2024-09-10T09:15:52.757 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-10T09:15:52.785 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-10T09:15:52.869 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/rishabh-2024-09-10_06:58:10-fs-wip-rishabh-testing-20240909.165402-squid-distro-default-smithi/7898972 branch: wip-rishabh-testing-20240909.165402-squid description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/legacy wsync/yes} objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/no 5-quiesce/with-quiesce 6-workunit/suites/dbench}} email: null first_in_suite: false job_id: '7898972' kernel: client: branch: testing kdb: 1 sha1: distro ktype: upstream last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules name: rishabh-2024-09-10_06:58:10-fs-wip-rishabh-testing-20240909.165402-squid-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-rishabh-testing-20240909.165402-squid ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd max_mds: 5 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true mds_export_ephemeral_random_max: 0.1 osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 mon warn on pool no app: false osd: bluestore block size: 96636764160 bluestore compression mode: aggressive bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10000 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - online, but wants - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - MON_DOWN - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - Degraded data redundancy - slow metadata IO - SLOW_OPS - slow request sha1: 1cf819b7212a6048e92fc78c4eff963344f6d56b subvols: create: 2 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported install: ceph: flavor: default sha1: 1cf819b7212a6048e92fc78c4eff963344f6d56b kclient: client.0: mount_subvol_num: 1 mntopts: - ms_mode=legacy - wsync syntax: v1 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-rishabh-testing-20240909.165402-squid sha1: 1cf819b7212a6048e92fc78c4eff963344f6d56b owner: scheduled_rishabh@teuthology priority: 70 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 9893 sha1: 1cf819b7212a6048e92fc78c4eff963344f6d56b sleep_before_teardown: 0 subset: 197/1024 suite: fs suite_branch: wip-rishabh-testing-20240909.165402-squid suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_1cf819b7212a6048e92fc78c4eff963344f6d56b/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 1cf819b7212a6048e92fc78c4eff963344f6d56b syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi005.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPQdTO5HqgF1mId4F4iFj0YDrnVFQ2xJKyqviH7JwPsyJaOmUX56uzXDtc/AnLXToNkT/VHbgdw48oOhHtpoD0E= smithi102.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOi0ZkfbTSn8BagGo+74RWHMQ9BffbJpv+RbubILpg5Ej80SNJOKTGsl4DPaYswZpEIU+UNnf8Zow2fSK5VuE5M= smithi159.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBAjDFWT02jo38nydrXBdUDMZjK8ipi4j1R2sO3lbrziSkoqLMteArVf3VKR44ESUvfLxIRQgCH9ZLKuV88V8WYA= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - sequential: - mgrmodules - kclient: null - exec: mon.a: - ceph fs set cephfs balance_automate false - ceph fs subvolumegroup pin cephfs qa random 0.10 - check-counter: {} - fwd_scrub: scrub_timeout: 900 sleep_between_iterations: 1 - quiescer: null - workunit: clients: all: - suites/dbench.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-10_06:58:10 tube: smithi user: rishabh verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856 2024-09-10T09:15:52.869 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_1cf819b7212a6048e92fc78c4eff963344f6d56b/qa; will attempt to use it 2024-09-10T09:15:52.870 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_1cf819b7212a6048e92fc78c4eff963344f6d56b/qa/tasks 2024-09-10T09:15:52.870 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-10T09:15:52.872 INFO:teuthology.task.internal:Checking packages... 2024-09-10T09:15:52.895 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '1cf819b7212a6048e92fc78c4eff963344f6d56b' 2024-09-10T09:15:52.895 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-10T09:15:52.895 INFO:teuthology.packaging:ref: None 2024-09-10T09:15:52.895 INFO:teuthology.packaging:tag: None 2024-09-10T09:15:52.895 INFO:teuthology.packaging:branch: wip-rishabh-testing-20240909.165402-squid 2024-09-10T09:15:52.895 INFO:teuthology.packaging:sha1: 1cf819b7212a6048e92fc78c4eff963344f6d56b 2024-09-10T09:15:52.895 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-rishabh-testing-20240909.165402-squid 2024-09-10T09:15:53.042 INFO:teuthology.task.internal:Found packages for ceph version 19.1.1-305.g1cf819b7 2024-09-10T09:15:53.044 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-10T09:15:53.050 INFO:teuthology.task.internal:no buildpackages task found 2024-09-10T09:15:53.051 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-10T09:15:53.095 INFO:teuthology.task.internal:Saving configuration 2024-09-10T09:15:53.114 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-10T09:15:53.119 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-10T09:15:53.144 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi005.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-09-10_06:58:10-fs-wip-rishabh-testing-20240909.165402-squid-distro-default-smithi/7898972', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-10 09:10:39.469533', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPQdTO5HqgF1mId4F4iFj0YDrnVFQ2xJKyqviH7JwPsyJaOmUX56uzXDtc/AnLXToNkT/VHbgdw48oOhHtpoD0E='} 2024-09-10T09:15:53.163 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi102.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-09-10_06:58:10-fs-wip-rishabh-testing-20240909.165402-squid-distro-default-smithi/7898972', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-10 09:10:39.466667', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOi0ZkfbTSn8BagGo+74RWHMQ9BffbJpv+RbubILpg5Ej80SNJOKTGsl4DPaYswZpEIU+UNnf8Zow2fSK5VuE5M='} 2024-09-10T09:15:53.187 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi159.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-09-10_06:58:10-fs-wip-rishabh-testing-20240909.165402-squid-distro-default-smithi/7898972', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-10 09:10:39.468213', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBAjDFWT02jo38nydrXBdUDMZjK8ipi4j1R2sO3lbrziSkoqLMteArVf3VKR44ESUvfLxIRQgCH9ZLKuV88V8WYA='} 2024-09-10T09:15:53.187 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-10T09:15:53.193 INFO:teuthology.task.internal:roles: ubuntu@smithi005.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-09-10T09:15:53.193 INFO:teuthology.task.internal:roles: ubuntu@smithi102.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-09-10T09:15:53.193 INFO:teuthology.task.internal:roles: ubuntu@smithi159.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-09-10T09:15:53.193 INFO:teuthology.run_tasks:Running task console_log... 2024-09-10T09:15:53.292 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f7a534d7e20>, signals=[15]) 2024-09-10T09:15:53.292 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-10T09:15:53.299 INFO:teuthology.task.internal:Opening connections... 2024-09-10T09:15:53.299 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi005.front.sepia.ceph.com 2024-09-10T09:15:53.300 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi005.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T09:15:53.380 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi102.front.sepia.ceph.com 2024-09-10T09:15:53.381 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi102.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T09:15:53.456 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi159.front.sepia.ceph.com 2024-09-10T09:15:53.457 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi159.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T09:15:53.529 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-10T09:15:53.536 DEBUG:teuthology.orchestra.run.smithi005:> uname -m 2024-09-10T09:15:53.555 INFO:teuthology.orchestra.run.smithi005.stdout:x86_64 2024-09-10T09:15:53.555 DEBUG:teuthology.orchestra.run.smithi005:> cat /etc/os-release 2024-09-10T09:15:53.613 INFO:teuthology.orchestra.run.smithi005.stdout:NAME="CentOS Stream" 2024-09-10T09:15:53.613 INFO:teuthology.orchestra.run.smithi005.stdout:VERSION="9" 2024-09-10T09:15:53.613 INFO:teuthology.orchestra.run.smithi005.stdout:ID="centos" 2024-09-10T09:15:53.613 INFO:teuthology.orchestra.run.smithi005.stdout:ID_LIKE="rhel fedora" 2024-09-10T09:15:53.613 INFO:teuthology.orchestra.run.smithi005.stdout:VERSION_ID="9" 2024-09-10T09:15:53.613 INFO:teuthology.orchestra.run.smithi005.stdout:PLATFORM_ID="platform:el9" 2024-09-10T09:15:53.613 INFO:teuthology.orchestra.run.smithi005.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-10T09:15:53.613 INFO:teuthology.orchestra.run.smithi005.stdout:ANSI_COLOR="0;31" 2024-09-10T09:15:53.614 INFO:teuthology.orchestra.run.smithi005.stdout:LOGO="fedora-logo-icon" 2024-09-10T09:15:53.614 INFO:teuthology.orchestra.run.smithi005.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-10T09:15:53.614 INFO:teuthology.orchestra.run.smithi005.stdout:HOME_URL="https://centos.org/" 2024-09-10T09:15:53.614 INFO:teuthology.orchestra.run.smithi005.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-10T09:15:53.614 INFO:teuthology.orchestra.run.smithi005.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-10T09:15:53.614 INFO:teuthology.orchestra.run.smithi005.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-10T09:15:53.614 INFO:teuthology.lock.ops:Updating smithi005.front.sepia.ceph.com on lock server 2024-09-10T09:15:53.638 DEBUG:teuthology.orchestra.run.smithi102:> uname -m 2024-09-10T09:15:53.655 INFO:teuthology.orchestra.run.smithi102.stdout:x86_64 2024-09-10T09:15:53.656 DEBUG:teuthology.orchestra.run.smithi102:> cat /etc/os-release 2024-09-10T09:15:53.712 INFO:teuthology.orchestra.run.smithi102.stdout:NAME="CentOS Stream" 2024-09-10T09:15:53.712 INFO:teuthology.orchestra.run.smithi102.stdout:VERSION="9" 2024-09-10T09:15:53.712 INFO:teuthology.orchestra.run.smithi102.stdout:ID="centos" 2024-09-10T09:15:53.713 INFO:teuthology.orchestra.run.smithi102.stdout:ID_LIKE="rhel fedora" 2024-09-10T09:15:53.713 INFO:teuthology.orchestra.run.smithi102.stdout:VERSION_ID="9" 2024-09-10T09:15:53.713 INFO:teuthology.orchestra.run.smithi102.stdout:PLATFORM_ID="platform:el9" 2024-09-10T09:15:53.713 INFO:teuthology.orchestra.run.smithi102.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-10T09:15:53.713 INFO:teuthology.orchestra.run.smithi102.stdout:ANSI_COLOR="0;31" 2024-09-10T09:15:53.713 INFO:teuthology.orchestra.run.smithi102.stdout:LOGO="fedora-logo-icon" 2024-09-10T09:15:53.713 INFO:teuthology.orchestra.run.smithi102.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-10T09:15:53.713 INFO:teuthology.orchestra.run.smithi102.stdout:HOME_URL="https://centos.org/" 2024-09-10T09:15:53.713 INFO:teuthology.orchestra.run.smithi102.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-10T09:15:53.713 INFO:teuthology.orchestra.run.smithi102.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-10T09:15:53.713 INFO:teuthology.orchestra.run.smithi102.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-10T09:15:53.714 INFO:teuthology.lock.ops:Updating smithi102.front.sepia.ceph.com on lock server 2024-09-10T09:15:53.729 DEBUG:teuthology.orchestra.run.smithi159:> uname -m 2024-09-10T09:15:53.748 INFO:teuthology.orchestra.run.smithi159.stdout:x86_64 2024-09-10T09:15:53.748 DEBUG:teuthology.orchestra.run.smithi159:> cat /etc/os-release 2024-09-10T09:15:53.807 INFO:teuthology.orchestra.run.smithi159.stdout:NAME="CentOS Stream" 2024-09-10T09:15:53.807 INFO:teuthology.orchestra.run.smithi159.stdout:VERSION="9" 2024-09-10T09:15:53.807 INFO:teuthology.orchestra.run.smithi159.stdout:ID="centos" 2024-09-10T09:15:53.807 INFO:teuthology.orchestra.run.smithi159.stdout:ID_LIKE="rhel fedora" 2024-09-10T09:15:53.807 INFO:teuthology.orchestra.run.smithi159.stdout:VERSION_ID="9" 2024-09-10T09:15:53.807 INFO:teuthology.orchestra.run.smithi159.stdout:PLATFORM_ID="platform:el9" 2024-09-10T09:15:53.807 INFO:teuthology.orchestra.run.smithi159.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-10T09:15:53.807 INFO:teuthology.orchestra.run.smithi159.stdout:ANSI_COLOR="0;31" 2024-09-10T09:15:53.808 INFO:teuthology.orchestra.run.smithi159.stdout:LOGO="fedora-logo-icon" 2024-09-10T09:15:53.808 INFO:teuthology.orchestra.run.smithi159.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-10T09:15:53.808 INFO:teuthology.orchestra.run.smithi159.stdout:HOME_URL="https://centos.org/" 2024-09-10T09:15:53.808 INFO:teuthology.orchestra.run.smithi159.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-10T09:15:53.808 INFO:teuthology.orchestra.run.smithi159.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-10T09:15:53.808 INFO:teuthology.orchestra.run.smithi159.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-10T09:15:53.808 INFO:teuthology.lock.ops:Updating smithi159.front.sepia.ceph.com on lock server 2024-09-10T09:15:53.821 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-10T09:15:53.831 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-10T09:15:53.853 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-10T09:15:53.853 DEBUG:teuthology.orchestra.run.smithi005:> test '!' -e /home/ubuntu/cephtest 2024-09-10T09:15:53.856 DEBUG:teuthology.orchestra.run.smithi102:> test '!' -e /home/ubuntu/cephtest 2024-09-10T09:15:53.858 DEBUG:teuthology.orchestra.run.smithi159:> test '!' -e /home/ubuntu/cephtest 2024-09-10T09:15:53.875 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-10T09:15:53.882 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-10T09:15:53.882 DEBUG:teuthology.orchestra.run.smithi005:> test -z $(ls -A /var/lib/ceph) 2024-09-10T09:15:53.913 DEBUG:teuthology.orchestra.run.smithi102:> test -z $(ls -A /var/lib/ceph) 2024-09-10T09:15:53.915 DEBUG:teuthology.orchestra.run.smithi159:> test -z $(ls -A /var/lib/ceph) 2024-09-10T09:15:54.139 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-10T09:15:54.209 INFO:teuthology.run_tasks:Running task kernel... 2024-09-10T09:15:54.223 INFO:teuthology.task.kernel:normalize config orig: {'client': {'branch': 'testing'}, 'kdb': 1, 'sha1': 'distro'} 2024-09-10T09:15:54.223 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'branch': 'testing'}} 2024-09-10T09:15:54.223 INFO:teuthology.task.kernel:config {'client.0': {'branch': 'testing'}}, timeout 300 2024-09-10T09:15:54.224 DEBUG:teuthology.orchestra.run.smithi005:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-10T09:15:54.241 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T09:15:54.241 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&ref=testing 2024-09-10T09:15:54.446 DEBUG:teuthology.task.kernel:sha1 for client.0 is 2ca44a2f35ca8f36983aaaa46d49312239ef519a 2024-09-10T09:15:54.446 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "2ca44a2f35ca8f36983aaaa46d49312239ef519a"... 2024-09-10T09:15:54.446 DEBUG:teuthology.orchestra.run.smithi005:> uname -r 2024-09-10T09:15:54.464 INFO:teuthology.orchestra.run.smithi005.stdout:5.14.0-503.el9.x86_64 2024-09-10T09:15:54.464 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-503.el9.x86_64 vs 2ca44a2f35ca8f36983aaaa46d49312239ef519a 2024-09-10T09:15:54.464 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-09-10T09:15:54.465 DEBUG:teuthology.orchestra.run.smithi005:> sudo yum upgrade -y linux-firmware 2024-09-10T09:15:57.362 INFO:teuthology.orchestra.run.smithi005.stdout:CentOS Stream 9 - BaseOS 6.9 MB/s | 8.2 MB 00:01 2024-09-10T09:16:03.938 INFO:teuthology.orchestra.run.smithi005.stdout:CentOS Stream 9 - AppStream 3.8 MB/s | 20 MB 00:05 2024-09-10T09:16:08.480 INFO:teuthology.orchestra.run.smithi005.stdout:CentOS Stream 9 - CRB 11 MB/s | 6.5 MB 00:00 2024-09-10T09:16:10.164 INFO:teuthology.orchestra.run.smithi005.stdout:CentOS Stream 9 - Extras packages 47 kB/s | 19 kB 00:00 2024-09-10T09:16:11.085 INFO:teuthology.orchestra.run.smithi005.stdout:Extra Packages for Enterprise Linux 29 MB/s | 23 MB 00:00 2024-09-10T09:16:16.724 INFO:teuthology.orchestra.run.smithi005.stdout:lab-extras 31 kB/s | 1.7 kB 00:00 2024-09-10T09:16:18.924 INFO:teuthology.orchestra.run.smithi005.stdout:Dependencies resolved. 2024-09-10T09:16:18.925 INFO:teuthology.orchestra.run.smithi005.stdout:Nothing to do. 2024-09-10T09:16:18.925 INFO:teuthology.orchestra.run.smithi005.stdout:Complete! 2024-09-10T09:16:19.041 INFO:teuthology.task.kernel:Downloading kernel 2ca44a2f35ca8f36983aaaa46d49312239ef519a on client.0... 2024-09-10T09:16:19.042 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=2ca44a2f35ca8f36983aaaa46d49312239ef519a 2024-09-10T09:16:19.195 INFO:teuthology.task.kernel:fetching, builder baseurl is https://3.chacra.ceph.com/r/kernel/testing/2ca44a2f35ca8f36983aaaa46d49312239ef519a/centos/9/flavors/default/x86_64/ 2024-09-10T09:16:19.195 DEBUG:teuthology.orchestra.run.smithi005:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.11.0_rc5_g2ca44a2f35ca-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://3.chacra.ceph.com/r/kernel/testing/2ca44a2f35ca8f36983aaaa46d49312239ef519a/centos/9/flavors/default/x86_64/ --input-file=- 2024-09-10T09:16:19.198 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi005.front.sepia.ceph.com to complete... 2024-09-10T09:16:25.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10 09:16:25 URL:https://3.chacra.ceph.com/r/kernel/testing/2ca44a2f35ca8f36983aaaa46d49312239ef519a/centos/9/flavors/default/x86_64/kernel-6.11.0_rc5_g2ca44a2f35ca-1.x86_64.rpm [359633503/359633503] -> "/tmp/kernel.x86_64.rpm" [1] 2024-09-10T09:16:25.490 INFO:teuthology.orchestra.run.smithi005.stderr:FINISHED --2024-09-10 09:16:25-- 2024-09-10T09:16:25.490 INFO:teuthology.orchestra.run.smithi005.stderr:Total wall clock time: 6.3s 2024-09-10T09:16:25.491 INFO:teuthology.orchestra.run.smithi005.stderr:Downloaded: 1 files, 343M in 6.0s (57.2 MB/s) 2024-09-10T09:16:25.493 INFO:teuthology.task.kernel:Installing kernel 2ca44a2f35ca8f36983aaaa46d49312239ef519a on client.0... 2024-09-10T09:16:25.494 DEBUG:teuthology.orchestra.run.smithi005:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-09-10T09:16:27.698 INFO:teuthology.orchestra.run.smithi005.stdout:Verifying... ######################################## 2024-09-10T09:16:27.862 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing... ######################################## 2024-09-10T09:16:27.964 INFO:teuthology.orchestra.run.smithi005.stdout:Updating / installing... 2024-09-10T09:16:36.259 INFO:teuthology.orchestra.run.smithi005.stdout:kernel-6.11.0_rc5_g2ca44a2f35ca-1 ######################################## 2024-09-10T09:17:16.280 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi005.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-09-10T09:17:16.281 DEBUG:teuthology.orchestra.run.smithi005:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-09-10T09:17:16.410 INFO:teuthology.orchestra.run.smithi005.stdout:/boot/System.map-6.11.0-rc5-g2ca44a2f35ca 2024-09-10T09:17:16.411 INFO:teuthology.orchestra.run.smithi005.stdout:/boot/config-6.11.0-rc5-g2ca44a2f35ca 2024-09-10T09:17:16.411 INFO:teuthology.orchestra.run.smithi005.stdout:/boot/vmlinuz-6.11.0-rc5-g2ca44a2f35ca 2024-09-10T09:17:16.411 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca 2024-09-10T09:17:16.411 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/System.map 2024-09-10T09:17:16.411 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/config 2024-09-10T09:17:16.411 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel 2024-09-10T09:17:16.411 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch 2024-09-10T09:17:16.412 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86 2024-09-10T09:17:16.412 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto 2024-09-10T09:17:16.412 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-09-10T09:17:16.412 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-09-10T09:17:16.412 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-09-10T09:17:16.412 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-09-10T09:17:16.412 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-09-10T09:17:16.412 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-09-10T09:17:16.412 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-09-10T09:17:16.412 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/crc32c-intel.ko 2024-09-10T09:17:16.412 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-09-10T09:17:16.412 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-09-10T09:17:16.413 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-09-10T09:17:16.413 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-09-10T09:17:16.413 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/serpent-avx2.ko 2024-09-10T09:17:16.413 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-09-10T09:17:16.413 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-09-10T09:17:16.413 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-09-10T09:17:16.413 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-09-10T09:17:16.413 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-09-10T09:17:16.413 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-09-10T09:17:16.413 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-09-10T09:17:16.413 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel 2024-09-10T09:17:16.416 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/cpu 2024-09-10T09:17:16.416 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/cpu/mce 2024-09-10T09:17:16.416 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-09-10T09:17:16.416 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm 2024-09-10T09:17:16.416 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm/kvm-amd.ko 2024-09-10T09:17:16.416 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm/kvm-intel.ko 2024-09-10T09:17:16.416 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm/kvm.ko 2024-09-10T09:17:16.417 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto 2024-09-10T09:17:16.417 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/ansi_cprng.ko 2024-09-10T09:17:16.417 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/anubis.ko 2024-09-10T09:17:16.417 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/arc4.ko 2024-09-10T09:17:16.417 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx 2024-09-10T09:17:16.417 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_memcpy.ko 2024-09-10T09:17:16.417 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_pq.ko 2024-09-10T09:17:16.417 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_raid6_recov.ko 2024-09-10T09:17:16.417 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_tx.ko 2024-09-10T09:17:16.417 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_xor.ko 2024-09-10T09:17:16.417 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/raid6test.ko 2024-09-10T09:17:16.417 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/authenc.ko 2024-09-10T09:17:16.418 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/authencesn.ko 2024-09-10T09:17:16.418 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/blake2b_generic.ko 2024-09-10T09:17:16.418 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/blowfish_common.ko 2024-09-10T09:17:16.418 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/blowfish_generic.ko 2024-09-10T09:17:16.418 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/camellia_generic.ko 2024-09-10T09:17:16.418 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cast5_generic.ko 2024-09-10T09:17:16.418 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cast6_generic.ko 2024-09-10T09:17:16.418 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cast_common.ko 2024-09-10T09:17:16.418 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/ccm.ko 2024-09-10T09:17:16.418 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cmac.ko 2024-09-10T09:17:16.418 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/crc32_generic.ko 2024-09-10T09:17:16.418 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/crypto_engine.ko 2024-09-10T09:17:16.418 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/crypto_user.ko 2024-09-10T09:17:16.419 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cts.ko 2024-09-10T09:17:16.419 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/des_generic.ko 2024-09-10T09:17:16.419 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/dh_generic.ko 2024-09-10T09:17:16.419 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/ecc.ko 2024-09-10T09:17:16.419 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/ecdh_generic.ko 2024-09-10T09:17:16.419 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/echainiv.ko 2024-09-10T09:17:16.419 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/essiv.ko 2024-09-10T09:17:16.419 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/fcrypt.ko 2024-09-10T09:17:16.419 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/khazad.ko 2024-09-10T09:17:16.419 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/lz4.ko 2024-09-10T09:17:16.419 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/lz4hc.ko 2024-09-10T09:17:16.419 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/md4.ko 2024-09-10T09:17:16.419 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/michael_mic.ko 2024-09-10T09:17:16.419 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/pcbc.ko 2024-09-10T09:17:16.420 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/pcrypt.ko 2024-09-10T09:17:16.420 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/rmd160.ko 2024-09-10T09:17:16.420 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/seed.ko 2024-09-10T09:17:16.420 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/serpent_generic.ko 2024-09-10T09:17:16.420 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/tcrypt.ko 2024-09-10T09:17:16.420 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/tea.ko 2024-09-10T09:17:16.420 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/twofish_common.ko 2024-09-10T09:17:16.420 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/twofish_generic.ko 2024-09-10T09:17:16.420 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/vmac.ko 2024-09-10T09:17:16.420 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/wp512.ko 2024-09-10T09:17:16.420 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/xcbc.ko 2024-09-10T09:17:16.420 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/xor.ko 2024-09-10T09:17:16.420 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/xxhash_generic.ko 2024-09-10T09:17:16.421 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers 2024-09-10T09:17:16.421 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi 2024-09-10T09:17:16.421 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/acpi_ipmi.ko 2024-09-10T09:17:16.421 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/acpi_pad.ko 2024-09-10T09:17:16.421 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/ec_sys.ko 2024-09-10T09:17:16.421 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/platform_profile.ko 2024-09-10T09:17:16.421 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/sbs.ko 2024-09-10T09:17:16.421 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/sbshc.ko 2024-09-10T09:17:16.421 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/video.ko 2024-09-10T09:17:16.421 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata 2024-09-10T09:17:16.421 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/acard-ahci.ko 2024-09-10T09:17:16.421 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/ahci_platform.ko 2024-09-10T09:17:16.421 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/ata_generic.ko 2024-09-10T09:17:16.422 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/libahci_platform.ko 2024-09-10T09:17:16.422 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_acpi.ko 2024-09-10T09:17:16.422 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ali.ko 2024-09-10T09:17:16.422 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_amd.ko 2024-09-10T09:17:16.422 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_artop.ko 2024-09-10T09:17:16.422 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_atiixp.ko 2024-09-10T09:17:16.422 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_atp867x.ko 2024-09-10T09:17:16.422 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_cmd640.ko 2024-09-10T09:17:16.422 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_cmd64x.ko 2024-09-10T09:17:16.422 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_cypress.ko 2024-09-10T09:17:16.422 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_efar.ko 2024-09-10T09:17:16.422 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt366.ko 2024-09-10T09:17:16.422 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt37x.ko 2024-09-10T09:17:16.422 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt3x2n.ko 2024-09-10T09:17:16.423 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt3x3.ko 2024-09-10T09:17:16.423 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_it8213.ko 2024-09-10T09:17:16.423 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_it821x.ko 2024-09-10T09:17:16.423 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_jmicron.ko 2024-09-10T09:17:16.423 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_marvell.ko 2024-09-10T09:17:16.423 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_mpiix.ko 2024-09-10T09:17:16.423 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_netcell.ko 2024-09-10T09:17:16.423 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ninja32.ko 2024-09-10T09:17:16.423 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ns87410.ko 2024-09-10T09:17:16.423 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ns87415.ko 2024-09-10T09:17:16.423 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_oldpiix.ko 2024-09-10T09:17:16.423 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_opti.ko 2024-09-10T09:17:16.423 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_optidma.ko 2024-09-10T09:17:16.424 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_pcmcia.ko 2024-09-10T09:17:16.424 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_pdc2027x.ko 2024-09-10T09:17:16.424 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-09-10T09:17:16.424 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_piccolo.ko 2024-09-10T09:17:16.424 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_rdc.ko 2024-09-10T09:17:16.424 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_sch.ko 2024-09-10T09:17:16.424 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_serverworks.ko 2024-09-10T09:17:16.424 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_sil680.ko 2024-09-10T09:17:16.424 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_sis.ko 2024-09-10T09:17:16.424 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_sl82c105.ko 2024-09-10T09:17:16.424 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_triflex.ko 2024-09-10T09:17:16.424 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_via.ko 2024-09-10T09:17:16.424 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pdc_adma.ko 2024-09-10T09:17:16.425 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_inic162x.ko 2024-09-10T09:17:16.425 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_mv.ko 2024-09-10T09:17:16.425 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_nv.ko 2024-09-10T09:17:16.425 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_promise.ko 2024-09-10T09:17:16.425 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_qstor.ko 2024-09-10T09:17:16.425 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sil.ko 2024-09-10T09:17:16.425 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sil24.ko 2024-09-10T09:17:16.425 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sis.ko 2024-09-10T09:17:16.425 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_svw.ko 2024-09-10T09:17:16.425 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sx4.ko 2024-09-10T09:17:16.425 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_uli.ko 2024-09-10T09:17:16.425 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_via.ko 2024-09-10T09:17:16.425 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_vsc.ko 2024-09-10T09:17:16.425 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/auxdisplay 2024-09-10T09:17:16.426 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/auxdisplay/ks0108.ko 2024-09-10T09:17:16.426 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/base 2024-09-10T09:17:16.426 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/base/regmap 2024-09-10T09:17:16.426 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/base/regmap/regmap-i2c.ko 2024-09-10T09:17:16.426 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/base/regmap/regmap-mmio.ko 2024-09-10T09:17:16.426 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/bcma 2024-09-10T09:17:16.426 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/bcma/bcma.ko 2024-09-10T09:17:16.426 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block 2024-09-10T09:17:16.426 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/aoe 2024-09-10T09:17:16.426 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/aoe/aoe.ko 2024-09-10T09:17:16.426 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/brd.ko 2024-09-10T09:17:16.426 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/drbd 2024-09-10T09:17:16.426 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/drbd/drbd.ko 2024-09-10T09:17:16.427 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/floppy.ko 2024-09-10T09:17:16.427 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/loop.ko 2024-09-10T09:17:16.427 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/mtip32xx 2024-09-10T09:17:16.427 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-09-10T09:17:16.427 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/nbd.ko 2024-09-10T09:17:16.427 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/null_blk 2024-09-10T09:17:16.427 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/null_blk/null_blk.ko 2024-09-10T09:17:16.427 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/pktcdvd.ko 2024-09-10T09:17:16.427 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/rbd.ko 2024-09-10T09:17:16.427 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/virtio_blk.ko 2024-09-10T09:17:16.427 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/xen-blkback 2024-09-10T09:17:16.427 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-09-10T09:17:16.427 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/xen-blkfront.ko 2024-09-10T09:17:16.427 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/zram 2024-09-10T09:17:16.428 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/zram/zram.ko 2024-09-10T09:17:16.428 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char 2024-09-10T09:17:16.428 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hangcheck-timer.ko 2024-09-10T09:17:16.428 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random 2024-09-10T09:17:16.428 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/amd-rng.ko 2024-09-10T09:17:16.428 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/intel-rng.ko 2024-09-10T09:17:16.428 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-09-10T09:17:16.428 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/via-rng.ko 2024-09-10T09:17:16.428 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/virtio-rng.ko 2024-09-10T09:17:16.428 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi 2024-09-10T09:17:16.428 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-09-10T09:17:16.428 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-09-10T09:17:16.428 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-09-10T09:17:16.429 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_si.ko 2024-09-10T09:17:16.429 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-09-10T09:17:16.429 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-09-10T09:17:16.429 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/lp.ko 2024-09-10T09:17:16.429 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/mwave 2024-09-10T09:17:16.429 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/mwave/mwave.ko 2024-09-10T09:17:16.429 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ppdev.ko 2024-09-10T09:17:16.429 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tlclk.ko 2024-09-10T09:17:16.429 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm 2024-09-10T09:17:16.429 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm.ko 2024-09-10T09:17:16.429 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_atmel.ko 2024-09-10T09:17:16.429 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_infineon.ko 2024-09-10T09:17:16.429 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_nsc.ko 2024-09-10T09:17:16.430 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_tis.ko 2024-09-10T09:17:16.430 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-09-10T09:17:16.430 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/uv_mmtimer.ko 2024-09-10T09:17:16.430 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/virtio_console.ko 2024-09-10T09:17:16.430 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq 2024-09-10T09:17:16.430 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-09-10T09:17:16.430 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-09-10T09:17:16.430 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/p4-clockmod.ko 2024-09-10T09:17:16.430 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-09-10T09:17:16.430 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/powernow-k8.ko 2024-09-10T09:17:16.430 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/speedstep-lib.ko 2024-09-10T09:17:16.430 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto 2024-09-10T09:17:16.430 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/ccp 2024-09-10T09:17:16.430 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-09-10T09:17:16.431 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/ccp/ccp.ko 2024-09-10T09:17:16.431 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel 2024-09-10T09:17:16.431 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat 2024-09-10T09:17:16.431 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_common 2024-09-10T09:17:16.431 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-09-10T09:17:16.431 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-09-10T09:17:16.431 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-09-10T09:17:16.431 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/padlock-aes.ko 2024-09-10T09:17:16.431 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/padlock-sha.ko 2024-09-10T09:17:16.431 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/virtio 2024-09-10T09:17:16.431 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-09-10T09:17:16.431 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dca 2024-09-10T09:17:16.431 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dca/dca.ko 2024-09-10T09:17:16.432 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/devfreq 2024-09-10T09:17:16.432 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-09-10T09:17:16.432 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma 2024-09-10T09:17:16.432 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/dw 2024-09-10T09:17:16.432 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/dw/dw_dmac.ko 2024-09-10T09:17:16.432 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/ioat 2024-09-10T09:17:16.432 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/ioat/ioatdma.ko 2024-09-10T09:17:16.432 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac 2024-09-10T09:17:16.432 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/amd64_edac.ko 2024-09-10T09:17:16.432 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/e752x_edac.ko 2024-09-10T09:17:16.432 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/edac_mce_amd.ko 2024-09-10T09:17:16.432 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i3000_edac.ko 2024-09-10T09:17:16.432 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i3200_edac.ko 2024-09-10T09:17:16.433 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i5100_edac.ko 2024-09-10T09:17:16.433 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i5400_edac.ko 2024-09-10T09:17:16.433 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i7300_edac.ko 2024-09-10T09:17:16.433 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i7core_edac.ko 2024-09-10T09:17:16.433 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i82975x_edac.ko 2024-09-10T09:17:16.433 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/ie31200_edac.ko 2024-09-10T09:17:16.433 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/sb_edac.ko 2024-09-10T09:17:16.433 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/x38_edac.ko 2024-09-10T09:17:16.433 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/firmware 2024-09-10T09:17:16.433 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/firmware/edd.ko 2024-09-10T09:17:16.433 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/firmware/iscsi_ibft.ko 2024-09-10T09:17:16.433 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio 2024-09-10T09:17:16.433 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-ich.ko 2024-09-10T09:17:16.434 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-viperboard.ko 2024-09-10T09:17:16.434 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid 2024-09-10T09:17:16.434 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-appleir.ko 2024-09-10T09:17:16.434 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-aureal.ko 2024-09-10T09:17:16.434 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-axff.ko 2024-09-10T09:17:16.434 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-dr.ko 2024-09-10T09:17:16.434 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-elecom.ko 2024-09-10T09:17:16.434 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-elo.ko 2024-09-10T09:17:16.434 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-emsff.ko 2024-09-10T09:17:16.434 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-gaff.ko 2024-09-10T09:17:16.434 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-gt683r.ko 2024-09-10T09:17:16.434 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-gyration.ko 2024-09-10T09:17:16.435 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-holtek-kbd.ko 2024-09-10T09:17:16.435 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-holtek-mouse.ko 2024-09-10T09:17:16.435 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-holtekff.ko 2024-09-10T09:17:16.435 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-hyperv.ko 2024-09-10T09:17:16.435 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-icade.ko 2024-09-10T09:17:16.435 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-keytouch.ko 2024-09-10T09:17:16.435 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-kye.ko 2024-09-10T09:17:16.435 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-lcpower.ko 2024-09-10T09:17:16.435 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-led.ko 2024-09-10T09:17:16.435 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-lenovo.ko 2024-09-10T09:17:16.435 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-logitech-dj.ko 2024-09-10T09:17:16.435 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-09-10T09:17:16.435 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-multitouch.ko 2024-09-10T09:17:16.435 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-ortek.ko 2024-09-10T09:17:16.436 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-penmount.ko 2024-09-10T09:17:16.436 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-petalynx.ko 2024-09-10T09:17:16.436 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-picolcd.ko 2024-09-10T09:17:16.436 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-pl.ko 2024-09-10T09:17:16.436 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-plantronics.ko 2024-09-10T09:17:16.436 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-primax.ko 2024-09-10T09:17:16.436 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-rmi.ko 2024-09-10T09:17:16.436 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-arvo.ko 2024-09-10T09:17:16.436 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-common.ko 2024-09-10T09:17:16.436 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-isku.ko 2024-09-10T09:17:16.436 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-kone.ko 2024-09-10T09:17:16.436 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-09-10T09:17:16.436 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-konepure.ko 2024-09-10T09:17:16.437 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-09-10T09:17:16.437 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-lua.ko 2024-09-10T09:17:16.437 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-pyra.ko 2024-09-10T09:17:16.437 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-ryos.ko 2024-09-10T09:17:16.437 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-savu.ko 2024-09-10T09:17:16.437 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat.ko 2024-09-10T09:17:16.437 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-saitek.ko 2024-09-10T09:17:16.437 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-samsung.ko 2024-09-10T09:17:16.437 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sensor-hub.ko 2024-09-10T09:17:16.437 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sjoy.ko 2024-09-10T09:17:16.437 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sony.ko 2024-09-10T09:17:16.437 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-speedlink.ko 2024-09-10T09:17:16.437 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-steelseries.ko 2024-09-10T09:17:16.438 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sunplus.ko 2024-09-10T09:17:16.438 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-thrustmaster.ko 2024-09-10T09:17:16.438 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-tivo.ko 2024-09-10T09:17:16.438 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-tmff.ko 2024-09-10T09:17:16.438 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-topseed.ko 2024-09-10T09:17:16.438 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-twinhan.ko 2024-09-10T09:17:16.438 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-uclogic.ko 2024-09-10T09:17:16.438 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-waltop.ko 2024-09-10T09:17:16.438 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-wiimote.ko 2024-09-10T09:17:16.438 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-xinmo.ko 2024-09-10T09:17:16.438 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-zpff.ko 2024-09-10T09:17:16.438 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-zydacron.ko 2024-09-10T09:17:16.438 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/uhid.ko 2024-09-10T09:17:16.439 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/wacom.ko 2024-09-10T09:17:16.439 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv 2024-09-10T09:17:16.439 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv/hv_balloon.ko 2024-09-10T09:17:16.439 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv/hv_utils.ko 2024-09-10T09:17:16.439 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv/hv_vmbus.ko 2024-09-10T09:17:16.439 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon 2024-09-10T09:17:16.439 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/abituguru.ko 2024-09-10T09:17:16.439 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/abituguru3.ko 2024-09-10T09:17:16.439 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/acpi_power_meter.ko 2024-09-10T09:17:16.439 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ad7414.ko 2024-09-10T09:17:16.439 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ad7418.ko 2024-09-10T09:17:16.439 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adc128d818.ko 2024-09-10T09:17:16.439 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1025.ko 2024-09-10T09:17:16.440 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1026.ko 2024-09-10T09:17:16.440 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1029.ko 2024-09-10T09:17:16.440 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1031.ko 2024-09-10T09:17:16.440 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm9240.ko 2024-09-10T09:17:16.440 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ads7828.ko 2024-09-10T09:17:16.440 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7410.ko 2024-09-10T09:17:16.440 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7411.ko 2024-09-10T09:17:16.440 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7462.ko 2024-09-10T09:17:16.440 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7470.ko 2024-09-10T09:17:16.440 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7475.ko 2024-09-10T09:17:16.440 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7x10.ko 2024-09-10T09:17:16.440 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/amc6821.ko 2024-09-10T09:17:16.441 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/applesmc.ko 2024-09-10T09:17:16.441 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/asb100.ko 2024-09-10T09:17:16.441 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/asc7621.ko 2024-09-10T09:17:16.441 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/asus_atk0110.ko 2024-09-10T09:17:16.441 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/atxp1.ko 2024-09-10T09:17:16.441 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/coretemp.ko 2024-09-10T09:17:16.441 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-09-10T09:17:16.441 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/dme1737.ko 2024-09-10T09:17:16.441 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ds1621.ko 2024-09-10T09:17:16.441 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ds620.ko 2024-09-10T09:17:16.441 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/emc1403.ko 2024-09-10T09:17:16.441 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/emc6w201.ko 2024-09-10T09:17:16.441 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/f71805f.ko 2024-09-10T09:17:16.441 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/f71882fg.ko 2024-09-10T09:17:16.442 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/f75375s.ko 2024-09-10T09:17:16.442 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/fam15h_power.ko 2024-09-10T09:17:16.442 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/fschmd.ko 2024-09-10T09:17:16.442 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/g760a.ko 2024-09-10T09:17:16.442 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/g762.ko 2024-09-10T09:17:16.442 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/gl518sm.ko 2024-09-10T09:17:16.442 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/gl520sm.ko 2024-09-10T09:17:16.442 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/hwmon-vid.ko 2024-09-10T09:17:16.442 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/i5500_temp.ko 2024-09-10T09:17:16.442 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/i5k_amb.ko 2024-09-10T09:17:16.442 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ibmaem.ko 2024-09-10T09:17:16.442 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ibmpex.ko 2024-09-10T09:17:16.442 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ina209.ko 2024-09-10T09:17:16.443 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ina2xx.ko 2024-09-10T09:17:16.443 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/it87.ko 2024-09-10T09:17:16.443 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/k10temp.ko 2024-09-10T09:17:16.443 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/k8temp.ko 2024-09-10T09:17:16.443 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lineage-pem.ko 2024-09-10T09:17:16.443 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm63.ko 2024-09-10T09:17:16.443 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm73.ko 2024-09-10T09:17:16.443 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm75.ko 2024-09-10T09:17:16.443 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm77.ko 2024-09-10T09:17:16.443 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm78.ko 2024-09-10T09:17:16.443 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm80.ko 2024-09-10T09:17:16.443 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm83.ko 2024-09-10T09:17:16.443 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm85.ko 2024-09-10T09:17:16.444 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm87.ko 2024-09-10T09:17:16.444 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm90.ko 2024-09-10T09:17:16.444 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm92.ko 2024-09-10T09:17:16.444 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm93.ko 2024-09-10T09:17:16.444 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm95234.ko 2024-09-10T09:17:16.444 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm95241.ko 2024-09-10T09:17:16.444 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm95245.ko 2024-09-10T09:17:16.444 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc2945.ko 2024-09-10T09:17:16.444 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4151.ko 2024-09-10T09:17:16.444 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4215.ko 2024-09-10T09:17:16.444 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4222.ko 2024-09-10T09:17:16.444 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4245.ko 2024-09-10T09:17:16.444 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4260.ko 2024-09-10T09:17:16.445 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4261.ko 2024-09-10T09:17:16.445 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max16065.ko 2024-09-10T09:17:16.445 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max1619.ko 2024-09-10T09:17:16.445 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max1668.ko 2024-09-10T09:17:16.445 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max197.ko 2024-09-10T09:17:16.445 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max6639.ko 2024-09-10T09:17:16.445 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max6650.ko 2024-09-10T09:17:16.445 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max6697.ko 2024-09-10T09:17:16.445 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/mcp3021.ko 2024-09-10T09:17:16.445 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct6683.ko 2024-09-10T09:17:16.445 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct6775-core.ko 2024-09-10T09:17:16.445 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct6775.ko 2024-09-10T09:17:16.445 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct7802.ko 2024-09-10T09:17:16.446 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pc87360.ko 2024-09-10T09:17:16.446 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pc87427.ko 2024-09-10T09:17:16.446 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pcf8591.ko 2024-09-10T09:17:16.446 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus 2024-09-10T09:17:16.446 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-09-10T09:17:16.446 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-09-10T09:17:16.446 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-09-10T09:17:16.446 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/max16064.ko 2024-09-10T09:17:16.446 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/max34440.ko 2024-09-10T09:17:16.446 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/max8688.ko 2024-09-10T09:17:16.446 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-09-10T09:17:16.446 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-09-10T09:17:16.446 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-09-10T09:17:16.447 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-09-10T09:17:16.447 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-09-10T09:17:16.447 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-09-10T09:17:16.447 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/powr1220.ko 2024-09-10T09:17:16.447 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sch5627.ko 2024-09-10T09:17:16.447 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sch5636.ko 2024-09-10T09:17:16.447 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sch56xx-common.ko 2024-09-10T09:17:16.447 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sht15.ko 2024-09-10T09:17:16.447 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sht21.ko 2024-09-10T09:17:16.447 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/shtc1.ko 2024-09-10T09:17:16.447 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sis5595.ko 2024-09-10T09:17:16.447 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/smsc47b397.ko 2024-09-10T09:17:16.447 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/smsc47m1.ko 2024-09-10T09:17:16.448 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/smsc47m192.ko 2024-09-10T09:17:16.448 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/thmc50.ko 2024-09-10T09:17:16.448 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp102.ko 2024-09-10T09:17:16.448 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp103.ko 2024-09-10T09:17:16.448 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp401.ko 2024-09-10T09:17:16.448 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp421.ko 2024-09-10T09:17:16.448 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/via-cputemp.ko 2024-09-10T09:17:16.448 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/via686a.ko 2024-09-10T09:17:16.448 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/vt1211.ko 2024-09-10T09:17:16.448 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/vt8231.ko 2024-09-10T09:17:16.448 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83627ehf.ko 2024-09-10T09:17:16.448 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83627hf.ko 2024-09-10T09:17:16.449 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83781d.ko 2024-09-10T09:17:16.449 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83791d.ko 2024-09-10T09:17:16.449 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83792d.ko 2024-09-10T09:17:16.449 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83793.ko 2024-09-10T09:17:16.449 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83795.ko 2024-09-10T09:17:16.449 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83l785ts.ko 2024-09-10T09:17:16.449 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83l786ng.ko 2024-09-10T09:17:16.449 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c 2024-09-10T09:17:16.449 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/algos 2024-09-10T09:17:16.449 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-09-10T09:17:16.449 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-09-10T09:17:16.449 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses 2024-09-10T09:17:16.449 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-09-10T09:17:16.450 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-09-10T09:17:16.450 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-09-10T09:17:16.450 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-09-10T09:17:16.450 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-09-10T09:17:16.450 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-09-10T09:17:16.450 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-09-10T09:17:16.450 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-09-10T09:17:16.450 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-i801.ko 2024-09-10T09:17:16.450 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-isch.ko 2024-09-10T09:17:16.450 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-09-10T09:17:16.450 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-09-10T09:17:16.450 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-09-10T09:17:16.450 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-parport.ko 2024-09-10T09:17:16.451 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-09-10T09:17:16.451 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-09-10T09:17:16.451 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-09-10T09:17:16.451 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-09-10T09:17:16.451 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-09-10T09:17:16.451 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-09-10T09:17:16.451 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-via.ko 2024-09-10T09:17:16.451 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-09-10T09:17:16.451 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-09-10T09:17:16.451 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-dev.ko 2024-09-10T09:17:16.451 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-mux.ko 2024-09-10T09:17:16.451 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-09-10T09:17:16.451 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-smbus.ko 2024-09-10T09:17:16.451 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-stub.ko 2024-09-10T09:17:16.452 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband 2024-09-10T09:17:16.452 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core 2024-09-10T09:17:16.452 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_cm.ko 2024-09-10T09:17:16.452 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_core.ko 2024-09-10T09:17:16.452 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_umad.ko 2024-09-10T09:17:16.452 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-09-10T09:17:16.452 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/iw_cm.ko 2024-09-10T09:17:16.452 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/rdma_cm.ko 2024-09-10T09:17:16.452 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-09-10T09:17:16.452 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw 2024-09-10T09:17:16.452 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/cxgb4 2024-09-10T09:17:16.452 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-09-10T09:17:16.452 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx4 2024-09-10T09:17:16.453 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-09-10T09:17:16.453 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx5 2024-09-10T09:17:16.453 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-09-10T09:17:16.453 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mthca 2024-09-10T09:17:16.453 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-09-10T09:17:16.453 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp 2024-09-10T09:17:16.453 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/ipoib 2024-09-10T09:17:16.453 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-09-10T09:17:16.453 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/iser 2024-09-10T09:17:16.453 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-09-10T09:17:16.453 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/isert 2024-09-10T09:17:16.453 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-09-10T09:17:16.453 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srp 2024-09-10T09:17:16.454 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-09-10T09:17:16.454 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srpt 2024-09-10T09:17:16.454 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-09-10T09:17:16.454 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input 2024-09-10T09:17:16.454 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport 2024-09-10T09:17:16.454 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-09-10T09:17:16.454 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport/fm801-gp.ko 2024-09-10T09:17:16.454 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport/gameport.ko 2024-09-10T09:17:16.454 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/joydev.ko 2024-09-10T09:17:16.454 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard 2024-09-10T09:17:16.454 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/gpio_keys.ko 2024-09-10T09:17:16.454 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse 2024-09-10T09:17:16.454 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/appletouch.ko 2024-09-10T09:17:16.455 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/bcm5974.ko 2024-09-10T09:17:16.455 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/cyapatp.ko 2024-09-10T09:17:16.455 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/elan_i2c.ko 2024-09-10T09:17:16.455 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/sermouse.ko 2024-09-10T09:17:16.455 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-09-10T09:17:16.455 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/synaptics_usb.ko 2024-09-10T09:17:16.455 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/vsxxxaa.ko 2024-09-10T09:17:16.455 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/rmi4 2024-09-10T09:17:16.455 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/rmi4/rmi_core.ko 2024-09-10T09:17:16.455 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio 2024-09-10T09:17:16.455 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/altera_ps2.ko 2024-09-10T09:17:16.455 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/arc_ps2.ko 2024-09-10T09:17:16.455 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-09-10T09:17:16.456 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/serio_raw.ko 2024-09-10T09:17:16.456 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/sparse-keymap.ko 2024-09-10T09:17:16.456 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds 2024-09-10T09:17:16.456 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-blinkm.ko 2024-09-10T09:17:16.456 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-lm3530.ko 2024-09-10T09:17:16.456 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-lp3944.ko 2024-09-10T09:17:16.456 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-ss4200.ko 2024-09-10T09:17:16.456 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger 2024-09-10T09:17:16.456 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-09-10T09:17:16.456 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-09-10T09:17:16.456 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-09-10T09:17:16.456 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-09-10T09:17:16.456 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-09-10T09:17:16.457 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-09-10T09:17:16.457 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-09-10T09:17:16.457 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-09-10T09:17:16.457 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md 2024-09-10T09:17:16.457 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/bcache 2024-09-10T09:17:16.457 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/bcache/bcache.ko 2024-09-10T09:17:16.457 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-bio-prison.ko 2024-09-10T09:17:16.457 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-cache-smq.ko 2024-09-10T09:17:16.457 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-cache.ko 2024-09-10T09:17:16.457 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-crypt.ko 2024-09-10T09:17:16.457 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-delay.ko 2024-09-10T09:17:16.457 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-flakey.ko 2024-09-10T09:17:16.457 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-log-userspace.ko 2024-09-10T09:17:16.458 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-multipath.ko 2024-09-10T09:17:16.458 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-queue-length.ko 2024-09-10T09:17:16.458 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-raid.ko 2024-09-10T09:17:16.458 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-round-robin.ko 2024-09-10T09:17:16.458 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-service-time.ko 2024-09-10T09:17:16.458 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-switch.ko 2024-09-10T09:17:16.458 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-thin-pool.ko 2024-09-10T09:17:16.458 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-verity.ko 2024-09-10T09:17:16.458 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/persistent-data 2024-09-10T09:17:16.458 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-09-10T09:17:16.458 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid0.ko 2024-09-10T09:17:16.458 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid1.ko 2024-09-10T09:17:16.458 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid10.ko 2024-09-10T09:17:16.459 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid456.ko 2024-09-10T09:17:16.459 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message 2024-09-10T09:17:16.459 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion 2024-09-10T09:17:16.459 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptbase.ko 2024-09-10T09:17:16.459 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptctl.ko 2024-09-10T09:17:16.459 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptfc.ko 2024-09-10T09:17:16.459 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptlan.ko 2024-09-10T09:17:16.459 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptsas.ko 2024-09-10T09:17:16.459 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptscsih.ko 2024-09-10T09:17:16.459 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptspi.ko 2024-09-10T09:17:16.459 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd 2024-09-10T09:17:16.459 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/lpc_ich.ko 2024-09-10T09:17:16.459 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/lpc_sch.ko 2024-09-10T09:17:16.460 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/mfd-core.ko 2024-09-10T09:17:16.460 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/sm501.ko 2024-09-10T09:17:16.460 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/viperboard.ko 2024-09-10T09:17:16.460 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/vx855.ko 2024-09-10T09:17:16.460 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/wl1273-core.ko 2024-09-10T09:17:16.460 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc 2024-09-10T09:17:16.460 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/altera-stapl 2024-09-10T09:17:16.460 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-09-10T09:17:16.460 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/apds9802als.ko 2024-09-10T09:17:16.460 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/apds990x.ko 2024-09-10T09:17:16.460 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/bh1770glc.ko 2024-09-10T09:17:16.460 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/cb710 2024-09-10T09:17:16.460 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/cb710/cb710.ko 2024-09-10T09:17:16.460 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom 2024-09-10T09:17:16.461 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom/at24.ko 2024-09-10T09:17:16.461 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-09-10T09:17:16.461 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom/max6875.ko 2024-09-10T09:17:16.461 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/enclosure.ko 2024-09-10T09:17:16.461 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/hpilo.ko 2024-09-10T09:17:16.461 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/isl29003.ko 2024-09-10T09:17:16.461 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/isl29020.ko 2024-09-10T09:17:16.461 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/lis3lv02d 2024-09-10T09:17:16.461 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-09-10T09:17:16.461 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-09-10T09:17:16.461 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei 2024-09-10T09:17:16.461 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei/mei-me.ko 2024-09-10T09:17:16.461 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei/mei-txe.ko 2024-09-10T09:17:16.462 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei/mei.ko 2024-09-10T09:17:16.462 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-gru 2024-09-10T09:17:16.462 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-gru/gru.ko 2024-09-10T09:17:16.462 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-xp 2024-09-10T09:17:16.462 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-xp/xp.ko 2024-09-10T09:17:16.462 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-xp/xpc.ko 2024-09-10T09:17:16.462 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-09-10T09:17:16.462 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/tifm_7xx1.ko 2024-09-10T09:17:16.462 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/tifm_core.ko 2024-09-10T09:17:16.462 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/tsl2550.ko 2024-09-10T09:17:16.462 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/vmw_balloon.ko 2024-09-10T09:17:16.462 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/vmw_vmci 2024-09-10T09:17:16.462 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-09-10T09:17:16.462 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net 2024-09-10T09:17:16.463 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/bonding 2024-09-10T09:17:16.463 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/bonding/bonding.ko 2024-09-10T09:17:16.463 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa 2024-09-10T09:17:16.463 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/b53 2024-09-10T09:17:16.463 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/b53/b53_common.ko 2024-09-10T09:17:16.463 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/bcm-sf2.ko 2024-09-10T09:17:16.463 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/mv88e6060.ko 2024-09-10T09:17:16.463 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/mv88e6xxx 2024-09-10T09:17:16.463 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-09-10T09:17:16.463 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dummy.ko 2024-09-10T09:17:16.463 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/eql.ko 2024-09-10T09:17:16.463 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet 2024-09-10T09:17:16.463 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com 2024-09-10T09:17:16.464 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-09-10T09:17:16.464 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-09-10T09:17:16.464 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-09-10T09:17:16.464 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-09-10T09:17:16.464 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390 2024-09-10T09:17:16.464 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/8390.ko 2024-09-10T09:17:16.464 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-09-10T09:17:16.464 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-09-10T09:17:16.464 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-09-10T09:17:16.464 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/adaptec 2024-09-10T09:17:16.464 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-09-10T09:17:16.464 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/agere 2024-09-10T09:17:16.465 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/agere/et131x.ko 2024-09-10T09:17:16.465 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/alteon 2024-09-10T09:17:16.465 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-09-10T09:17:16.465 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/altera 2024-09-10T09:17:16.465 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-09-10T09:17:16.465 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd 2024-09-10T09:17:16.465 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-09-10T09:17:16.465 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-09-10T09:17:16.465 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-09-10T09:17:16.465 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros 2024-09-10T09:17:16.465 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/alx 2024-09-10T09:17:16.465 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-09-10T09:17:16.465 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1c 2024-09-10T09:17:16.466 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-09-10T09:17:16.466 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1e 2024-09-10T09:17:16.466 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-09-10T09:17:16.466 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atlx 2024-09-10T09:17:16.466 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-09-10T09:17:16.466 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-09-10T09:17:16.466 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom 2024-09-10T09:17:16.466 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-09-10T09:17:16.466 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-09-10T09:17:16.466 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-09-10T09:17:16.466 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-09-10T09:17:16.466 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-09-10T09:17:16.466 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/genet 2024-09-10T09:17:16.467 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-09-10T09:17:16.467 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-09-10T09:17:16.467 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/brocade 2024-09-10T09:17:16.467 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/brocade/bna 2024-09-10T09:17:16.467 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-09-10T09:17:16.467 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio 2024-09-10T09:17:16.467 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb 2024-09-10T09:17:16.467 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-09-10T09:17:16.467 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-09-10T09:17:16.467 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-09-10T09:17:16.467 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-09-10T09:17:16.467 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-09-10T09:17:16.467 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-09-10T09:17:16.468 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-09-10T09:17:16.468 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-09-10T09:17:16.468 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-09-10T09:17:16.468 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/cisco 2024-09-10T09:17:16.468 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/cisco/enic 2024-09-10T09:17:16.468 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-09-10T09:17:16.468 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec 2024-09-10T09:17:16.468 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip 2024-09-10T09:17:16.468 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-09-10T09:17:16.468 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-09-10T09:17:16.468 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-09-10T09:17:16.468 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-09-10T09:17:16.468 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-09-10T09:17:16.469 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-09-10T09:17:16.469 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dlink 2024-09-10T09:17:16.469 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-09-10T09:17:16.469 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-09-10T09:17:16.469 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dnet.ko 2024-09-10T09:17:16.469 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/emulex 2024-09-10T09:17:16.469 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/emulex/benet 2024-09-10T09:17:16.469 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-09-10T09:17:16.469 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/ethoc.ko 2024-09-10T09:17:16.469 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/fealnx.ko 2024-09-10T09:17:16.469 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel 2024-09-10T09:17:16.469 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e100.ko 2024-09-10T09:17:16.469 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000 2024-09-10T09:17:16.470 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-09-10T09:17:16.470 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000e 2024-09-10T09:17:16.470 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-09-10T09:17:16.470 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/fm10k 2024-09-10T09:17:16.470 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-09-10T09:17:16.470 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/i40e 2024-09-10T09:17:16.470 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-09-10T09:17:16.470 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/iavf 2024-09-10T09:17:16.470 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-09-10T09:17:16.470 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igb 2024-09-10T09:17:16.470 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-09-10T09:17:16.470 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igbvf 2024-09-10T09:17:16.470 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-09-10T09:17:16.471 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbe 2024-09-10T09:17:16.471 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-09-10T09:17:16.471 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbevf 2024-09-10T09:17:16.471 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-09-10T09:17:16.471 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libeth 2024-09-10T09:17:16.471 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libeth/libeth.ko 2024-09-10T09:17:16.471 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libie 2024-09-10T09:17:16.471 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libie/libie.ko 2024-09-10T09:17:16.471 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/jme.ko 2024-09-10T09:17:16.471 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell 2024-09-10T09:17:16.471 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-09-10T09:17:16.471 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell/skge.ko 2024-09-10T09:17:16.471 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-09-10T09:17:16.472 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox 2024-09-10T09:17:16.472 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx4 2024-09-10T09:17:16.472 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-09-10T09:17:16.472 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-09-10T09:17:16.472 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx5 2024-09-10T09:17:16.472 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-09-10T09:17:16.472 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-09-10T09:17:16.472 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/micrel 2024-09-10T09:17:16.472 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-09-10T09:17:16.472 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/myricom 2024-09-10T09:17:16.472 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/myricom/myri10ge 2024-09-10T09:17:16.472 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-09-10T09:17:16.472 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/natsemi 2024-09-10T09:17:16.473 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-09-10T09:17:16.473 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-09-10T09:17:16.473 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/neterion 2024-09-10T09:17:16.473 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-09-10T09:17:16.473 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/nvidia 2024-09-10T09:17:16.473 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-09-10T09:17:16.473 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/packetengines 2024-09-10T09:17:16.473 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-09-10T09:17:16.473 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-09-10T09:17:16.473 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic 2024-09-10T09:17:16.473 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/netxen 2024-09-10T09:17:16.473 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-09-10T09:17:16.473 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-09-10T09:17:16.474 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-09-10T09:17:16.474 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-09-10T09:17:16.474 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/rdc 2024-09-10T09:17:16.474 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-09-10T09:17:16.474 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek 2024-09-10T09:17:16.474 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-09-10T09:17:16.474 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-09-10T09:17:16.474 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/atp.ko 2024-09-10T09:17:16.474 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-09-10T09:17:16.474 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/rocker 2024-09-10T09:17:16.474 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-09-10T09:17:16.474 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sfc 2024-09-10T09:17:16.474 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-09-10T09:17:16.474 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/silan 2024-09-10T09:17:16.475 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-09-10T09:17:16.475 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sis 2024-09-10T09:17:16.475 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sis/sis190.ko 2024-09-10T09:17:16.475 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sis/sis900.ko 2024-09-10T09:17:16.475 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc 2024-09-10T09:17:16.475 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-09-10T09:17:16.475 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-09-10T09:17:16.475 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-09-10T09:17:16.475 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-09-10T09:17:16.475 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro 2024-09-10T09:17:16.475 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/stmmac 2024-09-10T09:17:16.475 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-09-10T09:17:16.475 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-09-10T09:17:16.476 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun 2024-09-10T09:17:16.476 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/cassini.ko 2024-09-10T09:17:16.476 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/niu.ko 2024-09-10T09:17:16.476 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/sungem.ko 2024-09-10T09:17:16.476 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-09-10T09:17:16.476 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/tehuti 2024-09-10T09:17:16.476 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-09-10T09:17:16.476 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/ti 2024-09-10T09:17:16.476 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/ti/tlan.ko 2024-09-10T09:17:16.476 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/via 2024-09-10T09:17:16.476 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-09-10T09:17:16.476 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-09-10T09:17:16.476 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/wiznet 2024-09-10T09:17:16.477 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-09-10T09:17:16.477 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-09-10T09:17:16.477 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/xircom 2024-09-10T09:17:16.477 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-09-10T09:17:16.477 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/geneve.ko 2024-09-10T09:17:16.477 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/hyperv 2024-09-10T09:17:16.477 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-09-10T09:17:16.477 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ieee802154 2024-09-10T09:17:16.477 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ieee802154/fakelb.ko 2024-09-10T09:17:16.477 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ifb.ko 2024-09-10T09:17:16.477 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ipvlan 2024-09-10T09:17:16.477 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ipvlan/ipvlan.ko 2024-09-10T09:17:16.477 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/macvlan.ko 2024-09-10T09:17:16.477 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/macvtap.ko 2024-09-10T09:17:16.478 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio 2024-09-10T09:17:16.478 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio.ko 2024-09-10T09:17:16.478 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-09-10T09:17:16.478 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-09-10T09:17:16.478 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio/mdio-regmap.ko 2024-09-10T09:17:16.478 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mii.ko 2024-09-10T09:17:16.478 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/net_failover.ko 2024-09-10T09:17:16.478 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/netconsole.ko 2024-09-10T09:17:16.478 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/nlmon.ko 2024-09-10T09:17:16.478 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/pcs 2024-09-10T09:17:16.478 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/pcs/pcs-lynx.ko 2024-09-10T09:17:16.478 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-09-10T09:17:16.478 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy 2024-09-10T09:17:16.479 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/amd.ko 2024-09-10T09:17:16.479 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-09-10T09:17:16.479 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-09-10T09:17:16.479 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/bcm7xxx.ko 2024-09-10T09:17:16.479 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/bcm87xx.ko 2024-09-10T09:17:16.479 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/broadcom.ko 2024-09-10T09:17:16.479 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/cicada.ko 2024-09-10T09:17:16.479 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/davicom.ko 2024-09-10T09:17:16.479 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/dp83640.ko 2024-09-10T09:17:16.479 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/et1011c.ko 2024-09-10T09:17:16.479 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/icplus.ko 2024-09-10T09:17:16.479 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/lxt.ko 2024-09-10T09:17:16.479 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/marvell.ko 2024-09-10T09:17:16.480 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/micrel.ko 2024-09-10T09:17:16.480 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/national.ko 2024-09-10T09:17:16.480 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/phylink.ko 2024-09-10T09:17:16.480 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/qsemi.ko 2024-09-10T09:17:16.480 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/realtek.ko 2024-09-10T09:17:16.480 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/smsc.ko 2024-09-10T09:17:16.480 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/ste10Xp.ko 2024-09-10T09:17:16.480 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/vitesse.ko 2024-09-10T09:17:16.481 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp 2024-09-10T09:17:16.481 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/bsd_comp.ko 2024-09-10T09:17:16.481 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_async.ko 2024-09-10T09:17:16.481 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_deflate.ko 2024-09-10T09:17:16.481 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_generic.ko 2024-09-10T09:17:16.481 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_mppe.ko 2024-09-10T09:17:16.481 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_synctty.ko 2024-09-10T09:17:16.481 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/pppoe.ko 2024-09-10T09:17:16.481 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/pppox.ko 2024-09-10T09:17:16.481 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/pptp.ko 2024-09-10T09:17:16.482 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/slip 2024-09-10T09:17:16.482 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/slip/slhc.ko 2024-09-10T09:17:16.482 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/slip/slip.ko 2024-09-10T09:17:16.482 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/sungem_phy.ko 2024-09-10T09:17:16.482 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/tap.ko 2024-09-10T09:17:16.482 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team 2024-09-10T09:17:16.482 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team.ko 2024-09-10T09:17:16.482 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_activebackup.ko 2024-09-10T09:17:16.482 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_broadcast.ko 2024-09-10T09:17:16.482 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-09-10T09:17:16.482 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_random.ko 2024-09-10T09:17:16.482 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-09-10T09:17:16.483 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/tun.ko 2024-09-10T09:17:16.483 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/veth.ko 2024-09-10T09:17:16.483 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/virtio_net.ko 2024-09-10T09:17:16.483 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vmxnet3 2024-09-10T09:17:16.483 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-09-10T09:17:16.483 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vxlan 2024-09-10T09:17:16.483 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vxlan/vxlan.ko 2024-09-10T09:17:16.483 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/xen-netback 2024-09-10T09:17:16.483 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/xen-netback/xen-netback.ko 2024-09-10T09:17:16.483 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/xen-netfront.ko 2024-09-10T09:17:16.483 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ntb 2024-09-10T09:17:16.483 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ntb/ntb.ko 2024-09-10T09:17:16.483 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme 2024-09-10T09:17:16.484 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme/host 2024-09-10T09:17:16.484 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme/host/nvme-core.ko 2024-09-10T09:17:16.484 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme/host/nvme.ko 2024-09-10T09:17:16.484 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport 2024-09-10T09:17:16.484 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport.ko 2024-09-10T09:17:16.484 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport_cs.ko 2024-09-10T09:17:16.484 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport_pc.ko 2024-09-10T09:17:16.484 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport_serial.ko 2024-09-10T09:17:16.484 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci 2024-09-10T09:17:16.484 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/hotplug 2024-09-10T09:17:16.484 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-09-10T09:17:16.484 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/pcie 2024-09-10T09:17:16.484 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/pcie/aer_inject.ko 2024-09-10T09:17:16.485 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/xen-pcifront.ko 2024-09-10T09:17:16.485 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia 2024-09-10T09:17:16.485 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/i82092.ko 2024-09-10T09:17:16.485 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/pd6729.ko 2024-09-10T09:17:16.485 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/yenta_socket.ko 2024-09-10T09:17:16.485 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pinctrl 2024-09-10T09:17:16.485 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pinctrl/intel 2024-09-10T09:17:16.485 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-09-10T09:17:16.485 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform 2024-09-10T09:17:16.485 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86 2024-09-10T09:17:16.485 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/acer-wmi.ko 2024-09-10T09:17:16.485 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/acerhdf.ko 2024-09-10T09:17:16.485 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/apple-gmux.ko 2024-09-10T09:17:16.485 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/asus-laptop.ko 2024-09-10T09:17:16.486 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-09-10T09:17:16.486 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/asus-wmi.ko 2024-09-10T09:17:16.486 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/classmate-laptop.ko 2024-09-10T09:17:16.486 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-09-10T09:17:16.486 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-09-10T09:17:16.486 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-09-10T09:17:16.486 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-09-10T09:17:16.486 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/hdaps.ko 2024-09-10T09:17:16.486 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/intel 2024-09-10T09:17:16.486 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-09-10T09:17:16.486 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/intel_ips.ko 2024-09-10T09:17:16.486 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/msi-wmi.ko 2024-09-10T09:17:16.486 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/mxm-wmi.ko 2024-09-10T09:17:16.486 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-09-10T09:17:16.487 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/samsung-laptop.ko 2024-09-10T09:17:16.487 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/samsung-q10.ko 2024-09-10T09:17:16.487 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/topstar-laptop.ko 2024-09-10T09:17:16.487 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-09-10T09:17:16.487 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/toshiba_haps.ko 2024-09-10T09:17:16.487 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/wmi-bmof.ko 2024-09-10T09:17:16.487 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/wmi.ko 2024-09-10T09:17:16.487 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/powercap 2024-09-10T09:17:16.487 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/powercap/intel_rapl_common.ko 2024-09-10T09:17:16.487 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/powercap/intel_rapl_msr.ko 2024-09-10T09:17:16.487 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps 2024-09-10T09:17:16.487 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients 2024-09-10T09:17:16.488 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients/pps-gpio.ko 2024-09-10T09:17:16.488 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients/pps-ldisc.ko 2024-09-10T09:17:16.488 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients/pps_parport.ko 2024-09-10T09:17:16.488 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/pps_core.ko 2024-09-10T09:17:16.488 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ptp 2024-09-10T09:17:16.488 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ptp/ptp.ko 2024-09-10T09:17:16.488 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ptp/ptp_kvm.ko 2024-09-10T09:17:16.488 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras 2024-09-10T09:17:16.488 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd 2024-09-10T09:17:16.488 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd/atl 2024-09-10T09:17:16.489 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd/atl/amd_atl.ko 2024-09-10T09:17:16.489 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd/fmpm.ko 2024-09-10T09:17:16.489 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc 2024-09-10T09:17:16.489 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-bq32k.ko 2024-09-10T09:17:16.489 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1286.ko 2024-09-10T09:17:16.489 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1307.ko 2024-09-10T09:17:16.489 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1374.ko 2024-09-10T09:17:16.489 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1511.ko 2024-09-10T09:17:16.489 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1553.ko 2024-09-10T09:17:16.489 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1672.ko 2024-09-10T09:17:16.489 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1742.ko 2024-09-10T09:17:16.489 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds2404.ko 2024-09-10T09:17:16.489 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds3232.ko 2024-09-10T09:17:16.490 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-em3027.ko 2024-09-10T09:17:16.490 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-fm3130.ko 2024-09-10T09:17:16.490 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-isl12022.ko 2024-09-10T09:17:16.490 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-isl1208.ko 2024-09-10T09:17:16.490 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-m41t80.ko 2024-09-10T09:17:16.490 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-m48t35.ko 2024-09-10T09:17:16.490 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-m48t59.ko 2024-09-10T09:17:16.490 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-max6900.ko 2024-09-10T09:17:16.490 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-msm6242.ko 2024-09-10T09:17:16.490 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf2127.ko 2024-09-10T09:17:16.490 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf85063.ko 2024-09-10T09:17:16.490 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf8523.ko 2024-09-10T09:17:16.490 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf8563.ko 2024-09-10T09:17:16.490 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf8583.ko 2024-09-10T09:17:16.491 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rp5c01.ko 2024-09-10T09:17:16.491 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rs5c372.ko 2024-09-10T09:17:16.491 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-09-10T09:17:16.491 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rx8025.ko 2024-09-10T09:17:16.491 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rx8581.ko 2024-09-10T09:17:16.491 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-09-10T09:17:16.491 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-x1205.ko 2024-09-10T09:17:16.491 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi 2024-09-10T09:17:16.491 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/3w-9xxx.ko 2024-09-10T09:17:16.491 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/3w-sas.ko 2024-09-10T09:17:16.491 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/3w-xxxx.ko 2024-09-10T09:17:16.491 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/BusLogic.ko 2024-09-10T09:17:16.492 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/a100u2w.ko 2024-09-10T09:17:16.492 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aacraid 2024-09-10T09:17:16.492 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aacraid/aacraid.ko 2024-09-10T09:17:16.492 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/advansys.ko 2024-09-10T09:17:16.492 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aic7xxx 2024-09-10T09:17:16.492 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-09-10T09:17:16.492 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-09-10T09:17:16.492 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/am53c974.ko 2024-09-10T09:17:16.492 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/arcmsr 2024-09-10T09:17:16.492 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-09-10T09:17:16.492 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/atp870u.ko 2024-09-10T09:17:16.492 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/be2iscsi 2024-09-10T09:17:16.492 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-09-10T09:17:16.493 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bfa 2024-09-10T09:17:16.493 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bfa/bfa.ko 2024-09-10T09:17:16.493 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2fc 2024-09-10T09:17:16.493 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-09-10T09:17:16.493 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2i 2024-09-10T09:17:16.493 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-09-10T09:17:16.493 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ch.ko 2024-09-10T09:17:16.493 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/csiostor 2024-09-10T09:17:16.493 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/csiostor/csiostor.ko 2024-09-10T09:17:16.493 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi 2024-09-10T09:17:16.493 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb3i 2024-09-10T09:17:16.493 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-09-10T09:17:16.493 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb4i 2024-09-10T09:17:16.493 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-09-10T09:17:16.494 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-09-10T09:17:16.494 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/dc395x.ko 2024-09-10T09:17:16.494 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/device_handler 2024-09-10T09:17:16.494 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-09-10T09:17:16.494 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-09-10T09:17:16.494 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-09-10T09:17:16.494 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-09-10T09:17:16.494 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/dmx3191d.ko 2024-09-10T09:17:16.494 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/esas2r 2024-09-10T09:17:16.494 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/esas2r/esas2r.ko 2024-09-10T09:17:16.494 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/esp_scsi.ko 2024-09-10T09:17:16.494 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fcoe 2024-09-10T09:17:16.494 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fcoe/fcoe.ko 2024-09-10T09:17:16.495 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-09-10T09:17:16.495 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fnic 2024-09-10T09:17:16.495 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fnic/fnic.ko 2024-09-10T09:17:16.495 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/hpsa.ko 2024-09-10T09:17:16.495 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/hptiop.ko 2024-09-10T09:17:16.495 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/hv_storvsc.ko 2024-09-10T09:17:16.495 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/initio.ko 2024-09-10T09:17:16.495 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ipr.ko 2024-09-10T09:17:16.495 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ips.ko 2024-09-10T09:17:16.495 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/isci 2024-09-10T09:17:16.495 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/isci/isci.ko 2024-09-10T09:17:16.495 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-09-10T09:17:16.495 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/iscsi_tcp.ko 2024-09-10T09:17:16.495 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libfc 2024-09-10T09:17:16.496 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libfc/libfc.ko 2024-09-10T09:17:16.496 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libiscsi.ko 2024-09-10T09:17:16.496 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libiscsi_tcp.ko 2024-09-10T09:17:16.496 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libsas 2024-09-10T09:17:16.496 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libsas/libsas.ko 2024-09-10T09:17:16.496 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/lpfc 2024-09-10T09:17:16.496 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/lpfc/lpfc.ko 2024-09-10T09:17:16.496 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid 2024-09-10T09:17:16.496 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid.ko 2024-09-10T09:17:16.496 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-09-10T09:17:16.496 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-09-10T09:17:16.496 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-09-10T09:17:16.496 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mpt3sas 2024-09-10T09:17:16.496 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-09-10T09:17:16.497 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mvsas 2024-09-10T09:17:16.497 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mvsas/mvsas.ko 2024-09-10T09:17:16.497 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mvumi.ko 2024-09-10T09:17:16.497 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pm8001 2024-09-10T09:17:16.497 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-09-10T09:17:16.497 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pmcraid.ko 2024-09-10T09:17:16.497 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla1280.ko 2024-09-10T09:17:16.497 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla2xxx 2024-09-10T09:17:16.497 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-09-10T09:17:16.497 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-09-10T09:17:16.497 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla4xxx 2024-09-10T09:17:16.497 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-09-10T09:17:16.497 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/raid_class.ko 2024-09-10T09:17:16.498 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_debug.ko 2024-09-10T09:17:16.498 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_fc.ko 2024-09-10T09:17:16.498 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-09-10T09:17:16.498 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_sas.ko 2024-09-10T09:17:16.498 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_spi.ko 2024-09-10T09:17:16.498 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_srp.ko 2024-09-10T09:17:16.498 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ses.ko 2024-09-10T09:17:16.498 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/st.ko 2024-09-10T09:17:16.498 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/stex.ko 2024-09-10T09:17:16.498 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/sym53c8xx_2 2024-09-10T09:17:16.498 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-09-10T09:17:16.498 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/virtio_scsi.ko 2024-09-10T09:17:16.498 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/vmw_pvscsi.ko 2024-09-10T09:17:16.498 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/wd719x.ko 2024-09-10T09:17:16.499 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ssb 2024-09-10T09:17:16.499 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ssb/ssb.ko 2024-09-10T09:17:16.499 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target 2024-09-10T09:17:16.499 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/iscsi 2024-09-10T09:17:16.499 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-09-10T09:17:16.499 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/loopback 2024-09-10T09:17:16.499 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/loopback/tcm_loop.ko 2024-09-10T09:17:16.499 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_file.ko 2024-09-10T09:17:16.499 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_iblock.ko 2024-09-10T09:17:16.499 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_mod.ko 2024-09-10T09:17:16.499 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_pscsi.ko 2024-09-10T09:17:16.499 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_user.ko 2024-09-10T09:17:16.499 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/tcm_fc 2024-09-10T09:17:16.499 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-09-10T09:17:16.500 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal 2024-09-10T09:17:16.500 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel 2024-09-10T09:17:16.500 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal 2024-09-10T09:17:16.500 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-09-10T09:17:16.500 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-09-10T09:17:16.500 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-09-10T09:17:16.500 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-09-10T09:17:16.500 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-09-10T09:17:16.500 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-09-10T09:17:16.500 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-09-10T09:17:16.500 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-09-10T09:17:16.500 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-09-10T09:17:16.501 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-09-10T09:17:16.501 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-09-10T09:17:16.501 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-09-10T09:17:16.501 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-09-10T09:17:16.501 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-09-10T09:17:16.501 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-09-10T09:17:16.501 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-09-10T09:17:16.501 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-09-10T09:17:16.501 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-09-10T09:17:16.501 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty 2024-09-10T09:17:16.501 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/ipwireless 2024-09-10T09:17:16.501 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-09-10T09:17:16.501 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/n_gsm.ko 2024-09-10T09:17:16.501 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/n_hdlc.ko 2024-09-10T09:17:16.502 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/nozomi.ko 2024-09-10T09:17:16.502 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial 2024-09-10T09:17:16.502 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/8250 2024-09-10T09:17:16.502 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-09-10T09:17:16.502 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/arc_uart.ko 2024-09-10T09:17:16.502 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/jsm 2024-09-10T09:17:16.502 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/jsm/jsm.ko 2024-09-10T09:17:16.502 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs 2024-09-10T09:17:16.502 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/core 2024-09-10T09:17:16.502 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/core/ufshcd-core.ko 2024-09-10T09:17:16.502 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/host 2024-09-10T09:17:16.502 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-09-10T09:17:16.502 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio 2024-09-10T09:17:16.502 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio.ko 2024-09-10T09:17:16.503 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_aec.ko 2024-09-10T09:17:16.503 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_cif.ko 2024-09-10T09:17:16.503 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_pci_generic.ko 2024-09-10T09:17:16.503 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_sercos3.ko 2024-09-10T09:17:16.503 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb 2024-09-10T09:17:16.503 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class 2024-09-10T09:17:16.503 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/cdc-acm.ko 2024-09-10T09:17:16.503 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/cdc-wdm.ko 2024-09-10T09:17:16.503 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/usblp.ko 2024-09-10T09:17:16.503 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/usbtmc.ko 2024-09-10T09:17:16.503 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host 2024-09-10T09:17:16.503 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/sl811-hcd.ko 2024-09-10T09:17:16.503 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/image 2024-09-10T09:17:16.503 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/image/mdc800.ko 2024-09-10T09:17:16.504 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/image/microtek.ko 2024-09-10T09:17:16.504 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc 2024-09-10T09:17:16.504 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/adutux.ko 2024-09-10T09:17:16.504 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/appledisplay.ko 2024-09-10T09:17:16.504 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/emi26.ko 2024-09-10T09:17:16.504 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/emi62.ko 2024-09-10T09:17:16.504 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/ezusb.ko 2024-09-10T09:17:16.504 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/idmouse.ko 2024-09-10T09:17:16.504 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/iowarrior.ko 2024-09-10T09:17:16.504 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/isight_firmware.ko 2024-09-10T09:17:16.504 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/ldusb.ko 2024-09-10T09:17:16.504 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/legousbtower.ko 2024-09-10T09:17:16.504 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/sisusbvga 2024-09-10T09:17:16.505 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-09-10T09:17:16.505 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/trancevibrator.ko 2024-09-10T09:17:16.505 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/usb3503.ko 2024-09-10T09:17:16.505 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/usblcd.ko 2024-09-10T09:17:16.505 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/usbsevseg.ko 2024-09-10T09:17:16.505 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/uss720.ko 2024-09-10T09:17:16.505 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/yurex.ko 2024-09-10T09:17:16.505 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/phy 2024-09-10T09:17:16.505 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/phy/phy-generic.ko 2024-09-10T09:17:16.505 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial 2024-09-10T09:17:16.505 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/aircable.ko 2024-09-10T09:17:16.505 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ark3116.ko 2024-09-10T09:17:16.505 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/belkin_sa.ko 2024-09-10T09:17:16.506 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ch341.ko 2024-09-10T09:17:16.506 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/cp210x.ko 2024-09-10T09:17:16.506 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/cyberjack.ko 2024-09-10T09:17:16.506 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/cypress_m8.ko 2024-09-10T09:17:16.506 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/digi_acceleport.ko 2024-09-10T09:17:16.506 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/empeg.ko 2024-09-10T09:17:16.506 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ftdi_sio.ko 2024-09-10T09:17:16.506 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/garmin_gps.ko 2024-09-10T09:17:16.506 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/io_edgeport.ko 2024-09-10T09:17:16.506 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/io_ti.ko 2024-09-10T09:17:16.506 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ipaq.ko 2024-09-10T09:17:16.506 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ipw.ko 2024-09-10T09:17:16.506 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ir-usb.ko 2024-09-10T09:17:16.507 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-09-10T09:17:16.507 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/keyspan.ko 2024-09-10T09:17:16.507 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/keyspan_pda.ko 2024-09-10T09:17:16.507 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/kl5kusb105.ko 2024-09-10T09:17:16.507 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/kobil_sct.ko 2024-09-10T09:17:16.507 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/mct_u232.ko 2024-09-10T09:17:16.507 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/mos7720.ko 2024-09-10T09:17:16.507 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/mos7840.ko 2024-09-10T09:17:16.507 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/navman.ko 2024-09-10T09:17:16.507 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/omninet.ko 2024-09-10T09:17:16.507 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/opticon.ko 2024-09-10T09:17:16.507 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/option.ko 2024-09-10T09:17:16.507 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/oti6858.ko 2024-09-10T09:17:16.507 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/pl2303.ko 2024-09-10T09:17:16.508 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/qcaux.ko 2024-09-10T09:17:16.508 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/qcserial.ko 2024-09-10T09:17:16.508 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/quatech2.ko 2024-09-10T09:17:16.508 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/safe_serial.ko 2024-09-10T09:17:16.508 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/sierra.ko 2024-09-10T09:17:16.508 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/spcp8x5.ko 2024-09-10T09:17:16.508 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ssu100.ko 2024-09-10T09:17:16.508 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/symbolserial.ko 2024-09-10T09:17:16.508 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-09-10T09:17:16.508 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-09-10T09:17:16.508 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/usb_debug.ko 2024-09-10T09:17:16.508 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/usb_wwan.ko 2024-09-10T09:17:16.508 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/visor.ko 2024-09-10T09:17:16.509 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/whiteheat.ko 2024-09-10T09:17:16.509 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/xsens_mt.ko 2024-09-10T09:17:16.509 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage 2024-09-10T09:17:16.509 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/uas.ko 2024-09-10T09:17:16.509 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-alauda.ko 2024-09-10T09:17:16.509 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-cypress.ko 2024-09-10T09:17:16.509 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-datafab.ko 2024-09-10T09:17:16.509 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-09-10T09:17:16.509 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-freecom.ko 2024-09-10T09:17:16.509 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-isd200.ko 2024-09-10T09:17:16.509 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-09-10T09:17:16.509 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-karma.ko 2024-09-10T09:17:16.509 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-onetouch.ko 2024-09-10T09:17:16.510 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-realtek.ko 2024-09-10T09:17:16.510 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-sddr09.ko 2024-09-10T09:17:16.510 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-sddr55.ko 2024-09-10T09:17:16.510 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-usbat.ko 2024-09-10T09:17:16.510 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/usb-storage.ko 2024-09-10T09:17:16.510 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip 2024-09-10T09:17:16.510 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip/usbip-core.ko 2024-09-10T09:17:16.510 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip/usbip-host.ko 2024-09-10T09:17:16.510 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-09-10T09:17:16.510 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio 2024-09-10T09:17:16.510 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/pci 2024-09-10T09:17:16.510 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-09-10T09:17:16.510 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/pci/vfio-pci.ko 2024-09-10T09:17:16.511 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/vfio.ko 2024-09-10T09:17:16.511 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-09-10T09:17:16.511 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost 2024-09-10T09:17:16.511 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost.ko 2024-09-10T09:17:16.511 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost_iotlb.ko 2024-09-10T09:17:16.511 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost_net.ko 2024-09-10T09:17:16.511 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost_scsi.ko 2024-09-10T09:17:16.511 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video 2024-09-10T09:17:16.511 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight 2024-09-10T09:17:16.511 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/apple_bl.ko 2024-09-10T09:17:16.511 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/lcd.ko 2024-09-10T09:17:16.511 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/platform_lcd.ko 2024-09-10T09:17:16.511 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio 2024-09-10T09:17:16.511 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio.ko 2024-09-10T09:17:16.512 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_balloon.ko 2024-09-10T09:17:16.512 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_mmio.ko 2024-09-10T09:17:16.512 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_pci.ko 2024-09-10T09:17:16.512 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-09-10T09:17:16.512 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-09-10T09:17:16.512 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_ring.ko 2024-09-10T09:17:16.512 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1 2024-09-10T09:17:16.512 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/masters 2024-09-10T09:17:16.512 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/masters/ds2482.ko 2024-09-10T09:17:16.512 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/masters/ds2490.ko 2024-09-10T09:17:16.512 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves 2024-09-10T09:17:16.512 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-09-10T09:17:16.513 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-09-10T09:17:16.513 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-09-10T09:17:16.513 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-09-10T09:17:16.513 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-09-10T09:17:16.513 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-09-10T09:17:16.513 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-09-10T09:17:16.513 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-09-10T09:17:16.513 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-09-10T09:17:16.513 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_smem.ko 2024-09-10T09:17:16.513 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_therm.ko 2024-09-10T09:17:16.513 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/wire.ko 2024-09-10T09:17:16.513 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog 2024-09-10T09:17:16.513 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/alim1535_wdt.ko 2024-09-10T09:17:16.514 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/alim7101_wdt.ko 2024-09-10T09:17:16.514 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/f71808e_wdt.ko 2024-09-10T09:17:16.514 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/hpwdt.ko 2024-09-10T09:17:16.514 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/i6300esb.ko 2024-09-10T09:17:16.514 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-09-10T09:17:16.514 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/iTCO_wdt.ko 2024-09-10T09:17:16.514 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/ib700wdt.ko 2024-09-10T09:17:16.514 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/ibmasr.ko 2024-09-10T09:17:16.514 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-09-10T09:17:16.514 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/it8712f_wdt.ko 2024-09-10T09:17:16.514 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/it87_wdt.ko 2024-09-10T09:17:16.514 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/machzwd.ko 2024-09-10T09:17:16.514 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/nv_tco.ko 2024-09-10T09:17:16.515 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/pcwd_pci.ko 2024-09-10T09:17:16.515 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/pcwd_usb.ko 2024-09-10T09:17:16.515 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-09-10T09:17:16.515 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sch311x_wdt.ko 2024-09-10T09:17:16.515 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/softdog.ko 2024-09-10T09:17:16.515 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sp5100_tco.ko 2024-09-10T09:17:16.515 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/via_wdt.ko 2024-09-10T09:17:16.515 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-09-10T09:17:16.515 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/w83877f_wdt.ko 2024-09-10T09:17:16.515 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/w83977f_wdt.ko 2024-09-10T09:17:16.515 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/wdt_pci.ko 2024-09-10T09:17:16.515 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/xen_wdt.ko 2024-09-10T09:17:16.515 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen 2024-09-10T09:17:16.516 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-acpi-processor.ko 2024-09-10T09:17:16.516 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-evtchn.ko 2024-09-10T09:17:16.516 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-gntalloc.ko 2024-09-10T09:17:16.516 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-gntdev.ko 2024-09-10T09:17:16.516 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-pciback 2024-09-10T09:17:16.516 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-09-10T09:17:16.516 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-privcmd.ko 2024-09-10T09:17:16.516 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xenfs 2024-09-10T09:17:16.516 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xenfs/xenfs.ko 2024-09-10T09:17:16.516 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs 2024-09-10T09:17:16.516 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/9p 2024-09-10T09:17:16.516 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/9p/9p.ko 2024-09-10T09:17:16.516 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/affs 2024-09-10T09:17:16.517 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/affs/affs.ko 2024-09-10T09:17:16.517 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/befs 2024-09-10T09:17:16.517 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/befs/befs.ko 2024-09-10T09:17:16.517 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/binfmt_misc.ko 2024-09-10T09:17:16.517 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/btrfs 2024-09-10T09:17:16.517 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/btrfs/btrfs.ko 2024-09-10T09:17:16.517 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cachefiles 2024-09-10T09:17:16.517 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cachefiles/cachefiles.ko 2024-09-10T09:17:16.517 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ceph 2024-09-10T09:17:16.517 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ceph/ceph.ko 2024-09-10T09:17:16.517 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/coda 2024-09-10T09:17:16.517 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/coda/coda.ko 2024-09-10T09:17:16.517 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cramfs 2024-09-10T09:17:16.518 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cramfs/cramfs.ko 2024-09-10T09:17:16.518 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/dlm 2024-09-10T09:17:16.518 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/dlm/dlm.ko 2024-09-10T09:17:16.518 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ecryptfs 2024-09-10T09:17:16.518 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ecryptfs/ecryptfs.ko 2024-09-10T09:17:16.518 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/f2fs 2024-09-10T09:17:16.518 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/f2fs/f2fs.ko 2024-09-10T09:17:16.518 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fat 2024-09-10T09:17:16.518 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fat/fat.ko 2024-09-10T09:17:16.518 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fat/msdos.ko 2024-09-10T09:17:16.518 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fat/vfat.ko 2024-09-10T09:17:16.518 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fuse 2024-09-10T09:17:16.518 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fuse/cuse.ko 2024-09-10T09:17:16.518 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fuse/fuse.ko 2024-09-10T09:17:16.519 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/gfs2 2024-09-10T09:17:16.519 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/gfs2/gfs2.ko 2024-09-10T09:17:16.519 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfs 2024-09-10T09:17:16.519 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfs/hfs.ko 2024-09-10T09:17:16.519 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfsplus 2024-09-10T09:17:16.519 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfsplus/hfsplus.ko 2024-09-10T09:17:16.519 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/isofs 2024-09-10T09:17:16.519 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/isofs/isofs.ko 2024-09-10T09:17:16.519 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/jfs 2024-09-10T09:17:16.519 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/jfs/jfs.ko 2024-09-10T09:17:16.519 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/lockd 2024-09-10T09:17:16.519 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/lockd/lockd.ko 2024-09-10T09:17:16.519 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/minix 2024-09-10T09:17:16.520 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/minix/minix.ko 2024-09-10T09:17:16.520 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/netfs 2024-09-10T09:17:16.520 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/netfs/netfs.ko 2024-09-10T09:17:16.520 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs 2024-09-10T09:17:16.520 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/blocklayout 2024-09-10T09:17:16.520 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-09-10T09:17:16.520 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/filelayout 2024-09-10T09:17:16.520 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-09-10T09:17:16.520 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/flexfilelayout 2024-09-10T09:17:16.520 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-09-10T09:17:16.520 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/nfs.ko 2024-09-10T09:17:16.520 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/nfsv3.ko 2024-09-10T09:17:16.520 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/nfsv4.ko 2024-09-10T09:17:16.521 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs_common 2024-09-10T09:17:16.521 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs_common/grace.ko 2024-09-10T09:17:16.521 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs_common/nfs_acl.ko 2024-09-10T09:17:16.521 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfsd 2024-09-10T09:17:16.521 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfsd/nfsd.ko 2024-09-10T09:17:16.521 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nilfs2 2024-09-10T09:17:16.521 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nilfs2/nilfs2.ko 2024-09-10T09:17:16.521 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls 2024-09-10T09:17:16.521 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-celtic.ko 2024-09-10T09:17:16.521 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-centeuro.ko 2024-09-10T09:17:16.521 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-croatian.ko 2024-09-10T09:17:16.521 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-cyrillic.ko 2024-09-10T09:17:16.521 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-gaelic.ko 2024-09-10T09:17:16.521 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-greek.ko 2024-09-10T09:17:16.522 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-iceland.ko 2024-09-10T09:17:16.522 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-inuit.ko 2024-09-10T09:17:16.522 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-roman.ko 2024-09-10T09:17:16.522 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-romanian.ko 2024-09-10T09:17:16.522 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-turkish.ko 2024-09-10T09:17:16.522 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp1250.ko 2024-09-10T09:17:16.522 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp1251.ko 2024-09-10T09:17:16.522 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp1255.ko 2024-09-10T09:17:16.522 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp737.ko 2024-09-10T09:17:16.522 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp775.ko 2024-09-10T09:17:16.522 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp850.ko 2024-09-10T09:17:16.522 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp852.ko 2024-09-10T09:17:16.522 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp855.ko 2024-09-10T09:17:16.523 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp857.ko 2024-09-10T09:17:16.523 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp860.ko 2024-09-10T09:17:16.523 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp861.ko 2024-09-10T09:17:16.523 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp862.ko 2024-09-10T09:17:16.523 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp863.ko 2024-09-10T09:17:16.523 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp864.ko 2024-09-10T09:17:16.523 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp865.ko 2024-09-10T09:17:16.523 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp866.ko 2024-09-10T09:17:16.523 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp869.ko 2024-09-10T09:17:16.523 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp874.ko 2024-09-10T09:17:16.523 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp932.ko 2024-09-10T09:17:16.523 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp936.ko 2024-09-10T09:17:16.523 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp949.ko 2024-09-10T09:17:16.523 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp950.ko 2024-09-10T09:17:16.524 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_euc-jp.ko 2024-09-10T09:17:16.524 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-1.ko 2024-09-10T09:17:16.524 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-13.ko 2024-09-10T09:17:16.524 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-14.ko 2024-09-10T09:17:16.524 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-15.ko 2024-09-10T09:17:16.524 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-2.ko 2024-09-10T09:17:16.524 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-3.ko 2024-09-10T09:17:16.524 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-4.ko 2024-09-10T09:17:16.524 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-5.ko 2024-09-10T09:17:16.524 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-6.ko 2024-09-10T09:17:16.524 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-7.ko 2024-09-10T09:17:16.524 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-9.ko 2024-09-10T09:17:16.524 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_koi8-r.ko 2024-09-10T09:17:16.524 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_koi8-ru.ko 2024-09-10T09:17:16.525 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_koi8-u.ko 2024-09-10T09:17:16.525 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_ucs2_utils.ko 2024-09-10T09:17:16.525 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_utf8.ko 2024-09-10T09:17:16.525 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2 2024-09-10T09:17:16.525 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/cluster 2024-09-10T09:17:16.525 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-09-10T09:17:16.525 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlm 2024-09-10T09:17:16.525 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-09-10T09:17:16.525 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlmfs 2024-09-10T09:17:16.525 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-09-10T09:17:16.525 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2.ko 2024-09-10T09:17:16.525 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-09-10T09:17:16.525 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-09-10T09:17:16.526 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-09-10T09:17:16.526 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/overlayfs 2024-09-10T09:17:16.526 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/overlayfs/overlay.ko 2024-09-10T09:17:16.526 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/pstore 2024-09-10T09:17:16.526 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/pstore/ramoops.ko 2024-09-10T09:17:16.526 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/reiserfs 2024-09-10T09:17:16.526 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/reiserfs/reiserfs.ko 2024-09-10T09:17:16.526 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/romfs 2024-09-10T09:17:16.526 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/romfs/romfs.ko 2024-09-10T09:17:16.526 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb 2024-09-10T09:17:16.526 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/client 2024-09-10T09:17:16.526 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/client/cifs.ko 2024-09-10T09:17:16.526 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/common 2024-09-10T09:17:16.527 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/common/cifs_arc4.ko 2024-09-10T09:17:16.527 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/common/cifs_md4.ko 2024-09-10T09:17:16.527 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/squashfs 2024-09-10T09:17:16.527 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/squashfs/squashfs.ko 2024-09-10T09:17:16.527 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/sysv 2024-09-10T09:17:16.527 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/sysv/sysv.ko 2024-09-10T09:17:16.527 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/udf 2024-09-10T09:17:16.527 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/udf/udf.ko 2024-09-10T09:17:16.527 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ufs 2024-09-10T09:17:16.527 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ufs/ufs.ko 2024-09-10T09:17:16.527 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/xfs 2024-09-10T09:17:16.527 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/xfs/xfs.ko 2024-09-10T09:17:16.527 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel 2024-09-10T09:17:16.528 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/rcu 2024-09-10T09:17:16.528 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/rcu/rcutorture.ko 2024-09-10T09:17:16.528 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/torture.ko 2024-09-10T09:17:16.528 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/trace 2024-09-10T09:17:16.528 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-09-10T09:17:16.528 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib 2024-09-10T09:17:16.528 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/asn1_encoder.ko 2024-09-10T09:17:16.528 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crc-itu-t.ko 2024-09-10T09:17:16.528 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crc8.ko 2024-09-10T09:17:16.528 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crypto 2024-09-10T09:17:16.528 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crypto/libaescfb.ko 2024-09-10T09:17:16.528 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crypto/libarc4.ko 2024-09-10T09:17:16.528 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crypto/libdes.ko 2024-09-10T09:17:16.528 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/libcrc32c.ko 2024-09-10T09:17:16.529 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lru_cache.ko 2024-09-10T09:17:16.529 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lz4 2024-09-10T09:17:16.529 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lz4/lz4_compress.ko 2024-09-10T09:17:16.529 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lz4/lz4hc_compress.ko 2024-09-10T09:17:16.529 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/math 2024-09-10T09:17:16.529 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/math/cordic.ko 2024-09-10T09:17:16.529 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/raid6 2024-09-10T09:17:16.529 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/raid6/raid6_pq.ko 2024-09-10T09:17:16.529 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/reed_solomon 2024-09-10T09:17:16.529 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/reed_solomon/reed_solomon.ko 2024-09-10T09:17:16.529 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/ts_bm.ko 2024-09-10T09:17:16.529 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/ts_fsm.ko 2024-09-10T09:17:16.529 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/ts_kmp.ko 2024-09-10T09:17:16.529 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/zstd 2024-09-10T09:17:16.530 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/zstd/zstd_compress.ko 2024-09-10T09:17:16.530 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/mm 2024-09-10T09:17:16.530 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/mm/hwpoison-inject.ko 2024-09-10T09:17:16.530 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net 2024-09-10T09:17:16.530 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan 2024-09-10T09:17:16.530 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/6lowpan.ko 2024-09-10T09:17:16.530 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_dest.ko 2024-09-10T09:17:16.530 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_fragment.ko 2024-09-10T09:17:16.530 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_hop.ko 2024-09-10T09:17:16.530 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_ipv6.ko 2024-09-10T09:17:16.530 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_mobility.ko 2024-09-10T09:17:16.530 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_routing.ko 2024-09-10T09:17:16.530 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_udp.ko 2024-09-10T09:17:16.531 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802 2024-09-10T09:17:16.531 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/garp.ko 2024-09-10T09:17:16.531 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/mrp.ko 2024-09-10T09:17:16.531 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/p8022.ko 2024-09-10T09:17:16.531 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/psnap.ko 2024-09-10T09:17:16.531 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/stp.ko 2024-09-10T09:17:16.531 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/8021q 2024-09-10T09:17:16.531 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/8021q/8021q.ko 2024-09-10T09:17:16.531 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p 2024-09-10T09:17:16.531 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet.ko 2024-09-10T09:17:16.531 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet_fd.ko 2024-09-10T09:17:16.531 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet_rdma.ko 2024-09-10T09:17:16.531 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet_virtio.ko 2024-09-10T09:17:16.532 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/batman-adv 2024-09-10T09:17:16.532 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/batman-adv/batman-adv.ko 2024-09-10T09:17:16.532 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge 2024-09-10T09:17:16.532 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/br_netfilter.ko 2024-09-10T09:17:16.532 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/bridge.ko 2024-09-10T09:17:16.532 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter 2024-09-10T09:17:16.532 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-09-10T09:17:16.532 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_among.ko 2024-09-10T09:17:16.532 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_arp.ko 2024-09-10T09:17:16.532 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-09-10T09:17:16.532 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-09-10T09:17:16.532 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_ip.ko 2024-09-10T09:17:16.532 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-09-10T09:17:16.533 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_limit.ko 2024-09-10T09:17:16.533 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_log.ko 2024-09-10T09:17:16.533 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_mark.ko 2024-09-10T09:17:16.533 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-09-10T09:17:16.533 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-09-10T09:17:16.533 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-09-10T09:17:16.533 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-09-10T09:17:16.533 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_snat.ko 2024-09-10T09:17:16.533 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_stp.ko 2024-09-10T09:17:16.533 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-09-10T09:17:16.533 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-09-10T09:17:16.533 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-09-10T09:17:16.533 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-09-10T09:17:16.534 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtables.ko 2024-09-10T09:17:16.534 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-09-10T09:17:16.534 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-09-10T09:17:16.534 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-09-10T09:17:16.534 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ceph 2024-09-10T09:17:16.534 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ceph/libceph.ko 2024-09-10T09:17:16.534 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/core 2024-09-10T09:17:16.534 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/core/failover.ko 2024-09-10T09:17:16.534 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/core/pktgen.ko 2024-09-10T09:17:16.534 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp 2024-09-10T09:17:16.534 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp.ko 2024-09-10T09:17:16.534 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp_diag.ko 2024-09-10T09:17:16.534 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp_ipv4.ko 2024-09-10T09:17:16.535 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp_ipv6.ko 2024-09-10T09:17:16.535 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dns_resolver 2024-09-10T09:17:16.535 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dns_resolver/dns_resolver.ko 2024-09-10T09:17:16.535 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa 2024-09-10T09:17:16.535 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/dsa_core.ko 2024-09-10T09:17:16.535 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_brcm.ko 2024-09-10T09:17:16.535 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_dsa.ko 2024-09-10T09:17:16.535 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_none.ko 2024-09-10T09:17:16.535 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_trailer.ko 2024-09-10T09:17:16.535 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154 2024-09-10T09:17:16.535 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/6lowpan 2024-09-10T09:17:16.535 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-09-10T09:17:16.535 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/ieee802154.ko 2024-09-10T09:17:16.535 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/ieee802154_socket.ko 2024-09-10T09:17:16.536 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4 2024-09-10T09:17:16.536 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ah4.ko 2024-09-10T09:17:16.536 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/esp4.ko 2024-09-10T09:17:16.536 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/fou.ko 2024-09-10T09:17:16.536 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/gre.ko 2024-09-10T09:17:16.536 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/inet_diag.ko 2024-09-10T09:17:16.536 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ip_gre.ko 2024-09-10T09:17:16.536 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ip_tunnel.ko 2024-09-10T09:17:16.536 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ip_vti.ko 2024-09-10T09:17:16.536 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ipcomp.ko 2024-09-10T09:17:16.536 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ipip.ko 2024-09-10T09:17:16.536 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter 2024-09-10T09:17:16.537 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/arp_tables.ko 2024-09-10T09:17:16.537 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-09-10T09:17:16.537 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-09-10T09:17:16.537 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-09-10T09:17:16.537 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-09-10T09:17:16.537 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-09-10T09:17:16.537 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-09-10T09:17:16.537 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-09-10T09:17:16.537 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-09-10T09:17:16.537 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-09-10T09:17:16.537 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_security.ko 2024-09-10T09:17:16.537 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-09-10T09:17:16.537 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-09-10T09:17:16.538 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-09-10T09:17:16.538 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-09-10T09:17:16.538 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-09-10T09:17:16.538 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-09-10T09:17:16.538 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-09-10T09:17:16.538 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-09-10T09:17:16.538 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-09-10T09:17:16.538 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-09-10T09:17:16.538 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_bic.ko 2024-09-10T09:17:16.538 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_dctcp.ko 2024-09-10T09:17:16.538 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_diag.ko 2024-09-10T09:17:16.538 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_highspeed.ko 2024-09-10T09:17:16.538 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_htcp.ko 2024-09-10T09:17:16.538 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_hybla.ko 2024-09-10T09:17:16.539 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_illinois.ko 2024-09-10T09:17:16.539 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_lp.ko 2024-09-10T09:17:16.539 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_scalable.ko 2024-09-10T09:17:16.539 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_vegas.ko 2024-09-10T09:17:16.539 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_veno.ko 2024-09-10T09:17:16.539 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_westwood.ko 2024-09-10T09:17:16.539 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_yeah.ko 2024-09-10T09:17:16.539 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tunnel4.ko 2024-09-10T09:17:16.539 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/udp_diag.ko 2024-09-10T09:17:16.539 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/udp_tunnel.ko 2024-09-10T09:17:16.539 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/xfrm4_tunnel.ko 2024-09-10T09:17:16.539 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6 2024-09-10T09:17:16.539 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ah6.ko 2024-09-10T09:17:16.540 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/esp6.ko 2024-09-10T09:17:16.540 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/fou6.ko 2024-09-10T09:17:16.540 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ip6_tunnel.ko 2024-09-10T09:17:16.540 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-09-10T09:17:16.540 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ip6_vti.ko 2024-09-10T09:17:16.540 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ipcomp6.ko 2024-09-10T09:17:16.540 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter 2024-09-10T09:17:16.540 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-09-10T09:17:16.540 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-09-10T09:17:16.540 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-09-10T09:17:16.540 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-09-10T09:17:16.540 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-09-10T09:17:16.540 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-09-10T09:17:16.541 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-09-10T09:17:16.541 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-09-10T09:17:16.541 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-09-10T09:17:16.541 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-09-10T09:17:16.541 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-09-10T09:17:16.541 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-09-10T09:17:16.541 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-09-10T09:17:16.541 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-09-10T09:17:16.541 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-09-10T09:17:16.541 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-09-10T09:17:16.541 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-09-10T09:17:16.541 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-09-10T09:17:16.541 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-09-10T09:17:16.542 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-09-10T09:17:16.542 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-09-10T09:17:16.542 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-09-10T09:17:16.542 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-09-10T09:17:16.542 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-09-10T09:17:16.542 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/sit.ko 2024-09-10T09:17:16.542 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/tunnel6.ko 2024-09-10T09:17:16.542 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/xfrm6_tunnel.ko 2024-09-10T09:17:16.542 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/key 2024-09-10T09:17:16.542 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/key/af_key.ko 2024-09-10T09:17:16.542 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp 2024-09-10T09:17:16.542 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_core.ko 2024-09-10T09:17:16.542 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_debugfs.ko 2024-09-10T09:17:16.543 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_eth.ko 2024-09-10T09:17:16.543 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_ip.ko 2024-09-10T09:17:16.543 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_ip6.ko 2024-09-10T09:17:16.543 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_netlink.ko 2024-09-10T09:17:16.543 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_ppp.ko 2024-09-10T09:17:16.543 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/llc 2024-09-10T09:17:16.543 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/llc/llc.ko 2024-09-10T09:17:16.543 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mac802154 2024-09-10T09:17:16.543 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mac802154/mac802154.ko 2024-09-10T09:17:16.543 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mpls 2024-09-10T09:17:16.543 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mpls/mpls_gso.ko 2024-09-10T09:17:16.543 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter 2024-09-10T09:17:16.543 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset 2024-09-10T09:17:16.543 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set.ko 2024-09-10T09:17:16.544 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-09-10T09:17:16.544 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-09-10T09:17:16.544 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-09-10T09:17:16.544 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-09-10T09:17:16.544 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-09-10T09:17:16.544 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-09-10T09:17:16.544 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-09-10T09:17:16.544 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-09-10T09:17:16.544 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-09-10T09:17:16.544 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-09-10T09:17:16.544 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-09-10T09:17:16.544 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-09-10T09:17:16.545 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-09-10T09:17:16.545 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-09-10T09:17:16.545 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-09-10T09:17:16.545 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs 2024-09-10T09:17:16.545 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs.ko 2024-09-10T09:17:16.545 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-09-10T09:17:16.545 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-09-10T09:17:16.545 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-09-10T09:17:16.545 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-09-10T09:17:16.545 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-09-10T09:17:16.545 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-09-10T09:17:16.545 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-09-10T09:17:16.546 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-09-10T09:17:16.546 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-09-10T09:17:16.546 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-09-10T09:17:16.546 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-09-10T09:17:16.546 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-09-10T09:17:16.546 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-09-10T09:17:16.546 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conncount.ko 2024-09-10T09:17:16.546 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack.ko 2024-09-10T09:17:16.546 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-09-10T09:17:16.546 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-09-10T09:17:16.546 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-09-10T09:17:16.546 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_h323.ko 2024-09-10T09:17:16.546 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_irc.ko 2024-09-10T09:17:16.546 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-09-10T09:17:16.547 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-09-10T09:17:16.547 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-09-10T09:17:16.547 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_sane.ko 2024-09-10T09:17:16.547 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_sip.ko 2024-09-10T09:17:16.547 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-09-10T09:17:16.547 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-09-10T09:17:16.547 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_dup_netdev.ko 2024-09-10T09:17:16.547 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_flow_table.ko 2024-09-10T09:17:16.547 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_flow_table_inet.ko 2024-09-10T09:17:16.547 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_log_syslog.ko 2024-09-10T09:17:16.547 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat.ko 2024-09-10T09:17:16.547 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_amanda.ko 2024-09-10T09:17:16.548 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_ftp.ko 2024-09-10T09:17:16.548 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_irc.ko 2024-09-10T09:17:16.548 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_sip.ko 2024-09-10T09:17:16.548 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_tftp.ko 2024-09-10T09:17:16.548 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_synproxy_core.ko 2024-09-10T09:17:16.548 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_tables.ko 2024-09-10T09:17:16.548 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink.ko 2024-09-10T09:17:16.548 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_acct.ko 2024-09-10T09:17:16.548 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_log.ko 2024-09-10T09:17:16.548 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_osf.ko 2024-09-10T09:17:16.548 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_queue.ko 2024-09-10T09:17:16.548 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_chain_nat.ko 2024-09-10T09:17:16.549 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_compat.ko 2024-09-10T09:17:16.549 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_ct.ko 2024-09-10T09:17:16.549 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_dup_netdev.ko 2024-09-10T09:17:16.549 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_fib.ko 2024-09-10T09:17:16.549 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_fwd_netdev.ko 2024-09-10T09:17:16.549 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_hash.ko 2024-09-10T09:17:16.549 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_limit.ko 2024-09-10T09:17:16.549 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_log.ko 2024-09-10T09:17:16.549 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_masq.ko 2024-09-10T09:17:16.549 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_nat.ko 2024-09-10T09:17:16.549 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_numgen.ko 2024-09-10T09:17:16.549 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_queue.ko 2024-09-10T09:17:16.549 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_quota.ko 2024-09-10T09:17:16.549 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_redir.ko 2024-09-10T09:17:16.550 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_reject.ko 2024-09-10T09:17:16.550 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_reject_inet.ko 2024-09-10T09:17:16.550 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_socket.ko 2024-09-10T09:17:16.550 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_tunnel.ko 2024-09-10T09:17:16.550 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_AUDIT.ko 2024-09-10T09:17:16.550 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CHECKSUM.ko 2024-09-10T09:17:16.550 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CLASSIFY.ko 2024-09-10T09:17:16.550 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-09-10T09:17:16.550 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CT.ko 2024-09-10T09:17:16.550 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_DSCP.ko 2024-09-10T09:17:16.550 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_HL.ko 2024-09-10T09:17:16.550 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_HMARK.ko 2024-09-10T09:17:16.550 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_IDLETIMER.ko 2024-09-10T09:17:16.551 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_LED.ko 2024-09-10T09:17:16.551 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_LOG.ko 2024-09-10T09:17:16.551 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_MASQUERADE.ko 2024-09-10T09:17:16.551 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_NETMAP.ko 2024-09-10T09:17:16.551 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_NFLOG.ko 2024-09-10T09:17:16.551 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_NFQUEUE.ko 2024-09-10T09:17:16.551 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_RATEEST.ko 2024-09-10T09:17:16.551 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_REDIRECT.ko 2024-09-10T09:17:16.551 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_SECMARK.ko 2024-09-10T09:17:16.551 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TCPMSS.ko 2024-09-10T09:17:16.551 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-09-10T09:17:16.551 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TEE.ko 2024-09-10T09:17:16.551 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TPROXY.ko 2024-09-10T09:17:16.551 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TRACE.ko 2024-09-10T09:17:16.552 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_addrtype.ko 2024-09-10T09:17:16.552 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_bpf.ko 2024-09-10T09:17:16.552 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_cgroup.ko 2024-09-10T09:17:16.552 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_cluster.ko 2024-09-10T09:17:16.552 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_comment.ko 2024-09-10T09:17:16.552 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connbytes.ko 2024-09-10T09:17:16.552 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connlabel.ko 2024-09-10T09:17:16.552 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connlimit.ko 2024-09-10T09:17:16.552 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connmark.ko 2024-09-10T09:17:16.552 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_conntrack.ko 2024-09-10T09:17:16.552 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_cpu.ko 2024-09-10T09:17:16.552 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_dccp.ko 2024-09-10T09:17:16.553 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_devgroup.ko 2024-09-10T09:17:16.553 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_dscp.ko 2024-09-10T09:17:16.553 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_ecn.ko 2024-09-10T09:17:16.553 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_esp.ko 2024-09-10T09:17:16.553 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_hashlimit.ko 2024-09-10T09:17:16.553 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_helper.ko 2024-09-10T09:17:16.553 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_hl.ko 2024-09-10T09:17:16.553 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_ipcomp.ko 2024-09-10T09:17:16.553 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_iprange.ko 2024-09-10T09:17:16.553 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_ipvs.ko 2024-09-10T09:17:16.553 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_l2tp.ko 2024-09-10T09:17:16.553 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_length.ko 2024-09-10T09:17:16.553 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_limit.ko 2024-09-10T09:17:16.553 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_mac.ko 2024-09-10T09:17:16.554 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_mark.ko 2024-09-10T09:17:16.554 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_multiport.ko 2024-09-10T09:17:16.554 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_nat.ko 2024-09-10T09:17:16.554 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_nfacct.ko 2024-09-10T09:17:16.554 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_osf.ko 2024-09-10T09:17:16.554 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_owner.ko 2024-09-10T09:17:16.554 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_physdev.ko 2024-09-10T09:17:16.554 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_pkttype.ko 2024-09-10T09:17:16.554 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_policy.ko 2024-09-10T09:17:16.554 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_quota.ko 2024-09-10T09:17:16.554 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_rateest.ko 2024-09-10T09:17:16.554 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_realm.ko 2024-09-10T09:17:16.554 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_recent.ko 2024-09-10T09:17:16.554 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_sctp.ko 2024-09-10T09:17:16.555 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_set.ko 2024-09-10T09:17:16.555 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_socket.ko 2024-09-10T09:17:16.555 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_state.ko 2024-09-10T09:17:16.555 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_statistic.ko 2024-09-10T09:17:16.555 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_string.ko 2024-09-10T09:17:16.555 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_tcpmss.ko 2024-09-10T09:17:16.555 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_time.ko 2024-09-10T09:17:16.555 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_u32.ko 2024-09-10T09:17:16.555 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netlink 2024-09-10T09:17:16.555 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netlink/netlink_diag.ko 2024-09-10T09:17:16.555 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/nsh 2024-09-10T09:17:16.555 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/nsh/nsh.ko 2024-09-10T09:17:16.555 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch 2024-09-10T09:17:16.556 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/openvswitch.ko 2024-09-10T09:17:16.556 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/vport-geneve.ko 2024-09-10T09:17:16.556 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/vport-gre.ko 2024-09-10T09:17:16.556 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/vport-vxlan.ko 2024-09-10T09:17:16.556 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/packet 2024-09-10T09:17:16.556 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/packet/af_packet_diag.ko 2024-09-10T09:17:16.556 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds 2024-09-10T09:17:16.556 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds/rds.ko 2024-09-10T09:17:16.556 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds/rds_rdma.ko 2024-09-10T09:17:16.556 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds/rds_tcp.ko 2024-09-10T09:17:16.556 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched 2024-09-10T09:17:16.556 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_csum.ko 2024-09-10T09:17:16.556 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_gact.ko 2024-09-10T09:17:16.557 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_mirred.ko 2024-09-10T09:17:16.557 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_nat.ko 2024-09-10T09:17:16.557 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_pedit.ko 2024-09-10T09:17:16.557 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_police.ko 2024-09-10T09:17:16.557 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_simple.ko 2024-09-10T09:17:16.557 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_skbedit.ko 2024-09-10T09:17:16.557 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_vlan.ko 2024-09-10T09:17:16.557 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_basic.ko 2024-09-10T09:17:16.557 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_bpf.ko 2024-09-10T09:17:16.557 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_flow.ko 2024-09-10T09:17:16.557 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_fw.ko 2024-09-10T09:17:16.557 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_route.ko 2024-09-10T09:17:16.557 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_u32.ko 2024-09-10T09:17:16.558 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_cmp.ko 2024-09-10T09:17:16.558 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_ipset.ko 2024-09-10T09:17:16.558 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_meta.ko 2024-09-10T09:17:16.558 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_nbyte.ko 2024-09-10T09:17:16.558 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_text.ko 2024-09-10T09:17:16.558 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_u32.ko 2024-09-10T09:17:16.558 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_choke.ko 2024-09-10T09:17:16.558 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_codel.ko 2024-09-10T09:17:16.558 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_drr.ko 2024-09-10T09:17:16.558 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_fq.ko 2024-09-10T09:17:16.558 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_gred.ko 2024-09-10T09:17:16.558 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_hfsc.ko 2024-09-10T09:17:16.558 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_hhf.ko 2024-09-10T09:17:16.558 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_htb.ko 2024-09-10T09:17:16.559 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_ingress.ko 2024-09-10T09:17:16.559 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_mqprio.ko 2024-09-10T09:17:16.559 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_mqprio_lib.ko 2024-09-10T09:17:16.559 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_multiq.ko 2024-09-10T09:17:16.559 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_netem.ko 2024-09-10T09:17:16.559 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_pie.ko 2024-09-10T09:17:16.559 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_plug.ko 2024-09-10T09:17:16.559 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_prio.ko 2024-09-10T09:17:16.559 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_qfq.ko 2024-09-10T09:17:16.559 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_red.ko 2024-09-10T09:17:16.559 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_sfb.ko 2024-09-10T09:17:16.559 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_sfq.ko 2024-09-10T09:17:16.559 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_tbf.ko 2024-09-10T09:17:16.560 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_teql.ko 2024-09-10T09:17:16.560 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sctp 2024-09-10T09:17:16.560 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sctp/sctp.ko 2024-09-10T09:17:16.560 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sctp/sctp_diag.ko 2024-09-10T09:17:16.560 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc 2024-09-10T09:17:16.560 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/auth_gss 2024-09-10T09:17:16.560 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-09-10T09:17:16.560 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-09-10T09:17:16.560 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/sunrpc.ko 2024-09-10T09:17:16.560 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/xprtrdma 2024-09-10T09:17:16.560 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-09-10T09:17:16.560 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/tipc 2024-09-10T09:17:16.560 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/tipc/tipc.ko 2024-09-10T09:17:16.561 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/tipc/tipc_diag.ko 2024-09-10T09:17:16.561 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/unix 2024-09-10T09:17:16.561 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/unix/unix_diag.ko 2024-09-10T09:17:16.561 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock 2024-09-10T09:17:16.561 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-09-10T09:17:16.561 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-09-10T09:17:16.561 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vsock.ko 2024-09-10T09:17:16.561 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vsock_diag.ko 2024-09-10T09:17:16.561 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vsock_loopback.ko 2024-09-10T09:17:16.561 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/xfrm 2024-09-10T09:17:16.561 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/xfrm/xfrm_ipcomp.ko 2024-09-10T09:17:16.561 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security 2024-09-10T09:17:16.561 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security/keys 2024-09-10T09:17:16.562 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security/keys/encrypted-keys 2024-09-10T09:17:16.562 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-09-10T09:17:16.562 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security/keys/trusted-keys 2024-09-10T09:17:16.562 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security/keys/trusted-keys/trusted.ko 2024-09-10T09:17:16.562 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.alias 2024-09-10T09:17:16.562 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.alias.bin 2024-09-10T09:17:16.562 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin 2024-09-10T09:17:16.562 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin.alias.bin 2024-09-10T09:17:16.562 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin.bin 2024-09-10T09:17:16.562 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin.modinfo 2024-09-10T09:17:16.562 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.dep 2024-09-10T09:17:16.562 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.dep.bin 2024-09-10T09:17:16.562 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.devname 2024-09-10T09:17:16.563 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.order 2024-09-10T09:17:16.563 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.softdep 2024-09-10T09:17:16.563 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.symbols 2024-09-10T09:17:16.563 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.symbols.bin 2024-09-10T09:17:16.563 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.weakdep 2024-09-10T09:17:16.563 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/vmlinuz 2024-09-10T09:17:16.567 DEBUG:teuthology.task.kernel:get_image_version: 6.11.0-rc5-g2ca44a2f35ca 2024-09-10T09:17:16.567 DEBUG:teuthology.orchestra.run.smithi005:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-09-10T09:17:16.591 INFO:teuthology.orchestra.run.smithi005.stdout:postinstall scriptlet (using /bin/sh): 2024-09-10T09:17:16.592 INFO:teuthology.orchestra.run.smithi005.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-09-10T09:17:16.592 INFO:teuthology.orchestra.run.smithi005.stdout: /usr/bin/kernel-install add 6.11.0-rc5-g2ca44a2f35ca /lib/modules/6.11.0-rc5-g2ca44a2f35ca/vmlinuz 2024-09-10T09:17:16.592 INFO:teuthology.orchestra.run.smithi005.stdout:fi 2024-09-10T09:17:16.592 INFO:teuthology.orchestra.run.smithi005.stdout:for file in vmlinuz System.map config; do 2024-09-10T09:17:16.592 INFO:teuthology.orchestra.run.smithi005.stdout: if ! cmp --silent "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/${file}" "/boot/${file}-6.11.0-rc5-g2ca44a2f35ca"; then 2024-09-10T09:17:16.592 INFO:teuthology.orchestra.run.smithi005.stdout: cp "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/${file}" "/boot/${file}-6.11.0-rc5-g2ca44a2f35ca" 2024-09-10T09:17:16.592 INFO:teuthology.orchestra.run.smithi005.stdout: fi 2024-09-10T09:17:16.592 INFO:teuthology.orchestra.run.smithi005.stdout:done 2024-09-10T09:17:16.592 INFO:teuthology.orchestra.run.smithi005.stdout:if [ -d "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/dtb" ] && \ 2024-09-10T09:17:16.592 INFO:teuthology.orchestra.run.smithi005.stdout: ! diff -rq "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/dtb" "/boot/dtb-6.11.0-rc5-g2ca44a2f35ca" >/dev/null 2>&1; then 2024-09-10T09:17:16.592 INFO:teuthology.orchestra.run.smithi005.stdout: rm -rf "/boot/dtb-6.11.0-rc5-g2ca44a2f35ca" 2024-09-10T09:17:16.593 INFO:teuthology.orchestra.run.smithi005.stdout: cp -r "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/dtb" "/boot/dtb-6.11.0-rc5-g2ca44a2f35ca" 2024-09-10T09:17:16.593 INFO:teuthology.orchestra.run.smithi005.stdout:fi 2024-09-10T09:17:16.593 INFO:teuthology.orchestra.run.smithi005.stdout:if [ ! -e "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.dep" ]; then 2024-09-10T09:17:16.593 INFO:teuthology.orchestra.run.smithi005.stdout: /usr/sbin/depmod 6.11.0-rc5-g2ca44a2f35ca 2024-09-10T09:17:16.593 INFO:teuthology.orchestra.run.smithi005.stdout:fi 2024-09-10T09:17:16.593 INFO:teuthology.orchestra.run.smithi005.stdout:preuninstall scriptlet (using /bin/sh): 2024-09-10T09:17:16.593 INFO:teuthology.orchestra.run.smithi005.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-09-10T09:17:16.593 INFO:teuthology.orchestra.run.smithi005.stdout:kernel-install remove 6.11.0-rc5-g2ca44a2f35ca 2024-09-10T09:17:16.593 INFO:teuthology.orchestra.run.smithi005.stdout:fi 2024-09-10T09:17:16.594 DEBUG:teuthology.orchestra.run.smithi005:> sudo rpm -qi grub2-tools 2024-09-10T09:17:16.669 INFO:teuthology.orchestra.run.smithi005.stdout:Name : grub2-tools 2024-09-10T09:17:16.670 INFO:teuthology.orchestra.run.smithi005.stdout:Epoch : 1 2024-09-10T09:17:16.670 INFO:teuthology.orchestra.run.smithi005.stdout:Version : 2.06 2024-09-10T09:17:16.670 INFO:teuthology.orchestra.run.smithi005.stdout:Release : 82.el9 2024-09-10T09:17:16.670 INFO:teuthology.orchestra.run.smithi005.stdout:Architecture: x86_64 2024-09-10T09:17:16.670 INFO:teuthology.orchestra.run.smithi005.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-10T09:17:16.670 INFO:teuthology.orchestra.run.smithi005.stdout:Group : Unspecified 2024-09-10T09:17:16.670 INFO:teuthology.orchestra.run.smithi005.stdout:Size : 8274670 2024-09-10T09:17:16.670 INFO:teuthology.orchestra.run.smithi005.stdout:License : GPLv3+ 2024-09-10T09:17:16.670 INFO:teuthology.orchestra.run.smithi005.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-10T09:17:16.670 INFO:teuthology.orchestra.run.smithi005.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-10T09:17:16.670 INFO:teuthology.orchestra.run.smithi005.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-10T09:17:16.670 INFO:teuthology.orchestra.run.smithi005.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-10T09:17:16.670 INFO:teuthology.orchestra.run.smithi005.stdout:Packager : builder@centos.org 2024-09-10T09:17:16.671 INFO:teuthology.orchestra.run.smithi005.stdout:Vendor : CentOS 2024-09-10T09:17:16.671 INFO:teuthology.orchestra.run.smithi005.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-10T09:17:16.671 INFO:teuthology.orchestra.run.smithi005.stdout:Summary : Support tools for GRUB. 2024-09-10T09:17:16.671 INFO:teuthology.orchestra.run.smithi005.stdout:Description : 2024-09-10T09:17:16.671 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-10T09:17:16.671 INFO:teuthology.orchestra.run.smithi005.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-10T09:17:16.671 INFO:teuthology.orchestra.run.smithi005.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-10T09:17:16.671 INFO:teuthology.orchestra.run.smithi005.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-10T09:17:16.671 INFO:teuthology.orchestra.run.smithi005.stdout:hardware devices. 2024-09-10T09:17:16.671 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-10T09:17:16.671 INFO:teuthology.orchestra.run.smithi005.stdout:This subpackage provides tools for support of all platforms. 2024-09-10T09:17:16.672 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-10T09:17:16.672 INFO:teuthology.task.kernel:Updating grub on smithi005 to boot 6.11.0-rc5-g2ca44a2f35ca 2024-09-10T09:17:16.672 DEBUG:teuthology.orchestra.run.smithi005:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-10T09:17:17.258 INFO:teuthology.orchestra.run.smithi005.stderr:Generating grub configuration file ... 2024-09-10T09:17:18.675 INFO:teuthology.orchestra.run.smithi005.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-10T09:17:18.690 INFO:teuthology.orchestra.run.smithi005.stderr:done 2024-09-10T09:17:18.693 DEBUG:teuthology.orchestra.run.smithi005:> mktemp 2024-09-10T09:17:18.708 INFO:teuthology.orchestra.run.smithi005.stdout:/tmp/tmp.QHUx1D5nxk 2024-09-10T09:17:18.708 DEBUG:teuthology.orchestra.run.smithi005:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.QHUx1D5nxk 2024-09-10T09:17:18.775 DEBUG:teuthology.orchestra.run.smithi005:> sudo chmod 0666 /tmp/tmp.QHUx1D5nxk 2024-09-10T09:17:18.920 DEBUG:teuthology.orchestra.remote:smithi005:/tmp/tmp.QHUx1D5nxk is 6KB 2024-09-10T09:17:18.968 DEBUG:teuthology.orchestra.run.smithi005:> rm -fr /tmp/tmp.QHUx1D5nxk 2024-09-10T09:17:18.983 DEBUG:teuthology.orchestra.run.smithi005:> sudo /bin/ls /boot/loader/entries || true 2024-09-10T09:17:19.046 INFO:teuthology.orchestra.run.smithi005.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-10T09:17:19.046 INFO:teuthology.orchestra.run.smithi005.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-10T09:17:19.047 INFO:teuthology.orchestra.run.smithi005.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-10T09:17:19.047 INFO:teuthology.orchestra.run.smithi005.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-6.11.0-rc5-g2ca44a2f35ca.conf 2024-09-10T09:17:19.048 DEBUG:teuthology.orchestra.run.smithi005:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-6.11.0-rc5-g2ca44a2f35ca 2024-09-10T09:17:19.221 DEBUG:teuthology.orchestra.run.smithi005:> sudo shutdown -r now 2024-09-10T09:17:49.224 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-09-10T09:17:49.225 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi005.front.sepia.ceph.com' 2024-09-10T09:17:49.226 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi005.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T09:18:07.677 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.5 2024-09-10T09:18:16.678 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi005.front.sepia.ceph.com' 2024-09-10T09:18:16.679 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi005.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T09:19:16.682 DEBUG:teuthology.orchestra.remote:timed out 2024-09-10T09:19:28.683 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi005.front.sepia.ceph.com' 2024-09-10T09:19:28.684 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi005.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T09:19:28.955 DEBUG:teuthology.orchestra.run.smithi005:> true 2024-09-10T09:19:29.441 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi005.front.sepia.ceph.com' 2024-09-10T09:19:29.442 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "6.11.0-rc5-g2ca44a2f35ca"... 2024-09-10T09:19:29.442 DEBUG:teuthology.orchestra.run.smithi005:> uname -r 2024-09-10T09:19:29.477 INFO:teuthology.orchestra.run.smithi005.stdout:6.11.0-rc5-g2ca44a2f35ca 2024-09-10T09:19:29.477 DEBUG:teuthology.task.kernel:current kernel version is 6.11.0-rc5-g2ca44a2f35ca vs 6.11.0-rc5-g2ca44a2f35ca 2024-09-10T09:19:29.477 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-10T09:19:29.478 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-10T09:19:30.479 DEBUG:teuthology.parallel:result is None 2024-09-10T09:19:30.479 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-10T09:19:30.486 INFO:teuthology.task.internal:Creating test directory... 2024-09-10T09:19:30.486 DEBUG:teuthology.orchestra.run.smithi005:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-10T09:19:30.490 DEBUG:teuthology.orchestra.run.smithi102:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-10T09:19:30.492 DEBUG:teuthology.orchestra.run.smithi159:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-10T09:19:30.524 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-10T09:19:30.531 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-10T09:19:30.556 INFO:teuthology.task.internal:Creating archive directory... 2024-09-10T09:19:30.557 DEBUG:teuthology.orchestra.run.smithi005:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-10T09:19:30.568 DEBUG:teuthology.orchestra.run.smithi102:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-10T09:19:30.570 DEBUG:teuthology.orchestra.run.smithi159:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-10T09:19:30.672 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-10T09:19:30.680 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-10T09:19:30.681 DEBUG:teuthology.orchestra.run.smithi102:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-10T09:19:30.698 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T09:19:30.698 DEBUG:teuthology.orchestra.run.smithi159:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-10T09:19:30.715 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T09:19:30.716 DEBUG:teuthology.orchestra.run.smithi005:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-10T09:19:30.720 DEBUG:teuthology.orchestra.run.smithi102:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-10T09:19:30.741 DEBUG:teuthology.orchestra.run.smithi159:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-10T09:19:30.774 INFO:teuthology.orchestra.run.smithi102.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-10T09:19:30.793 INFO:teuthology.orchestra.run.smithi102.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-10T09:19:30.832 INFO:teuthology.orchestra.run.smithi159.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-10T09:19:30.840 INFO:teuthology.orchestra.run.smithi005.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-10T09:19:30.846 INFO:teuthology.orchestra.run.smithi159.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-10T09:19:30.873 INFO:teuthology.orchestra.run.smithi005.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-10T09:19:30.876 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-10T09:19:30.883 INFO:teuthology.task.internal:Configuring sudo... 2024-09-10T09:19:30.883 DEBUG:teuthology.orchestra.run.smithi005:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-10T09:19:30.887 DEBUG:teuthology.orchestra.run.smithi102:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-10T09:19:30.889 DEBUG:teuthology.orchestra.run.smithi159:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-10T09:19:30.941 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-10T09:19:30.950 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-10T09:19:30.950 DEBUG:teuthology.orchestra.run.smithi005:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-10T09:19:30.954 DEBUG:teuthology.orchestra.run.smithi102:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-10T09:19:30.960 DEBUG:teuthology.orchestra.run.smithi159:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-10T09:19:30.985 DEBUG:teuthology.orchestra.run.smithi005:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-10T09:19:31.087 DEBUG:teuthology.orchestra.run.smithi005:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-10T09:19:31.183 DEBUG:teuthology.orchestra.run.smithi005:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-10T09:19:31.245 DEBUG:teuthology.orchestra.run.smithi005:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-10T09:19:31.318 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-09-10T09:19:31.318 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-10T09:19:31.373 DEBUG:teuthology.orchestra.run.smithi102:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-10T09:19:31.417 DEBUG:teuthology.orchestra.run.smithi102:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-10T09:19:31.478 DEBUG:teuthology.orchestra.run.smithi102:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-10T09:19:31.557 DEBUG:teuthology.orchestra.run.smithi102:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-10T09:19:31.609 DEBUG:teuthology.orchestra.run.smithi102:> set -ex 2024-09-10T09:19:31.609 DEBUG:teuthology.orchestra.run.smithi102:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-10T09:19:31.679 DEBUG:teuthology.orchestra.run.smithi159:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-10T09:19:31.717 DEBUG:teuthology.orchestra.run.smithi159:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-10T09:19:31.780 DEBUG:teuthology.orchestra.run.smithi159:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-10T09:19:31.863 DEBUG:teuthology.orchestra.run.smithi159:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-10T09:19:31.910 DEBUG:teuthology.orchestra.run.smithi159:> set -ex 2024-09-10T09:19:31.911 DEBUG:teuthology.orchestra.run.smithi159:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-10T09:19:31.977 DEBUG:teuthology.orchestra.run.smithi005:> sudo service rsyslog restart 2024-09-10T09:19:31.982 DEBUG:teuthology.orchestra.run.smithi102:> sudo service rsyslog restart 2024-09-10T09:19:31.984 DEBUG:teuthology.orchestra.run.smithi159:> sudo service rsyslog restart 2024-09-10T09:19:32.023 INFO:teuthology.orchestra.run.smithi102.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-10T09:19:32.056 INFO:teuthology.orchestra.run.smithi159.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-10T09:19:32.060 INFO:teuthology.orchestra.run.smithi005.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-10T09:19:32.421 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-10T09:19:32.429 INFO:teuthology.task.internal:Starting timer... 2024-09-10T09:19:32.429 INFO:teuthology.run_tasks:Running task pcp... 2024-09-10T09:19:32.453 INFO:teuthology.run_tasks:Running task selinux... 2024-09-10T09:19:32.484 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-09-10T09:19:32.485 DEBUG:teuthology.orchestra.run.smithi005:> sudo service auditd rotate 2024-09-10T09:19:32.574 INFO:teuthology.orchestra.run.smithi005.stdout:Rotating logs: 2024-09-10T09:19:32.579 DEBUG:teuthology.orchestra.run.smithi102:> sudo service auditd rotate 2024-09-10T09:19:32.637 INFO:teuthology.orchestra.run.smithi102.stdout:Rotating logs: 2024-09-10T09:19:32.639 DEBUG:teuthology.orchestra.run.smithi159:> sudo service auditd rotate 2024-09-10T09:19:32.723 INFO:teuthology.orchestra.run.smithi159.stdout:Rotating logs: 2024-09-10T09:19:32.725 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-10T09:19:32.725 DEBUG:teuthology.orchestra.run.smithi005:> /usr/sbin/getenforce 2024-09-10T09:19:32.768 INFO:teuthology.orchestra.run.smithi005.stdout:Permissive 2024-09-10T09:19:32.768 DEBUG:teuthology.orchestra.run.smithi102:> /usr/sbin/getenforce 2024-09-10T09:19:32.797 INFO:teuthology.orchestra.run.smithi102.stdout:Permissive 2024-09-10T09:19:32.797 DEBUG:teuthology.orchestra.run.smithi159:> /usr/sbin/getenforce 2024-09-10T09:19:32.825 INFO:teuthology.orchestra.run.smithi159.stdout:Permissive 2024-09-10T09:19:32.826 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi005.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi102.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi159.front.sepia.ceph.com': 'permissive'} 2024-09-10T09:19:32.826 DEBUG:teuthology.orchestra.run.smithi005:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-10T09:19:32.883 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T09:19:32.884 DEBUG:teuthology.orchestra.run.smithi102:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-10T09:19:32.911 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T09:19:32.912 DEBUG:teuthology.orchestra.run.smithi159:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-10T09:19:32.940 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T09:19:32.941 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-10T09:19:32.941 DEBUG:teuthology.orchestra.run.smithi005:> sudo /usr/sbin/setenforce permissive 2024-09-10T09:19:33.012 DEBUG:teuthology.orchestra.run.smithi102:> sudo /usr/sbin/setenforce permissive 2024-09-10T09:19:33.045 DEBUG:teuthology.orchestra.run.smithi159:> sudo /usr/sbin/setenforce permissive 2024-09-10T09:19:33.081 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-10T09:19:33.090 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-09-10T09:19:33.093 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-09-10T09:19:33.216 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-10T09:19:33.226 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-10T09:19:33.227 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi005.front.sepia.ceph.com,smithi102.front.sepia.ceph.com,smithi159.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-10T09:24:13.019 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi005.front.sepia.ceph.com'), Remote(name='ubuntu@smithi102.front.sepia.ceph.com'), Remote(name='ubuntu@smithi159.front.sepia.ceph.com')] 2024-09-10T09:24:13.020 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi005.front.sepia.ceph.com' 2024-09-10T09:24:13.021 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi005.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T09:24:13.114 DEBUG:teuthology.orchestra.run.smithi005:> true 2024-09-10T09:24:13.223 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi005.front.sepia.ceph.com' 2024-09-10T09:24:13.224 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi102.front.sepia.ceph.com' 2024-09-10T09:24:13.224 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi102.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T09:24:13.304 DEBUG:teuthology.orchestra.run.smithi102:> true 2024-09-10T09:24:13.339 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi102.front.sepia.ceph.com' 2024-09-10T09:24:13.339 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi159.front.sepia.ceph.com' 2024-09-10T09:24:13.340 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi159.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T09:24:13.430 DEBUG:teuthology.orchestra.run.smithi159:> true 2024-09-10T09:24:13.514 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi159.front.sepia.ceph.com' 2024-09-10T09:24:13.514 INFO:teuthology.run_tasks:Running task clock... 2024-09-10T09:24:13.524 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-10T09:24:13.524 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-10T09:24:13.525 DEBUG:teuthology.orchestra.run.smithi005:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-10T09:24:13.528 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-10T09:24:13.528 DEBUG:teuthology.orchestra.run.smithi102:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-10T09:24:13.531 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-10T09:24:13.531 DEBUG:teuthology.orchestra.run.smithi159:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-10T09:24:13.558 INFO:teuthology.orchestra.run.smithi102.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-10T09:24:13.573 INFO:teuthology.orchestra.run.smithi102.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-10T09:24:13.583 INFO:teuthology.orchestra.run.smithi159.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-10T09:24:13.588 INFO:teuthology.orchestra.run.smithi005.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-10T09:24:13.597 INFO:teuthology.orchestra.run.smithi102.stderr:sudo: ntpd: command not found 2024-09-10T09:24:13.598 INFO:teuthology.orchestra.run.smithi159.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-10T09:24:13.608 INFO:teuthology.orchestra.run.smithi102.stdout:506 Cannot talk to daemon 2024-09-10T09:24:13.618 INFO:teuthology.orchestra.run.smithi005.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-10T09:24:13.624 INFO:teuthology.orchestra.run.smithi159.stderr:sudo: ntpd: command not found 2024-09-10T09:24:13.624 INFO:teuthology.orchestra.run.smithi102.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-10T09:24:13.634 INFO:teuthology.orchestra.run.smithi159.stdout:506 Cannot talk to daemon 2024-09-10T09:24:13.636 INFO:teuthology.orchestra.run.smithi102.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-10T09:24:13.647 INFO:teuthology.orchestra.run.smithi159.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-10T09:24:13.661 INFO:teuthology.orchestra.run.smithi159.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-10T09:24:13.670 INFO:teuthology.orchestra.run.smithi005.stderr:sudo: ntpd: command not found 2024-09-10T09:24:13.692 INFO:teuthology.orchestra.run.smithi102.stderr:bash: line 1: ntpq: command not found 2024-09-10T09:24:13.694 INFO:teuthology.orchestra.run.smithi102.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-10T09:24:13.694 INFO:teuthology.orchestra.run.smithi102.stdout:=============================================================================== 2024-09-10T09:24:13.694 INFO:teuthology.orchestra.run.smithi102.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T09:24:13.694 INFO:teuthology.orchestra.run.smithi102.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T09:24:13.694 INFO:teuthology.orchestra.run.smithi102.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T09:24:13.694 INFO:teuthology.orchestra.run.smithi102.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T09:24:13.695 INFO:teuthology.orchestra.run.smithi005.stdout:506 Cannot talk to daemon 2024-09-10T09:24:13.717 INFO:teuthology.orchestra.run.smithi159.stderr:bash: line 1: ntpq: command not found 2024-09-10T09:24:13.721 INFO:teuthology.orchestra.run.smithi159.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-10T09:24:13.721 INFO:teuthology.orchestra.run.smithi159.stdout:=============================================================================== 2024-09-10T09:24:13.721 INFO:teuthology.orchestra.run.smithi159.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T09:24:13.721 INFO:teuthology.orchestra.run.smithi159.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T09:24:13.721 INFO:teuthology.orchestra.run.smithi159.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T09:24:13.721 INFO:teuthology.orchestra.run.smithi159.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T09:24:13.724 INFO:teuthology.orchestra.run.smithi005.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-10T09:24:13.754 INFO:teuthology.orchestra.run.smithi005.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-10T09:24:13.858 INFO:teuthology.orchestra.run.smithi005.stderr:bash: line 1: ntpq: command not found 2024-09-10T09:24:13.864 INFO:teuthology.orchestra.run.smithi005.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-10T09:24:13.865 INFO:teuthology.orchestra.run.smithi005.stdout:=============================================================================== 2024-09-10T09:24:13.865 INFO:teuthology.orchestra.run.smithi005.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T09:24:13.865 INFO:teuthology.orchestra.run.smithi005.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T09:24:13.865 INFO:teuthology.orchestra.run.smithi005.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T09:24:13.865 INFO:teuthology.orchestra.run.smithi005.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T09:24:13.866 INFO:teuthology.run_tasks:Running task install... 2024-09-10T09:24:13.874 DEBUG:teuthology.task.install:project ceph 2024-09-10T09:24:13.874 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '1cf819b7212a6048e92fc78c4eff963344f6d56b'}} 2024-09-10T09:24:13.874 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '1cf819b7212a6048e92fc78c4eff963344f6d56b'} 2024-09-10T09:24:13.875 INFO:teuthology.task.install:Using flavor: default 2024-09-10T09:24:13.880 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-09-10T09:24:13.880 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-09-10T09:24:13.881 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '1cf819b7212a6048e92fc78c4eff963344f6d56b', 'tag': None, 'wait_for_package': False} 2024-09-10T09:24:13.881 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1cf819b7212a6048e92fc78c4eff963344f6d56b 2024-09-10T09:24:13.884 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '1cf819b7212a6048e92fc78c4eff963344f6d56b', 'tag': None, 'wait_for_package': False} 2024-09-10T09:24:13.884 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1cf819b7212a6048e92fc78c4eff963344f6d56b 2024-09-10T09:24:13.885 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '1cf819b7212a6048e92fc78c4eff963344f6d56b', 'tag': None, 'wait_for_package': False} 2024-09-10T09:24:13.885 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1cf819b7212a6048e92fc78c4eff963344f6d56b 2024-09-10T09:24:14.030 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/centos/9/flavors/default/ 2024-09-10T09:24:14.031 INFO:teuthology.task.install.rpm:Package version is 19.1.1-305.g1cf819b7 2024-09-10T09:24:14.039 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/centos/9/flavors/default/ 2024-09-10T09:24:14.039 INFO:teuthology.task.install.rpm:Package version is 19.1.1-305.g1cf819b7 2024-09-10T09:24:14.052 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/centos/9/flavors/default/ 2024-09-10T09:24:14.052 INFO:teuthology.task.install.rpm:Package version is 19.1.1-305.g1cf819b7 2024-09-10T09:24:14.145 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-10T09:24:14.145 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-09-10T09:24:14.145 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-10T09:24:14.153 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-10T09:24:14.154 DEBUG:teuthology.orchestra.run.smithi159:> set -ex 2024-09-10T09:24:14.154 DEBUG:teuthology.orchestra.run.smithi159:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-10T09:24:14.161 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-10T09:24:14.161 DEBUG:teuthology.orchestra.run.smithi102:> set -ex 2024-09-10T09:24:14.161 DEBUG:teuthology.orchestra.run.smithi102:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-10T09:24:14.180 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-10T09:24:14.180 DEBUG:teuthology.orchestra.run.smithi159:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/1cf819b7212a6048e92fc78c4eff963344f6d56b/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-10T09:24:14.187 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-10T09:24:14.187 DEBUG:teuthology.orchestra.run.smithi102:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/1cf819b7212a6048e92fc78c4eff963344f6d56b/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-10T09:24:14.203 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-10T09:24:14.204 DEBUG:teuthology.orchestra.run.smithi005:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/1cf819b7212a6048e92fc78c4eff963344f6d56b/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-10T09:24:14.247 DEBUG:teuthology.orchestra.run.smithi159:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-10T09:24:14.254 DEBUG:teuthology.orchestra.run.smithi102:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-10T09:24:14.304 DEBUG:teuthology.orchestra.run.smithi005:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-10T09:24:14.321 DEBUG:teuthology.orchestra.run.smithi159:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-10T09:24:14.328 DEBUG:teuthology.orchestra.run.smithi102:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-10T09:24:14.383 DEBUG:teuthology.orchestra.run.smithi005:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-10T09:24:14.386 INFO:teuthology.orchestra.run.smithi159.stdout:check_obsoletes = 1 2024-09-10T09:24:14.388 DEBUG:teuthology.orchestra.run.smithi159:> sudo yum clean all 2024-09-10T09:24:14.393 INFO:teuthology.orchestra.run.smithi102.stdout:check_obsoletes = 1 2024-09-10T09:24:14.394 DEBUG:teuthology.orchestra.run.smithi102:> sudo yum clean all 2024-09-10T09:24:14.439 INFO:teuthology.orchestra.run.smithi005.stdout:check_obsoletes = 1 2024-09-10T09:24:14.442 DEBUG:teuthology.orchestra.run.smithi005:> sudo yum clean all 2024-09-10T09:24:14.615 INFO:teuthology.orchestra.run.smithi159.stdout:45 files removed 2024-09-10T09:24:14.618 INFO:teuthology.orchestra.run.smithi102.stdout:45 files removed 2024-09-10T09:24:14.642 DEBUG:teuthology.orchestra.run.smithi159:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-10T09:24:14.644 DEBUG:teuthology.orchestra.run.smithi102:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-10T09:24:14.719 INFO:teuthology.orchestra.run.smithi005.stdout:45 files removed 2024-09-10T09:24:14.753 DEBUG:teuthology.orchestra.run.smithi005:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-10T09:24:15.189 INFO:teuthology.orchestra.run.smithi102.stdout:ceph packages for x86_64 314 kB/s | 84 kB 00:00 2024-09-10T09:24:15.193 INFO:teuthology.orchestra.run.smithi159.stdout:ceph packages for x86_64 303 kB/s | 84 kB 00:00 2024-09-10T09:24:15.473 INFO:teuthology.orchestra.run.smithi102.stdout:ceph noarch packages 49 kB/s | 12 kB 00:00 2024-09-10T09:24:15.485 INFO:teuthology.orchestra.run.smithi159.stdout:ceph noarch packages 47 kB/s | 12 kB 00:00 2024-09-10T09:24:15.499 INFO:teuthology.orchestra.run.smithi005.stdout:ceph packages for x86_64 256 kB/s | 84 kB 00:00 2024-09-10T09:24:15.740 INFO:teuthology.orchestra.run.smithi102.stdout:ceph source packages 8.2 kB/s | 1.9 kB 00:00 2024-09-10T09:24:15.761 INFO:teuthology.orchestra.run.smithi159.stdout:ceph source packages 7.9 kB/s | 1.9 kB 00:00 2024-09-10T09:24:15.817 INFO:teuthology.orchestra.run.smithi005.stdout:ceph noarch packages 46 kB/s | 12 kB 00:00 2024-09-10T09:24:16.120 INFO:teuthology.orchestra.run.smithi005.stdout:ceph source packages 7.1 kB/s | 1.9 kB 00:00 2024-09-10T09:24:16.703 INFO:teuthology.orchestra.run.smithi005.stdout:CentOS Stream 9 - BaseOS 15 MB/s | 8.2 MB 00:00 2024-09-10T09:24:18.349 INFO:teuthology.orchestra.run.smithi102.stdout:CentOS Stream 9 - BaseOS 3.2 MB/s | 8.2 MB 00:02 2024-09-10T09:24:18.677 INFO:teuthology.orchestra.run.smithi159.stdout:CentOS Stream 9 - BaseOS 2.9 MB/s | 8.2 MB 00:02 2024-09-10T09:24:20.233 INFO:teuthology.orchestra.run.smithi102.stdout:CentOS Stream 9 - AppStream 26 MB/s | 20 MB 00:00 2024-09-10T09:24:25.917 INFO:teuthology.orchestra.run.smithi102.stdout:CentOS Stream 9 - CRB 3.4 MB/s | 6.5 MB 00:01 2024-09-10T09:24:25.978 INFO:teuthology.orchestra.run.smithi005.stdout:CentOS Stream 9 - AppStream 2.5 MB/s | 20 MB 00:08 2024-09-10T09:24:27.502 INFO:teuthology.orchestra.run.smithi159.stdout:CentOS Stream 9 - AppStream 2.6 MB/s | 20 MB 00:07 2024-09-10T09:24:27.543 INFO:teuthology.orchestra.run.smithi102.stdout:CentOS Stream 9 - Extras packages 45 kB/s | 19 kB 00:00 2024-09-10T09:24:28.622 INFO:teuthology.orchestra.run.smithi102.stdout:Extra Packages for Enterprise Linux 24 MB/s | 23 MB 00:00 2024-09-10T09:24:31.837 INFO:teuthology.orchestra.run.smithi159.stdout:CentOS Stream 9 - CRB 13 MB/s | 6.5 MB 00:00 2024-09-10T09:24:32.063 INFO:teuthology.orchestra.run.smithi005.stdout:CentOS Stream 9 - CRB 3.4 MB/s | 6.5 MB 00:01 2024-09-10T09:24:33.287 INFO:teuthology.orchestra.run.smithi159.stdout:CentOS Stream 9 - Extras packages 76 kB/s | 19 kB 00:00 2024-09-10T09:24:34.052 INFO:teuthology.orchestra.run.smithi102.stdout:lab-extras 29 kB/s | 1.7 kB 00:00 2024-09-10T09:24:34.417 INFO:teuthology.orchestra.run.smithi159.stdout:Extra Packages for Enterprise Linux 22 MB/s | 23 MB 00:01 2024-09-10T09:24:34.488 INFO:teuthology.orchestra.run.smithi005.stdout:CentOS Stream 9 - Extras packages 17 kB/s | 19 kB 00:01 2024-09-10T09:24:35.676 INFO:teuthology.orchestra.run.smithi005.stdout:Extra Packages for Enterprise Linux 22 MB/s | 23 MB 00:01 2024-09-10T09:24:36.313 INFO:teuthology.orchestra.run.smithi102.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-10T09:24:36.313 INFO:teuthology.orchestra.run.smithi102.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-10T09:24:36.325 INFO:teuthology.orchestra.run.smithi102.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-10T09:24:36.325 INFO:teuthology.orchestra.run.smithi102.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-10T09:24:36.326 INFO:teuthology.orchestra.run.smithi102.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-10T09:24:36.327 INFO:teuthology.orchestra.run.smithi102.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-10T09:24:36.329 INFO:teuthology.orchestra.run.smithi102.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-10T09:24:36.331 INFO:teuthology.orchestra.run.smithi102.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-10T09:24:36.332 INFO:teuthology.orchestra.run.smithi102.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-10T09:24:36.395 INFO:teuthology.orchestra.run.smithi102.stdout:Dependencies resolved. 2024-09-10T09:24:36.414 INFO:teuthology.orchestra.run.smithi102.stdout:====================================================================================== 2024-09-10T09:24:36.414 INFO:teuthology.orchestra.run.smithi102.stdout: Package Arch Version Repository Size 2024-09-10T09:24:36.414 INFO:teuthology.orchestra.run.smithi102.stdout:====================================================================================== 2024-09-10T09:24:36.414 INFO:teuthology.orchestra.run.smithi102.stdout:Installing: 2024-09-10T09:24:36.414 INFO:teuthology.orchestra.run.smithi102.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-10T09:24:36.414 INFO:teuthology.orchestra.run.smithi102.stdout: ceph x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 6.5 k 2024-09-10T09:24:36.414 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-base x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 5.4 M 2024-09-10T09:24:36.415 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-fuse x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 897 k 2024-09-10T09:24:36.415 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-immutable-object-cache x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 145 k 2024-09-10T09:24:36.415 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-mgr x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 1.5 M 2024-09-10T09:24:36.415 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-mgr-cephadm noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 146 k 2024-09-10T09:24:36.415 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-mgr-dashboard noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 3.6 M 2024-09-10T09:24:36.415 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-mgr-diskprediction-local noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 7.4 M 2024-09-10T09:24:36.415 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-mgr-rook noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 49 k 2024-09-10T09:24:36.415 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-radosgw x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 11 M 2024-09-10T09:24:36.415 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-test x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 49 M 2024-09-10T09:24:36.415 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-volume noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 285 k 2024-09-10T09:24:36.416 INFO:teuthology.orchestra.run.smithi102.stdout: cephadm noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 766 k 2024-09-10T09:24:36.416 INFO:teuthology.orchestra.run.smithi102.stdout: cephfs-mirror x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 228 k 2024-09-10T09:24:36.416 INFO:teuthology.orchestra.run.smithi102.stdout: cephfs-shell x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 27 k 2024-09-10T09:24:36.416 INFO:teuthology.orchestra.run.smithi102.stdout: cephfs-top noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 18 k 2024-09-10T09:24:36.416 INFO:teuthology.orchestra.run.smithi102.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-10T09:24:36.416 INFO:teuthology.orchestra.run.smithi102.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-10T09:24:36.416 INFO:teuthology.orchestra.run.smithi102.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-10T09:24:36.416 INFO:teuthology.orchestra.run.smithi102.stdout: libcephfs-devel x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 32 k 2024-09-10T09:24:36.417 INFO:teuthology.orchestra.run.smithi102.stdout: libcephfs2 x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 751 k 2024-09-10T09:24:36.417 INFO:teuthology.orchestra.run.smithi102.stdout: librados-devel x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 127 k 2024-09-10T09:24:36.417 INFO:teuthology.orchestra.run.smithi102.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-10T09:24:36.417 INFO:teuthology.orchestra.run.smithi102.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-10T09:24:36.417 INFO:teuthology.orchestra.run.smithi102.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-10T09:24:36.417 INFO:teuthology.orchestra.run.smithi102.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-10T09:24:36.417 INFO:teuthology.orchestra.run.smithi102.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-10T09:24:36.417 INFO:teuthology.orchestra.run.smithi102.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-10T09:24:36.418 INFO:teuthology.orchestra.run.smithi102.stdout: python3-cephfs x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 161 k 2024-09-10T09:24:36.418 INFO:teuthology.orchestra.run.smithi102.stdout: python3-rados x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 320 k 2024-09-10T09:24:36.418 INFO:teuthology.orchestra.run.smithi102.stdout: python3-rbd x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 301 k 2024-09-10T09:24:36.418 INFO:teuthology.orchestra.run.smithi102.stdout: python3-rgw x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 99 k 2024-09-10T09:24:36.418 INFO:teuthology.orchestra.run.smithi102.stdout: rbd-fuse x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 85 k 2024-09-10T09:24:36.418 INFO:teuthology.orchestra.run.smithi102.stdout: rbd-mirror x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 3.1 M 2024-09-10T09:24:36.418 INFO:teuthology.orchestra.run.smithi102.stdout: rbd-nbd x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 170 k 2024-09-10T09:24:36.418 INFO:teuthology.orchestra.run.smithi102.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-10T09:24:36.418 INFO:teuthology.orchestra.run.smithi102.stdout:Upgrading: 2024-09-10T09:24:36.419 INFO:teuthology.orchestra.run.smithi102.stdout: NetworkManager x86_64 1:1.49.90-1.el9 baseos 2.3 M 2024-09-10T09:24:36.419 INFO:teuthology.orchestra.run.smithi102.stdout: NetworkManager-libnm x86_64 1:1.49.90-1.el9 baseos 1.8 M 2024-09-10T09:24:36.419 INFO:teuthology.orchestra.run.smithi102.stdout: NetworkManager-team x86_64 1:1.49.90-1.el9 baseos 41 k 2024-09-10T09:24:36.419 INFO:teuthology.orchestra.run.smithi102.stdout: NetworkManager-tui x86_64 1:1.49.90-1.el9 baseos 249 k 2024-09-10T09:24:36.419 INFO:teuthology.orchestra.run.smithi102.stdout: librados2 x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 3.4 M 2024-09-10T09:24:36.419 INFO:teuthology.orchestra.run.smithi102.stdout: librbd1 x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 3.2 M 2024-09-10T09:24:36.419 INFO:teuthology.orchestra.run.smithi102.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-10T09:24:36.419 INFO:teuthology.orchestra.run.smithi102.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-10T09:24:36.419 INFO:teuthology.orchestra.run.smithi102.stdout:Installing dependencies: 2024-09-10T09:24:36.419 INFO:teuthology.orchestra.run.smithi102.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-10T09:24:36.420 INFO:teuthology.orchestra.run.smithi102.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-10T09:24:36.420 INFO:teuthology.orchestra.run.smithi102.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-10T09:24:36.420 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-common x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 21 M 2024-09-10T09:24:36.420 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-grafana-dashboards noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 32 k 2024-09-10T09:24:36.420 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-mds x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 2.4 M 2024-09-10T09:24:36.420 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-mgr-modules-core noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 249 k 2024-09-10T09:24:36.420 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-mon x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 4.7 M 2024-09-10T09:24:36.420 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-osd x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 17 M 2024-09-10T09:24:36.420 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-prometheus-alerts noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 16 k 2024-09-10T09:24:36.420 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-selinux x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 25 k 2024-09-10T09:24:36.420 INFO:teuthology.orchestra.run.smithi102.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-10T09:24:36.420 INFO:teuthology.orchestra.run.smithi102.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-10T09:24:36.420 INFO:teuthology.orchestra.run.smithi102.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-10T09:24:36.421 INFO:teuthology.orchestra.run.smithi102.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-10T09:24:36.421 INFO:teuthology.orchestra.run.smithi102.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-09-10T09:24:36.421 INFO:teuthology.orchestra.run.smithi102.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-10T09:24:36.421 INFO:teuthology.orchestra.run.smithi102.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-10T09:24:36.421 INFO:teuthology.orchestra.run.smithi102.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-10T09:24:36.421 INFO:teuthology.orchestra.run.smithi102.stdout: libcephsqlite x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 164 k 2024-09-10T09:24:36.421 INFO:teuthology.orchestra.run.smithi102.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-10T09:24:36.421 INFO:teuthology.orchestra.run.smithi102.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-10T09:24:36.421 INFO:teuthology.orchestra.run.smithi102.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-10T09:24:36.421 INFO:teuthology.orchestra.run.smithi102.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-10T09:24:36.421 INFO:teuthology.orchestra.run.smithi102.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-10T09:24:36.421 INFO:teuthology.orchestra.run.smithi102.stdout: libradosstriper1 x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 506 k 2024-09-10T09:24:36.422 INFO:teuthology.orchestra.run.smithi102.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-10T09:24:36.422 INFO:teuthology.orchestra.run.smithi102.stdout: librgw2 x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 5.3 M 2024-09-10T09:24:36.422 INFO:teuthology.orchestra.run.smithi102.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-10T09:24:36.422 INFO:teuthology.orchestra.run.smithi102.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-10T09:24:36.422 INFO:teuthology.orchestra.run.smithi102.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-10T09:24:36.422 INFO:teuthology.orchestra.run.smithi102.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-10T09:24:36.422 INFO:teuthology.orchestra.run.smithi102.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-10T09:24:36.422 INFO:teuthology.orchestra.run.smithi102.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-10T09:24:36.422 INFO:teuthology.orchestra.run.smithi102.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-10T09:24:36.422 INFO:teuthology.orchestra.run.smithi102.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-10T09:24:36.422 INFO:teuthology.orchestra.run.smithi102.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-10T09:24:36.422 INFO:teuthology.orchestra.run.smithi102.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-10T09:24:36.423 INFO:teuthology.orchestra.run.smithi102.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-10T09:24:36.423 INFO:teuthology.orchestra.run.smithi102.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-10T09:24:36.423 INFO:teuthology.orchestra.run.smithi102.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-10T09:24:36.423 INFO:teuthology.orchestra.run.smithi102.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-10T09:24:36.423 INFO:teuthology.orchestra.run.smithi102.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-10T09:24:36.423 INFO:teuthology.orchestra.run.smithi102.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-10T09:24:36.423 INFO:teuthology.orchestra.run.smithi102.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-10T09:24:36.423 INFO:teuthology.orchestra.run.smithi102.stdout: python3-ceph-argparse x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 45 k 2024-09-10T09:24:36.423 INFO:teuthology.orchestra.run.smithi102.stdout: python3-ceph-common x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 136 k 2024-09-10T09:24:36.423 INFO:teuthology.orchestra.run.smithi102.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-10T09:24:36.423 INFO:teuthology.orchestra.run.smithi102.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-10T09:24:36.423 INFO:teuthology.orchestra.run.smithi102.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-10T09:24:36.424 INFO:teuthology.orchestra.run.smithi102.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-10T09:24:36.424 INFO:teuthology.orchestra.run.smithi102.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-10T09:24:36.424 INFO:teuthology.orchestra.run.smithi102.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-10T09:24:36.424 INFO:teuthology.orchestra.run.smithi102.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-10T09:24:36.424 INFO:teuthology.orchestra.run.smithi102.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-10T09:24:36.424 INFO:teuthology.orchestra.run.smithi102.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-10T09:24:36.424 INFO:teuthology.orchestra.run.smithi102.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-10T09:24:36.424 INFO:teuthology.orchestra.run.smithi102.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-10T09:24:36.424 INFO:teuthology.orchestra.run.smithi102.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-10T09:24:36.424 INFO:teuthology.orchestra.run.smithi102.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-10T09:24:36.424 INFO:teuthology.orchestra.run.smithi102.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-10T09:24:36.424 INFO:teuthology.orchestra.run.smithi102.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-10T09:24:36.425 INFO:teuthology.orchestra.run.smithi102.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-10T09:24:36.425 INFO:teuthology.orchestra.run.smithi102.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-10T09:24:36.425 INFO:teuthology.orchestra.run.smithi102.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-10T09:24:36.425 INFO:teuthology.orchestra.run.smithi102.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-10T09:24:36.425 INFO:teuthology.orchestra.run.smithi102.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-10T09:24:36.425 INFO:teuthology.orchestra.run.smithi102.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-10T09:24:36.425 INFO:teuthology.orchestra.run.smithi102.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-10T09:24:36.425 INFO:teuthology.orchestra.run.smithi102.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-10T09:24:36.425 INFO:teuthology.orchestra.run.smithi102.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-10T09:24:36.425 INFO:teuthology.orchestra.run.smithi102.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-10T09:24:36.425 INFO:teuthology.orchestra.run.smithi102.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-10T09:24:36.425 INFO:teuthology.orchestra.run.smithi102.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-10T09:24:36.426 INFO:teuthology.orchestra.run.smithi102.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-10T09:24:36.426 INFO:teuthology.orchestra.run.smithi102.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-10T09:24:36.426 INFO:teuthology.orchestra.run.smithi102.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-10T09:24:36.426 INFO:teuthology.orchestra.run.smithi102.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-10T09:24:36.426 INFO:teuthology.orchestra.run.smithi102.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-10T09:24:36.426 INFO:teuthology.orchestra.run.smithi102.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-10T09:24:36.426 INFO:teuthology.orchestra.run.smithi102.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-10T09:24:36.426 INFO:teuthology.orchestra.run.smithi102.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-10T09:24:36.426 INFO:teuthology.orchestra.run.smithi102.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-10T09:24:36.426 INFO:teuthology.orchestra.run.smithi102.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-10T09:24:36.426 INFO:teuthology.orchestra.run.smithi102.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-10T09:24:36.426 INFO:teuthology.orchestra.run.smithi102.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-10T09:24:36.427 INFO:teuthology.orchestra.run.smithi102.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-10T09:24:36.427 INFO:teuthology.orchestra.run.smithi102.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-10T09:24:36.427 INFO:teuthology.orchestra.run.smithi102.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-10T09:24:36.427 INFO:teuthology.orchestra.run.smithi102.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-10T09:24:36.427 INFO:teuthology.orchestra.run.smithi102.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-10T09:24:36.427 INFO:teuthology.orchestra.run.smithi102.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-10T09:24:36.427 INFO:teuthology.orchestra.run.smithi102.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-10T09:24:36.427 INFO:teuthology.orchestra.run.smithi102.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-10T09:24:36.427 INFO:teuthology.orchestra.run.smithi102.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-10T09:24:36.427 INFO:teuthology.orchestra.run.smithi102.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-10T09:24:36.427 INFO:teuthology.orchestra.run.smithi102.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-10T09:24:36.427 INFO:teuthology.orchestra.run.smithi102.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-10T09:24:36.427 INFO:teuthology.orchestra.run.smithi102.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-10T09:24:36.427 INFO:teuthology.orchestra.run.smithi102.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-10T09:24:36.428 INFO:teuthology.orchestra.run.smithi102.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-10T09:24:36.428 INFO:teuthology.orchestra.run.smithi102.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-10T09:24:36.428 INFO:teuthology.orchestra.run.smithi102.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-10T09:24:36.428 INFO:teuthology.orchestra.run.smithi102.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-10T09:24:36.428 INFO:teuthology.orchestra.run.smithi102.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-10T09:24:36.428 INFO:teuthology.orchestra.run.smithi102.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-10T09:24:36.428 INFO:teuthology.orchestra.run.smithi102.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-10T09:24:36.428 INFO:teuthology.orchestra.run.smithi102.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-10T09:24:36.428 INFO:teuthology.orchestra.run.smithi102.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-10T09:24:36.428 INFO:teuthology.orchestra.run.smithi102.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-10T09:24:36.428 INFO:teuthology.orchestra.run.smithi102.stdout:Installing weak dependencies: 2024-09-10T09:24:36.428 INFO:teuthology.orchestra.run.smithi102.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-10T09:24:36.428 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-10T09:24:36.429 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction Summary 2024-09-10T09:24:36.429 INFO:teuthology.orchestra.run.smithi102.stdout:====================================================================================== 2024-09-10T09:24:36.429 INFO:teuthology.orchestra.run.smithi102.stdout:Install 148 Packages 2024-09-10T09:24:36.429 INFO:teuthology.orchestra.run.smithi102.stdout:Upgrade 8 Packages 2024-09-10T09:24:36.429 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-10T09:24:36.429 INFO:teuthology.orchestra.run.smithi102.stdout:Total download size: 226 M 2024-09-10T09:24:36.430 INFO:teuthology.orchestra.run.smithi102.stdout:Downloading Packages: 2024-09-10T09:24:37.029 INFO:teuthology.orchestra.run.smithi102.stdout:(1/156): ceph-19.1.1-305.g1cf819b7.el9.x86_64.r 49 kB/s | 6.5 kB 00:00 2024-09-10T09:24:37.213 INFO:teuthology.orchestra.run.smithi102.stdout:(2/156): ceph-fuse-19.1.1-305.g1cf819b7.el9.x86 4.8 MB/s | 897 kB 00:00 2024-09-10T09:24:37.296 INFO:teuthology.orchestra.run.smithi102.stdout:(3/156): ceph-immutable-object-cache-19.1.1-305 1.7 MB/s | 145 kB 00:00 2024-09-10T09:24:37.455 INFO:teuthology.orchestra.run.smithi102.stdout:(4/156): ceph-base-19.1.1-305.g1cf819b7.el9.x86 9.7 MB/s | 5.4 MB 00:00 2024-09-10T09:24:37.605 INFO:teuthology.orchestra.run.smithi102.stdout:(5/156): ceph-mds-19.1.1-305.g1cf819b7.el9.x86_ 7.7 MB/s | 2.4 MB 00:00 2024-09-10T09:24:37.639 INFO:teuthology.orchestra.run.smithi102.stdout:(6/156): ceph-mgr-19.1.1-305.g1cf819b7.el9.x86_ 8.3 MB/s | 1.5 MB 00:00 2024-09-10T09:24:37.931 INFO:teuthology.orchestra.run.smithi102.stdout:(7/156): ceph-common-19.1.1-305.g1cf819b7.el9.x 21 MB/s | 21 MB 00:01 2024-09-10T09:24:38.006 INFO:teuthology.orchestra.run.smithi102.stdout:(8/156): ceph-mon-19.1.1-305.g1cf819b7.el9.x86_ 12 MB/s | 4.7 MB 00:00 2024-09-10T09:24:38.073 INFO:teuthology.orchestra.run.smithi102.stdout:(9/156): ceph-selinux-19.1.1-305.g1cf819b7.el9. 378 kB/s | 25 kB 00:00 2024-09-10T09:24:38.481 INFO:teuthology.orchestra.run.smithi102.stdout:(10/156): ceph-radosgw-19.1.1-305.g1cf819b7.el9 19 MB/s | 11 MB 00:00 2024-09-10T09:24:38.657 INFO:teuthology.orchestra.run.smithi102.stdout:(11/156): ceph-osd-19.1.1-305.g1cf819b7.el9.x86 17 MB/s | 17 MB 00:01 2024-09-10T09:24:38.682 INFO:teuthology.orchestra.run.smithi102.stdout:(12/156): cephfs-mirror-19.1.1-305.g1cf819b7.el 1.1 MB/s | 228 kB 00:00 2024-09-10T09:24:38.708 INFO:teuthology.orchestra.run.smithi102.stdout:(13/156): cephfs-shell-19.1.1-305.g1cf819b7.el9 541 kB/s | 27 kB 00:00 2024-09-10T09:24:38.733 INFO:teuthology.orchestra.run.smithi102.stdout:(14/156): libcephfs-devel-19.1.1-305.g1cf819b7. 632 kB/s | 32 kB 00:00 2024-09-10T09:24:38.783 INFO:teuthology.orchestra.run.smithi102.stdout:(15/156): libcephfs2-19.1.1-305.g1cf819b7.el9.x 9.7 MB/s | 751 kB 00:00 2024-09-10T09:24:38.808 INFO:teuthology.orchestra.run.smithi102.stdout:(16/156): libcephsqlite-19.1.1-305.g1cf819b7.el 2.1 MB/s | 164 kB 00:00 2024-09-10T09:24:38.834 INFO:teuthology.orchestra.run.smithi102.stdout:(17/156): librados-devel-19.1.1-305.g1cf819b7.e 2.5 MB/s | 127 kB 00:00 2024-09-10T09:24:38.867 INFO:teuthology.orchestra.run.smithi102.stdout:(18/156): libradosstriper1-19.1.1-305.g1cf819b7 8.5 MB/s | 506 kB 00:00 2024-09-10T09:24:38.926 INFO:teuthology.orchestra.run.smithi102.stdout:(19/156): python3-ceph-argparse-19.1.1-305.g1cf 774 kB/s | 45 kB 00:00 2024-09-10T09:24:38.993 INFO:teuthology.orchestra.run.smithi102.stdout:(20/156): python3-ceph-common-19.1.1-305.g1cf81 2.0 MB/s | 136 kB 00:00 2024-09-10T09:24:39.085 INFO:teuthology.orchestra.run.smithi102.stdout:(21/156): librgw2-19.1.1-305.g1cf819b7.el9.x86_ 21 MB/s | 5.3 MB 00:00 2024-09-10T09:24:39.110 INFO:teuthology.orchestra.run.smithi102.stdout:(22/156): python3-cephfs-19.1.1-305.g1cf819b7.e 1.3 MB/s | 161 kB 00:00 2024-09-10T09:24:39.144 INFO:teuthology.orchestra.run.smithi102.stdout:(23/156): python3-rados-19.1.1-305.g1cf819b7.el 5.4 MB/s | 320 kB 00:00 2024-09-10T09:24:39.169 INFO:teuthology.orchestra.run.smithi102.stdout:(24/156): python3-rbd-19.1.1-305.g1cf819b7.el9. 5.0 MB/s | 301 kB 00:00 2024-09-10T09:24:39.194 INFO:teuthology.orchestra.run.smithi102.stdout:(25/156): python3-rgw-19.1.1-305.g1cf819b7.el9. 1.9 MB/s | 99 kB 00:00 2024-09-10T09:24:39.219 INFO:teuthology.orchestra.run.smithi102.stdout:(26/156): rbd-fuse-19.1.1-305.g1cf819b7.el9.x86 1.7 MB/s | 85 kB 00:00 2024-09-10T09:24:39.278 INFO:teuthology.orchestra.run.smithi102.stdout:(27/156): rbd-nbd-19.1.1-305.g1cf819b7.el9.x86_ 2.8 MB/s | 170 kB 00:00 2024-09-10T09:24:39.345 INFO:teuthology.orchestra.run.smithi102.stdout:(28/156): rbd-mirror-19.1.1-305.g1cf819b7.el9.x 21 MB/s | 3.1 MB 00:00 2024-09-10T09:24:39.370 INFO:teuthology.orchestra.run.smithi102.stdout:(29/156): ceph-grafana-dashboards-19.1.1-305.g1 347 kB/s | 32 kB 00:00 2024-09-10T09:24:39.396 INFO:teuthology.orchestra.run.smithi102.stdout:(30/156): ceph-mgr-cephadm-19.1.1-305.g1cf819b7 2.8 MB/s | 146 kB 00:00 2024-09-10T09:24:39.621 INFO:teuthology.orchestra.run.smithi102.stdout:(31/156): ceph-mgr-dashboard-19.1.1-305.g1cf819 14 MB/s | 3.6 MB 00:00 2024-09-10T09:24:39.680 INFO:teuthology.orchestra.run.smithi102.stdout:(32/156): ceph-mgr-modules-core-19.1.1-305.g1cf 4.2 MB/s | 249 kB 00:00 2024-09-10T09:24:39.771 INFO:teuthology.orchestra.run.smithi102.stdout:(33/156): ceph-mgr-diskprediction-local-19.1.1- 20 MB/s | 7.4 MB 00:00 2024-09-10T09:24:39.788 INFO:teuthology.orchestra.run.smithi102.stdout:(34/156): ceph-mgr-rook-19.1.1-305.g1cf819b7.el 455 kB/s | 49 kB 00:00 2024-09-10T09:24:39.822 INFO:teuthology.orchestra.run.smithi102.stdout:(35/156): ceph-prometheus-alerts-19.1.1-305.g1c 327 kB/s | 16 kB 00:00 2024-09-10T09:24:39.856 INFO:teuthology.orchestra.run.smithi102.stdout:(36/156): ceph-volume-19.1.1-305.g1cf819b7.el9. 4.2 MB/s | 285 kB 00:00 2024-09-10T09:24:39.889 INFO:teuthology.orchestra.run.smithi159.stdout:lab-extras 29 kB/s | 1.7 kB 00:00 2024-09-10T09:24:39.889 INFO:teuthology.orchestra.run.smithi102.stdout:(37/156): cephadm-19.1.1-305.g1cf819b7.el9.noar 11 MB/s | 766 kB 00:00 2024-09-10T09:24:39.914 INFO:teuthology.orchestra.run.smithi102.stdout:(38/156): cephfs-top-19.1.1-305.g1cf819b7.el9.n 312 kB/s | 18 kB 00:00 2024-09-10T09:24:40.732 INFO:teuthology.orchestra.run.smithi102.stdout:(39/156): ceph-test-19.1.1-305.g1cf819b7.el9.x8 18 MB/s | 49 MB 00:02 2024-09-10T09:24:40.757 INFO:teuthology.orchestra.run.smithi102.stdout:(40/156): fuse-2.9.9-16.el9.x86_64.rpm 92 kB/s | 80 kB 00:00 2024-09-10T09:24:41.465 INFO:teuthology.orchestra.run.smithi102.stdout:(41/156): libquadmath-11.5.0-2.el9.x86_64.rpm 261 kB/s | 191 kB 00:00 2024-09-10T09:24:41.507 INFO:teuthology.orchestra.run.smithi102.stdout:(42/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 337 kB/s | 253 kB 00:00 2024-09-10T09:24:41.891 INFO:teuthology.orchestra.run.smithi102.stdout:(43/156): libgfortran-11.5.0-2.el9.x86_64.rpm 405 kB/s | 801 kB 00:01 2024-09-10T09:24:42.191 INFO:teuthology.orchestra.run.smithi102.stdout:(44/156): python3-ply-3.11-14.el9.noarch.rpm 156 kB/s | 106 kB 00:00 2024-09-10T09:24:42.201 INFO:teuthology.orchestra.run.smithi159.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-10T09:24:42.202 INFO:teuthology.orchestra.run.smithi159.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-10T09:24:42.215 INFO:teuthology.orchestra.run.smithi159.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-10T09:24:42.216 INFO:teuthology.orchestra.run.smithi159.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-10T09:24:42.217 INFO:teuthology.orchestra.run.smithi159.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-10T09:24:42.218 INFO:teuthology.orchestra.run.smithi159.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-10T09:24:42.220 INFO:teuthology.orchestra.run.smithi159.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-10T09:24:42.223 INFO:teuthology.orchestra.run.smithi159.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-10T09:24:42.224 INFO:teuthology.orchestra.run.smithi159.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-10T09:24:42.288 INFO:teuthology.orchestra.run.smithi159.stdout:Dependencies resolved. 2024-09-10T09:24:42.304 INFO:teuthology.orchestra.run.smithi159.stdout:====================================================================================== 2024-09-10T09:24:42.304 INFO:teuthology.orchestra.run.smithi159.stdout: Package Arch Version Repository Size 2024-09-10T09:24:42.304 INFO:teuthology.orchestra.run.smithi159.stdout:====================================================================================== 2024-09-10T09:24:42.304 INFO:teuthology.orchestra.run.smithi159.stdout:Installing: 2024-09-10T09:24:42.304 INFO:teuthology.orchestra.run.smithi159.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-10T09:24:42.304 INFO:teuthology.orchestra.run.smithi159.stdout: ceph x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 6.5 k 2024-09-10T09:24:42.304 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-base x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 5.4 M 2024-09-10T09:24:42.304 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-fuse x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 897 k 2024-09-10T09:24:42.304 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-immutable-object-cache x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 145 k 2024-09-10T09:24:42.305 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-mgr x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 1.5 M 2024-09-10T09:24:42.305 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-mgr-cephadm noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 146 k 2024-09-10T09:24:42.305 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-mgr-dashboard noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 3.6 M 2024-09-10T09:24:42.305 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-mgr-diskprediction-local noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 7.4 M 2024-09-10T09:24:42.305 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-mgr-rook noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 49 k 2024-09-10T09:24:42.305 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-radosgw x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 11 M 2024-09-10T09:24:42.305 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-test x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 49 M 2024-09-10T09:24:42.305 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-volume noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 285 k 2024-09-10T09:24:42.305 INFO:teuthology.orchestra.run.smithi159.stdout: cephadm noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 766 k 2024-09-10T09:24:42.306 INFO:teuthology.orchestra.run.smithi159.stdout: cephfs-mirror x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 228 k 2024-09-10T09:24:42.306 INFO:teuthology.orchestra.run.smithi159.stdout: cephfs-shell x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 27 k 2024-09-10T09:24:42.306 INFO:teuthology.orchestra.run.smithi159.stdout: cephfs-top noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 18 k 2024-09-10T09:24:42.306 INFO:teuthology.orchestra.run.smithi159.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-10T09:24:42.306 INFO:teuthology.orchestra.run.smithi159.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-10T09:24:42.306 INFO:teuthology.orchestra.run.smithi159.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-10T09:24:42.306 INFO:teuthology.orchestra.run.smithi159.stdout: libcephfs-devel x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 32 k 2024-09-10T09:24:42.306 INFO:teuthology.orchestra.run.smithi159.stdout: libcephfs2 x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 751 k 2024-09-10T09:24:42.306 INFO:teuthology.orchestra.run.smithi159.stdout: librados-devel x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 127 k 2024-09-10T09:24:42.307 INFO:teuthology.orchestra.run.smithi159.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-10T09:24:42.307 INFO:teuthology.orchestra.run.smithi159.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-10T09:24:42.307 INFO:teuthology.orchestra.run.smithi159.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-10T09:24:42.307 INFO:teuthology.orchestra.run.smithi159.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-10T09:24:42.307 INFO:teuthology.orchestra.run.smithi159.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-10T09:24:42.307 INFO:teuthology.orchestra.run.smithi159.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-10T09:24:42.307 INFO:teuthology.orchestra.run.smithi159.stdout: python3-cephfs x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 161 k 2024-09-10T09:24:42.307 INFO:teuthology.orchestra.run.smithi159.stdout: python3-rados x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 320 k 2024-09-10T09:24:42.307 INFO:teuthology.orchestra.run.smithi159.stdout: python3-rbd x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 301 k 2024-09-10T09:24:42.307 INFO:teuthology.orchestra.run.smithi159.stdout: python3-rgw x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 99 k 2024-09-10T09:24:42.307 INFO:teuthology.orchestra.run.smithi159.stdout: rbd-fuse x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 85 k 2024-09-10T09:24:42.307 INFO:teuthology.orchestra.run.smithi159.stdout: rbd-mirror x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 3.1 M 2024-09-10T09:24:42.308 INFO:teuthology.orchestra.run.smithi159.stdout: rbd-nbd x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 170 k 2024-09-10T09:24:42.308 INFO:teuthology.orchestra.run.smithi159.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-10T09:24:42.308 INFO:teuthology.orchestra.run.smithi159.stdout:Upgrading: 2024-09-10T09:24:42.308 INFO:teuthology.orchestra.run.smithi159.stdout: NetworkManager x86_64 1:1.49.90-1.el9 baseos 2.3 M 2024-09-10T09:24:42.308 INFO:teuthology.orchestra.run.smithi159.stdout: NetworkManager-libnm x86_64 1:1.49.90-1.el9 baseos 1.8 M 2024-09-10T09:24:42.308 INFO:teuthology.orchestra.run.smithi159.stdout: NetworkManager-team x86_64 1:1.49.90-1.el9 baseos 41 k 2024-09-10T09:24:42.308 INFO:teuthology.orchestra.run.smithi159.stdout: NetworkManager-tui x86_64 1:1.49.90-1.el9 baseos 249 k 2024-09-10T09:24:42.309 INFO:teuthology.orchestra.run.smithi159.stdout: librados2 x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 3.4 M 2024-09-10T09:24:42.309 INFO:teuthology.orchestra.run.smithi159.stdout: librbd1 x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 3.2 M 2024-09-10T09:24:42.309 INFO:teuthology.orchestra.run.smithi159.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-10T09:24:42.309 INFO:teuthology.orchestra.run.smithi159.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-10T09:24:42.309 INFO:teuthology.orchestra.run.smithi159.stdout:Installing dependencies: 2024-09-10T09:24:42.309 INFO:teuthology.orchestra.run.smithi159.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-10T09:24:42.309 INFO:teuthology.orchestra.run.smithi159.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-10T09:24:42.309 INFO:teuthology.orchestra.run.smithi159.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-10T09:24:42.309 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-common x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 21 M 2024-09-10T09:24:42.309 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-grafana-dashboards noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 32 k 2024-09-10T09:24:42.310 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-mds x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 2.4 M 2024-09-10T09:24:42.310 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-mgr-modules-core noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 249 k 2024-09-10T09:24:42.310 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-mon x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 4.7 M 2024-09-10T09:24:42.310 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-osd x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 17 M 2024-09-10T09:24:42.310 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-prometheus-alerts noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 16 k 2024-09-10T09:24:42.310 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-selinux x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 25 k 2024-09-10T09:24:42.311 INFO:teuthology.orchestra.run.smithi159.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-10T09:24:42.311 INFO:teuthology.orchestra.run.smithi159.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-10T09:24:42.311 INFO:teuthology.orchestra.run.smithi159.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-10T09:24:42.311 INFO:teuthology.orchestra.run.smithi159.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-10T09:24:42.311 INFO:teuthology.orchestra.run.smithi159.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-09-10T09:24:42.311 INFO:teuthology.orchestra.run.smithi159.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-10T09:24:42.312 INFO:teuthology.orchestra.run.smithi159.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-10T09:24:42.312 INFO:teuthology.orchestra.run.smithi159.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-10T09:24:42.312 INFO:teuthology.orchestra.run.smithi159.stdout: libcephsqlite x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 164 k 2024-09-10T09:24:42.312 INFO:teuthology.orchestra.run.smithi159.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-10T09:24:42.312 INFO:teuthology.orchestra.run.smithi159.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-10T09:24:42.312 INFO:teuthology.orchestra.run.smithi159.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-10T09:24:42.312 INFO:teuthology.orchestra.run.smithi159.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-10T09:24:42.313 INFO:teuthology.orchestra.run.smithi159.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-10T09:24:42.313 INFO:teuthology.orchestra.run.smithi159.stdout: libradosstriper1 x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 506 k 2024-09-10T09:24:42.313 INFO:teuthology.orchestra.run.smithi159.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-10T09:24:42.313 INFO:teuthology.orchestra.run.smithi159.stdout: librgw2 x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 5.3 M 2024-09-10T09:24:42.313 INFO:teuthology.orchestra.run.smithi159.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-10T09:24:42.313 INFO:teuthology.orchestra.run.smithi159.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-10T09:24:42.313 INFO:teuthology.orchestra.run.smithi159.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-10T09:24:42.314 INFO:teuthology.orchestra.run.smithi159.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-10T09:24:42.314 INFO:teuthology.orchestra.run.smithi159.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-10T09:24:42.314 INFO:teuthology.orchestra.run.smithi159.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-10T09:24:42.314 INFO:teuthology.orchestra.run.smithi159.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-10T09:24:42.314 INFO:teuthology.orchestra.run.smithi159.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-10T09:24:42.314 INFO:teuthology.orchestra.run.smithi159.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-10T09:24:42.315 INFO:teuthology.orchestra.run.smithi159.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-10T09:24:42.315 INFO:teuthology.orchestra.run.smithi159.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-10T09:24:42.315 INFO:teuthology.orchestra.run.smithi159.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-10T09:24:42.315 INFO:teuthology.orchestra.run.smithi159.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-10T09:24:42.315 INFO:teuthology.orchestra.run.smithi159.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-10T09:24:42.315 INFO:teuthology.orchestra.run.smithi159.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-10T09:24:42.315 INFO:teuthology.orchestra.run.smithi159.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-10T09:24:42.315 INFO:teuthology.orchestra.run.smithi159.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-10T09:24:42.316 INFO:teuthology.orchestra.run.smithi159.stdout: python3-ceph-argparse x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 45 k 2024-09-10T09:24:42.316 INFO:teuthology.orchestra.run.smithi159.stdout: python3-ceph-common x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 136 k 2024-09-10T09:24:42.316 INFO:teuthology.orchestra.run.smithi159.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-10T09:24:42.316 INFO:teuthology.orchestra.run.smithi159.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-10T09:24:42.316 INFO:teuthology.orchestra.run.smithi159.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-10T09:24:42.316 INFO:teuthology.orchestra.run.smithi159.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-10T09:24:42.316 INFO:teuthology.orchestra.run.smithi159.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-10T09:24:42.316 INFO:teuthology.orchestra.run.smithi159.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-10T09:24:42.317 INFO:teuthology.orchestra.run.smithi159.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-10T09:24:42.317 INFO:teuthology.orchestra.run.smithi159.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-10T09:24:42.317 INFO:teuthology.orchestra.run.smithi159.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-10T09:24:42.317 INFO:teuthology.orchestra.run.smithi159.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-10T09:24:42.317 INFO:teuthology.orchestra.run.smithi159.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-10T09:24:42.317 INFO:teuthology.orchestra.run.smithi159.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-10T09:24:42.317 INFO:teuthology.orchestra.run.smithi159.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-10T09:24:42.317 INFO:teuthology.orchestra.run.smithi159.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-10T09:24:42.318 INFO:teuthology.orchestra.run.smithi159.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-10T09:24:42.318 INFO:teuthology.orchestra.run.smithi159.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-10T09:24:42.318 INFO:teuthology.orchestra.run.smithi159.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-10T09:24:42.318 INFO:teuthology.orchestra.run.smithi159.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-10T09:24:42.318 INFO:teuthology.orchestra.run.smithi159.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-10T09:24:42.318 INFO:teuthology.orchestra.run.smithi159.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-10T09:24:42.318 INFO:teuthology.orchestra.run.smithi159.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-10T09:24:42.319 INFO:teuthology.orchestra.run.smithi159.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-10T09:24:42.319 INFO:teuthology.orchestra.run.smithi159.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-10T09:24:42.319 INFO:teuthology.orchestra.run.smithi159.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-10T09:24:42.319 INFO:teuthology.orchestra.run.smithi159.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-10T09:24:42.319 INFO:teuthology.orchestra.run.smithi159.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-10T09:24:42.319 INFO:teuthology.orchestra.run.smithi159.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-10T09:24:42.319 INFO:teuthology.orchestra.run.smithi159.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-10T09:24:42.319 INFO:teuthology.orchestra.run.smithi159.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-10T09:24:42.319 INFO:teuthology.orchestra.run.smithi159.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-10T09:24:42.320 INFO:teuthology.orchestra.run.smithi159.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-10T09:24:42.320 INFO:teuthology.orchestra.run.smithi159.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-10T09:24:42.320 INFO:teuthology.orchestra.run.smithi159.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-10T09:24:42.320 INFO:teuthology.orchestra.run.smithi159.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-10T09:24:42.320 INFO:teuthology.orchestra.run.smithi159.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-10T09:24:42.320 INFO:teuthology.orchestra.run.smithi159.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-10T09:24:42.320 INFO:teuthology.orchestra.run.smithi159.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-10T09:24:42.320 INFO:teuthology.orchestra.run.smithi159.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-10T09:24:42.321 INFO:teuthology.orchestra.run.smithi159.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-10T09:24:42.321 INFO:teuthology.orchestra.run.smithi159.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-10T09:24:42.321 INFO:teuthology.orchestra.run.smithi159.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-10T09:24:42.321 INFO:teuthology.orchestra.run.smithi159.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-10T09:24:42.321 INFO:teuthology.orchestra.run.smithi159.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-10T09:24:42.322 INFO:teuthology.orchestra.run.smithi159.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-10T09:24:42.322 INFO:teuthology.orchestra.run.smithi159.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-10T09:24:42.322 INFO:teuthology.orchestra.run.smithi159.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-10T09:24:42.322 INFO:teuthology.orchestra.run.smithi159.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-10T09:24:42.322 INFO:teuthology.orchestra.run.smithi159.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-10T09:24:42.322 INFO:teuthology.orchestra.run.smithi159.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-10T09:24:42.322 INFO:teuthology.orchestra.run.smithi159.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-10T09:24:42.322 INFO:teuthology.orchestra.run.smithi159.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-10T09:24:42.323 INFO:teuthology.orchestra.run.smithi159.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-10T09:24:42.323 INFO:teuthology.orchestra.run.smithi159.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-10T09:24:42.323 INFO:teuthology.orchestra.run.smithi159.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-10T09:24:42.323 INFO:teuthology.orchestra.run.smithi159.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-10T09:24:42.323 INFO:teuthology.orchestra.run.smithi159.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-10T09:24:42.323 INFO:teuthology.orchestra.run.smithi159.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-10T09:24:42.323 INFO:teuthology.orchestra.run.smithi159.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-10T09:24:42.323 INFO:teuthology.orchestra.run.smithi159.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-10T09:24:42.323 INFO:teuthology.orchestra.run.smithi159.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-10T09:24:42.324 INFO:teuthology.orchestra.run.smithi159.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-10T09:24:42.324 INFO:teuthology.orchestra.run.smithi159.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-10T09:24:42.324 INFO:teuthology.orchestra.run.smithi159.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-10T09:24:42.324 INFO:teuthology.orchestra.run.smithi159.stdout:Installing weak dependencies: 2024-09-10T09:24:42.324 INFO:teuthology.orchestra.run.smithi159.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-10T09:24:42.324 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-10T09:24:42.324 INFO:teuthology.orchestra.run.smithi159.stdout:Transaction Summary 2024-09-10T09:24:42.324 INFO:teuthology.orchestra.run.smithi159.stdout:====================================================================================== 2024-09-10T09:24:42.325 INFO:teuthology.orchestra.run.smithi159.stdout:Install 148 Packages 2024-09-10T09:24:42.325 INFO:teuthology.orchestra.run.smithi159.stdout:Upgrade 8 Packages 2024-09-10T09:24:42.325 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-10T09:24:42.326 INFO:teuthology.orchestra.run.smithi159.stdout:Total download size: 226 M 2024-09-10T09:24:42.326 INFO:teuthology.orchestra.run.smithi159.stdout:Downloading Packages: 2024-09-10T09:24:42.382 INFO:teuthology.orchestra.run.smithi005.stdout:lab-extras 31 kB/s | 1.7 kB 00:00 2024-09-10T09:24:42.392 INFO:teuthology.orchestra.run.smithi102.stdout:(45/156): python3-cryptography-36.0.1-4.el9.x86 1.3 MB/s | 1.2 MB 00:00 2024-09-10T09:24:42.667 INFO:teuthology.orchestra.run.smithi102.stdout:(46/156): python3-pycparser-2.20-6.el9.noarch.r 174 kB/s | 135 kB 00:00 2024-09-10T09:24:42.867 INFO:teuthology.orchestra.run.smithi102.stdout:(47/156): python3-requests-2.25.1-8.el9.noarch. 185 kB/s | 125 kB 00:00 2024-09-10T09:24:42.951 INFO:teuthology.orchestra.run.smithi102.stdout:(48/156): automake-1.16.2-8.el9.noarch.rpm 2.4 MB/s | 689 kB 00:00 2024-09-10T09:24:43.001 INFO:teuthology.orchestra.run.smithi102.stdout:(49/156): boost-program-options-1.75.0-8.el9.x8 2.1 MB/s | 107 kB 00:00 2024-09-10T09:24:43.051 INFO:teuthology.orchestra.run.smithi102.stdout:(50/156): elfutils-libelf-devel-0.191-4.el9.x86 435 kB/s | 22 kB 00:00 2024-09-10T09:24:43.126 INFO:teuthology.orchestra.run.smithi102.stdout:(51/156): python3-urllib3-1.26.5-6.el9.noarch.r 292 kB/s | 215 kB 00:00 2024-09-10T09:24:43.176 INFO:teuthology.orchestra.run.smithi102.stdout:(52/156): flex-2.6.4-9.el9.x86_64.rpm 2.4 MB/s | 313 kB 00:00 2024-09-10T09:24:43.235 INFO:teuthology.orchestra.run.smithi102.stdout:(53/156): flexiblas-3.0.4-8.el9.x86_64.rpm 302 kB/s | 33 kB 00:00 2024-09-10T09:24:43.269 INFO:teuthology.orchestra.run.smithi102.stdout:(54/156): bison-3.7.4-5.el9.x86_64.rpm 2.3 MB/s | 943 kB 00:00 2024-09-10T09:24:43.294 INFO:teuthology.orchestra.run.smithi102.stdout:(55/156): flexiblas-openblas-openmp-3.0.4-8.el9 311 kB/s | 18 kB 00:00 2024-09-10T09:24:43.319 INFO:teuthology.orchestra.run.smithi102.stdout:(56/156): libaio-devel-0.3.111-13.el9.x86_64.rp 222 kB/s | 11 kB 00:00 2024-09-10T09:24:43.369 INFO:teuthology.orchestra.run.smithi102.stdout:(57/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 910 kB/s | 45 kB 00:00 2024-09-10T09:24:43.395 INFO:teuthology.orchestra.run.smithi102.stdout:(58/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.6 MB/s | 160 kB 00:00 2024-09-10T09:24:43.570 INFO:teuthology.orchestra.run.smithi102.stdout:(59/156): librdkafka-1.6.1-102.el9.x86_64.rpm 3.2 MB/s | 662 kB 00:00 2024-09-10T09:24:43.595 INFO:teuthology.orchestra.run.smithi102.stdout:(60/156): libtool-2.4.6-46.el9.x86_64.rpm 2.8 MB/s | 581 kB 00:00 2024-09-10T09:24:43.629 INFO:teuthology.orchestra.run.smithi102.stdout:(61/156): libuuid-devel-2.37.4-20.el9.x86_64.rp 417 kB/s | 24 kB 00:00 2024-09-10T09:24:43.654 INFO:teuthology.orchestra.run.smithi102.stdout:(62/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 810 kB/s | 47 kB 00:00 2024-09-10T09:24:43.657 INFO:teuthology.orchestra.run.smithi159.stdout:(1/156): ceph-19.1.1-305.g1cf819b7.el9.x86_64.r 17 kB/s | 6.5 kB 00:00 2024-09-10T09:24:43.746 INFO:teuthology.orchestra.run.smithi102.stdout:(63/156): lua-5.4.4-4.el9.x86_64.rpm 2.0 MB/s | 188 kB 00:00 2024-09-10T09:24:43.772 INFO:teuthology.orchestra.run.smithi102.stdout:(64/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.0 MB/s | 292 kB 00:00 2024-09-10T09:24:43.797 INFO:teuthology.orchestra.run.smithi102.stdout:(65/156): openblas-0.3.26-2.el9.x86_64.rpm 767 kB/s | 38 kB 00:00 2024-09-10T09:24:43.956 INFO:teuthology.orchestra.run.smithi102.stdout:(66/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 3.8 MB/s | 3.0 MB 00:00 2024-09-10T09:24:43.999 INFO:teuthology.orchestra.run.smithi159.stdout:(2/156): ceph-fuse-19.1.1-305.g1cf819b7.el9.x86 2.6 MB/s | 897 kB 00:00 2024-09-10T09:24:44.074 INFO:teuthology.orchestra.run.smithi159.stdout:(3/156): ceph-base-19.1.1-305.g1cf819b7.el9.x86 6.7 MB/s | 5.4 MB 00:00 2024-09-10T09:24:44.099 INFO:teuthology.orchestra.run.smithi159.stdout:(4/156): ceph-immutable-object-cache-19.1.1-305 1.4 MB/s | 145 kB 00:00 2024-09-10T09:24:44.306 INFO:teuthology.orchestra.run.smithi102.stdout:(67/156): postgresql-13.16-1.el9.x86_64.rpm 4.5 MB/s | 1.6 MB 00:00 2024-09-10T09:24:44.408 INFO:teuthology.orchestra.run.smithi159.stdout:(5/156): ceph-mgr-19.1.1-305.g1cf819b7.el9.x86_ 4.9 MB/s | 1.5 MB 00:00 2024-09-10T09:24:44.506 INFO:teuthology.orchestra.run.smithi102.stdout:(68/156): postgresql-contrib-13.16-1.el9.x86_64 4.4 MB/s | 910 kB 00:00 2024-09-10T09:24:44.565 INFO:teuthology.orchestra.run.smithi102.stdout:(69/156): postgresql-private-libs-13.16-1.el9.x 2.3 MB/s | 137 kB 00:00 2024-09-10T09:24:44.790 INFO:teuthology.orchestra.run.smithi102.stdout:(70/156): openssl-devel-3.2.2-6.el9.x86_64.rpm 4.5 MB/s | 4.4 MB 00:00 2024-09-10T09:24:44.791 INFO:teuthology.orchestra.run.smithi159.stdout:(6/156): ceph-common-19.1.1-305.g1cf819b7.el9.x 14 MB/s | 21 MB 00:01 2024-09-10T09:24:44.842 INFO:teuthology.orchestra.run.smithi159.stdout:(7/156): ceph-mds-19.1.1-305.g1cf819b7.el9.x86_ 3.1 MB/s | 2.4 MB 00:00 2024-09-10T09:24:44.882 INFO:teuthology.orchestra.run.smithi102.stdout:(71/156): openblas-openmp-0.3.26-2.el9.x86_64.r 4.5 MB/s | 4.9 MB 00:01 2024-09-10T09:24:44.932 INFO:teuthology.orchestra.run.smithi102.stdout:(72/156): python3-attrs-20.3.0-7.el9.noarch.rpm 1.7 MB/s | 87 kB 00:00 2024-09-10T09:24:44.987 INFO:teuthology.orchestra.run.smithi005.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-10T09:24:44.987 INFO:teuthology.orchestra.run.smithi005.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-10T09:24:44.999 INFO:teuthology.orchestra.run.smithi005.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-10T09:24:45.000 INFO:teuthology.orchestra.run.smithi005.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-10T09:24:45.001 INFO:teuthology.orchestra.run.smithi005.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-10T09:24:45.002 INFO:teuthology.orchestra.run.smithi005.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-10T09:24:45.004 INFO:teuthology.orchestra.run.smithi005.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-10T09:24:45.007 INFO:teuthology.orchestra.run.smithi005.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-10T09:24:45.007 INFO:teuthology.orchestra.run.smithi005.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-10T09:24:45.074 INFO:teuthology.orchestra.run.smithi102.stdout:(73/156): protobuf-3.14.0-13.el9.x86_64.rpm 3.6 MB/s | 1.0 MB 00:00 2024-09-10T09:24:45.077 INFO:teuthology.orchestra.run.smithi005.stdout:Dependencies resolved. 2024-09-10T09:24:45.100 INFO:teuthology.orchestra.run.smithi159.stdout:(8/156): ceph-mon-19.1.1-305.g1cf819b7.el9.x86_ 6.8 MB/s | 4.7 MB 00:00 2024-09-10T09:24:45.105 INFO:teuthology.orchestra.run.smithi005.stdout:====================================================================================== 2024-09-10T09:24:45.105 INFO:teuthology.orchestra.run.smithi005.stdout: Package Arch Version Repository Size 2024-09-10T09:24:45.105 INFO:teuthology.orchestra.run.smithi005.stdout:====================================================================================== 2024-09-10T09:24:45.105 INFO:teuthology.orchestra.run.smithi005.stdout:Installing: 2024-09-10T09:24:45.105 INFO:teuthology.orchestra.run.smithi005.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-10T09:24:45.105 INFO:teuthology.orchestra.run.smithi005.stdout: ceph x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 6.5 k 2024-09-10T09:24:45.105 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-base x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 5.4 M 2024-09-10T09:24:45.106 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-fuse x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 897 k 2024-09-10T09:24:45.106 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-immutable-object-cache x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 145 k 2024-09-10T09:24:45.106 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-mgr x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 1.5 M 2024-09-10T09:24:45.106 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-mgr-cephadm noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 146 k 2024-09-10T09:24:45.106 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-mgr-dashboard noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 3.6 M 2024-09-10T09:24:45.106 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-mgr-diskprediction-local noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 7.4 M 2024-09-10T09:24:45.106 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-mgr-rook noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 49 k 2024-09-10T09:24:45.106 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-radosgw x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 11 M 2024-09-10T09:24:45.106 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-test x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 49 M 2024-09-10T09:24:45.106 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-volume noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 285 k 2024-09-10T09:24:45.107 INFO:teuthology.orchestra.run.smithi005.stdout: cephadm noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 766 k 2024-09-10T09:24:45.107 INFO:teuthology.orchestra.run.smithi005.stdout: cephfs-mirror x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 228 k 2024-09-10T09:24:45.107 INFO:teuthology.orchestra.run.smithi005.stdout: cephfs-shell x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 27 k 2024-09-10T09:24:45.107 INFO:teuthology.orchestra.run.smithi005.stdout: cephfs-top noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 18 k 2024-09-10T09:24:45.107 INFO:teuthology.orchestra.run.smithi005.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-10T09:24:45.107 INFO:teuthology.orchestra.run.smithi005.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-10T09:24:45.107 INFO:teuthology.orchestra.run.smithi005.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-10T09:24:45.107 INFO:teuthology.orchestra.run.smithi005.stdout: libcephfs-devel x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 32 k 2024-09-10T09:24:45.107 INFO:teuthology.orchestra.run.smithi005.stdout: libcephfs2 x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 751 k 2024-09-10T09:24:45.107 INFO:teuthology.orchestra.run.smithi005.stdout: librados-devel x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 127 k 2024-09-10T09:24:45.107 INFO:teuthology.orchestra.run.smithi005.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-10T09:24:45.107 INFO:teuthology.orchestra.run.smithi005.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-10T09:24:45.108 INFO:teuthology.orchestra.run.smithi005.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-10T09:24:45.108 INFO:teuthology.orchestra.run.smithi005.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-10T09:24:45.108 INFO:teuthology.orchestra.run.smithi005.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-10T09:24:45.108 INFO:teuthology.orchestra.run.smithi005.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-10T09:24:45.108 INFO:teuthology.orchestra.run.smithi005.stdout: python3-cephfs x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 161 k 2024-09-10T09:24:45.108 INFO:teuthology.orchestra.run.smithi005.stdout: python3-rados x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 320 k 2024-09-10T09:24:45.108 INFO:teuthology.orchestra.run.smithi005.stdout: python3-rbd x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 301 k 2024-09-10T09:24:45.108 INFO:teuthology.orchestra.run.smithi005.stdout: python3-rgw x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 99 k 2024-09-10T09:24:45.108 INFO:teuthology.orchestra.run.smithi005.stdout: rbd-fuse x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 85 k 2024-09-10T09:24:45.108 INFO:teuthology.orchestra.run.smithi005.stdout: rbd-mirror x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 3.1 M 2024-09-10T09:24:45.108 INFO:teuthology.orchestra.run.smithi005.stdout: rbd-nbd x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 170 k 2024-09-10T09:24:45.109 INFO:teuthology.orchestra.run.smithi005.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-10T09:24:45.109 INFO:teuthology.orchestra.run.smithi005.stdout:Upgrading: 2024-09-10T09:24:45.109 INFO:teuthology.orchestra.run.smithi005.stdout: NetworkManager x86_64 1:1.49.90-1.el9 baseos 2.3 M 2024-09-10T09:24:45.109 INFO:teuthology.orchestra.run.smithi005.stdout: NetworkManager-libnm x86_64 1:1.49.90-1.el9 baseos 1.8 M 2024-09-10T09:24:45.109 INFO:teuthology.orchestra.run.smithi005.stdout: NetworkManager-team x86_64 1:1.49.90-1.el9 baseos 41 k 2024-09-10T09:24:45.109 INFO:teuthology.orchestra.run.smithi005.stdout: NetworkManager-tui x86_64 1:1.49.90-1.el9 baseos 249 k 2024-09-10T09:24:45.109 INFO:teuthology.orchestra.run.smithi005.stdout: librados2 x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 3.4 M 2024-09-10T09:24:45.109 INFO:teuthology.orchestra.run.smithi005.stdout: librbd1 x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 3.2 M 2024-09-10T09:24:45.109 INFO:teuthology.orchestra.run.smithi005.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-10T09:24:45.109 INFO:teuthology.orchestra.run.smithi005.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-10T09:24:45.109 INFO:teuthology.orchestra.run.smithi005.stdout:Installing dependencies: 2024-09-10T09:24:45.109 INFO:teuthology.orchestra.run.smithi005.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-10T09:24:45.109 INFO:teuthology.orchestra.run.smithi005.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-10T09:24:45.110 INFO:teuthology.orchestra.run.smithi005.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-10T09:24:45.110 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-common x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 21 M 2024-09-10T09:24:45.110 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-grafana-dashboards noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 32 k 2024-09-10T09:24:45.110 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-mds x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 2.4 M 2024-09-10T09:24:45.110 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-mgr-modules-core noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 249 k 2024-09-10T09:24:45.110 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-mon x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 4.7 M 2024-09-10T09:24:45.110 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-osd x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 17 M 2024-09-10T09:24:45.110 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-prometheus-alerts noarch 2:19.1.1-305.g1cf819b7.el9 ceph-noarch 16 k 2024-09-10T09:24:45.110 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-selinux x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 25 k 2024-09-10T09:24:45.110 INFO:teuthology.orchestra.run.smithi005.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-10T09:24:45.110 INFO:teuthology.orchestra.run.smithi005.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-10T09:24:45.110 INFO:teuthology.orchestra.run.smithi005.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-10T09:24:45.111 INFO:teuthology.orchestra.run.smithi005.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-10T09:24:45.111 INFO:teuthology.orchestra.run.smithi005.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-09-10T09:24:45.111 INFO:teuthology.orchestra.run.smithi005.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-10T09:24:45.111 INFO:teuthology.orchestra.run.smithi005.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-10T09:24:45.111 INFO:teuthology.orchestra.run.smithi005.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-10T09:24:45.111 INFO:teuthology.orchestra.run.smithi005.stdout: libcephsqlite x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 164 k 2024-09-10T09:24:45.111 INFO:teuthology.orchestra.run.smithi005.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-10T09:24:45.111 INFO:teuthology.orchestra.run.smithi005.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-10T09:24:45.111 INFO:teuthology.orchestra.run.smithi005.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-10T09:24:45.111 INFO:teuthology.orchestra.run.smithi005.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-10T09:24:45.111 INFO:teuthology.orchestra.run.smithi005.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-10T09:24:45.111 INFO:teuthology.orchestra.run.smithi005.stdout: libradosstriper1 x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 506 k 2024-09-10T09:24:45.111 INFO:teuthology.orchestra.run.smithi005.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-10T09:24:45.112 INFO:teuthology.orchestra.run.smithi005.stdout: librgw2 x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 5.3 M 2024-09-10T09:24:45.112 INFO:teuthology.orchestra.run.smithi005.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-10T09:24:45.112 INFO:teuthology.orchestra.run.smithi005.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-10T09:24:45.112 INFO:teuthology.orchestra.run.smithi005.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-10T09:24:45.112 INFO:teuthology.orchestra.run.smithi005.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-10T09:24:45.112 INFO:teuthology.orchestra.run.smithi005.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-10T09:24:45.112 INFO:teuthology.orchestra.run.smithi005.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-10T09:24:45.112 INFO:teuthology.orchestra.run.smithi005.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-10T09:24:45.112 INFO:teuthology.orchestra.run.smithi005.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-10T09:24:45.112 INFO:teuthology.orchestra.run.smithi005.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-10T09:24:45.112 INFO:teuthology.orchestra.run.smithi005.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-10T09:24:45.112 INFO:teuthology.orchestra.run.smithi005.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-10T09:24:45.113 INFO:teuthology.orchestra.run.smithi005.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-10T09:24:45.113 INFO:teuthology.orchestra.run.smithi005.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-10T09:24:45.113 INFO:teuthology.orchestra.run.smithi005.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-10T09:24:45.113 INFO:teuthology.orchestra.run.smithi005.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-10T09:24:45.113 INFO:teuthology.orchestra.run.smithi005.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-10T09:24:45.113 INFO:teuthology.orchestra.run.smithi005.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-10T09:24:45.113 INFO:teuthology.orchestra.run.smithi005.stdout: python3-ceph-argparse x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 45 k 2024-09-10T09:24:45.113 INFO:teuthology.orchestra.run.smithi005.stdout: python3-ceph-common x86_64 2:19.1.1-305.g1cf819b7.el9 ceph 136 k 2024-09-10T09:24:45.113 INFO:teuthology.orchestra.run.smithi005.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-10T09:24:45.113 INFO:teuthology.orchestra.run.smithi005.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-10T09:24:45.113 INFO:teuthology.orchestra.run.smithi005.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-10T09:24:45.113 INFO:teuthology.orchestra.run.smithi005.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-10T09:24:45.114 INFO:teuthology.orchestra.run.smithi005.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-10T09:24:45.114 INFO:teuthology.orchestra.run.smithi005.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-10T09:24:45.114 INFO:teuthology.orchestra.run.smithi005.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-10T09:24:45.114 INFO:teuthology.orchestra.run.smithi005.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-10T09:24:45.114 INFO:teuthology.orchestra.run.smithi005.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-10T09:24:45.114 INFO:teuthology.orchestra.run.smithi005.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-10T09:24:45.114 INFO:teuthology.orchestra.run.smithi005.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-10T09:24:45.114 INFO:teuthology.orchestra.run.smithi005.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-10T09:24:45.114 INFO:teuthology.orchestra.run.smithi005.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-10T09:24:45.114 INFO:teuthology.orchestra.run.smithi005.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-10T09:24:45.114 INFO:teuthology.orchestra.run.smithi005.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-10T09:24:45.114 INFO:teuthology.orchestra.run.smithi005.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-10T09:24:45.114 INFO:teuthology.orchestra.run.smithi005.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-10T09:24:45.115 INFO:teuthology.orchestra.run.smithi005.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-10T09:24:45.115 INFO:teuthology.orchestra.run.smithi005.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-10T09:24:45.115 INFO:teuthology.orchestra.run.smithi005.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-10T09:24:45.115 INFO:teuthology.orchestra.run.smithi005.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-10T09:24:45.115 INFO:teuthology.orchestra.run.smithi005.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-10T09:24:45.115 INFO:teuthology.orchestra.run.smithi005.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-10T09:24:45.115 INFO:teuthology.orchestra.run.smithi005.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-10T09:24:45.115 INFO:teuthology.orchestra.run.smithi005.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-10T09:24:45.115 INFO:teuthology.orchestra.run.smithi005.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-10T09:24:45.115 INFO:teuthology.orchestra.run.smithi005.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-10T09:24:45.115 INFO:teuthology.orchestra.run.smithi005.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-10T09:24:45.115 INFO:teuthology.orchestra.run.smithi005.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-10T09:24:45.115 INFO:teuthology.orchestra.run.smithi005.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-10T09:24:45.116 INFO:teuthology.orchestra.run.smithi005.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-10T09:24:45.116 INFO:teuthology.orchestra.run.smithi005.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-10T09:24:45.116 INFO:teuthology.orchestra.run.smithi005.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-10T09:24:45.116 INFO:teuthology.orchestra.run.smithi005.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-10T09:24:45.116 INFO:teuthology.orchestra.run.smithi005.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-10T09:24:45.116 INFO:teuthology.orchestra.run.smithi005.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-10T09:24:45.116 INFO:teuthology.orchestra.run.smithi005.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-10T09:24:45.116 INFO:teuthology.orchestra.run.smithi005.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-10T09:24:45.116 INFO:teuthology.orchestra.run.smithi005.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-10T09:24:45.116 INFO:teuthology.orchestra.run.smithi005.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-10T09:24:45.116 INFO:teuthology.orchestra.run.smithi005.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-10T09:24:45.116 INFO:teuthology.orchestra.run.smithi005.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-10T09:24:45.117 INFO:teuthology.orchestra.run.smithi005.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-10T09:24:45.117 INFO:teuthology.orchestra.run.smithi005.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-10T09:24:45.117 INFO:teuthology.orchestra.run.smithi005.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-10T09:24:45.117 INFO:teuthology.orchestra.run.smithi005.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-10T09:24:45.117 INFO:teuthology.orchestra.run.smithi005.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-10T09:24:45.117 INFO:teuthology.orchestra.run.smithi005.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-10T09:24:45.117 INFO:teuthology.orchestra.run.smithi005.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-10T09:24:45.117 INFO:teuthology.orchestra.run.smithi005.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-10T09:24:45.117 INFO:teuthology.orchestra.run.smithi005.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-10T09:24:45.117 INFO:teuthology.orchestra.run.smithi005.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-10T09:24:45.117 INFO:teuthology.orchestra.run.smithi005.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-10T09:24:45.117 INFO:teuthology.orchestra.run.smithi005.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-10T09:24:45.117 INFO:teuthology.orchestra.run.smithi005.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-10T09:24:45.118 INFO:teuthology.orchestra.run.smithi005.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-10T09:24:45.118 INFO:teuthology.orchestra.run.smithi005.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-10T09:24:45.118 INFO:teuthology.orchestra.run.smithi005.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-10T09:24:45.118 INFO:teuthology.orchestra.run.smithi005.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-10T09:24:45.118 INFO:teuthology.orchestra.run.smithi005.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-10T09:24:45.118 INFO:teuthology.orchestra.run.smithi005.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-10T09:24:45.118 INFO:teuthology.orchestra.run.smithi005.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-10T09:24:45.118 INFO:teuthology.orchestra.run.smithi005.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-10T09:24:45.118 INFO:teuthology.orchestra.run.smithi005.stdout:Installing weak dependencies: 2024-09-10T09:24:45.118 INFO:teuthology.orchestra.run.smithi005.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-10T09:24:45.118 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-10T09:24:45.118 INFO:teuthology.orchestra.run.smithi005.stdout:Transaction Summary 2024-09-10T09:24:45.118 INFO:teuthology.orchestra.run.smithi005.stdout:====================================================================================== 2024-09-10T09:24:45.119 INFO:teuthology.orchestra.run.smithi005.stdout:Install 148 Packages 2024-09-10T09:24:45.119 INFO:teuthology.orchestra.run.smithi005.stdout:Upgrade 8 Packages 2024-09-10T09:24:45.119 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-10T09:24:45.119 INFO:teuthology.orchestra.run.smithi005.stdout:Total download size: 226 M 2024-09-10T09:24:45.119 INFO:teuthology.orchestra.run.smithi005.stdout:Downloading Packages: 2024-09-10T09:24:45.133 INFO:teuthology.orchestra.run.smithi102.stdout:(74/156): python3-colorama-0.4.6-3.el9.noarch.r 1.0 MB/s | 61 kB 00:00 2024-09-10T09:24:45.193 INFO:teuthology.orchestra.run.smithi159.stdout:(9/156): ceph-selinux-19.1.1-305.g1cf819b7.el9. 274 kB/s | 25 kB 00:00 2024-09-10T09:24:45.218 INFO:teuthology.orchestra.run.smithi102.stdout:(75/156): python3-devel-3.9.19-8.el9.x86_64.rpm 2.8 MB/s | 245 kB 00:00 2024-09-10T09:24:45.301 INFO:teuthology.orchestra.run.smithi102.stdout:(76/156): python3-jinja2-2.11.3-6.el9.noarch.rp 2.9 MB/s | 248 kB 00:00 2024-09-10T09:24:45.352 INFO:teuthology.orchestra.run.smithi102.stdout:(77/156): python3-jsonpatch-1.21-16.el9.noarch. 514 kB/s | 26 kB 00:00 2024-09-10T09:24:45.410 INFO:teuthology.orchestra.run.smithi102.stdout:(78/156): python3-jsonpointer-2.0-4.el9.noarch. 328 kB/s | 19 kB 00:00 2024-09-10T09:24:45.469 INFO:teuthology.orchestra.run.smithi102.stdout:(79/156): python3-mako-1.1.4-6.el9.noarch.rpm 2.9 MB/s | 172 kB 00:00 2024-09-10T09:24:45.527 INFO:teuthology.orchestra.run.smithi102.stdout:(80/156): python3-markupsafe-1.1.1-12.el9.x86_6 597 kB/s | 35 kB 00:00 2024-09-10T09:24:45.628 INFO:teuthology.orchestra.run.smithi102.stdout:(81/156): postgresql-server-13.16-1.el9.x86_64. 5.4 MB/s | 5.8 MB 00:01 2024-09-10T09:24:45.676 INFO:teuthology.orchestra.run.smithi159.stdout:(10/156): ceph-osd-19.1.1-305.g1cf819b7.el9.x86 19 MB/s | 17 MB 00:00 2024-09-10T09:24:45.712 INFO:teuthology.orchestra.run.smithi102.stdout:(82/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.9 MB/s | 250 kB 00:00 2024-09-10T09:24:45.770 INFO:teuthology.orchestra.run.smithi102.stdout:(83/156): python3-oauthlib-3.1.1-5.el9.noarch.r 3.7 MB/s | 222 kB 00:00 2024-09-10T09:24:45.801 INFO:teuthology.orchestra.run.smithi159.stdout:(11/156): ceph-radosgw-19.1.1-305.g1cf819b7.el9 11 MB/s | 11 MB 00:00 2024-09-10T09:24:45.820 INFO:teuthology.orchestra.run.smithi102.stdout:(84/156): python3-packaging-20.9-5.el9.noarch.r 1.5 MB/s | 77 kB 00:00 2024-09-10T09:24:45.826 INFO:teuthology.orchestra.run.smithi159.stdout:(12/156): cephfs-mirror-19.1.1-305.g1cf819b7.el 1.5 MB/s | 228 kB 00:00 2024-09-10T09:24:45.860 INFO:teuthology.orchestra.run.smithi159.stdout:(13/156): cephfs-shell-19.1.1-305.g1cf819b7.el9 462 kB/s | 27 kB 00:00 2024-09-10T09:24:45.871 INFO:teuthology.orchestra.run.smithi102.stdout:(85/156): python3-prettytable-0.7.2-27.el9.noar 845 kB/s | 42 kB 00:00 2024-09-10T09:24:45.894 INFO:teuthology.orchestra.run.smithi159.stdout:(14/156): libcephfs-devel-19.1.1-305.g1cf819b7. 474 kB/s | 32 kB 00:00 2024-09-10T09:24:45.953 INFO:teuthology.orchestra.run.smithi159.stdout:(15/156): libcephfs2-19.1.1-305.g1cf819b7.el9.x 8.0 MB/s | 751 kB 00:00 2024-09-10T09:24:45.954 INFO:teuthology.orchestra.run.smithi102.stdout:(86/156): python3-protobuf-3.14.0-13.el9.noarch 3.2 MB/s | 269 kB 00:00 2024-09-10T09:24:45.978 INFO:teuthology.orchestra.run.smithi159.stdout:(16/156): libcephsqlite-19.1.1-305.g1cf819b7.el 1.9 MB/s | 164 kB 00:00 2024-09-10T09:24:46.013 INFO:teuthology.orchestra.run.smithi102.stdout:(87/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.7 MB/s | 159 kB 00:00 2024-09-10T09:24:46.028 INFO:teuthology.orchestra.run.smithi159.stdout:(17/156): librados-devel-19.1.1-305.g1cf819b7.e 1.6 MB/s | 127 kB 00:00 2024-09-10T09:24:46.078 INFO:teuthology.orchestra.run.smithi159.stdout:(18/156): libradosstriper1-19.1.1-305.g1cf819b7 4.9 MB/s | 506 kB 00:00 2024-09-10T09:24:46.147 INFO:teuthology.orchestra.run.smithi102.stdout:(88/156): python3-babel-2.9.1-2.el9.noarch.rpm 4.9 MB/s | 6.0 MB 00:01 2024-09-10T09:24:46.170 INFO:teuthology.orchestra.run.smithi159.stdout:(19/156): python3-ceph-argparse-19.1.1-305.g1cf 492 kB/s | 45 kB 00:00 2024-09-10T09:24:46.172 INFO:teuthology.orchestra.run.smithi102.stdout:(89/156): python3-pyasn1-modules-0.4.8-6.el9.no 1.7 MB/s | 279 kB 00:00 2024-09-10T09:24:46.205 INFO:teuthology.orchestra.run.smithi102.stdout:(90/156): python3-pytz-2021.1-5.el9.noarch.rpm 873 kB/s | 51 kB 00:00 2024-09-10T09:24:46.230 INFO:teuthology.orchestra.run.smithi102.stdout:(91/156): python3-requests-oauthlib-1.3.0-12.el 917 kB/s | 54 kB 00:00 2024-09-10T09:24:46.270 INFO:teuthology.orchestra.run.smithi005.stdout:(1/156): ceph-19.1.1-305.g1cf819b7.el9.x86_64.r 19 kB/s | 6.5 kB 00:00 2024-09-10T09:24:46.281 INFO:teuthology.orchestra.run.smithi102.stdout:(92/156): python3-toml-0.10.2-6.el9.noarch.rpm 833 kB/s | 42 kB 00:00 2024-09-10T09:24:46.288 INFO:teuthology.orchestra.run.smithi159.stdout:(20/156): python3-ceph-common-19.1.1-305.g1cf81 1.1 MB/s | 136 kB 00:00 2024-09-10T09:24:46.340 INFO:teuthology.orchestra.run.smithi102.stdout:(93/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 757 kB/s | 44 kB 00:00 2024-09-10T09:24:46.388 INFO:teuthology.orchestra.run.smithi159.stdout:(21/156): python3-cephfs-19.1.1-305.g1cf819b7.e 1.6 MB/s | 161 kB 00:00 2024-09-10T09:24:46.400 INFO:teuthology.orchestra.run.smithi102.stdout:(94/156): qatlib-24.02.0-1.el9.x86_64.rpm 3.7 MB/s | 221 kB 00:00 2024-09-10T09:24:46.457 INFO:teuthology.orchestra.run.smithi102.stdout:(95/156): qatlib-service-24.02.0-1.el9.x86_64.r 614 kB/s | 36 kB 00:00 2024-09-10T09:24:46.515 INFO:teuthology.orchestra.run.smithi102.stdout:(96/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 814 kB/s | 47 kB 00:00 2024-09-10T09:24:46.538 INFO:teuthology.orchestra.run.smithi159.stdout:(22/156): librgw2-19.1.1-305.g1cf819b7.el9.x86_ 10 MB/s | 5.3 MB 00:00 2024-09-10T09:24:46.563 INFO:teuthology.orchestra.run.smithi159.stdout:(23/156): python3-rados-19.1.1-305.g1cf819b7.el 1.8 MB/s | 320 kB 00:00 2024-09-10T09:24:46.582 INFO:teuthology.orchestra.run.smithi102.stdout:(97/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 4.8 MB/s | 5.1 MB 00:01 2024-09-10T09:24:46.622 INFO:teuthology.orchestra.run.smithi159.stdout:(24/156): python3-rbd-19.1.1-305.g1cf819b7.el9. 3.5 MB/s | 301 kB 00:00 2024-09-10T09:24:46.641 INFO:teuthology.orchestra.run.smithi102.stdout:(98/156): socat-1.7.4.1-6.el9.x86_64.rpm 2.4 MB/s | 304 kB 00:00 2024-09-10T09:24:46.647 INFO:teuthology.orchestra.run.smithi159.stdout:(25/156): python3-rgw-19.1.1-305.g1cf819b7.el9. 1.2 MB/s | 99 kB 00:00 2024-09-10T09:24:46.658 INFO:teuthology.orchestra.run.smithi102.stdout:(99/156): uuid-1.6.2-55.el9.x86_64.rpm 768 kB/s | 58 kB 00:00 2024-09-10T09:24:46.692 INFO:teuthology.orchestra.run.smithi102.stdout:(100/156): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 1.9 MB/s | 96 kB 00:00 2024-09-10T09:24:46.697 INFO:teuthology.orchestra.run.smithi159.stdout:(26/156): rbd-fuse-19.1.1-305.g1cf819b7.el9.x86 1.1 MB/s | 85 kB 00:00 2024-09-10T09:24:46.709 INFO:teuthology.orchestra.run.smithi102.stdout:(101/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-09-10T09:24:46.734 INFO:teuthology.orchestra.run.smithi102.stdout:(102/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 1.0 MB/s | 45 kB 00:00 2024-09-10T09:24:46.820 INFO:teuthology.orchestra.run.smithi102.stdout:(103/156): lua-devel-5.4.4-4.el9.x86_64.rpm 201 kB/s | 22 kB 00:00 2024-09-10T09:24:46.868 INFO:teuthology.orchestra.run.smithi102.stdout:(104/156): abseil-cpp-20211102.0-3.el9.x86_64.r 11 MB/s | 548 kB 00:00 2024-09-10T09:24:46.893 INFO:teuthology.orchestra.run.smithi102.stdout:(105/156): gperftools-libs-2.9.1-2.el9.x86_64.r 12 MB/s | 309 kB 00:00 2024-09-10T09:24:46.918 INFO:teuthology.orchestra.run.smithi102.stdout:(106/156): grpc-data-1.46.7-10.el9.noarch.rpm 777 kB/s | 19 kB 00:00 2024-09-10T09:24:47.006 INFO:teuthology.orchestra.run.smithi159.stdout:(27/156): rbd-mirror-19.1.1-305.g1cf819b7.el9.x 8.6 MB/s | 3.1 MB 00:00 2024-09-10T09:24:47.020 INFO:teuthology.orchestra.run.smithi102.stdout:(107/156): libarrow-9.0.0-11.el9.x86_64.rpm 44 MB/s | 4.4 MB 00:00 2024-09-10T09:24:47.044 INFO:teuthology.orchestra.run.smithi102.stdout:(108/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 1.0 MB/s | 26 kB 00:00 2024-09-10T09:24:47.069 INFO:teuthology.orchestra.run.smithi102.stdout:(109/156): liboath-2.6.7-2.el9.x86_64.rpm 1.9 MB/s | 49 kB 00:00 2024-09-10T09:24:47.094 INFO:teuthology.orchestra.run.smithi102.stdout:(110/156): protobuf-compiler-3.14.0-13.el9.x86_ 2.3 MB/s | 863 kB 00:00 2024-09-10T09:24:47.098 INFO:teuthology.orchestra.run.smithi159.stdout:(28/156): ceph-grafana-dashboards-19.1.1-305.g1 347 kB/s | 32 kB 00:00 2024-09-10T09:24:47.111 INFO:teuthology.orchestra.run.smithi102.stdout:(111/156): libunwind-1.6.2-1.el9.x86_64.rpm 1.6 MB/s | 67 kB 00:00 2024-09-10T09:24:47.170 INFO:teuthology.orchestra.run.smithi102.stdout:(112/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 16 MB/s | 839 kB 00:00 2024-09-10T09:24:47.182 INFO:teuthology.orchestra.run.smithi159.stdout:(29/156): ceph-mgr-cephadm-19.1.1-305.g1cf819b7 1.7 MB/s | 146 kB 00:00 2024-09-10T09:24:47.195 INFO:teuthology.orchestra.run.smithi102.stdout:(113/156): luarocks-3.9.2-5.el9.noarch.rpm 1.5 MB/s | 151 kB 00:00 2024-09-10T09:24:47.220 INFO:teuthology.orchestra.run.smithi102.stdout:(114/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.7 MB/s | 43 kB 00:00 2024-09-10T09:24:47.246 INFO:teuthology.orchestra.run.smithi102.stdout:(115/156): python3-cachetools-4.2.4-1.el9.noarc 1.3 MB/s | 32 kB 00:00 2024-09-10T09:24:47.271 INFO:teuthology.orchestra.run.smithi102.stdout:(116/156): python3-certifi-2023.05.07-4.el9.noa 572 kB/s | 14 kB 00:00 2024-09-10T09:24:47.292 INFO:teuthology.orchestra.run.smithi005.stdout:(2/156): ceph-fuse-19.1.1-305.g1cf819b7.el9.x86 878 kB/s | 897 kB 00:01 2024-09-10T09:24:47.296 INFO:teuthology.orchestra.run.smithi102.stdout:(117/156): python3-asyncssh-2.13.2-5.el9.noarch 4.2 MB/s | 548 kB 00:00 2024-09-10T09:24:47.322 INFO:teuthology.orchestra.run.smithi102.stdout:(118/156): python3-cheroot-8.6.0-4.el9.noarch.r 3.3 MB/s | 172 kB 00:00 2024-09-10T09:24:47.355 INFO:teuthology.orchestra.run.smithi102.stdout:(119/156): python3-cherrypy-18.6.1-2.el9.noarch 6.0 MB/s | 358 kB 00:00 2024-09-10T09:24:47.381 INFO:teuthology.orchestra.run.smithi102.stdout:(120/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 5.1 MB/s | 303 kB 00:00 2024-09-10T09:24:47.406 INFO:teuthology.orchestra.run.smithi102.stdout:(121/156): python3-google-auth-2.34.0-1.el9.noa 8.7 MB/s | 220 kB 00:00 2024-09-10T09:24:47.432 INFO:teuthology.orchestra.run.smithi102.stdout:(122/156): python3-grpcio-tools-1.46.7-10.el9.x 5.7 MB/s | 144 kB 00:00 2024-09-10T09:24:47.456 INFO:teuthology.orchestra.run.smithi102.stdout:(123/156): python3-jaraco-8.2.1-3.el9.noarch.rp 426 kB/s | 11 kB 00:00 2024-09-10T09:24:47.482 INFO:teuthology.orchestra.run.smithi102.stdout:(124/156): python3-jaraco-classes-3.2.1-5.el9.n 708 kB/s | 18 kB 00:00 2024-09-10T09:24:47.499 INFO:teuthology.orchestra.run.smithi159.stdout:(30/156): ceph-mgr-dashboard-19.1.1-305.g1cf819 11 MB/s | 3.6 MB 00:00 2024-09-10T09:24:47.507 INFO:teuthology.orchestra.run.smithi102.stdout:(125/156): python3-jaraco-collections-3.0.0-8.e 931 kB/s | 23 kB 00:00 2024-09-10T09:24:47.508 INFO:teuthology.orchestra.run.smithi005.stdout:(3/156): ceph-immutable-object-cache-19.1.1-305 671 kB/s | 145 kB 00:00 2024-09-10T09:24:47.533 INFO:teuthology.orchestra.run.smithi102.stdout:(126/156): python3-jaraco-functools-3.5.0-2.el9 779 kB/s | 19 kB 00:00 2024-09-10T09:24:47.566 INFO:teuthology.orchestra.run.smithi102.stdout:(127/156): python3-jaraco-text-3.2.0-6.el9.noar 585 kB/s | 20 kB 00:00 2024-09-10T09:24:47.574 INFO:teuthology.orchestra.run.smithi159.stdout:(31/156): rbd-nbd-19.1.1-305.g1cf819b7.el9.x86_ 194 kB/s | 170 kB 00:00 2024-09-10T09:24:47.616 INFO:teuthology.orchestra.run.smithi102.stdout:(128/156): python3-grpcio-1.46.7-10.el9.x86_64. 8.7 MB/s | 2.0 MB 00:00 2024-09-10T09:24:47.642 INFO:teuthology.orchestra.run.smithi102.stdout:(129/156): python3-logutils-0.3.5-21.el9.noarch 1.8 MB/s | 46 kB 00:00 2024-09-10T09:24:47.667 INFO:teuthology.orchestra.run.smithi102.stdout:(130/156): python3-more-itertools-8.12.0-2.el9. 3.1 MB/s | 79 kB 00:00 2024-09-10T09:24:47.691 INFO:teuthology.orchestra.run.smithi159.stdout:(32/156): ceph-mgr-modules-core-19.1.1-305.g1cf 2.1 MB/s | 249 kB 00:00 2024-09-10T09:24:47.692 INFO:teuthology.orchestra.run.smithi102.stdout:(131/156): python3-natsort-7.1.1-5.el9.noarch.r 2.3 MB/s | 58 kB 00:00 2024-09-10T09:24:47.726 INFO:teuthology.orchestra.run.smithi102.stdout:(132/156): python3-kubernetes-26.1.0-2.el9.noar 6.4 MB/s | 1.0 MB 00:00 2024-09-10T09:24:47.751 INFO:teuthology.orchestra.run.smithi102.stdout:(133/156): python3-portend-3.1.0-2.el9.noarch.r 656 kB/s | 16 kB 00:00 2024-09-10T09:24:47.777 INFO:teuthology.orchestra.run.smithi102.stdout:(134/156): python3-pecan-1.4.2-3.el9.noarch.rpm 3.2 MB/s | 272 kB 00:00 2024-09-10T09:24:47.800 INFO:teuthology.orchestra.run.smithi159.stdout:(33/156): ceph-mgr-rook-19.1.1-305.g1cf819b7.el 455 kB/s | 49 kB 00:00 2024-09-10T09:24:47.802 INFO:teuthology.orchestra.run.smithi102.stdout:(135/156): python3-pyperclip-1.8.2-2.el9.noarch 1.1 MB/s | 28 kB 00:00 2024-09-10T09:24:47.819 INFO:teuthology.orchestra.run.smithi102.stdout:(136/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.3 MB/s | 90 kB 00:00 2024-09-10T09:24:47.844 INFO:teuthology.orchestra.run.smithi102.stdout:(137/156): python3-repoze-lru-0.7-16.el9.noarch 734 kB/s | 31 kB 00:00 2024-09-10T09:24:47.869 INFO:teuthology.orchestra.run.smithi102.stdout:(138/156): python3-rsa-4.9-2.el9.noarch.rpm 2.3 MB/s | 59 kB 00:00 2024-09-10T09:24:47.895 INFO:teuthology.orchestra.run.smithi102.stdout:(139/156): python3-routes-2.5.1-5.el9.noarch.rp 2.4 MB/s | 188 kB 00:00 2024-09-10T09:24:47.900 INFO:teuthology.orchestra.run.smithi159.stdout:(34/156): ceph-prometheus-alerts-19.1.1-305.g1c 165 kB/s | 16 kB 00:00 2024-09-10T09:24:47.920 INFO:teuthology.orchestra.run.smithi102.stdout:(140/156): python3-tempora-5.0.0-2.el9.noarch.r 713 kB/s | 36 kB 00:00 2024-09-10T09:24:47.945 INFO:teuthology.orchestra.run.smithi102.stdout:(141/156): python3-typing-extensions-4.12.2-2.e 2.9 MB/s | 74 kB 00:00 2024-09-10T09:24:47.970 INFO:teuthology.orchestra.run.smithi102.stdout:(142/156): python3-websocket-client-1.2.3-2.el9 3.5 MB/s | 90 kB 00:00 2024-09-10T09:24:47.996 INFO:teuthology.orchestra.run.smithi102.stdout:(143/156): python3-webob-1.8.8-2.el9.noarch.rpm 3.0 MB/s | 230 kB 00:00 2024-09-10T09:24:48.013 INFO:teuthology.orchestra.run.smithi102.stdout:(144/156): python3-zc-lockfile-2.0-10.el9.noarc 1.2 MB/s | 20 kB 00:00 2024-09-10T09:24:48.038 INFO:teuthology.orchestra.run.smithi102.stdout:(145/156): re2-20211101-20.el9.x86_64.rpm 7.6 MB/s | 191 kB 00:00 2024-09-10T09:24:48.071 INFO:teuthology.orchestra.run.smithi102.stdout:(146/156): python3-werkzeug-2.0.3-3.el9.1.noarc 4.1 MB/s | 427 kB 00:00 2024-09-10T09:24:48.100 INFO:teuthology.orchestra.run.smithi159.stdout:(35/156): ceph-mgr-diskprediction-local-19.1.1- 12 MB/s | 7.4 MB 00:00 2024-09-10T09:24:48.122 INFO:teuthology.orchestra.run.smithi102.stdout:(147/156): thrift-0.15.0-2.el9.x86_64.rpm 19 MB/s | 1.6 MB 00:00 2024-09-10T09:24:48.534 INFO:teuthology.orchestra.run.smithi159.stdout:(36/156): ceph-test-19.1.1-305.g1cf819b7.el9.x8 15 MB/s | 49 MB 00:03 2024-09-10T09:24:48.559 INFO:teuthology.orchestra.run.smithi159.stdout:(37/156): ceph-volume-19.1.1-305.g1cf819b7.el9. 433 kB/s | 285 kB 00:00 2024-09-10T09:24:48.593 INFO:teuthology.orchestra.run.smithi159.stdout:(38/156): cephadm-19.1.1-305.g1cf819b7.el9.noar 1.5 MB/s | 766 kB 00:00 2024-09-10T09:24:48.605 INFO:teuthology.orchestra.run.smithi102.stdout:(148/156): librados2-19.1.1-305.g1cf819b7.el9.x 6.4 MB/s | 3.4 MB 00:00 2024-09-10T09:24:48.610 INFO:teuthology.orchestra.run.smithi159.stdout:(39/156): cephfs-top-19.1.1-305.g1cf819b7.el9.n 243 kB/s | 18 kB 00:00 2024-09-10T09:24:48.656 INFO:teuthology.orchestra.run.smithi102.stdout:(149/156): librbd1-19.1.1-305.g1cf819b7.el9.x86 5.9 MB/s | 3.2 MB 00:00 2024-09-10T09:24:48.685 INFO:teuthology.orchestra.run.smithi159.stdout:(40/156): fuse-2.9.9-16.el9.x86_64.rpm 636 kB/s | 80 kB 00:00 2024-09-10T09:24:48.692 INFO:teuthology.orchestra.run.smithi005.stdout:(4/156): ceph-base-19.1.1-305.g1cf819b7.el9.x86 2.0 MB/s | 5.4 MB 00:02 2024-09-10T09:24:48.752 INFO:teuthology.orchestra.run.smithi159.stdout:(41/156): libquadmath-11.5.0-2.el9.x86_64.rpm 1.3 MB/s | 191 kB 00:00 2024-09-10T09:24:48.777 INFO:teuthology.orchestra.run.smithi159.stdout:(42/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.7 MB/s | 253 kB 00:00 2024-09-10T09:24:48.828 INFO:teuthology.orchestra.run.smithi159.stdout:(43/156): libgfortran-11.5.0-2.el9.x86_64.rpm 3.3 MB/s | 801 kB 00:00 2024-09-10T09:24:48.853 INFO:teuthology.orchestra.run.smithi159.stdout:(44/156): python3-ply-3.11-14.el9.noarch.rpm 1.4 MB/s | 106 kB 00:00 2024-09-10T09:24:48.873 INFO:teuthology.orchestra.run.smithi102.stdout:(150/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 6.1 MB/s | 16 MB 00:02 2024-09-10T09:24:48.878 INFO:teuthology.orchestra.run.smithi159.stdout:(45/156): python3-pycparser-2.20-6.el9.noarch.r 2.6 MB/s | 135 kB 00:00 2024-09-10T09:24:48.893 INFO:teuthology.orchestra.run.smithi005.stdout:(5/156): ceph-mgr-19.1.1-305.g1cf819b7.el9.x86_ 7.6 MB/s | 1.5 MB 00:00 2024-09-10T09:24:48.904 INFO:teuthology.orchestra.run.smithi159.stdout:(46/156): python3-requests-2.25.1-8.el9.noarch. 2.4 MB/s | 125 kB 00:00 2024-09-10T09:24:48.929 INFO:teuthology.orchestra.run.smithi159.stdout:(47/156): python3-urllib3-1.26.5-6.el9.noarch.r 4.2 MB/s | 215 kB 00:00 2024-09-10T09:24:48.962 INFO:teuthology.orchestra.run.smithi159.stdout:(48/156): python3-cryptography-36.0.1-4.el9.x86 5.8 MB/s | 1.2 MB 00:00 2024-09-10T09:24:49.018 INFO:teuthology.orchestra.run.smithi005.stdout:(6/156): ceph-mds-19.1.1-305.g1cf819b7.el9.x86_ 1.6 MB/s | 2.4 MB 00:01 2024-09-10T09:24:49.388 INFO:teuthology.orchestra.run.smithi159.stdout:(49/156): boost-program-options-1.75.0-8.el9.x8 251 kB/s | 107 kB 00:00 2024-09-10T09:24:49.410 INFO:teuthology.orchestra.run.smithi005.stdout:(7/156): ceph-mon-19.1.1-305.g1cf819b7.el9.x86_ 9.1 MB/s | 4.7 MB 00:00 2024-09-10T09:24:49.440 INFO:teuthology.orchestra.run.smithi102.stdout:(151/156): NetworkManager-team-1.49.90-1.el9.x8 72 kB/s | 41 kB 00:00 2024-09-10T09:24:49.505 INFO:teuthology.orchestra.run.smithi159.stdout:(50/156): elfutils-libelf-devel-0.191-4.el9.x86 186 kB/s | 22 kB 00:00 2024-09-10T09:24:49.627 INFO:teuthology.orchestra.run.smithi005.stdout:(8/156): ceph-common-19.1.1-305.g1cf819b7.el9.x 5.7 MB/s | 21 MB 00:03 2024-09-10T09:24:49.655 INFO:teuthology.orchestra.run.smithi159.stdout:(51/156): automake-1.16.2-8.el9.noarch.rpm 917 kB/s | 689 kB 00:00 2024-09-10T09:24:49.694 INFO:teuthology.orchestra.run.smithi005.stdout:(9/156): ceph-selinux-19.1.1-305.g1cf819b7.el9. 378 kB/s | 25 kB 00:00 2024-09-10T09:24:49.731 INFO:teuthology.orchestra.run.smithi102.stdout:(152/156): NetworkManager-1.49.90-1.el9.x86_64. 2.1 MB/s | 2.3 MB 00:01 2024-09-10T09:24:49.764 INFO:teuthology.orchestra.run.smithi159.stdout:(52/156): flexiblas-3.0.4-8.el9.x86_64.rpm 302 kB/s | 33 kB 00:00 2024-09-10T09:24:49.773 INFO:teuthology.orchestra.run.smithi102.stdout:(153/156): NetworkManager-libnm-1.49.90-1.el9.x 1.6 MB/s | 1.8 MB 00:01 2024-09-10T09:24:49.789 INFO:teuthology.orchestra.run.smithi159.stdout:(53/156): flex-2.6.4-9.el9.x86_64.rpm 1.1 MB/s | 313 kB 00:00 2024-09-10T09:24:49.814 INFO:teuthology.orchestra.run.smithi159.stdout:(54/156): bison-3.7.4-5.el9.x86_64.rpm 1.0 MB/s | 943 kB 00:00 2024-09-10T09:24:49.906 INFO:teuthology.orchestra.run.smithi159.stdout:(55/156): flexiblas-openblas-openmp-3.0.4-8.el9 155 kB/s | 18 kB 00:00 2024-09-10T09:24:49.931 INFO:teuthology.orchestra.run.smithi159.stdout:(56/156): libaio-devel-0.3.111-13.el9.x86_64.rp 95 kB/s | 11 kB 00:00 2024-09-10T09:24:50.023 INFO:teuthology.orchestra.run.smithi159.stdout:(57/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.3 MB/s | 160 kB 00:00 2024-09-10T09:24:50.049 INFO:teuthology.orchestra.run.smithi159.stdout:(58/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 387 kB/s | 45 kB 00:00 2024-09-10T09:24:50.166 INFO:teuthology.orchestra.run.smithi159.stdout:(59/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 7.4 MB/s | 3.0 MB 00:00 2024-09-10T09:24:50.198 INFO:teuthology.orchestra.run.smithi102.stdout:(154/156): NetworkManager-tui-1.49.90-1.el9.x86 328 kB/s | 249 kB 00:00 2024-09-10T09:24:50.277 INFO:teuthology.orchestra.run.smithi005.stdout:(10/156): ceph-radosgw-19.1.1-305.g1cf819b7.el9 12 MB/s | 11 MB 00:00 2024-09-10T09:24:50.282 INFO:teuthology.orchestra.run.smithi159.stdout:(60/156): libtool-2.4.6-46.el9.x86_64.rpm 2.4 MB/s | 581 kB 00:00 2024-09-10T09:24:50.308 INFO:teuthology.orchestra.run.smithi159.stdout:(61/156): libuuid-devel-2.37.4-20.el9.x86_64.rp 172 kB/s | 24 kB 00:00 2024-09-10T09:24:50.333 INFO:teuthology.orchestra.run.smithi159.stdout:(62/156): librdkafka-1.6.1-102.el9.x86_64.rpm 2.1 MB/s | 662 kB 00:00 2024-09-10T09:24:50.383 INFO:teuthology.orchestra.run.smithi159.stdout:(63/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 471 kB/s | 47 kB 00:00 2024-09-10T09:24:50.403 INFO:teuthology.orchestra.run.smithi005.stdout:(11/156): cephfs-mirror-19.1.1-305.g1cf819b7.el 1.8 MB/s | 228 kB 00:00 2024-09-10T09:24:50.417 INFO:teuthology.orchestra.run.smithi159.stdout:(64/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.6 MB/s | 292 kB 00:00 2024-09-10T09:24:50.442 INFO:teuthology.orchestra.run.smithi159.stdout:(65/156): lua-5.4.4-4.el9.x86_64.rpm 1.7 MB/s | 188 kB 00:00 2024-09-10T09:24:50.484 INFO:teuthology.orchestra.run.smithi159.stdout:(66/156): openblas-0.3.26-2.el9.x86_64.rpm 383 kB/s | 38 kB 00:00 2024-09-10T09:24:50.486 INFO:teuthology.orchestra.run.smithi005.stdout:(12/156): cephfs-shell-19.1.1-305.g1cf819b7.el9 326 kB/s | 27 kB 00:00 2024-09-10T09:24:50.570 INFO:teuthology.orchestra.run.smithi005.stdout:(13/156): libcephfs-devel-19.1.1-305.g1cf819b7. 381 kB/s | 32 kB 00:00 2024-09-10T09:24:50.665 INFO:teuthology.orchestra.run.smithi102.stdout:(155/156): openssl-3.2.2-6.el9.x86_64.rpm 1.5 MB/s | 1.4 MB 00:00 2024-09-10T09:24:50.670 INFO:teuthology.orchestra.run.smithi005.stdout:(14/156): libcephfs2-19.1.1-305.g1cf819b7.el9.x 7.4 MB/s | 751 kB 00:00 2024-09-10T09:24:50.701 INFO:teuthology.orchestra.run.smithi159.stdout:(67/156): postgresql-13.16-1.el9.x86_64.rpm 7.3 MB/s | 1.6 MB 00:00 2024-09-10T09:24:50.787 INFO:teuthology.orchestra.run.smithi005.stdout:(15/156): libcephsqlite-19.1.1-305.g1cf819b7.el 1.4 MB/s | 164 kB 00:00 2024-09-10T09:24:50.807 INFO:teuthology.orchestra.run.smithi102.stdout:(156/156): openssl-libs-3.2.2-6.el9.x86_64.rpm 2.3 MB/s | 2.4 MB 00:01 2024-09-10T09:24:50.813 INFO:teuthology.orchestra.run.smithi102.stdout:-------------------------------------------------------------------------------- 2024-09-10T09:24:50.813 INFO:teuthology.orchestra.run.smithi102.stdout:Total 16 MB/s | 226 MB 00:14 2024-09-10T09:24:50.836 INFO:teuthology.orchestra.run.smithi159.stdout:(68/156): postgresql-contrib-13.16-1.el9.x86_64 6.6 MB/s | 910 kB 00:00 2024-09-10T09:24:50.879 INFO:teuthology.orchestra.run.smithi005.stdout:(16/156): librados-devel-19.1.1-305.g1cf819b7.e 1.3 MB/s | 127 kB 00:00 2024-09-10T09:24:50.953 INFO:teuthology.orchestra.run.smithi159.stdout:(69/156): postgresql-private-libs-13.16-1.el9.x 1.2 MB/s | 137 kB 00:00 2024-09-10T09:24:50.963 INFO:teuthology.orchestra.run.smithi005.stdout:(17/156): libradosstriper1-19.1.1-305.g1cf819b7 5.9 MB/s | 506 kB 00:00 2024-09-10T09:24:51.287 INFO:teuthology.orchestra.run.smithi159.stdout:(70/156): postgresql-server-13.16-1.el9.x86_64. 17 MB/s | 5.8 MB 00:00 2024-09-10T09:24:51.404 INFO:teuthology.orchestra.run.smithi159.stdout:(71/156): protobuf-3.14.0-13.el9.x86_64.rpm 8.7 MB/s | 1.0 MB 00:00 2024-09-10T09:24:51.512 INFO:teuthology.orchestra.run.smithi159.stdout:(72/156): python3-attrs-20.3.0-7.el9.noarch.rpm 808 kB/s | 87 kB 00:00 2024-09-10T09:24:51.513 INFO:teuthology.orchestra.run.smithi005.stdout:(18/156): librgw2-19.1.1-305.g1cf819b7.el9.x86_ 9.7 MB/s | 5.3 MB 00:00 2024-09-10T09:24:51.555 INFO:teuthology.orchestra.run.smithi005.stdout:(19/156): python3-ceph-argparse-19.1.1-305.g1cf 1.1 MB/s | 45 kB 00:00 2024-09-10T09:24:51.596 INFO:teuthology.orchestra.run.smithi159.stdout:(73/156): openblas-openmp-0.3.26-2.el9.x86_64.r 4.2 MB/s | 4.9 MB 00:01 2024-09-10T09:24:51.631 INFO:teuthology.orchestra.run.smithi005.stdout:(20/156): python3-ceph-common-19.1.1-305.g1cf81 1.8 MB/s | 136 kB 00:00 2024-09-10T09:24:51.705 INFO:teuthology.orchestra.run.smithi159.stdout:(74/156): python3-colorama-0.4.6-3.el9.noarch.r 566 kB/s | 61 kB 00:00 2024-09-10T09:24:51.705 INFO:teuthology.orchestra.run.smithi005.stdout:(21/156): python3-cephfs-19.1.1-305.g1cf819b7.e 2.1 MB/s | 161 kB 00:00 2024-09-10T09:24:51.781 INFO:teuthology.orchestra.run.smithi005.stdout:(22/156): python3-rados-19.1.1-305.g1cf819b7.el 4.2 MB/s | 320 kB 00:00 2024-09-10T09:24:51.847 INFO:teuthology.orchestra.run.smithi159.stdout:(75/156): python3-babel-2.9.1-2.el9.noarch.rpm 18 MB/s | 6.0 MB 00:00 2024-09-10T09:24:51.964 INFO:teuthology.orchestra.run.smithi159.stdout:(76/156): python3-jinja2-2.11.3-6.el9.noarch.rp 2.1 MB/s | 248 kB 00:00 2024-09-10T09:24:52.049 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction check 2024-09-10T09:24:52.072 INFO:teuthology.orchestra.run.smithi159.stdout:(77/156): python3-jsonpatch-1.21-16.el9.noarch. 237 kB/s | 26 kB 00:00 2024-09-10T09:24:52.098 INFO:teuthology.orchestra.run.smithi159.stdout:(78/156): python3-devel-3.9.19-8.el9.x86_64.rpm 623 kB/s | 245 kB 00:00 2024-09-10T09:24:52.173 INFO:teuthology.orchestra.run.smithi159.stdout:(79/156): python3-jsonpointer-2.0-4.el9.noarch. 191 kB/s | 19 kB 00:00 2024-09-10T09:24:52.200 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction check succeeded. 2024-09-10T09:24:52.200 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction test 2024-09-10T09:24:52.232 INFO:teuthology.orchestra.run.smithi005.stdout:(23/156): ceph-test-19.1.1-305.g1cf819b7.el9.x8 19 MB/s | 49 MB 00:02 2024-09-10T09:24:52.273 INFO:teuthology.orchestra.run.smithi159.stdout:(80/156): python3-markupsafe-1.1.1-12.el9.x86_6 347 kB/s | 35 kB 00:00 2024-09-10T09:24:52.274 INFO:teuthology.orchestra.run.smithi005.stdout:(24/156): python3-rbd-19.1.1-305.g1cf819b7.el9. 611 kB/s | 301 kB 00:00 2024-09-10T09:24:52.298 INFO:teuthology.orchestra.run.smithi005.stdout:(25/156): python3-rgw-19.1.1-305.g1cf819b7.el9. 1.5 MB/s | 99 kB 00:00 2024-09-10T09:24:52.324 INFO:teuthology.orchestra.run.smithi005.stdout:(26/156): rbd-fuse-19.1.1-305.g1cf819b7.el9.x86 1.7 MB/s | 85 kB 00:00 2024-09-10T09:24:52.382 INFO:teuthology.orchestra.run.smithi159.stdout:(81/156): python3-mako-1.1.4-6.el9.noarch.rpm 606 kB/s | 172 kB 00:00 2024-09-10T09:24:52.391 INFO:teuthology.orchestra.run.smithi005.stdout:(27/156): rbd-nbd-19.1.1-305.g1cf819b7.el9.x86_ 2.5 MB/s | 170 kB 00:00 2024-09-10T09:24:52.491 INFO:teuthology.orchestra.run.smithi005.stdout:(28/156): rbd-mirror-19.1.1-305.g1cf819b7.el9.x 16 MB/s | 3.1 MB 00:00 2024-09-10T09:24:52.508 INFO:teuthology.orchestra.run.smithi005.stdout:(29/156): ceph-grafana-dashboards-19.1.1-305.g1 272 kB/s | 32 kB 00:00 2024-09-10T09:24:52.542 INFO:teuthology.orchestra.run.smithi005.stdout:(30/156): ceph-mgr-cephadm-19.1.1-305.g1cf819b7 2.8 MB/s | 146 kB 00:00 2024-09-10T09:24:52.582 INFO:teuthology.orchestra.run.smithi159.stdout:(82/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 17 MB/s | 5.1 MB 00:00 2024-09-10T09:24:52.691 INFO:teuthology.orchestra.run.smithi159.stdout:(83/156): python3-oauthlib-3.1.1-5.el9.noarch.r 2.0 MB/s | 222 kB 00:00 2024-09-10T09:24:52.791 INFO:teuthology.orchestra.run.smithi159.stdout:(84/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 613 kB/s | 250 kB 00:00 2024-09-10T09:24:52.809 INFO:teuthology.orchestra.run.smithi005.stdout:(31/156): ceph-mgr-dashboard-19.1.1-305.g1cf819 12 MB/s | 3.6 MB 00:00 2024-09-10T09:24:52.816 INFO:teuthology.orchestra.run.smithi159.stdout:(85/156): python3-packaging-20.9-5.el9.noarch.r 616 kB/s | 77 kB 00:00 2024-09-10T09:24:52.892 INFO:teuthology.orchestra.run.smithi159.stdout:(86/156): python3-prettytable-0.7.2-27.el9.noar 421 kB/s | 42 kB 00:00 2024-09-10T09:24:52.909 INFO:teuthology.orchestra.run.smithi005.stdout:(32/156): ceph-mgr-diskprediction-local-19.1.1- 20 MB/s | 7.4 MB 00:00 2024-09-10T09:24:52.925 INFO:teuthology.orchestra.run.smithi159.stdout:(87/156): python3-protobuf-3.14.0-13.el9.noarch 2.4 MB/s | 269 kB 00:00 2024-09-10T09:24:52.934 INFO:teuthology.orchestra.run.smithi005.stdout:(33/156): ceph-mgr-modules-core-19.1.1-305.g1cf 1.9 MB/s | 249 kB 00:00 2024-09-10T09:24:52.962 INFO:teuthology.orchestra.run.smithi005.stdout:(34/156): ceph-mgr-rook-19.1.1-305.g1cf819b7.el 985 kB/s | 49 kB 00:00 2024-09-10T09:24:52.985 INFO:teuthology.orchestra.run.smithi005.stdout:(35/156): ceph-prometheus-alerts-19.1.1-305.g1c 327 kB/s | 16 kB 00:00 2024-09-10T09:24:53.010 INFO:teuthology.orchestra.run.smithi005.stdout:(36/156): ceph-volume-19.1.1-305.g1cf819b7.el9. 5.6 MB/s | 285 kB 00:00 2024-09-10T09:24:53.034 INFO:teuthology.orchestra.run.smithi159.stdout:(88/156): python3-pyasn1-modules-0.4.8-6.el9.no 2.5 MB/s | 279 kB 00:00 2024-09-10T09:24:53.052 INFO:teuthology.orchestra.run.smithi005.stdout:(37/156): cephadm-19.1.1-305.g1cf819b7.el9.noar 11 MB/s | 766 kB 00:00 2024-09-10T09:24:53.078 INFO:teuthology.orchestra.run.smithi005.stdout:(38/156): cephfs-top-19.1.1-305.g1cf819b7.el9.n 273 kB/s | 18 kB 00:00 2024-09-10T09:24:53.143 INFO:teuthology.orchestra.run.smithi159.stdout:(89/156): python3-pytz-2021.1-5.el9.noarch.rpm 470 kB/s | 51 kB 00:00 2024-09-10T09:24:53.176 INFO:teuthology.orchestra.run.smithi159.stdout:(90/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 559 kB/s | 159 kB 00:00 2024-09-10T09:24:53.259 INFO:teuthology.orchestra.run.smithi159.stdout:(91/156): python3-requests-oauthlib-1.3.0-12.el 461 kB/s | 54 kB 00:00 2024-09-10T09:24:53.328 INFO:teuthology.orchestra.run.smithi005.stdout:(39/156): fuse-2.9.9-16.el9.x86_64.rpm 290 kB/s | 80 kB 00:00 2024-09-10T09:24:53.335 INFO:teuthology.orchestra.run.smithi159.stdout:(92/156): openssl-devel-3.2.2-6.el9.x86_64.rpm 1.5 MB/s | 4.4 MB 00:02 2024-09-10T09:24:53.360 INFO:teuthology.orchestra.run.smithi159.stdout:(93/156): python3-toml-0.10.2-6.el9.noarch.rpm 413 kB/s | 42 kB 00:00 2024-09-10T09:24:53.435 INFO:teuthology.orchestra.run.smithi159.stdout:(94/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 440 kB/s | 44 kB 00:00 2024-09-10T09:24:53.477 INFO:teuthology.orchestra.run.smithi159.stdout:(95/156): qatlib-24.02.0-1.el9.x86_64.rpm 1.8 MB/s | 221 kB 00:00 2024-09-10T09:24:53.544 INFO:teuthology.orchestra.run.smithi159.stdout:(96/156): qatlib-service-24.02.0-1.el9.x86_64.r 331 kB/s | 36 kB 00:00 2024-09-10T09:24:53.578 INFO:teuthology.orchestra.run.smithi159.stdout:(97/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 472 kB/s | 47 kB 00:00 2024-09-10T09:24:53.586 INFO:teuthology.orchestra.run.smithi005.stdout:(40/156): libquadmath-11.5.0-2.el9.x86_64.rpm 740 kB/s | 191 kB 00:00 2024-09-10T09:24:53.695 INFO:teuthology.orchestra.run.smithi159.stdout:(98/156): uuid-1.6.2-55.el9.x86_64.rpm 494 kB/s | 58 kB 00:00 2024-09-10T09:24:53.703 INFO:teuthology.orchestra.run.smithi005.stdout:(41/156): libgfortran-11.5.0-2.el9.x86_64.rpm 1.3 MB/s | 801 kB 00:00 2024-09-10T09:24:53.780 INFO:teuthology.orchestra.run.smithi005.stdout:(42/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.3 MB/s | 253 kB 00:00 2024-09-10T09:24:53.795 INFO:teuthology.orchestra.run.smithi159.stdout:(99/156): xfsprogs-devel-6.4.0-4.el9.x86_64.rpm 960 kB/s | 96 kB 00:00 2024-09-10T09:24:53.863 INFO:teuthology.orchestra.run.smithi005.stdout:(43/156): python3-ply-3.11-14.el9.noarch.rpm 1.2 MB/s | 106 kB 00:00 2024-09-10T09:24:53.904 INFO:teuthology.orchestra.run.smithi159.stdout:(100/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 588 kB/s | 64 kB 00:00 2024-09-10T09:24:53.938 INFO:teuthology.orchestra.run.smithi005.stdout:(44/156): python3-pycparser-2.20-6.el9.noarch.r 1.8 MB/s | 135 kB 00:00 2024-09-10T09:24:53.979 INFO:teuthology.orchestra.run.smithi159.stdout:(101/156): socat-1.7.4.1-6.el9.x86_64.rpm 700 kB/s | 304 kB 00:00 2024-09-10T09:24:53.996 INFO:teuthology.orchestra.run.smithi159.stdout:(102/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 488 kB/s | 45 kB 00:00 2024-09-10T09:24:54.005 INFO:teuthology.orchestra.run.smithi005.stdout:(45/156): python3-requests-2.25.1-8.el9.noarch. 1.8 MB/s | 125 kB 00:00 2024-09-10T09:24:54.018 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction test succeeded. 2024-09-10T09:24:54.019 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction 2024-09-10T09:24:54.055 INFO:teuthology.orchestra.run.smithi159.stdout:(103/156): lua-devel-5.4.4-4.el9.x86_64.rpm 297 kB/s | 22 kB 00:00 2024-09-10T09:24:54.105 INFO:teuthology.orchestra.run.smithi159.stdout:(104/156): abseil-cpp-20211102.0-3.el9.x86_64.r 11 MB/s | 548 kB 00:00 2024-09-10T09:24:54.122 INFO:teuthology.orchestra.run.smithi005.stdout:(46/156): python3-urllib3-1.26.5-6.el9.noarch.r 1.8 MB/s | 215 kB 00:00 2024-09-10T09:24:54.139 INFO:teuthology.orchestra.run.smithi159.stdout:(105/156): gperftools-libs-2.9.1-2.el9.x86_64.r 9.0 MB/s | 309 kB 00:00 2024-09-10T09:24:54.155 INFO:teuthology.orchestra.run.smithi159.stdout:(106/156): grpc-data-1.46.7-10.el9.noarch.rpm 1.1 MB/s | 19 kB 00:00 2024-09-10T09:24:54.172 INFO:teuthology.orchestra.run.smithi005.stdout:(47/156): python3-cryptography-36.0.1-4.el9.x86 2.6 MB/s | 1.2 MB 00:00 2024-09-10T09:24:54.265 INFO:teuthology.orchestra.run.smithi159.stdout:(107/156): libarrow-9.0.0-11.el9.x86_64.rpm 41 MB/s | 4.4 MB 00:00 2024-09-10T09:24:54.289 INFO:teuthology.orchestra.run.smithi159.stdout:(108/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 1.0 MB/s | 26 kB 00:00 2024-09-10T09:24:54.323 INFO:teuthology.orchestra.run.smithi159.stdout:(109/156): protobuf-compiler-3.14.0-13.el9.x86_ 2.6 MB/s | 863 kB 00:00 2024-09-10T09:24:54.357 INFO:teuthology.orchestra.run.smithi159.stdout:(110/156): liboath-2.6.7-2.el9.x86_64.rpm 725 kB/s | 49 kB 00:00 2024-09-10T09:24:54.382 INFO:teuthology.orchestra.run.smithi159.stdout:(111/156): luarocks-3.9.2-5.el9.noarch.rpm 6.4 MB/s | 151 kB 00:00 2024-09-10T09:24:54.406 INFO:teuthology.orchestra.run.smithi005.stdout:(48/156): ceph-osd-19.1.1-305.g1cf819b7.el9.x86 3.1 MB/s | 17 MB 00:05 2024-09-10T09:24:54.407 INFO:teuthology.orchestra.run.smithi159.stdout:(112/156): libunwind-1.6.2-1.el9.x86_64.rpm 805 kB/s | 67 kB 00:00 2024-09-10T09:24:54.448 INFO:teuthology.orchestra.run.smithi005.stdout:(49/156): automake-1.16.2-8.el9.noarch.rpm 2.1 MB/s | 689 kB 00:00 2024-09-10T09:24:54.458 INFO:teuthology.orchestra.run.smithi159.stdout:(113/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 11 MB/s | 839 kB 00:00 2024-09-10T09:24:54.483 INFO:teuthology.orchestra.run.smithi159.stdout:(114/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.7 MB/s | 43 kB 00:00 2024-09-10T09:24:54.500 INFO:teuthology.orchestra.run.smithi005.stdout:(50/156): elfutils-libelf-devel-0.191-4.el9.x86 433 kB/s | 22 kB 00:00 2024-09-10T09:24:54.508 INFO:teuthology.orchestra.run.smithi159.stdout:(115/156): python3-cachetools-4.2.4-1.el9.noarc 1.3 MB/s | 32 kB 00:00 2024-09-10T09:24:54.532 INFO:teuthology.orchestra.run.smithi005.stdout:(51/156): boost-program-options-1.75.0-8.el9.x8 855 kB/s | 107 kB 00:00 2024-09-10T09:24:54.534 INFO:teuthology.orchestra.run.smithi159.stdout:(116/156): python3-certifi-2023.05.07-4.el9.noa 568 kB/s | 14 kB 00:00 2024-09-10T09:24:54.559 INFO:teuthology.orchestra.run.smithi159.stdout:(117/156): python3-asyncssh-2.13.2-5.el9.noarch 3.5 MB/s | 548 kB 00:00 2024-09-10T09:24:54.582 INFO:teuthology.orchestra.run.smithi005.stdout:(52/156): flex-2.6.4-9.el9.x86_64.rpm 3.7 MB/s | 313 kB 00:00 2024-09-10T09:24:54.584 INFO:teuthology.orchestra.run.smithi159.stdout:(118/156): python3-cheroot-8.6.0-4.el9.noarch.r 3.3 MB/s | 172 kB 00:00 2024-09-10T09:24:54.618 INFO:teuthology.orchestra.run.smithi159.stdout:(119/156): python3-cherrypy-18.6.1-2.el9.noarch 6.0 MB/s | 358 kB 00:00 2024-09-10T09:24:54.641 INFO:teuthology.orchestra.run.smithi005.stdout:(53/156): bison-3.7.4-5.el9.x86_64.rpm 2.0 MB/s | 943 kB 00:00 2024-09-10T09:24:54.643 INFO:teuthology.orchestra.run.smithi159.stdout:(120/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 5.1 MB/s | 303 kB 00:00 2024-09-10T09:24:54.666 INFO:teuthology.orchestra.run.smithi005.stdout:(54/156): flexiblas-3.0.4-8.el9.x86_64.rpm 244 kB/s | 33 kB 00:00 2024-09-10T09:24:54.669 INFO:teuthology.orchestra.run.smithi159.stdout:(121/156): python3-google-auth-2.34.0-1.el9.noa 4.3 MB/s | 220 kB 00:00 2024-09-10T09:24:54.691 INFO:teuthology.orchestra.run.smithi005.stdout:(55/156): flexiblas-openblas-openmp-3.0.4-8.el9 364 kB/s | 18 kB 00:00 2024-09-10T09:24:54.702 INFO:teuthology.orchestra.run.smithi159.stdout:(122/156): python3-grpcio-tools-1.46.7-10.el9.x 4.3 MB/s | 144 kB 00:00 2024-09-10T09:24:54.717 INFO:teuthology.orchestra.run.smithi005.stdout:(56/156): libaio-devel-0.3.111-13.el9.x86_64.rp 222 kB/s | 11 kB 00:00 2024-09-10T09:24:54.727 INFO:teuthology.orchestra.run.smithi159.stdout:(123/156): python3-jaraco-8.2.1-3.el9.noarch.rp 426 kB/s | 11 kB 00:00 2024-09-10T09:24:54.742 INFO:teuthology.orchestra.run.smithi005.stdout:(57/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.1 MB/s | 160 kB 00:00 2024-09-10T09:24:54.752 INFO:teuthology.orchestra.run.smithi159.stdout:(124/156): python3-jaraco-classes-3.2.1-5.el9.n 713 kB/s | 18 kB 00:00 2024-09-10T09:24:54.767 INFO:teuthology.orchestra.run.smithi005.stdout:(58/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 902 kB/s | 45 kB 00:00 2024-09-10T09:24:54.777 INFO:teuthology.orchestra.run.smithi159.stdout:(125/156): python3-jaraco-collections-3.0.0-8.e 930 kB/s | 23 kB 00:00 2024-09-10T09:24:54.803 INFO:teuthology.orchestra.run.smithi159.stdout:(126/156): python3-jaraco-functools-3.5.0-2.el9 776 kB/s | 19 kB 00:00 2024-09-10T09:24:54.837 INFO:teuthology.orchestra.run.smithi159.stdout:(127/156): python3-jaraco-text-3.2.0-6.el9.noar 588 kB/s | 20 kB 00:00 2024-09-10T09:24:54.904 INFO:teuthology.orchestra.run.smithi159.stdout:(128/156): python3-grpcio-1.46.7-10.el9.x86_64. 7.8 MB/s | 2.0 MB 00:00 2024-09-10T09:24:54.909 INFO:teuthology.orchestra.run.smithi005.stdout:(59/156): libtool-2.4.6-46.el9.x86_64.rpm 4.0 MB/s | 581 kB 00:00 2024-09-10T09:24:54.929 INFO:teuthology.orchestra.run.smithi159.stdout:(129/156): python3-logutils-0.3.5-21.el9.noarch 1.8 MB/s | 46 kB 00:00 2024-09-10T09:24:54.963 INFO:teuthology.orchestra.run.smithi159.stdout:(130/156): python3-more-itertools-8.12.0-2.el9. 2.3 MB/s | 79 kB 00:00 2024-09-10T09:24:54.988 INFO:teuthology.orchestra.run.smithi159.stdout:(131/156): python3-natsort-7.1.1-5.el9.noarch.r 2.3 MB/s | 58 kB 00:00 2024-09-10T09:24:55.021 INFO:teuthology.orchestra.run.smithi159.stdout:(132/156): python3-pecan-1.4.2-3.el9.noarch.rpm 8.0 MB/s | 272 kB 00:00 2024-09-10T09:24:55.034 INFO:teuthology.orchestra.run.smithi005.stdout:(60/156): librdkafka-1.6.1-102.el9.x86_64.rpm 2.2 MB/s | 662 kB 00:00 2024-09-10T09:24:55.051 INFO:teuthology.orchestra.run.smithi005.stdout:(61/156): libuuid-devel-2.37.4-20.el9.x86_64.rp 173 kB/s | 24 kB 00:00 2024-09-10T09:24:55.055 INFO:teuthology.orchestra.run.smithi159.stdout:(133/156): python3-kubernetes-26.1.0-2.el9.noar 4.7 MB/s | 1.0 MB 00:00 2024-09-10T09:24:55.080 INFO:teuthology.orchestra.run.smithi159.stdout:(134/156): python3-portend-3.1.0-2.el9.noarch.r 280 kB/s | 16 kB 00:00 2024-09-10T09:24:55.085 INFO:teuthology.orchestra.run.smithi005.stdout:(62/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 947 kB/s | 47 kB 00:00 2024-09-10T09:24:55.106 INFO:teuthology.orchestra.run.smithi159.stdout:(135/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.8 MB/s | 90 kB 00:00 2024-09-10T09:24:55.131 INFO:teuthology.orchestra.run.smithi159.stdout:(136/156): python3-pyperclip-1.8.2-2.el9.noarch 562 kB/s | 28 kB 00:00 2024-09-10T09:24:55.157 INFO:teuthology.orchestra.run.smithi159.stdout:(137/156): python3-repoze-lru-0.7-16.el9.noarch 611 kB/s | 31 kB 00:00 2024-09-10T09:24:55.168 INFO:teuthology.orchestra.run.smithi005.stdout:(63/156): lua-5.4.4-4.el9.x86_64.rpm 2.2 MB/s | 188 kB 00:00 2024-09-10T09:24:55.181 INFO:teuthology.orchestra.run.smithi159.stdout:(138/156): python3-rsa-4.9-2.el9.noarch.rpm 2.3 MB/s | 59 kB 00:00 2024-09-10T09:24:55.194 INFO:teuthology.orchestra.run.smithi005.stdout:(64/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.0 MB/s | 292 kB 00:00 2024-09-10T09:24:55.207 INFO:teuthology.orchestra.run.smithi159.stdout:(139/156): python3-routes-2.5.1-5.el9.noarch.rp 2.4 MB/s | 188 kB 00:00 2024-09-10T09:24:55.219 INFO:teuthology.orchestra.run.smithi005.stdout:(65/156): openblas-0.3.26-2.el9.x86_64.rpm 767 kB/s | 38 kB 00:00 2024-09-10T09:24:55.232 INFO:teuthology.orchestra.run.smithi159.stdout:(140/156): python3-tempora-5.0.0-2.el9.noarch.r 711 kB/s | 36 kB 00:00 2024-09-10T09:24:55.258 INFO:teuthology.orchestra.run.smithi159.stdout:(141/156): python3-typing-extensions-4.12.2-2.e 1.4 MB/s | 74 kB 00:00 2024-09-10T09:24:55.283 INFO:teuthology.orchestra.run.smithi159.stdout:(142/156): python3-websocket-client-1.2.3-2.el9 3.5 MB/s | 90 kB 00:00 2024-09-10T09:24:55.308 INFO:teuthology.orchestra.run.smithi159.stdout:(143/156): python3-webob-1.8.8-2.el9.noarch.rpm 3.0 MB/s | 230 kB 00:00 2024-09-10T09:24:55.325 INFO:teuthology.orchestra.run.smithi159.stdout:(144/156): python3-zc-lockfile-2.0-10.el9.noarc 1.2 MB/s | 20 kB 00:00 2024-09-10T09:24:55.350 INFO:teuthology.orchestra.run.smithi159.stdout:(145/156): re2-20211101-20.el9.x86_64.rpm 7.5 MB/s | 191 kB 00:00 2024-09-10T09:24:55.384 INFO:teuthology.orchestra.run.smithi159.stdout:(146/156): python3-werkzeug-2.0.3-3.el9.1.noarc 4.1 MB/s | 427 kB 00:00 2024-09-10T09:24:55.443 INFO:teuthology.orchestra.run.smithi159.stdout:(147/156): thrift-0.15.0-2.el9.x86_64.rpm 17 MB/s | 1.6 MB 00:00 2024-09-10T09:24:55.470 INFO:teuthology.orchestra.run.smithi005.stdout:(66/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 3.4 MB/s | 3.0 MB 00:00 2024-09-10T09:24:55.635 INFO:teuthology.orchestra.run.smithi159.stdout:(148/156): librados2-19.1.1-305.g1cf819b7.el9.x 14 MB/s | 3.4 MB 00:00 2024-09-10T09:24:55.718 INFO:teuthology.orchestra.run.smithi159.stdout:(149/156): librbd1-19.1.1-305.g1cf819b7.el9.x86 11 MB/s | 3.2 MB 00:00 2024-09-10T09:24:55.852 INFO:teuthology.orchestra.run.smithi159.stdout:(150/156): NetworkManager-1.49.90-1.el9.x86_64. 11 MB/s | 2.3 MB 00:00 2024-09-10T09:24:55.902 INFO:teuthology.orchestra.run.smithi159.stdout:(151/156): NetworkManager-libnm-1.49.90-1.el9.x 9.8 MB/s | 1.8 MB 00:00 2024-09-10T09:24:55.927 INFO:teuthology.orchestra.run.smithi159.stdout:(152/156): NetworkManager-team-1.49.90-1.el9.x8 541 kB/s | 41 kB 00:00 2024-09-10T09:24:55.961 INFO:teuthology.orchestra.run.smithi159.stdout:(153/156): NetworkManager-tui-1.49.90-1.el9.x86 4.2 MB/s | 249 kB 00:00 2024-09-10T09:24:55.961 INFO:teuthology.orchestra.run.smithi005.stdout:(67/156): openssl-devel-3.2.2-6.el9.x86_64.rpm 6.0 MB/s | 4.4 MB 00:00 2024-09-10T09:24:56.003 INFO:teuthology.orchestra.run.smithi159.stdout:(154/156): openssl-3.2.2-6.el9.x86_64.rpm 18 MB/s | 1.4 MB 00:00 2024-09-10T09:24:56.070 INFO:teuthology.orchestra.run.smithi159.stdout:(155/156): openssl-libs-3.2.2-6.el9.x86_64.rpm 22 MB/s | 2.4 MB 00:00 2024-09-10T09:24:56.087 INFO:teuthology.orchestra.run.smithi005.stdout:(68/156): postgresql-13.16-1.el9.x86_64.rpm 2.6 MB/s | 1.6 MB 00:00 2024-09-10T09:24:56.195 INFO:teuthology.orchestra.run.smithi005.stdout:(69/156): postgresql-contrib-13.16-1.el9.x86_64 3.8 MB/s | 910 kB 00:00 2024-09-10T09:24:56.271 INFO:teuthology.orchestra.run.smithi005.stdout:(70/156): openblas-openmp-0.3.26-2.el9.x86_64.r 4.6 MB/s | 4.9 MB 00:01 2024-09-10T09:24:56.296 INFO:teuthology.orchestra.run.smithi005.stdout:(71/156): postgresql-private-libs-13.16-1.el9.x 658 kB/s | 137 kB 00:00 2024-09-10T09:24:56.355 INFO:teuthology.orchestra.run.smithi005.stdout:(72/156): python3-attrs-20.3.0-7.el9.noarch.rpm 1.5 MB/s | 87 kB 00:00 2024-09-10T09:24:56.663 INFO:teuthology.orchestra.run.smithi005.stdout:(73/156): protobuf-3.14.0-13.el9.x86_64.rpm 2.8 MB/s | 1.0 MB 00:00 2024-09-10T09:24:56.714 INFO:teuthology.orchestra.run.smithi005.stdout:(74/156): python3-colorama-0.4.6-3.el9.noarch.r 1.2 MB/s | 61 kB 00:00 2024-09-10T09:24:56.822 INFO:teuthology.orchestra.run.smithi005.stdout:(75/156): python3-devel-3.9.19-8.el9.x86_64.rpm 2.2 MB/s | 245 kB 00:00 2024-09-10T09:24:56.931 INFO:teuthology.orchestra.run.smithi005.stdout:(76/156): python3-jinja2-2.11.3-6.el9.noarch.rp 2.2 MB/s | 248 kB 00:00 2024-09-10T09:24:57.006 INFO:teuthology.orchestra.run.smithi005.stdout:(77/156): python3-jsonpatch-1.21-16.el9.noarch. 342 kB/s | 26 kB 00:00 2024-09-10T09:24:57.115 INFO:teuthology.orchestra.run.smithi005.stdout:(78/156): postgresql-server-13.16-1.el9.x86_64. 6.3 MB/s | 5.8 MB 00:00 2024-09-10T09:24:57.140 INFO:teuthology.orchestra.run.smithi005.stdout:(79/156): python3-jsonpointer-2.0-4.el9.noarch. 144 kB/s | 19 kB 00:00 2024-09-10T09:24:57.173 INFO:teuthology.orchestra.run.smithi005.stdout:(80/156): python3-mako-1.1.4-6.el9.noarch.rpm 2.9 MB/s | 172 kB 00:00 2024-09-10T09:24:57.199 INFO:teuthology.orchestra.run.smithi005.stdout:(81/156): python3-markupsafe-1.1.1-12.el9.x86_6 594 kB/s | 35 kB 00:00 2024-09-10T09:24:57.308 INFO:teuthology.orchestra.run.smithi005.stdout:(82/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.3 MB/s | 250 kB 00:00 2024-09-10T09:24:57.395 INFO:teuthology.orchestra.run.smithi159.stdout:(156/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 3.9 MB/s | 16 MB 00:04 2024-09-10T09:24:57.399 INFO:teuthology.orchestra.run.smithi005.stdout:(83/156): python3-babel-2.9.1-2.el9.noarch.rpm 5.7 MB/s | 6.0 MB 00:01 2024-09-10T09:24:57.400 INFO:teuthology.orchestra.run.smithi159.stdout:-------------------------------------------------------------------------------- 2024-09-10T09:24:57.400 INFO:teuthology.orchestra.run.smithi159.stdout:Total 15 MB/s | 226 MB 00:15 2024-09-10T09:24:57.425 INFO:teuthology.orchestra.run.smithi005.stdout:(84/156): python3-oauthlib-3.1.1-5.el9.noarch.r 1.9 MB/s | 222 kB 00:00 2024-09-10T09:24:57.450 INFO:teuthology.orchestra.run.smithi005.stdout:(85/156): python3-packaging-20.9-5.el9.noarch.r 1.5 MB/s | 77 kB 00:00 2024-09-10T09:24:57.503 INFO:teuthology.orchestra.run.smithi005.stdout:(86/156): python3-prettytable-0.7.2-27.el9.noar 541 kB/s | 42 kB 00:00 2024-09-10T09:24:57.559 INFO:teuthology.orchestra.run.smithi005.stdout:(87/156): python3-protobuf-3.14.0-13.el9.noarch 2.4 MB/s | 269 kB 00:00 2024-09-10T09:24:57.584 INFO:teuthology.orchestra.run.smithi005.stdout:(88/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.9 MB/s | 159 kB 00:00 2024-09-10T09:24:57.635 INFO:teuthology.orchestra.run.smithi005.stdout:(89/156): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-09-10T09:24:57.660 INFO:teuthology.orchestra.run.smithi005.stdout:(90/156): python3-pyasn1-modules-0.4.8-6.el9.no 2.7 MB/s | 279 kB 00:00 2024-09-10T09:24:57.685 INFO:teuthology.orchestra.run.smithi005.stdout:(91/156): python3-requests-oauthlib-1.3.0-12.el 1.0 MB/s | 54 kB 00:00 2024-09-10T09:24:57.736 INFO:teuthology.orchestra.run.smithi005.stdout:(92/156): python3-toml-0.10.2-6.el9.noarch.rpm 838 kB/s | 42 kB 00:00 2024-09-10T09:24:57.794 INFO:teuthology.orchestra.run.smithi005.stdout:(93/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 757 kB/s | 44 kB 00:00 2024-09-10T09:24:57.878 INFO:teuthology.orchestra.run.smithi005.stdout:(94/156): qatlib-24.02.0-1.el9.x86_64.rpm 2.6 MB/s | 221 kB 00:00 2024-09-10T09:24:57.945 INFO:teuthology.orchestra.run.smithi005.stdout:(95/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 6.6 MB/s | 5.1 MB 00:00 2024-09-10T09:24:57.970 INFO:teuthology.orchestra.run.smithi005.stdout:(96/156): qatlib-service-24.02.0-1.el9.x86_64.r 392 kB/s | 36 kB 00:00 2024-09-10T09:24:57.995 INFO:teuthology.orchestra.run.smithi005.stdout:(97/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 948 kB/s | 47 kB 00:00 2024-09-10T09:24:58.045 INFO:teuthology.orchestra.run.smithi005.stdout:(98/156): uuid-1.6.2-55.el9.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2024-09-10T09:24:58.088 INFO:teuthology.orchestra.run.smithi005.stdout:(99/156): socat-1.7.4.1-6.el9.x86_64.rpm 2.5 MB/s | 304 kB 00:00 2024-09-10T09:24:58.113 INFO:teuthology.orchestra.run.smithi005.stdout:(100/156): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 1.4 MB/s | 96 kB 00:00 2024-09-10T09:24:58.138 INFO:teuthology.orchestra.run.smithi005.stdout:(101/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-09-10T09:24:58.172 INFO:teuthology.orchestra.run.smithi005.stdout:(102/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 768 kB/s | 45 kB 00:00 2024-09-10T09:24:58.631 INFO:teuthology.orchestra.run.smithi159.stdout:Running transaction check 2024-09-10T09:24:58.681 INFO:teuthology.orchestra.run.smithi005.stdout:(103/156): lua-devel-5.4.4-4.el9.x86_64.rpm 41 kB/s | 22 kB 00:00 2024-09-10T09:24:58.747 INFO:teuthology.orchestra.run.smithi005.stdout:(104/156): abseil-cpp-20211102.0-3.el9.x86_64.r 8.1 MB/s | 548 kB 00:00 2024-09-10T09:24:58.781 INFO:teuthology.orchestra.run.smithi159.stdout:Transaction check succeeded. 2024-09-10T09:24:58.781 INFO:teuthology.orchestra.run.smithi159.stdout:Running transaction test 2024-09-10T09:24:58.782 INFO:teuthology.orchestra.run.smithi005.stdout:(105/156): gperftools-libs-2.9.1-2.el9.x86_64.r 9.2 MB/s | 309 kB 00:00 2024-09-10T09:24:58.806 INFO:teuthology.orchestra.run.smithi005.stdout:(106/156): grpc-data-1.46.7-10.el9.noarch.rpm 780 kB/s | 19 kB 00:00 2024-09-10T09:24:58.932 INFO:teuthology.orchestra.run.smithi005.stdout:(107/156): libarrow-9.0.0-11.el9.x86_64.rpm 35 MB/s | 4.4 MB 00:00 2024-09-10T09:24:58.957 INFO:teuthology.orchestra.run.smithi005.stdout:(108/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 1.0 MB/s | 26 kB 00:00 2024-09-10T09:24:59.063 INFO:teuthology.orchestra.run.smithi005.stdout:(109/156): liboath-2.6.7-2.el9.x86_64.rpm 1.1 MB/s | 49 kB 00:00 2024-09-10T09:24:59.063 INFO:teuthology.orchestra.run.smithi005.stdout:(110/156): libunwind-1.6.2-1.el9.x86_64.rpm 2.7 MB/s | 67 kB 00:00 2024-09-10T09:24:59.063 INFO:teuthology.orchestra.run.smithi005.stdout:(111/156): luarocks-3.9.2-5.el9.noarch.rpm 4.5 MB/s | 151 kB 00:00 2024-09-10T09:24:59.092 INFO:teuthology.orchestra.run.smithi005.stdout:(112/156): protobuf-compiler-3.14.0-13.el9.x86_ 939 kB/s | 863 kB 00:00 2024-09-10T09:24:59.151 INFO:teuthology.orchestra.run.smithi005.stdout:(113/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 9.0 MB/s | 839 kB 00:00 2024-09-10T09:24:59.192 INFO:teuthology.orchestra.run.smithi005.stdout:(114/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.0 MB/s | 43 kB 00:00 2024-09-10T09:24:59.235 INFO:teuthology.orchestra.run.smithi005.stdout:(115/156): python3-cachetools-4.2.4-1.el9.noarc 777 kB/s | 32 kB 00:00 2024-09-10T09:24:59.277 INFO:teuthology.orchestra.run.smithi005.stdout:(116/156): python3-certifi-2023.05.07-4.el9.noa 339 kB/s | 14 kB 00:00 2024-09-10T09:24:59.310 INFO:teuthology.orchestra.run.smithi005.stdout:(117/156): python3-cheroot-8.6.0-4.el9.noarch.r 5.1 MB/s | 172 kB 00:00 2024-09-10T09:24:59.344 INFO:teuthology.orchestra.run.smithi005.stdout:(118/156): python3-asyncssh-2.13.2-5.el9.noarch 2.1 MB/s | 548 kB 00:00 2024-09-10T09:24:59.377 INFO:teuthology.orchestra.run.smithi005.stdout:(119/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 9.0 MB/s | 303 kB 00:00 2024-09-10T09:24:59.434 INFO:teuthology.orchestra.run.smithi005.stdout:(120/156): python3-cherrypy-18.6.1-2.el9.noarch 2.8 MB/s | 358 kB 00:00 2024-09-10T09:24:59.464 INFO:teuthology.orchestra.run.smithi005.stdout:(121/156): python3-google-auth-2.34.0-1.el9.noa 2.5 MB/s | 220 kB 00:00 2024-09-10T09:24:59.498 INFO:teuthology.orchestra.run.smithi005.stdout:(122/156): python3-grpcio-tools-1.46.7-10.el9.x 4.3 MB/s | 144 kB 00:00 2024-09-10T09:24:59.540 INFO:teuthology.orchestra.run.smithi005.stdout:(123/156): python3-jaraco-8.2.1-3.el9.noarch.rp 257 kB/s | 11 kB 00:00 2024-09-10T09:24:59.582 INFO:teuthology.orchestra.run.smithi005.stdout:(124/156): python3-jaraco-classes-3.2.1-5.el9.n 426 kB/s | 18 kB 00:00 2024-09-10T09:24:59.624 INFO:teuthology.orchestra.run.smithi005.stdout:(125/156): python3-jaraco-collections-3.0.0-8.e 556 kB/s | 23 kB 00:00 2024-09-10T09:24:59.666 INFO:teuthology.orchestra.run.smithi005.stdout:(126/156): python3-jaraco-functools-3.5.0-2.el9 470 kB/s | 19 kB 00:00 2024-09-10T09:24:59.708 INFO:teuthology.orchestra.run.smithi005.stdout:(127/156): python3-jaraco-text-3.2.0-6.el9.noar 473 kB/s | 20 kB 00:00 2024-09-10T09:24:59.766 INFO:teuthology.orchestra.run.smithi005.stdout:(128/156): python3-kubernetes-26.1.0-2.el9.noar 18 MB/s | 1.0 MB 00:00 2024-09-10T09:24:59.808 INFO:teuthology.orchestra.run.smithi005.stdout:(129/156): python3-logutils-0.3.5-21.el9.noarch 1.1 MB/s | 46 kB 00:00 2024-09-10T09:24:59.858 INFO:teuthology.orchestra.run.smithi005.stdout:(130/156): python3-grpcio-1.46.7-10.el9.x86_64. 4.8 MB/s | 2.0 MB 00:00 2024-09-10T09:24:59.884 INFO:teuthology.orchestra.run.smithi005.stdout:(131/156): python3-more-itertools-8.12.0-2.el9. 1.0 MB/s | 79 kB 00:00 2024-09-10T09:24:59.926 INFO:teuthology.orchestra.run.smithi005.stdout:(132/156): python3-natsort-7.1.1-5.el9.noarch.r 861 kB/s | 58 kB 00:00 2024-09-10T09:24:59.968 INFO:teuthology.orchestra.run.smithi005.stdout:(133/156): python3-portend-3.1.0-2.el9.noarch.r 397 kB/s | 16 kB 00:00 2024-09-10T09:24:59.993 INFO:teuthology.orchestra.run.smithi005.stdout:(134/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 3.5 MB/s | 90 kB 00:00 2024-09-10T09:25:00.035 INFO:teuthology.orchestra.run.smithi005.stdout:(135/156): python3-pyperclip-1.8.2-2.el9.noarch 680 kB/s | 28 kB 00:00 2024-09-10T09:25:00.061 INFO:teuthology.orchestra.run.smithi005.stdout:(136/156): python3-pecan-1.4.2-3.el9.noarch.rpm 1.5 MB/s | 272 kB 00:00 2024-09-10T09:25:00.103 INFO:teuthology.orchestra.run.smithi005.stdout:(137/156): python3-repoze-lru-0.7-16.el9.noarch 459 kB/s | 31 kB 00:00 2024-09-10T09:25:00.145 INFO:teuthology.orchestra.run.smithi005.stdout:(138/156): python3-rsa-4.9-2.el9.noarch.rpm 1.4 MB/s | 59 kB 00:00 2024-09-10T09:25:00.169 INFO:teuthology.orchestra.run.smithi005.stdout:(139/156): python3-routes-2.5.1-5.el9.noarch.rp 1.7 MB/s | 188 kB 00:00 2024-09-10T09:25:00.212 INFO:teuthology.orchestra.run.smithi005.stdout:(140/156): python3-tempora-5.0.0-2.el9.noarch.r 535 kB/s | 36 kB 00:00 2024-09-10T09:25:00.236 INFO:teuthology.orchestra.run.smithi102.stdout: Preparing : 1/1 2024-09-10T09:25:00.237 INFO:teuthology.orchestra.run.smithi005.stdout:(141/156): python3-typing-extensions-4.12.2-2.e 1.1 MB/s | 74 kB 00:00 2024-09-10T09:25:00.262 INFO:teuthology.orchestra.run.smithi005.stdout:(142/156): python3-websocket-client-1.2.3-2.el9 3.6 MB/s | 90 kB 00:00 2024-09-10T09:25:00.288 INFO:teuthology.orchestra.run.smithi005.stdout:(143/156): python3-webob-1.8.8-2.el9.noarch.rpm 3.0 MB/s | 230 kB 00:00 2024-09-10T09:25:00.319 INFO:teuthology.orchestra.run.smithi102.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/164 2024-09-10T09:25:00.330 INFO:teuthology.orchestra.run.smithi005.stdout:(144/156): python3-zc-lockfile-2.0-10.el9.noarc 479 kB/s | 20 kB 00:00 2024-09-10T09:25:00.388 INFO:teuthology.orchestra.run.smithi005.stdout:(145/156): re2-20211101-20.el9.x86_64.rpm 3.2 MB/s | 191 kB 00:00 2024-09-10T09:25:00.430 INFO:teuthology.orchestra.run.smithi005.stdout:(146/156): python3-werkzeug-2.0.3-3.el9.1.noarc 2.5 MB/s | 427 kB 00:00 2024-09-10T09:25:00.459 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-09-10T09:25:00.611 INFO:teuthology.orchestra.run.smithi159.stdout:Transaction test succeeded. 2024-09-10T09:25:00.611 INFO:teuthology.orchestra.run.smithi159.stdout:Running transaction 2024-09-10T09:25:00.678 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-09-10T09:25:00.721 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-09-10T09:25:00.746 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-09-10T09:25:00.773 INFO:teuthology.orchestra.run.smithi005.stdout:(147/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 5.2 MB/s | 16 MB 00:03 2024-09-10T09:25:00.789 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-09-10T09:25:00.831 INFO:teuthology.orchestra.run.smithi005.stdout:(148/156): thrift-0.15.0-2.el9.x86_64.rpm 3.6 MB/s | 1.6 MB 00:00 2024-09-10T09:25:01.119 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-09-10T09:25:01.230 INFO:teuthology.orchestra.run.smithi102.stdout: Upgrading : librados2-2:19.1.1-305.g1cf819b7.el9.x86_64 8/164 2024-09-10T09:25:01.353 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: librados2-2:19.1.1-305.g1cf819b7.el9.x86_64 8/164 2024-09-10T09:25:01.381 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : libcephfs2-2:19.1.1-305.g1cf819b7.el9.x86_64 9/164 2024-09-10T09:25:01.472 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: libcephfs2-2:19.1.1-305.g1cf819b7.el9.x86_64 9/164 2024-09-10T09:25:01.502 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-rados-2:19.1.1-305.g1cf819b7.el9.x86_64 10/164 2024-09-10T09:25:01.523 INFO:teuthology.orchestra.run.smithi005.stdout:(149/156): librados2-19.1.1-305.g1cf819b7.el9.x 3.1 MB/s | 3.4 MB 00:01 2024-09-10T09:25:01.649 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/164 2024-09-10T09:25:01.682 INFO:teuthology.orchestra.run.smithi005.stdout:(150/156): librbd1-19.1.1-305.g1cf819b7.el9.x86 3.5 MB/s | 3.2 MB 00:00 2024-09-10T09:25:01.699 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : postgresql-13.16-1.el9.x86_64 12/164 2024-09-10T09:25:01.727 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : libcephsqlite-2:19.1.1-305.g1cf819b7.el9.x86_64 13/164 2024-09-10T09:25:01.822 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: libcephsqlite-2:19.1.1-305.g1cf819b7.el9.x86_64 13/164 2024-09-10T09:25:01.845 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : libradosstriper1-2:19.1.1-305.g1cf819b7.el9.x86_ 14/164 2024-09-10T09:25:01.891 INFO:teuthology.orchestra.run.smithi005.stdout:(151/156): NetworkManager-team-1.49.90-1.el9.x8 196 kB/s | 41 kB 00:00 2024-09-10T09:25:01.891 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: libradosstriper1-2:19.1.1-305.g1cf819b7.el9.x86_ 14/164 2024-09-10T09:25:01.912 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-10T09:25:01.933 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-10T09:25:01.949 INFO:teuthology.orchestra.run.smithi005.stdout:(152/156): NetworkManager-1.49.90-1.el9.x86_64. 2.1 MB/s | 2.3 MB 00:01 2024-09-10T09:25:02.131 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-10T09:25:02.193 INFO:teuthology.orchestra.run.smithi005.stdout:(153/156): NetworkManager-tui-1.49.90-1.el9.x86 829 kB/s | 249 kB 00:00 2024-09-10T09:25:02.272 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-10T09:25:02.408 INFO:teuthology.orchestra.run.smithi005.stdout:(154/156): openssl-3.2.2-6.el9.x86_64.rpm 3.0 MB/s | 1.4 MB 00:00 2024-09-10T09:25:02.438 INFO:teuthology.orchestra.run.smithi102.stdout: Upgrading : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 17/164 2024-09-10T09:25:02.492 INFO:teuthology.orchestra.run.smithi005.stdout:(155/156): NetworkManager-libnm-1.49.90-1.el9.x 1.9 MB/s | 1.8 MB 00:00 2024-09-10T09:25:02.543 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-10T09:25:02.612 INFO:teuthology.orchestra.run.smithi102.stdout: Upgrading : NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-10T09:25:02.832 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-10T09:25:02.955 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : re2-1:20211101-20.el9.x86_64 19/164 2024-09-10T09:25:03.048 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-09-10T09:25:03.091 INFO:teuthology.orchestra.run.smithi005.stdout:(156/156): openssl-libs-3.2.2-6.el9.x86_64.rpm 2.7 MB/s | 2.4 MB 00:00 2024-09-10T09:25:03.092 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 21/164 2024-09-10T09:25:03.098 INFO:teuthology.orchestra.run.smithi005.stdout:-------------------------------------------------------------------------------- 2024-09-10T09:25:03.098 INFO:teuthology.orchestra.run.smithi005.stdout:Total 13 MB/s | 226 MB 00:17 2024-09-10T09:25:03.127 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/164 2024-09-10T09:25:03.209 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/164 2024-09-10T09:25:03.253 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/164 2024-09-10T09:25:03.285 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/164 2024-09-10T09:25:03.329 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/164 2024-09-10T09:25:03.361 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/164 2024-09-10T09:25:03.389 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/164 2024-09-10T09:25:03.458 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/164 2024-09-10T09:25:03.520 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/164 2024-09-10T09:25:03.550 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/164 2024-09-10T09:25:03.594 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/164 2024-09-10T09:25:03.670 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/164 2024-09-10T09:25:03.705 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-ceph-common-2:19.1.1-305.g1cf819b7.el9.x 34/164 2024-09-10T09:25:03.747 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-ceph-argparse-2:19.1.1-305.g1cf819b7.el9 35/164 2024-09-10T09:25:03.777 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-cephfs-2:19.1.1-305.g1cf819b7.el9.x86_64 36/164 2024-09-10T09:25:03.846 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/164 2024-09-10T09:25:03.891 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/164 2024-09-10T09:25:03.974 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/164 2024-09-10T09:25:04.107 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/164 2024-09-10T09:25:04.160 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/164 2024-09-10T09:25:04.691 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/164 2024-09-10T09:25:04.775 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/164 2024-09-10T09:25:04.814 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/164 2024-09-10T09:25:04.857 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/164 2024-09-10T09:25:04.894 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : librados-devel-2:19.1.1-305.g1cf819b7.el9.x86_64 46/164 2024-09-10T09:25:04.936 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/164 2024-09-10T09:25:05.119 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/164 2024-09-10T09:25:05.189 INFO:teuthology.orchestra.run.smithi102.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/164 2024-09-10T09:25:05.190 INFO:teuthology.orchestra.run.smithi005.stdout:Running transaction check 2024-09-10T09:25:05.244 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/164 2024-09-10T09:25:05.318 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 51/164 2024-09-10T09:25:05.348 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 52/164 2024-09-10T09:25:05.384 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 53/164 2024-09-10T09:25:05.391 INFO:teuthology.orchestra.run.smithi005.stdout:Transaction check succeeded. 2024-09-10T09:25:05.392 INFO:teuthology.orchestra.run.smithi005.stdout:Running transaction test 2024-09-10T09:25:05.431 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 54/164 2024-09-10T09:25:05.467 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 55/164 2024-09-10T09:25:05.510 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 56/164 2024-09-10T09:25:05.553 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 57/164 2024-09-10T09:25:05.688 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 58/164 2024-09-10T09:25:05.729 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 59/164 2024-09-10T09:25:05.849 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 60/164 2024-09-10T09:25:06.067 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-09-10T09:25:06.105 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 62/164 2024-09-10T09:25:06.145 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-09-10T09:25:06.170 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 64/164 2024-09-10T09:25:06.195 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-09-10T09:25:06.270 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-09-10T09:25:06.397 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-09-10T09:25:06.428 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-09-10T09:25:06.456 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-09-10T09:25:06.490 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-09-10T09:25:06.529 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-09-10T09:25:06.561 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-09-10T09:25:06.591 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-09-10T09:25:06.626 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-09-10T09:25:06.659 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-09-10T09:25:06.700 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-09-10T09:25:06.735 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-09-10T09:25:06.822 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-09-10T09:25:06.868 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-09-10T09:25:06.897 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-09-10T09:25:06.998 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-09-10T09:25:07.047 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 82/164 2024-09-10T09:25:07.165 INFO:teuthology.orchestra.run.smithi159.stdout: Preparing : 1/1 2024-09-10T09:25:07.307 INFO:teuthology.orchestra.run.smithi159.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/164 2024-09-10T09:25:07.354 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-09-10T09:25:07.407 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-09-10T09:25:07.441 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-09-10T09:25:07.466 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-09-10T09:25:07.509 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-09-10T09:25:07.765 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 83/164 2024-09-10T09:25:07.820 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 84/164 2024-09-10T09:25:07.842 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-09-10T09:25:07.842 INFO:teuthology.orchestra.run.smithi005.stdout:Transaction test succeeded. 2024-09-10T09:25:07.843 INFO:teuthology.orchestra.run.smithi005.stdout:Running transaction 2024-09-10T09:25:07.853 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 85/164 2024-09-10T09:25:07.953 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 86/164 2024-09-10T09:25:07.959 INFO:teuthology.orchestra.run.smithi159.stdout: Upgrading : librados2-2:19.1.1-305.g1cf819b7.el9.x86_64 8/164 2024-09-10T09:25:08.000 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 87/164 2024-09-10T09:25:08.045 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 88/164 2024-09-10T09:25:08.082 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: librados2-2:19.1.1-305.g1cf819b7.el9.x86_64 8/164 2024-09-10T09:25:08.110 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : libcephfs2-2:19.1.1-305.g1cf819b7.el9.x86_64 9/164 2024-09-10T09:25:08.132 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-09-10T09:25:08.164 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-09-10T09:25:08.212 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: libcephfs2-2:19.1.1-305.g1cf819b7.el9.x86_64 9/164 2024-09-10T09:25:08.239 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-rados-2:19.1.1-305.g1cf819b7.el9.x86_64 10/164 2024-09-10T09:25:08.357 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : openblas-0.3.26-2.el9.x86_64 91/164 2024-09-10T09:25:08.387 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 92/164 2024-09-10T09:25:08.400 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/164 2024-09-10T09:25:08.456 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-09-10T09:25:08.461 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : postgresql-13.16-1.el9.x86_64 12/164 2024-09-10T09:25:08.489 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : libcephsqlite-2:19.1.1-305.g1cf819b7.el9.x86_64 13/164 2024-09-10T09:25:08.577 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: libcephsqlite-2:19.1.1-305.g1cf819b7.el9.x86_64 13/164 2024-09-10T09:25:08.599 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : libradosstriper1-2:19.1.1-305.g1cf819b7.el9.x86_ 14/164 2024-09-10T09:25:08.645 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: libradosstriper1-2:19.1.1-305.g1cf819b7.el9.x86_ 14/164 2024-09-10T09:25:08.666 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-10T09:25:08.692 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-10T09:25:08.876 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-10T09:25:09.006 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-10T09:25:09.094 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-09-10T09:25:09.160 INFO:teuthology.orchestra.run.smithi159.stdout: Upgrading : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 17/164 2024-09-10T09:25:09.265 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-10T09:25:09.341 INFO:teuthology.orchestra.run.smithi159.stdout: Upgrading : NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-10T09:25:09.360 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-09-10T09:25:09.574 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-10T09:25:09.682 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : re2-1:20211101-20.el9.x86_64 19/164 2024-09-10T09:25:09.777 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-09-10T09:25:09.813 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 21/164 2024-09-10T09:25:09.848 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/164 2024-09-10T09:25:09.914 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/164 2024-09-10T09:25:09.957 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/164 2024-09-10T09:25:09.989 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/164 2024-09-10T09:25:10.032 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/164 2024-09-10T09:25:10.065 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/164 2024-09-10T09:25:10.101 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/164 2024-09-10T09:25:10.178 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/164 2024-09-10T09:25:10.240 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/164 2024-09-10T09:25:10.271 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/164 2024-09-10T09:25:10.315 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/164 2024-09-10T09:25:10.412 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/164 2024-09-10T09:25:10.590 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-09-10T09:25:10.668 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-09-10T09:25:10.686 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-ceph-common-2:19.1.1-305.g1cf819b7.el9.x 34/164 2024-09-10T09:25:10.695 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-09-10T09:25:10.729 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 99/164 2024-09-10T09:25:10.735 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-ceph-argparse-2:19.1.1-305.g1cf819b7.el9 35/164 2024-09-10T09:25:10.764 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-cephfs-2:19.1.1-305.g1cf819b7.el9.x86_64 36/164 2024-09-10T09:25:10.833 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/164 2024-09-10T09:25:10.887 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/164 2024-09-10T09:25:10.965 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/164 2024-09-10T09:25:11.040 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-09-10T09:25:11.084 INFO:teuthology.orchestra.run.smithi102.stdout: Upgrading : librbd1-2:19.1.1-305.g1cf819b7.el9.x86_64 101/164 2024-09-10T09:25:11.107 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/164 2024-09-10T09:25:11.159 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/164 2024-09-10T09:25:11.183 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: librbd1-2:19.1.1-305.g1cf819b7.el9.x86_64 101/164 2024-09-10T09:25:11.213 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-rbd-2:19.1.1-305.g1cf819b7.el9.x86_64 102/164 2024-09-10T09:25:11.249 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-09-10T09:25:11.693 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/164 2024-09-10T09:25:11.733 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-09-10T09:25:11.770 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : librgw2-2:19.1.1-305.g1cf819b7.el9.x86_64 105/164 2024-09-10T09:25:11.780 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/164 2024-09-10T09:25:11.818 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/164 2024-09-10T09:25:11.841 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: librgw2-2:19.1.1-305.g1cf819b7.el9.x86_64 105/164 2024-09-10T09:25:11.860 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/164 2024-09-10T09:25:11.872 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-rgw-2:19.1.1-305.g1cf819b7.el9.x86_64 106/164 2024-09-10T09:25:11.899 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : librados-devel-2:19.1.1-305.g1cf819b7.el9.x86_64 46/164 2024-09-10T09:25:11.941 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/164 2024-09-10T09:25:12.100 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/164 2024-09-10T09:25:12.160 INFO:teuthology.orchestra.run.smithi159.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/164 2024-09-10T09:25:12.214 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/164 2024-09-10T09:25:12.287 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 51/164 2024-09-10T09:25:12.328 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 52/164 2024-09-10T09:25:12.364 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 53/164 2024-09-10T09:25:12.486 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 54/164 2024-09-10T09:25:12.731 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 55/164 2024-09-10T09:25:12.775 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 56/164 2024-09-10T09:25:12.816 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 57/164 2024-09-10T09:25:12.955 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 58/164 2024-09-10T09:25:13.000 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 59/164 2024-09-10T09:25:13.049 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 60/164 2024-09-10T09:25:13.154 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-09-10T09:25:13.202 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 62/164 2024-09-10T09:25:13.241 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-09-10T09:25:13.267 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 64/164 2024-09-10T09:25:13.293 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-09-10T09:25:13.367 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-09-10T09:25:13.494 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-09-10T09:25:13.533 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-09-10T09:25:13.560 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-09-10T09:25:13.603 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-09-10T09:25:13.643 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-09-10T09:25:13.674 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-09-10T09:25:13.720 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-09-10T09:25:13.755 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-09-10T09:25:13.788 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-09-10T09:25:13.830 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-09-10T09:25:13.864 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-09-10T09:25:13.951 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-09-10T09:25:13.995 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-09-10T09:25:14.026 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-09-10T09:25:14.113 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: ceph-common-2:19.1.1-305.g1cf819b7.el9.x86_64 107/164 2024-09-10T09:25:14.130 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-09-10T09:25:14.143 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-common-2:19.1.1-305.g1cf819b7.el9.x86_64 107/164 2024-09-10T09:25:14.185 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 82/164 2024-09-10T09:25:14.690 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: ceph-common-2:19.1.1-305.g1cf819b7.el9.x86_64 107/164 2024-09-10T09:25:14.730 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-base-2:19.1.1-305.g1cf819b7.el9.x86_64 108/164 2024-09-10T09:25:14.752 INFO:teuthology.orchestra.run.smithi005.stdout: Preparing : 1/1 2024-09-10T09:25:14.821 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: ceph-base-2:19.1.1-305.g1cf819b7.el9.x86_64 108/164 2024-09-10T09:25:14.821 INFO:teuthology.orchestra.run.smithi102.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-10T09:25:14.822 INFO:teuthology.orchestra.run.smithi102.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-10T09:25:14.822 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-10T09:25:14.856 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-selinux-2:19.1.1-305.g1cf819b7.el9.x86_64 109/164 2024-09-10T09:25:14.913 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 83/164 2024-09-10T09:25:14.958 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 84/164 2024-09-10T09:25:14.972 INFO:teuthology.orchestra.run.smithi005.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/164 2024-09-10T09:25:14.992 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 85/164 2024-09-10T09:25:15.023 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-09-10T09:25:15.094 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 86/164 2024-09-10T09:25:15.097 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-09-10T09:25:15.141 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-09-10T09:25:15.155 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 87/164 2024-09-10T09:25:15.172 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-09-10T09:25:15.208 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 88/164 2024-09-10T09:25:15.231 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-09-10T09:25:15.297 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-09-10T09:25:15.353 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-09-10T09:25:15.539 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : openblas-0.3.26-2.el9.x86_64 91/164 2024-09-10T09:25:15.567 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 92/164 2024-09-10T09:25:15.622 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-09-10T09:25:15.645 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-09-10T09:25:15.731 INFO:teuthology.orchestra.run.smithi005.stdout: Upgrading : librados2-2:19.1.1-305.g1cf819b7.el9.x86_64 8/164 2024-09-10T09:25:15.913 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: librados2-2:19.1.1-305.g1cf819b7.el9.x86_64 8/164 2024-09-10T09:25:15.949 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : libcephfs2-2:19.1.1-305.g1cf819b7.el9.x86_64 9/164 2024-09-10T09:25:16.097 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: libcephfs2-2:19.1.1-305.g1cf819b7.el9.x86_64 9/164 2024-09-10T09:25:16.139 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-rados-2:19.1.1-305.g1cf819b7.el9.x86_64 10/164 2024-09-10T09:25:16.294 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-09-10T09:25:16.364 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/164 2024-09-10T09:25:16.408 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-09-10T09:25:16.427 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : postgresql-13.16-1.el9.x86_64 12/164 2024-09-10T09:25:16.462 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : libcephsqlite-2:19.1.1-305.g1cf819b7.el9.x86_64 13/164 2024-09-10T09:25:16.624 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: libcephsqlite-2:19.1.1-305.g1cf819b7.el9.x86_64 13/164 2024-09-10T09:25:16.663 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : libradosstriper1-2:19.1.1-305.g1cf819b7.el9.x86_ 14/164 2024-09-10T09:25:16.801 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: libradosstriper1-2:19.1.1-305.g1cf819b7.el9.x86_ 14/164 2024-09-10T09:25:16.848 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-10T09:25:16.889 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-10T09:25:17.116 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-10T09:25:17.303 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-10T09:25:17.479 INFO:teuthology.orchestra.run.smithi005.stdout: Upgrading : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 17/164 2024-09-10T09:25:17.597 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-10T09:25:17.651 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-09-10T09:25:17.706 INFO:teuthology.orchestra.run.smithi005.stdout: Upgrading : NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-10T09:25:17.740 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-09-10T09:25:17.767 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-09-10T09:25:17.801 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 99/164 2024-09-10T09:25:18.011 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-10T09:25:18.112 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-09-10T09:25:18.164 INFO:teuthology.orchestra.run.smithi159.stdout: Upgrading : librbd1-2:19.1.1-305.g1cf819b7.el9.x86_64 101/164 2024-09-10T09:25:18.169 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : re2-1:20211101-20.el9.x86_64 19/164 2024-09-10T09:25:18.263 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: librbd1-2:19.1.1-305.g1cf819b7.el9.x86_64 101/164 2024-09-10T09:25:18.287 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-09-10T09:25:18.302 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-rbd-2:19.1.1-305.g1cf819b7.el9.x86_64 102/164 2024-09-10T09:25:18.330 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 21/164 2024-09-10T09:25:18.347 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-09-10T09:25:18.375 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/164 2024-09-10T09:25:18.459 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/164 2024-09-10T09:25:18.512 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/164 2024-09-10T09:25:18.549 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/164 2024-09-10T09:25:18.613 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/164 2024-09-10T09:25:18.651 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/164 2024-09-10T09:25:18.684 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/164 2024-09-10T09:25:18.776 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/164 2024-09-10T09:25:18.829 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-09-10T09:25:18.846 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/164 2024-09-10T09:25:18.867 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : librgw2-2:19.1.1-305.g1cf819b7.el9.x86_64 105/164 2024-09-10T09:25:18.880 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/164 2024-09-10T09:25:18.927 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/164 2024-09-10T09:25:18.938 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: librgw2-2:19.1.1-305.g1cf819b7.el9.x86_64 105/164 2024-09-10T09:25:18.969 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-rgw-2:19.1.1-305.g1cf819b7.el9.x86_64 106/164 2024-09-10T09:25:19.019 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/164 2024-09-10T09:25:19.059 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-ceph-common-2:19.1.1-305.g1cf819b7.el9.x 34/164 2024-09-10T09:25:19.118 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-ceph-argparse-2:19.1.1-305.g1cf819b7.el9 35/164 2024-09-10T09:25:19.156 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-cephfs-2:19.1.1-305.g1cf819b7.el9.x86_64 36/164 2024-09-10T09:25:19.250 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/164 2024-09-10T09:25:19.317 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/164 2024-09-10T09:25:19.413 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/164 2024-09-10T09:25:19.584 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/164 2024-09-10T09:25:19.656 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/164 2024-09-10T09:25:20.358 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/164 2024-09-10T09:25:20.473 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/164 2024-09-10T09:25:20.573 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/164 2024-09-10T09:25:20.700 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/164 2024-09-10T09:25:20.748 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : librados-devel-2:19.1.1-305.g1cf819b7.el9.x86_64 46/164 2024-09-10T09:25:20.801 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/164 2024-09-10T09:25:21.006 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/164 2024-09-10T09:25:21.076 INFO:teuthology.orchestra.run.smithi005.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/164 2024-09-10T09:25:21.145 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/164 2024-09-10T09:25:21.234 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 51/164 2024-09-10T09:25:21.285 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 52/164 2024-09-10T09:25:21.300 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: ceph-common-2:19.1.1-305.g1cf819b7.el9.x86_64 107/164 2024-09-10T09:25:21.325 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 53/164 2024-09-10T09:25:21.340 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : ceph-common-2:19.1.1-305.g1cf819b7.el9.x86_64 107/164 2024-09-10T09:25:21.385 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 54/164 2024-09-10T09:25:21.425 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 55/164 2024-09-10T09:25:21.470 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 56/164 2024-09-10T09:25:21.520 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 57/164 2024-09-10T09:25:21.711 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 58/164 2024-09-10T09:25:21.761 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 59/164 2024-09-10T09:25:21.804 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 60/164 2024-09-10T09:25:21.887 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: ceph-common-2:19.1.1-305.g1cf819b7.el9.x86_64 107/164 2024-09-10T09:25:21.927 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : ceph-base-2:19.1.1-305.g1cf819b7.el9.x86_64 108/164 2024-09-10T09:25:21.935 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-09-10T09:25:21.994 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 62/164 2024-09-10T09:25:22.020 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: ceph-base-2:19.1.1-305.g1cf819b7.el9.x86_64 108/164 2024-09-10T09:25:22.020 INFO:teuthology.orchestra.run.smithi159.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-10T09:25:22.020 INFO:teuthology.orchestra.run.smithi159.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-10T09:25:22.020 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-10T09:25:22.036 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-09-10T09:25:22.053 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : ceph-selinux-2:19.1.1-305.g1cf819b7.el9.x86_64 109/164 2024-09-10T09:25:22.067 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 64/164 2024-09-10T09:25:22.091 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-09-10T09:25:22.219 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-09-10T09:25:22.393 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-09-10T09:25:22.458 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-09-10T09:25:22.494 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-09-10T09:25:22.533 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-09-10T09:25:22.597 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-09-10T09:25:22.643 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-09-10T09:25:22.679 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-09-10T09:25:22.715 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-09-10T09:25:22.747 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-09-10T09:25:22.795 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-09-10T09:25:22.826 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-09-10T09:25:22.931 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-09-10T09:25:22.994 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-09-10T09:25:23.030 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-09-10T09:25:23.159 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-09-10T09:25:23.227 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 82/164 2024-09-10T09:25:24.175 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 83/164 2024-09-10T09:25:24.229 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 84/164 2024-09-10T09:25:24.263 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 85/164 2024-09-10T09:25:24.399 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 86/164 2024-09-10T09:25:24.619 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 87/164 2024-09-10T09:25:24.619 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 88/164 2024-09-10T09:25:24.650 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-09-10T09:25:24.689 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-09-10T09:25:24.938 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : openblas-0.3.26-2.el9.x86_64 91/164 2024-09-10T09:25:24.970 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 92/164 2024-09-10T09:25:25.071 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-09-10T09:25:25.918 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-09-10T09:25:26.071 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-09-10T09:25:26.569 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: ceph-selinux-2:19.1.1-305.g1cf819b7.el9.x86_64 109/164 2024-09-10T09:25:26.570 INFO:teuthology.orchestra.run.smithi102.stdout:skipping the directory /sys 2024-09-10T09:25:26.570 INFO:teuthology.orchestra.run.smithi102.stdout:skipping the directory /proc 2024-09-10T09:25:26.570 INFO:teuthology.orchestra.run.smithi102.stdout:skipping the directory /mnt 2024-09-10T09:25:26.570 INFO:teuthology.orchestra.run.smithi102.stdout:skipping the directory /var/tmp 2024-09-10T09:25:26.570 INFO:teuthology.orchestra.run.smithi102.stdout:skipping the directory /home 2024-09-10T09:25:26.570 INFO:teuthology.orchestra.run.smithi102.stdout:skipping the directory /root 2024-09-10T09:25:26.570 INFO:teuthology.orchestra.run.smithi102.stdout:skipping the directory /tmp 2024-09-10T09:25:26.570 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-10T09:25:26.743 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-volume-2:19.1.1-305.g1cf819b7.el9.noarch 110/164 2024-09-10T09:25:26.759 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: ceph-volume-2:19.1.1-305.g1cf819b7.el9.noarch 110/164 2024-09-10T09:25:26.760 INFO:teuthology.orchestra.run.smithi102.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T09:25:26.760 INFO:teuthology.orchestra.run.smithi102.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-10T09:25:26.760 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-10T09:25:27.725 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-09-10T09:25:27.834 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-09-10T09:25:27.870 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-09-10T09:25:27.903 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 99/164 2024-09-10T09:25:28.222 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-osd-2:19.1.1-305.g1cf819b7.el9.x86_64 111/164 2024-09-10T09:25:28.249 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: ceph-osd-2:19.1.1-305.g1cf819b7.el9.x86_64 111/164 2024-09-10T09:25:28.249 INFO:teuthology.orchestra.run.smithi102.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T09:25:28.249 INFO:teuthology.orchestra.run.smithi102.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-10T09:25:28.249 INFO:teuthology.orchestra.run.smithi102.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-10T09:25:28.249 INFO:teuthology.orchestra.run.smithi102.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-10T09:25:28.249 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-10T09:25:28.259 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-09-10T09:25:28.314 INFO:teuthology.orchestra.run.smithi005.stdout: Upgrading : librbd1-2:19.1.1-305.g1cf819b7.el9.x86_64 101/164 2024-09-10T09:25:28.481 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-mds-2:19.1.1-305.g1cf819b7.el9.x86_64 112/164 2024-09-10T09:25:28.486 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: librbd1-2:19.1.1-305.g1cf819b7.el9.x86_64 101/164 2024-09-10T09:25:28.505 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: ceph-mds-2:19.1.1-305.g1cf819b7.el9.x86_64 112/164 2024-09-10T09:25:28.505 INFO:teuthology.orchestra.run.smithi102.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T09:25:28.505 INFO:teuthology.orchestra.run.smithi102.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-10T09:25:28.505 INFO:teuthology.orchestra.run.smithi102.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-10T09:25:28.505 INFO:teuthology.orchestra.run.smithi102.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-10T09:25:28.505 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-10T09:25:28.529 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-rbd-2:19.1.1-305.g1cf819b7.el9.x86_64 102/164 2024-09-10T09:25:28.586 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-09-10T09:25:28.940 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-mon-2:19.1.1-305.g1cf819b7.el9.x86_64 113/164 2024-09-10T09:25:28.963 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: ceph-mon-2:19.1.1-305.g1cf819b7.el9.x86_64 113/164 2024-09-10T09:25:28.963 INFO:teuthology.orchestra.run.smithi102.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T09:25:28.963 INFO:teuthology.orchestra.run.smithi102.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-10T09:25:28.963 INFO:teuthology.orchestra.run.smithi102.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-10T09:25:28.964 INFO:teuthology.orchestra.run.smithi102.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-10T09:25:28.964 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-10T09:25:29.078 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-09-10T09:25:29.135 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-09-10T09:25:29.139 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-09-10T09:25:29.184 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : librgw2-2:19.1.1-305.g1cf819b7.el9.x86_64 105/164 2024-09-10T09:25:29.189 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-09-10T09:25:29.353 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: librgw2-2:19.1.1-305.g1cf819b7.el9.x86_64 105/164 2024-09-10T09:25:29.375 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-09-10T09:25:29.386 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-rgw-2:19.1.1-305.g1cf819b7.el9.x86_64 106/164 2024-09-10T09:25:29.423 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-09-10T09:25:29.492 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-09-10T09:25:29.581 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-09-10T09:25:29.711 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-09-10T09:25:29.774 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-09-10T09:25:29.814 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-09-10T09:25:29.842 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : fuse-2.9.9-16.el9.x86_64 124/164 2024-09-10T09:25:30.363 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: cephadm-2:19.1.1-305.g1cf819b7.el9.noarch 125/164 2024-09-10T09:25:30.391 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : cephadm-2:19.1.1-305.g1cf819b7.el9.noarch 125/164 2024-09-10T09:25:30.422 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-prometheus-alerts-2:19.1.1-305.g1cf819b7.el 126/164 2024-09-10T09:25:30.495 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-grafana-dashboards-2:19.1.1-305.g1cf819b7.e 127/164 2024-09-10T09:25:30.526 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-mgr-cephadm-2:19.1.1-305.g1cf819b7.el9.noar 128/164 2024-09-10T09:25:31.571 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.1.1-305.g1cf819b7.el9.noar 128/164 2024-09-10T09:25:31.607 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-mgr-dashboard-2:19.1.1-305.g1cf819b7.el9.no 129/164 2024-09-10T09:25:32.287 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: ceph-common-2:19.1.1-305.g1cf819b7.el9.x86_64 107/164 2024-09-10T09:25:32.333 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-common-2:19.1.1-305.g1cf819b7.el9.x86_64 107/164 2024-09-10T09:25:32.570 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.1.1-305.g1cf819b7.el9.no 129/164 2024-09-10T09:25:32.599 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-mgr-diskprediction-local-2:19.1.1-305.g1cf8 130/164 2024-09-10T09:25:32.703 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.1.1-305.g1cf8 130/164 2024-09-10T09:25:32.860 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-mgr-modules-core-2:19.1.1-305.g1cf819b7.el9 131/164 2024-09-10T09:25:32.893 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-mgr-2:19.1.1-305.g1cf819b7.el9.x86_64 132/164 2024-09-10T09:25:32.917 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: ceph-mgr-2:19.1.1-305.g1cf819b7.el9.x86_64 132/164 2024-09-10T09:25:32.917 INFO:teuthology.orchestra.run.smithi102.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T09:25:32.917 INFO:teuthology.orchestra.run.smithi102.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-10T09:25:32.918 INFO:teuthology.orchestra.run.smithi102.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-10T09:25:32.918 INFO:teuthology.orchestra.run.smithi102.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-10T09:25:32.918 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-10T09:25:32.969 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-mgr-rook-2:19.1.1-305.g1cf819b7.el9.noarch 133/164 2024-09-10T09:25:32.981 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: ceph-mgr-rook-2:19.1.1-305.g1cf819b7.el9.noarch 133/164 2024-09-10T09:25:32.994 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: ceph-common-2:19.1.1-305.g1cf819b7.el9.x86_64 107/164 2024-09-10T09:25:33.045 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-base-2:19.1.1-305.g1cf819b7.el9.x86_64 108/164 2024-09-10T09:25:33.077 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-2:19.1.1-305.g1cf819b7.el9.x86_64 134/164 2024-09-10T09:25:33.134 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-fuse-2:19.1.1-305.g1cf819b7.el9.x86_64 135/164 2024-09-10T09:25:33.219 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: ceph-base-2:19.1.1-305.g1cf819b7.el9.x86_64 108/164 2024-09-10T09:25:33.219 INFO:teuthology.orchestra.run.smithi005.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-10T09:25:33.219 INFO:teuthology.orchestra.run.smithi005.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-10T09:25:33.219 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-10T09:25:33.262 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-selinux-2:19.1.1-305.g1cf819b7.el9.x86_64 109/164 2024-09-10T09:25:33.869 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: ceph-selinux-2:19.1.1-305.g1cf819b7.el9.x86_64 109/164 2024-09-10T09:25:33.869 INFO:teuthology.orchestra.run.smithi159.stdout:skipping the directory /sys 2024-09-10T09:25:33.869 INFO:teuthology.orchestra.run.smithi159.stdout:skipping the directory /proc 2024-09-10T09:25:33.869 INFO:teuthology.orchestra.run.smithi159.stdout:skipping the directory /mnt 2024-09-10T09:25:33.869 INFO:teuthology.orchestra.run.smithi159.stdout:skipping the directory /var/tmp 2024-09-10T09:25:33.869 INFO:teuthology.orchestra.run.smithi159.stdout:skipping the directory /home 2024-09-10T09:25:33.870 INFO:teuthology.orchestra.run.smithi159.stdout:skipping the directory /root 2024-09-10T09:25:33.870 INFO:teuthology.orchestra.run.smithi159.stdout:skipping the directory /tmp 2024-09-10T09:25:33.870 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-10T09:25:34.040 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : ceph-volume-2:19.1.1-305.g1cf819b7.el9.noarch 110/164 2024-09-10T09:25:34.057 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: ceph-volume-2:19.1.1-305.g1cf819b7.el9.noarch 110/164 2024-09-10T09:25:34.057 INFO:teuthology.orchestra.run.smithi159.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T09:25:34.057 INFO:teuthology.orchestra.run.smithi159.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-10T09:25:34.057 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-10T09:25:34.066 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : libtool-2.4.6-46.el9.x86_64 136/164 2024-09-10T09:25:34.105 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-radosgw-2:19.1.1-305.g1cf819b7.el9.x86_64 137/164 2024-09-10T09:25:34.129 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: ceph-radosgw-2:19.1.1-305.g1cf819b7.el9.x86_64 137/164 2024-09-10T09:25:34.129 INFO:teuthology.orchestra.run.smithi102.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T09:25:34.129 INFO:teuthology.orchestra.run.smithi102.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-10T09:25:34.130 INFO:teuthology.orchestra.run.smithi102.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-10T09:25:34.130 INFO:teuthology.orchestra.run.smithi102.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-10T09:25:34.130 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-10T09:25:34.172 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-immutable-object-cache-2:19.1.1-305.g1cf819 138/164 2024-09-10T09:25:34.196 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.1.1-305.g1cf819 138/164 2024-09-10T09:25:34.196 INFO:teuthology.orchestra.run.smithi102.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T09:25:34.196 INFO:teuthology.orchestra.run.smithi102.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-10T09:25:34.196 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-10T09:25:34.248 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : cephfs-mirror-2:19.1.1-305.g1cf819b7.el9.x86_64 139/164 2024-09-10T09:25:34.271 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: cephfs-mirror-2:19.1.1-305.g1cf819b7.el9.x86_64 139/164 2024-09-10T09:25:34.271 INFO:teuthology.orchestra.run.smithi102.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T09:25:34.271 INFO:teuthology.orchestra.run.smithi102.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-10T09:25:34.272 INFO:teuthology.orchestra.run.smithi102.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-10T09:25:34.272 INFO:teuthology.orchestra.run.smithi102.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-10T09:25:34.272 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-10T09:25:34.573 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : rbd-mirror-2:19.1.1-305.g1cf819b7.el9.x86_64 140/164 2024-09-10T09:25:34.597 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: rbd-mirror-2:19.1.1-305.g1cf819b7.el9.x86_64 140/164 2024-09-10T09:25:34.598 INFO:teuthology.orchestra.run.smithi102.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T09:25:34.598 INFO:teuthology.orchestra.run.smithi102.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-10T09:25:34.598 INFO:teuthology.orchestra.run.smithi102.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-10T09:25:34.598 INFO:teuthology.orchestra.run.smithi102.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-10T09:25:34.598 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-10T09:25:35.527 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : ceph-osd-2:19.1.1-305.g1cf819b7.el9.x86_64 111/164 2024-09-10T09:25:35.554 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: ceph-osd-2:19.1.1-305.g1cf819b7.el9.x86_64 111/164 2024-09-10T09:25:35.554 INFO:teuthology.orchestra.run.smithi159.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T09:25:35.554 INFO:teuthology.orchestra.run.smithi159.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-10T09:25:35.554 INFO:teuthology.orchestra.run.smithi159.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-10T09:25:35.554 INFO:teuthology.orchestra.run.smithi159.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-10T09:25:35.555 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-10T09:25:35.785 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : ceph-mds-2:19.1.1-305.g1cf819b7.el9.x86_64 112/164 2024-09-10T09:25:35.809 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: ceph-mds-2:19.1.1-305.g1cf819b7.el9.x86_64 112/164 2024-09-10T09:25:35.809 INFO:teuthology.orchestra.run.smithi159.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T09:25:35.809 INFO:teuthology.orchestra.run.smithi159.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-10T09:25:35.809 INFO:teuthology.orchestra.run.smithi159.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-10T09:25:35.810 INFO:teuthology.orchestra.run.smithi159.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-10T09:25:35.810 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-10T09:25:36.245 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : ceph-mon-2:19.1.1-305.g1cf819b7.el9.x86_64 113/164 2024-09-10T09:25:36.268 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: ceph-mon-2:19.1.1-305.g1cf819b7.el9.x86_64 113/164 2024-09-10T09:25:36.268 INFO:teuthology.orchestra.run.smithi159.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T09:25:36.268 INFO:teuthology.orchestra.run.smithi159.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-10T09:25:36.269 INFO:teuthology.orchestra.run.smithi159.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-10T09:25:36.269 INFO:teuthology.orchestra.run.smithi159.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-10T09:25:36.269 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-10T09:25:36.383 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-09-10T09:25:36.440 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-09-10T09:25:36.495 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-09-10T09:25:36.683 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-09-10T09:25:36.737 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-09-10T09:25:36.816 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-09-10T09:25:36.904 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-09-10T09:25:37.045 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-09-10T09:25:37.121 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-09-10T09:25:37.161 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-09-10T09:25:37.189 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : fuse-2.9.9-16.el9.x86_64 124/164 2024-09-10T09:25:37.668 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: cephadm-2:19.1.1-305.g1cf819b7.el9.noarch 125/164 2024-09-10T09:25:37.696 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : cephadm-2:19.1.1-305.g1cf819b7.el9.noarch 125/164 2024-09-10T09:25:37.727 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : ceph-prometheus-alerts-2:19.1.1-305.g1cf819b7.el 126/164 2024-09-10T09:25:37.789 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : ceph-grafana-dashboards-2:19.1.1-305.g1cf819b7.e 127/164 2024-09-10T09:25:37.827 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : ceph-mgr-cephadm-2:19.1.1-305.g1cf819b7.el9.noar 128/164 2024-09-10T09:25:38.872 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.1.1-305.g1cf819b7.el9.noar 128/164 2024-09-10T09:25:38.912 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : ceph-mgr-dashboard-2:19.1.1-305.g1cf819b7.el9.no 129/164 2024-09-10T09:25:39.136 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-test-2:19.1.1-305.g1cf819b7.el9.x86_64 141/164 2024-09-10T09:25:39.177 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : rbd-fuse-2:19.1.1-305.g1cf819b7.el9.x86_64 142/164 2024-09-10T09:25:39.218 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : rbd-nbd-2:19.1.1-305.g1cf819b7.el9.x86_64 143/164 2024-09-10T09:25:39.259 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 144/164 2024-09-10T09:25:39.286 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 145/164 2024-09-10T09:25:39.464 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : cephfs-shell-2:19.1.1-305.g1cf819b7.el9.x86_64 146/164 2024-09-10T09:25:39.508 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 147/164 2024-09-10T09:25:39.538 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : libcephfs-devel-2:19.1.1-305.g1cf819b7.el9.x86_6 148/164 2024-09-10T09:25:39.568 INFO:teuthology.orchestra.run.smithi102.stdout: Upgrading : NetworkManager-team-1:1.49.90-1.el9.x86_64 149/164 2024-09-10T09:25:39.614 INFO:teuthology.orchestra.run.smithi102.stdout: Upgrading : NetworkManager-tui-1:1.49.90-1.el9.x86_64 150/164 2024-09-10T09:25:39.877 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.1.1-305.g1cf819b7.el9.no 129/164 2024-09-10T09:25:39.904 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : ceph-mgr-diskprediction-local-2:19.1.1-305.g1cf8 130/164 2024-09-10T09:25:40.007 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.1.1-305.g1cf8 130/164 2024-09-10T09:25:40.168 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : ceph-mgr-modules-core-2:19.1.1-305.g1cf819b7.el9 131/164 2024-09-10T09:25:40.207 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : ceph-mgr-2:19.1.1-305.g1cf819b7.el9.x86_64 132/164 2024-09-10T09:25:40.216 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-10T09:25:40.231 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: ceph-mgr-2:19.1.1-305.g1cf819b7.el9.x86_64 132/164 2024-09-10T09:25:40.232 INFO:teuthology.orchestra.run.smithi159.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T09:25:40.232 INFO:teuthology.orchestra.run.smithi159.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-10T09:25:40.232 INFO:teuthology.orchestra.run.smithi159.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-10T09:25:40.232 INFO:teuthology.orchestra.run.smithi159.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-10T09:25:40.232 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-10T09:25:40.262 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-10T09:25:40.282 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : ceph-mgr-rook-2:19.1.1-305.g1cf819b7.el9.noarch 133/164 2024-09-10T09:25:40.283 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-10T09:25:40.295 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: ceph-mgr-rook-2:19.1.1-305.g1cf819b7.el9.noarch 133/164 2024-09-10T09:25:40.392 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : ceph-2:19.1.1-305.g1cf819b7.el9.x86_64 134/164 2024-09-10T09:25:40.439 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : ceph-fuse-2:19.1.1-305.g1cf819b7.el9.x86_64 135/164 2024-09-10T09:25:41.101 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : cephfs-top-2:19.1.1-305.g1cf819b7.el9.noarch 152/164 2024-09-10T09:25:41.184 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 153/164 2024-09-10T09:25:41.233 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-09-10T09:25:41.322 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-09-10T09:25:41.374 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : libtool-2.4.6-46.el9.x86_64 136/164 2024-09-10T09:25:41.387 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-09-10T09:25:41.412 INFO:teuthology.orchestra.run.smithi102.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 157/164 2024-09-10T09:25:41.419 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : ceph-radosgw-2:19.1.1-305.g1cf819b7.el9.x86_64 137/164 2024-09-10T09:25:41.443 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: ceph-radosgw-2:19.1.1-305.g1cf819b7.el9.x86_64 137/164 2024-09-10T09:25:41.443 INFO:teuthology.orchestra.run.smithi159.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T09:25:41.443 INFO:teuthology.orchestra.run.smithi159.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-10T09:25:41.443 INFO:teuthology.orchestra.run.smithi159.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-10T09:25:41.443 INFO:teuthology.orchestra.run.smithi159.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-10T09:25:41.444 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-10T09:25:41.447 INFO:teuthology.orchestra.run.smithi102.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 158/164 2024-09-10T09:25:41.447 INFO:teuthology.orchestra.run.smithi102.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-10T09:25:41.486 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : ceph-immutable-object-cache-2:19.1.1-305.g1cf819 138/164 2024-09-10T09:25:41.509 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.1.1-305.g1cf819 138/164 2024-09-10T09:25:41.509 INFO:teuthology.orchestra.run.smithi159.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T09:25:41.509 INFO:teuthology.orchestra.run.smithi159.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-10T09:25:41.509 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-10T09:25:41.522 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-10T09:25:41.522 INFO:teuthology.orchestra.run.smithi102.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-10T09:25:41.561 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : cephfs-mirror-2:19.1.1-305.g1cf819b7.el9.x86_64 139/164 2024-09-10T09:25:41.585 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: cephfs-mirror-2:19.1.1-305.g1cf819b7.el9.x86_64 139/164 2024-09-10T09:25:41.585 INFO:teuthology.orchestra.run.smithi159.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T09:25:41.586 INFO:teuthology.orchestra.run.smithi159.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-10T09:25:41.586 INFO:teuthology.orchestra.run.smithi159.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-10T09:25:41.586 INFO:teuthology.orchestra.run.smithi159.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-10T09:25:41.586 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-10T09:25:41.590 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-10T09:25:41.615 INFO:teuthology.orchestra.run.smithi102.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 161/164 2024-09-10T09:25:41.620 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-10T09:25:41.621 INFO:teuthology.orchestra.run.smithi102.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-10T09:25:41.808 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-10T09:25:41.808 INFO:teuthology.orchestra.run.smithi102.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-10T09:25:41.887 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : rbd-mirror-2:19.1.1-305.g1cf819b7.el9.x86_64 140/164 2024-09-10T09:25:41.892 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-10T09:25:41.911 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: rbd-mirror-2:19.1.1-305.g1cf819b7.el9.x86_64 140/164 2024-09-10T09:25:41.911 INFO:teuthology.orchestra.run.smithi159.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T09:25:41.911 INFO:teuthology.orchestra.run.smithi159.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-10T09:25:41.912 INFO:teuthology.orchestra.run.smithi159.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-10T09:25:41.912 INFO:teuthology.orchestra.run.smithi159.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-10T09:25:41.912 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-10T09:25:41.955 INFO:teuthology.orchestra.run.smithi102.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-10T09:25:46.476 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: ceph-selinux-2:19.1.1-305.g1cf819b7.el9.x86_64 109/164 2024-09-10T09:25:46.476 INFO:teuthology.orchestra.run.smithi005.stdout:skipping the directory /sys 2024-09-10T09:25:46.477 INFO:teuthology.orchestra.run.smithi005.stdout:skipping the directory /proc 2024-09-10T09:25:46.477 INFO:teuthology.orchestra.run.smithi005.stdout:skipping the directory /mnt 2024-09-10T09:25:46.477 INFO:teuthology.orchestra.run.smithi005.stdout:skipping the directory /var/tmp 2024-09-10T09:25:46.477 INFO:teuthology.orchestra.run.smithi005.stdout:skipping the directory /home 2024-09-10T09:25:46.477 INFO:teuthology.orchestra.run.smithi005.stdout:skipping the directory /root 2024-09-10T09:25:46.477 INFO:teuthology.orchestra.run.smithi005.stdout:skipping the directory /tmp 2024-09-10T09:25:46.477 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-10T09:25:46.500 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : ceph-test-2:19.1.1-305.g1cf819b7.el9.x86_64 141/164 2024-09-10T09:25:46.541 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : rbd-fuse-2:19.1.1-305.g1cf819b7.el9.x86_64 142/164 2024-09-10T09:25:46.582 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : rbd-nbd-2:19.1.1-305.g1cf819b7.el9.x86_64 143/164 2024-09-10T09:25:46.614 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 144/164 2024-09-10T09:25:46.649 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 145/164 2024-09-10T09:25:46.693 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-volume-2:19.1.1-305.g1cf819b7.el9.noarch 110/164 2024-09-10T09:25:46.729 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: ceph-volume-2:19.1.1-305.g1cf819b7.el9.noarch 110/164 2024-09-10T09:25:46.729 INFO:teuthology.orchestra.run.smithi005.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T09:25:46.729 INFO:teuthology.orchestra.run.smithi005.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-10T09:25:46.730 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-10T09:25:46.840 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : cephfs-shell-2:19.1.1-305.g1cf819b7.el9.x86_64 146/164 2024-09-10T09:25:46.880 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 147/164 2024-09-10T09:25:46.910 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : libcephfs-devel-2:19.1.1-305.g1cf819b7.el9.x86_6 148/164 2024-09-10T09:25:46.939 INFO:teuthology.orchestra.run.smithi159.stdout: Upgrading : NetworkManager-team-1:1.49.90-1.el9.x86_64 149/164 2024-09-10T09:25:46.977 INFO:teuthology.orchestra.run.smithi159.stdout: Upgrading : NetworkManager-tui-1:1.49.90-1.el9.x86_64 150/164 2024-09-10T09:25:47.576 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-10T09:25:47.576 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-2:19.1.1-305.g1cf819b7.el9.x86_64 1/164 2024-09-10T09:25:47.576 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-base-2:19.1.1-305.g1cf819b7.el9.x86_64 2/164 2024-09-10T09:25:47.576 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-common-2:19.1.1-305.g1cf819b7.el9.x86_64 3/164 2024-09-10T09:25:47.576 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-fuse-2:19.1.1-305.g1cf819b7.el9.x86_64 4/164 2024-09-10T09:25:47.576 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-immutable-object-cache-2:19.1.1-305.g1cf819 5/164 2024-09-10T09:25:47.576 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-mds-2:19.1.1-305.g1cf819b7.el9.x86_64 6/164 2024-09-10T09:25:47.576 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-mgr-2:19.1.1-305.g1cf819b7.el9.x86_64 7/164 2024-09-10T09:25:47.577 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-mon-2:19.1.1-305.g1cf819b7.el9.x86_64 8/164 2024-09-10T09:25:47.577 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-osd-2:19.1.1-305.g1cf819b7.el9.x86_64 9/164 2024-09-10T09:25:47.577 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-radosgw-2:19.1.1-305.g1cf819b7.el9.x86_64 10/164 2024-09-10T09:25:47.577 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-selinux-2:19.1.1-305.g1cf819b7.el9.x86_64 11/164 2024-09-10T09:25:47.577 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-test-2:19.1.1-305.g1cf819b7.el9.x86_64 12/164 2024-09-10T09:25:47.577 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : cephfs-mirror-2:19.1.1-305.g1cf819b7.el9.x86_64 13/164 2024-09-10T09:25:47.577 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : cephfs-shell-2:19.1.1-305.g1cf819b7.el9.x86_64 14/164 2024-09-10T09:25:47.577 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : libcephfs-devel-2:19.1.1-305.g1cf819b7.el9.x86_6 15/164 2024-09-10T09:25:47.577 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : libcephfs2-2:19.1.1-305.g1cf819b7.el9.x86_64 16/164 2024-09-10T09:25:47.577 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : libcephsqlite-2:19.1.1-305.g1cf819b7.el9.x86_64 17/164 2024-09-10T09:25:47.577 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : librados-devel-2:19.1.1-305.g1cf819b7.el9.x86_64 18/164 2024-09-10T09:25:47.577 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : libradosstriper1-2:19.1.1-305.g1cf819b7.el9.x86_ 19/164 2024-09-10T09:25:47.577 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : librgw2-2:19.1.1-305.g1cf819b7.el9.x86_64 20/164 2024-09-10T09:25:47.578 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-ceph-argparse-2:19.1.1-305.g1cf819b7.el9 21/164 2024-09-10T09:25:47.578 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-ceph-common-2:19.1.1-305.g1cf819b7.el9.x 22/164 2024-09-10T09:25:47.578 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-cephfs-2:19.1.1-305.g1cf819b7.el9.x86_64 23/164 2024-09-10T09:25:47.578 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-rados-2:19.1.1-305.g1cf819b7.el9.x86_64 24/164 2024-09-10T09:25:47.578 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-rbd-2:19.1.1-305.g1cf819b7.el9.x86_64 25/164 2024-09-10T09:25:47.578 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-rgw-2:19.1.1-305.g1cf819b7.el9.x86_64 26/164 2024-09-10T09:25:47.578 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : rbd-fuse-2:19.1.1-305.g1cf819b7.el9.x86_64 27/164 2024-09-10T09:25:47.590 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : rbd-mirror-2:19.1.1-305.g1cf819b7.el9.x86_64 28/164 2024-09-10T09:25:47.590 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : rbd-nbd-2:19.1.1-305.g1cf819b7.el9.x86_64 29/164 2024-09-10T09:25:47.590 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-grafana-dashboards-2:19.1.1-305.g1cf819b7.e 30/164 2024-09-10T09:25:47.590 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-mgr-cephadm-2:19.1.1-305.g1cf819b7.el9.noar 31/164 2024-09-10T09:25:47.590 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-mgr-dashboard-2:19.1.1-305.g1cf819b7.el9.no 32/164 2024-09-10T09:25:47.590 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.1.1-305.g1cf8 33/164 2024-09-10T09:25:47.591 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-mgr-modules-core-2:19.1.1-305.g1cf819b7.el9 34/164 2024-09-10T09:25:47.591 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-mgr-rook-2:19.1.1-305.g1cf819b7.el9.noarch 35/164 2024-09-10T09:25:47.591 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-prometheus-alerts-2:19.1.1-305.g1cf819b7.el 36/164 2024-09-10T09:25:47.591 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-volume-2:19.1.1-305.g1cf819b7.el9.noarch 37/164 2024-09-10T09:25:47.591 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : cephadm-2:19.1.1-305.g1cf819b7.el9.noarch 38/164 2024-09-10T09:25:47.591 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : cephfs-top-2:19.1.1-305.g1cf819b7.el9.noarch 39/164 2024-09-10T09:25:47.591 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/164 2024-09-10T09:25:47.591 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/164 2024-09-10T09:25:47.591 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/164 2024-09-10T09:25:47.591 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-09-10T09:25:47.591 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-09-10T09:25:47.591 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-09-10T09:25:47.591 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-09-10T09:25:47.591 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-09-10T09:25:47.592 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/164 2024-09-10T09:25:47.592 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-09-10T09:25:47.592 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-09-10T09:25:47.592 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-09-10T09:25:47.592 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/164 2024-09-10T09:25:47.592 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-09-10T09:25:47.592 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-09-10T09:25:47.592 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-09-10T09:25:47.592 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-09-10T09:25:47.592 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-09-10T09:25:47.592 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-09-10T09:25:47.592 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-09-10T09:25:47.593 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-09-10T09:25:47.593 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 61/164 2024-09-10T09:25:47.593 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 62/164 2024-09-10T09:25:47.593 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-09-10T09:25:47.593 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-09-10T09:25:47.593 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-09-10T09:25:47.593 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 66/164 2024-09-10T09:25:47.593 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 67/164 2024-09-10T09:25:47.593 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 68/164 2024-09-10T09:25:47.593 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : postgresql-13.16-1.el9.x86_64 69/164 2024-09-10T09:25:47.593 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 70/164 2024-09-10T09:25:47.593 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 71/164 2024-09-10T09:25:47.593 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 72/164 2024-09-10T09:25:47.594 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-09-10T09:25:47.594 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-09-10T09:25:47.594 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-09-10T09:25:47.594 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/164 2024-09-10T09:25:47.594 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 77/164 2024-09-10T09:25:47.594 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/164 2024-09-10T09:25:47.594 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/164 2024-09-10T09:25:47.595 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/164 2024-09-10T09:25:47.595 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/164 2024-09-10T09:25:47.595 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/164 2024-09-10T09:25:47.595 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/164 2024-09-10T09:25:47.595 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/164 2024-09-10T09:25:47.595 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/164 2024-09-10T09:25:47.595 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/164 2024-09-10T09:25:47.595 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/164 2024-09-10T09:25:47.595 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/164 2024-09-10T09:25:47.595 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/164 2024-09-10T09:25:47.596 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/164 2024-09-10T09:25:47.596 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/164 2024-09-10T09:25:47.596 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/164 2024-09-10T09:25:47.596 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/164 2024-09-10T09:25:47.596 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/164 2024-09-10T09:25:47.596 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/164 2024-09-10T09:25:47.596 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/164 2024-09-10T09:25:47.596 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/164 2024-09-10T09:25:47.596 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/164 2024-09-10T09:25:47.596 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 99/164 2024-09-10T09:25:47.596 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/164 2024-09-10T09:25:47.596 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 101/164 2024-09-10T09:25:47.597 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/164 2024-09-10T09:25:47.597 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/164 2024-09-10T09:25:47.597 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/164 2024-09-10T09:25:47.597 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/164 2024-09-10T09:25:47.597 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/164 2024-09-10T09:25:47.597 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 107/164 2024-09-10T09:25:47.597 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/164 2024-09-10T09:25:47.597 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/164 2024-09-10T09:25:47.597 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/164 2024-09-10T09:25:47.597 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/164 2024-09-10T09:25:47.597 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/164 2024-09-10T09:25:47.597 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/164 2024-09-10T09:25:47.597 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/164 2024-09-10T09:25:47.598 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/164 2024-09-10T09:25:47.598 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/164 2024-09-10T09:25:47.598 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/164 2024-09-10T09:25:47.598 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 118/164 2024-09-10T09:25:47.598 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/164 2024-09-10T09:25:47.598 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/164 2024-09-10T09:25:47.598 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/164 2024-09-10T09:25:47.598 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 122/164 2024-09-10T09:25:47.598 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-09-10T09:25:47.598 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-09-10T09:25:47.598 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-09-10T09:25:47.598 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-09-10T09:25:47.598 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-09-10T09:25:47.598 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-09-10T09:25:47.599 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-09-10T09:25:47.599 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-09-10T09:25:47.599 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-09-10T09:25:47.599 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-09-10T09:25:47.599 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-09-10T09:25:47.599 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-09-10T09:25:47.599 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-09-10T09:25:47.599 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-09-10T09:25:47.599 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-09-10T09:25:47.599 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-09-10T09:25:47.599 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-09-10T09:25:47.600 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-09-10T09:25:47.600 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-09-10T09:25:47.600 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 142/164 2024-09-10T09:25:47.600 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 143/164 2024-09-10T09:25:47.600 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-09-10T09:25:47.600 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-09-10T09:25:47.600 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-09-10T09:25:47.600 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : re2-1:20211101-20.el9.x86_64 147/164 2024-09-10T09:25:47.600 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-09-10T09:25:47.600 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : librados2-2:19.1.1-305.g1cf819b7.el9.x86_64 149/164 2024-09-10T09:25:47.600 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-09-10T09:25:47.600 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : librbd1-2:19.1.1-305.g1cf819b7.el9.x86_64 151/164 2024-09-10T09:25:47.600 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-09-10T09:25:47.601 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : NetworkManager-1:1.49.90-1.el9.x86_64 153/164 2024-09-10T09:25:47.601 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 154/164 2024-09-10T09:25:47.601 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 155/164 2024-09-10T09:25:47.601 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 156/164 2024-09-10T09:25:47.601 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : NetworkManager-team-1:1.49.90-1.el9.x86_64 157/164 2024-09-10T09:25:47.601 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 158/164 2024-09-10T09:25:47.601 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : NetworkManager-tui-1:1.49.90-1.el9.x86_64 159/164 2024-09-10T09:25:47.601 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 160/164 2024-09-10T09:25:47.601 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 161/164 2024-09-10T09:25:47.601 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 162/164 2024-09-10T09:25:47.601 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 163/164 2024-09-10T09:25:47.605 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-10T09:25:47.642 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-10T09:25:47.663 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-10T09:25:48.397 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-osd-2:19.1.1-305.g1cf819b7.el9.x86_64 111/164 2024-09-10T09:25:48.456 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: ceph-osd-2:19.1.1-305.g1cf819b7.el9.x86_64 111/164 2024-09-10T09:25:48.456 INFO:teuthology.orchestra.run.smithi005.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T09:25:48.456 INFO:teuthology.orchestra.run.smithi005.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-10T09:25:48.456 INFO:teuthology.orchestra.run.smithi005.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-10T09:25:48.457 INFO:teuthology.orchestra.run.smithi005.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-10T09:25:48.457 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-10T09:25:48.514 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : cephfs-top-2:19.1.1-305.g1cf819b7.el9.noarch 152/164 2024-09-10T09:25:48.605 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 153/164 2024-09-10T09:25:48.645 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-09-10T09:25:48.728 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-09-10T09:25:48.756 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-mds-2:19.1.1-305.g1cf819b7.el9.x86_64 112/164 2024-09-10T09:25:48.758 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-09-10T09:25:48.784 INFO:teuthology.orchestra.run.smithi159.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 157/164 2024-09-10T09:25:48.807 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: ceph-mds-2:19.1.1-305.g1cf819b7.el9.x86_64 112/164 2024-09-10T09:25:48.808 INFO:teuthology.orchestra.run.smithi005.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T09:25:48.808 INFO:teuthology.orchestra.run.smithi005.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-10T09:25:48.808 INFO:teuthology.orchestra.run.smithi005.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-10T09:25:48.808 INFO:teuthology.orchestra.run.smithi005.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-10T09:25:48.808 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-10T09:25:48.818 INFO:teuthology.orchestra.run.smithi159.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 158/164 2024-09-10T09:25:48.818 INFO:teuthology.orchestra.run.smithi159.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-10T09:25:48.919 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-10T09:25:48.919 INFO:teuthology.orchestra.run.smithi159.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-10T09:25:49.029 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-10T09:25:49.053 INFO:teuthology.orchestra.run.smithi159.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 161/164 2024-09-10T09:25:49.058 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-10T09:25:49.059 INFO:teuthology.orchestra.run.smithi159.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-10T09:25:49.238 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-10T09:25:49.238 INFO:teuthology.orchestra.run.smithi159.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-10T09:25:49.322 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-10T09:25:49.332 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-mon-2:19.1.1-305.g1cf819b7.el9.x86_64 113/164 2024-09-10T09:25:49.383 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: ceph-mon-2:19.1.1-305.g1cf819b7.el9.x86_64 113/164 2024-09-10T09:25:49.383 INFO:teuthology.orchestra.run.smithi005.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T09:25:49.383 INFO:teuthology.orchestra.run.smithi005.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-10T09:25:49.383 INFO:teuthology.orchestra.run.smithi005.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-10T09:25:49.383 INFO:teuthology.orchestra.run.smithi005.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-10T09:25:49.383 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-10T09:25:49.385 INFO:teuthology.orchestra.run.smithi159.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-10T09:25:49.517 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-09-10T09:25:49.577 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-09-10T09:25:49.650 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-09-10T09:25:49.888 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-09-10T09:25:49.954 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-09-10T09:25:50.043 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-09-10T09:25:50.153 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-09-10T09:25:50.332 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-09-10T09:25:50.349 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-10T09:25:50.349 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-10T09:25:50.349 INFO:teuthology.orchestra.run.smithi102.stdout:Upgraded: 2024-09-10T09:25:50.350 INFO:teuthology.orchestra.run.smithi102.stdout: NetworkManager-1:1.49.90-1.el9.x86_64 2024-09-10T09:25:50.350 INFO:teuthology.orchestra.run.smithi102.stdout: NetworkManager-libnm-1:1.49.90-1.el9.x86_64 2024-09-10T09:25:50.350 INFO:teuthology.orchestra.run.smithi102.stdout: NetworkManager-team-1:1.49.90-1.el9.x86_64 2024-09-10T09:25:50.350 INFO:teuthology.orchestra.run.smithi102.stdout: NetworkManager-tui-1:1.49.90-1.el9.x86_64 2024-09-10T09:25:50.350 INFO:teuthology.orchestra.run.smithi102.stdout: librados2-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T09:25:50.350 INFO:teuthology.orchestra.run.smithi102.stdout: librbd1-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T09:25:50.351 INFO:teuthology.orchestra.run.smithi102.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-10T09:25:50.351 INFO:teuthology.orchestra.run.smithi102.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-10T09:25:50.351 INFO:teuthology.orchestra.run.smithi102.stdout:Installed: 2024-09-10T09:25:50.351 INFO:teuthology.orchestra.run.smithi102.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-10T09:25:50.351 INFO:teuthology.orchestra.run.smithi102.stdout: automake-1.16.2-8.el9.noarch 2024-09-10T09:25:50.351 INFO:teuthology.orchestra.run.smithi102.stdout: bison-3.7.4-5.el9.x86_64 2024-09-10T09:25:50.352 INFO:teuthology.orchestra.run.smithi102.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-10T09:25:50.352 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T09:25:50.352 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-base-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T09:25:50.352 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-common-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T09:25:50.352 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-fuse-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T09:25:50.353 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-grafana-dashboards-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T09:25:50.353 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-immutable-object-cache-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T09:25:50.353 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-mds-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T09:25:50.353 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-mgr-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T09:25:50.354 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-mgr-cephadm-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T09:25:50.354 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-mgr-dashboard-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T09:25:50.354 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-mgr-diskprediction-local-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T09:25:50.354 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-mgr-modules-core-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T09:25:50.354 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-mgr-rook-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T09:25:50.354 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-mon-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T09:25:50.355 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-osd-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T09:25:50.355 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-prometheus-alerts-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T09:25:50.355 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-radosgw-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T09:25:50.355 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-selinux-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T09:25:50.355 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-test-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T09:25:50.355 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-volume-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T09:25:50.355 INFO:teuthology.orchestra.run.smithi102.stdout: cephadm-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T09:25:50.356 INFO:teuthology.orchestra.run.smithi102.stdout: cephfs-mirror-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T09:25:50.356 INFO:teuthology.orchestra.run.smithi102.stdout: cephfs-shell-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T09:25:50.356 INFO:teuthology.orchestra.run.smithi102.stdout: cephfs-top-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T09:25:50.356 INFO:teuthology.orchestra.run.smithi102.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-10T09:25:50.357 INFO:teuthology.orchestra.run.smithi102.stdout: flex-2.6.4-9.el9.x86_64 2024-09-10T09:25:50.357 INFO:teuthology.orchestra.run.smithi102.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-10T09:25:50.357 INFO:teuthology.orchestra.run.smithi102.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-10T09:25:50.357 INFO:teuthology.orchestra.run.smithi102.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-10T09:25:50.357 INFO:teuthology.orchestra.run.smithi102.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-10T09:25:50.357 INFO:teuthology.orchestra.run.smithi102.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-09-10T09:25:50.357 INFO:teuthology.orchestra.run.smithi102.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-10T09:25:50.358 INFO:teuthology.orchestra.run.smithi102.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-10T09:25:50.358 INFO:teuthology.orchestra.run.smithi102.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-10T09:25:50.358 INFO:teuthology.orchestra.run.smithi102.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-10T09:25:50.358 INFO:teuthology.orchestra.run.smithi102.stdout: libcephfs-devel-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T09:25:50.358 INFO:teuthology.orchestra.run.smithi102.stdout: libcephfs2-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T09:25:50.358 INFO:teuthology.orchestra.run.smithi102.stdout: libcephsqlite-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T09:25:50.358 INFO:teuthology.orchestra.run.smithi102.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-10T09:25:50.359 INFO:teuthology.orchestra.run.smithi102.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-10T09:25:50.359 INFO:teuthology.orchestra.run.smithi102.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-10T09:25:50.359 INFO:teuthology.orchestra.run.smithi102.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-10T09:25:50.359 INFO:teuthology.orchestra.run.smithi102.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-10T09:25:50.359 INFO:teuthology.orchestra.run.smithi102.stdout: librados-devel-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T09:25:50.360 INFO:teuthology.orchestra.run.smithi102.stdout: libradosstriper1-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T09:25:50.360 INFO:teuthology.orchestra.run.smithi102.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-10T09:25:50.360 INFO:teuthology.orchestra.run.smithi102.stdout: librgw2-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T09:25:50.360 INFO:teuthology.orchestra.run.smithi102.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-10T09:25:50.360 INFO:teuthology.orchestra.run.smithi102.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-10T09:25:50.360 INFO:teuthology.orchestra.run.smithi102.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-10T09:25:50.360 INFO:teuthology.orchestra.run.smithi102.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-10T09:25:50.361 INFO:teuthology.orchestra.run.smithi102.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-10T09:25:50.361 INFO:teuthology.orchestra.run.smithi102.stdout: lua-5.4.4-4.el9.x86_64 2024-09-10T09:25:50.361 INFO:teuthology.orchestra.run.smithi102.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-10T09:25:50.361 INFO:teuthology.orchestra.run.smithi102.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-10T09:25:50.361 INFO:teuthology.orchestra.run.smithi102.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-10T09:25:50.362 INFO:teuthology.orchestra.run.smithi102.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-10T09:25:50.362 INFO:teuthology.orchestra.run.smithi102.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-10T09:25:50.362 INFO:teuthology.orchestra.run.smithi102.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-10T09:25:50.362 INFO:teuthology.orchestra.run.smithi102.stdout: postgresql-13.16-1.el9.x86_64 2024-09-10T09:25:50.362 INFO:teuthology.orchestra.run.smithi102.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-10T09:25:50.362 INFO:teuthology.orchestra.run.smithi102.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-10T09:25:50.362 INFO:teuthology.orchestra.run.smithi102.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-10T09:25:50.362 INFO:teuthology.orchestra.run.smithi102.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-10T09:25:50.362 INFO:teuthology.orchestra.run.smithi102.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-10T09:25:50.363 INFO:teuthology.orchestra.run.smithi102.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-10T09:25:50.363 INFO:teuthology.orchestra.run.smithi102.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-10T09:25:50.363 INFO:teuthology.orchestra.run.smithi102.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-10T09:25:50.363 INFO:teuthology.orchestra.run.smithi102.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-10T09:25:50.363 INFO:teuthology.orchestra.run.smithi102.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-10T09:25:50.363 INFO:teuthology.orchestra.run.smithi102.stdout: python3-ceph-argparse-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T09:25:50.363 INFO:teuthology.orchestra.run.smithi102.stdout: python3-ceph-common-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T09:25:50.363 INFO:teuthology.orchestra.run.smithi102.stdout: python3-cephfs-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T09:25:50.363 INFO:teuthology.orchestra.run.smithi102.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-10T09:25:50.363 INFO:teuthology.orchestra.run.smithi102.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-10T09:25:50.364 INFO:teuthology.orchestra.run.smithi102.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-10T09:25:50.364 INFO:teuthology.orchestra.run.smithi102.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-10T09:25:50.364 INFO:teuthology.orchestra.run.smithi102.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-10T09:25:50.364 INFO:teuthology.orchestra.run.smithi102.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-10T09:25:50.364 INFO:teuthology.orchestra.run.smithi102.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-10T09:25:50.364 INFO:teuthology.orchestra.run.smithi102.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-10T09:25:50.364 INFO:teuthology.orchestra.run.smithi102.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-10T09:25:50.365 INFO:teuthology.orchestra.run.smithi102.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-10T09:25:50.365 INFO:teuthology.orchestra.run.smithi102.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-10T09:25:50.365 INFO:teuthology.orchestra.run.smithi102.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-10T09:25:50.365 INFO:teuthology.orchestra.run.smithi102.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-10T09:25:50.365 INFO:teuthology.orchestra.run.smithi102.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-10T09:25:50.365 INFO:teuthology.orchestra.run.smithi102.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-10T09:25:50.365 INFO:teuthology.orchestra.run.smithi102.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-10T09:25:50.365 INFO:teuthology.orchestra.run.smithi102.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-10T09:25:50.365 INFO:teuthology.orchestra.run.smithi102.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-10T09:25:50.365 INFO:teuthology.orchestra.run.smithi102.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-10T09:25:50.365 INFO:teuthology.orchestra.run.smithi102.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-10T09:25:50.366 INFO:teuthology.orchestra.run.smithi102.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-10T09:25:50.366 INFO:teuthology.orchestra.run.smithi102.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-10T09:25:50.366 INFO:teuthology.orchestra.run.smithi102.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-10T09:25:50.366 INFO:teuthology.orchestra.run.smithi102.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-10T09:25:50.366 INFO:teuthology.orchestra.run.smithi102.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-10T09:25:50.367 INFO:teuthology.orchestra.run.smithi102.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-10T09:25:50.367 INFO:teuthology.orchestra.run.smithi102.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-10T09:25:50.367 INFO:teuthology.orchestra.run.smithi102.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-10T09:25:50.367 INFO:teuthology.orchestra.run.smithi102.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-10T09:25:50.367 INFO:teuthology.orchestra.run.smithi102.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-10T09:25:50.367 INFO:teuthology.orchestra.run.smithi102.stdout: python3-ply-3.11-14.el9.noarch 2024-09-10T09:25:50.367 INFO:teuthology.orchestra.run.smithi102.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-10T09:25:50.368 INFO:teuthology.orchestra.run.smithi102.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-10T09:25:50.368 INFO:teuthology.orchestra.run.smithi102.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-10T09:25:50.368 INFO:teuthology.orchestra.run.smithi102.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-10T09:25:50.368 INFO:teuthology.orchestra.run.smithi102.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-10T09:25:50.368 INFO:teuthology.orchestra.run.smithi102.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-10T09:25:50.368 INFO:teuthology.orchestra.run.smithi102.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-10T09:25:50.368 INFO:teuthology.orchestra.run.smithi102.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-10T09:25:50.368 INFO:teuthology.orchestra.run.smithi102.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-10T09:25:50.368 INFO:teuthology.orchestra.run.smithi102.stdout: python3-rados-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T09:25:50.368 INFO:teuthology.orchestra.run.smithi102.stdout: python3-rbd-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T09:25:50.369 INFO:teuthology.orchestra.run.smithi102.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-10T09:25:50.369 INFO:teuthology.orchestra.run.smithi102.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-10T09:25:50.369 INFO:teuthology.orchestra.run.smithi102.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-10T09:25:50.369 INFO:teuthology.orchestra.run.smithi102.stdout: python3-rgw-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T09:25:50.369 INFO:teuthology.orchestra.run.smithi102.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-10T09:25:50.369 INFO:teuthology.orchestra.run.smithi102.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-10T09:25:50.369 INFO:teuthology.orchestra.run.smithi102.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-10T09:25:50.369 INFO:teuthology.orchestra.run.smithi102.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-10T09:25:50.369 INFO:teuthology.orchestra.run.smithi102.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-10T09:25:50.369 INFO:teuthology.orchestra.run.smithi102.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-10T09:25:50.370 INFO:teuthology.orchestra.run.smithi102.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-10T09:25:50.370 INFO:teuthology.orchestra.run.smithi102.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-10T09:25:50.370 INFO:teuthology.orchestra.run.smithi102.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-10T09:25:50.370 INFO:teuthology.orchestra.run.smithi102.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-10T09:25:50.370 INFO:teuthology.orchestra.run.smithi102.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-10T09:25:50.370 INFO:teuthology.orchestra.run.smithi102.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-10T09:25:50.370 INFO:teuthology.orchestra.run.smithi102.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-10T09:25:50.370 INFO:teuthology.orchestra.run.smithi102.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-10T09:25:50.370 INFO:teuthology.orchestra.run.smithi102.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-10T09:25:50.370 INFO:teuthology.orchestra.run.smithi102.stdout: rbd-fuse-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T09:25:50.370 INFO:teuthology.orchestra.run.smithi102.stdout: rbd-mirror-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T09:25:50.370 INFO:teuthology.orchestra.run.smithi102.stdout: rbd-nbd-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T09:25:50.370 INFO:teuthology.orchestra.run.smithi102.stdout: re2-1:20211101-20.el9.x86_64 2024-09-10T09:25:50.371 INFO:teuthology.orchestra.run.smithi102.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-10T09:25:50.371 INFO:teuthology.orchestra.run.smithi102.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-10T09:25:50.371 INFO:teuthology.orchestra.run.smithi102.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-10T09:25:50.371 INFO:teuthology.orchestra.run.smithi102.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-10T09:25:50.371 INFO:teuthology.orchestra.run.smithi102.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-10T09:25:50.371 INFO:teuthology.orchestra.run.smithi102.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-10T09:25:50.371 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-10T09:25:50.371 INFO:teuthology.orchestra.run.smithi102.stdout:Complete! 2024-09-10T09:25:50.383 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-09-10T09:25:50.427 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-09-10T09:25:50.452 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : fuse-2.9.9-16.el9.x86_64 124/164 2024-09-10T09:25:50.773 DEBUG:teuthology.parallel:result is None 2024-09-10T09:25:51.068 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: cephadm-2:19.1.1-305.g1cf819b7.el9.noarch 125/164 2024-09-10T09:25:51.101 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : cephadm-2:19.1.1-305.g1cf819b7.el9.noarch 125/164 2024-09-10T09:25:51.135 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-prometheus-alerts-2:19.1.1-305.g1cf819b7.el 126/164 2024-09-10T09:25:51.213 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-grafana-dashboards-2:19.1.1-305.g1cf819b7.e 127/164 2024-09-10T09:25:51.273 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-mgr-cephadm-2:19.1.1-305.g1cf819b7.el9.noar 128/164 2024-09-10T09:25:52.579 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.1.1-305.g1cf819b7.el9.noar 128/164 2024-09-10T09:25:52.627 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-mgr-dashboard-2:19.1.1-305.g1cf819b7.el9.no 129/164 2024-09-10T09:25:53.782 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.1.1-305.g1cf819b7.el9.no 129/164 2024-09-10T09:25:53.809 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-mgr-diskprediction-local-2:19.1.1-305.g1cf8 130/164 2024-09-10T09:25:53.955 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.1.1-305.g1cf8 130/164 2024-09-10T09:25:54.140 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-mgr-modules-core-2:19.1.1-305.g1cf819b7.el9 131/164 2024-09-10T09:25:54.178 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-mgr-2:19.1.1-305.g1cf819b7.el9.x86_64 132/164 2024-09-10T09:25:54.229 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: ceph-mgr-2:19.1.1-305.g1cf819b7.el9.x86_64 132/164 2024-09-10T09:25:54.229 INFO:teuthology.orchestra.run.smithi005.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T09:25:54.229 INFO:teuthology.orchestra.run.smithi005.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-10T09:25:54.229 INFO:teuthology.orchestra.run.smithi005.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-10T09:25:54.229 INFO:teuthology.orchestra.run.smithi005.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-10T09:25:54.230 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-10T09:25:54.287 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-mgr-rook-2:19.1.1-305.g1cf819b7.el9.noarch 133/164 2024-09-10T09:25:54.314 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: ceph-mgr-rook-2:19.1.1-305.g1cf819b7.el9.noarch 133/164 2024-09-10T09:25:54.425 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-2:19.1.1-305.g1cf819b7.el9.x86_64 134/164 2024-09-10T09:25:54.503 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-fuse-2:19.1.1-305.g1cf819b7.el9.x86_64 135/164 2024-09-10T09:25:55.100 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-10T09:25:55.100 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : ceph-2:19.1.1-305.g1cf819b7.el9.x86_64 1/164 2024-09-10T09:25:55.101 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : ceph-base-2:19.1.1-305.g1cf819b7.el9.x86_64 2/164 2024-09-10T09:25:55.101 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : ceph-common-2:19.1.1-305.g1cf819b7.el9.x86_64 3/164 2024-09-10T09:25:55.101 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : ceph-fuse-2:19.1.1-305.g1cf819b7.el9.x86_64 4/164 2024-09-10T09:25:55.101 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : ceph-immutable-object-cache-2:19.1.1-305.g1cf819 5/164 2024-09-10T09:25:55.101 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : ceph-mds-2:19.1.1-305.g1cf819b7.el9.x86_64 6/164 2024-09-10T09:25:55.101 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : ceph-mgr-2:19.1.1-305.g1cf819b7.el9.x86_64 7/164 2024-09-10T09:25:55.101 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : ceph-mon-2:19.1.1-305.g1cf819b7.el9.x86_64 8/164 2024-09-10T09:25:55.101 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : ceph-osd-2:19.1.1-305.g1cf819b7.el9.x86_64 9/164 2024-09-10T09:25:55.101 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : ceph-radosgw-2:19.1.1-305.g1cf819b7.el9.x86_64 10/164 2024-09-10T09:25:55.101 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : ceph-selinux-2:19.1.1-305.g1cf819b7.el9.x86_64 11/164 2024-09-10T09:25:55.101 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : ceph-test-2:19.1.1-305.g1cf819b7.el9.x86_64 12/164 2024-09-10T09:25:55.101 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : cephfs-mirror-2:19.1.1-305.g1cf819b7.el9.x86_64 13/164 2024-09-10T09:25:55.101 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : cephfs-shell-2:19.1.1-305.g1cf819b7.el9.x86_64 14/164 2024-09-10T09:25:55.102 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : libcephfs-devel-2:19.1.1-305.g1cf819b7.el9.x86_6 15/164 2024-09-10T09:25:55.102 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : libcephfs2-2:19.1.1-305.g1cf819b7.el9.x86_64 16/164 2024-09-10T09:25:55.102 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : libcephsqlite-2:19.1.1-305.g1cf819b7.el9.x86_64 17/164 2024-09-10T09:25:55.102 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : librados-devel-2:19.1.1-305.g1cf819b7.el9.x86_64 18/164 2024-09-10T09:25:55.102 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : libradosstriper1-2:19.1.1-305.g1cf819b7.el9.x86_ 19/164 2024-09-10T09:25:55.102 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : librgw2-2:19.1.1-305.g1cf819b7.el9.x86_64 20/164 2024-09-10T09:25:55.102 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-ceph-argparse-2:19.1.1-305.g1cf819b7.el9 21/164 2024-09-10T09:25:55.102 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-ceph-common-2:19.1.1-305.g1cf819b7.el9.x 22/164 2024-09-10T09:25:55.102 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-cephfs-2:19.1.1-305.g1cf819b7.el9.x86_64 23/164 2024-09-10T09:25:55.102 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-rados-2:19.1.1-305.g1cf819b7.el9.x86_64 24/164 2024-09-10T09:25:55.102 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-rbd-2:19.1.1-305.g1cf819b7.el9.x86_64 25/164 2024-09-10T09:25:55.102 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-rgw-2:19.1.1-305.g1cf819b7.el9.x86_64 26/164 2024-09-10T09:25:55.102 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : rbd-fuse-2:19.1.1-305.g1cf819b7.el9.x86_64 27/164 2024-09-10T09:25:55.103 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : rbd-mirror-2:19.1.1-305.g1cf819b7.el9.x86_64 28/164 2024-09-10T09:25:55.103 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : rbd-nbd-2:19.1.1-305.g1cf819b7.el9.x86_64 29/164 2024-09-10T09:25:55.103 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : ceph-grafana-dashboards-2:19.1.1-305.g1cf819b7.e 30/164 2024-09-10T09:25:55.103 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : ceph-mgr-cephadm-2:19.1.1-305.g1cf819b7.el9.noar 31/164 2024-09-10T09:25:55.103 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : ceph-mgr-dashboard-2:19.1.1-305.g1cf819b7.el9.no 32/164 2024-09-10T09:25:55.103 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.1.1-305.g1cf8 33/164 2024-09-10T09:25:55.103 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : ceph-mgr-modules-core-2:19.1.1-305.g1cf819b7.el9 34/164 2024-09-10T09:25:55.103 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : ceph-mgr-rook-2:19.1.1-305.g1cf819b7.el9.noarch 35/164 2024-09-10T09:25:55.103 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : ceph-prometheus-alerts-2:19.1.1-305.g1cf819b7.el 36/164 2024-09-10T09:25:55.103 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : ceph-volume-2:19.1.1-305.g1cf819b7.el9.noarch 37/164 2024-09-10T09:25:55.103 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : cephadm-2:19.1.1-305.g1cf819b7.el9.noarch 38/164 2024-09-10T09:25:55.103 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : cephfs-top-2:19.1.1-305.g1cf819b7.el9.noarch 39/164 2024-09-10T09:25:55.103 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/164 2024-09-10T09:25:55.104 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/164 2024-09-10T09:25:55.104 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/164 2024-09-10T09:25:55.104 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-09-10T09:25:55.104 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-09-10T09:25:55.104 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-09-10T09:25:55.104 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-09-10T09:25:55.104 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-09-10T09:25:55.104 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/164 2024-09-10T09:25:55.104 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-09-10T09:25:55.104 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-09-10T09:25:55.104 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-09-10T09:25:55.104 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/164 2024-09-10T09:25:55.105 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-09-10T09:25:55.105 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-09-10T09:25:55.105 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-09-10T09:25:55.105 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-09-10T09:25:55.105 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-09-10T09:25:55.105 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-09-10T09:25:55.105 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-09-10T09:25:55.105 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-09-10T09:25:55.105 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 61/164 2024-09-10T09:25:55.105 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 62/164 2024-09-10T09:25:55.105 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-09-10T09:25:55.105 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-09-10T09:25:55.105 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-09-10T09:25:55.105 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 66/164 2024-09-10T09:25:55.106 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 67/164 2024-09-10T09:25:55.106 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 68/164 2024-09-10T09:25:55.106 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : postgresql-13.16-1.el9.x86_64 69/164 2024-09-10T09:25:55.106 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 70/164 2024-09-10T09:25:55.106 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 71/164 2024-09-10T09:25:55.106 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 72/164 2024-09-10T09:25:55.106 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-09-10T09:25:55.106 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-09-10T09:25:55.106 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-09-10T09:25:55.106 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/164 2024-09-10T09:25:55.106 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 77/164 2024-09-10T09:25:55.106 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/164 2024-09-10T09:25:55.106 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/164 2024-09-10T09:25:55.107 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/164 2024-09-10T09:25:55.107 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/164 2024-09-10T09:25:55.107 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/164 2024-09-10T09:25:55.107 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/164 2024-09-10T09:25:55.107 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/164 2024-09-10T09:25:55.107 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/164 2024-09-10T09:25:55.107 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/164 2024-09-10T09:25:55.107 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/164 2024-09-10T09:25:55.107 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/164 2024-09-10T09:25:55.107 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/164 2024-09-10T09:25:55.107 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/164 2024-09-10T09:25:55.107 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/164 2024-09-10T09:25:55.107 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/164 2024-09-10T09:25:55.108 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/164 2024-09-10T09:25:55.108 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/164 2024-09-10T09:25:55.108 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/164 2024-09-10T09:25:55.108 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/164 2024-09-10T09:25:55.108 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/164 2024-09-10T09:25:55.108 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/164 2024-09-10T09:25:55.108 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 99/164 2024-09-10T09:25:55.108 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/164 2024-09-10T09:25:55.108 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 101/164 2024-09-10T09:25:55.108 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/164 2024-09-10T09:25:55.108 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/164 2024-09-10T09:25:55.108 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/164 2024-09-10T09:25:55.108 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/164 2024-09-10T09:25:55.109 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/164 2024-09-10T09:25:55.109 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 107/164 2024-09-10T09:25:55.109 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/164 2024-09-10T09:25:55.109 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/164 2024-09-10T09:25:55.109 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/164 2024-09-10T09:25:55.109 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/164 2024-09-10T09:25:55.109 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/164 2024-09-10T09:25:55.109 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/164 2024-09-10T09:25:55.109 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/164 2024-09-10T09:25:55.109 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/164 2024-09-10T09:25:55.109 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/164 2024-09-10T09:25:55.109 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/164 2024-09-10T09:25:55.109 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 118/164 2024-09-10T09:25:55.109 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/164 2024-09-10T09:25:55.110 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/164 2024-09-10T09:25:55.110 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/164 2024-09-10T09:25:55.110 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 122/164 2024-09-10T09:25:55.110 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-09-10T09:25:55.110 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-09-10T09:25:55.110 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-09-10T09:25:55.110 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-09-10T09:25:55.110 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-09-10T09:25:55.110 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-09-10T09:25:55.110 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-09-10T09:25:55.110 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-09-10T09:25:55.110 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-09-10T09:25:55.110 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-09-10T09:25:55.111 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-09-10T09:25:55.111 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-09-10T09:25:55.111 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-09-10T09:25:55.111 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-09-10T09:25:55.111 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-09-10T09:25:55.111 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-09-10T09:25:55.111 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-09-10T09:25:55.111 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-09-10T09:25:55.111 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-09-10T09:25:55.111 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 142/164 2024-09-10T09:25:55.111 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 143/164 2024-09-10T09:25:55.111 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-09-10T09:25:55.111 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-09-10T09:25:55.111 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-09-10T09:25:55.112 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : re2-1:20211101-20.el9.x86_64 147/164 2024-09-10T09:25:55.112 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-09-10T09:25:55.112 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : librados2-2:19.1.1-305.g1cf819b7.el9.x86_64 149/164 2024-09-10T09:25:55.112 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-09-10T09:25:55.112 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : librbd1-2:19.1.1-305.g1cf819b7.el9.x86_64 151/164 2024-09-10T09:25:55.112 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-09-10T09:25:55.112 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : NetworkManager-1:1.49.90-1.el9.x86_64 153/164 2024-09-10T09:25:55.112 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 154/164 2024-09-10T09:25:55.112 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 155/164 2024-09-10T09:25:55.112 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 156/164 2024-09-10T09:25:55.112 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : NetworkManager-team-1:1.49.90-1.el9.x86_64 157/164 2024-09-10T09:25:55.112 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 158/164 2024-09-10T09:25:55.112 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : NetworkManager-tui-1:1.49.90-1.el9.x86_64 159/164 2024-09-10T09:25:55.113 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 160/164 2024-09-10T09:25:55.113 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 161/164 2024-09-10T09:25:55.113 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 162/164 2024-09-10T09:25:55.113 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 163/164 2024-09-10T09:25:55.576 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : libtool-2.4.6-46.el9.x86_64 136/164 2024-09-10T09:25:55.606 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-radosgw-2:19.1.1-305.g1cf819b7.el9.x86_64 137/164 2024-09-10T09:25:55.659 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: ceph-radosgw-2:19.1.1-305.g1cf819b7.el9.x86_64 137/164 2024-09-10T09:25:55.659 INFO:teuthology.orchestra.run.smithi005.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T09:25:55.659 INFO:teuthology.orchestra.run.smithi005.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-10T09:25:55.659 INFO:teuthology.orchestra.run.smithi005.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-10T09:25:55.659 INFO:teuthology.orchestra.run.smithi005.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-10T09:25:55.659 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-10T09:25:55.707 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-immutable-object-cache-2:19.1.1-305.g1cf819 138/164 2024-09-10T09:25:55.758 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.1.1-305.g1cf819 138/164 2024-09-10T09:25:55.758 INFO:teuthology.orchestra.run.smithi005.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T09:25:55.758 INFO:teuthology.orchestra.run.smithi005.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-10T09:25:55.758 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-10T09:25:55.808 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : cephfs-mirror-2:19.1.1-305.g1cf819b7.el9.x86_64 139/164 2024-09-10T09:25:55.859 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: cephfs-mirror-2:19.1.1-305.g1cf819b7.el9.x86_64 139/164 2024-09-10T09:25:55.859 INFO:teuthology.orchestra.run.smithi005.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T09:25:55.859 INFO:teuthology.orchestra.run.smithi005.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-10T09:25:55.859 INFO:teuthology.orchestra.run.smithi005.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-10T09:25:55.860 INFO:teuthology.orchestra.run.smithi005.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-10T09:25:55.860 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-10T09:25:56.209 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : rbd-mirror-2:19.1.1-305.g1cf819b7.el9.x86_64 140/164 2024-09-10T09:25:56.260 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: rbd-mirror-2:19.1.1-305.g1cf819b7.el9.x86_64 140/164 2024-09-10T09:25:56.260 INFO:teuthology.orchestra.run.smithi005.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T09:25:56.260 INFO:teuthology.orchestra.run.smithi005.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-10T09:25:56.260 INFO:teuthology.orchestra.run.smithi005.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-10T09:25:56.260 INFO:teuthology.orchestra.run.smithi005.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-10T09:25:56.260 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-10T09:25:57.834 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-10T09:25:57.834 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-10T09:25:57.834 INFO:teuthology.orchestra.run.smithi159.stdout:Upgraded: 2024-09-10T09:25:57.834 INFO:teuthology.orchestra.run.smithi159.stdout: NetworkManager-1:1.49.90-1.el9.x86_64 2024-09-10T09:25:57.834 INFO:teuthology.orchestra.run.smithi159.stdout: NetworkManager-libnm-1:1.49.90-1.el9.x86_64 2024-09-10T09:25:57.835 INFO:teuthology.orchestra.run.smithi159.stdout: NetworkManager-team-1:1.49.90-1.el9.x86_64 2024-09-10T09:25:57.835 INFO:teuthology.orchestra.run.smithi159.stdout: NetworkManager-tui-1:1.49.90-1.el9.x86_64 2024-09-10T09:25:57.835 INFO:teuthology.orchestra.run.smithi159.stdout: librados2-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T09:25:57.835 INFO:teuthology.orchestra.run.smithi159.stdout: librbd1-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T09:25:57.835 INFO:teuthology.orchestra.run.smithi159.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-10T09:25:57.835 INFO:teuthology.orchestra.run.smithi159.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-10T09:25:57.835 INFO:teuthology.orchestra.run.smithi159.stdout:Installed: 2024-09-10T09:25:57.835 INFO:teuthology.orchestra.run.smithi159.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-10T09:25:57.835 INFO:teuthology.orchestra.run.smithi159.stdout: automake-1.16.2-8.el9.noarch 2024-09-10T09:25:57.835 INFO:teuthology.orchestra.run.smithi159.stdout: bison-3.7.4-5.el9.x86_64 2024-09-10T09:25:57.835 INFO:teuthology.orchestra.run.smithi159.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-10T09:25:57.835 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T09:25:57.836 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-base-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T09:25:57.836 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-common-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T09:25:57.836 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-fuse-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T09:25:57.836 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-grafana-dashboards-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T09:25:57.836 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-immutable-object-cache-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T09:25:57.836 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-mds-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T09:25:57.836 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-mgr-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T09:25:57.836 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-mgr-cephadm-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T09:25:57.836 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-mgr-dashboard-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T09:25:57.836 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-mgr-diskprediction-local-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T09:25:57.836 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-mgr-modules-core-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T09:25:57.836 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-mgr-rook-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T09:25:57.837 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-mon-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T09:25:57.837 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-osd-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T09:25:57.837 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-prometheus-alerts-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T09:25:57.837 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-radosgw-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T09:25:57.837 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-selinux-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T09:25:57.837 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-test-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T09:25:57.837 INFO:teuthology.orchestra.run.smithi159.stdout: ceph-volume-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T09:25:57.837 INFO:teuthology.orchestra.run.smithi159.stdout: cephadm-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T09:25:57.837 INFO:teuthology.orchestra.run.smithi159.stdout: cephfs-mirror-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T09:25:57.837 INFO:teuthology.orchestra.run.smithi159.stdout: cephfs-shell-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T09:25:57.837 INFO:teuthology.orchestra.run.smithi159.stdout: cephfs-top-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T09:25:57.837 INFO:teuthology.orchestra.run.smithi159.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-10T09:25:57.838 INFO:teuthology.orchestra.run.smithi159.stdout: flex-2.6.4-9.el9.x86_64 2024-09-10T09:25:57.838 INFO:teuthology.orchestra.run.smithi159.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-10T09:25:57.838 INFO:teuthology.orchestra.run.smithi159.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-10T09:25:57.838 INFO:teuthology.orchestra.run.smithi159.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-10T09:25:57.838 INFO:teuthology.orchestra.run.smithi159.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-10T09:25:57.838 INFO:teuthology.orchestra.run.smithi159.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-09-10T09:25:57.838 INFO:teuthology.orchestra.run.smithi159.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-10T09:25:57.838 INFO:teuthology.orchestra.run.smithi159.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-10T09:25:57.838 INFO:teuthology.orchestra.run.smithi159.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-10T09:25:57.838 INFO:teuthology.orchestra.run.smithi159.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-10T09:25:57.838 INFO:teuthology.orchestra.run.smithi159.stdout: libcephfs-devel-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T09:25:57.838 INFO:teuthology.orchestra.run.smithi159.stdout: libcephfs2-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T09:25:57.838 INFO:teuthology.orchestra.run.smithi159.stdout: libcephsqlite-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T09:25:57.839 INFO:teuthology.orchestra.run.smithi159.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-10T09:25:57.839 INFO:teuthology.orchestra.run.smithi159.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-10T09:25:57.839 INFO:teuthology.orchestra.run.smithi159.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-10T09:25:57.839 INFO:teuthology.orchestra.run.smithi159.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-10T09:25:57.839 INFO:teuthology.orchestra.run.smithi159.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-10T09:25:57.839 INFO:teuthology.orchestra.run.smithi159.stdout: librados-devel-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T09:25:57.839 INFO:teuthology.orchestra.run.smithi159.stdout: libradosstriper1-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T09:25:57.839 INFO:teuthology.orchestra.run.smithi159.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-10T09:25:57.839 INFO:teuthology.orchestra.run.smithi159.stdout: librgw2-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T09:25:57.839 INFO:teuthology.orchestra.run.smithi159.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-10T09:25:57.839 INFO:teuthology.orchestra.run.smithi159.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-10T09:25:57.839 INFO:teuthology.orchestra.run.smithi159.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-10T09:25:57.840 INFO:teuthology.orchestra.run.smithi159.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-10T09:25:57.840 INFO:teuthology.orchestra.run.smithi159.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-10T09:25:57.840 INFO:teuthology.orchestra.run.smithi159.stdout: lua-5.4.4-4.el9.x86_64 2024-09-10T09:25:57.840 INFO:teuthology.orchestra.run.smithi159.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-10T09:25:57.840 INFO:teuthology.orchestra.run.smithi159.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-10T09:25:57.840 INFO:teuthology.orchestra.run.smithi159.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-10T09:25:57.840 INFO:teuthology.orchestra.run.smithi159.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-10T09:25:57.840 INFO:teuthology.orchestra.run.smithi159.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-10T09:25:57.840 INFO:teuthology.orchestra.run.smithi159.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-10T09:25:57.840 INFO:teuthology.orchestra.run.smithi159.stdout: postgresql-13.16-1.el9.x86_64 2024-09-10T09:25:57.840 INFO:teuthology.orchestra.run.smithi159.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-10T09:25:57.840 INFO:teuthology.orchestra.run.smithi159.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-10T09:25:57.841 INFO:teuthology.orchestra.run.smithi159.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-10T09:25:57.841 INFO:teuthology.orchestra.run.smithi159.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-10T09:25:57.841 INFO:teuthology.orchestra.run.smithi159.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-10T09:25:57.841 INFO:teuthology.orchestra.run.smithi159.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-10T09:25:57.841 INFO:teuthology.orchestra.run.smithi159.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-10T09:25:57.841 INFO:teuthology.orchestra.run.smithi159.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-10T09:25:57.841 INFO:teuthology.orchestra.run.smithi159.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-10T09:25:57.841 INFO:teuthology.orchestra.run.smithi159.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-10T09:25:57.841 INFO:teuthology.orchestra.run.smithi159.stdout: python3-ceph-argparse-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T09:25:57.841 INFO:teuthology.orchestra.run.smithi159.stdout: python3-ceph-common-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T09:25:57.841 INFO:teuthology.orchestra.run.smithi159.stdout: python3-cephfs-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T09:25:57.841 INFO:teuthology.orchestra.run.smithi159.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-10T09:25:57.841 INFO:teuthology.orchestra.run.smithi159.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-10T09:25:57.841 INFO:teuthology.orchestra.run.smithi159.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-10T09:25:57.842 INFO:teuthology.orchestra.run.smithi159.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-10T09:25:57.842 INFO:teuthology.orchestra.run.smithi159.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-10T09:25:57.842 INFO:teuthology.orchestra.run.smithi159.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-10T09:25:57.842 INFO:teuthology.orchestra.run.smithi159.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-10T09:25:57.842 INFO:teuthology.orchestra.run.smithi159.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-10T09:25:57.842 INFO:teuthology.orchestra.run.smithi159.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-10T09:25:57.842 INFO:teuthology.orchestra.run.smithi159.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-10T09:25:57.842 INFO:teuthology.orchestra.run.smithi159.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-10T09:25:57.842 INFO:teuthology.orchestra.run.smithi159.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-10T09:25:57.842 INFO:teuthology.orchestra.run.smithi159.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-10T09:25:57.842 INFO:teuthology.orchestra.run.smithi159.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-10T09:25:57.842 INFO:teuthology.orchestra.run.smithi159.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-10T09:25:57.842 INFO:teuthology.orchestra.run.smithi159.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-10T09:25:57.843 INFO:teuthology.orchestra.run.smithi159.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-10T09:25:57.843 INFO:teuthology.orchestra.run.smithi159.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-10T09:25:57.843 INFO:teuthology.orchestra.run.smithi159.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-10T09:25:57.843 INFO:teuthology.orchestra.run.smithi159.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-10T09:25:57.843 INFO:teuthology.orchestra.run.smithi159.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-10T09:25:57.843 INFO:teuthology.orchestra.run.smithi159.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-10T09:25:57.843 INFO:teuthology.orchestra.run.smithi159.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-10T09:25:57.843 INFO:teuthology.orchestra.run.smithi159.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-10T09:25:57.843 INFO:teuthology.orchestra.run.smithi159.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-10T09:25:57.843 INFO:teuthology.orchestra.run.smithi159.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-10T09:25:57.843 INFO:teuthology.orchestra.run.smithi159.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-10T09:25:57.843 INFO:teuthology.orchestra.run.smithi159.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-10T09:25:57.843 INFO:teuthology.orchestra.run.smithi159.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-10T09:25:57.844 INFO:teuthology.orchestra.run.smithi159.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-10T09:25:57.844 INFO:teuthology.orchestra.run.smithi159.stdout: python3-ply-3.11-14.el9.noarch 2024-09-10T09:25:57.844 INFO:teuthology.orchestra.run.smithi159.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-10T09:25:57.844 INFO:teuthology.orchestra.run.smithi159.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-10T09:25:57.844 INFO:teuthology.orchestra.run.smithi159.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-10T09:25:57.844 INFO:teuthology.orchestra.run.smithi159.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-10T09:25:57.844 INFO:teuthology.orchestra.run.smithi159.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-10T09:25:57.844 INFO:teuthology.orchestra.run.smithi159.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-10T09:25:57.844 INFO:teuthology.orchestra.run.smithi159.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-10T09:25:57.844 INFO:teuthology.orchestra.run.smithi159.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-10T09:25:57.844 INFO:teuthology.orchestra.run.smithi159.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-10T09:25:57.844 INFO:teuthology.orchestra.run.smithi159.stdout: python3-rados-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T09:25:57.844 INFO:teuthology.orchestra.run.smithi159.stdout: python3-rbd-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T09:25:57.845 INFO:teuthology.orchestra.run.smithi159.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-10T09:25:57.845 INFO:teuthology.orchestra.run.smithi159.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-10T09:25:57.845 INFO:teuthology.orchestra.run.smithi159.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-10T09:25:57.845 INFO:teuthology.orchestra.run.smithi159.stdout: python3-rgw-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T09:25:57.845 INFO:teuthology.orchestra.run.smithi159.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-10T09:25:57.845 INFO:teuthology.orchestra.run.smithi159.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-10T09:25:57.845 INFO:teuthology.orchestra.run.smithi159.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-10T09:25:57.845 INFO:teuthology.orchestra.run.smithi159.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-10T09:25:57.845 INFO:teuthology.orchestra.run.smithi159.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-10T09:25:57.845 INFO:teuthology.orchestra.run.smithi159.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-10T09:25:57.845 INFO:teuthology.orchestra.run.smithi159.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-10T09:25:57.845 INFO:teuthology.orchestra.run.smithi159.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-10T09:25:57.845 INFO:teuthology.orchestra.run.smithi159.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-10T09:25:57.846 INFO:teuthology.orchestra.run.smithi159.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-10T09:25:57.846 INFO:teuthology.orchestra.run.smithi159.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-10T09:25:57.846 INFO:teuthology.orchestra.run.smithi159.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-10T09:25:57.846 INFO:teuthology.orchestra.run.smithi159.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-10T09:25:57.846 INFO:teuthology.orchestra.run.smithi159.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-10T09:25:57.846 INFO:teuthology.orchestra.run.smithi159.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-10T09:25:57.846 INFO:teuthology.orchestra.run.smithi159.stdout: rbd-fuse-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T09:25:57.846 INFO:teuthology.orchestra.run.smithi159.stdout: rbd-mirror-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T09:25:57.846 INFO:teuthology.orchestra.run.smithi159.stdout: rbd-nbd-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T09:25:57.846 INFO:teuthology.orchestra.run.smithi159.stdout: re2-1:20211101-20.el9.x86_64 2024-09-10T09:25:57.846 INFO:teuthology.orchestra.run.smithi159.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-10T09:25:57.846 INFO:teuthology.orchestra.run.smithi159.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-10T09:25:57.846 INFO:teuthology.orchestra.run.smithi159.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-10T09:25:57.847 INFO:teuthology.orchestra.run.smithi159.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-10T09:25:57.847 INFO:teuthology.orchestra.run.smithi159.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-10T09:25:57.847 INFO:teuthology.orchestra.run.smithi159.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-10T09:25:57.847 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-10T09:25:57.847 INFO:teuthology.orchestra.run.smithi159.stdout:Complete! 2024-09-10T09:25:58.261 DEBUG:teuthology.parallel:result is None 2024-09-10T09:26:01.531 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-test-2:19.1.1-305.g1cf819b7.el9.x86_64 141/164 2024-09-10T09:26:01.582 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : rbd-fuse-2:19.1.1-305.g1cf819b7.el9.x86_64 142/164 2024-09-10T09:26:01.649 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : rbd-nbd-2:19.1.1-305.g1cf819b7.el9.x86_64 143/164 2024-09-10T09:26:01.696 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 144/164 2024-09-10T09:26:01.731 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 145/164 2024-09-10T09:26:01.980 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : cephfs-shell-2:19.1.1-305.g1cf819b7.el9.x86_64 146/164 2024-09-10T09:26:02.028 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 147/164 2024-09-10T09:26:02.058 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : libcephfs-devel-2:19.1.1-305.g1cf819b7.el9.x86_6 148/164 2024-09-10T09:26:02.089 INFO:teuthology.orchestra.run.smithi005.stdout: Upgrading : NetworkManager-team-1:1.49.90-1.el9.x86_64 149/164 2024-09-10T09:26:02.134 INFO:teuthology.orchestra.run.smithi005.stdout: Upgrading : NetworkManager-tui-1:1.49.90-1.el9.x86_64 150/164 2024-09-10T09:26:02.971 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-10T09:26:03.024 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-10T09:26:03.066 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-10T09:26:04.457 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : cephfs-top-2:19.1.1-305.g1cf819b7.el9.noarch 152/164 2024-09-10T09:26:04.563 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 153/164 2024-09-10T09:26:04.618 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-09-10T09:26:04.740 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-09-10T09:26:04.774 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-09-10T09:26:04.799 INFO:teuthology.orchestra.run.smithi005.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 157/164 2024-09-10T09:26:04.836 INFO:teuthology.orchestra.run.smithi005.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 158/164 2024-09-10T09:26:04.836 INFO:teuthology.orchestra.run.smithi005.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-10T09:26:04.985 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-10T09:26:04.986 INFO:teuthology.orchestra.run.smithi005.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-10T09:26:05.136 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-10T09:26:05.168 INFO:teuthology.orchestra.run.smithi005.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 161/164 2024-09-10T09:26:05.185 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-10T09:26:05.185 INFO:teuthology.orchestra.run.smithi005.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-10T09:26:05.429 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-10T09:26:05.430 INFO:teuthology.orchestra.run.smithi005.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-10T09:26:05.581 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-10T09:26:05.658 INFO:teuthology.orchestra.run.smithi005.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-10T09:26:05.918 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-10T09:26:05.918 INFO:teuthology.orchestra.run.smithi005.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-09-10T09:26:05.918 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-10T09:26:11.679 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-2:19.1.1-305.g1cf819b7.el9.x86_64 1/164 2024-09-10T09:26:11.680 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-base-2:19.1.1-305.g1cf819b7.el9.x86_64 2/164 2024-09-10T09:26:11.680 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-common-2:19.1.1-305.g1cf819b7.el9.x86_64 3/164 2024-09-10T09:26:11.680 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-fuse-2:19.1.1-305.g1cf819b7.el9.x86_64 4/164 2024-09-10T09:26:11.680 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-immutable-object-cache-2:19.1.1-305.g1cf819 5/164 2024-09-10T09:26:11.680 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-mds-2:19.1.1-305.g1cf819b7.el9.x86_64 6/164 2024-09-10T09:26:11.680 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-mgr-2:19.1.1-305.g1cf819b7.el9.x86_64 7/164 2024-09-10T09:26:11.680 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-mon-2:19.1.1-305.g1cf819b7.el9.x86_64 8/164 2024-09-10T09:26:11.680 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-osd-2:19.1.1-305.g1cf819b7.el9.x86_64 9/164 2024-09-10T09:26:11.680 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-radosgw-2:19.1.1-305.g1cf819b7.el9.x86_64 10/164 2024-09-10T09:26:11.680 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-selinux-2:19.1.1-305.g1cf819b7.el9.x86_64 11/164 2024-09-10T09:26:11.680 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-test-2:19.1.1-305.g1cf819b7.el9.x86_64 12/164 2024-09-10T09:26:11.681 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : cephfs-mirror-2:19.1.1-305.g1cf819b7.el9.x86_64 13/164 2024-09-10T09:26:11.681 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : cephfs-shell-2:19.1.1-305.g1cf819b7.el9.x86_64 14/164 2024-09-10T09:26:11.681 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : libcephfs-devel-2:19.1.1-305.g1cf819b7.el9.x86_6 15/164 2024-09-10T09:26:11.681 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : libcephfs2-2:19.1.1-305.g1cf819b7.el9.x86_64 16/164 2024-09-10T09:26:11.681 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : libcephsqlite-2:19.1.1-305.g1cf819b7.el9.x86_64 17/164 2024-09-10T09:26:11.681 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : librados-devel-2:19.1.1-305.g1cf819b7.el9.x86_64 18/164 2024-09-10T09:26:11.681 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : libradosstriper1-2:19.1.1-305.g1cf819b7.el9.x86_ 19/164 2024-09-10T09:26:11.681 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : librgw2-2:19.1.1-305.g1cf819b7.el9.x86_64 20/164 2024-09-10T09:26:11.681 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-ceph-argparse-2:19.1.1-305.g1cf819b7.el9 21/164 2024-09-10T09:26:11.681 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-ceph-common-2:19.1.1-305.g1cf819b7.el9.x 22/164 2024-09-10T09:26:11.681 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-cephfs-2:19.1.1-305.g1cf819b7.el9.x86_64 23/164 2024-09-10T09:26:11.681 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-rados-2:19.1.1-305.g1cf819b7.el9.x86_64 24/164 2024-09-10T09:26:11.681 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-rbd-2:19.1.1-305.g1cf819b7.el9.x86_64 25/164 2024-09-10T09:26:11.682 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-rgw-2:19.1.1-305.g1cf819b7.el9.x86_64 26/164 2024-09-10T09:26:11.682 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : rbd-fuse-2:19.1.1-305.g1cf819b7.el9.x86_64 27/164 2024-09-10T09:26:11.682 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : rbd-mirror-2:19.1.1-305.g1cf819b7.el9.x86_64 28/164 2024-09-10T09:26:11.682 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : rbd-nbd-2:19.1.1-305.g1cf819b7.el9.x86_64 29/164 2024-09-10T09:26:11.693 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-grafana-dashboards-2:19.1.1-305.g1cf819b7.e 30/164 2024-09-10T09:26:11.694 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-mgr-cephadm-2:19.1.1-305.g1cf819b7.el9.noar 31/164 2024-09-10T09:26:11.694 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-mgr-dashboard-2:19.1.1-305.g1cf819b7.el9.no 32/164 2024-09-10T09:26:11.694 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.1.1-305.g1cf8 33/164 2024-09-10T09:26:11.694 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-mgr-modules-core-2:19.1.1-305.g1cf819b7.el9 34/164 2024-09-10T09:26:11.694 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-mgr-rook-2:19.1.1-305.g1cf819b7.el9.noarch 35/164 2024-09-10T09:26:11.694 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-prometheus-alerts-2:19.1.1-305.g1cf819b7.el 36/164 2024-09-10T09:26:11.694 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-volume-2:19.1.1-305.g1cf819b7.el9.noarch 37/164 2024-09-10T09:26:11.694 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : cephadm-2:19.1.1-305.g1cf819b7.el9.noarch 38/164 2024-09-10T09:26:11.694 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : cephfs-top-2:19.1.1-305.g1cf819b7.el9.noarch 39/164 2024-09-10T09:26:11.694 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/164 2024-09-10T09:26:11.694 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/164 2024-09-10T09:26:11.695 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/164 2024-09-10T09:26:11.695 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-09-10T09:26:11.695 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-09-10T09:26:11.695 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-09-10T09:26:11.695 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-09-10T09:26:11.695 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-09-10T09:26:11.695 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/164 2024-09-10T09:26:11.695 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-09-10T09:26:11.695 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-09-10T09:26:11.695 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-09-10T09:26:11.695 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/164 2024-09-10T09:26:11.695 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-09-10T09:26:11.696 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-09-10T09:26:11.696 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-09-10T09:26:11.696 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-09-10T09:26:11.696 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-09-10T09:26:11.696 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-09-10T09:26:11.696 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-09-10T09:26:11.696 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-09-10T09:26:11.696 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 61/164 2024-09-10T09:26:11.696 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 62/164 2024-09-10T09:26:11.696 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-09-10T09:26:11.696 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-09-10T09:26:11.696 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-09-10T09:26:11.697 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 66/164 2024-09-10T09:26:11.697 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 67/164 2024-09-10T09:26:11.697 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 68/164 2024-09-10T09:26:11.697 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : postgresql-13.16-1.el9.x86_64 69/164 2024-09-10T09:26:11.699 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 70/164 2024-09-10T09:26:11.699 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 71/164 2024-09-10T09:26:11.700 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 72/164 2024-09-10T09:26:11.700 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-09-10T09:26:11.700 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-09-10T09:26:11.700 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-09-10T09:26:11.700 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/164 2024-09-10T09:26:11.700 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 77/164 2024-09-10T09:26:11.700 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/164 2024-09-10T09:26:11.700 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/164 2024-09-10T09:26:11.700 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/164 2024-09-10T09:26:11.701 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/164 2024-09-10T09:26:11.701 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/164 2024-09-10T09:26:11.701 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/164 2024-09-10T09:26:11.701 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/164 2024-09-10T09:26:11.701 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/164 2024-09-10T09:26:11.701 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/164 2024-09-10T09:26:11.701 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/164 2024-09-10T09:26:11.701 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/164 2024-09-10T09:26:11.701 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/164 2024-09-10T09:26:11.701 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/164 2024-09-10T09:26:11.701 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/164 2024-09-10T09:26:11.701 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/164 2024-09-10T09:26:11.701 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/164 2024-09-10T09:26:11.702 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/164 2024-09-10T09:26:11.702 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/164 2024-09-10T09:26:11.702 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/164 2024-09-10T09:26:11.702 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/164 2024-09-10T09:26:11.702 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/164 2024-09-10T09:26:11.702 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 99/164 2024-09-10T09:26:11.702 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/164 2024-09-10T09:26:11.702 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 101/164 2024-09-10T09:26:11.702 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/164 2024-09-10T09:26:11.702 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/164 2024-09-10T09:26:11.702 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/164 2024-09-10T09:26:11.702 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/164 2024-09-10T09:26:11.702 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/164 2024-09-10T09:26:11.703 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 107/164 2024-09-10T09:26:11.703 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/164 2024-09-10T09:26:11.703 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/164 2024-09-10T09:26:11.703 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/164 2024-09-10T09:26:11.703 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/164 2024-09-10T09:26:11.703 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/164 2024-09-10T09:26:11.703 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/164 2024-09-10T09:26:11.703 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/164 2024-09-10T09:26:11.703 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/164 2024-09-10T09:26:11.704 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/164 2024-09-10T09:26:11.704 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/164 2024-09-10T09:26:11.704 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 118/164 2024-09-10T09:26:11.704 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/164 2024-09-10T09:26:11.704 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/164 2024-09-10T09:26:11.704 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/164 2024-09-10T09:26:11.704 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 122/164 2024-09-10T09:26:11.704 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-09-10T09:26:11.704 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-09-10T09:26:11.704 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-09-10T09:26:11.704 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-09-10T09:26:11.704 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-09-10T09:26:11.704 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-09-10T09:26:11.705 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-09-10T09:26:11.705 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-09-10T09:26:11.705 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-09-10T09:26:11.705 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-09-10T09:26:11.705 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-09-10T09:26:11.705 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-09-10T09:26:11.705 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-09-10T09:26:11.705 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-09-10T09:26:11.706 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-09-10T09:26:11.706 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-09-10T09:26:11.706 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-09-10T09:26:11.706 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-09-10T09:26:11.706 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-09-10T09:26:11.706 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 142/164 2024-09-10T09:26:11.706 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 143/164 2024-09-10T09:26:11.706 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-09-10T09:26:11.706 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-09-10T09:26:11.706 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-09-10T09:26:11.706 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : re2-1:20211101-20.el9.x86_64 147/164 2024-09-10T09:26:11.706 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-09-10T09:26:11.706 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : librados2-2:19.1.1-305.g1cf819b7.el9.x86_64 149/164 2024-09-10T09:26:11.707 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-09-10T09:26:11.707 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : librbd1-2:19.1.1-305.g1cf819b7.el9.x86_64 151/164 2024-09-10T09:26:11.707 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-09-10T09:26:11.707 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : NetworkManager-1:1.49.90-1.el9.x86_64 153/164 2024-09-10T09:26:11.707 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 154/164 2024-09-10T09:26:11.707 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 155/164 2024-09-10T09:26:11.707 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 156/164 2024-09-10T09:26:11.707 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : NetworkManager-team-1:1.49.90-1.el9.x86_64 157/164 2024-09-10T09:26:11.707 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 158/164 2024-09-10T09:26:11.707 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : NetworkManager-tui-1:1.49.90-1.el9.x86_64 159/164 2024-09-10T09:26:11.708 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 160/164 2024-09-10T09:26:11.708 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 161/164 2024-09-10T09:26:11.708 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 162/164 2024-09-10T09:26:11.708 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 163/164 2024-09-10T09:26:13.886 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-10T09:26:13.886 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-10T09:26:13.887 INFO:teuthology.orchestra.run.smithi005.stdout:Upgraded: 2024-09-10T09:26:13.887 INFO:teuthology.orchestra.run.smithi005.stdout: NetworkManager-1:1.49.90-1.el9.x86_64 2024-09-10T09:26:13.887 INFO:teuthology.orchestra.run.smithi005.stdout: NetworkManager-libnm-1:1.49.90-1.el9.x86_64 2024-09-10T09:26:13.887 INFO:teuthology.orchestra.run.smithi005.stdout: NetworkManager-team-1:1.49.90-1.el9.x86_64 2024-09-10T09:26:13.887 INFO:teuthology.orchestra.run.smithi005.stdout: NetworkManager-tui-1:1.49.90-1.el9.x86_64 2024-09-10T09:26:13.887 INFO:teuthology.orchestra.run.smithi005.stdout: librados2-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T09:26:13.887 INFO:teuthology.orchestra.run.smithi005.stdout: librbd1-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T09:26:13.887 INFO:teuthology.orchestra.run.smithi005.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-10T09:26:13.887 INFO:teuthology.orchestra.run.smithi005.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-10T09:26:13.887 INFO:teuthology.orchestra.run.smithi005.stdout:Installed: 2024-09-10T09:26:13.887 INFO:teuthology.orchestra.run.smithi005.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-10T09:26:13.887 INFO:teuthology.orchestra.run.smithi005.stdout: automake-1.16.2-8.el9.noarch 2024-09-10T09:26:13.887 INFO:teuthology.orchestra.run.smithi005.stdout: bison-3.7.4-5.el9.x86_64 2024-09-10T09:26:13.888 INFO:teuthology.orchestra.run.smithi005.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-10T09:26:13.888 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T09:26:13.888 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-base-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T09:26:13.888 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-common-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T09:26:13.888 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-fuse-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T09:26:13.888 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-grafana-dashboards-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T09:26:13.888 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-immutable-object-cache-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T09:26:13.888 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-mds-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T09:26:13.888 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-mgr-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T09:26:13.888 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-mgr-cephadm-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T09:26:13.888 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-mgr-dashboard-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T09:26:13.888 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-mgr-diskprediction-local-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T09:26:13.888 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-mgr-modules-core-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T09:26:13.889 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-mgr-rook-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T09:26:13.889 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-mon-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T09:26:13.889 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-osd-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T09:26:13.889 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-prometheus-alerts-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T09:26:13.889 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-radosgw-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T09:26:13.889 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-selinux-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T09:26:13.889 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-test-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T09:26:13.889 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-volume-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T09:26:13.889 INFO:teuthology.orchestra.run.smithi005.stdout: cephadm-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T09:26:13.889 INFO:teuthology.orchestra.run.smithi005.stdout: cephfs-mirror-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T09:26:13.889 INFO:teuthology.orchestra.run.smithi005.stdout: cephfs-shell-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T09:26:13.889 INFO:teuthology.orchestra.run.smithi005.stdout: cephfs-top-2:19.1.1-305.g1cf819b7.el9.noarch 2024-09-10T09:26:13.890 INFO:teuthology.orchestra.run.smithi005.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-10T09:26:13.890 INFO:teuthology.orchestra.run.smithi005.stdout: flex-2.6.4-9.el9.x86_64 2024-09-10T09:26:13.890 INFO:teuthology.orchestra.run.smithi005.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-10T09:26:13.890 INFO:teuthology.orchestra.run.smithi005.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-10T09:26:13.890 INFO:teuthology.orchestra.run.smithi005.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-10T09:26:13.890 INFO:teuthology.orchestra.run.smithi005.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-10T09:26:13.890 INFO:teuthology.orchestra.run.smithi005.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-09-10T09:26:13.890 INFO:teuthology.orchestra.run.smithi005.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-10T09:26:13.890 INFO:teuthology.orchestra.run.smithi005.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-10T09:26:13.890 INFO:teuthology.orchestra.run.smithi005.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-10T09:26:13.890 INFO:teuthology.orchestra.run.smithi005.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-10T09:26:13.890 INFO:teuthology.orchestra.run.smithi005.stdout: libcephfs-devel-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T09:26:13.890 INFO:teuthology.orchestra.run.smithi005.stdout: libcephfs2-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T09:26:13.890 INFO:teuthology.orchestra.run.smithi005.stdout: libcephsqlite-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T09:26:13.891 INFO:teuthology.orchestra.run.smithi005.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-10T09:26:13.891 INFO:teuthology.orchestra.run.smithi005.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-10T09:26:13.891 INFO:teuthology.orchestra.run.smithi005.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-10T09:26:13.891 INFO:teuthology.orchestra.run.smithi005.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-10T09:26:13.891 INFO:teuthology.orchestra.run.smithi005.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-10T09:26:13.891 INFO:teuthology.orchestra.run.smithi005.stdout: librados-devel-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T09:26:13.891 INFO:teuthology.orchestra.run.smithi005.stdout: libradosstriper1-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T09:26:13.891 INFO:teuthology.orchestra.run.smithi005.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-10T09:26:13.891 INFO:teuthology.orchestra.run.smithi005.stdout: librgw2-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T09:26:13.891 INFO:teuthology.orchestra.run.smithi005.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-10T09:26:13.891 INFO:teuthology.orchestra.run.smithi005.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-10T09:26:13.891 INFO:teuthology.orchestra.run.smithi005.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-10T09:26:13.891 INFO:teuthology.orchestra.run.smithi005.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-10T09:26:13.892 INFO:teuthology.orchestra.run.smithi005.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-10T09:26:13.892 INFO:teuthology.orchestra.run.smithi005.stdout: lua-5.4.4-4.el9.x86_64 2024-09-10T09:26:13.892 INFO:teuthology.orchestra.run.smithi005.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-10T09:26:13.892 INFO:teuthology.orchestra.run.smithi005.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-10T09:26:13.892 INFO:teuthology.orchestra.run.smithi005.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-10T09:26:13.892 INFO:teuthology.orchestra.run.smithi005.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-10T09:26:13.892 INFO:teuthology.orchestra.run.smithi005.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-10T09:26:13.892 INFO:teuthology.orchestra.run.smithi005.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-10T09:26:13.892 INFO:teuthology.orchestra.run.smithi005.stdout: postgresql-13.16-1.el9.x86_64 2024-09-10T09:26:13.892 INFO:teuthology.orchestra.run.smithi005.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-10T09:26:13.892 INFO:teuthology.orchestra.run.smithi005.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-10T09:26:13.892 INFO:teuthology.orchestra.run.smithi005.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-10T09:26:13.892 INFO:teuthology.orchestra.run.smithi005.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-10T09:26:13.893 INFO:teuthology.orchestra.run.smithi005.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-10T09:26:13.893 INFO:teuthology.orchestra.run.smithi005.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-10T09:26:13.893 INFO:teuthology.orchestra.run.smithi005.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-10T09:26:13.893 INFO:teuthology.orchestra.run.smithi005.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-10T09:26:13.893 INFO:teuthology.orchestra.run.smithi005.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-10T09:26:13.893 INFO:teuthology.orchestra.run.smithi005.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-10T09:26:13.893 INFO:teuthology.orchestra.run.smithi005.stdout: python3-ceph-argparse-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T09:26:13.893 INFO:teuthology.orchestra.run.smithi005.stdout: python3-ceph-common-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T09:26:13.893 INFO:teuthology.orchestra.run.smithi005.stdout: python3-cephfs-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T09:26:13.893 INFO:teuthology.orchestra.run.smithi005.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-10T09:26:13.893 INFO:teuthology.orchestra.run.smithi005.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-10T09:26:13.893 INFO:teuthology.orchestra.run.smithi005.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-10T09:26:13.893 INFO:teuthology.orchestra.run.smithi005.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-10T09:26:13.894 INFO:teuthology.orchestra.run.smithi005.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-10T09:26:13.894 INFO:teuthology.orchestra.run.smithi005.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-10T09:26:13.894 INFO:teuthology.orchestra.run.smithi005.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-10T09:26:13.894 INFO:teuthology.orchestra.run.smithi005.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-10T09:26:13.894 INFO:teuthology.orchestra.run.smithi005.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-10T09:26:13.894 INFO:teuthology.orchestra.run.smithi005.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-10T09:26:13.894 INFO:teuthology.orchestra.run.smithi005.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-10T09:26:13.894 INFO:teuthology.orchestra.run.smithi005.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-10T09:26:13.894 INFO:teuthology.orchestra.run.smithi005.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-10T09:26:13.894 INFO:teuthology.orchestra.run.smithi005.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-10T09:26:13.894 INFO:teuthology.orchestra.run.smithi005.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-10T09:26:13.895 INFO:teuthology.orchestra.run.smithi005.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-10T09:26:13.895 INFO:teuthology.orchestra.run.smithi005.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-10T09:26:13.895 INFO:teuthology.orchestra.run.smithi005.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-10T09:26:13.895 INFO:teuthology.orchestra.run.smithi005.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-10T09:26:13.895 INFO:teuthology.orchestra.run.smithi005.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-10T09:26:13.895 INFO:teuthology.orchestra.run.smithi005.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-10T09:26:13.895 INFO:teuthology.orchestra.run.smithi005.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-10T09:26:13.895 INFO:teuthology.orchestra.run.smithi005.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-10T09:26:13.895 INFO:teuthology.orchestra.run.smithi005.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-10T09:26:13.895 INFO:teuthology.orchestra.run.smithi005.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-10T09:26:13.895 INFO:teuthology.orchestra.run.smithi005.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-10T09:26:13.895 INFO:teuthology.orchestra.run.smithi005.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-10T09:26:13.895 INFO:teuthology.orchestra.run.smithi005.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-10T09:26:13.895 INFO:teuthology.orchestra.run.smithi005.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-10T09:26:13.896 INFO:teuthology.orchestra.run.smithi005.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-10T09:26:13.896 INFO:teuthology.orchestra.run.smithi005.stdout: python3-ply-3.11-14.el9.noarch 2024-09-10T09:26:13.896 INFO:teuthology.orchestra.run.smithi005.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-10T09:26:13.896 INFO:teuthology.orchestra.run.smithi005.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-10T09:26:13.896 INFO:teuthology.orchestra.run.smithi005.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-10T09:26:13.896 INFO:teuthology.orchestra.run.smithi005.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-10T09:26:13.896 INFO:teuthology.orchestra.run.smithi005.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-10T09:26:13.896 INFO:teuthology.orchestra.run.smithi005.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-10T09:26:13.896 INFO:teuthology.orchestra.run.smithi005.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-10T09:26:13.896 INFO:teuthology.orchestra.run.smithi005.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-10T09:26:13.896 INFO:teuthology.orchestra.run.smithi005.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-10T09:26:13.896 INFO:teuthology.orchestra.run.smithi005.stdout: python3-rados-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T09:26:13.897 INFO:teuthology.orchestra.run.smithi005.stdout: python3-rbd-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T09:26:13.897 INFO:teuthology.orchestra.run.smithi005.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-10T09:26:13.897 INFO:teuthology.orchestra.run.smithi005.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-10T09:26:13.897 INFO:teuthology.orchestra.run.smithi005.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-10T09:26:13.897 INFO:teuthology.orchestra.run.smithi005.stdout: python3-rgw-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T09:26:13.897 INFO:teuthology.orchestra.run.smithi005.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-10T09:26:13.897 INFO:teuthology.orchestra.run.smithi005.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-10T09:26:13.897 INFO:teuthology.orchestra.run.smithi005.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-10T09:26:13.897 INFO:teuthology.orchestra.run.smithi005.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-10T09:26:13.897 INFO:teuthology.orchestra.run.smithi005.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-10T09:26:13.897 INFO:teuthology.orchestra.run.smithi005.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-10T09:26:13.897 INFO:teuthology.orchestra.run.smithi005.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-10T09:26:13.897 INFO:teuthology.orchestra.run.smithi005.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-10T09:26:13.897 INFO:teuthology.orchestra.run.smithi005.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-10T09:26:13.898 INFO:teuthology.orchestra.run.smithi005.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-10T09:26:13.898 INFO:teuthology.orchestra.run.smithi005.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-10T09:26:13.898 INFO:teuthology.orchestra.run.smithi005.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-10T09:26:13.898 INFO:teuthology.orchestra.run.smithi005.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-10T09:26:13.898 INFO:teuthology.orchestra.run.smithi005.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-10T09:26:13.898 INFO:teuthology.orchestra.run.smithi005.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-10T09:26:13.898 INFO:teuthology.orchestra.run.smithi005.stdout: rbd-fuse-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T09:26:13.898 INFO:teuthology.orchestra.run.smithi005.stdout: rbd-mirror-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T09:26:13.898 INFO:teuthology.orchestra.run.smithi005.stdout: rbd-nbd-2:19.1.1-305.g1cf819b7.el9.x86_64 2024-09-10T09:26:13.898 INFO:teuthology.orchestra.run.smithi005.stdout: re2-1:20211101-20.el9.x86_64 2024-09-10T09:26:13.898 INFO:teuthology.orchestra.run.smithi005.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-10T09:26:13.898 INFO:teuthology.orchestra.run.smithi005.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-10T09:26:13.898 INFO:teuthology.orchestra.run.smithi005.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-10T09:26:13.899 INFO:teuthology.orchestra.run.smithi005.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-10T09:26:13.899 INFO:teuthology.orchestra.run.smithi005.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-10T09:26:13.899 INFO:teuthology.orchestra.run.smithi005.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-10T09:26:13.899 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-10T09:26:13.899 INFO:teuthology.orchestra.run.smithi005.stdout:Complete! 2024-09-10T09:26:14.669 DEBUG:teuthology.parallel:result is None 2024-09-10T09:26:14.670 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1cf819b7212a6048e92fc78c4eff963344f6d56b 2024-09-10T09:26:14.814 DEBUG:teuthology.orchestra.run.smithi005:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-10T09:26:14.861 INFO:teuthology.orchestra.run.smithi005.stdout:19.1.1-305.g1cf819b7.el9 2024-09-10T09:26:14.862 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-305.g1cf819b7.el9 2024-09-10T09:26:14.862 INFO:teuthology.task.install:The correct ceph version 19.1.1-305.g1cf819b7 is installed. 2024-09-10T09:26:14.864 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1cf819b7212a6048e92fc78c4eff963344f6d56b 2024-09-10T09:26:15.013 DEBUG:teuthology.orchestra.run.smithi102:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-10T09:26:15.039 INFO:teuthology.orchestra.run.smithi102.stdout:19.1.1-305.g1cf819b7.el9 2024-09-10T09:26:15.039 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-305.g1cf819b7.el9 2024-09-10T09:26:15.039 INFO:teuthology.task.install:The correct ceph version 19.1.1-305.g1cf819b7 is installed. 2024-09-10T09:26:15.041 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1cf819b7212a6048e92fc78c4eff963344f6d56b 2024-09-10T09:26:15.190 DEBUG:teuthology.orchestra.run.smithi159:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-10T09:26:15.217 INFO:teuthology.orchestra.run.smithi159.stdout:19.1.1-305.g1cf819b7.el9 2024-09-10T09:26:15.217 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-305.g1cf819b7.el9 2024-09-10T09:26:15.217 INFO:teuthology.task.install:The correct ceph version 19.1.1-305.g1cf819b7 is installed. 2024-09-10T09:26:15.219 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-09-10T09:26:15.219 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-09-10T09:26:15.219 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-10T09:26:15.279 DEBUG:teuthology.orchestra.run.smithi102:> set -ex 2024-09-10T09:26:15.279 DEBUG:teuthology.orchestra.run.smithi102:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-10T09:26:15.310 DEBUG:teuthology.orchestra.run.smithi159:> set -ex 2024-09-10T09:26:15.310 DEBUG:teuthology.orchestra.run.smithi159:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-10T09:26:15.343 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-09-10T09:26:15.344 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-09-10T09:26:15.344 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd of=/usr/bin/daemon-helper 2024-09-10T09:26:15.406 DEBUG:teuthology.orchestra.run.smithi005:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-10T09:26:15.502 DEBUG:teuthology.orchestra.run.smithi102:> set -ex 2024-09-10T09:26:15.502 DEBUG:teuthology.orchestra.run.smithi102:> sudo dd of=/usr/bin/daemon-helper 2024-09-10T09:26:15.531 DEBUG:teuthology.orchestra.run.smithi102:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-10T09:26:15.596 DEBUG:teuthology.orchestra.run.smithi159:> set -ex 2024-09-10T09:26:15.596 DEBUG:teuthology.orchestra.run.smithi159:> sudo dd of=/usr/bin/daemon-helper 2024-09-10T09:26:15.627 DEBUG:teuthology.orchestra.run.smithi159:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-10T09:26:15.693 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-09-10T09:26:15.694 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-09-10T09:26:15.694 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-10T09:26:15.751 DEBUG:teuthology.orchestra.run.smithi005:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-10T09:26:15.849 DEBUG:teuthology.orchestra.run.smithi102:> set -ex 2024-09-10T09:26:15.849 DEBUG:teuthology.orchestra.run.smithi102:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-10T09:26:15.876 DEBUG:teuthology.orchestra.run.smithi102:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-10T09:26:15.944 DEBUG:teuthology.orchestra.run.smithi159:> set -ex 2024-09-10T09:26:15.944 DEBUG:teuthology.orchestra.run.smithi159:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-10T09:26:15.968 DEBUG:teuthology.orchestra.run.smithi159:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-10T09:26:16.034 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-09-10T09:26:16.034 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-09-10T09:26:16.035 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd of=/usr/bin/stdin-killer 2024-09-10T09:26:16.093 DEBUG:teuthology.orchestra.run.smithi005:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-10T09:26:16.196 DEBUG:teuthology.orchestra.run.smithi102:> set -ex 2024-09-10T09:26:16.196 DEBUG:teuthology.orchestra.run.smithi102:> sudo dd of=/usr/bin/stdin-killer 2024-09-10T09:26:16.221 DEBUG:teuthology.orchestra.run.smithi102:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-10T09:26:16.287 DEBUG:teuthology.orchestra.run.smithi159:> set -ex 2024-09-10T09:26:16.287 DEBUG:teuthology.orchestra.run.smithi159:> sudo dd of=/usr/bin/stdin-killer 2024-09-10T09:26:16.313 DEBUG:teuthology.orchestra.run.smithi159:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-10T09:26:16.378 INFO:teuthology.run_tasks:Running task cephadm... 2024-09-10T09:26:16.476 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 5, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_export_ephemeral_random_max': 0.1, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120, 'mon warn on pool no app': False}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10000}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'online, but wants', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy', 'slow metadata IO', 'SLOW_OPS', 'slow request'], 'sha1': '1cf819b7212a6048e92fc78c4eff963344f6d56b', 'subvols': {'create': 2}} 2024-09-10T09:26:16.477 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b 2024-09-10T09:26:16.477 INFO:tasks.cephadm:Cluster fsid is bade4b50-6f56-11ef-bcea-c7b262605968 2024-09-10T09:26:16.477 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-09-10T09:26:16.477 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.5', 'mon.b': '172.21.15.102', 'mon.c': '172.21.15.159'} 2024-09-10T09:26:16.478 INFO:tasks.cephadm:First mon is mon.a on smithi005 2024-09-10T09:26:16.478 INFO:tasks.cephadm:First mgr is x 2024-09-10T09:26:16.478 INFO:tasks.cephadm:Normalizing hostnames... 2024-09-10T09:26:16.478 DEBUG:teuthology.orchestra.run.smithi005:> sudo hostname $(hostname -s) 2024-09-10T09:26:16.535 DEBUG:teuthology.orchestra.run.smithi102:> sudo hostname $(hostname -s) 2024-09-10T09:26:16.560 DEBUG:teuthology.orchestra.run.smithi159:> sudo hostname $(hostname -s) 2024-09-10T09:26:16.585 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-09-10T09:26:16.586 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1cf819b7212a6048e92fc78c4eff963344f6d56b 2024-09-10T09:26:16.727 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': '1cf819b7212a6048e92fc78c4eff963344f6d56b', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/82914/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.1.1-305-g1cf819b7', 'node_name': '172.21.5.35+adami05', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.1.1-305.g1cf819b7'}, 'url': 'https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-09-09 22:14:38.445020', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-rishabh-testing-20240909.165402-squid', 'chacra_url': 'https://1.chacra.ceph.com/repos/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/centos/9/flavors/default/', 'archs': ['x86_64', 'source'], 'distro': 'centos'}, {'status': 'ready', 'sha1': '1cf819b7212a6048e92fc78c4eff963344f6d56b', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/82912/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.1.1-305-g1cf819b7', 'node_name': '172.21.2.7+braggi07', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.1.1-305.g1cf819b7'}, 'url': 'https://4.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-09-09 22:05:03.996701', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-rishabh-testing-20240909.165402-squid', 'chacra_url': 'https://4.chacra.ceph.com/repos/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/centos/9/flavors/default/', 'archs': ['x86_64', 'source'], 'distro': 'centos'}] 2024-09-10T09:26:16.865 INFO:tasks.util.chacra:got chacra host 1.chacra.ceph.com, ref wip-rishabh-testing-20240909.165402-squid, sha1 1cf819b7212a6048e92fc78c4eff963344f6d56b from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=1cf819b7212a6048e92fc78c4eff963344f6d56b 2024-09-10T09:26:16.866 INFO:tasks.cephadm:Discovered cachra url: https://1.chacra.ceph.com/binaries/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/centos/9/x86_64/flavors/default/cephadm 2024-09-10T09:26:16.866 INFO:tasks.cephadm:Downloading cephadm from url: https://1.chacra.ceph.com/binaries/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/centos/9/x86_64/flavors/default/cephadm 2024-09-10T09:26:16.867 DEBUG:teuthology.orchestra.run.smithi005:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-10T09:26:17.267 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-r--r--. 1 ubuntu ubuntu 784613 Sep 10 09:26 /home/ubuntu/cephtest/cephadm 2024-09-10T09:26:17.267 DEBUG:teuthology.orchestra.run.smithi102:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-10T09:26:17.654 INFO:teuthology.orchestra.run.smithi102.stdout:-rw-r--r--. 1 ubuntu ubuntu 784613 Sep 10 09:26 /home/ubuntu/cephtest/cephadm 2024-09-10T09:26:17.655 DEBUG:teuthology.orchestra.run.smithi159:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-10T09:26:18.096 INFO:teuthology.orchestra.run.smithi159.stdout:-rw-r--r--. 1 ubuntu ubuntu 784613 Sep 10 09:26 /home/ubuntu/cephtest/cephadm 2024-09-10T09:26:18.096 DEBUG:teuthology.orchestra.run.smithi005:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-10T09:26:18.135 DEBUG:teuthology.orchestra.run.smithi102:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-10T09:26:18.156 DEBUG:teuthology.orchestra.run.smithi159:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-10T09:26:18.191 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b on all hosts... 2024-09-10T09:26:18.191 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b pull 2024-09-10T09:26:18.195 DEBUG:teuthology.orchestra.run.smithi102:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b pull 2024-09-10T09:26:18.198 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b pull 2024-09-10T09:26:19.778 INFO:teuthology.orchestra.run.smithi159.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b... 2024-09-10T09:26:19.805 INFO:teuthology.orchestra.run.smithi102.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b... 2024-09-10T09:26:20.183 INFO:teuthology.orchestra.run.smithi005.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b... 2024-09-10T09:26:46.468 INFO:teuthology.orchestra.run.smithi102.stdout:{ 2024-09-10T09:26:46.469 INFO:teuthology.orchestra.run.smithi102.stdout: "ceph_version": "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)", 2024-09-10T09:26:46.469 INFO:teuthology.orchestra.run.smithi102.stdout: "image_id": "41d67112835771c3efca60c6b8fe297c7ddaecdc26cea26fd28aa18636e49005", 2024-09-10T09:26:46.469 INFO:teuthology.orchestra.run.smithi102.stdout: "repo_digests": [ 2024-09-10T09:26:46.469 INFO:teuthology.orchestra.run.smithi102.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559" 2024-09-10T09:26:46.469 INFO:teuthology.orchestra.run.smithi102.stdout: ] 2024-09-10T09:26:46.469 INFO:teuthology.orchestra.run.smithi102.stdout:} 2024-09-10T09:26:51.986 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-09-10T09:26:51.986 INFO:teuthology.orchestra.run.smithi159.stdout: "ceph_version": "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)", 2024-09-10T09:26:51.986 INFO:teuthology.orchestra.run.smithi159.stdout: "image_id": "41d67112835771c3efca60c6b8fe297c7ddaecdc26cea26fd28aa18636e49005", 2024-09-10T09:26:51.986 INFO:teuthology.orchestra.run.smithi159.stdout: "repo_digests": [ 2024-09-10T09:26:51.986 INFO:teuthology.orchestra.run.smithi159.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559" 2024-09-10T09:26:51.986 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-09-10T09:26:51.986 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-09-10T09:26:52.048 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-10T09:26:52.048 INFO:teuthology.orchestra.run.smithi005.stdout: "ceph_version": "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)", 2024-09-10T09:26:52.049 INFO:teuthology.orchestra.run.smithi005.stdout: "image_id": "41d67112835771c3efca60c6b8fe297c7ddaecdc26cea26fd28aa18636e49005", 2024-09-10T09:26:52.049 INFO:teuthology.orchestra.run.smithi005.stdout: "repo_digests": [ 2024-09-10T09:26:52.049 INFO:teuthology.orchestra.run.smithi005.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559" 2024-09-10T09:26:52.049 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-10T09:26:52.049 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-10T09:26:52.082 DEBUG:teuthology.orchestra.run.smithi005:> sudo mkdir -p /etc/ceph 2024-09-10T09:26:52.154 DEBUG:teuthology.orchestra.run.smithi102:> sudo mkdir -p /etc/ceph 2024-09-10T09:26:52.186 DEBUG:teuthology.orchestra.run.smithi159:> sudo mkdir -p /etc/ceph 2024-09-10T09:26:52.215 DEBUG:teuthology.orchestra.run.smithi005:> sudo chmod 777 /etc/ceph 2024-09-10T09:26:52.271 DEBUG:teuthology.orchestra.run.smithi102:> sudo chmod 777 /etc/ceph 2024-09-10T09:26:52.298 DEBUG:teuthology.orchestra.run.smithi159:> sudo chmod 777 /etc/ceph 2024-09-10T09:26:52.323 INFO:tasks.cephadm:Writing seed config... 2024-09-10T09:26:52.324 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-09-10T09:26:52.324 INFO:tasks.cephadm: override: [client] debug client = 20 2024-09-10T09:26:52.324 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-09-10T09:26:52.324 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-09-10T09:26:52.324 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-09-10T09:26:52.324 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2024-09-10T09:26:52.324 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-09-10T09:26:52.325 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-09-10T09:26:52.325 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-09-10T09:26:52.325 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-09-10T09:26:52.325 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-09-10T09:26:52.325 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-09-10T09:26:52.325 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-09-10T09:26:52.325 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-09-10T09:26:52.325 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-09-10T09:26:52.325 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-09-10T09:26:52.325 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-09-10T09:26:52.325 INFO:tasks.cephadm: override: [mds] mds_export_ephemeral_random_max = 0.1 2024-09-10T09:26:52.325 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-09-10T09:26:52.325 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-09-10T09:26:52.325 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-09-10T09:26:52.326 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-09-10T09:26:52.326 INFO:tasks.cephadm: override: [mgr] debug client = 20 2024-09-10T09:26:52.326 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-09-10T09:26:52.326 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-09-10T09:26:52.326 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-09-10T09:26:52.326 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-09-10T09:26:52.326 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-09-10T09:26:52.326 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-09-10T09:26:52.326 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-09-10T09:26:52.326 INFO:tasks.cephadm: override: [mon] mon warn on pool no app = False 2024-09-10T09:26:52.326 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-09-10T09:26:52.326 INFO:tasks.cephadm: override: [osd] bluestore compression mode = aggressive 2024-09-10T09:26:52.326 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-09-10T09:26:52.327 INFO:tasks.cephadm: override: [osd] debug bluefs = 20 2024-09-10T09:26:52.327 INFO:tasks.cephadm: override: [osd] debug bluestore = 20 2024-09-10T09:26:52.327 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-09-10T09:26:52.327 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-09-10T09:26:52.327 INFO:tasks.cephadm: override: [osd] debug rocksdb = 10 2024-09-10T09:26:52.327 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-09-10T09:26:52.327 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-09-10T09:26:52.327 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-09-10T09:26:52.327 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-09-10T09:26:52.327 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-09-10T09:26:52.327 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-09-10T09:26:52.327 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-09-10T09:26:52.327 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10000 2024-09-10T09:26:52.328 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-09-10T09:26:52.328 DEBUG:teuthology.orchestra.run.smithi005:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-09-10T09:26:52.363 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = bade4b50-6f56-11ef-bcea-c7b262605968 ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bluestore block size = 96636764160 bluestore compression mode = aggressive bluestore fsck on mount = True debug bluefs = 20 debug bluestore = 20 debug ms = 1 debug osd = 20 debug rocksdb = 10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10000 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug client = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 mon op complaint time = 120 mon warn on pool no app = False [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True mds_export_ephemeral_random_max = 0.1 osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-09-10T09:26:52.364 DEBUG:teuthology.orchestra.run.smithi005:mon.a> sudo journalctl -f -n 0 -u ceph-bade4b50-6f56-11ef-bcea-c7b262605968@mon.a.service 2024-09-10T09:26:52.408 DEBUG:teuthology.orchestra.run.smithi005:mgr.x> sudo journalctl -f -n 0 -u ceph-bade4b50-6f56-11ef-bcea-c7b262605968@mgr.x.service 2024-09-10T09:26:52.452 INFO:tasks.cephadm:Bootstrapping... 2024-09-10T09:26:52.452 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b -v bootstrap --fsid bade4b50-6f56-11ef-bcea-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.5 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-09-10T09:26:52.715 INFO:teuthology.orchestra.run.smithi005.stdout:-------------------------------------------------------------------------------- 2024-09-10T09:26:52.715 INFO:teuthology.orchestra.run.smithi005.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b', '-v', 'bootstrap', '--fsid', 'bade4b50-6f56-11ef-bcea-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.5', '--skip-admin-label'] 2024-09-10T09:26:52.716 INFO:teuthology.orchestra.run.smithi005.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-09-10T09:26:52.716 INFO:teuthology.orchestra.run.smithi005.stdout:Verifying podman|docker is present... 2024-09-10T09:26:52.760 INFO:teuthology.orchestra.run.smithi005.stdout:/bin/podman: stdout 5.2.2 2024-09-10T09:26:52.761 INFO:teuthology.orchestra.run.smithi005.stdout:Verifying lvm2 is present... 2024-09-10T09:26:52.761 INFO:teuthology.orchestra.run.smithi005.stdout:Verifying time synchronization is in place... 2024-09-10T09:26:52.775 INFO:teuthology.orchestra.run.smithi005.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-09-10T09:26:52.775 INFO:teuthology.orchestra.run.smithi005.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-09-10T09:26:52.787 INFO:teuthology.orchestra.run.smithi005.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-09-10T09:26:52.787 INFO:teuthology.orchestra.run.smithi005.stdout:systemctl: stdout inactive 2024-09-10T09:26:52.801 INFO:teuthology.orchestra.run.smithi005.stdout:systemctl: stdout enabled 2024-09-10T09:26:52.813 INFO:teuthology.orchestra.run.smithi005.stdout:systemctl: stdout active 2024-09-10T09:26:52.813 INFO:teuthology.orchestra.run.smithi005.stdout:Unit chronyd.service is enabled and running 2024-09-10T09:26:52.813 INFO:teuthology.orchestra.run.smithi005.stdout:Repeating the final host check... 2024-09-10T09:26:52.857 INFO:teuthology.orchestra.run.smithi005.stdout:/bin/podman: stdout 5.2.2 2024-09-10T09:26:52.857 INFO:teuthology.orchestra.run.smithi005.stdout:podman (/bin/podman) version 5.2.2 is present 2024-09-10T09:26:52.857 INFO:teuthology.orchestra.run.smithi005.stdout:systemctl is present 2024-09-10T09:26:52.857 INFO:teuthology.orchestra.run.smithi005.stdout:lvcreate is present 2024-09-10T09:26:52.870 INFO:teuthology.orchestra.run.smithi005.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-09-10T09:26:52.870 INFO:teuthology.orchestra.run.smithi005.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-09-10T09:26:52.882 INFO:teuthology.orchestra.run.smithi005.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-09-10T09:26:52.882 INFO:teuthology.orchestra.run.smithi005.stdout:systemctl: stdout inactive 2024-09-10T09:26:52.896 INFO:teuthology.orchestra.run.smithi005.stdout:systemctl: stdout enabled 2024-09-10T09:26:52.908 INFO:teuthology.orchestra.run.smithi005.stdout:systemctl: stdout active 2024-09-10T09:26:52.909 INFO:teuthology.orchestra.run.smithi005.stdout:Unit chronyd.service is enabled and running 2024-09-10T09:26:52.909 INFO:teuthology.orchestra.run.smithi005.stdout:Host looks OK 2024-09-10T09:26:52.909 INFO:teuthology.orchestra.run.smithi005.stdout:Cluster fsid: bade4b50-6f56-11ef-bcea-c7b262605968 2024-09-10T09:26:52.909 INFO:teuthology.orchestra.run.smithi005.stdout:Acquiring lock 140283401511936 on /run/cephadm/bade4b50-6f56-11ef-bcea-c7b262605968.lock 2024-09-10T09:26:52.909 INFO:teuthology.orchestra.run.smithi005.stdout:Lock 140283401511936 acquired on /run/cephadm/bade4b50-6f56-11ef-bcea-c7b262605968.lock 2024-09-10T09:26:52.909 INFO:teuthology.orchestra.run.smithi005.stdout:Verifying IP 172.21.15.5 port 3300 ... 2024-09-10T09:26:52.910 INFO:teuthology.orchestra.run.smithi005.stdout:Verifying IP 172.21.15.5 port 6789 ... 2024-09-10T09:26:52.910 INFO:teuthology.orchestra.run.smithi005.stdout:Base mon IP(s) is [172.21.15.5:3300, 172.21.15.5:6789], mon addrv is [v2:172.21.15.5:3300,v1:172.21.15.5:6789] 2024-09-10T09:26:52.917 INFO:teuthology.orchestra.run.smithi005.stdout:/sbin/ip: stdout default via 172.21.15.254 dev ens1f0 proto dhcp src 172.21.15.5 metric 100 2024-09-10T09:26:52.917 INFO:teuthology.orchestra.run.smithi005.stdout:/sbin/ip: stdout 172.21.0.0/20 dev ens1f0 proto kernel scope link src 172.21.15.5 metric 100 2024-09-10T09:26:52.923 INFO:teuthology.orchestra.run.smithi005.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-09-10T09:26:52.923 INFO:teuthology.orchestra.run.smithi005.stdout:/sbin/ip: stdout fe80::/64 dev ens1f0 proto kernel metric 256 pref medium 2024-09-10T09:26:52.923 INFO:teuthology.orchestra.run.smithi005.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev ens1f0 proto ra metric 1024 expires 1782sec hoplimit 64 pref medium 2024-09-10T09:26:52.929 INFO:teuthology.orchestra.run.smithi005.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-09-10T09:26:52.929 INFO:teuthology.orchestra.run.smithi005.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-09-10T09:26:52.929 INFO:teuthology.orchestra.run.smithi005.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-09-10T09:26:52.929 INFO:teuthology.orchestra.run.smithi005.stdout:/sbin/ip: stdout 4: ens1f0: mtu 1500 state UP qlen 1000 2024-09-10T09:26:52.929 INFO:teuthology.orchestra.run.smithi005.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:febd:1442/64 scope link 2024-09-10T09:26:52.929 INFO:teuthology.orchestra.run.smithi005.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-09-10T09:26:52.930 INFO:teuthology.orchestra.run.smithi005.stdout:Mon IP `172.21.15.5` is in CIDR network `172.21.0.0/20` 2024-09-10T09:26:52.930 INFO:teuthology.orchestra.run.smithi005.stdout:Mon IP `172.21.15.5` is in CIDR network `172.21.0.0/20` 2024-09-10T09:26:52.930 INFO:teuthology.orchestra.run.smithi005.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-09-10T09:26:52.930 INFO:teuthology.orchestra.run.smithi005.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-09-10T09:26:52.931 INFO:teuthology.orchestra.run.smithi005.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b... 2024-09-10T09:26:53.444 INFO:teuthology.orchestra.run.smithi005.stdout:/bin/podman: stdout 41d67112835771c3efca60c6b8fe297c7ddaecdc26cea26fd28aa18636e49005 2024-09-10T09:26:53.444 INFO:teuthology.orchestra.run.smithi005.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b... 2024-09-10T09:26:53.444 INFO:teuthology.orchestra.run.smithi005.stdout:/bin/podman: stderr Getting image source signatures 2024-09-10T09:26:53.444 INFO:teuthology.orchestra.run.smithi005.stdout:/bin/podman: stderr Copying blob sha256:e8b54c863393b7a8216a71737771b73b16a8e43ec7acf927c7faa175c255e551 2024-09-10T09:26:53.444 INFO:teuthology.orchestra.run.smithi005.stdout:/bin/podman: stderr Copying blob sha256:3965ca83761b6e221cd35b61f2ebac0e3b1f9d68a1ce39c1ada69153a1d73116 2024-09-10T09:26:53.444 INFO:teuthology.orchestra.run.smithi005.stdout:/bin/podman: stderr Copying config sha256:41d67112835771c3efca60c6b8fe297c7ddaecdc26cea26fd28aa18636e49005 2024-09-10T09:26:53.445 INFO:teuthology.orchestra.run.smithi005.stdout:/bin/podman: stderr Writing manifest to image destination 2024-09-10T09:26:54.496 INFO:teuthology.orchestra.run.smithi005.stdout:ceph: stdout ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable) 2024-09-10T09:26:54.496 INFO:teuthology.orchestra.run.smithi005.stdout:Ceph version: ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable) 2024-09-10T09:26:54.496 INFO:teuthology.orchestra.run.smithi005.stdout:Extracting ceph user uid/gid from container image... 2024-09-10T09:27:05.302 INFO:teuthology.orchestra.run.smithi005.stdout:stat: stdout 167 167 2024-09-10T09:27:05.302 INFO:teuthology.orchestra.run.smithi005.stdout:Creating initial keys... 2024-09-10T09:27:06.228 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph-authtool: stdout AQDpEOBmAEWbKRAAgRpX8jBjudBeIADiX9GSzA== 2024-09-10T09:27:07.319 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph-authtool: stdout AQDqEOBmFnPiJhAAInTZCg1Oa6nyT7qp3bC5dw== 2024-09-10T09:27:08.341 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph-authtool: stdout AQDrEOBmJMGgKRAA6GNgGaw8i8mlSa4owjgl5w== 2024-09-10T09:27:08.342 INFO:teuthology.orchestra.run.smithi005.stdout:Creating initial monmap... 2024-09-10T09:27:09.371 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-09-10T09:27:09.371 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-09-10T09:27:09.371 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to bade4b50-6f56-11ef-bcea-c7b262605968 2024-09-10T09:27:09.371 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-09-10T09:27:09.371 INFO:teuthology.orchestra.run.smithi005.stdout:monmaptool for a [v2:172.21.15.5:3300,v1:172.21.15.5:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-09-10T09:27:09.371 INFO:teuthology.orchestra.run.smithi005.stdout:setting min_mon_release = quincy 2024-09-10T09:27:09.371 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/monmaptool: set fsid to bade4b50-6f56-11ef-bcea-c7b262605968 2024-09-10T09:27:09.371 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-09-10T09:27:09.371 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-10T09:27:09.371 INFO:teuthology.orchestra.run.smithi005.stdout:Creating mon... 2024-09-10T09:27:11.227 INFO:teuthology.orchestra.run.smithi005.stdout:create mon.a on 2024-09-10T09:27:11.554 INFO:teuthology.orchestra.run.smithi005.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-09-10T09:27:11.773 INFO:teuthology.orchestra.run.smithi005.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-09-10T09:27:12.031 INFO:teuthology.orchestra.run.smithi005.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-bade4b50-6f56-11ef-bcea-c7b262605968.target → /etc/systemd/system/ceph-bade4b50-6f56-11ef-bcea-c7b262605968.target. 2024-09-10T09:27:12.031 INFO:teuthology.orchestra.run.smithi005.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-bade4b50-6f56-11ef-bcea-c7b262605968.target → /etc/systemd/system/ceph-bade4b50-6f56-11ef-bcea-c7b262605968.target. 2024-09-10T09:27:12.350 INFO:teuthology.orchestra.run.smithi005.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-bade4b50-6f56-11ef-bcea-c7b262605968@mon.a 2024-09-10T09:27:12.350 INFO:teuthology.orchestra.run.smithi005.stdout:systemctl: stderr Failed to reset failed state of unit ceph-bade4b50-6f56-11ef-bcea-c7b262605968@mon.a.service: Unit ceph-bade4b50-6f56-11ef-bcea-c7b262605968@mon.a.service not loaded. 2024-09-10T09:27:12.625 INFO:teuthology.orchestra.run.smithi005.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-bade4b50-6f56-11ef-bcea-c7b262605968.target.wants/ceph-bade4b50-6f56-11ef-bcea-c7b262605968@mon.a.service → /etc/systemd/system/ceph-bade4b50-6f56-11ef-bcea-c7b262605968@.service. 2024-09-10T09:27:13.234 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:12 smithi005 podman[29971]: 2024-09-10 09:27:12.980477073 +0000 UTC m=+0.127679523 container create f35ee6cfa51934597f699f8447673ac3c97aa4e60237cbeb7e756c6c766a3c8c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b, name=ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mon-a, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.37.2) 2024-09-10T09:27:13.234 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:13 smithi005 podman[29971]: 2024-09-10 09:27:13.232976276 +0000 UTC m=+0.380178756 container init f35ee6cfa51934597f699f8447673ac3c97aa4e60237cbeb7e756c6c766a3c8c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b, name=ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mon-a, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2) 2024-09-10T09:27:13.360 INFO:teuthology.orchestra.run.smithi005.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-10T09:27:13.360 INFO:teuthology.orchestra.run.smithi005.stdout:systemctl: stdout disabled 2024-09-10T09:27:13.373 INFO:teuthology.orchestra.run.smithi005.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-10T09:27:13.373 INFO:teuthology.orchestra.run.smithi005.stdout:systemctl: stdout inactive 2024-09-10T09:27:13.374 INFO:teuthology.orchestra.run.smithi005.stdout:firewalld.service is not enabled 2024-09-10T09:27:13.374 INFO:teuthology.orchestra.run.smithi005.stdout:Not possible to enable service . firewalld.service is not available 2024-09-10T09:27:13.374 INFO:teuthology.orchestra.run.smithi005.stdout:Waiting for mon to start... 2024-09-10T09:27:13.374 INFO:teuthology.orchestra.run.smithi005.stdout:Waiting for mon... 2024-09-10T09:27:13.516 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:13 smithi005 podman[29971]: 2024-09-10 09:27:13.239624945 +0000 UTC m=+0.386827396 container start f35ee6cfa51934597f699f8447673ac3c97aa4e60237cbeb7e756c6c766a3c8c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b, name=ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mon-a, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, maintainer=Guillaume Abrioux ) 2024-09-10T09:27:13.516 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:13 smithi005 ceph-mon[29990]: mkfs bade4b50-6f56-11ef-bcea-c7b262605968 2024-09-10T09:27:13.516 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:13 smithi005 ceph-mon[29990]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-10T09:27:13.516 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:13 smithi005 bash[29971]: f35ee6cfa51934597f699f8447673ac3c97aa4e60237cbeb7e756c6c766a3c8c 2024-09-10T09:27:13.516 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:13 smithi005 systemd[1]: Started Ceph mon.a for bade4b50-6f56-11ef-bcea-c7b262605968. 2024-09-10T09:27:14.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:14 smithi005 ceph-mon[29990]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-10T09:27:14.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:14 smithi005 ceph-mon[29990]: monmap epoch 1 2024-09-10T09:27:14.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:14 smithi005 ceph-mon[29990]: fsid bade4b50-6f56-11ef-bcea-c7b262605968 2024-09-10T09:27:14.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:14 smithi005 ceph-mon[29990]: last_changed 2024-09-10T09:27:08.776915+0000 2024-09-10T09:27:14.557 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:14 smithi005 ceph-mon[29990]: created 2024-09-10T09:27:08.776915+0000 2024-09-10T09:27:14.557 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:14 smithi005 ceph-mon[29990]: min_mon_release 19 (squid) 2024-09-10T09:27:14.557 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:14 smithi005 ceph-mon[29990]: election_strategy: 1 2024-09-10T09:27:14.557 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:14 smithi005 ceph-mon[29990]: 0: [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] mon.a 2024-09-10T09:27:14.557 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:14 smithi005 ceph-mon[29990]: fsmap 2024-09-10T09:27:14.557 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:14 smithi005 ceph-mon[29990]: osdmap e1: 0 total, 0 up, 0 in 2024-09-10T09:27:14.557 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:14 smithi005 ceph-mon[29990]: mgrmap e1: no daemons active 2024-09-10T09:27:14.557 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:14 smithi005 ceph-mon[29990]: from='client.? 172.21.15.5:0/1279352785' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-09-10T09:27:14.627 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout cluster: 2024-09-10T09:27:14.627 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout id: bade4b50-6f56-11ef-bcea-c7b262605968 2024-09-10T09:27:14.627 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-09-10T09:27:14.627 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout 2024-09-10T09:27:14.627 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout services: 2024-09-10T09:27:14.627 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.659964s) 2024-09-10T09:27:14.627 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-09-10T09:27:14.627 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-09-10T09:27:14.627 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout 2024-09-10T09:27:14.627 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout data: 2024-09-10T09:27:14.627 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-09-10T09:27:14.627 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-09-10T09:27:14.628 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-09-10T09:27:14.628 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout pgs: 2024-09-10T09:27:14.628 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout 2024-09-10T09:27:14.628 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:13.878+0000 7f7d05400640 1 Processor -- start 2024-09-10T09:27:14.628 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:13.878+0000 7f7d05400640 1 -- start start 2024-09-10T09:27:14.628 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:13.879+0000 7f7d05400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7d0007de00 0x7f7d0007c300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:27:14.628 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:13.879+0000 7f7d05400640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f7d0007c840 con 0x7f7d0007de00 2024-09-10T09:27:14.628 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:13.880+0000 7f7cfe800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7d0007de00 0x7f7d0007c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:27:14.628 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:13.880+0000 7f7cfe800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7d0007de00 0x7f7d0007c300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49980/0 (socket says 172.21.15.5:49980) 2024-09-10T09:27:14.628 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:13.880+0000 7f7cfe800640 1 -- 172.21.15.5:0/187988863 learned_addr learned my addr 172.21.15.5:0/187988863 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:27:14.628 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:13.880+0000 7f7cfe800640 1 -- 172.21.15.5:0/187988863 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d0007c9c0 con 0x7f7d0007de00 2024-09-10T09:27:14.628 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:13.883+0000 7f7cfe800640 1 --2- 172.21.15.5:0/187988863 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7d0007de00 0x7f7d0007c300 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f7cf0009820 tx=0x7f7cf002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a8a7cd729ebd0ad server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:27:14.629 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:13.883+0000 7f7cfde00640 1 -- 172.21.15.5:0/187988863 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7cf0037070 con 0x7f7d0007de00 2024-09-10T09:27:14.629 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:13.883+0000 7f7cfde00640 1 -- 172.21.15.5:0/187988863 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7cf0032440 con 0x7f7d0007de00 2024-09-10T09:27:14.629 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:13.883+0000 7f7cfde00640 1 -- 172.21.15.5:0/187988863 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7cf0032740 con 0x7f7d0007de00 2024-09-10T09:27:14.629 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:13.884+0000 7f7d05400640 1 -- 172.21.15.5:0/187988863 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7d0007de00 msgr2=0x7f7d0007c300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:27:14.629 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:13.884+0000 7f7d05400640 1 --2- 172.21.15.5:0/187988863 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7d0007de00 0x7f7d0007c300 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f7cf0009820 tx=0x7f7cf002a270 comp rx=0 tx=0).stop 2024-09-10T09:27:14.629 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:13.884+0000 7f7d05400640 1 -- 172.21.15.5:0/187988863 shutdown_connections 2024-09-10T09:27:14.629 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:13.884+0000 7f7d05400640 1 --2- 172.21.15.5:0/187988863 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7d0007de00 0x7f7d0007c300 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:27:14.629 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:13.884+0000 7f7d05400640 1 -- 172.21.15.5:0/187988863 >> 172.21.15.5:0/187988863 conn(0x7f7d00102e50 msgr2=0x7f7d00105270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:27:14.629 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:13.884+0000 7f7d05400640 1 -- 172.21.15.5:0/187988863 shutdown_connections 2024-09-10T09:27:14.629 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:13.884+0000 7f7d05400640 1 -- 172.21.15.5:0/187988863 wait complete. 2024-09-10T09:27:14.629 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:13.885+0000 7f7d05400640 1 Processor -- start 2024-09-10T09:27:14.629 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:13.885+0000 7f7d05400640 1 -- start start 2024-09-10T09:27:14.629 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:13.885+0000 7f7d05400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7d0007de00 0x7f7d0019cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:27:14.630 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:13.885+0000 7f7d05400640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f7d0007cf00 con 0x7f7d0007de00 2024-09-10T09:27:14.630 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:13.886+0000 7f7cfe800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7d0007de00 0x7f7d0019cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:27:14.630 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:13.886+0000 7f7cfe800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7d0007de00 0x7f7d0019cfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49996/0 (socket says 172.21.15.5:49996) 2024-09-10T09:27:14.630 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:13.886+0000 7f7cfe800640 1 -- 172.21.15.5:0/1279352785 learned_addr learned my addr 172.21.15.5:0/1279352785 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:27:14.630 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:13.886+0000 7f7cfe800640 1 -- 172.21.15.5:0/1279352785 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d0019d4f0 con 0x7f7d0007de00 2024-09-10T09:27:14.630 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:13.886+0000 7f7cfe800640 1 --2- 172.21.15.5:0/1279352785 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7d0007de00 0x7f7d0019cfb0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f7cf00090b0 tx=0x7f7cf00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:27:14.630 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:13.887+0000 7f7cf7e00640 1 -- 172.21.15.5:0/1279352785 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7cf0041070 con 0x7f7d0007de00 2024-09-10T09:27:14.630 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:13.887+0000 7f7cf7e00640 1 -- 172.21.15.5:0/1279352785 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7cf003c7a0 con 0x7f7d0007de00 2024-09-10T09:27:14.630 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:13.887+0000 7f7d05400640 1 -- 172.21.15.5:0/1279352785 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d0019d780 con 0x7f7d0007de00 2024-09-10T09:27:14.631 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:13.887+0000 7f7cf7e00640 1 -- 172.21.15.5:0/1279352785 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7cf0037040 con 0x7f7d0007de00 2024-09-10T09:27:14.631 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:13.887+0000 7f7d05400640 1 -- 172.21.15.5:0/1279352785 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d001a0470 con 0x7f7d0007de00 2024-09-10T09:27:14.631 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:13.887+0000 7f7cf7e00640 1 -- 172.21.15.5:0/1279352785 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f7cf0044070 con 0x7f7d0007de00 2024-09-10T09:27:14.631 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:13.887+0000 7f7cf7e00640 1 -- 172.21.15.5:0/1279352785 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f7cf003d3f0 con 0x7f7d0007de00 2024-09-10T09:27:14.631 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:13.888+0000 7f7d05400640 1 -- 172.21.15.5:0/1279352785 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d0007c380 con 0x7f7d0007de00 2024-09-10T09:27:14.631 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:13.891+0000 7f7cf7e00640 1 -- 172.21.15.5:0/1279352785 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f7cf0046450 con 0x7f7d0007de00 2024-09-10T09:27:14.631 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:13.956+0000 7f7d05400640 1 -- 172.21.15.5:0/1279352785 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7f7d0002d910 con 0x7f7d0007de00 2024-09-10T09:27:14.631 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:13.956+0000 7f7cf7e00640 1 -- 172.21.15.5:0/1279352785 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+317 (secure 0 0 0) 0x7f7cf0046630 con 0x7f7d0007de00 2024-09-10T09:27:14.631 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:13.957+0000 7f7d05400640 1 -- 172.21.15.5:0/1279352785 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7d0007de00 msgr2=0x7f7d0019cfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:27:14.631 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:13.957+0000 7f7d05400640 1 --2- 172.21.15.5:0/1279352785 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7d0007de00 0x7f7d0019cfb0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f7cf00090b0 tx=0x7f7cf00046c0 comp rx=0 tx=0).stop 2024-09-10T09:27:14.631 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:13.957+0000 7f7d05400640 1 -- 172.21.15.5:0/1279352785 shutdown_connections 2024-09-10T09:27:14.631 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:13.957+0000 7f7d05400640 1 --2- 172.21.15.5:0/1279352785 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7d0007de00 0x7f7d0019cfb0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:27:14.631 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:13.957+0000 7f7d05400640 1 -- 172.21.15.5:0/1279352785 >> 172.21.15.5:0/1279352785 conn(0x7f7d00102e50 msgr2=0x7f7d00103ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:27:14.632 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:13.957+0000 7f7d05400640 1 -- 172.21.15.5:0/1279352785 shutdown_connections 2024-09-10T09:27:14.632 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:13.958+0000 7f7d05400640 1 -- 172.21.15.5:0/1279352785 wait complete. 2024-09-10T09:27:14.632 INFO:teuthology.orchestra.run.smithi005.stdout:mon is available 2024-09-10T09:27:14.632 INFO:teuthology.orchestra.run.smithi005.stdout:Assimilating anything we can from ceph.conf... 2024-09-10T09:27:15.502 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:15 smithi005 ceph-mon[29990]: from='client.? 172.21.15.5:0/940888965' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-09-10T09:27:15.502 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:15 smithi005 ceph-mon[29990]: from='client.? 172.21.15.5:0/940888965' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-09-10T09:27:15.776 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout 2024-09-10T09:27:15.776 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout [global] 2024-09-10T09:27:15.776 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout fsid = bade4b50-6f56-11ef-bcea-c7b262605968 2024-09-10T09:27:15.776 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.5:3300,v1:172.21.15.5:6789] 2024-09-10T09:27:15.776 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-09-10T09:27:15.776 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-09-10T09:27:15.776 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-09-10T09:27:15.776 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-09-10T09:27:15.776 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout 2024-09-10T09:27:15.776 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout [mgr] 2024-09-10T09:27:15.776 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-09-10T09:27:15.776 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout 2024-09-10T09:27:15.777 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout [osd] 2024-09-10T09:27:15.777 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-09-10T09:27:15.777 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-09-10T09:27:15.777 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-09-10T09:27:15.777 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-09-10T09:27:15.777 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-09-10T09:27:15.777 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:15.091+0000 7f84f9200640 1 Processor -- start 2024-09-10T09:27:15.777 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:15.091+0000 7f84f9200640 1 -- start start 2024-09-10T09:27:15.777 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:15.091+0000 7f84f9200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f84f41076a0 0x7f84f4107aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:27:15.777 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:15.091+0000 7f84f9200640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f84f4108070 con 0x7f84f41076a0 2024-09-10T09:27:15.777 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:15.092+0000 7f84f1e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f84f41076a0 0x7f84f4107aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:27:15.777 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:15.092+0000 7f84f1e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f84f41076a0 0x7f84f4107aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50012/0 (socket says 172.21.15.5:50012) 2024-09-10T09:27:15.777 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:15.092+0000 7f84f1e00640 1 -- 172.21.15.5:0/2435196887 learned_addr learned my addr 172.21.15.5:0/2435196887 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:27:15.778 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:15.092+0000 7f84f1e00640 1 -- 172.21.15.5:0/2435196887 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84f4108800 con 0x7f84f41076a0 2024-09-10T09:27:15.778 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:15.093+0000 7f84f1e00640 1 --2- 172.21.15.5:0/2435196887 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f84f41076a0 0x7f84f4107aa0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f84e4009820 tx=0x7f84e402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19cc25886c380f12 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:27:15.778 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:15.093+0000 7f84f0a00640 1 -- 172.21.15.5:0/2435196887 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f84e4037070 con 0x7f84f41076a0 2024-09-10T09:27:15.778 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:15.093+0000 7f84f0a00640 1 -- 172.21.15.5:0/2435196887 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f84e4032440 con 0x7f84f41076a0 2024-09-10T09:27:15.778 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:15.093+0000 7f84f0a00640 1 -- 172.21.15.5:0/2435196887 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f84e4032740 con 0x7f84f41076a0 2024-09-10T09:27:15.778 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:15.094+0000 7f84f9200640 1 -- 172.21.15.5:0/2435196887 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f84f41076a0 msgr2=0x7f84f4107aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:27:15.778 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:15.094+0000 7f84f9200640 1 --2- 172.21.15.5:0/2435196887 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f84f41076a0 0x7f84f4107aa0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f84e4009820 tx=0x7f84e402a270 comp rx=0 tx=0).stop 2024-09-10T09:27:15.778 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:15.094+0000 7f84f9200640 1 -- 172.21.15.5:0/2435196887 shutdown_connections 2024-09-10T09:27:15.778 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:15.094+0000 7f84f9200640 1 --2- 172.21.15.5:0/2435196887 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f84f41076a0 0x7f84f4107aa0 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:27:15.778 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:15.094+0000 7f84f9200640 1 -- 172.21.15.5:0/2435196887 >> 172.21.15.5:0/2435196887 conn(0x7f84f4102e50 msgr2=0x7f84f4105270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:27:15.778 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:15.094+0000 7f84f9200640 1 -- 172.21.15.5:0/2435196887 shutdown_connections 2024-09-10T09:27:15.778 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:15.094+0000 7f84f9200640 1 -- 172.21.15.5:0/2435196887 wait complete. 2024-09-10T09:27:15.778 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:15.095+0000 7f84f9200640 1 Processor -- start 2024-09-10T09:27:15.779 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:15.095+0000 7f84f9200640 1 -- start start 2024-09-10T09:27:15.779 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:15.095+0000 7f84f9200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f84f41076a0 0x7f84f419d010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:27:15.779 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:15.095+0000 7f84f9200640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f84f4108da0 con 0x7f84f41076a0 2024-09-10T09:27:15.779 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:15.096+0000 7f84f1e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f84f41076a0 0x7f84f419d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:27:15.779 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:15.096+0000 7f84f1e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f84f41076a0 0x7f84f419d010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50024/0 (socket says 172.21.15.5:50024) 2024-09-10T09:27:15.779 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:15.096+0000 7f84f1e00640 1 -- 172.21.15.5:0/940888965 learned_addr learned my addr 172.21.15.5:0/940888965 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:27:15.779 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:15.096+0000 7f84f1e00640 1 -- 172.21.15.5:0/940888965 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84f419d550 con 0x7f84f41076a0 2024-09-10T09:27:15.779 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:15.096+0000 7f84f1e00640 1 --2- 172.21.15.5:0/940888965 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f84f41076a0 0x7f84f419d010 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f84e40090b0 tx=0x7f84e40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:27:15.779 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:15.097+0000 7f84eaa00640 1 -- 172.21.15.5:0/940888965 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f84e4041070 con 0x7f84f41076a0 2024-09-10T09:27:15.779 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:15.097+0000 7f84eaa00640 1 -- 172.21.15.5:0/940888965 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f84e403c7a0 con 0x7f84f41076a0 2024-09-10T09:27:15.779 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:15.097+0000 7f84eaa00640 1 -- 172.21.15.5:0/940888965 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f84e4037040 con 0x7f84f41076a0 2024-09-10T09:27:15.779 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:15.097+0000 7f84f9200640 1 -- 172.21.15.5:0/940888965 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84f419d7e0 con 0x7f84f41076a0 2024-09-10T09:27:15.779 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:15.097+0000 7f84f9200640 1 -- 172.21.15.5:0/940888965 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84f41a04d0 con 0x7f84f41076a0 2024-09-10T09:27:15.779 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:15.097+0000 7f84eaa00640 1 -- 172.21.15.5:0/940888965 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f84e4044070 con 0x7f84f41076a0 2024-09-10T09:27:15.780 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:15.097+0000 7f84eaa00640 1 -- 172.21.15.5:0/940888965 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f84e403d3f0 con 0x7f84f41076a0 2024-09-10T09:27:15.780 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:15.098+0000 7f84f9200640 1 -- 172.21.15.5:0/940888965 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f84f41093f0 con 0x7f84f41076a0 2024-09-10T09:27:15.780 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:15.101+0000 7f84eaa00640 1 -- 172.21.15.5:0/940888965 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f84e4046450 con 0x7f84f41076a0 2024-09-10T09:27:15.780 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:15.163+0000 7f84f9200640 1 -- 172.21.15.5:0/940888965 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f84f41a1ba0 con 0x7f84f41076a0 2024-09-10T09:27:15.780 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:15.168+0000 7f84eaa00640 1 -- 172.21.15.5:0/940888965 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) ==== 70+0+427 (secure 0 0 0) 0x7f84e4046630 con 0x7f84f41076a0 2024-09-10T09:27:15.780 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:15.169+0000 7f84eaa00640 1 -- 172.21.15.5:0/940888965 <== mon.0 v2:172.21.15.5:3300/0 8 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f84e40584b0 con 0x7f84f41076a0 2024-09-10T09:27:15.780 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:15.170+0000 7f84f9200640 1 -- 172.21.15.5:0/940888965 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f84f41076a0 msgr2=0x7f84f419d010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:27:15.780 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:15.170+0000 7f84f9200640 1 --2- 172.21.15.5:0/940888965 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f84f41076a0 0x7f84f419d010 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f84e40090b0 tx=0x7f84e40046c0 comp rx=0 tx=0).stop 2024-09-10T09:27:15.780 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:15.170+0000 7f84f9200640 1 -- 172.21.15.5:0/940888965 shutdown_connections 2024-09-10T09:27:15.780 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:15.170+0000 7f84f9200640 1 --2- 172.21.15.5:0/940888965 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f84f41076a0 0x7f84f419d010 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:27:15.780 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:15.170+0000 7f84f9200640 1 -- 172.21.15.5:0/940888965 >> 172.21.15.5:0/940888965 conn(0x7f84f4102e50 msgr2=0x7f84f407c0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:27:15.780 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:15.170+0000 7f84f9200640 1 -- 172.21.15.5:0/940888965 shutdown_connections 2024-09-10T09:27:15.780 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:15.170+0000 7f84f9200640 1 -- 172.21.15.5:0/940888965 wait complete. 2024-09-10T09:27:15.780 INFO:teuthology.orchestra.run.smithi005.stdout:Generating new minimal ceph.conf... 2024-09-10T09:27:16.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:16 smithi005 ceph-mon[29990]: from='client.? 172.21.15.5:0/3675893169' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:27:16.938 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:16.223+0000 7fdbdb800640 1 Processor -- start 2024-09-10T09:27:16.938 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:16.223+0000 7fdbdb800640 1 -- start start 2024-09-10T09:27:16.938 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:16.223+0000 7fdbdb800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdbd4109870 0x7fdbd4109c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:27:16.938 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:16.223+0000 7fdbdb800640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fdbd410a240 con 0x7fdbd4109870 2024-09-10T09:27:16.939 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:16.224+0000 7fdbd8e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdbd4109870 0x7fdbd4109c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:27:16.939 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:16.224+0000 7fdbd8e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdbd4109870 0x7fdbd4109c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50040/0 (socket says 172.21.15.5:50040) 2024-09-10T09:27:16.939 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:16.224+0000 7fdbd8e00640 1 -- 172.21.15.5:0/1554315739 learned_addr learned my addr 172.21.15.5:0/1554315739 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:27:16.939 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:16.224+0000 7fdbd8e00640 1 -- 172.21.15.5:0/1554315739 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbd410aa10 con 0x7fdbd4109870 2024-09-10T09:27:16.939 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:16.225+0000 7fdbd8e00640 1 --2- 172.21.15.5:0/1554315739 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdbd4109870 0x7fdbd4109c70 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fdbc8009820 tx=0x7fdbc802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dba0cacd444539ad server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:27:16.939 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:16.226+0000 7fdbd3400640 1 -- 172.21.15.5:0/1554315739 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdbc8037070 con 0x7fdbd4109870 2024-09-10T09:27:16.939 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:16.226+0000 7fdbd3400640 1 -- 172.21.15.5:0/1554315739 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fdbc8032900 con 0x7fdbd4109870 2024-09-10T09:27:16.939 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:16.226+0000 7fdbd3400640 1 -- 172.21.15.5:0/1554315739 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdbc8030850 con 0x7fdbd4109870 2024-09-10T09:27:16.939 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:16.226+0000 7fdbdb800640 1 -- 172.21.15.5:0/1554315739 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdbd4109870 msgr2=0x7fdbd4109c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:27:16.939 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:16.226+0000 7fdbdb800640 1 --2- 172.21.15.5:0/1554315739 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdbd4109870 0x7fdbd4109c70 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fdbc8009820 tx=0x7fdbc802a270 comp rx=0 tx=0).stop 2024-09-10T09:27:16.939 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:16.226+0000 7fdbdb800640 1 -- 172.21.15.5:0/1554315739 shutdown_connections 2024-09-10T09:27:16.939 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:16.226+0000 7fdbdb800640 1 --2- 172.21.15.5:0/1554315739 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdbd4109870 0x7fdbd4109c70 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:27:16.940 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:16.226+0000 7fdbdb800640 1 -- 172.21.15.5:0/1554315739 >> 172.21.15.5:0/1554315739 conn(0x7fdbd407cf40 msgr2=0x7fdbd407d350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:27:16.940 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:16.227+0000 7fdbdb800640 1 -- 172.21.15.5:0/1554315739 shutdown_connections 2024-09-10T09:27:16.940 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:16.227+0000 7fdbdb800640 1 -- 172.21.15.5:0/1554315739 wait complete. 2024-09-10T09:27:16.940 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:16.227+0000 7fdbdb800640 1 Processor -- start 2024-09-10T09:27:16.940 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:16.228+0000 7fdbdb800640 1 -- start start 2024-09-10T09:27:16.940 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:16.228+0000 7fdbdb800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdbd4109870 0x7fdbd419d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:27:16.940 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:16.228+0000 7fdbdb800640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fdbd410ddc0 con 0x7fdbd4109870 2024-09-10T09:27:16.940 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:16.228+0000 7fdbd8e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdbd4109870 0x7fdbd419d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:27:16.940 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:16.228+0000 7fdbd8e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdbd4109870 0x7fdbd419d7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50056/0 (socket says 172.21.15.5:50056) 2024-09-10T09:27:16.940 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:16.228+0000 7fdbd8e00640 1 -- 172.21.15.5:0/3675893169 learned_addr learned my addr 172.21.15.5:0/3675893169 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:27:16.940 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:16.228+0000 7fdbd8e00640 1 -- 172.21.15.5:0/3675893169 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbd419dd20 con 0x7fdbd4109870 2024-09-10T09:27:16.940 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:16.229+0000 7fdbd8e00640 1 --2- 172.21.15.5:0/3675893169 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdbd4109870 0x7fdbd419d7e0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fdbc8032ce0 tx=0x7fdbc8032d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:27:16.941 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:16.229+0000 7fdbd1600640 1 -- 172.21.15.5:0/3675893169 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdbc8042070 con 0x7fdbd4109870 2024-09-10T09:27:16.941 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:16.229+0000 7fdbd1600640 1 -- 172.21.15.5:0/3675893169 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fdbc80308c0 con 0x7fdbd4109870 2024-09-10T09:27:16.941 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:16.229+0000 7fdbd1600640 1 -- 172.21.15.5:0/3675893169 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdbc8037040 con 0x7fdbd4109870 2024-09-10T09:27:16.941 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:16.229+0000 7fdbdb800640 1 -- 172.21.15.5:0/3675893169 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdbd419dfb0 con 0x7fdbd4109870 2024-09-10T09:27:16.941 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:16.229+0000 7fdbdb800640 1 -- 172.21.15.5:0/3675893169 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdbd41a0ca0 con 0x7fdbd4109870 2024-09-10T09:27:16.941 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:16.229+0000 7fdbd1600640 1 -- 172.21.15.5:0/3675893169 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fdbc8045070 con 0x7fdbd4109870 2024-09-10T09:27:16.941 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:16.229+0000 7fdbd1600640 1 -- 172.21.15.5:0/3675893169 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fdbc8047b70 con 0x7fdbd4109870 2024-09-10T09:27:16.941 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:16.230+0000 7fdbdb800640 1 -- 172.21.15.5:0/3675893169 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdbd4109c70 con 0x7fdbd4109870 2024-09-10T09:27:16.941 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:16.233+0000 7fdbd1600640 1 -- 172.21.15.5:0/3675893169 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7fdbc803dbf0 con 0x7fdbd4109870 2024-09-10T09:27:16.941 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:16.297+0000 7fdbdb800640 1 -- 172.21.15.5:0/3675893169 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7fdbd41a1210 con 0x7fdbd4109870 2024-09-10T09:27:16.941 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:16.297+0000 7fdbd1600640 1 -- 172.21.15.5:0/3675893169 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) ==== 76+0+173 (secure 0 0 0) 0x7fdbc803d430 con 0x7fdbd4109870 2024-09-10T09:27:16.941 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:16.298+0000 7fdbdb800640 1 -- 172.21.15.5:0/3675893169 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdbd4109870 msgr2=0x7fdbd419d7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:27:16.941 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:16.298+0000 7fdbdb800640 1 --2- 172.21.15.5:0/3675893169 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdbd4109870 0x7fdbd419d7e0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fdbc8032ce0 tx=0x7fdbc8032d10 comp rx=0 tx=0).stop 2024-09-10T09:27:16.942 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:16.298+0000 7fdbdb800640 1 -- 172.21.15.5:0/3675893169 shutdown_connections 2024-09-10T09:27:16.942 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:16.299+0000 7fdbdb800640 1 --2- 172.21.15.5:0/3675893169 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdbd4109870 0x7fdbd419d7e0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:27:16.942 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:16.299+0000 7fdbdb800640 1 -- 172.21.15.5:0/3675893169 >> 172.21.15.5:0/3675893169 conn(0x7fdbd407cf40 msgr2=0x7fdbd4107440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:27:16.942 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:16.299+0000 7fdbdb800640 1 -- 172.21.15.5:0/3675893169 shutdown_connections 2024-09-10T09:27:16.942 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:16.299+0000 7fdbdb800640 1 -- 172.21.15.5:0/3675893169 wait complete. 2024-09-10T09:27:16.942 INFO:teuthology.orchestra.run.smithi005.stdout:Restarting the monitor... 2024-09-10T09:27:17.306 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:16 smithi005 systemd[1]: Stopping Ceph mon.a for bade4b50-6f56-11ef-bcea-c7b262605968... 2024-09-10T09:27:17.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:17 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mon-a[29986]: 2024-09-10T09:27:17.350+0000 7f9b86600640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T09:27:17.757 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:17 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mon-a[29986]: 2024-09-10T09:27:17.350+0000 7f9b86600640 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-09-10T09:27:17.757 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:17 smithi005 podman[30231]: 2024-09-10 09:27:17.504468209 +0000 UTC m=+0.307761983 container died f35ee6cfa51934597f699f8447673ac3c97aa4e60237cbeb7e756c6c766a3c8c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b, name=ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mon-a, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731) 2024-09-10T09:27:18.023 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:17 smithi005 podman[30231]: 2024-09-10 09:27:17.863029932 +0000 UTC m=+0.666323702 container cleanup f35ee6cfa51934597f699f8447673ac3c97aa4e60237cbeb7e756c6c766a3c8c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b, name=ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mon-a, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.build-date=20240731) 2024-09-10T09:27:18.023 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:17 smithi005 bash[30231]: ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mon-a 2024-09-10T09:27:18.274 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:18 smithi005 podman[30245]: 2024-09-10 09:27:18.022429276 +0000 UTC m=+0.507917996 container remove f35ee6cfa51934597f699f8447673ac3c97aa4e60237cbeb7e756c6c766a3c8c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b, name=ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mon-a, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS) 2024-09-10T09:27:18.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:18 smithi005 systemd[1]: ceph-bade4b50-6f56-11ef-bcea-c7b262605968@mon.a.service: Deactivated successfully. 2024-09-10T09:27:18.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:18 smithi005 systemd[1]: Stopped Ceph mon.a for bade4b50-6f56-11ef-bcea-c7b262605968. 2024-09-10T09:27:18.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:18 smithi005 systemd[1]: ceph-bade4b50-6f56-11ef-bcea-c7b262605968@mon.a.service: Consumed 1.227s CPU time. 2024-09-10T09:27:18.557 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:18 smithi005 systemd[1]: Starting Ceph mon.a for bade4b50-6f56-11ef-bcea-c7b262605968... 2024-09-10T09:27:18.883 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:18 smithi005 podman[30331]: 2024-09-10 09:27:18.606478244 +0000 UTC m=+0.122035253 container create 36f7420b52880ca9c13a94f30b28e7998c72e348aea621b0a554120f9a6a4df8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b, name=ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mon-a, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-09-10T09:27:18.883 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:18 smithi005 podman[30331]: 2024-09-10 09:27:18.511761457 +0000 UTC m=+0.027318478 image pull 41d67112835771c3efca60c6b8fe297c7ddaecdc26cea26fd28aa18636e49005 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b 2024-09-10T09:27:18.884 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:18 smithi005 podman[30331]: 2024-09-10 09:27:18.842469464 +0000 UTC m=+0.358026489 container init 36f7420b52880ca9c13a94f30b28e7998c72e348aea621b0a554120f9a6a4df8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b, name=ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mon-a, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-10T09:27:18.884 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:18 smithi005 podman[30331]: 2024-09-10 09:27:18.849056854 +0000 UTC m=+0.364613864 container start 36f7420b52880ca9c13a94f30b28e7998c72e348aea621b0a554120f9a6a4df8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b, name=ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mon-a, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-09-10T09:27:18.961 INFO:teuthology.orchestra.run.smithi005.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-09-10T09:27:19.137 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:18 smithi005 ceph-mon[30350]: set uid:gid to 167:167 (ceph:ceph) 2024-09-10T09:27:19.137 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:18 smithi005 ceph-mon[30350]: ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable), process ceph-mon, pid 2 2024-09-10T09:27:19.137 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:18 smithi005 ceph-mon[30350]: pidfile_write: ignore empty --pid-file 2024-09-10T09:27:19.137 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:18 smithi005 ceph-mon[30350]: load: jerasure load: lrc 2024-09-10T09:27:19.137 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:18 smithi005 ceph-mon[30350]: rocksdb: RocksDB version: 7.9.2 2024-09-10T09:27:19.138 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:18 smithi005 ceph-mon[30350]: rocksdb: Git sha 0 2024-09-10T09:27:19.138 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:18 smithi005 ceph-mon[30350]: rocksdb: Compile date 2024-09-09 20:43:47 2024-09-10T09:27:19.138 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:18 smithi005 ceph-mon[30350]: rocksdb: DB SUMMARY 2024-09-10T09:27:19.138 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:18 smithi005 ceph-mon[30350]: rocksdb: DB Session ID: U4DQNGW6BPH3NI2CPS2G 2024-09-10T09:27:19.138 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:18 smithi005 ceph-mon[30350]: rocksdb: CURRENT file: CURRENT 2024-09-10T09:27:19.138 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:18 smithi005 ceph-mon[30350]: rocksdb: IDENTITY file: IDENTITY 2024-09-10T09:27:19.139 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:18 smithi005 ceph-mon[30350]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-09-10T09:27:19.139 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:18 smithi005 ceph-mon[30350]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-09-10T09:27:19.139 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:18 smithi005 ceph-mon[30350]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 105386 ; 2024-09-10T09:27:19.139 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:18 smithi005 ceph-mon[30350]: rocksdb: Options.error_if_exists: 0 2024-09-10T09:27:19.139 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:18 smithi005 ceph-mon[30350]: rocksdb: Options.create_if_missing: 0 2024-09-10T09:27:19.140 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:18 smithi005 ceph-mon[30350]: rocksdb: Options.paranoid_checks: 1 2024-09-10T09:27:19.140 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:18 smithi005 ceph-mon[30350]: rocksdb: Options.flush_verify_memtable_count: 1 2024-09-10T09:27:19.140 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:18 smithi005 ceph-mon[30350]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-09-10T09:27:19.140 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:18 smithi005 ceph-mon[30350]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-09-10T09:27:19.140 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:18 smithi005 ceph-mon[30350]: rocksdb: Options.env: 0x5641066c9940 2024-09-10T09:27:19.141 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:18 smithi005 ceph-mon[30350]: rocksdb: Options.fs: PosixFileSystem 2024-09-10T09:27:19.141 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:18 smithi005 ceph-mon[30350]: rocksdb: Options.info_log: 0x56412d4376a0 2024-09-10T09:27:19.141 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:18 smithi005 ceph-mon[30350]: rocksdb: Options.max_file_opening_threads: 16 2024-09-10T09:27:19.141 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:18 smithi005 ceph-mon[30350]: rocksdb: Options.statistics: (nil) 2024-09-10T09:27:19.141 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:18 smithi005 ceph-mon[30350]: rocksdb: Options.use_fsync: 0 2024-09-10T09:27:19.142 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:18 smithi005 ceph-mon[30350]: rocksdb: Options.max_log_file_size: 0 2024-09-10T09:27:19.142 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:18 smithi005 ceph-mon[30350]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-10T09:27:19.142 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:18 smithi005 ceph-mon[30350]: rocksdb: Options.log_file_time_to_roll: 0 2024-09-10T09:27:19.142 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:18 smithi005 ceph-mon[30350]: rocksdb: Options.keep_log_file_num: 1000 2024-09-10T09:27:19.142 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:18 smithi005 ceph-mon[30350]: rocksdb: Options.recycle_log_file_num: 0 2024-09-10T09:27:19.143 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:18 smithi005 ceph-mon[30350]: rocksdb: Options.allow_fallocate: 1 2024-09-10T09:27:19.143 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:18 smithi005 ceph-mon[30350]: rocksdb: Options.allow_mmap_reads: 0 2024-09-10T09:27:19.143 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:18 smithi005 ceph-mon[30350]: rocksdb: Options.allow_mmap_writes: 0 2024-09-10T09:27:19.143 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:18 smithi005 ceph-mon[30350]: rocksdb: Options.use_direct_reads: 0 2024-09-10T09:27:19.143 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:18 smithi005 ceph-mon[30350]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-10T09:27:19.143 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:18 smithi005 ceph-mon[30350]: rocksdb: Options.create_missing_column_families: 0 2024-09-10T09:27:19.144 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:18 smithi005 ceph-mon[30350]: rocksdb: Options.db_log_dir: 2024-09-10T09:27:19.144 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:18 smithi005 ceph-mon[30350]: rocksdb: Options.wal_dir: 2024-09-10T09:27:19.144 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:18 smithi005 ceph-mon[30350]: rocksdb: Options.table_cache_numshardbits: 6 2024-09-10T09:27:19.144 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:18 smithi005 ceph-mon[30350]: rocksdb: Options.WAL_ttl_seconds: 0 2024-09-10T09:27:19.144 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:18 smithi005 ceph-mon[30350]: rocksdb: Options.WAL_size_limit_MB: 0 2024-09-10T09:27:19.144 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:18 smithi005 ceph-mon[30350]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-10T09:27:19.145 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:18 smithi005 ceph-mon[30350]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-10T09:27:19.145 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:18 smithi005 ceph-mon[30350]: rocksdb: Options.is_fd_close_on_exec: 1 2024-09-10T09:27:19.145 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:18 smithi005 ceph-mon[30350]: rocksdb: Options.advise_random_on_open: 1 2024-09-10T09:27:19.145 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:18 smithi005 ceph-mon[30350]: rocksdb: Options.db_write_buffer_size: 0 2024-09-10T09:27:19.146 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:18 smithi005 ceph-mon[30350]: rocksdb: Options.write_buffer_manager: 0x56412d446dc0 2024-09-10T09:27:19.146 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:18 smithi005 ceph-mon[30350]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-10T09:27:19.146 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:18 smithi005 ceph-mon[30350]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-10T09:27:19.146 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:18 smithi005 ceph-mon[30350]: rocksdb: Options.use_adaptive_mutex: 0 2024-09-10T09:27:19.146 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:18 smithi005 ceph-mon[30350]: rocksdb: Options.rate_limiter: (nil) 2024-09-10T09:27:19.146 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:18 smithi005 ceph-mon[30350]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-10T09:27:19.147 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:18 smithi005 ceph-mon[30350]: rocksdb: Options.wal_recovery_mode: 2 2024-09-10T09:27:19.147 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:18 smithi005 ceph-mon[30350]: rocksdb: Options.enable_thread_tracking: 0 2024-09-10T09:27:19.147 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:18 smithi005 ceph-mon[30350]: rocksdb: Options.enable_pipelined_write: 0 2024-09-10T09:27:19.147 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:18 smithi005 ceph-mon[30350]: rocksdb: Options.unordered_write: 0 2024-09-10T09:27:19.147 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:18 smithi005 ceph-mon[30350]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-10T09:27:19.147 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:18 smithi005 ceph-mon[30350]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-10T09:27:19.148 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:18 smithi005 ceph-mon[30350]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-10T09:27:19.148 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:18 smithi005 ceph-mon[30350]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-10T09:27:19.148 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:18 smithi005 ceph-mon[30350]: rocksdb: Options.row_cache: None 2024-09-10T09:27:19.148 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:18 smithi005 ceph-mon[30350]: rocksdb: Options.wal_filter: None 2024-09-10T09:27:19.149 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:18 smithi005 ceph-mon[30350]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-10T09:27:19.149 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:18 smithi005 ceph-mon[30350]: rocksdb: Options.allow_ingest_behind: 0 2024-09-10T09:27:19.149 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:18 smithi005 ceph-mon[30350]: rocksdb: Options.two_write_queues: 0 2024-09-10T09:27:19.149 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:18 smithi005 ceph-mon[30350]: rocksdb: Options.manual_wal_flush: 0 2024-09-10T09:27:19.149 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:18 smithi005 ceph-mon[30350]: rocksdb: Options.wal_compression: 0 2024-09-10T09:27:19.149 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:18 smithi005 ceph-mon[30350]: rocksdb: Options.atomic_flush: 0 2024-09-10T09:27:19.149 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:18 smithi005 ceph-mon[30350]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-10T09:27:19.149 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:18 smithi005 ceph-mon[30350]: rocksdb: Options.persist_stats_to_disk: 0 2024-09-10T09:27:19.150 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:18 smithi005 ceph-mon[30350]: rocksdb: Options.write_dbid_to_manifest: 0 2024-09-10T09:27:19.150 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:18 smithi005 ceph-mon[30350]: rocksdb: Options.log_readahead_size: 0 2024-09-10T09:27:19.150 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:18 smithi005 ceph-mon[30350]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-10T09:27:19.150 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:18 smithi005 ceph-mon[30350]: rocksdb: Options.best_efforts_recovery: 0 2024-09-10T09:27:19.150 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:18 smithi005 ceph-mon[30350]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-10T09:27:19.150 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:18 smithi005 ceph-mon[30350]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-10T09:27:19.151 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:18 smithi005 ceph-mon[30350]: rocksdb: Options.allow_data_in_errors: 0 2024-09-10T09:27:19.151 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:18 smithi005 ceph-mon[30350]: rocksdb: Options.db_host_id: __hostname__ 2024-09-10T09:27:19.151 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:18 smithi005 ceph-mon[30350]: rocksdb: Options.enforce_single_del_contracts: true 2024-09-10T09:27:19.153 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:18 smithi005 ceph-mon[30350]: rocksdb: Options.max_background_jobs: 2 2024-09-10T09:27:19.153 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:18 smithi005 ceph-mon[30350]: rocksdb: Options.max_background_compactions: -1 2024-09-10T09:27:19.153 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:18 smithi005 ceph-mon[30350]: rocksdb: Options.max_subcompactions: 1 2024-09-10T09:27:19.153 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:18 smithi005 ceph-mon[30350]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-10T09:27:19.153 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:18 smithi005 ceph-mon[30350]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-10T09:27:19.153 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:18 smithi005 ceph-mon[30350]: rocksdb: Options.delayed_write_rate : 16777216 2024-09-10T09:27:19.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:18 smithi005 ceph-mon[30350]: rocksdb: Options.max_total_wal_size: 0 2024-09-10T09:27:19.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:18 smithi005 ceph-mon[30350]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-10T09:27:19.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:18 smithi005 ceph-mon[30350]: rocksdb: Options.stats_dump_period_sec: 600 2024-09-10T09:27:19.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:18 smithi005 ceph-mon[30350]: rocksdb: Options.stats_persist_period_sec: 600 2024-09-10T09:27:19.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:18 smithi005 ceph-mon[30350]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-10T09:27:19.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:18 smithi005 ceph-mon[30350]: rocksdb: Options.max_open_files: -1 2024-09-10T09:27:19.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:18 smithi005 ceph-mon[30350]: rocksdb: Options.bytes_per_sync: 0 2024-09-10T09:27:19.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:18 smithi005 ceph-mon[30350]: rocksdb: Options.wal_bytes_per_sync: 0 2024-09-10T09:27:19.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:18 smithi005 ceph-mon[30350]: rocksdb: Options.strict_bytes_per_sync: 0 2024-09-10T09:27:19.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:18 smithi005 ceph-mon[30350]: rocksdb: Options.compaction_readahead_size: 0 2024-09-10T09:27:19.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:18 smithi005 ceph-mon[30350]: rocksdb: Options.max_background_flushes: -1 2024-09-10T09:27:19.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:18 smithi005 ceph-mon[30350]: rocksdb: Compression algorithms supported: 2024-09-10T09:27:19.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:18 smithi005 ceph-mon[30350]: rocksdb: kZSTD supported: 0 2024-09-10T09:27:19.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:18 smithi005 ceph-mon[30350]: rocksdb: kXpressCompression supported: 0 2024-09-10T09:27:19.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:18 smithi005 ceph-mon[30350]: rocksdb: kBZip2Compression supported: 0 2024-09-10T09:27:19.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:18 smithi005 ceph-mon[30350]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-10T09:27:19.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:18 smithi005 ceph-mon[30350]: rocksdb: kLZ4Compression supported: 1 2024-09-10T09:27:19.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:18 smithi005 ceph-mon[30350]: rocksdb: kZlibCompression supported: 1 2024-09-10T09:27:19.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:18 smithi005 ceph-mon[30350]: rocksdb: kLZ4HCCompression supported: 1 2024-09-10T09:27:19.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:18 smithi005 ceph-mon[30350]: rocksdb: kSnappyCompression supported: 1 2024-09-10T09:27:19.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:18 smithi005 ceph-mon[30350]: rocksdb: Fast CRC32 supported: Supported on x86 2024-09-10T09:27:19.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:18 smithi005 ceph-mon[30350]: rocksdb: DMutex implementation: pthread_mutex_t 2024-09-10T09:27:19.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:18 smithi005 ceph-mon[30350]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-09-10T09:27:19.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:18 smithi005 ceph-mon[30350]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-09-10T09:27:19.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:18 smithi005 ceph-mon[30350]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-10T09:27:19.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:18 smithi005 ceph-mon[30350]: rocksdb: Options.merge_operator: 2024-09-10T09:27:19.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:18 smithi005 ceph-mon[30350]: rocksdb: Options.compaction_filter: None 2024-09-10T09:27:19.156 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:18 smithi005 ceph-mon[30350]: rocksdb: Options.compaction_filter_factory: None 2024-09-10T09:27:19.156 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:18 smithi005 ceph-mon[30350]: rocksdb: Options.sst_partitioner_factory: None 2024-09-10T09:27:19.156 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:18 smithi005 ceph-mon[30350]: rocksdb: Options.memtable_factory: SkipListFactory 2024-09-10T09:27:19.156 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:18 smithi005 ceph-mon[30350]: rocksdb: Options.table_factory: BlockBasedTable 2024-09-10T09:27:19.156 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:18 smithi005 ceph-mon[30350]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x56412d437300) 2024-09-10T09:27:19.156 INFO:journalctl@ceph.mon.a.smithi005.stdout: cache_index_and_filter_blocks: 1 2024-09-10T09:27:19.156 INFO:journalctl@ceph.mon.a.smithi005.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-10T09:27:19.156 INFO:journalctl@ceph.mon.a.smithi005.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-10T09:27:19.156 INFO:journalctl@ceph.mon.a.smithi005.stdout: pin_top_level_index_and_filter: 1 2024-09-10T09:27:19.156 INFO:journalctl@ceph.mon.a.smithi005.stdout: index_type: 0 2024-09-10T09:27:19.156 INFO:journalctl@ceph.mon.a.smithi005.stdout: data_block_index_type: 0 2024-09-10T09:27:19.156 INFO:journalctl@ceph.mon.a.smithi005.stdout: index_shortening: 1 2024-09-10T09:27:19.157 INFO:journalctl@ceph.mon.a.smithi005.stdout: data_block_hash_table_util_ratio: 0.750000 2024-09-10T09:27:19.157 INFO:journalctl@ceph.mon.a.smithi005.stdout: checksum: 4 2024-09-10T09:27:19.157 INFO:journalctl@ceph.mon.a.smithi005.stdout: no_block_cache: 0 2024-09-10T09:27:19.157 INFO:journalctl@ceph.mon.a.smithi005.stdout: block_cache: 0x56412d42b1f0 2024-09-10T09:27:19.157 INFO:journalctl@ceph.mon.a.smithi005.stdout: block_cache_name: BinnedLRUCache 2024-09-10T09:27:19.157 INFO:journalctl@ceph.mon.a.smithi005.stdout: block_cache_options: 2024-09-10T09:27:19.157 INFO:journalctl@ceph.mon.a.smithi005.stdout: capacity : 536870912 2024-09-10T09:27:19.157 INFO:journalctl@ceph.mon.a.smithi005.stdout: num_shard_bits : 4 2024-09-10T09:27:19.157 INFO:journalctl@ceph.mon.a.smithi005.stdout: strict_capacity_limit : 0 2024-09-10T09:27:19.157 INFO:journalctl@ceph.mon.a.smithi005.stdout: high_pri_pool_ratio: 0.000 2024-09-10T09:27:19.157 INFO:journalctl@ceph.mon.a.smithi005.stdout: block_cache_compressed: (nil) 2024-09-10T09:27:19.157 INFO:journalctl@ceph.mon.a.smithi005.stdout: persistent_cache: (nil) 2024-09-10T09:27:19.157 INFO:journalctl@ceph.mon.a.smithi005.stdout: block_size: 4096 2024-09-10T09:27:19.157 INFO:journalctl@ceph.mon.a.smithi005.stdout: block_size_deviation: 10 2024-09-10T09:27:19.158 INFO:journalctl@ceph.mon.a.smithi005.stdout: block_restart_interval: 16 2024-09-10T09:27:19.158 INFO:journalctl@ceph.mon.a.smithi005.stdout: index_block_restart_interval: 1 2024-09-10T09:27:19.158 INFO:journalctl@ceph.mon.a.smithi005.stdout: metadata_block_size: 4096 2024-09-10T09:27:19.158 INFO:journalctl@ceph.mon.a.smithi005.stdout: partition_filters: 0 2024-09-10T09:27:19.158 INFO:journalctl@ceph.mon.a.smithi005.stdout: use_delta_encoding: 1 2024-09-10T09:27:19.158 INFO:journalctl@ceph.mon.a.smithi005.stdout: filter_policy: bloomfilter 2024-09-10T09:27:19.158 INFO:journalctl@ceph.mon.a.smithi005.stdout: whole_key_filtering: 1 2024-09-10T09:27:19.158 INFO:journalctl@ceph.mon.a.smithi005.stdout: verify_compression: 0 2024-09-10T09:27:19.158 INFO:journalctl@ceph.mon.a.smithi005.stdout: read_amp_bytes_per_bit: 0 2024-09-10T09:27:19.158 INFO:journalctl@ceph.mon.a.smithi005.stdout: format_version: 5 2024-09-10T09:27:19.158 INFO:journalctl@ceph.mon.a.smithi005.stdout: enable_index_compression: 1 2024-09-10T09:27:19.159 INFO:journalctl@ceph.mon.a.smithi005.stdout: block_align: 0 2024-09-10T09:27:19.159 INFO:journalctl@ceph.mon.a.smithi005.stdout: max_auto_readahead_size: 262144 2024-09-10T09:27:19.159 INFO:journalctl@ceph.mon.a.smithi005.stdout: prepopulate_block_cache: 0 2024-09-10T09:27:19.159 INFO:journalctl@ceph.mon.a.smithi005.stdout: initial_auto_readahead_size: 8192 2024-09-10T09:27:19.159 INFO:journalctl@ceph.mon.a.smithi005.stdout: num_file_reads_for_auto_readahead: 2 2024-09-10T09:27:19.159 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:18 smithi005 ceph-mon[30350]: rocksdb: Options.write_buffer_size: 33554432 2024-09-10T09:27:19.159 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:18 smithi005 ceph-mon[30350]: rocksdb: Options.max_write_buffer_number: 2 2024-09-10T09:27:19.159 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:18 smithi005 ceph-mon[30350]: rocksdb: Options.compression: NoCompression 2024-09-10T09:27:19.159 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:18 smithi005 ceph-mon[30350]: rocksdb: Options.bottommost_compression: Disabled 2024-09-10T09:27:19.159 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:18 smithi005 ceph-mon[30350]: rocksdb: Options.prefix_extractor: nullptr 2024-09-10T09:27:19.159 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:18 smithi005 ceph-mon[30350]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-10T09:27:19.160 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:18 smithi005 ceph-mon[30350]: rocksdb: Options.num_levels: 7 2024-09-10T09:27:19.160 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:18 smithi005 ceph-mon[30350]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-10T09:27:19.160 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:18 smithi005 ceph-mon[30350]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-10T09:27:19.160 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:18 smithi005 ceph-mon[30350]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-10T09:27:19.160 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:18 smithi005 ceph-mon[30350]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-10T09:27:19.160 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:18 smithi005 ceph-mon[30350]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-10T09:27:19.160 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:18 smithi005 ceph-mon[30350]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-10T09:27:19.160 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:18 smithi005 ceph-mon[30350]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-10T09:27:19.160 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:18 smithi005 ceph-mon[30350]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-10T09:27:19.160 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:18 smithi005 ceph-mon[30350]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-10T09:27:19.160 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:18 smithi005 ceph-mon[30350]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-10T09:27:19.160 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:18 smithi005 ceph-mon[30350]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-09-10T09:27:19.160 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:18 smithi005 ceph-mon[30350]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-09-10T09:27:19.161 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:18 smithi005 ceph-mon[30350]: rocksdb: Options.compression_opts.window_bits: -14 2024-09-10T09:27:19.161 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:18 smithi005 ceph-mon[30350]: rocksdb: Options.compression_opts.level: 32767 2024-09-10T09:27:19.161 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:18 smithi005 ceph-mon[30350]: rocksdb: Options.compression_opts.strategy: 0 2024-09-10T09:27:19.161 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:18 smithi005 ceph-mon[30350]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-10T09:27:19.161 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:18 smithi005 ceph-mon[30350]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-10T09:27:19.161 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:18 smithi005 ceph-mon[30350]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-09-10T09:27:19.161 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:18 smithi005 ceph-mon[30350]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-10T09:27:19.161 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:18 smithi005 ceph-mon[30350]: rocksdb: Options.compression_opts.enabled: false 2024-09-10T09:27:19.161 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:18 smithi005 ceph-mon[30350]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-09-10T09:27:19.161 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:18 smithi005 ceph-mon[30350]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-10T09:27:19.161 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:18 smithi005 ceph-mon[30350]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-10T09:27:19.161 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:18 smithi005 ceph-mon[30350]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-10T09:27:19.161 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:18 smithi005 ceph-mon[30350]: rocksdb: Options.target_file_size_base: 67108864 2024-09-10T09:27:19.162 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:18 smithi005 ceph-mon[30350]: rocksdb: Options.target_file_size_multiplier: 1 2024-09-10T09:27:19.162 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:18 smithi005 ceph-mon[30350]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-10T09:27:19.162 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:18 smithi005 ceph-mon[30350]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-10T09:27:19.162 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:18 smithi005 ceph-mon[30350]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-10T09:27:19.162 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:18 smithi005 ceph-mon[30350]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-10T09:27:19.162 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:18 smithi005 ceph-mon[30350]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-10T09:27:19.162 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:18 smithi005 ceph-mon[30350]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-10T09:27:19.162 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:18 smithi005 ceph-mon[30350]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-10T09:27:19.162 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:18 smithi005 ceph-mon[30350]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-10T09:27:19.162 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:18 smithi005 ceph-mon[30350]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-10T09:27:19.162 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:18 smithi005 ceph-mon[30350]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-10T09:27:19.162 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:18 smithi005 ceph-mon[30350]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-10T09:27:19.162 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:18 smithi005 ceph-mon[30350]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-10T09:27:19.162 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:18 smithi005 ceph-mon[30350]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-09-10T09:27:19.163 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:18 smithi005 ceph-mon[30350]: rocksdb: Options.arena_block_size: 1048576 2024-09-10T09:27:19.163 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:18 smithi005 ceph-mon[30350]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-10T09:27:19.163 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:18 smithi005 ceph-mon[30350]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-10T09:27:19.163 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:18 smithi005 ceph-mon[30350]: rocksdb: Options.disable_auto_compactions: 0 2024-09-10T09:27:19.163 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:18 smithi005 ceph-mon[30350]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-10T09:27:19.163 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:18 smithi005 ceph-mon[30350]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-10T09:27:19.163 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:18 smithi005 ceph-mon[30350]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-10T09:27:19.163 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:18 smithi005 ceph-mon[30350]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-10T09:27:19.163 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:18 smithi005 ceph-mon[30350]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-10T09:27:19.163 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:18 smithi005 ceph-mon[30350]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-10T09:27:19.163 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:18 smithi005 ceph-mon[30350]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-10T09:27:19.163 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:18 smithi005 ceph-mon[30350]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-10T09:27:19.164 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:18 smithi005 ceph-mon[30350]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-10T09:27:19.164 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:18 smithi005 ceph-mon[30350]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-10T09:27:19.164 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:18 smithi005 ceph-mon[30350]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-10T09:27:19.164 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:18 smithi005 ceph-mon[30350]: rocksdb: Options.inplace_update_support: 0 2024-09-10T09:27:19.164 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:18 smithi005 ceph-mon[30350]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-10T09:27:19.164 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:18 smithi005 ceph-mon[30350]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-10T09:27:19.164 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:18 smithi005 ceph-mon[30350]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-10T09:27:19.164 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:18 smithi005 ceph-mon[30350]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-10T09:27:19.164 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:18 smithi005 ceph-mon[30350]: rocksdb: Options.bloom_locality: 0 2024-09-10T09:27:19.164 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:18 smithi005 ceph-mon[30350]: rocksdb: Options.max_successive_merges: 0 2024-09-10T09:27:19.165 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:18 smithi005 ceph-mon[30350]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-10T09:27:19.165 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:18 smithi005 ceph-mon[30350]: rocksdb: Options.paranoid_file_checks: 0 2024-09-10T09:27:19.165 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:18 smithi005 ceph-mon[30350]: rocksdb: Options.force_consistency_checks: 1 2024-09-10T09:27:19.165 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:18 smithi005 ceph-mon[30350]: rocksdb: Options.report_bg_io_stats: 0 2024-09-10T09:27:19.165 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:18 smithi005 ceph-mon[30350]: rocksdb: Options.ttl: 2592000 2024-09-10T09:27:19.165 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:18 smithi005 ceph-mon[30350]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-10T09:27:19.165 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:18 smithi005 ceph-mon[30350]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-09-10T09:27:19.165 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:18 smithi005 ceph-mon[30350]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-09-10T09:27:19.165 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:18 smithi005 ceph-mon[30350]: rocksdb: Options.enable_blob_files: false 2024-09-10T09:27:19.165 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:18 smithi005 ceph-mon[30350]: rocksdb: Options.min_blob_size: 0 2024-09-10T09:27:19.165 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:18 smithi005 ceph-mon[30350]: rocksdb: Options.blob_file_size: 268435456 2024-09-10T09:27:19.165 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:18 smithi005 ceph-mon[30350]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-10T09:27:19.165 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:18 smithi005 ceph-mon[30350]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-10T09:27:19.165 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:18 smithi005 ceph-mon[30350]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-10T09:27:19.166 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:18 smithi005 ceph-mon[30350]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-09-10T09:27:19.166 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:18 smithi005 ceph-mon[30350]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-09-10T09:27:19.166 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:18 smithi005 ceph-mon[30350]: rocksdb: Options.blob_file_starting_level: 0 2024-09-10T09:27:19.166 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:18 smithi005 ceph-mon[30350]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-09-10T09:27:19.166 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:18 smithi005 ceph-mon[30350]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-09-10T09:27:19.166 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:18 smithi005 ceph-mon[30350]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-09-10T09:27:19.166 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:18 smithi005 ceph-mon[30350]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: d888b1cf-d8dd-4f9e-ae72-a5e461866f57 2024-09-10T09:27:19.166 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:18 smithi005 ceph-mon[30350]: rocksdb: EVENT_LOG_v1 {"time_micros": 1725960438891603, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-09-10T09:27:19.166 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:18 smithi005 ceph-mon[30350]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-09-10T09:27:19.166 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:18 smithi005 ceph-mon[30350]: rocksdb: EVENT_LOG_v1 {"time_micros": 1725960438893595, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 100434, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 328, "table_properties": {"data_size": 98328, "index_size": 298, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 773, "raw_key_size": 14320, "raw_average_key_size": 49, "raw_value_size": 90581, "raw_average_value_size": 315, "num_data_blocks": 12, "num_entries": 287, "num_filter_entries": 287, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1725960438, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d888b1cf-d8dd-4f9e-ae72-a5e461866f57", "db_session_id": "U4DQNGW6BPH3NI2CPS2G", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-09-10T09:27:19.167 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:18 smithi005 ceph-mon[30350]: rocksdb: EVENT_LOG_v1 {"time_micros": 1725960438893787, "job": 1, "event": "recovery_finished"} 2024-09-10T09:27:19.167 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:18 smithi005 ceph-mon[30350]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-09-10T09:27:19.167 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:18 smithi005 ceph-mon[30350]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-10T09:27:19.167 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:18 smithi005 ceph-mon[30350]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x56412d45ae00 2024-09-10T09:27:19.167 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:18 smithi005 ceph-mon[30350]: rocksdb: DB pointer 0x56412d556000 2024-09-10T09:27:19.167 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:18 smithi005 ceph-mon[30350]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-09-10T09:27:19.167 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:18 smithi005 ceph-mon[30350]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-09-10T09:27:19.167 INFO:journalctl@ceph.mon.a.smithi005.stdout: ** DB Stats ** 2024-09-10T09:27:19.167 INFO:journalctl@ceph.mon.a.smithi005.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-10T09:27:19.167 INFO:journalctl@ceph.mon.a.smithi005.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-10T09:27:19.167 INFO:journalctl@ceph.mon.a.smithi005.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-10T09:27:19.167 INFO:journalctl@ceph.mon.a.smithi005.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-10T09:27:19.167 INFO:journalctl@ceph.mon.a.smithi005.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-10T09:27:19.168 INFO:journalctl@ceph.mon.a.smithi005.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-10T09:27:19.168 INFO:journalctl@ceph.mon.a.smithi005.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-10T09:27:19.168 INFO:journalctl@ceph.mon.a.smithi005.stdout: 2024-09-10T09:27:19.168 INFO:journalctl@ceph.mon.a.smithi005.stdout: ** Compaction Stats [default] ** 2024-09-10T09:27:19.168 INFO:journalctl@ceph.mon.a.smithi005.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-10T09:27:19.168 INFO:journalctl@ceph.mon.a.smithi005.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-09-10T09:27:19.168 INFO:journalctl@ceph.mon.a.smithi005.stdout: L0 2/0 99.94 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 73.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-10T09:27:19.168 INFO:journalctl@ceph.mon.a.smithi005.stdout: Sum 2/0 99.94 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 73.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-10T09:27:19.168 INFO:journalctl@ceph.mon.a.smithi005.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 73.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-10T09:27:19.168 INFO:journalctl@ceph.mon.a.smithi005.stdout: 2024-09-10T09:27:19.169 INFO:journalctl@ceph.mon.a.smithi005.stdout: ** Compaction Stats [default] ** 2024-09-10T09:27:19.169 INFO:journalctl@ceph.mon.a.smithi005.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-10T09:27:19.169 INFO:journalctl@ceph.mon.a.smithi005.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-10T09:27:19.169 INFO:journalctl@ceph.mon.a.smithi005.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 73.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-10T09:27:19.169 INFO:journalctl@ceph.mon.a.smithi005.stdout: 2024-09-10T09:27:19.169 INFO:journalctl@ceph.mon.a.smithi005.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-09-10T09:27:19.169 INFO:journalctl@ceph.mon.a.smithi005.stdout: 2024-09-10T09:27:19.169 INFO:journalctl@ceph.mon.a.smithi005.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-10T09:27:19.169 INFO:journalctl@ceph.mon.a.smithi005.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-10T09:27:19.169 INFO:journalctl@ceph.mon.a.smithi005.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-10T09:27:19.169 INFO:journalctl@ceph.mon.a.smithi005.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-10T09:27:19.169 INFO:journalctl@ceph.mon.a.smithi005.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-10T09:27:19.170 INFO:journalctl@ceph.mon.a.smithi005.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-10T09:27:19.170 INFO:journalctl@ceph.mon.a.smithi005.stdout: Cumulative compaction: 0.00 GB write, 15.98 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-10T09:27:19.170 INFO:journalctl@ceph.mon.a.smithi005.stdout: Interval compaction: 0.00 GB write, 15.98 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-10T09:27:19.170 INFO:journalctl@ceph.mon.a.smithi005.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-10T09:27:19.170 INFO:journalctl@ceph.mon.a.smithi005.stdout: Block cache BinnedLRUCache@0x56412d42b1f0#2 capacity: 512.00 MB usage: 1.52 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1e-05 secs_since: 0 2024-09-10T09:27:19.170 INFO:journalctl@ceph.mon.a.smithi005.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,1.02 KB,0.000193715%) IndexBlock(2,0.50 KB,9.53674e-05%) Misc(1,0.00 KB,0%) 2024-09-10T09:27:19.170 INFO:journalctl@ceph.mon.a.smithi005.stdout: 2024-09-10T09:27:19.170 INFO:journalctl@ceph.mon.a.smithi005.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-10T09:27:19.170 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:18 smithi005 ceph-mon[30350]: starting mon.a rank 0 at public addrs [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] at bind addrs [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid bade4b50-6f56-11ef-bcea-c7b262605968 2024-09-10T09:27:19.170 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:18 smithi005 ceph-mon[30350]: mon.a@-1(???) e1 preinit fsid bade4b50-6f56-11ef-bcea-c7b262605968 2024-09-10T09:27:19.170 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:18 smithi005 ceph-mon[30350]: mon.a@-1(???).mds e1 new map 2024-09-10T09:27:19.170 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:18 smithi005 ceph-mon[30350]: mon.a@-1(???).mds e1 print_map 2024-09-10T09:27:19.170 INFO:journalctl@ceph.mon.a.smithi005.stdout: e1 2024-09-10T09:27:19.171 INFO:journalctl@ceph.mon.a.smithi005.stdout: btime 2024-09-10T09:27:13:298289+0000 2024-09-10T09:27:19.171 INFO:journalctl@ceph.mon.a.smithi005.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-10T09:27:19.171 INFO:journalctl@ceph.mon.a.smithi005.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-10T09:27:19.171 INFO:journalctl@ceph.mon.a.smithi005.stdout: legacy client fscid: -1 2024-09-10T09:27:19.171 INFO:journalctl@ceph.mon.a.smithi005.stdout: 2024-09-10T09:27:19.171 INFO:journalctl@ceph.mon.a.smithi005.stdout: No filesystems configured 2024-09-10T09:27:19.171 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:18 smithi005 ceph-mon[30350]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-10T09:27:19.171 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:18 smithi005 ceph-mon[30350]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-10T09:27:19.171 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:18 smithi005 ceph-mon[30350]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-10T09:27:19.171 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:18 smithi005 ceph-mon[30350]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-10T09:27:19.171 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:18 smithi005 ceph-mon[30350]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-09-10T09:27:19.171 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:18 smithi005 bash[30331]: 36f7420b52880ca9c13a94f30b28e7998c72e348aea621b0a554120f9a6a4df8 2024-09-10T09:27:19.171 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:18 smithi005 ceph-mon[30350]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-10T09:27:19.171 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:18 smithi005 ceph-mon[30350]: monmap epoch 1 2024-09-10T09:27:19.171 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:18 smithi005 ceph-mon[30350]: fsid bade4b50-6f56-11ef-bcea-c7b262605968 2024-09-10T09:27:19.172 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:18 smithi005 ceph-mon[30350]: last_changed 2024-09-10T09:27:08.776915+0000 2024-09-10T09:27:19.172 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:18 smithi005 ceph-mon[30350]: created 2024-09-10T09:27:08.776915+0000 2024-09-10T09:27:19.172 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:18 smithi005 ceph-mon[30350]: min_mon_release 19 (squid) 2024-09-10T09:27:19.172 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:18 smithi005 ceph-mon[30350]: election_strategy: 1 2024-09-10T09:27:19.172 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:18 smithi005 ceph-mon[30350]: 0: [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] mon.a 2024-09-10T09:27:19.172 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:18 smithi005 ceph-mon[30350]: fsmap 2024-09-10T09:27:19.172 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:18 smithi005 ceph-mon[30350]: osdmap e1: 0 total, 0 up, 0 in 2024-09-10T09:27:19.172 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:18 smithi005 ceph-mon[30350]: mgrmap e1: no daemons active 2024-09-10T09:27:19.172 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:18 smithi005 systemd[1]: Started Ceph mon.a for bade4b50-6f56-11ef-bcea-c7b262605968. 2024-09-10T09:27:20.193 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:19.508+0000 7fcbb6200640 1 Processor -- start 2024-09-10T09:27:20.194 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:19.508+0000 7fcbb6200640 1 -- start start 2024-09-10T09:27:20.194 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:19.508+0000 7fcbb6200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcbb007de80 0x7fcbb007c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:27:20.194 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:19.508+0000 7fcbb6200640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fcbb007c8b0 con 0x7fcbb007de80 2024-09-10T09:27:20.194 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:19.509+0000 7fcbb4e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcbb007de80 0x7fcbb007c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:27:20.194 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:19.509+0000 7fcbb4e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcbb007de80 0x7fcbb007c2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50068/0 (socket says 172.21.15.5:50068) 2024-09-10T09:27:20.194 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:19.509+0000 7fcbb4e00640 1 -- 172.21.15.5:0/2396728605 learned_addr learned my addr 172.21.15.5:0/2396728605 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:27:20.194 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:19.509+0000 7fcbb4e00640 1 -- 172.21.15.5:0/2396728605 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcbb007ca30 con 0x7fcbb007de80 2024-09-10T09:27:20.194 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:19.512+0000 7fcbb4e00640 1 --2- 172.21.15.5:0/2396728605 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcbb007de80 0x7fcbb007c2e0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fcba0009820 tx=0x7fcba002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac9bfcf5b78b7db5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:27:20.194 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:19.512+0000 7fcbafe00640 1 -- 172.21.15.5:0/2396728605 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcba0037070 con 0x7fcbb007de80 2024-09-10T09:27:20.194 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:19.512+0000 7fcbafe00640 1 -- 172.21.15.5:0/2396728605 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fcba0032900 con 0x7fcbb007de80 2024-09-10T09:27:20.194 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:19.512+0000 7fcbafe00640 1 -- 172.21.15.5:0/2396728605 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcba0030850 con 0x7fcbb007de80 2024-09-10T09:27:20.194 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:19.512+0000 7fcbb6200640 1 -- 172.21.15.5:0/2396728605 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcbb007de80 msgr2=0x7fcbb007c2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:27:20.195 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:19.513+0000 7fcbb6200640 1 --2- 172.21.15.5:0/2396728605 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcbb007de80 0x7fcbb007c2e0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fcba0009820 tx=0x7fcba002a270 comp rx=0 tx=0).stop 2024-09-10T09:27:20.195 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:19.513+0000 7fcbb6200640 1 -- 172.21.15.5:0/2396728605 shutdown_connections 2024-09-10T09:27:20.195 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:19.513+0000 7fcbb6200640 1 --2- 172.21.15.5:0/2396728605 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcbb007de80 0x7fcbb007c2e0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:27:20.195 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:19.513+0000 7fcbb6200640 1 -- 172.21.15.5:0/2396728605 >> 172.21.15.5:0/2396728605 conn(0x7fcbb0102c10 msgr2=0x7fcbb0105030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:27:20.195 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:19.513+0000 7fcbb6200640 1 -- 172.21.15.5:0/2396728605 shutdown_connections 2024-09-10T09:27:20.195 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:19.513+0000 7fcbb6200640 1 -- 172.21.15.5:0/2396728605 wait complete. 2024-09-10T09:27:20.195 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:19.514+0000 7fcbb6200640 1 Processor -- start 2024-09-10T09:27:20.195 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:19.514+0000 7fcbb6200640 1 -- start start 2024-09-10T09:27:20.195 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:19.514+0000 7fcbb6200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcbb007de80 0x7fcbb019d720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:27:20.195 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:19.514+0000 7fcbb6200640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fcbb010da70 con 0x7fcbb007de80 2024-09-10T09:27:20.195 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:19.514+0000 7fcbb4e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcbb007de80 0x7fcbb019d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:27:20.195 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:19.514+0000 7fcbb4e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcbb007de80 0x7fcbb019d720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50084/0 (socket says 172.21.15.5:50084) 2024-09-10T09:27:20.195 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:19.514+0000 7fcbb4e00640 1 -- 172.21.15.5:0/989985014 learned_addr learned my addr 172.21.15.5:0/989985014 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:27:20.195 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:19.515+0000 7fcbb4e00640 1 -- 172.21.15.5:0/989985014 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcbb019dc60 con 0x7fcbb007de80 2024-09-10T09:27:20.196 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:19.515+0000 7fcbb4e00640 1 --2- 172.21.15.5:0/989985014 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcbb007de80 0x7fcbb019d720 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fcba0031000 tx=0x7fcba00326e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:27:20.196 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:19.515+0000 7fcbae000640 1 -- 172.21.15.5:0/989985014 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcba0042070 con 0x7fcbb007de80 2024-09-10T09:27:20.196 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:19.515+0000 7fcbae000640 1 -- 172.21.15.5:0/989985014 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fcba0032be0 con 0x7fcbb007de80 2024-09-10T09:27:20.196 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:19.515+0000 7fcbb6200640 1 -- 172.21.15.5:0/989985014 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcbb019def0 con 0x7fcbb007de80 2024-09-10T09:27:20.196 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:19.515+0000 7fcbae000640 1 -- 172.21.15.5:0/989985014 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcba0037040 con 0x7fcbb007de80 2024-09-10T09:27:20.196 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:19.515+0000 7fcbb6200640 1 -- 172.21.15.5:0/989985014 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcbb01a0be0 con 0x7fcbb007de80 2024-09-10T09:27:20.196 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:19.516+0000 7fcbae000640 1 -- 172.21.15.5:0/989985014 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fcba0045070 con 0x7fcbb007de80 2024-09-10T09:27:20.196 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:19.516+0000 7fcbae000640 1 -- 172.21.15.5:0/989985014 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fcba0047830 con 0x7fcbb007de80 2024-09-10T09:27:20.196 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:19.516+0000 7fcbb6200640 1 -- 172.21.15.5:0/989985014 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcbb007c360 con 0x7fcbb007de80 2024-09-10T09:27:20.196 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:19.520+0000 7fcbae000640 1 -- 172.21.15.5:0/989985014 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7fcba004c030 con 0x7fcbb007de80 2024-09-10T09:27:20.196 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:19.582+0000 7fcbb6200640 1 -- 172.21.15.5:0/989985014 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) -- 0x7fcbb01a0db0 con 0x7fcbb007de80 2024-09-10T09:27:20.196 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:19.584+0000 7fcbae000640 1 -- 172.21.15.5:0/989985014 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) ==== 124+0+0 (secure 0 0 0) 0x7fcba0032d90 con 0x7fcbb007de80 2024-09-10T09:27:20.197 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:19.584+0000 7fcbb6200640 1 -- 172.21.15.5:0/989985014 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcbb007de80 msgr2=0x7fcbb019d720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:27:20.197 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:19.584+0000 7fcbb6200640 1 --2- 172.21.15.5:0/989985014 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcbb007de80 0x7fcbb019d720 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fcba0031000 tx=0x7fcba00326e0 comp rx=0 tx=0).stop 2024-09-10T09:27:20.197 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:19.584+0000 7fcbb6200640 1 -- 172.21.15.5:0/989985014 shutdown_connections 2024-09-10T09:27:20.197 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:19.584+0000 7fcbb6200640 1 --2- 172.21.15.5:0/989985014 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcbb007de80 0x7fcbb019d720 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:27:20.197 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:19.585+0000 7fcbb6200640 1 -- 172.21.15.5:0/989985014 >> 172.21.15.5:0/989985014 conn(0x7fcbb0102c10 msgr2=0x7fcbb0103830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:27:20.197 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:19.585+0000 7fcbb6200640 1 -- 172.21.15.5:0/989985014 shutdown_connections 2024-09-10T09:27:20.197 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:19.585+0000 7fcbb6200640 1 -- 172.21.15.5:0/989985014 wait complete. 2024-09-10T09:27:20.198 INFO:teuthology.orchestra.run.smithi005.stdout:Wrote config to /etc/ceph/ceph.conf 2024-09-10T09:27:20.215 INFO:teuthology.orchestra.run.smithi005.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-09-10T09:27:20.215 INFO:teuthology.orchestra.run.smithi005.stdout:Creating mgr... 2024-09-10T09:27:20.216 INFO:teuthology.orchestra.run.smithi005.stdout:Verifying port 0.0.0.0:9283 ... 2024-09-10T09:27:20.217 INFO:teuthology.orchestra.run.smithi005.stdout:Verifying port 0.0.0.0:8765 ... 2024-09-10T09:27:20.726 INFO:teuthology.orchestra.run.smithi005.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-bade4b50-6f56-11ef-bcea-c7b262605968@mgr.x 2024-09-10T09:27:20.726 INFO:teuthology.orchestra.run.smithi005.stdout:systemctl: stderr Failed to reset failed state of unit ceph-bade4b50-6f56-11ef-bcea-c7b262605968@mgr.x.service: Unit ceph-bade4b50-6f56-11ef-bcea-c7b262605968@mgr.x.service not loaded. 2024-09-10T09:27:20.951 INFO:teuthology.orchestra.run.smithi005.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-bade4b50-6f56-11ef-bcea-c7b262605968.target.wants/ceph-bade4b50-6f56-11ef-bcea-c7b262605968@mgr.x.service → /etc/systemd/system/ceph-bade4b50-6f56-11ef-bcea-c7b262605968@.service. 2024-09-10T09:27:20.974 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:20 smithi005 ceph-mon[30350]: from='client.? 172.21.15.5:0/989985014' entity='client.admin' 2024-09-10T09:27:21.292 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 10 09:27:20 smithi005 systemd[1]: Starting Ceph mgr.x for bade4b50-6f56-11ef-bcea-c7b262605968... 2024-09-10T09:27:21.556 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 10 09:27:21 smithi005 podman[30560]: 2024-09-10 09:27:21.195572013 +0000 UTC m=+0.027040961 image pull 41d67112835771c3efca60c6b8fe297c7ddaecdc26cea26fd28aa18636e49005 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b 2024-09-10T09:27:21.556 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 10 09:27:21 smithi005 podman[30560]: 2024-09-10 09:27:21.292920669 +0000 UTC m=+0.124389602 container create 65d4fbac1f457ada3a5151d42ec4b7808cb8e28032f36febab2944fc1446a53e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b, name=ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mgr-x, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux ) 2024-09-10T09:27:21.556 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 10 09:27:21 smithi005 podman[30560]: 2024-09-10 09:27:21.487659841 +0000 UTC m=+0.319128797 container init 65d4fbac1f457ada3a5151d42ec4b7808cb8e28032f36febab2944fc1446a53e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b, name=ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mgr-x, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, ceph=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-09-10T09:27:21.557 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 10 09:27:21 smithi005 podman[30560]: 2024-09-10 09:27:21.494279032 +0000 UTC m=+0.325747973 container start 65d4fbac1f457ada3a5151d42ec4b7808cb8e28032f36febab2944fc1446a53e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b, name=ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mgr-x, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-10T09:27:21.602 INFO:teuthology.orchestra.run.smithi005.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-10T09:27:21.602 INFO:teuthology.orchestra.run.smithi005.stdout:systemctl: stdout disabled 2024-09-10T09:27:21.616 INFO:teuthology.orchestra.run.smithi005.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-10T09:27:21.616 INFO:teuthology.orchestra.run.smithi005.stdout:systemctl: stdout inactive 2024-09-10T09:27:21.616 INFO:teuthology.orchestra.run.smithi005.stdout:firewalld.service is not enabled 2024-09-10T09:27:21.617 INFO:teuthology.orchestra.run.smithi005.stdout:Not possible to enable service . firewalld.service is not available 2024-09-10T09:27:21.635 INFO:teuthology.orchestra.run.smithi005.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-10T09:27:21.636 INFO:teuthology.orchestra.run.smithi005.stdout:systemctl: stdout disabled 2024-09-10T09:27:21.651 INFO:teuthology.orchestra.run.smithi005.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-10T09:27:21.651 INFO:teuthology.orchestra.run.smithi005.stdout:systemctl: stdout inactive 2024-09-10T09:27:21.651 INFO:teuthology.orchestra.run.smithi005.stdout:firewalld.service is not enabled 2024-09-10T09:27:21.652 INFO:teuthology.orchestra.run.smithi005.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-09-10T09:27:21.652 INFO:teuthology.orchestra.run.smithi005.stdout:Waiting for mgr to start... 2024-09-10T09:27:21.652 INFO:teuthology.orchestra.run.smithi005.stdout:Waiting for mgr... 2024-09-10T09:27:21.925 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 10 09:27:21 smithi005 bash[30560]: 65d4fbac1f457ada3a5151d42ec4b7808cb8e28032f36febab2944fc1446a53e 2024-09-10T09:27:21.925 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 10 09:27:21 smithi005 systemd[1]: Started Ceph mgr.x for bade4b50-6f56-11ef-bcea-c7b262605968. 2024-09-10T09:27:21.925 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 10 09:27:21 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mgr-x[30575]: 2024-09-10T09:27:21.683+0000 7fcc18a2d040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-10T09:27:21.925 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 10 09:27:21 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mgr-x[30575]: 2024-09-10T09:27:21.763+0000 7fcc18a2d040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-10T09:27:22.178 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 10 09:27:21 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mgr-x[30575]: 2024-09-10T09:27:21.966+0000 7fcc18a2d040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-10T09:27:22.179 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 10 09:27:22 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mgr-x[30575]: 2024-09-10T09:27:22.053+0000 7fcc18a2d040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-10T09:27:22.519 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:22 smithi005 ceph-mon[30350]: from='client.? 172.21.15.5:0/3870048464' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-10T09:27:22.519 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 10 09:27:22 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mgr-x[30575]: 2024-09-10T09:27:22.243+0000 7fcc18a2d040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-10T09:27:22.519 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 10 09:27:22 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mgr-x[30575]: 2024-09-10T09:27:22.425+0000 7fcc18a2d040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-10T09:27:22.817 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout 2024-09-10T09:27:22.819 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout { 2024-09-10T09:27:22.819 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "fsid": "bade4b50-6f56-11ef-bcea-c7b262605968", 2024-09-10T09:27:22.819 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "health": { 2024-09-10T09:27:22.819 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-10T09:27:22.819 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-10T09:27:22.819 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-10T09:27:22.820 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout }, 2024-09-10T09:27:22.820 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-10T09:27:22.820 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-10T09:27:22.820 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout 0 2024-09-10T09:27:22.820 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout ], 2024-09-10T09:27:22.820 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-10T09:27:22.820 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "a" 2024-09-10T09:27:22.820 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout ], 2024-09-10T09:27:22.820 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "quorum_age": 3, 2024-09-10T09:27:22.820 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-10T09:27:22.820 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T09:27:22.820 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-10T09:27:22.820 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-10T09:27:22.820 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout }, 2024-09-10T09:27:22.821 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-10T09:27:22.821 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T09:27:22.821 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-10T09:27:22.821 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-10T09:27:22.821 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-10T09:27:22.828 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-10T09:27:22.829 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-10T09:27:22.829 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-10T09:27:22.829 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout }, 2024-09-10T09:27:22.829 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-10T09:27:22.829 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-10T09:27:22.829 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-10T09:27:22.829 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-10T09:27:22.829 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-10T09:27:22.829 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-10T09:27:22.829 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-10T09:27:22.829 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-10T09:27:22.829 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-10T09:27:22.829 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout }, 2024-09-10T09:27:22.830 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-10T09:27:22.830 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T09:27:22.830 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "btime": "2024-09-10T09:27:13:298289+0000", 2024-09-10T09:27:22.830 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-10T09:27:22.830 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-10T09:27:22.830 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout }, 2024-09-10T09:27:22.830 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-10T09:27:22.830 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "available": false, 2024-09-10T09:27:22.830 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-10T09:27:22.830 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-10T09:27:22.830 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-10T09:27:22.830 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-10T09:27:22.830 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "restful" 2024-09-10T09:27:22.830 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout ], 2024-09-10T09:27:22.830 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-10T09:27:22.831 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout }, 2024-09-10T09:27:22.831 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-10T09:27:22.831 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T09:27:22.831 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "modified": "2024-09-10T09:27:13.299038+0000", 2024-09-10T09:27:22.831 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-10T09:27:22.831 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout }, 2024-09-10T09:27:22.831 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-10T09:27:22.831 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout } 2024-09-10T09:27:22.831 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:22.176+0000 7f63fca00640 1 Processor -- start 2024-09-10T09:27:22.831 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:22.176+0000 7f63fca00640 1 -- start start 2024-09-10T09:27:22.831 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:22.177+0000 7f63fca00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f63f8109920 0x7f63f8109d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:27:22.831 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:22.177+0000 7f63fca00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f63f810a310 con 0x7f63f8109920 2024-09-10T09:27:22.832 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:22.177+0000 7f63f5e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f63f8109920 0x7f63f8109d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:27:22.832 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:22.177+0000 7f63f5e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f63f8109920 0x7f63f8109d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51294/0 (socket says 172.21.15.5:51294) 2024-09-10T09:27:22.832 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:22.177+0000 7f63f5e00640 1 -- 172.21.15.5:0/3132512150 learned_addr learned my addr 172.21.15.5:0/3132512150 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:27:22.832 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:22.178+0000 7f63f5e00640 1 -- 172.21.15.5:0/3132512150 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63f810aa10 con 0x7f63f8109920 2024-09-10T09:27:22.832 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:22.179+0000 7f63f5e00640 1 --2- 172.21.15.5:0/3132512150 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f63f8109920 0x7f63f8109d40 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f63e8009820 tx=0x7f63e802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=230e4c017307228b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:27:22.832 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:22.179+0000 7f63f4a00640 1 -- 172.21.15.5:0/3132512150 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f63e8037070 con 0x7f63f8109920 2024-09-10T09:27:22.832 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:22.179+0000 7f63f4a00640 1 -- 172.21.15.5:0/3132512150 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f63e8032900 con 0x7f63f8109920 2024-09-10T09:27:22.832 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:22.179+0000 7f63f4a00640 1 -- 172.21.15.5:0/3132512150 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f63e8030850 con 0x7f63f8109920 2024-09-10T09:27:22.832 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:22.180+0000 7f63fca00640 1 -- 172.21.15.5:0/3132512150 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f63f8109920 msgr2=0x7f63f8109d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:27:22.832 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:22.180+0000 7f63fca00640 1 --2- 172.21.15.5:0/3132512150 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f63f8109920 0x7f63f8109d40 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f63e8009820 tx=0x7f63e802a270 comp rx=0 tx=0).stop 2024-09-10T09:27:22.832 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:22.180+0000 7f63fca00640 1 -- 172.21.15.5:0/3132512150 shutdown_connections 2024-09-10T09:27:22.832 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:22.180+0000 7f63fca00640 1 --2- 172.21.15.5:0/3132512150 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f63f8109920 0x7f63f8109d40 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:27:22.833 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:22.180+0000 7f63fca00640 1 -- 172.21.15.5:0/3132512150 >> 172.21.15.5:0/3132512150 conn(0x7f63f807cf40 msgr2=0x7f63f807d350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:27:22.833 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:22.180+0000 7f63fca00640 1 -- 172.21.15.5:0/3132512150 shutdown_connections 2024-09-10T09:27:22.833 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:22.180+0000 7f63fca00640 1 -- 172.21.15.5:0/3132512150 wait complete. 2024-09-10T09:27:22.833 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:22.182+0000 7f63fca00640 1 Processor -- start 2024-09-10T09:27:22.833 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:22.182+0000 7f63fca00640 1 -- start start 2024-09-10T09:27:22.833 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:22.182+0000 7f63fca00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f63f8109920 0x7f63f819d830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:27:22.833 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:22.182+0000 7f63fca00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f63f810ddc0 con 0x7f63f8109920 2024-09-10T09:27:22.833 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:22.183+0000 7f63f5e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f63f8109920 0x7f63f819d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:27:22.833 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:22.183+0000 7f63f5e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f63f8109920 0x7f63f819d830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51296/0 (socket says 172.21.15.5:51296) 2024-09-10T09:27:22.833 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:22.183+0000 7f63f5e00640 1 -- 172.21.15.5:0/3870048464 learned_addr learned my addr 172.21.15.5:0/3870048464 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:27:22.833 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:22.183+0000 7f63f5e00640 1 -- 172.21.15.5:0/3870048464 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63f819dd70 con 0x7f63f8109920 2024-09-10T09:27:22.833 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:22.183+0000 7f63f5e00640 1 --2- 172.21.15.5:0/3870048464 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f63f8109920 0x7f63f819d830 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f63e80097f0 tx=0x7f63e8030d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:27:22.833 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:22.184+0000 7f63eea00640 1 -- 172.21.15.5:0/3870048464 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f63e8037040 con 0x7f63f8109920 2024-09-10T09:27:22.833 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:22.184+0000 7f63eea00640 1 -- 172.21.15.5:0/3870048464 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f63e8040070 con 0x7f63f8109920 2024-09-10T09:27:22.834 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:22.184+0000 7f63fca00640 1 -- 172.21.15.5:0/3870048464 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63f819e000 con 0x7f63f8109920 2024-09-10T09:27:22.834 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:22.184+0000 7f63eea00640 1 -- 172.21.15.5:0/3870048464 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f63e803a390 con 0x7f63f8109920 2024-09-10T09:27:22.834 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:22.184+0000 7f63fca00640 1 -- 172.21.15.5:0/3870048464 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63f81a0cf0 con 0x7f63f8109920 2024-09-10T09:27:22.834 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:22.185+0000 7f63eea00640 1 -- 172.21.15.5:0/3870048464 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f63e8032900 con 0x7f63f8109920 2024-09-10T09:27:22.834 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:22.185+0000 7f63eea00640 1 -- 172.21.15.5:0/3870048464 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f63e80454f0 con 0x7f63f8109920 2024-09-10T09:27:22.834 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:22.185+0000 7f63fca00640 1 -- 172.21.15.5:0/3870048464 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f63f810df40 con 0x7f63f8109920 2024-09-10T09:27:22.834 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:22.188+0000 7f63eea00640 1 -- 172.21.15.5:0/3870048464 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f63e803b430 con 0x7f63f8109920 2024-09-10T09:27:22.834 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:22.255+0000 7f63fca00640 1 -- 172.21.15.5:0/3870048464 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f63f81a0ec0 con 0x7f63f8109920 2024-09-10T09:27:22.834 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:22.255+0000 7f63eea00640 1 -- 172.21.15.5:0/3870048464 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1291 (secure 0 0 0) 0x7f63e803b7c0 con 0x7f63f8109920 2024-09-10T09:27:22.834 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:22.257+0000 7f63fca00640 1 -- 172.21.15.5:0/3870048464 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f63f8109920 msgr2=0x7f63f819d830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:27:22.834 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:22.257+0000 7f63fca00640 1 --2- 172.21.15.5:0/3870048464 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f63f8109920 0x7f63f819d830 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f63e80097f0 tx=0x7f63e8030d80 comp rx=0 tx=0).stop 2024-09-10T09:27:22.834 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:22.257+0000 7f63fca00640 1 -- 172.21.15.5:0/3870048464 shutdown_connections 2024-09-10T09:27:22.834 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:22.257+0000 7f63fca00640 1 --2- 172.21.15.5:0/3870048464 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f63f8109920 0x7f63f819d830 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:27:22.834 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:22.257+0000 7f63fca00640 1 -- 172.21.15.5:0/3870048464 >> 172.21.15.5:0/3870048464 conn(0x7f63f807cf40 msgr2=0x7f63f8107540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:27:22.835 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:22.257+0000 7f63fca00640 1 -- 172.21.15.5:0/3870048464 shutdown_connections 2024-09-10T09:27:22.835 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:22.257+0000 7f63fca00640 1 -- 172.21.15.5:0/3870048464 wait complete. 2024-09-10T09:27:22.835 INFO:teuthology.orchestra.run.smithi005.stdout:mgr not available, waiting (1/15)... 2024-09-10T09:27:23.129 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 10 09:27:22 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mgr-x[30575]: 2024-09-10T09:27:22.830+0000 7fcc18a2d040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-10T09:27:23.129 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 10 09:27:22 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mgr-x[30575]: 2024-09-10T09:27:22.905+0000 7fcc18a2d040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-10T09:27:23.129 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 10 09:27:23 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mgr-x[30575]: 2024-09-10T09:27:23.127+0000 7fcc18a2d040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-10T09:27:24.556 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 10 09:27:24 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mgr-x[30575]: 2024-09-10T09:27:24.280+0000 7fcc18a2d040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-10T09:27:24.556 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 10 09:27:24 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mgr-x[30575]: 2024-09-10T09:27:24.364+0000 7fcc18a2d040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-10T09:27:25.516 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:25 smithi005 ceph-mon[30350]: from='client.? 172.21.15.5:0/2808896701' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-10T09:27:25.516 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 10 09:27:25 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mgr-x[30575]: 2024-09-10T09:27:25.366+0000 7fcc18a2d040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-10T09:27:25.516 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 10 09:27:25 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mgr-x[30575]: 2024-09-10T09:27:25.514+0000 7fcc18a2d040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-10T09:27:25.806 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 10 09:27:25 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mgr-x[30575]: 2024-09-10T09:27:25.687+0000 7fcc18a2d040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-10T09:27:25.806 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 10 09:27:25 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mgr-x[30575]: 2024-09-10T09:27:25.763+0000 7fcc18a2d040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-10T09:27:26.107 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 10 09:27:25 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mgr-x[30575]: 2024-09-10T09:27:25.945+0000 7fcc18a2d040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-10T09:27:26.107 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 10 09:27:26 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mgr-x[30575]: 2024-09-10T09:27:26.030+0000 7fcc18a2d040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-10T09:27:26.107 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 10 09:27:26 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mgr-x[30575]: 2024-09-10T09:27:26.106+0000 7fcc18a2d040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-10T09:27:26.150 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout 2024-09-10T09:27:26.150 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout { 2024-09-10T09:27:26.150 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "fsid": "bade4b50-6f56-11ef-bcea-c7b262605968", 2024-09-10T09:27:26.151 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "health": { 2024-09-10T09:27:26.151 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-10T09:27:26.151 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-10T09:27:26.151 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-10T09:27:26.151 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout }, 2024-09-10T09:27:26.151 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-10T09:27:26.151 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-10T09:27:26.151 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout 0 2024-09-10T09:27:26.151 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout ], 2024-09-10T09:27:26.151 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-10T09:27:26.151 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "a" 2024-09-10T09:27:26.152 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout ], 2024-09-10T09:27:26.152 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "quorum_age": 6, 2024-09-10T09:27:26.152 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-10T09:27:26.152 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T09:27:26.152 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-10T09:27:26.152 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-10T09:27:26.152 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout }, 2024-09-10T09:27:26.152 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-10T09:27:26.152 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T09:27:26.152 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-10T09:27:26.152 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-10T09:27:26.152 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-10T09:27:26.152 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-10T09:27:26.152 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-10T09:27:26.153 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-10T09:27:26.153 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout }, 2024-09-10T09:27:26.153 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-10T09:27:26.153 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-10T09:27:26.153 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-10T09:27:26.153 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-10T09:27:26.153 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-10T09:27:26.153 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-10T09:27:26.153 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-10T09:27:26.153 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-10T09:27:26.153 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-10T09:27:26.153 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout }, 2024-09-10T09:27:26.154 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-10T09:27:26.154 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T09:27:26.154 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "btime": "2024-09-10T09:27:13:298289+0000", 2024-09-10T09:27:26.154 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-10T09:27:26.154 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-10T09:27:26.154 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout }, 2024-09-10T09:27:26.154 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-10T09:27:26.154 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "available": false, 2024-09-10T09:27:26.154 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-10T09:27:26.154 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-10T09:27:26.154 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-10T09:27:26.154 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-10T09:27:26.154 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "restful" 2024-09-10T09:27:26.154 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout ], 2024-09-10T09:27:26.155 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-10T09:27:26.155 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout }, 2024-09-10T09:27:26.155 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-10T09:27:26.155 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T09:27:26.155 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "modified": "2024-09-10T09:27:13.299038+0000", 2024-09-10T09:27:26.155 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-10T09:27:26.155 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout }, 2024-09-10T09:27:26.155 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-10T09:27:26.155 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout } 2024-09-10T09:27:26.155 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:25.338+0000 7f6594400640 1 Processor -- start 2024-09-10T09:27:26.155 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:25.338+0000 7f6594400640 1 -- start start 2024-09-10T09:27:26.155 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:25.339+0000 7f6594400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f658c109920 0x7f658c109d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:27:26.155 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:25.339+0000 7f6594400640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f658c10a310 con 0x7f658c109920 2024-09-10T09:27:26.156 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:25.339+0000 7f6591a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f658c109920 0x7f658c109d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:27:26.156 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:25.339+0000 7f6591a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f658c109920 0x7f658c109d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51304/0 (socket says 172.21.15.5:51304) 2024-09-10T09:27:26.156 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:25.339+0000 7f6591a00640 1 -- 172.21.15.5:0/3485745099 learned_addr learned my addr 172.21.15.5:0/3485745099 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:27:26.156 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:25.340+0000 7f6591a00640 1 -- 172.21.15.5:0/3485745099 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f658c10aa10 con 0x7f658c109920 2024-09-10T09:27:26.156 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:25.341+0000 7f6591a00640 1 --2- 172.21.15.5:0/3485745099 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f658c109920 0x7f658c109d40 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f6580009820 tx=0x7f658002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29ac5ee9c0659a0a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:27:26.156 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:25.341+0000 7f658be00640 1 -- 172.21.15.5:0/3485745099 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6580037070 con 0x7f658c109920 2024-09-10T09:27:26.156 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:25.341+0000 7f658be00640 1 -- 172.21.15.5:0/3485745099 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f6580032900 con 0x7f658c109920 2024-09-10T09:27:26.156 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:25.341+0000 7f658be00640 1 -- 172.21.15.5:0/3485745099 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6580030850 con 0x7f658c109920 2024-09-10T09:27:26.156 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:25.342+0000 7f6594400640 1 -- 172.21.15.5:0/3485745099 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f658c109920 msgr2=0x7f658c109d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:27:26.156 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:25.342+0000 7f6594400640 1 --2- 172.21.15.5:0/3485745099 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f658c109920 0x7f658c109d40 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f6580009820 tx=0x7f658002a270 comp rx=0 tx=0).stop 2024-09-10T09:27:26.157 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:25.342+0000 7f6594400640 1 -- 172.21.15.5:0/3485745099 shutdown_connections 2024-09-10T09:27:26.157 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:25.342+0000 7f6594400640 1 --2- 172.21.15.5:0/3485745099 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f658c109920 0x7f658c109d40 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:27:26.157 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:25.342+0000 7f6594400640 1 -- 172.21.15.5:0/3485745099 >> 172.21.15.5:0/3485745099 conn(0x7f658c07cf40 msgr2=0x7f658c07d350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:27:26.157 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:25.342+0000 7f6594400640 1 -- 172.21.15.5:0/3485745099 shutdown_connections 2024-09-10T09:27:26.157 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:25.342+0000 7f6594400640 1 -- 172.21.15.5:0/3485745099 wait complete. 2024-09-10T09:27:26.157 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:25.343+0000 7f6594400640 1 Processor -- start 2024-09-10T09:27:26.157 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:25.344+0000 7f6594400640 1 -- start start 2024-09-10T09:27:26.157 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:25.344+0000 7f6594400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f658c109920 0x7f658c19d830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:27:26.157 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:25.344+0000 7f6594400640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f658c10ddc0 con 0x7f658c109920 2024-09-10T09:27:26.157 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:25.344+0000 7f6591a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f658c109920 0x7f658c19d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:27:26.157 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:25.344+0000 7f6591a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f658c109920 0x7f658c19d830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51320/0 (socket says 172.21.15.5:51320) 2024-09-10T09:27:26.157 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:25.344+0000 7f6591a00640 1 -- 172.21.15.5:0/2808896701 learned_addr learned my addr 172.21.15.5:0/2808896701 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:27:26.158 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:25.345+0000 7f6591a00640 1 -- 172.21.15.5:0/2808896701 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f658c19dd70 con 0x7f658c109920 2024-09-10T09:27:26.158 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:25.345+0000 7f6591a00640 1 --2- 172.21.15.5:0/2808896701 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f658c109920 0x7f658c19d830 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f65800097f0 tx=0x7f6580030d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:27:26.158 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:25.345+0000 7f658a000640 1 -- 172.21.15.5:0/2808896701 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6580037040 con 0x7f658c109920 2024-09-10T09:27:26.158 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:25.345+0000 7f6594400640 1 -- 172.21.15.5:0/2808896701 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f658c19e000 con 0x7f658c109920 2024-09-10T09:27:26.158 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:25.345+0000 7f658a000640 1 -- 172.21.15.5:0/2808896701 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f6580040070 con 0x7f658c109920 2024-09-10T09:27:26.158 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:25.345+0000 7f658a000640 1 -- 172.21.15.5:0/2808896701 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f658003a390 con 0x7f658c109920 2024-09-10T09:27:26.158 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:25.345+0000 7f6594400640 1 -- 172.21.15.5:0/2808896701 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f658c1a0cf0 con 0x7f658c109920 2024-09-10T09:27:26.158 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:25.346+0000 7f658a000640 1 -- 172.21.15.5:0/2808896701 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f658003a630 con 0x7f658c109920 2024-09-10T09:27:26.158 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:25.346+0000 7f658a000640 1 -- 172.21.15.5:0/2808896701 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f65800459a0 con 0x7f658c109920 2024-09-10T09:27:26.158 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:25.347+0000 7f6594400640 1 -- 172.21.15.5:0/2808896701 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f658c109dc0 con 0x7f658c109920 2024-09-10T09:27:26.158 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:25.349+0000 7f658a000640 1 -- 172.21.15.5:0/2808896701 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f6580045460 con 0x7f658c109920 2024-09-10T09:27:26.158 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:25.415+0000 7f6594400640 1 -- 172.21.15.5:0/2808896701 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f658c1a0ec0 con 0x7f658c109920 2024-09-10T09:27:26.158 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:25.416+0000 7f658a000640 1 -- 172.21.15.5:0/2808896701 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1291 (secure 0 0 0) 0x7f658003b430 con 0x7f658c109920 2024-09-10T09:27:26.159 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:25.417+0000 7f6594400640 1 -- 172.21.15.5:0/2808896701 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f658c109920 msgr2=0x7f658c19d830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:27:26.159 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:25.417+0000 7f6594400640 1 --2- 172.21.15.5:0/2808896701 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f658c109920 0x7f658c19d830 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f65800097f0 tx=0x7f6580030d80 comp rx=0 tx=0).stop 2024-09-10T09:27:26.159 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:25.417+0000 7f6594400640 1 -- 172.21.15.5:0/2808896701 shutdown_connections 2024-09-10T09:27:26.159 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:25.417+0000 7f6594400640 1 --2- 172.21.15.5:0/2808896701 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f658c109920 0x7f658c19d830 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:27:26.159 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:25.417+0000 7f6594400640 1 -- 172.21.15.5:0/2808896701 >> 172.21.15.5:0/2808896701 conn(0x7f658c07cf40 msgr2=0x7f658c107540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:27:26.159 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:25.417+0000 7f6594400640 1 -- 172.21.15.5:0/2808896701 shutdown_connections 2024-09-10T09:27:26.159 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:25.417+0000 7f6594400640 1 -- 172.21.15.5:0/2808896701 wait complete. 2024-09-10T09:27:26.159 INFO:teuthology.orchestra.run.smithi005.stdout:mgr not available, waiting (2/15)... 2024-09-10T09:27:26.378 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 10 09:27:26 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mgr-x[30575]: 2024-09-10T09:27:26.296+0000 7fcc18a2d040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-10T09:27:26.635 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 10 09:27:26 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mgr-x[30575]: 2024-09-10T09:27:26.377+0000 7fcc18a2d040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-10T09:27:26.635 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 10 09:27:26 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mgr-x[30575]: 2024-09-10T09:27:26.459+0000 7fcc18a2d040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-10T09:27:26.635 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 10 09:27:26 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mgr-x[30575]: 2024-09-10T09:27:26.560+0000 7fcc18a2d040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-10T09:27:26.635 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 10 09:27:26 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mgr-x[30575]: 2024-09-10T09:27:26.634+0000 7fcc18a2d040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-10T09:27:27.056 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 10 09:27:26 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mgr-x[30575]: 2024-09-10T09:27:26.707+0000 7fcc18a2d040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-10T09:27:28.056 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 10 09:27:27 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mgr-x[30575]: 2024-09-10T09:27:27.666+0000 7fcc18a2d040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-10T09:27:28.057 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:27 smithi005 ceph-mon[30350]: Activating manager daemon x 2024-09-10T09:27:28.057 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:27 smithi005 ceph-mon[30350]: mgrmap e2: x(active, starting, since 0.00407838s) 2024-09-10T09:27:28.057 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:27 smithi005 ceph-mon[30350]: from='mgr.14100 172.21.15.5:0/1794931009' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-10T09:27:28.057 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:27 smithi005 ceph-mon[30350]: from='mgr.14100 172.21.15.5:0/1794931009' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-10T09:27:28.057 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:27 smithi005 ceph-mon[30350]: from='mgr.14100 172.21.15.5:0/1794931009' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-10T09:27:28.058 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:27 smithi005 ceph-mon[30350]: from='mgr.14100 172.21.15.5:0/1794931009' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T09:27:28.058 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:27 smithi005 ceph-mon[30350]: from='mgr.14100 172.21.15.5:0/1794931009' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-10T09:27:28.058 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:27 smithi005 ceph-mon[30350]: Manager daemon x is now available 2024-09-10T09:27:28.058 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:27 smithi005 ceph-mon[30350]: from='mgr.14100 172.21.15.5:0/1794931009' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-10T09:27:28.058 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:27 smithi005 ceph-mon[30350]: from='mgr.14100 172.21.15.5:0/1794931009' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-10T09:27:28.058 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:27 smithi005 ceph-mon[30350]: from='mgr.14100 172.21.15.5:0/1794931009' entity='mgr.x' 2024-09-10T09:27:28.059 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:27 smithi005 ceph-mon[30350]: from='mgr.14100 172.21.15.5:0/1794931009' entity='mgr.x' 2024-09-10T09:27:28.059 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:27 smithi005 ceph-mon[30350]: from='mgr.14100 172.21.15.5:0/1794931009' entity='mgr.x' 2024-09-10T09:27:29.290 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout 2024-09-10T09:27:29.290 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout { 2024-09-10T09:27:29.291 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "fsid": "bade4b50-6f56-11ef-bcea-c7b262605968", 2024-09-10T09:27:29.291 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "health": { 2024-09-10T09:27:29.291 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-10T09:27:29.291 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-10T09:27:29.291 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-10T09:27:29.291 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout }, 2024-09-10T09:27:29.291 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-10T09:27:29.291 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-10T09:27:29.291 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout 0 2024-09-10T09:27:29.291 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout ], 2024-09-10T09:27:29.292 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-10T09:27:29.292 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "a" 2024-09-10T09:27:29.292 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout ], 2024-09-10T09:27:29.292 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "quorum_age": 9, 2024-09-10T09:27:29.292 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-10T09:27:29.292 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T09:27:29.292 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-10T09:27:29.292 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-10T09:27:29.292 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout }, 2024-09-10T09:27:29.292 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-10T09:27:29.292 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T09:27:29.292 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-10T09:27:29.292 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-10T09:27:29.293 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-10T09:27:29.302 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-10T09:27:29.302 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-10T09:27:29.302 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-10T09:27:29.302 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout }, 2024-09-10T09:27:29.302 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-10T09:27:29.302 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-10T09:27:29.302 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-10T09:27:29.302 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-10T09:27:29.302 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-10T09:27:29.302 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-10T09:27:29.302 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-10T09:27:29.303 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-10T09:27:29.303 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-10T09:27:29.303 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout }, 2024-09-10T09:27:29.303 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-10T09:27:29.303 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T09:27:29.303 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "btime": "2024-09-10T09:27:13:298289+0000", 2024-09-10T09:27:29.303 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-10T09:27:29.303 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-10T09:27:29.303 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout }, 2024-09-10T09:27:29.303 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-10T09:27:29.303 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-10T09:27:29.303 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-10T09:27:29.304 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-10T09:27:29.304 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-10T09:27:29.304 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-10T09:27:29.304 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "restful" 2024-09-10T09:27:29.304 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout ], 2024-09-10T09:27:29.304 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-10T09:27:29.304 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout }, 2024-09-10T09:27:29.304 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-10T09:27:29.304 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T09:27:29.304 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "modified": "2024-09-10T09:27:13.299038+0000", 2024-09-10T09:27:29.304 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-10T09:27:29.304 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout }, 2024-09-10T09:27:29.305 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-10T09:27:29.305 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout } 2024-09-10T09:27:29.305 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:28.609+0000 7f11ada00640 1 Processor -- start 2024-09-10T09:27:29.305 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:28.609+0000 7f11ada00640 1 -- start start 2024-09-10T09:27:29.305 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:28.609+0000 7f11ada00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f11a807de00 0x7f11a807c300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:27:29.305 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:28.609+0000 7f11ada00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f11a807c840 con 0x7f11a807de00 2024-09-10T09:27:29.305 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:28.610+0000 7f11a6a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f11a807de00 0x7f11a807c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:27:29.305 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:28.610+0000 7f11a6a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f11a807de00 0x7f11a807c300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51392/0 (socket says 172.21.15.5:51392) 2024-09-10T09:27:29.305 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:28.610+0000 7f11a6a00640 1 -- 172.21.15.5:0/419235004 learned_addr learned my addr 172.21.15.5:0/419235004 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:27:29.305 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:28.610+0000 7f11a6a00640 1 -- 172.21.15.5:0/419235004 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11a807c9c0 con 0x7f11a807de00 2024-09-10T09:27:29.305 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:28.611+0000 7f11a6a00640 1 --2- 172.21.15.5:0/419235004 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f11a807de00 0x7f11a807c300 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f1198009820 tx=0x7f119802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ac88d3cfe3fb9d1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:27:29.305 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:28.611+0000 7f11a5600640 1 -- 172.21.15.5:0/419235004 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1198037070 con 0x7f11a807de00 2024-09-10T09:27:29.305 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:28.611+0000 7f11a5600640 1 -- 172.21.15.5:0/419235004 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f1198032900 con 0x7f11a807de00 2024-09-10T09:27:29.306 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:28.611+0000 7f11a5600640 1 -- 172.21.15.5:0/419235004 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1198030850 con 0x7f11a807de00 2024-09-10T09:27:29.306 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:28.612+0000 7f11ada00640 1 -- 172.21.15.5:0/419235004 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f11a807de00 msgr2=0x7f11a807c300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:27:29.306 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:28.612+0000 7f11ada00640 1 --2- 172.21.15.5:0/419235004 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f11a807de00 0x7f11a807c300 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f1198009820 tx=0x7f119802a270 comp rx=0 tx=0).stop 2024-09-10T09:27:29.306 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:28.612+0000 7f11ada00640 1 -- 172.21.15.5:0/419235004 shutdown_connections 2024-09-10T09:27:29.306 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:28.612+0000 7f11ada00640 1 --2- 172.21.15.5:0/419235004 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f11a807de00 0x7f11a807c300 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:27:29.306 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:28.612+0000 7f11ada00640 1 -- 172.21.15.5:0/419235004 >> 172.21.15.5:0/419235004 conn(0x7f11a8102e50 msgr2=0x7f11a8105270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:27:29.306 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:28.612+0000 7f11ada00640 1 -- 172.21.15.5:0/419235004 shutdown_connections 2024-09-10T09:27:29.306 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:28.612+0000 7f11ada00640 1 -- 172.21.15.5:0/419235004 wait complete. 2024-09-10T09:27:29.306 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:28.613+0000 7f11ada00640 1 Processor -- start 2024-09-10T09:27:29.306 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:28.613+0000 7f11ada00640 1 -- start start 2024-09-10T09:27:29.306 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:28.614+0000 7f11ada00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f11a807de00 0x7f11a819d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:27:29.307 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:28.614+0000 7f11ada00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f11a810dc50 con 0x7f11a807de00 2024-09-10T09:27:29.307 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:28.614+0000 7f11a6a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f11a807de00 0x7f11a819d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:27:29.307 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:28.614+0000 7f11a6a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f11a807de00 0x7f11a819d870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51394/0 (socket says 172.21.15.5:51394) 2024-09-10T09:27:29.307 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:28.614+0000 7f11a6a00640 1 -- 172.21.15.5:0/1014488775 learned_addr learned my addr 172.21.15.5:0/1014488775 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:27:29.307 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:28.614+0000 7f11a6a00640 1 -- 172.21.15.5:0/1014488775 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11a819ddb0 con 0x7f11a807de00 2024-09-10T09:27:29.307 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:28.615+0000 7f11a6a00640 1 --2- 172.21.15.5:0/1014488775 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f11a807de00 0x7f11a819d870 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f11980097f0 tx=0x7f1198030ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:27:29.307 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:28.615+0000 7f119f400640 1 -- 172.21.15.5:0/1014488775 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1198042070 con 0x7f11a807de00 2024-09-10T09:27:29.307 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:28.615+0000 7f119f400640 1 -- 172.21.15.5:0/1014488775 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f1198032ca0 con 0x7f11a807de00 2024-09-10T09:27:29.307 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:28.615+0000 7f119f400640 1 -- 172.21.15.5:0/1014488775 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1198037040 con 0x7f11a807de00 2024-09-10T09:27:29.307 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:28.615+0000 7f11ada00640 1 -- 172.21.15.5:0/1014488775 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11a819e040 con 0x7f11a807de00 2024-09-10T09:27:29.307 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:28.615+0000 7f11ada00640 1 -- 172.21.15.5:0/1014488775 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11a81a0d30 con 0x7f11a807de00 2024-09-10T09:27:29.307 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:28.616+0000 7f119f400640 1 -- 172.21.15.5:0/1014488775 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 2) ==== 46043+0+0 (secure 0 0 0) 0x7f1198045070 con 0x7f11a807de00 2024-09-10T09:27:29.308 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:28.616+0000 7f11ada00640 1 -- 172.21.15.5:0/1014488775 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f11a807c380 con 0x7f11a807de00 2024-09-10T09:27:29.308 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:28.617+0000 7f119f400640 1 -- 172.21.15.5:0/1014488775 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f119806bb10 con 0x7f11a807de00 2024-09-10T09:27:29.308 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:28.619+0000 7f119f400640 1 -- 172.21.15.5:0/1014488775 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f1198045360 con 0x7f11a807de00 2024-09-10T09:27:29.308 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:28.673+0000 7f119f400640 1 -- 172.21.15.5:0/1014488775 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mgrmap(e 3) ==== 46170+0+0 (secure 0 0 0) 0x7f119803d430 con 0x7f11a807de00 2024-09-10T09:27:29.308 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:28.673+0000 7f119f400640 1 --2- 172.21.15.5:0/1014488775 >> [v2:172.21.15.5:6800/14425913,v1:172.21.15.5:6801/14425913] conn(0x7f118c039ac0 0x7f118c03beb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:27:29.308 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:28.674+0000 7f11a6000640 1 --2- 172.21.15.5:0/1014488775 >> [v2:172.21.15.5:6800/14425913,v1:172.21.15.5:6801/14425913] conn(0x7f118c039ac0 0x7f118c03beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:27:29.308 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:28.675+0000 7f11a6000640 1 --2- 172.21.15.5:0/1014488775 >> [v2:172.21.15.5:6800/14425913,v1:172.21.15.5:6801/14425913] conn(0x7f118c039ac0 0x7f118c03beb0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f11940098c0 tx=0x7f1194006db0 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:27:29.308 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:28.685+0000 7f11ada00640 1 -- 172.21.15.5:0/1014488775 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f11a81a1340 con 0x7f11a807de00 2024-09-10T09:27:29.308 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:28.686+0000 7f119f400640 1 -- 172.21.15.5:0/1014488775 <== mon.0 v2:172.21.15.5:3300/0 8 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1290 (secure 0 0 0) 0x7f1198042210 con 0x7f11a807de00 2024-09-10T09:27:29.309 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:28.687+0000 7f11ada00640 1 -- 172.21.15.5:0/1014488775 >> [v2:172.21.15.5:6800/14425913,v1:172.21.15.5:6801/14425913] conn(0x7f118c039ac0 msgr2=0x7f118c03beb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:27:29.309 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:28.687+0000 7f11ada00640 1 --2- 172.21.15.5:0/1014488775 >> [v2:172.21.15.5:6800/14425913,v1:172.21.15.5:6801/14425913] conn(0x7f118c039ac0 0x7f118c03beb0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f11940098c0 tx=0x7f1194006db0 comp rx=0 tx=0).stop 2024-09-10T09:27:29.309 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:28.687+0000 7f11ada00640 1 -- 172.21.15.5:0/1014488775 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f11a807de00 msgr2=0x7f11a819d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:27:29.309 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:28.687+0000 7f11ada00640 1 --2- 172.21.15.5:0/1014488775 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f11a807de00 0x7f11a819d870 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f11980097f0 tx=0x7f1198030ec0 comp rx=0 tx=0).stop 2024-09-10T09:27:29.309 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:28.687+0000 7f11ada00640 1 -- 172.21.15.5:0/1014488775 shutdown_connections 2024-09-10T09:27:29.309 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:28.687+0000 7f11ada00640 1 --2- 172.21.15.5:0/1014488775 >> [v2:172.21.15.5:6800/14425913,v1:172.21.15.5:6801/14425913] conn(0x7f118c039ac0 0x7f118c03beb0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:27:29.309 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:28.687+0000 7f11ada00640 1 --2- 172.21.15.5:0/1014488775 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f11a807de00 0x7f11a819d870 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:27:29.309 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:28.687+0000 7f11ada00640 1 -- 172.21.15.5:0/1014488775 >> 172.21.15.5:0/1014488775 conn(0x7f11a8102e50 msgr2=0x7f11a8103a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:27:29.309 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:28.687+0000 7f11ada00640 1 -- 172.21.15.5:0/1014488775 shutdown_connections 2024-09-10T09:27:29.309 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:28.687+0000 7f11ada00640 1 -- 172.21.15.5:0/1014488775 wait complete. 2024-09-10T09:27:29.309 INFO:teuthology.orchestra.run.smithi005.stdout:mgr is available 2024-09-10T09:27:29.800 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:29 smithi005 ceph-mon[30350]: mgrmap e3: x(active, since 1.00825s) 2024-09-10T09:27:29.800 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:29 smithi005 ceph-mon[30350]: from='client.? 172.21.15.5:0/1014488775' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-10T09:27:30.679 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout 2024-09-10T09:27:30.679 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout [global] 2024-09-10T09:27:30.679 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout fsid = bade4b50-6f56-11ef-bcea-c7b262605968 2024-09-10T09:27:30.679 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.5:3300,v1:172.21.15.5:6789] 2024-09-10T09:27:30.680 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-09-10T09:27:30.680 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-09-10T09:27:30.680 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-09-10T09:27:30.680 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-09-10T09:27:30.680 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout 2024-09-10T09:27:30.680 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout [mgr] 2024-09-10T09:27:30.680 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-09-10T09:27:30.680 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout 2024-09-10T09:27:30.680 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout [osd] 2024-09-10T09:27:30.680 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-09-10T09:27:30.680 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-09-10T09:27:30.680 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-09-10T09:27:30.680 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-09-10T09:27:30.680 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-09-10T09:27:30.681 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:29.798+0000 7f5ec4200640 1 Processor -- start 2024-09-10T09:27:30.681 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:29.798+0000 7f5ec4200640 1 -- start start 2024-09-10T09:27:30.681 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:29.799+0000 7f5ec4200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5ebc108cf0 0x7f5ebc1090f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:27:30.681 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:29.799+0000 7f5ec4200640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f5ebc107230 con 0x7f5ebc108cf0 2024-09-10T09:27:30.681 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:29.799+0000 7f5ec1800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5ebc108cf0 0x7f5ebc1090f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:27:30.681 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:29.799+0000 7f5ec1800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5ebc108cf0 0x7f5ebc1090f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51400/0 (socket says 172.21.15.5:51400) 2024-09-10T09:27:30.681 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:29.799+0000 7f5ec1800640 1 -- 172.21.15.5:0/2011765729 learned_addr learned my addr 172.21.15.5:0/2011765729 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:27:30.681 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:29.799+0000 7f5ec1800640 1 -- 172.21.15.5:0/2011765729 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ebc1073b0 con 0x7f5ebc108cf0 2024-09-10T09:27:30.681 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:29.800+0000 7f5ec1800640 1 --2- 172.21.15.5:0/2011765729 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5ebc108cf0 0x7f5ebc1090f0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f5eb0009820 tx=0x7f5eb002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4161e836005242 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:27:30.681 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:29.801+0000 7f5ebbe00640 1 -- 172.21.15.5:0/2011765729 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5eb0037070 con 0x7f5ebc108cf0 2024-09-10T09:27:30.681 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:29.801+0000 7f5ebbe00640 1 -- 172.21.15.5:0/2011765729 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f5eb0032900 con 0x7f5ebc108cf0 2024-09-10T09:27:30.681 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:29.801+0000 7f5ebbe00640 1 -- 172.21.15.5:0/2011765729 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5eb0030850 con 0x7f5ebc108cf0 2024-09-10T09:27:30.681 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:29.801+0000 7f5ec4200640 1 -- 172.21.15.5:0/2011765729 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5ebc108cf0 msgr2=0x7f5ebc1090f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:27:30.681 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:29.801+0000 7f5ec4200640 1 --2- 172.21.15.5:0/2011765729 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5ebc108cf0 0x7f5ebc1090f0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f5eb0009820 tx=0x7f5eb002a270 comp rx=0 tx=0).stop 2024-09-10T09:27:30.681 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:29.801+0000 7f5ec4200640 1 -- 172.21.15.5:0/2011765729 shutdown_connections 2024-09-10T09:27:30.682 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:29.801+0000 7f5ec4200640 1 --2- 172.21.15.5:0/2011765729 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5ebc108cf0 0x7f5ebc1090f0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:27:30.682 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:29.801+0000 7f5ec4200640 1 -- 172.21.15.5:0/2011765729 >> 172.21.15.5:0/2011765729 conn(0x7f5ebc102dc0 msgr2=0x7f5ebc1051e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:27:30.682 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:29.802+0000 7f5ec4200640 1 -- 172.21.15.5:0/2011765729 shutdown_connections 2024-09-10T09:27:30.682 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:29.802+0000 7f5ec4200640 1 -- 172.21.15.5:0/2011765729 wait complete. 2024-09-10T09:27:30.682 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:29.802+0000 7f5ec4200640 1 Processor -- start 2024-09-10T09:27:30.682 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:29.803+0000 7f5ec4200640 1 -- start start 2024-09-10T09:27:30.682 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:29.803+0000 7f5ec4200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5ebc108cf0 0x7f5ebc07db70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:27:30.682 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:29.803+0000 7f5ec4200640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f5ebc10bbc0 con 0x7f5ebc108cf0 2024-09-10T09:27:30.682 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:29.803+0000 7f5ec1800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5ebc108cf0 0x7f5ebc07db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:27:30.682 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:29.803+0000 7f5ec1800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5ebc108cf0 0x7f5ebc07db70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51412/0 (socket says 172.21.15.5:51412) 2024-09-10T09:27:30.682 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:29.803+0000 7f5ec1800640 1 -- 172.21.15.5:0/2713337841 learned_addr learned my addr 172.21.15.5:0/2713337841 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:27:30.682 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:29.803+0000 7f5ec1800640 1 -- 172.21.15.5:0/2713337841 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ebc07e0b0 con 0x7f5ebc108cf0 2024-09-10T09:27:30.682 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:29.804+0000 7f5ec1800640 1 --2- 172.21.15.5:0/2713337841 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5ebc108cf0 0x7f5ebc07db70 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f5eb00097f0 tx=0x7f5eb0030d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:27:30.685 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:29.804+0000 7f5eba000640 1 -- 172.21.15.5:0/2713337841 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5eb0037040 con 0x7f5ebc108cf0 2024-09-10T09:27:30.685 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:29.804+0000 7f5eba000640 1 -- 172.21.15.5:0/2713337841 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f5eb0040070 con 0x7f5ebc108cf0 2024-09-10T09:27:30.685 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:29.804+0000 7f5ec4200640 1 -- 172.21.15.5:0/2713337841 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ebc07c2a0 con 0x7f5ebc108cf0 2024-09-10T09:27:30.685 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:29.804+0000 7f5eba000640 1 -- 172.21.15.5:0/2713337841 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5eb003a390 con 0x7f5ebc108cf0 2024-09-10T09:27:30.685 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:29.804+0000 7f5ec4200640 1 -- 172.21.15.5:0/2713337841 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ebc07c780 con 0x7f5ebc108cf0 2024-09-10T09:27:30.685 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:29.805+0000 7f5eba000640 1 -- 172.21.15.5:0/2713337841 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 4) ==== 46276+0+0 (secure 0 0 0) 0x7f5eb003b430 con 0x7f5ebc108cf0 2024-09-10T09:27:30.685 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:29.805+0000 7f5ec4200640 1 -- 172.21.15.5:0/2713337841 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ebc109170 con 0x7f5ebc108cf0 2024-09-10T09:27:30.685 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:29.806+0000 7f5eba000640 1 --2- 172.21.15.5:0/2713337841 >> [v2:172.21.15.5:6800/14425913,v1:172.21.15.5:6801/14425913] conn(0x7f5ea4038f70 0x7f5ea403b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:27:30.685 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:29.806+0000 7f5eba000640 1 -- 172.21.15.5:0/2713337841 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f5eb006c090 con 0x7f5ebc108cf0 2024-09-10T09:27:30.685 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:29.806+0000 7f5ec0e00640 1 --2- 172.21.15.5:0/2713337841 >> [v2:172.21.15.5:6800/14425913,v1:172.21.15.5:6801/14425913] conn(0x7f5ea4038f70 0x7f5ea403b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:27:30.685 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:29.807+0000 7f5ec0e00640 1 --2- 172.21.15.5:0/2713337841 >> [v2:172.21.15.5:6800/14425913,v1:172.21.15.5:6801/14425913] conn(0x7f5ea4038f70 0x7f5ea403b430 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f5ea80098c0 tx=0x7f5ea8006db0 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:27:30.686 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:29.812+0000 7f5eba000640 1 -- 172.21.15.5:0/2713337841 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f5eb00371e0 con 0x7f5ebc108cf0 2024-09-10T09:27:30.686 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:29.998+0000 7f5ec4200640 1 -- 172.21.15.5:0/2713337841 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f5ebc07ce90 con 0x7f5ebc108cf0 2024-09-10T09:27:30.686 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:29.999+0000 7f5eba000640 1 -- 172.21.15.5:0/2713337841 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) ==== 70+0+427 (secure 0 0 0) 0x7f5eb006bdf0 con 0x7f5ebc108cf0 2024-09-10T09:27:30.686 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:30.001+0000 7f5ec4200640 1 -- 172.21.15.5:0/2713337841 >> [v2:172.21.15.5:6800/14425913,v1:172.21.15.5:6801/14425913] conn(0x7f5ea4038f70 msgr2=0x7f5ea403b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:27:30.686 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:30.001+0000 7f5ec4200640 1 --2- 172.21.15.5:0/2713337841 >> [v2:172.21.15.5:6800/14425913,v1:172.21.15.5:6801/14425913] conn(0x7f5ea4038f70 0x7f5ea403b430 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f5ea80098c0 tx=0x7f5ea8006db0 comp rx=0 tx=0).stop 2024-09-10T09:27:30.686 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:30.001+0000 7f5ec4200640 1 -- 172.21.15.5:0/2713337841 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5ebc108cf0 msgr2=0x7f5ebc07db70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:27:30.686 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:30.001+0000 7f5ec4200640 1 --2- 172.21.15.5:0/2713337841 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5ebc108cf0 0x7f5ebc07db70 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f5eb00097f0 tx=0x7f5eb0030d80 comp rx=0 tx=0).stop 2024-09-10T09:27:30.686 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:30.002+0000 7f5ec4200640 1 -- 172.21.15.5:0/2713337841 shutdown_connections 2024-09-10T09:27:30.686 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:30.002+0000 7f5ec4200640 1 --2- 172.21.15.5:0/2713337841 >> [v2:172.21.15.5:6800/14425913,v1:172.21.15.5:6801/14425913] conn(0x7f5ea4038f70 0x7f5ea403b430 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:27:30.686 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:30.002+0000 7f5ec4200640 1 --2- 172.21.15.5:0/2713337841 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5ebc108cf0 0x7f5ebc07db70 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:27:30.686 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:30.002+0000 7f5ec4200640 1 -- 172.21.15.5:0/2713337841 >> 172.21.15.5:0/2713337841 conn(0x7f5ebc102dc0 msgr2=0x7f5ebc103920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:27:30.686 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:30.002+0000 7f5ec4200640 1 -- 172.21.15.5:0/2713337841 shutdown_connections 2024-09-10T09:27:30.686 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:30.002+0000 7f5ec4200640 1 -- 172.21.15.5:0/2713337841 wait complete. 2024-09-10T09:27:30.686 INFO:teuthology.orchestra.run.smithi005.stdout:Enabling cephadm module... 2024-09-10T09:27:31.016 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:30 smithi005 ceph-mon[30350]: mgrmap e4: x(active, since 2s) 2024-09-10T09:27:31.016 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:30 smithi005 ceph-mon[30350]: from='client.? 172.21.15.5:0/2713337841' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-09-10T09:27:31.777 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:31 smithi005 ceph-mon[30350]: from='client.? 172.21.15.5:0/2417613891' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-09-10T09:27:32.041 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 10 09:27:31 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mgr-x[30575]: ignoring --setuser ceph since I am not root 2024-09-10T09:27:32.041 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 10 09:27:31 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mgr-x[30575]: ignoring --setgroup ceph since I am not root 2024-09-10T09:27:32.041 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 10 09:27:31 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mgr-x[30575]: 2024-09-10T09:27:31.903+0000 7fbc94236040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-10T09:27:32.041 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 10 09:27:31 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mgr-x[30575]: 2024-09-10T09:27:31.975+0000 7fbc94236040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-10T09:27:32.306 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 10 09:27:32 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mgr-x[30575]: 2024-09-10T09:27:32.167+0000 7fbc94236040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-10T09:27:32.306 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 10 09:27:32 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mgr-x[30575]: 2024-09-10T09:27:32.251+0000 7fbc94236040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-10T09:27:32.323 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:31.242+0000 7fc69ec00640 1 Processor -- start 2024-09-10T09:27:32.323 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:31.242+0000 7fc69ec00640 1 -- start start 2024-09-10T09:27:32.323 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:31.242+0000 7fc69ec00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc69807de00 0x7fc69807c300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:27:32.323 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:31.242+0000 7fc69ec00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc69807c840 con 0x7fc69807de00 2024-09-10T09:27:32.323 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:31.243+0000 7fc697e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc69807de00 0x7fc69807c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:27:32.323 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:31.243+0000 7fc697e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc69807de00 0x7fc69807c300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52278/0 (socket says 172.21.15.5:52278) 2024-09-10T09:27:32.323 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:31.243+0000 7fc697e00640 1 -- 172.21.15.5:0/1797151097 learned_addr learned my addr 172.21.15.5:0/1797151097 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:27:32.323 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:31.243+0000 7fc697e00640 1 -- 172.21.15.5:0/1797151097 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc69807c9c0 con 0x7fc69807de00 2024-09-10T09:27:32.323 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:31.244+0000 7fc697e00640 1 --2- 172.21.15.5:0/1797151097 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc69807de00 0x7fc69807c300 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fc688009820 tx=0x7fc68802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e43ca159801b2c9f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:27:32.324 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:31.244+0000 7fc697400640 1 -- 172.21.15.5:0/1797151097 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc688037070 con 0x7fc69807de00 2024-09-10T09:27:32.324 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:31.244+0000 7fc697400640 1 -- 172.21.15.5:0/1797151097 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fc688032900 con 0x7fc69807de00 2024-09-10T09:27:32.324 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:31.244+0000 7fc697400640 1 -- 172.21.15.5:0/1797151097 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc688030850 con 0x7fc69807de00 2024-09-10T09:27:32.324 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:31.245+0000 7fc69ec00640 1 -- 172.21.15.5:0/1797151097 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc69807de00 msgr2=0x7fc69807c300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:27:32.324 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:31.245+0000 7fc69ec00640 1 --2- 172.21.15.5:0/1797151097 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc69807de00 0x7fc69807c300 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fc688009820 tx=0x7fc68802a270 comp rx=0 tx=0).stop 2024-09-10T09:27:32.324 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:31.245+0000 7fc69ec00640 1 -- 172.21.15.5:0/1797151097 shutdown_connections 2024-09-10T09:27:32.324 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:31.245+0000 7fc69ec00640 1 --2- 172.21.15.5:0/1797151097 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc69807de00 0x7fc69807c300 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:27:32.324 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:31.245+0000 7fc69ec00640 1 -- 172.21.15.5:0/1797151097 >> 172.21.15.5:0/1797151097 conn(0x7fc698102e50 msgr2=0x7fc698105270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:27:32.324 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:31.245+0000 7fc69ec00640 1 -- 172.21.15.5:0/1797151097 shutdown_connections 2024-09-10T09:27:32.324 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:31.245+0000 7fc69ec00640 1 -- 172.21.15.5:0/1797151097 wait complete. 2024-09-10T09:27:32.324 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:31.246+0000 7fc69ec00640 1 Processor -- start 2024-09-10T09:27:32.324 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:31.246+0000 7fc69ec00640 1 -- start start 2024-09-10T09:27:32.325 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:31.247+0000 7fc69ec00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc69807de00 0x7fc69819d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:27:32.325 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:31.247+0000 7fc69ec00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc69810dc50 con 0x7fc69807de00 2024-09-10T09:27:32.325 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:31.247+0000 7fc697e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc69807de00 0x7fc69819d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:27:32.325 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:31.247+0000 7fc697e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc69807de00 0x7fc69819d870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52294/0 (socket says 172.21.15.5:52294) 2024-09-10T09:27:32.325 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:31.247+0000 7fc697e00640 1 -- 172.21.15.5:0/2417613891 learned_addr learned my addr 172.21.15.5:0/2417613891 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:27:32.325 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:31.247+0000 7fc697e00640 1 -- 172.21.15.5:0/2417613891 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc69819ddb0 con 0x7fc69807de00 2024-09-10T09:27:32.325 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:31.247+0000 7fc697e00640 1 --2- 172.21.15.5:0/2417613891 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc69807de00 0x7fc69819d870 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fc688031000 tx=0x7fc6880326e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:27:32.325 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:31.248+0000 7fc695600640 1 -- 172.21.15.5:0/2417613891 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc688042070 con 0x7fc69807de00 2024-09-10T09:27:32.325 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:31.248+0000 7fc695600640 1 -- 172.21.15.5:0/2417613891 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fc688032be0 con 0x7fc69807de00 2024-09-10T09:27:32.325 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:31.248+0000 7fc695600640 1 -- 172.21.15.5:0/2417613891 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc688037040 con 0x7fc69807de00 2024-09-10T09:27:32.325 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:31.248+0000 7fc69ec00640 1 -- 172.21.15.5:0/2417613891 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc69819e040 con 0x7fc69807de00 2024-09-10T09:27:32.325 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:31.248+0000 7fc69ec00640 1 -- 172.21.15.5:0/2417613891 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6981a0d30 con 0x7fc69807de00 2024-09-10T09:27:32.325 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:31.249+0000 7fc695600640 1 -- 172.21.15.5:0/2417613891 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 4) ==== 46276+0+0 (secure 0 0 0) 0x7fc688045070 con 0x7fc69807de00 2024-09-10T09:27:32.326 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:31.249+0000 7fc69ec00640 1 -- 172.21.15.5:0/2417613891 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc69807c380 con 0x7fc69807de00 2024-09-10T09:27:32.326 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:31.249+0000 7fc695600640 1 --2- 172.21.15.5:0/2417613891 >> [v2:172.21.15.5:6800/14425913,v1:172.21.15.5:6801/14425913] conn(0x7fc67c038f70 0x7fc67c03b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:27:32.326 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:31.249+0000 7fc695600640 1 -- 172.21.15.5:0/2417613891 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fc68806bf00 con 0x7fc69807de00 2024-09-10T09:27:32.326 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:31.250+0000 7fc68f400640 1 --2- 172.21.15.5:0/2417613891 >> [v2:172.21.15.5:6800/14425913,v1:172.21.15.5:6801/14425913] conn(0x7fc67c038f70 0x7fc67c03b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:27:32.326 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:31.250+0000 7fc68f400640 1 --2- 172.21.15.5:0/2417613891 >> [v2:172.21.15.5:6800/14425913,v1:172.21.15.5:6801/14425913] conn(0x7fc67c038f70 0x7fc67c03b430 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fc6800098c0 tx=0x7fc680006db0 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:27:32.326 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:31.254+0000 7fc695600640 1 -- 172.21.15.5:0/2417613891 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc68802f2a0 con 0x7fc69807de00 2024-09-10T09:27:32.326 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:31.487+0000 7fc69ec00640 1 -- 172.21.15.5:0/2417613891 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) -- 0x7fc6981a1340 con 0x7fc69807de00 2024-09-10T09:27:32.326 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:31.679+0000 7fc695600640 1 -- 172.21.15.5:0/2417613891 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) ==== 86+0+0 (secure 0 0 0) 0x7fc688032d90 con 0x7fc69807de00 2024-09-10T09:27:32.326 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:31.681+0000 7fc695600640 1 -- 172.21.15.5:0/2417613891 <== mon.0 v2:172.21.15.5:3300/0 8 ==== mgrmap(e 5) ==== 46287+0+0 (secure 0 0 0) 0x7fc68803d430 con 0x7fc69807de00 2024-09-10T09:27:32.326 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:31.682+0000 7fc69ec00640 1 -- 172.21.15.5:0/2417613891 >> [v2:172.21.15.5:6800/14425913,v1:172.21.15.5:6801/14425913] conn(0x7fc67c038f70 msgr2=0x7fc67c03b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:27:32.326 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:31.683+0000 7fc69ec00640 1 --2- 172.21.15.5:0/2417613891 >> [v2:172.21.15.5:6800/14425913,v1:172.21.15.5:6801/14425913] conn(0x7fc67c038f70 0x7fc67c03b430 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fc6800098c0 tx=0x7fc680006db0 comp rx=0 tx=0).stop 2024-09-10T09:27:32.326 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:31.683+0000 7fc69ec00640 1 -- 172.21.15.5:0/2417613891 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc69807de00 msgr2=0x7fc69819d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:27:32.327 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:31.683+0000 7fc69ec00640 1 --2- 172.21.15.5:0/2417613891 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc69807de00 0x7fc69819d870 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fc688031000 tx=0x7fc6880326e0 comp rx=0 tx=0).stop 2024-09-10T09:27:32.327 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:31.683+0000 7fc69ec00640 1 -- 172.21.15.5:0/2417613891 shutdown_connections 2024-09-10T09:27:32.327 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:31.683+0000 7fc69ec00640 1 --2- 172.21.15.5:0/2417613891 >> [v2:172.21.15.5:6800/14425913,v1:172.21.15.5:6801/14425913] conn(0x7fc67c038f70 0x7fc67c03b430 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:27:32.327 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:31.683+0000 7fc69ec00640 1 --2- 172.21.15.5:0/2417613891 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc69807de00 0x7fc69819d870 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:27:32.327 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:31.683+0000 7fc69ec00640 1 -- 172.21.15.5:0/2417613891 >> 172.21.15.5:0/2417613891 conn(0x7fc698102e50 msgr2=0x7fc698103a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:27:32.327 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:31.683+0000 7fc69ec00640 1 -- 172.21.15.5:0/2417613891 shutdown_connections 2024-09-10T09:27:32.327 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:31.683+0000 7fc69ec00640 1 -- 172.21.15.5:0/2417613891 wait complete. 2024-09-10T09:27:32.584 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 10 09:27:32 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mgr-x[30575]: 2024-09-10T09:27:32.443+0000 7fbc94236040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-10T09:27:32.848 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 10 09:27:32 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mgr-x[30575]: 2024-09-10T09:27:32.618+0000 7fbc94236040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-10T09:27:32.849 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:32 smithi005 ceph-mon[30350]: from='client.? 172.21.15.5:0/2417613891' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-09-10T09:27:32.849 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:32 smithi005 ceph-mon[30350]: mgrmap e5: x(active, since 4s) 2024-09-10T09:27:33.099 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 10 09:27:33 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mgr-x[30575]: 2024-09-10T09:27:33.004+0000 7fbc94236040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-10T09:27:33.099 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 10 09:27:33 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mgr-x[30575]: 2024-09-10T09:27:33.089+0000 7fbc94236040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-10T09:27:33.422 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 10 09:27:33 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mgr-x[30575]: 2024-09-10T09:27:33.321+0000 7fbc94236040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-10T09:27:33.728 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout { 2024-09-10T09:27:33.728 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-09-10T09:27:33.728 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-10T09:27:33.728 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-09-10T09:27:33.728 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-09-10T09:27:33.728 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout } 2024-09-10T09:27:33.728 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:32.847+0000 7f8f29200640 1 Processor -- start 2024-09-10T09:27:33.729 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:32.847+0000 7f8f29200640 1 -- start start 2024-09-10T09:27:33.729 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:32.847+0000 7f8f29200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8f24109970 0x7f8f24109d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:27:33.729 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:32.847+0000 7f8f29200640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f8f2410a340 con 0x7f8f24109970 2024-09-10T09:27:33.729 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:32.848+0000 7f8f23400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8f24109970 0x7f8f24109d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:27:33.729 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:32.848+0000 7f8f23400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8f24109970 0x7f8f24109d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52318/0 (socket says 172.21.15.5:52318) 2024-09-10T09:27:33.729 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:32.848+0000 7f8f23400640 1 -- 172.21.15.5:0/3935496324 learned_addr learned my addr 172.21.15.5:0/3935496324 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:27:33.729 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:32.848+0000 7f8f23400640 1 -- 172.21.15.5:0/3935496324 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f2410a810 con 0x7f8f24109970 2024-09-10T09:27:33.729 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:32.849+0000 7f8f23400640 1 --2- 172.21.15.5:0/3935496324 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8f24109970 0x7f8f24109d70 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f8f14009820 tx=0x7f8f1402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7444364c03825579 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:27:33.729 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:32.849+0000 7f8f22a00640 1 -- 172.21.15.5:0/3935496324 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8f14037070 con 0x7f8f24109970 2024-09-10T09:27:33.729 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:32.849+0000 7f8f22a00640 1 -- 172.21.15.5:0/3935496324 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f8f14032900 con 0x7f8f24109970 2024-09-10T09:27:33.729 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:32.849+0000 7f8f22a00640 1 -- 172.21.15.5:0/3935496324 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8f14030850 con 0x7f8f24109970 2024-09-10T09:27:33.729 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:32.850+0000 7f8f29200640 1 -- 172.21.15.5:0/3935496324 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8f24109970 msgr2=0x7f8f24109d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:27:33.730 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:32.850+0000 7f8f29200640 1 --2- 172.21.15.5:0/3935496324 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8f24109970 0x7f8f24109d70 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f8f14009820 tx=0x7f8f1402a270 comp rx=0 tx=0).stop 2024-09-10T09:27:33.730 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:32.850+0000 7f8f29200640 1 -- 172.21.15.5:0/3935496324 shutdown_connections 2024-09-10T09:27:33.730 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:32.850+0000 7f8f29200640 1 --2- 172.21.15.5:0/3935496324 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8f24109970 0x7f8f24109d70 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:27:33.730 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:32.850+0000 7f8f29200640 1 -- 172.21.15.5:0/3935496324 >> 172.21.15.5:0/3935496324 conn(0x7f8f2407cb90 msgr2=0x7f8f2407d000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:27:33.730 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:32.850+0000 7f8f29200640 1 -- 172.21.15.5:0/3935496324 shutdown_connections 2024-09-10T09:27:33.730 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:32.850+0000 7f8f29200640 1 -- 172.21.15.5:0/3935496324 wait complete. 2024-09-10T09:27:33.730 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:32.851+0000 7f8f29200640 1 Processor -- start 2024-09-10T09:27:33.730 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:32.851+0000 7f8f29200640 1 -- start start 2024-09-10T09:27:33.730 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:32.852+0000 7f8f29200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8f24109970 0x7f8f2419d700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:27:33.730 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:32.852+0000 7f8f29200640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f8f2410dbc0 con 0x7f8f24109970 2024-09-10T09:27:33.730 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:32.852+0000 7f8f23400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8f24109970 0x7f8f2419d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:27:33.730 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:32.852+0000 7f8f23400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8f24109970 0x7f8f2419d700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52332/0 (socket says 172.21.15.5:52332) 2024-09-10T09:27:33.731 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:32.852+0000 7f8f23400640 1 -- 172.21.15.5:0/3969722955 learned_addr learned my addr 172.21.15.5:0/3969722955 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:27:33.731 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:32.852+0000 7f8f23400640 1 -- 172.21.15.5:0/3969722955 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f2419dc40 con 0x7f8f24109970 2024-09-10T09:27:33.731 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:32.853+0000 7f8f23400640 1 --2- 172.21.15.5:0/3969722955 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8f24109970 0x7f8f2419d700 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f8f14032710 tx=0x7f8f14032740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:27:33.731 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:32.853+0000 7f8f20c00640 1 -- 172.21.15.5:0/3969722955 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8f14042070 con 0x7f8f24109970 2024-09-10T09:27:33.731 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:32.853+0000 7f8f20c00640 1 -- 172.21.15.5:0/3969722955 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f8f14032ca0 con 0x7f8f24109970 2024-09-10T09:27:33.731 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:32.853+0000 7f8f20c00640 1 -- 172.21.15.5:0/3969722955 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8f14037040 con 0x7f8f24109970 2024-09-10T09:27:33.731 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:32.853+0000 7f8f29200640 1 -- 172.21.15.5:0/3969722955 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f2419ded0 con 0x7f8f24109970 2024-09-10T09:27:33.731 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:32.853+0000 7f8f29200640 1 -- 172.21.15.5:0/3969722955 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f241a0bc0 con 0x7f8f24109970 2024-09-10T09:27:33.731 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:32.854+0000 7f8f20c00640 1 -- 172.21.15.5:0/3969722955 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 46287+0+0 (secure 0 0 0) 0x7f8f14045070 con 0x7f8f24109970 2024-09-10T09:27:33.731 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:32.854+0000 7f8f29200640 1 -- 172.21.15.5:0/3969722955 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f24109df0 con 0x7f8f24109970 2024-09-10T09:27:33.731 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:32.855+0000 7f8f20c00640 1 --2- 172.21.15.5:0/3969722955 >> [v2:172.21.15.5:6800/14425913,v1:172.21.15.5:6801/14425913] conn(0x7f8f08038fc0 0x7f8f0803b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:27:33.731 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:32.855+0000 7f8f20c00640 1 -- 172.21.15.5:0/3969722955 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f8f1406be40 con 0x7f8f24109970 2024-09-10T09:27:33.732 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:32.855+0000 7f8f1be00640 1 -- 172.21.15.5:0/3969722955 >> [v2:172.21.15.5:6800/14425913,v1:172.21.15.5:6801/14425913] conn(0x7f8f08038fc0 msgr2=0x7f8f0803b480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.5:6800/14425913 2024-09-10T09:27:33.732 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:32.855+0000 7f8f1be00640 1 --2- 172.21.15.5:0/3969722955 >> [v2:172.21.15.5:6800/14425913,v1:172.21.15.5:6801/14425913] conn(0x7f8f08038fc0 0x7f8f0803b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T09:27:33.732 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:32.860+0000 7f8f20c00640 1 -- 172.21.15.5:0/3969722955 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f8f14045360 con 0x7f8f24109970 2024-09-10T09:27:33.732 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:33.056+0000 7f8f1be00640 1 -- 172.21.15.5:0/3969722955 >> [v2:172.21.15.5:6800/14425913,v1:172.21.15.5:6801/14425913] conn(0x7f8f08038fc0 msgr2=0x7f8f0803b480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.5:6800/14425913 2024-09-10T09:27:33.732 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:33.056+0000 7f8f1be00640 1 --2- 172.21.15.5:0/3969722955 >> [v2:172.21.15.5:6800/14425913,v1:172.21.15.5:6801/14425913] conn(0x7f8f08038fc0 0x7f8f0803b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-10T09:27:33.732 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:33.095+0000 7f8f29200640 1 -- 172.21.15.5:0/3969722955 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f8f241a1240 con 0x7f8f24109970 2024-09-10T09:27:33.732 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:33.095+0000 7f8f20c00640 1 -- 172.21.15.5:0/3969722955 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) ==== 56+0+88 (secure 0 0 0) 0x7f8f1406b860 con 0x7f8f24109970 2024-09-10T09:27:33.732 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:33.098+0000 7f8f29200640 1 -- 172.21.15.5:0/3969722955 >> [v2:172.21.15.5:6800/14425913,v1:172.21.15.5:6801/14425913] conn(0x7f8f08038fc0 msgr2=0x7f8f0803b480 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T09:27:33.732 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:33.098+0000 7f8f29200640 1 --2- 172.21.15.5:0/3969722955 >> [v2:172.21.15.5:6800/14425913,v1:172.21.15.5:6801/14425913] conn(0x7f8f08038fc0 0x7f8f0803b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:27:33.732 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:33.098+0000 7f8f29200640 1 -- 172.21.15.5:0/3969722955 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8f24109970 msgr2=0x7f8f2419d700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:27:33.732 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:33.098+0000 7f8f29200640 1 --2- 172.21.15.5:0/3969722955 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8f24109970 0x7f8f2419d700 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f8f14032710 tx=0x7f8f14032740 comp rx=0 tx=0).stop 2024-09-10T09:27:33.732 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:33.098+0000 7f8f29200640 1 -- 172.21.15.5:0/3969722955 shutdown_connections 2024-09-10T09:27:33.732 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:33.098+0000 7f8f29200640 1 --2- 172.21.15.5:0/3969722955 >> [v2:172.21.15.5:6800/14425913,v1:172.21.15.5:6801/14425913] conn(0x7f8f08038fc0 0x7f8f0803b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:27:33.733 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:33.098+0000 7f8f29200640 1 --2- 172.21.15.5:0/3969722955 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8f24109970 0x7f8f2419d700 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:27:33.733 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:33.098+0000 7f8f29200640 1 -- 172.21.15.5:0/3969722955 >> 172.21.15.5:0/3969722955 conn(0x7f8f2407cb90 msgr2=0x7f8f24107640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:27:33.733 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:33.098+0000 7f8f29200640 1 -- 172.21.15.5:0/3969722955 shutdown_connections 2024-09-10T09:27:33.733 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:33.098+0000 7f8f29200640 1 -- 172.21.15.5:0/3969722955 wait complete. 2024-09-10T09:27:33.733 INFO:teuthology.orchestra.run.smithi005.stdout:Waiting for the mgr to restart... 2024-09-10T09:27:33.733 INFO:teuthology.orchestra.run.smithi005.stdout:Waiting for mgr epoch 5... 2024-09-10T09:27:33.984 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:33 smithi005 ceph-mon[30350]: from='client.? 172.21.15.5:0/3969722955' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-09-10T09:27:34.806 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 10 09:27:34 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mgr-x[30575]: 2024-09-10T09:27:34.483+0000 7fbc94236040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-10T09:27:34.806 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 10 09:27:34 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mgr-x[30575]: 2024-09-10T09:27:34.567+0000 7fbc94236040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-10T09:27:35.806 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 10 09:27:35 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mgr-x[30575]: 2024-09-10T09:27:35.512+0000 7fbc94236040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-10T09:27:35.806 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 10 09:27:35 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mgr-x[30575]: 2024-09-10T09:27:35.653+0000 7fbc94236040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-10T09:27:36.077 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 10 09:27:35 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mgr-x[30575]: 2024-09-10T09:27:35.823+0000 7fbc94236040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-10T09:27:36.077 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 10 09:27:35 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mgr-x[30575]: 2024-09-10T09:27:35.898+0000 7fbc94236040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-10T09:27:36.077 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 10 09:27:36 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mgr-x[30575]: 2024-09-10T09:27:36.076+0000 7fbc94236040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-10T09:27:36.422 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 10 09:27:36 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mgr-x[30575]: 2024-09-10T09:27:36.160+0000 7fbc94236040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-10T09:27:36.422 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 10 09:27:36 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mgr-x[30575]: 2024-09-10T09:27:36.234+0000 7fbc94236040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-10T09:27:36.422 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 10 09:27:36 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mgr-x[30575]: 2024-09-10T09:27:36.421+0000 7fbc94236040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-10T09:27:36.684 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 10 09:27:36 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mgr-x[30575]: 2024-09-10T09:27:36.501+0000 7fbc94236040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-10T09:27:36.684 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 10 09:27:36 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mgr-x[30575]: 2024-09-10T09:27:36.583+0000 7fbc94236040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-10T09:27:36.684 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 10 09:27:36 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mgr-x[30575]: 2024-09-10T09:27:36.683+0000 7fbc94236040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-10T09:27:37.056 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 10 09:27:36 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mgr-x[30575]: 2024-09-10T09:27:36.756+0000 7fbc94236040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-10T09:27:37.056 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 10 09:27:36 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mgr-x[30575]: 2024-09-10T09:27:36.829+0000 7fbc94236040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-10T09:27:38.056 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 10 09:27:37 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mgr-x[30575]: 2024-09-10T09:27:37.776+0000 7fbc94236040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-10T09:27:38.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:37 smithi005 ceph-mon[30350]: Active manager daemon x restarted 2024-09-10T09:27:38.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:37 smithi005 ceph-mon[30350]: Activating manager daemon x 2024-09-10T09:27:38.057 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:37 smithi005 ceph-mon[30350]: osdmap e2: 0 total, 0 up, 0 in 2024-09-10T09:27:38.057 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:37 smithi005 ceph-mon[30350]: mgrmap e6: x(active, starting, since 0.00411872s) 2024-09-10T09:27:38.057 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:37 smithi005 ceph-mon[30350]: from='mgr.14118 172.21.15.5:0/1435659661' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T09:27:38.057 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:37 smithi005 ceph-mon[30350]: from='mgr.14118 172.21.15.5:0/1435659661' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-10T09:27:38.057 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:37 smithi005 ceph-mon[30350]: from='mgr.14118 172.21.15.5:0/1435659661' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-10T09:27:38.057 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:37 smithi005 ceph-mon[30350]: from='mgr.14118 172.21.15.5:0/1435659661' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-10T09:27:38.057 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:37 smithi005 ceph-mon[30350]: from='mgr.14118 172.21.15.5:0/1435659661' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-10T09:27:38.057 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:37 smithi005 ceph-mon[30350]: Manager daemon x is now available 2024-09-10T09:27:38.057 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:37 smithi005 ceph-mon[30350]: from='mgr.14118 172.21.15.5:0/1435659661' entity='mgr.x' 2024-09-10T09:27:38.057 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:37 smithi005 ceph-mon[30350]: from='mgr.14118 172.21.15.5:0/1435659661' entity='mgr.x' 2024-09-10T09:27:38.057 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:37 smithi005 ceph-mon[30350]: from='mgr.14118 172.21.15.5:0/1435659661' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:27:38.057 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:37 smithi005 ceph-mon[30350]: from='mgr.14118 172.21.15.5:0/1435659661' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:27:39.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:38 smithi005 ceph-mon[30350]: Found migration_current of "None". Setting to last migration. 2024-09-10T09:27:39.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:38 smithi005 ceph-mon[30350]: from='mgr.14118 172.21.15.5:0/1435659661' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-10T09:27:39.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:38 smithi005 ceph-mon[30350]: from='mgr.14118 172.21.15.5:0/1435659661' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-10T09:27:39.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:38 smithi005 ceph-mon[30350]: mgrmap e7: x(active, since 1.00723s) 2024-09-10T09:27:39.468 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout { 2024-09-10T09:27:39.468 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-09-10T09:27:39.468 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "initialized": true 2024-09-10T09:27:39.468 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout } 2024-09-10T09:27:39.468 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:34.219+0000 7f1c57e00640 1 Processor -- start 2024-09-10T09:27:39.468 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:34.219+0000 7f1c57e00640 1 -- start start 2024-09-10T09:27:39.468 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:34.219+0000 7f1c57e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1c50109830 0x7f1c50109c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:27:39.468 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:34.219+0000 7f1c57e00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1c5010a200 con 0x7f1c50109830 2024-09-10T09:27:39.469 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:34.220+0000 7f1c55400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1c50109830 0x7f1c50109c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:27:39.469 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:34.220+0000 7f1c55400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1c50109830 0x7f1c50109c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52348/0 (socket says 172.21.15.5:52348) 2024-09-10T09:27:39.469 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:34.220+0000 7f1c55400640 1 -- 172.21.15.5:0/3511546958 learned_addr learned my addr 172.21.15.5:0/3511546958 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:27:39.469 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:34.220+0000 7f1c55400640 1 -- 172.21.15.5:0/3511546958 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c5010aa40 con 0x7f1c50109830 2024-09-10T09:27:39.469 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:34.222+0000 7f1c55400640 1 --2- 172.21.15.5:0/3511546958 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1c50109830 0x7f1c50109c30 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f1c44009820 tx=0x7f1c4402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49a4f4a932e285fd server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:27:39.469 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:34.222+0000 7f1c4fe00640 1 -- 172.21.15.5:0/3511546958 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1c44037070 con 0x7f1c50109830 2024-09-10T09:27:39.469 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:34.222+0000 7f1c4fe00640 1 -- 172.21.15.5:0/3511546958 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f1c44032900 con 0x7f1c50109830 2024-09-10T09:27:39.469 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:34.222+0000 7f1c4fe00640 1 -- 172.21.15.5:0/3511546958 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1c44030850 con 0x7f1c50109830 2024-09-10T09:27:39.469 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:34.223+0000 7f1c57e00640 1 -- 172.21.15.5:0/3511546958 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1c50109830 msgr2=0x7f1c50109c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:27:39.470 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:34.223+0000 7f1c57e00640 1 --2- 172.21.15.5:0/3511546958 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1c50109830 0x7f1c50109c30 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f1c44009820 tx=0x7f1c4402a270 comp rx=0 tx=0).stop 2024-09-10T09:27:39.470 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:34.223+0000 7f1c57e00640 1 -- 172.21.15.5:0/3511546958 shutdown_connections 2024-09-10T09:27:39.470 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:34.223+0000 7f1c57e00640 1 --2- 172.21.15.5:0/3511546958 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1c50109830 0x7f1c50109c30 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:27:39.470 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:34.223+0000 7f1c57e00640 1 -- 172.21.15.5:0/3511546958 >> 172.21.15.5:0/3511546958 conn(0x7f1c5007ce40 msgr2=0x7f1c5007d270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:27:39.470 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:34.223+0000 7f1c57e00640 1 -- 172.21.15.5:0/3511546958 shutdown_connections 2024-09-10T09:27:39.470 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:34.223+0000 7f1c57e00640 1 -- 172.21.15.5:0/3511546958 wait complete. 2024-09-10T09:27:39.470 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:34.224+0000 7f1c57e00640 1 Processor -- start 2024-09-10T09:27:39.470 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:34.224+0000 7f1c57e00640 1 -- start start 2024-09-10T09:27:39.470 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:34.224+0000 7f1c57e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1c50109830 0x7f1c5019d900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:27:39.470 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:34.224+0000 7f1c57e00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1c5010ddf0 con 0x7f1c50109830 2024-09-10T09:27:39.470 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:34.225+0000 7f1c55400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1c50109830 0x7f1c5019d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:27:39.470 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:34.225+0000 7f1c55400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1c50109830 0x7f1c5019d900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52360/0 (socket says 172.21.15.5:52360) 2024-09-10T09:27:39.471 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:34.225+0000 7f1c55400640 1 -- 172.21.15.5:0/3297063746 learned_addr learned my addr 172.21.15.5:0/3297063746 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:27:39.471 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:34.225+0000 7f1c55400640 1 -- 172.21.15.5:0/3297063746 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c5019de40 con 0x7f1c50109830 2024-09-10T09:27:39.471 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:34.226+0000 7f1c55400640 1 --2- 172.21.15.5:0/3297063746 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1c50109830 0x7f1c5019d900 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f1c440097f0 tx=0x7f1c44030ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:27:39.471 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:34.226+0000 7f1c4e000640 1 -- 172.21.15.5:0/3297063746 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1c44042070 con 0x7f1c50109830 2024-09-10T09:27:39.471 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:34.226+0000 7f1c4e000640 1 -- 172.21.15.5:0/3297063746 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f1c44032ca0 con 0x7f1c50109830 2024-09-10T09:27:39.471 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:34.226+0000 7f1c4e000640 1 -- 172.21.15.5:0/3297063746 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1c44037040 con 0x7f1c50109830 2024-09-10T09:27:39.471 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:34.226+0000 7f1c57e00640 1 -- 172.21.15.5:0/3297063746 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c5019e0d0 con 0x7f1c50109830 2024-09-10T09:27:39.471 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:34.226+0000 7f1c57e00640 1 -- 172.21.15.5:0/3297063746 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c501a0dc0 con 0x7f1c50109830 2024-09-10T09:27:39.471 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:34.227+0000 7f1c4e000640 1 -- 172.21.15.5:0/3297063746 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 46287+0+0 (secure 0 0 0) 0x7f1c44045070 con 0x7f1c50109830 2024-09-10T09:27:39.471 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:34.228+0000 7f1c4e000640 1 --2- 172.21.15.5:0/3297063746 >> [v2:172.21.15.5:6800/14425913,v1:172.21.15.5:6801/14425913] conn(0x7f1c38038fc0 0x7f1c3803b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:27:39.471 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:34.228+0000 7f1c4e000640 1 -- 172.21.15.5:0/3297063746 --> [v2:172.21.15.5:6800/14425913,v1:172.21.15.5:6801/14425913] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f1c3803bb90 con 0x7f1c38038fc0 2024-09-10T09:27:39.472 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:34.228+0000 7f1c4e000640 1 -- 172.21.15.5:0/3297063746 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f1c4406be80 con 0x7f1c50109830 2024-09-10T09:27:39.472 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:34.228+0000 7f1c54a00640 1 -- 172.21.15.5:0/3297063746 >> [v2:172.21.15.5:6800/14425913,v1:172.21.15.5:6801/14425913] conn(0x7f1c38038fc0 msgr2=0x7f1c3803b480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.5:6800/14425913 2024-09-10T09:27:39.472 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:34.228+0000 7f1c54a00640 1 --2- 172.21.15.5:0/3297063746 >> [v2:172.21.15.5:6800/14425913,v1:172.21.15.5:6801/14425913] conn(0x7f1c38038fc0 0x7f1c3803b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T09:27:39.472 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:34.429+0000 7f1c54a00640 1 -- 172.21.15.5:0/3297063746 >> [v2:172.21.15.5:6800/14425913,v1:172.21.15.5:6801/14425913] conn(0x7f1c38038fc0 msgr2=0x7f1c3803b480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.5:6800/14425913 2024-09-10T09:27:39.472 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:34.429+0000 7f1c54a00640 1 --2- 172.21.15.5:0/3297063746 >> [v2:172.21.15.5:6800/14425913,v1:172.21.15.5:6801/14425913] conn(0x7f1c38038fc0 0x7f1c3803b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-10T09:27:39.472 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:34.830+0000 7f1c54a00640 1 -- 172.21.15.5:0/3297063746 >> [v2:172.21.15.5:6800/14425913,v1:172.21.15.5:6801/14425913] conn(0x7f1c38038fc0 msgr2=0x7f1c3803b480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.5:6800/14425913 2024-09-10T09:27:39.472 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:34.830+0000 7f1c54a00640 1 --2- 172.21.15.5:0/3297063746 >> [v2:172.21.15.5:6800/14425913,v1:172.21.15.5:6801/14425913] conn(0x7f1c38038fc0 0x7f1c3803b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-10T09:27:39.472 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:35.631+0000 7f1c54a00640 1 -- 172.21.15.5:0/3297063746 >> [v2:172.21.15.5:6800/14425913,v1:172.21.15.5:6801/14425913] conn(0x7f1c38038fc0 msgr2=0x7f1c3803b480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.5:6800/14425913 2024-09-10T09:27:39.472 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:35.631+0000 7f1c54a00640 1 --2- 172.21.15.5:0/3297063746 >> [v2:172.21.15.5:6800/14425913,v1:172.21.15.5:6801/14425913] conn(0x7f1c38038fc0 0x7f1c3803b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-10T09:27:39.472 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:37.232+0000 7f1c54a00640 1 -- 172.21.15.5:0/3297063746 >> [v2:172.21.15.5:6800/14425913,v1:172.21.15.5:6801/14425913] conn(0x7f1c38038fc0 msgr2=0x7f1c3803b480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.5:6800/14425913 2024-09-10T09:27:39.472 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:37.233+0000 7f1c54a00640 1 --2- 172.21.15.5:0/3297063746 >> [v2:172.21.15.5:6800/14425913,v1:172.21.15.5:6801/14425913] conn(0x7f1c38038fc0 0x7f1c3803b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-09-10T09:27:39.472 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:37.780+0000 7f1c4e000640 1 -- 172.21.15.5:0/3297063746 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mgrmap(e 6) ==== 46054+0+0 (secure 0 0 0) 0x7f1c4403c660 con 0x7f1c50109830 2024-09-10T09:27:39.472 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:37.780+0000 7f1c4e000640 1 -- 172.21.15.5:0/3297063746 >> [v2:172.21.15.5:6800/14425913,v1:172.21.15.5:6801/14425913] conn(0x7f1c38038fc0 msgr2=0x7f1c3803b480 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T09:27:39.473 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:37.780+0000 7f1c4e000640 1 --2- 172.21.15.5:0/3297063746 >> [v2:172.21.15.5:6800/14425913,v1:172.21.15.5:6801/14425913] conn(0x7f1c38038fc0 0x7f1c3803b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:27:39.473 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:38.781+0000 7f1c4e000640 1 -- 172.21.15.5:0/3297063746 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mgrmap(e 7) ==== 46181+0+0 (secure 0 0 0) 0x7f1c4406dd70 con 0x7f1c50109830 2024-09-10T09:27:39.473 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:38.781+0000 7f1c4e000640 1 --2- 172.21.15.5:0/3297063746 >> [v2:172.21.15.5:6800/1527422686,v1:172.21.15.5:6801/1527422686] conn(0x7f1c3803c8f0 0x7f1c3803ece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:27:39.473 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:38.782+0000 7f1c4e000640 1 -- 172.21.15.5:0/3297063746 --> [v2:172.21.15.5:6800/1527422686,v1:172.21.15.5:6801/1527422686] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f1c3803bb90 con 0x7f1c3803c8f0 2024-09-10T09:27:39.473 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:38.782+0000 7f1c54a00640 1 --2- 172.21.15.5:0/3297063746 >> [v2:172.21.15.5:6800/1527422686,v1:172.21.15.5:6801/1527422686] conn(0x7f1c3803c8f0 0x7f1c3803ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:27:39.473 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:38.784+0000 7f1c54a00640 1 --2- 172.21.15.5:0/3297063746 >> [v2:172.21.15.5:6800/1527422686,v1:172.21.15.5:6801/1527422686] conn(0x7f1c3803c8f0 0x7f1c3803ece0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f1c3c003980 tx=0x7f1c3c0092b0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:27:39.473 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:38.785+0000 7f1c4e000640 1 -- 172.21.15.5:0/3297063746 <== mgr.14118 v2:172.21.15.5:6800/1527422686 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9564 (secure 0 0 0) 0x7f1c3803bb90 con 0x7f1c3803c8f0 2024-09-10T09:27:39.473 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:38.793+0000 7f1c57e00640 1 -- 172.21.15.5:0/3297063746 --> [v2:172.21.15.5:6800/1527422686,v1:172.21.15.5:6801/1527422686] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f1c50109c30 con 0x7f1c3803c8f0 2024-09-10T09:27:39.473 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:38.793+0000 7f1c4e000640 1 -- 172.21.15.5:0/3297063746 <== mgr.14118 v2:172.21.15.5:6800/1527422686 2 ==== command_reply(tid 1: 0 ) ==== 8+0+51 (secure 0 0 0) 0x7f1c50109c30 con 0x7f1c3803c8f0 2024-09-10T09:27:39.473 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:38.794+0000 7f1c57e00640 1 -- 172.21.15.5:0/3297063746 >> [v2:172.21.15.5:6800/1527422686,v1:172.21.15.5:6801/1527422686] conn(0x7f1c3803c8f0 msgr2=0x7f1c3803ece0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:27:39.473 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:38.794+0000 7f1c57e00640 1 --2- 172.21.15.5:0/3297063746 >> [v2:172.21.15.5:6800/1527422686,v1:172.21.15.5:6801/1527422686] conn(0x7f1c3803c8f0 0x7f1c3803ece0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f1c3c003980 tx=0x7f1c3c0092b0 comp rx=0 tx=0).stop 2024-09-10T09:27:39.473 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:38.794+0000 7f1c57e00640 1 -- 172.21.15.5:0/3297063746 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1c50109830 msgr2=0x7f1c5019d900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:27:39.473 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:38.794+0000 7f1c57e00640 1 --2- 172.21.15.5:0/3297063746 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1c50109830 0x7f1c5019d900 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f1c440097f0 tx=0x7f1c44030ec0 comp rx=0 tx=0).stop 2024-09-10T09:27:39.474 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:38.794+0000 7f1c57e00640 1 -- 172.21.15.5:0/3297063746 shutdown_connections 2024-09-10T09:27:39.474 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:38.794+0000 7f1c57e00640 1 --2- 172.21.15.5:0/3297063746 >> [v2:172.21.15.5:6800/1527422686,v1:172.21.15.5:6801/1527422686] conn(0x7f1c3803c8f0 0x7f1c3803ece0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:27:39.474 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:38.794+0000 7f1c57e00640 1 --2- 172.21.15.5:0/3297063746 >> [v2:172.21.15.5:6800/14425913,v1:172.21.15.5:6801/14425913] conn(0x7f1c38038fc0 0x7f1c3803b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:27:39.474 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:38.794+0000 7f1c57e00640 1 --2- 172.21.15.5:0/3297063746 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1c50109830 0x7f1c5019d900 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:27:39.474 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:38.794+0000 7f1c57e00640 1 -- 172.21.15.5:0/3297063746 >> 172.21.15.5:0/3297063746 conn(0x7f1c5007ce40 msgr2=0x7f1c501075b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:27:39.474 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:38.794+0000 7f1c57e00640 1 -- 172.21.15.5:0/3297063746 shutdown_connections 2024-09-10T09:27:39.474 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:38.794+0000 7f1c57e00640 1 -- 172.21.15.5:0/3297063746 wait complete. 2024-09-10T09:27:39.474 INFO:teuthology.orchestra.run.smithi005.stdout:mgr epoch 5 is available 2024-09-10T09:27:39.474 INFO:teuthology.orchestra.run.smithi005.stdout:Setting orchestrator backend to cephadm... 2024-09-10T09:27:40.569 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:40 smithi005 ceph-mon[30350]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-09-10T09:27:40.569 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:40 smithi005 ceph-mon[30350]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-09-10T09:27:40.569 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:40 smithi005 ceph-mon[30350]: from='mgr.14118 172.21.15.5:0/1435659661' entity='mgr.x' 2024-09-10T09:27:40.569 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:40 smithi005 ceph-mon[30350]: from='mgr.14118 172.21.15.5:0/1435659661' entity='mgr.x' 2024-09-10T09:27:40.569 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:40 smithi005 ceph-mon[30350]: from='mgr.14118 172.21.15.5:0/1435659661' entity='mgr.x' 2024-09-10T09:27:40.569 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:40 smithi005 ceph-mon[30350]: from='mgr.14118 172.21.15.5:0/1435659661' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:27:40.570 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 10 09:27:40 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mgr-x[30575]: 2024-09-10T09:27:40.472+0000 7fbbec600640 -1 log_channel(cephadm) log [ERR] : [10/Sep/2024:09:27:40] ENGINE Error in HTTPServer.serve 2024-09-10T09:27:40.570 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 10 09:27:40 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mgr-x[30575]: Traceback (most recent call last): 2024-09-10T09:27:40.570 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 10 09:27:40 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mgr-x[30575]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-10T09:27:40.570 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 10 09:27:40 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mgr-x[30575]: self._connections.run(self.expiration_interval) 2024-09-10T09:27:40.570 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 10 09:27:40 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mgr-x[30575]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-10T09:27:40.570 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 10 09:27:40 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mgr-x[30575]: self._run(expiration_interval) 2024-09-10T09:27:40.570 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 10 09:27:40 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mgr-x[30575]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-10T09:27:40.570 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 10 09:27:40 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mgr-x[30575]: new_conn = self._from_server_socket(self.server.socket) 2024-09-10T09:27:40.570 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 10 09:27:40 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mgr-x[30575]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-10T09:27:40.570 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 10 09:27:40 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mgr-x[30575]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-10T09:27:40.570 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 10 09:27:40 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mgr-x[30575]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-10T09:27:40.570 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 10 09:27:40 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mgr-x[30575]: s = self.context.wrap_socket( 2024-09-10T09:27:40.571 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 10 09:27:40 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mgr-x[30575]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-10T09:27:40.571 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 10 09:27:40 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mgr-x[30575]: return self.sslsocket_class._create( 2024-09-10T09:27:40.571 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 10 09:27:40 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mgr-x[30575]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-10T09:27:40.571 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 10 09:27:40 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mgr-x[30575]: self.do_handshake() 2024-09-10T09:27:40.571 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 10 09:27:40 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mgr-x[30575]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-10T09:27:40.571 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 10 09:27:40 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mgr-x[30575]: self._sslobj.do_handshake() 2024-09-10T09:27:40.571 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 10 09:27:40 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mgr-x[30575]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-10T09:27:40.571 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 10 09:27:40 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mgr-x[30575]: 2024-09-10T09:27:40.882 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:39.950+0000 7fbf74c00640 1 Processor -- start 2024-09-10T09:27:40.882 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:39.950+0000 7fbf74c00640 1 -- start start 2024-09-10T09:27:40.882 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:39.951+0000 7fbf74c00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbf70109920 0x7fbf70109d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:27:40.882 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:39.951+0000 7fbf74c00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fbf7010a310 con 0x7fbf70109920 2024-09-10T09:27:40.882 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:39.951+0000 7fbf6de00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbf70109920 0x7fbf70109d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:27:40.882 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:39.951+0000 7fbf6de00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbf70109920 0x7fbf70109d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52424/0 (socket says 172.21.15.5:52424) 2024-09-10T09:27:40.882 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:39.951+0000 7fbf6de00640 1 -- 172.21.15.5:0/1718172616 learned_addr learned my addr 172.21.15.5:0/1718172616 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:27:40.883 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:39.951+0000 7fbf6de00640 1 -- 172.21.15.5:0/1718172616 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf7010aa10 con 0x7fbf70109920 2024-09-10T09:27:40.883 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:39.952+0000 7fbf6de00640 1 --2- 172.21.15.5:0/1718172616 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbf70109920 0x7fbf70109d40 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fbf60009820 tx=0x7fbf6002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1197bddc3c050af0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:27:40.883 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:39.953+0000 7fbf6ca00640 1 -- 172.21.15.5:0/1718172616 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbf60037070 con 0x7fbf70109920 2024-09-10T09:27:40.883 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:39.953+0000 7fbf6ca00640 1 -- 172.21.15.5:0/1718172616 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fbf60032900 con 0x7fbf70109920 2024-09-10T09:27:40.883 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:39.953+0000 7fbf6ca00640 1 -- 172.21.15.5:0/1718172616 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbf60030850 con 0x7fbf70109920 2024-09-10T09:27:40.883 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:39.953+0000 7fbf74c00640 1 -- 172.21.15.5:0/1718172616 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbf70109920 msgr2=0x7fbf70109d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:27:40.883 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:39.953+0000 7fbf74c00640 1 --2- 172.21.15.5:0/1718172616 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbf70109920 0x7fbf70109d40 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fbf60009820 tx=0x7fbf6002a270 comp rx=0 tx=0).stop 2024-09-10T09:27:40.883 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:39.953+0000 7fbf74c00640 1 -- 172.21.15.5:0/1718172616 shutdown_connections 2024-09-10T09:27:40.883 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:39.953+0000 7fbf74c00640 1 --2- 172.21.15.5:0/1718172616 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbf70109920 0x7fbf70109d40 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:27:40.883 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:39.953+0000 7fbf74c00640 1 -- 172.21.15.5:0/1718172616 >> 172.21.15.5:0/1718172616 conn(0x7fbf7007cf40 msgr2=0x7fbf7007d350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:27:40.883 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:39.954+0000 7fbf74c00640 1 -- 172.21.15.5:0/1718172616 shutdown_connections 2024-09-10T09:27:40.883 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:39.954+0000 7fbf74c00640 1 -- 172.21.15.5:0/1718172616 wait complete. 2024-09-10T09:27:40.883 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:39.954+0000 7fbf74c00640 1 Processor -- start 2024-09-10T09:27:40.884 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:39.955+0000 7fbf74c00640 1 -- start start 2024-09-10T09:27:40.884 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:39.955+0000 7fbf74c00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbf70109920 0x7fbf7019d800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:27:40.884 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:39.955+0000 7fbf74c00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fbf7010ddc0 con 0x7fbf70109920 2024-09-10T09:27:40.884 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:39.955+0000 7fbf6de00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbf70109920 0x7fbf7019d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:27:40.884 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:39.955+0000 7fbf6de00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbf70109920 0x7fbf7019d800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52440/0 (socket says 172.21.15.5:52440) 2024-09-10T09:27:40.884 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:39.955+0000 7fbf6de00640 1 -- 172.21.15.5:0/1787881361 learned_addr learned my addr 172.21.15.5:0/1787881361 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:27:40.884 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:39.955+0000 7fbf6de00640 1 -- 172.21.15.5:0/1787881361 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf7019dd40 con 0x7fbf70109920 2024-09-10T09:27:40.884 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:39.956+0000 7fbf6de00640 1 --2- 172.21.15.5:0/1787881361 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbf70109920 0x7fbf7019d800 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fbf600097f0 tx=0x7fbf60030d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:27:40.884 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:39.956+0000 7fbf66a00640 1 -- 172.21.15.5:0/1787881361 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbf60037040 con 0x7fbf70109920 2024-09-10T09:27:40.884 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:39.956+0000 7fbf66a00640 1 -- 172.21.15.5:0/1787881361 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fbf60040070 con 0x7fbf70109920 2024-09-10T09:27:40.884 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:39.956+0000 7fbf66a00640 1 -- 172.21.15.5:0/1787881361 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbf6003a390 con 0x7fbf70109920 2024-09-10T09:27:40.884 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:39.956+0000 7fbf74c00640 1 -- 172.21.15.5:0/1787881361 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf7019dfd0 con 0x7fbf70109920 2024-09-10T09:27:40.884 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:39.956+0000 7fbf74c00640 1 -- 172.21.15.5:0/1787881361 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf701a0cc0 con 0x7fbf70109920 2024-09-10T09:27:40.885 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:39.957+0000 7fbf74c00640 1 -- 172.21.15.5:0/1787881361 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf7010df40 con 0x7fbf70109920 2024-09-10T09:27:40.885 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:39.958+0000 7fbf66a00640 1 -- 172.21.15.5:0/1787881361 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 7) ==== 46181+0+0 (secure 0 0 0) 0x7fbf6003b430 con 0x7fbf70109920 2024-09-10T09:27:40.885 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:39.958+0000 7fbf66a00640 1 --2- 172.21.15.5:0/1787881361 >> [v2:172.21.15.5:6800/1527422686,v1:172.21.15.5:6801/1527422686] conn(0x7fbf54038eb0 0x7fbf5403b370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:27:40.885 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:39.958+0000 7fbf66a00640 1 -- 172.21.15.5:0/1787881361 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fbf6006c300 con 0x7fbf70109920 2024-09-10T09:27:40.885 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:39.959+0000 7fbf6d400640 1 --2- 172.21.15.5:0/1787881361 >> [v2:172.21.15.5:6800/1527422686,v1:172.21.15.5:6801/1527422686] conn(0x7fbf54038eb0 0x7fbf5403b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:27:40.885 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:39.959+0000 7fbf6d400640 1 --2- 172.21.15.5:0/1787881361 >> [v2:172.21.15.5:6800/1527422686,v1:172.21.15.5:6801/1527422686] conn(0x7fbf54038eb0 0x7fbf5403b370 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fbf5c0098c0 tx=0x7fbf5c006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:27:40.885 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:39.964+0000 7fbf66a00640 1 -- 172.21.15.5:0/1787881361 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fbf60032900 con 0x7fbf70109920 2024-09-10T09:27:40.885 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:40.162+0000 7fbf74c00640 1 -- 172.21.15.5:0/1787881361 --> [v2:172.21.15.5:6800/1527422686,v1:172.21.15.5:6801/1527422686] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) -- 0x7fbf70107ca0 con 0x7fbf54038eb0 2024-09-10T09:27:40.885 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:40.168+0000 7fbf66a00640 1 -- 172.21.15.5:0/1787881361 <== mgr.14118 v2:172.21.15.5:6800/1527422686 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fbf70107ca0 con 0x7fbf54038eb0 2024-09-10T09:27:40.885 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:40.170+0000 7fbf74c00640 1 -- 172.21.15.5:0/1787881361 >> [v2:172.21.15.5:6800/1527422686,v1:172.21.15.5:6801/1527422686] conn(0x7fbf54038eb0 msgr2=0x7fbf5403b370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:27:40.885 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:40.170+0000 7fbf74c00640 1 --2- 172.21.15.5:0/1787881361 >> [v2:172.21.15.5:6800/1527422686,v1:172.21.15.5:6801/1527422686] conn(0x7fbf54038eb0 0x7fbf5403b370 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fbf5c0098c0 tx=0x7fbf5c006db0 comp rx=0 tx=0).stop 2024-09-10T09:27:40.885 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:40.170+0000 7fbf74c00640 1 -- 172.21.15.5:0/1787881361 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbf70109920 msgr2=0x7fbf7019d800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:27:40.886 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:40.170+0000 7fbf74c00640 1 --2- 172.21.15.5:0/1787881361 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbf70109920 0x7fbf7019d800 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fbf600097f0 tx=0x7fbf60030d80 comp rx=0 tx=0).stop 2024-09-10T09:27:40.886 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:40.170+0000 7fbf74c00640 1 -- 172.21.15.5:0/1787881361 shutdown_connections 2024-09-10T09:27:40.886 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:40.170+0000 7fbf74c00640 1 --2- 172.21.15.5:0/1787881361 >> [v2:172.21.15.5:6800/1527422686,v1:172.21.15.5:6801/1527422686] conn(0x7fbf54038eb0 0x7fbf5403b370 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:27:40.886 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:40.170+0000 7fbf74c00640 1 --2- 172.21.15.5:0/1787881361 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbf70109920 0x7fbf7019d800 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:27:40.886 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:40.170+0000 7fbf74c00640 1 -- 172.21.15.5:0/1787881361 >> 172.21.15.5:0/1787881361 conn(0x7fbf7007cf40 msgr2=0x7fbf70107590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:27:40.886 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:40.171+0000 7fbf74c00640 1 -- 172.21.15.5:0/1787881361 shutdown_connections 2024-09-10T09:27:40.886 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:40.171+0000 7fbf74c00640 1 -- 172.21.15.5:0/1787881361 wait complete. 2024-09-10T09:27:41.540 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:41 smithi005 ceph-mon[30350]: from='client.14130 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T09:27:41.541 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:41 smithi005 ceph-mon[30350]: [10/Sep/2024:09:27:40] ENGINE Bus STARTING 2024-09-10T09:27:41.541 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:41 smithi005 ceph-mon[30350]: mgrmap e8: x(active, since 2s) 2024-09-10T09:27:41.541 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:41 smithi005 ceph-mon[30350]: [10/Sep/2024:09:27:40] ENGINE Serving on http://172.21.15.5:8765 2024-09-10T09:27:41.541 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:41 smithi005 ceph-mon[30350]: [10/Sep/2024:09:27:40] ENGINE Serving on https://172.21.15.5:7150 2024-09-10T09:27:41.541 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:41 smithi005 ceph-mon[30350]: [10/Sep/2024:09:27:40] ENGINE Bus STARTED 2024-09-10T09:27:41.541 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:41 smithi005 ceph-mon[30350]: from='mgr.14118 172.21.15.5:0/1435659661' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:27:41.541 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:41 smithi005 ceph-mon[30350]: [10/Sep/2024:09:27:40] ENGINE Error in HTTPServer.serve 2024-09-10T09:27:41.541 INFO:journalctl@ceph.mon.a.smithi005.stdout: Traceback (most recent call last): 2024-09-10T09:27:41.541 INFO:journalctl@ceph.mon.a.smithi005.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-10T09:27:41.541 INFO:journalctl@ceph.mon.a.smithi005.stdout: self._connections.run(self.expiration_interval) 2024-09-10T09:27:41.541 INFO:journalctl@ceph.mon.a.smithi005.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-10T09:27:41.541 INFO:journalctl@ceph.mon.a.smithi005.stdout: self._run(expiration_interval) 2024-09-10T09:27:41.542 INFO:journalctl@ceph.mon.a.smithi005.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-10T09:27:41.542 INFO:journalctl@ceph.mon.a.smithi005.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-10T09:27:41.542 INFO:journalctl@ceph.mon.a.smithi005.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-10T09:27:41.542 INFO:journalctl@ceph.mon.a.smithi005.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-10T09:27:41.542 INFO:journalctl@ceph.mon.a.smithi005.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-10T09:27:41.542 INFO:journalctl@ceph.mon.a.smithi005.stdout: s = self.context.wrap_socket( 2024-09-10T09:27:41.542 INFO:journalctl@ceph.mon.a.smithi005.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-10T09:27:41.542 INFO:journalctl@ceph.mon.a.smithi005.stdout: return self.sslsocket_class._create( 2024-09-10T09:27:41.542 INFO:journalctl@ceph.mon.a.smithi005.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-10T09:27:41.542 INFO:journalctl@ceph.mon.a.smithi005.stdout: self.do_handshake() 2024-09-10T09:27:41.542 INFO:journalctl@ceph.mon.a.smithi005.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-10T09:27:41.542 INFO:journalctl@ceph.mon.a.smithi005.stdout: self._sslobj.do_handshake() 2024-09-10T09:27:41.543 INFO:journalctl@ceph.mon.a.smithi005.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-10T09:27:42.197 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout value unchanged 2024-09-10T09:27:42.198 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:41.340+0000 7f6795400640 1 Processor -- start 2024-09-10T09:27:42.198 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:41.340+0000 7f6795400640 1 -- start start 2024-09-10T09:27:42.198 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:41.340+0000 7f6795400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6790109970 0x7f6790109d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:27:42.198 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:41.340+0000 7f6795400640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f679010a340 con 0x7f6790109970 2024-09-10T09:27:42.198 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:41.341+0000 7f678fe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6790109970 0x7f6790109d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:27:42.198 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:41.341+0000 7f678fe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6790109970 0x7f6790109d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51216/0 (socket says 172.21.15.5:51216) 2024-09-10T09:27:42.198 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:41.341+0000 7f678fe00640 1 -- 172.21.15.5:0/455257384 learned_addr learned my addr 172.21.15.5:0/455257384 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:27:42.198 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:41.341+0000 7f678fe00640 1 -- 172.21.15.5:0/455257384 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f679010a840 con 0x7f6790109970 2024-09-10T09:27:42.198 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:41.342+0000 7f678fe00640 1 --2- 172.21.15.5:0/455257384 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6790109970 0x7f6790109d70 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f6780009820 tx=0x7f678002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34f0304eb6271d70 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:27:42.198 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:41.342+0000 7f678ea00640 1 -- 172.21.15.5:0/455257384 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6780037070 con 0x7f6790109970 2024-09-10T09:27:42.198 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:41.342+0000 7f678ea00640 1 -- 172.21.15.5:0/455257384 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f6780032900 con 0x7f6790109970 2024-09-10T09:27:42.198 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:41.342+0000 7f678ea00640 1 -- 172.21.15.5:0/455257384 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6780030850 con 0x7f6790109970 2024-09-10T09:27:42.199 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:41.343+0000 7f6795400640 1 -- 172.21.15.5:0/455257384 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6790109970 msgr2=0x7f6790109d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:27:42.199 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:41.343+0000 7f6795400640 1 --2- 172.21.15.5:0/455257384 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6790109970 0x7f6790109d70 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f6780009820 tx=0x7f678002a270 comp rx=0 tx=0).stop 2024-09-10T09:27:42.199 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:41.343+0000 7f6795400640 1 -- 172.21.15.5:0/455257384 shutdown_connections 2024-09-10T09:27:42.199 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:41.343+0000 7f6795400640 1 --2- 172.21.15.5:0/455257384 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6790109970 0x7f6790109d70 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:27:42.199 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:41.343+0000 7f6795400640 1 -- 172.21.15.5:0/455257384 >> 172.21.15.5:0/455257384 conn(0x7f679007cca0 msgr2=0x7f679007d0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:27:42.199 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:41.343+0000 7f6795400640 1 -- 172.21.15.5:0/455257384 shutdown_connections 2024-09-10T09:27:42.199 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:41.343+0000 7f6795400640 1 -- 172.21.15.5:0/455257384 wait complete. 2024-09-10T09:27:42.199 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:41.344+0000 7f6795400640 1 Processor -- start 2024-09-10T09:27:42.199 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:41.344+0000 7f6795400640 1 -- start start 2024-09-10T09:27:42.199 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:41.345+0000 7f6795400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6790109970 0x7f679019d730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:27:42.199 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:41.345+0000 7f6795400640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f679010dbf0 con 0x7f6790109970 2024-09-10T09:27:42.199 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:41.345+0000 7f678fe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6790109970 0x7f679019d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:27:42.199 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:41.345+0000 7f678fe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6790109970 0x7f679019d730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51222/0 (socket says 172.21.15.5:51222) 2024-09-10T09:27:42.199 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:41.345+0000 7f678fe00640 1 -- 172.21.15.5:0/1590240543 learned_addr learned my addr 172.21.15.5:0/1590240543 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:27:42.200 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:41.345+0000 7f678fe00640 1 -- 172.21.15.5:0/1590240543 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f679019dc70 con 0x7f6790109970 2024-09-10T09:27:42.200 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:41.345+0000 7f678fe00640 1 --2- 172.21.15.5:0/1590240543 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6790109970 0x7f679019d730 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f67800097f0 tx=0x7f6780030d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:27:42.200 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:41.346+0000 7f678cc00640 1 -- 172.21.15.5:0/1590240543 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6780042070 con 0x7f6790109970 2024-09-10T09:27:42.200 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:41.346+0000 7f678cc00640 1 -- 172.21.15.5:0/1590240543 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f6780032be0 con 0x7f6790109970 2024-09-10T09:27:42.200 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:41.346+0000 7f678cc00640 1 -- 172.21.15.5:0/1590240543 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6780037040 con 0x7f6790109970 2024-09-10T09:27:42.200 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:41.346+0000 7f6795400640 1 -- 172.21.15.5:0/1590240543 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f679019df00 con 0x7f6790109970 2024-09-10T09:27:42.200 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:41.346+0000 7f6795400640 1 -- 172.21.15.5:0/1590240543 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67901a0bf0 con 0x7f6790109970 2024-09-10T09:27:42.200 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:41.347+0000 7f678cc00640 1 -- 172.21.15.5:0/1590240543 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f6780045070 con 0x7f6790109970 2024-09-10T09:27:42.200 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:41.347+0000 7f6795400640 1 -- 172.21.15.5:0/1590240543 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6790109df0 con 0x7f6790109970 2024-09-10T09:27:42.200 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:41.347+0000 7f678cc00640 1 --2- 172.21.15.5:0/1590240543 >> [v2:172.21.15.5:6800/1527422686,v1:172.21.15.5:6801/1527422686] conn(0x7f6774038fc0 0x7f677403b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:27:42.201 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:41.347+0000 7f678cc00640 1 -- 172.21.15.5:0/1590240543 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f678006bec0 con 0x7f6790109970 2024-09-10T09:27:42.201 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:41.348+0000 7f678f400640 1 --2- 172.21.15.5:0/1590240543 >> [v2:172.21.15.5:6800/1527422686,v1:172.21.15.5:6801/1527422686] conn(0x7f6774038fc0 0x7f677403b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:27:42.201 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:41.348+0000 7f678f400640 1 --2- 172.21.15.5:0/1590240543 >> [v2:172.21.15.5:6800/1527422686,v1:172.21.15.5:6801/1527422686] conn(0x7f6774038fc0 0x7f677403b480 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f677c0098c0 tx=0x7f677c006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:27:42.201 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:41.353+0000 7f678cc00640 1 -- 172.21.15.5:0/1590240543 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f678006b810 con 0x7f6790109970 2024-09-10T09:27:42.201 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:41.538+0000 7f6795400640 1 -- 172.21.15.5:0/1590240543 --> [v2:172.21.15.5:6800/1527422686,v1:172.21.15.5:6801/1527422686] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) -- 0x7f6790107d10 con 0x7f6774038fc0 2024-09-10T09:27:42.201 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:41.539+0000 7f678cc00640 1 -- 172.21.15.5:0/1590240543 <== mgr.14118 v2:172.21.15.5:6800/1527422686 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+16 (secure 0 0 0) 0x7f6790107d10 con 0x7f6774038fc0 2024-09-10T09:27:42.201 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:41.541+0000 7f6795400640 1 -- 172.21.15.5:0/1590240543 >> [v2:172.21.15.5:6800/1527422686,v1:172.21.15.5:6801/1527422686] conn(0x7f6774038fc0 msgr2=0x7f677403b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:27:42.201 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:41.541+0000 7f6795400640 1 --2- 172.21.15.5:0/1590240543 >> [v2:172.21.15.5:6800/1527422686,v1:172.21.15.5:6801/1527422686] conn(0x7f6774038fc0 0x7f677403b480 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f677c0098c0 tx=0x7f677c006db0 comp rx=0 tx=0).stop 2024-09-10T09:27:42.201 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:41.541+0000 7f6795400640 1 -- 172.21.15.5:0/1590240543 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6790109970 msgr2=0x7f679019d730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:27:42.201 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:41.541+0000 7f6795400640 1 --2- 172.21.15.5:0/1590240543 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6790109970 0x7f679019d730 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f67800097f0 tx=0x7f6780030d80 comp rx=0 tx=0).stop 2024-09-10T09:27:42.201 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:41.542+0000 7f6795400640 1 -- 172.21.15.5:0/1590240543 shutdown_connections 2024-09-10T09:27:42.202 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:41.542+0000 7f6795400640 1 --2- 172.21.15.5:0/1590240543 >> [v2:172.21.15.5:6800/1527422686,v1:172.21.15.5:6801/1527422686] conn(0x7f6774038fc0 0x7f677403b480 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:27:42.202 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:41.542+0000 7f6795400640 1 --2- 172.21.15.5:0/1590240543 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6790109970 0x7f679019d730 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:27:42.202 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:41.542+0000 7f6795400640 1 -- 172.21.15.5:0/1590240543 >> 172.21.15.5:0/1590240543 conn(0x7f679007cca0 msgr2=0x7f6790107600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:27:42.202 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:41.542+0000 7f6795400640 1 -- 172.21.15.5:0/1590240543 shutdown_connections 2024-09-10T09:27:42.202 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:41.542+0000 7f6795400640 1 -- 172.21.15.5:0/1590240543 wait complete. 2024-09-10T09:27:42.202 INFO:teuthology.orchestra.run.smithi005.stdout:Generating ssh key... 2024-09-10T09:27:42.460 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:42 smithi005 ceph-mon[30350]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T09:27:43.525 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 10 09:27:43 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mgr-x[30575]: Generating public/private rsa key pair. 2024-09-10T09:27:43.525 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 10 09:27:43 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mgr-x[30575]: Your identification has been saved in /tmp/tmpdj6lwoqg/key 2024-09-10T09:27:43.525 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 10 09:27:43 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mgr-x[30575]: Your public key has been saved in /tmp/tmpdj6lwoqg/key.pub 2024-09-10T09:27:43.525 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 10 09:27:43 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mgr-x[30575]: The key fingerprint is: 2024-09-10T09:27:43.526 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 10 09:27:43 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mgr-x[30575]: SHA256:b1nx/VNeAS1Be9sdzft996UOoT+wE8JTmkaKVPYOpDI ceph-bade4b50-6f56-11ef-bcea-c7b262605968 2024-09-10T09:27:43.526 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 10 09:27:43 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mgr-x[30575]: The key's randomart image is: 2024-09-10T09:27:43.526 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 10 09:27:43 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mgr-x[30575]: +---[RSA 3072]----+ 2024-09-10T09:27:43.526 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 10 09:27:43 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mgr-x[30575]: | .+o | 2024-09-10T09:27:43.526 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 10 09:27:43 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mgr-x[30575]: | + .oo.| 2024-09-10T09:27:43.526 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 10 09:27:43 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mgr-x[30575]: | = . ...+o| 2024-09-10T09:27:43.526 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 10 09:27:43 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mgr-x[30575]: | E o . o . o..B| 2024-09-10T09:27:43.526 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 10 09:27:43 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mgr-x[30575]: | + . *S+ o .o*| 2024-09-10T09:27:43.526 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 10 09:27:43 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mgr-x[30575]: | . . O.o+ . .*| 2024-09-10T09:27:43.526 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 10 09:27:43 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mgr-x[30575]: | . o=+. .O| 2024-09-10T09:27:43.526 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 10 09:27:43 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mgr-x[30575]: | .o... .*| 2024-09-10T09:27:43.526 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 10 09:27:43 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mgr-x[30575]: | ..oo .| 2024-09-10T09:27:43.526 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 10 09:27:43 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mgr-x[30575]: +----[SHA256]-----+ 2024-09-10T09:27:43.839 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:42.688+0000 7f01dce00640 1 Processor -- start 2024-09-10T09:27:43.839 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:42.688+0000 7f01dce00640 1 -- start start 2024-09-10T09:27:43.839 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:42.688+0000 7f01dce00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f01d8107680 0x7f01d8107a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:27:43.839 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:42.688+0000 7f01dce00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f01d8107fc0 con 0x7f01d8107680 2024-09-10T09:27:43.839 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:42.689+0000 7f01d5e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f01d8107680 0x7f01d8107a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:27:43.839 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:42.689+0000 7f01d5e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f01d8107680 0x7f01d8107a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51234/0 (socket says 172.21.15.5:51234) 2024-09-10T09:27:43.840 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:42.689+0000 7f01d5e00640 1 -- 172.21.15.5:0/2849657706 learned_addr learned my addr 172.21.15.5:0/2849657706 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:27:43.840 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:42.689+0000 7f01d5e00640 1 -- 172.21.15.5:0/2849657706 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01d8108140 con 0x7f01d8107680 2024-09-10T09:27:43.840 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:42.690+0000 7f01d5e00640 1 --2- 172.21.15.5:0/2849657706 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f01d8107680 0x7f01d8107a80 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f01c8009820 tx=0x7f01c802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e43595b1633c2c6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:27:43.840 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:42.690+0000 7f01d4a00640 1 -- 172.21.15.5:0/2849657706 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f01c8037070 con 0x7f01d8107680 2024-09-10T09:27:43.840 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:42.690+0000 7f01d4a00640 1 -- 172.21.15.5:0/2849657706 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f01c8032900 con 0x7f01d8107680 2024-09-10T09:27:43.840 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:42.690+0000 7f01d4a00640 1 -- 172.21.15.5:0/2849657706 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f01c8030850 con 0x7f01d8107680 2024-09-10T09:27:43.840 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:42.691+0000 7f01dce00640 1 -- 172.21.15.5:0/2849657706 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f01d8107680 msgr2=0x7f01d8107a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:27:43.841 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:42.691+0000 7f01dce00640 1 --2- 172.21.15.5:0/2849657706 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f01d8107680 0x7f01d8107a80 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f01c8009820 tx=0x7f01c802a270 comp rx=0 tx=0).stop 2024-09-10T09:27:43.841 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:42.691+0000 7f01dce00640 1 -- 172.21.15.5:0/2849657706 shutdown_connections 2024-09-10T09:27:43.841 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:42.691+0000 7f01dce00640 1 --2- 172.21.15.5:0/2849657706 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f01d8107680 0x7f01d8107a80 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:27:43.841 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:42.691+0000 7f01dce00640 1 -- 172.21.15.5:0/2849657706 >> 172.21.15.5:0/2849657706 conn(0x7f01d8102eb0 msgr2=0x7f01d81052f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:27:43.841 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:42.691+0000 7f01dce00640 1 -- 172.21.15.5:0/2849657706 shutdown_connections 2024-09-10T09:27:43.841 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:42.691+0000 7f01dce00640 1 -- 172.21.15.5:0/2849657706 wait complete. 2024-09-10T09:27:43.841 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:42.692+0000 7f01dce00640 1 Processor -- start 2024-09-10T09:27:43.841 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:42.692+0000 7f01dce00640 1 -- start start 2024-09-10T09:27:43.841 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:42.693+0000 7f01dce00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f01d8107680 0x7f01d807db70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:27:43.841 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:42.693+0000 7f01dce00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f01d810baf0 con 0x7f01d8107680 2024-09-10T09:27:43.841 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:42.693+0000 7f01d5e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f01d8107680 0x7f01d807db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:27:43.841 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:42.693+0000 7f01d5e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f01d8107680 0x7f01d807db70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51246/0 (socket says 172.21.15.5:51246) 2024-09-10T09:27:43.841 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:42.693+0000 7f01d5e00640 1 -- 172.21.15.5:0/3765847602 learned_addr learned my addr 172.21.15.5:0/3765847602 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:27:43.842 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:42.693+0000 7f01d5e00640 1 -- 172.21.15.5:0/3765847602 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01d807e0b0 con 0x7f01d8107680 2024-09-10T09:27:43.842 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:42.693+0000 7f01d5e00640 1 --2- 172.21.15.5:0/3765847602 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f01d8107680 0x7f01d807db70 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f01c80097f0 tx=0x7f01c8030ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:27:43.842 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:42.694+0000 7f01cea00640 1 -- 172.21.15.5:0/3765847602 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f01c8042070 con 0x7f01d8107680 2024-09-10T09:27:43.842 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:42.694+0000 7f01cea00640 1 -- 172.21.15.5:0/3765847602 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f01c8032ca0 con 0x7f01d8107680 2024-09-10T09:27:43.842 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:42.694+0000 7f01cea00640 1 -- 172.21.15.5:0/3765847602 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f01c8037040 con 0x7f01d8107680 2024-09-10T09:27:43.842 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:42.694+0000 7f01dce00640 1 -- 172.21.15.5:0/3765847602 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01d807c2a0 con 0x7f01d8107680 2024-09-10T09:27:43.842 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:42.694+0000 7f01dce00640 1 -- 172.21.15.5:0/3765847602 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01d807c7e0 con 0x7f01d8107680 2024-09-10T09:27:43.842 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:42.695+0000 7f01cea00640 1 -- 172.21.15.5:0/3765847602 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f01c8045070 con 0x7f01d8107680 2024-09-10T09:27:43.842 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:42.695+0000 7f01dce00640 1 -- 172.21.15.5:0/3765847602 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f01d8107b00 con 0x7f01d8107680 2024-09-10T09:27:43.842 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:42.695+0000 7f01cea00640 1 --2- 172.21.15.5:0/3765847602 >> [v2:172.21.15.5:6800/1527422686,v1:172.21.15.5:6801/1527422686] conn(0x7f01bc038fc0 0x7f01bc03b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:27:43.842 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:42.695+0000 7f01cea00640 1 -- 172.21.15.5:0/3765847602 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f01c806bff0 con 0x7f01d8107680 2024-09-10T09:27:43.842 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:42.696+0000 7f01d5400640 1 --2- 172.21.15.5:0/3765847602 >> [v2:172.21.15.5:6800/1527422686,v1:172.21.15.5:6801/1527422686] conn(0x7f01bc038fc0 0x7f01bc03b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:27:43.842 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:42.696+0000 7f01d5400640 1 --2- 172.21.15.5:0/3765847602 >> [v2:172.21.15.5:6800/1527422686,v1:172.21.15.5:6801/1527422686] conn(0x7f01bc038fc0 0x7f01bc03b480 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f01c40098c0 tx=0x7f01c4006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:27:43.843 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:42.700+0000 7f01cea00640 1 -- 172.21.15.5:0/3765847602 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f01c806b940 con 0x7f01d8107680 2024-09-10T09:27:43.843 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:42.888+0000 7f01dce00640 1 -- 172.21.15.5:0/3765847602 --> [v2:172.21.15.5:6800/1527422686,v1:172.21.15.5:6801/1527422686] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) -- 0x7f01d807d130 con 0x7f01bc038fc0 2024-09-10T09:27:43.843 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:43.198+0000 7f01cea00640 1 -- 172.21.15.5:0/3765847602 <== mgr.14118 v2:172.21.15.5:6800/1527422686 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f01d807d130 con 0x7f01bc038fc0 2024-09-10T09:27:43.843 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:43.200+0000 7f01dce00640 1 -- 172.21.15.5:0/3765847602 >> [v2:172.21.15.5:6800/1527422686,v1:172.21.15.5:6801/1527422686] conn(0x7f01bc038fc0 msgr2=0x7f01bc03b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:27:43.843 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:43.201+0000 7f01dce00640 1 --2- 172.21.15.5:0/3765847602 >> [v2:172.21.15.5:6800/1527422686,v1:172.21.15.5:6801/1527422686] conn(0x7f01bc038fc0 0x7f01bc03b480 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f01c40098c0 tx=0x7f01c4006db0 comp rx=0 tx=0).stop 2024-09-10T09:27:43.843 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:43.201+0000 7f01dce00640 1 -- 172.21.15.5:0/3765847602 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f01d8107680 msgr2=0x7f01d807db70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:27:43.843 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:43.201+0000 7f01dce00640 1 --2- 172.21.15.5:0/3765847602 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f01d8107680 0x7f01d807db70 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f01c80097f0 tx=0x7f01c8030ec0 comp rx=0 tx=0).stop 2024-09-10T09:27:43.843 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:43.201+0000 7f01dce00640 1 -- 172.21.15.5:0/3765847602 shutdown_connections 2024-09-10T09:27:43.843 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:43.201+0000 7f01dce00640 1 --2- 172.21.15.5:0/3765847602 >> [v2:172.21.15.5:6800/1527422686,v1:172.21.15.5:6801/1527422686] conn(0x7f01bc038fc0 0x7f01bc03b480 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:27:43.843 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:43.201+0000 7f01dce00640 1 --2- 172.21.15.5:0/3765847602 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f01d8107680 0x7f01d807db70 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:27:43.843 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:43.201+0000 7f01dce00640 1 -- 172.21.15.5:0/3765847602 >> 172.21.15.5:0/3765847602 conn(0x7f01d8102eb0 msgr2=0x7f01d81039f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:27:43.844 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:43.201+0000 7f01dce00640 1 -- 172.21.15.5:0/3765847602 shutdown_connections 2024-09-10T09:27:43.844 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:43.201+0000 7f01dce00640 1 -- 172.21.15.5:0/3765847602 wait complete. 2024-09-10T09:27:44.510 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:44 smithi005 ceph-mon[30350]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T09:27:44.510 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:44 smithi005 ceph-mon[30350]: Generating ssh key... 2024-09-10T09:27:44.510 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:44 smithi005 ceph-mon[30350]: from='mgr.14118 172.21.15.5:0/1435659661' entity='mgr.x' 2024-09-10T09:27:44.510 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:44 smithi005 ceph-mon[30350]: from='mgr.14118 172.21.15.5:0/1435659661' entity='mgr.x' 2024-09-10T09:27:45.009 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQD3kMvhYY/c4Qg+wIxyzdtZdZXPJOF45PYPNYa0JHRy7azcDpeNQRYqM52JifHuAGXk5BPEviPo/KtpoByh8e68S0RPaFm8199+lYksO823Rh1qDYeCVSVCb2embg/NzX1Z5IFQ/hyZBUY3fes5iGEwiBQCq3C+98h0dZkfC7knZdLNLA0P+XPImJSPIfA4LnhTc6Q8yVHdU3+ta4O6wIIyoqBKpCCE8ClLcr9ZRYjHU57f6QYFGQKYjRwsXltvrwE+dpxYA3dvc93n0k2u5nKc0qQdY4BQMNjkjYQqguhzVZlT7mVI8ommP2iUNaw9UBEby+9oo2zF45B1Ky9JLpFBVMletw7w9bJHeX5y6+TfJBNSJWpVksUpntX7ylMNZXUB5jGCUEZNn7B+ZDmSGf8gil5A2sqezcgFjIaUXetcToG/ulyEvPLVfuuxStw2O3pWj2KB0KCnLrSp9viwxiYAiKpW+YqHtNlDvD47Na54ZUjh1oJARIe3zjGYLWbYq0E= ceph-bade4b50-6f56-11ef-bcea-c7b262605968 2024-09-10T09:27:45.009 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:44.308+0000 7f7dc6c00640 1 Processor -- start 2024-09-10T09:27:45.009 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:44.308+0000 7f7dc6c00640 1 -- start start 2024-09-10T09:27:45.009 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:44.308+0000 7f7dc6c00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7dc0107680 0x7f7dc0107a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:27:45.009 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:44.308+0000 7f7dc6c00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f7dc0107fc0 con 0x7f7dc0107680 2024-09-10T09:27:45.009 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:44.309+0000 7f7dbfe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7dc0107680 0x7f7dc0107a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:27:45.009 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:44.309+0000 7f7dbfe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7dc0107680 0x7f7dc0107a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51248/0 (socket says 172.21.15.5:51248) 2024-09-10T09:27:45.010 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:44.309+0000 7f7dbfe00640 1 -- 172.21.15.5:0/408021542 learned_addr learned my addr 172.21.15.5:0/408021542 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:27:45.010 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:44.309+0000 7f7dbfe00640 1 -- 172.21.15.5:0/408021542 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7dc0108140 con 0x7f7dc0107680 2024-09-10T09:27:45.010 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:44.310+0000 7f7dbfe00640 1 --2- 172.21.15.5:0/408021542 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7dc0107680 0x7f7dc0107a80 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f7db0009820 tx=0x7f7db002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5f8e538174145bf server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:27:45.010 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:44.310+0000 7f7dbea00640 1 -- 172.21.15.5:0/408021542 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7db0037070 con 0x7f7dc0107680 2024-09-10T09:27:45.010 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:44.310+0000 7f7dbea00640 1 -- 172.21.15.5:0/408021542 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f7db0032900 con 0x7f7dc0107680 2024-09-10T09:27:45.010 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:44.310+0000 7f7dbea00640 1 -- 172.21.15.5:0/408021542 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7db0030850 con 0x7f7dc0107680 2024-09-10T09:27:45.010 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:44.311+0000 7f7dc6c00640 1 -- 172.21.15.5:0/408021542 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7dc0107680 msgr2=0x7f7dc0107a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:27:45.010 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:44.311+0000 7f7dc6c00640 1 --2- 172.21.15.5:0/408021542 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7dc0107680 0x7f7dc0107a80 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f7db0009820 tx=0x7f7db002a270 comp rx=0 tx=0).stop 2024-09-10T09:27:45.010 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:44.311+0000 7f7dc6c00640 1 -- 172.21.15.5:0/408021542 shutdown_connections 2024-09-10T09:27:45.010 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:44.311+0000 7f7dc6c00640 1 --2- 172.21.15.5:0/408021542 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7dc0107680 0x7f7dc0107a80 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:27:45.010 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:44.311+0000 7f7dc6c00640 1 -- 172.21.15.5:0/408021542 >> 172.21.15.5:0/408021542 conn(0x7f7dc0102eb0 msgr2=0x7f7dc01052f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:27:45.010 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:44.311+0000 7f7dc6c00640 1 -- 172.21.15.5:0/408021542 shutdown_connections 2024-09-10T09:27:45.011 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:44.311+0000 7f7dc6c00640 1 -- 172.21.15.5:0/408021542 wait complete. 2024-09-10T09:27:45.011 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:44.312+0000 7f7dc6c00640 1 Processor -- start 2024-09-10T09:27:45.011 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:44.312+0000 7f7dc6c00640 1 -- start start 2024-09-10T09:27:45.011 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:44.313+0000 7f7dc6c00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7dc0107680 0x7f7dc019d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:27:45.011 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:44.313+0000 7f7dc6c00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f7dc010baf0 con 0x7f7dc0107680 2024-09-10T09:27:45.011 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:44.313+0000 7f7dbfe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7dc0107680 0x7f7dc019d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:27:45.011 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:44.313+0000 7f7dbfe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7dc0107680 0x7f7dc019d8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51250/0 (socket says 172.21.15.5:51250) 2024-09-10T09:27:45.011 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:44.313+0000 7f7dbfe00640 1 -- 172.21.15.5:0/2564554269 learned_addr learned my addr 172.21.15.5:0/2564554269 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:27:45.011 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:44.313+0000 7f7dbfe00640 1 -- 172.21.15.5:0/2564554269 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7dc019dde0 con 0x7f7dc0107680 2024-09-10T09:27:45.011 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:44.313+0000 7f7dbfe00640 1 --2- 172.21.15.5:0/2564554269 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7dc0107680 0x7f7dc019d8a0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f7db0031000 tx=0x7f7db00326e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:27:45.011 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:44.314+0000 7f7dbcc00640 1 -- 172.21.15.5:0/2564554269 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7db0042070 con 0x7f7dc0107680 2024-09-10T09:27:45.011 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:44.314+0000 7f7dbcc00640 1 -- 172.21.15.5:0/2564554269 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f7db0032be0 con 0x7f7dc0107680 2024-09-10T09:27:45.011 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:44.314+0000 7f7dc6c00640 1 -- 172.21.15.5:0/2564554269 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7dc019e070 con 0x7f7dc0107680 2024-09-10T09:27:45.011 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:44.314+0000 7f7dbcc00640 1 -- 172.21.15.5:0/2564554269 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7db0037040 con 0x7f7dc0107680 2024-09-10T09:27:45.012 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:44.314+0000 7f7dc6c00640 1 -- 172.21.15.5:0/2564554269 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7dc01a0d60 con 0x7f7dc0107680 2024-09-10T09:27:45.012 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:44.315+0000 7f7dbcc00640 1 -- 172.21.15.5:0/2564554269 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f7db0045070 con 0x7f7dc0107680 2024-09-10T09:27:45.012 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:44.315+0000 7f7dc6c00640 1 -- 172.21.15.5:0/2564554269 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7dc0107b00 con 0x7f7dc0107680 2024-09-10T09:27:45.012 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:44.316+0000 7f7dbcc00640 1 --2- 172.21.15.5:0/2564554269 >> [v2:172.21.15.5:6800/1527422686,v1:172.21.15.5:6801/1527422686] conn(0x7f7da4038fc0 0x7f7da403b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:27:45.012 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:44.316+0000 7f7dbcc00640 1 -- 172.21.15.5:0/2564554269 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f7db006bff0 con 0x7f7dc0107680 2024-09-10T09:27:45.012 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:44.316+0000 7f7dbf400640 1 --2- 172.21.15.5:0/2564554269 >> [v2:172.21.15.5:6800/1527422686,v1:172.21.15.5:6801/1527422686] conn(0x7f7da4038fc0 0x7f7da403b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:27:45.012 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:44.317+0000 7f7dbf400640 1 --2- 172.21.15.5:0/2564554269 >> [v2:172.21.15.5:6800/1527422686,v1:172.21.15.5:6801/1527422686] conn(0x7f7da4038fc0 0x7f7da403b480 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f7dac0098c0 tx=0x7f7dac006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:27:45.012 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:44.321+0000 7f7dbcc00640 1 -- 172.21.15.5:0/2564554269 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f7db002f2a0 con 0x7f7dc0107680 2024-09-10T09:27:45.012 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:44.507+0000 7f7dc6c00640 1 -- 172.21.15.5:0/2564554269 --> [v2:172.21.15.5:6800/1527422686,v1:172.21.15.5:6801/1527422686] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) -- 0x7f7dc01040c0 con 0x7f7da4038fc0 2024-09-10T09:27:45.012 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:44.508+0000 7f7dbcc00640 1 -- 172.21.15.5:0/2564554269 <== mgr.14118 v2:172.21.15.5:6800/1527422686 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+595 (secure 0 0 0) 0x7f7dc01040c0 con 0x7f7da4038fc0 2024-09-10T09:27:45.012 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:44.511+0000 7f7dc6c00640 1 -- 172.21.15.5:0/2564554269 >> [v2:172.21.15.5:6800/1527422686,v1:172.21.15.5:6801/1527422686] conn(0x7f7da4038fc0 msgr2=0x7f7da403b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:27:45.012 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:44.512+0000 7f7dc6c00640 1 --2- 172.21.15.5:0/2564554269 >> [v2:172.21.15.5:6800/1527422686,v1:172.21.15.5:6801/1527422686] conn(0x7f7da4038fc0 0x7f7da403b480 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f7dac0098c0 tx=0x7f7dac006db0 comp rx=0 tx=0).stop 2024-09-10T09:27:45.013 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:44.512+0000 7f7dc6c00640 1 -- 172.21.15.5:0/2564554269 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7dc0107680 msgr2=0x7f7dc019d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:27:45.013 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:44.512+0000 7f7dc6c00640 1 --2- 172.21.15.5:0/2564554269 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7dc0107680 0x7f7dc019d8a0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f7db0031000 tx=0x7f7db00326e0 comp rx=0 tx=0).stop 2024-09-10T09:27:45.013 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:44.512+0000 7f7dc6c00640 1 -- 172.21.15.5:0/2564554269 shutdown_connections 2024-09-10T09:27:45.013 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:44.512+0000 7f7dc6c00640 1 --2- 172.21.15.5:0/2564554269 >> [v2:172.21.15.5:6800/1527422686,v1:172.21.15.5:6801/1527422686] conn(0x7f7da4038fc0 0x7f7da403b480 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:27:45.013 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:44.512+0000 7f7dc6c00640 1 --2- 172.21.15.5:0/2564554269 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7dc0107680 0x7f7dc019d8a0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:27:45.013 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:44.512+0000 7f7dc6c00640 1 -- 172.21.15.5:0/2564554269 >> 172.21.15.5:0/2564554269 conn(0x7f7dc0102eb0 msgr2=0x7f7dc01039b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:27:45.013 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:44.512+0000 7f7dc6c00640 1 -- 172.21.15.5:0/2564554269 shutdown_connections 2024-09-10T09:27:45.013 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:44.512+0000 7f7dc6c00640 1 -- 172.21.15.5:0/2564554269 wait complete. 2024-09-10T09:27:45.013 INFO:teuthology.orchestra.run.smithi005.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-09-10T09:27:45.013 INFO:teuthology.orchestra.run.smithi005.stdout:Adding key to root@localhost authorized_keys... 2024-09-10T09:27:45.017 INFO:teuthology.orchestra.run.smithi005.stdout:Adding host smithi005... 2024-09-10T09:27:45.451 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:45 smithi005 ceph-mon[30350]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T09:27:46.306 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:46 smithi005 ceph-mon[30350]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi005", "addr": "172.21.15.5", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T09:27:48.003 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:47 smithi005 ceph-mon[30350]: Deploying cephadm binary to smithi005 2024-09-10T09:27:50.985 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout Added host 'smithi005' with addr '172.21.15.5' 2024-09-10T09:27:50.985 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:45.560+0000 7fdcbd000640 1 Processor -- start 2024-09-10T09:27:50.985 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:45.561+0000 7fdcbd000640 1 -- start start 2024-09-10T09:27:50.985 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:45.561+0000 7fdcbd000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdcb8107680 0x7fdcb8107a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:27:50.985 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:45.561+0000 7fdcbd000640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fdcb8107fc0 con 0x7fdcb8107680 2024-09-10T09:27:50.985 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:45.561+0000 7fdcb5e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdcb8107680 0x7fdcb8107a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:27:50.985 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:45.561+0000 7fdcb5e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdcb8107680 0x7fdcb8107a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51256/0 (socket says 172.21.15.5:51256) 2024-09-10T09:27:50.985 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:45.561+0000 7fdcb5e00640 1 -- 172.21.15.5:0/4181361197 learned_addr learned my addr 172.21.15.5:0/4181361197 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:27:50.985 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:45.561+0000 7fdcb5e00640 1 -- 172.21.15.5:0/4181361197 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdcb8108140 con 0x7fdcb8107680 2024-09-10T09:27:50.985 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:45.562+0000 7fdcb5e00640 1 --2- 172.21.15.5:0/4181361197 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdcb8107680 0x7fdcb8107a80 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fdca8009820 tx=0x7fdca802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6874a45de1fc098 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:27:50.986 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:45.563+0000 7fdcb5400640 1 -- 172.21.15.5:0/4181361197 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdca8037070 con 0x7fdcb8107680 2024-09-10T09:27:50.986 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:45.563+0000 7fdcb5400640 1 -- 172.21.15.5:0/4181361197 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fdca8032900 con 0x7fdcb8107680 2024-09-10T09:27:50.986 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:45.563+0000 7fdcb5400640 1 -- 172.21.15.5:0/4181361197 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdca8030850 con 0x7fdcb8107680 2024-09-10T09:27:50.986 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:45.563+0000 7fdcbd000640 1 -- 172.21.15.5:0/4181361197 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdcb8107680 msgr2=0x7fdcb8107a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:27:50.986 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:45.563+0000 7fdcbd000640 1 --2- 172.21.15.5:0/4181361197 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdcb8107680 0x7fdcb8107a80 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fdca8009820 tx=0x7fdca802a270 comp rx=0 tx=0).stop 2024-09-10T09:27:50.986 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:45.563+0000 7fdcbd000640 1 -- 172.21.15.5:0/4181361197 shutdown_connections 2024-09-10T09:27:50.986 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:45.563+0000 7fdcbd000640 1 --2- 172.21.15.5:0/4181361197 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdcb8107680 0x7fdcb8107a80 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:27:50.986 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:45.563+0000 7fdcbd000640 1 -- 172.21.15.5:0/4181361197 >> 172.21.15.5:0/4181361197 conn(0x7fdcb8102eb0 msgr2=0x7fdcb81052f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:27:50.986 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:45.564+0000 7fdcbd000640 1 -- 172.21.15.5:0/4181361197 shutdown_connections 2024-09-10T09:27:50.986 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:45.564+0000 7fdcbd000640 1 -- 172.21.15.5:0/4181361197 wait complete. 2024-09-10T09:27:50.986 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:45.564+0000 7fdcbd000640 1 Processor -- start 2024-09-10T09:27:50.986 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:45.565+0000 7fdcbd000640 1 -- start start 2024-09-10T09:27:50.986 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:45.565+0000 7fdcbd000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdcb8107680 0x7fdcb819b640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:27:50.987 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:45.565+0000 7fdcbd000640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fdcb810baf0 con 0x7fdcb8107680 2024-09-10T09:27:50.987 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:45.565+0000 7fdcb5e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdcb8107680 0x7fdcb819b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:27:50.987 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:45.565+0000 7fdcb5e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdcb8107680 0x7fdcb819b640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51258/0 (socket says 172.21.15.5:51258) 2024-09-10T09:27:50.987 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:45.565+0000 7fdcb5e00640 1 -- 172.21.15.5:0/332697255 learned_addr learned my addr 172.21.15.5:0/332697255 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:27:50.987 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:45.566+0000 7fdcb5e00640 1 -- 172.21.15.5:0/332697255 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdcb819bb80 con 0x7fdcb8107680 2024-09-10T09:27:50.987 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:45.566+0000 7fdcb5e00640 1 --2- 172.21.15.5:0/332697255 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdcb8107680 0x7fdcb819b640 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fdca80097f0 tx=0x7fdca8030ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:27:50.987 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:45.566+0000 7fdcaea00640 1 -- 172.21.15.5:0/332697255 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdca8042070 con 0x7fdcb8107680 2024-09-10T09:27:50.987 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:45.566+0000 7fdcaea00640 1 -- 172.21.15.5:0/332697255 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fdca8032ca0 con 0x7fdcb8107680 2024-09-10T09:27:50.987 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:45.566+0000 7fdcaea00640 1 -- 172.21.15.5:0/332697255 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdca8037040 con 0x7fdcb8107680 2024-09-10T09:27:50.987 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:45.566+0000 7fdcbd000640 1 -- 172.21.15.5:0/332697255 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdcb819be10 con 0x7fdcb8107680 2024-09-10T09:27:50.987 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:45.566+0000 7fdcbd000640 1 -- 172.21.15.5:0/332697255 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdcb819eb00 con 0x7fdcb8107680 2024-09-10T09:27:50.987 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:45.567+0000 7fdcaea00640 1 -- 172.21.15.5:0/332697255 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7fdca8045070 con 0x7fdcb8107680 2024-09-10T09:27:50.987 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:45.567+0000 7fdcbd000640 1 -- 172.21.15.5:0/332697255 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdcb8107b00 con 0x7fdcb8107680 2024-09-10T09:27:50.987 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:45.568+0000 7fdcaea00640 1 --2- 172.21.15.5:0/332697255 >> [v2:172.21.15.5:6800/1527422686,v1:172.21.15.5:6801/1527422686] conn(0x7fdc9c038f70 0x7fdc9c03b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:27:50.988 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:45.568+0000 7fdcaea00640 1 -- 172.21.15.5:0/332697255 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fdca806c040 con 0x7fdcb8107680 2024-09-10T09:27:50.988 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:45.568+0000 7fdcafe00640 1 --2- 172.21.15.5:0/332697255 >> [v2:172.21.15.5:6800/1527422686,v1:172.21.15.5:6801/1527422686] conn(0x7fdc9c038f70 0x7fdc9c03b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:27:50.988 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:45.569+0000 7fdcafe00640 1 --2- 172.21.15.5:0/332697255 >> [v2:172.21.15.5:6800/1527422686,v1:172.21.15.5:6801/1527422686] conn(0x7fdc9c038f70 0x7fdc9c03b430 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fdca00098c0 tx=0x7fdca0006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:27:50.988 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:45.573+0000 7fdcaea00640 1 -- 172.21.15.5:0/332697255 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fdca8045360 con 0x7fdcb8107680 2024-09-10T09:27:50.988 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:45.761+0000 7fdcbd000640 1 -- 172.21.15.5:0/332697255 --> [v2:172.21.15.5:6800/1527422686,v1:172.21.15.5:6801/1527422686] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi005", "addr": "172.21.15.5", "target": ["mon-mgr", ""]}) -- 0x7fdcb8104100 con 0x7fdc9c038f70 2024-09-10T09:27:50.988 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:50.249+0000 7fdcaea00640 1 -- 172.21.15.5:0/332697255 <== mgr.14118 v2:172.21.15.5:6800/1527422686 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+47 (secure 0 0 0) 0x7fdcb8104100 con 0x7fdc9c038f70 2024-09-10T09:27:50.988 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:50.252+0000 7fdcbd000640 1 -- 172.21.15.5:0/332697255 >> [v2:172.21.15.5:6800/1527422686,v1:172.21.15.5:6801/1527422686] conn(0x7fdc9c038f70 msgr2=0x7fdc9c03b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:27:50.988 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:50.252+0000 7fdcbd000640 1 --2- 172.21.15.5:0/332697255 >> [v2:172.21.15.5:6800/1527422686,v1:172.21.15.5:6801/1527422686] conn(0x7fdc9c038f70 0x7fdc9c03b430 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fdca00098c0 tx=0x7fdca0006db0 comp rx=0 tx=0).stop 2024-09-10T09:27:50.988 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:50.252+0000 7fdcbd000640 1 -- 172.21.15.5:0/332697255 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdcb8107680 msgr2=0x7fdcb819b640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:27:50.988 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:50.252+0000 7fdcbd000640 1 --2- 172.21.15.5:0/332697255 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdcb8107680 0x7fdcb819b640 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fdca80097f0 tx=0x7fdca8030ec0 comp rx=0 tx=0).stop 2024-09-10T09:27:50.988 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:50.252+0000 7fdcbd000640 1 -- 172.21.15.5:0/332697255 shutdown_connections 2024-09-10T09:27:50.988 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:50.252+0000 7fdcbd000640 1 --2- 172.21.15.5:0/332697255 >> [v2:172.21.15.5:6800/1527422686,v1:172.21.15.5:6801/1527422686] conn(0x7fdc9c038f70 0x7fdc9c03b430 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:27:50.988 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:50.252+0000 7fdcbd000640 1 --2- 172.21.15.5:0/332697255 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdcb8107680 0x7fdcb819b640 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:27:50.988 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:50.252+0000 7fdcbd000640 1 -- 172.21.15.5:0/332697255 >> 172.21.15.5:0/332697255 conn(0x7fdcb8102eb0 msgr2=0x7fdcb81039f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:27:50.989 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:50.252+0000 7fdcbd000640 1 -- 172.21.15.5:0/332697255 shutdown_connections 2024-09-10T09:27:50.989 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:50.252+0000 7fdcbd000640 1 -- 172.21.15.5:0/332697255 wait complete. 2024-09-10T09:27:50.989 INFO:teuthology.orchestra.run.smithi005.stdout:Deploying unmanaged mon service... 2024-09-10T09:27:51.518 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:51 smithi005 ceph-mon[30350]: from='mgr.14118 172.21.15.5:0/1435659661' entity='mgr.x' 2024-09-10T09:27:51.518 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:51 smithi005 ceph-mon[30350]: Added host smithi005 2024-09-10T09:27:51.518 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:51 smithi005 ceph-mon[30350]: from='mgr.14118 172.21.15.5:0/1435659661' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:27:52.746 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-09-10T09:27:52.747 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:51.516+0000 7fc69fe00640 1 Processor -- start 2024-09-10T09:27:52.747 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:51.516+0000 7fc69fe00640 1 -- start start 2024-09-10T09:27:52.747 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:51.516+0000 7fc69fe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc6a007dee0 0x7fc6a007c340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:27:52.747 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:51.516+0000 7fc69fe00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc6a007c910 con 0x7fc6a007dee0 2024-09-10T09:27:52.747 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:51.517+0000 7fc69ea00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc6a007dee0 0x7fc6a007c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:27:52.747 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:51.517+0000 7fc69ea00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc6a007dee0 0x7fc6a007c340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40836/0 (socket says 172.21.15.5:40836) 2024-09-10T09:27:52.747 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:51.517+0000 7fc69ea00640 1 -- 172.21.15.5:0/694303546 learned_addr learned my addr 172.21.15.5:0/694303546 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:27:52.747 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:51.517+0000 7fc69ea00640 1 -- 172.21.15.5:0/694303546 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6a007ca90 con 0x7fc6a007dee0 2024-09-10T09:27:52.747 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:51.518+0000 7fc69ea00640 1 --2- 172.21.15.5:0/694303546 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc6a007dee0 0x7fc6a007c340 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fc690009820 tx=0x7fc69002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ee444479ceeb148 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:27:52.748 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:51.518+0000 7fc69d600640 1 -- 172.21.15.5:0/694303546 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc690037070 con 0x7fc6a007dee0 2024-09-10T09:27:52.748 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:51.519+0000 7fc69d600640 1 -- 172.21.15.5:0/694303546 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fc690032900 con 0x7fc6a007dee0 2024-09-10T09:27:52.748 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:51.519+0000 7fc69d600640 1 -- 172.21.15.5:0/694303546 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc690030850 con 0x7fc6a007dee0 2024-09-10T09:27:52.748 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:51.519+0000 7fc69fe00640 1 -- 172.21.15.5:0/694303546 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc6a007dee0 msgr2=0x7fc6a007c340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:27:52.748 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:51.519+0000 7fc69fe00640 1 --2- 172.21.15.5:0/694303546 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc6a007dee0 0x7fc6a007c340 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fc690009820 tx=0x7fc69002a270 comp rx=0 tx=0).stop 2024-09-10T09:27:52.748 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:51.519+0000 7fc69fe00640 1 -- 172.21.15.5:0/694303546 shutdown_connections 2024-09-10T09:27:52.748 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:51.519+0000 7fc69fe00640 1 --2- 172.21.15.5:0/694303546 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc6a007dee0 0x7fc6a007c340 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:27:52.748 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:51.519+0000 7fc69fe00640 1 -- 172.21.15.5:0/694303546 >> 172.21.15.5:0/694303546 conn(0x7fc6a0102c10 msgr2=0x7fc6a0105030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:27:52.748 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:51.519+0000 7fc69fe00640 1 -- 172.21.15.5:0/694303546 shutdown_connections 2024-09-10T09:27:52.748 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:51.519+0000 7fc69fe00640 1 -- 172.21.15.5:0/694303546 wait complete. 2024-09-10T09:27:52.749 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:51.520+0000 7fc69fe00640 1 Processor -- start 2024-09-10T09:27:52.749 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:51.520+0000 7fc69fe00640 1 -- start start 2024-09-10T09:27:52.749 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:51.521+0000 7fc69fe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc6a007dee0 0x7fc6a019d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:27:52.749 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:51.521+0000 7fc69fe00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc6a010da70 con 0x7fc6a007dee0 2024-09-10T09:27:52.749 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:51.521+0000 7fc69ea00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc6a007dee0 0x7fc6a019d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:27:52.749 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:51.521+0000 7fc69ea00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc6a007dee0 0x7fc6a019d6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40844/0 (socket says 172.21.15.5:40844) 2024-09-10T09:27:52.749 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:51.521+0000 7fc69ea00640 1 -- 172.21.15.5:0/1539301185 learned_addr learned my addr 172.21.15.5:0/1539301185 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:27:52.749 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:51.521+0000 7fc69ea00640 1 -- 172.21.15.5:0/1539301185 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6a019dc10 con 0x7fc6a007dee0 2024-09-10T09:27:52.749 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:51.522+0000 7fc69ea00640 1 --2- 172.21.15.5:0/1539301185 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc6a007dee0 0x7fc6a019d6d0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fc690009940 tx=0x7fc6900326e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:27:52.749 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:51.522+0000 7fc697400640 1 -- 172.21.15.5:0/1539301185 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc690042070 con 0x7fc6a007dee0 2024-09-10T09:27:52.749 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:51.522+0000 7fc697400640 1 -- 172.21.15.5:0/1539301185 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fc690032be0 con 0x7fc6a007dee0 2024-09-10T09:27:52.750 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:51.522+0000 7fc69fe00640 1 -- 172.21.15.5:0/1539301185 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6a019dea0 con 0x7fc6a007dee0 2024-09-10T09:27:52.750 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:51.522+0000 7fc697400640 1 -- 172.21.15.5:0/1539301185 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc690037040 con 0x7fc6a007dee0 2024-09-10T09:27:52.750 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:51.522+0000 7fc69fe00640 1 -- 172.21.15.5:0/1539301185 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6a01a0b90 con 0x7fc6a007dee0 2024-09-10T09:27:52.750 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:51.523+0000 7fc697400640 1 -- 172.21.15.5:0/1539301185 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7fc690045070 con 0x7fc6a007dee0 2024-09-10T09:27:52.750 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:51.523+0000 7fc69fe00640 1 -- 172.21.15.5:0/1539301185 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc6a007c3c0 con 0x7fc6a007dee0 2024-09-10T09:27:52.750 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:51.524+0000 7fc697400640 1 --2- 172.21.15.5:0/1539301185 >> [v2:172.21.15.5:6800/1527422686,v1:172.21.15.5:6801/1527422686] conn(0x7fc684038f70 0x7fc68403b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:27:52.750 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:51.524+0000 7fc697400640 1 -- 172.21.15.5:0/1539301185 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fc69006bfb0 con 0x7fc6a007dee0 2024-09-10T09:27:52.750 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:51.524+0000 7fc69e000640 1 --2- 172.21.15.5:0/1539301185 >> [v2:172.21.15.5:6800/1527422686,v1:172.21.15.5:6801/1527422686] conn(0x7fc684038f70 0x7fc68403b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:27:52.750 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:51.525+0000 7fc69e000640 1 --2- 172.21.15.5:0/1539301185 >> [v2:172.21.15.5:6800/1527422686,v1:172.21.15.5:6801/1527422686] conn(0x7fc684038f70 0x7fc68403b430 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fc68c0098c0 tx=0x7fc68c006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:27:52.750 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:51.529+0000 7fc697400640 1 -- 172.21.15.5:0/1539301185 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc69006b900 con 0x7fc6a007dee0 2024-09-10T09:27:52.751 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:51.735+0000 7fc69fe00640 1 -- 172.21.15.5:0/1539301185 --> [v2:172.21.15.5:6800/1527422686,v1:172.21.15.5:6801/1527422686] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7fc6a0103ef0 con 0x7fc684038f70 2024-09-10T09:27:52.751 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:51.740+0000 7fc697400640 1 -- 172.21.15.5:0/1539301185 <== mgr.14118 v2:172.21.15.5:6800/1527422686 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fc6a0103ef0 con 0x7fc684038f70 2024-09-10T09:27:52.751 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:51.742+0000 7fc69fe00640 1 -- 172.21.15.5:0/1539301185 >> [v2:172.21.15.5:6800/1527422686,v1:172.21.15.5:6801/1527422686] conn(0x7fc684038f70 msgr2=0x7fc68403b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:27:52.751 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:51.742+0000 7fc69fe00640 1 --2- 172.21.15.5:0/1539301185 >> [v2:172.21.15.5:6800/1527422686,v1:172.21.15.5:6801/1527422686] conn(0x7fc684038f70 0x7fc68403b430 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fc68c0098c0 tx=0x7fc68c006db0 comp rx=0 tx=0).stop 2024-09-10T09:27:52.751 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:51.742+0000 7fc69fe00640 1 -- 172.21.15.5:0/1539301185 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc6a007dee0 msgr2=0x7fc6a019d6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:27:52.751 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:51.742+0000 7fc69fe00640 1 --2- 172.21.15.5:0/1539301185 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc6a007dee0 0x7fc6a019d6d0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fc690009940 tx=0x7fc6900326e0 comp rx=0 tx=0).stop 2024-09-10T09:27:52.751 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:51.742+0000 7fc69fe00640 1 -- 172.21.15.5:0/1539301185 shutdown_connections 2024-09-10T09:27:52.751 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:51.742+0000 7fc69fe00640 1 --2- 172.21.15.5:0/1539301185 >> [v2:172.21.15.5:6800/1527422686,v1:172.21.15.5:6801/1527422686] conn(0x7fc684038f70 0x7fc68403b430 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:27:52.751 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:51.742+0000 7fc69fe00640 1 --2- 172.21.15.5:0/1539301185 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc6a007dee0 0x7fc6a019d6d0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:27:52.751 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:51.742+0000 7fc69fe00640 1 -- 172.21.15.5:0/1539301185 >> 172.21.15.5:0/1539301185 conn(0x7fc6a0102c10 msgr2=0x7fc6a01037e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:27:52.751 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:51.742+0000 7fc69fe00640 1 -- 172.21.15.5:0/1539301185 shutdown_connections 2024-09-10T09:27:52.752 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:51.743+0000 7fc69fe00640 1 -- 172.21.15.5:0/1539301185 wait complete. 2024-09-10T09:27:52.752 INFO:teuthology.orchestra.run.smithi005.stdout:Deploying unmanaged mgr service... 2024-09-10T09:27:52.883 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:52 smithi005 ceph-mon[30350]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T09:27:52.883 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:52 smithi005 ceph-mon[30350]: Saving service mon spec with placement count:5 2024-09-10T09:27:52.883 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:52 smithi005 ceph-mon[30350]: from='mgr.14118 172.21.15.5:0/1435659661' entity='mgr.x' 2024-09-10T09:27:54.607 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-09-10T09:27:54.608 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:53.409+0000 7f8cd3200640 1 Processor -- start 2024-09-10T09:27:54.608 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:53.409+0000 7f8cd3200640 1 -- start start 2024-09-10T09:27:54.608 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:53.409+0000 7f8cd3200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8ccc07ddb0 0x7f8ccc07c320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:27:54.608 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:53.409+0000 7f8cd3200640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f8ccc07c860 con 0x7f8ccc07ddb0 2024-09-10T09:27:54.608 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:53.410+0000 7f8ccbe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8ccc07ddb0 0x7f8ccc07c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:27:54.608 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:53.410+0000 7f8ccbe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8ccc07ddb0 0x7f8ccc07c320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40858/0 (socket says 172.21.15.5:40858) 2024-09-10T09:27:54.608 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:53.410+0000 7f8ccbe00640 1 -- 172.21.15.5:0/3625754155 learned_addr learned my addr 172.21.15.5:0/3625754155 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:27:54.608 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:53.410+0000 7f8ccbe00640 1 -- 172.21.15.5:0/3625754155 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ccc07c9e0 con 0x7f8ccc07ddb0 2024-09-10T09:27:54.608 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:53.411+0000 7f8ccbe00640 1 --2- 172.21.15.5:0/3625754155 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8ccc07ddb0 0x7f8ccc07c320 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f8cbc009820 tx=0x7f8cbc02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd0db9686bdb84f1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:27:54.608 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:53.411+0000 7f8ccaa00640 1 -- 172.21.15.5:0/3625754155 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8cbc037070 con 0x7f8ccc07ddb0 2024-09-10T09:27:54.608 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:53.412+0000 7f8ccaa00640 1 -- 172.21.15.5:0/3625754155 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f8cbc032900 con 0x7f8ccc07ddb0 2024-09-10T09:27:54.609 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:53.412+0000 7f8ccaa00640 1 -- 172.21.15.5:0/3625754155 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8cbc030850 con 0x7f8ccc07ddb0 2024-09-10T09:27:54.609 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:53.412+0000 7f8cd3200640 1 -- 172.21.15.5:0/3625754155 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8ccc07ddb0 msgr2=0x7f8ccc07c320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:27:54.609 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:53.412+0000 7f8cd3200640 1 --2- 172.21.15.5:0/3625754155 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8ccc07ddb0 0x7f8ccc07c320 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f8cbc009820 tx=0x7f8cbc02a270 comp rx=0 tx=0).stop 2024-09-10T09:27:54.609 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:53.412+0000 7f8cd3200640 1 -- 172.21.15.5:0/3625754155 shutdown_connections 2024-09-10T09:27:54.609 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:53.412+0000 7f8cd3200640 1 --2- 172.21.15.5:0/3625754155 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8ccc07ddb0 0x7f8ccc07c320 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:27:54.609 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:53.412+0000 7f8cd3200640 1 -- 172.21.15.5:0/3625754155 >> 172.21.15.5:0/3625754155 conn(0x7f8ccc102eb0 msgr2=0x7f8ccc1052f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:27:54.609 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:53.412+0000 7f8cd3200640 1 -- 172.21.15.5:0/3625754155 shutdown_connections 2024-09-10T09:27:54.609 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:53.412+0000 7f8cd3200640 1 -- 172.21.15.5:0/3625754155 wait complete. 2024-09-10T09:27:54.609 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:53.413+0000 7f8cd3200640 1 Processor -- start 2024-09-10T09:27:54.609 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:53.413+0000 7f8cd3200640 1 -- start start 2024-09-10T09:27:54.609 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:53.414+0000 7f8cd3200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8ccc07ddb0 0x7f8ccc19d890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:27:54.609 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:53.414+0000 7f8cd3200640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f8ccc10dbe0 con 0x7f8ccc07ddb0 2024-09-10T09:27:54.610 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:53.414+0000 7f8ccbe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8ccc07ddb0 0x7f8ccc19d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:27:54.610 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:53.414+0000 7f8ccbe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8ccc07ddb0 0x7f8ccc19d890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40860/0 (socket says 172.21.15.5:40860) 2024-09-10T09:27:54.610 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:53.414+0000 7f8ccbe00640 1 -- 172.21.15.5:0/428462048 learned_addr learned my addr 172.21.15.5:0/428462048 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:27:54.610 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:53.414+0000 7f8ccbe00640 1 -- 172.21.15.5:0/428462048 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ccc19ddd0 con 0x7f8ccc07ddb0 2024-09-10T09:27:54.610 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:53.414+0000 7f8ccbe00640 1 --2- 172.21.15.5:0/428462048 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8ccc07ddb0 0x7f8ccc19d890 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f8cbc0097f0 tx=0x7f8cbc030ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:27:54.610 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:53.415+0000 7f8cc8c00640 1 -- 172.21.15.5:0/428462048 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8cbc042070 con 0x7f8ccc07ddb0 2024-09-10T09:27:54.610 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:53.415+0000 7f8cc8c00640 1 -- 172.21.15.5:0/428462048 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f8cbc032ca0 con 0x7f8ccc07ddb0 2024-09-10T09:27:54.610 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:53.415+0000 7f8cc8c00640 1 -- 172.21.15.5:0/428462048 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8cbc037040 con 0x7f8ccc07ddb0 2024-09-10T09:27:54.610 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:53.415+0000 7f8cd3200640 1 -- 172.21.15.5:0/428462048 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ccc19e060 con 0x7f8ccc07ddb0 2024-09-10T09:27:54.610 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:53.415+0000 7f8cd3200640 1 -- 172.21.15.5:0/428462048 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ccc1a0d50 con 0x7f8ccc07ddb0 2024-09-10T09:27:54.610 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:53.416+0000 7f8cc8c00640 1 -- 172.21.15.5:0/428462048 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f8cbc045070 con 0x7f8ccc07ddb0 2024-09-10T09:27:54.610 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:53.416+0000 7f8cd3200640 1 -- 172.21.15.5:0/428462048 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ccc07c3a0 con 0x7f8ccc07ddb0 2024-09-10T09:27:54.610 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:53.417+0000 7f8cc8c00640 1 --2- 172.21.15.5:0/428462048 >> [v2:172.21.15.5:6800/1527422686,v1:172.21.15.5:6801/1527422686] conn(0x7f8cb0038fc0 0x7f8cb003b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:27:54.611 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:53.417+0000 7f8cc8c00640 1 -- 172.21.15.5:0/428462048 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f8cbc06bfe0 con 0x7f8ccc07ddb0 2024-09-10T09:27:54.611 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:53.417+0000 7f8ccb400640 1 --2- 172.21.15.5:0/428462048 >> [v2:172.21.15.5:6800/1527422686,v1:172.21.15.5:6801/1527422686] conn(0x7f8cb0038fc0 0x7f8cb003b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:27:54.611 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:53.418+0000 7f8ccb400640 1 --2- 172.21.15.5:0/428462048 >> [v2:172.21.15.5:6800/1527422686,v1:172.21.15.5:6801/1527422686] conn(0x7f8cb0038fc0 0x7f8cb003b480 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f8cb80098c0 tx=0x7f8cb8006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:27:54.611 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:53.422+0000 7f8cc8c00640 1 -- 172.21.15.5:0/428462048 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f8cbc045360 con 0x7f8ccc07ddb0 2024-09-10T09:27:54.611 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:53.622+0000 7f8cd3200640 1 -- 172.21.15.5:0/428462048 --> [v2:172.21.15.5:6800/1527422686,v1:172.21.15.5:6801/1527422686] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f8ccc104190 con 0x7f8cb0038fc0 2024-09-10T09:27:54.611 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:53.626+0000 7f8cc8c00640 1 -- 172.21.15.5:0/428462048 <== mgr.14118 v2:172.21.15.5:6800/1527422686 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f8ccc104190 con 0x7f8cb0038fc0 2024-09-10T09:27:54.611 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:53.628+0000 7f8cd3200640 1 -- 172.21.15.5:0/428462048 >> [v2:172.21.15.5:6800/1527422686,v1:172.21.15.5:6801/1527422686] conn(0x7f8cb0038fc0 msgr2=0x7f8cb003b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:27:54.611 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:53.628+0000 7f8cd3200640 1 --2- 172.21.15.5:0/428462048 >> [v2:172.21.15.5:6800/1527422686,v1:172.21.15.5:6801/1527422686] conn(0x7f8cb0038fc0 0x7f8cb003b480 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f8cb80098c0 tx=0x7f8cb8006db0 comp rx=0 tx=0).stop 2024-09-10T09:27:54.611 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:53.628+0000 7f8cd3200640 1 -- 172.21.15.5:0/428462048 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8ccc07ddb0 msgr2=0x7f8ccc19d890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:27:54.611 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:53.628+0000 7f8cd3200640 1 --2- 172.21.15.5:0/428462048 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8ccc07ddb0 0x7f8ccc19d890 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f8cbc0097f0 tx=0x7f8cbc030ec0 comp rx=0 tx=0).stop 2024-09-10T09:27:54.611 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:53.629+0000 7f8cd3200640 1 -- 172.21.15.5:0/428462048 shutdown_connections 2024-09-10T09:27:54.611 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:53.629+0000 7f8cd3200640 1 --2- 172.21.15.5:0/428462048 >> [v2:172.21.15.5:6800/1527422686,v1:172.21.15.5:6801/1527422686] conn(0x7f8cb0038fc0 0x7f8cb003b480 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:27:54.612 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:53.629+0000 7f8cd3200640 1 --2- 172.21.15.5:0/428462048 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8ccc07ddb0 0x7f8ccc19d890 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:27:54.612 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:53.629+0000 7f8cd3200640 1 -- 172.21.15.5:0/428462048 >> 172.21.15.5:0/428462048 conn(0x7f8ccc102eb0 msgr2=0x7f8ccc103a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:27:54.612 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:53.629+0000 7f8cd3200640 1 -- 172.21.15.5:0/428462048 shutdown_connections 2024-09-10T09:27:54.612 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:53.629+0000 7f8cd3200640 1 -- 172.21.15.5:0/428462048 wait complete. 2024-09-10T09:27:54.882 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:54 smithi005 ceph-mon[30350]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T09:27:54.882 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:54 smithi005 ceph-mon[30350]: Saving service mgr spec with placement count:2 2024-09-10T09:27:54.882 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:54 smithi005 ceph-mon[30350]: from='mgr.14118 172.21.15.5:0/1435659661' entity='mgr.x' 2024-09-10T09:27:54.882 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:54 smithi005 ceph-mon[30350]: from='mgr.14118 172.21.15.5:0/1435659661' entity='mgr.x' 2024-09-10T09:27:54.882 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:54 smithi005 ceph-mon[30350]: from='mgr.14118 172.21.15.5:0/1435659661' entity='mgr.x' 2024-09-10T09:27:56.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:56 smithi005 ceph-mon[30350]: from='client.? 172.21.15.5:0/3459567327' entity='client.admin' 2024-09-10T09:27:56.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:56 smithi005 ceph-mon[30350]: from='mgr.14118 172.21.15.5:0/1435659661' entity='mgr.x' 2024-09-10T09:27:56.565 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:55.128+0000 7fce79a00640 1 Processor -- start 2024-09-10T09:27:56.565 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:55.129+0000 7fce79a00640 1 -- start start 2024-09-10T09:27:56.565 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:55.129+0000 7fce79a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fce7407dee0 0x7fce7407c340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:27:56.565 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:55.129+0000 7fce79a00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fce7407c910 con 0x7fce7407dee0 2024-09-10T09:27:56.565 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:55.130+0000 7fce72a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fce7407dee0 0x7fce7407c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:27:56.565 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:55.130+0000 7fce72a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fce7407dee0 0x7fce7407c340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40864/0 (socket says 172.21.15.5:40864) 2024-09-10T09:27:56.565 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:55.130+0000 7fce72a00640 1 -- 172.21.15.5:0/683932018 learned_addr learned my addr 172.21.15.5:0/683932018 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:27:56.565 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:55.130+0000 7fce72a00640 1 -- 172.21.15.5:0/683932018 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce7407ca90 con 0x7fce7407dee0 2024-09-10T09:27:56.565 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:55.131+0000 7fce72a00640 1 --2- 172.21.15.5:0/683932018 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fce7407dee0 0x7fce7407c340 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fce64009820 tx=0x7fce6402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2b9d41361405eb8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:27:56.565 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:55.131+0000 7fce72000640 1 -- 172.21.15.5:0/683932018 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fce64037070 con 0x7fce7407dee0 2024-09-10T09:27:56.566 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:55.131+0000 7fce72000640 1 -- 172.21.15.5:0/683932018 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fce64032920 con 0x7fce7407dee0 2024-09-10T09:27:56.566 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:55.131+0000 7fce72000640 1 -- 172.21.15.5:0/683932018 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fce64030810 con 0x7fce7407dee0 2024-09-10T09:27:56.566 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:55.132+0000 7fce79a00640 1 -- 172.21.15.5:0/683932018 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fce7407dee0 msgr2=0x7fce7407c340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:27:56.566 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:55.132+0000 7fce79a00640 1 --2- 172.21.15.5:0/683932018 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fce7407dee0 0x7fce7407c340 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fce64009820 tx=0x7fce6402a270 comp rx=0 tx=0).stop 2024-09-10T09:27:56.566 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:55.132+0000 7fce79a00640 1 -- 172.21.15.5:0/683932018 shutdown_connections 2024-09-10T09:27:56.566 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:55.132+0000 7fce79a00640 1 --2- 172.21.15.5:0/683932018 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fce7407dee0 0x7fce7407c340 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:27:56.566 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:55.132+0000 7fce79a00640 1 -- 172.21.15.5:0/683932018 >> 172.21.15.5:0/683932018 conn(0x7fce74102d90 msgr2=0x7fce741051f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:27:56.566 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:55.132+0000 7fce79a00640 1 -- 172.21.15.5:0/683932018 shutdown_connections 2024-09-10T09:27:56.566 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:55.133+0000 7fce79a00640 1 -- 172.21.15.5:0/683932018 wait complete. 2024-09-10T09:27:56.566 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:55.133+0000 7fce79a00640 1 Processor -- start 2024-09-10T09:27:56.566 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:55.134+0000 7fce79a00640 1 -- start start 2024-09-10T09:27:56.566 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:55.134+0000 7fce79a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fce7407dee0 0x7fce7419d820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:27:56.566 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:55.134+0000 7fce79a00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fce7410dcb0 con 0x7fce7407dee0 2024-09-10T09:27:56.567 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:55.134+0000 7fce72a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fce7407dee0 0x7fce7419d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:27:56.567 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:55.134+0000 7fce72a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fce7407dee0 0x7fce7419d820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40874/0 (socket says 172.21.15.5:40874) 2024-09-10T09:27:56.567 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:55.134+0000 7fce72a00640 1 -- 172.21.15.5:0/3459567327 learned_addr learned my addr 172.21.15.5:0/3459567327 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:27:56.567 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:55.135+0000 7fce72a00640 1 -- 172.21.15.5:0/3459567327 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce7419dd60 con 0x7fce7407dee0 2024-09-10T09:27:56.567 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:55.135+0000 7fce72a00640 1 --2- 172.21.15.5:0/3459567327 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fce7407dee0 0x7fce7419d820 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fce640097f0 tx=0x7fce640327b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:27:56.567 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:55.135+0000 7fce6be00640 1 -- 172.21.15.5:0/3459567327 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fce64037040 con 0x7fce7407dee0 2024-09-10T09:27:56.567 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:55.135+0000 7fce6be00640 1 -- 172.21.15.5:0/3459567327 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fce64030cc0 con 0x7fce7407dee0 2024-09-10T09:27:56.567 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:55.135+0000 7fce79a00640 1 -- 172.21.15.5:0/3459567327 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce7419dff0 con 0x7fce7407dee0 2024-09-10T09:27:56.567 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:55.135+0000 7fce79a00640 1 -- 172.21.15.5:0/3459567327 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce741a0ce0 con 0x7fce7407dee0 2024-09-10T09:27:56.567 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:55.136+0000 7fce6be00640 1 -- 172.21.15.5:0/3459567327 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fce6403a640 con 0x7fce7407dee0 2024-09-10T09:27:56.567 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:55.137+0000 7fce6be00640 1 -- 172.21.15.5:0/3459567327 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7fce6403a8f0 con 0x7fce7407dee0 2024-09-10T09:27:56.567 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:55.137+0000 7fce79a00640 1 -- 172.21.15.5:0/3459567327 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fce500052c0 con 0x7fce7407dee0 2024-09-10T09:27:56.567 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:55.137+0000 7fce6be00640 1 --2- 172.21.15.5:0/3459567327 >> [v2:172.21.15.5:6800/1527422686,v1:172.21.15.5:6801/1527422686] conn(0x7fce58038fc0 0x7fce5803b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:27:56.568 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:55.137+0000 7fce6be00640 1 -- 172.21.15.5:0/3459567327 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fce6406c570 con 0x7fce7407dee0 2024-09-10T09:27:56.568 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:55.138+0000 7fce6a000640 1 --2- 172.21.15.5:0/3459567327 >> [v2:172.21.15.5:6800/1527422686,v1:172.21.15.5:6801/1527422686] conn(0x7fce58038fc0 0x7fce5803b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:27:56.568 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:55.138+0000 7fce6a000640 1 --2- 172.21.15.5:0/3459567327 >> [v2:172.21.15.5:6800/1527422686,v1:172.21.15.5:6801/1527422686] conn(0x7fce58038fc0 0x7fce5803b480 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fce5c0098c0 tx=0x7fce5c006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:27:56.568 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:55.147+0000 7fce6be00640 1 -- 172.21.15.5:0/3459567327 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fce640328e0 con 0x7fce7407dee0 2024-09-10T09:27:56.568 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:55.339+0000 7fce79a00640 1 -- 172.21.15.5:0/3459567327 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) -- 0x7fce500055b0 con 0x7fce7407dee0 2024-09-10T09:27:56.568 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:55.341+0000 7fce6be00640 1 -- 172.21.15.5:0/3459567327 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) ==== 142+0+0 (secure 0 0 0) 0x7fce64031c00 con 0x7fce7407dee0 2024-09-10T09:27:56.568 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:55.344+0000 7fce79a00640 1 -- 172.21.15.5:0/3459567327 >> [v2:172.21.15.5:6800/1527422686,v1:172.21.15.5:6801/1527422686] conn(0x7fce58038fc0 msgr2=0x7fce5803b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:27:56.568 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:55.344+0000 7fce79a00640 1 --2- 172.21.15.5:0/3459567327 >> [v2:172.21.15.5:6800/1527422686,v1:172.21.15.5:6801/1527422686] conn(0x7fce58038fc0 0x7fce5803b480 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fce5c0098c0 tx=0x7fce5c006db0 comp rx=0 tx=0).stop 2024-09-10T09:27:56.568 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:55.344+0000 7fce79a00640 1 -- 172.21.15.5:0/3459567327 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fce7407dee0 msgr2=0x7fce7419d820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:27:56.569 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:55.344+0000 7fce79a00640 1 --2- 172.21.15.5:0/3459567327 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fce7407dee0 0x7fce7419d820 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fce640097f0 tx=0x7fce640327b0 comp rx=0 tx=0).stop 2024-09-10T09:27:56.569 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:55.344+0000 7fce79a00640 1 -- 172.21.15.5:0/3459567327 shutdown_connections 2024-09-10T09:27:56.569 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:55.344+0000 7fce79a00640 1 --2- 172.21.15.5:0/3459567327 >> [v2:172.21.15.5:6800/1527422686,v1:172.21.15.5:6801/1527422686] conn(0x7fce58038fc0 0x7fce5803b480 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:27:56.569 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:55.344+0000 7fce79a00640 1 --2- 172.21.15.5:0/3459567327 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fce7407dee0 0x7fce7419d820 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:27:56.569 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:55.344+0000 7fce79a00640 1 -- 172.21.15.5:0/3459567327 >> 172.21.15.5:0/3459567327 conn(0x7fce74102d90 msgr2=0x7fce74103970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:27:56.569 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:55.344+0000 7fce79a00640 1 -- 172.21.15.5:0/3459567327 shutdown_connections 2024-09-10T09:27:56.569 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:55.344+0000 7fce79a00640 1 -- 172.21.15.5:0/3459567327 wait complete. 2024-09-10T09:27:57.972 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:57.108+0000 7fbd69800640 1 Processor -- start 2024-09-10T09:27:57.972 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:57.108+0000 7fbd69800640 1 -- start start 2024-09-10T09:27:57.972 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:57.108+0000 7fbd69800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbd64107680 0x7fbd64107a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:27:57.973 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:57.108+0000 7fbd69800640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fbd64108050 con 0x7fbd64107680 2024-09-10T09:27:57.973 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:57.108+0000 7fbd62a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbd64107680 0x7fbd64107a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:27:57.973 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:57.108+0000 7fbd62a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbd64107680 0x7fbd64107a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40876/0 (socket says 172.21.15.5:40876) 2024-09-10T09:27:57.973 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:57.108+0000 7fbd62a00640 1 -- 172.21.15.5:0/2680508555 learned_addr learned my addr 172.21.15.5:0/2680508555 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:27:57.973 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:57.109+0000 7fbd62a00640 1 -- 172.21.15.5:0/2680508555 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd64108830 con 0x7fbd64107680 2024-09-10T09:27:57.973 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:57.110+0000 7fbd62a00640 1 --2- 172.21.15.5:0/2680508555 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbd64107680 0x7fbd64107a80 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fbd54009820 tx=0x7fbd5402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f72c5444a1f338f9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:27:57.973 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:57.110+0000 7fbd61600640 1 -- 172.21.15.5:0/2680508555 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbd54037070 con 0x7fbd64107680 2024-09-10T09:27:57.973 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:57.110+0000 7fbd61600640 1 -- 172.21.15.5:0/2680508555 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbd54032920 con 0x7fbd64107680 2024-09-10T09:27:57.973 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:57.110+0000 7fbd61600640 1 -- 172.21.15.5:0/2680508555 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbd54030810 con 0x7fbd64107680 2024-09-10T09:27:57.973 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:57.111+0000 7fbd69800640 1 -- 172.21.15.5:0/2680508555 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbd64107680 msgr2=0x7fbd64107a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:27:57.973 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:57.111+0000 7fbd69800640 1 --2- 172.21.15.5:0/2680508555 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbd64107680 0x7fbd64107a80 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fbd54009820 tx=0x7fbd5402a270 comp rx=0 tx=0).stop 2024-09-10T09:27:57.973 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:57.111+0000 7fbd69800640 1 -- 172.21.15.5:0/2680508555 shutdown_connections 2024-09-10T09:27:57.974 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:57.111+0000 7fbd69800640 1 --2- 172.21.15.5:0/2680508555 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbd64107680 0x7fbd64107a80 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:27:57.974 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:57.111+0000 7fbd69800640 1 -- 172.21.15.5:0/2680508555 >> 172.21.15.5:0/2680508555 conn(0x7fbd64102e50 msgr2=0x7fbd64105270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:27:57.974 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:57.111+0000 7fbd69800640 1 -- 172.21.15.5:0/2680508555 shutdown_connections 2024-09-10T09:27:57.974 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:57.111+0000 7fbd69800640 1 -- 172.21.15.5:0/2680508555 wait complete. 2024-09-10T09:27:57.974 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:57.112+0000 7fbd69800640 1 Processor -- start 2024-09-10T09:27:57.974 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:57.112+0000 7fbd69800640 1 -- start start 2024-09-10T09:27:57.974 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:57.113+0000 7fbd69800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbd64107680 0x7fbd6419d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:27:57.974 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:57.113+0000 7fbd69800640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fbd6410bbe0 con 0x7fbd64107680 2024-09-10T09:27:57.974 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:57.113+0000 7fbd62a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbd64107680 0x7fbd6419d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:27:57.974 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:57.113+0000 7fbd62a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbd64107680 0x7fbd6419d8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40888/0 (socket says 172.21.15.5:40888) 2024-09-10T09:27:57.974 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:57.113+0000 7fbd62a00640 1 -- 172.21.15.5:0/3684066396 learned_addr learned my addr 172.21.15.5:0/3684066396 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:27:57.974 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:57.113+0000 7fbd62a00640 1 -- 172.21.15.5:0/3684066396 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd6419de10 con 0x7fbd64107680 2024-09-10T09:27:57.974 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:57.114+0000 7fbd62a00640 1 --2- 172.21.15.5:0/3684066396 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbd64107680 0x7fbd6419d8d0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fbd540097f0 tx=0x7fbd540327b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:27:57.975 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:57.114+0000 7fbd5b400640 1 -- 172.21.15.5:0/3684066396 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbd54037040 con 0x7fbd64107680 2024-09-10T09:27:57.975 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:57.114+0000 7fbd5b400640 1 -- 172.21.15.5:0/3684066396 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbd54030c40 con 0x7fbd64107680 2024-09-10T09:27:57.975 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:57.114+0000 7fbd5b400640 1 -- 172.21.15.5:0/3684066396 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbd5403a5b0 con 0x7fbd64107680 2024-09-10T09:27:57.975 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:57.114+0000 7fbd69800640 1 -- 172.21.15.5:0/3684066396 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd6419e0a0 con 0x7fbd64107680 2024-09-10T09:27:57.975 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:57.114+0000 7fbd69800640 1 -- 172.21.15.5:0/3684066396 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd641a0d90 con 0x7fbd64107680 2024-09-10T09:27:57.975 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:57.116+0000 7fbd5b400640 1 -- 172.21.15.5:0/3684066396 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7fbd5403b3f0 con 0x7fbd64107680 2024-09-10T09:27:57.975 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:57.116+0000 7fbd69800640 1 -- 172.21.15.5:0/3684066396 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd6410bd60 con 0x7fbd64107680 2024-09-10T09:27:57.975 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:57.116+0000 7fbd5b400640 1 --2- 172.21.15.5:0/3684066396 >> [v2:172.21.15.5:6800/1527422686,v1:172.21.15.5:6801/1527422686] conn(0x7fbd48038f70 0x7fbd4803b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:27:57.975 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:57.116+0000 7fbd5b400640 1 -- 172.21.15.5:0/3684066396 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fbd5406c480 con 0x7fbd64107680 2024-09-10T09:27:57.975 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:57.117+0000 7fbd62000640 1 --2- 172.21.15.5:0/3684066396 >> [v2:172.21.15.5:6800/1527422686,v1:172.21.15.5:6801/1527422686] conn(0x7fbd48038f70 0x7fbd4803b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:27:57.975 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:57.117+0000 7fbd62000640 1 --2- 172.21.15.5:0/3684066396 >> [v2:172.21.15.5:6800/1527422686,v1:172.21.15.5:6801/1527422686] conn(0x7fbd48038f70 0x7fbd4803b430 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fbd500098c0 tx=0x7fbd50006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:27:57.975 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:57.125+0000 7fbd5b400640 1 -- 172.21.15.5:0/3684066396 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fbd540313a0 con 0x7fbd64107680 2024-09-10T09:27:57.975 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:57.319+0000 7fbd69800640 1 -- 172.21.15.5:0/3684066396 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) -- 0x7fbd64107b00 con 0x7fbd64107680 2024-09-10T09:27:57.975 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:57.322+0000 7fbd5b400640 1 -- 172.21.15.5:0/3684066396 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) ==== 130+0+0 (secure 0 0 0) 0x7fbd5406bdd0 con 0x7fbd64107680 2024-09-10T09:27:57.976 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:57.325+0000 7fbd69800640 1 -- 172.21.15.5:0/3684066396 >> [v2:172.21.15.5:6800/1527422686,v1:172.21.15.5:6801/1527422686] conn(0x7fbd48038f70 msgr2=0x7fbd4803b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:27:57.976 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:57.325+0000 7fbd69800640 1 --2- 172.21.15.5:0/3684066396 >> [v2:172.21.15.5:6800/1527422686,v1:172.21.15.5:6801/1527422686] conn(0x7fbd48038f70 0x7fbd4803b430 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fbd500098c0 tx=0x7fbd50006db0 comp rx=0 tx=0).stop 2024-09-10T09:27:57.976 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:57.325+0000 7fbd69800640 1 -- 172.21.15.5:0/3684066396 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbd64107680 msgr2=0x7fbd6419d8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:27:57.976 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:57.325+0000 7fbd69800640 1 --2- 172.21.15.5:0/3684066396 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbd64107680 0x7fbd6419d8d0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fbd540097f0 tx=0x7fbd540327b0 comp rx=0 tx=0).stop 2024-09-10T09:27:57.976 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:57.325+0000 7fbd69800640 1 -- 172.21.15.5:0/3684066396 shutdown_connections 2024-09-10T09:27:57.976 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:57.325+0000 7fbd69800640 1 --2- 172.21.15.5:0/3684066396 >> [v2:172.21.15.5:6800/1527422686,v1:172.21.15.5:6801/1527422686] conn(0x7fbd48038f70 0x7fbd4803b430 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:27:57.976 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:57.325+0000 7fbd69800640 1 --2- 172.21.15.5:0/3684066396 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbd64107680 0x7fbd6419d8d0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:27:57.976 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:57.325+0000 7fbd69800640 1 -- 172.21.15.5:0/3684066396 >> 172.21.15.5:0/3684066396 conn(0x7fbd64102e50 msgr2=0x7fbd641039c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:27:57.976 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:57.325+0000 7fbd69800640 1 -- 172.21.15.5:0/3684066396 shutdown_connections 2024-09-10T09:27:57.976 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:57.326+0000 7fbd69800640 1 -- 172.21.15.5:0/3684066396 wait complete. 2024-09-10T09:27:57.976 INFO:teuthology.orchestra.run.smithi005.stdout:Enabling the dashboard module... 2024-09-10T09:27:58.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:58 smithi005 ceph-mon[30350]: from='client.? 172.21.15.5:0/3684066396' entity='client.admin' 2024-09-10T09:27:58.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:58 smithi005 ceph-mon[30350]: from='mgr.14118 172.21.15.5:0/1435659661' entity='mgr.x' 2024-09-10T09:27:58.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:58 smithi005 ceph-mon[30350]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T09:27:59.573 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:27:59 smithi005 ceph-mon[30350]: from='client.? 172.21.15.5:0/3842848504' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-09-10T09:27:59.573 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 10 09:27:59 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mgr-x[30575]: ignoring --setuser ceph since I am not root 2024-09-10T09:27:59.573 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 10 09:27:59 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mgr-x[30575]: ignoring --setgroup ceph since I am not root 2024-09-10T09:27:59.574 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 10 09:27:59 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mgr-x[30575]: 2024-09-10T09:27:59.572+0000 7f570a5dc040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-10T09:27:59.838 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 10 09:27:59 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mgr-x[30575]: 2024-09-10T09:27:59.646+0000 7f570a5dc040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-10T09:28:00.027 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:58.424+0000 7f5e4aa00640 1 Processor -- start 2024-09-10T09:28:00.027 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:58.424+0000 7f5e4aa00640 1 -- start start 2024-09-10T09:28:00.027 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:58.424+0000 7f5e4aa00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5e441076a0 0x7f5e44107aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:28:00.027 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:58.424+0000 7f5e4aa00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f5e44108070 con 0x7f5e441076a0 2024-09-10T09:28:00.027 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:58.425+0000 7f5e43e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5e441076a0 0x7f5e44107aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:28:00.028 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:58.425+0000 7f5e43e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5e441076a0 0x7f5e44107aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40892/0 (socket says 172.21.15.5:40892) 2024-09-10T09:28:00.028 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:58.425+0000 7f5e43e00640 1 -- 172.21.15.5:0/291163601 learned_addr learned my addr 172.21.15.5:0/291163601 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:28:00.028 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:58.425+0000 7f5e43e00640 1 -- 172.21.15.5:0/291163601 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e44108800 con 0x7f5e441076a0 2024-09-10T09:28:00.028 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:58.426+0000 7f5e43e00640 1 --2- 172.21.15.5:0/291163601 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5e441076a0 0x7f5e44107aa0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f5e34009820 tx=0x7f5e3402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a61101b8b4cea05d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:28:00.028 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:58.426+0000 7f5e42a00640 1 -- 172.21.15.5:0/291163601 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5e34037070 con 0x7f5e441076a0 2024-09-10T09:28:00.028 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:58.426+0000 7f5e42a00640 1 -- 172.21.15.5:0/291163601 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5e34032920 con 0x7f5e441076a0 2024-09-10T09:28:00.028 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:58.426+0000 7f5e42a00640 1 -- 172.21.15.5:0/291163601 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5e34030810 con 0x7f5e441076a0 2024-09-10T09:28:00.028 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:58.427+0000 7f5e4aa00640 1 -- 172.21.15.5:0/291163601 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5e441076a0 msgr2=0x7f5e44107aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:28:00.028 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:58.427+0000 7f5e4aa00640 1 --2- 172.21.15.5:0/291163601 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5e441076a0 0x7f5e44107aa0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f5e34009820 tx=0x7f5e3402a270 comp rx=0 tx=0).stop 2024-09-10T09:28:00.028 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:58.427+0000 7f5e4aa00640 1 -- 172.21.15.5:0/291163601 shutdown_connections 2024-09-10T09:28:00.028 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:58.427+0000 7f5e4aa00640 1 --2- 172.21.15.5:0/291163601 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5e441076a0 0x7f5e44107aa0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:28:00.028 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:58.427+0000 7f5e4aa00640 1 -- 172.21.15.5:0/291163601 >> 172.21.15.5:0/291163601 conn(0x7f5e44102e50 msgr2=0x7f5e44105270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:28:00.028 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:58.427+0000 7f5e4aa00640 1 -- 172.21.15.5:0/291163601 shutdown_connections 2024-09-10T09:28:00.029 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:58.427+0000 7f5e4aa00640 1 -- 172.21.15.5:0/291163601 wait complete. 2024-09-10T09:28:00.029 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:58.428+0000 7f5e4aa00640 1 Processor -- start 2024-09-10T09:28:00.029 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:58.428+0000 7f5e4aa00640 1 -- start start 2024-09-10T09:28:00.029 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:58.429+0000 7f5e4aa00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5e441076a0 0x7f5e4407db70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:28:00.029 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:58.429+0000 7f5e4aa00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f5e4410bbb0 con 0x7f5e441076a0 2024-09-10T09:28:00.029 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:58.429+0000 7f5e43e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5e441076a0 0x7f5e4407db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:28:00.029 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:58.429+0000 7f5e43e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5e441076a0 0x7f5e4407db70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40908/0 (socket says 172.21.15.5:40908) 2024-09-10T09:28:00.029 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:58.429+0000 7f5e43e00640 1 -- 172.21.15.5:0/3842848504 learned_addr learned my addr 172.21.15.5:0/3842848504 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:28:00.029 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:58.429+0000 7f5e43e00640 1 -- 172.21.15.5:0/3842848504 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e4407e0b0 con 0x7f5e441076a0 2024-09-10T09:28:00.029 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:58.430+0000 7f5e43e00640 1 --2- 172.21.15.5:0/3842848504 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5e441076a0 0x7f5e4407db70 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f5e340097f0 tx=0x7f5e340327b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:28:00.029 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:58.430+0000 7f5e40c00640 1 -- 172.21.15.5:0/3842848504 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5e34037040 con 0x7f5e441076a0 2024-09-10T09:28:00.029 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:58.430+0000 7f5e40c00640 1 -- 172.21.15.5:0/3842848504 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5e34030c40 con 0x7f5e441076a0 2024-09-10T09:28:00.029 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:58.430+0000 7f5e4aa00640 1 -- 172.21.15.5:0/3842848504 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e4407c2a0 con 0x7f5e441076a0 2024-09-10T09:28:00.029 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:58.430+0000 7f5e4aa00640 1 -- 172.21.15.5:0/3842848504 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e4407c7e0 con 0x7f5e441076a0 2024-09-10T09:28:00.030 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:58.430+0000 7f5e40c00640 1 -- 172.21.15.5:0/3842848504 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5e3403a5b0 con 0x7f5e441076a0 2024-09-10T09:28:00.030 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:58.431+0000 7f5e40c00640 1 -- 172.21.15.5:0/3842848504 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f5e3403a7a0 con 0x7f5e441076a0 2024-09-10T09:28:00.030 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:58.431+0000 7f5e4aa00640 1 -- 172.21.15.5:0/3842848504 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5e4410bd30 con 0x7f5e441076a0 2024-09-10T09:28:00.030 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:58.431+0000 7f5e40c00640 1 --2- 172.21.15.5:0/3842848504 >> [v2:172.21.15.5:6800/1527422686,v1:172.21.15.5:6801/1527422686] conn(0x7f5e28038fc0 0x7f5e2803b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:28:00.030 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:58.432+0000 7f5e40c00640 1 -- 172.21.15.5:0/3842848504 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f5e3406c390 con 0x7f5e441076a0 2024-09-10T09:28:00.030 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:58.432+0000 7f5e43400640 1 --2- 172.21.15.5:0/3842848504 >> [v2:172.21.15.5:6800/1527422686,v1:172.21.15.5:6801/1527422686] conn(0x7f5e28038fc0 0x7f5e2803b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:28:00.030 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:58.432+0000 7f5e43400640 1 --2- 172.21.15.5:0/3842848504 >> [v2:172.21.15.5:6800/1527422686,v1:172.21.15.5:6801/1527422686] conn(0x7f5e28038fc0 0x7f5e2803b480 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f5e300098c0 tx=0x7f5e30006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:28:00.030 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:58.437+0000 7f5e40c00640 1 -- 172.21.15.5:0/3842848504 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f5e340371e0 con 0x7f5e441076a0 2024-09-10T09:28:00.030 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:58.688+0000 7f5e4aa00640 1 -- 172.21.15.5:0/3842848504 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) -- 0x7f5e4407cf70 con 0x7f5e441076a0 2024-09-10T09:28:00.030 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:59.325+0000 7f5e40c00640 1 -- 172.21.15.5:0/3842848504 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) ==== 88+0+0 (secure 0 0 0) 0x7f5e3403b3f0 con 0x7f5e441076a0 2024-09-10T09:28:00.030 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:59.327+0000 7f5e40c00640 1 -- 172.21.15.5:0/3842848504 <== mon.0 v2:172.21.15.5:3300/0 8 ==== mgrmap(e 9) ==== 46300+0+0 (secure 0 0 0) 0x7f5e3403aa90 con 0x7f5e441076a0 2024-09-10T09:28:00.030 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:59.328+0000 7f5e4aa00640 1 -- 172.21.15.5:0/3842848504 >> [v2:172.21.15.5:6800/1527422686,v1:172.21.15.5:6801/1527422686] conn(0x7f5e28038fc0 msgr2=0x7f5e2803b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:28:00.031 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:59.328+0000 7f5e4aa00640 1 --2- 172.21.15.5:0/3842848504 >> [v2:172.21.15.5:6800/1527422686,v1:172.21.15.5:6801/1527422686] conn(0x7f5e28038fc0 0x7f5e2803b480 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f5e300098c0 tx=0x7f5e30006db0 comp rx=0 tx=0).stop 2024-09-10T09:28:00.031 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:59.328+0000 7f5e4aa00640 1 -- 172.21.15.5:0/3842848504 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5e441076a0 msgr2=0x7f5e4407db70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:28:00.031 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:59.328+0000 7f5e4aa00640 1 --2- 172.21.15.5:0/3842848504 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5e441076a0 0x7f5e4407db70 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f5e340097f0 tx=0x7f5e340327b0 comp rx=0 tx=0).stop 2024-09-10T09:28:00.031 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:59.328+0000 7f5e4aa00640 1 -- 172.21.15.5:0/3842848504 shutdown_connections 2024-09-10T09:28:00.031 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:59.328+0000 7f5e4aa00640 1 --2- 172.21.15.5:0/3842848504 >> [v2:172.21.15.5:6800/1527422686,v1:172.21.15.5:6801/1527422686] conn(0x7f5e28038fc0 0x7f5e2803b480 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:28:00.031 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:59.328+0000 7f5e4aa00640 1 --2- 172.21.15.5:0/3842848504 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5e441076a0 0x7f5e4407db70 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:28:00.031 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:59.328+0000 7f5e4aa00640 1 -- 172.21.15.5:0/3842848504 >> 172.21.15.5:0/3842848504 conn(0x7f5e44102e50 msgr2=0x7f5e441039e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:28:00.031 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:59.329+0000 7f5e4aa00640 1 -- 172.21.15.5:0/3842848504 shutdown_connections 2024-09-10T09:28:00.031 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:27:59.329+0000 7f5e4aa00640 1 -- 172.21.15.5:0/3842848504 wait complete. 2024-09-10T09:28:00.101 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 10 09:27:59 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mgr-x[30575]: 2024-09-10T09:27:59.837+0000 7f570a5dc040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-10T09:28:00.101 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 10 09:27:59 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mgr-x[30575]: 2024-09-10T09:27:59.919+0000 7f570a5dc040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-10T09:28:00.101 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 10 09:28:00 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mgr-x[30575]: 2024-09-10T09:28:00.100+0000 7f570a5dc040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-10T09:28:00.393 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 10 09:28:00 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mgr-x[30575]: 2024-09-10T09:28:00.271+0000 7f570a5dc040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-10T09:28:00.394 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:28:00 smithi005 ceph-mon[30350]: from='client.? 172.21.15.5:0/3842848504' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-09-10T09:28:00.394 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:28:00 smithi005 ceph-mon[30350]: mgrmap e9: x(active, since 21s) 2024-09-10T09:28:00.957 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 10 09:28:00 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mgr-x[30575]: 2024-09-10T09:28:00.657+0000 7f570a5dc040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-10T09:28:00.958 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 10 09:28:00 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mgr-x[30575]: 2024-09-10T09:28:00.733+0000 7f570a5dc040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-10T09:28:00.958 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 10 09:28:00 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mgr-x[30575]: 2024-09-10T09:28:00.956+0000 7f570a5dc040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-10T09:28:01.639 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:28:01 smithi005 ceph-mon[30350]: from='client.? 172.21.15.5:0/1963020441' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-09-10T09:28:01.644 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout { 2024-09-10T09:28:01.650 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-09-10T09:28:01.650 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-10T09:28:01.650 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-09-10T09:28:01.650 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-09-10T09:28:01.651 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout } 2024-09-10T09:28:01.651 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:00.787+0000 7fac0ce00640 1 Processor -- start 2024-09-10T09:28:01.651 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:00.787+0000 7fac0ce00640 1 -- start start 2024-09-10T09:28:01.651 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:00.787+0000 7fac0ce00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fac08107680 0x7fac08107a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:28:01.651 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:00.787+0000 7fac0ce00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fac08108050 con 0x7fac08107680 2024-09-10T09:28:01.651 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:00.788+0000 7fac05e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fac08107680 0x7fac08107a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:28:01.651 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:00.788+0000 7fac05e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fac08107680 0x7fac08107a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40932/0 (socket says 172.21.15.5:40932) 2024-09-10T09:28:01.651 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:00.788+0000 7fac05e00640 1 -- 172.21.15.5:0/206839427 learned_addr learned my addr 172.21.15.5:0/206839427 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:28:01.651 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:00.788+0000 7fac05e00640 1 -- 172.21.15.5:0/206839427 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac08108830 con 0x7fac08107680 2024-09-10T09:28:01.651 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:00.789+0000 7fac05e00640 1 --2- 172.21.15.5:0/206839427 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fac08107680 0x7fac08107a80 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fabf8009820 tx=0x7fabf802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30a09c293e5715ad server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:28:01.651 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:00.790+0000 7fac04a00640 1 -- 172.21.15.5:0/206839427 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fabf8037070 con 0x7fac08107680 2024-09-10T09:28:01.651 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:00.790+0000 7fac04a00640 1 -- 172.21.15.5:0/206839427 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fabf8032920 con 0x7fac08107680 2024-09-10T09:28:01.651 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:00.790+0000 7fac04a00640 1 -- 172.21.15.5:0/206839427 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fabf8030810 con 0x7fac08107680 2024-09-10T09:28:01.652 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:00.791+0000 7fac0ce00640 1 -- 172.21.15.5:0/206839427 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fac08107680 msgr2=0x7fac08107a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:28:01.652 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:00.791+0000 7fac0ce00640 1 --2- 172.21.15.5:0/206839427 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fac08107680 0x7fac08107a80 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fabf8009820 tx=0x7fabf802a270 comp rx=0 tx=0).stop 2024-09-10T09:28:01.652 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:00.791+0000 7fac0ce00640 1 -- 172.21.15.5:0/206839427 shutdown_connections 2024-09-10T09:28:01.652 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:00.791+0000 7fac0ce00640 1 --2- 172.21.15.5:0/206839427 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fac08107680 0x7fac08107a80 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:28:01.652 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:00.791+0000 7fac0ce00640 1 -- 172.21.15.5:0/206839427 >> 172.21.15.5:0/206839427 conn(0x7fac08102e50 msgr2=0x7fac08105270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:28:01.652 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:00.791+0000 7fac0ce00640 1 -- 172.21.15.5:0/206839427 shutdown_connections 2024-09-10T09:28:01.652 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:00.791+0000 7fac0ce00640 1 -- 172.21.15.5:0/206839427 wait complete. 2024-09-10T09:28:01.652 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:00.792+0000 7fac0ce00640 1 Processor -- start 2024-09-10T09:28:01.652 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:00.792+0000 7fac0ce00640 1 -- start start 2024-09-10T09:28:01.652 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:00.792+0000 7fac0ce00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fac08107680 0x7fac0819d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:28:01.652 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:00.792+0000 7fac0ce00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fac0810bbe0 con 0x7fac08107680 2024-09-10T09:28:01.652 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:00.793+0000 7fac05e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fac08107680 0x7fac0819d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:28:01.653 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:00.793+0000 7fac05e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fac08107680 0x7fac0819d8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40946/0 (socket says 172.21.15.5:40946) 2024-09-10T09:28:01.653 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:00.793+0000 7fac05e00640 1 -- 172.21.15.5:0/1963020441 learned_addr learned my addr 172.21.15.5:0/1963020441 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:28:01.653 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:00.793+0000 7fac05e00640 1 -- 172.21.15.5:0/1963020441 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac0819de10 con 0x7fac08107680 2024-09-10T09:28:01.653 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:00.793+0000 7fac05e00640 1 --2- 172.21.15.5:0/1963020441 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fac08107680 0x7fac0819d8d0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fabf8030560 tx=0x7fabf8032670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:28:01.653 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:00.794+0000 7fabfea00640 1 -- 172.21.15.5:0/1963020441 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fabf8037040 con 0x7fac08107680 2024-09-10T09:28:01.653 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:00.794+0000 7fac0ce00640 1 -- 172.21.15.5:0/1963020441 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac0819e0a0 con 0x7fac08107680 2024-09-10T09:28:01.653 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:00.794+0000 7fabfea00640 1 -- 172.21.15.5:0/1963020441 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fabf8040070 con 0x7fac08107680 2024-09-10T09:28:01.653 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:00.794+0000 7fabfea00640 1 -- 172.21.15.5:0/1963020441 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fabf803a6d0 con 0x7fac08107680 2024-09-10T09:28:01.653 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:00.794+0000 7fac0ce00640 1 -- 172.21.15.5:0/1963020441 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac081a0d90 con 0x7fac08107680 2024-09-10T09:28:01.653 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:00.795+0000 7fac0ce00640 1 -- 172.21.15.5:0/1963020441 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac0810bd60 con 0x7fac08107680 2024-09-10T09:28:01.653 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:00.795+0000 7fabfea00640 1 -- 172.21.15.5:0/1963020441 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 9) ==== 46300+0+0 (secure 0 0 0) 0x7fabf803a870 con 0x7fac08107680 2024-09-10T09:28:01.653 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:00.796+0000 7fabfea00640 1 --2- 172.21.15.5:0/1963020441 >> [v2:172.21.15.5:6800/1527422686,v1:172.21.15.5:6801/1527422686] conn(0x7fabec039020 0x7fabec03b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:28:01.653 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:00.796+0000 7fabfea00640 1 -- 172.21.15.5:0/1963020441 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fabf806c740 con 0x7fac08107680 2024-09-10T09:28:01.653 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:00.796+0000 7fac05400640 1 -- 172.21.15.5:0/1963020441 >> [v2:172.21.15.5:6800/1527422686,v1:172.21.15.5:6801/1527422686] conn(0x7fabec039020 msgr2=0x7fabec03b4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.5:6800/1527422686 2024-09-10T09:28:01.654 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:00.796+0000 7fac05400640 1 --2- 172.21.15.5:0/1963020441 >> [v2:172.21.15.5:6800/1527422686,v1:172.21.15.5:6801/1527422686] conn(0x7fabec039020 0x7fabec03b4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T09:28:01.654 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:00.801+0000 7fabfea00640 1 -- 172.21.15.5:0/1963020441 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fabf8044c70 con 0x7fac08107680 2024-09-10T09:28:01.654 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:00.997+0000 7fac05400640 1 -- 172.21.15.5:0/1963020441 >> [v2:172.21.15.5:6800/1527422686,v1:172.21.15.5:6801/1527422686] conn(0x7fabec039020 msgr2=0x7fabec03b4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.5:6800/1527422686 2024-09-10T09:28:01.654 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:00.997+0000 7fac05400640 1 --2- 172.21.15.5:0/1963020441 >> [v2:172.21.15.5:6800/1527422686,v1:172.21.15.5:6801/1527422686] conn(0x7fabec039020 0x7fabec03b4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-10T09:28:01.654 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:01.033+0000 7fac0ce00640 1 -- 172.21.15.5:0/1963020441 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7fac081a0f60 con 0x7fac08107680 2024-09-10T09:28:01.654 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:01.034+0000 7fabfea00640 1 -- 172.21.15.5:0/1963020441 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) ==== 56+0+88 (secure 0 0 0) 0x7fabf80328f0 con 0x7fac08107680 2024-09-10T09:28:01.654 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:01.036+0000 7fac0ce00640 1 -- 172.21.15.5:0/1963020441 >> [v2:172.21.15.5:6800/1527422686,v1:172.21.15.5:6801/1527422686] conn(0x7fabec039020 msgr2=0x7fabec03b4e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T09:28:01.654 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:01.036+0000 7fac0ce00640 1 --2- 172.21.15.5:0/1963020441 >> [v2:172.21.15.5:6800/1527422686,v1:172.21.15.5:6801/1527422686] conn(0x7fabec039020 0x7fabec03b4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:28:01.654 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:01.036+0000 7fac0ce00640 1 -- 172.21.15.5:0/1963020441 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fac08107680 msgr2=0x7fac0819d8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:28:01.654 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:01.036+0000 7fac0ce00640 1 --2- 172.21.15.5:0/1963020441 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fac08107680 0x7fac0819d8d0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fabf8030560 tx=0x7fabf8032670 comp rx=0 tx=0).stop 2024-09-10T09:28:01.654 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:01.036+0000 7fac0ce00640 1 -- 172.21.15.5:0/1963020441 shutdown_connections 2024-09-10T09:28:01.654 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:01.036+0000 7fac0ce00640 1 --2- 172.21.15.5:0/1963020441 >> [v2:172.21.15.5:6800/1527422686,v1:172.21.15.5:6801/1527422686] conn(0x7fabec039020 0x7fabec03b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:28:01.654 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:01.036+0000 7fac0ce00640 1 --2- 172.21.15.5:0/1963020441 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fac08107680 0x7fac0819d8d0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:28:01.655 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:01.036+0000 7fac0ce00640 1 -- 172.21.15.5:0/1963020441 >> 172.21.15.5:0/1963020441 conn(0x7fac08102e50 msgr2=0x7fac081039c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:28:01.655 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:01.036+0000 7fac0ce00640 1 -- 172.21.15.5:0/1963020441 shutdown_connections 2024-09-10T09:28:01.655 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:01.036+0000 7fac0ce00640 1 -- 172.21.15.5:0/1963020441 wait complete. 2024-09-10T09:28:01.655 INFO:teuthology.orchestra.run.smithi005.stdout:Waiting for the mgr to restart... 2024-09-10T09:28:01.655 INFO:teuthology.orchestra.run.smithi005.stdout:Waiting for mgr epoch 9... 2024-09-10T09:28:02.191 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 10 09:28:02 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mgr-x[30575]: 2024-09-10T09:28:02.128+0000 7f570a5dc040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-10T09:28:02.556 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 10 09:28:02 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mgr-x[30575]: 2024-09-10T09:28:02.215+0000 7f570a5dc040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-10T09:28:03.475 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 10 09:28:03 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mgr-x[30575]: 2024-09-10T09:28:03.162+0000 7f570a5dc040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-10T09:28:03.475 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 10 09:28:03 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mgr-x[30575]: 2024-09-10T09:28:03.303+0000 7f570a5dc040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-10T09:28:03.475 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 10 09:28:03 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mgr-x[30575]: 2024-09-10T09:28:03.474+0000 7f570a5dc040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-10T09:28:03.728 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 10 09:28:03 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mgr-x[30575]: 2024-09-10T09:28:03.549+0000 7f570a5dc040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-10T09:28:04.056 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 10 09:28:03 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mgr-x[30575]: 2024-09-10T09:28:03.727+0000 7f570a5dc040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-10T09:28:04.056 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 10 09:28:03 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mgr-x[30575]: 2024-09-10T09:28:03.812+0000 7f570a5dc040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-10T09:28:04.056 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 10 09:28:03 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mgr-x[30575]: 2024-09-10T09:28:03.886+0000 7f570a5dc040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-10T09:28:04.338 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 10 09:28:04 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mgr-x[30575]: 2024-09-10T09:28:04.074+0000 7f570a5dc040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-10T09:28:04.338 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 10 09:28:04 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mgr-x[30575]: 2024-09-10T09:28:04.154+0000 7f570a5dc040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-10T09:28:04.338 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 10 09:28:04 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mgr-x[30575]: 2024-09-10T09:28:04.236+0000 7f570a5dc040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-10T09:28:04.338 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 10 09:28:04 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mgr-x[30575]: 2024-09-10T09:28:04.336+0000 7f570a5dc040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-10T09:28:04.806 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 10 09:28:04 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mgr-x[30575]: 2024-09-10T09:28:04.410+0000 7f570a5dc040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-10T09:28:04.806 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 10 09:28:04 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mgr-x[30575]: 2024-09-10T09:28:04.482+0000 7f570a5dc040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-10T09:28:05.489 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 10 09:28:05 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mgr-x[30575]: 2024-09-10T09:28:05.434+0000 7f570a5dc040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-10T09:28:05.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:28:05 smithi005 ceph-mon[30350]: Active manager daemon x restarted 2024-09-10T09:28:05.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:28:05 smithi005 ceph-mon[30350]: Activating manager daemon x 2024-09-10T09:28:05.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:28:05 smithi005 ceph-mon[30350]: osdmap e3: 0 total, 0 up, 0 in 2024-09-10T09:28:05.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:28:05 smithi005 ceph-mon[30350]: mgrmap e10: x(active, starting, since 0.0037399s) 2024-09-10T09:28:05.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:28:05 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T09:28:05.807 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:28:05 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-10T09:28:05.807 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:28:05 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-10T09:28:05.807 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:28:05 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-10T09:28:05.807 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:28:05 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-10T09:28:05.807 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:28:05 smithi005 ceph-mon[30350]: Manager daemon x is now available 2024-09-10T09:28:05.807 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:28:05 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:28:06.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:28:06 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-10T09:28:06.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:28:06 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-10T09:28:06.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:28:06 smithi005 ceph-mon[30350]: mgrmap e11: x(active, since 1.00758s) 2024-09-10T09:28:06.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:28:06 smithi005 ceph-mon[30350]: from='client.14154 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-09-10T09:28:06.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:28:06 smithi005 ceph-mon[30350]: from='client.14154 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-09-10T09:28:07.628 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout { 2024-09-10T09:28:07.628 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-09-10T09:28:07.628 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "initialized": true 2024-09-10T09:28:07.628 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout } 2024-09-10T09:28:07.628 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:02.190+0000 7f3747200640 1 Processor -- start 2024-09-10T09:28:07.628 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:02.190+0000 7f3747200640 1 -- start start 2024-09-10T09:28:07.628 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:02.190+0000 7f3747200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3748107440 0x7f3748107840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:28:07.628 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:02.190+0000 7f3747200640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f3748107e10 con 0x7f3748107440 2024-09-10T09:28:07.628 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:02.191+0000 7f3745e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3748107440 0x7f3748107840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:28:07.628 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:02.191+0000 7f3745e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3748107440 0x7f3748107840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50126/0 (socket says 172.21.15.5:50126) 2024-09-10T09:28:07.628 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:02.191+0000 7f3745e00640 1 -- 172.21.15.5:0/836396048 learned_addr learned my addr 172.21.15.5:0/836396048 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:28:07.629 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:02.191+0000 7f3745e00640 1 -- 172.21.15.5:0/836396048 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3748108640 con 0x7f3748107440 2024-09-10T09:28:07.629 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:02.193+0000 7f3745e00640 1 --2- 172.21.15.5:0/836396048 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3748107440 0x7f3748107840 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f3738009820 tx=0x7f373802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfac938d576774ea server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:28:07.629 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:02.193+0000 7f3745400640 1 -- 172.21.15.5:0/836396048 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3738037070 con 0x7f3748107440 2024-09-10T09:28:07.629 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:02.193+0000 7f3745400640 1 -- 172.21.15.5:0/836396048 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3738032920 con 0x7f3748107440 2024-09-10T09:28:07.629 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:02.193+0000 7f3745400640 1 -- 172.21.15.5:0/836396048 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3738030810 con 0x7f3748107440 2024-09-10T09:28:07.629 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:02.194+0000 7f3747200640 1 -- 172.21.15.5:0/836396048 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3748107440 msgr2=0x7f3748107840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:28:07.629 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:02.194+0000 7f3747200640 1 --2- 172.21.15.5:0/836396048 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3748107440 0x7f3748107840 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f3738009820 tx=0x7f373802a270 comp rx=0 tx=0).stop 2024-09-10T09:28:07.629 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:02.194+0000 7f3747200640 1 -- 172.21.15.5:0/836396048 shutdown_connections 2024-09-10T09:28:07.629 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:02.194+0000 7f3747200640 1 --2- 172.21.15.5:0/836396048 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3748107440 0x7f3748107840 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:28:07.629 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:02.194+0000 7f3747200640 1 -- 172.21.15.5:0/836396048 >> 172.21.15.5:0/836396048 conn(0x7f3748102c10 msgr2=0x7f3748105030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:28:07.629 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:02.194+0000 7f3747200640 1 -- 172.21.15.5:0/836396048 shutdown_connections 2024-09-10T09:28:07.629 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:02.194+0000 7f3747200640 1 -- 172.21.15.5:0/836396048 wait complete. 2024-09-10T09:28:07.630 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:02.195+0000 7f3747200640 1 Processor -- start 2024-09-10T09:28:07.630 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:02.195+0000 7f3747200640 1 -- start start 2024-09-10T09:28:07.630 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:02.195+0000 7f3747200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3748107440 0x7f374819d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:28:07.630 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:02.195+0000 7f3747200640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f374807c260 con 0x7f3748107440 2024-09-10T09:28:07.630 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:02.196+0000 7f3745e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3748107440 0x7f374819d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:28:07.630 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:02.196+0000 7f3745e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3748107440 0x7f374819d6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50134/0 (socket says 172.21.15.5:50134) 2024-09-10T09:28:07.630 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:02.196+0000 7f3745e00640 1 -- 172.21.15.5:0/3384621393 learned_addr learned my addr 172.21.15.5:0/3384621393 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:28:07.630 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:02.196+0000 7f3745e00640 1 -- 172.21.15.5:0/3384621393 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f374819dc30 con 0x7f3748107440 2024-09-10T09:28:07.630 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:02.196+0000 7f3745e00640 1 --2- 172.21.15.5:0/3384621393 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3748107440 0x7f374819d6f0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f37380097f0 tx=0x7f37380327b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:28:07.630 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:02.197+0000 7f373ea00640 1 -- 172.21.15.5:0/3384621393 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3738037040 con 0x7f3748107440 2024-09-10T09:28:07.630 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:02.197+0000 7f373ea00640 1 -- 172.21.15.5:0/3384621393 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3738030c40 con 0x7f3748107440 2024-09-10T09:28:07.630 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:02.197+0000 7f3747200640 1 -- 172.21.15.5:0/3384621393 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f374819dec0 con 0x7f3748107440 2024-09-10T09:28:07.631 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:02.197+0000 7f373ea00640 1 -- 172.21.15.5:0/3384621393 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f373803a5b0 con 0x7f3748107440 2024-09-10T09:28:07.631 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:02.197+0000 7f3747200640 1 -- 172.21.15.5:0/3384621393 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37481a0bb0 con 0x7f3748107440 2024-09-10T09:28:07.631 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:02.198+0000 7f373ea00640 1 -- 172.21.15.5:0/3384621393 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 9) ==== 46300+0+0 (secure 0 0 0) 0x7f373803a7a0 con 0x7f3748107440 2024-09-10T09:28:07.631 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:02.199+0000 7f373ea00640 1 --2- 172.21.15.5:0/3384621393 >> [v2:172.21.15.5:6800/1527422686,v1:172.21.15.5:6801/1527422686] conn(0x7f372c039020 0x7f372c03b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:28:07.631 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:02.199+0000 7f373ea00640 1 -- 172.21.15.5:0/3384621393 --> [v2:172.21.15.5:6800/1527422686,v1:172.21.15.5:6801/1527422686] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f372c03bbf0 con 0x7f372c039020 2024-09-10T09:28:07.631 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:02.199+0000 7f373ea00640 1 -- 172.21.15.5:0/3384621393 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f373806c4c0 con 0x7f3748107440 2024-09-10T09:28:07.631 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:02.199+0000 7f373fe00640 1 -- 172.21.15.5:0/3384621393 >> [v2:172.21.15.5:6800/1527422686,v1:172.21.15.5:6801/1527422686] conn(0x7f372c039020 msgr2=0x7f372c03b4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.5:6800/1527422686 2024-09-10T09:28:07.631 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:02.199+0000 7f373fe00640 1 --2- 172.21.15.5:0/3384621393 >> [v2:172.21.15.5:6800/1527422686,v1:172.21.15.5:6801/1527422686] conn(0x7f372c039020 0x7f372c03b4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T09:28:07.631 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:02.400+0000 7f373fe00640 1 -- 172.21.15.5:0/3384621393 >> [v2:172.21.15.5:6800/1527422686,v1:172.21.15.5:6801/1527422686] conn(0x7f372c039020 msgr2=0x7f372c03b4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.5:6800/1527422686 2024-09-10T09:28:07.631 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:02.400+0000 7f373fe00640 1 --2- 172.21.15.5:0/3384621393 >> [v2:172.21.15.5:6800/1527422686,v1:172.21.15.5:6801/1527422686] conn(0x7f372c039020 0x7f372c03b4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-10T09:28:07.631 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:02.801+0000 7f373fe00640 1 -- 172.21.15.5:0/3384621393 >> [v2:172.21.15.5:6800/1527422686,v1:172.21.15.5:6801/1527422686] conn(0x7f372c039020 msgr2=0x7f372c03b4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.5:6800/1527422686 2024-09-10T09:28:07.631 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:02.801+0000 7f373fe00640 1 --2- 172.21.15.5:0/3384621393 >> [v2:172.21.15.5:6800/1527422686,v1:172.21.15.5:6801/1527422686] conn(0x7f372c039020 0x7f372c03b4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-10T09:28:07.632 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:03.602+0000 7f373fe00640 1 -- 172.21.15.5:0/3384621393 >> [v2:172.21.15.5:6800/1527422686,v1:172.21.15.5:6801/1527422686] conn(0x7f372c039020 msgr2=0x7f372c03b4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.5:6800/1527422686 2024-09-10T09:28:07.632 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:03.602+0000 7f373fe00640 1 --2- 172.21.15.5:0/3384621393 >> [v2:172.21.15.5:6800/1527422686,v1:172.21.15.5:6801/1527422686] conn(0x7f372c039020 0x7f372c03b4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-10T09:28:07.632 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:05.203+0000 7f373fe00640 1 -- 172.21.15.5:0/3384621393 >> [v2:172.21.15.5:6800/1527422686,v1:172.21.15.5:6801/1527422686] conn(0x7f372c039020 msgr2=0x7f372c03b4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.5:6800/1527422686 2024-09-10T09:28:07.632 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:05.204+0000 7f373fe00640 1 --2- 172.21.15.5:0/3384621393 >> [v2:172.21.15.5:6800/1527422686,v1:172.21.15.5:6801/1527422686] conn(0x7f372c039020 0x7f372c03b4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-09-10T09:28:07.632 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:05.438+0000 7f373ea00640 1 -- 172.21.15.5:0/3384621393 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mgrmap(e 10) ==== 46067+0+0 (secure 0 0 0) 0x7f373802fd60 con 0x7f3748107440 2024-09-10T09:28:07.632 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:05.438+0000 7f373ea00640 1 -- 172.21.15.5:0/3384621393 >> [v2:172.21.15.5:6800/1527422686,v1:172.21.15.5:6801/1527422686] conn(0x7f372c039020 msgr2=0x7f372c03b4e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T09:28:07.632 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:05.438+0000 7f373ea00640 1 --2- 172.21.15.5:0/3384621393 >> [v2:172.21.15.5:6800/1527422686,v1:172.21.15.5:6801/1527422686] conn(0x7f372c039020 0x7f372c03b4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:28:07.632 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:06.441+0000 7f373ea00640 1 -- 172.21.15.5:0/3384621393 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mgrmap(e 11) ==== 46194+0+0 (secure 0 0 0) 0x7f373806e9d0 con 0x7f3748107440 2024-09-10T09:28:07.632 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:06.441+0000 7f373ea00640 1 --2- 172.21.15.5:0/3384621393 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f372c03cae0 0x7f372c03eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:28:07.632 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:06.441+0000 7f373ea00640 1 -- 172.21.15.5:0/3384621393 --> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f3738046e70 con 0x7f372c03cae0 2024-09-10T09:28:07.632 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:06.442+0000 7f373fe00640 1 --2- 172.21.15.5:0/3384621393 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f372c03cae0 0x7f372c03eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:28:07.632 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:06.443+0000 7f373fe00640 1 --2- 172.21.15.5:0/3384621393 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f372c03cae0 0x7f372c03eed0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f3730003980 tx=0x7f37300092b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:28:07.632 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:06.444+0000 7f373ea00640 1 -- 172.21.15.5:0/3384621393 <== mgr.14150 v2:172.21.15.5:6800/2707348081 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9564 (secure 0 0 0) 0x7f3738046e70 con 0x7f372c03cae0 2024-09-10T09:28:07.633 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:06.452+0000 7f3747200640 1 -- 172.21.15.5:0/3384621393 --> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f3748107840 con 0x7f372c03cae0 2024-09-10T09:28:07.633 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:06.452+0000 7f373ea00640 1 -- 172.21.15.5:0/3384621393 <== mgr.14150 v2:172.21.15.5:6800/2707348081 2 ==== command_reply(tid 1: 0 ) ==== 8+0+52 (secure 0 0 0) 0x7f3748107840 con 0x7f372c03cae0 2024-09-10T09:28:07.633 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:06.452+0000 7f3747200640 1 -- 172.21.15.5:0/3384621393 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f372c03cae0 msgr2=0x7f372c03eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:28:07.633 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:06.452+0000 7f3747200640 1 --2- 172.21.15.5:0/3384621393 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f372c03cae0 0x7f372c03eed0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f3730003980 tx=0x7f37300092b0 comp rx=0 tx=0).stop 2024-09-10T09:28:07.633 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:06.452+0000 7f3747200640 1 -- 172.21.15.5:0/3384621393 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3748107440 msgr2=0x7f374819d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:28:07.633 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:06.452+0000 7f3747200640 1 --2- 172.21.15.5:0/3384621393 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3748107440 0x7f374819d6f0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f37380097f0 tx=0x7f37380327b0 comp rx=0 tx=0).stop 2024-09-10T09:28:07.633 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:06.453+0000 7f3747200640 1 -- 172.21.15.5:0/3384621393 shutdown_connections 2024-09-10T09:28:07.633 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:06.453+0000 7f3747200640 1 --2- 172.21.15.5:0/3384621393 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f372c03cae0 0x7f372c03eed0 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:28:07.633 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:06.453+0000 7f3747200640 1 --2- 172.21.15.5:0/3384621393 >> [v2:172.21.15.5:6800/1527422686,v1:172.21.15.5:6801/1527422686] conn(0x7f372c039020 0x7f372c03b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:28:07.633 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:06.453+0000 7f3747200640 1 --2- 172.21.15.5:0/3384621393 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3748107440 0x7f374819d6f0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:28:07.633 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:06.453+0000 7f3747200640 1 -- 172.21.15.5:0/3384621393 >> 172.21.15.5:0/3384621393 conn(0x7f3748102c10 msgr2=0x7f3748194990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:28:07.633 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:06.453+0000 7f3747200640 1 -- 172.21.15.5:0/3384621393 shutdown_connections 2024-09-10T09:28:07.633 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:06.453+0000 7f3747200640 1 -- 172.21.15.5:0/3384621393 wait complete. 2024-09-10T09:28:07.634 INFO:teuthology.orchestra.run.smithi005.stdout:mgr epoch 9 is available 2024-09-10T09:28:07.634 INFO:teuthology.orchestra.run.smithi005.stdout:Generating a dashboard self-signed certificate... 2024-09-10T09:28:07.761 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:28:07 smithi005 ceph-mon[30350]: [10/Sep/2024:09:28:07] ENGINE Bus STARTING 2024-09-10T09:28:07.761 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:28:07 smithi005 ceph-mon[30350]: [10/Sep/2024:09:28:07] ENGINE Serving on http://172.21.15.5:8765 2024-09-10T09:28:07.761 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 10 09:28:07 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mgr-x[30575]: 2024-09-10T09:28:07.523+0000 7f5660000640 -1 log_channel(cephadm) log [ERR] : [10/Sep/2024:09:28:07] ENGINE Error in HTTPServer.serve 2024-09-10T09:28:07.762 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 10 09:28:07 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mgr-x[30575]: Traceback (most recent call last): 2024-09-10T09:28:07.762 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 10 09:28:07 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mgr-x[30575]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-10T09:28:07.762 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 10 09:28:07 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mgr-x[30575]: self._connections.run(self.expiration_interval) 2024-09-10T09:28:07.762 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 10 09:28:07 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mgr-x[30575]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-10T09:28:07.762 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 10 09:28:07 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mgr-x[30575]: self._run(expiration_interval) 2024-09-10T09:28:07.762 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 10 09:28:07 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mgr-x[30575]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-10T09:28:07.762 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 10 09:28:07 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mgr-x[30575]: new_conn = self._from_server_socket(self.server.socket) 2024-09-10T09:28:07.762 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 10 09:28:07 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mgr-x[30575]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-10T09:28:07.762 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 10 09:28:07 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mgr-x[30575]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-10T09:28:07.762 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 10 09:28:07 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mgr-x[30575]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-10T09:28:07.762 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 10 09:28:07 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mgr-x[30575]: s = self.context.wrap_socket( 2024-09-10T09:28:07.762 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 10 09:28:07 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mgr-x[30575]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-10T09:28:07.762 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 10 09:28:07 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mgr-x[30575]: return self.sslsocket_class._create( 2024-09-10T09:28:07.763 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 10 09:28:07 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mgr-x[30575]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-10T09:28:07.763 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 10 09:28:07 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mgr-x[30575]: self.do_handshake() 2024-09-10T09:28:07.763 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 10 09:28:07 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mgr-x[30575]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-10T09:28:07.763 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 10 09:28:07 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mgr-x[30575]: self._sslobj.do_handshake() 2024-09-10T09:28:07.763 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 10 09:28:07 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mgr-x[30575]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-10T09:28:07.763 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 10 09:28:07 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mgr-x[30575]: 2024-09-10T09:28:08.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:28:08 smithi005 ceph-mon[30350]: mgrmap e12: x(active, since 2s) 2024-09-10T09:28:08.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:28:08 smithi005 ceph-mon[30350]: [10/Sep/2024:09:28:07] ENGINE Serving on https://172.21.15.5:7150 2024-09-10T09:28:08.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:28:08 smithi005 ceph-mon[30350]: [10/Sep/2024:09:28:07] ENGINE Bus STARTED 2024-09-10T09:28:08.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:28:08 smithi005 ceph-mon[30350]: [10/Sep/2024:09:28:07] ENGINE Error in HTTPServer.serve 2024-09-10T09:28:08.806 INFO:journalctl@ceph.mon.a.smithi005.stdout: Traceback (most recent call last): 2024-09-10T09:28:08.806 INFO:journalctl@ceph.mon.a.smithi005.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-10T09:28:08.807 INFO:journalctl@ceph.mon.a.smithi005.stdout: self._connections.run(self.expiration_interval) 2024-09-10T09:28:08.807 INFO:journalctl@ceph.mon.a.smithi005.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-10T09:28:08.807 INFO:journalctl@ceph.mon.a.smithi005.stdout: self._run(expiration_interval) 2024-09-10T09:28:08.807 INFO:journalctl@ceph.mon.a.smithi005.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-10T09:28:08.807 INFO:journalctl@ceph.mon.a.smithi005.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-10T09:28:08.807 INFO:journalctl@ceph.mon.a.smithi005.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-10T09:28:08.807 INFO:journalctl@ceph.mon.a.smithi005.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-10T09:28:08.807 INFO:journalctl@ceph.mon.a.smithi005.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-10T09:28:08.807 INFO:journalctl@ceph.mon.a.smithi005.stdout: s = self.context.wrap_socket( 2024-09-10T09:28:08.807 INFO:journalctl@ceph.mon.a.smithi005.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-10T09:28:08.807 INFO:journalctl@ceph.mon.a.smithi005.stdout: return self.sslsocket_class._create( 2024-09-10T09:28:08.807 INFO:journalctl@ceph.mon.a.smithi005.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-10T09:28:08.807 INFO:journalctl@ceph.mon.a.smithi005.stdout: self.do_handshake() 2024-09-10T09:28:08.807 INFO:journalctl@ceph.mon.a.smithi005.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-10T09:28:08.808 INFO:journalctl@ceph.mon.a.smithi005.stdout: self._sslobj.do_handshake() 2024-09-10T09:28:08.808 INFO:journalctl@ceph.mon.a.smithi005.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-10T09:28:08.808 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:28:08 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:28:08.808 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:28:08 smithi005 ceph-mon[30350]: from='client.14162 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T09:28:08.808 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:28:08 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:28:08.808 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:28:08 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:28:09.181 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-09-10T09:28:09.181 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:08.145+0000 7efe8e200640 1 Processor -- start 2024-09-10T09:28:09.181 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:08.145+0000 7efe8e200640 1 -- start start 2024-09-10T09:28:09.182 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:08.145+0000 7efe8e200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efe881098c0 0x7efe88109cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:28:09.182 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:08.145+0000 7efe8e200640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7efe8810a290 con 0x7efe881098c0 2024-09-10T09:28:09.182 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:08.146+0000 7efe87400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efe881098c0 0x7efe88109cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:28:09.182 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:08.146+0000 7efe87400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efe881098c0 0x7efe88109cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50188/0 (socket says 172.21.15.5:50188) 2024-09-10T09:28:09.182 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:08.146+0000 7efe87400640 1 -- 172.21.15.5:0/319236833 learned_addr learned my addr 172.21.15.5:0/319236833 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:28:09.182 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:08.146+0000 7efe87400640 1 -- 172.21.15.5:0/319236833 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe8810aa70 con 0x7efe881098c0 2024-09-10T09:28:09.182 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:08.147+0000 7efe87400640 1 --2- 172.21.15.5:0/319236833 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efe881098c0 0x7efe88109cc0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7efe78009820 tx=0x7efe7802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81998a8b08497da9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:28:09.182 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:08.148+0000 7efe86000640 1 -- 172.21.15.5:0/319236833 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efe78037070 con 0x7efe881098c0 2024-09-10T09:28:09.182 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:08.148+0000 7efe86000640 1 -- 172.21.15.5:0/319236833 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efe78032920 con 0x7efe881098c0 2024-09-10T09:28:09.182 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:08.148+0000 7efe86000640 1 -- 172.21.15.5:0/319236833 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efe78030810 con 0x7efe881098c0 2024-09-10T09:28:09.182 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:08.148+0000 7efe8e200640 1 -- 172.21.15.5:0/319236833 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efe881098c0 msgr2=0x7efe88109cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:28:09.182 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:08.148+0000 7efe8e200640 1 --2- 172.21.15.5:0/319236833 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efe881098c0 0x7efe88109cc0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7efe78009820 tx=0x7efe7802a270 comp rx=0 tx=0).stop 2024-09-10T09:28:09.182 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:08.148+0000 7efe8e200640 1 -- 172.21.15.5:0/319236833 shutdown_connections 2024-09-10T09:28:09.183 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:08.148+0000 7efe8e200640 1 --2- 172.21.15.5:0/319236833 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efe881098c0 0x7efe88109cc0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:28:09.183 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:08.148+0000 7efe8e200640 1 -- 172.21.15.5:0/319236833 >> 172.21.15.5:0/319236833 conn(0x7efe8807cf40 msgr2=0x7efe8807d350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:28:09.183 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:08.149+0000 7efe8e200640 1 -- 172.21.15.5:0/319236833 shutdown_connections 2024-09-10T09:28:09.183 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:08.149+0000 7efe8e200640 1 -- 172.21.15.5:0/319236833 wait complete. 2024-09-10T09:28:09.183 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:08.149+0000 7efe8e200640 1 Processor -- start 2024-09-10T09:28:09.183 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:08.150+0000 7efe8e200640 1 -- start start 2024-09-10T09:28:09.183 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:08.150+0000 7efe8e200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efe881098c0 0x7efe8819d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:28:09.183 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:08.150+0000 7efe8e200640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7efe8810de20 con 0x7efe881098c0 2024-09-10T09:28:09.183 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:08.150+0000 7efe87400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efe881098c0 0x7efe8819d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:28:09.183 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:08.150+0000 7efe87400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efe881098c0 0x7efe8819d8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50196/0 (socket says 172.21.15.5:50196) 2024-09-10T09:28:09.183 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:08.150+0000 7efe87400640 1 -- 172.21.15.5:0/1549210142 learned_addr learned my addr 172.21.15.5:0/1549210142 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:28:09.183 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:08.150+0000 7efe87400640 1 -- 172.21.15.5:0/1549210142 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe8819ddf0 con 0x7efe881098c0 2024-09-10T09:28:09.183 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:08.151+0000 7efe87400640 1 --2- 172.21.15.5:0/1549210142 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efe881098c0 0x7efe8819d8b0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7efe78030560 tx=0x7efe78032670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:28:09.183 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:08.151+0000 7efe7fe00640 1 -- 172.21.15.5:0/1549210142 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efe78037040 con 0x7efe881098c0 2024-09-10T09:28:09.184 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:08.151+0000 7efe7fe00640 1 -- 172.21.15.5:0/1549210142 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efe78040070 con 0x7efe881098c0 2024-09-10T09:28:09.184 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:08.151+0000 7efe7fe00640 1 -- 172.21.15.5:0/1549210142 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efe7803a740 con 0x7efe881098c0 2024-09-10T09:28:09.184 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:08.151+0000 7efe8e200640 1 -- 172.21.15.5:0/1549210142 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe8819e080 con 0x7efe881098c0 2024-09-10T09:28:09.184 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:08.151+0000 7efe8e200640 1 -- 172.21.15.5:0/1549210142 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe881a0d70 con 0x7efe881098c0 2024-09-10T09:28:09.184 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:08.152+0000 7efe7fe00640 1 -- 172.21.15.5:0/1549210142 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 12) ==== 46300+0+0 (secure 0 0 0) 0x7efe7803bb90 con 0x7efe881098c0 2024-09-10T09:28:09.184 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:08.152+0000 7efe8e200640 1 -- 172.21.15.5:0/1549210142 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe88109d40 con 0x7efe881098c0 2024-09-10T09:28:09.184 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:08.153+0000 7efe7fe00640 1 --2- 172.21.15.5:0/1549210142 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7efe6c039020 0x7efe6c03b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:28:09.184 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:08.153+0000 7efe7fe00640 1 -- 172.21.15.5:0/1549210142 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7efe7806c720 con 0x7efe881098c0 2024-09-10T09:28:09.184 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:08.153+0000 7efe86a00640 1 --2- 172.21.15.5:0/1549210142 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7efe6c039020 0x7efe6c03b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:28:09.184 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:08.154+0000 7efe86a00640 1 --2- 172.21.15.5:0/1549210142 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7efe6c039020 0x7efe6c03b4e0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7efe740098c0 tx=0x7efe74006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:28:09.184 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:08.162+0000 7efe7fe00640 1 -- 172.21.15.5:0/1549210142 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7efe7804a200 con 0x7efe881098c0 2024-09-10T09:28:09.184 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:08.356+0000 7efe8e200640 1 -- 172.21.15.5:0/1549210142 --> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) -- 0x7efe88107c60 con 0x7efe6c039020 2024-09-10T09:28:09.184 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:08.461+0000 7efe7fe00640 1 -- 172.21.15.5:0/1549210142 <== mgr.14150 v2:172.21.15.5:6800/2707348081 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+32 (secure 0 0 0) 0x7efe88107c60 con 0x7efe6c039020 2024-09-10T09:28:09.185 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:08.464+0000 7efe8e200640 1 -- 172.21.15.5:0/1549210142 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7efe6c039020 msgr2=0x7efe6c03b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:28:09.185 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:08.464+0000 7efe8e200640 1 --2- 172.21.15.5:0/1549210142 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7efe6c039020 0x7efe6c03b4e0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7efe740098c0 tx=0x7efe74006db0 comp rx=0 tx=0).stop 2024-09-10T09:28:09.185 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:08.464+0000 7efe8e200640 1 -- 172.21.15.5:0/1549210142 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efe881098c0 msgr2=0x7efe8819d8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:28:09.185 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:08.464+0000 7efe8e200640 1 --2- 172.21.15.5:0/1549210142 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efe881098c0 0x7efe8819d8b0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7efe78030560 tx=0x7efe78032670 comp rx=0 tx=0).stop 2024-09-10T09:28:09.185 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:08.464+0000 7efe8e200640 1 -- 172.21.15.5:0/1549210142 shutdown_connections 2024-09-10T09:28:09.185 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:08.464+0000 7efe8e200640 1 --2- 172.21.15.5:0/1549210142 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7efe6c039020 0x7efe6c03b4e0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:28:09.185 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:08.464+0000 7efe8e200640 1 --2- 172.21.15.5:0/1549210142 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efe881098c0 0x7efe8819d8b0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:28:09.185 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:08.464+0000 7efe8e200640 1 -- 172.21.15.5:0/1549210142 >> 172.21.15.5:0/1549210142 conn(0x7efe8807cf40 msgr2=0x7efe88107550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:28:09.185 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:08.464+0000 7efe8e200640 1 -- 172.21.15.5:0/1549210142 shutdown_connections 2024-09-10T09:28:09.185 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:08.464+0000 7efe8e200640 1 -- 172.21.15.5:0/1549210142 wait complete. 2024-09-10T09:28:09.185 INFO:teuthology.orchestra.run.smithi005.stdout:Creating initial admin user... 2024-09-10T09:28:10.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:28:10 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:28:10.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:28:10 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:28:11.468 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:28:11 smithi005 ceph-mon[30350]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T09:28:11.478 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$YigWAg9JFyrbEywcQXjHqeiT8dtl.SXDRRFdrA0vkhcEVSdYqvqJS", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1725960490, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-09-10T09:28:11.478 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:09.721+0000 7ff85f200640 1 Processor -- start 2024-09-10T09:28:11.478 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:09.722+0000 7ff85f200640 1 -- start start 2024-09-10T09:28:11.478 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:09.722+0000 7ff85f200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff860109970 0x7ff860109d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:28:11.478 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:09.722+0000 7ff85f200640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff86010a340 con 0x7ff860109970 2024-09-10T09:28:11.478 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:09.722+0000 7ff85de00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff860109970 0x7ff860109d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:28:11.478 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:09.722+0000 7ff85de00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff860109970 0x7ff860109d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50210/0 (socket says 172.21.15.5:50210) 2024-09-10T09:28:11.478 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:09.722+0000 7ff85de00640 1 -- 172.21.15.5:0/3223943474 learned_addr learned my addr 172.21.15.5:0/3223943474 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:28:11.478 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:09.723+0000 7ff85de00640 1 -- 172.21.15.5:0/3223943474 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff86010a840 con 0x7ff860109970 2024-09-10T09:28:11.478 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:09.724+0000 7ff85de00640 1 --2- 172.21.15.5:0/3223943474 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff860109970 0x7ff860109d70 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7ff850009e80 tx=0x7ff85002a7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8f055c8f16171fe server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:28:11.478 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:09.724+0000 7ff85d400640 1 -- 172.21.15.5:0/3223943474 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff850036070 con 0x7ff860109970 2024-09-10T09:28:11.479 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:09.724+0000 7ff85d400640 1 -- 172.21.15.5:0/3223943474 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff8500318e0 con 0x7ff860109970 2024-09-10T09:28:11.479 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:09.724+0000 7ff85d400640 1 -- 172.21.15.5:0/3223943474 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff850007a00 con 0x7ff860109970 2024-09-10T09:28:11.479 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:09.725+0000 7ff85f200640 1 -- 172.21.15.5:0/3223943474 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff860109970 msgr2=0x7ff860109d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:28:11.479 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:09.725+0000 7ff85f200640 1 --2- 172.21.15.5:0/3223943474 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff860109970 0x7ff860109d70 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7ff850009e80 tx=0x7ff85002a7a0 comp rx=0 tx=0).stop 2024-09-10T09:28:11.479 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:09.725+0000 7ff85f200640 1 -- 172.21.15.5:0/3223943474 shutdown_connections 2024-09-10T09:28:11.479 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:09.725+0000 7ff85f200640 1 --2- 172.21.15.5:0/3223943474 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff860109970 0x7ff860109d70 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:28:11.479 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:09.725+0000 7ff85f200640 1 -- 172.21.15.5:0/3223943474 >> 172.21.15.5:0/3223943474 conn(0x7ff86007cca0 msgr2=0x7ff86007d0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:28:11.479 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:09.725+0000 7ff85f200640 1 -- 172.21.15.5:0/3223943474 shutdown_connections 2024-09-10T09:28:11.479 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:09.725+0000 7ff85f200640 1 -- 172.21.15.5:0/3223943474 wait complete. 2024-09-10T09:28:11.479 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:09.726+0000 7ff85f200640 1 Processor -- start 2024-09-10T09:28:11.479 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:09.727+0000 7ff85f200640 1 -- start start 2024-09-10T09:28:11.479 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:09.727+0000 7ff85f200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff860109970 0x7ff86019d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:28:11.479 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:09.727+0000 7ff85f200640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff86010dbf0 con 0x7ff860109970 2024-09-10T09:28:11.480 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:09.727+0000 7ff85de00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff860109970 0x7ff86019d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:28:11.480 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:09.727+0000 7ff85de00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff860109970 0x7ff86019d6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50214/0 (socket says 172.21.15.5:50214) 2024-09-10T09:28:11.480 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:09.727+0000 7ff85de00640 1 -- 172.21.15.5:0/2980542896 learned_addr learned my addr 172.21.15.5:0/2980542896 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:28:11.480 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:09.727+0000 7ff85de00640 1 -- 172.21.15.5:0/2980542896 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff86019dc30 con 0x7ff860109970 2024-09-10T09:28:11.480 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:09.728+0000 7ff85de00640 1 --2- 172.21.15.5:0/2980542896 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff860109970 0x7ff86019d6f0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7ff85002acc0 tx=0x7ff850031cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:28:11.480 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:09.728+0000 7ff857400640 1 -- 172.21.15.5:0/2980542896 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff850036050 con 0x7ff860109970 2024-09-10T09:28:11.480 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:09.728+0000 7ff857400640 1 -- 172.21.15.5:0/2980542896 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff850037040 con 0x7ff860109970 2024-09-10T09:28:11.480 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:09.728+0000 7ff857400640 1 -- 172.21.15.5:0/2980542896 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff85003b920 con 0x7ff860109970 2024-09-10T09:28:11.480 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:09.728+0000 7ff85f200640 1 -- 172.21.15.5:0/2980542896 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff86019dec0 con 0x7ff860109970 2024-09-10T09:28:11.480 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:09.728+0000 7ff85f200640 1 -- 172.21.15.5:0/2980542896 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8601a0bb0 con 0x7ff860109970 2024-09-10T09:28:11.480 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:09.729+0000 7ff857400640 1 -- 172.21.15.5:0/2980542896 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 12) ==== 46300+0+0 (secure 0 0 0) 0x7ff850044020 con 0x7ff860109970 2024-09-10T09:28:11.480 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:09.730+0000 7ff85f200640 1 -- 172.21.15.5:0/2980542896 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff860109df0 con 0x7ff860109970 2024-09-10T09:28:11.480 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:09.730+0000 7ff857400640 1 --2- 172.21.15.5:0/2980542896 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7ff844038fc0 0x7ff84403b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:28:11.481 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:09.730+0000 7ff857400640 1 -- 172.21.15.5:0/2980542896 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7ff850042020 con 0x7ff860109970 2024-09-10T09:28:11.481 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:09.730+0000 7ff855400640 1 --2- 172.21.15.5:0/2980542896 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7ff844038fc0 0x7ff84403b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:28:11.481 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:09.731+0000 7ff855400640 1 --2- 172.21.15.5:0/2980542896 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7ff844038fc0 0x7ff84403b480 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7ff8480098c0 tx=0x7ff848006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:28:11.481 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:09.736+0000 7ff857400640 1 -- 172.21.15.5:0/2980542896 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff85004b020 con 0x7ff860109970 2024-09-10T09:28:11.481 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:09.943+0000 7ff85f200640 1 -- 172.21.15.5:0/2980542896 --> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) -- 0x7ff8601a0e40 con 0x7ff844038fc0 2024-09-10T09:28:11.481 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:10.193+0000 7ff857400640 1 -- 172.21.15.5:0/2980542896 <== mgr.14150 v2:172.21.15.5:6800/2707348081 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+252 (secure 0 0 0) 0x7ff8601a0e40 con 0x7ff844038fc0 2024-09-10T09:28:11.481 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:10.196+0000 7ff85f200640 1 -- 172.21.15.5:0/2980542896 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7ff844038fc0 msgr2=0x7ff84403b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:28:11.481 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:10.196+0000 7ff85f200640 1 --2- 172.21.15.5:0/2980542896 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7ff844038fc0 0x7ff84403b480 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7ff8480098c0 tx=0x7ff848006db0 comp rx=0 tx=0).stop 2024-09-10T09:28:11.481 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:10.196+0000 7ff85f200640 1 -- 172.21.15.5:0/2980542896 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff860109970 msgr2=0x7ff86019d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:28:11.481 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:10.196+0000 7ff85f200640 1 --2- 172.21.15.5:0/2980542896 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff860109970 0x7ff86019d6f0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7ff85002acc0 tx=0x7ff850031cc0 comp rx=0 tx=0).stop 2024-09-10T09:28:11.481 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:10.196+0000 7ff85f200640 1 -- 172.21.15.5:0/2980542896 shutdown_connections 2024-09-10T09:28:11.481 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:10.196+0000 7ff85f200640 1 --2- 172.21.15.5:0/2980542896 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7ff844038fc0 0x7ff84403b480 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:28:11.481 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:10.196+0000 7ff85f200640 1 --2- 172.21.15.5:0/2980542896 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff860109970 0x7ff86019d6f0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:28:11.482 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:10.196+0000 7ff85f200640 1 -- 172.21.15.5:0/2980542896 >> 172.21.15.5:0/2980542896 conn(0x7ff86007cca0 msgr2=0x7ff8601075c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:28:11.482 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:10.196+0000 7ff85f200640 1 -- 172.21.15.5:0/2980542896 shutdown_connections 2024-09-10T09:28:11.482 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:10.196+0000 7ff85f200640 1 -- 172.21.15.5:0/2980542896 wait complete. 2024-09-10T09:28:11.482 INFO:teuthology.orchestra.run.smithi005.stdout:Fetching dashboard port number... 2024-09-10T09:28:12.513 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:28:12 smithi005 ceph-mon[30350]: from='client.? 172.21.15.5:0/1572429553' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-09-10T09:28:12.776 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout 8443 2024-09-10T09:28:12.776 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:11.963+0000 7f4292600640 1 Processor -- start 2024-09-10T09:28:12.776 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:11.963+0000 7f4292600640 1 -- start start 2024-09-10T09:28:12.776 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:11.964+0000 7f4292600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f428c107680 0x7f428c107a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:28:12.776 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:11.964+0000 7f4292600640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f428c108050 con 0x7f428c107680 2024-09-10T09:28:12.776 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:11.964+0000 7f428b400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f428c107680 0x7f428c107a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:28:12.776 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:11.964+0000 7f428b400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f428c107680 0x7f428c107a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53712/0 (socket says 172.21.15.5:53712) 2024-09-10T09:28:12.776 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:11.964+0000 7f428b400640 1 -- 172.21.15.5:0/283807685 learned_addr learned my addr 172.21.15.5:0/283807685 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:28:12.776 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:11.965+0000 7f428b400640 1 -- 172.21.15.5:0/283807685 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f428c1087e0 con 0x7f428c107680 2024-09-10T09:28:12.777 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:11.966+0000 7f428b400640 1 --2- 172.21.15.5:0/283807685 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f428c107680 0x7f428c107a80 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f427c009820 tx=0x7f427c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4873f01f456496e0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:28:12.777 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:11.966+0000 7f428aa00640 1 -- 172.21.15.5:0/283807685 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f427c037070 con 0x7f428c107680 2024-09-10T09:28:12.777 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:11.966+0000 7f428aa00640 1 -- 172.21.15.5:0/283807685 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f427c032920 con 0x7f428c107680 2024-09-10T09:28:12.777 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:11.966+0000 7f428aa00640 1 -- 172.21.15.5:0/283807685 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f427c030810 con 0x7f428c107680 2024-09-10T09:28:12.777 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:11.967+0000 7f4292600640 1 -- 172.21.15.5:0/283807685 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f428c107680 msgr2=0x7f428c107a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:28:12.777 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:11.967+0000 7f4292600640 1 --2- 172.21.15.5:0/283807685 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f428c107680 0x7f428c107a80 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f427c009820 tx=0x7f427c02a270 comp rx=0 tx=0).stop 2024-09-10T09:28:12.777 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:11.967+0000 7f4292600640 1 -- 172.21.15.5:0/283807685 shutdown_connections 2024-09-10T09:28:12.777 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:11.967+0000 7f4292600640 1 --2- 172.21.15.5:0/283807685 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f428c107680 0x7f428c107a80 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:28:12.777 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:11.967+0000 7f4292600640 1 -- 172.21.15.5:0/283807685 >> 172.21.15.5:0/283807685 conn(0x7f428c102e10 msgr2=0x7f428c105250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:28:12.777 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:11.967+0000 7f4292600640 1 -- 172.21.15.5:0/283807685 shutdown_connections 2024-09-10T09:28:12.778 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:11.967+0000 7f4292600640 1 -- 172.21.15.5:0/283807685 wait complete. 2024-09-10T09:28:12.778 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:11.968+0000 7f4292600640 1 Processor -- start 2024-09-10T09:28:12.778 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:11.968+0000 7f4292600640 1 -- start start 2024-09-10T09:28:12.778 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:11.968+0000 7f4292600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f428c107680 0x7f428c07db70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:28:12.778 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:11.968+0000 7f4292600640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f428c10bb90 con 0x7f428c107680 2024-09-10T09:28:12.778 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:11.968+0000 7f428b400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f428c107680 0x7f428c07db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:28:12.778 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:11.968+0000 7f428b400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f428c107680 0x7f428c07db70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53722/0 (socket says 172.21.15.5:53722) 2024-09-10T09:28:12.778 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:11.968+0000 7f428b400640 1 -- 172.21.15.5:0/1572429553 learned_addr learned my addr 172.21.15.5:0/1572429553 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:28:12.778 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:11.969+0000 7f428b400640 1 -- 172.21.15.5:0/1572429553 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f428c07e0b0 con 0x7f428c107680 2024-09-10T09:28:12.778 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:11.969+0000 7f428b400640 1 --2- 172.21.15.5:0/1572429553 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f428c107680 0x7f428c07db70 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f427c0097f0 tx=0x7f427c0327b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:28:12.778 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:11.969+0000 7f4288c00640 1 -- 172.21.15.5:0/1572429553 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f427c037040 con 0x7f428c107680 2024-09-10T09:28:12.778 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:11.969+0000 7f4288c00640 1 -- 172.21.15.5:0/1572429553 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f427c030c40 con 0x7f428c107680 2024-09-10T09:28:12.778 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:11.969+0000 7f4288c00640 1 -- 172.21.15.5:0/1572429553 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f427c03a5b0 con 0x7f428c107680 2024-09-10T09:28:12.778 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:11.969+0000 7f4292600640 1 -- 172.21.15.5:0/1572429553 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f428c07c2a0 con 0x7f428c107680 2024-09-10T09:28:12.778 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:11.969+0000 7f4292600640 1 -- 172.21.15.5:0/1572429553 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f428c07c7e0 con 0x7f428c107680 2024-09-10T09:28:12.779 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:11.970+0000 7f4288c00640 1 -- 172.21.15.5:0/1572429553 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 12) ==== 46300+0+0 (secure 0 0 0) 0x7f427c03a7a0 con 0x7f428c107680 2024-09-10T09:28:12.779 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:11.970+0000 7f4292600640 1 -- 172.21.15.5:0/1572429553 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f428c10bd10 con 0x7f428c107680 2024-09-10T09:28:12.779 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:11.971+0000 7f4288c00640 1 --2- 172.21.15.5:0/1572429553 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f4270039020 0x7f427003b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:28:12.779 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:11.971+0000 7f4288c00640 1 -- 172.21.15.5:0/1572429553 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f427c06c5a0 con 0x7f428c107680 2024-09-10T09:28:12.779 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:11.971+0000 7f4282a00640 1 --2- 172.21.15.5:0/1572429553 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f4270039020 0x7f427003b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:28:12.779 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:11.972+0000 7f4282a00640 1 --2- 172.21.15.5:0/1572429553 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f4270039020 0x7f427003b4e0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f42740098c0 tx=0x7f4274006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:28:12.779 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:11.976+0000 7f4288c00640 1 -- 172.21.15.5:0/1572429553 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f427c0371e0 con 0x7f428c107680 2024-09-10T09:28:12.779 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:12.164+0000 7f4292600640 1 -- 172.21.15.5:0/1572429553 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) -- 0x7f428c107b00 con 0x7f428c107680 2024-09-10T09:28:12.779 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:12.164+0000 7f4288c00640 1 -- 172.21.15.5:0/1572429553 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) ==== 112+0+5 (secure 0 0 0) 0x7f427c03b3f0 con 0x7f428c107680 2024-09-10T09:28:12.779 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:12.166+0000 7f4292600640 1 -- 172.21.15.5:0/1572429553 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f4270039020 msgr2=0x7f427003b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:28:12.779 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:12.166+0000 7f4292600640 1 --2- 172.21.15.5:0/1572429553 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f4270039020 0x7f427003b4e0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f42740098c0 tx=0x7f4274006db0 comp rx=0 tx=0).stop 2024-09-10T09:28:12.779 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:12.166+0000 7f4292600640 1 -- 172.21.15.5:0/1572429553 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f428c107680 msgr2=0x7f428c07db70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:28:12.779 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:12.166+0000 7f4292600640 1 --2- 172.21.15.5:0/1572429553 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f428c107680 0x7f428c07db70 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f427c0097f0 tx=0x7f427c0327b0 comp rx=0 tx=0).stop 2024-09-10T09:28:12.779 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:12.166+0000 7f4292600640 1 -- 172.21.15.5:0/1572429553 shutdown_connections 2024-09-10T09:28:12.779 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:12.166+0000 7f4292600640 1 --2- 172.21.15.5:0/1572429553 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f4270039020 0x7f427003b4e0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:28:12.780 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:12.166+0000 7f4292600640 1 --2- 172.21.15.5:0/1572429553 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f428c107680 0x7f428c07db70 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:28:12.780 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:12.166+0000 7f4292600640 1 -- 172.21.15.5:0/1572429553 >> 172.21.15.5:0/1572429553 conn(0x7f428c102e10 msgr2=0x7f428c1039d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:28:12.780 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:12.167+0000 7f4292600640 1 -- 172.21.15.5:0/1572429553 shutdown_connections 2024-09-10T09:28:12.780 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:12.167+0000 7f4292600640 1 -- 172.21.15.5:0/1572429553 wait complete. 2024-09-10T09:28:12.793 INFO:teuthology.orchestra.run.smithi005.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-10T09:28:12.793 INFO:teuthology.orchestra.run.smithi005.stdout:systemctl: stdout disabled 2024-09-10T09:28:12.806 INFO:teuthology.orchestra.run.smithi005.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-10T09:28:12.806 INFO:teuthology.orchestra.run.smithi005.stdout:systemctl: stdout inactive 2024-09-10T09:28:12.806 INFO:teuthology.orchestra.run.smithi005.stdout:firewalld.service is not enabled 2024-09-10T09:28:12.806 INFO:teuthology.orchestra.run.smithi005.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-09-10T09:28:12.809 INFO:teuthology.orchestra.run.smithi005.stdout:Ceph Dashboard is now available at: 2024-09-10T09:28:12.809 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-10T09:28:12.809 INFO:teuthology.orchestra.run.smithi005.stdout: URL: https://smithi005.front.sepia.ceph.com:8443/ 2024-09-10T09:28:12.809 INFO:teuthology.orchestra.run.smithi005.stdout: User: admin 2024-09-10T09:28:12.809 INFO:teuthology.orchestra.run.smithi005.stdout: Password: l77awx630u 2024-09-10T09:28:12.810 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-10T09:28:12.810 INFO:teuthology.orchestra.run.smithi005.stdout:Saving cluster configuration to /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/config directory 2024-09-10T09:28:13.329 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:28:13 smithi005 ceph-mon[30350]: mgrmap e13: x(active, since 6s) 2024-09-10T09:28:14.273 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:13.328+0000 7f227a800640 1 Processor -- start 2024-09-10T09:28:14.273 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:13.328+0000 7f227a800640 1 -- start start 2024-09-10T09:28:14.274 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:13.329+0000 7f227a800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2274109870 0x7f2274109c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:28:14.274 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:13.329+0000 7f227a800640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f227410a240 con 0x7f2274109870 2024-09-10T09:28:14.274 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:13.329+0000 7f2273400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2274109870 0x7f2274109c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:28:14.274 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:13.329+0000 7f2273400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2274109870 0x7f2274109c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53738/0 (socket says 172.21.15.5:53738) 2024-09-10T09:28:14.274 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:13.329+0000 7f2273400640 1 -- 172.21.15.5:0/3568293983 learned_addr learned my addr 172.21.15.5:0/3568293983 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:28:14.274 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:13.330+0000 7f2273400640 1 -- 172.21.15.5:0/3568293983 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f227410aa10 con 0x7f2274109870 2024-09-10T09:28:14.274 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:13.331+0000 7f2273400640 1 --2- 172.21.15.5:0/3568293983 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2274109870 0x7f2274109c70 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f2264009820 tx=0x7f226402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f12ed111ac882d16 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:28:14.274 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:13.331+0000 7f2272a00640 1 -- 172.21.15.5:0/3568293983 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2264037070 con 0x7f2274109870 2024-09-10T09:28:14.274 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:13.331+0000 7f2272a00640 1 -- 172.21.15.5:0/3568293983 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2264032920 con 0x7f2274109870 2024-09-10T09:28:14.274 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:13.331+0000 7f2272a00640 1 -- 172.21.15.5:0/3568293983 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2264030810 con 0x7f2274109870 2024-09-10T09:28:14.274 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:13.331+0000 7f227a800640 1 -- 172.21.15.5:0/3568293983 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2274109870 msgr2=0x7f2274109c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:28:14.274 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:13.331+0000 7f227a800640 1 --2- 172.21.15.5:0/3568293983 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2274109870 0x7f2274109c70 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f2264009820 tx=0x7f226402a270 comp rx=0 tx=0).stop 2024-09-10T09:28:14.274 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:13.332+0000 7f227a800640 1 -- 172.21.15.5:0/3568293983 shutdown_connections 2024-09-10T09:28:14.275 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:13.332+0000 7f227a800640 1 --2- 172.21.15.5:0/3568293983 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2274109870 0x7f2274109c70 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:28:14.275 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:13.332+0000 7f227a800640 1 -- 172.21.15.5:0/3568293983 >> 172.21.15.5:0/3568293983 conn(0x7f227407cf40 msgr2=0x7f227407d350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:28:14.275 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:13.332+0000 7f227a800640 1 -- 172.21.15.5:0/3568293983 shutdown_connections 2024-09-10T09:28:14.275 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:13.332+0000 7f227a800640 1 -- 172.21.15.5:0/3568293983 wait complete. 2024-09-10T09:28:14.275 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:13.333+0000 7f227a800640 1 Processor -- start 2024-09-10T09:28:14.275 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:13.333+0000 7f227a800640 1 -- start start 2024-09-10T09:28:14.275 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:13.333+0000 7f227a800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2274109870 0x7f227419d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:28:14.275 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:13.333+0000 7f227a800640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f227410ddc0 con 0x7f2274109870 2024-09-10T09:28:14.275 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:13.333+0000 7f2273400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2274109870 0x7f227419d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:28:14.275 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:13.333+0000 7f2273400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2274109870 0x7f227419d7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53746/0 (socket says 172.21.15.5:53746) 2024-09-10T09:28:14.275 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:13.333+0000 7f2273400640 1 -- 172.21.15.5:0/2480902383 learned_addr learned my addr 172.21.15.5:0/2480902383 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:28:14.275 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:13.334+0000 7f2273400640 1 -- 172.21.15.5:0/2480902383 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f227419dd10 con 0x7f2274109870 2024-09-10T09:28:14.275 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:13.334+0000 7f2273400640 1 --2- 172.21.15.5:0/2480902383 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2274109870 0x7f227419d7d0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f22640097f0 tx=0x7f22640327b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:28:14.275 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:13.334+0000 7f2270c00640 1 -- 172.21.15.5:0/2480902383 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2264037040 con 0x7f2274109870 2024-09-10T09:28:14.276 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:13.334+0000 7f227a800640 1 -- 172.21.15.5:0/2480902383 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f227419dfa0 con 0x7f2274109870 2024-09-10T09:28:14.276 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:13.334+0000 7f2270c00640 1 -- 172.21.15.5:0/2480902383 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2264030cc0 con 0x7f2274109870 2024-09-10T09:28:14.276 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:13.334+0000 7f2270c00640 1 -- 172.21.15.5:0/2480902383 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f226403a6b0 con 0x7f2274109870 2024-09-10T09:28:14.276 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:13.334+0000 7f227a800640 1 -- 172.21.15.5:0/2480902383 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22741a0c90 con 0x7f2274109870 2024-09-10T09:28:14.276 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:13.335+0000 7f2270c00640 1 -- 172.21.15.5:0/2480902383 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 13) ==== 46342+0+0 (secure 0 0 0) 0x7f226403b3f0 con 0x7f2274109870 2024-09-10T09:28:14.276 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:13.335+0000 7f227a800640 1 -- 172.21.15.5:0/2480902383 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2274109cf0 con 0x7f2274109870 2024-09-10T09:28:14.276 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:13.336+0000 7f2270c00640 1 --2- 172.21.15.5:0/2480902383 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f2258039060 0x7f225803b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:28:14.276 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:13.336+0000 7f2270c00640 1 -- 172.21.15.5:0/2480902383 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f226406c7b0 con 0x7f2274109870 2024-09-10T09:28:14.276 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:13.336+0000 7f226be00640 1 --2- 172.21.15.5:0/2480902383 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f2258039060 0x7f225803b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:28:14.276 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:13.337+0000 7f226be00640 1 --2- 172.21.15.5:0/2480902383 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f2258039060 0x7f225803b520 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f225c0098c0 tx=0x7f225c006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:28:14.276 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:13.340+0000 7f2270c00640 1 -- 172.21.15.5:0/2480902383 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f226403a8a0 con 0x7f2274109870 2024-09-10T09:28:14.276 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:13.616+0000 7f227a800640 1 -- 172.21.15.5:0/2480902383 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) -- 0x7f22741a0e60 con 0x7f2274109870 2024-09-10T09:28:14.276 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:13.618+0000 7f2270c00640 1 -- 172.21.15.5:0/2480902383 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v28)=0 set mgr/dashboard/cluster/status v28) ==== 153+0+0 (secure 0 0 0) 0x7f226403baf0 con 0x7f2274109870 2024-09-10T09:28:14.276 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-09-10T09:28:14.277 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:13.621+0000 7f227a800640 1 -- 172.21.15.5:0/2480902383 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f2258039060 msgr2=0x7f225803b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:28:14.277 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:13.621+0000 7f227a800640 1 --2- 172.21.15.5:0/2480902383 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f2258039060 0x7f225803b520 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f225c0098c0 tx=0x7f225c006db0 comp rx=0 tx=0).stop 2024-09-10T09:28:14.277 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:13.621+0000 7f227a800640 1 -- 172.21.15.5:0/2480902383 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2274109870 msgr2=0x7f227419d7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:28:14.277 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:13.621+0000 7f227a800640 1 --2- 172.21.15.5:0/2480902383 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2274109870 0x7f227419d7d0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f22640097f0 tx=0x7f22640327b0 comp rx=0 tx=0).stop 2024-09-10T09:28:14.277 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:13.621+0000 7f227a800640 1 -- 172.21.15.5:0/2480902383 shutdown_connections 2024-09-10T09:28:14.277 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:13.621+0000 7f227a800640 1 --2- 172.21.15.5:0/2480902383 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f2258039060 0x7f225803b520 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:28:14.277 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:13.621+0000 7f227a800640 1 --2- 172.21.15.5:0/2480902383 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2274109870 0x7f227419d7d0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:28:14.277 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:13.621+0000 7f227a800640 1 -- 172.21.15.5:0/2480902383 >> 172.21.15.5:0/2480902383 conn(0x7f227407cf40 msgr2=0x7f2274107540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:28:14.277 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:13.621+0000 7f227a800640 1 -- 172.21.15.5:0/2480902383 shutdown_connections 2024-09-10T09:28:14.277 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-10T09:28:13.621+0000 7f227a800640 1 -- 172.21.15.5:0/2480902383 wait complete. 2024-09-10T09:28:14.277 INFO:teuthology.orchestra.run.smithi005.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-09-10T09:28:14.277 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-10T09:28:14.278 INFO:teuthology.orchestra.run.smithi005.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-09-10T09:28:14.278 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-10T09:28:14.278 INFO:teuthology.orchestra.run.smithi005.stdout:Or, if you are only running a single cluster on this host: 2024-09-10T09:28:14.278 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-10T09:28:14.278 INFO:teuthology.orchestra.run.smithi005.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-09-10T09:28:14.278 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-10T09:28:14.278 INFO:teuthology.orchestra.run.smithi005.stdout:Please consider enabling telemetry to help improve Ceph: 2024-09-10T09:28:14.278 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-10T09:28:14.278 INFO:teuthology.orchestra.run.smithi005.stdout: ceph telemetry on 2024-09-10T09:28:14.278 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-10T09:28:14.278 INFO:teuthology.orchestra.run.smithi005.stdout:For more information see: 2024-09-10T09:28:14.278 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-10T09:28:14.278 INFO:teuthology.orchestra.run.smithi005.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-09-10T09:28:14.279 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-10T09:28:14.279 INFO:teuthology.orchestra.run.smithi005.stdout:Bootstrap complete. 2024-09-10T09:28:14.328 INFO:tasks.cephadm:Fetching config... 2024-09-10T09:28:14.328 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-09-10T09:28:14.328 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-09-10T09:28:14.363 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-09-10T09:28:14.364 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-09-10T09:28:14.364 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-09-10T09:28:14.440 INFO:tasks.cephadm:Fetching mon keyring... 2024-09-10T09:28:14.440 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-09-10T09:28:14.440 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd if=/var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/keyring of=/dev/stdout 2024-09-10T09:28:14.537 INFO:tasks.cephadm:Fetching pub ssh key... 2024-09-10T09:28:14.537 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-09-10T09:28:14.537 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-09-10T09:28:14.612 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-09-10T09:28:14.612 DEBUG:teuthology.orchestra.run.smithi005:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQD3kMvhYY/c4Qg+wIxyzdtZdZXPJOF45PYPNYa0JHRy7azcDpeNQRYqM52JifHuAGXk5BPEviPo/KtpoByh8e68S0RPaFm8199+lYksO823Rh1qDYeCVSVCb2embg/NzX1Z5IFQ/hyZBUY3fes5iGEwiBQCq3C+98h0dZkfC7knZdLNLA0P+XPImJSPIfA4LnhTc6Q8yVHdU3+ta4O6wIIyoqBKpCCE8ClLcr9ZRYjHU57f6QYFGQKYjRwsXltvrwE+dpxYA3dvc93n0k2u5nKc0qQdY4BQMNjkjYQqguhzVZlT7mVI8ommP2iUNaw9UBEby+9oo2zF45B1Ky9JLpFBVMletw7w9bJHeX5y6+TfJBNSJWpVksUpntX7ylMNZXUB5jGCUEZNn7B+ZDmSGf8gil5A2sqezcgFjIaUXetcToG/ulyEvPLVfuuxStw2O3pWj2KB0KCnLrSp9viwxiYAiKpW+YqHtNlDvD47Na54ZUjh1oJARIe3zjGYLWbYq0E= ceph-bade4b50-6f56-11ef-bcea-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-10T09:28:14.733 INFO:teuthology.orchestra.run.smithi005.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQD3kMvhYY/c4Qg+wIxyzdtZdZXPJOF45PYPNYa0JHRy7azcDpeNQRYqM52JifHuAGXk5BPEviPo/KtpoByh8e68S0RPaFm8199+lYksO823Rh1qDYeCVSVCb2embg/NzX1Z5IFQ/hyZBUY3fes5iGEwiBQCq3C+98h0dZkfC7knZdLNLA0P+XPImJSPIfA4LnhTc6Q8yVHdU3+ta4O6wIIyoqBKpCCE8ClLcr9ZRYjHU57f6QYFGQKYjRwsXltvrwE+dpxYA3dvc93n0k2u5nKc0qQdY4BQMNjkjYQqguhzVZlT7mVI8ommP2iUNaw9UBEby+9oo2zF45B1Ky9JLpFBVMletw7w9bJHeX5y6+TfJBNSJWpVksUpntX7ylMNZXUB5jGCUEZNn7B+ZDmSGf8gil5A2sqezcgFjIaUXetcToG/ulyEvPLVfuuxStw2O3pWj2KB0KCnLrSp9viwxiYAiKpW+YqHtNlDvD47Na54ZUjh1oJARIe3zjGYLWbYq0E= ceph-bade4b50-6f56-11ef-bcea-c7b262605968 2024-09-10T09:28:14.758 DEBUG:teuthology.orchestra.run.smithi102:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQD3kMvhYY/c4Qg+wIxyzdtZdZXPJOF45PYPNYa0JHRy7azcDpeNQRYqM52JifHuAGXk5BPEviPo/KtpoByh8e68S0RPaFm8199+lYksO823Rh1qDYeCVSVCb2embg/NzX1Z5IFQ/hyZBUY3fes5iGEwiBQCq3C+98h0dZkfC7knZdLNLA0P+XPImJSPIfA4LnhTc6Q8yVHdU3+ta4O6wIIyoqBKpCCE8ClLcr9ZRYjHU57f6QYFGQKYjRwsXltvrwE+dpxYA3dvc93n0k2u5nKc0qQdY4BQMNjkjYQqguhzVZlT7mVI8ommP2iUNaw9UBEby+9oo2zF45B1Ky9JLpFBVMletw7w9bJHeX5y6+TfJBNSJWpVksUpntX7ylMNZXUB5jGCUEZNn7B+ZDmSGf8gil5A2sqezcgFjIaUXetcToG/ulyEvPLVfuuxStw2O3pWj2KB0KCnLrSp9viwxiYAiKpW+YqHtNlDvD47Na54ZUjh1oJARIe3zjGYLWbYq0E= ceph-bade4b50-6f56-11ef-bcea-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-10T09:28:14.798 INFO:teuthology.orchestra.run.smithi102.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQD3kMvhYY/c4Qg+wIxyzdtZdZXPJOF45PYPNYa0JHRy7azcDpeNQRYqM52JifHuAGXk5BPEviPo/KtpoByh8e68S0RPaFm8199+lYksO823Rh1qDYeCVSVCb2embg/NzX1Z5IFQ/hyZBUY3fes5iGEwiBQCq3C+98h0dZkfC7knZdLNLA0P+XPImJSPIfA4LnhTc6Q8yVHdU3+ta4O6wIIyoqBKpCCE8ClLcr9ZRYjHU57f6QYFGQKYjRwsXltvrwE+dpxYA3dvc93n0k2u5nKc0qQdY4BQMNjkjYQqguhzVZlT7mVI8ommP2iUNaw9UBEby+9oo2zF45B1Ky9JLpFBVMletw7w9bJHeX5y6+TfJBNSJWpVksUpntX7ylMNZXUB5jGCUEZNn7B+ZDmSGf8gil5A2sqezcgFjIaUXetcToG/ulyEvPLVfuuxStw2O3pWj2KB0KCnLrSp9viwxiYAiKpW+YqHtNlDvD47Na54ZUjh1oJARIe3zjGYLWbYq0E= ceph-bade4b50-6f56-11ef-bcea-c7b262605968 2024-09-10T09:28:14.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:28:14 smithi005 ceph-mon[30350]: from='client.? 172.21.15.5:0/2480902383' entity='client.admin' 2024-09-10T09:28:14.810 DEBUG:teuthology.orchestra.run.smithi159:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQD3kMvhYY/c4Qg+wIxyzdtZdZXPJOF45PYPNYa0JHRy7azcDpeNQRYqM52JifHuAGXk5BPEviPo/KtpoByh8e68S0RPaFm8199+lYksO823Rh1qDYeCVSVCb2embg/NzX1Z5IFQ/hyZBUY3fes5iGEwiBQCq3C+98h0dZkfC7knZdLNLA0P+XPImJSPIfA4LnhTc6Q8yVHdU3+ta4O6wIIyoqBKpCCE8ClLcr9ZRYjHU57f6QYFGQKYjRwsXltvrwE+dpxYA3dvc93n0k2u5nKc0qQdY4BQMNjkjYQqguhzVZlT7mVI8ommP2iUNaw9UBEby+9oo2zF45B1Ky9JLpFBVMletw7w9bJHeX5y6+TfJBNSJWpVksUpntX7ylMNZXUB5jGCUEZNn7B+ZDmSGf8gil5A2sqezcgFjIaUXetcToG/ulyEvPLVfuuxStw2O3pWj2KB0KCnLrSp9viwxiYAiKpW+YqHtNlDvD47Na54ZUjh1oJARIe3zjGYLWbYq0E= ceph-bade4b50-6f56-11ef-bcea-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-10T09:28:14.852 INFO:teuthology.orchestra.run.smithi159.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQD3kMvhYY/c4Qg+wIxyzdtZdZXPJOF45PYPNYa0JHRy7azcDpeNQRYqM52JifHuAGXk5BPEviPo/KtpoByh8e68S0RPaFm8199+lYksO823Rh1qDYeCVSVCb2embg/NzX1Z5IFQ/hyZBUY3fes5iGEwiBQCq3C+98h0dZkfC7knZdLNLA0P+XPImJSPIfA4LnhTc6Q8yVHdU3+ta4O6wIIyoqBKpCCE8ClLcr9ZRYjHU57f6QYFGQKYjRwsXltvrwE+dpxYA3dvc93n0k2u5nKc0qQdY4BQMNjkjYQqguhzVZlT7mVI8ommP2iUNaw9UBEby+9oo2zF45B1Ky9JLpFBVMletw7w9bJHeX5y6+TfJBNSJWpVksUpntX7ylMNZXUB5jGCUEZNn7B+ZDmSGf8gil5A2sqezcgFjIaUXetcToG/ulyEvPLVfuuxStw2O3pWj2KB0KCnLrSp9viwxiYAiKpW+YqHtNlDvD47Na54ZUjh1oJARIe3zjGYLWbYq0E= ceph-bade4b50-6f56-11ef-bcea-c7b262605968 2024-09-10T09:28:14.865 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-09-10T09:28:15.244 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:28:15.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:15.761+0000 7f9aaf200640 1 -- 172.21.15.5:0/1539597663 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9aa8076ba0 msgr2=0x7f9aa8076fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:28:15.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:15.761+0000 7f9aaf200640 1 --2- 172.21.15.5:0/1539597663 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9aa8076ba0 0x7f9aa8076fa0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f9a98009880 tx=0x7f9a9802a5d0 comp rx=0 tx=0).stop 2024-09-10T09:28:15.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:15.761+0000 7f9aaf200640 1 -- 172.21.15.5:0/1539597663 shutdown_connections 2024-09-10T09:28:15.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:15.761+0000 7f9aaf200640 1 --2- 172.21.15.5:0/1539597663 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9aa8076ba0 0x7f9aa8076fa0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:28:15.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:15.761+0000 7f9aaf200640 1 -- 172.21.15.5:0/1539597663 >> 172.21.15.5:0/1539597663 conn(0x7f9aa80fefc0 msgr2=0x7f9aa8101400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:28:15.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:15.761+0000 7f9aaf200640 1 -- 172.21.15.5:0/1539597663 shutdown_connections 2024-09-10T09:28:15.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:15.761+0000 7f9aaf200640 1 -- 172.21.15.5:0/1539597663 wait complete. 2024-09-10T09:28:15.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:15.762+0000 7f9aaf200640 1 Processor -- start 2024-09-10T09:28:15.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:15.763+0000 7f9aaf200640 1 -- start start 2024-09-10T09:28:15.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:15.763+0000 7f9aaf200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9aa8076ba0 0x7f9aa819d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:28:15.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:15.763+0000 7f9aaf200640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f9aa810dcf0 con 0x7f9aa8076ba0 2024-09-10T09:28:15.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:15.763+0000 7f9aa7e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9aa8076ba0 0x7f9aa819d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:28:15.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:15.763+0000 7f9aa7e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9aa8076ba0 0x7f9aa819d360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53756/0 (socket says 172.21.15.5:53756) 2024-09-10T09:28:15.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:15.763+0000 7f9aa7e00640 1 -- 172.21.15.5:0/3630588573 learned_addr learned my addr 172.21.15.5:0/3630588573 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:28:15.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:15.764+0000 7f9aa7e00640 1 -- 172.21.15.5:0/3630588573 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9aa819d8a0 con 0x7f9aa8076ba0 2024-09-10T09:28:15.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:15.764+0000 7f9aa7e00640 1 --2- 172.21.15.5:0/3630588573 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9aa8076ba0 0x7f9aa819d360 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f9a98004f60 tx=0x7f9a980048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:28:15.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:15.765+0000 7f9aa4c00640 1 -- 172.21.15.5:0/3630588573 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9a9803f070 con 0x7f9aa8076ba0 2024-09-10T09:28:15.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:15.765+0000 7f9aa4c00640 1 -- 172.21.15.5:0/3630588573 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9a98003ad0 con 0x7f9aa8076ba0 2024-09-10T09:28:15.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:15.765+0000 7f9aa4c00640 1 -- 172.21.15.5:0/3630588573 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9a98035030 con 0x7f9aa8076ba0 2024-09-10T09:28:15.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:15.765+0000 7f9aaf200640 1 -- 172.21.15.5:0/3630588573 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9aa819db30 con 0x7f9aa8076ba0 2024-09-10T09:28:15.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:15.765+0000 7f9aaf200640 1 -- 172.21.15.5:0/3630588573 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9aa81a0820 con 0x7f9aa8076ba0 2024-09-10T09:28:15.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:15.766+0000 7f9aaf200640 1 -- 172.21.15.5:0/3630588573 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9aa810e470 con 0x7f9aa8076ba0 2024-09-10T09:28:15.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:15.766+0000 7f9aa4c00640 1 -- 172.21.15.5:0/3630588573 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 13) ==== 46342+0+0 (secure 0 0 0) 0x7f9a98042070 con 0x7f9aa8076ba0 2024-09-10T09:28:15.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:15.767+0000 7f9aa4c00640 1 --2- 172.21.15.5:0/3630588573 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f9a8c0390c0 0x7f9a8c03b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:28:15.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:15.767+0000 7f9aa4c00640 1 -- 172.21.15.5:0/3630588573 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f9a9806ce40 con 0x7f9aa8076ba0 2024-09-10T09:28:15.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:15.767+0000 7f9aa7400640 1 --2- 172.21.15.5:0/3630588573 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f9a8c0390c0 0x7f9a8c03b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:28:15.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:15.768+0000 7f9aa7400640 1 --2- 172.21.15.5:0/3630588573 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f9a8c0390c0 0x7f9a8c03b580 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f9a940098c0 tx=0x7f9a94006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:28:15.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:15.773+0000 7f9aa4c00640 1 -- 172.21.15.5:0/3630588573 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f9a9803a3c0 con 0x7f9aa8076ba0 2024-09-10T09:28:15.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:15.964+0000 7f9aaf200640 1 -- 172.21.15.5:0/3630588573 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) -- 0x7f9aa802d830 con 0x7f9aa8076ba0 2024-09-10T09:28:15.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:15.967+0000 7f9aa4c00640 1 -- 172.21.15.5:0/3630588573 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) ==== 125+0+0 (secure 0 0 0) 0x7f9a98030d90 con 0x7f9aa8076ba0 2024-09-10T09:28:15.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:15.970+0000 7f9aaf200640 1 -- 172.21.15.5:0/3630588573 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f9a8c0390c0 msgr2=0x7f9a8c03b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:28:15.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:15.970+0000 7f9aaf200640 1 --2- 172.21.15.5:0/3630588573 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f9a8c0390c0 0x7f9a8c03b580 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f9a940098c0 tx=0x7f9a94006db0 comp rx=0 tx=0).stop 2024-09-10T09:28:15.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:15.970+0000 7f9aaf200640 1 -- 172.21.15.5:0/3630588573 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9aa8076ba0 msgr2=0x7f9aa819d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:28:15.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:15.970+0000 7f9aaf200640 1 --2- 172.21.15.5:0/3630588573 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9aa8076ba0 0x7f9aa819d360 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f9a98004f60 tx=0x7f9a980048f0 comp rx=0 tx=0).stop 2024-09-10T09:28:15.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:15.970+0000 7f9aaf200640 1 -- 172.21.15.5:0/3630588573 shutdown_connections 2024-09-10T09:28:15.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:15.970+0000 7f9aaf200640 1 --2- 172.21.15.5:0/3630588573 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f9a8c0390c0 0x7f9a8c03b580 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:28:15.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:15.970+0000 7f9aaf200640 1 --2- 172.21.15.5:0/3630588573 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9aa8076ba0 0x7f9aa819d360 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:28:15.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:15.970+0000 7f9aaf200640 1 -- 172.21.15.5:0/3630588573 >> 172.21.15.5:0/3630588573 conn(0x7f9aa80fefc0 msgr2=0x7f9aa80ffa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:28:15.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:15.970+0000 7f9aaf200640 1 -- 172.21.15.5:0/3630588573 shutdown_connections 2024-09-10T09:28:15.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:15.970+0000 7f9aaf200640 1 -- 172.21.15.5:0/3630588573 wait complete. 2024-09-10T09:28:17.182 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-09-10T09:28:17.182 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-09-10T09:28:17.231 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:28:16 smithi005 ceph-mon[30350]: from='client.? 172.21.15.5:0/3630588573' entity='client.admin' 2024-09-10T09:28:17.737 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:28:18.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:18.292+0000 7f2f0fa00640 1 -- 172.21.15.5:0/2660740905 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2f081037f0 msgr2=0x7f2f08103bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:28:18.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:18.292+0000 7f2f0fa00640 1 --2- 172.21.15.5:0/2660740905 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2f081037f0 0x7f2f08103bf0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f2efc009880 tx=0x7f2efc02a5d0 comp rx=0 tx=0).stop 2024-09-10T09:28:18.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:18.292+0000 7f2f0fa00640 1 -- 172.21.15.5:0/2660740905 shutdown_connections 2024-09-10T09:28:18.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:18.292+0000 7f2f0fa00640 1 --2- 172.21.15.5:0/2660740905 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2f081037f0 0x7f2f08103bf0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:28:18.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:18.292+0000 7f2f0fa00640 1 -- 172.21.15.5:0/2660740905 >> 172.21.15.5:0/2660740905 conn(0x7f2f080fefc0 msgr2=0x7f2f08101400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:28:18.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:18.293+0000 7f2f0fa00640 1 -- 172.21.15.5:0/2660740905 shutdown_connections 2024-09-10T09:28:18.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:18.293+0000 7f2f0fa00640 1 -- 172.21.15.5:0/2660740905 wait complete. 2024-09-10T09:28:18.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:18.293+0000 7f2f0fa00640 1 Processor -- start 2024-09-10T09:28:18.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:18.294+0000 7f2f0fa00640 1 -- start start 2024-09-10T09:28:18.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:18.294+0000 7f2f0fa00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2f081037f0 0x7f2f08198f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:28:18.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:18.294+0000 7f2f0fa00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f2f08109900 con 0x7f2f081037f0 2024-09-10T09:28:18.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:18.294+0000 7f2f0d000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2f081037f0 0x7f2f08198f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:28:18.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:18.294+0000 7f2f0d000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2f081037f0 0x7f2f08198f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53774/0 (socket says 172.21.15.5:53774) 2024-09-10T09:28:18.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:18.294+0000 7f2f0d000640 1 -- 172.21.15.5:0/1937307692 learned_addr learned my addr 172.21.15.5:0/1937307692 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:28:18.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:18.295+0000 7f2f0d000640 1 -- 172.21.15.5:0/1937307692 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f081994b0 con 0x7f2f081037f0 2024-09-10T09:28:18.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:18.295+0000 7f2f0d000640 1 --2- 172.21.15.5:0/1937307692 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2f081037f0 0x7f2f08198f70 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f2efc004f60 tx=0x7f2efc0048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:28:18.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:18.295+0000 7f2f05600640 1 -- 172.21.15.5:0/1937307692 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2efc03f070 con 0x7f2f081037f0 2024-09-10T09:28:18.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:18.295+0000 7f2f05600640 1 -- 172.21.15.5:0/1937307692 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2efc004cc0 con 0x7f2f081037f0 2024-09-10T09:28:18.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:18.295+0000 7f2f05600640 1 -- 172.21.15.5:0/1937307692 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2efc035030 con 0x7f2f081037f0 2024-09-10T09:28:18.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:18.295+0000 7f2f0fa00640 1 -- 172.21.15.5:0/1937307692 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f08199740 con 0x7f2f081037f0 2024-09-10T09:28:18.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:18.295+0000 7f2f0fa00640 1 -- 172.21.15.5:0/1937307692 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f0819c430 con 0x7f2f081037f0 2024-09-10T09:28:18.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:18.297+0000 7f2f0fa00640 1 -- 172.21.15.5:0/1937307692 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f0810a080 con 0x7f2f081037f0 2024-09-10T09:28:18.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:18.297+0000 7f2f05600640 1 -- 172.21.15.5:0/1937307692 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 13) ==== 46342+0+0 (secure 0 0 0) 0x7f2efc042070 con 0x7f2f081037f0 2024-09-10T09:28:18.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:18.297+0000 7f2f05600640 1 --2- 172.21.15.5:0/1937307692 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f2ef0039060 0x7f2ef003b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:28:18.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:18.298+0000 7f2f05600640 1 -- 172.21.15.5:0/1937307692 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f2efc06ce00 con 0x7f2f081037f0 2024-09-10T09:28:18.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:18.298+0000 7f2f07e00640 1 --2- 172.21.15.5:0/1937307692 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f2ef0039060 0x7f2ef003b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:28:18.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:18.299+0000 7f2f07e00640 1 --2- 172.21.15.5:0/1937307692 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f2ef0039060 0x7f2ef003b520 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f2ef40098c0 tx=0x7f2ef4006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:28:18.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:18.302+0000 7f2f05600640 1 -- 172.21.15.5:0/1937307692 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f2efc0035f0 con 0x7f2f081037f0 2024-09-10T09:28:18.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:18.493+0000 7f2f0fa00640 1 -- 172.21.15.5:0/1937307692 --> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) -- 0x7f2f0802d830 con 0x7f2ef0039060 2024-09-10T09:28:18.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:18.498+0000 7f2f05600640 1 -- 172.21.15.5:0/1937307692 <== mgr.14150 v2:172.21.15.5:6800/2707348081 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f2f0802d830 con 0x7f2ef0039060 2024-09-10T09:28:18.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:18.500+0000 7f2f0fa00640 1 -- 172.21.15.5:0/1937307692 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f2ef0039060 msgr2=0x7f2ef003b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:28:18.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:18.500+0000 7f2f0fa00640 1 --2- 172.21.15.5:0/1937307692 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f2ef0039060 0x7f2ef003b520 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f2ef40098c0 tx=0x7f2ef4006db0 comp rx=0 tx=0).stop 2024-09-10T09:28:18.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:18.500+0000 7f2f0fa00640 1 -- 172.21.15.5:0/1937307692 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2f081037f0 msgr2=0x7f2f08198f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:28:18.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:18.500+0000 7f2f0fa00640 1 --2- 172.21.15.5:0/1937307692 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2f081037f0 0x7f2f08198f70 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f2efc004f60 tx=0x7f2efc0048f0 comp rx=0 tx=0).stop 2024-09-10T09:28:18.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:18.500+0000 7f2f0fa00640 1 -- 172.21.15.5:0/1937307692 shutdown_connections 2024-09-10T09:28:18.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:18.500+0000 7f2f0fa00640 1 --2- 172.21.15.5:0/1937307692 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f2ef0039060 0x7f2ef003b520 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:28:18.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:18.500+0000 7f2f0fa00640 1 --2- 172.21.15.5:0/1937307692 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2f081037f0 0x7f2f08198f70 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:28:18.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:18.500+0000 7f2f0fa00640 1 -- 172.21.15.5:0/1937307692 >> 172.21.15.5:0/1937307692 conn(0x7f2f080fefc0 msgr2=0x7f2f080ff9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:28:18.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:18.500+0000 7f2f0fa00640 1 -- 172.21.15.5:0/1937307692 shutdown_connections 2024-09-10T09:28:18.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:18.500+0000 7f2f0fa00640 1 -- 172.21.15.5:0/1937307692 wait complete. 2024-09-10T09:28:19.274 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi102 2024-09-10T09:28:19.274 DEBUG:teuthology.orchestra.run.smithi102:> set -ex 2024-09-10T09:28:19.275 DEBUG:teuthology.orchestra.run.smithi102:> dd of=/etc/ceph/ceph.conf 2024-09-10T09:28:19.293 DEBUG:teuthology.orchestra.run.smithi102:> set -ex 2024-09-10T09:28:19.293 DEBUG:teuthology.orchestra.run.smithi102:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-10T09:28:19.352 INFO:tasks.cephadm:Adding host smithi102 to orchestrator... 2024-09-10T09:28:19.352 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph orch host add smithi102 2024-09-10T09:28:19.790 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:28:19 smithi005 ceph-mon[30350]: from='client.14172 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T09:28:19.830 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:28:19 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:28:19.914 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:28:20.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:20.476+0000 7ffad4a00640 1 -- 172.21.15.5:0/2007990432 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffad0076ba0 msgr2=0x7ffad0076fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:28:20.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:20.476+0000 7ffad4a00640 1 --2- 172.21.15.5:0/2007990432 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffad0076ba0 0x7ffad0076fa0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7ffac0009880 tx=0x7ffac002a5d0 comp rx=0 tx=0).stop 2024-09-10T09:28:20.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:20.477+0000 7ffad4a00640 1 -- 172.21.15.5:0/2007990432 shutdown_connections 2024-09-10T09:28:20.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:20.477+0000 7ffad4a00640 1 --2- 172.21.15.5:0/2007990432 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffad0076ba0 0x7ffad0076fa0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:28:20.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:20.477+0000 7ffad4a00640 1 -- 172.21.15.5:0/2007990432 >> 172.21.15.5:0/2007990432 conn(0x7ffad00fefc0 msgr2=0x7ffad0101400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:28:20.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:20.477+0000 7ffad4a00640 1 -- 172.21.15.5:0/2007990432 shutdown_connections 2024-09-10T09:28:20.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:20.477+0000 7ffad4a00640 1 -- 172.21.15.5:0/2007990432 wait complete. 2024-09-10T09:28:20.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:20.478+0000 7ffad4a00640 1 Processor -- start 2024-09-10T09:28:20.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:20.478+0000 7ffad4a00640 1 -- start start 2024-09-10T09:28:20.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:20.478+0000 7ffad4a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffad0076ba0 0x7ffad019d340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:28:20.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:20.478+0000 7ffad4a00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ffad010dcf0 con 0x7ffad0076ba0 2024-09-10T09:28:20.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:20.479+0000 7ffacde00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffad0076ba0 0x7ffad019d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:28:20.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:20.479+0000 7ffacde00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffad0076ba0 0x7ffad019d340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53792/0 (socket says 172.21.15.5:53792) 2024-09-10T09:28:20.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:20.479+0000 7ffacde00640 1 -- 172.21.15.5:0/4083991037 learned_addr learned my addr 172.21.15.5:0/4083991037 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:28:20.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:20.479+0000 7ffacde00640 1 -- 172.21.15.5:0/4083991037 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffad019d880 con 0x7ffad0076ba0 2024-09-10T09:28:20.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:20.479+0000 7ffacde00640 1 --2- 172.21.15.5:0/4083991037 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffad0076ba0 0x7ffad019d340 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7ffac0004f60 tx=0x7ffac00048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:28:20.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:20.480+0000 7ffac6a00640 1 -- 172.21.15.5:0/4083991037 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ffac003f070 con 0x7ffad0076ba0 2024-09-10T09:28:20.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:20.480+0000 7ffac6a00640 1 -- 172.21.15.5:0/4083991037 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffac0003ad0 con 0x7ffad0076ba0 2024-09-10T09:28:20.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:20.480+0000 7ffac6a00640 1 -- 172.21.15.5:0/4083991037 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ffac0035030 con 0x7ffad0076ba0 2024-09-10T09:28:20.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:20.480+0000 7ffad4a00640 1 -- 172.21.15.5:0/4083991037 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffad019db10 con 0x7ffad0076ba0 2024-09-10T09:28:20.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:20.480+0000 7ffad4a00640 1 -- 172.21.15.5:0/4083991037 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffad01a0800 con 0x7ffad0076ba0 2024-09-10T09:28:20.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:20.481+0000 7ffac6a00640 1 -- 172.21.15.5:0/4083991037 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 13) ==== 46342+0+0 (secure 0 0 0) 0x7ffac0042070 con 0x7ffad0076ba0 2024-09-10T09:28:20.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:20.482+0000 7ffac6a00640 1 --2- 172.21.15.5:0/4083991037 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7ffab40390c0 0x7ffab403b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:28:20.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:20.482+0000 7ffac6a00640 1 -- 172.21.15.5:0/4083991037 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7ffac006ce10 con 0x7ffad0076ba0 2024-09-10T09:28:20.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:20.482+0000 7ffad4a00640 1 -- 172.21.15.5:0/4083991037 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffad010e470 con 0x7ffad0076ba0 2024-09-10T09:28:20.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:20.482+0000 7ffacd400640 1 --2- 172.21.15.5:0/4083991037 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7ffab40390c0 0x7ffab403b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:28:20.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:20.483+0000 7ffacd400640 1 --2- 172.21.15.5:0/4083991037 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7ffab40390c0 0x7ffab403b580 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7ffabc0098c0 tx=0x7ffabc006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:28:20.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:20.488+0000 7ffac6a00640 1 -- 172.21.15.5:0/4083991037 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ffac004a260 con 0x7ffad0076ba0 2024-09-10T09:28:20.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:20.702+0000 7ffad4a00640 1 -- 172.21.15.5:0/4083991037 --> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi102", "target": ["mon-mgr", ""]}) -- 0x7ffad0062630 con 0x7ffab40390c0 2024-09-10T09:28:21.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:28:20 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:28:21.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:28:20 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:28:21.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:28:20 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi005", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:28:21.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:28:20 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:28:21.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:28:20 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:28:21.057 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:28:20 smithi005 ceph-mon[30350]: Updating smithi005:/etc/ceph/ceph.conf 2024-09-10T09:28:21.057 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:28:20 smithi005 ceph-mon[30350]: Updating smithi005:/var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T09:28:22.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:28:21 smithi005 ceph-mon[30350]: from='client.14174 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi102", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T09:28:22.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:28:21 smithi005 ceph-mon[30350]: Updating smithi005:/etc/ceph/ceph.client.admin.keyring 2024-09-10T09:28:22.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:28:21 smithi005 ceph-mon[30350]: Updating smithi005:/var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-10T09:28:22.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:28:21 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:28:22.057 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:28:21 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:28:22.057 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:28:21 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:28:23.306 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:28:22 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:28:23.306 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:28:22 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:28:23.306 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:28:22 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:28:23.306 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:28:22 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:28:23.306 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:28:22 smithi005 ceph-mon[30350]: Deploying cephadm binary to smithi102 2024-09-10T09:28:24.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:24.607+0000 7ffac6a00640 1 -- 172.21.15.5:0/4083991037 <== mgr.14150 v2:172.21.15.5:6800/2707348081 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7ffad0062630 con 0x7ffab40390c0 2024-09-10T09:28:24.609 INFO:teuthology.orchestra.run.smithi005.stdout:Added host 'smithi102' with addr '172.21.15.102' 2024-09-10T09:28:24.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:24.609+0000 7ffad4a00640 1 -- 172.21.15.5:0/4083991037 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7ffab40390c0 msgr2=0x7ffab403b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:28:24.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:24.609+0000 7ffad4a00640 1 --2- 172.21.15.5:0/4083991037 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7ffab40390c0 0x7ffab403b580 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7ffabc0098c0 tx=0x7ffabc006db0 comp rx=0 tx=0).stop 2024-09-10T09:28:24.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:24.609+0000 7ffad4a00640 1 -- 172.21.15.5:0/4083991037 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffad0076ba0 msgr2=0x7ffad019d340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:28:24.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:24.609+0000 7ffad4a00640 1 --2- 172.21.15.5:0/4083991037 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffad0076ba0 0x7ffad019d340 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7ffac0004f60 tx=0x7ffac00048f0 comp rx=0 tx=0).stop 2024-09-10T09:28:24.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:24.609+0000 7ffad4a00640 1 -- 172.21.15.5:0/4083991037 shutdown_connections 2024-09-10T09:28:24.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:24.609+0000 7ffad4a00640 1 --2- 172.21.15.5:0/4083991037 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7ffab40390c0 0x7ffab403b580 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:28:24.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:24.609+0000 7ffad4a00640 1 --2- 172.21.15.5:0/4083991037 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffad0076ba0 0x7ffad019d340 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:28:24.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:24.609+0000 7ffad4a00640 1 -- 172.21.15.5:0/4083991037 >> 172.21.15.5:0/4083991037 conn(0x7ffad00fefc0 msgr2=0x7ffad00ffab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:28:24.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:24.610+0000 7ffad4a00640 1 -- 172.21.15.5:0/4083991037 shutdown_connections 2024-09-10T09:28:24.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:24.610+0000 7ffad4a00640 1 -- 172.21.15.5:0/4083991037 wait complete. 2024-09-10T09:28:25.230 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph orch host ls --format=json 2024-09-10T09:28:25.595 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:28:25.877 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:28:25 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:28:25.877 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:28:25 smithi005 ceph-mon[30350]: Added host smithi102 2024-09-10T09:28:25.877 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:28:25 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:28:25.877 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:28:25 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:28:25.877 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:28:25 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:28:25.877 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:28:25 smithi005 ceph-mon[30350]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T09:28:26.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:26.081+0000 7f32b9600640 1 -- 172.21.15.5:0/1277366084 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f32b4109780 msgr2=0x7f32b4109b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:28:26.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:26.081+0000 7f32b9600640 1 --2- 172.21.15.5:0/1277366084 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f32b4109780 0x7f32b4109b60 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f32a4009880 tx=0x7f32a402a5d0 comp rx=0 tx=0).stop 2024-09-10T09:28:26.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:26.082+0000 7f32b9600640 1 -- 172.21.15.5:0/1277366084 shutdown_connections 2024-09-10T09:28:26.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:26.082+0000 7f32b9600640 1 --2- 172.21.15.5:0/1277366084 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f32b4109780 0x7f32b4109b60 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:28:26.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:26.082+0000 7f32b9600640 1 -- 172.21.15.5:0/1277366084 >> 172.21.15.5:0/1277366084 conn(0x7f32b40ff480 msgr2=0x7f32b41018a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:28:26.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:26.082+0000 7f32b9600640 1 -- 172.21.15.5:0/1277366084 shutdown_connections 2024-09-10T09:28:26.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:26.082+0000 7f32b9600640 1 -- 172.21.15.5:0/1277366084 wait complete. 2024-09-10T09:28:26.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:26.083+0000 7f32b9600640 1 Processor -- start 2024-09-10T09:28:26.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:26.083+0000 7f32b9600640 1 -- start start 2024-09-10T09:28:26.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:26.083+0000 7f32b9600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f32b4109780 0x7f32b4199320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:28:26.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:26.083+0000 7f32b9600640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f32b410de40 con 0x7f32b4109780 2024-09-10T09:28:26.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:26.083+0000 7f32b2a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f32b4109780 0x7f32b4199320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:28:26.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:26.083+0000 7f32b2a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f32b4109780 0x7f32b4199320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42444/0 (socket says 172.21.15.5:42444) 2024-09-10T09:28:26.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:26.083+0000 7f32b2a00640 1 -- 172.21.15.5:0/1740979613 learned_addr learned my addr 172.21.15.5:0/1740979613 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:28:26.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:26.084+0000 7f32b2a00640 1 -- 172.21.15.5:0/1740979613 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32b4199860 con 0x7f32b4109780 2024-09-10T09:28:26.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:26.084+0000 7f32b2a00640 1 --2- 172.21.15.5:0/1740979613 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f32b4109780 0x7f32b4199320 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f32a4009230 tx=0x7f32a4003d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:28:26.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:26.084+0000 7f32ab400640 1 -- 172.21.15.5:0/1740979613 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f32a403f070 con 0x7f32b4109780 2024-09-10T09:28:26.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:26.084+0000 7f32b9600640 1 -- 172.21.15.5:0/1740979613 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32b4199b50 con 0x7f32b4109780 2024-09-10T09:28:26.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:26.084+0000 7f32ab400640 1 -- 172.21.15.5:0/1740979613 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f32a403bc80 con 0x7f32b4109780 2024-09-10T09:28:26.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:26.084+0000 7f32b9600640 1 -- 172.21.15.5:0/1740979613 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32b419dbf0 con 0x7f32b4109780 2024-09-10T09:28:26.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:26.085+0000 7f32ab400640 1 -- 172.21.15.5:0/1740979613 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f32a4049060 con 0x7f32b4109780 2024-09-10T09:28:26.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:26.086+0000 7f32ab400640 1 -- 172.21.15.5:0/1740979613 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 13) ==== 46342+0+0 (secure 0 0 0) 0x7f32a4042070 con 0x7f32b4109780 2024-09-10T09:28:26.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:26.086+0000 7f32b9600640 1 -- 172.21.15.5:0/1740979613 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f32b4104ec0 con 0x7f32b4109780 2024-09-10T09:28:26.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:26.086+0000 7f32ab400640 1 --2- 172.21.15.5:0/1740979613 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f3298039060 0x7f329803b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:28:26.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:26.086+0000 7f32ab400640 1 -- 172.21.15.5:0/1740979613 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f32a406cd80 con 0x7f32b4109780 2024-09-10T09:28:26.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:26.086+0000 7f32abe00640 1 --2- 172.21.15.5:0/1740979613 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f3298039060 0x7f329803b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:28:26.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:26.087+0000 7f32abe00640 1 --2- 172.21.15.5:0/1740979613 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f3298039060 0x7f329803b520 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f329c0098c0 tx=0x7f329c006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:28:26.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:26.091+0000 7f32ab400640 1 -- 172.21.15.5:0/1740979613 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f32a4031d40 con 0x7f32b4109780 2024-09-10T09:28:26.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:26.283+0000 7f32b9600640 1 -- 172.21.15.5:0/1740979613 --> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f32b419a680 con 0x7f3298039060 2024-09-10T09:28:26.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:26.284+0000 7f32ab400640 1 -- 172.21.15.5:0/1740979613 <== mgr.14150 v2:172.21.15.5:6800/2707348081 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+159 (secure 0 0 0) 0x7f32b419a680 con 0x7f3298039060 2024-09-10T09:28:26.285 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-10T09:28:26.285 INFO:teuthology.orchestra.run.smithi005.stdout:[{"addr": "172.21.15.5", "hostname": "smithi005", "labels": [], "status": ""}, {"addr": "172.21.15.102", "hostname": "smithi102", "labels": [], "status": ""}] 2024-09-10T09:28:26.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:26.287+0000 7f32b9600640 1 -- 172.21.15.5:0/1740979613 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f3298039060 msgr2=0x7f329803b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:28:26.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:26.287+0000 7f32b9600640 1 --2- 172.21.15.5:0/1740979613 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f3298039060 0x7f329803b520 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f329c0098c0 tx=0x7f329c006db0 comp rx=0 tx=0).stop 2024-09-10T09:28:26.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:26.287+0000 7f32b9600640 1 -- 172.21.15.5:0/1740979613 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f32b4109780 msgr2=0x7f32b4199320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:28:26.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:26.287+0000 7f32b9600640 1 --2- 172.21.15.5:0/1740979613 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f32b4109780 0x7f32b4199320 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f32a4009230 tx=0x7f32a4003d40 comp rx=0 tx=0).stop 2024-09-10T09:28:26.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:26.287+0000 7f32b9600640 1 -- 172.21.15.5:0/1740979613 shutdown_connections 2024-09-10T09:28:26.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:26.287+0000 7f32b9600640 1 --2- 172.21.15.5:0/1740979613 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f3298039060 0x7f329803b520 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:28:26.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:26.287+0000 7f32b9600640 1 --2- 172.21.15.5:0/1740979613 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f32b4109780 0x7f32b4199320 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:28:26.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:26.287+0000 7f32b9600640 1 -- 172.21.15.5:0/1740979613 >> 172.21.15.5:0/1740979613 conn(0x7f32b40ff480 msgr2=0x7f32b4100e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:28:26.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:26.287+0000 7f32b9600640 1 -- 172.21.15.5:0/1740979613 shutdown_connections 2024-09-10T09:28:26.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:26.287+0000 7f32b9600640 1 -- 172.21.15.5:0/1740979613 wait complete. 2024-09-10T09:28:27.140 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi159 2024-09-10T09:28:27.140 DEBUG:teuthology.orchestra.run.smithi159:> set -ex 2024-09-10T09:28:27.140 DEBUG:teuthology.orchestra.run.smithi159:> dd of=/etc/ceph/ceph.conf 2024-09-10T09:28:27.157 DEBUG:teuthology.orchestra.run.smithi159:> set -ex 2024-09-10T09:28:27.157 DEBUG:teuthology.orchestra.run.smithi159:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-10T09:28:27.214 INFO:tasks.cephadm:Adding host smithi159 to orchestrator... 2024-09-10T09:28:27.214 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph orch host add smithi159 2024-09-10T09:28:27.265 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:28:26 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:28:27.265 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:28:26 smithi005 ceph-mon[30350]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T09:28:27.582 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:28:27.975 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:28:27 smithi005 ceph-mon[30350]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T09:28:28.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:28.086+0000 7f0fda200640 1 -- 172.21.15.5:0/2141880552 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0fd4074620 msgr2=0x7f0fd4074a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:28:28.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:28.086+0000 7f0fda200640 1 --2- 172.21.15.5:0/2141880552 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0fd4074620 0x7f0fd4074a00 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f0fc4009880 tx=0x7f0fc402a670 comp rx=0 tx=0).stop 2024-09-10T09:28:28.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:28.086+0000 7f0fda200640 1 -- 172.21.15.5:0/2141880552 shutdown_connections 2024-09-10T09:28:28.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:28.086+0000 7f0fda200640 1 --2- 172.21.15.5:0/2141880552 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0fd4074620 0x7f0fd4074a00 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:28:28.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:28.086+0000 7f0fda200640 1 -- 172.21.15.5:0/2141880552 >> 172.21.15.5:0/2141880552 conn(0x7f0fd40fd290 msgr2=0x7f0fd40ff6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:28:28.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:28.086+0000 7f0fda200640 1 -- 172.21.15.5:0/2141880552 shutdown_connections 2024-09-10T09:28:28.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:28.086+0000 7f0fda200640 1 -- 172.21.15.5:0/2141880552 wait complete. 2024-09-10T09:28:28.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:28.087+0000 7f0fda200640 1 Processor -- start 2024-09-10T09:28:28.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:28.087+0000 7f0fda200640 1 -- start start 2024-09-10T09:28:28.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:28.088+0000 7f0fda200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0fd4074620 0x7f0fd41993e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:28:28.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:28.088+0000 7f0fda200640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f0fd410dec0 con 0x7f0fd4074620 2024-09-10T09:28:28.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:28.088+0000 7f0fd3400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0fd4074620 0x7f0fd41993e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:28:28.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:28.088+0000 7f0fd3400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0fd4074620 0x7f0fd41993e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42464/0 (socket says 172.21.15.5:42464) 2024-09-10T09:28:28.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:28.088+0000 7f0fd3400640 1 -- 172.21.15.5:0/2284071531 learned_addr learned my addr 172.21.15.5:0/2284071531 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:28:28.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:28.088+0000 7f0fd3400640 1 -- 172.21.15.5:0/2284071531 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0fd4199920 con 0x7f0fd4074620 2024-09-10T09:28:28.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:28.089+0000 7f0fd3400640 1 --2- 172.21.15.5:0/2284071531 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0fd4074620 0x7f0fd41993e0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f0fc4009230 tx=0x7f0fc4003d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:28:28.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:28.089+0000 7f0fcbe00640 1 -- 172.21.15.5:0/2284071531 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0fc403f070 con 0x7f0fd4074620 2024-09-10T09:28:28.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:28.089+0000 7f0fda200640 1 -- 172.21.15.5:0/2284071531 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0fd4199bb0 con 0x7f0fd4074620 2024-09-10T09:28:28.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:28.089+0000 7f0fcbe00640 1 -- 172.21.15.5:0/2284071531 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0fc403bd40 con 0x7f0fd4074620 2024-09-10T09:28:28.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:28.089+0000 7f0fcbe00640 1 -- 172.21.15.5:0/2284071531 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0fc4035030 con 0x7f0fd4074620 2024-09-10T09:28:28.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:28.089+0000 7f0fda200640 1 -- 172.21.15.5:0/2284071531 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0fd419a010 con 0x7f0fd4074620 2024-09-10T09:28:28.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:28.090+0000 7f0fcbe00640 1 -- 172.21.15.5:0/2284071531 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 13) ==== 46342+0+0 (secure 0 0 0) 0x7f0fc4042070 con 0x7f0fd4074620 2024-09-10T09:28:28.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:28.090+0000 7f0fc9600640 1 -- 172.21.15.5:0/2284071531 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0fd4075cd0 con 0x7f0fd4074620 2024-09-10T09:28:28.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:28.091+0000 7f0fcbe00640 1 --2- 172.21.15.5:0/2284071531 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f0fb8039060 0x7f0fb803b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:28:28.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:28.091+0000 7f0fcbe00640 1 -- 172.21.15.5:0/2284071531 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f0fc406ccf0 con 0x7f0fd4074620 2024-09-10T09:28:28.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:28.091+0000 7f0fd2a00640 1 --2- 172.21.15.5:0/2284071531 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f0fb8039060 0x7f0fb803b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:28:28.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:28.092+0000 7f0fd2a00640 1 --2- 172.21.15.5:0/2284071531 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f0fb8039060 0x7f0fb803b520 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f0fc00098c0 tx=0x7f0fc0006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:28:28.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:28.096+0000 7f0fcbe00640 1 -- 172.21.15.5:0/2284071531 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f0fc403a3c0 con 0x7f0fd4074620 2024-09-10T09:28:28.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:28.286+0000 7f0fc9600640 1 -- 172.21.15.5:0/2284071531 --> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi159", "target": ["mon-mgr", ""]}) -- 0x7f0fd419a740 con 0x7f0fb8039060 2024-09-10T09:28:29.306 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:28:28 smithi005 ceph-mon[30350]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi159", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T09:28:29.306 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:28:28 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:28:29.306 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:28:28 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:28:29.306 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:28:28 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:28:29.306 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:28:28 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:28:29.307 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:28:28 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi102", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:28:29.307 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:28:28 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:28:29.307 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:28:28 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:28:30.306 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:28:29 smithi005 ceph-mon[30350]: Updating smithi102:/etc/ceph/ceph.conf 2024-09-10T09:28:30.306 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:28:29 smithi005 ceph-mon[30350]: Updating smithi102:/var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T09:28:30.306 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:28:29 smithi005 ceph-mon[30350]: Updating smithi102:/etc/ceph/ceph.client.admin.keyring 2024-09-10T09:28:30.306 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:28:29 smithi005 ceph-mon[30350]: Updating smithi102:/var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-10T09:28:30.306 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:28:29 smithi005 ceph-mon[30350]: Deploying cephadm binary to smithi159 2024-09-10T09:28:30.306 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:28:29 smithi005 ceph-mon[30350]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T09:28:30.307 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:28:29 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:28:30.307 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:28:29 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:28:30.307 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:28:29 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:28:31.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:28:31 smithi005 ceph-mon[30350]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T09:28:32.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:32.186+0000 7f0fcbe00640 1 -- 172.21.15.5:0/2284071531 <== mgr.14150 v2:172.21.15.5:6800/2707348081 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7f0fd419a740 con 0x7f0fb8039060 2024-09-10T09:28:32.189 INFO:teuthology.orchestra.run.smithi005.stdout:Added host 'smithi159' with addr '172.21.15.159' 2024-09-10T09:28:32.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:32.189+0000 7f0fc9600640 1 -- 172.21.15.5:0/2284071531 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f0fb8039060 msgr2=0x7f0fb803b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:28:32.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:32.189+0000 7f0fc9600640 1 --2- 172.21.15.5:0/2284071531 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f0fb8039060 0x7f0fb803b520 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f0fc00098c0 tx=0x7f0fc0006db0 comp rx=0 tx=0).stop 2024-09-10T09:28:32.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:32.189+0000 7f0fc9600640 1 -- 172.21.15.5:0/2284071531 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0fd4074620 msgr2=0x7f0fd41993e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:28:32.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:32.189+0000 7f0fc9600640 1 --2- 172.21.15.5:0/2284071531 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0fd4074620 0x7f0fd41993e0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f0fc4009230 tx=0x7f0fc4003d80 comp rx=0 tx=0).stop 2024-09-10T09:28:32.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:32.189+0000 7f0fc9600640 1 -- 172.21.15.5:0/2284071531 shutdown_connections 2024-09-10T09:28:32.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:32.189+0000 7f0fc9600640 1 --2- 172.21.15.5:0/2284071531 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f0fb8039060 0x7f0fb803b520 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:28:32.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:32.189+0000 7f0fc9600640 1 --2- 172.21.15.5:0/2284071531 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0fd4074620 0x7f0fd41993e0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:28:32.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:32.189+0000 7f0fc9600640 1 -- 172.21.15.5:0/2284071531 >> 172.21.15.5:0/2284071531 conn(0x7f0fd40fd290 msgr2=0x7f0fd4107840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:28:32.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:32.189+0000 7f0fc9600640 1 -- 172.21.15.5:0/2284071531 shutdown_connections 2024-09-10T09:28:32.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:32.189+0000 7f0fc9600640 1 -- 172.21.15.5:0/2284071531 wait complete. 2024-09-10T09:28:32.840 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph orch host ls --format=json 2024-09-10T09:28:33.205 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:28:33.466 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:28:33 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:28:33.466 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:28:33 smithi005 ceph-mon[30350]: Added host smithi159 2024-09-10T09:28:33.466 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:28:33 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:28:33.466 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:28:33 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:28:33.466 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:28:33 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:28:33.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:33.717+0000 7f5a86000640 1 -- 172.21.15.5:0/787259080 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5a80076720 msgr2=0x7f5a80076b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:28:33.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:33.717+0000 7f5a86000640 1 --2- 172.21.15.5:0/787259080 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5a80076720 0x7f5a80076b00 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f5a74009880 tx=0x7f5a7402a5d0 comp rx=0 tx=0).stop 2024-09-10T09:28:33.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:33.717+0000 7f5a86000640 1 -- 172.21.15.5:0/787259080 shutdown_connections 2024-09-10T09:28:33.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:33.717+0000 7f5a86000640 1 --2- 172.21.15.5:0/787259080 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5a80076720 0x7f5a80076b00 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:28:33.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:33.717+0000 7f5a86000640 1 -- 172.21.15.5:0/787259080 >> 172.21.15.5:0/787259080 conn(0x7f5a800ff2c0 msgr2=0x7f5a801016e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:28:33.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:33.717+0000 7f5a86000640 1 -- 172.21.15.5:0/787259080 shutdown_connections 2024-09-10T09:28:33.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:33.717+0000 7f5a86000640 1 -- 172.21.15.5:0/787259080 wait complete. 2024-09-10T09:28:33.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:33.718+0000 7f5a86000640 1 Processor -- start 2024-09-10T09:28:33.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:33.719+0000 7f5a86000640 1 -- start start 2024-09-10T09:28:33.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:33.719+0000 7f5a86000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5a80076720 0x7f5a8019d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:28:33.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:33.719+0000 7f5a86000640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f5a801121d0 con 0x7f5a80076720 2024-09-10T09:28:33.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:33.719+0000 7f5a84c00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5a80076720 0x7f5a8019d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:28:33.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:33.719+0000 7f5a84c00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5a80076720 0x7f5a8019d6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37736/0 (socket says 172.21.15.5:37736) 2024-09-10T09:28:33.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:33.719+0000 7f5a84c00640 1 -- 172.21.15.5:0/3395051842 learned_addr learned my addr 172.21.15.5:0/3395051842 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:28:33.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:33.720+0000 7f5a84c00640 1 -- 172.21.15.5:0/3395051842 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a8019dbf0 con 0x7f5a80076720 2024-09-10T09:28:33.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:33.720+0000 7f5a84c00640 1 --2- 172.21.15.5:0/3395051842 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5a80076720 0x7f5a8019d6b0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f5a74009230 tx=0x7f5a74003d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:28:33.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:33.720+0000 7f5a7d600640 1 -- 172.21.15.5:0/3395051842 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5a7403f070 con 0x7f5a80076720 2024-09-10T09:28:33.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:33.720+0000 7f5a7d600640 1 -- 172.21.15.5:0/3395051842 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5a7403bd40 con 0x7f5a80076720 2024-09-10T09:28:33.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:33.720+0000 7f5a86000640 1 -- 172.21.15.5:0/3395051842 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a8019de80 con 0x7f5a80076720 2024-09-10T09:28:33.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:33.720+0000 7f5a7d600640 1 -- 172.21.15.5:0/3395051842 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5a74035030 con 0x7f5a80076720 2024-09-10T09:28:33.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:33.720+0000 7f5a86000640 1 -- 172.21.15.5:0/3395051842 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a8019e2e0 con 0x7f5a80076720 2024-09-10T09:28:33.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:33.721+0000 7f5a7d600640 1 -- 172.21.15.5:0/3395051842 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 13) ==== 46342+0+0 (secure 0 0 0) 0x7f5a74042070 con 0x7f5a80076720 2024-09-10T09:28:33.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:33.721+0000 7f5a86000640 1 -- 172.21.15.5:0/3395051842 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5a80077e60 con 0x7f5a80076720 2024-09-10T09:28:33.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:33.722+0000 7f5a7d600640 1 --2- 172.21.15.5:0/3395051842 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f5a68039060 0x7f5a6803b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:28:33.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:33.722+0000 7f5a7d600640 1 -- 172.21.15.5:0/3395051842 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f5a7406cd20 con 0x7f5a80076720 2024-09-10T09:28:33.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:33.722+0000 7f5a7fe00640 1 --2- 172.21.15.5:0/3395051842 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f5a68039060 0x7f5a6803b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:28:33.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:33.723+0000 7f5a7fe00640 1 --2- 172.21.15.5:0/3395051842 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f5a68039060 0x7f5a6803b520 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f5a6c0098c0 tx=0x7f5a6c006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:28:33.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:33.727+0000 7f5a7d600640 1 -- 172.21.15.5:0/3395051842 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f5a74030d60 con 0x7f5a80076720 2024-09-10T09:28:33.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:33.919+0000 7f5a86000640 1 -- 172.21.15.5:0/3395051842 --> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f5a8019ea10 con 0x7f5a68039060 2024-09-10T09:28:33.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:33.920+0000 7f5a7d600640 1 -- 172.21.15.5:0/3395051842 <== mgr.14150 v2:172.21.15.5:6800/2707348081 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+239 (secure 0 0 0) 0x7f5a8019ea10 con 0x7f5a68039060 2024-09-10T09:28:33.923 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-10T09:28:33.923 INFO:teuthology.orchestra.run.smithi005.stdout:[{"addr": "172.21.15.5", "hostname": "smithi005", "labels": [], "status": ""}, {"addr": "172.21.15.102", "hostname": "smithi102", "labels": [], "status": ""}, {"addr": "172.21.15.159", "hostname": "smithi159", "labels": [], "status": ""}] 2024-09-10T09:28:33.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:33.923+0000 7f5a86000640 1 -- 172.21.15.5:0/3395051842 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f5a68039060 msgr2=0x7f5a6803b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:28:33.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:33.923+0000 7f5a86000640 1 --2- 172.21.15.5:0/3395051842 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f5a68039060 0x7f5a6803b520 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f5a6c0098c0 tx=0x7f5a6c006db0 comp rx=0 tx=0).stop 2024-09-10T09:28:33.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:33.923+0000 7f5a86000640 1 -- 172.21.15.5:0/3395051842 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5a80076720 msgr2=0x7f5a8019d6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:28:33.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:33.923+0000 7f5a86000640 1 --2- 172.21.15.5:0/3395051842 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5a80076720 0x7f5a8019d6b0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f5a74009230 tx=0x7f5a74003d80 comp rx=0 tx=0).stop 2024-09-10T09:28:33.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:33.923+0000 7f5a86000640 1 -- 172.21.15.5:0/3395051842 shutdown_connections 2024-09-10T09:28:33.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:33.923+0000 7f5a86000640 1 --2- 172.21.15.5:0/3395051842 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f5a68039060 0x7f5a6803b520 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:28:33.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:33.923+0000 7f5a86000640 1 --2- 172.21.15.5:0/3395051842 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5a80076720 0x7f5a8019d6b0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:28:33.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:33.923+0000 7f5a86000640 1 -- 172.21.15.5:0/3395051842 >> 172.21.15.5:0/3395051842 conn(0x7f5a800ff2c0 msgr2=0x7f5a8010dcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:28:33.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:33.923+0000 7f5a86000640 1 -- 172.21.15.5:0/3395051842 shutdown_connections 2024-09-10T09:28:33.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:33.923+0000 7f5a86000640 1 -- 172.21.15.5:0/3395051842 wait complete. 2024-09-10T09:28:34.473 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:28:34 smithi005 ceph-mon[30350]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T09:28:34.473 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:28:34 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:28:34.606 INFO:tasks.cephadm:Setting crush tunables to default 2024-09-10T09:28:34.607 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph osd crush tunables default 2024-09-10T09:28:34.970 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:28:35.361 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:28:35 smithi005 ceph-mon[30350]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T09:28:35.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:35.466+0000 7f9e63800640 1 -- 172.21.15.5:0/2631368606 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9e5c107660 msgr2=0x7f9e5c107a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:28:35.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:35.466+0000 7f9e63800640 1 --2- 172.21.15.5:0/2631368606 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9e5c107660 0x7f9e5c107a40 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f9e50009880 tx=0x7f9e5002a5d0 comp rx=0 tx=0).stop 2024-09-10T09:28:35.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:35.466+0000 7f9e63800640 1 -- 172.21.15.5:0/2631368606 shutdown_connections 2024-09-10T09:28:35.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:35.466+0000 7f9e63800640 1 --2- 172.21.15.5:0/2631368606 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9e5c107660 0x7f9e5c107a40 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:28:35.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:35.466+0000 7f9e63800640 1 -- 172.21.15.5:0/2631368606 >> 172.21.15.5:0/2631368606 conn(0x7f9e5c0fd290 msgr2=0x7f9e5c0ff6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:28:35.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:35.467+0000 7f9e63800640 1 -- 172.21.15.5:0/2631368606 shutdown_connections 2024-09-10T09:28:35.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:35.467+0000 7f9e63800640 1 -- 172.21.15.5:0/2631368606 wait complete. 2024-09-10T09:28:35.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:35.467+0000 7f9e63800640 1 Processor -- start 2024-09-10T09:28:35.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:35.468+0000 7f9e63800640 1 -- start start 2024-09-10T09:28:35.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:35.468+0000 7f9e63800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9e5c107660 0x7f9e5c0744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:28:35.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:35.468+0000 7f9e63800640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f9e5c10bc90 con 0x7f9e5c107660 2024-09-10T09:28:35.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:35.468+0000 7f9e60e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9e5c107660 0x7f9e5c0744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:28:35.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:35.468+0000 7f9e60e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9e5c107660 0x7f9e5c0744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37766/0 (socket says 172.21.15.5:37766) 2024-09-10T09:28:35.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:35.468+0000 7f9e60e00640 1 -- 172.21.15.5:0/2279099030 learned_addr learned my addr 172.21.15.5:0/2279099030 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:28:35.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:35.469+0000 7f9e60e00640 1 -- 172.21.15.5:0/2279099030 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e5c0763e0 con 0x7f9e5c107660 2024-09-10T09:28:35.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:35.469+0000 7f9e60e00640 1 --2- 172.21.15.5:0/2279099030 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9e5c107660 0x7f9e5c0744f0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f9e50009230 tx=0x7f9e50003d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:28:35.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:35.469+0000 7f9e59600640 1 -- 172.21.15.5:0/2279099030 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9e5003f070 con 0x7f9e5c107660 2024-09-10T09:28:35.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:35.469+0000 7f9e59600640 1 -- 172.21.15.5:0/2279099030 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9e5003bc80 con 0x7f9e5c107660 2024-09-10T09:28:35.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:35.469+0000 7f9e59600640 1 -- 172.21.15.5:0/2279099030 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9e50035030 con 0x7f9e5c107660 2024-09-10T09:28:35.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:35.469+0000 7f9e63800640 1 -- 172.21.15.5:0/2279099030 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e5c074a30 con 0x7f9e5c107660 2024-09-10T09:28:35.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:35.469+0000 7f9e63800640 1 -- 172.21.15.5:0/2279099030 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e5c074ef0 con 0x7f9e5c107660 2024-09-10T09:28:35.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:35.471+0000 7f9e63800640 1 -- 172.21.15.5:0/2279099030 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9e5c102cd0 con 0x7f9e5c107660 2024-09-10T09:28:35.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:35.471+0000 7f9e59600640 1 -- 172.21.15.5:0/2279099030 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 13) ==== 46342+0+0 (secure 0 0 0) 0x7f9e50042070 con 0x7f9e5c107660 2024-09-10T09:28:35.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:35.472+0000 7f9e59600640 1 --2- 172.21.15.5:0/2279099030 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f9e440390c0 0x7f9e4403b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:28:35.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:35.472+0000 7f9e59600640 1 -- 172.21.15.5:0/2279099030 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f9e5006cd80 con 0x7f9e5c107660 2024-09-10T09:28:35.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:35.472+0000 7f9e5be00640 1 --2- 172.21.15.5:0/2279099030 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f9e440390c0 0x7f9e4403b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:28:35.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:35.473+0000 7f9e5be00640 1 --2- 172.21.15.5:0/2279099030 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f9e440390c0 0x7f9e4403b580 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f9e480098c0 tx=0x7f9e48006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:28:35.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:35.477+0000 7f9e59600640 1 -- 172.21.15.5:0/2279099030 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f9e500372d0 con 0x7f9e5c107660 2024-09-10T09:28:35.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:35.670+0000 7f9e63800640 1 -- 172.21.15.5:0/2279099030 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7f9e5c075a20 con 0x7f9e5c107660 2024-09-10T09:28:36.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:36.192+0000 7f9e59600640 1 -- 172.21.15.5:0/2279099030 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) ==== 124+0+0 (secure 0 0 0) 0x7f9e50031dd0 con 0x7f9e5c107660 2024-09-10T09:28:36.195 INFO:teuthology.orchestra.run.smithi005.stderr:adjusted tunables profile to default 2024-09-10T09:28:36.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:36.195+0000 7f9e63800640 1 -- 172.21.15.5:0/2279099030 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f9e440390c0 msgr2=0x7f9e4403b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:28:36.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:36.195+0000 7f9e63800640 1 --2- 172.21.15.5:0/2279099030 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f9e440390c0 0x7f9e4403b580 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f9e480098c0 tx=0x7f9e48006db0 comp rx=0 tx=0).stop 2024-09-10T09:28:36.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:36.195+0000 7f9e63800640 1 -- 172.21.15.5:0/2279099030 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9e5c107660 msgr2=0x7f9e5c0744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:28:36.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:36.195+0000 7f9e63800640 1 --2- 172.21.15.5:0/2279099030 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9e5c107660 0x7f9e5c0744f0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f9e50009230 tx=0x7f9e50003d40 comp rx=0 tx=0).stop 2024-09-10T09:28:36.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:36.195+0000 7f9e63800640 1 -- 172.21.15.5:0/2279099030 shutdown_connections 2024-09-10T09:28:36.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:36.195+0000 7f9e63800640 1 --2- 172.21.15.5:0/2279099030 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f9e440390c0 0x7f9e4403b580 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:28:36.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:36.195+0000 7f9e63800640 1 --2- 172.21.15.5:0/2279099030 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9e5c107660 0x7f9e5c0744f0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:28:36.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:36.195+0000 7f9e63800640 1 -- 172.21.15.5:0/2279099030 >> 172.21.15.5:0/2279099030 conn(0x7f9e5c0fd290 msgr2=0x7f9e5c0feb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:28:36.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:36.196+0000 7f9e63800640 1 -- 172.21.15.5:0/2279099030 shutdown_connections 2024-09-10T09:28:36.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:36.196+0000 7f9e63800640 1 -- 172.21.15.5:0/2279099030 wait complete. 2024-09-10T09:28:36.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:28:36 smithi005 ceph-mon[30350]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T09:28:36.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:28:36 smithi005 ceph-mon[30350]: from='client.? 172.21.15.5:0/2279099030' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-09-10T09:28:36.968 INFO:tasks.cephadm:Adding mon.a on smithi005 2024-09-10T09:28:36.969 INFO:tasks.cephadm:Adding mon.b on smithi102 2024-09-10T09:28:36.969 INFO:tasks.cephadm:Adding mon.c on smithi159 2024-09-10T09:28:36.969 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph orch apply mon '3;smithi005:172.21.15.5=a;smithi102:172.21.15.102=b;smithi159:172.21.15.159=c' 2024-09-10T09:28:37.212 INFO:teuthology.orchestra.run.smithi159.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T09:28:37.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:28:37 smithi005 ceph-mon[30350]: from='client.? 172.21.15.5:0/2279099030' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-10T09:28:37.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:28:37 smithi005 ceph-mon[30350]: osdmap e4: 0 total, 0 up, 0 in 2024-09-10T09:28:37.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:28:37 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:28:37.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:28:37 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:28:37.557 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:28:37 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:28:37.557 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:28:37 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:28:37.557 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:28:37 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi159", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:28:37.557 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:28:37 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:28:37.557 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:28:37 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:28:37.557 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:28:37 smithi005 ceph-mon[30350]: Updating smithi159:/etc/ceph/ceph.conf 2024-09-10T09:28:37.678 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:28:37.654+0000 7fd6e6ca8640 1 -- 172.21.15.159:0/1385802977 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd6e0102730 msgr2=0x7fd6e0102b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:28:37.678 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:28:37.654+0000 7fd6e6ca8640 1 --2- 172.21.15.159:0/1385802977 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd6e0102730 0x7fd6e0102b10 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fd6d4009810 tx=0x7fd6d402a540 comp rx=0 tx=0).stop 2024-09-10T09:28:37.678 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:28:37.654+0000 7fd6e6ca8640 1 -- 172.21.15.159:0/1385802977 shutdown_connections 2024-09-10T09:28:37.678 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:28:37.654+0000 7fd6e6ca8640 1 --2- 172.21.15.159:0/1385802977 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd6e0102730 0x7fd6e0102b10 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:28:37.678 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:28:37.654+0000 7fd6e6ca8640 1 -- 172.21.15.159:0/1385802977 >> 172.21.15.159:0/1385802977 conn(0x7fd6e0077310 msgr2=0x7fd6e00fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:28:37.679 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:28:37.654+0000 7fd6e6ca8640 1 -- 172.21.15.159:0/1385802977 shutdown_connections 2024-09-10T09:28:37.679 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:28:37.654+0000 7fd6e6ca8640 1 -- 172.21.15.159:0/1385802977 wait complete. 2024-09-10T09:28:37.679 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:28:37.655+0000 7fd6e6ca8640 1 Processor -- start 2024-09-10T09:28:37.679 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:28:37.655+0000 7fd6e6ca8640 1 -- start start 2024-09-10T09:28:37.679 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:28:37.655+0000 7fd6e6ca8640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd6e0102730 0x7fd6e01971b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:28:37.679 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:28:37.655+0000 7fd6e6ca8640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd6e010bc80 con 0x7fd6e0102730 2024-09-10T09:28:37.679 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:28:37.655+0000 7fd6e4a44640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd6e0102730 0x7fd6e01971b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:28:37.679 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:28:37.655+0000 7fd6e4a44640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd6e0102730 0x7fd6e01971b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.159:44626/0 (socket says 172.21.15.159:44626) 2024-09-10T09:28:37.679 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:28:37.655+0000 7fd6e4a44640 1 -- 172.21.15.159:0/909911548 learned_addr learned my addr 172.21.15.159:0/909911548 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-09-10T09:28:37.680 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:28:37.656+0000 7fd6e4a44640 1 -- 172.21.15.159:0/909911548 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6e01976f0 con 0x7fd6e0102730 2024-09-10T09:28:37.680 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:28:37.656+0000 7fd6e4a44640 1 --2- 172.21.15.159:0/909911548 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd6e0102730 0x7fd6e01971b0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fd6d40045e0 tx=0x7fd6d4004610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:28:37.680 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:28:37.656+0000 7fd6ddffb640 1 -- 172.21.15.159:0/909911548 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd6d4030e70 con 0x7fd6e0102730 2024-09-10T09:28:37.680 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:28:37.656+0000 7fd6e6ca8640 1 -- 172.21.15.159:0/909911548 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6e0197980 con 0x7fd6e0102730 2024-09-10T09:28:37.680 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:28:37.656+0000 7fd6ddffb640 1 -- 172.21.15.159:0/909911548 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd6d40318e0 con 0x7fd6e0102730 2024-09-10T09:28:37.680 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:28:37.656+0000 7fd6ddffb640 1 -- 172.21.15.159:0/909911548 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd6d40049c0 con 0x7fd6e0102730 2024-09-10T09:28:37.680 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:28:37.656+0000 7fd6e6ca8640 1 -- 172.21.15.159:0/909911548 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6e0197de0 con 0x7fd6e0102730 2024-09-10T09:28:37.680 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:28:37.657+0000 7fd6ddffb640 1 -- 172.21.15.159:0/909911548 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 13) ==== 46342+0+0 (secure 0 0 0) 0x7fd6d4039040 con 0x7fd6e0102730 2024-09-10T09:28:37.680 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:28:37.657+0000 7fd6e6ca8640 1 -- 172.21.15.159:0/909911548 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd6e0069f90 con 0x7fd6e0102730 2024-09-10T09:28:37.681 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:28:37.658+0000 7fd6ddffb640 1 --2- 172.21.15.159:0/909911548 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fd6d0039060 0x7fd6d003b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:28:37.681 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:28:37.658+0000 7fd6ddffb640 1 -- 172.21.15.159:0/909911548 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fd6d406d2c0 con 0x7fd6e0102730 2024-09-10T09:28:37.681 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:28:37.658+0000 7fd6dffff640 1 --2- 172.21.15.159:0/909911548 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fd6d0039060 0x7fd6d003b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:28:37.681 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:28:37.658+0000 7fd6dffff640 1 --2- 172.21.15.159:0/909911548 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fd6d0039060 0x7fd6d003b520 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fd6cc0098c0 tx=0x7fd6cc006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:28:37.681 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:28:37.662+0000 7fd6ddffb640 1 -- 172.21.15.159:0/909911548 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd6d4040030 con 0x7fd6e0102730 2024-09-10T09:28:37.845 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:28:37.844+0000 7fd6e6ca8640 1 -- 172.21.15.159:0/909911548 --> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi005:172.21.15.5=a;smithi102:172.21.15.102=b;smithi159:172.21.15.159=c", "target": ["mon-mgr", ""]}) -- 0x7fd6e0198510 con 0x7fd6d0039060 2024-09-10T09:28:37.850 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:28:37.849+0000 7fd6ddffb640 1 -- 172.21.15.159:0/909911548 <== mgr.14150 v2:172.21.15.5:6800/2707348081 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fd6e0198510 con 0x7fd6d0039060 2024-09-10T09:28:37.850 INFO:teuthology.orchestra.run.smithi159.stdout:Scheduled mon update... 2024-09-10T09:28:37.853 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:28:37.852+0000 7fd6e6ca8640 1 -- 172.21.15.159:0/909911548 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fd6d0039060 msgr2=0x7fd6d003b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:28:37.853 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:28:37.852+0000 7fd6e6ca8640 1 --2- 172.21.15.159:0/909911548 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fd6d0039060 0x7fd6d003b520 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fd6cc0098c0 tx=0x7fd6cc006db0 comp rx=0 tx=0).stop 2024-09-10T09:28:37.853 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:28:37.852+0000 7fd6e6ca8640 1 -- 172.21.15.159:0/909911548 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd6e0102730 msgr2=0x7fd6e01971b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:28:37.853 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:28:37.852+0000 7fd6e6ca8640 1 --2- 172.21.15.159:0/909911548 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd6e0102730 0x7fd6e01971b0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fd6d40045e0 tx=0x7fd6d4004610 comp rx=0 tx=0).stop 2024-09-10T09:28:37.853 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:28:37.852+0000 7fd6e6ca8640 1 -- 172.21.15.159:0/909911548 shutdown_connections 2024-09-10T09:28:37.853 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:28:37.852+0000 7fd6e6ca8640 1 --2- 172.21.15.159:0/909911548 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fd6d0039060 0x7fd6d003b520 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:28:37.853 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:28:37.852+0000 7fd6e6ca8640 1 --2- 172.21.15.159:0/909911548 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd6e0102730 0x7fd6e01971b0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:28:37.853 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:28:37.852+0000 7fd6e6ca8640 1 -- 172.21.15.159:0/909911548 >> 172.21.15.159:0/909911548 conn(0x7fd6e0077310 msgr2=0x7fd6e01054c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:28:37.853 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:28:37.852+0000 7fd6e6ca8640 1 -- 172.21.15.159:0/909911548 shutdown_connections 2024-09-10T09:28:37.853 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:28:37.852+0000 7fd6e6ca8640 1 -- 172.21.15.159:0/909911548 wait complete. 2024-09-10T09:28:38.409 DEBUG:teuthology.orchestra.run.smithi102:mon.b> sudo journalctl -f -n 0 -u ceph-bade4b50-6f56-11ef-bcea-c7b262605968@mon.b.service 2024-09-10T09:28:38.412 DEBUG:teuthology.orchestra.run.smithi159:mon.c> sudo journalctl -f -n 0 -u ceph-bade4b50-6f56-11ef-bcea-c7b262605968@mon.c.service 2024-09-10T09:28:38.416 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-10T09:28:38.416 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph mon dump -f json 2024-09-10T09:28:38.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:28:38 smithi005 ceph-mon[30350]: Updating smithi159:/var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T09:28:38.807 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:28:38 smithi005 ceph-mon[30350]: Updating smithi159:/etc/ceph/ceph.client.admin.keyring 2024-09-10T09:28:38.807 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:28:38 smithi005 ceph-mon[30350]: Updating smithi159:/var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-10T09:28:38.807 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:28:38 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:28:38.807 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:28:38 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:28:38.807 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:28:38 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:28:38.807 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:28:38 smithi005 ceph-mon[30350]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T09:28:38.807 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:28:38 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:28:38.807 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:28:38 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:28:38.807 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:28:38 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:28:38.807 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:28:38 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:28:38.808 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:28:38 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:28:38.808 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:28:38 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-10T09:28:38.808 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:28:38 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:28:39.097 INFO:teuthology.orchestra.run.smithi159.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T09:28:39.510 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:28:39.509+0000 7f0dc5173640 1 -- 172.21.15.159:0/113305112 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0dc01027b0 msgr2=0x7f0dc0102b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:28:39.510 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:28:39.509+0000 7f0dc5173640 1 --2- 172.21.15.159:0/113305112 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0dc01027b0 0x7f0dc0102b90 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f0db0009880 tx=0x7f0db002a670 comp rx=0 tx=0).stop 2024-09-10T09:28:39.510 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:28:39.509+0000 7f0dc5173640 1 -- 172.21.15.159:0/113305112 shutdown_connections 2024-09-10T09:28:39.510 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:28:39.509+0000 7f0dc5173640 1 --2- 172.21.15.159:0/113305112 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0dc01027b0 0x7f0dc0102b90 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:28:39.510 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:28:39.509+0000 7f0dc5173640 1 -- 172.21.15.159:0/113305112 >> 172.21.15.159:0/113305112 conn(0x7f0dc00fc1b0 msgr2=0x7f0dc00fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:28:39.511 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:28:39.509+0000 7f0dc5173640 1 -- 172.21.15.159:0/113305112 shutdown_connections 2024-09-10T09:28:39.511 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:28:39.509+0000 7f0dc5173640 1 -- 172.21.15.159:0/113305112 wait complete. 2024-09-10T09:28:39.511 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:28:39.510+0000 7f0dc5173640 1 Processor -- start 2024-09-10T09:28:39.512 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:28:39.510+0000 7f0dc5173640 1 -- start start 2024-09-10T09:28:39.512 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:28:39.510+0000 7f0dc5173640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0dc01027b0 0x7f0dc01993d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:28:39.512 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:28:39.510+0000 7f0dc5173640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f0dc010bcb0 con 0x7f0dc01027b0 2024-09-10T09:28:39.512 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:28:39.510+0000 7f0dbed9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0dc01027b0 0x7f0dc01993d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:28:39.512 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:28:39.511+0000 7f0dbed9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0dc01027b0 0x7f0dc01993d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.159:44632/0 (socket says 172.21.15.159:44632) 2024-09-10T09:28:39.512 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:28:39.511+0000 7f0dbed9d640 1 -- 172.21.15.159:0/173151215 learned_addr learned my addr 172.21.15.159:0/173151215 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-09-10T09:28:39.512 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:28:39.511+0000 7f0dbed9d640 1 -- 172.21.15.159:0/173151215 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0dc0199910 con 0x7f0dc01027b0 2024-09-10T09:28:39.513 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:28:39.511+0000 7f0dbed9d640 1 --2- 172.21.15.159:0/173151215 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0dc01027b0 0x7f0dc01993d0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f0db0009230 tx=0x7f0db0003d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:28:39.513 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:28:39.511+0000 7f0da7fff640 1 -- 172.21.15.159:0/173151215 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0db003f070 con 0x7f0dc01027b0 2024-09-10T09:28:39.513 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:28:39.511+0000 7f0dc5173640 1 -- 172.21.15.159:0/173151215 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0dc0199ba0 con 0x7f0dc01027b0 2024-09-10T09:28:39.513 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:28:39.511+0000 7f0dc5173640 1 -- 172.21.15.159:0/173151215 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0dc019a000 con 0x7f0dc01027b0 2024-09-10T09:28:39.513 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:28:39.511+0000 7f0da7fff640 1 -- 172.21.15.159:0/173151215 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0db003bc80 con 0x7f0dc01027b0 2024-09-10T09:28:39.513 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:28:39.511+0000 7f0da7fff640 1 -- 172.21.15.159:0/173151215 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0db0049060 con 0x7f0dc01027b0 2024-09-10T09:28:39.514 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:28:39.512+0000 7f0dc5173640 1 -- 172.21.15.159:0/173151215 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0dac005290 con 0x7f0dc01027b0 2024-09-10T09:28:39.514 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:28:39.513+0000 7f0da7fff640 1 -- 172.21.15.159:0/173151215 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 13) ==== 46342+0+0 (secure 0 0 0) 0x7f0db0042070 con 0x7f0dc01027b0 2024-09-10T09:28:39.514 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:28:39.513+0000 7f0da7fff640 1 --2- 172.21.15.159:0/173151215 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f0da8039060 0x7f0da803b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:28:39.514 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:28:39.513+0000 7f0da7fff640 1 -- 172.21.15.159:0/173151215 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f0db006c960 con 0x7f0dc01027b0 2024-09-10T09:28:39.514 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:28:39.513+0000 7f0dbe59c640 1 --2- 172.21.15.159:0/173151215 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f0da8039060 0x7f0da803b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:28:39.515 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:28:39.514+0000 7f0dbe59c640 1 --2- 172.21.15.159:0/173151215 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f0da8039060 0x7f0da803b520 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f0db40098c0 tx=0x7f0db4006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:28:39.519 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:28:39.518+0000 7f0da7fff640 1 -- 172.21.15.159:0/173151215 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f0db0040260 con 0x7f0dc01027b0 2024-09-10T09:28:39.754 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:28:39.753+0000 7f0dc5173640 1 -- 172.21.15.159:0/173151215 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f0dac005580 con 0x7f0dc01027b0 2024-09-10T09:28:39.755 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:28:39.754+0000 7f0da7fff640 1 -- 172.21.15.159:0/173151215 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+741 (secure 0 0 0) 0x7f0db0044e10 con 0x7f0dc01027b0 2024-09-10T09:28:39.755 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-10T09:28:39.755 INFO:teuthology.orchestra.run.smithi159.stdout:{"epoch":1,"fsid":"bade4b50-6f56-11ef-bcea-c7b262605968","modified":"2024-09-10T09:27:08.776915Z","created":"2024-09-10T09:27:08.776915Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:3300","nonce":0},{"type":"v1","addr":"172.21.15.5:6789","nonce":0}]},"addr":"172.21.15.5:6789/0","public_addr":"172.21.15.5:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-10T09:28:39.756 INFO:teuthology.orchestra.run.smithi159.stderr:dumped monmap epoch 1 2024-09-10T09:28:39.758 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:28:39.756+0000 7f0dc5173640 1 -- 172.21.15.159:0/173151215 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f0da8039060 msgr2=0x7f0da803b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:28:39.758 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:28:39.756+0000 7f0dc5173640 1 --2- 172.21.15.159:0/173151215 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f0da8039060 0x7f0da803b520 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f0db40098c0 tx=0x7f0db4006db0 comp rx=0 tx=0).stop 2024-09-10T09:28:39.758 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:28:39.756+0000 7f0dc5173640 1 -- 172.21.15.159:0/173151215 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0dc01027b0 msgr2=0x7f0dc01993d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:28:39.758 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:28:39.756+0000 7f0dc5173640 1 --2- 172.21.15.159:0/173151215 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0dc01027b0 0x7f0dc01993d0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f0db0009230 tx=0x7f0db0003d40 comp rx=0 tx=0).stop 2024-09-10T09:28:39.758 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:28:39.757+0000 7f0dc5173640 1 -- 172.21.15.159:0/173151215 shutdown_connections 2024-09-10T09:28:39.758 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:28:39.757+0000 7f0dc5173640 1 --2- 172.21.15.159:0/173151215 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f0da8039060 0x7f0da803b520 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:28:39.759 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:28:39.757+0000 7f0dc5173640 1 --2- 172.21.15.159:0/173151215 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0dc01027b0 0x7f0dc01993d0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:28:39.759 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:28:39.757+0000 7f0dc5173640 1 -- 172.21.15.159:0/173151215 >> 172.21.15.159:0/173151215 conn(0x7f0dc00fc1b0 msgr2=0x7f0dc0107970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:28:39.759 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:28:39.757+0000 7f0dc5173640 1 -- 172.21.15.159:0/173151215 shutdown_connections 2024-09-10T09:28:39.759 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:28:39.757+0000 7f0dc5173640 1 -- 172.21.15.159:0/173151215 wait complete. 2024-09-10T09:28:39.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:28:39 smithi005 ceph-mon[30350]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi005:172.21.15.5=a;smithi102:172.21.15.102=b;smithi159:172.21.15.159=c", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T09:28:39.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:28:39 smithi005 ceph-mon[30350]: Saving service mon spec with placement smithi005:172.21.15.5=a;smithi102:172.21.15.102=b;smithi159:172.21.15.159=c;count:3 2024-09-10T09:28:39.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:28:39 smithi005 ceph-mon[30350]: Deploying daemon mon.c on smithi159 2024-09-10T09:28:40.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:28:40 smithi005 ceph-mon[30350]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T09:28:40.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:28:40 smithi005 ceph-mon[30350]: from='client.? 172.21.15.159:0/173151215' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-10T09:28:41.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:28:41 smithi005 ceph-mon[30350]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T09:28:42.120 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-10T09:28:42.120 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph mon dump -f json 2024-09-10T09:28:42.487 INFO:teuthology.orchestra.run.smithi159.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.c/config 2024-09-10T09:28:42.930 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:28:42.928+0000 7f047a6ff640 1 -- 172.21.15.159:0/2049225306 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0474106630 msgr2=0x7f0474106a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:28:42.930 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:28:42.928+0000 7f047a6ff640 1 --2- 172.21.15.159:0/2049225306 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0474106630 0x7f0474106a10 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f0464009880 tx=0x7f046402a5d0 comp rx=0 tx=0).stop 2024-09-10T09:28:42.931 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:28:42.928+0000 7f047a6ff640 1 -- 172.21.15.159:0/2049225306 shutdown_connections 2024-09-10T09:28:42.931 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:28:42.928+0000 7f047a6ff640 1 --2- 172.21.15.159:0/2049225306 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0474106630 0x7f0474106a10 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:28:42.931 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:28:42.928+0000 7f047a6ff640 1 -- 172.21.15.159:0/2049225306 >> 172.21.15.159:0/2049225306 conn(0x7f04740fc220 msgr2=0x7f04740fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:28:42.931 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:28:42.928+0000 7f047a6ff640 1 -- 172.21.15.159:0/2049225306 shutdown_connections 2024-09-10T09:28:42.931 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:28:42.928+0000 7f047a6ff640 1 -- 172.21.15.159:0/2049225306 wait complete. 2024-09-10T09:28:42.931 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:28:42.928+0000 7f047a6ff640 1 Processor -- start 2024-09-10T09:28:42.931 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:28:42.928+0000 7f047a6ff640 1 -- start start 2024-09-10T09:28:42.931 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:28:42.929+0000 7f047a6ff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0474106630 0x7f04741021f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:28:42.931 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:28:42.929+0000 7f047a6ff640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f047410ac60 con 0x7f0474106630 2024-09-10T09:28:42.931 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:28:42.929+0000 7f0473fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0474106630 0x7f04741021f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:28:42.931 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:28:42.929+0000 7f0473fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0474106630 0x7f04741021f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.159:44646/0 (socket says 172.21.15.159:44646) 2024-09-10T09:28:42.931 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:28:42.929+0000 7f0473fff640 1 -- 172.21.15.159:0/3833375885 learned_addr learned my addr 172.21.15.159:0/3833375885 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-09-10T09:28:42.931 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:28:42.929+0000 7f0473fff640 1 -- 172.21.15.159:0/3833375885 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04741040e0 con 0x7f0474106630 2024-09-10T09:28:42.931 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:28:42.930+0000 7f0473fff640 1 --2- 172.21.15.159:0/3833375885 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0474106630 0x7f04741021f0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f0464009230 tx=0x7f0464003d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:28:42.933 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:28:42.930+0000 7f04717fa640 1 -- 172.21.15.159:0/3833375885 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f046403f070 con 0x7f0474106630 2024-09-10T09:28:42.933 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:28:42.930+0000 7f04717fa640 1 -- 172.21.15.159:0/3833375885 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f046403bc80 con 0x7f0474106630 2024-09-10T09:28:42.933 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:28:42.930+0000 7f047a6ff640 1 -- 172.21.15.159:0/3833375885 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0474102730 con 0x7f0474106630 2024-09-10T09:28:42.933 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:28:42.930+0000 7f04717fa640 1 -- 172.21.15.159:0/3833375885 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0464035030 con 0x7f0474106630 2024-09-10T09:28:42.933 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:28:42.930+0000 7f047a6ff640 1 -- 172.21.15.159:0/3833375885 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0474102bf0 con 0x7f0474106630 2024-09-10T09:28:42.933 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:28:42.931+0000 7f047a6ff640 1 -- 172.21.15.159:0/3833375885 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0474101ca0 con 0x7f0474106630 2024-09-10T09:28:42.933 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:28:42.931+0000 7f04717fa640 1 -- 172.21.15.159:0/3833375885 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 13) ==== 46342+0+0 (secure 0 0 0) 0x7f0464042070 con 0x7f0474106630 2024-09-10T09:28:42.933 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:28:42.931+0000 7f04717fa640 1 --2- 172.21.15.159:0/3833375885 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f045c0390c0 0x7f045c03b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:28:42.933 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:28:42.931+0000 7f04717fa640 1 -- 172.21.15.159:0/3833375885 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f046406cdb0 con 0x7f0474106630 2024-09-10T09:28:42.933 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:28:42.932+0000 7f04737fe640 1 --2- 172.21.15.159:0/3833375885 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f045c0390c0 0x7f045c03b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:28:42.934 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:28:42.932+0000 7f04737fe640 1 --2- 172.21.15.159:0/3833375885 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f045c0390c0 0x7f045c03b580 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f04680098c0 tx=0x7f0468006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:28:42.937 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:28:42.936+0000 7f04717fa640 1 -- 172.21.15.159:0/3833375885 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f04640372d0 con 0x7f0474106630 2024-09-10T09:28:43.176 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:28:43.175+0000 7f047a6ff640 1 -- 172.21.15.159:0/3833375885 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f0474103720 con 0x7f0474106630 2024-09-10T09:28:43.176 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:28:43.176+0000 7f04717fa640 1 -- 172.21.15.159:0/3833375885 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+741 (secure 0 0 0) 0x7f0464031e20 con 0x7f0474106630 2024-09-10T09:28:43.177 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-10T09:28:43.177 INFO:teuthology.orchestra.run.smithi159.stdout:{"epoch":1,"fsid":"bade4b50-6f56-11ef-bcea-c7b262605968","modified":"2024-09-10T09:27:08.776915Z","created":"2024-09-10T09:27:08.776915Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:3300","nonce":0},{"type":"v1","addr":"172.21.15.5:6789","nonce":0}]},"addr":"172.21.15.5:6789/0","public_addr":"172.21.15.5:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-10T09:28:43.177 INFO:teuthology.orchestra.run.smithi159.stderr:dumped monmap epoch 1 2024-09-10T09:28:43.179 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:28:43.178+0000 7f047a6ff640 1 -- 172.21.15.159:0/3833375885 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f045c0390c0 msgr2=0x7f045c03b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:28:43.179 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:28:43.178+0000 7f047a6ff640 1 --2- 172.21.15.159:0/3833375885 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f045c0390c0 0x7f045c03b580 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f04680098c0 tx=0x7f0468006db0 comp rx=0 tx=0).stop 2024-09-10T09:28:43.179 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:28:43.178+0000 7f047a6ff640 1 -- 172.21.15.159:0/3833375885 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0474106630 msgr2=0x7f04741021f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:28:43.179 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:28:43.178+0000 7f047a6ff640 1 --2- 172.21.15.159:0/3833375885 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0474106630 0x7f04741021f0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f0464009230 tx=0x7f0464003d40 comp rx=0 tx=0).stop 2024-09-10T09:28:43.179 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:28:43.178+0000 7f047a6ff640 1 -- 172.21.15.159:0/3833375885 shutdown_connections 2024-09-10T09:28:43.179 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:28:43.178+0000 7f047a6ff640 1 --2- 172.21.15.159:0/3833375885 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f045c0390c0 0x7f045c03b580 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:28:43.179 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:28:43.178+0000 7f047a6ff640 1 --2- 172.21.15.159:0/3833375885 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0474106630 0x7f04741021f0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:28:43.179 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:28:43.178+0000 7f047a6ff640 1 -- 172.21.15.159:0/3833375885 >> 172.21.15.159:0/3833375885 conn(0x7f04740fc220 msgr2=0x7f04740fe4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:28:43.179 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:28:43.178+0000 7f047a6ff640 1 -- 172.21.15.159:0/3833375885 shutdown_connections 2024-09-10T09:28:43.179 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:28:43.178+0000 7f047a6ff640 1 -- 172.21.15.159:0/3833375885 wait complete. 2024-09-10T09:28:43.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:28:43 smithi005 ceph-mon[30350]: from='client.? 172.21.15.159:0/3833375885' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-10T09:28:45.159 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-10T09:28:45.159 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph mon dump -f json 2024-09-10T09:28:45.428 INFO:teuthology.orchestra.run.smithi159.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.c/config 2024-09-10T09:28:48.849 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:28:48.847+0000 7f62e7b1a640 1 -- 172.21.15.159:0/2631365832 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f62e0107660 msgr2=0x7f62c8005580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:28:48.849 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:28:48.847+0000 7f62e7b1a640 1 --2- 172.21.15.159:0/2631365832 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f62e0107660 0x7f62c8005580 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f62d0011c50 tx=0x7f62d002be50 comp rx=0 tx=0).stop 2024-09-10T09:28:48.849 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:28:48.847+0000 7f62e7b1a640 1 -- 172.21.15.159:0/2631365832 shutdown_connections 2024-09-10T09:28:48.849 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:28:48.847+0000 7f62e7b1a640 1 --2- 172.21.15.159:0/2631365832 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f62e0107660 0x7f62c8005580 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:28:48.849 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:28:48.847+0000 7f62e7b1a640 1 -- 172.21.15.159:0/2631365832 >> 172.21.15.159:0/2631365832 conn(0x7f62e00fd320 msgr2=0x7f62e00ff740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:28:48.849 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:28:48.847+0000 7f62e7b1a640 1 -- 172.21.15.159:0/2631365832 shutdown_connections 2024-09-10T09:28:48.849 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:28:48.847+0000 7f62e7b1a640 1 -- 172.21.15.159:0/2631365832 wait complete. 2024-09-10T09:28:48.849 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:28:48.848+0000 7f62e7b1a640 1 Processor -- start 2024-09-10T09:28:48.849 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:28:48.848+0000 7f62e7b1a640 1 -- start start 2024-09-10T09:28:48.850 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:28:48.848+0000 7f62e7b1a640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f62e0107660 0x7f62e01994b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:28:48.850 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:28:48.848+0000 7f62e7b1a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f62e01999f0 0x7f62e019dd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:28:48.850 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:28:48.848+0000 7f62e7b1a640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f62e0109780 con 0x7f62e01999f0 2024-09-10T09:28:48.850 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:28:48.848+0000 7f62e7b1a640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f62e0109600 con 0x7f62e0107660 2024-09-10T09:28:48.851 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:28:48.848+0000 7f62e58b6640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f62e0107660 0x7f62e01994b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:28:48.851 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:28:48.848+0000 7f62e58b6640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f62e0107660 0x7f62e01994b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:49290/0 (socket says 172.21.15.159:49290) 2024-09-10T09:28:48.851 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:28:48.848+0000 7f62e58b6640 1 -- 172.21.15.159:0/74186351 learned_addr learned my addr 172.21.15.159:0/74186351 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-09-10T09:28:48.851 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:28:48.848+0000 7f62e58b6640 1 -- 172.21.15.159:0/74186351 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f62e0107660 msgr2=0x7f62e01994b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-09-10T09:28:48.852 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:28:48.848+0000 7f62e58b6640 1 -- 172.21.15.159:0/74186351 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f62e0107660 msgr2=0x7f62e01994b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-10T09:28:48.852 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:28:48.848+0000 7f62e58b6640 1 --2- 172.21.15.159:0/74186351 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f62e0107660 0x7f62e01994b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-10T09:28:48.852 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:28:48.848+0000 7f62e58b6640 1 --2- 172.21.15.159:0/74186351 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f62e0107660 0x7f62e01994b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T09:28:48.852 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:28:48.849+0000 7f62e50b5640 1 --2- 172.21.15.159:0/74186351 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f62e01999f0 0x7f62e019dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:28:48.852 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:28:48.849+0000 7f62e50b5640 1 -- 172.21.15.159:0/74186351 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f62e0107660 msgr2=0x7f62e01994b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T09:28:48.852 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:28:48.849+0000 7f62e50b5640 1 --2- 172.21.15.159:0/74186351 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f62e0107660 0x7f62e01994b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:28:48.852 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:28:48.849+0000 7f62e50b5640 1 -- 172.21.15.159:0/74186351 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62e019a010 con 0x7f62e01999f0 2024-09-10T09:28:48.852 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:28:48.849+0000 7f62e50b5640 1 --2- 172.21.15.159:0/74186351 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f62e01999f0 0x7f62e019dd80 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f62d400b9b0 tx=0x7f62d400be70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:28:48.852 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:28:48.849+0000 7f62deffd640 1 -- 172.21.15.159:0/74186351 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f62d400c920 con 0x7f62e01999f0 2024-09-10T09:28:48.852 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:28:48.849+0000 7f62e7b1a640 1 -- 172.21.15.159:0/74186351 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62e019e2c0 con 0x7f62e01999f0 2024-09-10T09:28:48.852 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:28:48.850+0000 7f62deffd640 1 -- 172.21.15.159:0/74186351 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f62d4014440 con 0x7f62e01999f0 2024-09-10T09:28:48.852 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:28:48.850+0000 7f62e7b1a640 1 -- 172.21.15.159:0/74186351 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62e019e800 con 0x7f62e01999f0 2024-09-10T09:28:48.852 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:28:48.850+0000 7f62deffd640 1 -- 172.21.15.159:0/74186351 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f62d4012500 con 0x7f62e01999f0 2024-09-10T09:28:48.853 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:28:48.851+0000 7f62deffd640 1 -- 172.21.15.159:0/74186351 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 13) ==== 46342+0+0 (secure 0 0 0) 0x7f62d401c070 con 0x7f62e01999f0 2024-09-10T09:28:48.853 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:28:48.851+0000 7f62e7b1a640 1 -- 172.21.15.159:0/74186351 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f62e0102da0 con 0x7f62e01999f0 2024-09-10T09:28:48.854 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:28:48.851+0000 7f62deffd640 1 --2- 172.21.15.159:0/74186351 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f62cc038e50 0x7f62cc03b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:28:48.854 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:28:48.851+0000 7f62deffd640 1 -- 172.21.15.159:0/74186351 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f62d404b900 con 0x7f62e01999f0 2024-09-10T09:28:48.854 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:28:48.852+0000 7f62e58b6640 1 --2- 172.21.15.159:0/74186351 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f62cc038e50 0x7f62cc03b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:28:48.854 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:28:48.852+0000 7f62e58b6640 1 --2- 172.21.15.159:0/74186351 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f62cc038e50 0x7f62cc03b310 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f62d0005940 tx=0x7f62d00058b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:28:48.858 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:28:48.857+0000 7f62deffd640 1 -- 172.21.15.159:0/74186351 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f62d40137a0 con 0x7f62e01999f0 2024-09-10T09:28:49.057 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:28:48 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T09:28:49.057 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:28:48 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T09:28:49.057 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:28:48 smithi005 ceph-mon[30350]: mon.a calling monitor election 2024-09-10T09:28:49.057 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:28:48 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T09:28:49.057 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:28:48 smithi005 ceph-mon[30350]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T09:28:49.057 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:28:48 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T09:28:49.057 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:28:48 smithi005 ceph-mon[30350]: mon.c calling monitor election 2024-09-10T09:28:49.057 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:28:48 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T09:28:49.057 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:28:48 smithi005 ceph-mon[30350]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T09:28:49.057 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:28:48 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T09:28:49.057 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:28:48 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T09:28:49.057 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:28:48 smithi005 ceph-mon[30350]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-10T09:28:49.058 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:28:48 smithi005 ceph-mon[30350]: monmap epoch 2 2024-09-10T09:28:49.058 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:28:48 smithi005 ceph-mon[30350]: fsid bade4b50-6f56-11ef-bcea-c7b262605968 2024-09-10T09:28:49.058 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:28:48 smithi005 ceph-mon[30350]: last_changed 2024-09-10T09:28:43.568819+0000 2024-09-10T09:28:49.058 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:28:48 smithi005 ceph-mon[30350]: created 2024-09-10T09:27:08.776915+0000 2024-09-10T09:28:49.058 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:28:48 smithi005 ceph-mon[30350]: min_mon_release 19 (squid) 2024-09-10T09:28:49.058 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:28:48 smithi005 ceph-mon[30350]: election_strategy: 1 2024-09-10T09:28:49.058 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:28:48 smithi005 ceph-mon[30350]: 0: [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] mon.a 2024-09-10T09:28:49.058 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:28:48 smithi005 ceph-mon[30350]: 1: [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] mon.c 2024-09-10T09:28:49.058 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:28:48 smithi005 ceph-mon[30350]: fsmap 2024-09-10T09:28:49.058 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:28:48 smithi005 ceph-mon[30350]: osdmap e4: 0 total, 0 up, 0 in 2024-09-10T09:28:49.058 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:28:48 smithi005 ceph-mon[30350]: mgrmap e13: x(active, since 43s) 2024-09-10T09:28:49.058 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:28:48 smithi005 ceph-mon[30350]: overall HEALTH_OK 2024-09-10T09:28:49.058 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:28:48 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:28:49.058 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:28:48 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:28:49.059 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:28:48 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:28:49.059 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:28:48 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-10T09:28:49.059 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:28:48 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:28:49.092 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:28:49.091+0000 7f62e7b1a640 1 -- 172.21.15.159:0/74186351 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f62e019eba0 con 0x7f62e01999f0 2024-09-10T09:28:49.093 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:28:49.092+0000 7f62deffd640 1 -- 172.21.15.159:0/74186351 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1011 (secure 0 0 0) 0x7f62d400e090 con 0x7f62e01999f0 2024-09-10T09:28:49.093 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-10T09:28:49.093 INFO:teuthology.orchestra.run.smithi159.stdout:{"epoch":2,"fsid":"bade4b50-6f56-11ef-bcea-c7b262605968","modified":"2024-09-10T09:28:43.568819Z","created":"2024-09-10T09:27:08.776915Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:3300","nonce":0},{"type":"v1","addr":"172.21.15.5:6789","nonce":0}]},"addr":"172.21.15.5:6789/0","public_addr":"172.21.15.5:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:3300","nonce":0},{"type":"v1","addr":"172.21.15.159:6789","nonce":0}]},"addr":"172.21.15.159:6789/0","public_addr":"172.21.15.159:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-10T09:28:49.093 INFO:teuthology.orchestra.run.smithi159.stderr:dumped monmap epoch 2 2024-09-10T09:28:49.095 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:28:49.094+0000 7f62e7b1a640 1 -- 172.21.15.159:0/74186351 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f62cc038e50 msgr2=0x7f62cc03b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:28:49.095 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:28:49.094+0000 7f62e7b1a640 1 --2- 172.21.15.159:0/74186351 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f62cc038e50 0x7f62cc03b310 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f62d0005940 tx=0x7f62d00058b0 comp rx=0 tx=0).stop 2024-09-10T09:28:49.095 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:28:49.094+0000 7f62e7b1a640 1 -- 172.21.15.159:0/74186351 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f62e01999f0 msgr2=0x7f62e019dd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:28:49.095 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:28:49.094+0000 7f62e7b1a640 1 --2- 172.21.15.159:0/74186351 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f62e01999f0 0x7f62e019dd80 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f62d400b9b0 tx=0x7f62d400be70 comp rx=0 tx=0).stop 2024-09-10T09:28:49.095 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:28:49.094+0000 7f62e7b1a640 1 -- 172.21.15.159:0/74186351 shutdown_connections 2024-09-10T09:28:49.095 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:28:49.094+0000 7f62e7b1a640 1 --2- 172.21.15.159:0/74186351 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f62cc038e50 0x7f62cc03b310 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:28:49.095 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:28:49.094+0000 7f62e7b1a640 1 --2- 172.21.15.159:0/74186351 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f62e01999f0 0x7f62e019dd80 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:28:49.096 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:28:49.094+0000 7f62e7b1a640 1 --2- 172.21.15.159:0/74186351 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f62e0107660 0x7f62e01994b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:28:49.096 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:28:49.094+0000 7f62e7b1a640 1 -- 172.21.15.159:0/74186351 >> 172.21.15.159:0/74186351 conn(0x7f62e00fd320 msgr2=0x7f62e0074ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:28:49.096 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:28:49.094+0000 7f62e7b1a640 1 -- 172.21.15.159:0/74186351 shutdown_connections 2024-09-10T09:28:49.096 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:28:49.094+0000 7f62e7b1a640 1 -- 172.21.15.159:0/74186351 wait complete. 2024-09-10T09:28:50.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:28:49 smithi005 ceph-mon[30350]: Deploying daemon mon.b on smithi102 2024-09-10T09:28:50.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:28:49 smithi005 ceph-mon[30350]: from='client.? 172.21.15.159:0/74186351' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-10T09:28:50.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:28:49 smithi005 ceph-mon[30350]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T09:28:50.057 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:28:49 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T09:28:50.715 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-10T09:28:50.716 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph mon dump -f json 2024-09-10T09:28:50.966 INFO:teuthology.orchestra.run.smithi159.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.c/config 2024-09-10T09:28:51.437 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:28:51.434+0000 7f4034be4640 1 -- 172.21.15.159:0/1912140615 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4030107660 msgr2=0x7f4030107a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:28:51.437 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:28:51.435+0000 7f4034be4640 1 --2- 172.21.15.159:0/1912140615 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4030107660 0x7f4030107a40 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f4020009880 tx=0x7f402002a5d0 comp rx=0 tx=0).stop 2024-09-10T09:28:51.437 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:28:51.435+0000 7f4034be4640 1 -- 172.21.15.159:0/1912140615 shutdown_connections 2024-09-10T09:28:51.437 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:28:51.435+0000 7f4034be4640 1 --2- 172.21.15.159:0/1912140615 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4030107660 0x7f4030107a40 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:28:51.438 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:28:51.435+0000 7f4034be4640 1 -- 172.21.15.159:0/1912140615 >> 172.21.15.159:0/1912140615 conn(0x7f40300fd320 msgr2=0x7f40300ff740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:28:51.438 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:28:51.435+0000 7f4034be4640 1 -- 172.21.15.159:0/1912140615 shutdown_connections 2024-09-10T09:28:51.438 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:28:51.435+0000 7f4034be4640 1 -- 172.21.15.159:0/1912140615 wait complete. 2024-09-10T09:28:51.438 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:28:51.436+0000 7f4034be4640 1 Processor -- start 2024-09-10T09:28:51.438 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:28:51.436+0000 7f4034be4640 1 -- start start 2024-09-10T09:28:51.438 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:28:51.436+0000 7f4034be4640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4030107660 0x7f40301973c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:28:51.438 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:28:51.436+0000 7f4034be4640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4030197900 0x7f403019bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:28:51.438 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:28:51.436+0000 7f4034be4640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f403010bd20 con 0x7f4030197900 2024-09-10T09:28:51.438 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:28:51.436+0000 7f4034be4640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f403010bba0 con 0x7f4030107660 2024-09-10T09:28:51.438 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:28:51.436+0000 7f402e59c640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4030107660 0x7f40301973c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:28:51.438 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:28:51.436+0000 7f402e59c640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4030107660 0x7f40301973c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:49298/0 (socket says 172.21.15.159:49298) 2024-09-10T09:28:51.438 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:28:51.436+0000 7f402e59c640 1 -- 172.21.15.159:0/4130106616 learned_addr learned my addr 172.21.15.159:0/4130106616 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-09-10T09:28:51.439 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:28:51.436+0000 7f402e59c640 1 -- 172.21.15.159:0/4130106616 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4030107660 msgr2=0x7f40301973c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-09-10T09:28:51.439 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:28:51.436+0000 7f402e59c640 1 -- 172.21.15.159:0/4130106616 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4030107660 msgr2=0x7f40301973c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-10T09:28:51.439 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:28:51.436+0000 7f402e59c640 1 --2- 172.21.15.159:0/4130106616 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4030107660 0x7f40301973c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-10T09:28:51.439 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:28:51.436+0000 7f402e59c640 1 --2- 172.21.15.159:0/4130106616 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4030107660 0x7f40301973c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T09:28:51.439 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:28:51.436+0000 7f402dd9b640 1 --2- 172.21.15.159:0/4130106616 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4030197900 0x7f403019bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:28:51.440 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:28:51.437+0000 7f402dd9b640 1 -- 172.21.15.159:0/4130106616 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4030107660 msgr2=0x7f40301973c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T09:28:51.440 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:28:51.437+0000 7f402dd9b640 1 --2- 172.21.15.159:0/4130106616 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4030107660 0x7f40301973c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:28:51.440 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:28:51.437+0000 7f402dd9b640 1 -- 172.21.15.159:0/4130106616 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4030197f20 con 0x7f4030197900 2024-09-10T09:28:51.441 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:28:51.437+0000 7f402dd9b640 1 --2- 172.21.15.159:0/4130106616 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4030197900 0x7f403019bc90 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f402400b3d0 tx=0x7f402400b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:28:51.441 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:28:51.437+0000 7f401f7fe640 1 -- 172.21.15.159:0/4130106616 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f4024011630 con 0x7f4030197900 2024-09-10T09:28:51.441 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:28:51.437+0000 7f4034be4640 1 -- 172.21.15.159:0/4130106616 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f403019c1d0 con 0x7f4030197900 2024-09-10T09:28:51.441 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:28:51.437+0000 7f401f7fe640 1 -- 172.21.15.159:0/4130106616 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4024011cb0 con 0x7f4030197900 2024-09-10T09:28:51.441 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:28:51.437+0000 7f401f7fe640 1 -- 172.21.15.159:0/4130106616 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f4024010df0 con 0x7f4030197900 2024-09-10T09:28:51.441 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:28:51.437+0000 7f4034be4640 1 -- 172.21.15.159:0/4130106616 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f403019c710 con 0x7f4030197900 2024-09-10T09:28:51.441 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:28:51.438+0000 7f4034be4640 1 -- 172.21.15.159:0/4130106616 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4030102da0 con 0x7f4030197900 2024-09-10T09:28:51.441 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:28:51.438+0000 7f401f7fe640 1 -- 172.21.15.159:0/4130106616 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 13) ==== 46342+0+0 (secure 0 0 0) 0x7f4024018020 con 0x7f4030197900 2024-09-10T09:28:51.441 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:28:51.439+0000 7f401f7fe640 1 --2- 172.21.15.159:0/4130106616 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f4018038e00 0x7f401803b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:28:51.441 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:28:51.439+0000 7f401f7fe640 1 -- 172.21.15.159:0/4130106616 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f402404bcc0 con 0x7f4030197900 2024-09-10T09:28:51.441 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:28:51.439+0000 7f402e59c640 1 --2- 172.21.15.159:0/4130106616 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f4018038e00 0x7f401803b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:28:51.441 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:28:51.440+0000 7f402e59c640 1 --2- 172.21.15.159:0/4130106616 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f4018038e00 0x7f401803b2c0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f40200096c0 tx=0x7f4020005a40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:28:51.445 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:28:51.444+0000 7f401f7fe640 1 -- 172.21.15.159:0/4130106616 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f402401f020 con 0x7f4030197900 2024-09-10T09:28:51.677 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:28:51.676+0000 7f4034be4640 1 -- 172.21.15.159:0/4130106616 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f403019ca20 con 0x7f4030197900 2024-09-10T09:28:51.677 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:28:51 smithi159 ceph-mon[35513]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T09:28:51.678 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:28:51.677+0000 7f401f7fe640 1 -- 172.21.15.159:0/4130106616 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1011 (secure 0 0 0) 0x7f40240172c0 con 0x7f4030197900 2024-09-10T09:28:51.678 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-10T09:28:51.678 INFO:teuthology.orchestra.run.smithi159.stdout:{"epoch":2,"fsid":"bade4b50-6f56-11ef-bcea-c7b262605968","modified":"2024-09-10T09:28:43.568819Z","created":"2024-09-10T09:27:08.776915Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:3300","nonce":0},{"type":"v1","addr":"172.21.15.5:6789","nonce":0}]},"addr":"172.21.15.5:6789/0","public_addr":"172.21.15.5:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:3300","nonce":0},{"type":"v1","addr":"172.21.15.159:6789","nonce":0}]},"addr":"172.21.15.159:6789/0","public_addr":"172.21.15.159:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-10T09:28:51.678 INFO:teuthology.orchestra.run.smithi159.stderr:dumped monmap epoch 2 2024-09-10T09:28:51.680 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:28:51.679+0000 7f4034be4640 1 -- 172.21.15.159:0/4130106616 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f4018038e00 msgr2=0x7f401803b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:28:51.680 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:28:51.679+0000 7f4034be4640 1 --2- 172.21.15.159:0/4130106616 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f4018038e00 0x7f401803b2c0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f40200096c0 tx=0x7f4020005a40 comp rx=0 tx=0).stop 2024-09-10T09:28:51.680 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:28:51.679+0000 7f4034be4640 1 -- 172.21.15.159:0/4130106616 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4030197900 msgr2=0x7f403019bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:28:51.681 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:28:51.679+0000 7f4034be4640 1 --2- 172.21.15.159:0/4130106616 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4030197900 0x7f403019bc90 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f402400b3d0 tx=0x7f402400b890 comp rx=0 tx=0).stop 2024-09-10T09:28:51.681 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:28:51.679+0000 7f4034be4640 1 -- 172.21.15.159:0/4130106616 shutdown_connections 2024-09-10T09:28:51.681 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:28:51.679+0000 7f4034be4640 1 --2- 172.21.15.159:0/4130106616 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f4018038e00 0x7f401803b2c0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:28:51.681 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:28:51.679+0000 7f4034be4640 1 --2- 172.21.15.159:0/4130106616 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4030197900 0x7f403019bc90 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:28:51.681 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:28:51.679+0000 7f4034be4640 1 --2- 172.21.15.159:0/4130106616 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4030107660 0x7f40301973c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:28:51.681 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:28:51.679+0000 7f4034be4640 1 -- 172.21.15.159:0/4130106616 >> 172.21.15.159:0/4130106616 conn(0x7f40300fd320 msgr2=0x7f40300fee10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:28:51.681 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:28:51.679+0000 7f4034be4640 1 -- 172.21.15.159:0/4130106616 shutdown_connections 2024-09-10T09:28:51.681 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:28:51.679+0000 7f4034be4640 1 -- 172.21.15.159:0/4130106616 wait complete. 2024-09-10T09:28:51.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:28:51 smithi005 ceph-mon[30350]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T09:28:52.167 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 podman[35423]: 2024-09-10 09:28:52.145427685 +0000 UTC m=+0.339620664 container start 36a004c0c1829086524204befeaf8e702aae1ff698b9d0024da65c178ff10952 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mon-b, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b) 2024-09-10T09:28:52.635 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: set uid:gid to 167:167 (ceph:ceph) 2024-09-10T09:28:52.635 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable), process ceph-mon, pid 2 2024-09-10T09:28:52.635 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: pidfile_write: ignore empty --pid-file 2024-09-10T09:28:52.635 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: load: jerasure load: lrc 2024-09-10T09:28:52.635 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: rocksdb: RocksDB version: 7.9.2 2024-09-10T09:28:52.635 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: rocksdb: Git sha 0 2024-09-10T09:28:52.635 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: rocksdb: Compile date 2024-09-09 20:43:47 2024-09-10T09:28:52.635 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: rocksdb: DB SUMMARY 2024-09-10T09:28:52.635 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: rocksdb: DB Session ID: YO8JUIB6CAUXQOGT90PC 2024-09-10T09:28:52.636 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: rocksdb: CURRENT file: CURRENT 2024-09-10T09:28:52.636 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: rocksdb: IDENTITY file: IDENTITY 2024-09-10T09:28:52.636 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-09-10T09:28:52.636 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: rocksdb: SST files in /var/lib/ceph/mon/ceph-b/store.db dir, Total Num: 0, files: 2024-09-10T09:28:52.636 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-b/store.db: 000004.log size: 636 ; 2024-09-10T09:28:52.636 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: rocksdb: Options.error_if_exists: 0 2024-09-10T09:28:52.636 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: rocksdb: Options.create_if_missing: 0 2024-09-10T09:28:52.636 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: rocksdb: Options.paranoid_checks: 1 2024-09-10T09:28:52.636 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: rocksdb: Options.flush_verify_memtable_count: 1 2024-09-10T09:28:52.636 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-09-10T09:28:52.636 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-09-10T09:28:52.636 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: rocksdb: Options.env: 0x5653fa532940 2024-09-10T09:28:52.636 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: rocksdb: Options.fs: PosixFileSystem 2024-09-10T09:28:52.637 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: rocksdb: Options.info_log: 0x5653fc7336e0 2024-09-10T09:28:52.637 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: rocksdb: Options.max_file_opening_threads: 16 2024-09-10T09:28:52.637 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: rocksdb: Options.statistics: (nil) 2024-09-10T09:28:52.637 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: rocksdb: Options.use_fsync: 0 2024-09-10T09:28:52.637 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: rocksdb: Options.max_log_file_size: 0 2024-09-10T09:28:52.637 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-10T09:28:52.637 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: rocksdb: Options.log_file_time_to_roll: 0 2024-09-10T09:28:52.637 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: rocksdb: Options.keep_log_file_num: 1000 2024-09-10T09:28:52.637 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: rocksdb: Options.recycle_log_file_num: 0 2024-09-10T09:28:52.637 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: rocksdb: Options.allow_fallocate: 1 2024-09-10T09:28:52.637 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: rocksdb: Options.allow_mmap_reads: 0 2024-09-10T09:28:52.637 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: rocksdb: Options.allow_mmap_writes: 0 2024-09-10T09:28:52.638 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: rocksdb: Options.use_direct_reads: 0 2024-09-10T09:28:52.638 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-10T09:28:52.638 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: rocksdb: Options.create_missing_column_families: 0 2024-09-10T09:28:52.638 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: rocksdb: Options.db_log_dir: 2024-09-10T09:28:52.638 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: rocksdb: Options.wal_dir: 2024-09-10T09:28:52.638 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: rocksdb: Options.table_cache_numshardbits: 6 2024-09-10T09:28:52.640 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: rocksdb: Options.WAL_ttl_seconds: 0 2024-09-10T09:28:52.640 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: rocksdb: Options.WAL_size_limit_MB: 0 2024-09-10T09:28:52.640 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-10T09:28:52.640 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-10T09:28:52.640 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: rocksdb: Options.is_fd_close_on_exec: 1 2024-09-10T09:28:52.640 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: rocksdb: Options.advise_random_on_open: 1 2024-09-10T09:28:52.640 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: rocksdb: Options.db_write_buffer_size: 0 2024-09-10T09:28:52.640 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: rocksdb: Options.write_buffer_manager: 0x5653fc742dc0 2024-09-10T09:28:52.640 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-10T09:28:52.641 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-10T09:28:52.641 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: rocksdb: Options.use_adaptive_mutex: 0 2024-09-10T09:28:52.641 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: rocksdb: Options.rate_limiter: (nil) 2024-09-10T09:28:52.641 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-10T09:28:52.641 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: rocksdb: Options.wal_recovery_mode: 2 2024-09-10T09:28:52.641 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: rocksdb: Options.enable_thread_tracking: 0 2024-09-10T09:28:52.641 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: rocksdb: Options.enable_pipelined_write: 0 2024-09-10T09:28:52.641 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: rocksdb: Options.unordered_write: 0 2024-09-10T09:28:52.641 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-10T09:28:52.641 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-10T09:28:52.641 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-10T09:28:52.641 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-10T09:28:52.642 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: rocksdb: Options.row_cache: None 2024-09-10T09:28:52.642 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: rocksdb: Options.wal_filter: None 2024-09-10T09:28:52.642 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-10T09:28:52.642 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: rocksdb: Options.allow_ingest_behind: 0 2024-09-10T09:28:52.642 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: rocksdb: Options.two_write_queues: 0 2024-09-10T09:28:52.642 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: rocksdb: Options.manual_wal_flush: 0 2024-09-10T09:28:52.642 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: rocksdb: Options.wal_compression: 0 2024-09-10T09:28:52.642 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: rocksdb: Options.atomic_flush: 0 2024-09-10T09:28:52.642 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-10T09:28:52.642 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: rocksdb: Options.persist_stats_to_disk: 0 2024-09-10T09:28:52.642 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: rocksdb: Options.write_dbid_to_manifest: 0 2024-09-10T09:28:52.642 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: rocksdb: Options.log_readahead_size: 0 2024-09-10T09:28:52.643 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-10T09:28:52.643 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: rocksdb: Options.best_efforts_recovery: 0 2024-09-10T09:28:52.643 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-10T09:28:52.643 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-10T09:28:52.643 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: rocksdb: Options.allow_data_in_errors: 0 2024-09-10T09:28:52.643 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: rocksdb: Options.db_host_id: __hostname__ 2024-09-10T09:28:52.643 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: rocksdb: Options.enforce_single_del_contracts: true 2024-09-10T09:28:52.643 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: rocksdb: Options.max_background_jobs: 2 2024-09-10T09:28:52.643 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: rocksdb: Options.max_background_compactions: -1 2024-09-10T09:28:52.643 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: rocksdb: Options.max_subcompactions: 1 2024-09-10T09:28:52.643 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-10T09:28:52.643 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-10T09:28:52.643 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: rocksdb: Options.delayed_write_rate : 16777216 2024-09-10T09:28:52.644 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: rocksdb: Options.max_total_wal_size: 0 2024-09-10T09:28:52.644 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-10T09:28:52.644 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: rocksdb: Options.stats_dump_period_sec: 600 2024-09-10T09:28:52.644 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: rocksdb: Options.stats_persist_period_sec: 600 2024-09-10T09:28:52.644 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-10T09:28:52.644 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: rocksdb: Options.max_open_files: -1 2024-09-10T09:28:52.644 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: rocksdb: Options.bytes_per_sync: 0 2024-09-10T09:28:52.644 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: rocksdb: Options.wal_bytes_per_sync: 0 2024-09-10T09:28:52.644 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: rocksdb: Options.strict_bytes_per_sync: 0 2024-09-10T09:28:52.644 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: rocksdb: Options.compaction_readahead_size: 0 2024-09-10T09:28:52.644 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: rocksdb: Options.max_background_flushes: -1 2024-09-10T09:28:52.644 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: rocksdb: Compression algorithms supported: 2024-09-10T09:28:52.644 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: rocksdb: kZSTD supported: 0 2024-09-10T09:28:52.645 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: rocksdb: kXpressCompression supported: 0 2024-09-10T09:28:52.645 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: rocksdb: kBZip2Compression supported: 0 2024-09-10T09:28:52.645 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-10T09:28:52.645 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: rocksdb: kLZ4Compression supported: 1 2024-09-10T09:28:52.645 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: rocksdb: kZlibCompression supported: 1 2024-09-10T09:28:52.645 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: rocksdb: kLZ4HCCompression supported: 1 2024-09-10T09:28:52.645 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: rocksdb: kSnappyCompression supported: 1 2024-09-10T09:28:52.645 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: rocksdb: Fast CRC32 supported: Supported on x86 2024-09-10T09:28:52.645 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: rocksdb: DMutex implementation: pthread_mutex_t 2024-09-10T09:28:52.645 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 2024-09-10T09:28:52.645 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-09-10T09:28:52.645 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-10T09:28:52.648 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: rocksdb: Options.merge_operator: 2024-09-10T09:28:52.648 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: rocksdb: Options.compaction_filter: None 2024-09-10T09:28:52.648 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: rocksdb: Options.compaction_filter_factory: None 2024-09-10T09:28:52.648 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: rocksdb: Options.sst_partitioner_factory: None 2024-09-10T09:28:52.648 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: rocksdb: Options.memtable_factory: SkipListFactory 2024-09-10T09:28:52.648 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: rocksdb: Options.table_factory: BlockBasedTable 2024-09-10T09:28:52.648 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x5653fc733320) 2024-09-10T09:28:52.648 INFO:journalctl@ceph.mon.b.smithi102.stdout: cache_index_and_filter_blocks: 1 2024-09-10T09:28:52.648 INFO:journalctl@ceph.mon.b.smithi102.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-10T09:28:52.648 INFO:journalctl@ceph.mon.b.smithi102.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-10T09:28:52.648 INFO:journalctl@ceph.mon.b.smithi102.stdout: pin_top_level_index_and_filter: 1 2024-09-10T09:28:52.649 INFO:journalctl@ceph.mon.b.smithi102.stdout: index_type: 0 2024-09-10T09:28:52.649 INFO:journalctl@ceph.mon.b.smithi102.stdout: data_block_index_type: 0 2024-09-10T09:28:52.649 INFO:journalctl@ceph.mon.b.smithi102.stdout: index_shortening: 1 2024-09-10T09:28:52.649 INFO:journalctl@ceph.mon.b.smithi102.stdout: data_block_hash_table_util_ratio: 0.750000 2024-09-10T09:28:52.649 INFO:journalctl@ceph.mon.b.smithi102.stdout: checksum: 4 2024-09-10T09:28:52.649 INFO:journalctl@ceph.mon.b.smithi102.stdout: no_block_cache: 0 2024-09-10T09:28:52.649 INFO:journalctl@ceph.mon.b.smithi102.stdout: block_cache: 0x5653fc7271f0 2024-09-10T09:28:52.649 INFO:journalctl@ceph.mon.b.smithi102.stdout: block_cache_name: BinnedLRUCache 2024-09-10T09:28:52.649 INFO:journalctl@ceph.mon.b.smithi102.stdout: block_cache_options: 2024-09-10T09:28:52.650 INFO:journalctl@ceph.mon.b.smithi102.stdout: capacity : 536870912 2024-09-10T09:28:52.650 INFO:journalctl@ceph.mon.b.smithi102.stdout: num_shard_bits : 4 2024-09-10T09:28:52.650 INFO:journalctl@ceph.mon.b.smithi102.stdout: strict_capacity_limit : 0 2024-09-10T09:28:52.650 INFO:journalctl@ceph.mon.b.smithi102.stdout: high_pri_pool_ratio: 0.000 2024-09-10T09:28:52.650 INFO:journalctl@ceph.mon.b.smithi102.stdout: block_cache_compressed: (nil) 2024-09-10T09:28:52.650 INFO:journalctl@ceph.mon.b.smithi102.stdout: persistent_cache: (nil) 2024-09-10T09:28:52.650 INFO:journalctl@ceph.mon.b.smithi102.stdout: block_size: 4096 2024-09-10T09:28:52.651 INFO:journalctl@ceph.mon.b.smithi102.stdout: block_size_deviation: 10 2024-09-10T09:28:52.651 INFO:journalctl@ceph.mon.b.smithi102.stdout: block_restart_interval: 16 2024-09-10T09:28:52.651 INFO:journalctl@ceph.mon.b.smithi102.stdout: index_block_restart_interval: 1 2024-09-10T09:28:52.651 INFO:journalctl@ceph.mon.b.smithi102.stdout: metadata_block_size: 4096 2024-09-10T09:28:52.651 INFO:journalctl@ceph.mon.b.smithi102.stdout: partition_filters: 0 2024-09-10T09:28:52.651 INFO:journalctl@ceph.mon.b.smithi102.stdout: use_delta_encoding: 1 2024-09-10T09:28:52.652 INFO:journalctl@ceph.mon.b.smithi102.stdout: filter_policy: bloomfilter 2024-09-10T09:28:52.652 INFO:journalctl@ceph.mon.b.smithi102.stdout: whole_key_filtering: 1 2024-09-10T09:28:52.652 INFO:journalctl@ceph.mon.b.smithi102.stdout: verify_compression: 0 2024-09-10T09:28:52.652 INFO:journalctl@ceph.mon.b.smithi102.stdout: read_amp_bytes_per_bit: 0 2024-09-10T09:28:52.652 INFO:journalctl@ceph.mon.b.smithi102.stdout: format_version: 5 2024-09-10T09:28:52.652 INFO:journalctl@ceph.mon.b.smithi102.stdout: enable_index_compression: 1 2024-09-10T09:28:52.653 INFO:journalctl@ceph.mon.b.smithi102.stdout: block_align: 0 2024-09-10T09:28:52.653 INFO:journalctl@ceph.mon.b.smithi102.stdout: max_auto_readahead_size: 262144 2024-09-10T09:28:52.653 INFO:journalctl@ceph.mon.b.smithi102.stdout: prepopulate_block_cache: 0 2024-09-10T09:28:52.653 INFO:journalctl@ceph.mon.b.smithi102.stdout: initial_auto_readahead_size: 8192 2024-09-10T09:28:52.653 INFO:journalctl@ceph.mon.b.smithi102.stdout: num_file_reads_for_auto_readahead: 2 2024-09-10T09:28:52.653 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: rocksdb: Options.write_buffer_size: 33554432 2024-09-10T09:28:52.653 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: rocksdb: Options.max_write_buffer_number: 2 2024-09-10T09:28:52.653 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: rocksdb: Options.compression: NoCompression 2024-09-10T09:28:52.653 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: rocksdb: Options.bottommost_compression: Disabled 2024-09-10T09:28:52.653 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: rocksdb: Options.prefix_extractor: nullptr 2024-09-10T09:28:52.653 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-10T09:28:52.653 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: rocksdb: Options.num_levels: 7 2024-09-10T09:28:52.654 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-10T09:28:52.654 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-10T09:28:52.654 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-10T09:28:52.654 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-10T09:28:52.654 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-10T09:28:52.654 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-10T09:28:52.654 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-10T09:28:52.654 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-10T09:28:52.654 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-10T09:28:52.654 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-10T09:28:52.654 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-09-10T09:28:52.654 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-09-10T09:28:52.654 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: rocksdb: Options.compression_opts.window_bits: -14 2024-09-10T09:28:52.654 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: rocksdb: Options.compression_opts.level: 32767 2024-09-10T09:28:52.655 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: rocksdb: Options.compression_opts.strategy: 0 2024-09-10T09:28:52.655 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-10T09:28:52.655 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-10T09:28:52.655 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-09-10T09:28:52.655 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-10T09:28:52.655 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: rocksdb: Options.compression_opts.enabled: false 2024-09-10T09:28:52.655 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-09-10T09:28:52.655 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-10T09:28:52.655 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-10T09:28:52.655 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-10T09:28:52.655 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: rocksdb: Options.target_file_size_base: 67108864 2024-09-10T09:28:52.655 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: rocksdb: Options.target_file_size_multiplier: 1 2024-09-10T09:28:52.655 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-10T09:28:52.656 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-10T09:28:52.656 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-10T09:28:52.656 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-10T09:28:52.656 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-10T09:28:52.656 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-10T09:28:52.656 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-10T09:28:52.656 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-10T09:28:52.656 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-10T09:28:52.656 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-10T09:28:52.656 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-10T09:28:52.656 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-10T09:28:52.656 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-09-10T09:28:52.656 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: rocksdb: Options.arena_block_size: 1048576 2024-09-10T09:28:52.656 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-10T09:28:52.657 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-10T09:28:52.657 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: rocksdb: Options.disable_auto_compactions: 0 2024-09-10T09:28:52.657 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-10T09:28:52.657 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-10T09:28:52.657 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-10T09:28:52.657 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-10T09:28:52.657 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-10T09:28:52.657 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-10T09:28:52.657 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-10T09:28:52.657 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-10T09:28:52.657 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-10T09:28:52.657 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-10T09:28:52.657 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-10T09:28:52.658 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: rocksdb: Options.inplace_update_support: 0 2024-09-10T09:28:52.658 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-10T09:28:52.658 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-10T09:28:52.658 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-10T09:28:52.658 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-10T09:28:52.658 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: rocksdb: Options.bloom_locality: 0 2024-09-10T09:28:52.658 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: rocksdb: Options.max_successive_merges: 0 2024-09-10T09:28:52.658 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-10T09:28:52.658 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: rocksdb: Options.paranoid_file_checks: 0 2024-09-10T09:28:52.658 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: rocksdb: Options.force_consistency_checks: 1 2024-09-10T09:28:52.659 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: rocksdb: Options.report_bg_io_stats: 0 2024-09-10T09:28:52.659 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: rocksdb: Options.ttl: 2592000 2024-09-10T09:28:52.659 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-10T09:28:52.659 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-09-10T09:28:52.659 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-09-10T09:28:52.659 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: rocksdb: Options.enable_blob_files: false 2024-09-10T09:28:52.659 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: rocksdb: Options.min_blob_size: 0 2024-09-10T09:28:52.659 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: rocksdb: Options.blob_file_size: 268435456 2024-09-10T09:28:52.659 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-10T09:28:52.659 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-10T09:28:52.659 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-10T09:28:52.659 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-09-10T09:28:52.659 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-09-10T09:28:52.660 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: rocksdb: Options.blob_file_starting_level: 0 2024-09-10T09:28:52.660 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-09-10T09:28:52.660 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-09-10T09:28:52.660 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-09-10T09:28:52.660 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 7ce9c47c-4f55-4358-96cd-28fce411366f 2024-09-10T09:28:52.660 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: rocksdb: EVENT_LOG_v1 {"time_micros": 1725960532172612, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-09-10T09:28:52.660 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-09-10T09:28:52.660 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: rocksdb: EVENT_LOG_v1 {"time_micros": 1725960532173018, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1773, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 648, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 526, "raw_average_value_size": 105, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1725960532, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "7ce9c47c-4f55-4358-96cd-28fce411366f", "db_session_id": "YO8JUIB6CAUXQOGT90PC", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-09-10T09:28:52.660 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: rocksdb: EVENT_LOG_v1 {"time_micros": 1725960532173118, "job": 1, "event": "recovery_finished"} 2024-09-10T09:28:52.660 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-09-10T09:28:52.661 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-b/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-10T09:28:52.661 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x5653fc754e00 2024-09-10T09:28:52.661 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: rocksdb: DB pointer 0x5653fc850000 2024-09-10T09:28:52.661 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: mon.b does not exist in monmap, will attempt to join an existing cluster 2024-09-10T09:28:52.661 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-09-10T09:28:52.661 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-09-10T09:28:52.661 INFO:journalctl@ceph.mon.b.smithi102.stdout: ** DB Stats ** 2024-09-10T09:28:52.661 INFO:journalctl@ceph.mon.b.smithi102.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-10T09:28:52.661 INFO:journalctl@ceph.mon.b.smithi102.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-10T09:28:52.661 INFO:journalctl@ceph.mon.b.smithi102.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-10T09:28:52.661 INFO:journalctl@ceph.mon.b.smithi102.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-10T09:28:52.662 INFO:journalctl@ceph.mon.b.smithi102.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-10T09:28:52.662 INFO:journalctl@ceph.mon.b.smithi102.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-10T09:28:52.662 INFO:journalctl@ceph.mon.b.smithi102.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-10T09:28:52.662 INFO:journalctl@ceph.mon.b.smithi102.stdout: 2024-09-10T09:28:52.662 INFO:journalctl@ceph.mon.b.smithi102.stdout: ** Compaction Stats [default] ** 2024-09-10T09:28:52.662 INFO:journalctl@ceph.mon.b.smithi102.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-10T09:28:52.662 INFO:journalctl@ceph.mon.b.smithi102.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-09-10T09:28:52.662 INFO:journalctl@ceph.mon.b.smithi102.stdout: L0 1/0 1.73 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.4 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-10T09:28:52.662 INFO:journalctl@ceph.mon.b.smithi102.stdout: Sum 1/0 1.73 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.4 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-10T09:28:52.662 INFO:journalctl@ceph.mon.b.smithi102.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.4 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-10T09:28:52.662 INFO:journalctl@ceph.mon.b.smithi102.stdout: 2024-09-10T09:28:52.662 INFO:journalctl@ceph.mon.b.smithi102.stdout: ** Compaction Stats [default] ** 2024-09-10T09:28:52.662 INFO:journalctl@ceph.mon.b.smithi102.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-10T09:28:52.663 INFO:journalctl@ceph.mon.b.smithi102.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-10T09:28:52.663 INFO:journalctl@ceph.mon.b.smithi102.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 4.4 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-10T09:28:52.663 INFO:journalctl@ceph.mon.b.smithi102.stdout: 2024-09-10T09:28:52.663 INFO:journalctl@ceph.mon.b.smithi102.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-09-10T09:28:52.663 INFO:journalctl@ceph.mon.b.smithi102.stdout: 2024-09-10T09:28:52.663 INFO:journalctl@ceph.mon.b.smithi102.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-10T09:28:52.663 INFO:journalctl@ceph.mon.b.smithi102.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-10T09:28:52.663 INFO:journalctl@ceph.mon.b.smithi102.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-10T09:28:52.664 INFO:journalctl@ceph.mon.b.smithi102.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-10T09:28:52.664 INFO:journalctl@ceph.mon.b.smithi102.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-10T09:28:52.664 INFO:journalctl@ceph.mon.b.smithi102.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-10T09:28:52.664 INFO:journalctl@ceph.mon.b.smithi102.stdout: Cumulative compaction: 0.00 GB write, 0.59 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-10T09:28:52.664 INFO:journalctl@ceph.mon.b.smithi102.stdout: Interval compaction: 0.00 GB write, 0.59 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-10T09:28:52.664 INFO:journalctl@ceph.mon.b.smithi102.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-10T09:28:52.665 INFO:journalctl@ceph.mon.b.smithi102.stdout: Block cache BinnedLRUCache@0x5653fc7271f0#2 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-09-10T09:28:52.665 INFO:journalctl@ceph.mon.b.smithi102.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-09-10T09:28:52.665 INFO:journalctl@ceph.mon.b.smithi102.stdout: 2024-09-10T09:28:52.665 INFO:journalctl@ceph.mon.b.smithi102.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-10T09:28:52.665 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: using public_addr v2:172.21.15.102:0/0 -> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] 2024-09-10T09:28:52.665 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: starting mon.b rank -1 at public addrs [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] at bind addrs [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] mon_data /var/lib/ceph/mon/ceph-b fsid bade4b50-6f56-11ef-bcea-c7b262605968 2024-09-10T09:28:52.666 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: mon.b@-1(???) e0 preinit fsid bade4b50-6f56-11ef-bcea-c7b262605968 2024-09-10T09:28:52.666 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: mon.b@-1(synchronizing).mds e1 new map 2024-09-10T09:28:52.666 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: mon.b@-1(synchronizing).mds e1 print_map 2024-09-10T09:28:52.666 INFO:journalctl@ceph.mon.b.smithi102.stdout: e1 2024-09-10T09:28:52.666 INFO:journalctl@ceph.mon.b.smithi102.stdout: btime 2024-09-10T09:27:13:298289+0000 2024-09-10T09:28:52.666 INFO:journalctl@ceph.mon.b.smithi102.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-10T09:28:52.666 INFO:journalctl@ceph.mon.b.smithi102.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-10T09:28:52.667 INFO:journalctl@ceph.mon.b.smithi102.stdout: legacy client fscid: -1 2024-09-10T09:28:52.667 INFO:journalctl@ceph.mon.b.smithi102.stdout: 2024-09-10T09:28:52.667 INFO:journalctl@ceph.mon.b.smithi102.stdout: No filesystems configured 2024-09-10T09:28:52.667 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: mon.b@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-09-10T09:28:52.667 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: mon.b@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-09-10T09:28:52.667 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: mon.b@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-09-10T09:28:52.667 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: mon.b@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-09-10T09:28:52.667 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: mon.b@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-09-10T09:28:52.668 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: mon.b@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-09-10T09:28:52.668 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: mon.b@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-10T09:28:52.668 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-10T09:28:52.668 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-10T09:28:52.668 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-10T09:28:52.668 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T09:28:52.668 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:28:52.669 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: Added host smithi159 2024-09-10T09:28:52.669 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:28:52.669 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:28:52.669 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:28:52.669 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T09:28:52.669 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:28:52.670 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T09:28:52.670 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T09:28:52.670 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: from='client.? 172.21.15.5:0/2279099030' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-09-10T09:28:52.670 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: from='client.? 172.21.15.5:0/2279099030' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-10T09:28:52.670 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: osdmap e4: 0 total, 0 up, 0 in 2024-09-10T09:28:52.670 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:28:52.671 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:28:52.671 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:28:52.671 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:28:52.671 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi159", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:28:52.671 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:28:52.671 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:28:52.671 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: Updating smithi159:/etc/ceph/ceph.conf 2024-09-10T09:28:52.671 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: Updating smithi159:/var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T09:28:52.672 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: Updating smithi159:/etc/ceph/ceph.client.admin.keyring 2024-09-10T09:28:52.672 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: Updating smithi159:/var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-10T09:28:52.672 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:28:52.672 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:28:52.672 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:28:52.672 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T09:28:52.672 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:28:52.672 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:28:52.672 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:28:52.672 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:28:52.672 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:28:52.672 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-10T09:28:52.673 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:28:52.673 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi005:172.21.15.5=a;smithi102:172.21.15.102=b;smithi159:172.21.15.159=c", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T09:28:52.673 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: Saving service mon spec with placement smithi005:172.21.15.5=a;smithi102:172.21.15.102=b;smithi159:172.21.15.159=c;count:3 2024-09-10T09:28:52.673 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: Deploying daemon mon.c on smithi159 2024-09-10T09:28:52.673 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T09:28:52.673 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: from='client.? 172.21.15.159:0/173151215' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-10T09:28:52.673 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T09:28:52.673 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: from='client.? 172.21.15.159:0/3833375885' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-10T09:28:52.673 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T09:28:52.673 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T09:28:52.673 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: mon.a calling monitor election 2024-09-10T09:28:52.673 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T09:28:52.673 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T09:28:52.674 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T09:28:52.674 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: mon.c calling monitor election 2024-09-10T09:28:52.674 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T09:28:52.674 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T09:28:52.674 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T09:28:52.674 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T09:28:52.674 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-10T09:28:52.674 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: monmap epoch 2 2024-09-10T09:28:52.674 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: fsid bade4b50-6f56-11ef-bcea-c7b262605968 2024-09-10T09:28:52.674 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: last_changed 2024-09-10T09:28:43.568819+0000 2024-09-10T09:28:52.674 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: created 2024-09-10T09:27:08.776915+0000 2024-09-10T09:28:52.674 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: min_mon_release 19 (squid) 2024-09-10T09:28:52.674 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: election_strategy: 1 2024-09-10T09:28:52.674 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: 0: [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] mon.a 2024-09-10T09:28:52.674 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: 1: [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] mon.c 2024-09-10T09:28:52.675 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: fsmap 2024-09-10T09:28:52.675 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: osdmap e4: 0 total, 0 up, 0 in 2024-09-10T09:28:52.675 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: mgrmap e13: x(active, since 43s) 2024-09-10T09:28:52.675 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: overall HEALTH_OK 2024-09-10T09:28:52.675 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:28:52.675 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:28:52.675 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:28:52.675 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-10T09:28:52.675 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:28:52.675 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: Deploying daemon mon.b on smithi102 2024-09-10T09:28:52.675 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: from='client.? 172.21.15.159:0/74186351' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-10T09:28:52.675 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T09:28:52.675 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T09:28:52.675 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T09:28:52.676 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 ceph-mon[35442]: mon.b@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-09-10T09:28:52.676 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 bash[35423]: 36a004c0c1829086524204befeaf8e702aae1ff698b9d0024da65c178ff10952 2024-09-10T09:28:52.676 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:52 smithi102 systemd[1]: Started Ceph mon.b for bade4b50-6f56-11ef-bcea-c7b262605968. 2024-09-10T09:28:53.321 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-10T09:28:53.321 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph mon dump -f json 2024-09-10T09:28:53.571 INFO:teuthology.orchestra.run.smithi159.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.c/config 2024-09-10T09:28:57.220 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:28:57.218+0000 7f40b320b640 1 -- 172.21.15.159:0/848245184 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f40ac106630 msgr2=0x7f4094007790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:28:57.220 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:28:57.218+0000 7f40b320b640 1 --2- 172.21.15.159:0/848245184 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f40ac106630 0x7f4094007790 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f409c013ce0 tx=0x7f409c02c350 comp rx=0 tx=0).stop 2024-09-10T09:28:57.220 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:28:57.218+0000 7f40b320b640 1 -- 172.21.15.159:0/848245184 shutdown_connections 2024-09-10T09:28:57.220 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:28:57.218+0000 7f40b320b640 1 --2- 172.21.15.159:0/848245184 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f40ac106630 0x7f4094007790 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:28:57.220 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:28:57.218+0000 7f40b320b640 1 -- 172.21.15.159:0/848245184 >> 172.21.15.159:0/848245184 conn(0x7f40ac0fc220 msgr2=0x7f40ac0fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:28:57.220 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:28:57.218+0000 7f40b320b640 1 -- 172.21.15.159:0/848245184 shutdown_connections 2024-09-10T09:28:57.220 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:28:57.218+0000 7f40b320b640 1 -- 172.21.15.159:0/848245184 wait complete. 2024-09-10T09:28:57.220 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:28:57.219+0000 7f40b320b640 1 Processor -- start 2024-09-10T09:28:57.220 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:28:57.219+0000 7f40b320b640 1 -- start start 2024-09-10T09:28:57.221 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:28:57.219+0000 7f40b320b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f40ac106630 0x7f40ac1995d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:28:57.221 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:28:57.219+0000 7f40b320b640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f40ac199b10 0x7f40ac19dea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:28:57.221 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:28:57.219+0000 7f40b320b640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f40ac19e3e0 0x7f40ac19e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:28:57.221 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:28:57.219+0000 7f40b320b640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f40ac107550 con 0x7f40ac106630 2024-09-10T09:28:57.221 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:28:57.219+0000 7f40b320b640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f40ac1073d0 con 0x7f40ac19e3e0 2024-09-10T09:28:57.221 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:28:57.219+0000 7f40b320b640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f40ac1076d0 con 0x7f40ac199b10 2024-09-10T09:28:57.222 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:28:57.219+0000 7f40abfff640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f40ac199b10 0x7f40ac19dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:28:57.223 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:28:57.219+0000 7f40abfff640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f40ac199b10 0x7f40ac19dea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:42394/0 (socket says 172.21.15.159:42394) 2024-09-10T09:28:57.223 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:28:57.219+0000 7f40abfff640 1 -- 172.21.15.159:0/478084434 learned_addr learned my addr 172.21.15.159:0/478084434 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-09-10T09:28:57.223 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:28:57.219+0000 7f40b0fa7640 1 --2- 172.21.15.159:0/478084434 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f40ac106630 0x7f40ac1995d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:28:57.223 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:28:57.219+0000 7f40abfff640 1 -- 172.21.15.159:0/478084434 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f40ac199b10 msgr2=0x7f40ac19dea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-09-10T09:28:57.223 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:28:57.219+0000 7f40abfff640 1 -- 172.21.15.159:0/478084434 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f40ac199b10 msgr2=0x7f40ac19dea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-10T09:28:57.223 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:28:57.219+0000 7f40abfff640 1 --2- 172.21.15.159:0/478084434 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f40ac199b10 0x7f40ac19dea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-10T09:28:57.223 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:28:57.219+0000 7f40abfff640 1 --2- 172.21.15.159:0/478084434 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f40ac199b10 0x7f40ac19dea0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T09:28:57.223 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:28:57.219+0000 7f40b17a8640 1 --2- 172.21.15.159:0/478084434 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f40ac19e3e0 0x7f40ac19e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:28:57.223 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:28:57.220+0000 7f40b17a8640 1 -- 172.21.15.159:0/478084434 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f40ac19e3e0 msgr2=0x7f40ac19e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 14 2024-09-10T09:28:57.223 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:28:57.220+0000 7f40b17a8640 1 -- 172.21.15.159:0/478084434 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f40ac19e3e0 msgr2=0x7f40ac19e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-10T09:28:57.223 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:28:57.220+0000 7f40b17a8640 1 --2- 172.21.15.159:0/478084434 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f40ac19e3e0 0x7f40ac19e890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-10T09:28:57.223 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:28:57.220+0000 7f40b17a8640 1 --2- 172.21.15.159:0/478084434 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f40ac19e3e0 0x7f40ac19e890 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T09:28:57.224 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:28:57.220+0000 7f40b0fa7640 1 -- 172.21.15.159:0/478084434 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f40ac199b10 msgr2=0x7f40ac19dea0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T09:28:57.224 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:28:57.220+0000 7f40b0fa7640 1 --2- 172.21.15.159:0/478084434 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f40ac199b10 0x7f40ac19dea0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:28:57.224 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:28:57.220+0000 7f40b0fa7640 1 -- 172.21.15.159:0/478084434 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f40ac19e3e0 msgr2=0x7f40ac19e890 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T09:28:57.224 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:28:57.220+0000 7f40b0fa7640 1 --2- 172.21.15.159:0/478084434 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f40ac19e3e0 0x7f40ac19e890 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:28:57.224 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:28:57.220+0000 7f40b0fa7640 1 -- 172.21.15.159:0/478084434 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40ac1a6150 con 0x7f40ac106630 2024-09-10T09:28:57.224 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:28:57.220+0000 7f40b0fa7640 1 --2- 172.21.15.159:0/478084434 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f40ac106630 0x7f40ac1995d0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f409c013ce0 tx=0x7f409c0052f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:28:57.224 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:28:57.220+0000 7f40a9ffb640 1 -- 172.21.15.159:0/478084434 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f409c032050 con 0x7f40ac106630 2024-09-10T09:28:57.224 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:28:57.220+0000 7f40a9ffb640 1 -- 172.21.15.159:0/478084434 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f409c012900 con 0x7f40ac106630 2024-09-10T09:28:57.224 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:28:57.220+0000 7f40b320b640 1 -- 172.21.15.159:0/478084434 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40ac1a6320 con 0x7f40ac106630 2024-09-10T09:28:57.224 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:28:57.220+0000 7f40a9ffb640 1 -- 172.21.15.159:0/478084434 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f409c013240 con 0x7f40ac106630 2024-09-10T09:28:57.224 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:28:57.221+0000 7f40b320b640 1 -- 172.21.15.159:0/478084434 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40ac1a6830 con 0x7f40ac106630 2024-09-10T09:28:57.225 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:28:57.222+0000 7f40b320b640 1 -- 172.21.15.159:0/478084434 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f40ac108710 con 0x7f40ac106630 2024-09-10T09:28:57.225 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:28:57.223+0000 7f40a9ffb640 1 -- 172.21.15.159:0/478084434 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 13) ==== 46342+0+0 (secure 0 0 0) 0x7f409c004880 con 0x7f40ac106630 2024-09-10T09:28:57.225 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:28:57.223+0000 7f40a9ffb640 1 --2- 172.21.15.159:0/478084434 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f40980394f0 0x7f409803b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:28:57.225 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:28:57.223+0000 7f40a9ffb640 1 -- 172.21.15.159:0/478084434 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f409c06d890 con 0x7f40ac106630 2024-09-10T09:28:57.225 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:28:57.224+0000 7f40abfff640 1 --2- 172.21.15.159:0/478084434 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f40980394f0 0x7f409803b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:28:57.225 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:28:57.224+0000 7f40abfff640 1 --2- 172.21.15.159:0/478084434 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f40980394f0 0x7f409803b9b0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f40a0009100 tx=0x7f40a0008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:28:57.229 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:28:57.228+0000 7f40a9ffb640 1 -- 172.21.15.159:0/478084434 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f409c048e00 con 0x7f40ac106630 2024-09-10T09:28:57.470 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:28:57.469+0000 7f40b320b640 1 -- 172.21.15.159:0/478084434 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f40ac19a930 con 0x7f40ac106630 2024-09-10T09:28:57.470 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:28:57 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T09:28:57.470 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:28:57 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T09:28:57.470 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:28:57 smithi159 ceph-mon[35513]: mon.a calling monitor election 2024-09-10T09:28:57.471 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:28:57 smithi159 ceph-mon[35513]: mon.c calling monitor election 2024-09-10T09:28:57.471 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:28:57 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T09:28:57.471 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:28:57 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T09:28:57.471 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:28:57 smithi159 ceph-mon[35513]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T09:28:57.471 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:28:57 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T09:28:57.471 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:28:57 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T09:28:57.471 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:28:57 smithi159 ceph-mon[35513]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T09:28:57.471 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:28:57 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T09:28:57.471 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:28:57 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T09:28:57.471 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:28:57 smithi159 ceph-mon[35513]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-10T09:28:57.471 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:28:57 smithi159 ceph-mon[35513]: monmap epoch 3 2024-09-10T09:28:57.471 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:28:57 smithi159 ceph-mon[35513]: fsid bade4b50-6f56-11ef-bcea-c7b262605968 2024-09-10T09:28:57.471 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:28:57 smithi159 ceph-mon[35513]: last_changed 2024-09-10T09:28:52.197965+0000 2024-09-10T09:28:57.472 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:28:57 smithi159 ceph-mon[35513]: created 2024-09-10T09:27:08.776915+0000 2024-09-10T09:28:57.472 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:28:57 smithi159 ceph-mon[35513]: min_mon_release 19 (squid) 2024-09-10T09:28:57.472 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:28:57 smithi159 ceph-mon[35513]: election_strategy: 1 2024-09-10T09:28:57.472 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:28:57 smithi159 ceph-mon[35513]: 0: [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] mon.a 2024-09-10T09:28:57.472 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:28:57 smithi159 ceph-mon[35513]: 1: [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] mon.c 2024-09-10T09:28:57.472 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:28:57 smithi159 ceph-mon[35513]: 2: [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] mon.b 2024-09-10T09:28:57.472 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:28:57 smithi159 ceph-mon[35513]: fsmap 2024-09-10T09:28:57.472 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:28:57 smithi159 ceph-mon[35513]: osdmap e4: 0 total, 0 up, 0 in 2024-09-10T09:28:57.472 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:28:57 smithi159 ceph-mon[35513]: mgrmap e13: x(active, since 51s) 2024-09-10T09:28:57.472 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:28:57 smithi159 ceph-mon[35513]: overall HEALTH_OK 2024-09-10T09:28:57.472 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:28:57 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:28:57.472 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:28:57 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:28:57.472 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:28:57 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:28:57.472 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:28:57 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:28:57.473 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:28:57.470+0000 7f40a9ffb640 1 -- 172.21.15.159:0/478084434 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) ==== 95+0+1279 (secure 0 0 0) 0x7f409c04f020 con 0x7f40ac106630 2024-09-10T09:28:57.473 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-10T09:28:57.473 INFO:teuthology.orchestra.run.smithi159.stdout:{"epoch":3,"fsid":"bade4b50-6f56-11ef-bcea-c7b262605968","modified":"2024-09-10T09:28:52.197965Z","created":"2024-09-10T09:27:08.776915Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:3300","nonce":0},{"type":"v1","addr":"172.21.15.5:6789","nonce":0}]},"addr":"172.21.15.5:6789/0","public_addr":"172.21.15.5:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:3300","nonce":0},{"type":"v1","addr":"172.21.15.159:6789","nonce":0}]},"addr":"172.21.15.159:6789/0","public_addr":"172.21.15.159:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:3300","nonce":0},{"type":"v1","addr":"172.21.15.102:6789","nonce":0}]},"addr":"172.21.15.102:6789/0","public_addr":"172.21.15.102:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-10T09:28:57.474 INFO:teuthology.orchestra.run.smithi159.stderr:dumped monmap epoch 3 2024-09-10T09:28:57.474 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:28:57.472+0000 7f40b320b640 1 -- 172.21.15.159:0/478084434 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f40980394f0 msgr2=0x7f409803b9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:28:57.474 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:28:57.472+0000 7f40b320b640 1 --2- 172.21.15.159:0/478084434 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f40980394f0 0x7f409803b9b0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f40a0009100 tx=0x7f40a0008040 comp rx=0 tx=0).stop 2024-09-10T09:28:57.474 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:28:57.472+0000 7f40b320b640 1 -- 172.21.15.159:0/478084434 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f40ac106630 msgr2=0x7f40ac1995d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:28:57.474 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:28:57.472+0000 7f40b320b640 1 --2- 172.21.15.159:0/478084434 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f40ac106630 0x7f40ac1995d0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f409c013ce0 tx=0x7f409c0052f0 comp rx=0 tx=0).stop 2024-09-10T09:28:57.474 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:28:57.472+0000 7f40b320b640 1 -- 172.21.15.159:0/478084434 shutdown_connections 2024-09-10T09:28:57.474 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:28:57.472+0000 7f40b320b640 1 --2- 172.21.15.159:0/478084434 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f40980394f0 0x7f409803b9b0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:28:57.475 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:28:57.472+0000 7f40b320b640 1 --2- 172.21.15.159:0/478084434 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f40ac19e3e0 0x7f40ac19e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:28:57.475 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:28:57.472+0000 7f40b320b640 1 --2- 172.21.15.159:0/478084434 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f40ac199b10 0x7f40ac19dea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:28:57.475 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:28:57.472+0000 7f40b320b640 1 --2- 172.21.15.159:0/478084434 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f40ac106630 0x7f40ac1995d0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:28:57.475 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:28:57.472+0000 7f40b320b640 1 -- 172.21.15.159:0/478084434 >> 172.21.15.159:0/478084434 conn(0x7f40ac0fc220 msgr2=0x7f40ac0fe0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:28:57.475 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:28:57.472+0000 7f40b320b640 1 -- 172.21.15.159:0/478084434 shutdown_connections 2024-09-10T09:28:57.475 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:28:57.472+0000 7f40b320b640 1 -- 172.21.15.159:0/478084434 wait complete. 2024-09-10T09:28:57.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:28:57 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T09:28:57.557 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:28:57 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T09:28:57.557 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:28:57 smithi005 ceph-mon[30350]: mon.a calling monitor election 2024-09-10T09:28:57.557 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:28:57 smithi005 ceph-mon[30350]: mon.c calling monitor election 2024-09-10T09:28:57.557 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:28:57 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T09:28:57.557 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:28:57 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T09:28:57.557 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:28:57 smithi005 ceph-mon[30350]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T09:28:57.557 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:28:57 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T09:28:57.557 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:28:57 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T09:28:57.557 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:28:57 smithi005 ceph-mon[30350]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T09:28:57.557 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:28:57 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T09:28:57.558 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:28:57 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T09:28:57.558 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:28:57 smithi005 ceph-mon[30350]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-10T09:28:57.558 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:28:57 smithi005 ceph-mon[30350]: monmap epoch 3 2024-09-10T09:28:57.558 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:28:57 smithi005 ceph-mon[30350]: fsid bade4b50-6f56-11ef-bcea-c7b262605968 2024-09-10T09:28:57.558 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:28:57 smithi005 ceph-mon[30350]: last_changed 2024-09-10T09:28:52.197965+0000 2024-09-10T09:28:57.558 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:28:57 smithi005 ceph-mon[30350]: created 2024-09-10T09:27:08.776915+0000 2024-09-10T09:28:57.558 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:28:57 smithi005 ceph-mon[30350]: min_mon_release 19 (squid) 2024-09-10T09:28:57.558 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:28:57 smithi005 ceph-mon[30350]: election_strategy: 1 2024-09-10T09:28:57.558 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:28:57 smithi005 ceph-mon[30350]: 0: [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] mon.a 2024-09-10T09:28:57.558 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:28:57 smithi005 ceph-mon[30350]: 1: [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] mon.c 2024-09-10T09:28:57.559 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:28:57 smithi005 ceph-mon[30350]: 2: [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] mon.b 2024-09-10T09:28:57.559 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:28:57 smithi005 ceph-mon[30350]: fsmap 2024-09-10T09:28:57.559 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:28:57 smithi005 ceph-mon[30350]: osdmap e4: 0 total, 0 up, 0 in 2024-09-10T09:28:57.559 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:28:57 smithi005 ceph-mon[30350]: mgrmap e13: x(active, since 51s) 2024-09-10T09:28:57.559 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:28:57 smithi005 ceph-mon[30350]: overall HEALTH_OK 2024-09-10T09:28:57.559 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:28:57 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:28:57.559 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:28:57 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:28:57.559 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:28:57 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:28:57.559 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:28:57 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:28:58.178 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-09-10T09:28:58.178 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph config generate-minimal-conf 2024-09-10T09:28:58.537 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:28:58 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:28:58.537 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:28:58 smithi159 ceph-mon[35513]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T09:28:58.537 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:28:58 smithi159 ceph-mon[35513]: from='client.? 172.21.15.159:0/478084434' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-10T09:28:58.537 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:28:58 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T09:28:58.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:28:58 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:28:58.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:28:58 smithi005 ceph-mon[30350]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T09:28:58.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:28:58 smithi005 ceph-mon[30350]: from='client.? 172.21.15.159:0/478084434' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-10T09:28:58.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:28:58 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T09:28:58.560 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:28:59.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:59.106+0000 7f6b0d600640 1 -- 172.21.15.5:0/103793695 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6b08109770 msgr2=0x7f6b08109b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:28:59.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:59.106+0000 7f6b0d600640 1 --2- 172.21.15.5:0/103793695 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6b08109770 0x7f6b08109b50 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f6af8009880 tx=0x7f6af802a5d0 comp rx=0 tx=0).stop 2024-09-10T09:28:59.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:59.106+0000 7f6b0d600640 1 -- 172.21.15.5:0/103793695 shutdown_connections 2024-09-10T09:28:59.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:59.106+0000 7f6b0d600640 1 --2- 172.21.15.5:0/103793695 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6b08109770 0x7f6b08109b50 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:28:59.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:59.106+0000 7f6b0d600640 1 -- 172.21.15.5:0/103793695 >> 172.21.15.5:0/103793695 conn(0x7f6b080ff450 msgr2=0x7f6b08101870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:28:59.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:59.106+0000 7f6b0d600640 1 -- 172.21.15.5:0/103793695 shutdown_connections 2024-09-10T09:28:59.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:59.106+0000 7f6b0d600640 1 -- 172.21.15.5:0/103793695 wait complete. 2024-09-10T09:28:59.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:59.107+0000 7f6b0d600640 1 Processor -- start 2024-09-10T09:28:59.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:59.108+0000 7f6b0d600640 1 -- start start 2024-09-10T09:28:59.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:59.108+0000 7f6b0d600640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6b08109770 0x7f6b08199630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:28:59.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:59.108+0000 7f6b0d600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6b08199b70 0x7f6b0819df00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:28:59.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:59.108+0000 7f6b0d600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6b0819e440 0x7f6b0819e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:28:59.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:59.108+0000 7f6b0d600640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f6b0810de30 con 0x7f6b0819e440 2024-09-10T09:28:59.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:59.108+0000 7f6b0d600640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f6b0810dcb0 con 0x7f6b08199b70 2024-09-10T09:28:59.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:59.108+0000 7f6b0d600640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f6b0810dfb0 con 0x7f6b08109770 2024-09-10T09:28:59.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:59.108+0000 7f6b06a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6b08109770 0x7f6b08199630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:28:59.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:59.108+0000 7f6b06a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6b08109770 0x7f6b08199630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:45378/0 (socket says 172.21.15.5:45378) 2024-09-10T09:28:59.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:59.108+0000 7f6b07400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6b0819e440 0x7f6b0819e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:28:59.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:59.108+0000 7f6b06a00640 1 -- 172.21.15.5:0/1035334287 learned_addr learned my addr 172.21.15.5:0/1035334287 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:28:59.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:59.108+0000 7f6b06000640 1 --2- 172.21.15.5:0/1035334287 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6b08199b70 0x7f6b0819df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:28:59.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:59.109+0000 7f6b06000640 1 -- 172.21.15.5:0/1035334287 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6b08199b70 msgr2=0x7f6b0819df00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-09-10T09:28:59.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:59.109+0000 7f6b06000640 1 -- 172.21.15.5:0/1035334287 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6b08199b70 msgr2=0x7f6b0819df00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-10T09:28:59.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:59.109+0000 7f6b06000640 1 --2- 172.21.15.5:0/1035334287 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6b08199b70 0x7f6b0819df00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-10T09:28:59.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:59.109+0000 7f6b06000640 1 --2- 172.21.15.5:0/1035334287 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6b08199b70 0x7f6b0819df00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T09:28:59.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:59.109+0000 7f6b07400640 1 -- 172.21.15.5:0/1035334287 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6b08109770 msgr2=0x7f6b08199630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:28:59.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:59.109+0000 7f6b07400640 1 --2- 172.21.15.5:0/1035334287 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6b08109770 0x7f6b08199630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:28:59.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:59.109+0000 7f6b07400640 1 -- 172.21.15.5:0/1035334287 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6b08199b70 msgr2=0x7f6b0819df00 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T09:28:59.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:59.109+0000 7f6b07400640 1 --2- 172.21.15.5:0/1035334287 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6b08199b70 0x7f6b0819df00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:28:59.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:59.109+0000 7f6b07400640 1 -- 172.21.15.5:0/1035334287 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b081a61b0 con 0x7f6b0819e440 2024-09-10T09:28:59.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:59.109+0000 7f6b06a00640 1 --2- 172.21.15.5:0/1035334287 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6b08109770 0x7f6b08199630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:28:59.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:59.109+0000 7f6b07400640 1 --2- 172.21.15.5:0/1035334287 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6b0819e440 0x7f6b0819e8f0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f6b0000bd50 tx=0x7f6b0000be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:28:59.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:59.109+0000 7f6aff400640 1 -- 172.21.15.5:0/1035334287 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b00019070 con 0x7f6b0819e440 2024-09-10T09:28:59.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:59.110+0000 7f6aff400640 1 -- 172.21.15.5:0/1035334287 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6b00014920 con 0x7f6b0819e440 2024-09-10T09:28:59.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:59.110+0000 7f6b0d600640 1 -- 172.21.15.5:0/1035334287 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b081a63e0 con 0x7f6b0819e440 2024-09-10T09:28:59.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:59.110+0000 7f6aff400640 1 -- 172.21.15.5:0/1035334287 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b00012990 con 0x7f6b0819e440 2024-09-10T09:28:59.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:59.110+0000 7f6b0d600640 1 -- 172.21.15.5:0/1035334287 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b081a68f0 con 0x7f6b0819e440 2024-09-10T09:28:59.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:59.111+0000 7f6aff400640 1 -- 172.21.15.5:0/1035334287 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 13) ==== 46342+0+0 (secure 0 0 0) 0x7f6b0001c070 con 0x7f6b0819e440 2024-09-10T09:28:59.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:59.111+0000 7f6b0d600640 1 -- 172.21.15.5:0/1035334287 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6b08109bd0 con 0x7f6b0819e440 2024-09-10T09:28:59.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:59.111+0000 7f6aff400640 1 --2- 172.21.15.5:0/1035334287 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f6aec0394f0 0x7f6aec03b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:28:59.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:59.111+0000 7f6aff400640 1 -- 172.21.15.5:0/1035334287 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f6b0004be40 con 0x7f6b0819e440 2024-09-10T09:28:59.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:59.112+0000 7f6b06a00640 1 --2- 172.21.15.5:0/1035334287 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f6aec0394f0 0x7f6aec03b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:28:59.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:59.112+0000 7f6b06a00640 1 --2- 172.21.15.5:0/1035334287 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f6aec0394f0 0x7f6aec03b9b0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f6af80096c0 tx=0x7f6af802aad0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:28:59.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:59.117+0000 7f6aff400640 1 -- 172.21.15.5:0/1035334287 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f6b00050070 con 0x7f6b0819e440 2024-09-10T09:28:59.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:59.315+0000 7f6b0d600640 1 -- 172.21.15.5:0/1035334287 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7f6b081a6c50 con 0x7f6b0819e440 2024-09-10T09:28:59.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:59.315+0000 7f6aff400640 1 -- 172.21.15.5:0/1035334287 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) ==== 76+0+273 (secure 0 0 0) 0x7f6b00012b90 con 0x7f6b0819e440 2024-09-10T09:28:59.317 INFO:teuthology.orchestra.run.smithi005.stdout:# minimal ceph.conf for bade4b50-6f56-11ef-bcea-c7b262605968 2024-09-10T09:28:59.317 INFO:teuthology.orchestra.run.smithi005.stdout:[global] 2024-09-10T09:28:59.317 INFO:teuthology.orchestra.run.smithi005.stdout: fsid = bade4b50-6f56-11ef-bcea-c7b262605968 2024-09-10T09:28:59.317 INFO:teuthology.orchestra.run.smithi005.stdout: mon_host = [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] 2024-09-10T09:28:59.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:59.318+0000 7f6b0d600640 1 -- 172.21.15.5:0/1035334287 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f6aec0394f0 msgr2=0x7f6aec03b9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:28:59.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:59.318+0000 7f6b0d600640 1 --2- 172.21.15.5:0/1035334287 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f6aec0394f0 0x7f6aec03b9b0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f6af80096c0 tx=0x7f6af802aad0 comp rx=0 tx=0).stop 2024-09-10T09:28:59.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:59.318+0000 7f6b0d600640 1 -- 172.21.15.5:0/1035334287 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6b0819e440 msgr2=0x7f6b0819e8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:28:59.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:59.318+0000 7f6b0d600640 1 --2- 172.21.15.5:0/1035334287 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6b0819e440 0x7f6b0819e8f0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f6b0000bd50 tx=0x7f6b0000be40 comp rx=0 tx=0).stop 2024-09-10T09:28:59.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:59.318+0000 7f6b0d600640 1 -- 172.21.15.5:0/1035334287 shutdown_connections 2024-09-10T09:28:59.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:59.318+0000 7f6b0d600640 1 --2- 172.21.15.5:0/1035334287 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f6aec0394f0 0x7f6aec03b9b0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:28:59.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:59.318+0000 7f6b0d600640 1 --2- 172.21.15.5:0/1035334287 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6b0819e440 0x7f6b0819e8f0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:28:59.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:59.318+0000 7f6b0d600640 1 --2- 172.21.15.5:0/1035334287 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6b08199b70 0x7f6b0819df00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:28:59.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:59.318+0000 7f6b0d600640 1 --2- 172.21.15.5:0/1035334287 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6b08109770 0x7f6b08199630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:28:59.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:59.318+0000 7f6b0d600640 1 -- 172.21.15.5:0/1035334287 >> 172.21.15.5:0/1035334287 conn(0x7f6b080ff450 msgr2=0x7f6b08100ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:28:59.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:59.318+0000 7f6b0d600640 1 -- 172.21.15.5:0/1035334287 shutdown_connections 2024-09-10T09:28:59.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:28:59.319+0000 7f6b0d600640 1 -- 172.21.15.5:0/1035334287 wait complete. 2024-09-10T09:28:59.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:59 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T09:28:59.635 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:59 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T09:28:59.635 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:59 smithi102 ceph-mon[35442]: mon.a calling monitor election 2024-09-10T09:28:59.635 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:59 smithi102 ceph-mon[35442]: mon.c calling monitor election 2024-09-10T09:28:59.635 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:59 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T09:28:59.635 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:59 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T09:28:59.635 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:59 smithi102 ceph-mon[35442]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T09:28:59.635 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:59 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T09:28:59.635 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:59 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T09:28:59.635 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:59 smithi102 ceph-mon[35442]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T09:28:59.635 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:59 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T09:28:59.635 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:59 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T09:28:59.636 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:59 smithi102 ceph-mon[35442]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-10T09:28:59.636 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:59 smithi102 ceph-mon[35442]: monmap epoch 3 2024-09-10T09:28:59.636 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:59 smithi102 ceph-mon[35442]: fsid bade4b50-6f56-11ef-bcea-c7b262605968 2024-09-10T09:28:59.636 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:59 smithi102 ceph-mon[35442]: last_changed 2024-09-10T09:28:52.197965+0000 2024-09-10T09:28:59.636 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:59 smithi102 ceph-mon[35442]: created 2024-09-10T09:27:08.776915+0000 2024-09-10T09:28:59.636 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:59 smithi102 ceph-mon[35442]: min_mon_release 19 (squid) 2024-09-10T09:28:59.636 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:59 smithi102 ceph-mon[35442]: election_strategy: 1 2024-09-10T09:28:59.636 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:59 smithi102 ceph-mon[35442]: 0: [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] mon.a 2024-09-10T09:28:59.636 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:59 smithi102 ceph-mon[35442]: 1: [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] mon.c 2024-09-10T09:28:59.636 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:59 smithi102 ceph-mon[35442]: 2: [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] mon.b 2024-09-10T09:28:59.637 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:59 smithi102 ceph-mon[35442]: fsmap 2024-09-10T09:28:59.637 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:59 smithi102 ceph-mon[35442]: osdmap e4: 0 total, 0 up, 0 in 2024-09-10T09:28:59.637 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:59 smithi102 ceph-mon[35442]: mgrmap e13: x(active, since 51s) 2024-09-10T09:28:59.637 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:59 smithi102 ceph-mon[35442]: overall HEALTH_OK 2024-09-10T09:28:59.637 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:59 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:28:59.637 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:59 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:28:59.637 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:59 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:28:59.637 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:59 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:28:59.637 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:59 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:28:59.637 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:59 smithi102 ceph-mon[35442]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T09:28:59.637 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:59 smithi102 ceph-mon[35442]: from='client.? 172.21.15.159:0/478084434' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-10T09:28:59.638 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:59 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T09:29:00.009 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:28:59 smithi005 ceph-mon[30350]: mon.b calling monitor election 2024-09-10T09:29:00.009 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:28:59 smithi005 ceph-mon[30350]: Updating smithi005:/etc/ceph/ceph.conf 2024-09-10T09:29:00.009 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:28:59 smithi005 ceph-mon[30350]: Updating smithi102:/etc/ceph/ceph.conf 2024-09-10T09:29:00.009 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:28:59 smithi005 ceph-mon[30350]: Updating smithi159:/etc/ceph/ceph.conf 2024-09-10T09:29:00.010 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:28:59 smithi005 ceph-mon[30350]: Updating smithi159:/var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T09:29:00.010 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:28:59 smithi005 ceph-mon[30350]: Updating smithi102:/var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T09:29:00.010 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:28:59 smithi005 ceph-mon[30350]: mon.b calling monitor election 2024-09-10T09:29:00.010 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:28:59 smithi005 ceph-mon[30350]: mon.c calling monitor election 2024-09-10T09:29:00.010 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:28:59 smithi005 ceph-mon[30350]: mon.a calling monitor election 2024-09-10T09:29:00.010 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:28:59 smithi005 ceph-mon[30350]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-09-10T09:29:00.010 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:28:59 smithi005 ceph-mon[30350]: monmap epoch 3 2024-09-10T09:29:00.010 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:28:59 smithi005 ceph-mon[30350]: fsid bade4b50-6f56-11ef-bcea-c7b262605968 2024-09-10T09:29:00.010 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:28:59 smithi005 ceph-mon[30350]: last_changed 2024-09-10T09:28:52.197965+0000 2024-09-10T09:29:00.010 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:28:59 smithi005 ceph-mon[30350]: created 2024-09-10T09:27:08.776915+0000 2024-09-10T09:29:00.010 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:28:59 smithi005 ceph-mon[30350]: min_mon_release 19 (squid) 2024-09-10T09:29:00.010 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:28:59 smithi005 ceph-mon[30350]: election_strategy: 1 2024-09-10T09:29:00.010 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:28:59 smithi005 ceph-mon[30350]: 0: [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] mon.a 2024-09-10T09:29:00.010 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:28:59 smithi005 ceph-mon[30350]: 1: [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] mon.c 2024-09-10T09:29:00.011 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:28:59 smithi005 ceph-mon[30350]: 2: [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] mon.b 2024-09-10T09:29:00.011 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:28:59 smithi005 ceph-mon[30350]: fsmap 2024-09-10T09:29:00.011 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:28:59 smithi005 ceph-mon[30350]: osdmap e4: 0 total, 0 up, 0 in 2024-09-10T09:29:00.011 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:28:59 smithi005 ceph-mon[30350]: mgrmap e13: x(active, since 53s) 2024-09-10T09:29:00.011 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:28:59 smithi005 ceph-mon[30350]: overall HEALTH_OK 2024-09-10T09:29:00.011 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:28:59 smithi005 ceph-mon[30350]: from='client.? 172.21.15.5:0/1035334287' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:29:00.011 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:28:59 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:29:00.011 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:28:59 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:29:00.011 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:28:59 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:29:00.011 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:28:59 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:29:00.011 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:28:59 smithi005 ceph-mon[30350]: Updating smithi005:/var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T09:29:00.011 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:28:59 smithi005 ceph-mon[30350]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T09:29:00.049 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-09-10T09:29:00.050 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-09-10T09:29:00.050 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd of=/etc/ceph/ceph.conf 2024-09-10T09:29:00.111 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-09-10T09:29:00.111 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-10T09:29:00.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:59 smithi102 ceph-mon[35442]: mon.b calling monitor election 2024-09-10T09:29:00.135 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:59 smithi102 ceph-mon[35442]: Updating smithi005:/etc/ceph/ceph.conf 2024-09-10T09:29:00.135 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:59 smithi102 ceph-mon[35442]: Updating smithi102:/etc/ceph/ceph.conf 2024-09-10T09:29:00.135 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:59 smithi102 ceph-mon[35442]: Updating smithi159:/etc/ceph/ceph.conf 2024-09-10T09:29:00.135 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:59 smithi102 ceph-mon[35442]: Updating smithi159:/var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T09:29:00.135 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:59 smithi102 ceph-mon[35442]: Updating smithi102:/var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T09:29:00.135 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:59 smithi102 ceph-mon[35442]: mon.b calling monitor election 2024-09-10T09:29:00.135 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:59 smithi102 ceph-mon[35442]: mon.c calling monitor election 2024-09-10T09:29:00.135 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:59 smithi102 ceph-mon[35442]: mon.a calling monitor election 2024-09-10T09:29:00.135 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:59 smithi102 ceph-mon[35442]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-09-10T09:29:00.135 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:59 smithi102 ceph-mon[35442]: monmap epoch 3 2024-09-10T09:29:00.135 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:59 smithi102 ceph-mon[35442]: fsid bade4b50-6f56-11ef-bcea-c7b262605968 2024-09-10T09:29:00.135 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:59 smithi102 ceph-mon[35442]: last_changed 2024-09-10T09:28:52.197965+0000 2024-09-10T09:29:00.135 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:59 smithi102 ceph-mon[35442]: created 2024-09-10T09:27:08.776915+0000 2024-09-10T09:29:00.135 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:59 smithi102 ceph-mon[35442]: min_mon_release 19 (squid) 2024-09-10T09:29:00.136 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:59 smithi102 ceph-mon[35442]: election_strategy: 1 2024-09-10T09:29:00.136 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:59 smithi102 ceph-mon[35442]: 0: [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] mon.a 2024-09-10T09:29:00.136 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:59 smithi102 ceph-mon[35442]: 1: [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] mon.c 2024-09-10T09:29:00.136 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:59 smithi102 ceph-mon[35442]: 2: [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] mon.b 2024-09-10T09:29:00.136 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:59 smithi102 ceph-mon[35442]: fsmap 2024-09-10T09:29:00.136 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:59 smithi102 ceph-mon[35442]: osdmap e4: 0 total, 0 up, 0 in 2024-09-10T09:29:00.136 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:59 smithi102 ceph-mon[35442]: mgrmap e13: x(active, since 53s) 2024-09-10T09:29:00.136 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:59 smithi102 ceph-mon[35442]: overall HEALTH_OK 2024-09-10T09:29:00.136 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:59 smithi102 ceph-mon[35442]: from='client.? 172.21.15.5:0/1035334287' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:29:00.136 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:59 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:29:00.136 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:59 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:29:00.136 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:59 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:29:00.136 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:59 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:29:00.136 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:59 smithi102 ceph-mon[35442]: Updating smithi005:/var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T09:29:00.136 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:28:59 smithi102 ceph-mon[35442]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T09:29:00.207 DEBUG:teuthology.orchestra.run.smithi102:> set -ex 2024-09-10T09:29:00.207 DEBUG:teuthology.orchestra.run.smithi102:> sudo dd of=/etc/ceph/ceph.conf 2024-09-10T09:29:00.236 DEBUG:teuthology.orchestra.run.smithi102:> set -ex 2024-09-10T09:29:00.236 DEBUG:teuthology.orchestra.run.smithi102:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-10T09:29:00.301 DEBUG:teuthology.orchestra.run.smithi159:> set -ex 2024-09-10T09:29:00.302 DEBUG:teuthology.orchestra.run.smithi159:> sudo dd of=/etc/ceph/ceph.conf 2024-09-10T09:29:00.325 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:28:59 smithi159 ceph-mon[35513]: mon.b calling monitor election 2024-09-10T09:29:00.325 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:28:59 smithi159 ceph-mon[35513]: Updating smithi005:/etc/ceph/ceph.conf 2024-09-10T09:29:00.325 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:28:59 smithi159 ceph-mon[35513]: Updating smithi102:/etc/ceph/ceph.conf 2024-09-10T09:29:00.325 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:28:59 smithi159 ceph-mon[35513]: Updating smithi159:/etc/ceph/ceph.conf 2024-09-10T09:29:00.325 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:28:59 smithi159 ceph-mon[35513]: Updating smithi159:/var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T09:29:00.326 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:28:59 smithi159 ceph-mon[35513]: Updating smithi102:/var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T09:29:00.326 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:28:59 smithi159 ceph-mon[35513]: mon.b calling monitor election 2024-09-10T09:29:00.326 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:28:59 smithi159 ceph-mon[35513]: mon.c calling monitor election 2024-09-10T09:29:00.326 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:28:59 smithi159 ceph-mon[35513]: mon.a calling monitor election 2024-09-10T09:29:00.326 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:28:59 smithi159 ceph-mon[35513]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-09-10T09:29:00.326 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:28:59 smithi159 ceph-mon[35513]: monmap epoch 3 2024-09-10T09:29:00.326 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:28:59 smithi159 ceph-mon[35513]: fsid bade4b50-6f56-11ef-bcea-c7b262605968 2024-09-10T09:29:00.326 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:28:59 smithi159 ceph-mon[35513]: last_changed 2024-09-10T09:28:52.197965+0000 2024-09-10T09:29:00.326 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:28:59 smithi159 ceph-mon[35513]: created 2024-09-10T09:27:08.776915+0000 2024-09-10T09:29:00.326 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:28:59 smithi159 ceph-mon[35513]: min_mon_release 19 (squid) 2024-09-10T09:29:00.326 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:28:59 smithi159 ceph-mon[35513]: election_strategy: 1 2024-09-10T09:29:00.326 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:28:59 smithi159 ceph-mon[35513]: 0: [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] mon.a 2024-09-10T09:29:00.326 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:28:59 smithi159 ceph-mon[35513]: 1: [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] mon.c 2024-09-10T09:29:00.326 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:28:59 smithi159 ceph-mon[35513]: 2: [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] mon.b 2024-09-10T09:29:00.327 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:28:59 smithi159 ceph-mon[35513]: fsmap 2024-09-10T09:29:00.327 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:28:59 smithi159 ceph-mon[35513]: osdmap e4: 0 total, 0 up, 0 in 2024-09-10T09:29:00.327 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:28:59 smithi159 ceph-mon[35513]: mgrmap e13: x(active, since 53s) 2024-09-10T09:29:00.327 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:28:59 smithi159 ceph-mon[35513]: overall HEALTH_OK 2024-09-10T09:29:00.327 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:28:59 smithi159 ceph-mon[35513]: from='client.? 172.21.15.5:0/1035334287' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:29:00.327 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:28:59 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:29:00.327 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:28:59 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:29:00.327 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:28:59 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:29:00.327 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:28:59 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:29:00.327 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:28:59 smithi159 ceph-mon[35513]: Updating smithi005:/var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T09:29:00.327 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:28:59 smithi159 ceph-mon[35513]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T09:29:00.330 DEBUG:teuthology.orchestra.run.smithi159:> set -ex 2024-09-10T09:29:00.330 DEBUG:teuthology.orchestra.run.smithi159:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-10T09:29:00.396 INFO:tasks.cephadm:Adding mgr.x on smithi005 2024-09-10T09:29:00.396 INFO:tasks.cephadm:Adding mgr.y on smithi102 2024-09-10T09:29:00.397 INFO:tasks.cephadm:Adding mgr.z on smithi159 2024-09-10T09:29:00.397 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph orch apply mgr '3;smithi005=x;smithi102=y;smithi159=z' 2024-09-10T09:29:00.675 INFO:teuthology.orchestra.run.smithi159.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.c/config 2024-09-10T09:29:01.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:29:00 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:29:01.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:29:00 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:29:01.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:29:00 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:29:01.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:29:00 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:29:01.057 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:29:00 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:29:01.057 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:29:00 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:29:01.057 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:29:00 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:29:01.057 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:29:00 smithi005 ceph-mon[30350]: Reconfiguring mon.a (unknown last config time)... 2024-09-10T09:29:01.057 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:29:00 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-10T09:29:01.057 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:29:00 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-10T09:29:01.057 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:29:00 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:29:01.057 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:29:00 smithi005 ceph-mon[30350]: Reconfiguring daemon mon.a on smithi005 2024-09-10T09:29:01.057 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:29:00 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T09:29:01.057 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:29:00 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:29:01.098 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:29:00 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:29:01.098 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:29:00 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:29:01.098 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:29:00 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:29:01.098 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:29:00 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:29:01.098 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:29:00 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:29:01.099 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:29:00 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:29:01.099 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:29:00 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:29:01.099 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:29:00 smithi159 ceph-mon[35513]: Reconfiguring mon.a (unknown last config time)... 2024-09-10T09:29:01.099 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:29:00 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-10T09:29:01.099 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:29:00 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-10T09:29:01.099 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:29:00 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:29:01.099 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:29:00 smithi159 ceph-mon[35513]: Reconfiguring daemon mon.a on smithi005 2024-09-10T09:29:01.099 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:29:00 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T09:29:01.099 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:29:00 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:29:01.108 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:29:01.105+0000 7fec43142640 1 -- 172.21.15.159:0/3679505181 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fec3c1016e0 msgr2=0x7fec3c101ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:29:01.108 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:29:01.105+0000 7fec43142640 1 --2- 172.21.15.159:0/3679505181 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fec3c1016e0 0x7fec3c101ae0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fec2c009880 tx=0x7fec2c02a5d0 comp rx=0 tx=0).stop 2024-09-10T09:29:01.108 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:29:01.105+0000 7fec43142640 1 -- 172.21.15.159:0/3679505181 shutdown_connections 2024-09-10T09:29:01.108 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:29:01.105+0000 7fec43142640 1 --2- 172.21.15.159:0/3679505181 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fec3c1016e0 0x7fec3c101ae0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:29:01.108 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:29:01.105+0000 7fec43142640 1 -- 172.21.15.159:0/3679505181 >> 172.21.15.159:0/3679505181 conn(0x7fec3c0fce90 msgr2=0x7fec3c0ff2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:29:01.108 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:29:01.105+0000 7fec43142640 1 -- 172.21.15.159:0/3679505181 shutdown_connections 2024-09-10T09:29:01.109 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:29:01.105+0000 7fec43142640 1 -- 172.21.15.159:0/3679505181 wait complete. 2024-09-10T09:29:01.109 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:29:01.106+0000 7fec43142640 1 Processor -- start 2024-09-10T09:29:01.109 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:29:01.106+0000 7fec43142640 1 -- start start 2024-09-10T09:29:01.109 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:29:01.106+0000 7fec43142640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fec3c1016e0 0x7fec3c196fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:29:01.109 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:29:01.106+0000 7fec43142640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fec3c197510 0x7fec3c19c580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:29:01.109 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:29:01.106+0000 7fec43142640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fec3c197990 0x7fec3c19eb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:29:01.109 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:29:01.106+0000 7fec43142640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fec3c1077d0 con 0x7fec3c197990 2024-09-10T09:29:01.109 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:29:01.106+0000 7fec43142640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fec3c107650 con 0x7fec3c197510 2024-09-10T09:29:01.109 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:29:01.106+0000 7fec43142640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fec3c107950 con 0x7fec3c1016e0 2024-09-10T09:29:01.109 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:29:01.106+0000 7fec40ede640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fec3c1016e0 0x7fec3c196fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:29:01.110 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:29:01.106+0000 7fec40ede640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fec3c1016e0 0x7fec3c196fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:42400/0 (socket says 172.21.15.159:42400) 2024-09-10T09:29:01.110 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:29:01.106+0000 7fec40ede640 1 -- 172.21.15.159:0/2639954351 learned_addr learned my addr 172.21.15.159:0/2639954351 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-09-10T09:29:01.110 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:29:01.106+0000 7fec416df640 1 --2- 172.21.15.159:0/2639954351 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fec3c197990 0x7fec3c19eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:29:01.110 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:29:01.106+0000 7fec3bfff640 1 --2- 172.21.15.159:0/2639954351 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fec3c197510 0x7fec3c19c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:29:01.110 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:29:01.106+0000 7fec40ede640 1 -- 172.21.15.159:0/2639954351 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fec3c197510 msgr2=0x7fec3c19c580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:29:01.110 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:29:01.106+0000 7fec40ede640 1 --2- 172.21.15.159:0/2639954351 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fec3c197510 0x7fec3c19c580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:29:01.110 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:29:01.106+0000 7fec40ede640 1 -- 172.21.15.159:0/2639954351 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fec3c197990 msgr2=0x7fec3c19eb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:29:01.110 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:29:01.106+0000 7fec40ede640 1 --2- 172.21.15.159:0/2639954351 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fec3c197990 0x7fec3c19eb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:29:01.110 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:29:01.106+0000 7fec40ede640 1 -- 172.21.15.159:0/2639954351 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec3c19f3d0 con 0x7fec3c1016e0 2024-09-10T09:29:01.110 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:29:01.106+0000 7fec416df640 1 --2- 172.21.15.159:0/2639954351 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fec3c197990 0x7fec3c19eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:29:01.110 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:29:01.107+0000 7fec3bfff640 1 --2- 172.21.15.159:0/2639954351 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fec3c197510 0x7fec3c19c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T09:29:01.110 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:29:01.107+0000 7fec40ede640 1 --2- 172.21.15.159:0/2639954351 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fec3c1016e0 0x7fec3c196fd0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fec2c031610 tx=0x7fec2c0043c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:29:01.110 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:29:01.107+0000 7fec39ffb640 1 -- 172.21.15.159:0/2639954351 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec2c03f030 con 0x7fec3c1016e0 2024-09-10T09:29:01.111 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:29:01.107+0000 7fec43142640 1 -- 172.21.15.159:0/2639954351 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec3c19f660 con 0x7fec3c1016e0 2024-09-10T09:29:01.111 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:29:01.107+0000 7fec39ffb640 1 -- 172.21.15.159:0/2639954351 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fec2c003ce0 con 0x7fec3c1016e0 2024-09-10T09:29:01.111 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:29:01.107+0000 7fec39ffb640 1 -- 172.21.15.159:0/2639954351 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec2c035070 con 0x7fec3c1016e0 2024-09-10T09:29:01.111 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:29:01.107+0000 7fec43142640 1 -- 172.21.15.159:0/2639954351 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec3c19fc10 con 0x7fec3c1016e0 2024-09-10T09:29:01.112 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:29:01.108+0000 7fec39ffb640 1 -- 172.21.15.159:0/2639954351 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 13) ==== 46342+0+0 (secure 0 0 0) 0x7fec2c042030 con 0x7fec3c1016e0 2024-09-10T09:29:01.112 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:29:01.108+0000 7fec39ffb640 1 --2- 172.21.15.159:0/2639954351 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fec240394f0 0x7fec2403b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:29:01.112 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:29:01.108+0000 7fec43142640 1 -- 172.21.15.159:0/2639954351 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fec3c190590 con 0x7fec3c1016e0 2024-09-10T09:29:01.112 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:29:01.108+0000 7fec39ffb640 1 -- 172.21.15.159:0/2639954351 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fec2c06ca60 con 0x7fec3c1016e0 2024-09-10T09:29:01.112 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:29:01.109+0000 7fec3bfff640 1 --2- 172.21.15.159:0/2639954351 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fec240394f0 0x7fec2403b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:29:01.112 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:29:01.109+0000 7fec3bfff640 1 --2- 172.21.15.159:0/2639954351 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fec240394f0 0x7fec2403b9b0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fec3c1984f0 tx=0x7fec300074e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:29:01.114 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:29:01.113+0000 7fec39ffb640 1 -- 172.21.15.159:0/2639954351 <== mon.1 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fec2c03ac70 con 0x7fec3c1016e0 2024-09-10T09:29:01.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:29:00 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:29:01.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:29:00 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:29:01.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:29:00 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:29:01.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:29:00 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:29:01.135 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:29:00 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:29:01.135 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:29:00 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:29:01.135 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:29:00 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:29:01.135 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:29:00 smithi102 ceph-mon[35442]: Reconfiguring mon.a (unknown last config time)... 2024-09-10T09:29:01.135 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:29:00 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-10T09:29:01.135 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:29:00 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-10T09:29:01.135 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:29:00 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:29:01.135 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:29:00 smithi102 ceph-mon[35442]: Reconfiguring daemon mon.a on smithi005 2024-09-10T09:29:01.135 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:29:00 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T09:29:01.135 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:29:00 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:29:01.290 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:29:01.289+0000 7fec43142640 1 -- 172.21.15.159:0/2639954351 --> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi005=x;smithi102=y;smithi159=z", "target": ["mon-mgr", ""]}) -- 0x7fec3c0626c0 con 0x7fec240394f0 2024-09-10T09:29:01.306 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:29:01.305+0000 7fec39ffb640 1 -- 172.21.15.159:0/2639954351 <== mgr.14150 v2:172.21.15.5:6800/2707348081 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fec3c0626c0 con 0x7fec240394f0 2024-09-10T09:29:01.306 INFO:teuthology.orchestra.run.smithi159.stdout:Scheduled mgr update... 2024-09-10T09:29:01.308 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:29:01.307+0000 7fec43142640 1 -- 172.21.15.159:0/2639954351 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fec240394f0 msgr2=0x7fec2403b9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:29:01.308 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:29:01.307+0000 7fec43142640 1 --2- 172.21.15.159:0/2639954351 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fec240394f0 0x7fec2403b9b0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fec3c1984f0 tx=0x7fec300074e0 comp rx=0 tx=0).stop 2024-09-10T09:29:01.308 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:29:01.307+0000 7fec43142640 1 -- 172.21.15.159:0/2639954351 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fec3c1016e0 msgr2=0x7fec3c196fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:29:01.308 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:29:01.307+0000 7fec43142640 1 --2- 172.21.15.159:0/2639954351 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fec3c1016e0 0x7fec3c196fd0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fec2c031610 tx=0x7fec2c0043c0 comp rx=0 tx=0).stop 2024-09-10T09:29:01.308 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:29:01.307+0000 7fec43142640 1 -- 172.21.15.159:0/2639954351 shutdown_connections 2024-09-10T09:29:01.309 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:29:01.307+0000 7fec43142640 1 --2- 172.21.15.159:0/2639954351 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fec240394f0 0x7fec2403b9b0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:29:01.309 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:29:01.307+0000 7fec43142640 1 --2- 172.21.15.159:0/2639954351 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fec3c197990 0x7fec3c19eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:29:01.309 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:29:01.307+0000 7fec43142640 1 --2- 172.21.15.159:0/2639954351 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fec3c197510 0x7fec3c19c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:29:01.309 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:29:01.307+0000 7fec43142640 1 --2- 172.21.15.159:0/2639954351 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fec3c1016e0 0x7fec3c196fd0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:29:01.309 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:29:01.307+0000 7fec43142640 1 -- 172.21.15.159:0/2639954351 >> 172.21.15.159:0/2639954351 conn(0x7fec3c0fce90 msgr2=0x7fec3c0fd9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:29:01.309 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:29:01.307+0000 7fec43142640 1 -- 172.21.15.159:0/2639954351 shutdown_connections 2024-09-10T09:29:01.309 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:29:01.307+0000 7fec43142640 1 -- 172.21.15.159:0/2639954351 wait complete. 2024-09-10T09:29:01.556 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 10 09:29:01 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mgr-x[30575]: 2024-09-10T09:29:01.198+0000 7f56a7400640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-09-10T09:29:01.870 DEBUG:teuthology.orchestra.run.smithi102:mgr.y> sudo journalctl -f -n 0 -u ceph-bade4b50-6f56-11ef-bcea-c7b262605968@mgr.y.service 2024-09-10T09:29:01.873 DEBUG:teuthology.orchestra.run.smithi159:mgr.z> sudo journalctl -f -n 0 -u ceph-bade4b50-6f56-11ef-bcea-c7b262605968@mgr.z.service 2024-09-10T09:29:01.875 INFO:tasks.cephadm:Deploying OSDs... 2024-09-10T09:29:01.875 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-09-10T09:29:01.876 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/scratch_devs of=/dev/stdout 2024-09-10T09:29:01.911 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-10T09:29:01.911 DEBUG:teuthology.orchestra.run.smithi005:> stat /dev/vg_nvme/lv_1 2024-09-10T09:29:01.987 INFO:teuthology.orchestra.run.smithi005.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-10T09:29:01.987 INFO:teuthology.orchestra.run.smithi005.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T09:29:01.987 INFO:teuthology.orchestra.run.smithi005.stdout:Device: 6h/6d Inode: 631 Links: 1 2024-09-10T09:29:01.987 INFO:teuthology.orchestra.run.smithi005.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T09:29:01.987 INFO:teuthology.orchestra.run.smithi005.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T09:29:01.987 INFO:teuthology.orchestra.run.smithi005.stdout:Access: 2024-09-10 09:28:18.230175789 +0000 2024-09-10T09:29:01.987 INFO:teuthology.orchestra.run.smithi005.stdout:Modify: 2024-09-10 09:23:49.358322543 +0000 2024-09-10T09:29:01.987 INFO:teuthology.orchestra.run.smithi005.stdout:Change: 2024-09-10 09:23:49.358322543 +0000 2024-09-10T09:29:01.988 INFO:teuthology.orchestra.run.smithi005.stdout: Birth: 2024-09-10 09:23:49.358322543 +0000 2024-09-10T09:29:01.988 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-10T09:29:02.086 INFO:teuthology.orchestra.run.smithi005.stderr:1+0 records in 2024-09-10T09:29:02.086 INFO:teuthology.orchestra.run.smithi005.stderr:1+0 records out 2024-09-10T09:29:02.086 INFO:teuthology.orchestra.run.smithi005.stderr:512 bytes copied, 0.000181976 s, 2.8 MB/s 2024-09-10T09:29:02.089 DEBUG:teuthology.orchestra.run.smithi005:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-10T09:29:02.127 DEBUG:teuthology.orchestra.run.smithi005:> stat /dev/vg_nvme/lv_2 2024-09-10T09:29:02.203 INFO:teuthology.orchestra.run.smithi005.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-10T09:29:02.203 INFO:teuthology.orchestra.run.smithi005.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T09:29:02.203 INFO:teuthology.orchestra.run.smithi005.stdout:Device: 6h/6d Inode: 650 Links: 1 2024-09-10T09:29:02.203 INFO:teuthology.orchestra.run.smithi005.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T09:29:02.203 INFO:teuthology.orchestra.run.smithi005.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T09:29:02.203 INFO:teuthology.orchestra.run.smithi005.stdout:Access: 2024-09-10 09:28:18.280174767 +0000 2024-09-10T09:29:02.203 INFO:teuthology.orchestra.run.smithi005.stdout:Modify: 2024-09-10 09:23:49.829312350 +0000 2024-09-10T09:29:02.203 INFO:teuthology.orchestra.run.smithi005.stdout:Change: 2024-09-10 09:23:49.829312350 +0000 2024-09-10T09:29:02.203 INFO:teuthology.orchestra.run.smithi005.stdout: Birth: 2024-09-10 09:23:49.829312350 +0000 2024-09-10T09:29:02.204 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-10T09:29:02.295 INFO:teuthology.orchestra.run.smithi005.stderr:1+0 records in 2024-09-10T09:29:02.296 INFO:teuthology.orchestra.run.smithi005.stderr:1+0 records out 2024-09-10T09:29:02.296 INFO:teuthology.orchestra.run.smithi005.stderr:512 bytes copied, 0.000163946 s, 3.1 MB/s 2024-09-10T09:29:02.298 DEBUG:teuthology.orchestra.run.smithi005:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-10T09:29:02.338 DEBUG:teuthology.orchestra.run.smithi005:> stat /dev/vg_nvme/lv_3 2024-09-10T09:29:02.344 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:29:02 smithi005 ceph-mon[30350]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi005=x;smithi102=y;smithi159=z", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T09:29:02.344 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:29:02 smithi005 ceph-mon[30350]: Saving service mgr spec with placement smithi005=x;smithi102=y;smithi159=z;count:3 2024-09-10T09:29:02.344 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:29:02 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:29:02.344 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:29:02 smithi005 ceph-mon[30350]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T09:29:02.345 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:29:02 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:29:02.345 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:29:02 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:29:02.345 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:29:02 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-10T09:29:02.345 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:29:02 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-10T09:29:02.345 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:29:02 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:29:02.382 INFO:teuthology.orchestra.run.smithi005.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-10T09:29:02.382 INFO:teuthology.orchestra.run.smithi005.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T09:29:02.382 INFO:teuthology.orchestra.run.smithi005.stdout:Device: 6h/6d Inode: 671 Links: 1 2024-09-10T09:29:02.382 INFO:teuthology.orchestra.run.smithi005.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T09:29:02.382 INFO:teuthology.orchestra.run.smithi005.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T09:29:02.382 INFO:teuthology.orchestra.run.smithi005.stdout:Access: 2024-09-10 09:28:18.335173643 +0000 2024-09-10T09:29:02.382 INFO:teuthology.orchestra.run.smithi005.stdout:Modify: 2024-09-10 09:23:50.306302028 +0000 2024-09-10T09:29:02.382 INFO:teuthology.orchestra.run.smithi005.stdout:Change: 2024-09-10 09:23:50.306302028 +0000 2024-09-10T09:29:02.382 INFO:teuthology.orchestra.run.smithi005.stdout: Birth: 2024-09-10 09:23:50.306302028 +0000 2024-09-10T09:29:02.382 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-10T09:29:02.477 INFO:teuthology.orchestra.run.smithi005.stderr:1+0 records in 2024-09-10T09:29:02.478 INFO:teuthology.orchestra.run.smithi005.stderr:1+0 records out 2024-09-10T09:29:02.478 INFO:teuthology.orchestra.run.smithi005.stderr:512 bytes copied, 0.000180998 s, 2.8 MB/s 2024-09-10T09:29:02.480 DEBUG:teuthology.orchestra.run.smithi005:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-10T09:29:02.518 DEBUG:teuthology.orchestra.run.smithi005:> stat /dev/vg_nvme/lv_4 2024-09-10T09:29:02.594 INFO:teuthology.orchestra.run.smithi005.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-10T09:29:02.594 INFO:teuthology.orchestra.run.smithi005.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T09:29:02.594 INFO:teuthology.orchestra.run.smithi005.stdout:Device: 6h/6d Inode: 688 Links: 1 2024-09-10T09:29:02.594 INFO:teuthology.orchestra.run.smithi005.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T09:29:02.594 INFO:teuthology.orchestra.run.smithi005.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T09:29:02.594 INFO:teuthology.orchestra.run.smithi005.stdout:Access: 2024-09-10 09:28:18.385172622 +0000 2024-09-10T09:29:02.594 INFO:teuthology.orchestra.run.smithi005.stdout:Modify: 2024-09-10 09:23:50.797291404 +0000 2024-09-10T09:29:02.594 INFO:teuthology.orchestra.run.smithi005.stdout:Change: 2024-09-10 09:23:50.797291404 +0000 2024-09-10T09:29:02.594 INFO:teuthology.orchestra.run.smithi005.stdout: Birth: 2024-09-10 09:23:50.797291404 +0000 2024-09-10T09:29:02.595 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-10T09:29:02.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:29:02 smithi159 ceph-mon[35513]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi005=x;smithi102=y;smithi159=z", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T09:29:02.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:29:02 smithi159 ceph-mon[35513]: Saving service mgr spec with placement smithi005=x;smithi102=y;smithi159=z;count:3 2024-09-10T09:29:02.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:29:02 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:29:02.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:29:02 smithi159 ceph-mon[35513]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T09:29:02.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:29:02 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:29:02.601 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:29:02 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:29:02.601 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:29:02 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-10T09:29:02.601 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:29:02 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-10T09:29:02.601 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:29:02 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:29:02.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:29:02 smithi102 ceph-mon[35442]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi005=x;smithi102=y;smithi159=z", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T09:29:02.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:29:02 smithi102 ceph-mon[35442]: Saving service mgr spec with placement smithi005=x;smithi102=y;smithi159=z;count:3 2024-09-10T09:29:02.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:29:02 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:29:02.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:29:02 smithi102 ceph-mon[35442]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T09:29:02.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:29:02 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:29:02.635 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:29:02 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:29:02.635 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:29:02 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-10T09:29:02.635 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:29:02 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-10T09:29:02.635 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:29:02 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:29:02.688 INFO:teuthology.orchestra.run.smithi005.stderr:1+0 records in 2024-09-10T09:29:02.688 INFO:teuthology.orchestra.run.smithi005.stderr:1+0 records out 2024-09-10T09:29:02.688 INFO:teuthology.orchestra.run.smithi005.stderr:512 bytes copied, 0.000183804 s, 2.8 MB/s 2024-09-10T09:29:02.691 DEBUG:teuthology.orchestra.run.smithi005:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-10T09:29:02.728 DEBUG:teuthology.orchestra.run.smithi102:> set -ex 2024-09-10T09:29:02.728 DEBUG:teuthology.orchestra.run.smithi102:> dd if=/scratch_devs of=/dev/stdout 2024-09-10T09:29:02.748 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-10T09:29:02.748 DEBUG:teuthology.orchestra.run.smithi102:> stat /dev/vg_nvme/lv_1 2024-09-10T09:29:02.806 INFO:teuthology.orchestra.run.smithi102.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-10T09:29:02.806 INFO:teuthology.orchestra.run.smithi102.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T09:29:02.806 INFO:teuthology.orchestra.run.smithi102.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-09-10T09:29:02.806 INFO:teuthology.orchestra.run.smithi102.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T09:29:02.806 INFO:teuthology.orchestra.run.smithi102.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T09:29:02.806 INFO:teuthology.orchestra.run.smithi102.stdout:Access: 2024-09-10 09:28:51.643296971 +0000 2024-09-10T09:29:02.806 INFO:teuthology.orchestra.run.smithi102.stdout:Modify: 2024-09-10 09:23:13.423729424 +0000 2024-09-10T09:29:02.806 INFO:teuthology.orchestra.run.smithi102.stdout:Change: 2024-09-10 09:23:13.423729424 +0000 2024-09-10T09:29:02.807 INFO:teuthology.orchestra.run.smithi102.stdout: Birth: 2024-09-10 09:23:13.423729424 +0000 2024-09-10T09:29:02.807 DEBUG:teuthology.orchestra.run.smithi102:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-10T09:29:02.873 INFO:teuthology.orchestra.run.smithi102.stderr:1+0 records in 2024-09-10T09:29:02.873 INFO:teuthology.orchestra.run.smithi102.stderr:1+0 records out 2024-09-10T09:29:02.873 INFO:teuthology.orchestra.run.smithi102.stderr:512 bytes copied, 0.00021976 s, 2.3 MB/s 2024-09-10T09:29:02.875 DEBUG:teuthology.orchestra.run.smithi102:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-10T09:29:02.933 DEBUG:teuthology.orchestra.run.smithi102:> stat /dev/vg_nvme/lv_2 2024-09-10T09:29:02.989 INFO:teuthology.orchestra.run.smithi102.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-10T09:29:02.989 INFO:teuthology.orchestra.run.smithi102.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T09:29:02.989 INFO:teuthology.orchestra.run.smithi102.stdout:Device: 5h/5d Inode: 683 Links: 1 2024-09-10T09:29:02.989 INFO:teuthology.orchestra.run.smithi102.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T09:29:02.989 INFO:teuthology.orchestra.run.smithi102.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T09:29:02.990 INFO:teuthology.orchestra.run.smithi102.stdout:Access: 2024-09-10 09:28:51.643296971 +0000 2024-09-10T09:29:02.990 INFO:teuthology.orchestra.run.smithi102.stdout:Modify: 2024-09-10 09:23:13.746722182 +0000 2024-09-10T09:29:02.990 INFO:teuthology.orchestra.run.smithi102.stdout:Change: 2024-09-10 09:23:13.746722182 +0000 2024-09-10T09:29:02.990 INFO:teuthology.orchestra.run.smithi102.stdout: Birth: 2024-09-10 09:23:13.746722182 +0000 2024-09-10T09:29:02.990 DEBUG:teuthology.orchestra.run.smithi102:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-10T09:29:03.054 INFO:teuthology.orchestra.run.smithi102.stderr:1+0 records in 2024-09-10T09:29:03.054 INFO:teuthology.orchestra.run.smithi102.stderr:1+0 records out 2024-09-10T09:29:03.055 INFO:teuthology.orchestra.run.smithi102.stderr:512 bytes copied, 0.000225929 s, 2.3 MB/s 2024-09-10T09:29:03.056 DEBUG:teuthology.orchestra.run.smithi102:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-10T09:29:03.113 DEBUG:teuthology.orchestra.run.smithi102:> stat /dev/vg_nvme/lv_3 2024-09-10T09:29:03.169 INFO:teuthology.orchestra.run.smithi102.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-10T09:29:03.169 INFO:teuthology.orchestra.run.smithi102.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T09:29:03.169 INFO:teuthology.orchestra.run.smithi102.stdout:Device: 5h/5d Inode: 704 Links: 1 2024-09-10T09:29:03.169 INFO:teuthology.orchestra.run.smithi102.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T09:29:03.169 INFO:teuthology.orchestra.run.smithi102.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T09:29:03.169 INFO:teuthology.orchestra.run.smithi102.stdout:Access: 2024-09-10 09:28:51.644296950 +0000 2024-09-10T09:29:03.169 INFO:teuthology.orchestra.run.smithi102.stdout:Modify: 2024-09-10 09:23:14.066715008 +0000 2024-09-10T09:29:03.169 INFO:teuthology.orchestra.run.smithi102.stdout:Change: 2024-09-10 09:23:14.066715008 +0000 2024-09-10T09:29:03.169 INFO:teuthology.orchestra.run.smithi102.stdout: Birth: 2024-09-10 09:23:14.066715008 +0000 2024-09-10T09:29:03.169 DEBUG:teuthology.orchestra.run.smithi102:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-10T09:29:03.232 INFO:teuthology.orchestra.run.smithi102.stderr:1+0 records in 2024-09-10T09:29:03.232 INFO:teuthology.orchestra.run.smithi102.stderr:1+0 records out 2024-09-10T09:29:03.233 INFO:teuthology.orchestra.run.smithi102.stderr:512 bytes copied, 0.000204895 s, 2.5 MB/s 2024-09-10T09:29:03.234 DEBUG:teuthology.orchestra.run.smithi102:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-10T09:29:03.290 DEBUG:teuthology.orchestra.run.smithi102:> stat /dev/vg_nvme/lv_4 2024-09-10T09:29:03.346 INFO:teuthology.orchestra.run.smithi102.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-10T09:29:03.346 INFO:teuthology.orchestra.run.smithi102.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T09:29:03.346 INFO:teuthology.orchestra.run.smithi102.stdout:Device: 5h/5d Inode: 724 Links: 1 2024-09-10T09:29:03.346 INFO:teuthology.orchestra.run.smithi102.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T09:29:03.346 INFO:teuthology.orchestra.run.smithi102.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T09:29:03.346 INFO:teuthology.orchestra.run.smithi102.stdout:Access: 2024-09-10 09:28:51.644296950 +0000 2024-09-10T09:29:03.346 INFO:teuthology.orchestra.run.smithi102.stdout:Modify: 2024-09-10 09:23:14.373708126 +0000 2024-09-10T09:29:03.346 INFO:teuthology.orchestra.run.smithi102.stdout:Change: 2024-09-10 09:23:14.373708126 +0000 2024-09-10T09:29:03.346 INFO:teuthology.orchestra.run.smithi102.stdout: Birth: 2024-09-10 09:23:14.373708126 +0000 2024-09-10T09:29:03.347 DEBUG:teuthology.orchestra.run.smithi102:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-10T09:29:03.409 INFO:teuthology.orchestra.run.smithi102.stderr:1+0 records in 2024-09-10T09:29:03.409 INFO:teuthology.orchestra.run.smithi102.stderr:1+0 records out 2024-09-10T09:29:03.409 INFO:teuthology.orchestra.run.smithi102.stderr:512 bytes copied, 0.000215618 s, 2.4 MB/s 2024-09-10T09:29:03.410 DEBUG:teuthology.orchestra.run.smithi102:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-10T09:29:03.411 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:29:03 smithi005 ceph-mon[30350]: Reconfiguring mgr.x (unknown last config time)... 2024-09-10T09:29:03.411 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:29:03 smithi005 ceph-mon[30350]: Reconfiguring daemon mgr.x on smithi005 2024-09-10T09:29:03.467 DEBUG:teuthology.orchestra.run.smithi159:> set -ex 2024-09-10T09:29:03.467 DEBUG:teuthology.orchestra.run.smithi159:> dd if=/scratch_devs of=/dev/stdout 2024-09-10T09:29:03.486 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-10T09:29:03.486 DEBUG:teuthology.orchestra.run.smithi159:> stat /dev/vg_nvme/lv_1 2024-09-10T09:29:03.545 INFO:teuthology.orchestra.run.smithi159.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-10T09:29:03.546 INFO:teuthology.orchestra.run.smithi159.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T09:29:03.546 INFO:teuthology.orchestra.run.smithi159.stdout:Device: 5h/5d Inode: 664 Links: 1 2024-09-10T09:29:03.546 INFO:teuthology.orchestra.run.smithi159.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T09:29:03.546 INFO:teuthology.orchestra.run.smithi159.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T09:29:03.546 INFO:teuthology.orchestra.run.smithi159.stdout:Access: 2024-09-10 09:28:42.457582333 +0000 2024-09-10T09:29:03.546 INFO:teuthology.orchestra.run.smithi159.stdout:Modify: 2024-09-10 09:23:16.535847448 +0000 2024-09-10T09:29:03.546 INFO:teuthology.orchestra.run.smithi159.stdout:Change: 2024-09-10 09:23:16.535847448 +0000 2024-09-10T09:29:03.546 INFO:teuthology.orchestra.run.smithi159.stdout: Birth: 2024-09-10 09:23:16.535847448 +0000 2024-09-10T09:29:03.547 DEBUG:teuthology.orchestra.run.smithi159:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-10T09:29:03.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:29:03 smithi159 ceph-mon[35513]: Reconfiguring mgr.x (unknown last config time)... 2024-09-10T09:29:03.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:29:03 smithi159 ceph-mon[35513]: Reconfiguring daemon mgr.x on smithi005 2024-09-10T09:29:03.613 INFO:teuthology.orchestra.run.smithi159.stderr:1+0 records in 2024-09-10T09:29:03.614 INFO:teuthology.orchestra.run.smithi159.stderr:1+0 records out 2024-09-10T09:29:03.614 INFO:teuthology.orchestra.run.smithi159.stderr:512 bytes copied, 0.000356809 s, 1.4 MB/s 2024-09-10T09:29:03.615 DEBUG:teuthology.orchestra.run.smithi159:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-10T09:29:03.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:29:03 smithi102 ceph-mon[35442]: Reconfiguring mgr.x (unknown last config time)... 2024-09-10T09:29:03.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:29:03 smithi102 ceph-mon[35442]: Reconfiguring daemon mgr.x on smithi005 2024-09-10T09:29:03.674 DEBUG:teuthology.orchestra.run.smithi159:> stat /dev/vg_nvme/lv_2 2024-09-10T09:29:03.731 INFO:teuthology.orchestra.run.smithi159.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-10T09:29:03.731 INFO:teuthology.orchestra.run.smithi159.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T09:29:03.731 INFO:teuthology.orchestra.run.smithi159.stdout:Device: 5h/5d Inode: 685 Links: 1 2024-09-10T09:29:03.732 INFO:teuthology.orchestra.run.smithi159.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T09:29:03.732 INFO:teuthology.orchestra.run.smithi159.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T09:29:03.732 INFO:teuthology.orchestra.run.smithi159.stdout:Access: 2024-09-10 09:28:42.458582313 +0000 2024-09-10T09:29:03.732 INFO:teuthology.orchestra.run.smithi159.stdout:Modify: 2024-09-10 09:23:16.879840454 +0000 2024-09-10T09:29:03.732 INFO:teuthology.orchestra.run.smithi159.stdout:Change: 2024-09-10 09:23:16.879840454 +0000 2024-09-10T09:29:03.732 INFO:teuthology.orchestra.run.smithi159.stdout: Birth: 2024-09-10 09:23:16.879840454 +0000 2024-09-10T09:29:03.732 DEBUG:teuthology.orchestra.run.smithi159:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-10T09:29:03.798 INFO:teuthology.orchestra.run.smithi159.stderr:1+0 records in 2024-09-10T09:29:03.798 INFO:teuthology.orchestra.run.smithi159.stderr:1+0 records out 2024-09-10T09:29:03.798 INFO:teuthology.orchestra.run.smithi159.stderr:512 bytes copied, 0.000204937 s, 2.5 MB/s 2024-09-10T09:29:03.799 DEBUG:teuthology.orchestra.run.smithi159:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-10T09:29:03.857 DEBUG:teuthology.orchestra.run.smithi159:> stat /dev/vg_nvme/lv_3 2024-09-10T09:29:03.916 INFO:teuthology.orchestra.run.smithi159.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-10T09:29:03.916 INFO:teuthology.orchestra.run.smithi159.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T09:29:03.916 INFO:teuthology.orchestra.run.smithi159.stdout:Device: 5h/5d Inode: 705 Links: 1 2024-09-10T09:29:03.916 INFO:teuthology.orchestra.run.smithi159.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T09:29:03.917 INFO:teuthology.orchestra.run.smithi159.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T09:29:03.917 INFO:teuthology.orchestra.run.smithi159.stdout:Access: 2024-09-10 09:28:42.458582313 +0000 2024-09-10T09:29:03.917 INFO:teuthology.orchestra.run.smithi159.stdout:Modify: 2024-09-10 09:23:17.215833622 +0000 2024-09-10T09:29:03.917 INFO:teuthology.orchestra.run.smithi159.stdout:Change: 2024-09-10 09:23:17.215833622 +0000 2024-09-10T09:29:03.917 INFO:teuthology.orchestra.run.smithi159.stdout: Birth: 2024-09-10 09:23:17.215833622 +0000 2024-09-10T09:29:03.917 DEBUG:teuthology.orchestra.run.smithi159:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-10T09:29:03.982 INFO:teuthology.orchestra.run.smithi159.stderr:1+0 records in 2024-09-10T09:29:03.982 INFO:teuthology.orchestra.run.smithi159.stderr:1+0 records out 2024-09-10T09:29:03.982 INFO:teuthology.orchestra.run.smithi159.stderr:512 bytes copied, 0.000207461 s, 2.5 MB/s 2024-09-10T09:29:03.983 DEBUG:teuthology.orchestra.run.smithi159:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-10T09:29:04.041 DEBUG:teuthology.orchestra.run.smithi159:> stat /dev/vg_nvme/lv_4 2024-09-10T09:29:04.097 INFO:teuthology.orchestra.run.smithi159.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-10T09:29:04.097 INFO:teuthology.orchestra.run.smithi159.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T09:29:04.097 INFO:teuthology.orchestra.run.smithi159.stdout:Device: 5h/5d Inode: 722 Links: 1 2024-09-10T09:29:04.097 INFO:teuthology.orchestra.run.smithi159.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T09:29:04.097 INFO:teuthology.orchestra.run.smithi159.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T09:29:04.097 INFO:teuthology.orchestra.run.smithi159.stdout:Access: 2024-09-10 09:28:42.459582293 +0000 2024-09-10T09:29:04.097 INFO:teuthology.orchestra.run.smithi159.stdout:Modify: 2024-09-10 09:23:17.557826668 +0000 2024-09-10T09:29:04.097 INFO:teuthology.orchestra.run.smithi159.stdout:Change: 2024-09-10 09:23:17.557826668 +0000 2024-09-10T09:29:04.097 INFO:teuthology.orchestra.run.smithi159.stdout: Birth: 2024-09-10 09:23:17.557826668 +0000 2024-09-10T09:29:04.097 DEBUG:teuthology.orchestra.run.smithi159:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-10T09:29:04.161 INFO:teuthology.orchestra.run.smithi159.stderr:1+0 records in 2024-09-10T09:29:04.161 INFO:teuthology.orchestra.run.smithi159.stderr:1+0 records out 2024-09-10T09:29:04.161 INFO:teuthology.orchestra.run.smithi159.stderr:512 bytes copied, 0.000233727 s, 2.2 MB/s 2024-09-10T09:29:04.162 DEBUG:teuthology.orchestra.run.smithi159:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-10T09:29:04.219 INFO:tasks.cephadm:Deploying osd.0 on smithi005 with /dev/vg_nvme/lv_4... 2024-09-10T09:29:04.219 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_4 2024-09-10T09:29:04.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:29:04 smithi005 ceph-mon[30350]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T09:29:04.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:29:04 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:29:04.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:29:04 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:29:04.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:29:04 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-10T09:29:04.557 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:29:04 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-10T09:29:04.557 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:29:04 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:29:04.585 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:29:04 smithi102 ceph-mon[35442]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T09:29:04.585 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:29:04 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:29:04.585 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:29:04 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:29:04.585 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:29:04 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-10T09:29:04.585 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:29:04 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-10T09:29:04.585 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:29:04 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:29:04.587 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:29:04.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:29:04 smithi159 ceph-mon[35513]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T09:29:04.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:29:04 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:29:04.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:29:04 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:29:04.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:29:04 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-10T09:29:04.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:29:04 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-10T09:29:04.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:29:04 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:29:05.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:29:05 smithi159 ceph-mon[35513]: Reconfiguring mon.b (monmap changed)... 2024-09-10T09:29:05.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:29:05 smithi159 ceph-mon[35513]: Reconfiguring daemon mon.b on smithi102 2024-09-10T09:29:05.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:29:05 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:29:05.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:29:05 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:29:05.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:29:05 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-10T09:29:05.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:29:05 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-10T09:29:05.601 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:29:05 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:29:05.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:29:05 smithi102 ceph-mon[35442]: Reconfiguring mon.b (monmap changed)... 2024-09-10T09:29:05.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:29:05 smithi102 ceph-mon[35442]: Reconfiguring daemon mon.b on smithi102 2024-09-10T09:29:05.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:29:05 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:29:05.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:29:05 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:29:05.635 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:29:05 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-10T09:29:05.635 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:29:05 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-10T09:29:05.635 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:29:05 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:29:05.744 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:29:05 smithi005 ceph-mon[30350]: Reconfiguring mon.b (monmap changed)... 2024-09-10T09:29:05.744 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:29:05 smithi005 ceph-mon[30350]: Reconfiguring daemon mon.b on smithi102 2024-09-10T09:29:05.745 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:29:05 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:29:05.745 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:29:05 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:29:05.745 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:29:05 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-10T09:29:05.745 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:29:05 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-10T09:29:05.745 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:29:05 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:29:06.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:29:06 smithi159 ceph-mon[35513]: Reconfiguring mon.c (monmap changed)... 2024-09-10T09:29:06.601 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:29:06 smithi159 ceph-mon[35513]: Reconfiguring daemon mon.c on smithi159 2024-09-10T09:29:06.601 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:29:06 smithi159 ceph-mon[35513]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T09:29:06.601 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:29:06 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:29:06.601 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:29:06 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:29:06.601 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:29:06 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:29:06.601 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:29:06 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:29:06.601 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:29:06 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:29:06.601 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:29:06 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:29:06.601 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:29:06 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-10T09:29:06.601 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:29:06 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-10T09:29:06.601 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:29:06 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-10T09:29:06.601 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:29:06 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:29:06.614 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:29:06 smithi005 ceph-mon[30350]: Reconfiguring mon.c (monmap changed)... 2024-09-10T09:29:06.615 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:29:06 smithi005 ceph-mon[30350]: Reconfiguring daemon mon.c on smithi159 2024-09-10T09:29:06.615 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:29:06 smithi005 ceph-mon[30350]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T09:29:06.615 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:29:06 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:29:06.615 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:29:06 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:29:06.615 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:29:06 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:29:06.615 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:29:06 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:29:06.615 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:29:06 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:29:06.615 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:29:06 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:29:06.615 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:29:06 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-10T09:29:06.615 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:29:06 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-10T09:29:06.615 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:29:06 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-10T09:29:06.615 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:29:06 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:29:06.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:29:06 smithi102 ceph-mon[35442]: Reconfiguring mon.c (monmap changed)... 2024-09-10T09:29:06.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:29:06 smithi102 ceph-mon[35442]: Reconfiguring daemon mon.c on smithi159 2024-09-10T09:29:06.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:29:06 smithi102 ceph-mon[35442]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T09:29:06.635 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:29:06 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:29:06.635 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:29:06 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:29:06.635 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:29:06 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:29:06.635 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:29:06 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:29:06.635 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:29:06 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:29:06.635 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:29:06 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:29:06.635 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:29:06 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-10T09:29:06.635 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:29:06 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-10T09:29:06.635 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:29:06 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-10T09:29:06.635 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:29:06 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:29:07.435 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:29:07 smithi005 ceph-mon[30350]: Deploying daemon mgr.z on smithi159 2024-09-10T09:29:07.442 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-10T09:29:07.475 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi005:vg_nvme/lv_4 2024-09-10T09:29:07.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:29:07 smithi159 ceph-mon[35513]: Deploying daemon mgr.z on smithi159 2024-09-10T09:29:07.601 INFO:journalctl@ceph.mgr.z.smithi159.stdout:Sep 10 09:29:07 smithi159 systemd[1]: Starting Ceph mgr.z for bade4b50-6f56-11ef-bcea-c7b262605968... 2024-09-10T09:29:07.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:29:07 smithi102 ceph-mon[35442]: Deploying daemon mgr.z on smithi159 2024-09-10T09:29:07.844 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:29:08.002 INFO:journalctl@ceph.mgr.z.smithi159.stdout:Sep 10 09:29:07 smithi159 podman[37300]: 2024-09-10 09:29:07.733354557 +0000 UTC m=+0.101577217 container create 0d11d57166b6b33dfe0757c0b933af912dc56640a6ab7006781e8b50d8a1b02b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mgr-z, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-09-10T09:29:08.003 INFO:journalctl@ceph.mgr.z.smithi159.stdout:Sep 10 09:29:07 smithi159 podman[37300]: 2024-09-10 09:29:07.647476236 +0000 UTC m=+0.015698855 image pull 41d67112835771c3efca60c6b8fe297c7ddaecdc26cea26fd28aa18636e49005 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 2024-09-10T09:29:08.003 INFO:journalctl@ceph.mgr.z.smithi159.stdout:Sep 10 09:29:07 smithi159 podman[37300]: 2024-09-10 09:29:07.927349789 +0000 UTC m=+0.295572407 container init 0d11d57166b6b33dfe0757c0b933af912dc56640a6ab7006781e8b50d8a1b02b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mgr-z, ceph=True, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T09:29:08.003 INFO:journalctl@ceph.mgr.z.smithi159.stdout:Sep 10 09:29:07 smithi159 podman[37300]: 2024-09-10 09:29:07.930150364 +0000 UTC m=+0.298372989 container start 0d11d57166b6b33dfe0757c0b933af912dc56640a6ab7006781e8b50d8a1b02b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mgr-z, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-10T09:29:08.003 INFO:journalctl@ceph.mgr.z.smithi159.stdout:Sep 10 09:29:07 smithi159 ceph-mgr[37319]: -- 172.21.15.159:0/618975202 <== mon.1 v2:172.21.15.159:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x561983cf30e0 con 0x561983cd0800 2024-09-10T09:29:08.003 INFO:journalctl@ceph.mgr.z.smithi159.stdout:Sep 10 09:29:08 smithi159 bash[37300]: 0d11d57166b6b33dfe0757c0b933af912dc56640a6ab7006781e8b50d8a1b02b 2024-09-10T09:29:08.259 INFO:journalctl@ceph.mgr.z.smithi159.stdout:Sep 10 09:29:08 smithi159 systemd[1]: Started Ceph mgr.z for bade4b50-6f56-11ef-bcea-c7b262605968. 2024-09-10T09:29:08.259 INFO:journalctl@ceph.mgr.z.smithi159.stdout:Sep 10 09:29:08 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mgr-z[37315]: 2024-09-10T09:29:08.052+0000 7fc8f4725040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-10T09:29:08.259 INFO:journalctl@ceph.mgr.z.smithi159.stdout:Sep 10 09:29:08 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mgr-z[37315]: 2024-09-10T09:29:08.110+0000 7fc8f4725040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-10T09:29:08.259 INFO:journalctl@ceph.mgr.z.smithi159.stdout:Sep 10 09:29:08 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mgr-z[37315]: 2024-09-10T09:29:08.258+0000 7fc8f4725040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-10T09:29:08.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:29:08.312+0000 7f2335a00640 1 -- 172.21.15.5:0/2156094887 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2330103e00 msgr2=0x7f2330104200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:29:08.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:29:08.312+0000 7f2335a00640 1 --2- 172.21.15.5:0/2156094887 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2330103e00 0x7f2330104200 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f2320009900 tx=0x7f232002a3f0 comp rx=0 tx=0).stop 2024-09-10T09:29:08.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:29:08.312+0000 7f2335a00640 1 -- 172.21.15.5:0/2156094887 shutdown_connections 2024-09-10T09:29:08.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:29:08.312+0000 7f2335a00640 1 --2- 172.21.15.5:0/2156094887 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2330105920 0x7f233010c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:29:08.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:29:08.312+0000 7f2335a00640 1 --2- 172.21.15.5:0/2156094887 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2330104f60 0x7f23301053e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:29:08.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:29:08.312+0000 7f2335a00640 1 --2- 172.21.15.5:0/2156094887 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2330103e00 0x7f2330104200 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:29:08.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:29:08.312+0000 7f2335a00640 1 -- 172.21.15.5:0/2156094887 >> 172.21.15.5:0/2156094887 conn(0x7f23300ff5b0 msgr2=0x7f23301019d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:29:08.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:29:08.312+0000 7f2335a00640 1 -- 172.21.15.5:0/2156094887 shutdown_connections 2024-09-10T09:29:08.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:29:08.312+0000 7f2335a00640 1 -- 172.21.15.5:0/2156094887 wait complete. 2024-09-10T09:29:08.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:29:08.313+0000 7f2335a00640 1 Processor -- start 2024-09-10T09:29:08.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:29:08.313+0000 7f2335a00640 1 -- start start 2024-09-10T09:29:08.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:29:08.313+0000 7f2335a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2330103e00 0x7f233019a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:29:08.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:29:08.313+0000 7f2335a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2330104f60 0x7f233019a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:29:08.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:29:08.313+0000 7f2335a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2330105920 0x7f23301a1690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:29:08.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:29:08.313+0000 7f2335a00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f233010fb00 con 0x7f2330105920 2024-09-10T09:29:08.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:29:08.313+0000 7f2335a00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f233010f980 con 0x7f2330103e00 2024-09-10T09:29:08.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:29:08.313+0000 7f2335a00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f233010fc80 con 0x7f2330104f60 2024-09-10T09:29:08.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:29:08.314+0000 7f232ea00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2330103e00 0x7f233019a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:29:08.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:29:08.314+0000 7f2326000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2330104f60 0x7f233019a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:29:08.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:29:08.314+0000 7f232ea00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2330103e00 0x7f233019a0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:34580/0 (socket says 172.21.15.5:34580) 2024-09-10T09:29:08.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:29:08.314+0000 7f232ea00640 1 -- 172.21.15.5:0/1042520641 learned_addr learned my addr 172.21.15.5:0/1042520641 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:29:08.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:29:08.314+0000 7f232f400640 1 --2- 172.21.15.5:0/1042520641 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2330105920 0x7f23301a1690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:29:08.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:29:08.314+0000 7f2326000640 1 -- 172.21.15.5:0/1042520641 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2330103e00 msgr2=0x7f233019a0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:29:08.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:29:08.314+0000 7f2326000640 1 --2- 172.21.15.5:0/1042520641 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2330103e00 0x7f233019a0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:29:08.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:29:08.314+0000 7f2326000640 1 -- 172.21.15.5:0/1042520641 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2330105920 msgr2=0x7f23301a1690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:29:08.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:29:08.314+0000 7f2326000640 1 --2- 172.21.15.5:0/1042520641 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2330105920 0x7f23301a1690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:29:08.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:29:08.314+0000 7f2326000640 1 -- 172.21.15.5:0/1042520641 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23301a1d00 con 0x7f2330104f60 2024-09-10T09:29:08.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:29:08.314+0000 7f232ea00640 1 --2- 172.21.15.5:0/1042520641 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2330103e00 0x7f233019a0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:29:08.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:29:08.314+0000 7f232f400640 1 --2- 172.21.15.5:0/1042520641 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2330105920 0x7f23301a1690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:29:08.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:29:08.314+0000 7f2326000640 1 --2- 172.21.15.5:0/1042520641 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2330104f60 0x7f233019a610 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f231800e9c0 tx=0x7f231800ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:29:08.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:29:08.315+0000 7f2327e00640 1 -- 172.21.15.5:0/1042520641 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f231800ccc0 con 0x7f2330104f60 2024-09-10T09:29:08.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:29:08.315+0000 7f2327e00640 1 -- 172.21.15.5:0/1042520641 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f23180035f0 con 0x7f2330104f60 2024-09-10T09:29:08.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:29:08.315+0000 7f2327e00640 1 -- 172.21.15.5:0/1042520641 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f231800ce60 con 0x7f2330104f60 2024-09-10T09:29:08.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:29:08.315+0000 7f2335a00640 1 -- 172.21.15.5:0/1042520641 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23301a1ff0 con 0x7f2330104f60 2024-09-10T09:29:08.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:29:08.315+0000 7f2335a00640 1 -- 172.21.15.5:0/1042520641 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23301a24a0 con 0x7f2330104f60 2024-09-10T09:29:08.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:29:08.316+0000 7f2327e00640 1 -- 172.21.15.5:0/1042520641 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 13) ==== 46342+0+0 (secure 0 0 0) 0x7f2318004030 con 0x7f2330104f60 2024-09-10T09:29:08.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:29:08.316+0000 7f2335a00640 1 -- 172.21.15.5:0/1042520641 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f233010b120 con 0x7f2330104f60 2024-09-10T09:29:08.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:29:08.316+0000 7f2327e00640 1 --2- 172.21.15.5:0/1042520641 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f23140392c0 0x7f231403b780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:29:08.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:29:08.316+0000 7f2327e00640 1 -- 172.21.15.5:0/1042520641 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f231804c930 con 0x7f2330104f60 2024-09-10T09:29:08.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:29:08.317+0000 7f232ea00640 1 --2- 172.21.15.5:0/1042520641 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f23140392c0 0x7f231403b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:29:08.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:29:08.317+0000 7f232ea00640 1 --2- 172.21.15.5:0/1042520641 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f23140392c0 0x7f231403b780 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f23200096c0 tx=0x7f23200315b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:29:08.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:29:08.322+0000 7f2327e00640 1 -- 172.21.15.5:0/1042520641 <== mon.1 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f2318004320 con 0x7f2330104f60 2024-09-10T09:29:08.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:29:08.519+0000 7f2335a00640 1 -- 172.21.15.5:0/1042520641 --> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi005:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f2330062630 con 0x7f23140392c0 2024-09-10T09:29:08.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:29:08 smithi005 ceph-mon[30350]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T09:29:08.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:29:08 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:29:08.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:29:08 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:29:08.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:29:08 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:29:08.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:29:08 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-10T09:29:08.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:29:08 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-10T09:29:08.557 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:29:08 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-10T09:29:08.557 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:29:08 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:29:08.600 INFO:journalctl@ceph.mgr.z.smithi159.stdout:Sep 10 09:29:08 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mgr-z[37315]: 2024-09-10T09:29:08.325+0000 7fc8f4725040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-10T09:29:08.600 INFO:journalctl@ceph.mgr.z.smithi159.stdout:Sep 10 09:29:08 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mgr-z[37315]: 2024-09-10T09:29:08.473+0000 7fc8f4725040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-10T09:29:08.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:29:08 smithi159 ceph-mon[35513]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T09:29:08.601 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:29:08 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:29:08.601 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:29:08 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:29:08.601 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:29:08 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:29:08.601 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:29:08 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-10T09:29:08.601 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:29:08 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-10T09:29:08.601 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:29:08 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-10T09:29:08.601 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:29:08 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:29:08.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:29:08 smithi102 ceph-mon[35442]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T09:29:08.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:29:08 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:29:08.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:29:08 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:29:08.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:29:08 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:29:08.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:29:08 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-10T09:29:08.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:29:08 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-10T09:29:08.635 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:29:08 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-10T09:29:08.635 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:29:08 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:29:08.925 INFO:journalctl@ceph.mgr.z.smithi159.stdout:Sep 10 09:29:08 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mgr-z[37315]: 2024-09-10T09:29:08.613+0000 7fc8f4725040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-10T09:29:08.925 INFO:journalctl@ceph.mgr.z.smithi159.stdout:Sep 10 09:29:08 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mgr-z[37315]: 2024-09-10T09:29:08.924+0000 7fc8f4725040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-10T09:29:09.332 INFO:journalctl@ceph.mgr.z.smithi159.stdout:Sep 10 09:29:08 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mgr-z[37315]: 2024-09-10T09:29:08.984+0000 7fc8f4725040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-10T09:29:09.332 INFO:journalctl@ceph.mgr.z.smithi159.stdout:Sep 10 09:29:09 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mgr-z[37315]: 2024-09-10T09:29:09.172+0000 7fc8f4725040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-10T09:29:09.433 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:29:09 smithi005 ceph-mon[30350]: Deploying daemon mgr.y on smithi102 2024-09-10T09:29:09.433 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:29:09 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T09:29:09.433 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:29:09 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T09:29:09.434 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:29:09 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:29:09.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:29:09 smithi102 ceph-mon[35442]: Deploying daemon mgr.y on smithi102 2024-09-10T09:29:09.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:29:09 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T09:29:09.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:29:09 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T09:29:09.577 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:29:09 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:29:09.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:29:09 smithi159 ceph-mon[35513]: Deploying daemon mgr.y on smithi102 2024-09-10T09:29:09.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:29:09 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T09:29:09.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:29:09 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T09:29:09.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:29:09 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:29:09.884 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 10 09:29:09 smithi102 systemd[1]: Starting Ceph mgr.y for bade4b50-6f56-11ef-bcea-c7b262605968... 2024-09-10T09:29:10.287 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 10 09:29:09 smithi102 podman[36789]: 2024-09-10 09:29:09.867928751 +0000 UTC m=+0.024699220 image pull 41d67112835771c3efca60c6b8fe297c7ddaecdc26cea26fd28aa18636e49005 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 2024-09-10T09:29:10.287 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 10 09:29:09 smithi102 podman[36789]: 2024-09-10 09:29:09.976110001 +0000 UTC m=+0.132880460 container create b5d70fe88cc5c41b2162b86c1aa35742629352abbe658eaa71fcef1aecb226d0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mgr-y, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS) 2024-09-10T09:29:10.288 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 10 09:29:10 smithi102 podman[36789]: 2024-09-10 09:29:10.186812159 +0000 UTC m=+0.343582619 container init b5d70fe88cc5c41b2162b86c1aa35742629352abbe658eaa71fcef1aecb226d0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mgr-y, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2) 2024-09-10T09:29:10.288 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 10 09:29:10 smithi102 podman[36789]: 2024-09-10 09:29:10.189645686 +0000 UTC m=+0.346416155 container start b5d70fe88cc5c41b2162b86c1aa35742629352abbe658eaa71fcef1aecb226d0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mgr-y, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux ) 2024-09-10T09:29:10.288 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 10 09:29:10 smithi102 ceph-mgr[36808]: -- 172.21.15.102:0/2231282141 <== mon.2 v2:172.21.15.102:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5577164d10e0 con 0x5577164af400 2024-09-10T09:29:10.288 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 10 09:29:10 smithi102 bash[36789]: b5d70fe88cc5c41b2162b86c1aa35742629352abbe658eaa71fcef1aecb226d0 2024-09-10T09:29:10.594 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:29:10 smithi102 ceph-mon[35442]: from='client.24115 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi005:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T09:29:10.594 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:29:10 smithi102 ceph-mon[35442]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T09:29:10.595 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 10 09:29:10 smithi102 systemd[1]: Started Ceph mgr.y for bade4b50-6f56-11ef-bcea-c7b262605968. 2024-09-10T09:29:10.595 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 10 09:29:10 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mgr-y[36804]: 2024-09-10T09:29:10.309+0000 7f412b83e040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-10T09:29:10.595 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 10 09:29:10 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mgr-y[36804]: 2024-09-10T09:29:10.371+0000 7f412b83e040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-10T09:29:10.595 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 10 09:29:10 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mgr-y[36804]: 2024-09-10T09:29:10.523+0000 7f412b83e040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-10T09:29:10.595 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 10 09:29:10 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mgr-y[36804]: 2024-09-10T09:29:10.593+0000 7f412b83e040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-10T09:29:10.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:29:10 smithi159 ceph-mon[35513]: from='client.24115 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi005:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T09:29:10.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:29:10 smithi159 ceph-mon[35513]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T09:29:10.600 INFO:journalctl@ceph.mgr.z.smithi159.stdout:Sep 10 09:29:10 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mgr-z[37315]: 2024-09-10T09:29:10.107+0000 7fc8f4725040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-10T09:29:10.600 INFO:journalctl@ceph.mgr.z.smithi159.stdout:Sep 10 09:29:10 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mgr-z[37315]: 2024-09-10T09:29:10.176+0000 7fc8f4725040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-10T09:29:10.707 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:29:10 smithi005 ceph-mon[30350]: from='client.24115 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi005:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T09:29:10.707 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:29:10 smithi005 ceph-mon[30350]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T09:29:10.884 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 10 09:29:10 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mgr-y[36804]: 2024-09-10T09:29:10.747+0000 7f412b83e040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-10T09:29:11.206 INFO:journalctl@ceph.mgr.z.smithi159.stdout:Sep 10 09:29:10 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mgr-z[37315]: 2024-09-10T09:29:10.934+0000 7fc8f4725040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-10T09:29:11.206 INFO:journalctl@ceph.mgr.z.smithi159.stdout:Sep 10 09:29:11 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mgr-z[37315]: 2024-09-10T09:29:11.051+0000 7fc8f4725040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-10T09:29:11.218 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 10 09:29:10 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mgr-y[36804]: 2024-09-10T09:29:10.908+0000 7f412b83e040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-10T09:29:11.483 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 10 09:29:11 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mgr-y[36804]: 2024-09-10T09:29:11.233+0000 7f412b83e040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-10T09:29:11.483 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 10 09:29:11 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mgr-y[36804]: 2024-09-10T09:29:11.293+0000 7f412b83e040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-10T09:29:11.483 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:29:11 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:29:11.484 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:29:11 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:29:11.484 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:29:11 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:29:11.484 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:29:11 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:29:11.484 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:29:11 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:29:11.484 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:29:11 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:29:11.494 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:29:11 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:29:11.494 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:29:11 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:29:11.494 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:29:11 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:29:11.494 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:29:11 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:29:11.494 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:29:11 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:29:11.494 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:29:11 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:29:11.494 INFO:journalctl@ceph.mgr.z.smithi159.stdout:Sep 10 09:29:11 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mgr-z[37315]: 2024-09-10T09:29:11.204+0000 7fc8f4725040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-10T09:29:11.494 INFO:journalctl@ceph.mgr.z.smithi159.stdout:Sep 10 09:29:11 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mgr-z[37315]: 2024-09-10T09:29:11.266+0000 7fc8f4725040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-10T09:29:11.495 INFO:journalctl@ceph.mgr.z.smithi159.stdout:Sep 10 09:29:11 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mgr-z[37315]: 2024-09-10T09:29:11.425+0000 7fc8f4725040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-10T09:29:11.495 INFO:journalctl@ceph.mgr.z.smithi159.stdout:Sep 10 09:29:11 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mgr-z[37315]: 2024-09-10T09:29:11.493+0000 7fc8f4725040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-10T09:29:11.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:29:11 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:29:11.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:29:11 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:29:11.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:29:11 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:29:11.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:29:11 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:29:11.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:29:11 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:29:11.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:29:11 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:29:11.795 INFO:journalctl@ceph.mgr.z.smithi159.stdout:Sep 10 09:29:11 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mgr-z[37315]: 2024-09-10T09:29:11.554+0000 7fc8f4725040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-10T09:29:11.795 INFO:journalctl@ceph.mgr.z.smithi159.stdout:Sep 10 09:29:11 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mgr-z[37315]: 2024-09-10T09:29:11.723+0000 7fc8f4725040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-10T09:29:11.795 INFO:journalctl@ceph.mgr.z.smithi159.stdout:Sep 10 09:29:11 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mgr-z[37315]: 2024-09-10T09:29:11.793+0000 7fc8f4725040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-10T09:29:11.884 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 10 09:29:11 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mgr-y[36804]: 2024-09-10T09:29:11.482+0000 7f412b83e040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-10T09:29:12.065 INFO:journalctl@ceph.mgr.z.smithi159.stdout:Sep 10 09:29:11 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mgr-z[37315]: 2024-09-10T09:29:11.860+0000 7fc8f4725040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-10T09:29:12.065 INFO:journalctl@ceph.mgr.z.smithi159.stdout:Sep 10 09:29:11 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mgr-z[37315]: 2024-09-10T09:29:11.945+0000 7fc8f4725040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-10T09:29:12.065 INFO:journalctl@ceph.mgr.z.smithi159.stdout:Sep 10 09:29:12 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mgr-z[37315]: 2024-09-10T09:29:12.005+0000 7fc8f4725040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-10T09:29:12.065 INFO:journalctl@ceph.mgr.z.smithi159.stdout:Sep 10 09:29:12 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mgr-z[37315]: 2024-09-10T09:29:12.063+0000 7fc8f4725040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-10T09:29:12.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:29:12 smithi005 ceph-mon[30350]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T09:29:12.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:29:12 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:29:12.557 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:29:12 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:29:12.557 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:29:12 smithi005 ceph-mon[30350]: from='client.? 172.21.15.5:0/2610296064' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "085749a8-e882-4377-92dd-34bb266c0f0b"}]: dispatch 2024-09-10T09:29:12.557 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:29:12 smithi005 ceph-mon[30350]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "085749a8-e882-4377-92dd-34bb266c0f0b"}]: dispatch 2024-09-10T09:29:12.557 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:29:12 smithi005 ceph-mon[30350]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "085749a8-e882-4377-92dd-34bb266c0f0b"}]': finished 2024-09-10T09:29:12.557 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:29:12 smithi005 ceph-mon[30350]: osdmap e5: 1 total, 0 up, 1 in 2024-09-10T09:29:12.557 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:29:12 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T09:29:12.557 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:29:12 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:29:12.557 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:29:12 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:29:12.557 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:29:12 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:29:12.557 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:29:12 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:29:12.557 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:29:12 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:29:12.557 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:29:12 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:29:12.557 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:29:12 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:29:12.558 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:29:12 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:29:12.558 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:29:12 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:29:12.558 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:29:12 smithi005 ceph-mon[30350]: from='client.? 172.21.15.5:0/1960216282' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T09:29:12.634 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 10 09:29:12 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mgr-y[36804]: 2024-09-10T09:29:12.372+0000 7f412b83e040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-10T09:29:12.634 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 10 09:29:12 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mgr-y[36804]: 2024-09-10T09:29:12.437+0000 7f412b83e040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-10T09:29:12.635 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:29:12 smithi102 ceph-mon[35442]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T09:29:12.635 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:29:12 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:29:12.635 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:29:12 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:29:12.635 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:29:12 smithi102 ceph-mon[35442]: from='client.? 172.21.15.5:0/2610296064' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "085749a8-e882-4377-92dd-34bb266c0f0b"}]: dispatch 2024-09-10T09:29:12.635 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:29:12 smithi102 ceph-mon[35442]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "085749a8-e882-4377-92dd-34bb266c0f0b"}]: dispatch 2024-09-10T09:29:12.635 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:29:12 smithi102 ceph-mon[35442]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "085749a8-e882-4377-92dd-34bb266c0f0b"}]': finished 2024-09-10T09:29:12.635 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:29:12 smithi102 ceph-mon[35442]: osdmap e5: 1 total, 0 up, 1 in 2024-09-10T09:29:12.635 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:29:12 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T09:29:12.635 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:29:12 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:29:12.635 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:29:12 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:29:12.635 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:29:12 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:29:12.635 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:29:12 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:29:12.636 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:29:12 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:29:12.636 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:29:12 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:29:12.636 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:29:12 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:29:12.636 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:29:12 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:29:12.636 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:29:12 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:29:12.636 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:29:12 smithi102 ceph-mon[35442]: from='client.? 172.21.15.5:0/1960216282' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T09:29:12.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:29:12 smithi159 ceph-mon[35513]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T09:29:12.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:29:12 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:29:12.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:29:12 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:29:12.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:29:12 smithi159 ceph-mon[35513]: from='client.? 172.21.15.5:0/2610296064' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "085749a8-e882-4377-92dd-34bb266c0f0b"}]: dispatch 2024-09-10T09:29:12.851 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:29:12 smithi159 ceph-mon[35513]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "085749a8-e882-4377-92dd-34bb266c0f0b"}]: dispatch 2024-09-10T09:29:12.851 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:29:12 smithi159 ceph-mon[35513]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "085749a8-e882-4377-92dd-34bb266c0f0b"}]': finished 2024-09-10T09:29:12.851 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:29:12 smithi159 ceph-mon[35513]: osdmap e5: 1 total, 0 up, 1 in 2024-09-10T09:29:12.851 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:29:12 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T09:29:12.851 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:29:12 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:29:12.851 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:29:12 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:29:12.851 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:29:12 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:29:12.851 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:29:12 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:29:12.851 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:29:12 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:29:12.851 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:29:12 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:29:12.851 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:29:12 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:29:12.851 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:29:12 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:29:12.851 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:29:12 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:29:12.851 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:29:12 smithi159 ceph-mon[35513]: from='client.? 172.21.15.5:0/1960216282' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T09:29:13.350 INFO:journalctl@ceph.mgr.z.smithi159.stdout:Sep 10 09:29:12 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mgr-z[37315]: 2024-09-10T09:29:12.848+0000 7fc8f4725040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-10T09:29:13.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:29:13.370+0000 7f2327e00640 1 -- 172.21.15.5:0/1042520641 <== mon.1 v2:172.21.15.159:3300/0 7 ==== mgrmap(e 14) ==== 92059+0+0 (secure 0 0 0) 0x7f2318021910 con 0x7f2330104f60 2024-09-10T09:29:13.396 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 10 09:29:13 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mgr-y[36804]: 2024-09-10T09:29:13.140+0000 7f412b83e040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-10T09:29:13.396 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 10 09:29:13 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mgr-y[36804]: 2024-09-10T09:29:13.253+0000 7f412b83e040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-10T09:29:13.396 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 10 09:29:13 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mgr-y[36804]: 2024-09-10T09:29:13.395+0000 7f412b83e040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-10T09:29:13.397 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:29:13 smithi102 ceph-mon[35442]: Standby manager daemon z started 2024-09-10T09:29:13.397 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:29:13 smithi102 ceph-mon[35442]: from='mgr.? 172.21.15.159:0/771958673' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-10T09:29:13.397 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:29:13 smithi102 ceph-mon[35442]: from='mgr.? 172.21.15.159:0/771958673' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-10T09:29:13.397 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:29:13 smithi102 ceph-mon[35442]: from='mgr.? 172.21.15.159:0/771958673' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-10T09:29:13.397 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:29:13 smithi102 ceph-mon[35442]: from='mgr.? 172.21.15.159:0/771958673' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-10T09:29:13.669 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 10 09:29:13 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mgr-y[36804]: 2024-09-10T09:29:13.455+0000 7f412b83e040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-10T09:29:13.669 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 10 09:29:13 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mgr-y[36804]: 2024-09-10T09:29:13.601+0000 7f412b83e040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-10T09:29:13.669 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 10 09:29:13 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mgr-y[36804]: 2024-09-10T09:29:13.667+0000 7f412b83e040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-10T09:29:13.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:29:13 smithi005 ceph-mon[30350]: Standby manager daemon z started 2024-09-10T09:29:13.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:29:13 smithi005 ceph-mon[30350]: from='mgr.? 172.21.15.159:0/771958673' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-10T09:29:13.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:29:13 smithi005 ceph-mon[30350]: from='mgr.? 172.21.15.159:0/771958673' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-10T09:29:13.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:29:13 smithi005 ceph-mon[30350]: from='mgr.? 172.21.15.159:0/771958673' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-10T09:29:13.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:29:13 smithi005 ceph-mon[30350]: from='mgr.? 172.21.15.159:0/771958673' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-10T09:29:13.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:29:13 smithi159 ceph-mon[35513]: Standby manager daemon z started 2024-09-10T09:29:13.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:29:13 smithi159 ceph-mon[35513]: from='mgr.? 172.21.15.159:0/771958673' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-10T09:29:13.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:29:13 smithi159 ceph-mon[35513]: from='mgr.? 172.21.15.159:0/771958673' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-10T09:29:13.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:29:13 smithi159 ceph-mon[35513]: from='mgr.? 172.21.15.159:0/771958673' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-10T09:29:13.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:29:13 smithi159 ceph-mon[35513]: from='mgr.? 172.21.15.159:0/771958673' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-10T09:29:13.946 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 10 09:29:13 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mgr-y[36804]: 2024-09-10T09:29:13.726+0000 7f412b83e040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-10T09:29:13.946 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 10 09:29:13 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mgr-y[36804]: 2024-09-10T09:29:13.880+0000 7f412b83e040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-10T09:29:13.946 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 10 09:29:13 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mgr-y[36804]: 2024-09-10T09:29:13.944+0000 7f412b83e040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-10T09:29:14.208 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 10 09:29:14 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mgr-y[36804]: 2024-09-10T09:29:14.009+0000 7f412b83e040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-10T09:29:14.208 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 10 09:29:14 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mgr-y[36804]: 2024-09-10T09:29:14.090+0000 7f412b83e040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-10T09:29:14.208 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 10 09:29:14 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mgr-y[36804]: 2024-09-10T09:29:14.148+0000 7f412b83e040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-10T09:29:14.208 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 10 09:29:14 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mgr-y[36804]: 2024-09-10T09:29:14.206+0000 7f412b83e040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-10T09:29:14.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:29:14 smithi102 ceph-mon[35442]: mgrmap e14: x(active, since 67s), standbys: z 2024-09-10T09:29:14.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:29:14 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-10T09:29:14.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:29:14 smithi102 ceph-mon[35442]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T09:29:14.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:29:14 smithi005 ceph-mon[30350]: mgrmap e14: x(active, since 67s), standbys: z 2024-09-10T09:29:14.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:29:14 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-10T09:29:14.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:29:14 smithi005 ceph-mon[30350]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T09:29:14.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:29:14 smithi159 ceph-mon[35513]: mgrmap e14: x(active, since 67s), standbys: z 2024-09-10T09:29:14.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:29:14 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-10T09:29:14.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:29:14 smithi159 ceph-mon[35513]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T09:29:15.374 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 10 09:29:14 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mgr-y[36804]: 2024-09-10T09:29:14.980+0000 7f412b83e040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-10T09:29:15.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:29:15.385+0000 7f2327e00640 1 -- 172.21.15.5:0/1042520641 <== mon.1 v2:172.21.15.159:3300/0 8 ==== mgrmap(e 15) ==== 137776+0+0 (secure 0 0 0) 0x7f23180213f0 con 0x7f2330104f60 2024-09-10T09:29:15.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:29:15 smithi102 ceph-mon[35442]: Standby manager daemon y started 2024-09-10T09:29:15.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:29:15 smithi102 ceph-mon[35442]: from='mgr.? 172.21.15.102:0/1555312305' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-10T09:29:15.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:29:15 smithi102 ceph-mon[35442]: from='mgr.? 172.21.15.102:0/1555312305' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-10T09:29:15.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:29:15 smithi102 ceph-mon[35442]: from='mgr.? 172.21.15.102:0/1555312305' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-10T09:29:15.635 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:29:15 smithi102 ceph-mon[35442]: from='mgr.? 172.21.15.102:0/1555312305' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-10T09:29:15.643 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:29:15 smithi005 ceph-mon[30350]: Standby manager daemon y started 2024-09-10T09:29:15.643 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:29:15 smithi005 ceph-mon[30350]: from='mgr.? 172.21.15.102:0/1555312305' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-10T09:29:15.643 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:29:15 smithi005 ceph-mon[30350]: from='mgr.? 172.21.15.102:0/1555312305' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-10T09:29:15.643 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:29:15 smithi005 ceph-mon[30350]: from='mgr.? 172.21.15.102:0/1555312305' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-10T09:29:15.643 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:29:15 smithi005 ceph-mon[30350]: from='mgr.? 172.21.15.102:0/1555312305' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-10T09:29:15.751 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:29:15 smithi159 ceph-mon[35513]: Standby manager daemon y started 2024-09-10T09:29:15.752 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:29:15 smithi159 ceph-mon[35513]: from='mgr.? 172.21.15.102:0/1555312305' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-10T09:29:15.752 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:29:15 smithi159 ceph-mon[35513]: from='mgr.? 172.21.15.102:0/1555312305' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-10T09:29:15.752 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:29:15 smithi159 ceph-mon[35513]: from='mgr.? 172.21.15.102:0/1555312305' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-10T09:29:15.752 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:29:15 smithi159 ceph-mon[35513]: from='mgr.? 172.21.15.102:0/1555312305' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-10T09:29:16.711 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:29:16 smithi005 ceph-mon[30350]: mgrmap e15: x(active, since 69s), standbys: z, y 2024-09-10T09:29:16.712 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:29:16 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-10T09:29:16.712 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:29:16 smithi005 ceph-mon[30350]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T09:29:16.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:29:16 smithi159 ceph-mon[35513]: mgrmap e15: x(active, since 69s), standbys: z, y 2024-09-10T09:29:16.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:29:16 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-10T09:29:16.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:29:16 smithi159 ceph-mon[35513]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T09:29:16.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:29:16 smithi102 ceph-mon[35442]: mgrmap e15: x(active, since 69s), standbys: z, y 2024-09-10T09:29:16.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:29:16 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-10T09:29:16.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:29:16 smithi102 ceph-mon[35442]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T09:29:17.598 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:29:17 smithi005 ceph-mon[30350]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T09:29:17.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:29:17 smithi159 ceph-mon[35513]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T09:29:17.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:29:17 smithi102 ceph-mon[35442]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T09:29:19.657 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:29:19 smithi005 ceph-mon[30350]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T09:29:19.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:29:19 smithi159 ceph-mon[35513]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T09:29:19.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:29:19 smithi102 ceph-mon[35442]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T09:29:20.551 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:29:20 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-10T09:29:20.551 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:29:20 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:29:20.552 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:29:20 smithi005 ceph-mon[30350]: Deploying daemon osd.0 on smithi005 2024-09-10T09:29:20.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:29:20 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-10T09:29:20.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:29:20 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:29:20.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:29:20 smithi159 ceph-mon[35513]: Deploying daemon osd.0 on smithi005 2024-09-10T09:29:20.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:29:20 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-10T09:29:20.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:29:20 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:29:20.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:29:20 smithi102 ceph-mon[35442]: Deploying daemon osd.0 on smithi005 2024-09-10T09:29:21.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:29:21 smithi005 ceph-mon[30350]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T09:29:21.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:29:21 smithi159 ceph-mon[35513]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T09:29:21.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:29:21 smithi102 ceph-mon[35442]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T09:29:23.760 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:29:23 smithi005 ceph-mon[30350]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T09:29:23.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:29:23 smithi159 ceph-mon[35513]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T09:29:23.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:29:23 smithi102 ceph-mon[35442]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T09:29:25.803 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:29:25 smithi159 ceph-mon[35513]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T09:29:25.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:29:25 smithi005 ceph-mon[30350]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T09:29:25.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:29:25 smithi102 ceph-mon[35442]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T09:29:27.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:29:27 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:29:27.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:29:27 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:29:27.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:29:27 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:29:27.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:29:27 smithi005 ceph-mon[30350]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T09:29:27.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:29:27 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:29:27.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:29:27 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:29:27.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:29:27 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:29:27.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:29:27 smithi159 ceph-mon[35513]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T09:29:27.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:29:27 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:29:27.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:29:27 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:29:27.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:29:27 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:29:27.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:29:27 smithi102 ceph-mon[35442]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T09:29:29.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:29:29 smithi005 ceph-mon[30350]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T09:29:29.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:29:29 smithi159 ceph-mon[35513]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T09:29:29.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:29:29 smithi102 ceph-mon[35442]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T09:29:31.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:29:30 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:29:31.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:29:30 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:29:31.135 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:29:30 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:29:31.135 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:29:30 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:29:31.135 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:29:30 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:29:31.306 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:29:30 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:29:31.306 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:29:30 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:29:31.307 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:29:30 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:29:31.307 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:29:30 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:29:31.307 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:29:30 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:29:31.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:29:30 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:29:31.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:29:30 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:29:31.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:29:30 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:29:31.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:29:30 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:29:31.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:29:30 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:29:32.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:29:31 smithi102 ceph-mon[35442]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T09:29:32.191 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:29:31 smithi005 ceph-mon[30350]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T09:29:32.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:29:32.235+0000 7f2327e00640 1 -- 172.21.15.5:0/1042520641 <== mgr.14150 v2:172.21.15.5:6800/2707348081 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f2330062630 con 0x7f23140392c0 2024-09-10T09:29:32.236 INFO:teuthology.orchestra.run.smithi005.stdout:Created osd(s) 0 on host 'smithi005' 2024-09-10T09:29:32.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:29:32.237+0000 7f2335a00640 1 -- 172.21.15.5:0/1042520641 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f23140392c0 msgr2=0x7f231403b780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:29:32.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:29:32.237+0000 7f2335a00640 1 --2- 172.21.15.5:0/1042520641 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f23140392c0 0x7f231403b780 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f23200096c0 tx=0x7f23200315b0 comp rx=0 tx=0).stop 2024-09-10T09:29:32.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:29:32.237+0000 7f2335a00640 1 -- 172.21.15.5:0/1042520641 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2330104f60 msgr2=0x7f233019a610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:29:32.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:29:32.237+0000 7f2335a00640 1 --2- 172.21.15.5:0/1042520641 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2330104f60 0x7f233019a610 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f231800e9c0 tx=0x7f231800ee80 comp rx=0 tx=0).stop 2024-09-10T09:29:32.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:29:32.237+0000 7f2335a00640 1 -- 172.21.15.5:0/1042520641 shutdown_connections 2024-09-10T09:29:32.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:29:32.237+0000 7f2335a00640 1 --2- 172.21.15.5:0/1042520641 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f23140392c0 0x7f231403b780 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:29:32.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:29:32.237+0000 7f2335a00640 1 --2- 172.21.15.5:0/1042520641 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2330105920 0x7f23301a1690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:29:32.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:29:32.237+0000 7f2335a00640 1 --2- 172.21.15.5:0/1042520641 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2330104f60 0x7f233019a610 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:29:32.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:29:32.237+0000 7f2335a00640 1 --2- 172.21.15.5:0/1042520641 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2330103e00 0x7f233019a0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:29:32.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:29:32.237+0000 7f2335a00640 1 -- 172.21.15.5:0/1042520641 >> 172.21.15.5:0/1042520641 conn(0x7f23300ff5b0 msgr2=0x7f23301082f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:29:32.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:29:32.237+0000 7f2335a00640 1 -- 172.21.15.5:0/1042520641 shutdown_connections 2024-09-10T09:29:32.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:29:32.237+0000 7f2335a00640 1 -- 172.21.15.5:0/1042520641 wait complete. 2024-09-10T09:29:32.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:29:31 smithi159 ceph-mon[35513]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T09:29:32.944 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:29:32 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:29:32.945 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:29:32 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:29:32.945 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:29:32 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:29:32.983 DEBUG:teuthology.orchestra.run.smithi005:osd.0> sudo journalctl -f -n 0 -u ceph-bade4b50-6f56-11ef-bcea-c7b262605968@osd.0.service 2024-09-10T09:29:32.987 INFO:tasks.cephadm:Deploying osd.1 on smithi102 with /dev/vg_nvme/lv_4... 2024-09-10T09:29:32.987 DEBUG:teuthology.orchestra.run.smithi102:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_4 2024-09-10T09:29:33.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:29:32 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:29:33.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:29:32 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:29:33.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:29:32 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:29:33.241 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T09:29:33.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:29:32 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:29:33.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:29:32 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:29:33.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:29:32 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:29:33.806 INFO:journalctl@ceph.osd.0.smithi005.stdout:Sep 10 09:29:33 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-0[44625]: 2024-09-10T09:29:33.393+0000 7f7a2d615640 -1 osd.0 0 log_to_monitors true 2024-09-10T09:29:34.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:29:33 smithi102 ceph-mon[35442]: from='osd.0 [v2:172.21.15.5:6802/954535266,v1:172.21.15.5:6803/954535266]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-10T09:29:34.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:29:33 smithi102 ceph-mon[35442]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T09:29:34.276 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:29:33 smithi005 ceph-mon[30350]: from='osd.0 [v2:172.21.15.5:6802/954535266,v1:172.21.15.5:6803/954535266]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-10T09:29:34.276 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:29:33 smithi005 ceph-mon[30350]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T09:29:34.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:29:33 smithi159 ceph-mon[35513]: from='osd.0 [v2:172.21.15.5:6802/954535266,v1:172.21.15.5:6803/954535266]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-10T09:29:34.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:29:33 smithi159 ceph-mon[35513]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T09:29:35.049 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:29:34 smithi102 ceph-mon[35442]: from='osd.0 [v2:172.21.15.5:6802/954535266,v1:172.21.15.5:6803/954535266]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-10T09:29:35.049 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:29:34 smithi102 ceph-mon[35442]: osdmap e6: 1 total, 0 up, 1 in 2024-09-10T09:29:35.049 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:29:34 smithi102 ceph-mon[35442]: from='osd.0 [v2:172.21.15.5:6802/954535266,v1:172.21.15.5:6803/954535266]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi005", "root=default"]}]: dispatch 2024-09-10T09:29:35.049 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:29:34 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T09:29:35.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:29:34 smithi005 ceph-mon[30350]: from='osd.0 [v2:172.21.15.5:6802/954535266,v1:172.21.15.5:6803/954535266]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-10T09:29:35.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:29:34 smithi005 ceph-mon[30350]: osdmap e6: 1 total, 0 up, 1 in 2024-09-10T09:29:35.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:29:34 smithi005 ceph-mon[30350]: from='osd.0 [v2:172.21.15.5:6802/954535266,v1:172.21.15.5:6803/954535266]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi005", "root=default"]}]: dispatch 2024-09-10T09:29:35.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:29:34 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T09:29:35.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:29:34 smithi159 ceph-mon[35513]: from='osd.0 [v2:172.21.15.5:6802/954535266,v1:172.21.15.5:6803/954535266]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-10T09:29:35.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:29:34 smithi159 ceph-mon[35513]: osdmap e6: 1 total, 0 up, 1 in 2024-09-10T09:29:35.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:29:34 smithi159 ceph-mon[35513]: from='osd.0 [v2:172.21.15.5:6802/954535266,v1:172.21.15.5:6803/954535266]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi005", "root=default"]}]: dispatch 2024-09-10T09:29:35.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:29:34 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T09:29:35.667 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-10T09:29:35.698 DEBUG:teuthology.orchestra.run.smithi102:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi102:vg_nvme/lv_4 2024-09-10T09:29:35.806 INFO:journalctl@ceph.osd.0.smithi005.stdout:Sep 10 09:29:35 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-0[44625]: 2024-09-10T09:29:35.452+0000 7f7a28c00640 -1 osd.0 0 waiting for initial osdmap 2024-09-10T09:29:35.806 INFO:journalctl@ceph.osd.0.smithi005.stdout:Sep 10 09:29:35 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-0[44625]: 2024-09-10T09:29:35.467+0000 7f7a1e200640 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T09:29:35.945 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T09:29:36.071 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:29:35 smithi102 ceph-mon[35442]: from='osd.0 [v2:172.21.15.5:6802/954535266,v1:172.21.15.5:6803/954535266]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi005", "root=default"]}]': finished 2024-09-10T09:29:36.071 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:29:35 smithi102 ceph-mon[35442]: osdmap e7: 1 total, 0 up, 1 in 2024-09-10T09:29:36.071 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:29:35 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T09:29:36.072 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:29:35 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T09:29:36.072 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:29:35 smithi102 ceph-mon[35442]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T09:29:36.072 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:29:35 smithi102 ceph-mon[35442]: from='osd.0 [v2:172.21.15.5:6802/954535266,v1:172.21.15.5:6803/954535266]' entity='osd.0' 2024-09-10T09:29:36.133 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:29:35 smithi005 ceph-mon[30350]: from='osd.0 [v2:172.21.15.5:6802/954535266,v1:172.21.15.5:6803/954535266]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi005", "root=default"]}]': finished 2024-09-10T09:29:36.133 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:29:35 smithi005 ceph-mon[30350]: osdmap e7: 1 total, 0 up, 1 in 2024-09-10T09:29:36.133 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:29:35 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T09:29:36.134 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:29:35 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T09:29:36.134 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:29:35 smithi005 ceph-mon[30350]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T09:29:36.134 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:29:35 smithi005 ceph-mon[30350]: from='osd.0 [v2:172.21.15.5:6802/954535266,v1:172.21.15.5:6803/954535266]' entity='osd.0' 2024-09-10T09:29:36.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:29:35 smithi159 ceph-mon[35513]: from='osd.0 [v2:172.21.15.5:6802/954535266,v1:172.21.15.5:6803/954535266]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi005", "root=default"]}]': finished 2024-09-10T09:29:36.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:29:35 smithi159 ceph-mon[35513]: osdmap e7: 1 total, 0 up, 1 in 2024-09-10T09:29:36.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:29:35 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T09:29:36.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:29:35 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T09:29:36.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:29:35 smithi159 ceph-mon[35513]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T09:29:36.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:29:35 smithi159 ceph-mon[35513]: from='osd.0 [v2:172.21.15.5:6802/954535266,v1:172.21.15.5:6803/954535266]' entity='osd.0' 2024-09-10T09:29:36.457 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-10T09:29:36.454+0000 7f9529392640 1 -- 172.21.15.102:0/1184479606 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9524101c00 msgr2=0x7f9524102000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:29:36.457 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-10T09:29:36.454+0000 7f9529392640 1 --2- 172.21.15.102:0/1184479606 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9524101c00 0x7f9524102000 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f9518009900 tx=0x7f951802a410 comp rx=0 tx=0).stop 2024-09-10T09:29:36.457 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-10T09:29:36.454+0000 7f9529392640 1 -- 172.21.15.102:0/1184479606 shutdown_connections 2024-09-10T09:29:36.457 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-10T09:29:36.454+0000 7f9529392640 1 --2- 172.21.15.102:0/1184479606 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f95241037c0 0x7f952410a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:29:36.457 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-10T09:29:36.454+0000 7f9529392640 1 --2- 172.21.15.102:0/1184479606 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9524102e00 0x7f9524103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:29:36.457 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-10T09:29:36.454+0000 7f9529392640 1 --2- 172.21.15.102:0/1184479606 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9524101c00 0x7f9524102000 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:29:36.458 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-10T09:29:36.454+0000 7f9529392640 1 -- 172.21.15.102:0/1184479606 >> 172.21.15.102:0/1184479606 conn(0x7f95240fd390 msgr2=0x7f95240ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:29:36.458 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-10T09:29:36.454+0000 7f9529392640 1 -- 172.21.15.102:0/1184479606 shutdown_connections 2024-09-10T09:29:36.458 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-10T09:29:36.454+0000 7f9529392640 1 -- 172.21.15.102:0/1184479606 wait complete. 2024-09-10T09:29:36.458 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-10T09:29:36.455+0000 7f9529392640 1 Processor -- start 2024-09-10T09:29:36.458 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-10T09:29:36.455+0000 7f9529392640 1 -- start start 2024-09-10T09:29:36.458 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-10T09:29:36.455+0000 7f9529392640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9524101c00 0x7f9524197f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:29:36.458 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-10T09:29:36.455+0000 7f9529392640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9524102e00 0x7f9524198470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:29:36.458 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-10T09:29:36.455+0000 7f9529392640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f95241037c0 0x7f952419f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:29:36.458 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-10T09:29:36.455+0000 7f9529392640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f952410d8b0 con 0x7f9524101c00 2024-09-10T09:29:36.458 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-10T09:29:36.455+0000 7f9529392640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f952410d730 con 0x7f9524102e00 2024-09-10T09:29:36.458 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-10T09:29:36.455+0000 7f9529392640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f952410da30 con 0x7f95241037c0 2024-09-10T09:29:36.458 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-10T09:29:36.455+0000 7f95227fc640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9524102e00 0x7f9524198470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:29:36.459 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-10T09:29:36.456+0000 7f95227fc640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9524102e00 0x7f9524198470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:47930/0 (socket says 172.21.15.102:47930) 2024-09-10T09:29:36.459 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-10T09:29:36.456+0000 7f95227fc640 1 -- 172.21.15.102:0/1603314518 learned_addr learned my addr 172.21.15.102:0/1603314518 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-10T09:29:36.459 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-10T09:29:36.456+0000 7f95237fe640 1 --2- 172.21.15.102:0/1603314518 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f95241037c0 0x7f952419f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:29:36.459 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-10T09:29:36.456+0000 7f9522ffd640 1 --2- 172.21.15.102:0/1603314518 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9524101c00 0x7f9524197f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:29:36.459 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-10T09:29:36.456+0000 7f95227fc640 1 -- 172.21.15.102:0/1603314518 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f95241037c0 msgr2=0x7f952419f4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:29:36.459 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-10T09:29:36.456+0000 7f95227fc640 1 --2- 172.21.15.102:0/1603314518 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f95241037c0 0x7f952419f4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:29:36.459 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-10T09:29:36.456+0000 7f95227fc640 1 -- 172.21.15.102:0/1603314518 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9524101c00 msgr2=0x7f9524197f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:29:36.459 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-10T09:29:36.456+0000 7f95227fc640 1 --2- 172.21.15.102:0/1603314518 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9524101c00 0x7f9524197f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:29:36.459 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-10T09:29:36.456+0000 7f95227fc640 1 -- 172.21.15.102:0/1603314518 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9524074420 con 0x7f9524102e00 2024-09-10T09:29:36.459 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-10T09:29:36.456+0000 7f95227fc640 1 --2- 172.21.15.102:0/1603314518 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9524102e00 0x7f9524198470 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f951400d950 tx=0x7f951400de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:29:36.460 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-10T09:29:36.458+0000 7f950bfff640 1 -- 172.21.15.102:0/1603314518 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9514017070 con 0x7f9524102e00 2024-09-10T09:29:36.460 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-10T09:29:36.458+0000 7f950bfff640 1 -- 172.21.15.102:0/1603314518 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9514009c30 con 0x7f9524102e00 2024-09-10T09:29:36.460 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-10T09:29:36.458+0000 7f9529392640 1 -- 172.21.15.102:0/1603314518 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9524074710 con 0x7f9524102e00 2024-09-10T09:29:36.460 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-10T09:29:36.458+0000 7f950bfff640 1 -- 172.21.15.102:0/1603314518 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f951400f6b0 con 0x7f9524102e00 2024-09-10T09:29:36.460 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-10T09:29:36.458+0000 7f9529392640 1 -- 172.21.15.102:0/1603314518 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9524074ca0 con 0x7f9524102e00 2024-09-10T09:29:36.460 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-10T09:29:36.459+0000 7f9529392640 1 -- 172.21.15.102:0/1603314518 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f952402d7c0 con 0x7f9524102e00 2024-09-10T09:29:36.461 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-10T09:29:36.460+0000 7f950bfff640 1 -- 172.21.15.102:0/1603314518 <== mon.2 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 15) ==== 137776+0+0 (secure 0 0 0) 0x7f95140109d0 con 0x7f9524102e00 2024-09-10T09:29:36.462 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-10T09:29:36.461+0000 7f950bfff640 1 --2- 172.21.15.102:0/1603314518 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f950c0a2ba0 0x7f950c0a5060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:29:36.462 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-10T09:29:36.461+0000 7f950bfff640 1 -- 172.21.15.102:0/1603314518 <== mon.2 v2:172.21.15.102:3300/0 5 ==== osd_map(8..8 src has 1..8) ==== 1767+0+0 (secure 0 0 0) 0x7f95140cd100 con 0x7f9524102e00 2024-09-10T09:29:36.462 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-10T09:29:36.461+0000 7f9522ffd640 1 --2- 172.21.15.102:0/1603314518 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f950c0a2ba0 0x7f950c0a5060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:29:36.462 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-10T09:29:36.462+0000 7f9522ffd640 1 --2- 172.21.15.102:0/1603314518 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f950c0a2ba0 0x7f950c0a5060 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f951802aa00 tx=0x7f9518031400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:29:36.467 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-10T09:29:36.466+0000 7f950bfff640 1 -- 172.21.15.102:0/1603314518 <== mon.2 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f951409bad0 con 0x7f9524102e00 2024-09-10T09:29:36.648 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-10T09:29:36.647+0000 7f9529392640 1 -- 172.21.15.102:0/1603314518 --> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi102:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f9524108b80 con 0x7f950c0a2ba0 2024-09-10T09:29:37.045 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:29:36 smithi102 ceph-mon[35442]: purged_snaps scrub starts 2024-09-10T09:29:37.045 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:29:36 smithi102 ceph-mon[35442]: purged_snaps scrub ok 2024-09-10T09:29:37.045 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:29:36 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T09:29:37.045 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:29:36 smithi102 ceph-mon[35442]: osd.0 [v2:172.21.15.5:6802/954535266,v1:172.21.15.5:6803/954535266] boot 2024-09-10T09:29:37.045 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:29:36 smithi102 ceph-mon[35442]: osdmap e8: 1 total, 1 up, 1 in 2024-09-10T09:29:37.045 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:29:36 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T09:29:37.045 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:29:36 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:29:37.045 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:29:36 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:29:37.045 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:29:36 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:29:37.045 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:29:36 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:29:37.046 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:29:36 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:29:37.046 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:29:36 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:29:37.046 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:29:36 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:29:37.046 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:29:36 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T09:29:37.046 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:29:36 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T09:29:37.046 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:29:36 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:29:37.306 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:29:36 smithi005 ceph-mon[30350]: purged_snaps scrub starts 2024-09-10T09:29:37.307 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:29:36 smithi005 ceph-mon[30350]: purged_snaps scrub ok 2024-09-10T09:29:37.307 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:29:36 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T09:29:37.307 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:29:36 smithi005 ceph-mon[30350]: osd.0 [v2:172.21.15.5:6802/954535266,v1:172.21.15.5:6803/954535266] boot 2024-09-10T09:29:37.307 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:29:36 smithi005 ceph-mon[30350]: osdmap e8: 1 total, 1 up, 1 in 2024-09-10T09:29:37.307 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:29:36 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T09:29:37.307 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:29:36 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:29:37.307 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:29:36 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:29:37.307 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:29:36 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:29:37.307 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:29:36 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:29:37.307 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:29:36 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:29:37.307 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:29:36 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:29:37.307 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:29:36 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:29:37.308 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:29:36 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T09:29:37.308 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:29:36 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T09:29:37.308 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:29:36 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:29:37.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:29:36 smithi159 ceph-mon[35513]: purged_snaps scrub starts 2024-09-10T09:29:37.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:29:36 smithi159 ceph-mon[35513]: purged_snaps scrub ok 2024-09-10T09:29:37.351 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:29:36 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T09:29:37.351 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:29:36 smithi159 ceph-mon[35513]: osd.0 [v2:172.21.15.5:6802/954535266,v1:172.21.15.5:6803/954535266] boot 2024-09-10T09:29:37.351 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:29:36 smithi159 ceph-mon[35513]: osdmap e8: 1 total, 1 up, 1 in 2024-09-10T09:29:37.351 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:29:36 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T09:29:37.351 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:29:36 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:29:37.351 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:29:36 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:29:37.351 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:29:36 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:29:37.351 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:29:36 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:29:37.351 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:29:36 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:29:37.351 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:29:36 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:29:37.351 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:29:36 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:29:37.351 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:29:36 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T09:29:37.352 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:29:36 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T09:29:37.352 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:29:36 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:29:38.064 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:29:37 smithi102 ceph-mon[35442]: Detected new or changed devices on smithi005 2024-09-10T09:29:38.064 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:29:37 smithi102 ceph-mon[35442]: Adjusting osd_memory_target on smithi005 to 17094M 2024-09-10T09:29:38.064 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:29:37 smithi102 ceph-mon[35442]: from='client.24137 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi102:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T09:29:38.064 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:29:37 smithi102 ceph-mon[35442]: pgmap v44: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-10T09:29:38.306 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:29:37 smithi005 ceph-mon[30350]: Detected new or changed devices on smithi005 2024-09-10T09:29:38.306 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:29:37 smithi005 ceph-mon[30350]: Adjusting osd_memory_target on smithi005 to 17094M 2024-09-10T09:29:38.306 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:29:37 smithi005 ceph-mon[30350]: from='client.24137 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi102:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T09:29:38.307 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:29:37 smithi005 ceph-mon[30350]: pgmap v44: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-10T09:29:38.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:29:37 smithi159 ceph-mon[35513]: Detected new or changed devices on smithi005 2024-09-10T09:29:38.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:29:37 smithi159 ceph-mon[35513]: Adjusting osd_memory_target on smithi005 to 17094M 2024-09-10T09:29:38.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:29:37 smithi159 ceph-mon[35513]: from='client.24137 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi102:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T09:29:38.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:29:37 smithi159 ceph-mon[35513]: pgmap v44: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-10T09:29:39.180 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:29:38 smithi102 ceph-mon[35442]: osdmap e9: 1 total, 1 up, 1 in 2024-09-10T09:29:39.180 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:29:38 smithi102 ceph-mon[35442]: from='client.? 172.21.15.102:0/511544952' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0a76c795-2573-4a92-969a-2ec5dd65c094"}]: dispatch 2024-09-10T09:29:39.180 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:29:38 smithi102 ceph-mon[35442]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0a76c795-2573-4a92-969a-2ec5dd65c094"}]: dispatch 2024-09-10T09:29:39.180 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:29:38 smithi102 ceph-mon[35442]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0a76c795-2573-4a92-969a-2ec5dd65c094"}]': finished 2024-09-10T09:29:39.180 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:29:38 smithi102 ceph-mon[35442]: osdmap e10: 2 total, 1 up, 2 in 2024-09-10T09:29:39.181 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:29:38 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T09:29:39.306 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:29:38 smithi005 ceph-mon[30350]: osdmap e9: 1 total, 1 up, 1 in 2024-09-10T09:29:39.306 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:29:38 smithi005 ceph-mon[30350]: from='client.? 172.21.15.102:0/511544952' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0a76c795-2573-4a92-969a-2ec5dd65c094"}]: dispatch 2024-09-10T09:29:39.306 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:29:38 smithi005 ceph-mon[30350]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0a76c795-2573-4a92-969a-2ec5dd65c094"}]: dispatch 2024-09-10T09:29:39.306 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:29:38 smithi005 ceph-mon[30350]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0a76c795-2573-4a92-969a-2ec5dd65c094"}]': finished 2024-09-10T09:29:39.307 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:29:38 smithi005 ceph-mon[30350]: osdmap e10: 2 total, 1 up, 2 in 2024-09-10T09:29:39.307 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:29:38 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T09:29:39.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:29:38 smithi159 ceph-mon[35513]: osdmap e9: 1 total, 1 up, 1 in 2024-09-10T09:29:39.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:29:38 smithi159 ceph-mon[35513]: from='client.? 172.21.15.102:0/511544952' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0a76c795-2573-4a92-969a-2ec5dd65c094"}]: dispatch 2024-09-10T09:29:39.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:29:38 smithi159 ceph-mon[35513]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0a76c795-2573-4a92-969a-2ec5dd65c094"}]: dispatch 2024-09-10T09:29:39.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:29:38 smithi159 ceph-mon[35513]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0a76c795-2573-4a92-969a-2ec5dd65c094"}]': finished 2024-09-10T09:29:39.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:29:38 smithi159 ceph-mon[35513]: osdmap e10: 2 total, 1 up, 2 in 2024-09-10T09:29:39.351 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:29:38 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T09:29:40.179 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:29:39 smithi005 ceph-mon[30350]: from='client.? 172.21.15.102:0/3360706615' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T09:29:40.179 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:29:39 smithi005 ceph-mon[30350]: pgmap v47: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-10T09:29:40.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:29:39 smithi159 ceph-mon[35513]: from='client.? 172.21.15.102:0/3360706615' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T09:29:40.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:29:39 smithi159 ceph-mon[35513]: pgmap v47: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-10T09:29:40.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:29:39 smithi102 ceph-mon[35442]: from='client.? 172.21.15.102:0/3360706615' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T09:29:40.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:29:39 smithi102 ceph-mon[35442]: pgmap v47: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-10T09:29:41.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:29:41 smithi005 ceph-mon[30350]: pgmap v48: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-10T09:29:41.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:29:41 smithi159 ceph-mon[35513]: pgmap v48: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-10T09:29:41.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:29:41 smithi102 ceph-mon[35442]: pgmap v48: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-10T09:29:43.724 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:29:43 smithi102 ceph-mon[35442]: pgmap v49: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-10T09:29:43.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:29:43 smithi005 ceph-mon[30350]: pgmap v49: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-10T09:29:43.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:29:43 smithi159 ceph-mon[35513]: pgmap v49: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-10T09:29:45.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:29:45 smithi005 ceph-mon[30350]: pgmap v50: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-10T09:29:45.831 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:29:45 smithi102 ceph-mon[35442]: pgmap v50: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-10T09:29:45.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:29:45 smithi159 ceph-mon[35513]: pgmap v50: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-10T09:29:47.306 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:29:47 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-10T09:29:47.306 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:29:47 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:29:47.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:29:47 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-10T09:29:47.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:29:47 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:29:47.376 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:29:47 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-10T09:29:47.376 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:29:47 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:29:48.297 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:29:48 smithi102 ceph-mon[35442]: Deploying daemon osd.1 on smithi102 2024-09-10T09:29:48.297 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:29:48 smithi102 ceph-mon[35442]: pgmap v51: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-10T09:29:48.306 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:29:48 smithi005 ceph-mon[30350]: Deploying daemon osd.1 on smithi102 2024-09-10T09:29:48.306 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:29:48 smithi005 ceph-mon[30350]: pgmap v51: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-10T09:29:48.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:29:48 smithi159 ceph-mon[35513]: Deploying daemon osd.1 on smithi102 2024-09-10T09:29:48.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:29:48 smithi159 ceph-mon[35513]: pgmap v51: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-10T09:29:49.773 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:29:49 smithi102 ceph-mon[35442]: pgmap v52: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-10T09:29:49.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:29:49 smithi005 ceph-mon[30350]: pgmap v52: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-10T09:29:49.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:29:49 smithi159 ceph-mon[35513]: pgmap v52: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-10T09:29:51.746 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:29:51 smithi102 ceph-mon[35442]: pgmap v53: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-10T09:29:51.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:29:51 smithi005 ceph-mon[30350]: pgmap v53: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-10T09:29:51.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:29:51 smithi159 ceph-mon[35513]: pgmap v53: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-10T09:29:53.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:29:53 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:29:53.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:29:53 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:29:53.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:29:53 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:29:53.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:29:53 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:29:53.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:29:53 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:29:53.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:29:53 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:29:53.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:29:53 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:29:53.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:29:53 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:29:53.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:29:53 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:29:54.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:29:54 smithi005 ceph-mon[30350]: pgmap v54: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-10T09:29:54.598 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:29:54 smithi102 ceph-mon[35442]: pgmap v54: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-10T09:29:54.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:29:54 smithi159 ceph-mon[35513]: pgmap v54: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-10T09:29:56.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:29:56 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:29:56.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:29:56 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:29:56.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:29:56 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:29:56.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:29:56 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:29:56.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:29:56 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:29:56.385 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:29:56 smithi102 ceph-mon[35442]: pgmap v55: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-10T09:29:56.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:29:56 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:29:56.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:29:56 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:29:56.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:29:56 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:29:56.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:29:56 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:29:56.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:29:56 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:29:56.557 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:29:56 smithi005 ceph-mon[30350]: pgmap v55: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-10T09:29:56.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:29:56 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:29:56.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:29:56 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:29:56.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:29:56 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:29:56.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:29:56 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:29:56.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:29:56 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:29:56.601 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:29:56 smithi159 ceph-mon[35513]: pgmap v55: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-10T09:29:57.143 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-10T09:29:57.142+0000 7f950bfff640 1 -- 172.21.15.102:0/1603314518 <== mgr.14150 v2:172.21.15.5:6800/2707348081 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f9524108b80 con 0x7f950c0a2ba0 2024-09-10T09:29:57.143 INFO:teuthology.orchestra.run.smithi102.stdout:Created osd(s) 1 on host 'smithi102' 2024-09-10T09:29:57.145 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-10T09:29:57.144+0000 7f9529392640 1 -- 172.21.15.102:0/1603314518 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f950c0a2ba0 msgr2=0x7f950c0a5060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:29:57.145 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-10T09:29:57.144+0000 7f9529392640 1 --2- 172.21.15.102:0/1603314518 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f950c0a2ba0 0x7f950c0a5060 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f951802aa00 tx=0x7f9518031400 comp rx=0 tx=0).stop 2024-09-10T09:29:57.146 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-10T09:29:57.144+0000 7f9529392640 1 -- 172.21.15.102:0/1603314518 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9524102e00 msgr2=0x7f9524198470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:29:57.146 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-10T09:29:57.144+0000 7f9529392640 1 --2- 172.21.15.102:0/1603314518 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9524102e00 0x7f9524198470 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f951400d950 tx=0x7f951400de10 comp rx=0 tx=0).stop 2024-09-10T09:29:57.146 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-10T09:29:57.144+0000 7f9529392640 1 -- 172.21.15.102:0/1603314518 shutdown_connections 2024-09-10T09:29:57.146 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-10T09:29:57.144+0000 7f9529392640 1 --2- 172.21.15.102:0/1603314518 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f950c0a2ba0 0x7f950c0a5060 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:29:57.146 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-10T09:29:57.144+0000 7f9529392640 1 --2- 172.21.15.102:0/1603314518 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f95241037c0 0x7f952419f4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:29:57.146 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-10T09:29:57.144+0000 7f9529392640 1 --2- 172.21.15.102:0/1603314518 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9524102e00 0x7f9524198470 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:29:57.146 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-10T09:29:57.144+0000 7f9529392640 1 --2- 172.21.15.102:0/1603314518 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9524101c00 0x7f9524197f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:29:57.146 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-10T09:29:57.144+0000 7f9529392640 1 -- 172.21.15.102:0/1603314518 >> 172.21.15.102:0/1603314518 conn(0x7f95240fd390 msgr2=0x7f95241061a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:29:57.146 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-10T09:29:57.144+0000 7f9529392640 1 -- 172.21.15.102:0/1603314518 shutdown_connections 2024-09-10T09:29:57.146 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-10T09:29:57.144+0000 7f9529392640 1 -- 172.21.15.102:0/1603314518 wait complete. 2024-09-10T09:29:57.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:29:57 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:29:57.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:29:57 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:29:57.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:29:57 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:29:57.753 DEBUG:teuthology.orchestra.run.smithi102:osd.1> sudo journalctl -f -n 0 -u ceph-bade4b50-6f56-11ef-bcea-c7b262605968@osd.1.service 2024-09-10T09:29:57.756 INFO:tasks.cephadm:Deploying osd.2 on smithi159 with /dev/vg_nvme/lv_4... 2024-09-10T09:29:57.756 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_4 2024-09-10T09:29:58.011 INFO:teuthology.orchestra.run.smithi159.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.c/config 2024-09-10T09:29:58.346 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:29:58 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:29:58.346 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:29:58 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:29:58.346 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:29:58 smithi102 ceph-mon[35442]: pgmap v56: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-10T09:29:58.413 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:29:58 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:29:58.414 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:29:58 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:29:58.414 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:29:58 smithi159 ceph-mon[35513]: pgmap v56: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-10T09:29:58.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:29:58 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:29:58.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:29:58 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:29:58.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:29:58 smithi005 ceph-mon[30350]: pgmap v56: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-10T09:29:59.545 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:29:59 smithi159 ceph-mon[35513]: from='osd.1 [v2:172.21.15.102:6800/733924375,v1:172.21.15.102:6801/733924375]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-10T09:29:59.545 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:29:59 smithi159 ceph-mon[35513]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-10T09:29:59.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:29:59 smithi005 ceph-mon[30350]: from='osd.1 [v2:172.21.15.102:6800/733924375,v1:172.21.15.102:6801/733924375]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-10T09:29:59.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:29:59 smithi005 ceph-mon[30350]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-10T09:29:59.635 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:29:59 smithi102 ceph-mon[35442]: from='osd.1 [v2:172.21.15.102:6800/733924375,v1:172.21.15.102:6801/733924375]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-10T09:29:59.635 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:29:59 smithi102 ceph-mon[35442]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-10T09:29:59.635 INFO:journalctl@ceph.osd.1.smithi102.stdout:Sep 10 09:29:59 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-1[39907]: 2024-09-10T09:29:59.209+0000 7f23f62f5640 -1 osd.1 0 log_to_monitors true 2024-09-10T09:30:00.310 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:30:00 smithi159 ceph-mon[35513]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-10T09:30:00.310 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:30:00 smithi159 ceph-mon[35513]: osdmap e11: 2 total, 1 up, 2 in 2024-09-10T09:30:00.310 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:30:00 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T09:30:00.310 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:30:00 smithi159 ceph-mon[35513]: from='osd.1 [v2:172.21.15.102:6800/733924375,v1:172.21.15.102:6801/733924375]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]: dispatch 2024-09-10T09:30:00.310 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:30:00 smithi159 ceph-mon[35513]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]: dispatch 2024-09-10T09:30:00.311 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:30:00 smithi159 ceph-mon[35513]: pgmap v58: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-10T09:30:00.311 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:30:00 smithi159 ceph-mon[35513]: overall HEALTH_OK 2024-09-10T09:30:00.369 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-10T09:30:00.394 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi159:vg_nvme/lv_4 2024-09-10T09:30:00.523 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:30:00 smithi102 ceph-mon[35442]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-10T09:30:00.523 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:30:00 smithi102 ceph-mon[35442]: osdmap e11: 2 total, 1 up, 2 in 2024-09-10T09:30:00.523 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:30:00 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T09:30:00.524 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:30:00 smithi102 ceph-mon[35442]: from='osd.1 [v2:172.21.15.102:6800/733924375,v1:172.21.15.102:6801/733924375]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]: dispatch 2024-09-10T09:30:00.524 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:30:00 smithi102 ceph-mon[35442]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]: dispatch 2024-09-10T09:30:00.524 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:30:00 smithi102 ceph-mon[35442]: pgmap v58: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-10T09:30:00.524 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:30:00 smithi102 ceph-mon[35442]: overall HEALTH_OK 2024-09-10T09:30:00.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:30:00 smithi005 ceph-mon[30350]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-10T09:30:00.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:30:00 smithi005 ceph-mon[30350]: osdmap e11: 2 total, 1 up, 2 in 2024-09-10T09:30:00.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:30:00 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T09:30:00.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:30:00 smithi005 ceph-mon[30350]: from='osd.1 [v2:172.21.15.102:6800/733924375,v1:172.21.15.102:6801/733924375]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]: dispatch 2024-09-10T09:30:00.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:30:00 smithi005 ceph-mon[30350]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]: dispatch 2024-09-10T09:30:00.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:30:00 smithi005 ceph-mon[30350]: pgmap v58: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-10T09:30:00.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:30:00 smithi005 ceph-mon[30350]: overall HEALTH_OK 2024-09-10T09:30:00.647 INFO:teuthology.orchestra.run.smithi159.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.c/config 2024-09-10T09:30:00.884 INFO:journalctl@ceph.osd.1.smithi102.stdout:Sep 10 09:30:00 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-1[39907]: 2024-09-10T09:30:00.656+0000 7f23f2278640 -1 osd.1 0 waiting for initial osdmap 2024-09-10T09:30:00.885 INFO:journalctl@ceph.osd.1.smithi102.stdout:Sep 10 09:30:00 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-1[39907]: 2024-09-10T09:30:00.659+0000 7f23ea098640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T09:30:01.148 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:30:01.144+0000 7f1844861640 1 -- 172.21.15.159:0/2132812748 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1840107960 msgr2=0x7f1840109d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:30:01.148 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:30:01.144+0000 7f1844861640 1 --2- 172.21.15.159:0/2132812748 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1840107960 0x7f1840109d50 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f1838009ca0 tx=0x7f183802a8c0 comp rx=0 tx=0).stop 2024-09-10T09:30:01.148 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:30:01.144+0000 7f1844861640 1 -- 172.21.15.159:0/2132812748 shutdown_connections 2024-09-10T09:30:01.148 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:30:01.144+0000 7f1844861640 1 --2- 172.21.15.159:0/2132812748 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1840107960 0x7f1840109d50 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:30:01.148 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:30:01.144+0000 7f1844861640 1 --2- 172.21.15.159:0/2132812748 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1840105030 0x7f1840107420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:30:01.148 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:30:01.144+0000 7f1844861640 1 --2- 172.21.15.159:0/2132812748 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1840102700 0x7f1840104af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:30:01.148 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:30:01.144+0000 7f1844861640 1 -- 172.21.15.159:0/2132812748 >> 172.21.15.159:0/2132812748 conn(0x7f18400fc280 msgr2=0x7f18400fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:30:01.148 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:30:01.144+0000 7f1844861640 1 -- 172.21.15.159:0/2132812748 shutdown_connections 2024-09-10T09:30:01.149 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:30:01.144+0000 7f1844861640 1 -- 172.21.15.159:0/2132812748 wait complete. 2024-09-10T09:30:01.149 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:30:01.145+0000 7f1844861640 1 Processor -- start 2024-09-10T09:30:01.149 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:30:01.145+0000 7f1844861640 1 -- start start 2024-09-10T09:30:01.149 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:30:01.145+0000 7f1844861640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1840102700 0x7f1840197ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:30:01.149 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:30:01.145+0000 7f1844861640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1840105030 0x7f1840198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:30:01.149 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:30:01.145+0000 7f1844861640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1840107960 0x7f184019f490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:30:01.149 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:30:01.145+0000 7f1844861640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f184010d8b0 con 0x7f1840105030 2024-09-10T09:30:01.149 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:30:01.145+0000 7f1844861640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f184010d730 con 0x7f1840107960 2024-09-10T09:30:01.149 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:30:01.145+0000 7f1844861640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f184010da30 con 0x7f1840102700 2024-09-10T09:30:01.149 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:30:01.145+0000 7f183e59c640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1840102700 0x7f1840197ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:30:01.149 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:30:01.146+0000 7f183e59c640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1840102700 0x7f1840197ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:44650/0 (socket says 172.21.15.159:44650) 2024-09-10T09:30:01.150 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:30:01.146+0000 7f183e59c640 1 -- 172.21.15.159:0/2315333091 learned_addr learned my addr 172.21.15.159:0/2315333091 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-09-10T09:30:01.150 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:30:01.146+0000 7f183ed9d640 1 --2- 172.21.15.159:0/2315333091 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1840107960 0x7f184019f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:30:01.150 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:30:01.146+0000 7f183e59c640 1 -- 172.21.15.159:0/2315333091 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1840107960 msgr2=0x7f184019f490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:30:01.150 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:30:01.146+0000 7f183e59c640 1 --2- 172.21.15.159:0/2315333091 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1840107960 0x7f184019f490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:30:01.150 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:30:01.146+0000 7f183e59c640 1 -- 172.21.15.159:0/2315333091 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1840105030 msgr2=0x7f1840198410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:30:01.150 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:30:01.146+0000 7f183dd9b640 1 --2- 172.21.15.159:0/2315333091 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1840105030 0x7f1840198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:30:01.150 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:30:01.146+0000 7f183e59c640 1 --2- 172.21.15.159:0/2315333091 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1840105030 0x7f1840198410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:30:01.150 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:30:01.146+0000 7f183e59c640 1 -- 172.21.15.159:0/2315333091 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f184019fb00 con 0x7f1840102700 2024-09-10T09:30:01.150 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:30:01.146+0000 7f183dd9b640 1 --2- 172.21.15.159:0/2315333091 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1840105030 0x7f1840198410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T09:30:01.150 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:30:01.146+0000 7f183e59c640 1 --2- 172.21.15.159:0/2315333091 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1840102700 0x7f1840197ed0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f183400d950 tx=0x7f183400de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:30:01.150 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:30:01.146+0000 7f182f7fe640 1 -- 172.21.15.159:0/2315333091 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1834017070 con 0x7f1840102700 2024-09-10T09:30:01.151 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:30:01.146+0000 7f1844861640 1 -- 172.21.15.159:0/2315333091 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f184019fdf0 con 0x7f1840102700 2024-09-10T09:30:01.151 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:30:01.146+0000 7f182f7fe640 1 -- 172.21.15.159:0/2315333091 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1834009c30 con 0x7f1840102700 2024-09-10T09:30:01.151 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:30:01.146+0000 7f182f7fe640 1 -- 172.21.15.159:0/2315333091 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f183400f6b0 con 0x7f1840102700 2024-09-10T09:30:01.151 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:30:01.146+0000 7f1844861640 1 -- 172.21.15.159:0/2315333091 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18401a0240 con 0x7f1840102700 2024-09-10T09:30:01.152 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:30:01.147+0000 7f1844861640 1 -- 172.21.15.159:0/2315333091 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f18400fdd70 con 0x7f1840102700 2024-09-10T09:30:01.152 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:30:01.148+0000 7f182f7fe640 1 -- 172.21.15.159:0/2315333091 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 15) ==== 137776+0+0 (secure 0 0 0) 0x7f18340109d0 con 0x7f1840102700 2024-09-10T09:30:01.152 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:30:01.149+0000 7f182f7fe640 1 --2- 172.21.15.159:0/2315333091 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f18280a2b20 0x7f18280a4fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:30:01.152 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:30:01.149+0000 7f182f7fe640 1 -- 172.21.15.159:0/2315333091 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(12..12 src has 1..12) ==== 2029+0+0 (secure 0 0 0) 0x7f18340cd6b0 con 0x7f1840102700 2024-09-10T09:30:01.152 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:30:01.149+0000 7f183dd9b640 1 --2- 172.21.15.159:0/2315333091 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f18280a2b20 0x7f18280a4fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:30:01.152 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:30:01.150+0000 7f183dd9b640 1 --2- 172.21.15.159:0/2315333091 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f18280a2b20 0x7f18280a4fe0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f18401993f0 tx=0x7f1830005d30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:30:01.154 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:30:01.153+0000 7f182f7fe640 1 -- 172.21.15.159:0/2315333091 <== mon.1 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f183409bf80 con 0x7f1840102700 2024-09-10T09:30:01.317 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:30:01 smithi159 ceph-mon[35513]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]': finished 2024-09-10T09:30:01.317 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:30:01 smithi159 ceph-mon[35513]: osdmap e12: 2 total, 1 up, 2 in 2024-09-10T09:30:01.317 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:30:01 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T09:30:01.317 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:30:01 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T09:30:01.317 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:30:01 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:30:01.317 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:30:01 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:30:01.317 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:30:01 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:30:01.318 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:30:01 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:30:01.318 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:30:01 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:30:01.318 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:30:01 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:30:01.318 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:30:01 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:30:01.318 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:30:01 smithi159 ceph-mon[35513]: from='osd.1 ' entity='osd.1' 2024-09-10T09:30:01.336 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:30:01.335+0000 7f1844861640 1 -- 172.21.15.159:0/2315333091 --> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi159:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f1840062630 con 0x7f18280a2b20 2024-09-10T09:30:01.533 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:30:01 smithi102 ceph-mon[35442]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]': finished 2024-09-10T09:30:01.533 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:30:01 smithi102 ceph-mon[35442]: osdmap e12: 2 total, 1 up, 2 in 2024-09-10T09:30:01.533 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:30:01 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T09:30:01.533 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:30:01 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T09:30:01.533 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:30:01 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:30:01.533 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:30:01 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:30:01.533 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:30:01 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:30:01.533 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:30:01 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:30:01.533 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:30:01 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:30:01.533 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:30:01 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:30:01.533 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:30:01 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:30:01.534 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:30:01 smithi102 ceph-mon[35442]: from='osd.1 ' entity='osd.1' 2024-09-10T09:30:01.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:30:01 smithi005 ceph-mon[30350]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]': finished 2024-09-10T09:30:01.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:30:01 smithi005 ceph-mon[30350]: osdmap e12: 2 total, 1 up, 2 in 2024-09-10T09:30:01.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:30:01 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T09:30:01.557 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:30:01 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T09:30:01.557 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:30:01 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:30:01.557 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:30:01 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:30:01.557 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:30:01 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:30:01.557 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:30:01 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:30:01.557 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:30:01 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:30:01.557 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:30:01 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:30:01.557 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:30:01 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:30:01.557 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:30:01 smithi005 ceph-mon[30350]: from='osd.1 ' entity='osd.1' 2024-09-10T09:30:02.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:30:02 smithi159 ceph-mon[35513]: purged_snaps scrub starts 2024-09-10T09:30:02.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:30:02 smithi159 ceph-mon[35513]: purged_snaps scrub ok 2024-09-10T09:30:02.351 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:30:02 smithi159 ceph-mon[35513]: Detected new or changed devices on smithi102 2024-09-10T09:30:02.351 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:30:02 smithi159 ceph-mon[35513]: Adjusting osd_memory_target on smithi102 to 17088M 2024-09-10T09:30:02.351 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:30:02 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T09:30:02.351 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:30:02 smithi159 ceph-mon[35513]: from='client.24166 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi159:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T09:30:02.351 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:30:02 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T09:30:02.351 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:30:02 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T09:30:02.351 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:30:02 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:30:02.351 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:30:02 smithi159 ceph-mon[35513]: pgmap v60: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-10T09:30:02.351 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:30:02 smithi159 ceph-mon[35513]: osd.1 [v2:172.21.15.102:6800/733924375,v1:172.21.15.102:6801/733924375] boot 2024-09-10T09:30:02.351 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:30:02 smithi159 ceph-mon[35513]: osdmap e13: 2 total, 2 up, 2 in 2024-09-10T09:30:02.351 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:30:02 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T09:30:02.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:30:02 smithi005 ceph-mon[30350]: purged_snaps scrub starts 2024-09-10T09:30:02.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:30:02 smithi005 ceph-mon[30350]: purged_snaps scrub ok 2024-09-10T09:30:02.557 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:30:02 smithi005 ceph-mon[30350]: Detected new or changed devices on smithi102 2024-09-10T09:30:02.557 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:30:02 smithi005 ceph-mon[30350]: Adjusting osd_memory_target on smithi102 to 17088M 2024-09-10T09:30:02.557 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:30:02 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T09:30:02.557 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:30:02 smithi005 ceph-mon[30350]: from='client.24166 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi159:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T09:30:02.557 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:30:02 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T09:30:02.557 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:30:02 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T09:30:02.557 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:30:02 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:30:02.557 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:30:02 smithi005 ceph-mon[30350]: pgmap v60: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-10T09:30:02.557 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:30:02 smithi005 ceph-mon[30350]: osd.1 [v2:172.21.15.102:6800/733924375,v1:172.21.15.102:6801/733924375] boot 2024-09-10T09:30:02.557 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:30:02 smithi005 ceph-mon[30350]: osdmap e13: 2 total, 2 up, 2 in 2024-09-10T09:30:02.557 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:30:02 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T09:30:02.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:30:02 smithi102 ceph-mon[35442]: purged_snaps scrub starts 2024-09-10T09:30:02.635 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:30:02 smithi102 ceph-mon[35442]: purged_snaps scrub ok 2024-09-10T09:30:02.635 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:30:02 smithi102 ceph-mon[35442]: Detected new or changed devices on smithi102 2024-09-10T09:30:02.635 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:30:02 smithi102 ceph-mon[35442]: Adjusting osd_memory_target on smithi102 to 17088M 2024-09-10T09:30:02.635 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:30:02 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T09:30:02.635 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:30:02 smithi102 ceph-mon[35442]: from='client.24166 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi159:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T09:30:02.635 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:30:02 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T09:30:02.635 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:30:02 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T09:30:02.635 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:30:02 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:30:02.635 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:30:02 smithi102 ceph-mon[35442]: pgmap v60: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-10T09:30:02.635 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:30:02 smithi102 ceph-mon[35442]: osd.1 [v2:172.21.15.102:6800/733924375,v1:172.21.15.102:6801/733924375] boot 2024-09-10T09:30:02.635 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:30:02 smithi102 ceph-mon[35442]: osdmap e13: 2 total, 2 up, 2 in 2024-09-10T09:30:02.636 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:30:02 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T09:30:03.922 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:30:03 smithi159 ceph-mon[35513]: osdmap e14: 2 total, 2 up, 2 in 2024-09-10T09:30:03.923 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:30:03 smithi159 ceph-mon[35513]: pgmap v63: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-10T09:30:04.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:30:03 smithi005 ceph-mon[30350]: osdmap e14: 2 total, 2 up, 2 in 2024-09-10T09:30:04.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:30:03 smithi005 ceph-mon[30350]: pgmap v63: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-10T09:30:04.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:30:03 smithi102 ceph-mon[35442]: osdmap e14: 2 total, 2 up, 2 in 2024-09-10T09:30:04.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:30:03 smithi102 ceph-mon[35442]: pgmap v63: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-10T09:30:05.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:30:04 smithi005 ceph-mon[30350]: from='client.? 172.21.15.159:0/515574031' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0c98fcb8-60ab-4473-8995-1d9e97c577e6"}]: dispatch 2024-09-10T09:30:05.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:30:04 smithi005 ceph-mon[30350]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0c98fcb8-60ab-4473-8995-1d9e97c577e6"}]: dispatch 2024-09-10T09:30:05.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:30:04 smithi005 ceph-mon[30350]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0c98fcb8-60ab-4473-8995-1d9e97c577e6"}]': finished 2024-09-10T09:30:05.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:30:04 smithi005 ceph-mon[30350]: osdmap e15: 3 total, 2 up, 3 in 2024-09-10T09:30:05.057 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:30:04 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T09:30:05.057 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:30:04 smithi005 ceph-mon[30350]: from='client.? 172.21.15.159:0/1632148719' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T09:30:05.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:30:04 smithi159 ceph-mon[35513]: from='client.? 172.21.15.159:0/515574031' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0c98fcb8-60ab-4473-8995-1d9e97c577e6"}]: dispatch 2024-09-10T09:30:05.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:30:04 smithi159 ceph-mon[35513]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0c98fcb8-60ab-4473-8995-1d9e97c577e6"}]: dispatch 2024-09-10T09:30:05.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:30:04 smithi159 ceph-mon[35513]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0c98fcb8-60ab-4473-8995-1d9e97c577e6"}]': finished 2024-09-10T09:30:05.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:30:04 smithi159 ceph-mon[35513]: osdmap e15: 3 total, 2 up, 3 in 2024-09-10T09:30:05.101 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:30:04 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T09:30:05.101 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:30:04 smithi159 ceph-mon[35513]: from='client.? 172.21.15.159:0/1632148719' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T09:30:05.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:30:04 smithi102 ceph-mon[35442]: from='client.? 172.21.15.159:0/515574031' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0c98fcb8-60ab-4473-8995-1d9e97c577e6"}]: dispatch 2024-09-10T09:30:05.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:30:04 smithi102 ceph-mon[35442]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0c98fcb8-60ab-4473-8995-1d9e97c577e6"}]: dispatch 2024-09-10T09:30:05.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:30:04 smithi102 ceph-mon[35442]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0c98fcb8-60ab-4473-8995-1d9e97c577e6"}]': finished 2024-09-10T09:30:05.135 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:30:04 smithi102 ceph-mon[35442]: osdmap e15: 3 total, 2 up, 3 in 2024-09-10T09:30:05.135 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:30:04 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T09:30:05.135 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:30:04 smithi102 ceph-mon[35442]: from='client.? 172.21.15.159:0/1632148719' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T09:30:06.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:30:05 smithi005 ceph-mon[30350]: pgmap v65: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-10T09:30:06.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:30:05 smithi159 ceph-mon[35513]: pgmap v65: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-10T09:30:06.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:30:05 smithi102 ceph-mon[35442]: pgmap v65: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-10T09:30:07.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:30:07 smithi005 ceph-mon[30350]: pgmap v66: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-10T09:30:07.808 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:30:07 smithi159 ceph-mon[35513]: pgmap v66: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-10T09:30:07.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:30:07 smithi102 ceph-mon[35442]: pgmap v66: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-10T09:30:09.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:30:09 smithi005 ceph-mon[30350]: pgmap v67: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-10T09:30:09.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:30:09 smithi159 ceph-mon[35513]: pgmap v67: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-10T09:30:09.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:30:09 smithi102 ceph-mon[35442]: pgmap v67: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-10T09:30:11.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:30:11 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-10T09:30:11.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:30:11 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:30:11.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:30:11 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-10T09:30:11.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:30:11 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:30:11.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:30:11 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-10T09:30:11.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:30:11 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:30:12.630 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:30:12 smithi159 ceph-mon[35513]: Deploying daemon osd.2 on smithi159 2024-09-10T09:30:12.631 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:30:12 smithi159 ceph-mon[35513]: pgmap v68: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-10T09:30:12.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:30:12 smithi102 ceph-mon[35442]: Deploying daemon osd.2 on smithi159 2024-09-10T09:30:12.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:30:12 smithi102 ceph-mon[35442]: pgmap v68: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-10T09:30:12.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:30:12 smithi005 ceph-mon[30350]: Deploying daemon osd.2 on smithi159 2024-09-10T09:30:12.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:30:12 smithi005 ceph-mon[30350]: pgmap v68: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-10T09:30:13.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:30:13 smithi005 ceph-mon[30350]: pgmap v69: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-10T09:30:13.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:30:13 smithi159 ceph-mon[35513]: pgmap v69: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-10T09:30:13.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:30:13 smithi102 ceph-mon[35442]: pgmap v69: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-10T09:30:15.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:30:15 smithi005 ceph-mon[30350]: pgmap v70: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-10T09:30:15.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:30:15 smithi159 ceph-mon[35513]: pgmap v70: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-10T09:30:15.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:30:15 smithi102 ceph-mon[35442]: pgmap v70: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-10T09:30:17.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:30:17 smithi159 ceph-mon[35513]: pgmap v71: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-10T09:30:17.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:30:17 smithi005 ceph-mon[30350]: pgmap v71: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-10T09:30:17.885 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:30:17 smithi102 ceph-mon[35442]: pgmap v71: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-10T09:30:18.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:30:18 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:30:18.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:30:18 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:30:18.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:30:18 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:30:18.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:30:18 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:30:18.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:30:18 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:30:18.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:30:18 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:30:18.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:30:18 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:30:18.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:30:18 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:30:18.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:30:18 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:30:19.785 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:30:19 smithi102 ceph-mon[35442]: pgmap v72: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-10T09:30:19.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:30:19 smithi005 ceph-mon[30350]: pgmap v72: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-10T09:30:19.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:30:19 smithi159 ceph-mon[35513]: pgmap v72: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-10T09:30:21.348 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:30:21 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:30:21.348 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:30:21 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:30:21.348 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:30:21 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:30:21.349 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:30:21 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:30:21.349 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:30:21 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:30:21.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:30:21 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:30:21.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:30:21 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:30:21.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:30:21 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:30:21.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:30:21 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:30:21.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:30:21 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:30:21.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:30:21 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:30:21.557 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:30:21 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:30:21.557 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:30:21 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:30:21.557 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:30:21 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:30:21.557 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:30:21 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:30:21.598 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:30:21.596+0000 7f182f7fe640 1 -- 172.21.15.159:0/2315333091 <== mgr.14150 v2:172.21.15.5:6800/2707348081 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f1840062630 con 0x7f18280a2b20 2024-09-10T09:30:21.599 INFO:teuthology.orchestra.run.smithi159.stdout:Created osd(s) 2 on host 'smithi159' 2024-09-10T09:30:21.600 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:30:21.599+0000 7f1844861640 1 -- 172.21.15.159:0/2315333091 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f18280a2b20 msgr2=0x7f18280a4fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:30:21.600 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:30:21.599+0000 7f1844861640 1 --2- 172.21.15.159:0/2315333091 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f18280a2b20 0x7f18280a4fe0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f18401993f0 tx=0x7f1830005d30 comp rx=0 tx=0).stop 2024-09-10T09:30:21.601 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:30:21.599+0000 7f1844861640 1 -- 172.21.15.159:0/2315333091 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1840102700 msgr2=0x7f1840197ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:30:21.601 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:30:21.599+0000 7f1844861640 1 --2- 172.21.15.159:0/2315333091 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1840102700 0x7f1840197ed0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f183400d950 tx=0x7f183400de10 comp rx=0 tx=0).stop 2024-09-10T09:30:21.601 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:30:21.599+0000 7f1844861640 1 -- 172.21.15.159:0/2315333091 shutdown_connections 2024-09-10T09:30:21.601 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:30:21.599+0000 7f1844861640 1 --2- 172.21.15.159:0/2315333091 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f18280a2b20 0x7f18280a4fe0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:30:21.601 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:30:21.599+0000 7f1844861640 1 --2- 172.21.15.159:0/2315333091 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1840107960 0x7f184019f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:30:21.602 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:30:21.599+0000 7f1844861640 1 --2- 172.21.15.159:0/2315333091 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1840105030 0x7f1840198410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:30:21.602 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:30:21.599+0000 7f1844861640 1 --2- 172.21.15.159:0/2315333091 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1840102700 0x7f1840197ed0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:30:21.602 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:30:21.599+0000 7f1844861640 1 -- 172.21.15.159:0/2315333091 >> 172.21.15.159:0/2315333091 conn(0x7f18400fc280 msgr2=0x7f1840102fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:30:21.602 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:30:21.599+0000 7f1844861640 1 -- 172.21.15.159:0/2315333091 shutdown_connections 2024-09-10T09:30:21.602 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:30:21.599+0000 7f1844861640 1 -- 172.21.15.159:0/2315333091 wait complete. 2024-09-10T09:30:22.219 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:30:22 smithi159 ceph-mon[35513]: pgmap v73: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-10T09:30:22.219 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:30:22 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:30:22.219 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:30:22 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:30:22.219 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:30:22 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:30:22.250 DEBUG:teuthology.orchestra.run.smithi159:osd.2> sudo journalctl -f -n 0 -u ceph-bade4b50-6f56-11ef-bcea-c7b262605968@osd.2.service 2024-09-10T09:30:22.252 INFO:tasks.cephadm:Deploying osd.3 on smithi005 with /dev/vg_nvme/lv_3... 2024-09-10T09:30:22.252 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_3 2024-09-10T09:30:22.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:30:22 smithi102 ceph-mon[35442]: pgmap v73: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-10T09:30:22.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:30:22 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:30:22.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:30:22 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:30:22.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:30:22 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:30:22.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:30:22 smithi005 ceph-mon[30350]: pgmap v73: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-10T09:30:22.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:30:22 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:30:22.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:30:22 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:30:22.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:30:22 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:30:22.629 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:30:23.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:30:23 smithi159 ceph-mon[35513]: from='osd.2 [v2:172.21.15.159:6800/2236204189,v1:172.21.15.159:6801/2236204189]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-10T09:30:23.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:30:23 smithi159 ceph-mon[35513]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-10T09:30:23.601 INFO:journalctl@ceph.osd.2.smithi159.stdout:Sep 10 09:30:23 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-2[40490]: 2024-09-10T09:30:23.327+0000 7f41784db640 -1 osd.2 0 log_to_monitors true 2024-09-10T09:30:23.795 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:30:23 smithi005 ceph-mon[30350]: from='osd.2 [v2:172.21.15.159:6800/2236204189,v1:172.21.15.159:6801/2236204189]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-10T09:30:23.795 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:30:23 smithi005 ceph-mon[30350]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-10T09:30:23.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:30:23 smithi102 ceph-mon[35442]: from='osd.2 [v2:172.21.15.159:6800/2236204189,v1:172.21.15.159:6801/2236204189]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-10T09:30:23.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:30:23 smithi102 ceph-mon[35442]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-10T09:30:24.741 INFO:journalctl@ceph.osd.2.smithi159.stdout:Sep 10 09:30:24 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-2[40490]: 2024-09-10T09:30:24.398+0000 7f417445e640 -1 osd.2 0 waiting for initial osdmap 2024-09-10T09:30:24.741 INFO:journalctl@ceph.osd.2.smithi159.stdout:Sep 10 09:30:24 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-2[40490]: 2024-09-10T09:30:24.403+0000 7f4170286640 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T09:30:24.742 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:30:24 smithi159 ceph-mon[35513]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-10T09:30:24.742 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:30:24 smithi159 ceph-mon[35513]: osdmap e16: 3 total, 2 up, 3 in 2024-09-10T09:30:24.742 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:30:24 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T09:30:24.742 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:30:24 smithi159 ceph-mon[35513]: from='osd.2 [v2:172.21.15.159:6800/2236204189,v1:172.21.15.159:6801/2236204189]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]: dispatch 2024-09-10T09:30:24.742 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:30:24 smithi159 ceph-mon[35513]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]: dispatch 2024-09-10T09:30:24.742 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:30:24 smithi159 ceph-mon[35513]: pgmap v75: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-10T09:30:24.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:30:24 smithi005 ceph-mon[30350]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-10T09:30:24.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:30:24 smithi005 ceph-mon[30350]: osdmap e16: 3 total, 2 up, 3 in 2024-09-10T09:30:24.807 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:30:24 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T09:30:24.807 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:30:24 smithi005 ceph-mon[30350]: from='osd.2 [v2:172.21.15.159:6800/2236204189,v1:172.21.15.159:6801/2236204189]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]: dispatch 2024-09-10T09:30:24.807 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:30:24 smithi005 ceph-mon[30350]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]: dispatch 2024-09-10T09:30:24.807 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:30:24 smithi005 ceph-mon[30350]: pgmap v75: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-10T09:30:24.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:30:24 smithi102 ceph-mon[35442]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-10T09:30:24.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:30:24 smithi102 ceph-mon[35442]: osdmap e16: 3 total, 2 up, 3 in 2024-09-10T09:30:24.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:30:24 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T09:30:24.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:30:24 smithi102 ceph-mon[35442]: from='osd.2 [v2:172.21.15.159:6800/2236204189,v1:172.21.15.159:6801/2236204189]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]: dispatch 2024-09-10T09:30:24.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:30:24 smithi102 ceph-mon[35442]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]: dispatch 2024-09-10T09:30:24.885 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:30:24 smithi102 ceph-mon[35442]: pgmap v75: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-10T09:30:25.758 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:30:25 smithi005 ceph-mon[30350]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]': finished 2024-09-10T09:30:25.759 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:30:25 smithi005 ceph-mon[30350]: osdmap e17: 3 total, 2 up, 3 in 2024-09-10T09:30:25.759 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:30:25 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T09:30:25.759 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:30:25 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T09:30:25.759 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:30:25 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:30:25.759 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:30:25 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:30:25.759 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:30:25 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:30:25.759 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:30:25 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:30:25.759 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:30:25 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:30:25.759 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:30:25 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:30:25.759 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:30:25 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:30:25.819 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-10T09:30:25.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:30:25 smithi159 ceph-mon[35513]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]': finished 2024-09-10T09:30:25.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:30:25 smithi159 ceph-mon[35513]: osdmap e17: 3 total, 2 up, 3 in 2024-09-10T09:30:25.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:30:25 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T09:30:25.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:30:25 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T09:30:25.851 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:30:25 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:30:25.851 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:30:25 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:30:25.851 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:30:25 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:30:25.851 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:30:25 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:30:25.851 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:30:25 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:30:25.851 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:30:25 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:30:25.851 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:30:25 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:30:25.853 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi005:vg_nvme/lv_3 2024-09-10T09:30:25.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:30:25 smithi102 ceph-mon[35442]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]': finished 2024-09-10T09:30:25.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:30:25 smithi102 ceph-mon[35442]: osdmap e17: 3 total, 2 up, 3 in 2024-09-10T09:30:25.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:30:25 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T09:30:25.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:30:25 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T09:30:25.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:30:25 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:30:25.885 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:30:25 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:30:25.885 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:30:25 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:30:25.885 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:30:25 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:30:25.885 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:30:25 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:30:25.885 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:30:25 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:30:25.885 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:30:25 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:30:26.228 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:30:26.646 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:30:26 smithi005 ceph-mon[30350]: purged_snaps scrub starts 2024-09-10T09:30:26.646 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:30:26 smithi005 ceph-mon[30350]: purged_snaps scrub ok 2024-09-10T09:30:26.646 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:30:26 smithi005 ceph-mon[30350]: Detected new or changed devices on smithi159 2024-09-10T09:30:26.646 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:30:26 smithi005 ceph-mon[30350]: Adjusting osd_memory_target on smithi159 to 17088M 2024-09-10T09:30:26.646 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:30:26 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T09:30:26.646 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:30:26 smithi005 ceph-mon[30350]: osd.2 [v2:172.21.15.159:6800/2236204189,v1:172.21.15.159:6801/2236204189] boot 2024-09-10T09:30:26.646 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:30:26 smithi005 ceph-mon[30350]: osdmap e18: 3 total, 3 up, 3 in 2024-09-10T09:30:26.646 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:30:26 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T09:30:26.647 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:30:26 smithi005 ceph-mon[30350]: pgmap v78: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-10T09:30:26.647 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:30:26 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-09-10T09:30:26.830 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:30:26 smithi159 ceph-mon[35513]: purged_snaps scrub starts 2024-09-10T09:30:26.830 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:30:26 smithi159 ceph-mon[35513]: purged_snaps scrub ok 2024-09-10T09:30:26.830 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:30:26 smithi159 ceph-mon[35513]: Detected new or changed devices on smithi159 2024-09-10T09:30:26.831 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:30:26 smithi159 ceph-mon[35513]: Adjusting osd_memory_target on smithi159 to 17088M 2024-09-10T09:30:26.831 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:30:26 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T09:30:26.831 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:30:26 smithi159 ceph-mon[35513]: osd.2 [v2:172.21.15.159:6800/2236204189,v1:172.21.15.159:6801/2236204189] boot 2024-09-10T09:30:26.831 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:30:26 smithi159 ceph-mon[35513]: osdmap e18: 3 total, 3 up, 3 in 2024-09-10T09:30:26.831 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:30:26 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T09:30:26.831 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:30:26 smithi159 ceph-mon[35513]: pgmap v78: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-10T09:30:26.831 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:30:26 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-09-10T09:30:26.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:30:26.857+0000 7f12ee200640 1 -- 172.21.15.5:0/459285217 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f12e8104f60 msgr2=0x7f12e81053e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:30:26.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:30:26.857+0000 7f12ee200640 1 --2- 172.21.15.5:0/459285217 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f12e8104f60 0x7f12e81053e0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f12d0009900 tx=0x7f12d002a3f0 comp rx=0 tx=0).stop 2024-09-10T09:30:26.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:30:26.857+0000 7f12ee200640 1 -- 172.21.15.5:0/459285217 shutdown_connections 2024-09-10T09:30:26.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:30:26.857+0000 7f12ee200640 1 --2- 172.21.15.5:0/459285217 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f12e8105920 0x7f12e810c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:30:26.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:30:26.857+0000 7f12ee200640 1 --2- 172.21.15.5:0/459285217 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f12e8104f60 0x7f12e81053e0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:30:26.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:30:26.857+0000 7f12ee200640 1 --2- 172.21.15.5:0/459285217 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f12e8103e00 0x7f12e8104200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:30:26.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:30:26.857+0000 7f12ee200640 1 -- 172.21.15.5:0/459285217 >> 172.21.15.5:0/459285217 conn(0x7f12e80ff5b0 msgr2=0x7f12e81019d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:30:26.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:30:26.857+0000 7f12ee200640 1 -- 172.21.15.5:0/459285217 shutdown_connections 2024-09-10T09:30:26.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:30:26.857+0000 7f12ee200640 1 -- 172.21.15.5:0/459285217 wait complete. 2024-09-10T09:30:26.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:30:26.858+0000 7f12ee200640 1 Processor -- start 2024-09-10T09:30:26.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:30:26.858+0000 7f12ee200640 1 -- start start 2024-09-10T09:30:26.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:30:26.859+0000 7f12ee200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f12e8103e00 0x7f12e819a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:30:26.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:30:26.859+0000 7f12ee200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f12e8104f60 0x7f12e819a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:30:26.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:30:26.859+0000 7f12ee200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f12e8105920 0x7f12e81a16b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:30:26.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:30:26.859+0000 7f12ee200640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f12e810fb00 con 0x7f12e8103e00 2024-09-10T09:30:26.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:30:26.859+0000 7f12ee200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f12e810f980 con 0x7f12e8105920 2024-09-10T09:30:26.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:30:26.859+0000 7f12ee200640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f12e810fc80 con 0x7f12e8104f60 2024-09-10T09:30:26.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:30:26.859+0000 7f12e7400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f12e8103e00 0x7f12e819a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:30:26.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:30:26.859+0000 7f12dea00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f12e8104f60 0x7f12e819a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:30:26.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:30:26.859+0000 7f12dea00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f12e8104f60 0x7f12e819a630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:38750/0 (socket says 172.21.15.5:38750) 2024-09-10T09:30:26.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:30:26.859+0000 7f12dea00640 1 -- 172.21.15.5:0/1797591602 learned_addr learned my addr 172.21.15.5:0/1797591602 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:30:26.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:30:26.859+0000 7f12e7e00640 1 --2- 172.21.15.5:0/1797591602 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f12e8105920 0x7f12e81a16b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:30:26.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:30:26.859+0000 7f12dea00640 1 -- 172.21.15.5:0/1797591602 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f12e8105920 msgr2=0x7f12e81a16b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:30:26.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:30:26.859+0000 7f12dea00640 1 --2- 172.21.15.5:0/1797591602 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f12e8105920 0x7f12e81a16b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:30:26.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:30:26.860+0000 7f12dea00640 1 -- 172.21.15.5:0/1797591602 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f12e8103e00 msgr2=0x7f12e819a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:30:26.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:30:26.860+0000 7f12dea00640 1 --2- 172.21.15.5:0/1797591602 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f12e8103e00 0x7f12e819a0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:30:26.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:30:26.860+0000 7f12dea00640 1 -- 172.21.15.5:0/1797591602 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12e81a1d20 con 0x7f12e8104f60 2024-09-10T09:30:26.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:30:26.860+0000 7f12dea00640 1 --2- 172.21.15.5:0/1797591602 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f12e8104f60 0x7f12e819a630 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f12d0031440 tx=0x7f12d0032ce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:30:26.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:30:26.860+0000 7f12e4c00640 1 -- 172.21.15.5:0/1797591602 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12d003f030 con 0x7f12e8104f60 2024-09-10T09:30:26.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:30:26.860+0000 7f12e4c00640 1 -- 172.21.15.5:0/1797591602 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f12d0003ce0 con 0x7f12e8104f60 2024-09-10T09:30:26.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:30:26.860+0000 7f12ee200640 1 -- 172.21.15.5:0/1797591602 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12e81a1fb0 con 0x7f12e8104f60 2024-09-10T09:30:26.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:30:26.860+0000 7f12e4c00640 1 -- 172.21.15.5:0/1797591602 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12d0035070 con 0x7f12e8104f60 2024-09-10T09:30:26.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:30:26.860+0000 7f12ee200640 1 -- 172.21.15.5:0/1797591602 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12e81a23a0 con 0x7f12e8104f60 2024-09-10T09:30:26.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:30:26.861+0000 7f12ee200640 1 -- 172.21.15.5:0/1797591602 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f12e810b240 con 0x7f12e8104f60 2024-09-10T09:30:26.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:30:26.863+0000 7f12e4c00640 1 -- 172.21.15.5:0/1797591602 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 15) ==== 137776+0+0 (secure 0 0 0) 0x7f12d0042030 con 0x7f12e8104f60 2024-09-10T09:30:26.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:30:26.864+0000 7f12e4c00640 1 --2- 172.21.15.5:0/1797591602 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f12cc0a2b20 0x7f12cc0a4fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:30:26.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:30:26.865+0000 7f12e4c00640 1 -- 172.21.15.5:0/1797591602 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 3226+0+0 (secure 0 0 0) 0x7f12d00edf50 con 0x7f12e8104f60 2024-09-10T09:30:26.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:30:26.865+0000 7f12e7400640 1 --2- 172.21.15.5:0/1797591602 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f12cc0a2b20 0x7f12cc0a4fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:30:26.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:30:26.865+0000 7f12e7400640 1 --2- 172.21.15.5:0/1797591602 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f12cc0a2b20 0x7f12cc0a4fe0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f12d8005fd0 tx=0x7f12d8009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:30:26.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:30:26.869+0000 7f12e4c00640 1 -- 172.21.15.5:0/1797591602 <== mon.1 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f12d00bbc80 con 0x7f12e8104f60 2024-09-10T09:30:26.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:30:26 smithi102 ceph-mon[35442]: purged_snaps scrub starts 2024-09-10T09:30:26.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:30:26 smithi102 ceph-mon[35442]: purged_snaps scrub ok 2024-09-10T09:30:26.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:30:26 smithi102 ceph-mon[35442]: Detected new or changed devices on smithi159 2024-09-10T09:30:26.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:30:26 smithi102 ceph-mon[35442]: Adjusting osd_memory_target on smithi159 to 17088M 2024-09-10T09:30:26.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:30:26 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T09:30:26.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:30:26 smithi102 ceph-mon[35442]: osd.2 [v2:172.21.15.159:6800/2236204189,v1:172.21.15.159:6801/2236204189] boot 2024-09-10T09:30:26.885 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:30:26 smithi102 ceph-mon[35442]: osdmap e18: 3 total, 3 up, 3 in 2024-09-10T09:30:26.885 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:30:26 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T09:30:26.885 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:30:26 smithi102 ceph-mon[35442]: pgmap v78: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-10T09:30:26.885 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:30:26 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-09-10T09:30:27.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:30:27.064+0000 7f12ee200640 1 -- 172.21.15.5:0/1797591602 --> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi005:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f12e8062630 con 0x7f12cc0a2b20 2024-09-10T09:30:27.673 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:30:27 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-10T09:30:27.673 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:30:27 smithi005 ceph-mon[30350]: osdmap e19: 3 total, 3 up, 3 in 2024-09-10T09:30:27.673 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:30:27 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-10T09:30:27.673 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:30:27 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T09:30:27.674 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:30:27 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T09:30:27.674 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:30:27 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:30:27.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:30:27 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-10T09:30:27.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:30:27 smithi159 ceph-mon[35513]: osdmap e19: 3 total, 3 up, 3 in 2024-09-10T09:30:27.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:30:27 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-10T09:30:27.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:30:27 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T09:30:27.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:30:27 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T09:30:27.851 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:30:27 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:30:27.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:30:27 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-10T09:30:27.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:30:27 smithi102 ceph-mon[35442]: osdmap e19: 3 total, 3 up, 3 in 2024-09-10T09:30:27.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:30:27 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-10T09:30:27.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:30:27 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T09:30:27.885 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:30:27 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T09:30:27.885 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:30:27 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:30:28.306 INFO:journalctl@ceph.osd.0.smithi005.stdout:Sep 10 09:30:28 smithi005 sudo[49226]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-10T09:30:28.306 INFO:journalctl@ceph.osd.0.smithi005.stdout:Sep 10 09:30:28 smithi005 sudo[49226]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T09:30:28.306 INFO:journalctl@ceph.osd.0.smithi005.stdout:Sep 10 09:30:28 smithi005 sudo[49226]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T09:30:28.306 INFO:journalctl@ceph.osd.0.smithi005.stdout:Sep 10 09:30:28 smithi005 sudo[49226]: pam_unix(sudo:session): session closed for user root 2024-09-10T09:30:28.714 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:30:28 smithi005 ceph-mon[30350]: from='client.24193 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi005:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T09:30:28.714 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:30:28 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-10T09:30:28.714 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:30:28 smithi005 ceph-mon[30350]: osdmap e20: 3 total, 3 up, 3 in 2024-09-10T09:30:28.715 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:30:28 smithi005 ceph-mon[30350]: pgmap v81: 1 pgs: 1 creating+peering; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-10T09:30:28.715 INFO:journalctl@ceph.osd.0.smithi005.stdout:Sep 10 09:30:28 smithi005 sudo[49234]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-10T09:30:28.715 INFO:journalctl@ceph.osd.0.smithi005.stdout:Sep 10 09:30:28 smithi005 sudo[49234]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T09:30:28.715 INFO:journalctl@ceph.osd.0.smithi005.stdout:Sep 10 09:30:28 smithi005 sudo[49234]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T09:30:28.715 INFO:journalctl@ceph.osd.0.smithi005.stdout:Sep 10 09:30:28 smithi005 sudo[49234]: pam_unix(sudo:session): session closed for user root 2024-09-10T09:30:28.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:30:28 smithi159 ceph-mon[35513]: from='client.24193 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi005:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T09:30:28.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:30:28 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-10T09:30:28.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:30:28 smithi159 ceph-mon[35513]: osdmap e20: 3 total, 3 up, 3 in 2024-09-10T09:30:28.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:30:28 smithi159 ceph-mon[35513]: pgmap v81: 1 pgs: 1 creating+peering; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-10T09:30:28.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:30:28 smithi102 ceph-mon[35442]: from='client.24193 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi005:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T09:30:28.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:30:28 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-10T09:30:28.885 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:30:28 smithi102 ceph-mon[35442]: osdmap e20: 3 total, 3 up, 3 in 2024-09-10T09:30:28.885 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:30:28 smithi102 ceph-mon[35442]: pgmap v81: 1 pgs: 1 creating+peering; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-10T09:30:29.184 INFO:journalctl@ceph.osd.1.smithi102.stdout:Sep 10 09:30:28 smithi102 sudo[43601]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-10T09:30:29.184 INFO:journalctl@ceph.osd.1.smithi102.stdout:Sep 10 09:30:28 smithi102 sudo[43601]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T09:30:29.184 INFO:journalctl@ceph.osd.1.smithi102.stdout:Sep 10 09:30:28 smithi102 sudo[43601]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T09:30:29.184 INFO:journalctl@ceph.osd.1.smithi102.stdout:Sep 10 09:30:28 smithi102 sudo[43601]: pam_unix(sudo:session): session closed for user root 2024-09-10T09:30:29.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:30:29 smithi005 ceph-mon[30350]: osdmap e21: 3 total, 3 up, 3 in 2024-09-10T09:30:29.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:30:29 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:30:29.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:30:29 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:30:29.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:30:29 smithi102 ceph-mon[35442]: osdmap e21: 3 total, 3 up, 3 in 2024-09-10T09:30:29.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:30:29 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:30:29.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:30:29 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:30:29.635 INFO:journalctl@ceph.osd.1.smithi102.stdout:Sep 10 09:30:29 smithi102 sudo[43604]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-10T09:30:29.635 INFO:journalctl@ceph.osd.1.smithi102.stdout:Sep 10 09:30:29 smithi102 sudo[43604]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T09:30:29.635 INFO:journalctl@ceph.osd.1.smithi102.stdout:Sep 10 09:30:29 smithi102 sudo[43604]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T09:30:29.635 INFO:journalctl@ceph.osd.1.smithi102.stdout:Sep 10 09:30:29 smithi102 sudo[43604]: pam_unix(sudo:session): session closed for user root 2024-09-10T09:30:29.708 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:30:29 smithi159 ceph-mon[35513]: osdmap e21: 3 total, 3 up, 3 in 2024-09-10T09:30:29.708 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:30:29 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:30:29.708 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:30:29 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:30:29.708 INFO:journalctl@ceph.osd.2.smithi159.stdout:Sep 10 09:30:29 smithi159 sudo[44096]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-10T09:30:29.709 INFO:journalctl@ceph.osd.2.smithi159.stdout:Sep 10 09:30:29 smithi159 sudo[44096]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T09:30:29.709 INFO:journalctl@ceph.osd.2.smithi159.stdout:Sep 10 09:30:29 smithi159 sudo[44096]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T09:30:29.709 INFO:journalctl@ceph.osd.2.smithi159.stdout:Sep 10 09:30:29 smithi159 sudo[44096]: pam_unix(sudo:session): session closed for user root 2024-09-10T09:30:30.100 INFO:journalctl@ceph.osd.2.smithi159.stdout:Sep 10 09:30:29 smithi159 sudo[44099]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-10T09:30:30.100 INFO:journalctl@ceph.osd.2.smithi159.stdout:Sep 10 09:30:29 smithi159 sudo[44099]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T09:30:30.100 INFO:journalctl@ceph.osd.2.smithi159.stdout:Sep 10 09:30:29 smithi159 sudo[44099]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T09:30:30.100 INFO:journalctl@ceph.osd.2.smithi159.stdout:Sep 10 09:30:29 smithi159 sudo[44099]: pam_unix(sudo:session): session closed for user root 2024-09-10T09:30:30.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:30:30 smithi005 sudo[49308]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-10T09:30:30.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:30:30 smithi005 sudo[49308]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T09:30:30.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:30:30 smithi005 sudo[49308]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T09:30:30.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:30:30 smithi005 sudo[49308]: pam_unix(sudo:session): session closed for user root 2024-09-10T09:30:31.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:30:30 smithi102 ceph-mon[35442]: pgmap v83: 1 pgs: 1 creating+peering; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-10T09:30:31.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:30:30 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:30:31.135 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:30:30 smithi102 ceph-mon[35442]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-10T09:30:31.152 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:30:30 smithi005 sudo[49311]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-10T09:30:31.152 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:30:30 smithi005 sudo[49311]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T09:30:31.152 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:30:30 smithi005 sudo[49311]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T09:30:31.152 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:30:30 smithi005 sudo[49311]: pam_unix(sudo:session): session closed for user root 2024-09-10T09:30:31.152 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:30:30 smithi005 ceph-mon[30350]: pgmap v83: 1 pgs: 1 creating+peering; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-10T09:30:31.152 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:30:30 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:30:31.152 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:30:30 smithi005 ceph-mon[30350]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-10T09:30:31.342 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:30:31 smithi159 sudo[44102]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-10T09:30:31.342 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:30:31 smithi159 sudo[44102]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T09:30:31.342 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:30:31 smithi159 sudo[44102]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T09:30:31.343 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:30:31 smithi159 sudo[44102]: pam_unix(sudo:session): session closed for user root 2024-09-10T09:30:31.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:30:31.356+0000 7f12e4c00640 1 -- 172.21.15.5:0/1797591602 <== mon.1 v2:172.21.15.159:3300/0 7 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7f12d003b460 con 0x7f12e8104f60 2024-09-10T09:30:31.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:30:31 smithi159 sudo[44105]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-10T09:30:31.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:30:31 smithi159 sudo[44105]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T09:30:31.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:30:31 smithi159 sudo[44105]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T09:30:31.601 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:30:31 smithi159 sudo[44105]: pam_unix(sudo:session): session closed for user root 2024-09-10T09:30:31.601 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:30:31 smithi159 ceph-mon[35513]: pgmap v83: 1 pgs: 1 creating+peering; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-10T09:30:31.601 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:30:31 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:30:31.601 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:30:31 smithi159 ceph-mon[35513]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-10T09:30:31.839 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:30:31 smithi102 sudo[43607]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-10T09:30:31.840 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:30:31 smithi102 sudo[43607]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T09:30:31.840 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:30:31 smithi102 sudo[43607]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T09:30:31.840 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:30:31 smithi102 sudo[43607]: pam_unix(sudo:session): session closed for user root 2024-09-10T09:30:32.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:30:31 smithi102 sudo[43610]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-10T09:30:32.135 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:30:31 smithi102 sudo[43610]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T09:30:32.135 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:30:31 smithi102 sudo[43610]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T09:30:32.135 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:30:31 smithi102 sudo[43610]: pam_unix(sudo:session): session closed for user root 2024-09-10T09:30:32.135 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:30:31 smithi102 ceph-mon[35442]: from='client.? 172.21.15.5:0/3059925665' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "88eecdeb-5cee-4f78-a3f9-a66bf4bf772a"}]: dispatch 2024-09-10T09:30:32.135 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:30:31 smithi102 ceph-mon[35442]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-10T09:30:32.135 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:30:31 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T09:30:32.135 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:30:31 smithi102 ceph-mon[35442]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "88eecdeb-5cee-4f78-a3f9-a66bf4bf772a"}]: dispatch 2024-09-10T09:30:32.135 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:30:31 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T09:30:32.135 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:30:31 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T09:30:32.135 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:30:31 smithi102 ceph-mon[35442]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-10T09:30:32.135 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:30:31 smithi102 ceph-mon[35442]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "88eecdeb-5cee-4f78-a3f9-a66bf4bf772a"}]': finished 2024-09-10T09:30:32.135 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:30:31 smithi102 ceph-mon[35442]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-09-10T09:30:32.136 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:30:31 smithi102 ceph-mon[35442]: osdmap e22: 4 total, 3 up, 4 in 2024-09-10T09:30:32.136 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:30:31 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T09:30:32.136 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:30:31 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T09:30:32.136 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:30:31 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T09:30:32.136 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:30:31 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T09:30:32.136 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:30:31 smithi102 ceph-mon[35442]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-10T09:30:32.136 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:30:31 smithi102 ceph-mon[35442]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-10T09:30:32.136 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:30:31 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T09:30:32.136 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:30:31 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T09:30:32.136 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:30:31 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T09:30:32.136 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:30:31 smithi102 ceph-mon[35442]: pgmap v85: 1 pgs: 1 creating+peering; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-10T09:30:32.136 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:30:31 smithi102 ceph-mon[35442]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-10T09:30:32.136 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:30:31 smithi102 ceph-mon[35442]: from='client.? 172.21.15.5:0/3883368919' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T09:30:32.306 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:30:31 smithi005 ceph-mon[30350]: from='client.? 172.21.15.5:0/3059925665' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "88eecdeb-5cee-4f78-a3f9-a66bf4bf772a"}]: dispatch 2024-09-10T09:30:32.307 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:30:31 smithi005 ceph-mon[30350]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-10T09:30:32.307 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:30:31 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T09:30:32.307 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:30:31 smithi005 ceph-mon[30350]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "88eecdeb-5cee-4f78-a3f9-a66bf4bf772a"}]: dispatch 2024-09-10T09:30:32.307 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:30:31 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T09:30:32.307 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:30:31 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T09:30:32.307 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:30:31 smithi005 ceph-mon[30350]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-10T09:30:32.307 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:30:31 smithi005 ceph-mon[30350]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "88eecdeb-5cee-4f78-a3f9-a66bf4bf772a"}]': finished 2024-09-10T09:30:32.307 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:30:31 smithi005 ceph-mon[30350]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-09-10T09:30:32.307 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:30:31 smithi005 ceph-mon[30350]: osdmap e22: 4 total, 3 up, 4 in 2024-09-10T09:30:32.307 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:30:31 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T09:30:32.307 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:30:31 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T09:30:32.308 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:30:31 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T09:30:32.308 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:30:31 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T09:30:32.308 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:30:31 smithi005 ceph-mon[30350]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-10T09:30:32.308 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:30:31 smithi005 ceph-mon[30350]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-10T09:30:32.308 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:30:31 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T09:30:32.308 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:30:31 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T09:30:32.308 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:30:31 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T09:30:32.308 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:30:31 smithi005 ceph-mon[30350]: pgmap v85: 1 pgs: 1 creating+peering; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-10T09:30:32.308 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:30:31 smithi005 ceph-mon[30350]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-10T09:30:32.308 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:30:31 smithi005 ceph-mon[30350]: from='client.? 172.21.15.5:0/3883368919' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T09:30:32.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:30:31 smithi159 ceph-mon[35513]: from='client.? 172.21.15.5:0/3059925665' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "88eecdeb-5cee-4f78-a3f9-a66bf4bf772a"}]: dispatch 2024-09-10T09:30:32.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:30:31 smithi159 ceph-mon[35513]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-10T09:30:32.351 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:30:31 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T09:30:32.351 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:30:31 smithi159 ceph-mon[35513]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "88eecdeb-5cee-4f78-a3f9-a66bf4bf772a"}]: dispatch 2024-09-10T09:30:32.351 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:30:31 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T09:30:32.351 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:30:31 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T09:30:32.351 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:30:31 smithi159 ceph-mon[35513]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-10T09:30:32.351 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:30:31 smithi159 ceph-mon[35513]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "88eecdeb-5cee-4f78-a3f9-a66bf4bf772a"}]': finished 2024-09-10T09:30:32.351 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:30:31 smithi159 ceph-mon[35513]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-09-10T09:30:32.351 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:30:31 smithi159 ceph-mon[35513]: osdmap e22: 4 total, 3 up, 4 in 2024-09-10T09:30:32.351 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:30:31 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T09:30:32.351 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:30:31 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T09:30:32.351 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:30:31 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T09:30:32.351 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:30:31 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T09:30:32.351 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:30:31 smithi159 ceph-mon[35513]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-10T09:30:32.352 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:30:31 smithi159 ceph-mon[35513]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-10T09:30:32.352 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:30:31 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T09:30:32.352 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:30:31 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T09:30:32.352 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:30:31 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T09:30:32.352 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:30:31 smithi159 ceph-mon[35513]: pgmap v85: 1 pgs: 1 creating+peering; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-10T09:30:32.352 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:30:31 smithi159 ceph-mon[35513]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-10T09:30:32.352 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:30:31 smithi159 ceph-mon[35513]: from='client.? 172.21.15.5:0/3883368919' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T09:30:33.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:30:33 smithi005 ceph-mon[30350]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-10T09:30:33.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:30:33 smithi159 ceph-mon[35513]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-10T09:30:33.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:30:33 smithi102 ceph-mon[35442]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-10T09:30:35.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:30:35 smithi005 ceph-mon[30350]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-10T09:30:35.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:30:35 smithi159 ceph-mon[35513]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-10T09:30:35.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:30:35 smithi102 ceph-mon[35442]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-10T09:30:37.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:30:37 smithi005 ceph-mon[30350]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-10T09:30:37.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:30:37 smithi159 ceph-mon[35513]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-10T09:30:37.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:30:37 smithi102 ceph-mon[35442]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-10T09:30:39.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:30:39 smithi005 ceph-mon[30350]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-10T09:30:39.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:30:39 smithi159 ceph-mon[35513]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-10T09:30:39.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:30:39 smithi102 ceph-mon[35442]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-10T09:30:40.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:30:40 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-10T09:30:40.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:30:40 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:30:40.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:30:40 smithi005 ceph-mon[30350]: Deploying daemon osd.3 on smithi005 2024-09-10T09:30:40.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:30:40 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-10T09:30:40.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:30:40 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:30:40.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:30:40 smithi159 ceph-mon[35513]: Deploying daemon osd.3 on smithi005 2024-09-10T09:30:40.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:30:40 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-10T09:30:40.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:30:40 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:30:40.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:30:40 smithi102 ceph-mon[35442]: Deploying daemon osd.3 on smithi005 2024-09-10T09:30:41.748 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:30:41 smithi005 ceph-mon[30350]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-10T09:30:41.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:30:41 smithi159 ceph-mon[35513]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-10T09:30:41.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:30:41 smithi102 ceph-mon[35442]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-10T09:30:43.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:30:43 smithi005 ceph-mon[30350]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-10T09:30:43.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:30:43 smithi159 ceph-mon[35513]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-10T09:30:43.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:30:43 smithi102 ceph-mon[35442]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-10T09:30:45.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:30:45 smithi005 ceph-mon[30350]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-10T09:30:45.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:30:45 smithi159 ceph-mon[35513]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-10T09:30:45.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:30:45 smithi102 ceph-mon[35442]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-10T09:30:47.696 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:30:47 smithi005 ceph-mon[30350]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-10T09:30:47.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:30:47 smithi159 ceph-mon[35513]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-10T09:30:47.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:30:47 smithi102 ceph-mon[35442]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-10T09:30:48.799 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:30:48 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:30:48.799 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:30:48 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:30:48.799 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:30:48 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:30:48.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:30:48 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:30:48.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:30:48 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:30:48.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:30:48 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:30:48.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:30:48 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:30:48.885 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:30:48 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:30:48.885 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:30:48 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:30:49.785 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:30:49 smithi102 ceph-mon[35442]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-10T09:30:49.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:30:49 smithi005 ceph-mon[30350]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-10T09:30:49.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:30:49 smithi159 ceph-mon[35513]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-10T09:30:52.242 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:30:52 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:30:52.242 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:30:52 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:30:52.242 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:30:52 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:30:52.243 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:30:52 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:30:52.243 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:30:52 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:30:52.243 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:30:52 smithi005 ceph-mon[30350]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-10T09:30:52.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:30:52 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:30:52.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:30:52 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:30:52.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:30:52 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:30:52.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:30:52 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:30:52.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:30:52 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:30:52.351 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:30:52 smithi159 ceph-mon[35513]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-10T09:30:52.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:30:52 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:30:52.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:30:52 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:30:52.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:30:52 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:30:52.385 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:30:52 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:30:52.385 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:30:52 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:30:52.385 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:30:52 smithi102 ceph-mon[35442]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-10T09:30:53.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:30:53.034+0000 7f12e4c00640 1 -- 172.21.15.5:0/1797591602 <== mgr.14150 v2:172.21.15.5:6800/2707348081 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f12e8062630 con 0x7f12cc0a2b20 2024-09-10T09:30:53.036 INFO:teuthology.orchestra.run.smithi005.stdout:Created osd(s) 3 on host 'smithi005' 2024-09-10T09:30:53.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:30:53.036+0000 7f12ee200640 1 -- 172.21.15.5:0/1797591602 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f12cc0a2b20 msgr2=0x7f12cc0a4fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:30:53.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:30:53.036+0000 7f12ee200640 1 --2- 172.21.15.5:0/1797591602 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f12cc0a2b20 0x7f12cc0a4fe0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f12d8005fd0 tx=0x7f12d8009500 comp rx=0 tx=0).stop 2024-09-10T09:30:53.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:30:53.036+0000 7f12ee200640 1 -- 172.21.15.5:0/1797591602 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f12e8104f60 msgr2=0x7f12e819a630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:30:53.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:30:53.036+0000 7f12ee200640 1 --2- 172.21.15.5:0/1797591602 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f12e8104f60 0x7f12e819a630 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f12d0031440 tx=0x7f12d0032ce0 comp rx=0 tx=0).stop 2024-09-10T09:30:53.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:30:53.036+0000 7f12ee200640 1 -- 172.21.15.5:0/1797591602 shutdown_connections 2024-09-10T09:30:53.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:30:53.036+0000 7f12ee200640 1 --2- 172.21.15.5:0/1797591602 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f12cc0a2b20 0x7f12cc0a4fe0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:30:53.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:30:53.036+0000 7f12ee200640 1 --2- 172.21.15.5:0/1797591602 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f12e8105920 0x7f12e81a16b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:30:53.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:30:53.036+0000 7f12ee200640 1 --2- 172.21.15.5:0/1797591602 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f12e8104f60 0x7f12e819a630 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:30:53.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:30:53.036+0000 7f12ee200640 1 --2- 172.21.15.5:0/1797591602 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f12e8103e00 0x7f12e819a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:30:53.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:30:53.036+0000 7f12ee200640 1 -- 172.21.15.5:0/1797591602 >> 172.21.15.5:0/1797591602 conn(0x7f12e80ff5b0 msgr2=0x7f12e8108410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:30:53.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:30:53.036+0000 7f12ee200640 1 -- 172.21.15.5:0/1797591602 shutdown_connections 2024-09-10T09:30:53.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:30:53.037+0000 7f12ee200640 1 -- 172.21.15.5:0/1797591602 wait complete. 2024-09-10T09:30:53.306 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:30:53 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:30:53.306 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:30:53 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:30:53.306 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:30:53 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:30:53.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:30:53 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:30:53.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:30:53 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:30:53.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:30:53 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:30:53.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:30:53 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:30:53.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:30:53 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:30:53.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:30:53 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:30:53.771 DEBUG:teuthology.orchestra.run.smithi005:osd.3> sudo journalctl -f -n 0 -u ceph-bade4b50-6f56-11ef-bcea-c7b262605968@osd.3.service 2024-09-10T09:30:53.775 INFO:tasks.cephadm:Deploying osd.4 on smithi102 with /dev/vg_nvme/lv_3... 2024-09-10T09:30:53.775 DEBUG:teuthology.orchestra.run.smithi102:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_3 2024-09-10T09:30:54.020 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T09:30:54.306 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:30:54 smithi005 ceph-mon[30350]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-10T09:30:54.306 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:30:54 smithi005 ceph-mon[30350]: from='osd.3 [v2:172.21.15.5:6810/2221468725,v1:172.21.15.5:6811/2221468725]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-10T09:30:54.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:30:54 smithi159 ceph-mon[35513]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-10T09:30:54.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:30:54 smithi159 ceph-mon[35513]: from='osd.3 [v2:172.21.15.5:6810/2221468725,v1:172.21.15.5:6811/2221468725]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-10T09:30:54.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:30:54 smithi102 ceph-mon[35442]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-10T09:30:54.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:30:54 smithi102 ceph-mon[35442]: from='osd.3 [v2:172.21.15.5:6810/2221468725,v1:172.21.15.5:6811/2221468725]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-10T09:30:55.314 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:30:55 smithi102 ceph-mon[35442]: from='osd.3 [v2:172.21.15.5:6810/2221468725,v1:172.21.15.5:6811/2221468725]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-10T09:30:55.314 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:30:55 smithi102 ceph-mon[35442]: osdmap e23: 4 total, 3 up, 4 in 2024-09-10T09:30:55.314 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:30:55 smithi102 ceph-mon[35442]: from='osd.3 [v2:172.21.15.5:6810/2221468725,v1:172.21.15.5:6811/2221468725]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi005", "root=default"]}]: dispatch 2024-09-10T09:30:55.314 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:30:55 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T09:30:55.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:30:55 smithi159 ceph-mon[35513]: from='osd.3 [v2:172.21.15.5:6810/2221468725,v1:172.21.15.5:6811/2221468725]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-10T09:30:55.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:30:55 smithi159 ceph-mon[35513]: osdmap e23: 4 total, 3 up, 4 in 2024-09-10T09:30:55.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:30:55 smithi159 ceph-mon[35513]: from='osd.3 [v2:172.21.15.5:6810/2221468725,v1:172.21.15.5:6811/2221468725]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi005", "root=default"]}]: dispatch 2024-09-10T09:30:55.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:30:55 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T09:30:55.371 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:30:55 smithi005 ceph-mon[30350]: from='osd.3 [v2:172.21.15.5:6810/2221468725,v1:172.21.15.5:6811/2221468725]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-10T09:30:55.371 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:30:55 smithi005 ceph-mon[30350]: osdmap e23: 4 total, 3 up, 4 in 2024-09-10T09:30:55.371 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:30:55 smithi005 ceph-mon[30350]: from='osd.3 [v2:172.21.15.5:6810/2221468725,v1:172.21.15.5:6811/2221468725]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi005", "root=default"]}]: dispatch 2024-09-10T09:30:55.372 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:30:55 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T09:30:55.372 INFO:journalctl@ceph.osd.3.smithi005.stdout:Sep 10 09:30:55 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-3[51376]: 2024-09-10T09:30:55.080+0000 7fb365c00640 -1 osd.3 0 waiting for initial osdmap 2024-09-10T09:30:55.372 INFO:journalctl@ceph.osd.3.smithi005.stdout:Sep 10 09:30:55 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-3[51376]: 2024-09-10T09:30:55.086+0000 7fb360c00640 -1 osd.3 24 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T09:30:56.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:30:56 smithi159 ceph-mon[35513]: from='osd.3 [v2:172.21.15.5:6810/2221468725,v1:172.21.15.5:6811/2221468725]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi005", "root=default"]}]': finished 2024-09-10T09:30:56.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:30:56 smithi159 ceph-mon[35513]: osdmap e24: 4 total, 3 up, 4 in 2024-09-10T09:30:56.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:30:56 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T09:30:56.351 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:30:56 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T09:30:56.351 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:30:56 smithi159 ceph-mon[35513]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-10T09:30:56.351 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:30:56 smithi159 ceph-mon[35513]: osd.3 [v2:172.21.15.5:6810/2221468725,v1:172.21.15.5:6811/2221468725] boot 2024-09-10T09:30:56.351 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:30:56 smithi159 ceph-mon[35513]: osdmap e25: 4 total, 4 up, 4 in 2024-09-10T09:30:56.351 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:30:56 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T09:30:56.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:30:56 smithi102 ceph-mon[35442]: from='osd.3 [v2:172.21.15.5:6810/2221468725,v1:172.21.15.5:6811/2221468725]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi005", "root=default"]}]': finished 2024-09-10T09:30:56.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:30:56 smithi102 ceph-mon[35442]: osdmap e24: 4 total, 3 up, 4 in 2024-09-10T09:30:56.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:30:56 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T09:30:56.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:30:56 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T09:30:56.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:30:56 smithi102 ceph-mon[35442]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-10T09:30:56.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:30:56 smithi102 ceph-mon[35442]: osd.3 [v2:172.21.15.5:6810/2221468725,v1:172.21.15.5:6811/2221468725] boot 2024-09-10T09:30:56.385 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:30:56 smithi102 ceph-mon[35442]: osdmap e25: 4 total, 4 up, 4 in 2024-09-10T09:30:56.385 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:30:56 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T09:30:56.457 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-10T09:30:56.483 DEBUG:teuthology.orchestra.run.smithi102:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi102:vg_nvme/lv_3 2024-09-10T09:30:56.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:30:56 smithi005 ceph-mon[30350]: from='osd.3 [v2:172.21.15.5:6810/2221468725,v1:172.21.15.5:6811/2221468725]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi005", "root=default"]}]': finished 2024-09-10T09:30:56.557 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:30:56 smithi005 ceph-mon[30350]: osdmap e24: 4 total, 3 up, 4 in 2024-09-10T09:30:56.557 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:30:56 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T09:30:56.557 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:30:56 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T09:30:56.557 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:30:56 smithi005 ceph-mon[30350]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-10T09:30:56.557 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:30:56 smithi005 ceph-mon[30350]: osd.3 [v2:172.21.15.5:6810/2221468725,v1:172.21.15.5:6811/2221468725] boot 2024-09-10T09:30:56.557 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:30:56 smithi005 ceph-mon[30350]: osdmap e25: 4 total, 4 up, 4 in 2024-09-10T09:30:56.557 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:30:56 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T09:30:56.741 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T09:30:57.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:30:57 smithi102 ceph-mon[35442]: purged_snaps scrub starts 2024-09-10T09:30:57.135 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:30:57 smithi102 ceph-mon[35442]: purged_snaps scrub ok 2024-09-10T09:30:57.135 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:30:57 smithi102 ceph-mon[35442]: osdmap e26: 4 total, 4 up, 4 in 2024-09-10T09:30:57.215 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-10T09:30:57.181+0000 7f6f39c76640 1 -- 172.21.15.102:0/3517674421 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6f341078e0 msgr2=0x7f6f34109d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:30:57.216 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-10T09:30:57.181+0000 7f6f39c76640 1 --2- 172.21.15.102:0/3517674421 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6f341078e0 0x7f6f34109d30 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f6f2c009900 tx=0x7f6f2c02a410 comp rx=0 tx=0).stop 2024-09-10T09:30:57.216 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-10T09:30:57.182+0000 7f6f39c76640 1 -- 172.21.15.102:0/3517674421 shutdown_connections 2024-09-10T09:30:57.216 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-10T09:30:57.182+0000 7f6f39c76640 1 --2- 172.21.15.102:0/3517674421 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6f341078e0 0x7f6f34109d30 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:30:57.216 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-10T09:30:57.182+0000 7f6f39c76640 1 --2- 172.21.15.102:0/3517674421 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6f34104fb0 0x7f6f341073a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:30:57.216 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-10T09:30:57.182+0000 7f6f39c76640 1 --2- 172.21.15.102:0/3517674421 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6f34102680 0x7f6f34104a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:30:57.216 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-10T09:30:57.182+0000 7f6f39c76640 1 -- 172.21.15.102:0/3517674421 >> 172.21.15.102:0/3517674421 conn(0x7f6f340fc2a0 msgr2=0x7f6f340fe6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:30:57.216 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-10T09:30:57.182+0000 7f6f39c76640 1 -- 172.21.15.102:0/3517674421 shutdown_connections 2024-09-10T09:30:57.216 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-10T09:30:57.182+0000 7f6f39c76640 1 -- 172.21.15.102:0/3517674421 wait complete. 2024-09-10T09:30:57.216 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-10T09:30:57.182+0000 7f6f39c76640 1 Processor -- start 2024-09-10T09:30:57.216 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-10T09:30:57.182+0000 7f6f39c76640 1 -- start start 2024-09-10T09:30:57.216 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-10T09:30:57.183+0000 7f6f39c76640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6f34102680 0x7f6f34197f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:30:57.217 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-10T09:30:57.183+0000 7f6f39c76640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6f34104fb0 0x7f6f341984a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:30:57.217 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-10T09:30:57.183+0000 7f6f39c76640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6f341078e0 0x7f6f3419f520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:30:57.217 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-10T09:30:57.183+0000 7f6f39c76640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f6f3410d900 con 0x7f6f34102680 2024-09-10T09:30:57.217 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-10T09:30:57.183+0000 7f6f39c76640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f6f3410d780 con 0x7f6f341078e0 2024-09-10T09:30:57.217 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-10T09:30:57.183+0000 7f6f39c76640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f6f3410da80 con 0x7f6f34104fb0 2024-09-10T09:30:57.217 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-10T09:30:57.183+0000 7f6f33fff640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6f341078e0 0x7f6f3419f520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:30:57.217 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-10T09:30:57.183+0000 7f6f33fff640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6f341078e0 0x7f6f3419f520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:50324/0 (socket says 172.21.15.102:50324) 2024-09-10T09:30:57.217 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-10T09:30:57.183+0000 7f6f33fff640 1 -- 172.21.15.102:0/4129293495 learned_addr learned my addr 172.21.15.102:0/4129293495 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-10T09:30:57.217 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-10T09:30:57.183+0000 7f6f337fe640 1 --2- 172.21.15.102:0/4129293495 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6f34102680 0x7f6f34197f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:30:57.217 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-10T09:30:57.183+0000 7f6f32ffd640 1 --2- 172.21.15.102:0/4129293495 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6f34104fb0 0x7f6f341984a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:30:57.217 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-10T09:30:57.183+0000 7f6f33fff640 1 -- 172.21.15.102:0/4129293495 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6f34104fb0 msgr2=0x7f6f341984a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:30:57.217 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-10T09:30:57.183+0000 7f6f33fff640 1 --2- 172.21.15.102:0/4129293495 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6f34104fb0 0x7f6f341984a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:30:57.218 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-10T09:30:57.183+0000 7f6f33fff640 1 -- 172.21.15.102:0/4129293495 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6f34102680 msgr2=0x7f6f34197f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:30:57.218 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-10T09:30:57.183+0000 7f6f33fff640 1 --2- 172.21.15.102:0/4129293495 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6f34102680 0x7f6f34197f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:30:57.218 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-10T09:30:57.183+0000 7f6f33fff640 1 -- 172.21.15.102:0/4129293495 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f3419fb90 con 0x7f6f341078e0 2024-09-10T09:30:57.218 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-10T09:30:57.183+0000 7f6f337fe640 1 --2- 172.21.15.102:0/4129293495 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6f34102680 0x7f6f34197f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:30:57.218 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-10T09:30:57.183+0000 7f6f33fff640 1 --2- 172.21.15.102:0/4129293495 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6f341078e0 0x7f6f3419f520 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f6f2c0314e0 tx=0x7f6f2c0043c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:30:57.218 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-10T09:30:57.183+0000 7f6f30ff9640 1 -- 172.21.15.102:0/4129293495 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f2c03f030 con 0x7f6f341078e0 2024-09-10T09:30:57.218 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-10T09:30:57.183+0000 7f6f30ff9640 1 -- 172.21.15.102:0/4129293495 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6f2c003ce0 con 0x7f6f341078e0 2024-09-10T09:30:57.218 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-10T09:30:57.183+0000 7f6f39c76640 1 -- 172.21.15.102:0/4129293495 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f3419fe20 con 0x7f6f341078e0 2024-09-10T09:30:57.218 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-10T09:30:57.183+0000 7f6f30ff9640 1 -- 172.21.15.102:0/4129293495 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f2c035070 con 0x7f6f341078e0 2024-09-10T09:30:57.218 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-10T09:30:57.183+0000 7f6f39c76640 1 -- 172.21.15.102:0/4129293495 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f341a02f0 con 0x7f6f341078e0 2024-09-10T09:30:57.218 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-10T09:30:57.185+0000 7f6f39c76640 1 -- 172.21.15.102:0/4129293495 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6f340fdd90 con 0x7f6f341078e0 2024-09-10T09:30:57.218 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-10T09:30:57.186+0000 7f6f30ff9640 1 -- 172.21.15.102:0/4129293495 <== mon.2 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7f6f2c042030 con 0x7f6f341078e0 2024-09-10T09:30:57.218 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-10T09:30:57.186+0000 7f6f30ff9640 1 --2- 172.21.15.102:0/4129293495 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f6f1c0a2bb0 0x7f6f1c0a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:30:57.218 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-10T09:30:57.186+0000 7f6f30ff9640 1 -- 172.21.15.102:0/4129293495 <== mon.2 v2:172.21.15.102:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 3698+0+0 (secure 0 0 0) 0x7f6f2c0ee7e0 con 0x7f6f341078e0 2024-09-10T09:30:57.219 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-10T09:30:57.187+0000 7f6f337fe640 1 --2- 172.21.15.102:0/4129293495 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f6f1c0a2bb0 0x7f6f1c0a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:30:57.219 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-10T09:30:57.187+0000 7f6f337fe640 1 --2- 172.21.15.102:0/4129293495 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f6f1c0a2bb0 0x7f6f1c0a5070 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f6f24009fa0 tx=0x7f6f24009570 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:30:57.219 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-10T09:30:57.191+0000 7f6f30ff9640 1 -- 172.21.15.102:0/4129293495 <== mon.2 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f6f2c0bca00 con 0x7f6f341078e0 2024-09-10T09:30:57.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:30:57 smithi159 ceph-mon[35513]: purged_snaps scrub starts 2024-09-10T09:30:57.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:30:57 smithi159 ceph-mon[35513]: purged_snaps scrub ok 2024-09-10T09:30:57.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:30:57 smithi159 ceph-mon[35513]: osdmap e26: 4 total, 4 up, 4 in 2024-09-10T09:30:57.357 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:30:57 smithi005 ceph-mon[30350]: purged_snaps scrub starts 2024-09-10T09:30:57.357 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:30:57 smithi005 ceph-mon[30350]: purged_snaps scrub ok 2024-09-10T09:30:57.357 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:30:57 smithi005 ceph-mon[30350]: osdmap e26: 4 total, 4 up, 4 in 2024-09-10T09:30:57.367 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-10T09:30:57.366+0000 7f6f39c76640 1 -- 172.21.15.102:0/4129293495 --> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi102:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f6f34062630 con 0x7f6f1c0a2bb0 2024-09-10T09:30:58.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:30:58 smithi159 ceph-mon[35513]: from='client.24215 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi102:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T09:30:58.351 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:30:58 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T09:30:58.351 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:30:58 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T09:30:58.351 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:30:58 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:30:58.351 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:30:58 smithi159 ceph-mon[35513]: pgmap v102: 1 pgs: 1 remapped+peering; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-10T09:30:58.351 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:30:58 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:30:58.351 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:30:58 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:30:58.351 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:30:58 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:30:58.351 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:30:58 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:30:58.351 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:30:58 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:30:58.351 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:30:58 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:30:58.351 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:30:58 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:30:58.351 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:30:58 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:30:58.352 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:30:58 smithi159 ceph-mon[35513]: osdmap e27: 4 total, 4 up, 4 in 2024-09-10T09:30:58.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:30:58 smithi102 ceph-mon[35442]: from='client.24215 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi102:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T09:30:58.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:30:58 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T09:30:58.385 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:30:58 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T09:30:58.385 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:30:58 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:30:58.385 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:30:58 smithi102 ceph-mon[35442]: pgmap v102: 1 pgs: 1 remapped+peering; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-10T09:30:58.385 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:30:58 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:30:58.385 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:30:58 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:30:58.385 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:30:58 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:30:58.385 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:30:58 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:30:58.385 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:30:58 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:30:58.385 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:30:58 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:30:58.385 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:30:58 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:30:58.385 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:30:58 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:30:58.385 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:30:58 smithi102 ceph-mon[35442]: osdmap e27: 4 total, 4 up, 4 in 2024-09-10T09:30:58.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:30:58 smithi005 ceph-mon[30350]: from='client.24215 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi102:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T09:30:58.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:30:58 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T09:30:58.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:30:58 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T09:30:58.557 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:30:58 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:30:58.557 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:30:58 smithi005 ceph-mon[30350]: pgmap v102: 1 pgs: 1 remapped+peering; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-10T09:30:58.557 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:30:58 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:30:58.557 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:30:58 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:30:58.557 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:30:58 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:30:58.557 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:30:58 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:30:58.557 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:30:58 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:30:58.557 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:30:58 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:30:58.557 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:30:58 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:30:58.557 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:30:58 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:30:58.558 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:30:58 smithi005 ceph-mon[30350]: osdmap e27: 4 total, 4 up, 4 in 2024-09-10T09:30:59.352 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:30:59 smithi159 ceph-mon[35513]: Detected new or changed devices on smithi005 2024-09-10T09:30:59.352 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:30:59 smithi159 ceph-mon[35513]: Adjusting osd_memory_target on smithi005 to 8547M 2024-09-10T09:30:59.352 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:30:59 smithi159 ceph-mon[35513]: osdmap e28: 4 total, 4 up, 4 in 2024-09-10T09:30:59.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:30:59 smithi102 ceph-mon[35442]: Detected new or changed devices on smithi005 2024-09-10T09:30:59.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:30:59 smithi102 ceph-mon[35442]: Adjusting osd_memory_target on smithi005 to 8547M 2024-09-10T09:30:59.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:30:59 smithi102 ceph-mon[35442]: osdmap e28: 4 total, 4 up, 4 in 2024-09-10T09:30:59.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:30:59 smithi005 ceph-mon[30350]: Detected new or changed devices on smithi005 2024-09-10T09:30:59.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:30:59 smithi005 ceph-mon[30350]: Adjusting osd_memory_target on smithi005 to 8547M 2024-09-10T09:30:59.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:30:59 smithi005 ceph-mon[30350]: osdmap e28: 4 total, 4 up, 4 in 2024-09-10T09:31:00.293 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:31:00 smithi102 ceph-mon[35442]: pgmap v105: 1 pgs: 1 remapped+peering; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-10T09:31:00.293 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:31:00 smithi102 ceph-mon[35442]: from='client.? 172.21.15.102:0/2385393595' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f2e7314d-7938-4668-9011-45c175f66b48"}]: dispatch 2024-09-10T09:31:00.294 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:31:00 smithi102 ceph-mon[35442]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f2e7314d-7938-4668-9011-45c175f66b48"}]: dispatch 2024-09-10T09:31:00.294 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:31:00 smithi102 ceph-mon[35442]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f2e7314d-7938-4668-9011-45c175f66b48"}]': finished 2024-09-10T09:31:00.294 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:31:00 smithi102 ceph-mon[35442]: osdmap e29: 5 total, 4 up, 5 in 2024-09-10T09:31:00.294 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:31:00 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T09:31:00.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:31:00 smithi159 ceph-mon[35513]: pgmap v105: 1 pgs: 1 remapped+peering; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-10T09:31:00.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:31:00 smithi159 ceph-mon[35513]: from='client.? 172.21.15.102:0/2385393595' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f2e7314d-7938-4668-9011-45c175f66b48"}]: dispatch 2024-09-10T09:31:00.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:31:00 smithi159 ceph-mon[35513]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f2e7314d-7938-4668-9011-45c175f66b48"}]: dispatch 2024-09-10T09:31:00.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:31:00 smithi159 ceph-mon[35513]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f2e7314d-7938-4668-9011-45c175f66b48"}]': finished 2024-09-10T09:31:00.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:31:00 smithi159 ceph-mon[35513]: osdmap e29: 5 total, 4 up, 5 in 2024-09-10T09:31:00.351 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:31:00 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T09:31:00.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:31:00 smithi005 ceph-mon[30350]: pgmap v105: 1 pgs: 1 remapped+peering; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-10T09:31:00.557 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:31:00 smithi005 ceph-mon[30350]: from='client.? 172.21.15.102:0/2385393595' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f2e7314d-7938-4668-9011-45c175f66b48"}]: dispatch 2024-09-10T09:31:00.557 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:31:00 smithi005 ceph-mon[30350]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f2e7314d-7938-4668-9011-45c175f66b48"}]: dispatch 2024-09-10T09:31:00.557 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:31:00 smithi005 ceph-mon[30350]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f2e7314d-7938-4668-9011-45c175f66b48"}]': finished 2024-09-10T09:31:00.557 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:31:00 smithi005 ceph-mon[30350]: osdmap e29: 5 total, 4 up, 5 in 2024-09-10T09:31:00.557 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:31:00 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T09:31:01.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:31:01 smithi159 ceph-mon[35513]: from='client.? 172.21.15.102:0/1892759529' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T09:31:01.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:31:01 smithi102 ceph-mon[35442]: from='client.? 172.21.15.102:0/1892759529' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T09:31:01.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:31:01 smithi005 ceph-mon[30350]: from='client.? 172.21.15.102:0/1892759529' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T09:31:02.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:31:02 smithi159 ceph-mon[35513]: pgmap v107: 1 pgs: 1 remapped+peering; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-10T09:31:02.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:31:02 smithi102 ceph-mon[35442]: pgmap v107: 1 pgs: 1 remapped+peering; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-10T09:31:02.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:31:02 smithi005 ceph-mon[30350]: pgmap v107: 1 pgs: 1 remapped+peering; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-10T09:31:03.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:31:03 smithi005 ceph-mon[30350]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-10T09:31:03.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:31:03 smithi159 ceph-mon[35513]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-10T09:31:03.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:31:03 smithi102 ceph-mon[35442]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-10T09:31:05.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:31:05 smithi005 ceph-mon[30350]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 78 KiB/s, 0 objects/s recovering 2024-09-10T09:31:05.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:31:05 smithi159 ceph-mon[35513]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 78 KiB/s, 0 objects/s recovering 2024-09-10T09:31:05.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:31:05 smithi102 ceph-mon[35442]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 78 KiB/s, 0 objects/s recovering 2024-09-10T09:31:07.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:31:07 smithi102 ceph-mon[35442]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 68 KiB/s, 0 objects/s recovering 2024-09-10T09:31:07.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:31:07 smithi005 ceph-mon[30350]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 68 KiB/s, 0 objects/s recovering 2024-09-10T09:31:07.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:31:07 smithi159 ceph-mon[35513]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 68 KiB/s, 0 objects/s recovering 2024-09-10T09:31:08.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:31:08 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-10T09:31:08.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:31:08 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:31:08.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:31:08 smithi102 ceph-mon[35442]: Deploying daemon osd.4 on smithi102 2024-09-10T09:31:08.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:31:08 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-10T09:31:08.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:31:08 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:31:08.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:31:08 smithi005 ceph-mon[30350]: Deploying daemon osd.4 on smithi102 2024-09-10T09:31:08.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:31:08 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-10T09:31:08.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:31:08 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:31:08.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:31:08 smithi159 ceph-mon[35513]: Deploying daemon osd.4 on smithi102 2024-09-10T09:31:09.800 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:31:09 smithi102 ceph-mon[35442]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-10T09:31:09.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:31:09 smithi005 ceph-mon[30350]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-10T09:31:09.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:31:09 smithi159 ceph-mon[35513]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-10T09:31:11.777 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:31:11 smithi102 ceph-mon[35442]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 49 KiB/s, 0 objects/s recovering 2024-09-10T09:31:11.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:31:11 smithi005 ceph-mon[30350]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 49 KiB/s, 0 objects/s recovering 2024-09-10T09:31:11.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:31:11 smithi159 ceph-mon[35513]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 49 KiB/s, 0 objects/s recovering 2024-09-10T09:31:13.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:31:13 smithi102 ceph-mon[35442]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-10T09:31:13.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:31:13 smithi005 ceph-mon[30350]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-10T09:31:13.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:31:13 smithi159 ceph-mon[35513]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-10T09:31:14.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:31:14 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:31:14.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:31:14 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:31:14.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:31:14 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:31:15.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:31:14 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:31:15.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:31:14 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:31:15.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:31:14 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:31:15.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:31:14 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:31:15.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:31:14 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:31:15.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:31:14 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:31:16.001 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:31:15 smithi159 ceph-mon[35513]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-10T09:31:16.021 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:31:15 smithi102 ceph-mon[35442]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-10T09:31:16.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:31:15 smithi005 ceph-mon[30350]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-10T09:31:18.198 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-10T09:31:18.197+0000 7f6f30ff9640 1 -- 172.21.15.102:0/4129293495 <== mgr.14150 v2:172.21.15.5:6800/2707348081 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f6f34062630 con 0x7f6f1c0a2bb0 2024-09-10T09:31:18.198 INFO:teuthology.orchestra.run.smithi102.stdout:Created osd(s) 4 on host 'smithi102' 2024-09-10T09:31:18.200 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-10T09:31:18.199+0000 7f6f39c76640 1 -- 172.21.15.102:0/4129293495 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f6f1c0a2bb0 msgr2=0x7f6f1c0a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:31:18.200 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-10T09:31:18.199+0000 7f6f39c76640 1 --2- 172.21.15.102:0/4129293495 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f6f1c0a2bb0 0x7f6f1c0a5070 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f6f24009fa0 tx=0x7f6f24009570 comp rx=0 tx=0).stop 2024-09-10T09:31:18.200 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-10T09:31:18.199+0000 7f6f39c76640 1 -- 172.21.15.102:0/4129293495 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6f341078e0 msgr2=0x7f6f3419f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:31:18.200 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-10T09:31:18.199+0000 7f6f39c76640 1 --2- 172.21.15.102:0/4129293495 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6f341078e0 0x7f6f3419f520 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f6f2c0314e0 tx=0x7f6f2c0043c0 comp rx=0 tx=0).stop 2024-09-10T09:31:18.200 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-10T09:31:18.199+0000 7f6f39c76640 1 -- 172.21.15.102:0/4129293495 shutdown_connections 2024-09-10T09:31:18.201 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-10T09:31:18.199+0000 7f6f39c76640 1 --2- 172.21.15.102:0/4129293495 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f6f1c0a2bb0 0x7f6f1c0a5070 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:31:18.201 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-10T09:31:18.199+0000 7f6f39c76640 1 --2- 172.21.15.102:0/4129293495 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6f341078e0 0x7f6f3419f520 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:31:18.201 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-10T09:31:18.199+0000 7f6f39c76640 1 --2- 172.21.15.102:0/4129293495 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6f34104fb0 0x7f6f341984a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:31:18.201 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-10T09:31:18.199+0000 7f6f39c76640 1 --2- 172.21.15.102:0/4129293495 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6f34102680 0x7f6f34197f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:31:18.201 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-10T09:31:18.199+0000 7f6f39c76640 1 -- 172.21.15.102:0/4129293495 >> 172.21.15.102:0/4129293495 conn(0x7f6f340fc2a0 msgr2=0x7f6f34102f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:31:18.201 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-10T09:31:18.199+0000 7f6f39c76640 1 -- 172.21.15.102:0/4129293495 shutdown_connections 2024-09-10T09:31:18.201 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-10T09:31:18.199+0000 7f6f39c76640 1 -- 172.21.15.102:0/4129293495 wait complete. 2024-09-10T09:31:18.527 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:31:18 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:31:18.527 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:31:18 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:31:18.527 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:31:18 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:31:18.528 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:31:18 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:31:18.528 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:31:18 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:31:18.528 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:31:18 smithi102 ceph-mon[35442]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-10T09:31:18.528 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:31:18 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:31:18.528 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:31:18 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:31:18.528 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:31:18 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:31:18.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:31:18 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:31:18.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:31:18 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:31:18.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:31:18 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:31:18.807 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:31:18 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:31:18.807 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:31:18 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:31:18.807 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:31:18 smithi005 ceph-mon[30350]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-10T09:31:18.807 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:31:18 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:31:18.807 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:31:18 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:31:18.807 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:31:18 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:31:18.839 DEBUG:teuthology.orchestra.run.smithi102:osd.4> sudo journalctl -f -n 0 -u ceph-bade4b50-6f56-11ef-bcea-c7b262605968@osd.4.service 2024-09-10T09:31:18.841 INFO:tasks.cephadm:Deploying osd.5 on smithi159 with /dev/vg_nvme/lv_3... 2024-09-10T09:31:18.841 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_3 2024-09-10T09:31:18.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:31:18 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:31:18.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:31:18 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:31:18.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:31:18 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:31:18.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:31:18 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:31:18.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:31:18 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:31:18.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:31:18 smithi159 ceph-mon[35513]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-10T09:31:18.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:31:18 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:31:18.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:31:18 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:31:18.851 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:31:18 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:31:19.093 INFO:teuthology.orchestra.run.smithi159.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.c/config 2024-09-10T09:31:19.808 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:31:19 smithi005 ceph-mon[30350]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-10T09:31:19.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:31:19 smithi159 ceph-mon[35513]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-10T09:31:19.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:31:19 smithi102 ceph-mon[35442]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-10T09:31:20.511 INFO:journalctl@ceph.osd.4.smithi102.stdout:Sep 10 09:31:20 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-4[46359]: 2024-09-10T09:31:20.438+0000 7f004f238640 -1 osd.4 0 log_to_monitors true 2024-09-10T09:31:20.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:31:20 smithi005 ceph-mon[30350]: from='osd.4 [v2:172.21.15.102:6808/594465753,v1:172.21.15.102:6809/594465753]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-10T09:31:20.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:31:20 smithi005 ceph-mon[30350]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-10T09:31:20.849 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:31:20 smithi159 ceph-mon[35513]: from='osd.4 [v2:172.21.15.102:6808/594465753,v1:172.21.15.102:6809/594465753]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-10T09:31:20.849 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:31:20 smithi159 ceph-mon[35513]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-10T09:31:20.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:31:20 smithi102 ceph-mon[35442]: from='osd.4 [v2:172.21.15.102:6808/594465753,v1:172.21.15.102:6809/594465753]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-10T09:31:20.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:31:20 smithi102 ceph-mon[35442]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-10T09:31:21.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:31:21 smithi102 ceph-mon[35442]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-10T09:31:21.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:31:21 smithi102 ceph-mon[35442]: osdmap e30: 5 total, 4 up, 5 in 2024-09-10T09:31:21.635 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:31:21 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T09:31:21.635 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:31:21 smithi102 ceph-mon[35442]: from='osd.4 [v2:172.21.15.102:6808/594465753,v1:172.21.15.102:6809/594465753]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]: dispatch 2024-09-10T09:31:21.635 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:31:21 smithi102 ceph-mon[35442]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]: dispatch 2024-09-10T09:31:21.635 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:31:21 smithi102 ceph-mon[35442]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-10T09:31:21.780 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:31:21 smithi159 ceph-mon[35513]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-10T09:31:21.780 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:31:21 smithi159 ceph-mon[35513]: osdmap e30: 5 total, 4 up, 5 in 2024-09-10T09:31:21.780 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:31:21 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T09:31:21.780 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:31:21 smithi159 ceph-mon[35513]: from='osd.4 [v2:172.21.15.102:6808/594465753,v1:172.21.15.102:6809/594465753]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]: dispatch 2024-09-10T09:31:21.780 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:31:21 smithi159 ceph-mon[35513]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]: dispatch 2024-09-10T09:31:21.780 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:31:21 smithi159 ceph-mon[35513]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-10T09:31:21.782 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-10T09:31:21.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:31:21 smithi005 ceph-mon[30350]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-10T09:31:21.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:31:21 smithi005 ceph-mon[30350]: osdmap e30: 5 total, 4 up, 5 in 2024-09-10T09:31:21.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:31:21 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T09:31:21.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:31:21 smithi005 ceph-mon[30350]: from='osd.4 [v2:172.21.15.102:6808/594465753,v1:172.21.15.102:6809/594465753]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]: dispatch 2024-09-10T09:31:21.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:31:21 smithi005 ceph-mon[30350]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]: dispatch 2024-09-10T09:31:21.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:31:21 smithi005 ceph-mon[30350]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-10T09:31:21.809 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi159:vg_nvme/lv_3 2024-09-10T09:31:22.094 INFO:teuthology.orchestra.run.smithi159.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.c/config 2024-09-10T09:31:22.384 INFO:journalctl@ceph.osd.4.smithi102.stdout:Sep 10 09:31:21 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-4[46359]: 2024-09-10T09:31:21.936+0000 7f004b1bb640 -1 osd.4 0 waiting for initial osdmap 2024-09-10T09:31:22.384 INFO:journalctl@ceph.osd.4.smithi102.stdout:Sep 10 09:31:21 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-4[46359]: 2024-09-10T09:31:21.942+0000 7f0042fdb640 -1 osd.4 31 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T09:31:22.650 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:31:22.624+0000 7f56ef902640 1 -- 172.21.15.159:0/3044675268 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f56e8100490 msgr2=0x7f56e81028a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:31:22.650 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:31:22.624+0000 7f56ef902640 1 --2- 172.21.15.159:0/3044675268 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f56e8100490 0x7f56e81028a0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f56e4009900 tx=0x7f56e402a410 comp rx=0 tx=0).stop 2024-09-10T09:31:22.650 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:31:22.624+0000 7f56ef902640 1 -- 172.21.15.159:0/3044675268 shutdown_connections 2024-09-10T09:31:22.650 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:31:22.624+0000 7f56ef902640 1 --2- 172.21.15.159:0/3044675268 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f56e8105790 0x7f56e8105c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:31:22.650 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:31:22.624+0000 7f56ef902640 1 --2- 172.21.15.159:0/3044675268 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f56e8102de0 0x7f56e8105250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:31:22.650 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:31:22.624+0000 7f56ef902640 1 --2- 172.21.15.159:0/3044675268 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f56e8100490 0x7f56e81028a0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:31:22.650 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:31:22.624+0000 7f56ef902640 1 -- 172.21.15.159:0/3044675268 >> 172.21.15.159:0/3044675268 conn(0x7f56e80fc2a0 msgr2=0x7f56e80fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:31:22.651 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:31:22.625+0000 7f56ef902640 1 -- 172.21.15.159:0/3044675268 shutdown_connections 2024-09-10T09:31:22.651 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:31:22.625+0000 7f56ef902640 1 -- 172.21.15.159:0/3044675268 wait complete. 2024-09-10T09:31:22.651 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:31:22.625+0000 7f56ef902640 1 Processor -- start 2024-09-10T09:31:22.651 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:31:22.625+0000 7f56ef902640 1 -- start start 2024-09-10T09:31:22.651 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:31:22.625+0000 7f56ef902640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f56e8100490 0x7f56e819e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:31:22.651 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:31:22.625+0000 7f56ef902640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f56e8102de0 0x7f56e819ea30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:31:22.651 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:31:22.625+0000 7f56ef902640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f56e8105790 0x7f56e81a5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:31:22.651 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:31:22.625+0000 7f56ef902640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f56e8113e70 con 0x7f56e8100490 2024-09-10T09:31:22.651 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:31:22.625+0000 7f56ef902640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f56e8113cf0 con 0x7f56e8105790 2024-09-10T09:31:22.651 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:31:22.625+0000 7f56ef902640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f56e8113ff0 con 0x7f56e8102de0 2024-09-10T09:31:22.651 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:31:22.625+0000 7f56ece9d640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f56e8102de0 0x7f56e819ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:31:22.651 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:31:22.625+0000 7f56ece9d640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f56e8102de0 0x7f56e819ea30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:53268/0 (socket says 172.21.15.159:53268) 2024-09-10T09:31:22.651 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:31:22.625+0000 7f56ece9d640 1 -- 172.21.15.159:0/1742782977 learned_addr learned my addr 172.21.15.159:0/1742782977 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-09-10T09:31:22.652 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:31:22.626+0000 7f56ed69e640 1 --2- 172.21.15.159:0/1742782977 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f56e8100490 0x7f56e819e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:31:22.652 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:31:22.626+0000 7f56ece9d640 1 -- 172.21.15.159:0/1742782977 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f56e8105790 msgr2=0x7f56e81a5ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:31:22.652 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:31:22.626+0000 7f56ece9d640 1 --2- 172.21.15.159:0/1742782977 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f56e8105790 0x7f56e81a5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:31:22.652 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:31:22.626+0000 7f56ece9d640 1 -- 172.21.15.159:0/1742782977 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f56e8100490 msgr2=0x7f56e819e4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:31:22.652 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:31:22.626+0000 7f56ece9d640 1 --2- 172.21.15.159:0/1742782977 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f56e8100490 0x7f56e819e4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:31:22.652 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:31:22.626+0000 7f56ece9d640 1 -- 172.21.15.159:0/1742782977 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56e81a6120 con 0x7f56e8102de0 2024-09-10T09:31:22.652 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:31:22.626+0000 7f56ece9d640 1 --2- 172.21.15.159:0/1742782977 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f56e8102de0 0x7f56e819ea30 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f56d800d950 tx=0x7f56d800de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:31:22.652 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:31:22.626+0000 7f56de7fc640 1 -- 172.21.15.159:0/1742782977 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56d8017070 con 0x7f56e8102de0 2024-09-10T09:31:22.652 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:31:22.626+0000 7f56de7fc640 1 -- 172.21.15.159:0/1742782977 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f56d8009c30 con 0x7f56e8102de0 2024-09-10T09:31:22.652 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:31:22.626+0000 7f56ef902640 1 -- 172.21.15.159:0/1742782977 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56e81a6410 con 0x7f56e8102de0 2024-09-10T09:31:22.652 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:31:22.626+0000 7f56de7fc640 1 -- 172.21.15.159:0/1742782977 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56d800f6b0 con 0x7f56e8102de0 2024-09-10T09:31:22.652 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:31:22.626+0000 7f56ef902640 1 -- 172.21.15.159:0/1742782977 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56e81a6950 con 0x7f56e8102de0 2024-09-10T09:31:22.653 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:31:22.627+0000 7f56ef902640 1 -- 172.21.15.159:0/1742782977 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f56e80fdd90 con 0x7f56e8102de0 2024-09-10T09:31:22.653 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:31:22.628+0000 7f56de7fc640 1 -- 172.21.15.159:0/1742782977 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7f56d80109d0 con 0x7f56e8102de0 2024-09-10T09:31:22.653 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:31:22.629+0000 7f56de7fc640 1 --2- 172.21.15.159:0/1742782977 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f56d40a2bb0 0x7f56d40a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:31:22.653 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:31:22.629+0000 7f56de7fc640 1 -- 172.21.15.159:0/1742782977 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 3821+0+0 (secure 0 0 0) 0x7f56d80cdb60 con 0x7f56e8102de0 2024-09-10T09:31:22.653 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:31:22.629+0000 7f56ed69e640 1 --2- 172.21.15.159:0/1742782977 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f56d40a2bb0 0x7f56d40a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:31:22.653 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:31:22.630+0000 7f56ed69e640 1 --2- 172.21.15.159:0/1742782977 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f56d40a2bb0 0x7f56d40a5070 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f56e4000c00 tx=0x7f56e40315d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:31:22.653 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:31:22.632+0000 7f56de7fc640 1 -- 172.21.15.159:0/1742782977 <== mon.1 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f56d809bc50 con 0x7f56e8102de0 2024-09-10T09:31:22.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:31:22 smithi005 ceph-mon[30350]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]': finished 2024-09-10T09:31:22.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:31:22 smithi005 ceph-mon[30350]: osdmap e31: 5 total, 4 up, 5 in 2024-09-10T09:31:22.807 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:31:22 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T09:31:22.807 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:31:22 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T09:31:22.807 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:31:22 smithi005 ceph-mon[30350]: Detected new or changed devices on smithi102 2024-09-10T09:31:22.807 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:31:22 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:31:22.807 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:31:22 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:31:22.807 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:31:22 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:31:22.807 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:31:22 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:31:22.807 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:31:22 smithi005 ceph-mon[30350]: Adjusting osd_memory_target on smithi102 to 8544M 2024-09-10T09:31:22.807 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:31:22 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:31:22.807 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:31:22 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:31:22.807 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:31:22 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:31:22.807 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:31:22 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:31:22.807 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:31:22 smithi005 ceph-mon[30350]: from='osd.4 ' entity='osd.4' 2024-09-10T09:31:22.809 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:31:22.808+0000 7f56ef902640 1 -- 172.21.15.159:0/1742782977 --> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi159:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f56e8062630 con 0x7f56d40a2bb0 2024-09-10T09:31:22.810 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:31:22 smithi159 ceph-mon[35513]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]': finished 2024-09-10T09:31:22.810 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:31:22 smithi159 ceph-mon[35513]: osdmap e31: 5 total, 4 up, 5 in 2024-09-10T09:31:22.810 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:31:22 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T09:31:22.810 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:31:22 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T09:31:22.810 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:31:22 smithi159 ceph-mon[35513]: Detected new or changed devices on smithi102 2024-09-10T09:31:22.810 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:31:22 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:31:22.810 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:31:22 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:31:22.810 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:31:22 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:31:22.810 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:31:22 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:31:22.810 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:31:22 smithi159 ceph-mon[35513]: Adjusting osd_memory_target on smithi102 to 8544M 2024-09-10T09:31:22.810 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:31:22 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:31:22.810 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:31:22 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:31:22.811 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:31:22 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:31:22.811 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:31:22 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:31:22.811 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:31:22 smithi159 ceph-mon[35513]: from='osd.4 ' entity='osd.4' 2024-09-10T09:31:22.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:31:22 smithi102 ceph-mon[35442]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]': finished 2024-09-10T09:31:22.885 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:31:22 smithi102 ceph-mon[35442]: osdmap e31: 5 total, 4 up, 5 in 2024-09-10T09:31:22.885 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:31:22 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T09:31:22.885 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:31:22 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T09:31:22.885 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:31:22 smithi102 ceph-mon[35442]: Detected new or changed devices on smithi102 2024-09-10T09:31:22.885 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:31:22 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:31:22.885 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:31:22 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:31:22.885 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:31:22 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:31:22.885 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:31:22 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:31:22.885 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:31:22 smithi102 ceph-mon[35442]: Adjusting osd_memory_target on smithi102 to 8544M 2024-09-10T09:31:22.885 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:31:22 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:31:22.885 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:31:22 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:31:22.886 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:31:22 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:31:22.886 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:31:22 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:31:22.886 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:31:22 smithi102 ceph-mon[35442]: from='osd.4 ' entity='osd.4' 2024-09-10T09:31:23.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:31:23 smithi005 ceph-mon[30350]: purged_snaps scrub starts 2024-09-10T09:31:23.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:31:23 smithi005 ceph-mon[30350]: purged_snaps scrub ok 2024-09-10T09:31:23.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:31:23 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T09:31:23.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:31:23 smithi005 ceph-mon[30350]: from='client.24241 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi159:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T09:31:23.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:31:23 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T09:31:23.807 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:31:23 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T09:31:23.807 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:31:23 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:31:23.807 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:31:23 smithi005 ceph-mon[30350]: osd.4 [v2:172.21.15.102:6808/594465753,v1:172.21.15.102:6809/594465753] boot 2024-09-10T09:31:23.807 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:31:23 smithi005 ceph-mon[30350]: osdmap e32: 5 total, 5 up, 5 in 2024-09-10T09:31:23.807 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:31:23 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T09:31:23.807 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:31:23 smithi005 ceph-mon[30350]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-10T09:31:23.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:31:23 smithi159 ceph-mon[35513]: purged_snaps scrub starts 2024-09-10T09:31:23.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:31:23 smithi159 ceph-mon[35513]: purged_snaps scrub ok 2024-09-10T09:31:23.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:31:23 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T09:31:23.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:31:23 smithi159 ceph-mon[35513]: from='client.24241 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi159:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T09:31:23.851 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:31:23 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T09:31:23.851 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:31:23 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T09:31:23.851 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:31:23 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:31:23.851 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:31:23 smithi159 ceph-mon[35513]: osd.4 [v2:172.21.15.102:6808/594465753,v1:172.21.15.102:6809/594465753] boot 2024-09-10T09:31:23.851 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:31:23 smithi159 ceph-mon[35513]: osdmap e32: 5 total, 5 up, 5 in 2024-09-10T09:31:23.851 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:31:23 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T09:31:23.851 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:31:23 smithi159 ceph-mon[35513]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-10T09:31:23.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:31:23 smithi102 ceph-mon[35442]: purged_snaps scrub starts 2024-09-10T09:31:23.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:31:23 smithi102 ceph-mon[35442]: purged_snaps scrub ok 2024-09-10T09:31:23.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:31:23 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T09:31:23.885 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:31:23 smithi102 ceph-mon[35442]: from='client.24241 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi159:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T09:31:23.885 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:31:23 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T09:31:23.885 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:31:23 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T09:31:23.885 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:31:23 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:31:23.885 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:31:23 smithi102 ceph-mon[35442]: osd.4 [v2:172.21.15.102:6808/594465753,v1:172.21.15.102:6809/594465753] boot 2024-09-10T09:31:23.885 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:31:23 smithi102 ceph-mon[35442]: osdmap e32: 5 total, 5 up, 5 in 2024-09-10T09:31:23.885 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:31:23 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T09:31:23.885 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:31:23 smithi102 ceph-mon[35442]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-10T09:31:25.306 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:31:24 smithi005 ceph-mon[30350]: osdmap e33: 5 total, 5 up, 5 in 2024-09-10T09:31:25.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:31:24 smithi159 ceph-mon[35513]: osdmap e33: 5 total, 5 up, 5 in 2024-09-10T09:31:25.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:31:24 smithi102 ceph-mon[35442]: osdmap e33: 5 total, 5 up, 5 in 2024-09-10T09:31:26.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:31:25 smithi159 ceph-mon[35513]: from='client.? 172.21.15.159:0/2545116690' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0c0366d3-658b-4bf0-ba17-aff28a02e948"}]: dispatch 2024-09-10T09:31:26.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:31:25 smithi159 ceph-mon[35513]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0c0366d3-658b-4bf0-ba17-aff28a02e948"}]: dispatch 2024-09-10T09:31:26.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:31:25 smithi159 ceph-mon[35513]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0c0366d3-658b-4bf0-ba17-aff28a02e948"}]': finished 2024-09-10T09:31:26.101 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:31:25 smithi159 ceph-mon[35513]: osdmap e34: 6 total, 5 up, 6 in 2024-09-10T09:31:26.101 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:31:25 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T09:31:26.101 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:31:25 smithi159 ceph-mon[35513]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-10T09:31:26.101 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:31:25 smithi159 ceph-mon[35513]: from='client.? 172.21.15.159:0/3901939194' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T09:31:26.306 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:31:25 smithi005 ceph-mon[30350]: from='client.? 172.21.15.159:0/2545116690' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0c0366d3-658b-4bf0-ba17-aff28a02e948"}]: dispatch 2024-09-10T09:31:26.306 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:31:25 smithi005 ceph-mon[30350]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0c0366d3-658b-4bf0-ba17-aff28a02e948"}]: dispatch 2024-09-10T09:31:26.307 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:31:25 smithi005 ceph-mon[30350]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0c0366d3-658b-4bf0-ba17-aff28a02e948"}]': finished 2024-09-10T09:31:26.307 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:31:25 smithi005 ceph-mon[30350]: osdmap e34: 6 total, 5 up, 6 in 2024-09-10T09:31:26.307 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:31:25 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T09:31:26.307 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:31:25 smithi005 ceph-mon[30350]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-10T09:31:26.307 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:31:25 smithi005 ceph-mon[30350]: from='client.? 172.21.15.159:0/3901939194' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T09:31:26.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:31:25 smithi102 ceph-mon[35442]: from='client.? 172.21.15.159:0/2545116690' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0c0366d3-658b-4bf0-ba17-aff28a02e948"}]: dispatch 2024-09-10T09:31:26.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:31:25 smithi102 ceph-mon[35442]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0c0366d3-658b-4bf0-ba17-aff28a02e948"}]: dispatch 2024-09-10T09:31:26.385 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:31:25 smithi102 ceph-mon[35442]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0c0366d3-658b-4bf0-ba17-aff28a02e948"}]': finished 2024-09-10T09:31:26.385 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:31:25 smithi102 ceph-mon[35442]: osdmap e34: 6 total, 5 up, 6 in 2024-09-10T09:31:26.385 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:31:25 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T09:31:26.385 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:31:25 smithi102 ceph-mon[35442]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-10T09:31:26.385 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:31:25 smithi102 ceph-mon[35442]: from='client.? 172.21.15.159:0/3901939194' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T09:31:27.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:31:27 smithi005 ceph-mon[30350]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-10T09:31:27.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:31:27 smithi159 ceph-mon[35513]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-10T09:31:27.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:31:27 smithi102 ceph-mon[35442]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-10T09:31:29.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:31:29 smithi005 ceph-mon[30350]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-10T09:31:29.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:31:29 smithi159 ceph-mon[35513]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-10T09:31:29.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:31:29 smithi102 ceph-mon[35442]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-10T09:31:31.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:31:31 smithi005 ceph-mon[30350]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-10T09:31:31.847 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:31:31 smithi159 ceph-mon[35513]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-10T09:31:31.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:31:31 smithi102 ceph-mon[35442]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-10T09:31:33.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:31:33 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-10T09:31:33.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:31:33 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:31:33.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:31:33 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-10T09:31:33.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:31:33 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:31:33.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:31:33 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-10T09:31:33.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:31:33 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:31:34.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:31:34 smithi005 ceph-mon[30350]: Deploying daemon osd.5 on smithi159 2024-09-10T09:31:34.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:31:34 smithi005 ceph-mon[30350]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-10T09:31:34.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:31:34 smithi159 ceph-mon[35513]: Deploying daemon osd.5 on smithi159 2024-09-10T09:31:34.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:31:34 smithi159 ceph-mon[35513]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-10T09:31:34.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:31:34 smithi102 ceph-mon[35442]: Deploying daemon osd.5 on smithi159 2024-09-10T09:31:34.635 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:31:34 smithi102 ceph-mon[35442]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-10T09:31:35.736 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:31:35 smithi159 ceph-mon[35513]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-10T09:31:35.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:31:35 smithi005 ceph-mon[30350]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-10T09:31:35.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:31:35 smithi102 ceph-mon[35442]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-10T09:31:37.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:31:37 smithi005 ceph-mon[30350]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-10T09:31:37.845 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:31:37 smithi159 ceph-mon[35513]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-10T09:31:37.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:31:37 smithi102 ceph-mon[35442]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-10T09:31:39.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:31:39 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:31:39.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:31:39 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:31:39.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:31:39 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:31:39.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:31:39 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:31:39.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:31:39 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:31:39.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:31:39 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:31:39.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:31:39 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:31:39.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:31:39 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:31:39.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:31:39 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:31:40.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:31:40 smithi005 ceph-mon[30350]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-10T09:31:40.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:31:40 smithi159 ceph-mon[35513]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-10T09:31:40.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:31:40 smithi102 ceph-mon[35442]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-10T09:31:41.762 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:31:41 smithi159 ceph-mon[35513]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-10T09:31:41.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:31:41 smithi005 ceph-mon[30350]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-10T09:31:41.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:31:41 smithi102 ceph-mon[35442]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-10T09:31:43.483 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:31:43 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:31:43.483 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:31:43 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:31:43.483 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:31:43 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:31:43.483 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:31:43 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:31:43.483 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:31:43 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:31:43.523 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:31:43.521+0000 7f56de7fc640 1 -- 172.21.15.159:0/1742782977 <== mgr.14150 v2:172.21.15.5:6800/2707348081 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f56e8062630 con 0x7f56d40a2bb0 2024-09-10T09:31:43.524 INFO:teuthology.orchestra.run.smithi159.stdout:Created osd(s) 5 on host 'smithi159' 2024-09-10T09:31:43.526 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:31:43.524+0000 7f56ef902640 1 -- 172.21.15.159:0/1742782977 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f56d40a2bb0 msgr2=0x7f56d40a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:31:43.526 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:31:43.524+0000 7f56ef902640 1 --2- 172.21.15.159:0/1742782977 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f56d40a2bb0 0x7f56d40a5070 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f56e4000c00 tx=0x7f56e40315d0 comp rx=0 tx=0).stop 2024-09-10T09:31:43.526 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:31:43.524+0000 7f56ef902640 1 -- 172.21.15.159:0/1742782977 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f56e8102de0 msgr2=0x7f56e819ea30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:31:43.526 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:31:43.524+0000 7f56ef902640 1 --2- 172.21.15.159:0/1742782977 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f56e8102de0 0x7f56e819ea30 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f56d800d950 tx=0x7f56d800de10 comp rx=0 tx=0).stop 2024-09-10T09:31:43.526 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:31:43.524+0000 7f56ef902640 1 -- 172.21.15.159:0/1742782977 shutdown_connections 2024-09-10T09:31:43.526 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:31:43.524+0000 7f56ef902640 1 --2- 172.21.15.159:0/1742782977 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f56d40a2bb0 0x7f56d40a5070 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:31:43.526 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:31:43.524+0000 7f56ef902640 1 --2- 172.21.15.159:0/1742782977 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f56e8105790 0x7f56e81a5ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:31:43.526 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:31:43.524+0000 7f56ef902640 1 --2- 172.21.15.159:0/1742782977 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f56e8102de0 0x7f56e819ea30 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:31:43.526 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:31:43.524+0000 7f56ef902640 1 --2- 172.21.15.159:0/1742782977 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f56e8100490 0x7f56e819e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:31:43.527 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:31:43.524+0000 7f56ef902640 1 -- 172.21.15.159:0/1742782977 >> 172.21.15.159:0/1742782977 conn(0x7f56e80fc2a0 msgr2=0x7f56e8103840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:31:43.527 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:31:43.524+0000 7f56ef902640 1 -- 172.21.15.159:0/1742782977 shutdown_connections 2024-09-10T09:31:43.527 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:31:43.524+0000 7f56ef902640 1 -- 172.21.15.159:0/1742782977 wait complete. 2024-09-10T09:31:43.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:31:43 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:31:43.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:31:43 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:31:43.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:31:43 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:31:43.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:31:43 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:31:43.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:31:43 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:31:43.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:31:43 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:31:43.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:31:43 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:31:43.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:31:43 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:31:43.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:31:43 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:31:43.635 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:31:43 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:31:44.207 DEBUG:teuthology.orchestra.run.smithi159:osd.5> sudo journalctl -f -n 0 -u ceph-bade4b50-6f56-11ef-bcea-c7b262605968@osd.5.service 2024-09-10T09:31:44.209 INFO:tasks.cephadm:Deploying osd.6 on smithi005 with /dev/vg_nvme/lv_2... 2024-09-10T09:31:44.209 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_2 2024-09-10T09:31:44.514 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:31:44 smithi159 ceph-mon[35513]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-10T09:31:44.514 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:31:44 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:31:44.514 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:31:44 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:31:44.514 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:31:44 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:31:44.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:31:44 smithi005 ceph-mon[30350]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-10T09:31:44.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:31:44 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:31:44.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:31:44 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:31:44.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:31:44 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:31:44.591 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:31:44.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:31:44 smithi102 ceph-mon[35442]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-10T09:31:44.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:31:44 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:31:44.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:31:44 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:31:44.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:31:44 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:31:45.707 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:31:45 smithi159 ceph-mon[35513]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-10T09:31:45.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:31:45 smithi005 ceph-mon[30350]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-10T09:31:45.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:31:45 smithi102 ceph-mon[35442]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-10T09:31:46.101 INFO:journalctl@ceph.osd.5.smithi159.stdout:Sep 10 09:31:45 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-5[46947]: 2024-09-10T09:31:45.708+0000 7f87f8929640 -1 osd.5 0 log_to_monitors true 2024-09-10T09:31:46.793 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:31:46 smithi159 ceph-mon[35513]: from='osd.5 [v2:172.21.15.159:6808/3230280402,v1:172.21.15.159:6809/3230280402]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-10T09:31:46.793 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:31:46 smithi159 ceph-mon[35513]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-10T09:31:46.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:31:46 smithi005 ceph-mon[30350]: from='osd.5 [v2:172.21.15.159:6808/3230280402,v1:172.21.15.159:6809/3230280402]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-10T09:31:46.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:31:46 smithi005 ceph-mon[30350]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-10T09:31:46.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:31:46 smithi102 ceph-mon[35442]: from='osd.5 [v2:172.21.15.159:6808/3230280402,v1:172.21.15.159:6809/3230280402]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-10T09:31:46.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:31:46 smithi102 ceph-mon[35442]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-10T09:31:47.798 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:31:47 smithi005 ceph-mon[30350]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-10T09:31:47.798 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:31:47 smithi005 ceph-mon[30350]: osdmap e35: 6 total, 5 up, 6 in 2024-09-10T09:31:47.799 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:31:47 smithi005 ceph-mon[30350]: from='osd.5 [v2:172.21.15.159:6808/3230280402,v1:172.21.15.159:6809/3230280402]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]: dispatch 2024-09-10T09:31:47.799 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:31:47 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T09:31:47.799 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:31:47 smithi005 ceph-mon[30350]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]: dispatch 2024-09-10T09:31:47.799 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:31:47 smithi005 ceph-mon[30350]: Detected new or changed devices on smithi159 2024-09-10T09:31:47.799 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:31:47 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:31:47.799 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:31:47 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:31:47.799 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:31:47 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:31:47.799 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:31:47 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:31:47.799 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:31:47 smithi005 ceph-mon[30350]: Adjusting osd_memory_target on smithi159 to 8544M 2024-09-10T09:31:47.799 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:31:47 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:31:47.799 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:31:47 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:31:47.799 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:31:47 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:31:47.799 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:31:47 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:31:47.799 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:31:47 smithi005 ceph-mon[30350]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-10T09:31:47.850 INFO:journalctl@ceph.osd.5.smithi159.stdout:Sep 10 09:31:47 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-5[46947]: 2024-09-10T09:31:47.553+0000 7f87f48ac640 -1 osd.5 0 waiting for initial osdmap 2024-09-10T09:31:47.850 INFO:journalctl@ceph.osd.5.smithi159.stdout:Sep 10 09:31:47 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-5[46947]: 2024-09-10T09:31:47.560+0000 7f87f06d4640 -1 osd.5 36 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T09:31:47.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:31:47 smithi159 ceph-mon[35513]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-10T09:31:47.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:31:47 smithi159 ceph-mon[35513]: osdmap e35: 6 total, 5 up, 6 in 2024-09-10T09:31:47.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:31:47 smithi159 ceph-mon[35513]: from='osd.5 [v2:172.21.15.159:6808/3230280402,v1:172.21.15.159:6809/3230280402]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]: dispatch 2024-09-10T09:31:47.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:31:47 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T09:31:47.851 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:31:47 smithi159 ceph-mon[35513]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]: dispatch 2024-09-10T09:31:47.851 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:31:47 smithi159 ceph-mon[35513]: Detected new or changed devices on smithi159 2024-09-10T09:31:47.851 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:31:47 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:31:47.851 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:31:47 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:31:47.851 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:31:47 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:31:47.851 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:31:47 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:31:47.851 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:31:47 smithi159 ceph-mon[35513]: Adjusting osd_memory_target on smithi159 to 8544M 2024-09-10T09:31:47.851 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:31:47 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:31:47.851 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:31:47 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:31:47.851 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:31:47 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:31:47.851 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:31:47 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:31:47.851 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:31:47 smithi159 ceph-mon[35513]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-10T09:31:47.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:31:47 smithi102 ceph-mon[35442]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-10T09:31:47.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:31:47 smithi102 ceph-mon[35442]: osdmap e35: 6 total, 5 up, 6 in 2024-09-10T09:31:47.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:31:47 smithi102 ceph-mon[35442]: from='osd.5 [v2:172.21.15.159:6808/3230280402,v1:172.21.15.159:6809/3230280402]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]: dispatch 2024-09-10T09:31:47.885 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:31:47 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T09:31:47.885 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:31:47 smithi102 ceph-mon[35442]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]: dispatch 2024-09-10T09:31:47.885 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:31:47 smithi102 ceph-mon[35442]: Detected new or changed devices on smithi159 2024-09-10T09:31:47.885 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:31:47 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:31:47.885 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:31:47 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:31:47.885 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:31:47 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:31:47.885 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:31:47 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:31:47.885 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:31:47 smithi102 ceph-mon[35442]: Adjusting osd_memory_target on smithi159 to 8544M 2024-09-10T09:31:47.885 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:31:47 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:31:47.885 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:31:47 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:31:47.885 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:31:47 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:31:47.885 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:31:47 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:31:47.885 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:31:47 smithi102 ceph-mon[35442]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-10T09:31:47.886 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-10T09:31:47.921 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi005:vg_nvme/lv_2 2024-09-10T09:31:48.297 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:31:48.704 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:31:48 smithi005 ceph-mon[30350]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]': finished 2024-09-10T09:31:48.704 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:31:48 smithi005 ceph-mon[30350]: osdmap e36: 6 total, 5 up, 6 in 2024-09-10T09:31:48.704 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:31:48 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T09:31:48.704 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:31:48 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T09:31:48.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:31:48 smithi159 ceph-mon[35513]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]': finished 2024-09-10T09:31:48.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:31:48 smithi159 ceph-mon[35513]: osdmap e36: 6 total, 5 up, 6 in 2024-09-10T09:31:48.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:31:48 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T09:31:48.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:31:48 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T09:31:48.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:31:48 smithi102 ceph-mon[35442]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]': finished 2024-09-10T09:31:48.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:31:48 smithi102 ceph-mon[35442]: osdmap e36: 6 total, 5 up, 6 in 2024-09-10T09:31:48.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:31:48 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T09:31:48.885 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:31:48 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T09:31:48.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:31:48.889+0000 7ff800600640 1 -- 172.21.15.5:0/1365079741 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff7f8077df0 msgr2=0x7ff7f8078250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:31:48.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:31:48.889+0000 7ff800600640 1 --2- 172.21.15.5:0/1365079741 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff7f8077df0 0x7ff7f8078250 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7ff7e4009900 tx=0x7ff7e402a410 comp rx=0 tx=0).stop 2024-09-10T09:31:48.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:31:48.889+0000 7ff800600640 1 -- 172.21.15.5:0/1365079741 shutdown_connections 2024-09-10T09:31:48.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:31:48.889+0000 7ff800600640 1 --2- 172.21.15.5:0/1365079741 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff7f8078790 0x7ff7f8079030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:31:48.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:31:48.889+0000 7ff800600640 1 --2- 172.21.15.5:0/1365079741 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff7f8077df0 0x7ff7f8078250 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:31:48.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:31:48.889+0000 7ff800600640 1 --2- 172.21.15.5:0/1365079741 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff7f8076ba0 0x7ff7f8076fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:31:48.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:31:48.889+0000 7ff800600640 1 -- 172.21.15.5:0/1365079741 >> 172.21.15.5:0/1365079741 conn(0x7ff7f80ff5b0 msgr2=0x7ff7f81019d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:31:48.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:31:48.890+0000 7ff800600640 1 -- 172.21.15.5:0/1365079741 shutdown_connections 2024-09-10T09:31:48.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:31:48.890+0000 7ff800600640 1 -- 172.21.15.5:0/1365079741 wait complete. 2024-09-10T09:31:48.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:31:48.890+0000 7ff800600640 1 Processor -- start 2024-09-10T09:31:48.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:31:48.891+0000 7ff800600640 1 -- start start 2024-09-10T09:31:48.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:31:48.891+0000 7ff800600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff7f8076ba0 0x7ff7f819e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:31:48.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:31:48.891+0000 7ff800600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff7f8077df0 0x7ff7f819ea00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:31:48.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:31:48.891+0000 7ff800600640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff7f8078790 0x7ff7f81a5a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:31:48.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:31:48.891+0000 7ff800600640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff7f8113f10 con 0x7ff7f8076ba0 2024-09-10T09:31:48.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:31:48.891+0000 7ff800600640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7ff7f8113d90 con 0x7ff7f8077df0 2024-09-10T09:31:48.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:31:48.891+0000 7ff800600640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7ff7f8114090 con 0x7ff7f8078790 2024-09-10T09:31:48.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:31:48.891+0000 7ff7fdc00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff7f8076ba0 0x7ff7f819e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:31:48.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:31:48.891+0000 7ff7fdc00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff7f8076ba0 0x7ff7f819e4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36900/0 (socket says 172.21.15.5:36900) 2024-09-10T09:31:48.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:31:48.891+0000 7ff7fdc00640 1 -- 172.21.15.5:0/4218010693 learned_addr learned my addr 172.21.15.5:0/4218010693 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:31:48.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:31:48.891+0000 7ff7fd200640 1 --2- 172.21.15.5:0/4218010693 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff7f8077df0 0x7ff7f819ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:31:48.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:31:48.891+0000 7ff7fe600640 1 --2- 172.21.15.5:0/4218010693 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff7f8078790 0x7ff7f81a5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:31:48.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:31:48.892+0000 7ff7fdc00640 1 -- 172.21.15.5:0/4218010693 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff7f8078790 msgr2=0x7ff7f81a5a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:31:48.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:31:48.892+0000 7ff7fdc00640 1 --2- 172.21.15.5:0/4218010693 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff7f8078790 0x7ff7f81a5a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:31:48.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:31:48.892+0000 7ff7fdc00640 1 -- 172.21.15.5:0/4218010693 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff7f8077df0 msgr2=0x7ff7f819ea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:31:48.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:31:48.892+0000 7ff7fdc00640 1 --2- 172.21.15.5:0/4218010693 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff7f8077df0 0x7ff7f819ea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:31:48.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:31:48.892+0000 7ff7fdc00640 1 -- 172.21.15.5:0/4218010693 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7f81a60f0 con 0x7ff7f8076ba0 2024-09-10T09:31:48.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:31:48.892+0000 7ff7fdc00640 1 --2- 172.21.15.5:0/4218010693 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff7f8076ba0 0x7ff7f819e4c0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7ff7ec00e9f0 tx=0x7ff7ec00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:31:48.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:31:48.892+0000 7ff7f6000640 1 -- 172.21.15.5:0/4218010693 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7ec00cd80 con 0x7ff7f8076ba0 2024-09-10T09:31:48.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:31:48.892+0000 7ff800600640 1 -- 172.21.15.5:0/4218010693 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7f81a63e0 con 0x7ff7f8076ba0 2024-09-10T09:31:48.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:31:48.892+0000 7ff7f6000640 1 -- 172.21.15.5:0/4218010693 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff7ec010910 con 0x7ff7f8076ba0 2024-09-10T09:31:48.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:31:48.892+0000 7ff800600640 1 -- 172.21.15.5:0/4218010693 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7f81a6940 con 0x7ff7f8076ba0 2024-09-10T09:31:48.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:31:48.893+0000 7ff7f6000640 1 -- 172.21.15.5:0/4218010693 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7ec0188d0 con 0x7ff7f8076ba0 2024-09-10T09:31:48.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:31:48.893+0000 7ff800600640 1 -- 172.21.15.5:0/4218010693 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff7f8110780 con 0x7ff7f8076ba0 2024-09-10T09:31:48.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:31:48.895+0000 7ff7f6000640 1 -- 172.21.15.5:0/4218010693 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7ff7ec010ac0 con 0x7ff7f8076ba0 2024-09-10T09:31:48.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:31:48.897+0000 7ff7f6000640 1 --2- 172.21.15.5:0/4218010693 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7ff7e00a2bb0 0x7ff7e00a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:31:48.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:31:48.897+0000 7ff7f6000640 1 -- 172.21.15.5:0/4218010693 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 4566+0+0 (secure 0 0 0) 0x7ff7ec0ce520 con 0x7ff7f8076ba0 2024-09-10T09:31:48.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:31:48.897+0000 7ff7fd200640 1 --2- 172.21.15.5:0/4218010693 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7ff7e00a2bb0 0x7ff7e00a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:31:48.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:31:48.897+0000 7ff7fd200640 1 --2- 172.21.15.5:0/4218010693 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7ff7e00a2bb0 0x7ff7e00a5070 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7ff7e4031440 tx=0x7ff7e40315b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:31:48.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:31:48.901+0000 7ff7f6000640 1 -- 172.21.15.5:0/4218010693 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff7ec09bc30 con 0x7ff7f8076ba0 2024-09-10T09:31:49.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:31:49.099+0000 7ff800600640 1 -- 172.21.15.5:0/4218010693 --> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi005:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7ff7f8062630 con 0x7ff7e00a2bb0 2024-09-10T09:31:49.840 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:31:49 smithi005 ceph-mon[30350]: purged_snaps scrub starts 2024-09-10T09:31:49.840 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:31:49 smithi005 ceph-mon[30350]: purged_snaps scrub ok 2024-09-10T09:31:49.840 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:31:49 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T09:31:49.840 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:31:49 smithi005 ceph-mon[30350]: osd.5 [v2:172.21.15.159:6808/3230280402,v1:172.21.15.159:6809/3230280402] boot 2024-09-10T09:31:49.840 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:31:49 smithi005 ceph-mon[30350]: osdmap e37: 6 total, 6 up, 6 in 2024-09-10T09:31:49.840 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:31:49 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T09:31:49.841 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:31:49 smithi005 ceph-mon[30350]: osdmap e38: 6 total, 6 up, 6 in 2024-09-10T09:31:49.841 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:31:49 smithi005 ceph-mon[30350]: from='client.14322 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi005:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T09:31:49.841 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:31:49 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T09:31:49.841 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:31:49 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T09:31:49.841 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:31:49 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:31:49.841 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:31:49 smithi005 ceph-mon[30350]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-10T09:31:49.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:31:49 smithi159 ceph-mon[35513]: purged_snaps scrub starts 2024-09-10T09:31:49.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:31:49 smithi159 ceph-mon[35513]: purged_snaps scrub ok 2024-09-10T09:31:49.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:31:49 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T09:31:49.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:31:49 smithi159 ceph-mon[35513]: osd.5 [v2:172.21.15.159:6808/3230280402,v1:172.21.15.159:6809/3230280402] boot 2024-09-10T09:31:49.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:31:49 smithi159 ceph-mon[35513]: osdmap e37: 6 total, 6 up, 6 in 2024-09-10T09:31:49.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:31:49 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T09:31:49.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:31:49 smithi159 ceph-mon[35513]: osdmap e38: 6 total, 6 up, 6 in 2024-09-10T09:31:49.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:31:49 smithi159 ceph-mon[35513]: from='client.14322 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi005:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T09:31:49.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:31:49 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T09:31:49.851 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:31:49 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T09:31:49.851 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:31:49 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:31:49.851 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:31:49 smithi159 ceph-mon[35513]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-10T09:31:49.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:31:49 smithi102 ceph-mon[35442]: purged_snaps scrub starts 2024-09-10T09:31:49.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:31:49 smithi102 ceph-mon[35442]: purged_snaps scrub ok 2024-09-10T09:31:49.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:31:49 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T09:31:49.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:31:49 smithi102 ceph-mon[35442]: osd.5 [v2:172.21.15.159:6808/3230280402,v1:172.21.15.159:6809/3230280402] boot 2024-09-10T09:31:49.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:31:49 smithi102 ceph-mon[35442]: osdmap e37: 6 total, 6 up, 6 in 2024-09-10T09:31:49.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:31:49 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T09:31:49.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:31:49 smithi102 ceph-mon[35442]: osdmap e38: 6 total, 6 up, 6 in 2024-09-10T09:31:49.885 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:31:49 smithi102 ceph-mon[35442]: from='client.14322 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi005:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T09:31:49.885 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:31:49 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T09:31:49.885 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:31:49 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T09:31:49.885 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:31:49 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:31:49.885 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:31:49 smithi102 ceph-mon[35442]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-09-10T09:31:51.055 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:31:50 smithi005 ceph-mon[30350]: osdmap e39: 6 total, 6 up, 6 in 2024-09-10T09:31:51.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:31:50 smithi159 ceph-mon[35513]: osdmap e39: 6 total, 6 up, 6 in 2024-09-10T09:31:51.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:31:50 smithi102 ceph-mon[35442]: osdmap e39: 6 total, 6 up, 6 in 2024-09-10T09:31:52.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:31:51 smithi005 ceph-mon[30350]: osdmap e40: 6 total, 6 up, 6 in 2024-09-10T09:31:52.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:31:51 smithi005 ceph-mon[30350]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-09-10T09:31:52.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:31:51 smithi159 ceph-mon[35513]: osdmap e40: 6 total, 6 up, 6 in 2024-09-10T09:31:52.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:31:51 smithi159 ceph-mon[35513]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-09-10T09:31:52.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:31:51 smithi102 ceph-mon[35442]: osdmap e40: 6 total, 6 up, 6 in 2024-09-10T09:31:52.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:31:51 smithi102 ceph-mon[35442]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-09-10T09:31:53.306 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:31:52 smithi005 ceph-mon[30350]: osdmap e41: 6 total, 6 up, 6 in 2024-09-10T09:31:53.306 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:31:52 smithi005 ceph-mon[30350]: from='client.? 172.21.15.5:0/4279414030' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "682670a6-51d5-46e8-af6e-dc2d8c1cf407"}]: dispatch 2024-09-10T09:31:53.306 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:31:52 smithi005 ceph-mon[30350]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "682670a6-51d5-46e8-af6e-dc2d8c1cf407"}]: dispatch 2024-09-10T09:31:53.306 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:31:52 smithi005 ceph-mon[30350]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "682670a6-51d5-46e8-af6e-dc2d8c1cf407"}]': finished 2024-09-10T09:31:53.306 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:31:52 smithi005 ceph-mon[30350]: osdmap e42: 7 total, 6 up, 7 in 2024-09-10T09:31:53.307 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:31:52 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T09:31:53.307 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:31:52 smithi005 ceph-mon[30350]: from='client.? 172.21.15.5:0/1843242155' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T09:31:53.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:31:52 smithi159 ceph-mon[35513]: osdmap e41: 6 total, 6 up, 6 in 2024-09-10T09:31:53.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:31:52 smithi159 ceph-mon[35513]: from='client.? 172.21.15.5:0/4279414030' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "682670a6-51d5-46e8-af6e-dc2d8c1cf407"}]: dispatch 2024-09-10T09:31:53.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:31:52 smithi159 ceph-mon[35513]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "682670a6-51d5-46e8-af6e-dc2d8c1cf407"}]: dispatch 2024-09-10T09:31:53.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:31:52 smithi159 ceph-mon[35513]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "682670a6-51d5-46e8-af6e-dc2d8c1cf407"}]': finished 2024-09-10T09:31:53.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:31:52 smithi159 ceph-mon[35513]: osdmap e42: 7 total, 6 up, 7 in 2024-09-10T09:31:53.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:31:52 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T09:31:53.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:31:52 smithi159 ceph-mon[35513]: from='client.? 172.21.15.5:0/1843242155' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T09:31:53.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:31:52 smithi102 ceph-mon[35442]: osdmap e41: 6 total, 6 up, 6 in 2024-09-10T09:31:53.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:31:52 smithi102 ceph-mon[35442]: from='client.? 172.21.15.5:0/4279414030' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "682670a6-51d5-46e8-af6e-dc2d8c1cf407"}]: dispatch 2024-09-10T09:31:53.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:31:52 smithi102 ceph-mon[35442]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "682670a6-51d5-46e8-af6e-dc2d8c1cf407"}]: dispatch 2024-09-10T09:31:53.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:31:52 smithi102 ceph-mon[35442]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "682670a6-51d5-46e8-af6e-dc2d8c1cf407"}]': finished 2024-09-10T09:31:53.385 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:31:52 smithi102 ceph-mon[35442]: osdmap e42: 7 total, 6 up, 7 in 2024-09-10T09:31:53.385 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:31:52 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T09:31:53.385 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:31:52 smithi102 ceph-mon[35442]: from='client.? 172.21.15.5:0/1843242155' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T09:31:54.306 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:31:53 smithi005 ceph-mon[30350]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-09-10T09:31:54.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:31:53 smithi159 ceph-mon[35513]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-09-10T09:31:54.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:31:53 smithi102 ceph-mon[35442]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-09-10T09:31:55.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:31:55 smithi005 ceph-mon[30350]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 104 KiB/s, 0 objects/s recovering 2024-09-10T09:31:55.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:31:55 smithi159 ceph-mon[35513]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 104 KiB/s, 0 objects/s recovering 2024-09-10T09:31:55.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:31:55 smithi102 ceph-mon[35442]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 104 KiB/s, 0 objects/s recovering 2024-09-10T09:31:57.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:31:57 smithi005 ceph-mon[30350]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 88 KiB/s, 0 objects/s recovering 2024-09-10T09:31:57.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:31:57 smithi159 ceph-mon[35513]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 88 KiB/s, 0 objects/s recovering 2024-09-10T09:31:57.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:31:57 smithi102 ceph-mon[35442]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 88 KiB/s, 0 objects/s recovering 2024-09-10T09:31:59.654 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:31:59 smithi005 ceph-mon[30350]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-10T09:31:59.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:31:59 smithi159 ceph-mon[35513]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-10T09:31:59.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:31:59 smithi102 ceph-mon[35442]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-10T09:32:01.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:32:01 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-09-10T09:32:01.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:32:01 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:32:01.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:32:01 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-09-10T09:32:01.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:32:01 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:32:01.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:32:01 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-09-10T09:32:01.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:32:01 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:32:02.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:32:02 smithi005 ceph-mon[30350]: Deploying daemon osd.6 on smithi005 2024-09-10T09:32:02.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:32:02 smithi005 ceph-mon[30350]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-09-10T09:32:02.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:32:02 smithi102 ceph-mon[35442]: Deploying daemon osd.6 on smithi005 2024-09-10T09:32:02.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:32:02 smithi102 ceph-mon[35442]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-09-10T09:32:02.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:32:02 smithi159 ceph-mon[35513]: Deploying daemon osd.6 on smithi005 2024-09-10T09:32:02.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:32:02 smithi159 ceph-mon[35513]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-09-10T09:32:03.731 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:32:03 smithi005 ceph-mon[30350]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 51 KiB/s, 0 objects/s recovering 2024-09-10T09:32:03.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:32:03 smithi159 ceph-mon[35513]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 51 KiB/s, 0 objects/s recovering 2024-09-10T09:32:03.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:32:03 smithi102 ceph-mon[35442]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 51 KiB/s, 0 objects/s recovering 2024-09-10T09:32:05.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:32:05 smithi005 ceph-mon[30350]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-09-10T09:32:05.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:32:05 smithi159 ceph-mon[35513]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-09-10T09:32:05.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:32:05 smithi102 ceph-mon[35442]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-09-10T09:32:07.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:32:07 smithi005 ceph-mon[30350]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-09-10T09:32:07.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:32:07 smithi159 ceph-mon[35513]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-09-10T09:32:07.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:32:07 smithi102 ceph-mon[35442]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-09-10T09:32:09.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:32:09 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:32:09.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:32:09 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:32:09.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:32:09 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:32:09.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:32:09 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:32:09.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:32:09 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:32:09.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:32:09 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:32:09.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:32:09 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:32:09.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:32:09 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:32:09.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:32:09 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:32:10.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:32:10 smithi005 ceph-mon[30350]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-09-10T09:32:10.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:32:10 smithi159 ceph-mon[35513]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-09-10T09:32:10.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:32:10 smithi102 ceph-mon[35442]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-09-10T09:32:11.804 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:32:11 smithi005 ceph-mon[30350]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-09-10T09:32:11.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:32:11 smithi159 ceph-mon[35513]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-09-10T09:32:11.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:32:11 smithi102 ceph-mon[35442]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-09-10T09:32:13.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:32:13 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:32:13.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:32:13 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:32:13.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:32:13 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:32:13.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:32:13 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:32:13.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:32:13 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:32:13.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:32:13 smithi005 ceph-mon[30350]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-09-10T09:32:13.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:32:13 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:32:13.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:32:13 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:32:13.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:32:13 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:32:13.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:32:13 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:32:13.851 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:32:13 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:32:13.851 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:32:13 smithi159 ceph-mon[35513]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-09-10T09:32:13.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:32:13 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:32:13.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:32:13 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:32:13.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:32:13 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:32:13.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:32:13 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:32:13.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:32:13 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:32:13.885 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:32:13 smithi102 ceph-mon[35442]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-09-10T09:32:14.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:32:14.163+0000 7ff7f6000640 1 -- 172.21.15.5:0/4218010693 <== mgr.14150 v2:172.21.15.5:6800/2707348081 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7ff7f8062630 con 0x7ff7e00a2bb0 2024-09-10T09:32:14.164 INFO:teuthology.orchestra.run.smithi005.stdout:Created osd(s) 6 on host 'smithi005' 2024-09-10T09:32:14.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:32:14.165+0000 7ff800600640 1 -- 172.21.15.5:0/4218010693 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7ff7e00a2bb0 msgr2=0x7ff7e00a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:32:14.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:32:14.165+0000 7ff800600640 1 --2- 172.21.15.5:0/4218010693 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7ff7e00a2bb0 0x7ff7e00a5070 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7ff7e4031440 tx=0x7ff7e40315b0 comp rx=0 tx=0).stop 2024-09-10T09:32:14.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:32:14.165+0000 7ff800600640 1 -- 172.21.15.5:0/4218010693 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff7f8076ba0 msgr2=0x7ff7f819e4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:32:14.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:32:14.165+0000 7ff800600640 1 --2- 172.21.15.5:0/4218010693 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff7f8076ba0 0x7ff7f819e4c0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7ff7ec00e9f0 tx=0x7ff7ec00eeb0 comp rx=0 tx=0).stop 2024-09-10T09:32:14.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:32:14.165+0000 7ff800600640 1 -- 172.21.15.5:0/4218010693 shutdown_connections 2024-09-10T09:32:14.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:32:14.165+0000 7ff800600640 1 --2- 172.21.15.5:0/4218010693 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7ff7e00a2bb0 0x7ff7e00a5070 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:32:14.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:32:14.165+0000 7ff800600640 1 --2- 172.21.15.5:0/4218010693 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff7f8078790 0x7ff7f81a5a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:32:14.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:32:14.165+0000 7ff800600640 1 --2- 172.21.15.5:0/4218010693 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff7f8077df0 0x7ff7f819ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:32:14.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:32:14.165+0000 7ff800600640 1 --2- 172.21.15.5:0/4218010693 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff7f8076ba0 0x7ff7f819e4c0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:32:14.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:32:14.165+0000 7ff800600640 1 -- 172.21.15.5:0/4218010693 >> 172.21.15.5:0/4218010693 conn(0x7ff7f80ff5b0 msgr2=0x7ff7f8101040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:32:14.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:32:14.166+0000 7ff800600640 1 -- 172.21.15.5:0/4218010693 shutdown_connections 2024-09-10T09:32:14.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:32:14.166+0000 7ff800600640 1 -- 172.21.15.5:0/4218010693 wait complete. 2024-09-10T09:32:14.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:32:14 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:32:14.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:32:14 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:32:14.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:32:14 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:32:14.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:32:14 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:32:14.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:32:14 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:32:14.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:32:14 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:32:14.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:32:14 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:32:14.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:32:14 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:32:14.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:32:14 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:32:14.925 DEBUG:teuthology.orchestra.run.smithi005:osd.6> sudo journalctl -f -n 0 -u ceph-bade4b50-6f56-11ef-bcea-c7b262605968@osd.6.service 2024-09-10T09:32:14.929 INFO:tasks.cephadm:Deploying osd.7 on smithi102 with /dev/vg_nvme/lv_2... 2024-09-10T09:32:14.929 DEBUG:teuthology.orchestra.run.smithi102:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_2 2024-09-10T09:32:15.181 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T09:32:15.556 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 10 09:32:15 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-6[58305]: 2024-09-10T09:32:15.187+0000 7f732fa1b640 -1 osd.6 0 log_to_monitors true 2024-09-10T09:32:15.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:32:15 smithi159 ceph-mon[35513]: from='osd.6 [v2:172.21.15.5:6818/1907651140,v1:172.21.15.5:6819/1907651140]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-10T09:32:15.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:32:15 smithi159 ceph-mon[35513]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-09-10T09:32:15.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:32:15 smithi102 ceph-mon[35442]: from='osd.6 [v2:172.21.15.5:6818/1907651140,v1:172.21.15.5:6819/1907651140]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-10T09:32:15.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:32:15 smithi102 ceph-mon[35442]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-09-10T09:32:16.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:32:15 smithi005 ceph-mon[30350]: from='osd.6 [v2:172.21.15.5:6818/1907651140,v1:172.21.15.5:6819/1907651140]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-10T09:32:16.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:32:15 smithi005 ceph-mon[30350]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-09-10T09:32:16.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:32:16 smithi102 ceph-mon[35442]: from='osd.6 [v2:172.21.15.5:6818/1907651140,v1:172.21.15.5:6819/1907651140]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-10T09:32:16.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:32:16 smithi102 ceph-mon[35442]: osdmap e43: 7 total, 6 up, 7 in 2024-09-10T09:32:16.635 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:32:16 smithi102 ceph-mon[35442]: from='osd.6 [v2:172.21.15.5:6818/1907651140,v1:172.21.15.5:6819/1907651140]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi005", "root=default"]}]: dispatch 2024-09-10T09:32:16.635 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:32:16 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T09:32:16.777 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:32:16 smithi005 ceph-mon[30350]: from='osd.6 [v2:172.21.15.5:6818/1907651140,v1:172.21.15.5:6819/1907651140]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-10T09:32:16.777 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:32:16 smithi005 ceph-mon[30350]: osdmap e43: 7 total, 6 up, 7 in 2024-09-10T09:32:16.777 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:32:16 smithi005 ceph-mon[30350]: from='osd.6 [v2:172.21.15.5:6818/1907651140,v1:172.21.15.5:6819/1907651140]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi005", "root=default"]}]: dispatch 2024-09-10T09:32:16.777 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:32:16 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T09:32:17.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:32:16 smithi159 ceph-mon[35513]: from='osd.6 [v2:172.21.15.5:6818/1907651140,v1:172.21.15.5:6819/1907651140]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-10T09:32:17.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:32:16 smithi159 ceph-mon[35513]: osdmap e43: 7 total, 6 up, 7 in 2024-09-10T09:32:17.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:32:16 smithi159 ceph-mon[35513]: from='osd.6 [v2:172.21.15.5:6818/1907651140,v1:172.21.15.5:6819/1907651140]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi005", "root=default"]}]: dispatch 2024-09-10T09:32:17.101 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:32:16 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T09:32:17.306 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 10 09:32:17 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-6[58305]: 2024-09-10T09:32:17.098+0000 7f732a600640 -1 osd.6 0 waiting for initial osdmap 2024-09-10T09:32:17.306 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 10 09:32:17 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-6[58305]: 2024-09-10T09:32:17.126+0000 7f7320600640 -1 osd.6 44 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T09:32:17.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:32:17 smithi102 ceph-mon[35442]: from='osd.6 [v2:172.21.15.5:6818/1907651140,v1:172.21.15.5:6819/1907651140]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi005", "root=default"]}]': finished 2024-09-10T09:32:17.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:32:17 smithi102 ceph-mon[35442]: osdmap e44: 7 total, 6 up, 7 in 2024-09-10T09:32:17.635 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:32:17 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T09:32:17.635 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:32:17 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T09:32:17.635 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:32:17 smithi102 ceph-mon[35442]: from='osd.6 [v2:172.21.15.5:6818/1907651140,v1:172.21.15.5:6819/1907651140]' entity='osd.6' 2024-09-10T09:32:17.635 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:32:17 smithi102 ceph-mon[35442]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-09-10T09:32:17.835 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-10T09:32:17.861 DEBUG:teuthology.orchestra.run.smithi102:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi102:vg_nvme/lv_2 2024-09-10T09:32:17.937 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:32:17 smithi005 ceph-mon[30350]: from='osd.6 [v2:172.21.15.5:6818/1907651140,v1:172.21.15.5:6819/1907651140]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi005", "root=default"]}]': finished 2024-09-10T09:32:17.937 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:32:17 smithi005 ceph-mon[30350]: osdmap e44: 7 total, 6 up, 7 in 2024-09-10T09:32:17.937 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:32:17 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T09:32:17.937 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:32:17 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T09:32:17.937 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:32:17 smithi005 ceph-mon[30350]: from='osd.6 [v2:172.21.15.5:6818/1907651140,v1:172.21.15.5:6819/1907651140]' entity='osd.6' 2024-09-10T09:32:17.937 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:32:17 smithi005 ceph-mon[30350]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-09-10T09:32:18.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:32:17 smithi159 ceph-mon[35513]: from='osd.6 [v2:172.21.15.5:6818/1907651140,v1:172.21.15.5:6819/1907651140]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi005", "root=default"]}]': finished 2024-09-10T09:32:18.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:32:17 smithi159 ceph-mon[35513]: osdmap e44: 7 total, 6 up, 7 in 2024-09-10T09:32:18.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:32:17 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T09:32:18.101 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:32:17 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T09:32:18.101 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:32:17 smithi159 ceph-mon[35513]: from='osd.6 [v2:172.21.15.5:6818/1907651140,v1:172.21.15.5:6819/1907651140]' entity='osd.6' 2024-09-10T09:32:18.101 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:32:17 smithi159 ceph-mon[35513]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-09-10T09:32:18.108 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T09:32:18.627 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-10T09:32:18.609+0000 7ff8b0bfd640 1 -- 172.21.15.102:0/560212974 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff8ac101d10 msgr2=0x7ff8ac102190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:32:18.628 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-10T09:32:18.609+0000 7ff8b0bfd640 1 --2- 172.21.15.102:0/560212974 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff8ac101d10 0x7ff8ac102190 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7ff8a0009900 tx=0x7ff8a002a410 comp rx=0 tx=0).stop 2024-09-10T09:32:18.628 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-10T09:32:18.610+0000 7ff8b0bfd640 1 -- 172.21.15.102:0/560212974 shutdown_connections 2024-09-10T09:32:18.628 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-10T09:32:18.610+0000 7ff8b0bfd640 1 --2- 172.21.15.102:0/560212974 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff8ac1026d0 0x7ff8ac108f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:32:18.628 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-10T09:32:18.610+0000 7ff8b0bfd640 1 --2- 172.21.15.102:0/560212974 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff8ac101d10 0x7ff8ac102190 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:32:18.628 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-10T09:32:18.610+0000 7ff8b0bfd640 1 --2- 172.21.15.102:0/560212974 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff8ac100b10 0x7ff8ac100f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:32:18.628 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-10T09:32:18.610+0000 7ff8b0bfd640 1 -- 172.21.15.102:0/560212974 >> 172.21.15.102:0/560212974 conn(0x7ff8ac0fc2a0 msgr2=0x7ff8ac0fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:32:18.628 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-10T09:32:18.610+0000 7ff8b0bfd640 1 -- 172.21.15.102:0/560212974 shutdown_connections 2024-09-10T09:32:18.628 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-10T09:32:18.610+0000 7ff8b0bfd640 1 -- 172.21.15.102:0/560212974 wait complete. 2024-09-10T09:32:18.628 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-10T09:32:18.610+0000 7ff8b0bfd640 1 Processor -- start 2024-09-10T09:32:18.628 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-10T09:32:18.610+0000 7ff8b0bfd640 1 -- start start 2024-09-10T09:32:18.628 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-10T09:32:18.611+0000 7ff8b0bfd640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff8ac100b10 0x7ff8ac06b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:32:18.629 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-10T09:32:18.611+0000 7ff8b0bfd640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff8ac101d10 0x7ff8ac19cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:32:18.629 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-10T09:32:18.611+0000 7ff8b0bfd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff8ac1026d0 0x7ff8ac19f290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:32:18.629 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-10T09:32:18.611+0000 7ff8b0bfd640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff8ac10c7e0 con 0x7ff8ac1026d0 2024-09-10T09:32:18.629 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-10T09:32:18.611+0000 7ff8b0bfd640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7ff8ac10c660 con 0x7ff8ac101d10 2024-09-10T09:32:18.629 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-10T09:32:18.611+0000 7ff8b0bfd640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7ff8ac10c960 con 0x7ff8ac100b10 2024-09-10T09:32:18.629 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-10T09:32:18.611+0000 7ff8a9d9b640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff8ac101d10 0x7ff8ac19cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:32:18.629 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-10T09:32:18.611+0000 7ff8a9d9b640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff8ac101d10 0x7ff8ac19cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:51532/0 (socket says 172.21.15.102:51532) 2024-09-10T09:32:18.629 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-10T09:32:18.611+0000 7ff8a9d9b640 1 -- 172.21.15.102:0/2016661440 learned_addr learned my addr 172.21.15.102:0/2016661440 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-10T09:32:18.629 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-10T09:32:18.611+0000 7ff8aa59c640 1 --2- 172.21.15.102:0/2016661440 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff8ac100b10 0x7ff8ac06b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:32:18.629 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-10T09:32:18.611+0000 7ff8aad9d640 1 --2- 172.21.15.102:0/2016661440 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff8ac1026d0 0x7ff8ac19f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:32:18.629 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-10T09:32:18.611+0000 7ff8a9d9b640 1 -- 172.21.15.102:0/2016661440 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff8ac100b10 msgr2=0x7ff8ac06b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:32:18.630 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-10T09:32:18.611+0000 7ff8a9d9b640 1 --2- 172.21.15.102:0/2016661440 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff8ac100b10 0x7ff8ac06b2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:32:18.630 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-10T09:32:18.611+0000 7ff8a9d9b640 1 -- 172.21.15.102:0/2016661440 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff8ac1026d0 msgr2=0x7ff8ac19f290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:32:18.630 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-10T09:32:18.611+0000 7ff8a9d9b640 1 --2- 172.21.15.102:0/2016661440 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff8ac1026d0 0x7ff8ac19f290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:32:18.630 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-10T09:32:18.611+0000 7ff8a9d9b640 1 -- 172.21.15.102:0/2016661440 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8ac19fba0 con 0x7ff8ac101d10 2024-09-10T09:32:18.630 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-10T09:32:18.611+0000 7ff8a9d9b640 1 --2- 172.21.15.102:0/2016661440 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff8ac101d10 0x7ff8ac19cc70 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7ff8a0031440 tx=0x7ff8a0032e20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:32:18.630 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-10T09:32:18.611+0000 7ff89b7fe640 1 -- 172.21.15.102:0/2016661440 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8a003f030 con 0x7ff8ac101d10 2024-09-10T09:32:18.630 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-10T09:32:18.611+0000 7ff89b7fe640 1 -- 172.21.15.102:0/2016661440 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff8a0003ce0 con 0x7ff8ac101d10 2024-09-10T09:32:18.630 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-10T09:32:18.611+0000 7ff8b0bfd640 1 -- 172.21.15.102:0/2016661440 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8ac19fdd0 con 0x7ff8ac101d10 2024-09-10T09:32:18.630 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-10T09:32:18.611+0000 7ff89b7fe640 1 -- 172.21.15.102:0/2016661440 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8a0035070 con 0x7ff8ac101d10 2024-09-10T09:32:18.630 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-10T09:32:18.611+0000 7ff8b0bfd640 1 -- 172.21.15.102:0/2016661440 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8ac1a0250 con 0x7ff8ac101d10 2024-09-10T09:32:18.630 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-10T09:32:18.612+0000 7ff8b0bfd640 1 -- 172.21.15.102:0/2016661440 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff8ac107ec0 con 0x7ff8ac101d10 2024-09-10T09:32:18.631 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-10T09:32:18.613+0000 7ff89b7fe640 1 -- 172.21.15.102:0/2016661440 <== mon.2 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7ff8a0042030 con 0x7ff8ac101d10 2024-09-10T09:32:18.631 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-10T09:32:18.614+0000 7ff89b7fe640 1 --2- 172.21.15.102:0/2016661440 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7ff8940a2b80 0x7ff8940a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:32:18.631 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-10T09:32:18.614+0000 7ff89b7fe640 1 -- 172.21.15.102:0/2016661440 <== mon.2 v2:172.21.15.102:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 4965+0+0 (secure 0 0 0) 0x7ff8a00eea20 con 0x7ff8ac101d10 2024-09-10T09:32:18.631 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-10T09:32:18.614+0000 7ff8aa59c640 1 --2- 172.21.15.102:0/2016661440 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7ff8940a2b80 0x7ff8940a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:32:18.631 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-10T09:32:18.615+0000 7ff8aa59c640 1 --2- 172.21.15.102:0/2016661440 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7ff8940a2b80 0x7ff8940a5040 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7ff89c005fd0 tx=0x7ff89c005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:32:18.631 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-10T09:32:18.618+0000 7ff89b7fe640 1 -- 172.21.15.102:0/2016661440 <== mon.2 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff8a00bc690 con 0x7ff8ac101d10 2024-09-10T09:32:18.791 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-10T09:32:18.790+0000 7ff8b0bfd640 1 -- 172.21.15.102:0/2016661440 --> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi102:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7ff8ac062630 con 0x7ff8940a2b80 2024-09-10T09:32:18.792 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:32:18 smithi102 ceph-mon[35442]: purged_snaps scrub starts 2024-09-10T09:32:18.792 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:32:18 smithi102 ceph-mon[35442]: purged_snaps scrub ok 2024-09-10T09:32:18.792 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:32:18 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T09:32:18.792 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:32:18 smithi102 ceph-mon[35442]: osd.6 [v2:172.21.15.5:6818/1907651140,v1:172.21.15.5:6819/1907651140] boot 2024-09-10T09:32:18.792 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:32:18 smithi102 ceph-mon[35442]: osdmap e45: 7 total, 7 up, 7 in 2024-09-10T09:32:18.792 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:32:18 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T09:32:19.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:32:18 smithi005 ceph-mon[30350]: purged_snaps scrub starts 2024-09-10T09:32:19.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:32:18 smithi005 ceph-mon[30350]: purged_snaps scrub ok 2024-09-10T09:32:19.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:32:18 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T09:32:19.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:32:18 smithi005 ceph-mon[30350]: osd.6 [v2:172.21.15.5:6818/1907651140,v1:172.21.15.5:6819/1907651140] boot 2024-09-10T09:32:19.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:32:18 smithi005 ceph-mon[30350]: osdmap e45: 7 total, 7 up, 7 in 2024-09-10T09:32:19.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:32:18 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T09:32:19.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:32:18 smithi159 ceph-mon[35513]: purged_snaps scrub starts 2024-09-10T09:32:19.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:32:18 smithi159 ceph-mon[35513]: purged_snaps scrub ok 2024-09-10T09:32:19.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:32:18 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T09:32:19.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:32:18 smithi159 ceph-mon[35513]: osd.6 [v2:172.21.15.5:6818/1907651140,v1:172.21.15.5:6819/1907651140] boot 2024-09-10T09:32:19.101 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:32:18 smithi159 ceph-mon[35513]: osdmap e45: 7 total, 7 up, 7 in 2024-09-10T09:32:19.101 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:32:18 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T09:32:19.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:32:19 smithi005 ceph-mon[30350]: from='client.24281 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi102:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T09:32:19.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:32:19 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T09:32:19.807 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:32:19 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T09:32:19.807 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:32:19 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:32:19.807 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:32:19 smithi005 ceph-mon[30350]: osdmap e46: 7 total, 7 up, 7 in 2024-09-10T09:32:19.807 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:32:19 smithi005 ceph-mon[30350]: Detected new or changed devices on smithi005 2024-09-10T09:32:19.807 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:32:19 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:32:19.807 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:32:19 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:32:19.807 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:32:19 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:32:19.807 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:32:19 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:32:19.807 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:32:19 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:32:19.807 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:32:19 smithi005 ceph-mon[30350]: Adjusting osd_memory_target on smithi005 to 5698M 2024-09-10T09:32:19.807 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:32:19 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:32:19.807 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:32:19 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:32:19.807 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:32:19 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:32:19.807 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:32:19 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:32:19.808 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:32:19 smithi005 ceph-mon[30350]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-10T09:32:19.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:32:19 smithi102 ceph-mon[35442]: from='client.24281 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi102:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T09:32:19.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:32:19 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T09:32:19.885 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:32:19 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T09:32:19.885 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:32:19 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:32:19.885 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:32:19 smithi102 ceph-mon[35442]: osdmap e46: 7 total, 7 up, 7 in 2024-09-10T09:32:19.885 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:32:19 smithi102 ceph-mon[35442]: Detected new or changed devices on smithi005 2024-09-10T09:32:19.885 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:32:19 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:32:19.885 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:32:19 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:32:19.885 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:32:19 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:32:19.885 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:32:19 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:32:19.885 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:32:19 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:32:19.885 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:32:19 smithi102 ceph-mon[35442]: Adjusting osd_memory_target on smithi005 to 5698M 2024-09-10T09:32:19.885 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:32:19 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:32:19.885 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:32:19 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:32:19.885 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:32:19 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:32:19.886 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:32:19 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:32:19.886 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:32:19 smithi102 ceph-mon[35442]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-10T09:32:20.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:32:19 smithi159 ceph-mon[35513]: from='client.24281 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi102:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T09:32:20.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:32:19 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T09:32:20.101 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:32:19 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T09:32:20.101 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:32:19 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:32:20.101 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:32:19 smithi159 ceph-mon[35513]: osdmap e46: 7 total, 7 up, 7 in 2024-09-10T09:32:20.101 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:32:19 smithi159 ceph-mon[35513]: Detected new or changed devices on smithi005 2024-09-10T09:32:20.101 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:32:19 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:32:20.101 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:32:19 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:32:20.101 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:32:19 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:32:20.101 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:32:19 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:32:20.101 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:32:19 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:32:20.101 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:32:19 smithi159 ceph-mon[35513]: Adjusting osd_memory_target on smithi005 to 5698M 2024-09-10T09:32:20.101 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:32:19 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:32:20.101 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:32:19 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:32:20.101 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:32:19 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:32:20.102 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:32:19 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:32:20.102 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:32:19 smithi159 ceph-mon[35513]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-10T09:32:21.524 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:32:21 smithi102 ceph-mon[35442]: from='client.? 172.21.15.102:0/2243734917' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8ae0f4a9-99e7-4563-aa2a-15169684a8b0"}]: dispatch 2024-09-10T09:32:21.525 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:32:21 smithi102 ceph-mon[35442]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8ae0f4a9-99e7-4563-aa2a-15169684a8b0"}]: dispatch 2024-09-10T09:32:21.525 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:32:21 smithi102 ceph-mon[35442]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8ae0f4a9-99e7-4563-aa2a-15169684a8b0"}]': finished 2024-09-10T09:32:21.525 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:32:21 smithi102 ceph-mon[35442]: osdmap e47: 8 total, 7 up, 8 in 2024-09-10T09:32:21.525 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:32:21 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T09:32:21.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:32:21 smithi005 ceph-mon[30350]: from='client.? 172.21.15.102:0/2243734917' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8ae0f4a9-99e7-4563-aa2a-15169684a8b0"}]: dispatch 2024-09-10T09:32:21.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:32:21 smithi005 ceph-mon[30350]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8ae0f4a9-99e7-4563-aa2a-15169684a8b0"}]: dispatch 2024-09-10T09:32:21.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:32:21 smithi005 ceph-mon[30350]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8ae0f4a9-99e7-4563-aa2a-15169684a8b0"}]': finished 2024-09-10T09:32:21.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:32:21 smithi005 ceph-mon[30350]: osdmap e47: 8 total, 7 up, 8 in 2024-09-10T09:32:21.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:32:21 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T09:32:21.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:32:21 smithi159 ceph-mon[35513]: from='client.? 172.21.15.102:0/2243734917' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8ae0f4a9-99e7-4563-aa2a-15169684a8b0"}]: dispatch 2024-09-10T09:32:21.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:32:21 smithi159 ceph-mon[35513]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8ae0f4a9-99e7-4563-aa2a-15169684a8b0"}]: dispatch 2024-09-10T09:32:21.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:32:21 smithi159 ceph-mon[35513]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8ae0f4a9-99e7-4563-aa2a-15169684a8b0"}]': finished 2024-09-10T09:32:21.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:32:21 smithi159 ceph-mon[35513]: osdmap e47: 8 total, 7 up, 8 in 2024-09-10T09:32:21.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:32:21 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T09:32:22.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:32:22 smithi005 ceph-mon[30350]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-10T09:32:22.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:32:22 smithi005 ceph-mon[30350]: from='client.? 172.21.15.102:0/1987604719' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T09:32:22.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:32:22 smithi159 ceph-mon[35513]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-10T09:32:22.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:32:22 smithi159 ceph-mon[35513]: from='client.? 172.21.15.102:0/1987604719' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T09:32:22.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:32:22 smithi102 ceph-mon[35442]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-10T09:32:22.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:32:22 smithi102 ceph-mon[35442]: from='client.? 172.21.15.102:0/1987604719' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T09:32:23.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:32:23 smithi005 ceph-mon[30350]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-10T09:32:23.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:32:23 smithi159 ceph-mon[35513]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-10T09:32:23.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:32:23 smithi102 ceph-mon[35442]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-10T09:32:25.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:32:25 smithi005 ceph-mon[30350]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-10T09:32:25.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:32:25 smithi159 ceph-mon[35513]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-10T09:32:25.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:32:25 smithi102 ceph-mon[35442]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-10T09:32:27.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:32:27 smithi005 ceph-mon[30350]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-10T09:32:27.807 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:32:27 smithi102 ceph-mon[35442]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-10T09:32:27.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:32:27 smithi159 ceph-mon[35513]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-10T09:32:29.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:32:29 smithi005 ceph-mon[30350]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-10T09:32:29.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:32:29 smithi159 ceph-mon[35513]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-10T09:32:29.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:32:29 smithi102 ceph-mon[35442]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-10T09:32:30.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:32:30 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-09-10T09:32:30.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:32:30 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:32:30.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:32:30 smithi102 ceph-mon[35442]: Deploying daemon osd.7 on smithi102 2024-09-10T09:32:30.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:32:30 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-09-10T09:32:30.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:32:30 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:32:30.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:32:30 smithi005 ceph-mon[30350]: Deploying daemon osd.7 on smithi102 2024-09-10T09:32:30.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:32:30 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-09-10T09:32:30.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:32:30 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:32:30.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:32:30 smithi159 ceph-mon[35513]: Deploying daemon osd.7 on smithi102 2024-09-10T09:32:31.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:32:31 smithi005 ceph-mon[30350]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-10T09:32:31.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:32:31 smithi159 ceph-mon[35513]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-10T09:32:31.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:32:31 smithi102 ceph-mon[35442]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-10T09:32:33.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:32:33 smithi102 ceph-mon[35442]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-10T09:32:33.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:32:33 smithi005 ceph-mon[30350]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-10T09:32:33.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:32:33 smithi159 ceph-mon[35513]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-10T09:32:35.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:32:35 smithi005 ceph-mon[30350]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-10T09:32:35.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:32:35 smithi159 ceph-mon[35513]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-10T09:32:35.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:32:35 smithi102 ceph-mon[35442]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-10T09:32:36.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:32:36 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:32:36.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:32:36 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:32:36.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:32:36 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:32:36.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:32:36 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:32:36.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:32:36 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:32:36.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:32:36 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:32:36.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:32:36 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:32:36.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:32:36 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:32:36.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:32:36 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:32:37.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:32:37 smithi159 ceph-mon[35513]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-10T09:32:37.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:32:37 smithi102 ceph-mon[35442]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-10T09:32:38.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:32:37 smithi005 ceph-mon[30350]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-10T09:32:39.265 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-10T09:32:39.264+0000 7ff89b7fe640 1 -- 172.21.15.102:0/2016661440 <== mgr.14150 v2:172.21.15.5:6800/2707348081 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7ff8ac062630 con 0x7ff8940a2b80 2024-09-10T09:32:39.265 INFO:teuthology.orchestra.run.smithi102.stdout:Created osd(s) 7 on host 'smithi102' 2024-09-10T09:32:39.267 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-10T09:32:39.266+0000 7ff8b0bfd640 1 -- 172.21.15.102:0/2016661440 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7ff8940a2b80 msgr2=0x7ff8940a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:32:39.267 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-10T09:32:39.266+0000 7ff8b0bfd640 1 --2- 172.21.15.102:0/2016661440 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7ff8940a2b80 0x7ff8940a5040 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7ff89c005fd0 tx=0x7ff89c005850 comp rx=0 tx=0).stop 2024-09-10T09:32:39.267 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-10T09:32:39.266+0000 7ff8b0bfd640 1 -- 172.21.15.102:0/2016661440 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff8ac101d10 msgr2=0x7ff8ac19cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:32:39.267 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-10T09:32:39.266+0000 7ff8b0bfd640 1 --2- 172.21.15.102:0/2016661440 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff8ac101d10 0x7ff8ac19cc70 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7ff8a0031440 tx=0x7ff8a0032e20 comp rx=0 tx=0).stop 2024-09-10T09:32:39.267 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-10T09:32:39.266+0000 7ff8b0bfd640 1 -- 172.21.15.102:0/2016661440 shutdown_connections 2024-09-10T09:32:39.267 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-10T09:32:39.266+0000 7ff8b0bfd640 1 --2- 172.21.15.102:0/2016661440 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7ff8940a2b80 0x7ff8940a5040 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:32:39.267 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-10T09:32:39.266+0000 7ff8b0bfd640 1 --2- 172.21.15.102:0/2016661440 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff8ac1026d0 0x7ff8ac19f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:32:39.268 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-10T09:32:39.266+0000 7ff8b0bfd640 1 --2- 172.21.15.102:0/2016661440 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff8ac101d10 0x7ff8ac19cc70 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:32:39.268 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-10T09:32:39.266+0000 7ff8b0bfd640 1 --2- 172.21.15.102:0/2016661440 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff8ac100b10 0x7ff8ac06b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:32:39.268 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-10T09:32:39.266+0000 7ff8b0bfd640 1 -- 172.21.15.102:0/2016661440 >> 172.21.15.102:0/2016661440 conn(0x7ff8ac0fc2a0 msgr2=0x7ff8ac105090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:32:39.268 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-10T09:32:39.266+0000 7ff8b0bfd640 1 -- 172.21.15.102:0/2016661440 shutdown_connections 2024-09-10T09:32:39.268 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-10T09:32:39.266+0000 7ff8b0bfd640 1 -- 172.21.15.102:0/2016661440 wait complete. 2024-09-10T09:32:39.924 DEBUG:teuthology.orchestra.run.smithi102:osd.7> sudo journalctl -f -n 0 -u ceph-bade4b50-6f56-11ef-bcea-c7b262605968@osd.7.service 2024-09-10T09:32:39.927 INFO:tasks.cephadm:Deploying osd.8 on smithi159 with /dev/vg_nvme/lv_2... 2024-09-10T09:32:39.927 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_2 2024-09-10T09:32:40.182 INFO:teuthology.orchestra.run.smithi159.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.c/config 2024-09-10T09:32:40.458 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:32:40 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:32:40.458 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:32:40 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:32:40.459 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:32:40 smithi102 ceph-mon[35442]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-10T09:32:40.459 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:32:40 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:32:40.459 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:32:40 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:32:40.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:32:40 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:32:40.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:32:40 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:32:40.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:32:40 smithi159 ceph-mon[35513]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-10T09:32:40.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:32:40 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:32:40.511 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:32:40 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:32:40.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:32:40 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:32:40.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:32:40 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:32:40.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:32:40 smithi005 ceph-mon[30350]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-10T09:32:40.557 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:32:40 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:32:40.557 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:32:40 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:32:41.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:32:41 smithi102 ceph-mon[35442]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-10T09:32:41.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:32:41 smithi102 ceph-mon[35442]: from='osd.7 [v2:172.21.15.102:6816/692040360,v1:172.21.15.102:6817/692040360]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-10T09:32:41.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:32:41 smithi102 ceph-mon[35442]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-10T09:32:41.634 INFO:journalctl@ceph.osd.7.smithi102.stdout:Sep 10 09:32:41 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-7[52880]: 2024-09-10T09:32:41.524+0000 7ff947da9640 -1 osd.7 0 log_to_monitors true 2024-09-10T09:32:41.692 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:32:41 smithi159 ceph-mon[35513]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-10T09:32:41.692 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:32:41 smithi159 ceph-mon[35513]: from='osd.7 [v2:172.21.15.102:6816/692040360,v1:172.21.15.102:6817/692040360]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-10T09:32:41.692 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:32:41 smithi159 ceph-mon[35513]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-10T09:32:41.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:32:41 smithi005 ceph-mon[30350]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-10T09:32:41.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:32:41 smithi005 ceph-mon[30350]: from='osd.7 [v2:172.21.15.102:6816/692040360,v1:172.21.15.102:6817/692040360]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-10T09:32:41.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:32:41 smithi005 ceph-mon[30350]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-10T09:32:42.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:32:42 smithi159 ceph-mon[35513]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-10T09:32:42.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:32:42 smithi159 ceph-mon[35513]: osdmap e48: 8 total, 7 up, 8 in 2024-09-10T09:32:42.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:32:42 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T09:32:42.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:32:42 smithi159 ceph-mon[35513]: from='osd.7 [v2:172.21.15.102:6816/692040360,v1:172.21.15.102:6817/692040360]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]: dispatch 2024-09-10T09:32:42.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:32:42 smithi159 ceph-mon[35513]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]: dispatch 2024-09-10T09:32:42.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:32:42 smithi102 ceph-mon[35442]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-10T09:32:42.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:32:42 smithi102 ceph-mon[35442]: osdmap e48: 8 total, 7 up, 8 in 2024-09-10T09:32:42.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:32:42 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T09:32:42.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:32:42 smithi102 ceph-mon[35442]: from='osd.7 [v2:172.21.15.102:6816/692040360,v1:172.21.15.102:6817/692040360]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]: dispatch 2024-09-10T09:32:42.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:32:42 smithi102 ceph-mon[35442]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]: dispatch 2024-09-10T09:32:42.885 INFO:journalctl@ceph.osd.7.smithi102.stdout:Sep 10 09:32:42 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-7[52880]: 2024-09-10T09:32:42.588+0000 7ff943d2c640 -1 osd.7 0 waiting for initial osdmap 2024-09-10T09:32:42.885 INFO:journalctl@ceph.osd.7.smithi102.stdout:Sep 10 09:32:42 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-7[52880]: 2024-09-10T09:32:42.598+0000 7ff93fb54640 -1 osd.7 49 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T09:32:42.993 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-10T09:32:43.020 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi159:vg_nvme/lv_2 2024-09-10T09:32:43.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:32:42 smithi005 ceph-mon[30350]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-10T09:32:43.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:32:42 smithi005 ceph-mon[30350]: osdmap e48: 8 total, 7 up, 8 in 2024-09-10T09:32:43.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:32:42 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T09:32:43.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:32:42 smithi005 ceph-mon[30350]: from='osd.7 [v2:172.21.15.102:6816/692040360,v1:172.21.15.102:6817/692040360]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]: dispatch 2024-09-10T09:32:43.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:32:42 smithi005 ceph-mon[30350]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]: dispatch 2024-09-10T09:32:43.266 INFO:teuthology.orchestra.run.smithi159.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.c/config 2024-09-10T09:32:43.662 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:32:43 smithi159 ceph-mon[35513]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]': finished 2024-09-10T09:32:43.662 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:32:43 smithi159 ceph-mon[35513]: osdmap e49: 8 total, 7 up, 8 in 2024-09-10T09:32:43.662 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:32:43 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T09:32:43.662 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:32:43 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T09:32:43.662 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:32:43 smithi159 ceph-mon[35513]: Detected new or changed devices on smithi102 2024-09-10T09:32:43.662 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:32:43 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:32:43.662 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:32:43 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:32:43.662 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:32:43 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:32:43.662 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:32:43 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:32:43.663 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:32:43 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:32:43.663 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:32:43 smithi159 ceph-mon[35513]: Adjusting osd_memory_target on smithi102 to 5696M 2024-09-10T09:32:43.663 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:32:43 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:32:43.663 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:32:43 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:32:43.663 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:32:43 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:32:43.663 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:32:43 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:32:43.663 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:32:43 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:32:43.663 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:32:43 smithi159 ceph-mon[35513]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-10T09:32:43.761 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:32:43.745+0000 7fac4acb9640 1 -- 172.21.15.159:0/1694599924 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fac44100ad0 msgr2=0x7fac44100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:32:43.762 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:32:43.745+0000 7fac4acb9640 1 --2- 172.21.15.159:0/1694599924 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fac44100ad0 0x7fac44100ed0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fac40009900 tx=0x7fac4002a410 comp rx=0 tx=0).stop 2024-09-10T09:32:43.762 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:32:43.746+0000 7fac4acb9640 1 -- 172.21.15.159:0/1694599924 shutdown_connections 2024-09-10T09:32:43.762 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:32:43.746+0000 7fac4acb9640 1 --2- 172.21.15.159:0/1694599924 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fac44102690 0x7fac44108f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:32:43.762 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:32:43.746+0000 7fac4acb9640 1 --2- 172.21.15.159:0/1694599924 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fac44101cd0 0x7fac44102150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:32:43.762 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:32:43.746+0000 7fac4acb9640 1 --2- 172.21.15.159:0/1694599924 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fac44100ad0 0x7fac44100ed0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:32:43.762 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:32:43.746+0000 7fac4acb9640 1 -- 172.21.15.159:0/1694599924 >> 172.21.15.159:0/1694599924 conn(0x7fac440fc280 msgr2=0x7fac440fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:32:43.762 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:32:43.746+0000 7fac4acb9640 1 -- 172.21.15.159:0/1694599924 shutdown_connections 2024-09-10T09:32:43.762 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:32:43.746+0000 7fac4acb9640 1 -- 172.21.15.159:0/1694599924 wait complete. 2024-09-10T09:32:43.762 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:32:43.746+0000 7fac4acb9640 1 Processor -- start 2024-09-10T09:32:43.762 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:32:43.746+0000 7fac4acb9640 1 -- start start 2024-09-10T09:32:43.762 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:32:43.746+0000 7fac4acb9640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fac44100ad0 0x7fac4410b450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:32:43.762 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:32:43.746+0000 7fac4acb9640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fac44101cd0 0x7fac44109aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:32:43.762 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:32:43.746+0000 7fac4acb9640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fac44102690 0x7fac44109fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:32:43.763 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:32:43.746+0000 7fac4acb9640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fac4410d880 con 0x7fac44102690 2024-09-10T09:32:43.763 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:32:43.746+0000 7fac4acb9640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fac4410d700 con 0x7fac44101cd0 2024-09-10T09:32:43.763 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:32:43.746+0000 7fac4acb9640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fac4410da00 con 0x7fac44100ad0 2024-09-10T09:32:43.763 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:32:43.746+0000 7fac48a55640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fac44100ad0 0x7fac4410b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:32:43.763 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:32:43.746+0000 7fac48a55640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fac44100ad0 0x7fac4410b450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:46054/0 (socket says 172.21.15.159:46054) 2024-09-10T09:32:43.763 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:32:43.746+0000 7fac48a55640 1 -- 172.21.15.159:0/2216346776 learned_addr learned my addr 172.21.15.159:0/2216346776 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-09-10T09:32:43.763 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:32:43.747+0000 7fac3bfff640 1 --2- 172.21.15.159:0/2216346776 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fac44101cd0 0x7fac44109aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:32:43.763 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:32:43.747+0000 7fac49256640 1 --2- 172.21.15.159:0/2216346776 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fac44102690 0x7fac44109fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:32:43.763 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:32:43.747+0000 7fac48a55640 1 -- 172.21.15.159:0/2216346776 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fac44101cd0 msgr2=0x7fac44109aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:32:43.763 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:32:43.747+0000 7fac48a55640 1 --2- 172.21.15.159:0/2216346776 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fac44101cd0 0x7fac44109aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:32:43.763 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:32:43.747+0000 7fac48a55640 1 -- 172.21.15.159:0/2216346776 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fac44102690 msgr2=0x7fac44109fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:32:43.764 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:32:43.747+0000 7fac48a55640 1 --2- 172.21.15.159:0/2216346776 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fac44102690 0x7fac44109fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:32:43.764 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:32:43.747+0000 7fac48a55640 1 -- 172.21.15.159:0/2216346776 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac4410a8a0 con 0x7fac44100ad0 2024-09-10T09:32:43.764 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:32:43.747+0000 7fac49256640 1 --2- 172.21.15.159:0/2216346776 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fac44102690 0x7fac44109fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:32:43.764 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:32:43.747+0000 7fac48a55640 1 --2- 172.21.15.159:0/2216346776 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fac44100ad0 0x7fac4410b450 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fac40031610 tx=0x7fac400043b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:32:43.764 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:32:43.747+0000 7fac39ffb640 1 -- 172.21.15.159:0/2216346776 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac4003f030 con 0x7fac44100ad0 2024-09-10T09:32:43.764 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:32:43.747+0000 7fac39ffb640 1 -- 172.21.15.159:0/2216346776 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fac40003ce0 con 0x7fac44100ad0 2024-09-10T09:32:43.764 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:32:43.747+0000 7fac39ffb640 1 -- 172.21.15.159:0/2216346776 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac40035070 con 0x7fac44100ad0 2024-09-10T09:32:43.764 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:32:43.747+0000 7fac4acb9640 1 -- 172.21.15.159:0/2216346776 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac441a2060 con 0x7fac44100ad0 2024-09-10T09:32:43.764 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:32:43.747+0000 7fac4acb9640 1 -- 172.21.15.159:0/2216346776 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac441a2510 con 0x7fac44100ad0 2024-09-10T09:32:43.764 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:32:43.748+0000 7fac4acb9640 1 -- 172.21.15.159:0/2216346776 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac44107ec0 con 0x7fac44100ad0 2024-09-10T09:32:43.764 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:32:43.749+0000 7fac39ffb640 1 -- 172.21.15.159:0/2216346776 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7fac40042030 con 0x7fac44100ad0 2024-09-10T09:32:43.764 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:32:43.750+0000 7fac39ffb640 1 --2- 172.21.15.159:0/2216346776 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fac2c0a2b80 0x7fac2c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:32:43.764 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:32:43.750+0000 7fac39ffb640 1 -- 172.21.15.159:0/2216346776 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5430+0+0 (secure 0 0 0) 0x7fac400eeac0 con 0x7fac44100ad0 2024-09-10T09:32:43.765 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:32:43.750+0000 7fac3bfff640 1 --2- 172.21.15.159:0/2216346776 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fac2c0a2b80 0x7fac2c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:32:43.765 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:32:43.751+0000 7fac3bfff640 1 --2- 172.21.15.159:0/2216346776 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fac2c0a2b80 0x7fac2c0a5040 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fac4410b1d0 tx=0x7fac3400a250 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:32:43.765 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:32:43.754+0000 7fac39ffb640 1 -- 172.21.15.159:0/2216346776 <== mon.1 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fac400bc540 con 0x7fac44100ad0 2024-09-10T09:32:43.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:32:43 smithi102 ceph-mon[35442]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]': finished 2024-09-10T09:32:43.885 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:32:43 smithi102 ceph-mon[35442]: osdmap e49: 8 total, 7 up, 8 in 2024-09-10T09:32:43.885 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:32:43 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T09:32:43.885 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:32:43 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T09:32:43.885 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:32:43 smithi102 ceph-mon[35442]: Detected new or changed devices on smithi102 2024-09-10T09:32:43.885 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:32:43 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:32:43.885 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:32:43 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:32:43.885 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:32:43 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:32:43.885 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:32:43 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:32:43.885 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:32:43 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:32:43.885 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:32:43 smithi102 ceph-mon[35442]: Adjusting osd_memory_target on smithi102 to 5696M 2024-09-10T09:32:43.885 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:32:43 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:32:43.885 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:32:43 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:32:43.885 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:32:43 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:32:43.885 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:32:43 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:32:43.886 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:32:43 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:32:43.886 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:32:43 smithi102 ceph-mon[35442]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-10T09:32:43.934 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:32:43.933+0000 7fac4acb9640 1 -- 172.21.15.159:0/2216346776 --> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi159:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7fac44062630 con 0x7fac2c0a2b80 2024-09-10T09:32:44.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:32:43 smithi005 ceph-mon[30350]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]': finished 2024-09-10T09:32:44.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:32:43 smithi005 ceph-mon[30350]: osdmap e49: 8 total, 7 up, 8 in 2024-09-10T09:32:44.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:32:43 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T09:32:44.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:32:43 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T09:32:44.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:32:43 smithi005 ceph-mon[30350]: Detected new or changed devices on smithi102 2024-09-10T09:32:44.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:32:43 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:32:44.057 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:32:43 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:32:44.057 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:32:43 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:32:44.057 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:32:43 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:32:44.057 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:32:43 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:32:44.057 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:32:43 smithi005 ceph-mon[30350]: Adjusting osd_memory_target on smithi102 to 5696M 2024-09-10T09:32:44.057 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:32:43 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:32:44.057 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:32:43 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:32:44.057 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:32:43 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:32:44.057 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:32:43 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:32:44.057 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:32:43 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:32:44.057 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:32:43 smithi005 ceph-mon[30350]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-09-10T09:32:44.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:32:44 smithi159 ceph-mon[35513]: purged_snaps scrub starts 2024-09-10T09:32:44.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:32:44 smithi159 ceph-mon[35513]: purged_snaps scrub ok 2024-09-10T09:32:44.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:32:44 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T09:32:44.851 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:32:44 smithi159 ceph-mon[35513]: osd.7 [v2:172.21.15.102:6816/692040360,v1:172.21.15.102:6817/692040360] boot 2024-09-10T09:32:44.851 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:32:44 smithi159 ceph-mon[35513]: osdmap e50: 8 total, 8 up, 8 in 2024-09-10T09:32:44.851 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:32:44 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T09:32:44.851 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:32:44 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:32:44.851 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:32:44 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:32:44.851 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:32:44 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:32:44.851 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:32:44 smithi159 ceph-mon[35513]: from='client.24319 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi159:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T09:32:44.851 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:32:44 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T09:32:44.851 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:32:44 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T09:32:44.851 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:32:44 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:32:44.851 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:32:44 smithi159 ceph-mon[35513]: osdmap e51: 8 total, 8 up, 8 in 2024-09-10T09:32:44.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:32:44 smithi102 ceph-mon[35442]: purged_snaps scrub starts 2024-09-10T09:32:44.885 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:32:44 smithi102 ceph-mon[35442]: purged_snaps scrub ok 2024-09-10T09:32:44.885 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:32:44 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T09:32:44.885 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:32:44 smithi102 ceph-mon[35442]: osd.7 [v2:172.21.15.102:6816/692040360,v1:172.21.15.102:6817/692040360] boot 2024-09-10T09:32:44.885 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:32:44 smithi102 ceph-mon[35442]: osdmap e50: 8 total, 8 up, 8 in 2024-09-10T09:32:44.885 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:32:44 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T09:32:44.885 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:32:44 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:32:44.885 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:32:44 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:32:44.885 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:32:44 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:32:44.885 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:32:44 smithi102 ceph-mon[35442]: from='client.24319 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi159:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T09:32:44.885 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:32:44 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T09:32:44.885 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:32:44 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T09:32:44.885 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:32:44 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:32:44.886 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:32:44 smithi102 ceph-mon[35442]: osdmap e51: 8 total, 8 up, 8 in 2024-09-10T09:32:45.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:32:44 smithi005 ceph-mon[30350]: purged_snaps scrub starts 2024-09-10T09:32:45.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:32:44 smithi005 ceph-mon[30350]: purged_snaps scrub ok 2024-09-10T09:32:45.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:32:44 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T09:32:45.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:32:44 smithi005 ceph-mon[30350]: osd.7 [v2:172.21.15.102:6816/692040360,v1:172.21.15.102:6817/692040360] boot 2024-09-10T09:32:45.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:32:44 smithi005 ceph-mon[30350]: osdmap e50: 8 total, 8 up, 8 in 2024-09-10T09:32:45.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:32:44 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T09:32:45.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:32:44 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:32:45.057 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:32:44 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:32:45.057 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:32:44 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:32:45.057 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:32:44 smithi005 ceph-mon[30350]: from='client.24319 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi159:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T09:32:45.057 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:32:44 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T09:32:45.057 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:32:44 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T09:32:45.057 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:32:44 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:32:45.057 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:32:44 smithi005 ceph-mon[30350]: osdmap e51: 8 total, 8 up, 8 in 2024-09-10T09:32:46.306 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:32:45 smithi005 ceph-mon[30350]: osdmap e52: 8 total, 8 up, 8 in 2024-09-10T09:32:46.306 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:32:45 smithi005 ceph-mon[30350]: pgmap v182: 1 pgs: 1 remapped+peering; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-09-10T09:32:46.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:32:45 smithi159 ceph-mon[35513]: osdmap e52: 8 total, 8 up, 8 in 2024-09-10T09:32:46.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:32:45 smithi159 ceph-mon[35513]: pgmap v182: 1 pgs: 1 remapped+peering; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-09-10T09:32:46.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:32:45 smithi102 ceph-mon[35442]: osdmap e52: 8 total, 8 up, 8 in 2024-09-10T09:32:46.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:32:45 smithi102 ceph-mon[35442]: pgmap v182: 1 pgs: 1 remapped+peering; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-09-10T09:32:47.306 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:32:46 smithi005 ceph-mon[30350]: osdmap e53: 8 total, 8 up, 8 in 2024-09-10T09:32:47.306 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:32:46 smithi005 ceph-mon[30350]: from='client.? 172.21.15.159:0/920277639' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e37c845e-ced9-4413-9bbe-bc090537c2bc"}]: dispatch 2024-09-10T09:32:47.306 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:32:46 smithi005 ceph-mon[30350]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e37c845e-ced9-4413-9bbe-bc090537c2bc"}]: dispatch 2024-09-10T09:32:47.306 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:32:46 smithi005 ceph-mon[30350]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e37c845e-ced9-4413-9bbe-bc090537c2bc"}]': finished 2024-09-10T09:32:47.307 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:32:46 smithi005 ceph-mon[30350]: osdmap e54: 9 total, 8 up, 9 in 2024-09-10T09:32:47.307 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:32:46 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-10T09:32:47.307 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:32:46 smithi005 ceph-mon[30350]: from='client.? 172.21.15.159:0/2347326575' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T09:32:47.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:32:46 smithi159 ceph-mon[35513]: osdmap e53: 8 total, 8 up, 8 in 2024-09-10T09:32:47.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:32:46 smithi159 ceph-mon[35513]: from='client.? 172.21.15.159:0/920277639' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e37c845e-ced9-4413-9bbe-bc090537c2bc"}]: dispatch 2024-09-10T09:32:47.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:32:46 smithi159 ceph-mon[35513]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e37c845e-ced9-4413-9bbe-bc090537c2bc"}]: dispatch 2024-09-10T09:32:47.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:32:46 smithi159 ceph-mon[35513]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e37c845e-ced9-4413-9bbe-bc090537c2bc"}]': finished 2024-09-10T09:32:47.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:32:46 smithi159 ceph-mon[35513]: osdmap e54: 9 total, 8 up, 9 in 2024-09-10T09:32:47.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:32:46 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-10T09:32:47.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:32:46 smithi159 ceph-mon[35513]: from='client.? 172.21.15.159:0/2347326575' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T09:32:47.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:32:46 smithi102 ceph-mon[35442]: osdmap e53: 8 total, 8 up, 8 in 2024-09-10T09:32:47.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:32:46 smithi102 ceph-mon[35442]: from='client.? 172.21.15.159:0/920277639' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e37c845e-ced9-4413-9bbe-bc090537c2bc"}]: dispatch 2024-09-10T09:32:47.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:32:46 smithi102 ceph-mon[35442]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e37c845e-ced9-4413-9bbe-bc090537c2bc"}]: dispatch 2024-09-10T09:32:47.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:32:46 smithi102 ceph-mon[35442]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e37c845e-ced9-4413-9bbe-bc090537c2bc"}]': finished 2024-09-10T09:32:47.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:32:46 smithi102 ceph-mon[35442]: osdmap e54: 9 total, 8 up, 9 in 2024-09-10T09:32:47.385 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:32:46 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-10T09:32:47.385 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:32:46 smithi102 ceph-mon[35442]: from='client.? 172.21.15.159:0/2347326575' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T09:32:48.306 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:32:47 smithi005 ceph-mon[30350]: pgmap v185: 1 pgs: 1 remapped+peering; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-09-10T09:32:48.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:32:47 smithi159 ceph-mon[35513]: pgmap v185: 1 pgs: 1 remapped+peering; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-09-10T09:32:48.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:32:47 smithi102 ceph-mon[35442]: pgmap v185: 1 pgs: 1 remapped+peering; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-09-10T09:32:49.285 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:32:48 smithi102 ceph-mon[35442]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-09-10T09:32:49.306 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:32:48 smithi005 ceph-mon[30350]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-09-10T09:32:49.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:32:48 smithi159 ceph-mon[35513]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-09-10T09:32:50.306 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:32:49 smithi005 ceph-mon[30350]: pgmap v186: 1 pgs: 1 remapped+peering; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-09-10T09:32:50.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:32:49 smithi159 ceph-mon[35513]: pgmap v186: 1 pgs: 1 remapped+peering; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-09-10T09:32:50.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:32:49 smithi102 ceph-mon[35442]: pgmap v186: 1 pgs: 1 remapped+peering; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-09-10T09:32:51.777 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:32:51 smithi159 ceph-mon[35513]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-09-10T09:32:51.778 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:32:51 smithi159 ceph-mon[35513]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive, 1 pg peering) 2024-09-10T09:32:51.778 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:32:51 smithi159 ceph-mon[35513]: Cluster is now healthy 2024-09-10T09:32:51.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:32:51 smithi005 ceph-mon[30350]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-09-10T09:32:51.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:32:51 smithi005 ceph-mon[30350]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive, 1 pg peering) 2024-09-10T09:32:51.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:32:51 smithi005 ceph-mon[30350]: Cluster is now healthy 2024-09-10T09:32:51.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:32:51 smithi102 ceph-mon[35442]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-09-10T09:32:51.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:32:51 smithi102 ceph-mon[35442]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive, 1 pg peering) 2024-09-10T09:32:51.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:32:51 smithi102 ceph-mon[35442]: Cluster is now healthy 2024-09-10T09:32:53.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:32:53 smithi005 ceph-mon[30350]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-09-10T09:32:53.827 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:32:53 smithi159 ceph-mon[35513]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-09-10T09:32:53.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:32:53 smithi102 ceph-mon[35442]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-09-10T09:32:55.306 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:32:54 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-09-10T09:32:55.306 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:32:54 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:32:55.332 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:32:54 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-09-10T09:32:55.332 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:32:54 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:32:55.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:32:54 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-09-10T09:32:55.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:32:54 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:32:56.269 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:32:55 smithi159 ceph-mon[35513]: Deploying daemon osd.8 on smithi159 2024-09-10T09:32:56.269 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:32:55 smithi159 ceph-mon[35513]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-09-10T09:32:56.306 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:32:55 smithi005 ceph-mon[30350]: Deploying daemon osd.8 on smithi159 2024-09-10T09:32:56.306 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:32:55 smithi005 ceph-mon[30350]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-09-10T09:32:56.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:32:55 smithi102 ceph-mon[35442]: Deploying daemon osd.8 on smithi159 2024-09-10T09:32:56.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:32:55 smithi102 ceph-mon[35442]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-09-10T09:32:57.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:32:57 smithi159 ceph-mon[35513]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-09-10T09:32:57.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:32:57 smithi005 ceph-mon[30350]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-09-10T09:32:57.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:32:57 smithi102 ceph-mon[35442]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-09-10T09:32:59.658 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:32:59 smithi159 ceph-mon[35513]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-09-10T09:32:59.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:32:59 smithi005 ceph-mon[30350]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-09-10T09:32:59.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:32:59 smithi102 ceph-mon[35442]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-09-10T09:33:01.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:33:00 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:33:01.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:33:00 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:33:01.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:33:00 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:33:01.306 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:33:00 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:33:01.306 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:33:00 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:33:01.306 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:33:00 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:33:01.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:33:00 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:33:01.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:33:00 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:33:01.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:33:00 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:33:02.289 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:33:01 smithi159 ceph-mon[35513]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-09-10T09:33:02.306 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:33:01 smithi005 ceph-mon[30350]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-09-10T09:33:02.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:33:01 smithi102 ceph-mon[35442]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-09-10T09:33:03.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:33:03 smithi005 ceph-mon[30350]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-09-10T09:33:03.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:33:03 smithi159 ceph-mon[35513]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-09-10T09:33:03.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:33:03 smithi102 ceph-mon[35442]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-09-10T09:33:04.635 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:33:04.634+0000 7fac39ffb640 1 -- 172.21.15.159:0/2216346776 <== mgr.14150 v2:172.21.15.5:6800/2707348081 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fac44062630 con 0x7fac2c0a2b80 2024-09-10T09:33:04.635 INFO:teuthology.orchestra.run.smithi159.stdout:Created osd(s) 8 on host 'smithi159' 2024-09-10T09:33:04.637 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:33:04.636+0000 7fac4acb9640 1 -- 172.21.15.159:0/2216346776 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fac2c0a2b80 msgr2=0x7fac2c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:33:04.638 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:33:04.636+0000 7fac4acb9640 1 --2- 172.21.15.159:0/2216346776 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fac2c0a2b80 0x7fac2c0a5040 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fac4410b1d0 tx=0x7fac3400a250 comp rx=0 tx=0).stop 2024-09-10T09:33:04.638 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:33:04.636+0000 7fac4acb9640 1 -- 172.21.15.159:0/2216346776 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fac44100ad0 msgr2=0x7fac4410b450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:33:04.638 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:33:04.636+0000 7fac4acb9640 1 --2- 172.21.15.159:0/2216346776 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fac44100ad0 0x7fac4410b450 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fac40031610 tx=0x7fac400043b0 comp rx=0 tx=0).stop 2024-09-10T09:33:04.638 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:33:04.637+0000 7fac4acb9640 1 -- 172.21.15.159:0/2216346776 shutdown_connections 2024-09-10T09:33:04.638 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:33:04.637+0000 7fac4acb9640 1 --2- 172.21.15.159:0/2216346776 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fac2c0a2b80 0x7fac2c0a5040 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:33:04.638 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:33:04.637+0000 7fac4acb9640 1 --2- 172.21.15.159:0/2216346776 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fac44102690 0x7fac44109fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:33:04.638 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:33:04.637+0000 7fac4acb9640 1 --2- 172.21.15.159:0/2216346776 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fac44101cd0 0x7fac44109aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:33:04.638 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:33:04.637+0000 7fac4acb9640 1 --2- 172.21.15.159:0/2216346776 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fac44100ad0 0x7fac4410b450 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:33:04.638 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:33:04.637+0000 7fac4acb9640 1 -- 172.21.15.159:0/2216346776 >> 172.21.15.159:0/2216346776 conn(0x7fac440fc280 msgr2=0x7fac44105090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:33:04.638 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:33:04.637+0000 7fac4acb9640 1 -- 172.21.15.159:0/2216346776 shutdown_connections 2024-09-10T09:33:04.638 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:33:04.637+0000 7fac4acb9640 1 -- 172.21.15.159:0/2216346776 wait complete. 2024-09-10T09:33:04.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:33:04 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:33:04.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:33:04 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:33:04.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:33:04 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:33:04.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:33:04 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:33:04.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:33:04 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:33:04.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:33:04 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:33:04.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:33:04 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:33:04.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:33:04 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:33:04.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:33:04 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:33:04.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:33:04 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:33:05.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:33:04 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:33:05.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:33:04 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:33:05.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:33:04 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:33:05.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:33:04 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:33:05.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:33:04 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:33:05.273 DEBUG:teuthology.orchestra.run.smithi159:osd.8> sudo journalctl -f -n 0 -u ceph-bade4b50-6f56-11ef-bcea-c7b262605968@osd.8.service 2024-09-10T09:33:05.275 INFO:tasks.cephadm:Deploying osd.9 on smithi005 with /dev/vg_nvme/lv_1... 2024-09-10T09:33:05.276 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_1 2024-09-10T09:33:05.663 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:33:05.842 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:33:05 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:33:05.842 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:33:05 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:33:05.843 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:33:05 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:33:05.843 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:33:05 smithi159 ceph-mon[35513]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-09-10T09:33:05.850 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:33:05 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:33:05.850 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:33:05 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:33:05.850 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:33:05 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:33:05.850 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:33:05 smithi005 ceph-mon[30350]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-09-10T09:33:05.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:33:05 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:33:05.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:33:05 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:33:05.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:33:05 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:33:05.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:33:05 smithi102 ceph-mon[35442]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-09-10T09:33:07.326 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:33:07 smithi005 ceph-mon[30350]: from='osd.8 [v2:172.21.15.159:6816/2107291799,v1:172.21.15.159:6817/2107291799]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-10T09:33:07.326 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:33:07 smithi005 ceph-mon[30350]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-10T09:33:07.527 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:33:07 smithi159 ceph-mon[35513]: from='osd.8 [v2:172.21.15.159:6816/2107291799,v1:172.21.15.159:6817/2107291799]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-10T09:33:07.527 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:33:07 smithi159 ceph-mon[35513]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-10T09:33:07.528 INFO:journalctl@ceph.osd.8.smithi159.stdout:Sep 10 09:33:07 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-8[53439]: 2024-09-10T09:33:07.171+0000 7f7422361640 -1 osd.8 0 log_to_monitors true 2024-09-10T09:33:07.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:33:07 smithi102 ceph-mon[35442]: from='osd.8 [v2:172.21.15.159:6816/2107291799,v1:172.21.15.159:6817/2107291799]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-10T09:33:07.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:33:07 smithi102 ceph-mon[35442]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-10T09:33:08.306 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:33:08 smithi005 ceph-mon[30350]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-09-10T09:33:08.306 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:33:08 smithi005 ceph-mon[30350]: osdmap e55: 9 total, 8 up, 9 in 2024-09-10T09:33:08.306 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:33:08 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-10T09:33:08.306 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:33:08 smithi005 ceph-mon[30350]: from='osd.8 [v2:172.21.15.159:6816/2107291799,v1:172.21.15.159:6817/2107291799]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]: dispatch 2024-09-10T09:33:08.306 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:33:08 smithi005 ceph-mon[30350]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]: dispatch 2024-09-10T09:33:08.306 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:33:08 smithi005 ceph-mon[30350]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-09-10T09:33:08.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:33:08 smithi159 ceph-mon[35513]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-09-10T09:33:08.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:33:08 smithi159 ceph-mon[35513]: osdmap e55: 9 total, 8 up, 9 in 2024-09-10T09:33:08.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:33:08 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-10T09:33:08.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:33:08 smithi159 ceph-mon[35513]: from='osd.8 [v2:172.21.15.159:6816/2107291799,v1:172.21.15.159:6817/2107291799]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]: dispatch 2024-09-10T09:33:08.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:33:08 smithi159 ceph-mon[35513]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]: dispatch 2024-09-10T09:33:08.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:33:08 smithi159 ceph-mon[35513]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-09-10T09:33:08.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:33:08 smithi102 ceph-mon[35442]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-09-10T09:33:08.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:33:08 smithi102 ceph-mon[35442]: osdmap e55: 9 total, 8 up, 9 in 2024-09-10T09:33:08.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:33:08 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-10T09:33:08.635 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:33:08 smithi102 ceph-mon[35442]: from='osd.8 [v2:172.21.15.159:6816/2107291799,v1:172.21.15.159:6817/2107291799]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]: dispatch 2024-09-10T09:33:08.635 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:33:08 smithi102 ceph-mon[35442]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]: dispatch 2024-09-10T09:33:08.635 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:33:08 smithi102 ceph-mon[35442]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-09-10T09:33:09.074 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-10T09:33:09.100 INFO:journalctl@ceph.osd.8.smithi159.stdout:Sep 10 09:33:08 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-8[53439]: 2024-09-10T09:33:08.656+0000 7f741e2e4640 -1 osd.8 0 waiting for initial osdmap 2024-09-10T09:33:09.100 INFO:journalctl@ceph.osd.8.smithi159.stdout:Sep 10 09:33:08 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-8[53439]: 2024-09-10T09:33:08.666+0000 7f7416104640 -1 osd.8 56 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T09:33:09.106 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi005:vg_nvme/lv_1 2024-09-10T09:33:09.479 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:33:09.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:33:09 smithi005 ceph-mon[30350]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]': finished 2024-09-10T09:33:09.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:33:09 smithi005 ceph-mon[30350]: osdmap e56: 9 total, 8 up, 9 in 2024-09-10T09:33:09.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:33:09 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-10T09:33:09.557 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:33:09 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-10T09:33:09.557 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:33:09 smithi005 ceph-mon[30350]: Detected new or changed devices on smithi159 2024-09-10T09:33:09.557 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:33:09 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:33:09.557 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:33:09 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:33:09.557 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:33:09 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:33:09.557 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:33:09 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:33:09.557 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:33:09 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:33:09.557 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:33:09 smithi005 ceph-mon[30350]: Adjusting osd_memory_target on smithi159 to 5696M 2024-09-10T09:33:09.557 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:33:09 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:33:09.557 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:33:09 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:33:09.557 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:33:09 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:33:09.557 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:33:09 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:33:09.557 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:33:09 smithi005 ceph-mon[30350]: from='osd.8 ' entity='osd.8' 2024-09-10T09:33:09.558 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:33:09 smithi005 ceph-mon[30350]: osd.8 [v2:172.21.15.159:6816/2107291799,v1:172.21.15.159:6817/2107291799] boot 2024-09-10T09:33:09.558 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:33:09 smithi005 ceph-mon[30350]: osdmap e57: 9 total, 9 up, 9 in 2024-09-10T09:33:09.558 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:33:09 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-10T09:33:09.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:33:09 smithi159 ceph-mon[35513]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]': finished 2024-09-10T09:33:09.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:33:09 smithi159 ceph-mon[35513]: osdmap e56: 9 total, 8 up, 9 in 2024-09-10T09:33:09.601 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:33:09 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-10T09:33:09.601 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:33:09 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-10T09:33:09.601 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:33:09 smithi159 ceph-mon[35513]: Detected new or changed devices on smithi159 2024-09-10T09:33:09.601 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:33:09 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:33:09.601 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:33:09 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:33:09.601 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:33:09 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:33:09.601 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:33:09 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:33:09.601 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:33:09 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:33:09.601 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:33:09 smithi159 ceph-mon[35513]: Adjusting osd_memory_target on smithi159 to 5696M 2024-09-10T09:33:09.601 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:33:09 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:33:09.601 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:33:09 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:33:09.601 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:33:09 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:33:09.601 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:33:09 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:33:09.602 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:33:09 smithi159 ceph-mon[35513]: from='osd.8 ' entity='osd.8' 2024-09-10T09:33:09.602 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:33:09 smithi159 ceph-mon[35513]: osd.8 [v2:172.21.15.159:6816/2107291799,v1:172.21.15.159:6817/2107291799] boot 2024-09-10T09:33:09.602 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:33:09 smithi159 ceph-mon[35513]: osdmap e57: 9 total, 9 up, 9 in 2024-09-10T09:33:09.602 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:33:09 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-10T09:33:09.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:33:09 smithi102 ceph-mon[35442]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]': finished 2024-09-10T09:33:09.635 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:33:09 smithi102 ceph-mon[35442]: osdmap e56: 9 total, 8 up, 9 in 2024-09-10T09:33:09.635 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:33:09 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-10T09:33:09.635 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:33:09 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-10T09:33:09.635 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:33:09 smithi102 ceph-mon[35442]: Detected new or changed devices on smithi159 2024-09-10T09:33:09.635 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:33:09 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:33:09.635 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:33:09 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:33:09.635 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:33:09 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:33:09.635 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:33:09 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:33:09.635 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:33:09 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:33:09.635 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:33:09 smithi102 ceph-mon[35442]: Adjusting osd_memory_target on smithi159 to 5696M 2024-09-10T09:33:09.635 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:33:09 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:33:09.636 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:33:09 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:33:09.636 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:33:09 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:33:09.636 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:33:09 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:33:09.636 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:33:09 smithi102 ceph-mon[35442]: from='osd.8 ' entity='osd.8' 2024-09-10T09:33:09.636 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:33:09 smithi102 ceph-mon[35442]: osd.8 [v2:172.21.15.159:6816/2107291799,v1:172.21.15.159:6817/2107291799] boot 2024-09-10T09:33:09.636 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:33:09 smithi102 ceph-mon[35442]: osdmap e57: 9 total, 9 up, 9 in 2024-09-10T09:33:09.636 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:33:09 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-10T09:33:10.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:33:10.105+0000 7f930f000640 1 -- 172.21.15.5:0/2142456857 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9308105920 msgr2=0x7f930810c240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:33:10.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:33:10.105+0000 7f930f000640 1 --2- 172.21.15.5:0/2142456857 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9308105920 0x7f930810c240 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f9300009ca0 tx=0x7f930002a8c0 comp rx=0 tx=0).stop 2024-09-10T09:33:10.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:33:10.105+0000 7f930f000640 1 -- 172.21.15.5:0/2142456857 shutdown_connections 2024-09-10T09:33:10.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:33:10.105+0000 7f930f000640 1 --2- 172.21.15.5:0/2142456857 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9308105920 0x7f930810c240 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:33:10.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:33:10.105+0000 7f930f000640 1 --2- 172.21.15.5:0/2142456857 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9308104f60 0x7f93081053e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:33:10.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:33:10.105+0000 7f930f000640 1 --2- 172.21.15.5:0/2142456857 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9308103e00 0x7f9308104200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:33:10.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:33:10.105+0000 7f930f000640 1 -- 172.21.15.5:0/2142456857 >> 172.21.15.5:0/2142456857 conn(0x7f93080ff5b0 msgr2=0x7f93081019d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:33:10.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:33:10.105+0000 7f930f000640 1 -- 172.21.15.5:0/2142456857 shutdown_connections 2024-09-10T09:33:10.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:33:10.105+0000 7f930f000640 1 -- 172.21.15.5:0/2142456857 wait complete. 2024-09-10T09:33:10.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:33:10.106+0000 7f930f000640 1 Processor -- start 2024-09-10T09:33:10.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:33:10.106+0000 7f930f000640 1 -- start start 2024-09-10T09:33:10.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:33:10.106+0000 7f930f000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9308103e00 0x7f930819a070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:33:10.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:33:10.107+0000 7f930f000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9308104f60 0x7f930819a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:33:10.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:33:10.107+0000 7f930f000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9308105920 0x7f93081a1630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:33:10.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:33:10.107+0000 7f930f000640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f930810fb00 con 0x7f9308103e00 2024-09-10T09:33:10.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:33:10.107+0000 7f930f000640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f930810f980 con 0x7f9308104f60 2024-09-10T09:33:10.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:33:10.107+0000 7f930f000640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f930810fc80 con 0x7f9308105920 2024-09-10T09:33:10.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:33:10.107+0000 7f9307e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9308103e00 0x7f930819a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:33:10.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:33:10.107+0000 7f9307e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9308103e00 0x7f930819a070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55664/0 (socket says 172.21.15.5:55664) 2024-09-10T09:33:10.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:33:10.107+0000 7f9307e00640 1 -- 172.21.15.5:0/2552952825 learned_addr learned my addr 172.21.15.5:0/2552952825 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:33:10.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:33:10.107+0000 7f92ffe00640 1 --2- 172.21.15.5:0/2552952825 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9308104f60 0x7f930819a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:33:10.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:33:10.107+0000 7f930d000640 1 --2- 172.21.15.5:0/2552952825 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9308105920 0x7f93081a1630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:33:10.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:33:10.107+0000 7f9307e00640 1 -- 172.21.15.5:0/2552952825 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9308105920 msgr2=0x7f93081a1630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:33:10.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:33:10.107+0000 7f9307e00640 1 --2- 172.21.15.5:0/2552952825 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9308105920 0x7f93081a1630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:33:10.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:33:10.107+0000 7f9307e00640 1 -- 172.21.15.5:0/2552952825 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9308104f60 msgr2=0x7f930819a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:33:10.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:33:10.107+0000 7f9307e00640 1 --2- 172.21.15.5:0/2552952825 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9308104f60 0x7f930819a5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:33:10.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:33:10.107+0000 7f9307e00640 1 -- 172.21.15.5:0/2552952825 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93081a1ca0 con 0x7f9308103e00 2024-09-10T09:33:10.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:33:10.107+0000 7f92ffe00640 1 --2- 172.21.15.5:0/2552952825 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9308104f60 0x7f930819a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:33:10.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:33:10.108+0000 7f9307e00640 1 --2- 172.21.15.5:0/2552952825 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9308103e00 0x7f930819a070 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f92f800e9f0 tx=0x7f92f800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:33:10.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:33:10.108+0000 7f9305600640 1 -- 172.21.15.5:0/2552952825 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92f800cd80 con 0x7f9308103e00 2024-09-10T09:33:10.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:33:10.108+0000 7f9305600640 1 -- 172.21.15.5:0/2552952825 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f92f8005500 con 0x7f9308103e00 2024-09-10T09:33:10.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:33:10.108+0000 7f930f000640 1 -- 172.21.15.5:0/2552952825 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93081a1f90 con 0x7f9308103e00 2024-09-10T09:33:10.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:33:10.108+0000 7f9305600640 1 -- 172.21.15.5:0/2552952825 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92f8010690 con 0x7f9308103e00 2024-09-10T09:33:10.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:33:10.108+0000 7f930f000640 1 -- 172.21.15.5:0/2552952825 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93081a2390 con 0x7f9308103e00 2024-09-10T09:33:10.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:33:10.109+0000 7f930f000640 1 -- 172.21.15.5:0/2552952825 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f930810b120 con 0x7f9308103e00 2024-09-10T09:33:10.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:33:10.111+0000 7f9305600640 1 -- 172.21.15.5:0/2552952825 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7f92f80046c0 con 0x7f9308103e00 2024-09-10T09:33:10.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:33:10.113+0000 7f9305600640 1 --2- 172.21.15.5:0/2552952825 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f92ec0a2bb0 0x7f92ec0a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:33:10.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:33:10.113+0000 7f9305600640 1 -- 172.21.15.5:0/2552952825 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5858+0+0 (secure 0 0 0) 0x7f92f80ced20 con 0x7f9308103e00 2024-09-10T09:33:10.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:33:10.113+0000 7f92ffe00640 1 --2- 172.21.15.5:0/2552952825 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f92ec0a2bb0 0x7f92ec0a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:33:10.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:33:10.114+0000 7f92ffe00640 1 --2- 172.21.15.5:0/2552952825 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f92ec0a2bb0 0x7f92ec0a5070 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f930819b590 tx=0x7f92f0005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:33:10.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:33:10.116+0000 7f9305600640 1 -- 172.21.15.5:0/2552952825 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f92f80541b0 con 0x7f9308103e00 2024-09-10T09:33:10.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:33:10.312+0000 7f930f000640 1 -- 172.21.15.5:0/2552952825 --> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi005:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f9308062630 con 0x7f92ec0a2bb0 2024-09-10T09:33:10.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:33:10 smithi005 ceph-mon[30350]: purged_snaps scrub starts 2024-09-10T09:33:10.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:33:10 smithi005 ceph-mon[30350]: purged_snaps scrub ok 2024-09-10T09:33:10.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:33:10 smithi005 ceph-mon[30350]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-09-10T09:33:10.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:33:10 smithi005 ceph-mon[30350]: osdmap e58: 9 total, 9 up, 9 in 2024-09-10T09:33:10.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:33:10 smithi159 ceph-mon[35513]: purged_snaps scrub starts 2024-09-10T09:33:10.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:33:10 smithi159 ceph-mon[35513]: purged_snaps scrub ok 2024-09-10T09:33:10.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:33:10 smithi159 ceph-mon[35513]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-09-10T09:33:10.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:33:10 smithi159 ceph-mon[35513]: osdmap e58: 9 total, 9 up, 9 in 2024-09-10T09:33:10.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:33:10 smithi102 ceph-mon[35442]: purged_snaps scrub starts 2024-09-10T09:33:10.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:33:10 smithi102 ceph-mon[35442]: purged_snaps scrub ok 2024-09-10T09:33:10.635 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:33:10 smithi102 ceph-mon[35442]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-09-10T09:33:10.635 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:33:10 smithi102 ceph-mon[35442]: osdmap e58: 9 total, 9 up, 9 in 2024-09-10T09:33:11.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:33:11 smithi005 ceph-mon[30350]: from='client.14370 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi005:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T09:33:11.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:33:11 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T09:33:11.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:33:11 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T09:33:11.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:33:11 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:33:11.557 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:33:11 smithi005 ceph-mon[30350]: osdmap e59: 9 total, 9 up, 9 in 2024-09-10T09:33:11.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:33:11 smithi159 ceph-mon[35513]: from='client.14370 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi005:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T09:33:11.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:33:11 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T09:33:11.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:33:11 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T09:33:11.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:33:11 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:33:11.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:33:11 smithi159 ceph-mon[35513]: osdmap e59: 9 total, 9 up, 9 in 2024-09-10T09:33:11.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:33:11 smithi102 ceph-mon[35442]: from='client.14370 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi005:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T09:33:11.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:33:11 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T09:33:11.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:33:11 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T09:33:11.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:33:11 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:33:11.635 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:33:11 smithi102 ceph-mon[35442]: osdmap e59: 9 total, 9 up, 9 in 2024-09-10T09:33:12.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:33:12 smithi005 ceph-mon[30350]: pgmap v202: 1 pgs: 1 remapped+peering; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-09-10T09:33:12.557 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:33:12 smithi005 ceph-mon[30350]: osdmap e60: 9 total, 9 up, 9 in 2024-09-10T09:33:12.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:33:12 smithi159 ceph-mon[35513]: pgmap v202: 1 pgs: 1 remapped+peering; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-09-10T09:33:12.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:33:12 smithi159 ceph-mon[35513]: osdmap e60: 9 total, 9 up, 9 in 2024-09-10T09:33:12.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:33:12 smithi102 ceph-mon[35442]: pgmap v202: 1 pgs: 1 remapped+peering; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-09-10T09:33:12.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:33:12 smithi102 ceph-mon[35442]: osdmap e60: 9 total, 9 up, 9 in 2024-09-10T09:33:14.249 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:33:13 smithi005 ceph-mon[30350]: osdmap e61: 9 total, 9 up, 9 in 2024-09-10T09:33:14.250 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:33:13 smithi005 ceph-mon[30350]: pgmap v205: 1 pgs: 1 remapped+peering; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-09-10T09:33:14.250 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:33:13 smithi005 ceph-mon[30350]: from='client.? 172.21.15.5:0/3875580177' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "91a5b6cc-08da-47be-af12-d0896a849309"}]: dispatch 2024-09-10T09:33:14.250 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:33:13 smithi005 ceph-mon[30350]: from='client.? 172.21.15.5:0/3875580177' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "91a5b6cc-08da-47be-af12-d0896a849309"}]': finished 2024-09-10T09:33:14.250 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:33:13 smithi005 ceph-mon[30350]: osdmap e62: 10 total, 9 up, 10 in 2024-09-10T09:33:14.250 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:33:13 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-10T09:33:14.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:33:13 smithi159 ceph-mon[35513]: osdmap e61: 9 total, 9 up, 9 in 2024-09-10T09:33:14.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:33:13 smithi159 ceph-mon[35513]: pgmap v205: 1 pgs: 1 remapped+peering; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-09-10T09:33:14.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:33:13 smithi159 ceph-mon[35513]: from='client.? 172.21.15.5:0/3875580177' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "91a5b6cc-08da-47be-af12-d0896a849309"}]: dispatch 2024-09-10T09:33:14.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:33:13 smithi159 ceph-mon[35513]: from='client.? 172.21.15.5:0/3875580177' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "91a5b6cc-08da-47be-af12-d0896a849309"}]': finished 2024-09-10T09:33:14.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:33:13 smithi159 ceph-mon[35513]: osdmap e62: 10 total, 9 up, 10 in 2024-09-10T09:33:14.351 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:33:13 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-10T09:33:14.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:33:13 smithi102 ceph-mon[35442]: osdmap e61: 9 total, 9 up, 9 in 2024-09-10T09:33:14.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:33:13 smithi102 ceph-mon[35442]: pgmap v205: 1 pgs: 1 remapped+peering; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-09-10T09:33:14.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:33:13 smithi102 ceph-mon[35442]: from='client.? 172.21.15.5:0/3875580177' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "91a5b6cc-08da-47be-af12-d0896a849309"}]: dispatch 2024-09-10T09:33:14.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:33:13 smithi102 ceph-mon[35442]: from='client.? 172.21.15.5:0/3875580177' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "91a5b6cc-08da-47be-af12-d0896a849309"}]': finished 2024-09-10T09:33:14.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:33:13 smithi102 ceph-mon[35442]: osdmap e62: 10 total, 9 up, 10 in 2024-09-10T09:33:14.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:33:13 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-10T09:33:15.306 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:33:14 smithi005 ceph-mon[30350]: from='client.? 172.21.15.5:0/3053856656' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T09:33:15.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:33:14 smithi159 ceph-mon[35513]: from='client.? 172.21.15.5:0/3053856656' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T09:33:15.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:33:14 smithi102 ceph-mon[35442]: from='client.? 172.21.15.5:0/3053856656' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T09:33:16.251 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:33:15 smithi159 ceph-mon[35513]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 127 KiB/s, 0 objects/s recovering 2024-09-10T09:33:16.306 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:33:15 smithi005 ceph-mon[30350]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 127 KiB/s, 0 objects/s recovering 2024-09-10T09:33:16.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:33:15 smithi102 ceph-mon[35442]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 127 KiB/s, 0 objects/s recovering 2024-09-10T09:33:17.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:33:17 smithi159 ceph-mon[35513]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-10T09:33:17.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:33:17 smithi102 ceph-mon[35442]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-10T09:33:18.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:33:17 smithi005 ceph-mon[30350]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-10T09:33:19.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:33:19 smithi159 ceph-mon[35513]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail; 77 KiB/s, 0 objects/s recovering 2024-09-10T09:33:19.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:33:19 smithi102 ceph-mon[35442]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail; 77 KiB/s, 0 objects/s recovering 2024-09-10T09:33:20.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:33:19 smithi005 ceph-mon[30350]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail; 77 KiB/s, 0 objects/s recovering 2024-09-10T09:33:21.683 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:33:21 smithi005 ceph-mon[30350]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail; 68 KiB/s, 0 objects/s recovering 2024-09-10T09:33:21.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:33:21 smithi159 ceph-mon[35513]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail; 68 KiB/s, 0 objects/s recovering 2024-09-10T09:33:21.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:33:21 smithi102 ceph-mon[35442]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail; 68 KiB/s, 0 objects/s recovering 2024-09-10T09:33:23.306 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:33:23 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-09-10T09:33:23.306 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:33:23 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:33:23.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:33:23 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-09-10T09:33:23.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:33:23 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:33:23.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:33:23 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-09-10T09:33:23.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:33:23 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:33:24.306 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:33:24 smithi005 ceph-mon[30350]: Deploying daemon osd.9 on smithi005 2024-09-10T09:33:24.306 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:33:24 smithi005 ceph-mon[30350]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-10T09:33:24.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:33:24 smithi102 ceph-mon[35442]: Deploying daemon osd.9 on smithi005 2024-09-10T09:33:24.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:33:24 smithi102 ceph-mon[35442]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-10T09:33:24.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:33:24 smithi159 ceph-mon[35513]: Deploying daemon osd.9 on smithi005 2024-09-10T09:33:24.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:33:24 smithi159 ceph-mon[35513]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-10T09:33:25.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:33:25 smithi005 ceph-mon[30350]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-10T09:33:25.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:33:25 smithi159 ceph-mon[35513]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-10T09:33:25.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:33:25 smithi102 ceph-mon[35442]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-10T09:33:27.769 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:33:27 smithi005 ceph-mon[30350]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-09-10T09:33:27.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:33:27 smithi159 ceph-mon[35513]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-09-10T09:33:27.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:33:27 smithi102 ceph-mon[35442]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-09-10T09:33:29.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:33:29 smithi159 ceph-mon[35513]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-09-10T09:33:29.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:33:29 smithi102 ceph-mon[35442]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-09-10T09:33:29.935 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:33:29 smithi005 ceph-mon[30350]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-09-10T09:33:30.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:33:30 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:33:30.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:33:30 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:33:30.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:33:30 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:33:31.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:33:30 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:33:31.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:33:30 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:33:31.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:33:30 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:33:31.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:33:30 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:33:31.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:33:30 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:33:31.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:33:30 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:33:32.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:33:31 smithi005 ceph-mon[30350]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-09-10T09:33:32.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:33:31 smithi159 ceph-mon[35513]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-09-10T09:33:32.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:33:31 smithi102 ceph-mon[35442]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-09-10T09:33:33.783 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:33:33 smithi005 ceph-mon[30350]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-09-10T09:33:33.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:33:33 smithi159 ceph-mon[35513]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-09-10T09:33:33.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:33:33 smithi102 ceph-mon[35442]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-09-10T09:33:34.673 INFO:teuthology.orchestra.run.smithi005.stdout:Created osd(s) 9 on host 'smithi005' 2024-09-10T09:33:34.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:33:34.670+0000 7f9305600640 1 -- 172.21.15.5:0/2552952825 <== mgr.14150 v2:172.21.15.5:6800/2707348081 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f9308062630 con 0x7f92ec0a2bb0 2024-09-10T09:33:34.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:33:34.673+0000 7f930f000640 1 -- 172.21.15.5:0/2552952825 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f92ec0a2bb0 msgr2=0x7f92ec0a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:33:34.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:33:34.673+0000 7f930f000640 1 --2- 172.21.15.5:0/2552952825 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f92ec0a2bb0 0x7f92ec0a5070 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f930819b590 tx=0x7f92f0005f90 comp rx=0 tx=0).stop 2024-09-10T09:33:34.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:33:34.673+0000 7f930f000640 1 -- 172.21.15.5:0/2552952825 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9308103e00 msgr2=0x7f930819a070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:33:34.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:33:34.673+0000 7f930f000640 1 --2- 172.21.15.5:0/2552952825 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9308103e00 0x7f930819a070 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f92f800e9f0 tx=0x7f92f800edf0 comp rx=0 tx=0).stop 2024-09-10T09:33:34.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:33:34.673+0000 7f930f000640 1 -- 172.21.15.5:0/2552952825 shutdown_connections 2024-09-10T09:33:34.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:33:34.673+0000 7f930f000640 1 --2- 172.21.15.5:0/2552952825 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f92ec0a2bb0 0x7f92ec0a5070 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:33:34.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:33:34.673+0000 7f930f000640 1 --2- 172.21.15.5:0/2552952825 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9308105920 0x7f93081a1630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:33:34.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:33:34.673+0000 7f930f000640 1 --2- 172.21.15.5:0/2552952825 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9308104f60 0x7f930819a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:33:34.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:33:34.673+0000 7f930f000640 1 --2- 172.21.15.5:0/2552952825 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9308103e00 0x7f930819a070 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:33:34.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:33:34.673+0000 7f930f000640 1 -- 172.21.15.5:0/2552952825 >> 172.21.15.5:0/2552952825 conn(0x7f93080ff5b0 msgr2=0x7f93081082f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:33:34.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:33:34.674+0000 7f930f000640 1 -- 172.21.15.5:0/2552952825 shutdown_connections 2024-09-10T09:33:34.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:33:34.674+0000 7f930f000640 1 -- 172.21.15.5:0/2552952825 wait complete. 2024-09-10T09:33:35.450 DEBUG:teuthology.orchestra.run.smithi005:osd.9> sudo journalctl -f -n 0 -u ceph-bade4b50-6f56-11ef-bcea-c7b262605968@osd.9.service 2024-09-10T09:33:35.454 INFO:tasks.cephadm:Deploying osd.10 on smithi102 with /dev/vg_nvme/lv_1... 2024-09-10T09:33:35.455 DEBUG:teuthology.orchestra.run.smithi102:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_1 2024-09-10T09:33:35.708 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T09:33:35.828 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:33:35 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:33:35.828 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:33:35 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:33:35.828 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:33:35 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:33:35.829 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:33:35 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:33:35.829 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:33:35 smithi102 ceph-mon[35442]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-09-10T09:33:36.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:33:35 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:33:36.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:33:35 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:33:36.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:33:35 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:33:36.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:33:35 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:33:36.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:33:35 smithi005 ceph-mon[30350]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-09-10T09:33:36.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:33:35 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:33:36.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:33:35 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:33:36.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:33:35 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:33:36.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:33:35 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:33:36.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:33:35 smithi159 ceph-mon[35513]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-09-10T09:33:36.668 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 10 09:33:36 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-9[65288]: 2024-09-10T09:33:36.425+0000 7f84bcc2f640 -1 osd.9 0 log_to_monitors true 2024-09-10T09:33:36.957 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:33:36 smithi102 ceph-mon[35442]: from='osd.9 [v2:172.21.15.5:6826/902431281,v1:172.21.15.5:6827/902431281]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-09-10T09:33:36.957 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:33:36 smithi102 ceph-mon[35442]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-09-10T09:33:37.035 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:33:36 smithi005 ceph-mon[30350]: from='osd.9 [v2:172.21.15.5:6826/902431281,v1:172.21.15.5:6827/902431281]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-09-10T09:33:37.035 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:33:36 smithi005 ceph-mon[30350]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-09-10T09:33:37.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:33:36 smithi159 ceph-mon[35513]: from='osd.9 [v2:172.21.15.5:6826/902431281,v1:172.21.15.5:6827/902431281]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-09-10T09:33:37.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:33:36 smithi159 ceph-mon[35513]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-09-10T09:33:37.758 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:33:37 smithi005 ceph-mon[30350]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-09-10T09:33:37.758 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:33:37 smithi005 ceph-mon[30350]: osdmap e63: 10 total, 9 up, 10 in 2024-09-10T09:33:37.759 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:33:37 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-10T09:33:37.759 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:33:37 smithi005 ceph-mon[30350]: from='osd.9 [v2:172.21.15.5:6826/902431281,v1:172.21.15.5:6827/902431281]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi005", "root=default"]}]: dispatch 2024-09-10T09:33:37.759 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:33:37 smithi005 ceph-mon[30350]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi005", "root=default"]}]: dispatch 2024-09-10T09:33:37.759 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:33:37 smithi005 ceph-mon[30350]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-09-10T09:33:37.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:33:37 smithi102 ceph-mon[35442]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-09-10T09:33:37.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:33:37 smithi102 ceph-mon[35442]: osdmap e63: 10 total, 9 up, 10 in 2024-09-10T09:33:37.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:33:37 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-10T09:33:37.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:33:37 smithi102 ceph-mon[35442]: from='osd.9 [v2:172.21.15.5:6826/902431281,v1:172.21.15.5:6827/902431281]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi005", "root=default"]}]: dispatch 2024-09-10T09:33:37.885 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:33:37 smithi102 ceph-mon[35442]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi005", "root=default"]}]: dispatch 2024-09-10T09:33:37.885 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:33:37 smithi102 ceph-mon[35442]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-09-10T09:33:38.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:33:37 smithi159 ceph-mon[35513]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-09-10T09:33:38.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:33:37 smithi159 ceph-mon[35513]: osdmap e63: 10 total, 9 up, 10 in 2024-09-10T09:33:38.101 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:33:37 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-10T09:33:38.101 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:33:37 smithi159 ceph-mon[35513]: from='osd.9 [v2:172.21.15.5:6826/902431281,v1:172.21.15.5:6827/902431281]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi005", "root=default"]}]: dispatch 2024-09-10T09:33:38.101 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:33:37 smithi159 ceph-mon[35513]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi005", "root=default"]}]: dispatch 2024-09-10T09:33:38.101 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:33:37 smithi159 ceph-mon[35513]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-09-10T09:33:38.423 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-10T09:33:38.452 DEBUG:teuthology.orchestra.run.smithi102:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi102:vg_nvme/lv_1 2024-09-10T09:33:38.556 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 10 09:33:38 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-9[65288]: 2024-09-10T09:33:38.182+0000 7f84b7e00640 -1 osd.9 0 waiting for initial osdmap 2024-09-10T09:33:38.557 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 10 09:33:38 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-9[65288]: 2024-09-10T09:33:38.210+0000 7f84ad400640 -1 osd.9 64 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T09:33:38.704 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T09:33:38.934 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:33:38 smithi102 ceph-mon[35442]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi005", "root=default"]}]': finished 2024-09-10T09:33:38.934 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:33:38 smithi102 ceph-mon[35442]: osdmap e64: 10 total, 9 up, 10 in 2024-09-10T09:33:38.934 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:33:38 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-10T09:33:38.934 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:33:38 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-10T09:33:38.935 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:33:38 smithi102 ceph-mon[35442]: from='osd.9 ' entity='osd.9' 2024-09-10T09:33:38.993 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:33:38 smithi005 ceph-mon[30350]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi005", "root=default"]}]': finished 2024-09-10T09:33:38.993 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:33:38 smithi005 ceph-mon[30350]: osdmap e64: 10 total, 9 up, 10 in 2024-09-10T09:33:38.993 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:33:38 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-10T09:33:38.993 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:33:38 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-10T09:33:38.993 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:33:38 smithi005 ceph-mon[30350]: from='osd.9 ' entity='osd.9' 2024-09-10T09:33:39.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:33:38 smithi159 ceph-mon[35513]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi005", "root=default"]}]': finished 2024-09-10T09:33:39.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:33:38 smithi159 ceph-mon[35513]: osdmap e64: 10 total, 9 up, 10 in 2024-09-10T09:33:39.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:33:38 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-10T09:33:39.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:33:38 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-10T09:33:39.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:33:38 smithi159 ceph-mon[35513]: from='osd.9 ' entity='osd.9' 2024-09-10T09:33:39.277 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-10T09:33:39.235+0000 7f823f5d2640 1 -- 172.21.15.102:0/2607837634 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8238102690 msgr2=0x7f8238108f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:33:39.277 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-10T09:33:39.235+0000 7f823f5d2640 1 --2- 172.21.15.102:0/2607837634 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8238102690 0x7f8238108f20 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f8230009ca0 tx=0x7f823002a8c0 comp rx=0 tx=0).stop 2024-09-10T09:33:39.277 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-10T09:33:39.235+0000 7f823f5d2640 1 -- 172.21.15.102:0/2607837634 shutdown_connections 2024-09-10T09:33:39.277 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-10T09:33:39.235+0000 7f823f5d2640 1 --2- 172.21.15.102:0/2607837634 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8238102690 0x7f8238108f20 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:33:39.277 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-10T09:33:39.235+0000 7f823f5d2640 1 --2- 172.21.15.102:0/2607837634 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8238101cd0 0x7f8238102150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:33:39.277 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-10T09:33:39.235+0000 7f823f5d2640 1 --2- 172.21.15.102:0/2607837634 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8238100ad0 0x7f8238100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:33:39.277 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-10T09:33:39.235+0000 7f823f5d2640 1 -- 172.21.15.102:0/2607837634 >> 172.21.15.102:0/2607837634 conn(0x7f82380fc280 msgr2=0x7f82380fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:33:39.277 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-10T09:33:39.235+0000 7f823f5d2640 1 -- 172.21.15.102:0/2607837634 shutdown_connections 2024-09-10T09:33:39.278 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-10T09:33:39.235+0000 7f823f5d2640 1 -- 172.21.15.102:0/2607837634 wait complete. 2024-09-10T09:33:39.278 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-10T09:33:39.235+0000 7f823f5d2640 1 Processor -- start 2024-09-10T09:33:39.278 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-10T09:33:39.236+0000 7f823f5d2640 1 -- start start 2024-09-10T09:33:39.278 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-10T09:33:39.236+0000 7f823f5d2640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8238100ad0 0x7f823806ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:33:39.278 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-10T09:33:39.236+0000 7f823f5d2640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8238101cd0 0x7f823819cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:33:39.278 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-10T09:33:39.236+0000 7f823f5d2640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8238102690 0x7f823819f210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:33:39.278 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-10T09:33:39.236+0000 7f823f5d2640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f823810c750 con 0x7f8238101cd0 2024-09-10T09:33:39.278 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-10T09:33:39.236+0000 7f823f5d2640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f823810c5d0 con 0x7f8238102690 2024-09-10T09:33:39.278 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-10T09:33:39.236+0000 7f823f5d2640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f823810c8d0 con 0x7f8238100ad0 2024-09-10T09:33:39.278 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-10T09:33:39.236+0000 7f823db6f640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8238102690 0x7f823819f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:33:39.278 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-10T09:33:39.236+0000 7f823db6f640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8238102690 0x7f823819f210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.102:52942/0 (socket says 172.21.15.102:52942) 2024-09-10T09:33:39.279 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-10T09:33:39.236+0000 7f823db6f640 1 -- 172.21.15.102:0/2490420677 learned_addr learned my addr 172.21.15.102:0/2490420677 (peer_addr_for_me v2:172.21.15.102:0/0) 2024-09-10T09:33:39.279 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-10T09:33:39.236+0000 7f823d36e640 1 --2- 172.21.15.102:0/2490420677 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8238100ad0 0x7f823806ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:33:39.279 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-10T09:33:39.236+0000 7f823cb6d640 1 --2- 172.21.15.102:0/2490420677 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8238101cd0 0x7f823819cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:33:39.279 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-10T09:33:39.236+0000 7f823db6f640 1 -- 172.21.15.102:0/2490420677 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8238100ad0 msgr2=0x7f823806ab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:33:39.279 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-10T09:33:39.236+0000 7f823db6f640 1 --2- 172.21.15.102:0/2490420677 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8238100ad0 0x7f823806ab10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:33:39.279 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-10T09:33:39.236+0000 7f823db6f640 1 -- 172.21.15.102:0/2490420677 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8238101cd0 msgr2=0x7f823819cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:33:39.279 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-10T09:33:39.236+0000 7f823db6f640 1 --2- 172.21.15.102:0/2490420677 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8238101cd0 0x7f823819cbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:33:39.279 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-10T09:33:39.236+0000 7f823db6f640 1 -- 172.21.15.102:0/2490420677 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f823819fad0 con 0x7f8238102690 2024-09-10T09:33:39.279 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-10T09:33:39.237+0000 7f823db6f640 1 --2- 172.21.15.102:0/2490420677 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8238102690 0x7f823819f210 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f8230005e30 tx=0x7f8230031ee0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:33:39.279 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-10T09:33:39.237+0000 7f822e7fc640 1 -- 172.21.15.102:0/2490420677 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8230003c30 con 0x7f8238102690 2024-09-10T09:33:39.279 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-10T09:33:39.237+0000 7f822e7fc640 1 -- 172.21.15.102:0/2490420677 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8230003dd0 con 0x7f8238102690 2024-09-10T09:33:39.279 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-10T09:33:39.237+0000 7f823f5d2640 1 -- 172.21.15.102:0/2490420677 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f823819fd60 con 0x7f8238102690 2024-09-10T09:33:39.280 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-10T09:33:39.237+0000 7f822e7fc640 1 -- 172.21.15.102:0/2490420677 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f823002fde0 con 0x7f8238102690 2024-09-10T09:33:39.280 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-10T09:33:39.237+0000 7f823f5d2640 1 -- 172.21.15.102:0/2490420677 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82381a0240 con 0x7f8238102690 2024-09-10T09:33:39.280 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-10T09:33:39.239+0000 7f823f5d2640 1 -- 172.21.15.102:0/2490420677 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8238107e80 con 0x7f8238102690 2024-09-10T09:33:39.280 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-10T09:33:39.239+0000 7f822e7fc640 1 -- 172.21.15.102:0/2490420677 <== mon.2 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7f8230004050 con 0x7f8238102690 2024-09-10T09:33:39.280 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-10T09:33:39.240+0000 7f822e7fc640 1 --2- 172.21.15.102:0/2490420677 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f82240a2b80 0x7f82240a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:33:39.280 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-10T09:33:39.240+0000 7f822e7fc640 1 -- 172.21.15.102:0/2490420677 <== mon.2 v2:172.21.15.102:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 6261+0+0 (secure 0 0 0) 0x7f82300efcb0 con 0x7f8238102690 2024-09-10T09:33:39.280 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-10T09:33:39.240+0000 7f823d36e640 1 --2- 172.21.15.102:0/2490420677 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f82240a2b80 0x7f82240a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:33:39.280 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-10T09:33:39.241+0000 7f823d36e640 1 --2- 172.21.15.102:0/2490420677 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f82240a2b80 0x7f82240a5040 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f823806b050 tx=0x7f822800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:33:39.280 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-10T09:33:39.244+0000 7f822e7fc640 1 -- 172.21.15.102:0/2490420677 <== mon.2 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f82300bd530 con 0x7f8238102690 2024-09-10T09:33:39.424 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-10T09:33:39.423+0000 7f823f5d2640 1 -- 172.21.15.102:0/2490420677 --> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi102:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f8238062630 con 0x7f82240a2b80 2024-09-10T09:33:40.051 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:33:39 smithi102 ceph-mon[35442]: purged_snaps scrub starts 2024-09-10T09:33:40.051 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:33:39 smithi102 ceph-mon[35442]: purged_snaps scrub ok 2024-09-10T09:33:40.051 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:33:39 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-10T09:33:40.051 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:33:39 smithi102 ceph-mon[35442]: osd.9 [v2:172.21.15.5:6826/902431281,v1:172.21.15.5:6827/902431281] boot 2024-09-10T09:33:40.051 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:33:39 smithi102 ceph-mon[35442]: osdmap e65: 10 total, 10 up, 10 in 2024-09-10T09:33:40.051 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:33:39 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-10T09:33:40.052 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:33:39 smithi102 ceph-mon[35442]: from='client.24356 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi102:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T09:33:40.052 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:33:39 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T09:33:40.052 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:33:39 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T09:33:40.052 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:33:39 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:33:40.052 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:33:39 smithi102 ceph-mon[35442]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-09-10T09:33:40.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:33:39 smithi005 ceph-mon[30350]: purged_snaps scrub starts 2024-09-10T09:33:40.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:33:39 smithi005 ceph-mon[30350]: purged_snaps scrub ok 2024-09-10T09:33:40.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:33:39 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-10T09:33:40.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:33:39 smithi005 ceph-mon[30350]: osd.9 [v2:172.21.15.5:6826/902431281,v1:172.21.15.5:6827/902431281] boot 2024-09-10T09:33:40.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:33:39 smithi005 ceph-mon[30350]: osdmap e65: 10 total, 10 up, 10 in 2024-09-10T09:33:40.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:33:39 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-10T09:33:40.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:33:39 smithi005 ceph-mon[30350]: from='client.24356 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi102:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T09:33:40.057 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:33:39 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T09:33:40.057 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:33:39 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T09:33:40.057 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:33:39 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:33:40.057 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:33:39 smithi005 ceph-mon[30350]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-09-10T09:33:40.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:33:39 smithi159 ceph-mon[35513]: purged_snaps scrub starts 2024-09-10T09:33:40.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:33:39 smithi159 ceph-mon[35513]: purged_snaps scrub ok 2024-09-10T09:33:40.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:33:39 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-10T09:33:40.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:33:39 smithi159 ceph-mon[35513]: osd.9 [v2:172.21.15.5:6826/902431281,v1:172.21.15.5:6827/902431281] boot 2024-09-10T09:33:40.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:33:39 smithi159 ceph-mon[35513]: osdmap e65: 10 total, 10 up, 10 in 2024-09-10T09:33:40.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:33:39 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-10T09:33:40.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:33:39 smithi159 ceph-mon[35513]: from='client.24356 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi102:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T09:33:40.101 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:33:39 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T09:33:40.101 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:33:39 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T09:33:40.101 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:33:39 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:33:40.101 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:33:39 smithi159 ceph-mon[35513]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-09-10T09:33:41.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:33:40 smithi159 ceph-mon[35513]: Detected new or changed devices on smithi005 2024-09-10T09:33:41.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:33:40 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:33:41.101 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:33:40 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:33:41.101 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:33:40 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:33:41.101 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:33:40 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:33:41.101 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:33:40 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:33:41.101 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:33:40 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:33:41.101 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:33:40 smithi159 ceph-mon[35513]: Adjusting osd_memory_target on smithi005 to 4273M 2024-09-10T09:33:41.101 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:33:40 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:33:41.101 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:33:40 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:33:41.101 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:33:40 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:33:41.101 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:33:40 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:33:41.101 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:33:40 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:33:41.101 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:33:40 smithi159 ceph-mon[35513]: osdmap e66: 10 total, 10 up, 10 in 2024-09-10T09:33:41.101 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:33:40 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:33:41.102 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:33:40 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:33:41.102 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:33:40 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:33:41.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:33:40 smithi102 ceph-mon[35442]: Detected new or changed devices on smithi005 2024-09-10T09:33:41.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:33:40 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:33:41.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:33:40 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:33:41.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:33:40 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:33:41.135 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:33:40 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:33:41.135 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:33:40 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:33:41.135 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:33:40 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:33:41.135 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:33:40 smithi102 ceph-mon[35442]: Adjusting osd_memory_target on smithi005 to 4273M 2024-09-10T09:33:41.135 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:33:40 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:33:41.135 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:33:40 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:33:41.135 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:33:40 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:33:41.135 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:33:40 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:33:41.135 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:33:40 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:33:41.135 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:33:40 smithi102 ceph-mon[35442]: osdmap e66: 10 total, 10 up, 10 in 2024-09-10T09:33:41.135 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:33:40 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:33:41.135 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:33:40 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:33:41.135 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:33:40 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:33:41.306 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:33:40 smithi005 ceph-mon[30350]: Detected new or changed devices on smithi005 2024-09-10T09:33:41.306 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:33:40 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:33:41.306 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:33:40 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:33:41.306 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:33:40 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:33:41.306 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:33:40 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:33:41.307 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:33:40 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:33:41.307 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:33:40 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:33:41.307 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:33:40 smithi005 ceph-mon[30350]: Adjusting osd_memory_target on smithi005 to 4273M 2024-09-10T09:33:41.307 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:33:40 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:33:41.307 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:33:40 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:33:41.307 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:33:40 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:33:41.307 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:33:40 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:33:41.307 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:33:40 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:33:41.307 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:33:40 smithi005 ceph-mon[30350]: osdmap e66: 10 total, 10 up, 10 in 2024-09-10T09:33:41.307 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:33:40 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:33:41.307 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:33:40 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:33:41.307 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:33:40 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:33:42.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:33:41 smithi102 ceph-mon[35442]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-09-10T09:33:42.306 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:33:41 smithi005 ceph-mon[30350]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-09-10T09:33:42.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:33:41 smithi159 ceph-mon[35513]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-09-10T09:33:43.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:33:42 smithi102 ceph-mon[35442]: from='client.? 172.21.15.102:0/1119653723' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2211030c-b00e-402b-b4d7-c587eb29a490"}]: dispatch 2024-09-10T09:33:43.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:33:42 smithi102 ceph-mon[35442]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2211030c-b00e-402b-b4d7-c587eb29a490"}]: dispatch 2024-09-10T09:33:43.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:33:42 smithi102 ceph-mon[35442]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2211030c-b00e-402b-b4d7-c587eb29a490"}]': finished 2024-09-10T09:33:43.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:33:42 smithi102 ceph-mon[35442]: osdmap e67: 11 total, 10 up, 11 in 2024-09-10T09:33:43.135 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:33:42 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-10T09:33:43.135 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:33:42 smithi102 ceph-mon[35442]: from='client.? 172.21.15.102:0/3724793740' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T09:33:43.306 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:33:42 smithi005 ceph-mon[30350]: from='client.? 172.21.15.102:0/1119653723' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2211030c-b00e-402b-b4d7-c587eb29a490"}]: dispatch 2024-09-10T09:33:43.306 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:33:42 smithi005 ceph-mon[30350]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2211030c-b00e-402b-b4d7-c587eb29a490"}]: dispatch 2024-09-10T09:33:43.306 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:33:42 smithi005 ceph-mon[30350]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2211030c-b00e-402b-b4d7-c587eb29a490"}]': finished 2024-09-10T09:33:43.306 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:33:42 smithi005 ceph-mon[30350]: osdmap e67: 11 total, 10 up, 11 in 2024-09-10T09:33:43.306 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:33:42 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-10T09:33:43.306 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:33:42 smithi005 ceph-mon[30350]: from='client.? 172.21.15.102:0/3724793740' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T09:33:43.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:33:42 smithi159 ceph-mon[35513]: from='client.? 172.21.15.102:0/1119653723' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2211030c-b00e-402b-b4d7-c587eb29a490"}]: dispatch 2024-09-10T09:33:43.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:33:42 smithi159 ceph-mon[35513]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2211030c-b00e-402b-b4d7-c587eb29a490"}]: dispatch 2024-09-10T09:33:43.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:33:42 smithi159 ceph-mon[35513]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2211030c-b00e-402b-b4d7-c587eb29a490"}]': finished 2024-09-10T09:33:43.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:33:42 smithi159 ceph-mon[35513]: osdmap e67: 11 total, 10 up, 11 in 2024-09-10T09:33:43.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:33:42 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-10T09:33:43.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:33:42 smithi159 ceph-mon[35513]: from='client.? 172.21.15.102:0/3724793740' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T09:33:44.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:33:43 smithi102 ceph-mon[35442]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-09-10T09:33:44.306 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:33:43 smithi005 ceph-mon[30350]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-09-10T09:33:44.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:33:43 smithi159 ceph-mon[35513]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-09-10T09:33:45.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:33:45 smithi159 ceph-mon[35513]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-09-10T09:33:45.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:33:45 smithi102 ceph-mon[35442]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-09-10T09:33:46.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:33:45 smithi005 ceph-mon[30350]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-09-10T09:33:47.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:33:47 smithi159 ceph-mon[35513]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-09-10T09:33:47.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:33:47 smithi102 ceph-mon[35442]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-09-10T09:33:48.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:33:47 smithi005 ceph-mon[30350]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-09-10T09:33:49.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:33:49 smithi159 ceph-mon[35513]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-09-10T09:33:49.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:33:49 smithi102 ceph-mon[35442]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-09-10T09:33:49.985 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:33:49 smithi005 ceph-mon[30350]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-09-10T09:33:50.837 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:33:50 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-09-10T09:33:50.837 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:33:50 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:33:50.837 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:33:50 smithi102 ceph-mon[35442]: Deploying daemon osd.10 on smithi102 2024-09-10T09:33:50.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:33:50 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-09-10T09:33:50.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:33:50 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:33:50.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:33:50 smithi159 ceph-mon[35513]: Deploying daemon osd.10 on smithi102 2024-09-10T09:33:51.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:33:50 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-09-10T09:33:51.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:33:50 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:33:51.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:33:50 smithi005 ceph-mon[30350]: Deploying daemon osd.10 on smithi102 2024-09-10T09:33:51.731 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:33:51 smithi005 ceph-mon[30350]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-09-10T09:33:51.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:33:51 smithi159 ceph-mon[35513]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-09-10T09:33:51.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:33:51 smithi102 ceph-mon[35442]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-09-10T09:33:53.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:33:53 smithi159 ceph-mon[35513]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-09-10T09:33:53.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:33:53 smithi102 ceph-mon[35442]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-09-10T09:33:54.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:33:53 smithi005 ceph-mon[30350]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-09-10T09:33:55.822 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:33:55 smithi102 ceph-mon[35442]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-09-10T09:33:55.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:33:55 smithi159 ceph-mon[35513]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-09-10T09:33:56.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:33:55 smithi005 ceph-mon[30350]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-09-10T09:33:56.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:33:56 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:33:56.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:33:56 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:33:56.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:33:56 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:33:56.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:33:56 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:33:56.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:33:56 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:33:56.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:33:56 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:33:57.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:33:56 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:33:57.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:33:56 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:33:57.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:33:56 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:33:57.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:33:57 smithi159 ceph-mon[35513]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-09-10T09:33:57.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:33:57 smithi102 ceph-mon[35442]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-09-10T09:33:58.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:33:57 smithi005 ceph-mon[30350]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-09-10T09:34:00.114 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-10T09:34:00.113+0000 7f822e7fc640 1 -- 172.21.15.102:0/2490420677 <== mgr.14150 v2:172.21.15.5:6800/2707348081 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f8238062630 con 0x7f82240a2b80 2024-09-10T09:34:00.115 INFO:teuthology.orchestra.run.smithi102.stdout:Created osd(s) 10 on host 'smithi102' 2024-09-10T09:34:00.117 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-10T09:34:00.115+0000 7f823f5d2640 1 -- 172.21.15.102:0/2490420677 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f82240a2b80 msgr2=0x7f82240a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:34:00.117 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-10T09:34:00.115+0000 7f823f5d2640 1 --2- 172.21.15.102:0/2490420677 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f82240a2b80 0x7f82240a5040 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f823806b050 tx=0x7f822800b390 comp rx=0 tx=0).stop 2024-09-10T09:34:00.117 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-10T09:34:00.115+0000 7f823f5d2640 1 -- 172.21.15.102:0/2490420677 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8238102690 msgr2=0x7f823819f210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:34:00.117 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-10T09:34:00.115+0000 7f823f5d2640 1 --2- 172.21.15.102:0/2490420677 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8238102690 0x7f823819f210 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f8230005e30 tx=0x7f8230031ee0 comp rx=0 tx=0).stop 2024-09-10T09:34:00.117 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-10T09:34:00.115+0000 7f823f5d2640 1 -- 172.21.15.102:0/2490420677 shutdown_connections 2024-09-10T09:34:00.117 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-10T09:34:00.116+0000 7f823f5d2640 1 --2- 172.21.15.102:0/2490420677 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f82240a2b80 0x7f82240a5040 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:34:00.117 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-10T09:34:00.116+0000 7f823f5d2640 1 --2- 172.21.15.102:0/2490420677 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8238102690 0x7f823819f210 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:34:00.117 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-10T09:34:00.116+0000 7f823f5d2640 1 --2- 172.21.15.102:0/2490420677 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8238101cd0 0x7f823819cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:34:00.117 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-10T09:34:00.116+0000 7f823f5d2640 1 --2- 172.21.15.102:0/2490420677 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8238100ad0 0x7f823806ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:34:00.117 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-10T09:34:00.116+0000 7f823f5d2640 1 -- 172.21.15.102:0/2490420677 >> 172.21.15.102:0/2490420677 conn(0x7f82380fc280 msgr2=0x7f8238105050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:34:00.118 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-10T09:34:00.116+0000 7f823f5d2640 1 -- 172.21.15.102:0/2490420677 shutdown_connections 2024-09-10T09:34:00.118 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-10T09:34:00.116+0000 7f823f5d2640 1 -- 172.21.15.102:0/2490420677 wait complete. 2024-09-10T09:34:00.786 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:34:00 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:34:00.786 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:34:00 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:34:00.786 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:34:00 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:34:00.786 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:34:00 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:34:00.786 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:34:00 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:34:00.786 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:34:00 smithi102 ceph-mon[35442]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-09-10T09:34:00.786 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:34:00 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:34:00.786 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:34:00 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:34:00.786 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:34:00 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:34:00.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:34:00 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:34:00.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:34:00 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:34:00.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:34:00 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:34:00.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:34:00 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:34:00.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:34:00 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:34:00.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:34:00 smithi005 ceph-mon[30350]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-09-10T09:34:00.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:34:00 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:34:00.807 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:34:00 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:34:00.807 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:34:00 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:34:00.810 DEBUG:teuthology.orchestra.run.smithi102:osd.10> sudo journalctl -f -n 0 -u ceph-bade4b50-6f56-11ef-bcea-c7b262605968@osd.10.service 2024-09-10T09:34:00.813 INFO:tasks.cephadm:Deploying osd.11 on smithi159 with /dev/vg_nvme/lv_1... 2024-09-10T09:34:00.813 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_1 2024-09-10T09:34:00.841 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:34:00 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:34:00.842 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:34:00 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:34:00.842 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:34:00 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:34:00.842 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:34:00 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:34:00.842 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:34:00 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:34:00.842 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:34:00 smithi159 ceph-mon[35513]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-09-10T09:34:00.842 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:34:00 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:34:00.842 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:34:00 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:34:00.842 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:34:00 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:34:01.072 INFO:teuthology.orchestra.run.smithi159.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.c/config 2024-09-10T09:34:01.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:34:01 smithi159 ceph-mon[35513]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-09-10T09:34:01.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:34:01 smithi102 ceph-mon[35442]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-09-10T09:34:02.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:34:01 smithi005 ceph-mon[30350]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-09-10T09:34:02.316 INFO:journalctl@ceph.osd.10.smithi102.stdout:Sep 10 09:34:02 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-10[59874]: 2024-09-10T09:34:02.083+0000 7f40f9d87640 -1 osd.10 0 log_to_monitors true 2024-09-10T09:34:02.773 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:34:02 smithi159 ceph-mon[35513]: from='osd.10 [v2:172.21.15.102:6824/2853806211,v1:172.21.15.102:6825/2853806211]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-10T09:34:02.773 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:34:02 smithi159 ceph-mon[35513]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-10T09:34:02.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:34:02 smithi102 ceph-mon[35442]: from='osd.10 [v2:172.21.15.102:6824/2853806211,v1:172.21.15.102:6825/2853806211]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-10T09:34:02.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:34:02 smithi102 ceph-mon[35442]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-10T09:34:03.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:34:02 smithi005 ceph-mon[30350]: from='osd.10 [v2:172.21.15.102:6824/2853806211,v1:172.21.15.102:6825/2853806211]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-10T09:34:03.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:34:02 smithi005 ceph-mon[30350]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-10T09:34:03.711 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:34:03 smithi102 ceph-mon[35442]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-09-10T09:34:03.712 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:34:03 smithi102 ceph-mon[35442]: osdmap e68: 11 total, 10 up, 11 in 2024-09-10T09:34:03.712 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:34:03 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-10T09:34:03.712 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:34:03 smithi102 ceph-mon[35442]: from='osd.10 [v2:172.21.15.102:6824/2853806211,v1:172.21.15.102:6825/2853806211]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]: dispatch 2024-09-10T09:34:03.712 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:34:03 smithi102 ceph-mon[35442]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]: dispatch 2024-09-10T09:34:03.712 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:34:03 smithi102 ceph-mon[35442]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-09-10T09:34:03.712 INFO:journalctl@ceph.osd.10.smithi102.stdout:Sep 10 09:34:03 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-10[59874]: 2024-09-10T09:34:03.586+0000 7f40f5d0a640 -1 osd.10 0 waiting for initial osdmap 2024-09-10T09:34:03.712 INFO:journalctl@ceph.osd.10.smithi102.stdout:Sep 10 09:34:03 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-10[59874]: 2024-09-10T09:34:03.598+0000 7f40f1b32640 -1 osd.10 69 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T09:34:03.787 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:34:03 smithi159 ceph-mon[35513]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-09-10T09:34:03.787 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:34:03 smithi159 ceph-mon[35513]: osdmap e68: 11 total, 10 up, 11 in 2024-09-10T09:34:03.787 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:34:03 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-10T09:34:03.787 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:34:03 smithi159 ceph-mon[35513]: from='osd.10 [v2:172.21.15.102:6824/2853806211,v1:172.21.15.102:6825/2853806211]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]: dispatch 2024-09-10T09:34:03.787 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:34:03 smithi159 ceph-mon[35513]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]: dispatch 2024-09-10T09:34:03.787 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:34:03 smithi159 ceph-mon[35513]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-09-10T09:34:03.790 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-09-10T09:34:03.817 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi159:vg_nvme/lv_1 2024-09-10T09:34:04.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:34:03 smithi005 ceph-mon[30350]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-09-10T09:34:04.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:34:03 smithi005 ceph-mon[30350]: osdmap e68: 11 total, 10 up, 11 in 2024-09-10T09:34:04.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:34:03 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-10T09:34:04.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:34:03 smithi005 ceph-mon[30350]: from='osd.10 [v2:172.21.15.102:6824/2853806211,v1:172.21.15.102:6825/2853806211]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]: dispatch 2024-09-10T09:34:04.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:34:03 smithi005 ceph-mon[30350]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]: dispatch 2024-09-10T09:34:04.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:34:03 smithi005 ceph-mon[30350]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-09-10T09:34:04.115 INFO:teuthology.orchestra.run.smithi159.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.c/config 2024-09-10T09:34:04.617 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:34:04.589+0000 7f6785ae2640 1 -- 172.21.15.159:0/2736495161 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6780101c00 msgr2=0x7f6780102000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:34:04.617 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:34:04.589+0000 7f6785ae2640 1 --2- 172.21.15.159:0/2736495161 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6780101c00 0x7f6780102000 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f6770009900 tx=0x7f677002a410 comp rx=0 tx=0).stop 2024-09-10T09:34:04.618 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:34:04.589+0000 7f6785ae2640 1 -- 172.21.15.159:0/2736495161 shutdown_connections 2024-09-10T09:34:04.618 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:34:04.589+0000 7f6785ae2640 1 --2- 172.21.15.159:0/2736495161 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f67801037c0 0x7f678010a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:34:04.618 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:34:04.589+0000 7f6785ae2640 1 --2- 172.21.15.159:0/2736495161 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6780102e00 0x7f6780103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:34:04.618 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:34:04.589+0000 7f6785ae2640 1 --2- 172.21.15.159:0/2736495161 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6780101c00 0x7f6780102000 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:34:04.618 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:34:04.589+0000 7f6785ae2640 1 -- 172.21.15.159:0/2736495161 >> 172.21.15.159:0/2736495161 conn(0x7f67800fd390 msgr2=0x7f67800ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:34:04.618 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:34:04.590+0000 7f6785ae2640 1 -- 172.21.15.159:0/2736495161 shutdown_connections 2024-09-10T09:34:04.619 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:34:04.590+0000 7f6785ae2640 1 -- 172.21.15.159:0/2736495161 wait complete. 2024-09-10T09:34:04.619 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:34:04.590+0000 7f6785ae2640 1 Processor -- start 2024-09-10T09:34:04.619 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:34:04.590+0000 7f6785ae2640 1 -- start start 2024-09-10T09:34:04.619 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:34:04.590+0000 7f6785ae2640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6780101c00 0x7f6780075f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:34:04.619 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:34:04.590+0000 7f6785ae2640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6780102e00 0x7f67800745e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:34:04.619 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:34:04.590+0000 7f6785ae2640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f67801037c0 0x7f6780074b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:34:04.620 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:34:04.590+0000 7f6785ae2640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f678010d890 con 0x7f6780102e00 2024-09-10T09:34:04.620 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:34:04.590+0000 7f6785ae2640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f678010d710 con 0x7f67801037c0 2024-09-10T09:34:04.620 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:34:04.590+0000 7f6785ae2640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f678010da10 con 0x7f6780101c00 2024-09-10T09:34:04.620 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:34:04.590+0000 7f677f7fe640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6780101c00 0x7f6780075f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:34:04.620 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:34:04.590+0000 7f677f7fe640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6780101c00 0x7f6780075f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:37816/0 (socket says 172.21.15.159:37816) 2024-09-10T09:34:04.620 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:34:04.590+0000 7f677f7fe640 1 -- 172.21.15.159:0/3174999807 learned_addr learned my addr 172.21.15.159:0/3174999807 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-09-10T09:34:04.621 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:34:04.591+0000 7f677ffff640 1 --2- 172.21.15.159:0/3174999807 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f67801037c0 0x7f6780074b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:34:04.621 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:34:04.591+0000 7f677f7fe640 1 -- 172.21.15.159:0/3174999807 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f67801037c0 msgr2=0x7f6780074b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:34:04.621 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:34:04.591+0000 7f677f7fe640 1 --2- 172.21.15.159:0/3174999807 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f67801037c0 0x7f6780074b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:34:04.621 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:34:04.591+0000 7f677f7fe640 1 -- 172.21.15.159:0/3174999807 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6780102e00 msgr2=0x7f67800745e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:34:04.621 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:34:04.591+0000 7f677f7fe640 1 --2- 172.21.15.159:0/3174999807 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6780102e00 0x7f67800745e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:34:04.622 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:34:04.591+0000 7f677f7fe640 1 -- 172.21.15.159:0/3174999807 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67800753e0 con 0x7f6780101c00 2024-09-10T09:34:04.622 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:34:04.591+0000 7f677ffff640 1 --2- 172.21.15.159:0/3174999807 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f67801037c0 0x7f6780074b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:34:04.622 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:34:04.591+0000 7f677f7fe640 1 --2- 172.21.15.159:0/3174999807 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6780101c00 0x7f6780075f90 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f6770031610 tx=0x7f67700043b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:34:04.622 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:34:04.591+0000 7f677d7fa640 1 -- 172.21.15.159:0/3174999807 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f677003f030 con 0x7f6780101c00 2024-09-10T09:34:04.622 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:34:04.591+0000 7f6785ae2640 1 -- 172.21.15.159:0/3174999807 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67801a2010 con 0x7f6780101c00 2024-09-10T09:34:04.622 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:34:04.591+0000 7f677d7fa640 1 -- 172.21.15.159:0/3174999807 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6770003ce0 con 0x7f6780101c00 2024-09-10T09:34:04.623 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:34:04.591+0000 7f677d7fa640 1 -- 172.21.15.159:0/3174999807 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6770035070 con 0x7f6780101c00 2024-09-10T09:34:04.623 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:34:04.591+0000 7f6785ae2640 1 -- 172.21.15.159:0/3174999807 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67801a2510 con 0x7f6780101c00 2024-09-10T09:34:04.623 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:34:04.592+0000 7f6785ae2640 1 -- 172.21.15.159:0/3174999807 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6780108ff0 con 0x7f6780101c00 2024-09-10T09:34:04.623 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:34:04.593+0000 7f677d7fa640 1 -- 172.21.15.159:0/3174999807 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7f6770042030 con 0x7f6780101c00 2024-09-10T09:34:04.623 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:34:04.594+0000 7f677d7fa640 1 --2- 172.21.15.159:0/3174999807 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f676c0a2b80 0x7f676c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:34:04.623 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:34:04.594+0000 7f677d7fa640 1 -- 172.21.15.159:0/3174999807 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 6727+0+0 (secure 0 0 0) 0x7f67700ef2f0 con 0x7f6780101c00 2024-09-10T09:34:04.624 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:34:04.594+0000 7f6777fff640 1 --2- 172.21.15.159:0/3174999807 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f676c0a2b80 0x7f676c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:34:04.624 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:34:04.595+0000 7f6777fff640 1 --2- 172.21.15.159:0/3174999807 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f676c0a2b80 0x7f676c0a5040 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f6780075d10 tx=0x7f6768005bf0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:34:04.624 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:34:04.597+0000 7f677d7fa640 1 -- 172.21.15.159:0/3174999807 <== mon.1 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f67700bc860 con 0x7f6780101c00 2024-09-10T09:34:04.773 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:34:04.772+0000 7f6785ae2640 1 -- 172.21.15.159:0/3174999807 --> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi159:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f6780062630 con 0x7f676c0a2b80 2024-09-10T09:34:04.774 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:34:04 smithi159 ceph-mon[35513]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]': finished 2024-09-10T09:34:04.774 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:34:04 smithi159 ceph-mon[35513]: osdmap e69: 11 total, 10 up, 11 in 2024-09-10T09:34:04.774 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:34:04 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-10T09:34:04.774 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:34:04 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-10T09:34:04.774 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:34:04 smithi159 ceph-mon[35513]: Detected new or changed devices on smithi102 2024-09-10T09:34:04.774 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:34:04 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:34:04.774 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:34:04 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:34:04.774 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:34:04 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:34:04.775 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:34:04 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:34:04.775 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:34:04 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:34:04.775 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:34:04 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:34:04.775 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:34:04 smithi159 ceph-mon[35513]: Adjusting osd_memory_target on smithi102 to 4272M 2024-09-10T09:34:04.775 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:34:04 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:34:04.775 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:34:04 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:34:04.775 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:34:04 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:34:04.775 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:34:04 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:34:04.775 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:34:04 smithi159 ceph-mon[35513]: osd.10 [v2:172.21.15.102:6824/2853806211,v1:172.21.15.102:6825/2853806211] boot 2024-09-10T09:34:04.775 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:34:04 smithi159 ceph-mon[35513]: osdmap e70: 11 total, 11 up, 11 in 2024-09-10T09:34:04.775 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:34:04 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-10T09:34:04.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:34:04 smithi102 ceph-mon[35442]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]': finished 2024-09-10T09:34:04.885 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:34:04 smithi102 ceph-mon[35442]: osdmap e69: 11 total, 10 up, 11 in 2024-09-10T09:34:04.885 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:34:04 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-10T09:34:04.885 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:34:04 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-10T09:34:04.885 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:34:04 smithi102 ceph-mon[35442]: Detected new or changed devices on smithi102 2024-09-10T09:34:04.885 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:34:04 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:34:04.885 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:34:04 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:34:04.885 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:34:04 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:34:04.885 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:34:04 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:34:04.885 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:34:04 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:34:04.885 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:34:04 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:34:04.885 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:34:04 smithi102 ceph-mon[35442]: Adjusting osd_memory_target on smithi102 to 4272M 2024-09-10T09:34:04.885 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:34:04 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:34:04.885 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:34:04 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:34:04.885 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:34:04 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:34:04.886 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:34:04 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:34:04.886 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:34:04 smithi102 ceph-mon[35442]: osd.10 [v2:172.21.15.102:6824/2853806211,v1:172.21.15.102:6825/2853806211] boot 2024-09-10T09:34:04.886 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:34:04 smithi102 ceph-mon[35442]: osdmap e70: 11 total, 11 up, 11 in 2024-09-10T09:34:04.886 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:34:04 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-10T09:34:05.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:34:04 smithi005 ceph-mon[30350]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi102", "root=default"]}]': finished 2024-09-10T09:34:05.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:34:04 smithi005 ceph-mon[30350]: osdmap e69: 11 total, 10 up, 11 in 2024-09-10T09:34:05.057 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:34:04 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-10T09:34:05.057 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:34:04 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-10T09:34:05.057 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:34:04 smithi005 ceph-mon[30350]: Detected new or changed devices on smithi102 2024-09-10T09:34:05.057 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:34:04 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:34:05.057 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:34:04 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:34:05.057 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:34:04 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:34:05.057 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:34:04 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:34:05.057 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:34:04 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:34:05.057 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:34:04 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:34:05.057 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:34:04 smithi005 ceph-mon[30350]: Adjusting osd_memory_target on smithi102 to 4272M 2024-09-10T09:34:05.057 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:34:04 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:34:05.057 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:34:04 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:34:05.058 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:34:04 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:34:05.058 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:34:04 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:34:05.058 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:34:04 smithi005 ceph-mon[30350]: osd.10 [v2:172.21.15.102:6824/2853806211,v1:172.21.15.102:6825/2853806211] boot 2024-09-10T09:34:05.058 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:34:04 smithi005 ceph-mon[30350]: osdmap e70: 11 total, 11 up, 11 in 2024-09-10T09:34:05.058 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:34:04 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-10T09:34:05.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:34:05 smithi159 ceph-mon[35513]: purged_snaps scrub starts 2024-09-10T09:34:05.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:34:05 smithi159 ceph-mon[35513]: purged_snaps scrub ok 2024-09-10T09:34:05.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:34:05 smithi159 ceph-mon[35513]: from='client.24391 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi159:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T09:34:05.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:34:05 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T09:34:05.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:34:05 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T09:34:05.851 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:34:05 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:34:05.851 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:34:05 smithi159 ceph-mon[35513]: osdmap e71: 11 total, 11 up, 11 in 2024-09-10T09:34:05.851 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:34:05 smithi159 ceph-mon[35513]: pgmap v241: 1 pgs: 1 remapped+peering; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail 2024-09-10T09:34:05.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:34:05 smithi102 ceph-mon[35442]: purged_snaps scrub starts 2024-09-10T09:34:05.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:34:05 smithi102 ceph-mon[35442]: purged_snaps scrub ok 2024-09-10T09:34:05.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:34:05 smithi102 ceph-mon[35442]: from='client.24391 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi159:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T09:34:05.885 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:34:05 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T09:34:05.885 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:34:05 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T09:34:05.885 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:34:05 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:34:05.885 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:34:05 smithi102 ceph-mon[35442]: osdmap e71: 11 total, 11 up, 11 in 2024-09-10T09:34:05.885 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:34:05 smithi102 ceph-mon[35442]: pgmap v241: 1 pgs: 1 remapped+peering; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail 2024-09-10T09:34:06.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:34:05 smithi005 ceph-mon[30350]: purged_snaps scrub starts 2024-09-10T09:34:06.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:34:05 smithi005 ceph-mon[30350]: purged_snaps scrub ok 2024-09-10T09:34:06.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:34:05 smithi005 ceph-mon[30350]: from='client.24391 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi159:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T09:34:06.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:34:05 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T09:34:06.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:34:05 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T09:34:06.057 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:34:05 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:34:06.057 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:34:05 smithi005 ceph-mon[30350]: osdmap e71: 11 total, 11 up, 11 in 2024-09-10T09:34:06.057 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:34:05 smithi005 ceph-mon[30350]: pgmap v241: 1 pgs: 1 remapped+peering; 577 KiB data, 322 MiB used, 983 GiB / 983 GiB avail 2024-09-10T09:34:07.306 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:34:07 smithi005 ceph-mon[30350]: osdmap e72: 11 total, 11 up, 11 in 2024-09-10T09:34:07.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:34:07 smithi159 ceph-mon[35513]: osdmap e72: 11 total, 11 up, 11 in 2024-09-10T09:34:07.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:34:07 smithi102 ceph-mon[35442]: osdmap e72: 11 total, 11 up, 11 in 2024-09-10T09:34:08.306 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:34:08 smithi005 ceph-mon[30350]: osdmap e73: 11 total, 11 up, 11 in 2024-09-10T09:34:08.306 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:34:08 smithi005 ceph-mon[30350]: from='client.? 172.21.15.159:0/2422474246' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4e9699ec-8ff2-4005-9d84-af5600b3a2ee"}]: dispatch 2024-09-10T09:34:08.306 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:34:08 smithi005 ceph-mon[30350]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4e9699ec-8ff2-4005-9d84-af5600b3a2ee"}]: dispatch 2024-09-10T09:34:08.306 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:34:08 smithi005 ceph-mon[30350]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4e9699ec-8ff2-4005-9d84-af5600b3a2ee"}]': finished 2024-09-10T09:34:08.306 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:34:08 smithi005 ceph-mon[30350]: osdmap e74: 12 total, 11 up, 12 in 2024-09-10T09:34:08.306 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:34:08 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-10T09:34:08.307 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:34:08 smithi005 ceph-mon[30350]: pgmap v245: 1 pgs: 1 remapped+peering; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-09-10T09:34:08.307 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:34:08 smithi005 ceph-mon[30350]: from='client.? 172.21.15.159:0/2673648720' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T09:34:08.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:34:08 smithi159 ceph-mon[35513]: osdmap e73: 11 total, 11 up, 11 in 2024-09-10T09:34:08.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:34:08 smithi159 ceph-mon[35513]: from='client.? 172.21.15.159:0/2422474246' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4e9699ec-8ff2-4005-9d84-af5600b3a2ee"}]: dispatch 2024-09-10T09:34:08.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:34:08 smithi159 ceph-mon[35513]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4e9699ec-8ff2-4005-9d84-af5600b3a2ee"}]: dispatch 2024-09-10T09:34:08.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:34:08 smithi159 ceph-mon[35513]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4e9699ec-8ff2-4005-9d84-af5600b3a2ee"}]': finished 2024-09-10T09:34:08.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:34:08 smithi159 ceph-mon[35513]: osdmap e74: 12 total, 11 up, 12 in 2024-09-10T09:34:08.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:34:08 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-10T09:34:08.351 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:34:08 smithi159 ceph-mon[35513]: pgmap v245: 1 pgs: 1 remapped+peering; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-09-10T09:34:08.351 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:34:08 smithi159 ceph-mon[35513]: from='client.? 172.21.15.159:0/2673648720' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T09:34:08.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:34:08 smithi102 ceph-mon[35442]: osdmap e73: 11 total, 11 up, 11 in 2024-09-10T09:34:08.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:34:08 smithi102 ceph-mon[35442]: from='client.? 172.21.15.159:0/2422474246' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4e9699ec-8ff2-4005-9d84-af5600b3a2ee"}]: dispatch 2024-09-10T09:34:08.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:34:08 smithi102 ceph-mon[35442]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4e9699ec-8ff2-4005-9d84-af5600b3a2ee"}]: dispatch 2024-09-10T09:34:08.385 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:34:08 smithi102 ceph-mon[35442]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4e9699ec-8ff2-4005-9d84-af5600b3a2ee"}]': finished 2024-09-10T09:34:08.385 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:34:08 smithi102 ceph-mon[35442]: osdmap e74: 12 total, 11 up, 12 in 2024-09-10T09:34:08.385 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:34:08 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-10T09:34:08.385 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:34:08 smithi102 ceph-mon[35442]: pgmap v245: 1 pgs: 1 remapped+peering; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-09-10T09:34:08.385 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:34:08 smithi102 ceph-mon[35442]: from='client.? 172.21.15.159:0/2673648720' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T09:34:09.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:34:09 smithi159 ceph-mon[35513]: pgmap v246: 1 pgs: 1 remapped+peering; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-09-10T09:34:09.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:34:09 smithi102 ceph-mon[35442]: pgmap v246: 1 pgs: 1 remapped+peering; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-09-10T09:34:10.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:34:09 smithi005 ceph-mon[30350]: pgmap v246: 1 pgs: 1 remapped+peering; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-09-10T09:34:11.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:34:11 smithi159 ceph-mon[35513]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-10T09:34:11.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:34:11 smithi102 ceph-mon[35442]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-10T09:34:12.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:34:11 smithi005 ceph-mon[30350]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-10T09:34:13.746 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:34:13 smithi159 ceph-mon[35513]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 77 KiB/s, 0 objects/s recovering 2024-09-10T09:34:13.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:34:13 smithi102 ceph-mon[35442]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 77 KiB/s, 0 objects/s recovering 2024-09-10T09:34:14.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:34:13 smithi005 ceph-mon[30350]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 77 KiB/s, 0 objects/s recovering 2024-09-10T09:34:15.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:34:15 smithi159 ceph-mon[35513]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 68 KiB/s, 0 objects/s recovering 2024-09-10T09:34:15.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:34:15 smithi102 ceph-mon[35442]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 68 KiB/s, 0 objects/s recovering 2024-09-10T09:34:16.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:34:15 smithi005 ceph-mon[30350]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 68 KiB/s, 0 objects/s recovering 2024-09-10T09:34:16.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:34:16 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-09-10T09:34:16.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:34:16 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:34:16.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:34:16 smithi159 ceph-mon[35513]: Deploying daemon osd.11 on smithi159 2024-09-10T09:34:16.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:34:16 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-09-10T09:34:16.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:34:16 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:34:16.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:34:16 smithi102 ceph-mon[35442]: Deploying daemon osd.11 on smithi159 2024-09-10T09:34:17.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:34:16 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-09-10T09:34:17.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:34:16 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:34:17.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:34:16 smithi005 ceph-mon[30350]: Deploying daemon osd.11 on smithi159 2024-09-10T09:34:17.704 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:34:17 smithi159 ceph-mon[35513]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 57 KiB/s, 0 objects/s recovering 2024-09-10T09:34:17.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:34:17 smithi102 ceph-mon[35442]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 57 KiB/s, 0 objects/s recovering 2024-09-10T09:34:18.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:34:17 smithi005 ceph-mon[30350]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 57 KiB/s, 0 objects/s recovering 2024-09-10T09:34:19.642 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:34:19 smithi159 ceph-mon[35513]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-10T09:34:19.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:34:19 smithi102 ceph-mon[35442]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-10T09:34:20.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:34:19 smithi005 ceph-mon[30350]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-10T09:34:21.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:34:21 smithi159 ceph-mon[35513]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-10T09:34:21.858 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:34:21 smithi005 ceph-mon[30350]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-10T09:34:21.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:34:21 smithi102 ceph-mon[35442]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-10T09:34:22.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:34:22 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:34:22.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:34:22 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:34:22.851 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:34:22 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:34:22.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:34:22 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:34:22.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:34:22 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:34:22.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:34:22 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:34:23.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:34:22 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:34:23.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:34:22 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:34:23.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:34:22 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:34:23.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:34:23 smithi159 ceph-mon[35513]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-09-10T09:34:23.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:34:23 smithi102 ceph-mon[35442]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-09-10T09:34:24.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:34:23 smithi005 ceph-mon[30350]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-09-10T09:34:25.319 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:34:25.317+0000 7f677d7fa640 1 -- 172.21.15.159:0/3174999807 <== mgr.14150 v2:172.21.15.5:6800/2707348081 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f6780062630 con 0x7f676c0a2b80 2024-09-10T09:34:25.320 INFO:teuthology.orchestra.run.smithi159.stdout:Created osd(s) 11 on host 'smithi159' 2024-09-10T09:34:25.321 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:34:25.320+0000 7f6785ae2640 1 -- 172.21.15.159:0/3174999807 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f676c0a2b80 msgr2=0x7f676c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:34:25.321 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:34:25.320+0000 7f6785ae2640 1 --2- 172.21.15.159:0/3174999807 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f676c0a2b80 0x7f676c0a5040 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f6780075d10 tx=0x7f6768005bf0 comp rx=0 tx=0).stop 2024-09-10T09:34:25.321 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:34:25.320+0000 7f6785ae2640 1 -- 172.21.15.159:0/3174999807 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6780101c00 msgr2=0x7f6780075f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:34:25.321 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:34:25.320+0000 7f6785ae2640 1 --2- 172.21.15.159:0/3174999807 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6780101c00 0x7f6780075f90 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f6770031610 tx=0x7f67700043b0 comp rx=0 tx=0).stop 2024-09-10T09:34:25.321 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:34:25.320+0000 7f6785ae2640 1 -- 172.21.15.159:0/3174999807 shutdown_connections 2024-09-10T09:34:25.321 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:34:25.320+0000 7f6785ae2640 1 --2- 172.21.15.159:0/3174999807 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f676c0a2b80 0x7f676c0a5040 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:34:25.321 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:34:25.320+0000 7f6785ae2640 1 --2- 172.21.15.159:0/3174999807 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f67801037c0 0x7f6780074b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:34:25.321 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:34:25.320+0000 7f6785ae2640 1 --2- 172.21.15.159:0/3174999807 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6780102e00 0x7f67800745e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:34:25.322 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:34:25.320+0000 7f6785ae2640 1 --2- 172.21.15.159:0/3174999807 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6780101c00 0x7f6780075f90 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:34:25.322 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:34:25.320+0000 7f6785ae2640 1 -- 172.21.15.159:0/3174999807 >> 172.21.15.159:0/3174999807 conn(0x7f67800fd390 msgr2=0x7f67801061c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:34:25.322 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:34:25.320+0000 7f6785ae2640 1 -- 172.21.15.159:0/3174999807 shutdown_connections 2024-09-10T09:34:25.322 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:34:25.320+0000 7f6785ae2640 1 -- 172.21.15.159:0/3174999807 wait complete. 2024-09-10T09:34:25.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:34:25 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:34:25.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:34:25 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:34:25.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:34:25 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:34:25.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:34:25 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:34:25.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:34:25 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:34:25.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:34:25 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:34:25.807 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:34:25 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:34:25.807 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:34:25 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:34:25.807 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:34:25 smithi005 ceph-mon[30350]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-09-10T09:34:25.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:34:25 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:34:25.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:34:25 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:34:25.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:34:25 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:34:25.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:34:25 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:34:25.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:34:25 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:34:25.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:34:25 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:34:25.851 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:34:25 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:34:25.851 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:34:25 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:34:25.851 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:34:25 smithi159 ceph-mon[35513]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-09-10T09:34:25.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:34:25 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:34:25.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:34:25 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:34:25.885 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:34:25 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:34:25.885 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:34:25 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:34:25.885 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:34:25 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:34:25.885 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:34:25 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:34:25.885 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:34:25 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:34:25.885 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:34:25 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:34:25.885 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:34:25 smithi102 ceph-mon[35442]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-09-10T09:34:26.025 DEBUG:teuthology.orchestra.run.smithi159:osd.11> sudo journalctl -f -n 0 -u ceph-bade4b50-6f56-11ef-bcea-c7b262605968@osd.11.service 2024-09-10T09:34:26.027 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-09-10T09:34:26.027 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph osd stat -f json 2024-09-10T09:34:26.412 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:34:26.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:26.964+0000 7f6099c00640 1 -- 172.21.15.5:0/3650240881 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6094101c00 msgr2=0x7f6094102000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:34:26.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:26.964+0000 7f6099c00640 1 --2- 172.21.15.5:0/3650240881 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6094101c00 0x7f6094102000 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f6084009900 tx=0x7f608402a410 comp rx=0 tx=0).stop 2024-09-10T09:34:26.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:26.965+0000 7f6099c00640 1 -- 172.21.15.5:0/3650240881 shutdown_connections 2024-09-10T09:34:26.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:26.965+0000 7f6099c00640 1 --2- 172.21.15.5:0/3650240881 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f60941037c0 0x7f609410a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:34:26.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:26.965+0000 7f6099c00640 1 --2- 172.21.15.5:0/3650240881 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6094102e00 0x7f6094103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:34:26.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:26.965+0000 7f6099c00640 1 --2- 172.21.15.5:0/3650240881 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6094101c00 0x7f6094102000 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:34:26.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:26.965+0000 7f6099c00640 1 -- 172.21.15.5:0/3650240881 >> 172.21.15.5:0/3650240881 conn(0x7f60940fd390 msgr2=0x7f60940ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:34:26.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:26.965+0000 7f6099c00640 1 -- 172.21.15.5:0/3650240881 shutdown_connections 2024-09-10T09:34:26.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:26.965+0000 7f6099c00640 1 -- 172.21.15.5:0/3650240881 wait complete. 2024-09-10T09:34:26.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:26.966+0000 7f6099c00640 1 Processor -- start 2024-09-10T09:34:26.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:26.966+0000 7f6099c00640 1 -- start start 2024-09-10T09:34:26.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:26.966+0000 7f6099c00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6094101c00 0x7f6094107ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:34:26.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:26.966+0000 7f6099c00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6094102e00 0x7f6094108010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:34:26.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:26.966+0000 7f6099c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f60941037c0 0x7f6094104630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:34:26.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:26.966+0000 7f6099c00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f609410d8e0 con 0x7f6094102e00 2024-09-10T09:34:26.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:26.966+0000 7f6099c00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f609410d760 con 0x7f60941037c0 2024-09-10T09:34:26.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:26.966+0000 7f6099c00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f609410da60 con 0x7f6094101c00 2024-09-10T09:34:26.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:26.966+0000 7f6092a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6094101c00 0x7f6094107ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:34:26.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:26.966+0000 7f6092000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6094102e00 0x7f6094108010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:34:26.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:26.966+0000 7f6092a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6094101c00 0x7f6094107ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:48834/0 (socket says 172.21.15.5:48834) 2024-09-10T09:34:26.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:26.966+0000 7f6092a00640 1 -- 172.21.15.5:0/3673637959 learned_addr learned my addr 172.21.15.5:0/3673637959 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:34:26.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:26.966+0000 7f6093400640 1 --2- 172.21.15.5:0/3673637959 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f60941037c0 0x7f6094104630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:34:26.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:26.967+0000 7f6092a00640 1 -- 172.21.15.5:0/3673637959 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f60941037c0 msgr2=0x7f6094104630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:34:26.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:26.967+0000 7f6092a00640 1 --2- 172.21.15.5:0/3673637959 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f60941037c0 0x7f6094104630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:34:26.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:26.967+0000 7f6092a00640 1 -- 172.21.15.5:0/3673637959 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6094102e00 msgr2=0x7f6094108010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:34:26.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:26.967+0000 7f6092a00640 1 --2- 172.21.15.5:0/3673637959 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6094102e00 0x7f6094108010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:34:26.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:26.967+0000 7f6092a00640 1 -- 172.21.15.5:0/3673637959 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6094104ef0 con 0x7f6094101c00 2024-09-10T09:34:26.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:26.967+0000 7f6092000640 1 --2- 172.21.15.5:0/3673637959 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6094102e00 0x7f6094108010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:34:26.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:26.967+0000 7f6092a00640 1 --2- 172.21.15.5:0/3673637959 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6094101c00 0x7f6094107ad0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f6084031440 tx=0x7f6084032ce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:34:26.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:26.967+0000 7f608b400640 1 -- 172.21.15.5:0/3673637959 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f608403f030 con 0x7f6094101c00 2024-09-10T09:34:26.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:26.967+0000 7f608b400640 1 -- 172.21.15.5:0/3673637959 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6084003ce0 con 0x7f6094101c00 2024-09-10T09:34:26.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:26.968+0000 7f6099c00640 1 -- 172.21.15.5:0/3673637959 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60941a6590 con 0x7f6094101c00 2024-09-10T09:34:26.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:26.968+0000 7f608b400640 1 -- 172.21.15.5:0/3673637959 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6084035070 con 0x7f6094101c00 2024-09-10T09:34:26.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:26.968+0000 7f6099c00640 1 -- 172.21.15.5:0/3673637959 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60941a6930 con 0x7f6094101c00 2024-09-10T09:34:26.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:26.969+0000 7f6099c00640 1 -- 172.21.15.5:0/3673637959 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f60941a6b90 con 0x7f6094101c00 2024-09-10T09:34:26.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:26.971+0000 7f608b400640 1 -- 172.21.15.5:0/3673637959 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7f6084042030 con 0x7f6094101c00 2024-09-10T09:34:26.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:26.972+0000 7f608b400640 1 --2- 172.21.15.5:0/3673637959 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f60780a2c00 0x7f60780a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:34:26.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:26.972+0000 7f608b400640 1 -- 172.21.15.5:0/3673637959 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 6809+0+0 (secure 0 0 0) 0x7f60840eefe0 con 0x7f6094101c00 2024-09-10T09:34:26.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:26.972+0000 7f6092000640 1 --2- 172.21.15.5:0/3673637959 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f60780a2c00 0x7f60780a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:34:26.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:26.973+0000 7f6092000640 1 --2- 172.21.15.5:0/3673637959 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f60780a2c00 0x7f60780a50c0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f6094105a50 tx=0x7f6080005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:34:26.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:26.976+0000 7f608b400640 1 -- 172.21.15.5:0/3673637959 <== mon.1 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f60840bbe30 con 0x7f6094101c00 2024-09-10T09:34:27.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:27.161+0000 7f6099c00640 1 -- 172.21.15.5:0/3673637959 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f60941a7220 con 0x7f6094101c00 2024-09-10T09:34:27.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:27.161+0000 7f608b400640 1 -- 172.21.15.5:0/3673637959 <== mon.1 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v74) ==== 74+0+133 (secure 0 0 0) 0x7f60840c0130 con 0x7f6094101c00 2024-09-10T09:34:27.162 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-10T09:34:27.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:27.163+0000 7f6099c00640 1 -- 172.21.15.5:0/3673637959 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f60780a2c00 msgr2=0x7f60780a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:34:27.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:27.163+0000 7f6099c00640 1 --2- 172.21.15.5:0/3673637959 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f60780a2c00 0x7f60780a50c0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f6094105a50 tx=0x7f6080005f90 comp rx=0 tx=0).stop 2024-09-10T09:34:27.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:27.163+0000 7f6099c00640 1 -- 172.21.15.5:0/3673637959 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6094101c00 msgr2=0x7f6094107ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:34:27.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:27.163+0000 7f6099c00640 1 --2- 172.21.15.5:0/3673637959 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6094101c00 0x7f6094107ad0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f6084031440 tx=0x7f6084032ce0 comp rx=0 tx=0).stop 2024-09-10T09:34:27.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:27.164+0000 7f6099c00640 1 -- 172.21.15.5:0/3673637959 shutdown_connections 2024-09-10T09:34:27.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:27.164+0000 7f6099c00640 1 --2- 172.21.15.5:0/3673637959 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f60780a2c00 0x7f60780a50c0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:34:27.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:27.164+0000 7f6099c00640 1 --2- 172.21.15.5:0/3673637959 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f60941037c0 0x7f6094104630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:34:27.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:27.164+0000 7f6099c00640 1 --2- 172.21.15.5:0/3673637959 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6094102e00 0x7f6094108010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:34:27.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:27.164+0000 7f6099c00640 1 --2- 172.21.15.5:0/3673637959 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6094101c00 0x7f6094107ad0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:34:27.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:27.164+0000 7f6099c00640 1 -- 172.21.15.5:0/3673637959 >> 172.21.15.5:0/3673637959 conn(0x7f60940fd390 msgr2=0x7f6094074420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:34:27.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:27.164+0000 7f6099c00640 1 -- 172.21.15.5:0/3673637959 shutdown_connections 2024-09-10T09:34:27.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:27.164+0000 7f6099c00640 1 -- 172.21.15.5:0/3673637959 wait complete. 2024-09-10T09:34:27.476 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:34:27 smithi159 ceph-mon[35513]: from='client.? 172.21.15.5:0/3673637959' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-10T09:34:27.528 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:34:27 smithi005 ceph-mon[30350]: from='client.? 172.21.15.5:0/3673637959' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-10T09:34:27.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:34:27 smithi102 ceph-mon[35442]: from='client.? 172.21.15.5:0/3673637959' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-10T09:34:28.100 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 10 09:34:27 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-11[60437]: 2024-09-10T09:34:27.850+0000 7fa798f2d640 -1 osd.11 0 log_to_monitors true 2024-09-10T09:34:28.121 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":74,"num_osds":12,"num_up_osds":11,"osd_up_since":1725960843,"num_in_osds":12,"osd_in_since":1725960847,"num_remapped_pgs":0} 2024-09-10T09:34:28.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:34:28 smithi005 ceph-mon[30350]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-09-10T09:34:28.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:34:28 smithi005 ceph-mon[30350]: from='osd.11 [v2:172.21.15.159:6824/969941218,v1:172.21.15.159:6825/969941218]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-10T09:34:28.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:34:28 smithi005 ceph-mon[30350]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-10T09:34:28.564 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:34:28 smithi159 ceph-mon[35513]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-09-10T09:34:28.564 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:34:28 smithi159 ceph-mon[35513]: from='osd.11 [v2:172.21.15.159:6824/969941218,v1:172.21.15.159:6825/969941218]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-10T09:34:28.564 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:34:28 smithi159 ceph-mon[35513]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-10T09:34:28.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:34:28 smithi102 ceph-mon[35442]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-09-10T09:34:28.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:34:28 smithi102 ceph-mon[35442]: from='osd.11 [v2:172.21.15.159:6824/969941218,v1:172.21.15.159:6825/969941218]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-10T09:34:28.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:34:28 smithi102 ceph-mon[35442]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-10T09:34:29.122 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph osd stat -f json 2024-09-10T09:34:29.498 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:34:29.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:34:29 smithi005 ceph-mon[30350]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-09-10T09:34:29.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:34:29 smithi005 ceph-mon[30350]: from='osd.11 [v2:172.21.15.159:6824/969941218,v1:172.21.15.159:6825/969941218]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]: dispatch 2024-09-10T09:34:29.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:34:29 smithi005 ceph-mon[30350]: osdmap e75: 12 total, 11 up, 12 in 2024-09-10T09:34:29.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:34:29 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-10T09:34:29.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:34:29 smithi005 ceph-mon[30350]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]: dispatch 2024-09-10T09:34:29.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:34:29 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:34:29.557 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:34:29 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:34:29.557 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:34:29 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:34:29.557 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:34:29 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:34:29.557 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:34:29 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:34:29.557 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:34:29 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:34:29.557 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:34:29 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:34:29.557 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:34:29 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:34:29.557 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:34:29 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:34:29.557 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:34:29 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:34:29.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:34:29 smithi159 ceph-mon[35513]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-09-10T09:34:29.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:34:29 smithi159 ceph-mon[35513]: from='osd.11 [v2:172.21.15.159:6824/969941218,v1:172.21.15.159:6825/969941218]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]: dispatch 2024-09-10T09:34:29.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:34:29 smithi159 ceph-mon[35513]: osdmap e75: 12 total, 11 up, 12 in 2024-09-10T09:34:29.601 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:34:29 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-10T09:34:29.601 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:34:29 smithi159 ceph-mon[35513]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]: dispatch 2024-09-10T09:34:29.601 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:34:29 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:34:29.601 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:34:29 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:34:29.601 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:34:29 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:34:29.601 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:34:29 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:34:29.601 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:34:29 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:34:29.601 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:34:29 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:34:29.601 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:34:29 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:34:29.601 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:34:29 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:34:29.601 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:34:29 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:34:29.601 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:34:29 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:34:29.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:34:29 smithi102 ceph-mon[35442]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-09-10T09:34:29.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:34:29 smithi102 ceph-mon[35442]: from='osd.11 [v2:172.21.15.159:6824/969941218,v1:172.21.15.159:6825/969941218]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]: dispatch 2024-09-10T09:34:29.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:34:29 smithi102 ceph-mon[35442]: osdmap e75: 12 total, 11 up, 12 in 2024-09-10T09:34:29.635 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:34:29 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-10T09:34:29.635 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:34:29 smithi102 ceph-mon[35442]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]: dispatch 2024-09-10T09:34:29.635 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:34:29 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:34:29.635 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:34:29 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:34:29.635 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:34:29 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:34:29.635 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:34:29 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:34:29.635 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:34:29 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:34:29.635 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:34:29 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:34:29.635 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:34:29 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:34:29.635 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:34:29 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:34:29.635 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:34:29 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:34:29.635 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:34:29 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:34:30.070 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 10 09:34:29 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-11[60437]: 2024-09-10T09:34:29.726+0000 7fa794eb0640 -1 osd.11 0 waiting for initial osdmap 2024-09-10T09:34:30.070 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 10 09:34:29 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-11[60437]: 2024-09-10T09:34:29.742+0000 7fa78ccd0640 -1 osd.11 76 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T09:34:30.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:30.081+0000 7f77dde00640 1 -- 172.21.15.5:0/1944100106 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f77d8104f60 msgr2=0x7f77d81053e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:34:30.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:30.081+0000 7f77dde00640 1 --2- 172.21.15.5:0/1944100106 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f77d8104f60 0x7f77d81053e0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f77c0009900 tx=0x7f77c002a3f0 comp rx=0 tx=0).stop 2024-09-10T09:34:30.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:30.081+0000 7f77dde00640 1 -- 172.21.15.5:0/1944100106 shutdown_connections 2024-09-10T09:34:30.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:30.081+0000 7f77dde00640 1 --2- 172.21.15.5:0/1944100106 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f77d8105920 0x7f77d810c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:34:30.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:30.081+0000 7f77dde00640 1 --2- 172.21.15.5:0/1944100106 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f77d8104f60 0x7f77d81053e0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:34:30.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:30.081+0000 7f77dde00640 1 --2- 172.21.15.5:0/1944100106 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f77d8103e00 0x7f77d8104200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:34:30.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:30.081+0000 7f77dde00640 1 -- 172.21.15.5:0/1944100106 >> 172.21.15.5:0/1944100106 conn(0x7f77d80ff5b0 msgr2=0x7f77d81019d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:34:30.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:30.081+0000 7f77dde00640 1 -- 172.21.15.5:0/1944100106 shutdown_connections 2024-09-10T09:34:30.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:30.081+0000 7f77dde00640 1 -- 172.21.15.5:0/1944100106 wait complete. 2024-09-10T09:34:30.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:30.082+0000 7f77dde00640 1 Processor -- start 2024-09-10T09:34:30.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:30.082+0000 7f77dde00640 1 -- start start 2024-09-10T09:34:30.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:30.082+0000 7f77dde00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f77d8103e00 0x7f77d819a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:34:30.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:30.082+0000 7f77dde00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f77d8104f60 0x7f77d819a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:34:30.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:30.082+0000 7f77dde00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f77d8105920 0x7f77d81a16a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:34:30.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:30.082+0000 7f77dde00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f77d810fae0 con 0x7f77d8103e00 2024-09-10T09:34:30.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:30.082+0000 7f77dde00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f77d810f960 con 0x7f77d8104f60 2024-09-10T09:34:30.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:30.082+0000 7f77dde00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f77d810fc60 con 0x7f77d8105920 2024-09-10T09:34:30.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:30.082+0000 7f77d6a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f77d8103e00 0x7f77d819a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:34:30.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:30.082+0000 7f77d6a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f77d8103e00 0x7f77d819a0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37650/0 (socket says 172.21.15.5:37650) 2024-09-10T09:34:30.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:30.082+0000 7f77d6a00640 1 -- 172.21.15.5:0/944670160 learned_addr learned my addr 172.21.15.5:0/944670160 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:34:30.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:30.083+0000 7f77d7400640 1 --2- 172.21.15.5:0/944670160 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f77d8105920 0x7f77d81a16a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:34:30.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:30.083+0000 7f77ce000640 1 --2- 172.21.15.5:0/944670160 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f77d8104f60 0x7f77d819a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:34:30.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:30.083+0000 7f77d6a00640 1 -- 172.21.15.5:0/944670160 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f77d8105920 msgr2=0x7f77d81a16a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:34:30.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:30.083+0000 7f77d6a00640 1 --2- 172.21.15.5:0/944670160 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f77d8105920 0x7f77d81a16a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:34:30.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:30.083+0000 7f77d6a00640 1 -- 172.21.15.5:0/944670160 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f77d8104f60 msgr2=0x7f77d819a620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:34:30.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:30.083+0000 7f77d6a00640 1 --2- 172.21.15.5:0/944670160 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f77d8104f60 0x7f77d819a620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:34:30.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:30.083+0000 7f77d6a00640 1 -- 172.21.15.5:0/944670160 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77d81a1d10 con 0x7f77d8103e00 2024-09-10T09:34:30.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:30.083+0000 7f77d6a00640 1 --2- 172.21.15.5:0/944670160 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f77d8103e00 0x7f77d819a0e0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f77c800e9f0 tx=0x7f77c800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:34:30.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:30.084+0000 7f77dca00640 1 -- 172.21.15.5:0/944670160 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77c800cd80 con 0x7f77d8103e00 2024-09-10T09:34:30.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:30.084+0000 7f77dca00640 1 -- 172.21.15.5:0/944670160 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f77c8010910 con 0x7f77d8103e00 2024-09-10T09:34:30.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:30.084+0000 7f77dca00640 1 -- 172.21.15.5:0/944670160 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77c8018990 con 0x7f77d8103e00 2024-09-10T09:34:30.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:30.084+0000 7f77dde00640 1 -- 172.21.15.5:0/944670160 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77d81a1fa0 con 0x7f77d8103e00 2024-09-10T09:34:30.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:30.084+0000 7f77dde00640 1 -- 172.21.15.5:0/944670160 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77d81a2400 con 0x7f77d8103e00 2024-09-10T09:34:30.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:30.085+0000 7f77dde00640 1 -- 172.21.15.5:0/944670160 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f77d810b1a0 con 0x7f77d8103e00 2024-09-10T09:34:30.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:30.087+0000 7f77dca00640 1 -- 172.21.15.5:0/944670160 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7f77c8010ac0 con 0x7f77d8103e00 2024-09-10T09:34:30.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:30.088+0000 7f77dca00640 1 --2- 172.21.15.5:0/944670160 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f77bc0a2c30 0x7f77bc0a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:34:30.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:30.088+0000 7f77dca00640 1 -- 172.21.15.5:0/944670160 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 6847+0+0 (secure 0 0 0) 0x7f77c80cf130 con 0x7f77d8103e00 2024-09-10T09:34:30.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:30.088+0000 7f77ce000640 1 --2- 172.21.15.5:0/944670160 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f77bc0a2c30 0x7f77bc0a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:34:30.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:30.089+0000 7f77ce000640 1 --2- 172.21.15.5:0/944670160 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f77bc0a2c30 0x7f77bc0a50f0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f77c002a9e0 tx=0x7f77c00315b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:34:30.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:30.091+0000 7f77dca00640 1 -- 172.21.15.5:0/944670160 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f77c809bf60 con 0x7f77d8103e00 2024-09-10T09:34:30.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:30.276+0000 7f77dde00640 1 -- 172.21.15.5:0/944670160 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f77d802da70 con 0x7f77d8103e00 2024-09-10T09:34:30.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:30.276+0000 7f77dca00640 1 -- 172.21.15.5:0/944670160 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v76) ==== 74+0+133 (secure 0 0 0) 0x7f77c80a0260 con 0x7f77d8103e00 2024-09-10T09:34:30.277 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-10T09:34:30.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:30.278+0000 7f77dde00640 1 -- 172.21.15.5:0/944670160 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f77bc0a2c30 msgr2=0x7f77bc0a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:34:30.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:30.278+0000 7f77dde00640 1 --2- 172.21.15.5:0/944670160 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f77bc0a2c30 0x7f77bc0a50f0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f77c002a9e0 tx=0x7f77c00315b0 comp rx=0 tx=0).stop 2024-09-10T09:34:30.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:30.278+0000 7f77dde00640 1 -- 172.21.15.5:0/944670160 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f77d8103e00 msgr2=0x7f77d819a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:34:30.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:30.278+0000 7f77dde00640 1 --2- 172.21.15.5:0/944670160 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f77d8103e00 0x7f77d819a0e0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f77c800e9f0 tx=0x7f77c800eeb0 comp rx=0 tx=0).stop 2024-09-10T09:34:30.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:30.278+0000 7f77dde00640 1 -- 172.21.15.5:0/944670160 shutdown_connections 2024-09-10T09:34:30.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:30.278+0000 7f77dde00640 1 --2- 172.21.15.5:0/944670160 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f77bc0a2c30 0x7f77bc0a50f0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:34:30.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:30.278+0000 7f77dde00640 1 --2- 172.21.15.5:0/944670160 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f77d8105920 0x7f77d81a16a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:34:30.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:30.278+0000 7f77dde00640 1 --2- 172.21.15.5:0/944670160 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f77d8104f60 0x7f77d819a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:34:30.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:30.278+0000 7f77dde00640 1 --2- 172.21.15.5:0/944670160 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f77d8103e00 0x7f77d819a0e0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:34:30.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:30.278+0000 7f77dde00640 1 -- 172.21.15.5:0/944670160 >> 172.21.15.5:0/944670160 conn(0x7f77d80ff5b0 msgr2=0x7f77d8108370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:34:30.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:30.279+0000 7f77dde00640 1 -- 172.21.15.5:0/944670160 shutdown_connections 2024-09-10T09:34:30.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:30.279+0000 7f77dde00640 1 -- 172.21.15.5:0/944670160 wait complete. 2024-09-10T09:34:30.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:34:30 smithi005 ceph-mon[30350]: Detected new or changed devices on smithi159 2024-09-10T09:34:30.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:34:30 smithi005 ceph-mon[30350]: Adjusting osd_memory_target on smithi159 to 4272M 2024-09-10T09:34:30.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:34:30 smithi005 ceph-mon[30350]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]': finished 2024-09-10T09:34:30.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:34:30 smithi005 ceph-mon[30350]: osdmap e76: 12 total, 11 up, 12 in 2024-09-10T09:34:30.557 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:34:30 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-10T09:34:30.557 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:34:30 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-10T09:34:30.557 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:34:30 smithi005 ceph-mon[30350]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-09-10T09:34:30.557 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:34:30 smithi005 ceph-mon[30350]: from='osd.11 ' entity='osd.11' 2024-09-10T09:34:30.557 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:34:30 smithi005 ceph-mon[30350]: from='client.? 172.21.15.5:0/944670160' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-10T09:34:30.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:34:30 smithi159 ceph-mon[35513]: Detected new or changed devices on smithi159 2024-09-10T09:34:30.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:34:30 smithi159 ceph-mon[35513]: Adjusting osd_memory_target on smithi159 to 4272M 2024-09-10T09:34:30.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:34:30 smithi159 ceph-mon[35513]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]': finished 2024-09-10T09:34:30.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:34:30 smithi159 ceph-mon[35513]: osdmap e76: 12 total, 11 up, 12 in 2024-09-10T09:34:30.601 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:34:30 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-10T09:34:30.601 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:34:30 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-10T09:34:30.601 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:34:30 smithi159 ceph-mon[35513]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-09-10T09:34:30.601 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:34:30 smithi159 ceph-mon[35513]: from='osd.11 ' entity='osd.11' 2024-09-10T09:34:30.601 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:34:30 smithi159 ceph-mon[35513]: from='client.? 172.21.15.5:0/944670160' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-10T09:34:30.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:34:30 smithi102 ceph-mon[35442]: Detected new or changed devices on smithi159 2024-09-10T09:34:30.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:34:30 smithi102 ceph-mon[35442]: Adjusting osd_memory_target on smithi159 to 4272M 2024-09-10T09:34:30.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:34:30 smithi102 ceph-mon[35442]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]': finished 2024-09-10T09:34:30.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:34:30 smithi102 ceph-mon[35442]: osdmap e76: 12 total, 11 up, 12 in 2024-09-10T09:34:30.635 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:34:30 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-10T09:34:30.635 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:34:30 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-10T09:34:30.635 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:34:30 smithi102 ceph-mon[35442]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-09-10T09:34:30.635 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:34:30 smithi102 ceph-mon[35442]: from='osd.11 ' entity='osd.11' 2024-09-10T09:34:30.635 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:34:30 smithi102 ceph-mon[35442]: from='client.? 172.21.15.5:0/944670160' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-10T09:34:31.268 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":76,"num_osds":12,"num_up_osds":11,"osd_up_since":1725960843,"num_in_osds":12,"osd_in_since":1725960847,"num_remapped_pgs":0} 2024-09-10T09:34:31.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:34:31 smithi005 ceph-mon[30350]: purged_snaps scrub starts 2024-09-10T09:34:31.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:34:31 smithi005 ceph-mon[30350]: purged_snaps scrub ok 2024-09-10T09:34:31.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:34:31 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-10T09:34:31.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:34:31 smithi005 ceph-mon[30350]: osd.11 [v2:172.21.15.159:6824/969941218,v1:172.21.15.159:6825/969941218] boot 2024-09-10T09:34:31.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:34:31 smithi005 ceph-mon[30350]: osdmap e77: 12 total, 12 up, 12 in 2024-09-10T09:34:31.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:34:31 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-10T09:34:31.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:34:31 smithi159 ceph-mon[35513]: purged_snaps scrub starts 2024-09-10T09:34:31.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:34:31 smithi159 ceph-mon[35513]: purged_snaps scrub ok 2024-09-10T09:34:31.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:34:31 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-10T09:34:31.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:34:31 smithi159 ceph-mon[35513]: osd.11 [v2:172.21.15.159:6824/969941218,v1:172.21.15.159:6825/969941218] boot 2024-09-10T09:34:31.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:34:31 smithi159 ceph-mon[35513]: osdmap e77: 12 total, 12 up, 12 in 2024-09-10T09:34:31.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:34:31 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-10T09:34:31.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:34:31 smithi102 ceph-mon[35442]: purged_snaps scrub starts 2024-09-10T09:34:31.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:34:31 smithi102 ceph-mon[35442]: purged_snaps scrub ok 2024-09-10T09:34:31.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:34:31 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-10T09:34:31.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:34:31 smithi102 ceph-mon[35442]: osd.11 [v2:172.21.15.159:6824/969941218,v1:172.21.15.159:6825/969941218] boot 2024-09-10T09:34:31.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:34:31 smithi102 ceph-mon[35442]: osdmap e77: 12 total, 12 up, 12 in 2024-09-10T09:34:31.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:34:31 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-10T09:34:32.269 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph osd stat -f json 2024-09-10T09:34:32.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:34:32 smithi159 ceph-mon[35513]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T09:34:32.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:34:32 smithi159 ceph-mon[35513]: osdmap e78: 12 total, 12 up, 12 in 2024-09-10T09:34:32.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:34:32 smithi102 ceph-mon[35442]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T09:34:32.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:34:32 smithi102 ceph-mon[35442]: osdmap e78: 12 total, 12 up, 12 in 2024-09-10T09:34:32.651 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:34:32.790 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:34:32 smithi005 ceph-mon[30350]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T09:34:32.790 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:34:32 smithi005 ceph-mon[30350]: osdmap e78: 12 total, 12 up, 12 in 2024-09-10T09:34:33.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:33.208+0000 7f7650e00640 1 -- 172.21.15.5:0/2939165035 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f764c104f60 msgr2=0x7f764c1053e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:34:33.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:33.208+0000 7f7650e00640 1 --2- 172.21.15.5:0/2939165035 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f764c104f60 0x7f764c1053e0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f7634009900 tx=0x7f763402a3f0 comp rx=0 tx=0).stop 2024-09-10T09:34:33.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:33.208+0000 7f7650e00640 1 -- 172.21.15.5:0/2939165035 shutdown_connections 2024-09-10T09:34:33.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:33.208+0000 7f7650e00640 1 --2- 172.21.15.5:0/2939165035 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f764c105920 0x7f764c10c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:34:33.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:33.208+0000 7f7650e00640 1 --2- 172.21.15.5:0/2939165035 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f764c104f60 0x7f764c1053e0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:34:33.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:33.208+0000 7f7650e00640 1 --2- 172.21.15.5:0/2939165035 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f764c103e00 0x7f764c104200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:34:33.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:33.208+0000 7f7650e00640 1 -- 172.21.15.5:0/2939165035 >> 172.21.15.5:0/2939165035 conn(0x7f764c0ff5b0 msgr2=0x7f764c1019d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:34:33.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:33.208+0000 7f7650e00640 1 -- 172.21.15.5:0/2939165035 shutdown_connections 2024-09-10T09:34:33.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:33.208+0000 7f7650e00640 1 -- 172.21.15.5:0/2939165035 wait complete. 2024-09-10T09:34:33.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:33.209+0000 7f7650e00640 1 Processor -- start 2024-09-10T09:34:33.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:33.209+0000 7f7650e00640 1 -- start start 2024-09-10T09:34:33.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:33.209+0000 7f7650e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f764c103e00 0x7f764c19a130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:34:33.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:33.209+0000 7f7650e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f764c104f60 0x7f764c19a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:34:33.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:33.209+0000 7f7650e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f764c105920 0x7f764c1a16f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:34:33.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:33.209+0000 7f7650e00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f764c10fae0 con 0x7f764c105920 2024-09-10T09:34:33.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:33.209+0000 7f7650e00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f764c10f960 con 0x7f764c103e00 2024-09-10T09:34:33.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:33.209+0000 7f7650e00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f764c10fc60 con 0x7f764c104f60 2024-09-10T09:34:33.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:33.210+0000 7f7643e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f764c104f60 0x7f764c19a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:34:33.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:33.210+0000 7f764a800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f764c105920 0x7f764c1a16f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:34:33.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:33.210+0000 7f7649e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f764c103e00 0x7f764c19a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:34:33.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:33.210+0000 7f7643e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f764c104f60 0x7f764c19a670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:58782/0 (socket says 172.21.15.5:58782) 2024-09-10T09:34:33.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:33.210+0000 7f7643e00640 1 -- 172.21.15.5:0/500642492 learned_addr learned my addr 172.21.15.5:0/500642492 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:34:33.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:33.210+0000 7f7649e00640 1 -- 172.21.15.5:0/500642492 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f764c104f60 msgr2=0x7f764c19a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:34:33.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:33.210+0000 7f7649e00640 1 --2- 172.21.15.5:0/500642492 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f764c104f60 0x7f764c19a670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:34:33.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:33.210+0000 7f7649e00640 1 -- 172.21.15.5:0/500642492 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f764c105920 msgr2=0x7f764c1a16f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:34:33.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:33.210+0000 7f7649e00640 1 --2- 172.21.15.5:0/500642492 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f764c105920 0x7f764c1a16f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:34:33.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:33.210+0000 7f7649e00640 1 -- 172.21.15.5:0/500642492 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f764c1a1d60 con 0x7f764c103e00 2024-09-10T09:34:33.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:33.210+0000 7f7643e00640 1 --2- 172.21.15.5:0/500642492 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f764c104f60 0x7f764c19a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:34:33.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:33.210+0000 7f7649e00640 1 --2- 172.21.15.5:0/500642492 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f764c103e00 0x7f764c19a130 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f763c00e9f0 tx=0x7f763c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:34:33.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:33.211+0000 7f7642a00640 1 -- 172.21.15.5:0/500642492 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f763c00ccc0 con 0x7f764c103e00 2024-09-10T09:34:33.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:33.211+0000 7f7650e00640 1 -- 172.21.15.5:0/500642492 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f764c1a1ff0 con 0x7f764c103e00 2024-09-10T09:34:33.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:33.211+0000 7f7642a00640 1 -- 172.21.15.5:0/500642492 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f763c010910 con 0x7f764c103e00 2024-09-10T09:34:33.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:33.211+0000 7f7650e00640 1 -- 172.21.15.5:0/500642492 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f764c1a24f0 con 0x7f764c103e00 2024-09-10T09:34:33.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:33.211+0000 7f7642a00640 1 -- 172.21.15.5:0/500642492 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f763c018960 con 0x7f764c103e00 2024-09-10T09:34:33.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:33.212+0000 7f7650e00640 1 -- 172.21.15.5:0/500642492 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7628005290 con 0x7f764c103e00 2024-09-10T09:34:33.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:33.214+0000 7f7642a00640 1 -- 172.21.15.5:0/500642492 <== mon.2 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7f763c010ac0 con 0x7f764c103e00 2024-09-10T09:34:33.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:33.215+0000 7f7642a00640 1 --2- 172.21.15.5:0/500642492 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f76300a2c30 0x7f76300a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:34:33.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:33.215+0000 7f7642a00640 1 -- 172.21.15.5:0/500642492 <== mon.2 v2:172.21.15.102:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7127+0+0 (secure 0 0 0) 0x7f763c0cf370 con 0x7f764c103e00 2024-09-10T09:34:33.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:33.216+0000 7f7643e00640 1 --2- 172.21.15.5:0/500642492 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f76300a2c30 0x7f76300a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:34:33.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:33.216+0000 7f7643e00640 1 --2- 172.21.15.5:0/500642492 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f76300a2c30 0x7f76300a50f0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f7634000c00 tx=0x7f76340315b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:34:33.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:33.217+0000 7f7642a00640 1 -- 172.21.15.5:0/500642492 <== mon.2 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f763c09c080 con 0x7f764c103e00 2024-09-10T09:34:33.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:33.402+0000 7f7650e00640 1 -- 172.21.15.5:0/500642492 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f7628005850 con 0x7f764c103e00 2024-09-10T09:34:33.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:33.403+0000 7f7642a00640 1 -- 172.21.15.5:0/500642492 <== mon.2 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v78) ==== 74+0+133 (secure 0 0 0) 0x7f763c0a0380 con 0x7f764c103e00 2024-09-10T09:34:33.404 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-10T09:34:33.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:33.406+0000 7f7650e00640 1 -- 172.21.15.5:0/500642492 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f76300a2c30 msgr2=0x7f76300a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:34:33.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:33.406+0000 7f7650e00640 1 --2- 172.21.15.5:0/500642492 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f76300a2c30 0x7f76300a50f0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f7634000c00 tx=0x7f76340315b0 comp rx=0 tx=0).stop 2024-09-10T09:34:33.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:33.406+0000 7f7650e00640 1 -- 172.21.15.5:0/500642492 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f764c103e00 msgr2=0x7f764c19a130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:34:33.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:33.406+0000 7f7650e00640 1 --2- 172.21.15.5:0/500642492 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f764c103e00 0x7f764c19a130 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f763c00e9f0 tx=0x7f763c00eeb0 comp rx=0 tx=0).stop 2024-09-10T09:34:33.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:33.406+0000 7f7650e00640 1 -- 172.21.15.5:0/500642492 shutdown_connections 2024-09-10T09:34:33.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:33.406+0000 7f7650e00640 1 --2- 172.21.15.5:0/500642492 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f76300a2c30 0x7f76300a50f0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:34:33.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:33.406+0000 7f7650e00640 1 --2- 172.21.15.5:0/500642492 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f764c105920 0x7f764c1a16f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:34:33.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:33.406+0000 7f7650e00640 1 --2- 172.21.15.5:0/500642492 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f764c104f60 0x7f764c19a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:34:33.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:33.406+0000 7f7650e00640 1 --2- 172.21.15.5:0/500642492 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f764c103e00 0x7f764c19a130 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:34:33.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:33.406+0000 7f7650e00640 1 -- 172.21.15.5:0/500642492 >> 172.21.15.5:0/500642492 conn(0x7f764c0ff5b0 msgr2=0x7f764c108320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:34:33.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:33.406+0000 7f7650e00640 1 -- 172.21.15.5:0/500642492 shutdown_connections 2024-09-10T09:34:33.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:33.406+0000 7f7650e00640 1 -- 172.21.15.5:0/500642492 wait complete. 2024-09-10T09:34:33.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:34:33 smithi005 ceph-mon[30350]: from='client.? 172.21.15.5:0/500642492' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-10T09:34:33.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:34:33 smithi159 ceph-mon[35513]: from='client.? 172.21.15.5:0/500642492' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-10T09:34:33.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:34:33 smithi102 ceph-mon[35442]: from='client.? 172.21.15.5:0/500642492' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-10T09:34:34.256 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":78,"num_osds":12,"num_up_osds":12,"osd_up_since":1725960870,"num_in_osds":12,"osd_in_since":1725960847,"num_remapped_pgs":0} 2024-09-10T09:34:34.257 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-10T09:34:34.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:34:34 smithi005 ceph-mon[30350]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T09:34:34.633 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:34:34.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:34:34 smithi159 ceph-mon[35513]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T09:34:34.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:34:34 smithi102 ceph-mon[35442]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T09:34:35.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:35.166+0000 7feb3ea00640 1 -- 172.21.15.5:0/2667587814 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feb3810e420 msgr2=0x7feb38110810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:34:35.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:35.166+0000 7feb3ea00640 1 --2- 172.21.15.5:0/2667587814 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feb3810e420 0x7feb38110810 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7feb30009ca0 tx=0x7feb3002a950 comp rx=0 tx=0).stop 2024-09-10T09:34:35.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:35.166+0000 7feb3ea00640 1 -- 172.21.15.5:0/2667587814 shutdown_connections 2024-09-10T09:34:35.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:35.166+0000 7feb3ea00640 1 --2- 172.21.15.5:0/2667587814 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feb3810e420 0x7feb38110810 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:34:35.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:35.166+0000 7feb3ea00640 1 --2- 172.21.15.5:0/2667587814 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7feb38074ee0 0x7feb3810dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:34:35.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:35.166+0000 7feb3ea00640 1 --2- 172.21.15.5:0/2667587814 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7feb380745c0 0x7feb380749a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:34:35.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:35.166+0000 7feb3ea00640 1 -- 172.21.15.5:0/2667587814 >> 172.21.15.5:0/2667587814 conn(0x7feb380fd8c0 msgr2=0x7feb380ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:34:35.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:35.166+0000 7feb3ea00640 1 -- 172.21.15.5:0/2667587814 shutdown_connections 2024-09-10T09:34:35.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:35.166+0000 7feb3ea00640 1 -- 172.21.15.5:0/2667587814 wait complete. 2024-09-10T09:34:35.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:35.167+0000 7feb3ea00640 1 Processor -- start 2024-09-10T09:34:35.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:35.167+0000 7feb3ea00640 1 -- start start 2024-09-10T09:34:35.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:35.167+0000 7feb3ea00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7feb380745c0 0x7feb3819a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:34:35.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:35.167+0000 7feb3ea00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feb38074ee0 0x7feb3819ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:34:35.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:35.167+0000 7feb3ea00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7feb3810e420 0x7feb3819ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:34:35.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:35.167+0000 7feb3ea00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7feb381141b0 con 0x7feb38074ee0 2024-09-10T09:34:35.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:35.167+0000 7feb3ea00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7feb38114030 con 0x7feb380745c0 2024-09-10T09:34:35.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:35.167+0000 7feb3ea00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7feb38114330 con 0x7feb3810e420 2024-09-10T09:34:35.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:35.168+0000 7feb2f400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feb38074ee0 0x7feb3819ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:34:35.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:35.168+0000 7feb2f400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feb38074ee0 0x7feb3819ab90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33114/0 (socket says 172.21.15.5:33114) 2024-09-10T09:34:35.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:35.168+0000 7feb2f400640 1 -- 172.21.15.5:0/889484304 learned_addr learned my addr 172.21.15.5:0/889484304 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:34:35.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:35.168+0000 7feb37e00640 1 --2- 172.21.15.5:0/889484304 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7feb380745c0 0x7feb3819a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:34:35.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:35.168+0000 7feb3ca00640 1 --2- 172.21.15.5:0/889484304 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7feb3810e420 0x7feb3819ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:34:35.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:35.168+0000 7feb2f400640 1 -- 172.21.15.5:0/889484304 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7feb3810e420 msgr2=0x7feb3819ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:34:35.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:35.168+0000 7feb2f400640 1 --2- 172.21.15.5:0/889484304 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7feb3810e420 0x7feb3819ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:34:35.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:35.168+0000 7feb2f400640 1 -- 172.21.15.5:0/889484304 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7feb380745c0 msgr2=0x7feb3819a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:34:35.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:35.168+0000 7feb2f400640 1 --2- 172.21.15.5:0/889484304 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7feb380745c0 0x7feb3819a650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:34:35.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:35.168+0000 7feb2f400640 1 -- 172.21.15.5:0/889484304 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb3819f600 con 0x7feb38074ee0 2024-09-10T09:34:35.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:35.168+0000 7feb2f400640 1 --2- 172.21.15.5:0/889484304 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feb38074ee0 0x7feb3819ab90 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7feb2000d950 tx=0x7feb2000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:34:35.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:35.169+0000 7feb35600640 1 -- 172.21.15.5:0/889484304 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb20017070 con 0x7feb38074ee0 2024-09-10T09:34:35.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:35.169+0000 7feb35600640 1 -- 172.21.15.5:0/889484304 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feb20009c30 con 0x7feb38074ee0 2024-09-10T09:34:35.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:35.169+0000 7feb3ea00640 1 -- 172.21.15.5:0/889484304 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb3819f8f0 con 0x7feb38074ee0 2024-09-10T09:34:35.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:35.169+0000 7feb35600640 1 -- 172.21.15.5:0/889484304 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb2000f6b0 con 0x7feb38074ee0 2024-09-10T09:34:35.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:35.169+0000 7feb3ea00640 1 -- 172.21.15.5:0/889484304 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb381a71d0 con 0x7feb38074ee0 2024-09-10T09:34:35.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:35.170+0000 7feb3ea00640 1 -- 172.21.15.5:0/889484304 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feb380749a0 con 0x7feb38074ee0 2024-09-10T09:34:35.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:35.174+0000 7feb35600640 1 -- 172.21.15.5:0/889484304 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7feb200109d0 con 0x7feb38074ee0 2024-09-10T09:34:35.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:35.175+0000 7feb35600640 1 --2- 172.21.15.5:0/889484304 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7feb1c0a2c00 0x7feb1c0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:34:35.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:35.175+0000 7feb35600640 1 -- 172.21.15.5:0/889484304 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7127+0+0 (secure 0 0 0) 0x7feb200ce690 con 0x7feb38074ee0 2024-09-10T09:34:35.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:35.175+0000 7feb37e00640 1 --2- 172.21.15.5:0/889484304 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7feb1c0a2c00 0x7feb1c0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:34:35.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:35.176+0000 7feb37e00640 1 --2- 172.21.15.5:0/889484304 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7feb1c0a2c00 0x7feb1c0a50c0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7feb28005fd0 tx=0x7feb2800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:34:35.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:35.178+0000 7feb35600640 1 -- 172.21.15.5:0/889484304 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7feb20009de0 con 0x7feb38074ee0 2024-09-10T09:34:35.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:35.363+0000 7feb3ea00640 1 -- 172.21.15.5:0/889484304 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7feb3802d7c0 con 0x7feb38074ee0 2024-09-10T09:34:35.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:35.364+0000 7feb35600640 1 -- 172.21.15.5:0/889484304 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v78) ==== 74+0+18382 (secure 0 0 0) 0x7feb2009b420 con 0x7feb38074ee0 2024-09-10T09:34:35.365 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-10T09:34:35.366 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":78,"fsid":"bade4b50-6f56-11ef-bcea-c7b262605968","created":"2024-09-10T09:27:13.298498+0000","modified":"2024-09-10T09:34:31.731882+0000","last_up_change":"2024-09-10T09:34:30.730066+0000","last_in_change":"2024-09-10T09:34:07.340995+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T09:30:26.042255+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"085749a8-e882-4377-92dd-34bb266c0f0b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":954535266},{"type":"v1","addr":"172.21.15.5:6803","nonce":954535266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":954535266},{"type":"v1","addr":"172.21.15.5:6805","nonce":954535266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":954535266},{"type":"v1","addr":"172.21.15.5:6809","nonce":954535266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":954535266},{"type":"v1","addr":"172.21.15.5:6807","nonce":954535266}]},"public_addr":"172.21.15.5:6803/954535266","cluster_addr":"172.21.15.5:6805/954535266","heartbeat_back_addr":"172.21.15.5:6809/954535266","heartbeat_front_addr":"172.21.15.5:6807/954535266","state":["exists","up"]},{"osd":1,"uuid":"0a76c795-2573-4a92-969a-2ec5dd65c094","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6800","nonce":733924375},{"type":"v1","addr":"172.21.15.102:6801","nonce":733924375}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6802","nonce":733924375},{"type":"v1","addr":"172.21.15.102:6803","nonce":733924375}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6806","nonce":733924375},{"type":"v1","addr":"172.21.15.102:6807","nonce":733924375}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6804","nonce":733924375},{"type":"v1","addr":"172.21.15.102:6805","nonce":733924375}]},"public_addr":"172.21.15.102:6801/733924375","cluster_addr":"172.21.15.102:6803/733924375","heartbeat_back_addr":"172.21.15.102:6807/733924375","heartbeat_front_addr":"172.21.15.102:6805/733924375","state":["exists","up"]},{"osd":2,"uuid":"0c98fcb8-60ab-4473-8995-1d9e97c577e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6800","nonce":2236204189},{"type":"v1","addr":"172.21.15.159:6801","nonce":2236204189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":2236204189},{"type":"v1","addr":"172.21.15.159:6803","nonce":2236204189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":2236204189},{"type":"v1","addr":"172.21.15.159:6807","nonce":2236204189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":2236204189},{"type":"v1","addr":"172.21.15.159:6805","nonce":2236204189}]},"public_addr":"172.21.15.159:6801/2236204189","cluster_addr":"172.21.15.159:6803/2236204189","heartbeat_back_addr":"172.21.15.159:6807/2236204189","heartbeat_front_addr":"172.21.15.159:6805/2236204189","state":["exists","up"]},{"osd":3,"uuid":"88eecdeb-5cee-4f78-a3f9-a66bf4bf772a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":2221468725},{"type":"v1","addr":"172.21.15.5:6811","nonce":2221468725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":2221468725},{"type":"v1","addr":"172.21.15.5:6813","nonce":2221468725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6816","nonce":2221468725},{"type":"v1","addr":"172.21.15.5:6817","nonce":2221468725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":2221468725},{"type":"v1","addr":"172.21.15.5:6815","nonce":2221468725}]},"public_addr":"172.21.15.5:6811/2221468725","cluster_addr":"172.21.15.5:6813/2221468725","heartbeat_back_addr":"172.21.15.5:6817/2221468725","heartbeat_front_addr":"172.21.15.5:6815/2221468725","state":["exists","up"]},{"osd":4,"uuid":"f2e7314d-7938-4668-9011-45c175f66b48","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6808","nonce":594465753},{"type":"v1","addr":"172.21.15.102:6809","nonce":594465753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6810","nonce":594465753},{"type":"v1","addr":"172.21.15.102:6811","nonce":594465753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6814","nonce":594465753},{"type":"v1","addr":"172.21.15.102:6815","nonce":594465753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6812","nonce":594465753},{"type":"v1","addr":"172.21.15.102:6813","nonce":594465753}]},"public_addr":"172.21.15.102:6809/594465753","cluster_addr":"172.21.15.102:6811/594465753","heartbeat_back_addr":"172.21.15.102:6815/594465753","heartbeat_front_addr":"172.21.15.102:6813/594465753","state":["exists","up"]},{"osd":5,"uuid":"0c0366d3-658b-4bf0-ba17-aff28a02e948","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":3230280402},{"type":"v1","addr":"172.21.15.159:6809","nonce":3230280402}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":3230280402},{"type":"v1","addr":"172.21.15.159:6811","nonce":3230280402}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":3230280402},{"type":"v1","addr":"172.21.15.159:6815","nonce":3230280402}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":3230280402},{"type":"v1","addr":"172.21.15.159:6813","nonce":3230280402}]},"public_addr":"172.21.15.159:6809/3230280402","cluster_addr":"172.21.15.159:6811/3230280402","heartbeat_back_addr":"172.21.15.159:6815/3230280402","heartbeat_front_addr":"172.21.15.159:6813/3230280402","state":["exists","up"]},{"osd":6,"uuid":"682670a6-51d5-46e8-af6e-dc2d8c1cf407","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":1907651140},{"type":"v1","addr":"172.21.15.5:6819","nonce":1907651140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":1907651140},{"type":"v1","addr":"172.21.15.5:6821","nonce":1907651140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":1907651140},{"type":"v1","addr":"172.21.15.5:6825","nonce":1907651140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":1907651140},{"type":"v1","addr":"172.21.15.5:6823","nonce":1907651140}]},"public_addr":"172.21.15.5:6819/1907651140","cluster_addr":"172.21.15.5:6821/1907651140","heartbeat_back_addr":"172.21.15.5:6825/1907651140","heartbeat_front_addr":"172.21.15.5:6823/1907651140","state":["exists","up"]},{"osd":7,"uuid":"8ae0f4a9-99e7-4563-aa2a-15169684a8b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6816","nonce":692040360},{"type":"v1","addr":"172.21.15.102:6817","nonce":692040360}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6818","nonce":692040360},{"type":"v1","addr":"172.21.15.102:6819","nonce":692040360}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6822","nonce":692040360},{"type":"v1","addr":"172.21.15.102:6823","nonce":692040360}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6820","nonce":692040360},{"type":"v1","addr":"172.21.15.102:6821","nonce":692040360}]},"public_addr":"172.21.15.102:6817/692040360","cluster_addr":"172.21.15.102:6819/692040360","heartbeat_back_addr":"172.21.15.102:6823/692040360","heartbeat_front_addr":"172.21.15.102:6821/692040360","state":["exists","up"]},{"osd":8,"uuid":"e37c845e-ced9-4413-9bbe-bc090537c2bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":2107291799},{"type":"v1","addr":"172.21.15.159:6817","nonce":2107291799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":2107291799},{"type":"v1","addr":"172.21.15.159:6819","nonce":2107291799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":2107291799},{"type":"v1","addr":"172.21.15.159:6823","nonce":2107291799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":2107291799},{"type":"v1","addr":"172.21.15.159:6821","nonce":2107291799}]},"public_addr":"172.21.15.159:6817/2107291799","cluster_addr":"172.21.15.159:6819/2107291799","heartbeat_back_addr":"172.21.15.159:6823/2107291799","heartbeat_front_addr":"172.21.15.159:6821/2107291799","state":["exists","up"]},{"osd":9,"uuid":"91a5b6cc-08da-47be-af12-d0896a849309","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":902431281},{"type":"v1","addr":"172.21.15.5:6827","nonce":902431281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":902431281},{"type":"v1","addr":"172.21.15.5:6829","nonce":902431281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":902431281},{"type":"v1","addr":"172.21.15.5:6833","nonce":902431281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":902431281},{"type":"v1","addr":"172.21.15.5:6831","nonce":902431281}]},"public_addr":"172.21.15.5:6827/902431281","cluster_addr":"172.21.15.5:6829/902431281","heartbeat_back_addr":"172.21.15.5:6833/902431281","heartbeat_front_addr":"172.21.15.5:6831/902431281","state":["exists","up"]},{"osd":10,"uuid":"2211030c-b00e-402b-b4d7-c587eb29a490","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6824","nonce":2853806211},{"type":"v1","addr":"172.21.15.102:6825","nonce":2853806211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6826","nonce":2853806211},{"type":"v1","addr":"172.21.15.102:6827","nonce":2853806211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6830","nonce":2853806211},{"type":"v1","addr":"172.21.15.102:6831","nonce":2853806211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6828","nonce":2853806211},{"type":"v1","addr":"172.21.15.102:6829","nonce":2853806211}]},"public_addr":"172.21.15.102:6825/2853806211","cluster_addr":"172.21.15.102:6827/2853806211","heartbeat_back_addr":"172.21.15.102:6831/2853806211","heartbeat_front_addr":"172.21.15.102:6829/2853806211","state":["exists","up"]},{"osd":11,"uuid":"4e9699ec-8ff2-4005-9d84-af5600b3a2ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6824","nonce":969941218},{"type":"v1","addr":"172.21.15.159:6825","nonce":969941218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":969941218},{"type":"v1","addr":"172.21.15.159:6827","nonce":969941218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6830","nonce":969941218},{"type":"v1","addr":"172.21.15.159:6831","nonce":969941218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":969941218},{"type":"v1","addr":"172.21.15.159:6829","nonce":969941218}]},"public_addr":"172.21.15.159:6825/969941218","cluster_addr":"172.21.15.159:6827/969941218","heartbeat_back_addr":"172.21.15.159:6831/969941218","heartbeat_front_addr":"172.21.15.159:6829/969941218","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:29:34.416490+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:30:00.246101+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:30:24.300464+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:30:54.663623+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:31:21.426292+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:31:46.738914+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:32:16.196571+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:32:42.549243+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:33:08.178651+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:33:37.477911+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:34:03.114026+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:34:28.891924+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.5:6801/1527422686":"2024-09-11T09:28:05.436742+0000","172.21.15.5:0/2276144345":"2024-09-11T09:28:05.436742+0000","172.21.15.5:0/412040104":"2024-09-11T09:28:05.436742+0000","172.21.15.5:6800/1527422686":"2024-09-11T09:28:05.436742+0000","172.21.15.5:0/1856581170":"2024-09-11T09:28:05.436742+0000","172.21.15.5:0/2847668298":"2024-09-11T09:27:37.778500+0000","172.21.15.5:6801/14425913":"2024-09-11T09:27:37.778500+0000","172.21.15.5:0/166910782":"2024-09-11T09:27:37.778500+0000","172.21.15.5:6800/14425913":"2024-09-11T09:27:37.778500+0000","172.21.15.5:0/1711858179":"2024-09-11T09:27:37.778500+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T09:34:35.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:35.367+0000 7feb3ea00640 1 -- 172.21.15.5:0/889484304 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7feb1c0a2c00 msgr2=0x7feb1c0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:34:35.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:35.367+0000 7feb3ea00640 1 --2- 172.21.15.5:0/889484304 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7feb1c0a2c00 0x7feb1c0a50c0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7feb28005fd0 tx=0x7feb2800b390 comp rx=0 tx=0).stop 2024-09-10T09:34:35.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:35.367+0000 7feb3ea00640 1 -- 172.21.15.5:0/889484304 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feb38074ee0 msgr2=0x7feb3819ab90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:34:35.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:35.367+0000 7feb3ea00640 1 --2- 172.21.15.5:0/889484304 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feb38074ee0 0x7feb3819ab90 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7feb2000d950 tx=0x7feb2000de10 comp rx=0 tx=0).stop 2024-09-10T09:34:35.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:35.367+0000 7feb3ea00640 1 -- 172.21.15.5:0/889484304 shutdown_connections 2024-09-10T09:34:35.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:35.367+0000 7feb3ea00640 1 --2- 172.21.15.5:0/889484304 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7feb1c0a2c00 0x7feb1c0a50c0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:34:35.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:35.367+0000 7feb3ea00640 1 --2- 172.21.15.5:0/889484304 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7feb3810e420 0x7feb3819ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:34:35.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:35.367+0000 7feb3ea00640 1 --2- 172.21.15.5:0/889484304 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feb38074ee0 0x7feb3819ab90 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:34:35.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:35.367+0000 7feb3ea00640 1 --2- 172.21.15.5:0/889484304 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7feb380745c0 0x7feb3819a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:34:35.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:35.367+0000 7feb3ea00640 1 -- 172.21.15.5:0/889484304 >> 172.21.15.5:0/889484304 conn(0x7feb380fd8c0 msgr2=0x7feb380fe520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:34:35.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:35.367+0000 7feb3ea00640 1 -- 172.21.15.5:0/889484304 shutdown_connections 2024-09-10T09:34:35.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:35.367+0000 7feb3ea00640 1 -- 172.21.15.5:0/889484304 wait complete. 2024-09-10T09:34:35.714 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:34:35 smithi005 ceph-mon[30350]: from='client.? 172.21.15.5:0/889484304' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T09:34:35.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:34:35 smithi159 ceph-mon[35513]: from='client.? 172.21.15.5:0/889484304' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T09:34:35.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:34:35 smithi102 ceph-mon[35442]: from='client.? 172.21.15.5:0/889484304' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T09:34:36.200 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-10T09:30:26.042255+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-10T09:34:36.201 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-09-10T09:34:36.572 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:34:36.727 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:34:36 smithi005 ceph-mon[30350]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T09:34:36.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:34:36 smithi159 ceph-mon[35513]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T09:34:36.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:34:36 smithi102 ceph-mon[35442]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T09:34:37.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:37.119+0000 7f0131800640 1 -- 172.21.15.5:0/3479894588 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f012c103ff0 msgr2=0x7f012c1043d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:34:37.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:37.119+0000 7f0131800640 1 --2- 172.21.15.5:0/3479894588 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f012c103ff0 0x7f012c1043d0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f011c009900 tx=0x7f011c02a410 comp rx=0 tx=0).stop 2024-09-10T09:34:37.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:37.119+0000 7f0131800640 1 -- 172.21.15.5:0/3479894588 shutdown_connections 2024-09-10T09:34:37.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:37.119+0000 7f0131800640 1 --2- 172.21.15.5:0/3479894588 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f012c109120 0x7f012c110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:34:37.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:37.119+0000 7f0131800640 1 --2- 172.21.15.5:0/3479894588 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f012c1049a0 0x7f012c1089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:34:37.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:37.119+0000 7f0131800640 1 --2- 172.21.15.5:0/3479894588 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f012c103ff0 0x7f012c1043d0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:34:37.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:37.119+0000 7f0131800640 1 -- 172.21.15.5:0/3479894588 >> 172.21.15.5:0/3479894588 conn(0x7f012c0ffa90 msgr2=0x7f012c101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:34:37.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:37.120+0000 7f0131800640 1 -- 172.21.15.5:0/3479894588 shutdown_connections 2024-09-10T09:34:37.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:37.120+0000 7f0131800640 1 -- 172.21.15.5:0/3479894588 wait complete. 2024-09-10T09:34:37.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:37.120+0000 7f0131800640 1 Processor -- start 2024-09-10T09:34:37.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:37.121+0000 7f0131800640 1 -- start start 2024-09-10T09:34:37.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:37.121+0000 7f0131800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f012c103ff0 0x7f012c19a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:34:37.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:37.121+0000 7f0131800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f012c1049a0 0x7f012c19abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:34:37.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:37.121+0000 7f0131800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f012c109120 0x7f012c19ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:34:37.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:37.121+0000 7f0131800640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f012c114170 con 0x7f012c103ff0 2024-09-10T09:34:37.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:37.121+0000 7f0131800640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f012c113ff0 con 0x7f012c109120 2024-09-10T09:34:37.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:37.121+0000 7f0131800640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f012c1142f0 con 0x7f012c1049a0 2024-09-10T09:34:37.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:37.121+0000 7f012aa00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f012c103ff0 0x7f012c19a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:34:37.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:37.121+0000 7f012aa00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f012c103ff0 0x7f012c19a670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33132/0 (socket says 172.21.15.5:33132) 2024-09-10T09:34:37.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:37.121+0000 7f012aa00640 1 -- 172.21.15.5:0/2013728440 learned_addr learned my addr 172.21.15.5:0/2013728440 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:34:37.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:37.121+0000 7f0122000640 1 --2- 172.21.15.5:0/2013728440 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f012c1049a0 0x7f012c19abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:34:37.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:37.122+0000 7f012b400640 1 --2- 172.21.15.5:0/2013728440 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f012c109120 0x7f012c19ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:34:37.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:37.122+0000 7f012aa00640 1 -- 172.21.15.5:0/2013728440 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f012c1049a0 msgr2=0x7f012c19abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:34:37.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:37.122+0000 7f012aa00640 1 --2- 172.21.15.5:0/2013728440 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f012c1049a0 0x7f012c19abb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:34:37.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:37.122+0000 7f012aa00640 1 -- 172.21.15.5:0/2013728440 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f012c109120 msgr2=0x7f012c19ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:34:37.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:37.122+0000 7f012aa00640 1 --2- 172.21.15.5:0/2013728440 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f012c109120 0x7f012c19ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:34:37.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:37.122+0000 7f012aa00640 1 -- 172.21.15.5:0/2013728440 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f012c19f620 con 0x7f012c103ff0 2024-09-10T09:34:37.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:37.122+0000 7f012aa00640 1 --2- 172.21.15.5:0/2013728440 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f012c103ff0 0x7f012c19a670 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f011c004770 tx=0x7f011c003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:34:37.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:37.123+0000 7f0123e00640 1 -- 172.21.15.5:0/2013728440 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f011c03f070 con 0x7f012c103ff0 2024-09-10T09:34:37.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:37.123+0000 7f0123e00640 1 -- 172.21.15.5:0/2013728440 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f011c031ca0 con 0x7f012c103ff0 2024-09-10T09:34:37.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:37.123+0000 7f0131800640 1 -- 172.21.15.5:0/2013728440 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f012c19f8b0 con 0x7f012c103ff0 2024-09-10T09:34:37.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:37.123+0000 7f0123e00640 1 -- 172.21.15.5:0/2013728440 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f011c035030 con 0x7f012c103ff0 2024-09-10T09:34:37.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:37.123+0000 7f0131800640 1 -- 172.21.15.5:0/2013728440 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f012c1a71f0 con 0x7f012c103ff0 2024-09-10T09:34:37.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:37.124+0000 7f0131800640 1 -- 172.21.15.5:0/2013728440 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f012c1054d0 con 0x7f012c103ff0 2024-09-10T09:34:37.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:37.126+0000 7f0123e00640 1 -- 172.21.15.5:0/2013728440 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7f011c042070 con 0x7f012c103ff0 2024-09-10T09:34:37.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:37.127+0000 7f0123e00640 1 --2- 172.21.15.5:0/2013728440 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f01100a2b80 0x7f01100a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:34:37.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:37.127+0000 7f0123e00640 1 -- 172.21.15.5:0/2013728440 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7127+0+0 (secure 0 0 0) 0x7f011c0ef2e0 con 0x7f012c103ff0 2024-09-10T09:34:37.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:37.127+0000 7f0122000640 1 --2- 172.21.15.5:0/2013728440 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f01100a2b80 0x7f01100a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:34:37.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:37.128+0000 7f0122000640 1 --2- 172.21.15.5:0/2013728440 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f01100a2b80 0x7f01100a5040 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f012c0fff40 tx=0x7f0114005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:34:37.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:37.131+0000 7f0123e00640 1 -- 172.21.15.5:0/2013728440 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f011c0bbff0 con 0x7f012c103ff0 2024-09-10T09:34:37.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:37.317+0000 7f0131800640 1 -- 172.21.15.5:0/2013728440 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f012c10c300 con 0x7f012c103ff0 2024-09-10T09:34:37.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:37.318+0000 7f0123e00640 1 -- 172.21.15.5:0/2013728440 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v78) ==== 93+0+10 (secure 0 0 0) 0x7f011c0c02f0 con 0x7f012c103ff0 2024-09-10T09:34:37.319 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 1 2024-09-10T09:34:37.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:37.320+0000 7f0131800640 1 -- 172.21.15.5:0/2013728440 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f01100a2b80 msgr2=0x7f01100a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:34:37.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:37.320+0000 7f0131800640 1 --2- 172.21.15.5:0/2013728440 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f01100a2b80 0x7f01100a5040 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f012c0fff40 tx=0x7f0114005f90 comp rx=0 tx=0).stop 2024-09-10T09:34:37.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:37.320+0000 7f0131800640 1 -- 172.21.15.5:0/2013728440 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f012c103ff0 msgr2=0x7f012c19a670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:34:37.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:37.320+0000 7f0131800640 1 --2- 172.21.15.5:0/2013728440 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f012c103ff0 0x7f012c19a670 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f011c004770 tx=0x7f011c003c10 comp rx=0 tx=0).stop 2024-09-10T09:34:37.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:37.320+0000 7f0131800640 1 -- 172.21.15.5:0/2013728440 shutdown_connections 2024-09-10T09:34:37.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:37.320+0000 7f0131800640 1 --2- 172.21.15.5:0/2013728440 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f01100a2b80 0x7f01100a5040 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:34:37.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:37.320+0000 7f0131800640 1 --2- 172.21.15.5:0/2013728440 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f012c109120 0x7f012c19ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:34:37.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:37.320+0000 7f0131800640 1 --2- 172.21.15.5:0/2013728440 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f012c1049a0 0x7f012c19abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:34:37.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:37.320+0000 7f0131800640 1 --2- 172.21.15.5:0/2013728440 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f012c103ff0 0x7f012c19a670 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:34:37.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:37.320+0000 7f0131800640 1 -- 172.21.15.5:0/2013728440 >> 172.21.15.5:0/2013728440 conn(0x7f012c0ffa90 msgr2=0x7f012c100d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:34:37.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:37.321+0000 7f0131800640 1 -- 172.21.15.5:0/2013728440 shutdown_connections 2024-09-10T09:34:37.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:37.321+0000 7f0131800640 1 -- 172.21.15.5:0/2013728440 wait complete. 2024-09-10T09:34:37.659 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:34:37 smithi005 ceph-mon[30350]: from='client.? 172.21.15.5:0/2013728440' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-10T09:34:37.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:34:37 smithi159 ceph-mon[35513]: from='client.? 172.21.15.5:0/2013728440' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-10T09:34:37.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:34:37 smithi102 ceph-mon[35442]: from='client.? 172.21.15.5:0/2013728440' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-10T09:34:38.048 INFO:tasks.cephadm:Adding mds.a on smithi005 2024-09-10T09:34:38.048 INFO:tasks.cephadm:Adding mds.d on smithi005 2024-09-10T09:34:38.048 INFO:tasks.cephadm:Adding mds.g on smithi005 2024-09-10T09:34:38.048 INFO:tasks.cephadm:Adding mds.j on smithi005 2024-09-10T09:34:38.048 INFO:tasks.cephadm:Adding mds.b on smithi102 2024-09-10T09:34:38.048 INFO:tasks.cephadm:Adding mds.e on smithi102 2024-09-10T09:34:38.049 INFO:tasks.cephadm:Adding mds.h on smithi102 2024-09-10T09:34:38.049 INFO:tasks.cephadm:Adding mds.k on smithi102 2024-09-10T09:34:38.049 INFO:tasks.cephadm:Adding mds.c on smithi159 2024-09-10T09:34:38.049 INFO:tasks.cephadm:Adding mds.f on smithi159 2024-09-10T09:34:38.049 INFO:tasks.cephadm:Adding mds.i on smithi159 2024-09-10T09:34:38.049 INFO:tasks.cephadm:Adding mds.l on smithi159 2024-09-10T09:34:38.049 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph orch apply mds all '12;smithi005=a;smithi005=d;smithi005=g;smithi005=j;smithi102=b;smithi102=e;smithi102=h;smithi102=k;smithi159=c;smithi159=f;smithi159=i;smithi159=l' 2024-09-10T09:34:38.306 INFO:teuthology.orchestra.run.smithi159.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.c/config 2024-09-10T09:34:38.719 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:34:38 smithi159 ceph-mon[35513]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T09:34:38.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:34:38 smithi005 ceph-mon[30350]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T09:34:38.827 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:34:38.804+0000 7f698a326640 1 -- 172.21.15.159:0/47600652 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6984100ad0 msgr2=0x7f6984100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:34:38.827 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:34:38.804+0000 7f698a326640 1 --2- 172.21.15.159:0/47600652 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6984100ad0 0x7f6984100ed0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f6974009900 tx=0x7f697402a410 comp rx=0 tx=0).stop 2024-09-10T09:34:38.827 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:34:38.804+0000 7f698a326640 1 -- 172.21.15.159:0/47600652 shutdown_connections 2024-09-10T09:34:38.827 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:34:38.805+0000 7f698a326640 1 --2- 172.21.15.159:0/47600652 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6984102690 0x7f6984108f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:34:38.827 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:34:38.805+0000 7f698a326640 1 --2- 172.21.15.159:0/47600652 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6984101cd0 0x7f6984102150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:34:38.827 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:34:38.805+0000 7f698a326640 1 --2- 172.21.15.159:0/47600652 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6984100ad0 0x7f6984100ed0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:34:38.827 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:34:38.805+0000 7f698a326640 1 -- 172.21.15.159:0/47600652 >> 172.21.15.159:0/47600652 conn(0x7f69840fc280 msgr2=0x7f69840fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:34:38.827 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:34:38.805+0000 7f698a326640 1 -- 172.21.15.159:0/47600652 shutdown_connections 2024-09-10T09:34:38.827 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:34:38.805+0000 7f698a326640 1 -- 172.21.15.159:0/47600652 wait complete. 2024-09-10T09:34:38.827 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:34:38.805+0000 7f698a326640 1 Processor -- start 2024-09-10T09:34:38.827 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:34:38.805+0000 7f698a326640 1 -- start start 2024-09-10T09:34:38.827 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:34:38.806+0000 7f698a326640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6984100ad0 0x7f698410b450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:34:38.828 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:34:38.806+0000 7f698a326640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6984101cd0 0x7f6984109aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:34:38.828 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:34:38.806+0000 7f698a326640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6984102690 0x7f6984109fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:34:38.828 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:34:38.806+0000 7f698a326640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f698410d940 con 0x7f6984101cd0 2024-09-10T09:34:38.828 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:34:38.806+0000 7f698a326640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f698410d7c0 con 0x7f6984100ad0 2024-09-10T09:34:38.828 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:34:38.806+0000 7f698a326640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f698410dac0 con 0x7f6984102690 2024-09-10T09:34:38.828 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:34:38.806+0000 7f69888c3640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6984102690 0x7f6984109fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:34:38.828 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:34:38.806+0000 7f69888c3640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6984102690 0x7f6984109fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.159:53182/0 (socket says 172.21.15.159:53182) 2024-09-10T09:34:38.828 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:34:38.806+0000 7f69888c3640 1 -- 172.21.15.159:0/2605170498 learned_addr learned my addr 172.21.15.159:0/2605170498 (peer_addr_for_me v2:172.21.15.159:0/0) 2024-09-10T09:34:38.828 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:34:38.806+0000 7f6983fff640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6984100ad0 0x7f698410b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:34:38.828 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:34:38.806+0000 7f69837fe640 1 --2- 172.21.15.159:0/2605170498 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6984101cd0 0x7f6984109aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:34:38.828 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:34:38.806+0000 7f69888c3640 1 -- 172.21.15.159:0/2605170498 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6984100ad0 msgr2=0x7f698410b450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:34:38.828 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:34:38.806+0000 7f69888c3640 1 --2- 172.21.15.159:0/2605170498 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6984100ad0 0x7f698410b450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:34:38.828 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:34:38.806+0000 7f69888c3640 1 -- 172.21.15.159:0/2605170498 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6984101cd0 msgr2=0x7f6984109aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:34:38.829 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:34:38.806+0000 7f69888c3640 1 --2- 172.21.15.159:0/2605170498 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6984101cd0 0x7f6984109aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:34:38.829 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:34:38.806+0000 7f69888c3640 1 -- 172.21.15.159:0/2605170498 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f698410a8a0 con 0x7f6984102690 2024-09-10T09:34:38.829 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:34:38.806+0000 7f69888c3640 1 --2- 172.21.15.159:0/2605170498 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6984102690 0x7f6984109fe0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f697c00b970 tx=0x7f697c00be30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:34:38.829 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:34:38.806+0000 7f69817fa640 1 -- 172.21.15.159:0/2605170498 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f697c019070 con 0x7f6984102690 2024-09-10T09:34:38.829 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:34:38.806+0000 7f698a326640 1 -- 172.21.15.159:0/2605170498 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69841a21e0 con 0x7f6984102690 2024-09-10T09:34:38.829 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:34:38.806+0000 7f69817fa640 1 -- 172.21.15.159:0/2605170498 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f697c00cb40 con 0x7f6984102690 2024-09-10T09:34:38.829 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:34:38.806+0000 7f69817fa640 1 -- 172.21.15.159:0/2605170498 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f697c007610 con 0x7f6984102690 2024-09-10T09:34:38.829 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:34:38.806+0000 7f698a326640 1 -- 172.21.15.159:0/2605170498 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69841a27c0 con 0x7f6984102690 2024-09-10T09:34:38.829 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:34:38.808+0000 7f698a326640 1 -- 172.21.15.159:0/2605170498 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6984107f10 con 0x7f6984102690 2024-09-10T09:34:38.829 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:34:38.809+0000 7f69817fa640 1 -- 172.21.15.159:0/2605170498 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7f697c01c070 con 0x7f6984102690 2024-09-10T09:34:38.829 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:34:38.810+0000 7f69817fa640 1 --2- 172.21.15.159:0/2605170498 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f696c0a2b80 0x7f696c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:34:38.829 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:34:38.810+0000 7f69817fa640 1 -- 172.21.15.159:0/2605170498 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7127+0+0 (secure 0 0 0) 0x7f697c0ce5a0 con 0x7f6984102690 2024-09-10T09:34:38.829 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:34:38.810+0000 7f6983fff640 1 --2- 172.21.15.159:0/2605170498 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f696c0a2b80 0x7f696c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:34:38.829 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:34:38.811+0000 7f6983fff640 1 --2- 172.21.15.159:0/2605170498 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f696c0a2b80 0x7f696c0a5040 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f6974005ab0 tx=0x7f6974000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:34:38.830 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:34:38.813+0000 7f69817fa640 1 -- 172.21.15.159:0/2605170498 <== mon.1 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f697c09b330 con 0x7f6984102690 2024-09-10T09:34:38.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:34:38 smithi102 ceph-mon[35442]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T09:34:38.997 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:34:38.996+0000 7f698a326640 1 -- 172.21.15.159:0/2605170498 --> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi005=a;smithi005=d;smithi005=g;smithi005=j;smithi102=b;smithi102=e;smithi102=h;smithi102=k;smithi159=c;smithi159=f;smithi159=i;smithi159=l", "target": ["mon-mgr", ""]}) -- 0x7f6984062730 con 0x7f696c0a2b80 2024-09-10T09:34:39.005 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:34:39.005+0000 7f69817fa640 1 -- 172.21.15.159:0/2605170498 <== mgr.14150 v2:172.21.15.5:6800/2707348081 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (secure 0 0 0) 0x7f6984062730 con 0x7f696c0a2b80 2024-09-10T09:34:39.006 INFO:teuthology.orchestra.run.smithi159.stdout:Scheduled mds.all update... 2024-09-10T09:34:39.008 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:34:39.007+0000 7f698a326640 1 -- 172.21.15.159:0/2605170498 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f696c0a2b80 msgr2=0x7f696c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:34:39.008 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:34:39.007+0000 7f698a326640 1 --2- 172.21.15.159:0/2605170498 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f696c0a2b80 0x7f696c0a5040 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f6974005ab0 tx=0x7f6974000bc0 comp rx=0 tx=0).stop 2024-09-10T09:34:39.008 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:34:39.007+0000 7f698a326640 1 -- 172.21.15.159:0/2605170498 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6984102690 msgr2=0x7f6984109fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:34:39.008 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:34:39.007+0000 7f698a326640 1 --2- 172.21.15.159:0/2605170498 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6984102690 0x7f6984109fe0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f697c00b970 tx=0x7f697c00be30 comp rx=0 tx=0).stop 2024-09-10T09:34:39.008 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:34:39.007+0000 7f698a326640 1 -- 172.21.15.159:0/2605170498 shutdown_connections 2024-09-10T09:34:39.008 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:34:39.007+0000 7f698a326640 1 --2- 172.21.15.159:0/2605170498 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f696c0a2b80 0x7f696c0a5040 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:34:39.008 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:34:39.007+0000 7f698a326640 1 --2- 172.21.15.159:0/2605170498 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6984102690 0x7f6984109fe0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:34:39.009 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:34:39.007+0000 7f698a326640 1 --2- 172.21.15.159:0/2605170498 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6984101cd0 0x7f6984109aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:34:39.009 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:34:39.007+0000 7f698a326640 1 --2- 172.21.15.159:0/2605170498 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6984100ad0 0x7f698410b450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:34:39.009 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:34:39.007+0000 7f698a326640 1 -- 172.21.15.159:0/2605170498 >> 172.21.15.159:0/2605170498 conn(0x7f69840fc280 msgr2=0x7f69841050e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:34:39.009 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:34:39.007+0000 7f698a326640 1 -- 172.21.15.159:0/2605170498 shutdown_connections 2024-09-10T09:34:39.009 INFO:teuthology.orchestra.run.smithi159.stderr:2024-09-10T09:34:39.007+0000 7f698a326640 1 -- 172.21.15.159:0/2605170498 wait complete. 2024-09-10T09:34:39.926 DEBUG:teuthology.orchestra.run.smithi005:mds.a> sudo journalctl -f -n 0 -u ceph-bade4b50-6f56-11ef-bcea-c7b262605968@mds.a.service 2024-09-10T09:34:39.931 DEBUG:teuthology.orchestra.run.smithi005:mds.d> sudo journalctl -f -n 0 -u ceph-bade4b50-6f56-11ef-bcea-c7b262605968@mds.d.service 2024-09-10T09:34:39.977 DEBUG:teuthology.orchestra.run.smithi005:mds.g> sudo journalctl -f -n 0 -u ceph-bade4b50-6f56-11ef-bcea-c7b262605968@mds.g.service 2024-09-10T09:34:40.023 DEBUG:teuthology.orchestra.run.smithi005:mds.j> sudo journalctl -f -n 0 -u ceph-bade4b50-6f56-11ef-bcea-c7b262605968@mds.j.service 2024-09-10T09:34:40.068 DEBUG:teuthology.orchestra.run.smithi102:mds.b> sudo journalctl -f -n 0 -u ceph-bade4b50-6f56-11ef-bcea-c7b262605968@mds.b.service 2024-09-10T09:34:40.071 DEBUG:teuthology.orchestra.run.smithi102:mds.e> sudo journalctl -f -n 0 -u ceph-bade4b50-6f56-11ef-bcea-c7b262605968@mds.e.service 2024-09-10T09:34:40.114 DEBUG:teuthology.orchestra.run.smithi102:mds.h> sudo journalctl -f -n 0 -u ceph-bade4b50-6f56-11ef-bcea-c7b262605968@mds.h.service 2024-09-10T09:34:40.157 DEBUG:teuthology.orchestra.run.smithi102:mds.k> sudo journalctl -f -n 0 -u ceph-bade4b50-6f56-11ef-bcea-c7b262605968@mds.k.service 2024-09-10T09:34:40.166 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:34:40 smithi159 ceph-mon[35513]: from='client.24448 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi005=a;smithi005=d;smithi005=g;smithi005=j;smithi102=b;smithi102=e;smithi102=h;smithi102=k;smithi159=c;smithi159=f;smithi159=i;smithi159=l", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T09:34:40.166 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:34:40 smithi159 ceph-mon[35513]: Saving service mds.all spec with placement smithi005=a;smithi005=d;smithi005=g;smithi005=j;smithi102=b;smithi102=e;smithi102=h;smithi102=k;smithi159=c;smithi159=f;smithi159=i;smithi159=l;count:12 2024-09-10T09:34:40.166 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:34:40 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:34:40.166 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:34:40 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:34:40.166 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:34:40 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:34:40.166 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:34:40 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:34:40.167 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:34:40 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:34:40.167 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:34:40 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T09:34:40.167 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:34:40 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T09:34:40.167 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:34:40 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:34:40.167 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:34:40 smithi159 ceph-mon[35513]: Deploying daemon mds.l on smithi159 2024-09-10T09:34:40.167 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:34:40 smithi159 ceph-mon[35513]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T09:34:40.200 DEBUG:teuthology.orchestra.run.smithi159:mds.c> sudo journalctl -f -n 0 -u ceph-bade4b50-6f56-11ef-bcea-c7b262605968@mds.c.service 2024-09-10T09:34:40.202 DEBUG:teuthology.orchestra.run.smithi159:mds.f> sudo journalctl -f -n 0 -u ceph-bade4b50-6f56-11ef-bcea-c7b262605968@mds.f.service 2024-09-10T09:34:40.245 DEBUG:teuthology.orchestra.run.smithi159:mds.i> sudo journalctl -f -n 0 -u ceph-bade4b50-6f56-11ef-bcea-c7b262605968@mds.i.service 2024-09-10T09:34:40.288 DEBUG:teuthology.orchestra.run.smithi159:mds.l> sudo journalctl -f -n 0 -u ceph-bade4b50-6f56-11ef-bcea-c7b262605968@mds.l.service 2024-09-10T09:34:40.306 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:34:40 smithi005 ceph-mon[30350]: from='client.24448 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi005=a;smithi005=d;smithi005=g;smithi005=j;smithi102=b;smithi102=e;smithi102=h;smithi102=k;smithi159=c;smithi159=f;smithi159=i;smithi159=l", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T09:34:40.306 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:34:40 smithi005 ceph-mon[30350]: Saving service mds.all spec with placement smithi005=a;smithi005=d;smithi005=g;smithi005=j;smithi102=b;smithi102=e;smithi102=h;smithi102=k;smithi159=c;smithi159=f;smithi159=i;smithi159=l;count:12 2024-09-10T09:34:40.307 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:34:40 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:34:40.307 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:34:40 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:34:40.307 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:34:40 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:34:40.307 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:34:40 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:34:40.307 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:34:40 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:34:40.307 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:34:40 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T09:34:40.307 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:34:40 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T09:34:40.307 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:34:40 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:34:40.307 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:34:40 smithi005 ceph-mon[30350]: Deploying daemon mds.l on smithi159 2024-09-10T09:34:40.307 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:34:40 smithi005 ceph-mon[30350]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T09:34:40.331 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-09-10T09:34:40.332 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T09:34:40.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:34:40 smithi102 ceph-mon[35442]: from='client.24448 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi005=a;smithi005=d;smithi005=g;smithi005=j;smithi102=b;smithi102=e;smithi102=h;smithi102=k;smithi159=c;smithi159=f;smithi159=i;smithi159=l", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T09:34:40.385 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:34:40 smithi102 ceph-mon[35442]: Saving service mds.all spec with placement smithi005=a;smithi005=d;smithi005=g;smithi005=j;smithi102=b;smithi102=e;smithi102=h;smithi102=k;smithi159=c;smithi159=f;smithi159=i;smithi159=l;count:12 2024-09-10T09:34:40.385 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:34:40 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:34:40.385 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:34:40 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:34:40.385 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:34:40 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:34:40.385 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:34:40 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:34:40.385 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:34:40 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:34:40.385 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:34:40 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T09:34:40.385 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:34:40 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T09:34:40.385 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:34:40 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:34:40.385 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:34:40 smithi102 ceph-mon[35442]: Deploying daemon mds.l on smithi159 2024-09-10T09:34:40.386 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:34:40 smithi102 ceph-mon[35442]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T09:34:40.714 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:34:41.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:41.278+0000 7f1404c00640 1 -- 172.21.15.5:0/3445021629 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1400074eb0 msgr2=0x7f1400112650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:34:41.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:41.278+0000 7f1404c00640 1 --2- 172.21.15.5:0/3445021629 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1400074eb0 0x7f1400112650 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f13e8009900 tx=0x7f13e802a480 comp rx=0 tx=0).stop 2024-09-10T09:34:41.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:41.278+0000 7f1404c00640 1 -- 172.21.15.5:0/3445021629 shutdown_connections 2024-09-10T09:34:41.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:41.278+0000 7f1404c00640 1 --2- 172.21.15.5:0/3445021629 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1400112b90 0x7f1400114f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:34:41.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:41.278+0000 7f1404c00640 1 --2- 172.21.15.5:0/3445021629 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1400074eb0 0x7f1400112650 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:34:41.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:41.278+0000 7f1404c00640 1 --2- 172.21.15.5:0/3445021629 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1400074590 0x7f1400074970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:34:41.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:41.278+0000 7f1404c00640 1 -- 172.21.15.5:0/3445021629 >> 172.21.15.5:0/3445021629 conn(0x7f14000fd8c0 msgr2=0x7f14000ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:34:41.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:41.278+0000 7f1404c00640 1 -- 172.21.15.5:0/3445021629 shutdown_connections 2024-09-10T09:34:41.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:41.278+0000 7f1404c00640 1 -- 172.21.15.5:0/3445021629 wait complete. 2024-09-10T09:34:41.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:41.279+0000 7f1404c00640 1 Processor -- start 2024-09-10T09:34:41.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:41.279+0000 7f1404c00640 1 -- start start 2024-09-10T09:34:41.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:41.280+0000 7f1404c00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1400074590 0x7f140019eb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:34:41.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:41.280+0000 7f1404c00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1400074eb0 0x7f140019f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:34:41.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:41.280+0000 7f1404c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1400112b90 0x7f14001a3440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:34:41.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:41.280+0000 7f1404c00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f14001185b0 con 0x7f1400074eb0 2024-09-10T09:34:41.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:41.280+0000 7f1404c00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f1400118430 con 0x7f1400112b90 2024-09-10T09:34:41.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:41.280+0000 7f1404c00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f1400118730 con 0x7f1400074590 2024-09-10T09:34:41.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:41.280+0000 7f13fde00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1400074590 0x7f140019eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:34:41.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:41.280+0000 7f13f5400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1400074eb0 0x7f140019f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:34:41.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:41.280+0000 7f13fde00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1400074590 0x7f140019eb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:39050/0 (socket says 172.21.15.5:39050) 2024-09-10T09:34:41.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:41.280+0000 7f13fde00640 1 -- 172.21.15.5:0/1990678553 learned_addr learned my addr 172.21.15.5:0/1990678553 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:34:41.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:41.280+0000 7f13fe800640 1 --2- 172.21.15.5:0/1990678553 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1400112b90 0x7f14001a3440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:34:41.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:41.280+0000 7f13f5400640 1 -- 172.21.15.5:0/1990678553 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1400074590 msgr2=0x7f140019eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:34:41.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:41.280+0000 7f13f5400640 1 --2- 172.21.15.5:0/1990678553 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1400074590 0x7f140019eb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:34:41.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:41.280+0000 7f13f5400640 1 -- 172.21.15.5:0/1990678553 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1400112b90 msgr2=0x7f14001a3440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:34:41.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:41.280+0000 7f13f5400640 1 --2- 172.21.15.5:0/1990678553 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1400112b90 0x7f14001a3440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:34:41.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:41.280+0000 7f13f5400640 1 -- 172.21.15.5:0/1990678553 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14001a3b70 con 0x7f1400074eb0 2024-09-10T09:34:41.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:41.281+0000 7f13f5400640 1 --2- 172.21.15.5:0/1990678553 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1400074eb0 0x7f140019f080 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f13e802a9a0 tx=0x7f13e8003b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:34:41.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:41.281+0000 7f13f7400640 1 -- 172.21.15.5:0/1990678553 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13e803f070 con 0x7f1400074eb0 2024-09-10T09:34:41.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:41.281+0000 7f13f7400640 1 -- 172.21.15.5:0/1990678553 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f13e8031c90 con 0x7f1400074eb0 2024-09-10T09:34:41.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:41.281+0000 7f1404c00640 1 -- 172.21.15.5:0/1990678553 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14001a3e00 con 0x7f1400074eb0 2024-09-10T09:34:41.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:41.281+0000 7f13f7400640 1 -- 172.21.15.5:0/1990678553 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13e8035030 con 0x7f1400074eb0 2024-09-10T09:34:41.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:41.281+0000 7f1404c00640 1 -- 172.21.15.5:0/1990678553 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14001ab6f0 con 0x7f1400074eb0 2024-09-10T09:34:41.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:41.282+0000 7f1404c00640 1 -- 172.21.15.5:0/1990678553 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1400074970 con 0x7f1400074eb0 2024-09-10T09:34:41.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:41.284+0000 7f13f7400640 1 -- 172.21.15.5:0/1990678553 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7f13e8042070 con 0x7f1400074eb0 2024-09-10T09:34:41.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:41.286+0000 7f13f7400640 1 --2- 172.21.15.5:0/1990678553 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f13e40a2c00 0x7f13e40a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:34:41.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:41.286+0000 7f13f7400640 1 -- 172.21.15.5:0/1990678553 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7127+0+0 (secure 0 0 0) 0x7f13e80efae0 con 0x7f1400074eb0 2024-09-10T09:34:41.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:41.286+0000 7f13fde00640 1 --2- 172.21.15.5:0/1990678553 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f13e40a2c00 0x7f13e40a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:34:41.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:41.286+0000 7f13fde00640 1 --2- 172.21.15.5:0/1990678553 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f13e40a2c00 0x7f13e40a50c0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f13f0009c90 tx=0x7f13f0005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:34:41.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:41.288+0000 7f13f7400640 1 -- 172.21.15.5:0/1990678553 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f13e80bc7f0 con 0x7f1400074eb0 2024-09-10T09:34:41.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:41.511+0000 7f1404c00640 1 -- 172.21.15.5:0/1990678553 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f140019fea0 con 0x7f1400074eb0 2024-09-10T09:34:41.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:41.512+0000 7f13f7400640 1 -- 172.21.15.5:0/1990678553 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) ==== 93+0+611 (secure 0 0 0) 0x7f13e80c0af0 con 0x7f1400074eb0 2024-09-10T09:34:41.513 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-10T09:34:41.513 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":1,"btime":"2024-09-10T09:27:13:298289+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2024-09-10T09:34:41.513 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 1 2024-09-10T09:34:41.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:41.514+0000 7f1404c00640 1 -- 172.21.15.5:0/1990678553 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f13e40a2c00 msgr2=0x7f13e40a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:34:41.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:41.514+0000 7f1404c00640 1 --2- 172.21.15.5:0/1990678553 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f13e40a2c00 0x7f13e40a50c0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f13f0009c90 tx=0x7f13f0005f90 comp rx=0 tx=0).stop 2024-09-10T09:34:41.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:41.514+0000 7f1404c00640 1 -- 172.21.15.5:0/1990678553 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1400074eb0 msgr2=0x7f140019f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:34:41.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:41.514+0000 7f1404c00640 1 --2- 172.21.15.5:0/1990678553 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1400074eb0 0x7f140019f080 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f13e802a9a0 tx=0x7f13e8003b20 comp rx=0 tx=0).stop 2024-09-10T09:34:41.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:41.514+0000 7f1404c00640 1 -- 172.21.15.5:0/1990678553 shutdown_connections 2024-09-10T09:34:41.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:41.514+0000 7f1404c00640 1 --2- 172.21.15.5:0/1990678553 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f13e40a2c00 0x7f13e40a50c0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:34:41.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:41.514+0000 7f1404c00640 1 --2- 172.21.15.5:0/1990678553 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1400112b90 0x7f14001a3440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:34:41.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:41.514+0000 7f1404c00640 1 --2- 172.21.15.5:0/1990678553 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1400074eb0 0x7f140019f080 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:34:41.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:41.514+0000 7f1404c00640 1 --2- 172.21.15.5:0/1990678553 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1400074590 0x7f140019eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:34:41.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:41.515+0000 7f1404c00640 1 -- 172.21.15.5:0/1990678553 >> 172.21.15.5:0/1990678553 conn(0x7f14000fd8c0 msgr2=0x7f14000ff050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:34:41.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:41.515+0000 7f1404c00640 1 -- 172.21.15.5:0/1990678553 shutdown_connections 2024-09-10T09:34:41.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:41.515+0000 7f1404c00640 1 -- 172.21.15.5:0/1990678553 wait complete. 2024-09-10T09:34:41.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:34:41 smithi005 ceph-mon[30350]: from='client.? 172.21.15.5:0/1990678553' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:34:41.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:34:41 smithi005 ceph-mon[30350]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T09:34:41.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:34:41 smithi159 ceph-mon[35513]: from='client.? 172.21.15.5:0/1990678553' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:34:41.851 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:34:41 smithi159 ceph-mon[35513]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T09:34:41.851 INFO:journalctl@ceph.mds.l.smithi159.stdout:Sep 10 09:34:41 smithi159 systemd[1]: Starting Ceph mds.l for bade4b50-6f56-11ef-bcea-c7b262605968... 2024-09-10T09:34:41.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:34:41 smithi102 ceph-mon[35442]: from='client.? 172.21.15.5:0/1990678553' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:34:41.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:34:41 smithi102 ceph-mon[35442]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T09:34:42.179 INFO:journalctl@ceph.mds.l.smithi159.stdout:Sep 10 09:34:41 smithi159 podman[64655]: 2024-09-10 09:34:41.822348163 +0000 UTC m=+0.015727016 image pull 41d67112835771c3efca60c6b8fe297c7ddaecdc26cea26fd28aa18636e49005 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 2024-09-10T09:34:42.179 INFO:journalctl@ceph.mds.l.smithi159.stdout:Sep 10 09:34:41 smithi159 podman[64655]: 2024-09-10 09:34:41.934543359 +0000 UTC m=+0.127922249 container create 8b40c3762f535df0c1c66505a9fe26fede5d1ab0eaaa2235af7675efa86092c1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mds-l, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-09-10T09:34:42.179 INFO:journalctl@ceph.mds.l.smithi159.stdout:Sep 10 09:34:42 smithi159 podman[64655]: 2024-09-10 09:34:42.178777138 +0000 UTC m=+0.372155985 container init 8b40c3762f535df0c1c66505a9fe26fede5d1ab0eaaa2235af7675efa86092c1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mds-l, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, ceph=True) 2024-09-10T09:34:42.600 INFO:journalctl@ceph.mds.l.smithi159.stdout:Sep 10 09:34:42 smithi159 podman[64655]: 2024-09-10 09:34:42.181736294 +0000 UTC m=+0.375115142 container start 8b40c3762f535df0c1c66505a9fe26fede5d1ab0eaaa2235af7675efa86092c1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mds-l, org.label-schema.build-date=20240731, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0) 2024-09-10T09:34:42.600 INFO:journalctl@ceph.mds.l.smithi159.stdout:Sep 10 09:34:42 smithi159 ceph-mds[64674]: -- 172.21.15.159:0/3837474231 <== mon.1 v2:172.21.15.159:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x559963c71c20 con 0x559963c4f000 2024-09-10T09:34:42.600 INFO:journalctl@ceph.mds.l.smithi159.stdout:Sep 10 09:34:42 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mds-l[64670]: starting mds.l at 2024-09-10T09:34:42.600 INFO:journalctl@ceph.mds.l.smithi159.stdout:Sep 10 09:34:42 smithi159 bash[64655]: 8b40c3762f535df0c1c66505a9fe26fede5d1ab0eaaa2235af7675efa86092c1 2024-09-10T09:34:42.601 INFO:journalctl@ceph.mds.l.smithi159.stdout:Sep 10 09:34:42 smithi159 systemd[1]: Started Ceph mds.l for bade4b50-6f56-11ef-bcea-c7b262605968. 2024-09-10T09:34:43.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:34:43 smithi102 ceph-mon[35442]: mds.? [v2:172.21.15.159:6832/356445159,v1:172.21.15.159:6833/356445159] up:boot 2024-09-10T09:34:43.385 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:34:43 smithi102 ceph-mon[35442]: fsmap 1 up:standby 2024-09-10T09:34:43.385 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:34:43 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-10T09:34:43.385 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:34:43 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:34:43.385 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:34:43 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:34:43.385 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:34:43 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:34:43.385 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:34:43 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T09:34:43.385 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:34:43 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T09:34:43.385 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:34:43 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:34:43.385 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:34:43 smithi102 ceph-mon[35442]: Deploying daemon mds.k on smithi102 2024-09-10T09:34:43.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:34:43 smithi005 ceph-mon[30350]: mds.? [v2:172.21.15.159:6832/356445159,v1:172.21.15.159:6833/356445159] up:boot 2024-09-10T09:34:43.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:34:43 smithi005 ceph-mon[30350]: fsmap 1 up:standby 2024-09-10T09:34:43.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:34:43 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-10T09:34:43.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:34:43 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:34:43.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:34:43 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:34:43.557 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:34:43 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:34:43.557 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:34:43 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T09:34:43.557 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:34:43 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T09:34:43.557 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:34:43 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:34:43.557 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:34:43 smithi005 ceph-mon[30350]: Deploying daemon mds.k on smithi102 2024-09-10T09:34:43.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:34:43 smithi159 ceph-mon[35513]: mds.? [v2:172.21.15.159:6832/356445159,v1:172.21.15.159:6833/356445159] up:boot 2024-09-10T09:34:43.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:34:43 smithi159 ceph-mon[35513]: fsmap 1 up:standby 2024-09-10T09:34:43.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:34:43 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-10T09:34:43.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:34:43 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:34:43.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:34:43 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:34:43.601 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:34:43 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:34:43.601 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:34:43 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T09:34:43.601 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:34:43 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T09:34:43.601 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:34:43 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:34:43.601 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:34:43 smithi159 ceph-mon[35513]: Deploying daemon mds.k on smithi102 2024-09-10T09:34:44.307 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T09:34:44.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:34:44 smithi005 ceph-mon[30350]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T09:34:44.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:34:44 smithi159 ceph-mon[35513]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T09:34:44.631 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:34:44 smithi102 ceph-mon[35442]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T09:34:44.631 INFO:journalctl@ceph.mds.k.smithi102.stdout:Sep 10 09:34:44 smithi102 systemd[1]: Starting Ceph mds.k for bade4b50-6f56-11ef-bcea-c7b262605968... 2024-09-10T09:34:44.689 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:34:44.884 INFO:journalctl@ceph.mds.k.smithi102.stdout:Sep 10 09:34:44 smithi102 podman[64264]: 2024-09-10 09:34:44.532439298 +0000 UTC m=+0.015801350 image pull 41d67112835771c3efca60c6b8fe297c7ddaecdc26cea26fd28aa18636e49005 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 2024-09-10T09:34:44.885 INFO:journalctl@ceph.mds.k.smithi102.stdout:Sep 10 09:34:44 smithi102 podman[64264]: 2024-09-10 09:34:44.663856653 +0000 UTC m=+0.147218692 container create a2e3425919b256e9588efcab9ebf4425d6025cc88b7afac9602aafb62003d167 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mds-k, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-09-10T09:34:45.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:45.247+0000 7fb00b200640 1 -- 172.21.15.5:0/23427348 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb00c102360 msgr2=0x7fb00c10e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:34:45.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:45.247+0000 7fb00b200640 1 --2- 172.21.15.5:0/23427348 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb00c102360 0x7fb00c10e890 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fb004009be0 tx=0x7fb00402a6c0 comp rx=0 tx=0).stop 2024-09-10T09:34:45.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:45.247+0000 7fb00b200640 1 -- 172.21.15.5:0/23427348 shutdown_connections 2024-09-10T09:34:45.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:45.247+0000 7fb00b200640 1 --2- 172.21.15.5:0/23427348 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb00c102360 0x7fb00c10e890 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:34:45.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:45.247+0000 7fb00b200640 1 --2- 172.21.15.5:0/23427348 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb00c1019c0 0x7fb00c101e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:34:45.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:45.247+0000 7fb00b200640 1 --2- 172.21.15.5:0/23427348 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb00c1079c0 0x7fb00c107da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:34:45.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:45.247+0000 7fb00b200640 1 -- 172.21.15.5:0/23427348 >> 172.21.15.5:0/23427348 conn(0x7fb00c0fd740 msgr2=0x7fb00c0ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:34:45.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:45.247+0000 7fb00b200640 1 -- 172.21.15.5:0/23427348 shutdown_connections 2024-09-10T09:34:45.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:45.247+0000 7fb00b200640 1 -- 172.21.15.5:0/23427348 wait complete. 2024-09-10T09:34:45.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:45.248+0000 7fb00b200640 1 Processor -- start 2024-09-10T09:34:45.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:45.248+0000 7fb00b200640 1 -- start start 2024-09-10T09:34:45.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:45.248+0000 7fb00b200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb00c1019c0 0x7fb00c19a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:34:45.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:45.248+0000 7fb00b200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb00c102360 0x7fb00c19a980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:34:45.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:45.248+0000 7fb00b200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb00c1079c0 0x7fb00c19ed10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:34:45.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:45.249+0000 7fb00b200640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fb00c111dc0 con 0x7fb00c1079c0 2024-09-10T09:34:45.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:45.249+0000 7fb00b200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fb00c111c40 con 0x7fb00c102360 2024-09-10T09:34:45.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:45.249+0000 7fb00b200640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fb00c111f40 con 0x7fb00c1019c0 2024-09-10T09:34:45.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:45.249+0000 7fb009400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb00c102360 0x7fb00c19a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:34:45.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:45.249+0000 7fb009e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb00c1019c0 0x7fb00c19a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:34:45.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:45.249+0000 7fb009400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb00c102360 0x7fb00c19a980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:38314/0 (socket says 172.21.15.5:38314) 2024-09-10T09:34:45.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:45.249+0000 7fb00a800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb00c1079c0 0x7fb00c19ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:34:45.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:45.249+0000 7fb009400640 1 -- 172.21.15.5:0/2695608789 learned_addr learned my addr 172.21.15.5:0/2695608789 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:34:45.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:45.249+0000 7fb009400640 1 -- 172.21.15.5:0/2695608789 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb00c1019c0 msgr2=0x7fb00c19a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:34:45.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:45.249+0000 7fb009400640 1 --2- 172.21.15.5:0/2695608789 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb00c1019c0 0x7fb00c19a440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:34:45.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:45.249+0000 7fb009400640 1 -- 172.21.15.5:0/2695608789 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb00c1079c0 msgr2=0x7fb00c19ed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:34:45.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:45.249+0000 7fb009400640 1 --2- 172.21.15.5:0/2695608789 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb00c1079c0 0x7fb00c19ed10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:34:45.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:45.249+0000 7fb009400640 1 -- 172.21.15.5:0/2695608789 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb00c19f3f0 con 0x7fb00c102360 2024-09-10T09:34:45.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:45.250+0000 7fb009400640 1 --2- 172.21.15.5:0/2695608789 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb00c102360 0x7fb00c19a980 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7faff800e9f0 tx=0x7faff800edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:34:45.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:45.250+0000 7fb002a00640 1 -- 172.21.15.5:0/2695608789 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faff800ccc0 con 0x7fb00c102360 2024-09-10T09:34:45.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:45.250+0000 7fb002a00640 1 -- 172.21.15.5:0/2695608789 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faff8010910 con 0x7fb00c102360 2024-09-10T09:34:45.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:45.250+0000 7fb002a00640 1 -- 172.21.15.5:0/2695608789 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faff80188a0 con 0x7fb00c102360 2024-09-10T09:34:45.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:45.250+0000 7fb00b200640 1 -- 172.21.15.5:0/2695608789 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb00c19f6e0 con 0x7fb00c102360 2024-09-10T09:34:45.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:45.250+0000 7fb00b200640 1 -- 172.21.15.5:0/2695608789 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb00c1a6fc0 con 0x7fb00c102360 2024-09-10T09:34:45.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:45.251+0000 7fb00b200640 1 -- 172.21.15.5:0/2695608789 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb00c101e20 con 0x7fb00c102360 2024-09-10T09:34:45.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:45.253+0000 7fb002a00640 1 -- 172.21.15.5:0/2695608789 <== mon.2 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7faff8010ac0 con 0x7fb00c102360 2024-09-10T09:34:45.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:45.255+0000 7fb002a00640 1 --2- 172.21.15.5:0/2695608789 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7faff00a2c00 0x7faff00a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:34:45.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:45.255+0000 7fb002a00640 1 -- 172.21.15.5:0/2695608789 <== mon.2 v2:172.21.15.102:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7127+0+0 (secure 0 0 0) 0x7faff80ceff0 con 0x7fb00c102360 2024-09-10T09:34:45.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:45.255+0000 7fb009e00640 1 --2- 172.21.15.5:0/2695608789 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7faff00a2c00 0x7faff00a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:34:45.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:45.256+0000 7fb009e00640 1 --2- 172.21.15.5:0/2695608789 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7faff00a2c00 0x7faff00a50c0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7faffc005fd0 tx=0x7faffc009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:34:45.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:45.259+0000 7fb002a00640 1 -- 172.21.15.5:0/2695608789 <== mon.2 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7faff809c350 con 0x7fb00c102360 2024-09-10T09:34:45.384 INFO:journalctl@ceph.mds.k.smithi102.stdout:Sep 10 09:34:44 smithi102 podman[64264]: 2024-09-10 09:34:44.925409303 +0000 UTC m=+0.408771343 container init a2e3425919b256e9588efcab9ebf4425d6025cc88b7afac9602aafb62003d167 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mds-k, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T09:34:45.384 INFO:journalctl@ceph.mds.k.smithi102.stdout:Sep 10 09:34:44 smithi102 podman[64264]: 2024-09-10 09:34:44.92822424 +0000 UTC m=+0.411586283 container start a2e3425919b256e9588efcab9ebf4425d6025cc88b7afac9602aafb62003d167 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mds-k, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-09-10T09:34:45.385 INFO:journalctl@ceph.mds.k.smithi102.stdout:Sep 10 09:34:44 smithi102 ceph-mds[64283]: -- 172.21.15.102:0/4200244234 <== mon.2 v2:172.21.15.102:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x561fd960f860 con 0x561fd95ec800 2024-09-10T09:34:45.385 INFO:journalctl@ceph.mds.k.smithi102.stdout:Sep 10 09:34:44 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mds-k[64279]: starting mds.k at 2024-09-10T09:34:45.385 INFO:journalctl@ceph.mds.k.smithi102.stdout:Sep 10 09:34:45 smithi102 bash[64264]: a2e3425919b256e9588efcab9ebf4425d6025cc88b7afac9602aafb62003d167 2024-09-10T09:34:45.385 INFO:journalctl@ceph.mds.k.smithi102.stdout:Sep 10 09:34:45 smithi102 systemd[1]: Started Ceph mds.k for bade4b50-6f56-11ef-bcea-c7b262605968. 2024-09-10T09:34:45.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:45.483+0000 7fb00b200640 1 -- 172.21.15.5:0/2695608789 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb00c1a7190 con 0x7fb00c102360 2024-09-10T09:34:45.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:45.484+0000 7fb002a00640 1 -- 172.21.15.5:0/2695608789 <== mon.2 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+2263 (secure 0 0 0) 0x7faff809bc50 con 0x7fb00c102360 2024-09-10T09:34:45.485 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-10T09:34:45.485 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":3,"btime":"2024-09-10T09:34:45:218718+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6833/2457007427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":2457007427},{"type":"v1","addr":"172.21.15.102:6833","nonce":2457007427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24460,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6833/356445159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":356445159},{"type":"v1","addr":"172.21.15.159:6833","nonce":356445159}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-09-10T09:34:45.486 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 3 2024-09-10T09:34:45.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:45.487+0000 7fb00b200640 1 -- 172.21.15.5:0/2695608789 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7faff00a2c00 msgr2=0x7faff00a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:34:45.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:45.487+0000 7fb00b200640 1 --2- 172.21.15.5:0/2695608789 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7faff00a2c00 0x7faff00a50c0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7faffc005fd0 tx=0x7faffc009260 comp rx=0 tx=0).stop 2024-09-10T09:34:45.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:45.487+0000 7fb00b200640 1 -- 172.21.15.5:0/2695608789 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb00c102360 msgr2=0x7fb00c19a980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:34:45.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:45.487+0000 7fb00b200640 1 --2- 172.21.15.5:0/2695608789 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb00c102360 0x7fb00c19a980 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7faff800e9f0 tx=0x7faff800edf0 comp rx=0 tx=0).stop 2024-09-10T09:34:45.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:45.487+0000 7fb00b200640 1 -- 172.21.15.5:0/2695608789 shutdown_connections 2024-09-10T09:34:45.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:45.487+0000 7fb00b200640 1 --2- 172.21.15.5:0/2695608789 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7faff00a2c00 0x7faff00a50c0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:34:45.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:45.487+0000 7fb00b200640 1 --2- 172.21.15.5:0/2695608789 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb00c1079c0 0x7fb00c19ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:34:45.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:45.487+0000 7fb00b200640 1 --2- 172.21.15.5:0/2695608789 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb00c102360 0x7fb00c19a980 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:34:45.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:45.487+0000 7fb00b200640 1 --2- 172.21.15.5:0/2695608789 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb00c1019c0 0x7fb00c19a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:34:45.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:45.487+0000 7fb00b200640 1 -- 172.21.15.5:0/2695608789 >> 172.21.15.5:0/2695608789 conn(0x7fb00c0fd740 msgr2=0x7fb00c10cc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:34:45.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:45.487+0000 7fb00b200640 1 -- 172.21.15.5:0/2695608789 shutdown_connections 2024-09-10T09:34:45.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:45.487+0000 7fb00b200640 1 -- 172.21.15.5:0/2695608789 wait complete. 2024-09-10T09:34:46.219 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:34:46 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:34:46.219 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:34:46 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:34:46.219 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:34:46 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:34:46.219 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:34:46 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T09:34:46.219 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:34:46 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T09:34:46.219 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:34:46 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:34:46.219 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:34:46 smithi005 ceph-mon[30350]: Deploying daemon mds.j on smithi005 2024-09-10T09:34:46.220 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:34:46 smithi005 ceph-mon[30350]: mds.? [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] up:boot 2024-09-10T09:34:46.220 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:34:46 smithi005 ceph-mon[30350]: fsmap 2 up:standby 2024-09-10T09:34:46.220 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:34:46 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-10T09:34:46.220 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:34:46 smithi005 ceph-mon[30350]: from='client.? 172.21.15.5:0/2695608789' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:34:46.220 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:34:46 smithi005 ceph-mon[30350]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T09:34:46.345 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:34:46 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:34:46.345 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:34:46 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:34:46.345 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:34:46 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:34:46.345 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:34:46 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T09:34:46.345 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:34:46 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T09:34:46.346 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:34:46 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:34:46.346 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:34:46 smithi102 ceph-mon[35442]: Deploying daemon mds.j on smithi005 2024-09-10T09:34:46.346 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:34:46 smithi102 ceph-mon[35442]: mds.? [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] up:boot 2024-09-10T09:34:46.346 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:34:46 smithi102 ceph-mon[35442]: fsmap 2 up:standby 2024-09-10T09:34:46.346 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:34:46 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-10T09:34:46.346 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:34:46 smithi102 ceph-mon[35442]: from='client.? 172.21.15.5:0/2695608789' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:34:46.346 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:34:46 smithi102 ceph-mon[35442]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T09:34:46.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:34:46 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:34:46.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:34:46 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:34:46.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:34:46 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:34:46.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:34:46 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T09:34:46.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:34:46 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T09:34:46.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:34:46 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:34:46.351 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:34:46 smithi159 ceph-mon[35513]: Deploying daemon mds.j on smithi005 2024-09-10T09:34:46.351 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:34:46 smithi159 ceph-mon[35513]: mds.? [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] up:boot 2024-09-10T09:34:46.351 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:34:46 smithi159 ceph-mon[35513]: fsmap 2 up:standby 2024-09-10T09:34:46.351 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:34:46 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-10T09:34:46.351 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:34:46 smithi159 ceph-mon[35513]: from='client.? 172.21.15.5:0/2695608789' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:34:46.351 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:34:46 smithi159 ceph-mon[35513]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T09:34:47.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:34:47 smithi005 ceph-mon[30350]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T09:34:47.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:34:47 smithi159 ceph-mon[35513]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T09:34:47.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:34:47 smithi102 ceph-mon[35442]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T09:34:48.259 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T09:34:48.806 INFO:journalctl@ceph.mds.j.smithi005.stdout:Sep 10 09:34:48 smithi005 podman[70591]: 2024-09-10 09:34:48.547986536 +0000 UTC m=+0.163226342 container create b66502621e8fee3f628304fa814bfd3b260c93dee478196b082cced14005f7ec (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mds-j, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-09-10T09:34:49.008 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:34:49.153 INFO:journalctl@ceph.mds.j.smithi005.stdout:Sep 10 09:34:48 smithi005 podman[70591]: 2024-09-10 09:34:48.83475232 +0000 UTC m=+0.449992139 container init b66502621e8fee3f628304fa814bfd3b260c93dee478196b082cced14005f7ec (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mds-j, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux ) 2024-09-10T09:34:49.153 INFO:journalctl@ceph.mds.j.smithi005.stdout:Sep 10 09:34:48 smithi005 podman[70591]: 2024-09-10 09:34:48.841507779 +0000 UTC m=+0.456747583 container start b66502621e8fee3f628304fa814bfd3b260c93dee478196b082cced14005f7ec (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mds-j, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731) 2024-09-10T09:34:49.153 INFO:journalctl@ceph.mds.j.smithi005.stdout:Sep 10 09:34:48 smithi005 ceph-mds[70635]: -- 172.21.15.5:0/134901726 <== mon.0 v2:172.21.15.5:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55766d3a1860 con 0x55766d37f000 2024-09-10T09:34:49.153 INFO:journalctl@ceph.mds.j.smithi005.stdout:Sep 10 09:34:48 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mds-j[70631]: starting mds.j at 2024-09-10T09:34:49.153 INFO:journalctl@ceph.mds.j.smithi005.stdout:Sep 10 09:34:48 smithi005 bash[70591]: b66502621e8fee3f628304fa814bfd3b260c93dee478196b082cced14005f7ec 2024-09-10T09:34:49.153 INFO:journalctl@ceph.mds.j.smithi005.stdout:Sep 10 09:34:48 smithi005 systemd[1]: Started Ceph mds.j for bade4b50-6f56-11ef-bcea-c7b262605968. 2024-09-10T09:34:49.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:49.599+0000 7f990b200640 1 -- 172.21.15.5:0/2867216771 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f990c112870 msgr2=0x7f990c114c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:34:49.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:49.599+0000 7f990b200640 1 --2- 172.21.15.5:0/2867216771 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f990c112870 0x7f990c114c60 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f9904009ca0 tx=0x7f990402a8c0 comp rx=0 tx=0).stop 2024-09-10T09:34:49.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:49.599+0000 7f990b200640 1 -- 172.21.15.5:0/2867216771 shutdown_connections 2024-09-10T09:34:49.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:49.599+0000 7f990b200640 1 --2- 172.21.15.5:0/2867216771 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f990c112870 0x7f990c114c60 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:34:49.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:49.599+0000 7f990b200640 1 --2- 172.21.15.5:0/2867216771 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f990c077040 0x7f990c112330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:34:49.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:49.599+0000 7f990b200640 1 --2- 172.21.15.5:0/2867216771 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f990c076720 0x7f990c076b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:34:49.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:49.599+0000 7f990b200640 1 -- 172.21.15.5:0/2867216771 >> 172.21.15.5:0/2867216771 conn(0x7f990c0ff8d0 msgr2=0x7f990c101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:34:49.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:49.599+0000 7f990b200640 1 -- 172.21.15.5:0/2867216771 shutdown_connections 2024-09-10T09:34:49.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:49.599+0000 7f990b200640 1 -- 172.21.15.5:0/2867216771 wait complete. 2024-09-10T09:34:49.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:49.600+0000 7f990b200640 1 Processor -- start 2024-09-10T09:34:49.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:49.600+0000 7f990b200640 1 -- start start 2024-09-10T09:34:49.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:49.601+0000 7f990b200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f990c076720 0x7f990c1a45c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:34:49.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:49.601+0000 7f990b200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f990c077040 0x7f990c1a4b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:34:49.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:49.601+0000 7f990b200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f990c112870 0x7f990c19e690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:34:49.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:49.601+0000 7f990b200640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f990c1183b0 con 0x7f990c112870 2024-09-10T09:34:49.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:49.601+0000 7f990b200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f990c118230 con 0x7f990c076720 2024-09-10T09:34:49.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:49.601+0000 7f990b200640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f990c118530 con 0x7f990c077040 2024-09-10T09:34:49.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:49.601+0000 7f9909e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f990c076720 0x7f990c1a45c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:34:49.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:49.601+0000 7f9909e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f990c076720 0x7f990c1a45c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:38340/0 (socket says 172.21.15.5:38340) 2024-09-10T09:34:49.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:49.601+0000 7f9909e00640 1 -- 172.21.15.5:0/1691447964 learned_addr learned my addr 172.21.15.5:0/1691447964 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:34:49.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:49.601+0000 7f9909400640 1 --2- 172.21.15.5:0/1691447964 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f990c077040 0x7f990c1a4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:34:49.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:49.601+0000 7f990a800640 1 --2- 172.21.15.5:0/1691447964 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f990c112870 0x7f990c19e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:34:49.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:49.602+0000 7f990a800640 1 -- 172.21.15.5:0/1691447964 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f990c077040 msgr2=0x7f990c1a4b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:34:49.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:49.602+0000 7f990a800640 1 --2- 172.21.15.5:0/1691447964 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f990c077040 0x7f990c1a4b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:34:49.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:49.602+0000 7f990a800640 1 -- 172.21.15.5:0/1691447964 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f990c076720 msgr2=0x7f990c1a45c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:34:49.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:49.602+0000 7f990a800640 1 --2- 172.21.15.5:0/1691447964 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f990c076720 0x7f990c1a45c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:34:49.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:49.602+0000 7f990a800640 1 -- 172.21.15.5:0/1691447964 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f990c19ef00 con 0x7f990c112870 2024-09-10T09:34:49.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:49.602+0000 7f9909e00640 1 --2- 172.21.15.5:0/1691447964 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f990c076720 0x7f990c1a45c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:34:49.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:49.602+0000 7f990a800640 1 --2- 172.21.15.5:0/1691447964 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f990c112870 0x7f990c19e690 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f990402ade0 tx=0x7f99040319f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:34:49.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:49.602+0000 7f9902a00640 1 -- 172.21.15.5:0/1691447964 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9904003b70 con 0x7f990c112870 2024-09-10T09:34:49.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:49.602+0000 7f9902a00640 1 -- 172.21.15.5:0/1691447964 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9904003d10 con 0x7f990c112870 2024-09-10T09:34:49.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:49.602+0000 7f9902a00640 1 -- 172.21.15.5:0/1691447964 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f990402fc10 con 0x7f990c112870 2024-09-10T09:34:49.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:49.602+0000 7f990b200640 1 -- 172.21.15.5:0/1691447964 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f990c19f190 con 0x7f990c112870 2024-09-10T09:34:49.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:49.603+0000 7f990b200640 1 -- 172.21.15.5:0/1691447964 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f990c1ab370 con 0x7f990c112870 2024-09-10T09:34:49.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:49.604+0000 7f990b200640 1 -- 172.21.15.5:0/1691447964 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f990c077e60 con 0x7f990c112870 2024-09-10T09:34:49.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:49.606+0000 7f9902a00640 1 -- 172.21.15.5:0/1691447964 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7f9904004050 con 0x7f990c112870 2024-09-10T09:34:49.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:49.607+0000 7f9902a00640 1 --2- 172.21.15.5:0/1691447964 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f98f00a2b80 0x7f98f00a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:34:49.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:49.607+0000 7f9902a00640 1 -- 172.21.15.5:0/1691447964 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7127+0+0 (secure 0 0 0) 0x7f99040efce0 con 0x7f990c112870 2024-09-10T09:34:49.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:49.608+0000 7f9909e00640 1 --2- 172.21.15.5:0/1691447964 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f98f00a2b80 0x7f98f00a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:34:49.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:49.608+0000 7f9909e00640 1 --2- 172.21.15.5:0/1691447964 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f98f00a2b80 0x7f98f00a5040 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f98fc011550 tx=0x7f98fc005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:34:49.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:49.611+0000 7f9902a00640 1 -- 172.21.15.5:0/1691447964 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f99040bd160 con 0x7f990c112870 2024-09-10T09:34:49.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:49.851+0000 7f990b200640 1 -- 172.21.15.5:0/1691447964 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f990c10da00 con 0x7f990c112870 2024-09-10T09:34:49.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:49.852+0000 7f9902a00640 1 -- 172.21.15.5:0/1691447964 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) ==== 93+0+3085 (secure 0 0 0) 0x7f99040bca60 con 0x7f990c112870 2024-09-10T09:34:49.853 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-10T09:34:49.853 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":4,"btime":"2024-09-10T09:34:48:937702+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14454,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/2970594311","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2970594311},{"type":"v1","addr":"172.21.15.5:6835","nonce":2970594311}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6833/2457007427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":2457007427},{"type":"v1","addr":"172.21.15.102:6833","nonce":2457007427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24460,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6833/356445159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":356445159},{"type":"v1","addr":"172.21.15.159:6833","nonce":356445159}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-09-10T09:34:49.853 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 4 2024-09-10T09:34:49.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:49.854+0000 7f990b200640 1 -- 172.21.15.5:0/1691447964 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f98f00a2b80 msgr2=0x7f98f00a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:34:49.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:49.854+0000 7f990b200640 1 --2- 172.21.15.5:0/1691447964 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f98f00a2b80 0x7f98f00a5040 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f98fc011550 tx=0x7f98fc005f90 comp rx=0 tx=0).stop 2024-09-10T09:34:49.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:49.854+0000 7f990b200640 1 -- 172.21.15.5:0/1691447964 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f990c112870 msgr2=0x7f990c19e690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:34:49.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:49.854+0000 7f990b200640 1 --2- 172.21.15.5:0/1691447964 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f990c112870 0x7f990c19e690 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f990402ade0 tx=0x7f99040319f0 comp rx=0 tx=0).stop 2024-09-10T09:34:49.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:49.855+0000 7f990b200640 1 -- 172.21.15.5:0/1691447964 shutdown_connections 2024-09-10T09:34:49.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:49.855+0000 7f990b200640 1 --2- 172.21.15.5:0/1691447964 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f98f00a2b80 0x7f98f00a5040 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:34:49.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:49.855+0000 7f990b200640 1 --2- 172.21.15.5:0/1691447964 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f990c112870 0x7f990c19e690 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:34:49.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:49.855+0000 7f990b200640 1 --2- 172.21.15.5:0/1691447964 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f990c077040 0x7f990c1a4b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:34:49.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:49.855+0000 7f990b200640 1 --2- 172.21.15.5:0/1691447964 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f990c076720 0x7f990c1a45c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:34:49.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:49.855+0000 7f990b200640 1 -- 172.21.15.5:0/1691447964 >> 172.21.15.5:0/1691447964 conn(0x7f990c0ff8d0 msgr2=0x7f990c101cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:34:49.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:49.855+0000 7f990b200640 1 -- 172.21.15.5:0/1691447964 shutdown_connections 2024-09-10T09:34:49.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:49.855+0000 7f990b200640 1 -- 172.21.15.5:0/1691447964 wait complete. 2024-09-10T09:34:50.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:34:49 smithi005 ceph-mon[30350]: mds.? [v2:172.21.15.5:6834/2970594311,v1:172.21.15.5:6835/2970594311] up:boot 2024-09-10T09:34:50.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:34:49 smithi005 ceph-mon[30350]: fsmap 3 up:standby 2024-09-10T09:34:50.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:34:49 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-10T09:34:50.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:34:49 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:34:50.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:34:49 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:34:50.057 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:34:49 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:34:50.057 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:34:49 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T09:34:50.057 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:34:49 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T09:34:50.057 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:34:49 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:34:50.057 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:34:49 smithi005 ceph-mon[30350]: Deploying daemon mds.d on smithi005 2024-09-10T09:34:50.057 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:34:49 smithi005 ceph-mon[30350]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T09:34:50.057 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:34:49 smithi005 ceph-mon[30350]: from='client.? 172.21.15.5:0/1691447964' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:34:50.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:34:49 smithi159 ceph-mon[35513]: mds.? [v2:172.21.15.5:6834/2970594311,v1:172.21.15.5:6835/2970594311] up:boot 2024-09-10T09:34:50.351 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:34:49 smithi159 ceph-mon[35513]: fsmap 3 up:standby 2024-09-10T09:34:50.351 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:34:49 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-10T09:34:50.351 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:34:49 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:34:50.351 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:34:49 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:34:50.351 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:34:49 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:34:50.351 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:34:49 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T09:34:50.351 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:34:49 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T09:34:50.351 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:34:49 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:34:50.351 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:34:49 smithi159 ceph-mon[35513]: Deploying daemon mds.d on smithi005 2024-09-10T09:34:50.351 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:34:49 smithi159 ceph-mon[35513]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T09:34:50.351 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:34:49 smithi159 ceph-mon[35513]: from='client.? 172.21.15.5:0/1691447964' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:34:50.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:34:49 smithi102 ceph-mon[35442]: mds.? [v2:172.21.15.5:6834/2970594311,v1:172.21.15.5:6835/2970594311] up:boot 2024-09-10T09:34:50.385 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:34:49 smithi102 ceph-mon[35442]: fsmap 3 up:standby 2024-09-10T09:34:50.385 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:34:49 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-10T09:34:50.385 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:34:49 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:34:50.385 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:34:49 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:34:50.385 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:34:49 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:34:50.385 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:34:49 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T09:34:50.385 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:34:49 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T09:34:50.385 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:34:49 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:34:50.385 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:34:49 smithi102 ceph-mon[35442]: Deploying daemon mds.d on smithi005 2024-09-10T09:34:50.385 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:34:49 smithi102 ceph-mon[35442]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T09:34:50.385 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:34:49 smithi102 ceph-mon[35442]: from='client.? 172.21.15.5:0/1691447964' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:34:51.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:34:51 smithi159 ceph-mon[35513]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T09:34:51.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:34:51 smithi102 ceph-mon[35442]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T09:34:52.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:34:51 smithi005 ceph-mon[30350]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T09:34:53.019 INFO:journalctl@ceph.mds.d.smithi005.stdout:Sep 10 09:34:52 smithi005 podman[70990]: 2024-09-10 09:34:52.764939472 +0000 UTC m=+0.173009347 container create cbc4e0ec9a31bd14cfe177b45d10e437d01edf3925620126f547534fa1c141b8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mds-d, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, CEPH_POINT_RELEASE=) 2024-09-10T09:34:53.019 INFO:journalctl@ceph.mds.d.smithi005.stdout:Sep 10 09:34:53 smithi005 podman[70990]: 2024-09-10 09:34:53.018070979 +0000 UTC m=+0.426140846 container init cbc4e0ec9a31bd14cfe177b45d10e437d01edf3925620126f547534fa1c141b8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mds-d, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-09-10T09:34:53.176 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T09:34:53.306 INFO:journalctl@ceph.mds.d.smithi005.stdout:Sep 10 09:34:53 smithi005 podman[70990]: 2024-09-10 09:34:53.026643438 +0000 UTC m=+0.434713309 container start cbc4e0ec9a31bd14cfe177b45d10e437d01edf3925620126f547534fa1c141b8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mds-d, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-10T09:34:53.306 INFO:journalctl@ceph.mds.d.smithi005.stdout:Sep 10 09:34:53 smithi005 ceph-mds[71013]: -- 172.21.15.5:0/2584192823 <== mon.1 v2:172.21.15.159:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55edaab8b860 con 0x55edaab68800 2024-09-10T09:34:53.306 INFO:journalctl@ceph.mds.d.smithi005.stdout:Sep 10 09:34:53 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mds-d[71009]: starting mds.d at 2024-09-10T09:34:53.307 INFO:journalctl@ceph.mds.d.smithi005.stdout:Sep 10 09:34:53 smithi005 bash[70990]: cbc4e0ec9a31bd14cfe177b45d10e437d01edf3925620126f547534fa1c141b8 2024-09-10T09:34:53.307 INFO:journalctl@ceph.mds.d.smithi005.stdout:Sep 10 09:34:53 smithi005 systemd[1]: Started Ceph mds.d for bade4b50-6f56-11ef-bcea-c7b262605968. 2024-09-10T09:34:53.567 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:34:54.165 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:34:54 smithi005 ceph-mon[30350]: mds.? [v2:172.21.15.5:6836/2894314507,v1:172.21.15.5:6837/2894314507] up:boot 2024-09-10T09:34:54.165 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:34:54 smithi005 ceph-mon[30350]: fsmap 4 up:standby 2024-09-10T09:34:54.165 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:34:54 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-10T09:34:54.165 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:34:54 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:34:54.166 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:34:54 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:34:54.166 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:34:54 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:34:54.166 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:34:54 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T09:34:54.166 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:34:54 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T09:34:54.166 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:34:54 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:34:54.166 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:34:54 smithi005 ceph-mon[30350]: Deploying daemon mds.e on smithi102 2024-09-10T09:34:54.166 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:34:54 smithi005 ceph-mon[30350]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T09:34:54.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:54.179+0000 7f6ab0c00640 1 -- 172.21.15.5:0/4180187727 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6aac103ff0 msgr2=0x7f6aac1043d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:34:54.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:54.179+0000 7f6ab0c00640 1 --2- 172.21.15.5:0/4180187727 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6aac103ff0 0x7f6aac1043d0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f6a9c009860 tx=0x7f6a9c02a340 comp rx=0 tx=0).stop 2024-09-10T09:34:54.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:54.179+0000 7f6ab0c00640 1 -- 172.21.15.5:0/4180187727 shutdown_connections 2024-09-10T09:34:54.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:54.179+0000 7f6ab0c00640 1 --2- 172.21.15.5:0/4180187727 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6aac109120 0x7f6aac110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:34:54.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:54.179+0000 7f6ab0c00640 1 --2- 172.21.15.5:0/4180187727 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6aac1049a0 0x7f6aac1089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:34:54.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:54.179+0000 7f6ab0c00640 1 --2- 172.21.15.5:0/4180187727 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6aac103ff0 0x7f6aac1043d0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:34:54.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:54.179+0000 7f6ab0c00640 1 -- 172.21.15.5:0/4180187727 >> 172.21.15.5:0/4180187727 conn(0x7f6aac0ffa90 msgr2=0x7f6aac101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:34:54.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:54.179+0000 7f6ab0c00640 1 -- 172.21.15.5:0/4180187727 shutdown_connections 2024-09-10T09:34:54.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:54.179+0000 7f6ab0c00640 1 -- 172.21.15.5:0/4180187727 wait complete. 2024-09-10T09:34:54.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:54.180+0000 7f6ab0c00640 1 Processor -- start 2024-09-10T09:34:54.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:54.180+0000 7f6ab0c00640 1 -- start start 2024-09-10T09:34:54.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:54.180+0000 7f6ab0c00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6aac103ff0 0x7f6aac19a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:34:54.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:54.180+0000 7f6ab0c00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6aac1049a0 0x7f6aac19abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:34:54.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:54.180+0000 7f6ab0c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6aac109120 0x7f6aac19ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:34:54.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:54.181+0000 7f6ab0c00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f6aac114150 con 0x7f6aac1049a0 2024-09-10T09:34:54.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:54.181+0000 7f6ab0c00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f6aac113fd0 con 0x7f6aac109120 2024-09-10T09:34:54.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:54.181+0000 7f6ab0c00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f6aac1142d0 con 0x7f6aac103ff0 2024-09-10T09:34:54.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:54.181+0000 7f6aa9400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6aac1049a0 0x7f6aac19abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:34:54.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:54.181+0000 7f6aa9400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6aac1049a0 0x7f6aac19abb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43208/0 (socket says 172.21.15.5:43208) 2024-09-10T09:34:54.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:54.181+0000 7f6aa9400640 1 -- 172.21.15.5:0/2685892201 learned_addr learned my addr 172.21.15.5:0/2685892201 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:34:54.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:54.181+0000 7f6aa9e00640 1 --2- 172.21.15.5:0/2685892201 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6aac103ff0 0x7f6aac19a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:34:54.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:54.181+0000 7f6aaa800640 1 --2- 172.21.15.5:0/2685892201 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6aac109120 0x7f6aac19ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:34:54.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:54.181+0000 7f6aa9400640 1 -- 172.21.15.5:0/2685892201 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6aac103ff0 msgr2=0x7f6aac19a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:34:54.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:54.181+0000 7f6aa9400640 1 --2- 172.21.15.5:0/2685892201 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6aac103ff0 0x7f6aac19a670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:34:54.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:54.181+0000 7f6aa9400640 1 -- 172.21.15.5:0/2685892201 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6aac109120 msgr2=0x7f6aac19ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:34:54.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:54.181+0000 7f6aa9400640 1 --2- 172.21.15.5:0/2685892201 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6aac109120 0x7f6aac19ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:34:54.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:54.181+0000 7f6aa9400640 1 -- 172.21.15.5:0/2685892201 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6aac19f620 con 0x7f6aac1049a0 2024-09-10T09:34:54.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:54.181+0000 7f6aa9e00640 1 --2- 172.21.15.5:0/2685892201 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6aac103ff0 0x7f6aac19a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:34:54.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:54.182+0000 7f6aa9400640 1 --2- 172.21.15.5:0/2685892201 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6aac1049a0 0x7f6aac19abb0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f6a9800e9c0 tx=0x7f6a9800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:34:54.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:54.182+0000 7f6aa2a00640 1 -- 172.21.15.5:0/2685892201 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a9800ccc0 con 0x7f6aac1049a0 2024-09-10T09:34:54.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:54.182+0000 7f6aa2a00640 1 -- 172.21.15.5:0/2685892201 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6a98005500 con 0x7f6aac1049a0 2024-09-10T09:34:54.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:54.182+0000 7f6ab0c00640 1 -- 172.21.15.5:0/2685892201 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6aac19f910 con 0x7f6aac1049a0 2024-09-10T09:34:54.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:54.182+0000 7f6aa2a00640 1 -- 172.21.15.5:0/2685892201 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a98010430 con 0x7f6aac1049a0 2024-09-10T09:34:54.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:54.182+0000 7f6ab0c00640 1 -- 172.21.15.5:0/2685892201 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6aac1a71f0 con 0x7f6aac1049a0 2024-09-10T09:34:54.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:54.183+0000 7f6ab0c00640 1 -- 172.21.15.5:0/2685892201 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6aac1043d0 con 0x7f6aac1049a0 2024-09-10T09:34:54.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:54.185+0000 7f6aa2a00640 1 -- 172.21.15.5:0/2685892201 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7f6a980046c0 con 0x7f6aac1049a0 2024-09-10T09:34:54.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:54.187+0000 7f6aa2a00640 1 --2- 172.21.15.5:0/2685892201 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f6a900a2b80 0x7f6a900a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:34:54.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:54.187+0000 7f6aa2a00640 1 -- 172.21.15.5:0/2685892201 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7127+0+0 (secure 0 0 0) 0x7f6a980cef30 con 0x7f6aac1049a0 2024-09-10T09:34:54.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:54.187+0000 7f6aa9e00640 1 --2- 172.21.15.5:0/2685892201 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f6a900a2b80 0x7f6a900a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:34:54.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:54.188+0000 7f6aa9e00640 1 --2- 172.21.15.5:0/2685892201 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f6a900a2b80 0x7f6a900a5040 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f6a9c02a930 tx=0x7f6a9c0056d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:34:54.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:54.191+0000 7f6aa2a00640 1 -- 172.21.15.5:0/2685892201 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f6a9809c290 con 0x7f6aac1049a0 2024-09-10T09:34:54.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:34:54 smithi102 ceph-mon[35442]: mds.? [v2:172.21.15.5:6836/2894314507,v1:172.21.15.5:6837/2894314507] up:boot 2024-09-10T09:34:54.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:34:54 smithi102 ceph-mon[35442]: fsmap 4 up:standby 2024-09-10T09:34:54.385 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:34:54 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-10T09:34:54.385 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:34:54 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:34:54.385 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:34:54 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:34:54.385 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:34:54 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:34:54.385 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:34:54 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T09:34:54.385 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:34:54 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T09:34:54.385 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:34:54 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:34:54.385 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:34:54 smithi102 ceph-mon[35442]: Deploying daemon mds.e on smithi102 2024-09-10T09:34:54.385 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:34:54 smithi102 ceph-mon[35442]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T09:34:54.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:54.411+0000 7f6ab0c00640 1 -- 172.21.15.5:0/2685892201 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6aac19b9d0 con 0x7f6aac1049a0 2024-09-10T09:34:54.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:54.412+0000 7f6aa2a00640 1 -- 172.21.15.5:0/2685892201 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) ==== 93+0+3907 (secure 0 0 0) 0x7f6a9809bb90 con 0x7f6aac1049a0 2024-09-10T09:34:54.413 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-10T09:34:54.413 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":5,"btime":"2024-09-10T09:34:53:125640+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14454,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/2970594311","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2970594311},{"type":"v1","addr":"172.21.15.5:6835","nonce":2970594311}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6833/2457007427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":2457007427},{"type":"v1","addr":"172.21.15.102:6833","nonce":2457007427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24460,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6833/356445159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":356445159},{"type":"v1","addr":"172.21.15.159:6833","nonce":356445159}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24467,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/2894314507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":2894314507},{"type":"v1","addr":"172.21.15.5:6837","nonce":2894314507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[]} 2024-09-10T09:34:54.413 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 5 2024-09-10T09:34:54.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:54.414+0000 7f6ab0c00640 1 -- 172.21.15.5:0/2685892201 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f6a900a2b80 msgr2=0x7f6a900a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:34:54.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:54.414+0000 7f6ab0c00640 1 --2- 172.21.15.5:0/2685892201 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f6a900a2b80 0x7f6a900a5040 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f6a9c02a930 tx=0x7f6a9c0056d0 comp rx=0 tx=0).stop 2024-09-10T09:34:54.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:54.414+0000 7f6ab0c00640 1 -- 172.21.15.5:0/2685892201 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6aac1049a0 msgr2=0x7f6aac19abb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:34:54.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:54.414+0000 7f6ab0c00640 1 --2- 172.21.15.5:0/2685892201 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6aac1049a0 0x7f6aac19abb0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f6a9800e9c0 tx=0x7f6a9800ee80 comp rx=0 tx=0).stop 2024-09-10T09:34:54.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:54.414+0000 7f6ab0c00640 1 -- 172.21.15.5:0/2685892201 shutdown_connections 2024-09-10T09:34:54.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:54.414+0000 7f6ab0c00640 1 --2- 172.21.15.5:0/2685892201 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f6a900a2b80 0x7f6a900a5040 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:34:54.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:54.414+0000 7f6ab0c00640 1 --2- 172.21.15.5:0/2685892201 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6aac109120 0x7f6aac19ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:34:54.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:54.414+0000 7f6ab0c00640 1 --2- 172.21.15.5:0/2685892201 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6aac1049a0 0x7f6aac19abb0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:34:54.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:54.414+0000 7f6ab0c00640 1 --2- 172.21.15.5:0/2685892201 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6aac103ff0 0x7f6aac19a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:34:54.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:54.414+0000 7f6ab0c00640 1 -- 172.21.15.5:0/2685892201 >> 172.21.15.5:0/2685892201 conn(0x7f6aac0ffa90 msgr2=0x7f6aac100d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:34:54.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:54.414+0000 7f6ab0c00640 1 -- 172.21.15.5:0/2685892201 shutdown_connections 2024-09-10T09:34:54.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:54.414+0000 7f6ab0c00640 1 -- 172.21.15.5:0/2685892201 wait complete. 2024-09-10T09:34:54.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:34:54 smithi159 ceph-mon[35513]: mds.? [v2:172.21.15.5:6836/2894314507,v1:172.21.15.5:6837/2894314507] up:boot 2024-09-10T09:34:54.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:34:54 smithi159 ceph-mon[35513]: fsmap 4 up:standby 2024-09-10T09:34:54.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:34:54 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-10T09:34:54.601 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:34:54 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:34:54.601 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:34:54 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:34:54.601 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:34:54 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:34:54.601 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:34:54 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T09:34:54.601 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:34:54 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T09:34:54.601 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:34:54 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:34:54.601 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:34:54 smithi159 ceph-mon[35513]: Deploying daemon mds.e on smithi102 2024-09-10T09:34:54.601 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:34:54 smithi159 ceph-mon[35513]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T09:34:55.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:34:55 smithi102 ceph-mon[35442]: from='client.? 172.21.15.5:0/2685892201' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:34:55.384 INFO:journalctl@ceph.mds.e.smithi102.stdout:Sep 10 09:34:55 smithi102 systemd[1]: Starting Ceph mds.e for bade4b50-6f56-11ef-bcea-c7b262605968... 2024-09-10T09:34:55.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:34:55 smithi005 ceph-mon[30350]: from='client.? 172.21.15.5:0/2685892201' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:34:55.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:34:55 smithi159 ceph-mon[35513]: from='client.? 172.21.15.5:0/2685892201' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:34:55.849 INFO:journalctl@ceph.mds.e.smithi102.stdout:Sep 10 09:34:55 smithi102 podman[64577]: 2024-09-10 09:34:55.438211791 +0000 UTC m=+0.016052300 image pull 41d67112835771c3efca60c6b8fe297c7ddaecdc26cea26fd28aa18636e49005 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 2024-09-10T09:34:55.849 INFO:journalctl@ceph.mds.e.smithi102.stdout:Sep 10 09:34:55 smithi102 podman[64577]: 2024-09-10 09:34:55.562968201 +0000 UTC m=+0.140808701 container create b23eaeab4c90e6fb404c7bb6e4895de9ec81a002cf262d21f24bf4b3f93147de (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mds-e, GIT_CLEAN=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-09-10T09:34:56.134 INFO:journalctl@ceph.mds.e.smithi102.stdout:Sep 10 09:34:55 smithi102 podman[64577]: 2024-09-10 09:34:55.8492394 +0000 UTC m=+0.427079900 container init b23eaeab4c90e6fb404c7bb6e4895de9ec81a002cf262d21f24bf4b3f93147de (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mds-e, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-10T09:34:56.135 INFO:journalctl@ceph.mds.e.smithi102.stdout:Sep 10 09:34:55 smithi102 podman[64577]: 2024-09-10 09:34:55.851946456 +0000 UTC m=+0.429786957 container start b23eaeab4c90e6fb404c7bb6e4895de9ec81a002cf262d21f24bf4b3f93147de (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mds-e, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=) 2024-09-10T09:34:56.135 INFO:journalctl@ceph.mds.e.smithi102.stdout:Sep 10 09:34:55 smithi102 ceph-mds[64596]: -- 172.21.15.102:0/2880794844 <== mon.2 v2:172.21.15.102:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55ca155c9860 con 0x55ca155a7000 2024-09-10T09:34:56.135 INFO:journalctl@ceph.mds.e.smithi102.stdout:Sep 10 09:34:55 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mds-e[64592]: starting mds.e at 2024-09-10T09:34:56.135 INFO:journalctl@ceph.mds.e.smithi102.stdout:Sep 10 09:34:55 smithi102 bash[64577]: b23eaeab4c90e6fb404c7bb6e4895de9ec81a002cf262d21f24bf4b3f93147de 2024-09-10T09:34:56.135 INFO:journalctl@ceph.mds.e.smithi102.stdout:Sep 10 09:34:55 smithi102 systemd[1]: Started Ceph mds.e for bade4b50-6f56-11ef-bcea-c7b262605968. 2024-09-10T09:34:56.471 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:34:56 smithi159 ceph-mon[35513]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T09:34:56.472 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:34:56 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:34:56.472 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:34:56 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:34:56.472 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:34:56 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:34:56.472 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:34:56 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T09:34:56.472 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:34:56 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T09:34:56.472 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:34:56 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:34:56.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:34:56 smithi005 ceph-mon[30350]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T09:34:56.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:34:56 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:34:56.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:34:56 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:34:56.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:34:56 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:34:56.557 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:34:56 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T09:34:56.557 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:34:56 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T09:34:56.557 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:34:56 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:34:56.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:34:56 smithi102 ceph-mon[35442]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T09:34:56.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:34:56 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:34:56.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:34:56 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:34:56.635 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:34:56 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:34:56.635 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:34:56 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T09:34:56.635 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:34:56 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T09:34:56.635 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:34:56 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:34:57.182 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T09:34:57.474 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:34:57 smithi159 ceph-mon[35513]: Deploying daemon mds.f on smithi159 2024-09-10T09:34:57.474 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:34:57 smithi159 ceph-mon[35513]: mds.? [v2:172.21.15.102:6834/1647549111,v1:172.21.15.102:6835/1647549111] up:boot 2024-09-10T09:34:57.474 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:34:57 smithi159 ceph-mon[35513]: fsmap 5 up:standby 2024-09-10T09:34:57.474 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:34:57 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-10T09:34:57.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:34:57 smithi005 ceph-mon[30350]: Deploying daemon mds.f on smithi159 2024-09-10T09:34:57.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:34:57 smithi005 ceph-mon[30350]: mds.? [v2:172.21.15.102:6834/1647549111,v1:172.21.15.102:6835/1647549111] up:boot 2024-09-10T09:34:57.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:34:57 smithi005 ceph-mon[30350]: fsmap 5 up:standby 2024-09-10T09:34:57.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:34:57 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-10T09:34:57.570 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:34:57.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:34:57 smithi102 ceph-mon[35442]: Deploying daemon mds.f on smithi159 2024-09-10T09:34:57.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:34:57 smithi102 ceph-mon[35442]: mds.? [v2:172.21.15.102:6834/1647549111,v1:172.21.15.102:6835/1647549111] up:boot 2024-09-10T09:34:57.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:34:57 smithi102 ceph-mon[35442]: fsmap 5 up:standby 2024-09-10T09:34:57.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:34:57 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-10T09:34:58.100 INFO:journalctl@ceph.mds.f.smithi159.stdout:Sep 10 09:34:57 smithi159 systemd[1]: Starting Ceph mds.f for bade4b50-6f56-11ef-bcea-c7b262605968... 2024-09-10T09:34:58.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:58.144+0000 7fd218a00640 1 -- 172.21.15.5:0/263792601 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd214101de0 msgr2=0x7fd2141021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:34:58.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:58.144+0000 7fd218a00640 1 --2- 172.21.15.5:0/263792601 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd214101de0 0x7fd2141021c0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fd204009900 tx=0x7fd20402a3f0 comp rx=0 tx=0).stop 2024-09-10T09:34:58.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:58.145+0000 7fd218a00640 1 -- 172.21.15.5:0/263792601 shutdown_connections 2024-09-10T09:34:58.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:58.145+0000 7fd218a00640 1 --2- 172.21.15.5:0/263792601 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd214106f10 0x7fd21410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:34:58.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:58.145+0000 7fd218a00640 1 --2- 172.21.15.5:0/263792601 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd214102790 0x7fd2141067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:34:58.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:58.145+0000 7fd218a00640 1 --2- 172.21.15.5:0/263792601 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd214101de0 0x7fd2141021c0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:34:58.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:58.145+0000 7fd218a00640 1 -- 172.21.15.5:0/263792601 >> 172.21.15.5:0/263792601 conn(0x7fd2140fd8c0 msgr2=0x7fd2140ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:34:58.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:58.145+0000 7fd218a00640 1 -- 172.21.15.5:0/263792601 shutdown_connections 2024-09-10T09:34:58.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:58.145+0000 7fd218a00640 1 -- 172.21.15.5:0/263792601 wait complete. 2024-09-10T09:34:58.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:58.146+0000 7fd218a00640 1 Processor -- start 2024-09-10T09:34:58.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:58.146+0000 7fd218a00640 1 -- start start 2024-09-10T09:34:58.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:58.146+0000 7fd218a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd214101de0 0x7fd214198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:34:58.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:58.146+0000 7fd218a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd214102790 0x7fd214198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:34:58.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:58.146+0000 7fd218a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd214106f10 0x7fd21419cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:34:58.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:58.146+0000 7fd218a00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd214111f70 con 0x7fd214101de0 2024-09-10T09:34:58.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:58.146+0000 7fd218a00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fd214111df0 con 0x7fd214102790 2024-09-10T09:34:58.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:58.146+0000 7fd218a00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fd2141120f0 con 0x7fd214106f10 2024-09-10T09:34:58.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:58.146+0000 7fd211e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd214101de0 0x7fd214198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:34:58.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:58.147+0000 7fd211e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd214101de0 0x7fd214198430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43230/0 (socket says 172.21.15.5:43230) 2024-09-10T09:34:58.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:58.147+0000 7fd211e00640 1 -- 172.21.15.5:0/1535918158 learned_addr learned my addr 172.21.15.5:0/1535918158 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:34:58.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:58.147+0000 7fd212800640 1 --2- 172.21.15.5:0/1535918158 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd214106f10 0x7fd21419cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:34:58.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:58.147+0000 7fd209400640 1 --2- 172.21.15.5:0/1535918158 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd214102790 0x7fd214198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:34:58.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:58.147+0000 7fd211e00640 1 -- 172.21.15.5:0/1535918158 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd214106f10 msgr2=0x7fd21419cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:34:58.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:58.147+0000 7fd211e00640 1 --2- 172.21.15.5:0/1535918158 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd214106f10 0x7fd21419cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:34:58.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:58.147+0000 7fd211e00640 1 -- 172.21.15.5:0/1535918158 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd214102790 msgr2=0x7fd214198970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:34:58.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:58.147+0000 7fd211e00640 1 --2- 172.21.15.5:0/1535918158 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd214102790 0x7fd214198970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:34:58.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:58.147+0000 7fd211e00640 1 -- 172.21.15.5:0/1535918158 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd21419d3e0 con 0x7fd214101de0 2024-09-10T09:34:58.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:58.147+0000 7fd212800640 1 --2- 172.21.15.5:0/1535918158 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd214106f10 0x7fd21419cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:34:58.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:58.147+0000 7fd211e00640 1 --2- 172.21.15.5:0/1535918158 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd214101de0 0x7fd214198430 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fd204006010 tx=0x7fd2040042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:34:58.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:58.148+0000 7fd20b400640 1 -- 172.21.15.5:0/1535918158 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd20403f030 con 0x7fd214101de0 2024-09-10T09:34:58.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:58.148+0000 7fd20b400640 1 -- 172.21.15.5:0/1535918158 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd204003ce0 con 0x7fd214101de0 2024-09-10T09:34:58.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:58.148+0000 7fd218a00640 1 -- 172.21.15.5:0/1535918158 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd21419d670 con 0x7fd214101de0 2024-09-10T09:34:58.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:58.148+0000 7fd20b400640 1 -- 172.21.15.5:0/1535918158 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd204035070 con 0x7fd214101de0 2024-09-10T09:34:58.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:58.148+0000 7fd218a00640 1 -- 172.21.15.5:0/1535918158 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2141a4fb0 con 0x7fd214101de0 2024-09-10T09:34:58.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:58.149+0000 7fd218a00640 1 -- 172.21.15.5:0/1535918158 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd2141032c0 con 0x7fd214101de0 2024-09-10T09:34:58.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:58.151+0000 7fd20b400640 1 -- 172.21.15.5:0/1535918158 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7fd204042030 con 0x7fd214101de0 2024-09-10T09:34:58.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:58.152+0000 7fd20b400640 1 --2- 172.21.15.5:0/1535918158 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fd1f80a2b80 0x7fd1f80a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:34:58.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:58.152+0000 7fd20b400640 1 -- 172.21.15.5:0/1535918158 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7127+0+0 (secure 0 0 0) 0x7fd2040ef170 con 0x7fd214101de0 2024-09-10T09:34:58.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:58.152+0000 7fd209400640 1 --2- 172.21.15.5:0/1535918158 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fd1f80a2b80 0x7fd1f80a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:34:58.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:58.153+0000 7fd209400640 1 --2- 172.21.15.5:0/1535918158 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fd1f80a2b80 0x7fd1f80a5040 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fd2140ffe00 tx=0x7fd1fc00b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:34:58.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:58.156+0000 7fd20b400640 1 -- 172.21.15.5:0/1535918158 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd2040bbf40 con 0x7fd214101de0 2024-09-10T09:34:58.306 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:34:58 smithi005 ceph-mon[30350]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T09:34:58.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:58.380+0000 7fd218a00640 1 -- 172.21.15.5:0/1535918158 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd21410a0f0 con 0x7fd214101de0 2024-09-10T09:34:58.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:58.381+0000 7fd20b400640 1 -- 172.21.15.5:0/1535918158 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+4735 (secure 0 0 0) 0x7fd2040c0240 con 0x7fd214101de0 2024-09-10T09:34:58.383 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-10T09:34:58.383 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":6,"btime":"2024-09-10T09:34:56:213403+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14454,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/2970594311","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2970594311},{"type":"v1","addr":"172.21.15.5:6835","nonce":2970594311}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6833/2457007427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":2457007427},{"type":"v1","addr":"172.21.15.102:6833","nonce":2457007427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24460,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6833/356445159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":356445159},{"type":"v1","addr":"172.21.15.159:6833","nonce":356445159}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24467,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/2894314507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":2894314507},{"type":"v1","addr":"172.21.15.5:6837","nonce":2894314507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24479,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6835/1647549111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":1647549111},{"type":"v1","addr":"172.21.15.102:6835","nonce":1647549111}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-09-10T09:34:58.383 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 6 2024-09-10T09:34:58.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:58.384+0000 7fd218a00640 1 -- 172.21.15.5:0/1535918158 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fd1f80a2b80 msgr2=0x7fd1f80a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:34:58.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:58.384+0000 7fd218a00640 1 --2- 172.21.15.5:0/1535918158 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fd1f80a2b80 0x7fd1f80a5040 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fd2140ffe00 tx=0x7fd1fc00b410 comp rx=0 tx=0).stop 2024-09-10T09:34:58.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:58.384+0000 7fd218a00640 1 -- 172.21.15.5:0/1535918158 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd214101de0 msgr2=0x7fd214198430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:34:58.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:58.384+0000 7fd218a00640 1 --2- 172.21.15.5:0/1535918158 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd214101de0 0x7fd214198430 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fd204006010 tx=0x7fd2040042f0 comp rx=0 tx=0).stop 2024-09-10T09:34:58.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:58.384+0000 7fd218a00640 1 -- 172.21.15.5:0/1535918158 shutdown_connections 2024-09-10T09:34:58.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:58.384+0000 7fd218a00640 1 --2- 172.21.15.5:0/1535918158 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fd1f80a2b80 0x7fd1f80a5040 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:34:58.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:58.384+0000 7fd218a00640 1 --2- 172.21.15.5:0/1535918158 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd214106f10 0x7fd21419cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:34:58.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:58.384+0000 7fd218a00640 1 --2- 172.21.15.5:0/1535918158 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd214102790 0x7fd214198970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:34:58.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:58.384+0000 7fd218a00640 1 --2- 172.21.15.5:0/1535918158 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd214101de0 0x7fd214198430 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:34:58.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:58.384+0000 7fd218a00640 1 -- 172.21.15.5:0/1535918158 >> 172.21.15.5:0/1535918158 conn(0x7fd2140fd8c0 msgr2=0x7fd2140fecd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:34:58.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:58.384+0000 7fd218a00640 1 -- 172.21.15.5:0/1535918158 shutdown_connections 2024-09-10T09:34:58.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:34:58.384+0000 7fd218a00640 1 -- 172.21.15.5:0/1535918158 wait complete. 2024-09-10T09:34:58.484 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:34:58 smithi159 ceph-mon[35513]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T09:34:58.484 INFO:journalctl@ceph.mds.f.smithi159.stdout:Sep 10 09:34:58 smithi159 podman[64958]: 2024-09-10 09:34:58.096674267 +0000 UTC m=+0.015685140 image pull 41d67112835771c3efca60c6b8fe297c7ddaecdc26cea26fd28aa18636e49005 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 2024-09-10T09:34:58.485 INFO:journalctl@ceph.mds.f.smithi159.stdout:Sep 10 09:34:58 smithi159 podman[64958]: 2024-09-10 09:34:58.214469215 +0000 UTC m=+0.133480128 container create 206912c328a0446e5b2f47ba768c5cca7fa401e0ed50f09a8e9132b94a26182f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mds-f, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.37.2, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-09-10T09:34:58.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:34:58 smithi102 ceph-mon[35442]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T09:34:58.850 INFO:journalctl@ceph.mds.f.smithi159.stdout:Sep 10 09:34:58 smithi159 podman[64958]: 2024-09-10 09:34:58.483904374 +0000 UTC m=+0.402915243 container init 206912c328a0446e5b2f47ba768c5cca7fa401e0ed50f09a8e9132b94a26182f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mds-f, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, GIT_CLEAN=True) 2024-09-10T09:34:58.850 INFO:journalctl@ceph.mds.f.smithi159.stdout:Sep 10 09:34:58 smithi159 podman[64958]: 2024-09-10 09:34:58.487013829 +0000 UTC m=+0.406024702 container start 206912c328a0446e5b2f47ba768c5cca7fa401e0ed50f09a8e9132b94a26182f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mds-f, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS) 2024-09-10T09:34:58.850 INFO:journalctl@ceph.mds.f.smithi159.stdout:Sep 10 09:34:58 smithi159 ceph-mds[64977]: -- 172.21.15.159:0/3844724136 <== mon.1 v2:172.21.15.159:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55dd160ef860 con 0x55dd160cd400 2024-09-10T09:34:58.851 INFO:journalctl@ceph.mds.f.smithi159.stdout:Sep 10 09:34:58 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mds-f[64973]: starting mds.f at 2024-09-10T09:34:58.851 INFO:journalctl@ceph.mds.f.smithi159.stdout:Sep 10 09:34:58 smithi159 bash[64958]: 206912c328a0446e5b2f47ba768c5cca7fa401e0ed50f09a8e9132b94a26182f 2024-09-10T09:34:58.851 INFO:journalctl@ceph.mds.f.smithi159.stdout:Sep 10 09:34:58 smithi159 systemd[1]: Started Ceph mds.f for bade4b50-6f56-11ef-bcea-c7b262605968. 2024-09-10T09:34:59.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:34:59 smithi005 ceph-mon[30350]: from='client.? 172.21.15.5:0/1535918158' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:34:59.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:34:59 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:34:59.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:34:59 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:34:59.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:34:59 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:34:59.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:34:59 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T09:34:59.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:34:59 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T09:34:59.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:34:59 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:34:59.601 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:34:59 smithi159 ceph-mon[35513]: from='client.? 172.21.15.5:0/1535918158' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:34:59.601 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:34:59 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:34:59.601 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:34:59 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:34:59.601 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:34:59 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:34:59.601 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:34:59 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T09:34:59.601 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:34:59 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T09:34:59.601 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:34:59 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:34:59.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:34:59 smithi102 ceph-mon[35442]: from='client.? 172.21.15.5:0/1535918158' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:34:59.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:34:59 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:34:59.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:34:59 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:34:59.635 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:34:59 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:34:59.635 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:34:59 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T09:34:59.635 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:34:59 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T09:34:59.635 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:34:59 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:35:00.514 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:35:00 smithi159 ceph-mon[35513]: Deploying daemon mds.c on smithi159 2024-09-10T09:35:00.514 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:35:00 smithi159 ceph-mon[35513]: mds.? [v2:172.21.15.159:6834/952657242,v1:172.21.15.159:6835/952657242] up:boot 2024-09-10T09:35:00.514 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:35:00 smithi159 ceph-mon[35513]: fsmap 6 up:standby 2024-09-10T09:35:00.514 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:35:00 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-10T09:35:00.514 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:35:00 smithi159 ceph-mon[35513]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T09:35:00.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:35:00 smithi005 ceph-mon[30350]: Deploying daemon mds.c on smithi159 2024-09-10T09:35:00.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:35:00 smithi005 ceph-mon[30350]: mds.? [v2:172.21.15.159:6834/952657242,v1:172.21.15.159:6835/952657242] up:boot 2024-09-10T09:35:00.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:35:00 smithi005 ceph-mon[30350]: fsmap 6 up:standby 2024-09-10T09:35:00.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:35:00 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-10T09:35:00.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:35:00 smithi005 ceph-mon[30350]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T09:35:00.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:35:00 smithi102 ceph-mon[35442]: Deploying daemon mds.c on smithi159 2024-09-10T09:35:00.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:35:00 smithi102 ceph-mon[35442]: mds.? [v2:172.21.15.159:6834/952657242,v1:172.21.15.159:6835/952657242] up:boot 2024-09-10T09:35:00.635 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:35:00 smithi102 ceph-mon[35442]: fsmap 6 up:standby 2024-09-10T09:35:00.635 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:35:00 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-10T09:35:00.635 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:35:00 smithi102 ceph-mon[35442]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T09:35:00.850 INFO:journalctl@ceph.mds.c.smithi159.stdout:Sep 10 09:35:00 smithi159 systemd[1]: Starting Ceph mds.c for bade4b50-6f56-11ef-bcea-c7b262605968... 2024-09-10T09:35:01.183 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T09:35:01.333 INFO:journalctl@ceph.mds.c.smithi159.stdout:Sep 10 09:35:00 smithi159 podman[65260]: 2024-09-10 09:35:00.824391188 +0000 UTC m=+0.015756132 image pull 41d67112835771c3efca60c6b8fe297c7ddaecdc26cea26fd28aa18636e49005 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 2024-09-10T09:35:01.333 INFO:journalctl@ceph.mds.c.smithi159.stdout:Sep 10 09:35:00 smithi159 podman[65260]: 2024-09-10 09:35:00.945736399 +0000 UTC m=+0.137101390 container create d0d410063ea5681cb353cad93f1b4c5eb2f621442e3d855122b32108daaea28c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mds-c, io.buildah.version=1.37.2, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-09-10T09:35:01.333 INFO:journalctl@ceph.mds.c.smithi159.stdout:Sep 10 09:35:01 smithi159 podman[65260]: 2024-09-10 09:35:01.332239448 +0000 UTC m=+0.523604397 container init d0d410063ea5681cb353cad93f1b4c5eb2f621442e3d855122b32108daaea28c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mds-c, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.37.2) 2024-09-10T09:35:01.574 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:35:01.600 INFO:journalctl@ceph.mds.c.smithi159.stdout:Sep 10 09:35:01 smithi159 podman[65260]: 2024-09-10 09:35:01.335370588 +0000 UTC m=+0.526735534 container start d0d410063ea5681cb353cad93f1b4c5eb2f621442e3d855122b32108daaea28c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mds-c, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-10T09:35:01.601 INFO:journalctl@ceph.mds.c.smithi159.stdout:Sep 10 09:35:01 smithi159 ceph-mds[65280]: -- 172.21.15.159:0/1892934738 <== mon.1 v2:172.21.15.159:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x556259a1fc20 con 0x5562599fc800 2024-09-10T09:35:01.601 INFO:journalctl@ceph.mds.c.smithi159.stdout:Sep 10 09:35:01 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mds-c[65276]: starting mds.c at 2024-09-10T09:35:01.601 INFO:journalctl@ceph.mds.c.smithi159.stdout:Sep 10 09:35:01 smithi159 bash[65260]: d0d410063ea5681cb353cad93f1b4c5eb2f621442e3d855122b32108daaea28c 2024-09-10T09:35:01.601 INFO:journalctl@ceph.mds.c.smithi159.stdout:Sep 10 09:35:01 smithi159 systemd[1]: Started Ceph mds.c for bade4b50-6f56-11ef-bcea-c7b262605968. 2024-09-10T09:35:02.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:02.127+0000 7f1191a00640 1 -- 172.21.15.5:0/557738295 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f118c103ff0 msgr2=0x7f118c1043d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:35:02.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:02.127+0000 7f1191a00640 1 --2- 172.21.15.5:0/557738295 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f118c103ff0 0x7f118c1043d0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f117c009860 tx=0x7f117c02a340 comp rx=0 tx=0).stop 2024-09-10T09:35:02.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:02.127+0000 7f1191a00640 1 -- 172.21.15.5:0/557738295 shutdown_connections 2024-09-10T09:35:02.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:02.127+0000 7f1191a00640 1 --2- 172.21.15.5:0/557738295 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f118c109120 0x7f118c110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:35:02.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:02.127+0000 7f1191a00640 1 --2- 172.21.15.5:0/557738295 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f118c1049a0 0x7f118c1089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:35:02.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:02.127+0000 7f1191a00640 1 --2- 172.21.15.5:0/557738295 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f118c103ff0 0x7f118c1043d0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:35:02.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:02.127+0000 7f1191a00640 1 -- 172.21.15.5:0/557738295 >> 172.21.15.5:0/557738295 conn(0x7f118c0ffa90 msgr2=0x7f118c101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:35:02.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:02.127+0000 7f1191a00640 1 -- 172.21.15.5:0/557738295 shutdown_connections 2024-09-10T09:35:02.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:02.127+0000 7f1191a00640 1 -- 172.21.15.5:0/557738295 wait complete. 2024-09-10T09:35:02.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:02.128+0000 7f1191a00640 1 Processor -- start 2024-09-10T09:35:02.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:02.128+0000 7f1191a00640 1 -- start start 2024-09-10T09:35:02.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:02.128+0000 7f1191a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f118c103ff0 0x7f118c19a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:35:02.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:02.128+0000 7f1191a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f118c1049a0 0x7f118c19ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:35:02.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:02.128+0000 7f1191a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f118c109120 0x7f118c19eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:35:02.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:02.128+0000 7f1191a00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f118c114120 con 0x7f118c1049a0 2024-09-10T09:35:02.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:02.128+0000 7f1191a00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f118c113fa0 con 0x7f118c109120 2024-09-10T09:35:02.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:02.128+0000 7f1191a00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f118c1142a0 con 0x7f118c103ff0 2024-09-10T09:35:02.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:02.129+0000 7f1183e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f118c1049a0 0x7f118c19ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:35:02.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:02.129+0000 7f1183e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f118c1049a0 0x7f118c19ab60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37638/0 (socket says 172.21.15.5:37638) 2024-09-10T09:35:02.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:02.129+0000 7f1183e00640 1 -- 172.21.15.5:0/1107047103 learned_addr learned my addr 172.21.15.5:0/1107047103 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:35:02.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:02.129+0000 7f118b400640 1 --2- 172.21.15.5:0/1107047103 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f118c109120 0x7f118c19eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:35:02.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:02.129+0000 7f118aa00640 1 --2- 172.21.15.5:0/1107047103 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f118c103ff0 0x7f118c19a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:35:02.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:02.129+0000 7f118b400640 1 -- 172.21.15.5:0/1107047103 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f118c103ff0 msgr2=0x7f118c19a620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:35:02.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:02.129+0000 7f118b400640 1 --2- 172.21.15.5:0/1107047103 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f118c103ff0 0x7f118c19a620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:35:02.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:02.129+0000 7f118b400640 1 -- 172.21.15.5:0/1107047103 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f118c1049a0 msgr2=0x7f118c19ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:35:02.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:02.129+0000 7f118b400640 1 --2- 172.21.15.5:0/1107047103 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f118c1049a0 0x7f118c19ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:35:02.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:02.129+0000 7f118b400640 1 -- 172.21.15.5:0/1107047103 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f118c19f5d0 con 0x7f118c109120 2024-09-10T09:35:02.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:02.129+0000 7f118aa00640 1 --2- 172.21.15.5:0/1107047103 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f118c103ff0 0x7f118c19a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:35:02.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:02.129+0000 7f118b400640 1 --2- 172.21.15.5:0/1107047103 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f118c109120 0x7f118c19eef0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f118400efc0 tx=0x7f118400c800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:35:02.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:02.130+0000 7f1183400640 1 -- 172.21.15.5:0/1107047103 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1184003970 con 0x7f118c109120 2024-09-10T09:35:02.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:02.130+0000 7f1183400640 1 -- 172.21.15.5:0/1107047103 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1184004020 con 0x7f118c109120 2024-09-10T09:35:02.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:02.130+0000 7f1191a00640 1 -- 172.21.15.5:0/1107047103 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f118c19f8c0 con 0x7f118c109120 2024-09-10T09:35:02.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:02.130+0000 7f1183400640 1 -- 172.21.15.5:0/1107047103 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f118400b9f0 con 0x7f118c109120 2024-09-10T09:35:02.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:02.130+0000 7f1191a00640 1 -- 172.21.15.5:0/1107047103 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f118c1a71a0 con 0x7f118c109120 2024-09-10T09:35:02.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:02.131+0000 7f1191a00640 1 -- 172.21.15.5:0/1107047103 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f118c1043d0 con 0x7f118c109120 2024-09-10T09:35:02.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:02.133+0000 7f1183400640 1 -- 172.21.15.5:0/1107047103 <== mon.2 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7f118401d020 con 0x7f118c109120 2024-09-10T09:35:02.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:02.134+0000 7f1183400640 1 --2- 172.21.15.5:0/1107047103 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f11700a2b80 0x7f11700a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:35:02.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:02.134+0000 7f1183400640 1 -- 172.21.15.5:0/1107047103 <== mon.2 v2:172.21.15.102:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7127+0+0 (secure 0 0 0) 0x7f11840cea50 con 0x7f118c109120 2024-09-10T09:35:02.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:02.134+0000 7f118aa00640 1 --2- 172.21.15.5:0/1107047103 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f11700a2b80 0x7f11700a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:35:02.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:02.135+0000 7f118aa00640 1 --2- 172.21.15.5:0/1107047103 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f11700a2b80 0x7f11700a5040 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f117c02a930 tx=0x7f117c0056d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:35:02.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:02.138+0000 7f1183400640 1 -- 172.21.15.5:0/1107047103 <== mon.2 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f118409bdb0 con 0x7f118c109120 2024-09-10T09:35:02.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:02.360+0000 7f1191a00640 1 -- 172.21.15.5:0/1107047103 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f118c10c360 con 0x7f118c109120 2024-09-10T09:35:02.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:02.361+0000 7f1183400640 1 -- 172.21.15.5:0/1107047103 <== mon.2 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+6385 (secure 0 0 0) 0x7f118409b6b0 con 0x7f118c109120 2024-09-10T09:35:02.363 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-10T09:35:02.363 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":8,"btime":"2024-09-10T09:35:01:414335+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14454,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/2970594311","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2970594311},{"type":"v1","addr":"172.21.15.5:6835","nonce":2970594311}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6833/2457007427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":2457007427},{"type":"v1","addr":"172.21.15.102:6833","nonce":2457007427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24460,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6833/356445159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":356445159},{"type":"v1","addr":"172.21.15.159:6833","nonce":356445159}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24467,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/2894314507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":2894314507},{"type":"v1","addr":"172.21.15.5:6837","nonce":2894314507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24479,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6835/1647549111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":1647549111},{"type":"v1","addr":"172.21.15.102:6835","nonce":1647549111}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24505,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6835/952657242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":952657242},{"type":"v1","addr":"172.21.15.159:6835","nonce":952657242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24511,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6837/620627639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":620627639},{"type":"v1","addr":"172.21.15.159:6837","nonce":620627639}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-09-10T09:35:02.363 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 8 2024-09-10T09:35:02.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:02.364+0000 7f1191a00640 1 -- 172.21.15.5:0/1107047103 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f11700a2b80 msgr2=0x7f11700a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:35:02.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:02.364+0000 7f1191a00640 1 --2- 172.21.15.5:0/1107047103 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f11700a2b80 0x7f11700a5040 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f117c02a930 tx=0x7f117c0056d0 comp rx=0 tx=0).stop 2024-09-10T09:35:02.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:02.364+0000 7f1191a00640 1 -- 172.21.15.5:0/1107047103 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f118c109120 msgr2=0x7f118c19eef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:35:02.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:02.364+0000 7f1191a00640 1 --2- 172.21.15.5:0/1107047103 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f118c109120 0x7f118c19eef0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f118400efc0 tx=0x7f118400c800 comp rx=0 tx=0).stop 2024-09-10T09:35:02.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:02.364+0000 7f1191a00640 1 -- 172.21.15.5:0/1107047103 shutdown_connections 2024-09-10T09:35:02.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:02.364+0000 7f1191a00640 1 --2- 172.21.15.5:0/1107047103 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f11700a2b80 0x7f11700a5040 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:35:02.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:02.364+0000 7f1191a00640 1 --2- 172.21.15.5:0/1107047103 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f118c109120 0x7f118c19eef0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:35:02.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:02.364+0000 7f1191a00640 1 --2- 172.21.15.5:0/1107047103 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f118c1049a0 0x7f118c19ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:35:02.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:02.364+0000 7f1191a00640 1 --2- 172.21.15.5:0/1107047103 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f118c103ff0 0x7f118c19a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:35:02.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:02.364+0000 7f1191a00640 1 -- 172.21.15.5:0/1107047103 >> 172.21.15.5:0/1107047103 conn(0x7f118c0ffa90 msgr2=0x7f118c100de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:35:02.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:02.364+0000 7f1191a00640 1 -- 172.21.15.5:0/1107047103 shutdown_connections 2024-09-10T09:35:02.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:02.364+0000 7f1191a00640 1 -- 172.21.15.5:0/1107047103 wait complete. 2024-09-10T09:35:02.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:35:02 smithi102 ceph-mon[35442]: mds.? [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] up:boot 2024-09-10T09:35:02.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:35:02 smithi102 ceph-mon[35442]: fsmap 7 up:standby 2024-09-10T09:35:02.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:35:02 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-10T09:35:02.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:35:02 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:35:02.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:35:02 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:35:02.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:35:02 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:35:02.635 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:35:02 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T09:35:02.635 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:35:02 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T09:35:02.635 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:35:02 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:35:02.635 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:35:02 smithi102 ceph-mon[35442]: Deploying daemon mds.h on smithi102 2024-09-10T09:35:02.635 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:35:02 smithi102 ceph-mon[35442]: from='client.? 172.21.15.5:0/1107047103' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:35:02.726 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:35:02 smithi005 ceph-mon[30350]: mds.? [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] up:boot 2024-09-10T09:35:02.726 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:35:02 smithi005 ceph-mon[30350]: fsmap 7 up:standby 2024-09-10T09:35:02.726 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:35:02 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-10T09:35:02.726 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:35:02 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:35:02.727 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:35:02 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:35:02.727 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:35:02 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:35:02.727 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:35:02 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T09:35:02.727 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:35:02 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T09:35:02.727 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:35:02 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:35:02.727 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:35:02 smithi005 ceph-mon[30350]: Deploying daemon mds.h on smithi102 2024-09-10T09:35:02.727 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:35:02 smithi005 ceph-mon[30350]: from='client.? 172.21.15.5:0/1107047103' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:35:02.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:35:02 smithi159 ceph-mon[35513]: mds.? [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] up:boot 2024-09-10T09:35:02.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:35:02 smithi159 ceph-mon[35513]: fsmap 7 up:standby 2024-09-10T09:35:02.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:35:02 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-10T09:35:02.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:35:02 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:35:02.851 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:35:02 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:35:02.851 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:35:02 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:35:02.851 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:35:02 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T09:35:02.851 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:35:02 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T09:35:02.851 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:35:02 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:35:02.851 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:35:02 smithi159 ceph-mon[35513]: Deploying daemon mds.h on smithi102 2024-09-10T09:35:02.851 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:35:02 smithi159 ceph-mon[35513]: from='client.? 172.21.15.5:0/1107047103' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:35:03.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:35:03 smithi102 ceph-mon[35442]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T09:35:03.635 INFO:journalctl@ceph.mds.h.smithi102.stdout:Sep 10 09:35:03 smithi102 systemd[1]: Starting Ceph mds.h for bade4b50-6f56-11ef-bcea-c7b262605968... 2024-09-10T09:35:03.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:35:03 smithi005 ceph-mon[30350]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T09:35:03.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:35:03 smithi159 ceph-mon[35513]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T09:35:04.056 INFO:journalctl@ceph.mds.h.smithi102.stdout:Sep 10 09:35:03 smithi102 podman[64879]: 2024-09-10 09:35:03.641165178 +0000 UTC m=+0.016065349 image pull 41d67112835771c3efca60c6b8fe297c7ddaecdc26cea26fd28aa18636e49005 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 2024-09-10T09:35:04.056 INFO:journalctl@ceph.mds.h.smithi102.stdout:Sep 10 09:35:03 smithi102 podman[64879]: 2024-09-10 09:35:03.778123324 +0000 UTC m=+0.153023485 container create 0989d5e3d2ed54de7dd4a556306c46317db8753e6660efdb0f601ba19a420317 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mds-h, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-10T09:35:04.056 INFO:journalctl@ceph.mds.h.smithi102.stdout:Sep 10 09:35:04 smithi102 podman[64879]: 2024-09-10 09:35:04.056253271 +0000 UTC m=+0.431153441 container init 0989d5e3d2ed54de7dd4a556306c46317db8753e6660efdb0f601ba19a420317 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mds-h, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-10T09:35:04.384 INFO:journalctl@ceph.mds.h.smithi102.stdout:Sep 10 09:35:04 smithi102 podman[64879]: 2024-09-10 09:35:04.058983944 +0000 UTC m=+0.433884106 container start 0989d5e3d2ed54de7dd4a556306c46317db8753e6660efdb0f601ba19a420317 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mds-h, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-09-10T09:35:04.384 INFO:journalctl@ceph.mds.h.smithi102.stdout:Sep 10 09:35:04 smithi102 ceph-mds[64899]: -- 172.21.15.102:0/1771656866 <== mon.2 v2:172.21.15.102:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x555dbaf61860 con 0x555dbaf3f000 2024-09-10T09:35:04.384 INFO:journalctl@ceph.mds.h.smithi102.stdout:Sep 10 09:35:04 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mds-h[64895]: starting mds.h at 2024-09-10T09:35:04.384 INFO:journalctl@ceph.mds.h.smithi102.stdout:Sep 10 09:35:04 smithi102 bash[64879]: 0989d5e3d2ed54de7dd4a556306c46317db8753e6660efdb0f601ba19a420317 2024-09-10T09:35:04.385 INFO:journalctl@ceph.mds.h.smithi102.stdout:Sep 10 09:35:04 smithi102 systemd[1]: Started Ceph mds.h for bade4b50-6f56-11ef-bcea-c7b262605968. 2024-09-10T09:35:05.119 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T09:35:05.511 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:35:05.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:35:05 smithi005 ceph-mon[30350]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T09:35:05.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:35:05 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:35:05.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:35:05 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:35:05.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:35:05 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:35:05.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:35:05 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T09:35:05.557 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:35:05 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T09:35:05.557 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:35:05 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:35:05.557 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:35:05 smithi005 ceph-mon[30350]: Deploying daemon mds.i on smithi159 2024-09-10T09:35:05.557 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:35:05 smithi005 ceph-mon[30350]: mds.? [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] up:boot 2024-09-10T09:35:05.557 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:35:05 smithi005 ceph-mon[30350]: fsmap 8 up:standby 2024-09-10T09:35:05.557 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:35:05 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-10T09:35:05.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:35:05 smithi159 ceph-mon[35513]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T09:35:05.601 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:35:05 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:35:05.601 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:35:05 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:35:05.601 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:35:05 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:35:05.601 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:35:05 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T09:35:05.601 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:35:05 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T09:35:05.601 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:35:05 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:35:05.601 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:35:05 smithi159 ceph-mon[35513]: Deploying daemon mds.i on smithi159 2024-09-10T09:35:05.601 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:35:05 smithi159 ceph-mon[35513]: mds.? [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] up:boot 2024-09-10T09:35:05.601 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:35:05 smithi159 ceph-mon[35513]: fsmap 8 up:standby 2024-09-10T09:35:05.601 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:35:05 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-10T09:35:05.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:35:05 smithi102 ceph-mon[35442]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T09:35:05.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:35:05 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:35:05.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:35:05 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:35:05.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:35:05 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:35:05.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:35:05 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T09:35:05.635 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:35:05 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T09:35:05.635 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:35:05 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:35:05.635 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:35:05 smithi102 ceph-mon[35442]: Deploying daemon mds.i on smithi159 2024-09-10T09:35:05.635 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:35:05 smithi102 ceph-mon[35442]: mds.? [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] up:boot 2024-09-10T09:35:05.635 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:35:05 smithi102 ceph-mon[35442]: fsmap 8 up:standby 2024-09-10T09:35:05.635 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:35:05 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-10T09:35:06.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:06.069+0000 7fba2c200640 1 -- 172.21.15.5:0/26246273 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fba24074eb0 msgr2=0x7fba24112650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:35:06.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:06.069+0000 7fba2c200640 1 --2- 172.21.15.5:0/26246273 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fba24074eb0 0x7fba24112650 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fba10009860 tx=0x7fba1002a3d0 comp rx=0 tx=0).stop 2024-09-10T09:35:06.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:06.069+0000 7fba2c200640 1 -- 172.21.15.5:0/26246273 shutdown_connections 2024-09-10T09:35:06.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:06.069+0000 7fba2c200640 1 --2- 172.21.15.5:0/26246273 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fba24112b90 0x7fba24114f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:35:06.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:06.069+0000 7fba2c200640 1 --2- 172.21.15.5:0/26246273 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fba24074eb0 0x7fba24112650 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:35:06.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:06.069+0000 7fba2c200640 1 --2- 172.21.15.5:0/26246273 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fba24074590 0x7fba24074970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:35:06.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:06.069+0000 7fba2c200640 1 -- 172.21.15.5:0/26246273 >> 172.21.15.5:0/26246273 conn(0x7fba240fd8c0 msgr2=0x7fba240ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:35:06.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:06.069+0000 7fba2c200640 1 -- 172.21.15.5:0/26246273 shutdown_connections 2024-09-10T09:35:06.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:06.069+0000 7fba2c200640 1 -- 172.21.15.5:0/26246273 wait complete. 2024-09-10T09:35:06.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:06.070+0000 7fba2c200640 1 Processor -- start 2024-09-10T09:35:06.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:06.070+0000 7fba2c200640 1 -- start start 2024-09-10T09:35:06.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:06.070+0000 7fba2c200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fba24074590 0x7fba241a4700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:35:06.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:06.070+0000 7fba2c200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fba24074eb0 0x7fba241a4c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:35:06.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:06.070+0000 7fba2c200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fba24112b90 0x7fba2419e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:35:06.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:06.070+0000 7fba2c200640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fba24118590 con 0x7fba24074eb0 2024-09-10T09:35:06.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:06.070+0000 7fba2c200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fba24118410 con 0x7fba24112b90 2024-09-10T09:35:06.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:06.070+0000 7fba2c200640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fba24118710 con 0x7fba24074590 2024-09-10T09:35:06.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:06.070+0000 7fba29800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fba24074590 0x7fba241a4700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:35:06.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:06.070+0000 7fba29800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fba24074590 0x7fba241a4700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:45586/0 (socket says 172.21.15.5:45586) 2024-09-10T09:35:06.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:06.070+0000 7fba28e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fba24074eb0 0x7fba241a4c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:35:06.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:06.070+0000 7fba29800640 1 -- 172.21.15.5:0/3792747973 learned_addr learned my addr 172.21.15.5:0/3792747973 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:35:06.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:06.071+0000 7fba2a200640 1 --2- 172.21.15.5:0/3792747973 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fba24112b90 0x7fba2419e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:35:06.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:06.071+0000 7fba29800640 1 -- 172.21.15.5:0/3792747973 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fba24112b90 msgr2=0x7fba2419e8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:35:06.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:06.071+0000 7fba29800640 1 --2- 172.21.15.5:0/3792747973 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fba24112b90 0x7fba2419e8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:35:06.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:06.071+0000 7fba29800640 1 -- 172.21.15.5:0/3792747973 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fba24074eb0 msgr2=0x7fba241a4c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:35:06.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:06.071+0000 7fba29800640 1 --2- 172.21.15.5:0/3792747973 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fba24074eb0 0x7fba241a4c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:35:06.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:06.071+0000 7fba29800640 1 -- 172.21.15.5:0/3792747973 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba2419eff0 con 0x7fba24074590 2024-09-10T09:35:06.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:06.071+0000 7fba29800640 1 --2- 172.21.15.5:0/3792747973 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fba24074590 0x7fba241a4700 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fba1800e9c0 tx=0x7fba1800ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:35:06.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:06.071+0000 7fba22000640 1 -- 172.21.15.5:0/3792747973 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba1800ccc0 con 0x7fba24074590 2024-09-10T09:35:06.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:06.071+0000 7fba22000640 1 -- 172.21.15.5:0/3792747973 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fba18005500 con 0x7fba24074590 2024-09-10T09:35:06.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:06.071+0000 7fba2c200640 1 -- 172.21.15.5:0/3792747973 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba2419f2e0 con 0x7fba24074590 2024-09-10T09:35:06.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:06.071+0000 7fba22000640 1 -- 172.21.15.5:0/3792747973 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba18010430 con 0x7fba24074590 2024-09-10T09:35:06.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:06.071+0000 7fba2c200640 1 -- 172.21.15.5:0/3792747973 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba241ab5d0 con 0x7fba24074590 2024-09-10T09:35:06.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:06.072+0000 7fba2c200640 1 -- 172.21.15.5:0/3792747973 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fba24074970 con 0x7fba24074590 2024-09-10T09:35:06.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:06.074+0000 7fba22000640 1 -- 172.21.15.5:0/3792747973 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7fba180046c0 con 0x7fba24074590 2024-09-10T09:35:06.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:06.076+0000 7fba22000640 1 --2- 172.21.15.5:0/3792747973 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fba0c0a2c00 0x7fba0c0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:35:06.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:06.076+0000 7fba22000640 1 -- 172.21.15.5:0/3792747973 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7127+0+0 (secure 0 0 0) 0x7fba180cef40 con 0x7fba24074590 2024-09-10T09:35:06.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:06.076+0000 7fba28e00640 1 --2- 172.21.15.5:0/3792747973 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fba0c0a2c00 0x7fba0c0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:35:06.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:06.076+0000 7fba28e00640 1 --2- 172.21.15.5:0/3792747973 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fba0c0a2c00 0x7fba0c0a50c0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fba241a00b0 tx=0x7fba10005c90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:35:06.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:06.079+0000 7fba22000640 1 -- 172.21.15.5:0/3792747973 <== mon.1 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fba1809c2a0 con 0x7fba24074590 2024-09-10T09:35:06.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:06.304+0000 7fba2c200640 1 -- 172.21.15.5:0/3792747973 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fba2419fdf0 con 0x7fba24074590 2024-09-10T09:35:06.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:06.305+0000 7fba22000640 1 -- 172.21.15.5:0/3792747973 <== mon.1 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+7213 (secure 0 0 0) 0x7fba1809bba0 con 0x7fba24074590 2024-09-10T09:35:06.307 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-10T09:35:06.307 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":9,"btime":"2024-09-10T09:35:04:499679+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14454,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/2970594311","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2970594311},{"type":"v1","addr":"172.21.15.5:6835","nonce":2970594311}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6833/2457007427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":2457007427},{"type":"v1","addr":"172.21.15.102:6833","nonce":2457007427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24460,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6833/356445159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":356445159},{"type":"v1","addr":"172.21.15.159:6833","nonce":356445159}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24467,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/2894314507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":2894314507},{"type":"v1","addr":"172.21.15.5:6837","nonce":2894314507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24479,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6835/1647549111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":1647549111},{"type":"v1","addr":"172.21.15.102:6835","nonce":1647549111}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24505,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6835/952657242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":952657242},{"type":"v1","addr":"172.21.15.159:6835","nonce":952657242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24506,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/2689776764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":2689776764},{"type":"v1","addr":"172.21.15.102:6837","nonce":2689776764}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24511,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6837/620627639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":620627639},{"type":"v1","addr":"172.21.15.159:6837","nonce":620627639}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-09-10T09:35:06.307 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 9 2024-09-10T09:35:06.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:06.307+0000 7fba2c200640 1 -- 172.21.15.5:0/3792747973 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fba0c0a2c00 msgr2=0x7fba0c0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:35:06.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:06.307+0000 7fba2c200640 1 --2- 172.21.15.5:0/3792747973 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fba0c0a2c00 0x7fba0c0a50c0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fba241a00b0 tx=0x7fba10005c90 comp rx=0 tx=0).stop 2024-09-10T09:35:06.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:06.307+0000 7fba2c200640 1 -- 172.21.15.5:0/3792747973 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fba24074590 msgr2=0x7fba241a4700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:35:06.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:06.307+0000 7fba2c200640 1 --2- 172.21.15.5:0/3792747973 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fba24074590 0x7fba241a4700 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fba1800e9c0 tx=0x7fba1800ee80 comp rx=0 tx=0).stop 2024-09-10T09:35:06.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:06.308+0000 7fba2c200640 1 -- 172.21.15.5:0/3792747973 shutdown_connections 2024-09-10T09:35:06.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:06.308+0000 7fba2c200640 1 --2- 172.21.15.5:0/3792747973 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fba0c0a2c00 0x7fba0c0a50c0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:35:06.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:06.308+0000 7fba2c200640 1 --2- 172.21.15.5:0/3792747973 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fba24112b90 0x7fba2419e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:35:06.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:06.308+0000 7fba2c200640 1 --2- 172.21.15.5:0/3792747973 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fba24074eb0 0x7fba241a4c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:35:06.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:06.308+0000 7fba2c200640 1 --2- 172.21.15.5:0/3792747973 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fba24074590 0x7fba241a4700 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:35:06.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:06.308+0000 7fba2c200640 1 -- 172.21.15.5:0/3792747973 >> 172.21.15.5:0/3792747973 conn(0x7fba240fd8c0 msgr2=0x7fba240ff020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:35:06.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:06.308+0000 7fba2c200640 1 -- 172.21.15.5:0/3792747973 shutdown_connections 2024-09-10T09:35:06.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:06.308+0000 7fba2c200640 1 -- 172.21.15.5:0/3792747973 wait complete. 2024-09-10T09:35:06.538 INFO:journalctl@ceph.mds.i.smithi159.stdout:Sep 10 09:35:06 smithi159 systemd[1]: Starting Ceph mds.i for bade4b50-6f56-11ef-bcea-c7b262605968... 2024-09-10T09:35:06.824 INFO:journalctl@ceph.mds.i.smithi159.stdout:Sep 10 09:35:06 smithi159 podman[65562]: 2024-09-10 09:35:06.440518548 +0000 UTC m=+0.016443512 image pull 41d67112835771c3efca60c6b8fe297c7ddaecdc26cea26fd28aa18636e49005 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 2024-09-10T09:35:06.824 INFO:journalctl@ceph.mds.i.smithi159.stdout:Sep 10 09:35:06 smithi159 podman[65562]: 2024-09-10 09:35:06.56059327 +0000 UTC m=+0.136518224 container create dfe08e65daed26646feb9f6ede92ac54dda8f31d601c8c9167bd3ae02807dde0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mds-i, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2) 2024-09-10T09:35:06.824 INFO:journalctl@ceph.mds.i.smithi159.stdout:Sep 10 09:35:06 smithi159 podman[65562]: 2024-09-10 09:35:06.823478715 +0000 UTC m=+0.399403680 container init dfe08e65daed26646feb9f6ede92ac54dda8f31d601c8c9167bd3ae02807dde0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mds-i, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, ceph=True, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2) 2024-09-10T09:35:07.100 INFO:journalctl@ceph.mds.i.smithi159.stdout:Sep 10 09:35:06 smithi159 podman[65562]: 2024-09-10 09:35:06.826359058 +0000 UTC m=+0.402284010 container start dfe08e65daed26646feb9f6ede92ac54dda8f31d601c8c9167bd3ae02807dde0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mds-i, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731) 2024-09-10T09:35:07.100 INFO:journalctl@ceph.mds.i.smithi159.stdout:Sep 10 09:35:06 smithi159 ceph-mds[65581]: -- 172.21.15.159:0/2802983658 <== mon.1 v2:172.21.15.159:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55780b7a1860 con 0x55780b77f400 2024-09-10T09:35:07.100 INFO:journalctl@ceph.mds.i.smithi159.stdout:Sep 10 09:35:06 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mds-i[65577]: starting mds.i at 2024-09-10T09:35:07.100 INFO:journalctl@ceph.mds.i.smithi159.stdout:Sep 10 09:35:06 smithi159 bash[65562]: dfe08e65daed26646feb9f6ede92ac54dda8f31d601c8c9167bd3ae02807dde0 2024-09-10T09:35:07.100 INFO:journalctl@ceph.mds.i.smithi159.stdout:Sep 10 09:35:06 smithi159 systemd[1]: Started Ceph mds.i for bade4b50-6f56-11ef-bcea-c7b262605968. 2024-09-10T09:35:07.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:35:07 smithi005 ceph-mon[30350]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T09:35:07.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:35:07 smithi005 ceph-mon[30350]: from='client.? 172.21.15.5:0/3792747973' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:35:07.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:35:07 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:35:07.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:35:07 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:35:07.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:35:07 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:35:07.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:35:07 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T09:35:07.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:35:07 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T09:35:07.557 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:35:07 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:35:07.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:35:07 smithi159 ceph-mon[35513]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T09:35:07.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:35:07 smithi159 ceph-mon[35513]: from='client.? 172.21.15.5:0/3792747973' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:35:07.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:35:07 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:35:07.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:35:07 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:35:07.601 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:35:07 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:35:07.601 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:35:07 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T09:35:07.601 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:35:07 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T09:35:07.601 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:35:07 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:35:07.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:35:07 smithi102 ceph-mon[35442]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T09:35:07.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:35:07 smithi102 ceph-mon[35442]: from='client.? 172.21.15.5:0/3792747973' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:35:07.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:35:07 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:35:07.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:35:07 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:35:07.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:35:07 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:35:07.635 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:35:07 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T09:35:07.635 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:35:07 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T09:35:07.635 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:35:07 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:35:08.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:35:08 smithi005 ceph-mon[30350]: Deploying daemon mds.g on smithi005 2024-09-10T09:35:08.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:35:08 smithi005 ceph-mon[30350]: mds.? [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] up:boot 2024-09-10T09:35:08.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:35:08 smithi005 ceph-mon[30350]: fsmap 9 up:standby 2024-09-10T09:35:08.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:35:08 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-10T09:35:08.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:35:08 smithi159 ceph-mon[35513]: Deploying daemon mds.g on smithi005 2024-09-10T09:35:08.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:35:08 smithi159 ceph-mon[35513]: mds.? [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] up:boot 2024-09-10T09:35:08.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:35:08 smithi159 ceph-mon[35513]: fsmap 9 up:standby 2024-09-10T09:35:08.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:35:08 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-10T09:35:08.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:35:08 smithi102 ceph-mon[35442]: Deploying daemon mds.g on smithi005 2024-09-10T09:35:08.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:35:08 smithi102 ceph-mon[35442]: mds.? [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] up:boot 2024-09-10T09:35:08.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:35:08 smithi102 ceph-mon[35442]: fsmap 9 up:standby 2024-09-10T09:35:08.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:35:08 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-10T09:35:08.993 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T09:35:09.277 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:35:09 smithi005 ceph-mon[30350]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T09:35:09.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:35:09 smithi159 ceph-mon[35513]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T09:35:09.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:35:09 smithi102 ceph-mon[35442]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T09:35:09.637 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:35:10.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:10.234+0000 7fb29e000640 1 -- 172.21.15.5:0/1836638815 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb2981049a0 msgr2=0x7fb2981089f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:35:10.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:10.234+0000 7fb29e000640 1 --2- 172.21.15.5:0/1836638815 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb2981049a0 0x7fb2981089f0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fb284009900 tx=0x7fb28402a3f0 comp rx=0 tx=0).stop 2024-09-10T09:35:10.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:10.234+0000 7fb29e000640 1 -- 172.21.15.5:0/1836638815 shutdown_connections 2024-09-10T09:35:10.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:10.234+0000 7fb29e000640 1 --2- 172.21.15.5:0/1836638815 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb298109120 0x7fb298110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:35:10.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:10.234+0000 7fb29e000640 1 --2- 172.21.15.5:0/1836638815 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb2981049a0 0x7fb2981089f0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:35:10.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:10.234+0000 7fb29e000640 1 --2- 172.21.15.5:0/1836638815 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb298103ff0 0x7fb2981043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:35:10.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:10.234+0000 7fb29e000640 1 -- 172.21.15.5:0/1836638815 >> 172.21.15.5:0/1836638815 conn(0x7fb2980ffa90 msgr2=0x7fb298101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:35:10.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:10.234+0000 7fb29e000640 1 -- 172.21.15.5:0/1836638815 shutdown_connections 2024-09-10T09:35:10.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:10.235+0000 7fb29e000640 1 -- 172.21.15.5:0/1836638815 wait complete. 2024-09-10T09:35:10.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:10.236+0000 7fb29e000640 1 Processor -- start 2024-09-10T09:35:10.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:10.236+0000 7fb29e000640 1 -- start start 2024-09-10T09:35:10.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:10.237+0000 7fb29e000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb298103ff0 0x7fb29819a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:35:10.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:10.237+0000 7fb29e000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb2981049a0 0x7fb29819ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:35:10.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:10.237+0000 7fb29e000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb298109120 0x7fb29819ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:35:10.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:10.237+0000 7fb29e000640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fb298114190 con 0x7fb298103ff0 2024-09-10T09:35:10.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:10.237+0000 7fb29e000640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fb298114010 con 0x7fb2981049a0 2024-09-10T09:35:10.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:10.237+0000 7fb29e000640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fb298114310 con 0x7fb298109120 2024-09-10T09:35:10.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:10.237+0000 7fb296a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb2981049a0 0x7fb29819ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:35:10.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:10.237+0000 7fb297400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb298103ff0 0x7fb29819a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:35:10.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:10.237+0000 7fb296a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb2981049a0 0x7fb29819ab90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:40052/0 (socket says 172.21.15.5:40052) 2024-09-10T09:35:10.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:10.237+0000 7fb296a00640 1 -- 172.21.15.5:0/2116354844 learned_addr learned my addr 172.21.15.5:0/2116354844 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:35:10.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:10.237+0000 7fb297e00640 1 --2- 172.21.15.5:0/2116354844 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb298109120 0x7fb29819ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:35:10.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:10.238+0000 7fb296a00640 1 -- 172.21.15.5:0/2116354844 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb298109120 msgr2=0x7fb29819ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:35:10.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:10.238+0000 7fb296a00640 1 --2- 172.21.15.5:0/2116354844 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb298109120 0x7fb29819ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:35:10.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:10.238+0000 7fb296a00640 1 -- 172.21.15.5:0/2116354844 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb298103ff0 msgr2=0x7fb29819a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:35:10.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:10.238+0000 7fb296a00640 1 --2- 172.21.15.5:0/2116354844 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb298103ff0 0x7fb29819a650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:35:10.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:10.238+0000 7fb296a00640 1 -- 172.21.15.5:0/2116354844 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb29819f600 con 0x7fb2981049a0 2024-09-10T09:35:10.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:10.238+0000 7fb296a00640 1 --2- 172.21.15.5:0/2116354844 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb2981049a0 0x7fb29819ab90 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fb284004770 tx=0x7fb284003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:35:10.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:10.238+0000 7fb28fe00640 1 -- 172.21.15.5:0/2116354844 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb28403f070 con 0x7fb2981049a0 2024-09-10T09:35:10.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:10.238+0000 7fb28fe00640 1 -- 172.21.15.5:0/2116354844 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb284031ca0 con 0x7fb2981049a0 2024-09-10T09:35:10.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:10.238+0000 7fb29e000640 1 -- 172.21.15.5:0/2116354844 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb29819f890 con 0x7fb2981049a0 2024-09-10T09:35:10.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:10.238+0000 7fb28fe00640 1 -- 172.21.15.5:0/2116354844 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb284035030 con 0x7fb2981049a0 2024-09-10T09:35:10.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:10.238+0000 7fb29e000640 1 -- 172.21.15.5:0/2116354844 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2981a71d0 con 0x7fb2981049a0 2024-09-10T09:35:10.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:10.240+0000 7fb29e000640 1 -- 172.21.15.5:0/2116354844 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb2981043d0 con 0x7fb2981049a0 2024-09-10T09:35:10.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:10.242+0000 7fb28fe00640 1 -- 172.21.15.5:0/2116354844 <== mon.2 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7fb284042070 con 0x7fb2981049a0 2024-09-10T09:35:10.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:10.244+0000 7fb28fe00640 1 --2- 172.21.15.5:0/2116354844 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fb27c0a2c00 0x7fb27c0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:35:10.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:10.244+0000 7fb28fe00640 1 -- 172.21.15.5:0/2116354844 <== mon.2 v2:172.21.15.102:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7127+0+0 (secure 0 0 0) 0x7fb2840ef190 con 0x7fb2981049a0 2024-09-10T09:35:10.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:10.244+0000 7fb297400640 1 --2- 172.21.15.5:0/2116354844 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fb27c0a2c00 0x7fb27c0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:35:10.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:10.245+0000 7fb297400640 1 --2- 172.21.15.5:0/2116354844 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fb27c0a2c00 0x7fb27c0a50c0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fb288009bd0 tx=0x7fb288005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:35:10.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:10.248+0000 7fb28fe00640 1 -- 172.21.15.5:0/2116354844 <== mon.2 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb2840bc4f0 con 0x7fb2981049a0 2024-09-10T09:35:10.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:10.484+0000 7fb29e000640 1 -- 172.21.15.5:0/2116354844 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb29810c360 con 0x7fb2981049a0 2024-09-10T09:35:10.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:10.485+0000 7fb28fe00640 1 -- 172.21.15.5:0/2116354844 <== mon.2 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) ==== 94+0+8043 (secure 0 0 0) 0x7fb2840bbdf0 con 0x7fb2981049a0 2024-09-10T09:35:10.487 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-10T09:35:10.488 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":10,"btime":"2024-09-10T09:35:07:216715+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14454,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/2970594311","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2970594311},{"type":"v1","addr":"172.21.15.5:6835","nonce":2970594311}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6833/2457007427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":2457007427},{"type":"v1","addr":"172.21.15.102:6833","nonce":2457007427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24460,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6833/356445159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":356445159},{"type":"v1","addr":"172.21.15.159:6833","nonce":356445159}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24467,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/2894314507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":2894314507},{"type":"v1","addr":"172.21.15.5:6837","nonce":2894314507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24479,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6835/1647549111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":1647549111},{"type":"v1","addr":"172.21.15.102:6835","nonce":1647549111}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24505,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6835/952657242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":952657242},{"type":"v1","addr":"172.21.15.159:6835","nonce":952657242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24506,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/2689776764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":2689776764},{"type":"v1","addr":"172.21.15.102:6837","nonce":2689776764}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24511,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6837/620627639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":620627639},{"type":"v1","addr":"172.21.15.159:6837","nonce":620627639}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24535,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6839/2073739599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":2073739599},{"type":"v1","addr":"172.21.15.159:6839","nonce":2073739599}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-09-10T09:35:10.488 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 10 2024-09-10T09:35:10.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:10.488+0000 7fb29e000640 1 -- 172.21.15.5:0/2116354844 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fb27c0a2c00 msgr2=0x7fb27c0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:35:10.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:10.488+0000 7fb29e000640 1 --2- 172.21.15.5:0/2116354844 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fb27c0a2c00 0x7fb27c0a50c0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fb288009bd0 tx=0x7fb288005f90 comp rx=0 tx=0).stop 2024-09-10T09:35:10.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:10.488+0000 7fb29e000640 1 -- 172.21.15.5:0/2116354844 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb2981049a0 msgr2=0x7fb29819ab90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:35:10.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:10.488+0000 7fb29e000640 1 --2- 172.21.15.5:0/2116354844 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb2981049a0 0x7fb29819ab90 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fb284004770 tx=0x7fb284003c10 comp rx=0 tx=0).stop 2024-09-10T09:35:10.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:10.488+0000 7fb29e000640 1 -- 172.21.15.5:0/2116354844 shutdown_connections 2024-09-10T09:35:10.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:10.488+0000 7fb29e000640 1 --2- 172.21.15.5:0/2116354844 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fb27c0a2c00 0x7fb27c0a50c0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:35:10.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:10.488+0000 7fb29e000640 1 --2- 172.21.15.5:0/2116354844 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb298109120 0x7fb29819ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:35:10.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:10.488+0000 7fb29e000640 1 --2- 172.21.15.5:0/2116354844 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb2981049a0 0x7fb29819ab90 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:35:10.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:10.488+0000 7fb29e000640 1 --2- 172.21.15.5:0/2116354844 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb298103ff0 0x7fb29819a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:35:10.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:10.488+0000 7fb29e000640 1 -- 172.21.15.5:0/2116354844 >> 172.21.15.5:0/2116354844 conn(0x7fb2980ffa90 msgr2=0x7fb298100d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:35:10.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:10.489+0000 7fb29e000640 1 -- 172.21.15.5:0/2116354844 shutdown_connections 2024-09-10T09:35:10.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:10.489+0000 7fb29e000640 1 -- 172.21.15.5:0/2116354844 wait complete. 2024-09-10T09:35:11.529 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:35:11 smithi005 ceph-mon[30350]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T09:35:11.529 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:35:11 smithi005 ceph-mon[30350]: from='client.? 172.21.15.5:0/2116354844' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:35:11.529 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:35:11 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:35:11.529 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:35:11 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:35:11.529 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:35:11 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:35:11.529 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:35:11 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T09:35:11.530 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:35:11 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T09:35:11.530 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:35:11 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:35:11.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:35:11 smithi159 ceph-mon[35513]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T09:35:11.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:35:11 smithi159 ceph-mon[35513]: from='client.? 172.21.15.5:0/2116354844' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:35:11.601 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:35:11 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:35:11.601 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:35:11 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:35:11.601 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:35:11 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:35:11.601 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:35:11 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T09:35:11.601 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:35:11 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T09:35:11.601 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:35:11 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:35:11.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:35:11 smithi102 ceph-mon[35442]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T09:35:11.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:35:11 smithi102 ceph-mon[35442]: from='client.? 172.21.15.5:0/2116354844' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:35:11.635 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:35:11 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:35:11.635 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:35:11 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:35:11.635 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:35:11 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:35:11.635 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:35:11 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T09:35:11.635 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:35:11 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T09:35:11.635 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:35:11 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:35:12.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:35:12 smithi005 ceph-mon[30350]: Deploying daemon mds.a on smithi005 2024-09-10T09:35:12.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:35:12 smithi005 ceph-mon[30350]: mds.? [v2:172.21.15.5:6838/3616502174,v1:172.21.15.5:6839/3616502174] up:boot 2024-09-10T09:35:12.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:35:12 smithi005 ceph-mon[30350]: fsmap 10 up:standby 2024-09-10T09:35:12.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:35:12 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-10T09:35:12.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:35:12 smithi159 ceph-mon[35513]: Deploying daemon mds.a on smithi005 2024-09-10T09:35:12.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:35:12 smithi159 ceph-mon[35513]: mds.? [v2:172.21.15.5:6838/3616502174,v1:172.21.15.5:6839/3616502174] up:boot 2024-09-10T09:35:12.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:35:12 smithi159 ceph-mon[35513]: fsmap 10 up:standby 2024-09-10T09:35:12.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:35:12 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-10T09:35:12.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:35:12 smithi102 ceph-mon[35442]: Deploying daemon mds.a on smithi005 2024-09-10T09:35:12.635 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:35:12 smithi102 ceph-mon[35442]: mds.? [v2:172.21.15.5:6838/3616502174,v1:172.21.15.5:6839/3616502174] up:boot 2024-09-10T09:35:12.635 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:35:12 smithi102 ceph-mon[35442]: fsmap 10 up:standby 2024-09-10T09:35:12.635 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:35:12 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-10T09:35:13.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:35:13 smithi005 ceph-mon[30350]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T09:35:13.568 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T09:35:13.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:35:13 smithi159 ceph-mon[35513]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T09:35:13.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:35:13 smithi102 ceph-mon[35442]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T09:35:14.001 INFO:journalctl@ceph.mds.a.smithi005.stdout:Sep 10 09:35:13 smithi005 systemd[1]: Starting Ceph mds.a for bade4b50-6f56-11ef-bcea-c7b262605968... 2024-09-10T09:35:14.268 INFO:journalctl@ceph.mds.a.smithi005.stdout:Sep 10 09:35:14 smithi005 podman[72181]: 2024-09-10 09:35:13.906024957 +0000 UTC m=+0.035412523 image pull 41d67112835771c3efca60c6b8fe297c7ddaecdc26cea26fd28aa18636e49005 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 2024-09-10T09:35:14.268 INFO:journalctl@ceph.mds.a.smithi005.stdout:Sep 10 09:35:14 smithi005 podman[72181]: 2024-09-10 09:35:14.039676607 +0000 UTC m=+0.169064179 container create 0d820c5c04148b88bbaffdf4547e35e1f21930e21f65207af0c8df24b76b32da (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mds-a, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-09-10T09:35:14.268 INFO:journalctl@ceph.mds.a.smithi005.stdout:Sep 10 09:35:14 smithi005 podman[72181]: 2024-09-10 09:35:14.267663154 +0000 UTC m=+0.397050719 container init 0d820c5c04148b88bbaffdf4547e35e1f21930e21f65207af0c8df24b76b32da (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mds-a, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T09:35:14.401 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:35:14.549 INFO:journalctl@ceph.mds.a.smithi005.stdout:Sep 10 09:35:14 smithi005 podman[72181]: 2024-09-10 09:35:14.274510995 +0000 UTC m=+0.403898564 container start 0d820c5c04148b88bbaffdf4547e35e1f21930e21f65207af0c8df24b76b32da (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mds-a, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T09:35:14.549 INFO:journalctl@ceph.mds.a.smithi005.stdout:Sep 10 09:35:14 smithi005 ceph-mds[72224]: -- 172.21.15.5:0/2919243622 <== mon.0 v2:172.21.15.5:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55e16fdab860 con 0x55e16fd88800 2024-09-10T09:35:14.549 INFO:journalctl@ceph.mds.a.smithi005.stdout:Sep 10 09:35:14 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mds-a[72209]: starting mds.a at 2024-09-10T09:35:14.549 INFO:journalctl@ceph.mds.a.smithi005.stdout:Sep 10 09:35:14 smithi005 bash[72181]: 0d820c5c04148b88bbaffdf4547e35e1f21930e21f65207af0c8df24b76b32da 2024-09-10T09:35:14.549 INFO:journalctl@ceph.mds.a.smithi005.stdout:Sep 10 09:35:14 smithi005 systemd[1]: Started Ceph mds.a for bade4b50-6f56-11ef-bcea-c7b262605968. 2024-09-10T09:35:15.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:15.062+0000 7f8602000640 1 -- 172.21.15.5:0/1518269945 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f85fc076720 msgr2=0x7f85fc076b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:35:15.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:15.062+0000 7f8602000640 1 --2- 172.21.15.5:0/1518269945 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f85fc076720 0x7f85fc076b00 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f85ec009900 tx=0x7f85ec02a3f0 comp rx=0 tx=0).stop 2024-09-10T09:35:15.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:15.062+0000 7f8602000640 1 -- 172.21.15.5:0/1518269945 shutdown_connections 2024-09-10T09:35:15.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:15.062+0000 7f8602000640 1 --2- 172.21.15.5:0/1518269945 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f85fc112a30 0x7f85fc114e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:35:15.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:15.062+0000 7f8602000640 1 --2- 172.21.15.5:0/1518269945 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f85fc077040 0x7f85fc1124f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:35:15.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:15.062+0000 7f8602000640 1 --2- 172.21.15.5:0/1518269945 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f85fc076720 0x7f85fc076b00 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:35:15.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:15.062+0000 7f8602000640 1 -- 172.21.15.5:0/1518269945 >> 172.21.15.5:0/1518269945 conn(0x7f85fc0ffa90 msgr2=0x7f85fc101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:35:15.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:15.062+0000 7f8602000640 1 -- 172.21.15.5:0/1518269945 shutdown_connections 2024-09-10T09:35:15.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:15.062+0000 7f8602000640 1 -- 172.21.15.5:0/1518269945 wait complete. 2024-09-10T09:35:15.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:15.063+0000 7f8602000640 1 Processor -- start 2024-09-10T09:35:15.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:15.063+0000 7f8602000640 1 -- start start 2024-09-10T09:35:15.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:15.063+0000 7f8602000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f85fc076720 0x7f85fc1a4720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:35:15.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:15.063+0000 7f8602000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f85fc077040 0x7f85fc1a4c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:35:15.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:15.063+0000 7f8602000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f85fc112a30 0x7f85fc19e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:35:15.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:15.063+0000 7f8602000640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f85fc1185c0 con 0x7f85fc112a30 2024-09-10T09:35:15.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:15.063+0000 7f8602000640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f85fc118440 con 0x7f85fc077040 2024-09-10T09:35:15.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:15.063+0000 7f8602000640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f85fc118740 con 0x7f85fc076720 2024-09-10T09:35:15.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:15.063+0000 7f85f3e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f85fc077040 0x7f85fc1a4c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:35:15.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:15.063+0000 7f85f3e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f85fc077040 0x7f85fc1a4c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:50448/0 (socket says 172.21.15.5:50448) 2024-09-10T09:35:15.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:15.063+0000 7f85f3e00640 1 -- 172.21.15.5:0/382341499 learned_addr learned my addr 172.21.15.5:0/382341499 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:35:15.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:15.063+0000 7f85fbe00640 1 --2- 172.21.15.5:0/382341499 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f85fc112a30 0x7f85fc19e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:35:15.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:15.063+0000 7f85fb400640 1 --2- 172.21.15.5:0/382341499 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f85fc076720 0x7f85fc1a4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:35:15.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:15.064+0000 7f85f3e00640 1 -- 172.21.15.5:0/382341499 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f85fc076720 msgr2=0x7f85fc1a4720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:35:15.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:15.064+0000 7f85f3e00640 1 --2- 172.21.15.5:0/382341499 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f85fc076720 0x7f85fc1a4720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:35:15.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:15.064+0000 7f85f3e00640 1 -- 172.21.15.5:0/382341499 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f85fc112a30 msgr2=0x7f85fc19e8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:35:15.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:15.064+0000 7f85f3e00640 1 --2- 172.21.15.5:0/382341499 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f85fc112a30 0x7f85fc19e8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:35:15.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:15.064+0000 7f85f3e00640 1 -- 172.21.15.5:0/382341499 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85fc19f010 con 0x7f85fc077040 2024-09-10T09:35:15.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:15.064+0000 7f85f3e00640 1 --2- 172.21.15.5:0/382341499 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f85fc077040 0x7f85fc1a4c60 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f85e400e9f0 tx=0x7f85e400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:35:15.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:15.064+0000 7f85f8c00640 1 -- 172.21.15.5:0/382341499 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85e400cd80 con 0x7f85fc077040 2024-09-10T09:35:15.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:15.065+0000 7f85f8c00640 1 -- 172.21.15.5:0/382341499 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f85e4010910 con 0x7f85fc077040 2024-09-10T09:35:15.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:15.065+0000 7f8602000640 1 -- 172.21.15.5:0/382341499 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85fc19f300 con 0x7f85fc077040 2024-09-10T09:35:15.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:15.065+0000 7f85f8c00640 1 -- 172.21.15.5:0/382341499 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85e4018990 con 0x7f85fc077040 2024-09-10T09:35:15.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:15.065+0000 7f8602000640 1 -- 172.21.15.5:0/382341499 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85fc1ab5f0 con 0x7f85fc077040 2024-09-10T09:35:15.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:15.066+0000 7f8602000640 1 -- 172.21.15.5:0/382341499 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f85fc076b00 con 0x7f85fc077040 2024-09-10T09:35:15.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:15.068+0000 7f85f8c00640 1 -- 172.21.15.5:0/382341499 <== mon.2 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7f85e4010ac0 con 0x7f85fc077040 2024-09-10T09:35:15.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:15.069+0000 7f85f8c00640 1 --2- 172.21.15.5:0/382341499 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f85e00a2b80 0x7f85e00a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:35:15.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:15.070+0000 7f85f8c00640 1 -- 172.21.15.5:0/382341499 <== mon.2 v2:172.21.15.102:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7127+0+0 (secure 0 0 0) 0x7f85e40cf4f0 con 0x7f85fc077040 2024-09-10T09:35:15.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:15.070+0000 7f85fb400640 1 --2- 172.21.15.5:0/382341499 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f85e00a2b80 0x7f85e00a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:35:15.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:15.070+0000 7f85fb400640 1 --2- 172.21.15.5:0/382341499 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f85e00a2b80 0x7f85e00a5040 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f85fc19fa20 tx=0x7f85ec0058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:35:15.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:15.071+0000 7f85f8c00640 1 -- 172.21.15.5:0/382341499 <== mon.2 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f85e409c200 con 0x7f85fc077040 2024-09-10T09:35:15.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:15.294+0000 7f8602000640 1 -- 172.21.15.5:0/382341499 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f85fc02d7c0 con 0x7f85fc077040 2024-09-10T09:35:15.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:15.295+0000 7f85f8c00640 1 -- 172.21.15.5:0/382341499 <== mon.2 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) ==== 94+0+9689 (secure 0 0 0) 0x7f85e40a0500 con 0x7f85fc077040 2024-09-10T09:35:15.297 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-10T09:35:15.297 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":12,"btime":"2024-09-10T09:35:15:246589+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14454,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/2970594311","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2970594311},{"type":"v1","addr":"172.21.15.5:6835","nonce":2970594311}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14514,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6839/3616502174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":3616502174},{"type":"v1","addr":"172.21.15.5:6839","nonce":3616502174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14520,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6841/2073599842","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":2073599842},{"type":"v1","addr":"172.21.15.5:6841","nonce":2073599842}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6833/2457007427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":2457007427},{"type":"v1","addr":"172.21.15.102:6833","nonce":2457007427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24460,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6833/356445159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":356445159},{"type":"v1","addr":"172.21.15.159:6833","nonce":356445159}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24467,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/2894314507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":2894314507},{"type":"v1","addr":"172.21.15.5:6837","nonce":2894314507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24479,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6835/1647549111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":1647549111},{"type":"v1","addr":"172.21.15.102:6835","nonce":1647549111}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24505,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6835/952657242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":952657242},{"type":"v1","addr":"172.21.15.159:6835","nonce":952657242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24506,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/2689776764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":2689776764},{"type":"v1","addr":"172.21.15.102:6837","nonce":2689776764}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24511,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6837/620627639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":620627639},{"type":"v1","addr":"172.21.15.159:6837","nonce":620627639}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24535,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6839/2073739599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":2073739599},{"type":"v1","addr":"172.21.15.159:6839","nonce":2073739599}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-09-10T09:35:15.297 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 12 2024-09-10T09:35:15.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:15.298+0000 7f8602000640 1 -- 172.21.15.5:0/382341499 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f85e00a2b80 msgr2=0x7f85e00a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:35:15.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:15.298+0000 7f8602000640 1 --2- 172.21.15.5:0/382341499 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f85e00a2b80 0x7f85e00a5040 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f85fc19fa20 tx=0x7f85ec0058d0 comp rx=0 tx=0).stop 2024-09-10T09:35:15.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:15.298+0000 7f8602000640 1 -- 172.21.15.5:0/382341499 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f85fc077040 msgr2=0x7f85fc1a4c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:35:15.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:15.298+0000 7f8602000640 1 --2- 172.21.15.5:0/382341499 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f85fc077040 0x7f85fc1a4c60 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f85e400e9f0 tx=0x7f85e400eeb0 comp rx=0 tx=0).stop 2024-09-10T09:35:15.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:15.298+0000 7f8602000640 1 -- 172.21.15.5:0/382341499 shutdown_connections 2024-09-10T09:35:15.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:15.298+0000 7f8602000640 1 --2- 172.21.15.5:0/382341499 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f85e00a2b80 0x7f85e00a5040 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:35:15.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:15.298+0000 7f8602000640 1 --2- 172.21.15.5:0/382341499 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f85fc112a30 0x7f85fc19e8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:35:15.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:15.298+0000 7f8602000640 1 --2- 172.21.15.5:0/382341499 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f85fc077040 0x7f85fc1a4c60 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:35:15.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:15.298+0000 7f8602000640 1 --2- 172.21.15.5:0/382341499 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f85fc076720 0x7f85fc1a4720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:35:15.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:15.298+0000 7f8602000640 1 -- 172.21.15.5:0/382341499 >> 172.21.15.5:0/382341499 conn(0x7f85fc0ffa90 msgr2=0x7f85fc101e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:35:15.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:15.299+0000 7f8602000640 1 -- 172.21.15.5:0/382341499 shutdown_connections 2024-09-10T09:35:15.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:15.299+0000 7f8602000640 1 -- 172.21.15.5:0/382341499 wait complete. 2024-09-10T09:35:15.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:35:15 smithi005 ceph-mon[30350]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T09:35:15.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:35:15 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:35:15.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:35:15 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:35:15.557 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:35:15 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:35:15.557 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:35:15 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T09:35:15.557 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:35:15 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T09:35:15.557 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:35:15 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:35:15.557 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:35:15 smithi005 ceph-mon[30350]: Deploying daemon mds.b on smithi102 2024-09-10T09:35:15.557 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:35:15 smithi005 ceph-mon[30350]: mds.? [v2:172.21.15.5:6840/2073599842,v1:172.21.15.5:6841/2073599842] up:boot 2024-09-10T09:35:15.557 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:35:15 smithi005 ceph-mon[30350]: fsmap 11 up:standby 2024-09-10T09:35:15.557 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:35:15 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-10T09:35:15.557 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:35:15 smithi005 ceph-mon[30350]: from='client.? 172.21.15.5:0/382341499' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:35:15.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:35:15 smithi102 ceph-mon[35442]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T09:35:15.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:35:15 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:35:15.635 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:35:15 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:35:15.635 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:35:15 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:35:15.635 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:35:15 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T09:35:15.635 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:35:15 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T09:35:15.635 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:35:15 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:35:15.635 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:35:15 smithi102 ceph-mon[35442]: Deploying daemon mds.b on smithi102 2024-09-10T09:35:15.635 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:35:15 smithi102 ceph-mon[35442]: mds.? [v2:172.21.15.5:6840/2073599842,v1:172.21.15.5:6841/2073599842] up:boot 2024-09-10T09:35:15.635 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:35:15 smithi102 ceph-mon[35442]: fsmap 11 up:standby 2024-09-10T09:35:15.635 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:35:15 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-10T09:35:15.635 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:35:15 smithi102 ceph-mon[35442]: from='client.? 172.21.15.5:0/382341499' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:35:15.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:35:15 smithi159 ceph-mon[35513]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T09:35:15.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:35:15 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:35:15.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:35:15 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:35:15.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:35:15 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:35:15.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:35:15 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T09:35:15.851 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:35:15 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T09:35:15.851 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:35:15 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:35:15.851 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:35:15 smithi159 ceph-mon[35513]: Deploying daemon mds.b on smithi102 2024-09-10T09:35:15.851 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:35:15 smithi159 ceph-mon[35513]: mds.? [v2:172.21.15.5:6840/2073599842,v1:172.21.15.5:6841/2073599842] up:boot 2024-09-10T09:35:15.851 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:35:15 smithi159 ceph-mon[35513]: fsmap 11 up:standby 2024-09-10T09:35:15.851 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:35:15 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-10T09:35:15.851 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:35:15 smithi159 ceph-mon[35513]: from='client.? 172.21.15.5:0/382341499' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:35:16.635 INFO:journalctl@ceph.mds.b.smithi102.stdout:Sep 10 09:35:16 smithi102 systemd[1]: Starting Ceph mds.b for bade4b50-6f56-11ef-bcea-c7b262605968... 2024-09-10T09:35:16.920 INFO:journalctl@ceph.mds.b.smithi102.stdout:Sep 10 09:35:16 smithi102 podman[65182]: 2024-09-10 09:35:16.559873798 +0000 UTC m=+0.015719348 image pull 41d67112835771c3efca60c6b8fe297c7ddaecdc26cea26fd28aa18636e49005 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 2024-09-10T09:35:16.921 INFO:journalctl@ceph.mds.b.smithi102.stdout:Sep 10 09:35:16 smithi102 podman[65182]: 2024-09-10 09:35:16.6759384 +0000 UTC m=+0.131783943 container create 1283ac49c22fa17d6f39a54224510397b8507457965833fa2e2526ed63d98d70 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mds-b, io.buildah.version=1.37.2, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-09-10T09:35:16.921 INFO:journalctl@ceph.mds.b.smithi102.stdout:Sep 10 09:35:16 smithi102 podman[65182]: 2024-09-10 09:35:16.920387397 +0000 UTC m=+0.376232940 container init 1283ac49c22fa17d6f39a54224510397b8507457965833fa2e2526ed63d98d70 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mds-b, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T09:35:17.384 INFO:journalctl@ceph.mds.b.smithi102.stdout:Sep 10 09:35:16 smithi102 podman[65182]: 2024-09-10 09:35:16.923415172 +0000 UTC m=+0.379260716 container start 1283ac49c22fa17d6f39a54224510397b8507457965833fa2e2526ed63d98d70 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mds-b, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS) 2024-09-10T09:35:17.384 INFO:journalctl@ceph.mds.b.smithi102.stdout:Sep 10 09:35:16 smithi102 ceph-mds[65202]: -- 172.21.15.102:0/1371785094 <== mon.2 v2:172.21.15.102:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55e81585b860 con 0x55e815839000 2024-09-10T09:35:17.385 INFO:journalctl@ceph.mds.b.smithi102.stdout:Sep 10 09:35:16 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mds-b[65198]: starting mds.b at 2024-09-10T09:35:17.385 INFO:journalctl@ceph.mds.b.smithi102.stdout:Sep 10 09:35:17 smithi102 bash[65182]: 1283ac49c22fa17d6f39a54224510397b8507457965833fa2e2526ed63d98d70 2024-09-10T09:35:17.385 INFO:journalctl@ceph.mds.b.smithi102.stdout:Sep 10 09:35:17 smithi102 systemd[1]: Started Ceph mds.b for bade4b50-6f56-11ef-bcea-c7b262605968. 2024-09-10T09:35:17.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:35:17 smithi005 ceph-mon[30350]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T09:35:17.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:35:17 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:35:17.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:35:17 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:35:17.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:35:17 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:35:17.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:35:17 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:35:17.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:35:17 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:35:17.807 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:35:17 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:35:17.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:35:17 smithi159 ceph-mon[35513]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T09:35:17.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:35:17 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:35:17.851 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:35:17 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:35:17.851 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:35:17 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:35:17.851 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:35:17 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:35:17.851 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:35:17 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:35:17.851 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:35:17 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:35:17.868 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:35:17 smithi102 ceph-mon[35442]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T09:35:17.868 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:35:17 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:35:17.868 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:35:17 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:35:17.868 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:35:17 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:35:17.868 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:35:17 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:35:17.868 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:35:17 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:35:17.868 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:35:17 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:35:18.078 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T09:35:18.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:35:18 smithi102 ceph-mon[35442]: mds.? [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] up:boot 2024-09-10T09:35:18.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:35:18 smithi102 ceph-mon[35442]: fsmap 12 up:standby 2024-09-10T09:35:18.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:35:18 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-10T09:35:18.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:35:18 smithi005 ceph-mon[30350]: mds.? [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] up:boot 2024-09-10T09:35:18.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:35:18 smithi005 ceph-mon[30350]: fsmap 12 up:standby 2024-09-10T09:35:18.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:35:18 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-10T09:35:18.817 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:35:18.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:35:18 smithi159 ceph-mon[35513]: mds.? [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] up:boot 2024-09-10T09:35:18.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:35:18 smithi159 ceph-mon[35513]: fsmap 12 up:standby 2024-09-10T09:35:18.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:35:18 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-10T09:35:19.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:19.390+0000 7ff8fca00640 1 -- 172.21.15.5:0/3976794558 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff8f8106f20 msgr2=0x7ff8f810eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:35:19.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:19.390+0000 7ff8fca00640 1 --2- 172.21.15.5:0/3976794558 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff8f8106f20 0x7ff8f810eaa0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7ff8f0009ca0 tx=0x7ff8f002a8c0 comp rx=0 tx=0).stop 2024-09-10T09:35:19.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:19.390+0000 7ff8fca00640 1 -- 172.21.15.5:0/3976794558 shutdown_connections 2024-09-10T09:35:19.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:19.390+0000 7ff8fca00640 1 --2- 172.21.15.5:0/3976794558 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff8f8106f20 0x7ff8f810eaa0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:35:19.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:19.390+0000 7ff8fca00640 1 --2- 172.21.15.5:0/3976794558 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff8f81027a0 0x7ff8f81067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:35:19.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:19.390+0000 7ff8fca00640 1 --2- 172.21.15.5:0/3976794558 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff8f8101df0 0x7ff8f81021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:35:19.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:19.390+0000 7ff8fca00640 1 -- 172.21.15.5:0/3976794558 >> 172.21.15.5:0/3976794558 conn(0x7ff8f80fd8b0 msgr2=0x7ff8f80ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:35:19.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:19.390+0000 7ff8fca00640 1 -- 172.21.15.5:0/3976794558 shutdown_connections 2024-09-10T09:35:19.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:19.390+0000 7ff8fca00640 1 -- 172.21.15.5:0/3976794558 wait complete. 2024-09-10T09:35:19.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:19.391+0000 7ff8fca00640 1 Processor -- start 2024-09-10T09:35:19.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:19.391+0000 7ff8fca00640 1 -- start start 2024-09-10T09:35:19.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:19.392+0000 7ff8fca00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff8f8101df0 0x7ff8f819a530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:35:19.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:19.392+0000 7ff8fca00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff8f81027a0 0x7ff8f819aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:35:19.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:19.392+0000 7ff8fca00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff8f8106f20 0x7ff8f819ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:35:19.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:19.392+0000 7ff8fca00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff8f8111f80 con 0x7ff8f8101df0 2024-09-10T09:35:19.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:19.392+0000 7ff8fca00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7ff8f8111e00 con 0x7ff8f8106f20 2024-09-10T09:35:19.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:19.392+0000 7ff8fca00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7ff8f8112100 con 0x7ff8f81027a0 2024-09-10T09:35:19.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:19.392+0000 7ff8ed400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff8f81027a0 0x7ff8f819aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:35:19.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:19.392+0000 7ff8f5e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff8f8101df0 0x7ff8f819a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:35:19.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:19.392+0000 7ff8ed400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff8f81027a0 0x7ff8f819aa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:37476/0 (socket says 172.21.15.5:37476) 2024-09-10T09:35:19.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:19.392+0000 7ff8ed400640 1 -- 172.21.15.5:0/2076541366 learned_addr learned my addr 172.21.15.5:0/2076541366 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:35:19.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:19.392+0000 7ff8f6800640 1 --2- 172.21.15.5:0/2076541366 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff8f8106f20 0x7ff8f819ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:35:19.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:19.393+0000 7ff8ed400640 1 -- 172.21.15.5:0/2076541366 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff8f8106f20 msgr2=0x7ff8f819ee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:35:19.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:19.393+0000 7ff8ed400640 1 --2- 172.21.15.5:0/2076541366 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff8f8106f20 0x7ff8f819ee00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:35:19.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:19.393+0000 7ff8ed400640 1 -- 172.21.15.5:0/2076541366 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff8f8101df0 msgr2=0x7ff8f819a530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:35:19.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:19.393+0000 7ff8ed400640 1 --2- 172.21.15.5:0/2076541366 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff8f8101df0 0x7ff8f819a530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:35:19.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:19.393+0000 7ff8ed400640 1 -- 172.21.15.5:0/2076541366 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8f819f4e0 con 0x7ff8f81027a0 2024-09-10T09:35:19.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:19.393+0000 7ff8ed400640 1 --2- 172.21.15.5:0/2076541366 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff8f81027a0 0x7ff8f819aa70 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7ff8e000ea60 tx=0x7ff8e000ef20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:35:19.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:19.393+0000 7ff8ef400640 1 -- 172.21.15.5:0/2076541366 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8e000cde0 con 0x7ff8f81027a0 2024-09-10T09:35:19.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:19.393+0000 7ff8fca00640 1 -- 172.21.15.5:0/2076541366 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8f819f7d0 con 0x7ff8f81027a0 2024-09-10T09:35:19.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:19.393+0000 7ff8ef400640 1 -- 172.21.15.5:0/2076541366 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff8e00035f0 con 0x7ff8f81027a0 2024-09-10T09:35:19.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:19.393+0000 7ff8ef400640 1 -- 172.21.15.5:0/2076541366 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8e0010670 con 0x7ff8f81027a0 2024-09-10T09:35:19.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:19.394+0000 7ff8fca00640 1 -- 172.21.15.5:0/2076541366 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8f81a70b0 con 0x7ff8f81027a0 2024-09-10T09:35:19.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:19.395+0000 7ff8fca00640 1 -- 172.21.15.5:0/2076541366 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff8f81021d0 con 0x7ff8f81027a0 2024-09-10T09:35:19.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:19.397+0000 7ff8ef400640 1 -- 172.21.15.5:0/2076541366 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7ff8e00037a0 con 0x7ff8f81027a0 2024-09-10T09:35:19.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:19.398+0000 7ff8ef400640 1 --2- 172.21.15.5:0/2076541366 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7ff8d80a2c00 0x7ff8d80a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:35:19.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:19.398+0000 7ff8ef400640 1 -- 172.21.15.5:0/2076541366 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7127+0+0 (secure 0 0 0) 0x7ff8e00cedc0 con 0x7ff8f81027a0 2024-09-10T09:35:19.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:19.399+0000 7ff8f5e00640 1 --2- 172.21.15.5:0/2076541366 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7ff8d80a2c00 0x7ff8d80a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:35:19.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:19.400+0000 7ff8f5e00640 1 --2- 172.21.15.5:0/2076541366 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7ff8d80a2c00 0x7ff8d80a50c0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7ff8e8005d00 tx=0x7ff8e800b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:35:19.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:19.402+0000 7ff8ef400640 1 -- 172.21.15.5:0/2076541366 <== mon.1 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff8e009c120 con 0x7ff8f81027a0 2024-09-10T09:35:19.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:35:19 smithi005 ceph-mon[30350]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T09:35:19.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:35:19 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:35:19.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:35:19 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:35:19.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:35:19 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:35:19.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:35:19 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:35:19.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:19.635+0000 7ff8fca00640 1 -- 172.21.15.5:0/2076541366 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff8f81a7440 con 0x7ff8f81027a0 2024-09-10T09:35:19.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:19.636+0000 7ff8ef400640 1 -- 172.21.15.5:0/2076541366 <== mon.1 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+10515 (secure 0 0 0) 0x7ff8e009ba20 con 0x7ff8f81027a0 2024-09-10T09:35:19.638 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-10T09:35:19.638 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":13,"btime":"2024-09-10T09:35:17:466066+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14454,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/2970594311","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2970594311},{"type":"v1","addr":"172.21.15.5:6835","nonce":2970594311}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14514,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6839/3616502174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":3616502174},{"type":"v1","addr":"172.21.15.5:6839","nonce":3616502174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14520,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6841/2073599842","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":2073599842},{"type":"v1","addr":"172.21.15.5:6841","nonce":2073599842}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6833/2457007427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":2457007427},{"type":"v1","addr":"172.21.15.102:6833","nonce":2457007427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24460,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6833/356445159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":356445159},{"type":"v1","addr":"172.21.15.159:6833","nonce":356445159}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24467,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/2894314507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":2894314507},{"type":"v1","addr":"172.21.15.5:6837","nonce":2894314507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24479,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6835/1647549111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":1647549111},{"type":"v1","addr":"172.21.15.102:6835","nonce":1647549111}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24505,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6835/952657242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":952657242},{"type":"v1","addr":"172.21.15.159:6835","nonce":952657242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24506,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/2689776764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":2689776764},{"type":"v1","addr":"172.21.15.102:6837","nonce":2689776764}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24511,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6837/620627639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":620627639},{"type":"v1","addr":"172.21.15.159:6837","nonce":620627639}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24535,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6839/2073739599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":2073739599},{"type":"v1","addr":"172.21.15.159:6839","nonce":2073739599}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24542,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6839/335409749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6838","nonce":335409749},{"type":"v1","addr":"172.21.15.102:6839","nonce":335409749}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[]} 2024-09-10T09:35:19.638 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 13 2024-09-10T09:35:19.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:19.639+0000 7ff8fca00640 1 -- 172.21.15.5:0/2076541366 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7ff8d80a2c00 msgr2=0x7ff8d80a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:35:19.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:19.639+0000 7ff8fca00640 1 --2- 172.21.15.5:0/2076541366 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7ff8d80a2c00 0x7ff8d80a50c0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7ff8e8005d00 tx=0x7ff8e800b410 comp rx=0 tx=0).stop 2024-09-10T09:35:19.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:19.639+0000 7ff8fca00640 1 -- 172.21.15.5:0/2076541366 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff8f81027a0 msgr2=0x7ff8f819aa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:35:19.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:19.639+0000 7ff8fca00640 1 --2- 172.21.15.5:0/2076541366 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff8f81027a0 0x7ff8f819aa70 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7ff8e000ea60 tx=0x7ff8e000ef20 comp rx=0 tx=0).stop 2024-09-10T09:35:19.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:19.639+0000 7ff8fca00640 1 -- 172.21.15.5:0/2076541366 shutdown_connections 2024-09-10T09:35:19.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:19.639+0000 7ff8fca00640 1 --2- 172.21.15.5:0/2076541366 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7ff8d80a2c00 0x7ff8d80a50c0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:35:19.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:19.639+0000 7ff8fca00640 1 --2- 172.21.15.5:0/2076541366 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff8f8106f20 0x7ff8f819ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:35:19.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:19.639+0000 7ff8fca00640 1 --2- 172.21.15.5:0/2076541366 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff8f81027a0 0x7ff8f819aa70 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:35:19.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:19.639+0000 7ff8fca00640 1 --2- 172.21.15.5:0/2076541366 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff8f8101df0 0x7ff8f819a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:35:19.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:19.639+0000 7ff8fca00640 1 -- 172.21.15.5:0/2076541366 >> 172.21.15.5:0/2076541366 conn(0x7ff8f80fd8b0 msgr2=0x7ff8f80fe650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:35:19.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:19.639+0000 7ff8fca00640 1 -- 172.21.15.5:0/2076541366 shutdown_connections 2024-09-10T09:35:19.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:19.639+0000 7ff8fca00640 1 -- 172.21.15.5:0/2076541366 wait complete. 2024-09-10T09:35:19.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:35:19 smithi159 ceph-mon[35513]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T09:35:19.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:35:19 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:35:19.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:35:19 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:35:19.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:35:19 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:35:19.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:35:19 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:35:19.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:35:19 smithi102 ceph-mon[35442]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T09:35:19.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:35:19 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:35:19.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:35:19 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:35:19.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:35:19 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:35:19.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:35:19 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:35:20.423 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph --format=json-pretty osd lspools 2024-09-10T09:35:20.786 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:35:20 smithi102 ceph-mon[35442]: from='client.? 172.21.15.5:0/2076541366' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:35:20.786 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:35:20 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:35:20.786 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:35:20 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:35:20.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:35:20 smithi005 ceph-mon[30350]: from='client.? 172.21.15.5:0/2076541366' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:35:20.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:35:20 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:35:20.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:35:20 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:35:20.814 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:35:20.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:35:20 smithi159 ceph-mon[35513]: from='client.? 172.21.15.5:0/2076541366' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:35:20.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:35:20 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:35:20.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:35:20 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:35:21.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:21.377+0000 7fb5fac00640 1 -- 172.21.15.5:0/73363542 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb5f4112a30 msgr2=0x7fb5f4114e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:35:21.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:21.377+0000 7fb5fac00640 1 --2- 172.21.15.5:0/73363542 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb5f4112a30 0x7fb5f4114e20 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fb5ec009ca0 tx=0x7fb5ec02a8c0 comp rx=0 tx=0).stop 2024-09-10T09:35:21.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:21.377+0000 7fb5fac00640 1 -- 172.21.15.5:0/73363542 shutdown_connections 2024-09-10T09:35:21.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:21.377+0000 7fb5fac00640 1 --2- 172.21.15.5:0/73363542 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb5f4112a30 0x7fb5f4114e20 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:35:21.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:21.377+0000 7fb5fac00640 1 --2- 172.21.15.5:0/73363542 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb5f4077040 0x7fb5f41124f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:35:21.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:21.377+0000 7fb5fac00640 1 --2- 172.21.15.5:0/73363542 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb5f4076720 0x7fb5f4076b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:35:21.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:21.377+0000 7fb5fac00640 1 -- 172.21.15.5:0/73363542 >> 172.21.15.5:0/73363542 conn(0x7fb5f40ffa90 msgr2=0x7fb5f4101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:35:21.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:21.377+0000 7fb5fac00640 1 -- 172.21.15.5:0/73363542 shutdown_connections 2024-09-10T09:35:21.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:21.377+0000 7fb5fac00640 1 -- 172.21.15.5:0/73363542 wait complete. 2024-09-10T09:35:21.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:21.378+0000 7fb5fac00640 1 Processor -- start 2024-09-10T09:35:21.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:21.378+0000 7fb5fac00640 1 -- start start 2024-09-10T09:35:21.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:21.378+0000 7fb5fac00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb5f4076720 0x7fb5f41a46c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:35:21.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:21.378+0000 7fb5fac00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb5f4077040 0x7fb5f41a4c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:35:21.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:21.378+0000 7fb5fac00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb5f4112a30 0x7fb5f419e880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:35:21.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:21.378+0000 7fb5fac00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fb5f41185e0 con 0x7fb5f4112a30 2024-09-10T09:35:21.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:21.378+0000 7fb5fac00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fb5f4118460 con 0x7fb5f4077040 2024-09-10T09:35:21.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:21.378+0000 7fb5fac00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fb5f4118760 con 0x7fb5f4076720 2024-09-10T09:35:21.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:21.379+0000 7fb5f3e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb5f4076720 0x7fb5f41a46c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:35:21.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:21.379+0000 7fb5f3e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb5f4076720 0x7fb5f41a46c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:45994/0 (socket says 172.21.15.5:45994) 2024-09-10T09:35:21.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:21.379+0000 7fb5f3e00640 1 -- 172.21.15.5:0/2221944496 learned_addr learned my addr 172.21.15.5:0/2221944496 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:35:21.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:21.379+0000 7fb5f3400640 1 --2- 172.21.15.5:0/2221944496 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb5f4077040 0x7fb5f41a4c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:35:21.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:21.379+0000 7fb5f8c00640 1 --2- 172.21.15.5:0/2221944496 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb5f4112a30 0x7fb5f419e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:35:21.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:21.379+0000 7fb5f3e00640 1 -- 172.21.15.5:0/2221944496 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb5f4077040 msgr2=0x7fb5f41a4c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:35:21.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:21.379+0000 7fb5f3e00640 1 --2- 172.21.15.5:0/2221944496 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb5f4077040 0x7fb5f41a4c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:35:21.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:21.379+0000 7fb5f3e00640 1 -- 172.21.15.5:0/2221944496 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb5f4112a30 msgr2=0x7fb5f419e880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:35:21.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:21.379+0000 7fb5f3e00640 1 --2- 172.21.15.5:0/2221944496 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb5f4112a30 0x7fb5f419e880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:35:21.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:21.379+0000 7fb5f3e00640 1 -- 172.21.15.5:0/2221944496 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5f419efb0 con 0x7fb5f4076720 2024-09-10T09:35:21.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:21.379+0000 7fb5f3e00640 1 --2- 172.21.15.5:0/2221944496 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb5f4076720 0x7fb5f41a46c0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fb5e400e9f0 tx=0x7fb5e400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:35:21.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:21.380+0000 7fb5f0c00640 1 -- 172.21.15.5:0/2221944496 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5e400cd80 con 0x7fb5f4076720 2024-09-10T09:35:21.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:21.380+0000 7fb5f0c00640 1 -- 172.21.15.5:0/2221944496 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb5e4010910 con 0x7fb5f4076720 2024-09-10T09:35:21.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:21.380+0000 7fb5fac00640 1 -- 172.21.15.5:0/2221944496 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5f419f2a0 con 0x7fb5f4076720 2024-09-10T09:35:21.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:21.380+0000 7fb5f0c00640 1 -- 172.21.15.5:0/2221944496 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5e4018990 con 0x7fb5f4076720 2024-09-10T09:35:21.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:21.380+0000 7fb5fac00640 1 -- 172.21.15.5:0/2221944496 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5f41ab590 con 0x7fb5f4076720 2024-09-10T09:35:21.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:21.381+0000 7fb5fac00640 1 -- 172.21.15.5:0/2221944496 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb5f4076b00 con 0x7fb5f4076720 2024-09-10T09:35:21.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:21.383+0000 7fb5f0c00640 1 -- 172.21.15.5:0/2221944496 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7fb5e4010ac0 con 0x7fb5f4076720 2024-09-10T09:35:21.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:21.384+0000 7fb5f0c00640 1 --2- 172.21.15.5:0/2221944496 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fb5d80a2b80 0x7fb5d80a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:35:21.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:21.384+0000 7fb5f0c00640 1 -- 172.21.15.5:0/2221944496 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7127+0+0 (secure 0 0 0) 0x7fb5e40cf390 con 0x7fb5f4076720 2024-09-10T09:35:21.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:21.385+0000 7fb5f3400640 1 --2- 172.21.15.5:0/2221944496 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fb5d80a2b80 0x7fb5d80a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:35:21.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:21.385+0000 7fb5f3400640 1 --2- 172.21.15.5:0/2221944496 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fb5d80a2b80 0x7fb5d80a5040 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fb5f40792b0 tx=0x7fb5e0005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:35:21.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:21.386+0000 7fb5f0c00640 1 -- 172.21.15.5:0/2221944496 <== mon.1 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb5e409c0a0 con 0x7fb5f4076720 2024-09-10T09:35:21.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:35:21 smithi005 ceph-mon[30350]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T09:35:21.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:35:21 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:35:21.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:35:21 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:35:21.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:35:21 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:35:21.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:35:21 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:35:21.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:35:21 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:35:21.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:21.584+0000 7fb5fac00640 1 -- 172.21.15.5:0/2221944496 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fb5f419fdb0 con 0x7fb5f4076720 2024-09-10T09:35:21.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:21.585+0000 7fb5f0c00640 1 -- 172.21.15.5:0/2221944496 <== mon.1 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v78) ==== 84+0+65 (secure 0 0 0) 0x7fb5e40a03a0 con 0x7fb5f4076720 2024-09-10T09:35:21.586 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-10T09:35:21.586 INFO:teuthology.orchestra.run.smithi005.stdout:[ 2024-09-10T09:35:21.586 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-10T09:35:21.586 INFO:teuthology.orchestra.run.smithi005.stdout: "poolnum": 1, 2024-09-10T09:35:21.586 INFO:teuthology.orchestra.run.smithi005.stdout: "poolname": ".mgr" 2024-09-10T09:35:21.586 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T09:35:21.586 INFO:teuthology.orchestra.run.smithi005.stdout:] 2024-09-10T09:35:21.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:21.588+0000 7fb5fac00640 1 -- 172.21.15.5:0/2221944496 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fb5d80a2b80 msgr2=0x7fb5d80a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:35:21.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:21.588+0000 7fb5fac00640 1 --2- 172.21.15.5:0/2221944496 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fb5d80a2b80 0x7fb5d80a5040 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fb5f40792b0 tx=0x7fb5e0005f90 comp rx=0 tx=0).stop 2024-09-10T09:35:21.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:21.588+0000 7fb5fac00640 1 -- 172.21.15.5:0/2221944496 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb5f4076720 msgr2=0x7fb5f41a46c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:35:21.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:21.588+0000 7fb5fac00640 1 --2- 172.21.15.5:0/2221944496 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb5f4076720 0x7fb5f41a46c0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fb5e400e9f0 tx=0x7fb5e400eeb0 comp rx=0 tx=0).stop 2024-09-10T09:35:21.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:21.588+0000 7fb5fac00640 1 -- 172.21.15.5:0/2221944496 shutdown_connections 2024-09-10T09:35:21.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:21.588+0000 7fb5fac00640 1 --2- 172.21.15.5:0/2221944496 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fb5d80a2b80 0x7fb5d80a5040 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:35:21.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:21.588+0000 7fb5fac00640 1 --2- 172.21.15.5:0/2221944496 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb5f4112a30 0x7fb5f419e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:35:21.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:21.588+0000 7fb5fac00640 1 --2- 172.21.15.5:0/2221944496 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb5f4077040 0x7fb5f41a4c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:35:21.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:21.588+0000 7fb5fac00640 1 --2- 172.21.15.5:0/2221944496 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb5f4076720 0x7fb5f41a46c0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:35:21.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:21.588+0000 7fb5fac00640 1 -- 172.21.15.5:0/2221944496 >> 172.21.15.5:0/2221944496 conn(0x7fb5f40ffa90 msgr2=0x7fb5f4101e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:35:21.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:21.588+0000 7fb5fac00640 1 -- 172.21.15.5:0/2221944496 shutdown_connections 2024-09-10T09:35:21.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:21.588+0000 7fb5fac00640 1 -- 172.21.15.5:0/2221944496 wait complete. 2024-09-10T09:35:21.773 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:35:21 smithi159 ceph-mon[35513]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T09:35:21.773 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:35:21 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:35:21.773 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:35:21 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:35:21.773 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:35:21 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:35:21.773 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:35:21 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:35:21.774 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:35:21 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:35:21.779 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:35:21 smithi102 ceph-mon[35442]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T09:35:21.779 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:35:21 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:35:21.779 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:35:21 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:35:21.779 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:35:21 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:35:21.779 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:35:21 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:35:21.779 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:35:21 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:35:22.668 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-10T09:35:22.668 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-10T09:35:22.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:35:22 smithi005 ceph-mon[30350]: from='client.? 172.21.15.5:0/2221944496' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-09-10T09:35:22.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:35:22 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:35:22.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:35:22 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:35:22.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:35:22 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:35:22.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:35:22 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:35:22.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:35:22 smithi159 ceph-mon[35513]: from='client.? 172.21.15.5:0/2221944496' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-09-10T09:35:22.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:35:22 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:35:22.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:35:22 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:35:22.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:35:22 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:35:22.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:35:22 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:35:22.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:35:22 smithi102 ceph-mon[35442]: from='client.? 172.21.15.5:0/2221944496' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-09-10T09:35:22.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:35:22 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:35:22.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:35:22 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:35:22.885 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:35:22 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:35:22.885 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:35:22 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:35:23.073 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:35:23.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:23.678+0000 7f07f3600640 1 -- 172.21.15.5:0/3304352937 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f07ec104010 msgr2=0x7f07ec1043f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:35:23.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:23.678+0000 7f07f3600640 1 --2- 172.21.15.5:0/3304352937 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f07ec104010 0x7f07ec1043f0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f07e0009900 tx=0x7f07e002a410 comp rx=0 tx=0).stop 2024-09-10T09:35:23.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:23.678+0000 7f07f3600640 1 -- 172.21.15.5:0/3304352937 shutdown_connections 2024-09-10T09:35:23.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:23.678+0000 7f07f3600640 1 --2- 172.21.15.5:0/3304352937 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f07ec109140 0x7f07ec110cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:35:23.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:23.678+0000 7f07f3600640 1 --2- 172.21.15.5:0/3304352937 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f07ec1049c0 0x7f07ec108a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:35:23.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:23.678+0000 7f07f3600640 1 --2- 172.21.15.5:0/3304352937 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f07ec104010 0x7f07ec1043f0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:35:23.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:23.678+0000 7f07f3600640 1 -- 172.21.15.5:0/3304352937 >> 172.21.15.5:0/3304352937 conn(0x7f07ec0ffa50 msgr2=0x7f07ec101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:35:23.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:23.678+0000 7f07f3600640 1 -- 172.21.15.5:0/3304352937 shutdown_connections 2024-09-10T09:35:23.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:23.678+0000 7f07f3600640 1 -- 172.21.15.5:0/3304352937 wait complete. 2024-09-10T09:35:23.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:23.679+0000 7f07f3600640 1 Processor -- start 2024-09-10T09:35:23.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:23.679+0000 7f07f3600640 1 -- start start 2024-09-10T09:35:23.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:23.680+0000 7f07f3600640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f07ec104010 0x7f07ec19a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:35:23.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:23.680+0000 7f07f3600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f07ec1049c0 0x7f07ec19ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:35:23.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:23.680+0000 7f07f3600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f07ec109140 0x7f07ec19ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:35:23.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:23.680+0000 7f07f3600640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f07ec1141c0 con 0x7f07ec109140 2024-09-10T09:35:23.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:23.680+0000 7f07f3600640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f07ec114040 con 0x7f07ec1049c0 2024-09-10T09:35:23.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:23.680+0000 7f07f3600640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f07ec114340 con 0x7f07ec104010 2024-09-10T09:35:23.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:23.680+0000 7f07ebe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f07ec1049c0 0x7f07ec19ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:35:23.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:23.680+0000 7f07ebe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f07ec1049c0 0x7f07ec19ab80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:33132/0 (socket says 172.21.15.5:33132) 2024-09-10T09:35:23.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:23.680+0000 7f07ebe00640 1 -- 172.21.15.5:0/3404981449 learned_addr learned my addr 172.21.15.5:0/3404981449 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:35:23.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:23.680+0000 7f07f1600640 1 --2- 172.21.15.5:0/3404981449 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f07ec109140 0x7f07ec19ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:35:23.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:23.680+0000 7f07f0c00640 1 --2- 172.21.15.5:0/3404981449 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f07ec104010 0x7f07ec19a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:35:23.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:23.681+0000 7f07ebe00640 1 -- 172.21.15.5:0/3404981449 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f07ec104010 msgr2=0x7f07ec19a640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:35:23.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:23.681+0000 7f07ebe00640 1 --2- 172.21.15.5:0/3404981449 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f07ec104010 0x7f07ec19a640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:35:23.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:23.681+0000 7f07ebe00640 1 -- 172.21.15.5:0/3404981449 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f07ec109140 msgr2=0x7f07ec19ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:35:23.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:23.681+0000 7f07ebe00640 1 --2- 172.21.15.5:0/3404981449 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f07ec109140 0x7f07ec19ef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:35:23.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:23.681+0000 7f07ebe00640 1 -- 172.21.15.5:0/3404981449 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07ec19f5f0 con 0x7f07ec1049c0 2024-09-10T09:35:23.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:23.681+0000 7f07ebe00640 1 --2- 172.21.15.5:0/3404981449 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f07ec1049c0 0x7f07ec19ab80 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f07d800d950 tx=0x7f07d800de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:35:23.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:23.681+0000 7f07e9600640 1 -- 172.21.15.5:0/3404981449 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07d8017030 con 0x7f07ec1049c0 2024-09-10T09:35:23.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:23.681+0000 7f07f3600640 1 -- 172.21.15.5:0/3404981449 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07ec19f8e0 con 0x7f07ec1049c0 2024-09-10T09:35:23.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:23.681+0000 7f07e9600640 1 -- 172.21.15.5:0/3404981449 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f07d8004020 con 0x7f07ec1049c0 2024-09-10T09:35:23.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:23.681+0000 7f07e9600640 1 -- 172.21.15.5:0/3404981449 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07d80047a0 con 0x7f07ec1049c0 2024-09-10T09:35:23.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:23.681+0000 7f07f3600640 1 -- 172.21.15.5:0/3404981449 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07ec1a71c0 con 0x7f07ec1049c0 2024-09-10T09:35:23.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:23.683+0000 7f07f3600640 1 -- 172.21.15.5:0/3404981449 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f07ec1043f0 con 0x7f07ec1049c0 2024-09-10T09:35:23.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:23.685+0000 7f07e9600640 1 -- 172.21.15.5:0/3404981449 <== mon.2 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7f07d80041d0 con 0x7f07ec1049c0 2024-09-10T09:35:23.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:23.686+0000 7f07e9600640 1 --2- 172.21.15.5:0/3404981449 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f07d40a2c00 0x7f07d40a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:35:23.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:23.686+0000 7f07e9600640 1 -- 172.21.15.5:0/3404981449 <== mon.2 v2:172.21.15.102:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7127+0+0 (secure 0 0 0) 0x7f07d80ce5a0 con 0x7f07ec1049c0 2024-09-10T09:35:23.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:23.687+0000 7f07f0c00640 1 --2- 172.21.15.5:0/3404981449 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f07d40a2c00 0x7f07d40a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:35:23.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:23.687+0000 7f07f0c00640 1 --2- 172.21.15.5:0/3404981449 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f07d40a2c00 0x7f07d40a50c0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f07e0000c00 tx=0x7f07e0005b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:35:23.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:23.691+0000 7f07e9600640 1 -- 172.21.15.5:0/3404981449 <== mon.2 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f07d809b900 con 0x7f07ec1049c0 2024-09-10T09:35:23.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:35:23 smithi005 ceph-mon[30350]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T09:35:23.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:35:23 smithi159 ceph-mon[35513]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T09:35:23.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:23.883+0000 7f07f3600640 1 -- 172.21.15.5:0/3404981449 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f07ec02d7c0 con 0x7f07ec1049c0 2024-09-10T09:35:23.885 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:35:23 smithi102 ceph-mon[35442]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T09:35:24.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:24.522+0000 7f07e9600640 1 -- 172.21.15.5:0/3404981449 <== mon.2 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v79) ==== 138+0+0 (secure 0 0 0) 0x7f07d809b200 con 0x7f07ec1049c0 2024-09-10T09:35:24.523 INFO:teuthology.orchestra.run.smithi005.stderr:pool 'cephfs_metadata' created 2024-09-10T09:35:24.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:24.526+0000 7f07f3600640 1 -- 172.21.15.5:0/3404981449 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f07d40a2c00 msgr2=0x7f07d40a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:35:24.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:24.526+0000 7f07f3600640 1 --2- 172.21.15.5:0/3404981449 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f07d40a2c00 0x7f07d40a50c0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f07e0000c00 tx=0x7f07e0005b50 comp rx=0 tx=0).stop 2024-09-10T09:35:24.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:24.526+0000 7f07f3600640 1 -- 172.21.15.5:0/3404981449 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f07ec1049c0 msgr2=0x7f07ec19ab80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:35:24.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:24.526+0000 7f07f3600640 1 --2- 172.21.15.5:0/3404981449 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f07ec1049c0 0x7f07ec19ab80 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f07d800d950 tx=0x7f07d800de10 comp rx=0 tx=0).stop 2024-09-10T09:35:24.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:24.526+0000 7f07f3600640 1 -- 172.21.15.5:0/3404981449 shutdown_connections 2024-09-10T09:35:24.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:24.526+0000 7f07f3600640 1 --2- 172.21.15.5:0/3404981449 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f07d40a2c00 0x7f07d40a50c0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:35:24.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:24.526+0000 7f07f3600640 1 --2- 172.21.15.5:0/3404981449 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f07ec109140 0x7f07ec19ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:35:24.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:24.526+0000 7f07f3600640 1 --2- 172.21.15.5:0/3404981449 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f07ec1049c0 0x7f07ec19ab80 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:35:24.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:24.526+0000 7f07f3600640 1 --2- 172.21.15.5:0/3404981449 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f07ec104010 0x7f07ec19a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:35:24.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:24.526+0000 7f07f3600640 1 -- 172.21.15.5:0/3404981449 >> 172.21.15.5:0/3404981449 conn(0x7f07ec0ffa50 msgr2=0x7f07ec100d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:35:24.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:24.527+0000 7f07f3600640 1 -- 172.21.15.5:0/3404981449 shutdown_connections 2024-09-10T09:35:24.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:24.528+0000 7f07f3600640 1 -- 172.21.15.5:0/3404981449 wait complete. 2024-09-10T09:35:24.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:35:24 smithi005 ceph-mon[30350]: from='client.? 172.21.15.5:0/3404981449' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-09-10T09:35:24.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:35:24 smithi005 ceph-mon[30350]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-09-10T09:35:24.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:35:24 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:35:24.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:35:24 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:35:24.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:35:24 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:35:24.807 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:35:24 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:35:24.807 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:35:24 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:35:24.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:35:24 smithi159 ceph-mon[35513]: from='client.? 172.21.15.5:0/3404981449' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-09-10T09:35:24.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:35:24 smithi159 ceph-mon[35513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-09-10T09:35:24.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:35:24 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:35:24.851 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:35:24 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:35:24.851 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:35:24 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:35:24.851 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:35:24 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:35:24.851 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:35:24 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:35:24.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:35:24 smithi102 ceph-mon[35442]: from='client.? 172.21.15.5:0/3404981449' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-09-10T09:35:24.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:35:24 smithi102 ceph-mon[35442]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-09-10T09:35:24.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:35:24 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:35:24.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:35:24 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:35:24.885 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:35:24 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:35:24.885 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:35:24 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:35:24.885 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:35:24 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:35:25.415 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2024-09-10T09:35:25.801 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:35:25.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:35:25 smithi005 ceph-mon[30350]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T09:35:25.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:35:25 smithi005 ceph-mon[30350]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-09-10T09:35:25.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:35:25 smithi005 ceph-mon[30350]: osdmap e79: 12 total, 12 up, 12 in 2024-09-10T09:35:25.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:35:25 smithi159 ceph-mon[35513]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T09:35:25.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:35:25 smithi159 ceph-mon[35513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-09-10T09:35:25.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:35:25 smithi159 ceph-mon[35513]: osdmap e79: 12 total, 12 up, 12 in 2024-09-10T09:35:25.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:35:25 smithi102 ceph-mon[35442]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T09:35:25.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:35:25 smithi102 ceph-mon[35442]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-09-10T09:35:25.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:35:25 smithi102 ceph-mon[35442]: osdmap e79: 12 total, 12 up, 12 in 2024-09-10T09:35:26.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:26.430+0000 7f5e12600640 1 -- 172.21.15.5:0/2068309917 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5e0c103e30 msgr2=0x7f5e0c104210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:35:26.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:26.430+0000 7f5e12600640 1 --2- 172.21.15.5:0/2068309917 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5e0c103e30 0x7f5e0c104210 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f5e00009900 tx=0x7f5e0002a410 comp rx=0 tx=0).stop 2024-09-10T09:35:26.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:26.430+0000 7f5e12600640 1 -- 172.21.15.5:0/2068309917 shutdown_connections 2024-09-10T09:35:26.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:26.430+0000 7f5e12600640 1 --2- 172.21.15.5:0/2068309917 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5e0c108f60 0x7f5e0c110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:35:26.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:26.430+0000 7f5e12600640 1 --2- 172.21.15.5:0/2068309917 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5e0c1047e0 0x7f5e0c108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:35:26.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:26.430+0000 7f5e12600640 1 --2- 172.21.15.5:0/2068309917 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5e0c103e30 0x7f5e0c104210 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:35:26.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:26.430+0000 7f5e12600640 1 -- 172.21.15.5:0/2068309917 >> 172.21.15.5:0/2068309917 conn(0x7f5e0c0ff910 msgr2=0x7f5e0c101d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:35:26.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:26.430+0000 7f5e12600640 1 -- 172.21.15.5:0/2068309917 shutdown_connections 2024-09-10T09:35:26.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:26.430+0000 7f5e12600640 1 -- 172.21.15.5:0/2068309917 wait complete. 2024-09-10T09:35:26.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:26.431+0000 7f5e12600640 1 Processor -- start 2024-09-10T09:35:26.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:26.431+0000 7f5e12600640 1 -- start start 2024-09-10T09:35:26.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:26.432+0000 7f5e12600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5e0c103e30 0x7f5e0c19a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:35:26.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:26.432+0000 7f5e12600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5e0c1047e0 0x7f5e0c19aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:35:26.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:26.432+0000 7f5e12600640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5e0c108f60 0x7f5e0c19edc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:35:26.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:26.432+0000 7f5e12600640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f5e0c114010 con 0x7f5e0c1047e0 2024-09-10T09:35:26.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:26.432+0000 7f5e12600640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f5e0c113e90 con 0x7f5e0c103e30 2024-09-10T09:35:26.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:26.432+0000 7f5e12600640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f5e0c114190 con 0x7f5e0c108f60 2024-09-10T09:35:26.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:26.432+0000 7f5e0be00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5e0c1047e0 0x7f5e0c19aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:35:26.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:26.432+0000 7f5e11200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5e0c103e30 0x7f5e0c19a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:35:26.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:26.432+0000 7f5e11200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5e0c103e30 0x7f5e0c19a4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:33150/0 (socket says 172.21.15.5:33150) 2024-09-10T09:35:26.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:26.432+0000 7f5e11c00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5e0c108f60 0x7f5e0c19edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:35:26.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:26.432+0000 7f5e11200640 1 -- 172.21.15.5:0/3737458811 learned_addr learned my addr 172.21.15.5:0/3737458811 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:35:26.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:26.432+0000 7f5e0be00640 1 -- 172.21.15.5:0/3737458811 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5e0c108f60 msgr2=0x7f5e0c19edc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:35:26.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:26.432+0000 7f5e0be00640 1 --2- 172.21.15.5:0/3737458811 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5e0c108f60 0x7f5e0c19edc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:35:26.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:26.433+0000 7f5e0be00640 1 -- 172.21.15.5:0/3737458811 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5e0c103e30 msgr2=0x7f5e0c19a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:35:26.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:26.433+0000 7f5e0be00640 1 --2- 172.21.15.5:0/3737458811 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5e0c103e30 0x7f5e0c19a4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:35:26.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:26.433+0000 7f5e0be00640 1 -- 172.21.15.5:0/3737458811 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e0c19f4a0 con 0x7f5e0c1047e0 2024-09-10T09:35:26.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:26.433+0000 7f5e11c00640 1 --2- 172.21.15.5:0/3737458811 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5e0c108f60 0x7f5e0c19edc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:35:26.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:26.433+0000 7f5e0be00640 1 --2- 172.21.15.5:0/3737458811 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5e0c1047e0 0x7f5e0c19aa30 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f5df800e9f0 tx=0x7f5df800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:35:26.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:26.433+0000 7f5e09600640 1 -- 172.21.15.5:0/3737458811 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5df800ccc0 con 0x7f5e0c1047e0 2024-09-10T09:35:26.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:26.433+0000 7f5e09600640 1 -- 172.21.15.5:0/3737458811 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5df8005020 con 0x7f5e0c1047e0 2024-09-10T09:35:26.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:26.433+0000 7f5e12600640 1 -- 172.21.15.5:0/3737458811 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e0c19f790 con 0x7f5e0c1047e0 2024-09-10T09:35:26.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:26.433+0000 7f5e09600640 1 -- 172.21.15.5:0/3737458811 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5df80049e0 con 0x7f5e0c1047e0 2024-09-10T09:35:26.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:26.433+0000 7f5e12600640 1 -- 172.21.15.5:0/3737458811 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e0c1a7070 con 0x7f5e0c1047e0 2024-09-10T09:35:26.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:26.434+0000 7f5e12600640 1 -- 172.21.15.5:0/3737458811 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5e0c105310 con 0x7f5e0c1047e0 2024-09-10T09:35:26.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:26.437+0000 7f5e09600640 1 -- 172.21.15.5:0/3737458811 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7f5df8005560 con 0x7f5e0c1047e0 2024-09-10T09:35:26.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:26.438+0000 7f5e09600640 1 --2- 172.21.15.5:0/3737458811 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f5df40a2c00 0x7f5df40a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:35:26.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:26.438+0000 7f5e09600640 1 -- 172.21.15.5:0/3737458811 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7497+0+0 (secure 0 0 0) 0x7f5df80ced50 con 0x7f5e0c1047e0 2024-09-10T09:35:26.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:26.438+0000 7f5e11200640 1 --2- 172.21.15.5:0/3737458811 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f5df40a2c00 0x7f5df40a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:35:26.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:26.439+0000 7f5e11200640 1 --2- 172.21.15.5:0/3737458811 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f5df40a2c00 0x7f5df40a50c0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f5e000096c0 tx=0x7f5e000046c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:35:26.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:26.440+0000 7f5e09600640 1 -- 172.21.15.5:0/3737458811 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f5df80051d0 con 0x7f5e0c1047e0 2024-09-10T09:35:26.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:26.627+0000 7f5e12600640 1 -- 172.21.15.5:0/3737458811 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) -- 0x7f5e0c104210 con 0x7f5e0c1047e0 2024-09-10T09:35:26.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:35:26 smithi005 ceph-mon[30350]: osdmap e80: 12 total, 12 up, 12 in 2024-09-10T09:35:26.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:35:26 smithi005 ceph-mon[30350]: pgmap v290: 33 pgs: 14 creating+peering, 1 active+clean, 18 unknown; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T09:35:26.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:35:26 smithi005 ceph-mon[30350]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T09:35:26.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:35:26 smithi159 ceph-mon[35513]: osdmap e80: 12 total, 12 up, 12 in 2024-09-10T09:35:26.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:35:26 smithi159 ceph-mon[35513]: pgmap v290: 33 pgs: 14 creating+peering, 1 active+clean, 18 unknown; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T09:35:26.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:35:26 smithi159 ceph-mon[35513]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T09:35:26.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:35:26 smithi102 ceph-mon[35442]: osdmap e80: 12 total, 12 up, 12 in 2024-09-10T09:35:26.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:35:26 smithi102 ceph-mon[35442]: pgmap v290: 33 pgs: 14 creating+peering, 1 active+clean, 18 unknown; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T09:35:26.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:35:26 smithi102 ceph-mon[35442]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T09:35:27.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:27.543+0000 7f5e09600640 1 -- 172.21.15.5:0/3737458811 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v82) ==== 171+0+0 (secure 0 0 0) 0x7f5e0c104210 con 0x7f5e0c1047e0 2024-09-10T09:35:27.544 INFO:teuthology.orchestra.run.smithi005.stderr:pool 'cephfs_data' created 2024-09-10T09:35:27.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:27.546+0000 7f5e12600640 1 -- 172.21.15.5:0/3737458811 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f5df40a2c00 msgr2=0x7f5df40a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:35:27.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:27.546+0000 7f5e12600640 1 --2- 172.21.15.5:0/3737458811 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f5df40a2c00 0x7f5df40a50c0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f5e000096c0 tx=0x7f5e000046c0 comp rx=0 tx=0).stop 2024-09-10T09:35:27.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:27.546+0000 7f5e12600640 1 -- 172.21.15.5:0/3737458811 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5e0c1047e0 msgr2=0x7f5e0c19aa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:35:27.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:27.546+0000 7f5e12600640 1 --2- 172.21.15.5:0/3737458811 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5e0c1047e0 0x7f5e0c19aa30 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f5df800e9f0 tx=0x7f5df800eeb0 comp rx=0 tx=0).stop 2024-09-10T09:35:27.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:27.547+0000 7f5e12600640 1 -- 172.21.15.5:0/3737458811 shutdown_connections 2024-09-10T09:35:27.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:27.547+0000 7f5e12600640 1 --2- 172.21.15.5:0/3737458811 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f5df40a2c00 0x7f5df40a50c0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:35:27.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:27.547+0000 7f5e12600640 1 --2- 172.21.15.5:0/3737458811 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5e0c108f60 0x7f5e0c19edc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:35:27.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:27.547+0000 7f5e12600640 1 --2- 172.21.15.5:0/3737458811 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5e0c1047e0 0x7f5e0c19aa30 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:35:27.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:27.547+0000 7f5e12600640 1 --2- 172.21.15.5:0/3737458811 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5e0c103e30 0x7f5e0c19a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:35:27.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:27.547+0000 7f5e12600640 1 -- 172.21.15.5:0/3737458811 >> 172.21.15.5:0/3737458811 conn(0x7f5e0c0ff910 msgr2=0x7f5e0c100bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:35:27.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:27.547+0000 7f5e12600640 1 -- 172.21.15.5:0/3737458811 shutdown_connections 2024-09-10T09:35:27.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:27.547+0000 7f5e12600640 1 -- 172.21.15.5:0/3737458811 wait complete. 2024-09-10T09:35:27.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:35:27 smithi005 ceph-mon[30350]: osdmap e81: 12 total, 12 up, 12 in 2024-09-10T09:35:27.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:35:27 smithi005 ceph-mon[30350]: from='client.? 172.21.15.5:0/3737458811' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-09-10T09:35:27.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:35:27 smithi159 ceph-mon[35513]: osdmap e81: 12 total, 12 up, 12 in 2024-09-10T09:35:27.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:35:27 smithi159 ceph-mon[35513]: from='client.? 172.21.15.5:0/3737458811' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-09-10T09:35:27.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:35:27 smithi102 ceph-mon[35442]: osdmap e81: 12 total, 12 up, 12 in 2024-09-10T09:35:27.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:35:27 smithi102 ceph-mon[35442]: from='client.? 172.21.15.5:0/3737458811' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-09-10T09:35:28.437 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-10T09:35:28.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:35:28 smithi005 ceph-mon[30350]: pgmap v292: 33 pgs: 14 creating+peering, 7 active+clean, 12 unknown; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T09:35:28.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:35:28 smithi005 ceph-mon[30350]: from='client.? 172.21.15.5:0/3737458811' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-09-10T09:35:28.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:35:28 smithi005 ceph-mon[30350]: osdmap e82: 12 total, 12 up, 12 in 2024-09-10T09:35:28.837 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:35:28.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:35:28 smithi159 ceph-mon[35513]: pgmap v292: 33 pgs: 14 creating+peering, 7 active+clean, 12 unknown; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T09:35:28.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:35:28 smithi159 ceph-mon[35513]: from='client.? 172.21.15.5:0/3737458811' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-09-10T09:35:28.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:35:28 smithi159 ceph-mon[35513]: osdmap e82: 12 total, 12 up, 12 in 2024-09-10T09:35:28.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:35:28 smithi102 ceph-mon[35442]: pgmap v292: 33 pgs: 14 creating+peering, 7 active+clean, 12 unknown; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T09:35:28.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:35:28 smithi102 ceph-mon[35442]: from='client.? 172.21.15.5:0/3737458811' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-09-10T09:35:28.885 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:35:28 smithi102 ceph-mon[35442]: osdmap e82: 12 total, 12 up, 12 in 2024-09-10T09:35:29.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:29.387+0000 7f72a7400640 1 -- 172.21.15.5:0/908322200 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f72a0109120 msgr2=0x7f72a0110ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:35:29.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:29.387+0000 7f72a7400640 1 --2- 172.21.15.5:0/908322200 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f72a0109120 0x7f72a0110ca0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f7298009ca0 tx=0x7f729802a8c0 comp rx=0 tx=0).stop 2024-09-10T09:35:29.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:29.387+0000 7f72a7400640 1 -- 172.21.15.5:0/908322200 shutdown_connections 2024-09-10T09:35:29.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:29.387+0000 7f72a7400640 1 --2- 172.21.15.5:0/908322200 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f72a0109120 0x7f72a0110ca0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:35:29.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:29.387+0000 7f72a7400640 1 --2- 172.21.15.5:0/908322200 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f72a01049a0 0x7f72a01089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:35:29.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:29.387+0000 7f72a7400640 1 --2- 172.21.15.5:0/908322200 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f72a0103ff0 0x7f72a01043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:35:29.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:29.387+0000 7f72a7400640 1 -- 172.21.15.5:0/908322200 >> 172.21.15.5:0/908322200 conn(0x7f72a00ffa90 msgr2=0x7f72a0101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:35:29.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:29.387+0000 7f72a7400640 1 -- 172.21.15.5:0/908322200 shutdown_connections 2024-09-10T09:35:29.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:29.387+0000 7f72a7400640 1 -- 172.21.15.5:0/908322200 wait complete. 2024-09-10T09:35:29.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:29.388+0000 7f72a7400640 1 Processor -- start 2024-09-10T09:35:29.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:29.389+0000 7f72a7400640 1 -- start start 2024-09-10T09:35:29.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:29.389+0000 7f72a7400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f72a0103ff0 0x7f72a019a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:35:29.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:29.389+0000 7f72a7400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f72a01049a0 0x7f72a019aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:35:29.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:29.389+0000 7f72a7400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f72a0109120 0x7f72a019ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:35:29.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:29.389+0000 7f72a7400640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f72a0114150 con 0x7f72a0103ff0 2024-09-10T09:35:29.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:29.389+0000 7f72a7400640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f72a0113fd0 con 0x7f72a01049a0 2024-09-10T09:35:29.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:29.389+0000 7f72a7400640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f72a01142d0 con 0x7f72a0109120 2024-09-10T09:35:29.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:29.389+0000 7f72a4a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f72a0103ff0 0x7f72a019a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:35:29.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:29.389+0000 7f72a4a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f72a0103ff0 0x7f72a019a5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33972/0 (socket says 172.21.15.5:33972) 2024-09-10T09:35:29.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:29.389+0000 7f72a4a00640 1 -- 172.21.15.5:0/2837009389 learned_addr learned my addr 172.21.15.5:0/2837009389 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:35:29.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:29.389+0000 7f729fe00640 1 --2- 172.21.15.5:0/2837009389 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f72a01049a0 0x7f72a019aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:35:29.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:29.390+0000 7f72a5400640 1 --2- 172.21.15.5:0/2837009389 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f72a0109120 0x7f72a019ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:35:29.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:29.390+0000 7f72a4a00640 1 -- 172.21.15.5:0/2837009389 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f72a0109120 msgr2=0x7f72a019ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:35:29.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:29.390+0000 7f72a4a00640 1 --2- 172.21.15.5:0/2837009389 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f72a0109120 0x7f72a019ee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:35:29.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:29.390+0000 7f72a4a00640 1 -- 172.21.15.5:0/2837009389 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f72a01049a0 msgr2=0x7f72a019aaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:35:29.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:29.390+0000 7f72a4a00640 1 --2- 172.21.15.5:0/2837009389 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f72a01049a0 0x7f72a019aaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:35:29.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:29.390+0000 7f72a4a00640 1 -- 172.21.15.5:0/2837009389 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72a019f560 con 0x7f72a0103ff0 2024-09-10T09:35:29.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:29.390+0000 7f729fe00640 1 --2- 172.21.15.5:0/2837009389 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f72a01049a0 0x7f72a019aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:35:29.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:29.390+0000 7f72a4a00640 1 --2- 172.21.15.5:0/2837009389 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f72a0103ff0 0x7f72a019a5b0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f729400e9f0 tx=0x7f729400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:35:29.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:29.390+0000 7f729d600640 1 -- 172.21.15.5:0/2837009389 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f729400cd80 con 0x7f72a0103ff0 2024-09-10T09:35:29.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:29.391+0000 7f72a7400640 1 -- 172.21.15.5:0/2837009389 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72a019f850 con 0x7f72a0103ff0 2024-09-10T09:35:29.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:29.391+0000 7f729d600640 1 -- 172.21.15.5:0/2837009389 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7294010910 con 0x7f72a0103ff0 2024-09-10T09:35:29.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:29.391+0000 7f729d600640 1 -- 172.21.15.5:0/2837009389 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72940188f0 con 0x7f72a0103ff0 2024-09-10T09:35:29.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:29.391+0000 7f72a7400640 1 -- 172.21.15.5:0/2837009389 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72a01a7130 con 0x7f72a0103ff0 2024-09-10T09:35:29.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:29.392+0000 7f72a7400640 1 -- 172.21.15.5:0/2837009389 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f72a01043d0 con 0x7f72a0103ff0 2024-09-10T09:35:29.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:29.394+0000 7f729d600640 1 -- 172.21.15.5:0/2837009389 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7f7294010ac0 con 0x7f72a0103ff0 2024-09-10T09:35:29.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:29.396+0000 7f729d600640 1 --2- 172.21.15.5:0/2837009389 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f72880a2b80 0x7f72880a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:35:29.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:29.396+0000 7f729d600640 1 -- 172.21.15.5:0/2837009389 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7879+0+0 (secure 0 0 0) 0x7f72940cf2c0 con 0x7f72a0103ff0 2024-09-10T09:35:29.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:29.396+0000 7f729fe00640 1 --2- 172.21.15.5:0/2837009389 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f72880a2b80 0x7f72880a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:35:29.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:29.397+0000 7f729fe00640 1 --2- 172.21.15.5:0/2837009389 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f72880a2b80 0x7f72880a5040 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f72a00ffeb0 tx=0x7f728c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:35:29.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:29.400+0000 7f729d600640 1 -- 172.21.15.5:0/2837009389 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f729409bce0 con 0x7f72a0103ff0 2024-09-10T09:35:29.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:29.625+0000 7f72a7400640 1 -- 172.21.15.5:0/2837009389 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f72a019b910 con 0x7f72a0103ff0 2024-09-10T09:35:29.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:35:29 smithi005 ceph-mon[30350]: osdmap e83: 12 total, 12 up, 12 in 2024-09-10T09:35:29.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:35:29 smithi159 ceph-mon[35513]: osdmap e83: 12 total, 12 up, 12 in 2024-09-10T09:35:29.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:35:29 smithi102 ceph-mon[35442]: osdmap e83: 12 total, 12 up, 12 in 2024-09-10T09:35:30.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:30.562+0000 7f729d600640 1 -- 172.21.15.5:0/2837009389 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) ==== 172+0+0 (secure 0 0 0) 0x7f729409ffe0 con 0x7f72a0103ff0 2024-09-10T09:35:30.564 INFO:teuthology.orchestra.run.smithi005.stderr:new fs with metadata pool 2 and data pool 3 2024-09-10T09:35:30.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:30.566+0000 7f72a7400640 1 -- 172.21.15.5:0/2837009389 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f72880a2b80 msgr2=0x7f72880a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:35:30.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:30.566+0000 7f72a7400640 1 --2- 172.21.15.5:0/2837009389 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f72880a2b80 0x7f72880a5040 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f72a00ffeb0 tx=0x7f728c005f90 comp rx=0 tx=0).stop 2024-09-10T09:35:30.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:30.566+0000 7f72a7400640 1 -- 172.21.15.5:0/2837009389 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f72a0103ff0 msgr2=0x7f72a019a5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:35:30.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:30.566+0000 7f72a7400640 1 --2- 172.21.15.5:0/2837009389 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f72a0103ff0 0x7f72a019a5b0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f729400e9f0 tx=0x7f729400eeb0 comp rx=0 tx=0).stop 2024-09-10T09:35:30.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:30.566+0000 7f72a7400640 1 -- 172.21.15.5:0/2837009389 shutdown_connections 2024-09-10T09:35:30.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:30.566+0000 7f72a7400640 1 --2- 172.21.15.5:0/2837009389 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f72880a2b80 0x7f72880a5040 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:35:30.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:30.566+0000 7f72a7400640 1 --2- 172.21.15.5:0/2837009389 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f72a0109120 0x7f72a019ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:35:30.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:30.566+0000 7f72a7400640 1 --2- 172.21.15.5:0/2837009389 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f72a01049a0 0x7f72a019aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:35:30.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:30.566+0000 7f72a7400640 1 --2- 172.21.15.5:0/2837009389 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f72a0103ff0 0x7f72a019a5b0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:35:30.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:30.567+0000 7f72a7400640 1 -- 172.21.15.5:0/2837009389 >> 172.21.15.5:0/2837009389 conn(0x7f72a00ffa90 msgr2=0x7f72a0101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:35:30.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:30.567+0000 7f72a7400640 1 -- 172.21.15.5:0/2837009389 shutdown_connections 2024-09-10T09:35:30.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:30.567+0000 7f72a7400640 1 -- 172.21.15.5:0/2837009389 wait complete. 2024-09-10T09:35:30.837 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:35:30 smithi102 ceph-mon[35442]: pgmap v295: 97 pgs: 14 creating+peering, 31 active+clean, 52 unknown; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T09:35:30.837 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:35:30 smithi102 ceph-mon[35442]: osdmap e84: 12 total, 12 up, 12 in 2024-09-10T09:35:30.837 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:35:30 smithi102 ceph-mon[35442]: from='client.? 172.21.15.5:0/2837009389' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-09-10T09:35:30.837 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:35:30 smithi102 ceph-mon[35442]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-09-10T09:35:30.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:35:30 smithi159 ceph-mon[35513]: pgmap v295: 97 pgs: 14 creating+peering, 31 active+clean, 52 unknown; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T09:35:30.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:35:30 smithi159 ceph-mon[35513]: osdmap e84: 12 total, 12 up, 12 in 2024-09-10T09:35:30.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:35:30 smithi159 ceph-mon[35513]: from='client.? 172.21.15.5:0/2837009389' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-09-10T09:35:30.851 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:35:30 smithi159 ceph-mon[35513]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-09-10T09:35:30.918 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:35:30 smithi005 ceph-mon[30350]: pgmap v295: 97 pgs: 14 creating+peering, 31 active+clean, 52 unknown; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T09:35:30.918 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:35:30 smithi005 ceph-mon[30350]: osdmap e84: 12 total, 12 up, 12 in 2024-09-10T09:35:30.918 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:35:30 smithi005 ceph-mon[30350]: from='client.? 172.21.15.5:0/2837009389' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-09-10T09:35:30.918 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:35:30 smithi005 ceph-mon[30350]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-09-10T09:35:31.332 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2024-09-10T09:35:31.332 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2024-09-10T09:35:31.716 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:35:31.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:35:31 smithi159 ceph-mon[35513]: from='client.? 172.21.15.5:0/2837009389' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-09-10T09:35:31.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:35:31 smithi159 ceph-mon[35513]: osdmap e85: 12 total, 12 up, 12 in 2024-09-10T09:35:31.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:35:31 smithi159 ceph-mon[35513]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-09-10T09:35:31.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:35:31 smithi159 ceph-mon[35513]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-09-10T09:35:31.854 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:35:31 smithi005 ceph-mon[30350]: from='client.? 172.21.15.5:0/2837009389' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-09-10T09:35:31.854 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:35:31 smithi005 ceph-mon[30350]: osdmap e85: 12 total, 12 up, 12 in 2024-09-10T09:35:31.854 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:35:31 smithi005 ceph-mon[30350]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-09-10T09:35:31.854 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:35:31 smithi005 ceph-mon[30350]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-09-10T09:35:31.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:35:31 smithi102 ceph-mon[35442]: from='client.? 172.21.15.5:0/2837009389' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-09-10T09:35:31.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:35:31 smithi102 ceph-mon[35442]: osdmap e85: 12 total, 12 up, 12 in 2024-09-10T09:35:31.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:35:31 smithi102 ceph-mon[35442]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-09-10T09:35:31.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:35:31 smithi102 ceph-mon[35442]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-09-10T09:35:32.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:32.331+0000 7fd200800640 1 -- 172.21.15.5:0/4204422848 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd1f8109140 msgr2=0x7fd1f8110cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:35:32.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:32.331+0000 7fd200800640 1 --2- 172.21.15.5:0/4204422848 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd1f8109140 0x7fd1f8110cc0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fd1f4009ca0 tx=0x7fd1f402a8c0 comp rx=0 tx=0).stop 2024-09-10T09:35:32.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:32.331+0000 7fd200800640 1 -- 172.21.15.5:0/4204422848 shutdown_connections 2024-09-10T09:35:32.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:32.331+0000 7fd200800640 1 --2- 172.21.15.5:0/4204422848 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd1f8109140 0x7fd1f8110cc0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:35:32.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:32.331+0000 7fd200800640 1 --2- 172.21.15.5:0/4204422848 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd1f81049c0 0x7fd1f8108a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:35:32.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:32.331+0000 7fd200800640 1 --2- 172.21.15.5:0/4204422848 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd1f8104010 0x7fd1f81043f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:35:32.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:32.331+0000 7fd200800640 1 -- 172.21.15.5:0/4204422848 >> 172.21.15.5:0/4204422848 conn(0x7fd1f80ffa50 msgr2=0x7fd1f8101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:35:32.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:32.331+0000 7fd200800640 1 -- 172.21.15.5:0/4204422848 shutdown_connections 2024-09-10T09:35:32.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:32.331+0000 7fd200800640 1 -- 172.21.15.5:0/4204422848 wait complete. 2024-09-10T09:35:32.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:32.332+0000 7fd200800640 1 Processor -- start 2024-09-10T09:35:32.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:32.332+0000 7fd200800640 1 -- start start 2024-09-10T09:35:32.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:32.332+0000 7fd200800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd1f8104010 0x7fd1f819a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:35:32.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:32.332+0000 7fd200800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd1f81049c0 0x7fd1f819ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:35:32.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:32.332+0000 7fd200800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd1f8109140 0x7fd1f819ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:35:32.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:32.332+0000 7fd200800640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd1f8114170 con 0x7fd1f8104010 2024-09-10T09:35:32.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:32.332+0000 7fd200800640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fd1f8113ff0 con 0x7fd1f81049c0 2024-09-10T09:35:32.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:32.332+0000 7fd200800640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fd1f81142f0 con 0x7fd1f8109140 2024-09-10T09:35:32.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:32.333+0000 7fd1fde00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd1f8104010 0x7fd1f819a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:35:32.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:32.333+0000 7fd1fde00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd1f8104010 0x7fd1f819a630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41196/0 (socket says 172.21.15.5:41196) 2024-09-10T09:35:32.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:32.333+0000 7fd1fde00640 1 -- 172.21.15.5:0/2948751427 learned_addr learned my addr 172.21.15.5:0/2948751427 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:35:32.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:32.333+0000 7fd1fe800640 1 --2- 172.21.15.5:0/2948751427 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd1f8109140 0x7fd1f819ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:35:32.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:32.333+0000 7fd1fd400640 1 --2- 172.21.15.5:0/2948751427 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd1f81049c0 0x7fd1f819ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:35:32.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:32.333+0000 7fd1fde00640 1 -- 172.21.15.5:0/2948751427 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd1f8109140 msgr2=0x7fd1f819ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:35:32.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:32.333+0000 7fd1fde00640 1 --2- 172.21.15.5:0/2948751427 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd1f8109140 0x7fd1f819ef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:35:32.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:32.333+0000 7fd1fde00640 1 -- 172.21.15.5:0/2948751427 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd1f81049c0 msgr2=0x7fd1f819ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:35:32.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:32.333+0000 7fd1fde00640 1 --2- 172.21.15.5:0/2948751427 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd1f81049c0 0x7fd1f819ab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:35:32.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:32.333+0000 7fd1fde00640 1 -- 172.21.15.5:0/2948751427 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1f819f5e0 con 0x7fd1f8104010 2024-09-10T09:35:32.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:32.333+0000 7fd1fde00640 1 --2- 172.21.15.5:0/2948751427 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd1f8104010 0x7fd1f819a630 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fd1ec00e9f0 tx=0x7fd1ec00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:35:32.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:32.334+0000 7fd1f2a00640 1 -- 172.21.15.5:0/2948751427 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1ec00cd80 con 0x7fd1f8104010 2024-09-10T09:35:32.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:32.334+0000 7fd200800640 1 -- 172.21.15.5:0/2948751427 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1f819f8d0 con 0x7fd1f8104010 2024-09-10T09:35:32.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:32.334+0000 7fd1f2a00640 1 -- 172.21.15.5:0/2948751427 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd1ec010910 con 0x7fd1f8104010 2024-09-10T09:35:32.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:32.334+0000 7fd1f2a00640 1 -- 172.21.15.5:0/2948751427 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1ec018990 con 0x7fd1f8104010 2024-09-10T09:35:32.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:32.334+0000 7fd200800640 1 -- 172.21.15.5:0/2948751427 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1f81a71b0 con 0x7fd1f8104010 2024-09-10T09:35:32.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:32.335+0000 7fd200800640 1 -- 172.21.15.5:0/2948751427 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd1f81043f0 con 0x7fd1f8104010 2024-09-10T09:35:32.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:32.337+0000 7fd1f2a00640 1 -- 172.21.15.5:0/2948751427 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7fd1ec010ac0 con 0x7fd1f8104010 2024-09-10T09:35:32.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:32.338+0000 7fd1f2a00640 1 --2- 172.21.15.5:0/2948751427 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fd1e00a2b80 0x7fd1e00a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:35:32.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:32.338+0000 7fd1f2a00640 1 -- 172.21.15.5:0/2948751427 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7fd1ec0cf2a0 con 0x7fd1f8104010 2024-09-10T09:35:32.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:32.339+0000 7fd1fd400640 1 --2- 172.21.15.5:0/2948751427 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fd1e00a2b80 0x7fd1e00a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:35:32.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:32.339+0000 7fd1fd400640 1 --2- 172.21.15.5:0/2948751427 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fd1e00a2b80 0x7fd1e00a5040 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fd1f810ab10 tx=0x7fd1e8005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:35:32.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:32.343+0000 7fd1f2a00640 1 -- 172.21.15.5:0/2948751427 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd1ec09bc60 con 0x7fd1f8104010 2024-09-10T09:35:32.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:32.536+0000 7fd200800640 1 -- 172.21.15.5:0/2948751427 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) -- 0x7fd1f802d7c0 con 0x7fd1f8104010 2024-09-10T09:35:32.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:32.581+0000 7fd1f2a00640 1 -- 172.21.15.5:0/2948751427 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v86) ==== 157+0+0 (secure 0 0 0) 0x7fd1ec09ff60 con 0x7fd1f8104010 2024-09-10T09:35:32.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:32.584+0000 7fd200800640 1 -- 172.21.15.5:0/2948751427 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fd1e00a2b80 msgr2=0x7fd1e00a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:35:32.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:32.584+0000 7fd200800640 1 --2- 172.21.15.5:0/2948751427 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fd1e00a2b80 0x7fd1e00a5040 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fd1f810ab10 tx=0x7fd1e8005f90 comp rx=0 tx=0).stop 2024-09-10T09:35:32.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:32.584+0000 7fd200800640 1 -- 172.21.15.5:0/2948751427 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd1f8104010 msgr2=0x7fd1f819a630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:35:32.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:32.584+0000 7fd200800640 1 --2- 172.21.15.5:0/2948751427 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd1f8104010 0x7fd1f819a630 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fd1ec00e9f0 tx=0x7fd1ec00eeb0 comp rx=0 tx=0).stop 2024-09-10T09:35:32.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:32.584+0000 7fd200800640 1 -- 172.21.15.5:0/2948751427 shutdown_connections 2024-09-10T09:35:32.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:32.584+0000 7fd200800640 1 --2- 172.21.15.5:0/2948751427 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fd1e00a2b80 0x7fd1e00a5040 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:35:32.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:32.584+0000 7fd200800640 1 --2- 172.21.15.5:0/2948751427 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd1f8109140 0x7fd1f819ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:35:32.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:32.584+0000 7fd200800640 1 --2- 172.21.15.5:0/2948751427 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd1f81049c0 0x7fd1f819ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:35:32.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:32.584+0000 7fd200800640 1 --2- 172.21.15.5:0/2948751427 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd1f8104010 0x7fd1f819a630 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:35:32.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:32.584+0000 7fd200800640 1 -- 172.21.15.5:0/2948751427 >> 172.21.15.5:0/2948751427 conn(0x7fd1f80ffa50 msgr2=0x7fd1f8100d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:35:32.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:32.584+0000 7fd200800640 1 -- 172.21.15.5:0/2948751427 shutdown_connections 2024-09-10T09:35:32.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:32.584+0000 7fd200800640 1 -- 172.21.15.5:0/2948751427 wait complete. 2024-09-10T09:35:32.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:35:32 smithi005 ceph-mon[30350]: pgmap v298: 97 pgs: 83 active+clean, 14 unknown; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 0 op/s 2024-09-10T09:35:32.807 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:35:32 smithi005 ceph-mon[30350]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-09-10T09:35:32.807 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:35:32 smithi005 ceph-mon[30350]: Cluster is now healthy 2024-09-10T09:35:32.807 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:35:32 smithi005 ceph-mon[30350]: mds.? [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] up:active 2024-09-10T09:35:32.807 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:35:32 smithi005 ceph-mon[30350]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-10T09:35:32.807 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:35:32 smithi005 ceph-mon[30350]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-10T09:35:32.807 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:35:32 smithi005 ceph-mon[30350]: from='client.? 172.21.15.5:0/2948751427' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T09:35:32.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:35:32 smithi159 ceph-mon[35513]: pgmap v298: 97 pgs: 83 active+clean, 14 unknown; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 0 op/s 2024-09-10T09:35:32.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:35:32 smithi159 ceph-mon[35513]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-09-10T09:35:32.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:35:32 smithi159 ceph-mon[35513]: Cluster is now healthy 2024-09-10T09:35:32.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:35:32 smithi159 ceph-mon[35513]: mds.? [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] up:active 2024-09-10T09:35:32.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:35:32 smithi159 ceph-mon[35513]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-10T09:35:32.851 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:35:32 smithi159 ceph-mon[35513]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-10T09:35:32.851 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:35:32 smithi159 ceph-mon[35513]: from='client.? 172.21.15.5:0/2948751427' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T09:35:32.884 INFO:journalctl@ceph.mds.b.smithi102.stdout:Sep 10 09:35:32 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mds-b[65198]: 2024-09-10T09:35:32.477+0000 7f0951ce7640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-10T09:35:32.885 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:35:32 smithi102 ceph-mon[35442]: pgmap v298: 97 pgs: 83 active+clean, 14 unknown; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 0 op/s 2024-09-10T09:35:32.885 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:35:32 smithi102 ceph-mon[35442]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-09-10T09:35:32.885 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:35:32 smithi102 ceph-mon[35442]: Cluster is now healthy 2024-09-10T09:35:32.885 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:35:32 smithi102 ceph-mon[35442]: mds.? [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] up:active 2024-09-10T09:35:32.885 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:35:32 smithi102 ceph-mon[35442]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-10T09:35:32.885 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:35:32 smithi102 ceph-mon[35442]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-10T09:35:32.885 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:35:32 smithi102 ceph-mon[35442]: from='client.? 172.21.15.5:0/2948751427' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T09:35:33.300 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2024-09-10T09:35:33.680 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:35:33.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:35:33 smithi159 ceph-mon[35513]: from='client.? 172.21.15.5:0/2948751427' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-09-10T09:35:33.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:35:33 smithi159 ceph-mon[35513]: osdmap e86: 12 total, 12 up, 12 in 2024-09-10T09:35:33.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:35:33 smithi102 ceph-mon[35442]: from='client.? 172.21.15.5:0/2948751427' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-09-10T09:35:33.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:35:33 smithi102 ceph-mon[35442]: osdmap e86: 12 total, 12 up, 12 in 2024-09-10T09:35:33.989 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:35:33 smithi005 ceph-mon[30350]: from='client.? 172.21.15.5:0/2948751427' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-09-10T09:35:33.989 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:35:33 smithi005 ceph-mon[30350]: osdmap e86: 12 total, 12 up, 12 in 2024-09-10T09:35:34.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:34.253+0000 7f2f93400640 1 -- 172.21.15.5:0/325890949 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2f8c109120 msgr2=0x7f2f8c110ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:35:34.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:34.253+0000 7f2f93400640 1 --2- 172.21.15.5:0/325890949 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2f8c109120 0x7f2f8c110ca0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f2f84009ca0 tx=0x7f2f8402a8c0 comp rx=0 tx=0).stop 2024-09-10T09:35:34.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:34.253+0000 7f2f93400640 1 -- 172.21.15.5:0/325890949 shutdown_connections 2024-09-10T09:35:34.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:34.253+0000 7f2f93400640 1 --2- 172.21.15.5:0/325890949 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2f8c109120 0x7f2f8c110ca0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:35:34.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:34.253+0000 7f2f93400640 1 --2- 172.21.15.5:0/325890949 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2f8c1049a0 0x7f2f8c1089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:35:34.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:34.253+0000 7f2f93400640 1 --2- 172.21.15.5:0/325890949 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2f8c103ff0 0x7f2f8c1043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:35:34.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:34.253+0000 7f2f93400640 1 -- 172.21.15.5:0/325890949 >> 172.21.15.5:0/325890949 conn(0x7f2f8c0ffa90 msgr2=0x7f2f8c101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:35:34.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:34.253+0000 7f2f93400640 1 -- 172.21.15.5:0/325890949 shutdown_connections 2024-09-10T09:35:34.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:34.253+0000 7f2f93400640 1 -- 172.21.15.5:0/325890949 wait complete. 2024-09-10T09:35:34.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:34.254+0000 7f2f93400640 1 Processor -- start 2024-09-10T09:35:34.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:34.254+0000 7f2f93400640 1 -- start start 2024-09-10T09:35:34.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:34.255+0000 7f2f93400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2f8c103ff0 0x7f2f8c076700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:35:34.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:34.255+0000 7f2f93400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2f8c1049a0 0x7f2f8c076c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:35:34.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:34.255+0000 7f2f93400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2f8c109120 0x7f2f8c077180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:35:34.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:34.255+0000 7f2f93400640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f2f8c114130 con 0x7f2f8c103ff0 2024-09-10T09:35:34.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:34.255+0000 7f2f93400640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f2f8c113fb0 con 0x7f2f8c109120 2024-09-10T09:35:34.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:34.255+0000 7f2f93400640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f2f8c1142b0 con 0x7f2f8c1049a0 2024-09-10T09:35:34.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:34.255+0000 7f2f90a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2f8c103ff0 0x7f2f8c076700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:35:34.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:34.255+0000 7f2f90a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2f8c103ff0 0x7f2f8c076700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41222/0 (socket says 172.21.15.5:41222) 2024-09-10T09:35:34.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:34.255+0000 7f2f90a00640 1 -- 172.21.15.5:0/2117329130 learned_addr learned my addr 172.21.15.5:0/2117329130 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:35:34.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:34.255+0000 7f2f8be00640 1 --2- 172.21.15.5:0/2117329130 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2f8c1049a0 0x7f2f8c076c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:35:34.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:34.255+0000 7f2f91400640 1 --2- 172.21.15.5:0/2117329130 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2f8c109120 0x7f2f8c077180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:35:34.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:34.255+0000 7f2f90a00640 1 -- 172.21.15.5:0/2117329130 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2f8c1049a0 msgr2=0x7f2f8c076c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:35:34.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:34.255+0000 7f2f90a00640 1 --2- 172.21.15.5:0/2117329130 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2f8c1049a0 0x7f2f8c076c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:35:34.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:34.255+0000 7f2f90a00640 1 -- 172.21.15.5:0/2117329130 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2f8c109120 msgr2=0x7f2f8c077180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:35:34.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:34.255+0000 7f2f90a00640 1 --2- 172.21.15.5:0/2117329130 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2f8c109120 0x7f2f8c077180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:35:34.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:34.255+0000 7f2f90a00640 1 -- 172.21.15.5:0/2117329130 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f8c1aac30 con 0x7f2f8c103ff0 2024-09-10T09:35:34.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:34.256+0000 7f2f90a00640 1 --2- 172.21.15.5:0/2117329130 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2f8c103ff0 0x7f2f8c076700 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f2f8000ea20 tx=0x7f2f8000eee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:35:34.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:34.256+0000 7f2f89600640 1 -- 172.21.15.5:0/2117329130 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f8000cd80 con 0x7f2f8c103ff0 2024-09-10T09:35:34.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:34.256+0000 7f2f89600640 1 -- 172.21.15.5:0/2117329130 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2f80010910 con 0x7f2f8c103ff0 2024-09-10T09:35:34.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:34.256+0000 7f2f89600640 1 -- 172.21.15.5:0/2117329130 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f80018920 con 0x7f2f8c103ff0 2024-09-10T09:35:34.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:34.256+0000 7f2f93400640 1 -- 172.21.15.5:0/2117329130 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f8c1aaf20 con 0x7f2f8c103ff0 2024-09-10T09:35:34.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:34.256+0000 7f2f93400640 1 -- 172.21.15.5:0/2117329130 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f8c1ab3d0 con 0x7f2f8c103ff0 2024-09-10T09:35:34.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:34.258+0000 7f2f93400640 1 -- 172.21.15.5:0/2117329130 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f8c1054d0 con 0x7f2f8c103ff0 2024-09-10T09:35:34.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:34.259+0000 7f2f89600640 1 -- 172.21.15.5:0/2117329130 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7f2f80010ac0 con 0x7f2f8c103ff0 2024-09-10T09:35:34.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:34.261+0000 7f2f89600640 1 --2- 172.21.15.5:0/2117329130 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f2f740a2b80 0x7f2f740a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:35:34.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:34.261+0000 7f2f89600640 1 -- 172.21.15.5:0/2117329130 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8277+0+0 (secure 0 0 0) 0x7f2f800cf400 con 0x7f2f8c103ff0 2024-09-10T09:35:34.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:34.261+0000 7f2f8be00640 1 --2- 172.21.15.5:0/2117329130 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f2f740a2b80 0x7f2f740a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:35:34.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:34.262+0000 7f2f8be00640 1 --2- 172.21.15.5:0/2117329130 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f2f740a2b80 0x7f2f740a5040 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f2f8c077ec0 tx=0x7f2f7800b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:35:34.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:34.265+0000 7f2f89600640 1 -- 172.21.15.5:0/2117329130 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f2f8009bc80 con 0x7f2f8c103ff0 2024-09-10T09:35:34.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:34.452+0000 7f2f93400640 1 -- 172.21.15.5:0/2117329130 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f2f8c1ab7e0 con 0x7f2f8c103ff0 2024-09-10T09:35:34.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:35:34 smithi159 ceph-mon[35513]: pgmap v300: 97 pgs: 97 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s wr, 0 op/s 2024-09-10T09:35:34.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:35:34 smithi159 ceph-mon[35513]: from='client.? 172.21.15.5:0/2117329130' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-09-10T09:35:34.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:35:34 smithi102 ceph-mon[35442]: pgmap v300: 97 pgs: 97 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s wr, 0 op/s 2024-09-10T09:35:34.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:35:34 smithi102 ceph-mon[35442]: from='client.? 172.21.15.5:0/2117329130' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-09-10T09:35:35.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:35:34 smithi005 ceph-mon[30350]: pgmap v300: 97 pgs: 97 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s wr, 0 op/s 2024-09-10T09:35:35.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:35:34 smithi005 ceph-mon[30350]: from='client.? 172.21.15.5:0/2117329130' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-09-10T09:35:35.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:35.598+0000 7f2f89600640 1 -- 172.21.15.5:0/2117329130 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v88) ==== 228+0+0 (secure 0 0 0) 0x7f2f8009ff80 con 0x7f2f8c103ff0 2024-09-10T09:35:35.599 INFO:teuthology.orchestra.run.smithi005.stderr:pool 'cephfs_data_ec' created 2024-09-10T09:35:35.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:35.601+0000 7f2f93400640 1 -- 172.21.15.5:0/2117329130 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f2f740a2b80 msgr2=0x7f2f740a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:35:35.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:35.601+0000 7f2f93400640 1 --2- 172.21.15.5:0/2117329130 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f2f740a2b80 0x7f2f740a5040 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f2f8c077ec0 tx=0x7f2f7800b410 comp rx=0 tx=0).stop 2024-09-10T09:35:35.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:35.601+0000 7f2f93400640 1 -- 172.21.15.5:0/2117329130 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2f8c103ff0 msgr2=0x7f2f8c076700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:35:35.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:35.601+0000 7f2f93400640 1 --2- 172.21.15.5:0/2117329130 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2f8c103ff0 0x7f2f8c076700 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f2f8000ea20 tx=0x7f2f8000eee0 comp rx=0 tx=0).stop 2024-09-10T09:35:35.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:35.601+0000 7f2f93400640 1 -- 172.21.15.5:0/2117329130 shutdown_connections 2024-09-10T09:35:35.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:35.601+0000 7f2f93400640 1 --2- 172.21.15.5:0/2117329130 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f2f740a2b80 0x7f2f740a5040 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:35:35.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:35.601+0000 7f2f93400640 1 --2- 172.21.15.5:0/2117329130 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2f8c109120 0x7f2f8c077180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:35:35.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:35.601+0000 7f2f93400640 1 --2- 172.21.15.5:0/2117329130 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2f8c1049a0 0x7f2f8c076c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:35:35.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:35.601+0000 7f2f93400640 1 --2- 172.21.15.5:0/2117329130 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2f8c103ff0 0x7f2f8c076700 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:35:35.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:35.602+0000 7f2f93400640 1 -- 172.21.15.5:0/2117329130 >> 172.21.15.5:0/2117329130 conn(0x7f2f8c0ffa90 msgr2=0x7f2f8c101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:35:35.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:35.602+0000 7f2f93400640 1 -- 172.21.15.5:0/2117329130 shutdown_connections 2024-09-10T09:35:35.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:35.602+0000 7f2f93400640 1 -- 172.21.15.5:0/2117329130 wait complete. 2024-09-10T09:35:35.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:35:35 smithi159 ceph-mon[35513]: osdmap e87: 12 total, 12 up, 12 in 2024-09-10T09:35:35.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:35:35 smithi102 ceph-mon[35442]: osdmap e87: 12 total, 12 up, 12 in 2024-09-10T09:35:36.023 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:35:35 smithi005 ceph-mon[30350]: osdmap e87: 12 total, 12 up, 12 in 2024-09-10T09:35:36.547 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2024-09-10T09:35:36.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:35:36 smithi005 ceph-mon[30350]: pgmap v302: 97 pgs: 97 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s wr, 7 op/s 2024-09-10T09:35:36.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:35:36 smithi005 ceph-mon[30350]: from='client.? 172.21.15.5:0/2117329130' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-09-10T09:35:36.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:35:36 smithi005 ceph-mon[30350]: osdmap e88: 12 total, 12 up, 12 in 2024-09-10T09:35:36.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:35:36 smithi102 ceph-mon[35442]: pgmap v302: 97 pgs: 97 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s wr, 7 op/s 2024-09-10T09:35:36.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:35:36 smithi102 ceph-mon[35442]: from='client.? 172.21.15.5:0/2117329130' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-09-10T09:35:36.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:35:36 smithi102 ceph-mon[35442]: osdmap e88: 12 total, 12 up, 12 in 2024-09-10T09:35:36.945 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:35:37.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:35:36 smithi159 ceph-mon[35513]: pgmap v302: 97 pgs: 97 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s wr, 7 op/s 2024-09-10T09:35:37.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:35:36 smithi159 ceph-mon[35513]: from='client.? 172.21.15.5:0/2117329130' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-09-10T09:35:37.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:35:36 smithi159 ceph-mon[35513]: osdmap e88: 12 total, 12 up, 12 in 2024-09-10T09:35:37.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:37.528+0000 7f9fddc00640 1 -- 172.21.15.5:0/2206761098 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9fd8108f60 msgr2=0x7f9fd8110ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:35:37.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:37.528+0000 7f9fddc00640 1 --2- 172.21.15.5:0/2206761098 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9fd8108f60 0x7f9fd8110ae0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f9fd0009ca0 tx=0x7f9fd002a8c0 comp rx=0 tx=0).stop 2024-09-10T09:35:37.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:37.528+0000 7f9fddc00640 1 -- 172.21.15.5:0/2206761098 shutdown_connections 2024-09-10T09:35:37.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:37.528+0000 7f9fddc00640 1 --2- 172.21.15.5:0/2206761098 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9fd8108f60 0x7f9fd8110ae0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:35:37.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:37.528+0000 7f9fddc00640 1 --2- 172.21.15.5:0/2206761098 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9fd81047e0 0x7f9fd8108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:35:37.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:37.528+0000 7f9fddc00640 1 --2- 172.21.15.5:0/2206761098 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9fd8103e30 0x7f9fd8104210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:35:37.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:37.528+0000 7f9fddc00640 1 -- 172.21.15.5:0/2206761098 >> 172.21.15.5:0/2206761098 conn(0x7f9fd80ff8d0 msgr2=0x7f9fd8101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:35:37.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:37.528+0000 7f9fddc00640 1 -- 172.21.15.5:0/2206761098 shutdown_connections 2024-09-10T09:35:37.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:37.528+0000 7f9fddc00640 1 -- 172.21.15.5:0/2206761098 wait complete. 2024-09-10T09:35:37.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:37.529+0000 7f9fddc00640 1 Processor -- start 2024-09-10T09:35:37.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:37.529+0000 7f9fddc00640 1 -- start start 2024-09-10T09:35:37.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:37.529+0000 7f9fddc00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9fd8103e30 0x7f9fd819a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:35:37.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:37.529+0000 7f9fddc00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9fd81047e0 0x7f9fd819a930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:35:37.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:37.529+0000 7f9fddc00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9fd8108f60 0x7f9fd819ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:35:37.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:37.529+0000 7f9fddc00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f9fd8114010 con 0x7f9fd81047e0 2024-09-10T09:35:37.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:37.529+0000 7f9fddc00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f9fd8113e90 con 0x7f9fd8108f60 2024-09-10T09:35:37.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:37.529+0000 7f9fddc00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f9fd8114190 con 0x7f9fd8103e30 2024-09-10T09:35:37.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:37.530+0000 7f9fcfe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9fd81047e0 0x7f9fd819a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:35:37.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:37.530+0000 7f9fcfe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9fd81047e0 0x7f9fd819a930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41248/0 (socket says 172.21.15.5:41248) 2024-09-10T09:35:37.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:37.530+0000 7f9fcfe00640 1 -- 172.21.15.5:0/4044314694 learned_addr learned my addr 172.21.15.5:0/4044314694 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:35:37.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:37.530+0000 7f9fdd200640 1 --2- 172.21.15.5:0/4044314694 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9fd8108f60 0x7f9fd819ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:35:37.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:37.530+0000 7f9fd7e00640 1 --2- 172.21.15.5:0/4044314694 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9fd8103e30 0x7f9fd819a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:35:37.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:37.530+0000 7f9fcfe00640 1 -- 172.21.15.5:0/4044314694 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9fd8103e30 msgr2=0x7f9fd819a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:35:37.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:37.530+0000 7f9fcfe00640 1 --2- 172.21.15.5:0/4044314694 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9fd8103e30 0x7f9fd819a3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:35:37.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:37.530+0000 7f9fcfe00640 1 -- 172.21.15.5:0/4044314694 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9fd8108f60 msgr2=0x7f9fd819ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:35:37.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:37.530+0000 7f9fcfe00640 1 --2- 172.21.15.5:0/4044314694 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9fd8108f60 0x7f9fd819ecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:35:37.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:37.530+0000 7f9fcfe00640 1 -- 172.21.15.5:0/4044314694 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9fd819f3a0 con 0x7f9fd81047e0 2024-09-10T09:35:37.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:37.530+0000 7f9fd7e00640 1 --2- 172.21.15.5:0/4044314694 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9fd8103e30 0x7f9fd819a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:35:37.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:37.530+0000 7f9fcfe00640 1 --2- 172.21.15.5:0/4044314694 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9fd81047e0 0x7f9fd819a930 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f9fc000e9f0 tx=0x7f9fc000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:35:37.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:37.530+0000 7f9fd5600640 1 -- 172.21.15.5:0/4044314694 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fc000cd80 con 0x7f9fd81047e0 2024-09-10T09:35:37.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:37.531+0000 7f9fd5600640 1 -- 172.21.15.5:0/4044314694 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9fc0005500 con 0x7f9fd81047e0 2024-09-10T09:35:37.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:37.531+0000 7f9fddc00640 1 -- 172.21.15.5:0/4044314694 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9fd819f690 con 0x7f9fd81047e0 2024-09-10T09:35:37.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:37.531+0000 7f9fd5600640 1 -- 172.21.15.5:0/4044314694 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fc0010690 con 0x7f9fd81047e0 2024-09-10T09:35:37.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:37.531+0000 7f9fddc00640 1 -- 172.21.15.5:0/4044314694 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9fd81a6f70 con 0x7f9fd81047e0 2024-09-10T09:35:37.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:37.532+0000 7f9fddc00640 1 -- 172.21.15.5:0/4044314694 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9fd8104210 con 0x7f9fd81047e0 2024-09-10T09:35:37.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:37.534+0000 7f9fd5600640 1 -- 172.21.15.5:0/4044314694 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7f9fc00046c0 con 0x7f9fd81047e0 2024-09-10T09:35:37.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:37.535+0000 7f9fd5600640 1 --2- 172.21.15.5:0/4044314694 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f9fbc0a2c00 0x7f9fbc0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:35:37.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:37.535+0000 7f9fd5600640 1 -- 172.21.15.5:0/4044314694 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8770+0+0 (secure 0 0 0) 0x7f9fc00cf870 con 0x7f9fd81047e0 2024-09-10T09:35:37.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:37.535+0000 7f9fd7e00640 1 --2- 172.21.15.5:0/4044314694 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f9fbc0a2c00 0x7f9fbc0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:35:37.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:37.536+0000 7f9fd7e00640 1 --2- 172.21.15.5:0/4044314694 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f9fbc0a2c00 0x7f9fbc0a50c0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f9fc8009c20 tx=0x7f9fc8005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:35:37.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:37.538+0000 7f9fd5600640 1 -- 172.21.15.5:0/4044314694 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f9fc00541b0 con 0x7f9fd81047e0 2024-09-10T09:35:37.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:37.725+0000 7f9fddc00640 1 -- 172.21.15.5:0/4044314694 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) -- 0x7f9fd802d7c0 con 0x7f9fd81047e0 2024-09-10T09:35:37.726 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:35:37 smithi005 ceph-mon[30350]: osdmap e89: 12 total, 12 up, 12 in 2024-09-10T09:35:37.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:35:37 smithi102 ceph-mon[35442]: osdmap e89: 12 total, 12 up, 12 in 2024-09-10T09:35:38.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:35:37 smithi159 ceph-mon[35513]: osdmap e89: 12 total, 12 up, 12 in 2024-09-10T09:35:38.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:38.621+0000 7f9fd5600640 1 -- 172.21.15.5:0/4044314694 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v91) ==== 169+0+0 (secure 0 0 0) 0x7f9fc009c550 con 0x7f9fd81047e0 2024-09-10T09:35:38.622 INFO:teuthology.orchestra.run.smithi005.stderr:set pool 4 allow_ec_overwrites to true 2024-09-10T09:35:38.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:38.625+0000 7f9fddc00640 1 -- 172.21.15.5:0/4044314694 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f9fbc0a2c00 msgr2=0x7f9fbc0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:35:38.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:38.625+0000 7f9fddc00640 1 --2- 172.21.15.5:0/4044314694 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f9fbc0a2c00 0x7f9fbc0a50c0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f9fc8009c20 tx=0x7f9fc8005f90 comp rx=0 tx=0).stop 2024-09-10T09:35:38.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:38.625+0000 7f9fddc00640 1 -- 172.21.15.5:0/4044314694 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9fd81047e0 msgr2=0x7f9fd819a930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:35:38.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:38.625+0000 7f9fddc00640 1 --2- 172.21.15.5:0/4044314694 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9fd81047e0 0x7f9fd819a930 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f9fc000e9f0 tx=0x7f9fc000edf0 comp rx=0 tx=0).stop 2024-09-10T09:35:38.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:38.626+0000 7f9fddc00640 1 -- 172.21.15.5:0/4044314694 shutdown_connections 2024-09-10T09:35:38.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:38.626+0000 7f9fddc00640 1 --2- 172.21.15.5:0/4044314694 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f9fbc0a2c00 0x7f9fbc0a50c0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:35:38.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:38.626+0000 7f9fddc00640 1 --2- 172.21.15.5:0/4044314694 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9fd8108f60 0x7f9fd819ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:35:38.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:38.626+0000 7f9fddc00640 1 --2- 172.21.15.5:0/4044314694 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9fd81047e0 0x7f9fd819a930 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:35:38.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:38.626+0000 7f9fddc00640 1 --2- 172.21.15.5:0/4044314694 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9fd8103e30 0x7f9fd819a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:35:38.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:38.626+0000 7f9fddc00640 1 -- 172.21.15.5:0/4044314694 >> 172.21.15.5:0/4044314694 conn(0x7f9fd80ff8d0 msgr2=0x7f9fd8100ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:35:38.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:38.626+0000 7f9fddc00640 1 -- 172.21.15.5:0/4044314694 shutdown_connections 2024-09-10T09:35:38.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:38.626+0000 7f9fddc00640 1 -- 172.21.15.5:0/4044314694 wait complete. 2024-09-10T09:35:38.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:35:38 smithi102 ceph-mon[35442]: pgmap v305: 129 pgs: 9 creating+peering, 20 unknown, 100 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s wr, 8 op/s 2024-09-10T09:35:38.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:35:38 smithi102 ceph-mon[35442]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T09:35:38.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:35:38 smithi102 ceph-mon[35442]: osdmap e90: 12 total, 12 up, 12 in 2024-09-10T09:35:38.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:35:38 smithi102 ceph-mon[35442]: from='client.? 172.21.15.5:0/4044314694' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-09-10T09:35:39.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:35:38 smithi005 ceph-mon[30350]: pgmap v305: 129 pgs: 9 creating+peering, 20 unknown, 100 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s wr, 8 op/s 2024-09-10T09:35:39.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:35:38 smithi005 ceph-mon[30350]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T09:35:39.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:35:38 smithi005 ceph-mon[30350]: osdmap e90: 12 total, 12 up, 12 in 2024-09-10T09:35:39.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:35:38 smithi005 ceph-mon[30350]: from='client.? 172.21.15.5:0/4044314694' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-09-10T09:35:39.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:35:38 smithi159 ceph-mon[35513]: pgmap v305: 129 pgs: 9 creating+peering, 20 unknown, 100 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s wr, 8 op/s 2024-09-10T09:35:39.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:35:38 smithi159 ceph-mon[35513]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T09:35:39.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:35:38 smithi159 ceph-mon[35513]: osdmap e90: 12 total, 12 up, 12 in 2024-09-10T09:35:39.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:35:38 smithi159 ceph-mon[35513]: from='client.? 172.21.15.5:0/4044314694' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-09-10T09:35:39.547 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph fs add_data_pool cephfs cephfs_data_ec 2024-09-10T09:35:39.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:35:39 smithi005 ceph-mon[30350]: from='client.? 172.21.15.5:0/4044314694' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-09-10T09:35:39.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:35:39 smithi005 ceph-mon[30350]: osdmap e91: 12 total, 12 up, 12 in 2024-09-10T09:35:39.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:35:39 smithi102 ceph-mon[35442]: from='client.? 172.21.15.5:0/4044314694' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-09-10T09:35:39.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:35:39 smithi102 ceph-mon[35442]: osdmap e91: 12 total, 12 up, 12 in 2024-09-10T09:35:39.927 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:35:40.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:35:39 smithi159 ceph-mon[35513]: from='client.? 172.21.15.5:0/4044314694' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-09-10T09:35:40.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:35:39 smithi159 ceph-mon[35513]: osdmap e91: 12 total, 12 up, 12 in 2024-09-10T09:35:40.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:40.442+0000 7f7b71400640 1 -- 172.21.15.5:0/794164187 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7b6c077040 msgr2=0x7f7b6c1124f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:35:40.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:40.442+0000 7f7b71400640 1 --2- 172.21.15.5:0/794164187 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7b6c077040 0x7f7b6c1124f0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f7b58009900 tx=0x7f7b5802a410 comp rx=0 tx=0).stop 2024-09-10T09:35:40.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:40.442+0000 7f7b71400640 1 -- 172.21.15.5:0/794164187 shutdown_connections 2024-09-10T09:35:40.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:40.442+0000 7f7b71400640 1 --2- 172.21.15.5:0/794164187 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7b6c112a30 0x7f7b6c114e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:35:40.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:40.442+0000 7f7b71400640 1 --2- 172.21.15.5:0/794164187 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7b6c077040 0x7f7b6c1124f0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:35:40.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:40.442+0000 7f7b71400640 1 --2- 172.21.15.5:0/794164187 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7b6c076720 0x7f7b6c076b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:35:40.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:40.442+0000 7f7b71400640 1 -- 172.21.15.5:0/794164187 >> 172.21.15.5:0/794164187 conn(0x7f7b6c0ffa90 msgr2=0x7f7b6c101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:35:40.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:40.443+0000 7f7b71400640 1 -- 172.21.15.5:0/794164187 shutdown_connections 2024-09-10T09:35:40.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:40.443+0000 7f7b71400640 1 -- 172.21.15.5:0/794164187 wait complete. 2024-09-10T09:35:40.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:40.443+0000 7f7b71400640 1 Processor -- start 2024-09-10T09:35:40.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:40.444+0000 7f7b71400640 1 -- start start 2024-09-10T09:35:40.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:40.444+0000 7f7b71400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7b6c076720 0x7f7b6c1a4700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:35:40.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:40.444+0000 7f7b71400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7b6c077040 0x7f7b6c1a4c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:35:40.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:40.444+0000 7f7b71400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7b6c112a30 0x7f7b6c19e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:35:40.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:40.444+0000 7f7b71400640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f7b6c118600 con 0x7f7b6c112a30 2024-09-10T09:35:40.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:40.444+0000 7f7b71400640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f7b6c118480 con 0x7f7b6c076720 2024-09-10T09:35:40.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:40.444+0000 7f7b71400640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f7b6c118780 con 0x7f7b6c077040 2024-09-10T09:35:40.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:40.444+0000 7f7b6a800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7b6c076720 0x7f7b6c1a4700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:35:40.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:40.444+0000 7f7b69e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7b6c077040 0x7f7b6c1a4c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:35:40.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:40.444+0000 7f7b6b200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7b6c112a30 0x7f7b6c19e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:35:40.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:40.444+0000 7f7b69e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7b6c077040 0x7f7b6c1a4c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:43740/0 (socket says 172.21.15.5:43740) 2024-09-10T09:35:40.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:40.444+0000 7f7b69e00640 1 -- 172.21.15.5:0/1322560034 learned_addr learned my addr 172.21.15.5:0/1322560034 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:35:40.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:40.444+0000 7f7b6a800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7b6c076720 0x7f7b6c1a4700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:39972/0 (socket says 172.21.15.5:39972) 2024-09-10T09:35:40.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:40.445+0000 7f7b69e00640 1 -- 172.21.15.5:0/1322560034 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7b6c076720 msgr2=0x7f7b6c1a4700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:35:40.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:40.445+0000 7f7b69e00640 1 --2- 172.21.15.5:0/1322560034 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7b6c076720 0x7f7b6c1a4700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:35:40.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:40.445+0000 7f7b69e00640 1 -- 172.21.15.5:0/1322560034 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7b6c112a30 msgr2=0x7f7b6c19e8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:35:40.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:40.445+0000 7f7b69e00640 1 --2- 172.21.15.5:0/1322560034 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7b6c112a30 0x7f7b6c19e8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:35:40.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:40.445+0000 7f7b69e00640 1 -- 172.21.15.5:0/1322560034 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b6c19eff0 con 0x7f7b6c077040 2024-09-10T09:35:40.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:40.445+0000 7f7b69e00640 1 --2- 172.21.15.5:0/1322560034 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7b6c077040 0x7f7b6c1a4c40 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f7b58004770 tx=0x7f7b58003c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:35:40.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:40.446+0000 7f7b63400640 1 -- 172.21.15.5:0/1322560034 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b5803f070 con 0x7f7b6c077040 2024-09-10T09:35:40.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:40.446+0000 7f7b63400640 1 -- 172.21.15.5:0/1322560034 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b58031ca0 con 0x7f7b6c077040 2024-09-10T09:35:40.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:40.446+0000 7f7b63400640 1 -- 172.21.15.5:0/1322560034 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b58035030 con 0x7f7b6c077040 2024-09-10T09:35:40.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:40.446+0000 7f7b71400640 1 -- 172.21.15.5:0/1322560034 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b6c19f280 con 0x7f7b6c077040 2024-09-10T09:35:40.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:40.446+0000 7f7b71400640 1 -- 172.21.15.5:0/1322560034 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b6c1ab5d0 con 0x7f7b6c077040 2024-09-10T09:35:40.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:40.447+0000 7f7b71400640 1 -- 172.21.15.5:0/1322560034 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7b6c076b00 con 0x7f7b6c077040 2024-09-10T09:35:40.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:40.449+0000 7f7b63400640 1 -- 172.21.15.5:0/1322560034 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7f7b58042070 con 0x7f7b6c077040 2024-09-10T09:35:40.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:40.450+0000 7f7b63400640 1 --2- 172.21.15.5:0/1322560034 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f7b500a2c00 0x7f7b500a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:35:40.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:40.450+0000 7f7b63400640 1 -- 172.21.15.5:0/1322560034 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8770+0+0 (secure 0 0 0) 0x7f7b580efa30 con 0x7f7b6c077040 2024-09-10T09:35:40.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:40.451+0000 7f7b6a800640 1 --2- 172.21.15.5:0/1322560034 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f7b500a2c00 0x7f7b500a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:35:40.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:40.451+0000 7f7b6a800640 1 --2- 172.21.15.5:0/1322560034 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f7b500a2c00 0x7f7b500a50c0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f7b6c19fa00 tx=0x7f7b5c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:35:40.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:40.452+0000 7f7b63400640 1 -- 172.21.15.5:0/1322560034 <== mon.1 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f7b580bc180 con 0x7f7b6c077040 2024-09-10T09:35:40.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:40.674+0000 7f7b71400640 1 -- 172.21.15.5:0/1322560034 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) -- 0x7f7b6c19fdf0 con 0x7f7b6c077040 2024-09-10T09:35:40.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:35:40 smithi102 ceph-mon[35442]: pgmap v308: 129 pgs: 9 creating+peering, 9 unknown, 111 active+clean; 579 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T09:35:41.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:35:40 smithi005 ceph-mon[30350]: pgmap v308: 129 pgs: 9 creating+peering, 9 unknown, 111 active+clean; 579 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T09:35:41.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:35:40 smithi159 ceph-mon[35513]: pgmap v308: 129 pgs: 9 creating+peering, 9 unknown, 111 active+clean; 579 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T09:35:41.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:41.645+0000 7f7b63400640 1 -- 172.21.15.5:0/1322560034 <== mon.1 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 4 to fsmap v17) ==== 137+0+0 (secure 0 0 0) 0x7f7b6c19fdf0 con 0x7f7b6c077040 2024-09-10T09:35:41.646 INFO:teuthology.orchestra.run.smithi005.stderr:added data pool 4 to fsmap 2024-09-10T09:35:41.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:41.647+0000 7f7b71400640 1 -- 172.21.15.5:0/1322560034 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f7b500a2c00 msgr2=0x7f7b500a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:35:41.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:41.647+0000 7f7b71400640 1 --2- 172.21.15.5:0/1322560034 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f7b500a2c00 0x7f7b500a50c0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f7b6c19fa00 tx=0x7f7b5c005f90 comp rx=0 tx=0).stop 2024-09-10T09:35:41.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:41.648+0000 7f7b71400640 1 -- 172.21.15.5:0/1322560034 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7b6c077040 msgr2=0x7f7b6c1a4c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:35:41.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:41.648+0000 7f7b71400640 1 --2- 172.21.15.5:0/1322560034 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7b6c077040 0x7f7b6c1a4c40 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f7b58004770 tx=0x7f7b58003c10 comp rx=0 tx=0).stop 2024-09-10T09:35:41.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:41.648+0000 7f7b71400640 1 -- 172.21.15.5:0/1322560034 shutdown_connections 2024-09-10T09:35:41.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:41.648+0000 7f7b71400640 1 --2- 172.21.15.5:0/1322560034 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f7b500a2c00 0x7f7b500a50c0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:35:41.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:41.648+0000 7f7b71400640 1 --2- 172.21.15.5:0/1322560034 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7b6c112a30 0x7f7b6c19e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:35:41.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:41.648+0000 7f7b71400640 1 --2- 172.21.15.5:0/1322560034 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7b6c077040 0x7f7b6c1a4c40 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:35:41.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:41.648+0000 7f7b71400640 1 --2- 172.21.15.5:0/1322560034 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7b6c076720 0x7f7b6c1a4700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:35:41.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:41.648+0000 7f7b71400640 1 -- 172.21.15.5:0/1322560034 >> 172.21.15.5:0/1322560034 conn(0x7f7b6c0ffa90 msgr2=0x7f7b6c101e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:35:41.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:41.648+0000 7f7b71400640 1 -- 172.21.15.5:0/1322560034 shutdown_connections 2024-09-10T09:35:41.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:41.648+0000 7f7b71400640 1 -- 172.21.15.5:0/1322560034 wait complete. 2024-09-10T09:35:41.992 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:35:41 smithi005 ceph-mon[30350]: from='client.? 172.21.15.5:0/1322560034' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-09-10T09:35:41.992 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:35:41 smithi005 ceph-mon[30350]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-09-10T09:35:42.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:35:41 smithi159 ceph-mon[35513]: from='client.? 172.21.15.5:0/1322560034' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-09-10T09:35:42.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:35:41 smithi159 ceph-mon[35513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-09-10T09:35:42.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:35:41 smithi102 ceph-mon[35442]: from='client.? 172.21.15.5:0/1322560034' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-09-10T09:35:42.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:35:41 smithi102 ceph-mon[35442]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-09-10T09:35:42.506 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T09:35:42.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:35:42 smithi005 ceph-mon[30350]: pgmap v309: 129 pgs: 129 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T09:35:42.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:35:42 smithi005 ceph-mon[30350]: osdmap e92: 12 total, 12 up, 12 in 2024-09-10T09:35:42.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:35:42 smithi005 ceph-mon[30350]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-09-10T09:35:42.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:35:42 smithi005 ceph-mon[30350]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-10T09:35:42.884 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:35:43.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:35:42 smithi159 ceph-mon[35513]: pgmap v309: 129 pgs: 129 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T09:35:43.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:35:42 smithi159 ceph-mon[35513]: osdmap e92: 12 total, 12 up, 12 in 2024-09-10T09:35:43.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:35:42 smithi159 ceph-mon[35513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-09-10T09:35:43.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:35:42 smithi159 ceph-mon[35513]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-10T09:35:43.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:35:42 smithi102 ceph-mon[35442]: pgmap v309: 129 pgs: 129 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T09:35:43.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:35:42 smithi102 ceph-mon[35442]: osdmap e92: 12 total, 12 up, 12 in 2024-09-10T09:35:43.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:35:42 smithi102 ceph-mon[35442]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-09-10T09:35:43.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:35:42 smithi102 ceph-mon[35442]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-10T09:35:43.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:43.434+0000 7f31de600640 1 -- 172.21.15.5:0/3780593346 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f31d8074ee0 msgr2=0x7f31d810def0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:35:43.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:43.434+0000 7f31de600640 1 --2- 172.21.15.5:0/3780593346 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f31d8074ee0 0x7f31d810def0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f31c0009860 tx=0x7f31c002a3d0 comp rx=0 tx=0).stop 2024-09-10T09:35:43.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:43.434+0000 7f31de600640 1 -- 172.21.15.5:0/3780593346 shutdown_connections 2024-09-10T09:35:43.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:43.434+0000 7f31de600640 1 --2- 172.21.15.5:0/3780593346 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f31d810e430 0x7f31d8110820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:35:43.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:43.434+0000 7f31de600640 1 --2- 172.21.15.5:0/3780593346 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f31d8074ee0 0x7f31d810def0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:35:43.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:43.434+0000 7f31de600640 1 --2- 172.21.15.5:0/3780593346 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f31d80745c0 0x7f31d80749a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:35:43.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:43.434+0000 7f31de600640 1 -- 172.21.15.5:0/3780593346 >> 172.21.15.5:0/3780593346 conn(0x7f31d80fd8b0 msgr2=0x7f31d80ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:35:43.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:43.434+0000 7f31de600640 1 -- 172.21.15.5:0/3780593346 shutdown_connections 2024-09-10T09:35:43.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:43.434+0000 7f31de600640 1 -- 172.21.15.5:0/3780593346 wait complete. 2024-09-10T09:35:43.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:43.435+0000 7f31de600640 1 Processor -- start 2024-09-10T09:35:43.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:43.435+0000 7f31de600640 1 -- start start 2024-09-10T09:35:43.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:43.435+0000 7f31de600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f31d80745c0 0x7f31d819a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:35:43.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:43.435+0000 7f31de600640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f31d8074ee0 0x7f31d819ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:35:43.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:43.435+0000 7f31de600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f31d810e430 0x7f31d819efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:35:43.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:43.435+0000 7f31de600640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f31d8114170 con 0x7f31d810e430 2024-09-10T09:35:43.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:43.435+0000 7f31de600640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f31d8113ff0 con 0x7f31d80745c0 2024-09-10T09:35:43.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:43.435+0000 7f31de600640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f31d81142f0 con 0x7f31d8074ee0 2024-09-10T09:35:43.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:43.436+0000 7f31cea00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f31d8074ee0 0x7f31d819ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:35:43.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:43.436+0000 7f31d7e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f31d810e430 0x7f31d819efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:35:43.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:43.436+0000 7f31cea00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f31d8074ee0 0x7f31d819ac30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:51416/0 (socket says 172.21.15.5:51416) 2024-09-10T09:35:43.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:43.436+0000 7f31cea00640 1 -- 172.21.15.5:0/55647243 learned_addr learned my addr 172.21.15.5:0/55647243 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:35:43.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:43.436+0000 7f31d7400640 1 --2- 172.21.15.5:0/55647243 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f31d80745c0 0x7f31d819a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:35:43.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:43.436+0000 7f31cea00640 1 -- 172.21.15.5:0/55647243 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f31d80745c0 msgr2=0x7f31d819a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:35:43.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:43.436+0000 7f31cea00640 1 --2- 172.21.15.5:0/55647243 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f31d80745c0 0x7f31d819a6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:35:43.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:43.436+0000 7f31cea00640 1 -- 172.21.15.5:0/55647243 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f31d810e430 msgr2=0x7f31d819efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:35:43.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:43.436+0000 7f31cea00640 1 --2- 172.21.15.5:0/55647243 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f31d810e430 0x7f31d819efc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:35:43.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:43.436+0000 7f31cea00640 1 -- 172.21.15.5:0/55647243 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31d819f6a0 con 0x7f31d8074ee0 2024-09-10T09:35:43.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:43.436+0000 7f31d7400640 1 --2- 172.21.15.5:0/55647243 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f31d80745c0 0x7f31d819a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:35:43.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:43.436+0000 7f31d7e00640 1 --2- 172.21.15.5:0/55647243 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f31d810e430 0x7f31d819efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:35:43.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:43.437+0000 7f31cea00640 1 --2- 172.21.15.5:0/55647243 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f31d8074ee0 0x7f31d819ac30 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f31c002a8f0 tx=0x7f31c0030650 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:35:43.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:43.437+0000 7f31d4c00640 1 -- 172.21.15.5:0/55647243 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31c0040020 con 0x7f31d8074ee0 2024-09-10T09:35:43.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:43.437+0000 7f31d4c00640 1 -- 172.21.15.5:0/55647243 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f31c0004460 con 0x7f31d8074ee0 2024-09-10T09:35:43.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:43.437+0000 7f31de600640 1 -- 172.21.15.5:0/55647243 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31d819f930 con 0x7f31d8074ee0 2024-09-10T09:35:43.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:43.437+0000 7f31d4c00640 1 -- 172.21.15.5:0/55647243 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31c00046c0 con 0x7f31d8074ee0 2024-09-10T09:35:43.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:43.437+0000 7f31de600640 1 -- 172.21.15.5:0/55647243 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31d81a7270 con 0x7f31d8074ee0 2024-09-10T09:35:43.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:43.438+0000 7f31de600640 1 -- 172.21.15.5:0/55647243 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f31d8075c70 con 0x7f31d8074ee0 2024-09-10T09:35:43.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:43.440+0000 7f31d4c00640 1 -- 172.21.15.5:0/55647243 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7f31c0043020 con 0x7f31d8074ee0 2024-09-10T09:35:43.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:43.441+0000 7f31d4c00640 1 --2- 172.21.15.5:0/55647243 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f31bc0a2b80 0x7f31bc0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:35:43.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:43.442+0000 7f31d4c00640 1 -- 172.21.15.5:0/55647243 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8802+0+0 (secure 0 0 0) 0x7f31c00f0360 con 0x7f31d8074ee0 2024-09-10T09:35:43.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:43.442+0000 7f31d7400640 1 --2- 172.21.15.5:0/55647243 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f31bc0a2b80 0x7f31bc0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:35:43.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:43.442+0000 7f31d7400640 1 --2- 172.21.15.5:0/55647243 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f31bc0a2b80 0x7f31bc0a5040 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f31c8009fc0 tx=0x7f31c8005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:35:43.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:43.443+0000 7f31d4c00640 1 -- 172.21.15.5:0/55647243 <== mon.1 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f31c004a020 con 0x7f31d8074ee0 2024-09-10T09:35:43.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:43.667+0000 7f31de600640 1 -- 172.21.15.5:0/55647243 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f31d80749a0 con 0x7f31d8074ee0 2024-09-10T09:35:43.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:43.668+0000 7f31d4c00640 1 -- 172.21.15.5:0/55647243 <== mon.1 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+11833 (secure 0 0 0) 0x7f31d80749a0 con 0x7f31d8074ee0 2024-09-10T09:35:43.669 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-10T09:35:43.669 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":17,"btime":"2024-09-10T09:35:41:631650+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14454,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/2970594311","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2970594311},{"type":"v1","addr":"172.21.15.5:6835","nonce":2970594311}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14514,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6839/3616502174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":3616502174},{"type":"v1","addr":"172.21.15.5:6839","nonce":3616502174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14520,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6841/2073599842","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":2073599842},{"type":"v1","addr":"172.21.15.5:6841","nonce":2073599842}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6833/2457007427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":2457007427},{"type":"v1","addr":"172.21.15.102:6833","nonce":2457007427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24460,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6833/356445159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":356445159},{"type":"v1","addr":"172.21.15.159:6833","nonce":356445159}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24467,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/2894314507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":2894314507},{"type":"v1","addr":"172.21.15.5:6837","nonce":2894314507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24479,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6835/1647549111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":1647549111},{"type":"v1","addr":"172.21.15.102:6835","nonce":1647549111}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24505,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6835/952657242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":952657242},{"type":"v1","addr":"172.21.15.159:6835","nonce":952657242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24506,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/2689776764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":2689776764},{"type":"v1","addr":"172.21.15.102:6837","nonce":2689776764}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24511,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6837/620627639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":620627639},{"type":"v1","addr":"172.21.15.159:6837","nonce":620627639}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24535,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6839/2073739599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":2073739599},{"type":"v1","addr":"172.21.15.159:6839","nonce":2073739599}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T09:35:29.626287+0000","modified":"2024-09-10T09:35:40.676575+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24542},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24542":{"gid":24542,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.102:6839/335409749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6838","nonce":335409749},{"type":"v1","addr":"172.21.15.102:6839","nonce":335409749}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24542,"qdb_cluster":[24542]},"id":1}]} 2024-09-10T09:35:43.669 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 17 2024-09-10T09:35:43.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:43.670+0000 7f31de600640 1 -- 172.21.15.5:0/55647243 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f31bc0a2b80 msgr2=0x7f31bc0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:35:43.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:43.670+0000 7f31de600640 1 --2- 172.21.15.5:0/55647243 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f31bc0a2b80 0x7f31bc0a5040 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f31c8009fc0 tx=0x7f31c8005f90 comp rx=0 tx=0).stop 2024-09-10T09:35:43.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:43.670+0000 7f31de600640 1 -- 172.21.15.5:0/55647243 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f31d8074ee0 msgr2=0x7f31d819ac30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:35:43.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:43.670+0000 7f31de600640 1 --2- 172.21.15.5:0/55647243 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f31d8074ee0 0x7f31d819ac30 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f31c002a8f0 tx=0x7f31c0030650 comp rx=0 tx=0).stop 2024-09-10T09:35:43.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:43.670+0000 7f31de600640 1 -- 172.21.15.5:0/55647243 shutdown_connections 2024-09-10T09:35:43.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:43.670+0000 7f31de600640 1 --2- 172.21.15.5:0/55647243 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f31bc0a2b80 0x7f31bc0a5040 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:35:43.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:43.670+0000 7f31de600640 1 --2- 172.21.15.5:0/55647243 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f31d810e430 0x7f31d819efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:35:43.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:43.670+0000 7f31de600640 1 --2- 172.21.15.5:0/55647243 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f31d8074ee0 0x7f31d819ac30 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:35:43.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:43.670+0000 7f31de600640 1 --2- 172.21.15.5:0/55647243 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f31d80745c0 0x7f31d819a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:35:43.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:43.670+0000 7f31de600640 1 -- 172.21.15.5:0/55647243 >> 172.21.15.5:0/55647243 conn(0x7f31d80fd8b0 msgr2=0x7f31d80fefd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:35:43.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:43.671+0000 7f31de600640 1 -- 172.21.15.5:0/55647243 shutdown_connections 2024-09-10T09:35:43.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:43.671+0000 7f31de600640 1 -- 172.21.15.5:0/55647243 wait complete. 2024-09-10T09:35:44.048 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:35:43 smithi005 ceph-mon[30350]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-10T09:35:44.048 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:35:43 smithi005 ceph-mon[30350]: Cluster is now healthy 2024-09-10T09:35:44.048 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:35:43 smithi005 ceph-mon[30350]: from='client.? 172.21.15.5:0/55647243' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:35:44.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:35:43 smithi159 ceph-mon[35513]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-10T09:35:44.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:35:43 smithi159 ceph-mon[35513]: Cluster is now healthy 2024-09-10T09:35:44.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:35:43 smithi159 ceph-mon[35513]: from='client.? 172.21.15.5:0/55647243' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:35:44.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:35:43 smithi102 ceph-mon[35442]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-10T09:35:44.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:35:43 smithi102 ceph-mon[35442]: Cluster is now healthy 2024-09-10T09:35:44.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:35:43 smithi102 ceph-mon[35442]: from='client.? 172.21.15.5:0/55647243' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:35:44.442 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-10T09:35:44.825 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:35:44.964 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:35:44 smithi005 ceph-mon[30350]: pgmap v311: 129 pgs: 129 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T09:35:45.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:35:44 smithi159 ceph-mon[35513]: pgmap v311: 129 pgs: 129 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T09:35:45.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:35:44 smithi102 ceph-mon[35442]: pgmap v311: 129 pgs: 129 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T09:35:45.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:45.343+0000 7f5da5600640 1 -- 172.21.15.5:0/3996467810 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5da0074590 msgr2=0x7f5da0074970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:35:45.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:45.343+0000 7f5da5600640 1 --2- 172.21.15.5:0/3996467810 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5da0074590 0x7f5da0074970 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f5d90009900 tx=0x7f5d9002a480 comp rx=0 tx=0).stop 2024-09-10T09:35:45.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:45.343+0000 7f5da5600640 1 -- 172.21.15.5:0/3996467810 shutdown_connections 2024-09-10T09:35:45.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:45.343+0000 7f5da5600640 1 --2- 172.21.15.5:0/3996467810 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5da0112b90 0x7f5da0114f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:35:45.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:45.343+0000 7f5da5600640 1 --2- 172.21.15.5:0/3996467810 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5da0074eb0 0x7f5da0112650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:35:45.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:45.343+0000 7f5da5600640 1 --2- 172.21.15.5:0/3996467810 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5da0074590 0x7f5da0074970 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:35:45.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:45.343+0000 7f5da5600640 1 -- 172.21.15.5:0/3996467810 >> 172.21.15.5:0/3996467810 conn(0x7f5da00fd8c0 msgr2=0x7f5da00ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:35:45.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:45.343+0000 7f5da5600640 1 -- 172.21.15.5:0/3996467810 shutdown_connections 2024-09-10T09:35:45.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:45.343+0000 7f5da5600640 1 -- 172.21.15.5:0/3996467810 wait complete. 2024-09-10T09:35:45.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:45.344+0000 7f5da5600640 1 Processor -- start 2024-09-10T09:35:45.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:45.345+0000 7f5da5600640 1 -- start start 2024-09-10T09:35:45.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:45.345+0000 7f5da5600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5da0074590 0x7f5da019eb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:35:45.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:45.345+0000 7f5da5600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5da0074eb0 0x7f5da019f090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:35:45.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:45.345+0000 7f5da5600640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5da0112b90 0x7f5da01a3450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:35:45.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:45.345+0000 7f5da5600640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f5da01185c0 con 0x7f5da0074eb0 2024-09-10T09:35:45.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:45.345+0000 7f5da5600640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f5da0118440 con 0x7f5da0074590 2024-09-10T09:35:45.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:45.345+0000 7f5da5600640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f5da0118740 con 0x7f5da0112b90 2024-09-10T09:35:45.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:45.345+0000 7f5d97e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5da0074eb0 0x7f5da019f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:35:45.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:45.345+0000 7f5d97e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5da0074eb0 0x7f5da019f090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39476/0 (socket says 172.21.15.5:39476) 2024-09-10T09:35:45.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:45.345+0000 7f5d97e00640 1 -- 172.21.15.5:0/4043816940 learned_addr learned my addr 172.21.15.5:0/4043816940 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:35:45.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:45.345+0000 7f5d9ea00640 1 --2- 172.21.15.5:0/4043816940 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5da0074590 0x7f5da019eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:35:45.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:45.345+0000 7f5d9f400640 1 --2- 172.21.15.5:0/4043816940 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5da0112b90 0x7f5da01a3450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:35:45.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:45.346+0000 7f5d97e00640 1 -- 172.21.15.5:0/4043816940 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5da0112b90 msgr2=0x7f5da01a3450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:35:45.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:45.346+0000 7f5d97e00640 1 --2- 172.21.15.5:0/4043816940 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5da0112b90 0x7f5da01a3450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:35:45.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:45.346+0000 7f5d97e00640 1 -- 172.21.15.5:0/4043816940 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5da0074590 msgr2=0x7f5da019eb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:35:45.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:45.346+0000 7f5d97e00640 1 --2- 172.21.15.5:0/4043816940 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5da0074590 0x7f5da019eb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:35:45.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:45.346+0000 7f5d97e00640 1 -- 172.21.15.5:0/4043816940 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5da01a3b80 con 0x7f5da0074eb0 2024-09-10T09:35:45.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:45.346+0000 7f5d97e00640 1 --2- 172.21.15.5:0/4043816940 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5da0074eb0 0x7f5da019f090 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f5d8800e9f0 tx=0x7f5d8800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:35:45.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:45.346+0000 7f5d97400640 1 -- 172.21.15.5:0/4043816940 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d8800cd80 con 0x7f5da0074eb0 2024-09-10T09:35:45.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:45.346+0000 7f5da5600640 1 -- 172.21.15.5:0/4043816940 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5da01a3e70 con 0x7f5da0074eb0 2024-09-10T09:35:45.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:45.346+0000 7f5d97400640 1 -- 172.21.15.5:0/4043816940 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5d88005500 con 0x7f5da0074eb0 2024-09-10T09:35:45.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:45.347+0000 7f5d97400640 1 -- 172.21.15.5:0/4043816940 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d88018630 con 0x7f5da0074eb0 2024-09-10T09:35:45.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:45.347+0000 7f5da5600640 1 -- 172.21.15.5:0/4043816940 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5da01ab700 con 0x7f5da0074eb0 2024-09-10T09:35:45.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:45.348+0000 7f5da5600640 1 -- 172.21.15.5:0/4043816940 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5da0074970 con 0x7f5da0074eb0 2024-09-10T09:35:45.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:45.350+0000 7f5d97400640 1 -- 172.21.15.5:0/4043816940 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7f5d880046c0 con 0x7f5da0074eb0 2024-09-10T09:35:45.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:45.351+0000 7f5d97400640 1 --2- 172.21.15.5:0/4043816940 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f5d840a2b80 0x7f5d840a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:35:45.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:45.351+0000 7f5d97400640 1 -- 172.21.15.5:0/4043816940 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8802+0+0 (secure 0 0 0) 0x7f5d880cf7c0 con 0x7f5da0074eb0 2024-09-10T09:35:45.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:45.351+0000 7f5d9ea00640 1 --2- 172.21.15.5:0/4043816940 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f5d840a2b80 0x7f5d840a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:35:45.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:45.352+0000 7f5d9ea00640 1 --2- 172.21.15.5:0/4043816940 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f5d840a2b80 0x7f5d840a5040 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f5d90000c00 tx=0x7f5d90005b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:35:45.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:45.356+0000 7f5d97400640 1 -- 172.21.15.5:0/4043816940 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f5d8809c480 con 0x7f5da0074eb0 2024-09-10T09:35:45.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:45.542+0000 7f5da5600640 1 -- 172.21.15.5:0/4043816940 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5da0105000 con 0x7f5da0074eb0 2024-09-10T09:35:45.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:45.543+0000 7f5d97400640 1 -- 172.21.15.5:0/4043816940 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) ==== 74+0+24126 (secure 0 0 0) 0x7f5d8809bd80 con 0x7f5da0074eb0 2024-09-10T09:35:45.545 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-10T09:35:45.545 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":92,"fsid":"bade4b50-6f56-11ef-bcea-c7b262605968","created":"2024-09-10T09:27:13.298498+0000","modified":"2024-09-10T09:35:40.676433+0000","last_up_change":"2024-09-10T09:34:30.730066+0000","last_in_change":"2024-09-10T09:34:07.340995+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T09:30:26.042255+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T09:35:24.461031+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T09:35:27.155349+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-10T09:35:35.121462+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"085749a8-e882-4377-92dd-34bb266c0f0b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":954535266},{"type":"v1","addr":"172.21.15.5:6803","nonce":954535266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":954535266},{"type":"v1","addr":"172.21.15.5:6805","nonce":954535266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":954535266},{"type":"v1","addr":"172.21.15.5:6809","nonce":954535266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":954535266},{"type":"v1","addr":"172.21.15.5:6807","nonce":954535266}]},"public_addr":"172.21.15.5:6803/954535266","cluster_addr":"172.21.15.5:6805/954535266","heartbeat_back_addr":"172.21.15.5:6809/954535266","heartbeat_front_addr":"172.21.15.5:6807/954535266","state":["exists","up"]},{"osd":1,"uuid":"0a76c795-2573-4a92-969a-2ec5dd65c094","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6800","nonce":733924375},{"type":"v1","addr":"172.21.15.102:6801","nonce":733924375}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6802","nonce":733924375},{"type":"v1","addr":"172.21.15.102:6803","nonce":733924375}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6806","nonce":733924375},{"type":"v1","addr":"172.21.15.102:6807","nonce":733924375}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6804","nonce":733924375},{"type":"v1","addr":"172.21.15.102:6805","nonce":733924375}]},"public_addr":"172.21.15.102:6801/733924375","cluster_addr":"172.21.15.102:6803/733924375","heartbeat_back_addr":"172.21.15.102:6807/733924375","heartbeat_front_addr":"172.21.15.102:6805/733924375","state":["exists","up"]},{"osd":2,"uuid":"0c98fcb8-60ab-4473-8995-1d9e97c577e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6800","nonce":2236204189},{"type":"v1","addr":"172.21.15.159:6801","nonce":2236204189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":2236204189},{"type":"v1","addr":"172.21.15.159:6803","nonce":2236204189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":2236204189},{"type":"v1","addr":"172.21.15.159:6807","nonce":2236204189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":2236204189},{"type":"v1","addr":"172.21.15.159:6805","nonce":2236204189}]},"public_addr":"172.21.15.159:6801/2236204189","cluster_addr":"172.21.15.159:6803/2236204189","heartbeat_back_addr":"172.21.15.159:6807/2236204189","heartbeat_front_addr":"172.21.15.159:6805/2236204189","state":["exists","up"]},{"osd":3,"uuid":"88eecdeb-5cee-4f78-a3f9-a66bf4bf772a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":2221468725},{"type":"v1","addr":"172.21.15.5:6811","nonce":2221468725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":2221468725},{"type":"v1","addr":"172.21.15.5:6813","nonce":2221468725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6816","nonce":2221468725},{"type":"v1","addr":"172.21.15.5:6817","nonce":2221468725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":2221468725},{"type":"v1","addr":"172.21.15.5:6815","nonce":2221468725}]},"public_addr":"172.21.15.5:6811/2221468725","cluster_addr":"172.21.15.5:6813/2221468725","heartbeat_back_addr":"172.21.15.5:6817/2221468725","heartbeat_front_addr":"172.21.15.5:6815/2221468725","state":["exists","up"]},{"osd":4,"uuid":"f2e7314d-7938-4668-9011-45c175f66b48","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6808","nonce":594465753},{"type":"v1","addr":"172.21.15.102:6809","nonce":594465753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6810","nonce":594465753},{"type":"v1","addr":"172.21.15.102:6811","nonce":594465753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6814","nonce":594465753},{"type":"v1","addr":"172.21.15.102:6815","nonce":594465753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6812","nonce":594465753},{"type":"v1","addr":"172.21.15.102:6813","nonce":594465753}]},"public_addr":"172.21.15.102:6809/594465753","cluster_addr":"172.21.15.102:6811/594465753","heartbeat_back_addr":"172.21.15.102:6815/594465753","heartbeat_front_addr":"172.21.15.102:6813/594465753","state":["exists","up"]},{"osd":5,"uuid":"0c0366d3-658b-4bf0-ba17-aff28a02e948","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":3230280402},{"type":"v1","addr":"172.21.15.159:6809","nonce":3230280402}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":3230280402},{"type":"v1","addr":"172.21.15.159:6811","nonce":3230280402}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":3230280402},{"type":"v1","addr":"172.21.15.159:6815","nonce":3230280402}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":3230280402},{"type":"v1","addr":"172.21.15.159:6813","nonce":3230280402}]},"public_addr":"172.21.15.159:6809/3230280402","cluster_addr":"172.21.15.159:6811/3230280402","heartbeat_back_addr":"172.21.15.159:6815/3230280402","heartbeat_front_addr":"172.21.15.159:6813/3230280402","state":["exists","up"]},{"osd":6,"uuid":"682670a6-51d5-46e8-af6e-dc2d8c1cf407","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":1907651140},{"type":"v1","addr":"172.21.15.5:6819","nonce":1907651140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":1907651140},{"type":"v1","addr":"172.21.15.5:6821","nonce":1907651140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":1907651140},{"type":"v1","addr":"172.21.15.5:6825","nonce":1907651140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":1907651140},{"type":"v1","addr":"172.21.15.5:6823","nonce":1907651140}]},"public_addr":"172.21.15.5:6819/1907651140","cluster_addr":"172.21.15.5:6821/1907651140","heartbeat_back_addr":"172.21.15.5:6825/1907651140","heartbeat_front_addr":"172.21.15.5:6823/1907651140","state":["exists","up"]},{"osd":7,"uuid":"8ae0f4a9-99e7-4563-aa2a-15169684a8b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6816","nonce":692040360},{"type":"v1","addr":"172.21.15.102:6817","nonce":692040360}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6818","nonce":692040360},{"type":"v1","addr":"172.21.15.102:6819","nonce":692040360}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6822","nonce":692040360},{"type":"v1","addr":"172.21.15.102:6823","nonce":692040360}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6820","nonce":692040360},{"type":"v1","addr":"172.21.15.102:6821","nonce":692040360}]},"public_addr":"172.21.15.102:6817/692040360","cluster_addr":"172.21.15.102:6819/692040360","heartbeat_back_addr":"172.21.15.102:6823/692040360","heartbeat_front_addr":"172.21.15.102:6821/692040360","state":["exists","up"]},{"osd":8,"uuid":"e37c845e-ced9-4413-9bbe-bc090537c2bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":2107291799},{"type":"v1","addr":"172.21.15.159:6817","nonce":2107291799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":2107291799},{"type":"v1","addr":"172.21.15.159:6819","nonce":2107291799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":2107291799},{"type":"v1","addr":"172.21.15.159:6823","nonce":2107291799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":2107291799},{"type":"v1","addr":"172.21.15.159:6821","nonce":2107291799}]},"public_addr":"172.21.15.159:6817/2107291799","cluster_addr":"172.21.15.159:6819/2107291799","heartbeat_back_addr":"172.21.15.159:6823/2107291799","heartbeat_front_addr":"172.21.15.159:6821/2107291799","state":["exists","up"]},{"osd":9,"uuid":"91a5b6cc-08da-47be-af12-d0896a849309","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":902431281},{"type":"v1","addr":"172.21.15.5:6827","nonce":902431281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":902431281},{"type":"v1","addr":"172.21.15.5:6829","nonce":902431281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":902431281},{"type":"v1","addr":"172.21.15.5:6833","nonce":902431281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":902431281},{"type":"v1","addr":"172.21.15.5:6831","nonce":902431281}]},"public_addr":"172.21.15.5:6827/902431281","cluster_addr":"172.21.15.5:6829/902431281","heartbeat_back_addr":"172.21.15.5:6833/902431281","heartbeat_front_addr":"172.21.15.5:6831/902431281","state":["exists","up"]},{"osd":10,"uuid":"2211030c-b00e-402b-b4d7-c587eb29a490","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6824","nonce":2853806211},{"type":"v1","addr":"172.21.15.102:6825","nonce":2853806211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6826","nonce":2853806211},{"type":"v1","addr":"172.21.15.102:6827","nonce":2853806211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6830","nonce":2853806211},{"type":"v1","addr":"172.21.15.102:6831","nonce":2853806211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6828","nonce":2853806211},{"type":"v1","addr":"172.21.15.102:6829","nonce":2853806211}]},"public_addr":"172.21.15.102:6825/2853806211","cluster_addr":"172.21.15.102:6827/2853806211","heartbeat_back_addr":"172.21.15.102:6831/2853806211","heartbeat_front_addr":"172.21.15.102:6829/2853806211","state":["exists","up"]},{"osd":11,"uuid":"4e9699ec-8ff2-4005-9d84-af5600b3a2ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6824","nonce":969941218},{"type":"v1","addr":"172.21.15.159:6825","nonce":969941218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":969941218},{"type":"v1","addr":"172.21.15.159:6827","nonce":969941218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6830","nonce":969941218},{"type":"v1","addr":"172.21.15.159:6831","nonce":969941218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":969941218},{"type":"v1","addr":"172.21.15.159:6829","nonce":969941218}]},"public_addr":"172.21.15.159:6825/969941218","cluster_addr":"172.21.15.159:6827/969941218","heartbeat_back_addr":"172.21.15.159:6831/969941218","heartbeat_front_addr":"172.21.15.159:6829/969941218","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:29:34.416490+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:30:00.246101+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:30:24.300464+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:30:54.663623+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:31:21.426292+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:31:46.738914+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:32:16.196571+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:32:42.549243+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:33:08.178651+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:33:37.477911+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:34:03.114026+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:34:28.891924+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.5:6801/1527422686":"2024-09-11T09:28:05.436742+0000","172.21.15.5:0/2276144345":"2024-09-11T09:28:05.436742+0000","172.21.15.5:0/412040104":"2024-09-11T09:28:05.436742+0000","172.21.15.5:6800/1527422686":"2024-09-11T09:28:05.436742+0000","172.21.15.5:0/1856581170":"2024-09-11T09:28:05.436742+0000","172.21.15.5:0/2847668298":"2024-09-11T09:27:37.778500+0000","172.21.15.5:6801/14425913":"2024-09-11T09:27:37.778500+0000","172.21.15.5:0/166910782":"2024-09-11T09:27:37.778500+0000","172.21.15.5:6800/14425913":"2024-09-11T09:27:37.778500+0000","172.21.15.5:0/1711858179":"2024-09-11T09:27:37.778500+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T09:35:45.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:45.547+0000 7f5da5600640 1 -- 172.21.15.5:0/4043816940 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f5d840a2b80 msgr2=0x7f5d840a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:35:45.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:45.547+0000 7f5da5600640 1 --2- 172.21.15.5:0/4043816940 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f5d840a2b80 0x7f5d840a5040 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f5d90000c00 tx=0x7f5d90005b50 comp rx=0 tx=0).stop 2024-09-10T09:35:45.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:45.547+0000 7f5da5600640 1 -- 172.21.15.5:0/4043816940 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5da0074eb0 msgr2=0x7f5da019f090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:35:45.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:45.547+0000 7f5da5600640 1 --2- 172.21.15.5:0/4043816940 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5da0074eb0 0x7f5da019f090 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f5d8800e9f0 tx=0x7f5d8800edf0 comp rx=0 tx=0).stop 2024-09-10T09:35:45.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:45.547+0000 7f5da5600640 1 -- 172.21.15.5:0/4043816940 shutdown_connections 2024-09-10T09:35:45.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:45.547+0000 7f5da5600640 1 --2- 172.21.15.5:0/4043816940 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f5d840a2b80 0x7f5d840a5040 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:35:45.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:45.547+0000 7f5da5600640 1 --2- 172.21.15.5:0/4043816940 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5da0112b90 0x7f5da01a3450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:35:45.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:45.547+0000 7f5da5600640 1 --2- 172.21.15.5:0/4043816940 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5da0074eb0 0x7f5da019f090 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:35:45.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:45.547+0000 7f5da5600640 1 --2- 172.21.15.5:0/4043816940 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5da0074590 0x7f5da019eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:35:45.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:45.547+0000 7f5da5600640 1 -- 172.21.15.5:0/4043816940 >> 172.21.15.5:0/4043816940 conn(0x7f5da00fd8c0 msgr2=0x7f5da00ff050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:35:45.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:45.547+0000 7f5da5600640 1 -- 172.21.15.5:0/4043816940 shutdown_connections 2024-09-10T09:35:45.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:45.547+0000 7f5da5600640 1 -- 172.21.15.5:0/4043816940 wait complete. 2024-09-10T09:35:45.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:35:45 smithi005 ceph-mon[30350]: from='client.? 172.21.15.5:0/4043816940' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T09:35:46.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:35:45 smithi159 ceph-mon[35513]: from='client.? 172.21.15.5:0/4043816940' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T09:35:46.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:35:45 smithi102 ceph-mon[35442]: from='client.? 172.21.15.5:0/4043816940' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T09:35:46.453 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-10T09:35:46.830 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:35:46.969 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:35:46 smithi005 ceph-mon[30350]: pgmap v312: 129 pgs: 129 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T09:35:47.001 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:35:46 smithi159 ceph-mon[35513]: pgmap v312: 129 pgs: 129 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T09:35:47.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:35:46 smithi102 ceph-mon[35442]: pgmap v312: 129 pgs: 129 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T09:35:47.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:47.386+0000 7fe080200640 1 -- 172.21.15.5:0/3975048164 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe078109120 msgr2=0x7fe078110ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:35:47.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:47.386+0000 7fe080200640 1 --2- 172.21.15.5:0/3975048164 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe078109120 0x7fe078110ca0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fe070009ca0 tx=0x7fe07002a8c0 comp rx=0 tx=0).stop 2024-09-10T09:35:47.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:47.386+0000 7fe080200640 1 -- 172.21.15.5:0/3975048164 shutdown_connections 2024-09-10T09:35:47.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:47.386+0000 7fe080200640 1 --2- 172.21.15.5:0/3975048164 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe078109120 0x7fe078110ca0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:35:47.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:47.386+0000 7fe080200640 1 --2- 172.21.15.5:0/3975048164 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe0781049a0 0x7fe0781089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:35:47.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:47.386+0000 7fe080200640 1 --2- 172.21.15.5:0/3975048164 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe078103ff0 0x7fe0781043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:35:47.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:47.386+0000 7fe080200640 1 -- 172.21.15.5:0/3975048164 >> 172.21.15.5:0/3975048164 conn(0x7fe0780ffa90 msgr2=0x7fe078101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:35:47.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:47.386+0000 7fe080200640 1 -- 172.21.15.5:0/3975048164 shutdown_connections 2024-09-10T09:35:47.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:47.387+0000 7fe080200640 1 -- 172.21.15.5:0/3975048164 wait complete. 2024-09-10T09:35:47.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:47.387+0000 7fe080200640 1 Processor -- start 2024-09-10T09:35:47.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:47.388+0000 7fe080200640 1 -- start start 2024-09-10T09:35:47.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:47.388+0000 7fe080200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe078103ff0 0x7fe07819a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:35:47.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:47.388+0000 7fe080200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe0781049a0 0x7fe07819ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:35:47.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:47.388+0000 7fe080200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe078109120 0x7fe07819eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:35:47.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:47.388+0000 7fe080200640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe0781141a0 con 0x7fe078103ff0 2024-09-10T09:35:47.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:47.388+0000 7fe080200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fe078114020 con 0x7fe078109120 2024-09-10T09:35:47.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:47.388+0000 7fe080200640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fe078114320 con 0x7fe0781049a0 2024-09-10T09:35:47.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:47.388+0000 7fe07d800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe078103ff0 0x7fe07819a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:35:47.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:47.388+0000 7fe07ce00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe0781049a0 0x7fe07819ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:35:47.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:47.388+0000 7fe07e200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe078109120 0x7fe07819eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:35:47.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:47.388+0000 7fe07d800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe078103ff0 0x7fe07819a5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39500/0 (socket says 172.21.15.5:39500) 2024-09-10T09:35:47.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:47.388+0000 7fe07d800640 1 -- 172.21.15.5:0/2981430443 learned_addr learned my addr 172.21.15.5:0/2981430443 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:35:47.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:47.388+0000 7fe07e200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe078109120 0x7fe07819eeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:37350/0 (socket says 172.21.15.5:37350) 2024-09-10T09:35:47.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:47.388+0000 7fe07ce00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe0781049a0 0x7fe07819ab20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:51448/0 (socket says 172.21.15.5:51448) 2024-09-10T09:35:47.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:47.389+0000 7fe07ce00640 1 -- 172.21.15.5:0/2981430443 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe078109120 msgr2=0x7fe07819eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:35:47.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:47.389+0000 7fe07ce00640 1 --2- 172.21.15.5:0/2981430443 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe078109120 0x7fe07819eeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:35:47.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:47.389+0000 7fe07ce00640 1 -- 172.21.15.5:0/2981430443 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe078103ff0 msgr2=0x7fe07819a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:35:47.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:47.389+0000 7fe07ce00640 1 --2- 172.21.15.5:0/2981430443 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe078103ff0 0x7fe07819a5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:35:47.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:47.389+0000 7fe07ce00640 1 -- 172.21.15.5:0/2981430443 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe07819f590 con 0x7fe0781049a0 2024-09-10T09:35:47.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:47.389+0000 7fe07ce00640 1 --2- 172.21.15.5:0/2981430443 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe0781049a0 0x7fe07819ab20 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fe06400e9f0 tx=0x7fe06400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:35:47.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:47.389+0000 7fe076000640 1 -- 172.21.15.5:0/2981430443 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe06400ccc0 con 0x7fe0781049a0 2024-09-10T09:35:47.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:47.390+0000 7fe080200640 1 -- 172.21.15.5:0/2981430443 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe07819f880 con 0x7fe0781049a0 2024-09-10T09:35:47.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:47.390+0000 7fe080200640 1 -- 172.21.15.5:0/2981430443 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0781a7160 con 0x7fe0781049a0 2024-09-10T09:35:47.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:47.390+0000 7fe076000640 1 -- 172.21.15.5:0/2981430443 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe064005020 con 0x7fe0781049a0 2024-09-10T09:35:47.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:47.390+0000 7fe076000640 1 -- 172.21.15.5:0/2981430443 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe06401c070 con 0x7fe0781049a0 2024-09-10T09:35:47.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:47.391+0000 7fe080200640 1 -- 172.21.15.5:0/2981430443 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe0781043d0 con 0x7fe0781049a0 2024-09-10T09:35:47.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:47.393+0000 7fe076000640 1 -- 172.21.15.5:0/2981430443 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7fe064005540 con 0x7fe0781049a0 2024-09-10T09:35:47.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:47.394+0000 7fe076000640 1 --2- 172.21.15.5:0/2981430443 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fe0600a2c00 0x7fe0600a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:35:47.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:47.394+0000 7fe076000640 1 -- 172.21.15.5:0/2981430443 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8802+0+0 (secure 0 0 0) 0x7fe0640cf1e0 con 0x7fe0781049a0 2024-09-10T09:35:47.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:47.394+0000 7fe07d800640 1 --2- 172.21.15.5:0/2981430443 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fe0600a2c00 0x7fe0600a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:35:47.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:47.395+0000 7fe07d800640 1 --2- 172.21.15.5:0/2981430443 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fe0600a2c00 0x7fe0600a50c0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fe06c009c90 tx=0x7fe06c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:35:47.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:47.396+0000 7fe076000640 1 -- 172.21.15.5:0/2981430443 <== mon.1 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe06409bf30 con 0x7fe0781049a0 2024-09-10T09:35:47.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:47.585+0000 7fe080200640 1 -- 172.21.15.5:0/2981430443 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe07819b940 con 0x7fe0781049a0 2024-09-10T09:35:47.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:47.586+0000 7fe076000640 1 -- 172.21.15.5:0/2981430443 <== mon.1 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) ==== 74+0+24126 (secure 0 0 0) 0x7fe07819b940 con 0x7fe0781049a0 2024-09-10T09:35:47.589 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-10T09:35:47.589 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":92,"fsid":"bade4b50-6f56-11ef-bcea-c7b262605968","created":"2024-09-10T09:27:13.298498+0000","modified":"2024-09-10T09:35:40.676433+0000","last_up_change":"2024-09-10T09:34:30.730066+0000","last_in_change":"2024-09-10T09:34:07.340995+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T09:30:26.042255+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T09:35:24.461031+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T09:35:27.155349+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-10T09:35:35.121462+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"085749a8-e882-4377-92dd-34bb266c0f0b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":954535266},{"type":"v1","addr":"172.21.15.5:6803","nonce":954535266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":954535266},{"type":"v1","addr":"172.21.15.5:6805","nonce":954535266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":954535266},{"type":"v1","addr":"172.21.15.5:6809","nonce":954535266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":954535266},{"type":"v1","addr":"172.21.15.5:6807","nonce":954535266}]},"public_addr":"172.21.15.5:6803/954535266","cluster_addr":"172.21.15.5:6805/954535266","heartbeat_back_addr":"172.21.15.5:6809/954535266","heartbeat_front_addr":"172.21.15.5:6807/954535266","state":["exists","up"]},{"osd":1,"uuid":"0a76c795-2573-4a92-969a-2ec5dd65c094","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6800","nonce":733924375},{"type":"v1","addr":"172.21.15.102:6801","nonce":733924375}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6802","nonce":733924375},{"type":"v1","addr":"172.21.15.102:6803","nonce":733924375}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6806","nonce":733924375},{"type":"v1","addr":"172.21.15.102:6807","nonce":733924375}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6804","nonce":733924375},{"type":"v1","addr":"172.21.15.102:6805","nonce":733924375}]},"public_addr":"172.21.15.102:6801/733924375","cluster_addr":"172.21.15.102:6803/733924375","heartbeat_back_addr":"172.21.15.102:6807/733924375","heartbeat_front_addr":"172.21.15.102:6805/733924375","state":["exists","up"]},{"osd":2,"uuid":"0c98fcb8-60ab-4473-8995-1d9e97c577e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6800","nonce":2236204189},{"type":"v1","addr":"172.21.15.159:6801","nonce":2236204189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":2236204189},{"type":"v1","addr":"172.21.15.159:6803","nonce":2236204189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":2236204189},{"type":"v1","addr":"172.21.15.159:6807","nonce":2236204189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":2236204189},{"type":"v1","addr":"172.21.15.159:6805","nonce":2236204189}]},"public_addr":"172.21.15.159:6801/2236204189","cluster_addr":"172.21.15.159:6803/2236204189","heartbeat_back_addr":"172.21.15.159:6807/2236204189","heartbeat_front_addr":"172.21.15.159:6805/2236204189","state":["exists","up"]},{"osd":3,"uuid":"88eecdeb-5cee-4f78-a3f9-a66bf4bf772a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":2221468725},{"type":"v1","addr":"172.21.15.5:6811","nonce":2221468725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":2221468725},{"type":"v1","addr":"172.21.15.5:6813","nonce":2221468725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6816","nonce":2221468725},{"type":"v1","addr":"172.21.15.5:6817","nonce":2221468725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":2221468725},{"type":"v1","addr":"172.21.15.5:6815","nonce":2221468725}]},"public_addr":"172.21.15.5:6811/2221468725","cluster_addr":"172.21.15.5:6813/2221468725","heartbeat_back_addr":"172.21.15.5:6817/2221468725","heartbeat_front_addr":"172.21.15.5:6815/2221468725","state":["exists","up"]},{"osd":4,"uuid":"f2e7314d-7938-4668-9011-45c175f66b48","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6808","nonce":594465753},{"type":"v1","addr":"172.21.15.102:6809","nonce":594465753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6810","nonce":594465753},{"type":"v1","addr":"172.21.15.102:6811","nonce":594465753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6814","nonce":594465753},{"type":"v1","addr":"172.21.15.102:6815","nonce":594465753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6812","nonce":594465753},{"type":"v1","addr":"172.21.15.102:6813","nonce":594465753}]},"public_addr":"172.21.15.102:6809/594465753","cluster_addr":"172.21.15.102:6811/594465753","heartbeat_back_addr":"172.21.15.102:6815/594465753","heartbeat_front_addr":"172.21.15.102:6813/594465753","state":["exists","up"]},{"osd":5,"uuid":"0c0366d3-658b-4bf0-ba17-aff28a02e948","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":3230280402},{"type":"v1","addr":"172.21.15.159:6809","nonce":3230280402}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":3230280402},{"type":"v1","addr":"172.21.15.159:6811","nonce":3230280402}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":3230280402},{"type":"v1","addr":"172.21.15.159:6815","nonce":3230280402}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":3230280402},{"type":"v1","addr":"172.21.15.159:6813","nonce":3230280402}]},"public_addr":"172.21.15.159:6809/3230280402","cluster_addr":"172.21.15.159:6811/3230280402","heartbeat_back_addr":"172.21.15.159:6815/3230280402","heartbeat_front_addr":"172.21.15.159:6813/3230280402","state":["exists","up"]},{"osd":6,"uuid":"682670a6-51d5-46e8-af6e-dc2d8c1cf407","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":1907651140},{"type":"v1","addr":"172.21.15.5:6819","nonce":1907651140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":1907651140},{"type":"v1","addr":"172.21.15.5:6821","nonce":1907651140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":1907651140},{"type":"v1","addr":"172.21.15.5:6825","nonce":1907651140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":1907651140},{"type":"v1","addr":"172.21.15.5:6823","nonce":1907651140}]},"public_addr":"172.21.15.5:6819/1907651140","cluster_addr":"172.21.15.5:6821/1907651140","heartbeat_back_addr":"172.21.15.5:6825/1907651140","heartbeat_front_addr":"172.21.15.5:6823/1907651140","state":["exists","up"]},{"osd":7,"uuid":"8ae0f4a9-99e7-4563-aa2a-15169684a8b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6816","nonce":692040360},{"type":"v1","addr":"172.21.15.102:6817","nonce":692040360}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6818","nonce":692040360},{"type":"v1","addr":"172.21.15.102:6819","nonce":692040360}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6822","nonce":692040360},{"type":"v1","addr":"172.21.15.102:6823","nonce":692040360}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6820","nonce":692040360},{"type":"v1","addr":"172.21.15.102:6821","nonce":692040360}]},"public_addr":"172.21.15.102:6817/692040360","cluster_addr":"172.21.15.102:6819/692040360","heartbeat_back_addr":"172.21.15.102:6823/692040360","heartbeat_front_addr":"172.21.15.102:6821/692040360","state":["exists","up"]},{"osd":8,"uuid":"e37c845e-ced9-4413-9bbe-bc090537c2bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":2107291799},{"type":"v1","addr":"172.21.15.159:6817","nonce":2107291799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":2107291799},{"type":"v1","addr":"172.21.15.159:6819","nonce":2107291799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":2107291799},{"type":"v1","addr":"172.21.15.159:6823","nonce":2107291799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":2107291799},{"type":"v1","addr":"172.21.15.159:6821","nonce":2107291799}]},"public_addr":"172.21.15.159:6817/2107291799","cluster_addr":"172.21.15.159:6819/2107291799","heartbeat_back_addr":"172.21.15.159:6823/2107291799","heartbeat_front_addr":"172.21.15.159:6821/2107291799","state":["exists","up"]},{"osd":9,"uuid":"91a5b6cc-08da-47be-af12-d0896a849309","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":902431281},{"type":"v1","addr":"172.21.15.5:6827","nonce":902431281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":902431281},{"type":"v1","addr":"172.21.15.5:6829","nonce":902431281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":902431281},{"type":"v1","addr":"172.21.15.5:6833","nonce":902431281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":902431281},{"type":"v1","addr":"172.21.15.5:6831","nonce":902431281}]},"public_addr":"172.21.15.5:6827/902431281","cluster_addr":"172.21.15.5:6829/902431281","heartbeat_back_addr":"172.21.15.5:6833/902431281","heartbeat_front_addr":"172.21.15.5:6831/902431281","state":["exists","up"]},{"osd":10,"uuid":"2211030c-b00e-402b-b4d7-c587eb29a490","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6824","nonce":2853806211},{"type":"v1","addr":"172.21.15.102:6825","nonce":2853806211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6826","nonce":2853806211},{"type":"v1","addr":"172.21.15.102:6827","nonce":2853806211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6830","nonce":2853806211},{"type":"v1","addr":"172.21.15.102:6831","nonce":2853806211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6828","nonce":2853806211},{"type":"v1","addr":"172.21.15.102:6829","nonce":2853806211}]},"public_addr":"172.21.15.102:6825/2853806211","cluster_addr":"172.21.15.102:6827/2853806211","heartbeat_back_addr":"172.21.15.102:6831/2853806211","heartbeat_front_addr":"172.21.15.102:6829/2853806211","state":["exists","up"]},{"osd":11,"uuid":"4e9699ec-8ff2-4005-9d84-af5600b3a2ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6824","nonce":969941218},{"type":"v1","addr":"172.21.15.159:6825","nonce":969941218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":969941218},{"type":"v1","addr":"172.21.15.159:6827","nonce":969941218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6830","nonce":969941218},{"type":"v1","addr":"172.21.15.159:6831","nonce":969941218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":969941218},{"type":"v1","addr":"172.21.15.159:6829","nonce":969941218}]},"public_addr":"172.21.15.159:6825/969941218","cluster_addr":"172.21.15.159:6827/969941218","heartbeat_back_addr":"172.21.15.159:6831/969941218","heartbeat_front_addr":"172.21.15.159:6829/969941218","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:29:34.416490+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:30:00.246101+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:30:24.300464+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:30:54.663623+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:31:21.426292+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:31:46.738914+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:32:16.196571+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:32:42.549243+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:33:08.178651+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:33:37.477911+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:34:03.114026+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:34:28.891924+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.5:6801/1527422686":"2024-09-11T09:28:05.436742+0000","172.21.15.5:0/2276144345":"2024-09-11T09:28:05.436742+0000","172.21.15.5:0/412040104":"2024-09-11T09:28:05.436742+0000","172.21.15.5:6800/1527422686":"2024-09-11T09:28:05.436742+0000","172.21.15.5:0/1856581170":"2024-09-11T09:28:05.436742+0000","172.21.15.5:0/2847668298":"2024-09-11T09:27:37.778500+0000","172.21.15.5:6801/14425913":"2024-09-11T09:27:37.778500+0000","172.21.15.5:0/166910782":"2024-09-11T09:27:37.778500+0000","172.21.15.5:6800/14425913":"2024-09-11T09:27:37.778500+0000","172.21.15.5:0/1711858179":"2024-09-11T09:27:37.778500+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T09:35:47.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:47.589+0000 7fe080200640 1 -- 172.21.15.5:0/2981430443 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fe0600a2c00 msgr2=0x7fe0600a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:35:47.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:47.589+0000 7fe080200640 1 --2- 172.21.15.5:0/2981430443 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fe0600a2c00 0x7fe0600a50c0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fe06c009c90 tx=0x7fe06c005f90 comp rx=0 tx=0).stop 2024-09-10T09:35:47.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:47.589+0000 7fe080200640 1 -- 172.21.15.5:0/2981430443 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe0781049a0 msgr2=0x7fe07819ab20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:35:47.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:47.589+0000 7fe080200640 1 --2- 172.21.15.5:0/2981430443 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe0781049a0 0x7fe07819ab20 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fe06400e9f0 tx=0x7fe06400eeb0 comp rx=0 tx=0).stop 2024-09-10T09:35:47.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:47.589+0000 7fe080200640 1 -- 172.21.15.5:0/2981430443 shutdown_connections 2024-09-10T09:35:47.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:47.589+0000 7fe080200640 1 --2- 172.21.15.5:0/2981430443 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fe0600a2c00 0x7fe0600a50c0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:35:47.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:47.589+0000 7fe080200640 1 --2- 172.21.15.5:0/2981430443 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe078109120 0x7fe07819eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:35:47.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:47.589+0000 7fe080200640 1 --2- 172.21.15.5:0/2981430443 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe0781049a0 0x7fe07819ab20 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:35:47.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:47.589+0000 7fe080200640 1 --2- 172.21.15.5:0/2981430443 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe078103ff0 0x7fe07819a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:35:47.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:47.589+0000 7fe080200640 1 -- 172.21.15.5:0/2981430443 >> 172.21.15.5:0/2981430443 conn(0x7fe0780ffa90 msgr2=0x7fe0781004b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:35:47.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:47.589+0000 7fe080200640 1 -- 172.21.15.5:0/2981430443 shutdown_connections 2024-09-10T09:35:47.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:47.589+0000 7fe080200640 1 -- 172.21.15.5:0/2981430443 wait complete. 2024-09-10T09:35:47.944 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:35:47 smithi005 ceph-mon[30350]: from='client.? 172.21.15.5:0/2981430443' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T09:35:48.013 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:35:47 smithi159 ceph-mon[35513]: from='client.? 172.21.15.5:0/2981430443' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T09:35:48.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:35:47 smithi102 ceph-mon[35442]: from='client.? 172.21.15.5:0/2981430443' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T09:35:48.317 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T09:35:48.692 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:35:49.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:35:48 smithi005 ceph-mon[30350]: pgmap v313: 129 pgs: 129 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T09:35:49.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:35:48 smithi159 ceph-mon[35513]: pgmap v313: 129 pgs: 129 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T09:35:49.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:35:48 smithi102 ceph-mon[35442]: pgmap v313: 129 pgs: 129 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T09:35:49.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:49.319+0000 7f0a36600640 1 -- 172.21.15.5:0/1760628026 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0a301019c0 msgr2=0x7f0a30101e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:35:49.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:49.319+0000 7f0a36600640 1 --2- 172.21.15.5:0/1760628026 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0a301019c0 0x7f0a30101e20 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f0a1c009930 tx=0x7f0a1c02a420 comp rx=0 tx=0).stop 2024-09-10T09:35:49.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:49.319+0000 7f0a36600640 1 -- 172.21.15.5:0/1760628026 shutdown_connections 2024-09-10T09:35:49.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:49.319+0000 7f0a36600640 1 --2- 172.21.15.5:0/1760628026 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0a30102360 0x7f0a3010e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:35:49.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:49.319+0000 7f0a36600640 1 --2- 172.21.15.5:0/1760628026 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0a301019c0 0x7f0a30101e20 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:35:49.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:49.319+0000 7f0a36600640 1 --2- 172.21.15.5:0/1760628026 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0a301079c0 0x7f0a30107da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:35:49.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:49.319+0000 7f0a36600640 1 -- 172.21.15.5:0/1760628026 >> 172.21.15.5:0/1760628026 conn(0x7f0a300fd740 msgr2=0x7f0a300ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:35:49.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:49.319+0000 7f0a36600640 1 -- 172.21.15.5:0/1760628026 shutdown_connections 2024-09-10T09:35:49.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:49.319+0000 7f0a36600640 1 -- 172.21.15.5:0/1760628026 wait complete. 2024-09-10T09:35:49.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:49.321+0000 7f0a36600640 1 Processor -- start 2024-09-10T09:35:49.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:49.321+0000 7f0a36600640 1 -- start start 2024-09-10T09:35:49.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:49.321+0000 7f0a36600640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0a301019c0 0x7f0a300744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:35:49.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:49.321+0000 7f0a36600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0a30102360 0x7f0a30074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:35:49.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:49.321+0000 7f0a36600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0a301079c0 0x7f0a30074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:35:49.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:49.321+0000 7f0a36600640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f0a30111e10 con 0x7f0a30102360 2024-09-10T09:35:49.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:49.321+0000 7f0a36600640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f0a30111c90 con 0x7f0a301079c0 2024-09-10T09:35:49.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:49.322+0000 7f0a36600640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f0a30111f90 con 0x7f0a301019c0 2024-09-10T09:35:49.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:49.322+0000 7f0a2fe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0a30102360 0x7f0a30074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:35:49.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:49.322+0000 7f0a2fe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0a30102360 0x7f0a30074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39514/0 (socket says 172.21.15.5:39514) 2024-09-10T09:35:49.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:49.322+0000 7f0a2fe00640 1 -- 172.21.15.5:0/157429297 learned_addr learned my addr 172.21.15.5:0/157429297 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:35:49.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:49.322+0000 7f0a35200640 1 --2- 172.21.15.5:0/157429297 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0a301019c0 0x7f0a300744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:35:49.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:49.322+0000 7f0a35c00640 1 --2- 172.21.15.5:0/157429297 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0a301079c0 0x7f0a30074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:35:49.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:49.322+0000 7f0a2fe00640 1 -- 172.21.15.5:0/157429297 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0a301019c0 msgr2=0x7f0a300744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:35:49.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:49.322+0000 7f0a2fe00640 1 --2- 172.21.15.5:0/157429297 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0a301019c0 0x7f0a300744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:35:49.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:49.322+0000 7f0a2fe00640 1 -- 172.21.15.5:0/157429297 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0a301079c0 msgr2=0x7f0a30074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:35:49.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:49.322+0000 7f0a2fe00640 1 --2- 172.21.15.5:0/157429297 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0a301079c0 0x7f0a30074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:35:49.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:49.322+0000 7f0a2fe00640 1 -- 172.21.15.5:0/157429297 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a301a67b0 con 0x7f0a30102360 2024-09-10T09:35:49.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:49.322+0000 7f0a35200640 1 --2- 172.21.15.5:0/157429297 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0a301019c0 0x7f0a300744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:35:49.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:49.323+0000 7f0a2fe00640 1 --2- 172.21.15.5:0/157429297 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0a30102360 0x7f0a30074a30 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f0a1c02a940 tx=0x7f0a1c003b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:35:49.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:49.323+0000 7f0a2d600640 1 -- 172.21.15.5:0/157429297 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a1c03f070 con 0x7f0a30102360 2024-09-10T09:35:49.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:49.323+0000 7f0a2d600640 1 -- 172.21.15.5:0/157429297 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0a1c031c90 con 0x7f0a30102360 2024-09-10T09:35:49.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:49.323+0000 7f0a36600640 1 -- 172.21.15.5:0/157429297 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a301a6a40 con 0x7f0a30102360 2024-09-10T09:35:49.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:49.323+0000 7f0a2d600640 1 -- 172.21.15.5:0/157429297 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a1c035030 con 0x7f0a30102360 2024-09-10T09:35:49.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:49.323+0000 7f0a36600640 1 -- 172.21.15.5:0/157429297 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a301a6f20 con 0x7f0a30102360 2024-09-10T09:35:49.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:49.324+0000 7f0a36600640 1 -- 172.21.15.5:0/157429297 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0a30101e20 con 0x7f0a30102360 2024-09-10T09:35:49.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:49.326+0000 7f0a2d600640 1 -- 172.21.15.5:0/157429297 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7f0a1c042070 con 0x7f0a30102360 2024-09-10T09:35:49.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:49.328+0000 7f0a2d600640 1 --2- 172.21.15.5:0/157429297 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f0a180a2b80 0x7f0a180a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:35:49.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:49.328+0000 7f0a2d600640 1 -- 172.21.15.5:0/157429297 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8802+0+0 (secure 0 0 0) 0x7f0a1c0ef8e0 con 0x7f0a30102360 2024-09-10T09:35:49.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:49.328+0000 7f0a35200640 1 --2- 172.21.15.5:0/157429297 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f0a180a2b80 0x7f0a180a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:35:49.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:49.329+0000 7f0a35200640 1 --2- 172.21.15.5:0/157429297 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f0a180a2b80 0x7f0a180a5040 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f0a24009f40 tx=0x7f0a24005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:35:49.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:49.332+0000 7f0a2d600640 1 -- 172.21.15.5:0/157429297 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f0a1c0bbf50 con 0x7f0a30102360 2024-09-10T09:35:49.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:49.556+0000 7f0a36600640 1 -- 172.21.15.5:0/157429297 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0a30109f30 con 0x7f0a30102360 2024-09-10T09:35:49.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:49.557+0000 7f0a2d600640 1 -- 172.21.15.5:0/157429297 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+11833 (secure 0 0 0) 0x7f0a1c0c0250 con 0x7f0a30102360 2024-09-10T09:35:49.558 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-10T09:35:49.559 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":17,"btime":"2024-09-10T09:35:41:631650+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14454,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/2970594311","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2970594311},{"type":"v1","addr":"172.21.15.5:6835","nonce":2970594311}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14514,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6839/3616502174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":3616502174},{"type":"v1","addr":"172.21.15.5:6839","nonce":3616502174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14520,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6841/2073599842","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":2073599842},{"type":"v1","addr":"172.21.15.5:6841","nonce":2073599842}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24443,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6833/2457007427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":2457007427},{"type":"v1","addr":"172.21.15.102:6833","nonce":2457007427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24460,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6833/356445159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":356445159},{"type":"v1","addr":"172.21.15.159:6833","nonce":356445159}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24467,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/2894314507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":2894314507},{"type":"v1","addr":"172.21.15.5:6837","nonce":2894314507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24479,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6835/1647549111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":1647549111},{"type":"v1","addr":"172.21.15.102:6835","nonce":1647549111}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24505,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6835/952657242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":952657242},{"type":"v1","addr":"172.21.15.159:6835","nonce":952657242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24506,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.102:6837/2689776764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":2689776764},{"type":"v1","addr":"172.21.15.102:6837","nonce":2689776764}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24511,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6837/620627639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":620627639},{"type":"v1","addr":"172.21.15.159:6837","nonce":620627639}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24535,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.159:6839/2073739599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":2073739599},{"type":"v1","addr":"172.21.15.159:6839","nonce":2073739599}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T09:35:29.626287+0000","modified":"2024-09-10T09:35:40.676575+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24542},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24542":{"gid":24542,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.102:6839/335409749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6838","nonce":335409749},{"type":"v1","addr":"172.21.15.102:6839","nonce":335409749}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24542,"qdb_cluster":[24542]},"id":1}]} 2024-09-10T09:35:49.559 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 17 2024-09-10T09:35:49.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:49.559+0000 7f0a36600640 1 -- 172.21.15.5:0/157429297 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f0a180a2b80 msgr2=0x7f0a180a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:35:49.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:49.559+0000 7f0a36600640 1 --2- 172.21.15.5:0/157429297 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f0a180a2b80 0x7f0a180a5040 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f0a24009f40 tx=0x7f0a24005f90 comp rx=0 tx=0).stop 2024-09-10T09:35:49.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:49.559+0000 7f0a36600640 1 -- 172.21.15.5:0/157429297 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0a30102360 msgr2=0x7f0a30074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:35:49.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:49.559+0000 7f0a36600640 1 --2- 172.21.15.5:0/157429297 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0a30102360 0x7f0a30074a30 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f0a1c02a940 tx=0x7f0a1c003b20 comp rx=0 tx=0).stop 2024-09-10T09:35:49.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:49.559+0000 7f0a36600640 1 -- 172.21.15.5:0/157429297 shutdown_connections 2024-09-10T09:35:49.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:49.559+0000 7f0a36600640 1 --2- 172.21.15.5:0/157429297 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f0a180a2b80 0x7f0a180a5040 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:35:49.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:49.559+0000 7f0a36600640 1 --2- 172.21.15.5:0/157429297 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0a301079c0 0x7f0a30074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:35:49.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:49.559+0000 7f0a36600640 1 --2- 172.21.15.5:0/157429297 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0a30102360 0x7f0a30074a30 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:35:49.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:49.559+0000 7f0a36600640 1 --2- 172.21.15.5:0/157429297 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0a301019c0 0x7f0a300744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:35:49.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:49.559+0000 7f0a36600640 1 -- 172.21.15.5:0/157429297 >> 172.21.15.5:0/157429297 conn(0x7f0a300fd740 msgr2=0x7f0a300fe020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:35:49.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:49.559+0000 7f0a36600640 1 -- 172.21.15.5:0/157429297 shutdown_connections 2024-09-10T09:35:49.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:49.559+0000 7f0a36600640 1 -- 172.21.15.5:0/157429297 wait complete. 2024-09-10T09:35:49.889 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:35:49 smithi005 ceph-mon[30350]: from='client.? 172.21.15.5:0/157429297' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:35:50.035 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:35:49 smithi102 ceph-mon[35442]: from='client.? 172.21.15.5:0/157429297' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:35:50.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:35:49 smithi159 ceph-mon[35513]: from='client.? 172.21.15.5:0/157429297' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:35:50.423 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-10T09:35:50.803 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:35:50.967 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:35:50 smithi005 ceph-mon[30350]: pgmap v314: 129 pgs: 129 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T09:35:51.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:35:50 smithi159 ceph-mon[35513]: pgmap v314: 129 pgs: 129 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T09:35:51.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:35:50 smithi102 ceph-mon[35442]: pgmap v314: 129 pgs: 129 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T09:35:51.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:51.364+0000 7f5b2cc00640 1 -- 172.21.15.5:0/2516917015 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5b28102790 msgr2=0x7f5b281067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:35:51.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:51.364+0000 7f5b2cc00640 1 --2- 172.21.15.5:0/2516917015 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5b28102790 0x7f5b281067e0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f5b10009900 tx=0x7f5b1002a3f0 comp rx=0 tx=0).stop 2024-09-10T09:35:51.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:51.365+0000 7f5b2cc00640 1 -- 172.21.15.5:0/2516917015 shutdown_connections 2024-09-10T09:35:51.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:51.365+0000 7f5b2cc00640 1 --2- 172.21.15.5:0/2516917015 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5b28106f10 0x7f5b2810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:35:51.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:51.365+0000 7f5b2cc00640 1 --2- 172.21.15.5:0/2516917015 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5b28102790 0x7f5b281067e0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:35:51.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:51.365+0000 7f5b2cc00640 1 --2- 172.21.15.5:0/2516917015 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5b28101de0 0x7f5b281021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:35:51.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:51.365+0000 7f5b2cc00640 1 -- 172.21.15.5:0/2516917015 >> 172.21.15.5:0/2516917015 conn(0x7f5b280fd8c0 msgr2=0x7f5b280ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:35:51.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:51.365+0000 7f5b2cc00640 1 -- 172.21.15.5:0/2516917015 shutdown_connections 2024-09-10T09:35:51.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:51.365+0000 7f5b2cc00640 1 -- 172.21.15.5:0/2516917015 wait complete. 2024-09-10T09:35:51.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:51.366+0000 7f5b2cc00640 1 Processor -- start 2024-09-10T09:35:51.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:51.366+0000 7f5b2cc00640 1 -- start start 2024-09-10T09:35:51.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:51.366+0000 7f5b2cc00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5b28101de0 0x7f5b280744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:35:51.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:51.366+0000 7f5b2cc00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5b28102790 0x7f5b28074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:35:51.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:51.366+0000 7f5b2cc00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5b28106f10 0x7f5b28074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:35:51.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:51.366+0000 7f5b2cc00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f5b28111fc0 con 0x7f5b28101de0 2024-09-10T09:35:51.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:51.366+0000 7f5b2cc00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f5b28111e40 con 0x7f5b28102790 2024-09-10T09:35:51.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:51.366+0000 7f5b2cc00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f5b28112140 con 0x7f5b28106f10 2024-09-10T09:35:51.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:51.366+0000 7f5b25e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5b28101de0 0x7f5b280744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:35:51.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:51.367+0000 7f5b25e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5b28101de0 0x7f5b280744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43884/0 (socket says 172.21.15.5:43884) 2024-09-10T09:35:51.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:51.367+0000 7f5b25e00640 1 -- 172.21.15.5:0/3796749000 learned_addr learned my addr 172.21.15.5:0/3796749000 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:35:51.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:51.367+0000 7f5b1d400640 1 --2- 172.21.15.5:0/3796749000 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5b28102790 0x7f5b28074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:35:51.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:51.367+0000 7f5b26800640 1 --2- 172.21.15.5:0/3796749000 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5b28106f10 0x7f5b28074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:35:51.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:51.367+0000 7f5b25e00640 1 -- 172.21.15.5:0/3796749000 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5b28106f10 msgr2=0x7f5b28074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:35:51.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:51.367+0000 7f5b25e00640 1 --2- 172.21.15.5:0/3796749000 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5b28106f10 0x7f5b28074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:35:51.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:51.367+0000 7f5b25e00640 1 -- 172.21.15.5:0/3796749000 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5b28102790 msgr2=0x7f5b28074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:35:51.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:51.367+0000 7f5b25e00640 1 --2- 172.21.15.5:0/3796749000 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5b28102790 0x7f5b28074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:35:51.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:51.367+0000 7f5b25e00640 1 -- 172.21.15.5:0/3796749000 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b281a6990 con 0x7f5b28101de0 2024-09-10T09:35:51.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:51.367+0000 7f5b25e00640 1 --2- 172.21.15.5:0/3796749000 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5b28101de0 0x7f5b280744f0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f5b1800e9f0 tx=0x7f5b1800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:35:51.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:51.368+0000 7f5b1f400640 1 -- 172.21.15.5:0/3796749000 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b1800cd80 con 0x7f5b28101de0 2024-09-10T09:35:51.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:51.368+0000 7f5b1f400640 1 -- 172.21.15.5:0/3796749000 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5b18010910 con 0x7f5b28101de0 2024-09-10T09:35:51.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:51.368+0000 7f5b2cc00640 1 -- 172.21.15.5:0/3796749000 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b281a6c80 con 0x7f5b28101de0 2024-09-10T09:35:51.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:51.368+0000 7f5b1f400640 1 -- 172.21.15.5:0/3796749000 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b18018990 con 0x7f5b28101de0 2024-09-10T09:35:51.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:51.368+0000 7f5b2cc00640 1 -- 172.21.15.5:0/3796749000 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b281a7190 con 0x7f5b28101de0 2024-09-10T09:35:51.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:51.369+0000 7f5b2cc00640 1 -- 172.21.15.5:0/3796749000 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b281021c0 con 0x7f5b28101de0 2024-09-10T09:35:51.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:51.371+0000 7f5b1f400640 1 -- 172.21.15.5:0/3796749000 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7f5b18010ac0 con 0x7f5b28101de0 2024-09-10T09:35:51.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:51.372+0000 7f5b1f400640 1 --2- 172.21.15.5:0/3796749000 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f5b0c0a2c00 0x7f5b0c0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:35:51.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:51.372+0000 7f5b1f400640 1 -- 172.21.15.5:0/3796749000 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8802+0+0 (secure 0 0 0) 0x7f5b180cf550 con 0x7f5b28101de0 2024-09-10T09:35:51.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:51.372+0000 7f5b1d400640 1 --2- 172.21.15.5:0/3796749000 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f5b0c0a2c00 0x7f5b0c0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:35:51.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:51.373+0000 7f5b1d400640 1 --2- 172.21.15.5:0/3796749000 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f5b0c0a2c00 0x7f5b0c0a50c0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f5b10000c00 tx=0x7f5b10005b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:35:51.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:51.377+0000 7f5b1f400640 1 -- 172.21.15.5:0/3796749000 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f5b1809bbc0 con 0x7f5b28101de0 2024-09-10T09:35:51.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:51.564+0000 7f5b2cc00640 1 -- 172.21.15.5:0/3796749000 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5b2810a150 con 0x7f5b28101de0 2024-09-10T09:35:51.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:51.566+0000 7f5b1f400640 1 -- 172.21.15.5:0/3796749000 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) ==== 74+0+24126 (secure 0 0 0) 0x7f5b1809fec0 con 0x7f5b28101de0 2024-09-10T09:35:51.568 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-10T09:35:51.569 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":92,"fsid":"bade4b50-6f56-11ef-bcea-c7b262605968","created":"2024-09-10T09:27:13.298498+0000","modified":"2024-09-10T09:35:40.676433+0000","last_up_change":"2024-09-10T09:34:30.730066+0000","last_in_change":"2024-09-10T09:34:07.340995+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T09:30:26.042255+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T09:35:24.461031+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T09:35:27.155349+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-10T09:35:35.121462+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"085749a8-e882-4377-92dd-34bb266c0f0b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":954535266},{"type":"v1","addr":"172.21.15.5:6803","nonce":954535266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":954535266},{"type":"v1","addr":"172.21.15.5:6805","nonce":954535266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":954535266},{"type":"v1","addr":"172.21.15.5:6809","nonce":954535266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":954535266},{"type":"v1","addr":"172.21.15.5:6807","nonce":954535266}]},"public_addr":"172.21.15.5:6803/954535266","cluster_addr":"172.21.15.5:6805/954535266","heartbeat_back_addr":"172.21.15.5:6809/954535266","heartbeat_front_addr":"172.21.15.5:6807/954535266","state":["exists","up"]},{"osd":1,"uuid":"0a76c795-2573-4a92-969a-2ec5dd65c094","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6800","nonce":733924375},{"type":"v1","addr":"172.21.15.102:6801","nonce":733924375}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6802","nonce":733924375},{"type":"v1","addr":"172.21.15.102:6803","nonce":733924375}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6806","nonce":733924375},{"type":"v1","addr":"172.21.15.102:6807","nonce":733924375}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6804","nonce":733924375},{"type":"v1","addr":"172.21.15.102:6805","nonce":733924375}]},"public_addr":"172.21.15.102:6801/733924375","cluster_addr":"172.21.15.102:6803/733924375","heartbeat_back_addr":"172.21.15.102:6807/733924375","heartbeat_front_addr":"172.21.15.102:6805/733924375","state":["exists","up"]},{"osd":2,"uuid":"0c98fcb8-60ab-4473-8995-1d9e97c577e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6800","nonce":2236204189},{"type":"v1","addr":"172.21.15.159:6801","nonce":2236204189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":2236204189},{"type":"v1","addr":"172.21.15.159:6803","nonce":2236204189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":2236204189},{"type":"v1","addr":"172.21.15.159:6807","nonce":2236204189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":2236204189},{"type":"v1","addr":"172.21.15.159:6805","nonce":2236204189}]},"public_addr":"172.21.15.159:6801/2236204189","cluster_addr":"172.21.15.159:6803/2236204189","heartbeat_back_addr":"172.21.15.159:6807/2236204189","heartbeat_front_addr":"172.21.15.159:6805/2236204189","state":["exists","up"]},{"osd":3,"uuid":"88eecdeb-5cee-4f78-a3f9-a66bf4bf772a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":2221468725},{"type":"v1","addr":"172.21.15.5:6811","nonce":2221468725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":2221468725},{"type":"v1","addr":"172.21.15.5:6813","nonce":2221468725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6816","nonce":2221468725},{"type":"v1","addr":"172.21.15.5:6817","nonce":2221468725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":2221468725},{"type":"v1","addr":"172.21.15.5:6815","nonce":2221468725}]},"public_addr":"172.21.15.5:6811/2221468725","cluster_addr":"172.21.15.5:6813/2221468725","heartbeat_back_addr":"172.21.15.5:6817/2221468725","heartbeat_front_addr":"172.21.15.5:6815/2221468725","state":["exists","up"]},{"osd":4,"uuid":"f2e7314d-7938-4668-9011-45c175f66b48","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6808","nonce":594465753},{"type":"v1","addr":"172.21.15.102:6809","nonce":594465753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6810","nonce":594465753},{"type":"v1","addr":"172.21.15.102:6811","nonce":594465753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6814","nonce":594465753},{"type":"v1","addr":"172.21.15.102:6815","nonce":594465753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6812","nonce":594465753},{"type":"v1","addr":"172.21.15.102:6813","nonce":594465753}]},"public_addr":"172.21.15.102:6809/594465753","cluster_addr":"172.21.15.102:6811/594465753","heartbeat_back_addr":"172.21.15.102:6815/594465753","heartbeat_front_addr":"172.21.15.102:6813/594465753","state":["exists","up"]},{"osd":5,"uuid":"0c0366d3-658b-4bf0-ba17-aff28a02e948","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":3230280402},{"type":"v1","addr":"172.21.15.159:6809","nonce":3230280402}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":3230280402},{"type":"v1","addr":"172.21.15.159:6811","nonce":3230280402}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":3230280402},{"type":"v1","addr":"172.21.15.159:6815","nonce":3230280402}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":3230280402},{"type":"v1","addr":"172.21.15.159:6813","nonce":3230280402}]},"public_addr":"172.21.15.159:6809/3230280402","cluster_addr":"172.21.15.159:6811/3230280402","heartbeat_back_addr":"172.21.15.159:6815/3230280402","heartbeat_front_addr":"172.21.15.159:6813/3230280402","state":["exists","up"]},{"osd":6,"uuid":"682670a6-51d5-46e8-af6e-dc2d8c1cf407","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":1907651140},{"type":"v1","addr":"172.21.15.5:6819","nonce":1907651140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":1907651140},{"type":"v1","addr":"172.21.15.5:6821","nonce":1907651140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":1907651140},{"type":"v1","addr":"172.21.15.5:6825","nonce":1907651140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":1907651140},{"type":"v1","addr":"172.21.15.5:6823","nonce":1907651140}]},"public_addr":"172.21.15.5:6819/1907651140","cluster_addr":"172.21.15.5:6821/1907651140","heartbeat_back_addr":"172.21.15.5:6825/1907651140","heartbeat_front_addr":"172.21.15.5:6823/1907651140","state":["exists","up"]},{"osd":7,"uuid":"8ae0f4a9-99e7-4563-aa2a-15169684a8b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6816","nonce":692040360},{"type":"v1","addr":"172.21.15.102:6817","nonce":692040360}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6818","nonce":692040360},{"type":"v1","addr":"172.21.15.102:6819","nonce":692040360}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6822","nonce":692040360},{"type":"v1","addr":"172.21.15.102:6823","nonce":692040360}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6820","nonce":692040360},{"type":"v1","addr":"172.21.15.102:6821","nonce":692040360}]},"public_addr":"172.21.15.102:6817/692040360","cluster_addr":"172.21.15.102:6819/692040360","heartbeat_back_addr":"172.21.15.102:6823/692040360","heartbeat_front_addr":"172.21.15.102:6821/692040360","state":["exists","up"]},{"osd":8,"uuid":"e37c845e-ced9-4413-9bbe-bc090537c2bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":2107291799},{"type":"v1","addr":"172.21.15.159:6817","nonce":2107291799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":2107291799},{"type":"v1","addr":"172.21.15.159:6819","nonce":2107291799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":2107291799},{"type":"v1","addr":"172.21.15.159:6823","nonce":2107291799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":2107291799},{"type":"v1","addr":"172.21.15.159:6821","nonce":2107291799}]},"public_addr":"172.21.15.159:6817/2107291799","cluster_addr":"172.21.15.159:6819/2107291799","heartbeat_back_addr":"172.21.15.159:6823/2107291799","heartbeat_front_addr":"172.21.15.159:6821/2107291799","state":["exists","up"]},{"osd":9,"uuid":"91a5b6cc-08da-47be-af12-d0896a849309","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":902431281},{"type":"v1","addr":"172.21.15.5:6827","nonce":902431281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":902431281},{"type":"v1","addr":"172.21.15.5:6829","nonce":902431281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":902431281},{"type":"v1","addr":"172.21.15.5:6833","nonce":902431281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":902431281},{"type":"v1","addr":"172.21.15.5:6831","nonce":902431281}]},"public_addr":"172.21.15.5:6827/902431281","cluster_addr":"172.21.15.5:6829/902431281","heartbeat_back_addr":"172.21.15.5:6833/902431281","heartbeat_front_addr":"172.21.15.5:6831/902431281","state":["exists","up"]},{"osd":10,"uuid":"2211030c-b00e-402b-b4d7-c587eb29a490","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6824","nonce":2853806211},{"type":"v1","addr":"172.21.15.102:6825","nonce":2853806211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6826","nonce":2853806211},{"type":"v1","addr":"172.21.15.102:6827","nonce":2853806211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6830","nonce":2853806211},{"type":"v1","addr":"172.21.15.102:6831","nonce":2853806211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6828","nonce":2853806211},{"type":"v1","addr":"172.21.15.102:6829","nonce":2853806211}]},"public_addr":"172.21.15.102:6825/2853806211","cluster_addr":"172.21.15.102:6827/2853806211","heartbeat_back_addr":"172.21.15.102:6831/2853806211","heartbeat_front_addr":"172.21.15.102:6829/2853806211","state":["exists","up"]},{"osd":11,"uuid":"4e9699ec-8ff2-4005-9d84-af5600b3a2ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6824","nonce":969941218},{"type":"v1","addr":"172.21.15.159:6825","nonce":969941218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":969941218},{"type":"v1","addr":"172.21.15.159:6827","nonce":969941218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6830","nonce":969941218},{"type":"v1","addr":"172.21.15.159:6831","nonce":969941218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":969941218},{"type":"v1","addr":"172.21.15.159:6829","nonce":969941218}]},"public_addr":"172.21.15.159:6825/969941218","cluster_addr":"172.21.15.159:6827/969941218","heartbeat_back_addr":"172.21.15.159:6831/969941218","heartbeat_front_addr":"172.21.15.159:6829/969941218","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:29:34.416490+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:30:00.246101+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:30:24.300464+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:30:54.663623+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:31:21.426292+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:31:46.738914+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:32:16.196571+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:32:42.549243+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:33:08.178651+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:33:37.477911+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:34:03.114026+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:34:28.891924+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.5:6801/1527422686":"2024-09-11T09:28:05.436742+0000","172.21.15.5:0/2276144345":"2024-09-11T09:28:05.436742+0000","172.21.15.5:0/412040104":"2024-09-11T09:28:05.436742+0000","172.21.15.5:6800/1527422686":"2024-09-11T09:28:05.436742+0000","172.21.15.5:0/1856581170":"2024-09-11T09:28:05.436742+0000","172.21.15.5:0/2847668298":"2024-09-11T09:27:37.778500+0000","172.21.15.5:6801/14425913":"2024-09-11T09:27:37.778500+0000","172.21.15.5:0/166910782":"2024-09-11T09:27:37.778500+0000","172.21.15.5:6800/14425913":"2024-09-11T09:27:37.778500+0000","172.21.15.5:0/1711858179":"2024-09-11T09:27:37.778500+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T09:35:51.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:51.568+0000 7f5b2cc00640 1 -- 172.21.15.5:0/3796749000 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f5b0c0a2c00 msgr2=0x7f5b0c0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:35:51.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:51.568+0000 7f5b2cc00640 1 --2- 172.21.15.5:0/3796749000 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f5b0c0a2c00 0x7f5b0c0a50c0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f5b10000c00 tx=0x7f5b10005b50 comp rx=0 tx=0).stop 2024-09-10T09:35:51.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:51.568+0000 7f5b2cc00640 1 -- 172.21.15.5:0/3796749000 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5b28101de0 msgr2=0x7f5b280744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:35:51.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:51.568+0000 7f5b2cc00640 1 --2- 172.21.15.5:0/3796749000 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5b28101de0 0x7f5b280744f0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f5b1800e9f0 tx=0x7f5b1800eeb0 comp rx=0 tx=0).stop 2024-09-10T09:35:51.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:51.569+0000 7f5b2cc00640 1 -- 172.21.15.5:0/3796749000 shutdown_connections 2024-09-10T09:35:51.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:51.569+0000 7f5b2cc00640 1 --2- 172.21.15.5:0/3796749000 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f5b0c0a2c00 0x7f5b0c0a50c0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:35:51.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:51.569+0000 7f5b2cc00640 1 --2- 172.21.15.5:0/3796749000 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5b28106f10 0x7f5b28074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:35:51.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:51.569+0000 7f5b2cc00640 1 --2- 172.21.15.5:0/3796749000 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5b28102790 0x7f5b28074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:35:51.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:51.569+0000 7f5b2cc00640 1 --2- 172.21.15.5:0/3796749000 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5b28101de0 0x7f5b280744f0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:35:51.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:51.569+0000 7f5b2cc00640 1 -- 172.21.15.5:0/3796749000 >> 172.21.15.5:0/3796749000 conn(0x7f5b280fd8c0 msgr2=0x7f5b280fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:35:51.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:51.569+0000 7f5b2cc00640 1 -- 172.21.15.5:0/3796749000 shutdown_connections 2024-09-10T09:35:51.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:51.569+0000 7f5b2cc00640 1 -- 172.21.15.5:0/3796749000 wait complete. 2024-09-10T09:35:51.925 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:35:51 smithi005 ceph-mon[30350]: from='client.? 172.21.15.5:0/3796749000' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T09:35:52.101 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:35:51 smithi159 ceph-mon[35513]: from='client.? 172.21.15.5:0/3796749000' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T09:35:52.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:35:51 smithi102 ceph-mon[35442]: from='client.? 172.21.15.5:0/3796749000' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T09:35:52.569 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 17, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-10T09:35:29.626287+0000', 'modified': '2024-09-10T09:35:40.676575+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 24542}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24542': {'gid': 24542, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.102:6839/335409749', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.102:6838', 'nonce': 335409749}, {'type': 'v1', 'addr': '172.21.15.102:6839', 'nonce': 335409749}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1, 'qdb_leader': 24542, 'qdb_cluster': [24542]} 2024-09-10T09:35:52.569 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-10T09:35:52.569 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph tell mds.b status 2024-09-10T09:35:52.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:35:52 smithi005 ceph-mon[30350]: pgmap v315: 129 pgs: 129 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T09:35:52.959 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:35:53.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:35:52 smithi159 ceph-mon[35513]: pgmap v315: 129 pgs: 129 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T09:35:53.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:35:52 smithi102 ceph-mon[35442]: pgmap v315: 129 pgs: 129 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T09:35:53.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:53.559+0000 7fb47dc00640 1 -- 172.21.15.5:0/203637749 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb47810e420 msgr2=0x7fb478110810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:35:53.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:53.559+0000 7fb47dc00640 1 --2- 172.21.15.5:0/203637749 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb47810e420 0x7fb478110810 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fb470009ca0 tx=0x7fb47002a950 comp rx=0 tx=0).stop 2024-09-10T09:35:53.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:53.559+0000 7fb47dc00640 1 -- 172.21.15.5:0/203637749 shutdown_connections 2024-09-10T09:35:53.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:53.559+0000 7fb47dc00640 1 --2- 172.21.15.5:0/203637749 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb47810e420 0x7fb478110810 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:35:53.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:53.559+0000 7fb47dc00640 1 --2- 172.21.15.5:0/203637749 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb478074ee0 0x7fb47810dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:35:53.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:53.559+0000 7fb47dc00640 1 --2- 172.21.15.5:0/203637749 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb4780745c0 0x7fb4780749a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:35:53.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:53.559+0000 7fb47dc00640 1 -- 172.21.15.5:0/203637749 >> 172.21.15.5:0/203637749 conn(0x7fb4780fd8c0 msgr2=0x7fb4780ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:35:53.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:53.559+0000 7fb47dc00640 1 -- 172.21.15.5:0/203637749 shutdown_connections 2024-09-10T09:35:53.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:53.559+0000 7fb47dc00640 1 -- 172.21.15.5:0/203637749 wait complete. 2024-09-10T09:35:53.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:53.560+0000 7fb47dc00640 1 Processor -- start 2024-09-10T09:35:53.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:53.560+0000 7fb47dc00640 1 -- start start 2024-09-10T09:35:53.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:53.560+0000 7fb47dc00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb4780745c0 0x7fb47819a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:35:53.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:53.560+0000 7fb47dc00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb478074ee0 0x7fb47819abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:35:53.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:53.560+0000 7fb47dc00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb47810e420 0x7fb47819ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:35:53.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:53.560+0000 7fb47dc00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fb4781141c0 con 0x7fb4780745c0 2024-09-10T09:35:53.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:53.560+0000 7fb47dc00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fb478114040 con 0x7fb47810e420 2024-09-10T09:35:53.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:53.560+0000 7fb47dc00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fb478114340 con 0x7fb478074ee0 2024-09-10T09:35:53.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:53.560+0000 7fb476a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb4780745c0 0x7fb47819a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:35:53.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:53.560+0000 7fb476a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb4780745c0 0x7fb47819a670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43900/0 (socket says 172.21.15.5:43900) 2024-09-10T09:35:53.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:53.560+0000 7fb476a00640 1 -- 172.21.15.5:0/2504373313 learned_addr learned my addr 172.21.15.5:0/2504373313 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:35:53.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:53.561+0000 7fb477400640 1 --2- 172.21.15.5:0/2504373313 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb47810e420 0x7fb47819ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:35:53.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:53.561+0000 7fb476a00640 1 -- 172.21.15.5:0/2504373313 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb478074ee0 msgr2=0x7fb47819abb0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-10T09:35:53.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:53.561+0000 7fb476a00640 1 --2- 172.21.15.5:0/2504373313 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb478074ee0 0x7fb47819abb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:35:53.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:53.561+0000 7fb476a00640 1 -- 172.21.15.5:0/2504373313 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb47810e420 msgr2=0x7fb47819ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:35:53.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:53.561+0000 7fb476a00640 1 --2- 172.21.15.5:0/2504373313 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb47810e420 0x7fb47819ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:35:53.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:53.561+0000 7fb476a00640 1 -- 172.21.15.5:0/2504373313 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb47819f620 con 0x7fb4780745c0 2024-09-10T09:35:53.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:53.561+0000 7fb477400640 1 --2- 172.21.15.5:0/2504373313 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb47810e420 0x7fb47819ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:35:53.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:53.561+0000 7fb476a00640 1 --2- 172.21.15.5:0/2504373313 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb4780745c0 0x7fb47819a670 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fb46800e9f0 tx=0x7fb46800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:35:53.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:53.561+0000 7fb46f400640 1 -- 172.21.15.5:0/2504373313 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb46800cd80 con 0x7fb4780745c0 2024-09-10T09:35:53.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:53.561+0000 7fb46f400640 1 -- 172.21.15.5:0/2504373313 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb468010910 con 0x7fb4780745c0 2024-09-10T09:35:53.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:53.562+0000 7fb47dc00640 1 -- 172.21.15.5:0/2504373313 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb47819f910 con 0x7fb4780745c0 2024-09-10T09:35:53.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:53.562+0000 7fb46f400640 1 -- 172.21.15.5:0/2504373313 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb468018990 con 0x7fb4780745c0 2024-09-10T09:35:53.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:53.562+0000 7fb47dc00640 1 -- 172.21.15.5:0/2504373313 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4781a71f0 con 0x7fb4780745c0 2024-09-10T09:35:53.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:53.565+0000 7fb480875740 1 Processor -- start 2024-09-10T09:35:53.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:53.565+0000 7fb46f400640 1 -- 172.21.15.5:0/2504373313 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7fb468010ac0 con 0x7fb4780745c0 2024-09-10T09:35:53.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:53.565+0000 7fb480875740 1 -- start start 2024-09-10T09:35:53.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:53.565+0000 7fb480875740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5565e97ba330 0x5565e97ba710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:35:53.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:53.565+0000 7fb480875740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5565e97bac50 0x5565e97c4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:35:53.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:53.565+0000 7fb480875740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5565e97c5510 0x5565e97c7900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:35:53.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:53.565+0000 7fb480875740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5565e97200b0 con 0x5565e97ba330 2024-09-10T09:35:53.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:53.565+0000 7fb480875740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x5565e96c7ab0 con 0x5565e97c5510 2024-09-10T09:35:53.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:53.565+0000 7fb480875740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5565e971c070 con 0x5565e97bac50 2024-09-10T09:35:53.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:53.566+0000 7fb477400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5565e97ba330 0x5565e97ba710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:35:53.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:53.566+0000 7fb46fe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5565e97c5510 0x5565e97c7900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:35:53.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:53.566+0000 7fb476a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5565e97bac50 0x5565e97c4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:35:53.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:53.566+0000 7fb46fe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5565e97c5510 0x5565e97c7900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:58948/0 (socket says 172.21.15.5:58948) 2024-09-10T09:35:53.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:53.566+0000 7fb46fe00640 1 -- 172.21.15.5:0/514039600 learned_addr learned my addr 172.21.15.5:0/514039600 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:35:53.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:53.566+0000 7fb46fe00640 1 -- 172.21.15.5:0/514039600 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5565e97bac50 msgr2=0x5565e97c4fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:35:53.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:53.566+0000 7fb46fe00640 1 --2- 172.21.15.5:0/514039600 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5565e97bac50 0x5565e97c4fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:35:53.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:53.566+0000 7fb46fe00640 1 -- 172.21.15.5:0/514039600 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5565e97ba330 msgr2=0x5565e97ba710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:35:53.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:53.566+0000 7fb46fe00640 1 --2- 172.21.15.5:0/514039600 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5565e97ba330 0x5565e97ba710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:35:53.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:53.566+0000 7fb46fe00640 1 -- 172.21.15.5:0/514039600 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5565e96d44a0 con 0x5565e97c5510 2024-09-10T09:35:53.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:53.566+0000 7fb46fe00640 1 --2- 172.21.15.5:0/514039600 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5565e97c5510 0x5565e97c7900 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fb460009af0 tx=0x7fb46000e350 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e2914992d6a8f662 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:35:53.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:53.567+0000 7fb467e00640 1 -- 172.21.15.5:0/514039600 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb46000ed80 con 0x5565e97c5510 2024-09-10T09:35:53.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:53.567+0000 7fb467e00640 1 -- 172.21.15.5:0/514039600 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb460005500 con 0x5565e97c5510 2024-09-10T09:35:53.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:53.567+0000 7fb467e00640 1 -- 172.21.15.5:0/514039600 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb460010690 con 0x5565e97c5510 2024-09-10T09:35:53.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:53.567+0000 7fb46f400640 1 --2- 172.21.15.5:0/2504373313 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fb45c0a2b80 0x7fb45c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:35:53.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:53.567+0000 7fb46f400640 1 -- 172.21.15.5:0/2504373313 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8802+0+0 (secure 0 0 0) 0x7fb4680cf750 con 0x7fb4780745c0 2024-09-10T09:35:53.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:53.567+0000 7fb477400640 1 --2- 172.21.15.5:0/2504373313 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fb45c0a2b80 0x7fb45c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:35:53.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:53.567+0000 7fb480875740 1 -- 172.21.15.5:0/514039600 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5565e97c5510 msgr2=0x5565e97c7900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:35:53.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:53.567+0000 7fb480875740 1 --2- 172.21.15.5:0/514039600 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5565e97c5510 0x5565e97c7900 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fb460009af0 tx=0x7fb46000e350 comp rx=0 tx=0).stop 2024-09-10T09:35:53.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:53.568+0000 7fb480875740 1 -- 172.21.15.5:0/514039600 shutdown_connections 2024-09-10T09:35:53.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:53.568+0000 7fb480875740 1 --2- 172.21.15.5:0/514039600 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5565e97c5510 0x5565e97c7900 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:35:53.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:53.568+0000 7fb480875740 1 --2- 172.21.15.5:0/514039600 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5565e97bac50 0x5565e97c4fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:35:53.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:53.568+0000 7fb480875740 1 --2- 172.21.15.5:0/514039600 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5565e97ba330 0x5565e97ba710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:35:53.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:53.568+0000 7fb480875740 1 -- 172.21.15.5:0/514039600 >> 172.21.15.5:0/514039600 conn(0x5565e97b68c0 msgr2=0x5565e97b6ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:35:53.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:53.568+0000 7fb480875740 1 -- 172.21.15.5:0/514039600 shutdown_connections 2024-09-10T09:35:53.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:53.568+0000 7fb477400640 1 --2- 172.21.15.5:0/2504373313 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fb45c0a2b80 0x7fb45c0a5040 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fb470004af0 tx=0x7fb470004a40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:35:53.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:53.568+0000 7fb480875740 1 -- 172.21.15.5:0/514039600 wait complete. 2024-09-10T09:35:53.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:53.568+0000 7fb480875740 1 -- start start 2024-09-10T09:35:53.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:53.569+0000 7fb480875740 1 Processor -- start 2024-09-10T09:35:53.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:53.569+0000 7fb480875740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5565e97ba330 0x5565e97c2540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:35:53.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:53.569+0000 7fb480875740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5565e97bac50 0x5565e97c2a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:35:53.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:53.569+0000 7fb480875740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5565e97c5510 0x5565e97c2fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:35:53.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:53.569+0000 7fb480875740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5565e97c8060 con 0x5565e97c5510 2024-09-10T09:35:53.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:53.569+0000 7fb480875740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x5565e96a8920 con 0x5565e97bac50 2024-09-10T09:35:53.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:53.569+0000 7fb480875740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5565e97c81e0 con 0x5565e97ba330 2024-09-10T09:35:53.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:53.569+0000 7fb480875740 10 client.0 _ll_register_callbacks cb 0x5565e96cc8a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:35:53.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:53.569+0000 7fb476a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5565e97ba330 0x5565e97c2540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:35:53.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:53.569+0000 7fb476a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5565e97ba330 0x5565e97c2540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:40102/0 (socket says 172.21.15.5:40102) 2024-09-10T09:35:53.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:53.569+0000 7fb476a00640 1 -- 172.21.15.5:0/1768239506 learned_addr learned my addr 172.21.15.5:0/1768239506 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:35:53.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:53.570+0000 7fb46fe00640 1 --2- 172.21.15.5:0/1768239506 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5565e97bac50 0x5565e97c2a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:35:53.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:53.570+0000 7fb477400640 1 --2- 172.21.15.5:0/1768239506 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5565e97c5510 0x5565e97c2fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:35:53.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:53.570+0000 7fb476a00640 1 -- 172.21.15.5:0/1768239506 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5565e97bac50 msgr2=0x5565e97c2a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:35:53.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:53.570+0000 7fb476a00640 1 --2- 172.21.15.5:0/1768239506 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5565e97bac50 0x5565e97c2a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:35:53.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:53.570+0000 7fb476a00640 1 -- 172.21.15.5:0/1768239506 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5565e97c5510 msgr2=0x5565e97c2fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:35:53.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:53.570+0000 7fb476a00640 1 --2- 172.21.15.5:0/1768239506 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5565e97c5510 0x5565e97c2fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:35:53.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:53.570+0000 7fb476a00640 1 -- 172.21.15.5:0/1768239506 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5565e94912b0 con 0x5565e97ba330 2024-09-10T09:35:53.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:53.570+0000 7fb476a00640 1 --2- 172.21.15.5:0/1768239506 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5565e97ba330 0x5565e97c2540 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fb46809bca0 tx=0x7fb46809b850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=68abbee924581df6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:35:53.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:53.570+0000 7fb464c00640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-09-10T09:35:53.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:53.570+0000 7fb464c00640 1 -- 172.21.15.5:0/1768239506 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4680cfd40 con 0x5565e97ba330 2024-09-10T09:35:53.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:53.570+0000 7fb480875740 1 -- 172.21.15.5:0/1768239506 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb460005500 con 0x5565e97ba330 2024-09-10T09:35:53.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:53.570+0000 7fb464c00640 1 -- 172.21.15.5:0/1768239506 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8802+0+0 (secure 0 0 0) 0x7fb46809b160 con 0x5565e97ba330 2024-09-10T09:35:53.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:53.571+0000 7fb464c00640 1 -- 172.21.15.5:0/1768239506 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7fb46809bd10 con 0x5565e97ba330 2024-09-10T09:35:53.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:53.571+0000 7fb480875740 10 client.24652 fetch_fsmap learned FSMap version 17 2024-09-10T09:35:53.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:53.571+0000 7fb480875740 1 -- 172.21.15.5:0/1768239506 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=4+}) -- 0x5565e97d3e60 con 0x5565e97ba330 2024-09-10T09:35:53.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:53.571+0000 7fb464c00640 1 -- 172.21.15.5:0/1768239506 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb460005500 con 0x5565e97ba330 2024-09-10T09:35:53.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:53.571+0000 7fb464c00640 1 -- 172.21.15.5:0/1768239506 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 17) ==== 8350+0+0 (secure 0 0 0) 0x7fb468018c50 con 0x5565e97ba330 2024-09-10T09:35:53.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:53.571+0000 7fb480875740 10 client.24652 fetch_fsmap finished waiting for FSMap version 17 2024-09-10T09:35:53.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:53.571+0000 7fb480875740 10 client.24652 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-10T09:35:53.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:53.572+0000 7fb480875740 20 client.24652 populate_metadata read hostname 'smithi005' 2024-09-10T09:35:53.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:53.572+0000 7fb480875740 1 --2- 172.21.15.5:0/1768239506 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x5565e97d63a0 0x5565e97d7810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:35:53.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:53.572+0000 7fb480875740 4 client.24652 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:35:53.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:53.572+0000 7fb480875740 1 -- 172.21.15.5:0/1768239506 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5565e972d620 con 0x5565e97d63a0 2024-09-10T09:35:53.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:53.572+0000 7fb46fe00640 1 --2- 172.21.15.5:0/1768239506 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x5565e97d63a0 0x5565e97d7810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:35:53.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:53.572+0000 7fb46fe00640 1 --2- 172.21.15.5:0/1768239506 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x5565e97d63a0 0x5565e97d7810 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4558d8e802aeb8cb server_cookie=96f56612362d5f3d in_seq=0 out_seq=0 2024-09-10T09:35:53.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:53.572+0000 7fb464c00640 10 client.24652 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T09:35:53.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:53.574+0000 7fb464c00640 1 -- 172.21.15.5:0/1768239506 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5565e972d620 con 0x5565e97d63a0 2024-09-10T09:35:53.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:53.574+0000 7fb464c00640 10 client.24652 handle_command_reply: tid=0 2024-09-10T09:35:53.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:53.574+0000 7fb480875740 1 client.24652 shutdown 2024-09-10T09:35:53.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:53.574+0000 7fb480875740 1 -- 172.21.15.5:0/1768239506 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5565e97ba330 msgr2=0x5565e97c2540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:35:53.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:53.574+0000 7fb480875740 1 --2- 172.21.15.5:0/1768239506 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5565e97ba330 0x5565e97c2540 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fb46809bca0 tx=0x7fb46809b850 comp rx=0 tx=0).stop 2024-09-10T09:35:53.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:53.574+0000 7fb480875740 1 -- 172.21.15.5:0/1768239506 shutdown_connections 2024-09-10T09:35:53.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:53.574+0000 7fb480875740 1 --2- 172.21.15.5:0/1768239506 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x5565e97d63a0 0x5565e97d7810 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:35:53.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:53.574+0000 7fb480875740 1 --2- 172.21.15.5:0/1768239506 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5565e97c5510 0x5565e97c2fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:35:53.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:53.574+0000 7fb480875740 1 --2- 172.21.15.5:0/1768239506 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5565e97bac50 0x5565e97c2a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:35:53.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:53.574+0000 7fb480875740 1 --2- 172.21.15.5:0/1768239506 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5565e97ba330 0x5565e97c2540 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:35:53.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:53.574+0000 7fb464c00640 0 client.24652 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T09:35:53.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:53.574+0000 7fb480875740 1 -- 172.21.15.5:0/1768239506 >> 172.21.15.5:0/1768239506 conn(0x5565e97b68c0 msgr2=0x5565e97bc5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:35:53.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:53.574+0000 7fb480875740 1 -- 172.21.15.5:0/1768239506 shutdown_connections 2024-09-10T09:35:53.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:53.574+0000 7fb480875740 1 -- 172.21.15.5:0/1768239506 wait complete. 2024-09-10T09:35:53.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:53.574+0000 7fb480875740 20 client.24652 trim_cache size 0 max 16384 2024-09-10T09:35:53.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:53.593+0000 7fb480875740 1 Processor -- start 2024-09-10T09:35:53.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:53.593+0000 7fb480875740 1 -- start start 2024-09-10T09:35:53.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:53.594+0000 7fb480875740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5565e97ba330 0x5565e97c2ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:35:53.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:53.594+0000 7fb480875740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5565e97bac50 0x5565e97c3010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:35:53.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:53.594+0000 7fb480875740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5565e97c5510 0x5565e97bf800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:35:53.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:53.594+0000 7fb480875740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5565e97d5b50 con 0x5565e97c5510 2024-09-10T09:35:53.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:53.594+0000 7fb480875740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x5565e97d6220 con 0x5565e97ba330 2024-09-10T09:35:53.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:53.594+0000 7fb480875740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5565e97d68f0 con 0x5565e97bac50 2024-09-10T09:35:53.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:53.594+0000 7fb477400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5565e97c5510 0x5565e97bf800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:35:53.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:53.594+0000 7fb46fe00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5565e97bac50 0x5565e97c3010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:35:53.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:53.594+0000 7fb477400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5565e97c5510 0x5565e97bf800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43928/0 (socket says 172.21.15.5:43928) 2024-09-10T09:35:53.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:53.594+0000 7fb46fe00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5565e97bac50 0x5565e97c3010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:40114/0 (socket says 172.21.15.5:40114) 2024-09-10T09:35:53.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:53.594+0000 7fb477400640 1 -- 172.21.15.5:0/724685215 learned_addr learned my addr 172.21.15.5:0/724685215 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:35:53.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:53.594+0000 7fb476a00640 1 --2- 172.21.15.5:0/724685215 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5565e97ba330 0x5565e97c2ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:35:53.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:53.594+0000 7fb46fe00640 1 -- 172.21.15.5:0/724685215 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5565e97ba330 msgr2=0x5565e97c2ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:35:53.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:53.594+0000 7fb46fe00640 1 --2- 172.21.15.5:0/724685215 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5565e97ba330 0x5565e97c2ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:35:53.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:53.594+0000 7fb46fe00640 1 -- 172.21.15.5:0/724685215 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5565e97c5510 msgr2=0x5565e97bf800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:35:53.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:53.594+0000 7fb46fe00640 1 --2- 172.21.15.5:0/724685215 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5565e97c5510 0x5565e97bf800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:35:53.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:53.594+0000 7fb46fe00640 1 -- 172.21.15.5:0/724685215 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5565e97c4eb0 con 0x5565e97bac50 2024-09-10T09:35:53.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:53.595+0000 7fb46fe00640 1 --2- 172.21.15.5:0/724685215 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5565e97bac50 0x5565e97c3010 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fb46000a010 tx=0x7fb460004d20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=12cbe0f4430c791c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:35:53.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:53.595+0000 7fb467400640 1 -- 172.21.15.5:0/724685215 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb46001d040 con 0x5565e97bac50 2024-09-10T09:35:53.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:53.595+0000 7fb467400640 1 -- 172.21.15.5:0/724685215 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb460004b60 con 0x5565e97bac50 2024-09-10T09:35:53.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:53.595+0000 7fb467400640 1 -- 172.21.15.5:0/724685215 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb460005570 con 0x5565e97bac50 2024-09-10T09:35:53.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:53.596+0000 7fb480875740 1 -- 172.21.15.5:0/724685215 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5565e97bac50 msgr2=0x5565e97c3010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:35:53.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:53.596+0000 7fb480875740 1 --2- 172.21.15.5:0/724685215 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5565e97bac50 0x5565e97c3010 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fb46000a010 tx=0x7fb460004d20 comp rx=0 tx=0).stop 2024-09-10T09:35:53.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:53.596+0000 7fb480875740 1 -- 172.21.15.5:0/724685215 shutdown_connections 2024-09-10T09:35:53.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:53.596+0000 7fb480875740 1 --2- 172.21.15.5:0/724685215 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5565e97c5510 0x5565e97bf800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:35:53.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:53.596+0000 7fb480875740 1 --2- 172.21.15.5:0/724685215 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5565e97bac50 0x5565e97c3010 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:35:53.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:53.596+0000 7fb480875740 1 --2- 172.21.15.5:0/724685215 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5565e97ba330 0x5565e97c2ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:35:53.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:53.596+0000 7fb480875740 1 -- 172.21.15.5:0/724685215 >> 172.21.15.5:0/724685215 conn(0x5565e97b68c0 msgr2=0x5565e97b6ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:35:53.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:53.596+0000 7fb480875740 1 -- 172.21.15.5:0/724685215 shutdown_connections 2024-09-10T09:35:53.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:53.596+0000 7fb480875740 1 -- 172.21.15.5:0/724685215 wait complete. 2024-09-10T09:35:53.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:53.596+0000 7fb480875740 1 -- start start 2024-09-10T09:35:53.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:53.596+0000 7fb480875740 1 Processor -- start 2024-09-10T09:35:53.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:53.597+0000 7fb480875740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5565e97ba330 0x5565e97dff80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:35:53.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:53.597+0000 7fb480875740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5565e97bac50 0x5565e97daa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:35:53.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:53.597+0000 7fb480875740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5565e97c5510 0x5565e97daf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:35:53.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:53.597+0000 7fb480875740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5565e97d7690 con 0x5565e97bac50 2024-09-10T09:35:53.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:53.597+0000 7fb480875740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x5565e97d6fc0 con 0x5565e97c5510 2024-09-10T09:35:53.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:53.597+0000 7fb480875740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5565e97d7d60 con 0x5565e97ba330 2024-09-10T09:35:53.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:53.597+0000 7fb480875740 10 client.0 _ll_register_callbacks cb 0x7fb468009700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:35:53.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:53.597+0000 7fb46fe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5565e97bac50 0x5565e97daa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:35:53.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:53.597+0000 7fb476a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5565e97ba330 0x5565e97dff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:35:53.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:53.597+0000 7fb46fe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5565e97bac50 0x5565e97daa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43932/0 (socket says 172.21.15.5:43932) 2024-09-10T09:35:53.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:53.597+0000 7fb46fe00640 1 -- 172.21.15.5:0/2423551884 learned_addr learned my addr 172.21.15.5:0/2423551884 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:35:53.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:53.597+0000 7fb477400640 1 --2- 172.21.15.5:0/2423551884 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5565e97c5510 0x5565e97daf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:35:53.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:53.597+0000 7fb46fe00640 1 -- 172.21.15.5:0/2423551884 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5565e97ba330 msgr2=0x5565e97dff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:35:53.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:53.597+0000 7fb46fe00640 1 --2- 172.21.15.5:0/2423551884 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5565e97ba330 0x5565e97dff80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:35:53.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:53.598+0000 7fb46fe00640 1 -- 172.21.15.5:0/2423551884 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5565e97c5510 msgr2=0x5565e97daf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:35:53.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:53.598+0000 7fb46fe00640 1 --2- 172.21.15.5:0/2423551884 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5565e97c5510 0x5565e97daf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:35:53.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:53.598+0000 7fb46fe00640 1 -- 172.21.15.5:0/2423551884 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5565e97db4a0 con 0x5565e97bac50 2024-09-10T09:35:53.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:53.598+0000 7fb46fe00640 1 --2- 172.21.15.5:0/2423551884 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5565e97bac50 0x5565e97daa20 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fb460004e00 tx=0x7fb460004e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=671d6d6fdd8362be server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:35:53.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:53.598+0000 7fb464c00640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-10T09:35:53.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:53.598+0000 7fb464c00640 1 -- 172.21.15.5:0/2423551884 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb460004fd0 con 0x5565e97bac50 2024-09-10T09:35:53.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:53.598+0000 7fb480875740 1 -- 172.21.15.5:0/2423551884 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5565e9727b30 con 0x5565e97bac50 2024-09-10T09:35:53.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:53.598+0000 7fb464c00640 1 -- 172.21.15.5:0/2423551884 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8802+0+0 (secure 0 0 0) 0x7fb46000eb80 con 0x5565e97bac50 2024-09-10T09:35:53.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:53.599+0000 7fb464c00640 1 -- 172.21.15.5:0/2423551884 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7fb46000ed80 con 0x5565e97bac50 2024-09-10T09:35:53.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:53.599+0000 7fb480875740 10 client.14628 fetch_fsmap learned FSMap version 17 2024-09-10T09:35:53.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:53.599+0000 7fb480875740 1 -- 172.21.15.5:0/2423551884 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=4+}) -- 0x5565e98417c0 con 0x5565e97bac50 2024-09-10T09:35:53.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:53.599+0000 7fb464c00640 1 -- 172.21.15.5:0/2423551884 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x5565e9727b30 con 0x5565e97bac50 2024-09-10T09:35:53.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:53.599+0000 7fb464c00640 1 -- 172.21.15.5:0/2423551884 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 17) ==== 8350+0+0 (secure 0 0 0) 0x7fb460010700 con 0x5565e97bac50 2024-09-10T09:35:53.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:53.599+0000 7fb480875740 10 client.14628 fetch_fsmap finished waiting for FSMap version 17 2024-09-10T09:35:53.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:53.599+0000 7fb480875740 10 client.14628 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-10T09:35:53.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:53.599+0000 7fb480875740 20 client.14628 populate_metadata read hostname 'smithi005' 2024-09-10T09:35:53.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:53.599+0000 7fb480875740 1 --2- 172.21.15.5:0/2423551884 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x5565e9845b40 0x5565e9844850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:35:53.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:53.599+0000 7fb480875740 4 client.14628 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T09:35:53.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:53.599+0000 7fb480875740 1 -- 172.21.15.5:0/2423551884 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "status"}) -- 0x5565e96a3f80 con 0x5565e9845b40 2024-09-10T09:35:53.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:53.600+0000 7fb476a00640 1 --2- 172.21.15.5:0/2423551884 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x5565e9845b40 0x5565e9844850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:35:53.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:53.600+0000 7fb476a00640 1 --2- 172.21.15.5:0/2423551884 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x5565e9845b40 0x5565e9844850 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=433669354591d3dc server_cookie=264fbe8de190ed0 in_seq=0 out_seq=0 2024-09-10T09:35:53.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:53.600+0000 7fb464c00640 10 client.14628 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T09:35:53.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:53.600+0000 7fb464c00640 1 -- 172.21.15.5:0/2423551884 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+326 (crc 0 0 0) 0x5565e96a3f80 con 0x5565e9845b40 2024-09-10T09:35:53.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:53.600+0000 7fb464c00640 10 client.14628 handle_command_reply: tid=0 2024-09-10T09:35:53.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:53.600+0000 7fb480875740 1 client.14628 shutdown 2024-09-10T09:35:53.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:53.600+0000 7fb480875740 1 -- 172.21.15.5:0/2423551884 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5565e97bac50 msgr2=0x5565e97daa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:35:53.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:53.600+0000 7fb480875740 1 --2- 172.21.15.5:0/2423551884 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5565e97bac50 0x5565e97daa20 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fb460004e00 tx=0x7fb460004e30 comp rx=0 tx=0).stop 2024-09-10T09:35:53.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:53.601+0000 7fb480875740 1 -- 172.21.15.5:0/2423551884 shutdown_connections 2024-09-10T09:35:53.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:53.601+0000 7fb480875740 1 --2- 172.21.15.5:0/2423551884 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x5565e9845b40 0x5565e9844850 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:35:53.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:53.601+0000 7fb480875740 1 --2- 172.21.15.5:0/2423551884 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5565e97c5510 0x5565e97daf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:35:53.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:53.601+0000 7fb480875740 1 --2- 172.21.15.5:0/2423551884 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5565e97bac50 0x5565e97daa20 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:35:53.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:53.601+0000 7fb480875740 1 --2- 172.21.15.5:0/2423551884 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5565e97ba330 0x5565e97dff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:35:53.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:53.601+0000 7fb480875740 1 -- 172.21.15.5:0/2423551884 >> 172.21.15.5:0/2423551884 conn(0x5565e97b68c0 msgr2=0x5565e97d0430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:35:53.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:53.601+0000 7fb464c00640 0 client.14628 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T09:35:53.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:53.601+0000 7fb480875740 1 -- 172.21.15.5:0/2423551884 shutdown_connections 2024-09-10T09:35:53.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:53.601+0000 7fb480875740 1 -- 172.21.15.5:0/2423551884 wait complete. 2024-09-10T09:35:53.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:53.601+0000 7fb480875740 20 client.14628 trim_cache size 0 max 16384 2024-09-10T09:35:53.607 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-10T09:35:53.608 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_fsid": "bade4b50-6f56-11ef-bcea-c7b262605968", 2024-09-10T09:35:53.608 INFO:teuthology.orchestra.run.smithi005.stdout: "whoami": 0, 2024-09-10T09:35:53.608 INFO:teuthology.orchestra.run.smithi005.stdout: "id": 24542, 2024-09-10T09:35:53.608 INFO:teuthology.orchestra.run.smithi005.stdout: "want_state": "up:active", 2024-09-10T09:35:53.608 INFO:teuthology.orchestra.run.smithi005.stdout: "state": "up:active", 2024-09-10T09:35:53.608 INFO:teuthology.orchestra.run.smithi005.stdout: "fs_name": "cephfs", 2024-09-10T09:35:53.608 INFO:teuthology.orchestra.run.smithi005.stdout: "rank_uptime": 23.036806012, 2024-09-10T09:35:53.608 INFO:teuthology.orchestra.run.smithi005.stdout: "mdsmap_epoch": 17, 2024-09-10T09:35:53.608 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch": 92, 2024-09-10T09:35:53.608 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch_barrier": 92, 2024-09-10T09:35:53.608 INFO:teuthology.orchestra.run.smithi005.stdout: "uptime": 36.652753797999999 2024-09-10T09:35:53.608 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-10T09:35:53.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:53.602+0000 7fb47dc00640 1 -- 172.21.15.5:0/2504373313 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fb45c0a2b80 msgr2=0x7fb45c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:35:53.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:53.602+0000 7fb47dc00640 1 --2- 172.21.15.5:0/2504373313 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fb45c0a2b80 0x7fb45c0a5040 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fb470004af0 tx=0x7fb470004a40 comp rx=0 tx=0).stop 2024-09-10T09:35:53.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:53.602+0000 7fb47dc00640 1 -- 172.21.15.5:0/2504373313 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb4780745c0 msgr2=0x7fb47819a670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:35:53.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:53.602+0000 7fb47dc00640 1 --2- 172.21.15.5:0/2504373313 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb4780745c0 0x7fb47819a670 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fb46800e9f0 tx=0x7fb46800eeb0 comp rx=0 tx=0).stop 2024-09-10T09:35:53.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:53.602+0000 7fb47dc00640 1 -- 172.21.15.5:0/2504373313 shutdown_connections 2024-09-10T09:35:53.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:53.602+0000 7fb47dc00640 1 --2- 172.21.15.5:0/2504373313 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fb45c0a2b80 0x7fb45c0a5040 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:35:53.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:53.602+0000 7fb47dc00640 1 --2- 172.21.15.5:0/2504373313 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb47810e420 0x7fb47819ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:35:53.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:53.602+0000 7fb47dc00640 1 --2- 172.21.15.5:0/2504373313 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb478074ee0 0x7fb47819abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:35:53.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:53.602+0000 7fb47dc00640 1 --2- 172.21.15.5:0/2504373313 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb4780745c0 0x7fb47819a670 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:35:53.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:53.602+0000 7fb47dc00640 1 -- 172.21.15.5:0/2504373313 >> 172.21.15.5:0/2504373313 conn(0x7fb4780fd8c0 msgr2=0x7fb4780fe560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:35:53.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:53.602+0000 7fb47dc00640 1 -- 172.21.15.5:0/2504373313 shutdown_connections 2024-09-10T09:35:53.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:53.602+0000 7fb47dc00640 1 -- 172.21.15.5:0/2504373313 wait complete. 2024-09-10T09:35:54.503 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-10T09:35:54.503 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-09-10T09:35:54.504 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph fs ls 2024-09-10T09:35:54.884 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:35:55.023 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:35:54 smithi005 ceph-mon[30350]: pgmap v316: 129 pgs: 129 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T09:35:55.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:35:54 smithi159 ceph-mon[35513]: pgmap v316: 129 pgs: 129 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T09:35:55.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:35:54 smithi102 ceph-mon[35442]: pgmap v316: 129 pgs: 129 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T09:35:55.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:55.463+0000 7f45af200640 1 -- 172.21.15.5:0/3586861382 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f45a8102790 msgr2=0x7f45a81067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:35:55.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:55.463+0000 7f45af200640 1 --2- 172.21.15.5:0/3586861382 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f45a8102790 0x7f45a81067e0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f4590009860 tx=0x7f459002a340 comp rx=0 tx=0).stop 2024-09-10T09:35:55.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:55.464+0000 7f45af200640 1 -- 172.21.15.5:0/3586861382 shutdown_connections 2024-09-10T09:35:55.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:55.464+0000 7f45af200640 1 --2- 172.21.15.5:0/3586861382 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f45a8106f10 0x7f45a810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:35:55.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:55.464+0000 7f45af200640 1 --2- 172.21.15.5:0/3586861382 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f45a8102790 0x7f45a81067e0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:35:55.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:55.464+0000 7f45af200640 1 --2- 172.21.15.5:0/3586861382 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f45a8101de0 0x7f45a81021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:35:55.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:55.464+0000 7f45af200640 1 -- 172.21.15.5:0/3586861382 >> 172.21.15.5:0/3586861382 conn(0x7f45a80fd8c0 msgr2=0x7f45a80ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:35:55.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:55.464+0000 7f45af200640 1 -- 172.21.15.5:0/3586861382 shutdown_connections 2024-09-10T09:35:55.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:55.464+0000 7f45af200640 1 -- 172.21.15.5:0/3586861382 wait complete. 2024-09-10T09:35:55.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:55.465+0000 7f45af200640 1 Processor -- start 2024-09-10T09:35:55.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:55.465+0000 7f45af200640 1 -- start start 2024-09-10T09:35:55.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:55.465+0000 7f45af200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f45a8101de0 0x7f45a80744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:35:55.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:55.465+0000 7f45af200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f45a8102790 0x7f45a8074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:35:55.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:55.465+0000 7f45af200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f45a8106f10 0x7f45a8074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:35:55.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:55.465+0000 7f45af200640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f45a8111f60 con 0x7f45a8102790 2024-09-10T09:35:55.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:55.465+0000 7f45af200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f45a8111de0 con 0x7f45a8101de0 2024-09-10T09:35:55.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:55.465+0000 7f45af200640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f45a81120e0 con 0x7f45a8106f10 2024-09-10T09:35:55.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:55.466+0000 7f45a7e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f45a8101de0 0x7f45a80744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:35:55.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:55.466+0000 7f459fe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f45a8102790 0x7f45a8074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:35:55.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:55.466+0000 7f45a7e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f45a8101de0 0x7f45a80744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:58976/0 (socket says 172.21.15.5:58976) 2024-09-10T09:35:55.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:55.466+0000 7f45a7e00640 1 -- 172.21.15.5:0/1024407333 learned_addr learned my addr 172.21.15.5:0/1024407333 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:35:55.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:55.466+0000 7f45ad200640 1 --2- 172.21.15.5:0/1024407333 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f45a8106f10 0x7f45a8074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:35:55.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:55.466+0000 7f45a7e00640 1 -- 172.21.15.5:0/1024407333 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f45a8106f10 msgr2=0x7f45a8074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:35:55.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:55.466+0000 7f45a7e00640 1 --2- 172.21.15.5:0/1024407333 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f45a8106f10 0x7f45a8074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:35:55.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:55.466+0000 7f45a7e00640 1 -- 172.21.15.5:0/1024407333 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f45a8102790 msgr2=0x7f45a8074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:35:55.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:55.466+0000 7f45a7e00640 1 --2- 172.21.15.5:0/1024407333 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f45a8102790 0x7f45a8074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:35:55.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:55.466+0000 7f45a7e00640 1 -- 172.21.15.5:0/1024407333 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45a81a68f0 con 0x7f45a8101de0 2024-09-10T09:35:55.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:55.466+0000 7f45ad200640 1 --2- 172.21.15.5:0/1024407333 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f45a8106f10 0x7f45a8074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:35:55.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:55.466+0000 7f45a7e00640 1 --2- 172.21.15.5:0/1024407333 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f45a8101de0 0x7f45a80744f0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f459800e9c0 tx=0x7f459800ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:35:55.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:55.467+0000 7f45a5600640 1 -- 172.21.15.5:0/1024407333 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f459800cde0 con 0x7f45a8101de0 2024-09-10T09:35:55.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:55.467+0000 7f45af200640 1 -- 172.21.15.5:0/1024407333 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45a81a6be0 con 0x7f45a8101de0 2024-09-10T09:35:55.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:55.467+0000 7f45a5600640 1 -- 172.21.15.5:0/1024407333 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f45980035f0 con 0x7f45a8101de0 2024-09-10T09:35:55.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:55.467+0000 7f45a5600640 1 -- 172.21.15.5:0/1024407333 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4598010650 con 0x7f45a8101de0 2024-09-10T09:35:55.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:55.467+0000 7f45af200640 1 -- 172.21.15.5:0/1024407333 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45a81a7140 con 0x7f45a8101de0 2024-09-10T09:35:55.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:55.468+0000 7f45af200640 1 -- 172.21.15.5:0/1024407333 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f45a81021c0 con 0x7f45a8101de0 2024-09-10T09:35:55.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:55.470+0000 7f45a5600640 1 -- 172.21.15.5:0/1024407333 <== mon.2 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7f45980037a0 con 0x7f45a8101de0 2024-09-10T09:35:55.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:55.471+0000 7f45a5600640 1 --2- 172.21.15.5:0/1024407333 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f458c0a2b80 0x7f458c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:35:55.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:55.472+0000 7f45a5600640 1 -- 172.21.15.5:0/1024407333 <== mon.2 v2:172.21.15.102:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8802+0+0 (secure 0 0 0) 0x7f45980cf730 con 0x7f45a8101de0 2024-09-10T09:35:55.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:55.472+0000 7f459fe00640 1 --2- 172.21.15.5:0/1024407333 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f458c0a2b80 0x7f458c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:35:55.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:55.472+0000 7f459fe00640 1 --2- 172.21.15.5:0/1024407333 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f458c0a2b80 0x7f458c0a5040 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f459002a930 tx=0x7f45900056d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:35:55.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:55.474+0000 7f45a5600640 1 -- 172.21.15.5:0/1024407333 <== mon.2 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f459809c480 con 0x7f45a8101de0 2024-09-10T09:35:55.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:55.695+0000 7f45af200640 1 -- 172.21.15.5:0/1024407333 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f45a810a150 con 0x7f45a8101de0 2024-09-10T09:35:55.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:55.696+0000 7f45a5600640 1 -- 172.21.15.5:0/1024407333 <== mon.2 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) ==== 53+0+88 (secure 0 0 0) 0x7f459809bd80 con 0x7f45a8101de0 2024-09-10T09:35:55.698 INFO:teuthology.orchestra.run.smithi005.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-09-10T09:35:55.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:55.698+0000 7f45af200640 1 -- 172.21.15.5:0/1024407333 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f458c0a2b80 msgr2=0x7f458c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:35:55.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:55.698+0000 7f45af200640 1 --2- 172.21.15.5:0/1024407333 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f458c0a2b80 0x7f458c0a5040 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f459002a930 tx=0x7f45900056d0 comp rx=0 tx=0).stop 2024-09-10T09:35:55.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:55.698+0000 7f45af200640 1 -- 172.21.15.5:0/1024407333 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f45a8101de0 msgr2=0x7f45a80744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:35:55.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:55.698+0000 7f45af200640 1 --2- 172.21.15.5:0/1024407333 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f45a8101de0 0x7f45a80744f0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f459800e9c0 tx=0x7f459800ee80 comp rx=0 tx=0).stop 2024-09-10T09:35:55.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:55.698+0000 7f45af200640 1 -- 172.21.15.5:0/1024407333 shutdown_connections 2024-09-10T09:35:55.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:55.698+0000 7f45af200640 1 --2- 172.21.15.5:0/1024407333 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f458c0a2b80 0x7f458c0a5040 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:35:55.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:55.698+0000 7f45af200640 1 --2- 172.21.15.5:0/1024407333 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f45a8106f10 0x7f45a8074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:35:55.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:55.698+0000 7f45af200640 1 --2- 172.21.15.5:0/1024407333 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f45a8102790 0x7f45a8074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:35:55.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:55.698+0000 7f45af200640 1 --2- 172.21.15.5:0/1024407333 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f45a8101de0 0x7f45a80744f0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:35:55.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:55.698+0000 7f45af200640 1 -- 172.21.15.5:0/1024407333 >> 172.21.15.5:0/1024407333 conn(0x7f45a80fd8c0 msgr2=0x7f45a80fec40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:35:55.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:55.699+0000 7f45af200640 1 -- 172.21.15.5:0/1024407333 shutdown_connections 2024-09-10T09:35:55.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:55.699+0000 7f45af200640 1 -- 172.21.15.5:0/1024407333 wait complete. 2024-09-10T09:35:56.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:35:55 smithi005 ceph-mon[30350]: from='client.? 172.21.15.5:0/1024407333' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-10T09:35:56.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:35:55 smithi159 ceph-mon[35513]: from='client.? 172.21.15.5:0/1024407333' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-10T09:35:56.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:35:55 smithi102 ceph-mon[35442]: from='client.? 172.21.15.5:0/1024407333' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-10T09:35:56.593 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-10T09:35:56.594 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-10T09:35:56.594 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi005.front.sepia.ceph.com 2024-09-10T09:35:56.594 INFO:tasks.cephfs.mount:self.client.name = client.admin 2024-09-10T09:35:56.594 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-09-10T09:35:56.594 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-10T09:35:56.594 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-10T09:35:56.594 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-10T09:35:56.594 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2024-09-10T09:35:56.594 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T09:35:56.594 DEBUG:teuthology.orchestra.run.smithi005:> ip addr 2024-09-10T09:35:56.630 INFO:teuthology.orchestra.run.smithi005.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-10T09:35:56.631 INFO:teuthology.orchestra.run.smithi005.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-10T09:35:56.631 INFO:teuthology.orchestra.run.smithi005.stdout: inet 127.0.0.1/8 scope host lo 2024-09-10T09:35:56.631 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-09-10T09:35:56.631 INFO:teuthology.orchestra.run.smithi005.stdout: inet6 ::1/128 scope host 2024-09-10T09:35:56.631 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-09-10T09:35:56.631 INFO:teuthology.orchestra.run.smithi005.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T09:35:56.631 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 0c:c4:7a:6b:fd:de brd ff:ff:ff:ff:ff:ff 2024-09-10T09:35:56.631 INFO:teuthology.orchestra.run.smithi005.stdout: altname enp7s0f0 2024-09-10T09:35:56.631 INFO:teuthology.orchestra.run.smithi005.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T09:35:56.631 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 0c:c4:7a:6b:fd:df brd ff:ff:ff:ff:ff:ff 2024-09-10T09:35:56.631 INFO:teuthology.orchestra.run.smithi005.stdout: altname enp7s0f1 2024-09-10T09:35:56.631 INFO:teuthology.orchestra.run.smithi005.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-10T09:35:56.632 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 0c:c4:7a:bd:14:42 brd ff:ff:ff:ff:ff:ff 2024-09-10T09:35:56.632 INFO:teuthology.orchestra.run.smithi005.stdout: altname enp1s0f0 2024-09-10T09:35:56.632 INFO:teuthology.orchestra.run.smithi005.stdout: inet 172.21.15.5/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-09-10T09:35:56.632 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft 42182sec preferred_lft 42182sec 2024-09-10T09:35:56.632 INFO:teuthology.orchestra.run.smithi005.stdout: inet6 fe80::ec4:7aff:febd:1442/64 scope link 2024-09-10T09:35:56.632 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-09-10T09:35:56.632 INFO:teuthology.orchestra.run.smithi005.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T09:35:56.632 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 0c:c4:7a:bd:14:43 brd ff:ff:ff:ff:ff:ff 2024-09-10T09:35:56.632 INFO:teuthology.orchestra.run.smithi005.stdout: altname enp1s0f1 2024-09-10T09:35:56.633 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-09-10T09:35:56.634 DEBUG:teuthology.orchestra.run.smithi005:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T09:35:56.634 DEBUG:teuthology.orchestra.run.smithi005:> set -e 2024-09-10T09:35:56.634 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip link add name ceph-brx type bridge 2024-09-10T09:35:56.634 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip addr flush dev ceph-brx 2024-09-10T09:35:56.634 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip link set ceph-brx up 2024-09-10T09:35:56.634 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-09-10T09:35:56.634 DEBUG:teuthology.orchestra.run.smithi005:> ') 2024-09-10T09:35:56.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:56 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T09:35:56.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:35:56 smithi005 ceph-mon[30350]: pgmap v317: 129 pgs: 129 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T09:35:57.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:57 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T09:35:57.055 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T09:35:57.055 DEBUG:teuthology.orchestra.run.smithi005:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-10T09:35:57.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:35:56 smithi159 ceph-mon[35513]: pgmap v317: 129 pgs: 129 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T09:35:57.114 INFO:teuthology.orchestra.run.smithi005.stdout:1 2024-09-10T09:35:57.117 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T09:35:57.117 DEBUG:teuthology.orchestra.run.smithi005:> route 2024-09-10T09:35:57.135 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:35:56 smithi102 ceph-mon[35442]: pgmap v317: 129 pgs: 129 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T09:35:57.167 INFO:teuthology.orchestra.run.smithi005.stdout:Kernel IP routing table 2024-09-10T09:35:57.168 INFO:teuthology.orchestra.run.smithi005.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-10T09:35:57.168 INFO:teuthology.orchestra.run.smithi005.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-09-10T09:35:57.168 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-09-10T09:35:57.168 INFO:teuthology.orchestra.run.smithi005.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-10T09:35:57.169 DEBUG:teuthology.orchestra.run.smithi005:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T09:35:57.169 DEBUG:teuthology.orchestra.run.smithi005:> set -e 2024-09-10T09:35:57.169 DEBUG:teuthology.orchestra.run.smithi005:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-10T09:35:57.169 DEBUG:teuthology.orchestra.run.smithi005:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-10T09:35:57.169 DEBUG:teuthology.orchestra.run.smithi005:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-10T09:35:57.170 DEBUG:teuthology.orchestra.run.smithi005:> ') 2024-09-10T09:35:57.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:57 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T09:35:57.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:57 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T09:35:57.739 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T09:35:57.739 DEBUG:teuthology.orchestra.run.smithi005:> ip netns list 2024-09-10T09:35:57.775 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T09:35:57.775 DEBUG:teuthology.orchestra.run.smithi005:> ip netns list-id 2024-09-10T09:35:57.851 DEBUG:teuthology.orchestra.run.smithi005:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T09:35:57.851 DEBUG:teuthology.orchestra.run.smithi005:> set -e 2024-09-10T09:35:57.851 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-10T09:35:57.852 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2024-09-10T09:35:57.852 DEBUG:teuthology.orchestra.run.smithi005:> ') 2024-09-10T09:35:57.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:57 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T09:35:58.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:58 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T09:35:58.031 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2024-09-10T09:35:58.031 DEBUG:teuthology.orchestra.run.smithi005:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T09:35:58.031 DEBUG:teuthology.orchestra.run.smithi005:> set -e 2024-09-10T09:35:58.031 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2024-09-10T09:35:58.032 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-09-10T09:35:58.032 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2024-09-10T09:35:58.032 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2024-09-10T09:35:58.032 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2024-09-10T09:35:58.032 DEBUG:teuthology.orchestra.run.smithi005:> ') 2024-09-10T09:35:58.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:58 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T09:35:58.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:58 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T09:35:58.281 DEBUG:teuthology.orchestra.run.smithi005:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T09:35:58.282 DEBUG:teuthology.orchestra.run.smithi005:> set -e 2024-09-10T09:35:58.282 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip link set brx.0 up 2024-09-10T09:35:58.282 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip link set dev brx.0 master ceph-brx 2024-09-10T09:35:58.282 DEBUG:teuthology.orchestra.run.smithi005:> ') 2024-09-10T09:35:58.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:58 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T09:35:58.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:58 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T09:35:58.453 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2024-09-10T09:35:58.454 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T09:35:58.454 DEBUG:teuthology.orchestra.run.smithi005:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2024-09-10T09:35:58.489 INFO:teuthology.orchestra.run.smithi005.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2024-09-10T09:35:58.489 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T09:35:58.490 DEBUG:teuthology.orchestra.run.smithi005:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2024-09-10T09:35:58.565 DEBUG:teuthology.orchestra.run.smithi005:> sudo modprobe fuse 2024-09-10T09:35:58.662 DEBUG:teuthology.orchestra.run.smithi005:> cat /proc/self/mounts | awk '{print $2}' 2024-09-10T09:35:58.699 INFO:teuthology.orchestra.run.smithi005.stdout:/proc 2024-09-10T09:35:58.699 INFO:teuthology.orchestra.run.smithi005.stdout:/sys 2024-09-10T09:35:58.699 INFO:teuthology.orchestra.run.smithi005.stdout:/dev 2024-09-10T09:35:58.699 INFO:teuthology.orchestra.run.smithi005.stdout:/sys/kernel/security 2024-09-10T09:35:58.699 INFO:teuthology.orchestra.run.smithi005.stdout:/dev/shm 2024-09-10T09:35:58.699 INFO:teuthology.orchestra.run.smithi005.stdout:/dev/pts 2024-09-10T09:35:58.699 INFO:teuthology.orchestra.run.smithi005.stdout:/run 2024-09-10T09:35:58.699 INFO:teuthology.orchestra.run.smithi005.stdout:/sys/fs/cgroup 2024-09-10T09:35:58.699 INFO:teuthology.orchestra.run.smithi005.stdout:/sys/fs/pstore 2024-09-10T09:35:58.700 INFO:teuthology.orchestra.run.smithi005.stdout:/sys/fs/bpf 2024-09-10T09:35:58.700 INFO:teuthology.orchestra.run.smithi005.stdout:/ 2024-09-10T09:35:58.700 INFO:teuthology.orchestra.run.smithi005.stdout:/sys/fs/selinux 2024-09-10T09:35:58.700 INFO:teuthology.orchestra.run.smithi005.stdout:/proc/sys/fs/binfmt_misc 2024-09-10T09:35:58.700 INFO:teuthology.orchestra.run.smithi005.stdout:/dev/mqueue 2024-09-10T09:35:58.700 INFO:teuthology.orchestra.run.smithi005.stdout:/dev/hugepages 2024-09-10T09:35:58.700 INFO:teuthology.orchestra.run.smithi005.stdout:/sys/kernel/debug 2024-09-10T09:35:58.700 INFO:teuthology.orchestra.run.smithi005.stdout:/sys/kernel/tracing 2024-09-10T09:35:58.700 INFO:teuthology.orchestra.run.smithi005.stdout:/sys/fs/fuse/connections 2024-09-10T09:35:58.700 INFO:teuthology.orchestra.run.smithi005.stdout:/sys/kernel/config 2024-09-10T09:35:58.700 INFO:teuthology.orchestra.run.smithi005.stdout:/run/credentials/systemd-sysctl.service 2024-09-10T09:35:58.700 INFO:teuthology.orchestra.run.smithi005.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-10T09:35:58.701 INFO:teuthology.orchestra.run.smithi005.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-10T09:35:58.701 INFO:teuthology.orchestra.run.smithi005.stdout:/var/lib/nfs/rpc_pipefs 2024-09-10T09:35:58.701 INFO:teuthology.orchestra.run.smithi005.stdout:/run/user/1000 2024-09-10T09:35:58.701 INFO:teuthology.orchestra.run.smithi005.stdout:/var/lib/ceph 2024-09-10T09:35:58.701 INFO:teuthology.orchestra.run.smithi005.stdout:/var/lib/containers/storage/overlay 2024-09-10T09:35:58.701 INFO:teuthology.orchestra.run.smithi005.stdout:/var/lib/containers/storage/overlay/788531062f153cac5d13acefa427409819c71b10db5f4e018c8b04130b189813/merged 2024-09-10T09:35:58.701 INFO:teuthology.orchestra.run.smithi005.stdout:/var/lib/containers/storage/overlay/b482a958f1574ef3f89ee101c02de10b94f9d37dac8d7dbf9f0a9563a91e6cde/merged 2024-09-10T09:35:58.701 INFO:teuthology.orchestra.run.smithi005.stdout:/run/user/0 2024-09-10T09:35:58.701 INFO:teuthology.orchestra.run.smithi005.stdout:/proc/sys/fs/binfmt_misc 2024-09-10T09:35:58.701 INFO:teuthology.orchestra.run.smithi005.stdout:/var/lib/containers/storage/overlay/1fdf7ec3256c96ca4d2db5ab1880173e35409bdee3b42093ee4de511e221d3b5/merged 2024-09-10T09:35:58.701 INFO:teuthology.orchestra.run.smithi005.stdout:/var/lib/containers/storage/overlay/5cb9ca8fc20b723eabdf024cd027df359dcefe0d099ce811ef7cea13aa602e43/merged 2024-09-10T09:35:58.701 INFO:teuthology.orchestra.run.smithi005.stdout:/var/lib/containers/storage/overlay/98721cadf7ce6d357af7b45f0e2edb3cb3c84b54429e905d1efee4ce3272fd47/merged 2024-09-10T09:35:58.701 INFO:teuthology.orchestra.run.smithi005.stdout:/var/lib/containers/storage/overlay/bc0e61220221011a90372d5aacc3d40f97eef465e8b7c434ffeff35157971112/merged 2024-09-10T09:35:58.701 INFO:teuthology.orchestra.run.smithi005.stdout:/var/lib/containers/storage/overlay/48ae453ad08b8010a529babcc29cbf07bfa23d72cd277d4593728885c9ecc16f/merged 2024-09-10T09:35:58.702 INFO:teuthology.orchestra.run.smithi005.stdout:/var/lib/containers/storage/overlay/b1dbdcda67a5c7c9092e13ed6a14dc5d3a6c59d2b5405c1fe66bdf14d5bae916/merged 2024-09-10T09:35:58.702 INFO:teuthology.orchestra.run.smithi005.stdout:/var/lib/containers/storage/overlay/cfb6864f2200fc3d5a08145b38c3d7ab502e0a1568b3766f92f706f9f32ccfca/merged 2024-09-10T09:35:58.702 INFO:teuthology.orchestra.run.smithi005.stdout:/var/lib/containers/storage/overlay/7d07504a835781e99cb11dc6e06491da6eddbe4b285491f3eb755c20c8c12fff/merged 2024-09-10T09:35:58.702 INFO:teuthology.orchestra.run.smithi005.stdout:/run/netns 2024-09-10T09:35:58.702 INFO:teuthology.orchestra.run.smithi005.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-10T09:35:58.702 INFO:teuthology.orchestra.run.smithi005.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-10T09:35:58.703 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T09:35:58.703 DEBUG:teuthology.orchestra.run.smithi005:> ls /sys/fs/fuse/connections 2024-09-10T09:35:58.779 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-10T09:35:58.779 DEBUG:teuthology.orchestra.run.smithi005:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2024-09-10T09:35:58.824 DEBUG:teuthology.orchestra.run.smithi005:> sudo modprobe fuse 2024-09-10T09:35:58.882 DEBUG:teuthology.orchestra.run.smithi005:> cat /proc/self/mounts | awk '{print $2}' 2024-09-10T09:35:58.920 INFO:teuthology.orchestra.run.smithi005.stdout:/proc 2024-09-10T09:35:58.920 INFO:teuthology.orchestra.run.smithi005.stdout:/sys 2024-09-10T09:35:58.920 INFO:teuthology.orchestra.run.smithi005.stdout:/dev 2024-09-10T09:35:58.920 INFO:teuthology.orchestra.run.smithi005.stdout:/sys/kernel/security 2024-09-10T09:35:58.920 INFO:teuthology.orchestra.run.smithi005.stdout:/dev/shm 2024-09-10T09:35:58.920 INFO:teuthology.orchestra.run.smithi005.stdout:/dev/pts 2024-09-10T09:35:58.920 INFO:teuthology.orchestra.run.smithi005.stdout:/run 2024-09-10T09:35:58.921 INFO:teuthology.orchestra.run.smithi005.stdout:/sys/fs/cgroup 2024-09-10T09:35:58.921 INFO:teuthology.orchestra.run.smithi005.stdout:/sys/fs/pstore 2024-09-10T09:35:58.921 INFO:teuthology.orchestra.run.smithi005.stdout:/sys/fs/bpf 2024-09-10T09:35:58.921 INFO:teuthology.orchestra.run.smithi005.stdout:/ 2024-09-10T09:35:58.921 INFO:teuthology.orchestra.run.smithi005.stdout:/sys/fs/selinux 2024-09-10T09:35:58.921 INFO:teuthology.orchestra.run.smithi005.stdout:/proc/sys/fs/binfmt_misc 2024-09-10T09:35:58.921 INFO:teuthology.orchestra.run.smithi005.stdout:/dev/mqueue 2024-09-10T09:35:58.921 INFO:teuthology.orchestra.run.smithi005.stdout:/dev/hugepages 2024-09-10T09:35:58.921 INFO:teuthology.orchestra.run.smithi005.stdout:/sys/kernel/debug 2024-09-10T09:35:58.921 INFO:teuthology.orchestra.run.smithi005.stdout:/sys/kernel/tracing 2024-09-10T09:35:58.921 INFO:teuthology.orchestra.run.smithi005.stdout:/sys/fs/fuse/connections 2024-09-10T09:35:58.921 INFO:teuthology.orchestra.run.smithi005.stdout:/sys/kernel/config 2024-09-10T09:35:58.921 INFO:teuthology.orchestra.run.smithi005.stdout:/run/credentials/systemd-sysctl.service 2024-09-10T09:35:58.922 INFO:teuthology.orchestra.run.smithi005.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-10T09:35:58.922 INFO:teuthology.orchestra.run.smithi005.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-10T09:35:58.922 INFO:teuthology.orchestra.run.smithi005.stdout:/var/lib/nfs/rpc_pipefs 2024-09-10T09:35:58.922 INFO:teuthology.orchestra.run.smithi005.stdout:/run/user/1000 2024-09-10T09:35:58.922 INFO:teuthology.orchestra.run.smithi005.stdout:/var/lib/ceph 2024-09-10T09:35:58.922 INFO:teuthology.orchestra.run.smithi005.stdout:/var/lib/containers/storage/overlay 2024-09-10T09:35:58.922 INFO:teuthology.orchestra.run.smithi005.stdout:/var/lib/containers/storage/overlay/788531062f153cac5d13acefa427409819c71b10db5f4e018c8b04130b189813/merged 2024-09-10T09:35:58.922 INFO:teuthology.orchestra.run.smithi005.stdout:/var/lib/containers/storage/overlay/b482a958f1574ef3f89ee101c02de10b94f9d37dac8d7dbf9f0a9563a91e6cde/merged 2024-09-10T09:35:58.922 INFO:teuthology.orchestra.run.smithi005.stdout:/run/user/0 2024-09-10T09:35:58.922 INFO:teuthology.orchestra.run.smithi005.stdout:/proc/sys/fs/binfmt_misc 2024-09-10T09:35:58.922 INFO:teuthology.orchestra.run.smithi005.stdout:/var/lib/containers/storage/overlay/1fdf7ec3256c96ca4d2db5ab1880173e35409bdee3b42093ee4de511e221d3b5/merged 2024-09-10T09:35:58.922 INFO:teuthology.orchestra.run.smithi005.stdout:/var/lib/containers/storage/overlay/5cb9ca8fc20b723eabdf024cd027df359dcefe0d099ce811ef7cea13aa602e43/merged 2024-09-10T09:35:58.922 INFO:teuthology.orchestra.run.smithi005.stdout:/var/lib/containers/storage/overlay/98721cadf7ce6d357af7b45f0e2edb3cb3c84b54429e905d1efee4ce3272fd47/merged 2024-09-10T09:35:58.923 INFO:teuthology.orchestra.run.smithi005.stdout:/var/lib/containers/storage/overlay/bc0e61220221011a90372d5aacc3d40f97eef465e8b7c434ffeff35157971112/merged 2024-09-10T09:35:58.923 INFO:teuthology.orchestra.run.smithi005.stdout:/var/lib/containers/storage/overlay/48ae453ad08b8010a529babcc29cbf07bfa23d72cd277d4593728885c9ecc16f/merged 2024-09-10T09:35:58.923 INFO:teuthology.orchestra.run.smithi005.stdout:/var/lib/containers/storage/overlay/b1dbdcda67a5c7c9092e13ed6a14dc5d3a6c59d2b5405c1fe66bdf14d5bae916/merged 2024-09-10T09:35:58.923 INFO:teuthology.orchestra.run.smithi005.stdout:/var/lib/containers/storage/overlay/cfb6864f2200fc3d5a08145b38c3d7ab502e0a1568b3766f92f706f9f32ccfca/merged 2024-09-10T09:35:58.923 INFO:teuthology.orchestra.run.smithi005.stdout:/var/lib/containers/storage/overlay/7d07504a835781e99cb11dc6e06491da6eddbe4b285491f3eb755c20c8c12fff/merged 2024-09-10T09:35:58.923 INFO:teuthology.orchestra.run.smithi005.stdout:/run/netns 2024-09-10T09:35:58.923 INFO:teuthology.orchestra.run.smithi005.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-10T09:35:58.923 INFO:teuthology.orchestra.run.smithi005.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-10T09:35:58.924 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T09:35:58.924 DEBUG:teuthology.orchestra.run.smithi005:> ls /sys/fs/fuse/connections 2024-09-10T09:35:58.990 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi005.stderr:2024-09-10T09:35:58.989+0000 7fe68231a480 -1 init, newargv = 0x5647413b7440 newargc=15 2024-09-10T09:35:58.991 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi005.stderr:ceph-fuse[75409]: starting ceph client 2024-09-10T09:35:59.001 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi005.stderr:ceph-fuse[75409]: starting fuse 2024-09-10T09:35:59.004 INFO:teuthology.orchestra.run.smithi005.stdout:120 2024-09-10T09:35:59.004 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [120] 2024-09-10T09:35:59.004 DEBUG:teuthology.orchestra.run.smithi005:> sudo stdin-killer -- python3 -c ' 2024-09-10T09:35:59.005 DEBUG:teuthology.orchestra.run.smithi005:> import glob 2024-09-10T09:35:59.005 DEBUG:teuthology.orchestra.run.smithi005:> import re 2024-09-10T09:35:59.005 DEBUG:teuthology.orchestra.run.smithi005:> import os 2024-09-10T09:35:59.005 DEBUG:teuthology.orchestra.run.smithi005:> import subprocess 2024-09-10T09:35:59.005 DEBUG:teuthology.orchestra.run.smithi005:> 2024-09-10T09:35:59.005 DEBUG:teuthology.orchestra.run.smithi005:> def _find_admin_socket(client_name): 2024-09-10T09:35:59.005 DEBUG:teuthology.orchestra.run.smithi005:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2024-09-10T09:35:59.005 DEBUG:teuthology.orchestra.run.smithi005:> files = glob.glob(asok_path) 2024-09-10T09:35:59.005 DEBUG:teuthology.orchestra.run.smithi005:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2024-09-10T09:35:59.005 DEBUG:teuthology.orchestra.run.smithi005:> 2024-09-10T09:35:59.005 DEBUG:teuthology.orchestra.run.smithi005:> # Given a non-glob path, it better be there 2024-09-10T09:35:59.005 DEBUG:teuthology.orchestra.run.smithi005:> if "*" not in asok_path: 2024-09-10T09:35:59.005 DEBUG:teuthology.orchestra.run.smithi005:> assert(len(files) == 1) 2024-09-10T09:35:59.005 DEBUG:teuthology.orchestra.run.smithi005:> return files[0] 2024-09-10T09:35:59.006 DEBUG:teuthology.orchestra.run.smithi005:> 2024-09-10T09:35:59.006 DEBUG:teuthology.orchestra.run.smithi005:> for f in files: 2024-09-10T09:35:59.006 DEBUG:teuthology.orchestra.run.smithi005:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-10T09:35:59.006 DEBUG:teuthology.orchestra.run.smithi005:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-10T09:35:59.006 DEBUG:teuthology.orchestra.run.smithi005:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-10T09:35:59.006 DEBUG:teuthology.orchestra.run.smithi005:> contents = proc_f.read() 2024-09-10T09:35:59.006 DEBUG:teuthology.orchestra.run.smithi005:> if mountpoint in contents: 2024-09-10T09:35:59.006 DEBUG:teuthology.orchestra.run.smithi005:> return f 2024-09-10T09:35:59.006 DEBUG:teuthology.orchestra.run.smithi005:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-10T09:35:59.006 DEBUG:teuthology.orchestra.run.smithi005:> 2024-09-10T09:35:59.006 DEBUG:teuthology.orchestra.run.smithi005:> print(_find_admin_socket("client.admin")) 2024-09-10T09:35:59.006 DEBUG:teuthology.orchestra.run.smithi005:> ' 2024-09-10T09:35:59.055 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:35:58 smithi005 ceph-mon[30350]: pgmap v318: 129 pgs: 129 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T09:35:59.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:35:58 smithi159 ceph-mon[35513]: pgmap v318: 129 pgs: 129 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T09:35:59.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:35:58 smithi102 ceph-mon[35442]: pgmap v318: 129 pgs: 129 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T09:35:59.169 INFO:teuthology.orchestra.run.smithi005.stdout:/var/run/ceph/ceph-client.admin.75409.asok 2024-09-10T09:35:59.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:59 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T09:35:59.183 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.75409.asok 2024-09-10T09:35:59.183 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T09:35:59.183 DEBUG:teuthology.orchestra.run.smithi005:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.75409.asok status 2024-09-10T09:35:59.368 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-10T09:35:59.368 INFO:teuthology.orchestra.run.smithi005.stdout: "metadata": { 2024-09-10T09:35:59.369 INFO:teuthology.orchestra.run.smithi005.stdout: "ceph_sha1": "1cf819b7212a6048e92fc78c4eff963344f6d56b", 2024-09-10T09:35:59.369 INFO:teuthology.orchestra.run.smithi005.stdout: "ceph_version": "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)", 2024-09-10T09:35:59.369 INFO:teuthology.orchestra.run.smithi005.stdout: "entity_id": "admin", 2024-09-10T09:35:59.369 INFO:teuthology.orchestra.run.smithi005.stdout: "hostname": "smithi005", 2024-09-10T09:35:59.369 INFO:teuthology.orchestra.run.smithi005.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2024-09-10T09:35:59.369 INFO:teuthology.orchestra.run.smithi005.stdout: "pid": "75409", 2024-09-10T09:35:59.369 INFO:teuthology.orchestra.run.smithi005.stdout: "root": "/" 2024-09-10T09:35:59.369 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T09:35:59.369 INFO:teuthology.orchestra.run.smithi005.stdout: "dentry_count": 0, 2024-09-10T09:35:59.369 INFO:teuthology.orchestra.run.smithi005.stdout: "dentry_pinned_count": 0, 2024-09-10T09:35:59.369 INFO:teuthology.orchestra.run.smithi005.stdout: "id": 14640, 2024-09-10T09:35:59.369 INFO:teuthology.orchestra.run.smithi005.stdout: "inst": { 2024-09-10T09:35:59.369 INFO:teuthology.orchestra.run.smithi005.stdout: "name": { 2024-09-10T09:35:59.370 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "client", 2024-09-10T09:35:59.370 INFO:teuthology.orchestra.run.smithi005.stdout: "num": 14640 2024-09-10T09:35:59.370 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T09:35:59.370 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": { 2024-09-10T09:35:59.370 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-09-10T09:35:59.370 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "192.168.0.1:0", 2024-09-10T09:35:59.370 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 372974339 2024-09-10T09:35:59.370 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T09:35:59.370 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T09:35:59.370 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": { 2024-09-10T09:35:59.370 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-09-10T09:35:59.370 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "192.168.0.1:0", 2024-09-10T09:35:59.370 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 372974339 2024-09-10T09:35:59.371 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T09:35:59.371 INFO:teuthology.orchestra.run.smithi005.stdout: "inst_str": "client.14640 192.168.0.1:0/372974339", 2024-09-10T09:35:59.371 INFO:teuthology.orchestra.run.smithi005.stdout: "addr_str": "192.168.0.1:0/372974339", 2024-09-10T09:35:59.371 INFO:teuthology.orchestra.run.smithi005.stdout: "inode_count": 1, 2024-09-10T09:35:59.371 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_epoch": 17, 2024-09-10T09:35:59.371 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_epoch": 92, 2024-09-10T09:35:59.371 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_epoch_barrier": 0, 2024-09-10T09:35:59.371 INFO:teuthology.orchestra.run.smithi005.stdout: "blocklisted": false, 2024-09-10T09:35:59.371 INFO:teuthology.orchestra.run.smithi005.stdout: "fs_name": "cephfs" 2024-09-10T09:35:59.371 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-10T09:35:59.382 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T09:35:59.382 DEBUG:teuthology.orchestra.run.smithi005:> stat --file-system '--printf=%T 2024-09-10T09:35:59.382 DEBUG:teuthology.orchestra.run.smithi005:> ' -- /home/ubuntu/cephtest/mnt.admin 2024-09-10T09:35:59.418 INFO:teuthology.orchestra.run.smithi005.stdout:fuseblk 2024-09-10T09:35:59.418 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2024-09-10T09:35:59.419 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T09:35:59.419 DEBUG:teuthology.orchestra.run.smithi005:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2024-09-10T09:35:59.517 DEBUG:teuthology.orchestra.run.smithi005:> (cd /home/ubuntu/cephtest/mnt.admin && exec stdin-killer --timeout=900 -- bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2024-09-10T09:35:59.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:59 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T09:35:59.623 INFO:teuthology.orchestra.run.smithi005.stdout:# file: . 2024-09-10T09:35:59.624 INFO:teuthology.orchestra.run.smithi005.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2024-09-10T09:35:59.624 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-10T09:35:59.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:35:59 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T09:35:59.632 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-09-10T09:35:59.632 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T09:35:59.669 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph osd blocklist ls 2024-09-10T09:36:00.088 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:36:00.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:00.685+0000 7f0661800640 1 -- 172.21.15.5:0/866248777 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f065c076720 msgr2=0x7f065c076b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:36:00.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:00.685+0000 7f0661800640 1 --2- 172.21.15.5:0/866248777 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f065c076720 0x7f065c076b00 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f064c009900 tx=0x7f064c02a3f0 comp rx=0 tx=0).stop 2024-09-10T09:36:00.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:00.685+0000 7f0661800640 1 -- 172.21.15.5:0/866248777 shutdown_connections 2024-09-10T09:36:00.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:00.685+0000 7f0661800640 1 --2- 172.21.15.5:0/866248777 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f065c112a30 0x7f065c114e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:36:00.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:00.685+0000 7f0661800640 1 --2- 172.21.15.5:0/866248777 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f065c077040 0x7f065c1124f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:36:00.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:00.685+0000 7f0661800640 1 --2- 172.21.15.5:0/866248777 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f065c076720 0x7f065c076b00 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:36:00.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:00.685+0000 7f0661800640 1 -- 172.21.15.5:0/866248777 >> 172.21.15.5:0/866248777 conn(0x7f065c0ffa90 msgr2=0x7f065c101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:36:00.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:00.685+0000 7f0661800640 1 -- 172.21.15.5:0/866248777 shutdown_connections 2024-09-10T09:36:00.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:00.686+0000 7f0661800640 1 -- 172.21.15.5:0/866248777 wait complete. 2024-09-10T09:36:00.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:00.687+0000 7f0661800640 1 Processor -- start 2024-09-10T09:36:00.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:00.687+0000 7f0661800640 1 -- start start 2024-09-10T09:36:00.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:00.687+0000 7f0661800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f065c076720 0x7f065c1a4710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:36:00.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:00.687+0000 7f0661800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f065c077040 0x7f065c1a4c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:36:00.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:00.687+0000 7f0661800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f065c112a30 0x7f065c19e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:36:00.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:00.687+0000 7f0661800640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f065c1185c0 con 0x7f065c112a30 2024-09-10T09:36:00.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:00.687+0000 7f0661800640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f065c118440 con 0x7f065c077040 2024-09-10T09:36:00.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:00.687+0000 7f0661800640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f065c118740 con 0x7f065c076720 2024-09-10T09:36:00.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:00.688+0000 7f065aa00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f065c076720 0x7f065c1a4710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:36:00.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:00.688+0000 7f065aa00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f065c076720 0x7f065c1a4710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:40174/0 (socket says 172.21.15.5:40174) 2024-09-10T09:36:00.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:00.688+0000 7f065aa00640 1 -- 172.21.15.5:0/1578245538 learned_addr learned my addr 172.21.15.5:0/1578245538 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:36:00.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:00.688+0000 7f065b400640 1 --2- 172.21.15.5:0/1578245538 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f065c112a30 0x7f065c19e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:36:00.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:00.688+0000 7f065a000640 1 --2- 172.21.15.5:0/1578245538 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f065c077040 0x7f065c1a4c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:36:00.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:00.688+0000 7f065b400640 1 -- 172.21.15.5:0/1578245538 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f065c076720 msgr2=0x7f065c1a4710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:36:00.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:00.688+0000 7f065b400640 1 --2- 172.21.15.5:0/1578245538 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f065c076720 0x7f065c1a4710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:36:00.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:00.688+0000 7f065b400640 1 -- 172.21.15.5:0/1578245538 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f065c077040 msgr2=0x7f065c1a4c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:36:00.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:00.688+0000 7f065b400640 1 --2- 172.21.15.5:0/1578245538 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f065c077040 0x7f065c1a4c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:36:00.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:00.688+0000 7f065b400640 1 -- 172.21.15.5:0/1578245538 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f065c19f000 con 0x7f065c112a30 2024-09-10T09:36:00.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:00.688+0000 7f065aa00640 1 --2- 172.21.15.5:0/1578245538 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f065c076720 0x7f065c1a4710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:36:00.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:00.689+0000 7f065b400640 1 --2- 172.21.15.5:0/1578245538 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f065c112a30 0x7f065c19e8d0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f065c07aa10 tx=0x7f065400ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:36:00.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:00.689+0000 7f0653400640 1 -- 172.21.15.5:0/1578245538 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f065400cdf0 con 0x7f065c112a30 2024-09-10T09:36:00.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:00.689+0000 7f0653400640 1 -- 172.21.15.5:0/1578245538 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0654005020 con 0x7f065c112a30 2024-09-10T09:36:00.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:00.689+0000 7f0653400640 1 -- 172.21.15.5:0/1578245538 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0654010670 con 0x7f065c112a30 2024-09-10T09:36:00.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:00.689+0000 7f0661800640 1 -- 172.21.15.5:0/1578245538 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f065c19f2f0 con 0x7f065c112a30 2024-09-10T09:36:00.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:00.689+0000 7f0661800640 1 -- 172.21.15.5:0/1578245538 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f065c1ab5e0 con 0x7f065c112a30 2024-09-10T09:36:00.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:00.690+0000 7f0661800640 1 -- 172.21.15.5:0/1578245538 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f065c076b00 con 0x7f065c112a30 2024-09-10T09:36:00.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:00.692+0000 7f0653400640 1 -- 172.21.15.5:0/1578245538 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7f06540054d0 con 0x7f065c112a30 2024-09-10T09:36:00.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:00.693+0000 7f0653400640 1 --2- 172.21.15.5:0/1578245538 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f06400a2b80 0x7f06400a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:36:00.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:00.694+0000 7f0653400640 1 -- 172.21.15.5:0/1578245538 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8802+0+0 (secure 0 0 0) 0x7f06540cf890 con 0x7f065c112a30 2024-09-10T09:36:00.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:00.694+0000 7f065aa00640 1 --2- 172.21.15.5:0/1578245538 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f06400a2b80 0x7f06400a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:36:00.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:00.695+0000 7f065aa00640 1 --2- 172.21.15.5:0/1578245538 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f06400a2b80 0x7f06400a5040 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f064c031550 tx=0x7f064c005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:36:00.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:00.696+0000 7f0653400640 1 -- 172.21.15.5:0/1578245538 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f065409c550 con 0x7f065c112a30 2024-09-10T09:36:00.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:00.895+0000 7f0661800640 1 -- 172.21.15.5:0/1578245538 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f065c19fe00 con 0x7f065c112a30 2024-09-10T09:36:00.897 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:36:00 smithi005 ceph-mon[30350]: pgmap v319: 129 pgs: 129 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T09:36:00.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:00.895+0000 7f0653400640 1 -- 172.21.15.5:0/1578245538 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v92) ==== 81+0+576 (secure 0 0 0) 0x7f065c19fe00 con 0x7f065c112a30 2024-09-10T09:36:00.898 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5:6801/1527422686 2024-09-11T09:28:05.436742+0000 2024-09-10T09:36:00.898 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5:0/2276144345 2024-09-11T09:28:05.436742+0000 2024-09-10T09:36:00.898 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5:0/412040104 2024-09-11T09:28:05.436742+0000 2024-09-10T09:36:00.898 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5:6800/1527422686 2024-09-11T09:28:05.436742+0000 2024-09-10T09:36:00.898 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5:0/1856581170 2024-09-11T09:28:05.436742+0000 2024-09-10T09:36:00.898 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5:0/2847668298 2024-09-11T09:27:37.778500+0000 2024-09-10T09:36:00.898 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5:6801/14425913 2024-09-11T09:27:37.778500+0000 2024-09-10T09:36:00.898 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5:0/166910782 2024-09-11T09:27:37.778500+0000 2024-09-10T09:36:00.898 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5:6800/14425913 2024-09-11T09:27:37.778500+0000 2024-09-10T09:36:00.898 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5:0/1711858179 2024-09-11T09:27:37.778500+0000 2024-09-10T09:36:00.899 INFO:teuthology.orchestra.run.smithi005.stderr:listed 10 entries 2024-09-10T09:36:00.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:00.897+0000 7f0661800640 1 -- 172.21.15.5:0/1578245538 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f06400a2b80 msgr2=0x7f06400a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:36:00.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:00.897+0000 7f0661800640 1 --2- 172.21.15.5:0/1578245538 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f06400a2b80 0x7f06400a5040 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f064c031550 tx=0x7f064c005a20 comp rx=0 tx=0).stop 2024-09-10T09:36:00.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:00.897+0000 7f0661800640 1 -- 172.21.15.5:0/1578245538 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f065c112a30 msgr2=0x7f065c19e8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:36:00.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:00.897+0000 7f0661800640 1 --2- 172.21.15.5:0/1578245538 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f065c112a30 0x7f065c19e8d0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f065c07aa10 tx=0x7f065400ef10 comp rx=0 tx=0).stop 2024-09-10T09:36:00.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:00.898+0000 7f0661800640 1 -- 172.21.15.5:0/1578245538 shutdown_connections 2024-09-10T09:36:00.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:00.898+0000 7f0661800640 1 --2- 172.21.15.5:0/1578245538 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f06400a2b80 0x7f06400a5040 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:36:00.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:00.898+0000 7f0661800640 1 --2- 172.21.15.5:0/1578245538 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f065c112a30 0x7f065c19e8d0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:36:00.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:00.898+0000 7f0661800640 1 --2- 172.21.15.5:0/1578245538 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f065c077040 0x7f065c1a4c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:36:00.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:00.898+0000 7f0661800640 1 --2- 172.21.15.5:0/1578245538 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f065c076720 0x7f065c1a4710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:36:00.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:00.898+0000 7f0661800640 1 -- 172.21.15.5:0/1578245538 >> 172.21.15.5:0/1578245538 conn(0x7f065c0ffa90 msgr2=0x7f065c101e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:36:00.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:00.898+0000 7f0661800640 1 -- 172.21.15.5:0/1578245538 shutdown_connections 2024-09-10T09:36:00.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:00.898+0000 7f0661800640 1 -- 172.21.15.5:0/1578245538 wait complete. 2024-09-10T09:36:01.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:36:00 smithi159 ceph-mon[35513]: pgmap v319: 129 pgs: 129 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T09:36:01.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:36:00 smithi102 ceph-mon[35442]: pgmap v319: 129 pgs: 129 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T09:36:01.672 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-09-10T09:36:01.673 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T09:36:01.710 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph osd blocklist ls 2024-09-10T09:36:02.058 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:36:01 smithi005 ceph-mon[30350]: from='client.? 172.21.15.5:0/1578245538' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-10T09:36:02.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:36:01 smithi159 ceph-mon[35513]: from='client.? 172.21.15.5:0/1578245538' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-10T09:36:02.133 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:36:02.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:36:01 smithi102 ceph-mon[35442]: from='client.? 172.21.15.5:0/1578245538' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-10T09:36:02.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:02.671+0000 7f1467000640 1 -- 172.21.15.5:0/391606368 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f14601049c0 msgr2=0x7f1460108a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:36:02.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:02.671+0000 7f1467000640 1 --2- 172.21.15.5:0/391606368 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f14601049c0 0x7f1460108a10 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f1448009900 tx=0x7f144802a3f0 comp rx=0 tx=0).stop 2024-09-10T09:36:02.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:02.671+0000 7f1467000640 1 -- 172.21.15.5:0/391606368 shutdown_connections 2024-09-10T09:36:02.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:02.671+0000 7f1467000640 1 --2- 172.21.15.5:0/391606368 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1460109140 0x7f1460110cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:36:02.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:02.671+0000 7f1467000640 1 --2- 172.21.15.5:0/391606368 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f14601049c0 0x7f1460108a10 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:36:02.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:02.671+0000 7f1467000640 1 --2- 172.21.15.5:0/391606368 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1460104010 0x7f14601043f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:36:02.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:02.671+0000 7f1467000640 1 -- 172.21.15.5:0/391606368 >> 172.21.15.5:0/391606368 conn(0x7f14600ffa50 msgr2=0x7f1460101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:36:02.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:02.671+0000 7f1467000640 1 -- 172.21.15.5:0/391606368 shutdown_connections 2024-09-10T09:36:02.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:02.671+0000 7f1467000640 1 -- 172.21.15.5:0/391606368 wait complete. 2024-09-10T09:36:02.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:02.672+0000 7f1467000640 1 Processor -- start 2024-09-10T09:36:02.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:02.672+0000 7f1467000640 1 -- start start 2024-09-10T09:36:02.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:02.672+0000 7f1467000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1460104010 0x7f146019a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:36:02.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:02.673+0000 7f1467000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f14601049c0 0x7f146019ac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:36:02.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:02.673+0000 7f1467000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1460109140 0x7f146019f000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:36:02.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:02.673+0000 7f1467000640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1460114160 con 0x7f1460109140 2024-09-10T09:36:02.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:02.673+0000 7f1467000640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f1460113fe0 con 0x7f1460104010 2024-09-10T09:36:02.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:02.673+0000 7f1467000640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f14601142e0 con 0x7f14601049c0 2024-09-10T09:36:02.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:02.673+0000 7f145fe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1460104010 0x7f146019a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:36:02.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:02.673+0000 7f1465000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1460109140 0x7f146019f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:36:02.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:02.673+0000 7f145fe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1460104010 0x7f146019a730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:34576/0 (socket says 172.21.15.5:34576) 2024-09-10T09:36:02.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:02.673+0000 7f145fe00640 1 -- 172.21.15.5:0/189494268 learned_addr learned my addr 172.21.15.5:0/189494268 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:36:02.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:02.673+0000 7f1457e00640 1 --2- 172.21.15.5:0/189494268 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f14601049c0 0x7f146019ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:36:02.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:02.673+0000 7f145fe00640 1 -- 172.21.15.5:0/189494268 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f14601049c0 msgr2=0x7f146019ac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:36:02.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:02.673+0000 7f145fe00640 1 --2- 172.21.15.5:0/189494268 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f14601049c0 0x7f146019ac70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:36:02.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:02.673+0000 7f145fe00640 1 -- 172.21.15.5:0/189494268 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1460109140 msgr2=0x7f146019f000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:36:02.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:02.673+0000 7f145fe00640 1 --2- 172.21.15.5:0/189494268 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1460109140 0x7f146019f000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:36:02.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:02.673+0000 7f145fe00640 1 -- 172.21.15.5:0/189494268 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f146019f6e0 con 0x7f1460104010 2024-09-10T09:36:02.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:02.674+0000 7f145fe00640 1 --2- 172.21.15.5:0/189494268 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1460104010 0x7f146019a730 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f145000e9f0 tx=0x7f145000edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:36:02.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:02.674+0000 7f145d600640 1 -- 172.21.15.5:0/189494268 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f145000cd80 con 0x7f1460104010 2024-09-10T09:36:02.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:02.674+0000 7f1467000640 1 -- 172.21.15.5:0/189494268 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f146019f9d0 con 0x7f1460104010 2024-09-10T09:36:02.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:02.674+0000 7f145d600640 1 -- 172.21.15.5:0/189494268 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1450005500 con 0x7f1460104010 2024-09-10T09:36:02.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:02.674+0000 7f145d600640 1 -- 172.21.15.5:0/189494268 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1450010690 con 0x7f1460104010 2024-09-10T09:36:02.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:02.674+0000 7f1467000640 1 -- 172.21.15.5:0/189494268 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14601a72b0 con 0x7f1460104010 2024-09-10T09:36:02.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:02.675+0000 7f1467000640 1 -- 172.21.15.5:0/189494268 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f14601043f0 con 0x7f1460104010 2024-09-10T09:36:02.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:02.677+0000 7f145d600640 1 -- 172.21.15.5:0/189494268 <== mon.2 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7f14500046c0 con 0x7f1460104010 2024-09-10T09:36:02.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:02.679+0000 7f145d600640 1 --2- 172.21.15.5:0/189494268 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f14440a2c00 0x7f14440a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:36:02.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:02.679+0000 7f145d600640 1 -- 172.21.15.5:0/189494268 <== mon.2 v2:172.21.15.102:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8802+0+0 (secure 0 0 0) 0x7f14500cf840 con 0x7f1460104010 2024-09-10T09:36:02.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:02.679+0000 7f1457e00640 1 --2- 172.21.15.5:0/189494268 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f14440a2c00 0x7f14440a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:36:02.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:02.680+0000 7f1457e00640 1 --2- 172.21.15.5:0/189494268 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f14440a2c00 0x7f14440a50c0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f1448009230 tx=0x7f14480058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:36:02.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:02.681+0000 7f145d600640 1 -- 172.21.15.5:0/189494268 <== mon.2 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f14500541b0 con 0x7f1460104010 2024-09-10T09:36:02.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:02.878+0000 7f1467000640 1 -- 172.21.15.5:0/189494268 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f146019ba90 con 0x7f1460104010 2024-09-10T09:36:02.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:36:02 smithi005 ceph-mon[30350]: pgmap v320: 129 pgs: 129 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 0 op/s 2024-09-10T09:36:02.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:02.878+0000 7f145d600640 1 -- 172.21.15.5:0/189494268 <== mon.2 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v92) ==== 81+0+576 (secure 0 0 0) 0x7f145009c500 con 0x7f1460104010 2024-09-10T09:36:02.880 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5:6801/1527422686 2024-09-11T09:28:05.436742+0000 2024-09-10T09:36:02.880 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5:0/2276144345 2024-09-11T09:28:05.436742+0000 2024-09-10T09:36:02.880 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5:0/412040104 2024-09-11T09:28:05.436742+0000 2024-09-10T09:36:02.880 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5:6800/1527422686 2024-09-11T09:28:05.436742+0000 2024-09-10T09:36:02.880 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5:0/1856581170 2024-09-11T09:28:05.436742+0000 2024-09-10T09:36:02.880 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5:0/2847668298 2024-09-11T09:27:37.778500+0000 2024-09-10T09:36:02.880 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5:6801/14425913 2024-09-11T09:27:37.778500+0000 2024-09-10T09:36:02.880 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5:0/166910782 2024-09-11T09:27:37.778500+0000 2024-09-10T09:36:02.880 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5:6800/14425913 2024-09-11T09:27:37.778500+0000 2024-09-10T09:36:02.881 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5:0/1711858179 2024-09-11T09:27:37.778500+0000 2024-09-10T09:36:02.881 INFO:teuthology.orchestra.run.smithi005.stderr:listed 10 entries 2024-09-10T09:36:02.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:02.881+0000 7f1467000640 1 -- 172.21.15.5:0/189494268 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f14440a2c00 msgr2=0x7f14440a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:36:02.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:02.881+0000 7f1467000640 1 --2- 172.21.15.5:0/189494268 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f14440a2c00 0x7f14440a50c0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f1448009230 tx=0x7f14480058d0 comp rx=0 tx=0).stop 2024-09-10T09:36:02.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:02.881+0000 7f1467000640 1 -- 172.21.15.5:0/189494268 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1460104010 msgr2=0x7f146019a730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:36:02.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:02.881+0000 7f1467000640 1 --2- 172.21.15.5:0/189494268 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1460104010 0x7f146019a730 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f145000e9f0 tx=0x7f145000edf0 comp rx=0 tx=0).stop 2024-09-10T09:36:02.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:02.881+0000 7f1467000640 1 -- 172.21.15.5:0/189494268 shutdown_connections 2024-09-10T09:36:02.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:02.881+0000 7f1467000640 1 --2- 172.21.15.5:0/189494268 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f14440a2c00 0x7f14440a50c0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:36:02.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:02.881+0000 7f1467000640 1 --2- 172.21.15.5:0/189494268 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1460109140 0x7f146019f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:36:02.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:02.881+0000 7f1467000640 1 --2- 172.21.15.5:0/189494268 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f14601049c0 0x7f146019ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:36:02.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:02.881+0000 7f1467000640 1 --2- 172.21.15.5:0/189494268 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1460104010 0x7f146019a730 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:36:02.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:02.881+0000 7f1467000640 1 -- 172.21.15.5:0/189494268 >> 172.21.15.5:0/189494268 conn(0x7f14600ffa50 msgr2=0x7f1460100e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:36:02.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:02.881+0000 7f1467000640 1 -- 172.21.15.5:0/189494268 shutdown_connections 2024-09-10T09:36:02.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:02.881+0000 7f1467000640 1 -- 172.21.15.5:0/189494268 wait complete. 2024-09-10T09:36:03.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:36:02 smithi159 ceph-mon[35513]: pgmap v320: 129 pgs: 129 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 0 op/s 2024-09-10T09:36:03.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:36:02 smithi102 ceph-mon[35442]: pgmap v320: 129 pgs: 129 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 0 op/s 2024-09-10T09:36:03.649 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi005.front.sepia.ceph.com... 2024-09-10T09:36:03.649 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T09:36:03.649 DEBUG:teuthology.orchestra.run.smithi005:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2024-09-10T09:36:03.705 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi005.stderr:ceph-fuse[75409]: fuse finished with error 0 and tester_r 0 2024-09-10T09:36:03.708 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T09:36:03.769 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi005.front.sepia.ceph.com 2024-09-10T09:36:03.770 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T09:36:03.770 DEBUG:teuthology.orchestra.run.smithi005:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2024-09-10T09:36:03.812 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-10T09:36:03.901 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:36:03 smithi005 ceph-mon[30350]: from='client.? 172.21.15.5:0/189494268' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-10T09:36:04.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:36:03 smithi159 ceph-mon[35513]: from='client.? 172.21.15.5:0/189494268' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-10T09:36:04.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:36:03 smithi102 ceph-mon[35442]: from='client.? 172.21.15.5:0/189494268' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-10T09:36:04.230 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:36:04.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:04.787+0000 7f9d25c00640 1 -- 172.21.15.5:0/1662142936 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9d20074620 msgr2=0x7f9d20074a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:36:04.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:04.787+0000 7f9d25c00640 1 --2- 172.21.15.5:0/1662142936 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9d20074620 0x7f9d20074a00 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f9d10009900 tx=0x7f9d1002a480 comp rx=0 tx=0).stop 2024-09-10T09:36:04.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:04.788+0000 7f9d25c00640 1 -- 172.21.15.5:0/1662142936 shutdown_connections 2024-09-10T09:36:04.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:04.788+0000 7f9d25c00640 1 --2- 172.21.15.5:0/1662142936 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9d2010e3c0 0x7f9d20110810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:36:04.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:04.788+0000 7f9d25c00640 1 --2- 172.21.15.5:0/1662142936 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9d20074f40 0x7f9d2010de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:36:04.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:04.788+0000 7f9d25c00640 1 --2- 172.21.15.5:0/1662142936 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9d20074620 0x7f9d20074a00 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:36:04.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:04.788+0000 7f9d25c00640 1 -- 172.21.15.5:0/1662142936 >> 172.21.15.5:0/1662142936 conn(0x7f9d200fd8c0 msgr2=0x7f9d200ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:36:04.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:04.788+0000 7f9d25c00640 1 -- 172.21.15.5:0/1662142936 shutdown_connections 2024-09-10T09:36:04.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:04.788+0000 7f9d25c00640 1 -- 172.21.15.5:0/1662142936 wait complete. 2024-09-10T09:36:04.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:04.789+0000 7f9d25c00640 1 Processor -- start 2024-09-10T09:36:04.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:04.789+0000 7f9d25c00640 1 -- start start 2024-09-10T09:36:04.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:04.790+0000 7f9d25c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9d20074620 0x7f9d2019a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:36:04.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:04.790+0000 7f9d25c00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9d20074f40 0x7f9d2019aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:36:04.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:04.790+0000 7f9d25c00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9d2010e3c0 0x7f9d2019ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:36:04.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:04.790+0000 7f9d25c00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f9d201141c0 con 0x7f9d2010e3c0 2024-09-10T09:36:04.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:04.790+0000 7f9d25c00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f9d20114040 con 0x7f9d20074620 2024-09-10T09:36:04.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:04.790+0000 7f9d25c00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f9d20114340 con 0x7f9d20074f40 2024-09-10T09:36:04.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:04.790+0000 7f9d17e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9d20074f40 0x7f9d2019aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:36:04.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:04.790+0000 7f9d17e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9d20074f40 0x7f9d2019aba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:49188/0 (socket says 172.21.15.5:49188) 2024-09-10T09:36:04.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:04.790+0000 7f9d17e00640 1 -- 172.21.15.5:0/2960993299 learned_addr learned my addr 172.21.15.5:0/2960993299 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:36:04.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:04.790+0000 7f9d1f400640 1 --2- 172.21.15.5:0/2960993299 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9d2010e3c0 0x7f9d2019ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:36:04.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:04.790+0000 7f9d1ea00640 1 --2- 172.21.15.5:0/2960993299 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9d20074620 0x7f9d2019a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:36:04.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:04.791+0000 7f9d1f400640 1 -- 172.21.15.5:0/2960993299 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9d20074f40 msgr2=0x7f9d2019aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:36:04.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:04.791+0000 7f9d1f400640 1 --2- 172.21.15.5:0/2960993299 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9d20074f40 0x7f9d2019aba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:36:04.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:04.791+0000 7f9d1f400640 1 -- 172.21.15.5:0/2960993299 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9d20074620 msgr2=0x7f9d2019a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:36:04.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:04.791+0000 7f9d1f400640 1 --2- 172.21.15.5:0/2960993299 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9d20074620 0x7f9d2019a660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:36:04.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:04.791+0000 7f9d1f400640 1 -- 172.21.15.5:0/2960993299 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d2019f610 con 0x7f9d2010e3c0 2024-09-10T09:36:04.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:04.791+0000 7f9d17e00640 1 --2- 172.21.15.5:0/2960993299 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9d20074f40 0x7f9d2019aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:36:04.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:04.791+0000 7f9d1ea00640 1 --2- 172.21.15.5:0/2960993299 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9d20074620 0x7f9d2019a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:36:04.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:04.791+0000 7f9d1f400640 1 --2- 172.21.15.5:0/2960993299 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9d2010e3c0 0x7f9d2019ef30 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f9d18009fd0 tx=0x7f9d1800ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:36:04.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:04.791+0000 7f9d17400640 1 -- 172.21.15.5:0/2960993299 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d1800ce20 con 0x7f9d2010e3c0 2024-09-10T09:36:04.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:04.791+0000 7f9d17400640 1 -- 172.21.15.5:0/2960993299 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9d18005020 con 0x7f9d2010e3c0 2024-09-10T09:36:04.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:04.791+0000 7f9d17400640 1 -- 172.21.15.5:0/2960993299 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d18010720 con 0x7f9d2010e3c0 2024-09-10T09:36:04.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:04.791+0000 7f9d25c00640 1 -- 172.21.15.5:0/2960993299 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d2019f900 con 0x7f9d2010e3c0 2024-09-10T09:36:04.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:04.791+0000 7f9d25c00640 1 -- 172.21.15.5:0/2960993299 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d201a71e0 con 0x7f9d2010e3c0 2024-09-10T09:36:04.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:04.793+0000 7f9d25c00640 1 -- 172.21.15.5:0/2960993299 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9d20074a00 con 0x7f9d2010e3c0 2024-09-10T09:36:04.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:04.795+0000 7f9d17400640 1 -- 172.21.15.5:0/2960993299 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7f9d18005540 con 0x7f9d2010e3c0 2024-09-10T09:36:04.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:04.796+0000 7f9d17400640 1 --2- 172.21.15.5:0/2960993299 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f9d040a2c00 0x7f9d040a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:36:04.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:04.796+0000 7f9d17400640 1 -- 172.21.15.5:0/2960993299 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8802+0+0 (secure 0 0 0) 0x7f9d180cf7d0 con 0x7f9d2010e3c0 2024-09-10T09:36:04.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:04.796+0000 7f9d1ea00640 1 --2- 172.21.15.5:0/2960993299 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f9d040a2c00 0x7f9d040a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:36:04.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:04.797+0000 7f9d1ea00640 1 --2- 172.21.15.5:0/2960993299 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f9d040a2c00 0x7f9d040a50c0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f9d10000c00 tx=0x7f9d10031510 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:36:04.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:04.800+0000 7f9d17400640 1 -- 172.21.15.5:0/2960993299 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f9d1809c490 con 0x7f9d2010e3c0 2024-09-10T09:36:04.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:04.988+0000 7f9d25c00640 1 -- 172.21.15.5:0/2960993299 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9d2002d7c0 con 0x7f9d2010e3c0 2024-09-10T09:36:04.989 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:36:04 smithi005 ceph-mon[30350]: pgmap v321: 129 pgs: 129 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 0 op/s 2024-09-10T09:36:04.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:04.990+0000 7f9d17400640 1 -- 172.21.15.5:0/2960993299 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) ==== 74+0+24126 (secure 0 0 0) 0x7f9d1809bd90 con 0x7f9d2010e3c0 2024-09-10T09:36:04.991 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-10T09:36:04.991 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":92,"fsid":"bade4b50-6f56-11ef-bcea-c7b262605968","created":"2024-09-10T09:27:13.298498+0000","modified":"2024-09-10T09:35:40.676433+0000","last_up_change":"2024-09-10T09:34:30.730066+0000","last_in_change":"2024-09-10T09:34:07.340995+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T09:30:26.042255+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T09:35:24.461031+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T09:35:27.155349+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-10T09:35:35.121462+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"085749a8-e882-4377-92dd-34bb266c0f0b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":954535266},{"type":"v1","addr":"172.21.15.5:6803","nonce":954535266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":954535266},{"type":"v1","addr":"172.21.15.5:6805","nonce":954535266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":954535266},{"type":"v1","addr":"172.21.15.5:6809","nonce":954535266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":954535266},{"type":"v1","addr":"172.21.15.5:6807","nonce":954535266}]},"public_addr":"172.21.15.5:6803/954535266","cluster_addr":"172.21.15.5:6805/954535266","heartbeat_back_addr":"172.21.15.5:6809/954535266","heartbeat_front_addr":"172.21.15.5:6807/954535266","state":["exists","up"]},{"osd":1,"uuid":"0a76c795-2573-4a92-969a-2ec5dd65c094","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6800","nonce":733924375},{"type":"v1","addr":"172.21.15.102:6801","nonce":733924375}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6802","nonce":733924375},{"type":"v1","addr":"172.21.15.102:6803","nonce":733924375}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6806","nonce":733924375},{"type":"v1","addr":"172.21.15.102:6807","nonce":733924375}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6804","nonce":733924375},{"type":"v1","addr":"172.21.15.102:6805","nonce":733924375}]},"public_addr":"172.21.15.102:6801/733924375","cluster_addr":"172.21.15.102:6803/733924375","heartbeat_back_addr":"172.21.15.102:6807/733924375","heartbeat_front_addr":"172.21.15.102:6805/733924375","state":["exists","up"]},{"osd":2,"uuid":"0c98fcb8-60ab-4473-8995-1d9e97c577e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6800","nonce":2236204189},{"type":"v1","addr":"172.21.15.159:6801","nonce":2236204189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":2236204189},{"type":"v1","addr":"172.21.15.159:6803","nonce":2236204189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":2236204189},{"type":"v1","addr":"172.21.15.159:6807","nonce":2236204189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":2236204189},{"type":"v1","addr":"172.21.15.159:6805","nonce":2236204189}]},"public_addr":"172.21.15.159:6801/2236204189","cluster_addr":"172.21.15.159:6803/2236204189","heartbeat_back_addr":"172.21.15.159:6807/2236204189","heartbeat_front_addr":"172.21.15.159:6805/2236204189","state":["exists","up"]},{"osd":3,"uuid":"88eecdeb-5cee-4f78-a3f9-a66bf4bf772a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":2221468725},{"type":"v1","addr":"172.21.15.5:6811","nonce":2221468725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":2221468725},{"type":"v1","addr":"172.21.15.5:6813","nonce":2221468725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6816","nonce":2221468725},{"type":"v1","addr":"172.21.15.5:6817","nonce":2221468725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":2221468725},{"type":"v1","addr":"172.21.15.5:6815","nonce":2221468725}]},"public_addr":"172.21.15.5:6811/2221468725","cluster_addr":"172.21.15.5:6813/2221468725","heartbeat_back_addr":"172.21.15.5:6817/2221468725","heartbeat_front_addr":"172.21.15.5:6815/2221468725","state":["exists","up"]},{"osd":4,"uuid":"f2e7314d-7938-4668-9011-45c175f66b48","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6808","nonce":594465753},{"type":"v1","addr":"172.21.15.102:6809","nonce":594465753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6810","nonce":594465753},{"type":"v1","addr":"172.21.15.102:6811","nonce":594465753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6814","nonce":594465753},{"type":"v1","addr":"172.21.15.102:6815","nonce":594465753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6812","nonce":594465753},{"type":"v1","addr":"172.21.15.102:6813","nonce":594465753}]},"public_addr":"172.21.15.102:6809/594465753","cluster_addr":"172.21.15.102:6811/594465753","heartbeat_back_addr":"172.21.15.102:6815/594465753","heartbeat_front_addr":"172.21.15.102:6813/594465753","state":["exists","up"]},{"osd":5,"uuid":"0c0366d3-658b-4bf0-ba17-aff28a02e948","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":3230280402},{"type":"v1","addr":"172.21.15.159:6809","nonce":3230280402}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":3230280402},{"type":"v1","addr":"172.21.15.159:6811","nonce":3230280402}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":3230280402},{"type":"v1","addr":"172.21.15.159:6815","nonce":3230280402}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":3230280402},{"type":"v1","addr":"172.21.15.159:6813","nonce":3230280402}]},"public_addr":"172.21.15.159:6809/3230280402","cluster_addr":"172.21.15.159:6811/3230280402","heartbeat_back_addr":"172.21.15.159:6815/3230280402","heartbeat_front_addr":"172.21.15.159:6813/3230280402","state":["exists","up"]},{"osd":6,"uuid":"682670a6-51d5-46e8-af6e-dc2d8c1cf407","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":1907651140},{"type":"v1","addr":"172.21.15.5:6819","nonce":1907651140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":1907651140},{"type":"v1","addr":"172.21.15.5:6821","nonce":1907651140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":1907651140},{"type":"v1","addr":"172.21.15.5:6825","nonce":1907651140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":1907651140},{"type":"v1","addr":"172.21.15.5:6823","nonce":1907651140}]},"public_addr":"172.21.15.5:6819/1907651140","cluster_addr":"172.21.15.5:6821/1907651140","heartbeat_back_addr":"172.21.15.5:6825/1907651140","heartbeat_front_addr":"172.21.15.5:6823/1907651140","state":["exists","up"]},{"osd":7,"uuid":"8ae0f4a9-99e7-4563-aa2a-15169684a8b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6816","nonce":692040360},{"type":"v1","addr":"172.21.15.102:6817","nonce":692040360}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6818","nonce":692040360},{"type":"v1","addr":"172.21.15.102:6819","nonce":692040360}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6822","nonce":692040360},{"type":"v1","addr":"172.21.15.102:6823","nonce":692040360}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6820","nonce":692040360},{"type":"v1","addr":"172.21.15.102:6821","nonce":692040360}]},"public_addr":"172.21.15.102:6817/692040360","cluster_addr":"172.21.15.102:6819/692040360","heartbeat_back_addr":"172.21.15.102:6823/692040360","heartbeat_front_addr":"172.21.15.102:6821/692040360","state":["exists","up"]},{"osd":8,"uuid":"e37c845e-ced9-4413-9bbe-bc090537c2bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":2107291799},{"type":"v1","addr":"172.21.15.159:6817","nonce":2107291799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":2107291799},{"type":"v1","addr":"172.21.15.159:6819","nonce":2107291799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":2107291799},{"type":"v1","addr":"172.21.15.159:6823","nonce":2107291799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":2107291799},{"type":"v1","addr":"172.21.15.159:6821","nonce":2107291799}]},"public_addr":"172.21.15.159:6817/2107291799","cluster_addr":"172.21.15.159:6819/2107291799","heartbeat_back_addr":"172.21.15.159:6823/2107291799","heartbeat_front_addr":"172.21.15.159:6821/2107291799","state":["exists","up"]},{"osd":9,"uuid":"91a5b6cc-08da-47be-af12-d0896a849309","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":902431281},{"type":"v1","addr":"172.21.15.5:6827","nonce":902431281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":902431281},{"type":"v1","addr":"172.21.15.5:6829","nonce":902431281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":902431281},{"type":"v1","addr":"172.21.15.5:6833","nonce":902431281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":902431281},{"type":"v1","addr":"172.21.15.5:6831","nonce":902431281}]},"public_addr":"172.21.15.5:6827/902431281","cluster_addr":"172.21.15.5:6829/902431281","heartbeat_back_addr":"172.21.15.5:6833/902431281","heartbeat_front_addr":"172.21.15.5:6831/902431281","state":["exists","up"]},{"osd":10,"uuid":"2211030c-b00e-402b-b4d7-c587eb29a490","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6824","nonce":2853806211},{"type":"v1","addr":"172.21.15.102:6825","nonce":2853806211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6826","nonce":2853806211},{"type":"v1","addr":"172.21.15.102:6827","nonce":2853806211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6830","nonce":2853806211},{"type":"v1","addr":"172.21.15.102:6831","nonce":2853806211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6828","nonce":2853806211},{"type":"v1","addr":"172.21.15.102:6829","nonce":2853806211}]},"public_addr":"172.21.15.102:6825/2853806211","cluster_addr":"172.21.15.102:6827/2853806211","heartbeat_back_addr":"172.21.15.102:6831/2853806211","heartbeat_front_addr":"172.21.15.102:6829/2853806211","state":["exists","up"]},{"osd":11,"uuid":"4e9699ec-8ff2-4005-9d84-af5600b3a2ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6824","nonce":969941218},{"type":"v1","addr":"172.21.15.159:6825","nonce":969941218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":969941218},{"type":"v1","addr":"172.21.15.159:6827","nonce":969941218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6830","nonce":969941218},{"type":"v1","addr":"172.21.15.159:6831","nonce":969941218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":969941218},{"type":"v1","addr":"172.21.15.159:6829","nonce":969941218}]},"public_addr":"172.21.15.159:6825/969941218","cluster_addr":"172.21.15.159:6827/969941218","heartbeat_back_addr":"172.21.15.159:6831/969941218","heartbeat_front_addr":"172.21.15.159:6829/969941218","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:29:34.416490+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:30:00.246101+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:30:24.300464+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:30:54.663623+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:31:21.426292+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:31:46.738914+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:32:16.196571+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:32:42.549243+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:33:08.178651+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:33:37.477911+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:34:03.114026+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:34:28.891924+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.5:6801/1527422686":"2024-09-11T09:28:05.436742+0000","172.21.15.5:0/2276144345":"2024-09-11T09:28:05.436742+0000","172.21.15.5:0/412040104":"2024-09-11T09:28:05.436742+0000","172.21.15.5:6800/1527422686":"2024-09-11T09:28:05.436742+0000","172.21.15.5:0/1856581170":"2024-09-11T09:28:05.436742+0000","172.21.15.5:0/2847668298":"2024-09-11T09:27:37.778500+0000","172.21.15.5:6801/14425913":"2024-09-11T09:27:37.778500+0000","172.21.15.5:0/166910782":"2024-09-11T09:27:37.778500+0000","172.21.15.5:6800/14425913":"2024-09-11T09:27:37.778500+0000","172.21.15.5:0/1711858179":"2024-09-11T09:27:37.778500+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T09:36:04.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:04.992+0000 7f9d25c00640 1 -- 172.21.15.5:0/2960993299 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f9d040a2c00 msgr2=0x7f9d040a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:36:04.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:04.992+0000 7f9d25c00640 1 --2- 172.21.15.5:0/2960993299 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f9d040a2c00 0x7f9d040a50c0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f9d10000c00 tx=0x7f9d10031510 comp rx=0 tx=0).stop 2024-09-10T09:36:04.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:04.992+0000 7f9d25c00640 1 -- 172.21.15.5:0/2960993299 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9d2010e3c0 msgr2=0x7f9d2019ef30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:36:04.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:04.992+0000 7f9d25c00640 1 --2- 172.21.15.5:0/2960993299 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9d2010e3c0 0x7f9d2019ef30 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f9d18009fd0 tx=0x7f9d1800ef10 comp rx=0 tx=0).stop 2024-09-10T09:36:04.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:04.992+0000 7f9d25c00640 1 -- 172.21.15.5:0/2960993299 shutdown_connections 2024-09-10T09:36:04.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:04.992+0000 7f9d25c00640 1 --2- 172.21.15.5:0/2960993299 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f9d040a2c00 0x7f9d040a50c0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:36:04.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:04.992+0000 7f9d25c00640 1 --2- 172.21.15.5:0/2960993299 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9d2010e3c0 0x7f9d2019ef30 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:36:04.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:04.992+0000 7f9d25c00640 1 --2- 172.21.15.5:0/2960993299 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9d20074f40 0x7f9d2019aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:36:04.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:04.992+0000 7f9d25c00640 1 --2- 172.21.15.5:0/2960993299 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9d20074620 0x7f9d2019a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:36:04.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:04.992+0000 7f9d25c00640 1 -- 172.21.15.5:0/2960993299 >> 172.21.15.5:0/2960993299 conn(0x7f9d200fd8c0 msgr2=0x7f9d200fe520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:36:04.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:04.992+0000 7f9d25c00640 1 -- 172.21.15.5:0/2960993299 shutdown_connections 2024-09-10T09:36:04.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:04.992+0000 7f9d25c00640 1 -- 172.21.15.5:0/2960993299 wait complete. 2024-09-10T09:36:05.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:36:04 smithi159 ceph-mon[35513]: pgmap v321: 129 pgs: 129 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 0 op/s 2024-09-10T09:36:05.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:36:04 smithi102 ceph-mon[35442]: pgmap v321: 129 pgs: 129 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 0 op/s 2024-09-10T09:36:05.777 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-10T09:36:06.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:36:05 smithi005 ceph-mon[30350]: from='client.? 172.21.15.5:0/2960993299' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T09:36:06.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:36:05 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 8]}]: dispatch 2024-09-10T09:36:06.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:36:05 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [3, 9]}]: dispatch 2024-09-10T09:36:06.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:36:05 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 6]}]: dispatch 2024-09-10T09:36:06.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:36:05 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1e", "id": [3, 9, 5, 7]}]: dispatch 2024-09-10T09:36:06.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:36:05 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [4, 8]}]: dispatch 2024-09-10T09:36:06.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:36:05 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.8", "id": [5, 8]}]: dispatch 2024-09-10T09:36:06.057 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:36:05 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.c", "id": [9, 10]}]: dispatch 2024-09-10T09:36:06.057 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:36:05 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [9, 8]}]: dispatch 2024-09-10T09:36:06.057 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:36:05 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [9, 8]}]: dispatch 2024-09-10T09:36:06.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:36:05 smithi159 ceph-mon[35513]: from='client.? 172.21.15.5:0/2960993299' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T09:36:06.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:36:05 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 8]}]: dispatch 2024-09-10T09:36:06.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:36:05 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [3, 9]}]: dispatch 2024-09-10T09:36:06.101 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:36:05 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 6]}]: dispatch 2024-09-10T09:36:06.101 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:36:05 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1e", "id": [3, 9, 5, 7]}]: dispatch 2024-09-10T09:36:06.101 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:36:05 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [4, 8]}]: dispatch 2024-09-10T09:36:06.101 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:36:05 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.8", "id": [5, 8]}]: dispatch 2024-09-10T09:36:06.101 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:36:05 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.c", "id": [9, 10]}]: dispatch 2024-09-10T09:36:06.101 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:36:05 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [9, 8]}]: dispatch 2024-09-10T09:36:06.101 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:36:05 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [9, 8]}]: dispatch 2024-09-10T09:36:06.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:36:05 smithi102 ceph-mon[35442]: from='client.? 172.21.15.5:0/2960993299' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T09:36:06.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:36:05 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 8]}]: dispatch 2024-09-10T09:36:06.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:36:05 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [3, 9]}]: dispatch 2024-09-10T09:36:06.135 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:36:05 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 6]}]: dispatch 2024-09-10T09:36:06.135 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:36:05 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1e", "id": [3, 9, 5, 7]}]: dispatch 2024-09-10T09:36:06.135 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:36:05 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [4, 8]}]: dispatch 2024-09-10T09:36:06.135 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:36:05 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.8", "id": [5, 8]}]: dispatch 2024-09-10T09:36:06.135 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:36:05 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.c", "id": [9, 10]}]: dispatch 2024-09-10T09:36:06.135 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:36:05 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [9, 8]}]: dispatch 2024-09-10T09:36:06.135 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:36:05 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [9, 8]}]: dispatch 2024-09-10T09:36:06.157 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:36:06.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:06.717+0000 7f927cc00640 1 -- 172.21.15.5:0/725209663 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9278074510 msgr2=0x7f92780748f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:36:06.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:06.717+0000 7f927cc00640 1 --2- 172.21.15.5:0/725209663 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9278074510 0x7f92780748f0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f9268009900 tx=0x7f926802a480 comp rx=0 tx=0).stop 2024-09-10T09:36:06.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:06.717+0000 7f927cc00640 1 -- 172.21.15.5:0/725209663 shutdown_connections 2024-09-10T09:36:06.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:06.717+0000 7f927cc00640 1 --2- 172.21.15.5:0/725209663 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f927810e280 0x7f9278110670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:36:06.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:06.717+0000 7f927cc00640 1 --2- 172.21.15.5:0/725209663 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9278074e30 0x7f927810dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:36:06.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:06.717+0000 7f927cc00640 1 --2- 172.21.15.5:0/725209663 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9278074510 0x7f92780748f0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:36:06.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:06.717+0000 7f927cc00640 1 -- 172.21.15.5:0/725209663 >> 172.21.15.5:0/725209663 conn(0x7f92780fd740 msgr2=0x7f92780ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:36:06.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:06.717+0000 7f927cc00640 1 -- 172.21.15.5:0/725209663 shutdown_connections 2024-09-10T09:36:06.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:06.717+0000 7f927cc00640 1 -- 172.21.15.5:0/725209663 wait complete. 2024-09-10T09:36:06.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:06.718+0000 7f927cc00640 1 Processor -- start 2024-09-10T09:36:06.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:06.718+0000 7f927cc00640 1 -- start start 2024-09-10T09:36:06.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:06.718+0000 7f927cc00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9278074510 0x7f927819a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:36:06.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:06.718+0000 7f927cc00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9278074e30 0x7f927819aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:36:06.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:06.719+0000 7f927cc00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f927810e280 0x7f927819ee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:36:06.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:06.719+0000 7f927cc00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f9278113fb0 con 0x7f927810e280 2024-09-10T09:36:06.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:06.719+0000 7f927cc00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f9278113e30 con 0x7f9278074510 2024-09-10T09:36:06.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:06.719+0000 7f927cc00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f9278114130 con 0x7f9278074e30 2024-09-10T09:36:06.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:06.719+0000 7f9277e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f927810e280 0x7f927819ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:36:06.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:06.719+0000 7f9277e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f927810e280 0x7f927819ee10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39686/0 (socket says 172.21.15.5:39686) 2024-09-10T09:36:06.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:06.719+0000 7f9277e00640 1 -- 172.21.15.5:0/1055685689 learned_addr learned my addr 172.21.15.5:0/1055685689 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:36:06.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:06.719+0000 7f9277400640 1 --2- 172.21.15.5:0/1055685689 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9278074510 0x7f927819a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:36:06.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:06.719+0000 7f926ea00640 1 --2- 172.21.15.5:0/1055685689 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9278074e30 0x7f927819aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:36:06.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:06.719+0000 7f9277e00640 1 -- 172.21.15.5:0/1055685689 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9278074e30 msgr2=0x7f927819aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:36:06.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:06.719+0000 7f9277e00640 1 --2- 172.21.15.5:0/1055685689 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9278074e30 0x7f927819aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:36:06.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:06.719+0000 7f9277e00640 1 -- 172.21.15.5:0/1055685689 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9278074510 msgr2=0x7f927819a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:36:06.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:06.719+0000 7f9277e00640 1 --2- 172.21.15.5:0/1055685689 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9278074510 0x7f927819a540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:36:06.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:06.719+0000 7f9277e00640 1 -- 172.21.15.5:0/1055685689 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f927819f4f0 con 0x7f927810e280 2024-09-10T09:36:06.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:06.719+0000 7f926ea00640 1 --2- 172.21.15.5:0/1055685689 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9278074e30 0x7f927819aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:36:06.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:06.720+0000 7f9277e00640 1 --2- 172.21.15.5:0/1055685689 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f927810e280 0x7f927819ee10 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f927000dac0 tx=0x7f927000df80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:36:06.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:06.720+0000 7f9274c00640 1 -- 172.21.15.5:0/1055685689 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9270017070 con 0x7f927810e280 2024-09-10T09:36:06.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:06.720+0000 7f9274c00640 1 -- 172.21.15.5:0/1055685689 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f927000be30 con 0x7f927810e280 2024-09-10T09:36:06.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:06.720+0000 7f927cc00640 1 -- 172.21.15.5:0/1055685689 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f927819f7e0 con 0x7f927810e280 2024-09-10T09:36:06.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:06.720+0000 7f9274c00640 1 -- 172.21.15.5:0/1055685689 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f927000f890 con 0x7f927810e280 2024-09-10T09:36:06.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:06.720+0000 7f927cc00640 1 -- 172.21.15.5:0/1055685689 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92781a70c0 con 0x7f927810e280 2024-09-10T09:36:06.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:06.721+0000 7f927cc00640 1 -- 172.21.15.5:0/1055685689 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f92780748f0 con 0x7f927810e280 2024-09-10T09:36:06.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:06.723+0000 7f9274c00640 1 -- 172.21.15.5:0/1055685689 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7f927000fab0 con 0x7f927810e280 2024-09-10T09:36:06.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:06.725+0000 7f9274c00640 1 --2- 172.21.15.5:0/1055685689 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f925c0a2b80 0x7f925c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:36:06.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:06.725+0000 7f9274c00640 1 -- 172.21.15.5:0/1055685689 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9071+0+0 (secure 0 0 0) 0x7f92700cf2b0 con 0x7f927810e280 2024-09-10T09:36:06.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:06.726+0000 7f9277400640 1 --2- 172.21.15.5:0/1055685689 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f925c0a2b80 0x7f925c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:36:06.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:06.726+0000 7f9277400640 1 --2- 172.21.15.5:0/1055685689 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f925c0a2b80 0x7f925c0a5040 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f9268009230 tx=0x7f92680058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:36:06.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:06.729+0000 7f9274c00640 1 -- 172.21.15.5:0/1055685689 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f927009be70 con 0x7f927810e280 2024-09-10T09:36:06.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:06.927+0000 7f927cc00640 1 -- 172.21.15.5:0/1055685689 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9278109460 con 0x7f927810e280 2024-09-10T09:36:06.928 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:36:06 smithi005 ceph-mon[30350]: pgmap v322: 129 pgs: 129 active+clean; 581 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-09-10T09:36:06.928 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:36:06 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 8]}]': finished 2024-09-10T09:36:06.928 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:36:06 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [3, 9]}]': finished 2024-09-10T09:36:06.928 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:36:06 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 6]}]': finished 2024-09-10T09:36:06.928 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:36:06 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1e", "id": [3, 9, 5, 7]}]': finished 2024-09-10T09:36:06.929 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:36:06 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [4, 8]}]': finished 2024-09-10T09:36:06.929 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:36:06 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.8", "id": [5, 8]}]': finished 2024-09-10T09:36:06.929 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:36:06 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.c", "id": [9, 10]}]': finished 2024-09-10T09:36:06.929 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:36:06 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [9, 8]}]': finished 2024-09-10T09:36:06.929 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:36:06 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [9, 8]}]': finished 2024-09-10T09:36:06.929 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:36:06 smithi005 ceph-mon[30350]: osdmap e93: 12 total, 12 up, 12 in 2024-09-10T09:36:06.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:06.928+0000 7f9274c00640 1 -- 172.21.15.5:0/1055685689 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) ==== 74+0+24566 (secure 0 0 0) 0x7f927009b770 con 0x7f927810e280 2024-09-10T09:36:06.930 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-10T09:36:06.933 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":94,"fsid":"bade4b50-6f56-11ef-bcea-c7b262605968","created":"2024-09-10T09:27:13.298498+0000","modified":"2024-09-10T09:36:06.755744+0000","last_up_change":"2024-09-10T09:34:30.730066+0000","last_in_change":"2024-09-10T09:34:07.340995+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T09:30:26.042255+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T09:35:24.461031+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T09:35:27.155349+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-10T09:35:35.121462+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"085749a8-e882-4377-92dd-34bb266c0f0b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":954535266},{"type":"v1","addr":"172.21.15.5:6803","nonce":954535266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":954535266},{"type":"v1","addr":"172.21.15.5:6805","nonce":954535266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":954535266},{"type":"v1","addr":"172.21.15.5:6809","nonce":954535266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":954535266},{"type":"v1","addr":"172.21.15.5:6807","nonce":954535266}]},"public_addr":"172.21.15.5:6803/954535266","cluster_addr":"172.21.15.5:6805/954535266","heartbeat_back_addr":"172.21.15.5:6809/954535266","heartbeat_front_addr":"172.21.15.5:6807/954535266","state":["exists","up"]},{"osd":1,"uuid":"0a76c795-2573-4a92-969a-2ec5dd65c094","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6800","nonce":733924375},{"type":"v1","addr":"172.21.15.102:6801","nonce":733924375}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6802","nonce":733924375},{"type":"v1","addr":"172.21.15.102:6803","nonce":733924375}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6806","nonce":733924375},{"type":"v1","addr":"172.21.15.102:6807","nonce":733924375}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6804","nonce":733924375},{"type":"v1","addr":"172.21.15.102:6805","nonce":733924375}]},"public_addr":"172.21.15.102:6801/733924375","cluster_addr":"172.21.15.102:6803/733924375","heartbeat_back_addr":"172.21.15.102:6807/733924375","heartbeat_front_addr":"172.21.15.102:6805/733924375","state":["exists","up"]},{"osd":2,"uuid":"0c98fcb8-60ab-4473-8995-1d9e97c577e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6800","nonce":2236204189},{"type":"v1","addr":"172.21.15.159:6801","nonce":2236204189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":2236204189},{"type":"v1","addr":"172.21.15.159:6803","nonce":2236204189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":2236204189},{"type":"v1","addr":"172.21.15.159:6807","nonce":2236204189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":2236204189},{"type":"v1","addr":"172.21.15.159:6805","nonce":2236204189}]},"public_addr":"172.21.15.159:6801/2236204189","cluster_addr":"172.21.15.159:6803/2236204189","heartbeat_back_addr":"172.21.15.159:6807/2236204189","heartbeat_front_addr":"172.21.15.159:6805/2236204189","state":["exists","up"]},{"osd":3,"uuid":"88eecdeb-5cee-4f78-a3f9-a66bf4bf772a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":2221468725},{"type":"v1","addr":"172.21.15.5:6811","nonce":2221468725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":2221468725},{"type":"v1","addr":"172.21.15.5:6813","nonce":2221468725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6816","nonce":2221468725},{"type":"v1","addr":"172.21.15.5:6817","nonce":2221468725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":2221468725},{"type":"v1","addr":"172.21.15.5:6815","nonce":2221468725}]},"public_addr":"172.21.15.5:6811/2221468725","cluster_addr":"172.21.15.5:6813/2221468725","heartbeat_back_addr":"172.21.15.5:6817/2221468725","heartbeat_front_addr":"172.21.15.5:6815/2221468725","state":["exists","up"]},{"osd":4,"uuid":"f2e7314d-7938-4668-9011-45c175f66b48","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6808","nonce":594465753},{"type":"v1","addr":"172.21.15.102:6809","nonce":594465753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6810","nonce":594465753},{"type":"v1","addr":"172.21.15.102:6811","nonce":594465753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6814","nonce":594465753},{"type":"v1","addr":"172.21.15.102:6815","nonce":594465753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6812","nonce":594465753},{"type":"v1","addr":"172.21.15.102:6813","nonce":594465753}]},"public_addr":"172.21.15.102:6809/594465753","cluster_addr":"172.21.15.102:6811/594465753","heartbeat_back_addr":"172.21.15.102:6815/594465753","heartbeat_front_addr":"172.21.15.102:6813/594465753","state":["exists","up"]},{"osd":5,"uuid":"0c0366d3-658b-4bf0-ba17-aff28a02e948","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":3230280402},{"type":"v1","addr":"172.21.15.159:6809","nonce":3230280402}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":3230280402},{"type":"v1","addr":"172.21.15.159:6811","nonce":3230280402}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":3230280402},{"type":"v1","addr":"172.21.15.159:6815","nonce":3230280402}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":3230280402},{"type":"v1","addr":"172.21.15.159:6813","nonce":3230280402}]},"public_addr":"172.21.15.159:6809/3230280402","cluster_addr":"172.21.15.159:6811/3230280402","heartbeat_back_addr":"172.21.15.159:6815/3230280402","heartbeat_front_addr":"172.21.15.159:6813/3230280402","state":["exists","up"]},{"osd":6,"uuid":"682670a6-51d5-46e8-af6e-dc2d8c1cf407","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":1907651140},{"type":"v1","addr":"172.21.15.5:6819","nonce":1907651140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":1907651140},{"type":"v1","addr":"172.21.15.5:6821","nonce":1907651140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":1907651140},{"type":"v1","addr":"172.21.15.5:6825","nonce":1907651140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":1907651140},{"type":"v1","addr":"172.21.15.5:6823","nonce":1907651140}]},"public_addr":"172.21.15.5:6819/1907651140","cluster_addr":"172.21.15.5:6821/1907651140","heartbeat_back_addr":"172.21.15.5:6825/1907651140","heartbeat_front_addr":"172.21.15.5:6823/1907651140","state":["exists","up"]},{"osd":7,"uuid":"8ae0f4a9-99e7-4563-aa2a-15169684a8b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6816","nonce":692040360},{"type":"v1","addr":"172.21.15.102:6817","nonce":692040360}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6818","nonce":692040360},{"type":"v1","addr":"172.21.15.102:6819","nonce":692040360}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6822","nonce":692040360},{"type":"v1","addr":"172.21.15.102:6823","nonce":692040360}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6820","nonce":692040360},{"type":"v1","addr":"172.21.15.102:6821","nonce":692040360}]},"public_addr":"172.21.15.102:6817/692040360","cluster_addr":"172.21.15.102:6819/692040360","heartbeat_back_addr":"172.21.15.102:6823/692040360","heartbeat_front_addr":"172.21.15.102:6821/692040360","state":["exists","up"]},{"osd":8,"uuid":"e37c845e-ced9-4413-9bbe-bc090537c2bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":2107291799},{"type":"v1","addr":"172.21.15.159:6817","nonce":2107291799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":2107291799},{"type":"v1","addr":"172.21.15.159:6819","nonce":2107291799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":2107291799},{"type":"v1","addr":"172.21.15.159:6823","nonce":2107291799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":2107291799},{"type":"v1","addr":"172.21.15.159:6821","nonce":2107291799}]},"public_addr":"172.21.15.159:6817/2107291799","cluster_addr":"172.21.15.159:6819/2107291799","heartbeat_back_addr":"172.21.15.159:6823/2107291799","heartbeat_front_addr":"172.21.15.159:6821/2107291799","state":["exists","up"]},{"osd":9,"uuid":"91a5b6cc-08da-47be-af12-d0896a849309","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":902431281},{"type":"v1","addr":"172.21.15.5:6827","nonce":902431281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":902431281},{"type":"v1","addr":"172.21.15.5:6829","nonce":902431281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":902431281},{"type":"v1","addr":"172.21.15.5:6833","nonce":902431281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":902431281},{"type":"v1","addr":"172.21.15.5:6831","nonce":902431281}]},"public_addr":"172.21.15.5:6827/902431281","cluster_addr":"172.21.15.5:6829/902431281","heartbeat_back_addr":"172.21.15.5:6833/902431281","heartbeat_front_addr":"172.21.15.5:6831/902431281","state":["exists","up"]},{"osd":10,"uuid":"2211030c-b00e-402b-b4d7-c587eb29a490","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6824","nonce":2853806211},{"type":"v1","addr":"172.21.15.102:6825","nonce":2853806211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6826","nonce":2853806211},{"type":"v1","addr":"172.21.15.102:6827","nonce":2853806211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6830","nonce":2853806211},{"type":"v1","addr":"172.21.15.102:6831","nonce":2853806211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6828","nonce":2853806211},{"type":"v1","addr":"172.21.15.102:6829","nonce":2853806211}]},"public_addr":"172.21.15.102:6825/2853806211","cluster_addr":"172.21.15.102:6827/2853806211","heartbeat_back_addr":"172.21.15.102:6831/2853806211","heartbeat_front_addr":"172.21.15.102:6829/2853806211","state":["exists","up"]},{"osd":11,"uuid":"4e9699ec-8ff2-4005-9d84-af5600b3a2ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6824","nonce":969941218},{"type":"v1","addr":"172.21.15.159:6825","nonce":969941218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":969941218},{"type":"v1","addr":"172.21.15.159:6827","nonce":969941218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6830","nonce":969941218},{"type":"v1","addr":"172.21.15.159:6831","nonce":969941218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":969941218},{"type":"v1","addr":"172.21.15.159:6829","nonce":969941218}]},"public_addr":"172.21.15.159:6825/969941218","cluster_addr":"172.21.15.159:6827/969941218","heartbeat_back_addr":"172.21.15.159:6831/969941218","heartbeat_front_addr":"172.21.15.159:6829/969941218","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:29:34.416490+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:30:00.246101+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:30:24.300464+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:30:54.663623+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:31:21.426292+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:31:46.738914+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:32:16.196571+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:32:42.549243+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:33:08.178651+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:33:37.477911+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:34:03.114026+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:34:28.891924+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.0","mappings":[{"from":4,"to":8}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":6}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9},{"from":5,"to":7}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]},{"pgid":"4.8","mappings":[{"from":5,"to":8}]},{"pgid":"4.c","mappings":[{"from":9,"to":10}]},{"pgid":"4.18","mappings":[{"from":9,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.5:6801/1527422686":"2024-09-11T09:28:05.436742+0000","172.21.15.5:0/2276144345":"2024-09-11T09:28:05.436742+0000","172.21.15.5:0/412040104":"2024-09-11T09:28:05.436742+0000","172.21.15.5:6800/1527422686":"2024-09-11T09:28:05.436742+0000","172.21.15.5:0/1856581170":"2024-09-11T09:28:05.436742+0000","172.21.15.5:0/2847668298":"2024-09-11T09:27:37.778500+0000","172.21.15.5:6801/14425913":"2024-09-11T09:27:37.778500+0000","172.21.15.5:0/166910782":"2024-09-11T09:27:37.778500+0000","172.21.15.5:6800/14425913":"2024-09-11T09:27:37.778500+0000","172.21.15.5:0/1711858179":"2024-09-11T09:27:37.778500+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T09:36:06.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:06.931+0000 7f927cc00640 1 -- 172.21.15.5:0/1055685689 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f925c0a2b80 msgr2=0x7f925c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:36:06.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:06.931+0000 7f927cc00640 1 --2- 172.21.15.5:0/1055685689 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f925c0a2b80 0x7f925c0a5040 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f9268009230 tx=0x7f92680058d0 comp rx=0 tx=0).stop 2024-09-10T09:36:06.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:06.931+0000 7f927cc00640 1 -- 172.21.15.5:0/1055685689 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f927810e280 msgr2=0x7f927819ee10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:36:06.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:06.931+0000 7f927cc00640 1 --2- 172.21.15.5:0/1055685689 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f927810e280 0x7f927819ee10 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f927000dac0 tx=0x7f927000df80 comp rx=0 tx=0).stop 2024-09-10T09:36:06.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:06.931+0000 7f927cc00640 1 -- 172.21.15.5:0/1055685689 shutdown_connections 2024-09-10T09:36:06.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:06.931+0000 7f927cc00640 1 --2- 172.21.15.5:0/1055685689 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f925c0a2b80 0x7f925c0a5040 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:36:06.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:06.931+0000 7f927cc00640 1 --2- 172.21.15.5:0/1055685689 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f927810e280 0x7f927819ee10 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:36:06.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:06.931+0000 7f927cc00640 1 --2- 172.21.15.5:0/1055685689 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9278074e30 0x7f927819aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:36:06.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:06.931+0000 7f927cc00640 1 --2- 172.21.15.5:0/1055685689 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9278074510 0x7f927819a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:36:06.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:06.931+0000 7f927cc00640 1 -- 172.21.15.5:0/1055685689 >> 172.21.15.5:0/1055685689 conn(0x7f92780fd740 msgr2=0x7f92780fef70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:36:06.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:06.931+0000 7f927cc00640 1 -- 172.21.15.5:0/1055685689 shutdown_connections 2024-09-10T09:36:06.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:06.931+0000 7f927cc00640 1 -- 172.21.15.5:0/1055685689 wait complete. 2024-09-10T09:36:07.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:36:06 smithi159 ceph-mon[35513]: pgmap v322: 129 pgs: 129 active+clean; 581 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-09-10T09:36:07.101 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:36:06 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 8]}]': finished 2024-09-10T09:36:07.101 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:36:06 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [3, 9]}]': finished 2024-09-10T09:36:07.101 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:36:06 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 6]}]': finished 2024-09-10T09:36:07.101 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:36:06 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1e", "id": [3, 9, 5, 7]}]': finished 2024-09-10T09:36:07.101 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:36:06 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [4, 8]}]': finished 2024-09-10T09:36:07.101 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:36:06 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.8", "id": [5, 8]}]': finished 2024-09-10T09:36:07.101 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:36:06 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.c", "id": [9, 10]}]': finished 2024-09-10T09:36:07.101 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:36:06 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [9, 8]}]': finished 2024-09-10T09:36:07.101 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:36:06 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [9, 8]}]': finished 2024-09-10T09:36:07.101 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:36:06 smithi159 ceph-mon[35513]: osdmap e93: 12 total, 12 up, 12 in 2024-09-10T09:36:07.135 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:36:06 smithi102 ceph-mon[35442]: pgmap v322: 129 pgs: 129 active+clean; 581 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-09-10T09:36:07.135 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:36:06 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 8]}]': finished 2024-09-10T09:36:07.135 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:36:06 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [3, 9]}]': finished 2024-09-10T09:36:07.135 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:36:06 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 6]}]': finished 2024-09-10T09:36:07.135 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:36:06 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1e", "id": [3, 9, 5, 7]}]': finished 2024-09-10T09:36:07.135 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:36:06 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [4, 8]}]': finished 2024-09-10T09:36:07.135 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:36:06 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.8", "id": [5, 8]}]': finished 2024-09-10T09:36:07.135 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:36:06 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.c", "id": [9, 10]}]': finished 2024-09-10T09:36:07.135 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:36:06 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [9, 8]}]': finished 2024-09-10T09:36:07.135 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:36:06 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [9, 8]}]': finished 2024-09-10T09:36:07.136 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:36:06 smithi102 ceph-mon[35442]: osdmap e93: 12 total, 12 up, 12 in 2024-09-10T09:36:07.713 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph fs set cephfs standby_count_wanted 0 2024-09-10T09:36:08.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:36:07 smithi005 ceph-mon[30350]: osdmap e94: 12 total, 12 up, 12 in 2024-09-10T09:36:08.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:36:07 smithi005 ceph-mon[30350]: from='client.? 172.21.15.5:0/1055685689' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T09:36:08.092 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:36:08.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:36:07 smithi159 ceph-mon[35513]: osdmap e94: 12 total, 12 up, 12 in 2024-09-10T09:36:08.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:36:07 smithi159 ceph-mon[35513]: from='client.? 172.21.15.5:0/1055685689' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T09:36:08.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:36:07 smithi102 ceph-mon[35442]: osdmap e94: 12 total, 12 up, 12 in 2024-09-10T09:36:08.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:36:07 smithi102 ceph-mon[35442]: from='client.? 172.21.15.5:0/1055685689' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T09:36:08.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:08.676+0000 7fe069400640 1 -- 172.21.15.5:0/2862193112 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe0641049a0 msgr2=0x7fe0641089f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:36:08.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:08.676+0000 7fe069400640 1 --2- 172.21.15.5:0/2862193112 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe0641049a0 0x7fe0641089f0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fe04c009900 tx=0x7fe04c02a3f0 comp rx=0 tx=0).stop 2024-09-10T09:36:08.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:08.676+0000 7fe069400640 1 -- 172.21.15.5:0/2862193112 shutdown_connections 2024-09-10T09:36:08.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:08.676+0000 7fe069400640 1 --2- 172.21.15.5:0/2862193112 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe064109120 0x7fe064110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:36:08.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:08.676+0000 7fe069400640 1 --2- 172.21.15.5:0/2862193112 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe0641049a0 0x7fe0641089f0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:36:08.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:08.676+0000 7fe069400640 1 --2- 172.21.15.5:0/2862193112 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe064103ff0 0x7fe0641043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:36:08.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:08.676+0000 7fe069400640 1 -- 172.21.15.5:0/2862193112 >> 172.21.15.5:0/2862193112 conn(0x7fe0640ffa90 msgr2=0x7fe064101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:36:08.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:08.676+0000 7fe069400640 1 -- 172.21.15.5:0/2862193112 shutdown_connections 2024-09-10T09:36:08.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:08.676+0000 7fe069400640 1 -- 172.21.15.5:0/2862193112 wait complete. 2024-09-10T09:36:08.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:08.677+0000 7fe069400640 1 Processor -- start 2024-09-10T09:36:08.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:08.678+0000 7fe069400640 1 -- start start 2024-09-10T09:36:08.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:08.678+0000 7fe069400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe064103ff0 0x7fe06419a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:36:08.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:08.678+0000 7fe069400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe0641049a0 0x7fe06419abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:36:08.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:08.678+0000 7fe069400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe064109120 0x7fe06419ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:36:08.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:08.678+0000 7fe069400640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe064114120 con 0x7fe064109120 2024-09-10T09:36:08.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:08.678+0000 7fe069400640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fe064113fa0 con 0x7fe0641049a0 2024-09-10T09:36:08.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:08.678+0000 7fe069400640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fe0641142a0 con 0x7fe064103ff0 2024-09-10T09:36:08.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:08.678+0000 7fe062800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe064103ff0 0x7fe06419a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:36:08.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:08.678+0000 7fe062800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe064103ff0 0x7fe06419a6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:49220/0 (socket says 172.21.15.5:49220) 2024-09-10T09:36:08.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:08.678+0000 7fe062800640 1 -- 172.21.15.5:0/2240969832 learned_addr learned my addr 172.21.15.5:0/2240969832 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:36:08.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:08.678+0000 7fe063200640 1 --2- 172.21.15.5:0/2240969832 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe064109120 0x7fe06419ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:36:08.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:08.678+0000 7fe059e00640 1 --2- 172.21.15.5:0/2240969832 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe0641049a0 0x7fe06419abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:36:08.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:08.679+0000 7fe062800640 1 -- 172.21.15.5:0/2240969832 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe0641049a0 msgr2=0x7fe06419abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:36:08.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:08.679+0000 7fe062800640 1 --2- 172.21.15.5:0/2240969832 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe0641049a0 0x7fe06419abe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:36:08.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:08.679+0000 7fe062800640 1 -- 172.21.15.5:0/2240969832 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe064109120 msgr2=0x7fe06419ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:36:08.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:08.679+0000 7fe062800640 1 --2- 172.21.15.5:0/2240969832 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe064109120 0x7fe06419ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:36:08.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:08.679+0000 7fe062800640 1 -- 172.21.15.5:0/2240969832 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe06419f650 con 0x7fe064103ff0 2024-09-10T09:36:08.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:08.679+0000 7fe062800640 1 --2- 172.21.15.5:0/2240969832 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe064103ff0 0x7fe06419a6a0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fe05400e9f0 tx=0x7fe05400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:36:08.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:08.679+0000 7fe05be00640 1 -- 172.21.15.5:0/2240969832 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe05400cd80 con 0x7fe064103ff0 2024-09-10T09:36:08.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:08.679+0000 7fe069400640 1 -- 172.21.15.5:0/2240969832 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe06419f940 con 0x7fe064103ff0 2024-09-10T09:36:08.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:08.679+0000 7fe05be00640 1 -- 172.21.15.5:0/2240969832 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe054010910 con 0x7fe064103ff0 2024-09-10T09:36:08.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:08.680+0000 7fe05be00640 1 -- 172.21.15.5:0/2240969832 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0540188f0 con 0x7fe064103ff0 2024-09-10T09:36:08.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:08.680+0000 7fe069400640 1 -- 172.21.15.5:0/2240969832 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0641a7220 con 0x7fe064103ff0 2024-09-10T09:36:08.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:08.681+0000 7fe069400640 1 -- 172.21.15.5:0/2240969832 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe0641043d0 con 0x7fe064103ff0 2024-09-10T09:36:08.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:08.683+0000 7fe05be00640 1 -- 172.21.15.5:0/2240969832 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7fe054010ac0 con 0x7fe064103ff0 2024-09-10T09:36:08.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:08.684+0000 7fe05be00640 1 --2- 172.21.15.5:0/2240969832 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fe0480a2b80 0x7fe0480a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:36:08.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:08.684+0000 7fe05be00640 1 -- 172.21.15.5:0/2240969832 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9071+0+0 (secure 0 0 0) 0x7fe0540cfb90 con 0x7fe064103ff0 2024-09-10T09:36:08.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:08.685+0000 7fe059e00640 1 --2- 172.21.15.5:0/2240969832 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fe0480a2b80 0x7fe0480a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:36:08.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:08.685+0000 7fe059e00640 1 --2- 172.21.15.5:0/2240969832 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fe0480a2b80 0x7fe0480a5040 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fe04c009230 tx=0x7fe04c0058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:36:08.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:08.686+0000 7fe05be00640 1 -- 172.21.15.5:0/2240969832 <== mon.1 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe05409c100 con 0x7fe064103ff0 2024-09-10T09:36:08.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:36:08 smithi005 ceph-mon[30350]: pgmap v325: 129 pgs: 2 peering, 127 active+clean; 581 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 639 B/s wr, 0 op/s 2024-09-10T09:36:08.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:08.908+0000 7fe069400640 1 -- 172.21.15.5:0/2240969832 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fe06410c390 con 0x7fe064103ff0 2024-09-10T09:36:09.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:36:08 smithi159 ceph-mon[35513]: pgmap v325: 129 pgs: 2 peering, 127 active+clean; 581 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 639 B/s wr, 0 op/s 2024-09-10T09:36:09.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:36:08 smithi102 ceph-mon[35442]: pgmap v325: 129 pgs: 2 peering, 127 active+clean; 581 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 639 B/s wr, 0 op/s 2024-09-10T09:36:09.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:09.783+0000 7fe05be00640 1 -- 172.21.15.5:0/2240969832 <== mon.1 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v18) ==== 118+0+0 (secure 0 0 0) 0x7fe0540a0400 con 0x7fe064103ff0 2024-09-10T09:36:09.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:09.787+0000 7fe069400640 1 -- 172.21.15.5:0/2240969832 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fe0480a2b80 msgr2=0x7fe0480a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:36:09.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:09.787+0000 7fe069400640 1 --2- 172.21.15.5:0/2240969832 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fe0480a2b80 0x7fe0480a5040 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fe04c009230 tx=0x7fe04c0058d0 comp rx=0 tx=0).stop 2024-09-10T09:36:09.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:09.787+0000 7fe069400640 1 -- 172.21.15.5:0/2240969832 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe064103ff0 msgr2=0x7fe06419a6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:36:09.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:09.787+0000 7fe069400640 1 --2- 172.21.15.5:0/2240969832 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe064103ff0 0x7fe06419a6a0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fe05400e9f0 tx=0x7fe05400eeb0 comp rx=0 tx=0).stop 2024-09-10T09:36:09.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:09.787+0000 7fe069400640 1 -- 172.21.15.5:0/2240969832 shutdown_connections 2024-09-10T09:36:09.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:09.787+0000 7fe069400640 1 --2- 172.21.15.5:0/2240969832 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fe0480a2b80 0x7fe0480a5040 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:36:09.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:09.787+0000 7fe069400640 1 --2- 172.21.15.5:0/2240969832 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe064109120 0x7fe06419ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:36:09.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:09.787+0000 7fe069400640 1 --2- 172.21.15.5:0/2240969832 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe0641049a0 0x7fe06419abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:36:09.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:09.787+0000 7fe069400640 1 --2- 172.21.15.5:0/2240969832 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe064103ff0 0x7fe06419a6a0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:36:09.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:09.787+0000 7fe069400640 1 -- 172.21.15.5:0/2240969832 >> 172.21.15.5:0/2240969832 conn(0x7fe0640ffa90 msgr2=0x7fe064100d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:36:09.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:09.787+0000 7fe069400640 1 -- 172.21.15.5:0/2240969832 shutdown_connections 2024-09-10T09:36:09.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:09.787+0000 7fe069400640 1 -- 172.21.15.5:0/2240969832 wait complete. 2024-09-10T09:36:10.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:36:09 smithi005 ceph-mon[30350]: from='client.? 172.21.15.5:0/2240969832' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-09-10T09:36:10.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:36:09 smithi005 ceph-mon[30350]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-09-10T09:36:10.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:36:09 smithi159 ceph-mon[35513]: from='client.? 172.21.15.5:0/2240969832' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-09-10T09:36:10.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:36:09 smithi159 ceph-mon[35513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-09-10T09:36:10.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:36:09 smithi102 ceph-mon[35442]: from='client.? 172.21.15.5:0/2240969832' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-09-10T09:36:10.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:36:09 smithi102 ceph-mon[35442]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-09-10T09:36:10.572 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 5, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2}} 2024-09-10T09:36:10.573 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph fs set cephfs max_mds 5 2024-09-10T09:36:10.957 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:36:11.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:36:10 smithi005 ceph-mon[30350]: pgmap v326: 129 pgs: 2 peering, 127 active+clean; 581 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 383 B/s wr, 0 op/s 2024-09-10T09:36:11.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:36:10 smithi005 ceph-mon[30350]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-09-10T09:36:11.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:36:10 smithi005 ceph-mon[30350]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-10T09:36:11.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:36:10 smithi159 ceph-mon[35513]: pgmap v326: 129 pgs: 2 peering, 127 active+clean; 581 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 383 B/s wr, 0 op/s 2024-09-10T09:36:11.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:36:10 smithi159 ceph-mon[35513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-09-10T09:36:11.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:36:10 smithi159 ceph-mon[35513]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-10T09:36:11.135 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:36:10 smithi102 ceph-mon[35442]: pgmap v326: 129 pgs: 2 peering, 127 active+clean; 581 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 383 B/s wr, 0 op/s 2024-09-10T09:36:11.136 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:36:10 smithi102 ceph-mon[35442]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-09-10T09:36:11.136 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:36:10 smithi102 ceph-mon[35442]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-10T09:36:11.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:11.537+0000 7f2625200640 1 -- 172.21.15.5:0/784799166 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2620104010 msgr2=0x7f26201043f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:36:11.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:11.537+0000 7f2625200640 1 --2- 172.21.15.5:0/784799166 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2620104010 0x7f26201043f0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f2610009900 tx=0x7f261002a3f0 comp rx=0 tx=0).stop 2024-09-10T09:36:11.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:11.537+0000 7f2625200640 1 -- 172.21.15.5:0/784799166 shutdown_connections 2024-09-10T09:36:11.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:11.537+0000 7f2625200640 1 --2- 172.21.15.5:0/784799166 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2620109140 0x7f2620110cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:36:11.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:11.537+0000 7f2625200640 1 --2- 172.21.15.5:0/784799166 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f26201049c0 0x7f2620108a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:36:11.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:11.537+0000 7f2625200640 1 --2- 172.21.15.5:0/784799166 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2620104010 0x7f26201043f0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:36:11.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:11.537+0000 7f2625200640 1 -- 172.21.15.5:0/784799166 >> 172.21.15.5:0/784799166 conn(0x7f26200ffa50 msgr2=0x7f2620101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:36:11.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:11.537+0000 7f2625200640 1 -- 172.21.15.5:0/784799166 shutdown_connections 2024-09-10T09:36:11.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:11.537+0000 7f2625200640 1 -- 172.21.15.5:0/784799166 wait complete. 2024-09-10T09:36:11.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:11.538+0000 7f2625200640 1 Processor -- start 2024-09-10T09:36:11.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:11.538+0000 7f2625200640 1 -- start start 2024-09-10T09:36:11.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:11.539+0000 7f2625200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2620104010 0x7f262019a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:36:11.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:11.539+0000 7f2625200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f26201049c0 0x7f262019ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:36:11.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:11.539+0000 7f2625200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2620109140 0x7f262019eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:36:11.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:11.539+0000 7f2625200640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f2620114190 con 0x7f26201049c0 2024-09-10T09:36:11.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:11.539+0000 7f2625200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f2620114010 con 0x7f2620104010 2024-09-10T09:36:11.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:11.539+0000 7f2625200640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f2620114310 con 0x7f2620109140 2024-09-10T09:36:11.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:11.539+0000 7f261de00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2620104010 0x7f262019a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:36:11.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:11.539+0000 7f261de00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2620104010 0x7f262019a600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:51400/0 (socket says 172.21.15.5:51400) 2024-09-10T09:36:11.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:11.539+0000 7f261d400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f26201049c0 0x7f262019ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:36:11.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:11.539+0000 7f261de00640 1 -- 172.21.15.5:0/1848885023 learned_addr learned my addr 172.21.15.5:0/1848885023 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:36:11.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:11.539+0000 7f261e800640 1 --2- 172.21.15.5:0/1848885023 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2620109140 0x7f262019eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:36:11.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:11.540+0000 7f261d400640 1 -- 172.21.15.5:0/1848885023 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2620109140 msgr2=0x7f262019eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:36:11.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:11.540+0000 7f261d400640 1 --2- 172.21.15.5:0/1848885023 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2620109140 0x7f262019eed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:36:11.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:11.540+0000 7f261d400640 1 -- 172.21.15.5:0/1848885023 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2620104010 msgr2=0x7f262019a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:36:11.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:11.540+0000 7f261d400640 1 --2- 172.21.15.5:0/1848885023 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2620104010 0x7f262019a600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:36:11.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:11.540+0000 7f261d400640 1 -- 172.21.15.5:0/1848885023 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f262019f5b0 con 0x7f26201049c0 2024-09-10T09:36:11.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:11.540+0000 7f261d400640 1 --2- 172.21.15.5:0/1848885023 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f26201049c0 0x7f262019ab40 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f260c005fd0 tx=0x7f260c00dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:36:11.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:11.540+0000 7f2616a00640 1 -- 172.21.15.5:0/1848885023 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f260c009c30 con 0x7f26201049c0 2024-09-10T09:36:11.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:11.540+0000 7f2616a00640 1 -- 172.21.15.5:0/1848885023 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f260c009dd0 con 0x7f26201049c0 2024-09-10T09:36:11.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:11.540+0000 7f2616a00640 1 -- 172.21.15.5:0/1848885023 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f260c010460 con 0x7f26201049c0 2024-09-10T09:36:11.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:11.540+0000 7f2625200640 1 -- 172.21.15.5:0/1848885023 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f262019f8a0 con 0x7f26201049c0 2024-09-10T09:36:11.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:11.541+0000 7f2625200640 1 -- 172.21.15.5:0/1848885023 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26201a7180 con 0x7f26201049c0 2024-09-10T09:36:11.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:11.542+0000 7f2625200640 1 -- 172.21.15.5:0/1848885023 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f26201043f0 con 0x7f26201049c0 2024-09-10T09:36:11.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:11.544+0000 7f2616a00640 1 -- 172.21.15.5:0/1848885023 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7f260c003f50 con 0x7f26201049c0 2024-09-10T09:36:11.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:11.546+0000 7f2616a00640 1 --2- 172.21.15.5:0/1848885023 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f26040a2c00 0x7f26040a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:36:11.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:11.546+0000 7f2616a00640 1 -- 172.21.15.5:0/1848885023 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9071+0+0 (secure 0 0 0) 0x7f260c0cf1f0 con 0x7f26201049c0 2024-09-10T09:36:11.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:11.546+0000 7f261de00640 1 --2- 172.21.15.5:0/1848885023 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f26040a2c00 0x7f26040a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:36:11.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:11.547+0000 7f261de00640 1 --2- 172.21.15.5:0/1848885023 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f26040a2c00 0x7f26040a50c0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f26100096c0 tx=0x7f26100046c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:36:11.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:11.550+0000 7f2616a00640 1 -- 172.21.15.5:0/1848885023 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f260c017020 con 0x7f26201049c0 2024-09-10T09:36:11.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:11.773+0000 7f2625200640 1 -- 172.21.15.5:0/1848885023 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"} v 0) -- 0x7f262010bd90 con 0x7f26201049c0 2024-09-10T09:36:11.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:11.785+0000 7f2616a00640 1 -- 172.21.15.5:0/1848885023 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]=0 v19) ==== 105+0+0 (secure 0 0 0) 0x7f260c09b760 con 0x7f26201049c0 2024-09-10T09:36:11.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:11.788+0000 7f2625200640 1 -- 172.21.15.5:0/1848885023 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f26040a2c00 msgr2=0x7f26040a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:36:11.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:11.788+0000 7f2625200640 1 --2- 172.21.15.5:0/1848885023 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f26040a2c00 0x7f26040a50c0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f26100096c0 tx=0x7f26100046c0 comp rx=0 tx=0).stop 2024-09-10T09:36:11.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:11.788+0000 7f2625200640 1 -- 172.21.15.5:0/1848885023 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f26201049c0 msgr2=0x7f262019ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:36:11.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:11.788+0000 7f2625200640 1 --2- 172.21.15.5:0/1848885023 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f26201049c0 0x7f262019ab40 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f260c005fd0 tx=0x7f260c00dde0 comp rx=0 tx=0).stop 2024-09-10T09:36:11.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:11.788+0000 7f2625200640 1 -- 172.21.15.5:0/1848885023 shutdown_connections 2024-09-10T09:36:11.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:11.788+0000 7f2625200640 1 --2- 172.21.15.5:0/1848885023 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f26040a2c00 0x7f26040a50c0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:36:11.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:11.788+0000 7f2625200640 1 --2- 172.21.15.5:0/1848885023 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2620109140 0x7f262019eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:36:11.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:11.788+0000 7f2625200640 1 --2- 172.21.15.5:0/1848885023 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f26201049c0 0x7f262019ab40 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:36:11.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:11.788+0000 7f2625200640 1 --2- 172.21.15.5:0/1848885023 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2620104010 0x7f262019a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:36:11.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:11.788+0000 7f2625200640 1 -- 172.21.15.5:0/1848885023 >> 172.21.15.5:0/1848885023 conn(0x7f26200ffa50 msgr2=0x7f2620100d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:36:11.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:11.788+0000 7f2625200640 1 -- 172.21.15.5:0/1848885023 shutdown_connections 2024-09-10T09:36:11.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:11.788+0000 7f2625200640 1 -- 172.21.15.5:0/1848885023 wait complete. 2024-09-10T09:36:12.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:36:11 smithi005 ceph-mon[30350]: from='client.? 172.21.15.5:0/1848885023' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-09-10T09:36:12.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:36:11 smithi159 ceph-mon[35513]: from='client.? 172.21.15.5:0/1848885023' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-09-10T09:36:12.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:36:11 smithi102 ceph-mon[35442]: from='client.? 172.21.15.5:0/1848885023' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-09-10T09:36:12.601 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph fs set cephfs allow_standby_replay true 2024-09-10T09:36:12.986 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:36:13.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:36:12 smithi005 ceph-mon[30350]: pgmap v327: 129 pgs: 129 active+clean; 581 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 383 B/s wr, 0 op/s 2024-09-10T09:36:13.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:36:12 smithi005 ceph-mon[30350]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-09-10T09:36:13.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:36:12 smithi005 ceph-mon[30350]: from='client.? 172.21.15.5:0/1848885023' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-09-10T09:36:13.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:36:12 smithi005 ceph-mon[30350]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-09-10T09:36:13.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:36:12 smithi005 ceph-mon[30350]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-10T09:36:13.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:36:12 smithi005 ceph-mon[30350]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-09-10T09:36:13.057 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:36:12 smithi005 ceph-mon[30350]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-09-10T09:36:13.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:36:12 smithi159 ceph-mon[35513]: pgmap v327: 129 pgs: 129 active+clean; 581 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 383 B/s wr, 0 op/s 2024-09-10T09:36:13.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:36:12 smithi159 ceph-mon[35513]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-09-10T09:36:13.101 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:36:12 smithi159 ceph-mon[35513]: from='client.? 172.21.15.5:0/1848885023' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-09-10T09:36:13.101 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:36:12 smithi159 ceph-mon[35513]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-09-10T09:36:13.101 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:36:12 smithi159 ceph-mon[35513]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-10T09:36:13.101 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:36:12 smithi159 ceph-mon[35513]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-09-10T09:36:13.101 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:36:12 smithi159 ceph-mon[35513]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-09-10T09:36:13.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:36:12 smithi102 ceph-mon[35442]: pgmap v327: 129 pgs: 129 active+clean; 581 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 383 B/s wr, 0 op/s 2024-09-10T09:36:13.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:36:12 smithi102 ceph-mon[35442]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-09-10T09:36:13.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:36:12 smithi102 ceph-mon[35442]: from='client.? 172.21.15.5:0/1848885023' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-09-10T09:36:13.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:36:12 smithi102 ceph-mon[35442]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-09-10T09:36:13.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:36:12 smithi102 ceph-mon[35442]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-10T09:36:13.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:36:12 smithi102 ceph-mon[35442]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-09-10T09:36:13.135 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:36:12 smithi102 ceph-mon[35442]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-09-10T09:36:13.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:13.543+0000 7f54eda00640 1 -- 172.21.15.5:0/1358706468 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f54e81047e0 msgr2=0x7f54e8108830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:36:13.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:13.543+0000 7f54eda00640 1 --2- 172.21.15.5:0/1358706468 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f54e81047e0 0x7f54e8108830 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f54d4009900 tx=0x7f54d402a410 comp rx=0 tx=0).stop 2024-09-10T09:36:13.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:13.543+0000 7f54eda00640 1 -- 172.21.15.5:0/1358706468 shutdown_connections 2024-09-10T09:36:13.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:13.543+0000 7f54eda00640 1 --2- 172.21.15.5:0/1358706468 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f54e8108f60 0x7f54e8110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:36:13.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:13.543+0000 7f54eda00640 1 --2- 172.21.15.5:0/1358706468 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f54e81047e0 0x7f54e8108830 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:36:13.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:13.543+0000 7f54eda00640 1 --2- 172.21.15.5:0/1358706468 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f54e8103e30 0x7f54e8104210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:36:13.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:13.543+0000 7f54eda00640 1 -- 172.21.15.5:0/1358706468 >> 172.21.15.5:0/1358706468 conn(0x7f54e80ff910 msgr2=0x7f54e8101d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:36:13.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:13.543+0000 7f54eda00640 1 -- 172.21.15.5:0/1358706468 shutdown_connections 2024-09-10T09:36:13.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:13.543+0000 7f54eda00640 1 -- 172.21.15.5:0/1358706468 wait complete. 2024-09-10T09:36:13.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:13.544+0000 7f54eda00640 1 Processor -- start 2024-09-10T09:36:13.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:13.544+0000 7f54eda00640 1 -- start start 2024-09-10T09:36:13.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:13.544+0000 7f54eda00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f54e8103e30 0x7f54e819a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:36:13.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:13.544+0000 7f54eda00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f54e81047e0 0x7f54e819a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:36:13.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:13.544+0000 7f54eda00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f54e8108f60 0x7f54e819ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:36:13.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:13.544+0000 7f54eda00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f54e8113fc0 con 0x7f54e8108f60 2024-09-10T09:36:13.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:13.544+0000 7f54eda00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f54e8113e40 con 0x7f54e8103e30 2024-09-10T09:36:13.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:13.544+0000 7f54eda00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f54e8114140 con 0x7f54e81047e0 2024-09-10T09:36:13.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:13.545+0000 7f54ed000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f54e8108f60 0x7f54e819ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:36:13.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:13.545+0000 7f54ed000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f54e8108f60 0x7f54e819ed50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40014/0 (socket says 172.21.15.5:40014) 2024-09-10T09:36:13.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:13.545+0000 7f54e7400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f54e81047e0 0x7f54e819a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:36:13.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:13.545+0000 7f54ed000640 1 -- 172.21.15.5:0/547178059 learned_addr learned my addr 172.21.15.5:0/547178059 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:36:13.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:13.545+0000 7f54e7e00640 1 --2- 172.21.15.5:0/547178059 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f54e8103e30 0x7f54e819a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:36:13.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:13.545+0000 7f54e7e00640 1 -- 172.21.15.5:0/547178059 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f54e81047e0 msgr2=0x7f54e819a9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:36:13.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:13.545+0000 7f54e7e00640 1 --2- 172.21.15.5:0/547178059 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f54e81047e0 0x7f54e819a9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:36:13.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:13.545+0000 7f54e7e00640 1 -- 172.21.15.5:0/547178059 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f54e8108f60 msgr2=0x7f54e819ed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:36:13.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:13.545+0000 7f54e7e00640 1 --2- 172.21.15.5:0/547178059 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f54e8108f60 0x7f54e819ed50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:36:13.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:13.545+0000 7f54e7e00640 1 -- 172.21.15.5:0/547178059 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54e819f430 con 0x7f54e8103e30 2024-09-10T09:36:13.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:13.546+0000 7f54e7e00640 1 --2- 172.21.15.5:0/547178059 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f54e8103e30 0x7f54e819a480 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f54d800e9f0 tx=0x7f54d800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:36:13.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:13.546+0000 7f54e4c00640 1 -- 172.21.15.5:0/547178059 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54d800ccc0 con 0x7f54e8103e30 2024-09-10T09:36:13.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:13.546+0000 7f54eda00640 1 -- 172.21.15.5:0/547178059 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54e819f720 con 0x7f54e8103e30 2024-09-10T09:36:13.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:13.546+0000 7f54e4c00640 1 -- 172.21.15.5:0/547178059 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f54d8010910 con 0x7f54e8103e30 2024-09-10T09:36:13.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:13.546+0000 7f54e4c00640 1 -- 172.21.15.5:0/547178059 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54d8018960 con 0x7f54e8103e30 2024-09-10T09:36:13.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:13.546+0000 7f54eda00640 1 -- 172.21.15.5:0/547178059 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54e81a7000 con 0x7f54e8103e30 2024-09-10T09:36:13.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:13.547+0000 7f54eda00640 1 -- 172.21.15.5:0/547178059 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f54e8105310 con 0x7f54e8103e30 2024-09-10T09:36:13.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:13.549+0000 7f54e4c00640 1 -- 172.21.15.5:0/547178059 <== mon.2 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7f54d8010ac0 con 0x7f54e8103e30 2024-09-10T09:36:13.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:13.550+0000 7f54e4c00640 1 --2- 172.21.15.5:0/547178059 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f54cc0a2b80 0x7f54cc0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:36:13.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:13.551+0000 7f54e4c00640 1 -- 172.21.15.5:0/547178059 <== mon.2 v2:172.21.15.102:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9071+0+0 (secure 0 0 0) 0x7f54d80cfaf0 con 0x7f54e8103e30 2024-09-10T09:36:13.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:13.551+0000 7f54e7400640 1 --2- 172.21.15.5:0/547178059 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f54cc0a2b80 0x7f54cc0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:36:13.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:13.551+0000 7f54e7400640 1 --2- 172.21.15.5:0/547178059 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f54cc0a2b80 0x7f54cc0a5040 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f54d4009230 tx=0x7f54d40058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:36:13.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:13.553+0000 7f54e4c00640 1 -- 172.21.15.5:0/547178059 <== mon.2 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f54d809c060 con 0x7f54e8103e30 2024-09-10T09:36:13.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:13.777+0000 7f54eda00640 1 -- 172.21.15.5:0/547178059 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) -- 0x7f54e810c1a0 con 0x7f54e8103e30 2024-09-10T09:36:13.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:13.804+0000 7f54e4c00640 1 -- 172.21.15.5:0/547178059 <== mon.2 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v23) ==== 121+0+0 (secure 0 0 0) 0x7f54d80a0360 con 0x7f54e8103e30 2024-09-10T09:36:13.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:13.807+0000 7f54eda00640 1 -- 172.21.15.5:0/547178059 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f54cc0a2b80 msgr2=0x7f54cc0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:36:13.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:13.807+0000 7f54eda00640 1 --2- 172.21.15.5:0/547178059 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f54cc0a2b80 0x7f54cc0a5040 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f54d4009230 tx=0x7f54d40058d0 comp rx=0 tx=0).stop 2024-09-10T09:36:13.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:13.807+0000 7f54eda00640 1 -- 172.21.15.5:0/547178059 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f54e8103e30 msgr2=0x7f54e819a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:36:13.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:13.807+0000 7f54eda00640 1 --2- 172.21.15.5:0/547178059 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f54e8103e30 0x7f54e819a480 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f54d800e9f0 tx=0x7f54d800eeb0 comp rx=0 tx=0).stop 2024-09-10T09:36:13.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:13.807+0000 7f54eda00640 1 -- 172.21.15.5:0/547178059 shutdown_connections 2024-09-10T09:36:13.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:13.807+0000 7f54eda00640 1 --2- 172.21.15.5:0/547178059 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f54cc0a2b80 0x7f54cc0a5040 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:36:13.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:13.807+0000 7f54eda00640 1 --2- 172.21.15.5:0/547178059 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f54e8108f60 0x7f54e819ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:36:13.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:13.807+0000 7f54eda00640 1 --2- 172.21.15.5:0/547178059 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f54e81047e0 0x7f54e819a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:36:13.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:13.807+0000 7f54eda00640 1 --2- 172.21.15.5:0/547178059 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f54e8103e30 0x7f54e819a480 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:36:13.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:13.807+0000 7f54eda00640 1 -- 172.21.15.5:0/547178059 >> 172.21.15.5:0/547178059 conn(0x7f54e80ff910 msgr2=0x7f54e8100c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:36:13.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:13.807+0000 7f54eda00640 1 -- 172.21.15.5:0/547178059 shutdown_connections 2024-09-10T09:36:13.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:13.807+0000 7f54eda00640 1 -- 172.21.15.5:0/547178059 wait complete. 2024-09-10T09:36:14.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:36:13 smithi005 ceph-mon[30350]: mds.? [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] up:active 2024-09-10T09:36:14.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:36:13 smithi005 ceph-mon[30350]: daemon mds.c assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-09-10T09:36:14.057 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:36:13 smithi005 ceph-mon[30350]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-09-10T09:36:14.057 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:36:13 smithi005 ceph-mon[30350]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:creating} 9 up:standby 2024-09-10T09:36:14.057 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:36:13 smithi005 ceph-mon[30350]: daemon mds.c is now active in filesystem cephfs as rank 2 2024-09-10T09:36:14.057 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:36:13 smithi005 ceph-mon[30350]: from='client.? 172.21.15.5:0/547178059' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-09-10T09:36:14.057 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:36:13 smithi005 ceph-mon[30350]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-09-10T09:36:14.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:36:13 smithi159 ceph-mon[35513]: mds.? [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] up:active 2024-09-10T09:36:14.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:36:13 smithi159 ceph-mon[35513]: daemon mds.c assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-09-10T09:36:14.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:36:13 smithi159 ceph-mon[35513]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-09-10T09:36:14.101 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:36:13 smithi159 ceph-mon[35513]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:creating} 9 up:standby 2024-09-10T09:36:14.101 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:36:13 smithi159 ceph-mon[35513]: daemon mds.c is now active in filesystem cephfs as rank 2 2024-09-10T09:36:14.101 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:36:13 smithi159 ceph-mon[35513]: from='client.? 172.21.15.5:0/547178059' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-09-10T09:36:14.101 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:36:13 smithi159 ceph-mon[35513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-09-10T09:36:14.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:36:13 smithi102 ceph-mon[35442]: mds.? [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] up:active 2024-09-10T09:36:14.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:36:13 smithi102 ceph-mon[35442]: daemon mds.c assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-09-10T09:36:14.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:36:13 smithi102 ceph-mon[35442]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-09-10T09:36:14.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:36:13 smithi102 ceph-mon[35442]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:creating} 9 up:standby 2024-09-10T09:36:14.135 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:36:13 smithi102 ceph-mon[35442]: daemon mds.c is now active in filesystem cephfs as rank 2 2024-09-10T09:36:14.135 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:36:13 smithi102 ceph-mon[35442]: from='client.? 172.21.15.5:0/547178059' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-09-10T09:36:14.135 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:36:13 smithi102 ceph-mon[35442]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-09-10T09:36:14.577 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph fs set cephfs session_timeout 300 2024-09-10T09:36:14.957 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:36:15.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:36:14 smithi005 ceph-mon[30350]: pgmap v328: 129 pgs: 129 active+clean; 581 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2024-09-10T09:36:15.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:36:14 smithi005 ceph-mon[30350]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-10T09:36:15.057 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:36:14 smithi005 ceph-mon[30350]: mds.? [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] up:active 2024-09-10T09:36:15.057 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:36:14 smithi005 ceph-mon[30350]: daemon mds.h assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-09-10T09:36:15.057 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:36:14 smithi005 ceph-mon[30350]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 9 up:standby 2024-09-10T09:36:15.057 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:36:14 smithi005 ceph-mon[30350]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=c=up:active,3=h=up:creating} 8 up:standby 2024-09-10T09:36:15.057 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:36:14 smithi005 ceph-mon[30350]: daemon mds.h is now active in filesystem cephfs as rank 3 2024-09-10T09:36:15.057 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:36:14 smithi005 ceph-mon[30350]: mds.? [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] up:active 2024-09-10T09:36:15.057 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:36:14 smithi005 ceph-mon[30350]: daemon mds.k assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-09-10T09:36:15.057 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:36:14 smithi005 ceph-mon[30350]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-09-10T09:36:15.057 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:36:14 smithi005 ceph-mon[30350]: Cluster is now healthy 2024-09-10T09:36:15.057 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:36:14 smithi005 ceph-mon[30350]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=c=up:active,3=h=up:active} 4 up:standby-replay 4 up:standby 2024-09-10T09:36:15.057 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:36:14 smithi005 ceph-mon[30350]: fsmap cephfs:5 4 up:standby-replay 1 up:creating 3 up:standby 4 up:active 2024-09-10T09:36:15.057 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:36:14 smithi005 ceph-mon[30350]: daemon mds.k is now active in filesystem cephfs as rank 4 2024-09-10T09:36:15.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:36:14 smithi159 ceph-mon[35513]: pgmap v328: 129 pgs: 129 active+clean; 581 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2024-09-10T09:36:15.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:36:14 smithi159 ceph-mon[35513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-10T09:36:15.101 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:36:14 smithi159 ceph-mon[35513]: mds.? [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] up:active 2024-09-10T09:36:15.101 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:36:14 smithi159 ceph-mon[35513]: daemon mds.h assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-09-10T09:36:15.101 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:36:14 smithi159 ceph-mon[35513]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 9 up:standby 2024-09-10T09:36:15.101 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:36:14 smithi159 ceph-mon[35513]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=c=up:active,3=h=up:creating} 8 up:standby 2024-09-10T09:36:15.101 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:36:14 smithi159 ceph-mon[35513]: daemon mds.h is now active in filesystem cephfs as rank 3 2024-09-10T09:36:15.101 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:36:14 smithi159 ceph-mon[35513]: mds.? [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] up:active 2024-09-10T09:36:15.101 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:36:14 smithi159 ceph-mon[35513]: daemon mds.k assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-09-10T09:36:15.101 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:36:14 smithi159 ceph-mon[35513]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-09-10T09:36:15.101 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:36:14 smithi159 ceph-mon[35513]: Cluster is now healthy 2024-09-10T09:36:15.101 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:36:14 smithi159 ceph-mon[35513]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=c=up:active,3=h=up:active} 4 up:standby-replay 4 up:standby 2024-09-10T09:36:15.101 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:36:14 smithi159 ceph-mon[35513]: fsmap cephfs:5 4 up:standby-replay 1 up:creating 3 up:standby 4 up:active 2024-09-10T09:36:15.101 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:36:14 smithi159 ceph-mon[35513]: daemon mds.k is now active in filesystem cephfs as rank 4 2024-09-10T09:36:15.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:36:14 smithi102 ceph-mon[35442]: pgmap v328: 129 pgs: 129 active+clean; 581 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2024-09-10T09:36:15.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:36:14 smithi102 ceph-mon[35442]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-10T09:36:15.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:36:14 smithi102 ceph-mon[35442]: mds.? [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] up:active 2024-09-10T09:36:15.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:36:14 smithi102 ceph-mon[35442]: daemon mds.h assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-09-10T09:36:15.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:36:14 smithi102 ceph-mon[35442]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 9 up:standby 2024-09-10T09:36:15.135 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:36:14 smithi102 ceph-mon[35442]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=c=up:active,3=h=up:creating} 8 up:standby 2024-09-10T09:36:15.135 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:36:14 smithi102 ceph-mon[35442]: daemon mds.h is now active in filesystem cephfs as rank 3 2024-09-10T09:36:15.135 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:36:14 smithi102 ceph-mon[35442]: mds.? [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] up:active 2024-09-10T09:36:15.135 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:36:14 smithi102 ceph-mon[35442]: daemon mds.k assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-09-10T09:36:15.135 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:36:14 smithi102 ceph-mon[35442]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-09-10T09:36:15.135 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:36:14 smithi102 ceph-mon[35442]: Cluster is now healthy 2024-09-10T09:36:15.135 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:36:14 smithi102 ceph-mon[35442]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=c=up:active,3=h=up:active} 4 up:standby-replay 4 up:standby 2024-09-10T09:36:15.135 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:36:14 smithi102 ceph-mon[35442]: fsmap cephfs:5 4 up:standby-replay 1 up:creating 3 up:standby 4 up:active 2024-09-10T09:36:15.135 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:36:14 smithi102 ceph-mon[35442]: daemon mds.k is now active in filesystem cephfs as rank 4 2024-09-10T09:36:15.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:15.529+0000 7fe683e00640 1 -- 172.21.15.5:0/793926903 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe684074590 msgr2=0x7fe684074970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:36:15.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:15.529+0000 7fe683e00640 1 --2- 172.21.15.5:0/793926903 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe684074590 0x7fe684074970 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fe674009900 tx=0x7fe67402a490 comp rx=0 tx=0).stop 2024-09-10T09:36:15.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:15.529+0000 7fe683e00640 1 -- 172.21.15.5:0/793926903 shutdown_connections 2024-09-10T09:36:15.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:15.530+0000 7fe683e00640 1 --2- 172.21.15.5:0/793926903 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe6841129e0 0x7fe684114dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:36:15.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:15.530+0000 7fe683e00640 1 --2- 172.21.15.5:0/793926903 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe684074eb0 0x7fe6841124a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:36:15.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:15.530+0000 7fe683e00640 1 --2- 172.21.15.5:0/793926903 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe684074590 0x7fe684074970 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:36:15.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:15.530+0000 7fe683e00640 1 -- 172.21.15.5:0/793926903 >> 172.21.15.5:0/793926903 conn(0x7fe6840fd710 msgr2=0x7fe6840ffb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:36:15.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:15.530+0000 7fe683e00640 1 -- 172.21.15.5:0/793926903 shutdown_connections 2024-09-10T09:36:15.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:15.530+0000 7fe683e00640 1 -- 172.21.15.5:0/793926903 wait complete. 2024-09-10T09:36:15.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:15.531+0000 7fe683e00640 1 Processor -- start 2024-09-10T09:36:15.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:15.531+0000 7fe683e00640 1 -- start start 2024-09-10T09:36:15.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:15.531+0000 7fe683e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe684074590 0x7fe6841a4680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:36:15.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:15.531+0000 7fe683e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe684074eb0 0x7fe6841a4bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:36:15.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:15.531+0000 7fe683e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe6841129e0 0x7fe68419e750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:36:15.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:15.531+0000 7fe683e00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe6841183f0 con 0x7fe684074eb0 2024-09-10T09:36:15.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:15.531+0000 7fe683e00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fe684118270 con 0x7fe6841129e0 2024-09-10T09:36:15.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:15.531+0000 7fe683e00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fe684118570 con 0x7fe684074590 2024-09-10T09:36:15.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:15.531+0000 7fe67be00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe684074eb0 0x7fe6841a4bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:36:15.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:15.532+0000 7fe67be00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe684074eb0 0x7fe6841a4bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40042/0 (socket says 172.21.15.5:40042) 2024-09-10T09:36:15.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:15.532+0000 7fe67be00640 1 -- 172.21.15.5:0/2161528331 learned_addr learned my addr 172.21.15.5:0/2161528331 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:36:15.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:15.532+0000 7fe682a00640 1 --2- 172.21.15.5:0/2161528331 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe684074590 0x7fe6841a4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:36:15.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:15.532+0000 7fe683400640 1 --2- 172.21.15.5:0/2161528331 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe6841129e0 0x7fe68419e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:36:15.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:15.532+0000 7fe683400640 1 -- 172.21.15.5:0/2161528331 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe684074590 msgr2=0x7fe6841a4680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:36:15.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:15.532+0000 7fe683400640 1 --2- 172.21.15.5:0/2161528331 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe684074590 0x7fe6841a4680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:36:15.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:15.532+0000 7fe683400640 1 -- 172.21.15.5:0/2161528331 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe684074eb0 msgr2=0x7fe6841a4bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:36:15.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:15.532+0000 7fe683400640 1 --2- 172.21.15.5:0/2161528331 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe684074eb0 0x7fe6841a4bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:36:15.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:15.532+0000 7fe683400640 1 -- 172.21.15.5:0/2161528331 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe68419efc0 con 0x7fe6841129e0 2024-09-10T09:36:15.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:15.532+0000 7fe67be00640 1 --2- 172.21.15.5:0/2161528331 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe684074eb0 0x7fe6841a4bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:36:15.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:15.532+0000 7fe682a00640 1 --2- 172.21.15.5:0/2161528331 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe684074590 0x7fe6841a4680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:36:15.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:15.532+0000 7fe683400640 1 --2- 172.21.15.5:0/2161528331 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe6841129e0 0x7fe68419e750 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fe67c005fd0 tx=0x7fe67c00c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:36:15.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:15.533+0000 7fe67b400640 1 -- 172.21.15.5:0/2161528331 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe67c010630 con 0x7fe6841129e0 2024-09-10T09:36:15.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:15.533+0000 7fe67b400640 1 -- 172.21.15.5:0/2161528331 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe67c010cb0 con 0x7fe6841129e0 2024-09-10T09:36:15.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:15.533+0000 7fe683e00640 1 -- 172.21.15.5:0/2161528331 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe68419f2b0 con 0x7fe6841129e0 2024-09-10T09:36:15.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:15.533+0000 7fe67b400640 1 -- 172.21.15.5:0/2161528331 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe67c017c80 con 0x7fe6841129e0 2024-09-10T09:36:15.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:15.533+0000 7fe683e00640 1 -- 172.21.15.5:0/2161528331 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6841ab600 con 0x7fe6841129e0 2024-09-10T09:36:15.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:15.534+0000 7fe683e00640 1 -- 172.21.15.5:0/2161528331 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe684075cd0 con 0x7fe6841129e0 2024-09-10T09:36:15.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:15.536+0000 7fe67b400640 1 -- 172.21.15.5:0/2161528331 <== mon.2 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7fe67c0107d0 con 0x7fe6841129e0 2024-09-10T09:36:15.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:15.537+0000 7fe67b400640 1 --2- 172.21.15.5:0/2161528331 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fe6680a2c00 0x7fe6680a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:36:15.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:15.537+0000 7fe67b400640 1 -- 172.21.15.5:0/2161528331 <== mon.2 v2:172.21.15.102:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9071+0+0 (secure 0 0 0) 0x7fe67c0cfac0 con 0x7fe6841129e0 2024-09-10T09:36:15.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:15.538+0000 7fe682a00640 1 --2- 172.21.15.5:0/2161528331 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fe6680a2c00 0x7fe6680a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:36:15.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:15.538+0000 7fe682a00640 1 --2- 172.21.15.5:0/2161528331 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fe6680a2c00 0x7fe6680a50c0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fe674000c00 tx=0x7fe674031510 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:36:15.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:15.541+0000 7fe67b400640 1 -- 172.21.15.5:0/2161528331 <== mon.2 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe67c09c680 con 0x7fe6841129e0 2024-09-10T09:36:15.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:15.762+0000 7fe683e00640 1 -- 172.21.15.5:0/2161528331 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7fe684104e20 con 0x7fe6841129e0 2024-09-10T09:36:16.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:16.501+0000 7fe67b400640 1 -- 172.21.15.5:0/2161528331 <== mon.2 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v29) ==== 115+0+0 (secure 0 0 0) 0x7fe67c09bf80 con 0x7fe6841129e0 2024-09-10T09:36:16.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:16.503+0000 7fe683e00640 1 -- 172.21.15.5:0/2161528331 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fe6680a2c00 msgr2=0x7fe6680a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:36:16.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:16.503+0000 7fe683e00640 1 --2- 172.21.15.5:0/2161528331 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fe6680a2c00 0x7fe6680a50c0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fe674000c00 tx=0x7fe674031510 comp rx=0 tx=0).stop 2024-09-10T09:36:16.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:16.503+0000 7fe683e00640 1 -- 172.21.15.5:0/2161528331 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe6841129e0 msgr2=0x7fe68419e750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:36:16.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:16.503+0000 7fe683e00640 1 --2- 172.21.15.5:0/2161528331 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe6841129e0 0x7fe68419e750 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fe67c005fd0 tx=0x7fe67c00c6a0 comp rx=0 tx=0).stop 2024-09-10T09:36:16.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:16.503+0000 7fe683e00640 1 -- 172.21.15.5:0/2161528331 shutdown_connections 2024-09-10T09:36:16.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:16.503+0000 7fe683e00640 1 --2- 172.21.15.5:0/2161528331 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fe6680a2c00 0x7fe6680a50c0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:36:16.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:16.503+0000 7fe683e00640 1 --2- 172.21.15.5:0/2161528331 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe6841129e0 0x7fe68419e750 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:36:16.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:16.503+0000 7fe683e00640 1 --2- 172.21.15.5:0/2161528331 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe684074eb0 0x7fe6841a4bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:36:16.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:16.503+0000 7fe683e00640 1 --2- 172.21.15.5:0/2161528331 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe684074590 0x7fe6841a4680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:36:16.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:16.503+0000 7fe683e00640 1 -- 172.21.15.5:0/2161528331 >> 172.21.15.5:0/2161528331 conn(0x7fe6840fd710 msgr2=0x7fe6840fef10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:36:16.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:16.504+0000 7fe683e00640 1 -- 172.21.15.5:0/2161528331 shutdown_connections 2024-09-10T09:36:16.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:16.504+0000 7fe683e00640 1 -- 172.21.15.5:0/2161528331 wait complete. 2024-09-10T09:36:16.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:36:16 smithi005 ceph-mon[30350]: mds.? [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] up:active 2024-09-10T09:36:16.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:36:16 smithi005 ceph-mon[30350]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-09-10T09:36:16.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:36:16 smithi005 ceph-mon[30350]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T09:36:16.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:36:16 smithi005 ceph-mon[30350]: from='client.? 172.21.15.5:0/2161528331' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-09-10T09:36:16.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:36:16 smithi005 ceph-mon[30350]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-09-10T09:36:16.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:36:16 smithi159 ceph-mon[35513]: mds.? [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] up:active 2024-09-10T09:36:16.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:36:16 smithi159 ceph-mon[35513]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-09-10T09:36:16.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:36:16 smithi159 ceph-mon[35513]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T09:36:16.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:36:16 smithi159 ceph-mon[35513]: from='client.? 172.21.15.5:0/2161528331' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-09-10T09:36:16.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:36:16 smithi159 ceph-mon[35513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-09-10T09:36:16.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:36:16 smithi102 ceph-mon[35442]: mds.? [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] up:active 2024-09-10T09:36:16.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:36:16 smithi102 ceph-mon[35442]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-09-10T09:36:16.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:36:16 smithi102 ceph-mon[35442]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T09:36:16.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:36:16 smithi102 ceph-mon[35442]: from='client.? 172.21.15.5:0/2161528331' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-09-10T09:36:16.885 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:36:16 smithi102 ceph-mon[35442]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-09-10T09:36:17.263 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2} subvols for filesystem 'cephfs' 2024-09-10T09:36:17.489 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph fs subvolumegroup create cephfs qa 2024-09-10T09:36:17.752 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:36:17 smithi159 ceph-mon[35513]: pgmap v329: 129 pgs: 129 active+clean; 585 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 3.4 KiB/s wr, 11 op/s 2024-09-10T09:36:17.753 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:36:17 smithi159 ceph-mon[35513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-09-10T09:36:17.753 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:36:17 smithi159 ceph-mon[35513]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T09:36:17.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:36:17 smithi005 ceph-mon[30350]: pgmap v329: 129 pgs: 129 active+clean; 585 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 3.4 KiB/s wr, 11 op/s 2024-09-10T09:36:17.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:36:17 smithi005 ceph-mon[30350]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-09-10T09:36:17.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:36:17 smithi005 ceph-mon[30350]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T09:36:17.869 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:36:17.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:36:17 smithi102 ceph-mon[35442]: pgmap v329: 129 pgs: 129 active+clean; 585 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 3.4 KiB/s wr, 11 op/s 2024-09-10T09:36:17.885 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:36:17 smithi102 ceph-mon[35442]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-09-10T09:36:17.885 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:36:17 smithi102 ceph-mon[35442]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T09:36:18.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:18.530+0000 7f1053400640 1 -- 172.21.15.5:0/2224469763 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f104c109120 msgr2=0x7f104c110ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:36:18.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:18.530+0000 7f1053400640 1 --2- 172.21.15.5:0/2224469763 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f104c109120 0x7f104c110ca0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f1044009ca0 tx=0x7f104402a8c0 comp rx=0 tx=0).stop 2024-09-10T09:36:18.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:18.531+0000 7f1053400640 1 -- 172.21.15.5:0/2224469763 shutdown_connections 2024-09-10T09:36:18.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:18.531+0000 7f1053400640 1 --2- 172.21.15.5:0/2224469763 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f104c109120 0x7f104c110ca0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:36:18.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:18.531+0000 7f1053400640 1 --2- 172.21.15.5:0/2224469763 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f104c1049a0 0x7f104c1089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:36:18.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:18.531+0000 7f1053400640 1 --2- 172.21.15.5:0/2224469763 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f104c103ff0 0x7f104c1043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:36:18.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:18.531+0000 7f1053400640 1 -- 172.21.15.5:0/2224469763 >> 172.21.15.5:0/2224469763 conn(0x7f104c0ffa90 msgr2=0x7f104c101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:36:18.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:18.531+0000 7f1053400640 1 -- 172.21.15.5:0/2224469763 shutdown_connections 2024-09-10T09:36:18.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:18.531+0000 7f1053400640 1 -- 172.21.15.5:0/2224469763 wait complete. 2024-09-10T09:36:18.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:18.532+0000 7f1053400640 1 Processor -- start 2024-09-10T09:36:18.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:18.532+0000 7f1053400640 1 -- start start 2024-09-10T09:36:18.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:18.532+0000 7f1053400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f104c103ff0 0x7f104c19a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:36:18.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:18.532+0000 7f1053400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f104c1049a0 0x7f104c19aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:36:18.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:18.532+0000 7f1053400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f104c109120 0x7f104c19ee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:36:18.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:18.532+0000 7f1053400640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f104c114130 con 0x7f104c103ff0 2024-09-10T09:36:18.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:18.532+0000 7f1053400640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f104c113fb0 con 0x7f104c1049a0 2024-09-10T09:36:18.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:18.532+0000 7f1053400640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f104c1142b0 con 0x7f104c109120 2024-09-10T09:36:18.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:18.533+0000 7f1050a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f104c103ff0 0x7f104c19a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:36:18.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:18.533+0000 7f1050a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f104c103ff0 0x7f104c19a590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40062/0 (socket says 172.21.15.5:40062) 2024-09-10T09:36:18.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:18.533+0000 7f1050a00640 1 -- 172.21.15.5:0/3689179859 learned_addr learned my addr 172.21.15.5:0/3689179859 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:36:18.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:18.533+0000 7f104be00640 1 --2- 172.21.15.5:0/3689179859 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f104c1049a0 0x7f104c19aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:36:18.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:18.533+0000 7f1051400640 1 --2- 172.21.15.5:0/3689179859 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f104c109120 0x7f104c19ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:36:18.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:18.533+0000 7f1050a00640 1 -- 172.21.15.5:0/3689179859 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f104c109120 msgr2=0x7f104c19ee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:36:18.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:18.533+0000 7f1050a00640 1 --2- 172.21.15.5:0/3689179859 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f104c109120 0x7f104c19ee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:36:18.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:18.533+0000 7f1050a00640 1 -- 172.21.15.5:0/3689179859 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f104c1049a0 msgr2=0x7f104c19aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:36:18.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:18.533+0000 7f1050a00640 1 --2- 172.21.15.5:0/3689179859 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f104c1049a0 0x7f104c19aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:36:18.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:18.533+0000 7f1050a00640 1 -- 172.21.15.5:0/3689179859 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f104c19f540 con 0x7f104c103ff0 2024-09-10T09:36:18.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:18.534+0000 7f1050a00640 1 --2- 172.21.15.5:0/3689179859 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f104c103ff0 0x7f104c19a590 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f104000d950 tx=0x7f104000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:36:18.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:18.534+0000 7f1049600640 1 -- 172.21.15.5:0/3689179859 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1040017070 con 0x7f104c103ff0 2024-09-10T09:36:18.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:18.534+0000 7f1053400640 1 -- 172.21.15.5:0/3689179859 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f104c19f830 con 0x7f104c103ff0 2024-09-10T09:36:18.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:18.534+0000 7f1049600640 1 -- 172.21.15.5:0/3689179859 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1040009c30 con 0x7f104c103ff0 2024-09-10T09:36:18.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:18.534+0000 7f1049600640 1 -- 172.21.15.5:0/3689179859 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f104000f6b0 con 0x7f104c103ff0 2024-09-10T09:36:18.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:18.534+0000 7f1053400640 1 -- 172.21.15.5:0/3689179859 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f104c1a7110 con 0x7f104c103ff0 2024-09-10T09:36:18.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:18.535+0000 7f1053400640 1 -- 172.21.15.5:0/3689179859 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f104c1043d0 con 0x7f104c103ff0 2024-09-10T09:36:18.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:18.537+0000 7f1049600640 1 -- 172.21.15.5:0/3689179859 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7f10400109d0 con 0x7f104c103ff0 2024-09-10T09:36:18.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:18.539+0000 7f1049600640 1 --2- 172.21.15.5:0/3689179859 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f10340a2b80 0x7f10340a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:36:18.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:18.540+0000 7f1049600640 1 -- 172.21.15.5:0/3689179859 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9071+0+0 (secure 0 0 0) 0x7f10400cefa0 con 0x7f104c103ff0 2024-09-10T09:36:18.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:18.540+0000 7f104be00640 1 --2- 172.21.15.5:0/3689179859 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f10340a2b80 0x7f10340a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:36:18.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:18.540+0000 7f104be00640 1 --2- 172.21.15.5:0/3689179859 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f10340a2b80 0x7f10340a5040 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f104c10aa90 tx=0x7f1038005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:36:18.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:18.544+0000 7f1049600640 1 -- 172.21.15.5:0/3689179859 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f1040009de0 con 0x7f104c103ff0 2024-09-10T09:36:18.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:18.754+0000 7f1053400640 1 -- 172.21.15.5:0/3689179859 --> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f104c100d60 con 0x7f10340a2b80 2024-09-10T09:36:18.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:18.784+0000 7f1049600640 1 -- 172.21.15.5:0/3689179859 <== mgr.14150 v2:172.21.15.5:6800/2707348081 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f104c100d60 con 0x7f10340a2b80 2024-09-10T09:36:18.785 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 10 09:36:18 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mgr-x[30575]: 2024-09-10T09:36:18.760+0000 7f5667800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T09:36:18.785 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 10 09:36:18 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mgr-x[30575]: 2024-09-10T09:36:18.760+0000 7f5667800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T09:36:18.786 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 10 09:36:18 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mgr-x[30575]: 2024-09-10T09:36:18.760+0000 7f5667800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T09:36:18.786 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 10 09:36:18 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mgr-x[30575]: 2024-09-10T09:36:18.760+0000 7f5667800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T09:36:18.786 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 10 09:36:18 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mgr-x[30575]: 2024-09-10T09:36:18.760+0000 7f5667800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T09:36:18.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:18.786+0000 7f1053400640 1 -- 172.21.15.5:0/3689179859 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f10340a2b80 msgr2=0x7f10340a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:36:18.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:18.787+0000 7f1053400640 1 --2- 172.21.15.5:0/3689179859 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f10340a2b80 0x7f10340a5040 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f104c10aa90 tx=0x7f1038005f90 comp rx=0 tx=0).stop 2024-09-10T09:36:18.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:18.787+0000 7f1053400640 1 -- 172.21.15.5:0/3689179859 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f104c103ff0 msgr2=0x7f104c19a590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:36:18.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:18.787+0000 7f1053400640 1 --2- 172.21.15.5:0/3689179859 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f104c103ff0 0x7f104c19a590 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f104000d950 tx=0x7f104000de10 comp rx=0 tx=0).stop 2024-09-10T09:36:18.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:18.787+0000 7f1053400640 1 -- 172.21.15.5:0/3689179859 shutdown_connections 2024-09-10T09:36:18.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:18.787+0000 7f1053400640 1 --2- 172.21.15.5:0/3689179859 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f10340a2b80 0x7f10340a5040 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:36:18.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:18.787+0000 7f1053400640 1 --2- 172.21.15.5:0/3689179859 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f104c109120 0x7f104c19ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:36:18.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:18.787+0000 7f1053400640 1 --2- 172.21.15.5:0/3689179859 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f104c1049a0 0x7f104c19aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:36:18.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:18.787+0000 7f1053400640 1 --2- 172.21.15.5:0/3689179859 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f104c103ff0 0x7f104c19a590 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:36:18.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:18.787+0000 7f1053400640 1 -- 172.21.15.5:0/3689179859 >> 172.21.15.5:0/3689179859 conn(0x7f104c0ffa90 msgr2=0x7f104c101ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:36:18.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:18.787+0000 7f1053400640 1 -- 172.21.15.5:0/3689179859 shutdown_connections 2024-09-10T09:36:18.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:18.787+0000 7f1053400640 1 -- 172.21.15.5:0/3689179859 wait complete. 2024-09-10T09:36:19.642 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph fs subvolume create cephfs sv_0 --group_name qa 2024-09-10T09:36:19.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:36:19 smithi005 ceph-mon[30350]: pgmap v330: 129 pgs: 129 active+clean; 586 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 3.4 KiB/s wr, 13 op/s 2024-09-10T09:36:19.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:36:19 smithi159 ceph-mon[35513]: pgmap v330: 129 pgs: 129 active+clean; 586 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 3.4 KiB/s wr, 13 op/s 2024-09-10T09:36:19.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:36:19 smithi102 ceph-mon[35442]: pgmap v330: 129 pgs: 129 active+clean; 586 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 3.4 KiB/s wr, 13 op/s 2024-09-10T09:36:20.023 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:36:20.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:20.592+0000 7fe8b4a00640 1 -- 172.21.15.5:0/2841206614 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe8b00745c0 msgr2=0x7fe8b00749a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:36:20.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:20.592+0000 7fe8b4a00640 1 --2- 172.21.15.5:0/2841206614 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe8b00745c0 0x7fe8b00749a0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fe8a0009900 tx=0x7fe8a002a480 comp rx=0 tx=0).stop 2024-09-10T09:36:20.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:20.592+0000 7fe8b4a00640 1 -- 172.21.15.5:0/2841206614 shutdown_connections 2024-09-10T09:36:20.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:20.592+0000 7fe8b4a00640 1 --2- 172.21.15.5:0/2841206614 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe8b010e260 0x7fe8b0110650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:36:20.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:20.592+0000 7fe8b4a00640 1 --2- 172.21.15.5:0/2841206614 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe8b0074ee0 0x7fe8b010dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:36:20.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:20.592+0000 7fe8b4a00640 1 --2- 172.21.15.5:0/2841206614 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe8b00745c0 0x7fe8b00749a0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:36:20.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:20.592+0000 7fe8b4a00640 1 -- 172.21.15.5:0/2841206614 >> 172.21.15.5:0/2841206614 conn(0x7fe8b00fd6e0 msgr2=0x7fe8b00ffb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:36:20.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:20.593+0000 7fe8b4a00640 1 -- 172.21.15.5:0/2841206614 shutdown_connections 2024-09-10T09:36:20.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:20.593+0000 7fe8b4a00640 1 -- 172.21.15.5:0/2841206614 wait complete. 2024-09-10T09:36:20.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:20.594+0000 7fe8b4a00640 1 Processor -- start 2024-09-10T09:36:20.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:20.594+0000 7fe8b4a00640 1 -- start start 2024-09-10T09:36:20.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:20.595+0000 7fe8b4a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe8b00745c0 0x7fe8b019a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:36:20.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:20.595+0000 7fe8b4a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe8b0074ee0 0x7fe8b019a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:36:20.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:20.595+0000 7fe8b4a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe8b010e260 0x7fe8b019ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:36:20.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:20.595+0000 7fe8b4a00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe8b0114000 con 0x7fe8b0074ee0 2024-09-10T09:36:20.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:20.595+0000 7fe8b4a00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fe8b0113e80 con 0x7fe8b010e260 2024-09-10T09:36:20.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:20.595+0000 7fe8b4a00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fe8b0114180 con 0x7fe8b00745c0 2024-09-10T09:36:20.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:20.595+0000 7fe8a7e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe8b0074ee0 0x7fe8b019a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:36:20.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:20.595+0000 7fe8ae800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe8b00745c0 0x7fe8b019a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:36:20.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:20.595+0000 7fe8a7e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe8b0074ee0 0x7fe8b019a9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40096/0 (socket says 172.21.15.5:40096) 2024-09-10T09:36:20.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:20.595+0000 7fe8af200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe8b010e260 0x7fe8b019ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:36:20.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:20.595+0000 7fe8a7e00640 1 -- 172.21.15.5:0/1712669376 learned_addr learned my addr 172.21.15.5:0/1712669376 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:36:20.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:20.595+0000 7fe8af200640 1 -- 172.21.15.5:0/1712669376 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe8b00745c0 msgr2=0x7fe8b019a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:36:20.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:20.595+0000 7fe8af200640 1 --2- 172.21.15.5:0/1712669376 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe8b00745c0 0x7fe8b019a4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:36:20.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:20.595+0000 7fe8af200640 1 -- 172.21.15.5:0/1712669376 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe8b0074ee0 msgr2=0x7fe8b019a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:36:20.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:20.595+0000 7fe8af200640 1 --2- 172.21.15.5:0/1712669376 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe8b0074ee0 0x7fe8b019a9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:36:20.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:20.595+0000 7fe8af200640 1 -- 172.21.15.5:0/1712669376 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8b019f460 con 0x7fe8b010e260 2024-09-10T09:36:20.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:20.596+0000 7fe8af200640 1 --2- 172.21.15.5:0/1712669376 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe8b010e260 0x7fe8b019ed80 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fe8a8005fd0 tx=0x7fe8a800ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:36:20.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:20.596+0000 7fe8a7400640 1 -- 172.21.15.5:0/1712669376 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8a800cdf0 con 0x7fe8b010e260 2024-09-10T09:36:20.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:20.596+0000 7fe8b4a00640 1 -- 172.21.15.5:0/1712669376 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8b019f750 con 0x7fe8b010e260 2024-09-10T09:36:20.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:20.596+0000 7fe8a7400640 1 -- 172.21.15.5:0/1712669376 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe8a8005020 con 0x7fe8b010e260 2024-09-10T09:36:20.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:20.596+0000 7fe8a7400640 1 -- 172.21.15.5:0/1712669376 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8a80106d0 con 0x7fe8b010e260 2024-09-10T09:36:20.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:20.596+0000 7fe8b4a00640 1 -- 172.21.15.5:0/1712669376 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8b01a7030 con 0x7fe8b010e260 2024-09-10T09:36:20.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:20.597+0000 7fe8b4a00640 1 -- 172.21.15.5:0/1712669376 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe8b00749a0 con 0x7fe8b010e260 2024-09-10T09:36:20.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:20.600+0000 7fe8a7400640 1 -- 172.21.15.5:0/1712669376 <== mon.2 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7fe8a80054d0 con 0x7fe8b010e260 2024-09-10T09:36:20.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:20.601+0000 7fe8a7400640 1 --2- 172.21.15.5:0/1712669376 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fe8940a2c10 0x7fe8940a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:36:20.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:20.601+0000 7fe8a7400640 1 -- 172.21.15.5:0/1712669376 <== mon.2 v2:172.21.15.102:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9071+0+0 (secure 0 0 0) 0x7fe8a80cfc20 con 0x7fe8b010e260 2024-09-10T09:36:20.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:20.601+0000 7fe8ae800640 1 --2- 172.21.15.5:0/1712669376 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fe8940a2c10 0x7fe8940a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:36:20.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:20.602+0000 7fe8ae800640 1 --2- 172.21.15.5:0/1712669376 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fe8940a2c10 0x7fe8940a50d0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fe8a0000c00 tx=0x7fe8a0005b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:36:20.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:20.605+0000 7fe8a7400640 1 -- 172.21.15.5:0/1712669376 <== mon.2 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe8a809c680 con 0x7fe8b010e260 2024-09-10T09:36:20.785 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:36:20 smithi102 ceph-mon[35442]: from='client.14694 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T09:36:20.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:36:20 smithi005 ceph-mon[30350]: from='client.14694 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T09:36:20.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:20.816+0000 7fe8b4a00640 1 -- 172.21.15.5:0/1712669376 --> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7fe8b0062630 con 0x7fe8940a2c10 2024-09-10T09:36:20.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:20.841+0000 7fe8a7400640 1 -- 172.21.15.5:0/1712669376 <== mgr.14150 v2:172.21.15.5:6800/2707348081 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fe8b0062630 con 0x7fe8940a2c10 2024-09-10T09:36:20.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:20.844+0000 7fe8b4a00640 1 -- 172.21.15.5:0/1712669376 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fe8940a2c10 msgr2=0x7fe8940a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:36:20.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:20.844+0000 7fe8b4a00640 1 --2- 172.21.15.5:0/1712669376 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fe8940a2c10 0x7fe8940a50d0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fe8a0000c00 tx=0x7fe8a0005b50 comp rx=0 tx=0).stop 2024-09-10T09:36:20.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:20.844+0000 7fe8b4a00640 1 -- 172.21.15.5:0/1712669376 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe8b010e260 msgr2=0x7fe8b019ed80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:36:20.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:20.844+0000 7fe8b4a00640 1 --2- 172.21.15.5:0/1712669376 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe8b010e260 0x7fe8b019ed80 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fe8a8005fd0 tx=0x7fe8a800ef10 comp rx=0 tx=0).stop 2024-09-10T09:36:20.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:20.844+0000 7fe8b4a00640 1 -- 172.21.15.5:0/1712669376 shutdown_connections 2024-09-10T09:36:20.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:20.844+0000 7fe8b4a00640 1 --2- 172.21.15.5:0/1712669376 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fe8940a2c10 0x7fe8940a50d0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:36:20.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:20.844+0000 7fe8b4a00640 1 --2- 172.21.15.5:0/1712669376 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe8b010e260 0x7fe8b019ed80 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:36:20.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:20.844+0000 7fe8b4a00640 1 --2- 172.21.15.5:0/1712669376 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe8b0074ee0 0x7fe8b019a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:36:20.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:20.844+0000 7fe8b4a00640 1 --2- 172.21.15.5:0/1712669376 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe8b00745c0 0x7fe8b019a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:36:20.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:20.844+0000 7fe8b4a00640 1 -- 172.21.15.5:0/1712669376 >> 172.21.15.5:0/1712669376 conn(0x7fe8b00fd6e0 msgr2=0x7fe8b00fe380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:36:20.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:20.844+0000 7fe8b4a00640 1 -- 172.21.15.5:0/1712669376 shutdown_connections 2024-09-10T09:36:20.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:20.844+0000 7fe8b4a00640 1 -- 172.21.15.5:0/1712669376 wait complete. 2024-09-10T09:36:20.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:36:20 smithi159 ceph-mon[35513]: from='client.14694 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T09:36:21.621 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-09-10T09:36:21.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:36:21 smithi005 ceph-mon[30350]: pgmap v331: 129 pgs: 129 active+clean; 587 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 4.0 KiB/s wr, 17 op/s 2024-09-10T09:36:21.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:36:21 smithi005 ceph-mon[30350]: mgrmap e17: x(active, since 8m), standbys: z, y 2024-09-10T09:36:21.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:36:21 smithi159 ceph-mon[35513]: pgmap v331: 129 pgs: 129 active+clean; 587 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 4.0 KiB/s wr, 17 op/s 2024-09-10T09:36:21.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:36:21 smithi159 ceph-mon[35513]: mgrmap e17: x(active, since 8m), standbys: z, y 2024-09-10T09:36:21.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:36:21 smithi102 ceph-mon[35442]: pgmap v331: 129 pgs: 129 active+clean; 587 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 4.0 KiB/s wr, 17 op/s 2024-09-10T09:36:21.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:36:21 smithi102 ceph-mon[35442]: mgrmap e17: x(active, since 8m), standbys: z, y 2024-09-10T09:36:21.997 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:36:22.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:22.568+0000 7f0322200640 1 -- 172.21.15.5:0/1537584985 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f031c1079c0 msgr2=0x7f031c107da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:36:22.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:22.568+0000 7f0322200640 1 --2- 172.21.15.5:0/1537584985 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f031c1079c0 0x7f031c107da0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f0310009930 tx=0x7f031002a420 comp rx=0 tx=0).stop 2024-09-10T09:36:22.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:22.568+0000 7f0322200640 1 -- 172.21.15.5:0/1537584985 shutdown_connections 2024-09-10T09:36:22.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:22.568+0000 7f0322200640 1 --2- 172.21.15.5:0/1537584985 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f031c102360 0x7f031c10e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:36:22.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:22.568+0000 7f0322200640 1 --2- 172.21.15.5:0/1537584985 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f031c1019c0 0x7f031c101e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:36:22.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:22.568+0000 7f0322200640 1 --2- 172.21.15.5:0/1537584985 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f031c1079c0 0x7f031c107da0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:36:22.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:22.568+0000 7f0322200640 1 -- 172.21.15.5:0/1537584985 >> 172.21.15.5:0/1537584985 conn(0x7f031c0fd740 msgr2=0x7f031c0ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:36:22.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:22.568+0000 7f0322200640 1 -- 172.21.15.5:0/1537584985 shutdown_connections 2024-09-10T09:36:22.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:22.568+0000 7f0322200640 1 -- 172.21.15.5:0/1537584985 wait complete. 2024-09-10T09:36:22.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:22.569+0000 7f0322200640 1 Processor -- start 2024-09-10T09:36:22.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:22.569+0000 7f0322200640 1 -- start start 2024-09-10T09:36:22.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:22.569+0000 7f0322200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f031c1019c0 0x7f031c0744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:36:22.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:22.569+0000 7f0322200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f031c102360 0x7f031c074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:36:22.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:22.569+0000 7f0322200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f031c1079c0 0x7f031c074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:36:22.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:22.569+0000 7f0322200640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f031c111dc0 con 0x7f031c1079c0 2024-09-10T09:36:22.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:22.569+0000 7f0322200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f031c111c40 con 0x7f031c1019c0 2024-09-10T09:36:22.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:22.569+0000 7f0322200640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f031c111f40 con 0x7f031c102360 2024-09-10T09:36:22.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:22.570+0000 7f0320e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f031c1019c0 0x7f031c0744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:36:22.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:22.570+0000 7f0321800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f031c1079c0 0x7f031c074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:36:22.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:22.570+0000 7f0320e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f031c1019c0 0x7f031c0744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:49834/0 (socket says 172.21.15.5:49834) 2024-09-10T09:36:22.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:22.570+0000 7f0320e00640 1 -- 172.21.15.5:0/3659168424 learned_addr learned my addr 172.21.15.5:0/3659168424 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:36:22.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:22.570+0000 7f031be00640 1 --2- 172.21.15.5:0/3659168424 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f031c102360 0x7f031c074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:36:22.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:22.570+0000 7f0321800640 1 -- 172.21.15.5:0/3659168424 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f031c102360 msgr2=0x7f031c074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:36:22.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:22.570+0000 7f0321800640 1 --2- 172.21.15.5:0/3659168424 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f031c102360 0x7f031c074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:36:22.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:22.570+0000 7f0321800640 1 -- 172.21.15.5:0/3659168424 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f031c1019c0 msgr2=0x7f031c0744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:36:22.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:22.570+0000 7f0321800640 1 --2- 172.21.15.5:0/3659168424 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f031c1019c0 0x7f031c0744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:36:22.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:22.570+0000 7f0321800640 1 -- 172.21.15.5:0/3659168424 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f031c1a67a0 con 0x7f031c1079c0 2024-09-10T09:36:22.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:22.570+0000 7f0320e00640 1 --2- 172.21.15.5:0/3659168424 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f031c1019c0 0x7f031c0744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:36:22.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:22.571+0000 7f0321800640 1 --2- 172.21.15.5:0/3659168424 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f031c1079c0 0x7f031c074f70 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f0314009fd0 tx=0x7f031400ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:36:22.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:22.571+0000 7f0319600640 1 -- 172.21.15.5:0/3659168424 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f031400ce20 con 0x7f031c1079c0 2024-09-10T09:36:22.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:22.571+0000 7f0319600640 1 -- 172.21.15.5:0/3659168424 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0314005020 con 0x7f031c1079c0 2024-09-10T09:36:22.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:22.571+0000 7f0319600640 1 -- 172.21.15.5:0/3659168424 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0314010720 con 0x7f031c1079c0 2024-09-10T09:36:22.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:22.571+0000 7f0322200640 1 -- 172.21.15.5:0/3659168424 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f031c1a6a90 con 0x7f031c1079c0 2024-09-10T09:36:22.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:22.571+0000 7f0322200640 1 -- 172.21.15.5:0/3659168424 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f031c1a6f80 con 0x7f031c1079c0 2024-09-10T09:36:22.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:22.573+0000 7f0322200640 1 -- 172.21.15.5:0/3659168424 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f031c101e20 con 0x7f031c1079c0 2024-09-10T09:36:22.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:22.575+0000 7f0319600640 1 -- 172.21.15.5:0/3659168424 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f0314005540 con 0x7f031c1079c0 2024-09-10T09:36:22.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:22.576+0000 7f0319600640 1 --2- 172.21.15.5:0/3659168424 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f03040a2c10 0x7f03040a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:36:22.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:22.576+0000 7f0319600640 1 -- 172.21.15.5:0/3659168424 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9071+0+0 (secure 0 0 0) 0x7f03140cfbe0 con 0x7f031c1079c0 2024-09-10T09:36:22.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:22.577+0000 7f0320e00640 1 --2- 172.21.15.5:0/3659168424 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f03040a2c10 0x7f03040a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:36:22.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:22.578+0000 7f0320e00640 1 --2- 172.21.15.5:0/3659168424 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f03040a2c10 0x7f03040a50d0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f0310000c00 tx=0x7f0310009560 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:36:22.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:22.581+0000 7f0319600640 1 -- 172.21.15.5:0/3659168424 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f031409c640 con 0x7f031c1079c0 2024-09-10T09:36:22.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:22.794+0000 7f0322200640 1 -- 172.21.15.5:0/3659168424 --> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) -- 0x7f031c062630 con 0x7f03040a2c10 2024-09-10T09:36:22.795 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:36:22 smithi005 ceph-mon[30350]: from='client.24722 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T09:36:22.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:22.799+0000 7f0319600640 1 -- 172.21.15.5:0/3659168424 <== mgr.14150 v2:172.21.15.5:6800/2707348081 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f031c062630 con 0x7f03040a2c10 2024-09-10T09:36:22.800 INFO:teuthology.orchestra.run.smithi005.stdout:/volumes/qa/sv_0/bc101281-2871-40f2-8c3a-00df5dfd0497 2024-09-10T09:36:22.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:22.801+0000 7f0322200640 1 -- 172.21.15.5:0/3659168424 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f03040a2c10 msgr2=0x7f03040a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:36:22.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:22.801+0000 7f0322200640 1 --2- 172.21.15.5:0/3659168424 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f03040a2c10 0x7f03040a50d0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f0310000c00 tx=0x7f0310009560 comp rx=0 tx=0).stop 2024-09-10T09:36:22.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:22.802+0000 7f0322200640 1 -- 172.21.15.5:0/3659168424 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f031c1079c0 msgr2=0x7f031c074f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:36:22.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:22.802+0000 7f0322200640 1 --2- 172.21.15.5:0/3659168424 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f031c1079c0 0x7f031c074f70 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f0314009fd0 tx=0x7f031400ef10 comp rx=0 tx=0).stop 2024-09-10T09:36:22.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:22.802+0000 7f0322200640 1 -- 172.21.15.5:0/3659168424 shutdown_connections 2024-09-10T09:36:22.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:22.802+0000 7f0322200640 1 --2- 172.21.15.5:0/3659168424 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f03040a2c10 0x7f03040a50d0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:36:22.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:22.802+0000 7f0322200640 1 --2- 172.21.15.5:0/3659168424 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f031c1079c0 0x7f031c074f70 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:36:22.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:22.802+0000 7f0322200640 1 --2- 172.21.15.5:0/3659168424 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f031c102360 0x7f031c074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:36:22.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:22.802+0000 7f0322200640 1 --2- 172.21.15.5:0/3659168424 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f031c1019c0 0x7f031c0744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:36:22.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:22.802+0000 7f0322200640 1 -- 172.21.15.5:0/3659168424 >> 172.21.15.5:0/3659168424 conn(0x7f031c0fd740 msgr2=0x7f031c0ff0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:36:22.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:22.802+0000 7f0322200640 1 -- 172.21.15.5:0/3659168424 shutdown_connections 2024-09-10T09:36:22.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:22.802+0000 7f0322200640 1 -- 172.21.15.5:0/3659168424 wait complete. 2024-09-10T09:36:22.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:36:22 smithi159 ceph-mon[35513]: from='client.24722 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T09:36:22.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:36:22 smithi102 ceph-mon[35442]: from='client.24722 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T09:36:23.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:36:23 smithi005 ceph-mon[30350]: pgmap v332: 129 pgs: 129 active+clean; 594 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 5.0 KiB/s wr, 21 op/s 2024-09-10T09:36:23.704 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph fs subvolume create cephfs sv_1 --group_name qa 2024-09-10T09:36:23.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:36:23 smithi159 ceph-mon[35513]: pgmap v332: 129 pgs: 129 active+clean; 594 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 5.0 KiB/s wr, 21 op/s 2024-09-10T09:36:23.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:36:23 smithi102 ceph-mon[35442]: pgmap v332: 129 pgs: 129 active+clean; 594 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 5.0 KiB/s wr, 21 op/s 2024-09-10T09:36:24.087 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:36:24.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:24.673+0000 7f335fe00640 1 -- 172.21.15.5:0/897503902 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f33601019c0 msgr2=0x7f3360101e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:36:24.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:24.673+0000 7f335fe00640 1 --2- 172.21.15.5:0/897503902 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f33601019c0 0x7f3360101e20 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f334c009930 tx=0x7f334c02a420 comp rx=0 tx=0).stop 2024-09-10T09:36:24.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:24.673+0000 7f335fe00640 1 -- 172.21.15.5:0/897503902 shutdown_connections 2024-09-10T09:36:24.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:24.673+0000 7f335fe00640 1 --2- 172.21.15.5:0/897503902 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3360102360 0x7f336010e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:36:24.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:24.673+0000 7f335fe00640 1 --2- 172.21.15.5:0/897503902 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f33601019c0 0x7f3360101e20 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:36:24.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:24.673+0000 7f335fe00640 1 --2- 172.21.15.5:0/897503902 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f33601079c0 0x7f3360107da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:36:24.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:24.673+0000 7f335fe00640 1 -- 172.21.15.5:0/897503902 >> 172.21.15.5:0/897503902 conn(0x7f33600fd740 msgr2=0x7f33600ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:36:24.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:24.673+0000 7f335fe00640 1 -- 172.21.15.5:0/897503902 shutdown_connections 2024-09-10T09:36:24.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:24.673+0000 7f335fe00640 1 -- 172.21.15.5:0/897503902 wait complete. 2024-09-10T09:36:24.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:24.675+0000 7f335fe00640 1 Processor -- start 2024-09-10T09:36:24.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:24.675+0000 7f335fe00640 1 -- start start 2024-09-10T09:36:24.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:24.675+0000 7f335fe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f33601019c0 0x7f33601982b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:36:24.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:24.675+0000 7f335fe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3360102360 0x7f33601987f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:36:24.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:24.675+0000 7f335fe00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f33601079c0 0x7f336019cb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:36:24.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:24.675+0000 7f335fe00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f3360111e10 con 0x7f3360102360 2024-09-10T09:36:24.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:24.675+0000 7f335fe00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f3360111c90 con 0x7f33601019c0 2024-09-10T09:36:24.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:24.675+0000 7f335fe00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f3360111f90 con 0x7f33601079c0 2024-09-10T09:36:24.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:24.676+0000 7f335ea00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f33601019c0 0x7f33601982b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:36:24.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:24.676+0000 7f335ea00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f33601019c0 0x7f33601982b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:49856/0 (socket says 172.21.15.5:49856) 2024-09-10T09:36:24.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:24.676+0000 7f335ea00640 1 -- 172.21.15.5:0/512344152 learned_addr learned my addr 172.21.15.5:0/512344152 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:36:24.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:24.676+0000 7f335e000640 1 --2- 172.21.15.5:0/512344152 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3360102360 0x7f33601987f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:36:24.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:24.676+0000 7f335f400640 1 --2- 172.21.15.5:0/512344152 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f33601079c0 0x7f336019cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:36:24.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:24.676+0000 7f335ea00640 1 -- 172.21.15.5:0/512344152 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f33601079c0 msgr2=0x7f336019cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:36:24.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:24.676+0000 7f335ea00640 1 --2- 172.21.15.5:0/512344152 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f33601079c0 0x7f336019cb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:36:24.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:24.676+0000 7f335ea00640 1 -- 172.21.15.5:0/512344152 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3360102360 msgr2=0x7f33601987f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:36:24.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:24.676+0000 7f335ea00640 1 --2- 172.21.15.5:0/512344152 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3360102360 0x7f33601987f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:36:24.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:24.676+0000 7f335ea00640 1 -- 172.21.15.5:0/512344152 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f336019d260 con 0x7f33601019c0 2024-09-10T09:36:24.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:24.676+0000 7f335f400640 1 --2- 172.21.15.5:0/512344152 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f33601079c0 0x7f336019cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:36:24.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:24.676+0000 7f335ea00640 1 --2- 172.21.15.5:0/512344152 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f33601019c0 0x7f33601982b0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f335000e9f0 tx=0x7f335000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:36:24.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:24.677+0000 7f3357400640 1 -- 172.21.15.5:0/512344152 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f335000cd80 con 0x7f33601019c0 2024-09-10T09:36:24.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:24.677+0000 7f335fe00640 1 -- 172.21.15.5:0/512344152 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f336019d550 con 0x7f33601019c0 2024-09-10T09:36:24.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:24.677+0000 7f3357400640 1 -- 172.21.15.5:0/512344152 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3350010910 con 0x7f33601019c0 2024-09-10T09:36:24.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:24.677+0000 7f335fe00640 1 -- 172.21.15.5:0/512344152 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33601a4e30 con 0x7f33601019c0 2024-09-10T09:36:24.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:24.677+0000 7f3357400640 1 -- 172.21.15.5:0/512344152 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33500188f0 con 0x7f33601019c0 2024-09-10T09:36:24.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:24.678+0000 7f335fe00640 1 -- 172.21.15.5:0/512344152 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3360101e20 con 0x7f33601019c0 2024-09-10T09:36:24.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:24.680+0000 7f3357400640 1 -- 172.21.15.5:0/512344152 <== mon.2 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f3350010ac0 con 0x7f33601019c0 2024-09-10T09:36:24.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:24.681+0000 7f3357400640 1 --2- 172.21.15.5:0/512344152 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f33440a2c90 0x7f33440a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:36:24.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:24.682+0000 7f3357400640 1 -- 172.21.15.5:0/512344152 <== mon.2 v2:172.21.15.102:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9071+0+0 (secure 0 0 0) 0x7f33500cfd20 con 0x7f33601019c0 2024-09-10T09:36:24.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:24.682+0000 7f335e000640 1 --2- 172.21.15.5:0/512344152 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f33440a2c90 0x7f33440a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:36:24.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:24.683+0000 7f335e000640 1 --2- 172.21.15.5:0/512344152 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f33440a2c90 0x7f33440a5150 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f334c006010 tx=0x7f334c000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:36:24.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:24.683+0000 7f3357400640 1 -- 172.21.15.5:0/512344152 <== mon.2 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f335009c130 con 0x7f33601019c0 2024-09-10T09:36:24.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:36:24 smithi005 ceph-mon[30350]: from='client.14712 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T09:36:24.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:36:24 smithi159 ceph-mon[35513]: from='client.14712 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T09:36:24.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:36:24 smithi102 ceph-mon[35442]: from='client.14712 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T09:36:24.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:24.901+0000 7f335fe00640 1 -- 172.21.15.5:0/512344152 --> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f3360062630 con 0x7f33440a2c90 2024-09-10T09:36:24.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:24.924+0000 7f3357400640 1 -- 172.21.15.5:0/512344152 <== mgr.14150 v2:172.21.15.5:6800/2707348081 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f3360062630 con 0x7f33440a2c90 2024-09-10T09:36:24.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:24.926+0000 7f335fe00640 1 -- 172.21.15.5:0/512344152 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f33440a2c90 msgr2=0x7f33440a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:36:24.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:24.926+0000 7f335fe00640 1 --2- 172.21.15.5:0/512344152 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f33440a2c90 0x7f33440a5150 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f334c006010 tx=0x7f334c000bc0 comp rx=0 tx=0).stop 2024-09-10T09:36:24.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:24.926+0000 7f335fe00640 1 -- 172.21.15.5:0/512344152 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f33601019c0 msgr2=0x7f33601982b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:36:24.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:24.926+0000 7f335fe00640 1 --2- 172.21.15.5:0/512344152 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f33601019c0 0x7f33601982b0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f335000e9f0 tx=0x7f335000eeb0 comp rx=0 tx=0).stop 2024-09-10T09:36:24.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:24.926+0000 7f335fe00640 1 -- 172.21.15.5:0/512344152 shutdown_connections 2024-09-10T09:36:24.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:24.926+0000 7f335fe00640 1 --2- 172.21.15.5:0/512344152 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f33440a2c90 0x7f33440a5150 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:36:24.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:24.926+0000 7f335fe00640 1 --2- 172.21.15.5:0/512344152 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f33601079c0 0x7f336019cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:36:24.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:24.926+0000 7f335fe00640 1 --2- 172.21.15.5:0/512344152 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3360102360 0x7f33601987f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:36:24.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:24.926+0000 7f335fe00640 1 --2- 172.21.15.5:0/512344152 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f33601019c0 0x7f33601982b0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:36:24.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:24.926+0000 7f335fe00640 1 -- 172.21.15.5:0/512344152 >> 172.21.15.5:0/512344152 conn(0x7f33600fd740 msgr2=0x7f33600fdec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:36:24.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:24.926+0000 7f335fe00640 1 -- 172.21.15.5:0/512344152 shutdown_connections 2024-09-10T09:36:24.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:24.926+0000 7f335fe00640 1 -- 172.21.15.5:0/512344152 wait complete. 2024-09-10T09:36:25.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:36:25 smithi005 ceph-mon[30350]: pgmap v333: 129 pgs: 129 active+clean; 594 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 5.0 KiB/s wr, 21 op/s 2024-09-10T09:36:25.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:36:25 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:36:25.633 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-09-10T09:36:25.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:36:25 smithi102 ceph-mon[35442]: pgmap v333: 129 pgs: 129 active+clean; 594 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 5.0 KiB/s wr, 21 op/s 2024-09-10T09:36:25.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:36:25 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:36:25.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:36:25 smithi159 ceph-mon[35513]: pgmap v333: 129 pgs: 129 active+clean; 594 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 5.0 KiB/s wr, 21 op/s 2024-09-10T09:36:25.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:36:25 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:36:26.695 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:36:26.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:36:26 smithi005 ceph-mon[30350]: from='client.24734 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T09:36:26.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:36:26 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:36:26.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:36:26 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:36:26.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:36:26 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:36:26.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:36:26 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:36:26.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:36:26 smithi159 ceph-mon[35513]: from='client.24734 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T09:36:26.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:36:26 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:36:26.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:36:26 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:36:26.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:36:26 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:36:26.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:36:26 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:36:26.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:36:26 smithi102 ceph-mon[35442]: from='client.24734 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T09:36:26.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:36:26 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:36:26.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:36:26 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:36:26.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:36:26 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:36:26.885 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:36:26 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:36:27.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:27.268+0000 7fb2bae00640 1 -- 172.21.15.5:0/3692820334 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb2b41079c0 msgr2=0x7fb2b4107da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:36:27.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:27.268+0000 7fb2bae00640 1 --2- 172.21.15.5:0/3692820334 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb2b41079c0 0x7fb2b4107da0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fb2a8009930 tx=0x7fb2a802a440 comp rx=0 tx=0).stop 2024-09-10T09:36:27.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:27.268+0000 7fb2bae00640 1 -- 172.21.15.5:0/3692820334 shutdown_connections 2024-09-10T09:36:27.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:27.268+0000 7fb2bae00640 1 --2- 172.21.15.5:0/3692820334 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb2b4102360 0x7fb2b410e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:36:27.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:27.268+0000 7fb2bae00640 1 --2- 172.21.15.5:0/3692820334 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb2b41019c0 0x7fb2b4101e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:36:27.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:27.268+0000 7fb2bae00640 1 --2- 172.21.15.5:0/3692820334 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb2b41079c0 0x7fb2b4107da0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:36:27.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:27.268+0000 7fb2bae00640 1 -- 172.21.15.5:0/3692820334 >> 172.21.15.5:0/3692820334 conn(0x7fb2b40fd740 msgr2=0x7fb2b40ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:36:27.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:27.269+0000 7fb2bae00640 1 -- 172.21.15.5:0/3692820334 shutdown_connections 2024-09-10T09:36:27.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:27.269+0000 7fb2bae00640 1 -- 172.21.15.5:0/3692820334 wait complete. 2024-09-10T09:36:27.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:27.270+0000 7fb2bae00640 1 Processor -- start 2024-09-10T09:36:27.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:27.270+0000 7fb2bae00640 1 -- start start 2024-09-10T09:36:27.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:27.270+0000 7fb2bae00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb2b41019c0 0x7fb2b40744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:36:27.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:27.270+0000 7fb2bae00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb2b4102360 0x7fb2b4074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:36:27.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:27.270+0000 7fb2bae00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb2b41079c0 0x7fb2b4074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:36:27.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:27.270+0000 7fb2bae00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fb2b4111e10 con 0x7fb2b4102360 2024-09-10T09:36:27.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:27.270+0000 7fb2bae00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fb2b4111c90 con 0x7fb2b41019c0 2024-09-10T09:36:27.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:27.270+0000 7fb2bae00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fb2b4111f90 con 0x7fb2b41079c0 2024-09-10T09:36:27.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:27.270+0000 7fb2b9000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb2b4102360 0x7fb2b4074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:36:27.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:27.270+0000 7fb2b9000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb2b4102360 0x7fb2b4074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36836/0 (socket says 172.21.15.5:36836) 2024-09-10T09:36:27.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:27.270+0000 7fb2b9a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb2b41019c0 0x7fb2b40744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:36:27.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:27.271+0000 7fb2b9000640 1 -- 172.21.15.5:0/457792737 learned_addr learned my addr 172.21.15.5:0/457792737 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:36:27.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:27.271+0000 7fb2ba400640 1 --2- 172.21.15.5:0/457792737 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb2b41079c0 0x7fb2b4074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:36:27.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:27.271+0000 7fb2b9000640 1 -- 172.21.15.5:0/457792737 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb2b41079c0 msgr2=0x7fb2b4074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:36:27.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:27.271+0000 7fb2b9000640 1 --2- 172.21.15.5:0/457792737 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb2b41079c0 0x7fb2b4074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:36:27.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:27.271+0000 7fb2b9000640 1 -- 172.21.15.5:0/457792737 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb2b41019c0 msgr2=0x7fb2b40744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:36:27.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:27.271+0000 7fb2b9000640 1 --2- 172.21.15.5:0/457792737 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb2b41019c0 0x7fb2b40744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:36:27.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:27.271+0000 7fb2b9000640 1 -- 172.21.15.5:0/457792737 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2b41a6730 con 0x7fb2b4102360 2024-09-10T09:36:27.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:27.271+0000 7fb2b9000640 1 --2- 172.21.15.5:0/457792737 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb2b4102360 0x7fb2b4074a30 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fb2a000e9f0 tx=0x7fb2a000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:36:27.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:27.272+0000 7fb2b2000640 1 -- 172.21.15.5:0/457792737 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2a000cd80 con 0x7fb2b4102360 2024-09-10T09:36:27.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:27.272+0000 7fb2b2000640 1 -- 172.21.15.5:0/457792737 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb2a0010910 con 0x7fb2b4102360 2024-09-10T09:36:27.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:27.272+0000 7fb2b2000640 1 -- 172.21.15.5:0/457792737 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2a0018990 con 0x7fb2b4102360 2024-09-10T09:36:27.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:27.272+0000 7fb2bae00640 1 -- 172.21.15.5:0/457792737 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2b41a6a20 con 0x7fb2b4102360 2024-09-10T09:36:27.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:27.272+0000 7fb2bae00640 1 -- 172.21.15.5:0/457792737 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2b41a6f10 con 0x7fb2b4102360 2024-09-10T09:36:27.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:27.273+0000 7fb2bae00640 1 -- 172.21.15.5:0/457792737 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb2b4101e20 con 0x7fb2b4102360 2024-09-10T09:36:27.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:27.275+0000 7fb2b2000640 1 -- 172.21.15.5:0/457792737 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7fb2a0010ac0 con 0x7fb2b4102360 2024-09-10T09:36:27.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:27.276+0000 7fb2b2000640 1 --2- 172.21.15.5:0/457792737 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fb29c0a2c90 0x7fb29c0a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:36:27.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:27.277+0000 7fb2b2000640 1 -- 172.21.15.5:0/457792737 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9071+0+0 (secure 0 0 0) 0x7fb2a00cf7e0 con 0x7fb2b4102360 2024-09-10T09:36:27.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:27.277+0000 7fb2b9a00640 1 --2- 172.21.15.5:0/457792737 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fb29c0a2c90 0x7fb29c0a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:36:27.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:27.278+0000 7fb2b9a00640 1 --2- 172.21.15.5:0/457792737 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fb29c0a2c90 0x7fb29c0a5150 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fb2a8006010 tx=0x7fb2a8000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:36:27.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:27.281+0000 7fb2b2000640 1 -- 172.21.15.5:0/457792737 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb2a009bbf0 con 0x7fb2b4102360 2024-09-10T09:36:27.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:27.506+0000 7fb2bae00640 1 -- 172.21.15.5:0/457792737 --> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) -- 0x7fb2b4062630 con 0x7fb29c0a2c90 2024-09-10T09:36:27.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:27.512+0000 7fb2b2000640 1 -- 172.21.15.5:0/457792737 <== mgr.14150 v2:172.21.15.5:6800/2707348081 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7fb2b4062630 con 0x7fb29c0a2c90 2024-09-10T09:36:27.513 INFO:teuthology.orchestra.run.smithi005.stdout:/volumes/qa/sv_1/8c062e55-cb3c-4c81-90c0-78100d07381f 2024-09-10T09:36:27.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:27.515+0000 7fb2bae00640 1 -- 172.21.15.5:0/457792737 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fb29c0a2c90 msgr2=0x7fb29c0a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:36:27.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:27.515+0000 7fb2bae00640 1 --2- 172.21.15.5:0/457792737 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fb29c0a2c90 0x7fb29c0a5150 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fb2a8006010 tx=0x7fb2a8000bc0 comp rx=0 tx=0).stop 2024-09-10T09:36:27.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:27.515+0000 7fb2bae00640 1 -- 172.21.15.5:0/457792737 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb2b4102360 msgr2=0x7fb2b4074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:36:27.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:27.515+0000 7fb2bae00640 1 --2- 172.21.15.5:0/457792737 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb2b4102360 0x7fb2b4074a30 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fb2a000e9f0 tx=0x7fb2a000eeb0 comp rx=0 tx=0).stop 2024-09-10T09:36:27.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:27.515+0000 7fb2bae00640 1 -- 172.21.15.5:0/457792737 shutdown_connections 2024-09-10T09:36:27.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:27.515+0000 7fb2bae00640 1 --2- 172.21.15.5:0/457792737 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fb29c0a2c90 0x7fb29c0a5150 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:36:27.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:27.515+0000 7fb2bae00640 1 --2- 172.21.15.5:0/457792737 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb2b41079c0 0x7fb2b4074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:36:27.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:27.515+0000 7fb2bae00640 1 --2- 172.21.15.5:0/457792737 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb2b4102360 0x7fb2b4074a30 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:36:27.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:27.515+0000 7fb2bae00640 1 --2- 172.21.15.5:0/457792737 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb2b41019c0 0x7fb2b40744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:36:27.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:27.515+0000 7fb2bae00640 1 -- 172.21.15.5:0/457792737 >> 172.21.15.5:0/457792737 conn(0x7fb2b40fd740 msgr2=0x7fb2b40feea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:36:27.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:27.515+0000 7fb2bae00640 1 -- 172.21.15.5:0/457792737 shutdown_connections 2024-09-10T09:36:27.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:27.515+0000 7fb2bae00640 1 -- 172.21.15.5:0/457792737 wait complete. 2024-09-10T09:36:27.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:36:27 smithi005 ceph-mon[30350]: pgmap v334: 129 pgs: 129 active+clean; 612 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 6.9 KiB/s wr, 26 op/s 2024-09-10T09:36:27.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:36:27 smithi005 ceph-mon[30350]: from='client.14724 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T09:36:27.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:36:27 smithi159 ceph-mon[35513]: pgmap v334: 129 pgs: 129 active+clean; 612 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 6.9 KiB/s wr, 26 op/s 2024-09-10T09:36:27.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:36:27 smithi159 ceph-mon[35513]: from='client.14724 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T09:36:27.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:36:27 smithi102 ceph-mon[35442]: pgmap v334: 129 pgs: 129 active+clean; 612 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 6.9 KiB/s wr, 26 op/s 2024-09-10T09:36:27.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:36:27 smithi102 ceph-mon[35442]: from='client.14724 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T09:36:28.372 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T09:36:28.776 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:36:29.236 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:36:28 smithi005 ceph-mon[30350]: pgmap v335: 129 pgs: 129 active+clean; 612 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 4.2 KiB/s wr, 17 op/s 2024-09-10T09:36:29.237 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:36:28 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:36:29.237 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:36:28 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:36:29.237 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:36:28 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:36:29.237 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:36:28 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:36:29.237 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:36:28 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:36:29.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:36:28 smithi159 ceph-mon[35513]: pgmap v335: 129 pgs: 129 active+clean; 612 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 4.2 KiB/s wr, 17 op/s 2024-09-10T09:36:29.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:36:28 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:36:29.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:36:28 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:36:29.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:36:28 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:36:29.351 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:36:28 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:36:29.351 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:36:28 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:36:29.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:29.349+0000 7f52dd600640 1 -- 172.21.15.5:0/2244083650 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f52d8112b90 msgr2=0x7f52d8114f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:36:29.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:29.349+0000 7f52dd600640 1 --2- 172.21.15.5:0/2244083650 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f52d8112b90 0x7f52d8114f80 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f52d0009ca0 tx=0x7f52d002a950 comp rx=0 tx=0).stop 2024-09-10T09:36:29.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:29.349+0000 7f52dd600640 1 -- 172.21.15.5:0/2244083650 shutdown_connections 2024-09-10T09:36:29.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:29.349+0000 7f52dd600640 1 --2- 172.21.15.5:0/2244083650 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f52d8112b90 0x7f52d8114f80 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:36:29.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:29.349+0000 7f52dd600640 1 --2- 172.21.15.5:0/2244083650 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f52d8074eb0 0x7f52d8112650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:36:29.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:29.349+0000 7f52dd600640 1 --2- 172.21.15.5:0/2244083650 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f52d8074590 0x7f52d8074970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:36:29.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:29.349+0000 7f52dd600640 1 -- 172.21.15.5:0/2244083650 >> 172.21.15.5:0/2244083650 conn(0x7f52d80fd8c0 msgr2=0x7f52d80ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:36:29.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:29.349+0000 7f52dd600640 1 -- 172.21.15.5:0/2244083650 shutdown_connections 2024-09-10T09:36:29.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:29.349+0000 7f52dd600640 1 -- 172.21.15.5:0/2244083650 wait complete. 2024-09-10T09:36:29.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:29.350+0000 7f52dd600640 1 Processor -- start 2024-09-10T09:36:29.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:29.350+0000 7f52dd600640 1 -- start start 2024-09-10T09:36:29.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:29.350+0000 7f52dd600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f52d8074590 0x7f52d819eb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:36:29.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:29.350+0000 7f52dd600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f52d8074eb0 0x7f52d819f050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:36:29.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:29.350+0000 7f52dd600640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f52d8112b90 0x7f52d81a3410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:36:29.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:29.350+0000 7f52dd600640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f52d81185c0 con 0x7f52d8074590 2024-09-10T09:36:29.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:29.350+0000 7f52dd600640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f52d8118440 con 0x7f52d8074eb0 2024-09-10T09:36:29.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:29.350+0000 7f52dd600640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f52d8118740 con 0x7f52d8112b90 2024-09-10T09:36:29.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:29.351+0000 7f52d6a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f52d8074590 0x7f52d819eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:36:29.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:29.351+0000 7f52d6a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f52d8074590 0x7f52d819eb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36862/0 (socket says 172.21.15.5:36862) 2024-09-10T09:36:29.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:29.351+0000 7f52d6a00640 1 -- 172.21.15.5:0/3940209351 learned_addr learned my addr 172.21.15.5:0/3940209351 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:36:29.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:29.351+0000 7f52d7400640 1 --2- 172.21.15.5:0/3940209351 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f52d8112b90 0x7f52d81a3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:36:29.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:29.351+0000 7f52cfe00640 1 --2- 172.21.15.5:0/3940209351 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f52d8074eb0 0x7f52d819f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:36:29.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:29.351+0000 7f52d6a00640 1 -- 172.21.15.5:0/3940209351 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f52d8112b90 msgr2=0x7f52d81a3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:36:29.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:29.351+0000 7f52d6a00640 1 --2- 172.21.15.5:0/3940209351 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f52d8112b90 0x7f52d81a3410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:36:29.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:29.351+0000 7f52d6a00640 1 -- 172.21.15.5:0/3940209351 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f52d8074eb0 msgr2=0x7f52d819f050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:36:29.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:29.351+0000 7f52d6a00640 1 --2- 172.21.15.5:0/3940209351 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f52d8074eb0 0x7f52d819f050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:36:29.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:29.351+0000 7f52d6a00640 1 -- 172.21.15.5:0/3940209351 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52d81a3b40 con 0x7f52d8074590 2024-09-10T09:36:29.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:29.351+0000 7f52cfe00640 1 --2- 172.21.15.5:0/3940209351 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f52d8074eb0 0x7f52d819f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:36:29.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:29.351+0000 7f52d6a00640 1 --2- 172.21.15.5:0/3940209351 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f52d8074590 0x7f52d819eb10 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f52c800ea60 tx=0x7f52c800ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:36:29.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:29.352+0000 7f52cf400640 1 -- 172.21.15.5:0/3940209351 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52c800cde0 con 0x7f52d8074590 2024-09-10T09:36:29.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:29.352+0000 7f52dd600640 1 -- 172.21.15.5:0/3940209351 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52d81a3e30 con 0x7f52d8074590 2024-09-10T09:36:29.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:29.352+0000 7f52cf400640 1 -- 172.21.15.5:0/3940209351 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f52c80035f0 con 0x7f52d8074590 2024-09-10T09:36:29.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:29.352+0000 7f52cf400640 1 -- 172.21.15.5:0/3940209351 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52c8010670 con 0x7f52d8074590 2024-09-10T09:36:29.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:29.352+0000 7f52dd600640 1 -- 172.21.15.5:0/3940209351 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52d81ab6c0 con 0x7f52d8074590 2024-09-10T09:36:29.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:29.353+0000 7f52dd600640 1 -- 172.21.15.5:0/3940209351 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f52d8074970 con 0x7f52d8074590 2024-09-10T09:36:29.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:29.355+0000 7f52cf400640 1 -- 172.21.15.5:0/3940209351 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f52c80037a0 con 0x7f52d8074590 2024-09-10T09:36:29.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:29.356+0000 7f52cf400640 1 --2- 172.21.15.5:0/3940209351 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f52bc0a2c90 0x7f52bc0a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:36:29.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:29.356+0000 7f52cf400640 1 -- 172.21.15.5:0/3940209351 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9071+0+0 (secure 0 0 0) 0x7f52c80cf860 con 0x7f52d8074590 2024-09-10T09:36:29.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:29.356+0000 7f52cfe00640 1 --2- 172.21.15.5:0/3940209351 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f52bc0a2c90 0x7f52bc0a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:36:29.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:29.357+0000 7f52cfe00640 1 --2- 172.21.15.5:0/3940209351 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f52bc0a2c90 0x7f52bc0a5150 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f52d80fecf0 tx=0x7f52c0005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:36:29.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:29.360+0000 7f52cf400640 1 -- 172.21.15.5:0/3940209351 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f52c809c2c0 con 0x7f52d8074590 2024-09-10T09:36:29.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:36:28 smithi102 ceph-mon[35442]: pgmap v335: 129 pgs: 129 active+clean; 612 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 4.2 KiB/s wr, 17 op/s 2024-09-10T09:36:29.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:36:28 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:36:29.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:36:28 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:36:29.385 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:36:28 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:36:29.385 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:36:28 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:36:29.385 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:36:28 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:36:29.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:29.586+0000 7f52dd600640 1 -- 172.21.15.5:0/3940209351 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f52d8105030 con 0x7f52d8074590 2024-09-10T09:36:29.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:29.587+0000 7f52cf400640 1 -- 172.21.15.5:0/3940209351 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11969 (secure 0 0 0) 0x7f52c809bbc0 con 0x7f52d8074590 2024-09-10T09:36:29.588 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-10T09:36:29.588 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":29,"btime":"2024-09-10T09:36:16:492171+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14454,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/2970594311","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2970594311},{"type":"v1","addr":"172.21.15.5:6835","nonce":2970594311}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14514,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6839/3616502174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":3616502174},{"type":"v1","addr":"172.21.15.5:6839","nonce":3616502174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T09:35:29.626287+0000","modified":"2024-09-10T09:36:15.764196+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24542,"mds_1":24535,"mds_2":24511,"mds_3":24506,"mds_4":24443},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14520":{"gid":14520,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/2073599842","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":2073599842},{"type":"v1","addr":"172.21.15.5:6841","nonce":2073599842}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"k","rank":4,"incarnation":26,"state":"up:active","state_seq":24,"addr":"172.21.15.102:6833/2457007427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":2457007427},{"type":"v1","addr":"172.21.15.102:6833","nonce":2457007427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/356445159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":356445159},{"type":"v1","addr":"172.21.15.159:6833","nonce":356445159}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6837/2894314507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":2894314507},{"type":"v1","addr":"172.21.15.5:6837","nonce":2894314507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.102:6835/1647549111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":1647549111},{"type":"v1","addr":"172.21.15.102:6835","nonce":1647549111}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6835/952657242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":952657242},{"type":"v1","addr":"172.21.15.159:6835","nonce":952657242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":19,"addr":"172.21.15.102:6837/2689776764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":2689776764},{"type":"v1","addr":"172.21.15.102:6837","nonce":2689776764}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.159:6837/620627639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":620627639},{"type":"v1","addr":"172.21.15.159:6837","nonce":620627639}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24535":{"gid":24535,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":18,"addr":"172.21.15.159:6839/2073739599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":2073739599},{"type":"v1","addr":"172.21.15.159:6839","nonce":2073739599}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24542":{"gid":24542,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.102:6839/335409749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6838","nonce":335409749},{"type":"v1","addr":"172.21.15.102:6839","nonce":335409749}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24542,"qdb_cluster":[24443,24506,24511,24535,24542]},"id":1}]} 2024-09-10T09:36:29.589 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 29 2024-09-10T09:36:29.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:29.589+0000 7f52dd600640 1 -- 172.21.15.5:0/3940209351 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f52bc0a2c90 msgr2=0x7f52bc0a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:36:29.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:29.589+0000 7f52dd600640 1 --2- 172.21.15.5:0/3940209351 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f52bc0a2c90 0x7f52bc0a5150 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f52d80fecf0 tx=0x7f52c0005f90 comp rx=0 tx=0).stop 2024-09-10T09:36:29.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:29.589+0000 7f52dd600640 1 -- 172.21.15.5:0/3940209351 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f52d8074590 msgr2=0x7f52d819eb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:36:29.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:29.589+0000 7f52dd600640 1 --2- 172.21.15.5:0/3940209351 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f52d8074590 0x7f52d819eb10 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f52c800ea60 tx=0x7f52c800ef20 comp rx=0 tx=0).stop 2024-09-10T09:36:29.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:29.590+0000 7f52dd600640 1 -- 172.21.15.5:0/3940209351 shutdown_connections 2024-09-10T09:36:29.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:29.590+0000 7f52dd600640 1 --2- 172.21.15.5:0/3940209351 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f52bc0a2c90 0x7f52bc0a5150 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:36:29.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:29.590+0000 7f52dd600640 1 --2- 172.21.15.5:0/3940209351 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f52d8112b90 0x7f52d81a3410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:36:29.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:29.590+0000 7f52dd600640 1 --2- 172.21.15.5:0/3940209351 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f52d8074eb0 0x7f52d819f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:36:29.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:29.590+0000 7f52dd600640 1 --2- 172.21.15.5:0/3940209351 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f52d8074590 0x7f52d819eb10 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:36:29.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:29.590+0000 7f52dd600640 1 -- 172.21.15.5:0/3940209351 >> 172.21.15.5:0/3940209351 conn(0x7f52d80fd8c0 msgr2=0x7f52d80ff050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:36:29.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:29.590+0000 7f52dd600640 1 -- 172.21.15.5:0/3940209351 shutdown_connections 2024-09-10T09:36:29.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:29.590+0000 7f52dd600640 1 -- 172.21.15.5:0/3940209351 wait complete. 2024-09-10T09:36:30.306 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:36:29 smithi005 ceph-mon[30350]: from='client.? 172.21.15.5:0/3940209351' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:36:30.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:36:29 smithi159 ceph-mon[35513]: from='client.? 172.21.15.5:0/3940209351' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:36:30.354 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-10T09:36:30.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:36:29 smithi102 ceph-mon[35442]: from='client.? 172.21.15.5:0/3940209351' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:36:30.735 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:36:31.187 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:36:30 smithi005 ceph-mon[30350]: pgmap v336: 129 pgs: 129 active+clean; 612 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 3.9 KiB/s wr, 15 op/s 2024-09-10T09:36:31.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:31.300+0000 7fbc40a00640 1 -- 172.21.15.5:0/2656788812 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbc3c1079c0 msgr2=0x7fbc3c107da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:36:31.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:31.300+0000 7fbc40a00640 1 --2- 172.21.15.5:0/2656788812 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbc3c1079c0 0x7fbc3c107da0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fbc2c009cd0 tx=0x7fbc2c02a8e0 comp rx=0 tx=0).stop 2024-09-10T09:36:31.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:31.300+0000 7fbc40a00640 1 -- 172.21.15.5:0/2656788812 shutdown_connections 2024-09-10T09:36:31.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:31.300+0000 7fbc40a00640 1 --2- 172.21.15.5:0/2656788812 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbc3c102360 0x7fbc3c10e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:36:31.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:31.300+0000 7fbc40a00640 1 --2- 172.21.15.5:0/2656788812 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbc3c1019c0 0x7fbc3c101e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:36:31.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:31.300+0000 7fbc40a00640 1 --2- 172.21.15.5:0/2656788812 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbc3c1079c0 0x7fbc3c107da0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:36:31.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:31.300+0000 7fbc40a00640 1 -- 172.21.15.5:0/2656788812 >> 172.21.15.5:0/2656788812 conn(0x7fbc3c0fd740 msgr2=0x7fbc3c0ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:36:31.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:31.300+0000 7fbc40a00640 1 -- 172.21.15.5:0/2656788812 shutdown_connections 2024-09-10T09:36:31.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:31.300+0000 7fbc40a00640 1 -- 172.21.15.5:0/2656788812 wait complete. 2024-09-10T09:36:31.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:31.301+0000 7fbc40a00640 1 Processor -- start 2024-09-10T09:36:31.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:31.301+0000 7fbc40a00640 1 -- start start 2024-09-10T09:36:31.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:31.302+0000 7fbc40a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbc3c1019c0 0x7fbc3c19a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:36:31.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:31.302+0000 7fbc40a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbc3c102360 0x7fbc3c19a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:36:31.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:31.302+0000 7fbc40a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbc3c1079c0 0x7fbc3c19ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:36:31.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:31.302+0000 7fbc40a00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fbc3c111d90 con 0x7fbc3c1019c0 2024-09-10T09:36:31.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:31.302+0000 7fbc40a00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fbc3c111c10 con 0x7fbc3c1079c0 2024-09-10T09:36:31.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:31.302+0000 7fbc40a00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fbc3c111f10 con 0x7fbc3c102360 2024-09-10T09:36:31.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:31.302+0000 7fbc3b400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbc3c1019c0 0x7fbc3c19a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:36:31.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:31.302+0000 7fbc3b400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbc3c1019c0 0x7fbc3c19a4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46580/0 (socket says 172.21.15.5:46580) 2024-09-10T09:36:31.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:31.302+0000 7fbc3b400640 1 -- 172.21.15.5:0/2064461357 learned_addr learned my addr 172.21.15.5:0/2064461357 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:36:31.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:31.302+0000 7fbc32a00640 1 --2- 172.21.15.5:0/2064461357 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbc3c102360 0x7fbc3c19a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:36:31.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:31.302+0000 7fbc3be00640 1 --2- 172.21.15.5:0/2064461357 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbc3c1079c0 0x7fbc3c19ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:36:31.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:31.302+0000 7fbc3b400640 1 -- 172.21.15.5:0/2064461357 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbc3c102360 msgr2=0x7fbc3c19a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:36:31.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:31.302+0000 7fbc3b400640 1 --2- 172.21.15.5:0/2064461357 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbc3c102360 0x7fbc3c19a9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:36:31.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:31.302+0000 7fbc3b400640 1 -- 172.21.15.5:0/2064461357 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbc3c1079c0 msgr2=0x7fbc3c19ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:36:31.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:31.302+0000 7fbc3b400640 1 --2- 172.21.15.5:0/2064461357 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbc3c1079c0 0x7fbc3c19ed70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:36:31.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:31.303+0000 7fbc3b400640 1 -- 172.21.15.5:0/2064461357 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc3c19f450 con 0x7fbc3c1019c0 2024-09-10T09:36:31.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:31.303+0000 7fbc32a00640 1 --2- 172.21.15.5:0/2064461357 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbc3c102360 0x7fbc3c19a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:36:31.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:31.304+0000 7fbc3b400640 1 --2- 172.21.15.5:0/2064461357 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbc3c1019c0 0x7fbc3c19a4a0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fbc2c005e30 tx=0x7fbc2c032fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:36:31.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:31.304+0000 7fbc38c00640 1 -- 172.21.15.5:0/2064461357 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc2c004040 con 0x7fbc3c1019c0 2024-09-10T09:36:31.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:31.304+0000 7fbc38c00640 1 -- 172.21.15.5:0/2064461357 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbc2c0046c0 con 0x7fbc3c1019c0 2024-09-10T09:36:31.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:31.304+0000 7fbc38c00640 1 -- 172.21.15.5:0/2064461357 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc2c030e30 con 0x7fbc3c1019c0 2024-09-10T09:36:31.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:31.304+0000 7fbc40a00640 1 -- 172.21.15.5:0/2064461357 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc3c19f6e0 con 0x7fbc3c1019c0 2024-09-10T09:36:31.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:31.304+0000 7fbc40a00640 1 -- 172.21.15.5:0/2064461357 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc3c1a7020 con 0x7fbc3c1019c0 2024-09-10T09:36:31.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:31.305+0000 7fbc40a00640 1 -- 172.21.15.5:0/2064461357 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbc3c101e20 con 0x7fbc3c1019c0 2024-09-10T09:36:31.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:31.307+0000 7fbc38c00640 1 -- 172.21.15.5:0/2064461357 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7fbc2c0305a0 con 0x7fbc3c1019c0 2024-09-10T09:36:31.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:31.308+0000 7fbc38c00640 1 --2- 172.21.15.5:0/2064461357 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fbc200a2c90 0x7fbc200a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:36:31.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:31.309+0000 7fbc38c00640 1 -- 172.21.15.5:0/2064461357 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9071+0+0 (secure 0 0 0) 0x7fbc2c02e030 con 0x7fbc3c1019c0 2024-09-10T09:36:31.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:31.309+0000 7fbc32a00640 1 --2- 172.21.15.5:0/2064461357 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fbc200a2c90 0x7fbc200a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:36:31.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:31.309+0000 7fbc32a00640 1 --2- 172.21.15.5:0/2064461357 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fbc200a2c90 0x7fbc200a5150 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fbc3c106cc0 tx=0x7fbc2400a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:36:31.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:31.313+0000 7fbc38c00640 1 -- 172.21.15.5:0/2064461357 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fbc2c04b060 con 0x7fbc3c1019c0 2024-09-10T09:36:31.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:36:30 smithi159 ceph-mon[35513]: pgmap v336: 129 pgs: 129 active+clean; 612 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 3.9 KiB/s wr, 15 op/s 2024-09-10T09:36:31.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:36:30 smithi102 ceph-mon[35442]: pgmap v336: 129 pgs: 129 active+clean; 612 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 3.9 KiB/s wr, 15 op/s 2024-09-10T09:36:31.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:31.499+0000 7fbc40a00640 1 -- 172.21.15.5:0/2064461357 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fbc3c1a7220 con 0x7fbc3c1019c0 2024-09-10T09:36:31.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:31.500+0000 7fbc38c00640 1 -- 172.21.15.5:0/2064461357 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) ==== 74+0+24566 (secure 0 0 0) 0x7fbc2c0bd5c0 con 0x7fbc3c1019c0 2024-09-10T09:36:31.502 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-10T09:36:31.502 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":94,"fsid":"bade4b50-6f56-11ef-bcea-c7b262605968","created":"2024-09-10T09:27:13.298498+0000","modified":"2024-09-10T09:36:06.755744+0000","last_up_change":"2024-09-10T09:34:30.730066+0000","last_in_change":"2024-09-10T09:34:07.340995+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T09:30:26.042255+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T09:35:24.461031+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T09:35:27.155349+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-10T09:35:35.121462+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"085749a8-e882-4377-92dd-34bb266c0f0b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":954535266},{"type":"v1","addr":"172.21.15.5:6803","nonce":954535266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":954535266},{"type":"v1","addr":"172.21.15.5:6805","nonce":954535266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":954535266},{"type":"v1","addr":"172.21.15.5:6809","nonce":954535266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":954535266},{"type":"v1","addr":"172.21.15.5:6807","nonce":954535266}]},"public_addr":"172.21.15.5:6803/954535266","cluster_addr":"172.21.15.5:6805/954535266","heartbeat_back_addr":"172.21.15.5:6809/954535266","heartbeat_front_addr":"172.21.15.5:6807/954535266","state":["exists","up"]},{"osd":1,"uuid":"0a76c795-2573-4a92-969a-2ec5dd65c094","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6800","nonce":733924375},{"type":"v1","addr":"172.21.15.102:6801","nonce":733924375}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6802","nonce":733924375},{"type":"v1","addr":"172.21.15.102:6803","nonce":733924375}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6806","nonce":733924375},{"type":"v1","addr":"172.21.15.102:6807","nonce":733924375}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6804","nonce":733924375},{"type":"v1","addr":"172.21.15.102:6805","nonce":733924375}]},"public_addr":"172.21.15.102:6801/733924375","cluster_addr":"172.21.15.102:6803/733924375","heartbeat_back_addr":"172.21.15.102:6807/733924375","heartbeat_front_addr":"172.21.15.102:6805/733924375","state":["exists","up"]},{"osd":2,"uuid":"0c98fcb8-60ab-4473-8995-1d9e97c577e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6800","nonce":2236204189},{"type":"v1","addr":"172.21.15.159:6801","nonce":2236204189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":2236204189},{"type":"v1","addr":"172.21.15.159:6803","nonce":2236204189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":2236204189},{"type":"v1","addr":"172.21.15.159:6807","nonce":2236204189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":2236204189},{"type":"v1","addr":"172.21.15.159:6805","nonce":2236204189}]},"public_addr":"172.21.15.159:6801/2236204189","cluster_addr":"172.21.15.159:6803/2236204189","heartbeat_back_addr":"172.21.15.159:6807/2236204189","heartbeat_front_addr":"172.21.15.159:6805/2236204189","state":["exists","up"]},{"osd":3,"uuid":"88eecdeb-5cee-4f78-a3f9-a66bf4bf772a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":2221468725},{"type":"v1","addr":"172.21.15.5:6811","nonce":2221468725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":2221468725},{"type":"v1","addr":"172.21.15.5:6813","nonce":2221468725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6816","nonce":2221468725},{"type":"v1","addr":"172.21.15.5:6817","nonce":2221468725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":2221468725},{"type":"v1","addr":"172.21.15.5:6815","nonce":2221468725}]},"public_addr":"172.21.15.5:6811/2221468725","cluster_addr":"172.21.15.5:6813/2221468725","heartbeat_back_addr":"172.21.15.5:6817/2221468725","heartbeat_front_addr":"172.21.15.5:6815/2221468725","state":["exists","up"]},{"osd":4,"uuid":"f2e7314d-7938-4668-9011-45c175f66b48","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6808","nonce":594465753},{"type":"v1","addr":"172.21.15.102:6809","nonce":594465753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6810","nonce":594465753},{"type":"v1","addr":"172.21.15.102:6811","nonce":594465753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6814","nonce":594465753},{"type":"v1","addr":"172.21.15.102:6815","nonce":594465753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6812","nonce":594465753},{"type":"v1","addr":"172.21.15.102:6813","nonce":594465753}]},"public_addr":"172.21.15.102:6809/594465753","cluster_addr":"172.21.15.102:6811/594465753","heartbeat_back_addr":"172.21.15.102:6815/594465753","heartbeat_front_addr":"172.21.15.102:6813/594465753","state":["exists","up"]},{"osd":5,"uuid":"0c0366d3-658b-4bf0-ba17-aff28a02e948","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":3230280402},{"type":"v1","addr":"172.21.15.159:6809","nonce":3230280402}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":3230280402},{"type":"v1","addr":"172.21.15.159:6811","nonce":3230280402}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":3230280402},{"type":"v1","addr":"172.21.15.159:6815","nonce":3230280402}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":3230280402},{"type":"v1","addr":"172.21.15.159:6813","nonce":3230280402}]},"public_addr":"172.21.15.159:6809/3230280402","cluster_addr":"172.21.15.159:6811/3230280402","heartbeat_back_addr":"172.21.15.159:6815/3230280402","heartbeat_front_addr":"172.21.15.159:6813/3230280402","state":["exists","up"]},{"osd":6,"uuid":"682670a6-51d5-46e8-af6e-dc2d8c1cf407","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":1907651140},{"type":"v1","addr":"172.21.15.5:6819","nonce":1907651140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":1907651140},{"type":"v1","addr":"172.21.15.5:6821","nonce":1907651140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":1907651140},{"type":"v1","addr":"172.21.15.5:6825","nonce":1907651140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":1907651140},{"type":"v1","addr":"172.21.15.5:6823","nonce":1907651140}]},"public_addr":"172.21.15.5:6819/1907651140","cluster_addr":"172.21.15.5:6821/1907651140","heartbeat_back_addr":"172.21.15.5:6825/1907651140","heartbeat_front_addr":"172.21.15.5:6823/1907651140","state":["exists","up"]},{"osd":7,"uuid":"8ae0f4a9-99e7-4563-aa2a-15169684a8b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6816","nonce":692040360},{"type":"v1","addr":"172.21.15.102:6817","nonce":692040360}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6818","nonce":692040360},{"type":"v1","addr":"172.21.15.102:6819","nonce":692040360}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6822","nonce":692040360},{"type":"v1","addr":"172.21.15.102:6823","nonce":692040360}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6820","nonce":692040360},{"type":"v1","addr":"172.21.15.102:6821","nonce":692040360}]},"public_addr":"172.21.15.102:6817/692040360","cluster_addr":"172.21.15.102:6819/692040360","heartbeat_back_addr":"172.21.15.102:6823/692040360","heartbeat_front_addr":"172.21.15.102:6821/692040360","state":["exists","up"]},{"osd":8,"uuid":"e37c845e-ced9-4413-9bbe-bc090537c2bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":2107291799},{"type":"v1","addr":"172.21.15.159:6817","nonce":2107291799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":2107291799},{"type":"v1","addr":"172.21.15.159:6819","nonce":2107291799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":2107291799},{"type":"v1","addr":"172.21.15.159:6823","nonce":2107291799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":2107291799},{"type":"v1","addr":"172.21.15.159:6821","nonce":2107291799}]},"public_addr":"172.21.15.159:6817/2107291799","cluster_addr":"172.21.15.159:6819/2107291799","heartbeat_back_addr":"172.21.15.159:6823/2107291799","heartbeat_front_addr":"172.21.15.159:6821/2107291799","state":["exists","up"]},{"osd":9,"uuid":"91a5b6cc-08da-47be-af12-d0896a849309","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":902431281},{"type":"v1","addr":"172.21.15.5:6827","nonce":902431281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":902431281},{"type":"v1","addr":"172.21.15.5:6829","nonce":902431281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":902431281},{"type":"v1","addr":"172.21.15.5:6833","nonce":902431281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":902431281},{"type":"v1","addr":"172.21.15.5:6831","nonce":902431281}]},"public_addr":"172.21.15.5:6827/902431281","cluster_addr":"172.21.15.5:6829/902431281","heartbeat_back_addr":"172.21.15.5:6833/902431281","heartbeat_front_addr":"172.21.15.5:6831/902431281","state":["exists","up"]},{"osd":10,"uuid":"2211030c-b00e-402b-b4d7-c587eb29a490","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6824","nonce":2853806211},{"type":"v1","addr":"172.21.15.102:6825","nonce":2853806211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6826","nonce":2853806211},{"type":"v1","addr":"172.21.15.102:6827","nonce":2853806211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6830","nonce":2853806211},{"type":"v1","addr":"172.21.15.102:6831","nonce":2853806211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6828","nonce":2853806211},{"type":"v1","addr":"172.21.15.102:6829","nonce":2853806211}]},"public_addr":"172.21.15.102:6825/2853806211","cluster_addr":"172.21.15.102:6827/2853806211","heartbeat_back_addr":"172.21.15.102:6831/2853806211","heartbeat_front_addr":"172.21.15.102:6829/2853806211","state":["exists","up"]},{"osd":11,"uuid":"4e9699ec-8ff2-4005-9d84-af5600b3a2ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6824","nonce":969941218},{"type":"v1","addr":"172.21.15.159:6825","nonce":969941218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":969941218},{"type":"v1","addr":"172.21.15.159:6827","nonce":969941218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6830","nonce":969941218},{"type":"v1","addr":"172.21.15.159:6831","nonce":969941218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":969941218},{"type":"v1","addr":"172.21.15.159:6829","nonce":969941218}]},"public_addr":"172.21.15.159:6825/969941218","cluster_addr":"172.21.15.159:6827/969941218","heartbeat_back_addr":"172.21.15.159:6831/969941218","heartbeat_front_addr":"172.21.15.159:6829/969941218","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:29:34.416490+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:30:00.246101+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:30:24.300464+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:30:54.663623+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:31:21.426292+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:31:46.738914+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:32:16.196571+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:32:42.549243+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:33:08.178651+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:33:37.477911+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:34:03.114026+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:34:28.891924+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.0","mappings":[{"from":4,"to":8}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":6}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9},{"from":5,"to":7}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]},{"pgid":"4.8","mappings":[{"from":5,"to":8}]},{"pgid":"4.c","mappings":[{"from":9,"to":10}]},{"pgid":"4.18","mappings":[{"from":9,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.5:6801/1527422686":"2024-09-11T09:28:05.436742+0000","172.21.15.5:0/2276144345":"2024-09-11T09:28:05.436742+0000","172.21.15.5:0/412040104":"2024-09-11T09:28:05.436742+0000","172.21.15.5:6800/1527422686":"2024-09-11T09:28:05.436742+0000","172.21.15.5:0/1856581170":"2024-09-11T09:28:05.436742+0000","172.21.15.5:0/2847668298":"2024-09-11T09:27:37.778500+0000","172.21.15.5:6801/14425913":"2024-09-11T09:27:37.778500+0000","172.21.15.5:0/166910782":"2024-09-11T09:27:37.778500+0000","172.21.15.5:6800/14425913":"2024-09-11T09:27:37.778500+0000","172.21.15.5:0/1711858179":"2024-09-11T09:27:37.778500+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T09:36:31.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:31.503+0000 7fbc40a00640 1 -- 172.21.15.5:0/2064461357 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fbc200a2c90 msgr2=0x7fbc200a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:36:31.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:31.503+0000 7fbc40a00640 1 --2- 172.21.15.5:0/2064461357 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fbc200a2c90 0x7fbc200a5150 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fbc3c106cc0 tx=0x7fbc2400a2d0 comp rx=0 tx=0).stop 2024-09-10T09:36:31.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:31.503+0000 7fbc40a00640 1 -- 172.21.15.5:0/2064461357 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbc3c1019c0 msgr2=0x7fbc3c19a4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:36:31.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:31.503+0000 7fbc40a00640 1 --2- 172.21.15.5:0/2064461357 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbc3c1019c0 0x7fbc3c19a4a0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fbc2c005e30 tx=0x7fbc2c032fb0 comp rx=0 tx=0).stop 2024-09-10T09:36:31.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:31.503+0000 7fbc40a00640 1 -- 172.21.15.5:0/2064461357 shutdown_connections 2024-09-10T09:36:31.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:31.503+0000 7fbc40a00640 1 --2- 172.21.15.5:0/2064461357 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fbc200a2c90 0x7fbc200a5150 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:36:31.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:31.503+0000 7fbc40a00640 1 --2- 172.21.15.5:0/2064461357 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbc3c1079c0 0x7fbc3c19ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:36:31.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:31.503+0000 7fbc40a00640 1 --2- 172.21.15.5:0/2064461357 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbc3c102360 0x7fbc3c19a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:36:31.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:31.503+0000 7fbc40a00640 1 --2- 172.21.15.5:0/2064461357 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbc3c1019c0 0x7fbc3c19a4a0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:36:31.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:31.503+0000 7fbc40a00640 1 -- 172.21.15.5:0/2064461357 >> 172.21.15.5:0/2064461357 conn(0x7fbc3c0fd740 msgr2=0x7fbc3c10cc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:36:31.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:31.503+0000 7fbc40a00640 1 -- 172.21.15.5:0/2064461357 shutdown_connections 2024-09-10T09:36:31.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:31.503+0000 7fbc40a00640 1 -- 172.21.15.5:0/2064461357 wait complete. 2024-09-10T09:36:32.201 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:36:31 smithi005 ceph-mon[30350]: from='client.? 172.21.15.5:0/2064461357' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T09:36:32.238 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-10T09:36:32.239 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-10T09:36:32.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:36:31 smithi159 ceph-mon[35513]: from='client.? 172.21.15.5:0/2064461357' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T09:36:32.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:36:31 smithi102 ceph-mon[35442]: from='client.? 172.21.15.5:0/2064461357' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T09:36:32.625 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:36:33.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:36:32 smithi005 ceph-mon[30350]: pgmap v337: 129 pgs: 129 active+clean; 635 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.3 KiB/s wr, 15 op/s 2024-09-10T09:36:33.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:33.194+0000 7f300ca00640 1 -- 172.21.15.5:0/1958857931 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f30081079c0 msgr2=0x7f3008107da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:36:33.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:33.194+0000 7f300ca00640 1 --2- 172.21.15.5:0/1958857931 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f30081079c0 0x7f3008107da0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f2ff8009930 tx=0x7f2ff802a420 comp rx=0 tx=0).stop 2024-09-10T09:36:33.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:33.194+0000 7f300ca00640 1 -- 172.21.15.5:0/1958857931 shutdown_connections 2024-09-10T09:36:33.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:33.194+0000 7f300ca00640 1 --2- 172.21.15.5:0/1958857931 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3008102360 0x7f300810e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:36:33.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:33.194+0000 7f300ca00640 1 --2- 172.21.15.5:0/1958857931 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f30081019c0 0x7f3008101e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:36:33.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:33.194+0000 7f300ca00640 1 --2- 172.21.15.5:0/1958857931 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f30081079c0 0x7f3008107da0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:36:33.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:33.194+0000 7f300ca00640 1 -- 172.21.15.5:0/1958857931 >> 172.21.15.5:0/1958857931 conn(0x7f30080fd740 msgr2=0x7f30080ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:36:33.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:33.194+0000 7f300ca00640 1 -- 172.21.15.5:0/1958857931 shutdown_connections 2024-09-10T09:36:33.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:33.194+0000 7f300ca00640 1 -- 172.21.15.5:0/1958857931 wait complete. 2024-09-10T09:36:33.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:33.195+0000 7f300ca00640 1 Processor -- start 2024-09-10T09:36:33.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:33.196+0000 7f300ca00640 1 -- start start 2024-09-10T09:36:33.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:33.196+0000 7f300ca00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f30081019c0 0x7f30080744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:36:33.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:33.196+0000 7f300ca00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3008102360 0x7f3008074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:36:33.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:33.196+0000 7f300ca00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f30081079c0 0x7f3008074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:36:33.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:33.196+0000 7f300ca00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f3008111dd0 con 0x7f30081079c0 2024-09-10T09:36:33.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:33.196+0000 7f300ca00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f3008111c50 con 0x7f30081019c0 2024-09-10T09:36:33.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:33.196+0000 7f300ca00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f3008111f50 con 0x7f3008102360 2024-09-10T09:36:33.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:33.196+0000 7f3007e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f30081079c0 0x7f3008074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:36:33.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:33.196+0000 7f3006a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3008102360 0x7f3008074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:36:33.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:33.196+0000 7f3007400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f30081019c0 0x7f30080744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:36:33.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:33.196+0000 7f3007e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f30081079c0 0x7f3008074f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46604/0 (socket says 172.21.15.5:46604) 2024-09-10T09:36:33.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:33.196+0000 7f3007e00640 1 -- 172.21.15.5:0/1666216719 learned_addr learned my addr 172.21.15.5:0/1666216719 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:36:33.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:33.197+0000 7f3006a00640 1 -- 172.21.15.5:0/1666216719 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f30081019c0 msgr2=0x7f30080744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:36:33.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:33.197+0000 7f3006a00640 1 --2- 172.21.15.5:0/1666216719 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f30081019c0 0x7f30080744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:36:33.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:33.197+0000 7f3006a00640 1 -- 172.21.15.5:0/1666216719 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f30081079c0 msgr2=0x7f3008074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:36:33.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:33.197+0000 7f3006a00640 1 --2- 172.21.15.5:0/1666216719 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f30081079c0 0x7f3008074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:36:33.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:33.197+0000 7f3006a00640 1 -- 172.21.15.5:0/1666216719 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30081a67b0 con 0x7f3008102360 2024-09-10T09:36:33.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:33.197+0000 7f3007400640 1 --2- 172.21.15.5:0/1666216719 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f30081019c0 0x7f30080744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:36:33.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:33.197+0000 7f3006a00640 1 --2- 172.21.15.5:0/1666216719 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3008102360 0x7f3008074a30 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f2ff400e9f0 tx=0x7f2ff400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:36:33.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:33.197+0000 7f2fffe00640 1 -- 172.21.15.5:0/1666216719 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ff400ccc0 con 0x7f3008102360 2024-09-10T09:36:33.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:33.198+0000 7f300ca00640 1 -- 172.21.15.5:0/1666216719 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30081a6aa0 con 0x7f3008102360 2024-09-10T09:36:33.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:33.198+0000 7f2fffe00640 1 -- 172.21.15.5:0/1666216719 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2ff4010910 con 0x7f3008102360 2024-09-10T09:36:33.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:33.198+0000 7f2fffe00640 1 -- 172.21.15.5:0/1666216719 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ff4018960 con 0x7f3008102360 2024-09-10T09:36:33.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:33.198+0000 7f300ca00640 1 -- 172.21.15.5:0/1666216719 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30081a7030 con 0x7f3008102360 2024-09-10T09:36:33.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:33.199+0000 7f300ca00640 1 -- 172.21.15.5:0/1666216719 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3008101e20 con 0x7f3008102360 2024-09-10T09:36:33.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:33.201+0000 7f2fffe00640 1 -- 172.21.15.5:0/1666216719 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f2ff4010ac0 con 0x7f3008102360 2024-09-10T09:36:33.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:33.203+0000 7f2fffe00640 1 --2- 172.21.15.5:0/1666216719 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f2fec0a2c90 0x7f2fec0a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:36:33.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:33.203+0000 7f2fffe00640 1 -- 172.21.15.5:0/1666216719 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9071+0+0 (secure 0 0 0) 0x7f2ff40cf880 con 0x7f3008102360 2024-09-10T09:36:33.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:33.203+0000 7f3007400640 1 --2- 172.21.15.5:0/1666216719 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f2fec0a2c90 0x7f2fec0a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:36:33.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:33.204+0000 7f3007400640 1 --2- 172.21.15.5:0/1666216719 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f2fec0a2c90 0x7f2fec0a5150 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f2ff8006010 tx=0x7f2ff8031510 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:36:33.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:33.207+0000 7f2fffe00640 1 -- 172.21.15.5:0/1666216719 <== mon.1 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f2ff409bc90 con 0x7f3008102360 2024-09-10T09:36:33.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:36:32 smithi159 ceph-mon[35513]: pgmap v337: 129 pgs: 129 active+clean; 635 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.3 KiB/s wr, 15 op/s 2024-09-10T09:36:33.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:36:32 smithi102 ceph-mon[35442]: pgmap v337: 129 pgs: 129 active+clean; 635 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.3 KiB/s wr, 15 op/s 2024-09-10T09:36:33.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:33.396+0000 7f300ca00640 1 -- 172.21.15.5:0/1666216719 --> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f3008062630 con 0x7f2fec0a2c90 2024-09-10T09:36:33.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:33.407+0000 7f2fffe00640 1 -- 172.21.15.5:0/1666216719 <== mgr.14150 v2:172.21.15.5:6800/2707348081 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340683 (secure 0 0 0) 0x7f3008062630 con 0x7f2fec0a2c90 2024-09-10T09:36:33.413 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-10T09:36:33.415 INFO:teuthology.orchestra.run.smithi005.stderr:dumped all 2024-09-10T09:36:33.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:33.414+0000 7f300ca00640 1 -- 172.21.15.5:0/1666216719 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f2fec0a2c90 msgr2=0x7f2fec0a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:36:33.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:33.414+0000 7f300ca00640 1 --2- 172.21.15.5:0/1666216719 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f2fec0a2c90 0x7f2fec0a5150 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f2ff8006010 tx=0x7f2ff8031510 comp rx=0 tx=0).stop 2024-09-10T09:36:33.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:33.414+0000 7f300ca00640 1 -- 172.21.15.5:0/1666216719 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3008102360 msgr2=0x7f3008074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:36:33.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:33.414+0000 7f300ca00640 1 --2- 172.21.15.5:0/1666216719 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3008102360 0x7f3008074a30 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f2ff400e9f0 tx=0x7f2ff400eeb0 comp rx=0 tx=0).stop 2024-09-10T09:36:33.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:33.414+0000 7f300ca00640 1 -- 172.21.15.5:0/1666216719 shutdown_connections 2024-09-10T09:36:33.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:33.414+0000 7f300ca00640 1 --2- 172.21.15.5:0/1666216719 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f2fec0a2c90 0x7f2fec0a5150 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:36:33.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:33.414+0000 7f300ca00640 1 --2- 172.21.15.5:0/1666216719 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f30081079c0 0x7f3008074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:36:33.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:33.414+0000 7f300ca00640 1 --2- 172.21.15.5:0/1666216719 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3008102360 0x7f3008074a30 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:36:33.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:33.414+0000 7f300ca00640 1 --2- 172.21.15.5:0/1666216719 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f30081019c0 0x7f30080744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:36:33.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:33.414+0000 7f300ca00640 1 -- 172.21.15.5:0/1666216719 >> 172.21.15.5:0/1666216719 conn(0x7f30080fd740 msgr2=0x7f30080fdf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:36:33.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:33.414+0000 7f300ca00640 1 -- 172.21.15.5:0/1666216719 shutdown_connections 2024-09-10T09:36:33.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:33.414+0000 7f300ca00640 1 -- 172.21.15.5:0/1666216719 wait complete. 2024-09-10T09:36:34.057 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:36:33 smithi005 ceph-mon[30350]: from='client.24772 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T09:36:34.142 INFO:teuthology.orchestra.run.smithi005.stdout:{"pg_ready":true,"pg_map":{"version":337,"stamp":"2024-09-10T09:36:31.541747+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":650223,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":322,"num_read_kb":273,"num_write":424,"num_write_kb":3069,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":347,"ondisk_log_size":347,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":374472,"kb_used_data":12744,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124567352,"statfs":{"total":1151940427776,"available":1151556968448,"internally_reserved":0,"allocated":13049856,"data_stored":8332443,"data_compressed":426963,"data_compressed_allocated":577536,"data_compressed_original":2654208,"omap_allocated":19141,"internal_metadata":370193723},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":49589,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":150,"num_read_kb":124,"num_write":34,"num_write_kb":64,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002888"},"pg_stats":[{"pgid":"2.1f","version":"94'13","reported_seq":68,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:36:28.519655+0000","last_change":"2024-09-10T09:35:25.533085+0000","last_active":"2024-09-10T09:36:28.519655+0000","last_peered":"2024-09-10T09:36:28.519655+0000","last_clean":"2024-09-10T09:36:28.519655+0000","last_became_active":"2024-09-10T09:35:25.532729+0000","last_became_peered":"2024-09-10T09:35:25.532729+0000","last_unstale":"2024-09-10T09:36:28.519655+0000","last_undegraded":"2024-09-10T09:36:28.519655+0000","last_fullsized":"2024-09-10T09:36:28.519655+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:55:11.158279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":16,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":11,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:36:06.768036+0000","last_change":"2024-09-10T09:36:06.767356+0000","last_active":"2024-09-10T09:36:06.768036+0000","last_peered":"2024-09-10T09:36:06.768036+0000","last_clean":"2024-09-10T09:36:06.768036+0000","last_became_active":"2024-09-10T09:36:06.766931+0000","last_became_peered":"2024-09-10T09:36:06.766931+0000","last_unstale":"2024-09-10T09:36:06.768036+0000","last_undegraded":"2024-09-10T09:36:06.768036+0000","last_fullsized":"2024-09-10T09:36:06.768036+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:25:02.777327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,7],"acting":[9,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:36:06.760145+0000","last_change":"2024-09-10T09:35:36.629797+0000","last_active":"2024-09-10T09:36:06.760145+0000","last_peered":"2024-09-10T09:36:06.760145+0000","last_clean":"2024-09-10T09:36:06.760145+0000","last_became_active":"2024-09-10T09:35:36.629414+0000","last_became_peered":"2024-09-10T09:35:36.629414+0000","last_unstale":"2024-09-10T09:36:06.760145+0000","last_undegraded":"2024-09-10T09:36:06.760145+0000","last_fullsized":"2024-09-10T09:36:06.760145+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:42:44.610581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"94'2","reported_seq":42,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:36:06.764083+0000","last_change":"2024-09-10T09:35:25.852022+0000","last_active":"2024-09-10T09:36:06.764083+0000","last_peered":"2024-09-10T09:36:06.764083+0000","last_clean":"2024-09-10T09:36:06.764083+0000","last_became_active":"2024-09-10T09:35:25.851587+0000","last_became_peered":"2024-09-10T09:35:25.851587+0000","last_unstale":"2024-09-10T09:36:06.764083+0000","last_undegraded":"2024-09-10T09:36:06.764083+0000","last_fullsized":"2024-09-10T09:36:06.764083+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:37:26.085914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:36:06.760232+0000","last_change":"2024-09-10T09:35:28.579260+0000","last_active":"2024-09-10T09:36:06.760232+0000","last_peered":"2024-09-10T09:36:06.760232+0000","last_clean":"2024-09-10T09:36:06.760232+0000","last_became_active":"2024-09-10T09:35:28.574584+0000","last_became_peered":"2024-09-10T09:35:28.574584+0000","last_unstale":"2024-09-10T09:36:06.760232+0000","last_undegraded":"2024-09-10T09:36:06.760232+0000","last_fullsized":"2024-09-10T09:36:06.760232+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:31:19.277689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":31,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:36:06.874861+0000","last_change":"2024-09-10T09:36:06.874861+0000","last_active":"2024-09-10T09:36:06.874861+0000","last_peered":"2024-09-10T09:36:06.874861+0000","last_clean":"2024-09-10T09:36:06.874861+0000","last_became_active":"2024-09-10T09:36:06.873930+0000","last_became_peered":"2024-09-10T09:36:06.873930+0000","last_unstale":"2024-09-10T09:36:06.874861+0000","last_undegraded":"2024-09-10T09:36:06.874861+0000","last_fullsized":"2024-09-10T09:36:06.874861+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:35:01.354061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,8,3],"acting":[4,6,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"94'12","reported_seq":52,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:36:14.498394+0000","last_change":"2024-09-10T09:35:25.534571+0000","last_active":"2024-09-10T09:36:14.498394+0000","last_peered":"2024-09-10T09:36:14.498394+0000","last_clean":"2024-09-10T09:36:14.498394+0000","last_became_active":"2024-09-10T09:35:25.534235+0000","last_became_peered":"2024-09-10T09:35:25.534235+0000","last_unstale":"2024-09-10T09:36:14.498394+0000","last_undegraded":"2024-09-10T09:36:14.498394+0000","last_fullsized":"2024-09-10T09:36:14.498394+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:50:21.447542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:36:06.765451+0000","last_change":"2024-09-10T09:35:28.565541+0000","last_active":"2024-09-10T09:36:06.765451+0000","last_peered":"2024-09-10T09:36:06.765451+0000","last_clean":"2024-09-10T09:36:06.765451+0000","last_became_active":"2024-09-10T09:35:28.565095+0000","last_became_peered":"2024-09-10T09:35:28.565095+0000","last_unstale":"2024-09-10T09:36:06.765451+0000","last_undegraded":"2024-09-10T09:36:06.765451+0000","last_fullsized":"2024-09-10T09:36:06.765451+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:29:31.285401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:36:06.765379+0000","last_change":"2024-09-10T09:35:36.629277+0000","last_active":"2024-09-10T09:36:06.765379+0000","last_peered":"2024-09-10T09:36:06.765379+0000","last_clean":"2024-09-10T09:36:06.765379+0000","last_became_active":"2024-09-10T09:35:36.628687+0000","last_became_peered":"2024-09-10T09:35:36.628687+0000","last_unstale":"2024-09-10T09:36:06.765379+0000","last_undegraded":"2024-09-10T09:36:06.765379+0000","last_fullsized":"2024-09-10T09:36:06.765379+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:34:33.628827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"94'5","reported_seq":45,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:36:14.497342+0000","last_change":"2024-09-10T09:35:25.536137+0000","last_active":"2024-09-10T09:36:14.497342+0000","last_peered":"2024-09-10T09:36:14.497342+0000","last_clean":"2024-09-10T09:36:14.497342+0000","last_became_active":"2024-09-10T09:35:25.535728+0000","last_became_peered":"2024-09-10T09:35:25.535728+0000","last_unstale":"2024-09-10T09:36:14.497342+0000","last_undegraded":"2024-09-10T09:36:14.497342+0000","last_fullsized":"2024-09-10T09:36:14.497342+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:36:04.829901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:36:06.874425+0000","last_change":"2024-09-10T09:36:06.874425+0000","last_active":"2024-09-10T09:36:06.874425+0000","last_peered":"2024-09-10T09:36:06.874425+0000","last_clean":"2024-09-10T09:36:06.874425+0000","last_became_active":"2024-09-10T09:36:06.874050+0000","last_became_peered":"2024-09-10T09:36:06.874050+0000","last_unstale":"2024-09-10T09:36:06.874425+0000","last_undegraded":"2024-09-10T09:36:06.874425+0000","last_fullsized":"2024-09-10T09:36:06.874425+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:30:51.200335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,1],"acting":[5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:36:06.765164+0000","last_change":"2024-09-10T09:35:36.615314+0000","last_active":"2024-09-10T09:36:06.765164+0000","last_peered":"2024-09-10T09:36:06.765164+0000","last_clean":"2024-09-10T09:36:06.765164+0000","last_became_active":"2024-09-10T09:35:36.614820+0000","last_became_peered":"2024-09-10T09:35:36.614820+0000","last_unstale":"2024-09-10T09:36:06.765164+0000","last_undegraded":"2024-09-10T09:36:06.765164+0000","last_fullsized":"2024-09-10T09:36:06.765164+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:51:16.897935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:36:06.764124+0000","last_change":"2024-09-10T09:35:28.572346+0000","last_active":"2024-09-10T09:36:06.764124+0000","last_peered":"2024-09-10T09:36:06.764124+0000","last_clean":"2024-09-10T09:36:06.764124+0000","last_became_active":"2024-09-10T09:35:28.571839+0000","last_became_peered":"2024-09-10T09:35:28.571839+0000","last_unstale":"2024-09-10T09:36:06.764124+0000","last_undegraded":"2024-09-10T09:36:06.764124+0000","last_fullsized":"2024-09-10T09:36:06.764124+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:41:29.662441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"94'4","reported_seq":44,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:36:11.804692+0000","last_change":"2024-09-10T09:35:25.538747+0000","last_active":"2024-09-10T09:36:11.804692+0000","last_peered":"2024-09-10T09:36:11.804692+0000","last_clean":"2024-09-10T09:36:11.804692+0000","last_became_active":"2024-09-10T09:35:25.538330+0000","last_became_peered":"2024-09-10T09:35:25.538330+0000","last_unstale":"2024-09-10T09:36:11.804692+0000","last_undegraded":"2024-09-10T09:36:11.804692+0000","last_fullsized":"2024-09-10T09:36:11.804692+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:15:08.340688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:36:06.764134+0000","last_change":"2024-09-10T09:35:37.270106+0000","last_active":"2024-09-10T09:36:06.764134+0000","last_peered":"2024-09-10T09:36:06.764134+0000","last_clean":"2024-09-10T09:36:06.764134+0000","last_became_active":"2024-09-10T09:35:37.269575+0000","last_became_peered":"2024-09-10T09:35:37.269575+0000","last_unstale":"2024-09-10T09:36:06.764134+0000","last_undegraded":"2024-09-10T09:36:06.764134+0000","last_fullsized":"2024-09-10T09:36:06.764134+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:35:00.414602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:36:06.763337+0000","last_change":"2024-09-10T09:35:28.569230+0000","last_active":"2024-09-10T09:36:06.763337+0000","last_peered":"2024-09-10T09:36:06.763337+0000","last_clean":"2024-09-10T09:36:06.763337+0000","last_became_active":"2024-09-10T09:35:28.568266+0000","last_became_peered":"2024-09-10T09:35:28.568266+0000","last_unstale":"2024-09-10T09:36:06.763337+0000","last_undegraded":"2024-09-10T09:36:06.763337+0000","last_fullsized":"2024-09-10T09:36:06.763337+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:49:43.730686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"94'3","reported_seq":42,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:36:13.817585+0000","last_change":"2024-09-10T09:35:25.534856+0000","last_active":"2024-09-10T09:36:13.817585+0000","last_peered":"2024-09-10T09:36:13.817585+0000","last_clean":"2024-09-10T09:36:13.817585+0000","last_became_active":"2024-09-10T09:35:25.534471+0000","last_became_peered":"2024-09-10T09:35:25.534471+0000","last_unstale":"2024-09-10T09:36:13.817585+0000","last_undegraded":"2024-09-10T09:36:13.817585+0000","last_fullsized":"2024-09-10T09:36:13.817585+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:34:51.490556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"94'1","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:36:06.875782+0000","last_change":"2024-09-10T09:36:06.875056+0000","last_active":"2024-09-10T09:36:06.875782+0000","last_peered":"2024-09-10T09:36:06.875782+0000","last_clean":"2024-09-10T09:36:06.875782+0000","last_became_active":"2024-09-10T09:36:06.874400+0000","last_became_peered":"2024-09-10T09:36:06.874400+0000","last_unstale":"2024-09-10T09:36:06.875782+0000","last_undegraded":"2024-09-10T09:36:06.875782+0000","last_fullsized":"2024-09-10T09:36:06.875782+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:21:06.300712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,7,8],"acting":[4,10,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:36:06.762615+0000","last_change":"2024-09-10T09:35:28.571319+0000","last_active":"2024-09-10T09:36:06.762615+0000","last_peered":"2024-09-10T09:36:06.762615+0000","last_clean":"2024-09-10T09:36:06.762615+0000","last_became_active":"2024-09-10T09:35:28.570972+0000","last_became_peered":"2024-09-10T09:35:28.570972+0000","last_unstale":"2024-09-10T09:36:06.762615+0000","last_undegraded":"2024-09-10T09:36:06.762615+0000","last_fullsized":"2024-09-10T09:36:06.762615+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:45:19.300535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"94'7","reported_seq":47,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:36:14.497731+0000","last_change":"2024-09-10T09:35:25.530489+0000","last_active":"2024-09-10T09:36:14.497731+0000","last_peered":"2024-09-10T09:36:14.497731+0000","last_clean":"2024-09-10T09:36:14.497731+0000","last_became_active":"2024-09-10T09:35:25.530197+0000","last_became_peered":"2024-09-10T09:35:25.530197+0000","last_unstale":"2024-09-10T09:36:14.497731+0000","last_undegraded":"2024-09-10T09:36:14.497731+0000","last_fullsized":"2024-09-10T09:36:14.497731+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:28:57.539569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:36:06.761452+0000","last_change":"2024-09-10T09:35:36.630507+0000","last_active":"2024-09-10T09:36:06.761452+0000","last_peered":"2024-09-10T09:36:06.761452+0000","last_clean":"2024-09-10T09:36:06.761452+0000","last_became_active":"2024-09-10T09:35:36.630052+0000","last_became_peered":"2024-09-10T09:35:36.630052+0000","last_unstale":"2024-09-10T09:36:06.761452+0000","last_undegraded":"2024-09-10T09:36:06.761452+0000","last_fullsized":"2024-09-10T09:36:06.761452+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:13:42.691382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:36:06.766516+0000","last_change":"2024-09-10T09:35:28.565650+0000","last_active":"2024-09-10T09:36:06.766516+0000","last_peered":"2024-09-10T09:36:06.766516+0000","last_clean":"2024-09-10T09:36:06.766516+0000","last_became_active":"2024-09-10T09:35:28.565303+0000","last_became_peered":"2024-09-10T09:35:28.565303+0000","last_unstale":"2024-09-10T09:36:06.766516+0000","last_undegraded":"2024-09-10T09:36:06.766516+0000","last_fullsized":"2024-09-10T09:36:06.766516+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:48:08.965487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"94'2","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:36:11.806672+0000","last_change":"2024-09-10T09:35:25.851326+0000","last_active":"2024-09-10T09:36:11.806672+0000","last_peered":"2024-09-10T09:36:11.806672+0000","last_clean":"2024-09-10T09:36:11.806672+0000","last_became_active":"2024-09-10T09:35:25.850981+0000","last_became_peered":"2024-09-10T09:35:25.850981+0000","last_unstale":"2024-09-10T09:36:11.806672+0000","last_undegraded":"2024-09-10T09:36:11.806672+0000","last_fullsized":"2024-09-10T09:36:11.806672+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:50:28.877840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:36:06.765389+0000","last_change":"2024-09-10T09:35:37.270138+0000","last_active":"2024-09-10T09:36:06.765389+0000","last_peered":"2024-09-10T09:36:06.765389+0000","last_clean":"2024-09-10T09:36:06.765389+0000","last_became_active":"2024-09-10T09:35:37.269663+0000","last_became_peered":"2024-09-10T09:35:37.269663+0000","last_unstale":"2024-09-10T09:36:06.765389+0000","last_undegraded":"2024-09-10T09:36:06.765389+0000","last_fullsized":"2024-09-10T09:36:06.765389+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:18:00.105970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:36:06.759993+0000","last_change":"2024-09-10T09:35:28.576197+0000","last_active":"2024-09-10T09:36:06.759993+0000","last_peered":"2024-09-10T09:36:06.759993+0000","last_clean":"2024-09-10T09:36:06.759993+0000","last_became_active":"2024-09-10T09:35:28.575859+0000","last_became_peered":"2024-09-10T09:35:28.575859+0000","last_unstale":"2024-09-10T09:36:06.759993+0000","last_undegraded":"2024-09-10T09:36:06.759993+0000","last_fullsized":"2024-09-10T09:36:06.759993+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:09:39.458761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"94'7","reported_seq":46,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:36:14.498489+0000","last_change":"2024-09-10T09:35:25.990587+0000","last_active":"2024-09-10T09:36:14.498489+0000","last_peered":"2024-09-10T09:36:14.498489+0000","last_clean":"2024-09-10T09:36:14.498489+0000","last_became_active":"2024-09-10T09:35:25.990302+0000","last_became_peered":"2024-09-10T09:35:25.990302+0000","last_unstale":"2024-09-10T09:36:14.498489+0000","last_undegraded":"2024-09-10T09:36:14.498489+0000","last_fullsized":"2024-09-10T09:36:14.498489+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:18:41.244441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:36:06.760684+0000","last_change":"2024-09-10T09:35:36.612983+0000","last_active":"2024-09-10T09:36:06.760684+0000","last_peered":"2024-09-10T09:36:06.760684+0000","last_clean":"2024-09-10T09:36:06.760684+0000","last_became_active":"2024-09-10T09:35:36.612542+0000","last_became_peered":"2024-09-10T09:35:36.612542+0000","last_unstale":"2024-09-10T09:36:06.760684+0000","last_undegraded":"2024-09-10T09:36:06.760684+0000","last_fullsized":"2024-09-10T09:36:06.760684+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:50:16.298195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:36:06.763996+0000","last_change":"2024-09-10T09:35:28.564121+0000","last_active":"2024-09-10T09:36:06.763996+0000","last_peered":"2024-09-10T09:36:06.763996+0000","last_clean":"2024-09-10T09:36:06.763996+0000","last_became_active":"2024-09-10T09:35:28.562780+0000","last_became_peered":"2024-09-10T09:35:28.562780+0000","last_unstale":"2024-09-10T09:36:06.763996+0000","last_undegraded":"2024-09-10T09:36:06.763996+0000","last_fullsized":"2024-09-10T09:36:06.763996+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:53:15.690478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"94'13","reported_seq":52,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:36:27.579197+0000","last_change":"2024-09-10T09:35:25.532740+0000","last_active":"2024-09-10T09:36:27.579197+0000","last_peered":"2024-09-10T09:36:27.579197+0000","last_clean":"2024-09-10T09:36:27.579197+0000","last_became_active":"2024-09-10T09:35:25.532282+0000","last_became_peered":"2024-09-10T09:35:25.532282+0000","last_unstale":"2024-09-10T09:36:27.579197+0000","last_undegraded":"2024-09-10T09:36:27.579197+0000","last_fullsized":"2024-09-10T09:36:27.579197+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:52:17.323590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:36:06.762957+0000","last_change":"2024-09-10T09:35:37.270888+0000","last_active":"2024-09-10T09:36:06.762957+0000","last_peered":"2024-09-10T09:36:06.762957+0000","last_clean":"2024-09-10T09:36:06.762957+0000","last_became_active":"2024-09-10T09:35:37.270229+0000","last_became_peered":"2024-09-10T09:35:37.270229+0000","last_unstale":"2024-09-10T09:36:06.762957+0000","last_undegraded":"2024-09-10T09:36:06.762957+0000","last_fullsized":"2024-09-10T09:36:06.762957+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:03:54.223463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:36:06.873437+0000","last_change":"2024-09-10T09:35:28.572799+0000","last_active":"2024-09-10T09:36:06.873437+0000","last_peered":"2024-09-10T09:36:06.873437+0000","last_clean":"2024-09-10T09:36:06.873437+0000","last_became_active":"2024-09-10T09:35:28.572310+0000","last_became_peered":"2024-09-10T09:35:28.572310+0000","last_unstale":"2024-09-10T09:36:06.873437+0000","last_undegraded":"2024-09-10T09:36:06.873437+0000","last_fullsized":"2024-09-10T09:36:06.873437+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:33:51.446766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"94'4","reported_seq":44,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:36:14.485932+0000","last_change":"2024-09-10T09:35:25.851868+0000","last_active":"2024-09-10T09:36:14.485932+0000","last_peered":"2024-09-10T09:36:14.485932+0000","last_clean":"2024-09-10T09:36:14.485932+0000","last_became_active":"2024-09-10T09:35:25.851447+0000","last_became_peered":"2024-09-10T09:35:25.851447+0000","last_unstale":"2024-09-10T09:36:14.485932+0000","last_undegraded":"2024-09-10T09:36:14.485932+0000","last_fullsized":"2024-09-10T09:36:14.485932+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:15:57.706690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:36:06.761923+0000","last_change":"2024-09-10T09:35:36.619459+0000","last_active":"2024-09-10T09:36:06.761923+0000","last_peered":"2024-09-10T09:36:06.761923+0000","last_clean":"2024-09-10T09:36:06.761923+0000","last_became_active":"2024-09-10T09:35:36.619023+0000","last_became_peered":"2024-09-10T09:35:36.619023+0000","last_unstale":"2024-09-10T09:36:06.761923+0000","last_undegraded":"2024-09-10T09:36:06.761923+0000","last_fullsized":"2024-09-10T09:36:06.761923+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:07:32.999231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:36:06.873350+0000","last_change":"2024-09-10T09:35:28.575668+0000","last_active":"2024-09-10T09:36:06.873350+0000","last_peered":"2024-09-10T09:36:06.873350+0000","last_clean":"2024-09-10T09:36:06.873350+0000","last_became_active":"2024-09-10T09:35:28.575425+0000","last_became_peered":"2024-09-10T09:35:28.575425+0000","last_unstale":"2024-09-10T09:36:06.873350+0000","last_undegraded":"2024-09-10T09:36:06.873350+0000","last_fullsized":"2024-09-10T09:36:06.873350+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:13:27.120208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"94'1","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:36:06.761098+0000","last_change":"2024-09-10T09:35:25.854569+0000","last_active":"2024-09-10T09:36:06.761098+0000","last_peered":"2024-09-10T09:36:06.761098+0000","last_clean":"2024-09-10T09:36:06.761098+0000","last_became_active":"2024-09-10T09:35:25.854152+0000","last_became_peered":"2024-09-10T09:35:25.854152+0000","last_unstale":"2024-09-10T09:36:06.761098+0000","last_undegraded":"2024-09-10T09:36:06.761098+0000","last_fullsized":"2024-09-10T09:36:06.761098+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:27:15.584707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:36:06.766506+0000","last_change":"2024-09-10T09:35:37.270517+0000","last_active":"2024-09-10T09:36:06.766506+0000","last_peered":"2024-09-10T09:36:06.766506+0000","last_clean":"2024-09-10T09:36:06.766506+0000","last_became_active":"2024-09-10T09:35:37.269793+0000","last_became_peered":"2024-09-10T09:35:37.269793+0000","last_unstale":"2024-09-10T09:36:06.766506+0000","last_undegraded":"2024-09-10T09:36:06.766506+0000","last_fullsized":"2024-09-10T09:36:06.766506+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:28:58.272848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:36:06.873121+0000","last_change":"2024-09-10T09:35:28.572056+0000","last_active":"2024-09-10T09:36:06.873121+0000","last_peered":"2024-09-10T09:36:06.873121+0000","last_clean":"2024-09-10T09:36:06.873121+0000","last_became_active":"2024-09-10T09:35:28.571755+0000","last_became_peered":"2024-09-10T09:35:28.571755+0000","last_unstale":"2024-09-10T09:36:06.873121+0000","last_undegraded":"2024-09-10T09:36:06.873121+0000","last_fullsized":"2024-09-10T09:36:06.873121+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:45:02.739512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"94'6","reported_seq":61,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:36:28.520322+0000","last_change":"2024-09-10T09:35:25.531062+0000","last_active":"2024-09-10T09:36:28.520322+0000","last_peered":"2024-09-10T09:36:28.520322+0000","last_clean":"2024-09-10T09:36:28.520322+0000","last_became_active":"2024-09-10T09:35:25.530005+0000","last_became_peered":"2024-09-10T09:35:25.530005+0000","last_unstale":"2024-09-10T09:36:28.520322+0000","last_undegraded":"2024-09-10T09:36:28.520322+0000","last_fullsized":"2024-09-10T09:36:28.520322+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:42:35.330312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":981,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":17,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:36:06.873232+0000","last_change":"2024-09-10T09:35:36.623099+0000","last_active":"2024-09-10T09:36:06.873232+0000","last_peered":"2024-09-10T09:36:06.873232+0000","last_clean":"2024-09-10T09:36:06.873232+0000","last_became_active":"2024-09-10T09:35:36.622716+0000","last_became_peered":"2024-09-10T09:35:36.622716+0000","last_unstale":"2024-09-10T09:36:06.873232+0000","last_undegraded":"2024-09-10T09:36:06.873232+0000","last_fullsized":"2024-09-10T09:36:06.873232+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:10:38.495164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:36:06.769405+0000","last_change":"2024-09-10T09:36:06.769405+0000","last_active":"2024-09-10T09:36:06.769405+0000","last_peered":"2024-09-10T09:36:06.769405+0000","last_clean":"2024-09-10T09:36:06.769405+0000","last_became_active":"2024-09-10T09:36:06.769042+0000","last_became_peered":"2024-09-10T09:36:06.769042+0000","last_unstale":"2024-09-10T09:36:06.769405+0000","last_undegraded":"2024-09-10T09:36:06.769405+0000","last_fullsized":"2024-09-10T09:36:06.769405+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:01:49.340765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,8],"acting":[5,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"94'5","reported_seq":60,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:36:27.508041+0000","last_change":"2024-09-10T09:35:25.851430+0000","last_active":"2024-09-10T09:36:27.508041+0000","last_peered":"2024-09-10T09:36:27.508041+0000","last_clean":"2024-09-10T09:36:27.508041+0000","last_became_active":"2024-09-10T09:35:25.851067+0000","last_became_peered":"2024-09-10T09:35:25.851067+0000","last_unstale":"2024-09-10T09:36:27.508041+0000","last_undegraded":"2024-09-10T09:36:27.508041+0000","last_fullsized":"2024-09-10T09:36:27.508041+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:55:58.756207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":16,"num_read_kb":16,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:36:06.873609+0000","last_change":"2024-09-10T09:35:37.270361+0000","last_active":"2024-09-10T09:36:06.873609+0000","last_peered":"2024-09-10T09:36:06.873609+0000","last_clean":"2024-09-10T09:36:06.873609+0000","last_became_active":"2024-09-10T09:35:37.269632+0000","last_became_peered":"2024-09-10T09:35:37.269632+0000","last_unstale":"2024-09-10T09:36:06.873609+0000","last_undegraded":"2024-09-10T09:36:06.873609+0000","last_fullsized":"2024-09-10T09:36:06.873609+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:50:34.360497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:36:06.872892+0000","last_change":"2024-09-10T09:35:28.571064+0000","last_active":"2024-09-10T09:36:06.872892+0000","last_peered":"2024-09-10T09:36:06.872892+0000","last_clean":"2024-09-10T09:36:06.872892+0000","last_became_active":"2024-09-10T09:35:28.570691+0000","last_became_peered":"2024-09-10T09:35:28.570691+0000","last_unstale":"2024-09-10T09:36:06.872892+0000","last_undegraded":"2024-09-10T09:36:06.872892+0000","last_fullsized":"2024-09-10T09:36:06.872892+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:58:33.162501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"94'4","reported_seq":42,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:36:12.811401+0000","last_change":"2024-09-10T09:35:25.533950+0000","last_active":"2024-09-10T09:36:12.811401+0000","last_peered":"2024-09-10T09:36:12.811401+0000","last_clean":"2024-09-10T09:36:12.811401+0000","last_became_active":"2024-09-10T09:35:25.533612+0000","last_became_peered":"2024-09-10T09:35:25.533612+0000","last_unstale":"2024-09-10T09:36:12.811401+0000","last_undegraded":"2024-09-10T09:36:12.811401+0000","last_fullsized":"2024-09-10T09:36:12.811401+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:13:40.221467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:36:06.763637+0000","last_change":"2024-09-10T09:35:37.270358+0000","last_active":"2024-09-10T09:36:06.763637+0000","last_peered":"2024-09-10T09:36:06.763637+0000","last_clean":"2024-09-10T09:36:06.763637+0000","last_became_active":"2024-09-10T09:35:37.269822+0000","last_became_peered":"2024-09-10T09:35:37.269822+0000","last_unstale":"2024-09-10T09:36:06.763637+0000","last_undegraded":"2024-09-10T09:36:06.763637+0000","last_fullsized":"2024-09-10T09:36:06.763637+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:19:48.932596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:36:06.763498+0000","last_change":"2024-09-10T09:35:28.576244+0000","last_active":"2024-09-10T09:36:06.763498+0000","last_peered":"2024-09-10T09:36:06.763498+0000","last_clean":"2024-09-10T09:36:06.763498+0000","last_became_active":"2024-09-10T09:35:28.571801+0000","last_became_peered":"2024-09-10T09:35:28.571801+0000","last_unstale":"2024-09-10T09:36:06.763498+0000","last_undegraded":"2024-09-10T09:36:06.763498+0000","last_fullsized":"2024-09-10T09:36:06.763498+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:11:21.742220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"94'6","reported_seq":47,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:36:14.498654+0000","last_change":"2024-09-10T09:35:25.537118+0000","last_active":"2024-09-10T09:36:14.498654+0000","last_peered":"2024-09-10T09:36:14.498654+0000","last_clean":"2024-09-10T09:36:14.498654+0000","last_became_active":"2024-09-10T09:35:25.536718+0000","last_became_peered":"2024-09-10T09:35:25.536718+0000","last_unstale":"2024-09-10T09:36:14.498654+0000","last_undegraded":"2024-09-10T09:36:14.498654+0000","last_fullsized":"2024-09-10T09:36:14.498654+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:16:32.837476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":356,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:36:06.766849+0000","last_change":"2024-09-10T09:34:07.350825+0000","last_active":"2024-09-10T09:36:06.766849+0000","last_peered":"2024-09-10T09:36:06.766849+0000","last_clean":"2024-09-10T09:36:06.766849+0000","last_became_active":"2024-09-10T09:34:07.350325+0000","last_became_peered":"2024-09-10T09:34:07.350325+0000","last_unstale":"2024-09-10T09:36:06.766849+0000","last_undegraded":"2024-09-10T09:36:06.766849+0000","last_fullsized":"2024-09-10T09:36:06.766849+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:30:26.408262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:30:26.408262+0000","last_clean_scrub_stamp":"2024-09-10T09:30:26.408262+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:58:04.920443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"94'1","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:36:06.763785+0000","last_change":"2024-09-10T09:35:36.632941+0000","last_active":"2024-09-10T09:36:06.763785+0000","last_peered":"2024-09-10T09:36:06.763785+0000","last_clean":"2024-09-10T09:36:06.763785+0000","last_became_active":"2024-09-10T09:35:36.632527+0000","last_became_peered":"2024-09-10T09:35:36.632527+0000","last_unstale":"2024-09-10T09:36:06.763785+0000","last_undegraded":"2024-09-10T09:36:06.763785+0000","last_fullsized":"2024-09-10T09:36:06.763785+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:33:21.733772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:36:06.764063+0000","last_change":"2024-09-10T09:35:28.572010+0000","last_active":"2024-09-10T09:36:06.764063+0000","last_peered":"2024-09-10T09:36:06.764063+0000","last_clean":"2024-09-10T09:36:06.764063+0000","last_became_active":"2024-09-10T09:35:28.571623+0000","last_became_peered":"2024-09-10T09:35:28.571623+0000","last_unstale":"2024-09-10T09:36:06.764063+0000","last_undegraded":"2024-09-10T09:36:06.764063+0000","last_fullsized":"2024-09-10T09:36:06.764063+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:00:49.328663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"94'5","reported_seq":63,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:36:29.512235+0000","last_change":"2024-09-10T09:35:25.990328+0000","last_active":"2024-09-10T09:36:29.512235+0000","last_peered":"2024-09-10T09:36:29.512235+0000","last_clean":"2024-09-10T09:36:29.512235+0000","last_became_active":"2024-09-10T09:35:25.989946+0000","last_became_peered":"2024-09-10T09:35:25.989946+0000","last_unstale":"2024-09-10T09:36:29.512235+0000","last_undegraded":"2024-09-10T09:36:29.512235+0000","last_fullsized":"2024-09-10T09:36:29.512235+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:51:16.069913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":19,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:36:06.765767+0000","last_change":"2024-09-10T09:35:36.619013+0000","last_active":"2024-09-10T09:36:06.765767+0000","last_peered":"2024-09-10T09:36:06.765767+0000","last_clean":"2024-09-10T09:36:06.765767+0000","last_became_active":"2024-09-10T09:35:36.618506+0000","last_became_peered":"2024-09-10T09:35:36.618506+0000","last_unstale":"2024-09-10T09:36:06.765767+0000","last_undegraded":"2024-09-10T09:36:06.765767+0000","last_fullsized":"2024-09-10T09:36:06.765767+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:57:53.580569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:36:06.768573+0000","last_change":"2024-09-10T09:36:06.768573+0000","last_active":"2024-09-10T09:36:06.768573+0000","last_peered":"2024-09-10T09:36:06.768573+0000","last_clean":"2024-09-10T09:36:06.768573+0000","last_became_active":"2024-09-10T09:36:06.768228+0000","last_became_peered":"2024-09-10T09:36:06.768228+0000","last_unstale":"2024-09-10T09:36:06.768573+0000","last_undegraded":"2024-09-10T09:36:06.768573+0000","last_fullsized":"2024-09-10T09:36:06.768573+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:57:38.310713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"94'3","reported_seq":43,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:36:14.486020+0000","last_change":"2024-09-10T09:35:25.528795+0000","last_active":"2024-09-10T09:36:14.486020+0000","last_peered":"2024-09-10T09:36:14.486020+0000","last_clean":"2024-09-10T09:36:14.486020+0000","last_became_active":"2024-09-10T09:35:25.528475+0000","last_became_peered":"2024-09-10T09:35:25.528475+0000","last_unstale":"2024-09-10T09:36:14.486020+0000","last_undegraded":"2024-09-10T09:36:14.486020+0000","last_fullsized":"2024-09-10T09:36:14.486020+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:16:06.999186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:36:06.759828+0000","last_change":"2024-09-10T09:35:36.633109+0000","last_active":"2024-09-10T09:36:06.759828+0000","last_peered":"2024-09-10T09:36:06.759828+0000","last_clean":"2024-09-10T09:36:06.759828+0000","last_became_active":"2024-09-10T09:35:36.632682+0000","last_became_peered":"2024-09-10T09:35:36.632682+0000","last_unstale":"2024-09-10T09:36:06.759828+0000","last_undegraded":"2024-09-10T09:36:06.759828+0000","last_fullsized":"2024-09-10T09:36:06.759828+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:51:14.372649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:36:06.765964+0000","last_change":"2024-09-10T09:35:28.562103+0000","last_active":"2024-09-10T09:36:06.765964+0000","last_peered":"2024-09-10T09:36:06.765964+0000","last_clean":"2024-09-10T09:36:06.765964+0000","last_became_active":"2024-09-10T09:35:28.561765+0000","last_became_peered":"2024-09-10T09:35:28.561765+0000","last_unstale":"2024-09-10T09:36:06.765964+0000","last_undegraded":"2024-09-10T09:36:06.765964+0000","last_fullsized":"2024-09-10T09:36:06.765964+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:35:35.400569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"94'5","reported_seq":44,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:36:14.497766+0000","last_change":"2024-09-10T09:35:25.538536+0000","last_active":"2024-09-10T09:36:14.497766+0000","last_peered":"2024-09-10T09:36:14.497766+0000","last_clean":"2024-09-10T09:36:14.497766+0000","last_became_active":"2024-09-10T09:35:25.537944+0000","last_became_peered":"2024-09-10T09:35:25.537944+0000","last_unstale":"2024-09-10T09:36:14.497766+0000","last_undegraded":"2024-09-10T09:36:14.497766+0000","last_fullsized":"2024-09-10T09:36:14.497766+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:01:19.385381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":12,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:36:06.770468+0000","last_change":"2024-09-10T09:36:06.769534+0000","last_active":"2024-09-10T09:36:06.770468+0000","last_peered":"2024-09-10T09:36:06.770468+0000","last_clean":"2024-09-10T09:36:06.770468+0000","last_became_active":"2024-09-10T09:36:06.769015+0000","last_became_peered":"2024-09-10T09:36:06.769015+0000","last_unstale":"2024-09-10T09:36:06.770468+0000","last_undegraded":"2024-09-10T09:36:06.770468+0000","last_fullsized":"2024-09-10T09:36:06.770468+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:19:26.369737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7,1],"acting":[8,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:36:06.762409+0000","last_change":"2024-09-10T09:35:28.566735+0000","last_active":"2024-09-10T09:36:06.762409+0000","last_peered":"2024-09-10T09:36:06.762409+0000","last_clean":"2024-09-10T09:36:06.762409+0000","last_became_active":"2024-09-10T09:35:28.566400+0000","last_became_peered":"2024-09-10T09:35:28.566400+0000","last_unstale":"2024-09-10T09:36:06.762409+0000","last_undegraded":"2024-09-10T09:36:06.762409+0000","last_fullsized":"2024-09-10T09:36:06.762409+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:18:53.337816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"94'2","reported_seq":42,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:36:13.806778+0000","last_change":"2024-09-10T09:35:25.535783+0000","last_active":"2024-09-10T09:36:13.806778+0000","last_peered":"2024-09-10T09:36:13.806778+0000","last_clean":"2024-09-10T09:36:13.806778+0000","last_became_active":"2024-09-10T09:35:25.535411+0000","last_became_peered":"2024-09-10T09:35:25.535411+0000","last_unstale":"2024-09-10T09:36:13.806778+0000","last_undegraded":"2024-09-10T09:36:13.806778+0000","last_fullsized":"2024-09-10T09:36:13.806778+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:48:09.239083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:36:06.763837+0000","last_change":"2024-09-10T09:35:36.619380+0000","last_active":"2024-09-10T09:36:06.763837+0000","last_peered":"2024-09-10T09:36:06.763837+0000","last_clean":"2024-09-10T09:36:06.763837+0000","last_became_active":"2024-09-10T09:35:36.618953+0000","last_became_peered":"2024-09-10T09:35:36.618953+0000","last_unstale":"2024-09-10T09:36:06.763837+0000","last_undegraded":"2024-09-10T09:36:06.763837+0000","last_fullsized":"2024-09-10T09:36:06.763837+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:40:25.181408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:36:06.764051+0000","last_change":"2024-09-10T09:35:28.571396+0000","last_active":"2024-09-10T09:36:06.764051+0000","last_peered":"2024-09-10T09:36:06.764051+0000","last_clean":"2024-09-10T09:36:06.764051+0000","last_became_active":"2024-09-10T09:35:28.571063+0000","last_became_peered":"2024-09-10T09:35:28.571063+0000","last_unstale":"2024-09-10T09:36:06.764051+0000","last_undegraded":"2024-09-10T09:36:06.764051+0000","last_fullsized":"2024-09-10T09:36:06.764051+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:38:17.965133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"94'17","reported_seq":57,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:36:14.486303+0000","last_change":"2024-09-10T09:35:25.851773+0000","last_active":"2024-09-10T09:36:14.486303+0000","last_peered":"2024-09-10T09:36:14.486303+0000","last_clean":"2024-09-10T09:36:14.486303+0000","last_became_active":"2024-09-10T09:35:25.851384+0000","last_became_peered":"2024-09-10T09:35:25.851384+0000","last_unstale":"2024-09-10T09:36:14.486303+0000","last_undegraded":"2024-09-10T09:36:14.486303+0000","last_fullsized":"2024-09-10T09:36:14.486303+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:03:32.338643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:36:06.873579+0000","last_change":"2024-09-10T09:35:36.620971+0000","last_active":"2024-09-10T09:36:06.873579+0000","last_peered":"2024-09-10T09:36:06.873579+0000","last_clean":"2024-09-10T09:36:06.873579+0000","last_became_active":"2024-09-10T09:35:36.620343+0000","last_became_peered":"2024-09-10T09:35:36.620343+0000","last_unstale":"2024-09-10T09:36:06.873579+0000","last_undegraded":"2024-09-10T09:36:06.873579+0000","last_fullsized":"2024-09-10T09:36:06.873579+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:17:19.517882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:36:06.766522+0000","last_change":"2024-09-10T09:35:28.565643+0000","last_active":"2024-09-10T09:36:06.766522+0000","last_peered":"2024-09-10T09:36:06.766522+0000","last_clean":"2024-09-10T09:36:06.766522+0000","last_became_active":"2024-09-10T09:35:28.564217+0000","last_became_peered":"2024-09-10T09:35:28.564217+0000","last_unstale":"2024-09-10T09:36:06.766522+0000","last_undegraded":"2024-09-10T09:36:06.766522+0000","last_fullsized":"2024-09-10T09:36:06.766522+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:53:34.562690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"94'5","reported_seq":60,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:36:27.508597+0000","last_change":"2024-09-10T09:35:25.534167+0000","last_active":"2024-09-10T09:36:27.508597+0000","last_peered":"2024-09-10T09:36:27.508597+0000","last_clean":"2024-09-10T09:36:27.508597+0000","last_became_active":"2024-09-10T09:35:25.533719+0000","last_became_peered":"2024-09-10T09:35:25.533719+0000","last_unstale":"2024-09-10T09:36:27.508597+0000","last_undegraded":"2024-09-10T09:36:27.508597+0000","last_fullsized":"2024-09-10T09:36:27.508597+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:00:03.537355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":18,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:36:06.761923+0000","last_change":"2024-09-10T09:35:36.628716+0000","last_active":"2024-09-10T09:36:06.761923+0000","last_peered":"2024-09-10T09:36:06.761923+0000","last_clean":"2024-09-10T09:36:06.761923+0000","last_became_active":"2024-09-10T09:35:36.628085+0000","last_became_peered":"2024-09-10T09:35:36.628085+0000","last_unstale":"2024-09-10T09:36:06.761923+0000","last_undegraded":"2024-09-10T09:36:06.761923+0000","last_fullsized":"2024-09-10T09:36:06.761923+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:45:01.279403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:36:06.763771+0000","last_change":"2024-09-10T09:35:28.566701+0000","last_active":"2024-09-10T09:36:06.763771+0000","last_peered":"2024-09-10T09:36:06.763771+0000","last_clean":"2024-09-10T09:36:06.763771+0000","last_became_active":"2024-09-10T09:35:28.566391+0000","last_became_peered":"2024-09-10T09:35:28.566391+0000","last_unstale":"2024-09-10T09:36:06.763771+0000","last_undegraded":"2024-09-10T09:36:06.763771+0000","last_fullsized":"2024-09-10T09:36:06.763771+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:57:59.918706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"94'5","reported_seq":45,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:36:14.497797+0000","last_change":"2024-09-10T09:35:25.532510+0000","last_active":"2024-09-10T09:36:14.497797+0000","last_peered":"2024-09-10T09:36:14.497797+0000","last_clean":"2024-09-10T09:36:14.497797+0000","last_became_active":"2024-09-10T09:35:25.532193+0000","last_became_peered":"2024-09-10T09:35:25.532193+0000","last_unstale":"2024-09-10T09:36:14.497797+0000","last_undegraded":"2024-09-10T09:36:14.497797+0000","last_fullsized":"2024-09-10T09:36:14.497797+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:17:24.685178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:36:06.761950+0000","last_change":"2024-09-10T09:35:36.630742+0000","last_active":"2024-09-10T09:36:06.761950+0000","last_peered":"2024-09-10T09:36:06.761950+0000","last_clean":"2024-09-10T09:36:06.761950+0000","last_became_active":"2024-09-10T09:35:36.630335+0000","last_became_peered":"2024-09-10T09:35:36.630335+0000","last_unstale":"2024-09-10T09:36:06.761950+0000","last_undegraded":"2024-09-10T09:36:06.761950+0000","last_fullsized":"2024-09-10T09:36:06.761950+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:29:12.029390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:36:06.873091+0000","last_change":"2024-09-10T09:35:28.581817+0000","last_active":"2024-09-10T09:36:06.873091+0000","last_peered":"2024-09-10T09:36:06.873091+0000","last_clean":"2024-09-10T09:36:06.873091+0000","last_became_active":"2024-09-10T09:35:28.580316+0000","last_became_peered":"2024-09-10T09:35:28.580316+0000","last_unstale":"2024-09-10T09:36:06.873091+0000","last_undegraded":"2024-09-10T09:36:06.873091+0000","last_fullsized":"2024-09-10T09:36:06.873091+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:04:08.249564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"94'9","reported_seq":50,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:36:14.498084+0000","last_change":"2024-09-10T09:35:25.537474+0000","last_active":"2024-09-10T09:36:14.498084+0000","last_peered":"2024-09-10T09:36:14.498084+0000","last_clean":"2024-09-10T09:36:14.498084+0000","last_became_active":"2024-09-10T09:35:25.537096+0000","last_became_peered":"2024-09-10T09:35:25.537096+0000","last_unstale":"2024-09-10T09:36:14.498084+0000","last_undegraded":"2024-09-10T09:36:14.498084+0000","last_fullsized":"2024-09-10T09:36:14.498084+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:13:06.008458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:36:06.763503+0000","last_change":"2024-09-10T09:35:36.622788+0000","last_active":"2024-09-10T09:36:06.763503+0000","last_peered":"2024-09-10T09:36:06.763503+0000","last_clean":"2024-09-10T09:36:06.763503+0000","last_became_active":"2024-09-10T09:35:36.622377+0000","last_became_peered":"2024-09-10T09:35:36.622377+0000","last_unstale":"2024-09-10T09:36:06.763503+0000","last_undegraded":"2024-09-10T09:36:06.763503+0000","last_fullsized":"2024-09-10T09:36:06.763503+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:29:56.680864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:36:06.767059+0000","last_change":"2024-09-10T09:35:28.564529+0000","last_active":"2024-09-10T09:36:06.767059+0000","last_peered":"2024-09-10T09:36:06.767059+0000","last_clean":"2024-09-10T09:36:06.767059+0000","last_became_active":"2024-09-10T09:35:28.564223+0000","last_became_peered":"2024-09-10T09:35:28.564223+0000","last_unstale":"2024-09-10T09:36:06.767059+0000","last_undegraded":"2024-09-10T09:36:06.767059+0000","last_fullsized":"2024-09-10T09:36:06.767059+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:21:24.296578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"94'31","reported_seq":109,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:36:29.512195+0000","last_change":"2024-09-10T09:35:25.534331+0000","last_active":"2024-09-10T09:36:29.512195+0000","last_peered":"2024-09-10T09:36:29.512195+0000","last_clean":"2024-09-10T09:36:29.512195+0000","last_became_active":"2024-09-10T09:35:25.533985+0000","last_became_peered":"2024-09-10T09:35:25.533985+0000","last_unstale":"2024-09-10T09:36:29.512195+0000","last_undegraded":"2024-09-10T09:36:29.512195+0000","last_fullsized":"2024-09-10T09:36:29.512195+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:14:37.050650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":52413,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":39,"num_read_kb":70,"num_write":33,"num_write_kb":63,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:36:06.760797+0000","last_change":"2024-09-10T09:35:36.632835+0000","last_active":"2024-09-10T09:36:06.760797+0000","last_peered":"2024-09-10T09:36:06.760797+0000","last_clean":"2024-09-10T09:36:06.760797+0000","last_became_active":"2024-09-10T09:35:36.630962+0000","last_became_peered":"2024-09-10T09:35:36.630962+0000","last_unstale":"2024-09-10T09:36:06.760797+0000","last_undegraded":"2024-09-10T09:36:06.760797+0000","last_fullsized":"2024-09-10T09:36:06.760797+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:59:27.613721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:36:06.765554+0000","last_change":"2024-09-10T09:35:28.567292+0000","last_active":"2024-09-10T09:36:06.765554+0000","last_peered":"2024-09-10T09:36:06.765554+0000","last_clean":"2024-09-10T09:36:06.765554+0000","last_became_active":"2024-09-10T09:35:28.566973+0000","last_became_peered":"2024-09-10T09:35:28.566973+0000","last_unstale":"2024-09-10T09:36:06.765554+0000","last_undegraded":"2024-09-10T09:36:06.765554+0000","last_fullsized":"2024-09-10T09:36:06.765554+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:05:37.251933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"94'5","reported_seq":45,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:36:13.816041+0000","last_change":"2024-09-10T09:35:25.852171+0000","last_active":"2024-09-10T09:36:13.816041+0000","last_peered":"2024-09-10T09:36:13.816041+0000","last_clean":"2024-09-10T09:36:13.816041+0000","last_became_active":"2024-09-10T09:35:25.851580+0000","last_became_peered":"2024-09-10T09:35:25.851580+0000","last_unstale":"2024-09-10T09:36:13.816041+0000","last_undegraded":"2024-09-10T09:36:13.816041+0000","last_fullsized":"2024-09-10T09:36:13.816041+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:03.263152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:36:06.765208+0000","last_change":"2024-09-10T09:35:36.629212+0000","last_active":"2024-09-10T09:36:06.765208+0000","last_peered":"2024-09-10T09:36:06.765208+0000","last_clean":"2024-09-10T09:36:06.765208+0000","last_became_active":"2024-09-10T09:35:36.628753+0000","last_became_peered":"2024-09-10T09:35:36.628753+0000","last_unstale":"2024-09-10T09:36:06.765208+0000","last_undegraded":"2024-09-10T09:36:06.765208+0000","last_fullsized":"2024-09-10T09:36:06.765208+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:54:45.311361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:36:06.760286+0000","last_change":"2024-09-10T09:35:28.572497+0000","last_active":"2024-09-10T09:36:06.760286+0000","last_peered":"2024-09-10T09:36:06.760286+0000","last_clean":"2024-09-10T09:36:06.760286+0000","last_became_active":"2024-09-10T09:35:28.569238+0000","last_became_peered":"2024-09-10T09:35:28.569238+0000","last_unstale":"2024-09-10T09:36:06.760286+0000","last_undegraded":"2024-09-10T09:36:06.760286+0000","last_fullsized":"2024-09-10T09:36:06.760286+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:38:47.189209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"94'4","reported_seq":42,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:36:14.497898+0000","last_change":"2024-09-10T09:35:25.536369+0000","last_active":"2024-09-10T09:36:14.497898+0000","last_peered":"2024-09-10T09:36:14.497898+0000","last_clean":"2024-09-10T09:36:14.497898+0000","last_became_active":"2024-09-10T09:35:25.535607+0000","last_became_peered":"2024-09-10T09:35:25.535607+0000","last_unstale":"2024-09-10T09:36:14.497898+0000","last_undegraded":"2024-09-10T09:36:14.497898+0000","last_fullsized":"2024-09-10T09:36:14.497898+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:49:18.621934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:36:06.760500+0000","last_change":"2024-09-10T09:35:36.627977+0000","last_active":"2024-09-10T09:36:06.760500+0000","last_peered":"2024-09-10T09:36:06.760500+0000","last_clean":"2024-09-10T09:36:06.760500+0000","last_became_active":"2024-09-10T09:35:36.626942+0000","last_became_peered":"2024-09-10T09:35:36.626942+0000","last_unstale":"2024-09-10T09:36:06.760500+0000","last_undegraded":"2024-09-10T09:36:06.760500+0000","last_fullsized":"2024-09-10T09:36:06.760500+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:37:18.209424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:36:06.766076+0000","last_change":"2024-09-10T09:35:28.562670+0000","last_active":"2024-09-10T09:36:06.766076+0000","last_peered":"2024-09-10T09:36:06.766076+0000","last_clean":"2024-09-10T09:36:06.766076+0000","last_became_active":"2024-09-10T09:35:28.562028+0000","last_became_peered":"2024-09-10T09:35:28.562028+0000","last_unstale":"2024-09-10T09:36:06.766076+0000","last_undegraded":"2024-09-10T09:36:06.766076+0000","last_fullsized":"2024-09-10T09:36:06.766076+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:37:15.176050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"94'4","reported_seq":75,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:36:29.511521+0000","last_change":"2024-09-10T09:35:25.529861+0000","last_active":"2024-09-10T09:36:29.511521+0000","last_peered":"2024-09-10T09:36:29.511521+0000","last_clean":"2024-09-10T09:36:29.511521+0000","last_became_active":"2024-09-10T09:35:25.529475+0000","last_became_peered":"2024-09-10T09:35:25.529475+0000","last_unstale":"2024-09-10T09:36:29.511521+0000","last_undegraded":"2024-09-10T09:36:29.511521+0000","last_fullsized":"2024-09-10T09:36:29.511521+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:44:27.086509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":32,"num_read_kb":32,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:36:06.763318+0000","last_change":"2024-09-10T09:35:36.615358+0000","last_active":"2024-09-10T09:36:06.763318+0000","last_peered":"2024-09-10T09:36:06.763318+0000","last_clean":"2024-09-10T09:36:06.763318+0000","last_became_active":"2024-09-10T09:35:36.614901+0000","last_became_peered":"2024-09-10T09:35:36.614901+0000","last_unstale":"2024-09-10T09:36:06.763318+0000","last_undegraded":"2024-09-10T09:36:06.763318+0000","last_fullsized":"2024-09-10T09:36:06.763318+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:56:54.248266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:36:06.759865+0000","last_change":"2024-09-10T09:35:36.629021+0000","last_active":"2024-09-10T09:36:06.759865+0000","last_peered":"2024-09-10T09:36:06.759865+0000","last_clean":"2024-09-10T09:36:06.759865+0000","last_became_active":"2024-09-10T09:35:36.628159+0000","last_became_peered":"2024-09-10T09:35:36.628159+0000","last_unstale":"2024-09-10T09:36:06.759865+0000","last_undegraded":"2024-09-10T09:36:06.759865+0000","last_fullsized":"2024-09-10T09:36:06.759865+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:50:48.615236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:36:06.760577+0000","last_change":"2024-09-10T09:35:28.579333+0000","last_active":"2024-09-10T09:36:06.760577+0000","last_peered":"2024-09-10T09:36:06.760577+0000","last_clean":"2024-09-10T09:36:06.760577+0000","last_became_active":"2024-09-10T09:35:28.578440+0000","last_became_peered":"2024-09-10T09:35:28.578440+0000","last_unstale":"2024-09-10T09:36:06.760577+0000","last_undegraded":"2024-09-10T09:36:06.760577+0000","last_fullsized":"2024-09-10T09:36:06.760577+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:07:28.701938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"94'3","reported_seq":43,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:36:14.490660+0000","last_change":"2024-09-10T09:35:25.530857+0000","last_active":"2024-09-10T09:36:14.490660+0000","last_peered":"2024-09-10T09:36:14.490660+0000","last_clean":"2024-09-10T09:36:14.490660+0000","last_became_active":"2024-09-10T09:35:25.530492+0000","last_became_peered":"2024-09-10T09:35:25.530492+0000","last_unstale":"2024-09-10T09:36:14.490660+0000","last_undegraded":"2024-09-10T09:36:14.490660+0000","last_fullsized":"2024-09-10T09:36:14.490660+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:07:21.726598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":31,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:36:06.770408+0000","last_change":"2024-09-10T09:36:06.770408+0000","last_active":"2024-09-10T09:36:06.770408+0000","last_peered":"2024-09-10T09:36:06.770408+0000","last_clean":"2024-09-10T09:36:06.770408+0000","last_became_active":"2024-09-10T09:36:06.769915+0000","last_became_peered":"2024-09-10T09:36:06.769915+0000","last_unstale":"2024-09-10T09:36:06.770408+0000","last_undegraded":"2024-09-10T09:36:06.770408+0000","last_fullsized":"2024-09-10T09:36:06.770408+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:56:09.795537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8,0],"acting":[11,4,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:36:06.763197+0000","last_change":"2024-09-10T09:35:28.570804+0000","last_active":"2024-09-10T09:36:06.763197+0000","last_peered":"2024-09-10T09:36:06.763197+0000","last_clean":"2024-09-10T09:36:06.763197+0000","last_became_active":"2024-09-10T09:35:28.570490+0000","last_became_peered":"2024-09-10T09:35:28.570490+0000","last_unstale":"2024-09-10T09:36:06.763197+0000","last_undegraded":"2024-09-10T09:36:06.763197+0000","last_fullsized":"2024-09-10T09:36:06.763197+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:13:38.999625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"94'3","reported_seq":42,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:36:12.814442+0000","last_change":"2024-09-10T09:35:25.536347+0000","last_active":"2024-09-10T09:36:12.814442+0000","last_peered":"2024-09-10T09:36:12.814442+0000","last_clean":"2024-09-10T09:36:12.814442+0000","last_became_active":"2024-09-10T09:35:25.535964+0000","last_became_peered":"2024-09-10T09:35:25.535964+0000","last_unstale":"2024-09-10T09:36:12.814442+0000","last_undegraded":"2024-09-10T09:36:12.814442+0000","last_fullsized":"2024-09-10T09:36:12.814442+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:36:49.035666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"94'7","reported_seq":46,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:36:13.818400+0000","last_change":"2024-09-10T09:35:25.532148+0000","last_active":"2024-09-10T09:36:13.818400+0000","last_peered":"2024-09-10T09:36:13.818400+0000","last_clean":"2024-09-10T09:36:13.818400+0000","last_became_active":"2024-09-10T09:35:25.531781+0000","last_became_peered":"2024-09-10T09:35:25.531781+0000","last_unstale":"2024-09-10T09:36:13.818400+0000","last_undegraded":"2024-09-10T09:36:13.818400+0000","last_fullsized":"2024-09-10T09:36:13.818400+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:52:46.740342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:36:06.763468+0000","last_change":"2024-09-10T09:35:28.578336+0000","last_active":"2024-09-10T09:36:06.763468+0000","last_peered":"2024-09-10T09:36:06.763468+0000","last_clean":"2024-09-10T09:36:06.763468+0000","last_became_active":"2024-09-10T09:35:28.577899+0000","last_became_peered":"2024-09-10T09:35:28.577899+0000","last_unstale":"2024-09-10T09:36:06.763468+0000","last_undegraded":"2024-09-10T09:36:06.763468+0000","last_fullsized":"2024-09-10T09:36:06.763468+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:13:04.478195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:36:06.760465+0000","last_change":"2024-09-10T09:35:36.630918+0000","last_active":"2024-09-10T09:36:06.760465+0000","last_peered":"2024-09-10T09:36:06.760465+0000","last_clean":"2024-09-10T09:36:06.760465+0000","last_became_active":"2024-09-10T09:35:36.630308+0000","last_became_peered":"2024-09-10T09:35:36.630308+0000","last_unstale":"2024-09-10T09:36:06.760465+0000","last_undegraded":"2024-09-10T09:36:06.760465+0000","last_fullsized":"2024-09-10T09:36:06.760465+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:25:44.954970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"94'6","reported_seq":64,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:36:28.510545+0000","last_change":"2024-09-10T09:35:25.537670+0000","last_active":"2024-09-10T09:36:28.510545+0000","last_peered":"2024-09-10T09:36:28.510545+0000","last_clean":"2024-09-10T09:36:28.510545+0000","last_became_active":"2024-09-10T09:35:25.537359+0000","last_became_peered":"2024-09-10T09:35:25.537359+0000","last_unstale":"2024-09-10T09:36:28.510545+0000","last_undegraded":"2024-09-10T09:36:28.510545+0000","last_fullsized":"2024-09-10T09:36:28.510545+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:06:55.074836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1015,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":19,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:36:06.766706+0000","last_change":"2024-09-10T09:35:28.567915+0000","last_active":"2024-09-10T09:36:06.766706+0000","last_peered":"2024-09-10T09:36:06.766706+0000","last_clean":"2024-09-10T09:36:06.766706+0000","last_became_active":"2024-09-10T09:35:28.567416+0000","last_became_peered":"2024-09-10T09:35:28.567416+0000","last_unstale":"2024-09-10T09:36:06.766706+0000","last_undegraded":"2024-09-10T09:36:06.766706+0000","last_fullsized":"2024-09-10T09:36:06.766706+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:07:25.704080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:36:06.760638+0000","last_change":"2024-09-10T09:35:36.615386+0000","last_active":"2024-09-10T09:36:06.760638+0000","last_peered":"2024-09-10T09:36:06.760638+0000","last_clean":"2024-09-10T09:36:06.760638+0000","last_became_active":"2024-09-10T09:35:36.614967+0000","last_became_peered":"2024-09-10T09:35:36.614967+0000","last_unstale":"2024-09-10T09:36:06.760638+0000","last_undegraded":"2024-09-10T09:36:06.760638+0000","last_fullsized":"2024-09-10T09:36:06.760638+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:47.620729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:36:06.762580+0000","last_change":"2024-09-10T09:35:28.564411+0000","last_active":"2024-09-10T09:36:06.762580+0000","last_peered":"2024-09-10T09:36:06.762580+0000","last_clean":"2024-09-10T09:36:06.762580+0000","last_became_active":"2024-09-10T09:35:28.564034+0000","last_became_peered":"2024-09-10T09:35:28.564034+0000","last_unstale":"2024-09-10T09:36:06.762580+0000","last_undegraded":"2024-09-10T09:36:06.762580+0000","last_fullsized":"2024-09-10T09:36:06.762580+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:38:17.687991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:36:06.760601+0000","last_change":"2024-09-10T09:35:28.564619+0000","last_active":"2024-09-10T09:36:06.760601+0000","last_peered":"2024-09-10T09:36:06.760601+0000","last_clean":"2024-09-10T09:36:06.760601+0000","last_became_active":"2024-09-10T09:35:28.563890+0000","last_became_peered":"2024-09-10T09:35:28.563890+0000","last_unstale":"2024-09-10T09:36:06.760601+0000","last_undegraded":"2024-09-10T09:36:06.760601+0000","last_fullsized":"2024-09-10T09:36:06.760601+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:51:42.481911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:36:06.760213+0000","last_change":"2024-09-10T09:35:28.558909+0000","last_active":"2024-09-10T09:36:06.760213+0000","last_peered":"2024-09-10T09:36:06.760213+0000","last_clean":"2024-09-10T09:36:06.760213+0000","last_became_active":"2024-09-10T09:35:28.558604+0000","last_became_peered":"2024-09-10T09:35:28.558604+0000","last_unstale":"2024-09-10T09:36:06.760213+0000","last_undegraded":"2024-09-10T09:36:06.760213+0000","last_fullsized":"2024-09-10T09:36:06.760213+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:11.928446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:36:06.759939+0000","last_change":"2024-09-10T09:35:28.574459+0000","last_active":"2024-09-10T09:36:06.759939+0000","last_peered":"2024-09-10T09:36:06.759939+0000","last_clean":"2024-09-10T09:36:06.759939+0000","last_became_active":"2024-09-10T09:35:28.574154+0000","last_became_peered":"2024-09-10T09:35:28.574154+0000","last_unstale":"2024-09-10T09:36:06.759939+0000","last_undegraded":"2024-09-10T09:36:06.759939+0000","last_fullsized":"2024-09-10T09:36:06.759939+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:59:09.650536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:36:06.873415+0000","last_change":"2024-09-10T09:35:28.576999+0000","last_active":"2024-09-10T09:36:06.873415+0000","last_peered":"2024-09-10T09:36:06.873415+0000","last_clean":"2024-09-10T09:36:06.873415+0000","last_became_active":"2024-09-10T09:35:28.575616+0000","last_became_peered":"2024-09-10T09:35:28.575616+0000","last_unstale":"2024-09-10T09:36:06.873415+0000","last_undegraded":"2024-09-10T09:36:06.873415+0000","last_fullsized":"2024-09-10T09:36:06.873415+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:18:08.771760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:36:06.762869+0000","last_change":"2024-09-10T09:35:28.564846+0000","last_active":"2024-09-10T09:36:06.762869+0000","last_peered":"2024-09-10T09:36:06.762869+0000","last_clean":"2024-09-10T09:36:06.762869+0000","last_became_active":"2024-09-10T09:35:28.564498+0000","last_became_peered":"2024-09-10T09:35:28.564498+0000","last_unstale":"2024-09-10T09:36:06.762869+0000","last_undegraded":"2024-09-10T09:36:06.762869+0000","last_fullsized":"2024-09-10T09:36:06.762869+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:33:43.041730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:36:06.760669+0000","last_change":"2024-09-10T09:35:28.558093+0000","last_active":"2024-09-10T09:36:06.760669+0000","last_peered":"2024-09-10T09:36:06.760669+0000","last_clean":"2024-09-10T09:36:06.760669+0000","last_became_active":"2024-09-10T09:35:28.557266+0000","last_became_peered":"2024-09-10T09:35:28.557266+0000","last_unstale":"2024-09-10T09:36:06.760669+0000","last_undegraded":"2024-09-10T09:36:06.760669+0000","last_fullsized":"2024-09-10T09:36:06.760669+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:51:30.946065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:36:06.770298+0000","last_change":"2024-09-10T09:36:06.770298+0000","last_active":"2024-09-10T09:36:06.770298+0000","last_peered":"2024-09-10T09:36:06.770298+0000","last_clean":"2024-09-10T09:36:06.770298+0000","last_became_active":"2024-09-10T09:36:06.769848+0000","last_became_peered":"2024-09-10T09:36:06.769848+0000","last_unstale":"2024-09-10T09:36:06.770298+0000","last_undegraded":"2024-09-10T09:36:06.770298+0000","last_fullsized":"2024-09-10T09:36:06.770298+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:00:30.470887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,8,2],"acting":[3,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:36:06.763611+0000","last_change":"2024-09-10T09:35:28.578158+0000","last_active":"2024-09-10T09:36:06.763611+0000","last_peered":"2024-09-10T09:36:06.763611+0000","last_clean":"2024-09-10T09:36:06.763611+0000","last_became_active":"2024-09-10T09:35:28.577824+0000","last_became_peered":"2024-09-10T09:35:28.577824+0000","last_unstale":"2024-09-10T09:36:06.763611+0000","last_undegraded":"2024-09-10T09:36:06.763611+0000","last_fullsized":"2024-09-10T09:36:06.763611+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:40:27.231164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:36:06.763546+0000","last_change":"2024-09-10T09:35:28.562670+0000","last_active":"2024-09-10T09:36:06.763546+0000","last_peered":"2024-09-10T09:36:06.763546+0000","last_clean":"2024-09-10T09:36:06.763546+0000","last_became_active":"2024-09-10T09:35:28.562311+0000","last_became_peered":"2024-09-10T09:35:28.562311+0000","last_unstale":"2024-09-10T09:36:06.763546+0000","last_undegraded":"2024-09-10T09:36:06.763546+0000","last_fullsized":"2024-09-10T09:36:06.763546+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:07:11.716142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:36:06.765363+0000","last_change":"2024-09-10T09:35:28.561124+0000","last_active":"2024-09-10T09:36:06.765363+0000","last_peered":"2024-09-10T09:36:06.765363+0000","last_clean":"2024-09-10T09:36:06.765363+0000","last_became_active":"2024-09-10T09:35:28.560777+0000","last_became_peered":"2024-09-10T09:35:28.560777+0000","last_unstale":"2024-09-10T09:36:06.765363+0000","last_undegraded":"2024-09-10T09:36:06.765363+0000","last_fullsized":"2024-09-10T09:36:06.765363+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:47:04.837596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:36:06.873206+0000","last_change":"2024-09-10T09:35:28.576790+0000","last_active":"2024-09-10T09:36:06.873206+0000","last_peered":"2024-09-10T09:36:06.873206+0000","last_clean":"2024-09-10T09:36:06.873206+0000","last_became_active":"2024-09-10T09:35:28.576464+0000","last_became_peered":"2024-09-10T09:35:28.576464+0000","last_unstale":"2024-09-10T09:36:06.873206+0000","last_undegraded":"2024-09-10T09:36:06.873206+0000","last_fullsized":"2024-09-10T09:36:06.873206+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:40:53.517577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:36:06.762086+0000","last_change":"2024-09-10T09:35:28.569935+0000","last_active":"2024-09-10T09:36:06.762086+0000","last_peered":"2024-09-10T09:36:06.762086+0000","last_clean":"2024-09-10T09:36:06.762086+0000","last_became_active":"2024-09-10T09:35:28.569539+0000","last_became_peered":"2024-09-10T09:35:28.569539+0000","last_unstale":"2024-09-10T09:36:06.762086+0000","last_undegraded":"2024-09-10T09:36:06.762086+0000","last_fullsized":"2024-09-10T09:36:06.762086+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:40:45.016269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:36:06.760825+0000","last_change":"2024-09-10T09:35:28.571619+0000","last_active":"2024-09-10T09:36:06.760825+0000","last_peered":"2024-09-10T09:36:06.760825+0000","last_clean":"2024-09-10T09:36:06.760825+0000","last_became_active":"2024-09-10T09:35:28.568591+0000","last_became_peered":"2024-09-10T09:35:28.568591+0000","last_unstale":"2024-09-10T09:36:06.760825+0000","last_undegraded":"2024-09-10T09:36:06.760825+0000","last_fullsized":"2024-09-10T09:36:06.760825+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:07:49.959712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:36:06.761923+0000","last_change":"2024-09-10T09:35:28.573336+0000","last_active":"2024-09-10T09:36:06.761923+0000","last_peered":"2024-09-10T09:36:06.761923+0000","last_clean":"2024-09-10T09:36:06.761923+0000","last_became_active":"2024-09-10T09:35:28.572991+0000","last_became_peered":"2024-09-10T09:35:28.572991+0000","last_unstale":"2024-09-10T09:36:06.761923+0000","last_undegraded":"2024-09-10T09:36:06.761923+0000","last_fullsized":"2024-09-10T09:36:06.761923+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:30:02.905869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:36:06.763406+0000","last_change":"2024-09-10T09:35:28.571945+0000","last_active":"2024-09-10T09:36:06.763406+0000","last_peered":"2024-09-10T09:36:06.763406+0000","last_clean":"2024-09-10T09:36:06.763406+0000","last_became_active":"2024-09-10T09:35:28.571546+0000","last_became_peered":"2024-09-10T09:35:28.571546+0000","last_unstale":"2024-09-10T09:36:06.763406+0000","last_undegraded":"2024-09-10T09:36:06.763406+0000","last_fullsized":"2024-09-10T09:36:06.763406+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:36:47.946864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:36:06.764770+0000","last_change":"2024-09-10T09:35:28.562712+0000","last_active":"2024-09-10T09:36:06.764770+0000","last_peered":"2024-09-10T09:36:06.764770+0000","last_clean":"2024-09-10T09:36:06.764770+0000","last_became_active":"2024-09-10T09:35:28.562396+0000","last_became_peered":"2024-09-10T09:35:28.562396+0000","last_unstale":"2024-09-10T09:36:06.764770+0000","last_undegraded":"2024-09-10T09:36:06.764770+0000","last_fullsized":"2024-09-10T09:36:06.764770+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:25:47.419162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:36:06.759492+0000","last_change":"2024-09-10T09:35:28.566354+0000","last_active":"2024-09-10T09:36:06.759492+0000","last_peered":"2024-09-10T09:36:06.759492+0000","last_clean":"2024-09-10T09:36:06.759492+0000","last_became_active":"2024-09-10T09:35:28.565248+0000","last_became_peered":"2024-09-10T09:35:28.565248+0000","last_unstale":"2024-09-10T09:36:06.759492+0000","last_undegraded":"2024-09-10T09:36:06.759492+0000","last_fullsized":"2024-09-10T09:36:06.759492+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:48:46.741010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:36:06.759606+0000","last_change":"2024-09-10T09:35:28.563331+0000","last_active":"2024-09-10T09:36:06.759606+0000","last_peered":"2024-09-10T09:36:06.759606+0000","last_clean":"2024-09-10T09:36:06.759606+0000","last_became_active":"2024-09-10T09:35:28.563020+0000","last_became_peered":"2024-09-10T09:35:28.563020+0000","last_unstale":"2024-09-10T09:36:06.759606+0000","last_undegraded":"2024-09-10T09:36:06.759606+0000","last_fullsized":"2024-09-10T09:36:06.759606+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:27.653551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:36:06.873157+0000","last_change":"2024-09-10T09:35:28.566687+0000","last_active":"2024-09-10T09:36:06.873157+0000","last_peered":"2024-09-10T09:36:06.873157+0000","last_clean":"2024-09-10T09:36:06.873157+0000","last_became_active":"2024-09-10T09:35:28.566253+0000","last_became_peered":"2024-09-10T09:35:28.566253+0000","last_unstale":"2024-09-10T09:36:06.873157+0000","last_undegraded":"2024-09-10T09:36:06.873157+0000","last_fullsized":"2024-09-10T09:36:06.873157+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:00:15.709725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:36:06.764878+0000","last_change":"2024-09-10T09:35:28.562251+0000","last_active":"2024-09-10T09:36:06.764878+0000","last_peered":"2024-09-10T09:36:06.764878+0000","last_clean":"2024-09-10T09:36:06.764878+0000","last_became_active":"2024-09-10T09:35:28.561933+0000","last_became_peered":"2024-09-10T09:35:28.561933+0000","last_unstale":"2024-09-10T09:36:06.764878+0000","last_undegraded":"2024-09-10T09:36:06.764878+0000","last_fullsized":"2024-09-10T09:36:06.764878+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:30:41.447954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:36:06.763944+0000","last_change":"2024-09-10T09:35:28.572164+0000","last_active":"2024-09-10T09:36:06.763944+0000","last_peered":"2024-09-10T09:36:06.763944+0000","last_clean":"2024-09-10T09:36:06.763944+0000","last_became_active":"2024-09-10T09:35:28.571825+0000","last_became_peered":"2024-09-10T09:35:28.571825+0000","last_unstale":"2024-09-10T09:36:06.763944+0000","last_undegraded":"2024-09-10T09:36:06.763944+0000","last_fullsized":"2024-09-10T09:36:06.763944+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:59:50.967269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:36:06.873657+0000","last_change":"2024-09-10T09:35:28.568098+0000","last_active":"2024-09-10T09:36:06.873657+0000","last_peered":"2024-09-10T09:36:06.873657+0000","last_clean":"2024-09-10T09:36:06.873657+0000","last_became_active":"2024-09-10T09:35:28.567765+0000","last_became_peered":"2024-09-10T09:35:28.567765+0000","last_unstale":"2024-09-10T09:36:06.873657+0000","last_undegraded":"2024-09-10T09:36:06.873657+0000","last_fullsized":"2024-09-10T09:36:06.873657+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:55:57.260320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:36:06.764041+0000","last_change":"2024-09-10T09:35:28.564123+0000","last_active":"2024-09-10T09:36:06.764041+0000","last_peered":"2024-09-10T09:36:06.764041+0000","last_clean":"2024-09-10T09:36:06.764041+0000","last_became_active":"2024-09-10T09:35:28.563814+0000","last_became_peered":"2024-09-10T09:35:28.563814+0000","last_unstale":"2024-09-10T09:36:06.764041+0000","last_undegraded":"2024-09-10T09:36:06.764041+0000","last_fullsized":"2024-09-10T09:36:06.764041+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:14:17.912087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:36:06.763268+0000","last_change":"2024-09-10T09:35:28.568601+0000","last_active":"2024-09-10T09:36:06.763268+0000","last_peered":"2024-09-10T09:36:06.763268+0000","last_clean":"2024-09-10T09:36:06.763268+0000","last_became_active":"2024-09-10T09:35:28.568303+0000","last_became_peered":"2024-09-10T09:35:28.568303+0000","last_unstale":"2024-09-10T09:36:06.763268+0000","last_undegraded":"2024-09-10T09:36:06.763268+0000","last_fullsized":"2024-09-10T09:36:06.763268+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:30:01.016420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:36:06.763766+0000","last_change":"2024-09-10T09:35:28.565217+0000","last_active":"2024-09-10T09:36:06.763766+0000","last_peered":"2024-09-10T09:36:06.763766+0000","last_clean":"2024-09-10T09:36:06.763766+0000","last_became_active":"2024-09-10T09:35:28.564261+0000","last_became_peered":"2024-09-10T09:35:28.564261+0000","last_unstale":"2024-09-10T09:36:06.763766+0000","last_undegraded":"2024-09-10T09:36:06.763766+0000","last_fullsized":"2024-09-10T09:36:06.763766+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:29:30.298487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:36:06.872976+0000","last_change":"2024-09-10T09:35:28.569225+0000","last_active":"2024-09-10T09:36:06.872976+0000","last_peered":"2024-09-10T09:36:06.872976+0000","last_clean":"2024-09-10T09:36:06.872976+0000","last_became_active":"2024-09-10T09:35:28.568891+0000","last_became_peered":"2024-09-10T09:35:28.568891+0000","last_unstale":"2024-09-10T09:36:06.872976+0000","last_undegraded":"2024-09-10T09:36:06.872976+0000","last_fullsized":"2024-09-10T09:36:06.872976+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:33:45.811781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:36:06.766738+0000","last_change":"2024-09-10T09:35:28.578362+0000","last_active":"2024-09-10T09:36:06.766738+0000","last_peered":"2024-09-10T09:36:06.766738+0000","last_clean":"2024-09-10T09:36:06.766738+0000","last_became_active":"2024-09-10T09:35:28.577867+0000","last_became_peered":"2024-09-10T09:35:28.577867+0000","last_unstale":"2024-09-10T09:36:06.766738+0000","last_undegraded":"2024-09-10T09:36:06.766738+0000","last_fullsized":"2024-09-10T09:36:06.766738+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:58:17.041663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:36:06.764779+0000","last_change":"2024-09-10T09:35:28.568183+0000","last_active":"2024-09-10T09:36:06.764779+0000","last_peered":"2024-09-10T09:36:06.764779+0000","last_clean":"2024-09-10T09:36:06.764779+0000","last_became_active":"2024-09-10T09:35:28.566775+0000","last_became_peered":"2024-09-10T09:35:28.566775+0000","last_unstale":"2024-09-10T09:36:06.764779+0000","last_undegraded":"2024-09-10T09:36:06.764779+0000","last_fullsized":"2024-09-10T09:36:06.764779+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:28:10.008889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:36:06.759752+0000","last_change":"2024-09-10T09:35:28.576122+0000","last_active":"2024-09-10T09:36:06.759752+0000","last_peered":"2024-09-10T09:36:06.759752+0000","last_clean":"2024-09-10T09:36:06.759752+0000","last_became_active":"2024-09-10T09:35:28.575831+0000","last_became_peered":"2024-09-10T09:35:28.575831+0000","last_unstale":"2024-09-10T09:36:06.759752+0000","last_undegraded":"2024-09-10T09:36:06.759752+0000","last_fullsized":"2024-09-10T09:36:06.759752+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:37:51.604658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:36:06.759864+0000","last_change":"2024-09-10T09:35:28.564441+0000","last_active":"2024-09-10T09:36:06.759864+0000","last_peered":"2024-09-10T09:36:06.759864+0000","last_clean":"2024-09-10T09:36:06.759864+0000","last_became_active":"2024-09-10T09:35:28.564095+0000","last_became_peered":"2024-09-10T09:35:28.564095+0000","last_unstale":"2024-09-10T09:36:06.759864+0000","last_undegraded":"2024-09-10T09:36:06.759864+0000","last_fullsized":"2024-09-10T09:36:06.759864+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:53:28.275361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:36:06.763287+0000","last_change":"2024-09-10T09:35:28.581921+0000","last_active":"2024-09-10T09:36:06.763287+0000","last_peered":"2024-09-10T09:36:06.763287+0000","last_clean":"2024-09-10T09:36:06.763287+0000","last_became_active":"2024-09-10T09:35:28.581573+0000","last_became_peered":"2024-09-10T09:35:28.581573+0000","last_unstale":"2024-09-10T09:36:06.763287+0000","last_undegraded":"2024-09-10T09:36:06.763287+0000","last_fullsized":"2024-09-10T09:36:06.763287+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:58:38.983934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":59621,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":196,"num_read_kb":164,"num_write":241,"num_write_kb":129,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":540672,"data_stored":178863,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":208,"ondisk_log_size":208,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":285627,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":77,"seq":330712481818,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32204,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712948,"statfs":{"total":95995035648,"available":95962058752,"internally_reserved":0,"allocated":1044480,"data_stored":535186,"data_compressed":26500,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":70,"seq":300647710750,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28184,"kb_used_data":1160,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716968,"statfs":{"total":95995035648,"available":95966175232,"internally_reserved":0,"allocated":1187840,"data_stored":1178511,"data_compressed":62821,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":65,"seq":279172874277,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32228,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712924,"statfs":{"total":95995035648,"available":95962034176,"internally_reserved":0,"allocated":1069056,"data_stored":535116,"data_compressed":26500,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1599,"internal_metadata":31914433},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":57,"seq":244813135913,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32224,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712928,"statfs":{"total":95995035648,"available":95962038272,"internally_reserved":0,"allocated":1064960,"data_stored":535494,"data_compressed":26500,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":50,"seq":214748364847,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28116,"kb_used_data":1092,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717036,"statfs":{"total":95995035648,"available":95966244864,"internally_reserved":0,"allocated":1118208,"data_stored":1132386,"data_compressed":62821,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":45,"seq":193273528372,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36364,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708788,"statfs":{"total":95995035648,"available":95957798912,"internally_reserved":0,"allocated":1044480,"data_stored":531013,"data_compressed":26500,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790011,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28124,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717028,"statfs":{"total":95995035648,"available":95966236672,"internally_reserved":0,"allocated":1126400,"data_stored":1127575,"data_compressed":62821,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953536,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32212,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712940,"statfs":{"total":95995035648,"available":95962050560,"internally_reserved":0,"allocated":1052672,"data_stored":534694,"data_compressed":26500,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182469,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28056,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717096,"statfs":{"total":95995035648,"available":95966306304,"internally_reserved":0,"allocated":1056768,"data_stored":530328,"data_compressed":26500,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411402,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28120,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717032,"statfs":{"total":95995035648,"available":95966240768,"internally_reserved":0,"allocated":1122304,"data_stored":584676,"data_compressed":26500,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574928,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32252,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712900,"statfs":{"total":95995035648,"available":95962009600,"internally_reserved":0,"allocated":1093632,"data_stored":575388,"data_compressed":26500,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738452,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36388,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708764,"statfs":{"total":95995035648,"available":95957774336,"internally_reserved":0,"allocated":1069056,"data_stored":532076,"data_compressed":26500,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34674,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34674,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34674,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36321,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36321,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36321,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36321,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36321,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":54558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":54778,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3213,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1115,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":54149,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T09:36:34.149 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-10T09:36:34.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:36:33 smithi159 ceph-mon[35513]: from='client.24772 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T09:36:34.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:36:33 smithi102 ceph-mon[35442]: from='client.24772 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T09:36:34.528 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:36:35.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:35.072+0000 7ff9ed000640 1 -- 172.21.15.5:0/1140680316 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff9e8074590 msgr2=0x7ff9e8074970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:36:35.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:35.072+0000 7ff9ed000640 1 --2- 172.21.15.5:0/1140680316 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff9e8074590 0x7ff9e8074970 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7ff9d8009900 tx=0x7ff9d802a480 comp rx=0 tx=0).stop 2024-09-10T09:36:35.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:35.072+0000 7ff9ed000640 1 -- 172.21.15.5:0/1140680316 shutdown_connections 2024-09-10T09:36:35.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:35.072+0000 7ff9ed000640 1 --2- 172.21.15.5:0/1140680316 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff9e8112b90 0x7ff9e8114f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:36:35.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:35.072+0000 7ff9ed000640 1 --2- 172.21.15.5:0/1140680316 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff9e8074eb0 0x7ff9e8112650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:36:35.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:35.072+0000 7ff9ed000640 1 --2- 172.21.15.5:0/1140680316 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff9e8074590 0x7ff9e8074970 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:36:35.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:35.072+0000 7ff9ed000640 1 -- 172.21.15.5:0/1140680316 >> 172.21.15.5:0/1140680316 conn(0x7ff9e80fd8c0 msgr2=0x7ff9e80ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:36:35.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:35.072+0000 7ff9ed000640 1 -- 172.21.15.5:0/1140680316 shutdown_connections 2024-09-10T09:36:35.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:35.072+0000 7ff9ed000640 1 -- 172.21.15.5:0/1140680316 wait complete. 2024-09-10T09:36:35.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:35.073+0000 7ff9ed000640 1 Processor -- start 2024-09-10T09:36:35.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:35.073+0000 7ff9ed000640 1 -- start start 2024-09-10T09:36:35.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:35.074+0000 7ff9ed000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff9e8074590 0x7ff9e81a4710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:36:35.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:35.074+0000 7ff9ed000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff9e8074eb0 0x7ff9e81a4c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:36:35.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:35.074+0000 7ff9ed000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff9e8112b90 0x7ff9e819e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:36:35.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:35.074+0000 7ff9ed000640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff9e81185a0 con 0x7ff9e8074590 2024-09-10T09:36:35.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:35.074+0000 7ff9ed000640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7ff9e8118420 con 0x7ff9e8112b90 2024-09-10T09:36:35.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:35.074+0000 7ff9ed000640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7ff9e8118720 con 0x7ff9e8074eb0 2024-09-10T09:36:35.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:35.074+0000 7ff9e5e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff9e8074590 0x7ff9e81a4710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:36:35.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:35.074+0000 7ff9e5e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff9e8074590 0x7ff9e81a4710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46628/0 (socket says 172.21.15.5:46628) 2024-09-10T09:36:35.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:35.074+0000 7ff9e5e00640 1 -- 172.21.15.5:0/914605453 learned_addr learned my addr 172.21.15.5:0/914605453 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:36:35.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:35.074+0000 7ff9dd400640 1 --2- 172.21.15.5:0/914605453 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff9e8074eb0 0x7ff9e81a4c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:36:35.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:35.074+0000 7ff9e6800640 1 --2- 172.21.15.5:0/914605453 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff9e8112b90 0x7ff9e819e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:36:35.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:35.074+0000 7ff9e5e00640 1 -- 172.21.15.5:0/914605453 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff9e8074eb0 msgr2=0x7ff9e81a4c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:36:35.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:35.074+0000 7ff9e5e00640 1 --2- 172.21.15.5:0/914605453 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff9e8074eb0 0x7ff9e81a4c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:36:35.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:35.074+0000 7ff9e5e00640 1 -- 172.21.15.5:0/914605453 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff9e8112b90 msgr2=0x7ff9e819e8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:36:35.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:35.074+0000 7ff9e5e00640 1 --2- 172.21.15.5:0/914605453 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff9e8112b90 0x7ff9e819e8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:36:35.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:35.075+0000 7ff9e5e00640 1 -- 172.21.15.5:0/914605453 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9e819f000 con 0x7ff9e8074590 2024-09-10T09:36:35.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:35.075+0000 7ff9e5e00640 1 --2- 172.21.15.5:0/914605453 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff9e8074590 0x7ff9e81a4710 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7ff9d8004770 tx=0x7ff9d8003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:36:35.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:35.075+0000 7ff9df400640 1 -- 172.21.15.5:0/914605453 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9d803f070 con 0x7ff9e8074590 2024-09-10T09:36:35.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:35.075+0000 7ff9df400640 1 -- 172.21.15.5:0/914605453 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff9d8031ca0 con 0x7ff9e8074590 2024-09-10T09:36:35.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:35.075+0000 7ff9df400640 1 -- 172.21.15.5:0/914605453 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9d8035030 con 0x7ff9e8074590 2024-09-10T09:36:35.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:35.075+0000 7ff9ed000640 1 -- 172.21.15.5:0/914605453 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9e819f290 con 0x7ff9e8074590 2024-09-10T09:36:35.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:35.075+0000 7ff9ed000640 1 -- 172.21.15.5:0/914605453 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9e81ab5e0 con 0x7ff9e8074590 2024-09-10T09:36:35.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:35.077+0000 7ff9ed000640 1 -- 172.21.15.5:0/914605453 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff9e8075cd0 con 0x7ff9e8074590 2024-09-10T09:36:35.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:35.078+0000 7ff9df400640 1 -- 172.21.15.5:0/914605453 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7ff9d8042070 con 0x7ff9e8074590 2024-09-10T09:36:35.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:35.080+0000 7ff9df400640 1 --2- 172.21.15.5:0/914605453 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7ff9cc0a2c90 0x7ff9cc0a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:36:35.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:35.080+0000 7ff9df400640 1 -- 172.21.15.5:0/914605453 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9071+0+0 (secure 0 0 0) 0x7ff9d80efd60 con 0x7ff9e8074590 2024-09-10T09:36:35.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:35.080+0000 7ff9dd400640 1 --2- 172.21.15.5:0/914605453 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7ff9cc0a2c90 0x7ff9cc0a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:36:35.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:35.081+0000 7ff9dd400640 1 --2- 172.21.15.5:0/914605453 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7ff9cc0a2c90 0x7ff9cc0a5150 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7ff9e80ffec0 tx=0x7ff9d000b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:36:35.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:35.082+0000 7ff9df400640 1 -- 172.21.15.5:0/914605453 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff9d80bc280 con 0x7ff9e8074590 2024-09-10T09:36:35.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:35.266+0000 7ff9ed000640 1 -- 172.21.15.5:0/914605453 --> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ff9e8062630 con 0x7ff9cc0a2c90 2024-09-10T09:36:35.267 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:36:34 smithi005 ceph-mon[30350]: pgmap v338: 129 pgs: 129 active+clean; 635 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 4.3 KiB/s wr, 10 op/s 2024-09-10T09:36:35.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:35.275+0000 7ff9df400640 1 -- 172.21.15.5:0/914605453 <== mgr.14150 v2:172.21.15.5:6800/2707348081 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340682 (secure 0 0 0) 0x7ff9e8062630 con 0x7ff9cc0a2c90 2024-09-10T09:36:35.282 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-10T09:36:35.284 INFO:teuthology.orchestra.run.smithi005.stderr:dumped all 2024-09-10T09:36:35.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:35.279+0000 7ff9ed000640 1 -- 172.21.15.5:0/914605453 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7ff9cc0a2c90 msgr2=0x7ff9cc0a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:36:35.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:35.279+0000 7ff9ed000640 1 --2- 172.21.15.5:0/914605453 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7ff9cc0a2c90 0x7ff9cc0a5150 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7ff9e80ffec0 tx=0x7ff9d000b410 comp rx=0 tx=0).stop 2024-09-10T09:36:35.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:35.279+0000 7ff9ed000640 1 -- 172.21.15.5:0/914605453 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff9e8074590 msgr2=0x7ff9e81a4710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:36:35.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:35.279+0000 7ff9ed000640 1 --2- 172.21.15.5:0/914605453 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff9e8074590 0x7ff9e81a4710 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7ff9d8004770 tx=0x7ff9d8003c10 comp rx=0 tx=0).stop 2024-09-10T09:36:35.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:35.280+0000 7ff9ed000640 1 -- 172.21.15.5:0/914605453 shutdown_connections 2024-09-10T09:36:35.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:35.280+0000 7ff9ed000640 1 --2- 172.21.15.5:0/914605453 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7ff9cc0a2c90 0x7ff9cc0a5150 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:36:35.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:35.280+0000 7ff9ed000640 1 --2- 172.21.15.5:0/914605453 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff9e8112b90 0x7ff9e819e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:36:35.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:35.280+0000 7ff9ed000640 1 --2- 172.21.15.5:0/914605453 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff9e8074eb0 0x7ff9e81a4c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:36:35.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:35.280+0000 7ff9ed000640 1 --2- 172.21.15.5:0/914605453 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff9e8074590 0x7ff9e81a4710 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:36:35.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:35.280+0000 7ff9ed000640 1 -- 172.21.15.5:0/914605453 >> 172.21.15.5:0/914605453 conn(0x7ff9e80fd8c0 msgr2=0x7ff9e80ff010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:36:35.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:35.280+0000 7ff9ed000640 1 -- 172.21.15.5:0/914605453 shutdown_connections 2024-09-10T09:36:35.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:35.280+0000 7ff9ed000640 1 -- 172.21.15.5:0/914605453 wait complete. 2024-09-10T09:36:35.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:36:34 smithi159 ceph-mon[35513]: pgmap v338: 129 pgs: 129 active+clean; 635 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 4.3 KiB/s wr, 10 op/s 2024-09-10T09:36:35.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:36:34 smithi102 ceph-mon[35442]: pgmap v338: 129 pgs: 129 active+clean; 635 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 4.3 KiB/s wr, 10 op/s 2024-09-10T09:36:36.104 INFO:teuthology.orchestra.run.smithi005.stdout:{"pg_ready":true,"pg_map":{"version":338,"stamp":"2024-09-10T09:36:33.542281+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":650223,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":322,"num_read_kb":273,"num_write":424,"num_write_kb":3069,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":347,"ondisk_log_size":347,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":374480,"kb_used_data":12752,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124567344,"statfs":{"total":1151940427776,"available":1151556960256,"internally_reserved":0,"allocated":13058048,"data_stored":8341511,"data_compressed":426963,"data_compressed_allocated":577536,"data_compressed_original":2654208,"omap_allocated":19141,"internal_metadata":370193723},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":42201,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":104,"num_read_kb":92,"num_write":26,"num_write_kb":52,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003013"},"pg_stats":[{"pgid":"2.1f","version":"94'13","reported_seq":68,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:36:28.519655+0000","last_change":"2024-09-10T09:35:25.533085+0000","last_active":"2024-09-10T09:36:28.519655+0000","last_peered":"2024-09-10T09:36:28.519655+0000","last_clean":"2024-09-10T09:36:28.519655+0000","last_became_active":"2024-09-10T09:35:25.532729+0000","last_became_peered":"2024-09-10T09:35:25.532729+0000","last_unstale":"2024-09-10T09:36:28.519655+0000","last_undegraded":"2024-09-10T09:36:28.519655+0000","last_fullsized":"2024-09-10T09:36:28.519655+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:55:11.158279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":16,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":11,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:36:06.768036+0000","last_change":"2024-09-10T09:36:06.767356+0000","last_active":"2024-09-10T09:36:06.768036+0000","last_peered":"2024-09-10T09:36:06.768036+0000","last_clean":"2024-09-10T09:36:06.768036+0000","last_became_active":"2024-09-10T09:36:06.766931+0000","last_became_peered":"2024-09-10T09:36:06.766931+0000","last_unstale":"2024-09-10T09:36:06.768036+0000","last_undegraded":"2024-09-10T09:36:06.768036+0000","last_fullsized":"2024-09-10T09:36:06.768036+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:25:02.777327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,7],"acting":[9,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:36:06.760145+0000","last_change":"2024-09-10T09:35:36.629797+0000","last_active":"2024-09-10T09:36:06.760145+0000","last_peered":"2024-09-10T09:36:06.760145+0000","last_clean":"2024-09-10T09:36:06.760145+0000","last_became_active":"2024-09-10T09:35:36.629414+0000","last_became_peered":"2024-09-10T09:35:36.629414+0000","last_unstale":"2024-09-10T09:36:06.760145+0000","last_undegraded":"2024-09-10T09:36:06.760145+0000","last_fullsized":"2024-09-10T09:36:06.760145+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:42:44.610581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"94'2","reported_seq":42,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:36:06.764083+0000","last_change":"2024-09-10T09:35:25.852022+0000","last_active":"2024-09-10T09:36:06.764083+0000","last_peered":"2024-09-10T09:36:06.764083+0000","last_clean":"2024-09-10T09:36:06.764083+0000","last_became_active":"2024-09-10T09:35:25.851587+0000","last_became_peered":"2024-09-10T09:35:25.851587+0000","last_unstale":"2024-09-10T09:36:06.764083+0000","last_undegraded":"2024-09-10T09:36:06.764083+0000","last_fullsized":"2024-09-10T09:36:06.764083+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:37:26.085914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:36:06.760232+0000","last_change":"2024-09-10T09:35:28.579260+0000","last_active":"2024-09-10T09:36:06.760232+0000","last_peered":"2024-09-10T09:36:06.760232+0000","last_clean":"2024-09-10T09:36:06.760232+0000","last_became_active":"2024-09-10T09:35:28.574584+0000","last_became_peered":"2024-09-10T09:35:28.574584+0000","last_unstale":"2024-09-10T09:36:06.760232+0000","last_undegraded":"2024-09-10T09:36:06.760232+0000","last_fullsized":"2024-09-10T09:36:06.760232+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:31:19.277689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":31,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:36:06.874861+0000","last_change":"2024-09-10T09:36:06.874861+0000","last_active":"2024-09-10T09:36:06.874861+0000","last_peered":"2024-09-10T09:36:06.874861+0000","last_clean":"2024-09-10T09:36:06.874861+0000","last_became_active":"2024-09-10T09:36:06.873930+0000","last_became_peered":"2024-09-10T09:36:06.873930+0000","last_unstale":"2024-09-10T09:36:06.874861+0000","last_undegraded":"2024-09-10T09:36:06.874861+0000","last_fullsized":"2024-09-10T09:36:06.874861+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:35:01.354061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,8,3],"acting":[4,6,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"94'12","reported_seq":52,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:36:14.498394+0000","last_change":"2024-09-10T09:35:25.534571+0000","last_active":"2024-09-10T09:36:14.498394+0000","last_peered":"2024-09-10T09:36:14.498394+0000","last_clean":"2024-09-10T09:36:14.498394+0000","last_became_active":"2024-09-10T09:35:25.534235+0000","last_became_peered":"2024-09-10T09:35:25.534235+0000","last_unstale":"2024-09-10T09:36:14.498394+0000","last_undegraded":"2024-09-10T09:36:14.498394+0000","last_fullsized":"2024-09-10T09:36:14.498394+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:50:21.447542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:36:06.765451+0000","last_change":"2024-09-10T09:35:28.565541+0000","last_active":"2024-09-10T09:36:06.765451+0000","last_peered":"2024-09-10T09:36:06.765451+0000","last_clean":"2024-09-10T09:36:06.765451+0000","last_became_active":"2024-09-10T09:35:28.565095+0000","last_became_peered":"2024-09-10T09:35:28.565095+0000","last_unstale":"2024-09-10T09:36:06.765451+0000","last_undegraded":"2024-09-10T09:36:06.765451+0000","last_fullsized":"2024-09-10T09:36:06.765451+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:29:31.285401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:36:06.765379+0000","last_change":"2024-09-10T09:35:36.629277+0000","last_active":"2024-09-10T09:36:06.765379+0000","last_peered":"2024-09-10T09:36:06.765379+0000","last_clean":"2024-09-10T09:36:06.765379+0000","last_became_active":"2024-09-10T09:35:36.628687+0000","last_became_peered":"2024-09-10T09:35:36.628687+0000","last_unstale":"2024-09-10T09:36:06.765379+0000","last_undegraded":"2024-09-10T09:36:06.765379+0000","last_fullsized":"2024-09-10T09:36:06.765379+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:34:33.628827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"94'5","reported_seq":45,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:36:14.497342+0000","last_change":"2024-09-10T09:35:25.536137+0000","last_active":"2024-09-10T09:36:14.497342+0000","last_peered":"2024-09-10T09:36:14.497342+0000","last_clean":"2024-09-10T09:36:14.497342+0000","last_became_active":"2024-09-10T09:35:25.535728+0000","last_became_peered":"2024-09-10T09:35:25.535728+0000","last_unstale":"2024-09-10T09:36:14.497342+0000","last_undegraded":"2024-09-10T09:36:14.497342+0000","last_fullsized":"2024-09-10T09:36:14.497342+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:36:04.829901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:36:06.874425+0000","last_change":"2024-09-10T09:36:06.874425+0000","last_active":"2024-09-10T09:36:06.874425+0000","last_peered":"2024-09-10T09:36:06.874425+0000","last_clean":"2024-09-10T09:36:06.874425+0000","last_became_active":"2024-09-10T09:36:06.874050+0000","last_became_peered":"2024-09-10T09:36:06.874050+0000","last_unstale":"2024-09-10T09:36:06.874425+0000","last_undegraded":"2024-09-10T09:36:06.874425+0000","last_fullsized":"2024-09-10T09:36:06.874425+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:30:51.200335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,1],"acting":[5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:36:06.765164+0000","last_change":"2024-09-10T09:35:36.615314+0000","last_active":"2024-09-10T09:36:06.765164+0000","last_peered":"2024-09-10T09:36:06.765164+0000","last_clean":"2024-09-10T09:36:06.765164+0000","last_became_active":"2024-09-10T09:35:36.614820+0000","last_became_peered":"2024-09-10T09:35:36.614820+0000","last_unstale":"2024-09-10T09:36:06.765164+0000","last_undegraded":"2024-09-10T09:36:06.765164+0000","last_fullsized":"2024-09-10T09:36:06.765164+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:51:16.897935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:36:06.764124+0000","last_change":"2024-09-10T09:35:28.572346+0000","last_active":"2024-09-10T09:36:06.764124+0000","last_peered":"2024-09-10T09:36:06.764124+0000","last_clean":"2024-09-10T09:36:06.764124+0000","last_became_active":"2024-09-10T09:35:28.571839+0000","last_became_peered":"2024-09-10T09:35:28.571839+0000","last_unstale":"2024-09-10T09:36:06.764124+0000","last_undegraded":"2024-09-10T09:36:06.764124+0000","last_fullsized":"2024-09-10T09:36:06.764124+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:41:29.662441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"94'4","reported_seq":44,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:36:11.804692+0000","last_change":"2024-09-10T09:35:25.538747+0000","last_active":"2024-09-10T09:36:11.804692+0000","last_peered":"2024-09-10T09:36:11.804692+0000","last_clean":"2024-09-10T09:36:11.804692+0000","last_became_active":"2024-09-10T09:35:25.538330+0000","last_became_peered":"2024-09-10T09:35:25.538330+0000","last_unstale":"2024-09-10T09:36:11.804692+0000","last_undegraded":"2024-09-10T09:36:11.804692+0000","last_fullsized":"2024-09-10T09:36:11.804692+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:15:08.340688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:36:06.764134+0000","last_change":"2024-09-10T09:35:37.270106+0000","last_active":"2024-09-10T09:36:06.764134+0000","last_peered":"2024-09-10T09:36:06.764134+0000","last_clean":"2024-09-10T09:36:06.764134+0000","last_became_active":"2024-09-10T09:35:37.269575+0000","last_became_peered":"2024-09-10T09:35:37.269575+0000","last_unstale":"2024-09-10T09:36:06.764134+0000","last_undegraded":"2024-09-10T09:36:06.764134+0000","last_fullsized":"2024-09-10T09:36:06.764134+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:35:00.414602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:36:06.763337+0000","last_change":"2024-09-10T09:35:28.569230+0000","last_active":"2024-09-10T09:36:06.763337+0000","last_peered":"2024-09-10T09:36:06.763337+0000","last_clean":"2024-09-10T09:36:06.763337+0000","last_became_active":"2024-09-10T09:35:28.568266+0000","last_became_peered":"2024-09-10T09:35:28.568266+0000","last_unstale":"2024-09-10T09:36:06.763337+0000","last_undegraded":"2024-09-10T09:36:06.763337+0000","last_fullsized":"2024-09-10T09:36:06.763337+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:49:43.730686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"94'3","reported_seq":42,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:36:13.817585+0000","last_change":"2024-09-10T09:35:25.534856+0000","last_active":"2024-09-10T09:36:13.817585+0000","last_peered":"2024-09-10T09:36:13.817585+0000","last_clean":"2024-09-10T09:36:13.817585+0000","last_became_active":"2024-09-10T09:35:25.534471+0000","last_became_peered":"2024-09-10T09:35:25.534471+0000","last_unstale":"2024-09-10T09:36:13.817585+0000","last_undegraded":"2024-09-10T09:36:13.817585+0000","last_fullsized":"2024-09-10T09:36:13.817585+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:34:51.490556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"94'1","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:36:06.875782+0000","last_change":"2024-09-10T09:36:06.875056+0000","last_active":"2024-09-10T09:36:06.875782+0000","last_peered":"2024-09-10T09:36:06.875782+0000","last_clean":"2024-09-10T09:36:06.875782+0000","last_became_active":"2024-09-10T09:36:06.874400+0000","last_became_peered":"2024-09-10T09:36:06.874400+0000","last_unstale":"2024-09-10T09:36:06.875782+0000","last_undegraded":"2024-09-10T09:36:06.875782+0000","last_fullsized":"2024-09-10T09:36:06.875782+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:21:06.300712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,7,8],"acting":[4,10,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:36:06.762615+0000","last_change":"2024-09-10T09:35:28.571319+0000","last_active":"2024-09-10T09:36:06.762615+0000","last_peered":"2024-09-10T09:36:06.762615+0000","last_clean":"2024-09-10T09:36:06.762615+0000","last_became_active":"2024-09-10T09:35:28.570972+0000","last_became_peered":"2024-09-10T09:35:28.570972+0000","last_unstale":"2024-09-10T09:36:06.762615+0000","last_undegraded":"2024-09-10T09:36:06.762615+0000","last_fullsized":"2024-09-10T09:36:06.762615+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:45:19.300535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"94'7","reported_seq":47,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:36:14.497731+0000","last_change":"2024-09-10T09:35:25.530489+0000","last_active":"2024-09-10T09:36:14.497731+0000","last_peered":"2024-09-10T09:36:14.497731+0000","last_clean":"2024-09-10T09:36:14.497731+0000","last_became_active":"2024-09-10T09:35:25.530197+0000","last_became_peered":"2024-09-10T09:35:25.530197+0000","last_unstale":"2024-09-10T09:36:14.497731+0000","last_undegraded":"2024-09-10T09:36:14.497731+0000","last_fullsized":"2024-09-10T09:36:14.497731+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:28:57.539569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:36:06.761452+0000","last_change":"2024-09-10T09:35:36.630507+0000","last_active":"2024-09-10T09:36:06.761452+0000","last_peered":"2024-09-10T09:36:06.761452+0000","last_clean":"2024-09-10T09:36:06.761452+0000","last_became_active":"2024-09-10T09:35:36.630052+0000","last_became_peered":"2024-09-10T09:35:36.630052+0000","last_unstale":"2024-09-10T09:36:06.761452+0000","last_undegraded":"2024-09-10T09:36:06.761452+0000","last_fullsized":"2024-09-10T09:36:06.761452+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:13:42.691382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:36:06.766516+0000","last_change":"2024-09-10T09:35:28.565650+0000","last_active":"2024-09-10T09:36:06.766516+0000","last_peered":"2024-09-10T09:36:06.766516+0000","last_clean":"2024-09-10T09:36:06.766516+0000","last_became_active":"2024-09-10T09:35:28.565303+0000","last_became_peered":"2024-09-10T09:35:28.565303+0000","last_unstale":"2024-09-10T09:36:06.766516+0000","last_undegraded":"2024-09-10T09:36:06.766516+0000","last_fullsized":"2024-09-10T09:36:06.766516+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:48:08.965487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"94'2","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:36:11.806672+0000","last_change":"2024-09-10T09:35:25.851326+0000","last_active":"2024-09-10T09:36:11.806672+0000","last_peered":"2024-09-10T09:36:11.806672+0000","last_clean":"2024-09-10T09:36:11.806672+0000","last_became_active":"2024-09-10T09:35:25.850981+0000","last_became_peered":"2024-09-10T09:35:25.850981+0000","last_unstale":"2024-09-10T09:36:11.806672+0000","last_undegraded":"2024-09-10T09:36:11.806672+0000","last_fullsized":"2024-09-10T09:36:11.806672+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:50:28.877840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:36:06.765389+0000","last_change":"2024-09-10T09:35:37.270138+0000","last_active":"2024-09-10T09:36:06.765389+0000","last_peered":"2024-09-10T09:36:06.765389+0000","last_clean":"2024-09-10T09:36:06.765389+0000","last_became_active":"2024-09-10T09:35:37.269663+0000","last_became_peered":"2024-09-10T09:35:37.269663+0000","last_unstale":"2024-09-10T09:36:06.765389+0000","last_undegraded":"2024-09-10T09:36:06.765389+0000","last_fullsized":"2024-09-10T09:36:06.765389+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:18:00.105970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:36:06.759993+0000","last_change":"2024-09-10T09:35:28.576197+0000","last_active":"2024-09-10T09:36:06.759993+0000","last_peered":"2024-09-10T09:36:06.759993+0000","last_clean":"2024-09-10T09:36:06.759993+0000","last_became_active":"2024-09-10T09:35:28.575859+0000","last_became_peered":"2024-09-10T09:35:28.575859+0000","last_unstale":"2024-09-10T09:36:06.759993+0000","last_undegraded":"2024-09-10T09:36:06.759993+0000","last_fullsized":"2024-09-10T09:36:06.759993+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:09:39.458761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"94'7","reported_seq":46,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:36:14.498489+0000","last_change":"2024-09-10T09:35:25.990587+0000","last_active":"2024-09-10T09:36:14.498489+0000","last_peered":"2024-09-10T09:36:14.498489+0000","last_clean":"2024-09-10T09:36:14.498489+0000","last_became_active":"2024-09-10T09:35:25.990302+0000","last_became_peered":"2024-09-10T09:35:25.990302+0000","last_unstale":"2024-09-10T09:36:14.498489+0000","last_undegraded":"2024-09-10T09:36:14.498489+0000","last_fullsized":"2024-09-10T09:36:14.498489+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:18:41.244441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:36:06.760684+0000","last_change":"2024-09-10T09:35:36.612983+0000","last_active":"2024-09-10T09:36:06.760684+0000","last_peered":"2024-09-10T09:36:06.760684+0000","last_clean":"2024-09-10T09:36:06.760684+0000","last_became_active":"2024-09-10T09:35:36.612542+0000","last_became_peered":"2024-09-10T09:35:36.612542+0000","last_unstale":"2024-09-10T09:36:06.760684+0000","last_undegraded":"2024-09-10T09:36:06.760684+0000","last_fullsized":"2024-09-10T09:36:06.760684+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:50:16.298195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:36:06.763996+0000","last_change":"2024-09-10T09:35:28.564121+0000","last_active":"2024-09-10T09:36:06.763996+0000","last_peered":"2024-09-10T09:36:06.763996+0000","last_clean":"2024-09-10T09:36:06.763996+0000","last_became_active":"2024-09-10T09:35:28.562780+0000","last_became_peered":"2024-09-10T09:35:28.562780+0000","last_unstale":"2024-09-10T09:36:06.763996+0000","last_undegraded":"2024-09-10T09:36:06.763996+0000","last_fullsized":"2024-09-10T09:36:06.763996+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:53:15.690478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"94'13","reported_seq":52,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:36:27.579197+0000","last_change":"2024-09-10T09:35:25.532740+0000","last_active":"2024-09-10T09:36:27.579197+0000","last_peered":"2024-09-10T09:36:27.579197+0000","last_clean":"2024-09-10T09:36:27.579197+0000","last_became_active":"2024-09-10T09:35:25.532282+0000","last_became_peered":"2024-09-10T09:35:25.532282+0000","last_unstale":"2024-09-10T09:36:27.579197+0000","last_undegraded":"2024-09-10T09:36:27.579197+0000","last_fullsized":"2024-09-10T09:36:27.579197+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:52:17.323590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:36:06.762957+0000","last_change":"2024-09-10T09:35:37.270888+0000","last_active":"2024-09-10T09:36:06.762957+0000","last_peered":"2024-09-10T09:36:06.762957+0000","last_clean":"2024-09-10T09:36:06.762957+0000","last_became_active":"2024-09-10T09:35:37.270229+0000","last_became_peered":"2024-09-10T09:35:37.270229+0000","last_unstale":"2024-09-10T09:36:06.762957+0000","last_undegraded":"2024-09-10T09:36:06.762957+0000","last_fullsized":"2024-09-10T09:36:06.762957+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:03:54.223463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:36:06.873437+0000","last_change":"2024-09-10T09:35:28.572799+0000","last_active":"2024-09-10T09:36:06.873437+0000","last_peered":"2024-09-10T09:36:06.873437+0000","last_clean":"2024-09-10T09:36:06.873437+0000","last_became_active":"2024-09-10T09:35:28.572310+0000","last_became_peered":"2024-09-10T09:35:28.572310+0000","last_unstale":"2024-09-10T09:36:06.873437+0000","last_undegraded":"2024-09-10T09:36:06.873437+0000","last_fullsized":"2024-09-10T09:36:06.873437+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:33:51.446766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"94'4","reported_seq":44,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:36:14.485932+0000","last_change":"2024-09-10T09:35:25.851868+0000","last_active":"2024-09-10T09:36:14.485932+0000","last_peered":"2024-09-10T09:36:14.485932+0000","last_clean":"2024-09-10T09:36:14.485932+0000","last_became_active":"2024-09-10T09:35:25.851447+0000","last_became_peered":"2024-09-10T09:35:25.851447+0000","last_unstale":"2024-09-10T09:36:14.485932+0000","last_undegraded":"2024-09-10T09:36:14.485932+0000","last_fullsized":"2024-09-10T09:36:14.485932+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:15:57.706690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:36:06.761923+0000","last_change":"2024-09-10T09:35:36.619459+0000","last_active":"2024-09-10T09:36:06.761923+0000","last_peered":"2024-09-10T09:36:06.761923+0000","last_clean":"2024-09-10T09:36:06.761923+0000","last_became_active":"2024-09-10T09:35:36.619023+0000","last_became_peered":"2024-09-10T09:35:36.619023+0000","last_unstale":"2024-09-10T09:36:06.761923+0000","last_undegraded":"2024-09-10T09:36:06.761923+0000","last_fullsized":"2024-09-10T09:36:06.761923+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:07:32.999231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:36:06.873350+0000","last_change":"2024-09-10T09:35:28.575668+0000","last_active":"2024-09-10T09:36:06.873350+0000","last_peered":"2024-09-10T09:36:06.873350+0000","last_clean":"2024-09-10T09:36:06.873350+0000","last_became_active":"2024-09-10T09:35:28.575425+0000","last_became_peered":"2024-09-10T09:35:28.575425+0000","last_unstale":"2024-09-10T09:36:06.873350+0000","last_undegraded":"2024-09-10T09:36:06.873350+0000","last_fullsized":"2024-09-10T09:36:06.873350+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:13:27.120208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"94'1","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:36:06.761098+0000","last_change":"2024-09-10T09:35:25.854569+0000","last_active":"2024-09-10T09:36:06.761098+0000","last_peered":"2024-09-10T09:36:06.761098+0000","last_clean":"2024-09-10T09:36:06.761098+0000","last_became_active":"2024-09-10T09:35:25.854152+0000","last_became_peered":"2024-09-10T09:35:25.854152+0000","last_unstale":"2024-09-10T09:36:06.761098+0000","last_undegraded":"2024-09-10T09:36:06.761098+0000","last_fullsized":"2024-09-10T09:36:06.761098+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:27:15.584707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:36:06.766506+0000","last_change":"2024-09-10T09:35:37.270517+0000","last_active":"2024-09-10T09:36:06.766506+0000","last_peered":"2024-09-10T09:36:06.766506+0000","last_clean":"2024-09-10T09:36:06.766506+0000","last_became_active":"2024-09-10T09:35:37.269793+0000","last_became_peered":"2024-09-10T09:35:37.269793+0000","last_unstale":"2024-09-10T09:36:06.766506+0000","last_undegraded":"2024-09-10T09:36:06.766506+0000","last_fullsized":"2024-09-10T09:36:06.766506+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:28:58.272848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:36:06.873121+0000","last_change":"2024-09-10T09:35:28.572056+0000","last_active":"2024-09-10T09:36:06.873121+0000","last_peered":"2024-09-10T09:36:06.873121+0000","last_clean":"2024-09-10T09:36:06.873121+0000","last_became_active":"2024-09-10T09:35:28.571755+0000","last_became_peered":"2024-09-10T09:35:28.571755+0000","last_unstale":"2024-09-10T09:36:06.873121+0000","last_undegraded":"2024-09-10T09:36:06.873121+0000","last_fullsized":"2024-09-10T09:36:06.873121+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:45:02.739512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"94'6","reported_seq":61,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:36:28.520322+0000","last_change":"2024-09-10T09:35:25.531062+0000","last_active":"2024-09-10T09:36:28.520322+0000","last_peered":"2024-09-10T09:36:28.520322+0000","last_clean":"2024-09-10T09:36:28.520322+0000","last_became_active":"2024-09-10T09:35:25.530005+0000","last_became_peered":"2024-09-10T09:35:25.530005+0000","last_unstale":"2024-09-10T09:36:28.520322+0000","last_undegraded":"2024-09-10T09:36:28.520322+0000","last_fullsized":"2024-09-10T09:36:28.520322+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:42:35.330312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":981,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":17,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:36:06.873232+0000","last_change":"2024-09-10T09:35:36.623099+0000","last_active":"2024-09-10T09:36:06.873232+0000","last_peered":"2024-09-10T09:36:06.873232+0000","last_clean":"2024-09-10T09:36:06.873232+0000","last_became_active":"2024-09-10T09:35:36.622716+0000","last_became_peered":"2024-09-10T09:35:36.622716+0000","last_unstale":"2024-09-10T09:36:06.873232+0000","last_undegraded":"2024-09-10T09:36:06.873232+0000","last_fullsized":"2024-09-10T09:36:06.873232+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:10:38.495164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:36:06.769405+0000","last_change":"2024-09-10T09:36:06.769405+0000","last_active":"2024-09-10T09:36:06.769405+0000","last_peered":"2024-09-10T09:36:06.769405+0000","last_clean":"2024-09-10T09:36:06.769405+0000","last_became_active":"2024-09-10T09:36:06.769042+0000","last_became_peered":"2024-09-10T09:36:06.769042+0000","last_unstale":"2024-09-10T09:36:06.769405+0000","last_undegraded":"2024-09-10T09:36:06.769405+0000","last_fullsized":"2024-09-10T09:36:06.769405+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:01:49.340765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,8],"acting":[5,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"94'5","reported_seq":60,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:36:27.508041+0000","last_change":"2024-09-10T09:35:25.851430+0000","last_active":"2024-09-10T09:36:27.508041+0000","last_peered":"2024-09-10T09:36:27.508041+0000","last_clean":"2024-09-10T09:36:27.508041+0000","last_became_active":"2024-09-10T09:35:25.851067+0000","last_became_peered":"2024-09-10T09:35:25.851067+0000","last_unstale":"2024-09-10T09:36:27.508041+0000","last_undegraded":"2024-09-10T09:36:27.508041+0000","last_fullsized":"2024-09-10T09:36:27.508041+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:55:58.756207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":16,"num_read_kb":16,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:36:06.873609+0000","last_change":"2024-09-10T09:35:37.270361+0000","last_active":"2024-09-10T09:36:06.873609+0000","last_peered":"2024-09-10T09:36:06.873609+0000","last_clean":"2024-09-10T09:36:06.873609+0000","last_became_active":"2024-09-10T09:35:37.269632+0000","last_became_peered":"2024-09-10T09:35:37.269632+0000","last_unstale":"2024-09-10T09:36:06.873609+0000","last_undegraded":"2024-09-10T09:36:06.873609+0000","last_fullsized":"2024-09-10T09:36:06.873609+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:50:34.360497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:36:06.872892+0000","last_change":"2024-09-10T09:35:28.571064+0000","last_active":"2024-09-10T09:36:06.872892+0000","last_peered":"2024-09-10T09:36:06.872892+0000","last_clean":"2024-09-10T09:36:06.872892+0000","last_became_active":"2024-09-10T09:35:28.570691+0000","last_became_peered":"2024-09-10T09:35:28.570691+0000","last_unstale":"2024-09-10T09:36:06.872892+0000","last_undegraded":"2024-09-10T09:36:06.872892+0000","last_fullsized":"2024-09-10T09:36:06.872892+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:58:33.162501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"94'4","reported_seq":42,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:36:12.811401+0000","last_change":"2024-09-10T09:35:25.533950+0000","last_active":"2024-09-10T09:36:12.811401+0000","last_peered":"2024-09-10T09:36:12.811401+0000","last_clean":"2024-09-10T09:36:12.811401+0000","last_became_active":"2024-09-10T09:35:25.533612+0000","last_became_peered":"2024-09-10T09:35:25.533612+0000","last_unstale":"2024-09-10T09:36:12.811401+0000","last_undegraded":"2024-09-10T09:36:12.811401+0000","last_fullsized":"2024-09-10T09:36:12.811401+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:13:40.221467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:36:06.763637+0000","last_change":"2024-09-10T09:35:37.270358+0000","last_active":"2024-09-10T09:36:06.763637+0000","last_peered":"2024-09-10T09:36:06.763637+0000","last_clean":"2024-09-10T09:36:06.763637+0000","last_became_active":"2024-09-10T09:35:37.269822+0000","last_became_peered":"2024-09-10T09:35:37.269822+0000","last_unstale":"2024-09-10T09:36:06.763637+0000","last_undegraded":"2024-09-10T09:36:06.763637+0000","last_fullsized":"2024-09-10T09:36:06.763637+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:19:48.932596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:36:06.763498+0000","last_change":"2024-09-10T09:35:28.576244+0000","last_active":"2024-09-10T09:36:06.763498+0000","last_peered":"2024-09-10T09:36:06.763498+0000","last_clean":"2024-09-10T09:36:06.763498+0000","last_became_active":"2024-09-10T09:35:28.571801+0000","last_became_peered":"2024-09-10T09:35:28.571801+0000","last_unstale":"2024-09-10T09:36:06.763498+0000","last_undegraded":"2024-09-10T09:36:06.763498+0000","last_fullsized":"2024-09-10T09:36:06.763498+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:11:21.742220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"94'6","reported_seq":47,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:36:14.498654+0000","last_change":"2024-09-10T09:35:25.537118+0000","last_active":"2024-09-10T09:36:14.498654+0000","last_peered":"2024-09-10T09:36:14.498654+0000","last_clean":"2024-09-10T09:36:14.498654+0000","last_became_active":"2024-09-10T09:35:25.536718+0000","last_became_peered":"2024-09-10T09:35:25.536718+0000","last_unstale":"2024-09-10T09:36:14.498654+0000","last_undegraded":"2024-09-10T09:36:14.498654+0000","last_fullsized":"2024-09-10T09:36:14.498654+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:16:32.837476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":356,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:36:06.766849+0000","last_change":"2024-09-10T09:34:07.350825+0000","last_active":"2024-09-10T09:36:06.766849+0000","last_peered":"2024-09-10T09:36:06.766849+0000","last_clean":"2024-09-10T09:36:06.766849+0000","last_became_active":"2024-09-10T09:34:07.350325+0000","last_became_peered":"2024-09-10T09:34:07.350325+0000","last_unstale":"2024-09-10T09:36:06.766849+0000","last_undegraded":"2024-09-10T09:36:06.766849+0000","last_fullsized":"2024-09-10T09:36:06.766849+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:30:26.408262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:30:26.408262+0000","last_clean_scrub_stamp":"2024-09-10T09:30:26.408262+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:58:04.920443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"94'1","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:36:06.763785+0000","last_change":"2024-09-10T09:35:36.632941+0000","last_active":"2024-09-10T09:36:06.763785+0000","last_peered":"2024-09-10T09:36:06.763785+0000","last_clean":"2024-09-10T09:36:06.763785+0000","last_became_active":"2024-09-10T09:35:36.632527+0000","last_became_peered":"2024-09-10T09:35:36.632527+0000","last_unstale":"2024-09-10T09:36:06.763785+0000","last_undegraded":"2024-09-10T09:36:06.763785+0000","last_fullsized":"2024-09-10T09:36:06.763785+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:33:21.733772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:36:06.764063+0000","last_change":"2024-09-10T09:35:28.572010+0000","last_active":"2024-09-10T09:36:06.764063+0000","last_peered":"2024-09-10T09:36:06.764063+0000","last_clean":"2024-09-10T09:36:06.764063+0000","last_became_active":"2024-09-10T09:35:28.571623+0000","last_became_peered":"2024-09-10T09:35:28.571623+0000","last_unstale":"2024-09-10T09:36:06.764063+0000","last_undegraded":"2024-09-10T09:36:06.764063+0000","last_fullsized":"2024-09-10T09:36:06.764063+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:00:49.328663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"94'5","reported_seq":63,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:36:29.512235+0000","last_change":"2024-09-10T09:35:25.990328+0000","last_active":"2024-09-10T09:36:29.512235+0000","last_peered":"2024-09-10T09:36:29.512235+0000","last_clean":"2024-09-10T09:36:29.512235+0000","last_became_active":"2024-09-10T09:35:25.989946+0000","last_became_peered":"2024-09-10T09:35:25.989946+0000","last_unstale":"2024-09-10T09:36:29.512235+0000","last_undegraded":"2024-09-10T09:36:29.512235+0000","last_fullsized":"2024-09-10T09:36:29.512235+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:51:16.069913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":19,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:36:06.765767+0000","last_change":"2024-09-10T09:35:36.619013+0000","last_active":"2024-09-10T09:36:06.765767+0000","last_peered":"2024-09-10T09:36:06.765767+0000","last_clean":"2024-09-10T09:36:06.765767+0000","last_became_active":"2024-09-10T09:35:36.618506+0000","last_became_peered":"2024-09-10T09:35:36.618506+0000","last_unstale":"2024-09-10T09:36:06.765767+0000","last_undegraded":"2024-09-10T09:36:06.765767+0000","last_fullsized":"2024-09-10T09:36:06.765767+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:57:53.580569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:36:06.768573+0000","last_change":"2024-09-10T09:36:06.768573+0000","last_active":"2024-09-10T09:36:06.768573+0000","last_peered":"2024-09-10T09:36:06.768573+0000","last_clean":"2024-09-10T09:36:06.768573+0000","last_became_active":"2024-09-10T09:36:06.768228+0000","last_became_peered":"2024-09-10T09:36:06.768228+0000","last_unstale":"2024-09-10T09:36:06.768573+0000","last_undegraded":"2024-09-10T09:36:06.768573+0000","last_fullsized":"2024-09-10T09:36:06.768573+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:57:38.310713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"94'3","reported_seq":43,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:36:14.486020+0000","last_change":"2024-09-10T09:35:25.528795+0000","last_active":"2024-09-10T09:36:14.486020+0000","last_peered":"2024-09-10T09:36:14.486020+0000","last_clean":"2024-09-10T09:36:14.486020+0000","last_became_active":"2024-09-10T09:35:25.528475+0000","last_became_peered":"2024-09-10T09:35:25.528475+0000","last_unstale":"2024-09-10T09:36:14.486020+0000","last_undegraded":"2024-09-10T09:36:14.486020+0000","last_fullsized":"2024-09-10T09:36:14.486020+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:16:06.999186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:36:06.759828+0000","last_change":"2024-09-10T09:35:36.633109+0000","last_active":"2024-09-10T09:36:06.759828+0000","last_peered":"2024-09-10T09:36:06.759828+0000","last_clean":"2024-09-10T09:36:06.759828+0000","last_became_active":"2024-09-10T09:35:36.632682+0000","last_became_peered":"2024-09-10T09:35:36.632682+0000","last_unstale":"2024-09-10T09:36:06.759828+0000","last_undegraded":"2024-09-10T09:36:06.759828+0000","last_fullsized":"2024-09-10T09:36:06.759828+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:51:14.372649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:36:06.765964+0000","last_change":"2024-09-10T09:35:28.562103+0000","last_active":"2024-09-10T09:36:06.765964+0000","last_peered":"2024-09-10T09:36:06.765964+0000","last_clean":"2024-09-10T09:36:06.765964+0000","last_became_active":"2024-09-10T09:35:28.561765+0000","last_became_peered":"2024-09-10T09:35:28.561765+0000","last_unstale":"2024-09-10T09:36:06.765964+0000","last_undegraded":"2024-09-10T09:36:06.765964+0000","last_fullsized":"2024-09-10T09:36:06.765964+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:35:35.400569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"94'5","reported_seq":44,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:36:14.497766+0000","last_change":"2024-09-10T09:35:25.538536+0000","last_active":"2024-09-10T09:36:14.497766+0000","last_peered":"2024-09-10T09:36:14.497766+0000","last_clean":"2024-09-10T09:36:14.497766+0000","last_became_active":"2024-09-10T09:35:25.537944+0000","last_became_peered":"2024-09-10T09:35:25.537944+0000","last_unstale":"2024-09-10T09:36:14.497766+0000","last_undegraded":"2024-09-10T09:36:14.497766+0000","last_fullsized":"2024-09-10T09:36:14.497766+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:01:19.385381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":12,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:36:06.770468+0000","last_change":"2024-09-10T09:36:06.769534+0000","last_active":"2024-09-10T09:36:06.770468+0000","last_peered":"2024-09-10T09:36:06.770468+0000","last_clean":"2024-09-10T09:36:06.770468+0000","last_became_active":"2024-09-10T09:36:06.769015+0000","last_became_peered":"2024-09-10T09:36:06.769015+0000","last_unstale":"2024-09-10T09:36:06.770468+0000","last_undegraded":"2024-09-10T09:36:06.770468+0000","last_fullsized":"2024-09-10T09:36:06.770468+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:19:26.369737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7,1],"acting":[8,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:36:06.762409+0000","last_change":"2024-09-10T09:35:28.566735+0000","last_active":"2024-09-10T09:36:06.762409+0000","last_peered":"2024-09-10T09:36:06.762409+0000","last_clean":"2024-09-10T09:36:06.762409+0000","last_became_active":"2024-09-10T09:35:28.566400+0000","last_became_peered":"2024-09-10T09:35:28.566400+0000","last_unstale":"2024-09-10T09:36:06.762409+0000","last_undegraded":"2024-09-10T09:36:06.762409+0000","last_fullsized":"2024-09-10T09:36:06.762409+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:18:53.337816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"94'2","reported_seq":42,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:36:13.806778+0000","last_change":"2024-09-10T09:35:25.535783+0000","last_active":"2024-09-10T09:36:13.806778+0000","last_peered":"2024-09-10T09:36:13.806778+0000","last_clean":"2024-09-10T09:36:13.806778+0000","last_became_active":"2024-09-10T09:35:25.535411+0000","last_became_peered":"2024-09-10T09:35:25.535411+0000","last_unstale":"2024-09-10T09:36:13.806778+0000","last_undegraded":"2024-09-10T09:36:13.806778+0000","last_fullsized":"2024-09-10T09:36:13.806778+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:48:09.239083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:36:06.763837+0000","last_change":"2024-09-10T09:35:36.619380+0000","last_active":"2024-09-10T09:36:06.763837+0000","last_peered":"2024-09-10T09:36:06.763837+0000","last_clean":"2024-09-10T09:36:06.763837+0000","last_became_active":"2024-09-10T09:35:36.618953+0000","last_became_peered":"2024-09-10T09:35:36.618953+0000","last_unstale":"2024-09-10T09:36:06.763837+0000","last_undegraded":"2024-09-10T09:36:06.763837+0000","last_fullsized":"2024-09-10T09:36:06.763837+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:40:25.181408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:36:06.764051+0000","last_change":"2024-09-10T09:35:28.571396+0000","last_active":"2024-09-10T09:36:06.764051+0000","last_peered":"2024-09-10T09:36:06.764051+0000","last_clean":"2024-09-10T09:36:06.764051+0000","last_became_active":"2024-09-10T09:35:28.571063+0000","last_became_peered":"2024-09-10T09:35:28.571063+0000","last_unstale":"2024-09-10T09:36:06.764051+0000","last_undegraded":"2024-09-10T09:36:06.764051+0000","last_fullsized":"2024-09-10T09:36:06.764051+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:38:17.965133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"94'17","reported_seq":57,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:36:14.486303+0000","last_change":"2024-09-10T09:35:25.851773+0000","last_active":"2024-09-10T09:36:14.486303+0000","last_peered":"2024-09-10T09:36:14.486303+0000","last_clean":"2024-09-10T09:36:14.486303+0000","last_became_active":"2024-09-10T09:35:25.851384+0000","last_became_peered":"2024-09-10T09:35:25.851384+0000","last_unstale":"2024-09-10T09:36:14.486303+0000","last_undegraded":"2024-09-10T09:36:14.486303+0000","last_fullsized":"2024-09-10T09:36:14.486303+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:03:32.338643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:36:06.873579+0000","last_change":"2024-09-10T09:35:36.620971+0000","last_active":"2024-09-10T09:36:06.873579+0000","last_peered":"2024-09-10T09:36:06.873579+0000","last_clean":"2024-09-10T09:36:06.873579+0000","last_became_active":"2024-09-10T09:35:36.620343+0000","last_became_peered":"2024-09-10T09:35:36.620343+0000","last_unstale":"2024-09-10T09:36:06.873579+0000","last_undegraded":"2024-09-10T09:36:06.873579+0000","last_fullsized":"2024-09-10T09:36:06.873579+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:17:19.517882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:36:06.766522+0000","last_change":"2024-09-10T09:35:28.565643+0000","last_active":"2024-09-10T09:36:06.766522+0000","last_peered":"2024-09-10T09:36:06.766522+0000","last_clean":"2024-09-10T09:36:06.766522+0000","last_became_active":"2024-09-10T09:35:28.564217+0000","last_became_peered":"2024-09-10T09:35:28.564217+0000","last_unstale":"2024-09-10T09:36:06.766522+0000","last_undegraded":"2024-09-10T09:36:06.766522+0000","last_fullsized":"2024-09-10T09:36:06.766522+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:53:34.562690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"94'5","reported_seq":60,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:36:27.508597+0000","last_change":"2024-09-10T09:35:25.534167+0000","last_active":"2024-09-10T09:36:27.508597+0000","last_peered":"2024-09-10T09:36:27.508597+0000","last_clean":"2024-09-10T09:36:27.508597+0000","last_became_active":"2024-09-10T09:35:25.533719+0000","last_became_peered":"2024-09-10T09:35:25.533719+0000","last_unstale":"2024-09-10T09:36:27.508597+0000","last_undegraded":"2024-09-10T09:36:27.508597+0000","last_fullsized":"2024-09-10T09:36:27.508597+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:00:03.537355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":18,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:36:06.761923+0000","last_change":"2024-09-10T09:35:36.628716+0000","last_active":"2024-09-10T09:36:06.761923+0000","last_peered":"2024-09-10T09:36:06.761923+0000","last_clean":"2024-09-10T09:36:06.761923+0000","last_became_active":"2024-09-10T09:35:36.628085+0000","last_became_peered":"2024-09-10T09:35:36.628085+0000","last_unstale":"2024-09-10T09:36:06.761923+0000","last_undegraded":"2024-09-10T09:36:06.761923+0000","last_fullsized":"2024-09-10T09:36:06.761923+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:45:01.279403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:36:06.763771+0000","last_change":"2024-09-10T09:35:28.566701+0000","last_active":"2024-09-10T09:36:06.763771+0000","last_peered":"2024-09-10T09:36:06.763771+0000","last_clean":"2024-09-10T09:36:06.763771+0000","last_became_active":"2024-09-10T09:35:28.566391+0000","last_became_peered":"2024-09-10T09:35:28.566391+0000","last_unstale":"2024-09-10T09:36:06.763771+0000","last_undegraded":"2024-09-10T09:36:06.763771+0000","last_fullsized":"2024-09-10T09:36:06.763771+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:57:59.918706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"94'5","reported_seq":45,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:36:14.497797+0000","last_change":"2024-09-10T09:35:25.532510+0000","last_active":"2024-09-10T09:36:14.497797+0000","last_peered":"2024-09-10T09:36:14.497797+0000","last_clean":"2024-09-10T09:36:14.497797+0000","last_became_active":"2024-09-10T09:35:25.532193+0000","last_became_peered":"2024-09-10T09:35:25.532193+0000","last_unstale":"2024-09-10T09:36:14.497797+0000","last_undegraded":"2024-09-10T09:36:14.497797+0000","last_fullsized":"2024-09-10T09:36:14.497797+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:17:24.685178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:36:06.761950+0000","last_change":"2024-09-10T09:35:36.630742+0000","last_active":"2024-09-10T09:36:06.761950+0000","last_peered":"2024-09-10T09:36:06.761950+0000","last_clean":"2024-09-10T09:36:06.761950+0000","last_became_active":"2024-09-10T09:35:36.630335+0000","last_became_peered":"2024-09-10T09:35:36.630335+0000","last_unstale":"2024-09-10T09:36:06.761950+0000","last_undegraded":"2024-09-10T09:36:06.761950+0000","last_fullsized":"2024-09-10T09:36:06.761950+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:29:12.029390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:36:06.873091+0000","last_change":"2024-09-10T09:35:28.581817+0000","last_active":"2024-09-10T09:36:06.873091+0000","last_peered":"2024-09-10T09:36:06.873091+0000","last_clean":"2024-09-10T09:36:06.873091+0000","last_became_active":"2024-09-10T09:35:28.580316+0000","last_became_peered":"2024-09-10T09:35:28.580316+0000","last_unstale":"2024-09-10T09:36:06.873091+0000","last_undegraded":"2024-09-10T09:36:06.873091+0000","last_fullsized":"2024-09-10T09:36:06.873091+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:04:08.249564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"94'9","reported_seq":50,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:36:14.498084+0000","last_change":"2024-09-10T09:35:25.537474+0000","last_active":"2024-09-10T09:36:14.498084+0000","last_peered":"2024-09-10T09:36:14.498084+0000","last_clean":"2024-09-10T09:36:14.498084+0000","last_became_active":"2024-09-10T09:35:25.537096+0000","last_became_peered":"2024-09-10T09:35:25.537096+0000","last_unstale":"2024-09-10T09:36:14.498084+0000","last_undegraded":"2024-09-10T09:36:14.498084+0000","last_fullsized":"2024-09-10T09:36:14.498084+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:13:06.008458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:36:06.763503+0000","last_change":"2024-09-10T09:35:36.622788+0000","last_active":"2024-09-10T09:36:06.763503+0000","last_peered":"2024-09-10T09:36:06.763503+0000","last_clean":"2024-09-10T09:36:06.763503+0000","last_became_active":"2024-09-10T09:35:36.622377+0000","last_became_peered":"2024-09-10T09:35:36.622377+0000","last_unstale":"2024-09-10T09:36:06.763503+0000","last_undegraded":"2024-09-10T09:36:06.763503+0000","last_fullsized":"2024-09-10T09:36:06.763503+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:29:56.680864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:36:06.767059+0000","last_change":"2024-09-10T09:35:28.564529+0000","last_active":"2024-09-10T09:36:06.767059+0000","last_peered":"2024-09-10T09:36:06.767059+0000","last_clean":"2024-09-10T09:36:06.767059+0000","last_became_active":"2024-09-10T09:35:28.564223+0000","last_became_peered":"2024-09-10T09:35:28.564223+0000","last_unstale":"2024-09-10T09:36:06.767059+0000","last_undegraded":"2024-09-10T09:36:06.767059+0000","last_fullsized":"2024-09-10T09:36:06.767059+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:21:24.296578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"94'31","reported_seq":109,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:36:29.512195+0000","last_change":"2024-09-10T09:35:25.534331+0000","last_active":"2024-09-10T09:36:29.512195+0000","last_peered":"2024-09-10T09:36:29.512195+0000","last_clean":"2024-09-10T09:36:29.512195+0000","last_became_active":"2024-09-10T09:35:25.533985+0000","last_became_peered":"2024-09-10T09:35:25.533985+0000","last_unstale":"2024-09-10T09:36:29.512195+0000","last_undegraded":"2024-09-10T09:36:29.512195+0000","last_fullsized":"2024-09-10T09:36:29.512195+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:14:37.050650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":52413,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":39,"num_read_kb":70,"num_write":33,"num_write_kb":63,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:36:06.760797+0000","last_change":"2024-09-10T09:35:36.632835+0000","last_active":"2024-09-10T09:36:06.760797+0000","last_peered":"2024-09-10T09:36:06.760797+0000","last_clean":"2024-09-10T09:36:06.760797+0000","last_became_active":"2024-09-10T09:35:36.630962+0000","last_became_peered":"2024-09-10T09:35:36.630962+0000","last_unstale":"2024-09-10T09:36:06.760797+0000","last_undegraded":"2024-09-10T09:36:06.760797+0000","last_fullsized":"2024-09-10T09:36:06.760797+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:59:27.613721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:36:06.765554+0000","last_change":"2024-09-10T09:35:28.567292+0000","last_active":"2024-09-10T09:36:06.765554+0000","last_peered":"2024-09-10T09:36:06.765554+0000","last_clean":"2024-09-10T09:36:06.765554+0000","last_became_active":"2024-09-10T09:35:28.566973+0000","last_became_peered":"2024-09-10T09:35:28.566973+0000","last_unstale":"2024-09-10T09:36:06.765554+0000","last_undegraded":"2024-09-10T09:36:06.765554+0000","last_fullsized":"2024-09-10T09:36:06.765554+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:05:37.251933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"94'5","reported_seq":45,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:36:13.816041+0000","last_change":"2024-09-10T09:35:25.852171+0000","last_active":"2024-09-10T09:36:13.816041+0000","last_peered":"2024-09-10T09:36:13.816041+0000","last_clean":"2024-09-10T09:36:13.816041+0000","last_became_active":"2024-09-10T09:35:25.851580+0000","last_became_peered":"2024-09-10T09:35:25.851580+0000","last_unstale":"2024-09-10T09:36:13.816041+0000","last_undegraded":"2024-09-10T09:36:13.816041+0000","last_fullsized":"2024-09-10T09:36:13.816041+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:03.263152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:36:06.765208+0000","last_change":"2024-09-10T09:35:36.629212+0000","last_active":"2024-09-10T09:36:06.765208+0000","last_peered":"2024-09-10T09:36:06.765208+0000","last_clean":"2024-09-10T09:36:06.765208+0000","last_became_active":"2024-09-10T09:35:36.628753+0000","last_became_peered":"2024-09-10T09:35:36.628753+0000","last_unstale":"2024-09-10T09:36:06.765208+0000","last_undegraded":"2024-09-10T09:36:06.765208+0000","last_fullsized":"2024-09-10T09:36:06.765208+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:54:45.311361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:36:06.760286+0000","last_change":"2024-09-10T09:35:28.572497+0000","last_active":"2024-09-10T09:36:06.760286+0000","last_peered":"2024-09-10T09:36:06.760286+0000","last_clean":"2024-09-10T09:36:06.760286+0000","last_became_active":"2024-09-10T09:35:28.569238+0000","last_became_peered":"2024-09-10T09:35:28.569238+0000","last_unstale":"2024-09-10T09:36:06.760286+0000","last_undegraded":"2024-09-10T09:36:06.760286+0000","last_fullsized":"2024-09-10T09:36:06.760286+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:38:47.189209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"94'4","reported_seq":42,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:36:14.497898+0000","last_change":"2024-09-10T09:35:25.536369+0000","last_active":"2024-09-10T09:36:14.497898+0000","last_peered":"2024-09-10T09:36:14.497898+0000","last_clean":"2024-09-10T09:36:14.497898+0000","last_became_active":"2024-09-10T09:35:25.535607+0000","last_became_peered":"2024-09-10T09:35:25.535607+0000","last_unstale":"2024-09-10T09:36:14.497898+0000","last_undegraded":"2024-09-10T09:36:14.497898+0000","last_fullsized":"2024-09-10T09:36:14.497898+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:49:18.621934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:36:06.760500+0000","last_change":"2024-09-10T09:35:36.627977+0000","last_active":"2024-09-10T09:36:06.760500+0000","last_peered":"2024-09-10T09:36:06.760500+0000","last_clean":"2024-09-10T09:36:06.760500+0000","last_became_active":"2024-09-10T09:35:36.626942+0000","last_became_peered":"2024-09-10T09:35:36.626942+0000","last_unstale":"2024-09-10T09:36:06.760500+0000","last_undegraded":"2024-09-10T09:36:06.760500+0000","last_fullsized":"2024-09-10T09:36:06.760500+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:37:18.209424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:36:06.766076+0000","last_change":"2024-09-10T09:35:28.562670+0000","last_active":"2024-09-10T09:36:06.766076+0000","last_peered":"2024-09-10T09:36:06.766076+0000","last_clean":"2024-09-10T09:36:06.766076+0000","last_became_active":"2024-09-10T09:35:28.562028+0000","last_became_peered":"2024-09-10T09:35:28.562028+0000","last_unstale":"2024-09-10T09:36:06.766076+0000","last_undegraded":"2024-09-10T09:36:06.766076+0000","last_fullsized":"2024-09-10T09:36:06.766076+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:37:15.176050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"94'4","reported_seq":75,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:36:29.511521+0000","last_change":"2024-09-10T09:35:25.529861+0000","last_active":"2024-09-10T09:36:29.511521+0000","last_peered":"2024-09-10T09:36:29.511521+0000","last_clean":"2024-09-10T09:36:29.511521+0000","last_became_active":"2024-09-10T09:35:25.529475+0000","last_became_peered":"2024-09-10T09:35:25.529475+0000","last_unstale":"2024-09-10T09:36:29.511521+0000","last_undegraded":"2024-09-10T09:36:29.511521+0000","last_fullsized":"2024-09-10T09:36:29.511521+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:44:27.086509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":32,"num_read_kb":32,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:36:06.763318+0000","last_change":"2024-09-10T09:35:36.615358+0000","last_active":"2024-09-10T09:36:06.763318+0000","last_peered":"2024-09-10T09:36:06.763318+0000","last_clean":"2024-09-10T09:36:06.763318+0000","last_became_active":"2024-09-10T09:35:36.614901+0000","last_became_peered":"2024-09-10T09:35:36.614901+0000","last_unstale":"2024-09-10T09:36:06.763318+0000","last_undegraded":"2024-09-10T09:36:06.763318+0000","last_fullsized":"2024-09-10T09:36:06.763318+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:56:54.248266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:36:06.759865+0000","last_change":"2024-09-10T09:35:36.629021+0000","last_active":"2024-09-10T09:36:06.759865+0000","last_peered":"2024-09-10T09:36:06.759865+0000","last_clean":"2024-09-10T09:36:06.759865+0000","last_became_active":"2024-09-10T09:35:36.628159+0000","last_became_peered":"2024-09-10T09:35:36.628159+0000","last_unstale":"2024-09-10T09:36:06.759865+0000","last_undegraded":"2024-09-10T09:36:06.759865+0000","last_fullsized":"2024-09-10T09:36:06.759865+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:50:48.615236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:36:06.760577+0000","last_change":"2024-09-10T09:35:28.579333+0000","last_active":"2024-09-10T09:36:06.760577+0000","last_peered":"2024-09-10T09:36:06.760577+0000","last_clean":"2024-09-10T09:36:06.760577+0000","last_became_active":"2024-09-10T09:35:28.578440+0000","last_became_peered":"2024-09-10T09:35:28.578440+0000","last_unstale":"2024-09-10T09:36:06.760577+0000","last_undegraded":"2024-09-10T09:36:06.760577+0000","last_fullsized":"2024-09-10T09:36:06.760577+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:07:28.701938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"94'3","reported_seq":43,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:36:14.490660+0000","last_change":"2024-09-10T09:35:25.530857+0000","last_active":"2024-09-10T09:36:14.490660+0000","last_peered":"2024-09-10T09:36:14.490660+0000","last_clean":"2024-09-10T09:36:14.490660+0000","last_became_active":"2024-09-10T09:35:25.530492+0000","last_became_peered":"2024-09-10T09:35:25.530492+0000","last_unstale":"2024-09-10T09:36:14.490660+0000","last_undegraded":"2024-09-10T09:36:14.490660+0000","last_fullsized":"2024-09-10T09:36:14.490660+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:07:21.726598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":31,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:36:06.770408+0000","last_change":"2024-09-10T09:36:06.770408+0000","last_active":"2024-09-10T09:36:06.770408+0000","last_peered":"2024-09-10T09:36:06.770408+0000","last_clean":"2024-09-10T09:36:06.770408+0000","last_became_active":"2024-09-10T09:36:06.769915+0000","last_became_peered":"2024-09-10T09:36:06.769915+0000","last_unstale":"2024-09-10T09:36:06.770408+0000","last_undegraded":"2024-09-10T09:36:06.770408+0000","last_fullsized":"2024-09-10T09:36:06.770408+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:56:09.795537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8,0],"acting":[11,4,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:36:06.763197+0000","last_change":"2024-09-10T09:35:28.570804+0000","last_active":"2024-09-10T09:36:06.763197+0000","last_peered":"2024-09-10T09:36:06.763197+0000","last_clean":"2024-09-10T09:36:06.763197+0000","last_became_active":"2024-09-10T09:35:28.570490+0000","last_became_peered":"2024-09-10T09:35:28.570490+0000","last_unstale":"2024-09-10T09:36:06.763197+0000","last_undegraded":"2024-09-10T09:36:06.763197+0000","last_fullsized":"2024-09-10T09:36:06.763197+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:13:38.999625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"94'3","reported_seq":42,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:36:12.814442+0000","last_change":"2024-09-10T09:35:25.536347+0000","last_active":"2024-09-10T09:36:12.814442+0000","last_peered":"2024-09-10T09:36:12.814442+0000","last_clean":"2024-09-10T09:36:12.814442+0000","last_became_active":"2024-09-10T09:35:25.535964+0000","last_became_peered":"2024-09-10T09:35:25.535964+0000","last_unstale":"2024-09-10T09:36:12.814442+0000","last_undegraded":"2024-09-10T09:36:12.814442+0000","last_fullsized":"2024-09-10T09:36:12.814442+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:36:49.035666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"94'7","reported_seq":46,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:36:13.818400+0000","last_change":"2024-09-10T09:35:25.532148+0000","last_active":"2024-09-10T09:36:13.818400+0000","last_peered":"2024-09-10T09:36:13.818400+0000","last_clean":"2024-09-10T09:36:13.818400+0000","last_became_active":"2024-09-10T09:35:25.531781+0000","last_became_peered":"2024-09-10T09:35:25.531781+0000","last_unstale":"2024-09-10T09:36:13.818400+0000","last_undegraded":"2024-09-10T09:36:13.818400+0000","last_fullsized":"2024-09-10T09:36:13.818400+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:52:46.740342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:36:06.763468+0000","last_change":"2024-09-10T09:35:28.578336+0000","last_active":"2024-09-10T09:36:06.763468+0000","last_peered":"2024-09-10T09:36:06.763468+0000","last_clean":"2024-09-10T09:36:06.763468+0000","last_became_active":"2024-09-10T09:35:28.577899+0000","last_became_peered":"2024-09-10T09:35:28.577899+0000","last_unstale":"2024-09-10T09:36:06.763468+0000","last_undegraded":"2024-09-10T09:36:06.763468+0000","last_fullsized":"2024-09-10T09:36:06.763468+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:13:04.478195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:36:06.760465+0000","last_change":"2024-09-10T09:35:36.630918+0000","last_active":"2024-09-10T09:36:06.760465+0000","last_peered":"2024-09-10T09:36:06.760465+0000","last_clean":"2024-09-10T09:36:06.760465+0000","last_became_active":"2024-09-10T09:35:36.630308+0000","last_became_peered":"2024-09-10T09:35:36.630308+0000","last_unstale":"2024-09-10T09:36:06.760465+0000","last_undegraded":"2024-09-10T09:36:06.760465+0000","last_fullsized":"2024-09-10T09:36:06.760465+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:25:44.954970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"94'6","reported_seq":64,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:36:28.510545+0000","last_change":"2024-09-10T09:35:25.537670+0000","last_active":"2024-09-10T09:36:28.510545+0000","last_peered":"2024-09-10T09:36:28.510545+0000","last_clean":"2024-09-10T09:36:28.510545+0000","last_became_active":"2024-09-10T09:35:25.537359+0000","last_became_peered":"2024-09-10T09:35:25.537359+0000","last_unstale":"2024-09-10T09:36:28.510545+0000","last_undegraded":"2024-09-10T09:36:28.510545+0000","last_fullsized":"2024-09-10T09:36:28.510545+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:06:55.074836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1015,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":19,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:36:06.766706+0000","last_change":"2024-09-10T09:35:28.567915+0000","last_active":"2024-09-10T09:36:06.766706+0000","last_peered":"2024-09-10T09:36:06.766706+0000","last_clean":"2024-09-10T09:36:06.766706+0000","last_became_active":"2024-09-10T09:35:28.567416+0000","last_became_peered":"2024-09-10T09:35:28.567416+0000","last_unstale":"2024-09-10T09:36:06.766706+0000","last_undegraded":"2024-09-10T09:36:06.766706+0000","last_fullsized":"2024-09-10T09:36:06.766706+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:07:25.704080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:36:06.760638+0000","last_change":"2024-09-10T09:35:36.615386+0000","last_active":"2024-09-10T09:36:06.760638+0000","last_peered":"2024-09-10T09:36:06.760638+0000","last_clean":"2024-09-10T09:36:06.760638+0000","last_became_active":"2024-09-10T09:35:36.614967+0000","last_became_peered":"2024-09-10T09:35:36.614967+0000","last_unstale":"2024-09-10T09:36:06.760638+0000","last_undegraded":"2024-09-10T09:36:06.760638+0000","last_fullsized":"2024-09-10T09:36:06.760638+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:47.620729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:36:06.762580+0000","last_change":"2024-09-10T09:35:28.564411+0000","last_active":"2024-09-10T09:36:06.762580+0000","last_peered":"2024-09-10T09:36:06.762580+0000","last_clean":"2024-09-10T09:36:06.762580+0000","last_became_active":"2024-09-10T09:35:28.564034+0000","last_became_peered":"2024-09-10T09:35:28.564034+0000","last_unstale":"2024-09-10T09:36:06.762580+0000","last_undegraded":"2024-09-10T09:36:06.762580+0000","last_fullsized":"2024-09-10T09:36:06.762580+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:38:17.687991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:36:06.760601+0000","last_change":"2024-09-10T09:35:28.564619+0000","last_active":"2024-09-10T09:36:06.760601+0000","last_peered":"2024-09-10T09:36:06.760601+0000","last_clean":"2024-09-10T09:36:06.760601+0000","last_became_active":"2024-09-10T09:35:28.563890+0000","last_became_peered":"2024-09-10T09:35:28.563890+0000","last_unstale":"2024-09-10T09:36:06.760601+0000","last_undegraded":"2024-09-10T09:36:06.760601+0000","last_fullsized":"2024-09-10T09:36:06.760601+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:51:42.481911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:36:06.760213+0000","last_change":"2024-09-10T09:35:28.558909+0000","last_active":"2024-09-10T09:36:06.760213+0000","last_peered":"2024-09-10T09:36:06.760213+0000","last_clean":"2024-09-10T09:36:06.760213+0000","last_became_active":"2024-09-10T09:35:28.558604+0000","last_became_peered":"2024-09-10T09:35:28.558604+0000","last_unstale":"2024-09-10T09:36:06.760213+0000","last_undegraded":"2024-09-10T09:36:06.760213+0000","last_fullsized":"2024-09-10T09:36:06.760213+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:11.928446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:36:06.759939+0000","last_change":"2024-09-10T09:35:28.574459+0000","last_active":"2024-09-10T09:36:06.759939+0000","last_peered":"2024-09-10T09:36:06.759939+0000","last_clean":"2024-09-10T09:36:06.759939+0000","last_became_active":"2024-09-10T09:35:28.574154+0000","last_became_peered":"2024-09-10T09:35:28.574154+0000","last_unstale":"2024-09-10T09:36:06.759939+0000","last_undegraded":"2024-09-10T09:36:06.759939+0000","last_fullsized":"2024-09-10T09:36:06.759939+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:59:09.650536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:36:06.873415+0000","last_change":"2024-09-10T09:35:28.576999+0000","last_active":"2024-09-10T09:36:06.873415+0000","last_peered":"2024-09-10T09:36:06.873415+0000","last_clean":"2024-09-10T09:36:06.873415+0000","last_became_active":"2024-09-10T09:35:28.575616+0000","last_became_peered":"2024-09-10T09:35:28.575616+0000","last_unstale":"2024-09-10T09:36:06.873415+0000","last_undegraded":"2024-09-10T09:36:06.873415+0000","last_fullsized":"2024-09-10T09:36:06.873415+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:18:08.771760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:36:06.762869+0000","last_change":"2024-09-10T09:35:28.564846+0000","last_active":"2024-09-10T09:36:06.762869+0000","last_peered":"2024-09-10T09:36:06.762869+0000","last_clean":"2024-09-10T09:36:06.762869+0000","last_became_active":"2024-09-10T09:35:28.564498+0000","last_became_peered":"2024-09-10T09:35:28.564498+0000","last_unstale":"2024-09-10T09:36:06.762869+0000","last_undegraded":"2024-09-10T09:36:06.762869+0000","last_fullsized":"2024-09-10T09:36:06.762869+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:33:43.041730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:36:06.760669+0000","last_change":"2024-09-10T09:35:28.558093+0000","last_active":"2024-09-10T09:36:06.760669+0000","last_peered":"2024-09-10T09:36:06.760669+0000","last_clean":"2024-09-10T09:36:06.760669+0000","last_became_active":"2024-09-10T09:35:28.557266+0000","last_became_peered":"2024-09-10T09:35:28.557266+0000","last_unstale":"2024-09-10T09:36:06.760669+0000","last_undegraded":"2024-09-10T09:36:06.760669+0000","last_fullsized":"2024-09-10T09:36:06.760669+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:51:30.946065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:36:06.770298+0000","last_change":"2024-09-10T09:36:06.770298+0000","last_active":"2024-09-10T09:36:06.770298+0000","last_peered":"2024-09-10T09:36:06.770298+0000","last_clean":"2024-09-10T09:36:06.770298+0000","last_became_active":"2024-09-10T09:36:06.769848+0000","last_became_peered":"2024-09-10T09:36:06.769848+0000","last_unstale":"2024-09-10T09:36:06.770298+0000","last_undegraded":"2024-09-10T09:36:06.770298+0000","last_fullsized":"2024-09-10T09:36:06.770298+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:00:30.470887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,8,2],"acting":[3,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:36:06.763611+0000","last_change":"2024-09-10T09:35:28.578158+0000","last_active":"2024-09-10T09:36:06.763611+0000","last_peered":"2024-09-10T09:36:06.763611+0000","last_clean":"2024-09-10T09:36:06.763611+0000","last_became_active":"2024-09-10T09:35:28.577824+0000","last_became_peered":"2024-09-10T09:35:28.577824+0000","last_unstale":"2024-09-10T09:36:06.763611+0000","last_undegraded":"2024-09-10T09:36:06.763611+0000","last_fullsized":"2024-09-10T09:36:06.763611+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:40:27.231164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:36:06.763546+0000","last_change":"2024-09-10T09:35:28.562670+0000","last_active":"2024-09-10T09:36:06.763546+0000","last_peered":"2024-09-10T09:36:06.763546+0000","last_clean":"2024-09-10T09:36:06.763546+0000","last_became_active":"2024-09-10T09:35:28.562311+0000","last_became_peered":"2024-09-10T09:35:28.562311+0000","last_unstale":"2024-09-10T09:36:06.763546+0000","last_undegraded":"2024-09-10T09:36:06.763546+0000","last_fullsized":"2024-09-10T09:36:06.763546+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:07:11.716142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:36:06.765363+0000","last_change":"2024-09-10T09:35:28.561124+0000","last_active":"2024-09-10T09:36:06.765363+0000","last_peered":"2024-09-10T09:36:06.765363+0000","last_clean":"2024-09-10T09:36:06.765363+0000","last_became_active":"2024-09-10T09:35:28.560777+0000","last_became_peered":"2024-09-10T09:35:28.560777+0000","last_unstale":"2024-09-10T09:36:06.765363+0000","last_undegraded":"2024-09-10T09:36:06.765363+0000","last_fullsized":"2024-09-10T09:36:06.765363+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:47:04.837596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:36:06.873206+0000","last_change":"2024-09-10T09:35:28.576790+0000","last_active":"2024-09-10T09:36:06.873206+0000","last_peered":"2024-09-10T09:36:06.873206+0000","last_clean":"2024-09-10T09:36:06.873206+0000","last_became_active":"2024-09-10T09:35:28.576464+0000","last_became_peered":"2024-09-10T09:35:28.576464+0000","last_unstale":"2024-09-10T09:36:06.873206+0000","last_undegraded":"2024-09-10T09:36:06.873206+0000","last_fullsized":"2024-09-10T09:36:06.873206+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:40:53.517577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:36:06.762086+0000","last_change":"2024-09-10T09:35:28.569935+0000","last_active":"2024-09-10T09:36:06.762086+0000","last_peered":"2024-09-10T09:36:06.762086+0000","last_clean":"2024-09-10T09:36:06.762086+0000","last_became_active":"2024-09-10T09:35:28.569539+0000","last_became_peered":"2024-09-10T09:35:28.569539+0000","last_unstale":"2024-09-10T09:36:06.762086+0000","last_undegraded":"2024-09-10T09:36:06.762086+0000","last_fullsized":"2024-09-10T09:36:06.762086+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:40:45.016269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:36:06.760825+0000","last_change":"2024-09-10T09:35:28.571619+0000","last_active":"2024-09-10T09:36:06.760825+0000","last_peered":"2024-09-10T09:36:06.760825+0000","last_clean":"2024-09-10T09:36:06.760825+0000","last_became_active":"2024-09-10T09:35:28.568591+0000","last_became_peered":"2024-09-10T09:35:28.568591+0000","last_unstale":"2024-09-10T09:36:06.760825+0000","last_undegraded":"2024-09-10T09:36:06.760825+0000","last_fullsized":"2024-09-10T09:36:06.760825+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:07:49.959712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:36:06.761923+0000","last_change":"2024-09-10T09:35:28.573336+0000","last_active":"2024-09-10T09:36:06.761923+0000","last_peered":"2024-09-10T09:36:06.761923+0000","last_clean":"2024-09-10T09:36:06.761923+0000","last_became_active":"2024-09-10T09:35:28.572991+0000","last_became_peered":"2024-09-10T09:35:28.572991+0000","last_unstale":"2024-09-10T09:36:06.761923+0000","last_undegraded":"2024-09-10T09:36:06.761923+0000","last_fullsized":"2024-09-10T09:36:06.761923+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:30:02.905869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:36:06.763406+0000","last_change":"2024-09-10T09:35:28.571945+0000","last_active":"2024-09-10T09:36:06.763406+0000","last_peered":"2024-09-10T09:36:06.763406+0000","last_clean":"2024-09-10T09:36:06.763406+0000","last_became_active":"2024-09-10T09:35:28.571546+0000","last_became_peered":"2024-09-10T09:35:28.571546+0000","last_unstale":"2024-09-10T09:36:06.763406+0000","last_undegraded":"2024-09-10T09:36:06.763406+0000","last_fullsized":"2024-09-10T09:36:06.763406+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:36:47.946864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:36:06.764770+0000","last_change":"2024-09-10T09:35:28.562712+0000","last_active":"2024-09-10T09:36:06.764770+0000","last_peered":"2024-09-10T09:36:06.764770+0000","last_clean":"2024-09-10T09:36:06.764770+0000","last_became_active":"2024-09-10T09:35:28.562396+0000","last_became_peered":"2024-09-10T09:35:28.562396+0000","last_unstale":"2024-09-10T09:36:06.764770+0000","last_undegraded":"2024-09-10T09:36:06.764770+0000","last_fullsized":"2024-09-10T09:36:06.764770+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:25:47.419162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:36:06.759492+0000","last_change":"2024-09-10T09:35:28.566354+0000","last_active":"2024-09-10T09:36:06.759492+0000","last_peered":"2024-09-10T09:36:06.759492+0000","last_clean":"2024-09-10T09:36:06.759492+0000","last_became_active":"2024-09-10T09:35:28.565248+0000","last_became_peered":"2024-09-10T09:35:28.565248+0000","last_unstale":"2024-09-10T09:36:06.759492+0000","last_undegraded":"2024-09-10T09:36:06.759492+0000","last_fullsized":"2024-09-10T09:36:06.759492+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:48:46.741010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:36:06.759606+0000","last_change":"2024-09-10T09:35:28.563331+0000","last_active":"2024-09-10T09:36:06.759606+0000","last_peered":"2024-09-10T09:36:06.759606+0000","last_clean":"2024-09-10T09:36:06.759606+0000","last_became_active":"2024-09-10T09:35:28.563020+0000","last_became_peered":"2024-09-10T09:35:28.563020+0000","last_unstale":"2024-09-10T09:36:06.759606+0000","last_undegraded":"2024-09-10T09:36:06.759606+0000","last_fullsized":"2024-09-10T09:36:06.759606+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:27.653551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:36:06.873157+0000","last_change":"2024-09-10T09:35:28.566687+0000","last_active":"2024-09-10T09:36:06.873157+0000","last_peered":"2024-09-10T09:36:06.873157+0000","last_clean":"2024-09-10T09:36:06.873157+0000","last_became_active":"2024-09-10T09:35:28.566253+0000","last_became_peered":"2024-09-10T09:35:28.566253+0000","last_unstale":"2024-09-10T09:36:06.873157+0000","last_undegraded":"2024-09-10T09:36:06.873157+0000","last_fullsized":"2024-09-10T09:36:06.873157+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:00:15.709725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:36:06.764878+0000","last_change":"2024-09-10T09:35:28.562251+0000","last_active":"2024-09-10T09:36:06.764878+0000","last_peered":"2024-09-10T09:36:06.764878+0000","last_clean":"2024-09-10T09:36:06.764878+0000","last_became_active":"2024-09-10T09:35:28.561933+0000","last_became_peered":"2024-09-10T09:35:28.561933+0000","last_unstale":"2024-09-10T09:36:06.764878+0000","last_undegraded":"2024-09-10T09:36:06.764878+0000","last_fullsized":"2024-09-10T09:36:06.764878+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:30:41.447954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:36:06.763944+0000","last_change":"2024-09-10T09:35:28.572164+0000","last_active":"2024-09-10T09:36:06.763944+0000","last_peered":"2024-09-10T09:36:06.763944+0000","last_clean":"2024-09-10T09:36:06.763944+0000","last_became_active":"2024-09-10T09:35:28.571825+0000","last_became_peered":"2024-09-10T09:35:28.571825+0000","last_unstale":"2024-09-10T09:36:06.763944+0000","last_undegraded":"2024-09-10T09:36:06.763944+0000","last_fullsized":"2024-09-10T09:36:06.763944+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:59:50.967269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:36:06.873657+0000","last_change":"2024-09-10T09:35:28.568098+0000","last_active":"2024-09-10T09:36:06.873657+0000","last_peered":"2024-09-10T09:36:06.873657+0000","last_clean":"2024-09-10T09:36:06.873657+0000","last_became_active":"2024-09-10T09:35:28.567765+0000","last_became_peered":"2024-09-10T09:35:28.567765+0000","last_unstale":"2024-09-10T09:36:06.873657+0000","last_undegraded":"2024-09-10T09:36:06.873657+0000","last_fullsized":"2024-09-10T09:36:06.873657+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:55:57.260320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:36:06.764041+0000","last_change":"2024-09-10T09:35:28.564123+0000","last_active":"2024-09-10T09:36:06.764041+0000","last_peered":"2024-09-10T09:36:06.764041+0000","last_clean":"2024-09-10T09:36:06.764041+0000","last_became_active":"2024-09-10T09:35:28.563814+0000","last_became_peered":"2024-09-10T09:35:28.563814+0000","last_unstale":"2024-09-10T09:36:06.764041+0000","last_undegraded":"2024-09-10T09:36:06.764041+0000","last_fullsized":"2024-09-10T09:36:06.764041+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:14:17.912087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:36:06.763268+0000","last_change":"2024-09-10T09:35:28.568601+0000","last_active":"2024-09-10T09:36:06.763268+0000","last_peered":"2024-09-10T09:36:06.763268+0000","last_clean":"2024-09-10T09:36:06.763268+0000","last_became_active":"2024-09-10T09:35:28.568303+0000","last_became_peered":"2024-09-10T09:35:28.568303+0000","last_unstale":"2024-09-10T09:36:06.763268+0000","last_undegraded":"2024-09-10T09:36:06.763268+0000","last_fullsized":"2024-09-10T09:36:06.763268+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:30:01.016420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:36:06.763766+0000","last_change":"2024-09-10T09:35:28.565217+0000","last_active":"2024-09-10T09:36:06.763766+0000","last_peered":"2024-09-10T09:36:06.763766+0000","last_clean":"2024-09-10T09:36:06.763766+0000","last_became_active":"2024-09-10T09:35:28.564261+0000","last_became_peered":"2024-09-10T09:35:28.564261+0000","last_unstale":"2024-09-10T09:36:06.763766+0000","last_undegraded":"2024-09-10T09:36:06.763766+0000","last_fullsized":"2024-09-10T09:36:06.763766+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:29:30.298487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:36:06.872976+0000","last_change":"2024-09-10T09:35:28.569225+0000","last_active":"2024-09-10T09:36:06.872976+0000","last_peered":"2024-09-10T09:36:06.872976+0000","last_clean":"2024-09-10T09:36:06.872976+0000","last_became_active":"2024-09-10T09:35:28.568891+0000","last_became_peered":"2024-09-10T09:35:28.568891+0000","last_unstale":"2024-09-10T09:36:06.872976+0000","last_undegraded":"2024-09-10T09:36:06.872976+0000","last_fullsized":"2024-09-10T09:36:06.872976+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:33:45.811781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:36:06.766738+0000","last_change":"2024-09-10T09:35:28.578362+0000","last_active":"2024-09-10T09:36:06.766738+0000","last_peered":"2024-09-10T09:36:06.766738+0000","last_clean":"2024-09-10T09:36:06.766738+0000","last_became_active":"2024-09-10T09:35:28.577867+0000","last_became_peered":"2024-09-10T09:35:28.577867+0000","last_unstale":"2024-09-10T09:36:06.766738+0000","last_undegraded":"2024-09-10T09:36:06.766738+0000","last_fullsized":"2024-09-10T09:36:06.766738+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:58:17.041663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:36:06.764779+0000","last_change":"2024-09-10T09:35:28.568183+0000","last_active":"2024-09-10T09:36:06.764779+0000","last_peered":"2024-09-10T09:36:06.764779+0000","last_clean":"2024-09-10T09:36:06.764779+0000","last_became_active":"2024-09-10T09:35:28.566775+0000","last_became_peered":"2024-09-10T09:35:28.566775+0000","last_unstale":"2024-09-10T09:36:06.764779+0000","last_undegraded":"2024-09-10T09:36:06.764779+0000","last_fullsized":"2024-09-10T09:36:06.764779+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:28:10.008889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:36:06.759752+0000","last_change":"2024-09-10T09:35:28.576122+0000","last_active":"2024-09-10T09:36:06.759752+0000","last_peered":"2024-09-10T09:36:06.759752+0000","last_clean":"2024-09-10T09:36:06.759752+0000","last_became_active":"2024-09-10T09:35:28.575831+0000","last_became_peered":"2024-09-10T09:35:28.575831+0000","last_unstale":"2024-09-10T09:36:06.759752+0000","last_undegraded":"2024-09-10T09:36:06.759752+0000","last_fullsized":"2024-09-10T09:36:06.759752+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:37:51.604658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:36:06.759864+0000","last_change":"2024-09-10T09:35:28.564441+0000","last_active":"2024-09-10T09:36:06.759864+0000","last_peered":"2024-09-10T09:36:06.759864+0000","last_clean":"2024-09-10T09:36:06.759864+0000","last_became_active":"2024-09-10T09:35:28.564095+0000","last_became_peered":"2024-09-10T09:35:28.564095+0000","last_unstale":"2024-09-10T09:36:06.759864+0000","last_undegraded":"2024-09-10T09:36:06.759864+0000","last_fullsized":"2024-09-10T09:36:06.759864+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:53:28.275361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-09-10T09:36:06.763287+0000","last_change":"2024-09-10T09:35:28.581921+0000","last_active":"2024-09-10T09:36:06.763287+0000","last_peered":"2024-09-10T09:36:06.763287+0000","last_clean":"2024-09-10T09:36:06.763287+0000","last_became_active":"2024-09-10T09:35:28.581573+0000","last_became_peered":"2024-09-10T09:35:28.581573+0000","last_unstale":"2024-09-10T09:36:06.763287+0000","last_undegraded":"2024-09-10T09:36:06.763287+0000","last_fullsized":"2024-09-10T09:36:06.763287+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:58:38.983934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":59621,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":196,"num_read_kb":164,"num_write":241,"num_write_kb":129,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":544768,"data_stored":183128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":208,"ondisk_log_size":208,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":285627,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":77,"seq":330712481818,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32204,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712948,"statfs":{"total":95995035648,"available":95962058752,"internally_reserved":0,"allocated":1044480,"data_stored":535186,"data_compressed":26500,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":70,"seq":300647710750,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28184,"kb_used_data":1160,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716968,"statfs":{"total":95995035648,"available":95966175232,"internally_reserved":0,"allocated":1187840,"data_stored":1178511,"data_compressed":62821,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":65,"seq":279172874277,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32228,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712924,"statfs":{"total":95995035648,"available":95962034176,"internally_reserved":0,"allocated":1069056,"data_stored":535116,"data_compressed":26500,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1599,"internal_metadata":31914433},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":57,"seq":244813135913,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32224,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712928,"statfs":{"total":95995035648,"available":95962038272,"internally_reserved":0,"allocated":1064960,"data_stored":535494,"data_compressed":26500,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":50,"seq":214748364847,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28116,"kb_used_data":1092,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717036,"statfs":{"total":95995035648,"available":95966244864,"internally_reserved":0,"allocated":1118208,"data_stored":1132386,"data_compressed":62821,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":45,"seq":193273528372,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36364,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708788,"statfs":{"total":95995035648,"available":95957798912,"internally_reserved":0,"allocated":1044480,"data_stored":531013,"data_compressed":26500,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790011,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28124,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717028,"statfs":{"total":95995035648,"available":95966236672,"internally_reserved":0,"allocated":1126400,"data_stored":1127575,"data_compressed":62821,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953536,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32212,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712940,"statfs":{"total":95995035648,"available":95962050560,"internally_reserved":0,"allocated":1052672,"data_stored":534694,"data_compressed":26500,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182469,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28056,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717096,"statfs":{"total":95995035648,"available":95966306304,"internally_reserved":0,"allocated":1056768,"data_stored":530328,"data_compressed":26500,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411402,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28120,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717032,"statfs":{"total":95995035648,"available":95966240768,"internally_reserved":0,"allocated":1122304,"data_stored":584676,"data_compressed":26500,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574929,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32260,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712892,"statfs":{"total":95995035648,"available":95962001408,"internally_reserved":0,"allocated":1101824,"data_stored":584456,"data_compressed":26500,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738453,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36388,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708764,"statfs":{"total":95995035648,"available":95957774336,"internally_reserved":0,"allocated":1069056,"data_stored":532076,"data_compressed":26500,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34674,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34674,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34674,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36321,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36321,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36321,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36321,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36321,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":58823,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":54778,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3213,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1115,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":54149,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T09:36:36.109 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-10T09:36:36.110 INFO:tasks.cephadm:Setting up client nodes... 2024-09-10T09:36:36.110 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-09-10T09:36:36.309 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:36:35 smithi005 ceph-mon[30350]: from='client.14748 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T09:36:36.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:36:35 smithi159 ceph-mon[35513]: from='client.14748 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T09:36:36.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:36:35 smithi102 ceph-mon[35442]: from='client.14748 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T09:36:36.490 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:36:37.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:37.081+0000 7f723c200640 1 -- 172.21.15.5:0/1610431542 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7234104f60 msgr2=0x7f72341053e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:36:37.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:37.081+0000 7f723c200640 1 --2- 172.21.15.5:0/1610431542 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7234104f60 0x7f72341053e0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f7220009860 tx=0x7f722002a340 comp rx=0 tx=0).stop 2024-09-10T09:36:37.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:37.081+0000 7f723c200640 1 -- 172.21.15.5:0/1610431542 shutdown_connections 2024-09-10T09:36:37.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:37.081+0000 7f723c200640 1 --2- 172.21.15.5:0/1610431542 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7234105920 0x7f723410c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:36:37.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:37.081+0000 7f723c200640 1 --2- 172.21.15.5:0/1610431542 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7234104f60 0x7f72341053e0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:36:37.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:37.081+0000 7f723c200640 1 --2- 172.21.15.5:0/1610431542 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7234103e00 0x7f7234104200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:36:37.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:37.081+0000 7f723c200640 1 -- 172.21.15.5:0/1610431542 >> 172.21.15.5:0/1610431542 conn(0x7f72340ff5b0 msgr2=0x7f72341019d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:36:37.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:37.081+0000 7f723c200640 1 -- 172.21.15.5:0/1610431542 shutdown_connections 2024-09-10T09:36:37.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:37.081+0000 7f723c200640 1 -- 172.21.15.5:0/1610431542 wait complete. 2024-09-10T09:36:37.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:37.082+0000 7f723c200640 1 Processor -- start 2024-09-10T09:36:37.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:37.082+0000 7f723c200640 1 -- start start 2024-09-10T09:36:37.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:37.082+0000 7f723c200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7234103e00 0x7f723419a140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:36:37.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:37.082+0000 7f723c200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7234104f60 0x7f723419a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:36:37.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:37.082+0000 7f723c200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7234105920 0x7f72341a1700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:36:37.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:37.082+0000 7f723c200640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f723410fa90 con 0x7f7234103e00 2024-09-10T09:36:37.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:37.082+0000 7f723c200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f723410f910 con 0x7f7234105920 2024-09-10T09:36:37.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:37.082+0000 7f723c200640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f723410fc10 con 0x7f7234104f60 2024-09-10T09:36:37.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:37.083+0000 7f7239800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7234103e00 0x7f723419a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:36:37.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:37.083+0000 7f7239800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7234103e00 0x7f723419a140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46646/0 (socket says 172.21.15.5:46646) 2024-09-10T09:36:37.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:37.083+0000 7f7239800640 1 -- 172.21.15.5:0/3028701131 learned_addr learned my addr 172.21.15.5:0/3028701131 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:36:37.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:37.083+0000 7f7238e00640 1 --2- 172.21.15.5:0/3028701131 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7234104f60 0x7f723419a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:36:37.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:37.083+0000 7f723a200640 1 --2- 172.21.15.5:0/3028701131 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7234105920 0x7f72341a1700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:36:37.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:37.083+0000 7f7239800640 1 -- 172.21.15.5:0/3028701131 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7234104f60 msgr2=0x7f723419a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:36:37.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:37.083+0000 7f7239800640 1 --2- 172.21.15.5:0/3028701131 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7234104f60 0x7f723419a680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:36:37.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:37.083+0000 7f7239800640 1 -- 172.21.15.5:0/3028701131 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7234105920 msgr2=0x7f72341a1700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:36:37.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:37.083+0000 7f7239800640 1 --2- 172.21.15.5:0/3028701131 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7234105920 0x7f72341a1700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:36:37.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:37.083+0000 7f7239800640 1 -- 172.21.15.5:0/3028701131 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72341a1d70 con 0x7f7234103e00 2024-09-10T09:36:37.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:37.083+0000 7f7239800640 1 --2- 172.21.15.5:0/3028701131 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7234103e00 0x7f723419a140 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f722800e9c0 tx=0x7f722800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:36:37.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:37.084+0000 7f7232000640 1 -- 172.21.15.5:0/3028701131 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f722800cde0 con 0x7f7234103e00 2024-09-10T09:36:37.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:37.084+0000 7f7232000640 1 -- 172.21.15.5:0/3028701131 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f72280035f0 con 0x7f7234103e00 2024-09-10T09:36:37.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:37.084+0000 7f723c200640 1 -- 172.21.15.5:0/3028701131 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72341a2060 con 0x7f7234103e00 2024-09-10T09:36:37.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:37.084+0000 7f7232000640 1 -- 172.21.15.5:0/3028701131 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7228010690 con 0x7f7234103e00 2024-09-10T09:36:37.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:37.084+0000 7f723c200640 1 -- 172.21.15.5:0/3028701131 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72341a25c0 con 0x7f7234103e00 2024-09-10T09:36:37.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:37.085+0000 7f723c200640 1 -- 172.21.15.5:0/3028701131 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f723410b290 con 0x7f7234103e00 2024-09-10T09:36:37.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:37.087+0000 7f7232000640 1 -- 172.21.15.5:0/3028701131 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f72280037a0 con 0x7f7234103e00 2024-09-10T09:36:37.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:37.088+0000 7f7232000640 1 --2- 172.21.15.5:0/3028701131 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f721c0a2cc0 0x7f721c0a5180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:36:37.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:37.088+0000 7f7232000640 1 -- 172.21.15.5:0/3028701131 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9071+0+0 (secure 0 0 0) 0x7f72280cf7a0 con 0x7f7234103e00 2024-09-10T09:36:37.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:37.088+0000 7f7238e00640 1 --2- 172.21.15.5:0/3028701131 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f721c0a2cc0 0x7f721c0a5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:36:37.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:37.089+0000 7f7238e00640 1 --2- 172.21.15.5:0/3028701131 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f721c0a2cc0 0x7f721c0a5180 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f7220030410 tx=0x7f7220030550 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:36:37.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:37.092+0000 7f7232000640 1 -- 172.21.15.5:0/3028701131 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f722809c200 con 0x7f7234103e00 2024-09-10T09:36:37.306 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:36:36 smithi005 ceph-mon[30350]: pgmap v339: 129 pgs: 129 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 4.7 KiB/s wr, 15 op/s 2024-09-10T09:36:37.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:37.344+0000 7f723c200640 1 -- 172.21.15.5:0/3028701131 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7f723402d7c0 con 0x7f7234103e00 2024-09-10T09:36:37.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:37.346+0000 7f7232000640 1 -- 172.21.15.5:0/3028701131 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) ==== 170+0+59 (secure 0 0 0) 0x7f722809bb00 con 0x7f7234103e00 2024-09-10T09:36:37.348 INFO:teuthology.orchestra.run.smithi005.stdout:[client.0] 2024-09-10T09:36:37.348 INFO:teuthology.orchestra.run.smithi005.stdout: key = AQAlE+Bmt1ijFBAAvVzSG/d5KPevyKKP9iZMsA== 2024-09-10T09:36:37.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:36:36 smithi159 ceph-mon[35513]: pgmap v339: 129 pgs: 129 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 4.7 KiB/s wr, 15 op/s 2024-09-10T09:36:37.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:37.348+0000 7f723c200640 1 -- 172.21.15.5:0/3028701131 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f721c0a2cc0 msgr2=0x7f721c0a5180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:36:37.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:37.348+0000 7f723c200640 1 --2- 172.21.15.5:0/3028701131 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f721c0a2cc0 0x7f721c0a5180 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f7220030410 tx=0x7f7220030550 comp rx=0 tx=0).stop 2024-09-10T09:36:37.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:37.349+0000 7f723c200640 1 -- 172.21.15.5:0/3028701131 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7234103e00 msgr2=0x7f723419a140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:36:37.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:37.349+0000 7f723c200640 1 --2- 172.21.15.5:0/3028701131 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7234103e00 0x7f723419a140 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f722800e9c0 tx=0x7f722800ee80 comp rx=0 tx=0).stop 2024-09-10T09:36:37.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:37.349+0000 7f723c200640 1 -- 172.21.15.5:0/3028701131 shutdown_connections 2024-09-10T09:36:37.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:37.349+0000 7f723c200640 1 --2- 172.21.15.5:0/3028701131 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f721c0a2cc0 0x7f721c0a5180 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:36:37.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:37.349+0000 7f723c200640 1 --2- 172.21.15.5:0/3028701131 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7234105920 0x7f72341a1700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:36:37.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:37.349+0000 7f723c200640 1 --2- 172.21.15.5:0/3028701131 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7234104f60 0x7f723419a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:36:37.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:37.349+0000 7f723c200640 1 --2- 172.21.15.5:0/3028701131 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7234103e00 0x7f723419a140 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:36:37.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:37.349+0000 7f723c200640 1 -- 172.21.15.5:0/3028701131 >> 172.21.15.5:0/3028701131 conn(0x7f72340ff5b0 msgr2=0x7f7234108460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:36:37.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:37.349+0000 7f723c200640 1 -- 172.21.15.5:0/3028701131 shutdown_connections 2024-09-10T09:36:37.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:37.349+0000 7f723c200640 1 -- 172.21.15.5:0/3028701131 wait complete. 2024-09-10T09:36:37.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:36:36 smithi102 ceph-mon[35442]: pgmap v339: 129 pgs: 129 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 4.7 KiB/s wr, 15 op/s 2024-09-10T09:36:38.060 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-09-10T09:36:38.060 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-09-10T09:36:38.060 DEBUG:teuthology.orchestra.run.smithi005:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-09-10T09:36:38.140 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-10T09:36:38.141 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-09-10T09:36:38.141 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph mgr dump --format=json 2024-09-10T09:36:38.306 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:36:37 smithi005 ceph-mon[30350]: from='client.? 172.21.15.5:0/3028701131' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-10T09:36:38.306 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:36:37 smithi005 ceph-mon[30350]: from='client.? 172.21.15.5:0/3028701131' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-10T09:36:38.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:36:37 smithi159 ceph-mon[35513]: from='client.? 172.21.15.5:0/3028701131' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-10T09:36:38.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:36:37 smithi159 ceph-mon[35513]: from='client.? 172.21.15.5:0/3028701131' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-10T09:36:38.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:36:37 smithi102 ceph-mon[35442]: from='client.? 172.21.15.5:0/3028701131' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-10T09:36:38.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:36:37 smithi102 ceph-mon[35442]: from='client.? 172.21.15.5:0/3028701131' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-10T09:36:38.558 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:36:39.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:39.229+0000 7f29df200640 1 -- 172.21.15.5:0/1354372383 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f29d8102d30 msgr2=0x7f29d8103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:36:39.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:39.229+0000 7f29df200640 1 --2- 172.21.15.5:0/1354372383 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f29d8102d30 0x7f29d8103110 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f29c8009900 tx=0x7f29c802a490 comp rx=0 tx=0).stop 2024-09-10T09:36:39.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:39.229+0000 7f29df200640 1 -- 172.21.15.5:0/1354372383 shutdown_connections 2024-09-10T09:36:39.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:39.229+0000 7f29df200640 1 --2- 172.21.15.5:0/1354372383 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f29d810c160 0x7f29d810e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:36:39.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:39.229+0000 7f29df200640 1 --2- 172.21.15.5:0/1354372383 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f29d8103650 0x7f29d810bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:36:39.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:39.229+0000 7f29df200640 1 --2- 172.21.15.5:0/1354372383 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f29d8102d30 0x7f29d8103110 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:36:39.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:39.229+0000 7f29df200640 1 -- 172.21.15.5:0/1354372383 >> 172.21.15.5:0/1354372383 conn(0x7f29d80fc7d0 msgr2=0x7f29d80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:36:39.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:39.229+0000 7f29df200640 1 -- 172.21.15.5:0/1354372383 shutdown_connections 2024-09-10T09:36:39.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:39.229+0000 7f29df200640 1 -- 172.21.15.5:0/1354372383 wait complete. 2024-09-10T09:36:39.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:39.230+0000 7f29df200640 1 Processor -- start 2024-09-10T09:36:39.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:39.230+0000 7f29df200640 1 -- start start 2024-09-10T09:36:39.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:39.230+0000 7f29df200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f29d8102d30 0x7f29d819a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:36:39.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:39.230+0000 7f29df200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f29d8103650 0x7f29d819ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:36:39.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:39.230+0000 7f29df200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f29d810c160 0x7f29d819ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:36:39.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:39.230+0000 7f29df200640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f29d8111f60 con 0x7f29d810c160 2024-09-10T09:36:39.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:39.230+0000 7f29df200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f29d8111de0 con 0x7f29d8103650 2024-09-10T09:36:39.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:39.230+0000 7f29df200640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f29d81120e0 con 0x7f29d8102d30 2024-09-10T09:36:39.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:39.230+0000 7f29dd200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f29d810c160 0x7f29d819ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:36:39.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:39.230+0000 7f29dd200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f29d810c160 0x7f29d819ef20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46674/0 (socket says 172.21.15.5:46674) 2024-09-10T09:36:39.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:39.230+0000 7f29dd200640 1 -- 172.21.15.5:0/3306121284 learned_addr learned my addr 172.21.15.5:0/3306121284 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:36:39.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:39.231+0000 7f29d7e00640 1 --2- 172.21.15.5:0/3306121284 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f29d8102d30 0x7f29d819a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:36:39.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:39.231+0000 7f29cfe00640 1 --2- 172.21.15.5:0/3306121284 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f29d8103650 0x7f29d819ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:36:39.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:39.231+0000 7f29dd200640 1 -- 172.21.15.5:0/3306121284 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f29d8102d30 msgr2=0x7f29d819a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:36:39.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:39.231+0000 7f29dd200640 1 --2- 172.21.15.5:0/3306121284 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f29d8102d30 0x7f29d819a650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:36:39.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:39.231+0000 7f29dd200640 1 -- 172.21.15.5:0/3306121284 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f29d8103650 msgr2=0x7f29d819ab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:36:39.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:39.231+0000 7f29dd200640 1 --2- 172.21.15.5:0/3306121284 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f29d8103650 0x7f29d819ab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:36:39.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:39.231+0000 7f29dd200640 1 -- 172.21.15.5:0/3306121284 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29d819f600 con 0x7f29d810c160 2024-09-10T09:36:39.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:39.231+0000 7f29cfe00640 1 --2- 172.21.15.5:0/3306121284 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f29d8103650 0x7f29d819ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:36:39.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:39.231+0000 7f29dd200640 1 --2- 172.21.15.5:0/3306121284 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f29d810c160 0x7f29d819ef20 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f29d000eaa0 tx=0x7f29d000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:36:39.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:39.231+0000 7f29d5600640 1 -- 172.21.15.5:0/3306121284 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29d000cdb0 con 0x7f29d810c160 2024-09-10T09:36:39.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:39.231+0000 7f29d5600640 1 -- 172.21.15.5:0/3306121284 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f29d0005020 con 0x7f29d810c160 2024-09-10T09:36:39.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:39.231+0000 7f29d5600640 1 -- 172.21.15.5:0/3306121284 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29d00116b0 con 0x7f29d810c160 2024-09-10T09:36:39.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:39.231+0000 7f29df200640 1 -- 172.21.15.5:0/3306121284 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29d819f8f0 con 0x7f29d810c160 2024-09-10T09:36:39.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:39.232+0000 7f29df200640 1 -- 172.21.15.5:0/3306121284 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29d81a71d0 con 0x7f29d810c160 2024-09-10T09:36:39.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:39.233+0000 7f29df200640 1 -- 172.21.15.5:0/3306121284 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f29d8103110 con 0x7f29d810c160 2024-09-10T09:36:39.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:39.234+0000 7f29d5600640 1 -- 172.21.15.5:0/3306121284 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f29d0005560 con 0x7f29d810c160 2024-09-10T09:36:39.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:39.236+0000 7f29d5600640 1 --2- 172.21.15.5:0/3306121284 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f29bc0a2c90 0x7f29bc0a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:36:39.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:39.236+0000 7f29d5600640 1 -- 172.21.15.5:0/3306121284 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9071+0+0 (secure 0 0 0) 0x7f29d00cf6f0 con 0x7f29d810c160 2024-09-10T09:36:39.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:39.236+0000 7f29d7e00640 1 --2- 172.21.15.5:0/3306121284 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f29bc0a2c90 0x7f29bc0a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:36:39.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:39.237+0000 7f29d7e00640 1 --2- 172.21.15.5:0/3306121284 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f29bc0a2c90 0x7f29bc0a5150 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f29c8009230 tx=0x7f29c80058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:36:39.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:39.240+0000 7f29d5600640 1 -- 172.21.15.5:0/3306121284 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f29d0016030 con 0x7f29d810c160 2024-09-10T09:36:39.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:39.480+0000 7f29df200640 1 -- 172.21.15.5:0/3306121284 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7f29d81a7540 con 0x7f29d810c160 2024-09-10T09:36:39.482 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:36:39 smithi005 ceph-mon[30350]: pgmap v340: 129 pgs: 129 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 2.8 KiB/s wr, 10 op/s 2024-09-10T09:36:39.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:39.488+0000 7f29d5600640 1 -- 172.21.15.5:0/3306121284 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) ==== 74+0+263191 (secure 0 0 0) 0x7f29d009c150 con 0x7f29d810c160 2024-09-10T09:36:39.493 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-10T09:36:39.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:39.492+0000 7f29df200640 1 -- 172.21.15.5:0/3306121284 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f29bc0a2c90 msgr2=0x7f29bc0a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:36:39.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:39.492+0000 7f29df200640 1 --2- 172.21.15.5:0/3306121284 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f29bc0a2c90 0x7f29bc0a5150 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f29c8009230 tx=0x7f29c80058d0 comp rx=0 tx=0).stop 2024-09-10T09:36:39.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:39.492+0000 7f29df200640 1 -- 172.21.15.5:0/3306121284 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f29d810c160 msgr2=0x7f29d819ef20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:36:39.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:39.492+0000 7f29df200640 1 --2- 172.21.15.5:0/3306121284 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f29d810c160 0x7f29d819ef20 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f29d000eaa0 tx=0x7f29d000ef60 comp rx=0 tx=0).stop 2024-09-10T09:36:39.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:39.492+0000 7f29df200640 1 -- 172.21.15.5:0/3306121284 shutdown_connections 2024-09-10T09:36:39.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:39.492+0000 7f29df200640 1 --2- 172.21.15.5:0/3306121284 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f29bc0a2c90 0x7f29bc0a5150 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:36:39.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:39.492+0000 7f29df200640 1 --2- 172.21.15.5:0/3306121284 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f29d810c160 0x7f29d819ef20 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:36:39.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:39.492+0000 7f29df200640 1 --2- 172.21.15.5:0/3306121284 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f29d8103650 0x7f29d819ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:36:39.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:39.492+0000 7f29df200640 1 --2- 172.21.15.5:0/3306121284 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f29d8102d30 0x7f29d819a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:36:39.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:39.492+0000 7f29df200640 1 -- 172.21.15.5:0/3306121284 >> 172.21.15.5:0/3306121284 conn(0x7f29d80fc7d0 msgr2=0x7f29d80fd3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:36:39.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:39.493+0000 7f29df200640 1 -- 172.21.15.5:0/3306121284 shutdown_connections 2024-09-10T09:36:39.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:39.493+0000 7f29df200640 1 -- 172.21.15.5:0/3306121284 wait complete. 2024-09-10T09:36:39.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:36:39 smithi102 ceph-mon[35442]: pgmap v340: 129 pgs: 129 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 2.8 KiB/s wr, 10 op/s 2024-09-10T09:36:39.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:36:39 smithi159 ceph-mon[35513]: pgmap v340: 129 pgs: 129 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 2.8 KiB/s wr, 10 op/s 2024-09-10T09:36:40.253 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":17,"flags":0,"active_gid":14150,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":2707348081},{"type":"v1","addr":"172.21.15.5:6801","nonce":2707348081}]},"active_addr":"172.21.15.5:6801/2707348081","active_change":"2024-09-10T09:28:05.436891+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24109,"name":"z","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24110,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.5:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.5:0","nonce":2690222669}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.5:0","nonce":837438180}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.5:0","nonce":1121510866}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.5:0","nonce":1314353105}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.5:0","nonce":3104520966}]}]} 2024-09-10T09:36:40.260 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-09-10T09:36:40.260 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-09-10T09:36:40.260 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-10T09:36:40.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:36:40 smithi005 ceph-mon[30350]: from='client.? 172.21.15.5:0/3306121284' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-10T09:36:40.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:36:40 smithi102 ceph-mon[35442]: from='client.? 172.21.15.5:0/3306121284' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-10T09:36:40.645 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:36:40.851 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:36:40 smithi159 ceph-mon[35513]: from='client.? 172.21.15.5:0/3306121284' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-10T09:36:41.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:41.226+0000 7f3107600640 1 -- 172.21.15.5:0/935911719 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3100112b90 msgr2=0x7f3100114f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:36:41.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:41.226+0000 7f3107600640 1 --2- 172.21.15.5:0/935911719 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3100112b90 0x7f3100114f80 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f30f8009ca0 tx=0x7f30f802a950 comp rx=0 tx=0).stop 2024-09-10T09:36:41.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:41.226+0000 7f3107600640 1 -- 172.21.15.5:0/935911719 shutdown_connections 2024-09-10T09:36:41.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:41.226+0000 7f3107600640 1 --2- 172.21.15.5:0/935911719 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3100112b90 0x7f3100114f80 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:36:41.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:41.226+0000 7f3107600640 1 --2- 172.21.15.5:0/935911719 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3100074eb0 0x7f3100112650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:36:41.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:41.226+0000 7f3107600640 1 --2- 172.21.15.5:0/935911719 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3100074590 0x7f3100074970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:36:41.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:41.226+0000 7f3107600640 1 -- 172.21.15.5:0/935911719 >> 172.21.15.5:0/935911719 conn(0x7f31000fd8c0 msgr2=0x7f31000ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:36:41.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:41.227+0000 7f3107600640 1 -- 172.21.15.5:0/935911719 shutdown_connections 2024-09-10T09:36:41.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:41.227+0000 7f3107600640 1 -- 172.21.15.5:0/935911719 wait complete. 2024-09-10T09:36:41.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:41.228+0000 7f3107600640 1 Processor -- start 2024-09-10T09:36:41.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:41.228+0000 7f3107600640 1 -- start start 2024-09-10T09:36:41.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:41.228+0000 7f3107600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3100074590 0x7f310019eb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:36:41.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:41.228+0000 7f3107600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3100074eb0 0x7f310019f050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:36:41.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:41.228+0000 7f3107600640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3100112b90 0x7f31001a3410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:36:41.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:41.228+0000 7f3107600640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f31001185c0 con 0x7f3100074eb0 2024-09-10T09:36:41.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:41.228+0000 7f3107600640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f3100118440 con 0x7f3100074590 2024-09-10T09:36:41.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:41.228+0000 7f3107600640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f3100118740 con 0x7f3100112b90 2024-09-10T09:36:41.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:41.228+0000 7f30ffe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3100074eb0 0x7f310019f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:36:41.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:41.228+0000 7f30ffe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3100074eb0 0x7f310019f050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58852/0 (socket says 172.21.15.5:58852) 2024-09-10T09:36:41.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:41.228+0000 7f3104c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3100074590 0x7f310019eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:36:41.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:41.228+0000 7f30ffe00640 1 -- 172.21.15.5:0/2487286885 learned_addr learned my addr 172.21.15.5:0/2487286885 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:36:41.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:41.229+0000 7f3105600640 1 --2- 172.21.15.5:0/2487286885 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3100112b90 0x7f31001a3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:36:41.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:41.229+0000 7f30ffe00640 1 -- 172.21.15.5:0/2487286885 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3100112b90 msgr2=0x7f31001a3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:36:41.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:41.229+0000 7f30ffe00640 1 --2- 172.21.15.5:0/2487286885 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3100112b90 0x7f31001a3410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:36:41.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:41.229+0000 7f30ffe00640 1 -- 172.21.15.5:0/2487286885 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3100074590 msgr2=0x7f310019eb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:36:41.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:41.229+0000 7f30ffe00640 1 --2- 172.21.15.5:0/2487286885 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3100074590 0x7f310019eb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:36:41.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:41.229+0000 7f30ffe00640 1 -- 172.21.15.5:0/2487286885 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31001a3b40 con 0x7f3100074eb0 2024-09-10T09:36:41.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:41.229+0000 7f3104c00640 1 --2- 172.21.15.5:0/2487286885 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3100074590 0x7f310019eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:36:41.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:41.229+0000 7f30ffe00640 1 --2- 172.21.15.5:0/2487286885 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3100074eb0 0x7f310019f050 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f30ec00e9f0 tx=0x7f30ec00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:36:41.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:41.230+0000 7f30fd600640 1 -- 172.21.15.5:0/2487286885 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30ec00cd80 con 0x7f3100074eb0 2024-09-10T09:36:41.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:41.230+0000 7f3107600640 1 -- 172.21.15.5:0/2487286885 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31001a3e30 con 0x7f3100074eb0 2024-09-10T09:36:41.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:41.230+0000 7f30fd600640 1 -- 172.21.15.5:0/2487286885 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f30ec005500 con 0x7f3100074eb0 2024-09-10T09:36:41.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:41.230+0000 7f30fd600640 1 -- 172.21.15.5:0/2487286885 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30ec010690 con 0x7f3100074eb0 2024-09-10T09:36:41.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:41.230+0000 7f3107600640 1 -- 172.21.15.5:0/2487286885 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31001ab6c0 con 0x7f3100074eb0 2024-09-10T09:36:41.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:41.231+0000 7f3107600640 1 -- 172.21.15.5:0/2487286885 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3100074970 con 0x7f3100074eb0 2024-09-10T09:36:41.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:41.233+0000 7f30fd600640 1 -- 172.21.15.5:0/2487286885 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f30ec0046c0 con 0x7f3100074eb0 2024-09-10T09:36:41.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:41.234+0000 7f30fd600640 1 --2- 172.21.15.5:0/2487286885 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f30e80a2c10 0x7f30e80a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:36:41.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:41.234+0000 7f30fd600640 1 -- 172.21.15.5:0/2487286885 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9071+0+0 (secure 0 0 0) 0x7f30ec0cf910 con 0x7f3100074eb0 2024-09-10T09:36:41.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:41.234+0000 7f3104c00640 1 --2- 172.21.15.5:0/2487286885 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f30e80a2c10 0x7f30e80a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:36:41.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:41.235+0000 7f3104c00640 1 --2- 172.21.15.5:0/2487286885 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f30e80a2c10 0x7f30e80a50d0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f30f4009fc0 tx=0x7f30f4005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:36:41.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:41.238+0000 7f30fd600640 1 -- 172.21.15.5:0/2487286885 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f30ec09c370 con 0x7f3100074eb0 2024-09-10T09:36:41.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:41.428+0000 7f3107600640 1 -- 172.21.15.5:0/2487286885 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f310019fe70 con 0x7f3100074eb0 2024-09-10T09:36:41.430 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:36:41 smithi005 ceph-mon[30350]: pgmap v341: 129 pgs: 129 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 2.8 KiB/s wr, 11 op/s 2024-09-10T09:36:41.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:41.430+0000 7f30fd600640 1 -- 172.21.15.5:0/2487286885 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) ==== 74+0+24566 (secure 0 0 0) 0x7f30ec09bc70 con 0x7f3100074eb0 2024-09-10T09:36:41.432 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-10T09:36:41.432 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":94,"fsid":"bade4b50-6f56-11ef-bcea-c7b262605968","created":"2024-09-10T09:27:13.298498+0000","modified":"2024-09-10T09:36:06.755744+0000","last_up_change":"2024-09-10T09:34:30.730066+0000","last_in_change":"2024-09-10T09:34:07.340995+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T09:30:26.042255+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T09:35:24.461031+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T09:35:27.155349+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-10T09:35:35.121462+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"085749a8-e882-4377-92dd-34bb266c0f0b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":954535266},{"type":"v1","addr":"172.21.15.5:6803","nonce":954535266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":954535266},{"type":"v1","addr":"172.21.15.5:6805","nonce":954535266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":954535266},{"type":"v1","addr":"172.21.15.5:6809","nonce":954535266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":954535266},{"type":"v1","addr":"172.21.15.5:6807","nonce":954535266}]},"public_addr":"172.21.15.5:6803/954535266","cluster_addr":"172.21.15.5:6805/954535266","heartbeat_back_addr":"172.21.15.5:6809/954535266","heartbeat_front_addr":"172.21.15.5:6807/954535266","state":["exists","up"]},{"osd":1,"uuid":"0a76c795-2573-4a92-969a-2ec5dd65c094","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6800","nonce":733924375},{"type":"v1","addr":"172.21.15.102:6801","nonce":733924375}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6802","nonce":733924375},{"type":"v1","addr":"172.21.15.102:6803","nonce":733924375}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6806","nonce":733924375},{"type":"v1","addr":"172.21.15.102:6807","nonce":733924375}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6804","nonce":733924375},{"type":"v1","addr":"172.21.15.102:6805","nonce":733924375}]},"public_addr":"172.21.15.102:6801/733924375","cluster_addr":"172.21.15.102:6803/733924375","heartbeat_back_addr":"172.21.15.102:6807/733924375","heartbeat_front_addr":"172.21.15.102:6805/733924375","state":["exists","up"]},{"osd":2,"uuid":"0c98fcb8-60ab-4473-8995-1d9e97c577e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6800","nonce":2236204189},{"type":"v1","addr":"172.21.15.159:6801","nonce":2236204189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":2236204189},{"type":"v1","addr":"172.21.15.159:6803","nonce":2236204189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":2236204189},{"type":"v1","addr":"172.21.15.159:6807","nonce":2236204189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":2236204189},{"type":"v1","addr":"172.21.15.159:6805","nonce":2236204189}]},"public_addr":"172.21.15.159:6801/2236204189","cluster_addr":"172.21.15.159:6803/2236204189","heartbeat_back_addr":"172.21.15.159:6807/2236204189","heartbeat_front_addr":"172.21.15.159:6805/2236204189","state":["exists","up"]},{"osd":3,"uuid":"88eecdeb-5cee-4f78-a3f9-a66bf4bf772a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":2221468725},{"type":"v1","addr":"172.21.15.5:6811","nonce":2221468725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":2221468725},{"type":"v1","addr":"172.21.15.5:6813","nonce":2221468725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6816","nonce":2221468725},{"type":"v1","addr":"172.21.15.5:6817","nonce":2221468725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":2221468725},{"type":"v1","addr":"172.21.15.5:6815","nonce":2221468725}]},"public_addr":"172.21.15.5:6811/2221468725","cluster_addr":"172.21.15.5:6813/2221468725","heartbeat_back_addr":"172.21.15.5:6817/2221468725","heartbeat_front_addr":"172.21.15.5:6815/2221468725","state":["exists","up"]},{"osd":4,"uuid":"f2e7314d-7938-4668-9011-45c175f66b48","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6808","nonce":594465753},{"type":"v1","addr":"172.21.15.102:6809","nonce":594465753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6810","nonce":594465753},{"type":"v1","addr":"172.21.15.102:6811","nonce":594465753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6814","nonce":594465753},{"type":"v1","addr":"172.21.15.102:6815","nonce":594465753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6812","nonce":594465753},{"type":"v1","addr":"172.21.15.102:6813","nonce":594465753}]},"public_addr":"172.21.15.102:6809/594465753","cluster_addr":"172.21.15.102:6811/594465753","heartbeat_back_addr":"172.21.15.102:6815/594465753","heartbeat_front_addr":"172.21.15.102:6813/594465753","state":["exists","up"]},{"osd":5,"uuid":"0c0366d3-658b-4bf0-ba17-aff28a02e948","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":3230280402},{"type":"v1","addr":"172.21.15.159:6809","nonce":3230280402}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":3230280402},{"type":"v1","addr":"172.21.15.159:6811","nonce":3230280402}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":3230280402},{"type":"v1","addr":"172.21.15.159:6815","nonce":3230280402}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":3230280402},{"type":"v1","addr":"172.21.15.159:6813","nonce":3230280402}]},"public_addr":"172.21.15.159:6809/3230280402","cluster_addr":"172.21.15.159:6811/3230280402","heartbeat_back_addr":"172.21.15.159:6815/3230280402","heartbeat_front_addr":"172.21.15.159:6813/3230280402","state":["exists","up"]},{"osd":6,"uuid":"682670a6-51d5-46e8-af6e-dc2d8c1cf407","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":1907651140},{"type":"v1","addr":"172.21.15.5:6819","nonce":1907651140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":1907651140},{"type":"v1","addr":"172.21.15.5:6821","nonce":1907651140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":1907651140},{"type":"v1","addr":"172.21.15.5:6825","nonce":1907651140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":1907651140},{"type":"v1","addr":"172.21.15.5:6823","nonce":1907651140}]},"public_addr":"172.21.15.5:6819/1907651140","cluster_addr":"172.21.15.5:6821/1907651140","heartbeat_back_addr":"172.21.15.5:6825/1907651140","heartbeat_front_addr":"172.21.15.5:6823/1907651140","state":["exists","up"]},{"osd":7,"uuid":"8ae0f4a9-99e7-4563-aa2a-15169684a8b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6816","nonce":692040360},{"type":"v1","addr":"172.21.15.102:6817","nonce":692040360}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6818","nonce":692040360},{"type":"v1","addr":"172.21.15.102:6819","nonce":692040360}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6822","nonce":692040360},{"type":"v1","addr":"172.21.15.102:6823","nonce":692040360}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6820","nonce":692040360},{"type":"v1","addr":"172.21.15.102:6821","nonce":692040360}]},"public_addr":"172.21.15.102:6817/692040360","cluster_addr":"172.21.15.102:6819/692040360","heartbeat_back_addr":"172.21.15.102:6823/692040360","heartbeat_front_addr":"172.21.15.102:6821/692040360","state":["exists","up"]},{"osd":8,"uuid":"e37c845e-ced9-4413-9bbe-bc090537c2bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":2107291799},{"type":"v1","addr":"172.21.15.159:6817","nonce":2107291799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":2107291799},{"type":"v1","addr":"172.21.15.159:6819","nonce":2107291799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":2107291799},{"type":"v1","addr":"172.21.15.159:6823","nonce":2107291799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":2107291799},{"type":"v1","addr":"172.21.15.159:6821","nonce":2107291799}]},"public_addr":"172.21.15.159:6817/2107291799","cluster_addr":"172.21.15.159:6819/2107291799","heartbeat_back_addr":"172.21.15.159:6823/2107291799","heartbeat_front_addr":"172.21.15.159:6821/2107291799","state":["exists","up"]},{"osd":9,"uuid":"91a5b6cc-08da-47be-af12-d0896a849309","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":902431281},{"type":"v1","addr":"172.21.15.5:6827","nonce":902431281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":902431281},{"type":"v1","addr":"172.21.15.5:6829","nonce":902431281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":902431281},{"type":"v1","addr":"172.21.15.5:6833","nonce":902431281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":902431281},{"type":"v1","addr":"172.21.15.5:6831","nonce":902431281}]},"public_addr":"172.21.15.5:6827/902431281","cluster_addr":"172.21.15.5:6829/902431281","heartbeat_back_addr":"172.21.15.5:6833/902431281","heartbeat_front_addr":"172.21.15.5:6831/902431281","state":["exists","up"]},{"osd":10,"uuid":"2211030c-b00e-402b-b4d7-c587eb29a490","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6824","nonce":2853806211},{"type":"v1","addr":"172.21.15.102:6825","nonce":2853806211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6826","nonce":2853806211},{"type":"v1","addr":"172.21.15.102:6827","nonce":2853806211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6830","nonce":2853806211},{"type":"v1","addr":"172.21.15.102:6831","nonce":2853806211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6828","nonce":2853806211},{"type":"v1","addr":"172.21.15.102:6829","nonce":2853806211}]},"public_addr":"172.21.15.102:6825/2853806211","cluster_addr":"172.21.15.102:6827/2853806211","heartbeat_back_addr":"172.21.15.102:6831/2853806211","heartbeat_front_addr":"172.21.15.102:6829/2853806211","state":["exists","up"]},{"osd":11,"uuid":"4e9699ec-8ff2-4005-9d84-af5600b3a2ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6824","nonce":969941218},{"type":"v1","addr":"172.21.15.159:6825","nonce":969941218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":969941218},{"type":"v1","addr":"172.21.15.159:6827","nonce":969941218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6830","nonce":969941218},{"type":"v1","addr":"172.21.15.159:6831","nonce":969941218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":969941218},{"type":"v1","addr":"172.21.15.159:6829","nonce":969941218}]},"public_addr":"172.21.15.159:6825/969941218","cluster_addr":"172.21.15.159:6827/969941218","heartbeat_back_addr":"172.21.15.159:6831/969941218","heartbeat_front_addr":"172.21.15.159:6829/969941218","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:29:34.416490+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:30:00.246101+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:30:24.300464+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:30:54.663623+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:31:21.426292+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:31:46.738914+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:32:16.196571+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:32:42.549243+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:33:08.178651+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:33:37.477911+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:34:03.114026+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:34:28.891924+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.0","mappings":[{"from":4,"to":8}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":6}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9},{"from":5,"to":7}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]},{"pgid":"4.8","mappings":[{"from":5,"to":8}]},{"pgid":"4.c","mappings":[{"from":9,"to":10}]},{"pgid":"4.18","mappings":[{"from":9,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.5:6801/1527422686":"2024-09-11T09:28:05.436742+0000","172.21.15.5:0/2276144345":"2024-09-11T09:28:05.436742+0000","172.21.15.5:0/412040104":"2024-09-11T09:28:05.436742+0000","172.21.15.5:6800/1527422686":"2024-09-11T09:28:05.436742+0000","172.21.15.5:0/1856581170":"2024-09-11T09:28:05.436742+0000","172.21.15.5:0/2847668298":"2024-09-11T09:27:37.778500+0000","172.21.15.5:6801/14425913":"2024-09-11T09:27:37.778500+0000","172.21.15.5:0/166910782":"2024-09-11T09:27:37.778500+0000","172.21.15.5:6800/14425913":"2024-09-11T09:27:37.778500+0000","172.21.15.5:0/1711858179":"2024-09-11T09:27:37.778500+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T09:36:41.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:41.432+0000 7f3107600640 1 -- 172.21.15.5:0/2487286885 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f30e80a2c10 msgr2=0x7f30e80a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:36:41.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:41.432+0000 7f3107600640 1 --2- 172.21.15.5:0/2487286885 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f30e80a2c10 0x7f30e80a50d0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f30f4009fc0 tx=0x7f30f4005f90 comp rx=0 tx=0).stop 2024-09-10T09:36:41.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:41.432+0000 7f3107600640 1 -- 172.21.15.5:0/2487286885 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3100074eb0 msgr2=0x7f310019f050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:36:41.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:41.432+0000 7f3107600640 1 --2- 172.21.15.5:0/2487286885 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3100074eb0 0x7f310019f050 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f30ec00e9f0 tx=0x7f30ec00edf0 comp rx=0 tx=0).stop 2024-09-10T09:36:41.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:41.432+0000 7f3107600640 1 -- 172.21.15.5:0/2487286885 shutdown_connections 2024-09-10T09:36:41.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:41.432+0000 7f3107600640 1 --2- 172.21.15.5:0/2487286885 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f30e80a2c10 0x7f30e80a50d0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:36:41.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:41.432+0000 7f3107600640 1 --2- 172.21.15.5:0/2487286885 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3100112b90 0x7f31001a3410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:36:41.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:41.432+0000 7f3107600640 1 --2- 172.21.15.5:0/2487286885 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3100074eb0 0x7f310019f050 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:36:41.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:41.432+0000 7f3107600640 1 --2- 172.21.15.5:0/2487286885 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3100074590 0x7f310019eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:36:41.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:41.432+0000 7f3107600640 1 -- 172.21.15.5:0/2487286885 >> 172.21.15.5:0/2487286885 conn(0x7f31000fd8c0 msgr2=0x7f31000ff050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:36:41.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:41.433+0000 7f3107600640 1 -- 172.21.15.5:0/2487286885 shutdown_connections 2024-09-10T09:36:41.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:41.433+0000 7f3107600640 1 -- 172.21.15.5:0/2487286885 wait complete. 2024-09-10T09:36:41.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:36:41 smithi102 ceph-mon[35442]: pgmap v341: 129 pgs: 129 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 2.8 KiB/s wr, 11 op/s 2024-09-10T09:36:41.852 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:36:41 smithi159 ceph-mon[35513]: pgmap v341: 129 pgs: 129 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 2.8 KiB/s wr, 11 op/s 2024-09-10T09:36:42.189 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-09-10T09:36:42.189 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-10T09:36:42.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:36:42 smithi005 ceph-mon[30350]: from='client.? 172.21.15.5:0/2487286885' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T09:36:42.567 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:36:42.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:36:42 smithi102 ceph-mon[35442]: from='client.? 172.21.15.5:0/2487286885' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T09:36:42.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:36:42 smithi159 ceph-mon[35513]: from='client.? 172.21.15.5:0/2487286885' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T09:36:43.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:43.183+0000 7f92b2000640 1 -- 172.21.15.5:0/4204851378 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f92ac1019c0 msgr2=0x7f92ac101e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:36:43.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:43.183+0000 7f92b2000640 1 --2- 172.21.15.5:0/4204851378 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f92ac1019c0 0x7f92ac101e20 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f9298009930 tx=0x7f929802a420 comp rx=0 tx=0).stop 2024-09-10T09:36:43.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:43.183+0000 7f92b2000640 1 -- 172.21.15.5:0/4204851378 shutdown_connections 2024-09-10T09:36:43.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:43.183+0000 7f92b2000640 1 --2- 172.21.15.5:0/4204851378 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f92ac102360 0x7f92ac10e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:36:43.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:43.183+0000 7f92b2000640 1 --2- 172.21.15.5:0/4204851378 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f92ac1019c0 0x7f92ac101e20 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:36:43.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:43.183+0000 7f92b2000640 1 --2- 172.21.15.5:0/4204851378 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f92ac1079c0 0x7f92ac107da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:36:43.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:43.183+0000 7f92b2000640 1 -- 172.21.15.5:0/4204851378 >> 172.21.15.5:0/4204851378 conn(0x7f92ac0fd740 msgr2=0x7f92ac0ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:36:43.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:43.183+0000 7f92b2000640 1 -- 172.21.15.5:0/4204851378 shutdown_connections 2024-09-10T09:36:43.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:43.183+0000 7f92b2000640 1 -- 172.21.15.5:0/4204851378 wait complete. 2024-09-10T09:36:43.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:43.184+0000 7f92b2000640 1 Processor -- start 2024-09-10T09:36:43.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:43.184+0000 7f92b2000640 1 -- start start 2024-09-10T09:36:43.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:43.184+0000 7f92b2000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f92ac1019c0 0x7f92ac19a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:36:43.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:43.184+0000 7f92b2000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f92ac102360 0x7f92ac19a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:36:43.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:43.184+0000 7f92b2000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f92ac1079c0 0x7f92ac19ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:36:43.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:43.184+0000 7f92b2000640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f92ac111dc0 con 0x7f92ac1019c0 2024-09-10T09:36:43.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:43.184+0000 7f92b2000640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f92ac111c40 con 0x7f92ac102360 2024-09-10T09:36:43.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:43.184+0000 7f92b2000640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f92ac111f40 con 0x7f92ac1079c0 2024-09-10T09:36:43.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:43.185+0000 7f92b0c00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f92ac1019c0 0x7f92ac19a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:36:43.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:43.185+0000 7f92b0c00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f92ac1019c0 0x7f92ac19a480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58872/0 (socket says 172.21.15.5:58872) 2024-09-10T09:36:43.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:43.185+0000 7f92b0c00640 1 -- 172.21.15.5:0/2027784864 learned_addr learned my addr 172.21.15.5:0/2027784864 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:36:43.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:43.185+0000 7f92abe00640 1 --2- 172.21.15.5:0/2027784864 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f92ac102360 0x7f92ac19a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:36:43.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:43.185+0000 7f92b1600640 1 --2- 172.21.15.5:0/2027784864 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f92ac1079c0 0x7f92ac19ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:36:43.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:43.185+0000 7f92b0c00640 1 -- 172.21.15.5:0/2027784864 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f92ac1079c0 msgr2=0x7f92ac19ed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:36:43.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:43.185+0000 7f92b0c00640 1 --2- 172.21.15.5:0/2027784864 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f92ac1079c0 0x7f92ac19ed50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:36:43.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:43.185+0000 7f92b0c00640 1 -- 172.21.15.5:0/2027784864 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f92ac102360 msgr2=0x7f92ac19a9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:36:43.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:43.185+0000 7f92b0c00640 1 --2- 172.21.15.5:0/2027784864 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f92ac102360 0x7f92ac19a9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:36:43.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:43.185+0000 7f92b0c00640 1 -- 172.21.15.5:0/2027784864 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92ac19f430 con 0x7f92ac1019c0 2024-09-10T09:36:43.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:43.186+0000 7f92b0c00640 1 --2- 172.21.15.5:0/2027784864 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f92ac1019c0 0x7f92ac19a480 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f92a000d950 tx=0x7f92a000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:36:43.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:43.186+0000 7f92a9600640 1 -- 172.21.15.5:0/2027784864 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92a0017030 con 0x7f92ac1019c0 2024-09-10T09:36:43.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:43.186+0000 7f92a9600640 1 -- 172.21.15.5:0/2027784864 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f92a0004020 con 0x7f92ac1019c0 2024-09-10T09:36:43.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:43.186+0000 7f92a9600640 1 -- 172.21.15.5:0/2027784864 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92a00047a0 con 0x7f92ac1019c0 2024-09-10T09:36:43.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:43.186+0000 7f92b2000640 1 -- 172.21.15.5:0/2027784864 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92ac19f720 con 0x7f92ac1019c0 2024-09-10T09:36:43.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:43.186+0000 7f92b2000640 1 -- 172.21.15.5:0/2027784864 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92ac1a7000 con 0x7f92ac1019c0 2024-09-10T09:36:43.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:43.187+0000 7f92b2000640 1 -- 172.21.15.5:0/2027784864 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f92ac101e20 con 0x7f92ac1019c0 2024-09-10T09:36:43.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:43.189+0000 7f92a9600640 1 -- 172.21.15.5:0/2027784864 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f92a00041d0 con 0x7f92ac1019c0 2024-09-10T09:36:43.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:43.190+0000 7f92a9600640 1 --2- 172.21.15.5:0/2027784864 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f92940a2c90 0x7f92940a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:36:43.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:43.190+0000 7f92a9600640 1 -- 172.21.15.5:0/2027784864 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9071+0+0 (secure 0 0 0) 0x7f92a00cedd0 con 0x7f92ac1019c0 2024-09-10T09:36:43.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:43.191+0000 7f92abe00640 1 --2- 172.21.15.5:0/2027784864 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f92940a2c90 0x7f92940a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:36:43.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:43.191+0000 7f92abe00640 1 --2- 172.21.15.5:0/2027784864 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f92940a2c90 0x7f92940a5150 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f9298009900 tx=0x7f92980058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:36:43.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:43.194+0000 7f92a9600640 1 -- 172.21.15.5:0/2027784864 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f92a009b830 con 0x7f92ac1019c0 2024-09-10T09:36:43.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:43.383+0000 7f92b2000640 1 -- 172.21.15.5:0/2027784864 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f92ac1a7260 con 0x7f92ac1019c0 2024-09-10T09:36:43.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:43.384+0000 7f92a9600640 1 -- 172.21.15.5:0/2027784864 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) ==== 74+0+24566 (secure 0 0 0) 0x7f92a009b130 con 0x7f92ac1019c0 2024-09-10T09:36:43.386 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-10T09:36:43.387 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":94,"fsid":"bade4b50-6f56-11ef-bcea-c7b262605968","created":"2024-09-10T09:27:13.298498+0000","modified":"2024-09-10T09:36:06.755744+0000","last_up_change":"2024-09-10T09:34:30.730066+0000","last_in_change":"2024-09-10T09:34:07.340995+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T09:30:26.042255+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T09:35:24.461031+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T09:35:27.155349+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-10T09:35:35.121462+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"085749a8-e882-4377-92dd-34bb266c0f0b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":954535266},{"type":"v1","addr":"172.21.15.5:6803","nonce":954535266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":954535266},{"type":"v1","addr":"172.21.15.5:6805","nonce":954535266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":954535266},{"type":"v1","addr":"172.21.15.5:6809","nonce":954535266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":954535266},{"type":"v1","addr":"172.21.15.5:6807","nonce":954535266}]},"public_addr":"172.21.15.5:6803/954535266","cluster_addr":"172.21.15.5:6805/954535266","heartbeat_back_addr":"172.21.15.5:6809/954535266","heartbeat_front_addr":"172.21.15.5:6807/954535266","state":["exists","up"]},{"osd":1,"uuid":"0a76c795-2573-4a92-969a-2ec5dd65c094","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6800","nonce":733924375},{"type":"v1","addr":"172.21.15.102:6801","nonce":733924375}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6802","nonce":733924375},{"type":"v1","addr":"172.21.15.102:6803","nonce":733924375}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6806","nonce":733924375},{"type":"v1","addr":"172.21.15.102:6807","nonce":733924375}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6804","nonce":733924375},{"type":"v1","addr":"172.21.15.102:6805","nonce":733924375}]},"public_addr":"172.21.15.102:6801/733924375","cluster_addr":"172.21.15.102:6803/733924375","heartbeat_back_addr":"172.21.15.102:6807/733924375","heartbeat_front_addr":"172.21.15.102:6805/733924375","state":["exists","up"]},{"osd":2,"uuid":"0c98fcb8-60ab-4473-8995-1d9e97c577e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6800","nonce":2236204189},{"type":"v1","addr":"172.21.15.159:6801","nonce":2236204189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":2236204189},{"type":"v1","addr":"172.21.15.159:6803","nonce":2236204189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":2236204189},{"type":"v1","addr":"172.21.15.159:6807","nonce":2236204189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":2236204189},{"type":"v1","addr":"172.21.15.159:6805","nonce":2236204189}]},"public_addr":"172.21.15.159:6801/2236204189","cluster_addr":"172.21.15.159:6803/2236204189","heartbeat_back_addr":"172.21.15.159:6807/2236204189","heartbeat_front_addr":"172.21.15.159:6805/2236204189","state":["exists","up"]},{"osd":3,"uuid":"88eecdeb-5cee-4f78-a3f9-a66bf4bf772a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":2221468725},{"type":"v1","addr":"172.21.15.5:6811","nonce":2221468725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":2221468725},{"type":"v1","addr":"172.21.15.5:6813","nonce":2221468725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6816","nonce":2221468725},{"type":"v1","addr":"172.21.15.5:6817","nonce":2221468725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":2221468725},{"type":"v1","addr":"172.21.15.5:6815","nonce":2221468725}]},"public_addr":"172.21.15.5:6811/2221468725","cluster_addr":"172.21.15.5:6813/2221468725","heartbeat_back_addr":"172.21.15.5:6817/2221468725","heartbeat_front_addr":"172.21.15.5:6815/2221468725","state":["exists","up"]},{"osd":4,"uuid":"f2e7314d-7938-4668-9011-45c175f66b48","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6808","nonce":594465753},{"type":"v1","addr":"172.21.15.102:6809","nonce":594465753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6810","nonce":594465753},{"type":"v1","addr":"172.21.15.102:6811","nonce":594465753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6814","nonce":594465753},{"type":"v1","addr":"172.21.15.102:6815","nonce":594465753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6812","nonce":594465753},{"type":"v1","addr":"172.21.15.102:6813","nonce":594465753}]},"public_addr":"172.21.15.102:6809/594465753","cluster_addr":"172.21.15.102:6811/594465753","heartbeat_back_addr":"172.21.15.102:6815/594465753","heartbeat_front_addr":"172.21.15.102:6813/594465753","state":["exists","up"]},{"osd":5,"uuid":"0c0366d3-658b-4bf0-ba17-aff28a02e948","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":3230280402},{"type":"v1","addr":"172.21.15.159:6809","nonce":3230280402}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":3230280402},{"type":"v1","addr":"172.21.15.159:6811","nonce":3230280402}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":3230280402},{"type":"v1","addr":"172.21.15.159:6815","nonce":3230280402}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":3230280402},{"type":"v1","addr":"172.21.15.159:6813","nonce":3230280402}]},"public_addr":"172.21.15.159:6809/3230280402","cluster_addr":"172.21.15.159:6811/3230280402","heartbeat_back_addr":"172.21.15.159:6815/3230280402","heartbeat_front_addr":"172.21.15.159:6813/3230280402","state":["exists","up"]},{"osd":6,"uuid":"682670a6-51d5-46e8-af6e-dc2d8c1cf407","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":1907651140},{"type":"v1","addr":"172.21.15.5:6819","nonce":1907651140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":1907651140},{"type":"v1","addr":"172.21.15.5:6821","nonce":1907651140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":1907651140},{"type":"v1","addr":"172.21.15.5:6825","nonce":1907651140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":1907651140},{"type":"v1","addr":"172.21.15.5:6823","nonce":1907651140}]},"public_addr":"172.21.15.5:6819/1907651140","cluster_addr":"172.21.15.5:6821/1907651140","heartbeat_back_addr":"172.21.15.5:6825/1907651140","heartbeat_front_addr":"172.21.15.5:6823/1907651140","state":["exists","up"]},{"osd":7,"uuid":"8ae0f4a9-99e7-4563-aa2a-15169684a8b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6816","nonce":692040360},{"type":"v1","addr":"172.21.15.102:6817","nonce":692040360}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6818","nonce":692040360},{"type":"v1","addr":"172.21.15.102:6819","nonce":692040360}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6822","nonce":692040360},{"type":"v1","addr":"172.21.15.102:6823","nonce":692040360}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6820","nonce":692040360},{"type":"v1","addr":"172.21.15.102:6821","nonce":692040360}]},"public_addr":"172.21.15.102:6817/692040360","cluster_addr":"172.21.15.102:6819/692040360","heartbeat_back_addr":"172.21.15.102:6823/692040360","heartbeat_front_addr":"172.21.15.102:6821/692040360","state":["exists","up"]},{"osd":8,"uuid":"e37c845e-ced9-4413-9bbe-bc090537c2bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":2107291799},{"type":"v1","addr":"172.21.15.159:6817","nonce":2107291799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":2107291799},{"type":"v1","addr":"172.21.15.159:6819","nonce":2107291799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":2107291799},{"type":"v1","addr":"172.21.15.159:6823","nonce":2107291799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":2107291799},{"type":"v1","addr":"172.21.15.159:6821","nonce":2107291799}]},"public_addr":"172.21.15.159:6817/2107291799","cluster_addr":"172.21.15.159:6819/2107291799","heartbeat_back_addr":"172.21.15.159:6823/2107291799","heartbeat_front_addr":"172.21.15.159:6821/2107291799","state":["exists","up"]},{"osd":9,"uuid":"91a5b6cc-08da-47be-af12-d0896a849309","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":902431281},{"type":"v1","addr":"172.21.15.5:6827","nonce":902431281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":902431281},{"type":"v1","addr":"172.21.15.5:6829","nonce":902431281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":902431281},{"type":"v1","addr":"172.21.15.5:6833","nonce":902431281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":902431281},{"type":"v1","addr":"172.21.15.5:6831","nonce":902431281}]},"public_addr":"172.21.15.5:6827/902431281","cluster_addr":"172.21.15.5:6829/902431281","heartbeat_back_addr":"172.21.15.5:6833/902431281","heartbeat_front_addr":"172.21.15.5:6831/902431281","state":["exists","up"]},{"osd":10,"uuid":"2211030c-b00e-402b-b4d7-c587eb29a490","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6824","nonce":2853806211},{"type":"v1","addr":"172.21.15.102:6825","nonce":2853806211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6826","nonce":2853806211},{"type":"v1","addr":"172.21.15.102:6827","nonce":2853806211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6830","nonce":2853806211},{"type":"v1","addr":"172.21.15.102:6831","nonce":2853806211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6828","nonce":2853806211},{"type":"v1","addr":"172.21.15.102:6829","nonce":2853806211}]},"public_addr":"172.21.15.102:6825/2853806211","cluster_addr":"172.21.15.102:6827/2853806211","heartbeat_back_addr":"172.21.15.102:6831/2853806211","heartbeat_front_addr":"172.21.15.102:6829/2853806211","state":["exists","up"]},{"osd":11,"uuid":"4e9699ec-8ff2-4005-9d84-af5600b3a2ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6824","nonce":969941218},{"type":"v1","addr":"172.21.15.159:6825","nonce":969941218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":969941218},{"type":"v1","addr":"172.21.15.159:6827","nonce":969941218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6830","nonce":969941218},{"type":"v1","addr":"172.21.15.159:6831","nonce":969941218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":969941218},{"type":"v1","addr":"172.21.15.159:6829","nonce":969941218}]},"public_addr":"172.21.15.159:6825/969941218","cluster_addr":"172.21.15.159:6827/969941218","heartbeat_back_addr":"172.21.15.159:6831/969941218","heartbeat_front_addr":"172.21.15.159:6829/969941218","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:29:34.416490+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:30:00.246101+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:30:24.300464+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:30:54.663623+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:31:21.426292+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:31:46.738914+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:32:16.196571+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:32:42.549243+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:33:08.178651+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:33:37.477911+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:34:03.114026+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:34:28.891924+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.0","mappings":[{"from":4,"to":8}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":6}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9},{"from":5,"to":7}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]},{"pgid":"4.8","mappings":[{"from":5,"to":8}]},{"pgid":"4.c","mappings":[{"from":9,"to":10}]},{"pgid":"4.18","mappings":[{"from":9,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.5:6801/1527422686":"2024-09-11T09:28:05.436742+0000","172.21.15.5:0/2276144345":"2024-09-11T09:28:05.436742+0000","172.21.15.5:0/412040104":"2024-09-11T09:28:05.436742+0000","172.21.15.5:6800/1527422686":"2024-09-11T09:28:05.436742+0000","172.21.15.5:0/1856581170":"2024-09-11T09:28:05.436742+0000","172.21.15.5:0/2847668298":"2024-09-11T09:27:37.778500+0000","172.21.15.5:6801/14425913":"2024-09-11T09:27:37.778500+0000","172.21.15.5:0/166910782":"2024-09-11T09:27:37.778500+0000","172.21.15.5:6800/14425913":"2024-09-11T09:27:37.778500+0000","172.21.15.5:0/1711858179":"2024-09-11T09:27:37.778500+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T09:36:43.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:43.387+0000 7f92b2000640 1 -- 172.21.15.5:0/2027784864 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f92940a2c90 msgr2=0x7f92940a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:36:43.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:43.387+0000 7f92b2000640 1 --2- 172.21.15.5:0/2027784864 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f92940a2c90 0x7f92940a5150 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f9298009900 tx=0x7f92980058d0 comp rx=0 tx=0).stop 2024-09-10T09:36:43.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:43.387+0000 7f92b2000640 1 -- 172.21.15.5:0/2027784864 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f92ac1019c0 msgr2=0x7f92ac19a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:36:43.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:43.387+0000 7f92b2000640 1 --2- 172.21.15.5:0/2027784864 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f92ac1019c0 0x7f92ac19a480 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f92a000d950 tx=0x7f92a000de10 comp rx=0 tx=0).stop 2024-09-10T09:36:43.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:43.387+0000 7f92b2000640 1 -- 172.21.15.5:0/2027784864 shutdown_connections 2024-09-10T09:36:43.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:43.387+0000 7f92b2000640 1 --2- 172.21.15.5:0/2027784864 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f92940a2c90 0x7f92940a5150 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:36:43.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:43.387+0000 7f92b2000640 1 --2- 172.21.15.5:0/2027784864 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f92ac1079c0 0x7f92ac19ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:36:43.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:43.387+0000 7f92b2000640 1 --2- 172.21.15.5:0/2027784864 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f92ac102360 0x7f92ac19a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:36:43.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:43.387+0000 7f92b2000640 1 --2- 172.21.15.5:0/2027784864 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f92ac1019c0 0x7f92ac19a480 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:36:43.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:43.387+0000 7f92b2000640 1 -- 172.21.15.5:0/2027784864 >> 172.21.15.5:0/2027784864 conn(0x7f92ac0fd740 msgr2=0x7f92ac10cf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:36:43.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:43.387+0000 7f92b2000640 1 -- 172.21.15.5:0/2027784864 shutdown_connections 2024-09-10T09:36:43.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:43.387+0000 7f92b2000640 1 -- 172.21.15.5:0/2027784864 wait complete. 2024-09-10T09:36:43.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:36:43 smithi102 ceph-mon[35442]: pgmap v342: 129 pgs: 129 active+clean; 645 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 3.4 KiB/s wr, 13 op/s 2024-09-10T09:36:43.675 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:36:43 smithi005 ceph-mon[30350]: pgmap v342: 129 pgs: 129 active+clean; 645 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 3.4 KiB/s wr, 13 op/s 2024-09-10T09:36:43.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:36:43 smithi159 ceph-mon[35513]: pgmap v342: 129 pgs: 129 active+clean; 645 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 3.4 KiB/s wr, 13 op/s 2024-09-10T09:36:44.167 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-09-10T09:36:44.168 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-09-10T09:36:44.168 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-09-10T09:36:44.168 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-09-10T09:36:44.169 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-09-10T09:36:44.169 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-09-10T09:36:44.170 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-09-10T09:36:44.170 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-09-10T09:36:44.171 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph tell osd.8 flush_pg_stats 2024-09-10T09:36:44.171 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph tell osd.9 flush_pg_stats 2024-09-10T09:36:44.172 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph tell osd.10 flush_pg_stats 2024-09-10T09:36:44.172 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph tell osd.11 flush_pg_stats 2024-09-10T09:36:44.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:36:44 smithi102 ceph-mon[35442]: from='client.? 172.21.15.5:0/2027784864' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T09:36:44.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:36:44 smithi005 ceph-mon[30350]: from='client.? 172.21.15.5:0/2027784864' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T09:36:44.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:36:44 smithi159 ceph-mon[35513]: from='client.? 172.21.15.5:0/2027784864' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T09:36:45.179 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:36:45.214 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:36:45.222 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:36:45.240 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:36:45.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:36:45 smithi102 ceph-mon[35442]: pgmap v343: 129 pgs: 129 active+clean; 645 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.1 KiB/s wr, 9 op/s 2024-09-10T09:36:45.710 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:36:45 smithi005 ceph-mon[30350]: pgmap v343: 129 pgs: 129 active+clean; 645 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.1 KiB/s wr, 9 op/s 2024-09-10T09:36:45.723 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:36:45.724 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:36:45.724 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:36:45.732 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:36:45.732 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:36:45.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:36:45 smithi159 ceph-mon[35513]: pgmap v343: 129 pgs: 129 active+clean; 645 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.1 KiB/s wr, 9 op/s 2024-09-10T09:36:46.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:46.171+0000 7f355d400640 1 -- 172.21.15.5:0/2926977698 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3558103e30 msgr2=0x7f3558104210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:36:46.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:46.171+0000 7f355d400640 1 --2- 172.21.15.5:0/2926977698 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3558103e30 0x7f3558104210 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f3548009900 tx=0x7f354802a410 comp rx=0 tx=0).stop 2024-09-10T09:36:46.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:46.171+0000 7f355d400640 1 -- 172.21.15.5:0/2926977698 shutdown_connections 2024-09-10T09:36:46.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:46.171+0000 7f355d400640 1 --2- 172.21.15.5:0/2926977698 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3558108f60 0x7f3558110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:36:46.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:46.171+0000 7f355d400640 1 --2- 172.21.15.5:0/2926977698 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f35581047e0 0x7f3558108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:36:46.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:46.171+0000 7f355d400640 1 --2- 172.21.15.5:0/2926977698 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3558103e30 0x7f3558104210 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:36:46.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:46.171+0000 7f355d400640 1 -- 172.21.15.5:0/2926977698 >> 172.21.15.5:0/2926977698 conn(0x7f35580ff8d0 msgr2=0x7f3558101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:36:46.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:46.171+0000 7f355d400640 1 -- 172.21.15.5:0/2926977698 shutdown_connections 2024-09-10T09:36:46.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:46.171+0000 7f355d400640 1 -- 172.21.15.5:0/2926977698 wait complete. 2024-09-10T09:36:46.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:46.172+0000 7f355d400640 1 Processor -- start 2024-09-10T09:36:46.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:46.172+0000 7f355d400640 1 -- start start 2024-09-10T09:36:46.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:46.172+0000 7f355d400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3558103e30 0x7f355819a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:36:46.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:46.173+0000 7f355d400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f35581047e0 0x7f355819a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:36:46.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:46.173+0000 7f355d400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3558108f60 0x7f355819ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:36:46.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:46.173+0000 7f355d400640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f3558114010 con 0x7f35581047e0 2024-09-10T09:36:46.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:46.173+0000 7f355d400640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f3558113e90 con 0x7f3558103e30 2024-09-10T09:36:46.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:46.173+0000 7f355d400640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f3558114190 con 0x7f3558108f60 2024-09-10T09:36:46.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:46.173+0000 7f3557400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f35581047e0 0x7f355819a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:36:46.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:46.173+0000 7f3557400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f35581047e0 0x7f355819a9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58898/0 (socket says 172.21.15.5:58898) 2024-09-10T09:36:46.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:46.173+0000 7f3557400640 1 -- 172.21.15.5:0/2097000938 learned_addr learned my addr 172.21.15.5:0/2097000938 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:36:46.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:46.173+0000 7f3557e00640 1 --2- 172.21.15.5:0/2097000938 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3558103e30 0x7f355819a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:36:46.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:46.173+0000 7f355ca00640 1 --2- 172.21.15.5:0/2097000938 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3558108f60 0x7f355819ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:36:46.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:46.173+0000 7f3557e00640 1 -- 172.21.15.5:0/2097000938 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3558108f60 msgr2=0x7f355819ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:36:46.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:46.173+0000 7f3557e00640 1 --2- 172.21.15.5:0/2097000938 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3558108f60 0x7f355819ed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:36:46.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:46.173+0000 7f3557e00640 1 -- 172.21.15.5:0/2097000938 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f35581047e0 msgr2=0x7f355819a9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:36:46.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:46.173+0000 7f3557e00640 1 --2- 172.21.15.5:0/2097000938 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f35581047e0 0x7f355819a9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:36:46.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:46.173+0000 7f3557e00640 1 -- 172.21.15.5:0/2097000938 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f355819f440 con 0x7f3558103e30 2024-09-10T09:36:46.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:46.173+0000 7f3557400640 1 --2- 172.21.15.5:0/2097000938 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f35581047e0 0x7f355819a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:36:46.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:46.174+0000 7f3557e00640 1 --2- 172.21.15.5:0/2097000938 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3558103e30 0x7f355819a490 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f354802a930 tx=0x7f3548003b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:36:46.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:46.174+0000 7f3554c00640 1 -- 172.21.15.5:0/2097000938 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f354803f070 con 0x7f3558103e30 2024-09-10T09:36:46.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:46.174+0000 7f355d400640 1 -- 172.21.15.5:0/2097000938 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f355819f6d0 con 0x7f3558103e30 2024-09-10T09:36:46.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:46.174+0000 7f3554c00640 1 -- 172.21.15.5:0/2097000938 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3548031c90 con 0x7f3558103e30 2024-09-10T09:36:46.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:46.174+0000 7f3554c00640 1 -- 172.21.15.5:0/2097000938 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3548035030 con 0x7f3558103e30 2024-09-10T09:36:46.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:46.174+0000 7f355d400640 1 -- 172.21.15.5:0/2097000938 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35581a7010 con 0x7f3558103e30 2024-09-10T09:36:46.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:46.175+0000 7f355d400640 1 -- 172.21.15.5:0/2097000938 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f3558062940 con 0x7f3558103e30 2024-09-10T09:36:46.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:46.178+0000 7f3554c00640 1 -- 172.21.15.5:0/2097000938 <== mon.2 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f3548042070 con 0x7f3558103e30 2024-09-10T09:36:46.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:46.179+0000 7f3554c00640 1 --2- 172.21.15.5:0/2097000938 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f353c0a2c10 0x7f353c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:36:46.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:46.179+0000 7f3554c00640 1 -- 172.21.15.5:0/2097000938 <== mon.2 v2:172.21.15.102:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9071+0+0 (secure 0 0 0) 0x7f35480f0520 con 0x7f3558103e30 2024-09-10T09:36:46.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:46.180+0000 7f3557400640 1 --2- 172.21.15.5:0/2097000938 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f353c0a2c10 0x7f353c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:36:46.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:46.180+0000 7f3554c00640 1 --2- 172.21.15.5:0/2097000938 >> [v2:172.21.15.102:6824/2853806211,v1:172.21.15.102:6825/2853806211] conn(0x7f353c0add60 0x7f353c0b0200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:36:46.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:46.180+0000 7f3554c00640 1 -- 172.21.15.5:0/2097000938 --> [v2:172.21.15.102:6824/2853806211,v1:172.21.15.102:6825/2853806211] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f354800b0b0 con 0x7f353c0add60 2024-09-10T09:36:46.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:46.180+0000 7f3554c00640 1 -- 172.21.15.5:0/2097000938 <== mon.2 v2:172.21.15.102:3300/0 6 ==== mon_get_version_reply(handle=1 version=94) ==== 24+0+0 (secure 0 0 0) 0x7f35480f08d0 con 0x7f3558103e30 2024-09-10T09:36:46.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:46.180+0000 7f355ca00640 1 --2- 172.21.15.5:0/2097000938 >> [v2:172.21.15.102:6824/2853806211,v1:172.21.15.102:6825/2853806211] conn(0x7f353c0add60 0x7f353c0b0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:36:46.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:46.180+0000 7f3557400640 1 --2- 172.21.15.5:0/2097000938 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f353c0a2c10 0x7f353c0a50d0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f35580ffcf0 tx=0x7f354400b4a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:36:46.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:46.181+0000 7f355ca00640 1 --2- 172.21.15.5:0/2097000938 >> [v2:172.21.15.102:6824/2853806211,v1:172.21.15.102:6825/2853806211] conn(0x7f353c0add60 0x7f353c0b0200 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:36:46.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:46.182+0000 7f3554c00640 1 -- 172.21.15.5:0/2097000938 <== osd.10 v2:172.21.15.102:6824/2853806211 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f354800b0b0 con 0x7f353c0add60 2024-09-10T09:36:46.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:46.196+0000 7f355d400640 1 -- 172.21.15.5:0/2097000938 --> [v2:172.21.15.102:6824/2853806211,v1:172.21.15.102:6825/2853806211] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f3558062d40 con 0x7f353c0add60 2024-09-10T09:36:46.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:46.197+0000 7f3554c00640 1 -- 172.21.15.5:0/2097000938 <== osd.10 v2:172.21.15.102:6824/2853806211 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f3558062d40 con 0x7f353c0add60 2024-09-10T09:36:46.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:46.197+0000 7f355d400640 1 -- 172.21.15.5:0/2097000938 >> [v2:172.21.15.102:6824/2853806211,v1:172.21.15.102:6825/2853806211] conn(0x7f353c0add60 msgr2=0x7f353c0b0200 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:36:46.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:46.197+0000 7f355d400640 1 --2- 172.21.15.5:0/2097000938 >> [v2:172.21.15.102:6824/2853806211,v1:172.21.15.102:6825/2853806211] conn(0x7f353c0add60 0x7f353c0b0200 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:36:46.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:46.197+0000 7f355d400640 1 -- 172.21.15.5:0/2097000938 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f353c0a2c10 msgr2=0x7f353c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:36:46.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:46.197+0000 7f355d400640 1 --2- 172.21.15.5:0/2097000938 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f353c0a2c10 0x7f353c0a50d0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f35580ffcf0 tx=0x7f354400b4a0 comp rx=0 tx=0).stop 2024-09-10T09:36:46.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:46.197+0000 7f355d400640 1 -- 172.21.15.5:0/2097000938 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3558103e30 msgr2=0x7f355819a490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:36:46.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:46.197+0000 7f355d400640 1 --2- 172.21.15.5:0/2097000938 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3558103e30 0x7f355819a490 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f354802a930 tx=0x7f3548003b90 comp rx=0 tx=0).stop 2024-09-10T09:36:46.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:46.197+0000 7f355ca00640 1 -- 172.21.15.5:0/2097000938 reap_dead start 2024-09-10T09:36:46.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:46.197+0000 7f355d400640 1 -- 172.21.15.5:0/2097000938 shutdown_connections 2024-09-10T09:36:46.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:46.197+0000 7f355d400640 1 -- 172.21.15.5:0/2097000938 >> 172.21.15.5:0/2097000938 conn(0x7f35580ff8d0 msgr2=0x7f3558100bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:36:46.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:46.197+0000 7f355d400640 1 -- 172.21.15.5:0/2097000938 shutdown_connections 2024-09-10T09:36:46.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:46.197+0000 7f355d400640 1 -- 172.21.15.5:0/2097000938 wait complete. 2024-09-10T09:36:46.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:46.626+0000 7fa41be00640 1 -- 172.21.15.5:0/762017536 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa41c077040 msgr2=0x7fa41c112330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:36:46.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:46.626+0000 7fa41be00640 1 --2- 172.21.15.5:0/762017536 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa41c077040 0x7fa41c112330 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fa408009900 tx=0x7fa40802a410 comp rx=0 tx=0).stop 2024-09-10T09:36:46.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:46.626+0000 7fa41be00640 1 -- 172.21.15.5:0/762017536 shutdown_connections 2024-09-10T09:36:46.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:46.626+0000 7fa41be00640 1 --2- 172.21.15.5:0/762017536 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa41c112870 0x7fa41c114c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:36:46.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:46.626+0000 7fa41be00640 1 --2- 172.21.15.5:0/762017536 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa41c077040 0x7fa41c112330 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:36:46.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:46.626+0000 7fa41be00640 1 --2- 172.21.15.5:0/762017536 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa41c076720 0x7fa41c076b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:36:46.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:46.626+0000 7fa41be00640 1 -- 172.21.15.5:0/762017536 >> 172.21.15.5:0/762017536 conn(0x7fa41c0ff8d0 msgr2=0x7fa41c101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:36:46.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:46.626+0000 7fa41be00640 1 -- 172.21.15.5:0/762017536 shutdown_connections 2024-09-10T09:36:46.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:46.626+0000 7fa41be00640 1 -- 172.21.15.5:0/762017536 wait complete. 2024-09-10T09:36:46.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:46.627+0000 7fa41be00640 1 Processor -- start 2024-09-10T09:36:46.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:46.627+0000 7fa41be00640 1 -- start start 2024-09-10T09:36:46.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:46.627+0000 7fa41be00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa41c076720 0x7fa41c1a4670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:36:46.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:46.627+0000 7fa41be00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa41c077040 0x7fa41c1a4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:36:46.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:46.627+0000 7fa41be00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa41c112870 0x7fa41c19e740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:36:46.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:46.627+0000 7fa41be00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa41c118410 con 0x7fa41c112870 2024-09-10T09:36:46.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:46.628+0000 7fa41be00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fa41c118290 con 0x7fa41c076720 2024-09-10T09:36:46.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:46.628+0000 7fa41be00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fa41c118590 con 0x7fa41c077040 2024-09-10T09:36:46.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:46.628+0000 7fa41b400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa41c112870 0x7fa41c19e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:36:46.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:46.628+0000 7fa41a000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa41c077040 0x7fa41c1a4bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:36:46.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:46.628+0000 7fa41b400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa41c112870 0x7fa41c19e740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58926/0 (socket says 172.21.15.5:58926) 2024-09-10T09:36:46.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:46.628+0000 7fa41b400640 1 -- 172.21.15.5:0/2735708205 learned_addr learned my addr 172.21.15.5:0/2735708205 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:36:46.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:46.628+0000 7fa41aa00640 1 --2- 172.21.15.5:0/2735708205 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa41c076720 0x7fa41c1a4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:36:46.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:46.628+0000 7fa41b400640 1 -- 172.21.15.5:0/2735708205 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa41c077040 msgr2=0x7fa41c1a4bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:36:46.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:46.628+0000 7fa41b400640 1 --2- 172.21.15.5:0/2735708205 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa41c077040 0x7fa41c1a4bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:36:46.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:46.628+0000 7fa41b400640 1 -- 172.21.15.5:0/2735708205 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa41c076720 msgr2=0x7fa41c1a4670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:36:46.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:46.628+0000 7fa41b400640 1 --2- 172.21.15.5:0/2735708205 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa41c076720 0x7fa41c1a4670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:36:46.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:46.628+0000 7fa41b400640 1 -- 172.21.15.5:0/2735708205 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa41c19efb0 con 0x7fa41c112870 2024-09-10T09:36:46.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:46.629+0000 7fa41b400640 1 --2- 172.21.15.5:0/2735708205 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa41c112870 0x7fa41c19e740 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fa41400eb10 tx=0x7fa41400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:36:46.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:46.629+0000 7fa413400640 1 -- 172.21.15.5:0/2735708205 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa414004b20 con 0x7fa41c112870 2024-09-10T09:36:46.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:46.629+0000 7fa413400640 1 -- 172.21.15.5:0/2735708205 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa414004cc0 con 0x7fa41c112870 2024-09-10T09:36:46.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:46.629+0000 7fa413400640 1 -- 172.21.15.5:0/2735708205 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa414010790 con 0x7fa41c112870 2024-09-10T09:36:46.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:46.629+0000 7fa41be00640 1 -- 172.21.15.5:0/2735708205 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa41c19f2a0 con 0x7fa41c112870 2024-09-10T09:36:46.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:46.629+0000 7fa41be00640 1 -- 172.21.15.5:0/2735708205 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa41c1ab420 con 0x7fa41c112870 2024-09-10T09:36:46.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:46.630+0000 7fa41be00640 1 -- 172.21.15.5:0/2735708205 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fa41c062980 con 0x7fa41c112870 2024-09-10T09:36:46.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:46.632+0000 7fa413400640 1 -- 172.21.15.5:0/2735708205 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7fa414005000 con 0x7fa41c112870 2024-09-10T09:36:46.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:46.633+0000 7fa413400640 1 --2- 172.21.15.5:0/2735708205 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fa4000a2c90 0x7fa4000a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:36:46.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:46.633+0000 7fa413400640 1 -- 172.21.15.5:0/2735708205 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9071+0+0 (secure 0 0 0) 0x7fa4140cfd20 con 0x7fa41c112870 2024-09-10T09:36:46.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:46.633+0000 7fa41aa00640 1 --2- 172.21.15.5:0/2735708205 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fa4000a2c90 0x7fa4000a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:36:46.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:46.633+0000 7fa413400640 1 --2- 172.21.15.5:0/2735708205 >> [v2:172.21.15.5:6826/902431281,v1:172.21.15.5:6827/902431281] conn(0x7fa4000adde0 0x7fa4000b0280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:36:46.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:46.634+0000 7fa413400640 1 -- 172.21.15.5:0/2735708205 --> [v2:172.21.15.5:6826/902431281,v1:172.21.15.5:6827/902431281] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa41400fe50 con 0x7fa4000adde0 2024-09-10T09:36:46.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:46.634+0000 7fa413400640 1 -- 172.21.15.5:0/2735708205 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_get_version_reply(handle=1 version=94) ==== 24+0+0 (secure 0 0 0) 0x7fa4140d00d0 con 0x7fa41c112870 2024-09-10T09:36:46.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:46.634+0000 7fa41a000640 1 --2- 172.21.15.5:0/2735708205 >> [v2:172.21.15.5:6826/902431281,v1:172.21.15.5:6827/902431281] conn(0x7fa4000adde0 0x7fa4000b0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:36:46.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:46.634+0000 7fa41aa00640 1 --2- 172.21.15.5:0/2735708205 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fa4000a2c90 0x7fa4000a5150 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fa40c009bb0 tx=0x7fa40c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:36:46.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:46.634+0000 7fa41a000640 1 --2- 172.21.15.5:0/2735708205 >> [v2:172.21.15.5:6826/902431281,v1:172.21.15.5:6827/902431281] conn(0x7fa4000adde0 0x7fa4000b0280 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:36:46.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:46.635+0000 7fa413400640 1 -- 172.21.15.5:0/2735708205 <== osd.9 v2:172.21.15.5:6826/902431281 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa41400fe50 con 0x7fa4000adde0 2024-09-10T09:36:46.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:46.650+0000 7fa41be00640 1 -- 172.21.15.5:0/2735708205 --> [v2:172.21.15.5:6826/902431281,v1:172.21.15.5:6827/902431281] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fa41c062e80 con 0x7fa4000adde0 2024-09-10T09:36:46.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:46.650+0000 7fa413400640 1 -- 172.21.15.5:0/2735708205 <== osd.9 v2:172.21.15.5:6826/902431281 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fa41c062e80 con 0x7fa4000adde0 2024-09-10T09:36:46.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:46.651+0000 7fa41be00640 1 -- 172.21.15.5:0/2735708205 >> [v2:172.21.15.5:6826/902431281,v1:172.21.15.5:6827/902431281] conn(0x7fa4000adde0 msgr2=0x7fa4000b0280 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:36:46.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:46.651+0000 7fa41be00640 1 --2- 172.21.15.5:0/2735708205 >> [v2:172.21.15.5:6826/902431281,v1:172.21.15.5:6827/902431281] conn(0x7fa4000adde0 0x7fa4000b0280 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:36:46.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:46.651+0000 7fa41be00640 1 -- 172.21.15.5:0/2735708205 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fa4000a2c90 msgr2=0x7fa4000a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:36:46.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:46.651+0000 7fa41be00640 1 --2- 172.21.15.5:0/2735708205 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fa4000a2c90 0x7fa4000a5150 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fa40c009bb0 tx=0x7fa40c005f90 comp rx=0 tx=0).stop 2024-09-10T09:36:46.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:46.651+0000 7fa41be00640 1 -- 172.21.15.5:0/2735708205 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa41c112870 msgr2=0x7fa41c19e740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:36:46.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:46.651+0000 7fa41be00640 1 --2- 172.21.15.5:0/2735708205 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa41c112870 0x7fa41c19e740 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fa41400eb10 tx=0x7fa41400c2d0 comp rx=0 tx=0).stop 2024-09-10T09:36:46.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:46.651+0000 7fa41b400640 1 -- 172.21.15.5:0/2735708205 reap_dead start 2024-09-10T09:36:46.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:46.651+0000 7fa41be00640 1 -- 172.21.15.5:0/2735708205 shutdown_connections 2024-09-10T09:36:46.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:46.651+0000 7fa41be00640 1 -- 172.21.15.5:0/2735708205 >> 172.21.15.5:0/2735708205 conn(0x7fa41c0ff8d0 msgr2=0x7fa41c101cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:36:46.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:46.651+0000 7fa41be00640 1 -- 172.21.15.5:0/2735708205 shutdown_connections 2024-09-10T09:36:46.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:46.651+0000 7fa41be00640 1 -- 172.21.15.5:0/2735708205 wait complete. 2024-09-10T09:36:47.641 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:36:47 smithi005 ceph-mon[30350]: pgmap v344: 129 pgs: 129 active+clean; 645 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.1 KiB/s wr, 12 op/s 2024-09-10T09:36:47.653 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:36:47 smithi159 ceph-mon[35513]: pgmap v344: 129 pgs: 129 active+clean; 645 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.1 KiB/s wr, 12 op/s 2024-09-10T09:36:47.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:47.860+0000 7fc6a3a00640 1 -- 172.21.15.5:0/2641282405 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc69c1049c0 msgr2=0x7fc69c108a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:36:47.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:47.860+0000 7fc6a3a00640 1 --2- 172.21.15.5:0/2641282405 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc69c1049c0 0x7fc69c108a10 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fc688009900 tx=0x7fc68802a410 comp rx=0 tx=0).stop 2024-09-10T09:36:47.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:47.860+0000 7fc6a3a00640 1 -- 172.21.15.5:0/2641282405 shutdown_connections 2024-09-10T09:36:47.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:47.860+0000 7fc6a3a00640 1 --2- 172.21.15.5:0/2641282405 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc69c109140 0x7fc69c110cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:36:47.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:47.860+0000 7fc6a3a00640 1 --2- 172.21.15.5:0/2641282405 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc69c1049c0 0x7fc69c108a10 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:36:47.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:47.860+0000 7fc6a3a00640 1 --2- 172.21.15.5:0/2641282405 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc69c104010 0x7fc69c1043f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:36:47.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:47.860+0000 7fc6a3a00640 1 -- 172.21.15.5:0/2641282405 >> 172.21.15.5:0/2641282405 conn(0x7fc69c0ffa50 msgr2=0x7fc69c101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:36:47.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:47.860+0000 7fc6a3a00640 1 -- 172.21.15.5:0/2641282405 shutdown_connections 2024-09-10T09:36:47.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:47.860+0000 7fc6a3a00640 1 -- 172.21.15.5:0/2641282405 wait complete. 2024-09-10T09:36:47.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:47.861+0000 7fc6a3a00640 1 Processor -- start 2024-09-10T09:36:47.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:47.861+0000 7fc6a3a00640 1 -- start start 2024-09-10T09:36:47.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:47.862+0000 7fc6a3a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc69c104010 0x7fc69c19a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:36:47.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:47.862+0000 7fc6a3a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc69c1049c0 0x7fc69c19abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:36:47.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:47.862+0000 7fc6a3a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc69c109140 0x7fc69c19ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:36:47.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:47.862+0000 7fc6a3a00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc69c1141b0 con 0x7fc69c104010 2024-09-10T09:36:47.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:47.862+0000 7fc6a3a00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fc69c114030 con 0x7fc69c1049c0 2024-09-10T09:36:47.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:47.862+0000 7fc6a3a00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fc69c114330 con 0x7fc69c109140 2024-09-10T09:36:47.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:47.862+0000 7fc6a1000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc69c104010 0x7fc69c19a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:36:47.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:47.862+0000 7fc69be00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc69c1049c0 0x7fc69c19abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:36:47.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:47.862+0000 7fc6a1a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc69c109140 0x7fc69c19ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:36:47.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:47.862+0000 7fc69be00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc69c1049c0 0x7fc69c19abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:47830/0 (socket says 172.21.15.5:47830) 2024-09-10T09:36:47.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:47.862+0000 7fc69be00640 1 -- 172.21.15.5:0/701021767 learned_addr learned my addr 172.21.15.5:0/701021767 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:36:47.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:47.862+0000 7fc6a1a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc69c109140 0x7fc69c19ef60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:51064/0 (socket says 172.21.15.5:51064) 2024-09-10T09:36:47.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:47.862+0000 7fc6a1000640 1 -- 172.21.15.5:0/701021767 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc69c109140 msgr2=0x7fc69c19ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:36:47.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:47.862+0000 7fc6a1000640 1 --2- 172.21.15.5:0/701021767 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc69c109140 0x7fc69c19ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:36:47.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:47.862+0000 7fc6a1000640 1 -- 172.21.15.5:0/701021767 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc69c1049c0 msgr2=0x7fc69c19abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:36:47.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:47.862+0000 7fc6a1000640 1 --2- 172.21.15.5:0/701021767 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc69c1049c0 0x7fc69c19abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:36:47.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:47.862+0000 7fc6a1000640 1 -- 172.21.15.5:0/701021767 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc69c19f640 con 0x7fc69c104010 2024-09-10T09:36:47.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:47.863+0000 7fc6a1000640 1 --2- 172.21.15.5:0/701021767 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc69c104010 0x7fc69c19a690 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fc690009fd0 tx=0x7fc69000ef90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:36:47.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:47.863+0000 7fc699600640 1 -- 172.21.15.5:0/701021767 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc690004b20 con 0x7fc69c104010 2024-09-10T09:36:47.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:47.863+0000 7fc699600640 1 -- 172.21.15.5:0/701021767 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc690004cc0 con 0x7fc69c104010 2024-09-10T09:36:47.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:47.863+0000 7fc6a3a00640 1 -- 172.21.15.5:0/701021767 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc69c19f930 con 0x7fc69c104010 2024-09-10T09:36:47.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:47.863+0000 7fc699600640 1 -- 172.21.15.5:0/701021767 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6900107b0 con 0x7fc69c104010 2024-09-10T09:36:47.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:47.863+0000 7fc6a3a00640 1 -- 172.21.15.5:0/701021767 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc69c1a7210 con 0x7fc69c104010 2024-09-10T09:36:47.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:47.864+0000 7fc6a3a00640 1 -- 172.21.15.5:0/701021767 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fc69c062980 con 0x7fc69c104010 2024-09-10T09:36:47.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:47.866+0000 7fc699600640 1 -- 172.21.15.5:0/701021767 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7fc690005020 con 0x7fc69c104010 2024-09-10T09:36:47.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:47.867+0000 7fc699600640 1 --2- 172.21.15.5:0/701021767 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fc6840a2c10 0x7fc6840a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:36:47.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:47.868+0000 7fc699600640 1 -- 172.21.15.5:0/701021767 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9071+0+0 (secure 0 0 0) 0x7fc6900cfe00 con 0x7fc69c104010 2024-09-10T09:36:47.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:47.868+0000 7fc69be00640 1 --2- 172.21.15.5:0/701021767 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fc6840a2c10 0x7fc6840a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:36:47.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:47.868+0000 7fc699600640 1 --2- 172.21.15.5:0/701021767 >> [v2:172.21.15.5:6802/954535266,v1:172.21.15.5:6803/954535266] conn(0x7fc6840add60 0x7fc6840b0200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:36:47.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:47.868+0000 7fc699600640 1 -- 172.21.15.5:0/701021767 --> [v2:172.21.15.5:6802/954535266,v1:172.21.15.5:6803/954535266] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc6840b08d0 con 0x7fc6840add60 2024-09-10T09:36:47.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:47.868+0000 7fc699600640 1 -- 172.21.15.5:0/701021767 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_get_version_reply(handle=1 version=94) ==== 24+0+0 (secure 0 0 0) 0x7fc6900d01b0 con 0x7fc69c104010 2024-09-10T09:36:47.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:47.868+0000 7fc6a1a00640 1 --2- 172.21.15.5:0/701021767 >> [v2:172.21.15.5:6802/954535266,v1:172.21.15.5:6803/954535266] conn(0x7fc6840add60 0x7fc6840b0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:36:47.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:47.868+0000 7fc69be00640 1 --2- 172.21.15.5:0/701021767 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fc6840a2c10 0x7fc6840a50d0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fc6880096c0 tx=0x7fc6880046c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:36:47.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:47.869+0000 7fc6a1a00640 1 --2- 172.21.15.5:0/701021767 >> [v2:172.21.15.5:6802/954535266,v1:172.21.15.5:6803/954535266] conn(0x7fc6840add60 0x7fc6840b0200 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:36:47.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:47.870+0000 7fc699600640 1 -- 172.21.15.5:0/701021767 <== osd.0 v2:172.21.15.5:6802/954535266 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc6840b08d0 con 0x7fc6840add60 2024-09-10T09:36:47.886 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:36:47 smithi102 ceph-mon[35442]: pgmap v344: 129 pgs: 129 active+clean; 645 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.1 KiB/s wr, 12 op/s 2024-09-10T09:36:47.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:47.884+0000 7fc6a3a00640 1 -- 172.21.15.5:0/701021767 --> [v2:172.21.15.5:6802/954535266,v1:172.21.15.5:6803/954535266] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fc69c062e00 con 0x7fc6840add60 2024-09-10T09:36:47.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:47.885+0000 7fc699600640 1 -- 172.21.15.5:0/701021767 <== osd.0 v2:172.21.15.5:6802/954535266 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fc69c062e00 con 0x7fc6840add60 2024-09-10T09:36:47.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:47.885+0000 7fc6a3a00640 1 -- 172.21.15.5:0/701021767 >> [v2:172.21.15.5:6802/954535266,v1:172.21.15.5:6803/954535266] conn(0x7fc6840add60 msgr2=0x7fc6840b0200 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:36:47.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:47.885+0000 7fc6a3a00640 1 --2- 172.21.15.5:0/701021767 >> [v2:172.21.15.5:6802/954535266,v1:172.21.15.5:6803/954535266] conn(0x7fc6840add60 0x7fc6840b0200 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:36:47.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:47.885+0000 7fc6a3a00640 1 -- 172.21.15.5:0/701021767 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fc6840a2c10 msgr2=0x7fc6840a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:36:47.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:47.885+0000 7fc6a3a00640 1 --2- 172.21.15.5:0/701021767 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fc6840a2c10 0x7fc6840a50d0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fc6880096c0 tx=0x7fc6880046c0 comp rx=0 tx=0).stop 2024-09-10T09:36:47.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:47.885+0000 7fc6a3a00640 1 -- 172.21.15.5:0/701021767 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc69c104010 msgr2=0x7fc69c19a690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:36:47.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:47.885+0000 7fc6a3a00640 1 --2- 172.21.15.5:0/701021767 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc69c104010 0x7fc69c19a690 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fc690009fd0 tx=0x7fc69000ef90 comp rx=0 tx=0).stop 2024-09-10T09:36:47.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:47.885+0000 7fc6a1a00640 1 -- 172.21.15.5:0/701021767 reap_dead start 2024-09-10T09:36:47.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:47.885+0000 7fc6a3a00640 1 -- 172.21.15.5:0/701021767 shutdown_connections 2024-09-10T09:36:47.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:47.885+0000 7fc6a3a00640 1 -- 172.21.15.5:0/701021767 >> 172.21.15.5:0/701021767 conn(0x7fc69c0ffa50 msgr2=0x7fc69c100d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:36:47.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:47.886+0000 7fc6a3a00640 1 -- 172.21.15.5:0/701021767 shutdown_connections 2024-09-10T09:36:47.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:47.886+0000 7fc6a3a00640 1 -- 172.21.15.5:0/701021767 wait complete. 2024-09-10T09:36:48.717 INFO:teuthology.orchestra.run.smithi005.stdout:300647710754 2024-09-10T09:36:48.718 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.10 2024-09-10T09:36:49.059 INFO:teuthology.orchestra.run.smithi005.stdout:279172874281 2024-09-10T09:36:49.059 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.9 2024-09-10T09:36:49.545 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:36:49 smithi005 ceph-mon[30350]: pgmap v345: 129 pgs: 129 active+clean; 645 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 682 B/s wr, 9 op/s 2024-09-10T09:36:49.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:49.518+0000 7f7680a00640 1 -- 172.21.15.5:0/2734017033 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f767c102360 msgr2=0x7f767c10e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:36:49.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:49.518+0000 7f7680a00640 1 --2- 172.21.15.5:0/2734017033 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f767c102360 0x7f767c10e890 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f7674009cd0 tx=0x7f767402a8e0 comp rx=0 tx=0).stop 2024-09-10T09:36:49.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:49.518+0000 7f7680a00640 1 -- 172.21.15.5:0/2734017033 shutdown_connections 2024-09-10T09:36:49.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:49.518+0000 7f7680a00640 1 --2- 172.21.15.5:0/2734017033 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f767c102360 0x7f767c10e890 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:36:49.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:49.518+0000 7f7680a00640 1 --2- 172.21.15.5:0/2734017033 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f767c1019c0 0x7f767c101e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:36:49.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:49.518+0000 7f7680a00640 1 --2- 172.21.15.5:0/2734017033 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f767c1079c0 0x7f767c107da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:36:49.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:49.518+0000 7f7680a00640 1 -- 172.21.15.5:0/2734017033 >> 172.21.15.5:0/2734017033 conn(0x7f767c0fd740 msgr2=0x7f767c0ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:36:49.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:49.518+0000 7f7680a00640 1 -- 172.21.15.5:0/2734017033 shutdown_connections 2024-09-10T09:36:49.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:49.518+0000 7f7680a00640 1 -- 172.21.15.5:0/2734017033 wait complete. 2024-09-10T09:36:49.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:49.519+0000 7f7680a00640 1 Processor -- start 2024-09-10T09:36:49.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:49.519+0000 7f7680a00640 1 -- start start 2024-09-10T09:36:49.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:49.519+0000 7f7680a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f767c1019c0 0x7f767c0744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:36:49.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:49.520+0000 7f7680a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f767c102360 0x7f767c074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:36:49.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:49.520+0000 7f7680a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f767c1079c0 0x7f767c074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:36:49.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:49.520+0000 7f7680a00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f767c111dc0 con 0x7f767c1019c0 2024-09-10T09:36:49.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:49.520+0000 7f7680a00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f767c111c40 con 0x7f767c1079c0 2024-09-10T09:36:49.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:49.520+0000 7f7680a00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f767c111f40 con 0x7f767c102360 2024-09-10T09:36:49.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:49.520+0000 7f767a800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f767c1019c0 0x7f767c0744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:36:49.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:49.520+0000 7f7679e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f767c102360 0x7f767c074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:36:49.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:49.520+0000 7f7679e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f767c102360 0x7f767c074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:51082/0 (socket says 172.21.15.5:51082) 2024-09-10T09:36:49.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:49.520+0000 7f767a800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f767c1019c0 0x7f767c0744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58950/0 (socket says 172.21.15.5:58950) 2024-09-10T09:36:49.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:49.520+0000 7f7679e00640 1 -- 172.21.15.5:0/2232115831 learned_addr learned my addr 172.21.15.5:0/2232115831 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:36:49.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:49.520+0000 7f767b200640 1 --2- 172.21.15.5:0/2232115831 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f767c1079c0 0x7f767c074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:36:49.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:49.520+0000 7f767a800640 1 -- 172.21.15.5:0/2232115831 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f767c102360 msgr2=0x7f767c074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:36:49.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:49.520+0000 7f767a800640 1 --2- 172.21.15.5:0/2232115831 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f767c102360 0x7f767c074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:36:49.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:49.520+0000 7f767a800640 1 -- 172.21.15.5:0/2232115831 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f767c1079c0 msgr2=0x7f767c074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:36:49.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:49.520+0000 7f767a800640 1 --2- 172.21.15.5:0/2232115831 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f767c1079c0 0x7f767c074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:36:49.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:49.520+0000 7f767a800640 1 -- 172.21.15.5:0/2232115831 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f767c1a6780 con 0x7f767c1019c0 2024-09-10T09:36:49.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:49.520+0000 7f7679e00640 1 --2- 172.21.15.5:0/2232115831 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f767c102360 0x7f767c074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:36:49.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:49.520+0000 7f767b200640 1 --2- 172.21.15.5:0/2232115831 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f767c1079c0 0x7f767c074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:36:49.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:49.521+0000 7f767a800640 1 --2- 172.21.15.5:0/2232115831 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f767c1019c0 0x7f767c0744f0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f766c00e9f0 tx=0x7f766c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:36:49.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:49.521+0000 7f7673400640 1 -- 172.21.15.5:0/2232115831 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f766c00ccc0 con 0x7f767c1019c0 2024-09-10T09:36:49.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:49.521+0000 7f7673400640 1 -- 172.21.15.5:0/2232115831 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f766c010910 con 0x7f767c1019c0 2024-09-10T09:36:49.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:49.521+0000 7f7680a00640 1 -- 172.21.15.5:0/2232115831 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f767c1a6a70 con 0x7f767c1019c0 2024-09-10T09:36:49.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:49.521+0000 7f7673400640 1 -- 172.21.15.5:0/2232115831 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f766c018960 con 0x7f767c1019c0 2024-09-10T09:36:49.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:49.521+0000 7f7680a00640 1 -- 172.21.15.5:0/2232115831 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f767c1a7000 con 0x7f767c1019c0 2024-09-10T09:36:49.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:49.522+0000 7f7680a00640 1 -- 172.21.15.5:0/2232115831 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f767c0628c0 con 0x7f767c1019c0 2024-09-10T09:36:49.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:49.524+0000 7f7673400640 1 -- 172.21.15.5:0/2232115831 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f766c010ac0 con 0x7f767c1019c0 2024-09-10T09:36:49.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:49.525+0000 7f7673400640 1 --2- 172.21.15.5:0/2232115831 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f76600a2c90 0x7f76600a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:36:49.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:49.525+0000 7f7673400640 1 -- 172.21.15.5:0/2232115831 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9071+0+0 (secure 0 0 0) 0x7f766c0d0090 con 0x7f767c1019c0 2024-09-10T09:36:49.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:49.525+0000 7f7679e00640 1 --2- 172.21.15.5:0/2232115831 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f76600a2c90 0x7f76600a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:36:49.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:49.525+0000 7f7673400640 1 --2- 172.21.15.5:0/2232115831 >> [v2:172.21.15.102:6816/692040360,v1:172.21.15.102:6817/692040360] conn(0x7f76600adde0 0x7f76600b0280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:36:49.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:49.526+0000 7f7673400640 1 -- 172.21.15.5:0/2232115831 --> [v2:172.21.15.102:6816/692040360,v1:172.21.15.102:6817/692040360] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f766c01bec0 con 0x7f76600adde0 2024-09-10T09:36:49.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:49.526+0000 7f7673400640 1 -- 172.21.15.5:0/2232115831 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_get_version_reply(handle=1 version=94) ==== 24+0+0 (secure 0 0 0) 0x7f766c0d0440 con 0x7f767c1019c0 2024-09-10T09:36:49.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:49.526+0000 7f767b200640 1 --2- 172.21.15.5:0/2232115831 >> [v2:172.21.15.102:6816/692040360,v1:172.21.15.102:6817/692040360] conn(0x7f76600adde0 0x7f76600b0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:36:49.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:49.526+0000 7f7679e00640 1 --2- 172.21.15.5:0/2232115831 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f76600a2c90 0x7f76600a5150 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f767c075f80 tx=0x7f7668005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:36:49.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:49.527+0000 7f767b200640 1 --2- 172.21.15.5:0/2232115831 >> [v2:172.21.15.102:6816/692040360,v1:172.21.15.102:6817/692040360] conn(0x7f76600adde0 0x7f76600b0280 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:36:49.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:49.527+0000 7f7673400640 1 -- 172.21.15.5:0/2232115831 <== osd.7 v2:172.21.15.102:6816/692040360 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f766c01bec0 con 0x7f76600adde0 2024-09-10T09:36:49.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:49.542+0000 7f7680a00640 1 -- 172.21.15.5:0/2232115831 --> [v2:172.21.15.102:6816/692040360,v1:172.21.15.102:6817/692040360] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f767c062cc0 con 0x7f76600adde0 2024-09-10T09:36:49.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:49.542+0000 7f7673400640 1 -- 172.21.15.5:0/2232115831 <== osd.7 v2:172.21.15.102:6816/692040360 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f767c062cc0 con 0x7f76600adde0 2024-09-10T09:36:49.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:49.543+0000 7f7680a00640 1 -- 172.21.15.5:0/2232115831 >> [v2:172.21.15.102:6816/692040360,v1:172.21.15.102:6817/692040360] conn(0x7f76600adde0 msgr2=0x7f76600b0280 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:36:49.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:49.543+0000 7f7680a00640 1 --2- 172.21.15.5:0/2232115831 >> [v2:172.21.15.102:6816/692040360,v1:172.21.15.102:6817/692040360] conn(0x7f76600adde0 0x7f76600b0280 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:36:49.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:49.543+0000 7f7680a00640 1 -- 172.21.15.5:0/2232115831 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f76600a2c90 msgr2=0x7f76600a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:36:49.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:49.543+0000 7f7680a00640 1 --2- 172.21.15.5:0/2232115831 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f76600a2c90 0x7f76600a5150 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f767c075f80 tx=0x7f7668005f90 comp rx=0 tx=0).stop 2024-09-10T09:36:49.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:49.543+0000 7f7680a00640 1 -- 172.21.15.5:0/2232115831 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f767c1019c0 msgr2=0x7f767c0744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:36:49.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:49.543+0000 7f7680a00640 1 --2- 172.21.15.5:0/2232115831 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f767c1019c0 0x7f767c0744f0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f766c00e9f0 tx=0x7f766c00eeb0 comp rx=0 tx=0).stop 2024-09-10T09:36:49.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:49.543+0000 7f767b200640 1 -- 172.21.15.5:0/2232115831 reap_dead start 2024-09-10T09:36:49.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:49.543+0000 7f7680a00640 1 -- 172.21.15.5:0/2232115831 shutdown_connections 2024-09-10T09:36:49.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:49.543+0000 7f7680a00640 1 -- 172.21.15.5:0/2232115831 >> 172.21.15.5:0/2232115831 conn(0x7f767c0fd740 msgr2=0x7f767c0fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:36:49.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:49.543+0000 7f7680a00640 1 -- 172.21.15.5:0/2232115831 shutdown_connections 2024-09-10T09:36:49.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:49.543+0000 7f7680a00640 1 -- 172.21.15.5:0/2232115831 wait complete. 2024-09-10T09:36:49.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:36:49 smithi159 ceph-mon[35513]: pgmap v345: 129 pgs: 129 active+clean; 645 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 682 B/s wr, 9 op/s 2024-09-10T09:36:49.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:36:49 smithi102 ceph-mon[35442]: pgmap v345: 129 pgs: 129 active+clean; 645 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 682 B/s wr, 9 op/s 2024-09-10T09:36:50.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:50.588+0000 7f3cf0c00640 1 -- 172.21.15.5:0/3691990242 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3cec109120 msgr2=0x7f3cec110ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:36:50.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:50.588+0000 7f3cf0c00640 1 --2- 172.21.15.5:0/3691990242 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3cec109120 0x7f3cec110ca0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f3ce4009be0 tx=0x7f3ce402a6c0 comp rx=0 tx=0).stop 2024-09-10T09:36:50.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:50.588+0000 7f3cf0c00640 1 -- 172.21.15.5:0/3691990242 shutdown_connections 2024-09-10T09:36:50.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:50.588+0000 7f3cf0c00640 1 --2- 172.21.15.5:0/3691990242 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3cec109120 0x7f3cec110ca0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:36:50.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:50.588+0000 7f3cf0c00640 1 --2- 172.21.15.5:0/3691990242 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3cec1049a0 0x7f3cec1089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:36:50.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:50.588+0000 7f3cf0c00640 1 --2- 172.21.15.5:0/3691990242 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3cec103ff0 0x7f3cec1043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:36:50.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:50.588+0000 7f3cf0c00640 1 -- 172.21.15.5:0/3691990242 >> 172.21.15.5:0/3691990242 conn(0x7f3cec0ffa90 msgr2=0x7f3cec101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:36:50.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:50.588+0000 7f3cf0c00640 1 -- 172.21.15.5:0/3691990242 shutdown_connections 2024-09-10T09:36:50.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:50.588+0000 7f3cf0c00640 1 -- 172.21.15.5:0/3691990242 wait complete. 2024-09-10T09:36:50.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:50.589+0000 7f3cf0c00640 1 Processor -- start 2024-09-10T09:36:50.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:50.590+0000 7f3cf0c00640 1 -- start start 2024-09-10T09:36:50.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:50.590+0000 7f3cf0c00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3cec103ff0 0x7f3cec19a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:36:50.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:50.590+0000 7f3cf0c00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3cec1049a0 0x7f3cec19ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:36:50.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:50.590+0000 7f3cf0c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3cec109120 0x7f3cec19eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:36:50.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:50.590+0000 7f3cf0c00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f3cec114130 con 0x7f3cec1049a0 2024-09-10T09:36:50.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:50.590+0000 7f3cf0c00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f3cec113fb0 con 0x7f3cec109120 2024-09-10T09:36:50.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:50.590+0000 7f3cf0c00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f3cec1142b0 con 0x7f3cec103ff0 2024-09-10T09:36:50.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:50.590+0000 7f3ce9e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3cec103ff0 0x7f3cec19a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:36:50.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:50.590+0000 7f3ce9e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3cec103ff0 0x7f3cec19a610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:51106/0 (socket says 172.21.15.5:51106) 2024-09-10T09:36:50.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:50.590+0000 7f3ce9e00640 1 -- 172.21.15.5:0/1699251154 learned_addr learned my addr 172.21.15.5:0/1699251154 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:36:50.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:50.591+0000 7f3ce9400640 1 --2- 172.21.15.5:0/1699251154 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3cec1049a0 0x7f3cec19ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:36:50.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:50.591+0000 7f3cea800640 1 --2- 172.21.15.5:0/1699251154 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3cec109120 0x7f3cec19eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:36:50.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:50.591+0000 7f3ce9e00640 1 -- 172.21.15.5:0/1699251154 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3cec109120 msgr2=0x7f3cec19eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:36:50.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:50.591+0000 7f3ce9e00640 1 --2- 172.21.15.5:0/1699251154 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3cec109120 0x7f3cec19eee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:36:50.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:50.591+0000 7f3ce9e00640 1 -- 172.21.15.5:0/1699251154 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3cec1049a0 msgr2=0x7f3cec19ab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:36:50.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:50.591+0000 7f3ce9e00640 1 --2- 172.21.15.5:0/1699251154 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3cec1049a0 0x7f3cec19ab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:36:50.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:50.591+0000 7f3ce9e00640 1 -- 172.21.15.5:0/1699251154 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3cec19f5c0 con 0x7f3cec103ff0 2024-09-10T09:36:50.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:50.591+0000 7f3ce9e00640 1 --2- 172.21.15.5:0/1699251154 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3cec103ff0 0x7f3cec19a610 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f3cdc00e9c0 tx=0x7f3cdc00ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:36:50.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:50.591+0000 7f3ce2a00640 1 -- 172.21.15.5:0/1699251154 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cdc00cde0 con 0x7f3cec103ff0 2024-09-10T09:36:50.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:50.592+0000 7f3cf0c00640 1 -- 172.21.15.5:0/1699251154 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3cec19f8b0 con 0x7f3cec103ff0 2024-09-10T09:36:50.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:50.592+0000 7f3ce2a00640 1 -- 172.21.15.5:0/1699251154 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3cdc0035f0 con 0x7f3cec103ff0 2024-09-10T09:36:50.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:50.592+0000 7f3ce2a00640 1 -- 172.21.15.5:0/1699251154 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cdc010650 con 0x7f3cec103ff0 2024-09-10T09:36:50.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:50.592+0000 7f3cf0c00640 1 -- 172.21.15.5:0/1699251154 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3cec1a7190 con 0x7f3cec103ff0 2024-09-10T09:36:50.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:50.593+0000 7f3cf0c00640 1 -- 172.21.15.5:0/1699251154 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f3cec062980 con 0x7f3cec103ff0 2024-09-10T09:36:50.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:50.595+0000 7f3ce2a00640 1 -- 172.21.15.5:0/1699251154 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f3cdc0037a0 con 0x7f3cec103ff0 2024-09-10T09:36:50.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:50.596+0000 7f3ce2a00640 1 --2- 172.21.15.5:0/1699251154 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f3cd00a2c90 0x7f3cd00a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:36:50.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:50.596+0000 7f3ce2a00640 1 -- 172.21.15.5:0/1699251154 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9071+0+0 (secure 0 0 0) 0x7f3cdc0cfab0 con 0x7f3cec103ff0 2024-09-10T09:36:50.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:50.596+0000 7f3ce9400640 1 --2- 172.21.15.5:0/1699251154 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f3cd00a2c90 0x7f3cd00a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:36:50.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:50.596+0000 7f3ce2a00640 1 --2- 172.21.15.5:0/1699251154 >> [v2:172.21.15.159:6816/2107291799,v1:172.21.15.159:6817/2107291799] conn(0x7f3cd00adde0 0x7f3cd00b0280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:36:50.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:50.596+0000 7f3ce2a00640 1 -- 172.21.15.5:0/1699251154 --> [v2:172.21.15.159:6816/2107291799,v1:172.21.15.159:6817/2107291799] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f3cdc009540 con 0x7f3cd00adde0 2024-09-10T09:36:50.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:50.596+0000 7f3ce2a00640 1 -- 172.21.15.5:0/1699251154 <== mon.1 v2:172.21.15.159:3300/0 6 ==== mon_get_version_reply(handle=1 version=94) ==== 24+0+0 (secure 0 0 0) 0x7f3cdc0cfe60 con 0x7f3cec103ff0 2024-09-10T09:36:50.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:50.597+0000 7f3cea800640 1 --2- 172.21.15.5:0/1699251154 >> [v2:172.21.15.159:6816/2107291799,v1:172.21.15.159:6817/2107291799] conn(0x7f3cd00adde0 0x7f3cd00b0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:36:50.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:50.597+0000 7f3ce9400640 1 --2- 172.21.15.5:0/1699251154 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f3cd00a2c90 0x7f3cd00a5150 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f3cec10a580 tx=0x7f3cd8005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:36:50.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:50.597+0000 7f3cea800640 1 --2- 172.21.15.5:0/1699251154 >> [v2:172.21.15.159:6816/2107291799,v1:172.21.15.159:6817/2107291799] conn(0x7f3cd00adde0 0x7f3cd00b0280 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:36:50.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:50.598+0000 7f3ce2a00640 1 -- 172.21.15.5:0/1699251154 <== osd.8 v2:172.21.15.159:6816/2107291799 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f3cdc009540 con 0x7f3cd00adde0 2024-09-10T09:36:50.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:50.613+0000 7f3cf0c00640 1 -- 172.21.15.5:0/1699251154 --> [v2:172.21.15.159:6816/2107291799,v1:172.21.15.159:6817/2107291799] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f3cec062d80 con 0x7f3cd00adde0 2024-09-10T09:36:50.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:50.614+0000 7f3ce2a00640 1 -- 172.21.15.5:0/1699251154 <== osd.8 v2:172.21.15.159:6816/2107291799 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f3cec062d80 con 0x7f3cd00adde0 2024-09-10T09:36:50.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:50.614+0000 7f3cf0c00640 1 -- 172.21.15.5:0/1699251154 >> [v2:172.21.15.159:6816/2107291799,v1:172.21.15.159:6817/2107291799] conn(0x7f3cd00adde0 msgr2=0x7f3cd00b0280 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:36:50.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:50.614+0000 7f3cf0c00640 1 --2- 172.21.15.5:0/1699251154 >> [v2:172.21.15.159:6816/2107291799,v1:172.21.15.159:6817/2107291799] conn(0x7f3cd00adde0 0x7f3cd00b0280 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:36:50.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:50.614+0000 7f3cf0c00640 1 -- 172.21.15.5:0/1699251154 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f3cd00a2c90 msgr2=0x7f3cd00a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:36:50.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:50.614+0000 7f3cf0c00640 1 --2- 172.21.15.5:0/1699251154 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f3cd00a2c90 0x7f3cd00a5150 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f3cec10a580 tx=0x7f3cd8005f90 comp rx=0 tx=0).stop 2024-09-10T09:36:50.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:50.614+0000 7f3cf0c00640 1 -- 172.21.15.5:0/1699251154 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3cec103ff0 msgr2=0x7f3cec19a610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:36:50.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:50.614+0000 7f3cf0c00640 1 --2- 172.21.15.5:0/1699251154 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3cec103ff0 0x7f3cec19a610 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f3cdc00e9c0 tx=0x7f3cdc00ee80 comp rx=0 tx=0).stop 2024-09-10T09:36:50.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:50.614+0000 7f3cea800640 1 -- 172.21.15.5:0/1699251154 reap_dead start 2024-09-10T09:36:50.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:50.614+0000 7f3cf0c00640 1 -- 172.21.15.5:0/1699251154 shutdown_connections 2024-09-10T09:36:50.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:50.614+0000 7f3cf0c00640 1 -- 172.21.15.5:0/1699251154 >> 172.21.15.5:0/1699251154 conn(0x7f3cec0ffa90 msgr2=0x7f3cec100d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:36:50.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:50.614+0000 7f3cf0c00640 1 -- 172.21.15.5:0/1699251154 shutdown_connections 2024-09-10T09:36:50.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:50.615+0000 7f3cf0c00640 1 -- 172.21.15.5:0/1699251154 wait complete. 2024-09-10T09:36:51.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:50.715+0000 7f0dc7e00640 1 -- 172.21.15.5:0/2031029977 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0dc8102340 msgr2=0x7f0dc810e870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:36:51.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:50.715+0000 7f0dc7e00640 1 --2- 172.21.15.5:0/2031029977 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0dc8102340 0x7f0dc810e870 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f0dc0009cd0 tx=0x7f0dc002a8e0 comp rx=0 tx=0).stop 2024-09-10T09:36:51.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:50.715+0000 7f0dc7e00640 1 -- 172.21.15.5:0/2031029977 shutdown_connections 2024-09-10T09:36:51.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:50.715+0000 7f0dc7e00640 1 --2- 172.21.15.5:0/2031029977 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0dc8102340 0x7f0dc810e870 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:36:51.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:50.715+0000 7f0dc7e00640 1 --2- 172.21.15.5:0/2031029977 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0dc81019a0 0x7f0dc8101e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:36:51.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:50.715+0000 7f0dc7e00640 1 --2- 172.21.15.5:0/2031029977 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0dc81079a0 0x7f0dc8107d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:36:51.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:50.715+0000 7f0dc7e00640 1 -- 172.21.15.5:0/2031029977 >> 172.21.15.5:0/2031029977 conn(0x7f0dc80fd700 msgr2=0x7f0dc80ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:36:51.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:50.715+0000 7f0dc7e00640 1 -- 172.21.15.5:0/2031029977 shutdown_connections 2024-09-10T09:36:51.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:50.715+0000 7f0dc7e00640 1 -- 172.21.15.5:0/2031029977 wait complete. 2024-09-10T09:36:51.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:50.716+0000 7f0dc7e00640 1 Processor -- start 2024-09-10T09:36:51.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:50.716+0000 7f0dc7e00640 1 -- start start 2024-09-10T09:36:51.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:50.717+0000 7f0dc7e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0dc81019a0 0x7f0dc80744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:36:51.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:50.717+0000 7f0dc7e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0dc8102340 0x7f0dc8074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:36:51.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:50.717+0000 7f0dc7e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0dc81079a0 0x7f0dc8074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:36:51.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:50.717+0000 7f0dc7e00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f0dc8111df0 con 0x7f0dc81019a0 2024-09-10T09:36:51.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:50.717+0000 7f0dc7e00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f0dc8111c70 con 0x7f0dc8102340 2024-09-10T09:36:51.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:50.717+0000 7f0dc7e00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f0dc8111f70 con 0x7f0dc81079a0 2024-09-10T09:36:51.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:50.717+0000 7f0dc6a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0dc81019a0 0x7f0dc80744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:36:51.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:50.717+0000 7f0dbfe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0dc8102340 0x7f0dc8074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:36:51.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:50.717+0000 7f0dbfe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0dc8102340 0x7f0dc8074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:47920/0 (socket says 172.21.15.5:47920) 2024-09-10T09:36:51.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:50.717+0000 7f0dbfe00640 1 -- 172.21.15.5:0/1390807655 learned_addr learned my addr 172.21.15.5:0/1390807655 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:36:51.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:50.717+0000 7f0dc7400640 1 --2- 172.21.15.5:0/1390807655 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0dc81079a0 0x7f0dc8074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:36:51.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:50.718+0000 7f0dbfe00640 1 -- 172.21.15.5:0/1390807655 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0dc81079a0 msgr2=0x7f0dc8074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:36:51.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:50.718+0000 7f0dbfe00640 1 --2- 172.21.15.5:0/1390807655 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0dc81079a0 0x7f0dc8074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:36:51.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:50.718+0000 7f0dbfe00640 1 -- 172.21.15.5:0/1390807655 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0dc81019a0 msgr2=0x7f0dc80744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:36:51.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:50.718+0000 7f0dbfe00640 1 --2- 172.21.15.5:0/1390807655 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0dc81019a0 0x7f0dc80744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:36:51.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:50.718+0000 7f0dbfe00640 1 -- 172.21.15.5:0/1390807655 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0dc81a66d0 con 0x7f0dc8102340 2024-09-10T09:36:51.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:50.718+0000 7f0dc6a00640 1 --2- 172.21.15.5:0/1390807655 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0dc81019a0 0x7f0dc80744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:36:51.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:50.718+0000 7f0dbfe00640 1 --2- 172.21.15.5:0/1390807655 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0dc8102340 0x7f0dc8074a30 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f0db000e9f0 tx=0x7f0db000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:36:51.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:50.718+0000 7f0dbf400640 1 -- 172.21.15.5:0/1390807655 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0db000cd80 con 0x7f0dc8102340 2024-09-10T09:36:51.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:50.718+0000 7f0dc7e00640 1 -- 172.21.15.5:0/1390807655 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0dc81a69c0 con 0x7f0dc8102340 2024-09-10T09:36:51.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:50.718+0000 7f0dbf400640 1 -- 172.21.15.5:0/1390807655 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0db0010910 con 0x7f0dc8102340 2024-09-10T09:36:51.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:50.718+0000 7f0dbf400640 1 -- 172.21.15.5:0/1390807655 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0db00188f0 con 0x7f0dc8102340 2024-09-10T09:36:51.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:50.719+0000 7f0dc7e00640 1 -- 172.21.15.5:0/1390807655 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0dc81a6f00 con 0x7f0dc8102340 2024-09-10T09:36:51.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:50.719+0000 7f0dc7e00640 1 -- 172.21.15.5:0/1390807655 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f0dc80ff830 con 0x7f0dc8102340 2024-09-10T09:36:51.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:50.722+0000 7f0dbf400640 1 -- 172.21.15.5:0/1390807655 <== mon.2 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f0db0010ac0 con 0x7f0dc8102340 2024-09-10T09:36:51.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:50.723+0000 7f0dbf400640 1 --2- 172.21.15.5:0/1390807655 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f0dac0a2c10 0x7f0dac0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:36:51.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:50.723+0000 7f0dbf400640 1 -- 172.21.15.5:0/1390807655 <== mon.2 v2:172.21.15.102:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9071+0+0 (secure 0 0 0) 0x7f0db00cfff0 con 0x7f0dc8102340 2024-09-10T09:36:51.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:50.723+0000 7f0dc6a00640 1 --2- 172.21.15.5:0/1390807655 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f0dac0a2c10 0x7f0dac0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:36:51.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:50.723+0000 7f0dbf400640 1 --2- 172.21.15.5:0/1390807655 >> [v2:172.21.15.102:6800/733924375,v1:172.21.15.102:6801/733924375] conn(0x7f0dac0add60 0x7f0dac0b0200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:36:51.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:50.723+0000 7f0dbf400640 1 -- 172.21.15.5:0/1390807655 --> [v2:172.21.15.102:6800/733924375,v1:172.21.15.102:6801/733924375] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f0dac0b08d0 con 0x7f0dac0add60 2024-09-10T09:36:51.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:50.723+0000 7f0dbf400640 1 -- 172.21.15.5:0/1390807655 <== mon.2 v2:172.21.15.102:3300/0 6 ==== mon_get_version_reply(handle=1 version=94) ==== 24+0+0 (secure 0 0 0) 0x7f0db00d0420 con 0x7f0dc8102340 2024-09-10T09:36:51.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:50.724+0000 7f0dc7400640 1 --2- 172.21.15.5:0/1390807655 >> [v2:172.21.15.102:6800/733924375,v1:172.21.15.102:6801/733924375] conn(0x7f0dac0add60 0x7f0dac0b0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:36:51.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:50.724+0000 7f0dc6a00640 1 --2- 172.21.15.5:0/1390807655 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f0dac0a2c10 0x7f0dac0a50d0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f0db8009f40 tx=0x7f0db8005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:36:51.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:50.724+0000 7f0dc7400640 1 --2- 172.21.15.5:0/1390807655 >> [v2:172.21.15.102:6800/733924375,v1:172.21.15.102:6801/733924375] conn(0x7f0dac0add60 0x7f0dac0b0200 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:36:51.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:50.725+0000 7f0dbf400640 1 -- 172.21.15.5:0/1390807655 <== osd.1 v2:172.21.15.102:6800/733924375 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f0dac0b08d0 con 0x7f0dac0add60 2024-09-10T09:36:51.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:50.740+0000 7f0dc7e00640 1 -- 172.21.15.5:0/1390807655 --> [v2:172.21.15.102:6800/733924375,v1:172.21.15.102:6801/733924375] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f0dc81030e0 con 0x7f0dac0add60 2024-09-10T09:36:51.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:50.740+0000 7f0dbf400640 1 -- 172.21.15.5:0/1390807655 <== osd.1 v2:172.21.15.102:6800/733924375 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f0dc81030e0 con 0x7f0dac0add60 2024-09-10T09:36:51.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:50.741+0000 7f0dc7e00640 1 -- 172.21.15.5:0/1390807655 >> [v2:172.21.15.102:6800/733924375,v1:172.21.15.102:6801/733924375] conn(0x7f0dac0add60 msgr2=0x7f0dac0b0200 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:36:51.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:50.741+0000 7f0dc7e00640 1 --2- 172.21.15.5:0/1390807655 >> [v2:172.21.15.102:6800/733924375,v1:172.21.15.102:6801/733924375] conn(0x7f0dac0add60 0x7f0dac0b0200 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:36:51.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:50.741+0000 7f0dc7e00640 1 -- 172.21.15.5:0/1390807655 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f0dac0a2c10 msgr2=0x7f0dac0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:36:51.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:50.741+0000 7f0dc7e00640 1 --2- 172.21.15.5:0/1390807655 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f0dac0a2c10 0x7f0dac0a50d0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f0db8009f40 tx=0x7f0db8005f90 comp rx=0 tx=0).stop 2024-09-10T09:36:51.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:50.741+0000 7f0dc7e00640 1 -- 172.21.15.5:0/1390807655 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0dc8102340 msgr2=0x7f0dc8074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:36:51.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:50.741+0000 7f0dc7e00640 1 --2- 172.21.15.5:0/1390807655 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0dc8102340 0x7f0dc8074a30 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f0db000e9f0 tx=0x7f0db000eeb0 comp rx=0 tx=0).stop 2024-09-10T09:36:51.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:50.741+0000 7f0dc7400640 1 -- 172.21.15.5:0/1390807655 reap_dead start 2024-09-10T09:36:51.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:50.741+0000 7f0dc7e00640 1 -- 172.21.15.5:0/1390807655 shutdown_connections 2024-09-10T09:36:51.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:50.741+0000 7f0dc7e00640 1 -- 172.21.15.5:0/1390807655 >> 172.21.15.5:0/1390807655 conn(0x7f0dc80fd700 msgr2=0x7f0dc80fee50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:36:51.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:50.741+0000 7f0dc7e00640 1 -- 172.21.15.5:0/1390807655 shutdown_connections 2024-09-10T09:36:51.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:50.741+0000 7f0dc7e00640 1 -- 172.21.15.5:0/1390807655 wait complete. 2024-09-10T09:36:51.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:36:51 smithi005 ceph-mon[30350]: pgmap v346: 129 pgs: 129 active+clean; 645 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 682 B/s wr, 10 op/s 2024-09-10T09:36:51.687 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:36:51 smithi102 ceph-mon[35442]: pgmap v346: 129 pgs: 129 active+clean; 645 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 682 B/s wr, 10 op/s 2024-09-10T09:36:51.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:36:51 smithi159 ceph-mon[35513]: pgmap v346: 129 pgs: 129 active+clean; 645 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 682 B/s wr, 10 op/s 2024-09-10T09:36:52.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:49.751+0000 7f44ee400640 1 -- 172.21.15.5:0/28524944 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f44e810e280 msgr2=0x7f44e8110670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:36:52.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:49.751+0000 7f44ee400640 1 --2- 172.21.15.5:0/28524944 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f44e810e280 0x7f44e8110670 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f44e0009ca0 tx=0x7f44e002a940 comp rx=0 tx=0).stop 2024-09-10T09:36:52.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:49.751+0000 7f44ee400640 1 -- 172.21.15.5:0/28524944 shutdown_connections 2024-09-10T09:36:52.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:49.751+0000 7f44ee400640 1 --2- 172.21.15.5:0/28524944 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f44e810e280 0x7f44e8110670 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:36:52.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:49.751+0000 7f44ee400640 1 --2- 172.21.15.5:0/28524944 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f44e8074e30 0x7f44e810dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:36:52.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:49.751+0000 7f44ee400640 1 --2- 172.21.15.5:0/28524944 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f44e8074510 0x7f44e80748f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:36:52.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:49.751+0000 7f44ee400640 1 -- 172.21.15.5:0/28524944 >> 172.21.15.5:0/28524944 conn(0x7f44e80fd740 msgr2=0x7f44e80ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:36:52.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:49.751+0000 7f44ee400640 1 -- 172.21.15.5:0/28524944 shutdown_connections 2024-09-10T09:36:52.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:49.751+0000 7f44ee400640 1 -- 172.21.15.5:0/28524944 wait complete. 2024-09-10T09:36:52.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:49.752+0000 7f44ee400640 1 Processor -- start 2024-09-10T09:36:52.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:49.752+0000 7f44ee400640 1 -- start start 2024-09-10T09:36:52.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:49.752+0000 7f44ee400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f44e8074510 0x7f44e819a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:36:52.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:49.752+0000 7f44ee400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f44e8074e30 0x7f44e819a980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:36:52.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:49.753+0000 7f44ee400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f44e810e280 0x7f44e819ed10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:36:52.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:49.753+0000 7f44ee400640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f44e8114000 con 0x7f44e8074510 2024-09-10T09:36:52.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:49.753+0000 7f44ee400640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f44e8113e80 con 0x7f44e810e280 2024-09-10T09:36:52.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:49.753+0000 7f44ee400640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f44e8114180 con 0x7f44e8074e30 2024-09-10T09:36:52.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:49.753+0000 7f44dfe00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f44e8074e30 0x7f44e819a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:36:52.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:49.753+0000 7f44ed000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f44e8074510 0x7f44e819a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:36:52.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:49.753+0000 7f44dfe00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f44e8074e30 0x7f44e819a980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:51088/0 (socket says 172.21.15.5:51088) 2024-09-10T09:36:52.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:49.753+0000 7f44dfe00640 1 -- 172.21.15.5:0/3605294543 learned_addr learned my addr 172.21.15.5:0/3605294543 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:36:52.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:49.753+0000 7f44eda00640 1 --2- 172.21.15.5:0/3605294543 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f44e810e280 0x7f44e819ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:36:52.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:49.753+0000 7f44dfe00640 1 -- 172.21.15.5:0/3605294543 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f44e810e280 msgr2=0x7f44e819ed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:36:52.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:49.753+0000 7f44dfe00640 1 --2- 172.21.15.5:0/3605294543 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f44e810e280 0x7f44e819ed10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:36:52.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:49.753+0000 7f44dfe00640 1 -- 172.21.15.5:0/3605294543 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f44e8074510 msgr2=0x7f44e819a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:36:52.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:49.753+0000 7f44dfe00640 1 --2- 172.21.15.5:0/3605294543 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f44e8074510 0x7f44e819a440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:36:52.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:49.753+0000 7f44dfe00640 1 -- 172.21.15.5:0/3605294543 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44e819f3f0 con 0x7f44e8074e30 2024-09-10T09:36:52.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:49.754+0000 7f44dfe00640 1 --2- 172.21.15.5:0/3605294543 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f44e8074e30 0x7f44e819a980 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f44d000ea60 tx=0x7f44d000ef20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:36:52.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:49.754+0000 7f44e6000640 1 -- 172.21.15.5:0/3605294543 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44d000cde0 con 0x7f44e8074e30 2024-09-10T09:36:52.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:49.754+0000 7f44e6000640 1 -- 172.21.15.5:0/3605294543 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f44d00035f0 con 0x7f44e8074e30 2024-09-10T09:36:52.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:49.754+0000 7f44ee400640 1 -- 172.21.15.5:0/3605294543 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44e819f6e0 con 0x7f44e8074e30 2024-09-10T09:36:52.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:49.754+0000 7f44e6000640 1 -- 172.21.15.5:0/3605294543 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44d0010670 con 0x7f44e8074e30 2024-09-10T09:36:52.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:49.754+0000 7f44ee400640 1 -- 172.21.15.5:0/3605294543 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44e81a6fc0 con 0x7f44e8074e30 2024-09-10T09:36:52.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:49.755+0000 7f44ee400640 1 -- 172.21.15.5:0/3605294543 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f44e8062940 con 0x7f44e8074e30 2024-09-10T09:36:52.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:49.757+0000 7f44e6000640 1 -- 172.21.15.5:0/3605294543 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f44d00037a0 con 0x7f44e8074e30 2024-09-10T09:36:52.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:49.758+0000 7f44e6000640 1 --2- 172.21.15.5:0/3605294543 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f44cc0a2c10 0x7f44cc0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:36:52.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:49.758+0000 7f44e6000640 1 -- 172.21.15.5:0/3605294543 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9071+0+0 (secure 0 0 0) 0x7f44d00cfae0 con 0x7f44e8074e30 2024-09-10T09:36:52.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:49.758+0000 7f44ed000640 1 --2- 172.21.15.5:0/3605294543 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f44cc0a2c10 0x7f44cc0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:36:52.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:49.759+0000 7f44e6000640 1 --2- 172.21.15.5:0/3605294543 >> [v2:172.21.15.102:6808/594465753,v1:172.21.15.102:6809/594465753] conn(0x7f44cc0add60 0x7f44cc0b0200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:36:52.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:49.759+0000 7f44e6000640 1 -- 172.21.15.5:0/3605294543 --> [v2:172.21.15.102:6808/594465753,v1:172.21.15.102:6809/594465753] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f44d0023e50 con 0x7f44cc0add60 2024-09-10T09:36:52.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:49.759+0000 7f44e6000640 1 -- 172.21.15.5:0/3605294543 <== mon.1 v2:172.21.15.159:3300/0 6 ==== mon_get_version_reply(handle=1 version=94) ==== 24+0+0 (secure 0 0 0) 0x7f44d00cfe90 con 0x7f44e8074e30 2024-09-10T09:36:52.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:49.759+0000 7f44eda00640 1 --2- 172.21.15.5:0/3605294543 >> [v2:172.21.15.102:6808/594465753,v1:172.21.15.102:6809/594465753] conn(0x7f44cc0add60 0x7f44cc0b0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:36:52.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:49.759+0000 7f44ed000640 1 --2- 172.21.15.5:0/3605294543 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f44cc0a2c10 0x7f44cc0a50d0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f44d8005d00 tx=0x7f44d800b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:36:52.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:49.760+0000 7f44eda00640 1 --2- 172.21.15.5:0/3605294543 >> [v2:172.21.15.102:6808/594465753,v1:172.21.15.102:6809/594465753] conn(0x7f44cc0add60 0x7f44cc0b0200 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:36:52.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:49.761+0000 7f44e6000640 1 -- 172.21.15.5:0/3605294543 <== osd.4 v2:172.21.15.102:6808/594465753 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f44d0023e50 con 0x7f44cc0add60 2024-09-10T09:36:52.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:49.776+0000 7f44ee400640 1 -- 172.21.15.5:0/3605294543 --> [v2:172.21.15.102:6808/594465753,v1:172.21.15.102:6809/594465753] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f44e8062dc0 con 0x7f44cc0add60 2024-09-10T09:36:52.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:49.776+0000 7f44e6000640 1 -- 172.21.15.5:0/3605294543 <== osd.4 v2:172.21.15.102:6808/594465753 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f44e8062dc0 con 0x7f44cc0add60 2024-09-10T09:36:52.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:49.776+0000 7f44ee400640 1 -- 172.21.15.5:0/3605294543 >> [v2:172.21.15.102:6808/594465753,v1:172.21.15.102:6809/594465753] conn(0x7f44cc0add60 msgr2=0x7f44cc0b0200 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:36:52.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:49.776+0000 7f44ee400640 1 --2- 172.21.15.5:0/3605294543 >> [v2:172.21.15.102:6808/594465753,v1:172.21.15.102:6809/594465753] conn(0x7f44cc0add60 0x7f44cc0b0200 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:36:52.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:49.776+0000 7f44ee400640 1 -- 172.21.15.5:0/3605294543 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f44cc0a2c10 msgr2=0x7f44cc0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:36:52.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:49.776+0000 7f44ee400640 1 --2- 172.21.15.5:0/3605294543 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f44cc0a2c10 0x7f44cc0a50d0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f44d8005d00 tx=0x7f44d800b410 comp rx=0 tx=0).stop 2024-09-10T09:36:52.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:49.776+0000 7f44ee400640 1 -- 172.21.15.5:0/3605294543 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f44e8074e30 msgr2=0x7f44e819a980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:36:52.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:49.776+0000 7f44ee400640 1 --2- 172.21.15.5:0/3605294543 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f44e8074e30 0x7f44e819a980 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f44d000ea60 tx=0x7f44d000ef20 comp rx=0 tx=0).stop 2024-09-10T09:36:52.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:49.776+0000 7f44eda00640 1 -- 172.21.15.5:0/3605294543 reap_dead start 2024-09-10T09:36:52.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:49.776+0000 7f44ee400640 1 -- 172.21.15.5:0/3605294543 shutdown_connections 2024-09-10T09:36:52.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:49.776+0000 7f44ee400640 1 -- 172.21.15.5:0/3605294543 >> 172.21.15.5:0/3605294543 conn(0x7f44e80fd740 msgr2=0x7f44e80fe3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:36:52.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:49.776+0000 7f44ee400640 1 -- 172.21.15.5:0/3605294543 shutdown_connections 2024-09-10T09:36:52.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:49.776+0000 7f44ee400640 1 -- 172.21.15.5:0/3605294543 wait complete. 2024-09-10T09:36:52.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:52.761+0000 7ffb3ca00640 1 -- 172.21.15.5:0/3505033781 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffb38074510 msgr2=0x7ffb380748f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:36:52.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:52.761+0000 7ffb3ca00640 1 --2- 172.21.15.5:0/3505033781 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffb38074510 0x7ffb380748f0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7ffb28009900 tx=0x7ffb2802a480 comp rx=0 tx=0).stop 2024-09-10T09:36:52.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:52.761+0000 7ffb3ca00640 1 -- 172.21.15.5:0/3505033781 shutdown_connections 2024-09-10T09:36:52.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:52.761+0000 7ffb3ca00640 1 --2- 172.21.15.5:0/3505033781 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ffb3810e280 0x7ffb38110670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:36:52.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:52.761+0000 7ffb3ca00640 1 --2- 172.21.15.5:0/3505033781 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ffb38074e30 0x7ffb3810dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:36:52.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:52.761+0000 7ffb3ca00640 1 --2- 172.21.15.5:0/3505033781 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffb38074510 0x7ffb380748f0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:36:52.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:52.761+0000 7ffb3ca00640 1 -- 172.21.15.5:0/3505033781 >> 172.21.15.5:0/3505033781 conn(0x7ffb380fd740 msgr2=0x7ffb380ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:36:52.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:52.761+0000 7ffb3ca00640 1 -- 172.21.15.5:0/3505033781 shutdown_connections 2024-09-10T09:36:52.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:52.761+0000 7ffb3ca00640 1 -- 172.21.15.5:0/3505033781 wait complete. 2024-09-10T09:36:52.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:52.762+0000 7ffb3ca00640 1 Processor -- start 2024-09-10T09:36:52.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:52.763+0000 7ffb3ca00640 1 -- start start 2024-09-10T09:36:52.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:52.763+0000 7ffb3ca00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffb38074510 0x7ffb3819a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:36:52.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:52.763+0000 7ffb3ca00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ffb38074e30 0x7ffb3819a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:36:52.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:52.763+0000 7ffb3ca00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ffb3810e280 0x7ffb3819ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:36:52.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:52.763+0000 7ffb3ca00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ffb38113fc0 con 0x7ffb38074510 2024-09-10T09:36:52.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:52.763+0000 7ffb3ca00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7ffb38113e40 con 0x7ffb38074e30 2024-09-10T09:36:52.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:52.763+0000 7ffb3ca00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7ffb38114140 con 0x7ffb3810e280 2024-09-10T09:36:52.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:52.763+0000 7ffb36800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffb38074510 0x7ffb3819a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:36:52.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:52.763+0000 7ffb36800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffb38074510 0x7ffb3819a4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35860/0 (socket says 172.21.15.5:35860) 2024-09-10T09:36:52.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:52.763+0000 7ffb36800640 1 -- 172.21.15.5:0/2282561876 learned_addr learned my addr 172.21.15.5:0/2282561876 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:36:52.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:52.764+0000 7ffb2fe00640 1 --2- 172.21.15.5:0/2282561876 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ffb38074e30 0x7ffb3819a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:36:52.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:52.764+0000 7ffb37200640 1 --2- 172.21.15.5:0/2282561876 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ffb3810e280 0x7ffb3819ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:36:52.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:52.764+0000 7ffb36800640 1 -- 172.21.15.5:0/2282561876 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ffb3810e280 msgr2=0x7ffb3819ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:36:52.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:52.764+0000 7ffb36800640 1 --2- 172.21.15.5:0/2282561876 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ffb3810e280 0x7ffb3819ed70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:36:52.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:52.764+0000 7ffb36800640 1 -- 172.21.15.5:0/2282561876 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ffb38074e30 msgr2=0x7ffb3819a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:36:52.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:52.764+0000 7ffb36800640 1 --2- 172.21.15.5:0/2282561876 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ffb38074e30 0x7ffb3819a9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:36:52.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:52.764+0000 7ffb36800640 1 -- 172.21.15.5:0/2282561876 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb3819f450 con 0x7ffb38074510 2024-09-10T09:36:52.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:52.765+0000 7ffb36800640 1 --2- 172.21.15.5:0/2282561876 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffb38074510 0x7ffb3819a4a0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7ffb28004770 tx=0x7ffb28003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:36:52.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:52.765+0000 7ffb2f400640 1 -- 172.21.15.5:0/2282561876 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb2803f070 con 0x7ffb38074510 2024-09-10T09:36:52.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:52.765+0000 7ffb2f400640 1 -- 172.21.15.5:0/2282561876 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffb28031ca0 con 0x7ffb38074510 2024-09-10T09:36:52.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:52.765+0000 7ffb2f400640 1 -- 172.21.15.5:0/2282561876 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb28035030 con 0x7ffb38074510 2024-09-10T09:36:52.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:52.765+0000 7ffb3ca00640 1 -- 172.21.15.5:0/2282561876 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb3819f6e0 con 0x7ffb38074510 2024-09-10T09:36:52.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:52.765+0000 7ffb3ca00640 1 -- 172.21.15.5:0/2282561876 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb381a7020 con 0x7ffb38074510 2024-09-10T09:36:52.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:52.766+0000 7ffb3ca00640 1 -- 172.21.15.5:0/2282561876 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7ffb38062980 con 0x7ffb38074510 2024-09-10T09:36:52.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:52.768+0000 7ffb2f400640 1 -- 172.21.15.5:0/2282561876 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7ffb28042070 con 0x7ffb38074510 2024-09-10T09:36:52.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:52.770+0000 7ffb2f400640 1 --2- 172.21.15.5:0/2282561876 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7ffb1c0a2c90 0x7ffb1c0a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:36:52.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:52.770+0000 7ffb2f400640 1 -- 172.21.15.5:0/2282561876 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9071+0+0 (secure 0 0 0) 0x7ffb280f0010 con 0x7ffb38074510 2024-09-10T09:36:52.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:52.770+0000 7ffb2fe00640 1 --2- 172.21.15.5:0/2282561876 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7ffb1c0a2c90 0x7ffb1c0a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:36:52.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:52.770+0000 7ffb2f400640 1 --2- 172.21.15.5:0/2282561876 >> [v2:172.21.15.159:6808/3230280402,v1:172.21.15.159:6809/3230280402] conn(0x7ffb1c0adcb0 0x7ffb1c0b0150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:36:52.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:52.770+0000 7ffb2f400640 1 -- 172.21.15.5:0/2282561876 --> [v2:172.21.15.159:6808/3230280402,v1:172.21.15.159:6809/3230280402] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ffb28038e90 con 0x7ffb1c0adcb0 2024-09-10T09:36:52.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:52.770+0000 7ffb2f400640 1 -- 172.21.15.5:0/2282561876 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_get_version_reply(handle=1 version=94) ==== 24+0+0 (secure 0 0 0) 0x7ffb280f0460 con 0x7ffb38074510 2024-09-10T09:36:52.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:52.771+0000 7ffb37200640 1 --2- 172.21.15.5:0/2282561876 >> [v2:172.21.15.159:6808/3230280402,v1:172.21.15.159:6809/3230280402] conn(0x7ffb1c0adcb0 0x7ffb1c0b0150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:36:52.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:52.771+0000 7ffb2fe00640 1 --2- 172.21.15.5:0/2282561876 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7ffb1c0a2c90 0x7ffb1c0a5150 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7ffb200095f0 tx=0x7ffb20005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:36:52.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:52.771+0000 7ffb37200640 1 --2- 172.21.15.5:0/2282561876 >> [v2:172.21.15.159:6808/3230280402,v1:172.21.15.159:6809/3230280402] conn(0x7ffb1c0adcb0 0x7ffb1c0b0150 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:36:52.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:52.772+0000 7ffb2f400640 1 -- 172.21.15.5:0/2282561876 <== osd.5 v2:172.21.15.159:6808/3230280402 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ffb28038e90 con 0x7ffb1c0adcb0 2024-09-10T09:36:52.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:52.787+0000 7ffb3ca00640 1 -- 172.21.15.5:0/2282561876 --> [v2:172.21.15.159:6808/3230280402,v1:172.21.15.159:6809/3230280402] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7ffb38062d80 con 0x7ffb1c0adcb0 2024-09-10T09:36:52.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:52.788+0000 7ffb2f400640 1 -- 172.21.15.5:0/2282561876 <== osd.5 v2:172.21.15.159:6808/3230280402 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7ffb38062d80 con 0x7ffb1c0adcb0 2024-09-10T09:36:52.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:52.789+0000 7ffb3ca00640 1 -- 172.21.15.5:0/2282561876 >> [v2:172.21.15.159:6808/3230280402,v1:172.21.15.159:6809/3230280402] conn(0x7ffb1c0adcb0 msgr2=0x7ffb1c0b0150 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:36:52.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:52.789+0000 7ffb3ca00640 1 --2- 172.21.15.5:0/2282561876 >> [v2:172.21.15.159:6808/3230280402,v1:172.21.15.159:6809/3230280402] conn(0x7ffb1c0adcb0 0x7ffb1c0b0150 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:36:52.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:52.789+0000 7ffb3ca00640 1 -- 172.21.15.5:0/2282561876 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7ffb1c0a2c90 msgr2=0x7ffb1c0a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:36:52.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:52.789+0000 7ffb3ca00640 1 --2- 172.21.15.5:0/2282561876 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7ffb1c0a2c90 0x7ffb1c0a5150 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7ffb200095f0 tx=0x7ffb20005f90 comp rx=0 tx=0).stop 2024-09-10T09:36:52.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:52.789+0000 7ffb3ca00640 1 -- 172.21.15.5:0/2282561876 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffb38074510 msgr2=0x7ffb3819a4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:36:52.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:52.789+0000 7ffb3ca00640 1 --2- 172.21.15.5:0/2282561876 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffb38074510 0x7ffb3819a4a0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7ffb28004770 tx=0x7ffb28003c10 comp rx=0 tx=0).stop 2024-09-10T09:36:52.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:52.789+0000 7ffb37200640 1 -- 172.21.15.5:0/2282561876 reap_dead start 2024-09-10T09:36:52.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:52.789+0000 7ffb3ca00640 1 -- 172.21.15.5:0/2282561876 shutdown_connections 2024-09-10T09:36:52.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:52.789+0000 7ffb3ca00640 1 -- 172.21.15.5:0/2282561876 >> 172.21.15.5:0/2282561876 conn(0x7ffb380fd740 msgr2=0x7ffb380fee20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:36:52.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:52.789+0000 7ffb3ca00640 1 -- 172.21.15.5:0/2282561876 shutdown_connections 2024-09-10T09:36:52.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:52.790+0000 7ffb3ca00640 1 -- 172.21.15.5:0/2282561876 wait complete. 2024-09-10T09:36:53.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:36:53 smithi005 ceph-mon[30350]: pgmap v347: 129 pgs: 129 active+clean; 645 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 682 B/s wr, 12 op/s 2024-09-10T09:36:53.595 INFO:teuthology.orchestra.run.smithi005.stdout:34359738457 2024-09-10T09:36:53.596 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-09-10T09:36:53.851 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:36:53 smithi159 ceph-mon[35513]: pgmap v347: 129 pgs: 129 active+clean; 645 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 682 B/s wr, 12 op/s 2024-09-10T09:36:53.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:36:53 smithi102 ceph-mon[35442]: pgmap v347: 129 pgs: 129 active+clean; 645 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 682 B/s wr, 12 op/s 2024-09-10T09:36:54.243 INFO:teuthology.orchestra.run.smithi005.stdout:244813135918 2024-09-10T09:36:54.243 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.8 2024-09-10T09:36:54.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:52.290+0000 7f0b8ec00640 1 -- 172.21.15.5:0/524965557 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0b88102360 msgr2=0x7f0b8810e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:36:54.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:52.290+0000 7f0b8ec00640 1 --2- 172.21.15.5:0/524965557 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0b88102360 0x7f0b8810e890 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f0b80009cd0 tx=0x7f0b8002a8f0 comp rx=0 tx=0).stop 2024-09-10T09:36:54.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:52.290+0000 7f0b8ec00640 1 -- 172.21.15.5:0/524965557 shutdown_connections 2024-09-10T09:36:54.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:52.290+0000 7f0b8ec00640 1 --2- 172.21.15.5:0/524965557 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0b88102360 0x7f0b8810e890 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:36:54.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:52.290+0000 7f0b8ec00640 1 --2- 172.21.15.5:0/524965557 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0b881019c0 0x7f0b88101e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:36:54.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:52.290+0000 7f0b8ec00640 1 --2- 172.21.15.5:0/524965557 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0b881079c0 0x7f0b88107da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:36:54.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:52.290+0000 7f0b8ec00640 1 -- 172.21.15.5:0/524965557 >> 172.21.15.5:0/524965557 conn(0x7f0b880fd740 msgr2=0x7f0b880ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:36:54.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:52.290+0000 7f0b8ec00640 1 -- 172.21.15.5:0/524965557 shutdown_connections 2024-09-10T09:36:54.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:52.290+0000 7f0b8ec00640 1 -- 172.21.15.5:0/524965557 wait complete. 2024-09-10T09:36:54.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:52.291+0000 7f0b8ec00640 1 Processor -- start 2024-09-10T09:36:54.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:52.291+0000 7f0b8ec00640 1 -- start start 2024-09-10T09:36:54.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:52.291+0000 7f0b8ec00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0b881019c0 0x7f0b880744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:36:54.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:52.291+0000 7f0b8ec00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0b88102360 0x7f0b88074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:36:54.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:52.291+0000 7f0b8ec00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0b881079c0 0x7f0b88074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:36:54.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:52.291+0000 7f0b8ec00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f0b88111dc0 con 0x7f0b881079c0 2024-09-10T09:36:54.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:52.291+0000 7f0b8ec00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f0b88111c40 con 0x7f0b881019c0 2024-09-10T09:36:54.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:52.291+0000 7f0b8ec00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f0b88111f40 con 0x7f0b88102360 2024-09-10T09:36:54.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:52.292+0000 7f0b8e200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0b881079c0 0x7f0b88074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:36:54.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:52.292+0000 7f0b8e200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0b881079c0 0x7f0b88074f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35844/0 (socket says 172.21.15.5:35844) 2024-09-10T09:36:54.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:52.292+0000 7f0b8e200640 1 -- 172.21.15.5:0/129393348 learned_addr learned my addr 172.21.15.5:0/129393348 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:36:54.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:52.292+0000 7f0b8d800640 1 --2- 172.21.15.5:0/129393348 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0b881019c0 0x7f0b880744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:36:54.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:52.292+0000 7f0b8ce00640 1 --2- 172.21.15.5:0/129393348 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0b88102360 0x7f0b88074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:36:54.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:52.292+0000 7f0b8d800640 1 -- 172.21.15.5:0/129393348 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0b88102360 msgr2=0x7f0b88074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:36:54.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:52.292+0000 7f0b8d800640 1 --2- 172.21.15.5:0/129393348 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0b88102360 0x7f0b88074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:36:54.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:52.292+0000 7f0b8d800640 1 -- 172.21.15.5:0/129393348 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0b881079c0 msgr2=0x7f0b88074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:36:54.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:52.292+0000 7f0b8d800640 1 --2- 172.21.15.5:0/129393348 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0b881079c0 0x7f0b88074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:36:54.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:52.292+0000 7f0b8d800640 1 -- 172.21.15.5:0/129393348 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b881a6740 con 0x7f0b881019c0 2024-09-10T09:36:54.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:52.292+0000 7f0b8e200640 1 --2- 172.21.15.5:0/129393348 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0b881079c0 0x7f0b88074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:36:54.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:52.293+0000 7f0b8d800640 1 --2- 172.21.15.5:0/129393348 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0b881019c0 0x7f0b880744f0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f0b7c00d6d0 tx=0x7f0b7c00db90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:36:54.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:52.293+0000 7f0b86000640 1 -- 172.21.15.5:0/129393348 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b7c017070 con 0x7f0b881019c0 2024-09-10T09:36:54.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:52.293+0000 7f0b8ec00640 1 -- 172.21.15.5:0/129393348 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b881a6a30 con 0x7f0b881019c0 2024-09-10T09:36:54.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:52.293+0000 7f0b86000640 1 -- 172.21.15.5:0/129393348 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0b7c009820 con 0x7f0b881019c0 2024-09-10T09:36:54.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:52.293+0000 7f0b86000640 1 -- 172.21.15.5:0/129393348 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b7c004cd0 con 0x7f0b881019c0 2024-09-10T09:36:54.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:52.293+0000 7f0b8ec00640 1 -- 172.21.15.5:0/129393348 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b881a6fc0 con 0x7f0b881019c0 2024-09-10T09:36:54.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:52.294+0000 7f0b8ec00640 1 -- 172.21.15.5:0/129393348 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f0b88062900 con 0x7f0b881019c0 2024-09-10T09:36:54.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:52.296+0000 7f0b86000640 1 -- 172.21.15.5:0/129393348 <== mon.2 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f0b7c0099d0 con 0x7f0b881019c0 2024-09-10T09:36:54.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:52.298+0000 7f0b86000640 1 --2- 172.21.15.5:0/129393348 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f0b700a2c10 0x7f0b700a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:36:54.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:52.298+0000 7f0b86000640 1 -- 172.21.15.5:0/129393348 <== mon.2 v2:172.21.15.102:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9071+0+0 (secure 0 0 0) 0x7f0b7c0cf390 con 0x7f0b881019c0 2024-09-10T09:36:54.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:52.298+0000 7f0b8ce00640 1 --2- 172.21.15.5:0/129393348 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f0b700a2c10 0x7f0b700a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:36:54.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:52.298+0000 7f0b86000640 1 --2- 172.21.15.5:0/129393348 >> [v2:172.21.15.159:6824/969941218,v1:172.21.15.159:6825/969941218] conn(0x7f0b700add60 0x7f0b700b0200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:36:54.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:52.298+0000 7f0b86000640 1 -- 172.21.15.5:0/129393348 --> [v2:172.21.15.159:6824/969941218,v1:172.21.15.159:6825/969941218] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f0b7c004e70 con 0x7f0b700add60 2024-09-10T09:36:54.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:52.298+0000 7f0b86000640 1 -- 172.21.15.5:0/129393348 <== mon.2 v2:172.21.15.102:3300/0 6 ==== mon_get_version_reply(handle=1 version=94) ==== 24+0+0 (secure 0 0 0) 0x7f0b7c0cf7e0 con 0x7f0b881019c0 2024-09-10T09:36:54.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:52.299+0000 7f0b8e200640 1 --2- 172.21.15.5:0/129393348 >> [v2:172.21.15.159:6824/969941218,v1:172.21.15.159:6825/969941218] conn(0x7f0b700add60 0x7f0b700b0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:36:54.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:52.299+0000 7f0b8ce00640 1 --2- 172.21.15.5:0/129393348 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f0b700a2c10 0x7f0b700a50d0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f0b88075f80 tx=0x7f0b7400b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:36:54.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:52.299+0000 7f0b8e200640 1 --2- 172.21.15.5:0/129393348 >> [v2:172.21.15.159:6824/969941218,v1:172.21.15.159:6825/969941218] conn(0x7f0b700add60 0x7f0b700b0200 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:36:54.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:52.300+0000 7f0b86000640 1 -- 172.21.15.5:0/129393348 <== osd.11 v2:172.21.15.159:6824/969941218 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f0b7c004e70 con 0x7f0b700add60 2024-09-10T09:36:54.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:52.315+0000 7f0b8ec00640 1 -- 172.21.15.5:0/129393348 --> [v2:172.21.15.159:6824/969941218,v1:172.21.15.159:6825/969941218] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f0b88062d00 con 0x7f0b700add60 2024-09-10T09:36:54.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:52.315+0000 7f0b86000640 1 -- 172.21.15.5:0/129393348 <== osd.11 v2:172.21.15.159:6824/969941218 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f0b88062d00 con 0x7f0b700add60 2024-09-10T09:36:54.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:52.316+0000 7f0b8ec00640 1 -- 172.21.15.5:0/129393348 >> [v2:172.21.15.159:6824/969941218,v1:172.21.15.159:6825/969941218] conn(0x7f0b700add60 msgr2=0x7f0b700b0200 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:36:54.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:52.316+0000 7f0b8ec00640 1 --2- 172.21.15.5:0/129393348 >> [v2:172.21.15.159:6824/969941218,v1:172.21.15.159:6825/969941218] conn(0x7f0b700add60 0x7f0b700b0200 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:36:54.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:52.316+0000 7f0b8ec00640 1 -- 172.21.15.5:0/129393348 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f0b700a2c10 msgr2=0x7f0b700a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:36:54.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:52.316+0000 7f0b8ec00640 1 --2- 172.21.15.5:0/129393348 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f0b700a2c10 0x7f0b700a50d0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f0b88075f80 tx=0x7f0b7400b390 comp rx=0 tx=0).stop 2024-09-10T09:36:54.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:52.316+0000 7f0b8ec00640 1 -- 172.21.15.5:0/129393348 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0b881019c0 msgr2=0x7f0b880744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:36:54.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:52.316+0000 7f0b8ec00640 1 --2- 172.21.15.5:0/129393348 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0b881019c0 0x7f0b880744f0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f0b7c00d6d0 tx=0x7f0b7c00db90 comp rx=0 tx=0).stop 2024-09-10T09:36:54.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:52.316+0000 7f0b8e200640 1 -- 172.21.15.5:0/129393348 reap_dead start 2024-09-10T09:36:54.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:52.316+0000 7f0b8ec00640 1 -- 172.21.15.5:0/129393348 shutdown_connections 2024-09-10T09:36:54.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:52.316+0000 7f0b8ec00640 1 -- 172.21.15.5:0/129393348 >> 172.21.15.5:0/129393348 conn(0x7f0b880fd740 msgr2=0x7f0b880fdfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:36:54.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:52.316+0000 7f0b8ec00640 1 -- 172.21.15.5:0/129393348 shutdown_connections 2024-09-10T09:36:54.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:52.316+0000 7f0b8ec00640 1 -- 172.21.15.5:0/129393348 wait complete. 2024-09-10T09:36:54.944 INFO:teuthology.orchestra.run.smithi005.stdout:55834574933 2024-09-10T09:36:54.949 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-09-10T09:36:55.407 INFO:teuthology.orchestra.run.smithi005.stdout:137438953541 2024-09-10T09:36:55.408 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-09-10T09:36:55.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:36:55 smithi005 ceph-mon[30350]: pgmap v348: 129 pgs: 129 active+clean; 645 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-10T09:36:55.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:36:55 smithi159 ceph-mon[35513]: pgmap v348: 129 pgs: 129 active+clean; 645 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-10T09:36:55.877 INFO:teuthology.orchestra.run.smithi005.stdout:330712481823 2024-09-10T09:36:55.878 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.11 2024-09-10T09:36:55.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:36:55 smithi102 ceph-mon[35442]: pgmap v348: 129 pgs: 129 active+clean; 645 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-10T09:36:56.170 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:36:56.200 INFO:teuthology.orchestra.run.smithi005.stdout:158913790016 2024-09-10T09:36:56.201 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-09-10T09:36:56.551 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:36:56.552 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:36:57.129 INFO:teuthology.orchestra.run.smithi005.stdout:214748364851 2024-09-10T09:36:57.129 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-09-10T09:36:57.372 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:36:57.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:57.373+0000 7fa8f9e00640 1 -- 172.21.15.5:0/378619428 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa8f41049b0 msgr2=0x7fa8f4108a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:36:57.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:57.373+0000 7fa8f9e00640 1 --2- 172.21.15.5:0/378619428 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa8f41049b0 0x7fa8f4108a00 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fa8e0009900 tx=0x7fa8e002a410 comp rx=0 tx=0).stop 2024-09-10T09:36:57.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:57.376+0000 7fa8f9e00640 1 -- 172.21.15.5:0/378619428 shutdown_connections 2024-09-10T09:36:57.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:57.376+0000 7fa8f9e00640 1 --2- 172.21.15.5:0/378619428 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa8f4109130 0x7fa8f4110cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:36:57.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:57.376+0000 7fa8f9e00640 1 --2- 172.21.15.5:0/378619428 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa8f41049b0 0x7fa8f4108a00 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:36:57.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:57.376+0000 7fa8f9e00640 1 --2- 172.21.15.5:0/378619428 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa8f4104000 0x7fa8f41043e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:36:57.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:57.376+0000 7fa8f9e00640 1 -- 172.21.15.5:0/378619428 >> 172.21.15.5:0/378619428 conn(0x7fa8f40ffa20 msgr2=0x7fa8f4101e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:36:57.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:57.376+0000 7fa8f9e00640 1 -- 172.21.15.5:0/378619428 shutdown_connections 2024-09-10T09:36:57.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:57.376+0000 7fa8f9e00640 1 -- 172.21.15.5:0/378619428 wait complete. 2024-09-10T09:36:57.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:57.378+0000 7fa8f9e00640 1 Processor -- start 2024-09-10T09:36:57.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:57.378+0000 7fa8f9e00640 1 -- start start 2024-09-10T09:36:57.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:57.378+0000 7fa8f9e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa8f4104000 0x7fa8f411a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:36:57.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:57.378+0000 7fa8f9e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa8f41049b0 0x7fa8f4110050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:36:57.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:57.378+0000 7fa8f9e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa8f4109130 0x7fa8f4110590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:36:57.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:57.379+0000 7fa8f2a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa8f4104000 0x7fa8f411a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:36:57.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:57.379+0000 7fa8f2a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa8f4104000 0x7fa8f411a2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35870/0 (socket says 172.21.15.5:35870) 2024-09-10T09:36:57.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:57.379+0000 7fa8f2000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa8f41049b0 0x7fa8f4110050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:36:57.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:57.379+0000 7fa8f3400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa8f4109130 0x7fa8f4110590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:36:57.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:57.379+0000 7fa8f3400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa8f4109130 0x7fa8f4110590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:36236/0 (socket says 172.21.15.5:36236) 2024-09-10T09:36:57.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:57.379+0000 7fa8f3400640 1 -- 172.21.15.5:0/702626516 learned_addr learned my addr 172.21.15.5:0/702626516 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:36:57.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:57.380+0000 7fa8f9e00640 1 -- 172.21.15.5:0/702626516 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa8f4078390 con 0x7fa8f4104000 2024-09-10T09:36:57.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:57.380+0000 7fa8f9e00640 1 -- 172.21.15.5:0/702626516 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fa8f4078210 con 0x7fa8f41049b0 2024-09-10T09:36:57.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:57.380+0000 7fa8f9e00640 1 -- 172.21.15.5:0/702626516 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fa8f4078510 con 0x7fa8f4109130 2024-09-10T09:36:57.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:57.382+0000 7fa8f3400640 1 -- 172.21.15.5:0/702626516 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa8f41049b0 msgr2=0x7fa8f4110050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:36:57.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:57.382+0000 7fa8f3400640 1 --2- 172.21.15.5:0/702626516 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa8f41049b0 0x7fa8f4110050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:36:57.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:57.382+0000 7fa8f3400640 1 -- 172.21.15.5:0/702626516 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa8f4104000 msgr2=0x7fa8f411a2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:36:57.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:57.382+0000 7fa8f3400640 1 --2- 172.21.15.5:0/702626516 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa8f4104000 0x7fa8f411a2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:36:57.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:57.382+0000 7fa8f3400640 1 -- 172.21.15.5:0/702626516 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8f4110cc0 con 0x7fa8f4109130 2024-09-10T09:36:57.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:57.382+0000 7fa8f2000640 1 --2- 172.21.15.5:0/702626516 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa8f41049b0 0x7fa8f4110050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:36:57.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:57.382+0000 7fa8f2a00640 1 --2- 172.21.15.5:0/702626516 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa8f4104000 0x7fa8f411a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:36:57.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:57.383+0000 7fa8f3400640 1 --2- 172.21.15.5:0/702626516 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa8f4109130 0x7fa8f4110590 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fa8ec00cce0 tx=0x7fa8ec00a620 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:36:57.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:57.383+0000 7fa8f8a00640 1 -- 172.21.15.5:0/702626516 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8ec00f450 con 0x7fa8f4109130 2024-09-10T09:36:57.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:57.383+0000 7fa8f8a00640 1 -- 172.21.15.5:0/702626516 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa8ec00fad0 con 0x7fa8f4109130 2024-09-10T09:36:57.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:57.384+0000 7fa8f8a00640 1 -- 172.21.15.5:0/702626516 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8ec018c10 con 0x7fa8f4109130 2024-09-10T09:36:57.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:57.384+0000 7fa8f9e00640 1 -- 172.21.15.5:0/702626516 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8f4115d10 con 0x7fa8f4109130 2024-09-10T09:36:57.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:57.384+0000 7fa8f9e00640 1 -- 172.21.15.5:0/702626516 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8f41161d0 con 0x7fa8f4109130 2024-09-10T09:36:57.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:57.386+0000 7fa8f9e00640 1 -- 172.21.15.5:0/702626516 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fa8d4001030 con 0x7fa8f4109130 2024-09-10T09:36:57.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:57.392+0000 7fa8f8a00640 1 -- 172.21.15.5:0/702626516 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7fa8ec018400 con 0x7fa8f4109130 2024-09-10T09:36:57.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:57.395+0000 7fa8f8a00640 1 --2- 172.21.15.5:0/702626516 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fa8d80a2d60 0x7fa8d80a5220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:36:57.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:57.395+0000 7fa8f8a00640 1 -- 172.21.15.5:0/702626516 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9071+0+0 (secure 0 0 0) 0x7fa8ec0cfb40 con 0x7fa8f4109130 2024-09-10T09:36:57.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:57.396+0000 7fa8f2a00640 1 --2- 172.21.15.5:0/702626516 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fa8d80a2d60 0x7fa8d80a5220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:36:57.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:57.396+0000 7fa8f8a00640 1 --2- 172.21.15.5:0/702626516 >> [v2:172.21.15.159:6800/2236204189,v1:172.21.15.159:6801/2236204189] conn(0x7fa8d80adeb0 0x7fa8d80b0350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:36:57.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:57.396+0000 7fa8f8a00640 1 -- 172.21.15.5:0/702626516 --> [v2:172.21.15.159:6800/2236204189,v1:172.21.15.159:6801/2236204189] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa8ec012ec0 con 0x7fa8d80adeb0 2024-09-10T09:36:57.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:57.396+0000 7fa8f8a00640 1 -- 172.21.15.5:0/702626516 <== mon.1 v2:172.21.15.159:3300/0 6 ==== mon_get_version_reply(handle=1 version=94) ==== 24+0+0 (secure 0 0 0) 0x7fa8ec0cff90 con 0x7fa8f4109130 2024-09-10T09:36:57.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:57.397+0000 7fa8f2a00640 1 --2- 172.21.15.5:0/702626516 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fa8d80a2d60 0x7fa8d80a5220 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fa8e400abe0 tx=0x7fa8e400a5c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:36:57.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:57.397+0000 7fa8f2000640 1 --2- 172.21.15.5:0/702626516 >> [v2:172.21.15.159:6800/2236204189,v1:172.21.15.159:6801/2236204189] conn(0x7fa8d80adeb0 0x7fa8d80b0350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:36:57.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:57.399+0000 7fa8f2000640 1 --2- 172.21.15.5:0/702626516 >> [v2:172.21.15.159:6800/2236204189,v1:172.21.15.159:6801/2236204189] conn(0x7fa8d80adeb0 0x7fa8d80b0350 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:36:57.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:57.400+0000 7fa8f8a00640 1 -- 172.21.15.5:0/702626516 <== osd.2 v2:172.21.15.159:6800/2236204189 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa8ec012ec0 con 0x7fa8d80adeb0 2024-09-10T09:36:57.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:57.417+0000 7fa8f9e00640 1 -- 172.21.15.5:0/702626516 --> [v2:172.21.15.159:6800/2236204189,v1:172.21.15.159:6801/2236204189] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fa8d4002dd0 con 0x7fa8d80adeb0 2024-09-10T09:36:57.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:57.417+0000 7fa8f8a00640 1 -- 172.21.15.5:0/702626516 <== osd.2 v2:172.21.15.159:6800/2236204189 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fa8d4002dd0 con 0x7fa8d80adeb0 2024-09-10T09:36:57.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:57.418+0000 7fa8f9e00640 1 -- 172.21.15.5:0/702626516 >> [v2:172.21.15.159:6800/2236204189,v1:172.21.15.159:6801/2236204189] conn(0x7fa8d80adeb0 msgr2=0x7fa8d80b0350 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:36:57.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:57.418+0000 7fa8f9e00640 1 --2- 172.21.15.5:0/702626516 >> [v2:172.21.15.159:6800/2236204189,v1:172.21.15.159:6801/2236204189] conn(0x7fa8d80adeb0 0x7fa8d80b0350 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:36:57.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:57.418+0000 7fa8f9e00640 1 -- 172.21.15.5:0/702626516 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fa8d80a2d60 msgr2=0x7fa8d80a5220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:36:57.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:57.418+0000 7fa8f9e00640 1 --2- 172.21.15.5:0/702626516 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fa8d80a2d60 0x7fa8d80a5220 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fa8e400abe0 tx=0x7fa8e400a5c0 comp rx=0 tx=0).stop 2024-09-10T09:36:57.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:57.418+0000 7fa8f9e00640 1 -- 172.21.15.5:0/702626516 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa8f4109130 msgr2=0x7fa8f4110590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:36:57.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:57.418+0000 7fa8f9e00640 1 --2- 172.21.15.5:0/702626516 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa8f4109130 0x7fa8f4110590 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fa8ec00cce0 tx=0x7fa8ec00a620 comp rx=0 tx=0).stop 2024-09-10T09:36:57.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:57.418+0000 7fa8f3400640 1 -- 172.21.15.5:0/702626516 reap_dead start 2024-09-10T09:36:57.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:57.419+0000 7fa8f9e00640 1 -- 172.21.15.5:0/702626516 shutdown_connections 2024-09-10T09:36:57.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:57.419+0000 7fa8f9e00640 1 -- 172.21.15.5:0/702626516 >> 172.21.15.5:0/702626516 conn(0x7fa8f40ffa20 msgr2=0x7fa8f4079aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:36:57.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:57.419+0000 7fa8f9e00640 1 -- 172.21.15.5:0/702626516 shutdown_connections 2024-09-10T09:36:57.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:57.419+0000 7fa8f9e00640 1 -- 172.21.15.5:0/702626516 wait complete. 2024-09-10T09:36:57.655 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:36:57 smithi005 ceph-mon[30350]: pgmap v349: 129 pgs: 129 active+clean; 645 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-10T09:36:57.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:36:57 smithi159 ceph-mon[35513]: pgmap v349: 129 pgs: 129 active+clean; 645 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-10T09:36:57.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:36:57 smithi102 ceph-mon[35442]: pgmap v349: 129 pgs: 129 active+clean; 645 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-10T09:36:58.722 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:36:58.726 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:36:58.727 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:36:58.730 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:36:58.777 INFO:teuthology.orchestra.run.smithi005.stdout:77309411408 2024-09-10T09:36:58.777 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-09-10T09:36:59.175 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:36:59.183 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:36:59.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:59.190+0000 7f917b000640 1 -- 172.21.15.5:0/335025329 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9174106f10 msgr2=0x7f917410ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:36:59.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:59.190+0000 7f917b000640 1 --2- 172.21.15.5:0/335025329 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9174106f10 0x7f917410ea90 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f916c009be0 tx=0x7f916c02a6c0 comp rx=0 tx=0).stop 2024-09-10T09:36:59.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:59.191+0000 7f917b000640 1 -- 172.21.15.5:0/335025329 shutdown_connections 2024-09-10T09:36:59.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:59.191+0000 7f917b000640 1 --2- 172.21.15.5:0/335025329 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9174106f10 0x7f917410ea90 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:36:59.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:59.191+0000 7f917b000640 1 --2- 172.21.15.5:0/335025329 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9174102790 0x7f91741067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:36:59.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:59.191+0000 7f917b000640 1 --2- 172.21.15.5:0/335025329 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9174101de0 0x7f91741021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:36:59.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:59.191+0000 7f917b000640 1 -- 172.21.15.5:0/335025329 >> 172.21.15.5:0/335025329 conn(0x7f91740fd8c0 msgr2=0x7f91740ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:36:59.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:59.191+0000 7f917b000640 1 -- 172.21.15.5:0/335025329 shutdown_connections 2024-09-10T09:36:59.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:59.191+0000 7f917b000640 1 -- 172.21.15.5:0/335025329 wait complete. 2024-09-10T09:36:59.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:59.192+0000 7f917b000640 1 Processor -- start 2024-09-10T09:36:59.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:59.192+0000 7f917b000640 1 -- start start 2024-09-10T09:36:59.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:59.193+0000 7f917b000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9174101de0 0x7f91741a46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:36:59.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:59.193+0000 7f917b000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9174102790 0x7f91741a4be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:36:59.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:59.193+0000 7f917b000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9174106f10 0x7f917410dc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:36:59.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:59.193+0000 7f917b000640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f9174111f70 con 0x7f9174106f10 2024-09-10T09:36:59.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:59.193+0000 7f917b000640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f9174111df0 con 0x7f9174101de0 2024-09-10T09:36:59.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:59.193+0000 7f917b000640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f91741120f0 con 0x7f9174102790 2024-09-10T09:36:59.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:59.193+0000 7f9173e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9174101de0 0x7f91741a46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:36:59.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:59.194+0000 7f9173e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9174101de0 0x7f91741a46a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:45110/0 (socket says 172.21.15.5:45110) 2024-09-10T09:36:59.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:59.194+0000 7f9173e00640 1 -- 172.21.15.5:0/3860809846 learned_addr learned my addr 172.21.15.5:0/3860809846 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:36:59.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:59.194+0000 7f9173400640 1 --2- 172.21.15.5:0/3860809846 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9174102790 0x7f91741a4be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:36:59.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:59.194+0000 7f9179000640 1 --2- 172.21.15.5:0/3860809846 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9174106f10 0x7f917410dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:36:59.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:59.194+0000 7f9173e00640 1 -- 172.21.15.5:0/3860809846 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9174102790 msgr2=0x7f91741a4be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:36:59.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:59.194+0000 7f9173e00640 1 --2- 172.21.15.5:0/3860809846 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9174102790 0x7f91741a4be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:36:59.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:59.194+0000 7f9173e00640 1 -- 172.21.15.5:0/3860809846 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9174106f10 msgr2=0x7f917410dc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:36:59.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:59.194+0000 7f9173e00640 1 --2- 172.21.15.5:0/3860809846 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9174106f10 0x7f917410dc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:36:59.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:59.194+0000 7f9173e00640 1 -- 172.21.15.5:0/3860809846 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f917410e3d0 con 0x7f9174101de0 2024-09-10T09:36:59.196 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:36:59.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:59.195+0000 7f9173e00640 1 --2- 172.21.15.5:0/3860809846 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9174101de0 0x7f91741a46a0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f916400e9c0 tx=0x7f916400ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:36:59.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:59.195+0000 7f9179000640 1 --2- 172.21.15.5:0/3860809846 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9174106f10 0x7f917410dc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T09:36:59.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:59.195+0000 7f9170c00640 1 -- 172.21.15.5:0/3860809846 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f916400cde0 con 0x7f9174101de0 2024-09-10T09:36:59.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:59.195+0000 7f917b000640 1 -- 172.21.15.5:0/3860809846 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f917410e6c0 con 0x7f9174101de0 2024-09-10T09:36:59.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:59.195+0000 7f9170c00640 1 -- 172.21.15.5:0/3860809846 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f91640035f0 con 0x7f9174101de0 2024-09-10T09:36:59.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:59.195+0000 7f9170c00640 1 -- 172.21.15.5:0/3860809846 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9164010690 con 0x7f9174101de0 2024-09-10T09:36:59.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:59.195+0000 7f917b000640 1 -- 172.21.15.5:0/3860809846 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91741ab500 con 0x7f9174101de0 2024-09-10T09:36:59.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:59.197+0000 7f917b000640 1 -- 172.21.15.5:0/3860809846 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f91740feee0 con 0x7f9174101de0 2024-09-10T09:36:59.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:59.199+0000 7f9170c00640 1 -- 172.21.15.5:0/3860809846 <== mon.2 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f91640037a0 con 0x7f9174101de0 2024-09-10T09:36:59.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:59.202+0000 7f9170c00640 1 --2- 172.21.15.5:0/3860809846 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f91580a2c10 0x7f91580a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:36:59.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:59.202+0000 7f9170c00640 1 -- 172.21.15.5:0/3860809846 <== mon.2 v2:172.21.15.102:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9071+0+0 (secure 0 0 0) 0x7f91640cf8c0 con 0x7f9174101de0 2024-09-10T09:36:59.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:59.202+0000 7f9170c00640 1 --2- 172.21.15.5:0/3860809846 >> [v2:172.21.15.5:6818/1907651140,v1:172.21.15.5:6819/1907651140] conn(0x7f91580add60 0x7f91580b0200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:36:59.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:59.202+0000 7f9170c00640 1 -- 172.21.15.5:0/3860809846 --> [v2:172.21.15.5:6818/1907651140,v1:172.21.15.5:6819/1907651140] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f9164018e50 con 0x7f91580add60 2024-09-10T09:36:59.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:59.202+0000 7f9170c00640 1 -- 172.21.15.5:0/3860809846 <== mon.2 v2:172.21.15.102:3300/0 6 ==== mon_get_version_reply(handle=1 version=94) ==== 24+0+0 (secure 0 0 0) 0x7f91640d42c0 con 0x7f9174101de0 2024-09-10T09:36:59.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:59.203+0000 7f9173400640 1 --2- 172.21.15.5:0/3860809846 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f91580a2c10 0x7f91580a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:36:59.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:59.204+0000 7f9179000640 1 --2- 172.21.15.5:0/3860809846 >> [v2:172.21.15.5:6818/1907651140,v1:172.21.15.5:6819/1907651140] conn(0x7f91580add60 0x7f91580b0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:36:59.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:59.204+0000 7f9173400640 1 --2- 172.21.15.5:0/3860809846 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f91580a2c10 0x7f91580a50d0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f9174076510 tx=0x7f9160005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:36:59.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:59.206+0000 7f9179000640 1 --2- 172.21.15.5:0/3860809846 >> [v2:172.21.15.5:6818/1907651140,v1:172.21.15.5:6819/1907651140] conn(0x7f91580add60 0x7f91580b0200 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:36:59.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:59.207+0000 7f9170c00640 1 -- 172.21.15.5:0/3860809846 <== osd.6 v2:172.21.15.5:6818/1907651140 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f9164018e50 con 0x7f91580add60 2024-09-10T09:36:59.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:59.236+0000 7f917b000640 1 -- 172.21.15.5:0/3860809846 --> [v2:172.21.15.5:6818/1907651140,v1:172.21.15.5:6819/1907651140] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f91741032c0 con 0x7f91580add60 2024-09-10T09:36:59.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:59.237+0000 7f9170c00640 1 -- 172.21.15.5:0/3860809846 <== osd.6 v2:172.21.15.5:6818/1907651140 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f91741032c0 con 0x7f91580add60 2024-09-10T09:36:59.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:59.238+0000 7f917b000640 1 -- 172.21.15.5:0/3860809846 >> [v2:172.21.15.5:6818/1907651140,v1:172.21.15.5:6819/1907651140] conn(0x7f91580add60 msgr2=0x7f91580b0200 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:36:59.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:59.238+0000 7f917b000640 1 --2- 172.21.15.5:0/3860809846 >> [v2:172.21.15.5:6818/1907651140,v1:172.21.15.5:6819/1907651140] conn(0x7f91580add60 0x7f91580b0200 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:36:59.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:59.238+0000 7f917b000640 1 -- 172.21.15.5:0/3860809846 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f91580a2c10 msgr2=0x7f91580a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:36:59.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:59.238+0000 7f917b000640 1 --2- 172.21.15.5:0/3860809846 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f91580a2c10 0x7f91580a50d0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f9174076510 tx=0x7f9160005f90 comp rx=0 tx=0).stop 2024-09-10T09:36:59.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:59.238+0000 7f917b000640 1 -- 172.21.15.5:0/3860809846 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9174101de0 msgr2=0x7f91741a46a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:36:59.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:59.238+0000 7f917b000640 1 --2- 172.21.15.5:0/3860809846 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9174101de0 0x7f91741a46a0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f916400e9c0 tx=0x7f916400ee80 comp rx=0 tx=0).stop 2024-09-10T09:36:59.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:59.238+0000 7f9179000640 1 -- 172.21.15.5:0/3860809846 reap_dead start 2024-09-10T09:36:59.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:59.239+0000 7f917b000640 1 -- 172.21.15.5:0/3860809846 shutdown_connections 2024-09-10T09:36:59.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:59.239+0000 7f917b000640 1 -- 172.21.15.5:0/3860809846 >> 172.21.15.5:0/3860809846 conn(0x7f91740fd8c0 msgr2=0x7f91740ffca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:36:59.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:59.239+0000 7f917b000640 1 -- 172.21.15.5:0/3860809846 shutdown_connections 2024-09-10T09:36:59.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:59.239+0000 7f917b000640 1 -- 172.21.15.5:0/3860809846 wait complete. 2024-09-10T09:36:59.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:36:59 smithi005 ceph-mon[30350]: pgmap v350: 129 pgs: 129 active+clean; 645 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-10T09:36:59.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:36:59 smithi159 ceph-mon[35513]: pgmap v350: 129 pgs: 129 active+clean; 645 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-10T09:36:59.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:36:59 smithi102 ceph-mon[35442]: pgmap v350: 129 pgs: 129 active+clean; 645 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-10T09:36:59.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:58.944+0000 7f04a6000640 1 -- 172.21.15.5:0/2217750127 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f04a0074ee0 msgr2=0x7f04a010dd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:36:59.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:58.944+0000 7f04a6000640 1 --2- 172.21.15.5:0/2217750127 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f04a0074ee0 0x7f04a010dd20 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f048c009900 tx=0x7f048c02a480 comp rx=0 tx=0).stop 2024-09-10T09:36:59.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:58.944+0000 7f04a6000640 1 -- 172.21.15.5:0/2217750127 shutdown_connections 2024-09-10T09:36:59.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:58.944+0000 7f04a6000640 1 --2- 172.21.15.5:0/2217750127 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f04a010e260 0x7f04a0110650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:36:59.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:58.944+0000 7f04a6000640 1 --2- 172.21.15.5:0/2217750127 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f04a0074ee0 0x7f04a010dd20 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:36:59.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:58.944+0000 7f04a6000640 1 --2- 172.21.15.5:0/2217750127 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f04a00745c0 0x7f04a00749a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:36:59.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:58.944+0000 7f04a6000640 1 -- 172.21.15.5:0/2217750127 >> 172.21.15.5:0/2217750127 conn(0x7f04a00fd740 msgr2=0x7f04a00ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:36:59.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:58.945+0000 7f04a6000640 1 -- 172.21.15.5:0/2217750127 shutdown_connections 2024-09-10T09:36:59.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:58.945+0000 7f04a6000640 1 -- 172.21.15.5:0/2217750127 wait complete. 2024-09-10T09:36:59.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:58.945+0000 7f04a6000640 1 Processor -- start 2024-09-10T09:36:59.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:58.946+0000 7f04a6000640 1 -- start start 2024-09-10T09:36:59.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:58.946+0000 7f04a6000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f04a00745c0 0x7f04a019a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:36:59.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:58.946+0000 7f04a6000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f04a0074ee0 0x7f04a019aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:36:59.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:58.946+0000 7f04a6000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f04a010e260 0x7f04a019edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:36:59.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:58.946+0000 7f04a6000640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f04a0114000 con 0x7f04a00745c0 2024-09-10T09:36:59.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:58.946+0000 7f04a6000640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f04a0113e80 con 0x7f04a0074ee0 2024-09-10T09:36:59.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:58.946+0000 7f04a6000640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f04a0114180 con 0x7f04a010e260 2024-09-10T09:36:59.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:58.946+0000 7f04a4c00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f04a00745c0 0x7f04a019a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:36:59.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:58.946+0000 7f04a4c00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f04a00745c0 0x7f04a019a4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35894/0 (socket says 172.21.15.5:35894) 2024-09-10T09:36:59.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:58.946+0000 7f04a4c00640 1 -- 172.21.15.5:0/787574535 learned_addr learned my addr 172.21.15.5:0/787574535 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:36:59.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:58.946+0000 7f049fe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f04a0074ee0 0x7f04a019aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:36:59.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:58.946+0000 7f04a5600640 1 --2- 172.21.15.5:0/787574535 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f04a010e260 0x7f04a019edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:36:59.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:58.947+0000 7f04a4c00640 1 -- 172.21.15.5:0/787574535 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f04a010e260 msgr2=0x7f04a019edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:36:59.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:58.947+0000 7f04a4c00640 1 --2- 172.21.15.5:0/787574535 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f04a010e260 0x7f04a019edb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:36:59.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:58.947+0000 7f04a4c00640 1 -- 172.21.15.5:0/787574535 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f04a0074ee0 msgr2=0x7f04a019aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:36:59.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:58.947+0000 7f04a4c00640 1 --2- 172.21.15.5:0/787574535 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f04a0074ee0 0x7f04a019aa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:36:59.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:58.947+0000 7f04a4c00640 1 -- 172.21.15.5:0/787574535 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04a019f490 con 0x7f04a00745c0 2024-09-10T09:36:59.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:58.947+0000 7f04a4c00640 1 --2- 172.21.15.5:0/787574535 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f04a00745c0 0x7f04a019a4e0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f049400d950 tx=0x7f049400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:36:59.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:58.947+0000 7f049d600640 1 -- 172.21.15.5:0/787574535 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0494017030 con 0x7f04a00745c0 2024-09-10T09:36:59.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:58.947+0000 7f049d600640 1 -- 172.21.15.5:0/787574535 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0494004020 con 0x7f04a00745c0 2024-09-10T09:36:59.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:58.947+0000 7f04a6000640 1 -- 172.21.15.5:0/787574535 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04a019f780 con 0x7f04a00745c0 2024-09-10T09:36:59.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:58.948+0000 7f04a6000640 1 -- 172.21.15.5:0/787574535 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04a01a7060 con 0x7f04a00745c0 2024-09-10T09:36:59.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:58.948+0000 7f049d600640 1 -- 172.21.15.5:0/787574535 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04940047a0 con 0x7f04a00745c0 2024-09-10T09:36:59.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:58.949+0000 7f04a6000640 1 -- 172.21.15.5:0/787574535 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f04a0062980 con 0x7f04a00745c0 2024-09-10T09:36:59.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:58.951+0000 7f049d600640 1 -- 172.21.15.5:0/787574535 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f04940041d0 con 0x7f04a00745c0 2024-09-10T09:36:59.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:58.952+0000 7f049d600640 1 --2- 172.21.15.5:0/787574535 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f04880a2c10 0x7f04880a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:36:59.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:58.952+0000 7f049d600640 1 -- 172.21.15.5:0/787574535 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9071+0+0 (secure 0 0 0) 0x7f04940cf080 con 0x7f04a00745c0 2024-09-10T09:36:59.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:58.952+0000 7f049fe00640 1 --2- 172.21.15.5:0/787574535 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f04880a2c10 0x7f04880a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:36:59.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:58.953+0000 7f049d600640 1 --2- 172.21.15.5:0/787574535 >> [v2:172.21.15.5:6810/2221468725,v1:172.21.15.5:6811/2221468725] conn(0x7f04880add60 0x7f04880b0200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:36:59.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:58.953+0000 7f049d600640 1 -- 172.21.15.5:0/787574535 --> [v2:172.21.15.5:6810/2221468725,v1:172.21.15.5:6811/2221468725] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f04880b08d0 con 0x7f04880add60 2024-09-10T09:36:59.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:58.953+0000 7f049d600640 1 -- 172.21.15.5:0/787574535 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_get_version_reply(handle=1 version=94) ==== 24+0+0 (secure 0 0 0) 0x7f04940cf430 con 0x7f04a00745c0 2024-09-10T09:36:59.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:58.953+0000 7f04a5600640 1 --2- 172.21.15.5:0/787574535 >> [v2:172.21.15.5:6810/2221468725,v1:172.21.15.5:6811/2221468725] conn(0x7f04880add60 0x7f04880b0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:36:59.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:58.953+0000 7f049fe00640 1 --2- 172.21.15.5:0/787574535 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f04880a2c10 0x7f04880a50d0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f048c000c00 tx=0x7f048c005b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:36:59.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:58.954+0000 7f04a5600640 1 --2- 172.21.15.5:0/787574535 >> [v2:172.21.15.5:6810/2221468725,v1:172.21.15.5:6811/2221468725] conn(0x7f04880add60 0x7f04880b0200 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:36:59.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:58.954+0000 7f049d600640 1 -- 172.21.15.5:0/787574535 <== osd.3 v2:172.21.15.5:6810/2221468725 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f04880b08d0 con 0x7f04880add60 2024-09-10T09:36:59.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:58.969+0000 7f04a6000640 1 -- 172.21.15.5:0/787574535 --> [v2:172.21.15.5:6810/2221468725,v1:172.21.15.5:6811/2221468725] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f04a0062d80 con 0x7f04880add60 2024-09-10T09:36:59.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:58.970+0000 7f049d600640 1 -- 172.21.15.5:0/787574535 <== osd.3 v2:172.21.15.5:6810/2221468725 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f04a0062d80 con 0x7f04880add60 2024-09-10T09:36:59.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:58.970+0000 7f04a6000640 1 -- 172.21.15.5:0/787574535 >> [v2:172.21.15.5:6810/2221468725,v1:172.21.15.5:6811/2221468725] conn(0x7f04880add60 msgr2=0x7f04880b0200 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:36:59.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:58.970+0000 7f04a6000640 1 --2- 172.21.15.5:0/787574535 >> [v2:172.21.15.5:6810/2221468725,v1:172.21.15.5:6811/2221468725] conn(0x7f04880add60 0x7f04880b0200 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:36:59.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:58.970+0000 7f04a6000640 1 -- 172.21.15.5:0/787574535 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f04880a2c10 msgr2=0x7f04880a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:36:59.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:58.970+0000 7f04a6000640 1 --2- 172.21.15.5:0/787574535 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f04880a2c10 0x7f04880a50d0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f048c000c00 tx=0x7f048c005b50 comp rx=0 tx=0).stop 2024-09-10T09:36:59.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:58.970+0000 7f04a6000640 1 -- 172.21.15.5:0/787574535 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f04a00745c0 msgr2=0x7f04a019a4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:36:59.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:58.970+0000 7f04a6000640 1 --2- 172.21.15.5:0/787574535 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f04a00745c0 0x7f04a019a4e0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f049400d950 tx=0x7f049400de10 comp rx=0 tx=0).stop 2024-09-10T09:36:59.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:58.970+0000 7f04a5600640 1 -- 172.21.15.5:0/787574535 reap_dead start 2024-09-10T09:36:59.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:58.971+0000 7f04a6000640 1 -- 172.21.15.5:0/787574535 shutdown_connections 2024-09-10T09:36:59.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:58.971+0000 7f04a6000640 1 -- 172.21.15.5:0/787574535 >> 172.21.15.5:0/787574535 conn(0x7f04a00fd740 msgr2=0x7f04a00fe440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:36:59.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:58.971+0000 7f04a6000640 1 -- 172.21.15.5:0/787574535 shutdown_connections 2024-09-10T09:36:59.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:36:58.971+0000 7f04a6000640 1 -- 172.21.15.5:0/787574535 wait complete. 2024-09-10T09:37:00.694 INFO:teuthology.orchestra.run.smithi005.stdout:193273528379 2024-09-10T09:37:00.694 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-09-10T09:37:01.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:01.609+0000 7f6c9e800640 1 -- 172.21.15.5:0/1677091567 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6c981079c0 msgr2=0x7f6c98107da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:37:01.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:01.609+0000 7f6c9e800640 1 --2- 172.21.15.5:0/1677091567 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6c981079c0 0x7f6c98107da0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f6c8c009930 tx=0x7f6c8c02a420 comp rx=0 tx=0).stop 2024-09-10T09:37:01.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:01.609+0000 7f6c9e800640 1 -- 172.21.15.5:0/1677091567 shutdown_connections 2024-09-10T09:37:01.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:01.609+0000 7f6c9e800640 1 --2- 172.21.15.5:0/1677091567 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6c98102360 0x7f6c9810e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:01.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:01.609+0000 7f6c9e800640 1 --2- 172.21.15.5:0/1677091567 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6c981019c0 0x7f6c98101e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:01.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:01.609+0000 7f6c9e800640 1 --2- 172.21.15.5:0/1677091567 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6c981079c0 0x7f6c98107da0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:01.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:01.609+0000 7f6c9e800640 1 -- 172.21.15.5:0/1677091567 >> 172.21.15.5:0/1677091567 conn(0x7f6c980fd740 msgr2=0x7f6c980ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:37:01.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:01.609+0000 7f6c9e800640 1 -- 172.21.15.5:0/1677091567 shutdown_connections 2024-09-10T09:37:01.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:01.610+0000 7f6c9e800640 1 -- 172.21.15.5:0/1677091567 wait complete. 2024-09-10T09:37:01.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:01.612+0000 7f6c9e800640 1 Processor -- start 2024-09-10T09:37:01.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:01.612+0000 7f6c9e800640 1 -- start start 2024-09-10T09:37:01.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:01.612+0000 7f6c9e800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6c981019c0 0x7f6c980744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:37:01.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:01.612+0000 7f6c9e800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6c98102360 0x7f6c98074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:37:01.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:01.612+0000 7f6c9e800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6c981079c0 0x7f6c98074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:37:01.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:01.612+0000 7f6c9e800640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f6c98111e10 con 0x7f6c98102360 2024-09-10T09:37:01.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:01.613+0000 7f6c9e800640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f6c98111c90 con 0x7f6c981079c0 2024-09-10T09:37:01.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:01.613+0000 7f6c9e800640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f6c98111f90 con 0x7f6c981019c0 2024-09-10T09:37:01.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:01.613+0000 7f6c9d400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6c981019c0 0x7f6c980744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:37:01.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:01.613+0000 7f6c9d400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6c981019c0 0x7f6c980744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:54642/0 (socket says 172.21.15.5:54642) 2024-09-10T09:37:01.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:01.613+0000 7f6c9de00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6c981079c0 0x7f6c98074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:37:01.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:01.613+0000 7f6c9d400640 1 -- 172.21.15.5:0/3268361912 learned_addr learned my addr 172.21.15.5:0/3268361912 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:37:01.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:01.613+0000 7f6c9de00640 1 -- 172.21.15.5:0/3268361912 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6c981019c0 msgr2=0x7f6c980744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:37:01.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:01.613+0000 7f6c9de00640 1 --2- 172.21.15.5:0/3268361912 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6c981019c0 0x7f6c980744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:01.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:01.613+0000 7f6c9de00640 1 -- 172.21.15.5:0/3268361912 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6c98102360 msgr2=0x7f6c98074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:37:01.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:01.613+0000 7f6c9de00640 1 --2- 172.21.15.5:0/3268361912 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6c98102360 0x7f6c98074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:01.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:01.613+0000 7f6c9de00640 1 -- 172.21.15.5:0/3268361912 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c981a6700 con 0x7f6c981079c0 2024-09-10T09:37:01.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:01.613+0000 7f6c9d400640 1 --2- 172.21.15.5:0/3268361912 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6c981019c0 0x7f6c980744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:37:01.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:01.614+0000 7f6c9de00640 1 --2- 172.21.15.5:0/3268361912 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6c981079c0 0x7f6c98074f70 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f6c90009fd0 tx=0x7f6c9000ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:37:01.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:01.614+0000 7f6c96000640 1 -- 172.21.15.5:0/3268361912 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c9000ce20 con 0x7f6c981079c0 2024-09-10T09:37:01.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:01.614+0000 7f6c96000640 1 -- 172.21.15.5:0/3268361912 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6c90005020 con 0x7f6c981079c0 2024-09-10T09:37:01.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:01.614+0000 7f6c96000640 1 -- 172.21.15.5:0/3268361912 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c90010720 con 0x7f6c981079c0 2024-09-10T09:37:01.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:01.614+0000 7f6c9e800640 1 -- 172.21.15.5:0/3268361912 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c981a69f0 con 0x7f6c981079c0 2024-09-10T09:37:01.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:01.614+0000 7f6c9e800640 1 -- 172.21.15.5:0/3268361912 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c981a6ee0 con 0x7f6c981079c0 2024-09-10T09:37:01.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:01.616+0000 7f6c9e800640 1 -- 172.21.15.5:0/3268361912 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6c98101e20 con 0x7f6c981079c0 2024-09-10T09:37:01.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:01.620+0000 7f6c96000640 1 -- 172.21.15.5:0/3268361912 <== mon.2 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f6c900051d0 con 0x7f6c981079c0 2024-09-10T09:37:01.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:01.621+0000 7f6c96000640 1 --2- 172.21.15.5:0/3268361912 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f6c800a2c90 0x7f6c800a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:37:01.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:01.621+0000 7f6c96000640 1 -- 172.21.15.5:0/3268361912 <== mon.2 v2:172.21.15.102:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9071+0+0 (secure 0 0 0) 0x7f6c900cfc60 con 0x7f6c981079c0 2024-09-10T09:37:01.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:01.621+0000 7f6c9d400640 1 --2- 172.21.15.5:0/3268361912 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f6c800a2c90 0x7f6c800a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:37:01.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:01.621+0000 7f6c96000640 1 -- 172.21.15.5:0/3268361912 <== mon.2 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f6c900173d0 con 0x7f6c981079c0 2024-09-10T09:37:01.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:01.622+0000 7f6c9d400640 1 --2- 172.21.15.5:0/3268361912 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f6c800a2c90 0x7f6c800a5150 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f6c8c005ab0 tx=0x7f6c8c005a40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:37:01.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:37:01 smithi005 ceph-mon[30350]: pgmap v351: 129 pgs: 129 active+clean; 645 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-10T09:37:01.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:01.823+0000 7f6c9e800640 1 -- 172.21.15.5:0/3268361912 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) -- 0x7f6c981a7300 con 0x7f6c981079c0 2024-09-10T09:37:01.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:01.823+0000 7f6c96000640 1 -- 172.21.15.5:0/3268361912 <== mon.2 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f6c9009c6c0 con 0x7f6c981079c0 2024-09-10T09:37:01.824 INFO:teuthology.orchestra.run.smithi005.stdout:300647710756 2024-09-10T09:37:01.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:01.826+0000 7f6c9e800640 1 -- 172.21.15.5:0/3268361912 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f6c800a2c90 msgr2=0x7f6c800a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:37:01.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:01.826+0000 7f6c9e800640 1 --2- 172.21.15.5:0/3268361912 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f6c800a2c90 0x7f6c800a5150 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f6c8c005ab0 tx=0x7f6c8c005a40 comp rx=0 tx=0).stop 2024-09-10T09:37:01.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:01.826+0000 7f6c9e800640 1 -- 172.21.15.5:0/3268361912 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6c981079c0 msgr2=0x7f6c98074f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:37:01.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:01.826+0000 7f6c9e800640 1 --2- 172.21.15.5:0/3268361912 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6c981079c0 0x7f6c98074f70 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f6c90009fd0 tx=0x7f6c9000ef10 comp rx=0 tx=0).stop 2024-09-10T09:37:01.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:01.826+0000 7f6c9e800640 1 -- 172.21.15.5:0/3268361912 shutdown_connections 2024-09-10T09:37:01.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:01.826+0000 7f6c9e800640 1 --2- 172.21.15.5:0/3268361912 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f6c800a2c90 0x7f6c800a5150 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:01.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:01.826+0000 7f6c9e800640 1 --2- 172.21.15.5:0/3268361912 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6c981079c0 0x7f6c98074f70 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:01.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:01.826+0000 7f6c9e800640 1 --2- 172.21.15.5:0/3268361912 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6c98102360 0x7f6c98074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:01.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:01.826+0000 7f6c9e800640 1 --2- 172.21.15.5:0/3268361912 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6c981019c0 0x7f6c980744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:01.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:01.826+0000 7f6c9e800640 1 -- 172.21.15.5:0/3268361912 >> 172.21.15.5:0/3268361912 conn(0x7f6c980fd740 msgr2=0x7f6c980fde60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:37:01.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:01.826+0000 7f6c9e800640 1 -- 172.21.15.5:0/3268361912 shutdown_connections 2024-09-10T09:37:01.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:01.826+0000 7f6c9e800640 1 -- 172.21.15.5:0/3268361912 wait complete. 2024-09-10T09:37:01.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:37:01 smithi159 ceph-mon[35513]: pgmap v351: 129 pgs: 129 active+clean; 645 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-10T09:37:01.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:37:01 smithi102 ceph-mon[35442]: pgmap v351: 129 pgs: 129 active+clean; 645 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-10T09:37:01.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:01.965+0000 7f9660400640 1 -- 172.21.15.5:0/283846718 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9658112a30 msgr2=0x7f9658114e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:37:01.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:01.965+0000 7f9660400640 1 --2- 172.21.15.5:0/283846718 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9658112a30 0x7f9658114e20 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f9650009ca0 tx=0x7f965002a8c0 comp rx=0 tx=0).stop 2024-09-10T09:37:01.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:01.965+0000 7f9660400640 1 -- 172.21.15.5:0/283846718 shutdown_connections 2024-09-10T09:37:01.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:01.965+0000 7f9660400640 1 --2- 172.21.15.5:0/283846718 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9658112a30 0x7f9658114e20 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:01.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:01.965+0000 7f9660400640 1 --2- 172.21.15.5:0/283846718 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9658077040 0x7f96581124f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:01.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:01.965+0000 7f9660400640 1 --2- 172.21.15.5:0/283846718 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9658076720 0x7f9658076b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:01.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:01.965+0000 7f9660400640 1 -- 172.21.15.5:0/283846718 >> 172.21.15.5:0/283846718 conn(0x7f96580ffa90 msgr2=0x7f9658101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:37:01.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:01.966+0000 7f9660400640 1 -- 172.21.15.5:0/283846718 shutdown_connections 2024-09-10T09:37:01.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:01.966+0000 7f9660400640 1 -- 172.21.15.5:0/283846718 wait complete. 2024-09-10T09:37:01.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:01.966+0000 7f9660400640 1 Processor -- start 2024-09-10T09:37:01.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:01.967+0000 7f9660400640 1 -- start start 2024-09-10T09:37:01.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:01.967+0000 7f9660400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9658076720 0x7f96581a46e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:37:01.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:01.967+0000 7f9660400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9658077040 0x7f96581a4c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:37:01.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:01.967+0000 7f9660400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9658112a30 0x7f965819e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:37:01.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:01.967+0000 7f9660400640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f96581185d0 con 0x7f9658112a30 2024-09-10T09:37:01.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:01.967+0000 7f9660400640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f9658118450 con 0x7f9658076720 2024-09-10T09:37:01.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:01.967+0000 7f9660400640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f9658118750 con 0x7f9658077040 2024-09-10T09:37:01.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:01.967+0000 7f965d000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9658077040 0x7f96581a4c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:37:01.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:01.967+0000 7f965e400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9658112a30 0x7f965819e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:37:01.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:01.967+0000 7f965d000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9658077040 0x7f96581a4c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:54686/0 (socket says 172.21.15.5:54686) 2024-09-10T09:37:01.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:01.967+0000 7f965d000640 1 -- 172.21.15.5:0/113360120 learned_addr learned my addr 172.21.15.5:0/113360120 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:37:01.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:01.967+0000 7f965da00640 1 --2- 172.21.15.5:0/113360120 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9658076720 0x7f96581a46e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:37:01.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:01.968+0000 7f965d000640 1 -- 172.21.15.5:0/113360120 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9658076720 msgr2=0x7f96581a46e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:37:01.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:01.968+0000 7f965d000640 1 --2- 172.21.15.5:0/113360120 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9658076720 0x7f96581a46e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:01.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:01.968+0000 7f965d000640 1 -- 172.21.15.5:0/113360120 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9658112a30 msgr2=0x7f965819e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:37:01.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:01.968+0000 7f965d000640 1 --2- 172.21.15.5:0/113360120 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9658112a30 0x7f965819e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:01.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:01.968+0000 7f965d000640 1 -- 172.21.15.5:0/113360120 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f965819efd0 con 0x7f9658077040 2024-09-10T09:37:01.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:01.968+0000 7f965d000640 1 --2- 172.21.15.5:0/113360120 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9658077040 0x7f96581a4c20 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f964400e9f0 tx=0x7f964400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:37:01.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:01.968+0000 7f9656000640 1 -- 172.21.15.5:0/113360120 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f964400cd80 con 0x7f9658077040 2024-09-10T09:37:01.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:01.969+0000 7f9656000640 1 -- 172.21.15.5:0/113360120 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9644010910 con 0x7f9658077040 2024-09-10T09:37:01.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:01.969+0000 7f9660400640 1 -- 172.21.15.5:0/113360120 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f965819f2c0 con 0x7f9658077040 2024-09-10T09:37:01.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:01.969+0000 7f9656000640 1 -- 172.21.15.5:0/113360120 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9644018990 con 0x7f9658077040 2024-09-10T09:37:01.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:01.969+0000 7f9660400640 1 -- 172.21.15.5:0/113360120 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96581ab5b0 con 0x7f9658077040 2024-09-10T09:37:01.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:01.970+0000 7f9660400640 1 -- 172.21.15.5:0/113360120 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9658076b00 con 0x7f9658077040 2024-09-10T09:37:01.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:01.973+0000 7f9656000640 1 -- 172.21.15.5:0/113360120 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f9644010ac0 con 0x7f9658077040 2024-09-10T09:37:01.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:01.976+0000 7f9656000640 1 --2- 172.21.15.5:0/113360120 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f96400a2c90 0x7f96400a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:37:01.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:01.976+0000 7f9656000640 1 -- 172.21.15.5:0/113360120 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9071+0+0 (secure 0 0 0) 0x7f96440cfcd0 con 0x7f9658077040 2024-09-10T09:37:01.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:01.976+0000 7f965da00640 1 --2- 172.21.15.5:0/113360120 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f96400a2c90 0x7f96400a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:37:01.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:01.976+0000 7f9656000640 1 -- 172.21.15.5:0/113360120 <== mon.1 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f9644098750 con 0x7f9658077040 2024-09-10T09:37:01.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:01.977+0000 7f965da00640 1 --2- 172.21.15.5:0/113360120 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f96400a2c90 0x7f96400a5150 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f965819f9e0 tx=0x7f964c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:37:02.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:02.169+0000 7f9660400640 1 -- 172.21.15.5:0/113360120 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7f965819fdd0 con 0x7f9658077040 2024-09-10T09:37:02.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:02.169+0000 7f9656000640 1 -- 172.21.15.5:0/113360120 <== mon.1 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f964409c0e0 con 0x7f9658077040 2024-09-10T09:37:02.170 INFO:teuthology.orchestra.run.smithi005.stdout:244813135920 2024-09-10T09:37:02.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:02.171+0000 7f9660400640 1 -- 172.21.15.5:0/113360120 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f96400a2c90 msgr2=0x7f96400a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:37:02.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:02.171+0000 7f9660400640 1 --2- 172.21.15.5:0/113360120 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f96400a2c90 0x7f96400a5150 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f965819f9e0 tx=0x7f964c005f90 comp rx=0 tx=0).stop 2024-09-10T09:37:02.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:02.171+0000 7f9660400640 1 -- 172.21.15.5:0/113360120 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9658077040 msgr2=0x7f96581a4c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:37:02.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:02.171+0000 7f9660400640 1 --2- 172.21.15.5:0/113360120 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9658077040 0x7f96581a4c20 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f964400e9f0 tx=0x7f964400eeb0 comp rx=0 tx=0).stop 2024-09-10T09:37:02.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:02.172+0000 7f9660400640 1 -- 172.21.15.5:0/113360120 shutdown_connections 2024-09-10T09:37:02.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:02.172+0000 7f9660400640 1 --2- 172.21.15.5:0/113360120 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f96400a2c90 0x7f96400a5150 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:02.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:02.172+0000 7f9660400640 1 --2- 172.21.15.5:0/113360120 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9658112a30 0x7f965819e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:02.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:02.172+0000 7f9660400640 1 --2- 172.21.15.5:0/113360120 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9658077040 0x7f96581a4c20 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:02.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:02.172+0000 7f9660400640 1 --2- 172.21.15.5:0/113360120 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9658076720 0x7f96581a46e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:02.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:02.172+0000 7f9660400640 1 -- 172.21.15.5:0/113360120 >> 172.21.15.5:0/113360120 conn(0x7f96580ffa90 msgr2=0x7f9658101e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:37:02.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:02.172+0000 7f9660400640 1 -- 172.21.15.5:0/113360120 shutdown_connections 2024-09-10T09:37:02.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:02.172+0000 7f9660400640 1 -- 172.21.15.5:0/113360120 wait complete. 2024-09-10T09:37:02.735 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:37:02 smithi005 ceph-mon[30350]: from='client.? 172.21.15.5:0/3268361912' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-09-10T09:37:02.735 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:37:02 smithi005 ceph-mon[30350]: from='client.? 172.21.15.5:0/2493145458' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-09-10T09:37:02.735 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:37:02 smithi005 ceph-mon[30350]: from='client.? 172.21.15.5:0/113360120' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-09-10T09:37:02.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:01.720+0000 7fd520e00640 1 -- 172.21.15.5:0/1085061207 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd51c102b90 msgr2=0x7fd51c102f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:37:02.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:01.720+0000 7fd520e00640 1 --2- 172.21.15.5:0/1085061207 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd51c102b90 0x7fd51c102f70 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fd50c009900 tx=0x7fd50c02a480 comp rx=0 tx=0).stop 2024-09-10T09:37:02.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:01.720+0000 7fd520e00640 1 -- 172.21.15.5:0/1085061207 shutdown_connections 2024-09-10T09:37:02.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:01.720+0000 7fd520e00640 1 --2- 172.21.15.5:0/1085061207 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd51c10bfc0 0x7fd51c10e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:02.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:01.720+0000 7fd520e00640 1 --2- 172.21.15.5:0/1085061207 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd51c1034b0 0x7fd51c10b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:02.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:01.720+0000 7fd520e00640 1 --2- 172.21.15.5:0/1085061207 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd51c102b90 0x7fd51c102f70 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:02.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:01.720+0000 7fd520e00640 1 -- 172.21.15.5:0/1085061207 >> 172.21.15.5:0/1085061207 conn(0x7fd51c0fc610 msgr2=0x7fd51c0fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:37:02.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:01.720+0000 7fd520e00640 1 -- 172.21.15.5:0/1085061207 shutdown_connections 2024-09-10T09:37:02.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:01.720+0000 7fd520e00640 1 -- 172.21.15.5:0/1085061207 wait complete. 2024-09-10T09:37:02.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:01.721+0000 7fd520e00640 1 Processor -- start 2024-09-10T09:37:02.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:01.721+0000 7fd520e00640 1 -- start start 2024-09-10T09:37:02.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:01.722+0000 7fd520e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd51c102b90 0x7fd51c1007f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:37:02.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:01.722+0000 7fd520e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd51c1034b0 0x7fd51c100d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:37:02.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:01.722+0000 7fd520e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd51c10bfc0 0x7fd51c101270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:37:02.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:01.722+0000 7fd520e00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd51c111dc0 con 0x7fd51c1034b0 2024-09-10T09:37:02.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:01.722+0000 7fd520e00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fd51c111c40 con 0x7fd51c10bfc0 2024-09-10T09:37:02.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:01.722+0000 7fd520e00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fd51c111f40 con 0x7fd51c102b90 2024-09-10T09:37:02.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:01.722+0000 7fd51b400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd51c102b90 0x7fd51c1007f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:37:02.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:01.722+0000 7fd51aa00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd51c1034b0 0x7fd51c100d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:37:02.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:01.722+0000 7fd51b400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd51c102b90 0x7fd51c1007f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:54662/0 (socket says 172.21.15.5:54662) 2024-09-10T09:37:02.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:01.722+0000 7fd51aa00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd51c1034b0 0x7fd51c100d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57216/0 (socket says 172.21.15.5:57216) 2024-09-10T09:37:02.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:01.722+0000 7fd51b400640 1 -- 172.21.15.5:0/2493145458 learned_addr learned my addr 172.21.15.5:0/2493145458 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:37:02.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:01.722+0000 7fd51be00640 1 --2- 172.21.15.5:0/2493145458 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd51c10bfc0 0x7fd51c101270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:37:02.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:01.723+0000 7fd51aa00640 1 -- 172.21.15.5:0/2493145458 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd51c102b90 msgr2=0x7fd51c1007f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:37:02.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:01.723+0000 7fd51aa00640 1 --2- 172.21.15.5:0/2493145458 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd51c102b90 0x7fd51c1007f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:02.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:01.723+0000 7fd51aa00640 1 -- 172.21.15.5:0/2493145458 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd51c10bfc0 msgr2=0x7fd51c101270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:37:02.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:01.723+0000 7fd51aa00640 1 --2- 172.21.15.5:0/2493145458 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd51c10bfc0 0x7fd51c101270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:02.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:01.723+0000 7fd51aa00640 1 -- 172.21.15.5:0/2493145458 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd51c1a67f0 con 0x7fd51c1034b0 2024-09-10T09:37:02.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:01.723+0000 7fd51aa00640 1 --2- 172.21.15.5:0/2493145458 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd51c1034b0 0x7fd51c100d30 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fd50800e7a0 tx=0x7fd50800ec60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:37:02.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:01.723+0000 7fd513e00640 1 -- 172.21.15.5:0/2493145458 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd50800ccc0 con 0x7fd51c1034b0 2024-09-10T09:37:02.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:01.723+0000 7fd513e00640 1 -- 172.21.15.5:0/2493145458 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd508010910 con 0x7fd51c1034b0 2024-09-10T09:37:02.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:01.723+0000 7fd520e00640 1 -- 172.21.15.5:0/2493145458 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd51c1a6ae0 con 0x7fd51c1034b0 2024-09-10T09:37:02.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:01.724+0000 7fd520e00640 1 -- 172.21.15.5:0/2493145458 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd51c1a7020 con 0x7fd51c1034b0 2024-09-10T09:37:02.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:01.724+0000 7fd513e00640 1 -- 172.21.15.5:0/2493145458 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd508018860 con 0x7fd51c1034b0 2024-09-10T09:37:02.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:01.725+0000 7fd520e00640 1 -- 172.21.15.5:0/2493145458 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd51c102f70 con 0x7fd51c1034b0 2024-09-10T09:37:02.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:01.728+0000 7fd513e00640 1 -- 172.21.15.5:0/2493145458 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7fd508010ac0 con 0x7fd51c1034b0 2024-09-10T09:37:02.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:01.729+0000 7fd513e00640 1 --2- 172.21.15.5:0/2493145458 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fd5000a2c10 0x7fd5000a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:37:02.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:01.729+0000 7fd513e00640 1 -- 172.21.15.5:0/2493145458 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9071+0+0 (secure 0 0 0) 0x7fd5080cf870 con 0x7fd51c1034b0 2024-09-10T09:37:02.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:01.729+0000 7fd51b400640 1 --2- 172.21.15.5:0/2493145458 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fd5000a2c10 0x7fd5000a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:37:02.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:01.730+0000 7fd51b400640 1 --2- 172.21.15.5:0/2493145458 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fd5000a2c10 0x7fd5000a50d0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fd50c0096c0 tx=0x7fd50c005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:37:02.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:01.733+0000 7fd513e00640 1 -- 172.21.15.5:0/2493145458 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd50809c2d0 con 0x7fd51c1034b0 2024-09-10T09:37:02.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:01.940+0000 7fd520e00640 1 -- 172.21.15.5:0/2493145458 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7fd51c107060 con 0x7fd51c1034b0 2024-09-10T09:37:02.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:01.940+0000 7fd513e00640 1 -- 172.21.15.5:0/2493145458 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7fd50809bbd0 con 0x7fd51c1034b0 2024-09-10T09:37:02.838 INFO:teuthology.orchestra.run.smithi005.stdout:330712481824 2024-09-10T09:37:02.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:01.943+0000 7fd520e00640 1 -- 172.21.15.5:0/2493145458 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fd5000a2c10 msgr2=0x7fd5000a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:37:02.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:01.943+0000 7fd520e00640 1 --2- 172.21.15.5:0/2493145458 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fd5000a2c10 0x7fd5000a50d0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fd50c0096c0 tx=0x7fd50c005a20 comp rx=0 tx=0).stop 2024-09-10T09:37:02.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:01.943+0000 7fd520e00640 1 -- 172.21.15.5:0/2493145458 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd51c1034b0 msgr2=0x7fd51c100d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:37:02.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:01.943+0000 7fd520e00640 1 --2- 172.21.15.5:0/2493145458 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd51c1034b0 0x7fd51c100d30 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fd50800e7a0 tx=0x7fd50800ec60 comp rx=0 tx=0).stop 2024-09-10T09:37:02.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:01.943+0000 7fd520e00640 1 -- 172.21.15.5:0/2493145458 shutdown_connections 2024-09-10T09:37:02.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:01.943+0000 7fd520e00640 1 --2- 172.21.15.5:0/2493145458 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fd5000a2c10 0x7fd5000a50d0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:02.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:01.943+0000 7fd520e00640 1 --2- 172.21.15.5:0/2493145458 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd51c10bfc0 0x7fd51c101270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:02.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:01.943+0000 7fd520e00640 1 --2- 172.21.15.5:0/2493145458 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd51c1034b0 0x7fd51c100d30 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:02.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:01.943+0000 7fd520e00640 1 --2- 172.21.15.5:0/2493145458 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd51c102b90 0x7fd51c1007f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:02.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:01.943+0000 7fd520e00640 1 -- 172.21.15.5:0/2493145458 >> 172.21.15.5:0/2493145458 conn(0x7fd51c0fc610 msgr2=0x7fd51c0fd310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:37:02.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:01.943+0000 7fd520e00640 1 -- 172.21.15.5:0/2493145458 shutdown_connections 2024-09-10T09:37:02.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:01.943+0000 7fd520e00640 1 -- 172.21.15.5:0/2493145458 wait complete. 2024-09-10T09:37:02.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:37:02 smithi159 ceph-mon[35513]: from='client.? 172.21.15.5:0/3268361912' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-09-10T09:37:02.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:37:02 smithi159 ceph-mon[35513]: from='client.? 172.21.15.5:0/2493145458' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-09-10T09:37:02.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:37:02 smithi159 ceph-mon[35513]: from='client.? 172.21.15.5:0/113360120' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-09-10T09:37:02.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:37:02 smithi102 ceph-mon[35442]: from='client.? 172.21.15.5:0/3268361912' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-09-10T09:37:02.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:37:02 smithi102 ceph-mon[35442]: from='client.? 172.21.15.5:0/2493145458' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-09-10T09:37:02.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:37:02 smithi102 ceph-mon[35442]: from='client.? 172.21.15.5:0/113360120' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-09-10T09:37:03.698 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:37:03 smithi005 ceph-mon[30350]: pgmap v352: 129 pgs: 129 active+clean; 645 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-10T09:37:03.736 INFO:teuthology.orchestra.run.smithi005.stdout:107374182475 2024-09-10T09:37:03.736 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-09-10T09:37:03.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:37:03 smithi159 ceph-mon[35513]: pgmap v352: 129 pgs: 129 active+clean; 645 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-10T09:37:03.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:37:03 smithi102 ceph-mon[35442]: pgmap v352: 129 pgs: 129 active+clean; 645 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-10T09:37:04.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:03.907+0000 7f2cae000640 1 -- 172.21.15.5:0/712554936 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2ca8102370 msgr2=0x7f2ca810e8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:37:04.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:03.907+0000 7f2cae000640 1 --2- 172.21.15.5:0/712554936 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2ca8102370 0x7f2ca810e8a0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f2ca0009ca0 tx=0x7f2ca002a8e0 comp rx=0 tx=0).stop 2024-09-10T09:37:04.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:03.907+0000 7f2cae000640 1 -- 172.21.15.5:0/712554936 shutdown_connections 2024-09-10T09:37:04.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:03.907+0000 7f2cae000640 1 --2- 172.21.15.5:0/712554936 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2ca8102370 0x7f2ca810e8a0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:04.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:03.907+0000 7f2cae000640 1 --2- 172.21.15.5:0/712554936 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2ca81019d0 0x7f2ca8101e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:04.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:03.907+0000 7f2cae000640 1 --2- 172.21.15.5:0/712554936 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2ca81079d0 0x7f2ca8107db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:04.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:03.907+0000 7f2cae000640 1 -- 172.21.15.5:0/712554936 >> 172.21.15.5:0/712554936 conn(0x7f2ca80fd710 msgr2=0x7f2ca80ffb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:37:04.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:03.907+0000 7f2cae000640 1 -- 172.21.15.5:0/712554936 shutdown_connections 2024-09-10T09:37:04.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:03.907+0000 7f2cae000640 1 -- 172.21.15.5:0/712554936 wait complete. 2024-09-10T09:37:04.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:03.909+0000 7f2cae000640 1 Processor -- start 2024-09-10T09:37:04.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:03.909+0000 7f2cae000640 1 -- start start 2024-09-10T09:37:04.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:03.910+0000 7f2cae000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2ca81019d0 0x7f2ca819a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:37:04.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:03.910+0000 7f2cae000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2ca8102370 0x7f2ca819a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:37:04.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:03.910+0000 7f2cae000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2ca81079d0 0x7f2ca819ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:37:04.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:03.910+0000 7f2cae000640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f2ca8111e10 con 0x7f2ca81019d0 2024-09-10T09:37:04.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:03.910+0000 7f2cae000640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f2ca8111c90 con 0x7f2ca81079d0 2024-09-10T09:37:04.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:03.910+0000 7f2cae000640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f2ca8111f90 con 0x7f2ca8102370 2024-09-10T09:37:04.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:03.910+0000 7f2cacc00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2ca81019d0 0x7f2ca819a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:37:04.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:03.910+0000 7f2cacc00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2ca81019d0 0x7f2ca819a4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57254/0 (socket says 172.21.15.5:57254) 2024-09-10T09:37:04.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:03.910+0000 7f2cacc00640 1 -- 172.21.15.5:0/424113812 learned_addr learned my addr 172.21.15.5:0/424113812 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:37:04.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:03.910+0000 7f2ca7e00640 1 --2- 172.21.15.5:0/424113812 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2ca8102370 0x7f2ca819a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:37:04.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:03.910+0000 7f2cad600640 1 --2- 172.21.15.5:0/424113812 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2ca81079d0 0x7f2ca819ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:37:04.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:03.911+0000 7f2cacc00640 1 -- 172.21.15.5:0/424113812 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2ca8102370 msgr2=0x7f2ca819a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:37:04.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:03.911+0000 7f2cacc00640 1 --2- 172.21.15.5:0/424113812 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2ca8102370 0x7f2ca819a9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:04.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:03.911+0000 7f2cacc00640 1 -- 172.21.15.5:0/424113812 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2ca81079d0 msgr2=0x7f2ca819ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:37:04.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:03.911+0000 7f2cacc00640 1 --2- 172.21.15.5:0/424113812 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2ca81079d0 0x7f2ca819ed70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:04.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:03.911+0000 7f2cacc00640 1 -- 172.21.15.5:0/424113812 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ca819f450 con 0x7f2ca81019d0 2024-09-10T09:37:04.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:03.911+0000 7f2cacc00640 1 --2- 172.21.15.5:0/424113812 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2ca81019d0 0x7f2ca819a4a0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f2c9c00b450 tx=0x7f2c9c00b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:37:04.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:03.911+0000 7f2ca5600640 1 -- 172.21.15.5:0/424113812 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c9c015020 con 0x7f2ca81019d0 2024-09-10T09:37:04.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:03.911+0000 7f2ca5600640 1 -- 172.21.15.5:0/424113812 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2c9c011d70 con 0x7f2ca81019d0 2024-09-10T09:37:04.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:03.911+0000 7f2ca5600640 1 -- 172.21.15.5:0/424113812 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c9c00f340 con 0x7f2ca81019d0 2024-09-10T09:37:04.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:03.911+0000 7f2cae000640 1 -- 172.21.15.5:0/424113812 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ca819f740 con 0x7f2ca81019d0 2024-09-10T09:37:04.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:03.912+0000 7f2cae000640 1 -- 172.21.15.5:0/424113812 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ca81a7020 con 0x7f2ca81019d0 2024-09-10T09:37:04.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:03.913+0000 7f2cae000640 1 -- 172.21.15.5:0/424113812 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ca8101e30 con 0x7f2ca81019d0 2024-09-10T09:37:04.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:03.915+0000 7f2ca5600640 1 -- 172.21.15.5:0/424113812 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f2c9c018020 con 0x7f2ca81019d0 2024-09-10T09:37:04.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:03.916+0000 7f2ca5600640 1 --2- 172.21.15.5:0/424113812 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f2c900a2d60 0x7f2c900a5220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:37:04.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:03.916+0000 7f2ca5600640 1 -- 172.21.15.5:0/424113812 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9071+0+0 (secure 0 0 0) 0x7f2c9c0cf1f0 con 0x7f2ca81019d0 2024-09-10T09:37:04.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:03.916+0000 7f2ca7e00640 1 --2- 172.21.15.5:0/424113812 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f2c900a2d60 0x7f2c900a5220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:37:04.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:03.917+0000 7f2ca7e00640 1 --2- 172.21.15.5:0/424113812 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f2c900a2d60 0x7f2c900a5220 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f2ca810ecc0 tx=0x7f2c94005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:37:04.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:03.919+0000 7f2ca5600640 1 -- 172.21.15.5:0/424113812 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f2c9c01f020 con 0x7f2ca81019d0 2024-09-10T09:37:04.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:04.129+0000 7f2cae000640 1 -- 172.21.15.5:0/424113812 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f2ca819b800 con 0x7f2ca81019d0 2024-09-10T09:37:04.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:04.130+0000 7f2ca5600640 1 -- 172.21.15.5:0/424113812 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f2ca819b800 con 0x7f2ca81019d0 2024-09-10T09:37:04.131 INFO:teuthology.orchestra.run.smithi005.stdout:55834574935 2024-09-10T09:37:04.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:04.132+0000 7f2cae000640 1 -- 172.21.15.5:0/424113812 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f2c900a2d60 msgr2=0x7f2c900a5220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:37:04.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:04.132+0000 7f2cae000640 1 --2- 172.21.15.5:0/424113812 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f2c900a2d60 0x7f2c900a5220 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f2ca810ecc0 tx=0x7f2c94005f90 comp rx=0 tx=0).stop 2024-09-10T09:37:04.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:04.132+0000 7f2cae000640 1 -- 172.21.15.5:0/424113812 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2ca81019d0 msgr2=0x7f2ca819a4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:37:04.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:04.132+0000 7f2cae000640 1 --2- 172.21.15.5:0/424113812 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2ca81019d0 0x7f2ca819a4a0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f2c9c00b450 tx=0x7f2c9c00b910 comp rx=0 tx=0).stop 2024-09-10T09:37:04.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:04.132+0000 7f2cae000640 1 -- 172.21.15.5:0/424113812 shutdown_connections 2024-09-10T09:37:04.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:04.132+0000 7f2cae000640 1 --2- 172.21.15.5:0/424113812 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f2c900a2d60 0x7f2c900a5220 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:04.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:04.132+0000 7f2cae000640 1 --2- 172.21.15.5:0/424113812 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2ca81079d0 0x7f2ca819ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:04.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:04.132+0000 7f2cae000640 1 --2- 172.21.15.5:0/424113812 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2ca8102370 0x7f2ca819a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:04.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:04.132+0000 7f2cae000640 1 --2- 172.21.15.5:0/424113812 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2ca81019d0 0x7f2ca819a4a0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:04.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:04.132+0000 7f2cae000640 1 -- 172.21.15.5:0/424113812 >> 172.21.15.5:0/424113812 conn(0x7f2ca80fd710 msgr2=0x7f2ca80fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:37:04.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:04.133+0000 7f2cae000640 1 -- 172.21.15.5:0/424113812 shutdown_connections 2024-09-10T09:37:04.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:04.133+0000 7f2cae000640 1 -- 172.21.15.5:0/424113812 wait complete. 2024-09-10T09:37:04.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:04.043+0000 7fe1f1a00640 1 -- 172.21.15.5:0/3264362960 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe1ec104970 msgr2=0x7fe1ec1089c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:37:04.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:04.043+0000 7fe1f1a00640 1 --2- 172.21.15.5:0/3264362960 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe1ec104970 0x7fe1ec1089c0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fe1dc009900 tx=0x7fe1dc02a440 comp rx=0 tx=0).stop 2024-09-10T09:37:04.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:04.043+0000 7fe1f1a00640 1 -- 172.21.15.5:0/3264362960 shutdown_connections 2024-09-10T09:37:04.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:04.043+0000 7fe1f1a00640 1 --2- 172.21.15.5:0/3264362960 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe1ec1090f0 0x7fe1ec110c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:04.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:04.043+0000 7fe1f1a00640 1 --2- 172.21.15.5:0/3264362960 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe1ec104970 0x7fe1ec1089c0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:04.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:04.043+0000 7fe1f1a00640 1 --2- 172.21.15.5:0/3264362960 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe1ec103fc0 0x7fe1ec1043a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:04.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:04.043+0000 7fe1f1a00640 1 -- 172.21.15.5:0/3264362960 >> 172.21.15.5:0/3264362960 conn(0x7fe1ec0ffa40 msgr2=0x7fe1ec101e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:37:04.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:04.046+0000 7fe1f1a00640 1 -- 172.21.15.5:0/3264362960 shutdown_connections 2024-09-10T09:37:04.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:04.046+0000 7fe1f1a00640 1 -- 172.21.15.5:0/3264362960 wait complete. 2024-09-10T09:37:04.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:04.047+0000 7fe1f1a00640 1 Processor -- start 2024-09-10T09:37:04.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:04.048+0000 7fe1f1a00640 1 -- start start 2024-09-10T09:37:04.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:04.048+0000 7fe1f1a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe1ec103fc0 0x7fe1ec076700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:37:04.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:04.048+0000 7fe1f1a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe1ec104970 0x7fe1ec076c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:37:04.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:04.048+0000 7fe1f1a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe1ec1090f0 0x7fe1ec077180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:37:04.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:04.048+0000 7fe1f1a00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe1ec114140 con 0x7fe1ec1090f0 2024-09-10T09:37:04.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:04.048+0000 7fe1f1a00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fe1ec113fc0 con 0x7fe1ec104970 2024-09-10T09:37:04.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:04.048+0000 7fe1f1a00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fe1ec1142c0 con 0x7fe1ec103fc0 2024-09-10T09:37:04.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:04.049+0000 7fe1eaa00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe1ec103fc0 0x7fe1ec076700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:37:04.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:04.049+0000 7fe1eaa00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe1ec103fc0 0x7fe1ec076700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:54730/0 (socket says 172.21.15.5:54730) 2024-09-10T09:37:04.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:04.049+0000 7fe1eaa00640 1 -- 172.21.15.5:0/182683218 learned_addr learned my addr 172.21.15.5:0/182683218 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:37:04.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:04.049+0000 7fe1ea000640 1 --2- 172.21.15.5:0/182683218 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe1ec104970 0x7fe1ec076c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:37:04.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:04.049+0000 7fe1eb400640 1 --2- 172.21.15.5:0/182683218 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe1ec1090f0 0x7fe1ec077180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:37:04.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:04.049+0000 7fe1eaa00640 1 -- 172.21.15.5:0/182683218 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe1ec104970 msgr2=0x7fe1ec076c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:37:04.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:04.049+0000 7fe1eaa00640 1 --2- 172.21.15.5:0/182683218 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe1ec104970 0x7fe1ec076c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:04.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:04.049+0000 7fe1eaa00640 1 -- 172.21.15.5:0/182683218 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe1ec1090f0 msgr2=0x7fe1ec077180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:37:04.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:04.050+0000 7fe1eaa00640 1 --2- 172.21.15.5:0/182683218 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe1ec1090f0 0x7fe1ec077180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:04.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:04.050+0000 7fe1eaa00640 1 -- 172.21.15.5:0/182683218 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1ec1aace0 con 0x7fe1ec103fc0 2024-09-10T09:37:04.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:04.050+0000 7fe1ea000640 1 --2- 172.21.15.5:0/182683218 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe1ec104970 0x7fe1ec076c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:37:04.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:04.050+0000 7fe1eb400640 1 --2- 172.21.15.5:0/182683218 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe1ec1090f0 0x7fe1ec077180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T09:37:04.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:04.050+0000 7fe1eaa00640 1 --2- 172.21.15.5:0/182683218 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe1ec103fc0 0x7fe1ec076700 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fe1d800ea20 tx=0x7fe1d800eee0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:37:04.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:04.050+0000 7fe1e3400640 1 -- 172.21.15.5:0/182683218 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1d800cd80 con 0x7fe1ec103fc0 2024-09-10T09:37:04.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:04.051+0000 7fe1e3400640 1 -- 172.21.15.5:0/182683218 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe1d8010910 con 0x7fe1ec103fc0 2024-09-10T09:37:04.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:04.051+0000 7fe1e3400640 1 -- 172.21.15.5:0/182683218 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1d8018920 con 0x7fe1ec103fc0 2024-09-10T09:37:04.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:04.051+0000 7fe1f1a00640 1 -- 172.21.15.5:0/182683218 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1ec1aafd0 con 0x7fe1ec103fc0 2024-09-10T09:37:04.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:04.052+0000 7fe1f1a00640 1 -- 172.21.15.5:0/182683218 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1ec1ab370 con 0x7fe1ec103fc0 2024-09-10T09:37:04.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:04.053+0000 7fe1f1a00640 1 -- 172.21.15.5:0/182683218 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe1ec1054a0 con 0x7fe1ec103fc0 2024-09-10T09:37:04.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:04.056+0000 7fe1e3400640 1 -- 172.21.15.5:0/182683218 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7fe1d8010ac0 con 0x7fe1ec103fc0 2024-09-10T09:37:04.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:04.057+0000 7fe1e3400640 1 --2- 172.21.15.5:0/182683218 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fe1d00a2c10 0x7fe1d00a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:37:04.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:04.058+0000 7fe1e3400640 1 -- 172.21.15.5:0/182683218 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9071+0+0 (secure 0 0 0) 0x7fe1d80cfb60 con 0x7fe1ec103fc0 2024-09-10T09:37:04.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:04.058+0000 7fe1ea000640 1 --2- 172.21.15.5:0/182683218 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fe1d00a2c10 0x7fe1d00a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:37:04.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:04.059+0000 7fe1ea000640 1 --2- 172.21.15.5:0/182683218 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fe1d00a2c10 0x7fe1d00a50d0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fe1dc0094b0 tx=0x7fe1dc000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:37:04.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:04.062+0000 7fe1e3400640 1 -- 172.21.15.5:0/182683218 <== mon.1 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe1d809c5c0 con 0x7fe1ec103fc0 2024-09-10T09:37:04.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:04.257+0000 7fe1f1a00640 1 -- 172.21.15.5:0/182683218 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) -- 0x7fe1ec10bda0 con 0x7fe1ec103fc0 2024-09-10T09:37:04.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:04.257+0000 7fe1e3400640 1 -- 172.21.15.5:0/182683218 <== mon.1 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fe1d809bec0 con 0x7fe1ec103fc0 2024-09-10T09:37:04.259 INFO:teuthology.orchestra.run.smithi005.stdout:279172874284 2024-09-10T09:37:04.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:04.260+0000 7fe1f1a00640 1 -- 172.21.15.5:0/182683218 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fe1d00a2c10 msgr2=0x7fe1d00a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:37:04.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:04.260+0000 7fe1f1a00640 1 --2- 172.21.15.5:0/182683218 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fe1d00a2c10 0x7fe1d00a50d0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fe1dc0094b0 tx=0x7fe1dc000bc0 comp rx=0 tx=0).stop 2024-09-10T09:37:04.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:04.260+0000 7fe1f1a00640 1 -- 172.21.15.5:0/182683218 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe1ec103fc0 msgr2=0x7fe1ec076700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:37:04.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:04.260+0000 7fe1f1a00640 1 --2- 172.21.15.5:0/182683218 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe1ec103fc0 0x7fe1ec076700 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fe1d800ea20 tx=0x7fe1d800eee0 comp rx=0 tx=0).stop 2024-09-10T09:37:04.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:04.261+0000 7fe1f1a00640 1 -- 172.21.15.5:0/182683218 shutdown_connections 2024-09-10T09:37:04.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:04.261+0000 7fe1f1a00640 1 --2- 172.21.15.5:0/182683218 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fe1d00a2c10 0x7fe1d00a50d0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:04.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:04.261+0000 7fe1f1a00640 1 --2- 172.21.15.5:0/182683218 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe1ec1090f0 0x7fe1ec077180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:04.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:04.261+0000 7fe1f1a00640 1 --2- 172.21.15.5:0/182683218 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe1ec104970 0x7fe1ec076c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:04.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:04.261+0000 7fe1f1a00640 1 --2- 172.21.15.5:0/182683218 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe1ec103fc0 0x7fe1ec076700 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:04.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:04.261+0000 7fe1f1a00640 1 -- 172.21.15.5:0/182683218 >> 172.21.15.5:0/182683218 conn(0x7fe1ec0ffa40 msgr2=0x7fe1ec100d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:37:04.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:04.261+0000 7fe1f1a00640 1 -- 172.21.15.5:0/182683218 shutdown_connections 2024-09-10T09:37:04.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:04.261+0000 7fe1f1a00640 1 -- 172.21.15.5:0/182683218 wait complete. 2024-09-10T09:37:04.793 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:37:04 smithi005 ceph-mon[30350]: from='client.? 172.21.15.5:0/424113812' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-10T09:37:04.794 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:37:04 smithi005 ceph-mon[30350]: from='client.? 172.21.15.5:0/182683218' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-09-10T09:37:04.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:37:04 smithi159 ceph-mon[35513]: from='client.? 172.21.15.5:0/424113812' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-10T09:37:04.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:37:04 smithi159 ceph-mon[35513]: from='client.? 172.21.15.5:0/182683218' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-09-10T09:37:04.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:37:04 smithi102 ceph-mon[35442]: from='client.? 172.21.15.5:0/424113812' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-10T09:37:04.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:37:04 smithi102 ceph-mon[35442]: from='client.? 172.21.15.5:0/182683218' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-09-10T09:37:05.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:05.557+0000 7f2d6f600640 1 -- 172.21.15.5:0/635147548 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2d68106f10 msgr2=0x7f2d6810ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:37:05.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:05.557+0000 7f2d6f600640 1 --2- 172.21.15.5:0/635147548 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2d68106f10 0x7f2d6810ea90 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f2d60009ca0 tx=0x7f2d6002a8c0 comp rx=0 tx=0).stop 2024-09-10T09:37:05.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:05.557+0000 7f2d6f600640 1 -- 172.21.15.5:0/635147548 shutdown_connections 2024-09-10T09:37:05.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:05.557+0000 7f2d6f600640 1 --2- 172.21.15.5:0/635147548 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2d68106f10 0x7f2d6810ea90 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:05.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:05.557+0000 7f2d6f600640 1 --2- 172.21.15.5:0/635147548 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2d68102790 0x7f2d681067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:05.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:05.557+0000 7f2d6f600640 1 --2- 172.21.15.5:0/635147548 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2d68101de0 0x7f2d681021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:05.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:05.557+0000 7f2d6f600640 1 -- 172.21.15.5:0/635147548 >> 172.21.15.5:0/635147548 conn(0x7f2d680fd8c0 msgr2=0x7f2d680ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:37:05.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:05.557+0000 7f2d6f600640 1 -- 172.21.15.5:0/635147548 shutdown_connections 2024-09-10T09:37:05.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:05.557+0000 7f2d6f600640 1 -- 172.21.15.5:0/635147548 wait complete. 2024-09-10T09:37:05.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:05.558+0000 7f2d6f600640 1 Processor -- start 2024-09-10T09:37:05.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:05.559+0000 7f2d6f600640 1 -- start start 2024-09-10T09:37:05.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:05.559+0000 7f2d6f600640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2d68101de0 0x7f2d680744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:37:05.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:05.559+0000 7f2d6f600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2d68102790 0x7f2d68074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:37:05.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:05.559+0000 7f2d6f600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2d68106f10 0x7f2d68074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:37:05.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:05.559+0000 7f2d6f600640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f2d68111f70 con 0x7f2d68106f10 2024-09-10T09:37:05.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:05.559+0000 7f2d6f600640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f2d68111df0 con 0x7f2d68102790 2024-09-10T09:37:05.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:05.559+0000 7f2d6f600640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f2d681120f0 con 0x7f2d68101de0 2024-09-10T09:37:05.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:05.559+0000 7f2d67e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2d68102790 0x7f2d68074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:37:05.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:05.559+0000 7f2d67e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2d68102790 0x7f2d68074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:50006/0 (socket says 172.21.15.5:50006) 2024-09-10T09:37:05.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:05.560+0000 7f2d67e00640 1 -- 172.21.15.5:0/3967427590 learned_addr learned my addr 172.21.15.5:0/3967427590 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:37:05.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:05.560+0000 7f2d6cc00640 1 --2- 172.21.15.5:0/3967427590 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2d68101de0 0x7f2d680744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:37:05.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:05.560+0000 7f2d6d600640 1 --2- 172.21.15.5:0/3967427590 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2d68106f10 0x7f2d68074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:37:05.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:05.560+0000 7f2d67e00640 1 -- 172.21.15.5:0/3967427590 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2d68101de0 msgr2=0x7f2d680744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:37:05.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:05.560+0000 7f2d67e00640 1 --2- 172.21.15.5:0/3967427590 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2d68101de0 0x7f2d680744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:05.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:05.560+0000 7f2d67e00640 1 -- 172.21.15.5:0/3967427590 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2d68106f10 msgr2=0x7f2d68074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:37:05.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:05.560+0000 7f2d67e00640 1 --2- 172.21.15.5:0/3967427590 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2d68106f10 0x7f2d68074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:05.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:05.560+0000 7f2d67e00640 1 -- 172.21.15.5:0/3967427590 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d681a68d0 con 0x7f2d68102790 2024-09-10T09:37:05.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:05.560+0000 7f2d6cc00640 1 --2- 172.21.15.5:0/3967427590 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2d68101de0 0x7f2d680744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:37:05.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:05.561+0000 7f2d67e00640 1 --2- 172.21.15.5:0/3967427590 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2d68102790 0x7f2d68074a30 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f2d5400e9f0 tx=0x7f2d5400edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:37:05.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:05.561+0000 7f2d65600640 1 -- 172.21.15.5:0/3967427590 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d5400cd80 con 0x7f2d68102790 2024-09-10T09:37:05.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:05.561+0000 7f2d65600640 1 -- 172.21.15.5:0/3967427590 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2d54005500 con 0x7f2d68102790 2024-09-10T09:37:05.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:05.561+0000 7f2d6f600640 1 -- 172.21.15.5:0/3967427590 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d681a6bc0 con 0x7f2d68102790 2024-09-10T09:37:05.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:05.561+0000 7f2d65600640 1 -- 172.21.15.5:0/3967427590 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d54010690 con 0x7f2d68102790 2024-09-10T09:37:05.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:05.561+0000 7f2d6f600640 1 -- 172.21.15.5:0/3967427590 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d681a70b0 con 0x7f2d68102790 2024-09-10T09:37:05.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:05.562+0000 7f2d6f600640 1 -- 172.21.15.5:0/3967427590 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2d681021c0 con 0x7f2d68102790 2024-09-10T09:37:05.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:05.565+0000 7f2d65600640 1 -- 172.21.15.5:0/3967427590 <== mon.2 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f2d540046c0 con 0x7f2d68102790 2024-09-10T09:37:05.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:05.566+0000 7f2d65600640 1 --2- 172.21.15.5:0/3967427590 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f2d4c0a2c10 0x7f2d4c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:37:05.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:05.566+0000 7f2d65600640 1 -- 172.21.15.5:0/3967427590 <== mon.2 v2:172.21.15.102:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9071+0+0 (secure 0 0 0) 0x7f2d540cf920 con 0x7f2d68102790 2024-09-10T09:37:05.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:05.566+0000 7f2d6cc00640 1 --2- 172.21.15.5:0/3967427590 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f2d4c0a2c10 0x7f2d4c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:37:05.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:05.568+0000 7f2d6cc00640 1 --2- 172.21.15.5:0/3967427590 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f2d4c0a2c10 0x7f2d4c0a50d0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f2d58009fc0 tx=0x7f2d58005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:37:05.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:05.570+0000 7f2d65600640 1 -- 172.21.15.5:0/3967427590 <== mon.2 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f2d5409c380 con 0x7f2d68102790 2024-09-10T09:37:05.692 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:37:05 smithi005 ceph-mon[30350]: pgmap v353: 129 pgs: 129 active+clean; 645 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-10T09:37:05.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:05.758+0000 7f2d6f600640 1 -- 172.21.15.5:0/3967427590 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f2d6810a120 con 0x7f2d68102790 2024-09-10T09:37:05.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:05.759+0000 7f2d65600640 1 -- 172.21.15.5:0/3967427590 <== mon.2 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f2d5409bc80 con 0x7f2d68102790 2024-09-10T09:37:05.760 INFO:teuthology.orchestra.run.smithi005.stdout:34359738460 2024-09-10T09:37:05.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:05.762+0000 7f2d6f600640 1 -- 172.21.15.5:0/3967427590 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f2d4c0a2c10 msgr2=0x7f2d4c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:37:05.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:05.762+0000 7f2d6f600640 1 --2- 172.21.15.5:0/3967427590 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f2d4c0a2c10 0x7f2d4c0a50d0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f2d58009fc0 tx=0x7f2d58005f90 comp rx=0 tx=0).stop 2024-09-10T09:37:05.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:05.762+0000 7f2d6f600640 1 -- 172.21.15.5:0/3967427590 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2d68102790 msgr2=0x7f2d68074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:37:05.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:05.762+0000 7f2d6f600640 1 --2- 172.21.15.5:0/3967427590 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2d68102790 0x7f2d68074a30 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f2d5400e9f0 tx=0x7f2d5400edf0 comp rx=0 tx=0).stop 2024-09-10T09:37:05.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:05.762+0000 7f2d6f600640 1 -- 172.21.15.5:0/3967427590 shutdown_connections 2024-09-10T09:37:05.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:05.762+0000 7f2d6f600640 1 --2- 172.21.15.5:0/3967427590 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f2d4c0a2c10 0x7f2d4c0a50d0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:05.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:05.762+0000 7f2d6f600640 1 --2- 172.21.15.5:0/3967427590 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2d68106f10 0x7f2d68074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:05.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:05.762+0000 7f2d6f600640 1 --2- 172.21.15.5:0/3967427590 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2d68102790 0x7f2d68074a30 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:05.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:05.762+0000 7f2d6f600640 1 --2- 172.21.15.5:0/3967427590 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2d68101de0 0x7f2d680744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:05.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:05.762+0000 7f2d6f600640 1 -- 172.21.15.5:0/3967427590 >> 172.21.15.5:0/3967427590 conn(0x7f2d680fd8c0 msgr2=0x7f2d680fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:37:05.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:05.762+0000 7f2d6f600640 1 -- 172.21.15.5:0/3967427590 shutdown_connections 2024-09-10T09:37:05.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:05.762+0000 7f2d6f600640 1 -- 172.21.15.5:0/3967427590 wait complete. 2024-09-10T09:37:05.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:37:05 smithi159 ceph-mon[35513]: pgmap v353: 129 pgs: 129 active+clean; 645 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-10T09:37:05.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:37:05 smithi102 ceph-mon[35442]: pgmap v353: 129 pgs: 129 active+clean; 645 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-10T09:37:06.072 INFO:tasks.cephadm.ceph_manager.ceph:need seq 279172874281 got 279172874284 for osd.9 2024-09-10T09:37:06.072 DEBUG:teuthology.parallel:result is None 2024-09-10T09:37:06.700 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:37:06 smithi005 ceph-mon[30350]: from='client.? 172.21.15.5:0/3967427590' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-10T09:37:06.700 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:37:06 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [8, 4]}]: dispatch 2024-09-10T09:37:06.701 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:37:06 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]: dispatch 2024-09-10T09:37:06.701 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:37:06 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2024-09-10T09:37:06.701 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:37:06 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 4]}]: dispatch 2024-09-10T09:37:06.701 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:37:06 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]: dispatch 2024-09-10T09:37:06.701 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:37:06 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.39", "id": [5, 6]}]: dispatch 2024-09-10T09:37:06.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:37:06 smithi159 ceph-mon[35513]: from='client.? 172.21.15.5:0/3967427590' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-10T09:37:06.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:37:06 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [8, 4]}]: dispatch 2024-09-10T09:37:06.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:37:06 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]: dispatch 2024-09-10T09:37:06.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:37:06 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2024-09-10T09:37:06.851 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:37:06 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 4]}]: dispatch 2024-09-10T09:37:06.851 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:37:06 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]: dispatch 2024-09-10T09:37:06.851 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:37:06 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.39", "id": [5, 6]}]: dispatch 2024-09-10T09:37:06.852 INFO:tasks.cephadm.ceph_manager.ceph:need seq 300647710754 got 300647710756 for osd.10 2024-09-10T09:37:06.852 DEBUG:teuthology.parallel:result is None 2024-09-10T09:37:06.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:37:06 smithi102 ceph-mon[35442]: from='client.? 172.21.15.5:0/3967427590' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-10T09:37:06.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:37:06 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [8, 4]}]: dispatch 2024-09-10T09:37:06.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:37:06 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]: dispatch 2024-09-10T09:37:06.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:37:06 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2024-09-10T09:37:06.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:37:06 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 4]}]: dispatch 2024-09-10T09:37:06.885 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:37:06 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]: dispatch 2024-09-10T09:37:06.885 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:37:06 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.39", "id": [5, 6]}]: dispatch 2024-09-10T09:37:07.239 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738457 got 34359738460 for osd.0 2024-09-10T09:37:07.240 DEBUG:teuthology.parallel:result is None 2024-09-10T09:37:07.698 INFO:tasks.cephadm.ceph_manager.ceph:need seq 330712481823 got 330712481824 for osd.11 2024-09-10T09:37:07.698 DEBUG:teuthology.parallel:result is None 2024-09-10T09:37:07.782 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:37:07 smithi005 ceph-mon[30350]: pgmap v354: 129 pgs: 129 active+clean; 645 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-09-10T09:37:07.782 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:37:07 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [8, 4]}]': finished 2024-09-10T09:37:07.782 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:37:07 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]': finished 2024-09-10T09:37:07.782 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:37:07 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2024-09-10T09:37:07.782 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:37:07 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 4]}]': finished 2024-09-10T09:37:07.782 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:37:07 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]': finished 2024-09-10T09:37:07.783 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:37:07 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.39", "id": [5, 6]}]': finished 2024-09-10T09:37:07.783 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:37:07 smithi005 ceph-mon[30350]: osdmap e95: 12 total, 12 up, 12 in 2024-09-10T09:37:07.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:37:07 smithi159 ceph-mon[35513]: pgmap v354: 129 pgs: 129 active+clean; 645 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-09-10T09:37:07.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:37:07 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [8, 4]}]': finished 2024-09-10T09:37:07.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:37:07 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]': finished 2024-09-10T09:37:07.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:37:07 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2024-09-10T09:37:07.851 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:37:07 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 4]}]': finished 2024-09-10T09:37:07.851 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:37:07 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]': finished 2024-09-10T09:37:07.851 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:37:07 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.39", "id": [5, 6]}]': finished 2024-09-10T09:37:07.851 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:37:07 smithi159 ceph-mon[35513]: osdmap e95: 12 total, 12 up, 12 in 2024-09-10T09:37:07.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:37:07 smithi102 ceph-mon[35442]: pgmap v354: 129 pgs: 129 active+clean; 645 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-09-10T09:37:07.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:37:07 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [8, 4]}]': finished 2024-09-10T09:37:07.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:37:07 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]': finished 2024-09-10T09:37:07.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:37:07 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2024-09-10T09:37:07.885 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:37:07 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 4]}]': finished 2024-09-10T09:37:07.885 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:37:07 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]': finished 2024-09-10T09:37:07.885 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:37:07 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.39", "id": [5, 6]}]': finished 2024-09-10T09:37:07.885 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:37:07 smithi102 ceph-mon[35442]: osdmap e95: 12 total, 12 up, 12 in 2024-09-10T09:37:08.672 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:37:08 smithi005 ceph-mon[30350]: osdmap e96: 12 total, 12 up, 12 in 2024-09-10T09:37:08.810 INFO:tasks.cephadm.ceph_manager.ceph:need seq 244813135918 got 244813135920 for osd.8 2024-09-10T09:37:08.810 DEBUG:teuthology.parallel:result is None 2024-09-10T09:37:08.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:37:08 smithi159 ceph-mon[35513]: osdmap e96: 12 total, 12 up, 12 in 2024-09-10T09:37:08.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:37:08 smithi102 ceph-mon[35442]: osdmap e96: 12 total, 12 up, 12 in 2024-09-10T09:37:09.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:08.985+0000 7ff857400640 1 -- 172.21.15.5:0/3813910300 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff850103ff0 msgr2=0x7ff8501043d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:37:09.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:08.985+0000 7ff857400640 1 --2- 172.21.15.5:0/3813910300 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff850103ff0 0x7ff8501043d0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7ff840009900 tx=0x7ff84002a3f0 comp rx=0 tx=0).stop 2024-09-10T09:37:09.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:08.985+0000 7ff857400640 1 -- 172.21.15.5:0/3813910300 shutdown_connections 2024-09-10T09:37:09.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:08.985+0000 7ff857400640 1 --2- 172.21.15.5:0/3813910300 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff850109120 0x7ff850110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:09.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:08.985+0000 7ff857400640 1 --2- 172.21.15.5:0/3813910300 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff8501049a0 0x7ff8501089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:09.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:08.985+0000 7ff857400640 1 --2- 172.21.15.5:0/3813910300 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff850103ff0 0x7ff8501043d0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:09.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:08.985+0000 7ff857400640 1 -- 172.21.15.5:0/3813910300 >> 172.21.15.5:0/3813910300 conn(0x7ff8500ffa90 msgr2=0x7ff850101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:37:09.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:08.985+0000 7ff857400640 1 -- 172.21.15.5:0/3813910300 shutdown_connections 2024-09-10T09:37:09.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:08.985+0000 7ff857400640 1 -- 172.21.15.5:0/3813910300 wait complete. 2024-09-10T09:37:09.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:08.986+0000 7ff857400640 1 Processor -- start 2024-09-10T09:37:09.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:08.986+0000 7ff857400640 1 -- start start 2024-09-10T09:37:09.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:08.987+0000 7ff857400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff850103ff0 0x7ff85019a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:37:09.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:08.987+0000 7ff857400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff8501049a0 0x7ff85019ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:37:09.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:08.987+0000 7ff857400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff850109120 0x7ff85019eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:37:09.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:08.987+0000 7ff857400640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff850114140 con 0x7ff850103ff0 2024-09-10T09:37:09.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:08.987+0000 7ff857400640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7ff850113fc0 con 0x7ff8501049a0 2024-09-10T09:37:09.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:08.987+0000 7ff857400640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7ff8501142c0 con 0x7ff850109120 2024-09-10T09:37:09.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:08.987+0000 7ff854a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff850103ff0 0x7ff85019a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:37:09.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:08.987+0000 7ff854a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff850103ff0 0x7ff85019a620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57326/0 (socket says 172.21.15.5:57326) 2024-09-10T09:37:09.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:08.987+0000 7ff854a00640 1 -- 172.21.15.5:0/2216277324 learned_addr learned my addr 172.21.15.5:0/2216277324 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:37:09.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:08.987+0000 7ff847e00640 1 --2- 172.21.15.5:0/2216277324 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff8501049a0 0x7ff85019ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:37:09.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:08.987+0000 7ff855400640 1 --2- 172.21.15.5:0/2216277324 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff850109120 0x7ff85019eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:37:09.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:08.988+0000 7ff854a00640 1 -- 172.21.15.5:0/2216277324 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff850109120 msgr2=0x7ff85019eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:37:09.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:08.988+0000 7ff854a00640 1 --2- 172.21.15.5:0/2216277324 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff850109120 0x7ff85019eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:09.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:08.988+0000 7ff854a00640 1 -- 172.21.15.5:0/2216277324 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff8501049a0 msgr2=0x7ff85019ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:37:09.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:08.988+0000 7ff854a00640 1 --2- 172.21.15.5:0/2216277324 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff8501049a0 0x7ff85019ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:09.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:08.988+0000 7ff854a00640 1 -- 172.21.15.5:0/2216277324 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff85019f5d0 con 0x7ff850103ff0 2024-09-10T09:37:09.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:08.988+0000 7ff854a00640 1 --2- 172.21.15.5:0/2216277324 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff850103ff0 0x7ff85019a620 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7ff840006010 tx=0x7ff8400042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:37:09.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:08.988+0000 7ff84e000640 1 -- 172.21.15.5:0/2216277324 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff84003f030 con 0x7ff850103ff0 2024-09-10T09:37:09.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:08.988+0000 7ff857400640 1 -- 172.21.15.5:0/2216277324 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff85019f860 con 0x7ff850103ff0 2024-09-10T09:37:09.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:08.988+0000 7ff84e000640 1 -- 172.21.15.5:0/2216277324 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff840003ce0 con 0x7ff850103ff0 2024-09-10T09:37:09.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:08.988+0000 7ff84e000640 1 -- 172.21.15.5:0/2216277324 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff840035070 con 0x7ff850103ff0 2024-09-10T09:37:09.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:08.988+0000 7ff857400640 1 -- 172.21.15.5:0/2216277324 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8501a71a0 con 0x7ff850103ff0 2024-09-10T09:37:09.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:08.989+0000 7ff857400640 1 -- 172.21.15.5:0/2216277324 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff8501043d0 con 0x7ff850103ff0 2024-09-10T09:37:09.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:08.991+0000 7ff84e000640 1 -- 172.21.15.5:0/2216277324 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7ff840042030 con 0x7ff850103ff0 2024-09-10T09:37:09.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:08.993+0000 7ff84e000640 1 --2- 172.21.15.5:0/2216277324 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7ff8340a2c90 0x7ff8340a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:37:09.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:08.994+0000 7ff84e000640 1 -- 172.21.15.5:0/2216277324 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7ff8400ef340 con 0x7ff850103ff0 2024-09-10T09:37:09.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:08.994+0000 7ff847e00640 1 --2- 172.21.15.5:0/2216277324 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7ff8340a2c90 0x7ff8340a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:37:09.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:08.994+0000 7ff847e00640 1 --2- 172.21.15.5:0/2216277324 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7ff8340a2c90 0x7ff8340a5150 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7ff8500fff90 tx=0x7ff83800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:37:09.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:08.998+0000 7ff84e000640 1 -- 172.21.15.5:0/2216277324 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff8400bb760 con 0x7ff850103ff0 2024-09-10T09:37:09.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:09.188+0000 7ff857400640 1 -- 172.21.15.5:0/2216277324 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7ff85010c300 con 0x7ff850103ff0 2024-09-10T09:37:09.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:09.189+0000 7ff84e000640 1 -- 172.21.15.5:0/2216277324 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7ff8400bfa60 con 0x7ff850103ff0 2024-09-10T09:37:09.190 INFO:teuthology.orchestra.run.smithi005.stdout:137438953544 2024-09-10T09:37:09.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:09.191+0000 7ff857400640 1 -- 172.21.15.5:0/2216277324 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7ff8340a2c90 msgr2=0x7ff8340a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:37:09.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:09.191+0000 7ff857400640 1 --2- 172.21.15.5:0/2216277324 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7ff8340a2c90 0x7ff8340a5150 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7ff8500fff90 tx=0x7ff83800b390 comp rx=0 tx=0).stop 2024-09-10T09:37:09.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:09.191+0000 7ff857400640 1 -- 172.21.15.5:0/2216277324 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff850103ff0 msgr2=0x7ff85019a620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:37:09.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:09.191+0000 7ff857400640 1 --2- 172.21.15.5:0/2216277324 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff850103ff0 0x7ff85019a620 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7ff840006010 tx=0x7ff8400042f0 comp rx=0 tx=0).stop 2024-09-10T09:37:09.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:09.191+0000 7ff857400640 1 -- 172.21.15.5:0/2216277324 shutdown_connections 2024-09-10T09:37:09.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:09.191+0000 7ff857400640 1 --2- 172.21.15.5:0/2216277324 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7ff8340a2c90 0x7ff8340a5150 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:09.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:09.191+0000 7ff857400640 1 --2- 172.21.15.5:0/2216277324 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff850109120 0x7ff85019eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:09.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:09.191+0000 7ff857400640 1 --2- 172.21.15.5:0/2216277324 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff8501049a0 0x7ff85019ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:09.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:09.191+0000 7ff857400640 1 --2- 172.21.15.5:0/2216277324 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff850103ff0 0x7ff85019a620 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:09.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:09.191+0000 7ff857400640 1 -- 172.21.15.5:0/2216277324 >> 172.21.15.5:0/2216277324 conn(0x7ff8500ffa90 msgr2=0x7ff850100d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:37:09.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:09.192+0000 7ff857400640 1 -- 172.21.15.5:0/2216277324 shutdown_connections 2024-09-10T09:37:09.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:09.192+0000 7ff857400640 1 -- 172.21.15.5:0/2216277324 wait complete. 2024-09-10T09:37:09.195 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574933 got 55834574935 for osd.1 2024-09-10T09:37:09.195 DEBUG:teuthology.parallel:result is None 2024-09-10T09:37:09.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:09.403+0000 7f93b8000640 1 -- 172.21.15.5:0/3197715392 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f93b0109120 msgr2=0x7f93b0110ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:37:09.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:09.403+0000 7f93b8000640 1 --2- 172.21.15.5:0/3197715392 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f93b0109120 0x7f93b0110ca0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f93a8009ca0 tx=0x7f93a802a8c0 comp rx=0 tx=0).stop 2024-09-10T09:37:09.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:09.403+0000 7f93b8000640 1 -- 172.21.15.5:0/3197715392 shutdown_connections 2024-09-10T09:37:09.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:09.403+0000 7f93b8000640 1 --2- 172.21.15.5:0/3197715392 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f93b0109120 0x7f93b0110ca0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:09.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:09.403+0000 7f93b8000640 1 --2- 172.21.15.5:0/3197715392 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f93b01049a0 0x7f93b01089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:09.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:09.403+0000 7f93b8000640 1 --2- 172.21.15.5:0/3197715392 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f93b0103ff0 0x7f93b01043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:09.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:09.403+0000 7f93b8000640 1 -- 172.21.15.5:0/3197715392 >> 172.21.15.5:0/3197715392 conn(0x7f93b00ffa90 msgr2=0x7f93b0101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:37:09.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:09.403+0000 7f93b8000640 1 -- 172.21.15.5:0/3197715392 shutdown_connections 2024-09-10T09:37:09.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:09.403+0000 7f93b8000640 1 -- 172.21.15.5:0/3197715392 wait complete. 2024-09-10T09:37:09.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:09.404+0000 7f93b8000640 1 Processor -- start 2024-09-10T09:37:09.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:09.404+0000 7f93b8000640 1 -- start start 2024-09-10T09:37:09.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:09.404+0000 7f93b8000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f93b0103ff0 0x7f93b019a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:37:09.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:09.404+0000 7f93b8000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f93b01049a0 0x7f93b019ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:37:09.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:09.404+0000 7f93b8000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f93b0109120 0x7f93b019eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:37:09.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:09.404+0000 7f93b8000640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f93b01141a0 con 0x7f93b01049a0 2024-09-10T09:37:09.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:09.404+0000 7f93b8000640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f93b0114020 con 0x7f93b0103ff0 2024-09-10T09:37:09.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:09.404+0000 7f93b8000640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f93b0114320 con 0x7f93b0109120 2024-09-10T09:37:09.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:09.405+0000 7f93b4c00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f93b01049a0 0x7f93b019ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:37:09.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:09.405+0000 7f93b5600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f93b0103ff0 0x7f93b019a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:37:09.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:09.405+0000 7f93b4c00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f93b01049a0 0x7f93b019ab20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57350/0 (socket says 172.21.15.5:57350) 2024-09-10T09:37:09.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:09.405+0000 7f93b4c00640 1 -- 172.21.15.5:0/352345948 learned_addr learned my addr 172.21.15.5:0/352345948 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:37:09.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:09.405+0000 7f93b6000640 1 --2- 172.21.15.5:0/352345948 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f93b0109120 0x7f93b019eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:37:09.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:09.405+0000 7f93b5600640 1 -- 172.21.15.5:0/352345948 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f93b0109120 msgr2=0x7f93b019eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:37:09.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:09.405+0000 7f93b5600640 1 --2- 172.21.15.5:0/352345948 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f93b0109120 0x7f93b019eeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:09.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:09.405+0000 7f93b5600640 1 -- 172.21.15.5:0/352345948 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f93b01049a0 msgr2=0x7f93b019ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:37:09.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:09.405+0000 7f93b5600640 1 --2- 172.21.15.5:0/352345948 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f93b01049a0 0x7f93b019ab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:09.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:09.405+0000 7f93b5600640 1 -- 172.21.15.5:0/352345948 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93b019f590 con 0x7f93b0103ff0 2024-09-10T09:37:09.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:09.405+0000 7f93b6000640 1 --2- 172.21.15.5:0/352345948 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f93b0109120 0x7f93b019eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:37:09.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:09.405+0000 7f93b5600640 1 --2- 172.21.15.5:0/352345948 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f93b0103ff0 0x7f93b019a5e0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f93a400e9f0 tx=0x7f93a400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:37:09.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:09.406+0000 7f93ae000640 1 -- 172.21.15.5:0/352345948 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93a400ccc0 con 0x7f93b0103ff0 2024-09-10T09:37:09.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:09.406+0000 7f93b8000640 1 -- 172.21.15.5:0/352345948 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93b019f880 con 0x7f93b0103ff0 2024-09-10T09:37:09.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:09.406+0000 7f93ae000640 1 -- 172.21.15.5:0/352345948 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f93a4005020 con 0x7f93b0103ff0 2024-09-10T09:37:09.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:09.406+0000 7f93ae000640 1 -- 172.21.15.5:0/352345948 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93a40049e0 con 0x7f93b0103ff0 2024-09-10T09:37:09.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:09.406+0000 7f93b8000640 1 -- 172.21.15.5:0/352345948 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93b01a7160 con 0x7f93b0103ff0 2024-09-10T09:37:09.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:09.407+0000 7f93b8000640 1 -- 172.21.15.5:0/352345948 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f93b01043d0 con 0x7f93b0103ff0 2024-09-10T09:37:09.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:09.409+0000 7f93ae000640 1 -- 172.21.15.5:0/352345948 <== mon.2 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f93a4005560 con 0x7f93b0103ff0 2024-09-10T09:37:09.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:09.410+0000 7f93ae000640 1 --2- 172.21.15.5:0/352345948 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f93980a2c10 0x7f93980a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:37:09.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:09.411+0000 7f93ae000640 1 -- 172.21.15.5:0/352345948 <== mon.2 v2:172.21.15.102:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f93a40cf5b0 con 0x7f93b0103ff0 2024-09-10T09:37:09.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:09.411+0000 7f93b4c00640 1 --2- 172.21.15.5:0/352345948 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f93980a2c10 0x7f93980a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:37:09.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:09.412+0000 7f93b4c00640 1 --2- 172.21.15.5:0/352345948 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f93980a2c10 0x7f93980a50d0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f93b010ab00 tx=0x7f939c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:37:09.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:09.413+0000 7f93ae000640 1 -- 172.21.15.5:0/352345948 <== mon.2 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f93a40051d0 con 0x7f93b0103ff0 2024-09-10T09:37:09.426 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:37:09.426 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:37:09.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:37:09 smithi005 ceph-mon[30350]: pgmap v357: 129 pgs: 129 active+clean; 645 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 11 op/s 2024-09-10T09:37:09.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:37:09 smithi005 ceph-mon[30350]: from='client.? 172.21.15.5:0/2216277324' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-10T09:37:09.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:09.617+0000 7f93b8000640 1 -- 172.21.15.5:0/352345948 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f93b019b940 con 0x7f93b0103ff0 2024-09-10T09:37:09.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:09.617+0000 7f93ae000640 1 -- 172.21.15.5:0/352345948 <== mon.2 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f93a409bfc0 con 0x7f93b0103ff0 2024-09-10T09:37:09.618 INFO:teuthology.orchestra.run.smithi005.stdout:158913790019 2024-09-10T09:37:09.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:09.619+0000 7f93b8000640 1 -- 172.21.15.5:0/352345948 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f93980a2c10 msgr2=0x7f93980a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:37:09.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:09.619+0000 7f93b8000640 1 --2- 172.21.15.5:0/352345948 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f93980a2c10 0x7f93980a50d0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f93b010ab00 tx=0x7f939c005f90 comp rx=0 tx=0).stop 2024-09-10T09:37:09.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:09.619+0000 7f93b8000640 1 -- 172.21.15.5:0/352345948 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f93b0103ff0 msgr2=0x7f93b019a5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:37:09.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:09.619+0000 7f93b8000640 1 --2- 172.21.15.5:0/352345948 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f93b0103ff0 0x7f93b019a5e0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f93a400e9f0 tx=0x7f93a400eeb0 comp rx=0 tx=0).stop 2024-09-10T09:37:09.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:09.619+0000 7f93b8000640 1 -- 172.21.15.5:0/352345948 shutdown_connections 2024-09-10T09:37:09.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:09.619+0000 7f93b8000640 1 --2- 172.21.15.5:0/352345948 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f93980a2c10 0x7f93980a50d0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:09.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:09.619+0000 7f93b8000640 1 --2- 172.21.15.5:0/352345948 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f93b0109120 0x7f93b019eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:09.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:09.619+0000 7f93b8000640 1 --2- 172.21.15.5:0/352345948 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f93b01049a0 0x7f93b019ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:09.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:09.619+0000 7f93b8000640 1 --2- 172.21.15.5:0/352345948 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f93b0103ff0 0x7f93b019a5e0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:09.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:09.619+0000 7f93b8000640 1 -- 172.21.15.5:0/352345948 >> 172.21.15.5:0/352345948 conn(0x7f93b00ffa90 msgr2=0x7f93b01005c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:37:09.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:09.620+0000 7f93b8000640 1 -- 172.21.15.5:0/352345948 shutdown_connections 2024-09-10T09:37:09.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:09.620+0000 7f93b8000640 1 -- 172.21.15.5:0/352345948 wait complete. 2024-09-10T09:37:09.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:37:09 smithi159 ceph-mon[35513]: pgmap v357: 129 pgs: 129 active+clean; 645 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 11 op/s 2024-09-10T09:37:09.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:37:09 smithi159 ceph-mon[35513]: from='client.? 172.21.15.5:0/2216277324' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-10T09:37:09.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:37:09 smithi102 ceph-mon[35442]: pgmap v357: 129 pgs: 129 active+clean; 645 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 11 op/s 2024-09-10T09:37:09.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:37:09 smithi102 ceph-mon[35442]: from='client.? 172.21.15.5:0/2216277324' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-10T09:37:10.541 INFO:tasks.cephadm.ceph_manager.ceph:need seq 137438953541 got 137438953544 for osd.4 2024-09-10T09:37:10.541 DEBUG:teuthology.parallel:result is None 2024-09-10T09:37:10.792 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:37:10 smithi005 ceph-mon[30350]: pgmap v358: 129 pgs: 129 active+clean; 645 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-10T09:37:10.792 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:37:10 smithi005 ceph-mon[30350]: from='client.? 172.21.15.5:0/352345948' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-10T09:37:10.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:37:10 smithi159 ceph-mon[35513]: pgmap v358: 129 pgs: 129 active+clean; 645 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-10T09:37:10.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:37:10 smithi159 ceph-mon[35513]: from='client.? 172.21.15.5:0/352345948' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-10T09:37:10.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:37:10 smithi102 ceph-mon[35442]: pgmap v358: 129 pgs: 129 active+clean; 645 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-09-10T09:37:10.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:37:10 smithi102 ceph-mon[35442]: from='client.? 172.21.15.5:0/352345948' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-10T09:37:11.070 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:37:11.071 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:37:11.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:11.279+0000 7fb542c00640 1 -- 172.21.15.5:0/972456519 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb53c1019c0 msgr2=0x7fb53c101e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:37:11.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:11.279+0000 7fb542c00640 1 --2- 172.21.15.5:0/972456519 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb53c1019c0 0x7fb53c101e20 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fb528009930 tx=0x7fb52802a420 comp rx=0 tx=0).stop 2024-09-10T09:37:11.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:11.279+0000 7fb542c00640 1 -- 172.21.15.5:0/972456519 shutdown_connections 2024-09-10T09:37:11.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:11.279+0000 7fb542c00640 1 --2- 172.21.15.5:0/972456519 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb53c102360 0x7fb53c10e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:11.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:11.279+0000 7fb542c00640 1 --2- 172.21.15.5:0/972456519 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb53c1019c0 0x7fb53c101e20 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:11.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:11.279+0000 7fb542c00640 1 --2- 172.21.15.5:0/972456519 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb53c1079c0 0x7fb53c107da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:11.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:11.279+0000 7fb542c00640 1 -- 172.21.15.5:0/972456519 >> 172.21.15.5:0/972456519 conn(0x7fb53c0fd740 msgr2=0x7fb53c0ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:37:11.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:11.279+0000 7fb542c00640 1 -- 172.21.15.5:0/972456519 shutdown_connections 2024-09-10T09:37:11.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:11.279+0000 7fb542c00640 1 -- 172.21.15.5:0/972456519 wait complete. 2024-09-10T09:37:11.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:11.280+0000 7fb542c00640 1 Processor -- start 2024-09-10T09:37:11.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:11.281+0000 7fb542c00640 1 -- start start 2024-09-10T09:37:11.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:11.281+0000 7fb542c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb53c1019c0 0x7fb53c1982d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:37:11.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:11.281+0000 7fb542c00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb53c102360 0x7fb53c198810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:37:11.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:11.281+0000 7fb542c00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb53c1079c0 0x7fb53c19cba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:37:11.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:11.281+0000 7fb542c00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fb53c111e10 con 0x7fb53c1079c0 2024-09-10T09:37:11.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:11.281+0000 7fb542c00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fb53c111c90 con 0x7fb53c1019c0 2024-09-10T09:37:11.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:11.281+0000 7fb542c00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fb53c111f90 con 0x7fb53c102360 2024-09-10T09:37:11.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:11.281+0000 7fb540e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb53c102360 0x7fb53c198810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:37:11.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:11.281+0000 7fb541800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb53c1019c0 0x7fb53c1982d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:37:11.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:11.281+0000 7fb540e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb53c102360 0x7fb53c198810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:48986/0 (socket says 172.21.15.5:48986) 2024-09-10T09:37:11.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:11.281+0000 7fb542200640 1 --2- 172.21.15.5:0/3444671437 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb53c1079c0 0x7fb53c19cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:37:11.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:11.281+0000 7fb540e00640 1 -- 172.21.15.5:0/3444671437 learned_addr learned my addr 172.21.15.5:0/3444671437 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:37:11.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:11.282+0000 7fb541800640 1 -- 172.21.15.5:0/3444671437 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb53c102360 msgr2=0x7fb53c198810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:37:11.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:11.282+0000 7fb541800640 1 --2- 172.21.15.5:0/3444671437 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb53c102360 0x7fb53c198810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:11.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:11.282+0000 7fb541800640 1 -- 172.21.15.5:0/3444671437 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb53c1079c0 msgr2=0x7fb53c19cba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:37:11.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:11.282+0000 7fb541800640 1 --2- 172.21.15.5:0/3444671437 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb53c1079c0 0x7fb53c19cba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:11.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:11.282+0000 7fb541800640 1 -- 172.21.15.5:0/3444671437 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb53c19d280 con 0x7fb53c1019c0 2024-09-10T09:37:11.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:11.282+0000 7fb541800640 1 --2- 172.21.15.5:0/3444671437 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb53c1019c0 0x7fb53c1982d0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fb530005fd0 tx=0x7fb53000dde0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:37:11.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:11.282+0000 7fb53a000640 1 -- 172.21.15.5:0/3444671437 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb530009c30 con 0x7fb53c1019c0 2024-09-10T09:37:11.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:11.283+0000 7fb53a000640 1 -- 172.21.15.5:0/3444671437 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb530009dd0 con 0x7fb53c1019c0 2024-09-10T09:37:11.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:11.283+0000 7fb542c00640 1 -- 172.21.15.5:0/3444671437 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb53c19d570 con 0x7fb53c1019c0 2024-09-10T09:37:11.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:11.283+0000 7fb53a000640 1 -- 172.21.15.5:0/3444671437 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb530010460 con 0x7fb53c1019c0 2024-09-10T09:37:11.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:11.283+0000 7fb542c00640 1 -- 172.21.15.5:0/3444671437 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb53c1a4e50 con 0x7fb53c1019c0 2024-09-10T09:37:11.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:11.284+0000 7fb542c00640 1 -- 172.21.15.5:0/3444671437 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb53c101e20 con 0x7fb53c1019c0 2024-09-10T09:37:11.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:11.286+0000 7fb53a000640 1 -- 172.21.15.5:0/3444671437 <== mon.2 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7fb530003f50 con 0x7fb53c1019c0 2024-09-10T09:37:11.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:11.287+0000 7fb53a000640 1 --2- 172.21.15.5:0/3444671437 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fb5240a2c90 0x7fb5240a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:37:11.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:11.287+0000 7fb53a000640 1 -- 172.21.15.5:0/3444671437 <== mon.2 v2:172.21.15.102:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fb5300cf3d0 con 0x7fb53c1019c0 2024-09-10T09:37:11.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:11.287+0000 7fb540e00640 1 --2- 172.21.15.5:0/3444671437 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fb5240a2c90 0x7fb5240a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:37:11.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:11.288+0000 7fb540e00640 1 --2- 172.21.15.5:0/3444671437 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fb5240a2c90 0x7fb5240a5150 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fb528005c40 tx=0x7fb528005b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:37:11.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:11.289+0000 7fb53a000640 1 -- 172.21.15.5:0/3444671437 <== mon.2 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb530017020 con 0x7fb53c1019c0 2024-09-10T09:37:11.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:11.479+0000 7fb542c00640 1 -- 172.21.15.5:0/3444671437 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7fb53c199630 con 0x7fb53c1019c0 2024-09-10T09:37:11.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:11.480+0000 7fb53a000640 1 -- 172.21.15.5:0/3444671437 <== mon.2 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fb53c199630 con 0x7fb53c1019c0 2024-09-10T09:37:11.480 INFO:teuthology.orchestra.run.smithi005.stdout:214748364855 2024-09-10T09:37:11.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:11.482+0000 7fb542c00640 1 -- 172.21.15.5:0/3444671437 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fb5240a2c90 msgr2=0x7fb5240a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:37:11.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:11.482+0000 7fb542c00640 1 --2- 172.21.15.5:0/3444671437 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fb5240a2c90 0x7fb5240a5150 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fb528005c40 tx=0x7fb528005b50 comp rx=0 tx=0).stop 2024-09-10T09:37:11.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:11.482+0000 7fb542c00640 1 -- 172.21.15.5:0/3444671437 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb53c1019c0 msgr2=0x7fb53c1982d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:37:11.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:11.482+0000 7fb542c00640 1 --2- 172.21.15.5:0/3444671437 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb53c1019c0 0x7fb53c1982d0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fb530005fd0 tx=0x7fb53000dde0 comp rx=0 tx=0).stop 2024-09-10T09:37:11.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:11.482+0000 7fb542c00640 1 -- 172.21.15.5:0/3444671437 shutdown_connections 2024-09-10T09:37:11.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:11.482+0000 7fb542c00640 1 --2- 172.21.15.5:0/3444671437 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fb5240a2c90 0x7fb5240a5150 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:11.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:11.482+0000 7fb542c00640 1 --2- 172.21.15.5:0/3444671437 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb53c1079c0 0x7fb53c19cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:11.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:11.482+0000 7fb542c00640 1 --2- 172.21.15.5:0/3444671437 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb53c102360 0x7fb53c198810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:11.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:11.482+0000 7fb542c00640 1 --2- 172.21.15.5:0/3444671437 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb53c1019c0 0x7fb53c1982d0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:11.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:11.482+0000 7fb542c00640 1 -- 172.21.15.5:0/3444671437 >> 172.21.15.5:0/3444671437 conn(0x7fb53c0fd740 msgr2=0x7fb53c0fdef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:37:11.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:11.482+0000 7fb542c00640 1 -- 172.21.15.5:0/3444671437 shutdown_connections 2024-09-10T09:37:11.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:11.482+0000 7fb542c00640 1 -- 172.21.15.5:0/3444671437 wait complete. 2024-09-10T09:37:11.788 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:37:11 smithi005 ceph-mon[30350]: from='client.? 172.21.15.5:0/3444671437' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-10T09:37:11.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:37:11 smithi159 ceph-mon[35513]: from='client.? 172.21.15.5:0/3444671437' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-10T09:37:11.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:37:11 smithi102 ceph-mon[35442]: from='client.? 172.21.15.5:0/3444671437' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-10T09:37:11.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:11.900+0000 7f5566a00640 1 -- 172.21.15.5:0/314391116 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f55601079c0 msgr2=0x7f5560107da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:37:11.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:11.900+0000 7f5566a00640 1 --2- 172.21.15.5:0/314391116 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f55601079c0 0x7f5560107da0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f5554009930 tx=0x7f555402a420 comp rx=0 tx=0).stop 2024-09-10T09:37:11.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:11.900+0000 7f5566a00640 1 -- 172.21.15.5:0/314391116 shutdown_connections 2024-09-10T09:37:11.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:11.900+0000 7f5566a00640 1 --2- 172.21.15.5:0/314391116 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5560102360 0x7f556010e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:11.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:11.900+0000 7f5566a00640 1 --2- 172.21.15.5:0/314391116 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f55601019c0 0x7f5560101e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:11.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:11.900+0000 7f5566a00640 1 --2- 172.21.15.5:0/314391116 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f55601079c0 0x7f5560107da0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:11.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:11.900+0000 7f5566a00640 1 -- 172.21.15.5:0/314391116 >> 172.21.15.5:0/314391116 conn(0x7f55600fd740 msgr2=0x7f55600ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:37:11.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:11.900+0000 7f5566a00640 1 -- 172.21.15.5:0/314391116 shutdown_connections 2024-09-10T09:37:11.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:11.900+0000 7f5566a00640 1 -- 172.21.15.5:0/314391116 wait complete. 2024-09-10T09:37:11.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:11.901+0000 7f5566a00640 1 Processor -- start 2024-09-10T09:37:11.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:11.901+0000 7f5566a00640 1 -- start start 2024-09-10T09:37:11.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:11.902+0000 7f5566a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f55601019c0 0x7f556019a510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:37:11.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:11.902+0000 7f5566a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5560102360 0x7f556019aa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:37:11.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:11.902+0000 7f5566a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f55601079c0 0x7f556019ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:37:11.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:11.902+0000 7f5566a00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f5560111dc0 con 0x7f55601079c0 2024-09-10T09:37:11.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:11.902+0000 7f5566a00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f5560111c40 con 0x7f55601019c0 2024-09-10T09:37:11.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:11.902+0000 7f5566a00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f5560111f40 con 0x7f5560102360 2024-09-10T09:37:11.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:11.902+0000 7f5566000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f55601079c0 0x7f556019ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:37:11.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:11.902+0000 7f5565600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f55601019c0 0x7f556019a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:37:11.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:11.902+0000 7f5565600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f55601019c0 0x7f556019a510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:55352/0 (socket says 172.21.15.5:55352) 2024-09-10T09:37:11.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:11.902+0000 7f5565600640 1 -- 172.21.15.5:0/1599511843 learned_addr learned my addr 172.21.15.5:0/1599511843 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:37:11.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:11.902+0000 7f5564c00640 1 --2- 172.21.15.5:0/1599511843 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5560102360 0x7f556019aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:37:11.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:11.903+0000 7f5565600640 1 -- 172.21.15.5:0/1599511843 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5560102360 msgr2=0x7f556019aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:37:11.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:11.903+0000 7f5565600640 1 --2- 172.21.15.5:0/1599511843 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5560102360 0x7f556019aa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:11.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:11.903+0000 7f5565600640 1 -- 172.21.15.5:0/1599511843 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f55601079c0 msgr2=0x7f556019ede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:37:11.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:11.903+0000 7f5565600640 1 --2- 172.21.15.5:0/1599511843 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f55601079c0 0x7f556019ede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:11.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:11.903+0000 7f5565600640 1 -- 172.21.15.5:0/1599511843 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f556019f4c0 con 0x7f55601019c0 2024-09-10T09:37:11.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:11.903+0000 7f5566000640 1 --2- 172.21.15.5:0/1599511843 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f55601079c0 0x7f556019ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:37:11.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:11.903+0000 7f5565600640 1 --2- 172.21.15.5:0/1599511843 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f55601019c0 0x7f556019a510 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f5554009230 tx=0x7f5554004270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:37:11.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:11.903+0000 7f555e000640 1 -- 172.21.15.5:0/1599511843 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f555403f030 con 0x7f55601019c0 2024-09-10T09:37:11.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:11.903+0000 7f5566a00640 1 -- 172.21.15.5:0/1599511843 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f556019f750 con 0x7f55601019c0 2024-09-10T09:37:11.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:11.903+0000 7f555e000640 1 -- 172.21.15.5:0/1599511843 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5554003ad0 con 0x7f55601019c0 2024-09-10T09:37:11.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:11.903+0000 7f555e000640 1 -- 172.21.15.5:0/1599511843 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5554035070 con 0x7f55601019c0 2024-09-10T09:37:11.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:11.903+0000 7f5566a00640 1 -- 172.21.15.5:0/1599511843 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55601a7090 con 0x7f55601019c0 2024-09-10T09:37:11.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:11.904+0000 7f5566a00640 1 -- 172.21.15.5:0/1599511843 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5560101e20 con 0x7f55601019c0 2024-09-10T09:37:11.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:11.907+0000 7f555e000640 1 -- 172.21.15.5:0/1599511843 <== mon.2 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f5554042030 con 0x7f55601019c0 2024-09-10T09:37:11.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:11.908+0000 7f555e000640 1 --2- 172.21.15.5:0/1599511843 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f55480a2c90 0x7f55480a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:37:11.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:11.908+0000 7f555e000640 1 -- 172.21.15.5:0/1599511843 <== mon.2 v2:172.21.15.102:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f55540efe40 con 0x7f55601019c0 2024-09-10T09:37:11.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:11.908+0000 7f5564c00640 1 --2- 172.21.15.5:0/1599511843 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f55480a2c90 0x7f55480a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:37:11.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:11.909+0000 7f5564c00640 1 --2- 172.21.15.5:0/1599511843 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f55480a2c90 0x7f55480a5150 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f556010ec90 tx=0x7f554c00b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:37:11.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:11.910+0000 7f555e000640 1 -- 172.21.15.5:0/1599511843 <== mon.2 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f55540bc1a0 con 0x7f55601019c0 2024-09-10T09:37:12.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:12.096+0000 7f5566a00640 1 -- 172.21.15.5:0/1599511843 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f55601a7260 con 0x7f55601019c0 2024-09-10T09:37:12.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:12.097+0000 7f555e000640 1 -- 172.21.15.5:0/1599511843 <== mon.2 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f55540c04a0 con 0x7f55601019c0 2024-09-10T09:37:12.098 INFO:teuthology.orchestra.run.smithi005.stdout:193273528381 2024-09-10T09:37:12.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:12.099+0000 7f5566a00640 1 -- 172.21.15.5:0/1599511843 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f55480a2c90 msgr2=0x7f55480a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:37:12.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:12.099+0000 7f5566a00640 1 --2- 172.21.15.5:0/1599511843 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f55480a2c90 0x7f55480a5150 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f556010ec90 tx=0x7f554c00b390 comp rx=0 tx=0).stop 2024-09-10T09:37:12.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:12.099+0000 7f5566a00640 1 -- 172.21.15.5:0/1599511843 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f55601019c0 msgr2=0x7f556019a510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:37:12.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:12.099+0000 7f5566a00640 1 --2- 172.21.15.5:0/1599511843 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f55601019c0 0x7f556019a510 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f5554009230 tx=0x7f5554004270 comp rx=0 tx=0).stop 2024-09-10T09:37:12.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:12.099+0000 7f5566a00640 1 -- 172.21.15.5:0/1599511843 shutdown_connections 2024-09-10T09:37:12.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:12.099+0000 7f5566a00640 1 --2- 172.21.15.5:0/1599511843 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f55480a2c90 0x7f55480a5150 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:12.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:12.099+0000 7f5566a00640 1 --2- 172.21.15.5:0/1599511843 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f55601079c0 0x7f556019ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:12.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:12.099+0000 7f5566a00640 1 --2- 172.21.15.5:0/1599511843 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5560102360 0x7f556019aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:12.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:12.099+0000 7f5566a00640 1 --2- 172.21.15.5:0/1599511843 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f55601019c0 0x7f556019a510 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:12.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:12.099+0000 7f5566a00640 1 -- 172.21.15.5:0/1599511843 >> 172.21.15.5:0/1599511843 conn(0x7f55600fd740 msgr2=0x7f55600fde70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:37:12.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:12.099+0000 7f5566a00640 1 -- 172.21.15.5:0/1599511843 shutdown_connections 2024-09-10T09:37:12.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:12.099+0000 7f5566a00640 1 -- 172.21.15.5:0/1599511843 wait complete. 2024-09-10T09:37:12.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:12.278+0000 7f3693200640 1 -- 172.21.15.5:0/4271709341 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f368c074f40 msgr2=0x7f368c10de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:37:12.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:12.278+0000 7f3693200640 1 --2- 172.21.15.5:0/4271709341 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f368c074f40 0x7f368c10de80 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f3674009900 tx=0x7f367402a480 comp rx=0 tx=0).stop 2024-09-10T09:37:12.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:12.278+0000 7f3693200640 1 -- 172.21.15.5:0/4271709341 shutdown_connections 2024-09-10T09:37:12.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:12.278+0000 7f3693200640 1 --2- 172.21.15.5:0/4271709341 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f368c10e3c0 0x7f368c110810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:12.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:12.278+0000 7f3693200640 1 --2- 172.21.15.5:0/4271709341 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f368c074f40 0x7f368c10de80 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:12.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:12.278+0000 7f3693200640 1 --2- 172.21.15.5:0/4271709341 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f368c074620 0x7f368c074a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:12.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:12.278+0000 7f3693200640 1 -- 172.21.15.5:0/4271709341 >> 172.21.15.5:0/4271709341 conn(0x7f368c0fd8c0 msgr2=0x7f368c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:37:12.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:12.278+0000 7f3693200640 1 -- 172.21.15.5:0/4271709341 shutdown_connections 2024-09-10T09:37:12.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:12.278+0000 7f3693200640 1 -- 172.21.15.5:0/4271709341 wait complete. 2024-09-10T09:37:12.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:12.279+0000 7f3693200640 1 Processor -- start 2024-09-10T09:37:12.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:12.279+0000 7f3693200640 1 -- start start 2024-09-10T09:37:12.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:12.279+0000 7f3693200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f368c074620 0x7f368c19a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:37:12.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:12.279+0000 7f3693200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f368c074f40 0x7f368c19abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:37:12.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:12.279+0000 7f3693200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f368c10e3c0 0x7f368c19ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:37:12.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:12.279+0000 7f3693200640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f368c114190 con 0x7f368c074620 2024-09-10T09:37:12.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:12.280+0000 7f3693200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f368c114010 con 0x7f368c10e3c0 2024-09-10T09:37:12.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:12.280+0000 7f3693200640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f368c114310 con 0x7f368c074f40 2024-09-10T09:37:12.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:12.280+0000 7f368be00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f368c074620 0x7f368c19a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:37:12.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:12.280+0000 7f3683400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f368c074f40 0x7f368c19abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:37:12.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:12.280+0000 7f368be00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f368c074620 0x7f368c19a670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45518/0 (socket says 172.21.15.5:45518) 2024-09-10T09:37:12.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:12.280+0000 7f3691200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f368c10e3c0 0x7f368c19ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:37:12.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:12.280+0000 7f368be00640 1 -- 172.21.15.5:0/3999847638 learned_addr learned my addr 172.21.15.5:0/3999847638 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:37:12.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:12.280+0000 7f3691200640 1 -- 172.21.15.5:0/3999847638 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f368c074f40 msgr2=0x7f368c19abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:37:12.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:12.280+0000 7f3691200640 1 --2- 172.21.15.5:0/3999847638 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f368c074f40 0x7f368c19abb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:12.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:12.280+0000 7f3691200640 1 -- 172.21.15.5:0/3999847638 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f368c074620 msgr2=0x7f368c19a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:37:12.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:12.280+0000 7f3691200640 1 --2- 172.21.15.5:0/3999847638 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f368c074620 0x7f368c19a670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:12.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:12.280+0000 7f3691200640 1 -- 172.21.15.5:0/3999847638 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f368c19f620 con 0x7f368c10e3c0 2024-09-10T09:37:12.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:12.280+0000 7f3683400640 1 --2- 172.21.15.5:0/3999847638 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f368c074f40 0x7f368c19abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:37:12.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:12.280+0000 7f3691200640 1 --2- 172.21.15.5:0/3999847638 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f368c10e3c0 0x7f368c19ef40 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f3684009fd0 tx=0x7f368400ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:37:12.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:12.281+0000 7f3689600640 1 -- 172.21.15.5:0/3999847638 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f368400ce20 con 0x7f368c10e3c0 2024-09-10T09:37:12.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:12.281+0000 7f3693200640 1 -- 172.21.15.5:0/3999847638 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f368c19f910 con 0x7f368c10e3c0 2024-09-10T09:37:12.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:12.281+0000 7f3689600640 1 -- 172.21.15.5:0/3999847638 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3684005020 con 0x7f368c10e3c0 2024-09-10T09:37:12.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:12.281+0000 7f3689600640 1 -- 172.21.15.5:0/3999847638 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3684010720 con 0x7f368c10e3c0 2024-09-10T09:37:12.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:12.281+0000 7f3693200640 1 -- 172.21.15.5:0/3999847638 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f368c1a71f0 con 0x7f368c10e3c0 2024-09-10T09:37:12.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:12.282+0000 7f3693200640 1 -- 172.21.15.5:0/3999847638 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f368c074a00 con 0x7f368c10e3c0 2024-09-10T09:37:12.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:12.284+0000 7f3689600640 1 -- 172.21.15.5:0/3999847638 <== mon.2 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f3684005540 con 0x7f368c10e3c0 2024-09-10T09:37:12.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:12.285+0000 7f3689600640 1 --2- 172.21.15.5:0/3999847638 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f36700a2c90 0x7f36700a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:37:12.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:12.286+0000 7f3689600640 1 -- 172.21.15.5:0/3999847638 <== mon.2 v2:172.21.15.102:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f36840cfba0 con 0x7f368c10e3c0 2024-09-10T09:37:12.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:12.286+0000 7f368be00640 1 --2- 172.21.15.5:0/3999847638 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f36700a2c90 0x7f36700a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:37:12.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:12.286+0000 7f368be00640 1 --2- 172.21.15.5:0/3999847638 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f36700a2c90 0x7f36700a5150 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f367c009c90 tx=0x7f367c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:37:12.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:12.288+0000 7f3689600640 1 -- 172.21.15.5:0/3999847638 <== mon.2 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f368409c550 con 0x7f368c10e3c0 2024-09-10T09:37:12.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:12.482+0000 7f3693200640 1 -- 172.21.15.5:0/3999847638 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f368c108fb0 con 0x7f368c10e3c0 2024-09-10T09:37:12.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:12.483+0000 7f3689600640 1 -- 172.21.15.5:0/3999847638 <== mon.2 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f368409be50 con 0x7f368c10e3c0 2024-09-10T09:37:12.605 INFO:teuthology.orchestra.run.smithi005.stdout:77309411410 2024-09-10T09:37:12.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:12.485+0000 7f3693200640 1 -- 172.21.15.5:0/3999847638 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f36700a2c90 msgr2=0x7f36700a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:37:12.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:12.485+0000 7f3693200640 1 --2- 172.21.15.5:0/3999847638 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f36700a2c90 0x7f36700a5150 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f367c009c90 tx=0x7f367c005f90 comp rx=0 tx=0).stop 2024-09-10T09:37:12.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:12.485+0000 7f3693200640 1 -- 172.21.15.5:0/3999847638 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f368c10e3c0 msgr2=0x7f368c19ef40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:37:12.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:12.485+0000 7f3693200640 1 --2- 172.21.15.5:0/3999847638 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f368c10e3c0 0x7f368c19ef40 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f3684009fd0 tx=0x7f368400ef10 comp rx=0 tx=0).stop 2024-09-10T09:37:12.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:12.485+0000 7f3693200640 1 -- 172.21.15.5:0/3999847638 shutdown_connections 2024-09-10T09:37:12.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:12.485+0000 7f3693200640 1 --2- 172.21.15.5:0/3999847638 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f36700a2c90 0x7f36700a5150 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:12.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:12.485+0000 7f3693200640 1 --2- 172.21.15.5:0/3999847638 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f368c10e3c0 0x7f368c19ef40 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:12.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:12.485+0000 7f3693200640 1 --2- 172.21.15.5:0/3999847638 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f368c074f40 0x7f368c19abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:12.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:12.485+0000 7f3693200640 1 --2- 172.21.15.5:0/3999847638 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f368c074620 0x7f368c19a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:12.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:12.485+0000 7f3693200640 1 -- 172.21.15.5:0/3999847638 >> 172.21.15.5:0/3999847638 conn(0x7f368c0fd8c0 msgr2=0x7f368c0fe560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:37:12.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:12.485+0000 7f3693200640 1 -- 172.21.15.5:0/3999847638 shutdown_connections 2024-09-10T09:37:12.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:12.485+0000 7f3693200640 1 -- 172.21.15.5:0/3999847638 wait complete. 2024-09-10T09:37:12.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:37:12 smithi005 ceph-mon[30350]: pgmap v359: 129 pgs: 129 active+clean; 645 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 11 op/s; 126 B/s, 0 keys/s, 1 objects/s recovering 2024-09-10T09:37:12.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:37:12 smithi005 ceph-mon[30350]: from='client.? 172.21.15.5:0/1599511843' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-10T09:37:12.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:37:12 smithi005 ceph-mon[30350]: from='client.? 172.21.15.5:0/3999847638' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-10T09:37:12.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:37:12 smithi159 ceph-mon[35513]: pgmap v359: 129 pgs: 129 active+clean; 645 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 11 op/s; 126 B/s, 0 keys/s, 1 objects/s recovering 2024-09-10T09:37:12.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:37:12 smithi159 ceph-mon[35513]: from='client.? 172.21.15.5:0/1599511843' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-10T09:37:12.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:37:12 smithi159 ceph-mon[35513]: from='client.? 172.21.15.5:0/3999847638' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-10T09:37:12.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:37:12 smithi102 ceph-mon[35442]: pgmap v359: 129 pgs: 129 active+clean; 645 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 11 op/s; 126 B/s, 0 keys/s, 1 objects/s recovering 2024-09-10T09:37:12.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:37:12 smithi102 ceph-mon[35442]: from='client.? 172.21.15.5:0/1599511843' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-10T09:37:12.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:37:12 smithi102 ceph-mon[35442]: from='client.? 172.21.15.5:0/3999847638' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-10T09:37:13.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:13.754+0000 7ff0dea00640 1 -- 172.21.15.5:0/1117696326 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff0d8109120 msgr2=0x7ff0d8110ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:37:13.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:13.754+0000 7ff0dea00640 1 --2- 172.21.15.5:0/1117696326 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff0d8109120 0x7ff0d8110ca0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7ff0d0009ca0 tx=0x7ff0d002a8c0 comp rx=0 tx=0).stop 2024-09-10T09:37:13.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:13.754+0000 7ff0dea00640 1 -- 172.21.15.5:0/1117696326 shutdown_connections 2024-09-10T09:37:13.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:13.754+0000 7ff0dea00640 1 --2- 172.21.15.5:0/1117696326 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff0d8109120 0x7ff0d8110ca0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:13.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:13.754+0000 7ff0dea00640 1 --2- 172.21.15.5:0/1117696326 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff0d81049a0 0x7ff0d81089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:13.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:13.754+0000 7ff0dea00640 1 --2- 172.21.15.5:0/1117696326 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff0d8103ff0 0x7ff0d81043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:13.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:13.754+0000 7ff0dea00640 1 -- 172.21.15.5:0/1117696326 >> 172.21.15.5:0/1117696326 conn(0x7ff0d80ffa90 msgr2=0x7ff0d8101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:37:13.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:13.754+0000 7ff0dea00640 1 -- 172.21.15.5:0/1117696326 shutdown_connections 2024-09-10T09:37:13.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:13.754+0000 7ff0dea00640 1 -- 172.21.15.5:0/1117696326 wait complete. 2024-09-10T09:37:13.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:13.755+0000 7ff0dea00640 1 Processor -- start 2024-09-10T09:37:13.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:13.755+0000 7ff0dea00640 1 -- start start 2024-09-10T09:37:13.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:13.756+0000 7ff0dea00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff0d8103ff0 0x7ff0d819a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:37:13.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:13.756+0000 7ff0dea00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff0d81049a0 0x7ff0d819ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:37:13.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:13.756+0000 7ff0dea00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff0d8109120 0x7ff0d819eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:37:13.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:13.756+0000 7ff0dea00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff0d81141a0 con 0x7ff0d81049a0 2024-09-10T09:37:13.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:13.756+0000 7ff0dea00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7ff0d8114020 con 0x7ff0d8103ff0 2024-09-10T09:37:13.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:13.756+0000 7ff0dea00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7ff0d8114320 con 0x7ff0d8109120 2024-09-10T09:37:13.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:13.756+0000 7ff0cfe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff0d81049a0 0x7ff0d819ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:37:13.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:13.756+0000 7ff0cfe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff0d81049a0 0x7ff0d819ab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45540/0 (socket says 172.21.15.5:45540) 2024-09-10T09:37:13.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:13.756+0000 7ff0cfe00640 1 -- 172.21.15.5:0/1504453401 learned_addr learned my addr 172.21.15.5:0/1504453401 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:37:13.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:13.756+0000 7ff0d7e00640 1 --2- 172.21.15.5:0/1504453401 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff0d8103ff0 0x7ff0d819a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:37:13.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:13.756+0000 7ff0dca00640 1 --2- 172.21.15.5:0/1504453401 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff0d8109120 0x7ff0d819eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:37:13.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:13.757+0000 7ff0d7e00640 1 -- 172.21.15.5:0/1504453401 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff0d8109120 msgr2=0x7ff0d819eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:37:13.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:13.757+0000 7ff0d7e00640 1 --2- 172.21.15.5:0/1504453401 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff0d8109120 0x7ff0d819eed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:13.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:13.757+0000 7ff0d7e00640 1 -- 172.21.15.5:0/1504453401 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff0d81049a0 msgr2=0x7ff0d819ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:37:13.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:13.757+0000 7ff0d7e00640 1 --2- 172.21.15.5:0/1504453401 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff0d81049a0 0x7ff0d819ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:13.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:13.757+0000 7ff0d7e00640 1 -- 172.21.15.5:0/1504453401 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0d819f5b0 con 0x7ff0d8103ff0 2024-09-10T09:37:13.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:13.757+0000 7ff0dca00640 1 --2- 172.21.15.5:0/1504453401 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff0d8109120 0x7ff0d819eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:37:13.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:13.757+0000 7ff0cfe00640 1 --2- 172.21.15.5:0/1504453401 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff0d81049a0 0x7ff0d819ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:37:13.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:13.757+0000 7ff0d7e00640 1 --2- 172.21.15.5:0/1504453401 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff0d8103ff0 0x7ff0d819a600 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7ff0c800e9f0 tx=0x7ff0c800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:37:13.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:13.757+0000 7ff0d5600640 1 -- 172.21.15.5:0/1504453401 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0c800cd80 con 0x7ff0d8103ff0 2024-09-10T09:37:13.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:13.757+0000 7ff0d5600640 1 -- 172.21.15.5:0/1504453401 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff0c80035f0 con 0x7ff0d8103ff0 2024-09-10T09:37:13.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:13.757+0000 7ff0dea00640 1 -- 172.21.15.5:0/1504453401 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0d819f8a0 con 0x7ff0d8103ff0 2024-09-10T09:37:13.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:13.757+0000 7ff0d5600640 1 -- 172.21.15.5:0/1504453401 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0c8010690 con 0x7ff0d8103ff0 2024-09-10T09:37:13.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:13.757+0000 7ff0dea00640 1 -- 172.21.15.5:0/1504453401 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0d81a7180 con 0x7ff0d8103ff0 2024-09-10T09:37:13.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:13.758+0000 7ff0dea00640 1 -- 172.21.15.5:0/1504453401 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff0d81043d0 con 0x7ff0d8103ff0 2024-09-10T09:37:13.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:13.760+0000 7ff0d5600640 1 -- 172.21.15.5:0/1504453401 <== mon.2 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7ff0c8003b30 con 0x7ff0d8103ff0 2024-09-10T09:37:13.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:13.762+0000 7ff0d5600640 1 --2- 172.21.15.5:0/1504453401 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7ff0bc0a2c10 0x7ff0bc0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:37:13.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:13.762+0000 7ff0d5600640 1 -- 172.21.15.5:0/1504453401 <== mon.2 v2:172.21.15.102:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7ff0c80cf9b0 con 0x7ff0d8103ff0 2024-09-10T09:37:13.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:13.762+0000 7ff0cfe00640 1 --2- 172.21.15.5:0/1504453401 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7ff0bc0a2c10 0x7ff0bc0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:37:13.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:13.762+0000 7ff0cfe00640 1 --2- 172.21.15.5:0/1504453401 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7ff0bc0a2c10 0x7ff0bc0a50d0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7ff0d810ab20 tx=0x7ff0c0005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:37:13.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:13.766+0000 7ff0d5600640 1 -- 172.21.15.5:0/1504453401 <== mon.2 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff0c809c360 con 0x7ff0d8103ff0 2024-09-10T09:37:13.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:13.956+0000 7ff0dea00640 1 -- 172.21.15.5:0/1504453401 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7ff0d819b960 con 0x7ff0d8103ff0 2024-09-10T09:37:13.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:13.956+0000 7ff0d5600640 1 -- 172.21.15.5:0/1504453401 <== mon.2 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7ff0c809bc60 con 0x7ff0d8103ff0 2024-09-10T09:37:13.957 INFO:teuthology.orchestra.run.smithi005.stdout:107374182478 2024-09-10T09:37:13.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:13.959+0000 7ff0dea00640 1 -- 172.21.15.5:0/1504453401 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7ff0bc0a2c10 msgr2=0x7ff0bc0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:37:13.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:13.959+0000 7ff0dea00640 1 --2- 172.21.15.5:0/1504453401 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7ff0bc0a2c10 0x7ff0bc0a50d0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7ff0d810ab20 tx=0x7ff0c0005f90 comp rx=0 tx=0).stop 2024-09-10T09:37:13.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:13.959+0000 7ff0dea00640 1 -- 172.21.15.5:0/1504453401 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff0d8103ff0 msgr2=0x7ff0d819a600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:37:13.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:13.959+0000 7ff0dea00640 1 --2- 172.21.15.5:0/1504453401 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff0d8103ff0 0x7ff0d819a600 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7ff0c800e9f0 tx=0x7ff0c800eeb0 comp rx=0 tx=0).stop 2024-09-10T09:37:13.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:13.959+0000 7ff0dea00640 1 -- 172.21.15.5:0/1504453401 shutdown_connections 2024-09-10T09:37:13.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:13.959+0000 7ff0dea00640 1 --2- 172.21.15.5:0/1504453401 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7ff0bc0a2c10 0x7ff0bc0a50d0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:13.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:13.959+0000 7ff0dea00640 1 --2- 172.21.15.5:0/1504453401 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff0d8109120 0x7ff0d819eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:13.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:13.959+0000 7ff0dea00640 1 --2- 172.21.15.5:0/1504453401 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff0d81049a0 0x7ff0d819ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:13.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:13.959+0000 7ff0dea00640 1 --2- 172.21.15.5:0/1504453401 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff0d8103ff0 0x7ff0d819a600 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:13.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:13.959+0000 7ff0dea00640 1 -- 172.21.15.5:0/1504453401 >> 172.21.15.5:0/1504453401 conn(0x7ff0d80ffa90 msgr2=0x7ff0d8101e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:37:13.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:13.959+0000 7ff0dea00640 1 -- 172.21.15.5:0/1504453401 shutdown_connections 2024-09-10T09:37:13.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:13.959+0000 7ff0dea00640 1 -- 172.21.15.5:0/1504453401 wait complete. 2024-09-10T09:37:14.307 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411408 got 77309411410 for osd.2 2024-09-10T09:37:14.307 DEBUG:teuthology.parallel:result is None 2024-09-10T09:37:14.877 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:37:14 smithi005 ceph-mon[30350]: pgmap v360: 129 pgs: 129 active+clean; 645 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s; 126 B/s, 0 keys/s, 1 objects/s recovering 2024-09-10T09:37:14.878 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:37:14 smithi005 ceph-mon[30350]: from='client.? 172.21.15.5:0/1504453401' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-10T09:37:14.880 INFO:tasks.cephadm.ceph_manager.ceph:need seq 158913790016 got 158913790019 for osd.5 2024-09-10T09:37:14.880 DEBUG:teuthology.parallel:result is None 2024-09-10T09:37:14.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:37:14 smithi102 ceph-mon[35442]: pgmap v360: 129 pgs: 129 active+clean; 645 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s; 126 B/s, 0 keys/s, 1 objects/s recovering 2024-09-10T09:37:14.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:37:14 smithi102 ceph-mon[35442]: from='client.? 172.21.15.5:0/1504453401' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-10T09:37:15.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:37:14 smithi159 ceph-mon[35513]: pgmap v360: 129 pgs: 129 active+clean; 645 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s; 126 B/s, 0 keys/s, 1 objects/s recovering 2024-09-10T09:37:15.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:37:14 smithi159 ceph-mon[35513]: from='client.? 172.21.15.5:0/1504453401' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-10T09:37:15.448 INFO:tasks.cephadm.ceph_manager.ceph:need seq 214748364851 got 214748364855 for osd.7 2024-09-10T09:37:15.448 DEBUG:teuthology.parallel:result is None 2024-09-10T09:37:15.845 INFO:tasks.cephadm.ceph_manager.ceph:need seq 193273528379 got 193273528381 for osd.6 2024-09-10T09:37:15.845 DEBUG:teuthology.parallel:result is None 2024-09-10T09:37:16.616 INFO:tasks.cephadm.ceph_manager.ceph:need seq 107374182475 got 107374182478 for osd.3 2024-09-10T09:37:16.616 DEBUG:teuthology.parallel:result is None 2024-09-10T09:37:16.616 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-10T09:37:16.616 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-10T09:37:16.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:37:16 smithi102 ceph-mon[35442]: pgmap v361: 129 pgs: 129 active+clean; 645 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 9 op/s; 112 B/s, 0 keys/s, 0 objects/s recovering 2024-09-10T09:37:16.998 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:37:17.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:37:16 smithi005 ceph-mon[30350]: pgmap v361: 129 pgs: 129 active+clean; 645 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 9 op/s; 112 B/s, 0 keys/s, 0 objects/s recovering 2024-09-10T09:37:17.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:37:16 smithi159 ceph-mon[35513]: pgmap v361: 129 pgs: 129 active+clean; 645 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 9 op/s; 112 B/s, 0 keys/s, 0 objects/s recovering 2024-09-10T09:37:17.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:17.561+0000 7f23f1600640 1 -- 172.21.15.5:0/1894014236 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f23ec102790 msgr2=0x7f23ec1067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:37:17.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:17.561+0000 7f23f1600640 1 --2- 172.21.15.5:0/1894014236 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f23ec102790 0x7f23ec1067e0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f23d4009900 tx=0x7f23d402a3f0 comp rx=0 tx=0).stop 2024-09-10T09:37:17.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:17.561+0000 7f23f1600640 1 -- 172.21.15.5:0/1894014236 shutdown_connections 2024-09-10T09:37:17.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:17.561+0000 7f23f1600640 1 --2- 172.21.15.5:0/1894014236 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f23ec106f10 0x7f23ec10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:17.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:17.561+0000 7f23f1600640 1 --2- 172.21.15.5:0/1894014236 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f23ec102790 0x7f23ec1067e0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:17.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:17.561+0000 7f23f1600640 1 --2- 172.21.15.5:0/1894014236 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f23ec101de0 0x7f23ec1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:17.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:17.561+0000 7f23f1600640 1 -- 172.21.15.5:0/1894014236 >> 172.21.15.5:0/1894014236 conn(0x7f23ec0fd8c0 msgr2=0x7f23ec0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:37:17.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:17.561+0000 7f23f1600640 1 -- 172.21.15.5:0/1894014236 shutdown_connections 2024-09-10T09:37:17.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:17.561+0000 7f23f1600640 1 -- 172.21.15.5:0/1894014236 wait complete. 2024-09-10T09:37:17.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:17.562+0000 7f23f1600640 1 Processor -- start 2024-09-10T09:37:17.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:17.562+0000 7f23f1600640 1 -- start start 2024-09-10T09:37:17.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:17.563+0000 7f23f1600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f23ec101de0 0x7f23ec19a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:37:17.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:17.563+0000 7f23f1600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f23ec102790 0x7f23ec19aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:37:17.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:17.563+0000 7f23f1600640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f23ec106f10 0x7f23ec19ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:37:17.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:17.563+0000 7f23f1600640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f23ec111fc0 con 0x7f23ec102790 2024-09-10T09:37:17.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:17.563+0000 7f23f1600640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f23ec111e40 con 0x7f23ec101de0 2024-09-10T09:37:17.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:17.563+0000 7f23f1600640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f23ec112140 con 0x7f23ec106f10 2024-09-10T09:37:17.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:17.563+0000 7f23eb400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f23ec106f10 0x7f23ec19ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:37:17.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:17.563+0000 7f23eb400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f23ec106f10 0x7f23ec19ef30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:49058/0 (socket says 172.21.15.5:49058) 2024-09-10T09:37:17.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:17.563+0000 7f23eb400640 1 -- 172.21.15.5:0/1124313038 learned_addr learned my addr 172.21.15.5:0/1124313038 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:37:17.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:17.563+0000 7f23e2000640 1 --2- 172.21.15.5:0/1124313038 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f23ec102790 0x7f23ec19aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:37:17.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:17.563+0000 7f23eaa00640 1 --2- 172.21.15.5:0/1124313038 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f23ec101de0 0x7f23ec19a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:37:17.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:17.564+0000 7f23eb400640 1 -- 172.21.15.5:0/1124313038 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f23ec101de0 msgr2=0x7f23ec19a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:37:17.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:17.564+0000 7f23eb400640 1 --2- 172.21.15.5:0/1124313038 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f23ec101de0 0x7f23ec19a660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:17.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:17.564+0000 7f23eb400640 1 -- 172.21.15.5:0/1124313038 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f23ec102790 msgr2=0x7f23ec19aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:37:17.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:17.564+0000 7f23eb400640 1 --2- 172.21.15.5:0/1124313038 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f23ec102790 0x7f23ec19aba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:17.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:17.564+0000 7f23eb400640 1 -- 172.21.15.5:0/1124313038 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23ec19f610 con 0x7f23ec106f10 2024-09-10T09:37:17.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:17.564+0000 7f23e2000640 1 --2- 172.21.15.5:0/1124313038 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f23ec102790 0x7f23ec19aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:37:17.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:17.564+0000 7f23eb400640 1 --2- 172.21.15.5:0/1124313038 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f23ec106f10 0x7f23ec19ef30 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f23e400eb10 tx=0x7f23e400c2d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:37:17.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:17.564+0000 7f23e3e00640 1 -- 172.21.15.5:0/1124313038 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23e4004b20 con 0x7f23ec106f10 2024-09-10T09:37:17.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:17.564+0000 7f23e3e00640 1 -- 172.21.15.5:0/1124313038 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f23e4004cc0 con 0x7f23ec106f10 2024-09-10T09:37:17.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:17.564+0000 7f23f1600640 1 -- 172.21.15.5:0/1124313038 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23ec19f900 con 0x7f23ec106f10 2024-09-10T09:37:17.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:17.564+0000 7f23e3e00640 1 -- 172.21.15.5:0/1124313038 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23e4010760 con 0x7f23ec106f10 2024-09-10T09:37:17.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:17.565+0000 7f23f1600640 1 -- 172.21.15.5:0/1124313038 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23ec1a71e0 con 0x7f23ec106f10 2024-09-10T09:37:17.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:17.566+0000 7f23f1600640 1 -- 172.21.15.5:0/1124313038 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f23ec1021c0 con 0x7f23ec106f10 2024-09-10T09:37:17.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:17.568+0000 7f23e3e00640 1 -- 172.21.15.5:0/1124313038 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f23e4005000 con 0x7f23ec106f10 2024-09-10T09:37:17.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:17.569+0000 7f23e3e00640 1 --2- 172.21.15.5:0/1124313038 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f23d00a2c10 0x7f23d00a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:37:17.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:17.569+0000 7f23e3e00640 1 -- 172.21.15.5:0/1124313038 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f23e40cfd30 con 0x7f23ec106f10 2024-09-10T09:37:17.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:17.569+0000 7f23eaa00640 1 --2- 172.21.15.5:0/1124313038 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f23d00a2c10 0x7f23d00a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:37:17.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:17.570+0000 7f23eaa00640 1 --2- 172.21.15.5:0/1124313038 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f23d00a2c10 0x7f23d00a50d0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f23dc009bd0 tx=0x7f23dc005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:37:17.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:17.571+0000 7f23e3e00640 1 -- 172.21.15.5:0/1124313038 <== mon.1 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f23e40541b0 con 0x7f23ec106f10 2024-09-10T09:37:17.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:17.755+0000 7f23f1600640 1 -- 172.21.15.5:0/1124313038 --> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f23ec10de60 con 0x7f23d00a2c10 2024-09-10T09:37:17.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:17.764+0000 7f23e3e00640 1 -- 172.21.15.5:0/1124313038 <== mgr.14150 v2:172.21.15.5:6800/2707348081 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340705 (secure 0 0 0) 0x7f23ec10de60 con 0x7f23d00a2c10 2024-09-10T09:37:17.768 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-10T09:37:17.770 INFO:teuthology.orchestra.run.smithi005.stderr:dumped all 2024-09-10T09:37:17.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:17.769+0000 7f23f1600640 1 -- 172.21.15.5:0/1124313038 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f23d00a2c10 msgr2=0x7f23d00a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:37:17.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:17.769+0000 7f23f1600640 1 --2- 172.21.15.5:0/1124313038 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f23d00a2c10 0x7f23d00a50d0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f23dc009bd0 tx=0x7f23dc005f90 comp rx=0 tx=0).stop 2024-09-10T09:37:17.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:17.769+0000 7f23f1600640 1 -- 172.21.15.5:0/1124313038 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f23ec106f10 msgr2=0x7f23ec19ef30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:37:17.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:17.769+0000 7f23f1600640 1 --2- 172.21.15.5:0/1124313038 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f23ec106f10 0x7f23ec19ef30 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f23e400eb10 tx=0x7f23e400c2d0 comp rx=0 tx=0).stop 2024-09-10T09:37:17.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:17.769+0000 7f23f1600640 1 -- 172.21.15.5:0/1124313038 shutdown_connections 2024-09-10T09:37:17.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:17.769+0000 7f23f1600640 1 --2- 172.21.15.5:0/1124313038 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f23d00a2c10 0x7f23d00a50d0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:17.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:17.769+0000 7f23f1600640 1 --2- 172.21.15.5:0/1124313038 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f23ec106f10 0x7f23ec19ef30 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:17.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:17.769+0000 7f23f1600640 1 --2- 172.21.15.5:0/1124313038 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f23ec102790 0x7f23ec19aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:17.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:17.769+0000 7f23f1600640 1 --2- 172.21.15.5:0/1124313038 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f23ec101de0 0x7f23ec19a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:17.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:17.769+0000 7f23f1600640 1 -- 172.21.15.5:0/1124313038 >> 172.21.15.5:0/1124313038 conn(0x7f23ec0fd8c0 msgr2=0x7f23ec0feb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:37:17.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:17.769+0000 7f23f1600640 1 -- 172.21.15.5:0/1124313038 shutdown_connections 2024-09-10T09:37:17.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:17.769+0000 7f23f1600640 1 -- 172.21.15.5:0/1124313038 wait complete. 2024-09-10T09:37:18.658 INFO:teuthology.orchestra.run.smithi005.stdout:{"pg_ready":true,"pg_map":{"version":362,"stamp":"2024-09-10T09:37:17.554892+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":660741,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":765,"num_read_kb":508,"num_write":432,"num_write_kb":3082,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":1181751,"num_keys_recovered":7,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":353,"ondisk_log_size":353,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":374880,"kb_used_data":13076,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124566944,"statfs":{"total":1151940427776,"available":1151556550656,"internally_reserved":0,"allocated":13389824,"data_stored":8602313,"data_compressed":498435,"data_compressed_allocated":675840,"data_compressed_original":2850816,"omap_allocated":19141,"internal_metadata":370193723},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":91,"num_read_kb":48,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":1015,"num_keys_recovered":7,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.032107"},"pg_stats":[{"pgid":"2.1f","version":"94'13","reported_seq":117,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:13.581313+0000","last_change":"2024-09-10T09:35:25.533085+0000","last_active":"2024-09-10T09:37:13.581313+0000","last_peered":"2024-09-10T09:37:13.581313+0000","last_clean":"2024-09-10T09:37:13.581313+0000","last_became_active":"2024-09-10T09:35:25.532729+0000","last_became_peered":"2024-09-10T09:35:25.532729+0000","last_unstale":"2024-09-10T09:37:13.581313+0000","last_undegraded":"2024-09-10T09:37:13.581313+0000","last_fullsized":"2024-09-10T09:37:13.581313+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:55:11.158279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":61,"num_read_kb":61,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":15,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.527458+0000","last_change":"2024-09-10T09:36:06.767356+0000","last_active":"2024-09-10T09:37:07.527458+0000","last_peered":"2024-09-10T09:37:07.527458+0000","last_clean":"2024-09-10T09:37:07.527458+0000","last_became_active":"2024-09-10T09:36:06.766931+0000","last_became_peered":"2024-09-10T09:36:06.766931+0000","last_unstale":"2024-09-10T09:37:07.527458+0000","last_undegraded":"2024-09-10T09:37:07.527458+0000","last_fullsized":"2024-09-10T09:37:07.527458+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:25:02.777327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,7],"acting":[9,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":28,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.527473+0000","last_change":"2024-09-10T09:35:36.629797+0000","last_active":"2024-09-10T09:37:07.527473+0000","last_peered":"2024-09-10T09:37:07.527473+0000","last_clean":"2024-09-10T09:37:07.527473+0000","last_became_active":"2024-09-10T09:35:36.629414+0000","last_became_peered":"2024-09-10T09:35:36.629414+0000","last_unstale":"2024-09-10T09:37:07.527473+0000","last_undegraded":"2024-09-10T09:37:07.527473+0000","last_fullsized":"2024-09-10T09:37:07.527473+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:42:44.610581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"94'2","reported_seq":46,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.524916+0000","last_change":"2024-09-10T09:35:25.852022+0000","last_active":"2024-09-10T09:37:07.524916+0000","last_peered":"2024-09-10T09:37:07.524916+0000","last_clean":"2024-09-10T09:37:07.524916+0000","last_became_active":"2024-09-10T09:35:25.851587+0000","last_became_peered":"2024-09-10T09:35:25.851587+0000","last_unstale":"2024-09-10T09:37:07.524916+0000","last_undegraded":"2024-09-10T09:37:07.524916+0000","last_fullsized":"2024-09-10T09:37:07.524916+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:37:26.085914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.761614+0000","last_change":"2024-09-10T09:35:28.579260+0000","last_active":"2024-09-10T09:37:07.761614+0000","last_peered":"2024-09-10T09:37:07.761614+0000","last_clean":"2024-09-10T09:37:07.761614+0000","last_became_active":"2024-09-10T09:35:28.574584+0000","last_became_peered":"2024-09-10T09:35:28.574584+0000","last_unstale":"2024-09-10T09:37:07.761614+0000","last_undegraded":"2024-09-10T09:37:07.761614+0000","last_fullsized":"2024-09-10T09:37:07.761614+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:31:19.277689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":35,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.568297+0000","last_change":"2024-09-10T09:36:06.874861+0000","last_active":"2024-09-10T09:37:07.568297+0000","last_peered":"2024-09-10T09:37:07.568297+0000","last_clean":"2024-09-10T09:37:07.568297+0000","last_became_active":"2024-09-10T09:36:06.873930+0000","last_became_peered":"2024-09-10T09:36:06.873930+0000","last_unstale":"2024-09-10T09:37:07.568297+0000","last_undegraded":"2024-09-10T09:37:07.568297+0000","last_fullsized":"2024-09-10T09:37:07.568297+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:35:01.354061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,8,3],"acting":[4,6,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"94'12","reported_seq":56,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.530293+0000","last_change":"2024-09-10T09:35:25.534571+0000","last_active":"2024-09-10T09:37:07.530293+0000","last_peered":"2024-09-10T09:37:07.530293+0000","last_clean":"2024-09-10T09:37:07.530293+0000","last_became_active":"2024-09-10T09:35:25.534235+0000","last_became_peered":"2024-09-10T09:35:25.534235+0000","last_unstale":"2024-09-10T09:37:07.530293+0000","last_undegraded":"2024-09-10T09:37:07.530293+0000","last_fullsized":"2024-09-10T09:37:07.530293+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:50:21.447542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.574647+0000","last_change":"2024-09-10T09:35:28.565541+0000","last_active":"2024-09-10T09:37:07.574647+0000","last_peered":"2024-09-10T09:37:07.574647+0000","last_clean":"2024-09-10T09:37:07.574647+0000","last_became_active":"2024-09-10T09:35:28.565095+0000","last_became_peered":"2024-09-10T09:35:28.565095+0000","last_unstale":"2024-09-10T09:37:07.574647+0000","last_undegraded":"2024-09-10T09:37:07.574647+0000","last_fullsized":"2024-09-10T09:37:07.574647+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:29:31.285401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":28,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.568540+0000","last_change":"2024-09-10T09:35:36.629277+0000","last_active":"2024-09-10T09:37:07.568540+0000","last_peered":"2024-09-10T09:37:07.568540+0000","last_clean":"2024-09-10T09:37:07.568540+0000","last_became_active":"2024-09-10T09:35:36.628687+0000","last_became_peered":"2024-09-10T09:35:36.628687+0000","last_unstale":"2024-09-10T09:37:07.568540+0000","last_undegraded":"2024-09-10T09:37:07.568540+0000","last_fullsized":"2024-09-10T09:37:07.568540+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:34:33.628827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"94'5","reported_seq":49,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.530106+0000","last_change":"2024-09-10T09:35:25.536137+0000","last_active":"2024-09-10T09:37:07.530106+0000","last_peered":"2024-09-10T09:37:07.530106+0000","last_clean":"2024-09-10T09:37:07.530106+0000","last_became_active":"2024-09-10T09:35:25.535728+0000","last_became_peered":"2024-09-10T09:35:25.535728+0000","last_unstale":"2024-09-10T09:37:07.530106+0000","last_undegraded":"2024-09-10T09:37:07.530106+0000","last_fullsized":"2024-09-10T09:37:07.530106+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:36:04.829901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":44,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.532371+0000","last_change":"2024-09-10T09:36:06.874425+0000","last_active":"2024-09-10T09:37:07.532371+0000","last_peered":"2024-09-10T09:37:07.532371+0000","last_clean":"2024-09-10T09:37:07.532371+0000","last_became_active":"2024-09-10T09:36:06.874050+0000","last_became_peered":"2024-09-10T09:36:06.874050+0000","last_unstale":"2024-09-10T09:37:07.532371+0000","last_undegraded":"2024-09-10T09:37:07.532371+0000","last_fullsized":"2024-09-10T09:37:07.532371+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:30:51.200335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,1],"acting":[5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":28,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.567999+0000","last_change":"2024-09-10T09:35:36.615314+0000","last_active":"2024-09-10T09:37:07.567999+0000","last_peered":"2024-09-10T09:37:07.567999+0000","last_clean":"2024-09-10T09:37:07.567999+0000","last_became_active":"2024-09-10T09:35:36.614820+0000","last_became_peered":"2024-09-10T09:35:36.614820+0000","last_unstale":"2024-09-10T09:37:07.567999+0000","last_undegraded":"2024-09-10T09:37:07.567999+0000","last_fullsized":"2024-09-10T09:37:07.567999+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:51:16.897935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.528727+0000","last_change":"2024-09-10T09:35:28.572346+0000","last_active":"2024-09-10T09:37:07.528727+0000","last_peered":"2024-09-10T09:37:07.528727+0000","last_clean":"2024-09-10T09:37:07.528727+0000","last_became_active":"2024-09-10T09:35:28.571839+0000","last_became_peered":"2024-09-10T09:35:28.571839+0000","last_unstale":"2024-09-10T09:37:07.528727+0000","last_undegraded":"2024-09-10T09:37:07.528727+0000","last_fullsized":"2024-09-10T09:37:07.528727+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:41:29.662441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"94'4","reported_seq":48,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.529863+0000","last_change":"2024-09-10T09:35:25.538747+0000","last_active":"2024-09-10T09:37:07.529863+0000","last_peered":"2024-09-10T09:37:07.529863+0000","last_clean":"2024-09-10T09:37:07.529863+0000","last_became_active":"2024-09-10T09:35:25.538330+0000","last_became_peered":"2024-09-10T09:35:25.538330+0000","last_unstale":"2024-09-10T09:37:07.529863+0000","last_undegraded":"2024-09-10T09:37:07.529863+0000","last_fullsized":"2024-09-10T09:37:07.529863+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:15:08.340688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":28,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.568378+0000","last_change":"2024-09-10T09:35:37.270106+0000","last_active":"2024-09-10T09:37:07.568378+0000","last_peered":"2024-09-10T09:37:07.568378+0000","last_clean":"2024-09-10T09:37:07.568378+0000","last_became_active":"2024-09-10T09:35:37.269575+0000","last_became_peered":"2024-09-10T09:35:37.269575+0000","last_unstale":"2024-09-10T09:37:07.568378+0000","last_undegraded":"2024-09-10T09:37:07.568378+0000","last_fullsized":"2024-09-10T09:37:07.568378+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:35:00.414602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.524435+0000","last_change":"2024-09-10T09:35:28.569230+0000","last_active":"2024-09-10T09:37:07.524435+0000","last_peered":"2024-09-10T09:37:07.524435+0000","last_clean":"2024-09-10T09:37:07.524435+0000","last_became_active":"2024-09-10T09:35:28.568266+0000","last_became_peered":"2024-09-10T09:35:28.568266+0000","last_unstale":"2024-09-10T09:37:07.524435+0000","last_undegraded":"2024-09-10T09:37:07.524435+0000","last_fullsized":"2024-09-10T09:37:07.524435+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:49:43.730686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"94'3","reported_seq":17,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.544707+0000","last_change":"2024-09-10T09:37:07.544298+0000","last_active":"2024-09-10T09:37:07.544707+0000","last_peered":"2024-09-10T09:37:07.544707+0000","last_clean":"2024-09-10T09:37:07.544707+0000","last_became_active":"2024-09-10T09:37:07.531815+0000","last_became_peered":"2024-09-10T09:37:07.531815+0000","last_unstale":"2024-09-10T09:37:07.544707+0000","last_undegraded":"2024-09-10T09:37:07.544707+0000","last_fullsized":"2024-09-10T09:37:07.544707+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:18:10.091895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"94'1","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.568418+0000","last_change":"2024-09-10T09:36:06.875056+0000","last_active":"2024-09-10T09:37:07.568418+0000","last_peered":"2024-09-10T09:37:07.568418+0000","last_clean":"2024-09-10T09:37:07.568418+0000","last_became_active":"2024-09-10T09:36:06.874400+0000","last_became_peered":"2024-09-10T09:36:06.874400+0000","last_unstale":"2024-09-10T09:37:07.568418+0000","last_undegraded":"2024-09-10T09:37:07.568418+0000","last_fullsized":"2024-09-10T09:37:07.568418+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:21:06.300712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,7,8],"acting":[4,10,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.526854+0000","last_change":"2024-09-10T09:35:28.571319+0000","last_active":"2024-09-10T09:37:07.526854+0000","last_peered":"2024-09-10T09:37:07.526854+0000","last_clean":"2024-09-10T09:37:07.526854+0000","last_became_active":"2024-09-10T09:35:28.570972+0000","last_became_peered":"2024-09-10T09:35:28.570972+0000","last_unstale":"2024-09-10T09:37:07.526854+0000","last_undegraded":"2024-09-10T09:37:07.526854+0000","last_fullsized":"2024-09-10T09:37:07.526854+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:45:19.300535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"94'7","reported_seq":51,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.524428+0000","last_change":"2024-09-10T09:35:25.530489+0000","last_active":"2024-09-10T09:37:07.524428+0000","last_peered":"2024-09-10T09:37:07.524428+0000","last_clean":"2024-09-10T09:37:07.524428+0000","last_became_active":"2024-09-10T09:35:25.530197+0000","last_became_peered":"2024-09-10T09:35:25.530197+0000","last_unstale":"2024-09-10T09:37:07.524428+0000","last_undegraded":"2024-09-10T09:37:07.524428+0000","last_fullsized":"2024-09-10T09:37:07.524428+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:28:57.539569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":28,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.527603+0000","last_change":"2024-09-10T09:35:36.630507+0000","last_active":"2024-09-10T09:37:07.527603+0000","last_peered":"2024-09-10T09:37:07.527603+0000","last_clean":"2024-09-10T09:37:07.527603+0000","last_became_active":"2024-09-10T09:35:36.630052+0000","last_became_peered":"2024-09-10T09:35:36.630052+0000","last_unstale":"2024-09-10T09:37:07.527603+0000","last_undegraded":"2024-09-10T09:37:07.527603+0000","last_fullsized":"2024-09-10T09:37:07.527603+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:13:42.691382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.529586+0000","last_change":"2024-09-10T09:35:28.565650+0000","last_active":"2024-09-10T09:37:07.529586+0000","last_peered":"2024-09-10T09:37:07.529586+0000","last_clean":"2024-09-10T09:37:07.529586+0000","last_became_active":"2024-09-10T09:35:28.565303+0000","last_became_peered":"2024-09-10T09:35:28.565303+0000","last_unstale":"2024-09-10T09:37:07.529586+0000","last_undegraded":"2024-09-10T09:37:07.529586+0000","last_fullsized":"2024-09-10T09:37:07.529586+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:48:08.965487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"94'2","reported_seq":45,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.574420+0000","last_change":"2024-09-10T09:35:25.851326+0000","last_active":"2024-09-10T09:37:07.574420+0000","last_peered":"2024-09-10T09:37:07.574420+0000","last_clean":"2024-09-10T09:37:07.574420+0000","last_became_active":"2024-09-10T09:35:25.850981+0000","last_became_peered":"2024-09-10T09:35:25.850981+0000","last_unstale":"2024-09-10T09:37:07.574420+0000","last_undegraded":"2024-09-10T09:37:07.574420+0000","last_fullsized":"2024-09-10T09:37:07.574420+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:50:28.877840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":28,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.568406+0000","last_change":"2024-09-10T09:35:37.270138+0000","last_active":"2024-09-10T09:37:07.568406+0000","last_peered":"2024-09-10T09:37:07.568406+0000","last_clean":"2024-09-10T09:37:07.568406+0000","last_became_active":"2024-09-10T09:35:37.269663+0000","last_became_peered":"2024-09-10T09:35:37.269663+0000","last_unstale":"2024-09-10T09:37:07.568406+0000","last_undegraded":"2024-09-10T09:37:07.568406+0000","last_fullsized":"2024-09-10T09:37:07.568406+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:18:00.105970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.761440+0000","last_change":"2024-09-10T09:35:28.576197+0000","last_active":"2024-09-10T09:37:07.761440+0000","last_peered":"2024-09-10T09:37:07.761440+0000","last_clean":"2024-09-10T09:37:07.761440+0000","last_became_active":"2024-09-10T09:35:28.575859+0000","last_became_peered":"2024-09-10T09:35:28.575859+0000","last_unstale":"2024-09-10T09:37:07.761440+0000","last_undegraded":"2024-09-10T09:37:07.761440+0000","last_fullsized":"2024-09-10T09:37:07.761440+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:09:39.458761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"94'7","reported_seq":50,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.568747+0000","last_change":"2024-09-10T09:35:25.990587+0000","last_active":"2024-09-10T09:37:07.568747+0000","last_peered":"2024-09-10T09:37:07.568747+0000","last_clean":"2024-09-10T09:37:07.568747+0000","last_became_active":"2024-09-10T09:35:25.990302+0000","last_became_peered":"2024-09-10T09:35:25.990302+0000","last_unstale":"2024-09-10T09:37:07.568747+0000","last_undegraded":"2024-09-10T09:37:07.568747+0000","last_fullsized":"2024-09-10T09:37:07.568747+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:18:41.244441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":28,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.527701+0000","last_change":"2024-09-10T09:35:36.612983+0000","last_active":"2024-09-10T09:37:07.527701+0000","last_peered":"2024-09-10T09:37:07.527701+0000","last_clean":"2024-09-10T09:37:07.527701+0000","last_became_active":"2024-09-10T09:35:36.612542+0000","last_became_peered":"2024-09-10T09:35:36.612542+0000","last_unstale":"2024-09-10T09:37:07.527701+0000","last_undegraded":"2024-09-10T09:37:07.527701+0000","last_fullsized":"2024-09-10T09:37:07.527701+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:50:16.298195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.524933+0000","last_change":"2024-09-10T09:35:28.564121+0000","last_active":"2024-09-10T09:37:07.524933+0000","last_peered":"2024-09-10T09:37:07.524933+0000","last_clean":"2024-09-10T09:37:07.524933+0000","last_became_active":"2024-09-10T09:35:28.562780+0000","last_became_peered":"2024-09-10T09:35:28.562780+0000","last_unstale":"2024-09-10T09:37:07.524933+0000","last_undegraded":"2024-09-10T09:37:07.524933+0000","last_fullsized":"2024-09-10T09:37:07.524933+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:53:15.690478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"94'15","reported_seq":74,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.541031+0000","last_change":"2024-09-10T09:37:07.541031+0000","last_active":"2024-09-10T09:37:07.541031+0000","last_peered":"2024-09-10T09:37:07.541031+0000","last_clean":"2024-09-10T09:37:07.541031+0000","last_became_active":"2024-09-10T09:37:07.529577+0000","last_became_peered":"2024-09-10T09:37:07.529577+0000","last_unstale":"2024-09-10T09:37:07.541031+0000","last_undegraded":"2024-09-10T09:37:07.541031+0000","last_fullsized":"2024-09-10T09:37:07.541031+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:14:13.440346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":7,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":28,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.524058+0000","last_change":"2024-09-10T09:35:37.270888+0000","last_active":"2024-09-10T09:37:07.524058+0000","last_peered":"2024-09-10T09:37:07.524058+0000","last_clean":"2024-09-10T09:37:07.524058+0000","last_became_active":"2024-09-10T09:35:37.270229+0000","last_became_peered":"2024-09-10T09:35:37.270229+0000","last_unstale":"2024-09-10T09:37:07.524058+0000","last_undegraded":"2024-09-10T09:37:07.524058+0000","last_fullsized":"2024-09-10T09:37:07.524058+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:03:54.223463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.538908+0000","last_change":"2024-09-10T09:35:28.572799+0000","last_active":"2024-09-10T09:37:07.538908+0000","last_peered":"2024-09-10T09:37:07.538908+0000","last_clean":"2024-09-10T09:37:07.538908+0000","last_became_active":"2024-09-10T09:35:28.572310+0000","last_became_peered":"2024-09-10T09:35:28.572310+0000","last_unstale":"2024-09-10T09:37:07.538908+0000","last_undegraded":"2024-09-10T09:37:07.538908+0000","last_fullsized":"2024-09-10T09:37:07.538908+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:33:51.446766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"94'4","reported_seq":48,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.530409+0000","last_change":"2024-09-10T09:35:25.851868+0000","last_active":"2024-09-10T09:37:07.530409+0000","last_peered":"2024-09-10T09:37:07.530409+0000","last_clean":"2024-09-10T09:37:07.530409+0000","last_became_active":"2024-09-10T09:35:25.851447+0000","last_became_peered":"2024-09-10T09:35:25.851447+0000","last_unstale":"2024-09-10T09:37:07.530409+0000","last_undegraded":"2024-09-10T09:37:07.530409+0000","last_fullsized":"2024-09-10T09:37:07.530409+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:15:57.706690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":28,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.526489+0000","last_change":"2024-09-10T09:35:36.619459+0000","last_active":"2024-09-10T09:37:07.526489+0000","last_peered":"2024-09-10T09:37:07.526489+0000","last_clean":"2024-09-10T09:37:07.526489+0000","last_became_active":"2024-09-10T09:35:36.619023+0000","last_became_peered":"2024-09-10T09:35:36.619023+0000","last_unstale":"2024-09-10T09:37:07.526489+0000","last_undegraded":"2024-09-10T09:37:07.526489+0000","last_fullsized":"2024-09-10T09:37:07.526489+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:07:32.999231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.525808+0000","last_change":"2024-09-10T09:35:28.575668+0000","last_active":"2024-09-10T09:37:07.525808+0000","last_peered":"2024-09-10T09:37:07.525808+0000","last_clean":"2024-09-10T09:37:07.525808+0000","last_became_active":"2024-09-10T09:35:28.575425+0000","last_became_peered":"2024-09-10T09:35:28.575425+0000","last_unstale":"2024-09-10T09:37:07.525808+0000","last_undegraded":"2024-09-10T09:37:07.525808+0000","last_fullsized":"2024-09-10T09:37:07.525808+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:13:27.120208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"94'1","reported_seq":44,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.527840+0000","last_change":"2024-09-10T09:35:25.854569+0000","last_active":"2024-09-10T09:37:07.527840+0000","last_peered":"2024-09-10T09:37:07.527840+0000","last_clean":"2024-09-10T09:37:07.527840+0000","last_became_active":"2024-09-10T09:35:25.854152+0000","last_became_peered":"2024-09-10T09:35:25.854152+0000","last_unstale":"2024-09-10T09:37:07.527840+0000","last_undegraded":"2024-09-10T09:37:07.527840+0000","last_fullsized":"2024-09-10T09:37:07.527840+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:27:15.584707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":28,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.529899+0000","last_change":"2024-09-10T09:35:37.270517+0000","last_active":"2024-09-10T09:37:07.529899+0000","last_peered":"2024-09-10T09:37:07.529899+0000","last_clean":"2024-09-10T09:37:07.529899+0000","last_became_active":"2024-09-10T09:35:37.269793+0000","last_became_peered":"2024-09-10T09:35:37.269793+0000","last_unstale":"2024-09-10T09:37:07.529899+0000","last_undegraded":"2024-09-10T09:37:07.529899+0000","last_fullsized":"2024-09-10T09:37:07.529899+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:28:58.272848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":44,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.762585+0000","last_change":"2024-09-10T09:37:07.762585+0000","last_active":"2024-09-10T09:37:07.762585+0000","last_peered":"2024-09-10T09:37:07.762585+0000","last_clean":"2024-09-10T09:37:07.762585+0000","last_became_active":"2024-09-10T09:37:07.762250+0000","last_became_peered":"2024-09-10T09:37:07.762250+0000","last_unstale":"2024-09-10T09:37:07.762585+0000","last_undegraded":"2024-09-10T09:37:07.762585+0000","last_fullsized":"2024-09-10T09:37:07.762585+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:38:01.751512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"94'6","reported_seq":110,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:13.581994+0000","last_change":"2024-09-10T09:35:25.531062+0000","last_active":"2024-09-10T09:37:13.581994+0000","last_peered":"2024-09-10T09:37:13.581994+0000","last_clean":"2024-09-10T09:37:13.581994+0000","last_became_active":"2024-09-10T09:35:25.530005+0000","last_became_peered":"2024-09-10T09:35:25.530005+0000","last_unstale":"2024-09-10T09:37:13.581994+0000","last_undegraded":"2024-09-10T09:37:13.581994+0000","last_fullsized":"2024-09-10T09:37:13.581994+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:42:35.330312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":981,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":62,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":28,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.538712+0000","last_change":"2024-09-10T09:35:36.623099+0000","last_active":"2024-09-10T09:37:07.538712+0000","last_peered":"2024-09-10T09:37:07.538712+0000","last_clean":"2024-09-10T09:37:07.538712+0000","last_became_active":"2024-09-10T09:35:36.622716+0000","last_became_peered":"2024-09-10T09:35:36.622716+0000","last_unstale":"2024-09-10T09:37:07.538712+0000","last_undegraded":"2024-09-10T09:37:07.538712+0000","last_fullsized":"2024-09-10T09:37:07.538712+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:10:38.495164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":44,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.530897+0000","last_change":"2024-09-10T09:36:06.769405+0000","last_active":"2024-09-10T09:37:07.530897+0000","last_peered":"2024-09-10T09:37:07.530897+0000","last_clean":"2024-09-10T09:37:07.530897+0000","last_became_active":"2024-09-10T09:36:06.769042+0000","last_became_peered":"2024-09-10T09:36:06.769042+0000","last_unstale":"2024-09-10T09:37:07.530897+0000","last_undegraded":"2024-09-10T09:37:07.530897+0000","last_fullsized":"2024-09-10T09:37:07.530897+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:01:49.340765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,8],"acting":[5,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"94'5","reported_seq":109,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:12.574777+0000","last_change":"2024-09-10T09:35:25.851430+0000","last_active":"2024-09-10T09:37:12.574777+0000","last_peered":"2024-09-10T09:37:12.574777+0000","last_clean":"2024-09-10T09:37:12.574777+0000","last_became_active":"2024-09-10T09:35:25.851067+0000","last_became_peered":"2024-09-10T09:35:25.851067+0000","last_unstale":"2024-09-10T09:37:12.574777+0000","last_undegraded":"2024-09-10T09:37:12.574777+0000","last_fullsized":"2024-09-10T09:37:12.574777+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:55:58.756207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":61,"num_read_kb":61,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":28,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.538595+0000","last_change":"2024-09-10T09:35:37.270361+0000","last_active":"2024-09-10T09:37:07.538595+0000","last_peered":"2024-09-10T09:37:07.538595+0000","last_clean":"2024-09-10T09:37:07.538595+0000","last_became_active":"2024-09-10T09:35:37.269632+0000","last_became_peered":"2024-09-10T09:35:37.269632+0000","last_unstale":"2024-09-10T09:37:07.538595+0000","last_undegraded":"2024-09-10T09:37:07.538595+0000","last_fullsized":"2024-09-10T09:37:07.538595+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:50:34.360497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.538361+0000","last_change":"2024-09-10T09:35:28.571064+0000","last_active":"2024-09-10T09:37:07.538361+0000","last_peered":"2024-09-10T09:37:07.538361+0000","last_clean":"2024-09-10T09:37:07.538361+0000","last_became_active":"2024-09-10T09:35:28.570691+0000","last_became_peered":"2024-09-10T09:35:28.570691+0000","last_unstale":"2024-09-10T09:37:07.538361+0000","last_undegraded":"2024-09-10T09:37:07.538361+0000","last_fullsized":"2024-09-10T09:37:07.538361+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:58:33.162501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"94'4","reported_seq":56,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.572247+0000","last_change":"2024-09-10T09:37:07.572247+0000","last_active":"2024-09-10T09:37:07.572247+0000","last_peered":"2024-09-10T09:37:07.572247+0000","last_clean":"2024-09-10T09:37:07.572247+0000","last_became_active":"2024-09-10T09:37:07.569410+0000","last_became_peered":"2024-09-10T09:37:07.569410+0000","last_unstale":"2024-09-10T09:37:07.572247+0000","last_undegraded":"2024-09-10T09:37:07.572247+0000","last_fullsized":"2024-09-10T09:37:07.572247+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:05:09.037084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,4],"acting":[7,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":28,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.532586+0000","last_change":"2024-09-10T09:35:37.270358+0000","last_active":"2024-09-10T09:37:07.532586+0000","last_peered":"2024-09-10T09:37:07.532586+0000","last_clean":"2024-09-10T09:37:07.532586+0000","last_became_active":"2024-09-10T09:35:37.269822+0000","last_became_peered":"2024-09-10T09:35:37.269822+0000","last_unstale":"2024-09-10T09:37:07.532586+0000","last_undegraded":"2024-09-10T09:37:07.532586+0000","last_fullsized":"2024-09-10T09:37:07.532586+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:19:48.932596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.524402+0000","last_change":"2024-09-10T09:35:28.576244+0000","last_active":"2024-09-10T09:37:07.524402+0000","last_peered":"2024-09-10T09:37:07.524402+0000","last_clean":"2024-09-10T09:37:07.524402+0000","last_became_active":"2024-09-10T09:35:28.571801+0000","last_became_peered":"2024-09-10T09:35:28.571801+0000","last_unstale":"2024-09-10T09:37:07.524402+0000","last_undegraded":"2024-09-10T09:37:07.524402+0000","last_fullsized":"2024-09-10T09:37:07.524402+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:11:21.742220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"94'6","reported_seq":51,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.568700+0000","last_change":"2024-09-10T09:35:25.537118+0000","last_active":"2024-09-10T09:37:07.568700+0000","last_peered":"2024-09-10T09:37:07.568700+0000","last_clean":"2024-09-10T09:37:07.568700+0000","last_became_active":"2024-09-10T09:35:25.536718+0000","last_became_peered":"2024-09-10T09:35:25.536718+0000","last_unstale":"2024-09-10T09:37:07.568700+0000","last_undegraded":"2024-09-10T09:37:07.568700+0000","last_fullsized":"2024-09-10T09:37:07.568700+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:16:32.837476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":361,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.528720+0000","last_change":"2024-09-10T09:34:07.350825+0000","last_active":"2024-09-10T09:37:07.528720+0000","last_peered":"2024-09-10T09:37:07.528720+0000","last_clean":"2024-09-10T09:37:07.528720+0000","last_became_active":"2024-09-10T09:34:07.350325+0000","last_became_peered":"2024-09-10T09:34:07.350325+0000","last_unstale":"2024-09-10T09:37:07.528720+0000","last_undegraded":"2024-09-10T09:37:07.528720+0000","last_fullsized":"2024-09-10T09:37:07.528720+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:30:26.408262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:30:26.408262+0000","last_clean_scrub_stamp":"2024-09-10T09:30:26.408262+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:58:04.920443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"94'1","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.529244+0000","last_change":"2024-09-10T09:35:36.632941+0000","last_active":"2024-09-10T09:37:07.529244+0000","last_peered":"2024-09-10T09:37:07.529244+0000","last_clean":"2024-09-10T09:37:07.529244+0000","last_became_active":"2024-09-10T09:35:36.632527+0000","last_became_peered":"2024-09-10T09:35:36.632527+0000","last_unstale":"2024-09-10T09:37:07.529244+0000","last_undegraded":"2024-09-10T09:37:07.529244+0000","last_fullsized":"2024-09-10T09:37:07.529244+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:33:21.733772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.529425+0000","last_change":"2024-09-10T09:35:28.572010+0000","last_active":"2024-09-10T09:37:07.529425+0000","last_peered":"2024-09-10T09:37:07.529425+0000","last_clean":"2024-09-10T09:37:07.529425+0000","last_became_active":"2024-09-10T09:35:28.571623+0000","last_became_peered":"2024-09-10T09:35:28.571623+0000","last_unstale":"2024-09-10T09:37:07.529425+0000","last_undegraded":"2024-09-10T09:37:07.529425+0000","last_fullsized":"2024-09-10T09:37:07.529425+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:00:49.328663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"94'5","reported_seq":112,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:14.577199+0000","last_change":"2024-09-10T09:35:25.990328+0000","last_active":"2024-09-10T09:37:14.577199+0000","last_peered":"2024-09-10T09:37:14.577199+0000","last_clean":"2024-09-10T09:37:14.577199+0000","last_became_active":"2024-09-10T09:35:25.989946+0000","last_became_peered":"2024-09-10T09:35:25.989946+0000","last_unstale":"2024-09-10T09:37:14.577199+0000","last_undegraded":"2024-09-10T09:37:14.577199+0000","last_fullsized":"2024-09-10T09:37:14.577199+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:51:16.069913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":64,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":28,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.574415+0000","last_change":"2024-09-10T09:35:36.619013+0000","last_active":"2024-09-10T09:37:07.574415+0000","last_peered":"2024-09-10T09:37:07.574415+0000","last_clean":"2024-09-10T09:37:07.574415+0000","last_became_active":"2024-09-10T09:35:36.618506+0000","last_became_peered":"2024-09-10T09:35:36.618506+0000","last_unstale":"2024-09-10T09:37:07.574415+0000","last_undegraded":"2024-09-10T09:37:07.574415+0000","last_fullsized":"2024-09-10T09:37:07.574415+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:57:53.580569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":42,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.529232+0000","last_change":"2024-09-10T09:36:06.768573+0000","last_active":"2024-09-10T09:37:07.529232+0000","last_peered":"2024-09-10T09:37:07.529232+0000","last_clean":"2024-09-10T09:37:07.529232+0000","last_became_active":"2024-09-10T09:36:06.768228+0000","last_became_peered":"2024-09-10T09:36:06.768228+0000","last_unstale":"2024-09-10T09:37:07.529232+0000","last_undegraded":"2024-09-10T09:37:07.529232+0000","last_fullsized":"2024-09-10T09:37:07.529232+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:57:38.310713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"94'3","reported_seq":47,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.532881+0000","last_change":"2024-09-10T09:35:25.528795+0000","last_active":"2024-09-10T09:37:07.532881+0000","last_peered":"2024-09-10T09:37:07.532881+0000","last_clean":"2024-09-10T09:37:07.532881+0000","last_became_active":"2024-09-10T09:35:25.528475+0000","last_became_peered":"2024-09-10T09:35:25.528475+0000","last_unstale":"2024-09-10T09:37:07.532881+0000","last_undegraded":"2024-09-10T09:37:07.532881+0000","last_fullsized":"2024-09-10T09:37:07.532881+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:16:06.999186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":28,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.761719+0000","last_change":"2024-09-10T09:35:36.633109+0000","last_active":"2024-09-10T09:37:07.761719+0000","last_peered":"2024-09-10T09:37:07.761719+0000","last_clean":"2024-09-10T09:37:07.761719+0000","last_became_active":"2024-09-10T09:35:36.632682+0000","last_became_peered":"2024-09-10T09:35:36.632682+0000","last_unstale":"2024-09-10T09:37:07.761719+0000","last_undegraded":"2024-09-10T09:37:07.761719+0000","last_fullsized":"2024-09-10T09:37:07.761719+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:51:14.372649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.529393+0000","last_change":"2024-09-10T09:35:28.562103+0000","last_active":"2024-09-10T09:37:07.529393+0000","last_peered":"2024-09-10T09:37:07.529393+0000","last_clean":"2024-09-10T09:37:07.529393+0000","last_became_active":"2024-09-10T09:35:28.561765+0000","last_became_peered":"2024-09-10T09:35:28.561765+0000","last_unstale":"2024-09-10T09:37:07.529393+0000","last_undegraded":"2024-09-10T09:37:07.529393+0000","last_fullsized":"2024-09-10T09:37:07.529393+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:35:35.400569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"94'5","reported_seq":48,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.539182+0000","last_change":"2024-09-10T09:35:25.538536+0000","last_active":"2024-09-10T09:37:07.539182+0000","last_peered":"2024-09-10T09:37:07.539182+0000","last_clean":"2024-09-10T09:37:07.539182+0000","last_became_active":"2024-09-10T09:35:25.537944+0000","last_became_peered":"2024-09-10T09:35:25.537944+0000","last_unstale":"2024-09-10T09:37:07.539182+0000","last_undegraded":"2024-09-10T09:37:07.539182+0000","last_fullsized":"2024-09-10T09:37:07.539182+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:01:19.385381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":16,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.529871+0000","last_change":"2024-09-10T09:36:06.769534+0000","last_active":"2024-09-10T09:37:07.529871+0000","last_peered":"2024-09-10T09:37:07.529871+0000","last_clean":"2024-09-10T09:37:07.529871+0000","last_became_active":"2024-09-10T09:36:06.769015+0000","last_became_peered":"2024-09-10T09:36:06.769015+0000","last_unstale":"2024-09-10T09:37:07.529871+0000","last_undegraded":"2024-09-10T09:37:07.529871+0000","last_fullsized":"2024-09-10T09:37:07.529871+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:19:26.369737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7,1],"acting":[8,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.526505+0000","last_change":"2024-09-10T09:35:28.566735+0000","last_active":"2024-09-10T09:37:07.526505+0000","last_peered":"2024-09-10T09:37:07.526505+0000","last_clean":"2024-09-10T09:37:07.526505+0000","last_became_active":"2024-09-10T09:35:28.566400+0000","last_became_peered":"2024-09-10T09:35:28.566400+0000","last_unstale":"2024-09-10T09:37:07.526505+0000","last_undegraded":"2024-09-10T09:37:07.526505+0000","last_fullsized":"2024-09-10T09:37:07.526505+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:18:53.337816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"94'2","reported_seq":46,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.530519+0000","last_change":"2024-09-10T09:35:25.535783+0000","last_active":"2024-09-10T09:37:07.530519+0000","last_peered":"2024-09-10T09:37:07.530519+0000","last_clean":"2024-09-10T09:37:07.530519+0000","last_became_active":"2024-09-10T09:35:25.535411+0000","last_became_peered":"2024-09-10T09:35:25.535411+0000","last_unstale":"2024-09-10T09:37:07.530519+0000","last_undegraded":"2024-09-10T09:37:07.530519+0000","last_fullsized":"2024-09-10T09:37:07.530519+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:48:09.239083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":28,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.531150+0000","last_change":"2024-09-10T09:35:36.619380+0000","last_active":"2024-09-10T09:37:07.531150+0000","last_peered":"2024-09-10T09:37:07.531150+0000","last_clean":"2024-09-10T09:37:07.531150+0000","last_became_active":"2024-09-10T09:35:36.618953+0000","last_became_peered":"2024-09-10T09:35:36.618953+0000","last_unstale":"2024-09-10T09:37:07.531150+0000","last_undegraded":"2024-09-10T09:37:07.531150+0000","last_fullsized":"2024-09-10T09:37:07.531150+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:40:25.181408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.531331+0000","last_change":"2024-09-10T09:35:28.571396+0000","last_active":"2024-09-10T09:37:07.531331+0000","last_peered":"2024-09-10T09:37:07.531331+0000","last_clean":"2024-09-10T09:37:07.531331+0000","last_became_active":"2024-09-10T09:35:28.571063+0000","last_became_peered":"2024-09-10T09:35:28.571063+0000","last_unstale":"2024-09-10T09:37:07.531331+0000","last_undegraded":"2024-09-10T09:37:07.531331+0000","last_fullsized":"2024-09-10T09:37:07.531331+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:38:17.965133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"94'17","reported_seq":61,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.530004+0000","last_change":"2024-09-10T09:35:25.851773+0000","last_active":"2024-09-10T09:37:07.530004+0000","last_peered":"2024-09-10T09:37:07.530004+0000","last_clean":"2024-09-10T09:37:07.530004+0000","last_became_active":"2024-09-10T09:35:25.851384+0000","last_became_peered":"2024-09-10T09:35:25.851384+0000","last_unstale":"2024-09-10T09:37:07.530004+0000","last_undegraded":"2024-09-10T09:37:07.530004+0000","last_fullsized":"2024-09-10T09:37:07.530004+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:03:32.338643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":28,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.539161+0000","last_change":"2024-09-10T09:35:36.620971+0000","last_active":"2024-09-10T09:37:07.539161+0000","last_peered":"2024-09-10T09:37:07.539161+0000","last_clean":"2024-09-10T09:37:07.539161+0000","last_became_active":"2024-09-10T09:35:36.620343+0000","last_became_peered":"2024-09-10T09:35:36.620343+0000","last_unstale":"2024-09-10T09:37:07.539161+0000","last_undegraded":"2024-09-10T09:37:07.539161+0000","last_fullsized":"2024-09-10T09:37:07.539161+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:17:19.517882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.528475+0000","last_change":"2024-09-10T09:35:28.565643+0000","last_active":"2024-09-10T09:37:07.528475+0000","last_peered":"2024-09-10T09:37:07.528475+0000","last_clean":"2024-09-10T09:37:07.528475+0000","last_became_active":"2024-09-10T09:35:28.564217+0000","last_became_peered":"2024-09-10T09:35:28.564217+0000","last_unstale":"2024-09-10T09:37:07.528475+0000","last_undegraded":"2024-09-10T09:37:07.528475+0000","last_fullsized":"2024-09-10T09:37:07.528475+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:53:34.562690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"94'5","reported_seq":109,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:12.575289+0000","last_change":"2024-09-10T09:35:25.534167+0000","last_active":"2024-09-10T09:37:12.575289+0000","last_peered":"2024-09-10T09:37:12.575289+0000","last_clean":"2024-09-10T09:37:12.575289+0000","last_became_active":"2024-09-10T09:35:25.533719+0000","last_became_peered":"2024-09-10T09:35:25.533719+0000","last_unstale":"2024-09-10T09:37:12.575289+0000","last_undegraded":"2024-09-10T09:37:12.575289+0000","last_fullsized":"2024-09-10T09:37:12.575289+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:00:03.537355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":63,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":28,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.527665+0000","last_change":"2024-09-10T09:35:36.628716+0000","last_active":"2024-09-10T09:37:07.527665+0000","last_peered":"2024-09-10T09:37:07.527665+0000","last_clean":"2024-09-10T09:37:07.527665+0000","last_became_active":"2024-09-10T09:35:36.628085+0000","last_became_peered":"2024-09-10T09:35:36.628085+0000","last_unstale":"2024-09-10T09:37:07.527665+0000","last_undegraded":"2024-09-10T09:37:07.527665+0000","last_fullsized":"2024-09-10T09:37:07.527665+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:45:01.279403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.528994+0000","last_change":"2024-09-10T09:35:28.566701+0000","last_active":"2024-09-10T09:37:07.528994+0000","last_peered":"2024-09-10T09:37:07.528994+0000","last_clean":"2024-09-10T09:37:07.528994+0000","last_became_active":"2024-09-10T09:35:28.566391+0000","last_became_peered":"2024-09-10T09:35:28.566391+0000","last_unstale":"2024-09-10T09:37:07.528994+0000","last_undegraded":"2024-09-10T09:37:07.528994+0000","last_fullsized":"2024-09-10T09:37:07.528994+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:57:59.918706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"94'5","reported_seq":49,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.531844+0000","last_change":"2024-09-10T09:35:25.532510+0000","last_active":"2024-09-10T09:37:07.531844+0000","last_peered":"2024-09-10T09:37:07.531844+0000","last_clean":"2024-09-10T09:37:07.531844+0000","last_became_active":"2024-09-10T09:35:25.532193+0000","last_became_peered":"2024-09-10T09:35:25.532193+0000","last_unstale":"2024-09-10T09:37:07.531844+0000","last_undegraded":"2024-09-10T09:37:07.531844+0000","last_fullsized":"2024-09-10T09:37:07.531844+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:17:24.685178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":28,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.526361+0000","last_change":"2024-09-10T09:35:36.630742+0000","last_active":"2024-09-10T09:37:07.526361+0000","last_peered":"2024-09-10T09:37:07.526361+0000","last_clean":"2024-09-10T09:37:07.526361+0000","last_became_active":"2024-09-10T09:35:36.630335+0000","last_became_peered":"2024-09-10T09:35:36.630335+0000","last_unstale":"2024-09-10T09:37:07.526361+0000","last_undegraded":"2024-09-10T09:37:07.526361+0000","last_fullsized":"2024-09-10T09:37:07.526361+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:29:12.029390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.538397+0000","last_change":"2024-09-10T09:35:28.581817+0000","last_active":"2024-09-10T09:37:07.538397+0000","last_peered":"2024-09-10T09:37:07.538397+0000","last_clean":"2024-09-10T09:37:07.538397+0000","last_became_active":"2024-09-10T09:35:28.580316+0000","last_became_peered":"2024-09-10T09:35:28.580316+0000","last_unstale":"2024-09-10T09:37:07.538397+0000","last_undegraded":"2024-09-10T09:37:07.538397+0000","last_fullsized":"2024-09-10T09:37:07.538397+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:04:08.249564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"94'9","reported_seq":54,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.528083+0000","last_change":"2024-09-10T09:35:25.537474+0000","last_active":"2024-09-10T09:37:07.528083+0000","last_peered":"2024-09-10T09:37:07.528083+0000","last_clean":"2024-09-10T09:37:07.528083+0000","last_became_active":"2024-09-10T09:35:25.537096+0000","last_became_peered":"2024-09-10T09:35:25.537096+0000","last_unstale":"2024-09-10T09:37:07.528083+0000","last_undegraded":"2024-09-10T09:37:07.528083+0000","last_fullsized":"2024-09-10T09:37:07.528083+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:13:06.008458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":28,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.532263+0000","last_change":"2024-09-10T09:35:36.622788+0000","last_active":"2024-09-10T09:37:07.532263+0000","last_peered":"2024-09-10T09:37:07.532263+0000","last_clean":"2024-09-10T09:37:07.532263+0000","last_became_active":"2024-09-10T09:35:36.622377+0000","last_became_peered":"2024-09-10T09:35:36.622377+0000","last_unstale":"2024-09-10T09:37:07.532263+0000","last_undegraded":"2024-09-10T09:37:07.532263+0000","last_fullsized":"2024-09-10T09:37:07.532263+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:29:56.680864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.530171+0000","last_change":"2024-09-10T09:35:28.564529+0000","last_active":"2024-09-10T09:37:07.530171+0000","last_peered":"2024-09-10T09:37:07.530171+0000","last_clean":"2024-09-10T09:37:07.530171+0000","last_became_active":"2024-09-10T09:35:28.564223+0000","last_became_peered":"2024-09-10T09:35:28.564223+0000","last_unstale":"2024-09-10T09:37:07.530171+0000","last_undegraded":"2024-09-10T09:37:07.530171+0000","last_fullsized":"2024-09-10T09:37:07.530171+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:21:24.296578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"94'35","reported_seq":207,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:14.577526+0000","last_change":"2024-09-10T09:35:25.534331+0000","last_active":"2024-09-10T09:37:14.577526+0000","last_peered":"2024-09-10T09:37:14.577526+0000","last_clean":"2024-09-10T09:37:14.577526+0000","last_became_active":"2024-09-10T09:35:25.533985+0000","last_became_peered":"2024-09-10T09:35:25.533985+0000","last_unstale":"2024-09-10T09:37:14.577526+0000","last_undegraded":"2024-09-10T09:37:14.577526+0000","last_fullsized":"2024-09-10T09:37:14.577526+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:14:37.050650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62931,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":129,"num_read_kb":126,"num_write":37,"num_write_kb":76,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":28,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.527877+0000","last_change":"2024-09-10T09:35:36.632835+0000","last_active":"2024-09-10T09:37:07.527877+0000","last_peered":"2024-09-10T09:37:07.527877+0000","last_clean":"2024-09-10T09:37:07.527877+0000","last_became_active":"2024-09-10T09:35:36.630962+0000","last_became_peered":"2024-09-10T09:35:36.630962+0000","last_unstale":"2024-09-10T09:37:07.527877+0000","last_undegraded":"2024-09-10T09:37:07.527877+0000","last_fullsized":"2024-09-10T09:37:07.527877+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:59:27.613721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.574757+0000","last_change":"2024-09-10T09:35:28.567292+0000","last_active":"2024-09-10T09:37:07.574757+0000","last_peered":"2024-09-10T09:37:07.574757+0000","last_clean":"2024-09-10T09:37:07.574757+0000","last_became_active":"2024-09-10T09:35:28.566973+0000","last_became_peered":"2024-09-10T09:35:28.566973+0000","last_unstale":"2024-09-10T09:37:07.574757+0000","last_undegraded":"2024-09-10T09:37:07.574757+0000","last_fullsized":"2024-09-10T09:37:07.574757+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:05:37.251933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"94'5","reported_seq":49,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.532712+0000","last_change":"2024-09-10T09:35:25.852171+0000","last_active":"2024-09-10T09:37:07.532712+0000","last_peered":"2024-09-10T09:37:07.532712+0000","last_clean":"2024-09-10T09:37:07.532712+0000","last_became_active":"2024-09-10T09:35:25.851580+0000","last_became_peered":"2024-09-10T09:35:25.851580+0000","last_unstale":"2024-09-10T09:37:07.532712+0000","last_undegraded":"2024-09-10T09:37:07.532712+0000","last_fullsized":"2024-09-10T09:37:07.532712+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:03.263152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":28,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.568288+0000","last_change":"2024-09-10T09:35:36.629212+0000","last_active":"2024-09-10T09:37:07.568288+0000","last_peered":"2024-09-10T09:37:07.568288+0000","last_clean":"2024-09-10T09:37:07.568288+0000","last_became_active":"2024-09-10T09:35:36.628753+0000","last_became_peered":"2024-09-10T09:35:36.628753+0000","last_unstale":"2024-09-10T09:37:07.568288+0000","last_undegraded":"2024-09-10T09:37:07.568288+0000","last_fullsized":"2024-09-10T09:37:07.568288+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:54:45.311361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.761606+0000","last_change":"2024-09-10T09:35:28.572497+0000","last_active":"2024-09-10T09:37:07.761606+0000","last_peered":"2024-09-10T09:37:07.761606+0000","last_clean":"2024-09-10T09:37:07.761606+0000","last_became_active":"2024-09-10T09:35:28.569238+0000","last_became_peered":"2024-09-10T09:35:28.569238+0000","last_unstale":"2024-09-10T09:37:07.761606+0000","last_undegraded":"2024-09-10T09:37:07.761606+0000","last_fullsized":"2024-09-10T09:37:07.761606+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:38:47.189209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"94'4","reported_seq":46,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.529513+0000","last_change":"2024-09-10T09:35:25.536369+0000","last_active":"2024-09-10T09:37:07.529513+0000","last_peered":"2024-09-10T09:37:07.529513+0000","last_clean":"2024-09-10T09:37:07.529513+0000","last_became_active":"2024-09-10T09:35:25.535607+0000","last_became_peered":"2024-09-10T09:35:25.535607+0000","last_unstale":"2024-09-10T09:37:07.529513+0000","last_undegraded":"2024-09-10T09:37:07.529513+0000","last_fullsized":"2024-09-10T09:37:07.529513+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:49:18.621934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":28,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.526991+0000","last_change":"2024-09-10T09:35:36.627977+0000","last_active":"2024-09-10T09:37:07.526991+0000","last_peered":"2024-09-10T09:37:07.526991+0000","last_clean":"2024-09-10T09:37:07.526991+0000","last_became_active":"2024-09-10T09:35:36.626942+0000","last_became_peered":"2024-09-10T09:35:36.626942+0000","last_unstale":"2024-09-10T09:37:07.526991+0000","last_undegraded":"2024-09-10T09:37:07.526991+0000","last_fullsized":"2024-09-10T09:37:07.526991+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:37:18.209424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.575017+0000","last_change":"2024-09-10T09:35:28.562670+0000","last_active":"2024-09-10T09:37:07.575017+0000","last_peered":"2024-09-10T09:37:07.575017+0000","last_clean":"2024-09-10T09:37:07.575017+0000","last_became_active":"2024-09-10T09:35:28.562028+0000","last_became_peered":"2024-09-10T09:35:28.562028+0000","last_unstale":"2024-09-10T09:37:07.575017+0000","last_undegraded":"2024-09-10T09:37:07.575017+0000","last_fullsized":"2024-09-10T09:37:07.575017+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:37:15.176050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"94'4","reported_seq":168,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:14.576692+0000","last_change":"2024-09-10T09:35:25.529861+0000","last_active":"2024-09-10T09:37:14.576692+0000","last_peered":"2024-09-10T09:37:14.576692+0000","last_clean":"2024-09-10T09:37:14.576692+0000","last_became_active":"2024-09-10T09:35:25.529475+0000","last_became_peered":"2024-09-10T09:35:25.529475+0000","last_unstale":"2024-09-10T09:37:14.576692+0000","last_undegraded":"2024-09-10T09:37:14.576692+0000","last_fullsized":"2024-09-10T09:37:14.576692+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:44:27.086509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":121,"num_read_kb":121,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":28,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.524255+0000","last_change":"2024-09-10T09:35:36.615358+0000","last_active":"2024-09-10T09:37:07.524255+0000","last_peered":"2024-09-10T09:37:07.524255+0000","last_clean":"2024-09-10T09:37:07.524255+0000","last_became_active":"2024-09-10T09:35:36.614901+0000","last_became_peered":"2024-09-10T09:35:36.614901+0000","last_unstale":"2024-09-10T09:37:07.524255+0000","last_undegraded":"2024-09-10T09:37:07.524255+0000","last_fullsized":"2024-09-10T09:37:07.524255+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:56:54.248266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":28,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.761229+0000","last_change":"2024-09-10T09:35:36.629021+0000","last_active":"2024-09-10T09:37:07.761229+0000","last_peered":"2024-09-10T09:37:07.761229+0000","last_clean":"2024-09-10T09:37:07.761229+0000","last_became_active":"2024-09-10T09:35:36.628159+0000","last_became_peered":"2024-09-10T09:35:36.628159+0000","last_unstale":"2024-09-10T09:37:07.761229+0000","last_undegraded":"2024-09-10T09:37:07.761229+0000","last_fullsized":"2024-09-10T09:37:07.761229+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:50:48.615236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.527161+0000","last_change":"2024-09-10T09:35:28.579333+0000","last_active":"2024-09-10T09:37:07.527161+0000","last_peered":"2024-09-10T09:37:07.527161+0000","last_clean":"2024-09-10T09:37:07.527161+0000","last_became_active":"2024-09-10T09:35:28.578440+0000","last_became_peered":"2024-09-10T09:35:28.578440+0000","last_unstale":"2024-09-10T09:37:07.527161+0000","last_undegraded":"2024-09-10T09:37:07.527161+0000","last_fullsized":"2024-09-10T09:37:07.527161+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:07:28.701938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"94'3","reported_seq":47,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.531014+0000","last_change":"2024-09-10T09:35:25.530857+0000","last_active":"2024-09-10T09:37:07.531014+0000","last_peered":"2024-09-10T09:37:07.531014+0000","last_clean":"2024-09-10T09:37:07.531014+0000","last_became_active":"2024-09-10T09:35:25.530492+0000","last_became_peered":"2024-09-10T09:35:25.530492+0000","last_unstale":"2024-09-10T09:37:07.531014+0000","last_undegraded":"2024-09-10T09:37:07.531014+0000","last_fullsized":"2024-09-10T09:37:07.531014+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:07:21.726598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":35,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.526386+0000","last_change":"2024-09-10T09:36:06.770408+0000","last_active":"2024-09-10T09:37:07.526386+0000","last_peered":"2024-09-10T09:37:07.526386+0000","last_clean":"2024-09-10T09:37:07.526386+0000","last_became_active":"2024-09-10T09:36:06.769915+0000","last_became_peered":"2024-09-10T09:36:06.769915+0000","last_unstale":"2024-09-10T09:37:07.526386+0000","last_undegraded":"2024-09-10T09:37:07.526386+0000","last_fullsized":"2024-09-10T09:37:07.526386+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:56:09.795537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8,0],"acting":[11,4,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.524227+0000","last_change":"2024-09-10T09:35:28.570804+0000","last_active":"2024-09-10T09:37:07.524227+0000","last_peered":"2024-09-10T09:37:07.524227+0000","last_clean":"2024-09-10T09:37:07.524227+0000","last_became_active":"2024-09-10T09:35:28.570490+0000","last_became_peered":"2024-09-10T09:35:28.570490+0000","last_unstale":"2024-09-10T09:37:07.524227+0000","last_undegraded":"2024-09-10T09:37:07.524227+0000","last_fullsized":"2024-09-10T09:37:07.524227+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:13:38.999625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"94'3","reported_seq":46,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.538391+0000","last_change":"2024-09-10T09:35:25.536347+0000","last_active":"2024-09-10T09:37:07.538391+0000","last_peered":"2024-09-10T09:37:07.538391+0000","last_clean":"2024-09-10T09:37:07.538391+0000","last_became_active":"2024-09-10T09:35:25.535964+0000","last_became_peered":"2024-09-10T09:35:25.535964+0000","last_unstale":"2024-09-10T09:37:07.538391+0000","last_undegraded":"2024-09-10T09:37:07.538391+0000","last_fullsized":"2024-09-10T09:37:07.538391+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:36:49.035666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"94'7","reported_seq":50,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.525762+0000","last_change":"2024-09-10T09:35:25.532148+0000","last_active":"2024-09-10T09:37:07.525762+0000","last_peered":"2024-09-10T09:37:07.525762+0000","last_clean":"2024-09-10T09:37:07.525762+0000","last_became_active":"2024-09-10T09:35:25.531781+0000","last_became_peered":"2024-09-10T09:35:25.531781+0000","last_unstale":"2024-09-10T09:37:07.525762+0000","last_undegraded":"2024-09-10T09:37:07.525762+0000","last_fullsized":"2024-09-10T09:37:07.525762+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:52:46.740342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.532588+0000","last_change":"2024-09-10T09:35:28.578336+0000","last_active":"2024-09-10T09:37:07.532588+0000","last_peered":"2024-09-10T09:37:07.532588+0000","last_clean":"2024-09-10T09:37:07.532588+0000","last_became_active":"2024-09-10T09:35:28.577899+0000","last_became_peered":"2024-09-10T09:35:28.577899+0000","last_unstale":"2024-09-10T09:37:07.532588+0000","last_undegraded":"2024-09-10T09:37:07.532588+0000","last_fullsized":"2024-09-10T09:37:07.532588+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:13:04.478195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":28,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.528073+0000","last_change":"2024-09-10T09:35:36.630918+0000","last_active":"2024-09-10T09:37:07.528073+0000","last_peered":"2024-09-10T09:37:07.528073+0000","last_clean":"2024-09-10T09:37:07.528073+0000","last_became_active":"2024-09-10T09:35:36.630308+0000","last_became_peered":"2024-09-10T09:35:36.630308+0000","last_unstale":"2024-09-10T09:37:07.528073+0000","last_undegraded":"2024-09-10T09:37:07.528073+0000","last_fullsized":"2024-09-10T09:37:07.528073+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:25:44.954970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"94'6","reported_seq":124,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:13.577265+0000","last_change":"2024-09-10T09:37:07.574386+0000","last_active":"2024-09-10T09:37:13.577265+0000","last_peered":"2024-09-10T09:37:13.577265+0000","last_clean":"2024-09-10T09:37:13.577265+0000","last_became_active":"2024-09-10T09:37:07.569370+0000","last_became_peered":"2024-09-10T09:37:07.569370+0000","last_unstale":"2024-09-10T09:37:13.577265+0000","last_undegraded":"2024-09-10T09:37:13.577265+0000","last_fullsized":"2024-09-10T09:37:13.577265+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:07:23.014789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1015,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":58,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":1015,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.530083+0000","last_change":"2024-09-10T09:35:28.567915+0000","last_active":"2024-09-10T09:37:07.530083+0000","last_peered":"2024-09-10T09:37:07.530083+0000","last_clean":"2024-09-10T09:37:07.530083+0000","last_became_active":"2024-09-10T09:35:28.567416+0000","last_became_peered":"2024-09-10T09:35:28.567416+0000","last_unstale":"2024-09-10T09:37:07.530083+0000","last_undegraded":"2024-09-10T09:37:07.530083+0000","last_fullsized":"2024-09-10T09:37:07.530083+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:07:25.704080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":28,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.528636+0000","last_change":"2024-09-10T09:35:36.615386+0000","last_active":"2024-09-10T09:37:07.528636+0000","last_peered":"2024-09-10T09:37:07.528636+0000","last_clean":"2024-09-10T09:37:07.528636+0000","last_became_active":"2024-09-10T09:35:36.614967+0000","last_became_peered":"2024-09-10T09:35:36.614967+0000","last_unstale":"2024-09-10T09:37:07.528636+0000","last_undegraded":"2024-09-10T09:37:07.528636+0000","last_fullsized":"2024-09-10T09:37:07.528636+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:47.620729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.526231+0000","last_change":"2024-09-10T09:35:28.564411+0000","last_active":"2024-09-10T09:37:07.526231+0000","last_peered":"2024-09-10T09:37:07.526231+0000","last_clean":"2024-09-10T09:37:07.526231+0000","last_became_active":"2024-09-10T09:35:28.564034+0000","last_became_peered":"2024-09-10T09:35:28.564034+0000","last_unstale":"2024-09-10T09:37:07.526231+0000","last_undegraded":"2024-09-10T09:37:07.526231+0000","last_fullsized":"2024-09-10T09:37:07.526231+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:38:17.687991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.527821+0000","last_change":"2024-09-10T09:35:28.564619+0000","last_active":"2024-09-10T09:37:07.527821+0000","last_peered":"2024-09-10T09:37:07.527821+0000","last_clean":"2024-09-10T09:37:07.527821+0000","last_became_active":"2024-09-10T09:35:28.563890+0000","last_became_peered":"2024-09-10T09:35:28.563890+0000","last_unstale":"2024-09-10T09:37:07.527821+0000","last_undegraded":"2024-09-10T09:37:07.527821+0000","last_fullsized":"2024-09-10T09:37:07.527821+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:51:42.481911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.527812+0000","last_change":"2024-09-10T09:35:28.558909+0000","last_active":"2024-09-10T09:37:07.527812+0000","last_peered":"2024-09-10T09:37:07.527812+0000","last_clean":"2024-09-10T09:37:07.527812+0000","last_became_active":"2024-09-10T09:35:28.558604+0000","last_became_peered":"2024-09-10T09:35:28.558604+0000","last_unstale":"2024-09-10T09:37:07.527812+0000","last_undegraded":"2024-09-10T09:37:07.527812+0000","last_fullsized":"2024-09-10T09:37:07.527812+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:11.928446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.761769+0000","last_change":"2024-09-10T09:35:28.574459+0000","last_active":"2024-09-10T09:37:07.761769+0000","last_peered":"2024-09-10T09:37:07.761769+0000","last_clean":"2024-09-10T09:37:07.761769+0000","last_became_active":"2024-09-10T09:35:28.574154+0000","last_became_peered":"2024-09-10T09:35:28.574154+0000","last_unstale":"2024-09-10T09:37:07.761769+0000","last_undegraded":"2024-09-10T09:37:07.761769+0000","last_fullsized":"2024-09-10T09:37:07.761769+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:59:09.650536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.538880+0000","last_change":"2024-09-10T09:35:28.576999+0000","last_active":"2024-09-10T09:37:07.538880+0000","last_peered":"2024-09-10T09:37:07.538880+0000","last_clean":"2024-09-10T09:37:07.538880+0000","last_became_active":"2024-09-10T09:35:28.575616+0000","last_became_peered":"2024-09-10T09:35:28.575616+0000","last_unstale":"2024-09-10T09:37:07.538880+0000","last_undegraded":"2024-09-10T09:37:07.538880+0000","last_fullsized":"2024-09-10T09:37:07.538880+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:18:08.771760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.527051+0000","last_change":"2024-09-10T09:35:28.564846+0000","last_active":"2024-09-10T09:37:07.527051+0000","last_peered":"2024-09-10T09:37:07.527051+0000","last_clean":"2024-09-10T09:37:07.527051+0000","last_became_active":"2024-09-10T09:35:28.564498+0000","last_became_peered":"2024-09-10T09:35:28.564498+0000","last_unstale":"2024-09-10T09:37:07.527051+0000","last_undegraded":"2024-09-10T09:37:07.527051+0000","last_fullsized":"2024-09-10T09:37:07.527051+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:33:43.041730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.528287+0000","last_change":"2024-09-10T09:35:28.558093+0000","last_active":"2024-09-10T09:37:07.528287+0000","last_peered":"2024-09-10T09:37:07.528287+0000","last_clean":"2024-09-10T09:37:07.528287+0000","last_became_active":"2024-09-10T09:35:28.557266+0000","last_became_peered":"2024-09-10T09:35:28.557266+0000","last_unstale":"2024-09-10T09:37:07.528287+0000","last_undegraded":"2024-09-10T09:37:07.528287+0000","last_fullsized":"2024-09-10T09:37:07.528287+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:51:30.946065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":44,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.524860+0000","last_change":"2024-09-10T09:36:06.770298+0000","last_active":"2024-09-10T09:37:07.524860+0000","last_peered":"2024-09-10T09:37:07.524860+0000","last_clean":"2024-09-10T09:37:07.524860+0000","last_became_active":"2024-09-10T09:36:06.769848+0000","last_became_peered":"2024-09-10T09:36:06.769848+0000","last_unstale":"2024-09-10T09:37:07.524860+0000","last_undegraded":"2024-09-10T09:37:07.524860+0000","last_fullsized":"2024-09-10T09:37:07.524860+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:00:30.470887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,8,2],"acting":[3,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.530788+0000","last_change":"2024-09-10T09:35:28.578158+0000","last_active":"2024-09-10T09:37:07.530788+0000","last_peered":"2024-09-10T09:37:07.530788+0000","last_clean":"2024-09-10T09:37:07.530788+0000","last_became_active":"2024-09-10T09:35:28.577824+0000","last_became_peered":"2024-09-10T09:35:28.577824+0000","last_unstale":"2024-09-10T09:37:07.530788+0000","last_undegraded":"2024-09-10T09:37:07.530788+0000","last_fullsized":"2024-09-10T09:37:07.530788+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:40:27.231164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.524324+0000","last_change":"2024-09-10T09:35:28.562670+0000","last_active":"2024-09-10T09:37:07.524324+0000","last_peered":"2024-09-10T09:37:07.524324+0000","last_clean":"2024-09-10T09:37:07.524324+0000","last_became_active":"2024-09-10T09:35:28.562311+0000","last_became_peered":"2024-09-10T09:35:28.562311+0000","last_unstale":"2024-09-10T09:37:07.524324+0000","last_undegraded":"2024-09-10T09:37:07.524324+0000","last_fullsized":"2024-09-10T09:37:07.524324+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:07:11.716142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.574601+0000","last_change":"2024-09-10T09:35:28.561124+0000","last_active":"2024-09-10T09:37:07.574601+0000","last_peered":"2024-09-10T09:37:07.574601+0000","last_clean":"2024-09-10T09:37:07.574601+0000","last_became_active":"2024-09-10T09:35:28.560777+0000","last_became_peered":"2024-09-10T09:35:28.560777+0000","last_unstale":"2024-09-10T09:37:07.574601+0000","last_undegraded":"2024-09-10T09:37:07.574601+0000","last_fullsized":"2024-09-10T09:37:07.574601+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:47:04.837596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.538921+0000","last_change":"2024-09-10T09:35:28.576790+0000","last_active":"2024-09-10T09:37:07.538921+0000","last_peered":"2024-09-10T09:37:07.538921+0000","last_clean":"2024-09-10T09:37:07.538921+0000","last_became_active":"2024-09-10T09:35:28.576464+0000","last_became_peered":"2024-09-10T09:35:28.576464+0000","last_unstale":"2024-09-10T09:37:07.538921+0000","last_undegraded":"2024-09-10T09:37:07.538921+0000","last_fullsized":"2024-09-10T09:37:07.538921+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:40:53.517577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.526717+0000","last_change":"2024-09-10T09:35:28.569935+0000","last_active":"2024-09-10T09:37:07.526717+0000","last_peered":"2024-09-10T09:37:07.526717+0000","last_clean":"2024-09-10T09:37:07.526717+0000","last_became_active":"2024-09-10T09:35:28.569539+0000","last_became_peered":"2024-09-10T09:35:28.569539+0000","last_unstale":"2024-09-10T09:37:07.526717+0000","last_undegraded":"2024-09-10T09:37:07.526717+0000","last_fullsized":"2024-09-10T09:37:07.526717+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:40:45.016269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.528743+0000","last_change":"2024-09-10T09:35:28.571619+0000","last_active":"2024-09-10T09:37:07.528743+0000","last_peered":"2024-09-10T09:37:07.528743+0000","last_clean":"2024-09-10T09:37:07.528743+0000","last_became_active":"2024-09-10T09:35:28.568591+0000","last_became_peered":"2024-09-10T09:35:28.568591+0000","last_unstale":"2024-09-10T09:37:07.528743+0000","last_undegraded":"2024-09-10T09:37:07.528743+0000","last_fullsized":"2024-09-10T09:37:07.528743+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:07:49.959712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.527811+0000","last_change":"2024-09-10T09:35:28.573336+0000","last_active":"2024-09-10T09:37:07.527811+0000","last_peered":"2024-09-10T09:37:07.527811+0000","last_clean":"2024-09-10T09:37:07.527811+0000","last_became_active":"2024-09-10T09:35:28.572991+0000","last_became_peered":"2024-09-10T09:35:28.572991+0000","last_unstale":"2024-09-10T09:37:07.527811+0000","last_undegraded":"2024-09-10T09:37:07.527811+0000","last_fullsized":"2024-09-10T09:37:07.527811+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:30:02.905869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.526885+0000","last_change":"2024-09-10T09:35:28.571945+0000","last_active":"2024-09-10T09:37:07.526885+0000","last_peered":"2024-09-10T09:37:07.526885+0000","last_clean":"2024-09-10T09:37:07.526885+0000","last_became_active":"2024-09-10T09:35:28.571546+0000","last_became_peered":"2024-09-10T09:35:28.571546+0000","last_unstale":"2024-09-10T09:37:07.526885+0000","last_undegraded":"2024-09-10T09:37:07.526885+0000","last_fullsized":"2024-09-10T09:37:07.526885+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:36:47.946864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.574295+0000","last_change":"2024-09-10T09:35:28.562712+0000","last_active":"2024-09-10T09:37:07.574295+0000","last_peered":"2024-09-10T09:37:07.574295+0000","last_clean":"2024-09-10T09:37:07.574295+0000","last_became_active":"2024-09-10T09:35:28.562396+0000","last_became_peered":"2024-09-10T09:35:28.562396+0000","last_unstale":"2024-09-10T09:37:07.574295+0000","last_undegraded":"2024-09-10T09:37:07.574295+0000","last_fullsized":"2024-09-10T09:37:07.574295+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:25:47.419162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.761275+0000","last_change":"2024-09-10T09:35:28.566354+0000","last_active":"2024-09-10T09:37:07.761275+0000","last_peered":"2024-09-10T09:37:07.761275+0000","last_clean":"2024-09-10T09:37:07.761275+0000","last_became_active":"2024-09-10T09:35:28.565248+0000","last_became_peered":"2024-09-10T09:35:28.565248+0000","last_unstale":"2024-09-10T09:37:07.761275+0000","last_undegraded":"2024-09-10T09:37:07.761275+0000","last_fullsized":"2024-09-10T09:37:07.761275+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:48:46.741010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.761172+0000","last_change":"2024-09-10T09:35:28.563331+0000","last_active":"2024-09-10T09:37:07.761172+0000","last_peered":"2024-09-10T09:37:07.761172+0000","last_clean":"2024-09-10T09:37:07.761172+0000","last_became_active":"2024-09-10T09:35:28.563020+0000","last_became_peered":"2024-09-10T09:35:28.563020+0000","last_unstale":"2024-09-10T09:37:07.761172+0000","last_undegraded":"2024-09-10T09:37:07.761172+0000","last_fullsized":"2024-09-10T09:37:07.761172+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:27.653551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.538817+0000","last_change":"2024-09-10T09:35:28.566687+0000","last_active":"2024-09-10T09:37:07.538817+0000","last_peered":"2024-09-10T09:37:07.538817+0000","last_clean":"2024-09-10T09:37:07.538817+0000","last_became_active":"2024-09-10T09:35:28.566253+0000","last_became_peered":"2024-09-10T09:35:28.566253+0000","last_unstale":"2024-09-10T09:37:07.538817+0000","last_undegraded":"2024-09-10T09:37:07.538817+0000","last_fullsized":"2024-09-10T09:37:07.538817+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:00:15.709725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.568759+0000","last_change":"2024-09-10T09:35:28.562251+0000","last_active":"2024-09-10T09:37:07.568759+0000","last_peered":"2024-09-10T09:37:07.568759+0000","last_clean":"2024-09-10T09:37:07.568759+0000","last_became_active":"2024-09-10T09:35:28.561933+0000","last_became_peered":"2024-09-10T09:35:28.561933+0000","last_unstale":"2024-09-10T09:37:07.568759+0000","last_undegraded":"2024-09-10T09:37:07.568759+0000","last_fullsized":"2024-09-10T09:37:07.568759+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:30:41.447954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.529401+0000","last_change":"2024-09-10T09:35:28.572164+0000","last_active":"2024-09-10T09:37:07.529401+0000","last_peered":"2024-09-10T09:37:07.529401+0000","last_clean":"2024-09-10T09:37:07.529401+0000","last_became_active":"2024-09-10T09:35:28.571825+0000","last_became_peered":"2024-09-10T09:35:28.571825+0000","last_unstale":"2024-09-10T09:37:07.529401+0000","last_undegraded":"2024-09-10T09:37:07.529401+0000","last_fullsized":"2024-09-10T09:37:07.529401+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:59:50.967269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.526142+0000","last_change":"2024-09-10T09:35:28.568098+0000","last_active":"2024-09-10T09:37:07.526142+0000","last_peered":"2024-09-10T09:37:07.526142+0000","last_clean":"2024-09-10T09:37:07.526142+0000","last_became_active":"2024-09-10T09:35:28.567765+0000","last_became_peered":"2024-09-10T09:35:28.567765+0000","last_unstale":"2024-09-10T09:37:07.526142+0000","last_undegraded":"2024-09-10T09:37:07.526142+0000","last_fullsized":"2024-09-10T09:37:07.526142+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:55:57.260320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.575147+0000","last_change":"2024-09-10T09:35:28.564123+0000","last_active":"2024-09-10T09:37:07.575147+0000","last_peered":"2024-09-10T09:37:07.575147+0000","last_clean":"2024-09-10T09:37:07.575147+0000","last_became_active":"2024-09-10T09:35:28.563814+0000","last_became_peered":"2024-09-10T09:35:28.563814+0000","last_unstale":"2024-09-10T09:37:07.575147+0000","last_undegraded":"2024-09-10T09:37:07.575147+0000","last_fullsized":"2024-09-10T09:37:07.575147+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:14:17.912087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.524203+0000","last_change":"2024-09-10T09:35:28.568601+0000","last_active":"2024-09-10T09:37:07.524203+0000","last_peered":"2024-09-10T09:37:07.524203+0000","last_clean":"2024-09-10T09:37:07.524203+0000","last_became_active":"2024-09-10T09:35:28.568303+0000","last_became_peered":"2024-09-10T09:35:28.568303+0000","last_unstale":"2024-09-10T09:37:07.524203+0000","last_undegraded":"2024-09-10T09:37:07.524203+0000","last_fullsized":"2024-09-10T09:37:07.524203+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:30:01.016420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":42,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.539577+0000","last_change":"2024-09-10T09:37:07.539577+0000","last_active":"2024-09-10T09:37:07.539577+0000","last_peered":"2024-09-10T09:37:07.539577+0000","last_clean":"2024-09-10T09:37:07.539577+0000","last_became_active":"2024-09-10T09:37:07.539206+0000","last_became_peered":"2024-09-10T09:37:07.539206+0000","last_unstale":"2024-09-10T09:37:07.539577+0000","last_undegraded":"2024-09-10T09:37:07.539577+0000","last_fullsized":"2024-09-10T09:37:07.539577+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:56:06.568226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.525650+0000","last_change":"2024-09-10T09:35:28.569225+0000","last_active":"2024-09-10T09:37:07.525650+0000","last_peered":"2024-09-10T09:37:07.525650+0000","last_clean":"2024-09-10T09:37:07.525650+0000","last_became_active":"2024-09-10T09:35:28.568891+0000","last_became_peered":"2024-09-10T09:35:28.568891+0000","last_unstale":"2024-09-10T09:37:07.525650+0000","last_undegraded":"2024-09-10T09:37:07.525650+0000","last_fullsized":"2024-09-10T09:37:07.525650+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:33:45.811781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.529728+0000","last_change":"2024-09-10T09:35:28.578362+0000","last_active":"2024-09-10T09:37:07.529728+0000","last_peered":"2024-09-10T09:37:07.529728+0000","last_clean":"2024-09-10T09:37:07.529728+0000","last_became_active":"2024-09-10T09:35:28.577867+0000","last_became_peered":"2024-09-10T09:35:28.577867+0000","last_unstale":"2024-09-10T09:37:07.529728+0000","last_undegraded":"2024-09-10T09:37:07.529728+0000","last_fullsized":"2024-09-10T09:37:07.529728+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:58:17.041663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.568770+0000","last_change":"2024-09-10T09:35:28.568183+0000","last_active":"2024-09-10T09:37:07.568770+0000","last_peered":"2024-09-10T09:37:07.568770+0000","last_clean":"2024-09-10T09:37:07.568770+0000","last_became_active":"2024-09-10T09:35:28.566775+0000","last_became_peered":"2024-09-10T09:35:28.566775+0000","last_unstale":"2024-09-10T09:37:07.568770+0000","last_undegraded":"2024-09-10T09:37:07.568770+0000","last_fullsized":"2024-09-10T09:37:07.568770+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:28:10.008889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.761305+0000","last_change":"2024-09-10T09:35:28.576122+0000","last_active":"2024-09-10T09:37:07.761305+0000","last_peered":"2024-09-10T09:37:07.761305+0000","last_clean":"2024-09-10T09:37:07.761305+0000","last_became_active":"2024-09-10T09:35:28.575831+0000","last_became_peered":"2024-09-10T09:35:28.575831+0000","last_unstale":"2024-09-10T09:37:07.761305+0000","last_undegraded":"2024-09-10T09:37:07.761305+0000","last_fullsized":"2024-09-10T09:37:07.761305+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:37:51.604658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.761353+0000","last_change":"2024-09-10T09:35:28.564441+0000","last_active":"2024-09-10T09:37:07.761353+0000","last_peered":"2024-09-10T09:37:07.761353+0000","last_clean":"2024-09-10T09:37:07.761353+0000","last_became_active":"2024-09-10T09:35:28.564095+0000","last_became_peered":"2024-09-10T09:35:28.564095+0000","last_unstale":"2024-09-10T09:37:07.761353+0000","last_undegraded":"2024-09-10T09:37:07.761353+0000","last_fullsized":"2024-09-10T09:37:07.761353+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:53:28.275361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.526738+0000","last_change":"2024-09-10T09:35:28.581921+0000","last_active":"2024-09-10T09:37:07.526738+0000","last_peered":"2024-09-10T09:37:07.526738+0000","last_clean":"2024-09-10T09:37:07.526738+0000","last_became_active":"2024-09-10T09:35:28.581573+0000","last_became_peered":"2024-09-10T09:35:28.581573+0000","last_unstale":"2024-09-10T09:37:07.526738+0000","last_undegraded":"2024-09-10T09:37:07.526738+0000","last_fullsized":"2024-09-10T09:37:07.526738+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:58:38.983934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":70139,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":639,"num_read_kb":399,"num_write":249,"num_write_kb":142,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":1015,"num_keys_recovered":7,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":577536,"data_stored":210417,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":214,"ondisk_log_size":214,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":285627,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":77,"seq":330712481828,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32236,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712916,"statfs":{"total":95995035648,"available":95962025984,"internally_reserved":0,"allocated":1069056,"data_stored":554290,"data_compressed":32456,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":70,"seq":300647710760,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28228,"kb_used_data":1196,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716924,"statfs":{"total":95995035648,"available":95966130176,"internally_reserved":0,"allocated":1224704,"data_stored":1208133,"data_compressed":68777,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":65,"seq":279172874287,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32260,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712892,"statfs":{"total":95995035648,"available":95962001408,"internally_reserved":0,"allocated":1093632,"data_stored":554220,"data_compressed":32456,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1599,"internal_metadata":31914433},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":57,"seq":244813135923,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32252,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712900,"statfs":{"total":95995035648,"available":95962009600,"internally_reserved":0,"allocated":1089536,"data_stored":554598,"data_compressed":32456,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":50,"seq":214748364857,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28148,"kb_used_data":1116,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717004,"statfs":{"total":95995035648,"available":95966212096,"internally_reserved":0,"allocated":1142784,"data_stored":1151490,"data_compressed":68777,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":45,"seq":193273528382,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36396,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708756,"statfs":{"total":95995035648,"available":95957766144,"internally_reserved":0,"allocated":1069056,"data_stored":550117,"data_compressed":32456,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790021,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28148,"kb_used_data":1124,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717004,"statfs":{"total":95995035648,"available":95966212096,"internally_reserved":0,"allocated":1150976,"data_stored":1146679,"data_compressed":68777,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953546,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32252,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712900,"statfs":{"total":95995035648,"available":95962009600,"internally_reserved":0,"allocated":1085440,"data_stored":554813,"data_compressed":32456,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182479,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28084,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717068,"statfs":{"total":95995035648,"available":95966277632,"internally_reserved":0,"allocated":1081344,"data_stored":549432,"data_compressed":32456,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411412,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28156,"kb_used_data":1124,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716996,"statfs":{"total":95995035648,"available":95966203904,"internally_reserved":0,"allocated":1150976,"data_stored":613283,"data_compressed":32456,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574939,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32304,"kb_used_data":1112,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712848,"statfs":{"total":95995035648,"available":95961956352,"internally_reserved":0,"allocated":1138688,"data_stored":614078,"data_compressed":32456,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738463,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36416,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708736,"statfs":{"total":95995035648,"available":95957745664,"internally_reserved":0,"allocated":1093632,"data_stored":551180,"data_compressed":32456,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34674,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34674,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34674,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36321,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36321,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36321,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36321,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36321,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":65076,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":64281,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1715,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3213,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1115,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":118784,"data_stored":64667,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T09:37:18.664 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-10T09:37:18.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:37:18 smithi102 ceph-mon[35442]: pgmap v362: 129 pgs: 129 active+clean; 645 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 9 op/s; 101 B/s, 0 keys/s, 0 objects/s recovering 2024-09-10T09:37:18.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:37:18 smithi102 ceph-mon[35442]: from='client.24946 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T09:37:18.902 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:37:18 smithi159 ceph-mon[35513]: pgmap v362: 129 pgs: 129 active+clean; 645 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 9 op/s; 101 B/s, 0 keys/s, 0 objects/s recovering 2024-09-10T09:37:18.902 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:37:18 smithi159 ceph-mon[35513]: from='client.24946 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T09:37:19.042 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:37:19.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:37:18 smithi005 ceph-mon[30350]: pgmap v362: 129 pgs: 129 active+clean; 645 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 9 op/s; 101 B/s, 0 keys/s, 0 objects/s recovering 2024-09-10T09:37:19.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:37:18 smithi005 ceph-mon[30350]: from='client.24946 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T09:37:19.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:19.634+0000 7f3d8c400640 1 -- 172.21.15.5:0/3223941592 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3d84074590 msgr2=0x7f3d84074970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:37:19.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:19.634+0000 7f3d8c400640 1 --2- 172.21.15.5:0/3223941592 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3d84074590 0x7f3d84074970 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f3d78009900 tx=0x7f3d7802a480 comp rx=0 tx=0).stop 2024-09-10T09:37:19.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:19.634+0000 7f3d8c400640 1 -- 172.21.15.5:0/3223941592 shutdown_connections 2024-09-10T09:37:19.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:19.634+0000 7f3d8c400640 1 --2- 172.21.15.5:0/3223941592 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3d84112b90 0x7f3d84114f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:19.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:19.634+0000 7f3d8c400640 1 --2- 172.21.15.5:0/3223941592 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3d84074eb0 0x7f3d84112650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:19.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:19.634+0000 7f3d8c400640 1 --2- 172.21.15.5:0/3223941592 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3d84074590 0x7f3d84074970 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:19.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:19.634+0000 7f3d8c400640 1 -- 172.21.15.5:0/3223941592 >> 172.21.15.5:0/3223941592 conn(0x7f3d840fd8c0 msgr2=0x7f3d840ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:37:19.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:19.634+0000 7f3d8c400640 1 -- 172.21.15.5:0/3223941592 shutdown_connections 2024-09-10T09:37:19.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:19.634+0000 7f3d8c400640 1 -- 172.21.15.5:0/3223941592 wait complete. 2024-09-10T09:37:19.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:19.635+0000 7f3d8c400640 1 Processor -- start 2024-09-10T09:37:19.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:19.635+0000 7f3d8c400640 1 -- start start 2024-09-10T09:37:19.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:19.635+0000 7f3d8c400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3d84074590 0x7f3d8419eb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:37:19.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:19.635+0000 7f3d8c400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3d84074eb0 0x7f3d8419f090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:37:19.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:19.635+0000 7f3d8c400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3d84112b90 0x7f3d841a3450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:37:19.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:19.635+0000 7f3d8c400640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f3d841185c0 con 0x7f3d84112b90 2024-09-10T09:37:19.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:19.635+0000 7f3d8c400640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f3d84118440 con 0x7f3d84074eb0 2024-09-10T09:37:19.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:19.635+0000 7f3d8c400640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f3d84118740 con 0x7f3d84074590 2024-09-10T09:37:19.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:19.636+0000 7f3d8a400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3d84112b90 0x7f3d841a3450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:37:19.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:19.636+0000 7f3d89a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3d84074590 0x7f3d8419eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:37:19.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:19.636+0000 7f3d89a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3d84074590 0x7f3d8419eb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:49078/0 (socket says 172.21.15.5:49078) 2024-09-10T09:37:19.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:19.636+0000 7f3d89000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3d84074eb0 0x7f3d8419f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:37:19.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:19.636+0000 7f3d89a00640 1 -- 172.21.15.5:0/2826702193 learned_addr learned my addr 172.21.15.5:0/2826702193 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:37:19.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:19.636+0000 7f3d89000640 1 -- 172.21.15.5:0/2826702193 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3d84074590 msgr2=0x7f3d8419eb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:37:19.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:19.636+0000 7f3d89000640 1 --2- 172.21.15.5:0/2826702193 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3d84074590 0x7f3d8419eb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:19.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:19.636+0000 7f3d89000640 1 -- 172.21.15.5:0/2826702193 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3d84112b90 msgr2=0x7f3d841a3450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:37:19.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:19.636+0000 7f3d89000640 1 --2- 172.21.15.5:0/2826702193 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3d84112b90 0x7f3d841a3450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:19.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:19.636+0000 7f3d89000640 1 -- 172.21.15.5:0/2826702193 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d841a3b80 con 0x7f3d84074eb0 2024-09-10T09:37:19.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:19.637+0000 7f3d89000640 1 --2- 172.21.15.5:0/2826702193 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3d84074eb0 0x7f3d8419f090 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f3d7000e9f0 tx=0x7f3d7000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:37:19.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:19.637+0000 7f3d82000640 1 -- 172.21.15.5:0/2826702193 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d7000ccc0 con 0x7f3d84074eb0 2024-09-10T09:37:19.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:19.637+0000 7f3d82000640 1 -- 172.21.15.5:0/2826702193 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3d70005020 con 0x7f3d84074eb0 2024-09-10T09:37:19.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:19.637+0000 7f3d82000640 1 -- 172.21.15.5:0/2826702193 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d700049e0 con 0x7f3d84074eb0 2024-09-10T09:37:19.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:19.637+0000 7f3d8c400640 1 -- 172.21.15.5:0/2826702193 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d841a3e70 con 0x7f3d84074eb0 2024-09-10T09:37:19.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:19.637+0000 7f3d8c400640 1 -- 172.21.15.5:0/2826702193 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d841ab700 con 0x7f3d84074eb0 2024-09-10T09:37:19.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:19.638+0000 7f3d8c400640 1 -- 172.21.15.5:0/2826702193 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3d84074970 con 0x7f3d84074eb0 2024-09-10T09:37:19.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:19.640+0000 7f3d82000640 1 -- 172.21.15.5:0/2826702193 <== mon.2 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f3d70005560 con 0x7f3d84074eb0 2024-09-10T09:37:19.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:19.641+0000 7f3d82000640 1 --2- 172.21.15.5:0/2826702193 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f3d6c0a2c10 0x7f3d6c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:37:19.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:19.642+0000 7f3d82000640 1 -- 172.21.15.5:0/2826702193 <== mon.2 v2:172.21.15.102:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f3d700cf5a0 con 0x7f3d84074eb0 2024-09-10T09:37:19.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:19.642+0000 7f3d89a00640 1 --2- 172.21.15.5:0/2826702193 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f3d6c0a2c10 0x7f3d6c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:37:19.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:19.642+0000 7f3d89a00640 1 --2- 172.21.15.5:0/2826702193 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f3d6c0a2c10 0x7f3d6c0a50d0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f3d780096c0 tx=0x7f3d780046c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:37:19.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:19.646+0000 7f3d82000640 1 -- 172.21.15.5:0/2826702193 <== mon.2 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f3d700051d0 con 0x7f3d84074eb0 2024-09-10T09:37:19.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:19.829+0000 7f3d8c400640 1 -- 172.21.15.5:0/2826702193 --> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f3d84105040 con 0x7f3d6c0a2c10 2024-09-10T09:37:19.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:19.838+0000 7f3d82000640 1 -- 172.21.15.5:0/2826702193 <== mgr.14150 v2:172.21.15.5:6800/2707348081 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340706 (secure 0 0 0) 0x7f3d780041c0 con 0x7f3d6c0a2c10 2024-09-10T09:37:19.840 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-10T09:37:19.851 INFO:teuthology.orchestra.run.smithi005.stderr:dumped all 2024-09-10T09:37:19.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:19.842+0000 7f3d8c400640 1 -- 172.21.15.5:0/2826702193 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f3d6c0a2c10 msgr2=0x7f3d6c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:37:19.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:19.842+0000 7f3d8c400640 1 --2- 172.21.15.5:0/2826702193 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f3d6c0a2c10 0x7f3d6c0a50d0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f3d780096c0 tx=0x7f3d780046c0 comp rx=0 tx=0).stop 2024-09-10T09:37:19.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:19.842+0000 7f3d8c400640 1 -- 172.21.15.5:0/2826702193 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3d84074eb0 msgr2=0x7f3d8419f090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:37:19.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:19.842+0000 7f3d8c400640 1 --2- 172.21.15.5:0/2826702193 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3d84074eb0 0x7f3d8419f090 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f3d7000e9f0 tx=0x7f3d7000eeb0 comp rx=0 tx=0).stop 2024-09-10T09:37:19.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:19.843+0000 7f3d8c400640 1 -- 172.21.15.5:0/2826702193 shutdown_connections 2024-09-10T09:37:19.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:19.843+0000 7f3d8c400640 1 --2- 172.21.15.5:0/2826702193 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f3d6c0a2c10 0x7f3d6c0a50d0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:19.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:19.843+0000 7f3d8c400640 1 --2- 172.21.15.5:0/2826702193 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3d84112b90 0x7f3d841a3450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:19.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:19.843+0000 7f3d8c400640 1 --2- 172.21.15.5:0/2826702193 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3d84074eb0 0x7f3d8419f090 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:19.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:19.843+0000 7f3d8c400640 1 --2- 172.21.15.5:0/2826702193 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3d84074590 0x7f3d8419eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:19.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:19.843+0000 7f3d8c400640 1 -- 172.21.15.5:0/2826702193 >> 172.21.15.5:0/2826702193 conn(0x7f3d840fd8c0 msgr2=0x7f3d840ff050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:37:19.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:19.843+0000 7f3d8c400640 1 -- 172.21.15.5:0/2826702193 shutdown_connections 2024-09-10T09:37:19.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:19.843+0000 7f3d8c400640 1 -- 172.21.15.5:0/2826702193 wait complete. 2024-09-10T09:37:20.610 INFO:teuthology.orchestra.run.smithi005.stdout:{"pg_ready":true,"pg_map":{"version":363,"stamp":"2024-09-10T09:37:19.555509+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":660741,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":775,"num_read_kb":513,"num_write":432,"num_write_kb":3082,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":1181751,"num_keys_recovered":7,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":353,"ondisk_log_size":353,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":374880,"kb_used_data":13076,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124566944,"statfs":{"total":1151940427776,"available":1151556550656,"internally_reserved":0,"allocated":13389824,"data_stored":8602313,"data_compressed":498435,"data_compressed_allocated":675840,"data_compressed_original":2850816,"omap_allocated":19141,"internal_metadata":370193723},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":101,"num_read_kb":53,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":1015,"num_keys_recovered":7,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003101"},"pg_stats":[{"pgid":"2.1f","version":"94'13","reported_seq":117,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:13.581313+0000","last_change":"2024-09-10T09:35:25.533085+0000","last_active":"2024-09-10T09:37:13.581313+0000","last_peered":"2024-09-10T09:37:13.581313+0000","last_clean":"2024-09-10T09:37:13.581313+0000","last_became_active":"2024-09-10T09:35:25.532729+0000","last_became_peered":"2024-09-10T09:35:25.532729+0000","last_unstale":"2024-09-10T09:37:13.581313+0000","last_undegraded":"2024-09-10T09:37:13.581313+0000","last_fullsized":"2024-09-10T09:37:13.581313+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:55:11.158279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":61,"num_read_kb":61,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":15,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.527458+0000","last_change":"2024-09-10T09:36:06.767356+0000","last_active":"2024-09-10T09:37:07.527458+0000","last_peered":"2024-09-10T09:37:07.527458+0000","last_clean":"2024-09-10T09:37:07.527458+0000","last_became_active":"2024-09-10T09:36:06.766931+0000","last_became_peered":"2024-09-10T09:36:06.766931+0000","last_unstale":"2024-09-10T09:37:07.527458+0000","last_undegraded":"2024-09-10T09:37:07.527458+0000","last_fullsized":"2024-09-10T09:37:07.527458+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:25:02.777327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,7],"acting":[9,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":28,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.527473+0000","last_change":"2024-09-10T09:35:36.629797+0000","last_active":"2024-09-10T09:37:07.527473+0000","last_peered":"2024-09-10T09:37:07.527473+0000","last_clean":"2024-09-10T09:37:07.527473+0000","last_became_active":"2024-09-10T09:35:36.629414+0000","last_became_peered":"2024-09-10T09:35:36.629414+0000","last_unstale":"2024-09-10T09:37:07.527473+0000","last_undegraded":"2024-09-10T09:37:07.527473+0000","last_fullsized":"2024-09-10T09:37:07.527473+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:42:44.610581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"94'2","reported_seq":46,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.524916+0000","last_change":"2024-09-10T09:35:25.852022+0000","last_active":"2024-09-10T09:37:07.524916+0000","last_peered":"2024-09-10T09:37:07.524916+0000","last_clean":"2024-09-10T09:37:07.524916+0000","last_became_active":"2024-09-10T09:35:25.851587+0000","last_became_peered":"2024-09-10T09:35:25.851587+0000","last_unstale":"2024-09-10T09:37:07.524916+0000","last_undegraded":"2024-09-10T09:37:07.524916+0000","last_fullsized":"2024-09-10T09:37:07.524916+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:37:26.085914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.761614+0000","last_change":"2024-09-10T09:35:28.579260+0000","last_active":"2024-09-10T09:37:07.761614+0000","last_peered":"2024-09-10T09:37:07.761614+0000","last_clean":"2024-09-10T09:37:07.761614+0000","last_became_active":"2024-09-10T09:35:28.574584+0000","last_became_peered":"2024-09-10T09:35:28.574584+0000","last_unstale":"2024-09-10T09:37:07.761614+0000","last_undegraded":"2024-09-10T09:37:07.761614+0000","last_fullsized":"2024-09-10T09:37:07.761614+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:31:19.277689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":35,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.568297+0000","last_change":"2024-09-10T09:36:06.874861+0000","last_active":"2024-09-10T09:37:07.568297+0000","last_peered":"2024-09-10T09:37:07.568297+0000","last_clean":"2024-09-10T09:37:07.568297+0000","last_became_active":"2024-09-10T09:36:06.873930+0000","last_became_peered":"2024-09-10T09:36:06.873930+0000","last_unstale":"2024-09-10T09:37:07.568297+0000","last_undegraded":"2024-09-10T09:37:07.568297+0000","last_fullsized":"2024-09-10T09:37:07.568297+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:35:01.354061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,8,3],"acting":[4,6,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"94'12","reported_seq":56,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.530293+0000","last_change":"2024-09-10T09:35:25.534571+0000","last_active":"2024-09-10T09:37:07.530293+0000","last_peered":"2024-09-10T09:37:07.530293+0000","last_clean":"2024-09-10T09:37:07.530293+0000","last_became_active":"2024-09-10T09:35:25.534235+0000","last_became_peered":"2024-09-10T09:35:25.534235+0000","last_unstale":"2024-09-10T09:37:07.530293+0000","last_undegraded":"2024-09-10T09:37:07.530293+0000","last_fullsized":"2024-09-10T09:37:07.530293+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:50:21.447542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.574647+0000","last_change":"2024-09-10T09:35:28.565541+0000","last_active":"2024-09-10T09:37:07.574647+0000","last_peered":"2024-09-10T09:37:07.574647+0000","last_clean":"2024-09-10T09:37:07.574647+0000","last_became_active":"2024-09-10T09:35:28.565095+0000","last_became_peered":"2024-09-10T09:35:28.565095+0000","last_unstale":"2024-09-10T09:37:07.574647+0000","last_undegraded":"2024-09-10T09:37:07.574647+0000","last_fullsized":"2024-09-10T09:37:07.574647+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:29:31.285401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":28,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.568540+0000","last_change":"2024-09-10T09:35:36.629277+0000","last_active":"2024-09-10T09:37:07.568540+0000","last_peered":"2024-09-10T09:37:07.568540+0000","last_clean":"2024-09-10T09:37:07.568540+0000","last_became_active":"2024-09-10T09:35:36.628687+0000","last_became_peered":"2024-09-10T09:35:36.628687+0000","last_unstale":"2024-09-10T09:37:07.568540+0000","last_undegraded":"2024-09-10T09:37:07.568540+0000","last_fullsized":"2024-09-10T09:37:07.568540+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:34:33.628827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"94'5","reported_seq":49,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.530106+0000","last_change":"2024-09-10T09:35:25.536137+0000","last_active":"2024-09-10T09:37:07.530106+0000","last_peered":"2024-09-10T09:37:07.530106+0000","last_clean":"2024-09-10T09:37:07.530106+0000","last_became_active":"2024-09-10T09:35:25.535728+0000","last_became_peered":"2024-09-10T09:35:25.535728+0000","last_unstale":"2024-09-10T09:37:07.530106+0000","last_undegraded":"2024-09-10T09:37:07.530106+0000","last_fullsized":"2024-09-10T09:37:07.530106+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:36:04.829901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":44,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.532371+0000","last_change":"2024-09-10T09:36:06.874425+0000","last_active":"2024-09-10T09:37:07.532371+0000","last_peered":"2024-09-10T09:37:07.532371+0000","last_clean":"2024-09-10T09:37:07.532371+0000","last_became_active":"2024-09-10T09:36:06.874050+0000","last_became_peered":"2024-09-10T09:36:06.874050+0000","last_unstale":"2024-09-10T09:37:07.532371+0000","last_undegraded":"2024-09-10T09:37:07.532371+0000","last_fullsized":"2024-09-10T09:37:07.532371+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:30:51.200335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,1],"acting":[5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":28,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.567999+0000","last_change":"2024-09-10T09:35:36.615314+0000","last_active":"2024-09-10T09:37:07.567999+0000","last_peered":"2024-09-10T09:37:07.567999+0000","last_clean":"2024-09-10T09:37:07.567999+0000","last_became_active":"2024-09-10T09:35:36.614820+0000","last_became_peered":"2024-09-10T09:35:36.614820+0000","last_unstale":"2024-09-10T09:37:07.567999+0000","last_undegraded":"2024-09-10T09:37:07.567999+0000","last_fullsized":"2024-09-10T09:37:07.567999+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:51:16.897935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.528727+0000","last_change":"2024-09-10T09:35:28.572346+0000","last_active":"2024-09-10T09:37:07.528727+0000","last_peered":"2024-09-10T09:37:07.528727+0000","last_clean":"2024-09-10T09:37:07.528727+0000","last_became_active":"2024-09-10T09:35:28.571839+0000","last_became_peered":"2024-09-10T09:35:28.571839+0000","last_unstale":"2024-09-10T09:37:07.528727+0000","last_undegraded":"2024-09-10T09:37:07.528727+0000","last_fullsized":"2024-09-10T09:37:07.528727+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:41:29.662441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"94'4","reported_seq":48,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.529863+0000","last_change":"2024-09-10T09:35:25.538747+0000","last_active":"2024-09-10T09:37:07.529863+0000","last_peered":"2024-09-10T09:37:07.529863+0000","last_clean":"2024-09-10T09:37:07.529863+0000","last_became_active":"2024-09-10T09:35:25.538330+0000","last_became_peered":"2024-09-10T09:35:25.538330+0000","last_unstale":"2024-09-10T09:37:07.529863+0000","last_undegraded":"2024-09-10T09:37:07.529863+0000","last_fullsized":"2024-09-10T09:37:07.529863+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:15:08.340688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":28,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.568378+0000","last_change":"2024-09-10T09:35:37.270106+0000","last_active":"2024-09-10T09:37:07.568378+0000","last_peered":"2024-09-10T09:37:07.568378+0000","last_clean":"2024-09-10T09:37:07.568378+0000","last_became_active":"2024-09-10T09:35:37.269575+0000","last_became_peered":"2024-09-10T09:35:37.269575+0000","last_unstale":"2024-09-10T09:37:07.568378+0000","last_undegraded":"2024-09-10T09:37:07.568378+0000","last_fullsized":"2024-09-10T09:37:07.568378+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:35:00.414602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.524435+0000","last_change":"2024-09-10T09:35:28.569230+0000","last_active":"2024-09-10T09:37:07.524435+0000","last_peered":"2024-09-10T09:37:07.524435+0000","last_clean":"2024-09-10T09:37:07.524435+0000","last_became_active":"2024-09-10T09:35:28.568266+0000","last_became_peered":"2024-09-10T09:35:28.568266+0000","last_unstale":"2024-09-10T09:37:07.524435+0000","last_undegraded":"2024-09-10T09:37:07.524435+0000","last_fullsized":"2024-09-10T09:37:07.524435+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:49:43.730686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"94'3","reported_seq":17,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.544707+0000","last_change":"2024-09-10T09:37:07.544298+0000","last_active":"2024-09-10T09:37:07.544707+0000","last_peered":"2024-09-10T09:37:07.544707+0000","last_clean":"2024-09-10T09:37:07.544707+0000","last_became_active":"2024-09-10T09:37:07.531815+0000","last_became_peered":"2024-09-10T09:37:07.531815+0000","last_unstale":"2024-09-10T09:37:07.544707+0000","last_undegraded":"2024-09-10T09:37:07.544707+0000","last_fullsized":"2024-09-10T09:37:07.544707+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:18:10.091895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"94'1","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.568418+0000","last_change":"2024-09-10T09:36:06.875056+0000","last_active":"2024-09-10T09:37:07.568418+0000","last_peered":"2024-09-10T09:37:07.568418+0000","last_clean":"2024-09-10T09:37:07.568418+0000","last_became_active":"2024-09-10T09:36:06.874400+0000","last_became_peered":"2024-09-10T09:36:06.874400+0000","last_unstale":"2024-09-10T09:37:07.568418+0000","last_undegraded":"2024-09-10T09:37:07.568418+0000","last_fullsized":"2024-09-10T09:37:07.568418+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:21:06.300712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,7,8],"acting":[4,10,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.526854+0000","last_change":"2024-09-10T09:35:28.571319+0000","last_active":"2024-09-10T09:37:07.526854+0000","last_peered":"2024-09-10T09:37:07.526854+0000","last_clean":"2024-09-10T09:37:07.526854+0000","last_became_active":"2024-09-10T09:35:28.570972+0000","last_became_peered":"2024-09-10T09:35:28.570972+0000","last_unstale":"2024-09-10T09:37:07.526854+0000","last_undegraded":"2024-09-10T09:37:07.526854+0000","last_fullsized":"2024-09-10T09:37:07.526854+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:45:19.300535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"94'7","reported_seq":51,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.524428+0000","last_change":"2024-09-10T09:35:25.530489+0000","last_active":"2024-09-10T09:37:07.524428+0000","last_peered":"2024-09-10T09:37:07.524428+0000","last_clean":"2024-09-10T09:37:07.524428+0000","last_became_active":"2024-09-10T09:35:25.530197+0000","last_became_peered":"2024-09-10T09:35:25.530197+0000","last_unstale":"2024-09-10T09:37:07.524428+0000","last_undegraded":"2024-09-10T09:37:07.524428+0000","last_fullsized":"2024-09-10T09:37:07.524428+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:28:57.539569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":28,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.527603+0000","last_change":"2024-09-10T09:35:36.630507+0000","last_active":"2024-09-10T09:37:07.527603+0000","last_peered":"2024-09-10T09:37:07.527603+0000","last_clean":"2024-09-10T09:37:07.527603+0000","last_became_active":"2024-09-10T09:35:36.630052+0000","last_became_peered":"2024-09-10T09:35:36.630052+0000","last_unstale":"2024-09-10T09:37:07.527603+0000","last_undegraded":"2024-09-10T09:37:07.527603+0000","last_fullsized":"2024-09-10T09:37:07.527603+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:13:42.691382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.529586+0000","last_change":"2024-09-10T09:35:28.565650+0000","last_active":"2024-09-10T09:37:07.529586+0000","last_peered":"2024-09-10T09:37:07.529586+0000","last_clean":"2024-09-10T09:37:07.529586+0000","last_became_active":"2024-09-10T09:35:28.565303+0000","last_became_peered":"2024-09-10T09:35:28.565303+0000","last_unstale":"2024-09-10T09:37:07.529586+0000","last_undegraded":"2024-09-10T09:37:07.529586+0000","last_fullsized":"2024-09-10T09:37:07.529586+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:48:08.965487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"94'2","reported_seq":45,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.574420+0000","last_change":"2024-09-10T09:35:25.851326+0000","last_active":"2024-09-10T09:37:07.574420+0000","last_peered":"2024-09-10T09:37:07.574420+0000","last_clean":"2024-09-10T09:37:07.574420+0000","last_became_active":"2024-09-10T09:35:25.850981+0000","last_became_peered":"2024-09-10T09:35:25.850981+0000","last_unstale":"2024-09-10T09:37:07.574420+0000","last_undegraded":"2024-09-10T09:37:07.574420+0000","last_fullsized":"2024-09-10T09:37:07.574420+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:50:28.877840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":28,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.568406+0000","last_change":"2024-09-10T09:35:37.270138+0000","last_active":"2024-09-10T09:37:07.568406+0000","last_peered":"2024-09-10T09:37:07.568406+0000","last_clean":"2024-09-10T09:37:07.568406+0000","last_became_active":"2024-09-10T09:35:37.269663+0000","last_became_peered":"2024-09-10T09:35:37.269663+0000","last_unstale":"2024-09-10T09:37:07.568406+0000","last_undegraded":"2024-09-10T09:37:07.568406+0000","last_fullsized":"2024-09-10T09:37:07.568406+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:18:00.105970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.761440+0000","last_change":"2024-09-10T09:35:28.576197+0000","last_active":"2024-09-10T09:37:07.761440+0000","last_peered":"2024-09-10T09:37:07.761440+0000","last_clean":"2024-09-10T09:37:07.761440+0000","last_became_active":"2024-09-10T09:35:28.575859+0000","last_became_peered":"2024-09-10T09:35:28.575859+0000","last_unstale":"2024-09-10T09:37:07.761440+0000","last_undegraded":"2024-09-10T09:37:07.761440+0000","last_fullsized":"2024-09-10T09:37:07.761440+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:09:39.458761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"94'7","reported_seq":50,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.568747+0000","last_change":"2024-09-10T09:35:25.990587+0000","last_active":"2024-09-10T09:37:07.568747+0000","last_peered":"2024-09-10T09:37:07.568747+0000","last_clean":"2024-09-10T09:37:07.568747+0000","last_became_active":"2024-09-10T09:35:25.990302+0000","last_became_peered":"2024-09-10T09:35:25.990302+0000","last_unstale":"2024-09-10T09:37:07.568747+0000","last_undegraded":"2024-09-10T09:37:07.568747+0000","last_fullsized":"2024-09-10T09:37:07.568747+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:18:41.244441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":28,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.527701+0000","last_change":"2024-09-10T09:35:36.612983+0000","last_active":"2024-09-10T09:37:07.527701+0000","last_peered":"2024-09-10T09:37:07.527701+0000","last_clean":"2024-09-10T09:37:07.527701+0000","last_became_active":"2024-09-10T09:35:36.612542+0000","last_became_peered":"2024-09-10T09:35:36.612542+0000","last_unstale":"2024-09-10T09:37:07.527701+0000","last_undegraded":"2024-09-10T09:37:07.527701+0000","last_fullsized":"2024-09-10T09:37:07.527701+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:50:16.298195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.524933+0000","last_change":"2024-09-10T09:35:28.564121+0000","last_active":"2024-09-10T09:37:07.524933+0000","last_peered":"2024-09-10T09:37:07.524933+0000","last_clean":"2024-09-10T09:37:07.524933+0000","last_became_active":"2024-09-10T09:35:28.562780+0000","last_became_peered":"2024-09-10T09:35:28.562780+0000","last_unstale":"2024-09-10T09:37:07.524933+0000","last_undegraded":"2024-09-10T09:37:07.524933+0000","last_fullsized":"2024-09-10T09:37:07.524933+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:53:15.690478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"94'15","reported_seq":74,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.541031+0000","last_change":"2024-09-10T09:37:07.541031+0000","last_active":"2024-09-10T09:37:07.541031+0000","last_peered":"2024-09-10T09:37:07.541031+0000","last_clean":"2024-09-10T09:37:07.541031+0000","last_became_active":"2024-09-10T09:37:07.529577+0000","last_became_peered":"2024-09-10T09:37:07.529577+0000","last_unstale":"2024-09-10T09:37:07.541031+0000","last_undegraded":"2024-09-10T09:37:07.541031+0000","last_fullsized":"2024-09-10T09:37:07.541031+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:14:13.440346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":7,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":28,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.524058+0000","last_change":"2024-09-10T09:35:37.270888+0000","last_active":"2024-09-10T09:37:07.524058+0000","last_peered":"2024-09-10T09:37:07.524058+0000","last_clean":"2024-09-10T09:37:07.524058+0000","last_became_active":"2024-09-10T09:35:37.270229+0000","last_became_peered":"2024-09-10T09:35:37.270229+0000","last_unstale":"2024-09-10T09:37:07.524058+0000","last_undegraded":"2024-09-10T09:37:07.524058+0000","last_fullsized":"2024-09-10T09:37:07.524058+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:03:54.223463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.538908+0000","last_change":"2024-09-10T09:35:28.572799+0000","last_active":"2024-09-10T09:37:07.538908+0000","last_peered":"2024-09-10T09:37:07.538908+0000","last_clean":"2024-09-10T09:37:07.538908+0000","last_became_active":"2024-09-10T09:35:28.572310+0000","last_became_peered":"2024-09-10T09:35:28.572310+0000","last_unstale":"2024-09-10T09:37:07.538908+0000","last_undegraded":"2024-09-10T09:37:07.538908+0000","last_fullsized":"2024-09-10T09:37:07.538908+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:33:51.446766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"94'4","reported_seq":48,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.530409+0000","last_change":"2024-09-10T09:35:25.851868+0000","last_active":"2024-09-10T09:37:07.530409+0000","last_peered":"2024-09-10T09:37:07.530409+0000","last_clean":"2024-09-10T09:37:07.530409+0000","last_became_active":"2024-09-10T09:35:25.851447+0000","last_became_peered":"2024-09-10T09:35:25.851447+0000","last_unstale":"2024-09-10T09:37:07.530409+0000","last_undegraded":"2024-09-10T09:37:07.530409+0000","last_fullsized":"2024-09-10T09:37:07.530409+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:15:57.706690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":28,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.526489+0000","last_change":"2024-09-10T09:35:36.619459+0000","last_active":"2024-09-10T09:37:07.526489+0000","last_peered":"2024-09-10T09:37:07.526489+0000","last_clean":"2024-09-10T09:37:07.526489+0000","last_became_active":"2024-09-10T09:35:36.619023+0000","last_became_peered":"2024-09-10T09:35:36.619023+0000","last_unstale":"2024-09-10T09:37:07.526489+0000","last_undegraded":"2024-09-10T09:37:07.526489+0000","last_fullsized":"2024-09-10T09:37:07.526489+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:07:32.999231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.525808+0000","last_change":"2024-09-10T09:35:28.575668+0000","last_active":"2024-09-10T09:37:07.525808+0000","last_peered":"2024-09-10T09:37:07.525808+0000","last_clean":"2024-09-10T09:37:07.525808+0000","last_became_active":"2024-09-10T09:35:28.575425+0000","last_became_peered":"2024-09-10T09:35:28.575425+0000","last_unstale":"2024-09-10T09:37:07.525808+0000","last_undegraded":"2024-09-10T09:37:07.525808+0000","last_fullsized":"2024-09-10T09:37:07.525808+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:13:27.120208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"94'1","reported_seq":44,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.527840+0000","last_change":"2024-09-10T09:35:25.854569+0000","last_active":"2024-09-10T09:37:07.527840+0000","last_peered":"2024-09-10T09:37:07.527840+0000","last_clean":"2024-09-10T09:37:07.527840+0000","last_became_active":"2024-09-10T09:35:25.854152+0000","last_became_peered":"2024-09-10T09:35:25.854152+0000","last_unstale":"2024-09-10T09:37:07.527840+0000","last_undegraded":"2024-09-10T09:37:07.527840+0000","last_fullsized":"2024-09-10T09:37:07.527840+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:27:15.584707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":28,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.529899+0000","last_change":"2024-09-10T09:35:37.270517+0000","last_active":"2024-09-10T09:37:07.529899+0000","last_peered":"2024-09-10T09:37:07.529899+0000","last_clean":"2024-09-10T09:37:07.529899+0000","last_became_active":"2024-09-10T09:35:37.269793+0000","last_became_peered":"2024-09-10T09:35:37.269793+0000","last_unstale":"2024-09-10T09:37:07.529899+0000","last_undegraded":"2024-09-10T09:37:07.529899+0000","last_fullsized":"2024-09-10T09:37:07.529899+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:28:58.272848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":44,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.762585+0000","last_change":"2024-09-10T09:37:07.762585+0000","last_active":"2024-09-10T09:37:07.762585+0000","last_peered":"2024-09-10T09:37:07.762585+0000","last_clean":"2024-09-10T09:37:07.762585+0000","last_became_active":"2024-09-10T09:37:07.762250+0000","last_became_peered":"2024-09-10T09:37:07.762250+0000","last_unstale":"2024-09-10T09:37:07.762585+0000","last_undegraded":"2024-09-10T09:37:07.762585+0000","last_fullsized":"2024-09-10T09:37:07.762585+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:38:01.751512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"94'6","reported_seq":110,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:13.581994+0000","last_change":"2024-09-10T09:35:25.531062+0000","last_active":"2024-09-10T09:37:13.581994+0000","last_peered":"2024-09-10T09:37:13.581994+0000","last_clean":"2024-09-10T09:37:13.581994+0000","last_became_active":"2024-09-10T09:35:25.530005+0000","last_became_peered":"2024-09-10T09:35:25.530005+0000","last_unstale":"2024-09-10T09:37:13.581994+0000","last_undegraded":"2024-09-10T09:37:13.581994+0000","last_fullsized":"2024-09-10T09:37:13.581994+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:42:35.330312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":981,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":62,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":28,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.538712+0000","last_change":"2024-09-10T09:35:36.623099+0000","last_active":"2024-09-10T09:37:07.538712+0000","last_peered":"2024-09-10T09:37:07.538712+0000","last_clean":"2024-09-10T09:37:07.538712+0000","last_became_active":"2024-09-10T09:35:36.622716+0000","last_became_peered":"2024-09-10T09:35:36.622716+0000","last_unstale":"2024-09-10T09:37:07.538712+0000","last_undegraded":"2024-09-10T09:37:07.538712+0000","last_fullsized":"2024-09-10T09:37:07.538712+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:10:38.495164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":44,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.530897+0000","last_change":"2024-09-10T09:36:06.769405+0000","last_active":"2024-09-10T09:37:07.530897+0000","last_peered":"2024-09-10T09:37:07.530897+0000","last_clean":"2024-09-10T09:37:07.530897+0000","last_became_active":"2024-09-10T09:36:06.769042+0000","last_became_peered":"2024-09-10T09:36:06.769042+0000","last_unstale":"2024-09-10T09:37:07.530897+0000","last_undegraded":"2024-09-10T09:37:07.530897+0000","last_fullsized":"2024-09-10T09:37:07.530897+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:01:49.340765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,8],"acting":[5,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"94'5","reported_seq":114,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:17.581784+0000","last_change":"2024-09-10T09:35:25.851430+0000","last_active":"2024-09-10T09:37:17.581784+0000","last_peered":"2024-09-10T09:37:17.581784+0000","last_clean":"2024-09-10T09:37:17.581784+0000","last_became_active":"2024-09-10T09:35:25.851067+0000","last_became_peered":"2024-09-10T09:35:25.851067+0000","last_unstale":"2024-09-10T09:37:17.581784+0000","last_undegraded":"2024-09-10T09:37:17.581784+0000","last_fullsized":"2024-09-10T09:37:17.581784+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:55:58.756207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":66,"num_read_kb":66,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":28,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.538595+0000","last_change":"2024-09-10T09:35:37.270361+0000","last_active":"2024-09-10T09:37:07.538595+0000","last_peered":"2024-09-10T09:37:07.538595+0000","last_clean":"2024-09-10T09:37:07.538595+0000","last_became_active":"2024-09-10T09:35:37.269632+0000","last_became_peered":"2024-09-10T09:35:37.269632+0000","last_unstale":"2024-09-10T09:37:07.538595+0000","last_undegraded":"2024-09-10T09:37:07.538595+0000","last_fullsized":"2024-09-10T09:37:07.538595+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:50:34.360497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.538361+0000","last_change":"2024-09-10T09:35:28.571064+0000","last_active":"2024-09-10T09:37:07.538361+0000","last_peered":"2024-09-10T09:37:07.538361+0000","last_clean":"2024-09-10T09:37:07.538361+0000","last_became_active":"2024-09-10T09:35:28.570691+0000","last_became_peered":"2024-09-10T09:35:28.570691+0000","last_unstale":"2024-09-10T09:37:07.538361+0000","last_undegraded":"2024-09-10T09:37:07.538361+0000","last_fullsized":"2024-09-10T09:37:07.538361+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:58:33.162501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"94'4","reported_seq":56,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.572247+0000","last_change":"2024-09-10T09:37:07.572247+0000","last_active":"2024-09-10T09:37:07.572247+0000","last_peered":"2024-09-10T09:37:07.572247+0000","last_clean":"2024-09-10T09:37:07.572247+0000","last_became_active":"2024-09-10T09:37:07.569410+0000","last_became_peered":"2024-09-10T09:37:07.569410+0000","last_unstale":"2024-09-10T09:37:07.572247+0000","last_undegraded":"2024-09-10T09:37:07.572247+0000","last_fullsized":"2024-09-10T09:37:07.572247+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:05:09.037084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,4],"acting":[7,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":28,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.532586+0000","last_change":"2024-09-10T09:35:37.270358+0000","last_active":"2024-09-10T09:37:07.532586+0000","last_peered":"2024-09-10T09:37:07.532586+0000","last_clean":"2024-09-10T09:37:07.532586+0000","last_became_active":"2024-09-10T09:35:37.269822+0000","last_became_peered":"2024-09-10T09:35:37.269822+0000","last_unstale":"2024-09-10T09:37:07.532586+0000","last_undegraded":"2024-09-10T09:37:07.532586+0000","last_fullsized":"2024-09-10T09:37:07.532586+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:19:48.932596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.524402+0000","last_change":"2024-09-10T09:35:28.576244+0000","last_active":"2024-09-10T09:37:07.524402+0000","last_peered":"2024-09-10T09:37:07.524402+0000","last_clean":"2024-09-10T09:37:07.524402+0000","last_became_active":"2024-09-10T09:35:28.571801+0000","last_became_peered":"2024-09-10T09:35:28.571801+0000","last_unstale":"2024-09-10T09:37:07.524402+0000","last_undegraded":"2024-09-10T09:37:07.524402+0000","last_fullsized":"2024-09-10T09:37:07.524402+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:11:21.742220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"94'6","reported_seq":51,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.568700+0000","last_change":"2024-09-10T09:35:25.537118+0000","last_active":"2024-09-10T09:37:07.568700+0000","last_peered":"2024-09-10T09:37:07.568700+0000","last_clean":"2024-09-10T09:37:07.568700+0000","last_became_active":"2024-09-10T09:35:25.536718+0000","last_became_peered":"2024-09-10T09:35:25.536718+0000","last_unstale":"2024-09-10T09:37:07.568700+0000","last_undegraded":"2024-09-10T09:37:07.568700+0000","last_fullsized":"2024-09-10T09:37:07.568700+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:16:32.837476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":361,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.528720+0000","last_change":"2024-09-10T09:34:07.350825+0000","last_active":"2024-09-10T09:37:07.528720+0000","last_peered":"2024-09-10T09:37:07.528720+0000","last_clean":"2024-09-10T09:37:07.528720+0000","last_became_active":"2024-09-10T09:34:07.350325+0000","last_became_peered":"2024-09-10T09:34:07.350325+0000","last_unstale":"2024-09-10T09:37:07.528720+0000","last_undegraded":"2024-09-10T09:37:07.528720+0000","last_fullsized":"2024-09-10T09:37:07.528720+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:30:26.408262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:30:26.408262+0000","last_clean_scrub_stamp":"2024-09-10T09:30:26.408262+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:58:04.920443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"94'1","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.529244+0000","last_change":"2024-09-10T09:35:36.632941+0000","last_active":"2024-09-10T09:37:07.529244+0000","last_peered":"2024-09-10T09:37:07.529244+0000","last_clean":"2024-09-10T09:37:07.529244+0000","last_became_active":"2024-09-10T09:35:36.632527+0000","last_became_peered":"2024-09-10T09:35:36.632527+0000","last_unstale":"2024-09-10T09:37:07.529244+0000","last_undegraded":"2024-09-10T09:37:07.529244+0000","last_fullsized":"2024-09-10T09:37:07.529244+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:33:21.733772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.529425+0000","last_change":"2024-09-10T09:35:28.572010+0000","last_active":"2024-09-10T09:37:07.529425+0000","last_peered":"2024-09-10T09:37:07.529425+0000","last_clean":"2024-09-10T09:37:07.529425+0000","last_became_active":"2024-09-10T09:35:28.571623+0000","last_became_peered":"2024-09-10T09:35:28.571623+0000","last_unstale":"2024-09-10T09:37:07.529425+0000","last_undegraded":"2024-09-10T09:37:07.529425+0000","last_fullsized":"2024-09-10T09:37:07.529425+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:00:49.328663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"94'5","reported_seq":112,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:14.577199+0000","last_change":"2024-09-10T09:35:25.990328+0000","last_active":"2024-09-10T09:37:14.577199+0000","last_peered":"2024-09-10T09:37:14.577199+0000","last_clean":"2024-09-10T09:37:14.577199+0000","last_became_active":"2024-09-10T09:35:25.989946+0000","last_became_peered":"2024-09-10T09:35:25.989946+0000","last_unstale":"2024-09-10T09:37:14.577199+0000","last_undegraded":"2024-09-10T09:37:14.577199+0000","last_fullsized":"2024-09-10T09:37:14.577199+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:51:16.069913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":64,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":28,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.574415+0000","last_change":"2024-09-10T09:35:36.619013+0000","last_active":"2024-09-10T09:37:07.574415+0000","last_peered":"2024-09-10T09:37:07.574415+0000","last_clean":"2024-09-10T09:37:07.574415+0000","last_became_active":"2024-09-10T09:35:36.618506+0000","last_became_peered":"2024-09-10T09:35:36.618506+0000","last_unstale":"2024-09-10T09:37:07.574415+0000","last_undegraded":"2024-09-10T09:37:07.574415+0000","last_fullsized":"2024-09-10T09:37:07.574415+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:57:53.580569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":42,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.529232+0000","last_change":"2024-09-10T09:36:06.768573+0000","last_active":"2024-09-10T09:37:07.529232+0000","last_peered":"2024-09-10T09:37:07.529232+0000","last_clean":"2024-09-10T09:37:07.529232+0000","last_became_active":"2024-09-10T09:36:06.768228+0000","last_became_peered":"2024-09-10T09:36:06.768228+0000","last_unstale":"2024-09-10T09:37:07.529232+0000","last_undegraded":"2024-09-10T09:37:07.529232+0000","last_fullsized":"2024-09-10T09:37:07.529232+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:57:38.310713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"94'3","reported_seq":47,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.532881+0000","last_change":"2024-09-10T09:35:25.528795+0000","last_active":"2024-09-10T09:37:07.532881+0000","last_peered":"2024-09-10T09:37:07.532881+0000","last_clean":"2024-09-10T09:37:07.532881+0000","last_became_active":"2024-09-10T09:35:25.528475+0000","last_became_peered":"2024-09-10T09:35:25.528475+0000","last_unstale":"2024-09-10T09:37:07.532881+0000","last_undegraded":"2024-09-10T09:37:07.532881+0000","last_fullsized":"2024-09-10T09:37:07.532881+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:16:06.999186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":28,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.761719+0000","last_change":"2024-09-10T09:35:36.633109+0000","last_active":"2024-09-10T09:37:07.761719+0000","last_peered":"2024-09-10T09:37:07.761719+0000","last_clean":"2024-09-10T09:37:07.761719+0000","last_became_active":"2024-09-10T09:35:36.632682+0000","last_became_peered":"2024-09-10T09:35:36.632682+0000","last_unstale":"2024-09-10T09:37:07.761719+0000","last_undegraded":"2024-09-10T09:37:07.761719+0000","last_fullsized":"2024-09-10T09:37:07.761719+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:51:14.372649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.529393+0000","last_change":"2024-09-10T09:35:28.562103+0000","last_active":"2024-09-10T09:37:07.529393+0000","last_peered":"2024-09-10T09:37:07.529393+0000","last_clean":"2024-09-10T09:37:07.529393+0000","last_became_active":"2024-09-10T09:35:28.561765+0000","last_became_peered":"2024-09-10T09:35:28.561765+0000","last_unstale":"2024-09-10T09:37:07.529393+0000","last_undegraded":"2024-09-10T09:37:07.529393+0000","last_fullsized":"2024-09-10T09:37:07.529393+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:35:35.400569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"94'5","reported_seq":48,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.539182+0000","last_change":"2024-09-10T09:35:25.538536+0000","last_active":"2024-09-10T09:37:07.539182+0000","last_peered":"2024-09-10T09:37:07.539182+0000","last_clean":"2024-09-10T09:37:07.539182+0000","last_became_active":"2024-09-10T09:35:25.537944+0000","last_became_peered":"2024-09-10T09:35:25.537944+0000","last_unstale":"2024-09-10T09:37:07.539182+0000","last_undegraded":"2024-09-10T09:37:07.539182+0000","last_fullsized":"2024-09-10T09:37:07.539182+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:01:19.385381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":16,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.529871+0000","last_change":"2024-09-10T09:36:06.769534+0000","last_active":"2024-09-10T09:37:07.529871+0000","last_peered":"2024-09-10T09:37:07.529871+0000","last_clean":"2024-09-10T09:37:07.529871+0000","last_became_active":"2024-09-10T09:36:06.769015+0000","last_became_peered":"2024-09-10T09:36:06.769015+0000","last_unstale":"2024-09-10T09:37:07.529871+0000","last_undegraded":"2024-09-10T09:37:07.529871+0000","last_fullsized":"2024-09-10T09:37:07.529871+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:19:26.369737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7,1],"acting":[8,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.526505+0000","last_change":"2024-09-10T09:35:28.566735+0000","last_active":"2024-09-10T09:37:07.526505+0000","last_peered":"2024-09-10T09:37:07.526505+0000","last_clean":"2024-09-10T09:37:07.526505+0000","last_became_active":"2024-09-10T09:35:28.566400+0000","last_became_peered":"2024-09-10T09:35:28.566400+0000","last_unstale":"2024-09-10T09:37:07.526505+0000","last_undegraded":"2024-09-10T09:37:07.526505+0000","last_fullsized":"2024-09-10T09:37:07.526505+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:18:53.337816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"94'2","reported_seq":46,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.530519+0000","last_change":"2024-09-10T09:35:25.535783+0000","last_active":"2024-09-10T09:37:07.530519+0000","last_peered":"2024-09-10T09:37:07.530519+0000","last_clean":"2024-09-10T09:37:07.530519+0000","last_became_active":"2024-09-10T09:35:25.535411+0000","last_became_peered":"2024-09-10T09:35:25.535411+0000","last_unstale":"2024-09-10T09:37:07.530519+0000","last_undegraded":"2024-09-10T09:37:07.530519+0000","last_fullsized":"2024-09-10T09:37:07.530519+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:48:09.239083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":28,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.531150+0000","last_change":"2024-09-10T09:35:36.619380+0000","last_active":"2024-09-10T09:37:07.531150+0000","last_peered":"2024-09-10T09:37:07.531150+0000","last_clean":"2024-09-10T09:37:07.531150+0000","last_became_active":"2024-09-10T09:35:36.618953+0000","last_became_peered":"2024-09-10T09:35:36.618953+0000","last_unstale":"2024-09-10T09:37:07.531150+0000","last_undegraded":"2024-09-10T09:37:07.531150+0000","last_fullsized":"2024-09-10T09:37:07.531150+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:40:25.181408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.531331+0000","last_change":"2024-09-10T09:35:28.571396+0000","last_active":"2024-09-10T09:37:07.531331+0000","last_peered":"2024-09-10T09:37:07.531331+0000","last_clean":"2024-09-10T09:37:07.531331+0000","last_became_active":"2024-09-10T09:35:28.571063+0000","last_became_peered":"2024-09-10T09:35:28.571063+0000","last_unstale":"2024-09-10T09:37:07.531331+0000","last_undegraded":"2024-09-10T09:37:07.531331+0000","last_fullsized":"2024-09-10T09:37:07.531331+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:38:17.965133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"94'17","reported_seq":61,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.530004+0000","last_change":"2024-09-10T09:35:25.851773+0000","last_active":"2024-09-10T09:37:07.530004+0000","last_peered":"2024-09-10T09:37:07.530004+0000","last_clean":"2024-09-10T09:37:07.530004+0000","last_became_active":"2024-09-10T09:35:25.851384+0000","last_became_peered":"2024-09-10T09:35:25.851384+0000","last_unstale":"2024-09-10T09:37:07.530004+0000","last_undegraded":"2024-09-10T09:37:07.530004+0000","last_fullsized":"2024-09-10T09:37:07.530004+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:03:32.338643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":28,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.539161+0000","last_change":"2024-09-10T09:35:36.620971+0000","last_active":"2024-09-10T09:37:07.539161+0000","last_peered":"2024-09-10T09:37:07.539161+0000","last_clean":"2024-09-10T09:37:07.539161+0000","last_became_active":"2024-09-10T09:35:36.620343+0000","last_became_peered":"2024-09-10T09:35:36.620343+0000","last_unstale":"2024-09-10T09:37:07.539161+0000","last_undegraded":"2024-09-10T09:37:07.539161+0000","last_fullsized":"2024-09-10T09:37:07.539161+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:17:19.517882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.528475+0000","last_change":"2024-09-10T09:35:28.565643+0000","last_active":"2024-09-10T09:37:07.528475+0000","last_peered":"2024-09-10T09:37:07.528475+0000","last_clean":"2024-09-10T09:37:07.528475+0000","last_became_active":"2024-09-10T09:35:28.564217+0000","last_became_peered":"2024-09-10T09:35:28.564217+0000","last_unstale":"2024-09-10T09:37:07.528475+0000","last_undegraded":"2024-09-10T09:37:07.528475+0000","last_fullsized":"2024-09-10T09:37:07.528475+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:53:34.562690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"94'5","reported_seq":114,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:17.582212+0000","last_change":"2024-09-10T09:35:25.534167+0000","last_active":"2024-09-10T09:37:17.582212+0000","last_peered":"2024-09-10T09:37:17.582212+0000","last_clean":"2024-09-10T09:37:17.582212+0000","last_became_active":"2024-09-10T09:35:25.533719+0000","last_became_peered":"2024-09-10T09:35:25.533719+0000","last_unstale":"2024-09-10T09:37:17.582212+0000","last_undegraded":"2024-09-10T09:37:17.582212+0000","last_fullsized":"2024-09-10T09:37:17.582212+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:00:03.537355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":68,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":28,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.527665+0000","last_change":"2024-09-10T09:35:36.628716+0000","last_active":"2024-09-10T09:37:07.527665+0000","last_peered":"2024-09-10T09:37:07.527665+0000","last_clean":"2024-09-10T09:37:07.527665+0000","last_became_active":"2024-09-10T09:35:36.628085+0000","last_became_peered":"2024-09-10T09:35:36.628085+0000","last_unstale":"2024-09-10T09:37:07.527665+0000","last_undegraded":"2024-09-10T09:37:07.527665+0000","last_fullsized":"2024-09-10T09:37:07.527665+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:45:01.279403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.528994+0000","last_change":"2024-09-10T09:35:28.566701+0000","last_active":"2024-09-10T09:37:07.528994+0000","last_peered":"2024-09-10T09:37:07.528994+0000","last_clean":"2024-09-10T09:37:07.528994+0000","last_became_active":"2024-09-10T09:35:28.566391+0000","last_became_peered":"2024-09-10T09:35:28.566391+0000","last_unstale":"2024-09-10T09:37:07.528994+0000","last_undegraded":"2024-09-10T09:37:07.528994+0000","last_fullsized":"2024-09-10T09:37:07.528994+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:57:59.918706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"94'5","reported_seq":49,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.531844+0000","last_change":"2024-09-10T09:35:25.532510+0000","last_active":"2024-09-10T09:37:07.531844+0000","last_peered":"2024-09-10T09:37:07.531844+0000","last_clean":"2024-09-10T09:37:07.531844+0000","last_became_active":"2024-09-10T09:35:25.532193+0000","last_became_peered":"2024-09-10T09:35:25.532193+0000","last_unstale":"2024-09-10T09:37:07.531844+0000","last_undegraded":"2024-09-10T09:37:07.531844+0000","last_fullsized":"2024-09-10T09:37:07.531844+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:17:24.685178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":28,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.526361+0000","last_change":"2024-09-10T09:35:36.630742+0000","last_active":"2024-09-10T09:37:07.526361+0000","last_peered":"2024-09-10T09:37:07.526361+0000","last_clean":"2024-09-10T09:37:07.526361+0000","last_became_active":"2024-09-10T09:35:36.630335+0000","last_became_peered":"2024-09-10T09:35:36.630335+0000","last_unstale":"2024-09-10T09:37:07.526361+0000","last_undegraded":"2024-09-10T09:37:07.526361+0000","last_fullsized":"2024-09-10T09:37:07.526361+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:29:12.029390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.538397+0000","last_change":"2024-09-10T09:35:28.581817+0000","last_active":"2024-09-10T09:37:07.538397+0000","last_peered":"2024-09-10T09:37:07.538397+0000","last_clean":"2024-09-10T09:37:07.538397+0000","last_became_active":"2024-09-10T09:35:28.580316+0000","last_became_peered":"2024-09-10T09:35:28.580316+0000","last_unstale":"2024-09-10T09:37:07.538397+0000","last_undegraded":"2024-09-10T09:37:07.538397+0000","last_fullsized":"2024-09-10T09:37:07.538397+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:04:08.249564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"94'9","reported_seq":54,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.528083+0000","last_change":"2024-09-10T09:35:25.537474+0000","last_active":"2024-09-10T09:37:07.528083+0000","last_peered":"2024-09-10T09:37:07.528083+0000","last_clean":"2024-09-10T09:37:07.528083+0000","last_became_active":"2024-09-10T09:35:25.537096+0000","last_became_peered":"2024-09-10T09:35:25.537096+0000","last_unstale":"2024-09-10T09:37:07.528083+0000","last_undegraded":"2024-09-10T09:37:07.528083+0000","last_fullsized":"2024-09-10T09:37:07.528083+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:13:06.008458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":28,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.532263+0000","last_change":"2024-09-10T09:35:36.622788+0000","last_active":"2024-09-10T09:37:07.532263+0000","last_peered":"2024-09-10T09:37:07.532263+0000","last_clean":"2024-09-10T09:37:07.532263+0000","last_became_active":"2024-09-10T09:35:36.622377+0000","last_became_peered":"2024-09-10T09:35:36.622377+0000","last_unstale":"2024-09-10T09:37:07.532263+0000","last_undegraded":"2024-09-10T09:37:07.532263+0000","last_fullsized":"2024-09-10T09:37:07.532263+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:29:56.680864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.530171+0000","last_change":"2024-09-10T09:35:28.564529+0000","last_active":"2024-09-10T09:37:07.530171+0000","last_peered":"2024-09-10T09:37:07.530171+0000","last_clean":"2024-09-10T09:37:07.530171+0000","last_became_active":"2024-09-10T09:35:28.564223+0000","last_became_peered":"2024-09-10T09:35:28.564223+0000","last_unstale":"2024-09-10T09:37:07.530171+0000","last_undegraded":"2024-09-10T09:37:07.530171+0000","last_fullsized":"2024-09-10T09:37:07.530171+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:21:24.296578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"94'35","reported_seq":207,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:14.577526+0000","last_change":"2024-09-10T09:35:25.534331+0000","last_active":"2024-09-10T09:37:14.577526+0000","last_peered":"2024-09-10T09:37:14.577526+0000","last_clean":"2024-09-10T09:37:14.577526+0000","last_became_active":"2024-09-10T09:35:25.533985+0000","last_became_peered":"2024-09-10T09:35:25.533985+0000","last_unstale":"2024-09-10T09:37:14.577526+0000","last_undegraded":"2024-09-10T09:37:14.577526+0000","last_fullsized":"2024-09-10T09:37:14.577526+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:14:37.050650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62931,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":129,"num_read_kb":126,"num_write":37,"num_write_kb":76,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":28,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.527877+0000","last_change":"2024-09-10T09:35:36.632835+0000","last_active":"2024-09-10T09:37:07.527877+0000","last_peered":"2024-09-10T09:37:07.527877+0000","last_clean":"2024-09-10T09:37:07.527877+0000","last_became_active":"2024-09-10T09:35:36.630962+0000","last_became_peered":"2024-09-10T09:35:36.630962+0000","last_unstale":"2024-09-10T09:37:07.527877+0000","last_undegraded":"2024-09-10T09:37:07.527877+0000","last_fullsized":"2024-09-10T09:37:07.527877+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:59:27.613721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.574757+0000","last_change":"2024-09-10T09:35:28.567292+0000","last_active":"2024-09-10T09:37:07.574757+0000","last_peered":"2024-09-10T09:37:07.574757+0000","last_clean":"2024-09-10T09:37:07.574757+0000","last_became_active":"2024-09-10T09:35:28.566973+0000","last_became_peered":"2024-09-10T09:35:28.566973+0000","last_unstale":"2024-09-10T09:37:07.574757+0000","last_undegraded":"2024-09-10T09:37:07.574757+0000","last_fullsized":"2024-09-10T09:37:07.574757+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:05:37.251933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"94'5","reported_seq":49,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.532712+0000","last_change":"2024-09-10T09:35:25.852171+0000","last_active":"2024-09-10T09:37:07.532712+0000","last_peered":"2024-09-10T09:37:07.532712+0000","last_clean":"2024-09-10T09:37:07.532712+0000","last_became_active":"2024-09-10T09:35:25.851580+0000","last_became_peered":"2024-09-10T09:35:25.851580+0000","last_unstale":"2024-09-10T09:37:07.532712+0000","last_undegraded":"2024-09-10T09:37:07.532712+0000","last_fullsized":"2024-09-10T09:37:07.532712+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:03.263152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":28,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.568288+0000","last_change":"2024-09-10T09:35:36.629212+0000","last_active":"2024-09-10T09:37:07.568288+0000","last_peered":"2024-09-10T09:37:07.568288+0000","last_clean":"2024-09-10T09:37:07.568288+0000","last_became_active":"2024-09-10T09:35:36.628753+0000","last_became_peered":"2024-09-10T09:35:36.628753+0000","last_unstale":"2024-09-10T09:37:07.568288+0000","last_undegraded":"2024-09-10T09:37:07.568288+0000","last_fullsized":"2024-09-10T09:37:07.568288+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:54:45.311361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.761606+0000","last_change":"2024-09-10T09:35:28.572497+0000","last_active":"2024-09-10T09:37:07.761606+0000","last_peered":"2024-09-10T09:37:07.761606+0000","last_clean":"2024-09-10T09:37:07.761606+0000","last_became_active":"2024-09-10T09:35:28.569238+0000","last_became_peered":"2024-09-10T09:35:28.569238+0000","last_unstale":"2024-09-10T09:37:07.761606+0000","last_undegraded":"2024-09-10T09:37:07.761606+0000","last_fullsized":"2024-09-10T09:37:07.761606+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:38:47.189209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"94'4","reported_seq":46,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.529513+0000","last_change":"2024-09-10T09:35:25.536369+0000","last_active":"2024-09-10T09:37:07.529513+0000","last_peered":"2024-09-10T09:37:07.529513+0000","last_clean":"2024-09-10T09:37:07.529513+0000","last_became_active":"2024-09-10T09:35:25.535607+0000","last_became_peered":"2024-09-10T09:35:25.535607+0000","last_unstale":"2024-09-10T09:37:07.529513+0000","last_undegraded":"2024-09-10T09:37:07.529513+0000","last_fullsized":"2024-09-10T09:37:07.529513+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:49:18.621934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":28,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.526991+0000","last_change":"2024-09-10T09:35:36.627977+0000","last_active":"2024-09-10T09:37:07.526991+0000","last_peered":"2024-09-10T09:37:07.526991+0000","last_clean":"2024-09-10T09:37:07.526991+0000","last_became_active":"2024-09-10T09:35:36.626942+0000","last_became_peered":"2024-09-10T09:35:36.626942+0000","last_unstale":"2024-09-10T09:37:07.526991+0000","last_undegraded":"2024-09-10T09:37:07.526991+0000","last_fullsized":"2024-09-10T09:37:07.526991+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:37:18.209424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.575017+0000","last_change":"2024-09-10T09:35:28.562670+0000","last_active":"2024-09-10T09:37:07.575017+0000","last_peered":"2024-09-10T09:37:07.575017+0000","last_clean":"2024-09-10T09:37:07.575017+0000","last_became_active":"2024-09-10T09:35:28.562028+0000","last_became_peered":"2024-09-10T09:35:28.562028+0000","last_unstale":"2024-09-10T09:37:07.575017+0000","last_undegraded":"2024-09-10T09:37:07.575017+0000","last_fullsized":"2024-09-10T09:37:07.575017+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:37:15.176050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"94'4","reported_seq":168,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:14.576692+0000","last_change":"2024-09-10T09:35:25.529861+0000","last_active":"2024-09-10T09:37:14.576692+0000","last_peered":"2024-09-10T09:37:14.576692+0000","last_clean":"2024-09-10T09:37:14.576692+0000","last_became_active":"2024-09-10T09:35:25.529475+0000","last_became_peered":"2024-09-10T09:35:25.529475+0000","last_unstale":"2024-09-10T09:37:14.576692+0000","last_undegraded":"2024-09-10T09:37:14.576692+0000","last_fullsized":"2024-09-10T09:37:14.576692+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:44:27.086509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":121,"num_read_kb":121,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":28,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.524255+0000","last_change":"2024-09-10T09:35:36.615358+0000","last_active":"2024-09-10T09:37:07.524255+0000","last_peered":"2024-09-10T09:37:07.524255+0000","last_clean":"2024-09-10T09:37:07.524255+0000","last_became_active":"2024-09-10T09:35:36.614901+0000","last_became_peered":"2024-09-10T09:35:36.614901+0000","last_unstale":"2024-09-10T09:37:07.524255+0000","last_undegraded":"2024-09-10T09:37:07.524255+0000","last_fullsized":"2024-09-10T09:37:07.524255+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:56:54.248266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":28,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.761229+0000","last_change":"2024-09-10T09:35:36.629021+0000","last_active":"2024-09-10T09:37:07.761229+0000","last_peered":"2024-09-10T09:37:07.761229+0000","last_clean":"2024-09-10T09:37:07.761229+0000","last_became_active":"2024-09-10T09:35:36.628159+0000","last_became_peered":"2024-09-10T09:35:36.628159+0000","last_unstale":"2024-09-10T09:37:07.761229+0000","last_undegraded":"2024-09-10T09:37:07.761229+0000","last_fullsized":"2024-09-10T09:37:07.761229+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:50:48.615236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.527161+0000","last_change":"2024-09-10T09:35:28.579333+0000","last_active":"2024-09-10T09:37:07.527161+0000","last_peered":"2024-09-10T09:37:07.527161+0000","last_clean":"2024-09-10T09:37:07.527161+0000","last_became_active":"2024-09-10T09:35:28.578440+0000","last_became_peered":"2024-09-10T09:35:28.578440+0000","last_unstale":"2024-09-10T09:37:07.527161+0000","last_undegraded":"2024-09-10T09:37:07.527161+0000","last_fullsized":"2024-09-10T09:37:07.527161+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:07:28.701938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"94'3","reported_seq":47,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.531014+0000","last_change":"2024-09-10T09:35:25.530857+0000","last_active":"2024-09-10T09:37:07.531014+0000","last_peered":"2024-09-10T09:37:07.531014+0000","last_clean":"2024-09-10T09:37:07.531014+0000","last_became_active":"2024-09-10T09:35:25.530492+0000","last_became_peered":"2024-09-10T09:35:25.530492+0000","last_unstale":"2024-09-10T09:37:07.531014+0000","last_undegraded":"2024-09-10T09:37:07.531014+0000","last_fullsized":"2024-09-10T09:37:07.531014+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:07:21.726598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":35,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.526386+0000","last_change":"2024-09-10T09:36:06.770408+0000","last_active":"2024-09-10T09:37:07.526386+0000","last_peered":"2024-09-10T09:37:07.526386+0000","last_clean":"2024-09-10T09:37:07.526386+0000","last_became_active":"2024-09-10T09:36:06.769915+0000","last_became_peered":"2024-09-10T09:36:06.769915+0000","last_unstale":"2024-09-10T09:37:07.526386+0000","last_undegraded":"2024-09-10T09:37:07.526386+0000","last_fullsized":"2024-09-10T09:37:07.526386+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:56:09.795537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8,0],"acting":[11,4,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.524227+0000","last_change":"2024-09-10T09:35:28.570804+0000","last_active":"2024-09-10T09:37:07.524227+0000","last_peered":"2024-09-10T09:37:07.524227+0000","last_clean":"2024-09-10T09:37:07.524227+0000","last_became_active":"2024-09-10T09:35:28.570490+0000","last_became_peered":"2024-09-10T09:35:28.570490+0000","last_unstale":"2024-09-10T09:37:07.524227+0000","last_undegraded":"2024-09-10T09:37:07.524227+0000","last_fullsized":"2024-09-10T09:37:07.524227+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:13:38.999625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"94'3","reported_seq":46,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.538391+0000","last_change":"2024-09-10T09:35:25.536347+0000","last_active":"2024-09-10T09:37:07.538391+0000","last_peered":"2024-09-10T09:37:07.538391+0000","last_clean":"2024-09-10T09:37:07.538391+0000","last_became_active":"2024-09-10T09:35:25.535964+0000","last_became_peered":"2024-09-10T09:35:25.535964+0000","last_unstale":"2024-09-10T09:37:07.538391+0000","last_undegraded":"2024-09-10T09:37:07.538391+0000","last_fullsized":"2024-09-10T09:37:07.538391+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:36:49.035666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"94'7","reported_seq":50,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.525762+0000","last_change":"2024-09-10T09:35:25.532148+0000","last_active":"2024-09-10T09:37:07.525762+0000","last_peered":"2024-09-10T09:37:07.525762+0000","last_clean":"2024-09-10T09:37:07.525762+0000","last_became_active":"2024-09-10T09:35:25.531781+0000","last_became_peered":"2024-09-10T09:35:25.531781+0000","last_unstale":"2024-09-10T09:37:07.525762+0000","last_undegraded":"2024-09-10T09:37:07.525762+0000","last_fullsized":"2024-09-10T09:37:07.525762+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:52:46.740342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.532588+0000","last_change":"2024-09-10T09:35:28.578336+0000","last_active":"2024-09-10T09:37:07.532588+0000","last_peered":"2024-09-10T09:37:07.532588+0000","last_clean":"2024-09-10T09:37:07.532588+0000","last_became_active":"2024-09-10T09:35:28.577899+0000","last_became_peered":"2024-09-10T09:35:28.577899+0000","last_unstale":"2024-09-10T09:37:07.532588+0000","last_undegraded":"2024-09-10T09:37:07.532588+0000","last_fullsized":"2024-09-10T09:37:07.532588+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:13:04.478195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":28,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.528073+0000","last_change":"2024-09-10T09:35:36.630918+0000","last_active":"2024-09-10T09:37:07.528073+0000","last_peered":"2024-09-10T09:37:07.528073+0000","last_clean":"2024-09-10T09:37:07.528073+0000","last_became_active":"2024-09-10T09:35:36.630308+0000","last_became_peered":"2024-09-10T09:35:36.630308+0000","last_unstale":"2024-09-10T09:37:07.528073+0000","last_undegraded":"2024-09-10T09:37:07.528073+0000","last_fullsized":"2024-09-10T09:37:07.528073+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:25:44.954970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"94'6","reported_seq":124,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:13.577265+0000","last_change":"2024-09-10T09:37:07.574386+0000","last_active":"2024-09-10T09:37:13.577265+0000","last_peered":"2024-09-10T09:37:13.577265+0000","last_clean":"2024-09-10T09:37:13.577265+0000","last_became_active":"2024-09-10T09:37:07.569370+0000","last_became_peered":"2024-09-10T09:37:07.569370+0000","last_unstale":"2024-09-10T09:37:13.577265+0000","last_undegraded":"2024-09-10T09:37:13.577265+0000","last_fullsized":"2024-09-10T09:37:13.577265+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:07:23.014789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1015,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":58,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":1015,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.530083+0000","last_change":"2024-09-10T09:35:28.567915+0000","last_active":"2024-09-10T09:37:07.530083+0000","last_peered":"2024-09-10T09:37:07.530083+0000","last_clean":"2024-09-10T09:37:07.530083+0000","last_became_active":"2024-09-10T09:35:28.567416+0000","last_became_peered":"2024-09-10T09:35:28.567416+0000","last_unstale":"2024-09-10T09:37:07.530083+0000","last_undegraded":"2024-09-10T09:37:07.530083+0000","last_fullsized":"2024-09-10T09:37:07.530083+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:07:25.704080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":28,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.528636+0000","last_change":"2024-09-10T09:35:36.615386+0000","last_active":"2024-09-10T09:37:07.528636+0000","last_peered":"2024-09-10T09:37:07.528636+0000","last_clean":"2024-09-10T09:37:07.528636+0000","last_became_active":"2024-09-10T09:35:36.614967+0000","last_became_peered":"2024-09-10T09:35:36.614967+0000","last_unstale":"2024-09-10T09:37:07.528636+0000","last_undegraded":"2024-09-10T09:37:07.528636+0000","last_fullsized":"2024-09-10T09:37:07.528636+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:47.620729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.526231+0000","last_change":"2024-09-10T09:35:28.564411+0000","last_active":"2024-09-10T09:37:07.526231+0000","last_peered":"2024-09-10T09:37:07.526231+0000","last_clean":"2024-09-10T09:37:07.526231+0000","last_became_active":"2024-09-10T09:35:28.564034+0000","last_became_peered":"2024-09-10T09:35:28.564034+0000","last_unstale":"2024-09-10T09:37:07.526231+0000","last_undegraded":"2024-09-10T09:37:07.526231+0000","last_fullsized":"2024-09-10T09:37:07.526231+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:38:17.687991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.527821+0000","last_change":"2024-09-10T09:35:28.564619+0000","last_active":"2024-09-10T09:37:07.527821+0000","last_peered":"2024-09-10T09:37:07.527821+0000","last_clean":"2024-09-10T09:37:07.527821+0000","last_became_active":"2024-09-10T09:35:28.563890+0000","last_became_peered":"2024-09-10T09:35:28.563890+0000","last_unstale":"2024-09-10T09:37:07.527821+0000","last_undegraded":"2024-09-10T09:37:07.527821+0000","last_fullsized":"2024-09-10T09:37:07.527821+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:51:42.481911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.527812+0000","last_change":"2024-09-10T09:35:28.558909+0000","last_active":"2024-09-10T09:37:07.527812+0000","last_peered":"2024-09-10T09:37:07.527812+0000","last_clean":"2024-09-10T09:37:07.527812+0000","last_became_active":"2024-09-10T09:35:28.558604+0000","last_became_peered":"2024-09-10T09:35:28.558604+0000","last_unstale":"2024-09-10T09:37:07.527812+0000","last_undegraded":"2024-09-10T09:37:07.527812+0000","last_fullsized":"2024-09-10T09:37:07.527812+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:11.928446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.761769+0000","last_change":"2024-09-10T09:35:28.574459+0000","last_active":"2024-09-10T09:37:07.761769+0000","last_peered":"2024-09-10T09:37:07.761769+0000","last_clean":"2024-09-10T09:37:07.761769+0000","last_became_active":"2024-09-10T09:35:28.574154+0000","last_became_peered":"2024-09-10T09:35:28.574154+0000","last_unstale":"2024-09-10T09:37:07.761769+0000","last_undegraded":"2024-09-10T09:37:07.761769+0000","last_fullsized":"2024-09-10T09:37:07.761769+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:59:09.650536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.538880+0000","last_change":"2024-09-10T09:35:28.576999+0000","last_active":"2024-09-10T09:37:07.538880+0000","last_peered":"2024-09-10T09:37:07.538880+0000","last_clean":"2024-09-10T09:37:07.538880+0000","last_became_active":"2024-09-10T09:35:28.575616+0000","last_became_peered":"2024-09-10T09:35:28.575616+0000","last_unstale":"2024-09-10T09:37:07.538880+0000","last_undegraded":"2024-09-10T09:37:07.538880+0000","last_fullsized":"2024-09-10T09:37:07.538880+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:18:08.771760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.527051+0000","last_change":"2024-09-10T09:35:28.564846+0000","last_active":"2024-09-10T09:37:07.527051+0000","last_peered":"2024-09-10T09:37:07.527051+0000","last_clean":"2024-09-10T09:37:07.527051+0000","last_became_active":"2024-09-10T09:35:28.564498+0000","last_became_peered":"2024-09-10T09:35:28.564498+0000","last_unstale":"2024-09-10T09:37:07.527051+0000","last_undegraded":"2024-09-10T09:37:07.527051+0000","last_fullsized":"2024-09-10T09:37:07.527051+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:33:43.041730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.528287+0000","last_change":"2024-09-10T09:35:28.558093+0000","last_active":"2024-09-10T09:37:07.528287+0000","last_peered":"2024-09-10T09:37:07.528287+0000","last_clean":"2024-09-10T09:37:07.528287+0000","last_became_active":"2024-09-10T09:35:28.557266+0000","last_became_peered":"2024-09-10T09:35:28.557266+0000","last_unstale":"2024-09-10T09:37:07.528287+0000","last_undegraded":"2024-09-10T09:37:07.528287+0000","last_fullsized":"2024-09-10T09:37:07.528287+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:51:30.946065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":44,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.524860+0000","last_change":"2024-09-10T09:36:06.770298+0000","last_active":"2024-09-10T09:37:07.524860+0000","last_peered":"2024-09-10T09:37:07.524860+0000","last_clean":"2024-09-10T09:37:07.524860+0000","last_became_active":"2024-09-10T09:36:06.769848+0000","last_became_peered":"2024-09-10T09:36:06.769848+0000","last_unstale":"2024-09-10T09:37:07.524860+0000","last_undegraded":"2024-09-10T09:37:07.524860+0000","last_fullsized":"2024-09-10T09:37:07.524860+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:00:30.470887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,8,2],"acting":[3,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.530788+0000","last_change":"2024-09-10T09:35:28.578158+0000","last_active":"2024-09-10T09:37:07.530788+0000","last_peered":"2024-09-10T09:37:07.530788+0000","last_clean":"2024-09-10T09:37:07.530788+0000","last_became_active":"2024-09-10T09:35:28.577824+0000","last_became_peered":"2024-09-10T09:35:28.577824+0000","last_unstale":"2024-09-10T09:37:07.530788+0000","last_undegraded":"2024-09-10T09:37:07.530788+0000","last_fullsized":"2024-09-10T09:37:07.530788+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:40:27.231164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.524324+0000","last_change":"2024-09-10T09:35:28.562670+0000","last_active":"2024-09-10T09:37:07.524324+0000","last_peered":"2024-09-10T09:37:07.524324+0000","last_clean":"2024-09-10T09:37:07.524324+0000","last_became_active":"2024-09-10T09:35:28.562311+0000","last_became_peered":"2024-09-10T09:35:28.562311+0000","last_unstale":"2024-09-10T09:37:07.524324+0000","last_undegraded":"2024-09-10T09:37:07.524324+0000","last_fullsized":"2024-09-10T09:37:07.524324+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:07:11.716142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.574601+0000","last_change":"2024-09-10T09:35:28.561124+0000","last_active":"2024-09-10T09:37:07.574601+0000","last_peered":"2024-09-10T09:37:07.574601+0000","last_clean":"2024-09-10T09:37:07.574601+0000","last_became_active":"2024-09-10T09:35:28.560777+0000","last_became_peered":"2024-09-10T09:35:28.560777+0000","last_unstale":"2024-09-10T09:37:07.574601+0000","last_undegraded":"2024-09-10T09:37:07.574601+0000","last_fullsized":"2024-09-10T09:37:07.574601+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:47:04.837596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.538921+0000","last_change":"2024-09-10T09:35:28.576790+0000","last_active":"2024-09-10T09:37:07.538921+0000","last_peered":"2024-09-10T09:37:07.538921+0000","last_clean":"2024-09-10T09:37:07.538921+0000","last_became_active":"2024-09-10T09:35:28.576464+0000","last_became_peered":"2024-09-10T09:35:28.576464+0000","last_unstale":"2024-09-10T09:37:07.538921+0000","last_undegraded":"2024-09-10T09:37:07.538921+0000","last_fullsized":"2024-09-10T09:37:07.538921+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:40:53.517577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.526717+0000","last_change":"2024-09-10T09:35:28.569935+0000","last_active":"2024-09-10T09:37:07.526717+0000","last_peered":"2024-09-10T09:37:07.526717+0000","last_clean":"2024-09-10T09:37:07.526717+0000","last_became_active":"2024-09-10T09:35:28.569539+0000","last_became_peered":"2024-09-10T09:35:28.569539+0000","last_unstale":"2024-09-10T09:37:07.526717+0000","last_undegraded":"2024-09-10T09:37:07.526717+0000","last_fullsized":"2024-09-10T09:37:07.526717+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:40:45.016269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.528743+0000","last_change":"2024-09-10T09:35:28.571619+0000","last_active":"2024-09-10T09:37:07.528743+0000","last_peered":"2024-09-10T09:37:07.528743+0000","last_clean":"2024-09-10T09:37:07.528743+0000","last_became_active":"2024-09-10T09:35:28.568591+0000","last_became_peered":"2024-09-10T09:35:28.568591+0000","last_unstale":"2024-09-10T09:37:07.528743+0000","last_undegraded":"2024-09-10T09:37:07.528743+0000","last_fullsized":"2024-09-10T09:37:07.528743+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:07:49.959712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.527811+0000","last_change":"2024-09-10T09:35:28.573336+0000","last_active":"2024-09-10T09:37:07.527811+0000","last_peered":"2024-09-10T09:37:07.527811+0000","last_clean":"2024-09-10T09:37:07.527811+0000","last_became_active":"2024-09-10T09:35:28.572991+0000","last_became_peered":"2024-09-10T09:35:28.572991+0000","last_unstale":"2024-09-10T09:37:07.527811+0000","last_undegraded":"2024-09-10T09:37:07.527811+0000","last_fullsized":"2024-09-10T09:37:07.527811+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:30:02.905869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.526885+0000","last_change":"2024-09-10T09:35:28.571945+0000","last_active":"2024-09-10T09:37:07.526885+0000","last_peered":"2024-09-10T09:37:07.526885+0000","last_clean":"2024-09-10T09:37:07.526885+0000","last_became_active":"2024-09-10T09:35:28.571546+0000","last_became_peered":"2024-09-10T09:35:28.571546+0000","last_unstale":"2024-09-10T09:37:07.526885+0000","last_undegraded":"2024-09-10T09:37:07.526885+0000","last_fullsized":"2024-09-10T09:37:07.526885+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:36:47.946864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.574295+0000","last_change":"2024-09-10T09:35:28.562712+0000","last_active":"2024-09-10T09:37:07.574295+0000","last_peered":"2024-09-10T09:37:07.574295+0000","last_clean":"2024-09-10T09:37:07.574295+0000","last_became_active":"2024-09-10T09:35:28.562396+0000","last_became_peered":"2024-09-10T09:35:28.562396+0000","last_unstale":"2024-09-10T09:37:07.574295+0000","last_undegraded":"2024-09-10T09:37:07.574295+0000","last_fullsized":"2024-09-10T09:37:07.574295+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:25:47.419162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.761275+0000","last_change":"2024-09-10T09:35:28.566354+0000","last_active":"2024-09-10T09:37:07.761275+0000","last_peered":"2024-09-10T09:37:07.761275+0000","last_clean":"2024-09-10T09:37:07.761275+0000","last_became_active":"2024-09-10T09:35:28.565248+0000","last_became_peered":"2024-09-10T09:35:28.565248+0000","last_unstale":"2024-09-10T09:37:07.761275+0000","last_undegraded":"2024-09-10T09:37:07.761275+0000","last_fullsized":"2024-09-10T09:37:07.761275+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:48:46.741010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.761172+0000","last_change":"2024-09-10T09:35:28.563331+0000","last_active":"2024-09-10T09:37:07.761172+0000","last_peered":"2024-09-10T09:37:07.761172+0000","last_clean":"2024-09-10T09:37:07.761172+0000","last_became_active":"2024-09-10T09:35:28.563020+0000","last_became_peered":"2024-09-10T09:35:28.563020+0000","last_unstale":"2024-09-10T09:37:07.761172+0000","last_undegraded":"2024-09-10T09:37:07.761172+0000","last_fullsized":"2024-09-10T09:37:07.761172+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:27.653551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.538817+0000","last_change":"2024-09-10T09:35:28.566687+0000","last_active":"2024-09-10T09:37:07.538817+0000","last_peered":"2024-09-10T09:37:07.538817+0000","last_clean":"2024-09-10T09:37:07.538817+0000","last_became_active":"2024-09-10T09:35:28.566253+0000","last_became_peered":"2024-09-10T09:35:28.566253+0000","last_unstale":"2024-09-10T09:37:07.538817+0000","last_undegraded":"2024-09-10T09:37:07.538817+0000","last_fullsized":"2024-09-10T09:37:07.538817+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:00:15.709725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.568759+0000","last_change":"2024-09-10T09:35:28.562251+0000","last_active":"2024-09-10T09:37:07.568759+0000","last_peered":"2024-09-10T09:37:07.568759+0000","last_clean":"2024-09-10T09:37:07.568759+0000","last_became_active":"2024-09-10T09:35:28.561933+0000","last_became_peered":"2024-09-10T09:35:28.561933+0000","last_unstale":"2024-09-10T09:37:07.568759+0000","last_undegraded":"2024-09-10T09:37:07.568759+0000","last_fullsized":"2024-09-10T09:37:07.568759+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:30:41.447954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.529401+0000","last_change":"2024-09-10T09:35:28.572164+0000","last_active":"2024-09-10T09:37:07.529401+0000","last_peered":"2024-09-10T09:37:07.529401+0000","last_clean":"2024-09-10T09:37:07.529401+0000","last_became_active":"2024-09-10T09:35:28.571825+0000","last_became_peered":"2024-09-10T09:35:28.571825+0000","last_unstale":"2024-09-10T09:37:07.529401+0000","last_undegraded":"2024-09-10T09:37:07.529401+0000","last_fullsized":"2024-09-10T09:37:07.529401+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:59:50.967269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.526142+0000","last_change":"2024-09-10T09:35:28.568098+0000","last_active":"2024-09-10T09:37:07.526142+0000","last_peered":"2024-09-10T09:37:07.526142+0000","last_clean":"2024-09-10T09:37:07.526142+0000","last_became_active":"2024-09-10T09:35:28.567765+0000","last_became_peered":"2024-09-10T09:35:28.567765+0000","last_unstale":"2024-09-10T09:37:07.526142+0000","last_undegraded":"2024-09-10T09:37:07.526142+0000","last_fullsized":"2024-09-10T09:37:07.526142+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:55:57.260320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.575147+0000","last_change":"2024-09-10T09:35:28.564123+0000","last_active":"2024-09-10T09:37:07.575147+0000","last_peered":"2024-09-10T09:37:07.575147+0000","last_clean":"2024-09-10T09:37:07.575147+0000","last_became_active":"2024-09-10T09:35:28.563814+0000","last_became_peered":"2024-09-10T09:35:28.563814+0000","last_unstale":"2024-09-10T09:37:07.575147+0000","last_undegraded":"2024-09-10T09:37:07.575147+0000","last_fullsized":"2024-09-10T09:37:07.575147+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:14:17.912087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.524203+0000","last_change":"2024-09-10T09:35:28.568601+0000","last_active":"2024-09-10T09:37:07.524203+0000","last_peered":"2024-09-10T09:37:07.524203+0000","last_clean":"2024-09-10T09:37:07.524203+0000","last_became_active":"2024-09-10T09:35:28.568303+0000","last_became_peered":"2024-09-10T09:35:28.568303+0000","last_unstale":"2024-09-10T09:37:07.524203+0000","last_undegraded":"2024-09-10T09:37:07.524203+0000","last_fullsized":"2024-09-10T09:37:07.524203+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:30:01.016420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":42,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.539577+0000","last_change":"2024-09-10T09:37:07.539577+0000","last_active":"2024-09-10T09:37:07.539577+0000","last_peered":"2024-09-10T09:37:07.539577+0000","last_clean":"2024-09-10T09:37:07.539577+0000","last_became_active":"2024-09-10T09:37:07.539206+0000","last_became_peered":"2024-09-10T09:37:07.539206+0000","last_unstale":"2024-09-10T09:37:07.539577+0000","last_undegraded":"2024-09-10T09:37:07.539577+0000","last_fullsized":"2024-09-10T09:37:07.539577+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:56:06.568226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.525650+0000","last_change":"2024-09-10T09:35:28.569225+0000","last_active":"2024-09-10T09:37:07.525650+0000","last_peered":"2024-09-10T09:37:07.525650+0000","last_clean":"2024-09-10T09:37:07.525650+0000","last_became_active":"2024-09-10T09:35:28.568891+0000","last_became_peered":"2024-09-10T09:35:28.568891+0000","last_unstale":"2024-09-10T09:37:07.525650+0000","last_undegraded":"2024-09-10T09:37:07.525650+0000","last_fullsized":"2024-09-10T09:37:07.525650+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:33:45.811781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.529728+0000","last_change":"2024-09-10T09:35:28.578362+0000","last_active":"2024-09-10T09:37:07.529728+0000","last_peered":"2024-09-10T09:37:07.529728+0000","last_clean":"2024-09-10T09:37:07.529728+0000","last_became_active":"2024-09-10T09:35:28.577867+0000","last_became_peered":"2024-09-10T09:35:28.577867+0000","last_unstale":"2024-09-10T09:37:07.529728+0000","last_undegraded":"2024-09-10T09:37:07.529728+0000","last_fullsized":"2024-09-10T09:37:07.529728+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:58:17.041663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.568770+0000","last_change":"2024-09-10T09:35:28.568183+0000","last_active":"2024-09-10T09:37:07.568770+0000","last_peered":"2024-09-10T09:37:07.568770+0000","last_clean":"2024-09-10T09:37:07.568770+0000","last_became_active":"2024-09-10T09:35:28.566775+0000","last_became_peered":"2024-09-10T09:35:28.566775+0000","last_unstale":"2024-09-10T09:37:07.568770+0000","last_undegraded":"2024-09-10T09:37:07.568770+0000","last_fullsized":"2024-09-10T09:37:07.568770+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:28:10.008889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.761305+0000","last_change":"2024-09-10T09:35:28.576122+0000","last_active":"2024-09-10T09:37:07.761305+0000","last_peered":"2024-09-10T09:37:07.761305+0000","last_clean":"2024-09-10T09:37:07.761305+0000","last_became_active":"2024-09-10T09:35:28.575831+0000","last_became_peered":"2024-09-10T09:35:28.575831+0000","last_unstale":"2024-09-10T09:37:07.761305+0000","last_undegraded":"2024-09-10T09:37:07.761305+0000","last_fullsized":"2024-09-10T09:37:07.761305+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:37:51.604658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.761353+0000","last_change":"2024-09-10T09:35:28.564441+0000","last_active":"2024-09-10T09:37:07.761353+0000","last_peered":"2024-09-10T09:37:07.761353+0000","last_clean":"2024-09-10T09:37:07.761353+0000","last_became_active":"2024-09-10T09:35:28.564095+0000","last_became_peered":"2024-09-10T09:35:28.564095+0000","last_unstale":"2024-09-10T09:37:07.761353+0000","last_undegraded":"2024-09-10T09:37:07.761353+0000","last_fullsized":"2024-09-10T09:37:07.761353+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:53:28.275361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":38,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.526738+0000","last_change":"2024-09-10T09:35:28.581921+0000","last_active":"2024-09-10T09:37:07.526738+0000","last_peered":"2024-09-10T09:37:07.526738+0000","last_clean":"2024-09-10T09:37:07.526738+0000","last_became_active":"2024-09-10T09:35:28.581573+0000","last_became_peered":"2024-09-10T09:35:28.581573+0000","last_unstale":"2024-09-10T09:37:07.526738+0000","last_undegraded":"2024-09-10T09:37:07.526738+0000","last_fullsized":"2024-09-10T09:37:07.526738+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:58:38.983934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":70139,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":649,"num_read_kb":404,"num_write":249,"num_write_kb":142,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":1015,"num_keys_recovered":7,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":577536,"data_stored":210417,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":214,"ondisk_log_size":214,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":285627,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":77,"seq":330712481828,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32236,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712916,"statfs":{"total":95995035648,"available":95962025984,"internally_reserved":0,"allocated":1069056,"data_stored":554290,"data_compressed":32456,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":70,"seq":300647710760,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28228,"kb_used_data":1196,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716924,"statfs":{"total":95995035648,"available":95966130176,"internally_reserved":0,"allocated":1224704,"data_stored":1208133,"data_compressed":68777,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":65,"seq":279172874287,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32260,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712892,"statfs":{"total":95995035648,"available":95962001408,"internally_reserved":0,"allocated":1093632,"data_stored":554220,"data_compressed":32456,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1599,"internal_metadata":31914433},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":57,"seq":244813135924,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32252,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712900,"statfs":{"total":95995035648,"available":95962009600,"internally_reserved":0,"allocated":1089536,"data_stored":554598,"data_compressed":32456,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":50,"seq":214748364857,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28148,"kb_used_data":1116,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717004,"statfs":{"total":95995035648,"available":95966212096,"internally_reserved":0,"allocated":1142784,"data_stored":1151490,"data_compressed":68777,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":45,"seq":193273528383,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36396,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708756,"statfs":{"total":95995035648,"available":95957766144,"internally_reserved":0,"allocated":1069056,"data_stored":550117,"data_compressed":32456,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790021,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28148,"kb_used_data":1124,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717004,"statfs":{"total":95995035648,"available":95966212096,"internally_reserved":0,"allocated":1150976,"data_stored":1146679,"data_compressed":68777,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953547,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32252,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712900,"statfs":{"total":95995035648,"available":95962009600,"internally_reserved":0,"allocated":1085440,"data_stored":554813,"data_compressed":32456,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182479,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28084,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717068,"statfs":{"total":95995035648,"available":95966277632,"internally_reserved":0,"allocated":1081344,"data_stored":549432,"data_compressed":32456,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411412,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28156,"kb_used_data":1124,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716996,"statfs":{"total":95995035648,"available":95966203904,"internally_reserved":0,"allocated":1150976,"data_stored":613283,"data_compressed":32456,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574939,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32304,"kb_used_data":1112,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712848,"statfs":{"total":95995035648,"available":95961956352,"internally_reserved":0,"allocated":1138688,"data_stored":614078,"data_compressed":32456,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738463,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36416,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708736,"statfs":{"total":95995035648,"available":95957745664,"internally_reserved":0,"allocated":1093632,"data_stored":551180,"data_compressed":32456,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34674,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34674,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34674,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36321,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36321,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36321,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36321,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36321,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":65076,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":64281,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1715,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3213,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1115,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":118784,"data_stored":64667,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T09:37:20.615 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-10T09:37:20.615 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-10T09:37:20.615 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-09-10T09:37:20.615 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph health --format=json 2024-09-10T09:37:20.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:37:20 smithi102 ceph-mon[35442]: pgmap v363: 129 pgs: 129 active+clean; 645 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 8 op/s; 84 B/s, 0 keys/s, 0 objects/s recovering 2024-09-10T09:37:20.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:37:20 smithi102 ceph-mon[35442]: from='client.24938 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T09:37:21.027 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:37:21.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:37:20 smithi005 ceph-mon[30350]: pgmap v363: 129 pgs: 129 active+clean; 645 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 8 op/s; 84 B/s, 0 keys/s, 0 objects/s recovering 2024-09-10T09:37:21.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:37:20 smithi005 ceph-mon[30350]: from='client.24938 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T09:37:21.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:37:20 smithi159 ceph-mon[35513]: pgmap v363: 129 pgs: 129 active+clean; 645 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 8 op/s; 84 B/s, 0 keys/s, 0 objects/s recovering 2024-09-10T09:37:21.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:37:20 smithi159 ceph-mon[35513]: from='client.24938 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T09:37:21.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:21.656+0000 7f273e000640 1 -- 172.21.15.5:0/920478264 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2738076720 msgr2=0x7f2738076b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:37:21.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:21.656+0000 7f273e000640 1 --2- 172.21.15.5:0/920478264 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2738076720 0x7f2738076b00 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f2728009860 tx=0x7f272802a340 comp rx=0 tx=0).stop 2024-09-10T09:37:21.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:21.656+0000 7f273e000640 1 -- 172.21.15.5:0/920478264 shutdown_connections 2024-09-10T09:37:21.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:21.656+0000 7f273e000640 1 --2- 172.21.15.5:0/920478264 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2738112a50 0x7f2738114e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:21.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:21.656+0000 7f273e000640 1 --2- 172.21.15.5:0/920478264 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2738077040 0x7f2738112510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:21.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:21.656+0000 7f273e000640 1 --2- 172.21.15.5:0/920478264 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2738076720 0x7f2738076b00 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:21.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:21.656+0000 7f273e000640 1 -- 172.21.15.5:0/920478264 >> 172.21.15.5:0/920478264 conn(0x7f27380ffa50 msgr2=0x7f2738101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:37:21.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:21.656+0000 7f273e000640 1 -- 172.21.15.5:0/920478264 shutdown_connections 2024-09-10T09:37:21.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:21.656+0000 7f273e000640 1 -- 172.21.15.5:0/920478264 wait complete. 2024-09-10T09:37:21.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:21.657+0000 7f273e000640 1 Processor -- start 2024-09-10T09:37:21.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:21.657+0000 7f273e000640 1 -- start start 2024-09-10T09:37:21.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:21.657+0000 7f273e000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2738076720 0x7f27381a4800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:37:21.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:21.657+0000 7f273e000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2738077040 0x7f27381a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:37:21.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:21.657+0000 7f273e000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2738112a50 0x7f273819e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:37:21.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:21.657+0000 7f273e000640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f27381185c0 con 0x7f2738112a50 2024-09-10T09:37:21.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:21.657+0000 7f273e000640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f2738118440 con 0x7f2738076720 2024-09-10T09:37:21.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:21.658+0000 7f273e000640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f2738118740 con 0x7f2738077040 2024-09-10T09:37:21.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:21.658+0000 7f2737400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2738076720 0x7f27381a4800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:37:21.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:21.658+0000 7f2736a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2738077040 0x7f27381a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:37:21.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:21.658+0000 7f2737e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2738112a50 0x7f273819e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:37:21.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:21.658+0000 7f2736a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2738077040 0x7f27381a4d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:36444/0 (socket says 172.21.15.5:36444) 2024-09-10T09:37:21.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:21.658+0000 7f2736a00640 1 -- 172.21.15.5:0/1572031506 learned_addr learned my addr 172.21.15.5:0/1572031506 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:37:21.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:21.658+0000 7f2737400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2738076720 0x7f27381a4800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:51930/0 (socket says 172.21.15.5:51930) 2024-09-10T09:37:21.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:21.658+0000 7f2737400640 1 -- 172.21.15.5:0/1572031506 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2738077040 msgr2=0x7f27381a4d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:37:21.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:21.658+0000 7f2737400640 1 --2- 172.21.15.5:0/1572031506 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2738077040 0x7f27381a4d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:21.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:21.658+0000 7f2737400640 1 -- 172.21.15.5:0/1572031506 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2738112a50 msgr2=0x7f273819e9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:37:21.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:21.658+0000 7f2737400640 1 --2- 172.21.15.5:0/1572031506 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2738112a50 0x7f273819e9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:21.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:21.658+0000 7f2737400640 1 -- 172.21.15.5:0/1572031506 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f273819f0f0 con 0x7f2738076720 2024-09-10T09:37:21.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:21.659+0000 7f2737400640 1 --2- 172.21.15.5:0/1572031506 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2738076720 0x7f27381a4800 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f2728035040 tx=0x7f2728033b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:37:21.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:21.659+0000 7f272fe00640 1 -- 172.21.15.5:0/1572031506 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f272803f020 con 0x7f2738076720 2024-09-10T09:37:21.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:21.659+0000 7f272fe00640 1 -- 172.21.15.5:0/1572031506 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2728030550 con 0x7f2738076720 2024-09-10T09:37:21.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:21.659+0000 7f273e000640 1 -- 172.21.15.5:0/1572031506 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f273819f380 con 0x7f2738076720 2024-09-10T09:37:21.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:21.659+0000 7f272fe00640 1 -- 172.21.15.5:0/1572031506 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2728037040 con 0x7f2738076720 2024-09-10T09:37:21.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:21.659+0000 7f273e000640 1 -- 172.21.15.5:0/1572031506 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27381ab6d0 con 0x7f2738076720 2024-09-10T09:37:21.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:21.661+0000 7f273e000640 1 -- 172.21.15.5:0/1572031506 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2738076b00 con 0x7f2738076720 2024-09-10T09:37:21.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:21.662+0000 7f272fe00640 1 -- 172.21.15.5:0/1572031506 <== mon.2 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f2728042020 con 0x7f2738076720 2024-09-10T09:37:21.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:21.664+0000 7f272fe00640 1 --2- 172.21.15.5:0/1572031506 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f271c0a2c90 0x7f271c0a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:37:21.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:21.664+0000 7f272fe00640 1 -- 172.21.15.5:0/1572031506 <== mon.2 v2:172.21.15.102:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f27280ef210 con 0x7f2738076720 2024-09-10T09:37:21.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:21.664+0000 7f2736a00640 1 --2- 172.21.15.5:0/1572031506 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f271c0a2c90 0x7f271c0a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:37:21.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:21.665+0000 7f2736a00640 1 --2- 172.21.15.5:0/1572031506 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f271c0a2c90 0x7f271c0a5150 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f27380792b0 tx=0x7f2724009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:37:21.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:21.668+0000 7f272fe00640 1 -- 172.21.15.5:0/1572031506 <== mon.2 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f2728049030 con 0x7f2738076720 2024-09-10T09:37:21.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:21.893+0000 7f273e000640 1 -- 172.21.15.5:0/1572031506 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f273819fef0 con 0x7f2738076720 2024-09-10T09:37:21.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:21.894+0000 7f272fe00640 1 -- 172.21.15.5:0/1572031506 <== mon.2 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f27280bbcd0 con 0x7f2738076720 2024-09-10T09:37:21.895 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-10T09:37:21.895 INFO:teuthology.orchestra.run.smithi005.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-10T09:37:21.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:21.896+0000 7f273e000640 1 -- 172.21.15.5:0/1572031506 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f271c0a2c90 msgr2=0x7f271c0a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:37:21.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:21.896+0000 7f273e000640 1 --2- 172.21.15.5:0/1572031506 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f271c0a2c90 0x7f271c0a5150 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f27380792b0 tx=0x7f2724009260 comp rx=0 tx=0).stop 2024-09-10T09:37:21.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:21.896+0000 7f273e000640 1 -- 172.21.15.5:0/1572031506 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2738076720 msgr2=0x7f27381a4800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:37:21.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:21.896+0000 7f273e000640 1 --2- 172.21.15.5:0/1572031506 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2738076720 0x7f27381a4800 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f2728035040 tx=0x7f2728033b60 comp rx=0 tx=0).stop 2024-09-10T09:37:21.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:21.896+0000 7f273e000640 1 -- 172.21.15.5:0/1572031506 shutdown_connections 2024-09-10T09:37:21.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:21.896+0000 7f273e000640 1 --2- 172.21.15.5:0/1572031506 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f271c0a2c90 0x7f271c0a5150 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:21.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:21.896+0000 7f273e000640 1 --2- 172.21.15.5:0/1572031506 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2738112a50 0x7f273819e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:21.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:21.896+0000 7f273e000640 1 --2- 172.21.15.5:0/1572031506 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2738077040 0x7f27381a4d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:21.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:21.896+0000 7f273e000640 1 --2- 172.21.15.5:0/1572031506 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2738076720 0x7f27381a4800 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:21.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:21.896+0000 7f273e000640 1 -- 172.21.15.5:0/1572031506 >> 172.21.15.5:0/1572031506 conn(0x7f27380ffa50 msgr2=0x7f2738114810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:37:21.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:21.896+0000 7f273e000640 1 -- 172.21.15.5:0/1572031506 shutdown_connections 2024-09-10T09:37:21.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:21.896+0000 7f273e000640 1 -- 172.21.15.5:0/1572031506 wait complete. 2024-09-10T09:37:22.671 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-09-10T09:37:22.671 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T09:37:22.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:37:22 smithi102 ceph-mon[35442]: pgmap v364: 129 pgs: 129 active+clean; 645 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 10 op/s; 84 B/s, 0 keys/s, 0 objects/s recovering 2024-09-10T09:37:22.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:37:22 smithi102 ceph-mon[35442]: from='client.? 172.21.15.5:0/1572031506' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-10T09:37:22.886 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:37:22 smithi005 ceph-mon[30350]: pgmap v364: 129 pgs: 129 active+clean; 645 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 10 op/s; 84 B/s, 0 keys/s, 0 objects/s recovering 2024-09-10T09:37:22.886 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:37:22 smithi005 ceph-mon[30350]: from='client.? 172.21.15.5:0/1572031506' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-10T09:37:23.057 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:37:23.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:37:22 smithi159 ceph-mon[35513]: pgmap v364: 129 pgs: 129 active+clean; 645 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 10 op/s; 84 B/s, 0 keys/s, 0 objects/s recovering 2024-09-10T09:37:23.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:37:22 smithi159 ceph-mon[35513]: from='client.? 172.21.15.5:0/1572031506' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-10T09:37:23.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:23.664+0000 7f7fd8a00640 1 -- 172.21.15.5:0/180512336 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7fd4109140 msgr2=0x7f7fd4110cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:37:23.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:23.664+0000 7f7fd8a00640 1 --2- 172.21.15.5:0/180512336 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7fd4109140 0x7f7fd4110cc0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f7fcc009ca0 tx=0x7f7fcc02a8c0 comp rx=0 tx=0).stop 2024-09-10T09:37:23.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:23.664+0000 7f7fd8a00640 1 -- 172.21.15.5:0/180512336 shutdown_connections 2024-09-10T09:37:23.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:23.664+0000 7f7fd8a00640 1 --2- 172.21.15.5:0/180512336 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7fd4109140 0x7f7fd4110cc0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:23.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:23.664+0000 7f7fd8a00640 1 --2- 172.21.15.5:0/180512336 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7fd41049c0 0x7f7fd4108a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:23.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:23.664+0000 7f7fd8a00640 1 --2- 172.21.15.5:0/180512336 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7fd4104010 0x7f7fd41043f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:23.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:23.664+0000 7f7fd8a00640 1 -- 172.21.15.5:0/180512336 >> 172.21.15.5:0/180512336 conn(0x7f7fd40ffa50 msgr2=0x7f7fd4101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:37:23.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:23.664+0000 7f7fd8a00640 1 -- 172.21.15.5:0/180512336 shutdown_connections 2024-09-10T09:37:23.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:23.664+0000 7f7fd8a00640 1 -- 172.21.15.5:0/180512336 wait complete. 2024-09-10T09:37:23.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:23.665+0000 7f7fd8a00640 1 Processor -- start 2024-09-10T09:37:23.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:23.665+0000 7f7fd8a00640 1 -- start start 2024-09-10T09:37:23.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:23.666+0000 7f7fd8a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7fd4104010 0x7f7fd419a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:37:23.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:23.666+0000 7f7fd8a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7fd41049c0 0x7f7fd419abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:37:23.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:23.666+0000 7f7fd8a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7fd4109140 0x7f7fd419ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:37:23.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:23.666+0000 7f7fd8a00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f7fd41141a0 con 0x7f7fd4104010 2024-09-10T09:37:23.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:23.666+0000 7f7fd8a00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f7fd4114020 con 0x7f7fd4109140 2024-09-10T09:37:23.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:23.666+0000 7f7fd8a00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f7fd4114320 con 0x7f7fd41049c0 2024-09-10T09:37:23.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:23.666+0000 7f7fd1e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7fd4104010 0x7f7fd419a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:37:23.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:23.666+0000 7f7fd1e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7fd4104010 0x7f7fd419a680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43212/0 (socket says 172.21.15.5:43212) 2024-09-10T09:37:23.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:23.666+0000 7f7fd1e00640 1 -- 172.21.15.5:0/627596636 learned_addr learned my addr 172.21.15.5:0/627596636 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:37:23.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:23.666+0000 7f7fd2800640 1 --2- 172.21.15.5:0/627596636 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7fd4109140 0x7f7fd419ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:37:23.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:23.666+0000 7f7fc9400640 1 --2- 172.21.15.5:0/627596636 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7fd41049c0 0x7f7fd419abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:37:23.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:23.666+0000 7f7fd2800640 1 -- 172.21.15.5:0/627596636 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7fd41049c0 msgr2=0x7f7fd419abc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:37:23.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:23.666+0000 7f7fd2800640 1 --2- 172.21.15.5:0/627596636 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7fd41049c0 0x7f7fd419abc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:23.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:23.666+0000 7f7fd2800640 1 -- 172.21.15.5:0/627596636 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7fd4104010 msgr2=0x7f7fd419a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:37:23.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:23.666+0000 7f7fd2800640 1 --2- 172.21.15.5:0/627596636 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7fd4104010 0x7f7fd419a680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:23.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:23.666+0000 7f7fd2800640 1 -- 172.21.15.5:0/627596636 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7fd419f630 con 0x7f7fd4109140 2024-09-10T09:37:23.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:23.666+0000 7f7fc9400640 1 --2- 172.21.15.5:0/627596636 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7fd41049c0 0x7f7fd419abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:37:23.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:23.667+0000 7f7fd2800640 1 --2- 172.21.15.5:0/627596636 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7fd4109140 0x7f7fd419ef50 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f7fcc005e30 tx=0x7f7fcc003f10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:37:23.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:23.667+0000 7f7fcb400640 1 -- 172.21.15.5:0/627596636 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fcc031650 con 0x7f7fd4109140 2024-09-10T09:37:23.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:23.667+0000 7f7fd8a00640 1 -- 172.21.15.5:0/627596636 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7fd419f8c0 con 0x7f7fd4109140 2024-09-10T09:37:23.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:23.667+0000 7f7fcb400640 1 -- 172.21.15.5:0/627596636 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7fcc0317f0 con 0x7f7fd4109140 2024-09-10T09:37:23.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:23.667+0000 7f7fcb400640 1 -- 172.21.15.5:0/627596636 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fcc02fa30 con 0x7f7fd4109140 2024-09-10T09:37:23.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:23.667+0000 7f7fd8a00640 1 -- 172.21.15.5:0/627596636 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7fd41a7200 con 0x7f7fd4109140 2024-09-10T09:37:23.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:23.669+0000 7f7fd8a00640 1 -- 172.21.15.5:0/627596636 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7fd41043f0 con 0x7f7fd4109140 2024-09-10T09:37:23.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:23.670+0000 7f7fcb400640 1 -- 172.21.15.5:0/627596636 <== mon.2 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f7fcc02fbd0 con 0x7f7fd4109140 2024-09-10T09:37:23.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:23.671+0000 7f7fcb400640 1 --2- 172.21.15.5:0/627596636 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f7fb80a2c10 0x7f7fb80a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:37:23.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:23.672+0000 7f7fcb400640 1 -- 172.21.15.5:0/627596636 <== mon.2 v2:172.21.15.102:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f7fcc0f07a0 con 0x7f7fd4109140 2024-09-10T09:37:23.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:23.672+0000 7f7fd1e00640 1 --2- 172.21.15.5:0/627596636 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f7fb80a2c10 0x7f7fb80a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:37:23.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:23.672+0000 7f7fd1e00640 1 --2- 172.21.15.5:0/627596636 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f7fb80a2c10 0x7f7fb80a50d0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f7fc4005fd0 tx=0x7f7fc400b480 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:37:23.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:23.676+0000 7f7fcb400640 1 -- 172.21.15.5:0/627596636 <== mon.2 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f7fcc0319a0 con 0x7f7fd4109140 2024-09-10T09:37:23.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:23.899+0000 7f7fd8a00640 1 -- 172.21.15.5:0/627596636 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7fd410c380 con 0x7f7fd4109140 2024-09-10T09:37:23.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:23.901+0000 7f7fcb400640 1 -- 172.21.15.5:0/627596636 <== mon.2 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11969 (secure 0 0 0) 0x7f7fcc0bd260 con 0x7f7fd4109140 2024-09-10T09:37:23.902 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-10T09:37:23.902 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":29,"btime":"2024-09-10T09:36:16:492171+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14454,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/2970594311","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2970594311},{"type":"v1","addr":"172.21.15.5:6835","nonce":2970594311}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14514,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6839/3616502174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":3616502174},{"type":"v1","addr":"172.21.15.5:6839","nonce":3616502174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T09:35:29.626287+0000","modified":"2024-09-10T09:36:15.764196+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24542,"mds_1":24535,"mds_2":24511,"mds_3":24506,"mds_4":24443},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14520":{"gid":14520,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/2073599842","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":2073599842},{"type":"v1","addr":"172.21.15.5:6841","nonce":2073599842}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"k","rank":4,"incarnation":26,"state":"up:active","state_seq":24,"addr":"172.21.15.102:6833/2457007427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":2457007427},{"type":"v1","addr":"172.21.15.102:6833","nonce":2457007427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/356445159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":356445159},{"type":"v1","addr":"172.21.15.159:6833","nonce":356445159}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6837/2894314507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":2894314507},{"type":"v1","addr":"172.21.15.5:6837","nonce":2894314507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.102:6835/1647549111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":1647549111},{"type":"v1","addr":"172.21.15.102:6835","nonce":1647549111}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6835/952657242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":952657242},{"type":"v1","addr":"172.21.15.159:6835","nonce":952657242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":19,"addr":"172.21.15.102:6837/2689776764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":2689776764},{"type":"v1","addr":"172.21.15.102:6837","nonce":2689776764}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.159:6837/620627639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":620627639},{"type":"v1","addr":"172.21.15.159:6837","nonce":620627639}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24535":{"gid":24535,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":18,"addr":"172.21.15.159:6839/2073739599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":2073739599},{"type":"v1","addr":"172.21.15.159:6839","nonce":2073739599}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24542":{"gid":24542,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.102:6839/335409749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6838","nonce":335409749},{"type":"v1","addr":"172.21.15.102:6839","nonce":335409749}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24542,"qdb_cluster":[24443,24506,24511,24535,24542]},"id":1}]} 2024-09-10T09:37:23.902 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 29 2024-09-10T09:37:23.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:23.903+0000 7f7fd8a00640 1 -- 172.21.15.5:0/627596636 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f7fb80a2c10 msgr2=0x7f7fb80a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:37:23.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:23.903+0000 7f7fd8a00640 1 --2- 172.21.15.5:0/627596636 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f7fb80a2c10 0x7f7fb80a50d0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f7fc4005fd0 tx=0x7f7fc400b480 comp rx=0 tx=0).stop 2024-09-10T09:37:23.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:23.903+0000 7f7fd8a00640 1 -- 172.21.15.5:0/627596636 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7fd4109140 msgr2=0x7f7fd419ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:37:23.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:23.903+0000 7f7fd8a00640 1 --2- 172.21.15.5:0/627596636 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7fd4109140 0x7f7fd419ef50 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f7fcc005e30 tx=0x7f7fcc003f10 comp rx=0 tx=0).stop 2024-09-10T09:37:23.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:23.903+0000 7f7fd8a00640 1 -- 172.21.15.5:0/627596636 shutdown_connections 2024-09-10T09:37:23.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:23.903+0000 7f7fd8a00640 1 --2- 172.21.15.5:0/627596636 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f7fb80a2c10 0x7f7fb80a50d0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:23.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:23.903+0000 7f7fd8a00640 1 --2- 172.21.15.5:0/627596636 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7fd4109140 0x7f7fd419ef50 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:23.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:23.903+0000 7f7fd8a00640 1 --2- 172.21.15.5:0/627596636 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7fd41049c0 0x7f7fd419abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:23.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:23.903+0000 7f7fd8a00640 1 --2- 172.21.15.5:0/627596636 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7fd4104010 0x7f7fd419a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:23.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:23.903+0000 7f7fd8a00640 1 -- 172.21.15.5:0/627596636 >> 172.21.15.5:0/627596636 conn(0x7f7fd40ffa50 msgr2=0x7f7fd4100d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:37:23.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:23.904+0000 7f7fd8a00640 1 -- 172.21.15.5:0/627596636 shutdown_connections 2024-09-10T09:37:23.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:23.904+0000 7f7fd8a00640 1 -- 172.21.15.5:0/627596636 wait complete. 2024-09-10T09:37:24.713 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T09:37:24.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:37:24 smithi102 ceph-mon[35442]: pgmap v365: 129 pgs: 129 active+clean; 645 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-10T09:37:24.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:37:24 smithi102 ceph-mon[35442]: from='client.? 172.21.15.5:0/627596636' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:37:25.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:37:24 smithi005 ceph-mon[30350]: pgmap v365: 129 pgs: 129 active+clean; 645 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-10T09:37:25.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:37:24 smithi005 ceph-mon[30350]: from='client.? 172.21.15.5:0/627596636' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:37:25.090 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:37:25.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:37:24 smithi159 ceph-mon[35513]: pgmap v365: 129 pgs: 129 active+clean; 645 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-10T09:37:25.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:37:24 smithi159 ceph-mon[35513]: from='client.? 172.21.15.5:0/627596636' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:37:25.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:25.679+0000 7f32b2400640 1 -- 172.21.15.5:0/3864072625 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f32ac112a30 msgr2=0x7f32ac114e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:37:25.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:25.679+0000 7f32b2400640 1 --2- 172.21.15.5:0/3864072625 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f32ac112a30 0x7f32ac114e20 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f32a4009860 tx=0x7f32a402a340 comp rx=0 tx=0).stop 2024-09-10T09:37:25.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:25.679+0000 7f32b2400640 1 -- 172.21.15.5:0/3864072625 shutdown_connections 2024-09-10T09:37:25.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:25.679+0000 7f32b2400640 1 --2- 172.21.15.5:0/3864072625 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f32ac112a30 0x7f32ac114e20 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:25.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:25.679+0000 7f32b2400640 1 --2- 172.21.15.5:0/3864072625 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f32ac077040 0x7f32ac1124f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:25.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:25.679+0000 7f32b2400640 1 --2- 172.21.15.5:0/3864072625 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f32ac076720 0x7f32ac076b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:25.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:25.679+0000 7f32b2400640 1 -- 172.21.15.5:0/3864072625 >> 172.21.15.5:0/3864072625 conn(0x7f32ac0ffa90 msgr2=0x7f32ac101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:37:25.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:25.679+0000 7f32b2400640 1 -- 172.21.15.5:0/3864072625 shutdown_connections 2024-09-10T09:37:25.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:25.679+0000 7f32b2400640 1 -- 172.21.15.5:0/3864072625 wait complete. 2024-09-10T09:37:25.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:25.680+0000 7f32b2400640 1 Processor -- start 2024-09-10T09:37:25.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:25.680+0000 7f32b2400640 1 -- start start 2024-09-10T09:37:25.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:25.681+0000 7f32b2400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f32ac076720 0x7f32ac1a46c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:37:25.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:25.681+0000 7f32b2400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f32ac077040 0x7f32ac1a4c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:37:25.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:25.681+0000 7f32b2400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f32ac112a30 0x7f32ac19e880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:37:25.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:25.681+0000 7f32b2400640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f32ac118590 con 0x7f32ac076720 2024-09-10T09:37:25.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:25.681+0000 7f32b2400640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f32ac118410 con 0x7f32ac077040 2024-09-10T09:37:25.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:25.681+0000 7f32b2400640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f32ac118710 con 0x7f32ac112a30 2024-09-10T09:37:25.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:25.681+0000 7f32ab400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f32ac076720 0x7f32ac1a46c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:37:25.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:25.681+0000 7f32ab400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f32ac076720 0x7f32ac1a46c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43228/0 (socket says 172.21.15.5:43228) 2024-09-10T09:37:25.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:25.681+0000 7f32ab400640 1 -- 172.21.15.5:0/3192179172 learned_addr learned my addr 172.21.15.5:0/3192179172 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:37:25.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:25.681+0000 7f32abe00640 1 --2- 172.21.15.5:0/3192179172 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f32ac112a30 0x7f32ac19e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:37:25.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:25.681+0000 7f32a3e00640 1 --2- 172.21.15.5:0/3192179172 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f32ac077040 0x7f32ac1a4c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:37:25.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:25.681+0000 7f32abe00640 1 -- 172.21.15.5:0/3192179172 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f32ac077040 msgr2=0x7f32ac1a4c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:37:25.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:25.681+0000 7f32abe00640 1 --2- 172.21.15.5:0/3192179172 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f32ac077040 0x7f32ac1a4c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:25.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:25.681+0000 7f32abe00640 1 -- 172.21.15.5:0/3192179172 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f32ac076720 msgr2=0x7f32ac1a46c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:37:25.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:25.681+0000 7f32abe00640 1 --2- 172.21.15.5:0/3192179172 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f32ac076720 0x7f32ac1a46c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:25.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:25.682+0000 7f32abe00640 1 -- 172.21.15.5:0/3192179172 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32ac19efb0 con 0x7f32ac112a30 2024-09-10T09:37:25.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:25.682+0000 7f32a3e00640 1 --2- 172.21.15.5:0/3192179172 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f32ac077040 0x7f32ac1a4c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:37:25.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:25.682+0000 7f32ab400640 1 --2- 172.21.15.5:0/3192179172 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f32ac076720 0x7f32ac1a46c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:37:25.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:25.682+0000 7f32abe00640 1 --2- 172.21.15.5:0/3192179172 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f32ac112a30 0x7f32ac19e880 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f32a4035040 tx=0x7f32a4033d10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:37:25.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:25.682+0000 7f32a8c00640 1 -- 172.21.15.5:0/3192179172 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32a4037040 con 0x7f32ac112a30 2024-09-10T09:37:25.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:25.682+0000 7f32b2400640 1 -- 172.21.15.5:0/3192179172 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32ac19f240 con 0x7f32ac112a30 2024-09-10T09:37:25.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:25.682+0000 7f32a8c00640 1 -- 172.21.15.5:0/3192179172 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f32a4040070 con 0x7f32ac112a30 2024-09-10T09:37:25.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:25.682+0000 7f32a8c00640 1 -- 172.21.15.5:0/3192179172 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32a40038e0 con 0x7f32ac112a30 2024-09-10T09:37:25.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:25.682+0000 7f32b2400640 1 -- 172.21.15.5:0/3192179172 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32ac1ab590 con 0x7f32ac112a30 2024-09-10T09:37:25.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:25.683+0000 7f32b2400640 1 -- 172.21.15.5:0/3192179172 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f32ac076b00 con 0x7f32ac112a30 2024-09-10T09:37:25.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:25.685+0000 7f32a8c00640 1 -- 172.21.15.5:0/3192179172 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f32a4003a80 con 0x7f32ac112a30 2024-09-10T09:37:25.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:25.687+0000 7f32a8c00640 1 --2- 172.21.15.5:0/3192179172 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f32900a2c90 0x7f32900a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:37:25.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:25.687+0000 7f32a8c00640 1 -- 172.21.15.5:0/3192179172 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f32a40ef730 con 0x7f32ac112a30 2024-09-10T09:37:25.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:25.687+0000 7f32ab400640 1 --2- 172.21.15.5:0/3192179172 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f32900a2c90 0x7f32900a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:37:25.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:25.687+0000 7f32ab400640 1 --2- 172.21.15.5:0/3192179172 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f32900a2c90 0x7f32900a5150 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f32ac19f9c0 tx=0x7f329c010040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:37:25.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:25.689+0000 7f32a8c00640 1 -- 172.21.15.5:0/3192179172 <== mon.1 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f32a40bc1f0 con 0x7f32ac112a30 2024-09-10T09:37:25.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:25.912+0000 7f32b2400640 1 -- 172.21.15.5:0/3192179172 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f32ac10dbc0 con 0x7f32ac112a30 2024-09-10T09:37:25.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:25.913+0000 7f32a8c00640 1 -- 172.21.15.5:0/3192179172 <== mon.1 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11969 (secure 0 0 0) 0x7f32a40bbaf0 con 0x7f32ac112a30 2024-09-10T09:37:25.916 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-10T09:37:25.916 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":29,"btime":"2024-09-10T09:36:16:492171+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14454,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/2970594311","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2970594311},{"type":"v1","addr":"172.21.15.5:6835","nonce":2970594311}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14514,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6839/3616502174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":3616502174},{"type":"v1","addr":"172.21.15.5:6839","nonce":3616502174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T09:35:29.626287+0000","modified":"2024-09-10T09:36:15.764196+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24542,"mds_1":24535,"mds_2":24511,"mds_3":24506,"mds_4":24443},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14520":{"gid":14520,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/2073599842","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":2073599842},{"type":"v1","addr":"172.21.15.5:6841","nonce":2073599842}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"k","rank":4,"incarnation":26,"state":"up:active","state_seq":24,"addr":"172.21.15.102:6833/2457007427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":2457007427},{"type":"v1","addr":"172.21.15.102:6833","nonce":2457007427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/356445159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":356445159},{"type":"v1","addr":"172.21.15.159:6833","nonce":356445159}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6837/2894314507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":2894314507},{"type":"v1","addr":"172.21.15.5:6837","nonce":2894314507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.102:6835/1647549111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":1647549111},{"type":"v1","addr":"172.21.15.102:6835","nonce":1647549111}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6835/952657242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":952657242},{"type":"v1","addr":"172.21.15.159:6835","nonce":952657242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":19,"addr":"172.21.15.102:6837/2689776764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":2689776764},{"type":"v1","addr":"172.21.15.102:6837","nonce":2689776764}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.159:6837/620627639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":620627639},{"type":"v1","addr":"172.21.15.159:6837","nonce":620627639}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24535":{"gid":24535,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":18,"addr":"172.21.15.159:6839/2073739599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":2073739599},{"type":"v1","addr":"172.21.15.159:6839","nonce":2073739599}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24542":{"gid":24542,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.102:6839/335409749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6838","nonce":335409749},{"type":"v1","addr":"172.21.15.102:6839","nonce":335409749}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24542,"qdb_cluster":[24443,24506,24511,24535,24542]},"id":1}]} 2024-09-10T09:37:25.916 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 29 2024-09-10T09:37:25.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:25.916+0000 7f32b2400640 1 -- 172.21.15.5:0/3192179172 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f32900a2c90 msgr2=0x7f32900a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:37:25.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:25.916+0000 7f32b2400640 1 --2- 172.21.15.5:0/3192179172 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f32900a2c90 0x7f32900a5150 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f32ac19f9c0 tx=0x7f329c010040 comp rx=0 tx=0).stop 2024-09-10T09:37:25.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:25.916+0000 7f32b2400640 1 -- 172.21.15.5:0/3192179172 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f32ac112a30 msgr2=0x7f32ac19e880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:37:25.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:25.916+0000 7f32b2400640 1 --2- 172.21.15.5:0/3192179172 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f32ac112a30 0x7f32ac19e880 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f32a4035040 tx=0x7f32a4033d10 comp rx=0 tx=0).stop 2024-09-10T09:37:25.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:25.916+0000 7f32b2400640 1 -- 172.21.15.5:0/3192179172 shutdown_connections 2024-09-10T09:37:25.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:25.916+0000 7f32b2400640 1 --2- 172.21.15.5:0/3192179172 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f32900a2c90 0x7f32900a5150 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:25.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:25.916+0000 7f32b2400640 1 --2- 172.21.15.5:0/3192179172 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f32ac112a30 0x7f32ac19e880 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:25.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:25.916+0000 7f32b2400640 1 --2- 172.21.15.5:0/3192179172 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f32ac077040 0x7f32ac1a4c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:25.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:25.916+0000 7f32b2400640 1 --2- 172.21.15.5:0/3192179172 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f32ac076720 0x7f32ac1a46c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:25.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:25.916+0000 7f32b2400640 1 -- 172.21.15.5:0/3192179172 >> 172.21.15.5:0/3192179172 conn(0x7f32ac0ffa90 msgr2=0x7f32ac101e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:37:25.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:25.916+0000 7f32b2400640 1 -- 172.21.15.5:0/3192179172 shutdown_connections 2024-09-10T09:37:25.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:25.916+0000 7f32b2400640 1 -- 172.21.15.5:0/3192179172 wait complete. 2024-09-10T09:37:26.676 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-10T09:37:26.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:37:26 smithi102 ceph-mon[35442]: pgmap v366: 129 pgs: 129 active+clean; 645 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-09-10T09:37:26.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:37:26 smithi102 ceph-mon[35442]: from='client.? 172.21.15.5:0/3192179172' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:37:27.053 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:37:27.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:37:26 smithi005 ceph-mon[30350]: pgmap v366: 129 pgs: 129 active+clean; 645 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-09-10T09:37:27.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:37:26 smithi005 ceph-mon[30350]: from='client.? 172.21.15.5:0/3192179172' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:37:27.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:37:26 smithi159 ceph-mon[35513]: pgmap v366: 129 pgs: 129 active+clean; 645 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-09-10T09:37:27.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:37:26 smithi159 ceph-mon[35513]: from='client.? 172.21.15.5:0/3192179172' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:37:27.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:27.619+0000 7fd3a3200640 1 -- 172.21.15.5:0/2395120740 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd3a41008c0 msgr2=0x7fd3a4104470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:37:27.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:27.619+0000 7fd3a3200640 1 --2- 172.21.15.5:0/2395120740 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd3a41008c0 0x7fd3a4104470 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fd39c009ca0 tx=0x7fd39c02a8c0 comp rx=0 tx=0).stop 2024-09-10T09:37:27.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:27.619+0000 7fd3a3200640 1 -- 172.21.15.5:0/2395120740 shutdown_connections 2024-09-10T09:37:27.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:27.619+0000 7fd3a3200640 1 --2- 172.21.15.5:0/2395120740 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd3a41008c0 0x7fd3a4104470 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:27.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:27.619+0000 7fd3a3200640 1 --2- 172.21.15.5:0/2395120740 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd3a40fff40 0x7fd3a4100380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:27.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:27.619+0000 7fd3a3200640 1 --2- 172.21.15.5:0/2395120740 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd3a4106a00 0x7fd3a4106de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:27.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:27.619+0000 7fd3a3200640 1 -- 172.21.15.5:0/2395120740 >> 172.21.15.5:0/2395120740 conn(0x7fd3a4077070 msgr2=0x7fd3a40fe480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:37:27.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:27.619+0000 7fd3a3200640 1 -- 172.21.15.5:0/2395120740 shutdown_connections 2024-09-10T09:37:27.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:27.619+0000 7fd3a3200640 1 -- 172.21.15.5:0/2395120740 wait complete. 2024-09-10T09:37:27.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:27.620+0000 7fd3a3200640 1 Processor -- start 2024-09-10T09:37:27.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:27.620+0000 7fd3a3200640 1 -- start start 2024-09-10T09:37:27.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:27.621+0000 7fd3a3200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd3a40fff40 0x7fd3a41a4560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:37:27.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:27.621+0000 7fd3a3200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd3a41008c0 0x7fd3a41a4aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:37:27.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:27.621+0000 7fd3a3200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd3a4106a00 0x7fd3a419e630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:37:27.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:27.621+0000 7fd3a3200640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd3a4118470 con 0x7fd3a40fff40 2024-09-10T09:37:27.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:27.621+0000 7fd3a3200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fd3a41182f0 con 0x7fd3a41008c0 2024-09-10T09:37:27.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:27.621+0000 7fd3a3200640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fd3a41185f0 con 0x7fd3a4106a00 2024-09-10T09:37:27.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:27.621+0000 7fd3a2800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd3a4106a00 0x7fd3a419e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:37:27.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:27.621+0000 7fd3a2800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd3a4106a00 0x7fd3a419e630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:36510/0 (socket says 172.21.15.5:36510) 2024-09-10T09:37:27.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:27.621+0000 7fd3a2800640 1 -- 172.21.15.5:0/283570637 learned_addr learned my addr 172.21.15.5:0/283570637 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:37:27.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:27.621+0000 7fd3a1e00640 1 --2- 172.21.15.5:0/283570637 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd3a40fff40 0x7fd3a41a4560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:37:27.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:27.621+0000 7fd399400640 1 --2- 172.21.15.5:0/283570637 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd3a41008c0 0x7fd3a41a4aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:37:27.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:27.621+0000 7fd3a2800640 1 -- 172.21.15.5:0/283570637 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd3a41008c0 msgr2=0x7fd3a41a4aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:37:27.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:27.621+0000 7fd3a2800640 1 --2- 172.21.15.5:0/283570637 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd3a41008c0 0x7fd3a41a4aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:27.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:27.621+0000 7fd3a2800640 1 -- 172.21.15.5:0/283570637 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd3a40fff40 msgr2=0x7fd3a41a4560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:37:27.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:27.621+0000 7fd3a2800640 1 --2- 172.21.15.5:0/283570637 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd3a40fff40 0x7fd3a41a4560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:27.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:27.621+0000 7fd3a2800640 1 -- 172.21.15.5:0/283570637 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3a419eea0 con 0x7fd3a4106a00 2024-09-10T09:37:27.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:27.622+0000 7fd399400640 1 --2- 172.21.15.5:0/283570637 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd3a41008c0 0x7fd3a41a4aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:37:27.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:27.622+0000 7fd3a2800640 1 --2- 172.21.15.5:0/283570637 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd3a4106a00 0x7fd3a419e630 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fd39c005e30 tx=0x7fd39c003f10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:37:27.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:27.622+0000 7fd39b400640 1 -- 172.21.15.5:0/283570637 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd39c031650 con 0x7fd3a4106a00 2024-09-10T09:37:27.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:27.622+0000 7fd3a3200640 1 -- 172.21.15.5:0/283570637 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3a419f130 con 0x7fd3a4106a00 2024-09-10T09:37:27.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:27.622+0000 7fd39b400640 1 -- 172.21.15.5:0/283570637 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd39c0317f0 con 0x7fd3a4106a00 2024-09-10T09:37:27.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:27.622+0000 7fd39b400640 1 -- 172.21.15.5:0/283570637 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd39c02fa30 con 0x7fd3a4106a00 2024-09-10T09:37:27.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:27.622+0000 7fd3a3200640 1 -- 172.21.15.5:0/283570637 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3a41ab310 con 0x7fd3a4106a00 2024-09-10T09:37:27.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:27.623+0000 7fd3a3200640 1 -- 172.21.15.5:0/283570637 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd3a4100f50 con 0x7fd3a4106a00 2024-09-10T09:37:27.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:27.625+0000 7fd39b400640 1 -- 172.21.15.5:0/283570637 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7fd39c02fbd0 con 0x7fd3a4106a00 2024-09-10T09:37:27.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:27.627+0000 7fd39b400640 1 --2- 172.21.15.5:0/283570637 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fd3880a2c10 0x7fd3880a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:37:27.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:27.627+0000 7fd39b400640 1 -- 172.21.15.5:0/283570637 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fd39c0f0920 con 0x7fd3a4106a00 2024-09-10T09:37:27.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:27.627+0000 7fd3a1e00640 1 --2- 172.21.15.5:0/283570637 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fd3880a2c10 0x7fd3880a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:37:27.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:27.628+0000 7fd3a1e00640 1 --2- 172.21.15.5:0/283570637 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fd3880a2c10 0x7fd3880a50d0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fd394009bd0 tx=0x7fd394005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:37:27.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:27.628+0000 7fd39b400640 1 -- 172.21.15.5:0/283570637 <== mon.1 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd39c0319a0 con 0x7fd3a4106a00 2024-09-10T09:37:27.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:27.814+0000 7fd3a3200640 1 -- 172.21.15.5:0/283570637 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd3a41ab4e0 con 0x7fd3a4106a00 2024-09-10T09:37:27.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:27.816+0000 7fd39b400640 1 -- 172.21.15.5:0/283570637 <== mon.1 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v96) ==== 74+0+24846 (secure 0 0 0) 0x7fd39c0bd3e0 con 0x7fd3a4106a00 2024-09-10T09:37:27.818 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-10T09:37:27.818 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":96,"fsid":"bade4b50-6f56-11ef-bcea-c7b262605968","created":"2024-09-10T09:27:13.298498+0000","modified":"2024-09-10T09:37:07.520520+0000","last_up_change":"2024-09-10T09:34:30.730066+0000","last_in_change":"2024-09-10T09:34:07.340995+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T09:30:26.042255+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T09:35:24.461031+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T09:35:27.155349+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-10T09:35:35.121462+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"085749a8-e882-4377-92dd-34bb266c0f0b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":954535266},{"type":"v1","addr":"172.21.15.5:6803","nonce":954535266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":954535266},{"type":"v1","addr":"172.21.15.5:6805","nonce":954535266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":954535266},{"type":"v1","addr":"172.21.15.5:6809","nonce":954535266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":954535266},{"type":"v1","addr":"172.21.15.5:6807","nonce":954535266}]},"public_addr":"172.21.15.5:6803/954535266","cluster_addr":"172.21.15.5:6805/954535266","heartbeat_back_addr":"172.21.15.5:6809/954535266","heartbeat_front_addr":"172.21.15.5:6807/954535266","state":["exists","up"]},{"osd":1,"uuid":"0a76c795-2573-4a92-969a-2ec5dd65c094","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6800","nonce":733924375},{"type":"v1","addr":"172.21.15.102:6801","nonce":733924375}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6802","nonce":733924375},{"type":"v1","addr":"172.21.15.102:6803","nonce":733924375}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6806","nonce":733924375},{"type":"v1","addr":"172.21.15.102:6807","nonce":733924375}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6804","nonce":733924375},{"type":"v1","addr":"172.21.15.102:6805","nonce":733924375}]},"public_addr":"172.21.15.102:6801/733924375","cluster_addr":"172.21.15.102:6803/733924375","heartbeat_back_addr":"172.21.15.102:6807/733924375","heartbeat_front_addr":"172.21.15.102:6805/733924375","state":["exists","up"]},{"osd":2,"uuid":"0c98fcb8-60ab-4473-8995-1d9e97c577e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6800","nonce":2236204189},{"type":"v1","addr":"172.21.15.159:6801","nonce":2236204189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":2236204189},{"type":"v1","addr":"172.21.15.159:6803","nonce":2236204189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":2236204189},{"type":"v1","addr":"172.21.15.159:6807","nonce":2236204189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":2236204189},{"type":"v1","addr":"172.21.15.159:6805","nonce":2236204189}]},"public_addr":"172.21.15.159:6801/2236204189","cluster_addr":"172.21.15.159:6803/2236204189","heartbeat_back_addr":"172.21.15.159:6807/2236204189","heartbeat_front_addr":"172.21.15.159:6805/2236204189","state":["exists","up"]},{"osd":3,"uuid":"88eecdeb-5cee-4f78-a3f9-a66bf4bf772a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":2221468725},{"type":"v1","addr":"172.21.15.5:6811","nonce":2221468725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":2221468725},{"type":"v1","addr":"172.21.15.5:6813","nonce":2221468725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6816","nonce":2221468725},{"type":"v1","addr":"172.21.15.5:6817","nonce":2221468725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":2221468725},{"type":"v1","addr":"172.21.15.5:6815","nonce":2221468725}]},"public_addr":"172.21.15.5:6811/2221468725","cluster_addr":"172.21.15.5:6813/2221468725","heartbeat_back_addr":"172.21.15.5:6817/2221468725","heartbeat_front_addr":"172.21.15.5:6815/2221468725","state":["exists","up"]},{"osd":4,"uuid":"f2e7314d-7938-4668-9011-45c175f66b48","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6808","nonce":594465753},{"type":"v1","addr":"172.21.15.102:6809","nonce":594465753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6810","nonce":594465753},{"type":"v1","addr":"172.21.15.102:6811","nonce":594465753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6814","nonce":594465753},{"type":"v1","addr":"172.21.15.102:6815","nonce":594465753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6812","nonce":594465753},{"type":"v1","addr":"172.21.15.102:6813","nonce":594465753}]},"public_addr":"172.21.15.102:6809/594465753","cluster_addr":"172.21.15.102:6811/594465753","heartbeat_back_addr":"172.21.15.102:6815/594465753","heartbeat_front_addr":"172.21.15.102:6813/594465753","state":["exists","up"]},{"osd":5,"uuid":"0c0366d3-658b-4bf0-ba17-aff28a02e948","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":3230280402},{"type":"v1","addr":"172.21.15.159:6809","nonce":3230280402}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":3230280402},{"type":"v1","addr":"172.21.15.159:6811","nonce":3230280402}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":3230280402},{"type":"v1","addr":"172.21.15.159:6815","nonce":3230280402}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":3230280402},{"type":"v1","addr":"172.21.15.159:6813","nonce":3230280402}]},"public_addr":"172.21.15.159:6809/3230280402","cluster_addr":"172.21.15.159:6811/3230280402","heartbeat_back_addr":"172.21.15.159:6815/3230280402","heartbeat_front_addr":"172.21.15.159:6813/3230280402","state":["exists","up"]},{"osd":6,"uuid":"682670a6-51d5-46e8-af6e-dc2d8c1cf407","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":1907651140},{"type":"v1","addr":"172.21.15.5:6819","nonce":1907651140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":1907651140},{"type":"v1","addr":"172.21.15.5:6821","nonce":1907651140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":1907651140},{"type":"v1","addr":"172.21.15.5:6825","nonce":1907651140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":1907651140},{"type":"v1","addr":"172.21.15.5:6823","nonce":1907651140}]},"public_addr":"172.21.15.5:6819/1907651140","cluster_addr":"172.21.15.5:6821/1907651140","heartbeat_back_addr":"172.21.15.5:6825/1907651140","heartbeat_front_addr":"172.21.15.5:6823/1907651140","state":["exists","up"]},{"osd":7,"uuid":"8ae0f4a9-99e7-4563-aa2a-15169684a8b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6816","nonce":692040360},{"type":"v1","addr":"172.21.15.102:6817","nonce":692040360}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6818","nonce":692040360},{"type":"v1","addr":"172.21.15.102:6819","nonce":692040360}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6822","nonce":692040360},{"type":"v1","addr":"172.21.15.102:6823","nonce":692040360}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6820","nonce":692040360},{"type":"v1","addr":"172.21.15.102:6821","nonce":692040360}]},"public_addr":"172.21.15.102:6817/692040360","cluster_addr":"172.21.15.102:6819/692040360","heartbeat_back_addr":"172.21.15.102:6823/692040360","heartbeat_front_addr":"172.21.15.102:6821/692040360","state":["exists","up"]},{"osd":8,"uuid":"e37c845e-ced9-4413-9bbe-bc090537c2bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":2107291799},{"type":"v1","addr":"172.21.15.159:6817","nonce":2107291799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":2107291799},{"type":"v1","addr":"172.21.15.159:6819","nonce":2107291799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":2107291799},{"type":"v1","addr":"172.21.15.159:6823","nonce":2107291799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":2107291799},{"type":"v1","addr":"172.21.15.159:6821","nonce":2107291799}]},"public_addr":"172.21.15.159:6817/2107291799","cluster_addr":"172.21.15.159:6819/2107291799","heartbeat_back_addr":"172.21.15.159:6823/2107291799","heartbeat_front_addr":"172.21.15.159:6821/2107291799","state":["exists","up"]},{"osd":9,"uuid":"91a5b6cc-08da-47be-af12-d0896a849309","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":902431281},{"type":"v1","addr":"172.21.15.5:6827","nonce":902431281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":902431281},{"type":"v1","addr":"172.21.15.5:6829","nonce":902431281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":902431281},{"type":"v1","addr":"172.21.15.5:6833","nonce":902431281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":902431281},{"type":"v1","addr":"172.21.15.5:6831","nonce":902431281}]},"public_addr":"172.21.15.5:6827/902431281","cluster_addr":"172.21.15.5:6829/902431281","heartbeat_back_addr":"172.21.15.5:6833/902431281","heartbeat_front_addr":"172.21.15.5:6831/902431281","state":["exists","up"]},{"osd":10,"uuid":"2211030c-b00e-402b-b4d7-c587eb29a490","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6824","nonce":2853806211},{"type":"v1","addr":"172.21.15.102:6825","nonce":2853806211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6826","nonce":2853806211},{"type":"v1","addr":"172.21.15.102:6827","nonce":2853806211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6830","nonce":2853806211},{"type":"v1","addr":"172.21.15.102:6831","nonce":2853806211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6828","nonce":2853806211},{"type":"v1","addr":"172.21.15.102:6829","nonce":2853806211}]},"public_addr":"172.21.15.102:6825/2853806211","cluster_addr":"172.21.15.102:6827/2853806211","heartbeat_back_addr":"172.21.15.102:6831/2853806211","heartbeat_front_addr":"172.21.15.102:6829/2853806211","state":["exists","up"]},{"osd":11,"uuid":"4e9699ec-8ff2-4005-9d84-af5600b3a2ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6824","nonce":969941218},{"type":"v1","addr":"172.21.15.159:6825","nonce":969941218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":969941218},{"type":"v1","addr":"172.21.15.159:6827","nonce":969941218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6830","nonce":969941218},{"type":"v1","addr":"172.21.15.159:6831","nonce":969941218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":969941218},{"type":"v1","addr":"172.21.15.159:6829","nonce":969941218}]},"public_addr":"172.21.15.159:6825/969941218","cluster_addr":"172.21.15.159:6827/969941218","heartbeat_back_addr":"172.21.15.159:6831/969941218","heartbeat_front_addr":"172.21.15.159:6829/969941218","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:29:34.416490+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:30:00.246101+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:30:24.300464+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:30:54.663623+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:31:21.426292+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:31:46.738914+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:32:16.196571+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:32:42.549243+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:33:08.178651+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:33:37.477911+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:34:03.114026+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:34:28.891924+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4}]},{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"3.0","mappings":[{"from":4,"to":8}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":6}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9},{"from":5,"to":7}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]},{"pgid":"3.39","mappings":[{"from":5,"to":6}]},{"pgid":"4.8","mappings":[{"from":5,"to":8}]},{"pgid":"4.c","mappings":[{"from":9,"to":10}]},{"pgid":"4.18","mappings":[{"from":9,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.5:6801/1527422686":"2024-09-11T09:28:05.436742+0000","172.21.15.5:0/2276144345":"2024-09-11T09:28:05.436742+0000","172.21.15.5:0/412040104":"2024-09-11T09:28:05.436742+0000","172.21.15.5:6800/1527422686":"2024-09-11T09:28:05.436742+0000","172.21.15.5:0/1856581170":"2024-09-11T09:28:05.436742+0000","172.21.15.5:0/2847668298":"2024-09-11T09:27:37.778500+0000","172.21.15.5:6801/14425913":"2024-09-11T09:27:37.778500+0000","172.21.15.5:0/166910782":"2024-09-11T09:27:37.778500+0000","172.21.15.5:6800/14425913":"2024-09-11T09:27:37.778500+0000","172.21.15.5:0/1711858179":"2024-09-11T09:27:37.778500+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T09:37:27.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:27.818+0000 7fd3a3200640 1 -- 172.21.15.5:0/283570637 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fd3880a2c10 msgr2=0x7fd3880a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:37:27.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:27.818+0000 7fd3a3200640 1 --2- 172.21.15.5:0/283570637 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fd3880a2c10 0x7fd3880a50d0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fd394009bd0 tx=0x7fd394005f90 comp rx=0 tx=0).stop 2024-09-10T09:37:27.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:27.818+0000 7fd3a3200640 1 -- 172.21.15.5:0/283570637 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd3a4106a00 msgr2=0x7fd3a419e630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:37:27.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:27.818+0000 7fd3a3200640 1 --2- 172.21.15.5:0/283570637 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd3a4106a00 0x7fd3a419e630 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fd39c005e30 tx=0x7fd39c003f10 comp rx=0 tx=0).stop 2024-09-10T09:37:27.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:27.818+0000 7fd3a3200640 1 -- 172.21.15.5:0/283570637 shutdown_connections 2024-09-10T09:37:27.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:27.818+0000 7fd3a3200640 1 --2- 172.21.15.5:0/283570637 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fd3880a2c10 0x7fd3880a50d0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:27.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:27.818+0000 7fd3a3200640 1 --2- 172.21.15.5:0/283570637 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd3a4106a00 0x7fd3a419e630 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:27.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:27.818+0000 7fd3a3200640 1 --2- 172.21.15.5:0/283570637 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd3a41008c0 0x7fd3a41a4aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:27.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:27.818+0000 7fd3a3200640 1 --2- 172.21.15.5:0/283570637 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd3a40fff40 0x7fd3a41a4560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:27.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:27.818+0000 7fd3a3200640 1 -- 172.21.15.5:0/283570637 >> 172.21.15.5:0/283570637 conn(0x7fd3a4077070 msgr2=0x7fd3a41022d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:37:27.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:27.819+0000 7fd3a3200640 1 -- 172.21.15.5:0/283570637 shutdown_connections 2024-09-10T09:37:27.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:27.819+0000 7fd3a3200640 1 -- 172.21.15.5:0/283570637 wait complete. 2024-09-10T09:37:28.590 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T09:37:28.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:37:28 smithi005 ceph-mon[30350]: pgmap v367: 129 pgs: 129 active+clean; 645 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-10T09:37:28.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:37:28 smithi005 ceph-mon[30350]: from='client.? 172.21.15.5:0/283570637' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T09:37:28.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:37:28 smithi102 ceph-mon[35442]: pgmap v367: 129 pgs: 129 active+clean; 645 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-10T09:37:28.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:37:28 smithi102 ceph-mon[35442]: from='client.? 172.21.15.5:0/283570637' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T09:37:28.975 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:37:29.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:37:28 smithi159 ceph-mon[35513]: pgmap v367: 129 pgs: 129 active+clean; 645 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-10T09:37:29.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:37:28 smithi159 ceph-mon[35513]: from='client.? 172.21.15.5:0/283570637' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T09:37:29.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:29.562+0000 7ff596200640 1 -- 172.21.15.5:0/296868478 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff5901019c0 msgr2=0x7ff590101e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:37:29.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:29.562+0000 7ff596200640 1 --2- 172.21.15.5:0/296868478 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff5901019c0 0x7ff590101e20 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7ff578009930 tx=0x7ff57802a420 comp rx=0 tx=0).stop 2024-09-10T09:37:29.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:29.563+0000 7ff596200640 1 -- 172.21.15.5:0/296868478 shutdown_connections 2024-09-10T09:37:29.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:29.563+0000 7ff596200640 1 --2- 172.21.15.5:0/296868478 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff590102360 0x7ff59010e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:29.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:29.563+0000 7ff596200640 1 --2- 172.21.15.5:0/296868478 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff5901019c0 0x7ff590101e20 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:29.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:29.563+0000 7ff596200640 1 --2- 172.21.15.5:0/296868478 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff5901079c0 0x7ff590107da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:29.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:29.563+0000 7ff596200640 1 -- 172.21.15.5:0/296868478 >> 172.21.15.5:0/296868478 conn(0x7ff5900fd740 msgr2=0x7ff5900ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:37:29.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:29.563+0000 7ff596200640 1 -- 172.21.15.5:0/296868478 shutdown_connections 2024-09-10T09:37:29.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:29.563+0000 7ff596200640 1 -- 172.21.15.5:0/296868478 wait complete. 2024-09-10T09:37:29.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:29.564+0000 7ff596200640 1 Processor -- start 2024-09-10T09:37:29.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:29.564+0000 7ff596200640 1 -- start start 2024-09-10T09:37:29.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:29.564+0000 7ff596200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff5901019c0 0x7ff590198310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:37:29.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:29.564+0000 7ff596200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff590102360 0x7ff590198850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:37:29.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:29.564+0000 7ff596200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff5901079c0 0x7ff59019cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:37:29.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:29.564+0000 7ff596200640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff590111e10 con 0x7ff5901079c0 2024-09-10T09:37:29.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:29.564+0000 7ff596200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7ff590111c90 con 0x7ff5901019c0 2024-09-10T09:37:29.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:29.564+0000 7ff596200640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7ff590111f90 con 0x7ff590102360 2024-09-10T09:37:29.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:29.565+0000 7ff587e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff590102360 0x7ff590198850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:37:29.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:29.565+0000 7ff587e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff590102360 0x7ff590198850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:36524/0 (socket says 172.21.15.5:36524) 2024-09-10T09:37:29.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:29.565+0000 7ff594e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff5901019c0 0x7ff590198310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:37:29.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:29.565+0000 7ff587e00640 1 -- 172.21.15.5:0/2460736983 learned_addr learned my addr 172.21.15.5:0/2460736983 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:37:29.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:29.565+0000 7ff595800640 1 --2- 172.21.15.5:0/2460736983 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff5901079c0 0x7ff59019cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:37:29.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:29.565+0000 7ff587e00640 1 -- 172.21.15.5:0/2460736983 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff5901019c0 msgr2=0x7ff590198310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:37:29.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:29.565+0000 7ff587e00640 1 --2- 172.21.15.5:0/2460736983 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff5901019c0 0x7ff590198310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:29.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:29.565+0000 7ff587e00640 1 -- 172.21.15.5:0/2460736983 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff5901079c0 msgr2=0x7ff59019cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:37:29.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:29.565+0000 7ff587e00640 1 --2- 172.21.15.5:0/2460736983 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff5901079c0 0x7ff59019cbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:29.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:29.565+0000 7ff587e00640 1 -- 172.21.15.5:0/2460736983 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff59019d2c0 con 0x7ff590102360 2024-09-10T09:37:29.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:29.565+0000 7ff595800640 1 --2- 172.21.15.5:0/2460736983 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff5901079c0 0x7ff59019cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:37:29.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:29.565+0000 7ff594e00640 1 --2- 172.21.15.5:0/2460736983 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff5901019c0 0x7ff590198310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:37:29.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:29.566+0000 7ff587e00640 1 --2- 172.21.15.5:0/2460736983 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff590102360 0x7ff590198850 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7ff57802a940 tx=0x7ff578003b20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:37:29.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:29.566+0000 7ff58e000640 1 -- 172.21.15.5:0/2460736983 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff57803f070 con 0x7ff590102360 2024-09-10T09:37:29.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:29.566+0000 7ff58e000640 1 -- 172.21.15.5:0/2460736983 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff578031c90 con 0x7ff590102360 2024-09-10T09:37:29.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:29.566+0000 7ff596200640 1 -- 172.21.15.5:0/2460736983 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff59019d550 con 0x7ff590102360 2024-09-10T09:37:29.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:29.566+0000 7ff58e000640 1 -- 172.21.15.5:0/2460736983 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff578035030 con 0x7ff590102360 2024-09-10T09:37:29.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:29.566+0000 7ff596200640 1 -- 172.21.15.5:0/2460736983 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff590074420 con 0x7ff590102360 2024-09-10T09:37:29.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:29.568+0000 7ff596200640 1 -- 172.21.15.5:0/2460736983 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff590103100 con 0x7ff590102360 2024-09-10T09:37:29.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:29.569+0000 7ff58e000640 1 -- 172.21.15.5:0/2460736983 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7ff578042070 con 0x7ff590102360 2024-09-10T09:37:29.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:29.570+0000 7ff58e000640 1 --2- 172.21.15.5:0/2460736983 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7ff5740a2c10 0x7ff5740a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:37:29.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:29.570+0000 7ff58e000640 1 -- 172.21.15.5:0/2460736983 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7ff5780ef940 con 0x7ff590102360 2024-09-10T09:37:29.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:29.571+0000 7ff594e00640 1 --2- 172.21.15.5:0/2460736983 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7ff5740a2c10 0x7ff5740a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:37:29.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:29.571+0000 7ff594e00640 1 --2- 172.21.15.5:0/2460736983 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7ff5740a2c10 0x7ff5740a50d0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7ff580009f40 tx=0x7ff580005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:37:29.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:29.574+0000 7ff58e000640 1 -- 172.21.15.5:0/2460736983 <== mon.1 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff5780bc2f0 con 0x7ff590102360 2024-09-10T09:37:29.803 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:37:29 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:37:29.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:37:29 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:37:29.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:29.804+0000 7ff596200640 1 -- 172.21.15.5:0/2460736983 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff590109f30 con 0x7ff590102360 2024-09-10T09:37:29.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:29.806+0000 7ff58e000640 1 -- 172.21.15.5:0/2460736983 <== mon.1 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11969 (secure 0 0 0) 0x7ff5780bbbf0 con 0x7ff590102360 2024-09-10T09:37:29.808 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-10T09:37:29.808 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 29 2024-09-10T09:37:29.808 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":29,"btime":"2024-09-10T09:36:16:492171+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14454,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/2970594311","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2970594311},{"type":"v1","addr":"172.21.15.5:6835","nonce":2970594311}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14514,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6839/3616502174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":3616502174},{"type":"v1","addr":"172.21.15.5:6839","nonce":3616502174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T09:35:29.626287+0000","modified":"2024-09-10T09:36:15.764196+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24542,"mds_1":24535,"mds_2":24511,"mds_3":24506,"mds_4":24443},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14520":{"gid":14520,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/2073599842","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":2073599842},{"type":"v1","addr":"172.21.15.5:6841","nonce":2073599842}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"k","rank":4,"incarnation":26,"state":"up:active","state_seq":24,"addr":"172.21.15.102:6833/2457007427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":2457007427},{"type":"v1","addr":"172.21.15.102:6833","nonce":2457007427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/356445159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":356445159},{"type":"v1","addr":"172.21.15.159:6833","nonce":356445159}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6837/2894314507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":2894314507},{"type":"v1","addr":"172.21.15.5:6837","nonce":2894314507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.102:6835/1647549111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":1647549111},{"type":"v1","addr":"172.21.15.102:6835","nonce":1647549111}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6835/952657242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":952657242},{"type":"v1","addr":"172.21.15.159:6835","nonce":952657242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":19,"addr":"172.21.15.102:6837/2689776764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":2689776764},{"type":"v1","addr":"172.21.15.102:6837","nonce":2689776764}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.159:6837/620627639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":620627639},{"type":"v1","addr":"172.21.15.159:6837","nonce":620627639}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24535":{"gid":24535,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":18,"addr":"172.21.15.159:6839/2073739599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":2073739599},{"type":"v1","addr":"172.21.15.159:6839","nonce":2073739599}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24542":{"gid":24542,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.102:6839/335409749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6838","nonce":335409749},{"type":"v1","addr":"172.21.15.102:6839","nonce":335409749}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24542,"qdb_cluster":[24443,24506,24511,24535,24542]},"id":1}]} 2024-09-10T09:37:29.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:29.808+0000 7ff596200640 1 -- 172.21.15.5:0/2460736983 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7ff5740a2c10 msgr2=0x7ff5740a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:37:29.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:29.808+0000 7ff596200640 1 --2- 172.21.15.5:0/2460736983 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7ff5740a2c10 0x7ff5740a50d0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7ff580009f40 tx=0x7ff580005f90 comp rx=0 tx=0).stop 2024-09-10T09:37:29.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:29.808+0000 7ff596200640 1 -- 172.21.15.5:0/2460736983 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff590102360 msgr2=0x7ff590198850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:37:29.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:29.808+0000 7ff596200640 1 --2- 172.21.15.5:0/2460736983 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff590102360 0x7ff590198850 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7ff57802a940 tx=0x7ff578003b20 comp rx=0 tx=0).stop 2024-09-10T09:37:29.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:29.808+0000 7ff596200640 1 -- 172.21.15.5:0/2460736983 shutdown_connections 2024-09-10T09:37:29.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:29.808+0000 7ff596200640 1 --2- 172.21.15.5:0/2460736983 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7ff5740a2c10 0x7ff5740a50d0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:29.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:29.808+0000 7ff596200640 1 --2- 172.21.15.5:0/2460736983 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff5901079c0 0x7ff59019cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:29.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:29.808+0000 7ff596200640 1 --2- 172.21.15.5:0/2460736983 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff590102360 0x7ff590198850 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:29.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:29.808+0000 7ff596200640 1 --2- 172.21.15.5:0/2460736983 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff5901019c0 0x7ff590198310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:29.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:29.808+0000 7ff596200640 1 -- 172.21.15.5:0/2460736983 >> 172.21.15.5:0/2460736983 conn(0x7ff5900fd740 msgr2=0x7ff5900fdfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:37:29.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:29.808+0000 7ff596200640 1 -- 172.21.15.5:0/2460736983 shutdown_connections 2024-09-10T09:37:29.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:29.808+0000 7ff596200640 1 -- 172.21.15.5:0/2460736983 wait complete. 2024-09-10T09:37:29.929 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:37:29 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:37:30.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:37:30 smithi005 ceph-mon[30350]: pgmap v368: 129 pgs: 129 active+clean; 645 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-09-10T09:37:30.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:37:30 smithi005 ceph-mon[30350]: from='client.? 172.21.15.5:0/2460736983' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:37:30.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:37:30 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:37:30.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:37:30 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:37:30.807 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:37:30 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:37:30.807 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:37:30 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:37:31.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:37:30 smithi159 ceph-mon[35513]: pgmap v368: 129 pgs: 129 active+clean; 645 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-09-10T09:37:31.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:37:30 smithi159 ceph-mon[35513]: from='client.? 172.21.15.5:0/2460736983' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:37:31.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:37:30 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:37:31.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:37:30 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:37:31.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:37:30 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:37:31.101 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:37:30 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:37:31.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:37:30 smithi102 ceph-mon[35442]: pgmap v368: 129 pgs: 129 active+clean; 645 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-09-10T09:37:31.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:37:30 smithi102 ceph-mon[35442]: from='client.? 172.21.15.5:0/2460736983' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:37:31.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:37:30 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:37:31.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:37:30 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:37:31.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:37:30 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:37:31.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:37:30 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:37:31.329 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 29, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-09-10T09:35:29.626287+0000', 'modified': '2024-09-10T09:36:15.764196+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24542, 'mds_1': 24535, 'mds_2': 24511, 'mds_3': 24506, 'mds_4': 24443}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14520': {'gid': 14520, 'name': 'a', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.5:6841/2073599842', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.5:6840', 'nonce': 2073599842}, {'type': 'v1', 'addr': '172.21.15.5:6841', 'nonce': 2073599842}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24443': {'gid': 24443, 'name': 'k', 'rank': 4, 'incarnation': 26, 'state': 'up:active', 'state_seq': 24, 'addr': '172.21.15.102:6833/2457007427', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.102:6832', 'nonce': 2457007427}, {'type': 'v1', 'addr': '172.21.15.102:6833', 'nonce': 2457007427}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24460': {'gid': 24460, 'name': 'l', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.159:6833/356445159', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.159:6832', 'nonce': 356445159}, {'type': 'v1', 'addr': '172.21.15.159:6833', 'nonce': 356445159}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24467': {'gid': 24467, 'name': 'd', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.5:6837/2894314507', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.5:6836', 'nonce': 2894314507}, {'type': 'v1', 'addr': '172.21.15.5:6837', 'nonce': 2894314507}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24479': {'gid': 24479, 'name': 'e', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.102:6835/1647549111', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.102:6834', 'nonce': 1647549111}, {'type': 'v1', 'addr': '172.21.15.102:6835', 'nonce': 1647549111}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24505': {'gid': 24505, 'name': 'f', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.159:6835/952657242', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.159:6834', 'nonce': 952657242}, {'type': 'v1', 'addr': '172.21.15.159:6835', 'nonce': 952657242}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24506': {'gid': 24506, 'name': 'h', 'rank': 3, 'incarnation': 24, 'state': 'up:active', 'state_seq': 19, 'addr': '172.21.15.102:6837/2689776764', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.102:6836', 'nonce': 2689776764}, {'type': 'v1', 'addr': '172.21.15.102:6837', 'nonce': 2689776764}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24511': {'gid': 24511, 'name': 'c', 'rank': 2, 'incarnation': 22, 'state': 'up:active', 'state_seq': 19, 'addr': '172.21.15.159:6837/620627639', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.159:6836', 'nonce': 620627639}, {'type': 'v1', 'addr': '172.21.15.159:6837', 'nonce': 620627639}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24535': {'gid': 24535, 'name': 'i', 'rank': 1, 'incarnation': 20, 'state': 'up:active', 'state_seq': 18, 'addr': '172.21.15.159:6839/2073739599', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.159:6838', 'nonce': 2073739599}, {'type': 'v1', 'addr': '172.21.15.159:6839', 'nonce': 2073739599}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24542': {'gid': 24542, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.102:6839/335409749', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.102:6838', 'nonce': 335409749}, {'type': 'v1', 'addr': '172.21.15.102:6839', 'nonce': 335409749}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24542, 'qdb_cluster': [24443, 24506, 24511, 24535, 24542]} 2024-09-10T09:37:31.329 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-09-10T09:37:31.329 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph tell mds.k status 2024-09-10T09:37:31.751 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:37:32.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:32.355+0000 7fd68fe00640 1 -- 172.21.15.5:0/579669552 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd6901079c0 msgr2=0x7fd690107da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:37:32.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:32.355+0000 7fd68fe00640 1 --2- 172.21.15.5:0/579669552 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd6901079c0 0x7fd690107da0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fd680009cd0 tx=0x7fd68002a8e0 comp rx=0 tx=0).stop 2024-09-10T09:37:32.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:32.355+0000 7fd68fe00640 1 -- 172.21.15.5:0/579669552 shutdown_connections 2024-09-10T09:37:32.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:32.355+0000 7fd68fe00640 1 --2- 172.21.15.5:0/579669552 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd690102360 0x7fd69010e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:32.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:32.355+0000 7fd68fe00640 1 --2- 172.21.15.5:0/579669552 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd6901019c0 0x7fd690101e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:32.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:32.355+0000 7fd68fe00640 1 --2- 172.21.15.5:0/579669552 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd6901079c0 0x7fd690107da0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:32.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:32.355+0000 7fd68fe00640 1 -- 172.21.15.5:0/579669552 >> 172.21.15.5:0/579669552 conn(0x7fd6900fd740 msgr2=0x7fd6900ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:37:32.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:32.355+0000 7fd68fe00640 1 -- 172.21.15.5:0/579669552 shutdown_connections 2024-09-10T09:37:32.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:32.355+0000 7fd68fe00640 1 -- 172.21.15.5:0/579669552 wait complete. 2024-09-10T09:37:32.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:32.356+0000 7fd68fe00640 1 Processor -- start 2024-09-10T09:37:32.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:32.356+0000 7fd68fe00640 1 -- start start 2024-09-10T09:37:32.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:32.357+0000 7fd68fe00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd6901019c0 0x7fd69019a510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:37:32.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:32.357+0000 7fd68fe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd690102360 0x7fd69019aa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:37:32.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:32.357+0000 7fd68fe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd6901079c0 0x7fd69019ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:37:32.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:32.357+0000 7fd68fe00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd690111e10 con 0x7fd6901079c0 2024-09-10T09:37:32.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:32.357+0000 7fd68fe00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fd690111c90 con 0x7fd690102360 2024-09-10T09:37:32.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:32.357+0000 7fd68fe00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fd690111f90 con 0x7fd6901019c0 2024-09-10T09:37:32.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:32.357+0000 7fd686000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd690102360 0x7fd69019aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:37:32.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:32.357+0000 7fd686000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd690102360 0x7fd69019aa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:48476/0 (socket says 172.21.15.5:48476) 2024-09-10T09:37:32.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:32.357+0000 7fd686000640 1 -- 172.21.15.5:0/2826890165 learned_addr learned my addr 172.21.15.5:0/2826890165 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:37:32.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:32.357+0000 7fd68ea00640 1 --2- 172.21.15.5:0/2826890165 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd6901019c0 0x7fd69019a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:37:32.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:32.357+0000 7fd68f400640 1 --2- 172.21.15.5:0/2826890165 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd6901079c0 0x7fd69019ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:37:32.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:32.358+0000 7fd686000640 1 -- 172.21.15.5:0/2826890165 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd6901019c0 msgr2=0x7fd69019a510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:37:32.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:32.358+0000 7fd686000640 1 --2- 172.21.15.5:0/2826890165 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd6901019c0 0x7fd69019a510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:32.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:32.358+0000 7fd686000640 1 -- 172.21.15.5:0/2826890165 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd6901079c0 msgr2=0x7fd69019ede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:37:32.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:32.358+0000 7fd686000640 1 --2- 172.21.15.5:0/2826890165 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd6901079c0 0x7fd69019ede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:32.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:32.358+0000 7fd686000640 1 -- 172.21.15.5:0/2826890165 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd69019f4c0 con 0x7fd690102360 2024-09-10T09:37:32.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:32.358+0000 7fd686000640 1 --2- 172.21.15.5:0/2826890165 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd690102360 0x7fd69019aa50 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fd67800e9f0 tx=0x7fd67800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:37:32.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:32.358+0000 7fd687e00640 1 -- 172.21.15.5:0/2826890165 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd67800cd80 con 0x7fd690102360 2024-09-10T09:37:32.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:32.358+0000 7fd68fe00640 1 -- 172.21.15.5:0/2826890165 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd69019f7b0 con 0x7fd690102360 2024-09-10T09:37:32.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:32.358+0000 7fd687e00640 1 -- 172.21.15.5:0/2826890165 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd678010910 con 0x7fd690102360 2024-09-10T09:37:32.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:32.358+0000 7fd687e00640 1 -- 172.21.15.5:0/2826890165 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6780188f0 con 0x7fd690102360 2024-09-10T09:37:32.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:32.358+0000 7fd68fe00640 1 -- 172.21.15.5:0/2826890165 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6901a7090 con 0x7fd690102360 2024-09-10T09:37:32.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:32.361+0000 7fd697fd1740 1 Processor -- start 2024-09-10T09:37:32.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:32.362+0000 7fd687e00640 1 -- 172.21.15.5:0/2826890165 <== mon.2 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7fd678010ac0 con 0x7fd690102360 2024-09-10T09:37:32.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:32.362+0000 7fd697fd1740 1 -- start start 2024-09-10T09:37:32.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:32.362+0000 7fd697fd1740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5615e10d04e0 0x5615e10d08c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:37:32.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:32.362+0000 7fd697fd1740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5615e10d0e00 0x5615e10db180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:37:32.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:32.362+0000 7fd697fd1740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5615e10db6c0 0x5615e10ddab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:37:32.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:32.362+0000 7fd697fd1740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5615e10360b0 con 0x5615e10d0e00 2024-09-10T09:37:32.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:32.362+0000 7fd697fd1740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x5615e0fddab0 con 0x5615e10db6c0 2024-09-10T09:37:32.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:32.362+0000 7fd697fd1740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5615e1032070 con 0x5615e10d04e0 2024-09-10T09:37:32.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:32.362+0000 7fd68ea00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5615e10d0e00 0x5615e10db180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:37:32.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:32.362+0000 7fd68f400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5615e10d04e0 0x5615e10d08c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:37:32.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:32.362+0000 7fd68ea00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5615e10d0e00 0x5615e10db180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34132/0 (socket says 172.21.15.5:34132) 2024-09-10T09:37:32.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:32.362+0000 7fd68ea00640 1 -- 172.21.15.5:0/1368734089 learned_addr learned my addr 172.21.15.5:0/1368734089 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:37:32.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:32.363+0000 7fd686000640 1 --2- 172.21.15.5:0/1368734089 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5615e10db6c0 0x5615e10ddab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:37:32.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:32.363+0000 7fd68f400640 1 -- 172.21.15.5:0/1368734089 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5615e10db6c0 msgr2=0x5615e10ddab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:37:32.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:32.363+0000 7fd68f400640 1 --2- 172.21.15.5:0/1368734089 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5615e10db6c0 0x5615e10ddab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:32.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:32.363+0000 7fd68f400640 1 -- 172.21.15.5:0/1368734089 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5615e10d0e00 msgr2=0x5615e10db180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:37:32.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:32.363+0000 7fd68f400640 1 --2- 172.21.15.5:0/1368734089 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5615e10d0e00 0x5615e10db180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:32.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:32.363+0000 7fd68f400640 1 -- 172.21.15.5:0/1368734089 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5615e0fea4a0 con 0x5615e10d04e0 2024-09-10T09:37:32.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:32.363+0000 7fd686000640 1 --2- 172.21.15.5:0/1368734089 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5615e10db6c0 0x5615e10ddab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T09:37:32.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:32.363+0000 7fd68ea00640 1 --2- 172.21.15.5:0/1368734089 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5615e10d0e00 0x5615e10db180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:37:32.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:32.363+0000 7fd68f400640 1 --2- 172.21.15.5:0/1368734089 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5615e10d04e0 0x5615e10d08c0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fd688004530 tx=0x7fd68800ba50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1b0558a48be82a6b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:37:32.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:32.363+0000 7fd687e00640 1 --2- 172.21.15.5:0/2826890165 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fd6740a2c90 0x7fd6740a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:37:32.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:32.364+0000 7fd67f400640 1 -- 172.21.15.5:0/1368734089 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd688003ad0 con 0x5615e10d04e0 2024-09-10T09:37:32.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:32.364+0000 7fd687e00640 1 -- 172.21.15.5:0/2826890165 <== mon.2 v2:172.21.15.102:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fd6780cf960 con 0x7fd690102360 2024-09-10T09:37:32.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:32.364+0000 7fd68ea00640 1 --2- 172.21.15.5:0/2826890165 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fd6740a2c90 0x7fd6740a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:37:32.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:32.364+0000 7fd67f400640 1 -- 172.21.15.5:0/1368734089 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd688015070 con 0x5615e10d04e0 2024-09-10T09:37:32.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:32.364+0000 7fd67f400640 1 -- 172.21.15.5:0/1368734089 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6880106a0 con 0x5615e10d04e0 2024-09-10T09:37:32.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:32.364+0000 7fd697fd1740 1 -- 172.21.15.5:0/1368734089 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5615e10d04e0 msgr2=0x5615e10d08c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:37:32.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:32.364+0000 7fd697fd1740 1 --2- 172.21.15.5:0/1368734089 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5615e10d04e0 0x5615e10d08c0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fd688004530 tx=0x7fd68800ba50 comp rx=0 tx=0).stop 2024-09-10T09:37:32.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:32.365+0000 7fd68ea00640 1 --2- 172.21.15.5:0/2826890165 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fd6740a2c90 0x7fd6740a5150 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fd680009430 tx=0x7fd680009360 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:37:32.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:32.365+0000 7fd697fd1740 1 -- 172.21.15.5:0/1368734089 shutdown_connections 2024-09-10T09:37:32.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:32.365+0000 7fd697fd1740 1 --2- 172.21.15.5:0/1368734089 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5615e10db6c0 0x5615e10ddab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:32.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:32.365+0000 7fd697fd1740 1 --2- 172.21.15.5:0/1368734089 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5615e10d0e00 0x5615e10db180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:32.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:32.365+0000 7fd697fd1740 1 --2- 172.21.15.5:0/1368734089 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5615e10d04e0 0x5615e10d08c0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:32.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:32.365+0000 7fd697fd1740 1 -- 172.21.15.5:0/1368734089 >> 172.21.15.5:0/1368734089 conn(0x5615e10cca70 msgr2=0x5615e10cce50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:37:32.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:32.365+0000 7fd697fd1740 1 -- 172.21.15.5:0/1368734089 shutdown_connections 2024-09-10T09:37:32.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:32.365+0000 7fd697fd1740 1 -- 172.21.15.5:0/1368734089 wait complete. 2024-09-10T09:37:32.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:32.365+0000 7fd697fd1740 1 -- start start 2024-09-10T09:37:32.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:32.366+0000 7fd697fd1740 1 Processor -- start 2024-09-10T09:37:32.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:32.366+0000 7fd697fd1740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5615e10d04e0 0x5615e10d85c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:37:32.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:32.366+0000 7fd697fd1740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5615e10d0e00 0x5615e10d8b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:37:32.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:32.366+0000 7fd697fd1740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5615e10db6c0 0x5615e10d9040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:37:32.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:32.366+0000 7fd697fd1740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5615e10de330 con 0x5615e10db6c0 2024-09-10T09:37:32.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:32.366+0000 7fd697fd1740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x5615e0fbe920 con 0x5615e10d04e0 2024-09-10T09:37:32.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:32.366+0000 7fd697fd1740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5615e10de4b0 con 0x5615e10d0e00 2024-09-10T09:37:32.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:32.366+0000 7fd697fd1740 10 client.0 _ll_register_callbacks cb 0x5615e0fe28a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:37:32.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:32.366+0000 7fd68f400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5615e10db6c0 0x5615e10d9040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:37:32.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:32.366+0000 7fd68f400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5615e10db6c0 0x5615e10d9040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34146/0 (socket says 172.21.15.5:34146) 2024-09-10T09:37:32.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:32.366+0000 7fd68f400640 1 -- 172.21.15.5:0/3307433368 learned_addr learned my addr 172.21.15.5:0/3307433368 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:37:32.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:32.367+0000 7fd686000640 1 --2- 172.21.15.5:0/3307433368 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5615e10d0e00 0x5615e10d8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:37:32.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:32.367+0000 7fd68ea00640 1 --2- 172.21.15.5:0/3307433368 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5615e10d04e0 0x5615e10d85c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:37:32.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:32.367+0000 7fd68f400640 1 -- 172.21.15.5:0/3307433368 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5615e10d0e00 msgr2=0x5615e10d8b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:37:32.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:32.367+0000 7fd68f400640 1 --2- 172.21.15.5:0/3307433368 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5615e10d0e00 0x5615e10d8b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:32.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:32.367+0000 7fd68f400640 1 -- 172.21.15.5:0/3307433368 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5615e10d04e0 msgr2=0x5615e10d85c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:37:32.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:32.367+0000 7fd68f400640 1 --2- 172.21.15.5:0/3307433368 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5615e10d04e0 0x5615e10d85c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:32.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:32.367+0000 7fd68f400640 1 -- 172.21.15.5:0/3307433368 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5615e0da72b0 con 0x5615e10db6c0 2024-09-10T09:37:32.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:32.367+0000 7fd686000640 1 --2- 172.21.15.5:0/3307433368 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5615e10d0e00 0x5615e10d8b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:37:32.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:32.367+0000 7fd68ea00640 1 --2- 172.21.15.5:0/3307433368 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5615e10d04e0 0x5615e10d85c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:37:32.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:32.367+0000 7fd68f400640 1 --2- 172.21.15.5:0/3307433368 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5615e10db6c0 0x5615e10d9040 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fd688011d00 tx=0x7fd688011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f7260b6fe8ff895 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:37:32.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:32.367+0000 7fd673e00640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-10T09:37:32.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:32.368+0000 7fd673e00640 1 -- 172.21.15.5:0/3307433368 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd68802a060 con 0x5615e10db6c0 2024-09-10T09:37:32.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:32.368+0000 7fd673e00640 1 -- 172.21.15.5:0/3307433368 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fd688011710 con 0x5615e10db6c0 2024-09-10T09:37:32.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:32.368+0000 7fd697fd1740 1 -- 172.21.15.5:0/3307433368 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd688015070 con 0x5615e10db6c0 2024-09-10T09:37:32.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:32.368+0000 7fd673e00640 1 -- 172.21.15.5:0/3307433368 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fd688031050 con 0x5615e10db6c0 2024-09-10T09:37:32.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:32.368+0000 7fd697fd1740 10 client.14958 fetch_fsmap learned FSMap version 29 2024-09-10T09:37:32.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:32.368+0000 7fd697fd1740 1 -- 172.21.15.5:0/3307433368 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5615e10e9f60 con 0x5615e10db6c0 2024-09-10T09:37:32.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:32.368+0000 7fd673e00640 1 -- 172.21.15.5:0/3307433368 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd688015070 con 0x5615e10db6c0 2024-09-10T09:37:32.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:32.369+0000 7fd673e00640 1 -- 172.21.15.5:0/3307433368 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7fd68801ab10 con 0x5615e10db6c0 2024-09-10T09:37:32.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:32.369+0000 7fd697fd1740 10 client.14958 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T09:37:32.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:32.369+0000 7fd697fd1740 10 client.14958 resolve_mds: resolved name 'k' to daemon mds.k 2024-09-10T09:37:32.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:32.369+0000 7fd697fd1740 20 client.14958 populate_metadata read hostname 'smithi005' 2024-09-10T09:37:32.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:32.369+0000 7fd697fd1740 1 --2- 172.21.15.5:0/3307433368 >> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] conn(0x5615e10eb3c0 0x5615e10ed5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:37:32.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:32.369+0000 7fd697fd1740 4 client.14958 mds_command: new command op to 24443 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:37:32.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:32.369+0000 7fd697fd1740 1 -- 172.21.15.5:0/3307433368 --> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5615e1043620 con 0x5615e10eb3c0 2024-09-10T09:37:32.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:32.369+0000 7fd68ea00640 1 --2- 172.21.15.5:0/3307433368 >> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] conn(0x5615e10eb3c0 0x5615e10ed5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:37:32.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:32.370+0000 7fd68ea00640 1 --2- 172.21.15.5:0/3307433368 >> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] conn(0x5615e10eb3c0 0x5615e10ed5c0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=f67e634ded5d6d86 server_cookie=5a2a58b1bba21c8a in_seq=0 out_seq=0 2024-09-10T09:37:32.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:32.370+0000 7fd673e00640 10 client.14958 ms_handle_connect on v2:172.21.15.102:6832/2457007427 2024-09-10T09:37:32.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:32.371+0000 7fd673e00640 1 -- 172.21.15.5:0/3307433368 <== mds.4 v2:172.21.15.102:6832/2457007427 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5615e1043620 con 0x5615e10eb3c0 2024-09-10T09:37:32.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:32.371+0000 7fd673e00640 10 client.14958 handle_command_reply: tid=0 2024-09-10T09:37:32.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:32.371+0000 7fd697fd1740 1 client.14958 shutdown 2024-09-10T09:37:32.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:32.371+0000 7fd697fd1740 1 -- 172.21.15.5:0/3307433368 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5615e10db6c0 msgr2=0x5615e10d9040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:37:32.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:32.371+0000 7fd697fd1740 1 --2- 172.21.15.5:0/3307433368 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5615e10db6c0 0x5615e10d9040 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fd688011d00 tx=0x7fd688011a30 comp rx=0 tx=0).stop 2024-09-10T09:37:32.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:32.371+0000 7fd697fd1740 1 -- 172.21.15.5:0/3307433368 shutdown_connections 2024-09-10T09:37:32.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:32.371+0000 7fd697fd1740 1 --2- 172.21.15.5:0/3307433368 >> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] conn(0x5615e10eb3c0 0x5615e10ed5c0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:32.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:32.371+0000 7fd697fd1740 1 --2- 172.21.15.5:0/3307433368 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5615e10db6c0 0x5615e10d9040 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:32.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:32.371+0000 7fd697fd1740 1 --2- 172.21.15.5:0/3307433368 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5615e10d0e00 0x5615e10d8b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:32.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:32.371+0000 7fd697fd1740 1 --2- 172.21.15.5:0/3307433368 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5615e10d04e0 0x5615e10d85c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:32.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:32.371+0000 7fd697fd1740 1 -- 172.21.15.5:0/3307433368 >> 172.21.15.5:0/3307433368 conn(0x5615e10cca70 msgr2=0x5615e10d27a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:37:32.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:32.371+0000 7fd673e00640 0 client.14958 ms_handle_reset on v2:172.21.15.102:6832/2457007427 2024-09-10T09:37:32.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:32.372+0000 7fd697fd1740 1 -- 172.21.15.5:0/3307433368 shutdown_connections 2024-09-10T09:37:32.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:32.372+0000 7fd697fd1740 1 -- 172.21.15.5:0/3307433368 wait complete. 2024-09-10T09:37:32.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:32.372+0000 7fd697fd1740 20 client.14958 trim_cache size 0 max 16384 2024-09-10T09:37:32.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:32.395+0000 7fd697fd1740 1 Processor -- start 2024-09-10T09:37:32.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:32.395+0000 7fd697fd1740 1 -- start start 2024-09-10T09:37:32.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:32.396+0000 7fd697fd1740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5615e10d04e0 0x5615e10ec150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:37:32.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:32.396+0000 7fd697fd1740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5615e10d0e00 0x5615e10d2eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:37:32.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:32.396+0000 7fd697fd1740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5615e10db6c0 0x5615e10d8e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:37:32.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:32.396+0000 7fd697fd1740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5615e10eb900 con 0x5615e10d0e00 2024-09-10T09:37:32.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:32.396+0000 7fd697fd1740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x5615e10ebfd0 con 0x5615e10d04e0 2024-09-10T09:37:32.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:32.396+0000 7fd697fd1740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5615e10ec6a0 con 0x5615e10db6c0 2024-09-10T09:37:32.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:32.396+0000 7fd68ea00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5615e10d04e0 0x5615e10ec150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:37:32.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:32.396+0000 7fd686000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5615e10d0e00 0x5615e10d2eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:37:32.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:32.396+0000 7fd68ea00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5615e10d04e0 0x5615e10ec150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:48494/0 (socket says 172.21.15.5:48494) 2024-09-10T09:37:32.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:32.396+0000 7fd68ea00640 1 -- 172.21.15.5:0/713316906 learned_addr learned my addr 172.21.15.5:0/713316906 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:37:32.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:32.397+0000 7fd68f400640 1 --2- 172.21.15.5:0/713316906 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5615e10db6c0 0x5615e10d8e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:37:32.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:32.397+0000 7fd686000640 1 -- 172.21.15.5:0/713316906 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5615e10db6c0 msgr2=0x5615e10d8e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:37:32.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:32.397+0000 7fd686000640 1 --2- 172.21.15.5:0/713316906 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5615e10db6c0 0x5615e10d8e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:32.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:32.397+0000 7fd686000640 1 -- 172.21.15.5:0/713316906 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5615e10d04e0 msgr2=0x5615e10ec150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:37:32.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:32.397+0000 7fd686000640 1 --2- 172.21.15.5:0/713316906 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5615e10d04e0 0x5615e10ec150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:32.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:32.397+0000 7fd686000640 1 -- 172.21.15.5:0/713316906 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5615e10daf30 con 0x5615e10d0e00 2024-09-10T09:37:32.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:32.397+0000 7fd68ea00640 1 --2- 172.21.15.5:0/713316906 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5615e10d04e0 0x5615e10ec150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:37:32.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:32.397+0000 7fd686000640 1 --2- 172.21.15.5:0/713316906 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5615e10d0e00 0x5615e10d2eb0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fd6780ccec0 tx=0x7fd67809bd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fc40e36a1f3ba24 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:37:32.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:32.397+0000 7fd67ea00640 1 -- 172.21.15.5:0/713316906 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6780cd110 con 0x5615e10d0e00 2024-09-10T09:37:32.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:32.398+0000 7fd67ea00640 1 -- 172.21.15.5:0/713316906 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd6780cd2b0 con 0x5615e10d0e00 2024-09-10T09:37:32.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:32.398+0000 7fd67ea00640 1 -- 172.21.15.5:0/713316906 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd67809ab80 con 0x5615e10d0e00 2024-09-10T09:37:32.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:32.398+0000 7fd697fd1740 1 -- 172.21.15.5:0/713316906 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5615e10d0e00 msgr2=0x5615e10d2eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:37:32.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:32.398+0000 7fd697fd1740 1 --2- 172.21.15.5:0/713316906 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5615e10d0e00 0x5615e10d2eb0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fd6780ccec0 tx=0x7fd67809bd40 comp rx=0 tx=0).stop 2024-09-10T09:37:32.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:32.398+0000 7fd697fd1740 1 -- 172.21.15.5:0/713316906 shutdown_connections 2024-09-10T09:37:32.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:32.398+0000 7fd697fd1740 1 --2- 172.21.15.5:0/713316906 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5615e10db6c0 0x5615e10d8e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:32.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:32.398+0000 7fd697fd1740 1 --2- 172.21.15.5:0/713316906 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5615e10d0e00 0x5615e10d2eb0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:32.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:32.398+0000 7fd697fd1740 1 --2- 172.21.15.5:0/713316906 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5615e10d04e0 0x5615e10ec150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:32.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:32.398+0000 7fd697fd1740 1 -- 172.21.15.5:0/713316906 >> 172.21.15.5:0/713316906 conn(0x5615e10cca70 msgr2=0x5615e10d9e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:37:32.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:32.399+0000 7fd697fd1740 1 -- 172.21.15.5:0/713316906 shutdown_connections 2024-09-10T09:37:32.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:32.399+0000 7fd697fd1740 1 -- 172.21.15.5:0/713316906 wait complete. 2024-09-10T09:37:32.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:32.399+0000 7fd697fd1740 1 -- start start 2024-09-10T09:37:32.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:32.399+0000 7fd697fd1740 1 Processor -- start 2024-09-10T09:37:32.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:32.400+0000 7fd697fd1740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5615e10d04e0 0x5615e10d5070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:37:32.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:32.400+0000 7fd697fd1740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5615e10d0e00 0x5615e10d55b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:37:32.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:32.400+0000 7fd697fd1740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5615e10db6c0 0x5615e10f45e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:37:32.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:32.400+0000 7fd697fd1740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5615e10ed440 con 0x5615e10db6c0 2024-09-10T09:37:32.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:32.400+0000 7fd697fd1740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x5615e10ecd70 con 0x5615e10d04e0 2024-09-10T09:37:32.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:32.400+0000 7fd697fd1740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5615e10edb10 con 0x5615e10d0e00 2024-09-10T09:37:32.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:32.400+0000 7fd697fd1740 10 client.0 _ll_register_callbacks cb 0x7fd6880056d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:37:32.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:32.400+0000 7fd68f400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5615e10db6c0 0x5615e10f45e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:37:32.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:32.400+0000 7fd68f400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5615e10db6c0 0x5615e10f45e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34164/0 (socket says 172.21.15.5:34164) 2024-09-10T09:37:32.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:32.400+0000 7fd68f400640 1 -- 172.21.15.5:0/3881077662 learned_addr learned my addr 172.21.15.5:0/3881077662 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:37:32.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:32.400+0000 7fd686000640 1 --2- 172.21.15.5:0/3881077662 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5615e10d0e00 0x5615e10d55b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:37:32.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:32.400+0000 7fd68ea00640 1 --2- 172.21.15.5:0/3881077662 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5615e10d04e0 0x5615e10d5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:37:32.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:32.401+0000 7fd68f400640 1 -- 172.21.15.5:0/3881077662 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5615e10d0e00 msgr2=0x5615e10d55b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:37:32.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:32.401+0000 7fd68f400640 1 --2- 172.21.15.5:0/3881077662 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5615e10d0e00 0x5615e10d55b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:32.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:32.401+0000 7fd68f400640 1 -- 172.21.15.5:0/3881077662 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5615e10d04e0 msgr2=0x5615e10d5070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:37:32.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:32.401+0000 7fd68f400640 1 --2- 172.21.15.5:0/3881077662 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5615e10d04e0 0x5615e10d5070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:32.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:32.401+0000 7fd68f400640 1 -- 172.21.15.5:0/3881077662 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5615e10f4c20 con 0x5615e10db6c0 2024-09-10T09:37:32.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:32.401+0000 7fd686000640 1 --2- 172.21.15.5:0/3881077662 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5615e10d0e00 0x5615e10d55b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:37:32.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:32.401+0000 7fd68ea00640 1 --2- 172.21.15.5:0/3881077662 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5615e10d04e0 0x5615e10d5070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:37:32.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:32.401+0000 7fd68f400640 1 --2- 172.21.15.5:0/3881077662 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5615e10db6c0 0x5615e10f45e0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fd688011d70 tx=0x7fd6880106c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ee2466a1772f6bf server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:37:32.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:32.401+0000 7fd673e00640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-10T09:37:32.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:32.401+0000 7fd673e00640 1 -- 172.21.15.5:0/3881077662 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd68802a040 con 0x5615e10db6c0 2024-09-10T09:37:32.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:32.401+0000 7fd697fd1740 1 -- 172.21.15.5:0/3881077662 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5615e103db30 con 0x5615e10db6c0 2024-09-10T09:37:32.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:32.401+0000 7fd673e00640 1 -- 172.21.15.5:0/3881077662 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fd688022040 con 0x5615e10db6c0 2024-09-10T09:37:32.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:32.402+0000 7fd673e00640 1 -- 172.21.15.5:0/3881077662 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fd688042050 con 0x5615e10db6c0 2024-09-10T09:37:32.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:32.402+0000 7fd697fd1740 10 client.14964 fetch_fsmap learned FSMap version 29 2024-09-10T09:37:32.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:32.402+0000 7fd697fd1740 1 -- 172.21.15.5:0/3881077662 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5615e10f5fe0 con 0x5615e10db6c0 2024-09-10T09:37:32.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:32.402+0000 7fd673e00640 1 -- 172.21.15.5:0/3881077662 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x5615e103db30 con 0x5615e10db6c0 2024-09-10T09:37:32.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:32.402+0000 7fd673e00640 1 -- 172.21.15.5:0/3881077662 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7fd6880038d0 con 0x5615e10db6c0 2024-09-10T09:37:32.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:32.402+0000 7fd697fd1740 10 client.14964 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T09:37:32.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:32.402+0000 7fd697fd1740 10 client.14964 resolve_mds: resolved name 'k' to daemon mds.k 2024-09-10T09:37:32.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:32.402+0000 7fd697fd1740 20 client.14964 populate_metadata read hostname 'smithi005' 2024-09-10T09:37:32.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:32.402+0000 7fd697fd1740 1 --2- 172.21.15.5:0/3881077662 >> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] conn(0x5615e115ac40 0x5615e1159950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:37:32.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:32.402+0000 7fd697fd1740 4 client.14964 mds_command: new command op to 24443 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T09:37:32.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:32.402+0000 7fd697fd1740 1 -- 172.21.15.5:0/3881077662 --> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] -- command(tid 0: {"prefix": "status"}) -- 0x5615e0fb9f80 con 0x5615e115ac40 2024-09-10T09:37:32.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:32.403+0000 7fd68ea00640 1 --2- 172.21.15.5:0/3881077662 >> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] conn(0x5615e115ac40 0x5615e1159950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:37:32.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:32.403+0000 7fd68ea00640 1 --2- 172.21.15.5:0/3881077662 >> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] conn(0x5615e115ac40 0x5615e1159950 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=a54396735b890cf3 server_cookie=ce4b4561a64656c7 in_seq=0 out_seq=0 2024-09-10T09:37:32.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:32.403+0000 7fd673e00640 10 client.14964 ms_handle_connect on v2:172.21.15.102:6832/2457007427 2024-09-10T09:37:32.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:32.403+0000 7fd673e00640 1 -- 172.21.15.5:0/3881077662 <== mds.4 v2:172.21.15.102:6832/2457007427 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x5615e0fb9f80 con 0x5615e115ac40 2024-09-10T09:37:32.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:32.403+0000 7fd673e00640 10 client.14964 handle_command_reply: tid=0 2024-09-10T09:37:32.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:32.403+0000 7fd697fd1740 1 client.14964 shutdown 2024-09-10T09:37:32.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:32.404+0000 7fd697fd1740 1 -- 172.21.15.5:0/3881077662 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5615e10db6c0 msgr2=0x5615e10f45e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:37:32.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:32.404+0000 7fd697fd1740 1 --2- 172.21.15.5:0/3881077662 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5615e10db6c0 0x5615e10f45e0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fd688011d70 tx=0x7fd6880106c0 comp rx=0 tx=0).stop 2024-09-10T09:37:32.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:32.404+0000 7fd697fd1740 1 -- 172.21.15.5:0/3881077662 shutdown_connections 2024-09-10T09:37:32.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:32.404+0000 7fd697fd1740 1 --2- 172.21.15.5:0/3881077662 >> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] conn(0x5615e115ac40 0x5615e1159950 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:32.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:32.404+0000 7fd697fd1740 1 --2- 172.21.15.5:0/3881077662 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5615e10db6c0 0x5615e10f45e0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:32.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:32.404+0000 7fd697fd1740 1 --2- 172.21.15.5:0/3881077662 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5615e10d0e00 0x5615e10d55b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:32.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:32.404+0000 7fd697fd1740 1 --2- 172.21.15.5:0/3881077662 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5615e10d04e0 0x5615e10d5070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:32.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:32.404+0000 7fd673e00640 0 client.14964 ms_handle_reset on v2:172.21.15.102:6832/2457007427 2024-09-10T09:37:32.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:32.404+0000 7fd697fd1740 1 -- 172.21.15.5:0/3881077662 >> 172.21.15.5:0/3881077662 conn(0x5615e10cca70 msgr2=0x5615e10e6f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:37:32.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:32.404+0000 7fd697fd1740 1 -- 172.21.15.5:0/3881077662 shutdown_connections 2024-09-10T09:37:32.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:32.404+0000 7fd697fd1740 1 -- 172.21.15.5:0/3881077662 wait complete. 2024-09-10T09:37:32.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:32.404+0000 7fd697fd1740 20 client.14964 trim_cache size 0 max 16384 2024-09-10T09:37:32.409 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-10T09:37:32.409 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_fsid": "bade4b50-6f56-11ef-bcea-c7b262605968", 2024-09-10T09:37:32.410 INFO:teuthology.orchestra.run.smithi005.stdout: "whoami": 4, 2024-09-10T09:37:32.410 INFO:teuthology.orchestra.run.smithi005.stdout: "id": 24443, 2024-09-10T09:37:32.410 INFO:teuthology.orchestra.run.smithi005.stdout: "want_state": "up:active", 2024-09-10T09:37:32.410 INFO:teuthology.orchestra.run.smithi005.stdout: "state": "up:active", 2024-09-10T09:37:32.410 INFO:teuthology.orchestra.run.smithi005.stdout: "fs_name": "cephfs", 2024-09-10T09:37:32.410 INFO:teuthology.orchestra.run.smithi005.stdout: "rank_uptime": 77.917745046999997, 2024-09-10T09:37:32.410 INFO:teuthology.orchestra.run.smithi005.stdout: "mdsmap_epoch": 29, 2024-09-10T09:37:32.410 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch": 96, 2024-09-10T09:37:32.410 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch_barrier": 94, 2024-09-10T09:37:32.410 INFO:teuthology.orchestra.run.smithi005.stdout: "uptime": 167.45152636700001 2024-09-10T09:37:32.410 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-10T09:37:32.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:32.405+0000 7fd68fe00640 1 -- 172.21.15.5:0/2826890165 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fd6740a2c90 msgr2=0x7fd6740a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:37:32.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:32.405+0000 7fd68fe00640 1 --2- 172.21.15.5:0/2826890165 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fd6740a2c90 0x7fd6740a5150 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fd680009430 tx=0x7fd680009360 comp rx=0 tx=0).stop 2024-09-10T09:37:32.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:32.405+0000 7fd68fe00640 1 -- 172.21.15.5:0/2826890165 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd690102360 msgr2=0x7fd69019aa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:37:32.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:32.405+0000 7fd68fe00640 1 --2- 172.21.15.5:0/2826890165 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd690102360 0x7fd69019aa50 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fd67800e9f0 tx=0x7fd67800eeb0 comp rx=0 tx=0).stop 2024-09-10T09:37:32.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:32.405+0000 7fd68fe00640 1 -- 172.21.15.5:0/2826890165 shutdown_connections 2024-09-10T09:37:32.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:32.405+0000 7fd68fe00640 1 --2- 172.21.15.5:0/2826890165 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fd6740a2c90 0x7fd6740a5150 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:32.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:32.405+0000 7fd68fe00640 1 --2- 172.21.15.5:0/2826890165 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd6901079c0 0x7fd69019ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:32.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:32.405+0000 7fd68fe00640 1 --2- 172.21.15.5:0/2826890165 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd690102360 0x7fd69019aa50 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:32.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:32.405+0000 7fd68fe00640 1 --2- 172.21.15.5:0/2826890165 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd6901019c0 0x7fd69019a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:32.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:32.405+0000 7fd68fe00640 1 -- 172.21.15.5:0/2826890165 >> 172.21.15.5:0/2826890165 conn(0x7fd6900fd740 msgr2=0x7fd6900fed50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:37:32.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:32.405+0000 7fd68fe00640 1 -- 172.21.15.5:0/2826890165 shutdown_connections 2024-09-10T09:37:32.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:32.405+0000 7fd68fe00640 1 -- 172.21.15.5:0/2826890165 wait complete. 2024-09-10T09:37:32.925 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:37:32 smithi005 ceph-mon[30350]: pgmap v369: 129 pgs: 129 active+clean; 645 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-09-10T09:37:33.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:37:32 smithi159 ceph-mon[35513]: pgmap v369: 129 pgs: 129 active+clean; 645 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-09-10T09:37:33.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:37:32 smithi102 ceph-mon[35442]: pgmap v369: 129 pgs: 129 active+clean; 645 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-09-10T09:37:33.226 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph tell mds.h status 2024-09-10T09:37:33.619 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:37:34.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:37:33 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:37:34.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:37:33 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:37:34.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:37:33 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:37:34.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:37:33 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:37:34.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:37:33 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:37:34.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:37:33 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:37:34.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:37:33 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:37:34.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:37:33 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:37:34.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:37:33 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:37:34.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:37:33 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:37:34.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:37:33 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:37:34.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:37:33 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:37:34.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:37:33 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:37:34.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:37:33 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:37:34.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:37:33 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:37:34.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:34.182+0000 7f6afbc00640 1 -- 172.21.15.5:0/1267206644 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6af4077040 msgr2=0x7f6af4112510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:37:34.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:34.182+0000 7f6afbc00640 1 --2- 172.21.15.5:0/1267206644 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6af4077040 0x7f6af4112510 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f6ae0009900 tx=0x7f6ae002a3f0 comp rx=0 tx=0).stop 2024-09-10T09:37:34.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:34.182+0000 7f6afbc00640 1 -- 172.21.15.5:0/1267206644 shutdown_connections 2024-09-10T09:37:34.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:34.182+0000 7f6afbc00640 1 --2- 172.21.15.5:0/1267206644 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6af4112a50 0x7f6af4114e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:34.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:34.182+0000 7f6afbc00640 1 --2- 172.21.15.5:0/1267206644 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6af4077040 0x7f6af4112510 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:34.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:34.182+0000 7f6afbc00640 1 --2- 172.21.15.5:0/1267206644 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6af4076720 0x7f6af4076b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:34.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:34.182+0000 7f6afbc00640 1 -- 172.21.15.5:0/1267206644 >> 172.21.15.5:0/1267206644 conn(0x7f6af40ffa50 msgr2=0x7f6af4101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:37:34.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:34.182+0000 7f6afbc00640 1 -- 172.21.15.5:0/1267206644 shutdown_connections 2024-09-10T09:37:34.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:34.182+0000 7f6afbc00640 1 -- 172.21.15.5:0/1267206644 wait complete. 2024-09-10T09:37:34.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:34.183+0000 7f6afbc00640 1 Processor -- start 2024-09-10T09:37:34.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:34.183+0000 7f6afbc00640 1 -- start start 2024-09-10T09:37:34.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:34.183+0000 7f6afbc00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6af4076720 0x7f6af41a4710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:37:34.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:34.183+0000 7f6afbc00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6af4077040 0x7f6af41a4c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:37:34.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:34.183+0000 7f6afbc00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6af4112a50 0x7f6af419e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:37:34.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:34.183+0000 7f6afbc00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f6af41185c0 con 0x7f6af4076720 2024-09-10T09:37:34.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:34.183+0000 7f6afbc00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f6af4118440 con 0x7f6af4112a50 2024-09-10T09:37:34.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:34.184+0000 7f6afbc00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f6af4118740 con 0x7f6af4077040 2024-09-10T09:37:34.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:34.184+0000 7f6af9200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6af4076720 0x7f6af41a4710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:37:34.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:34.184+0000 7f6af9200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6af4076720 0x7f6af41a4710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34178/0 (socket says 172.21.15.5:34178) 2024-09-10T09:37:34.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:34.184+0000 7f6af9200640 1 -- 172.21.15.5:0/60670963 learned_addr learned my addr 172.21.15.5:0/60670963 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:37:34.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:34.184+0000 7f6af3e00640 1 --2- 172.21.15.5:0/60670963 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6af4077040 0x7f6af41a4c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:37:34.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:34.184+0000 7f6af9c00640 1 --2- 172.21.15.5:0/60670963 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6af4112a50 0x7f6af419e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:37:34.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:34.184+0000 7f6af9200640 1 -- 172.21.15.5:0/60670963 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6af4077040 msgr2=0x7f6af41a4c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:37:34.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:34.184+0000 7f6af9200640 1 --2- 172.21.15.5:0/60670963 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6af4077040 0x7f6af41a4c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:34.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:34.184+0000 7f6af9200640 1 -- 172.21.15.5:0/60670963 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6af4112a50 msgr2=0x7f6af419e8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:37:34.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:34.184+0000 7f6af9200640 1 --2- 172.21.15.5:0/60670963 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6af4112a50 0x7f6af419e8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:34.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:34.184+0000 7f6af9200640 1 -- 172.21.15.5:0/60670963 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6af419f000 con 0x7f6af4076720 2024-09-10T09:37:34.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:34.185+0000 7f6af9200640 1 --2- 172.21.15.5:0/60670963 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6af4076720 0x7f6af41a4710 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f6ae800e9f0 tx=0x7f6ae800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:37:34.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:34.185+0000 7f6af1600640 1 -- 172.21.15.5:0/60670963 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ae800cd80 con 0x7f6af4076720 2024-09-10T09:37:34.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:34.185+0000 7f6af1600640 1 -- 172.21.15.5:0/60670963 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6ae8010910 con 0x7f6af4076720 2024-09-10T09:37:34.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:34.185+0000 7f6af1600640 1 -- 172.21.15.5:0/60670963 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ae8018990 con 0x7f6af4076720 2024-09-10T09:37:34.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:34.185+0000 7f6afbc00640 1 -- 172.21.15.5:0/60670963 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6af419f2f0 con 0x7f6af4076720 2024-09-10T09:37:34.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:34.185+0000 7f6afbc00640 1 -- 172.21.15.5:0/60670963 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6af41ab5e0 con 0x7f6af4076720 2024-09-10T09:37:34.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:34.188+0000 7f6afe61c740 1 Processor -- start 2024-09-10T09:37:34.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:34.188+0000 7f6af1600640 1 -- 172.21.15.5:0/60670963 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f6ae8010ac0 con 0x7f6af4076720 2024-09-10T09:37:34.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:34.189+0000 7f6afe61c740 1 -- start start 2024-09-10T09:37:34.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:34.189+0000 7f6afe61c740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5642e5cac330 0x5642e5cac710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:37:34.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:34.189+0000 7f6afe61c740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5642e5cacc50 0x5642e5cb6fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:37:34.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:34.189+0000 7f6afe61c740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5642e5cb7510 0x5642e5cb9900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:37:34.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:34.189+0000 7f6afe61c740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5642e5c120b0 con 0x5642e5cacc50 2024-09-10T09:37:34.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:34.189+0000 7f6afe61c740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x5642e5bb9ab0 con 0x5642e5cb7510 2024-09-10T09:37:34.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:34.189+0000 7f6afe61c740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5642e5c0e070 con 0x5642e5cac330 2024-09-10T09:37:34.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:34.189+0000 7f6af9c00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5642e5cac330 0x5642e5cac710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:37:34.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:34.189+0000 7f6af9c00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5642e5cac330 0x5642e5cac710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:44920/0 (socket says 172.21.15.5:44920) 2024-09-10T09:37:34.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:34.189+0000 7f6af9c00640 1 -- 172.21.15.5:0/732571144 learned_addr learned my addr 172.21.15.5:0/732571144 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:37:34.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:34.189+0000 7f6af9200640 1 --2- 172.21.15.5:0/732571144 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5642e5cacc50 0x5642e5cb6fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:37:34.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:34.190+0000 7f6af3e00640 1 --2- 172.21.15.5:0/732571144 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5642e5cb7510 0x5642e5cb9900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:37:34.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:34.190+0000 7f6af9c00640 1 -- 172.21.15.5:0/732571144 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5642e5cb7510 msgr2=0x5642e5cb9900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:37:34.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:34.190+0000 7f6af9c00640 1 --2- 172.21.15.5:0/732571144 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5642e5cb7510 0x5642e5cb9900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:34.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:34.190+0000 7f6af9c00640 1 -- 172.21.15.5:0/732571144 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5642e5cacc50 msgr2=0x5642e5cb6fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:37:34.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:34.190+0000 7f6af9c00640 1 --2- 172.21.15.5:0/732571144 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5642e5cacc50 0x5642e5cb6fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:34.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:34.190+0000 7f6af9c00640 1 -- 172.21.15.5:0/732571144 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5642e5bc64a0 con 0x5642e5cac330 2024-09-10T09:37:34.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:34.190+0000 7f6af9200640 1 --2- 172.21.15.5:0/732571144 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5642e5cacc50 0x5642e5cb6fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:37:34.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:34.190+0000 7f6af9c00640 1 --2- 172.21.15.5:0/732571144 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5642e5cac330 0x5642e5cac710 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f6aec009f20 tx=0x7f6aec010860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2cb575b7c7b19eb3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:37:34.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:34.190+0000 7f6af1600640 1 --2- 172.21.15.5:0/60670963 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f6adc0a2c90 0x7f6adc0a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:37:34.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:34.191+0000 7f6ae6000640 1 -- 172.21.15.5:0/732571144 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6aec003c30 con 0x5642e5cac330 2024-09-10T09:37:34.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:34.191+0000 7f6af1600640 1 -- 172.21.15.5:0/60670963 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f6ae80cf890 con 0x7f6af4076720 2024-09-10T09:37:34.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:34.191+0000 7f6af9200640 1 --2- 172.21.15.5:0/60670963 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f6adc0a2c90 0x7f6adc0a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:37:34.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:34.191+0000 7f6ae6000640 1 -- 172.21.15.5:0/732571144 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6aec015070 con 0x5642e5cac330 2024-09-10T09:37:34.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:34.191+0000 7f6ae6000640 1 -- 172.21.15.5:0/732571144 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6aec011880 con 0x5642e5cac330 2024-09-10T09:37:34.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:34.191+0000 7f6afe61c740 1 -- 172.21.15.5:0/732571144 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5642e5cac330 msgr2=0x5642e5cac710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:37:34.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:34.191+0000 7f6af9200640 1 --2- 172.21.15.5:0/60670963 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f6adc0a2c90 0x7f6adc0a5150 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f6ae800c6a0 tx=0x7f6ae80d5530 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:37:34.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:34.191+0000 7f6afe61c740 1 --2- 172.21.15.5:0/732571144 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5642e5cac330 0x5642e5cac710 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f6aec009f20 tx=0x7f6aec010860 comp rx=0 tx=0).stop 2024-09-10T09:37:34.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:34.192+0000 7f6afe61c740 1 -- 172.21.15.5:0/732571144 shutdown_connections 2024-09-10T09:37:34.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:34.192+0000 7f6afe61c740 1 --2- 172.21.15.5:0/732571144 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5642e5cb7510 0x5642e5cb9900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:34.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:34.192+0000 7f6afe61c740 1 --2- 172.21.15.5:0/732571144 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5642e5cacc50 0x5642e5cb6fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:34.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:34.192+0000 7f6afe61c740 1 --2- 172.21.15.5:0/732571144 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5642e5cac330 0x5642e5cac710 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:34.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:34.192+0000 7f6afe61c740 1 -- 172.21.15.5:0/732571144 >> 172.21.15.5:0/732571144 conn(0x5642e5ca88c0 msgr2=0x5642e5ca8ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:37:34.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:34.192+0000 7f6afe61c740 1 -- 172.21.15.5:0/732571144 shutdown_connections 2024-09-10T09:37:34.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:34.192+0000 7f6afe61c740 1 -- 172.21.15.5:0/732571144 wait complete. 2024-09-10T09:37:34.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:34.192+0000 7f6afe61c740 1 -- start start 2024-09-10T09:37:34.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:34.192+0000 7f6afe61c740 1 Processor -- start 2024-09-10T09:37:34.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:34.193+0000 7f6afe61c740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5642e5cac330 0x5642e5cb45f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:37:34.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:34.193+0000 7f6afe61c740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5642e5cacc50 0x5642e5cb4b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:37:34.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:34.193+0000 7f6afe61c740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5642e5cb7510 0x5642e5cb5070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:37:34.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:34.193+0000 7f6afe61c740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5642e5cba060 con 0x5642e5cb7510 2024-09-10T09:37:34.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:34.193+0000 7f6afe61c740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x5642e5b9a920 con 0x5642e5cac330 2024-09-10T09:37:34.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:34.193+0000 7f6afe61c740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5642e5cba1e0 con 0x5642e5cacc50 2024-09-10T09:37:34.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:34.193+0000 7f6afe61c740 10 client.0 _ll_register_callbacks cb 0x5642e5bbe8a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:37:34.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:34.193+0000 7f6af9c00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5642e5cb7510 0x5642e5cb5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:37:34.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:34.193+0000 7f6af3e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5642e5cacc50 0x5642e5cb4b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:37:34.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:34.193+0000 7f6af3e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5642e5cacc50 0x5642e5cb4b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:44934/0 (socket says 172.21.15.5:44934) 2024-09-10T09:37:34.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:34.193+0000 7f6af3e00640 1 -- 172.21.15.5:0/1534225824 learned_addr learned my addr 172.21.15.5:0/1534225824 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:37:34.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:34.193+0000 7f6af9c00640 1 --2- 172.21.15.5:0/1534225824 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5642e5cb7510 0x5642e5cb5070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34194/0 (socket says 172.21.15.5:34194) 2024-09-10T09:37:34.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:34.194+0000 7f6af9c00640 1 --2- 172.21.15.5:0/1534225824 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5642e5cac330 0x5642e5cb45f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:37:34.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:34.194+0000 7f6af3e00640 1 -- 172.21.15.5:0/1534225824 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5642e5cac330 msgr2=0x5642e5cb45f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:37:34.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:34.194+0000 7f6af3e00640 1 --2- 172.21.15.5:0/1534225824 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5642e5cac330 0x5642e5cb45f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:34.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:34.194+0000 7f6af3e00640 1 -- 172.21.15.5:0/1534225824 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5642e5cb7510 msgr2=0x5642e5cb5070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:37:34.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:34.194+0000 7f6af3e00640 1 --2- 172.21.15.5:0/1534225824 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5642e5cb7510 0x5642e5cb5070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:34.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:34.194+0000 7f6af3e00640 1 -- 172.21.15.5:0/1534225824 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5642e59832b0 con 0x5642e5cacc50 2024-09-10T09:37:34.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:34.194+0000 7f6af9c00640 1 --2- 172.21.15.5:0/1534225824 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5642e5cb7510 0x5642e5cb5070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:37:34.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:34.194+0000 7f6af3e00640 1 --2- 172.21.15.5:0/1534225824 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5642e5cacc50 0x5642e5cb4b30 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f6ae002a910 tx=0x7f6ae0031850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f0436799f183891a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:37:34.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:34.194+0000 7f6adaa00640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-09-10T09:37:34.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:34.195+0000 7f6adaa00640 1 -- 172.21.15.5:0/1534225824 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ae0035040 con 0x5642e5cacc50 2024-09-10T09:37:34.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:34.195+0000 7f6afe61c740 1 -- 172.21.15.5:0/1534225824 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6aec015070 con 0x5642e5cacc50 2024-09-10T09:37:34.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:34.195+0000 7f6adaa00640 1 -- 172.21.15.5:0/1534225824 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f6ae0004240 con 0x5642e5cacc50 2024-09-10T09:37:34.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:34.195+0000 7f6adaa00640 1 -- 172.21.15.5:0/1534225824 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f6ae004f050 con 0x5642e5cacc50 2024-09-10T09:37:34.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:34.195+0000 7f6afe61c740 10 client.25012 fetch_fsmap learned FSMap version 29 2024-09-10T09:37:34.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:34.195+0000 7f6afe61c740 1 -- 172.21.15.5:0/1534225824 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5642e5cc5e40 con 0x5642e5cacc50 2024-09-10T09:37:34.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:34.195+0000 7f6adaa00640 1 -- 172.21.15.5:0/1534225824 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6aec015070 con 0x5642e5cacc50 2024-09-10T09:37:34.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:34.195+0000 7f6adaa00640 1 -- 172.21.15.5:0/1534225824 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f6ae0004aa0 con 0x5642e5cacc50 2024-09-10T09:37:34.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:34.196+0000 7f6afe61c740 10 client.25012 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T09:37:34.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:34.196+0000 7f6afe61c740 10 client.25012 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-10T09:37:34.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:34.196+0000 7f6afe61c740 20 client.25012 populate_metadata read hostname 'smithi005' 2024-09-10T09:37:34.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:34.196+0000 7f6afe61c740 1 --2- 172.21.15.5:0/1534225824 >> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] conn(0x5642e5cc8400 0x5642e5cc9870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:37:34.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:34.196+0000 7f6afe61c740 4 client.25012 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:37:34.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:34.196+0000 7f6afe61c740 1 -- 172.21.15.5:0/1534225824 --> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5642e5c1f620 con 0x5642e5cc8400 2024-09-10T09:37:34.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:34.196+0000 7f6af9c00640 1 --2- 172.21.15.5:0/1534225824 >> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] conn(0x5642e5cc8400 0x5642e5cc9870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:37:34.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:34.197+0000 7f6af9c00640 1 --2- 172.21.15.5:0/1534225824 >> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] conn(0x5642e5cc8400 0x5642e5cc9870 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=c9947f76c229db03 server_cookie=e150a3da652ba444 in_seq=0 out_seq=0 2024-09-10T09:37:34.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:34.197+0000 7f6adaa00640 10 client.25012 ms_handle_connect on v2:172.21.15.102:6836/2689776764 2024-09-10T09:37:34.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:34.197+0000 7f6adaa00640 1 -- 172.21.15.5:0/1534225824 <== mds.3 v2:172.21.15.102:6836/2689776764 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5642e5c1f620 con 0x5642e5cc8400 2024-09-10T09:37:34.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:34.197+0000 7f6adaa00640 10 client.25012 handle_command_reply: tid=0 2024-09-10T09:37:34.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:34.197+0000 7f6afe61c740 1 client.25012 shutdown 2024-09-10T09:37:34.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:34.198+0000 7f6afe61c740 1 -- 172.21.15.5:0/1534225824 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5642e5cacc50 msgr2=0x5642e5cb4b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:37:34.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:34.198+0000 7f6afe61c740 1 --2- 172.21.15.5:0/1534225824 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5642e5cacc50 0x5642e5cb4b30 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f6ae002a910 tx=0x7f6ae0031850 comp rx=0 tx=0).stop 2024-09-10T09:37:34.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:34.198+0000 7f6afe61c740 1 -- 172.21.15.5:0/1534225824 shutdown_connections 2024-09-10T09:37:34.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:34.198+0000 7f6afe61c740 1 --2- 172.21.15.5:0/1534225824 >> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] conn(0x5642e5cc8400 0x5642e5cc9870 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:34.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:34.198+0000 7f6afe61c740 1 --2- 172.21.15.5:0/1534225824 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5642e5cb7510 0x5642e5cb5070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:34.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:34.198+0000 7f6afe61c740 1 --2- 172.21.15.5:0/1534225824 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5642e5cacc50 0x5642e5cb4b30 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:34.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:34.198+0000 7f6adaa00640 0 client.25012 ms_handle_reset on v2:172.21.15.102:6836/2689776764 2024-09-10T09:37:34.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:34.198+0000 7f6afe61c740 1 --2- 172.21.15.5:0/1534225824 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5642e5cac330 0x5642e5cb45f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:34.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:34.198+0000 7f6afe61c740 1 -- 172.21.15.5:0/1534225824 >> 172.21.15.5:0/1534225824 conn(0x5642e5ca88c0 msgr2=0x5642e5cae5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:37:34.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:34.198+0000 7f6afe61c740 1 -- 172.21.15.5:0/1534225824 shutdown_connections 2024-09-10T09:37:34.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:34.198+0000 7f6afe61c740 1 -- 172.21.15.5:0/1534225824 wait complete. 2024-09-10T09:37:34.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:34.198+0000 7f6afe61c740 20 client.25012 trim_cache size 0 max 16384 2024-09-10T09:37:34.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:34.217+0000 7f6afe61c740 1 Processor -- start 2024-09-10T09:37:34.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:34.217+0000 7f6afe61c740 1 -- start start 2024-09-10T09:37:34.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:34.217+0000 7f6afe61c740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5642e5cac330 0x5642e5cb2f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:37:34.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:34.217+0000 7f6afe61c740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5642e5cacc50 0x5642e5cb3460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:37:34.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:34.217+0000 7f6afe61c740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5642e5cb7510 0x5642e5cb18d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:37:34.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:34.217+0000 7f6afe61c740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5642e5cc7bb0 con 0x5642e5cb7510 2024-09-10T09:37:34.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:34.217+0000 7f6afe61c740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x5642e5cc8280 con 0x5642e5cac330 2024-09-10T09:37:34.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:34.217+0000 7f6afe61c740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5642e5cc8950 con 0x5642e5cacc50 2024-09-10T09:37:34.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:34.218+0000 7f6af9c00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5642e5cb7510 0x5642e5cb18d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:37:34.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:34.218+0000 7f6af3e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5642e5cacc50 0x5642e5cb3460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:37:34.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:34.218+0000 7f6af9c00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5642e5cb7510 0x5642e5cb18d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34198/0 (socket says 172.21.15.5:34198) 2024-09-10T09:37:34.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:34.218+0000 7f6af9c00640 1 -- 172.21.15.5:0/1185947204 learned_addr learned my addr 172.21.15.5:0/1185947204 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:37:34.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:34.218+0000 7f6af9c00640 1 --2- 172.21.15.5:0/1185947204 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5642e5cac330 0x5642e5cb2f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:37:34.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:34.218+0000 7f6af3e00640 1 -- 172.21.15.5:0/1185947204 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5642e5cac330 msgr2=0x5642e5cb2f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:37:34.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:34.218+0000 7f6af3e00640 1 --2- 172.21.15.5:0/1185947204 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5642e5cac330 0x5642e5cb2f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:34.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:34.218+0000 7f6af3e00640 1 -- 172.21.15.5:0/1185947204 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5642e5cb7510 msgr2=0x5642e5cb18d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:37:34.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:34.218+0000 7f6af3e00640 1 --2- 172.21.15.5:0/1185947204 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5642e5cb7510 0x5642e5cb18d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:34.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:34.218+0000 7f6af3e00640 1 -- 172.21.15.5:0/1185947204 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5642e5cac120 con 0x5642e5cacc50 2024-09-10T09:37:34.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:34.218+0000 7f6af9c00640 1 --2- 172.21.15.5:0/1185947204 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5642e5cb7510 0x5642e5cb18d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:37:34.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:34.218+0000 7f6af3e00640 1 --2- 172.21.15.5:0/1185947204 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5642e5cacc50 0x5642e5cb3460 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f6ae00096c0 tx=0x7f6ae002a3f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=af3f6415188c6e05 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:37:34.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:34.219+0000 7f6ae5600640 1 -- 172.21.15.5:0/1185947204 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ae0035070 con 0x5642e5cacc50 2024-09-10T09:37:34.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:34.219+0000 7f6ae5600640 1 -- 172.21.15.5:0/1185947204 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6ae00046c0 con 0x5642e5cacc50 2024-09-10T09:37:34.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:34.219+0000 7f6ae5600640 1 -- 172.21.15.5:0/1185947204 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ae004c040 con 0x5642e5cacc50 2024-09-10T09:37:34.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:34.219+0000 7f6afe61c740 1 -- 172.21.15.5:0/1185947204 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5642e5cacc50 msgr2=0x5642e5cb3460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:37:34.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:34.219+0000 7f6afe61c740 1 --2- 172.21.15.5:0/1185947204 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5642e5cacc50 0x5642e5cb3460 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f6ae00096c0 tx=0x7f6ae002a3f0 comp rx=0 tx=0).stop 2024-09-10T09:37:34.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:34.220+0000 7f6afe61c740 1 -- 172.21.15.5:0/1185947204 shutdown_connections 2024-09-10T09:37:34.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:34.220+0000 7f6afe61c740 1 --2- 172.21.15.5:0/1185947204 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5642e5cb7510 0x5642e5cb18d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:34.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:34.220+0000 7f6afe61c740 1 --2- 172.21.15.5:0/1185947204 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5642e5cacc50 0x5642e5cb3460 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:34.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:34.220+0000 7f6afe61c740 1 --2- 172.21.15.5:0/1185947204 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5642e5cac330 0x5642e5cb2f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:34.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:34.220+0000 7f6afe61c740 1 -- 172.21.15.5:0/1185947204 >> 172.21.15.5:0/1185947204 conn(0x5642e5ca88c0 msgr2=0x5642e5ca8ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:37:34.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:34.220+0000 7f6afe61c740 1 -- 172.21.15.5:0/1185947204 shutdown_connections 2024-09-10T09:37:34.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:34.220+0000 7f6afe61c740 1 -- 172.21.15.5:0/1185947204 wait complete. 2024-09-10T09:37:34.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:34.220+0000 7f6afe61c740 1 -- start start 2024-09-10T09:37:34.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:34.220+0000 7f6afe61c740 1 Processor -- start 2024-09-10T09:37:34.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:34.221+0000 7f6afe61c740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5642e5cac330 0x5642e5cb1860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:37:34.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:34.221+0000 7f6afe61c740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5642e5cacc50 0x5642e5cb1da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:37:34.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:34.221+0000 7f6afe61c740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5642e5cb7510 0x5642e5cb22e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:37:34.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:34.221+0000 7f6afe61c740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5642e5cc96f0 con 0x5642e5cac330 2024-09-10T09:37:34.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:34.221+0000 7f6afe61c740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x5642e5cc9020 con 0x5642e5cacc50 2024-09-10T09:37:34.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:34.221+0000 7f6afe61c740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5642e5cc9dc0 con 0x5642e5cb7510 2024-09-10T09:37:34.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:34.221+0000 7f6afe61c740 10 client.0 _ll_register_callbacks cb 0x7f6ae00095f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:37:34.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:34.221+0000 7f6af9c00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5642e5cac330 0x5642e5cb1860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:37:34.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:34.221+0000 7f6af3e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5642e5cacc50 0x5642e5cb1da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:37:34.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:34.221+0000 7f6af9c00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5642e5cac330 0x5642e5cb1860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34200/0 (socket says 172.21.15.5:34200) 2024-09-10T09:37:34.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:34.221+0000 7f6af9c00640 1 -- 172.21.15.5:0/3497532696 learned_addr learned my addr 172.21.15.5:0/3497532696 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:37:34.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:34.221+0000 7f6af9c00640 1 --2- 172.21.15.5:0/3497532696 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5642e5cb7510 0x5642e5cb22e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:37:34.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:34.222+0000 7f6af3e00640 1 -- 172.21.15.5:0/3497532696 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5642e5cb7510 msgr2=0x5642e5cb22e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:37:34.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:34.222+0000 7f6af3e00640 1 --2- 172.21.15.5:0/3497532696 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5642e5cb7510 0x5642e5cb22e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:34.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:34.222+0000 7f6af3e00640 1 -- 172.21.15.5:0/3497532696 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5642e5cac330 msgr2=0x5642e5cb1860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:37:34.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:34.222+0000 7f6af3e00640 1 --2- 172.21.15.5:0/3497532696 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5642e5cac330 0x5642e5cb1860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:34.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:34.222+0000 7f6af3e00640 1 -- 172.21.15.5:0/3497532696 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5642e5cd0480 con 0x5642e5cacc50 2024-09-10T09:37:34.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:34.222+0000 7f6af9c00640 1 --2- 172.21.15.5:0/3497532696 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5642e5cac330 0x5642e5cb1860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:37:34.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:34.222+0000 7f6af3e00640 1 --2- 172.21.15.5:0/3497532696 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5642e5cacc50 0x5642e5cb1da0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x5642e5caee90 tx=0x7f6ae0005bd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=82bf5e22b87f2f21 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:37:34.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:34.222+0000 7f6adaa00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-09-10T09:37:34.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:34.222+0000 7f6adaa00640 1 -- 172.21.15.5:0/3497532696 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ae0035040 con 0x5642e5cacc50 2024-09-10T09:37:34.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:34.222+0000 7f6afe61c740 1 -- 172.21.15.5:0/3497532696 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6ae00046c0 con 0x5642e5cacc50 2024-09-10T09:37:34.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:34.222+0000 7f6adaa00640 1 -- 172.21.15.5:0/3497532696 <== mon.2 v2:172.21.15.102:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f6ae0005000 con 0x5642e5cacc50 2024-09-10T09:37:34.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:34.223+0000 7f6adaa00640 1 -- 172.21.15.5:0/3497532696 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f6ae006c050 con 0x5642e5cacc50 2024-09-10T09:37:34.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:34.223+0000 7f6afe61c740 10 client.25001 fetch_fsmap learned FSMap version 29 2024-09-10T09:37:34.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:34.223+0000 7f6afe61c740 1 -- 172.21.15.5:0/3497532696 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5642e5d32f80 con 0x5642e5cacc50 2024-09-10T09:37:34.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:34.223+0000 7f6adaa00640 1 -- 172.21.15.5:0/3497532696 <== mon.2 v2:172.21.15.102:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6ae00046c0 con 0x5642e5cacc50 2024-09-10T09:37:34.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:34.223+0000 7f6adaa00640 1 -- 172.21.15.5:0/3497532696 <== mon.2 v2:172.21.15.102:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f6ae0046b00 con 0x5642e5cacc50 2024-09-10T09:37:34.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:34.223+0000 7f6afe61c740 10 client.25001 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T09:37:34.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:34.223+0000 7f6afe61c740 10 client.25001 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-10T09:37:34.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:34.223+0000 7f6afe61c740 20 client.25001 populate_metadata read hostname 'smithi005' 2024-09-10T09:37:34.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:34.223+0000 7f6afe61c740 1 --2- 172.21.15.5:0/3497532696 >> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] conn(0x5642e5d376c0 0x5642e5d363d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:37:34.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:34.223+0000 7f6afe61c740 4 client.25001 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T09:37:34.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:34.223+0000 7f6afe61c740 1 -- 172.21.15.5:0/3497532696 --> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] -- command(tid 0: {"prefix": "status"}) -- 0x5642e5c19b30 con 0x5642e5d376c0 2024-09-10T09:37:34.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:34.224+0000 7f6af9c00640 1 --2- 172.21.15.5:0/3497532696 >> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] conn(0x5642e5d376c0 0x5642e5d363d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:37:34.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:34.224+0000 7f6af9c00640 1 --2- 172.21.15.5:0/3497532696 >> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] conn(0x5642e5d376c0 0x5642e5d363d0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=d6e49d1d1a81c22 server_cookie=83e9d6ef4a96c0f0 in_seq=0 out_seq=0 2024-09-10T09:37:34.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:34.224+0000 7f6adaa00640 10 client.25001 ms_handle_connect on v2:172.21.15.102:6836/2689776764 2024-09-10T09:37:34.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:34.224+0000 7f6adaa00640 1 -- 172.21.15.5:0/3497532696 <== mds.3 v2:172.21.15.102:6836/2689776764 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x5642e5c19b30 con 0x5642e5d376c0 2024-09-10T09:37:34.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:34.224+0000 7f6adaa00640 10 client.25001 handle_command_reply: tid=0 2024-09-10T09:37:34.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:34.225+0000 7f6afe61c740 1 client.25001 shutdown 2024-09-10T09:37:34.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:34.225+0000 7f6afe61c740 1 -- 172.21.15.5:0/3497532696 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5642e5cacc50 msgr2=0x5642e5cb1da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:37:34.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:34.225+0000 7f6afe61c740 1 --2- 172.21.15.5:0/3497532696 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5642e5cacc50 0x5642e5cb1da0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x5642e5caee90 tx=0x7f6ae0005bd0 comp rx=0 tx=0).stop 2024-09-10T09:37:34.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:34.225+0000 7f6afe61c740 1 -- 172.21.15.5:0/3497532696 shutdown_connections 2024-09-10T09:37:34.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:34.225+0000 7f6afe61c740 1 --2- 172.21.15.5:0/3497532696 >> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] conn(0x5642e5d376c0 0x5642e5d363d0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:34.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:34.225+0000 7f6afe61c740 1 --2- 172.21.15.5:0/3497532696 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5642e5cb7510 0x5642e5cb22e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:34.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:34.225+0000 7f6afe61c740 1 --2- 172.21.15.5:0/3497532696 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5642e5cacc50 0x5642e5cb1da0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:34.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:34.225+0000 7f6afe61c740 1 --2- 172.21.15.5:0/3497532696 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5642e5cac330 0x5642e5cb1860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:34.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:34.225+0000 7f6afe61c740 1 -- 172.21.15.5:0/3497532696 >> 172.21.15.5:0/3497532696 conn(0x5642e5ca88c0 msgr2=0x5642e5cb2e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:37:34.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:34.225+0000 7f6adaa00640 0 client.25001 ms_handle_reset on v2:172.21.15.102:6836/2689776764 2024-09-10T09:37:34.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:34.225+0000 7f6afe61c740 1 -- 172.21.15.5:0/3497532696 shutdown_connections 2024-09-10T09:37:34.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:34.225+0000 7f6afe61c740 1 -- 172.21.15.5:0/3497532696 wait complete. 2024-09-10T09:37:34.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:34.225+0000 7f6afe61c740 20 client.25001 trim_cache size 0 max 16384 2024-09-10T09:37:34.230 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-10T09:37:34.231 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_fsid": "bade4b50-6f56-11ef-bcea-c7b262605968", 2024-09-10T09:37:34.231 INFO:teuthology.orchestra.run.smithi005.stdout: "whoami": 3, 2024-09-10T09:37:34.231 INFO:teuthology.orchestra.run.smithi005.stdout: "id": 24506, 2024-09-10T09:37:34.231 INFO:teuthology.orchestra.run.smithi005.stdout: "want_state": "up:active", 2024-09-10T09:37:34.231 INFO:teuthology.orchestra.run.smithi005.stdout: "state": "up:active", 2024-09-10T09:37:34.231 INFO:teuthology.orchestra.run.smithi005.stdout: "fs_name": "cephfs", 2024-09-10T09:37:34.231 INFO:teuthology.orchestra.run.smithi005.stdout: "rank_uptime": 80.418432408000001, 2024-09-10T09:37:34.231 INFO:teuthology.orchestra.run.smithi005.stdout: "mdsmap_epoch": 29, 2024-09-10T09:37:34.231 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch": 96, 2024-09-10T09:37:34.231 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch_barrier": 94, 2024-09-10T09:37:34.231 INFO:teuthology.orchestra.run.smithi005.stdout: "uptime": 150.14154897500001 2024-09-10T09:37:34.231 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-10T09:37:34.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:34.226+0000 7f6afbc00640 1 -- 172.21.15.5:0/60670963 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f6adc0a2c90 msgr2=0x7f6adc0a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:37:34.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:34.226+0000 7f6afbc00640 1 --2- 172.21.15.5:0/60670963 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f6adc0a2c90 0x7f6adc0a5150 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f6ae800c6a0 tx=0x7f6ae80d5530 comp rx=0 tx=0).stop 2024-09-10T09:37:34.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:34.226+0000 7f6afbc00640 1 -- 172.21.15.5:0/60670963 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6af4076720 msgr2=0x7f6af41a4710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:37:34.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:34.226+0000 7f6afbc00640 1 --2- 172.21.15.5:0/60670963 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6af4076720 0x7f6af41a4710 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f6ae800e9f0 tx=0x7f6ae800eeb0 comp rx=0 tx=0).stop 2024-09-10T09:37:34.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:34.226+0000 7f6afbc00640 1 -- 172.21.15.5:0/60670963 shutdown_connections 2024-09-10T09:37:34.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:34.226+0000 7f6afbc00640 1 --2- 172.21.15.5:0/60670963 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f6adc0a2c90 0x7f6adc0a5150 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:34.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:34.226+0000 7f6afbc00640 1 --2- 172.21.15.5:0/60670963 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6af4112a50 0x7f6af419e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:34.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:34.226+0000 7f6afbc00640 1 --2- 172.21.15.5:0/60670963 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6af4077040 0x7f6af41a4c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:34.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:34.226+0000 7f6afbc00640 1 --2- 172.21.15.5:0/60670963 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6af4076720 0x7f6af41a4710 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:34.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:34.226+0000 7f6afbc00640 1 -- 172.21.15.5:0/60670963 >> 172.21.15.5:0/60670963 conn(0x7f6af40ffa50 msgr2=0x7f6af4101e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:37:34.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:34.226+0000 7f6afbc00640 1 -- 172.21.15.5:0/60670963 shutdown_connections 2024-09-10T09:37:34.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:34.227+0000 7f6afbc00640 1 -- 172.21.15.5:0/60670963 wait complete. 2024-09-10T09:37:34.962 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:37:34 smithi005 ceph-mon[30350]: pgmap v370: 129 pgs: 129 active+clean; 645 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-10T09:37:35.001 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph tell mds.c status 2024-09-10T09:37:35.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:37:34 smithi159 ceph-mon[35513]: pgmap v370: 129 pgs: 129 active+clean; 645 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-10T09:37:35.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:37:34 smithi102 ceph-mon[35442]: pgmap v370: 129 pgs: 129 active+clean; 645 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-10T09:37:35.380 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:37:35.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:35.989+0000 7f2d6dc00640 1 -- 172.21.15.5:0/1352654808 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2d68108f60 msgr2=0x7f2d68110ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:37:35.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:35.989+0000 7f2d6dc00640 1 --2- 172.21.15.5:0/1352654808 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2d68108f60 0x7f2d68110ae0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f2d60009ca0 tx=0x7f2d6002a8c0 comp rx=0 tx=0).stop 2024-09-10T09:37:35.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:35.989+0000 7f2d6dc00640 1 -- 172.21.15.5:0/1352654808 shutdown_connections 2024-09-10T09:37:35.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:35.989+0000 7f2d6dc00640 1 --2- 172.21.15.5:0/1352654808 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2d68108f60 0x7f2d68110ae0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:35.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:35.989+0000 7f2d6dc00640 1 --2- 172.21.15.5:0/1352654808 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2d681047e0 0x7f2d68108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:35.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:35.989+0000 7f2d6dc00640 1 --2- 172.21.15.5:0/1352654808 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2d68103e30 0x7f2d68104210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:35.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:35.989+0000 7f2d6dc00640 1 -- 172.21.15.5:0/1352654808 >> 172.21.15.5:0/1352654808 conn(0x7f2d680ff910 msgr2=0x7f2d68101d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:37:35.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:35.989+0000 7f2d6dc00640 1 -- 172.21.15.5:0/1352654808 shutdown_connections 2024-09-10T09:37:35.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:35.989+0000 7f2d6dc00640 1 -- 172.21.15.5:0/1352654808 wait complete. 2024-09-10T09:37:35.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:35.990+0000 7f2d6dc00640 1 Processor -- start 2024-09-10T09:37:35.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:35.990+0000 7f2d6dc00640 1 -- start start 2024-09-10T09:37:35.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:35.990+0000 7f2d6dc00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2d68103e30 0x7f2d6819a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:37:35.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:35.991+0000 7f2d6dc00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2d681047e0 0x7f2d6819a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:37:35.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:35.991+0000 7f2d6dc00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2d68108f60 0x7f2d6819ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:37:35.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:35.991+0000 7f2d6dc00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f2d68113fc0 con 0x7f2d68108f60 2024-09-10T09:37:35.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:35.991+0000 7f2d6dc00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f2d68113e40 con 0x7f2d68103e30 2024-09-10T09:37:35.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:35.991+0000 7f2d6dc00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f2d68114140 con 0x7f2d681047e0 2024-09-10T09:37:35.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:35.991+0000 7f2d6d200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2d68108f60 0x7f2d6819ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:37:35.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:35.991+0000 7f2d67e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2d68103e30 0x7f2d6819a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:37:35.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:35.991+0000 7f2d67400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2d681047e0 0x7f2d6819a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:37:35.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:35.991+0000 7f2d6d200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2d68108f60 0x7f2d6819ed70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34212/0 (socket says 172.21.15.5:34212) 2024-09-10T09:37:35.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:35.991+0000 7f2d67e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2d68103e30 0x7f2d6819a4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:48604/0 (socket says 172.21.15.5:48604) 2024-09-10T09:37:35.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:35.991+0000 7f2d67e00640 1 -- 172.21.15.5:0/3615119023 learned_addr learned my addr 172.21.15.5:0/3615119023 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:37:35.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:35.991+0000 7f2d67e00640 1 -- 172.21.15.5:0/3615119023 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2d681047e0 msgr2=0x7f2d6819a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:37:35.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:35.991+0000 7f2d67e00640 1 --2- 172.21.15.5:0/3615119023 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2d681047e0 0x7f2d6819a9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:35.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:35.991+0000 7f2d67e00640 1 -- 172.21.15.5:0/3615119023 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2d68108f60 msgr2=0x7f2d6819ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:37:35.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:35.991+0000 7f2d67e00640 1 --2- 172.21.15.5:0/3615119023 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2d68108f60 0x7f2d6819ed70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:35.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:35.991+0000 7f2d67e00640 1 -- 172.21.15.5:0/3615119023 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d6819f450 con 0x7f2d68103e30 2024-09-10T09:37:35.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:35.992+0000 7f2d67e00640 1 --2- 172.21.15.5:0/3615119023 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2d68103e30 0x7f2d6819a4a0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f2d5800e9f0 tx=0x7f2d5800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:37:35.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:35.992+0000 7f2d64c00640 1 -- 172.21.15.5:0/3615119023 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d5800cd80 con 0x7f2d68103e30 2024-09-10T09:37:35.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:35.992+0000 7f2d64c00640 1 -- 172.21.15.5:0/3615119023 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2d58010910 con 0x7f2d68103e30 2024-09-10T09:37:35.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:35.992+0000 7f2d6dc00640 1 -- 172.21.15.5:0/3615119023 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d6819f740 con 0x7f2d68103e30 2024-09-10T09:37:35.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:35.992+0000 7f2d64c00640 1 -- 172.21.15.5:0/3615119023 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d58018990 con 0x7f2d68103e30 2024-09-10T09:37:35.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:35.992+0000 7f2d6dc00640 1 -- 172.21.15.5:0/3615119023 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d681a7020 con 0x7f2d68103e30 2024-09-10T09:37:35.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:35.995+0000 7f2d64c00640 1 -- 172.21.15.5:0/3615119023 <== mon.2 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f2d58010ac0 con 0x7f2d68103e30 2024-09-10T09:37:35.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:35.995+0000 7f2d71e0f740 1 Processor -- start 2024-09-10T09:37:35.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:35.995+0000 7f2d71e0f740 1 -- start start 2024-09-10T09:37:35.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:35.996+0000 7f2d71e0f740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56407869b4e0 0x56407869b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:37:35.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:35.996+0000 7f2d71e0f740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56407869be00 0x5640786a6180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:37:35.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:35.996+0000 7f2d71e0f740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5640786a66c0 0x5640786a8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:37:35.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:35.996+0000 7f2d71e0f740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5640786010b0 con 0x56407869b4e0 2024-09-10T09:37:35.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:35.996+0000 7f2d71e0f740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x5640785a8ab0 con 0x56407869be00 2024-09-10T09:37:35.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:35.996+0000 7f2d71e0f740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5640785fd070 con 0x5640786a66c0 2024-09-10T09:37:35.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:35.996+0000 7f2d6d200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56407869b4e0 0x56407869b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:37:35.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:35.996+0000 7f2d67e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56407869be00 0x5640786a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:37:35.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:35.996+0000 7f2d6d200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56407869b4e0 0x56407869b8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34216/0 (socket says 172.21.15.5:34216) 2024-09-10T09:37:35.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:35.996+0000 7f2d6d200640 1 -- 172.21.15.5:0/1308781487 learned_addr learned my addr 172.21.15.5:0/1308781487 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:37:35.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:35.996+0000 7f2d67400640 1 --2- 172.21.15.5:0/1308781487 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5640786a66c0 0x5640786a8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:37:36.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:35.997+0000 7f2d6d200640 1 -- 172.21.15.5:0/1308781487 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5640786a66c0 msgr2=0x5640786a8ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:37:36.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:35.997+0000 7f2d6d200640 1 --2- 172.21.15.5:0/1308781487 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5640786a66c0 0x5640786a8ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:36.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:35.997+0000 7f2d6d200640 1 -- 172.21.15.5:0/1308781487 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56407869be00 msgr2=0x5640786a6180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:37:36.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:35.997+0000 7f2d6d200640 1 --2- 172.21.15.5:0/1308781487 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56407869be00 0x5640786a6180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:36.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:35.997+0000 7f2d6d200640 1 -- 172.21.15.5:0/1308781487 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5640785b54a0 con 0x56407869b4e0 2024-09-10T09:37:36.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:35.997+0000 7f2d67e00640 1 --2- 172.21.15.5:0/1308781487 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56407869be00 0x5640786a6180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:37:36.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:35.997+0000 7f2d64c00640 1 --2- 172.21.15.5:0/3615119023 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f2d4c0a2b80 0x7f2d4c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:37:36.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:35.997+0000 7f2d64c00640 1 -- 172.21.15.5:0/3615119023 <== mon.2 v2:172.21.15.102:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f2d580cf7a0 con 0x7f2d68103e30 2024-09-10T09:37:36.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:35.997+0000 7f2d67e00640 1 --2- 172.21.15.5:0/3615119023 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f2d4c0a2b80 0x7f2d4c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:37:36.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:35.997+0000 7f2d6d200640 1 --2- 172.21.15.5:0/1308781487 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56407869b4e0 0x56407869b8c0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f2d60009ca0 tx=0x7f2d60005f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52af065af6d9c2b7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:37:36.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:35.998+0000 7f2d5d600640 1 -- 172.21.15.5:0/1308781487 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d60031db0 con 0x56407869b4e0 2024-09-10T09:37:36.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:35.998+0000 7f2d5d600640 1 -- 172.21.15.5:0/1308781487 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2d60036040 con 0x56407869b4e0 2024-09-10T09:37:36.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:35.998+0000 7f2d5d600640 1 -- 172.21.15.5:0/1308781487 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d600042d0 con 0x56407869b4e0 2024-09-10T09:37:36.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:35.998+0000 7f2d71e0f740 1 -- 172.21.15.5:0/1308781487 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56407869b4e0 msgr2=0x56407869b8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:37:36.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:35.998+0000 7f2d67e00640 1 --2- 172.21.15.5:0/3615119023 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f2d4c0a2b80 0x7f2d4c0a5040 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f2d5800c6a0 tx=0x7f2d580d5530 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:37:36.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:35.998+0000 7f2d71e0f740 1 --2- 172.21.15.5:0/1308781487 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56407869b4e0 0x56407869b8c0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f2d60009ca0 tx=0x7f2d60005f10 comp rx=0 tx=0).stop 2024-09-10T09:37:36.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:35.998+0000 7f2d71e0f740 1 -- 172.21.15.5:0/1308781487 shutdown_connections 2024-09-10T09:37:36.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:35.998+0000 7f2d71e0f740 1 --2- 172.21.15.5:0/1308781487 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5640786a66c0 0x5640786a8ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:36.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:35.998+0000 7f2d71e0f740 1 --2- 172.21.15.5:0/1308781487 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56407869be00 0x5640786a6180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:36.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:35.998+0000 7f2d71e0f740 1 --2- 172.21.15.5:0/1308781487 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56407869b4e0 0x56407869b8c0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:36.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:35.999+0000 7f2d71e0f740 1 -- 172.21.15.5:0/1308781487 >> 172.21.15.5:0/1308781487 conn(0x564078697a70 msgr2=0x564078697e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:37:36.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:35.999+0000 7f2d71e0f740 1 -- 172.21.15.5:0/1308781487 shutdown_connections 2024-09-10T09:37:36.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:35.999+0000 7f2d71e0f740 1 -- 172.21.15.5:0/1308781487 wait complete. 2024-09-10T09:37:36.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:35.999+0000 7f2d71e0f740 1 -- start start 2024-09-10T09:37:36.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:35.999+0000 7f2d71e0f740 1 Processor -- start 2024-09-10T09:37:36.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:36.000+0000 7f2d71e0f740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56407869b4e0 0x5640786a3500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:37:36.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:36.000+0000 7f2d71e0f740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56407869be00 0x5640786a3a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:37:36.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:36.000+0000 7f2d71e0f740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5640786a66c0 0x5640786a3f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:37:36.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:36.000+0000 7f2d71e0f740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5640786a91b0 con 0x5640786a66c0 2024-09-10T09:37:36.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:36.000+0000 7f2d71e0f740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x564078589920 con 0x56407869be00 2024-09-10T09:37:36.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:36.000+0000 7f2d71e0f740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5640786a9330 con 0x56407869b4e0 2024-09-10T09:37:36.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:36.000+0000 7f2d71e0f740 10 client.0 _ll_register_callbacks cb 0x5640785ad8a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:37:36.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:36.000+0000 7f2d6d200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56407869b4e0 0x5640786a3500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:37:36.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:36.000+0000 7f2d67400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56407869be00 0x5640786a3a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:37:36.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:36.000+0000 7f2d6d200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56407869b4e0 0x5640786a3500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:44980/0 (socket says 172.21.15.5:44980) 2024-09-10T09:37:36.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:36.000+0000 7f2d6d200640 1 -- 172.21.15.5:0/1602966714 learned_addr learned my addr 172.21.15.5:0/1602966714 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:37:36.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:36.000+0000 7f2d6d200640 1 --2- 172.21.15.5:0/1602966714 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5640786a66c0 0x5640786a3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:37:36.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:36.001+0000 7f2d67400640 1 -- 172.21.15.5:0/1602966714 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56407869b4e0 msgr2=0x5640786a3500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:37:36.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:36.001+0000 7f2d67400640 1 --2- 172.21.15.5:0/1602966714 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56407869b4e0 0x5640786a3500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:36.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:36.001+0000 7f2d67400640 1 -- 172.21.15.5:0/1602966714 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5640786a66c0 msgr2=0x5640786a3f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:37:36.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:36.001+0000 7f2d67400640 1 --2- 172.21.15.5:0/1602966714 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5640786a66c0 0x5640786a3f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:36.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:36.001+0000 7f2d67400640 1 -- 172.21.15.5:0/1602966714 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5640783722b0 con 0x56407869be00 2024-09-10T09:37:36.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:36.001+0000 7f2d67400640 1 --2- 172.21.15.5:0/1602966714 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56407869be00 0x5640786a3a40 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f2d6810a820 tx=0x7f2d54012b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=798451ead62cb3d2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:37:36.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:36.001+0000 7f2d52000640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-09-10T09:37:36.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:36.001+0000 7f2d52000640 1 -- 172.21.15.5:0/1602966714 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d5400f920 con 0x56407869be00 2024-09-10T09:37:36.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:36.001+0000 7f2d52000640 1 -- 172.21.15.5:0/1602966714 <== mon.2 v2:172.21.15.102:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f2d5400fac0 con 0x56407869be00 2024-09-10T09:37:36.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:36.001+0000 7f2d71e0f740 1 -- 172.21.15.5:0/1602966714 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2d60036040 con 0x56407869be00 2024-09-10T09:37:36.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:36.002+0000 7f2d52000640 1 -- 172.21.15.5:0/1602966714 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f2d54025050 con 0x56407869be00 2024-09-10T09:37:36.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:36.002+0000 7f2d71e0f740 10 client.25013 fetch_fsmap learned FSMap version 29 2024-09-10T09:37:36.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:36.002+0000 7f2d71e0f740 1 -- 172.21.15.5:0/1602966714 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5640786b4e60 con 0x56407869be00 2024-09-10T09:37:36.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:36.002+0000 7f2d52000640 1 -- 172.21.15.5:0/1602966714 <== mon.2 v2:172.21.15.102:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2d60036040 con 0x56407869be00 2024-09-10T09:37:36.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:36.002+0000 7f2d52000640 1 -- 172.21.15.5:0/1602966714 <== mon.2 v2:172.21.15.102:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f2d54010430 con 0x56407869be00 2024-09-10T09:37:36.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:36.002+0000 7f2d71e0f740 10 client.25013 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T09:37:36.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:36.002+0000 7f2d71e0f740 10 client.25013 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-10T09:37:36.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:36.002+0000 7f2d71e0f740 20 client.25013 populate_metadata read hostname 'smithi005' 2024-09-10T09:37:36.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:36.002+0000 7f2d71e0f740 1 --2- 172.21.15.5:0/1602966714 >> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] conn(0x5640786b7490 0x5640786b8900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:37:36.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:36.002+0000 7f2d71e0f740 4 client.25013 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:37:36.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:36.002+0000 7f2d71e0f740 1 -- 172.21.15.5:0/1602966714 --> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56407860e620 con 0x5640786b7490 2024-09-10T09:37:36.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:36.003+0000 7f2d6d200640 1 --2- 172.21.15.5:0/1602966714 >> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] conn(0x5640786b7490 0x5640786b8900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:37:36.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:36.003+0000 7f2d6d200640 1 --2- 172.21.15.5:0/1602966714 >> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] conn(0x5640786b7490 0x5640786b8900 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=466b6c695388602e server_cookie=4bfc45ce0ea5ec16 in_seq=0 out_seq=0 2024-09-10T09:37:36.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:36.003+0000 7f2d52000640 10 client.25013 ms_handle_connect on v2:172.21.15.159:6836/620627639 2024-09-10T09:37:36.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:36.004+0000 7f2d52000640 1 -- 172.21.15.5:0/1602966714 <== mds.2 v2:172.21.15.159:6836/620627639 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56407860e620 con 0x5640786b7490 2024-09-10T09:37:36.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:36.004+0000 7f2d52000640 10 client.25013 handle_command_reply: tid=0 2024-09-10T09:37:36.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:36.005+0000 7f2d71e0f740 1 client.25013 shutdown 2024-09-10T09:37:36.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:36.005+0000 7f2d71e0f740 1 -- 172.21.15.5:0/1602966714 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56407869be00 msgr2=0x5640786a3a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:37:36.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:36.005+0000 7f2d71e0f740 1 --2- 172.21.15.5:0/1602966714 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56407869be00 0x5640786a3a40 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f2d6810a820 tx=0x7f2d54012b60 comp rx=0 tx=0).stop 2024-09-10T09:37:36.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:36.005+0000 7f2d71e0f740 1 -- 172.21.15.5:0/1602966714 shutdown_connections 2024-09-10T09:37:36.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:36.005+0000 7f2d71e0f740 1 --2- 172.21.15.5:0/1602966714 >> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] conn(0x5640786b7490 0x5640786b8900 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:36.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:36.005+0000 7f2d71e0f740 1 --2- 172.21.15.5:0/1602966714 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5640786a66c0 0x5640786a3f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:36.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:36.005+0000 7f2d71e0f740 1 --2- 172.21.15.5:0/1602966714 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56407869be00 0x5640786a3a40 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:36.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:36.005+0000 7f2d71e0f740 1 --2- 172.21.15.5:0/1602966714 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56407869b4e0 0x5640786a3500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:36.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:36.005+0000 7f2d52000640 0 client.25013 ms_handle_reset on v2:172.21.15.159:6836/620627639 2024-09-10T09:37:36.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:36.005+0000 7f2d71e0f740 1 -- 172.21.15.5:0/1602966714 >> 172.21.15.5:0/1602966714 conn(0x564078697a70 msgr2=0x564078696bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:37:36.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:36.005+0000 7f2d71e0f740 1 -- 172.21.15.5:0/1602966714 shutdown_connections 2024-09-10T09:37:36.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:36.005+0000 7f2d71e0f740 1 -- 172.21.15.5:0/1602966714 wait complete. 2024-09-10T09:37:36.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:36.005+0000 7f2d71e0f740 20 client.25013 trim_cache size 0 max 16384 2024-09-10T09:37:36.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:36.024+0000 7f2d71e0f740 1 Processor -- start 2024-09-10T09:37:36.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:36.024+0000 7f2d71e0f740 1 -- start start 2024-09-10T09:37:36.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:36.024+0000 7f2d71e0f740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56407869b4e0 0x56407869df40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:37:36.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:36.024+0000 7f2d71e0f740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56407869be00 0x56407869e800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:37:36.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:36.024+0000 7f2d71e0f740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5640786a66c0 0x5640786a0760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:37:36.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:36.024+0000 7f2d71e0f740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5640786b6c40 con 0x56407869b4e0 2024-09-10T09:37:36.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:36.024+0000 7f2d71e0f740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x5640786b7310 con 0x5640786a66c0 2024-09-10T09:37:36.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:36.024+0000 7f2d71e0f740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5640786b79e0 con 0x56407869be00 2024-09-10T09:37:36.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:36.024+0000 7f2d6d200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56407869b4e0 0x56407869df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:37:36.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:36.025+0000 7f2d6d200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56407869b4e0 0x56407869df40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34228/0 (socket says 172.21.15.5:34228) 2024-09-10T09:37:36.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:36.025+0000 7f2d6d200640 1 -- 172.21.15.5:0/237611395 learned_addr learned my addr 172.21.15.5:0/237611395 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:37:36.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:36.025+0000 7f2d67400640 1 --2- 172.21.15.5:0/237611395 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56407869be00 0x56407869e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:37:36.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:36.025+0000 7f2d6d200640 1 --2- 172.21.15.5:0/237611395 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5640786a66c0 0x5640786a0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:37:36.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:36.025+0000 7f2d6d200640 1 -- 172.21.15.5:0/237611395 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56407869be00 msgr2=0x56407869e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:37:36.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:36.025+0000 7f2d6d200640 1 --2- 172.21.15.5:0/237611395 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56407869be00 0x56407869e800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:36.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:36.025+0000 7f2d6d200640 1 -- 172.21.15.5:0/237611395 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5640786a66c0 msgr2=0x5640786a0760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:37:36.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:36.025+0000 7f2d6d200640 1 --2- 172.21.15.5:0/237611395 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5640786a66c0 0x5640786a0760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:36.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:36.025+0000 7f2d6d200640 1 -- 172.21.15.5:0/237611395 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5640786978a0 con 0x56407869b4e0 2024-09-10T09:37:36.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:36.025+0000 7f2d6d200640 1 --2- 172.21.15.5:0/237611395 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56407869b4e0 0x56407869df40 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f2d600317b0 tx=0x7f2d6002a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f5b756f1d89cb9d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:37:36.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:36.026+0000 7f2d5cc00640 1 -- 172.21.15.5:0/237611395 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d60031bb0 con 0x56407869b4e0 2024-09-10T09:37:36.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:36.026+0000 7f2d5cc00640 1 -- 172.21.15.5:0/237611395 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2d60036040 con 0x56407869b4e0 2024-09-10T09:37:36.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:36.026+0000 7f2d5cc00640 1 -- 172.21.15.5:0/237611395 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d60004420 con 0x56407869b4e0 2024-09-10T09:37:36.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:36.026+0000 7f2d71e0f740 1 -- 172.21.15.5:0/237611395 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56407869b4e0 msgr2=0x56407869df40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:37:36.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:36.026+0000 7f2d71e0f740 1 --2- 172.21.15.5:0/237611395 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56407869b4e0 0x56407869df40 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f2d600317b0 tx=0x7f2d6002a8c0 comp rx=0 tx=0).stop 2024-09-10T09:37:36.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:36.026+0000 7f2d71e0f740 1 -- 172.21.15.5:0/237611395 shutdown_connections 2024-09-10T09:37:36.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:36.026+0000 7f2d71e0f740 1 --2- 172.21.15.5:0/237611395 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5640786a66c0 0x5640786a0760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:36.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:36.026+0000 7f2d71e0f740 1 --2- 172.21.15.5:0/237611395 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56407869be00 0x56407869e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:36.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:36.026+0000 7f2d71e0f740 1 --2- 172.21.15.5:0/237611395 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56407869b4e0 0x56407869df40 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:36.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:36.026+0000 7f2d71e0f740 1 -- 172.21.15.5:0/237611395 >> 172.21.15.5:0/237611395 conn(0x564078697a70 msgr2=0x5640786a25f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:37:36.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:36.026+0000 7f2d71e0f740 1 -- 172.21.15.5:0/237611395 shutdown_connections 2024-09-10T09:37:36.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:36.026+0000 7f2d71e0f740 1 -- 172.21.15.5:0/237611395 wait complete. 2024-09-10T09:37:36.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:36.027+0000 7f2d71e0f740 1 -- start start 2024-09-10T09:37:36.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:36.027+0000 7f2d71e0f740 1 Processor -- start 2024-09-10T09:37:36.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:36.027+0000 7f2d71e0f740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56407869b4e0 0x5640786baa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:37:36.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:36.027+0000 7f2d71e0f740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56407869be00 0x5640786baf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:37:36.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:36.028+0000 7f2d71e0f740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5640786a66c0 0x5640786bb4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:37:36.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:36.028+0000 7f2d71e0f740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5640786b8780 con 0x56407869b4e0 2024-09-10T09:37:36.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:36.028+0000 7f2d71e0f740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x5640786b80b0 con 0x56407869be00 2024-09-10T09:37:36.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:36.028+0000 7f2d71e0f740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5640786b8e50 con 0x5640786a66c0 2024-09-10T09:37:36.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:36.028+0000 7f2d71e0f740 10 client.0 _ll_register_callbacks cb 0x7f2d54009de0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:37:36.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:36.028+0000 7f2d67400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56407869be00 0x5640786baf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:37:36.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:36.028+0000 7f2d67400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56407869be00 0x5640786baf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:48640/0 (socket says 172.21.15.5:48640) 2024-09-10T09:37:36.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:36.028+0000 7f2d67400640 1 -- 172.21.15.5:0/3644057676 learned_addr learned my addr 172.21.15.5:0/3644057676 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:37:36.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:36.028+0000 7f2d6d200640 1 --2- 172.21.15.5:0/3644057676 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56407869b4e0 0x5640786baa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:37:36.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:36.029+0000 7f2d67400640 1 -- 172.21.15.5:0/3644057676 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5640786a66c0 msgr2=0x5640786bb4a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T09:37:36.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:36.029+0000 7f2d67400640 1 --2- 172.21.15.5:0/3644057676 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5640786a66c0 0x5640786bb4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:36.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:36.029+0000 7f2d67400640 1 -- 172.21.15.5:0/3644057676 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56407869b4e0 msgr2=0x5640786baa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:37:36.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:36.029+0000 7f2d67400640 1 --2- 172.21.15.5:0/3644057676 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56407869b4e0 0x5640786baa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:36.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:36.029+0000 7f2d67400640 1 -- 172.21.15.5:0/3644057676 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56407869c370 con 0x56407869be00 2024-09-10T09:37:36.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:36.029+0000 7f2d6d200640 1 --2- 172.21.15.5:0/3644057676 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56407869b4e0 0x5640786baa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:37:36.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:36.029+0000 7f2d67400640 1 --2- 172.21.15.5:0/3644057676 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56407869be00 0x5640786baf60 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x5640786a4b40 tx=0x7f2d54010640 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b8a850c101fce564 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:37:36.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:36.029+0000 7f2d52000640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-09-10T09:37:36.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:36.029+0000 7f2d52000640 1 -- 172.21.15.5:0/3644057676 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d5402e060 con 0x56407869be00 2024-09-10T09:37:36.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:36.029+0000 7f2d52000640 1 -- 172.21.15.5:0/3644057676 <== mon.2 v2:172.21.15.102:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f2d54018070 con 0x56407869be00 2024-09-10T09:37:36.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:36.029+0000 7f2d71e0f740 1 -- 172.21.15.5:0/3644057676 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2d60036040 con 0x56407869be00 2024-09-10T09:37:36.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:36.030+0000 7f2d52000640 1 -- 172.21.15.5:0/3644057676 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f2d54035050 con 0x56407869be00 2024-09-10T09:37:36.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:36.030+0000 7f2d71e0f740 10 client.25019 fetch_fsmap learned FSMap version 29 2024-09-10T09:37:36.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:36.030+0000 7f2d71e0f740 1 -- 172.21.15.5:0/3644057676 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56407872a5f0 con 0x56407869be00 2024-09-10T09:37:36.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:36.030+0000 7f2d52000640 1 -- 172.21.15.5:0/3644057676 <== mon.2 v2:172.21.15.102:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2d60036040 con 0x56407869be00 2024-09-10T09:37:36.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:36.030+0000 7f2d52000640 1 -- 172.21.15.5:0/3644057676 <== mon.2 v2:172.21.15.102:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f2d54005330 con 0x56407869be00 2024-09-10T09:37:36.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:36.030+0000 7f2d71e0f740 10 client.25019 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T09:37:36.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:36.030+0000 7f2d71e0f740 10 client.25019 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-10T09:37:36.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:36.030+0000 7f2d71e0f740 20 client.25019 populate_metadata read hostname 'smithi005' 2024-09-10T09:37:36.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:36.030+0000 7f2d71e0f740 1 --2- 172.21.15.5:0/3644057676 >> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] conn(0x56407872f2a0 0x56407872dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:37:36.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:36.030+0000 7f2d71e0f740 4 client.25019 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T09:37:36.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:36.030+0000 7f2d71e0f740 1 -- 172.21.15.5:0/3644057676 --> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] -- command(tid 0: {"prefix": "status"}) -- 0x564078608b30 con 0x56407872f2a0 2024-09-10T09:37:36.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:36.031+0000 7f2d6d200640 1 --2- 172.21.15.5:0/3644057676 >> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] conn(0x56407872f2a0 0x56407872dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:37:36.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:36.031+0000 7f2d6d200640 1 --2- 172.21.15.5:0/3644057676 >> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] conn(0x56407872f2a0 0x56407872dfb0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=14122fe19bc291ec server_cookie=48d92af1afc14fa1 in_seq=0 out_seq=0 2024-09-10T09:37:36.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:36.031+0000 7f2d52000640 10 client.25019 ms_handle_connect on v2:172.21.15.159:6836/620627639 2024-09-10T09:37:36.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:36.031+0000 7f2d52000640 1 -- 172.21.15.5:0/3644057676 <== mds.2 v2:172.21.15.159:6836/620627639 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x564078608b30 con 0x56407872f2a0 2024-09-10T09:37:36.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:36.031+0000 7f2d52000640 10 client.25019 handle_command_reply: tid=0 2024-09-10T09:37:36.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:36.031+0000 7f2d71e0f740 1 client.25019 shutdown 2024-09-10T09:37:36.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:36.032+0000 7f2d71e0f740 1 -- 172.21.15.5:0/3644057676 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56407869be00 msgr2=0x5640786baf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:37:36.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:36.032+0000 7f2d71e0f740 1 --2- 172.21.15.5:0/3644057676 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56407869be00 0x5640786baf60 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x5640786a4b40 tx=0x7f2d54010640 comp rx=0 tx=0).stop 2024-09-10T09:37:36.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:36.032+0000 7f2d71e0f740 1 -- 172.21.15.5:0/3644057676 shutdown_connections 2024-09-10T09:37:36.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:36.032+0000 7f2d71e0f740 1 --2- 172.21.15.5:0/3644057676 >> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] conn(0x56407872f2a0 0x56407872dfb0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:36.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:36.032+0000 7f2d71e0f740 1 --2- 172.21.15.5:0/3644057676 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5640786a66c0 0x5640786bb4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:36.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:36.032+0000 7f2d71e0f740 1 --2- 172.21.15.5:0/3644057676 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56407869be00 0x5640786baf60 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:36.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:36.032+0000 7f2d71e0f740 1 --2- 172.21.15.5:0/3644057676 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56407869b4e0 0x5640786baa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:36.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:36.032+0000 7f2d52000640 0 client.25019 ms_handle_reset on v2:172.21.15.159:6836/620627639 2024-09-10T09:37:36.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:36.032+0000 7f2d71e0f740 1 -- 172.21.15.5:0/3644057676 >> 172.21.15.5:0/3644057676 conn(0x564078697a70 msgr2=0x5640786963b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:37:36.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:36.032+0000 7f2d71e0f740 1 -- 172.21.15.5:0/3644057676 shutdown_connections 2024-09-10T09:37:36.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:36.032+0000 7f2d71e0f740 1 -- 172.21.15.5:0/3644057676 wait complete. 2024-09-10T09:37:36.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:36.032+0000 7f2d71e0f740 20 client.25019 trim_cache size 0 max 16384 2024-09-10T09:37:36.037 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-10T09:37:36.037 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_fsid": "bade4b50-6f56-11ef-bcea-c7b262605968", 2024-09-10T09:37:36.037 INFO:teuthology.orchestra.run.smithi005.stdout: "whoami": 2, 2024-09-10T09:37:36.037 INFO:teuthology.orchestra.run.smithi005.stdout: "id": 24511, 2024-09-10T09:37:36.037 INFO:teuthology.orchestra.run.smithi005.stdout: "want_state": "up:active", 2024-09-10T09:37:36.037 INFO:teuthology.orchestra.run.smithi005.stdout: "state": "up:active", 2024-09-10T09:37:36.038 INFO:teuthology.orchestra.run.smithi005.stdout: "fs_name": "cephfs", 2024-09-10T09:37:36.038 INFO:teuthology.orchestra.run.smithi005.stdout: "rank_uptime": 83.231046887000005, 2024-09-10T09:37:36.038 INFO:teuthology.orchestra.run.smithi005.stdout: "mdsmap_epoch": 29, 2024-09-10T09:37:36.038 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch": 96, 2024-09-10T09:37:36.038 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch_barrier": 94, 2024-09-10T09:37:36.038 INFO:teuthology.orchestra.run.smithi005.stdout: "uptime": 154.67125562800001 2024-09-10T09:37:36.038 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-10T09:37:36.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:36.033+0000 7f2d6dc00640 1 -- 172.21.15.5:0/3615119023 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f2d4c0a2b80 msgr2=0x7f2d4c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:37:36.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:36.033+0000 7f2d6dc00640 1 --2- 172.21.15.5:0/3615119023 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f2d4c0a2b80 0x7f2d4c0a5040 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f2d5800c6a0 tx=0x7f2d580d5530 comp rx=0 tx=0).stop 2024-09-10T09:37:36.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:36.033+0000 7f2d6dc00640 1 -- 172.21.15.5:0/3615119023 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2d68103e30 msgr2=0x7f2d6819a4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:37:36.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:36.033+0000 7f2d6dc00640 1 --2- 172.21.15.5:0/3615119023 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2d68103e30 0x7f2d6819a4a0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f2d5800e9f0 tx=0x7f2d5800eeb0 comp rx=0 tx=0).stop 2024-09-10T09:37:36.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:36.033+0000 7f2d6dc00640 1 -- 172.21.15.5:0/3615119023 shutdown_connections 2024-09-10T09:37:36.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:36.033+0000 7f2d6dc00640 1 --2- 172.21.15.5:0/3615119023 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f2d4c0a2b80 0x7f2d4c0a5040 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:36.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:36.033+0000 7f2d6dc00640 1 --2- 172.21.15.5:0/3615119023 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2d68108f60 0x7f2d6819ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:36.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:36.033+0000 7f2d6dc00640 1 --2- 172.21.15.5:0/3615119023 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2d681047e0 0x7f2d6819a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:36.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:36.033+0000 7f2d6dc00640 1 --2- 172.21.15.5:0/3615119023 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2d68103e30 0x7f2d6819a4a0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:36.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:36.033+0000 7f2d6dc00640 1 -- 172.21.15.5:0/3615119023 >> 172.21.15.5:0/3615119023 conn(0x7f2d680ff910 msgr2=0x7f2d68100be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:37:36.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:36.033+0000 7f2d6dc00640 1 -- 172.21.15.5:0/3615119023 shutdown_connections 2024-09-10T09:37:36.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:36.033+0000 7f2d6dc00640 1 -- 172.21.15.5:0/3615119023 wait complete. 2024-09-10T09:37:36.857 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph tell mds.i status 2024-09-10T09:37:37.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:37:36 smithi005 ceph-mon[30350]: pgmap v371: 129 pgs: 129 active+clean; 645 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-09-10T09:37:37.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:37:36 smithi005 ceph-mon[30350]: mgrmap e18: x(active, since 9m), standbys: z, y 2024-09-10T09:37:37.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:37:36 smithi159 ceph-mon[35513]: pgmap v371: 129 pgs: 129 active+clean; 645 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-09-10T09:37:37.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:37:36 smithi159 ceph-mon[35513]: mgrmap e18: x(active, since 9m), standbys: z, y 2024-09-10T09:37:37.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:37:36 smithi102 ceph-mon[35442]: pgmap v371: 129 pgs: 129 active+clean; 645 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-09-10T09:37:37.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:37:36 smithi102 ceph-mon[35442]: mgrmap e18: x(active, since 9m), standbys: z, y 2024-09-10T09:37:37.246 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:37:37.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:37.915+0000 7fa5d4e00640 1 -- 172.21.15.5:0/3915806530 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa5d01049a0 msgr2=0x7fa5d01089f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:37:37.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:37.915+0000 7fa5d4e00640 1 --2- 172.21.15.5:0/3915806530 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa5d01049a0 0x7fa5d01089f0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fa5b8009860 tx=0x7fa5b802a340 comp rx=0 tx=0).stop 2024-09-10T09:37:37.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:37.915+0000 7fa5d4e00640 1 -- 172.21.15.5:0/3915806530 shutdown_connections 2024-09-10T09:37:37.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:37.915+0000 7fa5d4e00640 1 --2- 172.21.15.5:0/3915806530 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa5d0109120 0x7fa5d0110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:37.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:37.915+0000 7fa5d4e00640 1 --2- 172.21.15.5:0/3915806530 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa5d01049a0 0x7fa5d01089f0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:37.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:37.915+0000 7fa5d4e00640 1 --2- 172.21.15.5:0/3915806530 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa5d0103ff0 0x7fa5d01043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:37.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:37.915+0000 7fa5d4e00640 1 -- 172.21.15.5:0/3915806530 >> 172.21.15.5:0/3915806530 conn(0x7fa5d00ffa90 msgr2=0x7fa5d0101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:37:37.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:37.915+0000 7fa5d4e00640 1 -- 172.21.15.5:0/3915806530 shutdown_connections 2024-09-10T09:37:37.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:37.915+0000 7fa5d4e00640 1 -- 172.21.15.5:0/3915806530 wait complete. 2024-09-10T09:37:37.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:37.916+0000 7fa5d4e00640 1 Processor -- start 2024-09-10T09:37:37.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:37.916+0000 7fa5d4e00640 1 -- start start 2024-09-10T09:37:37.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:37.917+0000 7fa5d4e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa5d0103ff0 0x7fa5d019a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:37:37.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:37.917+0000 7fa5d4e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa5d01049a0 0x7fa5d019abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:37:37.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:37.917+0000 7fa5d4e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa5d0109120 0x7fa5d019ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:37:37.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:37.917+0000 7fa5d4e00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa5d0114140 con 0x7fa5d0103ff0 2024-09-10T09:37:37.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:37.917+0000 7fa5d4e00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fa5d0113fc0 con 0x7fa5d01049a0 2024-09-10T09:37:37.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:37.917+0000 7fa5d4e00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fa5d01142c0 con 0x7fa5d0109120 2024-09-10T09:37:37.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:37.917+0000 7fa5cde00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa5d0103ff0 0x7fa5d019a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:37:37.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:37.917+0000 7fa5ce800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa5d0109120 0x7fa5d019ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:37:37.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:37.917+0000 7fa5cde00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa5d0103ff0 0x7fa5d019a680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34256/0 (socket says 172.21.15.5:34256) 2024-09-10T09:37:37.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:37.917+0000 7fa5ce800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa5d0109120 0x7fa5d019ef50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:45026/0 (socket says 172.21.15.5:45026) 2024-09-10T09:37:37.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:37.917+0000 7fa5cde00640 1 -- 172.21.15.5:0/3941546655 learned_addr learned my addr 172.21.15.5:0/3941546655 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:37:37.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:37.917+0000 7fa5c7e00640 1 --2- 172.21.15.5:0/3941546655 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa5d01049a0 0x7fa5d019abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:37:37.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:37.917+0000 7fa5cde00640 1 -- 172.21.15.5:0/3941546655 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa5d0109120 msgr2=0x7fa5d019ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:37:37.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:37.917+0000 7fa5cde00640 1 --2- 172.21.15.5:0/3941546655 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa5d0109120 0x7fa5d019ef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:37.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:37.918+0000 7fa5cde00640 1 -- 172.21.15.5:0/3941546655 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa5d01049a0 msgr2=0x7fa5d019abc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:37:37.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:37.918+0000 7fa5cde00640 1 --2- 172.21.15.5:0/3941546655 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa5d01049a0 0x7fa5d019abc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:37.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:37.918+0000 7fa5cde00640 1 -- 172.21.15.5:0/3941546655 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5d019f630 con 0x7fa5d0103ff0 2024-09-10T09:37:37.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:37.918+0000 7fa5c7e00640 1 --2- 172.21.15.5:0/3941546655 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa5d01049a0 0x7fa5d019abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:37:37.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:37.918+0000 7fa5cde00640 1 --2- 172.21.15.5:0/3941546655 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa5d0103ff0 0x7fa5d019a680 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fa5c000e9f0 tx=0x7fa5c000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:37:37.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:37.918+0000 7fa5c6a00640 1 -- 172.21.15.5:0/3941546655 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5c000cd80 con 0x7fa5d0103ff0 2024-09-10T09:37:37.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:37.918+0000 7fa5c6a00640 1 -- 172.21.15.5:0/3941546655 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa5c0005500 con 0x7fa5d0103ff0 2024-09-10T09:37:37.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:37.918+0000 7fa5c6a00640 1 -- 172.21.15.5:0/3941546655 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5c0010690 con 0x7fa5d0103ff0 2024-09-10T09:37:37.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:37.918+0000 7fa5d4e00640 1 -- 172.21.15.5:0/3941546655 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5d019f920 con 0x7fa5d0103ff0 2024-09-10T09:37:37.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:37.918+0000 7fa5d4e00640 1 -- 172.21.15.5:0/3941546655 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5d01a7200 con 0x7fa5d0103ff0 2024-09-10T09:37:37.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:37.922+0000 7fa5d79d8740 1 Processor -- start 2024-09-10T09:37:37.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:37.922+0000 7fa5c6a00640 1 -- 172.21.15.5:0/3941546655 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7fa5c00046c0 con 0x7fa5d0103ff0 2024-09-10T09:37:37.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:37.922+0000 7fa5d79d8740 1 -- start start 2024-09-10T09:37:37.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:37.922+0000 7fa5d79d8740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561629af7330 0x561629af7710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:37:37.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:37.922+0000 7fa5d79d8740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561629af7c50 0x561629b01fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:37:37.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:37.922+0000 7fa5d79d8740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561629b02510 0x561629b04900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:37:37.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:37.922+0000 7fa5d79d8740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x561629a5d0b0 con 0x561629af7330 2024-09-10T09:37:37.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:37.922+0000 7fa5d79d8740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x561629a04ab0 con 0x561629b02510 2024-09-10T09:37:37.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:37.922+0000 7fa5d79d8740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x561629a59070 con 0x561629af7c50 2024-09-10T09:37:37.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:37.922+0000 7fa5cde00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561629af7c50 0x561629b01fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:37:37.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:37.922+0000 7fa5ce800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561629af7330 0x561629af7710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:37:37.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:37.922+0000 7fa5c7e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561629b02510 0x561629b04900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:37:37.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:37.922+0000 7fa5ce800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561629af7330 0x561629af7710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34272/0 (socket says 172.21.15.5:34272) 2024-09-10T09:37:37.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:37.922+0000 7fa5c7e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561629b02510 0x561629b04900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:48666/0 (socket says 172.21.15.5:48666) 2024-09-10T09:37:37.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:37.922+0000 7fa5cde00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561629af7c50 0x561629b01fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:45030/0 (socket says 172.21.15.5:45030) 2024-09-10T09:37:37.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:37.922+0000 7fa5ce800640 1 -- 172.21.15.5:0/217649115 learned_addr learned my addr 172.21.15.5:0/217649115 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:37:37.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:37.923+0000 7fa5ce800640 1 -- 172.21.15.5:0/217649115 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561629af7c50 msgr2=0x561629b01fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:37:37.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:37.923+0000 7fa5ce800640 1 --2- 172.21.15.5:0/217649115 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561629af7c50 0x561629b01fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:37.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:37.923+0000 7fa5ce800640 1 -- 172.21.15.5:0/217649115 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561629b02510 msgr2=0x561629b04900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:37:37.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:37.923+0000 7fa5ce800640 1 --2- 172.21.15.5:0/217649115 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561629b02510 0x561629b04900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:37.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:37.923+0000 7fa5ce800640 1 -- 172.21.15.5:0/217649115 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561629a114a0 con 0x561629af7330 2024-09-10T09:37:37.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:37.923+0000 7fa5c7e00640 1 --2- 172.21.15.5:0/217649115 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561629b02510 0x561629b04900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:37:37.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:37.923+0000 7fa5ce800640 1 --2- 172.21.15.5:0/217649115 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561629af7330 0x561629af7710 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fa5c80099f0 tx=0x7fa5c80095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7a3c546034d3581 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:37:37.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:37.924+0000 7fa5c6a00640 1 --2- 172.21.15.5:0/3941546655 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fa5b40a2c00 0x7fa5b40a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:37:37.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:37.924+0000 7fa5bf400640 1 -- 172.21.15.5:0/217649115 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5c801d070 con 0x561629af7330 2024-09-10T09:37:37.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:37.924+0000 7fa5bf400640 1 -- 172.21.15.5:0/217649115 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa5c8003e40 con 0x561629af7330 2024-09-10T09:37:37.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:37.924+0000 7fa5bf400640 1 -- 172.21.15.5:0/217649115 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5c800dba0 con 0x561629af7330 2024-09-10T09:37:37.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:37.924+0000 7fa5c6a00640 1 -- 172.21.15.5:0/3941546655 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fa5c00cf810 con 0x7fa5d0103ff0 2024-09-10T09:37:37.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:37.924+0000 7fa5cde00640 1 --2- 172.21.15.5:0/3941546655 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fa5b40a2c00 0x7fa5b40a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:37:37.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:37.925+0000 7fa5d79d8740 1 -- 172.21.15.5:0/217649115 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561629af7330 msgr2=0x561629af7710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:37:37.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:37.925+0000 7fa5d79d8740 1 --2- 172.21.15.5:0/217649115 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561629af7330 0x561629af7710 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fa5c80099f0 tx=0x7fa5c80095c0 comp rx=0 tx=0).stop 2024-09-10T09:37:37.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:37.925+0000 7fa5cde00640 1 --2- 172.21.15.5:0/3941546655 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fa5b40a2c00 0x7fa5b40a50c0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fa5c0098080 tx=0x7fa5c00d5530 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:37:37.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:37.925+0000 7fa5d79d8740 1 -- 172.21.15.5:0/217649115 shutdown_connections 2024-09-10T09:37:37.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:37.925+0000 7fa5d79d8740 1 --2- 172.21.15.5:0/217649115 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561629b02510 0x561629b04900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:37.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:37.925+0000 7fa5d79d8740 1 --2- 172.21.15.5:0/217649115 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561629af7c50 0x561629b01fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:37.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:37.925+0000 7fa5d79d8740 1 --2- 172.21.15.5:0/217649115 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561629af7330 0x561629af7710 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:37.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:37.925+0000 7fa5d79d8740 1 -- 172.21.15.5:0/217649115 >> 172.21.15.5:0/217649115 conn(0x561629af38c0 msgr2=0x561629af3ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:37:37.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:37.925+0000 7fa5d79d8740 1 -- 172.21.15.5:0/217649115 shutdown_connections 2024-09-10T09:37:37.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:37.925+0000 7fa5d79d8740 1 -- 172.21.15.5:0/217649115 wait complete. 2024-09-10T09:37:37.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:37.925+0000 7fa5d79d8740 1 -- start start 2024-09-10T09:37:37.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:37.926+0000 7fa5d79d8740 1 Processor -- start 2024-09-10T09:37:37.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:37.926+0000 7fa5d79d8740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561629af7330 0x561629aff5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:37:37.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:37.926+0000 7fa5d79d8740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561629af7c50 0x561629affaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:37:37.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:37.926+0000 7fa5d79d8740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561629b02510 0x561629b00030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:37:37.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:37.926+0000 7fa5d79d8740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x561629b05060 con 0x561629af7330 2024-09-10T09:37:37.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:37.926+0000 7fa5d79d8740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x5616299e5920 con 0x561629b02510 2024-09-10T09:37:37.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:37.926+0000 7fa5d79d8740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x561629b051e0 con 0x561629af7c50 2024-09-10T09:37:37.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:37.926+0000 7fa5d79d8740 10 client.0 _ll_register_callbacks cb 0x561629a098a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:37:37.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:37.926+0000 7fa5ce800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561629af7330 0x561629aff5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:37:37.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:37.926+0000 7fa5ce800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561629af7330 0x561629aff5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34282/0 (socket says 172.21.15.5:34282) 2024-09-10T09:37:37.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:37.926+0000 7fa5c7e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561629af7c50 0x561629affaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:37:37.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:37.926+0000 7fa5ce800640 1 -- 172.21.15.5:0/330343546 learned_addr learned my addr 172.21.15.5:0/330343546 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:37:37.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:37.927+0000 7fa5ce800640 1 --2- 172.21.15.5:0/330343546 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561629b02510 0x561629b00030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:37:37.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:37.927+0000 7fa5c7e00640 1 -- 172.21.15.5:0/330343546 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561629b02510 msgr2=0x561629b00030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:37:37.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:37.927+0000 7fa5c7e00640 1 --2- 172.21.15.5:0/330343546 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561629b02510 0x561629b00030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:37.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:37.927+0000 7fa5c7e00640 1 -- 172.21.15.5:0/330343546 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561629af7330 msgr2=0x561629aff5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:37:37.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:37.927+0000 7fa5c7e00640 1 --2- 172.21.15.5:0/330343546 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561629af7330 0x561629aff5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:37.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:37.927+0000 7fa5c7e00640 1 -- 172.21.15.5:0/330343546 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5616297ce2b0 con 0x561629af7c50 2024-09-10T09:37:37.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:37.927+0000 7fa5ce800640 1 --2- 172.21.15.5:0/330343546 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561629b02510 0x561629b00030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T09:37:37.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:37.927+0000 7fa5c7e00640 1 --2- 172.21.15.5:0/330343546 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561629af7c50 0x561629affaf0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fa5b8035040 tx=0x7fa5b80097b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3c35900dd0338f1b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:37:37.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:37.927+0000 7fa5b3e00640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-09-10T09:37:37.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:37.927+0000 7fa5b3e00640 1 -- 172.21.15.5:0/330343546 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5b8037040 con 0x561629af7c50 2024-09-10T09:37:37.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:37.928+0000 7fa5d79d8740 1 -- 172.21.15.5:0/330343546 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x561629a6a620 con 0x561629af7c50 2024-09-10T09:37:37.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:37.928+0000 7fa5b3e00640 1 -- 172.21.15.5:0/330343546 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fa5b8030960 con 0x561629af7c50 2024-09-10T09:37:37.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:37.928+0000 7fa5b3e00640 1 -- 172.21.15.5:0/330343546 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fa5b804b050 con 0x561629af7c50 2024-09-10T09:37:37.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:37.928+0000 7fa5d79d8740 10 client.25045 fetch_fsmap learned FSMap version 29 2024-09-10T09:37:37.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:37.928+0000 7fa5d79d8740 1 -- 172.21.15.5:0/330343546 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x561629b10e90 con 0x561629af7c50 2024-09-10T09:37:37.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:37.928+0000 7fa5b3e00640 1 -- 172.21.15.5:0/330343546 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x561629a6a620 con 0x561629af7c50 2024-09-10T09:37:37.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:37.929+0000 7fa5b3e00640 1 -- 172.21.15.5:0/330343546 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7fa5b803d720 con 0x561629af7c50 2024-09-10T09:37:37.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:37.929+0000 7fa5d79d8740 10 client.25045 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T09:37:37.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:37.929+0000 7fa5d79d8740 10 client.25045 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-10T09:37:37.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:37.929+0000 7fa5d79d8740 20 client.25045 populate_metadata read hostname 'smithi005' 2024-09-10T09:37:37.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:37.929+0000 7fa5d79d8740 1 --2- 172.21.15.5:0/330343546 >> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] conn(0x561629b13430 0x561629b148a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:37:37.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:37.929+0000 7fa5d79d8740 4 client.25045 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:37:37.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:37.929+0000 7fa5d79d8740 1 -- 172.21.15.5:0/330343546 --> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561629a64b30 con 0x561629b13430 2024-09-10T09:37:37.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:37.929+0000 7fa5ce800640 1 --2- 172.21.15.5:0/330343546 >> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] conn(0x561629b13430 0x561629b148a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:37:37.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:37.930+0000 7fa5ce800640 1 --2- 172.21.15.5:0/330343546 >> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] conn(0x561629b13430 0x561629b148a0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6d6c84a5a940c0d3 server_cookie=b16075ba2222f8e6 in_seq=0 out_seq=0 2024-09-10T09:37:37.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:37.930+0000 7fa5b3e00640 10 client.25045 ms_handle_connect on v2:172.21.15.159:6838/2073739599 2024-09-10T09:37:37.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:37.931+0000 7fa5b3e00640 1 -- 172.21.15.5:0/330343546 <== mds.1 v2:172.21.15.159:6838/2073739599 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561629a64b30 con 0x561629b13430 2024-09-10T09:37:37.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:37.931+0000 7fa5b3e00640 10 client.25045 handle_command_reply: tid=0 2024-09-10T09:37:37.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:37.931+0000 7fa5d79d8740 1 client.25045 shutdown 2024-09-10T09:37:37.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:37.931+0000 7fa5d79d8740 1 -- 172.21.15.5:0/330343546 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561629af7c50 msgr2=0x561629affaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:37:37.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:37.931+0000 7fa5d79d8740 1 --2- 172.21.15.5:0/330343546 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561629af7c50 0x561629affaf0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fa5b8035040 tx=0x7fa5b80097b0 comp rx=0 tx=0).stop 2024-09-10T09:37:37.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:37.931+0000 7fa5d79d8740 1 -- 172.21.15.5:0/330343546 shutdown_connections 2024-09-10T09:37:37.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:37.931+0000 7fa5d79d8740 1 --2- 172.21.15.5:0/330343546 >> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] conn(0x561629b13430 0x561629b148a0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:37.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:37.931+0000 7fa5d79d8740 1 --2- 172.21.15.5:0/330343546 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561629b02510 0x561629b00030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:37.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:37.931+0000 7fa5d79d8740 1 --2- 172.21.15.5:0/330343546 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561629af7c50 0x561629affaf0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:37.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:37.931+0000 7fa5d79d8740 1 --2- 172.21.15.5:0/330343546 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561629af7330 0x561629aff5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:37.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:37.931+0000 7fa5d79d8740 1 -- 172.21.15.5:0/330343546 >> 172.21.15.5:0/330343546 conn(0x561629af38c0 msgr2=0x561629af95a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:37:37.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:37.932+0000 7fa5b3e00640 0 client.25045 ms_handle_reset on v2:172.21.15.159:6838/2073739599 2024-09-10T09:37:37.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:37.932+0000 7fa5d79d8740 1 -- 172.21.15.5:0/330343546 shutdown_connections 2024-09-10T09:37:37.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:37.932+0000 7fa5d79d8740 1 -- 172.21.15.5:0/330343546 wait complete. 2024-09-10T09:37:37.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:37.932+0000 7fa5d79d8740 20 client.25045 trim_cache size 0 max 16384 2024-09-10T09:37:37.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:37.951+0000 7fa5d79d8740 1 Processor -- start 2024-09-10T09:37:37.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:37.951+0000 7fa5d79d8740 1 -- start start 2024-09-10T09:37:37.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:37.951+0000 7fa5d79d8740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561629af7330 0x561629b12d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:37:37.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:37.951+0000 7fa5d79d8740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561629af7c50 0x561629afa930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:37:37.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:37.951+0000 7fa5d79d8740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561629b02510 0x561629affe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:37:37.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:37.951+0000 7fa5d79d8740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x561629b12be0 con 0x561629af7330 2024-09-10T09:37:37.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:37.951+0000 7fa5d79d8740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x561629b132b0 con 0x561629af7c50 2024-09-10T09:37:37.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:37.951+0000 7fa5d79d8740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x561629b13980 con 0x561629b02510 2024-09-10T09:37:37.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:37.951+0000 7fa5ce800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561629af7330 0x561629b12d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:37:37.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:37.951+0000 7fa5ce800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561629af7330 0x561629b12d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34296/0 (socket says 172.21.15.5:34296) 2024-09-10T09:37:37.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:37.951+0000 7fa5ce800640 1 -- 172.21.15.5:0/3623901413 learned_addr learned my addr 172.21.15.5:0/3623901413 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:37:37.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:37.952+0000 7fa5c7e00640 1 --2- 172.21.15.5:0/3623901413 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561629af7c50 0x561629afa930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:37:37.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:37.952+0000 7fa5ce800640 1 --2- 172.21.15.5:0/3623901413 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561629b02510 0x561629affe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:37:37.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:37.952+0000 7fa5ce800640 1 -- 172.21.15.5:0/3623901413 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561629b02510 msgr2=0x561629affe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:37:37.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:37.952+0000 7fa5ce800640 1 --2- 172.21.15.5:0/3623901413 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561629b02510 0x561629affe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:37.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:37.952+0000 7fa5ce800640 1 -- 172.21.15.5:0/3623901413 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561629af7c50 msgr2=0x561629afa930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:37:37.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:37.952+0000 7fa5ce800640 1 --2- 172.21.15.5:0/3623901413 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561629af7c50 0x561629afa930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:37.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:37.952+0000 7fa5ce800640 1 -- 172.21.15.5:0/3623901413 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561629af7a80 con 0x561629af7330 2024-09-10T09:37:37.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:37.952+0000 7fa5ce800640 1 --2- 172.21.15.5:0/3623901413 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561629af7330 0x561629b12d60 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fa5c8005fd0 tx=0x7fa5c800d780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9a44c5d3c94b724 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:37:37.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:37.952+0000 7fa5bea00640 1 -- 172.21.15.5:0/3623901413 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5c801d040 con 0x561629af7330 2024-09-10T09:37:37.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:37.952+0000 7fa5bea00640 1 -- 172.21.15.5:0/3623901413 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa5c801e040 con 0x561629af7330 2024-09-10T09:37:37.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:37.952+0000 7fa5bea00640 1 -- 172.21.15.5:0/3623901413 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5c8017640 con 0x561629af7330 2024-09-10T09:37:37.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:37.953+0000 7fa5d79d8740 1 -- 172.21.15.5:0/3623901413 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561629af7330 msgr2=0x561629b12d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:37:37.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:37.953+0000 7fa5d79d8740 1 --2- 172.21.15.5:0/3623901413 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561629af7330 0x561629b12d60 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fa5c8005fd0 tx=0x7fa5c800d780 comp rx=0 tx=0).stop 2024-09-10T09:37:37.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:37.953+0000 7fa5d79d8740 1 -- 172.21.15.5:0/3623901413 shutdown_connections 2024-09-10T09:37:37.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:37.953+0000 7fa5d79d8740 1 --2- 172.21.15.5:0/3623901413 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561629b02510 0x561629affe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:37.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:37.953+0000 7fa5d79d8740 1 --2- 172.21.15.5:0/3623901413 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561629af7c50 0x561629afa930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:37.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:37.953+0000 7fa5d79d8740 1 --2- 172.21.15.5:0/3623901413 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561629af7330 0x561629b12d60 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:37.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:37.953+0000 7fa5d79d8740 1 -- 172.21.15.5:0/3623901413 >> 172.21.15.5:0/3623901413 conn(0x561629af38c0 msgr2=0x561629afe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:37:37.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:37.953+0000 7fa5d79d8740 1 -- 172.21.15.5:0/3623901413 shutdown_connections 2024-09-10T09:37:37.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:37.953+0000 7fa5d79d8740 1 -- 172.21.15.5:0/3623901413 wait complete. 2024-09-10T09:37:37.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:37.954+0000 7fa5d79d8740 1 -- start start 2024-09-10T09:37:37.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:37.954+0000 7fa5d79d8740 1 Processor -- start 2024-09-10T09:37:37.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:37.954+0000 7fa5d79d8740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561629af7330 0x561629b1d490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:37:37.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:37.954+0000 7fa5d79d8740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561629af7c50 0x561629afc4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:37:37.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:37.954+0000 7fa5d79d8740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561629b02510 0x561629afcc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:37:37.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:37.954+0000 7fa5d79d8740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x561629b14720 con 0x561629af7330 2024-09-10T09:37:37.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:37.954+0000 7fa5d79d8740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x561629b14050 con 0x561629af7c50 2024-09-10T09:37:37.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:37.954+0000 7fa5d79d8740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x561629b14df0 con 0x561629b02510 2024-09-10T09:37:37.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:37.954+0000 7fa5d79d8740 10 client.0 _ll_register_callbacks cb 0x7fa5b8004490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:37:37.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:37.955+0000 7fa5c7e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561629af7c50 0x561629afc4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:37:37.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:37.955+0000 7fa5ce800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561629af7330 0x561629b1d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:37:37.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:37.955+0000 7fa5c7e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561629af7c50 0x561629afc4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:48692/0 (socket says 172.21.15.5:48692) 2024-09-10T09:37:37.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:37.955+0000 7fa5c7e00640 1 -- 172.21.15.5:0/417528093 learned_addr learned my addr 172.21.15.5:0/417528093 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:37:37.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:37.955+0000 7fa5ce800640 1 --2- 172.21.15.5:0/417528093 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561629b02510 0x561629afcc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:37:37.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:37.955+0000 7fa5c7e00640 1 -- 172.21.15.5:0/417528093 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561629b02510 msgr2=0x561629afcc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:37:37.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:37.955+0000 7fa5c7e00640 1 --2- 172.21.15.5:0/417528093 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561629b02510 0x561629afcc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:37.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:37.955+0000 7fa5c7e00640 1 -- 172.21.15.5:0/417528093 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561629af7330 msgr2=0x561629b1d490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:37:37.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:37.955+0000 7fa5c7e00640 1 --2- 172.21.15.5:0/417528093 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561629af7330 0x561629b1d490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:37.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:37.955+0000 7fa5c7e00640 1 -- 172.21.15.5:0/417528093 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561629af36f0 con 0x561629af7c50 2024-09-10T09:37:37.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:37.955+0000 7fa5ce800640 1 --2- 172.21.15.5:0/417528093 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561629af7330 0x561629b1d490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:37:37.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:37.956+0000 7fa5c7e00640 1 --2- 172.21.15.5:0/417528093 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561629af7c50 0x561629afc4e0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fa5b803dd00 tx=0x7fa5b8030990 comp rx=0 tx=0).ready entity=mon.2 client_cookie=86e415eeb529b859 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:37:37.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:37.956+0000 7fa5b3e00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-09-10T09:37:37.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:37.956+0000 7fa5b3e00640 1 -- 172.21.15.5:0/417528093 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5b8037040 con 0x561629af7c50 2024-09-10T09:37:37.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:37.956+0000 7fa5d79d8740 1 -- 172.21.15.5:0/417528093 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa5c801e040 con 0x561629af7c50 2024-09-10T09:37:37.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:37.956+0000 7fa5b3e00640 1 -- 172.21.15.5:0/417528093 <== mon.2 v2:172.21.15.102:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fa5b803d860 con 0x561629af7c50 2024-09-10T09:37:37.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:37.956+0000 7fa5b3e00640 1 -- 172.21.15.5:0/417528093 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fa5b805f050 con 0x561629af7c50 2024-09-10T09:37:37.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:37.956+0000 7fa5d79d8740 10 client.25034 fetch_fsmap learned FSMap version 29 2024-09-10T09:37:37.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:37.956+0000 7fa5d79d8740 1 -- 172.21.15.5:0/417528093 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x561629b7db40 con 0x561629af7c50 2024-09-10T09:37:37.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:37.957+0000 7fa5b3e00640 1 -- 172.21.15.5:0/417528093 <== mon.2 v2:172.21.15.102:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa5c801e040 con 0x561629af7c50 2024-09-10T09:37:37.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:37.957+0000 7fa5b3e00640 1 -- 172.21.15.5:0/417528093 <== mon.2 v2:172.21.15.102:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7fa5b8046530 con 0x561629af7c50 2024-09-10T09:37:37.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:37.957+0000 7fa5d79d8740 10 client.25034 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T09:37:37.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:37.957+0000 7fa5d79d8740 10 client.25034 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-10T09:37:37.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:37.957+0000 7fa5d79d8740 20 client.25034 populate_metadata read hostname 'smithi005' 2024-09-10T09:37:37.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:37.957+0000 7fa5d79d8740 1 --2- 172.21.15.5:0/417528093 >> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] conn(0x561629b82720 0x561629b81430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:37:37.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:37.957+0000 7fa5d79d8740 4 client.25034 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T09:37:37.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:37.957+0000 7fa5d79d8740 1 -- 172.21.15.5:0/417528093 --> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] -- command(tid 0: {"prefix": "status"}) -- 0x5616299e0f80 con 0x561629b82720 2024-09-10T09:37:37.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:37.957+0000 7fa5ce800640 1 --2- 172.21.15.5:0/417528093 >> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] conn(0x561629b82720 0x561629b81430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:37:37.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:37.958+0000 7fa5ce800640 1 --2- 172.21.15.5:0/417528093 >> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] conn(0x561629b82720 0x561629b81430 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8e5594de6fed415c server_cookie=f3c0f584336d91f8 in_seq=0 out_seq=0 2024-09-10T09:37:37.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:37.958+0000 7fa5b3e00640 10 client.25034 ms_handle_connect on v2:172.21.15.159:6838/2073739599 2024-09-10T09:37:37.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:37.958+0000 7fa5b3e00640 1 -- 172.21.15.5:0/417528093 <== mds.1 v2:172.21.15.159:6838/2073739599 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x5616299e0f80 con 0x561629b82720 2024-09-10T09:37:37.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:37.958+0000 7fa5b3e00640 10 client.25034 handle_command_reply: tid=0 2024-09-10T09:37:37.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:37.958+0000 7fa5d79d8740 1 client.25034 shutdown 2024-09-10T09:37:37.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:37.958+0000 7fa5d79d8740 1 -- 172.21.15.5:0/417528093 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561629af7c50 msgr2=0x561629afc4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:37:37.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:37.958+0000 7fa5d79d8740 1 --2- 172.21.15.5:0/417528093 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561629af7c50 0x561629afc4e0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fa5b803dd00 tx=0x7fa5b8030990 comp rx=0 tx=0).stop 2024-09-10T09:37:37.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:37.958+0000 7fa5d79d8740 1 -- 172.21.15.5:0/417528093 shutdown_connections 2024-09-10T09:37:37.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:37.958+0000 7fa5d79d8740 1 --2- 172.21.15.5:0/417528093 >> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] conn(0x561629b82720 0x561629b81430 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:37.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:37.958+0000 7fa5d79d8740 1 --2- 172.21.15.5:0/417528093 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561629b02510 0x561629afcc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:37.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:37.958+0000 7fa5d79d8740 1 --2- 172.21.15.5:0/417528093 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561629af7c50 0x561629afc4e0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:37.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:37.958+0000 7fa5d79d8740 1 --2- 172.21.15.5:0/417528093 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561629af7330 0x561629b1d490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:37.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:37.958+0000 7fa5b3e00640 0 client.25034 ms_handle_reset on v2:172.21.15.159:6838/2073739599 2024-09-10T09:37:37.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:37.958+0000 7fa5d79d8740 1 -- 172.21.15.5:0/417528093 >> 172.21.15.5:0/417528093 conn(0x561629af38c0 msgr2=0x561629af2200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:37:37.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:37.959+0000 7fa5d79d8740 1 -- 172.21.15.5:0/417528093 shutdown_connections 2024-09-10T09:37:37.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:37.959+0000 7fa5d79d8740 1 -- 172.21.15.5:0/417528093 wait complete. 2024-09-10T09:37:37.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:37.959+0000 7fa5d79d8740 20 client.25034 trim_cache size 0 max 16384 2024-09-10T09:37:37.964 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-10T09:37:37.964 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_fsid": "bade4b50-6f56-11ef-bcea-c7b262605968", 2024-09-10T09:37:37.964 INFO:teuthology.orchestra.run.smithi005.stdout: "whoami": 1, 2024-09-10T09:37:37.965 INFO:teuthology.orchestra.run.smithi005.stdout: "id": 24535, 2024-09-10T09:37:37.965 INFO:teuthology.orchestra.run.smithi005.stdout: "want_state": "up:active", 2024-09-10T09:37:37.965 INFO:teuthology.orchestra.run.smithi005.stdout: "state": "up:active", 2024-09-10T09:37:37.965 INFO:teuthology.orchestra.run.smithi005.stdout: "fs_name": "cephfs", 2024-09-10T09:37:37.965 INFO:teuthology.orchestra.run.smithi005.stdout: "rank_uptime": 86.163691223000001, 2024-09-10T09:37:37.965 INFO:teuthology.orchestra.run.smithi005.stdout: "mdsmap_epoch": 29, 2024-09-10T09:37:37.965 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch": 96, 2024-09-10T09:37:37.965 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch_barrier": 94, 2024-09-10T09:37:37.965 INFO:teuthology.orchestra.run.smithi005.stdout: "uptime": 151.10754817500001 2024-09-10T09:37:37.965 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-10T09:37:37.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:37.959+0000 7fa5d4e00640 1 -- 172.21.15.5:0/3941546655 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fa5b40a2c00 msgr2=0x7fa5b40a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:37:37.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:37.959+0000 7fa5d4e00640 1 --2- 172.21.15.5:0/3941546655 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fa5b40a2c00 0x7fa5b40a50c0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fa5c0098080 tx=0x7fa5c00d5530 comp rx=0 tx=0).stop 2024-09-10T09:37:37.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:37.959+0000 7fa5d4e00640 1 -- 172.21.15.5:0/3941546655 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa5d0103ff0 msgr2=0x7fa5d019a680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:37:37.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:37.959+0000 7fa5d4e00640 1 --2- 172.21.15.5:0/3941546655 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa5d0103ff0 0x7fa5d019a680 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fa5c000e9f0 tx=0x7fa5c000edf0 comp rx=0 tx=0).stop 2024-09-10T09:37:37.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:37.960+0000 7fa5d4e00640 1 -- 172.21.15.5:0/3941546655 shutdown_connections 2024-09-10T09:37:37.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:37.960+0000 7fa5d4e00640 1 --2- 172.21.15.5:0/3941546655 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fa5b40a2c00 0x7fa5b40a50c0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:37.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:37.960+0000 7fa5d4e00640 1 --2- 172.21.15.5:0/3941546655 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa5d0109120 0x7fa5d019ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:37.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:37.960+0000 7fa5d4e00640 1 --2- 172.21.15.5:0/3941546655 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa5d01049a0 0x7fa5d019abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:37.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:37.960+0000 7fa5d4e00640 1 --2- 172.21.15.5:0/3941546655 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa5d0103ff0 0x7fa5d019a680 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:37.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:37.960+0000 7fa5d4e00640 1 -- 172.21.15.5:0/3941546655 >> 172.21.15.5:0/3941546655 conn(0x7fa5d00ffa90 msgr2=0x7fa5d0100d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:37:37.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:37.960+0000 7fa5d4e00640 1 -- 172.21.15.5:0/3941546655 shutdown_connections 2024-09-10T09:37:37.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:37.960+0000 7fa5d4e00640 1 -- 172.21.15.5:0/3941546655 wait complete. 2024-09-10T09:37:38.780 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph tell mds.b status 2024-09-10T09:37:39.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:37:38 smithi005 ceph-mon[30350]: pgmap v372: 129 pgs: 129 active+clean; 645 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-10T09:37:39.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:37:38 smithi159 ceph-mon[35513]: pgmap v372: 129 pgs: 129 active+clean; 645 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-10T09:37:39.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:37:38 smithi102 ceph-mon[35442]: pgmap v372: 129 pgs: 129 active+clean; 645 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-10T09:37:39.162 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:37:39.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:39.740+0000 7f63cbe00640 1 -- 172.21.15.5:0/1755014882 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f63cc102360 msgr2=0x7f63cc10e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:37:39.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:39.740+0000 7f63cbe00640 1 --2- 172.21.15.5:0/1755014882 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f63cc102360 0x7f63cc10e890 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f63c4009cd0 tx=0x7f63c402a8e0 comp rx=0 tx=0).stop 2024-09-10T09:37:39.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:39.740+0000 7f63cbe00640 1 -- 172.21.15.5:0/1755014882 shutdown_connections 2024-09-10T09:37:39.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:39.740+0000 7f63cbe00640 1 --2- 172.21.15.5:0/1755014882 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f63cc102360 0x7f63cc10e890 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:39.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:39.740+0000 7f63cbe00640 1 --2- 172.21.15.5:0/1755014882 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f63cc1019c0 0x7f63cc101e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:39.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:39.740+0000 7f63cbe00640 1 --2- 172.21.15.5:0/1755014882 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f63cc1079c0 0x7f63cc107da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:39.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:39.740+0000 7f63cbe00640 1 -- 172.21.15.5:0/1755014882 >> 172.21.15.5:0/1755014882 conn(0x7f63cc0fd740 msgr2=0x7f63cc0ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:37:39.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:39.740+0000 7f63cbe00640 1 -- 172.21.15.5:0/1755014882 shutdown_connections 2024-09-10T09:37:39.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:39.741+0000 7f63cbe00640 1 -- 172.21.15.5:0/1755014882 wait complete. 2024-09-10T09:37:39.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:39.741+0000 7f63cbe00640 1 Processor -- start 2024-09-10T09:37:39.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:39.742+0000 7f63cbe00640 1 -- start start 2024-09-10T09:37:39.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:39.742+0000 7f63cbe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f63cc1019c0 0x7f63cc19a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:37:39.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:39.742+0000 7f63cbe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f63cc102360 0x7f63cc19a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:37:39.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:39.742+0000 7f63cbe00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f63cc1079c0 0x7f63cc19ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:37:39.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:39.742+0000 7f63cbe00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f63cc111e10 con 0x7f63cc1019c0 2024-09-10T09:37:39.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:39.742+0000 7f63cbe00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f63cc111c90 con 0x7f63cc102360 2024-09-10T09:37:39.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:39.742+0000 7f63cbe00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f63cc111f90 con 0x7f63cc1079c0 2024-09-10T09:37:39.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:39.742+0000 7f63caa00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f63cc1019c0 0x7f63cc19a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:37:39.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:39.742+0000 7f63caa00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f63cc1019c0 0x7f63cc19a490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34320/0 (socket says 172.21.15.5:34320) 2024-09-10T09:37:39.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:39.742+0000 7f63caa00640 1 -- 172.21.15.5:0/78615915 learned_addr learned my addr 172.21.15.5:0/78615915 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:37:39.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:39.742+0000 7f63cb400640 1 --2- 172.21.15.5:0/78615915 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f63cc1079c0 0x7f63cc19ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:37:39.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:39.742+0000 7f63ca000640 1 --2- 172.21.15.5:0/78615915 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f63cc102360 0x7f63cc19a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:37:39.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:39.743+0000 7f63caa00640 1 -- 172.21.15.5:0/78615915 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f63cc1079c0 msgr2=0x7f63cc19ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:37:39.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:39.743+0000 7f63caa00640 1 --2- 172.21.15.5:0/78615915 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f63cc1079c0 0x7f63cc19ed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:39.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:39.743+0000 7f63caa00640 1 -- 172.21.15.5:0/78615915 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f63cc102360 msgr2=0x7f63cc19a9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:37:39.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:39.743+0000 7f63caa00640 1 --2- 172.21.15.5:0/78615915 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f63cc102360 0x7f63cc19a9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:39.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:39.743+0000 7f63caa00640 1 -- 172.21.15.5:0/78615915 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63cc19f440 con 0x7f63cc1019c0 2024-09-10T09:37:39.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:39.743+0000 7f63caa00640 1 --2- 172.21.15.5:0/78615915 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f63cc1019c0 0x7f63cc19a490 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f63bc00e9f0 tx=0x7f63bc00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:37:39.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:39.743+0000 7f63c3400640 1 -- 172.21.15.5:0/78615915 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63bc00cd80 con 0x7f63cc1019c0 2024-09-10T09:37:39.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:39.743+0000 7f63c3400640 1 -- 172.21.15.5:0/78615915 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f63bc005500 con 0x7f63cc1019c0 2024-09-10T09:37:39.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:39.743+0000 7f63c3400640 1 -- 172.21.15.5:0/78615915 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63bc010690 con 0x7f63cc1019c0 2024-09-10T09:37:39.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:39.743+0000 7f63cbe00640 1 -- 172.21.15.5:0/78615915 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63cc19f730 con 0x7f63cc1019c0 2024-09-10T09:37:39.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:39.743+0000 7f63cbe00640 1 -- 172.21.15.5:0/78615915 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63cc1a7010 con 0x7f63cc1019c0 2024-09-10T09:37:39.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:39.746+0000 7f63d4289740 1 Processor -- start 2024-09-10T09:37:39.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:39.746+0000 7f63c3400640 1 -- 172.21.15.5:0/78615915 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f63bc0046c0 con 0x7f63cc1019c0 2024-09-10T09:37:39.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:39.746+0000 7f63d4289740 1 -- start start 2024-09-10T09:37:39.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:39.747+0000 7f63d4289740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560af4a754e0 0x560af4a758c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:37:39.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:39.747+0000 7f63d4289740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560af4a75e00 0x560af4a80180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:37:39.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:39.747+0000 7f63d4289740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560af4a806c0 0x560af4a82ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:37:39.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:39.747+0000 7f63d4289740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x560af49db0b0 con 0x560af4a806c0 2024-09-10T09:37:39.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:39.747+0000 7f63d4289740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x560af4982ab0 con 0x560af4a754e0 2024-09-10T09:37:39.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:39.747+0000 7f63d4289740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x560af49d7070 con 0x560af4a75e00 2024-09-10T09:37:39.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:39.747+0000 7f63ca000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560af4a806c0 0x560af4a82ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:37:39.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:39.747+0000 7f63caa00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560af4a75e00 0x560af4a80180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:37:39.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:39.747+0000 7f63cb400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560af4a754e0 0x560af4a758c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:37:39.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:39.747+0000 7f63ca000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560af4a806c0 0x560af4a82ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34324/0 (socket says 172.21.15.5:34324) 2024-09-10T09:37:39.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:39.747+0000 7f63ca000640 1 -- 172.21.15.5:0/3492402071 learned_addr learned my addr 172.21.15.5:0/3492402071 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:37:39.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:39.748+0000 7f63cb400640 1 -- 172.21.15.5:0/3492402071 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560af4a75e00 msgr2=0x560af4a80180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:37:39.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:39.748+0000 7f63cb400640 1 --2- 172.21.15.5:0/3492402071 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560af4a75e00 0x560af4a80180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:39.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:39.748+0000 7f63cb400640 1 -- 172.21.15.5:0/3492402071 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560af4a806c0 msgr2=0x560af4a82ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:37:39.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:39.748+0000 7f63cb400640 1 --2- 172.21.15.5:0/3492402071 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560af4a806c0 0x560af4a82ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:39.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:39.748+0000 7f63cb400640 1 -- 172.21.15.5:0/3492402071 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560af498f4a0 con 0x560af4a754e0 2024-09-10T09:37:39.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:39.748+0000 7f63caa00640 1 --2- 172.21.15.5:0/3492402071 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560af4a75e00 0x560af4a80180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:37:39.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:39.748+0000 7f63cb400640 1 --2- 172.21.15.5:0/3492402071 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560af4a754e0 0x560af4a758c0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f63c4009ca0 tx=0x7f63c4009a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=66353918479b7a05 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:37:39.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:39.748+0000 7f63c3400640 1 --2- 172.21.15.5:0/78615915 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f63b00a2b80 0x7f63b00a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:37:39.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:39.748+0000 7f63c3400640 1 -- 172.21.15.5:0/78615915 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f63bc0cf830 con 0x7f63cc1019c0 2024-09-10T09:37:39.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:39.748+0000 7f63b7e00640 1 -- 172.21.15.5:0/3492402071 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63c4049070 con 0x560af4a754e0 2024-09-10T09:37:39.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:39.748+0000 7f63b7e00640 1 -- 172.21.15.5:0/3492402071 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f63c4036040 con 0x560af4a754e0 2024-09-10T09:37:39.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:39.748+0000 7f63b7e00640 1 -- 172.21.15.5:0/3492402071 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63c4004040 con 0x560af4a754e0 2024-09-10T09:37:39.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:39.748+0000 7f63caa00640 1 --2- 172.21.15.5:0/78615915 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f63b00a2b80 0x7f63b00a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:37:39.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:39.749+0000 7f63d4289740 1 -- 172.21.15.5:0/3492402071 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560af4a754e0 msgr2=0x560af4a758c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:37:39.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:39.749+0000 7f63caa00640 1 --2- 172.21.15.5:0/78615915 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f63b00a2b80 0x7f63b00a5040 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f63bc09bbd0 tx=0x7f63bc0d5530 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:37:39.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:39.749+0000 7f63d4289740 1 --2- 172.21.15.5:0/3492402071 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560af4a754e0 0x560af4a758c0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f63c4009ca0 tx=0x7f63c4009a20 comp rx=0 tx=0).stop 2024-09-10T09:37:39.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:39.749+0000 7f63d4289740 1 -- 172.21.15.5:0/3492402071 shutdown_connections 2024-09-10T09:37:39.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:39.749+0000 7f63d4289740 1 --2- 172.21.15.5:0/3492402071 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560af4a806c0 0x560af4a82ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:39.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:39.749+0000 7f63d4289740 1 --2- 172.21.15.5:0/3492402071 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560af4a75e00 0x560af4a80180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:39.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:39.749+0000 7f63d4289740 1 --2- 172.21.15.5:0/3492402071 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560af4a754e0 0x560af4a758c0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:39.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:39.749+0000 7f63d4289740 1 -- 172.21.15.5:0/3492402071 >> 172.21.15.5:0/3492402071 conn(0x560af4a71a70 msgr2=0x560af4a71e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:37:39.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:39.749+0000 7f63d4289740 1 -- 172.21.15.5:0/3492402071 shutdown_connections 2024-09-10T09:37:39.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:39.749+0000 7f63d4289740 1 -- 172.21.15.5:0/3492402071 wait complete. 2024-09-10T09:37:39.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:39.750+0000 7f63d4289740 1 -- start start 2024-09-10T09:37:39.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:39.750+0000 7f63d4289740 1 Processor -- start 2024-09-10T09:37:39.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:39.751+0000 7f63d4289740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560af4a754e0 0x560af4a7d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:37:39.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:39.751+0000 7f63d4289740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560af4a75e00 0x560af4a7db20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:37:39.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:39.751+0000 7f63d4289740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560af4a806c0 0x560af4a7e060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:37:39.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:39.751+0000 7f63d4289740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x560af4a83330 con 0x560af4a806c0 2024-09-10T09:37:39.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:39.751+0000 7f63d4289740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x560af4963920 con 0x560af4a754e0 2024-09-10T09:37:39.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:39.751+0000 7f63d4289740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x560af4a834b0 con 0x560af4a75e00 2024-09-10T09:37:39.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:39.751+0000 7f63d4289740 10 client.0 _ll_register_callbacks cb 0x560af49878a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:37:39.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:39.751+0000 7f63cb400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560af4a754e0 0x560af4a7d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:37:39.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:39.751+0000 7f63cb400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560af4a754e0 0x560af4a7d5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:48726/0 (socket says 172.21.15.5:48726) 2024-09-10T09:37:39.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:39.751+0000 7f63cb400640 1 -- 172.21.15.5:0/3395913961 learned_addr learned my addr 172.21.15.5:0/3395913961 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:37:39.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:39.751+0000 7f63cb400640 1 --2- 172.21.15.5:0/3395913961 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560af4a806c0 0x560af4a7e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:37:39.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:39.751+0000 7f63ca000640 1 --2- 172.21.15.5:0/3395913961 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560af4a75e00 0x560af4a7db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:37:39.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:39.751+0000 7f63cb400640 1 -- 172.21.15.5:0/3395913961 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560af4a75e00 msgr2=0x560af4a7db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:37:39.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:39.751+0000 7f63cb400640 1 --2- 172.21.15.5:0/3395913961 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560af4a75e00 0x560af4a7db20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:39.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:39.751+0000 7f63cb400640 1 -- 172.21.15.5:0/3395913961 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560af4a754e0 msgr2=0x560af4a7d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:37:39.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:39.751+0000 7f63cb400640 1 --2- 172.21.15.5:0/3395913961 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560af4a754e0 0x560af4a7d5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:39.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:39.751+0000 7f63cb400640 1 -- 172.21.15.5:0/3395913961 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560af474c2b0 con 0x560af4a806c0 2024-09-10T09:37:39.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:39.752+0000 7f63cb400640 1 --2- 172.21.15.5:0/3395913961 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560af4a806c0 0x560af4a7e060 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f63c4009bf0 tx=0x7f63c402a8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b7b0e938a0967a1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:37:39.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:39.752+0000 7f63b4c00640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-10T09:37:39.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:39.752+0000 7f63b4c00640 1 -- 172.21.15.5:0/3395913961 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63c4049030 con 0x560af4a806c0 2024-09-10T09:37:39.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:39.752+0000 7f63b4c00640 1 -- 172.21.15.5:0/3395913961 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f63c4041040 con 0x560af4a806c0 2024-09-10T09:37:39.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:39.752+0000 7f63d4289740 1 -- 172.21.15.5:0/3395913961 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f63c4036040 con 0x560af4a806c0 2024-09-10T09:37:39.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:39.752+0000 7f63b4c00640 1 -- 172.21.15.5:0/3395913961 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f63c405b050 con 0x560af4a806c0 2024-09-10T09:37:39.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:39.752+0000 7f63d4289740 10 client.15030 fetch_fsmap learned FSMap version 29 2024-09-10T09:37:39.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:39.752+0000 7f63d4289740 1 -- 172.21.15.5:0/3395913961 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x560af4a83330 con 0x560af4a806c0 2024-09-10T09:37:39.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:39.753+0000 7f63b4c00640 1 -- 172.21.15.5:0/3395913961 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f63c4036040 con 0x560af4a806c0 2024-09-10T09:37:39.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:39.753+0000 7f63b4c00640 1 -- 172.21.15.5:0/3395913961 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f63c4030070 con 0x560af4a806c0 2024-09-10T09:37:39.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:39.753+0000 7f63d4289740 10 client.15030 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T09:37:39.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:39.753+0000 7f63d4289740 10 client.15030 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-10T09:37:39.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:39.753+0000 7f63d4289740 20 client.15030 populate_metadata read hostname 'smithi005' 2024-09-10T09:37:39.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:39.753+0000 7f63d4289740 1 --2- 172.21.15.5:0/3395913961 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x560af4a91190 0x560af4a92600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:37:39.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:39.753+0000 7f63d4289740 4 client.15030 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:37:39.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:39.753+0000 7f63d4289740 1 -- 172.21.15.5:0/3395913961 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560af49e8620 con 0x560af4a91190 2024-09-10T09:37:39.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:39.753+0000 7f63ca000640 1 --2- 172.21.15.5:0/3395913961 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x560af4a91190 0x560af4a92600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:37:39.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:39.754+0000 7f63ca000640 1 --2- 172.21.15.5:0/3395913961 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x560af4a91190 0x560af4a92600 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fc186ef55fdb8830 server_cookie=deca262b57fac017 in_seq=0 out_seq=0 2024-09-10T09:37:39.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:39.754+0000 7f63b4c00640 10 client.15030 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T09:37:39.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:39.755+0000 7f63b4c00640 1 -- 172.21.15.5:0/3395913961 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560af49e8620 con 0x560af4a91190 2024-09-10T09:37:39.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:39.755+0000 7f63b4c00640 10 client.15030 handle_command_reply: tid=0 2024-09-10T09:37:39.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:39.755+0000 7f63d4289740 1 client.15030 shutdown 2024-09-10T09:37:39.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:39.755+0000 7f63d4289740 1 -- 172.21.15.5:0/3395913961 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560af4a806c0 msgr2=0x560af4a7e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:37:39.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:39.755+0000 7f63d4289740 1 --2- 172.21.15.5:0/3395913961 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560af4a806c0 0x560af4a7e060 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f63c4009bf0 tx=0x7f63c402a8e0 comp rx=0 tx=0).stop 2024-09-10T09:37:39.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:39.755+0000 7f63d4289740 1 -- 172.21.15.5:0/3395913961 shutdown_connections 2024-09-10T09:37:39.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:39.755+0000 7f63d4289740 1 --2- 172.21.15.5:0/3395913961 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x560af4a91190 0x560af4a92600 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:39.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:39.755+0000 7f63d4289740 1 --2- 172.21.15.5:0/3395913961 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560af4a806c0 0x560af4a7e060 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:39.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:39.755+0000 7f63d4289740 1 --2- 172.21.15.5:0/3395913961 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560af4a75e00 0x560af4a7db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:39.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:39.755+0000 7f63d4289740 1 --2- 172.21.15.5:0/3395913961 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560af4a754e0 0x560af4a7d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:39.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:39.755+0000 7f63b4c00640 0 client.15030 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T09:37:39.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:39.755+0000 7f63d4289740 1 -- 172.21.15.5:0/3395913961 >> 172.21.15.5:0/3395913961 conn(0x560af4a71a70 msgr2=0x560af4a70bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:37:39.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:39.755+0000 7f63d4289740 1 -- 172.21.15.5:0/3395913961 shutdown_connections 2024-09-10T09:37:39.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:39.755+0000 7f63d4289740 1 -- 172.21.15.5:0/3395913961 wait complete. 2024-09-10T09:37:39.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:39.756+0000 7f63d4289740 20 client.15030 trim_cache size 0 max 16384 2024-09-10T09:37:39.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:39.775+0000 7f63d4289740 1 Processor -- start 2024-09-10T09:37:39.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:39.775+0000 7f63d4289740 1 -- start start 2024-09-10T09:37:39.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:39.775+0000 7f63d4289740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560af4a754e0 0x560af4a6c840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:37:39.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:39.775+0000 7f63d4289740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560af4a75e00 0x560af4a7bf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:37:39.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:39.775+0000 7f63d4289740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560af4a806c0 0x560af4a7c490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:37:39.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:39.775+0000 7f63d4289740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x560af4a90940 con 0x560af4a75e00 2024-09-10T09:37:39.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:39.775+0000 7f63d4289740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x560af4a91010 con 0x560af4a806c0 2024-09-10T09:37:39.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:39.775+0000 7f63d4289740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x560af4a916e0 con 0x560af4a754e0 2024-09-10T09:37:39.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:39.776+0000 7f63ca000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560af4a75e00 0x560af4a7bf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:37:39.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:39.776+0000 7f63ca000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560af4a75e00 0x560af4a7bf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34342/0 (socket says 172.21.15.5:34342) 2024-09-10T09:37:39.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:39.776+0000 7f63cb400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560af4a754e0 0x560af4a6c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:37:39.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:39.776+0000 7f63ca000640 1 -- 172.21.15.5:0/3783901205 learned_addr learned my addr 172.21.15.5:0/3783901205 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:37:39.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:39.776+0000 7f63cb400640 1 --2- 172.21.15.5:0/3783901205 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560af4a806c0 0x560af4a7c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:37:39.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:39.776+0000 7f63cb400640 1 -- 172.21.15.5:0/3783901205 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560af4a806c0 msgr2=0x560af4a7c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:37:39.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:39.776+0000 7f63cb400640 1 --2- 172.21.15.5:0/3783901205 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560af4a806c0 0x560af4a7c490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:39.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:39.776+0000 7f63cb400640 1 -- 172.21.15.5:0/3783901205 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560af4a75e00 msgr2=0x560af4a7bf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:37:39.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:39.776+0000 7f63cb400640 1 --2- 172.21.15.5:0/3783901205 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560af4a75e00 0x560af4a7bf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:39.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:39.776+0000 7f63cb400640 1 -- 172.21.15.5:0/3783901205 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560af4a7bbd0 con 0x560af4a754e0 2024-09-10T09:37:39.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:39.776+0000 7f63cb400640 1 --2- 172.21.15.5:0/3783901205 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560af4a754e0 0x560af4a6c840 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f63c4030ad0 tx=0x7f63c4009920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fe473c54f47ac688 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:37:39.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:39.777+0000 7f63b7400640 1 -- 172.21.15.5:0/3783901205 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63c4049030 con 0x560af4a754e0 2024-09-10T09:37:39.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:39.777+0000 7f63b7400640 1 -- 172.21.15.5:0/3783901205 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f63c404a040 con 0x560af4a754e0 2024-09-10T09:37:39.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:39.777+0000 7f63b7400640 1 -- 172.21.15.5:0/3783901205 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63c4058040 con 0x560af4a754e0 2024-09-10T09:37:39.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:39.777+0000 7f63d4289740 1 -- 172.21.15.5:0/3783901205 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560af4a754e0 msgr2=0x560af4a6c840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:37:39.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:39.777+0000 7f63d4289740 1 --2- 172.21.15.5:0/3783901205 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560af4a754e0 0x560af4a6c840 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f63c4030ad0 tx=0x7f63c4009920 comp rx=0 tx=0).stop 2024-09-10T09:37:39.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:39.778+0000 7f63d4289740 1 -- 172.21.15.5:0/3783901205 shutdown_connections 2024-09-10T09:37:39.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:39.778+0000 7f63d4289740 1 --2- 172.21.15.5:0/3783901205 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560af4a806c0 0x560af4a7c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:39.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:39.778+0000 7f63d4289740 1 --2- 172.21.15.5:0/3783901205 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560af4a75e00 0x560af4a7bf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:39.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:39.778+0000 7f63d4289740 1 --2- 172.21.15.5:0/3783901205 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560af4a754e0 0x560af4a6c840 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:39.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:39.778+0000 7f63d4289740 1 -- 172.21.15.5:0/3783901205 >> 172.21.15.5:0/3783901205 conn(0x560af4a71a70 msgr2=0x560af4a90190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:37:39.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:39.778+0000 7f63d4289740 1 -- 172.21.15.5:0/3783901205 shutdown_connections 2024-09-10T09:37:39.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:39.778+0000 7f63d4289740 1 -- 172.21.15.5:0/3783901205 wait complete. 2024-09-10T09:37:39.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:39.778+0000 7f63d4289740 1 -- start start 2024-09-10T09:37:39.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:39.778+0000 7f63d4289740 1 Processor -- start 2024-09-10T09:37:39.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:39.779+0000 7f63d4289740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560af4a754e0 0x560af4a9afb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:37:39.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:39.779+0000 7f63d4289740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560af4a75e00 0x560af4a7a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:37:39.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:39.779+0000 7f63d4289740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560af4a806c0 0x560af4a7a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:37:39.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:39.779+0000 7f63d4289740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x560af4a92480 con 0x560af4a806c0 2024-09-10T09:37:39.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:39.779+0000 7f63d4289740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x560af4a91db0 con 0x560af4a754e0 2024-09-10T09:37:39.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:39.779+0000 7f63d4289740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x560af4a92b50 con 0x560af4a75e00 2024-09-10T09:37:39.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:39.779+0000 7f63d4289740 10 client.0 _ll_register_callbacks cb 0x7f63c4009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:37:39.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:39.779+0000 7f63ca000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560af4a75e00 0x560af4a7a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:37:39.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:39.779+0000 7f63cb400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560af4a754e0 0x560af4a9afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:37:39.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:39.779+0000 7f63ca000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560af4a75e00 0x560af4a7a3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:45106/0 (socket says 172.21.15.5:45106) 2024-09-10T09:37:39.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:39.779+0000 7f63ca000640 1 -- 172.21.15.5:0/4061715158 learned_addr learned my addr 172.21.15.5:0/4061715158 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:37:39.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:39.779+0000 7f63cb400640 1 --2- 172.21.15.5:0/4061715158 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560af4a806c0 0x560af4a7a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:37:39.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:39.779+0000 7f63ca000640 1 -- 172.21.15.5:0/4061715158 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560af4a754e0 msgr2=0x560af4a9afb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:37:39.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:39.779+0000 7f63ca000640 1 --2- 172.21.15.5:0/4061715158 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560af4a754e0 0x560af4a9afb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:39.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:39.780+0000 7f63ca000640 1 -- 172.21.15.5:0/4061715158 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560af4a806c0 msgr2=0x560af4a7a8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:37:39.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:39.780+0000 7f63ca000640 1 --2- 172.21.15.5:0/4061715158 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560af4a806c0 0x560af4a7a8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:39.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:39.780+0000 7f63ca000640 1 -- 172.21.15.5:0/4061715158 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560af4a7b0d0 con 0x560af4a75e00 2024-09-10T09:37:39.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:39.780+0000 7f63cb400640 1 --2- 172.21.15.5:0/4061715158 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560af4a806c0 0x560af4a7a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:37:39.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:39.780+0000 7f63ca000640 1 --2- 172.21.15.5:0/4061715158 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560af4a75e00 0x560af4a7a3b0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x560af4a72df0 tx=0x7f63b8020630 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e5a6c8393aaec92d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:37:39.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:39.780+0000 7f63b4c00640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-09-10T09:37:39.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:39.780+0000 7f63b4c00640 1 -- 172.21.15.5:0/4061715158 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63b8004230 con 0x560af4a75e00 2024-09-10T09:37:39.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:39.780+0000 7f63b4c00640 1 -- 172.21.15.5:0/4061715158 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f63b80043d0 con 0x560af4a75e00 2024-09-10T09:37:39.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:39.780+0000 7f63d4289740 1 -- 172.21.15.5:0/4061715158 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f63c404a040 con 0x560af4a75e00 2024-09-10T09:37:39.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:39.780+0000 7f63b4c00640 1 -- 172.21.15.5:0/4061715158 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f63b8010640 con 0x560af4a75e00 2024-09-10T09:37:39.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:39.781+0000 7f63d4289740 10 client.25069 fetch_fsmap learned FSMap version 29 2024-09-10T09:37:39.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:39.781+0000 7f63d4289740 1 -- 172.21.15.5:0/4061715158 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x560af4afb930 con 0x560af4a75e00 2024-09-10T09:37:39.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:39.781+0000 7f63b4c00640 1 -- 172.21.15.5:0/4061715158 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f63c404a040 con 0x560af4a75e00 2024-09-10T09:37:39.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:39.781+0000 7f63b4c00640 1 -- 172.21.15.5:0/4061715158 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f63b8025070 con 0x560af4a75e00 2024-09-10T09:37:39.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:39.781+0000 7f63d4289740 10 client.25069 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T09:37:39.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:39.781+0000 7f63d4289740 10 client.25069 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-10T09:37:39.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:39.781+0000 7f63d4289740 20 client.25069 populate_metadata read hostname 'smithi005' 2024-09-10T09:37:39.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:39.781+0000 7f63d4289740 1 --2- 172.21.15.5:0/4061715158 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x560af4b005e0 0x560af4aff2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:37:39.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:39.781+0000 7f63d4289740 4 client.25069 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T09:37:39.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:39.781+0000 7f63d4289740 1 -- 172.21.15.5:0/4061715158 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "status"}) -- 0x560af49e2b30 con 0x560af4b005e0 2024-09-10T09:37:39.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:39.781+0000 7f63cb400640 1 --2- 172.21.15.5:0/4061715158 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x560af4b005e0 0x560af4aff2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:37:39.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:39.782+0000 7f63cb400640 1 --2- 172.21.15.5:0/4061715158 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x560af4b005e0 0x560af4aff2f0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d8d86ad6ed3e5611 server_cookie=9005c39994c47625 in_seq=0 out_seq=0 2024-09-10T09:37:39.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:39.782+0000 7f63b4c00640 10 client.25069 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T09:37:39.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:39.782+0000 7f63b4c00640 1 -- 172.21.15.5:0/4061715158 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x560af49e2b30 con 0x560af4b005e0 2024-09-10T09:37:39.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:39.782+0000 7f63b4c00640 10 client.25069 handle_command_reply: tid=0 2024-09-10T09:37:39.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:39.782+0000 7f63d4289740 1 client.25069 shutdown 2024-09-10T09:37:39.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:39.782+0000 7f63d4289740 1 -- 172.21.15.5:0/4061715158 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560af4a75e00 msgr2=0x560af4a7a3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:37:39.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:39.782+0000 7f63d4289740 1 --2- 172.21.15.5:0/4061715158 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560af4a75e00 0x560af4a7a3b0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x560af4a72df0 tx=0x7f63b8020630 comp rx=0 tx=0).stop 2024-09-10T09:37:39.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:39.783+0000 7f63d4289740 1 -- 172.21.15.5:0/4061715158 shutdown_connections 2024-09-10T09:37:39.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:39.783+0000 7f63d4289740 1 --2- 172.21.15.5:0/4061715158 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x560af4b005e0 0x560af4aff2f0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:39.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:39.783+0000 7f63d4289740 1 --2- 172.21.15.5:0/4061715158 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560af4a806c0 0x560af4a7a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:39.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:39.783+0000 7f63d4289740 1 --2- 172.21.15.5:0/4061715158 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560af4a75e00 0x560af4a7a3b0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:39.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:39.783+0000 7f63d4289740 1 --2- 172.21.15.5:0/4061715158 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560af4a754e0 0x560af4a9afb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:39.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:39.783+0000 7f63d4289740 1 -- 172.21.15.5:0/4061715158 >> 172.21.15.5:0/4061715158 conn(0x560af4a71a70 msgr2=0x560af4a90190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:37:39.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:39.783+0000 7f63b4c00640 0 client.25069 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T09:37:39.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:39.783+0000 7f63d4289740 1 -- 172.21.15.5:0/4061715158 shutdown_connections 2024-09-10T09:37:39.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:39.783+0000 7f63d4289740 1 -- 172.21.15.5:0/4061715158 wait complete. 2024-09-10T09:37:39.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:39.783+0000 7f63d4289740 20 client.25069 trim_cache size 0 max 16384 2024-09-10T09:37:39.790 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-10T09:37:39.791 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_fsid": "bade4b50-6f56-11ef-bcea-c7b262605968", 2024-09-10T09:37:39.791 INFO:teuthology.orchestra.run.smithi005.stdout: "whoami": 0, 2024-09-10T09:37:39.791 INFO:teuthology.orchestra.run.smithi005.stdout: "id": 24542, 2024-09-10T09:37:39.791 INFO:teuthology.orchestra.run.smithi005.stdout: "want_state": "up:active", 2024-09-10T09:37:39.791 INFO:teuthology.orchestra.run.smithi005.stdout: "state": "up:active", 2024-09-10T09:37:39.791 INFO:teuthology.orchestra.run.smithi005.stdout: "fs_name": "cephfs", 2024-09-10T09:37:39.791 INFO:teuthology.orchestra.run.smithi005.stdout: "rank_uptime": 129.21772167099999, 2024-09-10T09:37:39.791 INFO:teuthology.orchestra.run.smithi005.stdout: "mdsmap_epoch": 29, 2024-09-10T09:37:39.791 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch": 96, 2024-09-10T09:37:39.791 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch_barrier": 94, 2024-09-10T09:37:39.791 INFO:teuthology.orchestra.run.smithi005.stdout: "uptime": 142.833668977 2024-09-10T09:37:39.791 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-10T09:37:39.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:39.783+0000 7f63cbe00640 1 -- 172.21.15.5:0/78615915 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f63b00a2b80 msgr2=0x7f63b00a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:37:39.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:39.783+0000 7f63cbe00640 1 --2- 172.21.15.5:0/78615915 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f63b00a2b80 0x7f63b00a5040 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f63bc09bbd0 tx=0x7f63bc0d5530 comp rx=0 tx=0).stop 2024-09-10T09:37:39.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:39.784+0000 7f63cbe00640 1 -- 172.21.15.5:0/78615915 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f63cc1019c0 msgr2=0x7f63cc19a490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:37:39.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:39.784+0000 7f63cbe00640 1 --2- 172.21.15.5:0/78615915 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f63cc1019c0 0x7f63cc19a490 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f63bc00e9f0 tx=0x7f63bc00edf0 comp rx=0 tx=0).stop 2024-09-10T09:37:39.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:39.784+0000 7f63cbe00640 1 -- 172.21.15.5:0/78615915 shutdown_connections 2024-09-10T09:37:39.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:39.784+0000 7f63cbe00640 1 --2- 172.21.15.5:0/78615915 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f63b00a2b80 0x7f63b00a5040 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:39.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:39.784+0000 7f63cbe00640 1 --2- 172.21.15.5:0/78615915 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f63cc1079c0 0x7f63cc19ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:39.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:39.784+0000 7f63cbe00640 1 --2- 172.21.15.5:0/78615915 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f63cc102360 0x7f63cc19a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:39.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:39.784+0000 7f63cbe00640 1 --2- 172.21.15.5:0/78615915 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f63cc1019c0 0x7f63cc19a490 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:39.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:39.784+0000 7f63cbe00640 1 -- 172.21.15.5:0/78615915 >> 172.21.15.5:0/78615915 conn(0x7f63cc0fd740 msgr2=0x7f63cc0fdd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:37:39.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:39.784+0000 7f63cbe00640 1 -- 172.21.15.5:0/78615915 shutdown_connections 2024-09-10T09:37:39.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:39.784+0000 7f63cbe00640 1 -- 172.21.15.5:0/78615915 wait complete. 2024-09-10T09:37:40.629 INFO:tasks.cephadm:Setup complete, yielding 2024-09-10T09:37:40.629 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-09-10T09:37:40.640 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi005.front.sepia.ceph.com 2024-09-10T09:37:40.640 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- bash -c 'ceph orch status' 2024-09-10T09:37:41.022 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:37:41.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:37:40 smithi005 ceph-mon[30350]: pgmap v373: 129 pgs: 129 active+clean; 645 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-10T09:37:41.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:37:40 smithi159 ceph-mon[35513]: pgmap v373: 129 pgs: 129 active+clean; 645 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-10T09:37:41.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:37:40 smithi102 ceph-mon[35442]: pgmap v373: 129 pgs: 129 active+clean; 645 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-10T09:37:41.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:41.585+0000 7fd053800640 1 -- 172.21.15.5:0/1209842079 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd04c102e00 msgr2=0x7fd04c103280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:37:41.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:41.585+0000 7fd053800640 1 --2- 172.21.15.5:0/1209842079 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd04c102e00 0x7fd04c103280 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fd038009900 tx=0x7fd03802a400 comp rx=0 tx=0).stop 2024-09-10T09:37:41.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:41.585+0000 7fd053800640 1 -- 172.21.15.5:0/1209842079 shutdown_connections 2024-09-10T09:37:41.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:41.585+0000 7fd053800640 1 --2- 172.21.15.5:0/1209842079 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd04c1037c0 0x7fd04c10a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:41.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:41.585+0000 7fd053800640 1 --2- 172.21.15.5:0/1209842079 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd04c102e00 0x7fd04c103280 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:41.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:41.585+0000 7fd053800640 1 --2- 172.21.15.5:0/1209842079 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd04c101c00 0x7fd04c102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:41.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:41.585+0000 7fd053800640 1 -- 172.21.15.5:0/1209842079 >> 172.21.15.5:0/1209842079 conn(0x7fd04c0fd390 msgr2=0x7fd04c0ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:37:41.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:41.585+0000 7fd053800640 1 -- 172.21.15.5:0/1209842079 shutdown_connections 2024-09-10T09:37:41.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:41.585+0000 7fd053800640 1 -- 172.21.15.5:0/1209842079 wait complete. 2024-09-10T09:37:41.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:41.586+0000 7fd053800640 1 Processor -- start 2024-09-10T09:37:41.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:41.586+0000 7fd053800640 1 -- start start 2024-09-10T09:37:41.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:41.587+0000 7fd053800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd04c101c00 0x7fd04c19a110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:37:41.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:41.587+0000 7fd053800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd04c102e00 0x7fd04c19a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:37:41.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:41.587+0000 7fd053800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd04c1037c0 0x7fd04c1a16d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:37:41.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:41.587+0000 7fd053800640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd04c10d860 con 0x7fd04c1037c0 2024-09-10T09:37:41.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:41.587+0000 7fd053800640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fd04c10d6e0 con 0x7fd04c101c00 2024-09-10T09:37:41.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:41.587+0000 7fd053800640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fd04c10d9e0 con 0x7fd04c102e00 2024-09-10T09:37:41.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:41.587+0000 7fd050e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd04c101c00 0x7fd04c19a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:37:41.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:41.587+0000 7fd050e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd04c101c00 0x7fd04c19a110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:59142/0 (socket says 172.21.15.5:59142) 2024-09-10T09:37:41.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:41.587+0000 7fd050e00640 1 -- 172.21.15.5:0/2854927766 learned_addr learned my addr 172.21.15.5:0/2854927766 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:37:41.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:41.587+0000 7fd04be00640 1 --2- 172.21.15.5:0/2854927766 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd04c102e00 0x7fd04c19a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:37:41.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:41.587+0000 7fd051800640 1 --2- 172.21.15.5:0/2854927766 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd04c1037c0 0x7fd04c1a16d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:37:41.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:41.587+0000 7fd050e00640 1 -- 172.21.15.5:0/2854927766 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd04c102e00 msgr2=0x7fd04c19a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:37:41.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:41.587+0000 7fd050e00640 1 --2- 172.21.15.5:0/2854927766 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd04c102e00 0x7fd04c19a650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:41.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:41.587+0000 7fd050e00640 1 -- 172.21.15.5:0/2854927766 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd04c1037c0 msgr2=0x7fd04c1a16d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:37:41.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:41.587+0000 7fd050e00640 1 --2- 172.21.15.5:0/2854927766 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd04c1037c0 0x7fd04c1a16d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:41.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:41.587+0000 7fd050e00640 1 -- 172.21.15.5:0/2854927766 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd04c1a1d40 con 0x7fd04c101c00 2024-09-10T09:37:41.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:41.588+0000 7fd050e00640 1 --2- 172.21.15.5:0/2854927766 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd04c101c00 0x7fd04c19a110 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fd04000d770 tx=0x7fd04000dc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:37:41.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:41.588+0000 7fd049600640 1 -- 172.21.15.5:0/2854927766 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd040017070 con 0x7fd04c101c00 2024-09-10T09:37:41.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:41.588+0000 7fd049600640 1 -- 172.21.15.5:0/2854927766 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd040009c30 con 0x7fd04c101c00 2024-09-10T09:37:41.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:41.588+0000 7fd049600640 1 -- 172.21.15.5:0/2854927766 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd040003dd0 con 0x7fd04c101c00 2024-09-10T09:37:41.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:41.588+0000 7fd053800640 1 -- 172.21.15.5:0/2854927766 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd04c1a1fd0 con 0x7fd04c101c00 2024-09-10T09:37:41.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:41.588+0000 7fd053800640 1 -- 172.21.15.5:0/2854927766 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd04c1a2480 con 0x7fd04c101c00 2024-09-10T09:37:41.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:41.589+0000 7fd053800640 1 -- 172.21.15.5:0/2854927766 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd04c1a26b0 con 0x7fd04c101c00 2024-09-10T09:37:41.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:41.591+0000 7fd049600640 1 -- 172.21.15.5:0/2854927766 <== mon.2 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7fd040003fb0 con 0x7fd04c101c00 2024-09-10T09:37:41.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:41.592+0000 7fd049600640 1 --2- 172.21.15.5:0/2854927766 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fd0340a2c30 0x7fd0340a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:37:41.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:41.593+0000 7fd049600640 1 -- 172.21.15.5:0/2854927766 <== mon.2 v2:172.21.15.102:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fd0400ceca0 con 0x7fd04c101c00 2024-09-10T09:37:41.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:41.593+0000 7fd04be00640 1 --2- 172.21.15.5:0/2854927766 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fd0340a2c30 0x7fd0340a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:37:41.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:41.593+0000 7fd04be00640 1 --2- 172.21.15.5:0/2854927766 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fd0340a2c30 0x7fd0340a50f0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fd0380314e0 tx=0x7fd038000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:37:41.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:41.595+0000 7fd049600640 1 -- 172.21.15.5:0/2854927766 <== mon.2 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd040009de0 con 0x7fd04c101c00 2024-09-10T09:37:41.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:41.797+0000 7fd053800640 1 -- 172.21.15.5:0/2854927766 --> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) -- 0x7fd04c109cf0 con 0x7fd0340a2c30 2024-09-10T09:37:41.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:41.799+0000 7fd049600640 1 -- 172.21.15.5:0/2854927766 <== mgr.14150 v2:172.21.15.5:6800/2707348081 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7fd04c109cf0 con 0x7fd0340a2c30 2024-09-10T09:37:41.800 INFO:teuthology.orchestra.run.smithi005.stdout:Backend: cephadm 2024-09-10T09:37:41.800 INFO:teuthology.orchestra.run.smithi005.stdout:Available: Yes 2024-09-10T09:37:41.800 INFO:teuthology.orchestra.run.smithi005.stdout:Paused: No 2024-09-10T09:37:41.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:41.801+0000 7fd053800640 1 -- 172.21.15.5:0/2854927766 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fd0340a2c30 msgr2=0x7fd0340a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:37:41.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:41.801+0000 7fd053800640 1 --2- 172.21.15.5:0/2854927766 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fd0340a2c30 0x7fd0340a50f0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fd0380314e0 tx=0x7fd038000bc0 comp rx=0 tx=0).stop 2024-09-10T09:37:41.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:41.801+0000 7fd053800640 1 -- 172.21.15.5:0/2854927766 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd04c101c00 msgr2=0x7fd04c19a110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:37:41.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:41.801+0000 7fd053800640 1 --2- 172.21.15.5:0/2854927766 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd04c101c00 0x7fd04c19a110 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fd04000d770 tx=0x7fd04000dc30 comp rx=0 tx=0).stop 2024-09-10T09:37:41.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:41.802+0000 7fd053800640 1 -- 172.21.15.5:0/2854927766 shutdown_connections 2024-09-10T09:37:41.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:41.802+0000 7fd053800640 1 --2- 172.21.15.5:0/2854927766 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fd0340a2c30 0x7fd0340a50f0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:41.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:41.802+0000 7fd053800640 1 --2- 172.21.15.5:0/2854927766 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd04c1037c0 0x7fd04c1a16d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:41.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:41.802+0000 7fd053800640 1 --2- 172.21.15.5:0/2854927766 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd04c102e00 0x7fd04c19a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:41.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:41.802+0000 7fd053800640 1 --2- 172.21.15.5:0/2854927766 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd04c101c00 0x7fd04c19a110 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:41.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:41.802+0000 7fd053800640 1 -- 172.21.15.5:0/2854927766 >> 172.21.15.5:0/2854927766 conn(0x7fd04c0fd390 msgr2=0x7fd04c1085d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:37:41.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:41.802+0000 7fd053800640 1 -- 172.21.15.5:0/2854927766 shutdown_connections 2024-09-10T09:37:41.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:41.802+0000 7fd053800640 1 -- 172.21.15.5:0/2854927766 wait complete. 2024-09-10T09:37:42.514 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- bash -c 'ceph orch ps' 2024-09-10T09:37:42.893 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:37:43.033 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:37:42 smithi005 ceph-mon[30350]: pgmap v374: 129 pgs: 129 active+clean; 646 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 597 B/s wr, 12 op/s 2024-09-10T09:37:43.033 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:37:42 smithi005 ceph-mon[30350]: from='client.25058 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T09:37:43.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:37:42 smithi159 ceph-mon[35513]: pgmap v374: 129 pgs: 129 active+clean; 646 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 597 B/s wr, 12 op/s 2024-09-10T09:37:43.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:37:42 smithi159 ceph-mon[35513]: from='client.25058 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T09:37:43.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:37:42 smithi102 ceph-mon[35442]: pgmap v374: 129 pgs: 129 active+clean; 646 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 597 B/s wr, 12 op/s 2024-09-10T09:37:43.135 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:37:42 smithi102 ceph-mon[35442]: from='client.25058 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T09:37:43.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:43.450+0000 7ffb67200640 1 -- 172.21.15.5:0/836262999 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffb68102c40 msgr2=0x7ffb681030c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:37:43.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:43.450+0000 7ffb67200640 1 --2- 172.21.15.5:0/836262999 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffb68102c40 0x7ffb681030c0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7ffb54009900 tx=0x7ffb5402a3f0 comp rx=0 tx=0).stop 2024-09-10T09:37:43.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:43.450+0000 7ffb67200640 1 -- 172.21.15.5:0/836262999 shutdown_connections 2024-09-10T09:37:43.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:43.450+0000 7ffb67200640 1 --2- 172.21.15.5:0/836262999 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ffb68103600 0x7ffb68109e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:43.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:43.450+0000 7ffb67200640 1 --2- 172.21.15.5:0/836262999 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffb68102c40 0x7ffb681030c0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:43.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:43.450+0000 7ffb67200640 1 --2- 172.21.15.5:0/836262999 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ffb68101a40 0x7ffb68101e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:43.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:43.450+0000 7ffb67200640 1 -- 172.21.15.5:0/836262999 >> 172.21.15.5:0/836262999 conn(0x7ffb680fd1f0 msgr2=0x7ffb680ff610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:37:43.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:43.451+0000 7ffb67200640 1 -- 172.21.15.5:0/836262999 shutdown_connections 2024-09-10T09:37:43.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:43.451+0000 7ffb67200640 1 -- 172.21.15.5:0/836262999 wait complete. 2024-09-10T09:37:43.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:43.451+0000 7ffb67200640 1 Processor -- start 2024-09-10T09:37:43.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:43.452+0000 7ffb67200640 1 -- start start 2024-09-10T09:37:43.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:43.452+0000 7ffb67200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ffb68101a40 0x7ffb68075f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:37:43.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:43.452+0000 7ffb67200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffb68102c40 0x7ffb680745e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:37:43.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:43.452+0000 7ffb67200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ffb68103600 0x7ffb68074b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:37:43.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:43.452+0000 7ffb67200640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ffb6810d710 con 0x7ffb68102c40 2024-09-10T09:37:43.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:43.452+0000 7ffb67200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7ffb6810d590 con 0x7ffb68101a40 2024-09-10T09:37:43.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:43.452+0000 7ffb67200640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7ffb6810d890 con 0x7ffb68103600 2024-09-10T09:37:43.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:43.452+0000 7ffb65400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffb68102c40 0x7ffb680745e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:37:43.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:43.452+0000 7ffb65400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffb68102c40 0x7ffb680745e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43546/0 (socket says 172.21.15.5:43546) 2024-09-10T09:37:43.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:43.452+0000 7ffb65400640 1 -- 172.21.15.5:0/522110243 learned_addr learned my addr 172.21.15.5:0/522110243 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:37:43.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:43.452+0000 7ffb65e00640 1 --2- 172.21.15.5:0/522110243 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ffb68101a40 0x7ffb68075f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:37:43.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:43.452+0000 7ffb66800640 1 --2- 172.21.15.5:0/522110243 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ffb68103600 0x7ffb68074b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:37:43.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:43.453+0000 7ffb65e00640 1 -- 172.21.15.5:0/522110243 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ffb68103600 msgr2=0x7ffb68074b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:37:43.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:43.453+0000 7ffb65e00640 1 --2- 172.21.15.5:0/522110243 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ffb68103600 0x7ffb68074b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:43.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:43.453+0000 7ffb65e00640 1 -- 172.21.15.5:0/522110243 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffb68102c40 msgr2=0x7ffb680745e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:37:43.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:43.453+0000 7ffb65e00640 1 --2- 172.21.15.5:0/522110243 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffb68102c40 0x7ffb680745e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:43.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:43.453+0000 7ffb65e00640 1 -- 172.21.15.5:0/522110243 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb680753e0 con 0x7ffb68101a40 2024-09-10T09:37:43.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:43.453+0000 7ffb65e00640 1 --2- 172.21.15.5:0/522110243 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ffb68101a40 0x7ffb68075f90 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7ffb5800e9c0 tx=0x7ffb5800ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:37:43.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:43.453+0000 7ffb5ea00640 1 -- 172.21.15.5:0/522110243 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb5800ccc0 con 0x7ffb68101a40 2024-09-10T09:37:43.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:43.453+0000 7ffb67200640 1 -- 172.21.15.5:0/522110243 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb681a1ea0 con 0x7ffb68101a40 2024-09-10T09:37:43.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:43.453+0000 7ffb5ea00640 1 -- 172.21.15.5:0/522110243 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffb580035f0 con 0x7ffb68101a40 2024-09-10T09:37:43.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:43.453+0000 7ffb5ea00640 1 -- 172.21.15.5:0/522110243 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb5800ce60 con 0x7ffb68101a40 2024-09-10T09:37:43.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:43.454+0000 7ffb67200640 1 -- 172.21.15.5:0/522110243 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb681a2290 con 0x7ffb68101a40 2024-09-10T09:37:43.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:43.455+0000 7ffb67200640 1 -- 172.21.15.5:0/522110243 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffb68108e10 con 0x7ffb68101a40 2024-09-10T09:37:43.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:43.457+0000 7ffb5ea00640 1 -- 172.21.15.5:0/522110243 <== mon.2 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7ffb580037a0 con 0x7ffb68101a40 2024-09-10T09:37:43.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:43.458+0000 7ffb5ea00640 1 --2- 172.21.15.5:0/522110243 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7ffb4c0a2b80 0x7ffb4c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:37:43.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:43.458+0000 7ffb5ea00640 1 -- 172.21.15.5:0/522110243 <== mon.2 v2:172.21.15.102:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7ffb580cf920 con 0x7ffb68101a40 2024-09-10T09:37:43.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:43.458+0000 7ffb65400640 1 --2- 172.21.15.5:0/522110243 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7ffb4c0a2b80 0x7ffb4c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:37:43.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:43.459+0000 7ffb65400640 1 --2- 172.21.15.5:0/522110243 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7ffb4c0a2b80 0x7ffb4c0a5040 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7ffb68075d10 tx=0x7ffb540315b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:37:43.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:43.460+0000 7ffb5ea00640 1 -- 172.21.15.5:0/522110243 <== mon.2 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ffb5809c4c0 con 0x7ffb68101a40 2024-09-10T09:37:43.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:43.656+0000 7ffb67200640 1 -- 172.21.15.5:0/522110243 --> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7ffb68062630 con 0x7ffb4c0a2b80 2024-09-10T09:37:43.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:43.669+0000 7ffb5ea00640 1 -- 172.21.15.5:0/522110243 <== mgr.14150 v2:172.21.15.5:6800/2707348081 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+4092 (secure 0 0 0) 0x7ffb68062630 con 0x7ffb4c0a2b80 2024-09-10T09:37:43.671 INFO:teuthology.orchestra.run.smithi005.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-10T09:37:43.671 INFO:teuthology.orchestra.run.smithi005.stdout:mds.a smithi005 running (2m) 10s ago 2m 16.8M - 19.1.1-305-g1cf819b7 41d671128357 0d820c5c0414 2024-09-10T09:37:43.671 INFO:teuthology.orchestra.run.smithi005.stdout:mds.b smithi102 running (2m) 13s ago 2m 26.7M - 19.1.1-305-g1cf819b7 41d671128357 1283ac49c22f 2024-09-10T09:37:43.671 INFO:teuthology.orchestra.run.smithi005.stdout:mds.c smithi159 running (2m) 13s ago 2m 21.3M - 19.1.1-305-g1cf819b7 41d671128357 d0d410063ea5 2024-09-10T09:37:43.671 INFO:teuthology.orchestra.run.smithi005.stdout:mds.d smithi005 running (2m) 10s ago 2m 16.8M - 19.1.1-305-g1cf819b7 41d671128357 cbc4e0ec9a31 2024-09-10T09:37:43.671 INFO:teuthology.orchestra.run.smithi005.stdout:mds.e smithi102 running (2m) 13s ago 2m 20.6M - 19.1.1-305-g1cf819b7 41d671128357 b23eaeab4c90 2024-09-10T09:37:43.671 INFO:teuthology.orchestra.run.smithi005.stdout:mds.f smithi159 running (2m) 13s ago 2m 18.8M - 19.1.1-305-g1cf819b7 41d671128357 206912c328a0 2024-09-10T09:37:43.671 INFO:teuthology.orchestra.run.smithi005.stdout:mds.g smithi005 running (2m) 10s ago 2m 12.9M - 19.1.1-305-g1cf819b7 41d671128357 3935bf04438d 2024-09-10T09:37:43.671 INFO:teuthology.orchestra.run.smithi005.stdout:mds.h smithi102 running (2m) 13s ago 2m 19.0M - 19.1.1-305-g1cf819b7 41d671128357 0989d5e3d2ed 2024-09-10T09:37:43.671 INFO:teuthology.orchestra.run.smithi005.stdout:mds.i smithi159 running (2m) 13s ago 2m 19.1M - 19.1.1-305-g1cf819b7 41d671128357 dfe08e65daed 2024-09-10T09:37:43.672 INFO:teuthology.orchestra.run.smithi005.stdout:mds.j smithi005 running (2m) 10s ago 2m 13.2M - 19.1.1-305-g1cf819b7 41d671128357 b66502621e8f 2024-09-10T09:37:43.672 INFO:teuthology.orchestra.run.smithi005.stdout:mds.k smithi102 running (2m) 13s ago 2m 21.4M - 19.1.1-305-g1cf819b7 41d671128357 a2e3425919b2 2024-09-10T09:37:43.672 INFO:teuthology.orchestra.run.smithi005.stdout:mds.l smithi159 running (3m) 13s ago 3m 20.1M - 19.1.1-305-g1cf819b7 41d671128357 8b40c3762f53 2024-09-10T09:37:43.672 INFO:teuthology.orchestra.run.smithi005.stdout:mgr.x smithi005 *:9283,8765 running (10m) 10s ago 10m 510M - 19.1.1-305-g1cf819b7 41d671128357 65d4fbac1f45 2024-09-10T09:37:43.672 INFO:teuthology.orchestra.run.smithi005.stdout:mgr.y smithi102 *:8443,8765 running (8m) 13s ago 8m 442M - 19.1.1-305-g1cf819b7 41d671128357 b5d70fe88cc5 2024-09-10T09:37:43.672 INFO:teuthology.orchestra.run.smithi005.stdout:mgr.z smithi159 *:8443,8765 running (8m) 13s ago 8m 441M - 19.1.1-305-g1cf819b7 41d671128357 0d11d57166b6 2024-09-10T09:37:43.672 INFO:teuthology.orchestra.run.smithi005.stdout:mon.a smithi005 running (10m) 10s ago 10m 50.2M 2048M 19.1.1-305-g1cf819b7 41d671128357 36f7420b5288 2024-09-10T09:37:43.672 INFO:teuthology.orchestra.run.smithi005.stdout:mon.b smithi102 running (8m) 13s ago 8m 41.6M 2048M 19.1.1-305-g1cf819b7 41d671128357 36a004c0c182 2024-09-10T09:37:43.672 INFO:teuthology.orchestra.run.smithi005.stdout:mon.c smithi159 running (9m) 13s ago 9m 44.3M 2048M 19.1.1-305-g1cf819b7 41d671128357 ae880eacaff2 2024-09-10T09:37:43.672 INFO:teuthology.orchestra.run.smithi005.stdout:osd.0 smithi005 running (8m) 10s ago 8m 55.6M 4273M 19.1.1-305-g1cf819b7 41d671128357 e5730e5c6ffc 2024-09-10T09:37:43.673 INFO:teuthology.orchestra.run.smithi005.stdout:osd.1 smithi102 running (7m) 13s ago 7m 58.4M 4272M 19.1.1-305-g1cf819b7 41d671128357 4033f25c36fa 2024-09-10T09:37:43.673 INFO:teuthology.orchestra.run.smithi005.stdout:osd.2 smithi159 running (7m) 13s ago 7m 47.2M 4272M 19.1.1-305-g1cf819b7 41d671128357 70e47655f99a 2024-09-10T09:37:43.673 INFO:teuthology.orchestra.run.smithi005.stdout:osd.3 smithi005 running (6m) 10s ago 6m 42.2M 4273M 19.1.1-305-g1cf819b7 41d671128357 5d96fa43618b 2024-09-10T09:37:43.673 INFO:teuthology.orchestra.run.smithi005.stdout:osd.4 smithi102 running (6m) 13s ago 6m 56.9M 4272M 19.1.1-305-g1cf819b7 41d671128357 e39fd048c401 2024-09-10T09:37:43.673 INFO:teuthology.orchestra.run.smithi005.stdout:osd.5 smithi159 running (6m) 13s ago 6m 46.6M 4272M 19.1.1-305-g1cf819b7 41d671128357 b693c1ca5c75 2024-09-10T09:37:43.673 INFO:teuthology.orchestra.run.smithi005.stdout:osd.6 smithi005 running (5m) 10s ago 5m 54.1M 4273M 19.1.1-305-g1cf819b7 41d671128357 d61f28e1ca26 2024-09-10T09:37:43.673 INFO:teuthology.orchestra.run.smithi005.stdout:osd.7 smithi102 running (5m) 13s ago 5m 44.6M 4272M 19.1.1-305-g1cf819b7 41d671128357 e6c83a922b2e 2024-09-10T09:37:43.673 INFO:teuthology.orchestra.run.smithi005.stdout:osd.8 smithi159 running (4m) 13s ago 4m 55.1M 4272M 19.1.1-305-g1cf819b7 41d671128357 18cf8016a892 2024-09-10T09:37:43.673 INFO:teuthology.orchestra.run.smithi005.stdout:osd.9 smithi005 running (4m) 10s ago 4m 56.5M 4273M 19.1.1-305-g1cf819b7 41d671128357 fd7e7a76bf30 2024-09-10T09:37:43.673 INFO:teuthology.orchestra.run.smithi005.stdout:osd.10 smithi102 running (3m) 13s ago 3m 43.5M 4272M 19.1.1-305-g1cf819b7 41d671128357 ac91c3d5affc 2024-09-10T09:37:43.673 INFO:teuthology.orchestra.run.smithi005.stdout:osd.11 smithi159 running (3m) 13s ago 3m 50.9M 4272M 19.1.1-305-g1cf819b7 41d671128357 6596a70bb228 2024-09-10T09:37:43.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:43.671+0000 7ffb67200640 1 -- 172.21.15.5:0/522110243 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7ffb4c0a2b80 msgr2=0x7ffb4c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:37:43.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:43.672+0000 7ffb67200640 1 --2- 172.21.15.5:0/522110243 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7ffb4c0a2b80 0x7ffb4c0a5040 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7ffb68075d10 tx=0x7ffb540315b0 comp rx=0 tx=0).stop 2024-09-10T09:37:43.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:43.672+0000 7ffb67200640 1 -- 172.21.15.5:0/522110243 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ffb68101a40 msgr2=0x7ffb68075f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:37:43.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:43.672+0000 7ffb67200640 1 --2- 172.21.15.5:0/522110243 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ffb68101a40 0x7ffb68075f90 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7ffb5800e9c0 tx=0x7ffb5800ee80 comp rx=0 tx=0).stop 2024-09-10T09:37:43.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:43.672+0000 7ffb67200640 1 -- 172.21.15.5:0/522110243 shutdown_connections 2024-09-10T09:37:43.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:43.672+0000 7ffb67200640 1 --2- 172.21.15.5:0/522110243 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7ffb4c0a2b80 0x7ffb4c0a5040 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:43.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:43.672+0000 7ffb67200640 1 --2- 172.21.15.5:0/522110243 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ffb68103600 0x7ffb68074b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:43.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:43.672+0000 7ffb67200640 1 --2- 172.21.15.5:0/522110243 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffb68102c40 0x7ffb680745e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:43.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:43.672+0000 7ffb67200640 1 --2- 172.21.15.5:0/522110243 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ffb68101a40 0x7ffb68075f90 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:43.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:43.672+0000 7ffb67200640 1 -- 172.21.15.5:0/522110243 >> 172.21.15.5:0/522110243 conn(0x7ffb680fd1f0 msgr2=0x7ffb68105fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:37:43.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:43.672+0000 7ffb67200640 1 -- 172.21.15.5:0/522110243 shutdown_connections 2024-09-10T09:37:43.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:43.672+0000 7ffb67200640 1 -- 172.21.15.5:0/522110243 wait complete. 2024-09-10T09:37:44.546 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- bash -c 'ceph orch ls' 2024-09-10T09:37:44.931 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:37:45.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:37:44 smithi005 ceph-mon[30350]: pgmap v375: 129 pgs: 129 active+clean; 646 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 597 B/s wr, 8 op/s 2024-09-10T09:37:45.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:37:44 smithi005 ceph-mon[30350]: from='client.25064 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T09:37:45.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:37:44 smithi159 ceph-mon[35513]: pgmap v375: 129 pgs: 129 active+clean; 646 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 597 B/s wr, 8 op/s 2024-09-10T09:37:45.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:37:44 smithi159 ceph-mon[35513]: from='client.25064 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T09:37:45.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:37:44 smithi102 ceph-mon[35442]: pgmap v375: 129 pgs: 129 active+clean; 646 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 597 B/s wr, 8 op/s 2024-09-10T09:37:45.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:37:44 smithi102 ceph-mon[35442]: from='client.25064 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T09:37:45.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:45.510+0000 7f578fe00640 1 -- 172.21.15.5:0/2265622401 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5790076ba0 msgr2=0x7f5790076fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:37:45.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:45.510+0000 7f578fe00640 1 --2- 172.21.15.5:0/2265622401 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5790076ba0 0x7f5790076fa0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f5780009900 tx=0x7f578002a3f0 comp rx=0 tx=0).stop 2024-09-10T09:37:45.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:45.510+0000 7f578fe00640 1 -- 172.21.15.5:0/2265622401 shutdown_connections 2024-09-10T09:37:45.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:45.510+0000 7f578fe00640 1 --2- 172.21.15.5:0/2265622401 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5790078790 0x7f5790079030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:45.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:45.510+0000 7f578fe00640 1 --2- 172.21.15.5:0/2265622401 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5790077df0 0x7f5790078250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:45.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:45.510+0000 7f578fe00640 1 --2- 172.21.15.5:0/2265622401 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5790076ba0 0x7f5790076fa0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:45.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:45.510+0000 7f578fe00640 1 -- 172.21.15.5:0/2265622401 >> 172.21.15.5:0/2265622401 conn(0x7f57900ff3e0 msgr2=0x7f5790101820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:37:45.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:45.510+0000 7f578fe00640 1 -- 172.21.15.5:0/2265622401 shutdown_connections 2024-09-10T09:37:45.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:45.510+0000 7f578fe00640 1 -- 172.21.15.5:0/2265622401 wait complete. 2024-09-10T09:37:45.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:45.511+0000 7f578fe00640 1 Processor -- start 2024-09-10T09:37:45.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:45.512+0000 7f578fe00640 1 -- start start 2024-09-10T09:37:45.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:45.512+0000 7f578fe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5790076ba0 0x7f579019e300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:37:45.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:45.512+0000 7f578fe00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5790077df0 0x7f579019e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:37:45.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:45.512+0000 7f578fe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5790078790 0x7f57901a58c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:37:45.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:45.512+0000 7f578fe00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f5790113d60 con 0x7f5790078790 2024-09-10T09:37:45.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:45.512+0000 7f578fe00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f5790113be0 con 0x7f5790076ba0 2024-09-10T09:37:45.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:45.512+0000 7f578fe00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f5790113ee0 con 0x7f5790077df0 2024-09-10T09:37:45.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:45.512+0000 7f578f400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5790078790 0x7f57901a58c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:37:45.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:45.512+0000 7f578f400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5790078790 0x7f57901a58c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43574/0 (socket says 172.21.15.5:43574) 2024-09-10T09:37:45.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:45.512+0000 7f578f400640 1 -- 172.21.15.5:0/4274893734 learned_addr learned my addr 172.21.15.5:0/4274893734 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:37:45.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:45.512+0000 7f578ea00640 1 --2- 172.21.15.5:0/4274893734 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5790076ba0 0x7f579019e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:37:45.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:45.512+0000 7f578e000640 1 --2- 172.21.15.5:0/4274893734 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5790077df0 0x7f579019e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:37:45.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:45.513+0000 7f578f400640 1 -- 172.21.15.5:0/4274893734 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5790077df0 msgr2=0x7f579019e840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:37:45.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:45.513+0000 7f578f400640 1 --2- 172.21.15.5:0/4274893734 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5790077df0 0x7f579019e840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:45.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:45.513+0000 7f578f400640 1 -- 172.21.15.5:0/4274893734 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5790076ba0 msgr2=0x7f579019e300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:37:45.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:45.513+0000 7f578f400640 1 --2- 172.21.15.5:0/4274893734 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5790076ba0 0x7f579019e300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:45.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:45.513+0000 7f578f400640 1 -- 172.21.15.5:0/4274893734 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57901a5f30 con 0x7f5790078790 2024-09-10T09:37:45.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:45.513+0000 7f578f400640 1 --2- 172.21.15.5:0/4274893734 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5790078790 0x7f57901a58c0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f578800efd0 tx=0x7f578800c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:37:45.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:45.513+0000 7f5794a00640 1 -- 172.21.15.5:0/4274893734 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5788003b70 con 0x7f5790078790 2024-09-10T09:37:45.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:45.513+0000 7f5794a00640 1 -- 172.21.15.5:0/4274893734 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5788003d10 con 0x7f5790078790 2024-09-10T09:37:45.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:45.513+0000 7f578fe00640 1 -- 172.21.15.5:0/4274893734 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57901a6220 con 0x7f5790078790 2024-09-10T09:37:45.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:45.513+0000 7f5794a00640 1 -- 172.21.15.5:0/4274893734 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57880109f0 con 0x7f5790078790 2024-09-10T09:37:45.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:45.514+0000 7f578fe00640 1 -- 172.21.15.5:0/4274893734 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57901a6710 con 0x7f5790078790 2024-09-10T09:37:45.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:45.515+0000 7f578fe00640 1 -- 172.21.15.5:0/4274893734 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f57901105d0 con 0x7f5790078790 2024-09-10T09:37:45.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:45.516+0000 7f5794a00640 1 -- 172.21.15.5:0/4274893734 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f5788003fe0 con 0x7f5790078790 2024-09-10T09:37:45.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:45.518+0000 7f5794a00640 1 --2- 172.21.15.5:0/4274893734 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f57740a2bb0 0x7f57740a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:37:45.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:45.518+0000 7f5794a00640 1 -- 172.21.15.5:0/4274893734 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f57880cf870 con 0x7f5790078790 2024-09-10T09:37:45.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:45.518+0000 7f578ea00640 1 --2- 172.21.15.5:0/4274893734 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f57740a2bb0 0x7f57740a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:37:45.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:45.519+0000 7f578ea00640 1 --2- 172.21.15.5:0/4274893734 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f57740a2bb0 0x7f57740a5070 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f5780031440 tx=0x7f57800315b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:37:45.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:45.522+0000 7f5794a00640 1 -- 172.21.15.5:0/4274893734 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f578809c380 con 0x7f5790078790 2024-09-10T09:37:45.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:45.716+0000 7f578fe00640 1 -- 172.21.15.5:0/4274893734 --> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) -- 0x7f5790062630 con 0x7f57740a2bb0 2024-09-10T09:37:45.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:45.723+0000 7f5794a00640 1 -- 172.21.15.5:0/4274893734 <== mgr.14150 v2:172.21.15.5:6800/2707348081 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+980 (secure 0 0 0) 0x7f5790062630 con 0x7f57740a2bb0 2024-09-10T09:37:45.723 INFO:teuthology.orchestra.run.smithi005.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-09-10T09:37:45.723 INFO:teuthology.orchestra.run.smithi005.stdout:mds.all 12/12 15s ago 3m smithi005=a;smithi005=d;smithi005=g;smithi005=j;smithi102=b;smithi102=e;smithi102=h;smithi102=k;smithi159=c;smithi159=f;smithi159=i;smithi159=l;count:12 2024-09-10T09:37:45.724 INFO:teuthology.orchestra.run.smithi005.stdout:mgr 3/3 15s ago 8m smithi005=x;smithi102=y;smithi159=z;count:3 2024-09-10T09:37:45.724 INFO:teuthology.orchestra.run.smithi005.stdout:mon 3/3 15s ago 9m smithi005:172.21.15.5=a;smithi102:172.21.15.102=b;smithi159:172.21.15.159=c;count:3 2024-09-10T09:37:45.724 INFO:teuthology.orchestra.run.smithi005.stdout:osd 12 15s ago - 2024-09-10T09:37:45.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:45.725+0000 7f578fe00640 1 -- 172.21.15.5:0/4274893734 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f57740a2bb0 msgr2=0x7f57740a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:37:45.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:45.725+0000 7f578fe00640 1 --2- 172.21.15.5:0/4274893734 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f57740a2bb0 0x7f57740a5070 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f5780031440 tx=0x7f57800315b0 comp rx=0 tx=0).stop 2024-09-10T09:37:45.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:45.725+0000 7f578fe00640 1 -- 172.21.15.5:0/4274893734 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5790078790 msgr2=0x7f57901a58c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:37:45.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:45.725+0000 7f578fe00640 1 --2- 172.21.15.5:0/4274893734 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5790078790 0x7f57901a58c0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f578800efd0 tx=0x7f578800c6a0 comp rx=0 tx=0).stop 2024-09-10T09:37:45.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:45.725+0000 7f578fe00640 1 -- 172.21.15.5:0/4274893734 shutdown_connections 2024-09-10T09:37:45.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:45.725+0000 7f578fe00640 1 --2- 172.21.15.5:0/4274893734 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f57740a2bb0 0x7f57740a5070 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:45.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:45.725+0000 7f578fe00640 1 --2- 172.21.15.5:0/4274893734 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5790078790 0x7f57901a58c0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:45.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:45.725+0000 7f578fe00640 1 --2- 172.21.15.5:0/4274893734 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5790077df0 0x7f579019e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:45.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:45.725+0000 7f578fe00640 1 --2- 172.21.15.5:0/4274893734 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5790076ba0 0x7f579019e300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:45.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:45.725+0000 7f578fe00640 1 -- 172.21.15.5:0/4274893734 >> 172.21.15.5:0/4274893734 conn(0x7f57900ff3e0 msgr2=0x7f5790100e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:37:45.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:45.725+0000 7f578fe00640 1 -- 172.21.15.5:0/4274893734 shutdown_connections 2024-09-10T09:37:45.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:45.725+0000 7f578fe00640 1 -- 172.21.15.5:0/4274893734 wait complete. 2024-09-10T09:37:46.604 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- bash -c 'ceph orch host ls' 2024-09-10T09:37:46.984 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:37:47.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:37:46 smithi005 ceph-mon[30350]: pgmap v376: 129 pgs: 129 active+clean; 646 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 597 B/s wr, 12 op/s 2024-09-10T09:37:47.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:37:46 smithi005 ceph-mon[30350]: from='client.15054 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T09:37:47.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:37:46 smithi159 ceph-mon[35513]: pgmap v376: 129 pgs: 129 active+clean; 646 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 597 B/s wr, 12 op/s 2024-09-10T09:37:47.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:37:46 smithi159 ceph-mon[35513]: from='client.15054 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T09:37:47.136 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:37:46 smithi102 ceph-mon[35442]: pgmap v376: 129 pgs: 129 active+clean; 646 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 597 B/s wr, 12 op/s 2024-09-10T09:37:47.136 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:37:46 smithi102 ceph-mon[35442]: from='client.15054 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T09:37:47.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:47.562+0000 7f7602000640 1 -- 172.21.15.5:0/403183860 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f75fc103c50 msgr2=0x7f75fc104050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:37:47.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:47.562+0000 7f7602000640 1 --2- 172.21.15.5:0/403183860 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f75fc103c50 0x7f75fc104050 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f75f0009900 tx=0x7f75f002a410 comp rx=0 tx=0).stop 2024-09-10T09:37:47.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:47.562+0000 7f7602000640 1 -- 172.21.15.5:0/403183860 shutdown_connections 2024-09-10T09:37:47.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:47.562+0000 7f7602000640 1 --2- 172.21.15.5:0/403183860 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f75fc105810 0x7f75fc10c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:47.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:47.562+0000 7f7602000640 1 --2- 172.21.15.5:0/403183860 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f75fc104e50 0x7f75fc1052d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:47.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:47.562+0000 7f7602000640 1 --2- 172.21.15.5:0/403183860 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f75fc103c50 0x7f75fc104050 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:47.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:47.562+0000 7f7602000640 1 -- 172.21.15.5:0/403183860 >> 172.21.15.5:0/403183860 conn(0x7f75fc0ff3e0 msgr2=0x7f75fc101820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:37:47.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:47.563+0000 7f7602000640 1 -- 172.21.15.5:0/403183860 shutdown_connections 2024-09-10T09:37:47.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:47.563+0000 7f7602000640 1 -- 172.21.15.5:0/403183860 wait complete. 2024-09-10T09:37:47.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:47.564+0000 7f7602000640 1 Processor -- start 2024-09-10T09:37:47.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:47.564+0000 7f7602000640 1 -- start start 2024-09-10T09:37:47.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:47.564+0000 7f7602000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f75fc103c50 0x7f75fc199f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:37:47.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:47.564+0000 7f7602000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f75fc104e50 0x7f75fc19a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:37:47.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:47.564+0000 7f7602000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f75fc105810 0x7f75fc1a14c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:37:47.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:47.564+0000 7f7602000640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f75fc10f8e0 con 0x7f75fc104e50 2024-09-10T09:37:47.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:47.564+0000 7f7602000640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f75fc10f760 con 0x7f75fc105810 2024-09-10T09:37:47.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:47.564+0000 7f7602000640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f75fc10fa60 con 0x7f75fc103c50 2024-09-10T09:37:47.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:47.564+0000 7f75fbe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f75fc104e50 0x7f75fc19a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:37:47.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:47.564+0000 7f75fbe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f75fc104e50 0x7f75fc19a440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43586/0 (socket says 172.21.15.5:43586) 2024-09-10T09:37:47.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:47.564+0000 7f75fbe00640 1 -- 172.21.15.5:0/1371574763 learned_addr learned my addr 172.21.15.5:0/1371574763 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:37:47.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:47.564+0000 7f7601600640 1 --2- 172.21.15.5:0/1371574763 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f75fc105810 0x7f75fc1a14c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:37:47.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:47.564+0000 7f7600c00640 1 --2- 172.21.15.5:0/1371574763 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f75fc103c50 0x7f75fc199f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:37:47.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:47.565+0000 7f75fbe00640 1 -- 172.21.15.5:0/1371574763 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f75fc103c50 msgr2=0x7f75fc199f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:37:47.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:47.565+0000 7f75fbe00640 1 --2- 172.21.15.5:0/1371574763 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f75fc103c50 0x7f75fc199f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:47.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:47.565+0000 7f75fbe00640 1 -- 172.21.15.5:0/1371574763 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f75fc105810 msgr2=0x7f75fc1a14c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:37:47.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:47.565+0000 7f75fbe00640 1 --2- 172.21.15.5:0/1371574763 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f75fc105810 0x7f75fc1a14c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:47.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:47.565+0000 7f75fbe00640 1 -- 172.21.15.5:0/1371574763 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75fc1a1b30 con 0x7f75fc104e50 2024-09-10T09:37:47.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:47.565+0000 7f75fbe00640 1 --2- 172.21.15.5:0/1371574763 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f75fc104e50 0x7f75fc19a440 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f75e800e9f0 tx=0x7f75e800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:37:47.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:47.565+0000 7f75f9600640 1 -- 172.21.15.5:0/1371574763 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75e800cd80 con 0x7f75fc104e50 2024-09-10T09:37:47.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:47.565+0000 7f75f9600640 1 -- 172.21.15.5:0/1371574763 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f75e8010910 con 0x7f75fc104e50 2024-09-10T09:37:47.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:47.565+0000 7f7602000640 1 -- 172.21.15.5:0/1371574763 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75fc1a1e20 con 0x7f75fc104e50 2024-09-10T09:37:47.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:47.565+0000 7f75f9600640 1 -- 172.21.15.5:0/1371574763 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75e8018990 con 0x7f75fc104e50 2024-09-10T09:37:47.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:47.566+0000 7f7602000640 1 -- 172.21.15.5:0/1371574763 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75fc1a2320 con 0x7f75fc104e50 2024-09-10T09:37:47.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:47.567+0000 7f7602000640 1 -- 172.21.15.5:0/1371574763 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f75fc10b0a0 con 0x7f75fc104e50 2024-09-10T09:37:47.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:47.568+0000 7f75f9600640 1 -- 172.21.15.5:0/1371574763 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f75e8010ac0 con 0x7f75fc104e50 2024-09-10T09:37:47.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:47.570+0000 7f75f9600640 1 --2- 172.21.15.5:0/1371574763 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f75e40a2c30 0x7f75e40a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:37:47.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:47.570+0000 7f75f9600640 1 -- 172.21.15.5:0/1371574763 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f75e80cf740 con 0x7f75fc104e50 2024-09-10T09:37:47.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:47.570+0000 7f7600c00640 1 --2- 172.21.15.5:0/1371574763 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f75e40a2c30 0x7f75e40a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:37:47.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:47.571+0000 7f7600c00640 1 --2- 172.21.15.5:0/1371574763 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f75e40a2c30 0x7f75e40a50f0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f75f0006010 tx=0x7f75f0000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:37:47.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:47.574+0000 7f75f9600640 1 -- 172.21.15.5:0/1371574763 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f75e809c250 con 0x7f75fc104e50 2024-09-10T09:37:47.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:47.768+0000 7f7602000640 1 -- 172.21.15.5:0/1371574763 --> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) -- 0x7f75fc062630 con 0x7f75e40a2c30 2024-09-10T09:37:47.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:47.769+0000 7f75f9600640 1 -- 172.21.15.5:0/1371574763 <== mgr.14150 v2:172.21.15.5:6800/2707348081 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+191 (secure 0 0 0) 0x7f75fc062630 con 0x7f75e40a2c30 2024-09-10T09:37:47.770 INFO:teuthology.orchestra.run.smithi005.stdout:HOST ADDR LABELS STATUS 2024-09-10T09:37:47.770 INFO:teuthology.orchestra.run.smithi005.stdout:smithi005 172.21.15.5 2024-09-10T09:37:47.770 INFO:teuthology.orchestra.run.smithi005.stdout:smithi102 172.21.15.102 2024-09-10T09:37:47.770 INFO:teuthology.orchestra.run.smithi005.stdout:smithi159 172.21.15.159 2024-09-10T09:37:47.770 INFO:teuthology.orchestra.run.smithi005.stdout:3 hosts in cluster 2024-09-10T09:37:47.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:47.771+0000 7f7602000640 1 -- 172.21.15.5:0/1371574763 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f75e40a2c30 msgr2=0x7f75e40a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:37:47.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:47.771+0000 7f7602000640 1 --2- 172.21.15.5:0/1371574763 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f75e40a2c30 0x7f75e40a50f0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f75f0006010 tx=0x7f75f0000bc0 comp rx=0 tx=0).stop 2024-09-10T09:37:47.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:47.771+0000 7f7602000640 1 -- 172.21.15.5:0/1371574763 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f75fc104e50 msgr2=0x7f75fc19a440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:37:47.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:47.771+0000 7f7602000640 1 --2- 172.21.15.5:0/1371574763 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f75fc104e50 0x7f75fc19a440 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f75e800e9f0 tx=0x7f75e800eeb0 comp rx=0 tx=0).stop 2024-09-10T09:37:47.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:47.772+0000 7f7602000640 1 -- 172.21.15.5:0/1371574763 shutdown_connections 2024-09-10T09:37:47.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:47.772+0000 7f7602000640 1 --2- 172.21.15.5:0/1371574763 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f75e40a2c30 0x7f75e40a50f0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:47.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:47.772+0000 7f7602000640 1 --2- 172.21.15.5:0/1371574763 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f75fc105810 0x7f75fc1a14c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:47.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:47.772+0000 7f7602000640 1 --2- 172.21.15.5:0/1371574763 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f75fc104e50 0x7f75fc19a440 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:47.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:47.772+0000 7f7602000640 1 --2- 172.21.15.5:0/1371574763 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f75fc103c50 0x7f75fc199f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:47.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:47.772+0000 7f7602000640 1 -- 172.21.15.5:0/1371574763 >> 172.21.15.5:0/1371574763 conn(0x7f75fc0ff3e0 msgr2=0x7f75fc108270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:37:47.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:47.772+0000 7f7602000640 1 -- 172.21.15.5:0/1371574763 shutdown_connections 2024-09-10T09:37:47.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:47.772+0000 7f7602000640 1 -- 172.21.15.5:0/1371574763 wait complete. 2024-09-10T09:37:48.574 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- bash -c 'ceph orch device ls' 2024-09-10T09:37:48.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:37:48 smithi005 ceph-mon[30350]: pgmap v377: 129 pgs: 129 active+clean; 646 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 597 B/s wr, 9 op/s 2024-09-10T09:37:48.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:37:48 smithi005 ceph-mon[30350]: from='client.15060 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T09:37:48.953 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:37:49.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:37:48 smithi159 ceph-mon[35513]: pgmap v377: 129 pgs: 129 active+clean; 646 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 597 B/s wr, 9 op/s 2024-09-10T09:37:49.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:37:48 smithi159 ceph-mon[35513]: from='client.15060 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T09:37:49.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:37:48 smithi102 ceph-mon[35442]: pgmap v377: 129 pgs: 129 active+clean; 646 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 597 B/s wr, 9 op/s 2024-09-10T09:37:49.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:37:48 smithi102 ceph-mon[35442]: from='client.15060 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T09:37:49.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:49.527+0000 7f7a14600640 1 -- 172.21.15.5:0/3753299117 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7a0c074c50 msgr2=0x7f7a0c0750b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:37:49.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:49.527+0000 7f7a14600640 1 --2- 172.21.15.5:0/3753299117 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7a0c074c50 0x7f7a0c0750b0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f79f8009860 tx=0x7f79f802a340 comp rx=0 tx=0).stop 2024-09-10T09:37:49.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:49.527+0000 7f7a14600640 1 -- 172.21.15.5:0/3753299117 shutdown_connections 2024-09-10T09:37:49.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:49.527+0000 7f7a14600640 1 --2- 172.21.15.5:0/3753299117 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7a0c10e1e0 0x7f7a0c1105d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:49.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:49.527+0000 7f7a14600640 1 --2- 172.21.15.5:0/3753299117 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7a0c074c50 0x7f7a0c0750b0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:49.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:49.527+0000 7f7a14600640 1 --2- 172.21.15.5:0/3753299117 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7a0c076220 0x7f7a0c074680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:49.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:49.527+0000 7f7a14600640 1 -- 172.21.15.5:0/3753299117 >> 172.21.15.5:0/3753299117 conn(0x7f7a0c0fd390 msgr2=0x7f7a0c0ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:37:49.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:49.527+0000 7f7a14600640 1 -- 172.21.15.5:0/3753299117 shutdown_connections 2024-09-10T09:37:49.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:49.527+0000 7f7a14600640 1 -- 172.21.15.5:0/3753299117 wait complete. 2024-09-10T09:37:49.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:49.528+0000 7f7a14600640 1 Processor -- start 2024-09-10T09:37:49.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:49.528+0000 7f7a14600640 1 -- start start 2024-09-10T09:37:49.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:49.529+0000 7f7a14600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7a0c074c50 0x7f7a0c19e560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:37:49.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:49.529+0000 7f7a14600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7a0c076220 0x7f7a0c19eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:37:49.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:49.529+0000 7f7a14600640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7a0c10e1e0 0x7f7a0c1a5b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:37:49.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:49.529+0000 7f7a14600640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f7a0c113ea0 con 0x7f7a0c076220 2024-09-10T09:37:49.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:49.529+0000 7f7a14600640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f7a0c113d20 con 0x7f7a0c074c50 2024-09-10T09:37:49.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:49.529+0000 7f7a14600640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f7a0c114020 con 0x7f7a0c10e1e0 2024-09-10T09:37:49.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:49.529+0000 7f7a11200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7a0c076220 0x7f7a0c19eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:37:49.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:49.529+0000 7f7a11200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7a0c076220 0x7f7a0c19eaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43602/0 (socket says 172.21.15.5:43602) 2024-09-10T09:37:49.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:49.529+0000 7f7a11200640 1 -- 172.21.15.5:0/2756209729 learned_addr learned my addr 172.21.15.5:0/2756209729 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:37:49.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:49.529+0000 7f7a11c00640 1 --2- 172.21.15.5:0/2756209729 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7a0c074c50 0x7f7a0c19e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:37:49.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:49.529+0000 7f7a12600640 1 --2- 172.21.15.5:0/2756209729 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7a0c10e1e0 0x7f7a0c1a5b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:37:49.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:49.529+0000 7f7a11200640 1 -- 172.21.15.5:0/2756209729 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7a0c10e1e0 msgr2=0x7f7a0c1a5b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:37:49.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:49.529+0000 7f7a11200640 1 --2- 172.21.15.5:0/2756209729 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7a0c10e1e0 0x7f7a0c1a5b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:49.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:49.529+0000 7f7a11200640 1 -- 172.21.15.5:0/2756209729 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7a0c074c50 msgr2=0x7f7a0c19e560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:37:49.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:49.530+0000 7f7a11200640 1 --2- 172.21.15.5:0/2756209729 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7a0c074c50 0x7f7a0c19e560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:49.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:49.530+0000 7f7a11200640 1 -- 172.21.15.5:0/2756209729 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a0c1a6190 con 0x7f7a0c076220 2024-09-10T09:37:49.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:49.530+0000 7f7a11200640 1 --2- 172.21.15.5:0/2756209729 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7a0c076220 0x7f7a0c19eaa0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f79f8035040 tx=0x7f79f8033d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:37:49.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:49.530+0000 7f7a0a000640 1 -- 172.21.15.5:0/2756209729 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79f8037040 con 0x7f7a0c076220 2024-09-10T09:37:49.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:49.530+0000 7f7a0a000640 1 -- 172.21.15.5:0/2756209729 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f79f8032770 con 0x7f7a0c076220 2024-09-10T09:37:49.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:49.530+0000 7f7a14600640 1 -- 172.21.15.5:0/2756209729 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a0c1a6420 con 0x7f7a0c076220 2024-09-10T09:37:49.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:49.530+0000 7f7a0a000640 1 -- 172.21.15.5:0/2756209729 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79f80046c0 con 0x7f7a0c076220 2024-09-10T09:37:49.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:49.530+0000 7f7a14600640 1 -- 172.21.15.5:0/2756209729 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a0c1a6900 con 0x7f7a0c076220 2024-09-10T09:37:49.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:49.532+0000 7f7a14600640 1 -- 172.21.15.5:0/2756209729 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7a0c1043b0 con 0x7f7a0c076220 2024-09-10T09:37:49.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:49.534+0000 7f7a0a000640 1 -- 172.21.15.5:0/2756209729 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f79f8004050 con 0x7f7a0c076220 2024-09-10T09:37:49.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:49.535+0000 7f7a0a000640 1 --2- 172.21.15.5:0/2756209729 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f79f40a2bb0 0x7f79f40a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:37:49.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:49.535+0000 7f7a0a000640 1 -- 172.21.15.5:0/2756209729 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f79f80ef2b0 con 0x7f7a0c076220 2024-09-10T09:37:49.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:49.535+0000 7f7a11c00640 1 --2- 172.21.15.5:0/2756209729 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f79f40a2bb0 0x7f79f40a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:37:49.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:49.536+0000 7f7a11c00640 1 --2- 172.21.15.5:0/2756209729 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f79f40a2bb0 0x7f79f40a5070 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f7a00009530 tx=0x7f7a00009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:37:49.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:49.540+0000 7f7a0a000640 1 -- 172.21.15.5:0/2756209729 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f79f80bbe20 con 0x7f7a0c076220 2024-09-10T09:37:49.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:49.749+0000 7f7a14600640 1 -- 172.21.15.5:0/2756209729 --> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) -- 0x7f7a0c062630 con 0x7f79f40a2bb0 2024-09-10T09:37:49.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:49.751+0000 7f7a0a000640 1 -- 172.21.15.5:0/2756209729 <== mgr.14150 v2:172.21.15.5:6800/2707348081 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+532 (secure 0 0 0) 0x7f7a0c062630 con 0x7f79f40a2bb0 2024-09-10T09:37:49.752 INFO:teuthology.orchestra.run.smithi005.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-09-10T09:37:49.752 INFO:teuthology.orchestra.run.smithi005.stdout:smithi005 /dev/nvme0n1 ssd INTEL_SSDPED1D480GA_PHMB751300EW480DGN 447G No 4m ago Has a FileSystem, LVM detected 2024-09-10T09:37:49.752 INFO:teuthology.orchestra.run.smithi005.stdout:smithi102 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620400VK400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-09-10T09:37:49.752 INFO:teuthology.orchestra.run.smithi005.stdout:smithi159 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT623300GH400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-09-10T09:37:49.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:49.753+0000 7f7a14600640 1 -- 172.21.15.5:0/2756209729 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f79f40a2bb0 msgr2=0x7f79f40a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:37:49.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:49.753+0000 7f7a14600640 1 --2- 172.21.15.5:0/2756209729 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f79f40a2bb0 0x7f79f40a5070 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f7a00009530 tx=0x7f7a00009260 comp rx=0 tx=0).stop 2024-09-10T09:37:49.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:49.753+0000 7f7a14600640 1 -- 172.21.15.5:0/2756209729 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7a0c076220 msgr2=0x7f7a0c19eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:37:49.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:49.753+0000 7f7a14600640 1 --2- 172.21.15.5:0/2756209729 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7a0c076220 0x7f7a0c19eaa0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f79f8035040 tx=0x7f79f8033d90 comp rx=0 tx=0).stop 2024-09-10T09:37:49.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:49.753+0000 7f7a14600640 1 -- 172.21.15.5:0/2756209729 shutdown_connections 2024-09-10T09:37:49.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:49.753+0000 7f7a14600640 1 --2- 172.21.15.5:0/2756209729 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f79f40a2bb0 0x7f79f40a5070 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:49.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:49.753+0000 7f7a14600640 1 --2- 172.21.15.5:0/2756209729 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7a0c10e1e0 0x7f7a0c1a5b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:49.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:49.753+0000 7f7a14600640 1 --2- 172.21.15.5:0/2756209729 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7a0c076220 0x7f7a0c19eaa0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:49.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:49.753+0000 7f7a14600640 1 --2- 172.21.15.5:0/2756209729 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7a0c074c50 0x7f7a0c19e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:49.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:49.753+0000 7f7a14600640 1 -- 172.21.15.5:0/2756209729 >> 172.21.15.5:0/2756209729 conn(0x7f7a0c0fd390 msgr2=0x7f7a0c101580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:37:49.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:49.753+0000 7f7a14600640 1 -- 172.21.15.5:0/2756209729 shutdown_connections 2024-09-10T09:37:49.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:49.754+0000 7f7a14600640 1 -- 172.21.15.5:0/2756209729 wait complete. 2024-09-10T09:37:50.550 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-09-10T09:37:50.559 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi005.front.sepia.ceph.com 2024-09-10T09:37:50.559 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- bash -c 'ceph fs dump' 2024-09-10T09:37:50.941 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:37:51.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:37:50 smithi005 ceph-mon[30350]: pgmap v378: 129 pgs: 129 active+clean; 646 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 597 B/s wr, 9 op/s 2024-09-10T09:37:51.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:37:50 smithi005 ceph-mon[30350]: from='client.15066 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T09:37:51.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:37:50 smithi159 ceph-mon[35513]: pgmap v378: 129 pgs: 129 active+clean; 646 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 597 B/s wr, 9 op/s 2024-09-10T09:37:51.101 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:37:50 smithi159 ceph-mon[35513]: from='client.15066 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T09:37:51.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:37:50 smithi102 ceph-mon[35442]: pgmap v378: 129 pgs: 129 active+clean; 646 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 597 B/s wr, 9 op/s 2024-09-10T09:37:51.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:37:50 smithi102 ceph-mon[35442]: from='client.15066 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T09:37:51.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:51.520+0000 7f8a61400640 1 -- 172.21.15.5:0/1417868031 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8a5c076ba0 msgr2=0x7f8a5c076fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:37:51.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:51.520+0000 7f8a61400640 1 --2- 172.21.15.5:0/1417868031 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8a5c076ba0 0x7f8a5c076fa0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f8a4c009900 tx=0x7f8a4c02a3f0 comp rx=0 tx=0).stop 2024-09-10T09:37:51.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:51.520+0000 7f8a61400640 1 -- 172.21.15.5:0/1417868031 shutdown_connections 2024-09-10T09:37:51.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:51.521+0000 7f8a61400640 1 --2- 172.21.15.5:0/1417868031 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8a5c078790 0x7f8a5c079030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:51.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:51.521+0000 7f8a61400640 1 --2- 172.21.15.5:0/1417868031 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8a5c077df0 0x7f8a5c078250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:51.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:51.521+0000 7f8a61400640 1 --2- 172.21.15.5:0/1417868031 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8a5c076ba0 0x7f8a5c076fa0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:51.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:51.521+0000 7f8a61400640 1 -- 172.21.15.5:0/1417868031 >> 172.21.15.5:0/1417868031 conn(0x7f8a5c0ff5b0 msgr2=0x7f8a5c1019d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:37:51.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:51.521+0000 7f8a61400640 1 -- 172.21.15.5:0/1417868031 shutdown_connections 2024-09-10T09:37:51.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:51.521+0000 7f8a61400640 1 -- 172.21.15.5:0/1417868031 wait complete. 2024-09-10T09:37:51.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:51.522+0000 7f8a61400640 1 Processor -- start 2024-09-10T09:37:51.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:51.522+0000 7f8a61400640 1 -- start start 2024-09-10T09:37:51.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:51.522+0000 7f8a61400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8a5c076ba0 0x7f8a5c19e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:37:51.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:51.522+0000 7f8a61400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8a5c077df0 0x7f8a5c19ea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:37:51.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:51.522+0000 7f8a61400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8a5c078790 0x7f8a5c1a5a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:37:51.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:51.522+0000 7f8a61400640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f8a5c113f10 con 0x7f8a5c078790 2024-09-10T09:37:51.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:51.522+0000 7f8a61400640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f8a5c113d90 con 0x7f8a5c077df0 2024-09-10T09:37:51.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:51.522+0000 7f8a61400640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f8a5c114090 con 0x7f8a5c076ba0 2024-09-10T09:37:51.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:51.522+0000 7f8a5b200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8a5c078790 0x7f8a5c1a5a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:37:51.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:51.523+0000 7f8a5b200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8a5c078790 0x7f8a5c1a5a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38584/0 (socket says 172.21.15.5:38584) 2024-09-10T09:37:51.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:51.523+0000 7f8a5b200640 1 -- 172.21.15.5:0/1226048188 learned_addr learned my addr 172.21.15.5:0/1226048188 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:37:51.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:51.523+0000 7f8a53e00640 1 --2- 172.21.15.5:0/1226048188 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8a5c077df0 0x7f8a5c19ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:37:51.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:51.523+0000 7f8a5a800640 1 --2- 172.21.15.5:0/1226048188 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8a5c076ba0 0x7f8a5c19e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:37:51.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:51.523+0000 7f8a5b200640 1 -- 172.21.15.5:0/1226048188 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8a5c076ba0 msgr2=0x7f8a5c19e4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:37:51.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:51.523+0000 7f8a5b200640 1 --2- 172.21.15.5:0/1226048188 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8a5c076ba0 0x7f8a5c19e4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:51.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:51.523+0000 7f8a5b200640 1 -- 172.21.15.5:0/1226048188 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8a5c077df0 msgr2=0x7f8a5c19ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:37:51.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:51.523+0000 7f8a5b200640 1 --2- 172.21.15.5:0/1226048188 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8a5c077df0 0x7f8a5c19ea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:51.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:51.523+0000 7f8a5b200640 1 -- 172.21.15.5:0/1226048188 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a5c1a6100 con 0x7f8a5c078790 2024-09-10T09:37:51.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:51.523+0000 7f8a53e00640 1 --2- 172.21.15.5:0/1226048188 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8a5c077df0 0x7f8a5c19ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:37:51.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:51.523+0000 7f8a5b200640 1 --2- 172.21.15.5:0/1226048188 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8a5c078790 0x7f8a5c1a5a90 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f8a5400eb10 tx=0x7f8a5400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:37:51.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:51.524+0000 7f8a53400640 1 -- 172.21.15.5:0/1226048188 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a54004b20 con 0x7f8a5c078790 2024-09-10T09:37:51.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:51.524+0000 7f8a53400640 1 -- 172.21.15.5:0/1226048188 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8a54004cc0 con 0x7f8a5c078790 2024-09-10T09:37:51.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:51.524+0000 7f8a61400640 1 -- 172.21.15.5:0/1226048188 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a5c1a63f0 con 0x7f8a5c078790 2024-09-10T09:37:51.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:51.524+0000 7f8a53400640 1 -- 172.21.15.5:0/1226048188 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a54010760 con 0x7f8a5c078790 2024-09-10T09:37:51.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:51.524+0000 7f8a61400640 1 -- 172.21.15.5:0/1226048188 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a5c1a68e0 con 0x7f8a5c078790 2024-09-10T09:37:51.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:51.525+0000 7f8a61400640 1 -- 172.21.15.5:0/1226048188 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a5c110780 con 0x7f8a5c078790 2024-09-10T09:37:51.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:51.527+0000 7f8a53400640 1 -- 172.21.15.5:0/1226048188 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f8a54005020 con 0x7f8a5c078790 2024-09-10T09:37:51.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:51.528+0000 7f8a53400640 1 --2- 172.21.15.5:0/1226048188 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f8a400a2bb0 0x7f8a400a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:37:51.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:51.528+0000 7f8a53400640 1 -- 172.21.15.5:0/1226048188 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f8a540cfa10 con 0x7f8a5c078790 2024-09-10T09:37:51.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:51.528+0000 7f8a5a800640 1 --2- 172.21.15.5:0/1226048188 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f8a400a2bb0 0x7f8a400a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:37:51.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:51.529+0000 7f8a5a800640 1 --2- 172.21.15.5:0/1226048188 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f8a400a2bb0 0x7f8a400a5070 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f8a4c031440 tx=0x7f8a4c0315b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:37:51.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:51.532+0000 7f8a53400640 1 -- 172.21.15.5:0/1226048188 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f8a5409c520 con 0x7f8a5c078790 2024-09-10T09:37:51.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:51.753+0000 7f8a61400640 1 -- 172.21.15.5:0/1226048188 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7f8a5c062df0 con 0x7f8a5c078790 2024-09-10T09:37:51.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:51.754+0000 7f8a53400640 1 -- 172.21.15.5:0/1226048188 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 29 v29) ==== 76+0+3126 (secure 0 0 0) 0x7f8a5409be20 con 0x7f8a5c078790 2024-09-10T09:37:51.755 INFO:teuthology.orchestra.run.smithi005.stdout:e29 2024-09-10T09:37:51.755 INFO:teuthology.orchestra.run.smithi005.stdout:btime 2024-09-10T09:36:16:492171+0000 2024-09-10T09:37:51.755 INFO:teuthology.orchestra.run.smithi005.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-10T09:37:51.756 INFO:teuthology.orchestra.run.smithi005.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-10T09:37:51.756 INFO:teuthology.orchestra.run.smithi005.stdout:legacy client fscid: 1 2024-09-10T09:37:51.756 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-10T09:37:51.756 INFO:teuthology.orchestra.run.smithi005.stdout:Filesystem 'cephfs' (1) 2024-09-10T09:37:51.756 INFO:teuthology.orchestra.run.smithi005.stdout:fs_name cephfs 2024-09-10T09:37:51.756 INFO:teuthology.orchestra.run.smithi005.stdout:epoch 29 2024-09-10T09:37:51.756 INFO:teuthology.orchestra.run.smithi005.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-09-10T09:37:51.756 INFO:teuthology.orchestra.run.smithi005.stdout:created 2024-09-10T09:35:29.626287+0000 2024-09-10T09:37:51.756 INFO:teuthology.orchestra.run.smithi005.stdout:modified 2024-09-10T09:36:15.764196+0000 2024-09-10T09:37:51.756 INFO:teuthology.orchestra.run.smithi005.stdout:tableserver 0 2024-09-10T09:37:51.756 INFO:teuthology.orchestra.run.smithi005.stdout:root 0 2024-09-10T09:37:51.756 INFO:teuthology.orchestra.run.smithi005.stdout:session_timeout 300 2024-09-10T09:37:51.756 INFO:teuthology.orchestra.run.smithi005.stdout:session_autoclose 300 2024-09-10T09:37:51.757 INFO:teuthology.orchestra.run.smithi005.stdout:max_file_size 1099511627776 2024-09-10T09:37:51.757 INFO:teuthology.orchestra.run.smithi005.stdout:max_xattr_size 65536 2024-09-10T09:37:51.757 INFO:teuthology.orchestra.run.smithi005.stdout:required_client_features {} 2024-09-10T09:37:51.757 INFO:teuthology.orchestra.run.smithi005.stdout:last_failure 0 2024-09-10T09:37:51.757 INFO:teuthology.orchestra.run.smithi005.stdout:last_failure_osd_epoch 0 2024-09-10T09:37:51.757 INFO:teuthology.orchestra.run.smithi005.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-10T09:37:51.757 INFO:teuthology.orchestra.run.smithi005.stdout:max_mds 5 2024-09-10T09:37:51.757 INFO:teuthology.orchestra.run.smithi005.stdout:in 0,1,2,3,4 2024-09-10T09:37:51.757 INFO:teuthology.orchestra.run.smithi005.stdout:up {0=24542,1=24535,2=24511,3=24506,4=24443} 2024-09-10T09:37:51.757 INFO:teuthology.orchestra.run.smithi005.stdout:failed 2024-09-10T09:37:51.757 INFO:teuthology.orchestra.run.smithi005.stdout:damaged 2024-09-10T09:37:51.757 INFO:teuthology.orchestra.run.smithi005.stdout:stopped 2024-09-10T09:37:51.757 INFO:teuthology.orchestra.run.smithi005.stdout:data_pools [3,4] 2024-09-10T09:37:51.757 INFO:teuthology.orchestra.run.smithi005.stdout:metadata_pool 2 2024-09-10T09:37:51.758 INFO:teuthology.orchestra.run.smithi005.stdout:inline_data disabled 2024-09-10T09:37:51.758 INFO:teuthology.orchestra.run.smithi005.stdout:balancer 2024-09-10T09:37:51.758 INFO:teuthology.orchestra.run.smithi005.stdout:bal_rank_mask -1 2024-09-10T09:37:51.758 INFO:teuthology.orchestra.run.smithi005.stdout:standby_count_wanted 0 2024-09-10T09:37:51.758 INFO:teuthology.orchestra.run.smithi005.stdout:qdb_cluster leader: 24542 members: 24443,24506,24511,24535,24542 2024-09-10T09:37:51.758 INFO:teuthology.orchestra.run.smithi005.stdout:[mds.b{0:24542} state up:active seq 5 addr [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T09:37:51.758 INFO:teuthology.orchestra.run.smithi005.stdout:[mds.f{0:24505} state up:standby-replay seq 1 addr [v2:172.21.15.159:6834/952657242,v1:172.21.15.159:6835/952657242] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T09:37:51.758 INFO:teuthology.orchestra.run.smithi005.stdout:[mds.i{1:24535} state up:active seq 18 addr [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T09:37:51.758 INFO:teuthology.orchestra.run.smithi005.stdout:[mds.e{1:24479} state up:standby-replay seq 1 addr [v2:172.21.15.102:6834/1647549111,v1:172.21.15.102:6835/1647549111] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T09:37:51.758 INFO:teuthology.orchestra.run.smithi005.stdout:[mds.c{2:24511} state up:active seq 19 addr [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T09:37:51.759 INFO:teuthology.orchestra.run.smithi005.stdout:[mds.d{2:24467} state up:standby-replay seq 1 addr [v2:172.21.15.5:6836/2894314507,v1:172.21.15.5:6837/2894314507] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T09:37:51.759 INFO:teuthology.orchestra.run.smithi005.stdout:[mds.h{3:24506} state up:active seq 19 addr [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T09:37:51.759 INFO:teuthology.orchestra.run.smithi005.stdout:[mds.l{3:24460} state up:standby-replay seq 1 addr [v2:172.21.15.159:6832/356445159,v1:172.21.15.159:6833/356445159] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T09:37:51.759 INFO:teuthology.orchestra.run.smithi005.stdout:[mds.k{4:24443} state up:active seq 24 addr [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T09:37:51.759 INFO:teuthology.orchestra.run.smithi005.stdout:[mds.a{4:14520} state up:standby-replay seq 1 addr [v2:172.21.15.5:6840/2073599842,v1:172.21.15.5:6841/2073599842] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T09:37:51.759 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-10T09:37:51.759 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-10T09:37:51.759 INFO:teuthology.orchestra.run.smithi005.stdout:Standby daemons: 2024-09-10T09:37:51.759 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-10T09:37:51.759 INFO:teuthology.orchestra.run.smithi005.stdout:[mds.j{-1:14454} state up:standby seq 1 addr [v2:172.21.15.5:6834/2970594311,v1:172.21.15.5:6835/2970594311] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T09:37:51.759 INFO:teuthology.orchestra.run.smithi005.stdout:[mds.g{-1:14514} state up:standby seq 1 addr [v2:172.21.15.5:6838/3616502174,v1:172.21.15.5:6839/3616502174] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T09:37:51.760 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 29 2024-09-10T09:37:51.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:51.756+0000 7f8a61400640 1 -- 172.21.15.5:0/1226048188 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f8a400a2bb0 msgr2=0x7f8a400a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:37:51.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:51.756+0000 7f8a61400640 1 --2- 172.21.15.5:0/1226048188 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f8a400a2bb0 0x7f8a400a5070 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f8a4c031440 tx=0x7f8a4c0315b0 comp rx=0 tx=0).stop 2024-09-10T09:37:51.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:51.756+0000 7f8a61400640 1 -- 172.21.15.5:0/1226048188 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8a5c078790 msgr2=0x7f8a5c1a5a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:37:51.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:51.756+0000 7f8a61400640 1 --2- 172.21.15.5:0/1226048188 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8a5c078790 0x7f8a5c1a5a90 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f8a5400eb10 tx=0x7f8a5400c2d0 comp rx=0 tx=0).stop 2024-09-10T09:37:51.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:51.756+0000 7f8a61400640 1 -- 172.21.15.5:0/1226048188 shutdown_connections 2024-09-10T09:37:51.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:51.756+0000 7f8a61400640 1 --2- 172.21.15.5:0/1226048188 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f8a400a2bb0 0x7f8a400a5070 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:51.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:51.756+0000 7f8a61400640 1 --2- 172.21.15.5:0/1226048188 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8a5c078790 0x7f8a5c1a5a90 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:51.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:51.756+0000 7f8a61400640 1 --2- 172.21.15.5:0/1226048188 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8a5c077df0 0x7f8a5c19ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:51.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:51.756+0000 7f8a61400640 1 --2- 172.21.15.5:0/1226048188 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8a5c076ba0 0x7f8a5c19e4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:51.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:51.756+0000 7f8a61400640 1 -- 172.21.15.5:0/1226048188 >> 172.21.15.5:0/1226048188 conn(0x7f8a5c0ff5b0 msgr2=0x7f8a5c101040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:37:51.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:51.757+0000 7f8a61400640 1 -- 172.21.15.5:0/1226048188 shutdown_connections 2024-09-10T09:37:51.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:51.757+0000 7f8a61400640 1 -- 172.21.15.5:0/1226048188 wait complete. 2024-09-10T09:37:52.035 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:37:51 smithi102 ceph-mon[35442]: from='client.? 172.21.15.5:0/1226048188' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-10T09:37:52.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:37:51 smithi005 ceph-mon[30350]: from='client.? 172.21.15.5:0/1226048188' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-10T09:37:52.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:37:51 smithi159 ceph-mon[35513]: from='client.? 172.21.15.5:0/1226048188' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-10T09:37:52.616 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- bash -c 'ceph osd dump' 2024-09-10T09:37:53.017 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:37:53.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:37:52 smithi005 ceph-mon[30350]: pgmap v379: 129 pgs: 129 active+clean; 646 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 597 B/s wr, 12 op/s 2024-09-10T09:37:53.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:37:52 smithi159 ceph-mon[35513]: pgmap v379: 129 pgs: 129 active+clean; 646 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 597 B/s wr, 12 op/s 2024-09-10T09:37:53.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:37:52 smithi102 ceph-mon[35442]: pgmap v379: 129 pgs: 129 active+clean; 646 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 597 B/s wr, 12 op/s 2024-09-10T09:37:53.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:53.656+0000 7f7d22e00640 1 -- 172.21.15.5:0/1551689626 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7d1c105920 msgr2=0x7f7d1c10c240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:37:53.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:53.656+0000 7f7d22e00640 1 --2- 172.21.15.5:0/1551689626 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7d1c105920 0x7f7d1c10c240 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f7d14009ca0 tx=0x7f7d1402a8c0 comp rx=0 tx=0).stop 2024-09-10T09:37:53.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:53.656+0000 7f7d22e00640 1 -- 172.21.15.5:0/1551689626 shutdown_connections 2024-09-10T09:37:53.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:53.656+0000 7f7d22e00640 1 --2- 172.21.15.5:0/1551689626 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7d1c105920 0x7f7d1c10c240 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:53.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:53.656+0000 7f7d22e00640 1 --2- 172.21.15.5:0/1551689626 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7d1c104f60 0x7f7d1c1053e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:53.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:53.656+0000 7f7d22e00640 1 --2- 172.21.15.5:0/1551689626 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7d1c103e00 0x7f7d1c104200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:53.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:53.656+0000 7f7d22e00640 1 -- 172.21.15.5:0/1551689626 >> 172.21.15.5:0/1551689626 conn(0x7f7d1c0ff5b0 msgr2=0x7f7d1c1019d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:37:53.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:53.656+0000 7f7d22e00640 1 -- 172.21.15.5:0/1551689626 shutdown_connections 2024-09-10T09:37:53.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:53.656+0000 7f7d22e00640 1 -- 172.21.15.5:0/1551689626 wait complete. 2024-09-10T09:37:53.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:53.657+0000 7f7d22e00640 1 Processor -- start 2024-09-10T09:37:53.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:53.657+0000 7f7d22e00640 1 -- start start 2024-09-10T09:37:53.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:53.657+0000 7f7d22e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7d1c103e00 0x7f7d1c19a070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:37:53.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:53.657+0000 7f7d22e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7d1c104f60 0x7f7d1c19a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:37:53.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:53.658+0000 7f7d22e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7d1c105920 0x7f7d1c1a1630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:37:53.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:53.658+0000 7f7d22e00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f7d1c10fab0 con 0x7f7d1c104f60 2024-09-10T09:37:53.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:53.658+0000 7f7d22e00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f7d1c10f930 con 0x7f7d1c105920 2024-09-10T09:37:53.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:53.658+0000 7f7d22e00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f7d1c10fc30 con 0x7f7d1c103e00 2024-09-10T09:37:53.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:53.658+0000 7f7d13e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7d1c104f60 0x7f7d1c19a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:37:53.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:53.658+0000 7f7d13e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7d1c104f60 0x7f7d1c19a5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38614/0 (socket says 172.21.15.5:38614) 2024-09-10T09:37:53.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:53.658+0000 7f7d13e00640 1 -- 172.21.15.5:0/572924317 learned_addr learned my addr 172.21.15.5:0/572924317 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:37:53.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:53.658+0000 7f7d20e00640 1 --2- 172.21.15.5:0/572924317 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7d1c105920 0x7f7d1c1a1630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:37:53.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:53.658+0000 7f7d1be00640 1 --2- 172.21.15.5:0/572924317 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7d1c103e00 0x7f7d1c19a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:37:53.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:53.658+0000 7f7d13e00640 1 -- 172.21.15.5:0/572924317 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7d1c103e00 msgr2=0x7f7d1c19a070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:37:53.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:53.658+0000 7f7d13e00640 1 --2- 172.21.15.5:0/572924317 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7d1c103e00 0x7f7d1c19a070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:53.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:53.658+0000 7f7d13e00640 1 -- 172.21.15.5:0/572924317 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7d1c105920 msgr2=0x7f7d1c1a1630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:37:53.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:53.658+0000 7f7d13e00640 1 --2- 172.21.15.5:0/572924317 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7d1c105920 0x7f7d1c1a1630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:53.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:53.658+0000 7f7d13e00640 1 -- 172.21.15.5:0/572924317 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d1c1a1ca0 con 0x7f7d1c104f60 2024-09-10T09:37:53.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:53.658+0000 7f7d20e00640 1 --2- 172.21.15.5:0/572924317 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7d1c105920 0x7f7d1c1a1630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:37:53.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:53.659+0000 7f7d13e00640 1 --2- 172.21.15.5:0/572924317 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7d1c104f60 0x7f7d1c19a5b0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f7d0400e9c0 tx=0x7f7d0400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:37:53.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:53.659+0000 7f7d19600640 1 -- 172.21.15.5:0/572924317 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d0400cd80 con 0x7f7d1c104f60 2024-09-10T09:37:53.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:53.659+0000 7f7d19600640 1 -- 172.21.15.5:0/572924317 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7d040035f0 con 0x7f7d1c104f60 2024-09-10T09:37:53.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:53.659+0000 7f7d22e00640 1 -- 172.21.15.5:0/572924317 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d1c1a1f90 con 0x7f7d1c104f60 2024-09-10T09:37:53.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:53.659+0000 7f7d19600640 1 -- 172.21.15.5:0/572924317 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d04010690 con 0x7f7d1c104f60 2024-09-10T09:37:53.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:53.659+0000 7f7d22e00640 1 -- 172.21.15.5:0/572924317 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d1c1a2440 con 0x7f7d1c104f60 2024-09-10T09:37:53.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:53.661+0000 7f7d22e00640 1 -- 172.21.15.5:0/572924317 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d1c10b120 con 0x7f7d1c104f60 2024-09-10T09:37:53.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:53.662+0000 7f7d19600640 1 -- 172.21.15.5:0/572924317 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f7d040037a0 con 0x7f7d1c104f60 2024-09-10T09:37:53.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:53.664+0000 7f7d19600640 1 --2- 172.21.15.5:0/572924317 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f7d000a2c30 0x7f7d000a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:37:53.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:53.664+0000 7f7d19600640 1 -- 172.21.15.5:0/572924317 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f7d040cf720 con 0x7f7d1c104f60 2024-09-10T09:37:53.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:53.664+0000 7f7d1be00640 1 --2- 172.21.15.5:0/572924317 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f7d000a2c30 0x7f7d000a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:37:53.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:53.665+0000 7f7d1be00640 1 --2- 172.21.15.5:0/572924317 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f7d000a2c30 0x7f7d000a50f0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f7d0c009bd0 tx=0x7f7d0c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:37:53.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:53.668+0000 7f7d19600640 1 -- 172.21.15.5:0/572924317 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f7d0409c230 con 0x7f7d1c104f60 2024-09-10T09:37:53.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:53.855+0000 7f7d22e00640 1 -- 172.21.15.5:0/572924317 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump"} v 0) -- 0x7f7d1c02d7c0 con 0x7f7d1c104f60 2024-09-10T09:37:53.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:53.856+0000 7f7d19600640 1 -- 172.21.15.5:0/572924317 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v96) ==== 56+0+5690 (secure 0 0 0) 0x7f7d0409bb30 con 0x7f7d1c104f60 2024-09-10T09:37:53.857 INFO:teuthology.orchestra.run.smithi005.stdout:epoch 96 2024-09-10T09:37:53.857 INFO:teuthology.orchestra.run.smithi005.stdout:fsid bade4b50-6f56-11ef-bcea-c7b262605968 2024-09-10T09:37:53.857 INFO:teuthology.orchestra.run.smithi005.stdout:created 2024-09-10T09:27:13.298498+0000 2024-09-10T09:37:53.857 INFO:teuthology.orchestra.run.smithi005.stdout:modified 2024-09-10T09:37:07.520520+0000 2024-09-10T09:37:53.857 INFO:teuthology.orchestra.run.smithi005.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-09-10T09:37:53.857 INFO:teuthology.orchestra.run.smithi005.stdout:crush_version 27 2024-09-10T09:37:53.857 INFO:teuthology.orchestra.run.smithi005.stdout:full_ratio 0.95 2024-09-10T09:37:53.857 INFO:teuthology.orchestra.run.smithi005.stdout:backfillfull_ratio 0.9 2024-09-10T09:37:53.857 INFO:teuthology.orchestra.run.smithi005.stdout:nearfull_ratio 0.85 2024-09-10T09:37:53.858 INFO:teuthology.orchestra.run.smithi005.stdout:require_min_compat_client luminous 2024-09-10T09:37:53.858 INFO:teuthology.orchestra.run.smithi005.stdout:min_compat_client luminous 2024-09-10T09:37:53.858 INFO:teuthology.orchestra.run.smithi005.stdout:require_osd_release squid 2024-09-10T09:37:53.858 INFO:teuthology.orchestra.run.smithi005.stdout:stretch_mode_enabled false 2024-09-10T09:37:53.858 INFO:teuthology.orchestra.run.smithi005.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 21 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-09-10T09:37:53.858 INFO:teuthology.orchestra.run.smithi005.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 85 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 2.25 2024-09-10T09:37:53.858 INFO:teuthology.orchestra.run.smithi005.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 85 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.05 application cephfs read_balance_score 1.50 2024-09-10T09:37:53.858 INFO:teuthology.orchestra.run.smithi005.stdout:pool 4 'cephfs_data_ec' erasure profile cephfs_data_ec size 4 min_size 3 crush_rule 1 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 92 flags hashpspool,ec_overwrites stripe_width 8192 pg_num_min 64 target_size_ratio 0.9 application cephfs 2024-09-10T09:37:53.858 INFO:teuthology.orchestra.run.smithi005.stdout:max_osd 12 2024-09-10T09:37:53.858 INFO:teuthology.orchestra.run.smithi005.stdout:osd.0 up in weight 1 up_from 8 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.5:6802/954535266,v1:172.21.15.5:6803/954535266] [v2:172.21.15.5:6804/954535266,v1:172.21.15.5:6805/954535266] exists,up 085749a8-e882-4377-92dd-34bb266c0f0b 2024-09-10T09:37:53.858 INFO:teuthology.orchestra.run.smithi005.stdout:osd.1 up in weight 1 up_from 13 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.102:6800/733924375,v1:172.21.15.102:6801/733924375] [v2:172.21.15.102:6802/733924375,v1:172.21.15.102:6803/733924375] exists,up 0a76c795-2573-4a92-969a-2ec5dd65c094 2024-09-10T09:37:53.858 INFO:teuthology.orchestra.run.smithi005.stdout:osd.2 up in weight 1 up_from 18 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.159:6800/2236204189,v1:172.21.15.159:6801/2236204189] [v2:172.21.15.159:6802/2236204189,v1:172.21.15.159:6803/2236204189] exists,up 0c98fcb8-60ab-4473-8995-1d9e97c577e6 2024-09-10T09:37:53.858 INFO:teuthology.orchestra.run.smithi005.stdout:osd.3 up in weight 1 up_from 25 up_thru 93 down_at 0 last_clean_interval [0,0) [v2:172.21.15.5:6810/2221468725,v1:172.21.15.5:6811/2221468725] [v2:172.21.15.5:6812/2221468725,v1:172.21.15.5:6813/2221468725] exists,up 88eecdeb-5cee-4f78-a3f9-a66bf4bf772a 2024-09-10T09:37:53.859 INFO:teuthology.orchestra.run.smithi005.stdout:osd.4 up in weight 1 up_from 32 up_thru 93 down_at 0 last_clean_interval [0,0) [v2:172.21.15.102:6808/594465753,v1:172.21.15.102:6809/594465753] [v2:172.21.15.102:6810/594465753,v1:172.21.15.102:6811/594465753] exists,up f2e7314d-7938-4668-9011-45c175f66b48 2024-09-10T09:37:53.859 INFO:teuthology.orchestra.run.smithi005.stdout:osd.5 up in weight 1 up_from 37 up_thru 95 down_at 0 last_clean_interval [0,0) [v2:172.21.15.159:6808/3230280402,v1:172.21.15.159:6809/3230280402] [v2:172.21.15.159:6810/3230280402,v1:172.21.15.159:6811/3230280402] exists,up 0c0366d3-658b-4bf0-ba17-aff28a02e948 2024-09-10T09:37:53.859 INFO:teuthology.orchestra.run.smithi005.stdout:osd.6 up in weight 1 up_from 45 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.5:6818/1907651140,v1:172.21.15.5:6819/1907651140] [v2:172.21.15.5:6820/1907651140,v1:172.21.15.5:6821/1907651140] exists,up 682670a6-51d5-46e8-af6e-dc2d8c1cf407 2024-09-10T09:37:53.859 INFO:teuthology.orchestra.run.smithi005.stdout:osd.7 up in weight 1 up_from 50 up_thru 95 down_at 0 last_clean_interval [0,0) [v2:172.21.15.102:6816/692040360,v1:172.21.15.102:6817/692040360] [v2:172.21.15.102:6818/692040360,v1:172.21.15.102:6819/692040360] exists,up 8ae0f4a9-99e7-4563-aa2a-15169684a8b0 2024-09-10T09:37:53.859 INFO:teuthology.orchestra.run.smithi005.stdout:osd.8 up in weight 1 up_from 57 up_thru 93 down_at 0 last_clean_interval [0,0) [v2:172.21.15.159:6816/2107291799,v1:172.21.15.159:6817/2107291799] [v2:172.21.15.159:6818/2107291799,v1:172.21.15.159:6819/2107291799] exists,up e37c845e-ced9-4413-9bbe-bc090537c2bc 2024-09-10T09:37:53.859 INFO:teuthology.orchestra.run.smithi005.stdout:osd.9 up in weight 1 up_from 65 up_thru 95 down_at 0 last_clean_interval [0,0) [v2:172.21.15.5:6826/902431281,v1:172.21.15.5:6827/902431281] [v2:172.21.15.5:6828/902431281,v1:172.21.15.5:6829/902431281] exists,up 91a5b6cc-08da-47be-af12-d0896a849309 2024-09-10T09:37:53.859 INFO:teuthology.orchestra.run.smithi005.stdout:osd.10 up in weight 1 up_from 70 up_thru 95 down_at 0 last_clean_interval [0,0) [v2:172.21.15.102:6824/2853806211,v1:172.21.15.102:6825/2853806211] [v2:172.21.15.102:6826/2853806211,v1:172.21.15.102:6827/2853806211] exists,up 2211030c-b00e-402b-b4d7-c587eb29a490 2024-09-10T09:37:53.859 INFO:teuthology.orchestra.run.smithi005.stdout:osd.11 up in weight 1 up_from 77 up_thru 95 down_at 0 last_clean_interval [0,0) [v2:172.21.15.159:6824/969941218,v1:172.21.15.159:6825/969941218] [v2:172.21.15.159:6826/969941218,v1:172.21.15.159:6827/969941218] exists,up 4e9699ec-8ff2-4005-9d84-af5600b3a2ee 2024-09-10T09:37:53.859 INFO:teuthology.orchestra.run.smithi005.stdout:pg_upmap_items 2.0 [8,4] 2024-09-10T09:37:53.859 INFO:teuthology.orchestra.run.smithi005.stdout:pg_upmap_items 2.6 [8,11] 2024-09-10T09:37:53.859 INFO:teuthology.orchestra.run.smithi005.stdout:pg_upmap_items 2.a [2,11] 2024-09-10T09:37:53.859 INFO:teuthology.orchestra.run.smithi005.stdout:pg_upmap_items 2.1b [2,4] 2024-09-10T09:37:53.859 INFO:teuthology.orchestra.run.smithi005.stdout:pg_upmap_items 3.0 [4,8] 2024-09-10T09:37:53.859 INFO:teuthology.orchestra.run.smithi005.stdout:pg_upmap_items 3.3 [4,9] 2024-09-10T09:37:53.860 INFO:teuthology.orchestra.run.smithi005.stdout:pg_upmap_items 3.c [3,9] 2024-09-10T09:37:53.860 INFO:teuthology.orchestra.run.smithi005.stdout:pg_upmap_items 3.1d [4,6] 2024-09-10T09:37:53.860 INFO:teuthology.orchestra.run.smithi005.stdout:pg_upmap_items 3.1e [3,9,5,7] 2024-09-10T09:37:53.860 INFO:teuthology.orchestra.run.smithi005.stdout:pg_upmap_items 3.27 [4,8] 2024-09-10T09:37:53.860 INFO:teuthology.orchestra.run.smithi005.stdout:pg_upmap_items 3.39 [5,6] 2024-09-10T09:37:53.860 INFO:teuthology.orchestra.run.smithi005.stdout:pg_upmap_items 4.8 [5,8] 2024-09-10T09:37:53.860 INFO:teuthology.orchestra.run.smithi005.stdout:pg_upmap_items 4.c [9,10] 2024-09-10T09:37:53.860 INFO:teuthology.orchestra.run.smithi005.stdout:pg_upmap_items 4.18 [9,8] 2024-09-10T09:37:53.860 INFO:teuthology.orchestra.run.smithi005.stdout:pg_upmap_items 4.1f [9,8] 2024-09-10T09:37:53.860 INFO:teuthology.orchestra.run.smithi005.stdout:blocklist 172.21.15.5:6801/1527422686 expires 2024-09-11T09:28:05.436742+0000 2024-09-10T09:37:53.861 INFO:teuthology.orchestra.run.smithi005.stdout:blocklist 172.21.15.5:0/2276144345 expires 2024-09-11T09:28:05.436742+0000 2024-09-10T09:37:53.861 INFO:teuthology.orchestra.run.smithi005.stdout:blocklist 172.21.15.5:0/412040104 expires 2024-09-11T09:28:05.436742+0000 2024-09-10T09:37:53.861 INFO:teuthology.orchestra.run.smithi005.stdout:blocklist 172.21.15.5:6800/1527422686 expires 2024-09-11T09:28:05.436742+0000 2024-09-10T09:37:53.861 INFO:teuthology.orchestra.run.smithi005.stdout:blocklist 172.21.15.5:0/1856581170 expires 2024-09-11T09:28:05.436742+0000 2024-09-10T09:37:53.861 INFO:teuthology.orchestra.run.smithi005.stdout:blocklist 172.21.15.5:0/2847668298 expires 2024-09-11T09:27:37.778500+0000 2024-09-10T09:37:53.861 INFO:teuthology.orchestra.run.smithi005.stdout:blocklist 172.21.15.5:6801/14425913 expires 2024-09-11T09:27:37.778500+0000 2024-09-10T09:37:53.861 INFO:teuthology.orchestra.run.smithi005.stdout:blocklist 172.21.15.5:0/166910782 expires 2024-09-11T09:27:37.778500+0000 2024-09-10T09:37:53.861 INFO:teuthology.orchestra.run.smithi005.stdout:blocklist 172.21.15.5:6800/14425913 expires 2024-09-11T09:27:37.778500+0000 2024-09-10T09:37:53.861 INFO:teuthology.orchestra.run.smithi005.stdout:blocklist 172.21.15.5:0/1711858179 expires 2024-09-11T09:27:37.778500+0000 2024-09-10T09:37:53.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:53.859+0000 7f7d22e00640 1 -- 172.21.15.5:0/572924317 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f7d000a2c30 msgr2=0x7f7d000a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:37:53.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:53.859+0000 7f7d22e00640 1 --2- 172.21.15.5:0/572924317 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f7d000a2c30 0x7f7d000a50f0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f7d0c009bd0 tx=0x7f7d0c005f90 comp rx=0 tx=0).stop 2024-09-10T09:37:53.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:53.859+0000 7f7d22e00640 1 -- 172.21.15.5:0/572924317 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7d1c104f60 msgr2=0x7f7d1c19a5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:37:53.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:53.859+0000 7f7d22e00640 1 --2- 172.21.15.5:0/572924317 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7d1c104f60 0x7f7d1c19a5b0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f7d0400e9c0 tx=0x7f7d0400ee80 comp rx=0 tx=0).stop 2024-09-10T09:37:53.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:53.859+0000 7f7d22e00640 1 -- 172.21.15.5:0/572924317 shutdown_connections 2024-09-10T09:37:53.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:53.859+0000 7f7d22e00640 1 --2- 172.21.15.5:0/572924317 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f7d000a2c30 0x7f7d000a50f0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:53.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:53.859+0000 7f7d22e00640 1 --2- 172.21.15.5:0/572924317 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7d1c105920 0x7f7d1c1a1630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:53.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:53.860+0000 7f7d22e00640 1 --2- 172.21.15.5:0/572924317 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7d1c104f60 0x7f7d1c19a5b0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:53.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:53.860+0000 7f7d22e00640 1 --2- 172.21.15.5:0/572924317 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7d1c103e00 0x7f7d1c19a070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:53.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:53.860+0000 7f7d22e00640 1 -- 172.21.15.5:0/572924317 >> 172.21.15.5:0/572924317 conn(0x7f7d1c0ff5b0 msgr2=0x7f7d1c1082f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:37:53.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:53.860+0000 7f7d22e00640 1 -- 172.21.15.5:0/572924317 shutdown_connections 2024-09-10T09:37:53.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:53.860+0000 7f7d22e00640 1 -- 172.21.15.5:0/572924317 wait complete. 2024-09-10T09:37:54.616 INFO:teuthology.run_tasks:Running task fs.ready... 2024-09-10T09:37:54.628 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T09:37:55.009 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:37:55.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:37:54 smithi005 ceph-mon[30350]: pgmap v380: 129 pgs: 129 active+clean; 646 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-10T09:37:55.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:37:54 smithi005 ceph-mon[30350]: from='client.? 172.21.15.5:0/572924317' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-09-10T09:37:55.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:37:54 smithi159 ceph-mon[35513]: pgmap v380: 129 pgs: 129 active+clean; 646 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-10T09:37:55.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:37:54 smithi159 ceph-mon[35513]: from='client.? 172.21.15.5:0/572924317' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-09-10T09:37:55.135 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:37:54 smithi102 ceph-mon[35442]: pgmap v380: 129 pgs: 129 active+clean; 646 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-10T09:37:55.135 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:37:54 smithi102 ceph-mon[35442]: from='client.? 172.21.15.5:0/572924317' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-09-10T09:37:55.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:55.578+0000 7fcd27200640 1 -- 172.21.15.5:0/1188511272 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcd20074590 msgr2=0x7fcd20074970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:37:55.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:55.578+0000 7fcd27200640 1 --2- 172.21.15.5:0/1188511272 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcd20074590 0x7fcd20074970 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fcd10009900 tx=0x7fcd1002a480 comp rx=0 tx=0).stop 2024-09-10T09:37:55.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:55.578+0000 7fcd27200640 1 -- 172.21.15.5:0/1188511272 shutdown_connections 2024-09-10T09:37:55.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:55.578+0000 7fcd27200640 1 --2- 172.21.15.5:0/1188511272 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcd20112b90 0x7fcd20114f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:55.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:55.578+0000 7fcd27200640 1 --2- 172.21.15.5:0/1188511272 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcd20074eb0 0x7fcd20112650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:55.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:55.578+0000 7fcd27200640 1 --2- 172.21.15.5:0/1188511272 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcd20074590 0x7fcd20074970 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:55.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:55.578+0000 7fcd27200640 1 -- 172.21.15.5:0/1188511272 >> 172.21.15.5:0/1188511272 conn(0x7fcd200fd8c0 msgr2=0x7fcd200ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:37:55.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:55.578+0000 7fcd27200640 1 -- 172.21.15.5:0/1188511272 shutdown_connections 2024-09-10T09:37:55.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:55.578+0000 7fcd27200640 1 -- 172.21.15.5:0/1188511272 wait complete. 2024-09-10T09:37:55.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:55.579+0000 7fcd27200640 1 Processor -- start 2024-09-10T09:37:55.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:55.579+0000 7fcd27200640 1 -- start start 2024-09-10T09:37:55.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:55.579+0000 7fcd27200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcd20074590 0x7fcd201a4760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:37:55.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:55.580+0000 7fcd27200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcd20074eb0 0x7fcd201a4ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:37:55.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:55.580+0000 7fcd27200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcd20112b90 0x7fcd2019e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:37:55.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:55.580+0000 7fcd27200640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fcd201185c0 con 0x7fcd20074eb0 2024-09-10T09:37:55.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:55.580+0000 7fcd27200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fcd20118440 con 0x7fcd20112b90 2024-09-10T09:37:55.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:55.580+0000 7fcd27200640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fcd20118740 con 0x7fcd20074590 2024-09-10T09:37:55.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:55.580+0000 7fcd17e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcd20074eb0 0x7fcd201a4ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:37:55.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:55.580+0000 7fcd1fe00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcd20074590 0x7fcd201a4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:37:55.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:55.580+0000 7fcd1fe00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcd20074590 0x7fcd201a4760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:35808/0 (socket says 172.21.15.5:35808) 2024-09-10T09:37:55.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:55.580+0000 7fcd1fe00640 1 -- 172.21.15.5:0/693103612 learned_addr learned my addr 172.21.15.5:0/693103612 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:37:55.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:55.580+0000 7fcd25200640 1 --2- 172.21.15.5:0/693103612 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcd20112b90 0x7fcd2019e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:37:55.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:55.580+0000 7fcd1fe00640 1 -- 172.21.15.5:0/693103612 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcd20112b90 msgr2=0x7fcd2019e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:37:55.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:55.580+0000 7fcd1fe00640 1 --2- 172.21.15.5:0/693103612 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcd20112b90 0x7fcd2019e920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:55.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:55.580+0000 7fcd1fe00640 1 -- 172.21.15.5:0/693103612 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcd20074eb0 msgr2=0x7fcd201a4ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:37:55.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:55.580+0000 7fcd1fe00640 1 --2- 172.21.15.5:0/693103612 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcd20074eb0 0x7fcd201a4ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:55.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:55.580+0000 7fcd1fe00640 1 -- 172.21.15.5:0/693103612 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd2019f050 con 0x7fcd20074590 2024-09-10T09:37:55.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:55.580+0000 7fcd17e00640 1 --2- 172.21.15.5:0/693103612 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcd20074eb0 0x7fcd201a4ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:37:55.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:55.581+0000 7fcd1fe00640 1 --2- 172.21.15.5:0/693103612 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcd20074590 0x7fcd201a4760 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fcd10004770 tx=0x7fcd10003c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:37:55.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:55.581+0000 7fcd1d600640 1 -- 172.21.15.5:0/693103612 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd1003f070 con 0x7fcd20074590 2024-09-10T09:37:55.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:55.581+0000 7fcd27200640 1 -- 172.21.15.5:0/693103612 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd2019f2e0 con 0x7fcd20074590 2024-09-10T09:37:55.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:55.581+0000 7fcd1d600640 1 -- 172.21.15.5:0/693103612 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcd10031ca0 con 0x7fcd20074590 2024-09-10T09:37:55.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:55.581+0000 7fcd1d600640 1 -- 172.21.15.5:0/693103612 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd10035030 con 0x7fcd20074590 2024-09-10T09:37:55.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:55.581+0000 7fcd27200640 1 -- 172.21.15.5:0/693103612 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd201ab630 con 0x7fcd20074590 2024-09-10T09:37:55.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:55.582+0000 7fcd27200640 1 -- 172.21.15.5:0/693103612 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd20074970 con 0x7fcd20074590 2024-09-10T09:37:55.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:55.584+0000 7fcd1d600640 1 -- 172.21.15.5:0/693103612 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7fcd10042070 con 0x7fcd20074590 2024-09-10T09:37:55.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:55.586+0000 7fcd1d600640 1 --2- 172.21.15.5:0/693103612 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fcd040a2b80 0x7fcd040a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:37:55.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:55.586+0000 7fcd1d600640 1 -- 172.21.15.5:0/693103612 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fcd100f0060 con 0x7fcd20074590 2024-09-10T09:37:55.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:55.586+0000 7fcd17e00640 1 --2- 172.21.15.5:0/693103612 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fcd040a2b80 0x7fcd040a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:37:55.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:55.587+0000 7fcd17e00640 1 --2- 172.21.15.5:0/693103612 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fcd040a2b80 0x7fcd040a5040 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fcd200ffec0 tx=0x7fcd08005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:37:55.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:55.588+0000 7fcd1d600640 1 -- 172.21.15.5:0/693103612 <== mon.1 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fcd100bc5e0 con 0x7fcd20074590 2024-09-10T09:37:55.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:55.811+0000 7fcd27200640 1 -- 172.21.15.5:0/693103612 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcd20104fd0 con 0x7fcd20074590 2024-09-10T09:37:55.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:55.812+0000 7fcd1d600640 1 -- 172.21.15.5:0/693103612 <== mon.1 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11969 (secure 0 0 0) 0x7fcd100c08e0 con 0x7fcd20074590 2024-09-10T09:37:55.813 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-10T09:37:55.813 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":29,"btime":"2024-09-10T09:36:16:492171+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14454,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/2970594311","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2970594311},{"type":"v1","addr":"172.21.15.5:6835","nonce":2970594311}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14514,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6839/3616502174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":3616502174},{"type":"v1","addr":"172.21.15.5:6839","nonce":3616502174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T09:35:29.626287+0000","modified":"2024-09-10T09:36:15.764196+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24542,"mds_1":24535,"mds_2":24511,"mds_3":24506,"mds_4":24443},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14520":{"gid":14520,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/2073599842","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":2073599842},{"type":"v1","addr":"172.21.15.5:6841","nonce":2073599842}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"k","rank":4,"incarnation":26,"state":"up:active","state_seq":24,"addr":"172.21.15.102:6833/2457007427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":2457007427},{"type":"v1","addr":"172.21.15.102:6833","nonce":2457007427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/356445159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":356445159},{"type":"v1","addr":"172.21.15.159:6833","nonce":356445159}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6837/2894314507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":2894314507},{"type":"v1","addr":"172.21.15.5:6837","nonce":2894314507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.102:6835/1647549111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":1647549111},{"type":"v1","addr":"172.21.15.102:6835","nonce":1647549111}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6835/952657242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":952657242},{"type":"v1","addr":"172.21.15.159:6835","nonce":952657242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":19,"addr":"172.21.15.102:6837/2689776764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":2689776764},{"type":"v1","addr":"172.21.15.102:6837","nonce":2689776764}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.159:6837/620627639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":620627639},{"type":"v1","addr":"172.21.15.159:6837","nonce":620627639}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24535":{"gid":24535,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":18,"addr":"172.21.15.159:6839/2073739599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":2073739599},{"type":"v1","addr":"172.21.15.159:6839","nonce":2073739599}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24542":{"gid":24542,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.102:6839/335409749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6838","nonce":335409749},{"type":"v1","addr":"172.21.15.102:6839","nonce":335409749}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24542,"qdb_cluster":[24443,24506,24511,24535,24542]},"id":1}]} 2024-09-10T09:37:55.814 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 29 2024-09-10T09:37:55.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:55.814+0000 7fcd27200640 1 -- 172.21.15.5:0/693103612 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fcd040a2b80 msgr2=0x7fcd040a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:37:55.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:55.814+0000 7fcd27200640 1 --2- 172.21.15.5:0/693103612 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fcd040a2b80 0x7fcd040a5040 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fcd200ffec0 tx=0x7fcd08005f90 comp rx=0 tx=0).stop 2024-09-10T09:37:55.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:55.814+0000 7fcd27200640 1 -- 172.21.15.5:0/693103612 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcd20074590 msgr2=0x7fcd201a4760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:37:55.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:55.814+0000 7fcd27200640 1 --2- 172.21.15.5:0/693103612 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcd20074590 0x7fcd201a4760 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fcd10004770 tx=0x7fcd10003c10 comp rx=0 tx=0).stop 2024-09-10T09:37:55.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:55.815+0000 7fcd27200640 1 -- 172.21.15.5:0/693103612 shutdown_connections 2024-09-10T09:37:55.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:55.815+0000 7fcd27200640 1 --2- 172.21.15.5:0/693103612 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fcd040a2b80 0x7fcd040a5040 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:55.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:55.815+0000 7fcd27200640 1 --2- 172.21.15.5:0/693103612 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcd20112b90 0x7fcd2019e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:55.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:55.815+0000 7fcd27200640 1 --2- 172.21.15.5:0/693103612 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcd20074eb0 0x7fcd201a4ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:55.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:55.815+0000 7fcd27200640 1 --2- 172.21.15.5:0/693103612 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcd20074590 0x7fcd201a4760 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:55.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:55.815+0000 7fcd27200640 1 -- 172.21.15.5:0/693103612 >> 172.21.15.5:0/693103612 conn(0x7fcd200fd8c0 msgr2=0x7fcd200ff050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:37:55.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:55.815+0000 7fcd27200640 1 -- 172.21.15.5:0/693103612 shutdown_connections 2024-09-10T09:37:55.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:55.815+0000 7fcd27200640 1 -- 172.21.15.5:0/693103612 wait complete. 2024-09-10T09:37:56.608 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T09:37:56.986 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:37:57.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:37:56 smithi005 ceph-mon[30350]: pgmap v381: 129 pgs: 129 active+clean; 646 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-09-10T09:37:57.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:37:56 smithi005 ceph-mon[30350]: from='client.? 172.21.15.5:0/693103612' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:37:57.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:37:56 smithi159 ceph-mon[35513]: pgmap v381: 129 pgs: 129 active+clean; 646 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-09-10T09:37:57.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:37:56 smithi159 ceph-mon[35513]: from='client.? 172.21.15.5:0/693103612' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:37:57.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:37:56 smithi102 ceph-mon[35442]: pgmap v381: 129 pgs: 129 active+clean; 646 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-09-10T09:37:57.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:37:56 smithi102 ceph-mon[35442]: from='client.? 172.21.15.5:0/693103612' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:37:57.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:57.536+0000 7f7158800640 1 -- 172.21.15.5:0/826984461 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7150109120 msgr2=0x7f7150110ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:37:57.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:57.536+0000 7f7158800640 1 --2- 172.21.15.5:0/826984461 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7150109120 0x7f7150110ca0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f714c009ca0 tx=0x7f714c02a8c0 comp rx=0 tx=0).stop 2024-09-10T09:37:57.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:57.537+0000 7f7158800640 1 -- 172.21.15.5:0/826984461 shutdown_connections 2024-09-10T09:37:57.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:57.537+0000 7f7158800640 1 --2- 172.21.15.5:0/826984461 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7150109120 0x7f7150110ca0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:57.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:57.537+0000 7f7158800640 1 --2- 172.21.15.5:0/826984461 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f71501049a0 0x7f71501089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:57.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:57.537+0000 7f7158800640 1 --2- 172.21.15.5:0/826984461 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7150103ff0 0x7f71501043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:57.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:57.537+0000 7f7158800640 1 -- 172.21.15.5:0/826984461 >> 172.21.15.5:0/826984461 conn(0x7f71500ffa90 msgr2=0x7f7150101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:37:57.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:57.537+0000 7f7158800640 1 -- 172.21.15.5:0/826984461 shutdown_connections 2024-09-10T09:37:57.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:57.537+0000 7f7158800640 1 -- 172.21.15.5:0/826984461 wait complete. 2024-09-10T09:37:57.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:57.538+0000 7f7158800640 1 Processor -- start 2024-09-10T09:37:57.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:57.538+0000 7f7158800640 1 -- start start 2024-09-10T09:37:57.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:57.538+0000 7f7158800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7150103ff0 0x7f715019a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:37:57.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:57.538+0000 7f7158800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f71501049a0 0x7f715019aae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:37:57.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:57.538+0000 7f7158800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7150109120 0x7f715019ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:37:57.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:57.538+0000 7f7158800640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f7150114180 con 0x7f7150109120 2024-09-10T09:37:57.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:57.538+0000 7f7158800640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f7150114000 con 0x7f7150103ff0 2024-09-10T09:37:57.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:57.538+0000 7f7158800640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f7150114300 con 0x7f71501049a0 2024-09-10T09:37:57.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:57.538+0000 7f7155e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7150103ff0 0x7f715019a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:37:57.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:57.538+0000 7f7156800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7150109120 0x7f715019ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:37:57.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:57.538+0000 7f7155400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f71501049a0 0x7f715019aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:37:57.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:57.538+0000 7f7155400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f71501049a0 0x7f715019aae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:35832/0 (socket says 172.21.15.5:35832) 2024-09-10T09:37:57.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:57.538+0000 7f7155e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7150103ff0 0x7f715019a5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:34774/0 (socket says 172.21.15.5:34774) 2024-09-10T09:37:57.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:57.538+0000 7f7155400640 1 -- 172.21.15.5:0/2799766768 learned_addr learned my addr 172.21.15.5:0/2799766768 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:37:57.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:57.539+0000 7f7156800640 1 -- 172.21.15.5:0/2799766768 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f71501049a0 msgr2=0x7f715019aae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:37:57.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:57.539+0000 7f7156800640 1 --2- 172.21.15.5:0/2799766768 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f71501049a0 0x7f715019aae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:57.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:57.539+0000 7f7156800640 1 -- 172.21.15.5:0/2799766768 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7150103ff0 msgr2=0x7f715019a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:37:57.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:57.539+0000 7f7156800640 1 --2- 172.21.15.5:0/2799766768 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7150103ff0 0x7f715019a5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:57.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:57.539+0000 7f7156800640 1 -- 172.21.15.5:0/2799766768 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f715019f550 con 0x7f7150109120 2024-09-10T09:37:57.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:57.539+0000 7f7155e00640 1 --2- 172.21.15.5:0/2799766768 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7150103ff0 0x7f715019a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:37:57.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:57.539+0000 7f7155400640 1 --2- 172.21.15.5:0/2799766768 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f71501049a0 0x7f715019aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:37:57.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:57.539+0000 7f7156800640 1 --2- 172.21.15.5:0/2799766768 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7150109120 0x7f715019ee70 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f714c005e30 tx=0x7f714c003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:37:57.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:57.540+0000 7f714aa00640 1 -- 172.21.15.5:0/2799766768 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f714c031650 con 0x7f7150109120 2024-09-10T09:37:57.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:57.540+0000 7f714aa00640 1 -- 172.21.15.5:0/2799766768 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f714c0317f0 con 0x7f7150109120 2024-09-10T09:37:57.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:57.540+0000 7f7158800640 1 -- 172.21.15.5:0/2799766768 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f715019f7e0 con 0x7f7150109120 2024-09-10T09:37:57.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:57.540+0000 7f7158800640 1 -- 172.21.15.5:0/2799766768 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71501a7120 con 0x7f7150109120 2024-09-10T09:37:57.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:57.540+0000 7f714aa00640 1 -- 172.21.15.5:0/2799766768 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f714c02fa30 con 0x7f7150109120 2024-09-10T09:37:57.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:57.541+0000 7f7158800640 1 -- 172.21.15.5:0/2799766768 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f71501043d0 con 0x7f7150109120 2024-09-10T09:37:57.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:57.545+0000 7f714aa00640 1 -- 172.21.15.5:0/2799766768 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f714c02fbd0 con 0x7f7150109120 2024-09-10T09:37:57.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:57.546+0000 7f714aa00640 1 --2- 172.21.15.5:0/2799766768 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f71380a2b80 0x7f71380a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:37:57.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:57.546+0000 7f714aa00640 1 -- 172.21.15.5:0/2799766768 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f714c0f08c0 con 0x7f7150109120 2024-09-10T09:37:57.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:57.546+0000 7f7155e00640 1 --2- 172.21.15.5:0/2799766768 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f71380a2b80 0x7f71380a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:37:57.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:57.547+0000 7f7155e00640 1 --2- 172.21.15.5:0/2799766768 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f71380a2b80 0x7f71380a5040 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f7144009fc0 tx=0x7f7144005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:37:57.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:57.549+0000 7f714aa00640 1 -- 172.21.15.5:0/2799766768 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f714c0bd460 con 0x7f7150109120 2024-09-10T09:37:57.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:57.770+0000 7f7158800640 1 -- 172.21.15.5:0/2799766768 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f71501a7350 con 0x7f7150109120 2024-09-10T09:37:57.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:57.771+0000 7f714aa00640 1 -- 172.21.15.5:0/2799766768 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11969 (secure 0 0 0) 0x7f714c0319a0 con 0x7f7150109120 2024-09-10T09:37:57.772 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-10T09:37:57.772 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":29,"btime":"2024-09-10T09:36:16:492171+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14454,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/2970594311","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2970594311},{"type":"v1","addr":"172.21.15.5:6835","nonce":2970594311}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14514,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6839/3616502174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":3616502174},{"type":"v1","addr":"172.21.15.5:6839","nonce":3616502174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T09:35:29.626287+0000","modified":"2024-09-10T09:36:15.764196+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24542,"mds_1":24535,"mds_2":24511,"mds_3":24506,"mds_4":24443},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14520":{"gid":14520,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/2073599842","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":2073599842},{"type":"v1","addr":"172.21.15.5:6841","nonce":2073599842}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"k","rank":4,"incarnation":26,"state":"up:active","state_seq":24,"addr":"172.21.15.102:6833/2457007427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":2457007427},{"type":"v1","addr":"172.21.15.102:6833","nonce":2457007427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/356445159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":356445159},{"type":"v1","addr":"172.21.15.159:6833","nonce":356445159}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6837/2894314507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":2894314507},{"type":"v1","addr":"172.21.15.5:6837","nonce":2894314507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.102:6835/1647549111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":1647549111},{"type":"v1","addr":"172.21.15.102:6835","nonce":1647549111}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6835/952657242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":952657242},{"type":"v1","addr":"172.21.15.159:6835","nonce":952657242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":19,"addr":"172.21.15.102:6837/2689776764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":2689776764},{"type":"v1","addr":"172.21.15.102:6837","nonce":2689776764}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.159:6837/620627639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":620627639},{"type":"v1","addr":"172.21.15.159:6837","nonce":620627639}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24535":{"gid":24535,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":18,"addr":"172.21.15.159:6839/2073739599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":2073739599},{"type":"v1","addr":"172.21.15.159:6839","nonce":2073739599}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24542":{"gid":24542,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.102:6839/335409749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6838","nonce":335409749},{"type":"v1","addr":"172.21.15.102:6839","nonce":335409749}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24542,"qdb_cluster":[24443,24506,24511,24535,24542]},"id":1}]} 2024-09-10T09:37:57.772 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 29 2024-09-10T09:37:57.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:57.774+0000 7f7158800640 1 -- 172.21.15.5:0/2799766768 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f71380a2b80 msgr2=0x7f71380a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:37:57.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:57.774+0000 7f7158800640 1 --2- 172.21.15.5:0/2799766768 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f71380a2b80 0x7f71380a5040 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f7144009fc0 tx=0x7f7144005f90 comp rx=0 tx=0).stop 2024-09-10T09:37:57.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:57.774+0000 7f7158800640 1 -- 172.21.15.5:0/2799766768 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7150109120 msgr2=0x7f715019ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:37:57.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:57.774+0000 7f7158800640 1 --2- 172.21.15.5:0/2799766768 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7150109120 0x7f715019ee70 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f714c005e30 tx=0x7f714c003f10 comp rx=0 tx=0).stop 2024-09-10T09:37:57.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:57.774+0000 7f7158800640 1 -- 172.21.15.5:0/2799766768 shutdown_connections 2024-09-10T09:37:57.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:57.774+0000 7f7158800640 1 --2- 172.21.15.5:0/2799766768 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f71380a2b80 0x7f71380a5040 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:57.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:57.774+0000 7f7158800640 1 --2- 172.21.15.5:0/2799766768 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7150109120 0x7f715019ee70 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:57.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:57.774+0000 7f7158800640 1 --2- 172.21.15.5:0/2799766768 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f71501049a0 0x7f715019aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:57.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:57.774+0000 7f7158800640 1 --2- 172.21.15.5:0/2799766768 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7150103ff0 0x7f715019a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:57.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:57.774+0000 7f7158800640 1 -- 172.21.15.5:0/2799766768 >> 172.21.15.5:0/2799766768 conn(0x7f71500ffa90 msgr2=0x7f71501005b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:37:57.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:57.774+0000 7f7158800640 1 -- 172.21.15.5:0/2799766768 shutdown_connections 2024-09-10T09:37:57.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:57.775+0000 7f7158800640 1 -- 172.21.15.5:0/2799766768 wait complete. 2024-09-10T09:37:58.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:37:57 smithi005 ceph-mon[30350]: from='client.? 172.21.15.5:0/2799766768' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:37:58.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:37:57 smithi159 ceph-mon[35513]: from='client.? 172.21.15.5:0/2799766768' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:37:58.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:37:57 smithi102 ceph-mon[35442]: from='client.? 172.21.15.5:0/2799766768' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:37:58.584 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-10T09:37:58.963 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:37:59.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:37:58 smithi005 ceph-mon[30350]: pgmap v382: 129 pgs: 129 active+clean; 646 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-09-10T09:37:59.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:37:58 smithi159 ceph-mon[35513]: pgmap v382: 129 pgs: 129 active+clean; 646 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-09-10T09:37:59.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:37:58 smithi102 ceph-mon[35442]: pgmap v382: 129 pgs: 129 active+clean; 646 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-09-10T09:37:59.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:59.512+0000 7fdbd8600640 1 -- 172.21.15.5:0/2444160264 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdbd0074ee0 msgr2=0x7fdbd010def0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:37:59.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:59.512+0000 7fdbd8600640 1 --2- 172.21.15.5:0/2444160264 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdbd0074ee0 0x7fdbd010def0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fdbbc009900 tx=0x7fdbbc02a480 comp rx=0 tx=0).stop 2024-09-10T09:37:59.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:59.512+0000 7fdbd8600640 1 -- 172.21.15.5:0/2444160264 shutdown_connections 2024-09-10T09:37:59.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:59.512+0000 7fdbd8600640 1 --2- 172.21.15.5:0/2444160264 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdbd010e430 0x7fdbd0110820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:59.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:59.512+0000 7fdbd8600640 1 --2- 172.21.15.5:0/2444160264 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdbd0074ee0 0x7fdbd010def0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:59.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:59.512+0000 7fdbd8600640 1 --2- 172.21.15.5:0/2444160264 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdbd00745c0 0x7fdbd00749a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:59.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:59.512+0000 7fdbd8600640 1 -- 172.21.15.5:0/2444160264 >> 172.21.15.5:0/2444160264 conn(0x7fdbd00fd8b0 msgr2=0x7fdbd00ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:37:59.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:59.512+0000 7fdbd8600640 1 -- 172.21.15.5:0/2444160264 shutdown_connections 2024-09-10T09:37:59.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:59.512+0000 7fdbd8600640 1 -- 172.21.15.5:0/2444160264 wait complete. 2024-09-10T09:37:59.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:59.513+0000 7fdbd8600640 1 Processor -- start 2024-09-10T09:37:59.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:59.513+0000 7fdbd8600640 1 -- start start 2024-09-10T09:37:59.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:59.513+0000 7fdbd8600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdbd00745c0 0x7fdbd019a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:37:59.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:59.514+0000 7fdbd8600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdbd0074ee0 0x7fdbd019ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:37:59.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:59.514+0000 7fdbd8600640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdbd010e430 0x7fdbd019ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:37:59.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:59.514+0000 7fdbd8600640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fdbd0114180 con 0x7fdbd00745c0 2024-09-10T09:37:59.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:59.514+0000 7fdbd8600640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fdbd0114000 con 0x7fdbd0074ee0 2024-09-10T09:37:59.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:59.514+0000 7fdbd8600640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fdbd0114300 con 0x7fdbd010e430 2024-09-10T09:37:59.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:59.514+0000 7fdbd5c00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdbd00745c0 0x7fdbd019a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:37:59.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:59.514+0000 7fdbd5200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdbd0074ee0 0x7fdbd019ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:37:59.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:59.514+0000 7fdbd5c00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdbd00745c0 0x7fdbd019a640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38662/0 (socket says 172.21.15.5:38662) 2024-09-10T09:37:59.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:59.514+0000 7fdbd5c00640 1 -- 172.21.15.5:0/520518990 learned_addr learned my addr 172.21.15.5:0/520518990 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:37:59.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:59.514+0000 7fdbd6600640 1 --2- 172.21.15.5:0/520518990 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdbd010e430 0x7fdbd019ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:37:59.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:59.514+0000 7fdbd5200640 1 -- 172.21.15.5:0/520518990 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdbd010e430 msgr2=0x7fdbd019ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:37:59.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:59.514+0000 7fdbd5200640 1 --2- 172.21.15.5:0/520518990 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdbd010e430 0x7fdbd019ef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:59.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:59.514+0000 7fdbd5200640 1 -- 172.21.15.5:0/520518990 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdbd00745c0 msgr2=0x7fdbd019a640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:37:59.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:59.514+0000 7fdbd5200640 1 --2- 172.21.15.5:0/520518990 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdbd00745c0 0x7fdbd019a640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:59.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:59.514+0000 7fdbd5200640 1 -- 172.21.15.5:0/520518990 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbd019f5f0 con 0x7fdbd0074ee0 2024-09-10T09:37:59.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:59.514+0000 7fdbd5c00640 1 --2- 172.21.15.5:0/520518990 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdbd00745c0 0x7fdbd019a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:37:59.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:59.515+0000 7fdbd5200640 1 --2- 172.21.15.5:0/520518990 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdbd0074ee0 0x7fdbd019ab80 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fdbbc005940 tx=0x7fdbbc0040c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:37:59.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:59.515+0000 7fdbce000640 1 -- 172.21.15.5:0/520518990 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbbc03f030 con 0x7fdbd0074ee0 2024-09-10T09:37:59.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:59.515+0000 7fdbd8600640 1 -- 172.21.15.5:0/520518990 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdbd019f880 con 0x7fdbd0074ee0 2024-09-10T09:37:59.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:59.515+0000 7fdbce000640 1 -- 172.21.15.5:0/520518990 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdbbc0044c0 con 0x7fdbd0074ee0 2024-09-10T09:37:59.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:59.515+0000 7fdbce000640 1 -- 172.21.15.5:0/520518990 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbbc035070 con 0x7fdbd0074ee0 2024-09-10T09:37:59.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:59.515+0000 7fdbd8600640 1 -- 172.21.15.5:0/520518990 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdbd01a71c0 con 0x7fdbd0074ee0 2024-09-10T09:37:59.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:59.516+0000 7fdbd8600640 1 -- 172.21.15.5:0/520518990 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdbd00749a0 con 0x7fdbd0074ee0 2024-09-10T09:37:59.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:59.518+0000 7fdbce000640 1 -- 172.21.15.5:0/520518990 <== mon.2 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7fdbbc042030 con 0x7fdbd0074ee0 2024-09-10T09:37:59.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:59.520+0000 7fdbce000640 1 --2- 172.21.15.5:0/520518990 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fdbb80a2c00 0x7fdbb80a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:37:59.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:59.520+0000 7fdbce000640 1 -- 172.21.15.5:0/520518990 <== mon.2 v2:172.21.15.102:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fdbbc0ef990 con 0x7fdbd0074ee0 2024-09-10T09:37:59.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:59.520+0000 7fdbd5c00640 1 --2- 172.21.15.5:0/520518990 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fdbb80a2c00 0x7fdbb80a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:37:59.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:59.520+0000 7fdbd5c00640 1 --2- 172.21.15.5:0/520518990 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fdbb80a2c00 0x7fdbb80a50c0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fdbc4011550 tx=0x7fdbc4005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:37:59.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:59.522+0000 7fdbce000640 1 -- 172.21.15.5:0/520518990 <== mon.2 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fdbbc0bbe50 con 0x7fdbd0074ee0 2024-09-10T09:37:59.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:59.710+0000 7fdbd8600640 1 -- 172.21.15.5:0/520518990 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fdbd0109020 con 0x7fdbd0074ee0 2024-09-10T09:37:59.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:59.712+0000 7fdbce000640 1 -- 172.21.15.5:0/520518990 <== mon.2 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v96) ==== 74+0+24846 (secure 0 0 0) 0x7fdbbc0c0150 con 0x7fdbd0074ee0 2024-09-10T09:37:59.714 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-10T09:37:59.714 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":96,"fsid":"bade4b50-6f56-11ef-bcea-c7b262605968","created":"2024-09-10T09:27:13.298498+0000","modified":"2024-09-10T09:37:07.520520+0000","last_up_change":"2024-09-10T09:34:30.730066+0000","last_in_change":"2024-09-10T09:34:07.340995+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T09:30:26.042255+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T09:35:24.461031+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T09:35:27.155349+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-10T09:35:35.121462+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"085749a8-e882-4377-92dd-34bb266c0f0b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":954535266},{"type":"v1","addr":"172.21.15.5:6803","nonce":954535266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":954535266},{"type":"v1","addr":"172.21.15.5:6805","nonce":954535266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":954535266},{"type":"v1","addr":"172.21.15.5:6809","nonce":954535266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":954535266},{"type":"v1","addr":"172.21.15.5:6807","nonce":954535266}]},"public_addr":"172.21.15.5:6803/954535266","cluster_addr":"172.21.15.5:6805/954535266","heartbeat_back_addr":"172.21.15.5:6809/954535266","heartbeat_front_addr":"172.21.15.5:6807/954535266","state":["exists","up"]},{"osd":1,"uuid":"0a76c795-2573-4a92-969a-2ec5dd65c094","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6800","nonce":733924375},{"type":"v1","addr":"172.21.15.102:6801","nonce":733924375}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6802","nonce":733924375},{"type":"v1","addr":"172.21.15.102:6803","nonce":733924375}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6806","nonce":733924375},{"type":"v1","addr":"172.21.15.102:6807","nonce":733924375}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6804","nonce":733924375},{"type":"v1","addr":"172.21.15.102:6805","nonce":733924375}]},"public_addr":"172.21.15.102:6801/733924375","cluster_addr":"172.21.15.102:6803/733924375","heartbeat_back_addr":"172.21.15.102:6807/733924375","heartbeat_front_addr":"172.21.15.102:6805/733924375","state":["exists","up"]},{"osd":2,"uuid":"0c98fcb8-60ab-4473-8995-1d9e97c577e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6800","nonce":2236204189},{"type":"v1","addr":"172.21.15.159:6801","nonce":2236204189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":2236204189},{"type":"v1","addr":"172.21.15.159:6803","nonce":2236204189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":2236204189},{"type":"v1","addr":"172.21.15.159:6807","nonce":2236204189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":2236204189},{"type":"v1","addr":"172.21.15.159:6805","nonce":2236204189}]},"public_addr":"172.21.15.159:6801/2236204189","cluster_addr":"172.21.15.159:6803/2236204189","heartbeat_back_addr":"172.21.15.159:6807/2236204189","heartbeat_front_addr":"172.21.15.159:6805/2236204189","state":["exists","up"]},{"osd":3,"uuid":"88eecdeb-5cee-4f78-a3f9-a66bf4bf772a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":2221468725},{"type":"v1","addr":"172.21.15.5:6811","nonce":2221468725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":2221468725},{"type":"v1","addr":"172.21.15.5:6813","nonce":2221468725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6816","nonce":2221468725},{"type":"v1","addr":"172.21.15.5:6817","nonce":2221468725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":2221468725},{"type":"v1","addr":"172.21.15.5:6815","nonce":2221468725}]},"public_addr":"172.21.15.5:6811/2221468725","cluster_addr":"172.21.15.5:6813/2221468725","heartbeat_back_addr":"172.21.15.5:6817/2221468725","heartbeat_front_addr":"172.21.15.5:6815/2221468725","state":["exists","up"]},{"osd":4,"uuid":"f2e7314d-7938-4668-9011-45c175f66b48","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6808","nonce":594465753},{"type":"v1","addr":"172.21.15.102:6809","nonce":594465753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6810","nonce":594465753},{"type":"v1","addr":"172.21.15.102:6811","nonce":594465753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6814","nonce":594465753},{"type":"v1","addr":"172.21.15.102:6815","nonce":594465753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6812","nonce":594465753},{"type":"v1","addr":"172.21.15.102:6813","nonce":594465753}]},"public_addr":"172.21.15.102:6809/594465753","cluster_addr":"172.21.15.102:6811/594465753","heartbeat_back_addr":"172.21.15.102:6815/594465753","heartbeat_front_addr":"172.21.15.102:6813/594465753","state":["exists","up"]},{"osd":5,"uuid":"0c0366d3-658b-4bf0-ba17-aff28a02e948","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":3230280402},{"type":"v1","addr":"172.21.15.159:6809","nonce":3230280402}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":3230280402},{"type":"v1","addr":"172.21.15.159:6811","nonce":3230280402}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":3230280402},{"type":"v1","addr":"172.21.15.159:6815","nonce":3230280402}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":3230280402},{"type":"v1","addr":"172.21.15.159:6813","nonce":3230280402}]},"public_addr":"172.21.15.159:6809/3230280402","cluster_addr":"172.21.15.159:6811/3230280402","heartbeat_back_addr":"172.21.15.159:6815/3230280402","heartbeat_front_addr":"172.21.15.159:6813/3230280402","state":["exists","up"]},{"osd":6,"uuid":"682670a6-51d5-46e8-af6e-dc2d8c1cf407","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":1907651140},{"type":"v1","addr":"172.21.15.5:6819","nonce":1907651140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":1907651140},{"type":"v1","addr":"172.21.15.5:6821","nonce":1907651140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":1907651140},{"type":"v1","addr":"172.21.15.5:6825","nonce":1907651140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":1907651140},{"type":"v1","addr":"172.21.15.5:6823","nonce":1907651140}]},"public_addr":"172.21.15.5:6819/1907651140","cluster_addr":"172.21.15.5:6821/1907651140","heartbeat_back_addr":"172.21.15.5:6825/1907651140","heartbeat_front_addr":"172.21.15.5:6823/1907651140","state":["exists","up"]},{"osd":7,"uuid":"8ae0f4a9-99e7-4563-aa2a-15169684a8b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6816","nonce":692040360},{"type":"v1","addr":"172.21.15.102:6817","nonce":692040360}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6818","nonce":692040360},{"type":"v1","addr":"172.21.15.102:6819","nonce":692040360}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6822","nonce":692040360},{"type":"v1","addr":"172.21.15.102:6823","nonce":692040360}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6820","nonce":692040360},{"type":"v1","addr":"172.21.15.102:6821","nonce":692040360}]},"public_addr":"172.21.15.102:6817/692040360","cluster_addr":"172.21.15.102:6819/692040360","heartbeat_back_addr":"172.21.15.102:6823/692040360","heartbeat_front_addr":"172.21.15.102:6821/692040360","state":["exists","up"]},{"osd":8,"uuid":"e37c845e-ced9-4413-9bbe-bc090537c2bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":2107291799},{"type":"v1","addr":"172.21.15.159:6817","nonce":2107291799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":2107291799},{"type":"v1","addr":"172.21.15.159:6819","nonce":2107291799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":2107291799},{"type":"v1","addr":"172.21.15.159:6823","nonce":2107291799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":2107291799},{"type":"v1","addr":"172.21.15.159:6821","nonce":2107291799}]},"public_addr":"172.21.15.159:6817/2107291799","cluster_addr":"172.21.15.159:6819/2107291799","heartbeat_back_addr":"172.21.15.159:6823/2107291799","heartbeat_front_addr":"172.21.15.159:6821/2107291799","state":["exists","up"]},{"osd":9,"uuid":"91a5b6cc-08da-47be-af12-d0896a849309","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":902431281},{"type":"v1","addr":"172.21.15.5:6827","nonce":902431281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":902431281},{"type":"v1","addr":"172.21.15.5:6829","nonce":902431281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":902431281},{"type":"v1","addr":"172.21.15.5:6833","nonce":902431281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":902431281},{"type":"v1","addr":"172.21.15.5:6831","nonce":902431281}]},"public_addr":"172.21.15.5:6827/902431281","cluster_addr":"172.21.15.5:6829/902431281","heartbeat_back_addr":"172.21.15.5:6833/902431281","heartbeat_front_addr":"172.21.15.5:6831/902431281","state":["exists","up"]},{"osd":10,"uuid":"2211030c-b00e-402b-b4d7-c587eb29a490","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6824","nonce":2853806211},{"type":"v1","addr":"172.21.15.102:6825","nonce":2853806211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6826","nonce":2853806211},{"type":"v1","addr":"172.21.15.102:6827","nonce":2853806211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6830","nonce":2853806211},{"type":"v1","addr":"172.21.15.102:6831","nonce":2853806211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6828","nonce":2853806211},{"type":"v1","addr":"172.21.15.102:6829","nonce":2853806211}]},"public_addr":"172.21.15.102:6825/2853806211","cluster_addr":"172.21.15.102:6827/2853806211","heartbeat_back_addr":"172.21.15.102:6831/2853806211","heartbeat_front_addr":"172.21.15.102:6829/2853806211","state":["exists","up"]},{"osd":11,"uuid":"4e9699ec-8ff2-4005-9d84-af5600b3a2ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6824","nonce":969941218},{"type":"v1","addr":"172.21.15.159:6825","nonce":969941218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":969941218},{"type":"v1","addr":"172.21.15.159:6827","nonce":969941218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6830","nonce":969941218},{"type":"v1","addr":"172.21.15.159:6831","nonce":969941218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":969941218},{"type":"v1","addr":"172.21.15.159:6829","nonce":969941218}]},"public_addr":"172.21.15.159:6825/969941218","cluster_addr":"172.21.15.159:6827/969941218","heartbeat_back_addr":"172.21.15.159:6831/969941218","heartbeat_front_addr":"172.21.15.159:6829/969941218","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:29:34.416490+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:30:00.246101+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:30:24.300464+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:30:54.663623+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:31:21.426292+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:31:46.738914+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:32:16.196571+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:32:42.549243+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:33:08.178651+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:33:37.477911+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:34:03.114026+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:34:28.891924+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4}]},{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"3.0","mappings":[{"from":4,"to":8}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":6}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9},{"from":5,"to":7}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]},{"pgid":"3.39","mappings":[{"from":5,"to":6}]},{"pgid":"4.8","mappings":[{"from":5,"to":8}]},{"pgid":"4.c","mappings":[{"from":9,"to":10}]},{"pgid":"4.18","mappings":[{"from":9,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.5:6801/1527422686":"2024-09-11T09:28:05.436742+0000","172.21.15.5:0/2276144345":"2024-09-11T09:28:05.436742+0000","172.21.15.5:0/412040104":"2024-09-11T09:28:05.436742+0000","172.21.15.5:6800/1527422686":"2024-09-11T09:28:05.436742+0000","172.21.15.5:0/1856581170":"2024-09-11T09:28:05.436742+0000","172.21.15.5:0/2847668298":"2024-09-11T09:27:37.778500+0000","172.21.15.5:6801/14425913":"2024-09-11T09:27:37.778500+0000","172.21.15.5:0/166910782":"2024-09-11T09:27:37.778500+0000","172.21.15.5:6800/14425913":"2024-09-11T09:27:37.778500+0000","172.21.15.5:0/1711858179":"2024-09-11T09:27:37.778500+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T09:37:59.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:59.715+0000 7fdbd8600640 1 -- 172.21.15.5:0/520518990 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fdbb80a2c00 msgr2=0x7fdbb80a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:37:59.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:59.715+0000 7fdbd8600640 1 --2- 172.21.15.5:0/520518990 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fdbb80a2c00 0x7fdbb80a50c0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fdbc4011550 tx=0x7fdbc4005f90 comp rx=0 tx=0).stop 2024-09-10T09:37:59.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:59.715+0000 7fdbd8600640 1 -- 172.21.15.5:0/520518990 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdbd0074ee0 msgr2=0x7fdbd019ab80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:37:59.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:59.715+0000 7fdbd8600640 1 --2- 172.21.15.5:0/520518990 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdbd0074ee0 0x7fdbd019ab80 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fdbbc005940 tx=0x7fdbbc0040c0 comp rx=0 tx=0).stop 2024-09-10T09:37:59.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:59.715+0000 7fdbd8600640 1 -- 172.21.15.5:0/520518990 shutdown_connections 2024-09-10T09:37:59.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:59.715+0000 7fdbd8600640 1 --2- 172.21.15.5:0/520518990 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fdbb80a2c00 0x7fdbb80a50c0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:59.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:59.715+0000 7fdbd8600640 1 --2- 172.21.15.5:0/520518990 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdbd010e430 0x7fdbd019ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:59.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:59.715+0000 7fdbd8600640 1 --2- 172.21.15.5:0/520518990 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdbd0074ee0 0x7fdbd019ab80 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:59.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:59.715+0000 7fdbd8600640 1 --2- 172.21.15.5:0/520518990 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdbd00745c0 0x7fdbd019a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:37:59.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:59.715+0000 7fdbd8600640 1 -- 172.21.15.5:0/520518990 >> 172.21.15.5:0/520518990 conn(0x7fdbd00fd8b0 msgr2=0x7fdbd00fe510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:37:59.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:59.715+0000 7fdbd8600640 1 -- 172.21.15.5:0/520518990 shutdown_connections 2024-09-10T09:37:59.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:37:59.715+0000 7fdbd8600640 1 -- 172.21.15.5:0/520518990 wait complete. 2024-09-10T09:38:00.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:37:59 smithi005 ceph-mon[30350]: from='client.? 172.21.15.5:0/520518990' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T09:38:00.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:37:59 smithi159 ceph-mon[35513]: from='client.? 172.21.15.5:0/520518990' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T09:38:00.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:37:59 smithi102 ceph-mon[35442]: from='client.? 172.21.15.5:0/520518990' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T09:38:00.513 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 29, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-09-10T09:35:29.626287+0000', 'modified': '2024-09-10T09:36:15.764196+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24542, 'mds_1': 24535, 'mds_2': 24511, 'mds_3': 24506, 'mds_4': 24443}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14520': {'gid': 14520, 'name': 'a', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.5:6841/2073599842', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.5:6840', 'nonce': 2073599842}, {'type': 'v1', 'addr': '172.21.15.5:6841', 'nonce': 2073599842}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24443': {'gid': 24443, 'name': 'k', 'rank': 4, 'incarnation': 26, 'state': 'up:active', 'state_seq': 24, 'addr': '172.21.15.102:6833/2457007427', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.102:6832', 'nonce': 2457007427}, {'type': 'v1', 'addr': '172.21.15.102:6833', 'nonce': 2457007427}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24460': {'gid': 24460, 'name': 'l', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.159:6833/356445159', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.159:6832', 'nonce': 356445159}, {'type': 'v1', 'addr': '172.21.15.159:6833', 'nonce': 356445159}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24467': {'gid': 24467, 'name': 'd', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.5:6837/2894314507', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.5:6836', 'nonce': 2894314507}, {'type': 'v1', 'addr': '172.21.15.5:6837', 'nonce': 2894314507}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24479': {'gid': 24479, 'name': 'e', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.102:6835/1647549111', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.102:6834', 'nonce': 1647549111}, {'type': 'v1', 'addr': '172.21.15.102:6835', 'nonce': 1647549111}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24505': {'gid': 24505, 'name': 'f', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.159:6835/952657242', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.159:6834', 'nonce': 952657242}, {'type': 'v1', 'addr': '172.21.15.159:6835', 'nonce': 952657242}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24506': {'gid': 24506, 'name': 'h', 'rank': 3, 'incarnation': 24, 'state': 'up:active', 'state_seq': 19, 'addr': '172.21.15.102:6837/2689776764', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.102:6836', 'nonce': 2689776764}, {'type': 'v1', 'addr': '172.21.15.102:6837', 'nonce': 2689776764}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24511': {'gid': 24511, 'name': 'c', 'rank': 2, 'incarnation': 22, 'state': 'up:active', 'state_seq': 19, 'addr': '172.21.15.159:6837/620627639', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.159:6836', 'nonce': 620627639}, {'type': 'v1', 'addr': '172.21.15.159:6837', 'nonce': 620627639}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24535': {'gid': 24535, 'name': 'i', 'rank': 1, 'incarnation': 20, 'state': 'up:active', 'state_seq': 18, 'addr': '172.21.15.159:6839/2073739599', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.159:6838', 'nonce': 2073739599}, {'type': 'v1', 'addr': '172.21.15.159:6839', 'nonce': 2073739599}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24542': {'gid': 24542, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.102:6839/335409749', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.102:6838', 'nonce': 335409749}, {'type': 'v1', 'addr': '172.21.15.102:6839', 'nonce': 335409749}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24542, 'qdb_cluster': [24443, 24506, 24511, 24535, 24542]} 2024-09-10T09:38:00.513 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-09-10T09:38:00.513 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph tell mds.k status 2024-09-10T09:38:00.897 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:38:01.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:38:00 smithi005 ceph-mon[30350]: pgmap v383: 129 pgs: 129 active+clean; 646 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-09-10T09:38:01.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:38:00 smithi159 ceph-mon[35513]: pgmap v383: 129 pgs: 129 active+clean; 646 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-09-10T09:38:01.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:38:00 smithi102 ceph-mon[35442]: pgmap v383: 129 pgs: 129 active+clean; 646 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-09-10T09:38:01.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:01.573+0000 7f1c9f200640 1 -- 172.21.15.5:0/41557198 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1ca0077040 msgr2=0x7f1ca0112330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:38:01.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:01.573+0000 7f1c9f200640 1 --2- 172.21.15.5:0/41557198 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1ca0077040 0x7f1ca0112330 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f1c8c009900 tx=0x7f1c8c02a410 comp rx=0 tx=0).stop 2024-09-10T09:38:01.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:01.573+0000 7f1c9f200640 1 -- 172.21.15.5:0/41557198 shutdown_connections 2024-09-10T09:38:01.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:01.573+0000 7f1c9f200640 1 --2- 172.21.15.5:0/41557198 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1ca0112870 0x7f1ca0114c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:01.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:01.573+0000 7f1c9f200640 1 --2- 172.21.15.5:0/41557198 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1ca0077040 0x7f1ca0112330 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:01.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:01.573+0000 7f1c9f200640 1 --2- 172.21.15.5:0/41557198 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1ca0076720 0x7f1ca0076b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:01.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:01.573+0000 7f1c9f200640 1 -- 172.21.15.5:0/41557198 >> 172.21.15.5:0/41557198 conn(0x7f1ca00ff910 msgr2=0x7f1ca0101d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:38:01.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:01.573+0000 7f1c9f200640 1 -- 172.21.15.5:0/41557198 shutdown_connections 2024-09-10T09:38:01.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:01.573+0000 7f1c9f200640 1 -- 172.21.15.5:0/41557198 wait complete. 2024-09-10T09:38:01.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:01.574+0000 7f1c9f200640 1 Processor -- start 2024-09-10T09:38:01.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:01.574+0000 7f1c9f200640 1 -- start start 2024-09-10T09:38:01.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:01.575+0000 7f1c9f200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1ca0076720 0x7f1ca01a4640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:01.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:01.575+0000 7f1c9f200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1ca0077040 0x7f1ca01a4b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:01.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:01.575+0000 7f1c9f200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1ca0112870 0x7f1ca019e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:01.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:01.575+0000 7f1c9f200640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1ca0118450 con 0x7f1ca0076720 2024-09-10T09:38:01.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:01.575+0000 7f1c9f200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f1ca01182d0 con 0x7f1ca0112870 2024-09-10T09:38:01.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:01.575+0000 7f1c9f200640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f1ca01185d0 con 0x7f1ca0077040 2024-09-10T09:38:01.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:01.575+0000 7f1c9de00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1ca0076720 0x7f1ca01a4640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:01.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:01.575+0000 7f1c9d400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1ca0077040 0x7f1ca01a4b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:01.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:01.575+0000 7f1c9d400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1ca0077040 0x7f1ca01a4b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:46308/0 (socket says 172.21.15.5:46308) 2024-09-10T09:38:01.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:01.575+0000 7f1c9d400640 1 -- 172.21.15.5:0/4100006579 learned_addr learned my addr 172.21.15.5:0/4100006579 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:38:01.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:01.575+0000 7f1c9e800640 1 --2- 172.21.15.5:0/4100006579 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1ca0112870 0x7f1ca019e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:01.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:01.576+0000 7f1c9d400640 1 -- 172.21.15.5:0/4100006579 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1ca0112870 msgr2=0x7f1ca019e7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:38:01.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:01.576+0000 7f1c9d400640 1 --2- 172.21.15.5:0/4100006579 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1ca0112870 0x7f1ca019e7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:01.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:01.576+0000 7f1c9d400640 1 -- 172.21.15.5:0/4100006579 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1ca0076720 msgr2=0x7f1ca01a4640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:38:01.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:01.576+0000 7f1c9d400640 1 --2- 172.21.15.5:0/4100006579 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1ca0076720 0x7f1ca01a4640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:01.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:01.576+0000 7f1c9d400640 1 -- 172.21.15.5:0/4100006579 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ca019ef60 con 0x7f1ca0077040 2024-09-10T09:38:01.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:01.576+0000 7f1c9e800640 1 --2- 172.21.15.5:0/4100006579 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1ca0112870 0x7f1ca019e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:38:01.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:01.576+0000 7f1c9d400640 1 --2- 172.21.15.5:0/4100006579 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1ca0077040 0x7f1ca01a4b80 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f1c8c004770 tx=0x7f1c8c003c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:38:01.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:01.576+0000 7f1c96a00640 1 -- 172.21.15.5:0/4100006579 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c8c03f070 con 0x7f1ca0077040 2024-09-10T09:38:01.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:01.576+0000 7f1c9f200640 1 -- 172.21.15.5:0/4100006579 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ca019f1f0 con 0x7f1ca0077040 2024-09-10T09:38:01.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:01.576+0000 7f1c96a00640 1 -- 172.21.15.5:0/4100006579 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1c8c031ca0 con 0x7f1ca0077040 2024-09-10T09:38:01.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:01.576+0000 7f1c96a00640 1 -- 172.21.15.5:0/4100006579 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c8c035030 con 0x7f1ca0077040 2024-09-10T09:38:01.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:01.577+0000 7f1c9f200640 1 -- 172.21.15.5:0/4100006579 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ca01ab4a0 con 0x7f1ca0077040 2024-09-10T09:38:01.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:01.579+0000 7f1ca7645740 1 Processor -- start 2024-09-10T09:38:01.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:01.580+0000 7f1ca7645740 1 -- start start 2024-09-10T09:38:01.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:01.580+0000 7f1c96a00640 1 -- 172.21.15.5:0/4100006579 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f1c8c042070 con 0x7f1ca0077040 2024-09-10T09:38:01.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:01.580+0000 7f1ca7645740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555d2b6214e0 0x555d2b6218c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:01.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:01.580+0000 7f1ca7645740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555d2b621e00 0x555d2b62c180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:01.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:01.580+0000 7f1ca7645740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555d2b62c6c0 0x555d2b62eab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:01.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:01.580+0000 7f1ca7645740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x555d2b5870b0 con 0x555d2b62c6c0 2024-09-10T09:38:01.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:01.580+0000 7f1ca7645740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x555d2b52eab0 con 0x555d2b6214e0 2024-09-10T09:38:01.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:01.580+0000 7f1ca7645740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x555d2b583070 con 0x555d2b621e00 2024-09-10T09:38:01.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:01.580+0000 7f1c9d400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555d2b62c6c0 0x555d2b62eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:01.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:01.580+0000 7f1c9de00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555d2b621e00 0x555d2b62c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:01.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:01.580+0000 7f1c9e800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555d2b6214e0 0x555d2b6218c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:01.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:01.580+0000 7f1c9de00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555d2b621e00 0x555d2b62c180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:46322/0 (socket says 172.21.15.5:46322) 2024-09-10T09:38:01.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:01.581+0000 7f1c9de00640 1 -- 172.21.15.5:0/1169776932 learned_addr learned my addr 172.21.15.5:0/1169776932 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:38:01.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:01.581+0000 7f1c9de00640 1 -- 172.21.15.5:0/1169776932 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555d2b6214e0 msgr2=0x555d2b6218c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:38:01.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:01.581+0000 7f1c9de00640 1 --2- 172.21.15.5:0/1169776932 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555d2b6214e0 0x555d2b6218c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:01.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:01.581+0000 7f1c9de00640 1 -- 172.21.15.5:0/1169776932 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555d2b62c6c0 msgr2=0x555d2b62eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:38:01.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:01.581+0000 7f1c9de00640 1 --2- 172.21.15.5:0/1169776932 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555d2b62c6c0 0x555d2b62eab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:01.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:01.581+0000 7f1c9de00640 1 -- 172.21.15.5:0/1169776932 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555d2b53b4a0 con 0x555d2b621e00 2024-09-10T09:38:01.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:01.581+0000 7f1c9e800640 1 --2- 172.21.15.5:0/1169776932 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555d2b6214e0 0x555d2b6218c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:38:01.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:01.581+0000 7f1c9d400640 1 --2- 172.21.15.5:0/1169776932 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555d2b62c6c0 0x555d2b62eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:38:01.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:01.581+0000 7f1c9de00640 1 --2- 172.21.15.5:0/1169776932 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555d2b621e00 0x555d2b62c180 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f1c90009d20 tx=0x7f1c90011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1992a7436ffc3fe5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:38:01.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:01.582+0000 7f1c8b000640 1 -- 172.21.15.5:0/1169776932 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c9001e070 con 0x555d2b621e00 2024-09-10T09:38:01.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:01.582+0000 7f1c96a00640 1 --2- 172.21.15.5:0/4100006579 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f1c840a2c00 0x7f1c840a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:01.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:01.582+0000 7f1c8b000640 1 -- 172.21.15.5:0/1169776932 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1c90019b10 con 0x555d2b621e00 2024-09-10T09:38:01.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:01.582+0000 7f1c8b000640 1 -- 172.21.15.5:0/1169776932 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c9000ebf0 con 0x555d2b621e00 2024-09-10T09:38:01.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:01.582+0000 7f1c96a00640 1 -- 172.21.15.5:0/4100006579 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f1c8c0ef670 con 0x7f1ca0077040 2024-09-10T09:38:01.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:01.582+0000 7f1c9e800640 1 --2- 172.21.15.5:0/4100006579 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f1c840a2c00 0x7f1c840a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:01.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:01.582+0000 7f1ca7645740 1 -- 172.21.15.5:0/1169776932 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555d2b621e00 msgr2=0x555d2b62c180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:38:01.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:01.582+0000 7f1ca7645740 1 --2- 172.21.15.5:0/1169776932 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555d2b621e00 0x555d2b62c180 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f1c90009d20 tx=0x7f1c90011810 comp rx=0 tx=0).stop 2024-09-10T09:38:01.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:01.583+0000 7f1ca7645740 1 -- 172.21.15.5:0/1169776932 shutdown_connections 2024-09-10T09:38:01.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:01.583+0000 7f1ca7645740 1 --2- 172.21.15.5:0/1169776932 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555d2b62c6c0 0x555d2b62eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:01.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:01.583+0000 7f1ca7645740 1 --2- 172.21.15.5:0/1169776932 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555d2b621e00 0x555d2b62c180 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:01.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:01.583+0000 7f1ca7645740 1 --2- 172.21.15.5:0/1169776932 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555d2b6214e0 0x555d2b6218c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:01.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:01.583+0000 7f1ca7645740 1 -- 172.21.15.5:0/1169776932 >> 172.21.15.5:0/1169776932 conn(0x555d2b61da70 msgr2=0x555d2b61de50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:38:01.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:01.583+0000 7f1c9e800640 1 --2- 172.21.15.5:0/4100006579 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f1c840a2c00 0x7f1c840a50c0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f1c98009fd0 tx=0x7f1c98009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:38:01.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:01.583+0000 7f1ca7645740 1 -- 172.21.15.5:0/1169776932 shutdown_connections 2024-09-10T09:38:01.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:01.583+0000 7f1ca7645740 1 -- 172.21.15.5:0/1169776932 wait complete. 2024-09-10T09:38:01.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:01.583+0000 7f1ca7645740 1 -- start start 2024-09-10T09:38:01.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:01.583+0000 7f1ca7645740 1 Processor -- start 2024-09-10T09:38:01.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:01.584+0000 7f1ca7645740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555d2b6214e0 0x555d2b6295d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:01.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:01.584+0000 7f1ca7645740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555d2b621e00 0x555d2b629b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:01.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:01.584+0000 7f1ca7645740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555d2b62c6c0 0x555d2b62a050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:01.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:01.584+0000 7f1ca7645740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x555d2b62f330 con 0x555d2b62c6c0 2024-09-10T09:38:01.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:01.584+0000 7f1ca7645740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x555d2b50f920 con 0x555d2b6214e0 2024-09-10T09:38:01.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:01.584+0000 7f1ca7645740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x555d2b62f4b0 con 0x555d2b621e00 2024-09-10T09:38:01.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:01.584+0000 7f1ca7645740 10 client.0 _ll_register_callbacks cb 0x555d2b5338a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:38:01.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:01.584+0000 7f1c9de00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555d2b6214e0 0x555d2b6295d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:01.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:01.584+0000 7f1c9e800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555d2b62c6c0 0x555d2b62a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:01.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:01.584+0000 7f1c9de00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555d2b6214e0 0x555d2b6295d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:44992/0 (socket says 172.21.15.5:44992) 2024-09-10T09:38:01.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:01.584+0000 7f1c9de00640 1 -- 172.21.15.5:0/191054825 learned_addr learned my addr 172.21.15.5:0/191054825 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:38:01.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:01.584+0000 7f1c9d400640 1 --2- 172.21.15.5:0/191054825 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555d2b621e00 0x555d2b629b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:01.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:01.585+0000 7f1c9de00640 1 -- 172.21.15.5:0/191054825 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555d2b621e00 msgr2=0x555d2b629b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:38:01.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:01.585+0000 7f1c9de00640 1 --2- 172.21.15.5:0/191054825 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555d2b621e00 0x555d2b629b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:01.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:01.585+0000 7f1c9de00640 1 -- 172.21.15.5:0/191054825 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555d2b62c6c0 msgr2=0x555d2b62a050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:38:01.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:01.585+0000 7f1c9de00640 1 --2- 172.21.15.5:0/191054825 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555d2b62c6c0 0x555d2b62a050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:01.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:01.585+0000 7f1c9de00640 1 -- 172.21.15.5:0/191054825 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555d2b2f82b0 con 0x555d2b6214e0 2024-09-10T09:38:01.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:01.585+0000 7f1c9d400640 1 --2- 172.21.15.5:0/191054825 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555d2b621e00 0x555d2b629b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:38:01.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:01.585+0000 7f1c9de00640 1 --2- 172.21.15.5:0/191054825 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555d2b6214e0 0x555d2b6295d0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f1c90019660 tx=0x7f1c90019690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1093a43757b63133 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:38:01.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:01.585+0000 7f1c7f400640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-09-10T09:38:01.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:01.585+0000 7f1c7f400640 1 -- 172.21.15.5:0/191054825 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c9001e070 con 0x555d2b6214e0 2024-09-10T09:38:01.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:01.585+0000 7f1ca7645740 1 -- 172.21.15.5:0/191054825 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1c90019b10 con 0x555d2b6214e0 2024-09-10T09:38:01.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:01.585+0000 7f1c7f400640 1 -- 172.21.15.5:0/191054825 <== mon.2 v2:172.21.15.102:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f1c9000eaf0 con 0x555d2b6214e0 2024-09-10T09:38:01.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:01.586+0000 7f1c7f400640 1 -- 172.21.15.5:0/191054825 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f1c90031050 con 0x555d2b6214e0 2024-09-10T09:38:01.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:01.586+0000 7f1ca7645740 10 client.25124 fetch_fsmap learned FSMap version 29 2024-09-10T09:38:01.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:01.586+0000 7f1ca7645740 1 -- 172.21.15.5:0/191054825 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x555d2b63b010 con 0x555d2b6214e0 2024-09-10T09:38:01.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:01.586+0000 7f1c7f400640 1 -- 172.21.15.5:0/191054825 <== mon.2 v2:172.21.15.102:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1c90019b10 con 0x555d2b6214e0 2024-09-10T09:38:01.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:01.586+0000 7f1c7f400640 1 -- 172.21.15.5:0/191054825 <== mon.2 v2:172.21.15.102:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f1c900046c0 con 0x555d2b6214e0 2024-09-10T09:38:01.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:01.586+0000 7f1ca7645740 10 client.25124 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T09:38:01.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:01.586+0000 7f1ca7645740 10 client.25124 resolve_mds: resolved name 'k' to daemon mds.k 2024-09-10T09:38:01.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:01.586+0000 7f1ca7645740 20 client.25124 populate_metadata read hostname 'smithi005' 2024-09-10T09:38:01.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:01.586+0000 7f1ca7645740 1 --2- 172.21.15.5:0/191054825 >> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] conn(0x555d2b63d420 0x555d2b63e890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:01.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:01.586+0000 7f1ca7645740 4 client.25124 mds_command: new command op to 24443 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:38:01.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:01.586+0000 7f1ca7645740 1 -- 172.21.15.5:0/191054825 --> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555d2b594620 con 0x555d2b63d420 2024-09-10T09:38:01.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:01.587+0000 7f1c9d400640 1 --2- 172.21.15.5:0/191054825 >> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] conn(0x555d2b63d420 0x555d2b63e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:01.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:01.587+0000 7f1c9d400640 1 --2- 172.21.15.5:0/191054825 >> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] conn(0x555d2b63d420 0x555d2b63e890 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=4996198d94781d7b server_cookie=50b7a466ce37942b in_seq=0 out_seq=0 2024-09-10T09:38:01.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:01.587+0000 7f1c7f400640 10 client.25124 ms_handle_connect on v2:172.21.15.102:6832/2457007427 2024-09-10T09:38:01.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:01.588+0000 7f1c7f400640 1 -- 172.21.15.5:0/191054825 <== mds.4 v2:172.21.15.102:6832/2457007427 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555d2b594620 con 0x555d2b63d420 2024-09-10T09:38:01.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:01.588+0000 7f1c7f400640 10 client.25124 handle_command_reply: tid=0 2024-09-10T09:38:01.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:01.588+0000 7f1ca7645740 1 client.25124 shutdown 2024-09-10T09:38:01.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:01.588+0000 7f1ca7645740 1 -- 172.21.15.5:0/191054825 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555d2b6214e0 msgr2=0x555d2b6295d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:38:01.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:01.588+0000 7f1ca7645740 1 --2- 172.21.15.5:0/191054825 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555d2b6214e0 0x555d2b6295d0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f1c90019660 tx=0x7f1c90019690 comp rx=0 tx=0).stop 2024-09-10T09:38:01.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:01.588+0000 7f1ca7645740 1 -- 172.21.15.5:0/191054825 shutdown_connections 2024-09-10T09:38:01.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:01.588+0000 7f1ca7645740 1 --2- 172.21.15.5:0/191054825 >> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] conn(0x555d2b63d420 0x555d2b63e890 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:01.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:01.588+0000 7f1ca7645740 1 --2- 172.21.15.5:0/191054825 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555d2b62c6c0 0x555d2b62a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:01.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:01.588+0000 7f1ca7645740 1 --2- 172.21.15.5:0/191054825 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555d2b621e00 0x555d2b629b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:01.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:01.588+0000 7f1c7f400640 0 client.25124 ms_handle_reset on v2:172.21.15.102:6832/2457007427 2024-09-10T09:38:01.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:01.588+0000 7f1ca7645740 1 --2- 172.21.15.5:0/191054825 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555d2b6214e0 0x555d2b6295d0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:01.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:01.588+0000 7f1ca7645740 1 -- 172.21.15.5:0/191054825 >> 172.21.15.5:0/191054825 conn(0x555d2b61da70 msgr2=0x555d2b623750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:38:01.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:01.589+0000 7f1ca7645740 1 -- 172.21.15.5:0/191054825 shutdown_connections 2024-09-10T09:38:01.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:01.589+0000 7f1ca7645740 1 -- 172.21.15.5:0/191054825 wait complete. 2024-09-10T09:38:01.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:01.589+0000 7f1ca7645740 20 client.25124 trim_cache size 0 max 16384 2024-09-10T09:38:01.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:01.607+0000 7f1ca7645740 1 Processor -- start 2024-09-10T09:38:01.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:01.607+0000 7f1ca7645740 1 -- start start 2024-09-10T09:38:01.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:01.608+0000 7f1ca7645740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555d2b6214e0 0x555d2b626830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:01.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:01.608+0000 7f1ca7645740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555d2b621e00 0x555d2b626d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:01.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:01.608+0000 7f1ca7645740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555d2b62c6c0 0x555d2b61c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:01.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:01.608+0000 7f1ca7645740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x555d2b63cbd0 con 0x555d2b62c6c0 2024-09-10T09:38:01.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:01.608+0000 7f1ca7645740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x555d2b63d2a0 con 0x555d2b6214e0 2024-09-10T09:38:01.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:01.608+0000 7f1ca7645740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x555d2b63d970 con 0x555d2b621e00 2024-09-10T09:38:01.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:01.608+0000 7f1c9e800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555d2b62c6c0 0x555d2b61c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:01.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:01.608+0000 7f1c9e800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555d2b62c6c0 0x555d2b61c3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51822/0 (socket says 172.21.15.5:51822) 2024-09-10T09:38:01.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:01.608+0000 7f1c9e800640 1 -- 172.21.15.5:0/3818594235 learned_addr learned my addr 172.21.15.5:0/3818594235 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:38:01.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:01.608+0000 7f1c9de00640 1 --2- 172.21.15.5:0/3818594235 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555d2b6214e0 0x555d2b626830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:01.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:01.608+0000 7f1c9d400640 1 --2- 172.21.15.5:0/3818594235 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555d2b621e00 0x555d2b626d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:01.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:01.608+0000 7f1c9de00640 1 -- 172.21.15.5:0/3818594235 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555d2b621e00 msgr2=0x555d2b626d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:38:01.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:01.609+0000 7f1c9de00640 1 --2- 172.21.15.5:0/3818594235 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555d2b621e00 0x555d2b626d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:01.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:01.609+0000 7f1c9de00640 1 -- 172.21.15.5:0/3818594235 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555d2b62c6c0 msgr2=0x555d2b61c3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:38:01.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:01.609+0000 7f1c9de00640 1 --2- 172.21.15.5:0/3818594235 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555d2b62c6c0 0x555d2b61c3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:01.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:01.609+0000 7f1c9de00640 1 -- 172.21.15.5:0/3818594235 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555d2b622330 con 0x555d2b6214e0 2024-09-10T09:38:01.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:01.609+0000 7f1c9d400640 1 --2- 172.21.15.5:0/3818594235 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555d2b621e00 0x555d2b626d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:38:01.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:01.609+0000 7f1c9de00640 1 --2- 172.21.15.5:0/3818594235 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555d2b6214e0 0x555d2b626830 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f1c90011850 tx=0x7f1c900059b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6a212aea268ddabf server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:38:01.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:01.609+0000 7f1c8a600640 1 -- 172.21.15.5:0/3818594235 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c9001e040 con 0x555d2b6214e0 2024-09-10T09:38:01.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:01.609+0000 7f1c8a600640 1 -- 172.21.15.5:0/3818594235 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1c90028070 con 0x555d2b6214e0 2024-09-10T09:38:01.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:01.609+0000 7f1c8a600640 1 -- 172.21.15.5:0/3818594235 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c9000ee90 con 0x555d2b6214e0 2024-09-10T09:38:01.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:01.610+0000 7f1ca7645740 1 -- 172.21.15.5:0/3818594235 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555d2b6214e0 msgr2=0x555d2b626830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:38:01.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:01.610+0000 7f1ca7645740 1 --2- 172.21.15.5:0/3818594235 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555d2b6214e0 0x555d2b626830 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f1c90011850 tx=0x7f1c900059b0 comp rx=0 tx=0).stop 2024-09-10T09:38:01.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:01.610+0000 7f1ca7645740 1 -- 172.21.15.5:0/3818594235 shutdown_connections 2024-09-10T09:38:01.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:01.610+0000 7f1ca7645740 1 --2- 172.21.15.5:0/3818594235 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555d2b62c6c0 0x555d2b61c3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:01.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:01.610+0000 7f1ca7645740 1 --2- 172.21.15.5:0/3818594235 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555d2b621e00 0x555d2b626d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:01.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:01.610+0000 7f1ca7645740 1 --2- 172.21.15.5:0/3818594235 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555d2b6214e0 0x555d2b626830 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:01.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:01.610+0000 7f1ca7645740 1 -- 172.21.15.5:0/3818594235 >> 172.21.15.5:0/3818594235 conn(0x555d2b61da70 msgr2=0x555d2b62b650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:38:01.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:01.610+0000 7f1ca7645740 1 -- 172.21.15.5:0/3818594235 shutdown_connections 2024-09-10T09:38:01.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:01.610+0000 7f1ca7645740 1 -- 172.21.15.5:0/3818594235 wait complete. 2024-09-10T09:38:01.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:01.610+0000 7f1ca7645740 1 -- start start 2024-09-10T09:38:01.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:01.611+0000 7f1ca7645740 1 Processor -- start 2024-09-10T09:38:01.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:01.611+0000 7f1ca7645740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555d2b6214e0 0x555d2b6470b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:01.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:01.611+0000 7f1ca7645740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555d2b621e00 0x555d2b6423e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:01.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:01.611+0000 7f1ca7645740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555d2b62c6c0 0x555d2b642920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:01.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:01.611+0000 7f1ca7645740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x555d2b63e710 con 0x555d2b621e00 2024-09-10T09:38:01.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:01.611+0000 7f1ca7645740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x555d2b63e040 con 0x555d2b62c6c0 2024-09-10T09:38:01.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:01.611+0000 7f1ca7645740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x555d2b63ede0 con 0x555d2b6214e0 2024-09-10T09:38:01.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:01.611+0000 7f1ca7645740 10 client.0 _ll_register_callbacks cb 0x7f1c98009ee0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:38:01.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:01.611+0000 7f1c9e800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555d2b62c6c0 0x555d2b642920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:01.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:01.611+0000 7f1c9e800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555d2b62c6c0 0x555d2b642920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:45016/0 (socket says 172.21.15.5:45016) 2024-09-10T09:38:01.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:01.611+0000 7f1c9e800640 1 -- 172.21.15.5:0/2824295571 learned_addr learned my addr 172.21.15.5:0/2824295571 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:38:01.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:01.611+0000 7f1c9d400640 1 --2- 172.21.15.5:0/2824295571 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555d2b621e00 0x555d2b6423e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:01.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:01.611+0000 7f1c9de00640 1 --2- 172.21.15.5:0/2824295571 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555d2b6214e0 0x555d2b6470b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:01.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:01.612+0000 7f1c9e800640 1 -- 172.21.15.5:0/2824295571 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555d2b6214e0 msgr2=0x555d2b6470b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:38:01.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:01.612+0000 7f1c9e800640 1 --2- 172.21.15.5:0/2824295571 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555d2b6214e0 0x555d2b6470b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:01.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:01.612+0000 7f1c9e800640 1 -- 172.21.15.5:0/2824295571 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555d2b621e00 msgr2=0x555d2b6423e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:38:01.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:01.612+0000 7f1c9e800640 1 --2- 172.21.15.5:0/2824295571 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555d2b621e00 0x555d2b6423e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:01.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:01.612+0000 7f1c9e800640 1 -- 172.21.15.5:0/2824295571 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555d2b642f60 con 0x555d2b62c6c0 2024-09-10T09:38:01.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:01.612+0000 7f1c9e800640 1 --2- 172.21.15.5:0/2824295571 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555d2b62c6c0 0x555d2b642920 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f1c98012290 tx=0x7f1c98004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cba3adb41f36b367 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:38:01.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:01.612+0000 7f1c7f400640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-09-10T09:38:01.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:01.612+0000 7f1c7f400640 1 -- 172.21.15.5:0/2824295571 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c9802f060 con 0x555d2b62c6c0 2024-09-10T09:38:01.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:01.612+0000 7f1ca7645740 1 -- 172.21.15.5:0/2824295571 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1c90028070 con 0x555d2b62c6c0 2024-09-10T09:38:01.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:01.613+0000 7f1c7f400640 1 -- 172.21.15.5:0/2824295571 <== mon.2 v2:172.21.15.102:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f1c98011bd0 con 0x555d2b62c6c0 2024-09-10T09:38:01.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:01.613+0000 7f1c7f400640 1 -- 172.21.15.5:0/2824295571 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f1c98036050 con 0x555d2b62c6c0 2024-09-10T09:38:01.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:01.613+0000 7f1ca7645740 10 client.25130 fetch_fsmap learned FSMap version 29 2024-09-10T09:38:01.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:01.613+0000 7f1ca7645740 1 -- 172.21.15.5:0/2824295571 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x555d2b6a79f0 con 0x555d2b62c6c0 2024-09-10T09:38:01.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:01.613+0000 7f1c7f400640 1 -- 172.21.15.5:0/2824295571 <== mon.2 v2:172.21.15.102:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1c90028070 con 0x555d2b62c6c0 2024-09-10T09:38:01.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:01.613+0000 7f1c7f400640 1 -- 172.21.15.5:0/2824295571 <== mon.2 v2:172.21.15.102:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f1c9801e410 con 0x555d2b62c6c0 2024-09-10T09:38:01.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:01.613+0000 7f1ca7645740 10 client.25130 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T09:38:01.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:01.614+0000 7f1ca7645740 10 client.25130 resolve_mds: resolved name 'k' to daemon mds.k 2024-09-10T09:38:01.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:01.614+0000 7f1ca7645740 20 client.25130 populate_metadata read hostname 'smithi005' 2024-09-10T09:38:01.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:01.614+0000 7f1ca7645740 1 --2- 172.21.15.5:0/2824295571 >> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] conn(0x555d2b6ac620 0x555d2b6ab330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:01.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:01.614+0000 7f1ca7645740 4 client.25130 mds_command: new command op to 24443 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T09:38:01.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:01.614+0000 7f1ca7645740 1 -- 172.21.15.5:0/2824295571 --> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] -- command(tid 0: {"prefix": "status"}) -- 0x555d2b63ca20 con 0x555d2b6ac620 2024-09-10T09:38:01.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:01.614+0000 7f1c9de00640 1 --2- 172.21.15.5:0/2824295571 >> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] conn(0x555d2b6ac620 0x555d2b6ab330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:01.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:01.614+0000 7f1c9de00640 1 --2- 172.21.15.5:0/2824295571 >> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] conn(0x555d2b6ac620 0x555d2b6ab330 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=9f3ede1733d4a82f server_cookie=b22d77d7457aaec0 in_seq=0 out_seq=0 2024-09-10T09:38:01.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:01.614+0000 7f1c7f400640 10 client.25130 ms_handle_connect on v2:172.21.15.102:6832/2457007427 2024-09-10T09:38:01.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:01.615+0000 7f1c7f400640 1 -- 172.21.15.5:0/2824295571 <== mds.4 v2:172.21.15.102:6832/2457007427 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x555d2b63ca20 con 0x555d2b6ac620 2024-09-10T09:38:01.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:01.615+0000 7f1c7f400640 10 client.25130 handle_command_reply: tid=0 2024-09-10T09:38:01.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:01.615+0000 7f1ca7645740 1 client.25130 shutdown 2024-09-10T09:38:01.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:01.615+0000 7f1ca7645740 1 -- 172.21.15.5:0/2824295571 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555d2b62c6c0 msgr2=0x555d2b642920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:38:01.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:01.615+0000 7f1ca7645740 1 --2- 172.21.15.5:0/2824295571 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555d2b62c6c0 0x555d2b642920 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f1c98012290 tx=0x7f1c98004a40 comp rx=0 tx=0).stop 2024-09-10T09:38:01.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:01.615+0000 7f1ca7645740 1 -- 172.21.15.5:0/2824295571 shutdown_connections 2024-09-10T09:38:01.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:01.615+0000 7f1ca7645740 1 --2- 172.21.15.5:0/2824295571 >> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] conn(0x555d2b6ac620 0x555d2b6ab330 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:01.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:01.615+0000 7f1ca7645740 1 --2- 172.21.15.5:0/2824295571 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555d2b62c6c0 0x555d2b642920 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:01.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:01.615+0000 7f1ca7645740 1 --2- 172.21.15.5:0/2824295571 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555d2b621e00 0x555d2b6423e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:01.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:01.615+0000 7f1ca7645740 1 --2- 172.21.15.5:0/2824295571 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555d2b6214e0 0x555d2b6470b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:01.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:01.615+0000 7f1ca7645740 1 -- 172.21.15.5:0/2824295571 >> 172.21.15.5:0/2824295571 conn(0x555d2b61da70 msgr2=0x555d2b61c3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:38:01.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:01.615+0000 7f1c7f400640 0 client.25130 ms_handle_reset on v2:172.21.15.102:6832/2457007427 2024-09-10T09:38:01.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:01.615+0000 7f1ca7645740 1 -- 172.21.15.5:0/2824295571 shutdown_connections 2024-09-10T09:38:01.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:01.615+0000 7f1ca7645740 1 -- 172.21.15.5:0/2824295571 wait complete. 2024-09-10T09:38:01.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:01.615+0000 7f1ca7645740 20 client.25130 trim_cache size 0 max 16384 2024-09-10T09:38:01.619 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-10T09:38:01.620 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_fsid": "bade4b50-6f56-11ef-bcea-c7b262605968", 2024-09-10T09:38:01.620 INFO:teuthology.orchestra.run.smithi005.stdout: "whoami": 4, 2024-09-10T09:38:01.620 INFO:teuthology.orchestra.run.smithi005.stdout: "id": 24443, 2024-09-10T09:38:01.620 INFO:teuthology.orchestra.run.smithi005.stdout: "want_state": "up:active", 2024-09-10T09:38:01.620 INFO:teuthology.orchestra.run.smithi005.stdout: "state": "up:active", 2024-09-10T09:38:01.620 INFO:teuthology.orchestra.run.smithi005.stdout: "fs_name": "cephfs", 2024-09-10T09:38:01.620 INFO:teuthology.orchestra.run.smithi005.stdout: "rank_uptime": 107.128016754, 2024-09-10T09:38:01.620 INFO:teuthology.orchestra.run.smithi005.stdout: "mdsmap_epoch": 29, 2024-09-10T09:38:01.620 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch": 96, 2024-09-10T09:38:01.620 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch_barrier": 94, 2024-09-10T09:38:01.620 INFO:teuthology.orchestra.run.smithi005.stdout: "uptime": 196.66179960700001 2024-09-10T09:38:01.620 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-10T09:38:01.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:01.616+0000 7f1c9f200640 1 -- 172.21.15.5:0/4100006579 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f1c840a2c00 msgr2=0x7f1c840a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:38:01.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:01.616+0000 7f1c9f200640 1 --2- 172.21.15.5:0/4100006579 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f1c840a2c00 0x7f1c840a50c0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f1c98009fd0 tx=0x7f1c98009500 comp rx=0 tx=0).stop 2024-09-10T09:38:01.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:01.616+0000 7f1c9f200640 1 -- 172.21.15.5:0/4100006579 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1ca0077040 msgr2=0x7f1ca01a4b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:38:01.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:01.616+0000 7f1c9f200640 1 --2- 172.21.15.5:0/4100006579 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1ca0077040 0x7f1ca01a4b80 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f1c8c004770 tx=0x7f1c8c003c10 comp rx=0 tx=0).stop 2024-09-10T09:38:01.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:01.616+0000 7f1c9f200640 1 -- 172.21.15.5:0/4100006579 shutdown_connections 2024-09-10T09:38:01.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:01.616+0000 7f1c9f200640 1 --2- 172.21.15.5:0/4100006579 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f1c840a2c00 0x7f1c840a50c0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:01.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:01.616+0000 7f1c9f200640 1 --2- 172.21.15.5:0/4100006579 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1ca0112870 0x7f1ca019e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:01.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:01.616+0000 7f1c9f200640 1 --2- 172.21.15.5:0/4100006579 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1ca0077040 0x7f1ca01a4b80 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:01.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:01.616+0000 7f1c9f200640 1 --2- 172.21.15.5:0/4100006579 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1ca0076720 0x7f1ca01a4640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:01.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:01.616+0000 7f1c9f200640 1 -- 172.21.15.5:0/4100006579 >> 172.21.15.5:0/4100006579 conn(0x7f1ca00ff910 msgr2=0x7f1ca0101d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:38:01.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:01.616+0000 7f1c9f200640 1 -- 172.21.15.5:0/4100006579 shutdown_connections 2024-09-10T09:38:01.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:01.616+0000 7f1c9f200640 1 -- 172.21.15.5:0/4100006579 wait complete. 2024-09-10T09:38:02.492 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph tell mds.h status 2024-09-10T09:38:02.887 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:38:03.049 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:38:02 smithi005 ceph-mon[30350]: pgmap v384: 129 pgs: 129 active+clean; 646 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-09-10T09:38:03.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:38:02 smithi159 ceph-mon[35513]: pgmap v384: 129 pgs: 129 active+clean; 646 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-09-10T09:38:03.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:38:02 smithi102 ceph-mon[35442]: pgmap v384: 129 pgs: 129 active+clean; 646 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-09-10T09:38:03.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:03.425+0000 7f8ca8e00640 1 -- 172.21.15.5:0/2997294891 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8ca4077040 msgr2=0x7f8ca41124f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:38:03.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:03.425+0000 7f8ca8e00640 1 --2- 172.21.15.5:0/2997294891 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8ca4077040 0x7f8ca41124f0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f8c90009900 tx=0x7f8c9002a410 comp rx=0 tx=0).stop 2024-09-10T09:38:03.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:03.425+0000 7f8ca8e00640 1 -- 172.21.15.5:0/2997294891 shutdown_connections 2024-09-10T09:38:03.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:03.425+0000 7f8ca8e00640 1 --2- 172.21.15.5:0/2997294891 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8ca4112a30 0x7f8ca4114e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:03.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:03.425+0000 7f8ca8e00640 1 --2- 172.21.15.5:0/2997294891 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8ca4077040 0x7f8ca41124f0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:03.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:03.425+0000 7f8ca8e00640 1 --2- 172.21.15.5:0/2997294891 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8ca4076720 0x7f8ca4076b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:03.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:03.425+0000 7f8ca8e00640 1 -- 172.21.15.5:0/2997294891 >> 172.21.15.5:0/2997294891 conn(0x7f8ca40ffa90 msgr2=0x7f8ca4101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:38:03.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:03.425+0000 7f8ca8e00640 1 -- 172.21.15.5:0/2997294891 shutdown_connections 2024-09-10T09:38:03.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:03.425+0000 7f8ca8e00640 1 -- 172.21.15.5:0/2997294891 wait complete. 2024-09-10T09:38:03.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:03.426+0000 7f8ca8e00640 1 Processor -- start 2024-09-10T09:38:03.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:03.426+0000 7f8ca8e00640 1 -- start start 2024-09-10T09:38:03.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:03.427+0000 7f8ca8e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8ca4076720 0x7f8ca41a4740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:03.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:03.427+0000 7f8ca8e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8ca4077040 0x7f8ca41a4c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:03.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:03.427+0000 7f8ca8e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8ca4112a30 0x7f8ca419e900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:03.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:03.427+0000 7f8ca8e00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f8ca41185f0 con 0x7f8ca4112a30 2024-09-10T09:38:03.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:03.427+0000 7f8ca8e00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f8ca4118470 con 0x7f8ca4077040 2024-09-10T09:38:03.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:03.427+0000 7f8ca8e00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f8ca4118770 con 0x7f8ca4076720 2024-09-10T09:38:03.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:03.427+0000 7f8ca2800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8ca4112a30 0x7f8ca419e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:03.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:03.427+0000 7f8ca2800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8ca4112a30 0x7f8ca419e900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51852/0 (socket says 172.21.15.5:51852) 2024-09-10T09:38:03.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:03.427+0000 7f8ca2800640 1 -- 172.21.15.5:0/3780254412 learned_addr learned my addr 172.21.15.5:0/3780254412 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:38:03.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:03.427+0000 7f8ca1e00640 1 --2- 172.21.15.5:0/3780254412 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8ca4076720 0x7f8ca41a4740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:03.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:03.427+0000 7f8ca1400640 1 --2- 172.21.15.5:0/3780254412 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8ca4077040 0x7f8ca41a4c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:03.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:03.427+0000 7f8ca2800640 1 -- 172.21.15.5:0/3780254412 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8ca4076720 msgr2=0x7f8ca41a4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:38:03.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:03.427+0000 7f8ca2800640 1 --2- 172.21.15.5:0/3780254412 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8ca4076720 0x7f8ca41a4740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:03.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:03.427+0000 7f8ca2800640 1 -- 172.21.15.5:0/3780254412 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8ca4077040 msgr2=0x7f8ca41a4c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:38:03.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:03.427+0000 7f8ca2800640 1 --2- 172.21.15.5:0/3780254412 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8ca4077040 0x7f8ca41a4c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:03.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:03.427+0000 7f8ca2800640 1 -- 172.21.15.5:0/3780254412 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ca419f030 con 0x7f8ca4112a30 2024-09-10T09:38:03.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:03.428+0000 7f8ca2800640 1 --2- 172.21.15.5:0/3780254412 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8ca4112a30 0x7f8ca419e900 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f8c9c00eb10 tx=0x7f8c9c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:38:03.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:03.428+0000 7f8c9aa00640 1 -- 172.21.15.5:0/3780254412 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c9c004b20 con 0x7f8ca4112a30 2024-09-10T09:38:03.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:03.428+0000 7f8c9aa00640 1 -- 172.21.15.5:0/3780254412 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8c9c004cc0 con 0x7f8ca4112a30 2024-09-10T09:38:03.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:03.428+0000 7f8ca8e00640 1 -- 172.21.15.5:0/3780254412 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ca419f320 con 0x7f8ca4112a30 2024-09-10T09:38:03.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:03.428+0000 7f8c9aa00640 1 -- 172.21.15.5:0/3780254412 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c9c010790 con 0x7f8ca4112a30 2024-09-10T09:38:03.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:03.428+0000 7f8ca8e00640 1 -- 172.21.15.5:0/3780254412 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ca41ab610 con 0x7f8ca4112a30 2024-09-10T09:38:03.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:03.431+0000 7f8c9aa00640 1 -- 172.21.15.5:0/3780254412 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f8c9c005020 con 0x7f8ca4112a30 2024-09-10T09:38:03.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:03.431+0000 7f8cab9ec740 1 Processor -- start 2024-09-10T09:38:03.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:03.431+0000 7f8cab9ec740 1 -- start start 2024-09-10T09:38:03.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:03.432+0000 7f8cab9ec740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5585a7fb6330 0x5585a7fb6710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:03.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:03.432+0000 7f8cab9ec740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5585a7fb6c50 0x5585a7fc0fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:03.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:03.432+0000 7f8cab9ec740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5585a7fc1510 0x5585a7fc3900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:03.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:03.432+0000 7f8cab9ec740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5585a7f1c0b0 con 0x5585a7fb6330 2024-09-10T09:38:03.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:03.432+0000 7f8cab9ec740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x5585a7ec3ab0 con 0x5585a7fb6c50 2024-09-10T09:38:03.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:03.432+0000 7f8cab9ec740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5585a7f18070 con 0x5585a7fc1510 2024-09-10T09:38:03.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:03.432+0000 7f8ca1400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5585a7fb6330 0x5585a7fb6710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:03.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:03.432+0000 7f8ca1e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5585a7fb6c50 0x5585a7fc0fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:03.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:03.432+0000 7f8ca1400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5585a7fb6330 0x5585a7fb6710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51856/0 (socket says 172.21.15.5:51856) 2024-09-10T09:38:03.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:03.432+0000 7f8ca1400640 1 -- 172.21.15.5:0/1380292329 learned_addr learned my addr 172.21.15.5:0/1380292329 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:38:03.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:03.433+0000 7f8ca1400640 1 --2- 172.21.15.5:0/1380292329 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5585a7fc1510 0x5585a7fc3900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:03.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:03.433+0000 7f8ca1400640 1 -- 172.21.15.5:0/1380292329 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5585a7fc1510 msgr2=0x5585a7fc3900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:38:03.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:03.433+0000 7f8ca1400640 1 --2- 172.21.15.5:0/1380292329 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5585a7fc1510 0x5585a7fc3900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:03.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:03.433+0000 7f8ca1400640 1 -- 172.21.15.5:0/1380292329 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5585a7fb6c50 msgr2=0x5585a7fc0fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:38:03.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:03.433+0000 7f8ca1400640 1 --2- 172.21.15.5:0/1380292329 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5585a7fb6c50 0x5585a7fc0fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:03.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:03.433+0000 7f8ca1400640 1 -- 172.21.15.5:0/1380292329 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5585a7ed04a0 con 0x5585a7fb6330 2024-09-10T09:38:03.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:03.433+0000 7f8ca1e00640 1 --2- 172.21.15.5:0/1380292329 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5585a7fb6c50 0x5585a7fc0fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:38:03.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:03.433+0000 7f8c9aa00640 1 --2- 172.21.15.5:0/3780254412 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f8c880a2b80 0x7f8c880a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:03.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:03.433+0000 7f8ca1400640 1 --2- 172.21.15.5:0/1380292329 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5585a7fb6330 0x5585a7fb6710 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f8c90009230 tx=0x7f8c90005bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ae63cb737a0e7cb server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:38:03.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:03.433+0000 7f8c9aa00640 1 -- 172.21.15.5:0/3780254412 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f8c9c0cfb90 con 0x7f8ca4112a30 2024-09-10T09:38:03.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:03.433+0000 7f8ca1e00640 1 --2- 172.21.15.5:0/3780254412 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f8c880a2b80 0x7f8c880a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:03.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:03.433+0000 7f8c8f400640 1 -- 172.21.15.5:0/1380292329 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c90044020 con 0x5585a7fb6330 2024-09-10T09:38:03.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:03.433+0000 7f8c8f400640 1 -- 172.21.15.5:0/1380292329 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8c900044a0 con 0x5585a7fb6330 2024-09-10T09:38:03.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:03.433+0000 7f8c8f400640 1 -- 172.21.15.5:0/1380292329 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c90035040 con 0x5585a7fb6330 2024-09-10T09:38:03.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:03.434+0000 7f8cab9ec740 1 -- 172.21.15.5:0/1380292329 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5585a7fb6330 msgr2=0x5585a7fb6710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:38:03.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:03.434+0000 7f8cab9ec740 1 --2- 172.21.15.5:0/1380292329 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5585a7fb6330 0x5585a7fb6710 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f8c90009230 tx=0x7f8c90005bc0 comp rx=0 tx=0).stop 2024-09-10T09:38:03.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:03.434+0000 7f8ca1e00640 1 --2- 172.21.15.5:0/3780254412 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f8c880a2b80 0x7f8c880a5040 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x5585a7fb3720 tx=0x7f8c94005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:38:03.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:03.435+0000 7f8cab9ec740 1 -- 172.21.15.5:0/1380292329 shutdown_connections 2024-09-10T09:38:03.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:03.435+0000 7f8cab9ec740 1 --2- 172.21.15.5:0/1380292329 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5585a7fc1510 0x5585a7fc3900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:03.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:03.435+0000 7f8cab9ec740 1 --2- 172.21.15.5:0/1380292329 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5585a7fb6c50 0x5585a7fc0fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:03.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:03.435+0000 7f8cab9ec740 1 --2- 172.21.15.5:0/1380292329 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5585a7fb6330 0x5585a7fb6710 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:03.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:03.435+0000 7f8cab9ec740 1 -- 172.21.15.5:0/1380292329 >> 172.21.15.5:0/1380292329 conn(0x5585a7fb28c0 msgr2=0x5585a7fb2ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:38:03.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:03.435+0000 7f8cab9ec740 1 -- 172.21.15.5:0/1380292329 shutdown_connections 2024-09-10T09:38:03.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:03.435+0000 7f8cab9ec740 1 -- 172.21.15.5:0/1380292329 wait complete. 2024-09-10T09:38:03.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:03.435+0000 7f8cab9ec740 1 -- start start 2024-09-10T09:38:03.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:03.435+0000 7f8cab9ec740 1 Processor -- start 2024-09-10T09:38:03.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:03.436+0000 7f8cab9ec740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5585a7fb6330 0x5585a7fbe590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:03.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:03.436+0000 7f8cab9ec740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5585a7fb6c50 0x5585a7fbead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:03.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:03.436+0000 7f8cab9ec740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5585a7fc1510 0x5585a7fbf010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:03.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:03.436+0000 7f8cab9ec740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5585a7fc4060 con 0x5585a7fc1510 2024-09-10T09:38:03.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:03.436+0000 7f8cab9ec740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x5585a7ea4920 con 0x5585a7fb6c50 2024-09-10T09:38:03.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:03.436+0000 7f8cab9ec740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5585a7fc41e0 con 0x5585a7fb6330 2024-09-10T09:38:03.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:03.436+0000 7f8cab9ec740 10 client.0 _ll_register_callbacks cb 0x5585a7ec88a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:38:03.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:03.436+0000 7f8ca1e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5585a7fb6330 0x5585a7fbe590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:03.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:03.436+0000 7f8ca1e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5585a7fb6330 0x5585a7fbe590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:46402/0 (socket says 172.21.15.5:46402) 2024-09-10T09:38:03.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:03.436+0000 7f8ca1e00640 1 -- 172.21.15.5:0/331804737 learned_addr learned my addr 172.21.15.5:0/331804737 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:38:03.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:03.436+0000 7f8ca1400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5585a7fb6c50 0x5585a7fbead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:03.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:03.436+0000 7f8ca2800640 1 --2- 172.21.15.5:0/331804737 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5585a7fc1510 0x5585a7fbf010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:03.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:03.437+0000 7f8ca1400640 1 -- 172.21.15.5:0/331804737 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5585a7fb6330 msgr2=0x5585a7fbe590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:38:03.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:03.437+0000 7f8ca1400640 1 --2- 172.21.15.5:0/331804737 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5585a7fb6330 0x5585a7fbe590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:03.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:03.437+0000 7f8ca1400640 1 -- 172.21.15.5:0/331804737 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5585a7fc1510 msgr2=0x5585a7fbf010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:38:03.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:03.437+0000 7f8ca1400640 1 --2- 172.21.15.5:0/331804737 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5585a7fc1510 0x5585a7fbf010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:03.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:03.437+0000 7f8ca1400640 1 -- 172.21.15.5:0/331804737 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5585a7c8d2b0 con 0x5585a7fb6c50 2024-09-10T09:38:03.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:03.437+0000 7f8ca1e00640 1 --2- 172.21.15.5:0/331804737 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5585a7fb6330 0x5585a7fbe590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:38:03.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:03.437+0000 7f8ca1400640 1 --2- 172.21.15.5:0/331804737 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5585a7fb6c50 0x5585a7fbead0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f8c9002a930 tx=0x7f8c90005850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=690e1c521693d9b5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:38:03.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:03.437+0000 7f8c83e00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-09-10T09:38:03.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:03.437+0000 7f8c83e00640 1 -- 172.21.15.5:0/331804737 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c9004d070 con 0x5585a7fb6c50 2024-09-10T09:38:03.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:03.437+0000 7f8cab9ec740 1 -- 172.21.15.5:0/331804737 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8c900044a0 con 0x5585a7fb6c50 2024-09-10T09:38:03.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:03.437+0000 7f8c83e00640 1 -- 172.21.15.5:0/331804737 <== mon.2 v2:172.21.15.102:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f8c900050d0 con 0x5585a7fb6c50 2024-09-10T09:38:03.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:03.438+0000 7f8c83e00640 1 -- 172.21.15.5:0/331804737 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f8c90056050 con 0x5585a7fb6c50 2024-09-10T09:38:03.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:03.438+0000 7f8cab9ec740 10 client.25142 fetch_fsmap learned FSMap version 29 2024-09-10T09:38:03.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:03.438+0000 7f8cab9ec740 1 -- 172.21.15.5:0/331804737 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5585a7fcfe10 con 0x5585a7fb6c50 2024-09-10T09:38:03.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:03.438+0000 7f8c83e00640 1 -- 172.21.15.5:0/331804737 <== mon.2 v2:172.21.15.102:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8c900044a0 con 0x5585a7fb6c50 2024-09-10T09:38:03.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:03.438+0000 7f8c83e00640 1 -- 172.21.15.5:0/331804737 <== mon.2 v2:172.21.15.102:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f8c90050020 con 0x5585a7fb6c50 2024-09-10T09:38:03.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:03.438+0000 7f8cab9ec740 10 client.25142 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T09:38:03.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:03.438+0000 7f8cab9ec740 10 client.25142 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-10T09:38:03.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:03.438+0000 7f8cab9ec740 20 client.25142 populate_metadata read hostname 'smithi005' 2024-09-10T09:38:03.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:03.438+0000 7f8cab9ec740 1 --2- 172.21.15.5:0/331804737 >> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] conn(0x5585a7fd1f20 0x5585a7fd3390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:03.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:03.438+0000 7f8cab9ec740 4 client.25142 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:38:03.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:03.438+0000 7f8cab9ec740 1 -- 172.21.15.5:0/331804737 --> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5585a7f29620 con 0x5585a7fd1f20 2024-09-10T09:38:03.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:03.439+0000 7f8ca1e00640 1 --2- 172.21.15.5:0/331804737 >> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] conn(0x5585a7fd1f20 0x5585a7fd3390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:03.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:03.439+0000 7f8ca1e00640 1 --2- 172.21.15.5:0/331804737 >> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] conn(0x5585a7fd1f20 0x5585a7fd3390 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=414afef84027e542 server_cookie=68a84d34e02a23eb in_seq=0 out_seq=0 2024-09-10T09:38:03.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:03.439+0000 7f8c83e00640 10 client.25142 ms_handle_connect on v2:172.21.15.102:6836/2689776764 2024-09-10T09:38:03.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:03.440+0000 7f8c83e00640 1 -- 172.21.15.5:0/331804737 <== mds.3 v2:172.21.15.102:6836/2689776764 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5585a7f29620 con 0x5585a7fd1f20 2024-09-10T09:38:03.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:03.440+0000 7f8c83e00640 10 client.25142 handle_command_reply: tid=0 2024-09-10T09:38:03.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:03.440+0000 7f8cab9ec740 1 client.25142 shutdown 2024-09-10T09:38:03.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:03.440+0000 7f8cab9ec740 1 -- 172.21.15.5:0/331804737 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5585a7fb6c50 msgr2=0x5585a7fbead0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:38:03.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:03.440+0000 7f8cab9ec740 1 --2- 172.21.15.5:0/331804737 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5585a7fb6c50 0x5585a7fbead0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f8c9002a930 tx=0x7f8c90005850 comp rx=0 tx=0).stop 2024-09-10T09:38:03.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:03.440+0000 7f8cab9ec740 1 -- 172.21.15.5:0/331804737 shutdown_connections 2024-09-10T09:38:03.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:03.440+0000 7f8cab9ec740 1 --2- 172.21.15.5:0/331804737 >> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] conn(0x5585a7fd1f20 0x5585a7fd3390 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:03.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:03.440+0000 7f8cab9ec740 1 --2- 172.21.15.5:0/331804737 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5585a7fc1510 0x5585a7fbf010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:03.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:03.440+0000 7f8cab9ec740 1 --2- 172.21.15.5:0/331804737 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5585a7fb6c50 0x5585a7fbead0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:03.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:03.440+0000 7f8cab9ec740 1 --2- 172.21.15.5:0/331804737 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5585a7fb6330 0x5585a7fbe590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:03.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:03.440+0000 7f8c83e00640 0 client.25142 ms_handle_reset on v2:172.21.15.102:6836/2689776764 2024-09-10T09:38:03.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:03.440+0000 7f8cab9ec740 1 -- 172.21.15.5:0/331804737 >> 172.21.15.5:0/331804737 conn(0x5585a7fb28c0 msgr2=0x5585a7fb85f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:38:03.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:03.440+0000 7f8cab9ec740 1 -- 172.21.15.5:0/331804737 shutdown_connections 2024-09-10T09:38:03.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:03.440+0000 7f8cab9ec740 1 -- 172.21.15.5:0/331804737 wait complete. 2024-09-10T09:38:03.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:03.441+0000 7f8cab9ec740 20 client.25142 trim_cache size 0 max 16384 2024-09-10T09:38:03.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:03.459+0000 7f8cab9ec740 1 Processor -- start 2024-09-10T09:38:03.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:03.459+0000 7f8cab9ec740 1 -- start start 2024-09-10T09:38:03.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:03.460+0000 7f8cab9ec740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5585a7fb6330 0x5585a7fb2210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:03.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:03.460+0000 7f8cab9ec740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5585a7fb6c50 0x5585a7fbee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:03.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:03.460+0000 7f8cab9ec740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5585a7fc1510 0x5585a7fc0500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:03.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:03.460+0000 7f8cab9ec740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5585a7fd16d0 con 0x5585a7fb6330 2024-09-10T09:38:03.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:03.460+0000 7f8cab9ec740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x5585a7fd1da0 con 0x5585a7fc1510 2024-09-10T09:38:03.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:03.460+0000 7f8cab9ec740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5585a7fd2470 con 0x5585a7fb6c50 2024-09-10T09:38:03.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:03.460+0000 7f8ca1e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5585a7fb6330 0x5585a7fb2210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:03.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:03.460+0000 7f8ca1e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5585a7fb6330 0x5585a7fb2210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51876/0 (socket says 172.21.15.5:51876) 2024-09-10T09:38:03.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:03.460+0000 7f8ca1e00640 1 -- 172.21.15.5:0/872469704 learned_addr learned my addr 172.21.15.5:0/872469704 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:38:03.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:03.460+0000 7f8ca1400640 1 --2- 172.21.15.5:0/872469704 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5585a7fb6c50 0x5585a7fbee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:03.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:03.460+0000 7f8ca2800640 1 --2- 172.21.15.5:0/872469704 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5585a7fc1510 0x5585a7fc0500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:03.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:03.460+0000 7f8ca1e00640 1 -- 172.21.15.5:0/872469704 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5585a7fb6c50 msgr2=0x5585a7fbee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:38:03.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:03.460+0000 7f8ca1e00640 1 --2- 172.21.15.5:0/872469704 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5585a7fb6c50 0x5585a7fbee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:03.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:03.460+0000 7f8ca1e00640 1 -- 172.21.15.5:0/872469704 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5585a7fc1510 msgr2=0x5585a7fc0500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:38:03.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:03.460+0000 7f8ca1e00640 1 --2- 172.21.15.5:0/872469704 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5585a7fc1510 0x5585a7fc0500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:03.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:03.460+0000 7f8ca1e00640 1 -- 172.21.15.5:0/872469704 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5585a7fb3970 con 0x5585a7fb6330 2024-09-10T09:38:03.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:03.461+0000 7f8ca1e00640 1 --2- 172.21.15.5:0/872469704 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5585a7fb6330 0x5585a7fb2210 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x5585a7fd3bc0 tx=0x7f8c9401b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfdffe2e95d45898 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:38:03.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:03.461+0000 7f8c8ea00640 1 -- 172.21.15.5:0/872469704 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c94003e90 con 0x5585a7fb6330 2024-09-10T09:38:03.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:03.461+0000 7f8c8ea00640 1 -- 172.21.15.5:0/872469704 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8c9401fce0 con 0x5585a7fb6330 2024-09-10T09:38:03.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:03.461+0000 7f8c8ea00640 1 -- 172.21.15.5:0/872469704 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c9401e420 con 0x5585a7fb6330 2024-09-10T09:38:03.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:03.462+0000 7f8cab9ec740 1 -- 172.21.15.5:0/872469704 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5585a7fb6330 msgr2=0x5585a7fb2210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:38:03.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:03.462+0000 7f8cab9ec740 1 --2- 172.21.15.5:0/872469704 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5585a7fb6330 0x5585a7fb2210 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x5585a7fd3bc0 tx=0x7f8c9401b000 comp rx=0 tx=0).stop 2024-09-10T09:38:03.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:03.462+0000 7f8cab9ec740 1 -- 172.21.15.5:0/872469704 shutdown_connections 2024-09-10T09:38:03.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:03.462+0000 7f8cab9ec740 1 --2- 172.21.15.5:0/872469704 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5585a7fc1510 0x5585a7fc0500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:03.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:03.462+0000 7f8cab9ec740 1 --2- 172.21.15.5:0/872469704 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5585a7fb6c50 0x5585a7fbee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:03.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:03.462+0000 7f8cab9ec740 1 --2- 172.21.15.5:0/872469704 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5585a7fb6330 0x5585a7fb2210 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:03.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:03.462+0000 7f8cab9ec740 1 -- 172.21.15.5:0/872469704 >> 172.21.15.5:0/872469704 conn(0x5585a7fb28c0 msgr2=0x5585a7fbfdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:38:03.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:03.462+0000 7f8cab9ec740 1 -- 172.21.15.5:0/872469704 shutdown_connections 2024-09-10T09:38:03.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:03.462+0000 7f8cab9ec740 1 -- 172.21.15.5:0/872469704 wait complete. 2024-09-10T09:38:03.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:03.462+0000 7f8cab9ec740 1 -- start start 2024-09-10T09:38:03.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:03.462+0000 7f8cab9ec740 1 Processor -- start 2024-09-10T09:38:03.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:03.463+0000 7f8cab9ec740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5585a7fb6330 0x5585a7fbb5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:03.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:03.463+0000 7f8cab9ec740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5585a7fb6c50 0x5585a7fbbb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:03.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:03.463+0000 7f8cab9ec740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5585a7fc1510 0x5585a7fbc050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:03.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:03.463+0000 7f8cab9ec740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5585a7fd3210 con 0x5585a7fb6330 2024-09-10T09:38:03.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:03.463+0000 7f8cab9ec740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x5585a7fd2b40 con 0x5585a7fc1510 2024-09-10T09:38:03.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:03.463+0000 7f8cab9ec740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5585a7fd38e0 con 0x5585a7fb6c50 2024-09-10T09:38:03.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:03.463+0000 7f8cab9ec740 10 client.0 _ll_register_callbacks cb 0x7f8c90009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:38:03.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:03.463+0000 7f8ca1e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5585a7fb6330 0x5585a7fbb5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:03.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:03.463+0000 7f8ca1e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5585a7fb6330 0x5585a7fbb5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51886/0 (socket says 172.21.15.5:51886) 2024-09-10T09:38:03.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:03.463+0000 7f8ca1e00640 1 -- 172.21.15.5:0/2711613307 learned_addr learned my addr 172.21.15.5:0/2711613307 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:38:03.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:03.463+0000 7f8ca1400640 1 --2- 172.21.15.5:0/2711613307 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5585a7fb6c50 0x5585a7fbbb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:03.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:03.463+0000 7f8ca2800640 1 --2- 172.21.15.5:0/2711613307 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5585a7fc1510 0x5585a7fbc050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:03.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:03.464+0000 7f8ca1e00640 1 -- 172.21.15.5:0/2711613307 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5585a7fb6c50 msgr2=0x5585a7fbbb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:38:03.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:03.464+0000 7f8ca1e00640 1 --2- 172.21.15.5:0/2711613307 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5585a7fb6c50 0x5585a7fbbb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:03.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:03.464+0000 7f8ca1e00640 1 -- 172.21.15.5:0/2711613307 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5585a7fc1510 msgr2=0x5585a7fbc050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:38:03.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:03.464+0000 7f8ca1e00640 1 --2- 172.21.15.5:0/2711613307 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5585a7fc1510 0x5585a7fbc050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:03.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:03.464+0000 7f8ca1e00640 1 -- 172.21.15.5:0/2711613307 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5585a7fb3550 con 0x5585a7fb6330 2024-09-10T09:38:03.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:03.464+0000 7f8ca2800640 1 --2- 172.21.15.5:0/2711613307 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5585a7fc1510 0x5585a7fbc050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:38:03.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:03.464+0000 7f8ca1e00640 1 --2- 172.21.15.5:0/2711613307 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5585a7fb6330 0x5585a7fbb5d0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f8c94009890 tx=0x7f8c940098f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8ab4be1c95c0411 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:38:03.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:03.464+0000 7f8c83e00640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-10T09:38:03.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:03.464+0000 7f8c83e00640 1 -- 172.21.15.5:0/2711613307 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c94003e90 con 0x5585a7fb6330 2024-09-10T09:38:03.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:03.464+0000 7f8c83e00640 1 -- 172.21.15.5:0/2711613307 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f8c9401f800 con 0x5585a7fb6330 2024-09-10T09:38:03.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:03.464+0000 7f8cab9ec740 1 -- 172.21.15.5:0/2711613307 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8c9401fce0 con 0x5585a7fb6330 2024-09-10T09:38:03.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:03.465+0000 7f8c83e00640 1 -- 172.21.15.5:0/2711613307 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f8c9401fa00 con 0x5585a7fb6330 2024-09-10T09:38:03.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:03.465+0000 7f8cab9ec740 10 client.15132 fetch_fsmap learned FSMap version 29 2024-09-10T09:38:03.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:03.465+0000 7f8cab9ec740 1 -- 172.21.15.5:0/2711613307 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5585a803c690 con 0x5585a7fb6330 2024-09-10T09:38:03.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:03.465+0000 7f8c83e00640 1 -- 172.21.15.5:0/2711613307 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8c9401fce0 con 0x5585a7fb6330 2024-09-10T09:38:03.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:03.465+0000 7f8c83e00640 1 -- 172.21.15.5:0/2711613307 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f8c9403d020 con 0x5585a7fb6330 2024-09-10T09:38:03.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:03.465+0000 7f8cab9ec740 10 client.15132 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T09:38:03.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:03.465+0000 7f8cab9ec740 10 client.15132 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-10T09:38:03.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:03.465+0000 7f8cab9ec740 20 client.15132 populate_metadata read hostname 'smithi005' 2024-09-10T09:38:03.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:03.465+0000 7f8cab9ec740 1 --2- 172.21.15.5:0/2711613307 >> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] conn(0x5585a8041080 0x5585a803fd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:03.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:03.465+0000 7f8cab9ec740 4 client.15132 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T09:38:03.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:03.465+0000 7f8cab9ec740 1 -- 172.21.15.5:0/2711613307 --> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] -- command(tid 0: {"prefix": "status"}) -- 0x5585a7f23b30 con 0x5585a8041080 2024-09-10T09:38:03.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:03.466+0000 7f8ca1400640 1 --2- 172.21.15.5:0/2711613307 >> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] conn(0x5585a8041080 0x5585a803fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:03.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:03.466+0000 7f8ca1400640 1 --2- 172.21.15.5:0/2711613307 >> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] conn(0x5585a8041080 0x5585a803fd90 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=853ff0aabca52502 server_cookie=f20b7225d2bc3b66 in_seq=0 out_seq=0 2024-09-10T09:38:03.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:03.466+0000 7f8c83e00640 10 client.15132 ms_handle_connect on v2:172.21.15.102:6836/2689776764 2024-09-10T09:38:03.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:03.466+0000 7f8c83e00640 1 -- 172.21.15.5:0/2711613307 <== mds.3 v2:172.21.15.102:6836/2689776764 1 ==== command_reply(tid 0: 0 ) ==== 8+0+322 (crc 0 0 0) 0x5585a7f23b30 con 0x5585a8041080 2024-09-10T09:38:03.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:03.466+0000 7f8c83e00640 10 client.15132 handle_command_reply: tid=0 2024-09-10T09:38:03.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:03.466+0000 7f8cab9ec740 1 client.15132 shutdown 2024-09-10T09:38:03.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:03.467+0000 7f8cab9ec740 1 -- 172.21.15.5:0/2711613307 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5585a7fb6330 msgr2=0x5585a7fbb5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:38:03.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:03.467+0000 7f8cab9ec740 1 --2- 172.21.15.5:0/2711613307 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5585a7fb6330 0x5585a7fbb5d0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f8c94009890 tx=0x7f8c940098f0 comp rx=0 tx=0).stop 2024-09-10T09:38:03.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:03.467+0000 7f8cab9ec740 1 -- 172.21.15.5:0/2711613307 shutdown_connections 2024-09-10T09:38:03.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:03.467+0000 7f8cab9ec740 1 --2- 172.21.15.5:0/2711613307 >> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] conn(0x5585a8041080 0x5585a803fd90 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:03.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:03.467+0000 7f8cab9ec740 1 --2- 172.21.15.5:0/2711613307 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5585a7fc1510 0x5585a7fbc050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:03.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:03.467+0000 7f8cab9ec740 1 --2- 172.21.15.5:0/2711613307 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5585a7fb6c50 0x5585a7fbbb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:03.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:03.467+0000 7f8c83e00640 0 client.15132 ms_handle_reset on v2:172.21.15.102:6836/2689776764 2024-09-10T09:38:03.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:03.467+0000 7f8cab9ec740 1 --2- 172.21.15.5:0/2711613307 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5585a7fb6330 0x5585a7fbb5d0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:03.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:03.467+0000 7f8cab9ec740 1 -- 172.21.15.5:0/2711613307 >> 172.21.15.5:0/2711613307 conn(0x5585a7fb28c0 msgr2=0x5585a7fbca80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:38:03.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:03.467+0000 7f8cab9ec740 1 -- 172.21.15.5:0/2711613307 shutdown_connections 2024-09-10T09:38:03.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:03.467+0000 7f8cab9ec740 1 -- 172.21.15.5:0/2711613307 wait complete. 2024-09-10T09:38:03.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:03.467+0000 7f8cab9ec740 20 client.15132 trim_cache size 0 max 16384 2024-09-10T09:38:03.472 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-10T09:38:03.472 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_fsid": "bade4b50-6f56-11ef-bcea-c7b262605968", 2024-09-10T09:38:03.472 INFO:teuthology.orchestra.run.smithi005.stdout: "whoami": 3, 2024-09-10T09:38:03.472 INFO:teuthology.orchestra.run.smithi005.stdout: "id": 24506, 2024-09-10T09:38:03.472 INFO:teuthology.orchestra.run.smithi005.stdout: "want_state": "up:active", 2024-09-10T09:38:03.472 INFO:teuthology.orchestra.run.smithi005.stdout: "state": "up:active", 2024-09-10T09:38:03.472 INFO:teuthology.orchestra.run.smithi005.stdout: "fs_name": "cephfs", 2024-09-10T09:38:03.472 INFO:teuthology.orchestra.run.smithi005.stdout: "rank_uptime": 109.659407376, 2024-09-10T09:38:03.472 INFO:teuthology.orchestra.run.smithi005.stdout: "mdsmap_epoch": 29, 2024-09-10T09:38:03.472 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch": 96, 2024-09-10T09:38:03.472 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch_barrier": 94, 2024-09-10T09:38:03.472 INFO:teuthology.orchestra.run.smithi005.stdout: "uptime": 179.382523432 2024-09-10T09:38:03.473 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-10T09:38:03.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:03.468+0000 7f8ca8e00640 1 -- 172.21.15.5:0/3780254412 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f8c880a2b80 msgr2=0x7f8c880a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:38:03.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:03.468+0000 7f8ca8e00640 1 --2- 172.21.15.5:0/3780254412 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f8c880a2b80 0x7f8c880a5040 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x5585a7fb3720 tx=0x7f8c94005f90 comp rx=0 tx=0).stop 2024-09-10T09:38:03.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:03.468+0000 7f8ca8e00640 1 -- 172.21.15.5:0/3780254412 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8ca4112a30 msgr2=0x7f8ca419e900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:38:03.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:03.468+0000 7f8ca8e00640 1 --2- 172.21.15.5:0/3780254412 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8ca4112a30 0x7f8ca419e900 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f8c9c00eb10 tx=0x7f8c9c00c2d0 comp rx=0 tx=0).stop 2024-09-10T09:38:03.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:03.468+0000 7f8ca8e00640 1 -- 172.21.15.5:0/3780254412 shutdown_connections 2024-09-10T09:38:03.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:03.468+0000 7f8ca8e00640 1 --2- 172.21.15.5:0/3780254412 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f8c880a2b80 0x7f8c880a5040 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:03.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:03.468+0000 7f8ca8e00640 1 --2- 172.21.15.5:0/3780254412 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8ca4112a30 0x7f8ca419e900 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:03.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:03.468+0000 7f8ca8e00640 1 --2- 172.21.15.5:0/3780254412 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8ca4077040 0x7f8ca41a4c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:03.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:03.468+0000 7f8ca8e00640 1 --2- 172.21.15.5:0/3780254412 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8ca4076720 0x7f8ca41a4740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:03.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:03.468+0000 7f8ca8e00640 1 -- 172.21.15.5:0/3780254412 >> 172.21.15.5:0/3780254412 conn(0x7f8ca40ffa90 msgr2=0x7f8ca4101e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:38:03.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:03.468+0000 7f8ca8e00640 1 -- 172.21.15.5:0/3780254412 shutdown_connections 2024-09-10T09:38:03.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:03.468+0000 7f8ca8e00640 1 -- 172.21.15.5:0/3780254412 wait complete. 2024-09-10T09:38:04.272 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph tell mds.c status 2024-09-10T09:38:04.650 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:38:05.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:38:04 smithi005 ceph-mon[30350]: pgmap v385: 129 pgs: 129 active+clean; 646 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-09-10T09:38:05.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:38:04 smithi159 ceph-mon[35513]: pgmap v385: 129 pgs: 129 active+clean; 646 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-09-10T09:38:05.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:38:04 smithi102 ceph-mon[35442]: pgmap v385: 129 pgs: 129 active+clean; 646 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-09-10T09:38:05.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:05.197+0000 7f2207200640 1 -- 172.21.15.5:0/1002545260 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2200109090 msgr2=0x7f2200110c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:38:05.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:05.197+0000 7f2207200640 1 --2- 172.21.15.5:0/1002545260 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2200109090 0x7f2200110c10 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f21fc009ca0 tx=0x7f21fc02a8c0 comp rx=0 tx=0).stop 2024-09-10T09:38:05.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:05.198+0000 7f2207200640 1 -- 172.21.15.5:0/1002545260 shutdown_connections 2024-09-10T09:38:05.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:05.198+0000 7f2207200640 1 --2- 172.21.15.5:0/1002545260 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2200109090 0x7f2200110c10 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:05.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:05.198+0000 7f2207200640 1 --2- 172.21.15.5:0/1002545260 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2200104800 0x7f2200108960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:05.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:05.198+0000 7f2207200640 1 --2- 172.21.15.5:0/1002545260 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2200103e50 0x7f2200104230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:05.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:05.198+0000 7f2207200640 1 -- 172.21.15.5:0/1002545260 >> 172.21.15.5:0/1002545260 conn(0x7f22000ff8b0 msgr2=0x7f2200101cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:38:05.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:05.198+0000 7f2207200640 1 -- 172.21.15.5:0/1002545260 shutdown_connections 2024-09-10T09:38:05.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:05.198+0000 7f2207200640 1 -- 172.21.15.5:0/1002545260 wait complete. 2024-09-10T09:38:05.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:05.199+0000 7f2207200640 1 Processor -- start 2024-09-10T09:38:05.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:05.199+0000 7f2207200640 1 -- start start 2024-09-10T09:38:05.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:05.199+0000 7f2207200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2200103e50 0x7f220019a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:05.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:05.199+0000 7f2207200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2200104800 0x7f220019aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:05.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:05.199+0000 7f2207200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2200109090 0x7f220019edc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:05.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:05.199+0000 7f2207200640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f2200114090 con 0x7f2200103e50 2024-09-10T09:38:05.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:05.199+0000 7f2207200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f2200113f10 con 0x7f2200109090 2024-09-10T09:38:05.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:05.199+0000 7f2207200640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f2200114210 con 0x7f2200104800 2024-09-10T09:38:05.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:05.199+0000 7f2205e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2200103e50 0x7f220019a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:05.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:05.199+0000 7f2205e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2200103e50 0x7f220019a4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51904/0 (socket says 172.21.15.5:51904) 2024-09-10T09:38:05.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:05.199+0000 7f2205e00640 1 -- 172.21.15.5:0/4049606165 learned_addr learned my addr 172.21.15.5:0/4049606165 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:38:05.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:05.199+0000 7f2206800640 1 --2- 172.21.15.5:0/4049606165 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2200109090 0x7f220019edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:05.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:05.200+0000 7f2205400640 1 --2- 172.21.15.5:0/4049606165 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2200104800 0x7f220019aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:05.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:05.200+0000 7f2205e00640 1 -- 172.21.15.5:0/4049606165 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2200104800 msgr2=0x7f220019aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:38:05.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:05.200+0000 7f2205e00640 1 --2- 172.21.15.5:0/4049606165 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2200104800 0x7f220019aa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:05.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:05.200+0000 7f2205e00640 1 -- 172.21.15.5:0/4049606165 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2200109090 msgr2=0x7f220019edc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:38:05.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:05.200+0000 7f2205e00640 1 --2- 172.21.15.5:0/4049606165 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2200109090 0x7f220019edc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:05.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:05.200+0000 7f2205e00640 1 -- 172.21.15.5:0/4049606165 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f220019f4a0 con 0x7f2200103e50 2024-09-10T09:38:05.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:05.200+0000 7f2205e00640 1 --2- 172.21.15.5:0/4049606165 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2200103e50 0x7f220019a4f0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f21f400e9f0 tx=0x7f21f400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:38:05.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:05.200+0000 7f21faa00640 1 -- 172.21.15.5:0/4049606165 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21f400cd80 con 0x7f2200103e50 2024-09-10T09:38:05.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:05.201+0000 7f21faa00640 1 -- 172.21.15.5:0/4049606165 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f21f4010910 con 0x7f2200103e50 2024-09-10T09:38:05.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:05.201+0000 7f2207200640 1 -- 172.21.15.5:0/4049606165 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f220019f790 con 0x7f2200103e50 2024-09-10T09:38:05.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:05.201+0000 7f21faa00640 1 -- 172.21.15.5:0/4049606165 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21f4018990 con 0x7f2200103e50 2024-09-10T09:38:05.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:05.201+0000 7f2207200640 1 -- 172.21.15.5:0/4049606165 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22001a7070 con 0x7f2200103e50 2024-09-10T09:38:05.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:05.204+0000 7f220b27d740 1 Processor -- start 2024-09-10T09:38:05.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:05.204+0000 7f220b27d740 1 -- start start 2024-09-10T09:38:05.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:05.204+0000 7f21faa00640 1 -- 172.21.15.5:0/4049606165 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f21f4010ac0 con 0x7f2200103e50 2024-09-10T09:38:05.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:05.204+0000 7f220b27d740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564e7f1834e0 0x564e7f1838c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:05.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:05.204+0000 7f220b27d740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564e7f183e00 0x564e7f18e180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:05.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:05.204+0000 7f220b27d740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564e7f18e6c0 0x564e7f190ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:05.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:05.204+0000 7f220b27d740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x564e7f0e90b0 con 0x564e7f183e00 2024-09-10T09:38:05.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:05.204+0000 7f220b27d740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x564e7f090ab0 con 0x564e7f1834e0 2024-09-10T09:38:05.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:05.204+0000 7f220b27d740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x564e7f0e5070 con 0x564e7f18e6c0 2024-09-10T09:38:05.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:05.205+0000 7f2206800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564e7f1834e0 0x564e7f1838c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:05.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:05.205+0000 7f2206800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564e7f1834e0 0x564e7f1838c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:45122/0 (socket says 172.21.15.5:45122) 2024-09-10T09:38:05.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:05.205+0000 7f2206800640 1 -- 172.21.15.5:0/993334954 learned_addr learned my addr 172.21.15.5:0/993334954 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:38:05.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:05.205+0000 7f2205e00640 1 --2- 172.21.15.5:0/993334954 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564e7f183e00 0x564e7f18e180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:05.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:05.205+0000 7f2205400640 1 --2- 172.21.15.5:0/993334954 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564e7f18e6c0 0x564e7f190ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:05.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:05.205+0000 7f2206800640 1 -- 172.21.15.5:0/993334954 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564e7f18e6c0 msgr2=0x564e7f190ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:38:05.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:05.205+0000 7f2206800640 1 --2- 172.21.15.5:0/993334954 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564e7f18e6c0 0x564e7f190ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:05.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:05.205+0000 7f2206800640 1 -- 172.21.15.5:0/993334954 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564e7f183e00 msgr2=0x564e7f18e180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:38:05.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:05.205+0000 7f2206800640 1 --2- 172.21.15.5:0/993334954 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564e7f183e00 0x564e7f18e180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:05.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:05.205+0000 7f2206800640 1 -- 172.21.15.5:0/993334954 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564e7f09d4a0 con 0x564e7f1834e0 2024-09-10T09:38:05.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:05.205+0000 7f2206800640 1 --2- 172.21.15.5:0/993334954 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564e7f1834e0 0x564e7f1838c0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f21fc009ca0 tx=0x7f21fc009800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f3387e95371ee356 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:38:05.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:05.206+0000 7f21ef000640 1 -- 172.21.15.5:0/993334954 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21fc031db0 con 0x564e7f1834e0 2024-09-10T09:38:05.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:05.206+0000 7f21faa00640 1 --2- 172.21.15.5:0/4049606165 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f21e80a2b80 0x7f21e80a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:05.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:05.206+0000 7f21ef000640 1 -- 172.21.15.5:0/993334954 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f21fc036040 con 0x564e7f1834e0 2024-09-10T09:38:05.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:05.206+0000 7f21ef000640 1 -- 172.21.15.5:0/993334954 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21fc004330 con 0x564e7f1834e0 2024-09-10T09:38:05.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:05.206+0000 7f21faa00640 1 -- 172.21.15.5:0/4049606165 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f21f40cfa20 con 0x7f2200103e50 2024-09-10T09:38:05.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:05.206+0000 7f2205e00640 1 --2- 172.21.15.5:0/4049606165 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f21e80a2b80 0x7f21e80a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:05.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:05.206+0000 7f220b27d740 1 -- 172.21.15.5:0/993334954 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564e7f1834e0 msgr2=0x564e7f1838c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:38:05.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:05.206+0000 7f220b27d740 1 --2- 172.21.15.5:0/993334954 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564e7f1834e0 0x564e7f1838c0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f21fc009ca0 tx=0x7f21fc009800 comp rx=0 tx=0).stop 2024-09-10T09:38:05.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:05.207+0000 7f220b27d740 1 -- 172.21.15.5:0/993334954 shutdown_connections 2024-09-10T09:38:05.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:05.207+0000 7f220b27d740 1 --2- 172.21.15.5:0/993334954 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564e7f18e6c0 0x564e7f190ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:05.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:05.207+0000 7f220b27d740 1 --2- 172.21.15.5:0/993334954 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564e7f183e00 0x564e7f18e180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:05.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:05.207+0000 7f220b27d740 1 --2- 172.21.15.5:0/993334954 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564e7f1834e0 0x564e7f1838c0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:05.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:05.207+0000 7f220b27d740 1 -- 172.21.15.5:0/993334954 >> 172.21.15.5:0/993334954 conn(0x564e7f17fa70 msgr2=0x564e7f17fe50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:38:05.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:05.207+0000 7f2205e00640 1 --2- 172.21.15.5:0/4049606165 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f21e80a2b80 0x7f21e80a5040 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f21f40cfcc0 tx=0x7f21f40d2f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:38:05.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:05.207+0000 7f220b27d740 1 -- 172.21.15.5:0/993334954 shutdown_connections 2024-09-10T09:38:05.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:05.207+0000 7f220b27d740 1 -- 172.21.15.5:0/993334954 wait complete. 2024-09-10T09:38:05.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:05.207+0000 7f220b27d740 1 -- start start 2024-09-10T09:38:05.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:05.208+0000 7f220b27d740 1 Processor -- start 2024-09-10T09:38:05.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:05.208+0000 7f220b27d740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564e7f1834e0 0x564e7f18b630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:05.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:05.208+0000 7f220b27d740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564e7f183e00 0x564e7f18bb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:05.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:05.208+0000 7f220b27d740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564e7f18e6c0 0x564e7f18c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:05.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:05.208+0000 7f220b27d740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x564e7f1911b0 con 0x564e7f18e6c0 2024-09-10T09:38:05.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:05.208+0000 7f220b27d740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x564e7f071920 con 0x564e7f1834e0 2024-09-10T09:38:05.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:05.208+0000 7f220b27d740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x564e7f191330 con 0x564e7f183e00 2024-09-10T09:38:05.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:05.208+0000 7f220b27d740 10 client.0 _ll_register_callbacks cb 0x564e7f0958a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:38:05.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:05.208+0000 7f2206800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564e7f18e6c0 0x564e7f18c0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:05.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:05.208+0000 7f2206800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564e7f18e6c0 0x564e7f18c0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51930/0 (socket says 172.21.15.5:51930) 2024-09-10T09:38:05.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:05.208+0000 7f2206800640 1 -- 172.21.15.5:0/877687932 learned_addr learned my addr 172.21.15.5:0/877687932 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:38:05.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:05.208+0000 7f2205400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564e7f183e00 0x564e7f18bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:05.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:05.208+0000 7f2206800640 1 --2- 172.21.15.5:0/877687932 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564e7f1834e0 0x564e7f18b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:05.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:05.209+0000 7f2205400640 1 -- 172.21.15.5:0/877687932 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564e7f1834e0 msgr2=0x564e7f18b630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:38:05.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:05.209+0000 7f2205400640 1 --2- 172.21.15.5:0/877687932 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564e7f1834e0 0x564e7f18b630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:05.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:05.209+0000 7f2205400640 1 -- 172.21.15.5:0/877687932 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564e7f18e6c0 msgr2=0x564e7f18c0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:38:05.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:05.209+0000 7f2205400640 1 --2- 172.21.15.5:0/877687932 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564e7f18e6c0 0x564e7f18c0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:05.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:05.209+0000 7f2205400640 1 -- 172.21.15.5:0/877687932 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564e7ee5a2b0 con 0x564e7f183e00 2024-09-10T09:38:05.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:05.209+0000 7f2205400640 1 --2- 172.21.15.5:0/877687932 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564e7f183e00 0x564e7f18bb70 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f21f000b9c0 tx=0x7f21f000be80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=89b3f76d65a4e6f2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:38:05.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:05.209+0000 7f21e3400640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-09-10T09:38:05.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:05.209+0000 7f21e3400640 1 -- 172.21.15.5:0/877687932 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21f0003e20 con 0x564e7f183e00 2024-09-10T09:38:05.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:05.209+0000 7f220b27d740 1 -- 172.21.15.5:0/877687932 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f21fc036040 con 0x564e7f183e00 2024-09-10T09:38:05.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:05.210+0000 7f21e3400640 1 -- 172.21.15.5:0/877687932 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f21f0005020 con 0x564e7f183e00 2024-09-10T09:38:05.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:05.210+0000 7f21e3400640 1 -- 172.21.15.5:0/877687932 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f21f0024050 con 0x564e7f183e00 2024-09-10T09:38:05.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:05.210+0000 7f220b27d740 10 client.25177 fetch_fsmap learned FSMap version 29 2024-09-10T09:38:05.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:05.210+0000 7f220b27d740 1 -- 172.21.15.5:0/877687932 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x564e7f19cf90 con 0x564e7f183e00 2024-09-10T09:38:05.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:05.210+0000 7f21e3400640 1 -- 172.21.15.5:0/877687932 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f21fc036040 con 0x564e7f183e00 2024-09-10T09:38:05.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:05.210+0000 7f21e3400640 1 -- 172.21.15.5:0/877687932 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f21f001b470 con 0x564e7f183e00 2024-09-10T09:38:05.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:05.211+0000 7f220b27d740 10 client.25177 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T09:38:05.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:05.211+0000 7f220b27d740 10 client.25177 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-10T09:38:05.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:05.211+0000 7f220b27d740 20 client.25177 populate_metadata read hostname 'smithi005' 2024-09-10T09:38:05.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:05.211+0000 7f220b27d740 1 --2- 172.21.15.5:0/877687932 >> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] conn(0x564e7f19f570 0x564e7f1a09e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:05.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:05.211+0000 7f220b27d740 4 client.25177 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:38:05.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:05.211+0000 7f220b27d740 1 -- 172.21.15.5:0/877687932 --> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564e7f0f6620 con 0x564e7f19f570 2024-09-10T09:38:05.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:05.211+0000 7f2206800640 1 --2- 172.21.15.5:0/877687932 >> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] conn(0x564e7f19f570 0x564e7f1a09e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:05.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:05.211+0000 7f2206800640 1 --2- 172.21.15.5:0/877687932 >> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] conn(0x564e7f19f570 0x564e7f1a09e0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2b60a3af7db2cf06 server_cookie=e40ceb72083b2103 in_seq=0 out_seq=0 2024-09-10T09:38:05.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:05.211+0000 7f21e3400640 10 client.25177 ms_handle_connect on v2:172.21.15.159:6836/620627639 2024-09-10T09:38:05.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:05.212+0000 7f21e3400640 1 -- 172.21.15.5:0/877687932 <== mds.2 v2:172.21.15.159:6836/620627639 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564e7f0f6620 con 0x564e7f19f570 2024-09-10T09:38:05.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:05.212+0000 7f21e3400640 10 client.25177 handle_command_reply: tid=0 2024-09-10T09:38:05.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:05.212+0000 7f220b27d740 1 client.25177 shutdown 2024-09-10T09:38:05.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:05.212+0000 7f220b27d740 1 -- 172.21.15.5:0/877687932 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564e7f183e00 msgr2=0x564e7f18bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:38:05.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:05.212+0000 7f220b27d740 1 --2- 172.21.15.5:0/877687932 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564e7f183e00 0x564e7f18bb70 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f21f000b9c0 tx=0x7f21f000be80 comp rx=0 tx=0).stop 2024-09-10T09:38:05.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:05.213+0000 7f220b27d740 1 -- 172.21.15.5:0/877687932 shutdown_connections 2024-09-10T09:38:05.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:05.213+0000 7f220b27d740 1 --2- 172.21.15.5:0/877687932 >> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] conn(0x564e7f19f570 0x564e7f1a09e0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:05.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:05.213+0000 7f220b27d740 1 --2- 172.21.15.5:0/877687932 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564e7f18e6c0 0x564e7f18c0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:05.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:05.213+0000 7f220b27d740 1 --2- 172.21.15.5:0/877687932 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564e7f183e00 0x564e7f18bb70 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:05.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:05.213+0000 7f220b27d740 1 --2- 172.21.15.5:0/877687932 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564e7f1834e0 0x564e7f18b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:05.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:05.213+0000 7f21e3400640 0 client.25177 ms_handle_reset on v2:172.21.15.159:6836/620627639 2024-09-10T09:38:05.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:05.213+0000 7f220b27d740 1 -- 172.21.15.5:0/877687932 >> 172.21.15.5:0/877687932 conn(0x564e7f17fa70 msgr2=0x564e7f17ebd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:38:05.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:05.213+0000 7f220b27d740 1 -- 172.21.15.5:0/877687932 shutdown_connections 2024-09-10T09:38:05.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:05.213+0000 7f220b27d740 1 -- 172.21.15.5:0/877687932 wait complete. 2024-09-10T09:38:05.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:05.213+0000 7f220b27d740 20 client.25177 trim_cache size 0 max 16384 2024-09-10T09:38:05.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:05.232+0000 7f220b27d740 1 Processor -- start 2024-09-10T09:38:05.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:05.232+0000 7f220b27d740 1 -- start start 2024-09-10T09:38:05.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:05.232+0000 7f220b27d740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564e7f1834e0 0x564e7f185f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:05.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:05.232+0000 7f220b27d740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564e7f183e00 0x564e7f17fe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:05.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:05.232+0000 7f220b27d740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564e7f18e6c0 0x564e7f18d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:05.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:05.232+0000 7f220b27d740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x564e7f19ed20 con 0x564e7f18e6c0 2024-09-10T09:38:05.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:05.232+0000 7f220b27d740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x564e7f19f3f0 con 0x564e7f183e00 2024-09-10T09:38:05.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:05.232+0000 7f220b27d740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x564e7f19fac0 con 0x564e7f1834e0 2024-09-10T09:38:05.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:05.233+0000 7f2206800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564e7f18e6c0 0x564e7f18d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:05.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:05.233+0000 7f2205400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564e7f183e00 0x564e7f17fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:05.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:05.233+0000 7f2206800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564e7f18e6c0 0x564e7f18d5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51942/0 (socket says 172.21.15.5:51942) 2024-09-10T09:38:05.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:05.233+0000 7f2206800640 1 -- 172.21.15.5:0/192198064 learned_addr learned my addr 172.21.15.5:0/192198064 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:38:05.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:05.233+0000 7f2206800640 1 --2- 172.21.15.5:0/192198064 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564e7f1834e0 0x564e7f185f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:05.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:05.233+0000 7f2205400640 1 -- 172.21.15.5:0/192198064 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564e7f1834e0 msgr2=0x564e7f185f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:38:05.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:05.233+0000 7f2205400640 1 --2- 172.21.15.5:0/192198064 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564e7f1834e0 0x564e7f185f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:05.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:05.233+0000 7f2205400640 1 -- 172.21.15.5:0/192198064 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564e7f18e6c0 msgr2=0x564e7f18d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:38:05.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:05.233+0000 7f2205400640 1 --2- 172.21.15.5:0/192198064 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564e7f18e6c0 0x564e7f18d5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:05.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:05.233+0000 7f2205400640 1 -- 172.21.15.5:0/192198064 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564e7f18a5e0 con 0x564e7f183e00 2024-09-10T09:38:05.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:05.233+0000 7f2206800640 1 --2- 172.21.15.5:0/192198064 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564e7f18e6c0 0x564e7f18d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:38:05.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:05.233+0000 7f2205400640 1 --2- 172.21.15.5:0/192198064 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564e7f183e00 0x564e7f17fe50 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f21f001b840 tx=0x7f21f00039c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1fd2cc16115ee44d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:38:05.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:05.234+0000 7f21ee600640 1 -- 172.21.15.5:0/192198064 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21f002c020 con 0x564e7f183e00 2024-09-10T09:38:05.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:05.234+0000 7f21ee600640 1 -- 172.21.15.5:0/192198064 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f21f000b400 con 0x564e7f183e00 2024-09-10T09:38:05.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:05.234+0000 7f21ee600640 1 -- 172.21.15.5:0/192198064 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21f001bc00 con 0x564e7f183e00 2024-09-10T09:38:05.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:05.234+0000 7f220b27d740 1 -- 172.21.15.5:0/192198064 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564e7f183e00 msgr2=0x564e7f17fe50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:38:05.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:05.234+0000 7f220b27d740 1 --2- 172.21.15.5:0/192198064 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564e7f183e00 0x564e7f17fe50 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f21f001b840 tx=0x7f21f00039c0 comp rx=0 tx=0).stop 2024-09-10T09:38:05.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:05.234+0000 7f220b27d740 1 -- 172.21.15.5:0/192198064 shutdown_connections 2024-09-10T09:38:05.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:05.234+0000 7f220b27d740 1 --2- 172.21.15.5:0/192198064 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564e7f18e6c0 0x564e7f18d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:05.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:05.234+0000 7f220b27d740 1 --2- 172.21.15.5:0/192198064 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564e7f183e00 0x564e7f17fe50 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:05.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:05.234+0000 7f220b27d740 1 --2- 172.21.15.5:0/192198064 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564e7f1834e0 0x564e7f185f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:05.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:05.234+0000 7f220b27d740 1 -- 172.21.15.5:0/192198064 >> 172.21.15.5:0/192198064 conn(0x564e7f17fa70 msgr2=0x564e7f18ce90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:38:05.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:05.234+0000 7f220b27d740 1 -- 172.21.15.5:0/192198064 shutdown_connections 2024-09-10T09:38:05.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:05.234+0000 7f220b27d740 1 -- 172.21.15.5:0/192198064 wait complete. 2024-09-10T09:38:05.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:05.235+0000 7f220b27d740 1 -- start start 2024-09-10T09:38:05.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:05.235+0000 7f220b27d740 1 Processor -- start 2024-09-10T09:38:05.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:05.235+0000 7f220b27d740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564e7f1834e0 0x564e7f1a8e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:05.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:05.235+0000 7f220b27d740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564e7f183e00 0x564e7f1a93a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:05.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:05.235+0000 7f220b27d740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564e7f18e6c0 0x564e7f1a3180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:05.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:05.235+0000 7f220b27d740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x564e7f1a0860 con 0x564e7f183e00 2024-09-10T09:38:05.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:05.235+0000 7f220b27d740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x564e7f1a0190 con 0x564e7f1834e0 2024-09-10T09:38:05.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:05.235+0000 7f220b27d740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x564e7f1a0f30 con 0x564e7f18e6c0 2024-09-10T09:38:05.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:05.236+0000 7f220b27d740 10 client.0 _ll_register_callbacks cb 0x7f21f0004420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:38:05.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:05.236+0000 7f2205400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564e7f183e00 0x564e7f1a93a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:05.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:05.236+0000 7f2205400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564e7f183e00 0x564e7f1a93a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51944/0 (socket says 172.21.15.5:51944) 2024-09-10T09:38:05.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:05.236+0000 7f2205400640 1 -- 172.21.15.5:0/2686743490 learned_addr learned my addr 172.21.15.5:0/2686743490 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:38:05.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:05.236+0000 7f2206800640 1 --2- 172.21.15.5:0/2686743490 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564e7f1834e0 0x564e7f1a8e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:05.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:05.236+0000 7f2206800640 1 --2- 172.21.15.5:0/2686743490 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564e7f18e6c0 0x564e7f1a3180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:05.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:05.236+0000 7f2205400640 1 -- 172.21.15.5:0/2686743490 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564e7f18e6c0 msgr2=0x564e7f1a3180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:38:05.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:05.236+0000 7f2205400640 1 --2- 172.21.15.5:0/2686743490 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564e7f18e6c0 0x564e7f1a3180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:05.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:05.236+0000 7f2205400640 1 -- 172.21.15.5:0/2686743490 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564e7f1834e0 msgr2=0x564e7f1a8e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:38:05.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:05.236+0000 7f2205400640 1 --2- 172.21.15.5:0/2686743490 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564e7f1834e0 0x564e7f1a8e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:05.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:05.236+0000 7f2205400640 1 -- 172.21.15.5:0/2686743490 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564e7f1a3960 con 0x564e7f183e00 2024-09-10T09:38:05.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:05.236+0000 7f2205400640 1 --2- 172.21.15.5:0/2686743490 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564e7f183e00 0x564e7f1a93a0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f21f001b960 tx=0x7f21f001bb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71fcd8dfd6130deb server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:38:05.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:05.236+0000 7f21e3400640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-10T09:38:05.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:05.237+0000 7f21e3400640 1 -- 172.21.15.5:0/2686743490 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21f002c030 con 0x564e7f183e00 2024-09-10T09:38:05.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:05.237+0000 7f21e3400640 1 -- 172.21.15.5:0/2686743490 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f21f0011640 con 0x564e7f183e00 2024-09-10T09:38:05.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:05.237+0000 7f220b27d740 1 -- 172.21.15.5:0/2686743490 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f21f000b400 con 0x564e7f183e00 2024-09-10T09:38:05.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:05.237+0000 7f21e3400640 1 -- 172.21.15.5:0/2686743490 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f21f0041050 con 0x564e7f183e00 2024-09-10T09:38:05.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:05.237+0000 7f220b27d740 10 client.15150 fetch_fsmap learned FSMap version 29 2024-09-10T09:38:05.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:05.237+0000 7f220b27d740 1 -- 172.21.15.5:0/2686743490 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x564e7f20a6e0 con 0x564e7f183e00 2024-09-10T09:38:05.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:05.237+0000 7f21e3400640 1 -- 172.21.15.5:0/2686743490 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f21f000b400 con 0x564e7f183e00 2024-09-10T09:38:05.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:05.238+0000 7f21e3400640 1 -- 172.21.15.5:0/2686743490 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f21f0028800 con 0x564e7f183e00 2024-09-10T09:38:05.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:05.238+0000 7f220b27d740 10 client.15150 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T09:38:05.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:05.238+0000 7f220b27d740 10 client.15150 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-10T09:38:05.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:05.238+0000 7f220b27d740 20 client.15150 populate_metadata read hostname 'smithi005' 2024-09-10T09:38:05.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:05.238+0000 7f220b27d740 1 --2- 172.21.15.5:0/2686743490 >> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] conn(0x564e7f20eed0 0x564e7f20dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:05.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:05.238+0000 7f220b27d740 4 client.15150 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T09:38:05.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:05.238+0000 7f220b27d740 1 -- 172.21.15.5:0/2686743490 --> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] -- command(tid 0: {"prefix": "status"}) -- 0x564e7f0f0b30 con 0x564e7f20eed0 2024-09-10T09:38:05.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:05.238+0000 7f2206800640 1 --2- 172.21.15.5:0/2686743490 >> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] conn(0x564e7f20eed0 0x564e7f20dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:05.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:05.239+0000 7f2206800640 1 --2- 172.21.15.5:0/2686743490 >> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] conn(0x564e7f20eed0 0x564e7f20dbe0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6e79a60f2f2566e6 server_cookie=be55152c8e70733 in_seq=0 out_seq=0 2024-09-10T09:38:05.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:05.239+0000 7f21e3400640 10 client.15150 ms_handle_connect on v2:172.21.15.159:6836/620627639 2024-09-10T09:38:05.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:05.239+0000 7f21e3400640 1 -- 172.21.15.5:0/2686743490 <== mds.2 v2:172.21.15.159:6836/620627639 1 ==== command_reply(tid 0: 0 ) ==== 8+0+321 (crc 0 0 0) 0x564e7f0f0b30 con 0x564e7f20eed0 2024-09-10T09:38:05.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:05.239+0000 7f21e3400640 10 client.15150 handle_command_reply: tid=0 2024-09-10T09:38:05.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:05.239+0000 7f220b27d740 1 client.15150 shutdown 2024-09-10T09:38:05.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:05.239+0000 7f220b27d740 1 -- 172.21.15.5:0/2686743490 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564e7f183e00 msgr2=0x564e7f1a93a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:38:05.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:05.239+0000 7f220b27d740 1 --2- 172.21.15.5:0/2686743490 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564e7f183e00 0x564e7f1a93a0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f21f001b960 tx=0x7f21f001bb40 comp rx=0 tx=0).stop 2024-09-10T09:38:05.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:05.239+0000 7f220b27d740 1 -- 172.21.15.5:0/2686743490 shutdown_connections 2024-09-10T09:38:05.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:05.239+0000 7f220b27d740 1 --2- 172.21.15.5:0/2686743490 >> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] conn(0x564e7f20eed0 0x564e7f20dbe0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:05.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:05.239+0000 7f220b27d740 1 --2- 172.21.15.5:0/2686743490 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564e7f18e6c0 0x564e7f1a3180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:05.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:05.239+0000 7f220b27d740 1 --2- 172.21.15.5:0/2686743490 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564e7f183e00 0x564e7f1a93a0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:05.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:05.239+0000 7f220b27d740 1 --2- 172.21.15.5:0/2686743490 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564e7f1834e0 0x564e7f1a8e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:05.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:05.239+0000 7f220b27d740 1 -- 172.21.15.5:0/2686743490 >> 172.21.15.5:0/2686743490 conn(0x564e7f17fa70 msgr2=0x564e7f199da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:38:05.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:05.239+0000 7f21e3400640 0 client.15150 ms_handle_reset on v2:172.21.15.159:6836/620627639 2024-09-10T09:38:05.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:05.240+0000 7f220b27d740 1 -- 172.21.15.5:0/2686743490 shutdown_connections 2024-09-10T09:38:05.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:05.240+0000 7f220b27d740 1 -- 172.21.15.5:0/2686743490 wait complete. 2024-09-10T09:38:05.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:05.240+0000 7f220b27d740 20 client.15150 trim_cache size 0 max 16384 2024-09-10T09:38:05.244 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-10T09:38:05.244 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_fsid": "bade4b50-6f56-11ef-bcea-c7b262605968", 2024-09-10T09:38:05.245 INFO:teuthology.orchestra.run.smithi005.stdout: "whoami": 2, 2024-09-10T09:38:05.245 INFO:teuthology.orchestra.run.smithi005.stdout: "id": 24511, 2024-09-10T09:38:05.245 INFO:teuthology.orchestra.run.smithi005.stdout: "want_state": "up:active", 2024-09-10T09:38:05.245 INFO:teuthology.orchestra.run.smithi005.stdout: "state": "up:active", 2024-09-10T09:38:05.245 INFO:teuthology.orchestra.run.smithi005.stdout: "fs_name": "cephfs", 2024-09-10T09:38:05.245 INFO:teuthology.orchestra.run.smithi005.stdout: "rank_uptime": 112.437692246, 2024-09-10T09:38:05.245 INFO:teuthology.orchestra.run.smithi005.stdout: "mdsmap_epoch": 29, 2024-09-10T09:38:05.245 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch": 96, 2024-09-10T09:38:05.245 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch_barrier": 94, 2024-09-10T09:38:05.245 INFO:teuthology.orchestra.run.smithi005.stdout: "uptime": 183.87790092 2024-09-10T09:38:05.245 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-10T09:38:05.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:05.240+0000 7f2207200640 1 -- 172.21.15.5:0/4049606165 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f21e80a2b80 msgr2=0x7f21e80a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:38:05.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:05.240+0000 7f2207200640 1 --2- 172.21.15.5:0/4049606165 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f21e80a2b80 0x7f21e80a5040 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f21f40cfcc0 tx=0x7f21f40d2f90 comp rx=0 tx=0).stop 2024-09-10T09:38:05.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:05.240+0000 7f2207200640 1 -- 172.21.15.5:0/4049606165 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2200103e50 msgr2=0x7f220019a4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:38:05.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:05.240+0000 7f2207200640 1 --2- 172.21.15.5:0/4049606165 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2200103e50 0x7f220019a4f0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f21f400e9f0 tx=0x7f21f400eeb0 comp rx=0 tx=0).stop 2024-09-10T09:38:05.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:05.240+0000 7f2207200640 1 -- 172.21.15.5:0/4049606165 shutdown_connections 2024-09-10T09:38:05.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:05.240+0000 7f2207200640 1 --2- 172.21.15.5:0/4049606165 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f21e80a2b80 0x7f21e80a5040 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:05.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:05.240+0000 7f2207200640 1 --2- 172.21.15.5:0/4049606165 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2200109090 0x7f220019edc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:05.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:05.240+0000 7f2207200640 1 --2- 172.21.15.5:0/4049606165 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2200104800 0x7f220019aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:05.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:05.240+0000 7f2207200640 1 --2- 172.21.15.5:0/4049606165 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2200103e50 0x7f220019a4f0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:05.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:05.240+0000 7f2207200640 1 -- 172.21.15.5:0/4049606165 >> 172.21.15.5:0/4049606165 conn(0x7f22000ff8b0 msgr2=0x7f2200100bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:38:05.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:05.241+0000 7f2207200640 1 -- 172.21.15.5:0/4049606165 shutdown_connections 2024-09-10T09:38:05.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:05.241+0000 7f2207200640 1 -- 172.21.15.5:0/4049606165 wait complete. 2024-09-10T09:38:06.050 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph tell mds.i status 2024-09-10T09:38:06.436 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:38:06.829 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:38:06 smithi005 ceph-mon[30350]: pgmap v386: 129 pgs: 129 active+clean; 646 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-09-10T09:38:07.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:07.078+0000 7fda53200640 1 -- 172.21.15.5:0/2208725755 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fda54108f60 msgr2=0x7fda54110ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:38:07.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:07.078+0000 7fda53200640 1 --2- 172.21.15.5:0/2208725755 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fda54108f60 0x7fda54110ae0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fda4c009900 tx=0x7fda4c02a410 comp rx=0 tx=0).stop 2024-09-10T09:38:07.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:07.078+0000 7fda53200640 1 -- 172.21.15.5:0/2208725755 shutdown_connections 2024-09-10T09:38:07.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:07.078+0000 7fda53200640 1 --2- 172.21.15.5:0/2208725755 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fda54108f60 0x7fda54110ae0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:07.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:07.078+0000 7fda53200640 1 --2- 172.21.15.5:0/2208725755 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fda541047e0 0x7fda54108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:07.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:07.078+0000 7fda53200640 1 --2- 172.21.15.5:0/2208725755 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fda54103e30 0x7fda54104210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:07.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:07.078+0000 7fda53200640 1 -- 172.21.15.5:0/2208725755 >> 172.21.15.5:0/2208725755 conn(0x7fda540ff8d0 msgr2=0x7fda54101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:38:07.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:07.078+0000 7fda53200640 1 -- 172.21.15.5:0/2208725755 shutdown_connections 2024-09-10T09:38:07.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:07.078+0000 7fda53200640 1 -- 172.21.15.5:0/2208725755 wait complete. 2024-09-10T09:38:07.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:07.079+0000 7fda53200640 1 Processor -- start 2024-09-10T09:38:07.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:07.079+0000 7fda53200640 1 -- start start 2024-09-10T09:38:07.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:07.080+0000 7fda53200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fda54103e30 0x7fda54076700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:07.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:07.080+0000 7fda53200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fda541047e0 0x7fda54076c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:07.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:07.080+0000 7fda53200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fda54108f60 0x7fda54077180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:07.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:07.080+0000 7fda53200640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fda54113fc0 con 0x7fda541047e0 2024-09-10T09:38:07.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:07.080+0000 7fda53200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fda54113e40 con 0x7fda54108f60 2024-09-10T09:38:07.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:07.080+0000 7fda53200640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fda54114140 con 0x7fda54103e30 2024-09-10T09:38:07.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:07.080+0000 7fda49400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fda541047e0 0x7fda54076c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:07.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:07.080+0000 7fda51e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fda54103e30 0x7fda54076700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:07.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:07.080+0000 7fda52800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fda54108f60 0x7fda54077180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:07.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:07.080+0000 7fda51e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fda54103e30 0x7fda54076700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:46506/0 (socket says 172.21.15.5:46506) 2024-09-10T09:38:07.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:07.080+0000 7fda49400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fda541047e0 0x7fda54076c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51964/0 (socket says 172.21.15.5:51964) 2024-09-10T09:38:07.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:07.080+0000 7fda51e00640 1 -- 172.21.15.5:0/2039111600 learned_addr learned my addr 172.21.15.5:0/2039111600 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:38:07.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:07.080+0000 7fda51e00640 1 -- 172.21.15.5:0/2039111600 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fda54108f60 msgr2=0x7fda54077180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:38:07.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:07.080+0000 7fda51e00640 1 --2- 172.21.15.5:0/2039111600 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fda54108f60 0x7fda54077180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:07.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:07.080+0000 7fda51e00640 1 -- 172.21.15.5:0/2039111600 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fda541047e0 msgr2=0x7fda54076c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:38:07.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:07.080+0000 7fda51e00640 1 --2- 172.21.15.5:0/2039111600 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fda541047e0 0x7fda54076c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:07.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:07.081+0000 7fda51e00640 1 -- 172.21.15.5:0/2039111600 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda541aaaa0 con 0x7fda54103e30 2024-09-10T09:38:07.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:07.081+0000 7fda49400640 1 --2- 172.21.15.5:0/2039111600 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fda541047e0 0x7fda54076c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:38:07.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:07.081+0000 7fda51e00640 1 --2- 172.21.15.5:0/2039111600 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fda54103e30 0x7fda54076700 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fda4400ebc0 tx=0x7fda4400c6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:38:07.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:07.081+0000 7fda4b400640 1 -- 172.21.15.5:0/2039111600 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda44004d20 con 0x7fda54103e30 2024-09-10T09:38:07.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:07.081+0000 7fda53200640 1 -- 172.21.15.5:0/2039111600 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda541aad90 con 0x7fda54103e30 2024-09-10T09:38:07.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:07.081+0000 7fda4b400640 1 -- 172.21.15.5:0/2039111600 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fda44005500 con 0x7fda54103e30 2024-09-10T09:38:07.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:07.081+0000 7fda4b400640 1 -- 172.21.15.5:0/2039111600 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda440107f0 con 0x7fda54103e30 2024-09-10T09:38:07.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:07.081+0000 7fda53200640 1 -- 172.21.15.5:0/2039111600 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda541ab270 con 0x7fda54103e30 2024-09-10T09:38:07.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:07.084+0000 7fda5bbf9740 1 Processor -- start 2024-09-10T09:38:07.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:07.085+0000 7fda5bbf9740 1 -- start start 2024-09-10T09:38:07.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:07.085+0000 7fda4b400640 1 -- 172.21.15.5:0/2039111600 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7fda440056b0 con 0x7fda54103e30 2024-09-10T09:38:07.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:07.085+0000 7fda5bbf9740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556c90be24e0 0x556c90be28c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:07.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:07.085+0000 7fda5bbf9740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556c90be2e00 0x556c90bed180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:07.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:07.085+0000 7fda5bbf9740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556c90bed6c0 0x556c90befab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:07.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:07.085+0000 7fda5bbf9740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x556c90b480b0 con 0x556c90be24e0 2024-09-10T09:38:07.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:07.085+0000 7fda5bbf9740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x556c90aefab0 con 0x556c90be2e00 2024-09-10T09:38:07.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:07.085+0000 7fda5bbf9740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x556c90b44070 con 0x556c90bed6c0 2024-09-10T09:38:07.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:07.085+0000 7fda52800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556c90be24e0 0x556c90be28c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:07.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:07.085+0000 7fda52800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556c90be24e0 0x556c90be28c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51966/0 (socket says 172.21.15.5:51966) 2024-09-10T09:38:07.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:07.085+0000 7fda52800640 1 -- 172.21.15.5:0/64014304 learned_addr learned my addr 172.21.15.5:0/64014304 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:38:07.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:07.085+0000 7fda51e00640 1 --2- 172.21.15.5:0/64014304 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556c90be2e00 0x556c90bed180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:07.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:07.086+0000 7fda49400640 1 --2- 172.21.15.5:0/64014304 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556c90bed6c0 0x556c90befab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:07.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:07.086+0000 7fda52800640 1 -- 172.21.15.5:0/64014304 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556c90bed6c0 msgr2=0x556c90befab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:38:07.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:07.086+0000 7fda52800640 1 --2- 172.21.15.5:0/64014304 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556c90bed6c0 0x556c90befab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:07.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:07.086+0000 7fda52800640 1 -- 172.21.15.5:0/64014304 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556c90be2e00 msgr2=0x556c90bed180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:38:07.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:07.086+0000 7fda52800640 1 --2- 172.21.15.5:0/64014304 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556c90be2e00 0x556c90bed180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:07.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:07.086+0000 7fda52800640 1 -- 172.21.15.5:0/64014304 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556c90afc4a0 con 0x556c90be24e0 2024-09-10T09:38:07.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:07.086+0000 7fda52800640 1 --2- 172.21.15.5:0/64014304 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556c90be24e0 0x556c90be28c0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fda4c033000 tx=0x7fda4c005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87883b39e3b36ba server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:38:07.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:07.087+0000 7fda43400640 1 -- 172.21.15.5:0/64014304 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda4c040070 con 0x556c90be24e0 2024-09-10T09:38:07.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:07.087+0000 7fda43400640 1 -- 172.21.15.5:0/64014304 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fda4c0317b0 con 0x556c90be24e0 2024-09-10T09:38:07.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:07.087+0000 7fda43400640 1 -- 172.21.15.5:0/64014304 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda4c035040 con 0x556c90be24e0 2024-09-10T09:38:07.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:07.087+0000 7fda4b400640 1 --2- 172.21.15.5:0/2039111600 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fda380a2c00 0x7fda380a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:07.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:07.087+0000 7fda4b400640 1 -- 172.21.15.5:0/2039111600 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fda440cfa10 con 0x7fda54103e30 2024-09-10T09:38:07.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:07.087+0000 7fda51e00640 1 --2- 172.21.15.5:0/2039111600 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fda380a2c00 0x7fda380a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:07.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:07.087+0000 7fda5bbf9740 1 -- 172.21.15.5:0/64014304 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556c90be24e0 msgr2=0x556c90be28c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:38:07.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:07.087+0000 7fda5bbf9740 1 --2- 172.21.15.5:0/64014304 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556c90be24e0 0x556c90be28c0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fda4c033000 tx=0x7fda4c005d50 comp rx=0 tx=0).stop 2024-09-10T09:38:07.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:07.087+0000 7fda5bbf9740 1 -- 172.21.15.5:0/64014304 shutdown_connections 2024-09-10T09:38:07.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:07.087+0000 7fda5bbf9740 1 --2- 172.21.15.5:0/64014304 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556c90bed6c0 0x556c90befab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:07.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:07.087+0000 7fda5bbf9740 1 --2- 172.21.15.5:0/64014304 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556c90be2e00 0x556c90bed180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:07.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:07.087+0000 7fda5bbf9740 1 --2- 172.21.15.5:0/64014304 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556c90be24e0 0x556c90be28c0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:07.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:07.087+0000 7fda5bbf9740 1 -- 172.21.15.5:0/64014304 >> 172.21.15.5:0/64014304 conn(0x556c90bdea70 msgr2=0x556c90bdee50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:38:07.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:07.088+0000 7fda5bbf9740 1 -- 172.21.15.5:0/64014304 shutdown_connections 2024-09-10T09:38:07.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:07.088+0000 7fda5bbf9740 1 -- 172.21.15.5:0/64014304 wait complete. 2024-09-10T09:38:07.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:07.088+0000 7fda51e00640 1 --2- 172.21.15.5:0/2039111600 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fda380a2c00 0x7fda380a50c0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fda440cd0c0 tx=0x7fda440d54b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:38:07.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:07.088+0000 7fda5bbf9740 1 -- start start 2024-09-10T09:38:07.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:07.088+0000 7fda5bbf9740 1 Processor -- start 2024-09-10T09:38:07.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:07.089+0000 7fda5bbf9740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556c90be24e0 0x556c90bea4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:07.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:07.089+0000 7fda5bbf9740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556c90be2e00 0x556c90beaa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:07.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:07.089+0000 7fda5bbf9740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556c90bed6c0 0x556c90beaf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:07.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:07.089+0000 7fda5bbf9740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x556c90bf0330 con 0x556c90bed6c0 2024-09-10T09:38:07.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:07.089+0000 7fda5bbf9740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x556c90ad0920 con 0x556c90be2e00 2024-09-10T09:38:07.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:07.089+0000 7fda5bbf9740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x556c90bf04b0 con 0x556c90be24e0 2024-09-10T09:38:07.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:07.089+0000 7fda5bbf9740 10 client.0 _ll_register_callbacks cb 0x556c90af48a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:38:07.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:07.089+0000 7fda52800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556c90be24e0 0x556c90bea4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:07.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:07.089+0000 7fda49400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556c90be2e00 0x556c90beaa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:07.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:07.089+0000 7fda49400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556c90be2e00 0x556c90beaa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:45192/0 (socket says 172.21.15.5:45192) 2024-09-10T09:38:07.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:07.089+0000 7fda49400640 1 -- 172.21.15.5:0/569896797 learned_addr learned my addr 172.21.15.5:0/569896797 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:38:07.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:07.089+0000 7fda52800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556c90be24e0 0x556c90bea4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:46522/0 (socket says 172.21.15.5:46522) 2024-09-10T09:38:07.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:07.089+0000 7fda52800640 1 --2- 172.21.15.5:0/569896797 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556c90bed6c0 0x556c90beaf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:07.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:07.090+0000 7fda49400640 1 -- 172.21.15.5:0/569896797 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556c90be24e0 msgr2=0x556c90bea4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:38:07.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:07.090+0000 7fda49400640 1 --2- 172.21.15.5:0/569896797 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556c90be24e0 0x556c90bea4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:07.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:07.090+0000 7fda49400640 1 -- 172.21.15.5:0/569896797 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556c90bed6c0 msgr2=0x556c90beaf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:38:07.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:07.090+0000 7fda49400640 1 --2- 172.21.15.5:0/569896797 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556c90bed6c0 0x556c90beaf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:07.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:07.090+0000 7fda49400640 1 -- 172.21.15.5:0/569896797 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556c908b92b0 con 0x556c90be2e00 2024-09-10T09:38:07.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:07.090+0000 7fda52800640 1 --2- 172.21.15.5:0/569896797 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556c90bed6c0 0x556c90beaf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:38:07.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:07.090+0000 7fda49400640 1 --2- 172.21.15.5:0/569896797 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556c90be2e00 0x556c90beaa20 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fda3c012d20 tx=0x7fda3c00f790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e0a87c0928302e2e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:38:07.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:07.090+0000 7fda37e00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-09-10T09:38:07.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:07.090+0000 7fda37e00640 1 -- 172.21.15.5:0/569896797 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda3c012110 con 0x556c90be2e00 2024-09-10T09:38:07.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:07.090+0000 7fda5bbf9740 1 -- 172.21.15.5:0/569896797 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fda4c0317b0 con 0x556c90be2e00 2024-09-10T09:38:07.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:07.090+0000 7fda37e00640 1 -- 172.21.15.5:0/569896797 <== mon.2 v2:172.21.15.102:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fda3c004050 con 0x556c90be2e00 2024-09-10T09:38:07.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:07.091+0000 7fda37e00640 1 -- 172.21.15.5:0/569896797 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fda3c0122b0 con 0x556c90be2e00 2024-09-10T09:38:07.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:07.091+0000 7fda5bbf9740 10 client.25178 fetch_fsmap learned FSMap version 29 2024-09-10T09:38:07.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:07.091+0000 7fda5bbf9740 1 -- 172.21.15.5:0/569896797 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x556c90bfbff0 con 0x556c90be2e00 2024-09-10T09:38:07.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:07.091+0000 7fda37e00640 1 -- 172.21.15.5:0/569896797 <== mon.2 v2:172.21.15.102:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fda4c0317b0 con 0x556c90be2e00 2024-09-10T09:38:07.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:07.091+0000 7fda37e00640 1 -- 172.21.15.5:0/569896797 <== mon.2 v2:172.21.15.102:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7fda3c010430 con 0x556c90be2e00 2024-09-10T09:38:07.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:07.091+0000 7fda5bbf9740 10 client.25178 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T09:38:07.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:07.091+0000 7fda5bbf9740 10 client.25178 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-10T09:38:07.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:07.091+0000 7fda5bbf9740 20 client.25178 populate_metadata read hostname 'smithi005' 2024-09-10T09:38:07.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:07.091+0000 7fda5bbf9740 1 --2- 172.21.15.5:0/569896797 >> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] conn(0x556c90bfe620 0x556c90bffa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:07.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:07.091+0000 7fda5bbf9740 4 client.25178 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:38:07.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:07.091+0000 7fda5bbf9740 1 -- 172.21.15.5:0/569896797 --> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556c90b55620 con 0x556c90bfe620 2024-09-10T09:38:07.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:07.092+0000 7fda52800640 1 --2- 172.21.15.5:0/569896797 >> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] conn(0x556c90bfe620 0x556c90bffa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:07.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:07.092+0000 7fda52800640 1 --2- 172.21.15.5:0/569896797 >> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] conn(0x556c90bfe620 0x556c90bffa90 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=31f821586acaa3a3 server_cookie=c15d5924299c873d in_seq=0 out_seq=0 2024-09-10T09:38:07.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:07.092+0000 7fda37e00640 10 client.25178 ms_handle_connect on v2:172.21.15.159:6838/2073739599 2024-09-10T09:38:07.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:07.093+0000 7fda37e00640 1 -- 172.21.15.5:0/569896797 <== mds.1 v2:172.21.15.159:6838/2073739599 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556c90b55620 con 0x556c90bfe620 2024-09-10T09:38:07.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:07.093+0000 7fda37e00640 10 client.25178 handle_command_reply: tid=0 2024-09-10T09:38:07.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:07.093+0000 7fda5bbf9740 1 client.25178 shutdown 2024-09-10T09:38:07.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:07.093+0000 7fda5bbf9740 1 -- 172.21.15.5:0/569896797 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556c90be2e00 msgr2=0x556c90beaa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:38:07.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:07.093+0000 7fda5bbf9740 1 --2- 172.21.15.5:0/569896797 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556c90be2e00 0x556c90beaa20 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fda3c012d20 tx=0x7fda3c00f790 comp rx=0 tx=0).stop 2024-09-10T09:38:07.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:07.093+0000 7fda5bbf9740 1 -- 172.21.15.5:0/569896797 shutdown_connections 2024-09-10T09:38:07.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:07.093+0000 7fda5bbf9740 1 --2- 172.21.15.5:0/569896797 >> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] conn(0x556c90bfe620 0x556c90bffa90 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:07.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:07.093+0000 7fda5bbf9740 1 --2- 172.21.15.5:0/569896797 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556c90bed6c0 0x556c90beaf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:07.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:07.093+0000 7fda5bbf9740 1 --2- 172.21.15.5:0/569896797 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556c90be2e00 0x556c90beaa20 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:07.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:07.093+0000 7fda37e00640 0 client.25178 ms_handle_reset on v2:172.21.15.159:6838/2073739599 2024-09-10T09:38:07.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:07.093+0000 7fda5bbf9740 1 --2- 172.21.15.5:0/569896797 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556c90be24e0 0x556c90bea4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:07.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:07.093+0000 7fda5bbf9740 1 -- 172.21.15.5:0/569896797 >> 172.21.15.5:0/569896797 conn(0x556c90bdea70 msgr2=0x556c90bddbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:38:07.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:07.094+0000 7fda5bbf9740 1 -- 172.21.15.5:0/569896797 shutdown_connections 2024-09-10T09:38:07.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:07.094+0000 7fda5bbf9740 1 -- 172.21.15.5:0/569896797 wait complete. 2024-09-10T09:38:07.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:07.094+0000 7fda5bbf9740 20 client.25178 trim_cache size 0 max 16384 2024-09-10T09:38:07.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:38:06 smithi159 ceph-mon[35513]: pgmap v386: 129 pgs: 129 active+clean; 646 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-09-10T09:38:07.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:07.112+0000 7fda5bbf9740 1 Processor -- start 2024-09-10T09:38:07.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:07.113+0000 7fda5bbf9740 1 -- start start 2024-09-10T09:38:07.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:07.113+0000 7fda5bbf9740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556c90be24e0 0x556c90bf9870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:07.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:07.113+0000 7fda5bbf9740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556c90be2e00 0x556c90beadb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:07.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:07.113+0000 7fda5bbf9740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556c90bed6c0 0x556c90be94c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:07.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:07.113+0000 7fda5bbf9740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x556c90bfddd0 con 0x556c90be24e0 2024-09-10T09:38:07.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:07.113+0000 7fda5bbf9740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x556c90bfe4a0 con 0x556c90be2e00 2024-09-10T09:38:07.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:07.113+0000 7fda5bbf9740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x556c90bfeb70 con 0x556c90bed6c0 2024-09-10T09:38:07.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:07.113+0000 7fda52800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556c90be24e0 0x556c90bf9870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:07.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:07.113+0000 7fda52800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556c90be24e0 0x556c90bf9870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51984/0 (socket says 172.21.15.5:51984) 2024-09-10T09:38:07.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:07.113+0000 7fda52800640 1 -- 172.21.15.5:0/3443369571 learned_addr learned my addr 172.21.15.5:0/3443369571 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:38:07.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:07.113+0000 7fda49400640 1 --2- 172.21.15.5:0/3443369571 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556c90be2e00 0x556c90beadb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:07.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:07.113+0000 7fda52800640 1 --2- 172.21.15.5:0/3443369571 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556c90bed6c0 0x556c90be94c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:07.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:07.114+0000 7fda52800640 1 -- 172.21.15.5:0/3443369571 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556c90bed6c0 msgr2=0x556c90be94c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:38:07.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:07.114+0000 7fda52800640 1 --2- 172.21.15.5:0/3443369571 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556c90bed6c0 0x556c90be94c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:07.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:07.114+0000 7fda52800640 1 -- 172.21.15.5:0/3443369571 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556c90be2e00 msgr2=0x556c90beadb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:38:07.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:07.114+0000 7fda52800640 1 --2- 172.21.15.5:0/3443369571 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556c90be2e00 0x556c90beadb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:07.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:07.114+0000 7fda52800640 1 -- 172.21.15.5:0/3443369571 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556c90bde8a0 con 0x556c90be24e0 2024-09-10T09:38:07.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:07.114+0000 7fda49400640 1 --2- 172.21.15.5:0/3443369571 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556c90be2e00 0x556c90beadb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:38:07.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:07.114+0000 7fda52800640 1 --2- 172.21.15.5:0/3443369571 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556c90be24e0 0x556c90bf9870 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fda4c0054d0 tx=0x7fda4c02a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=334a39404ec1460e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:38:07.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:07.114+0000 7fda42a00640 1 -- 172.21.15.5:0/3443369571 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda4c035040 con 0x556c90be24e0 2024-09-10T09:38:07.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:07.114+0000 7fda42a00640 1 -- 172.21.15.5:0/3443369571 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fda4c005570 con 0x556c90be24e0 2024-09-10T09:38:07.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:07.114+0000 7fda42a00640 1 -- 172.21.15.5:0/3443369571 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda4c040040 con 0x556c90be24e0 2024-09-10T09:38:07.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:07.115+0000 7fda5bbf9740 1 -- 172.21.15.5:0/3443369571 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556c90be24e0 msgr2=0x556c90bf9870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:38:07.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:07.115+0000 7fda5bbf9740 1 --2- 172.21.15.5:0/3443369571 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556c90be24e0 0x556c90bf9870 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fda4c0054d0 tx=0x7fda4c02a410 comp rx=0 tx=0).stop 2024-09-10T09:38:07.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:07.115+0000 7fda5bbf9740 1 -- 172.21.15.5:0/3443369571 shutdown_connections 2024-09-10T09:38:07.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:07.115+0000 7fda5bbf9740 1 --2- 172.21.15.5:0/3443369571 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556c90bed6c0 0x556c90be94c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:07.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:07.115+0000 7fda5bbf9740 1 --2- 172.21.15.5:0/3443369571 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556c90be2e00 0x556c90beadb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:07.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:07.115+0000 7fda5bbf9740 1 --2- 172.21.15.5:0/3443369571 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556c90be24e0 0x556c90bf9870 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:07.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:07.115+0000 7fda5bbf9740 1 -- 172.21.15.5:0/3443369571 >> 172.21.15.5:0/3443369571 conn(0x556c90bdea70 msgr2=0x556c90be8db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:38:07.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:07.115+0000 7fda5bbf9740 1 -- 172.21.15.5:0/3443369571 shutdown_connections 2024-09-10T09:38:07.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:07.115+0000 7fda5bbf9740 1 -- 172.21.15.5:0/3443369571 wait complete. 2024-09-10T09:38:07.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:07.115+0000 7fda5bbf9740 1 -- start start 2024-09-10T09:38:07.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:07.116+0000 7fda5bbf9740 1 Processor -- start 2024-09-10T09:38:07.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:07.116+0000 7fda5bbf9740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556c90be24e0 0x556c90c082e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:07.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:07.116+0000 7fda5bbf9740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556c90be2e00 0x556c90be7060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:07.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:07.116+0000 7fda5bbf9740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556c90bed6c0 0x556c90be75a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:07.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:07.116+0000 7fda5bbf9740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x556c90bff910 con 0x556c90be24e0 2024-09-10T09:38:07.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:07.116+0000 7fda5bbf9740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x556c90bff240 con 0x556c90bed6c0 2024-09-10T09:38:07.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:07.116+0000 7fda5bbf9740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x556c90bfffe0 con 0x556c90be2e00 2024-09-10T09:38:07.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:07.116+0000 7fda5bbf9740 10 client.0 _ll_register_callbacks cb 0x7fda3c012930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:38:07.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:07.116+0000 7fda52800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556c90be24e0 0x556c90c082e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:07.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:07.117+0000 7fda52800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556c90be24e0 0x556c90c082e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52000/0 (socket says 172.21.15.5:52000) 2024-09-10T09:38:07.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:07.117+0000 7fda52800640 1 -- 172.21.15.5:0/3336231458 learned_addr learned my addr 172.21.15.5:0/3336231458 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:38:07.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:07.117+0000 7fda49400640 1 --2- 172.21.15.5:0/3336231458 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556c90be2e00 0x556c90be7060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:07.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:07.117+0000 7fda52800640 1 --2- 172.21.15.5:0/3336231458 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556c90bed6c0 0x556c90be75a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:07.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:07.117+0000 7fda49400640 1 -- 172.21.15.5:0/3336231458 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556c90bed6c0 msgr2=0x556c90be75a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:38:07.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:07.117+0000 7fda49400640 1 --2- 172.21.15.5:0/3336231458 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556c90bed6c0 0x556c90be75a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:07.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:07.117+0000 7fda49400640 1 -- 172.21.15.5:0/3336231458 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556c90be24e0 msgr2=0x556c90c082e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:38:07.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:07.117+0000 7fda49400640 1 --2- 172.21.15.5:0/3336231458 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556c90be24e0 0x556c90c082e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:07.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:07.117+0000 7fda49400640 1 -- 172.21.15.5:0/3336231458 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556c90be7b90 con 0x556c90be2e00 2024-09-10T09:38:07.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:07.117+0000 7fda49400640 1 --2- 172.21.15.5:0/3336231458 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556c90be2e00 0x556c90be7060 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x556c90beba90 tx=0x7fda3c010660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=343ea77429cd7c6b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:38:07.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:07.117+0000 7fda37e00640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-09-10T09:38:07.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:07.118+0000 7fda37e00640 1 -- 172.21.15.5:0/3336231458 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda3c02d020 con 0x556c90be2e00 2024-09-10T09:38:07.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:07.118+0000 7fda5bbf9740 1 -- 172.21.15.5:0/3336231458 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fda4c005570 con 0x556c90be2e00 2024-09-10T09:38:07.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:07.118+0000 7fda37e00640 1 -- 172.21.15.5:0/3336231458 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fda3c005320 con 0x556c90be2e00 2024-09-10T09:38:07.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:07.118+0000 7fda37e00640 1 -- 172.21.15.5:0/3336231458 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fda3c012110 con 0x556c90be2e00 2024-09-10T09:38:07.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:07.118+0000 7fda5bbf9740 10 client.25201 fetch_fsmap learned FSMap version 29 2024-09-10T09:38:07.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:07.118+0000 7fda5bbf9740 1 -- 172.21.15.5:0/3336231458 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x556c90c68d30 con 0x556c90be2e00 2024-09-10T09:38:07.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:07.119+0000 7fda37e00640 1 -- 172.21.15.5:0/3336231458 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fda4c005570 con 0x556c90be2e00 2024-09-10T09:38:07.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:07.119+0000 7fda37e00640 1 -- 172.21.15.5:0/3336231458 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7fda3c003cd0 con 0x556c90be2e00 2024-09-10T09:38:07.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:07.119+0000 7fda5bbf9740 10 client.25201 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T09:38:07.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:07.119+0000 7fda5bbf9740 10 client.25201 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-10T09:38:07.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:07.119+0000 7fda5bbf9740 20 client.25201 populate_metadata read hostname 'smithi005' 2024-09-10T09:38:07.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:07.119+0000 7fda5bbf9740 1 --2- 172.21.15.5:0/3336231458 >> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] conn(0x556c90c6d770 0x556c90c6c480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:07.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:07.119+0000 7fda5bbf9740 4 client.25201 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T09:38:07.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:07.119+0000 7fda5bbf9740 1 -- 172.21.15.5:0/3336231458 --> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] -- command(tid 0: {"prefix": "status"}) -- 0x556c90b4fb30 con 0x556c90c6d770 2024-09-10T09:38:07.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:07.119+0000 7fda52800640 1 --2- 172.21.15.5:0/3336231458 >> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] conn(0x556c90c6d770 0x556c90c6c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:07.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:07.120+0000 7fda52800640 1 --2- 172.21.15.5:0/3336231458 >> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] conn(0x556c90c6d770 0x556c90c6c480 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=89f3d2f1256ea5b6 server_cookie=8e729b2ed28c463f in_seq=0 out_seq=0 2024-09-10T09:38:07.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:07.120+0000 7fda37e00640 10 client.25201 ms_handle_connect on v2:172.21.15.159:6838/2073739599 2024-09-10T09:38:07.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:07.120+0000 7fda37e00640 1 -- 172.21.15.5:0/3336231458 <== mds.1 v2:172.21.15.159:6838/2073739599 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x556c90b4fb30 con 0x556c90c6d770 2024-09-10T09:38:07.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:07.120+0000 7fda37e00640 10 client.25201 handle_command_reply: tid=0 2024-09-10T09:38:07.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:07.120+0000 7fda5bbf9740 1 client.25201 shutdown 2024-09-10T09:38:07.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:07.120+0000 7fda5bbf9740 1 -- 172.21.15.5:0/3336231458 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556c90be2e00 msgr2=0x556c90be7060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:38:07.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:07.120+0000 7fda5bbf9740 1 --2- 172.21.15.5:0/3336231458 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556c90be2e00 0x556c90be7060 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x556c90beba90 tx=0x7fda3c010660 comp rx=0 tx=0).stop 2024-09-10T09:38:07.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:07.120+0000 7fda5bbf9740 1 -- 172.21.15.5:0/3336231458 shutdown_connections 2024-09-10T09:38:07.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:07.120+0000 7fda5bbf9740 1 --2- 172.21.15.5:0/3336231458 >> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] conn(0x556c90c6d770 0x556c90c6c480 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:07.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:07.120+0000 7fda5bbf9740 1 --2- 172.21.15.5:0/3336231458 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556c90bed6c0 0x556c90be75a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:07.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:07.120+0000 7fda5bbf9740 1 --2- 172.21.15.5:0/3336231458 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556c90be2e00 0x556c90be7060 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:07.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:07.120+0000 7fda5bbf9740 1 --2- 172.21.15.5:0/3336231458 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556c90be24e0 0x556c90c082e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:07.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:07.120+0000 7fda37e00640 0 client.25201 ms_handle_reset on v2:172.21.15.159:6838/2073739599 2024-09-10T09:38:07.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:07.121+0000 7fda5bbf9740 1 -- 172.21.15.5:0/3336231458 >> 172.21.15.5:0/3336231458 conn(0x556c90bdea70 msgr2=0x556c90be95d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:38:07.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:07.121+0000 7fda5bbf9740 1 -- 172.21.15.5:0/3336231458 shutdown_connections 2024-09-10T09:38:07.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:07.121+0000 7fda5bbf9740 1 -- 172.21.15.5:0/3336231458 wait complete. 2024-09-10T09:38:07.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:07.121+0000 7fda5bbf9740 20 client.25201 trim_cache size 0 max 16384 2024-09-10T09:38:07.126 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-10T09:38:07.126 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_fsid": "bade4b50-6f56-11ef-bcea-c7b262605968", 2024-09-10T09:38:07.126 INFO:teuthology.orchestra.run.smithi005.stdout: "whoami": 1, 2024-09-10T09:38:07.126 INFO:teuthology.orchestra.run.smithi005.stdout: "id": 24535, 2024-09-10T09:38:07.126 INFO:teuthology.orchestra.run.smithi005.stdout: "want_state": "up:active", 2024-09-10T09:38:07.126 INFO:teuthology.orchestra.run.smithi005.stdout: "state": "up:active", 2024-09-10T09:38:07.126 INFO:teuthology.orchestra.run.smithi005.stdout: "fs_name": "cephfs", 2024-09-10T09:38:07.126 INFO:teuthology.orchestra.run.smithi005.stdout: "rank_uptime": 115.32481279700001, 2024-09-10T09:38:07.126 INFO:teuthology.orchestra.run.smithi005.stdout: "mdsmap_epoch": 29, 2024-09-10T09:38:07.126 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch": 96, 2024-09-10T09:38:07.126 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch_barrier": 94, 2024-09-10T09:38:07.126 INFO:teuthology.orchestra.run.smithi005.stdout: "uptime": 180.26867064499999 2024-09-10T09:38:07.127 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-10T09:38:07.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:07.122+0000 7fda53200640 1 -- 172.21.15.5:0/2039111600 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fda380a2c00 msgr2=0x7fda380a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:38:07.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:07.122+0000 7fda53200640 1 --2- 172.21.15.5:0/2039111600 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fda380a2c00 0x7fda380a50c0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fda440cd0c0 tx=0x7fda440d54b0 comp rx=0 tx=0).stop 2024-09-10T09:38:07.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:07.122+0000 7fda53200640 1 -- 172.21.15.5:0/2039111600 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fda54103e30 msgr2=0x7fda54076700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:38:07.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:07.122+0000 7fda53200640 1 --2- 172.21.15.5:0/2039111600 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fda54103e30 0x7fda54076700 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fda4400ebc0 tx=0x7fda4400c6a0 comp rx=0 tx=0).stop 2024-09-10T09:38:07.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:07.122+0000 7fda53200640 1 -- 172.21.15.5:0/2039111600 shutdown_connections 2024-09-10T09:38:07.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:07.122+0000 7fda53200640 1 --2- 172.21.15.5:0/2039111600 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fda380a2c00 0x7fda380a50c0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:07.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:07.122+0000 7fda53200640 1 --2- 172.21.15.5:0/2039111600 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fda54108f60 0x7fda54077180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:07.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:07.122+0000 7fda53200640 1 --2- 172.21.15.5:0/2039111600 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fda541047e0 0x7fda54076c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:07.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:07.122+0000 7fda53200640 1 --2- 172.21.15.5:0/2039111600 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fda54103e30 0x7fda54076700 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:07.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:07.122+0000 7fda53200640 1 -- 172.21.15.5:0/2039111600 >> 172.21.15.5:0/2039111600 conn(0x7fda540ff8d0 msgr2=0x7fda54100c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:38:07.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:07.122+0000 7fda53200640 1 -- 172.21.15.5:0/2039111600 shutdown_connections 2024-09-10T09:38:07.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:07.122+0000 7fda53200640 1 -- 172.21.15.5:0/2039111600 wait complete. 2024-09-10T09:38:07.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:38:06 smithi102 ceph-mon[35442]: pgmap v386: 129 pgs: 129 active+clean; 646 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-09-10T09:38:07.973 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph tell mds.b status 2024-09-10T09:38:08.352 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:38:08.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:08.944+0000 7f58d1600640 1 -- 172.21.15.5:0/97579853 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f58cc101de0 msgr2=0x7f58cc1021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:38:08.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:08.944+0000 7f58d1600640 1 --2- 172.21.15.5:0/97579853 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f58cc101de0 0x7f58cc1021c0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f58bc009900 tx=0x7f58bc02a3f0 comp rx=0 tx=0).stop 2024-09-10T09:38:08.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:08.945+0000 7f58d1600640 1 -- 172.21.15.5:0/97579853 shutdown_connections 2024-09-10T09:38:08.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:08.945+0000 7f58d1600640 1 --2- 172.21.15.5:0/97579853 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f58cc106f10 0x7f58cc10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:08.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:08.945+0000 7f58d1600640 1 --2- 172.21.15.5:0/97579853 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f58cc102790 0x7f58cc1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:08.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:08.945+0000 7f58d1600640 1 --2- 172.21.15.5:0/97579853 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f58cc101de0 0x7f58cc1021c0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:08.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:08.945+0000 7f58d1600640 1 -- 172.21.15.5:0/97579853 >> 172.21.15.5:0/97579853 conn(0x7f58cc0fd8c0 msgr2=0x7f58cc0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:38:08.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:08.945+0000 7f58d1600640 1 -- 172.21.15.5:0/97579853 shutdown_connections 2024-09-10T09:38:08.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:08.945+0000 7f58d1600640 1 -- 172.21.15.5:0/97579853 wait complete. 2024-09-10T09:38:08.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:08.946+0000 7f58d1600640 1 Processor -- start 2024-09-10T09:38:08.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:08.946+0000 7f58d1600640 1 -- start start 2024-09-10T09:38:08.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:08.946+0000 7f58d1600640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f58cc101de0 0x7f58cc19a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:08.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:08.946+0000 7f58d1600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f58cc102790 0x7f58cc19abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:08.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:08.946+0000 7f58d1600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f58cc106f10 0x7f58cc19ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:08.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:08.946+0000 7f58d1600640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f58cc111fc0 con 0x7f58cc102790 2024-09-10T09:38:08.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:08.946+0000 7f58d1600640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f58cc111e40 con 0x7f58cc106f10 2024-09-10T09:38:08.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:08.946+0000 7f58d1600640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f58cc112140 con 0x7f58cc101de0 2024-09-10T09:38:08.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:08.946+0000 7f58c3e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f58cc102790 0x7f58cc19abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:08.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:08.947+0000 7f58c3e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f58cc102790 0x7f58cc19abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52014/0 (socket says 172.21.15.5:52014) 2024-09-10T09:38:08.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:08.947+0000 7f58c3e00640 1 -- 172.21.15.5:0/951205973 learned_addr learned my addr 172.21.15.5:0/951205973 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:38:08.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:08.947+0000 7f58cb400640 1 --2- 172.21.15.5:0/951205973 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f58cc106f10 0x7f58cc19ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:08.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:08.947+0000 7f58caa00640 1 --2- 172.21.15.5:0/951205973 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f58cc101de0 0x7f58cc19a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:08.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:08.947+0000 7f58c3e00640 1 -- 172.21.15.5:0/951205973 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f58cc101de0 msgr2=0x7f58cc19a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:38:08.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:08.947+0000 7f58c3e00640 1 --2- 172.21.15.5:0/951205973 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f58cc101de0 0x7f58cc19a690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:08.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:08.947+0000 7f58c3e00640 1 -- 172.21.15.5:0/951205973 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f58cc106f10 msgr2=0x7f58cc19ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:38:08.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:08.947+0000 7f58c3e00640 1 --2- 172.21.15.5:0/951205973 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f58cc106f10 0x7f58cc19ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:08.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:08.947+0000 7f58c3e00640 1 -- 172.21.15.5:0/951205973 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58cc19f640 con 0x7f58cc102790 2024-09-10T09:38:08.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:08.947+0000 7f58c3e00640 1 --2- 172.21.15.5:0/951205973 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f58cc102790 0x7f58cc19abd0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f58b400d770 tx=0x7f58b400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:38:08.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:08.948+0000 7f58c3400640 1 -- 172.21.15.5:0/951205973 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58b4017070 con 0x7f58cc102790 2024-09-10T09:38:08.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:08.948+0000 7f58c3400640 1 -- 172.21.15.5:0/951205973 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f58b4005500 con 0x7f58cc102790 2024-09-10T09:38:08.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:08.948+0000 7f58c3400640 1 -- 172.21.15.5:0/951205973 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58b4004e00 con 0x7f58cc102790 2024-09-10T09:38:08.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:08.948+0000 7f58d1600640 1 -- 172.21.15.5:0/951205973 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58cc19f930 con 0x7f58cc102790 2024-09-10T09:38:08.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:08.948+0000 7f58d1600640 1 -- 172.21.15.5:0/951205973 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58cc1a7210 con 0x7f58cc102790 2024-09-10T09:38:08.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:08.951+0000 7f58d40a7740 1 Processor -- start 2024-09-10T09:38:08.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:08.951+0000 7f58c3400640 1 -- 172.21.15.5:0/951205973 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f58b4004fa0 con 0x7f58cc102790 2024-09-10T09:38:08.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:08.951+0000 7f58d40a7740 1 -- start start 2024-09-10T09:38:08.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:08.951+0000 7f58d40a7740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a161271330 0x55a161271710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:08.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:08.951+0000 7f58d40a7740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a161271c50 0x55a16127bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:08.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:08.951+0000 7f58d40a7740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a16127c510 0x55a16127e900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:08.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:08.951+0000 7f58d40a7740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55a1611d70b0 con 0x55a161271330 2024-09-10T09:38:08.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:08.951+0000 7f58d40a7740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55a16117eab0 con 0x55a161271c50 2024-09-10T09:38:08.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:08.951+0000 7f58d40a7740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55a1611d3070 con 0x55a16127c510 2024-09-10T09:38:08.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:08.952+0000 7f58cb400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a161271330 0x55a161271710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:08.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:08.952+0000 7f58c3e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a16127c510 0x55a16127e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:08.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:08.952+0000 7f58cb400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a161271330 0x55a161271710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52028/0 (socket says 172.21.15.5:52028) 2024-09-10T09:38:08.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:08.952+0000 7f58cb400640 1 -- 172.21.15.5:0/1358422874 learned_addr learned my addr 172.21.15.5:0/1358422874 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:38:08.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:08.952+0000 7f58caa00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a161271c50 0x55a16127bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:08.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:08.952+0000 7f58caa00640 1 -- 172.21.15.5:0/1358422874 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a16127c510 msgr2=0x55a16127e900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:38:08.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:08.952+0000 7f58caa00640 1 --2- 172.21.15.5:0/1358422874 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a16127c510 0x55a16127e900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:08.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:08.952+0000 7f58caa00640 1 -- 172.21.15.5:0/1358422874 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a161271330 msgr2=0x55a161271710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:38:08.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:08.952+0000 7f58caa00640 1 --2- 172.21.15.5:0/1358422874 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a161271330 0x55a161271710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:08.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:08.952+0000 7f58caa00640 1 -- 172.21.15.5:0/1358422874 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a16118b4a0 con 0x55a161271c50 2024-09-10T09:38:08.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:08.953+0000 7f58c3400640 1 --2- 172.21.15.5:0/951205973 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f58b00a2b80 0x7f58b00a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:08.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:08.953+0000 7f58caa00640 1 --2- 172.21.15.5:0/1358422874 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a161271c50 0x55a16127bfd0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f58bc02a9e0 tx=0x7f58bc0040f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e8be2dfb18847c6d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:38:08.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:08.953+0000 7f58c3400640 1 -- 172.21.15.5:0/951205973 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f58b40cee60 con 0x7f58cc102790 2024-09-10T09:38:08.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:08.953+0000 7f58cb400640 1 --2- 172.21.15.5:0/951205973 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f58b00a2b80 0x7f58b00a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:08.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:08.953+0000 7f58bb400640 1 -- 172.21.15.5:0/1358422874 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58bc035040 con 0x55a161271c50 2024-09-10T09:38:08.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:08.953+0000 7f58bb400640 1 -- 172.21.15.5:0/1358422874 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f58bc005520 con 0x55a161271c50 2024-09-10T09:38:08.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:08.953+0000 7f58bb400640 1 -- 172.21.15.5:0/1358422874 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58bc004e60 con 0x55a161271c50 2024-09-10T09:38:08.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:08.954+0000 7f58d40a7740 1 -- 172.21.15.5:0/1358422874 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a161271c50 msgr2=0x55a16127bfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:38:08.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:08.954+0000 7f58cb400640 1 --2- 172.21.15.5:0/951205973 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f58b00a2b80 0x7f58b00a5040 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f58c4005f40 tx=0x7f58c40096a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:38:08.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:08.954+0000 7f58d40a7740 1 --2- 172.21.15.5:0/1358422874 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a161271c50 0x55a16127bfd0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f58bc02a9e0 tx=0x7f58bc0040f0 comp rx=0 tx=0).stop 2024-09-10T09:38:08.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:08.954+0000 7f58d40a7740 1 -- 172.21.15.5:0/1358422874 shutdown_connections 2024-09-10T09:38:08.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:08.954+0000 7f58d40a7740 1 --2- 172.21.15.5:0/1358422874 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a16127c510 0x55a16127e900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:08.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:08.954+0000 7f58d40a7740 1 --2- 172.21.15.5:0/1358422874 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a161271c50 0x55a16127bfd0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:08.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:08.954+0000 7f58d40a7740 1 --2- 172.21.15.5:0/1358422874 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a161271330 0x55a161271710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:08.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:08.954+0000 7f58d40a7740 1 -- 172.21.15.5:0/1358422874 >> 172.21.15.5:0/1358422874 conn(0x55a16126d8c0 msgr2=0x55a16126dca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:38:08.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:08.954+0000 7f58d40a7740 1 -- 172.21.15.5:0/1358422874 shutdown_connections 2024-09-10T09:38:08.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:08.954+0000 7f58d40a7740 1 -- 172.21.15.5:0/1358422874 wait complete. 2024-09-10T09:38:08.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:08.955+0000 7f58d40a7740 1 -- start start 2024-09-10T09:38:08.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:08.955+0000 7f58d40a7740 1 Processor -- start 2024-09-10T09:38:08.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:08.955+0000 7f58d40a7740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a161271330 0x55a1612795b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:08.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:08.955+0000 7f58d40a7740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a161271c50 0x55a161279af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:08.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:08.955+0000 7f58d40a7740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a16127c510 0x55a16127a030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:08.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:08.955+0000 7f58d40a7740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55a16127f060 con 0x55a161271330 2024-09-10T09:38:08.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:08.955+0000 7f58d40a7740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55a16115f920 con 0x55a161271c50 2024-09-10T09:38:08.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:08.955+0000 7f58d40a7740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55a16127f1e0 con 0x55a16127c510 2024-09-10T09:38:08.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:08.955+0000 7f58d40a7740 10 client.0 _ll_register_callbacks cb 0x55a1611838a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:38:08.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:08.956+0000 7f58caa00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a161271330 0x55a1612795b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:08.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:08.956+0000 7f58caa00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a161271330 0x55a1612795b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52034/0 (socket says 172.21.15.5:52034) 2024-09-10T09:38:08.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:08.956+0000 7f58caa00640 1 -- 172.21.15.5:0/2620311287 learned_addr learned my addr 172.21.15.5:0/2620311287 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:38:08.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:08.956+0000 7f58c3e00640 1 --2- 172.21.15.5:0/2620311287 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a161271c50 0x55a161279af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:08.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:08.956+0000 7f58cb400640 1 --2- 172.21.15.5:0/2620311287 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a16127c510 0x55a16127a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:08.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:08.956+0000 7f58caa00640 1 -- 172.21.15.5:0/2620311287 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a16127c510 msgr2=0x55a16127a030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:38:08.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:08.956+0000 7f58caa00640 1 --2- 172.21.15.5:0/2620311287 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a16127c510 0x55a16127a030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:08.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:08.956+0000 7f58caa00640 1 -- 172.21.15.5:0/2620311287 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a161271c50 msgr2=0x55a161279af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:38:08.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:08.956+0000 7f58caa00640 1 --2- 172.21.15.5:0/2620311287 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a161271c50 0x55a161279af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:08.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:08.956+0000 7f58caa00640 1 -- 172.21.15.5:0/2620311287 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a160f482b0 con 0x55a161271330 2024-09-10T09:38:08.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:08.956+0000 7f58c3e00640 1 --2- 172.21.15.5:0/2620311287 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a161271c50 0x55a161279af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:38:08.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:08.956+0000 7f58caa00640 1 --2- 172.21.15.5:0/2620311287 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a161271330 0x55a1612795b0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f58bc031550 tx=0x7f58bc0045b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5cb96e2b30fb80c9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:38:08.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:08.956+0000 7f58afe00640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-10T09:38:08.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:08.957+0000 7f58afe00640 1 -- 172.21.15.5:0/2620311287 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58bc035050 con 0x55a161271330 2024-09-10T09:38:08.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:08.957+0000 7f58d40a7740 1 -- 172.21.15.5:0/2620311287 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f58bc005520 con 0x55a161271330 2024-09-10T09:38:08.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:08.957+0000 7f58afe00640 1 -- 172.21.15.5:0/2620311287 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f58bc004910 con 0x55a161271330 2024-09-10T09:38:08.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:08.957+0000 7f58afe00640 1 -- 172.21.15.5:0/2620311287 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f58bc059050 con 0x55a161271330 2024-09-10T09:38:08.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:08.957+0000 7f58d40a7740 10 client.15180 fetch_fsmap learned FSMap version 29 2024-09-10T09:38:08.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:08.957+0000 7f58d40a7740 1 -- 172.21.15.5:0/2620311287 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55a16128ae40 con 0x55a161271330 2024-09-10T09:38:08.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:08.957+0000 7f58afe00640 1 -- 172.21.15.5:0/2620311287 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f58bc005520 con 0x55a161271330 2024-09-10T09:38:08.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:08.957+0000 7f58afe00640 1 -- 172.21.15.5:0/2620311287 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f58bc0039b0 con 0x55a161271330 2024-09-10T09:38:08.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:08.958+0000 7f58d40a7740 10 client.15180 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T09:38:08.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:08.958+0000 7f58d40a7740 10 client.15180 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-10T09:38:08.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:08.958+0000 7f58d40a7740 20 client.15180 populate_metadata read hostname 'smithi005' 2024-09-10T09:38:08.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:08.958+0000 7f58d40a7740 1 --2- 172.21.15.5:0/2620311287 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55a16128cf70 0x55a16128e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:08.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:08.958+0000 7f58d40a7740 4 client.15180 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:38:08.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:08.958+0000 7f58d40a7740 1 -- 172.21.15.5:0/2620311287 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a1611e4620 con 0x55a16128cf70 2024-09-10T09:38:08.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:08.958+0000 7f58c3e00640 1 --2- 172.21.15.5:0/2620311287 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55a16128cf70 0x55a16128e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:08.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:08.959+0000 7f58afe00640 10 client.15180 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T09:38:08.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:08.958+0000 7f58c3e00640 1 --2- 172.21.15.5:0/2620311287 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55a16128cf70 0x55a16128e3e0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=966a6a49ef28fd39 server_cookie=165e498436e06666 in_seq=0 out_seq=0 2024-09-10T09:38:08.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:08.959+0000 7f58afe00640 1 -- 172.21.15.5:0/2620311287 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a1611e4620 con 0x55a16128cf70 2024-09-10T09:38:08.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:08.959+0000 7f58afe00640 10 client.15180 handle_command_reply: tid=0 2024-09-10T09:38:08.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:08.959+0000 7f58d40a7740 1 client.15180 shutdown 2024-09-10T09:38:08.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:08.960+0000 7f58d40a7740 1 -- 172.21.15.5:0/2620311287 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a161271330 msgr2=0x55a1612795b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:38:08.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:08.960+0000 7f58d40a7740 1 --2- 172.21.15.5:0/2620311287 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a161271330 0x55a1612795b0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f58bc031550 tx=0x7f58bc0045b0 comp rx=0 tx=0).stop 2024-09-10T09:38:08.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:08.960+0000 7f58d40a7740 1 -- 172.21.15.5:0/2620311287 shutdown_connections 2024-09-10T09:38:08.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:08.960+0000 7f58d40a7740 1 --2- 172.21.15.5:0/2620311287 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55a16128cf70 0x55a16128e3e0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:08.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:08.960+0000 7f58d40a7740 1 --2- 172.21.15.5:0/2620311287 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a16127c510 0x55a16127a030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:08.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:08.960+0000 7f58d40a7740 1 --2- 172.21.15.5:0/2620311287 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a161271c50 0x55a161279af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:08.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:08.960+0000 7f58d40a7740 1 --2- 172.21.15.5:0/2620311287 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a161271330 0x55a1612795b0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:08.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:08.960+0000 7f58afe00640 0 client.15180 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T09:38:08.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:08.960+0000 7f58d40a7740 1 -- 172.21.15.5:0/2620311287 >> 172.21.15.5:0/2620311287 conn(0x55a16126d8c0 msgr2=0x55a1612735f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:38:08.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:08.960+0000 7f58d40a7740 1 -- 172.21.15.5:0/2620311287 shutdown_connections 2024-09-10T09:38:08.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:08.960+0000 7f58d40a7740 1 -- 172.21.15.5:0/2620311287 wait complete. 2024-09-10T09:38:08.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:08.960+0000 7f58d40a7740 20 client.15180 trim_cache size 0 max 16384 2024-09-10T09:38:08.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:08.979+0000 7f58d40a7740 1 Processor -- start 2024-09-10T09:38:08.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:08.979+0000 7f58d40a7740 1 -- start start 2024-09-10T09:38:08.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:08.979+0000 7f58d40a7740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a161271330 0x55a16128c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:08.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:08.979+0000 7f58d40a7740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a161271c50 0x55a161273d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:08.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:08.979+0000 7f58d40a7740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a16127c510 0x55a16127b520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:08.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:08.979+0000 7f58d40a7740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55a16128c720 con 0x55a16127c510 2024-09-10T09:38:08.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:08.979+0000 7f58d40a7740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55a16128cdf0 con 0x55a161271330 2024-09-10T09:38:08.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:08.979+0000 7f58d40a7740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55a16128d4c0 con 0x55a161271c50 2024-09-10T09:38:08.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:08.980+0000 7f58cb400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a16127c510 0x55a16127b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:08.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:08.980+0000 7f58cb400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a16127c510 0x55a16127b520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52048/0 (socket says 172.21.15.5:52048) 2024-09-10T09:38:08.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:08.980+0000 7f58cb400640 1 -- 172.21.15.5:0/1522273764 learned_addr learned my addr 172.21.15.5:0/1522273764 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:38:08.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:08.980+0000 7f58caa00640 1 --2- 172.21.15.5:0/1522273764 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a161271330 0x55a16128c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:08.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:08.980+0000 7f58c3e00640 1 --2- 172.21.15.5:0/1522273764 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a161271c50 0x55a161273d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:08.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:08.980+0000 7f58caa00640 1 -- 172.21.15.5:0/1522273764 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a161271c50 msgr2=0x55a161273d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:38:08.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:08.980+0000 7f58caa00640 1 --2- 172.21.15.5:0/1522273764 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a161271c50 0x55a161273d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:08.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:08.980+0000 7f58caa00640 1 -- 172.21.15.5:0/1522273764 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a16127c510 msgr2=0x55a16127b520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:38:08.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:08.980+0000 7f58caa00640 1 --2- 172.21.15.5:0/1522273764 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a16127c510 0x55a16127b520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:08.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:08.980+0000 7f58caa00640 1 -- 172.21.15.5:0/1522273764 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a16126ea50 con 0x55a161271330 2024-09-10T09:38:08.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:08.980+0000 7f58c3e00640 1 --2- 172.21.15.5:0/1522273764 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a161271c50 0x55a161273d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:38:08.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:08.980+0000 7f58caa00640 1 --2- 172.21.15.5:0/1522273764 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a161271330 0x55a16128c8a0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f58bc003f40 tx=0x7f58bc000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3c0f59d237676f41 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:38:08.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:08.981+0000 7f58baa00640 1 -- 172.21.15.5:0/1522273764 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58bc035070 con 0x55a161271330 2024-09-10T09:38:08.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:08.981+0000 7f58baa00640 1 -- 172.21.15.5:0/1522273764 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f58bc003ad0 con 0x55a161271330 2024-09-10T09:38:08.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:08.981+0000 7f58baa00640 1 -- 172.21.15.5:0/1522273764 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58bc04e6d0 con 0x55a161271330 2024-09-10T09:38:08.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:08.981+0000 7f58d40a7740 1 -- 172.21.15.5:0/1522273764 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a161271330 msgr2=0x55a16128c8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:38:08.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:08.981+0000 7f58d40a7740 1 --2- 172.21.15.5:0/1522273764 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a161271330 0x55a16128c8a0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f58bc003f40 tx=0x7f58bc000bc0 comp rx=0 tx=0).stop 2024-09-10T09:38:08.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:08.981+0000 7f58d40a7740 1 -- 172.21.15.5:0/1522273764 shutdown_connections 2024-09-10T09:38:08.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:08.981+0000 7f58d40a7740 1 --2- 172.21.15.5:0/1522273764 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a16127c510 0x55a16127b520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:08.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:08.981+0000 7f58d40a7740 1 --2- 172.21.15.5:0/1522273764 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a161271c50 0x55a161273d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:08.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:08.981+0000 7f58d40a7740 1 --2- 172.21.15.5:0/1522273764 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a161271330 0x55a16128c8a0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:08.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:08.981+0000 7f58d40a7740 1 -- 172.21.15.5:0/1522273764 >> 172.21.15.5:0/1522273764 conn(0x55a16126d8c0 msgr2=0x55a16127ae10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:38:08.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:08.981+0000 7f58d40a7740 1 -- 172.21.15.5:0/1522273764 shutdown_connections 2024-09-10T09:38:08.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:08.981+0000 7f58d40a7740 1 -- 172.21.15.5:0/1522273764 wait complete. 2024-09-10T09:38:08.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:08.982+0000 7f58d40a7740 1 -- start start 2024-09-10T09:38:08.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:08.982+0000 7f58d40a7740 1 Processor -- start 2024-09-10T09:38:08.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:08.982+0000 7f58d40a7740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a161271330 0x55a161296e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:08.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:08.982+0000 7f58d40a7740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a161271c50 0x55a161276430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:08.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:08.982+0000 7f58d40a7740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a16127c510 0x55a1612769d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:08.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:08.982+0000 7f58d40a7740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55a16128e260 con 0x55a161271c50 2024-09-10T09:38:08.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:08.982+0000 7f58d40a7740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55a16128db90 con 0x55a161271330 2024-09-10T09:38:08.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:08.982+0000 7f58d40a7740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55a16128e930 con 0x55a16127c510 2024-09-10T09:38:08.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:08.982+0000 7f58d40a7740 10 client.0 _ll_register_callbacks cb 0x7f58bc004040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:38:08.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:08.983+0000 7f58c3e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a161271c50 0x55a161276430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:08.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:08.983+0000 7f58c3e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a161271c50 0x55a161276430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52054/0 (socket says 172.21.15.5:52054) 2024-09-10T09:38:08.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:08.983+0000 7f58c3e00640 1 -- 172.21.15.5:0/1643980654 learned_addr learned my addr 172.21.15.5:0/1643980654 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:38:08.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:08.983+0000 7f58caa00640 1 --2- 172.21.15.5:0/1643980654 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a161271330 0x55a161296e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:08.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:08.983+0000 7f58cb400640 1 --2- 172.21.15.5:0/1643980654 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a16127c510 0x55a1612769d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:08.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:08.983+0000 7f58c3e00640 1 -- 172.21.15.5:0/1643980654 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a16127c510 msgr2=0x55a1612769d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:38:08.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:08.983+0000 7f58c3e00640 1 --2- 172.21.15.5:0/1643980654 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a16127c510 0x55a1612769d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:08.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:08.983+0000 7f58c3e00640 1 -- 172.21.15.5:0/1643980654 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a161271330 msgr2=0x55a161296e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:38:08.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:08.983+0000 7f58c3e00640 1 --2- 172.21.15.5:0/1643980654 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a161271330 0x55a161296e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:08.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:08.983+0000 7f58c3e00640 1 -- 172.21.15.5:0/1643980654 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a161277200 con 0x55a161271c50 2024-09-10T09:38:08.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:08.983+0000 7f58caa00640 1 --2- 172.21.15.5:0/1643980654 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a161271330 0x55a161296e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:38:08.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:08.983+0000 7f58c3e00640 1 --2- 172.21.15.5:0/1643980654 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a161271c50 0x55a161276430 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f58b40cc4b0 tx=0x7f58b40bb040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70256c4e113ef3d3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:38:08.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:08.983+0000 7f58afe00640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-10T09:38:08.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:08.984+0000 7f58afe00640 1 -- 172.21.15.5:0/1643980654 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58b4020040 con 0x55a161271c50 2024-09-10T09:38:08.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:08.984+0000 7f58afe00640 1 -- 172.21.15.5:0/1643980654 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f58b40cc620 con 0x55a161271c50 2024-09-10T09:38:08.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:08.984+0000 7f58d40a7740 1 -- 172.21.15.5:0/1643980654 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f58bc003ad0 con 0x55a161271c50 2024-09-10T09:38:08.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:08.984+0000 7f58afe00640 1 -- 172.21.15.5:0/1643980654 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f58b40046c0 con 0x55a161271c50 2024-09-10T09:38:08.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:08.984+0000 7f58d40a7740 10 client.15186 fetch_fsmap learned FSMap version 29 2024-09-10T09:38:08.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:08.984+0000 7f58d40a7740 1 -- 172.21.15.5:0/1643980654 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55a1612f7770 con 0x55a161271c50 2024-09-10T09:38:08.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:08.984+0000 7f58afe00640 1 -- 172.21.15.5:0/1643980654 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f58bc003ad0 con 0x55a161271c50 2024-09-10T09:38:08.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:08.984+0000 7f58afe00640 1 -- 172.21.15.5:0/1643980654 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f58b4097990 con 0x55a161271c50 2024-09-10T09:38:08.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:08.984+0000 7f58d40a7740 10 client.15186 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T09:38:08.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:08.985+0000 7f58d40a7740 10 client.15186 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-10T09:38:08.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:08.985+0000 7f58d40a7740 20 client.15186 populate_metadata read hostname 'smithi005' 2024-09-10T09:38:08.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:08.985+0000 7f58d40a7740 1 --2- 172.21.15.5:0/1643980654 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55a1612fc310 0x55a1612fb020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:08.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:08.985+0000 7f58d40a7740 4 client.15186 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T09:38:08.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:08.985+0000 7f58d40a7740 1 -- 172.21.15.5:0/1643980654 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "status"}) -- 0x55a1611deb30 con 0x55a1612fc310 2024-09-10T09:38:08.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:08.985+0000 7f58caa00640 1 --2- 172.21.15.5:0/1643980654 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55a1612fc310 0x55a1612fb020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:08.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:08.985+0000 7f58caa00640 1 --2- 172.21.15.5:0/1643980654 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55a1612fc310 0x55a1612fb020 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f15ad7d2c0b725b3 server_cookie=7afc324b9aa5345e in_seq=0 out_seq=0 2024-09-10T09:38:08.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:08.985+0000 7f58afe00640 10 client.15186 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T09:38:08.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:08.986+0000 7f58afe00640 1 -- 172.21.15.5:0/1643980654 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55a1611deb30 con 0x55a1612fc310 2024-09-10T09:38:08.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:08.986+0000 7f58afe00640 10 client.15186 handle_command_reply: tid=0 2024-09-10T09:38:08.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:08.986+0000 7f58d40a7740 1 client.15186 shutdown 2024-09-10T09:38:08.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:08.986+0000 7f58d40a7740 1 -- 172.21.15.5:0/1643980654 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a161271c50 msgr2=0x55a161276430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:38:08.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:08.986+0000 7f58d40a7740 1 --2- 172.21.15.5:0/1643980654 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a161271c50 0x55a161276430 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f58b40cc4b0 tx=0x7f58b40bb040 comp rx=0 tx=0).stop 2024-09-10T09:38:08.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:08.986+0000 7f58d40a7740 1 -- 172.21.15.5:0/1643980654 shutdown_connections 2024-09-10T09:38:08.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:08.986+0000 7f58d40a7740 1 --2- 172.21.15.5:0/1643980654 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55a1612fc310 0x55a1612fb020 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:08.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:08.986+0000 7f58d40a7740 1 --2- 172.21.15.5:0/1643980654 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a16127c510 0x55a1612769d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:08.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:08.986+0000 7f58d40a7740 1 --2- 172.21.15.5:0/1643980654 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a161271c50 0x55a161276430 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:08.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:08.986+0000 7f58d40a7740 1 --2- 172.21.15.5:0/1643980654 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a161271330 0x55a161296e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:08.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:08.986+0000 7f58afe00640 0 client.15186 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T09:38:08.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:08.986+0000 7f58d40a7740 1 -- 172.21.15.5:0/1643980654 >> 172.21.15.5:0/1643980654 conn(0x55a16126d8c0 msgr2=0x55a161287c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:38:08.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:08.986+0000 7f58d40a7740 1 -- 172.21.15.5:0/1643980654 shutdown_connections 2024-09-10T09:38:08.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:08.986+0000 7f58d40a7740 1 -- 172.21.15.5:0/1643980654 wait complete. 2024-09-10T09:38:08.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:08.986+0000 7f58d40a7740 20 client.15186 trim_cache size 0 max 16384 2024-09-10T09:38:08.992 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-10T09:38:08.992 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_fsid": "bade4b50-6f56-11ef-bcea-c7b262605968", 2024-09-10T09:38:08.992 INFO:teuthology.orchestra.run.smithi005.stdout: "whoami": 0, 2024-09-10T09:38:08.992 INFO:teuthology.orchestra.run.smithi005.stdout: "id": 24542, 2024-09-10T09:38:08.992 INFO:teuthology.orchestra.run.smithi005.stdout: "want_state": "up:active", 2024-09-10T09:38:08.992 INFO:teuthology.orchestra.run.smithi005.stdout: "state": "up:active", 2024-09-10T09:38:08.992 INFO:teuthology.orchestra.run.smithi005.stdout: "fs_name": "cephfs", 2024-09-10T09:38:08.992 INFO:teuthology.orchestra.run.smithi005.stdout: "rank_uptime": 158.42117544499999, 2024-09-10T09:38:08.992 INFO:teuthology.orchestra.run.smithi005.stdout: "mdsmap_epoch": 29, 2024-09-10T09:38:08.993 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch": 96, 2024-09-10T09:38:08.993 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch_barrier": 94, 2024-09-10T09:38:08.993 INFO:teuthology.orchestra.run.smithi005.stdout: "uptime": 172.03712536099999 2024-09-10T09:38:08.993 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-10T09:38:08.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:08.987+0000 7f58d1600640 1 -- 172.21.15.5:0/951205973 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f58b00a2b80 msgr2=0x7f58b00a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:38:08.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:08.987+0000 7f58d1600640 1 --2- 172.21.15.5:0/951205973 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f58b00a2b80 0x7f58b00a5040 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f58c4005f40 tx=0x7f58c40096a0 comp rx=0 tx=0).stop 2024-09-10T09:38:08.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:08.987+0000 7f58d1600640 1 -- 172.21.15.5:0/951205973 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f58cc102790 msgr2=0x7f58cc19abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:38:08.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:08.987+0000 7f58d1600640 1 --2- 172.21.15.5:0/951205973 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f58cc102790 0x7f58cc19abd0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f58b400d770 tx=0x7f58b400dc30 comp rx=0 tx=0).stop 2024-09-10T09:38:08.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:08.987+0000 7f58d1600640 1 -- 172.21.15.5:0/951205973 shutdown_connections 2024-09-10T09:38:08.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:08.987+0000 7f58d1600640 1 --2- 172.21.15.5:0/951205973 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f58b00a2b80 0x7f58b00a5040 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:08.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:08.987+0000 7f58d1600640 1 --2- 172.21.15.5:0/951205973 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f58cc106f10 0x7f58cc19ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:08.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:08.987+0000 7f58d1600640 1 --2- 172.21.15.5:0/951205973 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f58cc102790 0x7f58cc19abd0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:08.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:08.987+0000 7f58d1600640 1 --2- 172.21.15.5:0/951205973 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f58cc101de0 0x7f58cc19a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:08.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:08.987+0000 7f58d1600640 1 -- 172.21.15.5:0/951205973 >> 172.21.15.5:0/951205973 conn(0x7f58cc0fd8c0 msgr2=0x7f58cc0feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:38:08.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:08.987+0000 7f58d1600640 1 -- 172.21.15.5:0/951205973 shutdown_connections 2024-09-10T09:38:08.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:08.987+0000 7f58d1600640 1 -- 172.21.15.5:0/951205973 wait complete. 2024-09-10T09:38:09.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:38:08 smithi159 ceph-mon[35513]: pgmap v387: 129 pgs: 129 active+clean; 646 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-10T09:38:09.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:38:08 smithi102 ceph-mon[35442]: pgmap v387: 129 pgs: 129 active+clean; 646 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-10T09:38:09.306 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:38:08 smithi005 ceph-mon[30350]: pgmap v387: 129 pgs: 129 active+clean; 646 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-10T09:38:09.926 INFO:teuthology.run_tasks:Running task sequential... 2024-09-10T09:38:09.939 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-09-10T09:38:09.939 INFO:teuthology.task.sequential:In sequential, running task print... 2024-09-10T09:38:09.940 INFO:teuthology.task.print:Enabling mgr modules 2024-09-10T09:38:09.940 INFO:teuthology.run_tasks:Running task kclient... 2024-09-10T09:38:09.981 INFO:tasks.kclient:Mounting kernel clients... 2024-09-10T09:38:09.981 INFO:tasks.kclient:config is {'client.0': {}} 2024-09-10T09:38:09.981 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T09:38:09.981 DEBUG:teuthology.orchestra.run.smithi005:> ip netns list 2024-09-10T09:38:10.019 INFO:teuthology.orchestra.run.smithi005.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2024-09-10T09:38:10.020 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T09:38:10.020 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-10T09:38:10.120 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T09:38:10.120 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip link delete ceph-brx 2024-09-10T09:38:10.460 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=legacy', 'wsync'], 'syntax': 'v1', 'mount_subvol_num': 1} 2024-09-10T09:38:10.460 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-10T09:38:10.460 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-10T09:38:10.460 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph fs ls 2024-09-10T09:38:10.853 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:38:11.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:38:10 smithi159 ceph-mon[35513]: pgmap v388: 129 pgs: 129 active+clean; 646 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-10T09:38:11.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:38:10 smithi102 ceph-mon[35442]: pgmap v388: 129 pgs: 129 active+clean; 646 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-10T09:38:11.158 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:38:10 smithi005 ceph-mon[30350]: pgmap v388: 129 pgs: 129 active+clean; 646 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-10T09:38:11.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:11.404+0000 7f8d7e000640 1 -- 172.21.15.5:0/2057211327 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8d78108f60 msgr2=0x7f8d78110ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:38:11.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:11.404+0000 7f8d7e000640 1 --2- 172.21.15.5:0/2057211327 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8d78108f60 0x7f8d78110ae0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f8d70009ca0 tx=0x7f8d7002a8c0 comp rx=0 tx=0).stop 2024-09-10T09:38:11.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:11.404+0000 7f8d7e000640 1 -- 172.21.15.5:0/2057211327 shutdown_connections 2024-09-10T09:38:11.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:11.404+0000 7f8d7e000640 1 --2- 172.21.15.5:0/2057211327 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8d78108f60 0x7f8d78110ae0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:11.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:11.404+0000 7f8d7e000640 1 --2- 172.21.15.5:0/2057211327 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8d781047e0 0x7f8d78108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:11.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:11.404+0000 7f8d7e000640 1 --2- 172.21.15.5:0/2057211327 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8d78103e30 0x7f8d78104210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:11.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:11.404+0000 7f8d7e000640 1 -- 172.21.15.5:0/2057211327 >> 172.21.15.5:0/2057211327 conn(0x7f8d780ff8d0 msgr2=0x7f8d78101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:38:11.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:11.404+0000 7f8d7e000640 1 -- 172.21.15.5:0/2057211327 shutdown_connections 2024-09-10T09:38:11.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:11.404+0000 7f8d7e000640 1 -- 172.21.15.5:0/2057211327 wait complete. 2024-09-10T09:38:11.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:11.405+0000 7f8d7e000640 1 Processor -- start 2024-09-10T09:38:11.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:11.405+0000 7f8d7e000640 1 -- start start 2024-09-10T09:38:11.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:11.406+0000 7f8d7e000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8d78103e30 0x7f8d7819a410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:11.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:11.406+0000 7f8d7e000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8d781047e0 0x7f8d7819a950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:11.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:11.406+0000 7f8d7e000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8d78108f60 0x7f8d7819ece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:11.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:11.406+0000 7f8d7e000640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f8d78114010 con 0x7f8d78103e30 2024-09-10T09:38:11.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:11.406+0000 7f8d7e000640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f8d78113e90 con 0x7f8d78108f60 2024-09-10T09:38:11.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:11.406+0000 7f8d7e000640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f8d78114190 con 0x7f8d781047e0 2024-09-10T09:38:11.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:11.406+0000 7f8d7cc00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8d78103e30 0x7f8d7819a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:11.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:11.406+0000 7f8d77e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8d781047e0 0x7f8d7819a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:11.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:11.406+0000 7f8d7cc00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8d78103e30 0x7f8d7819a410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60602/0 (socket says 172.21.15.5:60602) 2024-09-10T09:38:11.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:11.406+0000 7f8d7cc00640 1 -- 172.21.15.5:0/2635020477 learned_addr learned my addr 172.21.15.5:0/2635020477 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:38:11.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:11.406+0000 7f8d7d600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8d78108f60 0x7f8d7819ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:11.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:11.406+0000 7f8d7cc00640 1 -- 172.21.15.5:0/2635020477 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8d781047e0 msgr2=0x7f8d7819a950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:38:11.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:11.406+0000 7f8d7cc00640 1 --2- 172.21.15.5:0/2635020477 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8d781047e0 0x7f8d7819a950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:11.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:11.406+0000 7f8d7cc00640 1 -- 172.21.15.5:0/2635020477 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8d78108f60 msgr2=0x7f8d7819ece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:38:11.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:11.406+0000 7f8d7cc00640 1 --2- 172.21.15.5:0/2635020477 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8d78108f60 0x7f8d7819ece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:11.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:11.406+0000 7f8d7cc00640 1 -- 172.21.15.5:0/2635020477 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d7819f3c0 con 0x7f8d78103e30 2024-09-10T09:38:11.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:11.406+0000 7f8d77e00640 1 --2- 172.21.15.5:0/2635020477 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8d781047e0 0x7f8d7819a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:38:11.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:11.407+0000 7f8d7cc00640 1 --2- 172.21.15.5:0/2635020477 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8d78103e30 0x7f8d7819a410 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f8d6c00e9f0 tx=0x7f8d6c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:38:11.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:11.407+0000 7f8d75600640 1 -- 172.21.15.5:0/2635020477 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d6c00cd80 con 0x7f8d78103e30 2024-09-10T09:38:11.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:11.407+0000 7f8d7e000640 1 -- 172.21.15.5:0/2635020477 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d7819f6b0 con 0x7f8d78103e30 2024-09-10T09:38:11.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:11.407+0000 7f8d75600640 1 -- 172.21.15.5:0/2635020477 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8d6c005500 con 0x7f8d78103e30 2024-09-10T09:38:11.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:11.407+0000 7f8d75600640 1 -- 172.21.15.5:0/2635020477 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d6c018630 con 0x7f8d78103e30 2024-09-10T09:38:11.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:11.407+0000 7f8d7e000640 1 -- 172.21.15.5:0/2635020477 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d781a6f90 con 0x7f8d78103e30 2024-09-10T09:38:11.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:11.408+0000 7f8d7e000640 1 -- 172.21.15.5:0/2635020477 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8d78104210 con 0x7f8d78103e30 2024-09-10T09:38:11.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:11.410+0000 7f8d75600640 1 -- 172.21.15.5:0/2635020477 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f8d6c0046c0 con 0x7f8d78103e30 2024-09-10T09:38:11.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:11.412+0000 7f8d75600640 1 --2- 172.21.15.5:0/2635020477 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f8d600a2b80 0x7f8d600a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:11.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:11.412+0000 7f8d75600640 1 -- 172.21.15.5:0/2635020477 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f8d6c0cf9e0 con 0x7f8d78103e30 2024-09-10T09:38:11.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:11.412+0000 7f8d77e00640 1 --2- 172.21.15.5:0/2635020477 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f8d600a2b80 0x7f8d600a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:11.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:11.413+0000 7f8d77e00640 1 --2- 172.21.15.5:0/2635020477 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f8d600a2b80 0x7f8d600a5040 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f8d640095f0 tx=0x7f8d64005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:38:11.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:11.416+0000 7f8d75600640 1 -- 172.21.15.5:0/2635020477 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f8d6c09c4f0 con 0x7f8d78103e30 2024-09-10T09:38:11.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:11.644+0000 7f8d7e000640 1 -- 172.21.15.5:0/2635020477 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f8d7802d7c0 con 0x7f8d78103e30 2024-09-10T09:38:11.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:11.644+0000 7f8d75600640 1 -- 172.21.15.5:0/2635020477 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v29) ==== 53+0+88 (secure 0 0 0) 0x7f8d6c09bdf0 con 0x7f8d78103e30 2024-09-10T09:38:11.645 INFO:teuthology.orchestra.run.smithi005.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-09-10T09:38:11.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:11.646+0000 7f8d7e000640 1 -- 172.21.15.5:0/2635020477 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f8d600a2b80 msgr2=0x7f8d600a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:38:11.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:11.646+0000 7f8d7e000640 1 --2- 172.21.15.5:0/2635020477 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f8d600a2b80 0x7f8d600a5040 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f8d640095f0 tx=0x7f8d64005f90 comp rx=0 tx=0).stop 2024-09-10T09:38:11.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:11.646+0000 7f8d7e000640 1 -- 172.21.15.5:0/2635020477 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8d78103e30 msgr2=0x7f8d7819a410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:38:11.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:11.646+0000 7f8d7e000640 1 --2- 172.21.15.5:0/2635020477 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8d78103e30 0x7f8d7819a410 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f8d6c00e9f0 tx=0x7f8d6c00edf0 comp rx=0 tx=0).stop 2024-09-10T09:38:11.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:11.647+0000 7f8d7e000640 1 -- 172.21.15.5:0/2635020477 shutdown_connections 2024-09-10T09:38:11.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:11.647+0000 7f8d7e000640 1 --2- 172.21.15.5:0/2635020477 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f8d600a2b80 0x7f8d600a5040 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:11.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:11.647+0000 7f8d7e000640 1 --2- 172.21.15.5:0/2635020477 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8d78108f60 0x7f8d7819ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:11.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:11.647+0000 7f8d7e000640 1 --2- 172.21.15.5:0/2635020477 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8d781047e0 0x7f8d7819a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:11.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:11.647+0000 7f8d7e000640 1 --2- 172.21.15.5:0/2635020477 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8d78103e30 0x7f8d7819a410 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:11.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:11.647+0000 7f8d7e000640 1 -- 172.21.15.5:0/2635020477 >> 172.21.15.5:0/2635020477 conn(0x7f8d780ff8d0 msgr2=0x7f8d78100bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:38:11.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:11.647+0000 7f8d7e000640 1 -- 172.21.15.5:0/2635020477 shutdown_connections 2024-09-10T09:38:11.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:11.647+0000 7f8d7e000640 1 -- 172.21.15.5:0/2635020477 wait complete. 2024-09-10T09:38:12.046 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:38:11 smithi005 ceph-mon[30350]: from='client.? 172.21.15.5:0/2635020477' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-10T09:38:12.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:38:11 smithi159 ceph-mon[35513]: from='client.? 172.21.15.5:0/2635020477' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-10T09:38:12.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:38:11 smithi102 ceph-mon[35442]: from='client.? 172.21.15.5:0/2635020477' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-10T09:38:12.589 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-10T09:38:12.589 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-10T09:38:12.589 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi005.front.sepia.ceph.com 2024-09-10T09:38:12.589 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-10T09:38:12.589 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-10T09:38:12.589 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-10T09:38:12.589 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-10T09:38:12.589 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-10T09:38:12.589 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-10T09:38:12.590 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T09:38:12.590 DEBUG:teuthology.orchestra.run.smithi005:> ip addr 2024-09-10T09:38:12.627 INFO:teuthology.orchestra.run.smithi005.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-10T09:38:12.627 INFO:teuthology.orchestra.run.smithi005.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-10T09:38:12.627 INFO:teuthology.orchestra.run.smithi005.stdout: inet 127.0.0.1/8 scope host lo 2024-09-10T09:38:12.627 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-09-10T09:38:12.627 INFO:teuthology.orchestra.run.smithi005.stdout: inet6 ::1/128 scope host 2024-09-10T09:38:12.627 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-09-10T09:38:12.627 INFO:teuthology.orchestra.run.smithi005.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T09:38:12.628 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 0c:c4:7a:6b:fd:de brd ff:ff:ff:ff:ff:ff 2024-09-10T09:38:12.628 INFO:teuthology.orchestra.run.smithi005.stdout: altname enp7s0f0 2024-09-10T09:38:12.628 INFO:teuthology.orchestra.run.smithi005.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T09:38:12.628 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 0c:c4:7a:6b:fd:df brd ff:ff:ff:ff:ff:ff 2024-09-10T09:38:12.628 INFO:teuthology.orchestra.run.smithi005.stdout: altname enp7s0f1 2024-09-10T09:38:12.628 INFO:teuthology.orchestra.run.smithi005.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-10T09:38:12.628 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 0c:c4:7a:bd:14:42 brd ff:ff:ff:ff:ff:ff 2024-09-10T09:38:12.628 INFO:teuthology.orchestra.run.smithi005.stdout: altname enp1s0f0 2024-09-10T09:38:12.628 INFO:teuthology.orchestra.run.smithi005.stdout: inet 172.21.15.5/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-09-10T09:38:12.628 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft 42046sec preferred_lft 42046sec 2024-09-10T09:38:12.628 INFO:teuthology.orchestra.run.smithi005.stdout: inet6 fe80::ec4:7aff:febd:1442/64 scope link 2024-09-10T09:38:12.628 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-09-10T09:38:12.629 INFO:teuthology.orchestra.run.smithi005.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T09:38:12.629 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 0c:c4:7a:bd:14:43 brd ff:ff:ff:ff:ff:ff 2024-09-10T09:38:12.629 INFO:teuthology.orchestra.run.smithi005.stdout: altname enp1s0f1 2024-09-10T09:38:12.629 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-09-10T09:38:12.629 DEBUG:teuthology.orchestra.run.smithi005:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T09:38:12.629 DEBUG:teuthology.orchestra.run.smithi005:> set -e 2024-09-10T09:38:12.629 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip link add name ceph-brx type bridge 2024-09-10T09:38:12.629 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip addr flush dev ceph-brx 2024-09-10T09:38:12.629 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip link set ceph-brx up 2024-09-10T09:38:12.629 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-09-10T09:38:12.630 DEBUG:teuthology.orchestra.run.smithi005:> ') 2024-09-10T09:38:12.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:12 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T09:38:12.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:12 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T09:38:12.855 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T09:38:12.855 DEBUG:teuthology.orchestra.run.smithi005:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-10T09:38:12.911 INFO:teuthology.orchestra.run.smithi005.stdout:1 2024-09-10T09:38:12.914 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T09:38:12.914 DEBUG:teuthology.orchestra.run.smithi005:> route 2024-09-10T09:38:12.950 INFO:teuthology.orchestra.run.smithi005.stdout:Kernel IP routing table 2024-09-10T09:38:12.951 INFO:teuthology.orchestra.run.smithi005.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-10T09:38:12.951 INFO:teuthology.orchestra.run.smithi005.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-09-10T09:38:12.951 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-09-10T09:38:12.951 INFO:teuthology.orchestra.run.smithi005.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-10T09:38:12.951 DEBUG:teuthology.orchestra.run.smithi005:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T09:38:12.951 DEBUG:teuthology.orchestra.run.smithi005:> set -e 2024-09-10T09:38:12.952 DEBUG:teuthology.orchestra.run.smithi005:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-10T09:38:12.952 DEBUG:teuthology.orchestra.run.smithi005:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-10T09:38:12.952 DEBUG:teuthology.orchestra.run.smithi005:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-10T09:38:12.952 DEBUG:teuthology.orchestra.run.smithi005:> ') 2024-09-10T09:38:13.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:38:12 smithi005 ceph-mon[30350]: pgmap v389: 129 pgs: 129 active+clean; 646 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-09-10T09:38:13.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:13 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T09:38:13.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:38:12 smithi159 ceph-mon[35513]: pgmap v389: 129 pgs: 129 active+clean; 646 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-09-10T09:38:13.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:38:12 smithi102 ceph-mon[35442]: pgmap v389: 129 pgs: 129 active+clean; 646 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-09-10T09:38:13.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:13 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T09:38:13.150 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T09:38:13.151 DEBUG:teuthology.orchestra.run.smithi005:> ip netns list 2024-09-10T09:38:13.187 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T09:38:13.187 DEBUG:teuthology.orchestra.run.smithi005:> ip netns list-id 2024-09-10T09:38:13.263 DEBUG:teuthology.orchestra.run.smithi005:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T09:38:13.264 DEBUG:teuthology.orchestra.run.smithi005:> set -e 2024-09-10T09:38:13.264 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T09:38:13.264 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-09-10T09:38:13.264 DEBUG:teuthology.orchestra.run.smithi005:> ') 2024-09-10T09:38:13.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:13 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T09:38:13.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:13 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T09:38:13.441 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-09-10T09:38:13.441 DEBUG:teuthology.orchestra.run.smithi005:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T09:38:13.441 DEBUG:teuthology.orchestra.run.smithi005:> set -e 2024-09-10T09:38:13.441 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-09-10T09:38:13.441 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-09-10T09:38:13.441 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-09-10T09:38:13.441 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-09-10T09:38:13.441 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-09-10T09:38:13.441 DEBUG:teuthology.orchestra.run.smithi005:> ') 2024-09-10T09:38:13.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:13 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T09:38:13.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:13 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T09:38:13.663 DEBUG:teuthology.orchestra.run.smithi005:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T09:38:13.663 DEBUG:teuthology.orchestra.run.smithi005:> set -e 2024-09-10T09:38:13.663 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip link set brx.0 up 2024-09-10T09:38:13.664 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip link set dev brx.0 master ceph-brx 2024-09-10T09:38:13.664 DEBUG:teuthology.orchestra.run.smithi005:> ') 2024-09-10T09:38:13.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:13 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T09:38:13.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:13 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T09:38:13.831 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T09:38:13.831 DEBUG:teuthology.orchestra.run.smithi005:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-10T09:38:13.868 INFO:teuthology.orchestra.run.smithi005.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-10T09:38:13.868 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T09:38:13.868 DEBUG:teuthology.orchestra.run.smithi005:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-10T09:38:13.942 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2024-09-10T09:38:13.942 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2024-09-10T09:38:13.942 INFO:tasks.cephfs.kernel_mount:mounting using device: :/volumes/qa/sv_1/8c062e55-cb3c-4c81-90c0-78100d07381f 2024-09-10T09:38:13.943 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T09:38:13.943 DEBUG:teuthology.orchestra.run.smithi005:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/volumes/qa/sv_1/8c062e55-cb3c-4c81-90c0-78100d07381f /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=legacy,wsync 2024-09-10T09:38:14.071 INFO:teuthology.orchestra.run.smithi005.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=legacy,wsync 2024-09-10T09:38:14.071 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=legacy,wsync". 2024-09-10T09:38:14.072 INFO:teuthology.orchestra.run.smithi005.stdout:invalid new device string format 2024-09-10T09:38:14.722 INFO:teuthology.orchestra.run.smithi005.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=legacy,wsync 2024-09-10T09:38:14.722 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=legacy,wsync". 2024-09-10T09:38:14.723 INFO:teuthology.orchestra.run.smithi005.stdout:invalid new device string format 2024-09-10T09:38:14.723 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: resolved to: "172.21.15.5:6789,172.21.15.102:6789,172.21.15.159:6789" 2024-09-10T09:38:14.723 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.5:6789,172.21.15.102:6789,172.21.15.159:6789:/volumes/qa/sv_1/8c062e55-cb3c-4c81-90c0-78100d07381f 2024-09-10T09:38:14.723 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=legacy,wsync,key=0,fsid=bade4b50-6f56-11ef-bcea-c7b262605968" will pass to kernel 2024-09-10T09:38:14.725 INFO:teuthology.orchestra.run.smithi005.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-09-10T09:38:14.725 INFO:teuthology.orchestra.run.smithi005.stdout: You just mounted a file system that supports labels which does not 2024-09-10T09:38:14.725 INFO:teuthology.orchestra.run.smithi005.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-10T09:38:14.725 INFO:teuthology.orchestra.run.smithi005.stdout: applications will generate AVC messages and not be allowed access to 2024-09-10T09:38:14.725 INFO:teuthology.orchestra.run.smithi005.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-10T09:38:14.728 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-10T09:38:14.729 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T09:38:14.729 DEBUG:teuthology.orchestra.run.smithi005:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-10T09:38:14.792 DEBUG:teuthology.orchestra.run.smithi005:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-10T09:38:14.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:14 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T09:38:14.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:14 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T09:38:14.886 INFO:teuthology.orchestra.run.smithi005.stdout:client25228 2024-09-10T09:38:14.886 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-09-10T09:38:14.886 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T09:38:14.924 DEBUG:teuthology.orchestra.run.smithi005:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-10T09:38:15.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:15 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T09:38:15.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:15 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T09:38:15.054 INFO:teuthology.orchestra.run.smithi005.stdout:client25228 2024-09-10T09:38:15.054 DEBUG:teuthology.orchestra.run.smithi005:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-10T09:38:15.055 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:38:14 smithi005 ceph-mon[30350]: pgmap v390: 129 pgs: 129 active+clean; 646 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-10T09:38:15.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:38:14 smithi159 ceph-mon[35513]: pgmap v390: 129 pgs: 129 active+clean; 646 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-10T09:38:15.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:15 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T09:38:15.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:38:14 smithi102 ceph-mon[35442]: pgmap v390: 129 pgs: 129 active+clean; 646 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-10T09:38:15.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:15 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T09:38:15.145 INFO:teuthology.orchestra.run.smithi005.stdout:client25228 2024-09-10T09:38:15.145 DEBUG:teuthology.orchestra.run.smithi005:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/bade4b50-6f56-11ef-bcea-c7b262605968.client25228/status') 2024-09-10T09:38:15.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:15 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T09:38:15.246 INFO:teuthology.orchestra.run.smithi005.stdout:instance: client.25228 (3)172.21.15.5:0/2877161741 2024-09-10T09:38:15.246 INFO:teuthology.orchestra.run.smithi005.stdout:blocklisted: false 2024-09-10T09:38:15.246 INFO:teuthology.orchestra.run.smithi005.stderr:0+1 records in 2024-09-10T09:38:15.246 INFO:teuthology.orchestra.run.smithi005.stderr:0+1 records out 2024-09-10T09:38:15.246 INFO:teuthology.orchestra.run.smithi005.stderr:70 bytes copied, 5.8708e-05 s, 1.2 MB/s 2024-09-10T09:38:15.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:15 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T09:38:15.256 INFO:teuthology.run_tasks:Running task exec... 2024-09-10T09:38:15.266 INFO:teuthology.task.exec:Executing custom commands... 2024-09-10T09:38:15.266 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi005.front.sepia.ceph.com 2024-09-10T09:38:15.266 DEBUG:teuthology.orchestra.run.smithi005:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate false' 2024-09-10T09:38:15.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:15.421+0000 7efc43400640 1 -- 172.21.15.5:0/2843156503 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efc3c105910 msgr2=0x7efc3c10c1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:38:15.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:15.421+0000 7efc43400640 1 --2- 172.21.15.5:0/2843156503 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efc3c105910 0x7efc3c10c1a0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7efc34009ca0 tx=0x7efc3402a8c0 comp rx=0 tx=0).stop 2024-09-10T09:38:15.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:15.421+0000 7efc43400640 1 -- 172.21.15.5:0/2843156503 shutdown_connections 2024-09-10T09:38:15.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:15.421+0000 7efc43400640 1 --2- 172.21.15.5:0/2843156503 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efc3c105910 0x7efc3c10c1a0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:15.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:15.421+0000 7efc43400640 1 --2- 172.21.15.5:0/2843156503 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efc3c104f50 0x7efc3c1053d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:15.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:15.421+0000 7efc43400640 1 --2- 172.21.15.5:0/2843156503 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efc3c103d60 0x7efc3c104160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:15.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:15.421+0000 7efc43400640 1 -- 172.21.15.5:0/2843156503 >> 172.21.15.5:0/2843156503 conn(0x7efc3c0ff4e0 msgr2=0x7efc3c101900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:38:15.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:15.421+0000 7efc43400640 1 -- 172.21.15.5:0/2843156503 shutdown_connections 2024-09-10T09:38:15.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:15.421+0000 7efc43400640 1 -- 172.21.15.5:0/2843156503 wait complete. 2024-09-10T09:38:15.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:15.422+0000 7efc43400640 1 Processor -- start 2024-09-10T09:38:15.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:15.422+0000 7efc43400640 1 -- start start 2024-09-10T09:38:15.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:15.422+0000 7efc43400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efc3c103d60 0x7efc3c199f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:15.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:15.422+0000 7efc43400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efc3c104f50 0x7efc3c19a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:15.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:15.422+0000 7efc43400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efc3c105910 0x7efc3c1a14f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:15.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:15.422+0000 7efc43400640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7efc3c10fa10 con 0x7efc3c105910 2024-09-10T09:38:15.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:15.422+0000 7efc43400640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7efc3c10f890 con 0x7efc3c104f50 2024-09-10T09:38:15.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:15.422+0000 7efc43400640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7efc3c10fb90 con 0x7efc3c103d60 2024-09-10T09:38:15.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:15.422+0000 7efc41400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efc3c105910 0x7efc3c1a14f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:15.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:15.422+0000 7efc41400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efc3c105910 0x7efc3c1a14f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60616/0 (socket says 172.21.15.5:60616) 2024-09-10T09:38:15.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:15.422+0000 7efc41400640 1 -- 172.21.15.5:0/2648712749 learned_addr learned my addr 172.21.15.5:0/2648712749 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:38:15.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:15.422+0000 7efc33e00640 1 --2- 172.21.15.5:0/2648712749 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efc3c104f50 0x7efc3c19a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:15.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:15.422+0000 7efc40a00640 1 --2- 172.21.15.5:0/2648712749 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efc3c103d60 0x7efc3c199f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:15.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:15.423+0000 7efc41400640 1 -- 172.21.15.5:0/2648712749 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efc3c103d60 msgr2=0x7efc3c199f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:38:15.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:15.423+0000 7efc41400640 1 --2- 172.21.15.5:0/2648712749 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efc3c103d60 0x7efc3c199f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:15.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:15.423+0000 7efc41400640 1 -- 172.21.15.5:0/2648712749 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efc3c104f50 msgr2=0x7efc3c19a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:38:15.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:15.423+0000 7efc41400640 1 --2- 172.21.15.5:0/2648712749 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efc3c104f50 0x7efc3c19a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:15.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:15.423+0000 7efc41400640 1 -- 172.21.15.5:0/2648712749 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc3c1a1b60 con 0x7efc3c105910 2024-09-10T09:38:15.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:15.423+0000 7efc33e00640 1 --2- 172.21.15.5:0/2648712749 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efc3c104f50 0x7efc3c19a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:38:15.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:15.423+0000 7efc41400640 1 --2- 172.21.15.5:0/2648712749 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efc3c105910 0x7efc3c1a14f0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7efc34009430 tx=0x7efc340319f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:38:15.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:15.423+0000 7efc3a000640 1 -- 172.21.15.5:0/2648712749 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc34003b70 con 0x7efc3c105910 2024-09-10T09:38:15.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:15.423+0000 7efc43400640 1 -- 172.21.15.5:0/2648712749 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc3c1a1df0 con 0x7efc3c105910 2024-09-10T09:38:15.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:15.423+0000 7efc3a000640 1 -- 172.21.15.5:0/2648712749 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efc34003d10 con 0x7efc3c105910 2024-09-10T09:38:15.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:15.423+0000 7efc3a000640 1 -- 172.21.15.5:0/2648712749 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc3402fc50 con 0x7efc3c105910 2024-09-10T09:38:15.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:15.424+0000 7efc43400640 1 -- 172.21.15.5:0/2648712749 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc3c1a2300 con 0x7efc3c105910 2024-09-10T09:38:15.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:15.425+0000 7efc43400640 1 -- 172.21.15.5:0/2648712749 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc3c10b080 con 0x7efc3c105910 2024-09-10T09:38:15.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:15.427+0000 7efc3a000640 1 -- 172.21.15.5:0/2648712749 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7efc34004040 con 0x7efc3c105910 2024-09-10T09:38:15.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:15.428+0000 7efc3a000640 1 --2- 172.21.15.5:0/2648712749 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7efc200a2cb0 0x7efc200a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:15.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:15.428+0000 7efc3a000640 1 -- 172.21.15.5:0/2648712749 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7efc340f08a0 con 0x7efc3c105910 2024-09-10T09:38:15.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:15.428+0000 7efc40a00640 1 --2- 172.21.15.5:0/2648712749 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7efc200a2cb0 0x7efc200a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:15.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:15.429+0000 7efc40a00640 1 --2- 172.21.15.5:0/2648712749 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7efc200a2cb0 0x7efc200a5170 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7efc2c005fd0 tx=0x7efc2c00b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:38:15.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:15.432+0000 7efc3a000640 1 -- 172.21.15.5:0/2648712749 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7efc340bd3b0 con 0x7efc3c105910 2024-09-10T09:38:15.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:15.655+0000 7efc43400640 1 -- 172.21.15.5:0/2648712749 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"} v 0) -- 0x7efc3c062e90 con 0x7efc3c105910 2024-09-10T09:38:15.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:15.838+0000 7efc3a000640 1 -- 172.21.15.5:0/2648712749 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]=0 v30) ==== 118+0+0 (secure 0 0 0) 0x7efc340bccb0 con 0x7efc3c105910 2024-09-10T09:38:15.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:15.840+0000 7efc43400640 1 -- 172.21.15.5:0/2648712749 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7efc200a2cb0 msgr2=0x7efc200a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:38:15.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:15.840+0000 7efc43400640 1 --2- 172.21.15.5:0/2648712749 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7efc200a2cb0 0x7efc200a5170 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7efc2c005fd0 tx=0x7efc2c00b410 comp rx=0 tx=0).stop 2024-09-10T09:38:15.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:15.840+0000 7efc43400640 1 -- 172.21.15.5:0/2648712749 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efc3c105910 msgr2=0x7efc3c1a14f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:38:15.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:15.840+0000 7efc43400640 1 --2- 172.21.15.5:0/2648712749 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efc3c105910 0x7efc3c1a14f0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7efc34009430 tx=0x7efc340319f0 comp rx=0 tx=0).stop 2024-09-10T09:38:15.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:15.840+0000 7efc43400640 1 -- 172.21.15.5:0/2648712749 shutdown_connections 2024-09-10T09:38:15.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:15.840+0000 7efc43400640 1 --2- 172.21.15.5:0/2648712749 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7efc200a2cb0 0x7efc200a5170 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:15.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:15.840+0000 7efc43400640 1 --2- 172.21.15.5:0/2648712749 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efc3c105910 0x7efc3c1a14f0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:15.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:15.840+0000 7efc43400640 1 --2- 172.21.15.5:0/2648712749 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efc3c104f50 0x7efc3c19a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:15.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:15.840+0000 7efc43400640 1 --2- 172.21.15.5:0/2648712749 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efc3c103d60 0x7efc3c199f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:15.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:15.840+0000 7efc43400640 1 -- 172.21.15.5:0/2648712749 >> 172.21.15.5:0/2648712749 conn(0x7efc3c0ff4e0 msgr2=0x7efc3c108250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:38:15.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:15.841+0000 7efc43400640 1 -- 172.21.15.5:0/2648712749 shutdown_connections 2024-09-10T09:38:15.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:15.841+0000 7efc43400640 1 -- 172.21.15.5:0/2648712749 wait complete. 2024-09-10T09:38:15.858 DEBUG:teuthology.orchestra.run.smithi005:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs subvolumegroup pin cephfs qa random 0.10' 2024-09-10T09:38:16.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:16.014+0000 7f85a7a00640 1 -- 172.21.15.5:0/1130056458 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f85a0104f50 msgr2=0x7f85a01053d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:38:16.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:16.014+0000 7f85a7a00640 1 --2- 172.21.15.5:0/1130056458 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f85a0104f50 0x7f85a01053d0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f858c009900 tx=0x7f858c02a410 comp rx=0 tx=0).stop 2024-09-10T09:38:16.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:16.014+0000 7f85a7a00640 1 -- 172.21.15.5:0/1130056458 shutdown_connections 2024-09-10T09:38:16.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:16.014+0000 7f85a7a00640 1 --2- 172.21.15.5:0/1130056458 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f85a0105910 0x7f85a010c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:16.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:16.014+0000 7f85a7a00640 1 --2- 172.21.15.5:0/1130056458 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f85a0104f50 0x7f85a01053d0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:16.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:16.014+0000 7f85a7a00640 1 --2- 172.21.15.5:0/1130056458 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f85a0103d60 0x7f85a0104160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:16.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:16.014+0000 7f85a7a00640 1 -- 172.21.15.5:0/1130056458 >> 172.21.15.5:0/1130056458 conn(0x7f85a00ff4e0 msgr2=0x7f85a0101900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:38:16.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:16.014+0000 7f85a7a00640 1 -- 172.21.15.5:0/1130056458 shutdown_connections 2024-09-10T09:38:16.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:16.014+0000 7f85a7a00640 1 -- 172.21.15.5:0/1130056458 wait complete. 2024-09-10T09:38:16.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:16.015+0000 7f85a7a00640 1 Processor -- start 2024-09-10T09:38:16.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:16.015+0000 7f85a7a00640 1 -- start start 2024-09-10T09:38:16.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:16.016+0000 7f85a7a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f85a0103d60 0x7f85a0199fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:16.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:16.016+0000 7f85a7a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f85a0104f50 0x7f85a019a510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:16.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:16.016+0000 7f85a7a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f85a0105910 0x7f85a01a1580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:16.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:16.016+0000 7f85a7a00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f85a010f9e0 con 0x7f85a0104f50 2024-09-10T09:38:16.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:16.016+0000 7f85a7a00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f85a010f860 con 0x7f85a0103d60 2024-09-10T09:38:16.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:16.016+0000 7f85a7a00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f85a010fb60 con 0x7f85a0105910 2024-09-10T09:38:16.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:16.016+0000 7f859fe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f85a0104f50 0x7f85a019a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:16.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:16.016+0000 7f85a5000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f85a0103d60 0x7f85a0199fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:16.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:16.016+0000 7f85a5000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f85a0103d60 0x7f85a0199fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:44646/0 (socket says 172.21.15.5:44646) 2024-09-10T09:38:16.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:16.016+0000 7f85a5000640 1 -- 172.21.15.5:0/3295222155 learned_addr learned my addr 172.21.15.5:0/3295222155 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:38:16.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:16.016+0000 7f85a5a00640 1 --2- 172.21.15.5:0/3295222155 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f85a0105910 0x7f85a01a1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:16.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:16.016+0000 7f85a5000640 1 -- 172.21.15.5:0/3295222155 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f85a0105910 msgr2=0x7f85a01a1580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:38:16.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:16.016+0000 7f85a5000640 1 --2- 172.21.15.5:0/3295222155 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f85a0105910 0x7f85a01a1580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:16.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:16.016+0000 7f85a5000640 1 -- 172.21.15.5:0/3295222155 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f85a0104f50 msgr2=0x7f85a019a510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:38:16.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:16.016+0000 7f85a5000640 1 --2- 172.21.15.5:0/3295222155 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f85a0104f50 0x7f85a019a510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:16.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:16.016+0000 7f85a5000640 1 -- 172.21.15.5:0/3295222155 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85a01a1bf0 con 0x7f85a0103d60 2024-09-10T09:38:16.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:16.017+0000 7f85a5000640 1 --2- 172.21.15.5:0/3295222155 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f85a0103d60 0x7f85a0199fd0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f859400e9c0 tx=0x7f859400ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:38:16.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:16.017+0000 7f859d600640 1 -- 172.21.15.5:0/3295222155 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f859400cd80 con 0x7f85a0103d60 2024-09-10T09:38:16.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:16.017+0000 7f85a7a00640 1 -- 172.21.15.5:0/3295222155 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85a01a1ee0 con 0x7f85a0103d60 2024-09-10T09:38:16.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:16.017+0000 7f859d600640 1 -- 172.21.15.5:0/3295222155 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f85940035f0 con 0x7f85a0103d60 2024-09-10T09:38:16.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:16.017+0000 7f859d600640 1 -- 172.21.15.5:0/3295222155 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8594010690 con 0x7f85a0103d60 2024-09-10T09:38:16.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:16.017+0000 7f85a7a00640 1 -- 172.21.15.5:0/3295222155 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85a01a2340 con 0x7f85a0103d60 2024-09-10T09:38:16.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:16.018+0000 7f85a7a00640 1 -- 172.21.15.5:0/3295222155 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f85a010b0a0 con 0x7f85a0103d60 2024-09-10T09:38:16.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:16.020+0000 7f859d600640 1 -- 172.21.15.5:0/3295222155 <== mon.2 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f85940037a0 con 0x7f85a0103d60 2024-09-10T09:38:16.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:16.022+0000 7f859d600640 1 --2- 172.21.15.5:0/3295222155 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f85880a2c30 0x7f85880a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:16.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:16.022+0000 7f859d600640 1 -- 172.21.15.5:0/3295222155 <== mon.2 v2:172.21.15.102:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f85940cf7d0 con 0x7f85a0103d60 2024-09-10T09:38:16.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:16.022+0000 7f859fe00640 1 --2- 172.21.15.5:0/3295222155 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f85880a2c30 0x7f85880a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:16.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:16.023+0000 7f859fe00640 1 --2- 172.21.15.5:0/3295222155 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f85880a2c30 0x7f85880a50f0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f858c0314e0 tx=0x7f858c000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:38:16.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:16.023+0000 7f859d600640 1 -- 172.21.15.5:0/3295222155 <== mon.2 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f859409c2e0 con 0x7f85a0103d60 2024-09-10T09:38:16.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:38:15 smithi159 ceph-mon[35513]: from='client.? 172.21.15.5:0/2648712749' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-09-10T09:38:16.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:38:15 smithi102 ceph-mon[35442]: from='client.? 172.21.15.5:0/2648712749' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-09-10T09:38:16.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:16.233+0000 7f85a7a00640 1 -- 172.21.15.5:0/3295222155 --> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}) -- 0x7f85a00626d0 con 0x7f85880a2c30 2024-09-10T09:38:16.242 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:38:15 smithi005 ceph-mon[30350]: from='client.? 172.21.15.5:0/2648712749' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-09-10T09:38:16.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:16.250+0000 7f859d600640 1 -- 172.21.15.5:0/3295222155 <== mgr.14150 v2:172.21.15.5:6800/2707348081 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3 (secure 0 0 0) 0x7f85a00626d0 con 0x7f85880a2c30 2024-09-10T09:38:16.252 INFO:teuthology.orchestra.run.smithi005.stdout:{} 2024-09-10T09:38:16.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:16.252+0000 7f85a7a00640 1 -- 172.21.15.5:0/3295222155 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f85880a2c30 msgr2=0x7f85880a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:38:16.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:16.252+0000 7f85a7a00640 1 --2- 172.21.15.5:0/3295222155 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f85880a2c30 0x7f85880a50f0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f858c0314e0 tx=0x7f858c000bc0 comp rx=0 tx=0).stop 2024-09-10T09:38:16.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:16.252+0000 7f85a7a00640 1 -- 172.21.15.5:0/3295222155 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f85a0103d60 msgr2=0x7f85a0199fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:38:16.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:16.252+0000 7f85a7a00640 1 --2- 172.21.15.5:0/3295222155 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f85a0103d60 0x7f85a0199fd0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f859400e9c0 tx=0x7f859400ee80 comp rx=0 tx=0).stop 2024-09-10T09:38:16.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:16.253+0000 7f85a7a00640 1 -- 172.21.15.5:0/3295222155 shutdown_connections 2024-09-10T09:38:16.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:16.253+0000 7f85a7a00640 1 --2- 172.21.15.5:0/3295222155 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f85880a2c30 0x7f85880a50f0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:16.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:16.253+0000 7f85a7a00640 1 --2- 172.21.15.5:0/3295222155 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f85a0105910 0x7f85a01a1580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:16.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:16.253+0000 7f85a7a00640 1 --2- 172.21.15.5:0/3295222155 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f85a0104f50 0x7f85a019a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:16.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:16.253+0000 7f85a7a00640 1 --2- 172.21.15.5:0/3295222155 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f85a0103d60 0x7f85a0199fd0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:16.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:16.253+0000 7f85a7a00640 1 -- 172.21.15.5:0/3295222155 >> 172.21.15.5:0/3295222155 conn(0x7f85a00ff4e0 msgr2=0x7f85a0108270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:38:16.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:16.253+0000 7f85a7a00640 1 -- 172.21.15.5:0/3295222155 shutdown_connections 2024-09-10T09:38:16.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:16.253+0000 7f85a7a00640 1 -- 172.21.15.5:0/3295222155 wait complete. 2024-09-10T09:38:16.271 INFO:teuthology.run_tasks:Running task check-counter... 2024-09-10T09:38:16.283 INFO:teuthology.run_tasks:Running task fwd_scrub... 2024-09-10T09:38:16.311 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T09:38:16.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:16.468+0000 7f62d0000640 1 -- 172.21.15.5:0/3790468348 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f62c8104f20 msgr2=0x7f62c81053a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:38:16.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:16.468+0000 7f62d0000640 1 --2- 172.21.15.5:0/3790468348 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f62c8104f20 0x7f62c81053a0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f62b4009860 tx=0x7f62b402a340 comp rx=0 tx=0).stop 2024-09-10T09:38:16.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:16.468+0000 7f62d0000640 1 -- 172.21.15.5:0/3790468348 shutdown_connections 2024-09-10T09:38:16.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:16.468+0000 7f62d0000640 1 --2- 172.21.15.5:0/3790468348 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f62c81058e0 0x7f62c810c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:16.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:16.468+0000 7f62d0000640 1 --2- 172.21.15.5:0/3790468348 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f62c8104f20 0x7f62c81053a0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:16.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:16.468+0000 7f62d0000640 1 --2- 172.21.15.5:0/3790468348 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f62c8103d30 0x7f62c8104130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:16.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:16.468+0000 7f62d0000640 1 -- 172.21.15.5:0/3790468348 >> 172.21.15.5:0/3790468348 conn(0x7f62c80ff4b0 msgr2=0x7f62c81018d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:38:16.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:16.468+0000 7f62d0000640 1 -- 172.21.15.5:0/3790468348 shutdown_connections 2024-09-10T09:38:16.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:16.468+0000 7f62d0000640 1 -- 172.21.15.5:0/3790468348 wait complete. 2024-09-10T09:38:16.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:16.469+0000 7f62d0000640 1 Processor -- start 2024-09-10T09:38:16.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:16.469+0000 7f62d0000640 1 -- start start 2024-09-10T09:38:16.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:16.469+0000 7f62d0000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f62c8103d30 0x7f62c819a090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:16.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:16.469+0000 7f62d0000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f62c8104f20 0x7f62c819a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:16.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:16.470+0000 7f62d0000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f62c81058e0 0x7f62c81a1640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:16.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:16.470+0000 7f62d0000640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f62c810f9b0 con 0x7f62c8104f20 2024-09-10T09:38:16.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:16.470+0000 7f62d0000640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f62c810f830 con 0x7f62c81058e0 2024-09-10T09:38:16.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:16.470+0000 7f62d0000640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f62c810fb30 con 0x7f62c8103d30 2024-09-10T09:38:16.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:16.470+0000 7f62ccc00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f62c8104f20 0x7f62c819a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:16.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:16.470+0000 7f62ccc00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f62c8104f20 0x7f62c819a5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60682/0 (socket says 172.21.15.5:60682) 2024-09-10T09:38:16.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:16.470+0000 7f62ccc00640 1 -- 172.21.15.5:0/3827118271 learned_addr learned my addr 172.21.15.5:0/3827118271 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:38:16.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:16.470+0000 7f62cd600640 1 --2- 172.21.15.5:0/3827118271 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f62c8103d30 0x7f62c819a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:16.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:16.470+0000 7f62ce000640 1 --2- 172.21.15.5:0/3827118271 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f62c81058e0 0x7f62c81a1640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:16.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:16.470+0000 7f62ccc00640 1 -- 172.21.15.5:0/3827118271 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f62c8103d30 msgr2=0x7f62c819a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:38:16.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:16.470+0000 7f62ccc00640 1 --2- 172.21.15.5:0/3827118271 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f62c8103d30 0x7f62c819a090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:16.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:16.470+0000 7f62ccc00640 1 -- 172.21.15.5:0/3827118271 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f62c81058e0 msgr2=0x7f62c81a1640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:38:16.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:16.470+0000 7f62ccc00640 1 --2- 172.21.15.5:0/3827118271 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f62c81058e0 0x7f62c81a1640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:16.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:16.470+0000 7f62ccc00640 1 -- 172.21.15.5:0/3827118271 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62c81a1cb0 con 0x7f62c8104f20 2024-09-10T09:38:16.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:16.470+0000 7f62ce000640 1 --2- 172.21.15.5:0/3827118271 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f62c81058e0 0x7f62c81a1640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:38:16.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:16.471+0000 7f62ccc00640 1 --2- 172.21.15.5:0/3827118271 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f62c8104f20 0x7f62c819a5d0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f62b4005940 tx=0x7f62b4033d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:38:16.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:16.471+0000 7f62c6000640 1 -- 172.21.15.5:0/3827118271 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62b403f020 con 0x7f62c8104f20 2024-09-10T09:38:16.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:16.471+0000 7f62c6000640 1 -- 172.21.15.5:0/3827118271 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f62b4030850 con 0x7f62c8104f20 2024-09-10T09:38:16.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:16.471+0000 7f62d0000640 1 -- 172.21.15.5:0/3827118271 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62c81a1f40 con 0x7f62c8104f20 2024-09-10T09:38:16.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:16.471+0000 7f62c6000640 1 -- 172.21.15.5:0/3827118271 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62b4037040 con 0x7f62c8104f20 2024-09-10T09:38:16.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:16.471+0000 7f62d0000640 1 -- 172.21.15.5:0/3827118271 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62c81a23b0 con 0x7f62c8104f20 2024-09-10T09:38:16.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:16.472+0000 7f62d0000640 1 -- 172.21.15.5:0/3827118271 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f62c810b1c0 con 0x7f62c8104f20 2024-09-10T09:38:16.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:16.474+0000 7f62c6000640 1 -- 172.21.15.5:0/3827118271 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f62b4042020 con 0x7f62c8104f20 2024-09-10T09:38:16.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:16.475+0000 7f62c6000640 1 --2- 172.21.15.5:0/3827118271 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f62b00a2c10 0x7f62b00a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:16.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:16.475+0000 7f62c6000640 1 -- 172.21.15.5:0/3827118271 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f62b40ef7f0 con 0x7f62c8104f20 2024-09-10T09:38:16.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:16.475+0000 7f62cd600640 1 --2- 172.21.15.5:0/3827118271 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f62b00a2c10 0x7f62b00a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:16.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:16.476+0000 7f62cd600640 1 --2- 172.21.15.5:0/3827118271 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f62b00a2c10 0x7f62b00a50d0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f62c8104d80 tx=0x7f62bc005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:38:16.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:16.479+0000 7f62c6000640 1 -- 172.21.15.5:0/3827118271 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f62b4049020 con 0x7f62c8104f20 2024-09-10T09:38:16.556 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 10 09:38:16 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mgr-x[30575]: 2024-09-10T09:38:16.239+0000 7f5667800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T09:38:16.556 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 10 09:38:16 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mgr-x[30575]: 2024-09-10T09:38:16.239+0000 7f5667800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T09:38:16.556 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 10 09:38:16 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mgr-x[30575]: 2024-09-10T09:38:16.239+0000 7f5667800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T09:38:16.556 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 10 09:38:16 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mgr-x[30575]: 2024-09-10T09:38:16.239+0000 7f5667800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T09:38:16.556 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 10 09:38:16 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mgr-x[30575]: 2024-09-10T09:38:16.239+0000 7f5667800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T09:38:16.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:16.674+0000 7f62d0000640 1 -- 172.21.15.5:0/3827118271 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f62c8062e90 con 0x7f62c8104f20 2024-09-10T09:38:16.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:16.676+0000 7f62c6000640 1 -- 172.21.15.5:0/3827118271 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v96) ==== 74+0+24846 (secure 0 0 0) 0x7f62b40bc300 con 0x7f62c8104f20 2024-09-10T09:38:16.678 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-10T09:38:16.678 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":96,"fsid":"bade4b50-6f56-11ef-bcea-c7b262605968","created":"2024-09-10T09:27:13.298498+0000","modified":"2024-09-10T09:37:07.520520+0000","last_up_change":"2024-09-10T09:34:30.730066+0000","last_in_change":"2024-09-10T09:34:07.340995+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T09:30:26.042255+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T09:35:24.461031+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T09:35:27.155349+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-10T09:35:35.121462+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"085749a8-e882-4377-92dd-34bb266c0f0b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":954535266},{"type":"v1","addr":"172.21.15.5:6803","nonce":954535266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":954535266},{"type":"v1","addr":"172.21.15.5:6805","nonce":954535266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":954535266},{"type":"v1","addr":"172.21.15.5:6809","nonce":954535266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":954535266},{"type":"v1","addr":"172.21.15.5:6807","nonce":954535266}]},"public_addr":"172.21.15.5:6803/954535266","cluster_addr":"172.21.15.5:6805/954535266","heartbeat_back_addr":"172.21.15.5:6809/954535266","heartbeat_front_addr":"172.21.15.5:6807/954535266","state":["exists","up"]},{"osd":1,"uuid":"0a76c795-2573-4a92-969a-2ec5dd65c094","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6800","nonce":733924375},{"type":"v1","addr":"172.21.15.102:6801","nonce":733924375}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6802","nonce":733924375},{"type":"v1","addr":"172.21.15.102:6803","nonce":733924375}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6806","nonce":733924375},{"type":"v1","addr":"172.21.15.102:6807","nonce":733924375}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6804","nonce":733924375},{"type":"v1","addr":"172.21.15.102:6805","nonce":733924375}]},"public_addr":"172.21.15.102:6801/733924375","cluster_addr":"172.21.15.102:6803/733924375","heartbeat_back_addr":"172.21.15.102:6807/733924375","heartbeat_front_addr":"172.21.15.102:6805/733924375","state":["exists","up"]},{"osd":2,"uuid":"0c98fcb8-60ab-4473-8995-1d9e97c577e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6800","nonce":2236204189},{"type":"v1","addr":"172.21.15.159:6801","nonce":2236204189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":2236204189},{"type":"v1","addr":"172.21.15.159:6803","nonce":2236204189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":2236204189},{"type":"v1","addr":"172.21.15.159:6807","nonce":2236204189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":2236204189},{"type":"v1","addr":"172.21.15.159:6805","nonce":2236204189}]},"public_addr":"172.21.15.159:6801/2236204189","cluster_addr":"172.21.15.159:6803/2236204189","heartbeat_back_addr":"172.21.15.159:6807/2236204189","heartbeat_front_addr":"172.21.15.159:6805/2236204189","state":["exists","up"]},{"osd":3,"uuid":"88eecdeb-5cee-4f78-a3f9-a66bf4bf772a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":2221468725},{"type":"v1","addr":"172.21.15.5:6811","nonce":2221468725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":2221468725},{"type":"v1","addr":"172.21.15.5:6813","nonce":2221468725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6816","nonce":2221468725},{"type":"v1","addr":"172.21.15.5:6817","nonce":2221468725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":2221468725},{"type":"v1","addr":"172.21.15.5:6815","nonce":2221468725}]},"public_addr":"172.21.15.5:6811/2221468725","cluster_addr":"172.21.15.5:6813/2221468725","heartbeat_back_addr":"172.21.15.5:6817/2221468725","heartbeat_front_addr":"172.21.15.5:6815/2221468725","state":["exists","up"]},{"osd":4,"uuid":"f2e7314d-7938-4668-9011-45c175f66b48","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6808","nonce":594465753},{"type":"v1","addr":"172.21.15.102:6809","nonce":594465753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6810","nonce":594465753},{"type":"v1","addr":"172.21.15.102:6811","nonce":594465753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6814","nonce":594465753},{"type":"v1","addr":"172.21.15.102:6815","nonce":594465753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6812","nonce":594465753},{"type":"v1","addr":"172.21.15.102:6813","nonce":594465753}]},"public_addr":"172.21.15.102:6809/594465753","cluster_addr":"172.21.15.102:6811/594465753","heartbeat_back_addr":"172.21.15.102:6815/594465753","heartbeat_front_addr":"172.21.15.102:6813/594465753","state":["exists","up"]},{"osd":5,"uuid":"0c0366d3-658b-4bf0-ba17-aff28a02e948","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":3230280402},{"type":"v1","addr":"172.21.15.159:6809","nonce":3230280402}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":3230280402},{"type":"v1","addr":"172.21.15.159:6811","nonce":3230280402}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":3230280402},{"type":"v1","addr":"172.21.15.159:6815","nonce":3230280402}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":3230280402},{"type":"v1","addr":"172.21.15.159:6813","nonce":3230280402}]},"public_addr":"172.21.15.159:6809/3230280402","cluster_addr":"172.21.15.159:6811/3230280402","heartbeat_back_addr":"172.21.15.159:6815/3230280402","heartbeat_front_addr":"172.21.15.159:6813/3230280402","state":["exists","up"]},{"osd":6,"uuid":"682670a6-51d5-46e8-af6e-dc2d8c1cf407","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":1907651140},{"type":"v1","addr":"172.21.15.5:6819","nonce":1907651140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":1907651140},{"type":"v1","addr":"172.21.15.5:6821","nonce":1907651140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":1907651140},{"type":"v1","addr":"172.21.15.5:6825","nonce":1907651140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":1907651140},{"type":"v1","addr":"172.21.15.5:6823","nonce":1907651140}]},"public_addr":"172.21.15.5:6819/1907651140","cluster_addr":"172.21.15.5:6821/1907651140","heartbeat_back_addr":"172.21.15.5:6825/1907651140","heartbeat_front_addr":"172.21.15.5:6823/1907651140","state":["exists","up"]},{"osd":7,"uuid":"8ae0f4a9-99e7-4563-aa2a-15169684a8b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6816","nonce":692040360},{"type":"v1","addr":"172.21.15.102:6817","nonce":692040360}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6818","nonce":692040360},{"type":"v1","addr":"172.21.15.102:6819","nonce":692040360}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6822","nonce":692040360},{"type":"v1","addr":"172.21.15.102:6823","nonce":692040360}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6820","nonce":692040360},{"type":"v1","addr":"172.21.15.102:6821","nonce":692040360}]},"public_addr":"172.21.15.102:6817/692040360","cluster_addr":"172.21.15.102:6819/692040360","heartbeat_back_addr":"172.21.15.102:6823/692040360","heartbeat_front_addr":"172.21.15.102:6821/692040360","state":["exists","up"]},{"osd":8,"uuid":"e37c845e-ced9-4413-9bbe-bc090537c2bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":2107291799},{"type":"v1","addr":"172.21.15.159:6817","nonce":2107291799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":2107291799},{"type":"v1","addr":"172.21.15.159:6819","nonce":2107291799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":2107291799},{"type":"v1","addr":"172.21.15.159:6823","nonce":2107291799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":2107291799},{"type":"v1","addr":"172.21.15.159:6821","nonce":2107291799}]},"public_addr":"172.21.15.159:6817/2107291799","cluster_addr":"172.21.15.159:6819/2107291799","heartbeat_back_addr":"172.21.15.159:6823/2107291799","heartbeat_front_addr":"172.21.15.159:6821/2107291799","state":["exists","up"]},{"osd":9,"uuid":"91a5b6cc-08da-47be-af12-d0896a849309","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":902431281},{"type":"v1","addr":"172.21.15.5:6827","nonce":902431281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":902431281},{"type":"v1","addr":"172.21.15.5:6829","nonce":902431281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":902431281},{"type":"v1","addr":"172.21.15.5:6833","nonce":902431281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":902431281},{"type":"v1","addr":"172.21.15.5:6831","nonce":902431281}]},"public_addr":"172.21.15.5:6827/902431281","cluster_addr":"172.21.15.5:6829/902431281","heartbeat_back_addr":"172.21.15.5:6833/902431281","heartbeat_front_addr":"172.21.15.5:6831/902431281","state":["exists","up"]},{"osd":10,"uuid":"2211030c-b00e-402b-b4d7-c587eb29a490","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6824","nonce":2853806211},{"type":"v1","addr":"172.21.15.102:6825","nonce":2853806211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6826","nonce":2853806211},{"type":"v1","addr":"172.21.15.102:6827","nonce":2853806211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6830","nonce":2853806211},{"type":"v1","addr":"172.21.15.102:6831","nonce":2853806211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6828","nonce":2853806211},{"type":"v1","addr":"172.21.15.102:6829","nonce":2853806211}]},"public_addr":"172.21.15.102:6825/2853806211","cluster_addr":"172.21.15.102:6827/2853806211","heartbeat_back_addr":"172.21.15.102:6831/2853806211","heartbeat_front_addr":"172.21.15.102:6829/2853806211","state":["exists","up"]},{"osd":11,"uuid":"4e9699ec-8ff2-4005-9d84-af5600b3a2ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6824","nonce":969941218},{"type":"v1","addr":"172.21.15.159:6825","nonce":969941218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":969941218},{"type":"v1","addr":"172.21.15.159:6827","nonce":969941218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6830","nonce":969941218},{"type":"v1","addr":"172.21.15.159:6831","nonce":969941218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":969941218},{"type":"v1","addr":"172.21.15.159:6829","nonce":969941218}]},"public_addr":"172.21.15.159:6825/969941218","cluster_addr":"172.21.15.159:6827/969941218","heartbeat_back_addr":"172.21.15.159:6831/969941218","heartbeat_front_addr":"172.21.15.159:6829/969941218","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:29:34.416490+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:30:00.246101+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:30:24.300464+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:30:54.663623+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:31:21.426292+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:31:46.738914+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:32:16.196571+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:32:42.549243+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:33:08.178651+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:33:37.477911+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:34:03.114026+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:34:28.891924+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4}]},{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"3.0","mappings":[{"from":4,"to":8}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":6}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9},{"from":5,"to":7}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]},{"pgid":"3.39","mappings":[{"from":5,"to":6}]},{"pgid":"4.8","mappings":[{"from":5,"to":8}]},{"pgid":"4.c","mappings":[{"from":9,"to":10}]},{"pgid":"4.18","mappings":[{"from":9,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.5:6801/1527422686":"2024-09-11T09:28:05.436742+0000","172.21.15.5:0/2276144345":"2024-09-11T09:28:05.436742+0000","172.21.15.5:0/412040104":"2024-09-11T09:28:05.436742+0000","172.21.15.5:6800/1527422686":"2024-09-11T09:28:05.436742+0000","172.21.15.5:0/1856581170":"2024-09-11T09:28:05.436742+0000","172.21.15.5:0/2847668298":"2024-09-11T09:27:37.778500+0000","172.21.15.5:6801/14425913":"2024-09-11T09:27:37.778500+0000","172.21.15.5:0/166910782":"2024-09-11T09:27:37.778500+0000","172.21.15.5:6800/14425913":"2024-09-11T09:27:37.778500+0000","172.21.15.5:0/1711858179":"2024-09-11T09:27:37.778500+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T09:38:16.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:16.678+0000 7f62d0000640 1 -- 172.21.15.5:0/3827118271 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f62b00a2c10 msgr2=0x7f62b00a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:38:16.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:16.678+0000 7f62d0000640 1 --2- 172.21.15.5:0/3827118271 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f62b00a2c10 0x7f62b00a50d0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f62c8104d80 tx=0x7f62bc005f90 comp rx=0 tx=0).stop 2024-09-10T09:38:16.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:16.678+0000 7f62d0000640 1 -- 172.21.15.5:0/3827118271 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f62c8104f20 msgr2=0x7f62c819a5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:38:16.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:16.678+0000 7f62d0000640 1 --2- 172.21.15.5:0/3827118271 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f62c8104f20 0x7f62c819a5d0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f62b4005940 tx=0x7f62b4033d90 comp rx=0 tx=0).stop 2024-09-10T09:38:16.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:16.678+0000 7f62d0000640 1 -- 172.21.15.5:0/3827118271 shutdown_connections 2024-09-10T09:38:16.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:16.678+0000 7f62d0000640 1 --2- 172.21.15.5:0/3827118271 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f62b00a2c10 0x7f62b00a50d0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:16.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:16.678+0000 7f62d0000640 1 --2- 172.21.15.5:0/3827118271 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f62c81058e0 0x7f62c81a1640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:16.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:16.678+0000 7f62d0000640 1 --2- 172.21.15.5:0/3827118271 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f62c8104f20 0x7f62c819a5d0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:16.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:16.678+0000 7f62d0000640 1 --2- 172.21.15.5:0/3827118271 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f62c8103d30 0x7f62c819a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:16.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:16.678+0000 7f62d0000640 1 -- 172.21.15.5:0/3827118271 >> 172.21.15.5:0/3827118271 conn(0x7f62c80ff4b0 msgr2=0x7f62c8108390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:38:16.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:16.678+0000 7f62d0000640 1 -- 172.21.15.5:0/3827118271 shutdown_connections 2024-09-10T09:38:16.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:16.678+0000 7f62d0000640 1 -- 172.21.15.5:0/3827118271 wait complete. 2024-09-10T09:38:16.697 INFO:tasks.fwd_scrub.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-10T09:30:26.042255+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-10T09:35:24.461031+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-10T09:35:27.155349+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-09-10T09:35:35.121462+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '92', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-09-10T09:38:16.697 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-10T09:38:16.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:16.858+0000 7f0b7a600640 1 -- 172.21.15.5:0/3064954039 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0b741036e0 msgr2=0x7f0b74109f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:38:16.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:16.858+0000 7f0b7a600640 1 --2- 172.21.15.5:0/3064954039 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0b741036e0 0x7f0b74109f70 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f0b6c009be0 tx=0x7f0b6c02a6c0 comp rx=0 tx=0).stop 2024-09-10T09:38:16.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:16.858+0000 7f0b7a600640 1 -- 172.21.15.5:0/3064954039 shutdown_connections 2024-09-10T09:38:16.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:16.858+0000 7f0b7a600640 1 --2- 172.21.15.5:0/3064954039 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0b741036e0 0x7f0b74109f70 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:16.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:16.858+0000 7f0b7a600640 1 --2- 172.21.15.5:0/3064954039 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0b74102d20 0x7f0b741031a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:16.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:16.858+0000 7f0b7a600640 1 --2- 172.21.15.5:0/3064954039 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0b74101b30 0x7f0b74101f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:16.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:16.858+0000 7f0b7a600640 1 -- 172.21.15.5:0/3064954039 >> 172.21.15.5:0/3064954039 conn(0x7f0b740fd290 msgr2=0x7f0b740ff6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:38:16.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:16.858+0000 7f0b7a600640 1 -- 172.21.15.5:0/3064954039 shutdown_connections 2024-09-10T09:38:16.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:16.858+0000 7f0b7a600640 1 -- 172.21.15.5:0/3064954039 wait complete. 2024-09-10T09:38:16.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:16.859+0000 7f0b7a600640 1 Processor -- start 2024-09-10T09:38:16.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:16.859+0000 7f0b7a600640 1 -- start start 2024-09-10T09:38:16.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:16.860+0000 7f0b7a600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0b74101b30 0x7f0b74197e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:16.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:16.860+0000 7f0b7a600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0b74102d20 0x7f0b74198360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:16.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:16.860+0000 7f0b7a600640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0b741036e0 0x7f0b7419f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:16.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:16.860+0000 7f0b7a600640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f0b7410d790 con 0x7f0b74102d20 2024-09-10T09:38:16.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:16.860+0000 7f0b7a600640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f0b7410d610 con 0x7f0b74101b30 2024-09-10T09:38:16.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:16.860+0000 7f0b7a600640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f0b7410d910 con 0x7f0b741036e0 2024-09-10T09:38:16.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:16.860+0000 7f0b73400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0b74101b30 0x7f0b74197e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:16.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:16.860+0000 7f0b73e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0b741036e0 0x7f0b7419f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:16.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:16.860+0000 7f0b72a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0b74102d20 0x7f0b74198360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:16.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:16.860+0000 7f0b73400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0b74101b30 0x7f0b74197e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:44686/0 (socket says 172.21.15.5:44686) 2024-09-10T09:38:16.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:16.860+0000 7f0b73e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0b741036e0 0x7f0b7419f3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:52892/0 (socket says 172.21.15.5:52892) 2024-09-10T09:38:16.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:16.860+0000 7f0b73400640 1 -- 172.21.15.5:0/1230699103 learned_addr learned my addr 172.21.15.5:0/1230699103 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:38:16.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:16.860+0000 7f0b73e00640 1 -- 172.21.15.5:0/1230699103 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0b74101b30 msgr2=0x7f0b74197e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:38:16.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:16.860+0000 7f0b73e00640 1 --2- 172.21.15.5:0/1230699103 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0b74101b30 0x7f0b74197e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:16.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:16.860+0000 7f0b73e00640 1 -- 172.21.15.5:0/1230699103 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0b74102d20 msgr2=0x7f0b74198360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:38:16.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:16.860+0000 7f0b73e00640 1 --2- 172.21.15.5:0/1230699103 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0b74102d20 0x7f0b74198360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:16.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:16.860+0000 7f0b73e00640 1 -- 172.21.15.5:0/1230699103 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b7419fa40 con 0x7f0b741036e0 2024-09-10T09:38:16.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:16.861+0000 7f0b73400640 1 --2- 172.21.15.5:0/1230699103 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0b74101b30 0x7f0b74197e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:38:16.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:16.861+0000 7f0b73e00640 1 --2- 172.21.15.5:0/1230699103 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0b741036e0 0x7f0b7419f3d0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f0b6c034040 tx=0x7f0b6c0305d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:38:16.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:16.861+0000 7f0b6be00640 1 -- 172.21.15.5:0/1230699103 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b6c041020 con 0x7f0b741036e0 2024-09-10T09:38:16.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:16.861+0000 7f0b6be00640 1 -- 172.21.15.5:0/1230699103 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0b6c030740 con 0x7f0b741036e0 2024-09-10T09:38:16.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:16.861+0000 7f0b7a600640 1 -- 172.21.15.5:0/1230699103 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b7419fcd0 con 0x7f0b741036e0 2024-09-10T09:38:16.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:16.861+0000 7f0b6be00640 1 -- 172.21.15.5:0/1230699103 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b6c036040 con 0x7f0b741036e0 2024-09-10T09:38:16.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:16.861+0000 7f0b7a600640 1 -- 172.21.15.5:0/1230699103 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b741a0190 con 0x7f0b741036e0 2024-09-10T09:38:16.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:16.862+0000 7f0b7a600640 1 -- 172.21.15.5:0/1230699103 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b74108ec0 con 0x7f0b741036e0 2024-09-10T09:38:16.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:16.864+0000 7f0b6be00640 1 -- 172.21.15.5:0/1230699103 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f0b6c044020 con 0x7f0b741036e0 2024-09-10T09:38:16.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:16.865+0000 7f0b6be00640 1 --2- 172.21.15.5:0/1230699103 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f0b580a2b90 0x7f0b580a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:16.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:16.866+0000 7f0b6be00640 1 -- 172.21.15.5:0/1230699103 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f0b6c0f0090 con 0x7f0b741036e0 2024-09-10T09:38:16.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:16.866+0000 7f0b73400640 1 --2- 172.21.15.5:0/1230699103 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f0b580a2b90 0x7f0b580a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:16.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:16.867+0000 7f0b73400640 1 --2- 172.21.15.5:0/1230699103 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f0b580a2b90 0x7f0b580a5050 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f0b64011490 tx=0x7f0b64009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:38:16.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:16.869+0000 7f0b6be00640 1 -- 172.21.15.5:0/1230699103 <== mon.1 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f0b6c04b020 con 0x7f0b741036e0 2024-09-10T09:38:17.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:38:16 smithi005 ceph-mon[30350]: pgmap v391: 129 pgs: 129 active+clean; 646 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-10T09:38:17.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:38:16 smithi005 ceph-mon[30350]: from='client.? 172.21.15.5:0/2648712749' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-09-10T09:38:17.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:38:16 smithi005 ceph-mon[30350]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T09:38:17.057 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:38:16 smithi005 ceph-mon[30350]: from='client.25220 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T09:38:17.057 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:38:16 smithi005 ceph-mon[30350]: from='client.? 172.21.15.5:0/3827118271' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T09:38:17.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:17.057+0000 7f0b7a600640 1 -- 172.21.15.5:0/1230699103 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f0b74062e90 con 0x7f0b741036e0 2024-09-10T09:38:17.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:17.057+0000 7f0b6be00640 1 -- 172.21.15.5:0/1230699103 <== mon.1 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v96) ==== 93+0+10 (secure 0 0 0) 0x7f0b6c0bcba0 con 0x7f0b741036e0 2024-09-10T09:38:17.059 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 1 2024-09-10T09:38:17.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:17.060+0000 7f0b7a600640 1 -- 172.21.15.5:0/1230699103 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f0b580a2b90 msgr2=0x7f0b580a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:38:17.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:17.060+0000 7f0b7a600640 1 --2- 172.21.15.5:0/1230699103 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f0b580a2b90 0x7f0b580a5050 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f0b64011490 tx=0x7f0b64009260 comp rx=0 tx=0).stop 2024-09-10T09:38:17.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:17.060+0000 7f0b7a600640 1 -- 172.21.15.5:0/1230699103 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0b741036e0 msgr2=0x7f0b7419f3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:38:17.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:17.060+0000 7f0b7a600640 1 --2- 172.21.15.5:0/1230699103 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0b741036e0 0x7f0b7419f3d0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f0b6c034040 tx=0x7f0b6c0305d0 comp rx=0 tx=0).stop 2024-09-10T09:38:17.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:17.060+0000 7f0b7a600640 1 -- 172.21.15.5:0/1230699103 shutdown_connections 2024-09-10T09:38:17.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:17.060+0000 7f0b7a600640 1 --2- 172.21.15.5:0/1230699103 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f0b580a2b90 0x7f0b580a5050 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:17.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:17.060+0000 7f0b7a600640 1 --2- 172.21.15.5:0/1230699103 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0b741036e0 0x7f0b7419f3d0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:17.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:17.060+0000 7f0b7a600640 1 --2- 172.21.15.5:0/1230699103 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0b74102d20 0x7f0b74198360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:17.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:17.060+0000 7f0b7a600640 1 --2- 172.21.15.5:0/1230699103 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0b74101b30 0x7f0b74197e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:17.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:17.060+0000 7f0b7a600640 1 -- 172.21.15.5:0/1230699103 >> 172.21.15.5:0/1230699103 conn(0x7f0b740fd290 msgr2=0x7f0b74106090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:38:17.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:17.060+0000 7f0b7a600640 1 -- 172.21.15.5:0/1230699103 shutdown_connections 2024-09-10T09:38:17.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:17.060+0000 7f0b7a600640 1 -- 172.21.15.5:0/1230699103 wait complete. 2024-09-10T09:38:17.079 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-10T09:38:17.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:38:16 smithi159 ceph-mon[35513]: pgmap v391: 129 pgs: 129 active+clean; 646 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-10T09:38:17.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:38:16 smithi159 ceph-mon[35513]: from='client.? 172.21.15.5:0/2648712749' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-09-10T09:38:17.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:38:16 smithi159 ceph-mon[35513]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T09:38:17.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:38:16 smithi159 ceph-mon[35513]: from='client.25220 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T09:38:17.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:38:16 smithi159 ceph-mon[35513]: from='client.? 172.21.15.5:0/3827118271' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T09:38:17.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:38:16 smithi102 ceph-mon[35442]: pgmap v391: 129 pgs: 129 active+clean; 646 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-10T09:38:17.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:38:16 smithi102 ceph-mon[35442]: from='client.? 172.21.15.5:0/2648712749' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-09-10T09:38:17.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:38:16 smithi102 ceph-mon[35442]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T09:38:17.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:38:16 smithi102 ceph-mon[35442]: from='client.25220 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T09:38:17.135 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:38:16 smithi102 ceph-mon[35442]: from='client.? 172.21.15.5:0/3827118271' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T09:38:17.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:17.239+0000 7fd27b400640 1 -- 172.21.15.5:0/845351975 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd2741058d0 msgr2=0x7fd27410c160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:38:17.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:17.239+0000 7fd27b400640 1 --2- 172.21.15.5:0/845351975 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd2741058d0 0x7fd27410c160 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fd26c009ca0 tx=0x7fd26c02a8c0 comp rx=0 tx=0).stop 2024-09-10T09:38:17.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:17.239+0000 7fd27b400640 1 -- 172.21.15.5:0/845351975 shutdown_connections 2024-09-10T09:38:17.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:17.239+0000 7fd27b400640 1 --2- 172.21.15.5:0/845351975 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd2741058d0 0x7fd27410c160 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:17.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:17.239+0000 7fd27b400640 1 --2- 172.21.15.5:0/845351975 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd274104f10 0x7fd274105390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:17.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:17.239+0000 7fd27b400640 1 --2- 172.21.15.5:0/845351975 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd274103d20 0x7fd274104120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:17.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:17.240+0000 7fd27b400640 1 -- 172.21.15.5:0/845351975 >> 172.21.15.5:0/845351975 conn(0x7fd2740ff4a0 msgr2=0x7fd2741018c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:38:17.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:17.240+0000 7fd27b400640 1 -- 172.21.15.5:0/845351975 shutdown_connections 2024-09-10T09:38:17.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:17.240+0000 7fd27b400640 1 -- 172.21.15.5:0/845351975 wait complete. 2024-09-10T09:38:17.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:17.240+0000 7fd27b400640 1 Processor -- start 2024-09-10T09:38:17.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:17.241+0000 7fd27b400640 1 -- start start 2024-09-10T09:38:17.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:17.241+0000 7fd27b400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd274103d20 0x7fd274199fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:17.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:17.241+0000 7fd27b400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd274104f10 0x7fd27419a520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:17.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:17.241+0000 7fd27b400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd2741058d0 0x7fd2741a1590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:17.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:17.241+0000 7fd27b400640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd27410f9f0 con 0x7fd274103d20 2024-09-10T09:38:17.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:17.241+0000 7fd27b400640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fd27410f870 con 0x7fd2741058d0 2024-09-10T09:38:17.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:17.241+0000 7fd27b400640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fd27410fb70 con 0x7fd274104f10 2024-09-10T09:38:17.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:17.241+0000 7fd278a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd274103d20 0x7fd274199fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:17.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:17.241+0000 7fd278a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd274103d20 0x7fd274199fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60742/0 (socket says 172.21.15.5:60742) 2024-09-10T09:38:17.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:17.241+0000 7fd278a00640 1 -- 172.21.15.5:0/1217120734 learned_addr learned my addr 172.21.15.5:0/1217120734 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:38:17.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:17.241+0000 7fd273e00640 1 --2- 172.21.15.5:0/1217120734 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd274104f10 0x7fd27419a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:17.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:17.241+0000 7fd279400640 1 --2- 172.21.15.5:0/1217120734 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd2741058d0 0x7fd2741a1590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:17.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:17.242+0000 7fd278a00640 1 -- 172.21.15.5:0/1217120734 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd274104f10 msgr2=0x7fd27419a520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:38:17.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:17.242+0000 7fd278a00640 1 --2- 172.21.15.5:0/1217120734 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd274104f10 0x7fd27419a520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:17.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:17.242+0000 7fd278a00640 1 -- 172.21.15.5:0/1217120734 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd2741058d0 msgr2=0x7fd2741a1590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:38:17.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:17.242+0000 7fd278a00640 1 --2- 172.21.15.5:0/1217120734 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd2741058d0 0x7fd2741a1590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:17.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:17.242+0000 7fd278a00640 1 -- 172.21.15.5:0/1217120734 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2741a1c00 con 0x7fd274103d20 2024-09-10T09:38:17.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:17.242+0000 7fd279400640 1 --2- 172.21.15.5:0/1217120734 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd2741058d0 0x7fd2741a1590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:38:17.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:17.242+0000 7fd278a00640 1 --2- 172.21.15.5:0/1217120734 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd274103d20 0x7fd274199fe0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fd26800e9c0 tx=0x7fd26800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:38:17.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:17.242+0000 7fd271600640 1 -- 172.21.15.5:0/1217120734 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd26800cd80 con 0x7fd274103d20 2024-09-10T09:38:17.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:17.242+0000 7fd271600640 1 -- 172.21.15.5:0/1217120734 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd2680035f0 con 0x7fd274103d20 2024-09-10T09:38:17.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:17.242+0000 7fd271600640 1 -- 172.21.15.5:0/1217120734 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd268010690 con 0x7fd274103d20 2024-09-10T09:38:17.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:17.242+0000 7fd27b400640 1 -- 172.21.15.5:0/1217120734 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2741a1ef0 con 0x7fd274103d20 2024-09-10T09:38:17.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:17.242+0000 7fd27b400640 1 -- 172.21.15.5:0/1217120734 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2741a2300 con 0x7fd274103d20 2024-09-10T09:38:17.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:17.243+0000 7fd27b400640 1 -- 172.21.15.5:0/1217120734 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd27410b0c0 con 0x7fd274103d20 2024-09-10T09:38:17.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:17.245+0000 7fd271600640 1 -- 172.21.15.5:0/1217120734 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7fd2680037a0 con 0x7fd274103d20 2024-09-10T09:38:17.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:17.246+0000 7fd271600640 1 --2- 172.21.15.5:0/1217120734 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fd25c0a2c10 0x7fd25c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:17.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:17.247+0000 7fd271600640 1 -- 172.21.15.5:0/1217120734 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fd2680cf840 con 0x7fd274103d20 2024-09-10T09:38:17.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:17.247+0000 7fd273e00640 1 --2- 172.21.15.5:0/1217120734 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fd25c0a2c10 0x7fd25c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:17.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:17.247+0000 7fd273e00640 1 --2- 172.21.15.5:0/1217120734 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fd25c0a2c10 0x7fd25c0a50d0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fd27419b4f0 tx=0x7fd260005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:38:17.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:17.250+0000 7fd271600640 1 -- 172.21.15.5:0/1217120734 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd26809c350 con 0x7fd274103d20 2024-09-10T09:38:17.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:17.434+0000 7fd27b400640 1 -- 172.21.15.5:0/1217120734 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fd274062f40 con 0x7fd274103d20 2024-09-10T09:38:17.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:17.434+0000 7fd271600640 1 -- 172.21.15.5:0/1217120734 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v96) ==== 104+0+11 (secure 0 0 0) 0x7fd26809bc50 con 0x7fd274103d20 2024-09-10T09:38:17.436 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 32 2024-09-10T09:38:17.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:17.436+0000 7fd27b400640 1 -- 172.21.15.5:0/1217120734 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fd25c0a2c10 msgr2=0x7fd25c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:38:17.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:17.436+0000 7fd27b400640 1 --2- 172.21.15.5:0/1217120734 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fd25c0a2c10 0x7fd25c0a50d0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fd27419b4f0 tx=0x7fd260005f90 comp rx=0 tx=0).stop 2024-09-10T09:38:17.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:17.436+0000 7fd27b400640 1 -- 172.21.15.5:0/1217120734 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd274103d20 msgr2=0x7fd274199fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:38:17.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:17.436+0000 7fd27b400640 1 --2- 172.21.15.5:0/1217120734 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd274103d20 0x7fd274199fe0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fd26800e9c0 tx=0x7fd26800ee80 comp rx=0 tx=0).stop 2024-09-10T09:38:17.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:17.437+0000 7fd27b400640 1 -- 172.21.15.5:0/1217120734 shutdown_connections 2024-09-10T09:38:17.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:17.437+0000 7fd27b400640 1 --2- 172.21.15.5:0/1217120734 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fd25c0a2c10 0x7fd25c0a50d0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:17.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:17.437+0000 7fd27b400640 1 --2- 172.21.15.5:0/1217120734 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd2741058d0 0x7fd2741a1590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:17.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:17.437+0000 7fd27b400640 1 --2- 172.21.15.5:0/1217120734 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd274104f10 0x7fd27419a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:17.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:17.437+0000 7fd27b400640 1 --2- 172.21.15.5:0/1217120734 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd274103d20 0x7fd274199fe0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:17.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:17.437+0000 7fd27b400640 1 -- 172.21.15.5:0/1217120734 >> 172.21.15.5:0/1217120734 conn(0x7fd2740ff4a0 msgr2=0x7fd274108290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:38:17.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:17.437+0000 7fd27b400640 1 -- 172.21.15.5:0/1217120734 shutdown_connections 2024-09-10T09:38:17.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:17.437+0000 7fd27b400640 1 -- 172.21.15.5:0/1217120734 wait complete. 2024-09-10T09:38:17.455 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-10T09:38:17.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:17.614+0000 7f031e400640 1 -- 172.21.15.5:0/688107898 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f03181035b0 msgr2=0x7f0318109e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:38:17.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:17.614+0000 7f031e400640 1 --2- 172.21.15.5:0/688107898 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f03181035b0 0x7f0318109e40 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f0310009ca0 tx=0x7f031002a8c0 comp rx=0 tx=0).stop 2024-09-10T09:38:17.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:17.614+0000 7f031e400640 1 -- 172.21.15.5:0/688107898 shutdown_connections 2024-09-10T09:38:17.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:17.614+0000 7f031e400640 1 --2- 172.21.15.5:0/688107898 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f03181035b0 0x7f0318109e40 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:17.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:17.614+0000 7f031e400640 1 --2- 172.21.15.5:0/688107898 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0318102bf0 0x7f0318103070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:17.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:17.614+0000 7f031e400640 1 --2- 172.21.15.5:0/688107898 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0318101a00 0x7f0318101e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:17.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:17.614+0000 7f031e400640 1 -- 172.21.15.5:0/688107898 >> 172.21.15.5:0/688107898 conn(0x7f03180fd180 msgr2=0x7f03180ff5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:38:17.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:17.614+0000 7f031e400640 1 -- 172.21.15.5:0/688107898 shutdown_connections 2024-09-10T09:38:17.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:17.614+0000 7f031e400640 1 -- 172.21.15.5:0/688107898 wait complete. 2024-09-10T09:38:17.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:17.615+0000 7f031e400640 1 Processor -- start 2024-09-10T09:38:17.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:17.615+0000 7f031e400640 1 -- start start 2024-09-10T09:38:17.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:17.615+0000 7f031e400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0318101a00 0x7f0318075fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:17.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:17.615+0000 7f031e400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0318102bf0 0x7f03180745e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:17.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:17.615+0000 7f031e400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f03181035b0 0x7f0318074b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:17.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:17.616+0000 7f031e400640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f031810d6d0 con 0x7f0318102bf0 2024-09-10T09:38:17.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:17.616+0000 7f031e400640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f031810d550 con 0x7f0318101a00 2024-09-10T09:38:17.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:17.616+0000 7f031e400640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f031810d850 con 0x7f03181035b0 2024-09-10T09:38:17.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:17.616+0000 7f030fe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0318102bf0 0x7f03180745e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:17.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:17.616+0000 7f031da00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f03181035b0 0x7f0318074b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:17.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:17.616+0000 7f030fe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0318102bf0 0x7f03180745e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60750/0 (socket says 172.21.15.5:60750) 2024-09-10T09:38:17.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:17.616+0000 7f030fe00640 1 -- 172.21.15.5:0/2518545547 learned_addr learned my addr 172.21.15.5:0/2518545547 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:38:17.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:17.616+0000 7f031d000640 1 --2- 172.21.15.5:0/2518545547 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0318101a00 0x7f0318075fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:17.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:17.616+0000 7f030fe00640 1 -- 172.21.15.5:0/2518545547 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f03181035b0 msgr2=0x7f0318074b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:38:17.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:17.616+0000 7f030fe00640 1 --2- 172.21.15.5:0/2518545547 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f03181035b0 0x7f0318074b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:17.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:17.616+0000 7f030fe00640 1 -- 172.21.15.5:0/2518545547 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0318101a00 msgr2=0x7f0318075fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:38:17.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:17.616+0000 7f030fe00640 1 --2- 172.21.15.5:0/2518545547 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0318101a00 0x7f0318075fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:17.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:17.616+0000 7f030fe00640 1 -- 172.21.15.5:0/2518545547 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03180753b0 con 0x7f0318102bf0 2024-09-10T09:38:17.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:17.616+0000 7f031da00640 1 --2- 172.21.15.5:0/2518545547 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f03181035b0 0x7f0318074b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:38:17.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:17.617+0000 7f030fe00640 1 --2- 172.21.15.5:0/2518545547 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0318102bf0 0x7f03180745e0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f030000e9f0 tx=0x7f030000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:38:17.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:17.617+0000 7f0316000640 1 -- 172.21.15.5:0/2518545547 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f030000cd80 con 0x7f0318102bf0 2024-09-10T09:38:17.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:17.617+0000 7f0316000640 1 -- 172.21.15.5:0/2518545547 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0300010910 con 0x7f0318102bf0 2024-09-10T09:38:17.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:17.617+0000 7f0316000640 1 -- 172.21.15.5:0/2518545547 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0300018990 con 0x7f0318102bf0 2024-09-10T09:38:17.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:17.617+0000 7f031e400640 1 -- 172.21.15.5:0/2518545547 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03181a1db0 con 0x7f0318102bf0 2024-09-10T09:38:17.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:17.617+0000 7f031e400640 1 -- 172.21.15.5:0/2518545547 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03181a2220 con 0x7f0318102bf0 2024-09-10T09:38:17.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:17.618+0000 7f031e400640 1 -- 172.21.15.5:0/2518545547 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0318108df0 con 0x7f0318102bf0 2024-09-10T09:38:17.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:17.620+0000 7f0316000640 1 -- 172.21.15.5:0/2518545547 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f0300010ac0 con 0x7f0318102bf0 2024-09-10T09:38:17.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:17.621+0000 7f0316000640 1 --2- 172.21.15.5:0/2518545547 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f02fc0a2c10 0x7f02fc0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:17.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:17.621+0000 7f0316000640 1 -- 172.21.15.5:0/2518545547 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f03000cf710 con 0x7f0318102bf0 2024-09-10T09:38:17.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:17.621+0000 7f031d000640 1 --2- 172.21.15.5:0/2518545547 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f02fc0a2c10 0x7f02fc0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:17.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:17.622+0000 7f031d000640 1 --2- 172.21.15.5:0/2518545547 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f02fc0a2c10 0x7f02fc0a50d0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f0318102a50 tx=0x7f0308005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:38:17.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:17.625+0000 7f0316000640 1 -- 172.21.15.5:0/2518545547 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f030009bbd0 con 0x7f0318102bf0 2024-09-10T09:38:17.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:17.810+0000 7f031e400640 1 -- 172.21.15.5:0/2518545547 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f0318062e20 con 0x7f0318102bf0 2024-09-10T09:38:17.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:17.811+0000 7f0316000640 1 -- 172.21.15.5:0/2518545547 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v96) ==== 100+0+11 (secure 0 0 0) 0x7f030009fed0 con 0x7f0318102bf0 2024-09-10T09:38:17.813 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 64 2024-09-10T09:38:17.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:17.813+0000 7f031e400640 1 -- 172.21.15.5:0/2518545547 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f02fc0a2c10 msgr2=0x7f02fc0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:38:17.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:17.813+0000 7f031e400640 1 --2- 172.21.15.5:0/2518545547 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f02fc0a2c10 0x7f02fc0a50d0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f0318102a50 tx=0x7f0308005f90 comp rx=0 tx=0).stop 2024-09-10T09:38:17.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:17.813+0000 7f031e400640 1 -- 172.21.15.5:0/2518545547 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0318102bf0 msgr2=0x7f03180745e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:38:17.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:17.813+0000 7f031e400640 1 --2- 172.21.15.5:0/2518545547 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0318102bf0 0x7f03180745e0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f030000e9f0 tx=0x7f030000eeb0 comp rx=0 tx=0).stop 2024-09-10T09:38:17.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:17.813+0000 7f031e400640 1 -- 172.21.15.5:0/2518545547 shutdown_connections 2024-09-10T09:38:17.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:17.813+0000 7f031e400640 1 --2- 172.21.15.5:0/2518545547 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f02fc0a2c10 0x7f02fc0a50d0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:17.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:17.813+0000 7f031e400640 1 --2- 172.21.15.5:0/2518545547 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f03181035b0 0x7f0318074b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:17.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:17.813+0000 7f031e400640 1 --2- 172.21.15.5:0/2518545547 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0318102bf0 0x7f03180745e0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:17.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:17.813+0000 7f031e400640 1 --2- 172.21.15.5:0/2518545547 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0318101a00 0x7f0318075fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:17.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:17.813+0000 7f031e400640 1 -- 172.21.15.5:0/2518545547 >> 172.21.15.5:0/2518545547 conn(0x7f03180fd180 msgr2=0x7f0318105fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:38:17.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:17.813+0000 7f031e400640 1 -- 172.21.15.5:0/2518545547 shutdown_connections 2024-09-10T09:38:17.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:17.813+0000 7f031e400640 1 -- 172.21.15.5:0/2518545547 wait complete. 2024-09-10T09:38:17.832 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-09-10T09:38:17.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:17.996+0000 7f1797e00640 1 -- 172.21.15.5:0/2168124247 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1790104f10 msgr2=0x7f1790105390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:38:17.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:17.996+0000 7f1797e00640 1 --2- 172.21.15.5:0/2168124247 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1790104f10 0x7f1790105390 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f177c009900 tx=0x7f177c02a410 comp rx=0 tx=0).stop 2024-09-10T09:38:17.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:17.996+0000 7f1797e00640 1 -- 172.21.15.5:0/2168124247 shutdown_connections 2024-09-10T09:38:17.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:17.996+0000 7f1797e00640 1 --2- 172.21.15.5:0/2168124247 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f17901058d0 0x7f179010c160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:17.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:17.996+0000 7f1797e00640 1 --2- 172.21.15.5:0/2168124247 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1790104f10 0x7f1790105390 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:17.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:17.996+0000 7f1797e00640 1 --2- 172.21.15.5:0/2168124247 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1790103d20 0x7f1790104120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:17.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:17.996+0000 7f1797e00640 1 -- 172.21.15.5:0/2168124247 >> 172.21.15.5:0/2168124247 conn(0x7f17900ff4a0 msgr2=0x7f17901018c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:38:17.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:17.996+0000 7f1797e00640 1 -- 172.21.15.5:0/2168124247 shutdown_connections 2024-09-10T09:38:17.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:17.996+0000 7f1797e00640 1 -- 172.21.15.5:0/2168124247 wait complete. 2024-09-10T09:38:18.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:17.997+0000 7f1797e00640 1 Processor -- start 2024-09-10T09:38:18.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:17.997+0000 7f1797e00640 1 -- start start 2024-09-10T09:38:18.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:17.998+0000 7f1797e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1790103d20 0x7f179019a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:18.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:17.998+0000 7f1797e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1790104f10 0x7f179019a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:18.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:17.998+0000 7f1797e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f17901058d0 0x7f17901a1660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:18.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:17.998+0000 7f1797e00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f179010f9a0 con 0x7f17901058d0 2024-09-10T09:38:18.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:17.998+0000 7f1797e00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f179010f820 con 0x7f1790103d20 2024-09-10T09:38:18.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:17.998+0000 7f1797e00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f179010fb20 con 0x7f1790104f10 2024-09-10T09:38:18.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:17.998+0000 7f1795e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f17901058d0 0x7f17901a1660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:18.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:17.998+0000 7f1795e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f17901058d0 0x7f17901a1660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60778/0 (socket says 172.21.15.5:60778) 2024-09-10T09:38:18.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:17.998+0000 7f1795e00640 1 -- 172.21.15.5:0/3655206490 learned_addr learned my addr 172.21.15.5:0/3655206490 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:38:18.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:17.998+0000 7f1795400640 1 --2- 172.21.15.5:0/3655206490 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1790103d20 0x7f179019a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:18.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:17.998+0000 7f1794a00640 1 --2- 172.21.15.5:0/3655206490 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1790104f10 0x7f179019a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:18.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:17.998+0000 7f1795e00640 1 -- 172.21.15.5:0/3655206490 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1790104f10 msgr2=0x7f179019a5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:38:18.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:17.998+0000 7f1795e00640 1 --2- 172.21.15.5:0/3655206490 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1790104f10 0x7f179019a5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:18.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:17.998+0000 7f1795e00640 1 -- 172.21.15.5:0/3655206490 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1790103d20 msgr2=0x7f179019a0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:38:18.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:17.998+0000 7f1795e00640 1 --2- 172.21.15.5:0/3655206490 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1790103d20 0x7f179019a0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:18.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:17.998+0000 7f1795e00640 1 -- 172.21.15.5:0/3655206490 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17901a1cd0 con 0x7f17901058d0 2024-09-10T09:38:18.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:17.998+0000 7f1794a00640 1 --2- 172.21.15.5:0/3655206490 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1790104f10 0x7f179019a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:38:18.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:17.999+0000 7f1795e00640 1 --2- 172.21.15.5:0/3655206490 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f17901058d0 0x7f17901a1660 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f1788009fd0 tx=0x7f178800b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:38:18.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:17.999+0000 7f178e000640 1 -- 172.21.15.5:0/3655206490 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1788017070 con 0x7f17901058d0 2024-09-10T09:38:18.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:17.999+0000 7f178e000640 1 -- 172.21.15.5:0/3655206490 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1788003e50 con 0x7f17901058d0 2024-09-10T09:38:18.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:17.999+0000 7f1797e00640 1 -- 172.21.15.5:0/3655206490 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17901a1fc0 con 0x7f17901058d0 2024-09-10T09:38:18.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:17.999+0000 7f178e000640 1 -- 172.21.15.5:0/3655206490 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17880106c0 con 0x7f17901058d0 2024-09-10T09:38:18.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:17.999+0000 7f1797e00640 1 -- 172.21.15.5:0/3655206490 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17901a2430 con 0x7f17901058d0 2024-09-10T09:38:18.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:18.000+0000 7f1797e00640 1 -- 172.21.15.5:0/3655206490 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f179010b150 con 0x7f17901058d0 2024-09-10T09:38:18.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:18.002+0000 7f178e000640 1 -- 172.21.15.5:0/3655206490 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 19) ==== 137883+0+0 (secure 0 0 0) 0x7f1788003970 con 0x7f17901058d0 2024-09-10T09:38:18.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:18.003+0000 7f178e000640 1 --2- 172.21.15.5:0/3655206490 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f17780a2c20 0x7f17780a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:18.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:18.003+0000 7f178e000640 1 -- 172.21.15.5:0/3655206490 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f17880cf3c0 con 0x7f17901058d0 2024-09-10T09:38:18.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:18.004+0000 7f1795400640 1 --2- 172.21.15.5:0/3655206490 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f17780a2c20 0x7f17780a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:18.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:18.004+0000 7f1795400640 1 --2- 172.21.15.5:0/3655206490 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f17780a2c20 0x7f17780a50e0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f1790104d70 tx=0x7f1784005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:38:18.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:18.007+0000 7f178e000640 1 -- 172.21.15.5:0/3655206490 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f178809bd70 con 0x7f17901058d0 2024-09-10T09:38:18.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:38:17 smithi159 ceph-mon[35513]: from='client.? 172.21.15.5:0/1230699103' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-10T09:38:18.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:38:17 smithi159 ceph-mon[35513]: from='client.? 172.21.15.5:0/1217120734' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-10T09:38:18.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:38:17 smithi159 ceph-mon[35513]: from='client.? 172.21.15.5:0/2518545547' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-10T09:38:18.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:38:17 smithi102 ceph-mon[35442]: from='client.? 172.21.15.5:0/1230699103' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-10T09:38:18.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:38:17 smithi102 ceph-mon[35442]: from='client.? 172.21.15.5:0/1217120734' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-10T09:38:18.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:38:17 smithi102 ceph-mon[35442]: from='client.? 172.21.15.5:0/2518545547' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-10T09:38:18.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:18.192+0000 7f1797e00640 1 -- 172.21.15.5:0/3655206490 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f179002d970 con 0x7f17901058d0 2024-09-10T09:38:18.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:18.193+0000 7f178e000640 1 -- 172.21.15.5:0/3655206490 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v96) ==== 103+0+11 (secure 0 0 0) 0x7f178809b670 con 0x7f17901058d0 2024-09-10T09:38:18.195 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 32 2024-09-10T09:38:18.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:18.195+0000 7f1797e00640 1 -- 172.21.15.5:0/3655206490 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f17780a2c20 msgr2=0x7f17780a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:38:18.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:18.195+0000 7f1797e00640 1 --2- 172.21.15.5:0/3655206490 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f17780a2c20 0x7f17780a50e0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f1790104d70 tx=0x7f1784005f90 comp rx=0 tx=0).stop 2024-09-10T09:38:18.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:18.195+0000 7f1797e00640 1 -- 172.21.15.5:0/3655206490 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f17901058d0 msgr2=0x7f17901a1660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:38:18.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:18.195+0000 7f1797e00640 1 --2- 172.21.15.5:0/3655206490 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f17901058d0 0x7f17901a1660 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f1788009fd0 tx=0x7f178800b260 comp rx=0 tx=0).stop 2024-09-10T09:38:18.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:18.195+0000 7f1797e00640 1 -- 172.21.15.5:0/3655206490 shutdown_connections 2024-09-10T09:38:18.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:18.195+0000 7f1797e00640 1 --2- 172.21.15.5:0/3655206490 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f17780a2c20 0x7f17780a50e0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:18.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:18.195+0000 7f1797e00640 1 --2- 172.21.15.5:0/3655206490 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f17901058d0 0x7f17901a1660 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:18.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:18.195+0000 7f1797e00640 1 --2- 172.21.15.5:0/3655206490 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1790104f10 0x7f179019a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:18.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:18.195+0000 7f1797e00640 1 --2- 172.21.15.5:0/3655206490 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1790103d20 0x7f179019a0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:18.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:18.195+0000 7f1797e00640 1 -- 172.21.15.5:0/3655206490 >> 172.21.15.5:0/3655206490 conn(0x7f17900ff4a0 msgr2=0x7f1790108320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:38:18.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:18.196+0000 7f1797e00640 1 -- 172.21.15.5:0/3655206490 shutdown_connections 2024-09-10T09:38:18.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:18.196+0000 7f1797e00640 1 -- 172.21.15.5:0/3655206490 wait complete. 2024-09-10T09:38:18.213 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:38:17 smithi005 ceph-mon[30350]: from='client.? 172.21.15.5:0/1230699103' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-10T09:38:18.213 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:38:17 smithi005 ceph-mon[30350]: from='client.? 172.21.15.5:0/1217120734' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-10T09:38:18.213 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:38:17 smithi005 ceph-mon[30350]: from='client.? 172.21.15.5:0/2518545547' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-10T09:38:18.215 INFO:tasks.fwd_scrub:Wait for all MDSs to reach steady state... 2024-09-10T09:38:18.215 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T09:38:18.594 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:38:19.049 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:38:18 smithi005 ceph-mon[30350]: pgmap v392: 129 pgs: 129 active+clean; 646 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-10T09:38:19.049 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:38:18 smithi005 ceph-mon[30350]: mgrmap e19: x(active, since 10m), standbys: z, y 2024-09-10T09:38:19.050 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:38:18 smithi005 ceph-mon[30350]: from='client.? 172.21.15.5:0/3655206490' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-09-10T09:38:19.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:38:18 smithi102 ceph-mon[35442]: pgmap v392: 129 pgs: 129 active+clean; 646 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-10T09:38:19.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:38:18 smithi102 ceph-mon[35442]: mgrmap e19: x(active, since 10m), standbys: z, y 2024-09-10T09:38:19.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:38:18 smithi102 ceph-mon[35442]: from='client.? 172.21.15.5:0/3655206490' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-09-10T09:38:19.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:19.161+0000 7f5403800640 1 -- 172.21.15.5:0/4086614760 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f53fc074620 msgr2=0x7f53fc074a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:38:19.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:19.161+0000 7f5403800640 1 --2- 172.21.15.5:0/4086614760 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f53fc074620 0x7f53fc074a00 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f53f0009900 tx=0x7f53f002a490 comp rx=0 tx=0).stop 2024-09-10T09:38:19.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:19.162+0000 7f5403800640 1 -- 172.21.15.5:0/4086614760 shutdown_connections 2024-09-10T09:38:19.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:19.162+0000 7f5403800640 1 --2- 172.21.15.5:0/4086614760 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f53fc10e3b0 0x7f53fc110800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:19.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:19.162+0000 7f5403800640 1 --2- 172.21.15.5:0/4086614760 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f53fc074f40 0x7f53fc10de70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:19.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:19.162+0000 7f5403800640 1 --2- 172.21.15.5:0/4086614760 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f53fc074620 0x7f53fc074a00 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:19.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:19.162+0000 7f5403800640 1 -- 172.21.15.5:0/4086614760 >> 172.21.15.5:0/4086614760 conn(0x7f53fc0fd8b0 msgr2=0x7f53fc0ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:38:19.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:19.162+0000 7f5403800640 1 -- 172.21.15.5:0/4086614760 shutdown_connections 2024-09-10T09:38:19.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:19.162+0000 7f5403800640 1 -- 172.21.15.5:0/4086614760 wait complete. 2024-09-10T09:38:19.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:19.163+0000 7f5403800640 1 Processor -- start 2024-09-10T09:38:19.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:19.163+0000 7f5403800640 1 -- start start 2024-09-10T09:38:19.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:19.163+0000 7f5403800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f53fc074620 0x7f53fc19a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:19.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:19.164+0000 7f5403800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f53fc074f40 0x7f53fc19abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:19.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:19.164+0000 7f5403800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f53fc10e3b0 0x7f53fc19ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:19.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:19.164+0000 7f5403800640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f53fc114140 con 0x7f53fc10e3b0 2024-09-10T09:38:19.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:19.164+0000 7f5403800640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f53fc113fc0 con 0x7f53fc074620 2024-09-10T09:38:19.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:19.164+0000 7f5403800640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f53fc1142c0 con 0x7f53fc074f40 2024-09-10T09:38:19.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:19.164+0000 7f5401800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f53fc10e3b0 0x7f53fc19ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:19.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:19.164+0000 7f5400e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f53fc074620 0x7f53fc19a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:19.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:19.164+0000 7f5400e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f53fc074620 0x7f53fc19a680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:44758/0 (socket says 172.21.15.5:44758) 2024-09-10T09:38:19.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:19.164+0000 7f5400e00640 1 -- 172.21.15.5:0/3589179655 learned_addr learned my addr 172.21.15.5:0/3589179655 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:38:19.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:19.164+0000 7f5401800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f53fc10e3b0 0x7f53fc19ef50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60802/0 (socket says 172.21.15.5:60802) 2024-09-10T09:38:19.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:19.164+0000 7f53fbe00640 1 --2- 172.21.15.5:0/3589179655 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f53fc074f40 0x7f53fc19abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:19.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:19.164+0000 7f5400e00640 1 -- 172.21.15.5:0/3589179655 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f53fc074f40 msgr2=0x7f53fc19abc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:38:19.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:19.164+0000 7f5400e00640 1 --2- 172.21.15.5:0/3589179655 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f53fc074f40 0x7f53fc19abc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:19.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:19.164+0000 7f5400e00640 1 -- 172.21.15.5:0/3589179655 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f53fc10e3b0 msgr2=0x7f53fc19ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:38:19.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:19.164+0000 7f5400e00640 1 --2- 172.21.15.5:0/3589179655 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f53fc10e3b0 0x7f53fc19ef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:19.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:19.164+0000 7f5400e00640 1 -- 172.21.15.5:0/3589179655 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53fc19f630 con 0x7f53fc074620 2024-09-10T09:38:19.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:19.164+0000 7f5401800640 1 --2- 172.21.15.5:0/3589179655 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f53fc10e3b0 0x7f53fc19ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:38:19.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:19.165+0000 7f5400e00640 1 --2- 172.21.15.5:0/3589179655 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f53fc074620 0x7f53fc19a680 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f53f0006010 tx=0x7f53f00042f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:38:19.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:19.165+0000 7f53f9600640 1 -- 172.21.15.5:0/3589179655 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53f003f030 con 0x7f53fc074620 2024-09-10T09:38:19.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:19.165+0000 7f53f9600640 1 -- 172.21.15.5:0/3589179655 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f53f0003ce0 con 0x7f53fc074620 2024-09-10T09:38:19.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:19.165+0000 7f53f9600640 1 -- 172.21.15.5:0/3589179655 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53f0035070 con 0x7f53fc074620 2024-09-10T09:38:19.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:19.165+0000 7f5403800640 1 -- 172.21.15.5:0/3589179655 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53fc19f8c0 con 0x7f53fc074620 2024-09-10T09:38:19.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:19.165+0000 7f5403800640 1 -- 172.21.15.5:0/3589179655 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53fc1a7200 con 0x7f53fc074620 2024-09-10T09:38:19.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:19.166+0000 7f5403800640 1 -- 172.21.15.5:0/3589179655 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f53fc074a00 con 0x7f53fc074620 2024-09-10T09:38:19.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:19.168+0000 7f53f9600640 1 -- 172.21.15.5:0/3589179655 <== mon.2 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 19) ==== 137883+0+0 (secure 0 0 0) 0x7f53f0042030 con 0x7f53fc074620 2024-09-10T09:38:19.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:19.169+0000 7f53f9600640 1 --2- 172.21.15.5:0/3589179655 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f53e40a2c90 0x7f53e40a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:19.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:19.170+0000 7f53f9600640 1 -- 172.21.15.5:0/3589179655 <== mon.2 v2:172.21.15.102:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f53f00efe70 con 0x7f53fc074620 2024-09-10T09:38:19.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:19.170+0000 7f53fbe00640 1 --2- 172.21.15.5:0/3589179655 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f53e40a2c90 0x7f53e40a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:19.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:19.170+0000 7f53fbe00640 1 --2- 172.21.15.5:0/3589179655 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f53e40a2c90 0x7f53e40a5150 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f53fc10f7a0 tx=0x7f53e800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:38:19.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:19.172+0000 7f53f9600640 1 -- 172.21.15.5:0/3589179655 <== mon.2 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f53f00bc290 con 0x7f53fc074620 2024-09-10T09:38:19.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:38:18 smithi159 ceph-mon[35513]: pgmap v392: 129 pgs: 129 active+clean; 646 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-10T09:38:19.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:38:18 smithi159 ceph-mon[35513]: mgrmap e19: x(active, since 10m), standbys: z, y 2024-09-10T09:38:19.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:38:18 smithi159 ceph-mon[35513]: from='client.? 172.21.15.5:0/3655206490' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-09-10T09:38:19.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:19.395+0000 7f5403800640 1 -- 172.21.15.5:0/3589179655 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f53fc19b9e0 con 0x7f53fc074620 2024-09-10T09:38:19.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:19.396+0000 7f53f9600640 1 -- 172.21.15.5:0/3589179655 <== mon.2 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11969 (secure 0 0 0) 0x7f53fc19b9e0 con 0x7f53fc074620 2024-09-10T09:38:19.398 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-10T09:38:19.398 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":30,"btime":"2024-09-10T09:38:15:834198+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14454,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/2970594311","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2970594311},{"type":"v1","addr":"172.21.15.5:6835","nonce":2970594311}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14514,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6839/3616502174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":3616502174},{"type":"v1","addr":"172.21.15.5:6839","nonce":3616502174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T09:35:29.626287+0000","modified":"2024-09-10T09:38:15.658058+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24542,"mds_1":24535,"mds_2":24511,"mds_3":24506,"mds_4":24443},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14520":{"gid":14520,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/2073599842","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":2073599842},{"type":"v1","addr":"172.21.15.5:6841","nonce":2073599842}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"k","rank":4,"incarnation":26,"state":"up:active","state_seq":24,"addr":"172.21.15.102:6833/2457007427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":2457007427},{"type":"v1","addr":"172.21.15.102:6833","nonce":2457007427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/356445159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":356445159},{"type":"v1","addr":"172.21.15.159:6833","nonce":356445159}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6837/2894314507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":2894314507},{"type":"v1","addr":"172.21.15.5:6837","nonce":2894314507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.102:6835/1647549111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":1647549111},{"type":"v1","addr":"172.21.15.102:6835","nonce":1647549111}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6835/952657242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":952657242},{"type":"v1","addr":"172.21.15.159:6835","nonce":952657242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":19,"addr":"172.21.15.102:6837/2689776764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":2689776764},{"type":"v1","addr":"172.21.15.102:6837","nonce":2689776764}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.159:6837/620627639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":620627639},{"type":"v1","addr":"172.21.15.159:6837","nonce":620627639}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24535":{"gid":24535,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":18,"addr":"172.21.15.159:6839/2073739599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":2073739599},{"type":"v1","addr":"172.21.15.159:6839","nonce":2073739599}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24542":{"gid":24542,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.102:6839/335409749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6838","nonce":335409749},{"type":"v1","addr":"172.21.15.102:6839","nonce":335409749}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24542,"qdb_cluster":[24542,24535,24511,24506,24443]},"id":1}]} 2024-09-10T09:38:19.398 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 30 2024-09-10T09:38:19.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:19.399+0000 7f5403800640 1 -- 172.21.15.5:0/3589179655 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f53e40a2c90 msgr2=0x7f53e40a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:38:19.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:19.399+0000 7f5403800640 1 --2- 172.21.15.5:0/3589179655 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f53e40a2c90 0x7f53e40a5150 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f53fc10f7a0 tx=0x7f53e800b390 comp rx=0 tx=0).stop 2024-09-10T09:38:19.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:19.399+0000 7f5403800640 1 -- 172.21.15.5:0/3589179655 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f53fc074620 msgr2=0x7f53fc19a680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:38:19.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:19.399+0000 7f5403800640 1 --2- 172.21.15.5:0/3589179655 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f53fc074620 0x7f53fc19a680 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f53f0006010 tx=0x7f53f00042f0 comp rx=0 tx=0).stop 2024-09-10T09:38:19.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:19.399+0000 7f5403800640 1 -- 172.21.15.5:0/3589179655 shutdown_connections 2024-09-10T09:38:19.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:19.399+0000 7f5403800640 1 --2- 172.21.15.5:0/3589179655 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f53e40a2c90 0x7f53e40a5150 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:19.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:19.399+0000 7f5403800640 1 --2- 172.21.15.5:0/3589179655 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f53fc10e3b0 0x7f53fc19ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:19.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:19.399+0000 7f5403800640 1 --2- 172.21.15.5:0/3589179655 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f53fc074f40 0x7f53fc19abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:19.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:19.399+0000 7f5403800640 1 --2- 172.21.15.5:0/3589179655 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f53fc074620 0x7f53fc19a680 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:19.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:19.399+0000 7f5403800640 1 -- 172.21.15.5:0/3589179655 >> 172.21.15.5:0/3589179655 conn(0x7f53fc0fd8b0 msgr2=0x7f53fc0fe5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:38:19.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:19.399+0000 7f5403800640 1 -- 172.21.15.5:0/3589179655 shutdown_connections 2024-09-10T09:38:19.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:19.399+0000 7f5403800640 1 -- 172.21.15.5:0/3589179655 wait complete. 2024-09-10T09:38:20.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:38:19 smithi005 ceph-mon[30350]: from='client.? 172.21.15.5:0/3589179655' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:38:20.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:38:19 smithi102 ceph-mon[35442]: from='client.? 172.21.15.5:0/3589179655' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:38:20.280 INFO:tasks.fwd_scrub:Ready to start scrub thrashing 2024-09-10T09:38:20.281 INFO:tasks.fwd_scrub.ceph_manager:waiting for clean 2024-09-10T09:38:20.281 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T09:38:20.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:38:19 smithi159 ceph-mon[35513]: from='client.? 172.21.15.5:0/3589179655' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:38:20.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:20.438+0000 7fc8b8800640 1 -- 172.21.15.5:0/3724107129 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc8b01058e0 msgr2=0x7fc8b010c170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:38:20.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:20.438+0000 7fc8b8800640 1 --2- 172.21.15.5:0/3724107129 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc8b01058e0 0x7fc8b010c170 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fc8ac009ca0 tx=0x7fc8ac02a8c0 comp rx=0 tx=0).stop 2024-09-10T09:38:20.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:20.438+0000 7fc8b8800640 1 -- 172.21.15.5:0/3724107129 shutdown_connections 2024-09-10T09:38:20.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:20.438+0000 7fc8b8800640 1 --2- 172.21.15.5:0/3724107129 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc8b01058e0 0x7fc8b010c170 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:20.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:20.438+0000 7fc8b8800640 1 --2- 172.21.15.5:0/3724107129 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc8b0104f20 0x7fc8b01053a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:20.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:20.438+0000 7fc8b8800640 1 --2- 172.21.15.5:0/3724107129 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc8b0103d30 0x7fc8b0104130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:20.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:20.438+0000 7fc8b8800640 1 -- 172.21.15.5:0/3724107129 >> 172.21.15.5:0/3724107129 conn(0x7fc8b00ff4b0 msgr2=0x7fc8b01018d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:38:20.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:20.438+0000 7fc8b8800640 1 -- 172.21.15.5:0/3724107129 shutdown_connections 2024-09-10T09:38:20.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:20.438+0000 7fc8b8800640 1 -- 172.21.15.5:0/3724107129 wait complete. 2024-09-10T09:38:20.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:20.439+0000 7fc8b8800640 1 Processor -- start 2024-09-10T09:38:20.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:20.439+0000 7fc8b8800640 1 -- start start 2024-09-10T09:38:20.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:20.440+0000 7fc8b8800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc8b0103d30 0x7fc8b0199fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:20.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:20.440+0000 7fc8b8800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc8b0104f20 0x7fc8b019a510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:20.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:20.440+0000 7fc8b8800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc8b01058e0 0x7fc8b01a1580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:20.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:20.440+0000 7fc8b8800640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc8b010f9b0 con 0x7fc8b0104f20 2024-09-10T09:38:20.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:20.440+0000 7fc8b8800640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fc8b010f830 con 0x7fc8b01058e0 2024-09-10T09:38:20.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:20.440+0000 7fc8b8800640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fc8b010fb30 con 0x7fc8b0103d30 2024-09-10T09:38:20.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:20.440+0000 7fc8b5400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc8b0104f20 0x7fc8b019a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:20.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:20.440+0000 7fc8b5400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc8b0104f20 0x7fc8b019a510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60808/0 (socket says 172.21.15.5:60808) 2024-09-10T09:38:20.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:20.440+0000 7fc8b5400640 1 -- 172.21.15.5:0/159867018 learned_addr learned my addr 172.21.15.5:0/159867018 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:38:20.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:20.440+0000 7fc8b5e00640 1 --2- 172.21.15.5:0/159867018 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc8b0103d30 0x7fc8b0199fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:20.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:20.440+0000 7fc8b6800640 1 --2- 172.21.15.5:0/159867018 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc8b01058e0 0x7fc8b01a1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:20.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:20.440+0000 7fc8b5400640 1 -- 172.21.15.5:0/159867018 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc8b0103d30 msgr2=0x7fc8b0199fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:38:20.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:20.440+0000 7fc8b5400640 1 --2- 172.21.15.5:0/159867018 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc8b0103d30 0x7fc8b0199fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:20.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:20.440+0000 7fc8b5400640 1 -- 172.21.15.5:0/159867018 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc8b01058e0 msgr2=0x7fc8b01a1580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:38:20.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:20.440+0000 7fc8b5400640 1 --2- 172.21.15.5:0/159867018 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc8b01058e0 0x7fc8b01a1580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:20.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:20.440+0000 7fc8b5400640 1 -- 172.21.15.5:0/159867018 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8b01a1bf0 con 0x7fc8b0104f20 2024-09-10T09:38:20.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:20.441+0000 7fc8b5400640 1 --2- 172.21.15.5:0/159867018 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc8b0104f20 0x7fc8b019a510 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fc8a000e9f0 tx=0x7fc8a000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:38:20.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:20.441+0000 7fc8aaa00640 1 -- 172.21.15.5:0/159867018 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8a000cd80 con 0x7fc8b0104f20 2024-09-10T09:38:20.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:20.441+0000 7fc8aaa00640 1 -- 172.21.15.5:0/159867018 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc8a0010910 con 0x7fc8b0104f20 2024-09-10T09:38:20.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:20.441+0000 7fc8b8800640 1 -- 172.21.15.5:0/159867018 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8b01a1ee0 con 0x7fc8b0104f20 2024-09-10T09:38:20.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:20.441+0000 7fc8aaa00640 1 -- 172.21.15.5:0/159867018 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8a0018990 con 0x7fc8b0104f20 2024-09-10T09:38:20.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:20.441+0000 7fc8b8800640 1 -- 172.21.15.5:0/159867018 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8b01a23f0 con 0x7fc8b0104f20 2024-09-10T09:38:20.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:20.442+0000 7fc8b8800640 1 -- 172.21.15.5:0/159867018 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc8b010b050 con 0x7fc8b0104f20 2024-09-10T09:38:20.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:20.445+0000 7fc8aaa00640 1 -- 172.21.15.5:0/159867018 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 19) ==== 137883+0+0 (secure 0 0 0) 0x7fc8a0010ac0 con 0x7fc8b0104f20 2024-09-10T09:38:20.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:20.446+0000 7fc8aaa00640 1 --2- 172.21.15.5:0/159867018 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fc8980a2ca0 0x7fc8980a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:20.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:20.446+0000 7fc8aaa00640 1 -- 172.21.15.5:0/159867018 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fc8a00cf8a0 con 0x7fc8b0104f20 2024-09-10T09:38:20.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:20.446+0000 7fc8b5e00640 1 --2- 172.21.15.5:0/159867018 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fc8980a2ca0 0x7fc8980a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:20.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:20.447+0000 7fc8b5e00640 1 --2- 172.21.15.5:0/159867018 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fc8980a2ca0 0x7fc8980a5160 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fc8b0104d80 tx=0x7fc8a4005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:38:20.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:20.450+0000 7fc8aaa00640 1 -- 172.21.15.5:0/159867018 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc8a009bc00 con 0x7fc8b0104f20 2024-09-10T09:38:20.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:20.634+0000 7fc8b8800640 1 -- 172.21.15.5:0/159867018 --> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fc8b0062630 con 0x7fc8980a2ca0 2024-09-10T09:38:20.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:20.644+0000 7fc8aaa00640 1 -- 172.21.15.5:0/159867018 <== mgr.14150 v2:172.21.15.5:6800/2707348081 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340822 (secure 0 0 0) 0x7fc8b0062630 con 0x7fc8980a2ca0 2024-09-10T09:38:20.651 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-10T09:38:20.653 INFO:teuthology.orchestra.run.smithi005.stderr:dumped all 2024-09-10T09:38:20.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:20.649+0000 7fc8b8800640 1 -- 172.21.15.5:0/159867018 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fc8980a2ca0 msgr2=0x7fc8980a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:38:20.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:20.649+0000 7fc8b8800640 1 --2- 172.21.15.5:0/159867018 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fc8980a2ca0 0x7fc8980a5160 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fc8b0104d80 tx=0x7fc8a4005f90 comp rx=0 tx=0).stop 2024-09-10T09:38:20.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:20.649+0000 7fc8b8800640 1 -- 172.21.15.5:0/159867018 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc8b0104f20 msgr2=0x7fc8b019a510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:38:20.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:20.649+0000 7fc8b8800640 1 --2- 172.21.15.5:0/159867018 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc8b0104f20 0x7fc8b019a510 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fc8a000e9f0 tx=0x7fc8a000eeb0 comp rx=0 tx=0).stop 2024-09-10T09:38:20.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:20.649+0000 7fc8b8800640 1 -- 172.21.15.5:0/159867018 shutdown_connections 2024-09-10T09:38:20.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:20.649+0000 7fc8b8800640 1 --2- 172.21.15.5:0/159867018 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fc8980a2ca0 0x7fc8980a5160 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:20.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:20.649+0000 7fc8b8800640 1 --2- 172.21.15.5:0/159867018 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc8b01058e0 0x7fc8b01a1580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:20.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:20.649+0000 7fc8b8800640 1 --2- 172.21.15.5:0/159867018 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc8b0104f20 0x7fc8b019a510 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:20.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:20.649+0000 7fc8b8800640 1 --2- 172.21.15.5:0/159867018 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc8b0103d30 0x7fc8b0199fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:20.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:20.649+0000 7fc8b8800640 1 -- 172.21.15.5:0/159867018 >> 172.21.15.5:0/159867018 conn(0x7fc8b00ff4b0 msgr2=0x7fc8b0108220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:38:20.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:20.649+0000 7fc8b8800640 1 -- 172.21.15.5:0/159867018 shutdown_connections 2024-09-10T09:38:20.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:20.649+0000 7fc8b8800640 1 -- 172.21.15.5:0/159867018 wait complete. 2024-09-10T09:38:20.667 INFO:teuthology.orchestra.run.smithi005.stdout:{"pg_ready":true,"pg_map":{"version":393,"stamp":"2024-09-10T09:38:19.570873+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":661272,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1370,"num_read_kb":812,"num_write":443,"num_write_kb":3090,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":1181751,"num_keys_recovered":7,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":862,"ondisk_log_size":862,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":378972,"kb_used_data":13084,"kb_used_omap":18,"kb_used_meta":365677,"kb_avail":1124562852,"statfs":{"total":1151940427776,"available":1151552360448,"internally_reserved":0,"allocated":13398016,"data_stored":8614564,"data_compressed":498435,"data_compressed_allocated":675840,"data_compressed_original":2850816,"omap_allocated":19141,"internal_metadata":374453563},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":110,"num_read_kb":55,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003036"},"pg_stats":[{"pgid":"2.1f","version":"96'14","reported_seq":179,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:38:13.661865+0000","last_change":"2024-09-10T09:35:25.533085+0000","last_active":"2024-09-10T09:38:13.661865+0000","last_peered":"2024-09-10T09:38:13.661865+0000","last_clean":"2024-09-10T09:38:13.661865+0000","last_became_active":"2024-09-10T09:35:25.532729+0000","last_became_peered":"2024-09-10T09:35:25.532729+0000","last_unstale":"2024-09-10T09:38:13.661865+0000","last_undegraded":"2024-09-10T09:38:13.661865+0000","last_fullsized":"2024-09-10T09:38:13.661865+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:55:11.158279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":121,"num_read_kb":121,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"96'5","reported_seq":21,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.545575+0000","last_change":"2024-09-10T09:36:06.767356+0000","last_active":"2024-09-10T09:37:35.545575+0000","last_peered":"2024-09-10T09:37:35.545575+0000","last_clean":"2024-09-10T09:37:35.545575+0000","last_became_active":"2024-09-10T09:36:06.766931+0000","last_became_peered":"2024-09-10T09:36:06.766931+0000","last_unstale":"2024-09-10T09:37:35.545575+0000","last_undegraded":"2024-09-10T09:37:35.545575+0000","last_fullsized":"2024-09-10T09:37:35.545575+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:25:02.777327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,7],"acting":[9,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.527523+0000","last_change":"2024-09-10T09:35:36.629797+0000","last_active":"2024-09-10T09:37:07.527523+0000","last_peered":"2024-09-10T09:37:07.527523+0000","last_clean":"2024-09-10T09:37:07.527523+0000","last_became_active":"2024-09-10T09:35:36.629414+0000","last_became_peered":"2024-09-10T09:35:36.629414+0000","last_unstale":"2024-09-10T09:37:07.527523+0000","last_undegraded":"2024-09-10T09:37:07.527523+0000","last_fullsized":"2024-09-10T09:37:07.527523+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:42:44.610581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"94'2","reported_seq":47,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.524964+0000","last_change":"2024-09-10T09:35:25.852022+0000","last_active":"2024-09-10T09:37:07.524964+0000","last_peered":"2024-09-10T09:37:07.524964+0000","last_clean":"2024-09-10T09:37:07.524964+0000","last_became_active":"2024-09-10T09:35:25.851587+0000","last_became_peered":"2024-09-10T09:35:25.851587+0000","last_unstale":"2024-09-10T09:37:07.524964+0000","last_undegraded":"2024-09-10T09:37:07.524964+0000","last_fullsized":"2024-09-10T09:37:07.524964+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:37:26.085914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"96'6","reported_seq":43,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.556261+0000","last_change":"2024-09-10T09:35:28.579260+0000","last_active":"2024-09-10T09:37:35.556261+0000","last_peered":"2024-09-10T09:37:35.556261+0000","last_clean":"2024-09-10T09:37:35.556261+0000","last_became_active":"2024-09-10T09:35:28.574584+0000","last_became_peered":"2024-09-10T09:35:28.574584+0000","last_unstale":"2024-09-10T09:37:35.556261+0000","last_undegraded":"2024-09-10T09:37:35.556261+0000","last_fullsized":"2024-09-10T09:37:35.556261+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:31:19.277689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.568359+0000","last_change":"2024-09-10T09:36:06.874861+0000","last_active":"2024-09-10T09:37:07.568359+0000","last_peered":"2024-09-10T09:37:07.568359+0000","last_clean":"2024-09-10T09:37:07.568359+0000","last_became_active":"2024-09-10T09:36:06.873930+0000","last_became_peered":"2024-09-10T09:36:06.873930+0000","last_unstale":"2024-09-10T09:37:07.568359+0000","last_undegraded":"2024-09-10T09:37:07.568359+0000","last_fullsized":"2024-09-10T09:37:07.568359+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:35:01.354061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,8,3],"acting":[4,6,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"96'13","reported_seq":58,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:59.271922+0000","last_change":"2024-09-10T09:35:25.534571+0000","last_active":"2024-09-10T09:37:59.271922+0000","last_peered":"2024-09-10T09:37:59.271922+0000","last_clean":"2024-09-10T09:37:59.271922+0000","last_became_active":"2024-09-10T09:35:25.534235+0000","last_became_peered":"2024-09-10T09:35:25.534235+0000","last_unstale":"2024-09-10T09:37:59.271922+0000","last_undegraded":"2024-09-10T09:37:59.271922+0000","last_fullsized":"2024-09-10T09:37:59.271922+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:50:21.447542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"96'9","reported_seq":48,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.553039+0000","last_change":"2024-09-10T09:35:28.565541+0000","last_active":"2024-09-10T09:37:35.553039+0000","last_peered":"2024-09-10T09:37:35.553039+0000","last_clean":"2024-09-10T09:37:35.553039+0000","last_became_active":"2024-09-10T09:35:28.565095+0000","last_became_peered":"2024-09-10T09:35:28.565095+0000","last_unstale":"2024-09-10T09:37:35.553039+0000","last_undegraded":"2024-09-10T09:37:35.553039+0000","last_fullsized":"2024-09-10T09:37:35.553039+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:29:31.285401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.568600+0000","last_change":"2024-09-10T09:35:36.629277+0000","last_active":"2024-09-10T09:37:07.568600+0000","last_peered":"2024-09-10T09:37:07.568600+0000","last_clean":"2024-09-10T09:37:07.568600+0000","last_became_active":"2024-09-10T09:35:36.628687+0000","last_became_peered":"2024-09-10T09:35:36.628687+0000","last_unstale":"2024-09-10T09:37:07.568600+0000","last_undegraded":"2024-09-10T09:37:07.568600+0000","last_fullsized":"2024-09-10T09:37:07.568600+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:34:33.628827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"94'5","reported_seq":50,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.530161+0000","last_change":"2024-09-10T09:35:25.536137+0000","last_active":"2024-09-10T09:37:07.530161+0000","last_peered":"2024-09-10T09:37:07.530161+0000","last_clean":"2024-09-10T09:37:07.530161+0000","last_became_active":"2024-09-10T09:35:25.535728+0000","last_became_peered":"2024-09-10T09:35:25.535728+0000","last_unstale":"2024-09-10T09:37:07.530161+0000","last_undegraded":"2024-09-10T09:37:07.530161+0000","last_fullsized":"2024-09-10T09:37:07.530161+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:36:04.829901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"96'15","reported_seq":60,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.547598+0000","last_change":"2024-09-10T09:36:06.874425+0000","last_active":"2024-09-10T09:37:35.547598+0000","last_peered":"2024-09-10T09:37:35.547598+0000","last_clean":"2024-09-10T09:37:35.547598+0000","last_became_active":"2024-09-10T09:36:06.874050+0000","last_became_peered":"2024-09-10T09:36:06.874050+0000","last_unstale":"2024-09-10T09:37:35.547598+0000","last_undegraded":"2024-09-10T09:37:35.547598+0000","last_fullsized":"2024-09-10T09:37:35.547598+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:30:51.200335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,1],"acting":[5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.568073+0000","last_change":"2024-09-10T09:35:36.615314+0000","last_active":"2024-09-10T09:37:07.568073+0000","last_peered":"2024-09-10T09:37:07.568073+0000","last_clean":"2024-09-10T09:37:07.568073+0000","last_became_active":"2024-09-10T09:35:36.614820+0000","last_became_peered":"2024-09-10T09:35:36.614820+0000","last_unstale":"2024-09-10T09:37:07.568073+0000","last_undegraded":"2024-09-10T09:37:07.568073+0000","last_fullsized":"2024-09-10T09:37:07.568073+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:51:16.897935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"96'5","reported_seq":42,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.556084+0000","last_change":"2024-09-10T09:35:28.572346+0000","last_active":"2024-09-10T09:37:35.556084+0000","last_peered":"2024-09-10T09:37:35.556084+0000","last_clean":"2024-09-10T09:37:35.556084+0000","last_became_active":"2024-09-10T09:35:28.571839+0000","last_became_peered":"2024-09-10T09:35:28.571839+0000","last_unstale":"2024-09-10T09:37:35.556084+0000","last_undegraded":"2024-09-10T09:37:35.556084+0000","last_fullsized":"2024-09-10T09:37:35.556084+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:41:29.662441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"94'4","reported_seq":49,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.529918+0000","last_change":"2024-09-10T09:35:25.538747+0000","last_active":"2024-09-10T09:37:07.529918+0000","last_peered":"2024-09-10T09:37:07.529918+0000","last_clean":"2024-09-10T09:37:07.529918+0000","last_became_active":"2024-09-10T09:35:25.538330+0000","last_became_peered":"2024-09-10T09:35:25.538330+0000","last_unstale":"2024-09-10T09:37:07.529918+0000","last_undegraded":"2024-09-10T09:37:07.529918+0000","last_fullsized":"2024-09-10T09:37:07.529918+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:15:08.340688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.568436+0000","last_change":"2024-09-10T09:35:37.270106+0000","last_active":"2024-09-10T09:37:07.568436+0000","last_peered":"2024-09-10T09:37:07.568436+0000","last_clean":"2024-09-10T09:37:07.568436+0000","last_became_active":"2024-09-10T09:35:37.269575+0000","last_became_peered":"2024-09-10T09:35:37.269575+0000","last_unstale":"2024-09-10T09:37:07.568436+0000","last_undegraded":"2024-09-10T09:37:07.568436+0000","last_fullsized":"2024-09-10T09:37:07.568436+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:35:00.414602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"96'8","reported_seq":47,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.555784+0000","last_change":"2024-09-10T09:35:28.569230+0000","last_active":"2024-09-10T09:37:35.555784+0000","last_peered":"2024-09-10T09:37:35.555784+0000","last_clean":"2024-09-10T09:37:35.555784+0000","last_became_active":"2024-09-10T09:35:28.568266+0000","last_became_peered":"2024-09-10T09:35:28.568266+0000","last_unstale":"2024-09-10T09:37:35.555784+0000","last_undegraded":"2024-09-10T09:37:35.555784+0000","last_fullsized":"2024-09-10T09:37:35.555784+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:49:43.730686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"94'3","reported_seq":17,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.544707+0000","last_change":"2024-09-10T09:37:07.544298+0000","last_active":"2024-09-10T09:37:07.544707+0000","last_peered":"2024-09-10T09:37:07.544707+0000","last_clean":"2024-09-10T09:37:07.544707+0000","last_became_active":"2024-09-10T09:37:07.531815+0000","last_became_peered":"2024-09-10T09:37:07.531815+0000","last_unstale":"2024-09-10T09:37:07.544707+0000","last_undegraded":"2024-09-10T09:37:07.544707+0000","last_fullsized":"2024-09-10T09:37:07.544707+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:18:10.091895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"94'1","reported_seq":37,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.568484+0000","last_change":"2024-09-10T09:36:06.875056+0000","last_active":"2024-09-10T09:37:07.568484+0000","last_peered":"2024-09-10T09:37:07.568484+0000","last_clean":"2024-09-10T09:37:07.568484+0000","last_became_active":"2024-09-10T09:36:06.874400+0000","last_became_peered":"2024-09-10T09:36:06.874400+0000","last_unstale":"2024-09-10T09:37:07.568484+0000","last_undegraded":"2024-09-10T09:37:07.568484+0000","last_fullsized":"2024-09-10T09:37:07.568484+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:21:06.300712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,7,8],"acting":[4,10,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"96'9","reported_seq":48,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.552565+0000","last_change":"2024-09-10T09:35:28.571319+0000","last_active":"2024-09-10T09:37:35.552565+0000","last_peered":"2024-09-10T09:37:35.552565+0000","last_clean":"2024-09-10T09:37:35.552565+0000","last_became_active":"2024-09-10T09:35:28.570972+0000","last_became_peered":"2024-09-10T09:35:28.570972+0000","last_unstale":"2024-09-10T09:37:35.552565+0000","last_undegraded":"2024-09-10T09:37:35.552565+0000","last_fullsized":"2024-09-10T09:37:35.552565+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:45:19.300535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"94'7","reported_seq":52,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.524483+0000","last_change":"2024-09-10T09:35:25.530489+0000","last_active":"2024-09-10T09:37:07.524483+0000","last_peered":"2024-09-10T09:37:07.524483+0000","last_clean":"2024-09-10T09:37:07.524483+0000","last_became_active":"2024-09-10T09:35:25.530197+0000","last_became_peered":"2024-09-10T09:35:25.530197+0000","last_unstale":"2024-09-10T09:37:07.524483+0000","last_undegraded":"2024-09-10T09:37:07.524483+0000","last_fullsized":"2024-09-10T09:37:07.524483+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:28:57.539569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.527650+0000","last_change":"2024-09-10T09:35:36.630507+0000","last_active":"2024-09-10T09:37:07.527650+0000","last_peered":"2024-09-10T09:37:07.527650+0000","last_clean":"2024-09-10T09:37:07.527650+0000","last_became_active":"2024-09-10T09:35:36.630052+0000","last_became_peered":"2024-09-10T09:35:36.630052+0000","last_unstale":"2024-09-10T09:37:07.527650+0000","last_undegraded":"2024-09-10T09:37:07.527650+0000","last_fullsized":"2024-09-10T09:37:07.527650+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:13:42.691382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"96'2","reported_seq":41,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.532487+0000","last_change":"2024-09-10T09:35:28.565650+0000","last_active":"2024-09-10T09:37:35.532487+0000","last_peered":"2024-09-10T09:37:35.532487+0000","last_clean":"2024-09-10T09:37:35.532487+0000","last_became_active":"2024-09-10T09:35:28.565303+0000","last_became_peered":"2024-09-10T09:35:28.565303+0000","last_unstale":"2024-09-10T09:37:35.532487+0000","last_undegraded":"2024-09-10T09:37:35.532487+0000","last_fullsized":"2024-09-10T09:37:35.532487+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:48:08.965487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"94'2","reported_seq":46,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.574507+0000","last_change":"2024-09-10T09:35:25.851326+0000","last_active":"2024-09-10T09:37:07.574507+0000","last_peered":"2024-09-10T09:37:07.574507+0000","last_clean":"2024-09-10T09:37:07.574507+0000","last_became_active":"2024-09-10T09:35:25.850981+0000","last_became_peered":"2024-09-10T09:35:25.850981+0000","last_unstale":"2024-09-10T09:37:07.574507+0000","last_undegraded":"2024-09-10T09:37:07.574507+0000","last_fullsized":"2024-09-10T09:37:07.574507+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:50:28.877840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.568510+0000","last_change":"2024-09-10T09:35:37.270138+0000","last_active":"2024-09-10T09:37:07.568510+0000","last_peered":"2024-09-10T09:37:07.568510+0000","last_clean":"2024-09-10T09:37:07.568510+0000","last_became_active":"2024-09-10T09:35:37.269663+0000","last_became_peered":"2024-09-10T09:35:37.269663+0000","last_unstale":"2024-09-10T09:37:07.568510+0000","last_undegraded":"2024-09-10T09:37:07.568510+0000","last_fullsized":"2024-09-10T09:37:07.568510+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:18:00.105970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"96'7","reported_seq":44,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.554144+0000","last_change":"2024-09-10T09:35:28.576197+0000","last_active":"2024-09-10T09:37:35.554144+0000","last_peered":"2024-09-10T09:37:35.554144+0000","last_clean":"2024-09-10T09:37:35.554144+0000","last_became_active":"2024-09-10T09:35:28.575859+0000","last_became_peered":"2024-09-10T09:35:28.575859+0000","last_unstale":"2024-09-10T09:37:35.554144+0000","last_undegraded":"2024-09-10T09:37:35.554144+0000","last_fullsized":"2024-09-10T09:37:35.554144+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:09:39.458761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"94'7","reported_seq":51,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.568805+0000","last_change":"2024-09-10T09:35:25.990587+0000","last_active":"2024-09-10T09:37:07.568805+0000","last_peered":"2024-09-10T09:37:07.568805+0000","last_clean":"2024-09-10T09:37:07.568805+0000","last_became_active":"2024-09-10T09:35:25.990302+0000","last_became_peered":"2024-09-10T09:35:25.990302+0000","last_unstale":"2024-09-10T09:37:07.568805+0000","last_undegraded":"2024-09-10T09:37:07.568805+0000","last_fullsized":"2024-09-10T09:37:07.568805+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:18:41.244441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.527768+0000","last_change":"2024-09-10T09:35:36.612983+0000","last_active":"2024-09-10T09:37:07.527768+0000","last_peered":"2024-09-10T09:37:07.527768+0000","last_clean":"2024-09-10T09:37:07.527768+0000","last_became_active":"2024-09-10T09:35:36.612542+0000","last_became_peered":"2024-09-10T09:35:36.612542+0000","last_unstale":"2024-09-10T09:37:07.527768+0000","last_undegraded":"2024-09-10T09:37:07.527768+0000","last_fullsized":"2024-09-10T09:37:07.527768+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:50:16.298195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"96'5","reported_seq":44,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.554267+0000","last_change":"2024-09-10T09:35:28.564121+0000","last_active":"2024-09-10T09:37:35.554267+0000","last_peered":"2024-09-10T09:37:35.554267+0000","last_clean":"2024-09-10T09:37:35.554267+0000","last_became_active":"2024-09-10T09:35:28.562780+0000","last_became_peered":"2024-09-10T09:35:28.562780+0000","last_unstale":"2024-09-10T09:37:35.554267+0000","last_undegraded":"2024-09-10T09:37:35.554267+0000","last_fullsized":"2024-09-10T09:37:35.554267+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:53:15.690478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"96'16","reported_seq":76,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:34.702676+0000","last_change":"2024-09-10T09:37:07.541031+0000","last_active":"2024-09-10T09:37:34.702676+0000","last_peered":"2024-09-10T09:37:34.702676+0000","last_clean":"2024-09-10T09:37:34.702676+0000","last_became_active":"2024-09-10T09:37:07.529577+0000","last_became_peered":"2024-09-10T09:37:07.529577+0000","last_unstale":"2024-09-10T09:37:34.702676+0000","last_undegraded":"2024-09-10T09:37:34.702676+0000","last_fullsized":"2024-09-10T09:37:34.702676+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:14:13.440346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":29,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":7,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.524145+0000","last_change":"2024-09-10T09:35:37.270888+0000","last_active":"2024-09-10T09:37:07.524145+0000","last_peered":"2024-09-10T09:37:07.524145+0000","last_clean":"2024-09-10T09:37:07.524145+0000","last_became_active":"2024-09-10T09:35:37.270229+0000","last_became_peered":"2024-09-10T09:35:37.270229+0000","last_unstale":"2024-09-10T09:37:07.524145+0000","last_undegraded":"2024-09-10T09:37:07.524145+0000","last_fullsized":"2024-09-10T09:37:07.524145+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:03:54.223463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"96'9","reported_seq":48,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.547904+0000","last_change":"2024-09-10T09:35:28.572799+0000","last_active":"2024-09-10T09:37:35.547904+0000","last_peered":"2024-09-10T09:37:35.547904+0000","last_clean":"2024-09-10T09:37:35.547904+0000","last_became_active":"2024-09-10T09:35:28.572310+0000","last_became_peered":"2024-09-10T09:35:28.572310+0000","last_unstale":"2024-09-10T09:37:35.547904+0000","last_undegraded":"2024-09-10T09:37:35.547904+0000","last_fullsized":"2024-09-10T09:37:35.547904+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:33:51.446766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"94'4","reported_seq":49,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.530456+0000","last_change":"2024-09-10T09:35:25.851868+0000","last_active":"2024-09-10T09:37:07.530456+0000","last_peered":"2024-09-10T09:37:07.530456+0000","last_clean":"2024-09-10T09:37:07.530456+0000","last_became_active":"2024-09-10T09:35:25.851447+0000","last_became_peered":"2024-09-10T09:35:25.851447+0000","last_unstale":"2024-09-10T09:37:07.530456+0000","last_undegraded":"2024-09-10T09:37:07.530456+0000","last_fullsized":"2024-09-10T09:37:07.530456+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:15:57.706690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.526545+0000","last_change":"2024-09-10T09:35:36.619459+0000","last_active":"2024-09-10T09:37:07.526545+0000","last_peered":"2024-09-10T09:37:07.526545+0000","last_clean":"2024-09-10T09:37:07.526545+0000","last_became_active":"2024-09-10T09:35:36.619023+0000","last_became_peered":"2024-09-10T09:35:36.619023+0000","last_unstale":"2024-09-10T09:37:07.526545+0000","last_undegraded":"2024-09-10T09:37:07.526545+0000","last_fullsized":"2024-09-10T09:37:07.526545+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:07:32.999231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"96'4","reported_seq":43,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.548010+0000","last_change":"2024-09-10T09:35:28.575668+0000","last_active":"2024-09-10T09:37:35.548010+0000","last_peered":"2024-09-10T09:37:35.548010+0000","last_clean":"2024-09-10T09:37:35.548010+0000","last_became_active":"2024-09-10T09:35:28.575425+0000","last_became_peered":"2024-09-10T09:35:28.575425+0000","last_unstale":"2024-09-10T09:37:35.548010+0000","last_undegraded":"2024-09-10T09:37:35.548010+0000","last_fullsized":"2024-09-10T09:37:35.548010+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:13:27.120208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"94'1","reported_seq":45,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.528018+0000","last_change":"2024-09-10T09:35:25.854569+0000","last_active":"2024-09-10T09:37:07.528018+0000","last_peered":"2024-09-10T09:37:07.528018+0000","last_clean":"2024-09-10T09:37:07.528018+0000","last_became_active":"2024-09-10T09:35:25.854152+0000","last_became_peered":"2024-09-10T09:35:25.854152+0000","last_unstale":"2024-09-10T09:37:07.528018+0000","last_undegraded":"2024-09-10T09:37:07.528018+0000","last_fullsized":"2024-09-10T09:37:07.528018+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:27:15.584707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.529957+0000","last_change":"2024-09-10T09:35:37.270517+0000","last_active":"2024-09-10T09:37:07.529957+0000","last_peered":"2024-09-10T09:37:07.529957+0000","last_clean":"2024-09-10T09:37:07.529957+0000","last_became_active":"2024-09-10T09:35:37.269793+0000","last_became_peered":"2024-09-10T09:35:37.269793+0000","last_unstale":"2024-09-10T09:37:07.529957+0000","last_undegraded":"2024-09-10T09:37:07.529957+0000","last_fullsized":"2024-09-10T09:37:07.529957+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:28:58.272848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"96'7","reported_seq":52,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.556473+0000","last_change":"2024-09-10T09:37:07.762585+0000","last_active":"2024-09-10T09:37:35.556473+0000","last_peered":"2024-09-10T09:37:35.556473+0000","last_clean":"2024-09-10T09:37:35.556473+0000","last_became_active":"2024-09-10T09:37:07.762250+0000","last_became_peered":"2024-09-10T09:37:07.762250+0000","last_unstale":"2024-09-10T09:37:35.556473+0000","last_undegraded":"2024-09-10T09:37:35.556473+0000","last_fullsized":"2024-09-10T09:37:35.556473+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:38:01.751512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"96'7","reported_seq":172,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:38:12.661207+0000","last_change":"2024-09-10T09:35:25.531062+0000","last_active":"2024-09-10T09:38:12.661207+0000","last_peered":"2024-09-10T09:38:12.661207+0000","last_clean":"2024-09-10T09:38:12.661207+0000","last_became_active":"2024-09-10T09:35:25.530005+0000","last_became_peered":"2024-09-10T09:35:25.530005+0000","last_unstale":"2024-09-10T09:38:12.661207+0000","last_undegraded":"2024-09-10T09:38:12.661207+0000","last_fullsized":"2024-09-10T09:38:12.661207+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:42:35.330312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1122,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":122,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.538766+0000","last_change":"2024-09-10T09:35:36.623099+0000","last_active":"2024-09-10T09:37:07.538766+0000","last_peered":"2024-09-10T09:37:07.538766+0000","last_clean":"2024-09-10T09:37:07.538766+0000","last_became_active":"2024-09-10T09:35:36.622716+0000","last_became_peered":"2024-09-10T09:35:36.622716+0000","last_unstale":"2024-09-10T09:37:07.538766+0000","last_undegraded":"2024-09-10T09:37:07.538766+0000","last_fullsized":"2024-09-10T09:37:07.538766+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:10:38.495164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"96'8","reported_seq":53,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.543805+0000","last_change":"2024-09-10T09:36:06.769405+0000","last_active":"2024-09-10T09:37:35.543805+0000","last_peered":"2024-09-10T09:37:35.543805+0000","last_clean":"2024-09-10T09:37:35.543805+0000","last_became_active":"2024-09-10T09:36:06.769042+0000","last_became_peered":"2024-09-10T09:36:06.769042+0000","last_unstale":"2024-09-10T09:37:35.543805+0000","last_undegraded":"2024-09-10T09:37:35.543805+0000","last_fullsized":"2024-09-10T09:37:35.543805+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:01:49.340765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,8],"acting":[5,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"94'5","reported_seq":174,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:38:16.666205+0000","last_change":"2024-09-10T09:35:25.851430+0000","last_active":"2024-09-10T09:38:16.666205+0000","last_peered":"2024-09-10T09:38:16.666205+0000","last_clean":"2024-09-10T09:38:16.666205+0000","last_became_active":"2024-09-10T09:35:25.851067+0000","last_became_peered":"2024-09-10T09:35:25.851067+0000","last_unstale":"2024-09-10T09:38:16.666205+0000","last_undegraded":"2024-09-10T09:38:16.666205+0000","last_fullsized":"2024-09-10T09:38:16.666205+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:55:58.756207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":125,"num_read_kb":125,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.538676+0000","last_change":"2024-09-10T09:35:37.270361+0000","last_active":"2024-09-10T09:37:07.538676+0000","last_peered":"2024-09-10T09:37:07.538676+0000","last_clean":"2024-09-10T09:37:07.538676+0000","last_became_active":"2024-09-10T09:35:37.269632+0000","last_became_peered":"2024-09-10T09:35:37.269632+0000","last_unstale":"2024-09-10T09:37:07.538676+0000","last_undegraded":"2024-09-10T09:37:07.538676+0000","last_fullsized":"2024-09-10T09:37:07.538676+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:50:34.360497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"96'5","reported_seq":44,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.548275+0000","last_change":"2024-09-10T09:35:28.571064+0000","last_active":"2024-09-10T09:37:35.548275+0000","last_peered":"2024-09-10T09:37:35.548275+0000","last_clean":"2024-09-10T09:37:35.548275+0000","last_became_active":"2024-09-10T09:35:28.570691+0000","last_became_peered":"2024-09-10T09:35:28.570691+0000","last_unstale":"2024-09-10T09:37:35.548275+0000","last_undegraded":"2024-09-10T09:37:35.548275+0000","last_fullsized":"2024-09-10T09:37:35.548275+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:58:33.162501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"94'4","reported_seq":57,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.572759+0000","last_change":"2024-09-10T09:37:07.572247+0000","last_active":"2024-09-10T09:37:07.572759+0000","last_peered":"2024-09-10T09:37:07.572759+0000","last_clean":"2024-09-10T09:37:07.572759+0000","last_became_active":"2024-09-10T09:37:07.569410+0000","last_became_peered":"2024-09-10T09:37:07.569410+0000","last_unstale":"2024-09-10T09:37:07.572759+0000","last_undegraded":"2024-09-10T09:37:07.572759+0000","last_fullsized":"2024-09-10T09:37:07.572759+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:05:09.037084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,4],"acting":[7,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.532723+0000","last_change":"2024-09-10T09:35:37.270358+0000","last_active":"2024-09-10T09:37:07.532723+0000","last_peered":"2024-09-10T09:37:07.532723+0000","last_clean":"2024-09-10T09:37:07.532723+0000","last_became_active":"2024-09-10T09:35:37.269822+0000","last_became_peered":"2024-09-10T09:35:37.269822+0000","last_unstale":"2024-09-10T09:37:07.532723+0000","last_undegraded":"2024-09-10T09:37:07.532723+0000","last_fullsized":"2024-09-10T09:37:07.532723+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:19:48.932596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"96'8","reported_seq":47,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.553803+0000","last_change":"2024-09-10T09:35:28.576244+0000","last_active":"2024-09-10T09:37:35.553803+0000","last_peered":"2024-09-10T09:37:35.553803+0000","last_clean":"2024-09-10T09:37:35.553803+0000","last_became_active":"2024-09-10T09:35:28.571801+0000","last_became_peered":"2024-09-10T09:35:28.571801+0000","last_unstale":"2024-09-10T09:37:35.553803+0000","last_undegraded":"2024-09-10T09:37:35.553803+0000","last_fullsized":"2024-09-10T09:37:35.553803+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:11:21.742220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"94'6","reported_seq":52,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.568754+0000","last_change":"2024-09-10T09:35:25.537118+0000","last_active":"2024-09-10T09:37:07.568754+0000","last_peered":"2024-09-10T09:37:07.568754+0000","last_clean":"2024-09-10T09:37:07.568754+0000","last_became_active":"2024-09-10T09:35:25.536718+0000","last_became_peered":"2024-09-10T09:35:25.536718+0000","last_unstale":"2024-09-10T09:37:07.568754+0000","last_undegraded":"2024-09-10T09:37:07.568754+0000","last_fullsized":"2024-09-10T09:37:07.568754+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:16:32.837476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":361,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.528720+0000","last_change":"2024-09-10T09:34:07.350825+0000","last_active":"2024-09-10T09:37:07.528720+0000","last_peered":"2024-09-10T09:37:07.528720+0000","last_clean":"2024-09-10T09:37:07.528720+0000","last_became_active":"2024-09-10T09:34:07.350325+0000","last_became_peered":"2024-09-10T09:34:07.350325+0000","last_unstale":"2024-09-10T09:37:07.528720+0000","last_undegraded":"2024-09-10T09:37:07.528720+0000","last_fullsized":"2024-09-10T09:37:07.528720+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:30:26.408262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:30:26.408262+0000","last_clean_scrub_stamp":"2024-09-10T09:30:26.408262+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:58:04.920443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"94'1","reported_seq":30,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.529308+0000","last_change":"2024-09-10T09:35:36.632941+0000","last_active":"2024-09-10T09:37:07.529308+0000","last_peered":"2024-09-10T09:37:07.529308+0000","last_clean":"2024-09-10T09:37:07.529308+0000","last_became_active":"2024-09-10T09:35:36.632527+0000","last_became_peered":"2024-09-10T09:35:36.632527+0000","last_unstale":"2024-09-10T09:37:07.529308+0000","last_undegraded":"2024-09-10T09:37:07.529308+0000","last_fullsized":"2024-09-10T09:37:07.529308+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:33:21.733772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"96'7","reported_seq":44,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.556036+0000","last_change":"2024-09-10T09:35:28.572010+0000","last_active":"2024-09-10T09:37:35.556036+0000","last_peered":"2024-09-10T09:37:35.556036+0000","last_clean":"2024-09-10T09:37:35.556036+0000","last_became_active":"2024-09-10T09:35:28.571623+0000","last_became_peered":"2024-09-10T09:35:28.571623+0000","last_unstale":"2024-09-10T09:37:35.556036+0000","last_undegraded":"2024-09-10T09:37:35.556036+0000","last_fullsized":"2024-09-10T09:37:35.556036+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:00:49.328663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"94'5","reported_seq":173,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:38:14.664651+0000","last_change":"2024-09-10T09:35:25.990328+0000","last_active":"2024-09-10T09:38:14.664651+0000","last_peered":"2024-09-10T09:38:14.664651+0000","last_clean":"2024-09-10T09:38:14.664651+0000","last_became_active":"2024-09-10T09:35:25.989946+0000","last_became_peered":"2024-09-10T09:35:25.989946+0000","last_unstale":"2024-09-10T09:38:14.664651+0000","last_undegraded":"2024-09-10T09:38:14.664651+0000","last_fullsized":"2024-09-10T09:38:14.664651+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:51:16.069913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":124,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.574499+0000","last_change":"2024-09-10T09:35:36.619013+0000","last_active":"2024-09-10T09:37:07.574499+0000","last_peered":"2024-09-10T09:37:07.574499+0000","last_clean":"2024-09-10T09:37:07.574499+0000","last_became_active":"2024-09-10T09:35:36.618506+0000","last_became_peered":"2024-09-10T09:35:36.618506+0000","last_unstale":"2024-09-10T09:37:07.574499+0000","last_undegraded":"2024-09-10T09:37:07.574499+0000","last_fullsized":"2024-09-10T09:37:07.574499+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:57:53.580569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"96'6","reported_seq":49,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.553955+0000","last_change":"2024-09-10T09:36:06.768573+0000","last_active":"2024-09-10T09:37:35.553955+0000","last_peered":"2024-09-10T09:37:35.553955+0000","last_clean":"2024-09-10T09:37:35.553955+0000","last_became_active":"2024-09-10T09:36:06.768228+0000","last_became_peered":"2024-09-10T09:36:06.768228+0000","last_unstale":"2024-09-10T09:37:35.553955+0000","last_undegraded":"2024-09-10T09:37:35.553955+0000","last_fullsized":"2024-09-10T09:37:35.553955+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:57:38.310713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"94'3","reported_seq":48,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.533010+0000","last_change":"2024-09-10T09:35:25.528795+0000","last_active":"2024-09-10T09:37:07.533010+0000","last_peered":"2024-09-10T09:37:07.533010+0000","last_clean":"2024-09-10T09:37:07.533010+0000","last_became_active":"2024-09-10T09:35:25.528475+0000","last_became_peered":"2024-09-10T09:35:25.528475+0000","last_unstale":"2024-09-10T09:37:07.533010+0000","last_undegraded":"2024-09-10T09:37:07.533010+0000","last_fullsized":"2024-09-10T09:37:07.533010+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:16:06.999186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.761758+0000","last_change":"2024-09-10T09:35:36.633109+0000","last_active":"2024-09-10T09:37:07.761758+0000","last_peered":"2024-09-10T09:37:07.761758+0000","last_clean":"2024-09-10T09:37:07.761758+0000","last_became_active":"2024-09-10T09:35:36.632682+0000","last_became_peered":"2024-09-10T09:35:36.632682+0000","last_unstale":"2024-09-10T09:37:07.761758+0000","last_undegraded":"2024-09-10T09:37:07.761758+0000","last_fullsized":"2024-09-10T09:37:07.761758+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:51:14.372649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"96'9","reported_seq":46,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.552249+0000","last_change":"2024-09-10T09:35:28.562103+0000","last_active":"2024-09-10T09:37:35.552249+0000","last_peered":"2024-09-10T09:37:35.552249+0000","last_clean":"2024-09-10T09:37:35.552249+0000","last_became_active":"2024-09-10T09:35:28.561765+0000","last_became_peered":"2024-09-10T09:35:28.561765+0000","last_unstale":"2024-09-10T09:37:35.552249+0000","last_undegraded":"2024-09-10T09:37:35.552249+0000","last_fullsized":"2024-09-10T09:37:35.552249+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:35:35.400569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"94'5","reported_seq":49,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.539243+0000","last_change":"2024-09-10T09:35:25.538536+0000","last_active":"2024-09-10T09:37:07.539243+0000","last_peered":"2024-09-10T09:37:07.539243+0000","last_clean":"2024-09-10T09:37:07.539243+0000","last_became_active":"2024-09-10T09:35:25.537944+0000","last_became_peered":"2024-09-10T09:35:25.537944+0000","last_unstale":"2024-09-10T09:37:07.539243+0000","last_undegraded":"2024-09-10T09:37:07.539243+0000","last_fullsized":"2024-09-10T09:37:07.539243+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:01:19.385381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":17,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.529928+0000","last_change":"2024-09-10T09:36:06.769534+0000","last_active":"2024-09-10T09:37:07.529928+0000","last_peered":"2024-09-10T09:37:07.529928+0000","last_clean":"2024-09-10T09:37:07.529928+0000","last_became_active":"2024-09-10T09:36:06.769015+0000","last_became_peered":"2024-09-10T09:36:06.769015+0000","last_unstale":"2024-09-10T09:37:07.529928+0000","last_undegraded":"2024-09-10T09:37:07.529928+0000","last_fullsized":"2024-09-10T09:37:07.529928+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:19:26.369737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7,1],"acting":[8,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"96'5","reported_seq":44,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.539962+0000","last_change":"2024-09-10T09:35:28.566735+0000","last_active":"2024-09-10T09:37:35.539962+0000","last_peered":"2024-09-10T09:37:35.539962+0000","last_clean":"2024-09-10T09:37:35.539962+0000","last_became_active":"2024-09-10T09:35:28.566400+0000","last_became_peered":"2024-09-10T09:35:28.566400+0000","last_unstale":"2024-09-10T09:37:35.539962+0000","last_undegraded":"2024-09-10T09:37:35.539962+0000","last_fullsized":"2024-09-10T09:37:35.539962+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:18:53.337816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"94'2","reported_seq":47,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.530569+0000","last_change":"2024-09-10T09:35:25.535783+0000","last_active":"2024-09-10T09:37:07.530569+0000","last_peered":"2024-09-10T09:37:07.530569+0000","last_clean":"2024-09-10T09:37:07.530569+0000","last_became_active":"2024-09-10T09:35:25.535411+0000","last_became_peered":"2024-09-10T09:35:25.535411+0000","last_unstale":"2024-09-10T09:37:07.530569+0000","last_undegraded":"2024-09-10T09:37:07.530569+0000","last_fullsized":"2024-09-10T09:37:07.530569+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:48:09.239083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.531208+0000","last_change":"2024-09-10T09:35:36.619380+0000","last_active":"2024-09-10T09:37:07.531208+0000","last_peered":"2024-09-10T09:37:07.531208+0000","last_clean":"2024-09-10T09:37:07.531208+0000","last_became_active":"2024-09-10T09:35:36.618953+0000","last_became_peered":"2024-09-10T09:35:36.618953+0000","last_unstale":"2024-09-10T09:37:07.531208+0000","last_undegraded":"2024-09-10T09:37:07.531208+0000","last_fullsized":"2024-09-10T09:37:07.531208+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:40:25.181408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"96'6","reported_seq":45,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.546928+0000","last_change":"2024-09-10T09:35:28.571396+0000","last_active":"2024-09-10T09:37:35.546928+0000","last_peered":"2024-09-10T09:37:35.546928+0000","last_clean":"2024-09-10T09:37:35.546928+0000","last_became_active":"2024-09-10T09:35:28.571063+0000","last_became_peered":"2024-09-10T09:35:28.571063+0000","last_unstale":"2024-09-10T09:37:35.546928+0000","last_undegraded":"2024-09-10T09:37:35.546928+0000","last_fullsized":"2024-09-10T09:37:35.546928+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:38:17.965133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"94'17","reported_seq":62,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.530060+0000","last_change":"2024-09-10T09:35:25.851773+0000","last_active":"2024-09-10T09:37:07.530060+0000","last_peered":"2024-09-10T09:37:07.530060+0000","last_clean":"2024-09-10T09:37:07.530060+0000","last_became_active":"2024-09-10T09:35:25.851384+0000","last_became_peered":"2024-09-10T09:35:25.851384+0000","last_unstale":"2024-09-10T09:37:07.530060+0000","last_undegraded":"2024-09-10T09:37:07.530060+0000","last_fullsized":"2024-09-10T09:37:07.530060+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:03:32.338643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.539186+0000","last_change":"2024-09-10T09:35:36.620971+0000","last_active":"2024-09-10T09:37:07.539186+0000","last_peered":"2024-09-10T09:37:07.539186+0000","last_clean":"2024-09-10T09:37:07.539186+0000","last_became_active":"2024-09-10T09:35:36.620343+0000","last_became_peered":"2024-09-10T09:35:36.620343+0000","last_unstale":"2024-09-10T09:37:07.539186+0000","last_undegraded":"2024-09-10T09:37:07.539186+0000","last_fullsized":"2024-09-10T09:37:07.539186+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:17:19.517882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"96'6","reported_seq":43,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.553818+0000","last_change":"2024-09-10T09:35:28.565643+0000","last_active":"2024-09-10T09:37:35.553818+0000","last_peered":"2024-09-10T09:37:35.553818+0000","last_clean":"2024-09-10T09:37:35.553818+0000","last_became_active":"2024-09-10T09:35:28.564217+0000","last_became_peered":"2024-09-10T09:35:28.564217+0000","last_unstale":"2024-09-10T09:37:35.553818+0000","last_undegraded":"2024-09-10T09:37:35.553818+0000","last_fullsized":"2024-09-10T09:37:35.553818+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:53:34.562690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"94'5","reported_seq":174,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:38:16.666733+0000","last_change":"2024-09-10T09:35:25.534167+0000","last_active":"2024-09-10T09:38:16.666733+0000","last_peered":"2024-09-10T09:38:16.666733+0000","last_clean":"2024-09-10T09:38:16.666733+0000","last_became_active":"2024-09-10T09:35:25.533719+0000","last_became_peered":"2024-09-10T09:35:25.533719+0000","last_unstale":"2024-09-10T09:38:16.666733+0000","last_undegraded":"2024-09-10T09:38:16.666733+0000","last_fullsized":"2024-09-10T09:38:16.666733+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:00:03.537355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":127,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.527733+0000","last_change":"2024-09-10T09:35:36.628716+0000","last_active":"2024-09-10T09:37:07.527733+0000","last_peered":"2024-09-10T09:37:07.527733+0000","last_clean":"2024-09-10T09:37:07.527733+0000","last_became_active":"2024-09-10T09:35:36.628085+0000","last_became_peered":"2024-09-10T09:35:36.628085+0000","last_unstale":"2024-09-10T09:37:07.527733+0000","last_undegraded":"2024-09-10T09:37:07.527733+0000","last_fullsized":"2024-09-10T09:37:07.527733+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:45:01.279403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"96'15","reported_seq":52,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.557076+0000","last_change":"2024-09-10T09:35:28.566701+0000","last_active":"2024-09-10T09:37:35.557076+0000","last_peered":"2024-09-10T09:37:35.557076+0000","last_clean":"2024-09-10T09:37:35.557076+0000","last_became_active":"2024-09-10T09:35:28.566391+0000","last_became_peered":"2024-09-10T09:35:28.566391+0000","last_unstale":"2024-09-10T09:37:35.557076+0000","last_undegraded":"2024-09-10T09:37:35.557076+0000","last_fullsized":"2024-09-10T09:37:35.557076+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:57:59.918706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"94'5","reported_seq":50,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.531896+0000","last_change":"2024-09-10T09:35:25.532510+0000","last_active":"2024-09-10T09:37:07.531896+0000","last_peered":"2024-09-10T09:37:07.531896+0000","last_clean":"2024-09-10T09:37:07.531896+0000","last_became_active":"2024-09-10T09:35:25.532193+0000","last_became_peered":"2024-09-10T09:35:25.532193+0000","last_unstale":"2024-09-10T09:37:07.531896+0000","last_undegraded":"2024-09-10T09:37:07.531896+0000","last_fullsized":"2024-09-10T09:37:07.531896+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:17:24.685178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.526424+0000","last_change":"2024-09-10T09:35:36.630742+0000","last_active":"2024-09-10T09:37:07.526424+0000","last_peered":"2024-09-10T09:37:07.526424+0000","last_clean":"2024-09-10T09:37:07.526424+0000","last_became_active":"2024-09-10T09:35:36.630335+0000","last_became_peered":"2024-09-10T09:35:36.630335+0000","last_unstale":"2024-09-10T09:37:07.526424+0000","last_undegraded":"2024-09-10T09:37:07.526424+0000","last_fullsized":"2024-09-10T09:37:07.526424+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:29:12.029390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"96'9","reported_seq":48,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.552455+0000","last_change":"2024-09-10T09:35:28.581817+0000","last_active":"2024-09-10T09:37:35.552455+0000","last_peered":"2024-09-10T09:37:35.552455+0000","last_clean":"2024-09-10T09:37:35.552455+0000","last_became_active":"2024-09-10T09:35:28.580316+0000","last_became_peered":"2024-09-10T09:35:28.580316+0000","last_unstale":"2024-09-10T09:37:35.552455+0000","last_undegraded":"2024-09-10T09:37:35.552455+0000","last_fullsized":"2024-09-10T09:37:35.552455+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:04:08.249564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"94'9","reported_seq":55,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.528127+0000","last_change":"2024-09-10T09:35:25.537474+0000","last_active":"2024-09-10T09:37:07.528127+0000","last_peered":"2024-09-10T09:37:07.528127+0000","last_clean":"2024-09-10T09:37:07.528127+0000","last_became_active":"2024-09-10T09:35:25.537096+0000","last_became_peered":"2024-09-10T09:35:25.537096+0000","last_unstale":"2024-09-10T09:37:07.528127+0000","last_undegraded":"2024-09-10T09:37:07.528127+0000","last_fullsized":"2024-09-10T09:37:07.528127+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:13:06.008458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.532316+0000","last_change":"2024-09-10T09:35:36.622788+0000","last_active":"2024-09-10T09:37:07.532316+0000","last_peered":"2024-09-10T09:37:07.532316+0000","last_clean":"2024-09-10T09:37:07.532316+0000","last_became_active":"2024-09-10T09:35:36.622377+0000","last_became_peered":"2024-09-10T09:35:36.622377+0000","last_unstale":"2024-09-10T09:37:07.532316+0000","last_undegraded":"2024-09-10T09:37:07.532316+0000","last_fullsized":"2024-09-10T09:37:07.532316+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:29:56.680864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"96'6","reported_seq":45,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.534371+0000","last_change":"2024-09-10T09:35:28.564529+0000","last_active":"2024-09-10T09:37:35.534371+0000","last_peered":"2024-09-10T09:37:35.534371+0000","last_clean":"2024-09-10T09:37:35.534371+0000","last_became_active":"2024-09-10T09:35:28.564223+0000","last_became_peered":"2024-09-10T09:35:28.564223+0000","last_unstale":"2024-09-10T09:37:35.534371+0000","last_undegraded":"2024-09-10T09:37:35.534371+0000","last_fullsized":"2024-09-10T09:37:35.534371+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:21:24.296578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"96'38","reported_seq":330,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:38:13.662697+0000","last_change":"2024-09-10T09:35:25.534331+0000","last_active":"2024-09-10T09:38:13.662697+0000","last_peered":"2024-09-10T09:38:13.662697+0000","last_clean":"2024-09-10T09:38:13.662697+0000","last_became_active":"2024-09-10T09:35:25.533985+0000","last_became_peered":"2024-09-10T09:35:25.533985+0000","last_unstale":"2024-09-10T09:38:13.662697+0000","last_undegraded":"2024-09-10T09:38:13.662697+0000","last_fullsized":"2024-09-10T09:38:13.662697+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:14:37.050650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":63180,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":248,"num_read_kb":186,"num_write":40,"num_write_kb":79,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.527925+0000","last_change":"2024-09-10T09:35:36.632835+0000","last_active":"2024-09-10T09:37:07.527925+0000","last_peered":"2024-09-10T09:37:07.527925+0000","last_clean":"2024-09-10T09:37:07.527925+0000","last_became_active":"2024-09-10T09:35:36.630962+0000","last_became_peered":"2024-09-10T09:35:36.630962+0000","last_unstale":"2024-09-10T09:37:07.527925+0000","last_undegraded":"2024-09-10T09:37:07.527925+0000","last_fullsized":"2024-09-10T09:37:07.527925+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:59:27.613721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"96'11","reported_seq":50,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.546541+0000","last_change":"2024-09-10T09:35:28.567292+0000","last_active":"2024-09-10T09:37:35.546541+0000","last_peered":"2024-09-10T09:37:35.546541+0000","last_clean":"2024-09-10T09:37:35.546541+0000","last_became_active":"2024-09-10T09:35:28.566973+0000","last_became_peered":"2024-09-10T09:35:28.566973+0000","last_unstale":"2024-09-10T09:37:35.546541+0000","last_undegraded":"2024-09-10T09:37:35.546541+0000","last_fullsized":"2024-09-10T09:37:35.546541+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:05:37.251933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"94'5","reported_seq":50,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.532787+0000","last_change":"2024-09-10T09:35:25.852171+0000","last_active":"2024-09-10T09:37:07.532787+0000","last_peered":"2024-09-10T09:37:07.532787+0000","last_clean":"2024-09-10T09:37:07.532787+0000","last_became_active":"2024-09-10T09:35:25.851580+0000","last_became_peered":"2024-09-10T09:35:25.851580+0000","last_unstale":"2024-09-10T09:37:07.532787+0000","last_undegraded":"2024-09-10T09:37:07.532787+0000","last_fullsized":"2024-09-10T09:37:07.532787+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:03.263152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.568336+0000","last_change":"2024-09-10T09:35:36.629212+0000","last_active":"2024-09-10T09:37:07.568336+0000","last_peered":"2024-09-10T09:37:07.568336+0000","last_clean":"2024-09-10T09:37:07.568336+0000","last_became_active":"2024-09-10T09:35:36.628753+0000","last_became_peered":"2024-09-10T09:35:36.628753+0000","last_unstale":"2024-09-10T09:37:07.568336+0000","last_undegraded":"2024-09-10T09:37:07.568336+0000","last_fullsized":"2024-09-10T09:37:07.568336+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:54:45.311361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"96'9","reported_seq":46,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.553075+0000","last_change":"2024-09-10T09:35:28.572497+0000","last_active":"2024-09-10T09:37:35.553075+0000","last_peered":"2024-09-10T09:37:35.553075+0000","last_clean":"2024-09-10T09:37:35.553075+0000","last_became_active":"2024-09-10T09:35:28.569238+0000","last_became_peered":"2024-09-10T09:35:28.569238+0000","last_unstale":"2024-09-10T09:37:35.553075+0000","last_undegraded":"2024-09-10T09:37:35.553075+0000","last_fullsized":"2024-09-10T09:37:35.553075+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:38:47.189209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"94'4","reported_seq":47,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.529569+0000","last_change":"2024-09-10T09:35:25.536369+0000","last_active":"2024-09-10T09:37:07.529569+0000","last_peered":"2024-09-10T09:37:07.529569+0000","last_clean":"2024-09-10T09:37:07.529569+0000","last_became_active":"2024-09-10T09:35:25.535607+0000","last_became_peered":"2024-09-10T09:35:25.535607+0000","last_unstale":"2024-09-10T09:37:07.529569+0000","last_undegraded":"2024-09-10T09:37:07.529569+0000","last_fullsized":"2024-09-10T09:37:07.529569+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:49:18.621934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.527047+0000","last_change":"2024-09-10T09:35:36.627977+0000","last_active":"2024-09-10T09:37:07.527047+0000","last_peered":"2024-09-10T09:37:07.527047+0000","last_clean":"2024-09-10T09:37:07.527047+0000","last_became_active":"2024-09-10T09:35:36.626942+0000","last_became_peered":"2024-09-10T09:35:36.626942+0000","last_unstale":"2024-09-10T09:37:07.527047+0000","last_undegraded":"2024-09-10T09:37:07.527047+0000","last_fullsized":"2024-09-10T09:37:07.527047+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:37:18.209424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"96'10","reported_seq":49,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.553165+0000","last_change":"2024-09-10T09:35:28.562670+0000","last_active":"2024-09-10T09:37:35.553165+0000","last_peered":"2024-09-10T09:37:35.553165+0000","last_clean":"2024-09-10T09:37:35.553165+0000","last_became_active":"2024-09-10T09:35:28.562028+0000","last_became_peered":"2024-09-10T09:35:28.562028+0000","last_unstale":"2024-09-10T09:37:35.553165+0000","last_undegraded":"2024-09-10T09:37:35.553165+0000","last_fullsized":"2024-09-10T09:37:35.553165+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:37:15.176050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"96'5","reported_seq":288,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:38:13.662801+0000","last_change":"2024-09-10T09:35:25.529861+0000","last_active":"2024-09-10T09:38:13.662801+0000","last_peered":"2024-09-10T09:38:13.662801+0000","last_clean":"2024-09-10T09:38:13.662801+0000","last_became_active":"2024-09-10T09:35:25.529475+0000","last_became_peered":"2024-09-10T09:35:25.529475+0000","last_unstale":"2024-09-10T09:38:13.662801+0000","last_undegraded":"2024-09-10T09:38:13.662801+0000","last_fullsized":"2024-09-10T09:38:13.662801+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:44:27.086509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":239,"num_read_kb":239,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.524306+0000","last_change":"2024-09-10T09:35:36.615358+0000","last_active":"2024-09-10T09:37:07.524306+0000","last_peered":"2024-09-10T09:37:07.524306+0000","last_clean":"2024-09-10T09:37:07.524306+0000","last_became_active":"2024-09-10T09:35:36.614901+0000","last_became_peered":"2024-09-10T09:35:36.614901+0000","last_unstale":"2024-09-10T09:37:07.524306+0000","last_undegraded":"2024-09-10T09:37:07.524306+0000","last_fullsized":"2024-09-10T09:37:07.524306+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:56:54.248266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.761302+0000","last_change":"2024-09-10T09:35:36.629021+0000","last_active":"2024-09-10T09:37:07.761302+0000","last_peered":"2024-09-10T09:37:07.761302+0000","last_clean":"2024-09-10T09:37:07.761302+0000","last_became_active":"2024-09-10T09:35:36.628159+0000","last_became_peered":"2024-09-10T09:35:36.628159+0000","last_unstale":"2024-09-10T09:37:07.761302+0000","last_undegraded":"2024-09-10T09:37:07.761302+0000","last_fullsized":"2024-09-10T09:37:07.761302+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:50:48.615236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"96'11","reported_seq":48,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.545215+0000","last_change":"2024-09-10T09:35:28.579333+0000","last_active":"2024-09-10T09:37:35.545215+0000","last_peered":"2024-09-10T09:37:35.545215+0000","last_clean":"2024-09-10T09:37:35.545215+0000","last_became_active":"2024-09-10T09:35:28.578440+0000","last_became_peered":"2024-09-10T09:35:28.578440+0000","last_unstale":"2024-09-10T09:37:35.545215+0000","last_undegraded":"2024-09-10T09:37:35.545215+0000","last_fullsized":"2024-09-10T09:37:35.545215+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:07:28.701938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"94'3","reported_seq":48,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.531063+0000","last_change":"2024-09-10T09:35:25.530857+0000","last_active":"2024-09-10T09:37:07.531063+0000","last_peered":"2024-09-10T09:37:07.531063+0000","last_clean":"2024-09-10T09:37:07.531063+0000","last_became_active":"2024-09-10T09:35:25.530492+0000","last_became_peered":"2024-09-10T09:35:25.530492+0000","last_unstale":"2024-09-10T09:37:07.531063+0000","last_undegraded":"2024-09-10T09:37:07.531063+0000","last_fullsized":"2024-09-10T09:37:07.531063+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:07:21.726598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.526432+0000","last_change":"2024-09-10T09:36:06.770408+0000","last_active":"2024-09-10T09:37:07.526432+0000","last_peered":"2024-09-10T09:37:07.526432+0000","last_clean":"2024-09-10T09:37:07.526432+0000","last_became_active":"2024-09-10T09:36:06.769915+0000","last_became_peered":"2024-09-10T09:36:06.769915+0000","last_unstale":"2024-09-10T09:37:07.526432+0000","last_undegraded":"2024-09-10T09:37:07.526432+0000","last_fullsized":"2024-09-10T09:37:07.526432+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:56:09.795537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8,0],"acting":[11,4,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"96'8","reported_seq":47,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.543491+0000","last_change":"2024-09-10T09:35:28.570804+0000","last_active":"2024-09-10T09:37:35.543491+0000","last_peered":"2024-09-10T09:37:35.543491+0000","last_clean":"2024-09-10T09:37:35.543491+0000","last_became_active":"2024-09-10T09:35:28.570490+0000","last_became_peered":"2024-09-10T09:35:28.570490+0000","last_unstale":"2024-09-10T09:37:35.543491+0000","last_undegraded":"2024-09-10T09:37:35.543491+0000","last_fullsized":"2024-09-10T09:37:35.543491+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:13:38.999625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"94'3","reported_seq":47,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.538446+0000","last_change":"2024-09-10T09:35:25.536347+0000","last_active":"2024-09-10T09:37:07.538446+0000","last_peered":"2024-09-10T09:37:07.538446+0000","last_clean":"2024-09-10T09:37:07.538446+0000","last_became_active":"2024-09-10T09:35:25.535964+0000","last_became_peered":"2024-09-10T09:35:25.535964+0000","last_unstale":"2024-09-10T09:37:07.538446+0000","last_undegraded":"2024-09-10T09:37:07.538446+0000","last_fullsized":"2024-09-10T09:37:07.538446+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:36:49.035666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"94'7","reported_seq":51,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.525812+0000","last_change":"2024-09-10T09:35:25.532148+0000","last_active":"2024-09-10T09:37:07.525812+0000","last_peered":"2024-09-10T09:37:07.525812+0000","last_clean":"2024-09-10T09:37:07.525812+0000","last_became_active":"2024-09-10T09:35:25.531781+0000","last_became_peered":"2024-09-10T09:35:25.531781+0000","last_unstale":"2024-09-10T09:37:07.525812+0000","last_undegraded":"2024-09-10T09:37:07.525812+0000","last_fullsized":"2024-09-10T09:37:07.525812+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:52:46.740342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"96'8","reported_seq":47,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.549412+0000","last_change":"2024-09-10T09:35:28.578336+0000","last_active":"2024-09-10T09:37:35.549412+0000","last_peered":"2024-09-10T09:37:35.549412+0000","last_clean":"2024-09-10T09:37:35.549412+0000","last_became_active":"2024-09-10T09:35:28.577899+0000","last_became_peered":"2024-09-10T09:35:28.577899+0000","last_unstale":"2024-09-10T09:37:35.549412+0000","last_undegraded":"2024-09-10T09:37:35.549412+0000","last_fullsized":"2024-09-10T09:37:35.549412+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:13:04.478195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.528122+0000","last_change":"2024-09-10T09:35:36.630918+0000","last_active":"2024-09-10T09:37:07.528122+0000","last_peered":"2024-09-10T09:37:07.528122+0000","last_clean":"2024-09-10T09:37:07.528122+0000","last_became_active":"2024-09-10T09:35:36.630308+0000","last_became_peered":"2024-09-10T09:35:36.630308+0000","last_unstale":"2024-09-10T09:37:07.528122+0000","last_undegraded":"2024-09-10T09:37:07.528122+0000","last_fullsized":"2024-09-10T09:37:07.528122+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:25:44.954970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"96'7","reported_seq":186,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:38:12.661788+0000","last_change":"2024-09-10T09:37:07.574386+0000","last_active":"2024-09-10T09:38:12.661788+0000","last_peered":"2024-09-10T09:38:12.661788+0000","last_clean":"2024-09-10T09:38:12.661788+0000","last_became_active":"2024-09-10T09:37:07.569370+0000","last_became_peered":"2024-09-10T09:37:07.569370+0000","last_unstale":"2024-09-10T09:38:12.661788+0000","last_undegraded":"2024-09-10T09:38:12.661788+0000","last_fullsized":"2024-09-10T09:38:12.661788+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:07:23.014789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1156,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":118,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":1015,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"96'7","reported_seq":46,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.543648+0000","last_change":"2024-09-10T09:35:28.567915+0000","last_active":"2024-09-10T09:37:35.543648+0000","last_peered":"2024-09-10T09:37:35.543648+0000","last_clean":"2024-09-10T09:37:35.543648+0000","last_became_active":"2024-09-10T09:35:28.567416+0000","last_became_peered":"2024-09-10T09:35:28.567416+0000","last_unstale":"2024-09-10T09:37:35.543648+0000","last_undegraded":"2024-09-10T09:37:35.543648+0000","last_fullsized":"2024-09-10T09:37:35.543648+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:07:25.704080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.528686+0000","last_change":"2024-09-10T09:35:36.615386+0000","last_active":"2024-09-10T09:37:07.528686+0000","last_peered":"2024-09-10T09:37:07.528686+0000","last_clean":"2024-09-10T09:37:07.528686+0000","last_became_active":"2024-09-10T09:35:36.614967+0000","last_became_peered":"2024-09-10T09:35:36.614967+0000","last_unstale":"2024-09-10T09:37:07.528686+0000","last_undegraded":"2024-09-10T09:37:07.528686+0000","last_fullsized":"2024-09-10T09:37:07.528686+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:47.620729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"96'9","reported_seq":48,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.544481+0000","last_change":"2024-09-10T09:35:28.564411+0000","last_active":"2024-09-10T09:37:35.544481+0000","last_peered":"2024-09-10T09:37:35.544481+0000","last_clean":"2024-09-10T09:37:35.544481+0000","last_became_active":"2024-09-10T09:35:28.564034+0000","last_became_peered":"2024-09-10T09:35:28.564034+0000","last_unstale":"2024-09-10T09:37:35.544481+0000","last_undegraded":"2024-09-10T09:37:35.544481+0000","last_fullsized":"2024-09-10T09:37:35.544481+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:38:17.687991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"96'12","reported_seq":49,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.547277+0000","last_change":"2024-09-10T09:35:28.564619+0000","last_active":"2024-09-10T09:37:35.547277+0000","last_peered":"2024-09-10T09:37:35.547277+0000","last_clean":"2024-09-10T09:37:35.547277+0000","last_became_active":"2024-09-10T09:35:28.563890+0000","last_became_peered":"2024-09-10T09:35:28.563890+0000","last_unstale":"2024-09-10T09:37:35.547277+0000","last_undegraded":"2024-09-10T09:37:35.547277+0000","last_fullsized":"2024-09-10T09:37:35.547277+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:51:42.481911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"96'8","reported_seq":45,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.543410+0000","last_change":"2024-09-10T09:35:28.558909+0000","last_active":"2024-09-10T09:37:35.543410+0000","last_peered":"2024-09-10T09:37:35.543410+0000","last_clean":"2024-09-10T09:37:35.543410+0000","last_became_active":"2024-09-10T09:35:28.558604+0000","last_became_peered":"2024-09-10T09:35:28.558604+0000","last_unstale":"2024-09-10T09:37:35.543410+0000","last_undegraded":"2024-09-10T09:37:35.543410+0000","last_fullsized":"2024-09-10T09:37:35.543410+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:11.928446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"96'7","reported_seq":44,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.554844+0000","last_change":"2024-09-10T09:35:28.574459+0000","last_active":"2024-09-10T09:37:35.554844+0000","last_peered":"2024-09-10T09:37:35.554844+0000","last_clean":"2024-09-10T09:37:35.554844+0000","last_became_active":"2024-09-10T09:35:28.574154+0000","last_became_peered":"2024-09-10T09:35:28.574154+0000","last_unstale":"2024-09-10T09:37:35.554844+0000","last_undegraded":"2024-09-10T09:37:35.554844+0000","last_fullsized":"2024-09-10T09:37:35.554844+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:59:09.650536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"96'4","reported_seq":43,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.552571+0000","last_change":"2024-09-10T09:35:28.576999+0000","last_active":"2024-09-10T09:37:35.552571+0000","last_peered":"2024-09-10T09:37:35.552571+0000","last_clean":"2024-09-10T09:37:35.552571+0000","last_became_active":"2024-09-10T09:35:28.575616+0000","last_became_peered":"2024-09-10T09:35:28.575616+0000","last_unstale":"2024-09-10T09:37:35.552571+0000","last_undegraded":"2024-09-10T09:37:35.552571+0000","last_fullsized":"2024-09-10T09:37:35.552571+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:18:08.771760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"96'6","reported_seq":45,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.541496+0000","last_change":"2024-09-10T09:35:28.564846+0000","last_active":"2024-09-10T09:37:35.541496+0000","last_peered":"2024-09-10T09:37:35.541496+0000","last_clean":"2024-09-10T09:37:35.541496+0000","last_became_active":"2024-09-10T09:35:28.564498+0000","last_became_peered":"2024-09-10T09:35:28.564498+0000","last_unstale":"2024-09-10T09:37:35.541496+0000","last_undegraded":"2024-09-10T09:37:35.541496+0000","last_fullsized":"2024-09-10T09:37:35.541496+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:33:43.041730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"96'15","reported_seq":52,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.550975+0000","last_change":"2024-09-10T09:35:28.558093+0000","last_active":"2024-09-10T09:37:35.550975+0000","last_peered":"2024-09-10T09:37:35.550975+0000","last_clean":"2024-09-10T09:37:35.550975+0000","last_became_active":"2024-09-10T09:35:28.557266+0000","last_became_peered":"2024-09-10T09:35:28.557266+0000","last_unstale":"2024-09-10T09:37:35.550975+0000","last_undegraded":"2024-09-10T09:37:35.550975+0000","last_fullsized":"2024-09-10T09:37:35.550975+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:51:30.946065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"96'2","reported_seq":47,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.549558+0000","last_change":"2024-09-10T09:36:06.770298+0000","last_active":"2024-09-10T09:37:35.549558+0000","last_peered":"2024-09-10T09:37:35.549558+0000","last_clean":"2024-09-10T09:37:35.549558+0000","last_became_active":"2024-09-10T09:36:06.769848+0000","last_became_peered":"2024-09-10T09:36:06.769848+0000","last_unstale":"2024-09-10T09:37:35.549558+0000","last_undegraded":"2024-09-10T09:37:35.549558+0000","last_fullsized":"2024-09-10T09:37:35.549558+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:00:30.470887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,8,2],"acting":[3,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"96'6","reported_seq":45,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.549553+0000","last_change":"2024-09-10T09:35:28.578158+0000","last_active":"2024-09-10T09:37:35.549553+0000","last_peered":"2024-09-10T09:37:35.549553+0000","last_clean":"2024-09-10T09:37:35.549553+0000","last_became_active":"2024-09-10T09:35:28.577824+0000","last_became_peered":"2024-09-10T09:35:28.577824+0000","last_unstale":"2024-09-10T09:37:35.549553+0000","last_undegraded":"2024-09-10T09:37:35.549553+0000","last_fullsized":"2024-09-10T09:37:35.549553+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:40:27.231164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"96'8","reported_seq":47,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.550143+0000","last_change":"2024-09-10T09:35:28.562670+0000","last_active":"2024-09-10T09:37:35.550143+0000","last_peered":"2024-09-10T09:37:35.550143+0000","last_clean":"2024-09-10T09:37:35.550143+0000","last_became_active":"2024-09-10T09:35:28.562311+0000","last_became_peered":"2024-09-10T09:35:28.562311+0000","last_unstale":"2024-09-10T09:37:35.550143+0000","last_undegraded":"2024-09-10T09:37:35.550143+0000","last_fullsized":"2024-09-10T09:37:35.550143+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:07:11.716142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"96'10","reported_seq":49,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.556330+0000","last_change":"2024-09-10T09:35:28.561124+0000","last_active":"2024-09-10T09:37:35.556330+0000","last_peered":"2024-09-10T09:37:35.556330+0000","last_clean":"2024-09-10T09:37:35.556330+0000","last_became_active":"2024-09-10T09:35:28.560777+0000","last_became_peered":"2024-09-10T09:35:28.560777+0000","last_unstale":"2024-09-10T09:37:35.556330+0000","last_undegraded":"2024-09-10T09:37:35.556330+0000","last_fullsized":"2024-09-10T09:37:35.556330+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:47:04.837596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"96'11","reported_seq":50,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.556539+0000","last_change":"2024-09-10T09:35:28.576790+0000","last_active":"2024-09-10T09:37:35.556539+0000","last_peered":"2024-09-10T09:37:35.556539+0000","last_clean":"2024-09-10T09:37:35.556539+0000","last_became_active":"2024-09-10T09:35:28.576464+0000","last_became_peered":"2024-09-10T09:35:28.576464+0000","last_unstale":"2024-09-10T09:37:35.556539+0000","last_undegraded":"2024-09-10T09:37:35.556539+0000","last_fullsized":"2024-09-10T09:37:35.556539+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:40:53.517577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"96'8","reported_seq":47,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.539417+0000","last_change":"2024-09-10T09:35:28.569935+0000","last_active":"2024-09-10T09:37:35.539417+0000","last_peered":"2024-09-10T09:37:35.539417+0000","last_clean":"2024-09-10T09:37:35.539417+0000","last_became_active":"2024-09-10T09:35:28.569539+0000","last_became_peered":"2024-09-10T09:35:28.569539+0000","last_unstale":"2024-09-10T09:37:35.539417+0000","last_undegraded":"2024-09-10T09:37:35.539417+0000","last_fullsized":"2024-09-10T09:37:35.539417+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:40:45.016269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"96'7","reported_seq":44,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.550145+0000","last_change":"2024-09-10T09:35:28.571619+0000","last_active":"2024-09-10T09:37:35.550145+0000","last_peered":"2024-09-10T09:37:35.550145+0000","last_clean":"2024-09-10T09:37:35.550145+0000","last_became_active":"2024-09-10T09:35:28.568591+0000","last_became_peered":"2024-09-10T09:35:28.568591+0000","last_unstale":"2024-09-10T09:37:35.550145+0000","last_undegraded":"2024-09-10T09:37:35.550145+0000","last_fullsized":"2024-09-10T09:37:35.550145+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:07:49.959712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"96'10","reported_seq":47,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.551796+0000","last_change":"2024-09-10T09:35:28.573336+0000","last_active":"2024-09-10T09:37:35.551796+0000","last_peered":"2024-09-10T09:37:35.551796+0000","last_clean":"2024-09-10T09:37:35.551796+0000","last_became_active":"2024-09-10T09:35:28.572991+0000","last_became_peered":"2024-09-10T09:35:28.572991+0000","last_unstale":"2024-09-10T09:37:35.551796+0000","last_undegraded":"2024-09-10T09:37:35.551796+0000","last_fullsized":"2024-09-10T09:37:35.551796+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:30:02.905869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"96'8","reported_seq":47,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.552930+0000","last_change":"2024-09-10T09:35:28.571945+0000","last_active":"2024-09-10T09:37:35.552930+0000","last_peered":"2024-09-10T09:37:35.552930+0000","last_clean":"2024-09-10T09:37:35.552930+0000","last_became_active":"2024-09-10T09:35:28.571546+0000","last_became_peered":"2024-09-10T09:35:28.571546+0000","last_unstale":"2024-09-10T09:37:35.552930+0000","last_undegraded":"2024-09-10T09:37:35.552930+0000","last_fullsized":"2024-09-10T09:37:35.552930+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:36:47.946864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"96'9","reported_seq":48,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.556725+0000","last_change":"2024-09-10T09:35:28.562712+0000","last_active":"2024-09-10T09:37:35.556725+0000","last_peered":"2024-09-10T09:37:35.556725+0000","last_clean":"2024-09-10T09:37:35.556725+0000","last_became_active":"2024-09-10T09:35:28.562396+0000","last_became_peered":"2024-09-10T09:35:28.562396+0000","last_unstale":"2024-09-10T09:37:35.556725+0000","last_undegraded":"2024-09-10T09:37:35.556725+0000","last_fullsized":"2024-09-10T09:37:35.556725+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:25:47.419162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"96'7","reported_seq":44,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.553427+0000","last_change":"2024-09-10T09:35:28.566354+0000","last_active":"2024-09-10T09:37:35.553427+0000","last_peered":"2024-09-10T09:37:35.553427+0000","last_clean":"2024-09-10T09:37:35.553427+0000","last_became_active":"2024-09-10T09:35:28.565248+0000","last_became_peered":"2024-09-10T09:35:28.565248+0000","last_unstale":"2024-09-10T09:37:35.553427+0000","last_undegraded":"2024-09-10T09:37:35.553427+0000","last_fullsized":"2024-09-10T09:37:35.553427+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:48:46.741010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"96'7","reported_seq":44,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.550936+0000","last_change":"2024-09-10T09:35:28.563331+0000","last_active":"2024-09-10T09:37:35.550936+0000","last_peered":"2024-09-10T09:37:35.550936+0000","last_clean":"2024-09-10T09:37:35.550936+0000","last_became_active":"2024-09-10T09:35:28.563020+0000","last_became_peered":"2024-09-10T09:35:28.563020+0000","last_unstale":"2024-09-10T09:37:35.550936+0000","last_undegraded":"2024-09-10T09:37:35.550936+0000","last_fullsized":"2024-09-10T09:37:35.550936+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:27.653551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"96'10","reported_seq":49,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.542747+0000","last_change":"2024-09-10T09:35:28.566687+0000","last_active":"2024-09-10T09:37:35.542747+0000","last_peered":"2024-09-10T09:37:35.542747+0000","last_clean":"2024-09-10T09:37:35.542747+0000","last_became_active":"2024-09-10T09:35:28.566253+0000","last_became_peered":"2024-09-10T09:35:28.566253+0000","last_unstale":"2024-09-10T09:37:35.542747+0000","last_undegraded":"2024-09-10T09:37:35.542747+0000","last_fullsized":"2024-09-10T09:37:35.542747+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:00:15.709725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"96'7","reported_seq":44,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.538923+0000","last_change":"2024-09-10T09:35:28.562251+0000","last_active":"2024-09-10T09:37:35.538923+0000","last_peered":"2024-09-10T09:37:35.538923+0000","last_clean":"2024-09-10T09:37:35.538923+0000","last_became_active":"2024-09-10T09:35:28.561933+0000","last_became_peered":"2024-09-10T09:35:28.561933+0000","last_unstale":"2024-09-10T09:37:35.538923+0000","last_undegraded":"2024-09-10T09:37:35.538923+0000","last_fullsized":"2024-09-10T09:37:35.538923+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:30:41.447954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"96'6","reported_seq":43,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.556126+0000","last_change":"2024-09-10T09:35:28.572164+0000","last_active":"2024-09-10T09:37:35.556126+0000","last_peered":"2024-09-10T09:37:35.556126+0000","last_clean":"2024-09-10T09:37:35.556126+0000","last_became_active":"2024-09-10T09:35:28.571825+0000","last_became_peered":"2024-09-10T09:35:28.571825+0000","last_unstale":"2024-09-10T09:37:35.556126+0000","last_undegraded":"2024-09-10T09:37:35.556126+0000","last_fullsized":"2024-09-10T09:37:35.556126+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:59:50.967269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"96'16","reported_seq":55,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.550138+0000","last_change":"2024-09-10T09:35:28.568098+0000","last_active":"2024-09-10T09:37:35.550138+0000","last_peered":"2024-09-10T09:37:35.550138+0000","last_clean":"2024-09-10T09:37:35.550138+0000","last_became_active":"2024-09-10T09:35:28.567765+0000","last_became_peered":"2024-09-10T09:35:28.567765+0000","last_unstale":"2024-09-10T09:37:35.550138+0000","last_undegraded":"2024-09-10T09:37:35.550138+0000","last_fullsized":"2024-09-10T09:37:35.550138+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:55:57.260320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"96'11","reported_seq":50,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.553598+0000","last_change":"2024-09-10T09:35:28.564123+0000","last_active":"2024-09-10T09:37:35.553598+0000","last_peered":"2024-09-10T09:37:35.553598+0000","last_clean":"2024-09-10T09:37:35.553598+0000","last_became_active":"2024-09-10T09:35:28.563814+0000","last_became_peered":"2024-09-10T09:35:28.563814+0000","last_unstale":"2024-09-10T09:37:35.553598+0000","last_undegraded":"2024-09-10T09:37:35.553598+0000","last_fullsized":"2024-09-10T09:37:35.553598+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:14:17.912087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"96'9","reported_seq":48,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.549407+0000","last_change":"2024-09-10T09:35:28.568601+0000","last_active":"2024-09-10T09:37:35.549407+0000","last_peered":"2024-09-10T09:37:35.549407+0000","last_clean":"2024-09-10T09:37:35.549407+0000","last_became_active":"2024-09-10T09:35:28.568303+0000","last_became_peered":"2024-09-10T09:35:28.568303+0000","last_unstale":"2024-09-10T09:37:35.549407+0000","last_undegraded":"2024-09-10T09:37:35.549407+0000","last_fullsized":"2024-09-10T09:37:35.549407+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:30:01.016420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"96'5","reported_seq":48,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.556713+0000","last_change":"2024-09-10T09:37:07.539577+0000","last_active":"2024-09-10T09:37:35.556713+0000","last_peered":"2024-09-10T09:37:35.556713+0000","last_clean":"2024-09-10T09:37:35.556713+0000","last_became_active":"2024-09-10T09:37:07.539206+0000","last_became_peered":"2024-09-10T09:37:07.539206+0000","last_unstale":"2024-09-10T09:37:35.556713+0000","last_undegraded":"2024-09-10T09:37:35.556713+0000","last_fullsized":"2024-09-10T09:37:35.556713+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:56:06.568226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"96'6","reported_seq":45,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.541193+0000","last_change":"2024-09-10T09:35:28.569225+0000","last_active":"2024-09-10T09:37:35.541193+0000","last_peered":"2024-09-10T09:37:35.541193+0000","last_clean":"2024-09-10T09:37:35.541193+0000","last_became_active":"2024-09-10T09:35:28.568891+0000","last_became_peered":"2024-09-10T09:35:28.568891+0000","last_unstale":"2024-09-10T09:37:35.541193+0000","last_undegraded":"2024-09-10T09:37:35.541193+0000","last_fullsized":"2024-09-10T09:37:35.541193+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:33:45.811781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"96'4","reported_seq":43,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.544182+0000","last_change":"2024-09-10T09:35:28.578362+0000","last_active":"2024-09-10T09:37:35.544182+0000","last_peered":"2024-09-10T09:37:35.544182+0000","last_clean":"2024-09-10T09:37:35.544182+0000","last_became_active":"2024-09-10T09:35:28.577867+0000","last_became_peered":"2024-09-10T09:35:28.577867+0000","last_unstale":"2024-09-10T09:37:35.544182+0000","last_undegraded":"2024-09-10T09:37:35.544182+0000","last_fullsized":"2024-09-10T09:37:35.544182+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:58:17.041663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"96'8","reported_seq":45,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.537018+0000","last_change":"2024-09-10T09:35:28.568183+0000","last_active":"2024-09-10T09:37:35.537018+0000","last_peered":"2024-09-10T09:37:35.537018+0000","last_clean":"2024-09-10T09:37:35.537018+0000","last_became_active":"2024-09-10T09:35:28.566775+0000","last_became_peered":"2024-09-10T09:35:28.566775+0000","last_unstale":"2024-09-10T09:37:35.537018+0000","last_undegraded":"2024-09-10T09:37:35.537018+0000","last_fullsized":"2024-09-10T09:37:35.537018+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:28:10.008889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"96'7","reported_seq":44,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.553233+0000","last_change":"2024-09-10T09:35:28.576122+0000","last_active":"2024-09-10T09:37:35.553233+0000","last_peered":"2024-09-10T09:37:35.553233+0000","last_clean":"2024-09-10T09:37:35.553233+0000","last_became_active":"2024-09-10T09:35:28.575831+0000","last_became_peered":"2024-09-10T09:35:28.575831+0000","last_unstale":"2024-09-10T09:37:35.553233+0000","last_undegraded":"2024-09-10T09:37:35.553233+0000","last_fullsized":"2024-09-10T09:37:35.553233+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:37:51.604658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"96'4","reported_seq":41,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.554017+0000","last_change":"2024-09-10T09:35:28.564441+0000","last_active":"2024-09-10T09:37:35.554017+0000","last_peered":"2024-09-10T09:37:35.554017+0000","last_clean":"2024-09-10T09:37:35.554017+0000","last_became_active":"2024-09-10T09:35:28.564095+0000","last_became_peered":"2024-09-10T09:35:28.564095+0000","last_unstale":"2024-09-10T09:37:35.554017+0000","last_undegraded":"2024-09-10T09:37:35.554017+0000","last_fullsized":"2024-09-10T09:37:35.554017+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:53:28.275361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"96'8","reported_seq":47,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.553155+0000","last_change":"2024-09-10T09:35:28.581921+0000","last_active":"2024-09-10T09:37:35.553155+0000","last_peered":"2024-09-10T09:37:35.553155+0000","last_clean":"2024-09-10T09:37:35.553155+0000","last_became_active":"2024-09-10T09:35:28.581573+0000","last_became_peered":"2024-09-10T09:35:28.581573+0000","last_unstale":"2024-09-10T09:37:35.553155+0000","last_undegraded":"2024-09-10T09:37:35.553155+0000","last_fullsized":"2024-09-10T09:37:35.553155+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:58:38.983934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":70670,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1244,"num_read_kb":703,"num_write":260,"num_write_kb":150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":1015,"num_keys_recovered":7,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":589824,"data_stored":225188,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":223,"ondisk_log_size":223,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":285627,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":77,"seq":330712481840,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32228,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712924,"statfs":{"total":95995035648,"available":95962034176,"internally_reserved":0,"allocated":1069056,"data_stored":554290,"data_compressed":32456,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":70,"seq":300647710772,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28220,"kb_used_data":1196,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716932,"statfs":{"total":95995035648,"available":95966138368,"internally_reserved":0,"allocated":1224704,"data_stored":1208382,"data_compressed":68777,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":65,"seq":279172874299,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32252,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712900,"statfs":{"total":95995035648,"available":95962009600,"internally_reserved":0,"allocated":1093632,"data_stored":554220,"data_compressed":32456,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1599,"internal_metadata":31914433},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":57,"seq":244813135936,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32248,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712904,"statfs":{"total":95995035648,"available":95962013696,"internally_reserved":0,"allocated":1089536,"data_stored":554598,"data_compressed":32456,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":50,"seq":214748364869,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28140,"kb_used_data":1116,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717012,"statfs":{"total":95995035648,"available":95966220288,"internally_reserved":0,"allocated":1142784,"data_stored":1151772,"data_compressed":68777,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":45,"seq":193273528395,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36388,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708764,"statfs":{"total":95995035648,"available":95957774336,"internally_reserved":0,"allocated":1069056,"data_stored":550569,"data_compressed":32456,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790033,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28148,"kb_used_data":1124,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717004,"statfs":{"total":95995035648,"available":95966212096,"internally_reserved":0,"allocated":1150976,"data_stored":1146820,"data_compressed":68777,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953559,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32244,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712908,"statfs":{"total":95995035648,"available":95962017792,"internally_reserved":0,"allocated":1085440,"data_stored":554954,"data_compressed":32456,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182491,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28080,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717072,"statfs":{"total":95995035648,"available":95966281728,"internally_reserved":0,"allocated":1081344,"data_stored":549432,"data_compressed":32456,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411424,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28152,"kb_used_data":1128,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717000,"statfs":{"total":95995035648,"available":95966208000,"internally_reserved":0,"allocated":1155072,"data_stored":618550,"data_compressed":32456,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574951,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36460,"kb_used_data":1116,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708692,"statfs":{"total":95995035648,"available":95957700608,"internally_reserved":0,"allocated":1142784,"data_stored":619797,"data_compressed":32456,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738475,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36412,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708740,"statfs":{"total":95995035648,"available":95957749760,"internally_reserved":0,"allocated":1093632,"data_stored":551180,"data_compressed":32456,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34674,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34674,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34674,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36321,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36321,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36321,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36321,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36321,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":73315,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":69548,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3354,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1567,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4210,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":118784,"data_stored":64916,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T09:38:20.672 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T09:38:20.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:20.863+0000 7f55a3000640 1 -- 172.21.15.5:0/1703791256 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f559c074bd0 msgr2=0x7f559c075030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:38:20.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:20.863+0000 7f55a3000640 1 --2- 172.21.15.5:0/1703791256 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f559c074bd0 0x7f559c075030 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f5588009900 tx=0x7f558802a480 comp rx=0 tx=0).stop 2024-09-10T09:38:20.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:20.863+0000 7f55a3000640 1 -- 172.21.15.5:0/1703791256 shutdown_connections 2024-09-10T09:38:20.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:20.863+0000 7f55a3000640 1 --2- 172.21.15.5:0/1703791256 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f559c109820 0x7f559c10bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:20.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:20.863+0000 7f55a3000640 1 --2- 172.21.15.5:0/1703791256 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f559c074bd0 0x7f559c075030 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:20.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:20.863+0000 7f55a3000640 1 --2- 172.21.15.5:0/1703791256 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f559c0761b0 0x7f559c074600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:20.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:20.863+0000 7f55a3000640 1 -- 172.21.15.5:0/1703791256 >> 172.21.15.5:0/1703791256 conn(0x7f559c0fd110 msgr2=0x7f559c0ff530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:38:20.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:20.863+0000 7f55a3000640 1 -- 172.21.15.5:0/1703791256 shutdown_connections 2024-09-10T09:38:20.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:20.863+0000 7f55a3000640 1 -- 172.21.15.5:0/1703791256 wait complete. 2024-09-10T09:38:20.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:20.864+0000 7f55a3000640 1 Processor -- start 2024-09-10T09:38:20.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:20.864+0000 7f55a3000640 1 -- start start 2024-09-10T09:38:20.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:20.864+0000 7f55a3000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f559c074bd0 0x7f559c199eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:20.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:20.864+0000 7f55a3000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f559c0761b0 0x7f559c19a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:20.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:20.865+0000 7f55a3000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f559c109820 0x7f559c1a1460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:20.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:20.865+0000 7f55a3000640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f559c10f820 con 0x7f559c074bd0 2024-09-10T09:38:20.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:20.865+0000 7f55a3000640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f559c10f6a0 con 0x7f559c0761b0 2024-09-10T09:38:20.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:20.865+0000 7f55a3000640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f559c10f9a0 con 0x7f559c109820 2024-09-10T09:38:20.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:20.865+0000 7f55a1c00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f559c074bd0 0x7f559c199eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:20.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:20.865+0000 7f55a1c00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f559c074bd0 0x7f559c199eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60766/0 (socket says 172.21.15.5:60766) 2024-09-10T09:38:20.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:20.865+0000 7f55a1c00640 1 -- 172.21.15.5:0/2930211006 learned_addr learned my addr 172.21.15.5:0/2930211006 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:38:20.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:20.865+0000 7f55a1200640 1 --2- 172.21.15.5:0/2930211006 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f559c0761b0 0x7f559c19a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:20.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:20.865+0000 7f55a2600640 1 --2- 172.21.15.5:0/2930211006 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f559c109820 0x7f559c1a1460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:20.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:20.865+0000 7f55a1c00640 1 -- 172.21.15.5:0/2930211006 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f559c109820 msgr2=0x7f559c1a1460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:38:20.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:20.865+0000 7f55a1c00640 1 --2- 172.21.15.5:0/2930211006 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f559c109820 0x7f559c1a1460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:20.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:20.865+0000 7f55a1c00640 1 -- 172.21.15.5:0/2930211006 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f559c0761b0 msgr2=0x7f559c19a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:38:20.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:20.865+0000 7f55a1c00640 1 --2- 172.21.15.5:0/2930211006 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f559c0761b0 0x7f559c19a3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:20.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:20.865+0000 7f55a1c00640 1 -- 172.21.15.5:0/2930211006 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f559c1a1ad0 con 0x7f559c074bd0 2024-09-10T09:38:20.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:20.865+0000 7f55a1c00640 1 --2- 172.21.15.5:0/2930211006 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f559c074bd0 0x7f559c199eb0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f559000e9f0 tx=0x7f559000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:38:20.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:20.866+0000 7f559a000640 1 -- 172.21.15.5:0/2930211006 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f559000cd80 con 0x7f559c074bd0 2024-09-10T09:38:20.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:20.866+0000 7f559a000640 1 -- 172.21.15.5:0/2930211006 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5590010910 con 0x7f559c074bd0 2024-09-10T09:38:20.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:20.866+0000 7f559a000640 1 -- 172.21.15.5:0/2930211006 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5590018990 con 0x7f559c074bd0 2024-09-10T09:38:20.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:20.866+0000 7f55a3000640 1 -- 172.21.15.5:0/2930211006 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f559c1a1dc0 con 0x7f559c074bd0 2024-09-10T09:38:20.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:20.866+0000 7f55a3000640 1 -- 172.21.15.5:0/2930211006 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f559c1a21d0 con 0x7f559c074bd0 2024-09-10T09:38:20.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:20.867+0000 7f55a3000640 1 -- 172.21.15.5:0/2930211006 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f559c1085e0 con 0x7f559c074bd0 2024-09-10T09:38:20.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:20.869+0000 7f559a000640 1 -- 172.21.15.5:0/2930211006 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 19) ==== 137883+0+0 (secure 0 0 0) 0x7f5590010ac0 con 0x7f559c074bd0 2024-09-10T09:38:20.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:20.870+0000 7f559a000640 1 --2- 172.21.15.5:0/2930211006 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f55840a2ca0 0x7f55840a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:20.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:20.870+0000 7f559a000640 1 -- 172.21.15.5:0/2930211006 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f55900cf890 con 0x7f559c074bd0 2024-09-10T09:38:20.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:20.871+0000 7f55a1200640 1 --2- 172.21.15.5:0/2930211006 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f55840a2ca0 0x7f55840a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:20.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:20.871+0000 7f55a1200640 1 --2- 172.21.15.5:0/2930211006 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f55840a2ca0 0x7f55840a5160 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f5588031440 tx=0x7f55880315b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:38:20.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:20.874+0000 7f559a000640 1 -- 172.21.15.5:0/2930211006 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f559009bbf0 con 0x7f559c074bd0 2024-09-10T09:38:21.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:38:20 smithi005 ceph-mon[30350]: pgmap v393: 129 pgs: 129 active+clean; 646 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 85 B/s wr, 9 op/s 2024-09-10T09:38:21.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:21.057+0000 7f55a3000640 1 -- 172.21.15.5:0/2930211006 --> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f559c062630 con 0x7f55840a2ca0 2024-09-10T09:38:21.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:21.066+0000 7f559a000640 1 -- 172.21.15.5:0/2930211006 <== mgr.14150 v2:172.21.15.5:6800/2707348081 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340822 (secure 0 0 0) 0x7f559c062630 con 0x7f55840a2ca0 2024-09-10T09:38:21.071 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-10T09:38:21.073 INFO:teuthology.orchestra.run.smithi005.stderr:dumped all 2024-09-10T09:38:21.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:21.070+0000 7f55a3000640 1 -- 172.21.15.5:0/2930211006 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f55840a2ca0 msgr2=0x7f55840a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:38:21.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:21.070+0000 7f55a3000640 1 --2- 172.21.15.5:0/2930211006 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f55840a2ca0 0x7f55840a5160 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f5588031440 tx=0x7f55880315b0 comp rx=0 tx=0).stop 2024-09-10T09:38:21.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:21.070+0000 7f55a3000640 1 -- 172.21.15.5:0/2930211006 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f559c074bd0 msgr2=0x7f559c199eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:38:21.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:21.070+0000 7f55a3000640 1 --2- 172.21.15.5:0/2930211006 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f559c074bd0 0x7f559c199eb0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f559000e9f0 tx=0x7f559000eeb0 comp rx=0 tx=0).stop 2024-09-10T09:38:21.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:21.070+0000 7f55a3000640 1 -- 172.21.15.5:0/2930211006 shutdown_connections 2024-09-10T09:38:21.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:21.070+0000 7f55a3000640 1 --2- 172.21.15.5:0/2930211006 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f55840a2ca0 0x7f55840a5160 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:21.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:21.070+0000 7f55a3000640 1 --2- 172.21.15.5:0/2930211006 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f559c109820 0x7f559c1a1460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:21.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:21.070+0000 7f55a3000640 1 --2- 172.21.15.5:0/2930211006 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f559c0761b0 0x7f559c19a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:21.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:21.070+0000 7f55a3000640 1 --2- 172.21.15.5:0/2930211006 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f559c074bd0 0x7f559c199eb0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:21.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:21.070+0000 7f55a3000640 1 -- 172.21.15.5:0/2930211006 >> 172.21.15.5:0/2930211006 conn(0x7f559c0fd110 msgr2=0x7f559c1057b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:38:21.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:21.070+0000 7f55a3000640 1 -- 172.21.15.5:0/2930211006 shutdown_connections 2024-09-10T09:38:21.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:21.070+0000 7f55a3000640 1 -- 172.21.15.5:0/2930211006 wait complete. 2024-09-10T09:38:21.089 INFO:teuthology.orchestra.run.smithi005.stdout:{"pg_ready":true,"pg_map":{"version":393,"stamp":"2024-09-10T09:38:19.570873+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":661272,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1370,"num_read_kb":812,"num_write":443,"num_write_kb":3090,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":1181751,"num_keys_recovered":7,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":862,"ondisk_log_size":862,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":378972,"kb_used_data":13084,"kb_used_omap":18,"kb_used_meta":365677,"kb_avail":1124562852,"statfs":{"total":1151940427776,"available":1151552360448,"internally_reserved":0,"allocated":13398016,"data_stored":8614564,"data_compressed":498435,"data_compressed_allocated":675840,"data_compressed_original":2850816,"omap_allocated":19141,"internal_metadata":374453563},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":110,"num_read_kb":55,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003036"},"pg_stats":[{"pgid":"2.1f","version":"96'14","reported_seq":179,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:38:13.661865+0000","last_change":"2024-09-10T09:35:25.533085+0000","last_active":"2024-09-10T09:38:13.661865+0000","last_peered":"2024-09-10T09:38:13.661865+0000","last_clean":"2024-09-10T09:38:13.661865+0000","last_became_active":"2024-09-10T09:35:25.532729+0000","last_became_peered":"2024-09-10T09:35:25.532729+0000","last_unstale":"2024-09-10T09:38:13.661865+0000","last_undegraded":"2024-09-10T09:38:13.661865+0000","last_fullsized":"2024-09-10T09:38:13.661865+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:55:11.158279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":121,"num_read_kb":121,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"96'5","reported_seq":21,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.545575+0000","last_change":"2024-09-10T09:36:06.767356+0000","last_active":"2024-09-10T09:37:35.545575+0000","last_peered":"2024-09-10T09:37:35.545575+0000","last_clean":"2024-09-10T09:37:35.545575+0000","last_became_active":"2024-09-10T09:36:06.766931+0000","last_became_peered":"2024-09-10T09:36:06.766931+0000","last_unstale":"2024-09-10T09:37:35.545575+0000","last_undegraded":"2024-09-10T09:37:35.545575+0000","last_fullsized":"2024-09-10T09:37:35.545575+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:25:02.777327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,7],"acting":[9,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.527523+0000","last_change":"2024-09-10T09:35:36.629797+0000","last_active":"2024-09-10T09:37:07.527523+0000","last_peered":"2024-09-10T09:37:07.527523+0000","last_clean":"2024-09-10T09:37:07.527523+0000","last_became_active":"2024-09-10T09:35:36.629414+0000","last_became_peered":"2024-09-10T09:35:36.629414+0000","last_unstale":"2024-09-10T09:37:07.527523+0000","last_undegraded":"2024-09-10T09:37:07.527523+0000","last_fullsized":"2024-09-10T09:37:07.527523+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:42:44.610581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"94'2","reported_seq":47,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.524964+0000","last_change":"2024-09-10T09:35:25.852022+0000","last_active":"2024-09-10T09:37:07.524964+0000","last_peered":"2024-09-10T09:37:07.524964+0000","last_clean":"2024-09-10T09:37:07.524964+0000","last_became_active":"2024-09-10T09:35:25.851587+0000","last_became_peered":"2024-09-10T09:35:25.851587+0000","last_unstale":"2024-09-10T09:37:07.524964+0000","last_undegraded":"2024-09-10T09:37:07.524964+0000","last_fullsized":"2024-09-10T09:37:07.524964+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:37:26.085914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"96'6","reported_seq":43,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.556261+0000","last_change":"2024-09-10T09:35:28.579260+0000","last_active":"2024-09-10T09:37:35.556261+0000","last_peered":"2024-09-10T09:37:35.556261+0000","last_clean":"2024-09-10T09:37:35.556261+0000","last_became_active":"2024-09-10T09:35:28.574584+0000","last_became_peered":"2024-09-10T09:35:28.574584+0000","last_unstale":"2024-09-10T09:37:35.556261+0000","last_undegraded":"2024-09-10T09:37:35.556261+0000","last_fullsized":"2024-09-10T09:37:35.556261+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:31:19.277689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.568359+0000","last_change":"2024-09-10T09:36:06.874861+0000","last_active":"2024-09-10T09:37:07.568359+0000","last_peered":"2024-09-10T09:37:07.568359+0000","last_clean":"2024-09-10T09:37:07.568359+0000","last_became_active":"2024-09-10T09:36:06.873930+0000","last_became_peered":"2024-09-10T09:36:06.873930+0000","last_unstale":"2024-09-10T09:37:07.568359+0000","last_undegraded":"2024-09-10T09:37:07.568359+0000","last_fullsized":"2024-09-10T09:37:07.568359+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:35:01.354061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,8,3],"acting":[4,6,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"96'13","reported_seq":58,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:59.271922+0000","last_change":"2024-09-10T09:35:25.534571+0000","last_active":"2024-09-10T09:37:59.271922+0000","last_peered":"2024-09-10T09:37:59.271922+0000","last_clean":"2024-09-10T09:37:59.271922+0000","last_became_active":"2024-09-10T09:35:25.534235+0000","last_became_peered":"2024-09-10T09:35:25.534235+0000","last_unstale":"2024-09-10T09:37:59.271922+0000","last_undegraded":"2024-09-10T09:37:59.271922+0000","last_fullsized":"2024-09-10T09:37:59.271922+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:50:21.447542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"96'9","reported_seq":48,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.553039+0000","last_change":"2024-09-10T09:35:28.565541+0000","last_active":"2024-09-10T09:37:35.553039+0000","last_peered":"2024-09-10T09:37:35.553039+0000","last_clean":"2024-09-10T09:37:35.553039+0000","last_became_active":"2024-09-10T09:35:28.565095+0000","last_became_peered":"2024-09-10T09:35:28.565095+0000","last_unstale":"2024-09-10T09:37:35.553039+0000","last_undegraded":"2024-09-10T09:37:35.553039+0000","last_fullsized":"2024-09-10T09:37:35.553039+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:29:31.285401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.568600+0000","last_change":"2024-09-10T09:35:36.629277+0000","last_active":"2024-09-10T09:37:07.568600+0000","last_peered":"2024-09-10T09:37:07.568600+0000","last_clean":"2024-09-10T09:37:07.568600+0000","last_became_active":"2024-09-10T09:35:36.628687+0000","last_became_peered":"2024-09-10T09:35:36.628687+0000","last_unstale":"2024-09-10T09:37:07.568600+0000","last_undegraded":"2024-09-10T09:37:07.568600+0000","last_fullsized":"2024-09-10T09:37:07.568600+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:34:33.628827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"94'5","reported_seq":50,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.530161+0000","last_change":"2024-09-10T09:35:25.536137+0000","last_active":"2024-09-10T09:37:07.530161+0000","last_peered":"2024-09-10T09:37:07.530161+0000","last_clean":"2024-09-10T09:37:07.530161+0000","last_became_active":"2024-09-10T09:35:25.535728+0000","last_became_peered":"2024-09-10T09:35:25.535728+0000","last_unstale":"2024-09-10T09:37:07.530161+0000","last_undegraded":"2024-09-10T09:37:07.530161+0000","last_fullsized":"2024-09-10T09:37:07.530161+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:36:04.829901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"96'15","reported_seq":60,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.547598+0000","last_change":"2024-09-10T09:36:06.874425+0000","last_active":"2024-09-10T09:37:35.547598+0000","last_peered":"2024-09-10T09:37:35.547598+0000","last_clean":"2024-09-10T09:37:35.547598+0000","last_became_active":"2024-09-10T09:36:06.874050+0000","last_became_peered":"2024-09-10T09:36:06.874050+0000","last_unstale":"2024-09-10T09:37:35.547598+0000","last_undegraded":"2024-09-10T09:37:35.547598+0000","last_fullsized":"2024-09-10T09:37:35.547598+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:30:51.200335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,1],"acting":[5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.568073+0000","last_change":"2024-09-10T09:35:36.615314+0000","last_active":"2024-09-10T09:37:07.568073+0000","last_peered":"2024-09-10T09:37:07.568073+0000","last_clean":"2024-09-10T09:37:07.568073+0000","last_became_active":"2024-09-10T09:35:36.614820+0000","last_became_peered":"2024-09-10T09:35:36.614820+0000","last_unstale":"2024-09-10T09:37:07.568073+0000","last_undegraded":"2024-09-10T09:37:07.568073+0000","last_fullsized":"2024-09-10T09:37:07.568073+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:51:16.897935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"96'5","reported_seq":42,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.556084+0000","last_change":"2024-09-10T09:35:28.572346+0000","last_active":"2024-09-10T09:37:35.556084+0000","last_peered":"2024-09-10T09:37:35.556084+0000","last_clean":"2024-09-10T09:37:35.556084+0000","last_became_active":"2024-09-10T09:35:28.571839+0000","last_became_peered":"2024-09-10T09:35:28.571839+0000","last_unstale":"2024-09-10T09:37:35.556084+0000","last_undegraded":"2024-09-10T09:37:35.556084+0000","last_fullsized":"2024-09-10T09:37:35.556084+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:41:29.662441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"94'4","reported_seq":49,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.529918+0000","last_change":"2024-09-10T09:35:25.538747+0000","last_active":"2024-09-10T09:37:07.529918+0000","last_peered":"2024-09-10T09:37:07.529918+0000","last_clean":"2024-09-10T09:37:07.529918+0000","last_became_active":"2024-09-10T09:35:25.538330+0000","last_became_peered":"2024-09-10T09:35:25.538330+0000","last_unstale":"2024-09-10T09:37:07.529918+0000","last_undegraded":"2024-09-10T09:37:07.529918+0000","last_fullsized":"2024-09-10T09:37:07.529918+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:15:08.340688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.568436+0000","last_change":"2024-09-10T09:35:37.270106+0000","last_active":"2024-09-10T09:37:07.568436+0000","last_peered":"2024-09-10T09:37:07.568436+0000","last_clean":"2024-09-10T09:37:07.568436+0000","last_became_active":"2024-09-10T09:35:37.269575+0000","last_became_peered":"2024-09-10T09:35:37.269575+0000","last_unstale":"2024-09-10T09:37:07.568436+0000","last_undegraded":"2024-09-10T09:37:07.568436+0000","last_fullsized":"2024-09-10T09:37:07.568436+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:35:00.414602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"96'8","reported_seq":47,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.555784+0000","last_change":"2024-09-10T09:35:28.569230+0000","last_active":"2024-09-10T09:37:35.555784+0000","last_peered":"2024-09-10T09:37:35.555784+0000","last_clean":"2024-09-10T09:37:35.555784+0000","last_became_active":"2024-09-10T09:35:28.568266+0000","last_became_peered":"2024-09-10T09:35:28.568266+0000","last_unstale":"2024-09-10T09:37:35.555784+0000","last_undegraded":"2024-09-10T09:37:35.555784+0000","last_fullsized":"2024-09-10T09:37:35.555784+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:49:43.730686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"94'3","reported_seq":17,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.544707+0000","last_change":"2024-09-10T09:37:07.544298+0000","last_active":"2024-09-10T09:37:07.544707+0000","last_peered":"2024-09-10T09:37:07.544707+0000","last_clean":"2024-09-10T09:37:07.544707+0000","last_became_active":"2024-09-10T09:37:07.531815+0000","last_became_peered":"2024-09-10T09:37:07.531815+0000","last_unstale":"2024-09-10T09:37:07.544707+0000","last_undegraded":"2024-09-10T09:37:07.544707+0000","last_fullsized":"2024-09-10T09:37:07.544707+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:18:10.091895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"94'1","reported_seq":37,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.568484+0000","last_change":"2024-09-10T09:36:06.875056+0000","last_active":"2024-09-10T09:37:07.568484+0000","last_peered":"2024-09-10T09:37:07.568484+0000","last_clean":"2024-09-10T09:37:07.568484+0000","last_became_active":"2024-09-10T09:36:06.874400+0000","last_became_peered":"2024-09-10T09:36:06.874400+0000","last_unstale":"2024-09-10T09:37:07.568484+0000","last_undegraded":"2024-09-10T09:37:07.568484+0000","last_fullsized":"2024-09-10T09:37:07.568484+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:21:06.300712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,7,8],"acting":[4,10,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"96'9","reported_seq":48,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.552565+0000","last_change":"2024-09-10T09:35:28.571319+0000","last_active":"2024-09-10T09:37:35.552565+0000","last_peered":"2024-09-10T09:37:35.552565+0000","last_clean":"2024-09-10T09:37:35.552565+0000","last_became_active":"2024-09-10T09:35:28.570972+0000","last_became_peered":"2024-09-10T09:35:28.570972+0000","last_unstale":"2024-09-10T09:37:35.552565+0000","last_undegraded":"2024-09-10T09:37:35.552565+0000","last_fullsized":"2024-09-10T09:37:35.552565+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:45:19.300535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"94'7","reported_seq":52,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.524483+0000","last_change":"2024-09-10T09:35:25.530489+0000","last_active":"2024-09-10T09:37:07.524483+0000","last_peered":"2024-09-10T09:37:07.524483+0000","last_clean":"2024-09-10T09:37:07.524483+0000","last_became_active":"2024-09-10T09:35:25.530197+0000","last_became_peered":"2024-09-10T09:35:25.530197+0000","last_unstale":"2024-09-10T09:37:07.524483+0000","last_undegraded":"2024-09-10T09:37:07.524483+0000","last_fullsized":"2024-09-10T09:37:07.524483+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:28:57.539569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.527650+0000","last_change":"2024-09-10T09:35:36.630507+0000","last_active":"2024-09-10T09:37:07.527650+0000","last_peered":"2024-09-10T09:37:07.527650+0000","last_clean":"2024-09-10T09:37:07.527650+0000","last_became_active":"2024-09-10T09:35:36.630052+0000","last_became_peered":"2024-09-10T09:35:36.630052+0000","last_unstale":"2024-09-10T09:37:07.527650+0000","last_undegraded":"2024-09-10T09:37:07.527650+0000","last_fullsized":"2024-09-10T09:37:07.527650+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:13:42.691382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"96'2","reported_seq":41,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.532487+0000","last_change":"2024-09-10T09:35:28.565650+0000","last_active":"2024-09-10T09:37:35.532487+0000","last_peered":"2024-09-10T09:37:35.532487+0000","last_clean":"2024-09-10T09:37:35.532487+0000","last_became_active":"2024-09-10T09:35:28.565303+0000","last_became_peered":"2024-09-10T09:35:28.565303+0000","last_unstale":"2024-09-10T09:37:35.532487+0000","last_undegraded":"2024-09-10T09:37:35.532487+0000","last_fullsized":"2024-09-10T09:37:35.532487+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:48:08.965487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"94'2","reported_seq":46,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.574507+0000","last_change":"2024-09-10T09:35:25.851326+0000","last_active":"2024-09-10T09:37:07.574507+0000","last_peered":"2024-09-10T09:37:07.574507+0000","last_clean":"2024-09-10T09:37:07.574507+0000","last_became_active":"2024-09-10T09:35:25.850981+0000","last_became_peered":"2024-09-10T09:35:25.850981+0000","last_unstale":"2024-09-10T09:37:07.574507+0000","last_undegraded":"2024-09-10T09:37:07.574507+0000","last_fullsized":"2024-09-10T09:37:07.574507+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:50:28.877840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.568510+0000","last_change":"2024-09-10T09:35:37.270138+0000","last_active":"2024-09-10T09:37:07.568510+0000","last_peered":"2024-09-10T09:37:07.568510+0000","last_clean":"2024-09-10T09:37:07.568510+0000","last_became_active":"2024-09-10T09:35:37.269663+0000","last_became_peered":"2024-09-10T09:35:37.269663+0000","last_unstale":"2024-09-10T09:37:07.568510+0000","last_undegraded":"2024-09-10T09:37:07.568510+0000","last_fullsized":"2024-09-10T09:37:07.568510+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:18:00.105970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"96'7","reported_seq":44,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.554144+0000","last_change":"2024-09-10T09:35:28.576197+0000","last_active":"2024-09-10T09:37:35.554144+0000","last_peered":"2024-09-10T09:37:35.554144+0000","last_clean":"2024-09-10T09:37:35.554144+0000","last_became_active":"2024-09-10T09:35:28.575859+0000","last_became_peered":"2024-09-10T09:35:28.575859+0000","last_unstale":"2024-09-10T09:37:35.554144+0000","last_undegraded":"2024-09-10T09:37:35.554144+0000","last_fullsized":"2024-09-10T09:37:35.554144+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:09:39.458761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"94'7","reported_seq":51,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.568805+0000","last_change":"2024-09-10T09:35:25.990587+0000","last_active":"2024-09-10T09:37:07.568805+0000","last_peered":"2024-09-10T09:37:07.568805+0000","last_clean":"2024-09-10T09:37:07.568805+0000","last_became_active":"2024-09-10T09:35:25.990302+0000","last_became_peered":"2024-09-10T09:35:25.990302+0000","last_unstale":"2024-09-10T09:37:07.568805+0000","last_undegraded":"2024-09-10T09:37:07.568805+0000","last_fullsized":"2024-09-10T09:37:07.568805+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:18:41.244441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.527768+0000","last_change":"2024-09-10T09:35:36.612983+0000","last_active":"2024-09-10T09:37:07.527768+0000","last_peered":"2024-09-10T09:37:07.527768+0000","last_clean":"2024-09-10T09:37:07.527768+0000","last_became_active":"2024-09-10T09:35:36.612542+0000","last_became_peered":"2024-09-10T09:35:36.612542+0000","last_unstale":"2024-09-10T09:37:07.527768+0000","last_undegraded":"2024-09-10T09:37:07.527768+0000","last_fullsized":"2024-09-10T09:37:07.527768+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:50:16.298195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"96'5","reported_seq":44,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.554267+0000","last_change":"2024-09-10T09:35:28.564121+0000","last_active":"2024-09-10T09:37:35.554267+0000","last_peered":"2024-09-10T09:37:35.554267+0000","last_clean":"2024-09-10T09:37:35.554267+0000","last_became_active":"2024-09-10T09:35:28.562780+0000","last_became_peered":"2024-09-10T09:35:28.562780+0000","last_unstale":"2024-09-10T09:37:35.554267+0000","last_undegraded":"2024-09-10T09:37:35.554267+0000","last_fullsized":"2024-09-10T09:37:35.554267+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:53:15.690478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"96'16","reported_seq":76,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:34.702676+0000","last_change":"2024-09-10T09:37:07.541031+0000","last_active":"2024-09-10T09:37:34.702676+0000","last_peered":"2024-09-10T09:37:34.702676+0000","last_clean":"2024-09-10T09:37:34.702676+0000","last_became_active":"2024-09-10T09:37:07.529577+0000","last_became_peered":"2024-09-10T09:37:07.529577+0000","last_unstale":"2024-09-10T09:37:34.702676+0000","last_undegraded":"2024-09-10T09:37:34.702676+0000","last_fullsized":"2024-09-10T09:37:34.702676+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:14:13.440346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":29,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":7,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.524145+0000","last_change":"2024-09-10T09:35:37.270888+0000","last_active":"2024-09-10T09:37:07.524145+0000","last_peered":"2024-09-10T09:37:07.524145+0000","last_clean":"2024-09-10T09:37:07.524145+0000","last_became_active":"2024-09-10T09:35:37.270229+0000","last_became_peered":"2024-09-10T09:35:37.270229+0000","last_unstale":"2024-09-10T09:37:07.524145+0000","last_undegraded":"2024-09-10T09:37:07.524145+0000","last_fullsized":"2024-09-10T09:37:07.524145+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:03:54.223463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"96'9","reported_seq":48,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.547904+0000","last_change":"2024-09-10T09:35:28.572799+0000","last_active":"2024-09-10T09:37:35.547904+0000","last_peered":"2024-09-10T09:37:35.547904+0000","last_clean":"2024-09-10T09:37:35.547904+0000","last_became_active":"2024-09-10T09:35:28.572310+0000","last_became_peered":"2024-09-10T09:35:28.572310+0000","last_unstale":"2024-09-10T09:37:35.547904+0000","last_undegraded":"2024-09-10T09:37:35.547904+0000","last_fullsized":"2024-09-10T09:37:35.547904+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:33:51.446766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"94'4","reported_seq":49,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.530456+0000","last_change":"2024-09-10T09:35:25.851868+0000","last_active":"2024-09-10T09:37:07.530456+0000","last_peered":"2024-09-10T09:37:07.530456+0000","last_clean":"2024-09-10T09:37:07.530456+0000","last_became_active":"2024-09-10T09:35:25.851447+0000","last_became_peered":"2024-09-10T09:35:25.851447+0000","last_unstale":"2024-09-10T09:37:07.530456+0000","last_undegraded":"2024-09-10T09:37:07.530456+0000","last_fullsized":"2024-09-10T09:37:07.530456+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:15:57.706690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.526545+0000","last_change":"2024-09-10T09:35:36.619459+0000","last_active":"2024-09-10T09:37:07.526545+0000","last_peered":"2024-09-10T09:37:07.526545+0000","last_clean":"2024-09-10T09:37:07.526545+0000","last_became_active":"2024-09-10T09:35:36.619023+0000","last_became_peered":"2024-09-10T09:35:36.619023+0000","last_unstale":"2024-09-10T09:37:07.526545+0000","last_undegraded":"2024-09-10T09:37:07.526545+0000","last_fullsized":"2024-09-10T09:37:07.526545+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:07:32.999231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"96'4","reported_seq":43,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.548010+0000","last_change":"2024-09-10T09:35:28.575668+0000","last_active":"2024-09-10T09:37:35.548010+0000","last_peered":"2024-09-10T09:37:35.548010+0000","last_clean":"2024-09-10T09:37:35.548010+0000","last_became_active":"2024-09-10T09:35:28.575425+0000","last_became_peered":"2024-09-10T09:35:28.575425+0000","last_unstale":"2024-09-10T09:37:35.548010+0000","last_undegraded":"2024-09-10T09:37:35.548010+0000","last_fullsized":"2024-09-10T09:37:35.548010+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:13:27.120208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"94'1","reported_seq":45,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.528018+0000","last_change":"2024-09-10T09:35:25.854569+0000","last_active":"2024-09-10T09:37:07.528018+0000","last_peered":"2024-09-10T09:37:07.528018+0000","last_clean":"2024-09-10T09:37:07.528018+0000","last_became_active":"2024-09-10T09:35:25.854152+0000","last_became_peered":"2024-09-10T09:35:25.854152+0000","last_unstale":"2024-09-10T09:37:07.528018+0000","last_undegraded":"2024-09-10T09:37:07.528018+0000","last_fullsized":"2024-09-10T09:37:07.528018+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:27:15.584707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.529957+0000","last_change":"2024-09-10T09:35:37.270517+0000","last_active":"2024-09-10T09:37:07.529957+0000","last_peered":"2024-09-10T09:37:07.529957+0000","last_clean":"2024-09-10T09:37:07.529957+0000","last_became_active":"2024-09-10T09:35:37.269793+0000","last_became_peered":"2024-09-10T09:35:37.269793+0000","last_unstale":"2024-09-10T09:37:07.529957+0000","last_undegraded":"2024-09-10T09:37:07.529957+0000","last_fullsized":"2024-09-10T09:37:07.529957+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:28:58.272848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"96'7","reported_seq":52,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.556473+0000","last_change":"2024-09-10T09:37:07.762585+0000","last_active":"2024-09-10T09:37:35.556473+0000","last_peered":"2024-09-10T09:37:35.556473+0000","last_clean":"2024-09-10T09:37:35.556473+0000","last_became_active":"2024-09-10T09:37:07.762250+0000","last_became_peered":"2024-09-10T09:37:07.762250+0000","last_unstale":"2024-09-10T09:37:35.556473+0000","last_undegraded":"2024-09-10T09:37:35.556473+0000","last_fullsized":"2024-09-10T09:37:35.556473+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:38:01.751512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"96'7","reported_seq":172,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:38:12.661207+0000","last_change":"2024-09-10T09:35:25.531062+0000","last_active":"2024-09-10T09:38:12.661207+0000","last_peered":"2024-09-10T09:38:12.661207+0000","last_clean":"2024-09-10T09:38:12.661207+0000","last_became_active":"2024-09-10T09:35:25.530005+0000","last_became_peered":"2024-09-10T09:35:25.530005+0000","last_unstale":"2024-09-10T09:38:12.661207+0000","last_undegraded":"2024-09-10T09:38:12.661207+0000","last_fullsized":"2024-09-10T09:38:12.661207+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:42:35.330312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1122,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":122,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.538766+0000","last_change":"2024-09-10T09:35:36.623099+0000","last_active":"2024-09-10T09:37:07.538766+0000","last_peered":"2024-09-10T09:37:07.538766+0000","last_clean":"2024-09-10T09:37:07.538766+0000","last_became_active":"2024-09-10T09:35:36.622716+0000","last_became_peered":"2024-09-10T09:35:36.622716+0000","last_unstale":"2024-09-10T09:37:07.538766+0000","last_undegraded":"2024-09-10T09:37:07.538766+0000","last_fullsized":"2024-09-10T09:37:07.538766+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:10:38.495164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"96'8","reported_seq":53,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.543805+0000","last_change":"2024-09-10T09:36:06.769405+0000","last_active":"2024-09-10T09:37:35.543805+0000","last_peered":"2024-09-10T09:37:35.543805+0000","last_clean":"2024-09-10T09:37:35.543805+0000","last_became_active":"2024-09-10T09:36:06.769042+0000","last_became_peered":"2024-09-10T09:36:06.769042+0000","last_unstale":"2024-09-10T09:37:35.543805+0000","last_undegraded":"2024-09-10T09:37:35.543805+0000","last_fullsized":"2024-09-10T09:37:35.543805+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:01:49.340765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,8],"acting":[5,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"94'5","reported_seq":174,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:38:16.666205+0000","last_change":"2024-09-10T09:35:25.851430+0000","last_active":"2024-09-10T09:38:16.666205+0000","last_peered":"2024-09-10T09:38:16.666205+0000","last_clean":"2024-09-10T09:38:16.666205+0000","last_became_active":"2024-09-10T09:35:25.851067+0000","last_became_peered":"2024-09-10T09:35:25.851067+0000","last_unstale":"2024-09-10T09:38:16.666205+0000","last_undegraded":"2024-09-10T09:38:16.666205+0000","last_fullsized":"2024-09-10T09:38:16.666205+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:55:58.756207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":125,"num_read_kb":125,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.538676+0000","last_change":"2024-09-10T09:35:37.270361+0000","last_active":"2024-09-10T09:37:07.538676+0000","last_peered":"2024-09-10T09:37:07.538676+0000","last_clean":"2024-09-10T09:37:07.538676+0000","last_became_active":"2024-09-10T09:35:37.269632+0000","last_became_peered":"2024-09-10T09:35:37.269632+0000","last_unstale":"2024-09-10T09:37:07.538676+0000","last_undegraded":"2024-09-10T09:37:07.538676+0000","last_fullsized":"2024-09-10T09:37:07.538676+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:50:34.360497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"96'5","reported_seq":44,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.548275+0000","last_change":"2024-09-10T09:35:28.571064+0000","last_active":"2024-09-10T09:37:35.548275+0000","last_peered":"2024-09-10T09:37:35.548275+0000","last_clean":"2024-09-10T09:37:35.548275+0000","last_became_active":"2024-09-10T09:35:28.570691+0000","last_became_peered":"2024-09-10T09:35:28.570691+0000","last_unstale":"2024-09-10T09:37:35.548275+0000","last_undegraded":"2024-09-10T09:37:35.548275+0000","last_fullsized":"2024-09-10T09:37:35.548275+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:58:33.162501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"94'4","reported_seq":57,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.572759+0000","last_change":"2024-09-10T09:37:07.572247+0000","last_active":"2024-09-10T09:37:07.572759+0000","last_peered":"2024-09-10T09:37:07.572759+0000","last_clean":"2024-09-10T09:37:07.572759+0000","last_became_active":"2024-09-10T09:37:07.569410+0000","last_became_peered":"2024-09-10T09:37:07.569410+0000","last_unstale":"2024-09-10T09:37:07.572759+0000","last_undegraded":"2024-09-10T09:37:07.572759+0000","last_fullsized":"2024-09-10T09:37:07.572759+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:05:09.037084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,4],"acting":[7,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.532723+0000","last_change":"2024-09-10T09:35:37.270358+0000","last_active":"2024-09-10T09:37:07.532723+0000","last_peered":"2024-09-10T09:37:07.532723+0000","last_clean":"2024-09-10T09:37:07.532723+0000","last_became_active":"2024-09-10T09:35:37.269822+0000","last_became_peered":"2024-09-10T09:35:37.269822+0000","last_unstale":"2024-09-10T09:37:07.532723+0000","last_undegraded":"2024-09-10T09:37:07.532723+0000","last_fullsized":"2024-09-10T09:37:07.532723+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:19:48.932596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"96'8","reported_seq":47,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.553803+0000","last_change":"2024-09-10T09:35:28.576244+0000","last_active":"2024-09-10T09:37:35.553803+0000","last_peered":"2024-09-10T09:37:35.553803+0000","last_clean":"2024-09-10T09:37:35.553803+0000","last_became_active":"2024-09-10T09:35:28.571801+0000","last_became_peered":"2024-09-10T09:35:28.571801+0000","last_unstale":"2024-09-10T09:37:35.553803+0000","last_undegraded":"2024-09-10T09:37:35.553803+0000","last_fullsized":"2024-09-10T09:37:35.553803+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:11:21.742220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"94'6","reported_seq":52,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.568754+0000","last_change":"2024-09-10T09:35:25.537118+0000","last_active":"2024-09-10T09:37:07.568754+0000","last_peered":"2024-09-10T09:37:07.568754+0000","last_clean":"2024-09-10T09:37:07.568754+0000","last_became_active":"2024-09-10T09:35:25.536718+0000","last_became_peered":"2024-09-10T09:35:25.536718+0000","last_unstale":"2024-09-10T09:37:07.568754+0000","last_undegraded":"2024-09-10T09:37:07.568754+0000","last_fullsized":"2024-09-10T09:37:07.568754+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:16:32.837476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":361,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.528720+0000","last_change":"2024-09-10T09:34:07.350825+0000","last_active":"2024-09-10T09:37:07.528720+0000","last_peered":"2024-09-10T09:37:07.528720+0000","last_clean":"2024-09-10T09:37:07.528720+0000","last_became_active":"2024-09-10T09:34:07.350325+0000","last_became_peered":"2024-09-10T09:34:07.350325+0000","last_unstale":"2024-09-10T09:37:07.528720+0000","last_undegraded":"2024-09-10T09:37:07.528720+0000","last_fullsized":"2024-09-10T09:37:07.528720+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:30:26.408262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:30:26.408262+0000","last_clean_scrub_stamp":"2024-09-10T09:30:26.408262+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:58:04.920443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"94'1","reported_seq":30,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.529308+0000","last_change":"2024-09-10T09:35:36.632941+0000","last_active":"2024-09-10T09:37:07.529308+0000","last_peered":"2024-09-10T09:37:07.529308+0000","last_clean":"2024-09-10T09:37:07.529308+0000","last_became_active":"2024-09-10T09:35:36.632527+0000","last_became_peered":"2024-09-10T09:35:36.632527+0000","last_unstale":"2024-09-10T09:37:07.529308+0000","last_undegraded":"2024-09-10T09:37:07.529308+0000","last_fullsized":"2024-09-10T09:37:07.529308+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:33:21.733772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"96'7","reported_seq":44,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.556036+0000","last_change":"2024-09-10T09:35:28.572010+0000","last_active":"2024-09-10T09:37:35.556036+0000","last_peered":"2024-09-10T09:37:35.556036+0000","last_clean":"2024-09-10T09:37:35.556036+0000","last_became_active":"2024-09-10T09:35:28.571623+0000","last_became_peered":"2024-09-10T09:35:28.571623+0000","last_unstale":"2024-09-10T09:37:35.556036+0000","last_undegraded":"2024-09-10T09:37:35.556036+0000","last_fullsized":"2024-09-10T09:37:35.556036+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:00:49.328663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"94'5","reported_seq":173,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:38:14.664651+0000","last_change":"2024-09-10T09:35:25.990328+0000","last_active":"2024-09-10T09:38:14.664651+0000","last_peered":"2024-09-10T09:38:14.664651+0000","last_clean":"2024-09-10T09:38:14.664651+0000","last_became_active":"2024-09-10T09:35:25.989946+0000","last_became_peered":"2024-09-10T09:35:25.989946+0000","last_unstale":"2024-09-10T09:38:14.664651+0000","last_undegraded":"2024-09-10T09:38:14.664651+0000","last_fullsized":"2024-09-10T09:38:14.664651+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:51:16.069913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":124,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.574499+0000","last_change":"2024-09-10T09:35:36.619013+0000","last_active":"2024-09-10T09:37:07.574499+0000","last_peered":"2024-09-10T09:37:07.574499+0000","last_clean":"2024-09-10T09:37:07.574499+0000","last_became_active":"2024-09-10T09:35:36.618506+0000","last_became_peered":"2024-09-10T09:35:36.618506+0000","last_unstale":"2024-09-10T09:37:07.574499+0000","last_undegraded":"2024-09-10T09:37:07.574499+0000","last_fullsized":"2024-09-10T09:37:07.574499+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:57:53.580569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"96'6","reported_seq":49,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.553955+0000","last_change":"2024-09-10T09:36:06.768573+0000","last_active":"2024-09-10T09:37:35.553955+0000","last_peered":"2024-09-10T09:37:35.553955+0000","last_clean":"2024-09-10T09:37:35.553955+0000","last_became_active":"2024-09-10T09:36:06.768228+0000","last_became_peered":"2024-09-10T09:36:06.768228+0000","last_unstale":"2024-09-10T09:37:35.553955+0000","last_undegraded":"2024-09-10T09:37:35.553955+0000","last_fullsized":"2024-09-10T09:37:35.553955+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:57:38.310713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"94'3","reported_seq":48,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.533010+0000","last_change":"2024-09-10T09:35:25.528795+0000","last_active":"2024-09-10T09:37:07.533010+0000","last_peered":"2024-09-10T09:37:07.533010+0000","last_clean":"2024-09-10T09:37:07.533010+0000","last_became_active":"2024-09-10T09:35:25.528475+0000","last_became_peered":"2024-09-10T09:35:25.528475+0000","last_unstale":"2024-09-10T09:37:07.533010+0000","last_undegraded":"2024-09-10T09:37:07.533010+0000","last_fullsized":"2024-09-10T09:37:07.533010+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:16:06.999186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.761758+0000","last_change":"2024-09-10T09:35:36.633109+0000","last_active":"2024-09-10T09:37:07.761758+0000","last_peered":"2024-09-10T09:37:07.761758+0000","last_clean":"2024-09-10T09:37:07.761758+0000","last_became_active":"2024-09-10T09:35:36.632682+0000","last_became_peered":"2024-09-10T09:35:36.632682+0000","last_unstale":"2024-09-10T09:37:07.761758+0000","last_undegraded":"2024-09-10T09:37:07.761758+0000","last_fullsized":"2024-09-10T09:37:07.761758+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:51:14.372649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"96'9","reported_seq":46,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.552249+0000","last_change":"2024-09-10T09:35:28.562103+0000","last_active":"2024-09-10T09:37:35.552249+0000","last_peered":"2024-09-10T09:37:35.552249+0000","last_clean":"2024-09-10T09:37:35.552249+0000","last_became_active":"2024-09-10T09:35:28.561765+0000","last_became_peered":"2024-09-10T09:35:28.561765+0000","last_unstale":"2024-09-10T09:37:35.552249+0000","last_undegraded":"2024-09-10T09:37:35.552249+0000","last_fullsized":"2024-09-10T09:37:35.552249+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:35:35.400569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"94'5","reported_seq":49,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.539243+0000","last_change":"2024-09-10T09:35:25.538536+0000","last_active":"2024-09-10T09:37:07.539243+0000","last_peered":"2024-09-10T09:37:07.539243+0000","last_clean":"2024-09-10T09:37:07.539243+0000","last_became_active":"2024-09-10T09:35:25.537944+0000","last_became_peered":"2024-09-10T09:35:25.537944+0000","last_unstale":"2024-09-10T09:37:07.539243+0000","last_undegraded":"2024-09-10T09:37:07.539243+0000","last_fullsized":"2024-09-10T09:37:07.539243+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:01:19.385381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":17,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.529928+0000","last_change":"2024-09-10T09:36:06.769534+0000","last_active":"2024-09-10T09:37:07.529928+0000","last_peered":"2024-09-10T09:37:07.529928+0000","last_clean":"2024-09-10T09:37:07.529928+0000","last_became_active":"2024-09-10T09:36:06.769015+0000","last_became_peered":"2024-09-10T09:36:06.769015+0000","last_unstale":"2024-09-10T09:37:07.529928+0000","last_undegraded":"2024-09-10T09:37:07.529928+0000","last_fullsized":"2024-09-10T09:37:07.529928+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:19:26.369737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7,1],"acting":[8,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"96'5","reported_seq":44,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.539962+0000","last_change":"2024-09-10T09:35:28.566735+0000","last_active":"2024-09-10T09:37:35.539962+0000","last_peered":"2024-09-10T09:37:35.539962+0000","last_clean":"2024-09-10T09:37:35.539962+0000","last_became_active":"2024-09-10T09:35:28.566400+0000","last_became_peered":"2024-09-10T09:35:28.566400+0000","last_unstale":"2024-09-10T09:37:35.539962+0000","last_undegraded":"2024-09-10T09:37:35.539962+0000","last_fullsized":"2024-09-10T09:37:35.539962+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:18:53.337816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"94'2","reported_seq":47,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.530569+0000","last_change":"2024-09-10T09:35:25.535783+0000","last_active":"2024-09-10T09:37:07.530569+0000","last_peered":"2024-09-10T09:37:07.530569+0000","last_clean":"2024-09-10T09:37:07.530569+0000","last_became_active":"2024-09-10T09:35:25.535411+0000","last_became_peered":"2024-09-10T09:35:25.535411+0000","last_unstale":"2024-09-10T09:37:07.530569+0000","last_undegraded":"2024-09-10T09:37:07.530569+0000","last_fullsized":"2024-09-10T09:37:07.530569+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:48:09.239083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.531208+0000","last_change":"2024-09-10T09:35:36.619380+0000","last_active":"2024-09-10T09:37:07.531208+0000","last_peered":"2024-09-10T09:37:07.531208+0000","last_clean":"2024-09-10T09:37:07.531208+0000","last_became_active":"2024-09-10T09:35:36.618953+0000","last_became_peered":"2024-09-10T09:35:36.618953+0000","last_unstale":"2024-09-10T09:37:07.531208+0000","last_undegraded":"2024-09-10T09:37:07.531208+0000","last_fullsized":"2024-09-10T09:37:07.531208+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:40:25.181408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"96'6","reported_seq":45,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.546928+0000","last_change":"2024-09-10T09:35:28.571396+0000","last_active":"2024-09-10T09:37:35.546928+0000","last_peered":"2024-09-10T09:37:35.546928+0000","last_clean":"2024-09-10T09:37:35.546928+0000","last_became_active":"2024-09-10T09:35:28.571063+0000","last_became_peered":"2024-09-10T09:35:28.571063+0000","last_unstale":"2024-09-10T09:37:35.546928+0000","last_undegraded":"2024-09-10T09:37:35.546928+0000","last_fullsized":"2024-09-10T09:37:35.546928+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:38:17.965133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"94'17","reported_seq":62,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.530060+0000","last_change":"2024-09-10T09:35:25.851773+0000","last_active":"2024-09-10T09:37:07.530060+0000","last_peered":"2024-09-10T09:37:07.530060+0000","last_clean":"2024-09-10T09:37:07.530060+0000","last_became_active":"2024-09-10T09:35:25.851384+0000","last_became_peered":"2024-09-10T09:35:25.851384+0000","last_unstale":"2024-09-10T09:37:07.530060+0000","last_undegraded":"2024-09-10T09:37:07.530060+0000","last_fullsized":"2024-09-10T09:37:07.530060+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:03:32.338643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.539186+0000","last_change":"2024-09-10T09:35:36.620971+0000","last_active":"2024-09-10T09:37:07.539186+0000","last_peered":"2024-09-10T09:37:07.539186+0000","last_clean":"2024-09-10T09:37:07.539186+0000","last_became_active":"2024-09-10T09:35:36.620343+0000","last_became_peered":"2024-09-10T09:35:36.620343+0000","last_unstale":"2024-09-10T09:37:07.539186+0000","last_undegraded":"2024-09-10T09:37:07.539186+0000","last_fullsized":"2024-09-10T09:37:07.539186+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:17:19.517882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"96'6","reported_seq":43,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.553818+0000","last_change":"2024-09-10T09:35:28.565643+0000","last_active":"2024-09-10T09:37:35.553818+0000","last_peered":"2024-09-10T09:37:35.553818+0000","last_clean":"2024-09-10T09:37:35.553818+0000","last_became_active":"2024-09-10T09:35:28.564217+0000","last_became_peered":"2024-09-10T09:35:28.564217+0000","last_unstale":"2024-09-10T09:37:35.553818+0000","last_undegraded":"2024-09-10T09:37:35.553818+0000","last_fullsized":"2024-09-10T09:37:35.553818+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:53:34.562690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"94'5","reported_seq":174,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:38:16.666733+0000","last_change":"2024-09-10T09:35:25.534167+0000","last_active":"2024-09-10T09:38:16.666733+0000","last_peered":"2024-09-10T09:38:16.666733+0000","last_clean":"2024-09-10T09:38:16.666733+0000","last_became_active":"2024-09-10T09:35:25.533719+0000","last_became_peered":"2024-09-10T09:35:25.533719+0000","last_unstale":"2024-09-10T09:38:16.666733+0000","last_undegraded":"2024-09-10T09:38:16.666733+0000","last_fullsized":"2024-09-10T09:38:16.666733+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:00:03.537355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":127,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.527733+0000","last_change":"2024-09-10T09:35:36.628716+0000","last_active":"2024-09-10T09:37:07.527733+0000","last_peered":"2024-09-10T09:37:07.527733+0000","last_clean":"2024-09-10T09:37:07.527733+0000","last_became_active":"2024-09-10T09:35:36.628085+0000","last_became_peered":"2024-09-10T09:35:36.628085+0000","last_unstale":"2024-09-10T09:37:07.527733+0000","last_undegraded":"2024-09-10T09:37:07.527733+0000","last_fullsized":"2024-09-10T09:37:07.527733+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:45:01.279403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"96'15","reported_seq":52,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.557076+0000","last_change":"2024-09-10T09:35:28.566701+0000","last_active":"2024-09-10T09:37:35.557076+0000","last_peered":"2024-09-10T09:37:35.557076+0000","last_clean":"2024-09-10T09:37:35.557076+0000","last_became_active":"2024-09-10T09:35:28.566391+0000","last_became_peered":"2024-09-10T09:35:28.566391+0000","last_unstale":"2024-09-10T09:37:35.557076+0000","last_undegraded":"2024-09-10T09:37:35.557076+0000","last_fullsized":"2024-09-10T09:37:35.557076+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:57:59.918706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"94'5","reported_seq":50,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.531896+0000","last_change":"2024-09-10T09:35:25.532510+0000","last_active":"2024-09-10T09:37:07.531896+0000","last_peered":"2024-09-10T09:37:07.531896+0000","last_clean":"2024-09-10T09:37:07.531896+0000","last_became_active":"2024-09-10T09:35:25.532193+0000","last_became_peered":"2024-09-10T09:35:25.532193+0000","last_unstale":"2024-09-10T09:37:07.531896+0000","last_undegraded":"2024-09-10T09:37:07.531896+0000","last_fullsized":"2024-09-10T09:37:07.531896+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:17:24.685178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.526424+0000","last_change":"2024-09-10T09:35:36.630742+0000","last_active":"2024-09-10T09:37:07.526424+0000","last_peered":"2024-09-10T09:37:07.526424+0000","last_clean":"2024-09-10T09:37:07.526424+0000","last_became_active":"2024-09-10T09:35:36.630335+0000","last_became_peered":"2024-09-10T09:35:36.630335+0000","last_unstale":"2024-09-10T09:37:07.526424+0000","last_undegraded":"2024-09-10T09:37:07.526424+0000","last_fullsized":"2024-09-10T09:37:07.526424+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:29:12.029390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"96'9","reported_seq":48,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.552455+0000","last_change":"2024-09-10T09:35:28.581817+0000","last_active":"2024-09-10T09:37:35.552455+0000","last_peered":"2024-09-10T09:37:35.552455+0000","last_clean":"2024-09-10T09:37:35.552455+0000","last_became_active":"2024-09-10T09:35:28.580316+0000","last_became_peered":"2024-09-10T09:35:28.580316+0000","last_unstale":"2024-09-10T09:37:35.552455+0000","last_undegraded":"2024-09-10T09:37:35.552455+0000","last_fullsized":"2024-09-10T09:37:35.552455+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:04:08.249564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"94'9","reported_seq":55,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.528127+0000","last_change":"2024-09-10T09:35:25.537474+0000","last_active":"2024-09-10T09:37:07.528127+0000","last_peered":"2024-09-10T09:37:07.528127+0000","last_clean":"2024-09-10T09:37:07.528127+0000","last_became_active":"2024-09-10T09:35:25.537096+0000","last_became_peered":"2024-09-10T09:35:25.537096+0000","last_unstale":"2024-09-10T09:37:07.528127+0000","last_undegraded":"2024-09-10T09:37:07.528127+0000","last_fullsized":"2024-09-10T09:37:07.528127+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:13:06.008458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.532316+0000","last_change":"2024-09-10T09:35:36.622788+0000","last_active":"2024-09-10T09:37:07.532316+0000","last_peered":"2024-09-10T09:37:07.532316+0000","last_clean":"2024-09-10T09:37:07.532316+0000","last_became_active":"2024-09-10T09:35:36.622377+0000","last_became_peered":"2024-09-10T09:35:36.622377+0000","last_unstale":"2024-09-10T09:37:07.532316+0000","last_undegraded":"2024-09-10T09:37:07.532316+0000","last_fullsized":"2024-09-10T09:37:07.532316+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:29:56.680864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"96'6","reported_seq":45,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.534371+0000","last_change":"2024-09-10T09:35:28.564529+0000","last_active":"2024-09-10T09:37:35.534371+0000","last_peered":"2024-09-10T09:37:35.534371+0000","last_clean":"2024-09-10T09:37:35.534371+0000","last_became_active":"2024-09-10T09:35:28.564223+0000","last_became_peered":"2024-09-10T09:35:28.564223+0000","last_unstale":"2024-09-10T09:37:35.534371+0000","last_undegraded":"2024-09-10T09:37:35.534371+0000","last_fullsized":"2024-09-10T09:37:35.534371+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:21:24.296578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"96'38","reported_seq":330,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:38:13.662697+0000","last_change":"2024-09-10T09:35:25.534331+0000","last_active":"2024-09-10T09:38:13.662697+0000","last_peered":"2024-09-10T09:38:13.662697+0000","last_clean":"2024-09-10T09:38:13.662697+0000","last_became_active":"2024-09-10T09:35:25.533985+0000","last_became_peered":"2024-09-10T09:35:25.533985+0000","last_unstale":"2024-09-10T09:38:13.662697+0000","last_undegraded":"2024-09-10T09:38:13.662697+0000","last_fullsized":"2024-09-10T09:38:13.662697+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:14:37.050650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":63180,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":248,"num_read_kb":186,"num_write":40,"num_write_kb":79,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.527925+0000","last_change":"2024-09-10T09:35:36.632835+0000","last_active":"2024-09-10T09:37:07.527925+0000","last_peered":"2024-09-10T09:37:07.527925+0000","last_clean":"2024-09-10T09:37:07.527925+0000","last_became_active":"2024-09-10T09:35:36.630962+0000","last_became_peered":"2024-09-10T09:35:36.630962+0000","last_unstale":"2024-09-10T09:37:07.527925+0000","last_undegraded":"2024-09-10T09:37:07.527925+0000","last_fullsized":"2024-09-10T09:37:07.527925+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:59:27.613721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"96'11","reported_seq":50,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.546541+0000","last_change":"2024-09-10T09:35:28.567292+0000","last_active":"2024-09-10T09:37:35.546541+0000","last_peered":"2024-09-10T09:37:35.546541+0000","last_clean":"2024-09-10T09:37:35.546541+0000","last_became_active":"2024-09-10T09:35:28.566973+0000","last_became_peered":"2024-09-10T09:35:28.566973+0000","last_unstale":"2024-09-10T09:37:35.546541+0000","last_undegraded":"2024-09-10T09:37:35.546541+0000","last_fullsized":"2024-09-10T09:37:35.546541+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:05:37.251933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"94'5","reported_seq":50,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.532787+0000","last_change":"2024-09-10T09:35:25.852171+0000","last_active":"2024-09-10T09:37:07.532787+0000","last_peered":"2024-09-10T09:37:07.532787+0000","last_clean":"2024-09-10T09:37:07.532787+0000","last_became_active":"2024-09-10T09:35:25.851580+0000","last_became_peered":"2024-09-10T09:35:25.851580+0000","last_unstale":"2024-09-10T09:37:07.532787+0000","last_undegraded":"2024-09-10T09:37:07.532787+0000","last_fullsized":"2024-09-10T09:37:07.532787+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:03.263152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.568336+0000","last_change":"2024-09-10T09:35:36.629212+0000","last_active":"2024-09-10T09:37:07.568336+0000","last_peered":"2024-09-10T09:37:07.568336+0000","last_clean":"2024-09-10T09:37:07.568336+0000","last_became_active":"2024-09-10T09:35:36.628753+0000","last_became_peered":"2024-09-10T09:35:36.628753+0000","last_unstale":"2024-09-10T09:37:07.568336+0000","last_undegraded":"2024-09-10T09:37:07.568336+0000","last_fullsized":"2024-09-10T09:37:07.568336+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:54:45.311361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"96'9","reported_seq":46,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.553075+0000","last_change":"2024-09-10T09:35:28.572497+0000","last_active":"2024-09-10T09:37:35.553075+0000","last_peered":"2024-09-10T09:37:35.553075+0000","last_clean":"2024-09-10T09:37:35.553075+0000","last_became_active":"2024-09-10T09:35:28.569238+0000","last_became_peered":"2024-09-10T09:35:28.569238+0000","last_unstale":"2024-09-10T09:37:35.553075+0000","last_undegraded":"2024-09-10T09:37:35.553075+0000","last_fullsized":"2024-09-10T09:37:35.553075+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:38:47.189209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"94'4","reported_seq":47,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.529569+0000","last_change":"2024-09-10T09:35:25.536369+0000","last_active":"2024-09-10T09:37:07.529569+0000","last_peered":"2024-09-10T09:37:07.529569+0000","last_clean":"2024-09-10T09:37:07.529569+0000","last_became_active":"2024-09-10T09:35:25.535607+0000","last_became_peered":"2024-09-10T09:35:25.535607+0000","last_unstale":"2024-09-10T09:37:07.529569+0000","last_undegraded":"2024-09-10T09:37:07.529569+0000","last_fullsized":"2024-09-10T09:37:07.529569+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:49:18.621934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.527047+0000","last_change":"2024-09-10T09:35:36.627977+0000","last_active":"2024-09-10T09:37:07.527047+0000","last_peered":"2024-09-10T09:37:07.527047+0000","last_clean":"2024-09-10T09:37:07.527047+0000","last_became_active":"2024-09-10T09:35:36.626942+0000","last_became_peered":"2024-09-10T09:35:36.626942+0000","last_unstale":"2024-09-10T09:37:07.527047+0000","last_undegraded":"2024-09-10T09:37:07.527047+0000","last_fullsized":"2024-09-10T09:37:07.527047+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:37:18.209424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"96'10","reported_seq":49,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.553165+0000","last_change":"2024-09-10T09:35:28.562670+0000","last_active":"2024-09-10T09:37:35.553165+0000","last_peered":"2024-09-10T09:37:35.553165+0000","last_clean":"2024-09-10T09:37:35.553165+0000","last_became_active":"2024-09-10T09:35:28.562028+0000","last_became_peered":"2024-09-10T09:35:28.562028+0000","last_unstale":"2024-09-10T09:37:35.553165+0000","last_undegraded":"2024-09-10T09:37:35.553165+0000","last_fullsized":"2024-09-10T09:37:35.553165+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:37:15.176050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"96'5","reported_seq":288,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:38:13.662801+0000","last_change":"2024-09-10T09:35:25.529861+0000","last_active":"2024-09-10T09:38:13.662801+0000","last_peered":"2024-09-10T09:38:13.662801+0000","last_clean":"2024-09-10T09:38:13.662801+0000","last_became_active":"2024-09-10T09:35:25.529475+0000","last_became_peered":"2024-09-10T09:35:25.529475+0000","last_unstale":"2024-09-10T09:38:13.662801+0000","last_undegraded":"2024-09-10T09:38:13.662801+0000","last_fullsized":"2024-09-10T09:38:13.662801+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:44:27.086509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":239,"num_read_kb":239,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.524306+0000","last_change":"2024-09-10T09:35:36.615358+0000","last_active":"2024-09-10T09:37:07.524306+0000","last_peered":"2024-09-10T09:37:07.524306+0000","last_clean":"2024-09-10T09:37:07.524306+0000","last_became_active":"2024-09-10T09:35:36.614901+0000","last_became_peered":"2024-09-10T09:35:36.614901+0000","last_unstale":"2024-09-10T09:37:07.524306+0000","last_undegraded":"2024-09-10T09:37:07.524306+0000","last_fullsized":"2024-09-10T09:37:07.524306+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:56:54.248266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.761302+0000","last_change":"2024-09-10T09:35:36.629021+0000","last_active":"2024-09-10T09:37:07.761302+0000","last_peered":"2024-09-10T09:37:07.761302+0000","last_clean":"2024-09-10T09:37:07.761302+0000","last_became_active":"2024-09-10T09:35:36.628159+0000","last_became_peered":"2024-09-10T09:35:36.628159+0000","last_unstale":"2024-09-10T09:37:07.761302+0000","last_undegraded":"2024-09-10T09:37:07.761302+0000","last_fullsized":"2024-09-10T09:37:07.761302+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:50:48.615236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"96'11","reported_seq":48,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.545215+0000","last_change":"2024-09-10T09:35:28.579333+0000","last_active":"2024-09-10T09:37:35.545215+0000","last_peered":"2024-09-10T09:37:35.545215+0000","last_clean":"2024-09-10T09:37:35.545215+0000","last_became_active":"2024-09-10T09:35:28.578440+0000","last_became_peered":"2024-09-10T09:35:28.578440+0000","last_unstale":"2024-09-10T09:37:35.545215+0000","last_undegraded":"2024-09-10T09:37:35.545215+0000","last_fullsized":"2024-09-10T09:37:35.545215+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:07:28.701938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"94'3","reported_seq":48,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.531063+0000","last_change":"2024-09-10T09:35:25.530857+0000","last_active":"2024-09-10T09:37:07.531063+0000","last_peered":"2024-09-10T09:37:07.531063+0000","last_clean":"2024-09-10T09:37:07.531063+0000","last_became_active":"2024-09-10T09:35:25.530492+0000","last_became_peered":"2024-09-10T09:35:25.530492+0000","last_unstale":"2024-09-10T09:37:07.531063+0000","last_undegraded":"2024-09-10T09:37:07.531063+0000","last_fullsized":"2024-09-10T09:37:07.531063+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:07:21.726598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.526432+0000","last_change":"2024-09-10T09:36:06.770408+0000","last_active":"2024-09-10T09:37:07.526432+0000","last_peered":"2024-09-10T09:37:07.526432+0000","last_clean":"2024-09-10T09:37:07.526432+0000","last_became_active":"2024-09-10T09:36:06.769915+0000","last_became_peered":"2024-09-10T09:36:06.769915+0000","last_unstale":"2024-09-10T09:37:07.526432+0000","last_undegraded":"2024-09-10T09:37:07.526432+0000","last_fullsized":"2024-09-10T09:37:07.526432+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:56:09.795537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8,0],"acting":[11,4,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"96'8","reported_seq":47,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.543491+0000","last_change":"2024-09-10T09:35:28.570804+0000","last_active":"2024-09-10T09:37:35.543491+0000","last_peered":"2024-09-10T09:37:35.543491+0000","last_clean":"2024-09-10T09:37:35.543491+0000","last_became_active":"2024-09-10T09:35:28.570490+0000","last_became_peered":"2024-09-10T09:35:28.570490+0000","last_unstale":"2024-09-10T09:37:35.543491+0000","last_undegraded":"2024-09-10T09:37:35.543491+0000","last_fullsized":"2024-09-10T09:37:35.543491+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:13:38.999625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"94'3","reported_seq":47,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.538446+0000","last_change":"2024-09-10T09:35:25.536347+0000","last_active":"2024-09-10T09:37:07.538446+0000","last_peered":"2024-09-10T09:37:07.538446+0000","last_clean":"2024-09-10T09:37:07.538446+0000","last_became_active":"2024-09-10T09:35:25.535964+0000","last_became_peered":"2024-09-10T09:35:25.535964+0000","last_unstale":"2024-09-10T09:37:07.538446+0000","last_undegraded":"2024-09-10T09:37:07.538446+0000","last_fullsized":"2024-09-10T09:37:07.538446+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:36:49.035666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"94'7","reported_seq":51,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.525812+0000","last_change":"2024-09-10T09:35:25.532148+0000","last_active":"2024-09-10T09:37:07.525812+0000","last_peered":"2024-09-10T09:37:07.525812+0000","last_clean":"2024-09-10T09:37:07.525812+0000","last_became_active":"2024-09-10T09:35:25.531781+0000","last_became_peered":"2024-09-10T09:35:25.531781+0000","last_unstale":"2024-09-10T09:37:07.525812+0000","last_undegraded":"2024-09-10T09:37:07.525812+0000","last_fullsized":"2024-09-10T09:37:07.525812+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:52:46.740342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"96'8","reported_seq":47,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.549412+0000","last_change":"2024-09-10T09:35:28.578336+0000","last_active":"2024-09-10T09:37:35.549412+0000","last_peered":"2024-09-10T09:37:35.549412+0000","last_clean":"2024-09-10T09:37:35.549412+0000","last_became_active":"2024-09-10T09:35:28.577899+0000","last_became_peered":"2024-09-10T09:35:28.577899+0000","last_unstale":"2024-09-10T09:37:35.549412+0000","last_undegraded":"2024-09-10T09:37:35.549412+0000","last_fullsized":"2024-09-10T09:37:35.549412+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:13:04.478195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.528122+0000","last_change":"2024-09-10T09:35:36.630918+0000","last_active":"2024-09-10T09:37:07.528122+0000","last_peered":"2024-09-10T09:37:07.528122+0000","last_clean":"2024-09-10T09:37:07.528122+0000","last_became_active":"2024-09-10T09:35:36.630308+0000","last_became_peered":"2024-09-10T09:35:36.630308+0000","last_unstale":"2024-09-10T09:37:07.528122+0000","last_undegraded":"2024-09-10T09:37:07.528122+0000","last_fullsized":"2024-09-10T09:37:07.528122+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:25:44.954970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"96'7","reported_seq":186,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:38:12.661788+0000","last_change":"2024-09-10T09:37:07.574386+0000","last_active":"2024-09-10T09:38:12.661788+0000","last_peered":"2024-09-10T09:38:12.661788+0000","last_clean":"2024-09-10T09:38:12.661788+0000","last_became_active":"2024-09-10T09:37:07.569370+0000","last_became_peered":"2024-09-10T09:37:07.569370+0000","last_unstale":"2024-09-10T09:38:12.661788+0000","last_undegraded":"2024-09-10T09:38:12.661788+0000","last_fullsized":"2024-09-10T09:38:12.661788+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:07:23.014789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1156,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":118,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":1015,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"96'7","reported_seq":46,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.543648+0000","last_change":"2024-09-10T09:35:28.567915+0000","last_active":"2024-09-10T09:37:35.543648+0000","last_peered":"2024-09-10T09:37:35.543648+0000","last_clean":"2024-09-10T09:37:35.543648+0000","last_became_active":"2024-09-10T09:35:28.567416+0000","last_became_peered":"2024-09-10T09:35:28.567416+0000","last_unstale":"2024-09-10T09:37:35.543648+0000","last_undegraded":"2024-09-10T09:37:35.543648+0000","last_fullsized":"2024-09-10T09:37:35.543648+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:07:25.704080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.528686+0000","last_change":"2024-09-10T09:35:36.615386+0000","last_active":"2024-09-10T09:37:07.528686+0000","last_peered":"2024-09-10T09:37:07.528686+0000","last_clean":"2024-09-10T09:37:07.528686+0000","last_became_active":"2024-09-10T09:35:36.614967+0000","last_became_peered":"2024-09-10T09:35:36.614967+0000","last_unstale":"2024-09-10T09:37:07.528686+0000","last_undegraded":"2024-09-10T09:37:07.528686+0000","last_fullsized":"2024-09-10T09:37:07.528686+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:47.620729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"96'9","reported_seq":48,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.544481+0000","last_change":"2024-09-10T09:35:28.564411+0000","last_active":"2024-09-10T09:37:35.544481+0000","last_peered":"2024-09-10T09:37:35.544481+0000","last_clean":"2024-09-10T09:37:35.544481+0000","last_became_active":"2024-09-10T09:35:28.564034+0000","last_became_peered":"2024-09-10T09:35:28.564034+0000","last_unstale":"2024-09-10T09:37:35.544481+0000","last_undegraded":"2024-09-10T09:37:35.544481+0000","last_fullsized":"2024-09-10T09:37:35.544481+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:38:17.687991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"96'12","reported_seq":49,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.547277+0000","last_change":"2024-09-10T09:35:28.564619+0000","last_active":"2024-09-10T09:37:35.547277+0000","last_peered":"2024-09-10T09:37:35.547277+0000","last_clean":"2024-09-10T09:37:35.547277+0000","last_became_active":"2024-09-10T09:35:28.563890+0000","last_became_peered":"2024-09-10T09:35:28.563890+0000","last_unstale":"2024-09-10T09:37:35.547277+0000","last_undegraded":"2024-09-10T09:37:35.547277+0000","last_fullsized":"2024-09-10T09:37:35.547277+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:51:42.481911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"96'8","reported_seq":45,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.543410+0000","last_change":"2024-09-10T09:35:28.558909+0000","last_active":"2024-09-10T09:37:35.543410+0000","last_peered":"2024-09-10T09:37:35.543410+0000","last_clean":"2024-09-10T09:37:35.543410+0000","last_became_active":"2024-09-10T09:35:28.558604+0000","last_became_peered":"2024-09-10T09:35:28.558604+0000","last_unstale":"2024-09-10T09:37:35.543410+0000","last_undegraded":"2024-09-10T09:37:35.543410+0000","last_fullsized":"2024-09-10T09:37:35.543410+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:11.928446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"96'7","reported_seq":44,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.554844+0000","last_change":"2024-09-10T09:35:28.574459+0000","last_active":"2024-09-10T09:37:35.554844+0000","last_peered":"2024-09-10T09:37:35.554844+0000","last_clean":"2024-09-10T09:37:35.554844+0000","last_became_active":"2024-09-10T09:35:28.574154+0000","last_became_peered":"2024-09-10T09:35:28.574154+0000","last_unstale":"2024-09-10T09:37:35.554844+0000","last_undegraded":"2024-09-10T09:37:35.554844+0000","last_fullsized":"2024-09-10T09:37:35.554844+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:59:09.650536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"96'4","reported_seq":43,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.552571+0000","last_change":"2024-09-10T09:35:28.576999+0000","last_active":"2024-09-10T09:37:35.552571+0000","last_peered":"2024-09-10T09:37:35.552571+0000","last_clean":"2024-09-10T09:37:35.552571+0000","last_became_active":"2024-09-10T09:35:28.575616+0000","last_became_peered":"2024-09-10T09:35:28.575616+0000","last_unstale":"2024-09-10T09:37:35.552571+0000","last_undegraded":"2024-09-10T09:37:35.552571+0000","last_fullsized":"2024-09-10T09:37:35.552571+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:18:08.771760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"96'6","reported_seq":45,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.541496+0000","last_change":"2024-09-10T09:35:28.564846+0000","last_active":"2024-09-10T09:37:35.541496+0000","last_peered":"2024-09-10T09:37:35.541496+0000","last_clean":"2024-09-10T09:37:35.541496+0000","last_became_active":"2024-09-10T09:35:28.564498+0000","last_became_peered":"2024-09-10T09:35:28.564498+0000","last_unstale":"2024-09-10T09:37:35.541496+0000","last_undegraded":"2024-09-10T09:37:35.541496+0000","last_fullsized":"2024-09-10T09:37:35.541496+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:33:43.041730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"96'15","reported_seq":52,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.550975+0000","last_change":"2024-09-10T09:35:28.558093+0000","last_active":"2024-09-10T09:37:35.550975+0000","last_peered":"2024-09-10T09:37:35.550975+0000","last_clean":"2024-09-10T09:37:35.550975+0000","last_became_active":"2024-09-10T09:35:28.557266+0000","last_became_peered":"2024-09-10T09:35:28.557266+0000","last_unstale":"2024-09-10T09:37:35.550975+0000","last_undegraded":"2024-09-10T09:37:35.550975+0000","last_fullsized":"2024-09-10T09:37:35.550975+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:51:30.946065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"96'2","reported_seq":47,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.549558+0000","last_change":"2024-09-10T09:36:06.770298+0000","last_active":"2024-09-10T09:37:35.549558+0000","last_peered":"2024-09-10T09:37:35.549558+0000","last_clean":"2024-09-10T09:37:35.549558+0000","last_became_active":"2024-09-10T09:36:06.769848+0000","last_became_peered":"2024-09-10T09:36:06.769848+0000","last_unstale":"2024-09-10T09:37:35.549558+0000","last_undegraded":"2024-09-10T09:37:35.549558+0000","last_fullsized":"2024-09-10T09:37:35.549558+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:00:30.470887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,8,2],"acting":[3,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"96'6","reported_seq":45,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.549553+0000","last_change":"2024-09-10T09:35:28.578158+0000","last_active":"2024-09-10T09:37:35.549553+0000","last_peered":"2024-09-10T09:37:35.549553+0000","last_clean":"2024-09-10T09:37:35.549553+0000","last_became_active":"2024-09-10T09:35:28.577824+0000","last_became_peered":"2024-09-10T09:35:28.577824+0000","last_unstale":"2024-09-10T09:37:35.549553+0000","last_undegraded":"2024-09-10T09:37:35.549553+0000","last_fullsized":"2024-09-10T09:37:35.549553+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:40:27.231164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"96'8","reported_seq":47,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.550143+0000","last_change":"2024-09-10T09:35:28.562670+0000","last_active":"2024-09-10T09:37:35.550143+0000","last_peered":"2024-09-10T09:37:35.550143+0000","last_clean":"2024-09-10T09:37:35.550143+0000","last_became_active":"2024-09-10T09:35:28.562311+0000","last_became_peered":"2024-09-10T09:35:28.562311+0000","last_unstale":"2024-09-10T09:37:35.550143+0000","last_undegraded":"2024-09-10T09:37:35.550143+0000","last_fullsized":"2024-09-10T09:37:35.550143+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:07:11.716142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"96'10","reported_seq":49,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.556330+0000","last_change":"2024-09-10T09:35:28.561124+0000","last_active":"2024-09-10T09:37:35.556330+0000","last_peered":"2024-09-10T09:37:35.556330+0000","last_clean":"2024-09-10T09:37:35.556330+0000","last_became_active":"2024-09-10T09:35:28.560777+0000","last_became_peered":"2024-09-10T09:35:28.560777+0000","last_unstale":"2024-09-10T09:37:35.556330+0000","last_undegraded":"2024-09-10T09:37:35.556330+0000","last_fullsized":"2024-09-10T09:37:35.556330+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:47:04.837596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"96'11","reported_seq":50,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.556539+0000","last_change":"2024-09-10T09:35:28.576790+0000","last_active":"2024-09-10T09:37:35.556539+0000","last_peered":"2024-09-10T09:37:35.556539+0000","last_clean":"2024-09-10T09:37:35.556539+0000","last_became_active":"2024-09-10T09:35:28.576464+0000","last_became_peered":"2024-09-10T09:35:28.576464+0000","last_unstale":"2024-09-10T09:37:35.556539+0000","last_undegraded":"2024-09-10T09:37:35.556539+0000","last_fullsized":"2024-09-10T09:37:35.556539+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:40:53.517577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"96'8","reported_seq":47,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.539417+0000","last_change":"2024-09-10T09:35:28.569935+0000","last_active":"2024-09-10T09:37:35.539417+0000","last_peered":"2024-09-10T09:37:35.539417+0000","last_clean":"2024-09-10T09:37:35.539417+0000","last_became_active":"2024-09-10T09:35:28.569539+0000","last_became_peered":"2024-09-10T09:35:28.569539+0000","last_unstale":"2024-09-10T09:37:35.539417+0000","last_undegraded":"2024-09-10T09:37:35.539417+0000","last_fullsized":"2024-09-10T09:37:35.539417+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:40:45.016269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"96'7","reported_seq":44,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.550145+0000","last_change":"2024-09-10T09:35:28.571619+0000","last_active":"2024-09-10T09:37:35.550145+0000","last_peered":"2024-09-10T09:37:35.550145+0000","last_clean":"2024-09-10T09:37:35.550145+0000","last_became_active":"2024-09-10T09:35:28.568591+0000","last_became_peered":"2024-09-10T09:35:28.568591+0000","last_unstale":"2024-09-10T09:37:35.550145+0000","last_undegraded":"2024-09-10T09:37:35.550145+0000","last_fullsized":"2024-09-10T09:37:35.550145+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:07:49.959712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"96'10","reported_seq":47,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.551796+0000","last_change":"2024-09-10T09:35:28.573336+0000","last_active":"2024-09-10T09:37:35.551796+0000","last_peered":"2024-09-10T09:37:35.551796+0000","last_clean":"2024-09-10T09:37:35.551796+0000","last_became_active":"2024-09-10T09:35:28.572991+0000","last_became_peered":"2024-09-10T09:35:28.572991+0000","last_unstale":"2024-09-10T09:37:35.551796+0000","last_undegraded":"2024-09-10T09:37:35.551796+0000","last_fullsized":"2024-09-10T09:37:35.551796+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:30:02.905869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"96'8","reported_seq":47,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.552930+0000","last_change":"2024-09-10T09:35:28.571945+0000","last_active":"2024-09-10T09:37:35.552930+0000","last_peered":"2024-09-10T09:37:35.552930+0000","last_clean":"2024-09-10T09:37:35.552930+0000","last_became_active":"2024-09-10T09:35:28.571546+0000","last_became_peered":"2024-09-10T09:35:28.571546+0000","last_unstale":"2024-09-10T09:37:35.552930+0000","last_undegraded":"2024-09-10T09:37:35.552930+0000","last_fullsized":"2024-09-10T09:37:35.552930+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:36:47.946864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"96'9","reported_seq":48,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.556725+0000","last_change":"2024-09-10T09:35:28.562712+0000","last_active":"2024-09-10T09:37:35.556725+0000","last_peered":"2024-09-10T09:37:35.556725+0000","last_clean":"2024-09-10T09:37:35.556725+0000","last_became_active":"2024-09-10T09:35:28.562396+0000","last_became_peered":"2024-09-10T09:35:28.562396+0000","last_unstale":"2024-09-10T09:37:35.556725+0000","last_undegraded":"2024-09-10T09:37:35.556725+0000","last_fullsized":"2024-09-10T09:37:35.556725+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:25:47.419162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"96'7","reported_seq":44,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.553427+0000","last_change":"2024-09-10T09:35:28.566354+0000","last_active":"2024-09-10T09:37:35.553427+0000","last_peered":"2024-09-10T09:37:35.553427+0000","last_clean":"2024-09-10T09:37:35.553427+0000","last_became_active":"2024-09-10T09:35:28.565248+0000","last_became_peered":"2024-09-10T09:35:28.565248+0000","last_unstale":"2024-09-10T09:37:35.553427+0000","last_undegraded":"2024-09-10T09:37:35.553427+0000","last_fullsized":"2024-09-10T09:37:35.553427+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:48:46.741010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"96'7","reported_seq":44,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.550936+0000","last_change":"2024-09-10T09:35:28.563331+0000","last_active":"2024-09-10T09:37:35.550936+0000","last_peered":"2024-09-10T09:37:35.550936+0000","last_clean":"2024-09-10T09:37:35.550936+0000","last_became_active":"2024-09-10T09:35:28.563020+0000","last_became_peered":"2024-09-10T09:35:28.563020+0000","last_unstale":"2024-09-10T09:37:35.550936+0000","last_undegraded":"2024-09-10T09:37:35.550936+0000","last_fullsized":"2024-09-10T09:37:35.550936+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:27.653551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"96'10","reported_seq":49,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.542747+0000","last_change":"2024-09-10T09:35:28.566687+0000","last_active":"2024-09-10T09:37:35.542747+0000","last_peered":"2024-09-10T09:37:35.542747+0000","last_clean":"2024-09-10T09:37:35.542747+0000","last_became_active":"2024-09-10T09:35:28.566253+0000","last_became_peered":"2024-09-10T09:35:28.566253+0000","last_unstale":"2024-09-10T09:37:35.542747+0000","last_undegraded":"2024-09-10T09:37:35.542747+0000","last_fullsized":"2024-09-10T09:37:35.542747+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:00:15.709725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"96'7","reported_seq":44,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.538923+0000","last_change":"2024-09-10T09:35:28.562251+0000","last_active":"2024-09-10T09:37:35.538923+0000","last_peered":"2024-09-10T09:37:35.538923+0000","last_clean":"2024-09-10T09:37:35.538923+0000","last_became_active":"2024-09-10T09:35:28.561933+0000","last_became_peered":"2024-09-10T09:35:28.561933+0000","last_unstale":"2024-09-10T09:37:35.538923+0000","last_undegraded":"2024-09-10T09:37:35.538923+0000","last_fullsized":"2024-09-10T09:37:35.538923+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:30:41.447954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"96'6","reported_seq":43,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.556126+0000","last_change":"2024-09-10T09:35:28.572164+0000","last_active":"2024-09-10T09:37:35.556126+0000","last_peered":"2024-09-10T09:37:35.556126+0000","last_clean":"2024-09-10T09:37:35.556126+0000","last_became_active":"2024-09-10T09:35:28.571825+0000","last_became_peered":"2024-09-10T09:35:28.571825+0000","last_unstale":"2024-09-10T09:37:35.556126+0000","last_undegraded":"2024-09-10T09:37:35.556126+0000","last_fullsized":"2024-09-10T09:37:35.556126+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:59:50.967269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"96'16","reported_seq":55,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.550138+0000","last_change":"2024-09-10T09:35:28.568098+0000","last_active":"2024-09-10T09:37:35.550138+0000","last_peered":"2024-09-10T09:37:35.550138+0000","last_clean":"2024-09-10T09:37:35.550138+0000","last_became_active":"2024-09-10T09:35:28.567765+0000","last_became_peered":"2024-09-10T09:35:28.567765+0000","last_unstale":"2024-09-10T09:37:35.550138+0000","last_undegraded":"2024-09-10T09:37:35.550138+0000","last_fullsized":"2024-09-10T09:37:35.550138+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:55:57.260320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"96'11","reported_seq":50,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.553598+0000","last_change":"2024-09-10T09:35:28.564123+0000","last_active":"2024-09-10T09:37:35.553598+0000","last_peered":"2024-09-10T09:37:35.553598+0000","last_clean":"2024-09-10T09:37:35.553598+0000","last_became_active":"2024-09-10T09:35:28.563814+0000","last_became_peered":"2024-09-10T09:35:28.563814+0000","last_unstale":"2024-09-10T09:37:35.553598+0000","last_undegraded":"2024-09-10T09:37:35.553598+0000","last_fullsized":"2024-09-10T09:37:35.553598+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:14:17.912087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"96'9","reported_seq":48,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.549407+0000","last_change":"2024-09-10T09:35:28.568601+0000","last_active":"2024-09-10T09:37:35.549407+0000","last_peered":"2024-09-10T09:37:35.549407+0000","last_clean":"2024-09-10T09:37:35.549407+0000","last_became_active":"2024-09-10T09:35:28.568303+0000","last_became_peered":"2024-09-10T09:35:28.568303+0000","last_unstale":"2024-09-10T09:37:35.549407+0000","last_undegraded":"2024-09-10T09:37:35.549407+0000","last_fullsized":"2024-09-10T09:37:35.549407+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:30:01.016420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"96'5","reported_seq":48,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.556713+0000","last_change":"2024-09-10T09:37:07.539577+0000","last_active":"2024-09-10T09:37:35.556713+0000","last_peered":"2024-09-10T09:37:35.556713+0000","last_clean":"2024-09-10T09:37:35.556713+0000","last_became_active":"2024-09-10T09:37:07.539206+0000","last_became_peered":"2024-09-10T09:37:07.539206+0000","last_unstale":"2024-09-10T09:37:35.556713+0000","last_undegraded":"2024-09-10T09:37:35.556713+0000","last_fullsized":"2024-09-10T09:37:35.556713+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:56:06.568226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"96'6","reported_seq":45,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.541193+0000","last_change":"2024-09-10T09:35:28.569225+0000","last_active":"2024-09-10T09:37:35.541193+0000","last_peered":"2024-09-10T09:37:35.541193+0000","last_clean":"2024-09-10T09:37:35.541193+0000","last_became_active":"2024-09-10T09:35:28.568891+0000","last_became_peered":"2024-09-10T09:35:28.568891+0000","last_unstale":"2024-09-10T09:37:35.541193+0000","last_undegraded":"2024-09-10T09:37:35.541193+0000","last_fullsized":"2024-09-10T09:37:35.541193+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:33:45.811781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"96'4","reported_seq":43,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.544182+0000","last_change":"2024-09-10T09:35:28.578362+0000","last_active":"2024-09-10T09:37:35.544182+0000","last_peered":"2024-09-10T09:37:35.544182+0000","last_clean":"2024-09-10T09:37:35.544182+0000","last_became_active":"2024-09-10T09:35:28.577867+0000","last_became_peered":"2024-09-10T09:35:28.577867+0000","last_unstale":"2024-09-10T09:37:35.544182+0000","last_undegraded":"2024-09-10T09:37:35.544182+0000","last_fullsized":"2024-09-10T09:37:35.544182+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:58:17.041663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"96'8","reported_seq":45,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.537018+0000","last_change":"2024-09-10T09:35:28.568183+0000","last_active":"2024-09-10T09:37:35.537018+0000","last_peered":"2024-09-10T09:37:35.537018+0000","last_clean":"2024-09-10T09:37:35.537018+0000","last_became_active":"2024-09-10T09:35:28.566775+0000","last_became_peered":"2024-09-10T09:35:28.566775+0000","last_unstale":"2024-09-10T09:37:35.537018+0000","last_undegraded":"2024-09-10T09:37:35.537018+0000","last_fullsized":"2024-09-10T09:37:35.537018+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:28:10.008889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"96'7","reported_seq":44,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.553233+0000","last_change":"2024-09-10T09:35:28.576122+0000","last_active":"2024-09-10T09:37:35.553233+0000","last_peered":"2024-09-10T09:37:35.553233+0000","last_clean":"2024-09-10T09:37:35.553233+0000","last_became_active":"2024-09-10T09:35:28.575831+0000","last_became_peered":"2024-09-10T09:35:28.575831+0000","last_unstale":"2024-09-10T09:37:35.553233+0000","last_undegraded":"2024-09-10T09:37:35.553233+0000","last_fullsized":"2024-09-10T09:37:35.553233+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:37:51.604658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"96'4","reported_seq":41,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.554017+0000","last_change":"2024-09-10T09:35:28.564441+0000","last_active":"2024-09-10T09:37:35.554017+0000","last_peered":"2024-09-10T09:37:35.554017+0000","last_clean":"2024-09-10T09:37:35.554017+0000","last_became_active":"2024-09-10T09:35:28.564095+0000","last_became_peered":"2024-09-10T09:35:28.564095+0000","last_unstale":"2024-09-10T09:37:35.554017+0000","last_undegraded":"2024-09-10T09:37:35.554017+0000","last_fullsized":"2024-09-10T09:37:35.554017+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:53:28.275361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"96'8","reported_seq":47,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.553155+0000","last_change":"2024-09-10T09:35:28.581921+0000","last_active":"2024-09-10T09:37:35.553155+0000","last_peered":"2024-09-10T09:37:35.553155+0000","last_clean":"2024-09-10T09:37:35.553155+0000","last_became_active":"2024-09-10T09:35:28.581573+0000","last_became_peered":"2024-09-10T09:35:28.581573+0000","last_unstale":"2024-09-10T09:37:35.553155+0000","last_undegraded":"2024-09-10T09:37:35.553155+0000","last_fullsized":"2024-09-10T09:37:35.553155+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:58:38.983934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":70670,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1244,"num_read_kb":703,"num_write":260,"num_write_kb":150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":1015,"num_keys_recovered":7,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":589824,"data_stored":225188,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":223,"ondisk_log_size":223,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":285627,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":77,"seq":330712481840,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32228,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712924,"statfs":{"total":95995035648,"available":95962034176,"internally_reserved":0,"allocated":1069056,"data_stored":554290,"data_compressed":32456,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":70,"seq":300647710772,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28220,"kb_used_data":1196,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716932,"statfs":{"total":95995035648,"available":95966138368,"internally_reserved":0,"allocated":1224704,"data_stored":1208382,"data_compressed":68777,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":65,"seq":279172874299,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32252,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712900,"statfs":{"total":95995035648,"available":95962009600,"internally_reserved":0,"allocated":1093632,"data_stored":554220,"data_compressed":32456,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1599,"internal_metadata":31914433},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":57,"seq":244813135936,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32248,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712904,"statfs":{"total":95995035648,"available":95962013696,"internally_reserved":0,"allocated":1089536,"data_stored":554598,"data_compressed":32456,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":50,"seq":214748364869,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28140,"kb_used_data":1116,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717012,"statfs":{"total":95995035648,"available":95966220288,"internally_reserved":0,"allocated":1142784,"data_stored":1151772,"data_compressed":68777,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":45,"seq":193273528395,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36388,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708764,"statfs":{"total":95995035648,"available":95957774336,"internally_reserved":0,"allocated":1069056,"data_stored":550569,"data_compressed":32456,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790033,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28148,"kb_used_data":1124,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717004,"statfs":{"total":95995035648,"available":95966212096,"internally_reserved":0,"allocated":1150976,"data_stored":1146820,"data_compressed":68777,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953559,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32244,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712908,"statfs":{"total":95995035648,"available":95962017792,"internally_reserved":0,"allocated":1085440,"data_stored":554954,"data_compressed":32456,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182491,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28080,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717072,"statfs":{"total":95995035648,"available":95966281728,"internally_reserved":0,"allocated":1081344,"data_stored":549432,"data_compressed":32456,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411424,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28152,"kb_used_data":1128,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717000,"statfs":{"total":95995035648,"available":95966208000,"internally_reserved":0,"allocated":1155072,"data_stored":618550,"data_compressed":32456,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574951,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36460,"kb_used_data":1116,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708692,"statfs":{"total":95995035648,"available":95957700608,"internally_reserved":0,"allocated":1142784,"data_stored":619797,"data_compressed":32456,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738475,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36412,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708740,"statfs":{"total":95995035648,"available":95957749760,"internally_reserved":0,"allocated":1093632,"data_stored":551180,"data_compressed":32456,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34674,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34674,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34674,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36321,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36321,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36321,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36321,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36321,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":73315,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":69548,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3354,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1567,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4210,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":118784,"data_stored":64916,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T09:38:21.094 INFO:tasks.fwd_scrub.ceph_manager:clean! 2024-09-10T09:38:21.094 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T09:38:21.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:38:20 smithi102 ceph-mon[35442]: pgmap v393: 129 pgs: 129 active+clean; 646 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 85 B/s wr, 9 op/s 2024-09-10T09:38:21.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:21.291+0000 7f2087a00640 1 -- 172.21.15.5:0/1345061754 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2080104f20 msgr2=0x7f20801053a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:38:21.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:21.291+0000 7f2087a00640 1 --2- 172.21.15.5:0/1345061754 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2080104f20 0x7f20801053a0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f206c009900 tx=0x7f206c02a3f0 comp rx=0 tx=0).stop 2024-09-10T09:38:21.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:21.291+0000 7f2087a00640 1 -- 172.21.15.5:0/1345061754 shutdown_connections 2024-09-10T09:38:21.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:21.291+0000 7f2087a00640 1 --2- 172.21.15.5:0/1345061754 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f20801058e0 0x7f208010c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:21.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:21.291+0000 7f2087a00640 1 --2- 172.21.15.5:0/1345061754 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2080104f20 0x7f20801053a0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:21.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:21.291+0000 7f2087a00640 1 --2- 172.21.15.5:0/1345061754 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2080103d30 0x7f2080104130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:21.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:21.291+0000 7f2087a00640 1 -- 172.21.15.5:0/1345061754 >> 172.21.15.5:0/1345061754 conn(0x7f20800ff4b0 msgr2=0x7f20801018d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:38:21.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:21.291+0000 7f2087a00640 1 -- 172.21.15.5:0/1345061754 shutdown_connections 2024-09-10T09:38:21.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:21.291+0000 7f2087a00640 1 -- 172.21.15.5:0/1345061754 wait complete. 2024-09-10T09:38:21.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:21.292+0000 7f2087a00640 1 Processor -- start 2024-09-10T09:38:21.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:21.292+0000 7f2087a00640 1 -- start start 2024-09-10T09:38:21.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:21.293+0000 7f2087a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2080103d30 0x7f208019a000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:21.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:21.293+0000 7f2087a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2080104f20 0x7f208019a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:21.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:21.293+0000 7f2087a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f20801058e0 0x7f20801a15b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:21.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:21.293+0000 7f2087a00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f208010f9e0 con 0x7f20801058e0 2024-09-10T09:38:21.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:21.293+0000 7f2087a00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f208010f860 con 0x7f2080104f20 2024-09-10T09:38:21.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:21.293+0000 7f2087a00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f208010fb60 con 0x7f2080103d30 2024-09-10T09:38:21.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:21.293+0000 7f2085a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f20801058e0 0x7f20801a15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:21.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:21.293+0000 7f2085a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f20801058e0 0x7f20801a15b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60784/0 (socket says 172.21.15.5:60784) 2024-09-10T09:38:21.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:21.293+0000 7f2085a00640 1 -- 172.21.15.5:0/3108124606 learned_addr learned my addr 172.21.15.5:0/3108124606 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:38:21.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:21.293+0000 7f207fe00640 1 --2- 172.21.15.5:0/3108124606 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2080104f20 0x7f208019a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:21.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:21.293+0000 7f2085000640 1 --2- 172.21.15.5:0/3108124606 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2080103d30 0x7f208019a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:21.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:21.294+0000 7f2085a00640 1 -- 172.21.15.5:0/3108124606 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2080103d30 msgr2=0x7f208019a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:38:21.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:21.294+0000 7f2085a00640 1 --2- 172.21.15.5:0/3108124606 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2080103d30 0x7f208019a000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:21.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:21.294+0000 7f2085a00640 1 -- 172.21.15.5:0/3108124606 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2080104f20 msgr2=0x7f208019a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:38:21.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:21.294+0000 7f2085a00640 1 --2- 172.21.15.5:0/3108124606 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2080104f20 0x7f208019a540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:21.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:21.294+0000 7f2085a00640 1 -- 172.21.15.5:0/3108124606 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20801a1c20 con 0x7f20801058e0 2024-09-10T09:38:21.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:21.294+0000 7f207fe00640 1 --2- 172.21.15.5:0/3108124606 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2080104f20 0x7f208019a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:38:21.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:21.294+0000 7f2085a00640 1 --2- 172.21.15.5:0/3108124606 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f20801058e0 0x7f20801a15b0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f207800eb10 tx=0x7f207800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:38:21.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:21.294+0000 7f207d600640 1 -- 172.21.15.5:0/3108124606 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2078004b20 con 0x7f20801058e0 2024-09-10T09:38:21.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:21.294+0000 7f207d600640 1 -- 172.21.15.5:0/3108124606 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2078004cc0 con 0x7f20801058e0 2024-09-10T09:38:21.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:21.294+0000 7f207d600640 1 -- 172.21.15.5:0/3108124606 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2078010760 con 0x7f20801058e0 2024-09-10T09:38:21.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:21.294+0000 7f2087a00640 1 -- 172.21.15.5:0/3108124606 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20801a1f10 con 0x7f20801058e0 2024-09-10T09:38:21.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:21.295+0000 7f2087a00640 1 -- 172.21.15.5:0/3108124606 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20801a2380 con 0x7f20801058e0 2024-09-10T09:38:21.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:21.296+0000 7f2087a00640 1 -- 172.21.15.5:0/3108124606 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f208010b140 con 0x7f20801058e0 2024-09-10T09:38:21.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:21.298+0000 7f207d600640 1 -- 172.21.15.5:0/3108124606 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 19) ==== 137883+0+0 (secure 0 0 0) 0x7f2078005020 con 0x7f20801058e0 2024-09-10T09:38:21.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:21.299+0000 7f207d600640 1 --2- 172.21.15.5:0/3108124606 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f20680a2ca0 0x7f20680a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:21.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:21.299+0000 7f207d600640 1 -- 172.21.15.5:0/3108124606 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f20780cfb50 con 0x7f20801058e0 2024-09-10T09:38:21.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:21.299+0000 7f2085000640 1 --2- 172.21.15.5:0/3108124606 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f20680a2ca0 0x7f20680a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:21.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:21.300+0000 7f2085000640 1 --2- 172.21.15.5:0/3108124606 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f20680a2ca0 0x7f20680a5160 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f2080104d80 tx=0x7f2074005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:38:21.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:21.303+0000 7f207d600640 1 -- 172.21.15.5:0/3108124606 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f207809c500 con 0x7f20801058e0 2024-09-10T09:38:21.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:38:20 smithi159 ceph-mon[35513]: pgmap v393: 129 pgs: 129 active+clean; 646 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 85 B/s wr, 9 op/s 2024-09-10T09:38:21.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:21.491+0000 7f2087a00640 1 -- 172.21.15.5:0/3108124606 --> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f2080062630 con 0x7f20680a2ca0 2024-09-10T09:38:21.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:21.499+0000 7f207d600640 1 -- 172.21.15.5:0/3108124606 <== mgr.14150 v2:172.21.15.5:6800/2707348081 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340822 (secure 0 0 0) 0x7f2080062630 con 0x7f20680a2ca0 2024-09-10T09:38:21.505 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-10T09:38:21.507 INFO:teuthology.orchestra.run.smithi005.stderr:dumped all 2024-09-10T09:38:21.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:21.504+0000 7f2087a00640 1 -- 172.21.15.5:0/3108124606 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f20680a2ca0 msgr2=0x7f20680a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:38:21.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:21.504+0000 7f2087a00640 1 --2- 172.21.15.5:0/3108124606 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f20680a2ca0 0x7f20680a5160 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f2080104d80 tx=0x7f2074005f90 comp rx=0 tx=0).stop 2024-09-10T09:38:21.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:21.504+0000 7f2087a00640 1 -- 172.21.15.5:0/3108124606 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f20801058e0 msgr2=0x7f20801a15b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:38:21.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:21.504+0000 7f2087a00640 1 --2- 172.21.15.5:0/3108124606 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f20801058e0 0x7f20801a15b0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f207800eb10 tx=0x7f207800c2d0 comp rx=0 tx=0).stop 2024-09-10T09:38:21.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:21.504+0000 7f2087a00640 1 -- 172.21.15.5:0/3108124606 shutdown_connections 2024-09-10T09:38:21.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:21.504+0000 7f2087a00640 1 --2- 172.21.15.5:0/3108124606 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f20680a2ca0 0x7f20680a5160 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:21.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:21.504+0000 7f2087a00640 1 --2- 172.21.15.5:0/3108124606 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f20801058e0 0x7f20801a15b0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:21.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:21.504+0000 7f2087a00640 1 --2- 172.21.15.5:0/3108124606 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2080104f20 0x7f208019a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:21.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:21.504+0000 7f2087a00640 1 --2- 172.21.15.5:0/3108124606 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2080103d30 0x7f208019a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:21.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:21.504+0000 7f2087a00640 1 -- 172.21.15.5:0/3108124606 >> 172.21.15.5:0/3108124606 conn(0x7f20800ff4b0 msgr2=0x7f2080108310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:38:21.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:21.505+0000 7f2087a00640 1 -- 172.21.15.5:0/3108124606 shutdown_connections 2024-09-10T09:38:21.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:21.505+0000 7f2087a00640 1 -- 172.21.15.5:0/3108124606 wait complete. 2024-09-10T09:38:21.522 INFO:teuthology.orchestra.run.smithi005.stdout:{"pg_ready":true,"pg_map":{"version":393,"stamp":"2024-09-10T09:38:19.570873+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":661272,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1370,"num_read_kb":812,"num_write":443,"num_write_kb":3090,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":1181751,"num_keys_recovered":7,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":862,"ondisk_log_size":862,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":378972,"kb_used_data":13084,"kb_used_omap":18,"kb_used_meta":365677,"kb_avail":1124562852,"statfs":{"total":1151940427776,"available":1151552360448,"internally_reserved":0,"allocated":13398016,"data_stored":8614564,"data_compressed":498435,"data_compressed_allocated":675840,"data_compressed_original":2850816,"omap_allocated":19141,"internal_metadata":374453563},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":110,"num_read_kb":55,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003036"},"pg_stats":[{"pgid":"2.1f","version":"96'14","reported_seq":179,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:38:13.661865+0000","last_change":"2024-09-10T09:35:25.533085+0000","last_active":"2024-09-10T09:38:13.661865+0000","last_peered":"2024-09-10T09:38:13.661865+0000","last_clean":"2024-09-10T09:38:13.661865+0000","last_became_active":"2024-09-10T09:35:25.532729+0000","last_became_peered":"2024-09-10T09:35:25.532729+0000","last_unstale":"2024-09-10T09:38:13.661865+0000","last_undegraded":"2024-09-10T09:38:13.661865+0000","last_fullsized":"2024-09-10T09:38:13.661865+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:55:11.158279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":121,"num_read_kb":121,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"96'5","reported_seq":21,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.545575+0000","last_change":"2024-09-10T09:36:06.767356+0000","last_active":"2024-09-10T09:37:35.545575+0000","last_peered":"2024-09-10T09:37:35.545575+0000","last_clean":"2024-09-10T09:37:35.545575+0000","last_became_active":"2024-09-10T09:36:06.766931+0000","last_became_peered":"2024-09-10T09:36:06.766931+0000","last_unstale":"2024-09-10T09:37:35.545575+0000","last_undegraded":"2024-09-10T09:37:35.545575+0000","last_fullsized":"2024-09-10T09:37:35.545575+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:25:02.777327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,7],"acting":[9,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.527523+0000","last_change":"2024-09-10T09:35:36.629797+0000","last_active":"2024-09-10T09:37:07.527523+0000","last_peered":"2024-09-10T09:37:07.527523+0000","last_clean":"2024-09-10T09:37:07.527523+0000","last_became_active":"2024-09-10T09:35:36.629414+0000","last_became_peered":"2024-09-10T09:35:36.629414+0000","last_unstale":"2024-09-10T09:37:07.527523+0000","last_undegraded":"2024-09-10T09:37:07.527523+0000","last_fullsized":"2024-09-10T09:37:07.527523+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:42:44.610581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"94'2","reported_seq":47,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.524964+0000","last_change":"2024-09-10T09:35:25.852022+0000","last_active":"2024-09-10T09:37:07.524964+0000","last_peered":"2024-09-10T09:37:07.524964+0000","last_clean":"2024-09-10T09:37:07.524964+0000","last_became_active":"2024-09-10T09:35:25.851587+0000","last_became_peered":"2024-09-10T09:35:25.851587+0000","last_unstale":"2024-09-10T09:37:07.524964+0000","last_undegraded":"2024-09-10T09:37:07.524964+0000","last_fullsized":"2024-09-10T09:37:07.524964+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:37:26.085914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"96'6","reported_seq":43,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.556261+0000","last_change":"2024-09-10T09:35:28.579260+0000","last_active":"2024-09-10T09:37:35.556261+0000","last_peered":"2024-09-10T09:37:35.556261+0000","last_clean":"2024-09-10T09:37:35.556261+0000","last_became_active":"2024-09-10T09:35:28.574584+0000","last_became_peered":"2024-09-10T09:35:28.574584+0000","last_unstale":"2024-09-10T09:37:35.556261+0000","last_undegraded":"2024-09-10T09:37:35.556261+0000","last_fullsized":"2024-09-10T09:37:35.556261+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:31:19.277689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.568359+0000","last_change":"2024-09-10T09:36:06.874861+0000","last_active":"2024-09-10T09:37:07.568359+0000","last_peered":"2024-09-10T09:37:07.568359+0000","last_clean":"2024-09-10T09:37:07.568359+0000","last_became_active":"2024-09-10T09:36:06.873930+0000","last_became_peered":"2024-09-10T09:36:06.873930+0000","last_unstale":"2024-09-10T09:37:07.568359+0000","last_undegraded":"2024-09-10T09:37:07.568359+0000","last_fullsized":"2024-09-10T09:37:07.568359+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:35:01.354061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,8,3],"acting":[4,6,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"96'13","reported_seq":58,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:59.271922+0000","last_change":"2024-09-10T09:35:25.534571+0000","last_active":"2024-09-10T09:37:59.271922+0000","last_peered":"2024-09-10T09:37:59.271922+0000","last_clean":"2024-09-10T09:37:59.271922+0000","last_became_active":"2024-09-10T09:35:25.534235+0000","last_became_peered":"2024-09-10T09:35:25.534235+0000","last_unstale":"2024-09-10T09:37:59.271922+0000","last_undegraded":"2024-09-10T09:37:59.271922+0000","last_fullsized":"2024-09-10T09:37:59.271922+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:50:21.447542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"96'9","reported_seq":48,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.553039+0000","last_change":"2024-09-10T09:35:28.565541+0000","last_active":"2024-09-10T09:37:35.553039+0000","last_peered":"2024-09-10T09:37:35.553039+0000","last_clean":"2024-09-10T09:37:35.553039+0000","last_became_active":"2024-09-10T09:35:28.565095+0000","last_became_peered":"2024-09-10T09:35:28.565095+0000","last_unstale":"2024-09-10T09:37:35.553039+0000","last_undegraded":"2024-09-10T09:37:35.553039+0000","last_fullsized":"2024-09-10T09:37:35.553039+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:29:31.285401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.568600+0000","last_change":"2024-09-10T09:35:36.629277+0000","last_active":"2024-09-10T09:37:07.568600+0000","last_peered":"2024-09-10T09:37:07.568600+0000","last_clean":"2024-09-10T09:37:07.568600+0000","last_became_active":"2024-09-10T09:35:36.628687+0000","last_became_peered":"2024-09-10T09:35:36.628687+0000","last_unstale":"2024-09-10T09:37:07.568600+0000","last_undegraded":"2024-09-10T09:37:07.568600+0000","last_fullsized":"2024-09-10T09:37:07.568600+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:34:33.628827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"94'5","reported_seq":50,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.530161+0000","last_change":"2024-09-10T09:35:25.536137+0000","last_active":"2024-09-10T09:37:07.530161+0000","last_peered":"2024-09-10T09:37:07.530161+0000","last_clean":"2024-09-10T09:37:07.530161+0000","last_became_active":"2024-09-10T09:35:25.535728+0000","last_became_peered":"2024-09-10T09:35:25.535728+0000","last_unstale":"2024-09-10T09:37:07.530161+0000","last_undegraded":"2024-09-10T09:37:07.530161+0000","last_fullsized":"2024-09-10T09:37:07.530161+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:36:04.829901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"96'15","reported_seq":60,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.547598+0000","last_change":"2024-09-10T09:36:06.874425+0000","last_active":"2024-09-10T09:37:35.547598+0000","last_peered":"2024-09-10T09:37:35.547598+0000","last_clean":"2024-09-10T09:37:35.547598+0000","last_became_active":"2024-09-10T09:36:06.874050+0000","last_became_peered":"2024-09-10T09:36:06.874050+0000","last_unstale":"2024-09-10T09:37:35.547598+0000","last_undegraded":"2024-09-10T09:37:35.547598+0000","last_fullsized":"2024-09-10T09:37:35.547598+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:30:51.200335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,1],"acting":[5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.568073+0000","last_change":"2024-09-10T09:35:36.615314+0000","last_active":"2024-09-10T09:37:07.568073+0000","last_peered":"2024-09-10T09:37:07.568073+0000","last_clean":"2024-09-10T09:37:07.568073+0000","last_became_active":"2024-09-10T09:35:36.614820+0000","last_became_peered":"2024-09-10T09:35:36.614820+0000","last_unstale":"2024-09-10T09:37:07.568073+0000","last_undegraded":"2024-09-10T09:37:07.568073+0000","last_fullsized":"2024-09-10T09:37:07.568073+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:51:16.897935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"96'5","reported_seq":42,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.556084+0000","last_change":"2024-09-10T09:35:28.572346+0000","last_active":"2024-09-10T09:37:35.556084+0000","last_peered":"2024-09-10T09:37:35.556084+0000","last_clean":"2024-09-10T09:37:35.556084+0000","last_became_active":"2024-09-10T09:35:28.571839+0000","last_became_peered":"2024-09-10T09:35:28.571839+0000","last_unstale":"2024-09-10T09:37:35.556084+0000","last_undegraded":"2024-09-10T09:37:35.556084+0000","last_fullsized":"2024-09-10T09:37:35.556084+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:41:29.662441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"94'4","reported_seq":49,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.529918+0000","last_change":"2024-09-10T09:35:25.538747+0000","last_active":"2024-09-10T09:37:07.529918+0000","last_peered":"2024-09-10T09:37:07.529918+0000","last_clean":"2024-09-10T09:37:07.529918+0000","last_became_active":"2024-09-10T09:35:25.538330+0000","last_became_peered":"2024-09-10T09:35:25.538330+0000","last_unstale":"2024-09-10T09:37:07.529918+0000","last_undegraded":"2024-09-10T09:37:07.529918+0000","last_fullsized":"2024-09-10T09:37:07.529918+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:15:08.340688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.568436+0000","last_change":"2024-09-10T09:35:37.270106+0000","last_active":"2024-09-10T09:37:07.568436+0000","last_peered":"2024-09-10T09:37:07.568436+0000","last_clean":"2024-09-10T09:37:07.568436+0000","last_became_active":"2024-09-10T09:35:37.269575+0000","last_became_peered":"2024-09-10T09:35:37.269575+0000","last_unstale":"2024-09-10T09:37:07.568436+0000","last_undegraded":"2024-09-10T09:37:07.568436+0000","last_fullsized":"2024-09-10T09:37:07.568436+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:35:00.414602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"96'8","reported_seq":47,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.555784+0000","last_change":"2024-09-10T09:35:28.569230+0000","last_active":"2024-09-10T09:37:35.555784+0000","last_peered":"2024-09-10T09:37:35.555784+0000","last_clean":"2024-09-10T09:37:35.555784+0000","last_became_active":"2024-09-10T09:35:28.568266+0000","last_became_peered":"2024-09-10T09:35:28.568266+0000","last_unstale":"2024-09-10T09:37:35.555784+0000","last_undegraded":"2024-09-10T09:37:35.555784+0000","last_fullsized":"2024-09-10T09:37:35.555784+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:49:43.730686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"94'3","reported_seq":17,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.544707+0000","last_change":"2024-09-10T09:37:07.544298+0000","last_active":"2024-09-10T09:37:07.544707+0000","last_peered":"2024-09-10T09:37:07.544707+0000","last_clean":"2024-09-10T09:37:07.544707+0000","last_became_active":"2024-09-10T09:37:07.531815+0000","last_became_peered":"2024-09-10T09:37:07.531815+0000","last_unstale":"2024-09-10T09:37:07.544707+0000","last_undegraded":"2024-09-10T09:37:07.544707+0000","last_fullsized":"2024-09-10T09:37:07.544707+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:18:10.091895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"94'1","reported_seq":37,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.568484+0000","last_change":"2024-09-10T09:36:06.875056+0000","last_active":"2024-09-10T09:37:07.568484+0000","last_peered":"2024-09-10T09:37:07.568484+0000","last_clean":"2024-09-10T09:37:07.568484+0000","last_became_active":"2024-09-10T09:36:06.874400+0000","last_became_peered":"2024-09-10T09:36:06.874400+0000","last_unstale":"2024-09-10T09:37:07.568484+0000","last_undegraded":"2024-09-10T09:37:07.568484+0000","last_fullsized":"2024-09-10T09:37:07.568484+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:21:06.300712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,7,8],"acting":[4,10,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"96'9","reported_seq":48,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.552565+0000","last_change":"2024-09-10T09:35:28.571319+0000","last_active":"2024-09-10T09:37:35.552565+0000","last_peered":"2024-09-10T09:37:35.552565+0000","last_clean":"2024-09-10T09:37:35.552565+0000","last_became_active":"2024-09-10T09:35:28.570972+0000","last_became_peered":"2024-09-10T09:35:28.570972+0000","last_unstale":"2024-09-10T09:37:35.552565+0000","last_undegraded":"2024-09-10T09:37:35.552565+0000","last_fullsized":"2024-09-10T09:37:35.552565+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:45:19.300535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"94'7","reported_seq":52,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.524483+0000","last_change":"2024-09-10T09:35:25.530489+0000","last_active":"2024-09-10T09:37:07.524483+0000","last_peered":"2024-09-10T09:37:07.524483+0000","last_clean":"2024-09-10T09:37:07.524483+0000","last_became_active":"2024-09-10T09:35:25.530197+0000","last_became_peered":"2024-09-10T09:35:25.530197+0000","last_unstale":"2024-09-10T09:37:07.524483+0000","last_undegraded":"2024-09-10T09:37:07.524483+0000","last_fullsized":"2024-09-10T09:37:07.524483+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:28:57.539569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.527650+0000","last_change":"2024-09-10T09:35:36.630507+0000","last_active":"2024-09-10T09:37:07.527650+0000","last_peered":"2024-09-10T09:37:07.527650+0000","last_clean":"2024-09-10T09:37:07.527650+0000","last_became_active":"2024-09-10T09:35:36.630052+0000","last_became_peered":"2024-09-10T09:35:36.630052+0000","last_unstale":"2024-09-10T09:37:07.527650+0000","last_undegraded":"2024-09-10T09:37:07.527650+0000","last_fullsized":"2024-09-10T09:37:07.527650+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:13:42.691382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"96'2","reported_seq":41,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.532487+0000","last_change":"2024-09-10T09:35:28.565650+0000","last_active":"2024-09-10T09:37:35.532487+0000","last_peered":"2024-09-10T09:37:35.532487+0000","last_clean":"2024-09-10T09:37:35.532487+0000","last_became_active":"2024-09-10T09:35:28.565303+0000","last_became_peered":"2024-09-10T09:35:28.565303+0000","last_unstale":"2024-09-10T09:37:35.532487+0000","last_undegraded":"2024-09-10T09:37:35.532487+0000","last_fullsized":"2024-09-10T09:37:35.532487+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:48:08.965487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"94'2","reported_seq":46,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.574507+0000","last_change":"2024-09-10T09:35:25.851326+0000","last_active":"2024-09-10T09:37:07.574507+0000","last_peered":"2024-09-10T09:37:07.574507+0000","last_clean":"2024-09-10T09:37:07.574507+0000","last_became_active":"2024-09-10T09:35:25.850981+0000","last_became_peered":"2024-09-10T09:35:25.850981+0000","last_unstale":"2024-09-10T09:37:07.574507+0000","last_undegraded":"2024-09-10T09:37:07.574507+0000","last_fullsized":"2024-09-10T09:37:07.574507+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:50:28.877840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.568510+0000","last_change":"2024-09-10T09:35:37.270138+0000","last_active":"2024-09-10T09:37:07.568510+0000","last_peered":"2024-09-10T09:37:07.568510+0000","last_clean":"2024-09-10T09:37:07.568510+0000","last_became_active":"2024-09-10T09:35:37.269663+0000","last_became_peered":"2024-09-10T09:35:37.269663+0000","last_unstale":"2024-09-10T09:37:07.568510+0000","last_undegraded":"2024-09-10T09:37:07.568510+0000","last_fullsized":"2024-09-10T09:37:07.568510+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:18:00.105970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"96'7","reported_seq":44,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.554144+0000","last_change":"2024-09-10T09:35:28.576197+0000","last_active":"2024-09-10T09:37:35.554144+0000","last_peered":"2024-09-10T09:37:35.554144+0000","last_clean":"2024-09-10T09:37:35.554144+0000","last_became_active":"2024-09-10T09:35:28.575859+0000","last_became_peered":"2024-09-10T09:35:28.575859+0000","last_unstale":"2024-09-10T09:37:35.554144+0000","last_undegraded":"2024-09-10T09:37:35.554144+0000","last_fullsized":"2024-09-10T09:37:35.554144+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:09:39.458761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"94'7","reported_seq":51,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.568805+0000","last_change":"2024-09-10T09:35:25.990587+0000","last_active":"2024-09-10T09:37:07.568805+0000","last_peered":"2024-09-10T09:37:07.568805+0000","last_clean":"2024-09-10T09:37:07.568805+0000","last_became_active":"2024-09-10T09:35:25.990302+0000","last_became_peered":"2024-09-10T09:35:25.990302+0000","last_unstale":"2024-09-10T09:37:07.568805+0000","last_undegraded":"2024-09-10T09:37:07.568805+0000","last_fullsized":"2024-09-10T09:37:07.568805+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:18:41.244441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.527768+0000","last_change":"2024-09-10T09:35:36.612983+0000","last_active":"2024-09-10T09:37:07.527768+0000","last_peered":"2024-09-10T09:37:07.527768+0000","last_clean":"2024-09-10T09:37:07.527768+0000","last_became_active":"2024-09-10T09:35:36.612542+0000","last_became_peered":"2024-09-10T09:35:36.612542+0000","last_unstale":"2024-09-10T09:37:07.527768+0000","last_undegraded":"2024-09-10T09:37:07.527768+0000","last_fullsized":"2024-09-10T09:37:07.527768+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:50:16.298195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"96'5","reported_seq":44,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.554267+0000","last_change":"2024-09-10T09:35:28.564121+0000","last_active":"2024-09-10T09:37:35.554267+0000","last_peered":"2024-09-10T09:37:35.554267+0000","last_clean":"2024-09-10T09:37:35.554267+0000","last_became_active":"2024-09-10T09:35:28.562780+0000","last_became_peered":"2024-09-10T09:35:28.562780+0000","last_unstale":"2024-09-10T09:37:35.554267+0000","last_undegraded":"2024-09-10T09:37:35.554267+0000","last_fullsized":"2024-09-10T09:37:35.554267+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:53:15.690478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"96'16","reported_seq":76,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:34.702676+0000","last_change":"2024-09-10T09:37:07.541031+0000","last_active":"2024-09-10T09:37:34.702676+0000","last_peered":"2024-09-10T09:37:34.702676+0000","last_clean":"2024-09-10T09:37:34.702676+0000","last_became_active":"2024-09-10T09:37:07.529577+0000","last_became_peered":"2024-09-10T09:37:07.529577+0000","last_unstale":"2024-09-10T09:37:34.702676+0000","last_undegraded":"2024-09-10T09:37:34.702676+0000","last_fullsized":"2024-09-10T09:37:34.702676+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:14:13.440346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":29,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":7,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.524145+0000","last_change":"2024-09-10T09:35:37.270888+0000","last_active":"2024-09-10T09:37:07.524145+0000","last_peered":"2024-09-10T09:37:07.524145+0000","last_clean":"2024-09-10T09:37:07.524145+0000","last_became_active":"2024-09-10T09:35:37.270229+0000","last_became_peered":"2024-09-10T09:35:37.270229+0000","last_unstale":"2024-09-10T09:37:07.524145+0000","last_undegraded":"2024-09-10T09:37:07.524145+0000","last_fullsized":"2024-09-10T09:37:07.524145+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:03:54.223463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"96'9","reported_seq":48,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.547904+0000","last_change":"2024-09-10T09:35:28.572799+0000","last_active":"2024-09-10T09:37:35.547904+0000","last_peered":"2024-09-10T09:37:35.547904+0000","last_clean":"2024-09-10T09:37:35.547904+0000","last_became_active":"2024-09-10T09:35:28.572310+0000","last_became_peered":"2024-09-10T09:35:28.572310+0000","last_unstale":"2024-09-10T09:37:35.547904+0000","last_undegraded":"2024-09-10T09:37:35.547904+0000","last_fullsized":"2024-09-10T09:37:35.547904+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:33:51.446766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"94'4","reported_seq":49,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.530456+0000","last_change":"2024-09-10T09:35:25.851868+0000","last_active":"2024-09-10T09:37:07.530456+0000","last_peered":"2024-09-10T09:37:07.530456+0000","last_clean":"2024-09-10T09:37:07.530456+0000","last_became_active":"2024-09-10T09:35:25.851447+0000","last_became_peered":"2024-09-10T09:35:25.851447+0000","last_unstale":"2024-09-10T09:37:07.530456+0000","last_undegraded":"2024-09-10T09:37:07.530456+0000","last_fullsized":"2024-09-10T09:37:07.530456+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:15:57.706690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.526545+0000","last_change":"2024-09-10T09:35:36.619459+0000","last_active":"2024-09-10T09:37:07.526545+0000","last_peered":"2024-09-10T09:37:07.526545+0000","last_clean":"2024-09-10T09:37:07.526545+0000","last_became_active":"2024-09-10T09:35:36.619023+0000","last_became_peered":"2024-09-10T09:35:36.619023+0000","last_unstale":"2024-09-10T09:37:07.526545+0000","last_undegraded":"2024-09-10T09:37:07.526545+0000","last_fullsized":"2024-09-10T09:37:07.526545+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:07:32.999231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"96'4","reported_seq":43,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.548010+0000","last_change":"2024-09-10T09:35:28.575668+0000","last_active":"2024-09-10T09:37:35.548010+0000","last_peered":"2024-09-10T09:37:35.548010+0000","last_clean":"2024-09-10T09:37:35.548010+0000","last_became_active":"2024-09-10T09:35:28.575425+0000","last_became_peered":"2024-09-10T09:35:28.575425+0000","last_unstale":"2024-09-10T09:37:35.548010+0000","last_undegraded":"2024-09-10T09:37:35.548010+0000","last_fullsized":"2024-09-10T09:37:35.548010+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:13:27.120208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"94'1","reported_seq":45,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.528018+0000","last_change":"2024-09-10T09:35:25.854569+0000","last_active":"2024-09-10T09:37:07.528018+0000","last_peered":"2024-09-10T09:37:07.528018+0000","last_clean":"2024-09-10T09:37:07.528018+0000","last_became_active":"2024-09-10T09:35:25.854152+0000","last_became_peered":"2024-09-10T09:35:25.854152+0000","last_unstale":"2024-09-10T09:37:07.528018+0000","last_undegraded":"2024-09-10T09:37:07.528018+0000","last_fullsized":"2024-09-10T09:37:07.528018+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:27:15.584707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.529957+0000","last_change":"2024-09-10T09:35:37.270517+0000","last_active":"2024-09-10T09:37:07.529957+0000","last_peered":"2024-09-10T09:37:07.529957+0000","last_clean":"2024-09-10T09:37:07.529957+0000","last_became_active":"2024-09-10T09:35:37.269793+0000","last_became_peered":"2024-09-10T09:35:37.269793+0000","last_unstale":"2024-09-10T09:37:07.529957+0000","last_undegraded":"2024-09-10T09:37:07.529957+0000","last_fullsized":"2024-09-10T09:37:07.529957+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:28:58.272848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"96'7","reported_seq":52,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.556473+0000","last_change":"2024-09-10T09:37:07.762585+0000","last_active":"2024-09-10T09:37:35.556473+0000","last_peered":"2024-09-10T09:37:35.556473+0000","last_clean":"2024-09-10T09:37:35.556473+0000","last_became_active":"2024-09-10T09:37:07.762250+0000","last_became_peered":"2024-09-10T09:37:07.762250+0000","last_unstale":"2024-09-10T09:37:35.556473+0000","last_undegraded":"2024-09-10T09:37:35.556473+0000","last_fullsized":"2024-09-10T09:37:35.556473+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:38:01.751512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"96'7","reported_seq":172,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:38:12.661207+0000","last_change":"2024-09-10T09:35:25.531062+0000","last_active":"2024-09-10T09:38:12.661207+0000","last_peered":"2024-09-10T09:38:12.661207+0000","last_clean":"2024-09-10T09:38:12.661207+0000","last_became_active":"2024-09-10T09:35:25.530005+0000","last_became_peered":"2024-09-10T09:35:25.530005+0000","last_unstale":"2024-09-10T09:38:12.661207+0000","last_undegraded":"2024-09-10T09:38:12.661207+0000","last_fullsized":"2024-09-10T09:38:12.661207+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:42:35.330312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1122,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":122,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.538766+0000","last_change":"2024-09-10T09:35:36.623099+0000","last_active":"2024-09-10T09:37:07.538766+0000","last_peered":"2024-09-10T09:37:07.538766+0000","last_clean":"2024-09-10T09:37:07.538766+0000","last_became_active":"2024-09-10T09:35:36.622716+0000","last_became_peered":"2024-09-10T09:35:36.622716+0000","last_unstale":"2024-09-10T09:37:07.538766+0000","last_undegraded":"2024-09-10T09:37:07.538766+0000","last_fullsized":"2024-09-10T09:37:07.538766+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:10:38.495164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"96'8","reported_seq":53,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.543805+0000","last_change":"2024-09-10T09:36:06.769405+0000","last_active":"2024-09-10T09:37:35.543805+0000","last_peered":"2024-09-10T09:37:35.543805+0000","last_clean":"2024-09-10T09:37:35.543805+0000","last_became_active":"2024-09-10T09:36:06.769042+0000","last_became_peered":"2024-09-10T09:36:06.769042+0000","last_unstale":"2024-09-10T09:37:35.543805+0000","last_undegraded":"2024-09-10T09:37:35.543805+0000","last_fullsized":"2024-09-10T09:37:35.543805+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:01:49.340765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,8],"acting":[5,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"94'5","reported_seq":174,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:38:16.666205+0000","last_change":"2024-09-10T09:35:25.851430+0000","last_active":"2024-09-10T09:38:16.666205+0000","last_peered":"2024-09-10T09:38:16.666205+0000","last_clean":"2024-09-10T09:38:16.666205+0000","last_became_active":"2024-09-10T09:35:25.851067+0000","last_became_peered":"2024-09-10T09:35:25.851067+0000","last_unstale":"2024-09-10T09:38:16.666205+0000","last_undegraded":"2024-09-10T09:38:16.666205+0000","last_fullsized":"2024-09-10T09:38:16.666205+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:55:58.756207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":125,"num_read_kb":125,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.538676+0000","last_change":"2024-09-10T09:35:37.270361+0000","last_active":"2024-09-10T09:37:07.538676+0000","last_peered":"2024-09-10T09:37:07.538676+0000","last_clean":"2024-09-10T09:37:07.538676+0000","last_became_active":"2024-09-10T09:35:37.269632+0000","last_became_peered":"2024-09-10T09:35:37.269632+0000","last_unstale":"2024-09-10T09:37:07.538676+0000","last_undegraded":"2024-09-10T09:37:07.538676+0000","last_fullsized":"2024-09-10T09:37:07.538676+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:50:34.360497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"96'5","reported_seq":44,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.548275+0000","last_change":"2024-09-10T09:35:28.571064+0000","last_active":"2024-09-10T09:37:35.548275+0000","last_peered":"2024-09-10T09:37:35.548275+0000","last_clean":"2024-09-10T09:37:35.548275+0000","last_became_active":"2024-09-10T09:35:28.570691+0000","last_became_peered":"2024-09-10T09:35:28.570691+0000","last_unstale":"2024-09-10T09:37:35.548275+0000","last_undegraded":"2024-09-10T09:37:35.548275+0000","last_fullsized":"2024-09-10T09:37:35.548275+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:58:33.162501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"94'4","reported_seq":57,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.572759+0000","last_change":"2024-09-10T09:37:07.572247+0000","last_active":"2024-09-10T09:37:07.572759+0000","last_peered":"2024-09-10T09:37:07.572759+0000","last_clean":"2024-09-10T09:37:07.572759+0000","last_became_active":"2024-09-10T09:37:07.569410+0000","last_became_peered":"2024-09-10T09:37:07.569410+0000","last_unstale":"2024-09-10T09:37:07.572759+0000","last_undegraded":"2024-09-10T09:37:07.572759+0000","last_fullsized":"2024-09-10T09:37:07.572759+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:05:09.037084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,4],"acting":[7,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.532723+0000","last_change":"2024-09-10T09:35:37.270358+0000","last_active":"2024-09-10T09:37:07.532723+0000","last_peered":"2024-09-10T09:37:07.532723+0000","last_clean":"2024-09-10T09:37:07.532723+0000","last_became_active":"2024-09-10T09:35:37.269822+0000","last_became_peered":"2024-09-10T09:35:37.269822+0000","last_unstale":"2024-09-10T09:37:07.532723+0000","last_undegraded":"2024-09-10T09:37:07.532723+0000","last_fullsized":"2024-09-10T09:37:07.532723+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:19:48.932596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"96'8","reported_seq":47,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.553803+0000","last_change":"2024-09-10T09:35:28.576244+0000","last_active":"2024-09-10T09:37:35.553803+0000","last_peered":"2024-09-10T09:37:35.553803+0000","last_clean":"2024-09-10T09:37:35.553803+0000","last_became_active":"2024-09-10T09:35:28.571801+0000","last_became_peered":"2024-09-10T09:35:28.571801+0000","last_unstale":"2024-09-10T09:37:35.553803+0000","last_undegraded":"2024-09-10T09:37:35.553803+0000","last_fullsized":"2024-09-10T09:37:35.553803+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:11:21.742220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"94'6","reported_seq":52,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.568754+0000","last_change":"2024-09-10T09:35:25.537118+0000","last_active":"2024-09-10T09:37:07.568754+0000","last_peered":"2024-09-10T09:37:07.568754+0000","last_clean":"2024-09-10T09:37:07.568754+0000","last_became_active":"2024-09-10T09:35:25.536718+0000","last_became_peered":"2024-09-10T09:35:25.536718+0000","last_unstale":"2024-09-10T09:37:07.568754+0000","last_undegraded":"2024-09-10T09:37:07.568754+0000","last_fullsized":"2024-09-10T09:37:07.568754+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:16:32.837476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":361,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.528720+0000","last_change":"2024-09-10T09:34:07.350825+0000","last_active":"2024-09-10T09:37:07.528720+0000","last_peered":"2024-09-10T09:37:07.528720+0000","last_clean":"2024-09-10T09:37:07.528720+0000","last_became_active":"2024-09-10T09:34:07.350325+0000","last_became_peered":"2024-09-10T09:34:07.350325+0000","last_unstale":"2024-09-10T09:37:07.528720+0000","last_undegraded":"2024-09-10T09:37:07.528720+0000","last_fullsized":"2024-09-10T09:37:07.528720+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:30:26.408262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:30:26.408262+0000","last_clean_scrub_stamp":"2024-09-10T09:30:26.408262+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:58:04.920443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"94'1","reported_seq":30,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.529308+0000","last_change":"2024-09-10T09:35:36.632941+0000","last_active":"2024-09-10T09:37:07.529308+0000","last_peered":"2024-09-10T09:37:07.529308+0000","last_clean":"2024-09-10T09:37:07.529308+0000","last_became_active":"2024-09-10T09:35:36.632527+0000","last_became_peered":"2024-09-10T09:35:36.632527+0000","last_unstale":"2024-09-10T09:37:07.529308+0000","last_undegraded":"2024-09-10T09:37:07.529308+0000","last_fullsized":"2024-09-10T09:37:07.529308+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:33:21.733772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"96'7","reported_seq":44,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.556036+0000","last_change":"2024-09-10T09:35:28.572010+0000","last_active":"2024-09-10T09:37:35.556036+0000","last_peered":"2024-09-10T09:37:35.556036+0000","last_clean":"2024-09-10T09:37:35.556036+0000","last_became_active":"2024-09-10T09:35:28.571623+0000","last_became_peered":"2024-09-10T09:35:28.571623+0000","last_unstale":"2024-09-10T09:37:35.556036+0000","last_undegraded":"2024-09-10T09:37:35.556036+0000","last_fullsized":"2024-09-10T09:37:35.556036+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:00:49.328663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"94'5","reported_seq":173,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:38:14.664651+0000","last_change":"2024-09-10T09:35:25.990328+0000","last_active":"2024-09-10T09:38:14.664651+0000","last_peered":"2024-09-10T09:38:14.664651+0000","last_clean":"2024-09-10T09:38:14.664651+0000","last_became_active":"2024-09-10T09:35:25.989946+0000","last_became_peered":"2024-09-10T09:35:25.989946+0000","last_unstale":"2024-09-10T09:38:14.664651+0000","last_undegraded":"2024-09-10T09:38:14.664651+0000","last_fullsized":"2024-09-10T09:38:14.664651+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:51:16.069913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":124,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.574499+0000","last_change":"2024-09-10T09:35:36.619013+0000","last_active":"2024-09-10T09:37:07.574499+0000","last_peered":"2024-09-10T09:37:07.574499+0000","last_clean":"2024-09-10T09:37:07.574499+0000","last_became_active":"2024-09-10T09:35:36.618506+0000","last_became_peered":"2024-09-10T09:35:36.618506+0000","last_unstale":"2024-09-10T09:37:07.574499+0000","last_undegraded":"2024-09-10T09:37:07.574499+0000","last_fullsized":"2024-09-10T09:37:07.574499+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:57:53.580569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"96'6","reported_seq":49,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.553955+0000","last_change":"2024-09-10T09:36:06.768573+0000","last_active":"2024-09-10T09:37:35.553955+0000","last_peered":"2024-09-10T09:37:35.553955+0000","last_clean":"2024-09-10T09:37:35.553955+0000","last_became_active":"2024-09-10T09:36:06.768228+0000","last_became_peered":"2024-09-10T09:36:06.768228+0000","last_unstale":"2024-09-10T09:37:35.553955+0000","last_undegraded":"2024-09-10T09:37:35.553955+0000","last_fullsized":"2024-09-10T09:37:35.553955+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:57:38.310713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"94'3","reported_seq":48,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.533010+0000","last_change":"2024-09-10T09:35:25.528795+0000","last_active":"2024-09-10T09:37:07.533010+0000","last_peered":"2024-09-10T09:37:07.533010+0000","last_clean":"2024-09-10T09:37:07.533010+0000","last_became_active":"2024-09-10T09:35:25.528475+0000","last_became_peered":"2024-09-10T09:35:25.528475+0000","last_unstale":"2024-09-10T09:37:07.533010+0000","last_undegraded":"2024-09-10T09:37:07.533010+0000","last_fullsized":"2024-09-10T09:37:07.533010+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:16:06.999186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.761758+0000","last_change":"2024-09-10T09:35:36.633109+0000","last_active":"2024-09-10T09:37:07.761758+0000","last_peered":"2024-09-10T09:37:07.761758+0000","last_clean":"2024-09-10T09:37:07.761758+0000","last_became_active":"2024-09-10T09:35:36.632682+0000","last_became_peered":"2024-09-10T09:35:36.632682+0000","last_unstale":"2024-09-10T09:37:07.761758+0000","last_undegraded":"2024-09-10T09:37:07.761758+0000","last_fullsized":"2024-09-10T09:37:07.761758+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:51:14.372649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"96'9","reported_seq":46,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.552249+0000","last_change":"2024-09-10T09:35:28.562103+0000","last_active":"2024-09-10T09:37:35.552249+0000","last_peered":"2024-09-10T09:37:35.552249+0000","last_clean":"2024-09-10T09:37:35.552249+0000","last_became_active":"2024-09-10T09:35:28.561765+0000","last_became_peered":"2024-09-10T09:35:28.561765+0000","last_unstale":"2024-09-10T09:37:35.552249+0000","last_undegraded":"2024-09-10T09:37:35.552249+0000","last_fullsized":"2024-09-10T09:37:35.552249+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:35:35.400569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"94'5","reported_seq":49,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.539243+0000","last_change":"2024-09-10T09:35:25.538536+0000","last_active":"2024-09-10T09:37:07.539243+0000","last_peered":"2024-09-10T09:37:07.539243+0000","last_clean":"2024-09-10T09:37:07.539243+0000","last_became_active":"2024-09-10T09:35:25.537944+0000","last_became_peered":"2024-09-10T09:35:25.537944+0000","last_unstale":"2024-09-10T09:37:07.539243+0000","last_undegraded":"2024-09-10T09:37:07.539243+0000","last_fullsized":"2024-09-10T09:37:07.539243+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:01:19.385381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":17,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.529928+0000","last_change":"2024-09-10T09:36:06.769534+0000","last_active":"2024-09-10T09:37:07.529928+0000","last_peered":"2024-09-10T09:37:07.529928+0000","last_clean":"2024-09-10T09:37:07.529928+0000","last_became_active":"2024-09-10T09:36:06.769015+0000","last_became_peered":"2024-09-10T09:36:06.769015+0000","last_unstale":"2024-09-10T09:37:07.529928+0000","last_undegraded":"2024-09-10T09:37:07.529928+0000","last_fullsized":"2024-09-10T09:37:07.529928+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:19:26.369737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7,1],"acting":[8,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"96'5","reported_seq":44,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.539962+0000","last_change":"2024-09-10T09:35:28.566735+0000","last_active":"2024-09-10T09:37:35.539962+0000","last_peered":"2024-09-10T09:37:35.539962+0000","last_clean":"2024-09-10T09:37:35.539962+0000","last_became_active":"2024-09-10T09:35:28.566400+0000","last_became_peered":"2024-09-10T09:35:28.566400+0000","last_unstale":"2024-09-10T09:37:35.539962+0000","last_undegraded":"2024-09-10T09:37:35.539962+0000","last_fullsized":"2024-09-10T09:37:35.539962+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:18:53.337816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"94'2","reported_seq":47,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.530569+0000","last_change":"2024-09-10T09:35:25.535783+0000","last_active":"2024-09-10T09:37:07.530569+0000","last_peered":"2024-09-10T09:37:07.530569+0000","last_clean":"2024-09-10T09:37:07.530569+0000","last_became_active":"2024-09-10T09:35:25.535411+0000","last_became_peered":"2024-09-10T09:35:25.535411+0000","last_unstale":"2024-09-10T09:37:07.530569+0000","last_undegraded":"2024-09-10T09:37:07.530569+0000","last_fullsized":"2024-09-10T09:37:07.530569+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:48:09.239083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.531208+0000","last_change":"2024-09-10T09:35:36.619380+0000","last_active":"2024-09-10T09:37:07.531208+0000","last_peered":"2024-09-10T09:37:07.531208+0000","last_clean":"2024-09-10T09:37:07.531208+0000","last_became_active":"2024-09-10T09:35:36.618953+0000","last_became_peered":"2024-09-10T09:35:36.618953+0000","last_unstale":"2024-09-10T09:37:07.531208+0000","last_undegraded":"2024-09-10T09:37:07.531208+0000","last_fullsized":"2024-09-10T09:37:07.531208+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:40:25.181408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"96'6","reported_seq":45,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.546928+0000","last_change":"2024-09-10T09:35:28.571396+0000","last_active":"2024-09-10T09:37:35.546928+0000","last_peered":"2024-09-10T09:37:35.546928+0000","last_clean":"2024-09-10T09:37:35.546928+0000","last_became_active":"2024-09-10T09:35:28.571063+0000","last_became_peered":"2024-09-10T09:35:28.571063+0000","last_unstale":"2024-09-10T09:37:35.546928+0000","last_undegraded":"2024-09-10T09:37:35.546928+0000","last_fullsized":"2024-09-10T09:37:35.546928+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:38:17.965133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"94'17","reported_seq":62,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.530060+0000","last_change":"2024-09-10T09:35:25.851773+0000","last_active":"2024-09-10T09:37:07.530060+0000","last_peered":"2024-09-10T09:37:07.530060+0000","last_clean":"2024-09-10T09:37:07.530060+0000","last_became_active":"2024-09-10T09:35:25.851384+0000","last_became_peered":"2024-09-10T09:35:25.851384+0000","last_unstale":"2024-09-10T09:37:07.530060+0000","last_undegraded":"2024-09-10T09:37:07.530060+0000","last_fullsized":"2024-09-10T09:37:07.530060+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:03:32.338643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.539186+0000","last_change":"2024-09-10T09:35:36.620971+0000","last_active":"2024-09-10T09:37:07.539186+0000","last_peered":"2024-09-10T09:37:07.539186+0000","last_clean":"2024-09-10T09:37:07.539186+0000","last_became_active":"2024-09-10T09:35:36.620343+0000","last_became_peered":"2024-09-10T09:35:36.620343+0000","last_unstale":"2024-09-10T09:37:07.539186+0000","last_undegraded":"2024-09-10T09:37:07.539186+0000","last_fullsized":"2024-09-10T09:37:07.539186+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:17:19.517882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"96'6","reported_seq":43,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.553818+0000","last_change":"2024-09-10T09:35:28.565643+0000","last_active":"2024-09-10T09:37:35.553818+0000","last_peered":"2024-09-10T09:37:35.553818+0000","last_clean":"2024-09-10T09:37:35.553818+0000","last_became_active":"2024-09-10T09:35:28.564217+0000","last_became_peered":"2024-09-10T09:35:28.564217+0000","last_unstale":"2024-09-10T09:37:35.553818+0000","last_undegraded":"2024-09-10T09:37:35.553818+0000","last_fullsized":"2024-09-10T09:37:35.553818+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:53:34.562690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"94'5","reported_seq":174,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:38:16.666733+0000","last_change":"2024-09-10T09:35:25.534167+0000","last_active":"2024-09-10T09:38:16.666733+0000","last_peered":"2024-09-10T09:38:16.666733+0000","last_clean":"2024-09-10T09:38:16.666733+0000","last_became_active":"2024-09-10T09:35:25.533719+0000","last_became_peered":"2024-09-10T09:35:25.533719+0000","last_unstale":"2024-09-10T09:38:16.666733+0000","last_undegraded":"2024-09-10T09:38:16.666733+0000","last_fullsized":"2024-09-10T09:38:16.666733+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:00:03.537355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":127,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.527733+0000","last_change":"2024-09-10T09:35:36.628716+0000","last_active":"2024-09-10T09:37:07.527733+0000","last_peered":"2024-09-10T09:37:07.527733+0000","last_clean":"2024-09-10T09:37:07.527733+0000","last_became_active":"2024-09-10T09:35:36.628085+0000","last_became_peered":"2024-09-10T09:35:36.628085+0000","last_unstale":"2024-09-10T09:37:07.527733+0000","last_undegraded":"2024-09-10T09:37:07.527733+0000","last_fullsized":"2024-09-10T09:37:07.527733+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:45:01.279403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"96'15","reported_seq":52,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.557076+0000","last_change":"2024-09-10T09:35:28.566701+0000","last_active":"2024-09-10T09:37:35.557076+0000","last_peered":"2024-09-10T09:37:35.557076+0000","last_clean":"2024-09-10T09:37:35.557076+0000","last_became_active":"2024-09-10T09:35:28.566391+0000","last_became_peered":"2024-09-10T09:35:28.566391+0000","last_unstale":"2024-09-10T09:37:35.557076+0000","last_undegraded":"2024-09-10T09:37:35.557076+0000","last_fullsized":"2024-09-10T09:37:35.557076+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:57:59.918706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"94'5","reported_seq":50,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.531896+0000","last_change":"2024-09-10T09:35:25.532510+0000","last_active":"2024-09-10T09:37:07.531896+0000","last_peered":"2024-09-10T09:37:07.531896+0000","last_clean":"2024-09-10T09:37:07.531896+0000","last_became_active":"2024-09-10T09:35:25.532193+0000","last_became_peered":"2024-09-10T09:35:25.532193+0000","last_unstale":"2024-09-10T09:37:07.531896+0000","last_undegraded":"2024-09-10T09:37:07.531896+0000","last_fullsized":"2024-09-10T09:37:07.531896+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:17:24.685178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.526424+0000","last_change":"2024-09-10T09:35:36.630742+0000","last_active":"2024-09-10T09:37:07.526424+0000","last_peered":"2024-09-10T09:37:07.526424+0000","last_clean":"2024-09-10T09:37:07.526424+0000","last_became_active":"2024-09-10T09:35:36.630335+0000","last_became_peered":"2024-09-10T09:35:36.630335+0000","last_unstale":"2024-09-10T09:37:07.526424+0000","last_undegraded":"2024-09-10T09:37:07.526424+0000","last_fullsized":"2024-09-10T09:37:07.526424+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:29:12.029390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"96'9","reported_seq":48,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.552455+0000","last_change":"2024-09-10T09:35:28.581817+0000","last_active":"2024-09-10T09:37:35.552455+0000","last_peered":"2024-09-10T09:37:35.552455+0000","last_clean":"2024-09-10T09:37:35.552455+0000","last_became_active":"2024-09-10T09:35:28.580316+0000","last_became_peered":"2024-09-10T09:35:28.580316+0000","last_unstale":"2024-09-10T09:37:35.552455+0000","last_undegraded":"2024-09-10T09:37:35.552455+0000","last_fullsized":"2024-09-10T09:37:35.552455+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:04:08.249564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"94'9","reported_seq":55,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.528127+0000","last_change":"2024-09-10T09:35:25.537474+0000","last_active":"2024-09-10T09:37:07.528127+0000","last_peered":"2024-09-10T09:37:07.528127+0000","last_clean":"2024-09-10T09:37:07.528127+0000","last_became_active":"2024-09-10T09:35:25.537096+0000","last_became_peered":"2024-09-10T09:35:25.537096+0000","last_unstale":"2024-09-10T09:37:07.528127+0000","last_undegraded":"2024-09-10T09:37:07.528127+0000","last_fullsized":"2024-09-10T09:37:07.528127+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:13:06.008458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.532316+0000","last_change":"2024-09-10T09:35:36.622788+0000","last_active":"2024-09-10T09:37:07.532316+0000","last_peered":"2024-09-10T09:37:07.532316+0000","last_clean":"2024-09-10T09:37:07.532316+0000","last_became_active":"2024-09-10T09:35:36.622377+0000","last_became_peered":"2024-09-10T09:35:36.622377+0000","last_unstale":"2024-09-10T09:37:07.532316+0000","last_undegraded":"2024-09-10T09:37:07.532316+0000","last_fullsized":"2024-09-10T09:37:07.532316+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:29:56.680864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"96'6","reported_seq":45,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.534371+0000","last_change":"2024-09-10T09:35:28.564529+0000","last_active":"2024-09-10T09:37:35.534371+0000","last_peered":"2024-09-10T09:37:35.534371+0000","last_clean":"2024-09-10T09:37:35.534371+0000","last_became_active":"2024-09-10T09:35:28.564223+0000","last_became_peered":"2024-09-10T09:35:28.564223+0000","last_unstale":"2024-09-10T09:37:35.534371+0000","last_undegraded":"2024-09-10T09:37:35.534371+0000","last_fullsized":"2024-09-10T09:37:35.534371+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:21:24.296578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"96'38","reported_seq":330,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:38:13.662697+0000","last_change":"2024-09-10T09:35:25.534331+0000","last_active":"2024-09-10T09:38:13.662697+0000","last_peered":"2024-09-10T09:38:13.662697+0000","last_clean":"2024-09-10T09:38:13.662697+0000","last_became_active":"2024-09-10T09:35:25.533985+0000","last_became_peered":"2024-09-10T09:35:25.533985+0000","last_unstale":"2024-09-10T09:38:13.662697+0000","last_undegraded":"2024-09-10T09:38:13.662697+0000","last_fullsized":"2024-09-10T09:38:13.662697+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:14:37.050650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":63180,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":248,"num_read_kb":186,"num_write":40,"num_write_kb":79,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.527925+0000","last_change":"2024-09-10T09:35:36.632835+0000","last_active":"2024-09-10T09:37:07.527925+0000","last_peered":"2024-09-10T09:37:07.527925+0000","last_clean":"2024-09-10T09:37:07.527925+0000","last_became_active":"2024-09-10T09:35:36.630962+0000","last_became_peered":"2024-09-10T09:35:36.630962+0000","last_unstale":"2024-09-10T09:37:07.527925+0000","last_undegraded":"2024-09-10T09:37:07.527925+0000","last_fullsized":"2024-09-10T09:37:07.527925+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:59:27.613721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"96'11","reported_seq":50,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.546541+0000","last_change":"2024-09-10T09:35:28.567292+0000","last_active":"2024-09-10T09:37:35.546541+0000","last_peered":"2024-09-10T09:37:35.546541+0000","last_clean":"2024-09-10T09:37:35.546541+0000","last_became_active":"2024-09-10T09:35:28.566973+0000","last_became_peered":"2024-09-10T09:35:28.566973+0000","last_unstale":"2024-09-10T09:37:35.546541+0000","last_undegraded":"2024-09-10T09:37:35.546541+0000","last_fullsized":"2024-09-10T09:37:35.546541+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:05:37.251933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"94'5","reported_seq":50,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.532787+0000","last_change":"2024-09-10T09:35:25.852171+0000","last_active":"2024-09-10T09:37:07.532787+0000","last_peered":"2024-09-10T09:37:07.532787+0000","last_clean":"2024-09-10T09:37:07.532787+0000","last_became_active":"2024-09-10T09:35:25.851580+0000","last_became_peered":"2024-09-10T09:35:25.851580+0000","last_unstale":"2024-09-10T09:37:07.532787+0000","last_undegraded":"2024-09-10T09:37:07.532787+0000","last_fullsized":"2024-09-10T09:37:07.532787+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:03.263152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.568336+0000","last_change":"2024-09-10T09:35:36.629212+0000","last_active":"2024-09-10T09:37:07.568336+0000","last_peered":"2024-09-10T09:37:07.568336+0000","last_clean":"2024-09-10T09:37:07.568336+0000","last_became_active":"2024-09-10T09:35:36.628753+0000","last_became_peered":"2024-09-10T09:35:36.628753+0000","last_unstale":"2024-09-10T09:37:07.568336+0000","last_undegraded":"2024-09-10T09:37:07.568336+0000","last_fullsized":"2024-09-10T09:37:07.568336+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:54:45.311361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"96'9","reported_seq":46,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.553075+0000","last_change":"2024-09-10T09:35:28.572497+0000","last_active":"2024-09-10T09:37:35.553075+0000","last_peered":"2024-09-10T09:37:35.553075+0000","last_clean":"2024-09-10T09:37:35.553075+0000","last_became_active":"2024-09-10T09:35:28.569238+0000","last_became_peered":"2024-09-10T09:35:28.569238+0000","last_unstale":"2024-09-10T09:37:35.553075+0000","last_undegraded":"2024-09-10T09:37:35.553075+0000","last_fullsized":"2024-09-10T09:37:35.553075+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:38:47.189209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"94'4","reported_seq":47,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.529569+0000","last_change":"2024-09-10T09:35:25.536369+0000","last_active":"2024-09-10T09:37:07.529569+0000","last_peered":"2024-09-10T09:37:07.529569+0000","last_clean":"2024-09-10T09:37:07.529569+0000","last_became_active":"2024-09-10T09:35:25.535607+0000","last_became_peered":"2024-09-10T09:35:25.535607+0000","last_unstale":"2024-09-10T09:37:07.529569+0000","last_undegraded":"2024-09-10T09:37:07.529569+0000","last_fullsized":"2024-09-10T09:37:07.529569+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:49:18.621934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.527047+0000","last_change":"2024-09-10T09:35:36.627977+0000","last_active":"2024-09-10T09:37:07.527047+0000","last_peered":"2024-09-10T09:37:07.527047+0000","last_clean":"2024-09-10T09:37:07.527047+0000","last_became_active":"2024-09-10T09:35:36.626942+0000","last_became_peered":"2024-09-10T09:35:36.626942+0000","last_unstale":"2024-09-10T09:37:07.527047+0000","last_undegraded":"2024-09-10T09:37:07.527047+0000","last_fullsized":"2024-09-10T09:37:07.527047+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:37:18.209424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"96'10","reported_seq":49,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.553165+0000","last_change":"2024-09-10T09:35:28.562670+0000","last_active":"2024-09-10T09:37:35.553165+0000","last_peered":"2024-09-10T09:37:35.553165+0000","last_clean":"2024-09-10T09:37:35.553165+0000","last_became_active":"2024-09-10T09:35:28.562028+0000","last_became_peered":"2024-09-10T09:35:28.562028+0000","last_unstale":"2024-09-10T09:37:35.553165+0000","last_undegraded":"2024-09-10T09:37:35.553165+0000","last_fullsized":"2024-09-10T09:37:35.553165+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:37:15.176050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"96'5","reported_seq":288,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:38:13.662801+0000","last_change":"2024-09-10T09:35:25.529861+0000","last_active":"2024-09-10T09:38:13.662801+0000","last_peered":"2024-09-10T09:38:13.662801+0000","last_clean":"2024-09-10T09:38:13.662801+0000","last_became_active":"2024-09-10T09:35:25.529475+0000","last_became_peered":"2024-09-10T09:35:25.529475+0000","last_unstale":"2024-09-10T09:38:13.662801+0000","last_undegraded":"2024-09-10T09:38:13.662801+0000","last_fullsized":"2024-09-10T09:38:13.662801+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:44:27.086509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":239,"num_read_kb":239,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.524306+0000","last_change":"2024-09-10T09:35:36.615358+0000","last_active":"2024-09-10T09:37:07.524306+0000","last_peered":"2024-09-10T09:37:07.524306+0000","last_clean":"2024-09-10T09:37:07.524306+0000","last_became_active":"2024-09-10T09:35:36.614901+0000","last_became_peered":"2024-09-10T09:35:36.614901+0000","last_unstale":"2024-09-10T09:37:07.524306+0000","last_undegraded":"2024-09-10T09:37:07.524306+0000","last_fullsized":"2024-09-10T09:37:07.524306+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:56:54.248266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.761302+0000","last_change":"2024-09-10T09:35:36.629021+0000","last_active":"2024-09-10T09:37:07.761302+0000","last_peered":"2024-09-10T09:37:07.761302+0000","last_clean":"2024-09-10T09:37:07.761302+0000","last_became_active":"2024-09-10T09:35:36.628159+0000","last_became_peered":"2024-09-10T09:35:36.628159+0000","last_unstale":"2024-09-10T09:37:07.761302+0000","last_undegraded":"2024-09-10T09:37:07.761302+0000","last_fullsized":"2024-09-10T09:37:07.761302+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:50:48.615236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"96'11","reported_seq":48,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.545215+0000","last_change":"2024-09-10T09:35:28.579333+0000","last_active":"2024-09-10T09:37:35.545215+0000","last_peered":"2024-09-10T09:37:35.545215+0000","last_clean":"2024-09-10T09:37:35.545215+0000","last_became_active":"2024-09-10T09:35:28.578440+0000","last_became_peered":"2024-09-10T09:35:28.578440+0000","last_unstale":"2024-09-10T09:37:35.545215+0000","last_undegraded":"2024-09-10T09:37:35.545215+0000","last_fullsized":"2024-09-10T09:37:35.545215+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:07:28.701938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"94'3","reported_seq":48,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.531063+0000","last_change":"2024-09-10T09:35:25.530857+0000","last_active":"2024-09-10T09:37:07.531063+0000","last_peered":"2024-09-10T09:37:07.531063+0000","last_clean":"2024-09-10T09:37:07.531063+0000","last_became_active":"2024-09-10T09:35:25.530492+0000","last_became_peered":"2024-09-10T09:35:25.530492+0000","last_unstale":"2024-09-10T09:37:07.531063+0000","last_undegraded":"2024-09-10T09:37:07.531063+0000","last_fullsized":"2024-09-10T09:37:07.531063+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:07:21.726598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.526432+0000","last_change":"2024-09-10T09:36:06.770408+0000","last_active":"2024-09-10T09:37:07.526432+0000","last_peered":"2024-09-10T09:37:07.526432+0000","last_clean":"2024-09-10T09:37:07.526432+0000","last_became_active":"2024-09-10T09:36:06.769915+0000","last_became_peered":"2024-09-10T09:36:06.769915+0000","last_unstale":"2024-09-10T09:37:07.526432+0000","last_undegraded":"2024-09-10T09:37:07.526432+0000","last_fullsized":"2024-09-10T09:37:07.526432+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:56:09.795537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8,0],"acting":[11,4,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"96'8","reported_seq":47,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.543491+0000","last_change":"2024-09-10T09:35:28.570804+0000","last_active":"2024-09-10T09:37:35.543491+0000","last_peered":"2024-09-10T09:37:35.543491+0000","last_clean":"2024-09-10T09:37:35.543491+0000","last_became_active":"2024-09-10T09:35:28.570490+0000","last_became_peered":"2024-09-10T09:35:28.570490+0000","last_unstale":"2024-09-10T09:37:35.543491+0000","last_undegraded":"2024-09-10T09:37:35.543491+0000","last_fullsized":"2024-09-10T09:37:35.543491+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:13:38.999625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"94'3","reported_seq":47,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.538446+0000","last_change":"2024-09-10T09:35:25.536347+0000","last_active":"2024-09-10T09:37:07.538446+0000","last_peered":"2024-09-10T09:37:07.538446+0000","last_clean":"2024-09-10T09:37:07.538446+0000","last_became_active":"2024-09-10T09:35:25.535964+0000","last_became_peered":"2024-09-10T09:35:25.535964+0000","last_unstale":"2024-09-10T09:37:07.538446+0000","last_undegraded":"2024-09-10T09:37:07.538446+0000","last_fullsized":"2024-09-10T09:37:07.538446+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:36:49.035666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"94'7","reported_seq":51,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.525812+0000","last_change":"2024-09-10T09:35:25.532148+0000","last_active":"2024-09-10T09:37:07.525812+0000","last_peered":"2024-09-10T09:37:07.525812+0000","last_clean":"2024-09-10T09:37:07.525812+0000","last_became_active":"2024-09-10T09:35:25.531781+0000","last_became_peered":"2024-09-10T09:35:25.531781+0000","last_unstale":"2024-09-10T09:37:07.525812+0000","last_undegraded":"2024-09-10T09:37:07.525812+0000","last_fullsized":"2024-09-10T09:37:07.525812+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:52:46.740342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"96'8","reported_seq":47,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.549412+0000","last_change":"2024-09-10T09:35:28.578336+0000","last_active":"2024-09-10T09:37:35.549412+0000","last_peered":"2024-09-10T09:37:35.549412+0000","last_clean":"2024-09-10T09:37:35.549412+0000","last_became_active":"2024-09-10T09:35:28.577899+0000","last_became_peered":"2024-09-10T09:35:28.577899+0000","last_unstale":"2024-09-10T09:37:35.549412+0000","last_undegraded":"2024-09-10T09:37:35.549412+0000","last_fullsized":"2024-09-10T09:37:35.549412+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:13:04.478195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.528122+0000","last_change":"2024-09-10T09:35:36.630918+0000","last_active":"2024-09-10T09:37:07.528122+0000","last_peered":"2024-09-10T09:37:07.528122+0000","last_clean":"2024-09-10T09:37:07.528122+0000","last_became_active":"2024-09-10T09:35:36.630308+0000","last_became_peered":"2024-09-10T09:35:36.630308+0000","last_unstale":"2024-09-10T09:37:07.528122+0000","last_undegraded":"2024-09-10T09:37:07.528122+0000","last_fullsized":"2024-09-10T09:37:07.528122+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:25:44.954970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"96'7","reported_seq":186,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:38:12.661788+0000","last_change":"2024-09-10T09:37:07.574386+0000","last_active":"2024-09-10T09:38:12.661788+0000","last_peered":"2024-09-10T09:38:12.661788+0000","last_clean":"2024-09-10T09:38:12.661788+0000","last_became_active":"2024-09-10T09:37:07.569370+0000","last_became_peered":"2024-09-10T09:37:07.569370+0000","last_unstale":"2024-09-10T09:38:12.661788+0000","last_undegraded":"2024-09-10T09:38:12.661788+0000","last_fullsized":"2024-09-10T09:38:12.661788+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:07:23.014789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1156,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":118,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":1015,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"96'7","reported_seq":46,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.543648+0000","last_change":"2024-09-10T09:35:28.567915+0000","last_active":"2024-09-10T09:37:35.543648+0000","last_peered":"2024-09-10T09:37:35.543648+0000","last_clean":"2024-09-10T09:37:35.543648+0000","last_became_active":"2024-09-10T09:35:28.567416+0000","last_became_peered":"2024-09-10T09:35:28.567416+0000","last_unstale":"2024-09-10T09:37:35.543648+0000","last_undegraded":"2024-09-10T09:37:35.543648+0000","last_fullsized":"2024-09-10T09:37:35.543648+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:07:25.704080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.528686+0000","last_change":"2024-09-10T09:35:36.615386+0000","last_active":"2024-09-10T09:37:07.528686+0000","last_peered":"2024-09-10T09:37:07.528686+0000","last_clean":"2024-09-10T09:37:07.528686+0000","last_became_active":"2024-09-10T09:35:36.614967+0000","last_became_peered":"2024-09-10T09:35:36.614967+0000","last_unstale":"2024-09-10T09:37:07.528686+0000","last_undegraded":"2024-09-10T09:37:07.528686+0000","last_fullsized":"2024-09-10T09:37:07.528686+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:47.620729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"96'9","reported_seq":48,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.544481+0000","last_change":"2024-09-10T09:35:28.564411+0000","last_active":"2024-09-10T09:37:35.544481+0000","last_peered":"2024-09-10T09:37:35.544481+0000","last_clean":"2024-09-10T09:37:35.544481+0000","last_became_active":"2024-09-10T09:35:28.564034+0000","last_became_peered":"2024-09-10T09:35:28.564034+0000","last_unstale":"2024-09-10T09:37:35.544481+0000","last_undegraded":"2024-09-10T09:37:35.544481+0000","last_fullsized":"2024-09-10T09:37:35.544481+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:38:17.687991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"96'12","reported_seq":49,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.547277+0000","last_change":"2024-09-10T09:35:28.564619+0000","last_active":"2024-09-10T09:37:35.547277+0000","last_peered":"2024-09-10T09:37:35.547277+0000","last_clean":"2024-09-10T09:37:35.547277+0000","last_became_active":"2024-09-10T09:35:28.563890+0000","last_became_peered":"2024-09-10T09:35:28.563890+0000","last_unstale":"2024-09-10T09:37:35.547277+0000","last_undegraded":"2024-09-10T09:37:35.547277+0000","last_fullsized":"2024-09-10T09:37:35.547277+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:51:42.481911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"96'8","reported_seq":45,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.543410+0000","last_change":"2024-09-10T09:35:28.558909+0000","last_active":"2024-09-10T09:37:35.543410+0000","last_peered":"2024-09-10T09:37:35.543410+0000","last_clean":"2024-09-10T09:37:35.543410+0000","last_became_active":"2024-09-10T09:35:28.558604+0000","last_became_peered":"2024-09-10T09:35:28.558604+0000","last_unstale":"2024-09-10T09:37:35.543410+0000","last_undegraded":"2024-09-10T09:37:35.543410+0000","last_fullsized":"2024-09-10T09:37:35.543410+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:11.928446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"96'7","reported_seq":44,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.554844+0000","last_change":"2024-09-10T09:35:28.574459+0000","last_active":"2024-09-10T09:37:35.554844+0000","last_peered":"2024-09-10T09:37:35.554844+0000","last_clean":"2024-09-10T09:37:35.554844+0000","last_became_active":"2024-09-10T09:35:28.574154+0000","last_became_peered":"2024-09-10T09:35:28.574154+0000","last_unstale":"2024-09-10T09:37:35.554844+0000","last_undegraded":"2024-09-10T09:37:35.554844+0000","last_fullsized":"2024-09-10T09:37:35.554844+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:59:09.650536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"96'4","reported_seq":43,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.552571+0000","last_change":"2024-09-10T09:35:28.576999+0000","last_active":"2024-09-10T09:37:35.552571+0000","last_peered":"2024-09-10T09:37:35.552571+0000","last_clean":"2024-09-10T09:37:35.552571+0000","last_became_active":"2024-09-10T09:35:28.575616+0000","last_became_peered":"2024-09-10T09:35:28.575616+0000","last_unstale":"2024-09-10T09:37:35.552571+0000","last_undegraded":"2024-09-10T09:37:35.552571+0000","last_fullsized":"2024-09-10T09:37:35.552571+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:18:08.771760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"96'6","reported_seq":45,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.541496+0000","last_change":"2024-09-10T09:35:28.564846+0000","last_active":"2024-09-10T09:37:35.541496+0000","last_peered":"2024-09-10T09:37:35.541496+0000","last_clean":"2024-09-10T09:37:35.541496+0000","last_became_active":"2024-09-10T09:35:28.564498+0000","last_became_peered":"2024-09-10T09:35:28.564498+0000","last_unstale":"2024-09-10T09:37:35.541496+0000","last_undegraded":"2024-09-10T09:37:35.541496+0000","last_fullsized":"2024-09-10T09:37:35.541496+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:33:43.041730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"96'15","reported_seq":52,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.550975+0000","last_change":"2024-09-10T09:35:28.558093+0000","last_active":"2024-09-10T09:37:35.550975+0000","last_peered":"2024-09-10T09:37:35.550975+0000","last_clean":"2024-09-10T09:37:35.550975+0000","last_became_active":"2024-09-10T09:35:28.557266+0000","last_became_peered":"2024-09-10T09:35:28.557266+0000","last_unstale":"2024-09-10T09:37:35.550975+0000","last_undegraded":"2024-09-10T09:37:35.550975+0000","last_fullsized":"2024-09-10T09:37:35.550975+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:51:30.946065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"96'2","reported_seq":47,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.549558+0000","last_change":"2024-09-10T09:36:06.770298+0000","last_active":"2024-09-10T09:37:35.549558+0000","last_peered":"2024-09-10T09:37:35.549558+0000","last_clean":"2024-09-10T09:37:35.549558+0000","last_became_active":"2024-09-10T09:36:06.769848+0000","last_became_peered":"2024-09-10T09:36:06.769848+0000","last_unstale":"2024-09-10T09:37:35.549558+0000","last_undegraded":"2024-09-10T09:37:35.549558+0000","last_fullsized":"2024-09-10T09:37:35.549558+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:00:30.470887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,8,2],"acting":[3,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"96'6","reported_seq":45,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.549553+0000","last_change":"2024-09-10T09:35:28.578158+0000","last_active":"2024-09-10T09:37:35.549553+0000","last_peered":"2024-09-10T09:37:35.549553+0000","last_clean":"2024-09-10T09:37:35.549553+0000","last_became_active":"2024-09-10T09:35:28.577824+0000","last_became_peered":"2024-09-10T09:35:28.577824+0000","last_unstale":"2024-09-10T09:37:35.549553+0000","last_undegraded":"2024-09-10T09:37:35.549553+0000","last_fullsized":"2024-09-10T09:37:35.549553+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:40:27.231164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"96'8","reported_seq":47,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.550143+0000","last_change":"2024-09-10T09:35:28.562670+0000","last_active":"2024-09-10T09:37:35.550143+0000","last_peered":"2024-09-10T09:37:35.550143+0000","last_clean":"2024-09-10T09:37:35.550143+0000","last_became_active":"2024-09-10T09:35:28.562311+0000","last_became_peered":"2024-09-10T09:35:28.562311+0000","last_unstale":"2024-09-10T09:37:35.550143+0000","last_undegraded":"2024-09-10T09:37:35.550143+0000","last_fullsized":"2024-09-10T09:37:35.550143+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:07:11.716142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"96'10","reported_seq":49,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.556330+0000","last_change":"2024-09-10T09:35:28.561124+0000","last_active":"2024-09-10T09:37:35.556330+0000","last_peered":"2024-09-10T09:37:35.556330+0000","last_clean":"2024-09-10T09:37:35.556330+0000","last_became_active":"2024-09-10T09:35:28.560777+0000","last_became_peered":"2024-09-10T09:35:28.560777+0000","last_unstale":"2024-09-10T09:37:35.556330+0000","last_undegraded":"2024-09-10T09:37:35.556330+0000","last_fullsized":"2024-09-10T09:37:35.556330+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:47:04.837596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"96'11","reported_seq":50,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.556539+0000","last_change":"2024-09-10T09:35:28.576790+0000","last_active":"2024-09-10T09:37:35.556539+0000","last_peered":"2024-09-10T09:37:35.556539+0000","last_clean":"2024-09-10T09:37:35.556539+0000","last_became_active":"2024-09-10T09:35:28.576464+0000","last_became_peered":"2024-09-10T09:35:28.576464+0000","last_unstale":"2024-09-10T09:37:35.556539+0000","last_undegraded":"2024-09-10T09:37:35.556539+0000","last_fullsized":"2024-09-10T09:37:35.556539+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:40:53.517577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"96'8","reported_seq":47,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.539417+0000","last_change":"2024-09-10T09:35:28.569935+0000","last_active":"2024-09-10T09:37:35.539417+0000","last_peered":"2024-09-10T09:37:35.539417+0000","last_clean":"2024-09-10T09:37:35.539417+0000","last_became_active":"2024-09-10T09:35:28.569539+0000","last_became_peered":"2024-09-10T09:35:28.569539+0000","last_unstale":"2024-09-10T09:37:35.539417+0000","last_undegraded":"2024-09-10T09:37:35.539417+0000","last_fullsized":"2024-09-10T09:37:35.539417+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:40:45.016269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"96'7","reported_seq":44,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.550145+0000","last_change":"2024-09-10T09:35:28.571619+0000","last_active":"2024-09-10T09:37:35.550145+0000","last_peered":"2024-09-10T09:37:35.550145+0000","last_clean":"2024-09-10T09:37:35.550145+0000","last_became_active":"2024-09-10T09:35:28.568591+0000","last_became_peered":"2024-09-10T09:35:28.568591+0000","last_unstale":"2024-09-10T09:37:35.550145+0000","last_undegraded":"2024-09-10T09:37:35.550145+0000","last_fullsized":"2024-09-10T09:37:35.550145+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:07:49.959712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"96'10","reported_seq":47,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.551796+0000","last_change":"2024-09-10T09:35:28.573336+0000","last_active":"2024-09-10T09:37:35.551796+0000","last_peered":"2024-09-10T09:37:35.551796+0000","last_clean":"2024-09-10T09:37:35.551796+0000","last_became_active":"2024-09-10T09:35:28.572991+0000","last_became_peered":"2024-09-10T09:35:28.572991+0000","last_unstale":"2024-09-10T09:37:35.551796+0000","last_undegraded":"2024-09-10T09:37:35.551796+0000","last_fullsized":"2024-09-10T09:37:35.551796+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:30:02.905869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"96'8","reported_seq":47,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.552930+0000","last_change":"2024-09-10T09:35:28.571945+0000","last_active":"2024-09-10T09:37:35.552930+0000","last_peered":"2024-09-10T09:37:35.552930+0000","last_clean":"2024-09-10T09:37:35.552930+0000","last_became_active":"2024-09-10T09:35:28.571546+0000","last_became_peered":"2024-09-10T09:35:28.571546+0000","last_unstale":"2024-09-10T09:37:35.552930+0000","last_undegraded":"2024-09-10T09:37:35.552930+0000","last_fullsized":"2024-09-10T09:37:35.552930+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:36:47.946864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"96'9","reported_seq":48,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.556725+0000","last_change":"2024-09-10T09:35:28.562712+0000","last_active":"2024-09-10T09:37:35.556725+0000","last_peered":"2024-09-10T09:37:35.556725+0000","last_clean":"2024-09-10T09:37:35.556725+0000","last_became_active":"2024-09-10T09:35:28.562396+0000","last_became_peered":"2024-09-10T09:35:28.562396+0000","last_unstale":"2024-09-10T09:37:35.556725+0000","last_undegraded":"2024-09-10T09:37:35.556725+0000","last_fullsized":"2024-09-10T09:37:35.556725+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:25:47.419162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"96'7","reported_seq":44,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.553427+0000","last_change":"2024-09-10T09:35:28.566354+0000","last_active":"2024-09-10T09:37:35.553427+0000","last_peered":"2024-09-10T09:37:35.553427+0000","last_clean":"2024-09-10T09:37:35.553427+0000","last_became_active":"2024-09-10T09:35:28.565248+0000","last_became_peered":"2024-09-10T09:35:28.565248+0000","last_unstale":"2024-09-10T09:37:35.553427+0000","last_undegraded":"2024-09-10T09:37:35.553427+0000","last_fullsized":"2024-09-10T09:37:35.553427+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:48:46.741010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"96'7","reported_seq":44,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.550936+0000","last_change":"2024-09-10T09:35:28.563331+0000","last_active":"2024-09-10T09:37:35.550936+0000","last_peered":"2024-09-10T09:37:35.550936+0000","last_clean":"2024-09-10T09:37:35.550936+0000","last_became_active":"2024-09-10T09:35:28.563020+0000","last_became_peered":"2024-09-10T09:35:28.563020+0000","last_unstale":"2024-09-10T09:37:35.550936+0000","last_undegraded":"2024-09-10T09:37:35.550936+0000","last_fullsized":"2024-09-10T09:37:35.550936+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:27.653551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"96'10","reported_seq":49,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.542747+0000","last_change":"2024-09-10T09:35:28.566687+0000","last_active":"2024-09-10T09:37:35.542747+0000","last_peered":"2024-09-10T09:37:35.542747+0000","last_clean":"2024-09-10T09:37:35.542747+0000","last_became_active":"2024-09-10T09:35:28.566253+0000","last_became_peered":"2024-09-10T09:35:28.566253+0000","last_unstale":"2024-09-10T09:37:35.542747+0000","last_undegraded":"2024-09-10T09:37:35.542747+0000","last_fullsized":"2024-09-10T09:37:35.542747+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:00:15.709725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"96'7","reported_seq":44,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.538923+0000","last_change":"2024-09-10T09:35:28.562251+0000","last_active":"2024-09-10T09:37:35.538923+0000","last_peered":"2024-09-10T09:37:35.538923+0000","last_clean":"2024-09-10T09:37:35.538923+0000","last_became_active":"2024-09-10T09:35:28.561933+0000","last_became_peered":"2024-09-10T09:35:28.561933+0000","last_unstale":"2024-09-10T09:37:35.538923+0000","last_undegraded":"2024-09-10T09:37:35.538923+0000","last_fullsized":"2024-09-10T09:37:35.538923+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:30:41.447954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"96'6","reported_seq":43,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.556126+0000","last_change":"2024-09-10T09:35:28.572164+0000","last_active":"2024-09-10T09:37:35.556126+0000","last_peered":"2024-09-10T09:37:35.556126+0000","last_clean":"2024-09-10T09:37:35.556126+0000","last_became_active":"2024-09-10T09:35:28.571825+0000","last_became_peered":"2024-09-10T09:35:28.571825+0000","last_unstale":"2024-09-10T09:37:35.556126+0000","last_undegraded":"2024-09-10T09:37:35.556126+0000","last_fullsized":"2024-09-10T09:37:35.556126+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:59:50.967269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"96'16","reported_seq":55,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.550138+0000","last_change":"2024-09-10T09:35:28.568098+0000","last_active":"2024-09-10T09:37:35.550138+0000","last_peered":"2024-09-10T09:37:35.550138+0000","last_clean":"2024-09-10T09:37:35.550138+0000","last_became_active":"2024-09-10T09:35:28.567765+0000","last_became_peered":"2024-09-10T09:35:28.567765+0000","last_unstale":"2024-09-10T09:37:35.550138+0000","last_undegraded":"2024-09-10T09:37:35.550138+0000","last_fullsized":"2024-09-10T09:37:35.550138+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:55:57.260320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"96'11","reported_seq":50,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.553598+0000","last_change":"2024-09-10T09:35:28.564123+0000","last_active":"2024-09-10T09:37:35.553598+0000","last_peered":"2024-09-10T09:37:35.553598+0000","last_clean":"2024-09-10T09:37:35.553598+0000","last_became_active":"2024-09-10T09:35:28.563814+0000","last_became_peered":"2024-09-10T09:35:28.563814+0000","last_unstale":"2024-09-10T09:37:35.553598+0000","last_undegraded":"2024-09-10T09:37:35.553598+0000","last_fullsized":"2024-09-10T09:37:35.553598+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:14:17.912087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"96'9","reported_seq":48,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.549407+0000","last_change":"2024-09-10T09:35:28.568601+0000","last_active":"2024-09-10T09:37:35.549407+0000","last_peered":"2024-09-10T09:37:35.549407+0000","last_clean":"2024-09-10T09:37:35.549407+0000","last_became_active":"2024-09-10T09:35:28.568303+0000","last_became_peered":"2024-09-10T09:35:28.568303+0000","last_unstale":"2024-09-10T09:37:35.549407+0000","last_undegraded":"2024-09-10T09:37:35.549407+0000","last_fullsized":"2024-09-10T09:37:35.549407+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:30:01.016420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"96'5","reported_seq":48,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.556713+0000","last_change":"2024-09-10T09:37:07.539577+0000","last_active":"2024-09-10T09:37:35.556713+0000","last_peered":"2024-09-10T09:37:35.556713+0000","last_clean":"2024-09-10T09:37:35.556713+0000","last_became_active":"2024-09-10T09:37:07.539206+0000","last_became_peered":"2024-09-10T09:37:07.539206+0000","last_unstale":"2024-09-10T09:37:35.556713+0000","last_undegraded":"2024-09-10T09:37:35.556713+0000","last_fullsized":"2024-09-10T09:37:35.556713+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:56:06.568226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"96'6","reported_seq":45,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.541193+0000","last_change":"2024-09-10T09:35:28.569225+0000","last_active":"2024-09-10T09:37:35.541193+0000","last_peered":"2024-09-10T09:37:35.541193+0000","last_clean":"2024-09-10T09:37:35.541193+0000","last_became_active":"2024-09-10T09:35:28.568891+0000","last_became_peered":"2024-09-10T09:35:28.568891+0000","last_unstale":"2024-09-10T09:37:35.541193+0000","last_undegraded":"2024-09-10T09:37:35.541193+0000","last_fullsized":"2024-09-10T09:37:35.541193+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:33:45.811781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"96'4","reported_seq":43,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.544182+0000","last_change":"2024-09-10T09:35:28.578362+0000","last_active":"2024-09-10T09:37:35.544182+0000","last_peered":"2024-09-10T09:37:35.544182+0000","last_clean":"2024-09-10T09:37:35.544182+0000","last_became_active":"2024-09-10T09:35:28.577867+0000","last_became_peered":"2024-09-10T09:35:28.577867+0000","last_unstale":"2024-09-10T09:37:35.544182+0000","last_undegraded":"2024-09-10T09:37:35.544182+0000","last_fullsized":"2024-09-10T09:37:35.544182+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:58:17.041663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"96'8","reported_seq":45,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.537018+0000","last_change":"2024-09-10T09:35:28.568183+0000","last_active":"2024-09-10T09:37:35.537018+0000","last_peered":"2024-09-10T09:37:35.537018+0000","last_clean":"2024-09-10T09:37:35.537018+0000","last_became_active":"2024-09-10T09:35:28.566775+0000","last_became_peered":"2024-09-10T09:35:28.566775+0000","last_unstale":"2024-09-10T09:37:35.537018+0000","last_undegraded":"2024-09-10T09:37:35.537018+0000","last_fullsized":"2024-09-10T09:37:35.537018+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:28:10.008889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"96'7","reported_seq":44,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.553233+0000","last_change":"2024-09-10T09:35:28.576122+0000","last_active":"2024-09-10T09:37:35.553233+0000","last_peered":"2024-09-10T09:37:35.553233+0000","last_clean":"2024-09-10T09:37:35.553233+0000","last_became_active":"2024-09-10T09:35:28.575831+0000","last_became_peered":"2024-09-10T09:35:28.575831+0000","last_unstale":"2024-09-10T09:37:35.553233+0000","last_undegraded":"2024-09-10T09:37:35.553233+0000","last_fullsized":"2024-09-10T09:37:35.553233+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:37:51.604658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"96'4","reported_seq":41,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.554017+0000","last_change":"2024-09-10T09:35:28.564441+0000","last_active":"2024-09-10T09:37:35.554017+0000","last_peered":"2024-09-10T09:37:35.554017+0000","last_clean":"2024-09-10T09:37:35.554017+0000","last_became_active":"2024-09-10T09:35:28.564095+0000","last_became_peered":"2024-09-10T09:35:28.564095+0000","last_unstale":"2024-09-10T09:37:35.554017+0000","last_undegraded":"2024-09-10T09:37:35.554017+0000","last_fullsized":"2024-09-10T09:37:35.554017+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:53:28.275361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"96'8","reported_seq":47,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.553155+0000","last_change":"2024-09-10T09:35:28.581921+0000","last_active":"2024-09-10T09:37:35.553155+0000","last_peered":"2024-09-10T09:37:35.553155+0000","last_clean":"2024-09-10T09:37:35.553155+0000","last_became_active":"2024-09-10T09:35:28.581573+0000","last_became_peered":"2024-09-10T09:35:28.581573+0000","last_unstale":"2024-09-10T09:37:35.553155+0000","last_undegraded":"2024-09-10T09:37:35.553155+0000","last_fullsized":"2024-09-10T09:37:35.553155+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:58:38.983934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":70670,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1244,"num_read_kb":703,"num_write":260,"num_write_kb":150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":1015,"num_keys_recovered":7,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":589824,"data_stored":225188,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":223,"ondisk_log_size":223,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":285627,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":77,"seq":330712481840,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32228,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712924,"statfs":{"total":95995035648,"available":95962034176,"internally_reserved":0,"allocated":1069056,"data_stored":554290,"data_compressed":32456,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":70,"seq":300647710772,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28220,"kb_used_data":1196,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716932,"statfs":{"total":95995035648,"available":95966138368,"internally_reserved":0,"allocated":1224704,"data_stored":1208382,"data_compressed":68777,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":65,"seq":279172874299,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32252,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712900,"statfs":{"total":95995035648,"available":95962009600,"internally_reserved":0,"allocated":1093632,"data_stored":554220,"data_compressed":32456,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1599,"internal_metadata":31914433},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":57,"seq":244813135936,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32248,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712904,"statfs":{"total":95995035648,"available":95962013696,"internally_reserved":0,"allocated":1089536,"data_stored":554598,"data_compressed":32456,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":50,"seq":214748364869,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28140,"kb_used_data":1116,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717012,"statfs":{"total":95995035648,"available":95966220288,"internally_reserved":0,"allocated":1142784,"data_stored":1151772,"data_compressed":68777,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":45,"seq":193273528395,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36388,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708764,"statfs":{"total":95995035648,"available":95957774336,"internally_reserved":0,"allocated":1069056,"data_stored":550569,"data_compressed":32456,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790033,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28148,"kb_used_data":1124,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717004,"statfs":{"total":95995035648,"available":95966212096,"internally_reserved":0,"allocated":1150976,"data_stored":1146820,"data_compressed":68777,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953559,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32244,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712908,"statfs":{"total":95995035648,"available":95962017792,"internally_reserved":0,"allocated":1085440,"data_stored":554954,"data_compressed":32456,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182491,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28080,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717072,"statfs":{"total":95995035648,"available":95966281728,"internally_reserved":0,"allocated":1081344,"data_stored":549432,"data_compressed":32456,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411424,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28152,"kb_used_data":1128,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717000,"statfs":{"total":95995035648,"available":95966208000,"internally_reserved":0,"allocated":1155072,"data_stored":618550,"data_compressed":32456,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574951,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36460,"kb_used_data":1116,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708692,"statfs":{"total":95995035648,"available":95957700608,"internally_reserved":0,"allocated":1142784,"data_stored":619797,"data_compressed":32456,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738475,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36412,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708740,"statfs":{"total":95995035648,"available":95957749760,"internally_reserved":0,"allocated":1093632,"data_stored":551180,"data_compressed":32456,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34674,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34674,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34674,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36321,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36321,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36321,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36321,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36321,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":73315,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":69548,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3354,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1567,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4210,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":118784,"data_stored":64916,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T09:38:21.528 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T09:38:21.955 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:38:22.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:38:21 smithi005 ceph-mon[30350]: from='client.15252 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T09:38:22.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:38:21 smithi005 ceph-mon[30350]: from='client.15258 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T09:38:22.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:38:21 smithi005 ceph-mon[30350]: from='client.15264 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T09:38:22.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:38:21 smithi102 ceph-mon[35442]: from='client.15252 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T09:38:22.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:38:21 smithi102 ceph-mon[35442]: from='client.15258 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T09:38:22.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:38:21 smithi102 ceph-mon[35442]: from='client.15264 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T09:38:22.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:38:21 smithi159 ceph-mon[35513]: from='client.15252 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T09:38:22.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:38:21 smithi159 ceph-mon[35513]: from='client.15258 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T09:38:22.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:38:21 smithi159 ceph-mon[35513]: from='client.15264 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T09:38:22.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:22.569+0000 7f9c17200640 1 -- 172.21.15.5:0/1109147178 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9c181079c0 msgr2=0x7f9c18107da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:38:22.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:22.569+0000 7f9c17200640 1 --2- 172.21.15.5:0/1109147178 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9c181079c0 0x7f9c18107da0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f9c08009930 tx=0x7f9c0802a420 comp rx=0 tx=0).stop 2024-09-10T09:38:22.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:22.569+0000 7f9c17200640 1 -- 172.21.15.5:0/1109147178 shutdown_connections 2024-09-10T09:38:22.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:22.569+0000 7f9c17200640 1 --2- 172.21.15.5:0/1109147178 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9c18102360 0x7f9c1810e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:22.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:22.570+0000 7f9c17200640 1 --2- 172.21.15.5:0/1109147178 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9c181019c0 0x7f9c18101e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:22.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:22.570+0000 7f9c17200640 1 --2- 172.21.15.5:0/1109147178 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9c181079c0 0x7f9c18107da0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:22.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:22.570+0000 7f9c17200640 1 -- 172.21.15.5:0/1109147178 >> 172.21.15.5:0/1109147178 conn(0x7f9c180fd740 msgr2=0x7f9c180ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:38:22.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:22.570+0000 7f9c17200640 1 -- 172.21.15.5:0/1109147178 shutdown_connections 2024-09-10T09:38:22.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:22.570+0000 7f9c17200640 1 -- 172.21.15.5:0/1109147178 wait complete. 2024-09-10T09:38:22.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:22.571+0000 7f9c17200640 1 Processor -- start 2024-09-10T09:38:22.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:22.571+0000 7f9c17200640 1 -- start start 2024-09-10T09:38:22.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:22.571+0000 7f9c17200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9c181019c0 0x7f9c1819a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:22.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:22.571+0000 7f9c17200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9c18102360 0x7f9c1819aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:22.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:22.571+0000 7f9c17200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9c181079c0 0x7f9c1819edd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:22.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:22.571+0000 7f9c17200640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f9c18111e10 con 0x7f9c181019c0 2024-09-10T09:38:22.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:22.571+0000 7f9c17200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f9c18111c90 con 0x7f9c18102360 2024-09-10T09:38:22.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:22.572+0000 7f9c17200640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f9c18111f90 con 0x7f9c181079c0 2024-09-10T09:38:22.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:22.572+0000 7f9c15e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9c181019c0 0x7f9c1819a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:22.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:22.572+0000 7f9c15e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9c181019c0 0x7f9c1819a500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60796/0 (socket says 172.21.15.5:60796) 2024-09-10T09:38:22.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:22.572+0000 7f9c15e00640 1 -- 172.21.15.5:0/4112549201 learned_addr learned my addr 172.21.15.5:0/4112549201 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:38:22.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:22.572+0000 7f9c15400640 1 --2- 172.21.15.5:0/4112549201 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9c18102360 0x7f9c1819aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:22.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:22.572+0000 7f9c16800640 1 --2- 172.21.15.5:0/4112549201 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9c181079c0 0x7f9c1819edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:22.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:22.572+0000 7f9c15e00640 1 -- 172.21.15.5:0/4112549201 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9c181079c0 msgr2=0x7f9c1819edd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:38:22.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:22.572+0000 7f9c15e00640 1 --2- 172.21.15.5:0/4112549201 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9c181079c0 0x7f9c1819edd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:22.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:22.572+0000 7f9c15e00640 1 -- 172.21.15.5:0/4112549201 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9c18102360 msgr2=0x7f9c1819aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:38:22.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:22.572+0000 7f9c15e00640 1 --2- 172.21.15.5:0/4112549201 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9c18102360 0x7f9c1819aa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:22.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:22.572+0000 7f9c15e00640 1 -- 172.21.15.5:0/4112549201 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c1819f4b0 con 0x7f9c181019c0 2024-09-10T09:38:22.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:22.573+0000 7f9c15400640 1 --2- 172.21.15.5:0/4112549201 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9c18102360 0x7f9c1819aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:38:22.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:22.573+0000 7f9c15e00640 1 --2- 172.21.15.5:0/4112549201 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9c181019c0 0x7f9c1819a500 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f9c0802a940 tx=0x7f9c08003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:38:22.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:22.573+0000 7f9c0ea00640 1 -- 172.21.15.5:0/4112549201 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c0803f070 con 0x7f9c181019c0 2024-09-10T09:38:22.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:22.573+0000 7f9c0ea00640 1 -- 172.21.15.5:0/4112549201 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9c08031c90 con 0x7f9c181019c0 2024-09-10T09:38:22.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:22.573+0000 7f9c0ea00640 1 -- 172.21.15.5:0/4112549201 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c08035030 con 0x7f9c181019c0 2024-09-10T09:38:22.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:22.573+0000 7f9c17200640 1 -- 172.21.15.5:0/4112549201 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c1819f740 con 0x7f9c181019c0 2024-09-10T09:38:22.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:22.573+0000 7f9c17200640 1 -- 172.21.15.5:0/4112549201 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c1819fbd0 con 0x7f9c181019c0 2024-09-10T09:38:22.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:22.575+0000 7f9c17200640 1 -- 172.21.15.5:0/4112549201 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c18101e20 con 0x7f9c181019c0 2024-09-10T09:38:22.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:22.577+0000 7f9c0ea00640 1 -- 172.21.15.5:0/4112549201 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 19) ==== 137883+0+0 (secure 0 0 0) 0x7f9c08042070 con 0x7f9c181019c0 2024-09-10T09:38:22.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:22.578+0000 7f9c0ea00640 1 --2- 172.21.15.5:0/4112549201 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f9bfc0a2c10 0x7f9bfc0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:22.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:22.578+0000 7f9c0ea00640 1 -- 172.21.15.5:0/4112549201 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f9c080effe0 con 0x7f9c181019c0 2024-09-10T09:38:22.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:22.579+0000 7f9c15400640 1 --2- 172.21.15.5:0/4112549201 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f9bfc0a2c10 0x7f9bfc0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:22.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:22.579+0000 7f9c15400640 1 --2- 172.21.15.5:0/4112549201 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f9bfc0a2c10 0x7f9bfc0a50d0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f9c1810dfa0 tx=0x7f9c04005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:38:22.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:22.584+0000 7f9c0ea00640 1 -- 172.21.15.5:0/4112549201 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f9c080bc380 con 0x7f9c181019c0 2024-09-10T09:38:22.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:22.815+0000 7f9c17200640 1 -- 172.21.15.5:0/4112549201 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9c181a71a0 con 0x7f9c181019c0 2024-09-10T09:38:22.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:22.816+0000 7f9c0ea00640 1 -- 172.21.15.5:0/4112549201 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11969 (secure 0 0 0) 0x7f9c080c0680 con 0x7f9c181019c0 2024-09-10T09:38:22.818 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-10T09:38:22.818 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":30,"btime":"2024-09-10T09:38:15:834198+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14454,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/2970594311","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2970594311},{"type":"v1","addr":"172.21.15.5:6835","nonce":2970594311}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14514,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6839/3616502174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":3616502174},{"type":"v1","addr":"172.21.15.5:6839","nonce":3616502174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T09:35:29.626287+0000","modified":"2024-09-10T09:38:15.658058+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24542,"mds_1":24535,"mds_2":24511,"mds_3":24506,"mds_4":24443},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14520":{"gid":14520,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/2073599842","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":2073599842},{"type":"v1","addr":"172.21.15.5:6841","nonce":2073599842}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"k","rank":4,"incarnation":26,"state":"up:active","state_seq":24,"addr":"172.21.15.102:6833/2457007427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":2457007427},{"type":"v1","addr":"172.21.15.102:6833","nonce":2457007427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/356445159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":356445159},{"type":"v1","addr":"172.21.15.159:6833","nonce":356445159}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6837/2894314507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":2894314507},{"type":"v1","addr":"172.21.15.5:6837","nonce":2894314507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.102:6835/1647549111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":1647549111},{"type":"v1","addr":"172.21.15.102:6835","nonce":1647549111}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6835/952657242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":952657242},{"type":"v1","addr":"172.21.15.159:6835","nonce":952657242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":19,"addr":"172.21.15.102:6837/2689776764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":2689776764},{"type":"v1","addr":"172.21.15.102:6837","nonce":2689776764}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.159:6837/620627639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":620627639},{"type":"v1","addr":"172.21.15.159:6837","nonce":620627639}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24535":{"gid":24535,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":18,"addr":"172.21.15.159:6839/2073739599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":2073739599},{"type":"v1","addr":"172.21.15.159:6839","nonce":2073739599}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24542":{"gid":24542,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.102:6839/335409749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6838","nonce":335409749},{"type":"v1","addr":"172.21.15.102:6839","nonce":335409749}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24542,"qdb_cluster":[24542,24535,24511,24506,24443]},"id":1}]} 2024-09-10T09:38:22.819 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 30 2024-09-10T09:38:22.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:22.818+0000 7f9c17200640 1 -- 172.21.15.5:0/4112549201 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f9bfc0a2c10 msgr2=0x7f9bfc0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:38:22.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:22.818+0000 7f9c17200640 1 --2- 172.21.15.5:0/4112549201 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f9bfc0a2c10 0x7f9bfc0a50d0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f9c1810dfa0 tx=0x7f9c04005f90 comp rx=0 tx=0).stop 2024-09-10T09:38:22.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:22.818+0000 7f9c17200640 1 -- 172.21.15.5:0/4112549201 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9c181019c0 msgr2=0x7f9c1819a500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:38:22.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:22.818+0000 7f9c17200640 1 --2- 172.21.15.5:0/4112549201 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9c181019c0 0x7f9c1819a500 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f9c0802a940 tx=0x7f9c08003b90 comp rx=0 tx=0).stop 2024-09-10T09:38:22.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:22.819+0000 7f9c17200640 1 -- 172.21.15.5:0/4112549201 shutdown_connections 2024-09-10T09:38:22.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:22.819+0000 7f9c17200640 1 --2- 172.21.15.5:0/4112549201 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f9bfc0a2c10 0x7f9bfc0a50d0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:22.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:22.819+0000 7f9c17200640 1 --2- 172.21.15.5:0/4112549201 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9c181079c0 0x7f9c1819edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:22.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:22.819+0000 7f9c17200640 1 --2- 172.21.15.5:0/4112549201 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9c18102360 0x7f9c1819aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:22.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:22.819+0000 7f9c17200640 1 --2- 172.21.15.5:0/4112549201 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9c181019c0 0x7f9c1819a500 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:22.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:22.819+0000 7f9c17200640 1 -- 172.21.15.5:0/4112549201 >> 172.21.15.5:0/4112549201 conn(0x7f9c180fd740 msgr2=0x7f9c1810cfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:38:22.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:22.819+0000 7f9c17200640 1 -- 172.21.15.5:0/4112549201 shutdown_connections 2024-09-10T09:38:22.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:22.819+0000 7f9c17200640 1 -- 172.21.15.5:0/4112549201 wait complete. 2024-09-10T09:38:23.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:38:22 smithi102 ceph-mon[35442]: pgmap v394: 129 pgs: 129 active+clean; 653 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.2 KiB/s wr, 12 op/s 2024-09-10T09:38:23.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:38:22 smithi102 ceph-mon[35442]: from='client.? 172.21.15.5:0/4112549201' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:38:23.220 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:38:22 smithi005 ceph-mon[30350]: pgmap v394: 129 pgs: 129 active+clean; 653 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.2 KiB/s wr, 12 op/s 2024-09-10T09:38:23.220 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:38:22 smithi005 ceph-mon[30350]: from='client.? 172.21.15.5:0/4112549201' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:38:23.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:38:22 smithi159 ceph-mon[35513]: pgmap v394: 129 pgs: 129 active+clean; 653 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.2 KiB/s wr, 12 op/s 2024-09-10T09:38:23.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:38:22 smithi159 ceph-mon[35513]: from='client.? 172.21.15.5:0/4112549201' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:38:23.711 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-10T09:38:24.092 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:38:24.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:24.641+0000 7f8511a00640 1 -- 172.21.15.5:0/596746096 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f850c1019c0 msgr2=0x7f850c101e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:38:24.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:24.641+0000 7f8511a00640 1 --2- 172.21.15.5:0/596746096 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f850c1019c0 0x7f850c101e20 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f84f4009890 tx=0x7f84f402a360 comp rx=0 tx=0).stop 2024-09-10T09:38:24.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:24.641+0000 7f8511a00640 1 -- 172.21.15.5:0/596746096 shutdown_connections 2024-09-10T09:38:24.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:24.641+0000 7f8511a00640 1 --2- 172.21.15.5:0/596746096 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f850c102360 0x7f850c10e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:24.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:24.641+0000 7f8511a00640 1 --2- 172.21.15.5:0/596746096 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f850c1019c0 0x7f850c101e20 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:24.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:24.641+0000 7f8511a00640 1 --2- 172.21.15.5:0/596746096 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f850c1079c0 0x7f850c107da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:24.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:24.641+0000 7f8511a00640 1 -- 172.21.15.5:0/596746096 >> 172.21.15.5:0/596746096 conn(0x7f850c0fd6e0 msgr2=0x7f850c0ffb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:38:24.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:24.642+0000 7f8511a00640 1 -- 172.21.15.5:0/596746096 shutdown_connections 2024-09-10T09:38:24.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:24.642+0000 7f8511a00640 1 -- 172.21.15.5:0/596746096 wait complete. 2024-09-10T09:38:24.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:24.642+0000 7f8511a00640 1 Processor -- start 2024-09-10T09:38:24.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:24.643+0000 7f8511a00640 1 -- start start 2024-09-10T09:38:24.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:24.643+0000 7f8511a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f850c1019c0 0x7f850c19a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:24.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:24.643+0000 7f8511a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f850c102360 0x7f850c19a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:24.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:24.643+0000 7f8511a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f850c1079c0 0x7f850c19ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:24.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:24.643+0000 7f8511a00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f850c111dc0 con 0x7f850c1079c0 2024-09-10T09:38:24.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:24.643+0000 7f8511a00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f850c111c40 con 0x7f850c102360 2024-09-10T09:38:24.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:24.643+0000 7f8511a00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f850c111f40 con 0x7f850c1019c0 2024-09-10T09:38:24.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:24.643+0000 7f850be00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f850c1019c0 0x7f850c19a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:24.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:24.643+0000 7f8511000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f850c1079c0 0x7f850c19ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:24.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:24.643+0000 7f8503e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f850c102360 0x7f850c19a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:24.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:24.643+0000 7f850be00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f850c1019c0 0x7f850c19a470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:41490/0 (socket says 172.21.15.5:41490) 2024-09-10T09:38:24.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:24.643+0000 7f850be00640 1 -- 172.21.15.5:0/445628612 learned_addr learned my addr 172.21.15.5:0/445628612 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:38:24.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:24.644+0000 7f8503e00640 1 -- 172.21.15.5:0/445628612 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f850c1019c0 msgr2=0x7f850c19a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:38:24.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:24.644+0000 7f8503e00640 1 --2- 172.21.15.5:0/445628612 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f850c1019c0 0x7f850c19a470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:24.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:24.644+0000 7f8503e00640 1 -- 172.21.15.5:0/445628612 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f850c1079c0 msgr2=0x7f850c19ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:38:24.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:24.644+0000 7f8503e00640 1 --2- 172.21.15.5:0/445628612 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f850c1079c0 0x7f850c19ed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:24.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:24.644+0000 7f8503e00640 1 -- 172.21.15.5:0/445628612 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f850c19f420 con 0x7f850c102360 2024-09-10T09:38:24.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:24.644+0000 7f850be00640 1 --2- 172.21.15.5:0/445628612 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f850c1019c0 0x7f850c19a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:38:24.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:24.644+0000 7f8503e00640 1 --2- 172.21.15.5:0/445628612 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f850c102360 0x7f850c19a9b0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f84f4035040 tx=0x7f84f4033ae0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:38:24.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:24.644+0000 7f8509600640 1 -- 172.21.15.5:0/445628612 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84f4037040 con 0x7f850c102360 2024-09-10T09:38:24.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:24.644+0000 7f8511a00640 1 -- 172.21.15.5:0/445628612 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f850c19f6b0 con 0x7f850c102360 2024-09-10T09:38:24.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:24.644+0000 7f8509600640 1 -- 172.21.15.5:0/445628612 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f84f4033d90 con 0x7f850c102360 2024-09-10T09:38:24.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:24.644+0000 7f8509600640 1 -- 172.21.15.5:0/445628612 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84f4003830 con 0x7f850c102360 2024-09-10T09:38:24.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:24.644+0000 7f8511a00640 1 -- 172.21.15.5:0/445628612 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f850c1a6ff0 con 0x7f850c102360 2024-09-10T09:38:24.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:24.645+0000 7f8511a00640 1 -- 172.21.15.5:0/445628612 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f850c101e20 con 0x7f850c102360 2024-09-10T09:38:24.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:24.647+0000 7f8509600640 1 -- 172.21.15.5:0/445628612 <== mon.2 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 19) ==== 137883+0+0 (secure 0 0 0) 0x7f84f4042020 con 0x7f850c102360 2024-09-10T09:38:24.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:24.649+0000 7f8509600640 1 --2- 172.21.15.5:0/445628612 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f84f00a2c90 0x7f84f00a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:24.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:24.649+0000 7f8509600640 1 -- 172.21.15.5:0/445628612 <== mon.2 v2:172.21.15.102:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f84f40eedf0 con 0x7f850c102360 2024-09-10T09:38:24.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:24.649+0000 7f850be00640 1 --2- 172.21.15.5:0/445628612 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f84f00a2c90 0x7f84f00a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:24.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:24.650+0000 7f850be00640 1 --2- 172.21.15.5:0/445628612 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f84f00a2c90 0x7f84f00a5150 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f84fc0096c0 tx=0x7f84fc009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:38:24.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:24.651+0000 7f8509600640 1 -- 172.21.15.5:0/445628612 <== mon.2 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f84f4049020 con 0x7f850c102360 2024-09-10T09:38:24.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:24.838+0000 7f8511a00640 1 -- 172.21.15.5:0/445628612 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f850c1a71f0 con 0x7f850c102360 2024-09-10T09:38:24.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:24.840+0000 7f8509600640 1 -- 172.21.15.5:0/445628612 <== mon.2 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v96) ==== 74+0+24846 (secure 0 0 0) 0x7f84f40bb8b0 con 0x7f850c102360 2024-09-10T09:38:24.842 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-10T09:38:24.842 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":96,"fsid":"bade4b50-6f56-11ef-bcea-c7b262605968","created":"2024-09-10T09:27:13.298498+0000","modified":"2024-09-10T09:37:07.520520+0000","last_up_change":"2024-09-10T09:34:30.730066+0000","last_in_change":"2024-09-10T09:34:07.340995+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T09:30:26.042255+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T09:35:24.461031+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T09:35:27.155349+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-10T09:35:35.121462+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"085749a8-e882-4377-92dd-34bb266c0f0b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":954535266},{"type":"v1","addr":"172.21.15.5:6803","nonce":954535266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":954535266},{"type":"v1","addr":"172.21.15.5:6805","nonce":954535266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":954535266},{"type":"v1","addr":"172.21.15.5:6809","nonce":954535266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":954535266},{"type":"v1","addr":"172.21.15.5:6807","nonce":954535266}]},"public_addr":"172.21.15.5:6803/954535266","cluster_addr":"172.21.15.5:6805/954535266","heartbeat_back_addr":"172.21.15.5:6809/954535266","heartbeat_front_addr":"172.21.15.5:6807/954535266","state":["exists","up"]},{"osd":1,"uuid":"0a76c795-2573-4a92-969a-2ec5dd65c094","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6800","nonce":733924375},{"type":"v1","addr":"172.21.15.102:6801","nonce":733924375}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6802","nonce":733924375},{"type":"v1","addr":"172.21.15.102:6803","nonce":733924375}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6806","nonce":733924375},{"type":"v1","addr":"172.21.15.102:6807","nonce":733924375}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6804","nonce":733924375},{"type":"v1","addr":"172.21.15.102:6805","nonce":733924375}]},"public_addr":"172.21.15.102:6801/733924375","cluster_addr":"172.21.15.102:6803/733924375","heartbeat_back_addr":"172.21.15.102:6807/733924375","heartbeat_front_addr":"172.21.15.102:6805/733924375","state":["exists","up"]},{"osd":2,"uuid":"0c98fcb8-60ab-4473-8995-1d9e97c577e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6800","nonce":2236204189},{"type":"v1","addr":"172.21.15.159:6801","nonce":2236204189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":2236204189},{"type":"v1","addr":"172.21.15.159:6803","nonce":2236204189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":2236204189},{"type":"v1","addr":"172.21.15.159:6807","nonce":2236204189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":2236204189},{"type":"v1","addr":"172.21.15.159:6805","nonce":2236204189}]},"public_addr":"172.21.15.159:6801/2236204189","cluster_addr":"172.21.15.159:6803/2236204189","heartbeat_back_addr":"172.21.15.159:6807/2236204189","heartbeat_front_addr":"172.21.15.159:6805/2236204189","state":["exists","up"]},{"osd":3,"uuid":"88eecdeb-5cee-4f78-a3f9-a66bf4bf772a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":2221468725},{"type":"v1","addr":"172.21.15.5:6811","nonce":2221468725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":2221468725},{"type":"v1","addr":"172.21.15.5:6813","nonce":2221468725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6816","nonce":2221468725},{"type":"v1","addr":"172.21.15.5:6817","nonce":2221468725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":2221468725},{"type":"v1","addr":"172.21.15.5:6815","nonce":2221468725}]},"public_addr":"172.21.15.5:6811/2221468725","cluster_addr":"172.21.15.5:6813/2221468725","heartbeat_back_addr":"172.21.15.5:6817/2221468725","heartbeat_front_addr":"172.21.15.5:6815/2221468725","state":["exists","up"]},{"osd":4,"uuid":"f2e7314d-7938-4668-9011-45c175f66b48","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6808","nonce":594465753},{"type":"v1","addr":"172.21.15.102:6809","nonce":594465753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6810","nonce":594465753},{"type":"v1","addr":"172.21.15.102:6811","nonce":594465753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6814","nonce":594465753},{"type":"v1","addr":"172.21.15.102:6815","nonce":594465753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6812","nonce":594465753},{"type":"v1","addr":"172.21.15.102:6813","nonce":594465753}]},"public_addr":"172.21.15.102:6809/594465753","cluster_addr":"172.21.15.102:6811/594465753","heartbeat_back_addr":"172.21.15.102:6815/594465753","heartbeat_front_addr":"172.21.15.102:6813/594465753","state":["exists","up"]},{"osd":5,"uuid":"0c0366d3-658b-4bf0-ba17-aff28a02e948","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":3230280402},{"type":"v1","addr":"172.21.15.159:6809","nonce":3230280402}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":3230280402},{"type":"v1","addr":"172.21.15.159:6811","nonce":3230280402}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":3230280402},{"type":"v1","addr":"172.21.15.159:6815","nonce":3230280402}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":3230280402},{"type":"v1","addr":"172.21.15.159:6813","nonce":3230280402}]},"public_addr":"172.21.15.159:6809/3230280402","cluster_addr":"172.21.15.159:6811/3230280402","heartbeat_back_addr":"172.21.15.159:6815/3230280402","heartbeat_front_addr":"172.21.15.159:6813/3230280402","state":["exists","up"]},{"osd":6,"uuid":"682670a6-51d5-46e8-af6e-dc2d8c1cf407","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":1907651140},{"type":"v1","addr":"172.21.15.5:6819","nonce":1907651140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":1907651140},{"type":"v1","addr":"172.21.15.5:6821","nonce":1907651140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":1907651140},{"type":"v1","addr":"172.21.15.5:6825","nonce":1907651140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":1907651140},{"type":"v1","addr":"172.21.15.5:6823","nonce":1907651140}]},"public_addr":"172.21.15.5:6819/1907651140","cluster_addr":"172.21.15.5:6821/1907651140","heartbeat_back_addr":"172.21.15.5:6825/1907651140","heartbeat_front_addr":"172.21.15.5:6823/1907651140","state":["exists","up"]},{"osd":7,"uuid":"8ae0f4a9-99e7-4563-aa2a-15169684a8b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6816","nonce":692040360},{"type":"v1","addr":"172.21.15.102:6817","nonce":692040360}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6818","nonce":692040360},{"type":"v1","addr":"172.21.15.102:6819","nonce":692040360}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6822","nonce":692040360},{"type":"v1","addr":"172.21.15.102:6823","nonce":692040360}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6820","nonce":692040360},{"type":"v1","addr":"172.21.15.102:6821","nonce":692040360}]},"public_addr":"172.21.15.102:6817/692040360","cluster_addr":"172.21.15.102:6819/692040360","heartbeat_back_addr":"172.21.15.102:6823/692040360","heartbeat_front_addr":"172.21.15.102:6821/692040360","state":["exists","up"]},{"osd":8,"uuid":"e37c845e-ced9-4413-9bbe-bc090537c2bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":2107291799},{"type":"v1","addr":"172.21.15.159:6817","nonce":2107291799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":2107291799},{"type":"v1","addr":"172.21.15.159:6819","nonce":2107291799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":2107291799},{"type":"v1","addr":"172.21.15.159:6823","nonce":2107291799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":2107291799},{"type":"v1","addr":"172.21.15.159:6821","nonce":2107291799}]},"public_addr":"172.21.15.159:6817/2107291799","cluster_addr":"172.21.15.159:6819/2107291799","heartbeat_back_addr":"172.21.15.159:6823/2107291799","heartbeat_front_addr":"172.21.15.159:6821/2107291799","state":["exists","up"]},{"osd":9,"uuid":"91a5b6cc-08da-47be-af12-d0896a849309","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":902431281},{"type":"v1","addr":"172.21.15.5:6827","nonce":902431281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":902431281},{"type":"v1","addr":"172.21.15.5:6829","nonce":902431281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":902431281},{"type":"v1","addr":"172.21.15.5:6833","nonce":902431281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":902431281},{"type":"v1","addr":"172.21.15.5:6831","nonce":902431281}]},"public_addr":"172.21.15.5:6827/902431281","cluster_addr":"172.21.15.5:6829/902431281","heartbeat_back_addr":"172.21.15.5:6833/902431281","heartbeat_front_addr":"172.21.15.5:6831/902431281","state":["exists","up"]},{"osd":10,"uuid":"2211030c-b00e-402b-b4d7-c587eb29a490","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6824","nonce":2853806211},{"type":"v1","addr":"172.21.15.102:6825","nonce":2853806211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6826","nonce":2853806211},{"type":"v1","addr":"172.21.15.102:6827","nonce":2853806211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6830","nonce":2853806211},{"type":"v1","addr":"172.21.15.102:6831","nonce":2853806211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6828","nonce":2853806211},{"type":"v1","addr":"172.21.15.102:6829","nonce":2853806211}]},"public_addr":"172.21.15.102:6825/2853806211","cluster_addr":"172.21.15.102:6827/2853806211","heartbeat_back_addr":"172.21.15.102:6831/2853806211","heartbeat_front_addr":"172.21.15.102:6829/2853806211","state":["exists","up"]},{"osd":11,"uuid":"4e9699ec-8ff2-4005-9d84-af5600b3a2ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6824","nonce":969941218},{"type":"v1","addr":"172.21.15.159:6825","nonce":969941218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":969941218},{"type":"v1","addr":"172.21.15.159:6827","nonce":969941218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6830","nonce":969941218},{"type":"v1","addr":"172.21.15.159:6831","nonce":969941218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":969941218},{"type":"v1","addr":"172.21.15.159:6829","nonce":969941218}]},"public_addr":"172.21.15.159:6825/969941218","cluster_addr":"172.21.15.159:6827/969941218","heartbeat_back_addr":"172.21.15.159:6831/969941218","heartbeat_front_addr":"172.21.15.159:6829/969941218","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:29:34.416490+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:30:00.246101+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:30:24.300464+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:30:54.663623+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:31:21.426292+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:31:46.738914+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:32:16.196571+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:32:42.549243+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:33:08.178651+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:33:37.477911+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:34:03.114026+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:34:28.891924+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4}]},{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"3.0","mappings":[{"from":4,"to":8}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":6}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9},{"from":5,"to":7}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]},{"pgid":"3.39","mappings":[{"from":5,"to":6}]},{"pgid":"4.8","mappings":[{"from":5,"to":8}]},{"pgid":"4.c","mappings":[{"from":9,"to":10}]},{"pgid":"4.18","mappings":[{"from":9,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.5:6801/1527422686":"2024-09-11T09:28:05.436742+0000","172.21.15.5:0/2276144345":"2024-09-11T09:28:05.436742+0000","172.21.15.5:0/412040104":"2024-09-11T09:28:05.436742+0000","172.21.15.5:6800/1527422686":"2024-09-11T09:28:05.436742+0000","172.21.15.5:0/1856581170":"2024-09-11T09:28:05.436742+0000","172.21.15.5:0/2847668298":"2024-09-11T09:27:37.778500+0000","172.21.15.5:6801/14425913":"2024-09-11T09:27:37.778500+0000","172.21.15.5:0/166910782":"2024-09-11T09:27:37.778500+0000","172.21.15.5:6800/14425913":"2024-09-11T09:27:37.778500+0000","172.21.15.5:0/1711858179":"2024-09-11T09:27:37.778500+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T09:38:24.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:24.842+0000 7f8511a00640 1 -- 172.21.15.5:0/445628612 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f84f00a2c90 msgr2=0x7f84f00a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:38:24.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:24.842+0000 7f8511a00640 1 --2- 172.21.15.5:0/445628612 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f84f00a2c90 0x7f84f00a5150 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f84fc0096c0 tx=0x7f84fc009260 comp rx=0 tx=0).stop 2024-09-10T09:38:24.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:24.842+0000 7f8511a00640 1 -- 172.21.15.5:0/445628612 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f850c102360 msgr2=0x7f850c19a9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:38:24.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:24.842+0000 7f8511a00640 1 --2- 172.21.15.5:0/445628612 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f850c102360 0x7f850c19a9b0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f84f4035040 tx=0x7f84f4033ae0 comp rx=0 tx=0).stop 2024-09-10T09:38:24.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:24.842+0000 7f8511a00640 1 -- 172.21.15.5:0/445628612 shutdown_connections 2024-09-10T09:38:24.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:24.842+0000 7f8511a00640 1 --2- 172.21.15.5:0/445628612 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f84f00a2c90 0x7f84f00a5150 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:24.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:24.842+0000 7f8511a00640 1 --2- 172.21.15.5:0/445628612 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f850c1079c0 0x7f850c19ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:24.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:24.842+0000 7f8511a00640 1 --2- 172.21.15.5:0/445628612 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f850c102360 0x7f850c19a9b0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:24.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:24.842+0000 7f8511a00640 1 --2- 172.21.15.5:0/445628612 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f850c1019c0 0x7f850c19a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:24.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:24.842+0000 7f8511a00640 1 -- 172.21.15.5:0/445628612 >> 172.21.15.5:0/445628612 conn(0x7f850c0fd6e0 msgr2=0x7f850c0fdc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:38:24.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:24.843+0000 7f8511a00640 1 -- 172.21.15.5:0/445628612 shutdown_connections 2024-09-10T09:38:24.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:24.843+0000 7f8511a00640 1 -- 172.21.15.5:0/445628612 wait complete. 2024-09-10T09:38:25.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:38:24 smithi102 ceph-mon[35442]: pgmap v395: 129 pgs: 129 active+clean; 653 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.2 KiB/s wr, 9 op/s 2024-09-10T09:38:25.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:38:24 smithi102 ceph-mon[35442]: from='client.? 172.21.15.5:0/445628612' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T09:38:25.209 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:38:24 smithi005 ceph-mon[30350]: pgmap v395: 129 pgs: 129 active+clean; 653 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.2 KiB/s wr, 9 op/s 2024-09-10T09:38:25.209 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:38:24 smithi005 ceph-mon[30350]: from='client.? 172.21.15.5:0/445628612' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T09:38:25.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:38:24 smithi159 ceph-mon[35513]: pgmap v395: 129 pgs: 129 active+clean; 653 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.2 KiB/s wr, 9 op/s 2024-09-10T09:38:25.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:38:24 smithi159 ceph-mon[35513]: from='client.? 172.21.15.5:0/445628612' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T09:38:25.688 DEBUG:tasks.fwd_scrub:Yielding 2024-09-10T09:38:25.688 INFO:teuthology.run_tasks:Running task quiescer... 2024-09-10T09:38:25.702 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-10T09:38:25.702 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-10T09:38:25.702 INFO:tasks.fwd_scrub.fs.[cephfs]:start scrubbing fs: cephfs 2024-09-10T09:38:25.702 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-10T09:38:25.703 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-10T09:38:26.107 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:38:26.112 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:38:26.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:26.897+0000 7f5244800640 1 -- 172.21.15.5:0/3454572568 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f523c077040 msgr2=0x7f523c1124f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:38:26.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:26.897+0000 7f5244800640 1 --2- 172.21.15.5:0/3454572568 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f523c077040 0x7f523c1124f0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f522c009900 tx=0x7f522c02a3f0 comp rx=0 tx=0).stop 2024-09-10T09:38:26.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:26.897+0000 7f5244800640 1 -- 172.21.15.5:0/3454572568 shutdown_connections 2024-09-10T09:38:26.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:26.897+0000 7f5244800640 1 --2- 172.21.15.5:0/3454572568 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f523c112a30 0x7f523c114e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:26.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:26.897+0000 7f5244800640 1 --2- 172.21.15.5:0/3454572568 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f523c077040 0x7f523c1124f0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:26.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:26.897+0000 7f5244800640 1 --2- 172.21.15.5:0/3454572568 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f523c076720 0x7f523c076b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:26.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:26.897+0000 7f5244800640 1 -- 172.21.15.5:0/3454572568 >> 172.21.15.5:0/3454572568 conn(0x7f523c0ffa90 msgr2=0x7f523c101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:38:26.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:26.897+0000 7f5244800640 1 -- 172.21.15.5:0/3454572568 shutdown_connections 2024-09-10T09:38:26.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:26.897+0000 7f5244800640 1 -- 172.21.15.5:0/3454572568 wait complete. 2024-09-10T09:38:26.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:26.898+0000 7f5244800640 1 Processor -- start 2024-09-10T09:38:26.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:26.898+0000 7f5244800640 1 -- start start 2024-09-10T09:38:26.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:26.898+0000 7f5244800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f523c076720 0x7f523c1a4740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:26.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:26.898+0000 7f5244800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f523c077040 0x7f523c1a4c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:26.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:26.899+0000 7f5244800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f523c112a30 0x7f523c19e900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:26.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:26.899+0000 7f5244800640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f523c118610 con 0x7f523c076720 2024-09-10T09:38:26.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:26.899+0000 7f5244800640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f523c118490 con 0x7f523c112a30 2024-09-10T09:38:26.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:26.899+0000 7f5244800640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f523c118790 con 0x7f523c077040 2024-09-10T09:38:26.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:26.899+0000 7f5241e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f523c076720 0x7f523c1a4740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:26.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:26.899+0000 7f5241e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f523c076720 0x7f523c1a4740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60848/0 (socket says 172.21.15.5:60848) 2024-09-10T09:38:26.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:26.899+0000 7f5241e00640 1 -- 172.21.15.5:0/2577148275 learned_addr learned my addr 172.21.15.5:0/2577148275 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:38:26.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:26.899+0000 7f5241400640 1 --2- 172.21.15.5:0/2577148275 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f523c077040 0x7f523c1a4c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:26.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:26.899+0000 7f5242800640 1 --2- 172.21.15.5:0/2577148275 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f523c112a30 0x7f523c19e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:26.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:26.899+0000 7f5241e00640 1 -- 172.21.15.5:0/2577148275 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f523c077040 msgr2=0x7f523c1a4c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:38:26.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:26.899+0000 7f5241e00640 1 --2- 172.21.15.5:0/2577148275 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f523c077040 0x7f523c1a4c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:26.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:26.899+0000 7f5241e00640 1 -- 172.21.15.5:0/2577148275 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f523c112a30 msgr2=0x7f523c19e900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:38:26.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:26.899+0000 7f5241e00640 1 --2- 172.21.15.5:0/2577148275 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f523c112a30 0x7f523c19e900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:26.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:26.899+0000 7f5241e00640 1 -- 172.21.15.5:0/2577148275 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f523c19f030 con 0x7f523c076720 2024-09-10T09:38:26.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:26.900+0000 7f5241e00640 1 --2- 172.21.15.5:0/2577148275 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f523c076720 0x7f523c1a4740 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f523000e9f0 tx=0x7f523000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:38:26.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:26.900+0000 7f5236a00640 1 -- 172.21.15.5:0/2577148275 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f523000ccc0 con 0x7f523c076720 2024-09-10T09:38:26.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:26.900+0000 7f5236a00640 1 -- 172.21.15.5:0/2577148275 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f523000ce60 con 0x7f523c076720 2024-09-10T09:38:26.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:26.900+0000 7f5244800640 1 -- 172.21.15.5:0/2577148275 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f523c19f320 con 0x7f523c076720 2024-09-10T09:38:26.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:26.900+0000 7f5236a00640 1 -- 172.21.15.5:0/2577148275 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5230010690 con 0x7f523c076720 2024-09-10T09:38:26.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:26.900+0000 7f5244800640 1 -- 172.21.15.5:0/2577148275 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f523c1ab610 con 0x7f523c076720 2024-09-10T09:38:26.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:26.903+0000 7f5236a00640 1 -- 172.21.15.5:0/2577148275 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 19) ==== 137883+0+0 (secure 0 0 0) 0x7f52300046c0 con 0x7f523c076720 2024-09-10T09:38:26.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:26.903+0000 7f5247215740 1 Processor -- start 2024-09-10T09:38:26.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:26.904+0000 7f5247215740 1 -- start start 2024-09-10T09:38:26.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:26.904+0000 7f5247215740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559e114fcac0 0x559e114fcea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:26.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:26.904+0000 7f5247215740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559e114fd3e0 0x559e11507760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:26.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:26.904+0000 7f5247215740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559e11507ca0 0x559e1150a090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:26.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:26.904+0000 7f5247215740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x559e11463290 con 0x559e114fcac0 2024-09-10T09:38:26.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:26.904+0000 7f5247215740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x559e1140ab40 con 0x559e11507ca0 2024-09-10T09:38:26.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:26.904+0000 7f5247215740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x559e1145f250 con 0x559e114fd3e0 2024-09-10T09:38:26.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:26.904+0000 7f5242800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559e114fcac0 0x559e114fcea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:26.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:26.904+0000 7f5241400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559e11507ca0 0x559e1150a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:26.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:26.904+0000 7f5242800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559e114fcac0 0x559e114fcea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60862/0 (socket says 172.21.15.5:60862) 2024-09-10T09:38:26.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:26.904+0000 7f5242800640 1 -- 172.21.15.5:0/1645567762 learned_addr learned my addr 172.21.15.5:0/1645567762 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:38:26.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:26.905+0000 7f5241e00640 1 --2- 172.21.15.5:0/1645567762 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559e114fd3e0 0x559e11507760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:26.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:26.905+0000 7f5242800640 1 -- 172.21.15.5:0/1645567762 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559e114fd3e0 msgr2=0x559e11507760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:38:26.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:26.905+0000 7f5242800640 1 --2- 172.21.15.5:0/1645567762 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559e114fd3e0 0x559e11507760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:26.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:26.905+0000 7f5242800640 1 -- 172.21.15.5:0/1645567762 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559e11507ca0 msgr2=0x559e1150a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:38:26.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:26.905+0000 7f5242800640 1 --2- 172.21.15.5:0/1645567762 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559e11507ca0 0x559e1150a090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:26.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:26.905+0000 7f5242800640 1 -- 172.21.15.5:0/1645567762 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559e111d46a0 con 0x559e114fcac0 2024-09-10T09:38:26.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:26.905+0000 7f5236a00640 1 --2- 172.21.15.5:0/2577148275 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f52240a2c10 0x7f52240a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:26.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:26.905+0000 7f5242800640 1 --2- 172.21.15.5:0/1645567762 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559e114fcac0 0x559e114fcea0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f5238009f40 tx=0x7f52380118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=241eca193251de93 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:38:26.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:26.905+0000 7f5236a00640 1 -- 172.21.15.5:0/2577148275 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f52300cfa40 con 0x7f523c076720 2024-09-10T09:38:26.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:26.906+0000 7f5241e00640 1 --2- 172.21.15.5:0/2577148275 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f52240a2c10 0x7f52240a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:26.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:26.906+0000 7f522b000640 1 -- 172.21.15.5:0/1645567762 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f523801e070 con 0x559e114fcac0 2024-09-10T09:38:26.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:26.906+0000 7f522b000640 1 -- 172.21.15.5:0/1645567762 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5238003c30 con 0x559e114fcac0 2024-09-10T09:38:26.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:26.906+0000 7f522b000640 1 -- 172.21.15.5:0/1645567762 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f523800f9c0 con 0x559e114fcac0 2024-09-10T09:38:26.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:26.907+0000 7f5247215740 1 -- 172.21.15.5:0/1645567762 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559e114fcac0 msgr2=0x559e114fcea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:38:26.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:26.907+0000 7f5241e00640 1 --2- 172.21.15.5:0/2577148275 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f52240a2c10 0x7f52240a50d0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f523000c6a0 tx=0x7f52300d34f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:38:26.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:26.907+0000 7f5247215740 1 --2- 172.21.15.5:0/1645567762 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559e114fcac0 0x559e114fcea0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f5238009f40 tx=0x7f52380118d0 comp rx=0 tx=0).stop 2024-09-10T09:38:26.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:26.907+0000 7f5247215740 1 -- 172.21.15.5:0/1645567762 shutdown_connections 2024-09-10T09:38:26.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:26.907+0000 7f5247215740 1 --2- 172.21.15.5:0/1645567762 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559e11507ca0 0x559e1150a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:26.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:26.907+0000 7f5247215740 1 --2- 172.21.15.5:0/1645567762 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559e114fd3e0 0x559e11507760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:26.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:26.907+0000 7f5247215740 1 --2- 172.21.15.5:0/1645567762 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559e114fcac0 0x559e114fcea0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:26.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:26.907+0000 7f5247215740 1 -- 172.21.15.5:0/1645567762 >> 172.21.15.5:0/1645567762 conn(0x559e114f9050 msgr2=0x559e114f9430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:38:26.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:26.907+0000 7f5247215740 1 -- 172.21.15.5:0/1645567762 shutdown_connections 2024-09-10T09:38:26.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:26.907+0000 7f5247215740 1 -- 172.21.15.5:0/1645567762 wait complete. 2024-09-10T09:38:26.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:26.907+0000 7f5247215740 1 -- start start 2024-09-10T09:38:26.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:26.908+0000 7f5247215740 1 Processor -- start 2024-09-10T09:38:26.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:26.908+0000 7f5247215740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559e114fcac0 0x559e115075e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:26.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:26.908+0000 7f5247215740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559e114fd3e0 0x559e11504890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:26.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:26.908+0000 7f5247215740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559e11507ca0 0x559e11504dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:26.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:26.908+0000 7f5247215740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x559e1150a910 con 0x559e11507ca0 2024-09-10T09:38:26.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:26.908+0000 7f5247215740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x559e1141dfa0 con 0x559e114fcac0 2024-09-10T09:38:26.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:26.908+0000 7f5247215740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x559e1150aa90 con 0x559e114fd3e0 2024-09-10T09:38:26.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:26.908+0000 7f5247215740 10 client.0 _ll_register_callbacks cb 0x559e11171ba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:38:26.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:26.908+0000 7f5242800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559e114fcac0 0x559e115075e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:26.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:26.908+0000 7f5242800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559e11507ca0 0x559e11504dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:26.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:26.908+0000 7f5241400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559e114fd3e0 0x559e11504890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:26.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:26.908+0000 7f5242800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559e11507ca0 0x559e11504dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60870/0 (socket says 172.21.15.5:60870) 2024-09-10T09:38:26.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:26.908+0000 7f5242800640 1 -- 172.21.15.5:0/904724654 learned_addr learned my addr 172.21.15.5:0/904724654 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:38:26.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:26.909+0000 7f5241400640 1 -- 172.21.15.5:0/904724654 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559e114fcac0 msgr2=0x559e115075e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:38:26.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:26.909+0000 7f5241400640 1 --2- 172.21.15.5:0/904724654 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559e114fcac0 0x559e115075e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:26.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:26.909+0000 7f5241400640 1 -- 172.21.15.5:0/904724654 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559e11507ca0 msgr2=0x559e11504dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:38:26.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:26.909+0000 7f5241400640 1 --2- 172.21.15.5:0/904724654 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559e11507ca0 0x559e11504dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:26.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:26.909+0000 7f5241400640 1 -- 172.21.15.5:0/904724654 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559e11505600 con 0x559e114fd3e0 2024-09-10T09:38:26.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:26.909+0000 7f5241400640 1 --2- 172.21.15.5:0/904724654 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559e114fd3e0 0x559e11504890 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f522c02a9e0 tx=0x7f522c004390 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cb02e09fa7c9ed55 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:38:26.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:26.909+0000 7f521f400640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-09-10T09:38:26.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:26.909+0000 7f521f400640 1 -- 172.21.15.5:0/904724654 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f522c035040 con 0x559e114fd3e0 2024-09-10T09:38:26.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:26.910+0000 7f5247215740 1 -- 172.21.15.5:0/904724654 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5238003c30 con 0x559e114fd3e0 2024-09-10T09:38:26.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:26.910+0000 7f521f400640 1 -- 172.21.15.5:0/904724654 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f522c005450 con 0x559e114fd3e0 2024-09-10T09:38:26.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:26.910+0000 7f521f400640 1 -- 172.21.15.5:0/904724654 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f522c04f050 con 0x559e114fd3e0 2024-09-10T09:38:26.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:26.910+0000 7f5247215740 10 client.25324 fetch_fsmap learned FSMap version 30 2024-09-10T09:38:26.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:26.910+0000 7f5247215740 1 -- 172.21.15.5:0/904724654 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x559e11516870 con 0x559e114fd3e0 2024-09-10T09:38:26.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:26.910+0000 7f521f400640 1 -- 172.21.15.5:0/904724654 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5238003c30 con 0x559e114fd3e0 2024-09-10T09:38:26.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:26.910+0000 7f521f400640 1 -- 172.21.15.5:0/904724654 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f522c003830 con 0x559e114fd3e0 2024-09-10T09:38:26.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:26.911+0000 7f5247215740 10 client.25324 fetch_fsmap finished waiting for FSMap version 30 2024-09-10T09:38:26.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:26.911+0000 7f5247215740 10 client.25324 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T09:38:26.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:26.911+0000 7f5247215740 20 client.25324 populate_metadata read hostname 'smithi005' 2024-09-10T09:38:26.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:26.911+0000 7f5247215740 1 --2- 172.21.15.5:0/904724654 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x559e11517c80 0x559e1151a070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:26.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:26.911+0000 7f5247215740 4 client.25324 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:38:26.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:26.911+0000 7f5247215740 1 -- 172.21.15.5:0/904724654 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559e11470800 con 0x559e11517c80 2024-09-10T09:38:26.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:26.911+0000 7f5242800640 1 --2- 172.21.15.5:0/904724654 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x559e11517c80 0x559e1151a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:26.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:26.911+0000 7f5242800640 1 --2- 172.21.15.5:0/904724654 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x559e11517c80 0x559e1151a070 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=660fa60f8284b3c server_cookie=f49e82e1d2cf4862 in_seq=0 out_seq=0 2024-09-10T09:38:26.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:26.911+0000 7f521f400640 10 client.25324 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T09:38:26.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:26.912+0000 7f521f400640 1 -- 172.21.15.5:0/904724654 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559e11470800 con 0x559e11517c80 2024-09-10T09:38:26.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:26.912+0000 7f521f400640 10 client.25324 handle_command_reply: tid=0 2024-09-10T09:38:26.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:26.912+0000 7f5247215740 1 client.25324 shutdown 2024-09-10T09:38:26.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:26.912+0000 7f5247215740 1 -- 172.21.15.5:0/904724654 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559e114fd3e0 msgr2=0x559e11504890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:38:26.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:26.912+0000 7f5247215740 1 --2- 172.21.15.5:0/904724654 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559e114fd3e0 0x559e11504890 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f522c02a9e0 tx=0x7f522c004390 comp rx=0 tx=0).stop 2024-09-10T09:38:26.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:26.913+0000 7f5247215740 1 -- 172.21.15.5:0/904724654 shutdown_connections 2024-09-10T09:38:26.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:26.913+0000 7f5247215740 1 --2- 172.21.15.5:0/904724654 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x559e11517c80 0x559e1151a070 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:26.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:26.913+0000 7f5247215740 1 --2- 172.21.15.5:0/904724654 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559e11507ca0 0x559e11504dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:26.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:26.913+0000 7f5247215740 1 --2- 172.21.15.5:0/904724654 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559e114fd3e0 0x559e11504890 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:26.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:26.913+0000 7f5247215740 1 --2- 172.21.15.5:0/904724654 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559e114fcac0 0x559e115075e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:26.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:26.913+0000 7f521f400640 0 client.25324 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T09:38:26.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:26.913+0000 7f5247215740 1 -- 172.21.15.5:0/904724654 >> 172.21.15.5:0/904724654 conn(0x559e114f9050 msgr2=0x559e114f81c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:38:26.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:26.913+0000 7f5247215740 1 -- 172.21.15.5:0/904724654 shutdown_connections 2024-09-10T09:38:26.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:26.913+0000 7f5247215740 1 -- 172.21.15.5:0/904724654 wait complete. 2024-09-10T09:38:26.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:26.913+0000 7f5247215740 20 client.25324 trim_cache size 0 max 16384 2024-09-10T09:38:26.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:26.931+0000 7f5247215740 1 Processor -- start 2024-09-10T09:38:26.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:26.931+0000 7f5247215740 1 -- start start 2024-09-10T09:38:26.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:26.932+0000 7f5247215740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559e114fcac0 0x559e11503870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:26.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:26.932+0000 7f5247215740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559e114fd3e0 0x559e1151f910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:26.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:26.932+0000 7f5247215740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559e11507ca0 0x559e1151fe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:26.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:26.932+0000 7f5247215740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f523c1b5400 con 0x559e114fd3e0 2024-09-10T09:38:26.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:26.932+0000 7f5247215740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f523c1b6f40 con 0x559e114fcac0 2024-09-10T09:38:26.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:26.932+0000 7f5247215740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f523c1b5ad0 con 0x559e11507ca0 2024-09-10T09:38:26.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:26.932+0000 7f5241400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559e114fd3e0 0x559e1151f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:26.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:26.932+0000 7f5241400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559e114fd3e0 0x559e1151f910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60878/0 (socket says 172.21.15.5:60878) 2024-09-10T09:38:26.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:26.932+0000 7f5241400640 1 -- 172.21.15.5:0/321753232 learned_addr learned my addr 172.21.15.5:0/321753232 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:38:26.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:26.932+0000 7f5242800640 1 --2- 172.21.15.5:0/321753232 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559e114fcac0 0x559e11503870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:26.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:26.932+0000 7f5242800640 1 --2- 172.21.15.5:0/321753232 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559e11507ca0 0x559e1151fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:26.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:26.932+0000 7f5241400640 1 -- 172.21.15.5:0/321753232 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559e11507ca0 msgr2=0x559e1151fe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:38:26.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:26.932+0000 7f5241400640 1 --2- 172.21.15.5:0/321753232 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559e11507ca0 0x559e1151fe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:26.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:26.932+0000 7f5241400640 1 -- 172.21.15.5:0/321753232 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559e114fcac0 msgr2=0x559e11503870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:38:26.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:26.932+0000 7f5241400640 1 --2- 172.21.15.5:0/321753232 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559e114fcac0 0x559e11503870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:26.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:26.932+0000 7f5241400640 1 -- 172.21.15.5:0/321753232 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559e1151a3e0 con 0x559e114fd3e0 2024-09-10T09:38:26.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:26.932+0000 7f5242800640 1 --2- 172.21.15.5:0/321753232 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559e11507ca0 0x559e1151fe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T09:38:26.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:26.933+0000 7f5241400640 1 --2- 172.21.15.5:0/321753232 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559e114fd3e0 0x559e1151f910 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f522c000c00 tx=0x7f522c005d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bda15c7663022dc server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:38:26.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:26.933+0000 7f522a600640 1 -- 172.21.15.5:0/321753232 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f522c035030 con 0x559e114fd3e0 2024-09-10T09:38:26.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:26.933+0000 7f522a600640 1 -- 172.21.15.5:0/321753232 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f522c041070 con 0x559e114fd3e0 2024-09-10T09:38:26.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:26.933+0000 7f522a600640 1 -- 172.21.15.5:0/321753232 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f522c031700 con 0x559e114fd3e0 2024-09-10T09:38:26.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:26.934+0000 7f5247215740 1 -- 172.21.15.5:0/321753232 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559e114fd3e0 msgr2=0x559e1151f910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:38:26.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:26.934+0000 7f5247215740 1 --2- 172.21.15.5:0/321753232 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559e114fd3e0 0x559e1151f910 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f522c000c00 tx=0x7f522c005d90 comp rx=0 tx=0).stop 2024-09-10T09:38:26.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:26.934+0000 7f5247215740 1 -- 172.21.15.5:0/321753232 shutdown_connections 2024-09-10T09:38:26.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:26.934+0000 7f5247215740 1 --2- 172.21.15.5:0/321753232 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559e11507ca0 0x559e1151fe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:26.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:26.934+0000 7f5247215740 1 --2- 172.21.15.5:0/321753232 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559e114fd3e0 0x559e1151f910 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:26.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:26.934+0000 7f5247215740 1 --2- 172.21.15.5:0/321753232 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559e114fcac0 0x559e11503870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:26.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:26.934+0000 7f5247215740 1 -- 172.21.15.5:0/321753232 >> 172.21.15.5:0/321753232 conn(0x559e114f9050 msgr2=0x559e11503160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:38:26.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:26.934+0000 7f5247215740 1 -- 172.21.15.5:0/321753232 shutdown_connections 2024-09-10T09:38:26.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:26.934+0000 7f5247215740 1 -- 172.21.15.5:0/321753232 wait complete. 2024-09-10T09:38:26.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:26.935+0000 7f5247215740 1 -- start start 2024-09-10T09:38:26.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:26.935+0000 7f5247215740 1 Processor -- start 2024-09-10T09:38:26.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:26.935+0000 7f5247215740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559e114fcac0 0x559e1150bc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:26.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:26.935+0000 7f5247215740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559e114fd3e0 0x559e1150c160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:26.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:26.935+0000 7f5247215740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559e11507ca0 0x559e1150c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:26.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:26.935+0000 7f5247215740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f523c1b6870 con 0x559e11507ca0 2024-09-10T09:38:26.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:26.935+0000 7f5247215740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f523c1b61a0 con 0x559e114fcac0 2024-09-10T09:38:26.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:26.935+0000 7f5247215740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f523c1b33e0 con 0x559e114fd3e0 2024-09-10T09:38:26.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:26.935+0000 7f5247215740 10 client.0 _ll_register_callbacks cb 0x7f522c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:38:26.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:26.936+0000 7f5242800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559e11507ca0 0x559e1150c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:26.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:26.936+0000 7f5242800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559e11507ca0 0x559e1150c6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60886/0 (socket says 172.21.15.5:60886) 2024-09-10T09:38:26.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:26.936+0000 7f5242800640 1 -- 172.21.15.5:0/17844583 learned_addr learned my addr 172.21.15.5:0/17844583 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:38:26.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:26.936+0000 7f5242800640 1 --2- 172.21.15.5:0/17844583 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559e114fcac0 0x559e1150bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:26.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:26.936+0000 7f5241400640 1 --2- 172.21.15.5:0/17844583 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559e114fd3e0 0x559e1150c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:26.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:26.936+0000 7f5242800640 1 -- 172.21.15.5:0/17844583 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559e114fd3e0 msgr2=0x559e1150c160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:38:26.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:26.936+0000 7f5242800640 1 --2- 172.21.15.5:0/17844583 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559e114fd3e0 0x559e1150c160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:26.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:26.936+0000 7f5242800640 1 -- 172.21.15.5:0/17844583 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559e114fcac0 msgr2=0x559e1150bc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:38:26.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:26.936+0000 7f5242800640 1 --2- 172.21.15.5:0/17844583 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559e114fcac0 0x559e1150bc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:26.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:26.936+0000 7f5242800640 1 -- 172.21.15.5:0/17844583 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559e11581a80 con 0x559e11507ca0 2024-09-10T09:38:26.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:26.936+0000 7f5241400640 1 --2- 172.21.15.5:0/17844583 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559e114fd3e0 0x559e1150c160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:38:26.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:26.937+0000 7f5242800640 1 --2- 172.21.15.5:0/17844583 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559e11507ca0 0x559e1150c6a0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f52380099a0 tx=0x7f523802a040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce91cdfc529124de server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:38:26.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:26.937+0000 7f521f400640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-10T09:38:26.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:26.937+0000 7f521f400640 1 -- 172.21.15.5:0/17844583 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f523801e040 con 0x559e11507ca0 2024-09-10T09:38:26.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:26.937+0000 7f5247215740 1 -- 172.21.15.5:0/17844583 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f522c041070 con 0x559e11507ca0 2024-09-10T09:38:26.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:26.937+0000 7f521f400640 1 -- 172.21.15.5:0/17844583 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f5238003fa0 con 0x559e11507ca0 2024-09-10T09:38:26.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:26.937+0000 7f521f400640 1 -- 172.21.15.5:0/17844583 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f52380041a0 con 0x559e11507ca0 2024-09-10T09:38:26.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:26.937+0000 7f5247215740 10 client.15294 fetch_fsmap learned FSMap version 30 2024-09-10T09:38:26.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:26.937+0000 7f5247215740 1 -- 172.21.15.5:0/17844583 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x559e11582d10 con 0x559e11507ca0 2024-09-10T09:38:26.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:26.937+0000 7f521f400640 1 -- 172.21.15.5:0/17844583 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f522c041070 con 0x559e11507ca0 2024-09-10T09:38:26.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:26.938+0000 7f521f400640 1 -- 172.21.15.5:0/17844583 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f5238019430 con 0x559e11507ca0 2024-09-10T09:38:26.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:26.938+0000 7f5247215740 10 client.15294 fetch_fsmap finished waiting for FSMap version 30 2024-09-10T09:38:26.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:26.938+0000 7f5247215740 10 client.15294 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T09:38:26.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:26.938+0000 7f5247215740 20 client.15294 populate_metadata read hostname 'smithi005' 2024-09-10T09:38:26.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:26.938+0000 7f5247215740 1 --2- 172.21.15.5:0/17844583 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x559e115831f0 0x559e115855e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:26.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:26.938+0000 7f5247215740 4 client.15294 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-10T09:38:26.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:26.938+0000 7f5247215740 1 -- 172.21.15.5:0/17844583 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x559e1146ad10 con 0x559e115831f0 2024-09-10T09:38:26.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:26.938+0000 7f5241400640 1 --2- 172.21.15.5:0/17844583 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x559e115831f0 0x559e115855e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:26.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:26.939+0000 7f5241400640 1 --2- 172.21.15.5:0/17844583 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x559e115831f0 0x559e115855e0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=76bbaaea5ab72952 server_cookie=16b4aefb85c5a024 in_seq=0 out_seq=0 2024-09-10T09:38:26.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:26.939+0000 7f521f400640 10 client.15294 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T09:38:26.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:26.940+0000 7f521f400640 1 -- 172.21.15.5:0/17844583 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x559e1146ad10 con 0x559e115831f0 2024-09-10T09:38:26.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:26.940+0000 7f521f400640 10 client.15294 handle_command_reply: tid=0 2024-09-10T09:38:26.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:26.940+0000 7f5247215740 1 client.15294 shutdown 2024-09-10T09:38:26.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:26.940+0000 7f5247215740 1 -- 172.21.15.5:0/17844583 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559e11507ca0 msgr2=0x559e1150c6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:38:26.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:26.940+0000 7f5247215740 1 --2- 172.21.15.5:0/17844583 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559e11507ca0 0x559e1150c6a0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f52380099a0 tx=0x7f523802a040 comp rx=0 tx=0).stop 2024-09-10T09:38:26.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:26.940+0000 7f5247215740 1 -- 172.21.15.5:0/17844583 shutdown_connections 2024-09-10T09:38:26.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:26.940+0000 7f5247215740 1 --2- 172.21.15.5:0/17844583 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x559e115831f0 0x559e115855e0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:26.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:26.940+0000 7f5247215740 1 --2- 172.21.15.5:0/17844583 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559e11507ca0 0x559e1150c6a0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:26.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:26.940+0000 7f5247215740 1 --2- 172.21.15.5:0/17844583 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559e114fd3e0 0x559e1150c160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:26.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:26.940+0000 7f5247215740 1 --2- 172.21.15.5:0/17844583 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559e114fcac0 0x559e1150bc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:26.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:26.940+0000 7f5247215740 1 -- 172.21.15.5:0/17844583 >> 172.21.15.5:0/17844583 conn(0x559e114f9050 msgr2=0x559e11503980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:38:26.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:26.940+0000 7f521f400640 0 client.15294 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T09:38:26.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:26.940+0000 7f5247215740 1 -- 172.21.15.5:0/17844583 shutdown_connections 2024-09-10T09:38:26.973 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-10T09:38:26.973 INFO:teuthology.orchestra.run.smithi005.stdout: "return_code": 0, 2024-09-10T09:38:26.973 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_tag": "a679c2e3-1405-4dbd-a91e-4744f2f0dd53", 2024-09-10T09:38:26.973 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": "asynchronous" 2024-09-10T09:38:26.973 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-10T09:38:26.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:26.940+0000 7f5247215740 1 -- 172.21.15.5:0/17844583 wait complete. 2024-09-10T09:38:26.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:26.941+0000 7f5247215740 20 client.15294 trim_cache size 0 max 16384 2024-09-10T09:38:26.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:26.941+0000 7f5244800640 1 -- 172.21.15.5:0/2577148275 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f52240a2c10 msgr2=0x7f52240a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:38:26.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:26.941+0000 7f5244800640 1 --2- 172.21.15.5:0/2577148275 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f52240a2c10 0x7f52240a50d0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f523000c6a0 tx=0x7f52300d34f0 comp rx=0 tx=0).stop 2024-09-10T09:38:26.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:26.941+0000 7f5244800640 1 -- 172.21.15.5:0/2577148275 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f523c076720 msgr2=0x7f523c1a4740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:38:26.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:26.941+0000 7f5244800640 1 --2- 172.21.15.5:0/2577148275 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f523c076720 0x7f523c1a4740 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f523000e9f0 tx=0x7f523000edf0 comp rx=0 tx=0).stop 2024-09-10T09:38:26.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:26.941+0000 7f5244800640 1 -- 172.21.15.5:0/2577148275 shutdown_connections 2024-09-10T09:38:26.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:26.941+0000 7f5244800640 1 --2- 172.21.15.5:0/2577148275 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f52240a2c10 0x7f52240a50d0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:26.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:26.941+0000 7f5244800640 1 --2- 172.21.15.5:0/2577148275 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f523c112a30 0x7f523c19e900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:26.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:26.941+0000 7f5244800640 1 --2- 172.21.15.5:0/2577148275 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f523c077040 0x7f523c1a4c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:26.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:26.941+0000 7f5244800640 1 --2- 172.21.15.5:0/2577148275 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f523c076720 0x7f523c1a4740 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:26.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:26.941+0000 7f5244800640 1 -- 172.21.15.5:0/2577148275 >> 172.21.15.5:0/2577148275 conn(0x7f523c0ffa90 msgr2=0x7f523c101e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:38:26.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:26.942+0000 7f5244800640 1 -- 172.21.15.5:0/2577148275 shutdown_connections 2024-09-10T09:38:26.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:26.942+0000 7f5244800640 1 -- 172.21.15.5:0/2577148275 wait complete. 2024-09-10T09:38:27.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:38:26 smithi005 ceph-mon[30350]: pgmap v396: 129 pgs: 129 active+clean; 653 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 1.2 KiB/s wr, 13 op/s 2024-09-10T09:38:27.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:38:26 smithi102 ceph-mon[35442]: pgmap v396: 129 pgs: 129 active+clean; 653 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 1.2 KiB/s wr, 13 op/s 2024-09-10T09:38:27.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:38:26 smithi159 ceph-mon[35513]: pgmap v396: 129 pgs: 129 active+clean; 653 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 1.2 KiB/s wr, 13 op/s 2024-09-10T09:38:27.771 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T09:38:27.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:26.984+0000 7f5656200640 1 -- 172.21.15.5:0/3063378229 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5650076720 msgr2=0x7f5650076b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:38:27.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:26.984+0000 7f5656200640 1 --2- 172.21.15.5:0/3063378229 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5650076720 0x7f5650076b00 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f5640009900 tx=0x7f564002a400 comp rx=0 tx=0).stop 2024-09-10T09:38:27.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:26.984+0000 7f5656200640 1 -- 172.21.15.5:0/3063378229 shutdown_connections 2024-09-10T09:38:27.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:26.984+0000 7f5656200640 1 --2- 172.21.15.5:0/3063378229 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5650112a50 0x7f5650114e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:27.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:26.984+0000 7f5656200640 1 --2- 172.21.15.5:0/3063378229 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5650077040 0x7f5650112510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:27.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:26.984+0000 7f5656200640 1 --2- 172.21.15.5:0/3063378229 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5650076720 0x7f5650076b00 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:27.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:26.984+0000 7f5656200640 1 -- 172.21.15.5:0/3063378229 >> 172.21.15.5:0/3063378229 conn(0x7f56500ffa50 msgr2=0x7f5650101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:38:27.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:26.984+0000 7f5656200640 1 -- 172.21.15.5:0/3063378229 shutdown_connections 2024-09-10T09:38:27.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:26.984+0000 7f5656200640 1 -- 172.21.15.5:0/3063378229 wait complete. 2024-09-10T09:38:27.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:26.985+0000 7f5656200640 1 Processor -- start 2024-09-10T09:38:27.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:26.985+0000 7f5656200640 1 -- start start 2024-09-10T09:38:27.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:26.986+0000 7f5656200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5650076720 0x7f56501a4760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:27.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:26.986+0000 7f5656200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5650077040 0x7f56501a4ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:27.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:26.986+0000 7f5656200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5650112a50 0x7f565019e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:27.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:26.986+0000 7f5656200640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f5650118610 con 0x7f5650112a50 2024-09-10T09:38:27.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:26.986+0000 7f5656200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f5650118490 con 0x7f5650077040 2024-09-10T09:38:27.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:26.986+0000 7f5656200640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f5650118790 con 0x7f5650076720 2024-09-10T09:38:27.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:26.986+0000 7f564ea00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5650077040 0x7f56501a4ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:27.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:26.986+0000 7f564f400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5650076720 0x7f56501a4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:27.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:26.986+0000 7f564ea00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5650077040 0x7f56501a4ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:57306/0 (socket says 172.21.15.5:57306) 2024-09-10T09:38:27.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:26.986+0000 7f564fe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5650112a50 0x7f565019e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:27.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:26.986+0000 7f564ea00640 1 -- 172.21.15.5:0/1964815959 learned_addr learned my addr 172.21.15.5:0/1964815959 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:38:27.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:26.986+0000 7f564ea00640 1 -- 172.21.15.5:0/1964815959 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5650076720 msgr2=0x7f56501a4760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:38:27.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:26.986+0000 7f564ea00640 1 --2- 172.21.15.5:0/1964815959 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5650076720 0x7f56501a4760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:27.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:26.986+0000 7f564ea00640 1 -- 172.21.15.5:0/1964815959 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5650112a50 msgr2=0x7f565019e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:38:27.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:26.986+0000 7f564ea00640 1 --2- 172.21.15.5:0/1964815959 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5650112a50 0x7f565019e920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:27.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:26.986+0000 7f564ea00640 1 -- 172.21.15.5:0/1964815959 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f565019f050 con 0x7f5650077040 2024-09-10T09:38:27.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:26.987+0000 7f564ea00640 1 --2- 172.21.15.5:0/1964815959 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5650077040 0x7f56501a4ca0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f563c00e9f0 tx=0x7f563c00edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:38:27.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:26.987+0000 7f5647e00640 1 -- 172.21.15.5:0/1964815959 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f563c00ccc0 con 0x7f5650077040 2024-09-10T09:38:27.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:26.987+0000 7f5656200640 1 -- 172.21.15.5:0/1964815959 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f565019f340 con 0x7f5650077040 2024-09-10T09:38:27.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:26.987+0000 7f5647e00640 1 -- 172.21.15.5:0/1964815959 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f563c010910 con 0x7f5650077040 2024-09-10T09:38:27.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:26.987+0000 7f5647e00640 1 -- 172.21.15.5:0/1964815959 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f563c0188a0 con 0x7f5650077040 2024-09-10T09:38:27.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:26.988+0000 7f5656200640 1 -- 172.21.15.5:0/1964815959 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56501ab630 con 0x7f5650077040 2024-09-10T09:38:27.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:26.989+0000 7f5656200640 1 -- 172.21.15.5:0/1964815959 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5650076b00 con 0x7f5650077040 2024-09-10T09:38:27.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:26.991+0000 7f5647e00640 1 -- 172.21.15.5:0/1964815959 <== mon.2 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 19) ==== 137883+0+0 (secure 0 0 0) 0x7f563c010ac0 con 0x7f5650077040 2024-09-10T09:38:27.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:26.993+0000 7f5647e00640 1 --2- 172.21.15.5:0/1964815959 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f56340a2c90 0x7f56340a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:27.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:26.993+0000 7f5647e00640 1 -- 172.21.15.5:0/1964815959 <== mon.2 v2:172.21.15.102:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f563c0cf9d0 con 0x7f5650077040 2024-09-10T09:38:27.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:26.993+0000 7f564f400640 1 --2- 172.21.15.5:0/1964815959 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f56340a2c90 0x7f56340a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:27.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:26.994+0000 7f564f400640 1 --2- 172.21.15.5:0/1964815959 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f56340a2c90 0x7f56340a5150 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f5640006010 tx=0x7f5640000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:38:27.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:26.997+0000 7f5647e00640 1 -- 172.21.15.5:0/1964815959 <== mon.2 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f563c09c380 con 0x7f5650077040 2024-09-10T09:38:27.890 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-10T09:38:27.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:27.179+0000 7f5656200640 1 -- 172.21.15.5:0/1964815959 --> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f56501146c0 con 0x7f56340a2c90 2024-09-10T09:38:27.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:27.187+0000 7f5647e00640 1 -- 172.21.15.5:0/1964815959 <== mgr.14150 v2:172.21.15.5:6800/2707348081 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340827 (secure 0 0 0) 0x7f56501146c0 con 0x7f56340a2c90 2024-09-10T09:38:27.895 INFO:teuthology.orchestra.run.smithi005.stderr:dumped all 2024-09-10T09:38:27.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:27.889+0000 7f5656200640 1 -- 172.21.15.5:0/1964815959 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f56340a2c90 msgr2=0x7f56340a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:38:27.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:27.889+0000 7f5656200640 1 --2- 172.21.15.5:0/1964815959 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f56340a2c90 0x7f56340a5150 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f5640006010 tx=0x7f5640000bc0 comp rx=0 tx=0).stop 2024-09-10T09:38:27.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:27.889+0000 7f5656200640 1 -- 172.21.15.5:0/1964815959 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5650077040 msgr2=0x7f56501a4ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:38:27.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:27.889+0000 7f5656200640 1 --2- 172.21.15.5:0/1964815959 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5650077040 0x7f56501a4ca0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f563c00e9f0 tx=0x7f563c00edf0 comp rx=0 tx=0).stop 2024-09-10T09:38:27.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:27.889+0000 7f5656200640 1 -- 172.21.15.5:0/1964815959 shutdown_connections 2024-09-10T09:38:27.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:27.889+0000 7f5656200640 1 --2- 172.21.15.5:0/1964815959 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f56340a2c90 0x7f56340a5150 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:27.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:27.889+0000 7f5656200640 1 --2- 172.21.15.5:0/1964815959 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5650112a50 0x7f565019e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:27.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:27.889+0000 7f5656200640 1 --2- 172.21.15.5:0/1964815959 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5650077040 0x7f56501a4ca0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:27.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:27.889+0000 7f5656200640 1 --2- 172.21.15.5:0/1964815959 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5650076720 0x7f56501a4760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:27.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:27.889+0000 7f5656200640 1 -- 172.21.15.5:0/1964815959 >> 172.21.15.5:0/1964815959 conn(0x7f56500ffa50 msgr2=0x7f5650101e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:38:27.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:27.890+0000 7f5656200640 1 -- 172.21.15.5:0/1964815959 shutdown_connections 2024-09-10T09:38:27.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:27.890+0000 7f5656200640 1 -- 172.21.15.5:0/1964815959 wait complete. 2024-09-10T09:38:28.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:38:27 smithi005 ceph-mon[30350]: scrub queued for path: / 2024-09-10T09:38:28.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:38:27 smithi005 ceph-mon[30350]: scrub summary: idle+waiting paths [/] 2024-09-10T09:38:28.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:38:27 smithi005 ceph-mon[30350]: scrub summary: active paths [/] 2024-09-10T09:38:28.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:38:27 smithi005 ceph-mon[30350]: scrub summary: idle+waiting paths [/] 2024-09-10T09:38:28.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:38:27 smithi005 ceph-mon[30350]: from='client.25316 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T09:38:28.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:38:27 smithi102 ceph-mon[35442]: scrub queued for path: / 2024-09-10T09:38:28.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:38:27 smithi102 ceph-mon[35442]: scrub summary: idle+waiting paths [/] 2024-09-10T09:38:28.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:38:27 smithi102 ceph-mon[35442]: scrub summary: active paths [/] 2024-09-10T09:38:28.135 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:38:27 smithi102 ceph-mon[35442]: scrub summary: idle+waiting paths [/] 2024-09-10T09:38:28.135 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:38:27 smithi102 ceph-mon[35442]: from='client.25316 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T09:38:28.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:38:27 smithi159 ceph-mon[35513]: scrub queued for path: / 2024-09-10T09:38:28.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:38:27 smithi159 ceph-mon[35513]: scrub summary: idle+waiting paths [/] 2024-09-10T09:38:28.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:38:27 smithi159 ceph-mon[35513]: scrub summary: active paths [/] 2024-09-10T09:38:28.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:38:27 smithi159 ceph-mon[35513]: scrub summary: idle+waiting paths [/] 2024-09-10T09:38:28.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:38:27 smithi159 ceph-mon[35513]: from='client.25316 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T09:38:28.699 INFO:teuthology.orchestra.run.smithi005.stdout:{"pg_ready":true,"pg_map":{"version":396,"stamp":"2024-09-10T09:38:25.572391+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":669121,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1459,"num_read_kb":867,"num_write":453,"num_write_kb":3104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":1181751,"num_keys_recovered":7,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":870,"ondisk_log_size":870,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":378988,"kb_used_data":13100,"kb_used_omap":18,"kb_used_meta":365677,"kb_avail":1124562836,"statfs":{"total":1151940427776,"available":1151552344064,"internally_reserved":0,"allocated":13414400,"data_stored":8627453,"data_compressed":498435,"data_compressed_allocated":675840,"data_compressed_original":2850816,"omap_allocated":19141,"internal_metadata":374453563},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":7849,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":149,"num_read_kb":85,"num_write":12,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003093"},"pg_stats":[{"pgid":"2.1f","version":"96'15","reported_seq":190,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:38:23.676913+0000","last_change":"2024-09-10T09:35:25.533085+0000","last_active":"2024-09-10T09:38:23.676913+0000","last_peered":"2024-09-10T09:38:23.676913+0000","last_clean":"2024-09-10T09:38:23.676913+0000","last_became_active":"2024-09-10T09:35:25.532729+0000","last_became_peered":"2024-09-10T09:35:25.532729+0000","last_unstale":"2024-09-10T09:38:23.676913+0000","last_undegraded":"2024-09-10T09:38:23.676913+0000","last_fullsized":"2024-09-10T09:38:23.676913+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:55:11.158279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":131,"num_read_kb":131,"num_write":16,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"96'5","reported_seq":21,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.545575+0000","last_change":"2024-09-10T09:36:06.767356+0000","last_active":"2024-09-10T09:37:35.545575+0000","last_peered":"2024-09-10T09:37:35.545575+0000","last_clean":"2024-09-10T09:37:35.545575+0000","last_became_active":"2024-09-10T09:36:06.766931+0000","last_became_peered":"2024-09-10T09:36:06.766931+0000","last_unstale":"2024-09-10T09:37:35.545575+0000","last_undegraded":"2024-09-10T09:37:35.545575+0000","last_fullsized":"2024-09-10T09:37:35.545575+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:25:02.777327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,7],"acting":[9,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.527523+0000","last_change":"2024-09-10T09:35:36.629797+0000","last_active":"2024-09-10T09:37:07.527523+0000","last_peered":"2024-09-10T09:37:07.527523+0000","last_clean":"2024-09-10T09:37:07.527523+0000","last_became_active":"2024-09-10T09:35:36.629414+0000","last_became_peered":"2024-09-10T09:35:36.629414+0000","last_unstale":"2024-09-10T09:37:07.527523+0000","last_undegraded":"2024-09-10T09:37:07.527523+0000","last_fullsized":"2024-09-10T09:37:07.527523+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:42:44.610581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"94'2","reported_seq":47,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.524964+0000","last_change":"2024-09-10T09:35:25.852022+0000","last_active":"2024-09-10T09:37:07.524964+0000","last_peered":"2024-09-10T09:37:07.524964+0000","last_clean":"2024-09-10T09:37:07.524964+0000","last_became_active":"2024-09-10T09:35:25.851587+0000","last_became_peered":"2024-09-10T09:35:25.851587+0000","last_unstale":"2024-09-10T09:37:07.524964+0000","last_undegraded":"2024-09-10T09:37:07.524964+0000","last_fullsized":"2024-09-10T09:37:07.524964+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:37:26.085914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"96'6","reported_seq":43,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.556261+0000","last_change":"2024-09-10T09:35:28.579260+0000","last_active":"2024-09-10T09:37:35.556261+0000","last_peered":"2024-09-10T09:37:35.556261+0000","last_clean":"2024-09-10T09:37:35.556261+0000","last_became_active":"2024-09-10T09:35:28.574584+0000","last_became_peered":"2024-09-10T09:35:28.574584+0000","last_unstale":"2024-09-10T09:37:35.556261+0000","last_undegraded":"2024-09-10T09:37:35.556261+0000","last_fullsized":"2024-09-10T09:37:35.556261+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:31:19.277689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.568359+0000","last_change":"2024-09-10T09:36:06.874861+0000","last_active":"2024-09-10T09:37:07.568359+0000","last_peered":"2024-09-10T09:37:07.568359+0000","last_clean":"2024-09-10T09:37:07.568359+0000","last_became_active":"2024-09-10T09:36:06.873930+0000","last_became_peered":"2024-09-10T09:36:06.873930+0000","last_unstale":"2024-09-10T09:37:07.568359+0000","last_undegraded":"2024-09-10T09:37:07.568359+0000","last_fullsized":"2024-09-10T09:37:07.568359+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:35:01.354061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,8,3],"acting":[4,6,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"96'13","reported_seq":58,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:59.271922+0000","last_change":"2024-09-10T09:35:25.534571+0000","last_active":"2024-09-10T09:37:59.271922+0000","last_peered":"2024-09-10T09:37:59.271922+0000","last_clean":"2024-09-10T09:37:59.271922+0000","last_became_active":"2024-09-10T09:35:25.534235+0000","last_became_peered":"2024-09-10T09:35:25.534235+0000","last_unstale":"2024-09-10T09:37:59.271922+0000","last_undegraded":"2024-09-10T09:37:59.271922+0000","last_fullsized":"2024-09-10T09:37:59.271922+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:50:21.447542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"96'9","reported_seq":48,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.553039+0000","last_change":"2024-09-10T09:35:28.565541+0000","last_active":"2024-09-10T09:37:35.553039+0000","last_peered":"2024-09-10T09:37:35.553039+0000","last_clean":"2024-09-10T09:37:35.553039+0000","last_became_active":"2024-09-10T09:35:28.565095+0000","last_became_peered":"2024-09-10T09:35:28.565095+0000","last_unstale":"2024-09-10T09:37:35.553039+0000","last_undegraded":"2024-09-10T09:37:35.553039+0000","last_fullsized":"2024-09-10T09:37:35.553039+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:29:31.285401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.568600+0000","last_change":"2024-09-10T09:35:36.629277+0000","last_active":"2024-09-10T09:37:07.568600+0000","last_peered":"2024-09-10T09:37:07.568600+0000","last_clean":"2024-09-10T09:37:07.568600+0000","last_became_active":"2024-09-10T09:35:36.628687+0000","last_became_peered":"2024-09-10T09:35:36.628687+0000","last_unstale":"2024-09-10T09:37:07.568600+0000","last_undegraded":"2024-09-10T09:37:07.568600+0000","last_fullsized":"2024-09-10T09:37:07.568600+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:34:33.628827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"94'5","reported_seq":50,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.530161+0000","last_change":"2024-09-10T09:35:25.536137+0000","last_active":"2024-09-10T09:37:07.530161+0000","last_peered":"2024-09-10T09:37:07.530161+0000","last_clean":"2024-09-10T09:37:07.530161+0000","last_became_active":"2024-09-10T09:35:25.535728+0000","last_became_peered":"2024-09-10T09:35:25.535728+0000","last_unstale":"2024-09-10T09:37:07.530161+0000","last_undegraded":"2024-09-10T09:37:07.530161+0000","last_fullsized":"2024-09-10T09:37:07.530161+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:36:04.829901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"96'15","reported_seq":60,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.547598+0000","last_change":"2024-09-10T09:36:06.874425+0000","last_active":"2024-09-10T09:37:35.547598+0000","last_peered":"2024-09-10T09:37:35.547598+0000","last_clean":"2024-09-10T09:37:35.547598+0000","last_became_active":"2024-09-10T09:36:06.874050+0000","last_became_peered":"2024-09-10T09:36:06.874050+0000","last_unstale":"2024-09-10T09:37:35.547598+0000","last_undegraded":"2024-09-10T09:37:35.547598+0000","last_fullsized":"2024-09-10T09:37:35.547598+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:30:51.200335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,1],"acting":[5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.568073+0000","last_change":"2024-09-10T09:35:36.615314+0000","last_active":"2024-09-10T09:37:07.568073+0000","last_peered":"2024-09-10T09:37:07.568073+0000","last_clean":"2024-09-10T09:37:07.568073+0000","last_became_active":"2024-09-10T09:35:36.614820+0000","last_became_peered":"2024-09-10T09:35:36.614820+0000","last_unstale":"2024-09-10T09:37:07.568073+0000","last_undegraded":"2024-09-10T09:37:07.568073+0000","last_fullsized":"2024-09-10T09:37:07.568073+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:51:16.897935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"96'5","reported_seq":42,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.556084+0000","last_change":"2024-09-10T09:35:28.572346+0000","last_active":"2024-09-10T09:37:35.556084+0000","last_peered":"2024-09-10T09:37:35.556084+0000","last_clean":"2024-09-10T09:37:35.556084+0000","last_became_active":"2024-09-10T09:35:28.571839+0000","last_became_peered":"2024-09-10T09:35:28.571839+0000","last_unstale":"2024-09-10T09:37:35.556084+0000","last_undegraded":"2024-09-10T09:37:35.556084+0000","last_fullsized":"2024-09-10T09:37:35.556084+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:41:29.662441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"94'4","reported_seq":49,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.529918+0000","last_change":"2024-09-10T09:35:25.538747+0000","last_active":"2024-09-10T09:37:07.529918+0000","last_peered":"2024-09-10T09:37:07.529918+0000","last_clean":"2024-09-10T09:37:07.529918+0000","last_became_active":"2024-09-10T09:35:25.538330+0000","last_became_peered":"2024-09-10T09:35:25.538330+0000","last_unstale":"2024-09-10T09:37:07.529918+0000","last_undegraded":"2024-09-10T09:37:07.529918+0000","last_fullsized":"2024-09-10T09:37:07.529918+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:15:08.340688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.568436+0000","last_change":"2024-09-10T09:35:37.270106+0000","last_active":"2024-09-10T09:37:07.568436+0000","last_peered":"2024-09-10T09:37:07.568436+0000","last_clean":"2024-09-10T09:37:07.568436+0000","last_became_active":"2024-09-10T09:35:37.269575+0000","last_became_peered":"2024-09-10T09:35:37.269575+0000","last_unstale":"2024-09-10T09:37:07.568436+0000","last_undegraded":"2024-09-10T09:37:07.568436+0000","last_fullsized":"2024-09-10T09:37:07.568436+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:35:00.414602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"96'8","reported_seq":47,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.555784+0000","last_change":"2024-09-10T09:35:28.569230+0000","last_active":"2024-09-10T09:37:35.555784+0000","last_peered":"2024-09-10T09:37:35.555784+0000","last_clean":"2024-09-10T09:37:35.555784+0000","last_became_active":"2024-09-10T09:35:28.568266+0000","last_became_peered":"2024-09-10T09:35:28.568266+0000","last_unstale":"2024-09-10T09:37:35.555784+0000","last_undegraded":"2024-09-10T09:37:35.555784+0000","last_fullsized":"2024-09-10T09:37:35.555784+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:49:43.730686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"94'3","reported_seq":17,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.544707+0000","last_change":"2024-09-10T09:37:07.544298+0000","last_active":"2024-09-10T09:37:07.544707+0000","last_peered":"2024-09-10T09:37:07.544707+0000","last_clean":"2024-09-10T09:37:07.544707+0000","last_became_active":"2024-09-10T09:37:07.531815+0000","last_became_peered":"2024-09-10T09:37:07.531815+0000","last_unstale":"2024-09-10T09:37:07.544707+0000","last_undegraded":"2024-09-10T09:37:07.544707+0000","last_fullsized":"2024-09-10T09:37:07.544707+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:18:10.091895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"94'1","reported_seq":37,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.568484+0000","last_change":"2024-09-10T09:36:06.875056+0000","last_active":"2024-09-10T09:37:07.568484+0000","last_peered":"2024-09-10T09:37:07.568484+0000","last_clean":"2024-09-10T09:37:07.568484+0000","last_became_active":"2024-09-10T09:36:06.874400+0000","last_became_peered":"2024-09-10T09:36:06.874400+0000","last_unstale":"2024-09-10T09:37:07.568484+0000","last_undegraded":"2024-09-10T09:37:07.568484+0000","last_fullsized":"2024-09-10T09:37:07.568484+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:21:06.300712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,7,8],"acting":[4,10,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"96'9","reported_seq":48,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.552565+0000","last_change":"2024-09-10T09:35:28.571319+0000","last_active":"2024-09-10T09:37:35.552565+0000","last_peered":"2024-09-10T09:37:35.552565+0000","last_clean":"2024-09-10T09:37:35.552565+0000","last_became_active":"2024-09-10T09:35:28.570972+0000","last_became_peered":"2024-09-10T09:35:28.570972+0000","last_unstale":"2024-09-10T09:37:35.552565+0000","last_undegraded":"2024-09-10T09:37:35.552565+0000","last_fullsized":"2024-09-10T09:37:35.552565+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:45:19.300535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"94'7","reported_seq":52,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.524483+0000","last_change":"2024-09-10T09:35:25.530489+0000","last_active":"2024-09-10T09:37:07.524483+0000","last_peered":"2024-09-10T09:37:07.524483+0000","last_clean":"2024-09-10T09:37:07.524483+0000","last_became_active":"2024-09-10T09:35:25.530197+0000","last_became_peered":"2024-09-10T09:35:25.530197+0000","last_unstale":"2024-09-10T09:37:07.524483+0000","last_undegraded":"2024-09-10T09:37:07.524483+0000","last_fullsized":"2024-09-10T09:37:07.524483+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:28:57.539569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.527650+0000","last_change":"2024-09-10T09:35:36.630507+0000","last_active":"2024-09-10T09:37:07.527650+0000","last_peered":"2024-09-10T09:37:07.527650+0000","last_clean":"2024-09-10T09:37:07.527650+0000","last_became_active":"2024-09-10T09:35:36.630052+0000","last_became_peered":"2024-09-10T09:35:36.630052+0000","last_unstale":"2024-09-10T09:37:07.527650+0000","last_undegraded":"2024-09-10T09:37:07.527650+0000","last_fullsized":"2024-09-10T09:37:07.527650+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:13:42.691382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"96'2","reported_seq":41,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.532487+0000","last_change":"2024-09-10T09:35:28.565650+0000","last_active":"2024-09-10T09:37:35.532487+0000","last_peered":"2024-09-10T09:37:35.532487+0000","last_clean":"2024-09-10T09:37:35.532487+0000","last_became_active":"2024-09-10T09:35:28.565303+0000","last_became_peered":"2024-09-10T09:35:28.565303+0000","last_unstale":"2024-09-10T09:37:35.532487+0000","last_undegraded":"2024-09-10T09:37:35.532487+0000","last_fullsized":"2024-09-10T09:37:35.532487+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:48:08.965487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"94'2","reported_seq":46,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.574507+0000","last_change":"2024-09-10T09:35:25.851326+0000","last_active":"2024-09-10T09:37:07.574507+0000","last_peered":"2024-09-10T09:37:07.574507+0000","last_clean":"2024-09-10T09:37:07.574507+0000","last_became_active":"2024-09-10T09:35:25.850981+0000","last_became_peered":"2024-09-10T09:35:25.850981+0000","last_unstale":"2024-09-10T09:37:07.574507+0000","last_undegraded":"2024-09-10T09:37:07.574507+0000","last_fullsized":"2024-09-10T09:37:07.574507+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:50:28.877840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.568510+0000","last_change":"2024-09-10T09:35:37.270138+0000","last_active":"2024-09-10T09:37:07.568510+0000","last_peered":"2024-09-10T09:37:07.568510+0000","last_clean":"2024-09-10T09:37:07.568510+0000","last_became_active":"2024-09-10T09:35:37.269663+0000","last_became_peered":"2024-09-10T09:35:37.269663+0000","last_unstale":"2024-09-10T09:37:07.568510+0000","last_undegraded":"2024-09-10T09:37:07.568510+0000","last_fullsized":"2024-09-10T09:37:07.568510+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:18:00.105970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"96'7","reported_seq":44,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.554144+0000","last_change":"2024-09-10T09:35:28.576197+0000","last_active":"2024-09-10T09:37:35.554144+0000","last_peered":"2024-09-10T09:37:35.554144+0000","last_clean":"2024-09-10T09:37:35.554144+0000","last_became_active":"2024-09-10T09:35:28.575859+0000","last_became_peered":"2024-09-10T09:35:28.575859+0000","last_unstale":"2024-09-10T09:37:35.554144+0000","last_undegraded":"2024-09-10T09:37:35.554144+0000","last_fullsized":"2024-09-10T09:37:35.554144+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:09:39.458761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"94'7","reported_seq":51,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.568805+0000","last_change":"2024-09-10T09:35:25.990587+0000","last_active":"2024-09-10T09:37:07.568805+0000","last_peered":"2024-09-10T09:37:07.568805+0000","last_clean":"2024-09-10T09:37:07.568805+0000","last_became_active":"2024-09-10T09:35:25.990302+0000","last_became_peered":"2024-09-10T09:35:25.990302+0000","last_unstale":"2024-09-10T09:37:07.568805+0000","last_undegraded":"2024-09-10T09:37:07.568805+0000","last_fullsized":"2024-09-10T09:37:07.568805+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:18:41.244441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.527768+0000","last_change":"2024-09-10T09:35:36.612983+0000","last_active":"2024-09-10T09:37:07.527768+0000","last_peered":"2024-09-10T09:37:07.527768+0000","last_clean":"2024-09-10T09:37:07.527768+0000","last_became_active":"2024-09-10T09:35:36.612542+0000","last_became_peered":"2024-09-10T09:35:36.612542+0000","last_unstale":"2024-09-10T09:37:07.527768+0000","last_undegraded":"2024-09-10T09:37:07.527768+0000","last_fullsized":"2024-09-10T09:37:07.527768+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:50:16.298195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"96'5","reported_seq":44,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.554267+0000","last_change":"2024-09-10T09:35:28.564121+0000","last_active":"2024-09-10T09:37:35.554267+0000","last_peered":"2024-09-10T09:37:35.554267+0000","last_clean":"2024-09-10T09:37:35.554267+0000","last_became_active":"2024-09-10T09:35:28.562780+0000","last_became_peered":"2024-09-10T09:35:28.562780+0000","last_unstale":"2024-09-10T09:37:35.554267+0000","last_undegraded":"2024-09-10T09:37:35.554267+0000","last_fullsized":"2024-09-10T09:37:35.554267+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:53:15.690478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"96'18","reported_seq":78,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:38:15.594145+0000","last_change":"2024-09-10T09:37:07.541031+0000","last_active":"2024-09-10T09:38:15.594145+0000","last_peered":"2024-09-10T09:38:15.594145+0000","last_clean":"2024-09-10T09:38:15.594145+0000","last_became_active":"2024-09-10T09:37:07.529577+0000","last_became_peered":"2024-09-10T09:37:07.529577+0000","last_unstale":"2024-09-10T09:38:15.594145+0000","last_undegraded":"2024-09-10T09:38:15.594145+0000","last_fullsized":"2024-09-10T09:38:15.594145+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:14:13.440346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":33,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":7,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.524145+0000","last_change":"2024-09-10T09:35:37.270888+0000","last_active":"2024-09-10T09:37:07.524145+0000","last_peered":"2024-09-10T09:37:07.524145+0000","last_clean":"2024-09-10T09:37:07.524145+0000","last_became_active":"2024-09-10T09:35:37.270229+0000","last_became_peered":"2024-09-10T09:35:37.270229+0000","last_unstale":"2024-09-10T09:37:07.524145+0000","last_undegraded":"2024-09-10T09:37:07.524145+0000","last_fullsized":"2024-09-10T09:37:07.524145+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:03:54.223463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"96'9","reported_seq":48,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.547904+0000","last_change":"2024-09-10T09:35:28.572799+0000","last_active":"2024-09-10T09:37:35.547904+0000","last_peered":"2024-09-10T09:37:35.547904+0000","last_clean":"2024-09-10T09:37:35.547904+0000","last_became_active":"2024-09-10T09:35:28.572310+0000","last_became_peered":"2024-09-10T09:35:28.572310+0000","last_unstale":"2024-09-10T09:37:35.547904+0000","last_undegraded":"2024-09-10T09:37:35.547904+0000","last_fullsized":"2024-09-10T09:37:35.547904+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:33:51.446766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"94'4","reported_seq":49,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.530456+0000","last_change":"2024-09-10T09:35:25.851868+0000","last_active":"2024-09-10T09:37:07.530456+0000","last_peered":"2024-09-10T09:37:07.530456+0000","last_clean":"2024-09-10T09:37:07.530456+0000","last_became_active":"2024-09-10T09:35:25.851447+0000","last_became_peered":"2024-09-10T09:35:25.851447+0000","last_unstale":"2024-09-10T09:37:07.530456+0000","last_undegraded":"2024-09-10T09:37:07.530456+0000","last_fullsized":"2024-09-10T09:37:07.530456+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:15:57.706690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.526545+0000","last_change":"2024-09-10T09:35:36.619459+0000","last_active":"2024-09-10T09:37:07.526545+0000","last_peered":"2024-09-10T09:37:07.526545+0000","last_clean":"2024-09-10T09:37:07.526545+0000","last_became_active":"2024-09-10T09:35:36.619023+0000","last_became_peered":"2024-09-10T09:35:36.619023+0000","last_unstale":"2024-09-10T09:37:07.526545+0000","last_undegraded":"2024-09-10T09:37:07.526545+0000","last_fullsized":"2024-09-10T09:37:07.526545+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:07:32.999231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"96'4","reported_seq":43,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.548010+0000","last_change":"2024-09-10T09:35:28.575668+0000","last_active":"2024-09-10T09:37:35.548010+0000","last_peered":"2024-09-10T09:37:35.548010+0000","last_clean":"2024-09-10T09:37:35.548010+0000","last_became_active":"2024-09-10T09:35:28.575425+0000","last_became_peered":"2024-09-10T09:35:28.575425+0000","last_unstale":"2024-09-10T09:37:35.548010+0000","last_undegraded":"2024-09-10T09:37:35.548010+0000","last_fullsized":"2024-09-10T09:37:35.548010+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:13:27.120208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"94'1","reported_seq":45,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.528018+0000","last_change":"2024-09-10T09:35:25.854569+0000","last_active":"2024-09-10T09:37:07.528018+0000","last_peered":"2024-09-10T09:37:07.528018+0000","last_clean":"2024-09-10T09:37:07.528018+0000","last_became_active":"2024-09-10T09:35:25.854152+0000","last_became_peered":"2024-09-10T09:35:25.854152+0000","last_unstale":"2024-09-10T09:37:07.528018+0000","last_undegraded":"2024-09-10T09:37:07.528018+0000","last_fullsized":"2024-09-10T09:37:07.528018+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:27:15.584707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.529957+0000","last_change":"2024-09-10T09:35:37.270517+0000","last_active":"2024-09-10T09:37:07.529957+0000","last_peered":"2024-09-10T09:37:07.529957+0000","last_clean":"2024-09-10T09:37:07.529957+0000","last_became_active":"2024-09-10T09:35:37.269793+0000","last_became_peered":"2024-09-10T09:35:37.269793+0000","last_unstale":"2024-09-10T09:37:07.529957+0000","last_undegraded":"2024-09-10T09:37:07.529957+0000","last_fullsized":"2024-09-10T09:37:07.529957+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:28:58.272848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"96'7","reported_seq":52,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.556473+0000","last_change":"2024-09-10T09:37:07.762585+0000","last_active":"2024-09-10T09:37:35.556473+0000","last_peered":"2024-09-10T09:37:35.556473+0000","last_clean":"2024-09-10T09:37:35.556473+0000","last_became_active":"2024-09-10T09:37:07.762250+0000","last_became_peered":"2024-09-10T09:37:07.762250+0000","last_unstale":"2024-09-10T09:37:35.556473+0000","last_undegraded":"2024-09-10T09:37:35.556473+0000","last_fullsized":"2024-09-10T09:37:35.556473+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:38:01.751512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"96'8","reported_seq":184,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:38:22.676195+0000","last_change":"2024-09-10T09:35:25.531062+0000","last_active":"2024-09-10T09:38:22.676195+0000","last_peered":"2024-09-10T09:38:22.676195+0000","last_clean":"2024-09-10T09:38:22.676195+0000","last_became_active":"2024-09-10T09:35:25.530005+0000","last_became_peered":"2024-09-10T09:35:25.530005+0000","last_unstale":"2024-09-10T09:38:22.676195+0000","last_undegraded":"2024-09-10T09:38:22.676195+0000","last_fullsized":"2024-09-10T09:38:22.676195+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:42:35.330312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1433,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":133,"num_read_kb":4,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.538766+0000","last_change":"2024-09-10T09:35:36.623099+0000","last_active":"2024-09-10T09:37:07.538766+0000","last_peered":"2024-09-10T09:37:07.538766+0000","last_clean":"2024-09-10T09:37:07.538766+0000","last_became_active":"2024-09-10T09:35:36.622716+0000","last_became_peered":"2024-09-10T09:35:36.622716+0000","last_unstale":"2024-09-10T09:37:07.538766+0000","last_undegraded":"2024-09-10T09:37:07.538766+0000","last_fullsized":"2024-09-10T09:37:07.538766+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:10:38.495164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"96'8","reported_seq":53,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.543805+0000","last_change":"2024-09-10T09:36:06.769405+0000","last_active":"2024-09-10T09:37:35.543805+0000","last_peered":"2024-09-10T09:37:35.543805+0000","last_clean":"2024-09-10T09:37:35.543805+0000","last_became_active":"2024-09-10T09:36:06.769042+0000","last_became_peered":"2024-09-10T09:36:06.769042+0000","last_unstale":"2024-09-10T09:37:35.543805+0000","last_undegraded":"2024-09-10T09:37:35.543805+0000","last_fullsized":"2024-09-10T09:37:35.543805+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:01:49.340765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,8],"acting":[5,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"94'5","reported_seq":179,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:38:21.673633+0000","last_change":"2024-09-10T09:35:25.851430+0000","last_active":"2024-09-10T09:38:21.673633+0000","last_peered":"2024-09-10T09:38:21.673633+0000","last_clean":"2024-09-10T09:38:21.673633+0000","last_became_active":"2024-09-10T09:35:25.851067+0000","last_became_peered":"2024-09-10T09:35:25.851067+0000","last_unstale":"2024-09-10T09:38:21.673633+0000","last_undegraded":"2024-09-10T09:38:21.673633+0000","last_fullsized":"2024-09-10T09:38:21.673633+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:55:58.756207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":130,"num_read_kb":130,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.538676+0000","last_change":"2024-09-10T09:35:37.270361+0000","last_active":"2024-09-10T09:37:07.538676+0000","last_peered":"2024-09-10T09:37:07.538676+0000","last_clean":"2024-09-10T09:37:07.538676+0000","last_became_active":"2024-09-10T09:35:37.269632+0000","last_became_peered":"2024-09-10T09:35:37.269632+0000","last_unstale":"2024-09-10T09:37:07.538676+0000","last_undegraded":"2024-09-10T09:37:07.538676+0000","last_fullsized":"2024-09-10T09:37:07.538676+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:50:34.360497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"96'5","reported_seq":44,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.548275+0000","last_change":"2024-09-10T09:35:28.571064+0000","last_active":"2024-09-10T09:37:35.548275+0000","last_peered":"2024-09-10T09:37:35.548275+0000","last_clean":"2024-09-10T09:37:35.548275+0000","last_became_active":"2024-09-10T09:35:28.570691+0000","last_became_peered":"2024-09-10T09:35:28.570691+0000","last_unstale":"2024-09-10T09:37:35.548275+0000","last_undegraded":"2024-09-10T09:37:35.548275+0000","last_fullsized":"2024-09-10T09:37:35.548275+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:58:33.162501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"94'4","reported_seq":57,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.572759+0000","last_change":"2024-09-10T09:37:07.572247+0000","last_active":"2024-09-10T09:37:07.572759+0000","last_peered":"2024-09-10T09:37:07.572759+0000","last_clean":"2024-09-10T09:37:07.572759+0000","last_became_active":"2024-09-10T09:37:07.569410+0000","last_became_peered":"2024-09-10T09:37:07.569410+0000","last_unstale":"2024-09-10T09:37:07.572759+0000","last_undegraded":"2024-09-10T09:37:07.572759+0000","last_fullsized":"2024-09-10T09:37:07.572759+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:05:09.037084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,4],"acting":[7,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.532723+0000","last_change":"2024-09-10T09:35:37.270358+0000","last_active":"2024-09-10T09:37:07.532723+0000","last_peered":"2024-09-10T09:37:07.532723+0000","last_clean":"2024-09-10T09:37:07.532723+0000","last_became_active":"2024-09-10T09:35:37.269822+0000","last_became_peered":"2024-09-10T09:35:37.269822+0000","last_unstale":"2024-09-10T09:37:07.532723+0000","last_undegraded":"2024-09-10T09:37:07.532723+0000","last_fullsized":"2024-09-10T09:37:07.532723+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:19:48.932596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"96'8","reported_seq":47,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.553803+0000","last_change":"2024-09-10T09:35:28.576244+0000","last_active":"2024-09-10T09:37:35.553803+0000","last_peered":"2024-09-10T09:37:35.553803+0000","last_clean":"2024-09-10T09:37:35.553803+0000","last_became_active":"2024-09-10T09:35:28.571801+0000","last_became_peered":"2024-09-10T09:35:28.571801+0000","last_unstale":"2024-09-10T09:37:35.553803+0000","last_undegraded":"2024-09-10T09:37:35.553803+0000","last_fullsized":"2024-09-10T09:37:35.553803+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:11:21.742220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"94'6","reported_seq":52,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.568754+0000","last_change":"2024-09-10T09:35:25.537118+0000","last_active":"2024-09-10T09:37:07.568754+0000","last_peered":"2024-09-10T09:37:07.568754+0000","last_clean":"2024-09-10T09:37:07.568754+0000","last_became_active":"2024-09-10T09:35:25.536718+0000","last_became_peered":"2024-09-10T09:35:25.536718+0000","last_unstale":"2024-09-10T09:37:07.568754+0000","last_undegraded":"2024-09-10T09:37:07.568754+0000","last_fullsized":"2024-09-10T09:37:07.568754+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:16:32.837476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":361,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.528720+0000","last_change":"2024-09-10T09:34:07.350825+0000","last_active":"2024-09-10T09:37:07.528720+0000","last_peered":"2024-09-10T09:37:07.528720+0000","last_clean":"2024-09-10T09:37:07.528720+0000","last_became_active":"2024-09-10T09:34:07.350325+0000","last_became_peered":"2024-09-10T09:34:07.350325+0000","last_unstale":"2024-09-10T09:37:07.528720+0000","last_undegraded":"2024-09-10T09:37:07.528720+0000","last_fullsized":"2024-09-10T09:37:07.528720+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:30:26.408262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:30:26.408262+0000","last_clean_scrub_stamp":"2024-09-10T09:30:26.408262+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:58:04.920443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"94'1","reported_seq":30,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.529308+0000","last_change":"2024-09-10T09:35:36.632941+0000","last_active":"2024-09-10T09:37:07.529308+0000","last_peered":"2024-09-10T09:37:07.529308+0000","last_clean":"2024-09-10T09:37:07.529308+0000","last_became_active":"2024-09-10T09:35:36.632527+0000","last_became_peered":"2024-09-10T09:35:36.632527+0000","last_unstale":"2024-09-10T09:37:07.529308+0000","last_undegraded":"2024-09-10T09:37:07.529308+0000","last_fullsized":"2024-09-10T09:37:07.529308+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:33:21.733772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"96'7","reported_seq":44,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.556036+0000","last_change":"2024-09-10T09:35:28.572010+0000","last_active":"2024-09-10T09:37:35.556036+0000","last_peered":"2024-09-10T09:37:35.556036+0000","last_clean":"2024-09-10T09:37:35.556036+0000","last_became_active":"2024-09-10T09:35:28.571623+0000","last_became_peered":"2024-09-10T09:35:28.571623+0000","last_unstale":"2024-09-10T09:37:35.556036+0000","last_undegraded":"2024-09-10T09:37:35.556036+0000","last_fullsized":"2024-09-10T09:37:35.556036+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:00:49.328663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"94'5","reported_seq":178,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:38:19.671689+0000","last_change":"2024-09-10T09:35:25.990328+0000","last_active":"2024-09-10T09:38:19.671689+0000","last_peered":"2024-09-10T09:38:19.671689+0000","last_clean":"2024-09-10T09:38:19.671689+0000","last_became_active":"2024-09-10T09:35:25.989946+0000","last_became_peered":"2024-09-10T09:35:25.989946+0000","last_unstale":"2024-09-10T09:38:19.671689+0000","last_undegraded":"2024-09-10T09:38:19.671689+0000","last_fullsized":"2024-09-10T09:38:19.671689+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:51:16.069913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":129,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.574499+0000","last_change":"2024-09-10T09:35:36.619013+0000","last_active":"2024-09-10T09:37:07.574499+0000","last_peered":"2024-09-10T09:37:07.574499+0000","last_clean":"2024-09-10T09:37:07.574499+0000","last_became_active":"2024-09-10T09:35:36.618506+0000","last_became_peered":"2024-09-10T09:35:36.618506+0000","last_unstale":"2024-09-10T09:37:07.574499+0000","last_undegraded":"2024-09-10T09:37:07.574499+0000","last_fullsized":"2024-09-10T09:37:07.574499+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:57:53.580569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"96'6","reported_seq":49,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.553955+0000","last_change":"2024-09-10T09:36:06.768573+0000","last_active":"2024-09-10T09:37:35.553955+0000","last_peered":"2024-09-10T09:37:35.553955+0000","last_clean":"2024-09-10T09:37:35.553955+0000","last_became_active":"2024-09-10T09:36:06.768228+0000","last_became_peered":"2024-09-10T09:36:06.768228+0000","last_unstale":"2024-09-10T09:37:35.553955+0000","last_undegraded":"2024-09-10T09:37:35.553955+0000","last_fullsized":"2024-09-10T09:37:35.553955+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:57:38.310713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"94'3","reported_seq":48,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.533010+0000","last_change":"2024-09-10T09:35:25.528795+0000","last_active":"2024-09-10T09:37:07.533010+0000","last_peered":"2024-09-10T09:37:07.533010+0000","last_clean":"2024-09-10T09:37:07.533010+0000","last_became_active":"2024-09-10T09:35:25.528475+0000","last_became_peered":"2024-09-10T09:35:25.528475+0000","last_unstale":"2024-09-10T09:37:07.533010+0000","last_undegraded":"2024-09-10T09:37:07.533010+0000","last_fullsized":"2024-09-10T09:37:07.533010+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:16:06.999186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.761758+0000","last_change":"2024-09-10T09:35:36.633109+0000","last_active":"2024-09-10T09:37:07.761758+0000","last_peered":"2024-09-10T09:37:07.761758+0000","last_clean":"2024-09-10T09:37:07.761758+0000","last_became_active":"2024-09-10T09:35:36.632682+0000","last_became_peered":"2024-09-10T09:35:36.632682+0000","last_unstale":"2024-09-10T09:37:07.761758+0000","last_undegraded":"2024-09-10T09:37:07.761758+0000","last_fullsized":"2024-09-10T09:37:07.761758+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:51:14.372649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"96'9","reported_seq":46,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.552249+0000","last_change":"2024-09-10T09:35:28.562103+0000","last_active":"2024-09-10T09:37:35.552249+0000","last_peered":"2024-09-10T09:37:35.552249+0000","last_clean":"2024-09-10T09:37:35.552249+0000","last_became_active":"2024-09-10T09:35:28.561765+0000","last_became_peered":"2024-09-10T09:35:28.561765+0000","last_unstale":"2024-09-10T09:37:35.552249+0000","last_undegraded":"2024-09-10T09:37:35.552249+0000","last_fullsized":"2024-09-10T09:37:35.552249+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:35:35.400569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"94'5","reported_seq":49,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.539243+0000","last_change":"2024-09-10T09:35:25.538536+0000","last_active":"2024-09-10T09:37:07.539243+0000","last_peered":"2024-09-10T09:37:07.539243+0000","last_clean":"2024-09-10T09:37:07.539243+0000","last_became_active":"2024-09-10T09:35:25.537944+0000","last_became_peered":"2024-09-10T09:35:25.537944+0000","last_unstale":"2024-09-10T09:37:07.539243+0000","last_undegraded":"2024-09-10T09:37:07.539243+0000","last_fullsized":"2024-09-10T09:37:07.539243+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:01:19.385381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":17,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.529928+0000","last_change":"2024-09-10T09:36:06.769534+0000","last_active":"2024-09-10T09:37:07.529928+0000","last_peered":"2024-09-10T09:37:07.529928+0000","last_clean":"2024-09-10T09:37:07.529928+0000","last_became_active":"2024-09-10T09:36:06.769015+0000","last_became_peered":"2024-09-10T09:36:06.769015+0000","last_unstale":"2024-09-10T09:37:07.529928+0000","last_undegraded":"2024-09-10T09:37:07.529928+0000","last_fullsized":"2024-09-10T09:37:07.529928+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:19:26.369737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7,1],"acting":[8,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"96'5","reported_seq":44,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.539962+0000","last_change":"2024-09-10T09:35:28.566735+0000","last_active":"2024-09-10T09:37:35.539962+0000","last_peered":"2024-09-10T09:37:35.539962+0000","last_clean":"2024-09-10T09:37:35.539962+0000","last_became_active":"2024-09-10T09:35:28.566400+0000","last_became_peered":"2024-09-10T09:35:28.566400+0000","last_unstale":"2024-09-10T09:37:35.539962+0000","last_undegraded":"2024-09-10T09:37:35.539962+0000","last_fullsized":"2024-09-10T09:37:35.539962+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:18:53.337816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"94'2","reported_seq":47,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.530569+0000","last_change":"2024-09-10T09:35:25.535783+0000","last_active":"2024-09-10T09:37:07.530569+0000","last_peered":"2024-09-10T09:37:07.530569+0000","last_clean":"2024-09-10T09:37:07.530569+0000","last_became_active":"2024-09-10T09:35:25.535411+0000","last_became_peered":"2024-09-10T09:35:25.535411+0000","last_unstale":"2024-09-10T09:37:07.530569+0000","last_undegraded":"2024-09-10T09:37:07.530569+0000","last_fullsized":"2024-09-10T09:37:07.530569+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:48:09.239083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.531208+0000","last_change":"2024-09-10T09:35:36.619380+0000","last_active":"2024-09-10T09:37:07.531208+0000","last_peered":"2024-09-10T09:37:07.531208+0000","last_clean":"2024-09-10T09:37:07.531208+0000","last_became_active":"2024-09-10T09:35:36.618953+0000","last_became_peered":"2024-09-10T09:35:36.618953+0000","last_unstale":"2024-09-10T09:37:07.531208+0000","last_undegraded":"2024-09-10T09:37:07.531208+0000","last_fullsized":"2024-09-10T09:37:07.531208+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:40:25.181408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"96'6","reported_seq":45,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.546928+0000","last_change":"2024-09-10T09:35:28.571396+0000","last_active":"2024-09-10T09:37:35.546928+0000","last_peered":"2024-09-10T09:37:35.546928+0000","last_clean":"2024-09-10T09:37:35.546928+0000","last_became_active":"2024-09-10T09:35:28.571063+0000","last_became_peered":"2024-09-10T09:35:28.571063+0000","last_unstale":"2024-09-10T09:37:35.546928+0000","last_undegraded":"2024-09-10T09:37:35.546928+0000","last_fullsized":"2024-09-10T09:37:35.546928+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:38:17.965133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"94'17","reported_seq":62,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.530060+0000","last_change":"2024-09-10T09:35:25.851773+0000","last_active":"2024-09-10T09:37:07.530060+0000","last_peered":"2024-09-10T09:37:07.530060+0000","last_clean":"2024-09-10T09:37:07.530060+0000","last_became_active":"2024-09-10T09:35:25.851384+0000","last_became_peered":"2024-09-10T09:35:25.851384+0000","last_unstale":"2024-09-10T09:37:07.530060+0000","last_undegraded":"2024-09-10T09:37:07.530060+0000","last_fullsized":"2024-09-10T09:37:07.530060+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:03:32.338643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.539186+0000","last_change":"2024-09-10T09:35:36.620971+0000","last_active":"2024-09-10T09:37:07.539186+0000","last_peered":"2024-09-10T09:37:07.539186+0000","last_clean":"2024-09-10T09:37:07.539186+0000","last_became_active":"2024-09-10T09:35:36.620343+0000","last_became_peered":"2024-09-10T09:35:36.620343+0000","last_unstale":"2024-09-10T09:37:07.539186+0000","last_undegraded":"2024-09-10T09:37:07.539186+0000","last_fullsized":"2024-09-10T09:37:07.539186+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:17:19.517882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"96'6","reported_seq":43,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.553818+0000","last_change":"2024-09-10T09:35:28.565643+0000","last_active":"2024-09-10T09:37:35.553818+0000","last_peered":"2024-09-10T09:37:35.553818+0000","last_clean":"2024-09-10T09:37:35.553818+0000","last_became_active":"2024-09-10T09:35:28.564217+0000","last_became_peered":"2024-09-10T09:35:28.564217+0000","last_unstale":"2024-09-10T09:37:35.553818+0000","last_undegraded":"2024-09-10T09:37:35.553818+0000","last_fullsized":"2024-09-10T09:37:35.553818+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:53:34.562690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"94'5","reported_seq":179,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:38:21.674381+0000","last_change":"2024-09-10T09:35:25.534167+0000","last_active":"2024-09-10T09:38:21.674381+0000","last_peered":"2024-09-10T09:38:21.674381+0000","last_clean":"2024-09-10T09:38:21.674381+0000","last_became_active":"2024-09-10T09:35:25.533719+0000","last_became_peered":"2024-09-10T09:35:25.533719+0000","last_unstale":"2024-09-10T09:38:21.674381+0000","last_undegraded":"2024-09-10T09:38:21.674381+0000","last_fullsized":"2024-09-10T09:38:21.674381+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:00:03.537355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":132,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.527733+0000","last_change":"2024-09-10T09:35:36.628716+0000","last_active":"2024-09-10T09:37:07.527733+0000","last_peered":"2024-09-10T09:37:07.527733+0000","last_clean":"2024-09-10T09:37:07.527733+0000","last_became_active":"2024-09-10T09:35:36.628085+0000","last_became_peered":"2024-09-10T09:35:36.628085+0000","last_unstale":"2024-09-10T09:37:07.527733+0000","last_undegraded":"2024-09-10T09:37:07.527733+0000","last_fullsized":"2024-09-10T09:37:07.527733+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:45:01.279403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"96'15","reported_seq":52,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.557076+0000","last_change":"2024-09-10T09:35:28.566701+0000","last_active":"2024-09-10T09:37:35.557076+0000","last_peered":"2024-09-10T09:37:35.557076+0000","last_clean":"2024-09-10T09:37:35.557076+0000","last_became_active":"2024-09-10T09:35:28.566391+0000","last_became_peered":"2024-09-10T09:35:28.566391+0000","last_unstale":"2024-09-10T09:37:35.557076+0000","last_undegraded":"2024-09-10T09:37:35.557076+0000","last_fullsized":"2024-09-10T09:37:35.557076+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:57:59.918706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"94'5","reported_seq":50,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.531896+0000","last_change":"2024-09-10T09:35:25.532510+0000","last_active":"2024-09-10T09:37:07.531896+0000","last_peered":"2024-09-10T09:37:07.531896+0000","last_clean":"2024-09-10T09:37:07.531896+0000","last_became_active":"2024-09-10T09:35:25.532193+0000","last_became_peered":"2024-09-10T09:35:25.532193+0000","last_unstale":"2024-09-10T09:37:07.531896+0000","last_undegraded":"2024-09-10T09:37:07.531896+0000","last_fullsized":"2024-09-10T09:37:07.531896+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:17:24.685178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.526424+0000","last_change":"2024-09-10T09:35:36.630742+0000","last_active":"2024-09-10T09:37:07.526424+0000","last_peered":"2024-09-10T09:37:07.526424+0000","last_clean":"2024-09-10T09:37:07.526424+0000","last_became_active":"2024-09-10T09:35:36.630335+0000","last_became_peered":"2024-09-10T09:35:36.630335+0000","last_unstale":"2024-09-10T09:37:07.526424+0000","last_undegraded":"2024-09-10T09:37:07.526424+0000","last_fullsized":"2024-09-10T09:37:07.526424+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:29:12.029390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"96'9","reported_seq":48,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.552455+0000","last_change":"2024-09-10T09:35:28.581817+0000","last_active":"2024-09-10T09:37:35.552455+0000","last_peered":"2024-09-10T09:37:35.552455+0000","last_clean":"2024-09-10T09:37:35.552455+0000","last_became_active":"2024-09-10T09:35:28.580316+0000","last_became_peered":"2024-09-10T09:35:28.580316+0000","last_unstale":"2024-09-10T09:37:35.552455+0000","last_undegraded":"2024-09-10T09:37:35.552455+0000","last_fullsized":"2024-09-10T09:37:35.552455+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:04:08.249564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"94'9","reported_seq":55,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.528127+0000","last_change":"2024-09-10T09:35:25.537474+0000","last_active":"2024-09-10T09:37:07.528127+0000","last_peered":"2024-09-10T09:37:07.528127+0000","last_clean":"2024-09-10T09:37:07.528127+0000","last_became_active":"2024-09-10T09:35:25.537096+0000","last_became_peered":"2024-09-10T09:35:25.537096+0000","last_unstale":"2024-09-10T09:37:07.528127+0000","last_undegraded":"2024-09-10T09:37:07.528127+0000","last_fullsized":"2024-09-10T09:37:07.528127+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:13:06.008458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.532316+0000","last_change":"2024-09-10T09:35:36.622788+0000","last_active":"2024-09-10T09:37:07.532316+0000","last_peered":"2024-09-10T09:37:07.532316+0000","last_clean":"2024-09-10T09:37:07.532316+0000","last_became_active":"2024-09-10T09:35:36.622377+0000","last_became_peered":"2024-09-10T09:35:36.622377+0000","last_unstale":"2024-09-10T09:37:07.532316+0000","last_undegraded":"2024-09-10T09:37:07.532316+0000","last_fullsized":"2024-09-10T09:37:07.532316+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:29:56.680864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"96'6","reported_seq":45,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.534371+0000","last_change":"2024-09-10T09:35:28.564529+0000","last_active":"2024-09-10T09:37:35.534371+0000","last_peered":"2024-09-10T09:37:35.534371+0000","last_clean":"2024-09-10T09:37:35.534371+0000","last_became_active":"2024-09-10T09:35:28.564223+0000","last_became_peered":"2024-09-10T09:35:28.564223+0000","last_unstale":"2024-09-10T09:37:35.534371+0000","last_undegraded":"2024-09-10T09:37:35.534371+0000","last_fullsized":"2024-09-10T09:37:35.534371+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:21:24.296578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"96'42","reported_seq":357,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:38:23.677498+0000","last_change":"2024-09-10T09:35:25.534331+0000","last_active":"2024-09-10T09:38:23.677498+0000","last_peered":"2024-09-10T09:38:23.677498+0000","last_clean":"2024-09-10T09:38:23.677498+0000","last_became_active":"2024-09-10T09:35:25.533985+0000","last_became_peered":"2024-09-10T09:35:25.533985+0000","last_unstale":"2024-09-10T09:38:23.677498+0000","last_undegraded":"2024-09-10T09:38:23.677498+0000","last_fullsized":"2024-09-10T09:38:23.677498+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":42,"log_dups_size":0,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:14:37.050650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70718,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":271,"num_read_kb":205,"num_write":44,"num_write_kb":89,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.527925+0000","last_change":"2024-09-10T09:35:36.632835+0000","last_active":"2024-09-10T09:37:07.527925+0000","last_peered":"2024-09-10T09:37:07.527925+0000","last_clean":"2024-09-10T09:37:07.527925+0000","last_became_active":"2024-09-10T09:35:36.630962+0000","last_became_peered":"2024-09-10T09:35:36.630962+0000","last_unstale":"2024-09-10T09:37:07.527925+0000","last_undegraded":"2024-09-10T09:37:07.527925+0000","last_fullsized":"2024-09-10T09:37:07.527925+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:59:27.613721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"96'11","reported_seq":50,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.546541+0000","last_change":"2024-09-10T09:35:28.567292+0000","last_active":"2024-09-10T09:37:35.546541+0000","last_peered":"2024-09-10T09:37:35.546541+0000","last_clean":"2024-09-10T09:37:35.546541+0000","last_became_active":"2024-09-10T09:35:28.566973+0000","last_became_peered":"2024-09-10T09:35:28.566973+0000","last_unstale":"2024-09-10T09:37:35.546541+0000","last_undegraded":"2024-09-10T09:37:35.546541+0000","last_fullsized":"2024-09-10T09:37:35.546541+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:05:37.251933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"94'5","reported_seq":50,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.532787+0000","last_change":"2024-09-10T09:35:25.852171+0000","last_active":"2024-09-10T09:37:07.532787+0000","last_peered":"2024-09-10T09:37:07.532787+0000","last_clean":"2024-09-10T09:37:07.532787+0000","last_became_active":"2024-09-10T09:35:25.851580+0000","last_became_peered":"2024-09-10T09:35:25.851580+0000","last_unstale":"2024-09-10T09:37:07.532787+0000","last_undegraded":"2024-09-10T09:37:07.532787+0000","last_fullsized":"2024-09-10T09:37:07.532787+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:03.263152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.568336+0000","last_change":"2024-09-10T09:35:36.629212+0000","last_active":"2024-09-10T09:37:07.568336+0000","last_peered":"2024-09-10T09:37:07.568336+0000","last_clean":"2024-09-10T09:37:07.568336+0000","last_became_active":"2024-09-10T09:35:36.628753+0000","last_became_peered":"2024-09-10T09:35:36.628753+0000","last_unstale":"2024-09-10T09:37:07.568336+0000","last_undegraded":"2024-09-10T09:37:07.568336+0000","last_fullsized":"2024-09-10T09:37:07.568336+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:54:45.311361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"96'9","reported_seq":46,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.553075+0000","last_change":"2024-09-10T09:35:28.572497+0000","last_active":"2024-09-10T09:37:35.553075+0000","last_peered":"2024-09-10T09:37:35.553075+0000","last_clean":"2024-09-10T09:37:35.553075+0000","last_became_active":"2024-09-10T09:35:28.569238+0000","last_became_peered":"2024-09-10T09:35:28.569238+0000","last_unstale":"2024-09-10T09:37:35.553075+0000","last_undegraded":"2024-09-10T09:37:35.553075+0000","last_fullsized":"2024-09-10T09:37:35.553075+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:38:47.189209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"94'4","reported_seq":47,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.529569+0000","last_change":"2024-09-10T09:35:25.536369+0000","last_active":"2024-09-10T09:37:07.529569+0000","last_peered":"2024-09-10T09:37:07.529569+0000","last_clean":"2024-09-10T09:37:07.529569+0000","last_became_active":"2024-09-10T09:35:25.535607+0000","last_became_peered":"2024-09-10T09:35:25.535607+0000","last_unstale":"2024-09-10T09:37:07.529569+0000","last_undegraded":"2024-09-10T09:37:07.529569+0000","last_fullsized":"2024-09-10T09:37:07.529569+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:49:18.621934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.527047+0000","last_change":"2024-09-10T09:35:36.627977+0000","last_active":"2024-09-10T09:37:07.527047+0000","last_peered":"2024-09-10T09:37:07.527047+0000","last_clean":"2024-09-10T09:37:07.527047+0000","last_became_active":"2024-09-10T09:35:36.626942+0000","last_became_peered":"2024-09-10T09:35:36.626942+0000","last_unstale":"2024-09-10T09:37:07.527047+0000","last_undegraded":"2024-09-10T09:37:07.527047+0000","last_fullsized":"2024-09-10T09:37:07.527047+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:37:18.209424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"96'10","reported_seq":49,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.553165+0000","last_change":"2024-09-10T09:35:28.562670+0000","last_active":"2024-09-10T09:37:35.553165+0000","last_peered":"2024-09-10T09:37:35.553165+0000","last_clean":"2024-09-10T09:37:35.553165+0000","last_became_active":"2024-09-10T09:35:28.562028+0000","last_became_peered":"2024-09-10T09:35:28.562028+0000","last_unstale":"2024-09-10T09:37:35.553165+0000","last_undegraded":"2024-09-10T09:37:35.553165+0000","last_fullsized":"2024-09-10T09:37:35.553165+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:37:15.176050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"96'5","reported_seq":308,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:38:23.677506+0000","last_change":"2024-09-10T09:35:25.529861+0000","last_active":"2024-09-10T09:38:23.677506+0000","last_peered":"2024-09-10T09:38:23.677506+0000","last_clean":"2024-09-10T09:38:23.677506+0000","last_became_active":"2024-09-10T09:35:25.529475+0000","last_became_peered":"2024-09-10T09:35:25.529475+0000","last_unstale":"2024-09-10T09:38:23.677506+0000","last_undegraded":"2024-09-10T09:38:23.677506+0000","last_fullsized":"2024-09-10T09:38:23.677506+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:44:27.086509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":259,"num_read_kb":259,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.524306+0000","last_change":"2024-09-10T09:35:36.615358+0000","last_active":"2024-09-10T09:37:07.524306+0000","last_peered":"2024-09-10T09:37:07.524306+0000","last_clean":"2024-09-10T09:37:07.524306+0000","last_became_active":"2024-09-10T09:35:36.614901+0000","last_became_peered":"2024-09-10T09:35:36.614901+0000","last_unstale":"2024-09-10T09:37:07.524306+0000","last_undegraded":"2024-09-10T09:37:07.524306+0000","last_fullsized":"2024-09-10T09:37:07.524306+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:56:54.248266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.761302+0000","last_change":"2024-09-10T09:35:36.629021+0000","last_active":"2024-09-10T09:37:07.761302+0000","last_peered":"2024-09-10T09:37:07.761302+0000","last_clean":"2024-09-10T09:37:07.761302+0000","last_became_active":"2024-09-10T09:35:36.628159+0000","last_became_peered":"2024-09-10T09:35:36.628159+0000","last_unstale":"2024-09-10T09:37:07.761302+0000","last_undegraded":"2024-09-10T09:37:07.761302+0000","last_fullsized":"2024-09-10T09:37:07.761302+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:50:48.615236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"96'11","reported_seq":48,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.545215+0000","last_change":"2024-09-10T09:35:28.579333+0000","last_active":"2024-09-10T09:37:35.545215+0000","last_peered":"2024-09-10T09:37:35.545215+0000","last_clean":"2024-09-10T09:37:35.545215+0000","last_became_active":"2024-09-10T09:35:28.578440+0000","last_became_peered":"2024-09-10T09:35:28.578440+0000","last_unstale":"2024-09-10T09:37:35.545215+0000","last_undegraded":"2024-09-10T09:37:35.545215+0000","last_fullsized":"2024-09-10T09:37:35.545215+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:07:28.701938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"94'3","reported_seq":48,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.531063+0000","last_change":"2024-09-10T09:35:25.530857+0000","last_active":"2024-09-10T09:37:07.531063+0000","last_peered":"2024-09-10T09:37:07.531063+0000","last_clean":"2024-09-10T09:37:07.531063+0000","last_became_active":"2024-09-10T09:35:25.530492+0000","last_became_peered":"2024-09-10T09:35:25.530492+0000","last_unstale":"2024-09-10T09:37:07.531063+0000","last_undegraded":"2024-09-10T09:37:07.531063+0000","last_fullsized":"2024-09-10T09:37:07.531063+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:07:21.726598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.526432+0000","last_change":"2024-09-10T09:36:06.770408+0000","last_active":"2024-09-10T09:37:07.526432+0000","last_peered":"2024-09-10T09:37:07.526432+0000","last_clean":"2024-09-10T09:37:07.526432+0000","last_became_active":"2024-09-10T09:36:06.769915+0000","last_became_peered":"2024-09-10T09:36:06.769915+0000","last_unstale":"2024-09-10T09:37:07.526432+0000","last_undegraded":"2024-09-10T09:37:07.526432+0000","last_fullsized":"2024-09-10T09:37:07.526432+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:56:09.795537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8,0],"acting":[11,4,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"96'8","reported_seq":47,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.543491+0000","last_change":"2024-09-10T09:35:28.570804+0000","last_active":"2024-09-10T09:37:35.543491+0000","last_peered":"2024-09-10T09:37:35.543491+0000","last_clean":"2024-09-10T09:37:35.543491+0000","last_became_active":"2024-09-10T09:35:28.570490+0000","last_became_peered":"2024-09-10T09:35:28.570490+0000","last_unstale":"2024-09-10T09:37:35.543491+0000","last_undegraded":"2024-09-10T09:37:35.543491+0000","last_fullsized":"2024-09-10T09:37:35.543491+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:13:38.999625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"94'3","reported_seq":47,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.538446+0000","last_change":"2024-09-10T09:35:25.536347+0000","last_active":"2024-09-10T09:37:07.538446+0000","last_peered":"2024-09-10T09:37:07.538446+0000","last_clean":"2024-09-10T09:37:07.538446+0000","last_became_active":"2024-09-10T09:35:25.535964+0000","last_became_peered":"2024-09-10T09:35:25.535964+0000","last_unstale":"2024-09-10T09:37:07.538446+0000","last_undegraded":"2024-09-10T09:37:07.538446+0000","last_fullsized":"2024-09-10T09:37:07.538446+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:36:49.035666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"94'7","reported_seq":51,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.525812+0000","last_change":"2024-09-10T09:35:25.532148+0000","last_active":"2024-09-10T09:37:07.525812+0000","last_peered":"2024-09-10T09:37:07.525812+0000","last_clean":"2024-09-10T09:37:07.525812+0000","last_became_active":"2024-09-10T09:35:25.531781+0000","last_became_peered":"2024-09-10T09:35:25.531781+0000","last_unstale":"2024-09-10T09:37:07.525812+0000","last_undegraded":"2024-09-10T09:37:07.525812+0000","last_fullsized":"2024-09-10T09:37:07.525812+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:52:46.740342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"96'8","reported_seq":47,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.549412+0000","last_change":"2024-09-10T09:35:28.578336+0000","last_active":"2024-09-10T09:37:35.549412+0000","last_peered":"2024-09-10T09:37:35.549412+0000","last_clean":"2024-09-10T09:37:35.549412+0000","last_became_active":"2024-09-10T09:35:28.577899+0000","last_became_peered":"2024-09-10T09:35:28.577899+0000","last_unstale":"2024-09-10T09:37:35.549412+0000","last_undegraded":"2024-09-10T09:37:35.549412+0000","last_fullsized":"2024-09-10T09:37:35.549412+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:13:04.478195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.528122+0000","last_change":"2024-09-10T09:35:36.630918+0000","last_active":"2024-09-10T09:37:07.528122+0000","last_peered":"2024-09-10T09:37:07.528122+0000","last_clean":"2024-09-10T09:37:07.528122+0000","last_became_active":"2024-09-10T09:35:36.630308+0000","last_became_peered":"2024-09-10T09:35:36.630308+0000","last_unstale":"2024-09-10T09:37:07.528122+0000","last_undegraded":"2024-09-10T09:37:07.528122+0000","last_fullsized":"2024-09-10T09:37:07.528122+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:25:44.954970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"96'7","reported_seq":196,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:38:22.676633+0000","last_change":"2024-09-10T09:37:07.574386+0000","last_active":"2024-09-10T09:38:22.676633+0000","last_peered":"2024-09-10T09:38:22.676633+0000","last_clean":"2024-09-10T09:38:22.676633+0000","last_became_active":"2024-09-10T09:37:07.569370+0000","last_became_peered":"2024-09-10T09:37:07.569370+0000","last_unstale":"2024-09-10T09:38:22.676633+0000","last_undegraded":"2024-09-10T09:38:22.676633+0000","last_fullsized":"2024-09-10T09:38:22.676633+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:07:23.014789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1156,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":128,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":1015,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"96'7","reported_seq":46,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.543648+0000","last_change":"2024-09-10T09:35:28.567915+0000","last_active":"2024-09-10T09:37:35.543648+0000","last_peered":"2024-09-10T09:37:35.543648+0000","last_clean":"2024-09-10T09:37:35.543648+0000","last_became_active":"2024-09-10T09:35:28.567416+0000","last_became_peered":"2024-09-10T09:35:28.567416+0000","last_unstale":"2024-09-10T09:37:35.543648+0000","last_undegraded":"2024-09-10T09:37:35.543648+0000","last_fullsized":"2024-09-10T09:37:35.543648+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:07:25.704080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.528686+0000","last_change":"2024-09-10T09:35:36.615386+0000","last_active":"2024-09-10T09:37:07.528686+0000","last_peered":"2024-09-10T09:37:07.528686+0000","last_clean":"2024-09-10T09:37:07.528686+0000","last_became_active":"2024-09-10T09:35:36.614967+0000","last_became_peered":"2024-09-10T09:35:36.614967+0000","last_unstale":"2024-09-10T09:37:07.528686+0000","last_undegraded":"2024-09-10T09:37:07.528686+0000","last_fullsized":"2024-09-10T09:37:07.528686+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:47.620729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"96'9","reported_seq":48,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.544481+0000","last_change":"2024-09-10T09:35:28.564411+0000","last_active":"2024-09-10T09:37:35.544481+0000","last_peered":"2024-09-10T09:37:35.544481+0000","last_clean":"2024-09-10T09:37:35.544481+0000","last_became_active":"2024-09-10T09:35:28.564034+0000","last_became_peered":"2024-09-10T09:35:28.564034+0000","last_unstale":"2024-09-10T09:37:35.544481+0000","last_undegraded":"2024-09-10T09:37:35.544481+0000","last_fullsized":"2024-09-10T09:37:35.544481+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:38:17.687991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"96'12","reported_seq":49,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.547277+0000","last_change":"2024-09-10T09:35:28.564619+0000","last_active":"2024-09-10T09:37:35.547277+0000","last_peered":"2024-09-10T09:37:35.547277+0000","last_clean":"2024-09-10T09:37:35.547277+0000","last_became_active":"2024-09-10T09:35:28.563890+0000","last_became_peered":"2024-09-10T09:35:28.563890+0000","last_unstale":"2024-09-10T09:37:35.547277+0000","last_undegraded":"2024-09-10T09:37:35.547277+0000","last_fullsized":"2024-09-10T09:37:35.547277+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:51:42.481911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"96'8","reported_seq":45,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.543410+0000","last_change":"2024-09-10T09:35:28.558909+0000","last_active":"2024-09-10T09:37:35.543410+0000","last_peered":"2024-09-10T09:37:35.543410+0000","last_clean":"2024-09-10T09:37:35.543410+0000","last_became_active":"2024-09-10T09:35:28.558604+0000","last_became_peered":"2024-09-10T09:35:28.558604+0000","last_unstale":"2024-09-10T09:37:35.543410+0000","last_undegraded":"2024-09-10T09:37:35.543410+0000","last_fullsized":"2024-09-10T09:37:35.543410+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:11.928446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"96'7","reported_seq":44,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.554844+0000","last_change":"2024-09-10T09:35:28.574459+0000","last_active":"2024-09-10T09:37:35.554844+0000","last_peered":"2024-09-10T09:37:35.554844+0000","last_clean":"2024-09-10T09:37:35.554844+0000","last_became_active":"2024-09-10T09:35:28.574154+0000","last_became_peered":"2024-09-10T09:35:28.574154+0000","last_unstale":"2024-09-10T09:37:35.554844+0000","last_undegraded":"2024-09-10T09:37:35.554844+0000","last_fullsized":"2024-09-10T09:37:35.554844+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:59:09.650536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"96'4","reported_seq":43,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.552571+0000","last_change":"2024-09-10T09:35:28.576999+0000","last_active":"2024-09-10T09:37:35.552571+0000","last_peered":"2024-09-10T09:37:35.552571+0000","last_clean":"2024-09-10T09:37:35.552571+0000","last_became_active":"2024-09-10T09:35:28.575616+0000","last_became_peered":"2024-09-10T09:35:28.575616+0000","last_unstale":"2024-09-10T09:37:35.552571+0000","last_undegraded":"2024-09-10T09:37:35.552571+0000","last_fullsized":"2024-09-10T09:37:35.552571+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:18:08.771760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"96'6","reported_seq":45,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.541496+0000","last_change":"2024-09-10T09:35:28.564846+0000","last_active":"2024-09-10T09:37:35.541496+0000","last_peered":"2024-09-10T09:37:35.541496+0000","last_clean":"2024-09-10T09:37:35.541496+0000","last_became_active":"2024-09-10T09:35:28.564498+0000","last_became_peered":"2024-09-10T09:35:28.564498+0000","last_unstale":"2024-09-10T09:37:35.541496+0000","last_undegraded":"2024-09-10T09:37:35.541496+0000","last_fullsized":"2024-09-10T09:37:35.541496+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:33:43.041730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"96'15","reported_seq":52,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.550975+0000","last_change":"2024-09-10T09:35:28.558093+0000","last_active":"2024-09-10T09:37:35.550975+0000","last_peered":"2024-09-10T09:37:35.550975+0000","last_clean":"2024-09-10T09:37:35.550975+0000","last_became_active":"2024-09-10T09:35:28.557266+0000","last_became_peered":"2024-09-10T09:35:28.557266+0000","last_unstale":"2024-09-10T09:37:35.550975+0000","last_undegraded":"2024-09-10T09:37:35.550975+0000","last_fullsized":"2024-09-10T09:37:35.550975+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:51:30.946065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"96'2","reported_seq":47,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.549558+0000","last_change":"2024-09-10T09:36:06.770298+0000","last_active":"2024-09-10T09:37:35.549558+0000","last_peered":"2024-09-10T09:37:35.549558+0000","last_clean":"2024-09-10T09:37:35.549558+0000","last_became_active":"2024-09-10T09:36:06.769848+0000","last_became_peered":"2024-09-10T09:36:06.769848+0000","last_unstale":"2024-09-10T09:37:35.549558+0000","last_undegraded":"2024-09-10T09:37:35.549558+0000","last_fullsized":"2024-09-10T09:37:35.549558+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:00:30.470887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,8,2],"acting":[3,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"96'6","reported_seq":45,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.549553+0000","last_change":"2024-09-10T09:35:28.578158+0000","last_active":"2024-09-10T09:37:35.549553+0000","last_peered":"2024-09-10T09:37:35.549553+0000","last_clean":"2024-09-10T09:37:35.549553+0000","last_became_active":"2024-09-10T09:35:28.577824+0000","last_became_peered":"2024-09-10T09:35:28.577824+0000","last_unstale":"2024-09-10T09:37:35.549553+0000","last_undegraded":"2024-09-10T09:37:35.549553+0000","last_fullsized":"2024-09-10T09:37:35.549553+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:40:27.231164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"96'8","reported_seq":47,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.550143+0000","last_change":"2024-09-10T09:35:28.562670+0000","last_active":"2024-09-10T09:37:35.550143+0000","last_peered":"2024-09-10T09:37:35.550143+0000","last_clean":"2024-09-10T09:37:35.550143+0000","last_became_active":"2024-09-10T09:35:28.562311+0000","last_became_peered":"2024-09-10T09:35:28.562311+0000","last_unstale":"2024-09-10T09:37:35.550143+0000","last_undegraded":"2024-09-10T09:37:35.550143+0000","last_fullsized":"2024-09-10T09:37:35.550143+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:07:11.716142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"96'10","reported_seq":49,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.556330+0000","last_change":"2024-09-10T09:35:28.561124+0000","last_active":"2024-09-10T09:37:35.556330+0000","last_peered":"2024-09-10T09:37:35.556330+0000","last_clean":"2024-09-10T09:37:35.556330+0000","last_became_active":"2024-09-10T09:35:28.560777+0000","last_became_peered":"2024-09-10T09:35:28.560777+0000","last_unstale":"2024-09-10T09:37:35.556330+0000","last_undegraded":"2024-09-10T09:37:35.556330+0000","last_fullsized":"2024-09-10T09:37:35.556330+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:47:04.837596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"96'11","reported_seq":50,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.556539+0000","last_change":"2024-09-10T09:35:28.576790+0000","last_active":"2024-09-10T09:37:35.556539+0000","last_peered":"2024-09-10T09:37:35.556539+0000","last_clean":"2024-09-10T09:37:35.556539+0000","last_became_active":"2024-09-10T09:35:28.576464+0000","last_became_peered":"2024-09-10T09:35:28.576464+0000","last_unstale":"2024-09-10T09:37:35.556539+0000","last_undegraded":"2024-09-10T09:37:35.556539+0000","last_fullsized":"2024-09-10T09:37:35.556539+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:40:53.517577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"96'8","reported_seq":47,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.539417+0000","last_change":"2024-09-10T09:35:28.569935+0000","last_active":"2024-09-10T09:37:35.539417+0000","last_peered":"2024-09-10T09:37:35.539417+0000","last_clean":"2024-09-10T09:37:35.539417+0000","last_became_active":"2024-09-10T09:35:28.569539+0000","last_became_peered":"2024-09-10T09:35:28.569539+0000","last_unstale":"2024-09-10T09:37:35.539417+0000","last_undegraded":"2024-09-10T09:37:35.539417+0000","last_fullsized":"2024-09-10T09:37:35.539417+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:40:45.016269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"96'7","reported_seq":44,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.550145+0000","last_change":"2024-09-10T09:35:28.571619+0000","last_active":"2024-09-10T09:37:35.550145+0000","last_peered":"2024-09-10T09:37:35.550145+0000","last_clean":"2024-09-10T09:37:35.550145+0000","last_became_active":"2024-09-10T09:35:28.568591+0000","last_became_peered":"2024-09-10T09:35:28.568591+0000","last_unstale":"2024-09-10T09:37:35.550145+0000","last_undegraded":"2024-09-10T09:37:35.550145+0000","last_fullsized":"2024-09-10T09:37:35.550145+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:07:49.959712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"96'10","reported_seq":47,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.551796+0000","last_change":"2024-09-10T09:35:28.573336+0000","last_active":"2024-09-10T09:37:35.551796+0000","last_peered":"2024-09-10T09:37:35.551796+0000","last_clean":"2024-09-10T09:37:35.551796+0000","last_became_active":"2024-09-10T09:35:28.572991+0000","last_became_peered":"2024-09-10T09:35:28.572991+0000","last_unstale":"2024-09-10T09:37:35.551796+0000","last_undegraded":"2024-09-10T09:37:35.551796+0000","last_fullsized":"2024-09-10T09:37:35.551796+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:30:02.905869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"96'8","reported_seq":47,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.552930+0000","last_change":"2024-09-10T09:35:28.571945+0000","last_active":"2024-09-10T09:37:35.552930+0000","last_peered":"2024-09-10T09:37:35.552930+0000","last_clean":"2024-09-10T09:37:35.552930+0000","last_became_active":"2024-09-10T09:35:28.571546+0000","last_became_peered":"2024-09-10T09:35:28.571546+0000","last_unstale":"2024-09-10T09:37:35.552930+0000","last_undegraded":"2024-09-10T09:37:35.552930+0000","last_fullsized":"2024-09-10T09:37:35.552930+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:36:47.946864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"96'9","reported_seq":48,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.556725+0000","last_change":"2024-09-10T09:35:28.562712+0000","last_active":"2024-09-10T09:37:35.556725+0000","last_peered":"2024-09-10T09:37:35.556725+0000","last_clean":"2024-09-10T09:37:35.556725+0000","last_became_active":"2024-09-10T09:35:28.562396+0000","last_became_peered":"2024-09-10T09:35:28.562396+0000","last_unstale":"2024-09-10T09:37:35.556725+0000","last_undegraded":"2024-09-10T09:37:35.556725+0000","last_fullsized":"2024-09-10T09:37:35.556725+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:25:47.419162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"96'7","reported_seq":44,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.553427+0000","last_change":"2024-09-10T09:35:28.566354+0000","last_active":"2024-09-10T09:37:35.553427+0000","last_peered":"2024-09-10T09:37:35.553427+0000","last_clean":"2024-09-10T09:37:35.553427+0000","last_became_active":"2024-09-10T09:35:28.565248+0000","last_became_peered":"2024-09-10T09:35:28.565248+0000","last_unstale":"2024-09-10T09:37:35.553427+0000","last_undegraded":"2024-09-10T09:37:35.553427+0000","last_fullsized":"2024-09-10T09:37:35.553427+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:48:46.741010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"96'7","reported_seq":44,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.550936+0000","last_change":"2024-09-10T09:35:28.563331+0000","last_active":"2024-09-10T09:37:35.550936+0000","last_peered":"2024-09-10T09:37:35.550936+0000","last_clean":"2024-09-10T09:37:35.550936+0000","last_became_active":"2024-09-10T09:35:28.563020+0000","last_became_peered":"2024-09-10T09:35:28.563020+0000","last_unstale":"2024-09-10T09:37:35.550936+0000","last_undegraded":"2024-09-10T09:37:35.550936+0000","last_fullsized":"2024-09-10T09:37:35.550936+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:27.653551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"96'10","reported_seq":49,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.542747+0000","last_change":"2024-09-10T09:35:28.566687+0000","last_active":"2024-09-10T09:37:35.542747+0000","last_peered":"2024-09-10T09:37:35.542747+0000","last_clean":"2024-09-10T09:37:35.542747+0000","last_became_active":"2024-09-10T09:35:28.566253+0000","last_became_peered":"2024-09-10T09:35:28.566253+0000","last_unstale":"2024-09-10T09:37:35.542747+0000","last_undegraded":"2024-09-10T09:37:35.542747+0000","last_fullsized":"2024-09-10T09:37:35.542747+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:00:15.709725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"96'7","reported_seq":44,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.538923+0000","last_change":"2024-09-10T09:35:28.562251+0000","last_active":"2024-09-10T09:37:35.538923+0000","last_peered":"2024-09-10T09:37:35.538923+0000","last_clean":"2024-09-10T09:37:35.538923+0000","last_became_active":"2024-09-10T09:35:28.561933+0000","last_became_peered":"2024-09-10T09:35:28.561933+0000","last_unstale":"2024-09-10T09:37:35.538923+0000","last_undegraded":"2024-09-10T09:37:35.538923+0000","last_fullsized":"2024-09-10T09:37:35.538923+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:30:41.447954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"96'6","reported_seq":43,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.556126+0000","last_change":"2024-09-10T09:35:28.572164+0000","last_active":"2024-09-10T09:37:35.556126+0000","last_peered":"2024-09-10T09:37:35.556126+0000","last_clean":"2024-09-10T09:37:35.556126+0000","last_became_active":"2024-09-10T09:35:28.571825+0000","last_became_peered":"2024-09-10T09:35:28.571825+0000","last_unstale":"2024-09-10T09:37:35.556126+0000","last_undegraded":"2024-09-10T09:37:35.556126+0000","last_fullsized":"2024-09-10T09:37:35.556126+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:59:50.967269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"96'16","reported_seq":55,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.550138+0000","last_change":"2024-09-10T09:35:28.568098+0000","last_active":"2024-09-10T09:37:35.550138+0000","last_peered":"2024-09-10T09:37:35.550138+0000","last_clean":"2024-09-10T09:37:35.550138+0000","last_became_active":"2024-09-10T09:35:28.567765+0000","last_became_peered":"2024-09-10T09:35:28.567765+0000","last_unstale":"2024-09-10T09:37:35.550138+0000","last_undegraded":"2024-09-10T09:37:35.550138+0000","last_fullsized":"2024-09-10T09:37:35.550138+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:55:57.260320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"96'11","reported_seq":50,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.553598+0000","last_change":"2024-09-10T09:35:28.564123+0000","last_active":"2024-09-10T09:37:35.553598+0000","last_peered":"2024-09-10T09:37:35.553598+0000","last_clean":"2024-09-10T09:37:35.553598+0000","last_became_active":"2024-09-10T09:35:28.563814+0000","last_became_peered":"2024-09-10T09:35:28.563814+0000","last_unstale":"2024-09-10T09:37:35.553598+0000","last_undegraded":"2024-09-10T09:37:35.553598+0000","last_fullsized":"2024-09-10T09:37:35.553598+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:14:17.912087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"96'9","reported_seq":48,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.549407+0000","last_change":"2024-09-10T09:35:28.568601+0000","last_active":"2024-09-10T09:37:35.549407+0000","last_peered":"2024-09-10T09:37:35.549407+0000","last_clean":"2024-09-10T09:37:35.549407+0000","last_became_active":"2024-09-10T09:35:28.568303+0000","last_became_peered":"2024-09-10T09:35:28.568303+0000","last_unstale":"2024-09-10T09:37:35.549407+0000","last_undegraded":"2024-09-10T09:37:35.549407+0000","last_fullsized":"2024-09-10T09:37:35.549407+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:30:01.016420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"96'5","reported_seq":48,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.556713+0000","last_change":"2024-09-10T09:37:07.539577+0000","last_active":"2024-09-10T09:37:35.556713+0000","last_peered":"2024-09-10T09:37:35.556713+0000","last_clean":"2024-09-10T09:37:35.556713+0000","last_became_active":"2024-09-10T09:37:07.539206+0000","last_became_peered":"2024-09-10T09:37:07.539206+0000","last_unstale":"2024-09-10T09:37:35.556713+0000","last_undegraded":"2024-09-10T09:37:35.556713+0000","last_fullsized":"2024-09-10T09:37:35.556713+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:56:06.568226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"96'6","reported_seq":45,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.541193+0000","last_change":"2024-09-10T09:35:28.569225+0000","last_active":"2024-09-10T09:37:35.541193+0000","last_peered":"2024-09-10T09:37:35.541193+0000","last_clean":"2024-09-10T09:37:35.541193+0000","last_became_active":"2024-09-10T09:35:28.568891+0000","last_became_peered":"2024-09-10T09:35:28.568891+0000","last_unstale":"2024-09-10T09:37:35.541193+0000","last_undegraded":"2024-09-10T09:37:35.541193+0000","last_fullsized":"2024-09-10T09:37:35.541193+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:33:45.811781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"96'4","reported_seq":43,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.544182+0000","last_change":"2024-09-10T09:35:28.578362+0000","last_active":"2024-09-10T09:37:35.544182+0000","last_peered":"2024-09-10T09:37:35.544182+0000","last_clean":"2024-09-10T09:37:35.544182+0000","last_became_active":"2024-09-10T09:35:28.577867+0000","last_became_peered":"2024-09-10T09:35:28.577867+0000","last_unstale":"2024-09-10T09:37:35.544182+0000","last_undegraded":"2024-09-10T09:37:35.544182+0000","last_fullsized":"2024-09-10T09:37:35.544182+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:58:17.041663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"96'8","reported_seq":45,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.537018+0000","last_change":"2024-09-10T09:35:28.568183+0000","last_active":"2024-09-10T09:37:35.537018+0000","last_peered":"2024-09-10T09:37:35.537018+0000","last_clean":"2024-09-10T09:37:35.537018+0000","last_became_active":"2024-09-10T09:35:28.566775+0000","last_became_peered":"2024-09-10T09:35:28.566775+0000","last_unstale":"2024-09-10T09:37:35.537018+0000","last_undegraded":"2024-09-10T09:37:35.537018+0000","last_fullsized":"2024-09-10T09:37:35.537018+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:28:10.008889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"96'7","reported_seq":44,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.553233+0000","last_change":"2024-09-10T09:35:28.576122+0000","last_active":"2024-09-10T09:37:35.553233+0000","last_peered":"2024-09-10T09:37:35.553233+0000","last_clean":"2024-09-10T09:37:35.553233+0000","last_became_active":"2024-09-10T09:35:28.575831+0000","last_became_peered":"2024-09-10T09:35:28.575831+0000","last_unstale":"2024-09-10T09:37:35.553233+0000","last_undegraded":"2024-09-10T09:37:35.553233+0000","last_fullsized":"2024-09-10T09:37:35.553233+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:37:51.604658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"96'4","reported_seq":41,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.554017+0000","last_change":"2024-09-10T09:35:28.564441+0000","last_active":"2024-09-10T09:37:35.554017+0000","last_peered":"2024-09-10T09:37:35.554017+0000","last_clean":"2024-09-10T09:37:35.554017+0000","last_became_active":"2024-09-10T09:35:28.564095+0000","last_became_peered":"2024-09-10T09:35:28.564095+0000","last_unstale":"2024-09-10T09:37:35.554017+0000","last_undegraded":"2024-09-10T09:37:35.554017+0000","last_fullsized":"2024-09-10T09:37:35.554017+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:53:28.275361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"96'8","reported_seq":47,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.553155+0000","last_change":"2024-09-10T09:35:28.581921+0000","last_active":"2024-09-10T09:37:35.553155+0000","last_peered":"2024-09-10T09:37:35.553155+0000","last_clean":"2024-09-10T09:37:35.553155+0000","last_became_active":"2024-09-10T09:35:28.581573+0000","last_became_peered":"2024-09-10T09:35:28.581573+0000","last_unstale":"2024-09-10T09:37:35.553155+0000","last_undegraded":"2024-09-10T09:37:35.553155+0000","last_fullsized":"2024-09-10T09:37:35.553155+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:58:38.983934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":78519,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1333,"num_read_kb":758,"num_write":270,"num_write_kb":164,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":1015,"num_keys_recovered":7,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":602112,"data_stored":235557,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":231,"ondisk_log_size":231,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":285627,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":77,"seq":330712481841,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32228,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712924,"statfs":{"total":95995035648,"available":95962034176,"internally_reserved":0,"allocated":1069056,"data_stored":554290,"data_compressed":32456,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":70,"seq":300647710774,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28228,"kb_used_data":1204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716924,"statfs":{"total":95995035648,"available":95966130176,"internally_reserved":0,"allocated":1232896,"data_stored":1215920,"data_compressed":68777,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":65,"seq":279172874301,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32252,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712900,"statfs":{"total":95995035648,"available":95962009600,"internally_reserved":0,"allocated":1093632,"data_stored":554220,"data_compressed":32456,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1599,"internal_metadata":31914433},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":57,"seq":244813135937,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32248,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712904,"statfs":{"total":95995035648,"available":95962013696,"internally_reserved":0,"allocated":1089536,"data_stored":554598,"data_compressed":32456,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":50,"seq":214748364871,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28140,"kb_used_data":1116,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717012,"statfs":{"total":95995035648,"available":95966220288,"internally_reserved":0,"allocated":1142784,"data_stored":1152083,"data_compressed":68777,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":45,"seq":193273528396,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36388,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708764,"statfs":{"total":95995035648,"available":95957774336,"internally_reserved":0,"allocated":1069056,"data_stored":550569,"data_compressed":32456,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790035,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28148,"kb_used_data":1124,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717004,"statfs":{"total":95995035648,"available":95966212096,"internally_reserved":0,"allocated":1150976,"data_stored":1146820,"data_compressed":68777,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953560,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32244,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712908,"statfs":{"total":95995035648,"available":95962017792,"internally_reserved":0,"allocated":1085440,"data_stored":554954,"data_compressed":32456,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182492,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28080,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717072,"statfs":{"total":95995035648,"available":95966281728,"internally_reserved":0,"allocated":1081344,"data_stored":549432,"data_compressed":32456,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411425,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28156,"kb_used_data":1132,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716996,"statfs":{"total":95995035648,"available":95966203904,"internally_reserved":0,"allocated":1159168,"data_stored":621070,"data_compressed":32456,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574952,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36464,"kb_used_data":1120,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708688,"statfs":{"total":95995035648,"available":95957696512,"internally_reserved":0,"allocated":1146880,"data_stored":622317,"data_compressed":32456,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738476,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36412,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708740,"statfs":{"total":95995035648,"available":95957749760,"internally_reserved":0,"allocated":1093632,"data_stored":551180,"data_compressed":32456,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34674,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34674,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34674,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36321,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36321,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36321,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36321,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36321,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":73315,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":72068,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3354,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1567,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4521,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":126976,"data_stored":72454,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T09:38:28.751 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-10T09:38:28.752 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:38:29.022 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:38:28 smithi005 ceph-mon[30350]: pgmap v397: 129 pgs: 129 active+clean; 653 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-09-10T09:38:29.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:38:28 smithi102 ceph-mon[35442]: pgmap v397: 129 pgs: 129 active+clean; 653 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-09-10T09:38:29.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:29.285+0000 7f1909200640 1 -- 172.21.15.5:0/3787664952 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f190410e260 msgr2=0x7f1904110650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:38:29.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:29.285+0000 7f1909200640 1 --2- 172.21.15.5:0/3787664952 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f190410e260 0x7f1904110650 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f18fc009ca0 tx=0x7f18fc02a950 comp rx=0 tx=0).stop 2024-09-10T09:38:29.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:29.285+0000 7f1909200640 1 -- 172.21.15.5:0/3787664952 shutdown_connections 2024-09-10T09:38:29.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:29.285+0000 7f1909200640 1 --2- 172.21.15.5:0/3787664952 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f190410e260 0x7f1904110650 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:29.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:29.285+0000 7f1909200640 1 --2- 172.21.15.5:0/3787664952 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1904074ee0 0x7f190410dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:29.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:29.285+0000 7f1909200640 1 --2- 172.21.15.5:0/3787664952 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f19040745c0 0x7f19040749a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:29.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:29.285+0000 7f1909200640 1 -- 172.21.15.5:0/3787664952 >> 172.21.15.5:0/3787664952 conn(0x7f19040fd740 msgr2=0x7f19040ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:38:29.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:29.285+0000 7f1909200640 1 -- 172.21.15.5:0/3787664952 shutdown_connections 2024-09-10T09:38:29.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:29.285+0000 7f1909200640 1 -- 172.21.15.5:0/3787664952 wait complete. 2024-09-10T09:38:29.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:29.287+0000 7f1909200640 1 Processor -- start 2024-09-10T09:38:29.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:29.287+0000 7f1909200640 1 -- start start 2024-09-10T09:38:29.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:29.287+0000 7f1909200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f19040745c0 0x7f190410db90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:29.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:29.287+0000 7f1909200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1904074ee0 0x7f190410ac40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:29.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:29.287+0000 7f1909200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f190410e260 0x7f190410b180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:29.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:29.287+0000 7f1909200640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1904114000 con 0x7f190410e260 2024-09-10T09:38:29.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:29.287+0000 7f1909200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f1904113e80 con 0x7f19040745c0 2024-09-10T09:38:29.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:29.287+0000 7f1909200640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f1904114180 con 0x7f1904074ee0 2024-09-10T09:38:29.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:29.288+0000 7f1903400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f19040745c0 0x7f190410db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:29.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:29.288+0000 7f1903400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f19040745c0 0x7f190410db90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:57334/0 (socket says 172.21.15.5:57334) 2024-09-10T09:38:29.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:29.288+0000 7f1903400640 1 -- 172.21.15.5:0/190514507 learned_addr learned my addr 172.21.15.5:0/190514507 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:38:29.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:29.288+0000 7f1902a00640 1 --2- 172.21.15.5:0/190514507 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1904074ee0 0x7f190410ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:29.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:29.288+0000 7f1903e00640 1 --2- 172.21.15.5:0/190514507 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f190410e260 0x7f190410b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:29.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:29.288+0000 7f1903400640 1 -- 172.21.15.5:0/190514507 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1904074ee0 msgr2=0x7f190410ac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:38:29.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:29.288+0000 7f1903400640 1 --2- 172.21.15.5:0/190514507 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1904074ee0 0x7f190410ac40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:29.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:29.288+0000 7f1903400640 1 -- 172.21.15.5:0/190514507 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f190410e260 msgr2=0x7f190410b180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:38:29.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:29.288+0000 7f1903400640 1 --2- 172.21.15.5:0/190514507 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f190410e260 0x7f190410b180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:29.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:29.288+0000 7f1903400640 1 -- 172.21.15.5:0/190514507 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f190410ba10 con 0x7f19040745c0 2024-09-10T09:38:29.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:29.289+0000 7f1903400640 1 --2- 172.21.15.5:0/190514507 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f19040745c0 0x7f190410db90 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f18f400e9f0 tx=0x7f18f400edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:38:29.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:29.289+0000 7f18fbe00640 1 -- 172.21.15.5:0/190514507 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18f400cd80 con 0x7f19040745c0 2024-09-10T09:38:29.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:29.289+0000 7f18fbe00640 1 -- 172.21.15.5:0/190514507 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f18f4005500 con 0x7f19040745c0 2024-09-10T09:38:29.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:29.289+0000 7f18fbe00640 1 -- 172.21.15.5:0/190514507 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18f4010690 con 0x7f19040745c0 2024-09-10T09:38:29.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:29.289+0000 7f1909200640 1 -- 172.21.15.5:0/190514507 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19041aade0 con 0x7f19040745c0 2024-09-10T09:38:29.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:29.289+0000 7f1909200640 1 -- 172.21.15.5:0/190514507 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19041ab1e0 con 0x7f19040745c0 2024-09-10T09:38:29.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:29.292+0000 7f18fbe00640 1 -- 172.21.15.5:0/190514507 <== mon.2 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 19) ==== 137883+0+0 (secure 0 0 0) 0x7f18f40046c0 con 0x7f19040745c0 2024-09-10T09:38:29.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:29.292+0000 7f190d20a740 1 Processor -- start 2024-09-10T09:38:29.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:29.293+0000 7f190d20a740 1 -- start start 2024-09-10T09:38:29.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:29.293+0000 7f190d20a740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563d23b80d50 0x563d23b81130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:29.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:29.293+0000 7f190d20a740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563d23b81670 0x563d23b8b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:29.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:29.293+0000 7f190d20a740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563d23b8bf30 0x563d23b8e320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:29.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:29.293+0000 7f190d20a740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x563d23ae73e0 con 0x563d23b81670 2024-09-10T09:38:29.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:29.293+0000 7f190d20a740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x563d23a8ed80 con 0x563d23b80d50 2024-09-10T09:38:29.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:29.293+0000 7f190d20a740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x563d23ae33a0 con 0x563d23b8bf30 2024-09-10T09:38:29.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:29.294+0000 7f1903400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563d23b81670 0x563d23b8b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:29.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:29.294+0000 7f1903400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563d23b81670 0x563d23b8b9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60928/0 (socket says 172.21.15.5:60928) 2024-09-10T09:38:29.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:29.294+0000 7f1903400640 1 -- 172.21.15.5:0/3653053316 learned_addr learned my addr 172.21.15.5:0/3653053316 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:38:29.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:29.294+0000 7f1903e00640 1 --2- 172.21.15.5:0/3653053316 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563d23b80d50 0x563d23b81130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:29.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:29.294+0000 7f1902a00640 1 --2- 172.21.15.5:0/3653053316 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563d23b8bf30 0x563d23b8e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:29.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:29.294+0000 7f1903400640 1 -- 172.21.15.5:0/3653053316 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563d23b8bf30 msgr2=0x563d23b8e320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:38:29.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:29.294+0000 7f1903400640 1 --2- 172.21.15.5:0/3653053316 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563d23b8bf30 0x563d23b8e320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:29.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:29.294+0000 7f1903400640 1 -- 172.21.15.5:0/3653053316 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563d23b80d50 msgr2=0x563d23b81130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:38:29.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:29.294+0000 7f1903400640 1 --2- 172.21.15.5:0/3653053316 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563d23b80d50 0x563d23b81130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:29.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:29.294+0000 7f18fbe00640 1 --2- 172.21.15.5:0/190514507 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f18e80a2c10 0x7f18e80a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:29.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:29.294+0000 7f1903400640 1 -- 172.21.15.5:0/3653053316 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563d23aadae0 con 0x563d23b81670 2024-09-10T09:38:29.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:29.294+0000 7f1902a00640 1 --2- 172.21.15.5:0/3653053316 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563d23b8bf30 0x563d23b8e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:38:29.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:29.294+0000 7f1903e00640 1 --2- 172.21.15.5:0/3653053316 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563d23b80d50 0x563d23b81130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:38:29.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:29.295+0000 7f18fbe00640 1 -- 172.21.15.5:0/190514507 <== mon.2 v2:172.21.15.102:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f18f40cfa20 con 0x7f19040745c0 2024-09-10T09:38:29.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:29.295+0000 7f1902a00640 1 --2- 172.21.15.5:0/190514507 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f18e80a2c10 0x7f18e80a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:29.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:29.295+0000 7f1903400640 1 --2- 172.21.15.5:0/3653053316 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563d23b81670 0x563d23b8b9f0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f18f40abce0 tx=0x7f18f4021000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d754b67d17e2247f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:38:29.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:29.295+0000 7f18f8c00640 1 -- 172.21.15.5:0/3653053316 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18f409baa0 con 0x563d23b81670 2024-09-10T09:38:29.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:29.295+0000 7f18f8c00640 1 -- 172.21.15.5:0/3653053316 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f18f40abd10 con 0x563d23b81670 2024-09-10T09:38:29.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:29.295+0000 7f18f8c00640 1 -- 172.21.15.5:0/3653053316 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18f409b630 con 0x563d23b81670 2024-09-10T09:38:29.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:29.296+0000 7f190d20a740 1 -- 172.21.15.5:0/3653053316 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563d23b81670 msgr2=0x563d23b8b9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:38:29.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:29.296+0000 7f1902a00640 1 --2- 172.21.15.5:0/190514507 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f18e80a2c10 0x7f18e80a50d0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x563d23b7e660 tx=0x7f18f0005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:38:29.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:29.296+0000 7f190d20a740 1 --2- 172.21.15.5:0/3653053316 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563d23b81670 0x563d23b8b9f0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f18f40abce0 tx=0x7f18f4021000 comp rx=0 tx=0).stop 2024-09-10T09:38:29.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:29.297+0000 7f190d20a740 1 -- 172.21.15.5:0/3653053316 shutdown_connections 2024-09-10T09:38:29.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:29.297+0000 7f190d20a740 1 --2- 172.21.15.5:0/3653053316 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563d23b8bf30 0x563d23b8e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:29.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:29.297+0000 7f190d20a740 1 --2- 172.21.15.5:0/3653053316 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563d23b81670 0x563d23b8b9f0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:29.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:29.297+0000 7f190d20a740 1 --2- 172.21.15.5:0/3653053316 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563d23b80d50 0x563d23b81130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:29.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:29.297+0000 7f190d20a740 1 -- 172.21.15.5:0/3653053316 >> 172.21.15.5:0/3653053316 conn(0x563d23b7d2e0 msgr2=0x563d23b7d6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:38:29.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:29.297+0000 7f190d20a740 1 -- 172.21.15.5:0/3653053316 shutdown_connections 2024-09-10T09:38:29.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:29.297+0000 7f190d20a740 1 -- 172.21.15.5:0/3653053316 wait complete. 2024-09-10T09:38:29.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:29.297+0000 7f190d20a740 1 -- start start 2024-09-10T09:38:29.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:29.298+0000 7f190d20a740 1 Processor -- start 2024-09-10T09:38:29.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:29.298+0000 7f190d20a740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563d23b80d50 0x563d23b89080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:29.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:29.298+0000 7f190d20a740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563d23b8bf30 0x563d23b895c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:29.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:29.298+0000 7f190d20a740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563d23b89b00 0x563d23b98010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:29.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:29.298+0000 7f190d20a740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x563d23b8eba0 con 0x563d23b80d50 2024-09-10T09:38:29.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:29.298+0000 7f190d20a740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x563d23aa20f0 con 0x563d23b89b00 2024-09-10T09:38:29.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:29.298+0000 7f190d20a740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x563d23b8ed20 con 0x563d23b8bf30 2024-09-10T09:38:29.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:29.298+0000 7f190d20a740 10 client.0 _ll_register_callbacks cb 0x563d23a939e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:38:29.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:29.299+0000 7f1903400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563d23b80d50 0x563d23b89080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:29.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:29.299+0000 7f1903400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563d23b80d50 0x563d23b89080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60936/0 (socket says 172.21.15.5:60936) 2024-09-10T09:38:29.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:29.299+0000 7f1903400640 1 -- 172.21.15.5:0/2777073183 learned_addr learned my addr 172.21.15.5:0/2777073183 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:38:29.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:29.299+0000 7f1903e00640 1 --2- 172.21.15.5:0/2777073183 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563d23b89b00 0x563d23b98010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:29.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:29.299+0000 7f1902a00640 1 --2- 172.21.15.5:0/2777073183 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563d23b8bf30 0x563d23b895c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:29.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:29.299+0000 7f1903400640 1 -- 172.21.15.5:0/2777073183 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563d23b8bf30 msgr2=0x563d23b895c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:38:29.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:29.299+0000 7f1903400640 1 --2- 172.21.15.5:0/2777073183 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563d23b8bf30 0x563d23b895c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:29.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:29.299+0000 7f1903400640 1 -- 172.21.15.5:0/2777073183 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563d23b89b00 msgr2=0x563d23b98010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:38:29.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:29.299+0000 7f1903400640 1 --2- 172.21.15.5:0/2777073183 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563d23b89b00 0x563d23b98010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:29.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:29.299+0000 7f1903400640 1 -- 172.21.15.5:0/2777073183 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563d238583c0 con 0x563d23b80d50 2024-09-10T09:38:29.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:29.299+0000 7f1903e00640 1 --2- 172.21.15.5:0/2777073183 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563d23b89b00 0x563d23b98010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:38:29.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:29.300+0000 7f1903400640 1 --2- 172.21.15.5:0/2777073183 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563d23b80d50 0x563d23b89080 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f18f40a4410 tx=0x7f18f4097e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60efdff744f3ce13 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:38:29.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:29.300+0000 7f18ed600640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-10T09:38:29.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:29.300+0000 7f18ed600640 1 -- 172.21.15.5:0/2777073183 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18f409b810 con 0x563d23b80d50 2024-09-10T09:38:29.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:29.300+0000 7f18ed600640 1 -- 172.21.15.5:0/2777073183 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f18f4004f40 con 0x563d23b80d50 2024-09-10T09:38:29.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:29.300+0000 7f190d20a740 1 -- 172.21.15.5:0/2777073183 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x563d23af4950 con 0x563d23b80d50 2024-09-10T09:38:29.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:29.300+0000 7f18ed600640 1 -- 172.21.15.5:0/2777073183 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f18f409b630 con 0x563d23b80d50 2024-09-10T09:38:29.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:29.300+0000 7f190d20a740 10 client.15312 fetch_fsmap learned FSMap version 30 2024-09-10T09:38:29.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:29.300+0000 7f190d20a740 1 -- 172.21.15.5:0/2777073183 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x563d23b9ae90 con 0x563d23b80d50 2024-09-10T09:38:29.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:29.301+0000 7f18ed600640 1 -- 172.21.15.5:0/2777073183 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x563d23af4950 con 0x563d23b80d50 2024-09-10T09:38:29.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:29.301+0000 7f18ed600640 1 -- 172.21.15.5:0/2777073183 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f18f40b3020 con 0x563d23b80d50 2024-09-10T09:38:29.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:29.301+0000 7f190d20a740 10 client.15312 fetch_fsmap finished waiting for FSMap version 30 2024-09-10T09:38:29.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:29.301+0000 7f190d20a740 10 client.15312 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T09:38:29.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:29.301+0000 7f190d20a740 20 client.15312 populate_metadata read hostname 'smithi005' 2024-09-10T09:38:29.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:29.301+0000 7f190d20a740 1 --2- 172.21.15.5:0/2777073183 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x563d23b9c1e0 0x563d23b9e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:29.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:29.301+0000 7f190d20a740 4 client.15312 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:38:29.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:29.301+0000 7f190d20a740 1 -- 172.21.15.5:0/2777073183 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563d23aeee60 con 0x563d23b9c1e0 2024-09-10T09:38:29.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:29.302+0000 7f1902a00640 1 --2- 172.21.15.5:0/2777073183 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x563d23b9c1e0 0x563d23b9e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:29.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:29.302+0000 7f1902a00640 1 --2- 172.21.15.5:0/2777073183 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x563d23b9c1e0 0x563d23b9e5d0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=71afc81dd038adf server_cookie=4c05bbd1e8c61548 in_seq=0 out_seq=0 2024-09-10T09:38:29.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:29.302+0000 7f18ed600640 10 client.15312 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T09:38:29.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:29.303+0000 7f18ed600640 1 -- 172.21.15.5:0/2777073183 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563d23aeee60 con 0x563d23b9c1e0 2024-09-10T09:38:29.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:29.303+0000 7f18ed600640 10 client.15312 handle_command_reply: tid=0 2024-09-10T09:38:29.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:29.303+0000 7f190d20a740 1 client.15312 shutdown 2024-09-10T09:38:29.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:29.303+0000 7f190d20a740 1 -- 172.21.15.5:0/2777073183 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563d23b80d50 msgr2=0x563d23b89080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:38:29.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:29.303+0000 7f190d20a740 1 --2- 172.21.15.5:0/2777073183 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563d23b80d50 0x563d23b89080 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f18f40a4410 tx=0x7f18f4097e20 comp rx=0 tx=0).stop 2024-09-10T09:38:29.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:29.303+0000 7f190d20a740 1 -- 172.21.15.5:0/2777073183 shutdown_connections 2024-09-10T09:38:29.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:29.303+0000 7f190d20a740 1 --2- 172.21.15.5:0/2777073183 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x563d23b9c1e0 0x563d23b9e5d0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:29.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:29.303+0000 7f190d20a740 1 --2- 172.21.15.5:0/2777073183 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563d23b89b00 0x563d23b98010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:29.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:29.303+0000 7f190d20a740 1 --2- 172.21.15.5:0/2777073183 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563d23b8bf30 0x563d23b895c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:29.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:29.303+0000 7f190d20a740 1 --2- 172.21.15.5:0/2777073183 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563d23b80d50 0x563d23b89080 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:29.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:29.303+0000 7f190d20a740 1 -- 172.21.15.5:0/2777073183 >> 172.21.15.5:0/2777073183 conn(0x563d23b7d2e0 msgr2=0x563d23b83010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:38:29.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:29.303+0000 7f18ed600640 0 client.15312 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T09:38:29.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:29.304+0000 7f190d20a740 1 -- 172.21.15.5:0/2777073183 shutdown_connections 2024-09-10T09:38:29.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:29.304+0000 7f190d20a740 1 -- 172.21.15.5:0/2777073183 wait complete. 2024-09-10T09:38:29.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:29.304+0000 7f190d20a740 20 client.15312 trim_cache size 0 max 16384 2024-09-10T09:38:29.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:29.323+0000 7f190d20a740 1 Processor -- start 2024-09-10T09:38:29.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:29.324+0000 7f190d20a740 1 -- start start 2024-09-10T09:38:29.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:29.324+0000 7f190d20a740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563d23b80d50 0x563d23b97f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:29.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:29.324+0000 7f190d20a740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563d23b89b00 0x563d23b7c290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:29.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:29.324+0000 7f190d20a740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563d23b8bf30 0x563d23b86340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:29.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:29.324+0000 7f190d20a740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f19041b5330 con 0x563d23b8bf30 2024-09-10T09:38:29.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:29.324+0000 7f190d20a740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f19041b6e70 con 0x563d23b89b00 2024-09-10T09:38:29.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:29.324+0000 7f190d20a740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f19041b5a00 con 0x563d23b80d50 2024-09-10T09:38:29.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:29.324+0000 7f1903e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563d23b8bf30 0x563d23b86340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:29.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:29.324+0000 7f1903400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563d23b80d50 0x563d23b97f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:29.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:29.324+0000 7f1902a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563d23b89b00 0x563d23b7c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:29.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:29.324+0000 7f1903e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563d23b8bf30 0x563d23b86340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60952/0 (socket says 172.21.15.5:60952) 2024-09-10T09:38:29.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:29.324+0000 7f1903e00640 1 -- 172.21.15.5:0/4054970811 learned_addr learned my addr 172.21.15.5:0/4054970811 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:38:29.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:29.325+0000 7f1902a00640 1 -- 172.21.15.5:0/4054970811 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563d23b80d50 msgr2=0x563d23b97f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:38:29.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:29.325+0000 7f1902a00640 1 --2- 172.21.15.5:0/4054970811 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563d23b80d50 0x563d23b97f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:29.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:29.325+0000 7f1902a00640 1 -- 172.21.15.5:0/4054970811 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563d23b8bf30 msgr2=0x563d23b86340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:38:29.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:29.325+0000 7f1902a00640 1 --2- 172.21.15.5:0/4054970811 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563d23b8bf30 0x563d23b86340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:29.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:29.325+0000 7f1902a00640 1 -- 172.21.15.5:0/4054970811 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563d23b8a020 con 0x563d23b89b00 2024-09-10T09:38:29.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:29.325+0000 7f1903e00640 1 --2- 172.21.15.5:0/4054970811 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563d23b8bf30 0x563d23b86340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:38:29.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:29.325+0000 7f1902a00640 1 --2- 172.21.15.5:0/4054970811 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563d23b89b00 0x563d23b7c290 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x563d23b8a630 tx=0x7f18f0017000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2cb6423430907031 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:38:29.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:29.326+0000 7f18efe00640 1 -- 172.21.15.5:0/4054970811 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18f00038d0 con 0x563d23b89b00 2024-09-10T09:38:29.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:29.326+0000 7f18efe00640 1 -- 172.21.15.5:0/4054970811 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f18f001f430 con 0x563d23b89b00 2024-09-10T09:38:29.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:29.326+0000 7f18efe00640 1 -- 172.21.15.5:0/4054970811 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18f002a660 con 0x563d23b89b00 2024-09-10T09:38:29.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:29.327+0000 7f190d20a740 1 -- 172.21.15.5:0/4054970811 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563d23b89b00 msgr2=0x563d23b7c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:38:29.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:29.327+0000 7f190d20a740 1 --2- 172.21.15.5:0/4054970811 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563d23b89b00 0x563d23b7c290 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x563d23b8a630 tx=0x7f18f0017000 comp rx=0 tx=0).stop 2024-09-10T09:38:29.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:29.327+0000 7f190d20a740 1 -- 172.21.15.5:0/4054970811 shutdown_connections 2024-09-10T09:38:29.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:29.327+0000 7f190d20a740 1 --2- 172.21.15.5:0/4054970811 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563d23b8bf30 0x563d23b86340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:29.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:29.327+0000 7f190d20a740 1 --2- 172.21.15.5:0/4054970811 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563d23b89b00 0x563d23b7c290 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:29.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:29.327+0000 7f190d20a740 1 --2- 172.21.15.5:0/4054970811 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563d23b80d50 0x563d23b97f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:29.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:29.327+0000 7f190d20a740 1 -- 172.21.15.5:0/4054970811 >> 172.21.15.5:0/4054970811 conn(0x563d23b7d2e0 msgr2=0x563d23b7bb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:38:29.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:29.327+0000 7f190d20a740 1 -- 172.21.15.5:0/4054970811 shutdown_connections 2024-09-10T09:38:29.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:29.327+0000 7f190d20a740 1 -- 172.21.15.5:0/4054970811 wait complete. 2024-09-10T09:38:29.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:29.328+0000 7f190d20a740 1 -- start start 2024-09-10T09:38:29.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:29.328+0000 7f190d20a740 1 Processor -- start 2024-09-10T09:38:29.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:29.328+0000 7f190d20a740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563d23b80d50 0x563d23ba1b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:29.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:29.328+0000 7f190d20a740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563d23b89b00 0x563d23b9eed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:29.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:29.328+0000 7f190d20a740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563d23b8bf30 0x563d23b9f410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:29.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:29.328+0000 7f190d20a740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f19041b67a0 con 0x563d23b8bf30 2024-09-10T09:38:29.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:29.328+0000 7f190d20a740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f19041b60d0 con 0x563d23b89b00 2024-09-10T09:38:29.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:29.328+0000 7f190d20a740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f19041b3310 con 0x563d23b80d50 2024-09-10T09:38:29.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:29.328+0000 7f190d20a740 10 client.0 _ll_register_callbacks cb 0x7f18f400cf20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:38:29.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:29.329+0000 7f1903e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563d23b8bf30 0x563d23b9f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:29.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:29.329+0000 7f1902a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563d23b89b00 0x563d23b9eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:29.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:29.329+0000 7f1902a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563d23b89b00 0x563d23b9eed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:57354/0 (socket says 172.21.15.5:57354) 2024-09-10T09:38:29.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:29.329+0000 7f1902a00640 1 -- 172.21.15.5:0/246292536 learned_addr learned my addr 172.21.15.5:0/246292536 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:38:29.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:29.329+0000 7f1903400640 1 --2- 172.21.15.5:0/246292536 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563d23b80d50 0x563d23ba1b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:29.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:29.329+0000 7f1902a00640 1 -- 172.21.15.5:0/246292536 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563d23b80d50 msgr2=0x563d23ba1b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:38:29.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:29.329+0000 7f1902a00640 1 --2- 172.21.15.5:0/246292536 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563d23b80d50 0x563d23ba1b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:29.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:29.329+0000 7f1902a00640 1 -- 172.21.15.5:0/246292536 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563d23b8bf30 msgr2=0x563d23b9f410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:38:29.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:29.329+0000 7f1902a00640 1 --2- 172.21.15.5:0/246292536 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563d23b8bf30 0x563d23b9f410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:29.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:29.329+0000 7f1902a00640 1 -- 172.21.15.5:0/246292536 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563d23b9fa50 con 0x563d23b89b00 2024-09-10T09:38:29.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:29.330+0000 7f1902a00640 1 --2- 172.21.15.5:0/246292536 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563d23b89b00 0x563d23b9eed0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f18f0009e10 tx=0x7f18f0009e40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=45cd1ed8a4150247 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:38:29.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:29.330+0000 7f18ed600640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-09-10T09:38:29.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:29.330+0000 7f18ed600640 1 -- 172.21.15.5:0/246292536 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18f0003ad0 con 0x563d23b89b00 2024-09-10T09:38:29.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:29.330+0000 7f190d20a740 1 -- 172.21.15.5:0/246292536 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f18f001f430 con 0x563d23b89b00 2024-09-10T09:38:29.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:29.330+0000 7f18ed600640 1 -- 172.21.15.5:0/246292536 <== mon.2 v2:172.21.15.102:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f18f0003c70 con 0x563d23b89b00 2024-09-10T09:38:29.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:29.330+0000 7f18ed600640 1 -- 172.21.15.5:0/246292536 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f18f0003e70 con 0x563d23b89b00 2024-09-10T09:38:29.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:29.331+0000 7f190d20a740 10 client.25334 fetch_fsmap learned FSMap version 30 2024-09-10T09:38:29.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:29.331+0000 7f190d20a740 1 -- 172.21.15.5:0/246292536 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x563d23c06960 con 0x563d23b89b00 2024-09-10T09:38:29.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:29.331+0000 7f18ed600640 1 -- 172.21.15.5:0/246292536 <== mon.2 v2:172.21.15.102:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f18f001f430 con 0x563d23b89b00 2024-09-10T09:38:29.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:29.331+0000 7f18ed600640 1 -- 172.21.15.5:0/246292536 <== mon.2 v2:172.21.15.102:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f18f0032070 con 0x563d23b89b00 2024-09-10T09:38:29.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:29.331+0000 7f190d20a740 10 client.25334 fetch_fsmap finished waiting for FSMap version 30 2024-09-10T09:38:29.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:29.331+0000 7f190d20a740 10 client.25334 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T09:38:29.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:29.331+0000 7f190d20a740 20 client.25334 populate_metadata read hostname 'smithi005' 2024-09-10T09:38:29.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:29.331+0000 7f190d20a740 1 --2- 172.21.15.5:0/246292536 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x563d23c070e0 0x563d23c09500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:29.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:29.331+0000 7f190d20a740 4 client.25334 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T09:38:29.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:29.331+0000 7f190d20a740 1 -- 172.21.15.5:0/246292536 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "scrub status"}) -- 0x563d23a6b0c0 con 0x563d23c070e0 2024-09-10T09:38:29.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:29.332+0000 7f1903400640 1 --2- 172.21.15.5:0/246292536 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x563d23c070e0 0x563d23c09500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:29.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:29.332+0000 7f18ed600640 10 client.25334 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T09:38:29.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:29.332+0000 7f1903400640 1 --2- 172.21.15.5:0/246292536 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x563d23c070e0 0x563d23c09500 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=de1ea93ae5799062 server_cookie=c90c0a6846d198c6 in_seq=0 out_seq=0 2024-09-10T09:38:29.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:29.333+0000 7f18ed600640 1 -- 172.21.15.5:0/246292536 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x563d23a6b0c0 con 0x563d23c070e0 2024-09-10T09:38:29.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:29.333+0000 7f18ed600640 10 client.25334 handle_command_reply: tid=0 2024-09-10T09:38:29.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:29.333+0000 7f190d20a740 1 client.25334 shutdown 2024-09-10T09:38:29.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:29.333+0000 7f190d20a740 1 -- 172.21.15.5:0/246292536 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563d23b89b00 msgr2=0x563d23b9eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:38:29.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:29.333+0000 7f190d20a740 1 --2- 172.21.15.5:0/246292536 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563d23b89b00 0x563d23b9eed0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f18f0009e10 tx=0x7f18f0009e40 comp rx=0 tx=0).stop 2024-09-10T09:38:29.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:29.333+0000 7f190d20a740 1 -- 172.21.15.5:0/246292536 shutdown_connections 2024-09-10T09:38:29.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:29.333+0000 7f190d20a740 1 --2- 172.21.15.5:0/246292536 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x563d23c070e0 0x563d23c09500 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:29.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:29.333+0000 7f190d20a740 1 --2- 172.21.15.5:0/246292536 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563d23b8bf30 0x563d23b9f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:29.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:29.333+0000 7f190d20a740 1 --2- 172.21.15.5:0/246292536 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563d23b89b00 0x563d23b9eed0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:29.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:29.333+0000 7f18ed600640 0 client.25334 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T09:38:29.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:29.333+0000 7f190d20a740 1 --2- 172.21.15.5:0/246292536 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563d23b80d50 0x563d23ba1b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:29.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:29.333+0000 7f190d20a740 1 -- 172.21.15.5:0/246292536 >> 172.21.15.5:0/246292536 conn(0x563d23b7d2e0 msgr2=0x563d23b82fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:38:29.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:29.333+0000 7f190d20a740 1 -- 172.21.15.5:0/246292536 shutdown_connections 2024-09-10T09:38:29.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:29.333+0000 7f190d20a740 1 -- 172.21.15.5:0/246292536 wait complete. 2024-09-10T09:38:29.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:29.334+0000 7f190d20a740 20 client.25334 trim_cache size 0 max 16384 2024-09-10T09:38:29.339 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-10T09:38:29.339 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-10T09:38:29.340 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2024-09-10T09:38:29.340 INFO:teuthology.orchestra.run.smithi005.stdout: "a679c2e3-1405-4dbd-a91e-4744f2f0dd53": { 2024-09-10T09:38:29.340 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2024-09-10T09:38:29.340 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "a679c2e3-1405-4dbd-a91e-4744f2f0dd53", 2024-09-10T09:38:29.340 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2024-09-10T09:38:29.340 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T09:38:29.340 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T09:38:29.340 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-10T09:38:29.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:29.334+0000 7f1909200640 1 -- 172.21.15.5:0/190514507 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f18e80a2c10 msgr2=0x7f18e80a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:38:29.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:29.334+0000 7f1909200640 1 --2- 172.21.15.5:0/190514507 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f18e80a2c10 0x7f18e80a50d0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x563d23b7e660 tx=0x7f18f0005f90 comp rx=0 tx=0).stop 2024-09-10T09:38:29.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:29.334+0000 7f1909200640 1 -- 172.21.15.5:0/190514507 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f19040745c0 msgr2=0x7f190410db90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:38:29.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:29.334+0000 7f1909200640 1 --2- 172.21.15.5:0/190514507 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f19040745c0 0x7f190410db90 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f18f400e9f0 tx=0x7f18f400edf0 comp rx=0 tx=0).stop 2024-09-10T09:38:29.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:29.335+0000 7f1909200640 1 -- 172.21.15.5:0/190514507 shutdown_connections 2024-09-10T09:38:29.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:29.335+0000 7f1909200640 1 --2- 172.21.15.5:0/190514507 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f18e80a2c10 0x7f18e80a50d0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:29.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:29.335+0000 7f1909200640 1 --2- 172.21.15.5:0/190514507 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f190410e260 0x7f190410b180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:29.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:29.335+0000 7f1909200640 1 --2- 172.21.15.5:0/190514507 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1904074ee0 0x7f190410ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:29.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:29.335+0000 7f1909200640 1 --2- 172.21.15.5:0/190514507 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f19040745c0 0x7f190410db90 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:29.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:29.335+0000 7f1909200640 1 -- 172.21.15.5:0/190514507 >> 172.21.15.5:0/190514507 conn(0x7f19040fd740 msgr2=0x7f19040fe440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:38:29.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:29.335+0000 7f1909200640 1 -- 172.21.15.5:0/190514507 shutdown_connections 2024-09-10T09:38:29.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:29.335+0000 7f1909200640 1 -- 172.21.15.5:0/190514507 wait complete. 2024-09-10T09:38:29.342 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:38:29.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:38:28 smithi159 ceph-mon[35513]: pgmap v397: 129 pgs: 129 active+clean; 653 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-09-10T09:38:30.392 INFO:tasks.cephfs.filesystem:scrub status for tag:a679c2e3-1405-4dbd-a91e-4744f2f0dd53 - {'path': '/', 'tag': 'a679c2e3-1405-4dbd-a91e-4744f2f0dd53', 'options': 'recursive,force'} 2024-09-10T09:38:30.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:30.678+0000 7f4264200640 1 -- 172.21.15.5:0/2189277095 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f425c109120 msgr2=0x7f425c110ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:38:30.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:30.678+0000 7f4264200640 1 --2- 172.21.15.5:0/2189277095 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f425c109120 0x7f425c110ca0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f4254009ca0 tx=0x7f425402a8c0 comp rx=0 tx=0).stop 2024-09-10T09:38:30.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:30.678+0000 7f4264200640 1 -- 172.21.15.5:0/2189277095 shutdown_connections 2024-09-10T09:38:30.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:30.678+0000 7f4264200640 1 --2- 172.21.15.5:0/2189277095 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f425c109120 0x7f425c110ca0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:30.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:30.678+0000 7f4264200640 1 --2- 172.21.15.5:0/2189277095 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f425c1049a0 0x7f425c1089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:30.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:30.678+0000 7f4264200640 1 --2- 172.21.15.5:0/2189277095 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f425c103ff0 0x7f425c1043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:30.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:30.678+0000 7f4264200640 1 -- 172.21.15.5:0/2189277095 >> 172.21.15.5:0/2189277095 conn(0x7f425c0ffa90 msgr2=0x7f425c101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:38:30.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:30.678+0000 7f4264200640 1 -- 172.21.15.5:0/2189277095 shutdown_connections 2024-09-10T09:38:30.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:30.678+0000 7f4264200640 1 -- 172.21.15.5:0/2189277095 wait complete. 2024-09-10T09:38:30.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:30.679+0000 7f4264200640 1 Processor -- start 2024-09-10T09:38:30.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:30.679+0000 7f4264200640 1 -- start start 2024-09-10T09:38:30.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:30.680+0000 7f4264200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f425c103ff0 0x7f425c19a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:30.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:30.680+0000 7f4264200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f425c1049a0 0x7f425c19ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:30.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:30.680+0000 7f4264200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f425c109120 0x7f425c19eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:30.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:30.680+0000 7f4264200640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f425c114130 con 0x7f425c1049a0 2024-09-10T09:38:30.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:30.680+0000 7f4264200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f425c113fb0 con 0x7f425c103ff0 2024-09-10T09:38:30.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:30.680+0000 7f4264200640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f425c1142b0 con 0x7f425c109120 2024-09-10T09:38:30.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:30.680+0000 7f4261800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f425c103ff0 0x7f425c19a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:30.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:30.680+0000 7f4260e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f425c1049a0 0x7f425c19ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:30.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:30.680+0000 7f4261800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f425c103ff0 0x7f425c19a610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:57360/0 (socket says 172.21.15.5:57360) 2024-09-10T09:38:30.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:30.680+0000 7f4261800640 1 -- 172.21.15.5:0/2546117064 learned_addr learned my addr 172.21.15.5:0/2546117064 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:38:30.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:30.680+0000 7f4262200640 1 --2- 172.21.15.5:0/2546117064 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f425c109120 0x7f425c19eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:30.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:30.681+0000 7f4261800640 1 -- 172.21.15.5:0/2546117064 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f425c109120 msgr2=0x7f425c19eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:38:30.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:30.681+0000 7f4261800640 1 --2- 172.21.15.5:0/2546117064 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f425c109120 0x7f425c19eee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:30.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:30.681+0000 7f4261800640 1 -- 172.21.15.5:0/2546117064 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f425c1049a0 msgr2=0x7f425c19ab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:38:30.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:30.681+0000 7f4261800640 1 --2- 172.21.15.5:0/2546117064 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f425c1049a0 0x7f425c19ab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:30.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:30.681+0000 7f4261800640 1 -- 172.21.15.5:0/2546117064 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f425c19f5c0 con 0x7f425c103ff0 2024-09-10T09:38:30.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:30.681+0000 7f4261800640 1 --2- 172.21.15.5:0/2546117064 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f425c103ff0 0x7f425c19a610 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f425000e9f0 tx=0x7f425000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:38:30.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:30.681+0000 7f425a000640 1 -- 172.21.15.5:0/2546117064 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f425000cd80 con 0x7f425c103ff0 2024-09-10T09:38:30.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:30.681+0000 7f4264200640 1 -- 172.21.15.5:0/2546117064 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f425c19f8b0 con 0x7f425c103ff0 2024-09-10T09:38:30.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:30.681+0000 7f425a000640 1 -- 172.21.15.5:0/2546117064 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4250010910 con 0x7f425c103ff0 2024-09-10T09:38:30.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:30.681+0000 7f425a000640 1 -- 172.21.15.5:0/2546117064 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42500188f0 con 0x7f425c103ff0 2024-09-10T09:38:30.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:30.681+0000 7f4264200640 1 -- 172.21.15.5:0/2546117064 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f425c1a7190 con 0x7f425c103ff0 2024-09-10T09:38:30.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:30.682+0000 7f4264200640 1 -- 172.21.15.5:0/2546117064 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f425c1043d0 con 0x7f425c103ff0 2024-09-10T09:38:30.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:30.684+0000 7f425a000640 1 -- 172.21.15.5:0/2546117064 <== mon.2 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 19) ==== 137883+0+0 (secure 0 0 0) 0x7f4250010ac0 con 0x7f425c103ff0 2024-09-10T09:38:30.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:30.686+0000 7f425a000640 1 --2- 172.21.15.5:0/2546117064 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f42440a2c10 0x7f42440a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:30.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:30.686+0000 7f425a000640 1 -- 172.21.15.5:0/2546117064 <== mon.2 v2:172.21.15.102:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f42500cfdb0 con 0x7f425c103ff0 2024-09-10T09:38:30.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:30.686+0000 7f4260e00640 1 --2- 172.21.15.5:0/2546117064 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f42440a2c10 0x7f42440a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:30.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:30.687+0000 7f4260e00640 1 --2- 172.21.15.5:0/2546117064 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f42440a2c10 0x7f42440a50d0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f425c10a9e0 tx=0x7f4248005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:38:30.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:30.688+0000 7f425a000640 1 -- 172.21.15.5:0/2546117064 <== mon.2 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f425009c110 con 0x7f425c103ff0 2024-09-10T09:38:30.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:30.871+0000 7f4264200640 1 -- 172.21.15.5:0/2546117064 --> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f425c100cc0 con 0x7f42440a2c10 2024-09-10T09:38:30.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:30.879+0000 7f425a000640 1 -- 172.21.15.5:0/2546117064 <== mgr.14150 v2:172.21.15.5:6800/2707348081 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340827 (secure 0 0 0) 0x7f425c100cc0 con 0x7f42440a2c10 2024-09-10T09:38:30.884 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-10T09:38:30.886 INFO:teuthology.orchestra.run.smithi005.stderr:dumped all 2024-09-10T09:38:30.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:30.884+0000 7f4264200640 1 -- 172.21.15.5:0/2546117064 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f42440a2c10 msgr2=0x7f42440a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:38:30.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:30.884+0000 7f4264200640 1 --2- 172.21.15.5:0/2546117064 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f42440a2c10 0x7f42440a50d0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f425c10a9e0 tx=0x7f4248005f90 comp rx=0 tx=0).stop 2024-09-10T09:38:30.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:30.884+0000 7f4264200640 1 -- 172.21.15.5:0/2546117064 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f425c103ff0 msgr2=0x7f425c19a610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:38:30.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:30.884+0000 7f4264200640 1 --2- 172.21.15.5:0/2546117064 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f425c103ff0 0x7f425c19a610 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f425000e9f0 tx=0x7f425000eeb0 comp rx=0 tx=0).stop 2024-09-10T09:38:30.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:30.884+0000 7f4264200640 1 -- 172.21.15.5:0/2546117064 shutdown_connections 2024-09-10T09:38:30.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:30.884+0000 7f4264200640 1 --2- 172.21.15.5:0/2546117064 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f42440a2c10 0x7f42440a50d0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:30.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:30.884+0000 7f4264200640 1 --2- 172.21.15.5:0/2546117064 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f425c109120 0x7f425c19eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:30.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:30.884+0000 7f4264200640 1 --2- 172.21.15.5:0/2546117064 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f425c1049a0 0x7f425c19ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:30.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:30.884+0000 7f4264200640 1 --2- 172.21.15.5:0/2546117064 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f425c103ff0 0x7f425c19a610 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:30.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:30.884+0000 7f4264200640 1 -- 172.21.15.5:0/2546117064 >> 172.21.15.5:0/2546117064 conn(0x7f425c0ffa90 msgr2=0x7f425c1005b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:38:30.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:30.884+0000 7f4264200640 1 -- 172.21.15.5:0/2546117064 shutdown_connections 2024-09-10T09:38:30.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:30.885+0000 7f4264200640 1 -- 172.21.15.5:0/2546117064 wait complete. 2024-09-10T09:38:30.946 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:38:30 smithi005 ceph-mon[30350]: pgmap v398: 129 pgs: 129 active+clean; 653 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-09-10T09:38:31.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:38:30 smithi102 ceph-mon[35442]: pgmap v398: 129 pgs: 129 active+clean; 653 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-09-10T09:38:31.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:38:30 smithi159 ceph-mon[35513]: pgmap v398: 129 pgs: 129 active+clean; 653 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-09-10T09:38:31.867 INFO:teuthology.orchestra.run.smithi005.stdout:{"pg_ready":true,"pg_map":{"version":398,"stamp":"2024-09-10T09:38:29.573510+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":669121,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1474,"num_read_kb":872,"num_write":453,"num_write_kb":3104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":1181751,"num_keys_recovered":7,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":870,"ondisk_log_size":870,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":378988,"kb_used_data":13100,"kb_used_omap":18,"kb_used_meta":365677,"kb_avail":1124562836,"statfs":{"total":1151940427776,"available":1151552344064,"internally_reserved":0,"allocated":13414400,"data_stored":8627453,"data_compressed":498435,"data_compressed_allocated":675840,"data_compressed_original":2850816,"omap_allocated":19141,"internal_metadata":374453563},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":7849,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":114,"num_read_kb":65,"num_write":12,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003161"},"pg_stats":[{"pgid":"2.1f","version":"96'15","reported_seq":190,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:38:23.676913+0000","last_change":"2024-09-10T09:35:25.533085+0000","last_active":"2024-09-10T09:38:23.676913+0000","last_peered":"2024-09-10T09:38:23.676913+0000","last_clean":"2024-09-10T09:38:23.676913+0000","last_became_active":"2024-09-10T09:35:25.532729+0000","last_became_peered":"2024-09-10T09:35:25.532729+0000","last_unstale":"2024-09-10T09:38:23.676913+0000","last_undegraded":"2024-09-10T09:38:23.676913+0000","last_fullsized":"2024-09-10T09:38:23.676913+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:55:11.158279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":131,"num_read_kb":131,"num_write":16,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"96'5","reported_seq":21,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.545575+0000","last_change":"2024-09-10T09:36:06.767356+0000","last_active":"2024-09-10T09:37:35.545575+0000","last_peered":"2024-09-10T09:37:35.545575+0000","last_clean":"2024-09-10T09:37:35.545575+0000","last_became_active":"2024-09-10T09:36:06.766931+0000","last_became_peered":"2024-09-10T09:36:06.766931+0000","last_unstale":"2024-09-10T09:37:35.545575+0000","last_undegraded":"2024-09-10T09:37:35.545575+0000","last_fullsized":"2024-09-10T09:37:35.545575+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:25:02.777327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,7],"acting":[9,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.527523+0000","last_change":"2024-09-10T09:35:36.629797+0000","last_active":"2024-09-10T09:37:07.527523+0000","last_peered":"2024-09-10T09:37:07.527523+0000","last_clean":"2024-09-10T09:37:07.527523+0000","last_became_active":"2024-09-10T09:35:36.629414+0000","last_became_peered":"2024-09-10T09:35:36.629414+0000","last_unstale":"2024-09-10T09:37:07.527523+0000","last_undegraded":"2024-09-10T09:37:07.527523+0000","last_fullsized":"2024-09-10T09:37:07.527523+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:42:44.610581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"94'2","reported_seq":47,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.524964+0000","last_change":"2024-09-10T09:35:25.852022+0000","last_active":"2024-09-10T09:37:07.524964+0000","last_peered":"2024-09-10T09:37:07.524964+0000","last_clean":"2024-09-10T09:37:07.524964+0000","last_became_active":"2024-09-10T09:35:25.851587+0000","last_became_peered":"2024-09-10T09:35:25.851587+0000","last_unstale":"2024-09-10T09:37:07.524964+0000","last_undegraded":"2024-09-10T09:37:07.524964+0000","last_fullsized":"2024-09-10T09:37:07.524964+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:37:26.085914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"96'6","reported_seq":43,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.556261+0000","last_change":"2024-09-10T09:35:28.579260+0000","last_active":"2024-09-10T09:37:35.556261+0000","last_peered":"2024-09-10T09:37:35.556261+0000","last_clean":"2024-09-10T09:37:35.556261+0000","last_became_active":"2024-09-10T09:35:28.574584+0000","last_became_peered":"2024-09-10T09:35:28.574584+0000","last_unstale":"2024-09-10T09:37:35.556261+0000","last_undegraded":"2024-09-10T09:37:35.556261+0000","last_fullsized":"2024-09-10T09:37:35.556261+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:31:19.277689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.568359+0000","last_change":"2024-09-10T09:36:06.874861+0000","last_active":"2024-09-10T09:37:07.568359+0000","last_peered":"2024-09-10T09:37:07.568359+0000","last_clean":"2024-09-10T09:37:07.568359+0000","last_became_active":"2024-09-10T09:36:06.873930+0000","last_became_peered":"2024-09-10T09:36:06.873930+0000","last_unstale":"2024-09-10T09:37:07.568359+0000","last_undegraded":"2024-09-10T09:37:07.568359+0000","last_fullsized":"2024-09-10T09:37:07.568359+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:35:01.354061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,8,3],"acting":[4,6,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"96'13","reported_seq":58,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:59.271922+0000","last_change":"2024-09-10T09:35:25.534571+0000","last_active":"2024-09-10T09:37:59.271922+0000","last_peered":"2024-09-10T09:37:59.271922+0000","last_clean":"2024-09-10T09:37:59.271922+0000","last_became_active":"2024-09-10T09:35:25.534235+0000","last_became_peered":"2024-09-10T09:35:25.534235+0000","last_unstale":"2024-09-10T09:37:59.271922+0000","last_undegraded":"2024-09-10T09:37:59.271922+0000","last_fullsized":"2024-09-10T09:37:59.271922+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:50:21.447542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"96'9","reported_seq":48,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.553039+0000","last_change":"2024-09-10T09:35:28.565541+0000","last_active":"2024-09-10T09:37:35.553039+0000","last_peered":"2024-09-10T09:37:35.553039+0000","last_clean":"2024-09-10T09:37:35.553039+0000","last_became_active":"2024-09-10T09:35:28.565095+0000","last_became_peered":"2024-09-10T09:35:28.565095+0000","last_unstale":"2024-09-10T09:37:35.553039+0000","last_undegraded":"2024-09-10T09:37:35.553039+0000","last_fullsized":"2024-09-10T09:37:35.553039+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:29:31.285401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.568600+0000","last_change":"2024-09-10T09:35:36.629277+0000","last_active":"2024-09-10T09:37:07.568600+0000","last_peered":"2024-09-10T09:37:07.568600+0000","last_clean":"2024-09-10T09:37:07.568600+0000","last_became_active":"2024-09-10T09:35:36.628687+0000","last_became_peered":"2024-09-10T09:35:36.628687+0000","last_unstale":"2024-09-10T09:37:07.568600+0000","last_undegraded":"2024-09-10T09:37:07.568600+0000","last_fullsized":"2024-09-10T09:37:07.568600+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:34:33.628827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"94'5","reported_seq":50,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.530161+0000","last_change":"2024-09-10T09:35:25.536137+0000","last_active":"2024-09-10T09:37:07.530161+0000","last_peered":"2024-09-10T09:37:07.530161+0000","last_clean":"2024-09-10T09:37:07.530161+0000","last_became_active":"2024-09-10T09:35:25.535728+0000","last_became_peered":"2024-09-10T09:35:25.535728+0000","last_unstale":"2024-09-10T09:37:07.530161+0000","last_undegraded":"2024-09-10T09:37:07.530161+0000","last_fullsized":"2024-09-10T09:37:07.530161+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:36:04.829901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"96'15","reported_seq":60,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.547598+0000","last_change":"2024-09-10T09:36:06.874425+0000","last_active":"2024-09-10T09:37:35.547598+0000","last_peered":"2024-09-10T09:37:35.547598+0000","last_clean":"2024-09-10T09:37:35.547598+0000","last_became_active":"2024-09-10T09:36:06.874050+0000","last_became_peered":"2024-09-10T09:36:06.874050+0000","last_unstale":"2024-09-10T09:37:35.547598+0000","last_undegraded":"2024-09-10T09:37:35.547598+0000","last_fullsized":"2024-09-10T09:37:35.547598+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:30:51.200335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,1],"acting":[5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.568073+0000","last_change":"2024-09-10T09:35:36.615314+0000","last_active":"2024-09-10T09:37:07.568073+0000","last_peered":"2024-09-10T09:37:07.568073+0000","last_clean":"2024-09-10T09:37:07.568073+0000","last_became_active":"2024-09-10T09:35:36.614820+0000","last_became_peered":"2024-09-10T09:35:36.614820+0000","last_unstale":"2024-09-10T09:37:07.568073+0000","last_undegraded":"2024-09-10T09:37:07.568073+0000","last_fullsized":"2024-09-10T09:37:07.568073+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:51:16.897935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"96'5","reported_seq":42,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.556084+0000","last_change":"2024-09-10T09:35:28.572346+0000","last_active":"2024-09-10T09:37:35.556084+0000","last_peered":"2024-09-10T09:37:35.556084+0000","last_clean":"2024-09-10T09:37:35.556084+0000","last_became_active":"2024-09-10T09:35:28.571839+0000","last_became_peered":"2024-09-10T09:35:28.571839+0000","last_unstale":"2024-09-10T09:37:35.556084+0000","last_undegraded":"2024-09-10T09:37:35.556084+0000","last_fullsized":"2024-09-10T09:37:35.556084+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:41:29.662441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"94'4","reported_seq":49,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.529918+0000","last_change":"2024-09-10T09:35:25.538747+0000","last_active":"2024-09-10T09:37:07.529918+0000","last_peered":"2024-09-10T09:37:07.529918+0000","last_clean":"2024-09-10T09:37:07.529918+0000","last_became_active":"2024-09-10T09:35:25.538330+0000","last_became_peered":"2024-09-10T09:35:25.538330+0000","last_unstale":"2024-09-10T09:37:07.529918+0000","last_undegraded":"2024-09-10T09:37:07.529918+0000","last_fullsized":"2024-09-10T09:37:07.529918+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:15:08.340688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.568436+0000","last_change":"2024-09-10T09:35:37.270106+0000","last_active":"2024-09-10T09:37:07.568436+0000","last_peered":"2024-09-10T09:37:07.568436+0000","last_clean":"2024-09-10T09:37:07.568436+0000","last_became_active":"2024-09-10T09:35:37.269575+0000","last_became_peered":"2024-09-10T09:35:37.269575+0000","last_unstale":"2024-09-10T09:37:07.568436+0000","last_undegraded":"2024-09-10T09:37:07.568436+0000","last_fullsized":"2024-09-10T09:37:07.568436+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:35:00.414602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"96'8","reported_seq":47,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.555784+0000","last_change":"2024-09-10T09:35:28.569230+0000","last_active":"2024-09-10T09:37:35.555784+0000","last_peered":"2024-09-10T09:37:35.555784+0000","last_clean":"2024-09-10T09:37:35.555784+0000","last_became_active":"2024-09-10T09:35:28.568266+0000","last_became_peered":"2024-09-10T09:35:28.568266+0000","last_unstale":"2024-09-10T09:37:35.555784+0000","last_undegraded":"2024-09-10T09:37:35.555784+0000","last_fullsized":"2024-09-10T09:37:35.555784+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:49:43.730686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"94'3","reported_seq":17,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.544707+0000","last_change":"2024-09-10T09:37:07.544298+0000","last_active":"2024-09-10T09:37:07.544707+0000","last_peered":"2024-09-10T09:37:07.544707+0000","last_clean":"2024-09-10T09:37:07.544707+0000","last_became_active":"2024-09-10T09:37:07.531815+0000","last_became_peered":"2024-09-10T09:37:07.531815+0000","last_unstale":"2024-09-10T09:37:07.544707+0000","last_undegraded":"2024-09-10T09:37:07.544707+0000","last_fullsized":"2024-09-10T09:37:07.544707+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:18:10.091895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"94'1","reported_seq":37,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.568484+0000","last_change":"2024-09-10T09:36:06.875056+0000","last_active":"2024-09-10T09:37:07.568484+0000","last_peered":"2024-09-10T09:37:07.568484+0000","last_clean":"2024-09-10T09:37:07.568484+0000","last_became_active":"2024-09-10T09:36:06.874400+0000","last_became_peered":"2024-09-10T09:36:06.874400+0000","last_unstale":"2024-09-10T09:37:07.568484+0000","last_undegraded":"2024-09-10T09:37:07.568484+0000","last_fullsized":"2024-09-10T09:37:07.568484+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:21:06.300712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,7,8],"acting":[4,10,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"96'9","reported_seq":48,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.552565+0000","last_change":"2024-09-10T09:35:28.571319+0000","last_active":"2024-09-10T09:37:35.552565+0000","last_peered":"2024-09-10T09:37:35.552565+0000","last_clean":"2024-09-10T09:37:35.552565+0000","last_became_active":"2024-09-10T09:35:28.570972+0000","last_became_peered":"2024-09-10T09:35:28.570972+0000","last_unstale":"2024-09-10T09:37:35.552565+0000","last_undegraded":"2024-09-10T09:37:35.552565+0000","last_fullsized":"2024-09-10T09:37:35.552565+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:45:19.300535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"94'7","reported_seq":52,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.524483+0000","last_change":"2024-09-10T09:35:25.530489+0000","last_active":"2024-09-10T09:37:07.524483+0000","last_peered":"2024-09-10T09:37:07.524483+0000","last_clean":"2024-09-10T09:37:07.524483+0000","last_became_active":"2024-09-10T09:35:25.530197+0000","last_became_peered":"2024-09-10T09:35:25.530197+0000","last_unstale":"2024-09-10T09:37:07.524483+0000","last_undegraded":"2024-09-10T09:37:07.524483+0000","last_fullsized":"2024-09-10T09:37:07.524483+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:28:57.539569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.527650+0000","last_change":"2024-09-10T09:35:36.630507+0000","last_active":"2024-09-10T09:37:07.527650+0000","last_peered":"2024-09-10T09:37:07.527650+0000","last_clean":"2024-09-10T09:37:07.527650+0000","last_became_active":"2024-09-10T09:35:36.630052+0000","last_became_peered":"2024-09-10T09:35:36.630052+0000","last_unstale":"2024-09-10T09:37:07.527650+0000","last_undegraded":"2024-09-10T09:37:07.527650+0000","last_fullsized":"2024-09-10T09:37:07.527650+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:13:42.691382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"96'2","reported_seq":41,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.532487+0000","last_change":"2024-09-10T09:35:28.565650+0000","last_active":"2024-09-10T09:37:35.532487+0000","last_peered":"2024-09-10T09:37:35.532487+0000","last_clean":"2024-09-10T09:37:35.532487+0000","last_became_active":"2024-09-10T09:35:28.565303+0000","last_became_peered":"2024-09-10T09:35:28.565303+0000","last_unstale":"2024-09-10T09:37:35.532487+0000","last_undegraded":"2024-09-10T09:37:35.532487+0000","last_fullsized":"2024-09-10T09:37:35.532487+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:48:08.965487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"94'2","reported_seq":46,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.574507+0000","last_change":"2024-09-10T09:35:25.851326+0000","last_active":"2024-09-10T09:37:07.574507+0000","last_peered":"2024-09-10T09:37:07.574507+0000","last_clean":"2024-09-10T09:37:07.574507+0000","last_became_active":"2024-09-10T09:35:25.850981+0000","last_became_peered":"2024-09-10T09:35:25.850981+0000","last_unstale":"2024-09-10T09:37:07.574507+0000","last_undegraded":"2024-09-10T09:37:07.574507+0000","last_fullsized":"2024-09-10T09:37:07.574507+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:50:28.877840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.568510+0000","last_change":"2024-09-10T09:35:37.270138+0000","last_active":"2024-09-10T09:37:07.568510+0000","last_peered":"2024-09-10T09:37:07.568510+0000","last_clean":"2024-09-10T09:37:07.568510+0000","last_became_active":"2024-09-10T09:35:37.269663+0000","last_became_peered":"2024-09-10T09:35:37.269663+0000","last_unstale":"2024-09-10T09:37:07.568510+0000","last_undegraded":"2024-09-10T09:37:07.568510+0000","last_fullsized":"2024-09-10T09:37:07.568510+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:18:00.105970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"96'7","reported_seq":44,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.554144+0000","last_change":"2024-09-10T09:35:28.576197+0000","last_active":"2024-09-10T09:37:35.554144+0000","last_peered":"2024-09-10T09:37:35.554144+0000","last_clean":"2024-09-10T09:37:35.554144+0000","last_became_active":"2024-09-10T09:35:28.575859+0000","last_became_peered":"2024-09-10T09:35:28.575859+0000","last_unstale":"2024-09-10T09:37:35.554144+0000","last_undegraded":"2024-09-10T09:37:35.554144+0000","last_fullsized":"2024-09-10T09:37:35.554144+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:09:39.458761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"94'7","reported_seq":51,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.568805+0000","last_change":"2024-09-10T09:35:25.990587+0000","last_active":"2024-09-10T09:37:07.568805+0000","last_peered":"2024-09-10T09:37:07.568805+0000","last_clean":"2024-09-10T09:37:07.568805+0000","last_became_active":"2024-09-10T09:35:25.990302+0000","last_became_peered":"2024-09-10T09:35:25.990302+0000","last_unstale":"2024-09-10T09:37:07.568805+0000","last_undegraded":"2024-09-10T09:37:07.568805+0000","last_fullsized":"2024-09-10T09:37:07.568805+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:18:41.244441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.527768+0000","last_change":"2024-09-10T09:35:36.612983+0000","last_active":"2024-09-10T09:37:07.527768+0000","last_peered":"2024-09-10T09:37:07.527768+0000","last_clean":"2024-09-10T09:37:07.527768+0000","last_became_active":"2024-09-10T09:35:36.612542+0000","last_became_peered":"2024-09-10T09:35:36.612542+0000","last_unstale":"2024-09-10T09:37:07.527768+0000","last_undegraded":"2024-09-10T09:37:07.527768+0000","last_fullsized":"2024-09-10T09:37:07.527768+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:50:16.298195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"96'5","reported_seq":44,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.554267+0000","last_change":"2024-09-10T09:35:28.564121+0000","last_active":"2024-09-10T09:37:35.554267+0000","last_peered":"2024-09-10T09:37:35.554267+0000","last_clean":"2024-09-10T09:37:35.554267+0000","last_became_active":"2024-09-10T09:35:28.562780+0000","last_became_peered":"2024-09-10T09:35:28.562780+0000","last_unstale":"2024-09-10T09:37:35.554267+0000","last_undegraded":"2024-09-10T09:37:35.554267+0000","last_fullsized":"2024-09-10T09:37:35.554267+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:53:15.690478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"96'18","reported_seq":78,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:38:15.594145+0000","last_change":"2024-09-10T09:37:07.541031+0000","last_active":"2024-09-10T09:38:15.594145+0000","last_peered":"2024-09-10T09:38:15.594145+0000","last_clean":"2024-09-10T09:38:15.594145+0000","last_became_active":"2024-09-10T09:37:07.529577+0000","last_became_peered":"2024-09-10T09:37:07.529577+0000","last_unstale":"2024-09-10T09:38:15.594145+0000","last_undegraded":"2024-09-10T09:38:15.594145+0000","last_fullsized":"2024-09-10T09:38:15.594145+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:14:13.440346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":33,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":7,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.524145+0000","last_change":"2024-09-10T09:35:37.270888+0000","last_active":"2024-09-10T09:37:07.524145+0000","last_peered":"2024-09-10T09:37:07.524145+0000","last_clean":"2024-09-10T09:37:07.524145+0000","last_became_active":"2024-09-10T09:35:37.270229+0000","last_became_peered":"2024-09-10T09:35:37.270229+0000","last_unstale":"2024-09-10T09:37:07.524145+0000","last_undegraded":"2024-09-10T09:37:07.524145+0000","last_fullsized":"2024-09-10T09:37:07.524145+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:03:54.223463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"96'9","reported_seq":48,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.547904+0000","last_change":"2024-09-10T09:35:28.572799+0000","last_active":"2024-09-10T09:37:35.547904+0000","last_peered":"2024-09-10T09:37:35.547904+0000","last_clean":"2024-09-10T09:37:35.547904+0000","last_became_active":"2024-09-10T09:35:28.572310+0000","last_became_peered":"2024-09-10T09:35:28.572310+0000","last_unstale":"2024-09-10T09:37:35.547904+0000","last_undegraded":"2024-09-10T09:37:35.547904+0000","last_fullsized":"2024-09-10T09:37:35.547904+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:33:51.446766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"94'4","reported_seq":49,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.530456+0000","last_change":"2024-09-10T09:35:25.851868+0000","last_active":"2024-09-10T09:37:07.530456+0000","last_peered":"2024-09-10T09:37:07.530456+0000","last_clean":"2024-09-10T09:37:07.530456+0000","last_became_active":"2024-09-10T09:35:25.851447+0000","last_became_peered":"2024-09-10T09:35:25.851447+0000","last_unstale":"2024-09-10T09:37:07.530456+0000","last_undegraded":"2024-09-10T09:37:07.530456+0000","last_fullsized":"2024-09-10T09:37:07.530456+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:15:57.706690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.526545+0000","last_change":"2024-09-10T09:35:36.619459+0000","last_active":"2024-09-10T09:37:07.526545+0000","last_peered":"2024-09-10T09:37:07.526545+0000","last_clean":"2024-09-10T09:37:07.526545+0000","last_became_active":"2024-09-10T09:35:36.619023+0000","last_became_peered":"2024-09-10T09:35:36.619023+0000","last_unstale":"2024-09-10T09:37:07.526545+0000","last_undegraded":"2024-09-10T09:37:07.526545+0000","last_fullsized":"2024-09-10T09:37:07.526545+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:07:32.999231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"96'4","reported_seq":43,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.548010+0000","last_change":"2024-09-10T09:35:28.575668+0000","last_active":"2024-09-10T09:37:35.548010+0000","last_peered":"2024-09-10T09:37:35.548010+0000","last_clean":"2024-09-10T09:37:35.548010+0000","last_became_active":"2024-09-10T09:35:28.575425+0000","last_became_peered":"2024-09-10T09:35:28.575425+0000","last_unstale":"2024-09-10T09:37:35.548010+0000","last_undegraded":"2024-09-10T09:37:35.548010+0000","last_fullsized":"2024-09-10T09:37:35.548010+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:13:27.120208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"94'1","reported_seq":45,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.528018+0000","last_change":"2024-09-10T09:35:25.854569+0000","last_active":"2024-09-10T09:37:07.528018+0000","last_peered":"2024-09-10T09:37:07.528018+0000","last_clean":"2024-09-10T09:37:07.528018+0000","last_became_active":"2024-09-10T09:35:25.854152+0000","last_became_peered":"2024-09-10T09:35:25.854152+0000","last_unstale":"2024-09-10T09:37:07.528018+0000","last_undegraded":"2024-09-10T09:37:07.528018+0000","last_fullsized":"2024-09-10T09:37:07.528018+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:27:15.584707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.529957+0000","last_change":"2024-09-10T09:35:37.270517+0000","last_active":"2024-09-10T09:37:07.529957+0000","last_peered":"2024-09-10T09:37:07.529957+0000","last_clean":"2024-09-10T09:37:07.529957+0000","last_became_active":"2024-09-10T09:35:37.269793+0000","last_became_peered":"2024-09-10T09:35:37.269793+0000","last_unstale":"2024-09-10T09:37:07.529957+0000","last_undegraded":"2024-09-10T09:37:07.529957+0000","last_fullsized":"2024-09-10T09:37:07.529957+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:28:58.272848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"96'7","reported_seq":52,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.556473+0000","last_change":"2024-09-10T09:37:07.762585+0000","last_active":"2024-09-10T09:37:35.556473+0000","last_peered":"2024-09-10T09:37:35.556473+0000","last_clean":"2024-09-10T09:37:35.556473+0000","last_became_active":"2024-09-10T09:37:07.762250+0000","last_became_peered":"2024-09-10T09:37:07.762250+0000","last_unstale":"2024-09-10T09:37:35.556473+0000","last_undegraded":"2024-09-10T09:37:35.556473+0000","last_fullsized":"2024-09-10T09:37:35.556473+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:38:01.751512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"96'8","reported_seq":184,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:38:22.676195+0000","last_change":"2024-09-10T09:35:25.531062+0000","last_active":"2024-09-10T09:38:22.676195+0000","last_peered":"2024-09-10T09:38:22.676195+0000","last_clean":"2024-09-10T09:38:22.676195+0000","last_became_active":"2024-09-10T09:35:25.530005+0000","last_became_peered":"2024-09-10T09:35:25.530005+0000","last_unstale":"2024-09-10T09:38:22.676195+0000","last_undegraded":"2024-09-10T09:38:22.676195+0000","last_fullsized":"2024-09-10T09:38:22.676195+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:42:35.330312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1433,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":133,"num_read_kb":4,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.538766+0000","last_change":"2024-09-10T09:35:36.623099+0000","last_active":"2024-09-10T09:37:07.538766+0000","last_peered":"2024-09-10T09:37:07.538766+0000","last_clean":"2024-09-10T09:37:07.538766+0000","last_became_active":"2024-09-10T09:35:36.622716+0000","last_became_peered":"2024-09-10T09:35:36.622716+0000","last_unstale":"2024-09-10T09:37:07.538766+0000","last_undegraded":"2024-09-10T09:37:07.538766+0000","last_fullsized":"2024-09-10T09:37:07.538766+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:10:38.495164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"96'8","reported_seq":53,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.543805+0000","last_change":"2024-09-10T09:36:06.769405+0000","last_active":"2024-09-10T09:37:35.543805+0000","last_peered":"2024-09-10T09:37:35.543805+0000","last_clean":"2024-09-10T09:37:35.543805+0000","last_became_active":"2024-09-10T09:36:06.769042+0000","last_became_peered":"2024-09-10T09:36:06.769042+0000","last_unstale":"2024-09-10T09:37:35.543805+0000","last_undegraded":"2024-09-10T09:37:35.543805+0000","last_fullsized":"2024-09-10T09:37:35.543805+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:01:49.340765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,8],"acting":[5,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"94'5","reported_seq":184,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:38:26.681235+0000","last_change":"2024-09-10T09:35:25.851430+0000","last_active":"2024-09-10T09:38:26.681235+0000","last_peered":"2024-09-10T09:38:26.681235+0000","last_clean":"2024-09-10T09:38:26.681235+0000","last_became_active":"2024-09-10T09:35:25.851067+0000","last_became_peered":"2024-09-10T09:35:25.851067+0000","last_unstale":"2024-09-10T09:38:26.681235+0000","last_undegraded":"2024-09-10T09:38:26.681235+0000","last_fullsized":"2024-09-10T09:38:26.681235+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:55:58.756207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":135,"num_read_kb":135,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.538676+0000","last_change":"2024-09-10T09:35:37.270361+0000","last_active":"2024-09-10T09:37:07.538676+0000","last_peered":"2024-09-10T09:37:07.538676+0000","last_clean":"2024-09-10T09:37:07.538676+0000","last_became_active":"2024-09-10T09:35:37.269632+0000","last_became_peered":"2024-09-10T09:35:37.269632+0000","last_unstale":"2024-09-10T09:37:07.538676+0000","last_undegraded":"2024-09-10T09:37:07.538676+0000","last_fullsized":"2024-09-10T09:37:07.538676+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:50:34.360497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"96'5","reported_seq":44,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.548275+0000","last_change":"2024-09-10T09:35:28.571064+0000","last_active":"2024-09-10T09:37:35.548275+0000","last_peered":"2024-09-10T09:37:35.548275+0000","last_clean":"2024-09-10T09:37:35.548275+0000","last_became_active":"2024-09-10T09:35:28.570691+0000","last_became_peered":"2024-09-10T09:35:28.570691+0000","last_unstale":"2024-09-10T09:37:35.548275+0000","last_undegraded":"2024-09-10T09:37:35.548275+0000","last_fullsized":"2024-09-10T09:37:35.548275+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:58:33.162501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"94'4","reported_seq":57,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.572759+0000","last_change":"2024-09-10T09:37:07.572247+0000","last_active":"2024-09-10T09:37:07.572759+0000","last_peered":"2024-09-10T09:37:07.572759+0000","last_clean":"2024-09-10T09:37:07.572759+0000","last_became_active":"2024-09-10T09:37:07.569410+0000","last_became_peered":"2024-09-10T09:37:07.569410+0000","last_unstale":"2024-09-10T09:37:07.572759+0000","last_undegraded":"2024-09-10T09:37:07.572759+0000","last_fullsized":"2024-09-10T09:37:07.572759+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:05:09.037084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,4],"acting":[7,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.532723+0000","last_change":"2024-09-10T09:35:37.270358+0000","last_active":"2024-09-10T09:37:07.532723+0000","last_peered":"2024-09-10T09:37:07.532723+0000","last_clean":"2024-09-10T09:37:07.532723+0000","last_became_active":"2024-09-10T09:35:37.269822+0000","last_became_peered":"2024-09-10T09:35:37.269822+0000","last_unstale":"2024-09-10T09:37:07.532723+0000","last_undegraded":"2024-09-10T09:37:07.532723+0000","last_fullsized":"2024-09-10T09:37:07.532723+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:19:48.932596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"96'8","reported_seq":47,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.553803+0000","last_change":"2024-09-10T09:35:28.576244+0000","last_active":"2024-09-10T09:37:35.553803+0000","last_peered":"2024-09-10T09:37:35.553803+0000","last_clean":"2024-09-10T09:37:35.553803+0000","last_became_active":"2024-09-10T09:35:28.571801+0000","last_became_peered":"2024-09-10T09:35:28.571801+0000","last_unstale":"2024-09-10T09:37:35.553803+0000","last_undegraded":"2024-09-10T09:37:35.553803+0000","last_fullsized":"2024-09-10T09:37:35.553803+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:11:21.742220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"94'6","reported_seq":52,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.568754+0000","last_change":"2024-09-10T09:35:25.537118+0000","last_active":"2024-09-10T09:37:07.568754+0000","last_peered":"2024-09-10T09:37:07.568754+0000","last_clean":"2024-09-10T09:37:07.568754+0000","last_became_active":"2024-09-10T09:35:25.536718+0000","last_became_peered":"2024-09-10T09:35:25.536718+0000","last_unstale":"2024-09-10T09:37:07.568754+0000","last_undegraded":"2024-09-10T09:37:07.568754+0000","last_fullsized":"2024-09-10T09:37:07.568754+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:16:32.837476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":361,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.528720+0000","last_change":"2024-09-10T09:34:07.350825+0000","last_active":"2024-09-10T09:37:07.528720+0000","last_peered":"2024-09-10T09:37:07.528720+0000","last_clean":"2024-09-10T09:37:07.528720+0000","last_became_active":"2024-09-10T09:34:07.350325+0000","last_became_peered":"2024-09-10T09:34:07.350325+0000","last_unstale":"2024-09-10T09:37:07.528720+0000","last_undegraded":"2024-09-10T09:37:07.528720+0000","last_fullsized":"2024-09-10T09:37:07.528720+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:30:26.408262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:30:26.408262+0000","last_clean_scrub_stamp":"2024-09-10T09:30:26.408262+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:58:04.920443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"94'1","reported_seq":30,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.529308+0000","last_change":"2024-09-10T09:35:36.632941+0000","last_active":"2024-09-10T09:37:07.529308+0000","last_peered":"2024-09-10T09:37:07.529308+0000","last_clean":"2024-09-10T09:37:07.529308+0000","last_became_active":"2024-09-10T09:35:36.632527+0000","last_became_peered":"2024-09-10T09:35:36.632527+0000","last_unstale":"2024-09-10T09:37:07.529308+0000","last_undegraded":"2024-09-10T09:37:07.529308+0000","last_fullsized":"2024-09-10T09:37:07.529308+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:33:21.733772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"96'7","reported_seq":44,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.556036+0000","last_change":"2024-09-10T09:35:28.572010+0000","last_active":"2024-09-10T09:37:35.556036+0000","last_peered":"2024-09-10T09:37:35.556036+0000","last_clean":"2024-09-10T09:37:35.556036+0000","last_became_active":"2024-09-10T09:35:28.571623+0000","last_became_peered":"2024-09-10T09:35:28.571623+0000","last_unstale":"2024-09-10T09:37:35.556036+0000","last_undegraded":"2024-09-10T09:37:35.556036+0000","last_fullsized":"2024-09-10T09:37:35.556036+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:00:49.328663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"94'5","reported_seq":183,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:38:24.679400+0000","last_change":"2024-09-10T09:35:25.990328+0000","last_active":"2024-09-10T09:38:24.679400+0000","last_peered":"2024-09-10T09:38:24.679400+0000","last_clean":"2024-09-10T09:38:24.679400+0000","last_became_active":"2024-09-10T09:35:25.989946+0000","last_became_peered":"2024-09-10T09:35:25.989946+0000","last_unstale":"2024-09-10T09:38:24.679400+0000","last_undegraded":"2024-09-10T09:38:24.679400+0000","last_fullsized":"2024-09-10T09:38:24.679400+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:51:16.069913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":134,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.574499+0000","last_change":"2024-09-10T09:35:36.619013+0000","last_active":"2024-09-10T09:37:07.574499+0000","last_peered":"2024-09-10T09:37:07.574499+0000","last_clean":"2024-09-10T09:37:07.574499+0000","last_became_active":"2024-09-10T09:35:36.618506+0000","last_became_peered":"2024-09-10T09:35:36.618506+0000","last_unstale":"2024-09-10T09:37:07.574499+0000","last_undegraded":"2024-09-10T09:37:07.574499+0000","last_fullsized":"2024-09-10T09:37:07.574499+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:57:53.580569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"96'6","reported_seq":49,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.553955+0000","last_change":"2024-09-10T09:36:06.768573+0000","last_active":"2024-09-10T09:37:35.553955+0000","last_peered":"2024-09-10T09:37:35.553955+0000","last_clean":"2024-09-10T09:37:35.553955+0000","last_became_active":"2024-09-10T09:36:06.768228+0000","last_became_peered":"2024-09-10T09:36:06.768228+0000","last_unstale":"2024-09-10T09:37:35.553955+0000","last_undegraded":"2024-09-10T09:37:35.553955+0000","last_fullsized":"2024-09-10T09:37:35.553955+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:57:38.310713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"94'3","reported_seq":48,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.533010+0000","last_change":"2024-09-10T09:35:25.528795+0000","last_active":"2024-09-10T09:37:07.533010+0000","last_peered":"2024-09-10T09:37:07.533010+0000","last_clean":"2024-09-10T09:37:07.533010+0000","last_became_active":"2024-09-10T09:35:25.528475+0000","last_became_peered":"2024-09-10T09:35:25.528475+0000","last_unstale":"2024-09-10T09:37:07.533010+0000","last_undegraded":"2024-09-10T09:37:07.533010+0000","last_fullsized":"2024-09-10T09:37:07.533010+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:16:06.999186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.761758+0000","last_change":"2024-09-10T09:35:36.633109+0000","last_active":"2024-09-10T09:37:07.761758+0000","last_peered":"2024-09-10T09:37:07.761758+0000","last_clean":"2024-09-10T09:37:07.761758+0000","last_became_active":"2024-09-10T09:35:36.632682+0000","last_became_peered":"2024-09-10T09:35:36.632682+0000","last_unstale":"2024-09-10T09:37:07.761758+0000","last_undegraded":"2024-09-10T09:37:07.761758+0000","last_fullsized":"2024-09-10T09:37:07.761758+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:51:14.372649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"96'9","reported_seq":46,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.552249+0000","last_change":"2024-09-10T09:35:28.562103+0000","last_active":"2024-09-10T09:37:35.552249+0000","last_peered":"2024-09-10T09:37:35.552249+0000","last_clean":"2024-09-10T09:37:35.552249+0000","last_became_active":"2024-09-10T09:35:28.561765+0000","last_became_peered":"2024-09-10T09:35:28.561765+0000","last_unstale":"2024-09-10T09:37:35.552249+0000","last_undegraded":"2024-09-10T09:37:35.552249+0000","last_fullsized":"2024-09-10T09:37:35.552249+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:35:35.400569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"94'5","reported_seq":49,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.539243+0000","last_change":"2024-09-10T09:35:25.538536+0000","last_active":"2024-09-10T09:37:07.539243+0000","last_peered":"2024-09-10T09:37:07.539243+0000","last_clean":"2024-09-10T09:37:07.539243+0000","last_became_active":"2024-09-10T09:35:25.537944+0000","last_became_peered":"2024-09-10T09:35:25.537944+0000","last_unstale":"2024-09-10T09:37:07.539243+0000","last_undegraded":"2024-09-10T09:37:07.539243+0000","last_fullsized":"2024-09-10T09:37:07.539243+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:01:19.385381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":17,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.529928+0000","last_change":"2024-09-10T09:36:06.769534+0000","last_active":"2024-09-10T09:37:07.529928+0000","last_peered":"2024-09-10T09:37:07.529928+0000","last_clean":"2024-09-10T09:37:07.529928+0000","last_became_active":"2024-09-10T09:36:06.769015+0000","last_became_peered":"2024-09-10T09:36:06.769015+0000","last_unstale":"2024-09-10T09:37:07.529928+0000","last_undegraded":"2024-09-10T09:37:07.529928+0000","last_fullsized":"2024-09-10T09:37:07.529928+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:19:26.369737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7,1],"acting":[8,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"96'5","reported_seq":44,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.539962+0000","last_change":"2024-09-10T09:35:28.566735+0000","last_active":"2024-09-10T09:37:35.539962+0000","last_peered":"2024-09-10T09:37:35.539962+0000","last_clean":"2024-09-10T09:37:35.539962+0000","last_became_active":"2024-09-10T09:35:28.566400+0000","last_became_peered":"2024-09-10T09:35:28.566400+0000","last_unstale":"2024-09-10T09:37:35.539962+0000","last_undegraded":"2024-09-10T09:37:35.539962+0000","last_fullsized":"2024-09-10T09:37:35.539962+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:18:53.337816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"94'2","reported_seq":47,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.530569+0000","last_change":"2024-09-10T09:35:25.535783+0000","last_active":"2024-09-10T09:37:07.530569+0000","last_peered":"2024-09-10T09:37:07.530569+0000","last_clean":"2024-09-10T09:37:07.530569+0000","last_became_active":"2024-09-10T09:35:25.535411+0000","last_became_peered":"2024-09-10T09:35:25.535411+0000","last_unstale":"2024-09-10T09:37:07.530569+0000","last_undegraded":"2024-09-10T09:37:07.530569+0000","last_fullsized":"2024-09-10T09:37:07.530569+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:48:09.239083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.531208+0000","last_change":"2024-09-10T09:35:36.619380+0000","last_active":"2024-09-10T09:37:07.531208+0000","last_peered":"2024-09-10T09:37:07.531208+0000","last_clean":"2024-09-10T09:37:07.531208+0000","last_became_active":"2024-09-10T09:35:36.618953+0000","last_became_peered":"2024-09-10T09:35:36.618953+0000","last_unstale":"2024-09-10T09:37:07.531208+0000","last_undegraded":"2024-09-10T09:37:07.531208+0000","last_fullsized":"2024-09-10T09:37:07.531208+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:40:25.181408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"96'6","reported_seq":45,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.546928+0000","last_change":"2024-09-10T09:35:28.571396+0000","last_active":"2024-09-10T09:37:35.546928+0000","last_peered":"2024-09-10T09:37:35.546928+0000","last_clean":"2024-09-10T09:37:35.546928+0000","last_became_active":"2024-09-10T09:35:28.571063+0000","last_became_peered":"2024-09-10T09:35:28.571063+0000","last_unstale":"2024-09-10T09:37:35.546928+0000","last_undegraded":"2024-09-10T09:37:35.546928+0000","last_fullsized":"2024-09-10T09:37:35.546928+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:38:17.965133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"94'17","reported_seq":62,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.530060+0000","last_change":"2024-09-10T09:35:25.851773+0000","last_active":"2024-09-10T09:37:07.530060+0000","last_peered":"2024-09-10T09:37:07.530060+0000","last_clean":"2024-09-10T09:37:07.530060+0000","last_became_active":"2024-09-10T09:35:25.851384+0000","last_became_peered":"2024-09-10T09:35:25.851384+0000","last_unstale":"2024-09-10T09:37:07.530060+0000","last_undegraded":"2024-09-10T09:37:07.530060+0000","last_fullsized":"2024-09-10T09:37:07.530060+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:03:32.338643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.539186+0000","last_change":"2024-09-10T09:35:36.620971+0000","last_active":"2024-09-10T09:37:07.539186+0000","last_peered":"2024-09-10T09:37:07.539186+0000","last_clean":"2024-09-10T09:37:07.539186+0000","last_became_active":"2024-09-10T09:35:36.620343+0000","last_became_peered":"2024-09-10T09:35:36.620343+0000","last_unstale":"2024-09-10T09:37:07.539186+0000","last_undegraded":"2024-09-10T09:37:07.539186+0000","last_fullsized":"2024-09-10T09:37:07.539186+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:17:19.517882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"96'6","reported_seq":43,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.553818+0000","last_change":"2024-09-10T09:35:28.565643+0000","last_active":"2024-09-10T09:37:35.553818+0000","last_peered":"2024-09-10T09:37:35.553818+0000","last_clean":"2024-09-10T09:37:35.553818+0000","last_became_active":"2024-09-10T09:35:28.564217+0000","last_became_peered":"2024-09-10T09:35:28.564217+0000","last_unstale":"2024-09-10T09:37:35.553818+0000","last_undegraded":"2024-09-10T09:37:35.553818+0000","last_fullsized":"2024-09-10T09:37:35.553818+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:53:34.562690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"94'5","reported_seq":184,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:38:26.681823+0000","last_change":"2024-09-10T09:35:25.534167+0000","last_active":"2024-09-10T09:38:26.681823+0000","last_peered":"2024-09-10T09:38:26.681823+0000","last_clean":"2024-09-10T09:38:26.681823+0000","last_became_active":"2024-09-10T09:35:25.533719+0000","last_became_peered":"2024-09-10T09:35:25.533719+0000","last_unstale":"2024-09-10T09:38:26.681823+0000","last_undegraded":"2024-09-10T09:38:26.681823+0000","last_fullsized":"2024-09-10T09:38:26.681823+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:00:03.537355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":137,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.527733+0000","last_change":"2024-09-10T09:35:36.628716+0000","last_active":"2024-09-10T09:37:07.527733+0000","last_peered":"2024-09-10T09:37:07.527733+0000","last_clean":"2024-09-10T09:37:07.527733+0000","last_became_active":"2024-09-10T09:35:36.628085+0000","last_became_peered":"2024-09-10T09:35:36.628085+0000","last_unstale":"2024-09-10T09:37:07.527733+0000","last_undegraded":"2024-09-10T09:37:07.527733+0000","last_fullsized":"2024-09-10T09:37:07.527733+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:45:01.279403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"96'15","reported_seq":52,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.557076+0000","last_change":"2024-09-10T09:35:28.566701+0000","last_active":"2024-09-10T09:37:35.557076+0000","last_peered":"2024-09-10T09:37:35.557076+0000","last_clean":"2024-09-10T09:37:35.557076+0000","last_became_active":"2024-09-10T09:35:28.566391+0000","last_became_peered":"2024-09-10T09:35:28.566391+0000","last_unstale":"2024-09-10T09:37:35.557076+0000","last_undegraded":"2024-09-10T09:37:35.557076+0000","last_fullsized":"2024-09-10T09:37:35.557076+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:57:59.918706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"94'5","reported_seq":50,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.531896+0000","last_change":"2024-09-10T09:35:25.532510+0000","last_active":"2024-09-10T09:37:07.531896+0000","last_peered":"2024-09-10T09:37:07.531896+0000","last_clean":"2024-09-10T09:37:07.531896+0000","last_became_active":"2024-09-10T09:35:25.532193+0000","last_became_peered":"2024-09-10T09:35:25.532193+0000","last_unstale":"2024-09-10T09:37:07.531896+0000","last_undegraded":"2024-09-10T09:37:07.531896+0000","last_fullsized":"2024-09-10T09:37:07.531896+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:17:24.685178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.526424+0000","last_change":"2024-09-10T09:35:36.630742+0000","last_active":"2024-09-10T09:37:07.526424+0000","last_peered":"2024-09-10T09:37:07.526424+0000","last_clean":"2024-09-10T09:37:07.526424+0000","last_became_active":"2024-09-10T09:35:36.630335+0000","last_became_peered":"2024-09-10T09:35:36.630335+0000","last_unstale":"2024-09-10T09:37:07.526424+0000","last_undegraded":"2024-09-10T09:37:07.526424+0000","last_fullsized":"2024-09-10T09:37:07.526424+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:29:12.029390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"96'9","reported_seq":48,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.552455+0000","last_change":"2024-09-10T09:35:28.581817+0000","last_active":"2024-09-10T09:37:35.552455+0000","last_peered":"2024-09-10T09:37:35.552455+0000","last_clean":"2024-09-10T09:37:35.552455+0000","last_became_active":"2024-09-10T09:35:28.580316+0000","last_became_peered":"2024-09-10T09:35:28.580316+0000","last_unstale":"2024-09-10T09:37:35.552455+0000","last_undegraded":"2024-09-10T09:37:35.552455+0000","last_fullsized":"2024-09-10T09:37:35.552455+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:04:08.249564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"94'9","reported_seq":55,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.528127+0000","last_change":"2024-09-10T09:35:25.537474+0000","last_active":"2024-09-10T09:37:07.528127+0000","last_peered":"2024-09-10T09:37:07.528127+0000","last_clean":"2024-09-10T09:37:07.528127+0000","last_became_active":"2024-09-10T09:35:25.537096+0000","last_became_peered":"2024-09-10T09:35:25.537096+0000","last_unstale":"2024-09-10T09:37:07.528127+0000","last_undegraded":"2024-09-10T09:37:07.528127+0000","last_fullsized":"2024-09-10T09:37:07.528127+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:13:06.008458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.532316+0000","last_change":"2024-09-10T09:35:36.622788+0000","last_active":"2024-09-10T09:37:07.532316+0000","last_peered":"2024-09-10T09:37:07.532316+0000","last_clean":"2024-09-10T09:37:07.532316+0000","last_became_active":"2024-09-10T09:35:36.622377+0000","last_became_peered":"2024-09-10T09:35:36.622377+0000","last_unstale":"2024-09-10T09:37:07.532316+0000","last_undegraded":"2024-09-10T09:37:07.532316+0000","last_fullsized":"2024-09-10T09:37:07.532316+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:29:56.680864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"96'6","reported_seq":45,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.534371+0000","last_change":"2024-09-10T09:35:28.564529+0000","last_active":"2024-09-10T09:37:35.534371+0000","last_peered":"2024-09-10T09:37:35.534371+0000","last_clean":"2024-09-10T09:37:35.534371+0000","last_became_active":"2024-09-10T09:35:28.564223+0000","last_became_peered":"2024-09-10T09:35:28.564223+0000","last_unstale":"2024-09-10T09:37:35.534371+0000","last_undegraded":"2024-09-10T09:37:35.534371+0000","last_fullsized":"2024-09-10T09:37:35.534371+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:21:24.296578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"96'42","reported_seq":357,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:38:23.677498+0000","last_change":"2024-09-10T09:35:25.534331+0000","last_active":"2024-09-10T09:38:23.677498+0000","last_peered":"2024-09-10T09:38:23.677498+0000","last_clean":"2024-09-10T09:38:23.677498+0000","last_became_active":"2024-09-10T09:35:25.533985+0000","last_became_peered":"2024-09-10T09:35:25.533985+0000","last_unstale":"2024-09-10T09:38:23.677498+0000","last_undegraded":"2024-09-10T09:38:23.677498+0000","last_fullsized":"2024-09-10T09:38:23.677498+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":42,"log_dups_size":0,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:14:37.050650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70718,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":271,"num_read_kb":205,"num_write":44,"num_write_kb":89,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.527925+0000","last_change":"2024-09-10T09:35:36.632835+0000","last_active":"2024-09-10T09:37:07.527925+0000","last_peered":"2024-09-10T09:37:07.527925+0000","last_clean":"2024-09-10T09:37:07.527925+0000","last_became_active":"2024-09-10T09:35:36.630962+0000","last_became_peered":"2024-09-10T09:35:36.630962+0000","last_unstale":"2024-09-10T09:37:07.527925+0000","last_undegraded":"2024-09-10T09:37:07.527925+0000","last_fullsized":"2024-09-10T09:37:07.527925+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:59:27.613721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"96'11","reported_seq":50,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.546541+0000","last_change":"2024-09-10T09:35:28.567292+0000","last_active":"2024-09-10T09:37:35.546541+0000","last_peered":"2024-09-10T09:37:35.546541+0000","last_clean":"2024-09-10T09:37:35.546541+0000","last_became_active":"2024-09-10T09:35:28.566973+0000","last_became_peered":"2024-09-10T09:35:28.566973+0000","last_unstale":"2024-09-10T09:37:35.546541+0000","last_undegraded":"2024-09-10T09:37:35.546541+0000","last_fullsized":"2024-09-10T09:37:35.546541+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:05:37.251933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"94'5","reported_seq":50,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.532787+0000","last_change":"2024-09-10T09:35:25.852171+0000","last_active":"2024-09-10T09:37:07.532787+0000","last_peered":"2024-09-10T09:37:07.532787+0000","last_clean":"2024-09-10T09:37:07.532787+0000","last_became_active":"2024-09-10T09:35:25.851580+0000","last_became_peered":"2024-09-10T09:35:25.851580+0000","last_unstale":"2024-09-10T09:37:07.532787+0000","last_undegraded":"2024-09-10T09:37:07.532787+0000","last_fullsized":"2024-09-10T09:37:07.532787+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:03.263152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.568336+0000","last_change":"2024-09-10T09:35:36.629212+0000","last_active":"2024-09-10T09:37:07.568336+0000","last_peered":"2024-09-10T09:37:07.568336+0000","last_clean":"2024-09-10T09:37:07.568336+0000","last_became_active":"2024-09-10T09:35:36.628753+0000","last_became_peered":"2024-09-10T09:35:36.628753+0000","last_unstale":"2024-09-10T09:37:07.568336+0000","last_undegraded":"2024-09-10T09:37:07.568336+0000","last_fullsized":"2024-09-10T09:37:07.568336+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:54:45.311361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"96'9","reported_seq":46,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.553075+0000","last_change":"2024-09-10T09:35:28.572497+0000","last_active":"2024-09-10T09:37:35.553075+0000","last_peered":"2024-09-10T09:37:35.553075+0000","last_clean":"2024-09-10T09:37:35.553075+0000","last_became_active":"2024-09-10T09:35:28.569238+0000","last_became_peered":"2024-09-10T09:35:28.569238+0000","last_unstale":"2024-09-10T09:37:35.553075+0000","last_undegraded":"2024-09-10T09:37:35.553075+0000","last_fullsized":"2024-09-10T09:37:35.553075+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:38:47.189209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"94'4","reported_seq":47,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.529569+0000","last_change":"2024-09-10T09:35:25.536369+0000","last_active":"2024-09-10T09:37:07.529569+0000","last_peered":"2024-09-10T09:37:07.529569+0000","last_clean":"2024-09-10T09:37:07.529569+0000","last_became_active":"2024-09-10T09:35:25.535607+0000","last_became_peered":"2024-09-10T09:35:25.535607+0000","last_unstale":"2024-09-10T09:37:07.529569+0000","last_undegraded":"2024-09-10T09:37:07.529569+0000","last_fullsized":"2024-09-10T09:37:07.529569+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:49:18.621934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.527047+0000","last_change":"2024-09-10T09:35:36.627977+0000","last_active":"2024-09-10T09:37:07.527047+0000","last_peered":"2024-09-10T09:37:07.527047+0000","last_clean":"2024-09-10T09:37:07.527047+0000","last_became_active":"2024-09-10T09:35:36.626942+0000","last_became_peered":"2024-09-10T09:35:36.626942+0000","last_unstale":"2024-09-10T09:37:07.527047+0000","last_undegraded":"2024-09-10T09:37:07.527047+0000","last_fullsized":"2024-09-10T09:37:07.527047+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:37:18.209424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"96'10","reported_seq":49,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.553165+0000","last_change":"2024-09-10T09:35:28.562670+0000","last_active":"2024-09-10T09:37:35.553165+0000","last_peered":"2024-09-10T09:37:35.553165+0000","last_clean":"2024-09-10T09:37:35.553165+0000","last_became_active":"2024-09-10T09:35:28.562028+0000","last_became_peered":"2024-09-10T09:35:28.562028+0000","last_unstale":"2024-09-10T09:37:35.553165+0000","last_undegraded":"2024-09-10T09:37:35.553165+0000","last_fullsized":"2024-09-10T09:37:35.553165+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:37:15.176050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"96'5","reported_seq":308,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:38:23.677506+0000","last_change":"2024-09-10T09:35:25.529861+0000","last_active":"2024-09-10T09:38:23.677506+0000","last_peered":"2024-09-10T09:38:23.677506+0000","last_clean":"2024-09-10T09:38:23.677506+0000","last_became_active":"2024-09-10T09:35:25.529475+0000","last_became_peered":"2024-09-10T09:35:25.529475+0000","last_unstale":"2024-09-10T09:38:23.677506+0000","last_undegraded":"2024-09-10T09:38:23.677506+0000","last_fullsized":"2024-09-10T09:38:23.677506+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:44:27.086509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":259,"num_read_kb":259,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.524306+0000","last_change":"2024-09-10T09:35:36.615358+0000","last_active":"2024-09-10T09:37:07.524306+0000","last_peered":"2024-09-10T09:37:07.524306+0000","last_clean":"2024-09-10T09:37:07.524306+0000","last_became_active":"2024-09-10T09:35:36.614901+0000","last_became_peered":"2024-09-10T09:35:36.614901+0000","last_unstale":"2024-09-10T09:37:07.524306+0000","last_undegraded":"2024-09-10T09:37:07.524306+0000","last_fullsized":"2024-09-10T09:37:07.524306+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:56:54.248266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.761302+0000","last_change":"2024-09-10T09:35:36.629021+0000","last_active":"2024-09-10T09:37:07.761302+0000","last_peered":"2024-09-10T09:37:07.761302+0000","last_clean":"2024-09-10T09:37:07.761302+0000","last_became_active":"2024-09-10T09:35:36.628159+0000","last_became_peered":"2024-09-10T09:35:36.628159+0000","last_unstale":"2024-09-10T09:37:07.761302+0000","last_undegraded":"2024-09-10T09:37:07.761302+0000","last_fullsized":"2024-09-10T09:37:07.761302+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:50:48.615236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"96'11","reported_seq":48,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.545215+0000","last_change":"2024-09-10T09:35:28.579333+0000","last_active":"2024-09-10T09:37:35.545215+0000","last_peered":"2024-09-10T09:37:35.545215+0000","last_clean":"2024-09-10T09:37:35.545215+0000","last_became_active":"2024-09-10T09:35:28.578440+0000","last_became_peered":"2024-09-10T09:35:28.578440+0000","last_unstale":"2024-09-10T09:37:35.545215+0000","last_undegraded":"2024-09-10T09:37:35.545215+0000","last_fullsized":"2024-09-10T09:37:35.545215+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:07:28.701938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"94'3","reported_seq":48,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.531063+0000","last_change":"2024-09-10T09:35:25.530857+0000","last_active":"2024-09-10T09:37:07.531063+0000","last_peered":"2024-09-10T09:37:07.531063+0000","last_clean":"2024-09-10T09:37:07.531063+0000","last_became_active":"2024-09-10T09:35:25.530492+0000","last_became_peered":"2024-09-10T09:35:25.530492+0000","last_unstale":"2024-09-10T09:37:07.531063+0000","last_undegraded":"2024-09-10T09:37:07.531063+0000","last_fullsized":"2024-09-10T09:37:07.531063+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:07:21.726598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.526432+0000","last_change":"2024-09-10T09:36:06.770408+0000","last_active":"2024-09-10T09:37:07.526432+0000","last_peered":"2024-09-10T09:37:07.526432+0000","last_clean":"2024-09-10T09:37:07.526432+0000","last_became_active":"2024-09-10T09:36:06.769915+0000","last_became_peered":"2024-09-10T09:36:06.769915+0000","last_unstale":"2024-09-10T09:37:07.526432+0000","last_undegraded":"2024-09-10T09:37:07.526432+0000","last_fullsized":"2024-09-10T09:37:07.526432+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:56:09.795537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8,0],"acting":[11,4,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"96'8","reported_seq":47,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.543491+0000","last_change":"2024-09-10T09:35:28.570804+0000","last_active":"2024-09-10T09:37:35.543491+0000","last_peered":"2024-09-10T09:37:35.543491+0000","last_clean":"2024-09-10T09:37:35.543491+0000","last_became_active":"2024-09-10T09:35:28.570490+0000","last_became_peered":"2024-09-10T09:35:28.570490+0000","last_unstale":"2024-09-10T09:37:35.543491+0000","last_undegraded":"2024-09-10T09:37:35.543491+0000","last_fullsized":"2024-09-10T09:37:35.543491+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:13:38.999625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"94'3","reported_seq":47,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.538446+0000","last_change":"2024-09-10T09:35:25.536347+0000","last_active":"2024-09-10T09:37:07.538446+0000","last_peered":"2024-09-10T09:37:07.538446+0000","last_clean":"2024-09-10T09:37:07.538446+0000","last_became_active":"2024-09-10T09:35:25.535964+0000","last_became_peered":"2024-09-10T09:35:25.535964+0000","last_unstale":"2024-09-10T09:37:07.538446+0000","last_undegraded":"2024-09-10T09:37:07.538446+0000","last_fullsized":"2024-09-10T09:37:07.538446+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:36:49.035666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"94'7","reported_seq":51,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.525812+0000","last_change":"2024-09-10T09:35:25.532148+0000","last_active":"2024-09-10T09:37:07.525812+0000","last_peered":"2024-09-10T09:37:07.525812+0000","last_clean":"2024-09-10T09:37:07.525812+0000","last_became_active":"2024-09-10T09:35:25.531781+0000","last_became_peered":"2024-09-10T09:35:25.531781+0000","last_unstale":"2024-09-10T09:37:07.525812+0000","last_undegraded":"2024-09-10T09:37:07.525812+0000","last_fullsized":"2024-09-10T09:37:07.525812+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:52:46.740342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"96'8","reported_seq":47,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.549412+0000","last_change":"2024-09-10T09:35:28.578336+0000","last_active":"2024-09-10T09:37:35.549412+0000","last_peered":"2024-09-10T09:37:35.549412+0000","last_clean":"2024-09-10T09:37:35.549412+0000","last_became_active":"2024-09-10T09:35:28.577899+0000","last_became_peered":"2024-09-10T09:35:28.577899+0000","last_unstale":"2024-09-10T09:37:35.549412+0000","last_undegraded":"2024-09-10T09:37:35.549412+0000","last_fullsized":"2024-09-10T09:37:35.549412+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:13:04.478195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.528122+0000","last_change":"2024-09-10T09:35:36.630918+0000","last_active":"2024-09-10T09:37:07.528122+0000","last_peered":"2024-09-10T09:37:07.528122+0000","last_clean":"2024-09-10T09:37:07.528122+0000","last_became_active":"2024-09-10T09:35:36.630308+0000","last_became_peered":"2024-09-10T09:35:36.630308+0000","last_unstale":"2024-09-10T09:37:07.528122+0000","last_undegraded":"2024-09-10T09:37:07.528122+0000","last_fullsized":"2024-09-10T09:37:07.528122+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:25:44.954970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"96'7","reported_seq":196,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:38:22.676633+0000","last_change":"2024-09-10T09:37:07.574386+0000","last_active":"2024-09-10T09:38:22.676633+0000","last_peered":"2024-09-10T09:38:22.676633+0000","last_clean":"2024-09-10T09:38:22.676633+0000","last_became_active":"2024-09-10T09:37:07.569370+0000","last_became_peered":"2024-09-10T09:37:07.569370+0000","last_unstale":"2024-09-10T09:38:22.676633+0000","last_undegraded":"2024-09-10T09:38:22.676633+0000","last_fullsized":"2024-09-10T09:38:22.676633+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:07:23.014789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1156,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":128,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":1015,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"96'7","reported_seq":46,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.543648+0000","last_change":"2024-09-10T09:35:28.567915+0000","last_active":"2024-09-10T09:37:35.543648+0000","last_peered":"2024-09-10T09:37:35.543648+0000","last_clean":"2024-09-10T09:37:35.543648+0000","last_became_active":"2024-09-10T09:35:28.567416+0000","last_became_peered":"2024-09-10T09:35:28.567416+0000","last_unstale":"2024-09-10T09:37:35.543648+0000","last_undegraded":"2024-09-10T09:37:35.543648+0000","last_fullsized":"2024-09-10T09:37:35.543648+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:07:25.704080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.528686+0000","last_change":"2024-09-10T09:35:36.615386+0000","last_active":"2024-09-10T09:37:07.528686+0000","last_peered":"2024-09-10T09:37:07.528686+0000","last_clean":"2024-09-10T09:37:07.528686+0000","last_became_active":"2024-09-10T09:35:36.614967+0000","last_became_peered":"2024-09-10T09:35:36.614967+0000","last_unstale":"2024-09-10T09:37:07.528686+0000","last_undegraded":"2024-09-10T09:37:07.528686+0000","last_fullsized":"2024-09-10T09:37:07.528686+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:47.620729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"96'9","reported_seq":48,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.544481+0000","last_change":"2024-09-10T09:35:28.564411+0000","last_active":"2024-09-10T09:37:35.544481+0000","last_peered":"2024-09-10T09:37:35.544481+0000","last_clean":"2024-09-10T09:37:35.544481+0000","last_became_active":"2024-09-10T09:35:28.564034+0000","last_became_peered":"2024-09-10T09:35:28.564034+0000","last_unstale":"2024-09-10T09:37:35.544481+0000","last_undegraded":"2024-09-10T09:37:35.544481+0000","last_fullsized":"2024-09-10T09:37:35.544481+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:38:17.687991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"96'12","reported_seq":49,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.547277+0000","last_change":"2024-09-10T09:35:28.564619+0000","last_active":"2024-09-10T09:37:35.547277+0000","last_peered":"2024-09-10T09:37:35.547277+0000","last_clean":"2024-09-10T09:37:35.547277+0000","last_became_active":"2024-09-10T09:35:28.563890+0000","last_became_peered":"2024-09-10T09:35:28.563890+0000","last_unstale":"2024-09-10T09:37:35.547277+0000","last_undegraded":"2024-09-10T09:37:35.547277+0000","last_fullsized":"2024-09-10T09:37:35.547277+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:51:42.481911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"96'8","reported_seq":45,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.543410+0000","last_change":"2024-09-10T09:35:28.558909+0000","last_active":"2024-09-10T09:37:35.543410+0000","last_peered":"2024-09-10T09:37:35.543410+0000","last_clean":"2024-09-10T09:37:35.543410+0000","last_became_active":"2024-09-10T09:35:28.558604+0000","last_became_peered":"2024-09-10T09:35:28.558604+0000","last_unstale":"2024-09-10T09:37:35.543410+0000","last_undegraded":"2024-09-10T09:37:35.543410+0000","last_fullsized":"2024-09-10T09:37:35.543410+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:11.928446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"96'7","reported_seq":44,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.554844+0000","last_change":"2024-09-10T09:35:28.574459+0000","last_active":"2024-09-10T09:37:35.554844+0000","last_peered":"2024-09-10T09:37:35.554844+0000","last_clean":"2024-09-10T09:37:35.554844+0000","last_became_active":"2024-09-10T09:35:28.574154+0000","last_became_peered":"2024-09-10T09:35:28.574154+0000","last_unstale":"2024-09-10T09:37:35.554844+0000","last_undegraded":"2024-09-10T09:37:35.554844+0000","last_fullsized":"2024-09-10T09:37:35.554844+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:59:09.650536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"96'4","reported_seq":43,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.552571+0000","last_change":"2024-09-10T09:35:28.576999+0000","last_active":"2024-09-10T09:37:35.552571+0000","last_peered":"2024-09-10T09:37:35.552571+0000","last_clean":"2024-09-10T09:37:35.552571+0000","last_became_active":"2024-09-10T09:35:28.575616+0000","last_became_peered":"2024-09-10T09:35:28.575616+0000","last_unstale":"2024-09-10T09:37:35.552571+0000","last_undegraded":"2024-09-10T09:37:35.552571+0000","last_fullsized":"2024-09-10T09:37:35.552571+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:18:08.771760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"96'6","reported_seq":45,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.541496+0000","last_change":"2024-09-10T09:35:28.564846+0000","last_active":"2024-09-10T09:37:35.541496+0000","last_peered":"2024-09-10T09:37:35.541496+0000","last_clean":"2024-09-10T09:37:35.541496+0000","last_became_active":"2024-09-10T09:35:28.564498+0000","last_became_peered":"2024-09-10T09:35:28.564498+0000","last_unstale":"2024-09-10T09:37:35.541496+0000","last_undegraded":"2024-09-10T09:37:35.541496+0000","last_fullsized":"2024-09-10T09:37:35.541496+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:33:43.041730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"96'15","reported_seq":52,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.550975+0000","last_change":"2024-09-10T09:35:28.558093+0000","last_active":"2024-09-10T09:37:35.550975+0000","last_peered":"2024-09-10T09:37:35.550975+0000","last_clean":"2024-09-10T09:37:35.550975+0000","last_became_active":"2024-09-10T09:35:28.557266+0000","last_became_peered":"2024-09-10T09:35:28.557266+0000","last_unstale":"2024-09-10T09:37:35.550975+0000","last_undegraded":"2024-09-10T09:37:35.550975+0000","last_fullsized":"2024-09-10T09:37:35.550975+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:51:30.946065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"96'2","reported_seq":47,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.549558+0000","last_change":"2024-09-10T09:36:06.770298+0000","last_active":"2024-09-10T09:37:35.549558+0000","last_peered":"2024-09-10T09:37:35.549558+0000","last_clean":"2024-09-10T09:37:35.549558+0000","last_became_active":"2024-09-10T09:36:06.769848+0000","last_became_peered":"2024-09-10T09:36:06.769848+0000","last_unstale":"2024-09-10T09:37:35.549558+0000","last_undegraded":"2024-09-10T09:37:35.549558+0000","last_fullsized":"2024-09-10T09:37:35.549558+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:00:30.470887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,8,2],"acting":[3,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"96'6","reported_seq":45,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.549553+0000","last_change":"2024-09-10T09:35:28.578158+0000","last_active":"2024-09-10T09:37:35.549553+0000","last_peered":"2024-09-10T09:37:35.549553+0000","last_clean":"2024-09-10T09:37:35.549553+0000","last_became_active":"2024-09-10T09:35:28.577824+0000","last_became_peered":"2024-09-10T09:35:28.577824+0000","last_unstale":"2024-09-10T09:37:35.549553+0000","last_undegraded":"2024-09-10T09:37:35.549553+0000","last_fullsized":"2024-09-10T09:37:35.549553+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:40:27.231164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"96'8","reported_seq":47,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.550143+0000","last_change":"2024-09-10T09:35:28.562670+0000","last_active":"2024-09-10T09:37:35.550143+0000","last_peered":"2024-09-10T09:37:35.550143+0000","last_clean":"2024-09-10T09:37:35.550143+0000","last_became_active":"2024-09-10T09:35:28.562311+0000","last_became_peered":"2024-09-10T09:35:28.562311+0000","last_unstale":"2024-09-10T09:37:35.550143+0000","last_undegraded":"2024-09-10T09:37:35.550143+0000","last_fullsized":"2024-09-10T09:37:35.550143+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:07:11.716142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"96'10","reported_seq":49,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.556330+0000","last_change":"2024-09-10T09:35:28.561124+0000","last_active":"2024-09-10T09:37:35.556330+0000","last_peered":"2024-09-10T09:37:35.556330+0000","last_clean":"2024-09-10T09:37:35.556330+0000","last_became_active":"2024-09-10T09:35:28.560777+0000","last_became_peered":"2024-09-10T09:35:28.560777+0000","last_unstale":"2024-09-10T09:37:35.556330+0000","last_undegraded":"2024-09-10T09:37:35.556330+0000","last_fullsized":"2024-09-10T09:37:35.556330+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:47:04.837596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"96'11","reported_seq":50,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.556539+0000","last_change":"2024-09-10T09:35:28.576790+0000","last_active":"2024-09-10T09:37:35.556539+0000","last_peered":"2024-09-10T09:37:35.556539+0000","last_clean":"2024-09-10T09:37:35.556539+0000","last_became_active":"2024-09-10T09:35:28.576464+0000","last_became_peered":"2024-09-10T09:35:28.576464+0000","last_unstale":"2024-09-10T09:37:35.556539+0000","last_undegraded":"2024-09-10T09:37:35.556539+0000","last_fullsized":"2024-09-10T09:37:35.556539+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:40:53.517577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"96'8","reported_seq":47,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.539417+0000","last_change":"2024-09-10T09:35:28.569935+0000","last_active":"2024-09-10T09:37:35.539417+0000","last_peered":"2024-09-10T09:37:35.539417+0000","last_clean":"2024-09-10T09:37:35.539417+0000","last_became_active":"2024-09-10T09:35:28.569539+0000","last_became_peered":"2024-09-10T09:35:28.569539+0000","last_unstale":"2024-09-10T09:37:35.539417+0000","last_undegraded":"2024-09-10T09:37:35.539417+0000","last_fullsized":"2024-09-10T09:37:35.539417+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:40:45.016269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"96'7","reported_seq":44,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.550145+0000","last_change":"2024-09-10T09:35:28.571619+0000","last_active":"2024-09-10T09:37:35.550145+0000","last_peered":"2024-09-10T09:37:35.550145+0000","last_clean":"2024-09-10T09:37:35.550145+0000","last_became_active":"2024-09-10T09:35:28.568591+0000","last_became_peered":"2024-09-10T09:35:28.568591+0000","last_unstale":"2024-09-10T09:37:35.550145+0000","last_undegraded":"2024-09-10T09:37:35.550145+0000","last_fullsized":"2024-09-10T09:37:35.550145+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:07:49.959712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"96'10","reported_seq":47,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.551796+0000","last_change":"2024-09-10T09:35:28.573336+0000","last_active":"2024-09-10T09:37:35.551796+0000","last_peered":"2024-09-10T09:37:35.551796+0000","last_clean":"2024-09-10T09:37:35.551796+0000","last_became_active":"2024-09-10T09:35:28.572991+0000","last_became_peered":"2024-09-10T09:35:28.572991+0000","last_unstale":"2024-09-10T09:37:35.551796+0000","last_undegraded":"2024-09-10T09:37:35.551796+0000","last_fullsized":"2024-09-10T09:37:35.551796+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:30:02.905869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"96'8","reported_seq":47,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.552930+0000","last_change":"2024-09-10T09:35:28.571945+0000","last_active":"2024-09-10T09:37:35.552930+0000","last_peered":"2024-09-10T09:37:35.552930+0000","last_clean":"2024-09-10T09:37:35.552930+0000","last_became_active":"2024-09-10T09:35:28.571546+0000","last_became_peered":"2024-09-10T09:35:28.571546+0000","last_unstale":"2024-09-10T09:37:35.552930+0000","last_undegraded":"2024-09-10T09:37:35.552930+0000","last_fullsized":"2024-09-10T09:37:35.552930+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:36:47.946864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"96'9","reported_seq":48,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.556725+0000","last_change":"2024-09-10T09:35:28.562712+0000","last_active":"2024-09-10T09:37:35.556725+0000","last_peered":"2024-09-10T09:37:35.556725+0000","last_clean":"2024-09-10T09:37:35.556725+0000","last_became_active":"2024-09-10T09:35:28.562396+0000","last_became_peered":"2024-09-10T09:35:28.562396+0000","last_unstale":"2024-09-10T09:37:35.556725+0000","last_undegraded":"2024-09-10T09:37:35.556725+0000","last_fullsized":"2024-09-10T09:37:35.556725+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:25:47.419162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"96'7","reported_seq":44,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.553427+0000","last_change":"2024-09-10T09:35:28.566354+0000","last_active":"2024-09-10T09:37:35.553427+0000","last_peered":"2024-09-10T09:37:35.553427+0000","last_clean":"2024-09-10T09:37:35.553427+0000","last_became_active":"2024-09-10T09:35:28.565248+0000","last_became_peered":"2024-09-10T09:35:28.565248+0000","last_unstale":"2024-09-10T09:37:35.553427+0000","last_undegraded":"2024-09-10T09:37:35.553427+0000","last_fullsized":"2024-09-10T09:37:35.553427+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:48:46.741010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"96'7","reported_seq":44,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.550936+0000","last_change":"2024-09-10T09:35:28.563331+0000","last_active":"2024-09-10T09:37:35.550936+0000","last_peered":"2024-09-10T09:37:35.550936+0000","last_clean":"2024-09-10T09:37:35.550936+0000","last_became_active":"2024-09-10T09:35:28.563020+0000","last_became_peered":"2024-09-10T09:35:28.563020+0000","last_unstale":"2024-09-10T09:37:35.550936+0000","last_undegraded":"2024-09-10T09:37:35.550936+0000","last_fullsized":"2024-09-10T09:37:35.550936+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:27.653551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"96'10","reported_seq":49,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.542747+0000","last_change":"2024-09-10T09:35:28.566687+0000","last_active":"2024-09-10T09:37:35.542747+0000","last_peered":"2024-09-10T09:37:35.542747+0000","last_clean":"2024-09-10T09:37:35.542747+0000","last_became_active":"2024-09-10T09:35:28.566253+0000","last_became_peered":"2024-09-10T09:35:28.566253+0000","last_unstale":"2024-09-10T09:37:35.542747+0000","last_undegraded":"2024-09-10T09:37:35.542747+0000","last_fullsized":"2024-09-10T09:37:35.542747+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:00:15.709725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"96'7","reported_seq":44,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.538923+0000","last_change":"2024-09-10T09:35:28.562251+0000","last_active":"2024-09-10T09:37:35.538923+0000","last_peered":"2024-09-10T09:37:35.538923+0000","last_clean":"2024-09-10T09:37:35.538923+0000","last_became_active":"2024-09-10T09:35:28.561933+0000","last_became_peered":"2024-09-10T09:35:28.561933+0000","last_unstale":"2024-09-10T09:37:35.538923+0000","last_undegraded":"2024-09-10T09:37:35.538923+0000","last_fullsized":"2024-09-10T09:37:35.538923+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:30:41.447954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"96'6","reported_seq":43,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.556126+0000","last_change":"2024-09-10T09:35:28.572164+0000","last_active":"2024-09-10T09:37:35.556126+0000","last_peered":"2024-09-10T09:37:35.556126+0000","last_clean":"2024-09-10T09:37:35.556126+0000","last_became_active":"2024-09-10T09:35:28.571825+0000","last_became_peered":"2024-09-10T09:35:28.571825+0000","last_unstale":"2024-09-10T09:37:35.556126+0000","last_undegraded":"2024-09-10T09:37:35.556126+0000","last_fullsized":"2024-09-10T09:37:35.556126+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:59:50.967269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"96'16","reported_seq":55,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.550138+0000","last_change":"2024-09-10T09:35:28.568098+0000","last_active":"2024-09-10T09:37:35.550138+0000","last_peered":"2024-09-10T09:37:35.550138+0000","last_clean":"2024-09-10T09:37:35.550138+0000","last_became_active":"2024-09-10T09:35:28.567765+0000","last_became_peered":"2024-09-10T09:35:28.567765+0000","last_unstale":"2024-09-10T09:37:35.550138+0000","last_undegraded":"2024-09-10T09:37:35.550138+0000","last_fullsized":"2024-09-10T09:37:35.550138+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:55:57.260320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"96'11","reported_seq":50,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.553598+0000","last_change":"2024-09-10T09:35:28.564123+0000","last_active":"2024-09-10T09:37:35.553598+0000","last_peered":"2024-09-10T09:37:35.553598+0000","last_clean":"2024-09-10T09:37:35.553598+0000","last_became_active":"2024-09-10T09:35:28.563814+0000","last_became_peered":"2024-09-10T09:35:28.563814+0000","last_unstale":"2024-09-10T09:37:35.553598+0000","last_undegraded":"2024-09-10T09:37:35.553598+0000","last_fullsized":"2024-09-10T09:37:35.553598+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:14:17.912087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"96'9","reported_seq":48,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.549407+0000","last_change":"2024-09-10T09:35:28.568601+0000","last_active":"2024-09-10T09:37:35.549407+0000","last_peered":"2024-09-10T09:37:35.549407+0000","last_clean":"2024-09-10T09:37:35.549407+0000","last_became_active":"2024-09-10T09:35:28.568303+0000","last_became_peered":"2024-09-10T09:35:28.568303+0000","last_unstale":"2024-09-10T09:37:35.549407+0000","last_undegraded":"2024-09-10T09:37:35.549407+0000","last_fullsized":"2024-09-10T09:37:35.549407+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:30:01.016420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"96'5","reported_seq":48,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.556713+0000","last_change":"2024-09-10T09:37:07.539577+0000","last_active":"2024-09-10T09:37:35.556713+0000","last_peered":"2024-09-10T09:37:35.556713+0000","last_clean":"2024-09-10T09:37:35.556713+0000","last_became_active":"2024-09-10T09:37:07.539206+0000","last_became_peered":"2024-09-10T09:37:07.539206+0000","last_unstale":"2024-09-10T09:37:35.556713+0000","last_undegraded":"2024-09-10T09:37:35.556713+0000","last_fullsized":"2024-09-10T09:37:35.556713+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:56:06.568226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"96'6","reported_seq":45,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.541193+0000","last_change":"2024-09-10T09:35:28.569225+0000","last_active":"2024-09-10T09:37:35.541193+0000","last_peered":"2024-09-10T09:37:35.541193+0000","last_clean":"2024-09-10T09:37:35.541193+0000","last_became_active":"2024-09-10T09:35:28.568891+0000","last_became_peered":"2024-09-10T09:35:28.568891+0000","last_unstale":"2024-09-10T09:37:35.541193+0000","last_undegraded":"2024-09-10T09:37:35.541193+0000","last_fullsized":"2024-09-10T09:37:35.541193+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:33:45.811781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"96'4","reported_seq":43,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.544182+0000","last_change":"2024-09-10T09:35:28.578362+0000","last_active":"2024-09-10T09:37:35.544182+0000","last_peered":"2024-09-10T09:37:35.544182+0000","last_clean":"2024-09-10T09:37:35.544182+0000","last_became_active":"2024-09-10T09:35:28.577867+0000","last_became_peered":"2024-09-10T09:35:28.577867+0000","last_unstale":"2024-09-10T09:37:35.544182+0000","last_undegraded":"2024-09-10T09:37:35.544182+0000","last_fullsized":"2024-09-10T09:37:35.544182+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:58:17.041663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"96'8","reported_seq":45,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.537018+0000","last_change":"2024-09-10T09:35:28.568183+0000","last_active":"2024-09-10T09:37:35.537018+0000","last_peered":"2024-09-10T09:37:35.537018+0000","last_clean":"2024-09-10T09:37:35.537018+0000","last_became_active":"2024-09-10T09:35:28.566775+0000","last_became_peered":"2024-09-10T09:35:28.566775+0000","last_unstale":"2024-09-10T09:37:35.537018+0000","last_undegraded":"2024-09-10T09:37:35.537018+0000","last_fullsized":"2024-09-10T09:37:35.537018+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:28:10.008889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"96'7","reported_seq":44,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.553233+0000","last_change":"2024-09-10T09:35:28.576122+0000","last_active":"2024-09-10T09:37:35.553233+0000","last_peered":"2024-09-10T09:37:35.553233+0000","last_clean":"2024-09-10T09:37:35.553233+0000","last_became_active":"2024-09-10T09:35:28.575831+0000","last_became_peered":"2024-09-10T09:35:28.575831+0000","last_unstale":"2024-09-10T09:37:35.553233+0000","last_undegraded":"2024-09-10T09:37:35.553233+0000","last_fullsized":"2024-09-10T09:37:35.553233+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:37:51.604658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"96'4","reported_seq":41,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.554017+0000","last_change":"2024-09-10T09:35:28.564441+0000","last_active":"2024-09-10T09:37:35.554017+0000","last_peered":"2024-09-10T09:37:35.554017+0000","last_clean":"2024-09-10T09:37:35.554017+0000","last_became_active":"2024-09-10T09:35:28.564095+0000","last_became_peered":"2024-09-10T09:35:28.564095+0000","last_unstale":"2024-09-10T09:37:35.554017+0000","last_undegraded":"2024-09-10T09:37:35.554017+0000","last_fullsized":"2024-09-10T09:37:35.554017+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:53:28.275361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"96'8","reported_seq":47,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.553155+0000","last_change":"2024-09-10T09:35:28.581921+0000","last_active":"2024-09-10T09:37:35.553155+0000","last_peered":"2024-09-10T09:37:35.553155+0000","last_clean":"2024-09-10T09:37:35.553155+0000","last_became_active":"2024-09-10T09:35:28.581573+0000","last_became_peered":"2024-09-10T09:35:28.581573+0000","last_unstale":"2024-09-10T09:37:35.553155+0000","last_undegraded":"2024-09-10T09:37:35.553155+0000","last_fullsized":"2024-09-10T09:37:35.553155+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:58:38.983934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":78519,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1348,"num_read_kb":763,"num_write":270,"num_write_kb":164,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":1015,"num_keys_recovered":7,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":602112,"data_stored":236695,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":231,"ondisk_log_size":231,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":285627,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":77,"seq":330712481842,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32228,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712924,"statfs":{"total":95995035648,"available":95962034176,"internally_reserved":0,"allocated":1069056,"data_stored":554290,"data_compressed":32456,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":70,"seq":300647710774,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28228,"kb_used_data":1204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716924,"statfs":{"total":95995035648,"available":95966130176,"internally_reserved":0,"allocated":1232896,"data_stored":1215920,"data_compressed":68777,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":65,"seq":279172874301,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32252,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712900,"statfs":{"total":95995035648,"available":95962009600,"internally_reserved":0,"allocated":1093632,"data_stored":554220,"data_compressed":32456,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1599,"internal_metadata":31914433},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":57,"seq":244813135938,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32248,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712904,"statfs":{"total":95995035648,"available":95962013696,"internally_reserved":0,"allocated":1089536,"data_stored":554598,"data_compressed":32456,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":50,"seq":214748364871,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28140,"kb_used_data":1116,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717012,"statfs":{"total":95995035648,"available":95966220288,"internally_reserved":0,"allocated":1142784,"data_stored":1152083,"data_compressed":68777,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":45,"seq":193273528397,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36388,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708764,"statfs":{"total":95995035648,"available":95957774336,"internally_reserved":0,"allocated":1069056,"data_stored":550569,"data_compressed":32456,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790035,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28148,"kb_used_data":1124,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717004,"statfs":{"total":95995035648,"available":95966212096,"internally_reserved":0,"allocated":1150976,"data_stored":1146820,"data_compressed":68777,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953561,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32244,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712908,"statfs":{"total":95995035648,"available":95962017792,"internally_reserved":0,"allocated":1085440,"data_stored":554954,"data_compressed":32456,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182493,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28080,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717072,"statfs":{"total":95995035648,"available":95966281728,"internally_reserved":0,"allocated":1081344,"data_stored":549432,"data_compressed":32456,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411426,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28156,"kb_used_data":1132,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716996,"statfs":{"total":95995035648,"available":95966203904,"internally_reserved":0,"allocated":1159168,"data_stored":621070,"data_compressed":32456,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574953,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36464,"kb_used_data":1120,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708688,"statfs":{"total":95995035648,"available":95957696512,"internally_reserved":0,"allocated":1146880,"data_stored":622317,"data_compressed":32456,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738477,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36412,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708740,"statfs":{"total":95995035648,"available":95957749760,"internally_reserved":0,"allocated":1093632,"data_stored":551180,"data_compressed":32456,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34674,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34674,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34674,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36321,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36321,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36321,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36321,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36321,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":74453,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":72068,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3354,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1567,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4521,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":126976,"data_stored":72454,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T09:38:31.873 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-10T09:38:31.873 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-10T09:38:32.256 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:38:32.306 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:38:31 smithi005 ceph-mon[30350]: from='client.25340 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T09:38:32.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:38:31 smithi159 ceph-mon[35513]: from='client.25340 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T09:38:32.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:38:31 smithi102 ceph-mon[35442]: from='client.25340 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T09:38:32.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:32.860+0000 7f28e3e00640 1 -- 172.21.15.5:0/296400151 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f28e41047e0 msgr2=0x7f28e4108830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:38:32.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:32.860+0000 7f28e3e00640 1 --2- 172.21.15.5:0/296400151 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f28e41047e0 0x7f28e4108830 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f28d0009900 tx=0x7f28d002a3f0 comp rx=0 tx=0).stop 2024-09-10T09:38:32.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:32.860+0000 7f28e3e00640 1 -- 172.21.15.5:0/296400151 shutdown_connections 2024-09-10T09:38:32.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:32.860+0000 7f28e3e00640 1 --2- 172.21.15.5:0/296400151 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f28e4108f60 0x7f28e4110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:32.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:32.860+0000 7f28e3e00640 1 --2- 172.21.15.5:0/296400151 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f28e41047e0 0x7f28e4108830 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:32.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:32.860+0000 7f28e3e00640 1 --2- 172.21.15.5:0/296400151 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f28e4103e30 0x7f28e4104210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:32.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:32.860+0000 7f28e3e00640 1 -- 172.21.15.5:0/296400151 >> 172.21.15.5:0/296400151 conn(0x7f28e40ff910 msgr2=0x7f28e4101d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:38:32.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:32.860+0000 7f28e3e00640 1 -- 172.21.15.5:0/296400151 shutdown_connections 2024-09-10T09:38:32.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:32.860+0000 7f28e3e00640 1 -- 172.21.15.5:0/296400151 wait complete. 2024-09-10T09:38:32.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:32.861+0000 7f28e3e00640 1 Processor -- start 2024-09-10T09:38:32.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:32.862+0000 7f28e3e00640 1 -- start start 2024-09-10T09:38:32.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:32.862+0000 7f28e3e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f28e4103e30 0x7f28e419a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:32.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:32.862+0000 7f28e3e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f28e41047e0 0x7f28e419aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:32.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:32.862+0000 7f28e3e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f28e4108f60 0x7f28e419eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:32.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:32.862+0000 7f28e3e00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f28e4113fc0 con 0x7f28e4103e30 2024-09-10T09:38:32.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:32.862+0000 7f28e3e00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f28e4113e40 con 0x7f28e41047e0 2024-09-10T09:38:32.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:32.862+0000 7f28e3e00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f28e4114140 con 0x7f28e4108f60 2024-09-10T09:38:32.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:32.862+0000 7f28e2a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f28e4103e30 0x7f28e419a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:32.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:32.862+0000 7f28e2a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f28e4103e30 0x7f28e419a4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52982/0 (socket says 172.21.15.5:52982) 2024-09-10T09:38:32.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:32.862+0000 7f28e2a00640 1 -- 172.21.15.5:0/3950094698 learned_addr learned my addr 172.21.15.5:0/3950094698 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:38:32.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:32.862+0000 7f28e2000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f28e41047e0 0x7f28e419aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:32.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:32.862+0000 7f28e3400640 1 --2- 172.21.15.5:0/3950094698 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f28e4108f60 0x7f28e419eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:32.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:32.863+0000 7f28e2a00640 1 -- 172.21.15.5:0/3950094698 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f28e4108f60 msgr2=0x7f28e419eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:38:32.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:32.863+0000 7f28e2a00640 1 --2- 172.21.15.5:0/3950094698 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f28e4108f60 0x7f28e419eda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:32.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:32.863+0000 7f28e2a00640 1 -- 172.21.15.5:0/3950094698 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f28e41047e0 msgr2=0x7f28e419aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:38:32.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:32.863+0000 7f28e2a00640 1 --2- 172.21.15.5:0/3950094698 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f28e41047e0 0x7f28e419aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:32.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:32.863+0000 7f28e2a00640 1 -- 172.21.15.5:0/3950094698 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28e419f480 con 0x7f28e4103e30 2024-09-10T09:38:32.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:32.863+0000 7f28e2a00640 1 --2- 172.21.15.5:0/3950094698 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f28e4103e30 0x7f28e419a4d0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f28d400d950 tx=0x7f28d400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:38:32.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:32.863+0000 7f28db400640 1 -- 172.21.15.5:0/3950094698 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28d4017030 con 0x7f28e4103e30 2024-09-10T09:38:32.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:32.863+0000 7f28db400640 1 -- 172.21.15.5:0/3950094698 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f28d4004020 con 0x7f28e4103e30 2024-09-10T09:38:32.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:32.863+0000 7f28e3e00640 1 -- 172.21.15.5:0/3950094698 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28e419f770 con 0x7f28e4103e30 2024-09-10T09:38:32.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:32.863+0000 7f28db400640 1 -- 172.21.15.5:0/3950094698 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28d40047a0 con 0x7f28e4103e30 2024-09-10T09:38:32.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:32.864+0000 7f28e3e00640 1 -- 172.21.15.5:0/3950094698 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28e41a7050 con 0x7f28e4103e30 2024-09-10T09:38:32.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:32.865+0000 7f28e3e00640 1 -- 172.21.15.5:0/3950094698 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f28e4105310 con 0x7f28e4103e30 2024-09-10T09:38:32.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:32.867+0000 7f28db400640 1 -- 172.21.15.5:0/3950094698 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 19) ==== 137883+0+0 (secure 0 0 0) 0x7f28d40041d0 con 0x7f28e4103e30 2024-09-10T09:38:32.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:32.868+0000 7f28db400640 1 --2- 172.21.15.5:0/3950094698 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f28c80a2c90 0x7f28c80a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:32.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:32.868+0000 7f28db400640 1 -- 172.21.15.5:0/3950094698 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f28d40cef00 con 0x7f28e4103e30 2024-09-10T09:38:32.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:32.868+0000 7f28e2000640 1 --2- 172.21.15.5:0/3950094698 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f28c80a2c90 0x7f28c80a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:32.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:32.869+0000 7f28e2000640 1 --2- 172.21.15.5:0/3950094698 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f28c80a2c90 0x7f28c80a5150 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f28d0009230 tx=0x7f28d00058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:38:32.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:32.872+0000 7f28db400640 1 -- 172.21.15.5:0/3950094698 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f28d409b8b0 con 0x7f28e4103e30 2024-09-10T09:38:33.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:38:32 smithi005 ceph-mon[30350]: pgmap v399: 129 pgs: 129 active+clean; 655 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 1.3 KiB/s wr, 13 op/s 2024-09-10T09:38:33.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:33.058+0000 7f28e3e00640 1 -- 172.21.15.5:0/3950094698 --> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f28e4062630 con 0x7f28c80a2c90 2024-09-10T09:38:33.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:33.067+0000 7f28db400640 1 -- 172.21.15.5:0/3950094698 <== mgr.14150 v2:172.21.15.5:6800/2707348081 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340827 (secure 0 0 0) 0x7f28e4062630 con 0x7f28c80a2c90 2024-09-10T09:38:33.072 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-10T09:38:33.073 INFO:teuthology.orchestra.run.smithi005.stderr:dumped all 2024-09-10T09:38:33.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:33.072+0000 7f28e3e00640 1 -- 172.21.15.5:0/3950094698 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f28c80a2c90 msgr2=0x7f28c80a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:38:33.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:33.072+0000 7f28e3e00640 1 --2- 172.21.15.5:0/3950094698 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f28c80a2c90 0x7f28c80a5150 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f28d0009230 tx=0x7f28d00058d0 comp rx=0 tx=0).stop 2024-09-10T09:38:33.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:33.072+0000 7f28e3e00640 1 -- 172.21.15.5:0/3950094698 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f28e4103e30 msgr2=0x7f28e419a4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:38:33.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:33.072+0000 7f28e3e00640 1 --2- 172.21.15.5:0/3950094698 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f28e4103e30 0x7f28e419a4d0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f28d400d950 tx=0x7f28d400de10 comp rx=0 tx=0).stop 2024-09-10T09:38:33.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:33.072+0000 7f28e3e00640 1 -- 172.21.15.5:0/3950094698 shutdown_connections 2024-09-10T09:38:33.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:33.072+0000 7f28e3e00640 1 --2- 172.21.15.5:0/3950094698 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f28c80a2c90 0x7f28c80a5150 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:33.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:33.072+0000 7f28e3e00640 1 --2- 172.21.15.5:0/3950094698 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f28e4108f60 0x7f28e419eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:33.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:33.072+0000 7f28e3e00640 1 --2- 172.21.15.5:0/3950094698 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f28e41047e0 0x7f28e419aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:33.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:33.072+0000 7f28e3e00640 1 --2- 172.21.15.5:0/3950094698 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f28e4103e30 0x7f28e419a4d0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:33.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:33.072+0000 7f28e3e00640 1 -- 172.21.15.5:0/3950094698 >> 172.21.15.5:0/3950094698 conn(0x7f28e40ff910 msgr2=0x7f28e4100cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:38:33.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:33.072+0000 7f28e3e00640 1 -- 172.21.15.5:0/3950094698 shutdown_connections 2024-09-10T09:38:33.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:33.072+0000 7f28e3e00640 1 -- 172.21.15.5:0/3950094698 wait complete. 2024-09-10T09:38:33.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:38:32 smithi159 ceph-mon[35513]: pgmap v399: 129 pgs: 129 active+clean; 655 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 1.3 KiB/s wr, 13 op/s 2024-09-10T09:38:33.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:38:32 smithi102 ceph-mon[35442]: pgmap v399: 129 pgs: 129 active+clean; 655 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 1.3 KiB/s wr, 13 op/s 2024-09-10T09:38:33.983 INFO:teuthology.orchestra.run.smithi005.stdout:{"pg_ready":true,"pg_map":{"version":399,"stamp":"2024-09-10T09:38:31.574190+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":670259,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1517,"num_read_kb":896,"num_write":454,"num_write_kb":3106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":1181751,"num_keys_recovered":7,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":871,"ondisk_log_size":871,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":378988,"kb_used_data":13100,"kb_used_omap":18,"kb_used_meta":365677,"kb_avail":1124562836,"statfs":{"total":1151940427776,"available":1151552344064,"internally_reserved":0,"allocated":13414400,"data_stored":8629729,"data_compressed":498435,"data_compressed_allocated":675840,"data_compressed_original":2850816,"omap_allocated":19141,"internal_metadata":374453563},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":8987,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":147,"num_read_kb":84,"num_write":11,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003317"},"pg_stats":[{"pgid":"2.1f","version":"96'15","reported_seq":196,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:38:28.684238+0000","last_change":"2024-09-10T09:35:25.533085+0000","last_active":"2024-09-10T09:38:28.684238+0000","last_peered":"2024-09-10T09:38:28.684238+0000","last_clean":"2024-09-10T09:38:28.684238+0000","last_became_active":"2024-09-10T09:35:25.532729+0000","last_became_peered":"2024-09-10T09:35:25.532729+0000","last_unstale":"2024-09-10T09:38:28.684238+0000","last_undegraded":"2024-09-10T09:38:28.684238+0000","last_fullsized":"2024-09-10T09:38:28.684238+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:55:11.158279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":137,"num_read_kb":137,"num_write":16,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"96'5","reported_seq":21,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.545575+0000","last_change":"2024-09-10T09:36:06.767356+0000","last_active":"2024-09-10T09:37:35.545575+0000","last_peered":"2024-09-10T09:37:35.545575+0000","last_clean":"2024-09-10T09:37:35.545575+0000","last_became_active":"2024-09-10T09:36:06.766931+0000","last_became_peered":"2024-09-10T09:36:06.766931+0000","last_unstale":"2024-09-10T09:37:35.545575+0000","last_undegraded":"2024-09-10T09:37:35.545575+0000","last_fullsized":"2024-09-10T09:37:35.545575+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:25:02.777327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,7],"acting":[9,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.527523+0000","last_change":"2024-09-10T09:35:36.629797+0000","last_active":"2024-09-10T09:37:07.527523+0000","last_peered":"2024-09-10T09:37:07.527523+0000","last_clean":"2024-09-10T09:37:07.527523+0000","last_became_active":"2024-09-10T09:35:36.629414+0000","last_became_peered":"2024-09-10T09:35:36.629414+0000","last_unstale":"2024-09-10T09:37:07.527523+0000","last_undegraded":"2024-09-10T09:37:07.527523+0000","last_fullsized":"2024-09-10T09:37:07.527523+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:42:44.610581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"94'2","reported_seq":47,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.524964+0000","last_change":"2024-09-10T09:35:25.852022+0000","last_active":"2024-09-10T09:37:07.524964+0000","last_peered":"2024-09-10T09:37:07.524964+0000","last_clean":"2024-09-10T09:37:07.524964+0000","last_became_active":"2024-09-10T09:35:25.851587+0000","last_became_peered":"2024-09-10T09:35:25.851587+0000","last_unstale":"2024-09-10T09:37:07.524964+0000","last_undegraded":"2024-09-10T09:37:07.524964+0000","last_fullsized":"2024-09-10T09:37:07.524964+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:37:26.085914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"96'6","reported_seq":43,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.556261+0000","last_change":"2024-09-10T09:35:28.579260+0000","last_active":"2024-09-10T09:37:35.556261+0000","last_peered":"2024-09-10T09:37:35.556261+0000","last_clean":"2024-09-10T09:37:35.556261+0000","last_became_active":"2024-09-10T09:35:28.574584+0000","last_became_peered":"2024-09-10T09:35:28.574584+0000","last_unstale":"2024-09-10T09:37:35.556261+0000","last_undegraded":"2024-09-10T09:37:35.556261+0000","last_fullsized":"2024-09-10T09:37:35.556261+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:31:19.277689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.568359+0000","last_change":"2024-09-10T09:36:06.874861+0000","last_active":"2024-09-10T09:37:07.568359+0000","last_peered":"2024-09-10T09:37:07.568359+0000","last_clean":"2024-09-10T09:37:07.568359+0000","last_became_active":"2024-09-10T09:36:06.873930+0000","last_became_peered":"2024-09-10T09:36:06.873930+0000","last_unstale":"2024-09-10T09:37:07.568359+0000","last_undegraded":"2024-09-10T09:37:07.568359+0000","last_fullsized":"2024-09-10T09:37:07.568359+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:35:01.354061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,8,3],"acting":[4,6,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"96'13","reported_seq":58,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:59.271922+0000","last_change":"2024-09-10T09:35:25.534571+0000","last_active":"2024-09-10T09:37:59.271922+0000","last_peered":"2024-09-10T09:37:59.271922+0000","last_clean":"2024-09-10T09:37:59.271922+0000","last_became_active":"2024-09-10T09:35:25.534235+0000","last_became_peered":"2024-09-10T09:35:25.534235+0000","last_unstale":"2024-09-10T09:37:59.271922+0000","last_undegraded":"2024-09-10T09:37:59.271922+0000","last_fullsized":"2024-09-10T09:37:59.271922+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:50:21.447542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"96'9","reported_seq":48,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.553039+0000","last_change":"2024-09-10T09:35:28.565541+0000","last_active":"2024-09-10T09:37:35.553039+0000","last_peered":"2024-09-10T09:37:35.553039+0000","last_clean":"2024-09-10T09:37:35.553039+0000","last_became_active":"2024-09-10T09:35:28.565095+0000","last_became_peered":"2024-09-10T09:35:28.565095+0000","last_unstale":"2024-09-10T09:37:35.553039+0000","last_undegraded":"2024-09-10T09:37:35.553039+0000","last_fullsized":"2024-09-10T09:37:35.553039+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:29:31.285401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.568600+0000","last_change":"2024-09-10T09:35:36.629277+0000","last_active":"2024-09-10T09:37:07.568600+0000","last_peered":"2024-09-10T09:37:07.568600+0000","last_clean":"2024-09-10T09:37:07.568600+0000","last_became_active":"2024-09-10T09:35:36.628687+0000","last_became_peered":"2024-09-10T09:35:36.628687+0000","last_unstale":"2024-09-10T09:37:07.568600+0000","last_undegraded":"2024-09-10T09:37:07.568600+0000","last_fullsized":"2024-09-10T09:37:07.568600+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:34:33.628827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"94'5","reported_seq":50,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.530161+0000","last_change":"2024-09-10T09:35:25.536137+0000","last_active":"2024-09-10T09:37:07.530161+0000","last_peered":"2024-09-10T09:37:07.530161+0000","last_clean":"2024-09-10T09:37:07.530161+0000","last_became_active":"2024-09-10T09:35:25.535728+0000","last_became_peered":"2024-09-10T09:35:25.535728+0000","last_unstale":"2024-09-10T09:37:07.530161+0000","last_undegraded":"2024-09-10T09:37:07.530161+0000","last_fullsized":"2024-09-10T09:37:07.530161+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:36:04.829901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"96'15","reported_seq":60,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.547598+0000","last_change":"2024-09-10T09:36:06.874425+0000","last_active":"2024-09-10T09:37:35.547598+0000","last_peered":"2024-09-10T09:37:35.547598+0000","last_clean":"2024-09-10T09:37:35.547598+0000","last_became_active":"2024-09-10T09:36:06.874050+0000","last_became_peered":"2024-09-10T09:36:06.874050+0000","last_unstale":"2024-09-10T09:37:35.547598+0000","last_undegraded":"2024-09-10T09:37:35.547598+0000","last_fullsized":"2024-09-10T09:37:35.547598+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:30:51.200335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,1],"acting":[5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.568073+0000","last_change":"2024-09-10T09:35:36.615314+0000","last_active":"2024-09-10T09:37:07.568073+0000","last_peered":"2024-09-10T09:37:07.568073+0000","last_clean":"2024-09-10T09:37:07.568073+0000","last_became_active":"2024-09-10T09:35:36.614820+0000","last_became_peered":"2024-09-10T09:35:36.614820+0000","last_unstale":"2024-09-10T09:37:07.568073+0000","last_undegraded":"2024-09-10T09:37:07.568073+0000","last_fullsized":"2024-09-10T09:37:07.568073+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:51:16.897935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"96'5","reported_seq":42,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.556084+0000","last_change":"2024-09-10T09:35:28.572346+0000","last_active":"2024-09-10T09:37:35.556084+0000","last_peered":"2024-09-10T09:37:35.556084+0000","last_clean":"2024-09-10T09:37:35.556084+0000","last_became_active":"2024-09-10T09:35:28.571839+0000","last_became_peered":"2024-09-10T09:35:28.571839+0000","last_unstale":"2024-09-10T09:37:35.556084+0000","last_undegraded":"2024-09-10T09:37:35.556084+0000","last_fullsized":"2024-09-10T09:37:35.556084+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:41:29.662441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"94'4","reported_seq":49,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.529918+0000","last_change":"2024-09-10T09:35:25.538747+0000","last_active":"2024-09-10T09:37:07.529918+0000","last_peered":"2024-09-10T09:37:07.529918+0000","last_clean":"2024-09-10T09:37:07.529918+0000","last_became_active":"2024-09-10T09:35:25.538330+0000","last_became_peered":"2024-09-10T09:35:25.538330+0000","last_unstale":"2024-09-10T09:37:07.529918+0000","last_undegraded":"2024-09-10T09:37:07.529918+0000","last_fullsized":"2024-09-10T09:37:07.529918+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:15:08.340688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.568436+0000","last_change":"2024-09-10T09:35:37.270106+0000","last_active":"2024-09-10T09:37:07.568436+0000","last_peered":"2024-09-10T09:37:07.568436+0000","last_clean":"2024-09-10T09:37:07.568436+0000","last_became_active":"2024-09-10T09:35:37.269575+0000","last_became_peered":"2024-09-10T09:35:37.269575+0000","last_unstale":"2024-09-10T09:37:07.568436+0000","last_undegraded":"2024-09-10T09:37:07.568436+0000","last_fullsized":"2024-09-10T09:37:07.568436+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:35:00.414602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"96'8","reported_seq":47,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.555784+0000","last_change":"2024-09-10T09:35:28.569230+0000","last_active":"2024-09-10T09:37:35.555784+0000","last_peered":"2024-09-10T09:37:35.555784+0000","last_clean":"2024-09-10T09:37:35.555784+0000","last_became_active":"2024-09-10T09:35:28.568266+0000","last_became_peered":"2024-09-10T09:35:28.568266+0000","last_unstale":"2024-09-10T09:37:35.555784+0000","last_undegraded":"2024-09-10T09:37:35.555784+0000","last_fullsized":"2024-09-10T09:37:35.555784+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:49:43.730686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"94'3","reported_seq":17,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.544707+0000","last_change":"2024-09-10T09:37:07.544298+0000","last_active":"2024-09-10T09:37:07.544707+0000","last_peered":"2024-09-10T09:37:07.544707+0000","last_clean":"2024-09-10T09:37:07.544707+0000","last_became_active":"2024-09-10T09:37:07.531815+0000","last_became_peered":"2024-09-10T09:37:07.531815+0000","last_unstale":"2024-09-10T09:37:07.544707+0000","last_undegraded":"2024-09-10T09:37:07.544707+0000","last_fullsized":"2024-09-10T09:37:07.544707+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:18:10.091895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"94'1","reported_seq":37,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.568484+0000","last_change":"2024-09-10T09:36:06.875056+0000","last_active":"2024-09-10T09:37:07.568484+0000","last_peered":"2024-09-10T09:37:07.568484+0000","last_clean":"2024-09-10T09:37:07.568484+0000","last_became_active":"2024-09-10T09:36:06.874400+0000","last_became_peered":"2024-09-10T09:36:06.874400+0000","last_unstale":"2024-09-10T09:37:07.568484+0000","last_undegraded":"2024-09-10T09:37:07.568484+0000","last_fullsized":"2024-09-10T09:37:07.568484+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:21:06.300712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,7,8],"acting":[4,10,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"96'9","reported_seq":48,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.552565+0000","last_change":"2024-09-10T09:35:28.571319+0000","last_active":"2024-09-10T09:37:35.552565+0000","last_peered":"2024-09-10T09:37:35.552565+0000","last_clean":"2024-09-10T09:37:35.552565+0000","last_became_active":"2024-09-10T09:35:28.570972+0000","last_became_peered":"2024-09-10T09:35:28.570972+0000","last_unstale":"2024-09-10T09:37:35.552565+0000","last_undegraded":"2024-09-10T09:37:35.552565+0000","last_fullsized":"2024-09-10T09:37:35.552565+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:45:19.300535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"94'7","reported_seq":52,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.524483+0000","last_change":"2024-09-10T09:35:25.530489+0000","last_active":"2024-09-10T09:37:07.524483+0000","last_peered":"2024-09-10T09:37:07.524483+0000","last_clean":"2024-09-10T09:37:07.524483+0000","last_became_active":"2024-09-10T09:35:25.530197+0000","last_became_peered":"2024-09-10T09:35:25.530197+0000","last_unstale":"2024-09-10T09:37:07.524483+0000","last_undegraded":"2024-09-10T09:37:07.524483+0000","last_fullsized":"2024-09-10T09:37:07.524483+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:28:57.539569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.527650+0000","last_change":"2024-09-10T09:35:36.630507+0000","last_active":"2024-09-10T09:37:07.527650+0000","last_peered":"2024-09-10T09:37:07.527650+0000","last_clean":"2024-09-10T09:37:07.527650+0000","last_became_active":"2024-09-10T09:35:36.630052+0000","last_became_peered":"2024-09-10T09:35:36.630052+0000","last_unstale":"2024-09-10T09:37:07.527650+0000","last_undegraded":"2024-09-10T09:37:07.527650+0000","last_fullsized":"2024-09-10T09:37:07.527650+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:13:42.691382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"96'2","reported_seq":41,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.532487+0000","last_change":"2024-09-10T09:35:28.565650+0000","last_active":"2024-09-10T09:37:35.532487+0000","last_peered":"2024-09-10T09:37:35.532487+0000","last_clean":"2024-09-10T09:37:35.532487+0000","last_became_active":"2024-09-10T09:35:28.565303+0000","last_became_peered":"2024-09-10T09:35:28.565303+0000","last_unstale":"2024-09-10T09:37:35.532487+0000","last_undegraded":"2024-09-10T09:37:35.532487+0000","last_fullsized":"2024-09-10T09:37:35.532487+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:48:08.965487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"94'2","reported_seq":46,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.574507+0000","last_change":"2024-09-10T09:35:25.851326+0000","last_active":"2024-09-10T09:37:07.574507+0000","last_peered":"2024-09-10T09:37:07.574507+0000","last_clean":"2024-09-10T09:37:07.574507+0000","last_became_active":"2024-09-10T09:35:25.850981+0000","last_became_peered":"2024-09-10T09:35:25.850981+0000","last_unstale":"2024-09-10T09:37:07.574507+0000","last_undegraded":"2024-09-10T09:37:07.574507+0000","last_fullsized":"2024-09-10T09:37:07.574507+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:50:28.877840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.568510+0000","last_change":"2024-09-10T09:35:37.270138+0000","last_active":"2024-09-10T09:37:07.568510+0000","last_peered":"2024-09-10T09:37:07.568510+0000","last_clean":"2024-09-10T09:37:07.568510+0000","last_became_active":"2024-09-10T09:35:37.269663+0000","last_became_peered":"2024-09-10T09:35:37.269663+0000","last_unstale":"2024-09-10T09:37:07.568510+0000","last_undegraded":"2024-09-10T09:37:07.568510+0000","last_fullsized":"2024-09-10T09:37:07.568510+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:18:00.105970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"96'7","reported_seq":44,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.554144+0000","last_change":"2024-09-10T09:35:28.576197+0000","last_active":"2024-09-10T09:37:35.554144+0000","last_peered":"2024-09-10T09:37:35.554144+0000","last_clean":"2024-09-10T09:37:35.554144+0000","last_became_active":"2024-09-10T09:35:28.575859+0000","last_became_peered":"2024-09-10T09:35:28.575859+0000","last_unstale":"2024-09-10T09:37:35.554144+0000","last_undegraded":"2024-09-10T09:37:35.554144+0000","last_fullsized":"2024-09-10T09:37:35.554144+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:09:39.458761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"94'7","reported_seq":51,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.568805+0000","last_change":"2024-09-10T09:35:25.990587+0000","last_active":"2024-09-10T09:37:07.568805+0000","last_peered":"2024-09-10T09:37:07.568805+0000","last_clean":"2024-09-10T09:37:07.568805+0000","last_became_active":"2024-09-10T09:35:25.990302+0000","last_became_peered":"2024-09-10T09:35:25.990302+0000","last_unstale":"2024-09-10T09:37:07.568805+0000","last_undegraded":"2024-09-10T09:37:07.568805+0000","last_fullsized":"2024-09-10T09:37:07.568805+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:18:41.244441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.527768+0000","last_change":"2024-09-10T09:35:36.612983+0000","last_active":"2024-09-10T09:37:07.527768+0000","last_peered":"2024-09-10T09:37:07.527768+0000","last_clean":"2024-09-10T09:37:07.527768+0000","last_became_active":"2024-09-10T09:35:36.612542+0000","last_became_peered":"2024-09-10T09:35:36.612542+0000","last_unstale":"2024-09-10T09:37:07.527768+0000","last_undegraded":"2024-09-10T09:37:07.527768+0000","last_fullsized":"2024-09-10T09:37:07.527768+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:50:16.298195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"96'5","reported_seq":44,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.554267+0000","last_change":"2024-09-10T09:35:28.564121+0000","last_active":"2024-09-10T09:37:35.554267+0000","last_peered":"2024-09-10T09:37:35.554267+0000","last_clean":"2024-09-10T09:37:35.554267+0000","last_became_active":"2024-09-10T09:35:28.562780+0000","last_became_peered":"2024-09-10T09:35:28.562780+0000","last_unstale":"2024-09-10T09:37:35.554267+0000","last_undegraded":"2024-09-10T09:37:35.554267+0000","last_fullsized":"2024-09-10T09:37:35.554267+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:53:15.690478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"96'18","reported_seq":78,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:38:15.594145+0000","last_change":"2024-09-10T09:37:07.541031+0000","last_active":"2024-09-10T09:38:15.594145+0000","last_peered":"2024-09-10T09:38:15.594145+0000","last_clean":"2024-09-10T09:38:15.594145+0000","last_became_active":"2024-09-10T09:37:07.529577+0000","last_became_peered":"2024-09-10T09:37:07.529577+0000","last_unstale":"2024-09-10T09:38:15.594145+0000","last_undegraded":"2024-09-10T09:38:15.594145+0000","last_fullsized":"2024-09-10T09:38:15.594145+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:14:13.440346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":33,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":7,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.524145+0000","last_change":"2024-09-10T09:35:37.270888+0000","last_active":"2024-09-10T09:37:07.524145+0000","last_peered":"2024-09-10T09:37:07.524145+0000","last_clean":"2024-09-10T09:37:07.524145+0000","last_became_active":"2024-09-10T09:35:37.270229+0000","last_became_peered":"2024-09-10T09:35:37.270229+0000","last_unstale":"2024-09-10T09:37:07.524145+0000","last_undegraded":"2024-09-10T09:37:07.524145+0000","last_fullsized":"2024-09-10T09:37:07.524145+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:03:54.223463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"96'9","reported_seq":48,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.547904+0000","last_change":"2024-09-10T09:35:28.572799+0000","last_active":"2024-09-10T09:37:35.547904+0000","last_peered":"2024-09-10T09:37:35.547904+0000","last_clean":"2024-09-10T09:37:35.547904+0000","last_became_active":"2024-09-10T09:35:28.572310+0000","last_became_peered":"2024-09-10T09:35:28.572310+0000","last_unstale":"2024-09-10T09:37:35.547904+0000","last_undegraded":"2024-09-10T09:37:35.547904+0000","last_fullsized":"2024-09-10T09:37:35.547904+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:33:51.446766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"94'4","reported_seq":49,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.530456+0000","last_change":"2024-09-10T09:35:25.851868+0000","last_active":"2024-09-10T09:37:07.530456+0000","last_peered":"2024-09-10T09:37:07.530456+0000","last_clean":"2024-09-10T09:37:07.530456+0000","last_became_active":"2024-09-10T09:35:25.851447+0000","last_became_peered":"2024-09-10T09:35:25.851447+0000","last_unstale":"2024-09-10T09:37:07.530456+0000","last_undegraded":"2024-09-10T09:37:07.530456+0000","last_fullsized":"2024-09-10T09:37:07.530456+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:15:57.706690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.526545+0000","last_change":"2024-09-10T09:35:36.619459+0000","last_active":"2024-09-10T09:37:07.526545+0000","last_peered":"2024-09-10T09:37:07.526545+0000","last_clean":"2024-09-10T09:37:07.526545+0000","last_became_active":"2024-09-10T09:35:36.619023+0000","last_became_peered":"2024-09-10T09:35:36.619023+0000","last_unstale":"2024-09-10T09:37:07.526545+0000","last_undegraded":"2024-09-10T09:37:07.526545+0000","last_fullsized":"2024-09-10T09:37:07.526545+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:07:32.999231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"96'4","reported_seq":43,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.548010+0000","last_change":"2024-09-10T09:35:28.575668+0000","last_active":"2024-09-10T09:37:35.548010+0000","last_peered":"2024-09-10T09:37:35.548010+0000","last_clean":"2024-09-10T09:37:35.548010+0000","last_became_active":"2024-09-10T09:35:28.575425+0000","last_became_peered":"2024-09-10T09:35:28.575425+0000","last_unstale":"2024-09-10T09:37:35.548010+0000","last_undegraded":"2024-09-10T09:37:35.548010+0000","last_fullsized":"2024-09-10T09:37:35.548010+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:13:27.120208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"94'1","reported_seq":45,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.528018+0000","last_change":"2024-09-10T09:35:25.854569+0000","last_active":"2024-09-10T09:37:07.528018+0000","last_peered":"2024-09-10T09:37:07.528018+0000","last_clean":"2024-09-10T09:37:07.528018+0000","last_became_active":"2024-09-10T09:35:25.854152+0000","last_became_peered":"2024-09-10T09:35:25.854152+0000","last_unstale":"2024-09-10T09:37:07.528018+0000","last_undegraded":"2024-09-10T09:37:07.528018+0000","last_fullsized":"2024-09-10T09:37:07.528018+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:27:15.584707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.529957+0000","last_change":"2024-09-10T09:35:37.270517+0000","last_active":"2024-09-10T09:37:07.529957+0000","last_peered":"2024-09-10T09:37:07.529957+0000","last_clean":"2024-09-10T09:37:07.529957+0000","last_became_active":"2024-09-10T09:35:37.269793+0000","last_became_peered":"2024-09-10T09:35:37.269793+0000","last_unstale":"2024-09-10T09:37:07.529957+0000","last_undegraded":"2024-09-10T09:37:07.529957+0000","last_fullsized":"2024-09-10T09:37:07.529957+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:28:58.272848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"96'7","reported_seq":52,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.556473+0000","last_change":"2024-09-10T09:37:07.762585+0000","last_active":"2024-09-10T09:37:35.556473+0000","last_peered":"2024-09-10T09:37:35.556473+0000","last_clean":"2024-09-10T09:37:35.556473+0000","last_became_active":"2024-09-10T09:37:07.762250+0000","last_became_peered":"2024-09-10T09:37:07.762250+0000","last_unstale":"2024-09-10T09:37:35.556473+0000","last_undegraded":"2024-09-10T09:37:35.556473+0000","last_fullsized":"2024-09-10T09:37:35.556473+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:38:01.751512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"96'8","reported_seq":189,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:38:27.683309+0000","last_change":"2024-09-10T09:35:25.531062+0000","last_active":"2024-09-10T09:38:27.683309+0000","last_peered":"2024-09-10T09:38:27.683309+0000","last_clean":"2024-09-10T09:38:27.683309+0000","last_became_active":"2024-09-10T09:35:25.530005+0000","last_became_peered":"2024-09-10T09:35:25.530005+0000","last_unstale":"2024-09-10T09:38:27.683309+0000","last_undegraded":"2024-09-10T09:38:27.683309+0000","last_fullsized":"2024-09-10T09:38:27.683309+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:42:35.330312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1433,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":138,"num_read_kb":4,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.538766+0000","last_change":"2024-09-10T09:35:36.623099+0000","last_active":"2024-09-10T09:37:07.538766+0000","last_peered":"2024-09-10T09:37:07.538766+0000","last_clean":"2024-09-10T09:37:07.538766+0000","last_became_active":"2024-09-10T09:35:36.622716+0000","last_became_peered":"2024-09-10T09:35:36.622716+0000","last_unstale":"2024-09-10T09:37:07.538766+0000","last_undegraded":"2024-09-10T09:37:07.538766+0000","last_fullsized":"2024-09-10T09:37:07.538766+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:10:38.495164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"96'8","reported_seq":53,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.543805+0000","last_change":"2024-09-10T09:36:06.769405+0000","last_active":"2024-09-10T09:37:35.543805+0000","last_peered":"2024-09-10T09:37:35.543805+0000","last_clean":"2024-09-10T09:37:35.543805+0000","last_became_active":"2024-09-10T09:36:06.769042+0000","last_became_peered":"2024-09-10T09:36:06.769042+0000","last_unstale":"2024-09-10T09:37:35.543805+0000","last_undegraded":"2024-09-10T09:37:35.543805+0000","last_fullsized":"2024-09-10T09:37:35.543805+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:01:49.340765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,8],"acting":[5,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"94'5","reported_seq":184,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:38:26.681235+0000","last_change":"2024-09-10T09:35:25.851430+0000","last_active":"2024-09-10T09:38:26.681235+0000","last_peered":"2024-09-10T09:38:26.681235+0000","last_clean":"2024-09-10T09:38:26.681235+0000","last_became_active":"2024-09-10T09:35:25.851067+0000","last_became_peered":"2024-09-10T09:35:25.851067+0000","last_unstale":"2024-09-10T09:38:26.681235+0000","last_undegraded":"2024-09-10T09:38:26.681235+0000","last_fullsized":"2024-09-10T09:38:26.681235+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:55:58.756207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":135,"num_read_kb":135,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.538676+0000","last_change":"2024-09-10T09:35:37.270361+0000","last_active":"2024-09-10T09:37:07.538676+0000","last_peered":"2024-09-10T09:37:07.538676+0000","last_clean":"2024-09-10T09:37:07.538676+0000","last_became_active":"2024-09-10T09:35:37.269632+0000","last_became_peered":"2024-09-10T09:35:37.269632+0000","last_unstale":"2024-09-10T09:37:07.538676+0000","last_undegraded":"2024-09-10T09:37:07.538676+0000","last_fullsized":"2024-09-10T09:37:07.538676+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:50:34.360497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"96'5","reported_seq":44,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.548275+0000","last_change":"2024-09-10T09:35:28.571064+0000","last_active":"2024-09-10T09:37:35.548275+0000","last_peered":"2024-09-10T09:37:35.548275+0000","last_clean":"2024-09-10T09:37:35.548275+0000","last_became_active":"2024-09-10T09:35:28.570691+0000","last_became_peered":"2024-09-10T09:35:28.570691+0000","last_unstale":"2024-09-10T09:37:35.548275+0000","last_undegraded":"2024-09-10T09:37:35.548275+0000","last_fullsized":"2024-09-10T09:37:35.548275+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:58:33.162501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"94'4","reported_seq":57,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.572759+0000","last_change":"2024-09-10T09:37:07.572247+0000","last_active":"2024-09-10T09:37:07.572759+0000","last_peered":"2024-09-10T09:37:07.572759+0000","last_clean":"2024-09-10T09:37:07.572759+0000","last_became_active":"2024-09-10T09:37:07.569410+0000","last_became_peered":"2024-09-10T09:37:07.569410+0000","last_unstale":"2024-09-10T09:37:07.572759+0000","last_undegraded":"2024-09-10T09:37:07.572759+0000","last_fullsized":"2024-09-10T09:37:07.572759+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:05:09.037084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,4],"acting":[7,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.532723+0000","last_change":"2024-09-10T09:35:37.270358+0000","last_active":"2024-09-10T09:37:07.532723+0000","last_peered":"2024-09-10T09:37:07.532723+0000","last_clean":"2024-09-10T09:37:07.532723+0000","last_became_active":"2024-09-10T09:35:37.269822+0000","last_became_peered":"2024-09-10T09:35:37.269822+0000","last_unstale":"2024-09-10T09:37:07.532723+0000","last_undegraded":"2024-09-10T09:37:07.532723+0000","last_fullsized":"2024-09-10T09:37:07.532723+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:19:48.932596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"96'8","reported_seq":47,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.553803+0000","last_change":"2024-09-10T09:35:28.576244+0000","last_active":"2024-09-10T09:37:35.553803+0000","last_peered":"2024-09-10T09:37:35.553803+0000","last_clean":"2024-09-10T09:37:35.553803+0000","last_became_active":"2024-09-10T09:35:28.571801+0000","last_became_peered":"2024-09-10T09:35:28.571801+0000","last_unstale":"2024-09-10T09:37:35.553803+0000","last_undegraded":"2024-09-10T09:37:35.553803+0000","last_fullsized":"2024-09-10T09:37:35.553803+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:11:21.742220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"94'6","reported_seq":52,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.568754+0000","last_change":"2024-09-10T09:35:25.537118+0000","last_active":"2024-09-10T09:37:07.568754+0000","last_peered":"2024-09-10T09:37:07.568754+0000","last_clean":"2024-09-10T09:37:07.568754+0000","last_became_active":"2024-09-10T09:35:25.536718+0000","last_became_peered":"2024-09-10T09:35:25.536718+0000","last_unstale":"2024-09-10T09:37:07.568754+0000","last_undegraded":"2024-09-10T09:37:07.568754+0000","last_fullsized":"2024-09-10T09:37:07.568754+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:16:32.837476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":361,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.528720+0000","last_change":"2024-09-10T09:34:07.350825+0000","last_active":"2024-09-10T09:37:07.528720+0000","last_peered":"2024-09-10T09:37:07.528720+0000","last_clean":"2024-09-10T09:37:07.528720+0000","last_became_active":"2024-09-10T09:34:07.350325+0000","last_became_peered":"2024-09-10T09:34:07.350325+0000","last_unstale":"2024-09-10T09:37:07.528720+0000","last_undegraded":"2024-09-10T09:37:07.528720+0000","last_fullsized":"2024-09-10T09:37:07.528720+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:30:26.408262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:30:26.408262+0000","last_clean_scrub_stamp":"2024-09-10T09:30:26.408262+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:58:04.920443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"94'1","reported_seq":30,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.529308+0000","last_change":"2024-09-10T09:35:36.632941+0000","last_active":"2024-09-10T09:37:07.529308+0000","last_peered":"2024-09-10T09:37:07.529308+0000","last_clean":"2024-09-10T09:37:07.529308+0000","last_became_active":"2024-09-10T09:35:36.632527+0000","last_became_peered":"2024-09-10T09:35:36.632527+0000","last_unstale":"2024-09-10T09:37:07.529308+0000","last_undegraded":"2024-09-10T09:37:07.529308+0000","last_fullsized":"2024-09-10T09:37:07.529308+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:33:21.733772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"96'7","reported_seq":44,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.556036+0000","last_change":"2024-09-10T09:35:28.572010+0000","last_active":"2024-09-10T09:37:35.556036+0000","last_peered":"2024-09-10T09:37:35.556036+0000","last_clean":"2024-09-10T09:37:35.556036+0000","last_became_active":"2024-09-10T09:35:28.571623+0000","last_became_peered":"2024-09-10T09:35:28.571623+0000","last_unstale":"2024-09-10T09:37:35.556036+0000","last_undegraded":"2024-09-10T09:37:35.556036+0000","last_fullsized":"2024-09-10T09:37:35.556036+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:00:49.328663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"94'5","reported_seq":188,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:38:29.686772+0000","last_change":"2024-09-10T09:35:25.990328+0000","last_active":"2024-09-10T09:38:29.686772+0000","last_peered":"2024-09-10T09:38:29.686772+0000","last_clean":"2024-09-10T09:38:29.686772+0000","last_became_active":"2024-09-10T09:35:25.989946+0000","last_became_peered":"2024-09-10T09:35:25.989946+0000","last_unstale":"2024-09-10T09:38:29.686772+0000","last_undegraded":"2024-09-10T09:38:29.686772+0000","last_fullsized":"2024-09-10T09:38:29.686772+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:51:16.069913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":139,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.574499+0000","last_change":"2024-09-10T09:35:36.619013+0000","last_active":"2024-09-10T09:37:07.574499+0000","last_peered":"2024-09-10T09:37:07.574499+0000","last_clean":"2024-09-10T09:37:07.574499+0000","last_became_active":"2024-09-10T09:35:36.618506+0000","last_became_peered":"2024-09-10T09:35:36.618506+0000","last_unstale":"2024-09-10T09:37:07.574499+0000","last_undegraded":"2024-09-10T09:37:07.574499+0000","last_fullsized":"2024-09-10T09:37:07.574499+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:57:53.580569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"96'6","reported_seq":49,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.553955+0000","last_change":"2024-09-10T09:36:06.768573+0000","last_active":"2024-09-10T09:37:35.553955+0000","last_peered":"2024-09-10T09:37:35.553955+0000","last_clean":"2024-09-10T09:37:35.553955+0000","last_became_active":"2024-09-10T09:36:06.768228+0000","last_became_peered":"2024-09-10T09:36:06.768228+0000","last_unstale":"2024-09-10T09:37:35.553955+0000","last_undegraded":"2024-09-10T09:37:35.553955+0000","last_fullsized":"2024-09-10T09:37:35.553955+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:57:38.310713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"94'3","reported_seq":48,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.533010+0000","last_change":"2024-09-10T09:35:25.528795+0000","last_active":"2024-09-10T09:37:07.533010+0000","last_peered":"2024-09-10T09:37:07.533010+0000","last_clean":"2024-09-10T09:37:07.533010+0000","last_became_active":"2024-09-10T09:35:25.528475+0000","last_became_peered":"2024-09-10T09:35:25.528475+0000","last_unstale":"2024-09-10T09:37:07.533010+0000","last_undegraded":"2024-09-10T09:37:07.533010+0000","last_fullsized":"2024-09-10T09:37:07.533010+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:16:06.999186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.761758+0000","last_change":"2024-09-10T09:35:36.633109+0000","last_active":"2024-09-10T09:37:07.761758+0000","last_peered":"2024-09-10T09:37:07.761758+0000","last_clean":"2024-09-10T09:37:07.761758+0000","last_became_active":"2024-09-10T09:35:36.632682+0000","last_became_peered":"2024-09-10T09:35:36.632682+0000","last_unstale":"2024-09-10T09:37:07.761758+0000","last_undegraded":"2024-09-10T09:37:07.761758+0000","last_fullsized":"2024-09-10T09:37:07.761758+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:51:14.372649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"96'9","reported_seq":46,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.552249+0000","last_change":"2024-09-10T09:35:28.562103+0000","last_active":"2024-09-10T09:37:35.552249+0000","last_peered":"2024-09-10T09:37:35.552249+0000","last_clean":"2024-09-10T09:37:35.552249+0000","last_became_active":"2024-09-10T09:35:28.561765+0000","last_became_peered":"2024-09-10T09:35:28.561765+0000","last_unstale":"2024-09-10T09:37:35.552249+0000","last_undegraded":"2024-09-10T09:37:35.552249+0000","last_fullsized":"2024-09-10T09:37:35.552249+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:35:35.400569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"94'5","reported_seq":49,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.539243+0000","last_change":"2024-09-10T09:35:25.538536+0000","last_active":"2024-09-10T09:37:07.539243+0000","last_peered":"2024-09-10T09:37:07.539243+0000","last_clean":"2024-09-10T09:37:07.539243+0000","last_became_active":"2024-09-10T09:35:25.537944+0000","last_became_peered":"2024-09-10T09:35:25.537944+0000","last_unstale":"2024-09-10T09:37:07.539243+0000","last_undegraded":"2024-09-10T09:37:07.539243+0000","last_fullsized":"2024-09-10T09:37:07.539243+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:01:19.385381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":17,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.529928+0000","last_change":"2024-09-10T09:36:06.769534+0000","last_active":"2024-09-10T09:37:07.529928+0000","last_peered":"2024-09-10T09:37:07.529928+0000","last_clean":"2024-09-10T09:37:07.529928+0000","last_became_active":"2024-09-10T09:36:06.769015+0000","last_became_peered":"2024-09-10T09:36:06.769015+0000","last_unstale":"2024-09-10T09:37:07.529928+0000","last_undegraded":"2024-09-10T09:37:07.529928+0000","last_fullsized":"2024-09-10T09:37:07.529928+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:19:26.369737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7,1],"acting":[8,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"96'5","reported_seq":44,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.539962+0000","last_change":"2024-09-10T09:35:28.566735+0000","last_active":"2024-09-10T09:37:35.539962+0000","last_peered":"2024-09-10T09:37:35.539962+0000","last_clean":"2024-09-10T09:37:35.539962+0000","last_became_active":"2024-09-10T09:35:28.566400+0000","last_became_peered":"2024-09-10T09:35:28.566400+0000","last_unstale":"2024-09-10T09:37:35.539962+0000","last_undegraded":"2024-09-10T09:37:35.539962+0000","last_fullsized":"2024-09-10T09:37:35.539962+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:18:53.337816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"94'2","reported_seq":47,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.530569+0000","last_change":"2024-09-10T09:35:25.535783+0000","last_active":"2024-09-10T09:37:07.530569+0000","last_peered":"2024-09-10T09:37:07.530569+0000","last_clean":"2024-09-10T09:37:07.530569+0000","last_became_active":"2024-09-10T09:35:25.535411+0000","last_became_peered":"2024-09-10T09:35:25.535411+0000","last_unstale":"2024-09-10T09:37:07.530569+0000","last_undegraded":"2024-09-10T09:37:07.530569+0000","last_fullsized":"2024-09-10T09:37:07.530569+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:48:09.239083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.531208+0000","last_change":"2024-09-10T09:35:36.619380+0000","last_active":"2024-09-10T09:37:07.531208+0000","last_peered":"2024-09-10T09:37:07.531208+0000","last_clean":"2024-09-10T09:37:07.531208+0000","last_became_active":"2024-09-10T09:35:36.618953+0000","last_became_peered":"2024-09-10T09:35:36.618953+0000","last_unstale":"2024-09-10T09:37:07.531208+0000","last_undegraded":"2024-09-10T09:37:07.531208+0000","last_fullsized":"2024-09-10T09:37:07.531208+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:40:25.181408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"96'6","reported_seq":45,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.546928+0000","last_change":"2024-09-10T09:35:28.571396+0000","last_active":"2024-09-10T09:37:35.546928+0000","last_peered":"2024-09-10T09:37:35.546928+0000","last_clean":"2024-09-10T09:37:35.546928+0000","last_became_active":"2024-09-10T09:35:28.571063+0000","last_became_peered":"2024-09-10T09:35:28.571063+0000","last_unstale":"2024-09-10T09:37:35.546928+0000","last_undegraded":"2024-09-10T09:37:35.546928+0000","last_fullsized":"2024-09-10T09:37:35.546928+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:38:17.965133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"94'17","reported_seq":62,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.530060+0000","last_change":"2024-09-10T09:35:25.851773+0000","last_active":"2024-09-10T09:37:07.530060+0000","last_peered":"2024-09-10T09:37:07.530060+0000","last_clean":"2024-09-10T09:37:07.530060+0000","last_became_active":"2024-09-10T09:35:25.851384+0000","last_became_peered":"2024-09-10T09:35:25.851384+0000","last_unstale":"2024-09-10T09:37:07.530060+0000","last_undegraded":"2024-09-10T09:37:07.530060+0000","last_fullsized":"2024-09-10T09:37:07.530060+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:03:32.338643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.539186+0000","last_change":"2024-09-10T09:35:36.620971+0000","last_active":"2024-09-10T09:37:07.539186+0000","last_peered":"2024-09-10T09:37:07.539186+0000","last_clean":"2024-09-10T09:37:07.539186+0000","last_became_active":"2024-09-10T09:35:36.620343+0000","last_became_peered":"2024-09-10T09:35:36.620343+0000","last_unstale":"2024-09-10T09:37:07.539186+0000","last_undegraded":"2024-09-10T09:37:07.539186+0000","last_fullsized":"2024-09-10T09:37:07.539186+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:17:19.517882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"96'6","reported_seq":43,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.553818+0000","last_change":"2024-09-10T09:35:28.565643+0000","last_active":"2024-09-10T09:37:35.553818+0000","last_peered":"2024-09-10T09:37:35.553818+0000","last_clean":"2024-09-10T09:37:35.553818+0000","last_became_active":"2024-09-10T09:35:28.564217+0000","last_became_peered":"2024-09-10T09:35:28.564217+0000","last_unstale":"2024-09-10T09:37:35.553818+0000","last_undegraded":"2024-09-10T09:37:35.553818+0000","last_fullsized":"2024-09-10T09:37:35.553818+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:53:34.562690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"94'5","reported_seq":184,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:38:26.681823+0000","last_change":"2024-09-10T09:35:25.534167+0000","last_active":"2024-09-10T09:38:26.681823+0000","last_peered":"2024-09-10T09:38:26.681823+0000","last_clean":"2024-09-10T09:38:26.681823+0000","last_became_active":"2024-09-10T09:35:25.533719+0000","last_became_peered":"2024-09-10T09:35:25.533719+0000","last_unstale":"2024-09-10T09:38:26.681823+0000","last_undegraded":"2024-09-10T09:38:26.681823+0000","last_fullsized":"2024-09-10T09:38:26.681823+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:00:03.537355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":137,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.527733+0000","last_change":"2024-09-10T09:35:36.628716+0000","last_active":"2024-09-10T09:37:07.527733+0000","last_peered":"2024-09-10T09:37:07.527733+0000","last_clean":"2024-09-10T09:37:07.527733+0000","last_became_active":"2024-09-10T09:35:36.628085+0000","last_became_peered":"2024-09-10T09:35:36.628085+0000","last_unstale":"2024-09-10T09:37:07.527733+0000","last_undegraded":"2024-09-10T09:37:07.527733+0000","last_fullsized":"2024-09-10T09:37:07.527733+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:45:01.279403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"96'15","reported_seq":52,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.557076+0000","last_change":"2024-09-10T09:35:28.566701+0000","last_active":"2024-09-10T09:37:35.557076+0000","last_peered":"2024-09-10T09:37:35.557076+0000","last_clean":"2024-09-10T09:37:35.557076+0000","last_became_active":"2024-09-10T09:35:28.566391+0000","last_became_peered":"2024-09-10T09:35:28.566391+0000","last_unstale":"2024-09-10T09:37:35.557076+0000","last_undegraded":"2024-09-10T09:37:35.557076+0000","last_fullsized":"2024-09-10T09:37:35.557076+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:57:59.918706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"94'5","reported_seq":50,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.531896+0000","last_change":"2024-09-10T09:35:25.532510+0000","last_active":"2024-09-10T09:37:07.531896+0000","last_peered":"2024-09-10T09:37:07.531896+0000","last_clean":"2024-09-10T09:37:07.531896+0000","last_became_active":"2024-09-10T09:35:25.532193+0000","last_became_peered":"2024-09-10T09:35:25.532193+0000","last_unstale":"2024-09-10T09:37:07.531896+0000","last_undegraded":"2024-09-10T09:37:07.531896+0000","last_fullsized":"2024-09-10T09:37:07.531896+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:17:24.685178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.526424+0000","last_change":"2024-09-10T09:35:36.630742+0000","last_active":"2024-09-10T09:37:07.526424+0000","last_peered":"2024-09-10T09:37:07.526424+0000","last_clean":"2024-09-10T09:37:07.526424+0000","last_became_active":"2024-09-10T09:35:36.630335+0000","last_became_peered":"2024-09-10T09:35:36.630335+0000","last_unstale":"2024-09-10T09:37:07.526424+0000","last_undegraded":"2024-09-10T09:37:07.526424+0000","last_fullsized":"2024-09-10T09:37:07.526424+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:29:12.029390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"96'9","reported_seq":48,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.552455+0000","last_change":"2024-09-10T09:35:28.581817+0000","last_active":"2024-09-10T09:37:35.552455+0000","last_peered":"2024-09-10T09:37:35.552455+0000","last_clean":"2024-09-10T09:37:35.552455+0000","last_became_active":"2024-09-10T09:35:28.580316+0000","last_became_peered":"2024-09-10T09:35:28.580316+0000","last_unstale":"2024-09-10T09:37:35.552455+0000","last_undegraded":"2024-09-10T09:37:35.552455+0000","last_fullsized":"2024-09-10T09:37:35.552455+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:04:08.249564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"94'9","reported_seq":55,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.528127+0000","last_change":"2024-09-10T09:35:25.537474+0000","last_active":"2024-09-10T09:37:07.528127+0000","last_peered":"2024-09-10T09:37:07.528127+0000","last_clean":"2024-09-10T09:37:07.528127+0000","last_became_active":"2024-09-10T09:35:25.537096+0000","last_became_peered":"2024-09-10T09:35:25.537096+0000","last_unstale":"2024-09-10T09:37:07.528127+0000","last_undegraded":"2024-09-10T09:37:07.528127+0000","last_fullsized":"2024-09-10T09:37:07.528127+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:13:06.008458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.532316+0000","last_change":"2024-09-10T09:35:36.622788+0000","last_active":"2024-09-10T09:37:07.532316+0000","last_peered":"2024-09-10T09:37:07.532316+0000","last_clean":"2024-09-10T09:37:07.532316+0000","last_became_active":"2024-09-10T09:35:36.622377+0000","last_became_peered":"2024-09-10T09:35:36.622377+0000","last_unstale":"2024-09-10T09:37:07.532316+0000","last_undegraded":"2024-09-10T09:37:07.532316+0000","last_fullsized":"2024-09-10T09:37:07.532316+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:29:56.680864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"96'6","reported_seq":45,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.534371+0000","last_change":"2024-09-10T09:35:28.564529+0000","last_active":"2024-09-10T09:37:35.534371+0000","last_peered":"2024-09-10T09:37:35.534371+0000","last_clean":"2024-09-10T09:37:35.534371+0000","last_became_active":"2024-09-10T09:35:28.564223+0000","last_became_peered":"2024-09-10T09:35:28.564223+0000","last_unstale":"2024-09-10T09:37:35.534371+0000","last_undegraded":"2024-09-10T09:37:35.534371+0000","last_fullsized":"2024-09-10T09:37:35.534371+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:21:24.296578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"96'43","reported_seq":370,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:38:28.685250+0000","last_change":"2024-09-10T09:35:25.534331+0000","last_active":"2024-09-10T09:38:28.685250+0000","last_peered":"2024-09-10T09:38:28.685250+0000","last_clean":"2024-09-10T09:38:28.685250+0000","last_became_active":"2024-09-10T09:35:25.533985+0000","last_became_peered":"2024-09-10T09:35:25.533985+0000","last_unstale":"2024-09-10T09:38:28.685250+0000","last_undegraded":"2024-09-10T09:38:28.685250+0000","last_fullsized":"2024-09-10T09:38:28.685250+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":43,"log_dups_size":0,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:14:37.050650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71856,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":283,"num_read_kb":213,"num_write":45,"num_write_kb":91,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.527925+0000","last_change":"2024-09-10T09:35:36.632835+0000","last_active":"2024-09-10T09:37:07.527925+0000","last_peered":"2024-09-10T09:37:07.527925+0000","last_clean":"2024-09-10T09:37:07.527925+0000","last_became_active":"2024-09-10T09:35:36.630962+0000","last_became_peered":"2024-09-10T09:35:36.630962+0000","last_unstale":"2024-09-10T09:37:07.527925+0000","last_undegraded":"2024-09-10T09:37:07.527925+0000","last_fullsized":"2024-09-10T09:37:07.527925+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:59:27.613721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"96'11","reported_seq":50,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.546541+0000","last_change":"2024-09-10T09:35:28.567292+0000","last_active":"2024-09-10T09:37:35.546541+0000","last_peered":"2024-09-10T09:37:35.546541+0000","last_clean":"2024-09-10T09:37:35.546541+0000","last_became_active":"2024-09-10T09:35:28.566973+0000","last_became_peered":"2024-09-10T09:35:28.566973+0000","last_unstale":"2024-09-10T09:37:35.546541+0000","last_undegraded":"2024-09-10T09:37:35.546541+0000","last_fullsized":"2024-09-10T09:37:35.546541+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:05:37.251933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"94'5","reported_seq":50,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.532787+0000","last_change":"2024-09-10T09:35:25.852171+0000","last_active":"2024-09-10T09:37:07.532787+0000","last_peered":"2024-09-10T09:37:07.532787+0000","last_clean":"2024-09-10T09:37:07.532787+0000","last_became_active":"2024-09-10T09:35:25.851580+0000","last_became_peered":"2024-09-10T09:35:25.851580+0000","last_unstale":"2024-09-10T09:37:07.532787+0000","last_undegraded":"2024-09-10T09:37:07.532787+0000","last_fullsized":"2024-09-10T09:37:07.532787+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:47:03.263152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.568336+0000","last_change":"2024-09-10T09:35:36.629212+0000","last_active":"2024-09-10T09:37:07.568336+0000","last_peered":"2024-09-10T09:37:07.568336+0000","last_clean":"2024-09-10T09:37:07.568336+0000","last_became_active":"2024-09-10T09:35:36.628753+0000","last_became_peered":"2024-09-10T09:35:36.628753+0000","last_unstale":"2024-09-10T09:37:07.568336+0000","last_undegraded":"2024-09-10T09:37:07.568336+0000","last_fullsized":"2024-09-10T09:37:07.568336+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:54:45.311361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"96'9","reported_seq":46,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.553075+0000","last_change":"2024-09-10T09:35:28.572497+0000","last_active":"2024-09-10T09:37:35.553075+0000","last_peered":"2024-09-10T09:37:35.553075+0000","last_clean":"2024-09-10T09:37:35.553075+0000","last_became_active":"2024-09-10T09:35:28.569238+0000","last_became_peered":"2024-09-10T09:35:28.569238+0000","last_unstale":"2024-09-10T09:37:35.553075+0000","last_undegraded":"2024-09-10T09:37:35.553075+0000","last_fullsized":"2024-09-10T09:37:35.553075+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:38:47.189209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"94'4","reported_seq":47,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.529569+0000","last_change":"2024-09-10T09:35:25.536369+0000","last_active":"2024-09-10T09:37:07.529569+0000","last_peered":"2024-09-10T09:37:07.529569+0000","last_clean":"2024-09-10T09:37:07.529569+0000","last_became_active":"2024-09-10T09:35:25.535607+0000","last_became_peered":"2024-09-10T09:35:25.535607+0000","last_unstale":"2024-09-10T09:37:07.529569+0000","last_undegraded":"2024-09-10T09:37:07.529569+0000","last_fullsized":"2024-09-10T09:37:07.529569+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:49:18.621934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.527047+0000","last_change":"2024-09-10T09:35:36.627977+0000","last_active":"2024-09-10T09:37:07.527047+0000","last_peered":"2024-09-10T09:37:07.527047+0000","last_clean":"2024-09-10T09:37:07.527047+0000","last_became_active":"2024-09-10T09:35:36.626942+0000","last_became_peered":"2024-09-10T09:35:36.626942+0000","last_unstale":"2024-09-10T09:37:07.527047+0000","last_undegraded":"2024-09-10T09:37:07.527047+0000","last_fullsized":"2024-09-10T09:37:07.527047+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:37:18.209424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"96'10","reported_seq":49,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.553165+0000","last_change":"2024-09-10T09:35:28.562670+0000","last_active":"2024-09-10T09:37:35.553165+0000","last_peered":"2024-09-10T09:37:35.553165+0000","last_clean":"2024-09-10T09:37:35.553165+0000","last_became_active":"2024-09-10T09:35:28.562028+0000","last_became_peered":"2024-09-10T09:35:28.562028+0000","last_unstale":"2024-09-10T09:37:35.553165+0000","last_undegraded":"2024-09-10T09:37:35.553165+0000","last_fullsized":"2024-09-10T09:37:35.553165+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:37:15.176050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"96'5","reported_seq":318,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:38:28.684925+0000","last_change":"2024-09-10T09:35:25.529861+0000","last_active":"2024-09-10T09:38:28.684925+0000","last_peered":"2024-09-10T09:38:28.684925+0000","last_clean":"2024-09-10T09:38:28.684925+0000","last_became_active":"2024-09-10T09:35:25.529475+0000","last_became_peered":"2024-09-10T09:35:25.529475+0000","last_unstale":"2024-09-10T09:38:28.684925+0000","last_undegraded":"2024-09-10T09:38:28.684925+0000","last_fullsized":"2024-09-10T09:38:28.684925+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:44:27.086509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":269,"num_read_kb":269,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.524306+0000","last_change":"2024-09-10T09:35:36.615358+0000","last_active":"2024-09-10T09:37:07.524306+0000","last_peered":"2024-09-10T09:37:07.524306+0000","last_clean":"2024-09-10T09:37:07.524306+0000","last_became_active":"2024-09-10T09:35:36.614901+0000","last_became_peered":"2024-09-10T09:35:36.614901+0000","last_unstale":"2024-09-10T09:37:07.524306+0000","last_undegraded":"2024-09-10T09:37:07.524306+0000","last_fullsized":"2024-09-10T09:37:07.524306+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:56:54.248266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.761302+0000","last_change":"2024-09-10T09:35:36.629021+0000","last_active":"2024-09-10T09:37:07.761302+0000","last_peered":"2024-09-10T09:37:07.761302+0000","last_clean":"2024-09-10T09:37:07.761302+0000","last_became_active":"2024-09-10T09:35:36.628159+0000","last_became_peered":"2024-09-10T09:35:36.628159+0000","last_unstale":"2024-09-10T09:37:07.761302+0000","last_undegraded":"2024-09-10T09:37:07.761302+0000","last_fullsized":"2024-09-10T09:37:07.761302+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:50:48.615236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"96'11","reported_seq":48,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.545215+0000","last_change":"2024-09-10T09:35:28.579333+0000","last_active":"2024-09-10T09:37:35.545215+0000","last_peered":"2024-09-10T09:37:35.545215+0000","last_clean":"2024-09-10T09:37:35.545215+0000","last_became_active":"2024-09-10T09:35:28.578440+0000","last_became_peered":"2024-09-10T09:35:28.578440+0000","last_unstale":"2024-09-10T09:37:35.545215+0000","last_undegraded":"2024-09-10T09:37:35.545215+0000","last_fullsized":"2024-09-10T09:37:35.545215+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:07:28.701938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"94'3","reported_seq":48,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.531063+0000","last_change":"2024-09-10T09:35:25.530857+0000","last_active":"2024-09-10T09:37:07.531063+0000","last_peered":"2024-09-10T09:37:07.531063+0000","last_clean":"2024-09-10T09:37:07.531063+0000","last_became_active":"2024-09-10T09:35:25.530492+0000","last_became_peered":"2024-09-10T09:35:25.530492+0000","last_unstale":"2024-09-10T09:37:07.531063+0000","last_undegraded":"2024-09-10T09:37:07.531063+0000","last_fullsized":"2024-09-10T09:37:07.531063+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:07:21.726598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":36,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.526432+0000","last_change":"2024-09-10T09:36:06.770408+0000","last_active":"2024-09-10T09:37:07.526432+0000","last_peered":"2024-09-10T09:37:07.526432+0000","last_clean":"2024-09-10T09:37:07.526432+0000","last_became_active":"2024-09-10T09:36:06.769915+0000","last_became_peered":"2024-09-10T09:36:06.769915+0000","last_unstale":"2024-09-10T09:37:07.526432+0000","last_undegraded":"2024-09-10T09:37:07.526432+0000","last_fullsized":"2024-09-10T09:37:07.526432+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:56:09.795537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8,0],"acting":[11,4,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"96'8","reported_seq":47,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.543491+0000","last_change":"2024-09-10T09:35:28.570804+0000","last_active":"2024-09-10T09:37:35.543491+0000","last_peered":"2024-09-10T09:37:35.543491+0000","last_clean":"2024-09-10T09:37:35.543491+0000","last_became_active":"2024-09-10T09:35:28.570490+0000","last_became_peered":"2024-09-10T09:35:28.570490+0000","last_unstale":"2024-09-10T09:37:35.543491+0000","last_undegraded":"2024-09-10T09:37:35.543491+0000","last_fullsized":"2024-09-10T09:37:35.543491+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:13:38.999625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"94'3","reported_seq":47,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.538446+0000","last_change":"2024-09-10T09:35:25.536347+0000","last_active":"2024-09-10T09:37:07.538446+0000","last_peered":"2024-09-10T09:37:07.538446+0000","last_clean":"2024-09-10T09:37:07.538446+0000","last_became_active":"2024-09-10T09:35:25.535964+0000","last_became_peered":"2024-09-10T09:35:25.535964+0000","last_unstale":"2024-09-10T09:37:07.538446+0000","last_undegraded":"2024-09-10T09:37:07.538446+0000","last_fullsized":"2024-09-10T09:37:07.538446+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:36:49.035666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"94'7","reported_seq":51,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.525812+0000","last_change":"2024-09-10T09:35:25.532148+0000","last_active":"2024-09-10T09:37:07.525812+0000","last_peered":"2024-09-10T09:37:07.525812+0000","last_clean":"2024-09-10T09:37:07.525812+0000","last_became_active":"2024-09-10T09:35:25.531781+0000","last_became_peered":"2024-09-10T09:35:25.531781+0000","last_unstale":"2024-09-10T09:37:07.525812+0000","last_undegraded":"2024-09-10T09:37:07.525812+0000","last_fullsized":"2024-09-10T09:37:07.525812+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:52:46.740342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"96'8","reported_seq":47,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.549412+0000","last_change":"2024-09-10T09:35:28.578336+0000","last_active":"2024-09-10T09:37:35.549412+0000","last_peered":"2024-09-10T09:37:35.549412+0000","last_clean":"2024-09-10T09:37:35.549412+0000","last_became_active":"2024-09-10T09:35:28.577899+0000","last_became_peered":"2024-09-10T09:35:28.577899+0000","last_unstale":"2024-09-10T09:37:35.549412+0000","last_undegraded":"2024-09-10T09:37:35.549412+0000","last_fullsized":"2024-09-10T09:37:35.549412+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:13:04.478195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.528122+0000","last_change":"2024-09-10T09:35:36.630918+0000","last_active":"2024-09-10T09:37:07.528122+0000","last_peered":"2024-09-10T09:37:07.528122+0000","last_clean":"2024-09-10T09:37:07.528122+0000","last_became_active":"2024-09-10T09:35:36.630308+0000","last_became_peered":"2024-09-10T09:35:36.630308+0000","last_unstale":"2024-09-10T09:37:07.528122+0000","last_undegraded":"2024-09-10T09:37:07.528122+0000","last_fullsized":"2024-09-10T09:37:07.528122+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:25:44.954970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"96'7","reported_seq":201,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:38:27.684031+0000","last_change":"2024-09-10T09:37:07.574386+0000","last_active":"2024-09-10T09:38:27.684031+0000","last_peered":"2024-09-10T09:38:27.684031+0000","last_clean":"2024-09-10T09:38:27.684031+0000","last_became_active":"2024-09-10T09:37:07.569370+0000","last_became_peered":"2024-09-10T09:37:07.569370+0000","last_unstale":"2024-09-10T09:38:27.684031+0000","last_undegraded":"2024-09-10T09:38:27.684031+0000","last_fullsized":"2024-09-10T09:38:27.684031+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:24.513528+0000","last_clean_scrub_stamp":"2024-09-10T09:35:24.513528+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:07:23.014789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1156,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":133,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":1015,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"96'7","reported_seq":46,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.543648+0000","last_change":"2024-09-10T09:35:28.567915+0000","last_active":"2024-09-10T09:37:35.543648+0000","last_peered":"2024-09-10T09:37:35.543648+0000","last_clean":"2024-09-10T09:37:35.543648+0000","last_became_active":"2024-09-10T09:35:28.567416+0000","last_became_peered":"2024-09-10T09:35:28.567416+0000","last_unstale":"2024-09-10T09:37:35.543648+0000","last_undegraded":"2024-09-10T09:37:35.543648+0000","last_fullsized":"2024-09-10T09:37:35.543648+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:07:25.704080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":29,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:07.528686+0000","last_change":"2024-09-10T09:35:36.615386+0000","last_active":"2024-09-10T09:37:07.528686+0000","last_peered":"2024-09-10T09:37:07.528686+0000","last_clean":"2024-09-10T09:37:07.528686+0000","last_became_active":"2024-09-10T09:35:36.614967+0000","last_became_peered":"2024-09-10T09:35:36.614967+0000","last_unstale":"2024-09-10T09:37:07.528686+0000","last_undegraded":"2024-09-10T09:37:07.528686+0000","last_fullsized":"2024-09-10T09:37:07.528686+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:35.593391+0000","last_clean_scrub_stamp":"2024-09-10T09:35:35.593391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:11:47.620729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"96'9","reported_seq":48,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.544481+0000","last_change":"2024-09-10T09:35:28.564411+0000","last_active":"2024-09-10T09:37:35.544481+0000","last_peered":"2024-09-10T09:37:35.544481+0000","last_clean":"2024-09-10T09:37:35.544481+0000","last_became_active":"2024-09-10T09:35:28.564034+0000","last_became_peered":"2024-09-10T09:35:28.564034+0000","last_unstale":"2024-09-10T09:37:35.544481+0000","last_undegraded":"2024-09-10T09:37:35.544481+0000","last_fullsized":"2024-09-10T09:37:35.544481+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:38:17.687991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"96'12","reported_seq":49,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.547277+0000","last_change":"2024-09-10T09:35:28.564619+0000","last_active":"2024-09-10T09:37:35.547277+0000","last_peered":"2024-09-10T09:37:35.547277+0000","last_clean":"2024-09-10T09:37:35.547277+0000","last_became_active":"2024-09-10T09:35:28.563890+0000","last_became_peered":"2024-09-10T09:35:28.563890+0000","last_unstale":"2024-09-10T09:37:35.547277+0000","last_undegraded":"2024-09-10T09:37:35.547277+0000","last_fullsized":"2024-09-10T09:37:35.547277+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:51:42.481911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"96'8","reported_seq":45,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.543410+0000","last_change":"2024-09-10T09:35:28.558909+0000","last_active":"2024-09-10T09:37:35.543410+0000","last_peered":"2024-09-10T09:37:35.543410+0000","last_clean":"2024-09-10T09:37:35.543410+0000","last_became_active":"2024-09-10T09:35:28.558604+0000","last_became_peered":"2024-09-10T09:35:28.558604+0000","last_unstale":"2024-09-10T09:37:35.543410+0000","last_undegraded":"2024-09-10T09:37:35.543410+0000","last_fullsized":"2024-09-10T09:37:35.543410+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:08:11.928446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"96'7","reported_seq":44,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.554844+0000","last_change":"2024-09-10T09:35:28.574459+0000","last_active":"2024-09-10T09:37:35.554844+0000","last_peered":"2024-09-10T09:37:35.554844+0000","last_clean":"2024-09-10T09:37:35.554844+0000","last_became_active":"2024-09-10T09:35:28.574154+0000","last_became_peered":"2024-09-10T09:35:28.574154+0000","last_unstale":"2024-09-10T09:37:35.554844+0000","last_undegraded":"2024-09-10T09:37:35.554844+0000","last_fullsized":"2024-09-10T09:37:35.554844+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:59:09.650536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"96'4","reported_seq":43,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.552571+0000","last_change":"2024-09-10T09:35:28.576999+0000","last_active":"2024-09-10T09:37:35.552571+0000","last_peered":"2024-09-10T09:37:35.552571+0000","last_clean":"2024-09-10T09:37:35.552571+0000","last_became_active":"2024-09-10T09:35:28.575616+0000","last_became_peered":"2024-09-10T09:35:28.575616+0000","last_unstale":"2024-09-10T09:37:35.552571+0000","last_undegraded":"2024-09-10T09:37:35.552571+0000","last_fullsized":"2024-09-10T09:37:35.552571+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:18:08.771760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"96'6","reported_seq":45,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.541496+0000","last_change":"2024-09-10T09:35:28.564846+0000","last_active":"2024-09-10T09:37:35.541496+0000","last_peered":"2024-09-10T09:37:35.541496+0000","last_clean":"2024-09-10T09:37:35.541496+0000","last_became_active":"2024-09-10T09:35:28.564498+0000","last_became_peered":"2024-09-10T09:35:28.564498+0000","last_unstale":"2024-09-10T09:37:35.541496+0000","last_undegraded":"2024-09-10T09:37:35.541496+0000","last_fullsized":"2024-09-10T09:37:35.541496+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:33:43.041730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"96'15","reported_seq":52,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.550975+0000","last_change":"2024-09-10T09:35:28.558093+0000","last_active":"2024-09-10T09:37:35.550975+0000","last_peered":"2024-09-10T09:37:35.550975+0000","last_clean":"2024-09-10T09:37:35.550975+0000","last_became_active":"2024-09-10T09:35:28.557266+0000","last_became_peered":"2024-09-10T09:35:28.557266+0000","last_unstale":"2024-09-10T09:37:35.550975+0000","last_undegraded":"2024-09-10T09:37:35.550975+0000","last_fullsized":"2024-09-10T09:37:35.550975+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:51:30.946065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"96'2","reported_seq":47,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.549558+0000","last_change":"2024-09-10T09:36:06.770298+0000","last_active":"2024-09-10T09:37:35.549558+0000","last_peered":"2024-09-10T09:37:35.549558+0000","last_clean":"2024-09-10T09:37:35.549558+0000","last_became_active":"2024-09-10T09:36:06.769848+0000","last_became_peered":"2024-09-10T09:36:06.769848+0000","last_unstale":"2024-09-10T09:37:35.549558+0000","last_undegraded":"2024-09-10T09:37:35.549558+0000","last_fullsized":"2024-09-10T09:37:35.549558+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:00:30.470887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,8,2],"acting":[3,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"96'6","reported_seq":45,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.549553+0000","last_change":"2024-09-10T09:35:28.578158+0000","last_active":"2024-09-10T09:37:35.549553+0000","last_peered":"2024-09-10T09:37:35.549553+0000","last_clean":"2024-09-10T09:37:35.549553+0000","last_became_active":"2024-09-10T09:35:28.577824+0000","last_became_peered":"2024-09-10T09:35:28.577824+0000","last_unstale":"2024-09-10T09:37:35.549553+0000","last_undegraded":"2024-09-10T09:37:35.549553+0000","last_fullsized":"2024-09-10T09:37:35.549553+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:40:27.231164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"96'8","reported_seq":47,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.550143+0000","last_change":"2024-09-10T09:35:28.562670+0000","last_active":"2024-09-10T09:37:35.550143+0000","last_peered":"2024-09-10T09:37:35.550143+0000","last_clean":"2024-09-10T09:37:35.550143+0000","last_became_active":"2024-09-10T09:35:28.562311+0000","last_became_peered":"2024-09-10T09:35:28.562311+0000","last_unstale":"2024-09-10T09:37:35.550143+0000","last_undegraded":"2024-09-10T09:37:35.550143+0000","last_fullsized":"2024-09-10T09:37:35.550143+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:07:11.716142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"96'10","reported_seq":49,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.556330+0000","last_change":"2024-09-10T09:35:28.561124+0000","last_active":"2024-09-10T09:37:35.556330+0000","last_peered":"2024-09-10T09:37:35.556330+0000","last_clean":"2024-09-10T09:37:35.556330+0000","last_became_active":"2024-09-10T09:35:28.560777+0000","last_became_peered":"2024-09-10T09:35:28.560777+0000","last_unstale":"2024-09-10T09:37:35.556330+0000","last_undegraded":"2024-09-10T09:37:35.556330+0000","last_fullsized":"2024-09-10T09:37:35.556330+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:47:04.837596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"96'11","reported_seq":50,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.556539+0000","last_change":"2024-09-10T09:35:28.576790+0000","last_active":"2024-09-10T09:37:35.556539+0000","last_peered":"2024-09-10T09:37:35.556539+0000","last_clean":"2024-09-10T09:37:35.556539+0000","last_became_active":"2024-09-10T09:35:28.576464+0000","last_became_peered":"2024-09-10T09:35:28.576464+0000","last_unstale":"2024-09-10T09:37:35.556539+0000","last_undegraded":"2024-09-10T09:37:35.556539+0000","last_fullsized":"2024-09-10T09:37:35.556539+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:40:53.517577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"96'8","reported_seq":47,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.539417+0000","last_change":"2024-09-10T09:35:28.569935+0000","last_active":"2024-09-10T09:37:35.539417+0000","last_peered":"2024-09-10T09:37:35.539417+0000","last_clean":"2024-09-10T09:37:35.539417+0000","last_became_active":"2024-09-10T09:35:28.569539+0000","last_became_peered":"2024-09-10T09:35:28.569539+0000","last_unstale":"2024-09-10T09:37:35.539417+0000","last_undegraded":"2024-09-10T09:37:35.539417+0000","last_fullsized":"2024-09-10T09:37:35.539417+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:40:45.016269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"96'7","reported_seq":44,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.550145+0000","last_change":"2024-09-10T09:35:28.571619+0000","last_active":"2024-09-10T09:37:35.550145+0000","last_peered":"2024-09-10T09:37:35.550145+0000","last_clean":"2024-09-10T09:37:35.550145+0000","last_became_active":"2024-09-10T09:35:28.568591+0000","last_became_peered":"2024-09-10T09:35:28.568591+0000","last_unstale":"2024-09-10T09:37:35.550145+0000","last_undegraded":"2024-09-10T09:37:35.550145+0000","last_fullsized":"2024-09-10T09:37:35.550145+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:07:49.959712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"96'10","reported_seq":47,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.551796+0000","last_change":"2024-09-10T09:35:28.573336+0000","last_active":"2024-09-10T09:37:35.551796+0000","last_peered":"2024-09-10T09:37:35.551796+0000","last_clean":"2024-09-10T09:37:35.551796+0000","last_became_active":"2024-09-10T09:35:28.572991+0000","last_became_peered":"2024-09-10T09:35:28.572991+0000","last_unstale":"2024-09-10T09:37:35.551796+0000","last_undegraded":"2024-09-10T09:37:35.551796+0000","last_fullsized":"2024-09-10T09:37:35.551796+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:30:02.905869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"96'8","reported_seq":47,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.552930+0000","last_change":"2024-09-10T09:35:28.571945+0000","last_active":"2024-09-10T09:37:35.552930+0000","last_peered":"2024-09-10T09:37:35.552930+0000","last_clean":"2024-09-10T09:37:35.552930+0000","last_became_active":"2024-09-10T09:35:28.571546+0000","last_became_peered":"2024-09-10T09:35:28.571546+0000","last_unstale":"2024-09-10T09:37:35.552930+0000","last_undegraded":"2024-09-10T09:37:35.552930+0000","last_fullsized":"2024-09-10T09:37:35.552930+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:36:47.946864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"96'9","reported_seq":48,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.556725+0000","last_change":"2024-09-10T09:35:28.562712+0000","last_active":"2024-09-10T09:37:35.556725+0000","last_peered":"2024-09-10T09:37:35.556725+0000","last_clean":"2024-09-10T09:37:35.556725+0000","last_became_active":"2024-09-10T09:35:28.562396+0000","last_became_peered":"2024-09-10T09:35:28.562396+0000","last_unstale":"2024-09-10T09:37:35.556725+0000","last_undegraded":"2024-09-10T09:37:35.556725+0000","last_fullsized":"2024-09-10T09:37:35.556725+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:25:47.419162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"96'7","reported_seq":44,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.553427+0000","last_change":"2024-09-10T09:35:28.566354+0000","last_active":"2024-09-10T09:37:35.553427+0000","last_peered":"2024-09-10T09:37:35.553427+0000","last_clean":"2024-09-10T09:37:35.553427+0000","last_became_active":"2024-09-10T09:35:28.565248+0000","last_became_peered":"2024-09-10T09:35:28.565248+0000","last_unstale":"2024-09-10T09:37:35.553427+0000","last_undegraded":"2024-09-10T09:37:35.553427+0000","last_fullsized":"2024-09-10T09:37:35.553427+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:48:46.741010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"96'7","reported_seq":44,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.550936+0000","last_change":"2024-09-10T09:35:28.563331+0000","last_active":"2024-09-10T09:37:35.550936+0000","last_peered":"2024-09-10T09:37:35.550936+0000","last_clean":"2024-09-10T09:37:35.550936+0000","last_became_active":"2024-09-10T09:35:28.563020+0000","last_became_peered":"2024-09-10T09:35:28.563020+0000","last_unstale":"2024-09-10T09:37:35.550936+0000","last_undegraded":"2024-09-10T09:37:35.550936+0000","last_fullsized":"2024-09-10T09:37:35.550936+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:54:27.653551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"96'10","reported_seq":49,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.542747+0000","last_change":"2024-09-10T09:35:28.566687+0000","last_active":"2024-09-10T09:37:35.542747+0000","last_peered":"2024-09-10T09:37:35.542747+0000","last_clean":"2024-09-10T09:37:35.542747+0000","last_became_active":"2024-09-10T09:35:28.566253+0000","last_became_peered":"2024-09-10T09:35:28.566253+0000","last_unstale":"2024-09-10T09:37:35.542747+0000","last_undegraded":"2024-09-10T09:37:35.542747+0000","last_fullsized":"2024-09-10T09:37:35.542747+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:00:15.709725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"96'7","reported_seq":44,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.538923+0000","last_change":"2024-09-10T09:35:28.562251+0000","last_active":"2024-09-10T09:37:35.538923+0000","last_peered":"2024-09-10T09:37:35.538923+0000","last_clean":"2024-09-10T09:37:35.538923+0000","last_became_active":"2024-09-10T09:35:28.561933+0000","last_became_peered":"2024-09-10T09:35:28.561933+0000","last_unstale":"2024-09-10T09:37:35.538923+0000","last_undegraded":"2024-09-10T09:37:35.538923+0000","last_fullsized":"2024-09-10T09:37:35.538923+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:30:41.447954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"96'6","reported_seq":43,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.556126+0000","last_change":"2024-09-10T09:35:28.572164+0000","last_active":"2024-09-10T09:37:35.556126+0000","last_peered":"2024-09-10T09:37:35.556126+0000","last_clean":"2024-09-10T09:37:35.556126+0000","last_became_active":"2024-09-10T09:35:28.571825+0000","last_became_peered":"2024-09-10T09:35:28.571825+0000","last_unstale":"2024-09-10T09:37:35.556126+0000","last_undegraded":"2024-09-10T09:37:35.556126+0000","last_fullsized":"2024-09-10T09:37:35.556126+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:59:50.967269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"96'16","reported_seq":55,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.550138+0000","last_change":"2024-09-10T09:35:28.568098+0000","last_active":"2024-09-10T09:37:35.550138+0000","last_peered":"2024-09-10T09:37:35.550138+0000","last_clean":"2024-09-10T09:37:35.550138+0000","last_became_active":"2024-09-10T09:35:28.567765+0000","last_became_peered":"2024-09-10T09:35:28.567765+0000","last_unstale":"2024-09-10T09:37:35.550138+0000","last_undegraded":"2024-09-10T09:37:35.550138+0000","last_fullsized":"2024-09-10T09:37:35.550138+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:55:57.260320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"96'11","reported_seq":50,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.553598+0000","last_change":"2024-09-10T09:35:28.564123+0000","last_active":"2024-09-10T09:37:35.553598+0000","last_peered":"2024-09-10T09:37:35.553598+0000","last_clean":"2024-09-10T09:37:35.553598+0000","last_became_active":"2024-09-10T09:35:28.563814+0000","last_became_peered":"2024-09-10T09:35:28.563814+0000","last_unstale":"2024-09-10T09:37:35.553598+0000","last_undegraded":"2024-09-10T09:37:35.553598+0000","last_fullsized":"2024-09-10T09:37:35.553598+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:14:17.912087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"96'9","reported_seq":48,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.549407+0000","last_change":"2024-09-10T09:35:28.568601+0000","last_active":"2024-09-10T09:37:35.549407+0000","last_peered":"2024-09-10T09:37:35.549407+0000","last_clean":"2024-09-10T09:37:35.549407+0000","last_became_active":"2024-09-10T09:35:28.568303+0000","last_became_peered":"2024-09-10T09:35:28.568303+0000","last_unstale":"2024-09-10T09:37:35.549407+0000","last_undegraded":"2024-09-10T09:37:35.549407+0000","last_fullsized":"2024-09-10T09:37:35.549407+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:30:01.016420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"96'5","reported_seq":48,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.556713+0000","last_change":"2024-09-10T09:37:07.539577+0000","last_active":"2024-09-10T09:37:35.556713+0000","last_peered":"2024-09-10T09:37:35.556713+0000","last_clean":"2024-09-10T09:37:35.556713+0000","last_became_active":"2024-09-10T09:37:07.539206+0000","last_became_peered":"2024-09-10T09:37:07.539206+0000","last_unstale":"2024-09-10T09:37:35.556713+0000","last_undegraded":"2024-09-10T09:37:35.556713+0000","last_fullsized":"2024-09-10T09:37:35.556713+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:56:06.568226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"96'6","reported_seq":45,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.541193+0000","last_change":"2024-09-10T09:35:28.569225+0000","last_active":"2024-09-10T09:37:35.541193+0000","last_peered":"2024-09-10T09:37:35.541193+0000","last_clean":"2024-09-10T09:37:35.541193+0000","last_became_active":"2024-09-10T09:35:28.568891+0000","last_became_peered":"2024-09-10T09:35:28.568891+0000","last_unstale":"2024-09-10T09:37:35.541193+0000","last_undegraded":"2024-09-10T09:37:35.541193+0000","last_fullsized":"2024-09-10T09:37:35.541193+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:33:45.811781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"96'4","reported_seq":43,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.544182+0000","last_change":"2024-09-10T09:35:28.578362+0000","last_active":"2024-09-10T09:37:35.544182+0000","last_peered":"2024-09-10T09:37:35.544182+0000","last_clean":"2024-09-10T09:37:35.544182+0000","last_became_active":"2024-09-10T09:35:28.577867+0000","last_became_peered":"2024-09-10T09:35:28.577867+0000","last_unstale":"2024-09-10T09:37:35.544182+0000","last_undegraded":"2024-09-10T09:37:35.544182+0000","last_fullsized":"2024-09-10T09:37:35.544182+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:58:17.041663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"96'8","reported_seq":45,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.537018+0000","last_change":"2024-09-10T09:35:28.568183+0000","last_active":"2024-09-10T09:37:35.537018+0000","last_peered":"2024-09-10T09:37:35.537018+0000","last_clean":"2024-09-10T09:37:35.537018+0000","last_became_active":"2024-09-10T09:35:28.566775+0000","last_became_peered":"2024-09-10T09:35:28.566775+0000","last_unstale":"2024-09-10T09:37:35.537018+0000","last_undegraded":"2024-09-10T09:37:35.537018+0000","last_fullsized":"2024-09-10T09:37:35.537018+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:28:10.008889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"96'7","reported_seq":44,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.553233+0000","last_change":"2024-09-10T09:35:28.576122+0000","last_active":"2024-09-10T09:37:35.553233+0000","last_peered":"2024-09-10T09:37:35.553233+0000","last_clean":"2024-09-10T09:37:35.553233+0000","last_became_active":"2024-09-10T09:35:28.575831+0000","last_became_peered":"2024-09-10T09:35:28.575831+0000","last_unstale":"2024-09-10T09:37:35.553233+0000","last_undegraded":"2024-09-10T09:37:35.553233+0000","last_fullsized":"2024-09-10T09:37:35.553233+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:37:51.604658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"96'4","reported_seq":41,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.554017+0000","last_change":"2024-09-10T09:35:28.564441+0000","last_active":"2024-09-10T09:37:35.554017+0000","last_peered":"2024-09-10T09:37:35.554017+0000","last_clean":"2024-09-10T09:37:35.554017+0000","last_became_active":"2024-09-10T09:35:28.564095+0000","last_became_peered":"2024-09-10T09:35:28.564095+0000","last_unstale":"2024-09-10T09:37:35.554017+0000","last_undegraded":"2024-09-10T09:37:35.554017+0000","last_fullsized":"2024-09-10T09:37:35.554017+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:53:28.275361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"96'8","reported_seq":47,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-10T09:37:35.553155+0000","last_change":"2024-09-10T09:35:28.581921+0000","last_active":"2024-09-10T09:37:35.553155+0000","last_peered":"2024-09-10T09:37:35.553155+0000","last_clean":"2024-09-10T09:37:35.553155+0000","last_became_active":"2024-09-10T09:35:28.581573+0000","last_became_peered":"2024-09-10T09:35:28.581573+0000","last_unstale":"2024-09-10T09:37:35.553155+0000","last_undegraded":"2024-09-10T09:37:35.553155+0000","last_fullsized":"2024-09-10T09:37:35.553155+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T09:35:27.536533+0000","last_clean_scrub_stamp":"2024-09-10T09:35:27.536533+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:58:38.983934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":79657,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1391,"num_read_kb":787,"num_write":271,"num_write_kb":166,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":1015,"num_keys_recovered":7,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":602112,"data_stored":238971,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":232,"ondisk_log_size":232,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":285627,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":77,"seq":330712481843,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32228,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712924,"statfs":{"total":95995035648,"available":95962034176,"internally_reserved":0,"allocated":1069056,"data_stored":554290,"data_compressed":32456,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":70,"seq":300647710775,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28228,"kb_used_data":1204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716924,"statfs":{"total":95995035648,"available":95966130176,"internally_reserved":0,"allocated":1232896,"data_stored":1217058,"data_compressed":68777,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":65,"seq":279172874302,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32252,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712900,"statfs":{"total":95995035648,"available":95962009600,"internally_reserved":0,"allocated":1093632,"data_stored":554220,"data_compressed":32456,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1599,"internal_metadata":31914433},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":57,"seq":244813135938,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32248,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712904,"statfs":{"total":95995035648,"available":95962013696,"internally_reserved":0,"allocated":1089536,"data_stored":554598,"data_compressed":32456,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":50,"seq":214748364872,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28140,"kb_used_data":1116,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717012,"statfs":{"total":95995035648,"available":95966220288,"internally_reserved":0,"allocated":1142784,"data_stored":1152083,"data_compressed":68777,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":45,"seq":193273528397,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36388,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708764,"statfs":{"total":95995035648,"available":95957774336,"internally_reserved":0,"allocated":1069056,"data_stored":550569,"data_compressed":32456,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790036,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28148,"kb_used_data":1124,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717004,"statfs":{"total":95995035648,"available":95966212096,"internally_reserved":0,"allocated":1150976,"data_stored":1146820,"data_compressed":68777,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953561,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32244,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712908,"statfs":{"total":95995035648,"available":95962017792,"internally_reserved":0,"allocated":1085440,"data_stored":554954,"data_compressed":32456,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182494,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28080,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717072,"statfs":{"total":95995035648,"available":95966281728,"internally_reserved":0,"allocated":1081344,"data_stored":549432,"data_compressed":32456,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411427,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28156,"kb_used_data":1132,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716996,"statfs":{"total":95995035648,"available":95966203904,"internally_reserved":0,"allocated":1159168,"data_stored":622208,"data_compressed":32456,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574953,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36464,"kb_used_data":1120,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708688,"statfs":{"total":95995035648,"available":95957696512,"internally_reserved":0,"allocated":1146880,"data_stored":622317,"data_compressed":32456,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738477,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36412,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708740,"statfs":{"total":95995035648,"available":95957749760,"internally_reserved":0,"allocated":1093632,"data_stored":551180,"data_compressed":32456,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34674,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34674,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34674,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36321,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36321,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36321,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36321,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36321,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":74453,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":73206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3354,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1567,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4521,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":126976,"data_stored":73592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T09:38:33.989 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T09:38:34.306 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:38:33 smithi005 ceph-mon[30350]: from='client.15330 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T09:38:34.306 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:38:33 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:38:34.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:38:33 smithi159 ceph-mon[35513]: from='client.15330 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T09:38:34.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:38:33 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:38:34.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:38:33 smithi102 ceph-mon[35442]: from='client.15330 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T09:38:34.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:38:33 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:38:34.387 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:38:34.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:34.973+0000 7f27cae00640 1 -- 172.21.15.5:0/485947459 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f27c4103e30 msgr2=0x7f27c4104210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:38:34.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:34.973+0000 7f27cae00640 1 --2- 172.21.15.5:0/485947459 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f27c4103e30 0x7f27c4104210 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f27b8009900 tx=0x7f27b802a400 comp rx=0 tx=0).stop 2024-09-10T09:38:34.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:34.973+0000 7f27cae00640 1 -- 172.21.15.5:0/485947459 shutdown_connections 2024-09-10T09:38:34.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:34.973+0000 7f27cae00640 1 --2- 172.21.15.5:0/485947459 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f27c4108f60 0x7f27c4110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:34.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:34.973+0000 7f27cae00640 1 --2- 172.21.15.5:0/485947459 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f27c41047e0 0x7f27c4108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:34.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:34.973+0000 7f27cae00640 1 --2- 172.21.15.5:0/485947459 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f27c4103e30 0x7f27c4104210 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:34.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:34.973+0000 7f27cae00640 1 -- 172.21.15.5:0/485947459 >> 172.21.15.5:0/485947459 conn(0x7f27c40ff910 msgr2=0x7f27c4101d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:38:34.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:34.973+0000 7f27cae00640 1 -- 172.21.15.5:0/485947459 shutdown_connections 2024-09-10T09:38:34.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:34.973+0000 7f27cae00640 1 -- 172.21.15.5:0/485947459 wait complete. 2024-09-10T09:38:34.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:34.974+0000 7f27cae00640 1 Processor -- start 2024-09-10T09:38:34.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:34.974+0000 7f27cae00640 1 -- start start 2024-09-10T09:38:34.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:34.975+0000 7f27cae00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f27c4103e30 0x7f27c419a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:34.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:34.975+0000 7f27cae00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f27c41047e0 0x7f27c419a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:34.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:34.975+0000 7f27cae00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f27c4108f60 0x7f27c419ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:34.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:34.975+0000 7f27cae00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f27c4114010 con 0x7f27c41047e0 2024-09-10T09:38:34.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:34.975+0000 7f27cae00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f27c4113e90 con 0x7f27c4103e30 2024-09-10T09:38:34.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:34.975+0000 7f27cae00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f27c4114190 con 0x7f27c4108f60 2024-09-10T09:38:34.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:34.975+0000 7f27c9a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f27c4103e30 0x7f27c419a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:34.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:34.975+0000 7f27c9a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f27c4103e30 0x7f27c419a470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:38590/0 (socket says 172.21.15.5:38590) 2024-09-10T09:38:34.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:34.975+0000 7f27c9000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f27c41047e0 0x7f27c419a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:34.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:34.975+0000 7f27c9a00640 1 -- 172.21.15.5:0/1228247652 learned_addr learned my addr 172.21.15.5:0/1228247652 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:38:34.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:34.975+0000 7f27ca400640 1 --2- 172.21.15.5:0/1228247652 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f27c4108f60 0x7f27c419ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:34.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:34.976+0000 7f27c9a00640 1 -- 172.21.15.5:0/1228247652 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f27c4108f60 msgr2=0x7f27c419ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:38:34.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:34.976+0000 7f27c9a00640 1 --2- 172.21.15.5:0/1228247652 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f27c4108f60 0x7f27c419ed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:34.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:34.976+0000 7f27c9a00640 1 -- 172.21.15.5:0/1228247652 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f27c41047e0 msgr2=0x7f27c419a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:38:34.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:34.976+0000 7f27c9a00640 1 --2- 172.21.15.5:0/1228247652 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f27c41047e0 0x7f27c419a9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:34.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:34.976+0000 7f27c9a00640 1 -- 172.21.15.5:0/1228247652 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27c419f420 con 0x7f27c4103e30 2024-09-10T09:38:34.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:34.976+0000 7f27c9000640 1 --2- 172.21.15.5:0/1228247652 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f27c41047e0 0x7f27c419a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:38:34.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:34.976+0000 7f27c9a00640 1 --2- 172.21.15.5:0/1228247652 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f27c4103e30 0x7f27c419a470 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f27b8004770 tx=0x7f27b8003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:38:34.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:34.976+0000 7f27c2000640 1 -- 172.21.15.5:0/1228247652 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27b803f070 con 0x7f27c4103e30 2024-09-10T09:38:34.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:34.976+0000 7f27cae00640 1 -- 172.21.15.5:0/1228247652 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27c419f6b0 con 0x7f27c4103e30 2024-09-10T09:38:34.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:34.976+0000 7f27c2000640 1 -- 172.21.15.5:0/1228247652 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f27b8031ca0 con 0x7f27c4103e30 2024-09-10T09:38:34.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:34.976+0000 7f27c2000640 1 -- 172.21.15.5:0/1228247652 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27b8035030 con 0x7f27c4103e30 2024-09-10T09:38:34.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:34.977+0000 7f27cae00640 1 -- 172.21.15.5:0/1228247652 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27c41a6ff0 con 0x7f27c4103e30 2024-09-10T09:38:34.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:34.978+0000 7f27cae00640 1 -- 172.21.15.5:0/1228247652 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f27c4105310 con 0x7f27c4103e30 2024-09-10T09:38:34.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:34.980+0000 7f27c2000640 1 -- 172.21.15.5:0/1228247652 <== mon.2 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 19) ==== 137883+0+0 (secure 0 0 0) 0x7f27b8042070 con 0x7f27c4103e30 2024-09-10T09:38:34.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:34.981+0000 7f27c2000640 1 --2- 172.21.15.5:0/1228247652 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f27ac0a2c90 0x7f27ac0a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:34.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:34.981+0000 7f27c2000640 1 -- 172.21.15.5:0/1228247652 <== mon.2 v2:172.21.15.102:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f27b80efec0 con 0x7f27c4103e30 2024-09-10T09:38:34.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:34.981+0000 7f27c9000640 1 --2- 172.21.15.5:0/1228247652 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f27ac0a2c90 0x7f27ac0a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:34.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:34.982+0000 7f27c9000640 1 --2- 172.21.15.5:0/1228247652 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f27ac0a2c90 0x7f27ac0a5150 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f27c40ffd30 tx=0x7f27b0005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:38:34.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:34.983+0000 7f27c2000640 1 -- 172.21.15.5:0/1228247652 <== mon.2 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f27b80bc2e0 con 0x7f27c4103e30 2024-09-10T09:38:35.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:35.225+0000 7f27cae00640 1 -- 172.21.15.5:0/1228247652 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f27c4104210 con 0x7f27c4103e30 2024-09-10T09:38:35.226 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:38:35 smithi005 ceph-mon[30350]: pgmap v400: 129 pgs: 129 active+clean; 655 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 170 B/s wr, 8 op/s 2024-09-10T09:38:35.226 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:38:35 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:38:35.226 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:38:35 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:38:35.226 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:38:35 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:38:35.226 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:38:35 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:38:35.227 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:38:35 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:38:35.227 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:38:35 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:38:35.227 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:38:35 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:38:35.227 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:38:35 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:38:35.227 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:38:35 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:38:35.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:35.226+0000 7f27c2000640 1 -- 172.21.15.5:0/1228247652 <== mon.2 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11969 (secure 0 0 0) 0x7f27b80c05e0 con 0x7f27c4103e30 2024-09-10T09:38:35.227 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-10T09:38:35.228 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 30 2024-09-10T09:38:35.228 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":30,"btime":"2024-09-10T09:38:15:834198+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14454,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/2970594311","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2970594311},{"type":"v1","addr":"172.21.15.5:6835","nonce":2970594311}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14514,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6839/3616502174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":3616502174},{"type":"v1","addr":"172.21.15.5:6839","nonce":3616502174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T09:35:29.626287+0000","modified":"2024-09-10T09:38:15.658058+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24542,"mds_1":24535,"mds_2":24511,"mds_3":24506,"mds_4":24443},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14520":{"gid":14520,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/2073599842","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":2073599842},{"type":"v1","addr":"172.21.15.5:6841","nonce":2073599842}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"k","rank":4,"incarnation":26,"state":"up:active","state_seq":24,"addr":"172.21.15.102:6833/2457007427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":2457007427},{"type":"v1","addr":"172.21.15.102:6833","nonce":2457007427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/356445159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":356445159},{"type":"v1","addr":"172.21.15.159:6833","nonce":356445159}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6837/2894314507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":2894314507},{"type":"v1","addr":"172.21.15.5:6837","nonce":2894314507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.102:6835/1647549111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":1647549111},{"type":"v1","addr":"172.21.15.102:6835","nonce":1647549111}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6835/952657242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":952657242},{"type":"v1","addr":"172.21.15.159:6835","nonce":952657242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":19,"addr":"172.21.15.102:6837/2689776764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":2689776764},{"type":"v1","addr":"172.21.15.102:6837","nonce":2689776764}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.159:6837/620627639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":620627639},{"type":"v1","addr":"172.21.15.159:6837","nonce":620627639}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24535":{"gid":24535,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":18,"addr":"172.21.15.159:6839/2073739599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":2073739599},{"type":"v1","addr":"172.21.15.159:6839","nonce":2073739599}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24542":{"gid":24542,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.102:6839/335409749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6838","nonce":335409749},{"type":"v1","addr":"172.21.15.102:6839","nonce":335409749}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24542,"qdb_cluster":[24542,24535,24511,24506,24443]},"id":1}]} 2024-09-10T09:38:35.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:35.228+0000 7f27cae00640 1 -- 172.21.15.5:0/1228247652 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f27ac0a2c90 msgr2=0x7f27ac0a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:38:35.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:35.228+0000 7f27cae00640 1 --2- 172.21.15.5:0/1228247652 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f27ac0a2c90 0x7f27ac0a5150 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f27c40ffd30 tx=0x7f27b0005f90 comp rx=0 tx=0).stop 2024-09-10T09:38:35.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:35.228+0000 7f27cae00640 1 -- 172.21.15.5:0/1228247652 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f27c4103e30 msgr2=0x7f27c419a470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:38:35.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:35.228+0000 7f27cae00640 1 --2- 172.21.15.5:0/1228247652 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f27c4103e30 0x7f27c419a470 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f27b8004770 tx=0x7f27b8003c10 comp rx=0 tx=0).stop 2024-09-10T09:38:35.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:35.228+0000 7f27cae00640 1 -- 172.21.15.5:0/1228247652 shutdown_connections 2024-09-10T09:38:35.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:35.228+0000 7f27cae00640 1 --2- 172.21.15.5:0/1228247652 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f27ac0a2c90 0x7f27ac0a5150 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:35.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:35.228+0000 7f27cae00640 1 --2- 172.21.15.5:0/1228247652 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f27c4108f60 0x7f27c419ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:35.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:35.228+0000 7f27cae00640 1 --2- 172.21.15.5:0/1228247652 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f27c41047e0 0x7f27c419a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:35.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:35.228+0000 7f27cae00640 1 --2- 172.21.15.5:0/1228247652 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f27c4103e30 0x7f27c419a470 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:35.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:35.228+0000 7f27cae00640 1 -- 172.21.15.5:0/1228247652 >> 172.21.15.5:0/1228247652 conn(0x7f27c40ff910 msgr2=0x7f27c4100bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:38:35.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:35.228+0000 7f27cae00640 1 -- 172.21.15.5:0/1228247652 shutdown_connections 2024-09-10T09:38:35.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:35.228+0000 7f27cae00640 1 -- 172.21.15.5:0/1228247652 wait complete. 2024-09-10T09:38:35.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:38:35 smithi159 ceph-mon[35513]: pgmap v400: 129 pgs: 129 active+clean; 655 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 170 B/s wr, 8 op/s 2024-09-10T09:38:35.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:38:35 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:38:35.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:38:35 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:38:35.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:38:35 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:38:35.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:38:35 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:38:35.351 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:38:35 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:38:35.351 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:38:35 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:38:35.351 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:38:35 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:38:35.351 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:38:35 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:38:35.351 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:38:35 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:38:35.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:38:35 smithi102 ceph-mon[35442]: pgmap v400: 129 pgs: 129 active+clean; 655 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 170 B/s wr, 8 op/s 2024-09-10T09:38:35.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:38:35 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:38:35.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:38:35 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:38:35.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:38:35 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:38:35.385 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:38:35 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:38:35.385 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:38:35 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:38:35.385 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:38:35 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:38:35.385 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:38:35 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:38:35.385 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:38:35 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:38:35.385 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:38:35 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:38:36.118 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T09:38:36.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:38:36 smithi159 ceph-mon[35513]: from='client.? 172.21.15.5:0/1228247652' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:38:36.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:38:36 smithi102 ceph-mon[35442]: from='client.? 172.21.15.5:0/1228247652' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:38:36.499 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:38:36.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:38:36 smithi005 ceph-mon[30350]: from='client.? 172.21.15.5:0/1228247652' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:38:37.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:37.068+0000 7fe1ebe00640 1 -- 172.21.15.5:0/3797868420 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe1ec074e30 msgr2=0x7fe1ec10dd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:38:37.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:37.068+0000 7fe1ebe00640 1 --2- 172.21.15.5:0/3797868420 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe1ec074e30 0x7fe1ec10dd40 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fe1d8009900 tx=0x7fe1d802a480 comp rx=0 tx=0).stop 2024-09-10T09:38:37.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:37.068+0000 7fe1ebe00640 1 -- 172.21.15.5:0/3797868420 shutdown_connections 2024-09-10T09:38:37.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:37.068+0000 7fe1ebe00640 1 --2- 172.21.15.5:0/3797868420 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe1ec10e280 0x7fe1ec110670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:37.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:37.068+0000 7fe1ebe00640 1 --2- 172.21.15.5:0/3797868420 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe1ec074e30 0x7fe1ec10dd40 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:37.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:37.068+0000 7fe1ebe00640 1 --2- 172.21.15.5:0/3797868420 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe1ec074510 0x7fe1ec0748f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:37.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:37.068+0000 7fe1ebe00640 1 -- 172.21.15.5:0/3797868420 >> 172.21.15.5:0/3797868420 conn(0x7fe1ec0fd740 msgr2=0x7fe1ec0ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:38:37.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:37.068+0000 7fe1ebe00640 1 -- 172.21.15.5:0/3797868420 shutdown_connections 2024-09-10T09:38:37.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:37.068+0000 7fe1ebe00640 1 -- 172.21.15.5:0/3797868420 wait complete. 2024-09-10T09:38:37.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:37.069+0000 7fe1ebe00640 1 Processor -- start 2024-09-10T09:38:37.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:37.069+0000 7fe1ebe00640 1 -- start start 2024-09-10T09:38:37.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:37.069+0000 7fe1ebe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe1ec074510 0x7fe1ec19a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:37.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:37.069+0000 7fe1ebe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe1ec074e30 0x7fe1ec19aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:37.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:37.069+0000 7fe1ebe00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe1ec10e280 0x7fe1ec19edc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:37.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:37.069+0000 7fe1ebe00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe1ec113fc0 con 0x7fe1ec074e30 2024-09-10T09:38:37.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:37.069+0000 7fe1ebe00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fe1ec113e40 con 0x7fe1ec074510 2024-09-10T09:38:37.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:37.069+0000 7fe1ebe00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fe1ec114140 con 0x7fe1ec10e280 2024-09-10T09:38:37.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:37.070+0000 7fe1eaa00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe1ec074510 0x7fe1ec19a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:37.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:37.070+0000 7fe1ea000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe1ec074e30 0x7fe1ec19aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:37.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:37.070+0000 7fe1eaa00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe1ec074510 0x7fe1ec19a4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:38612/0 (socket says 172.21.15.5:38612) 2024-09-10T09:38:37.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:37.070+0000 7fe1eb400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe1ec10e280 0x7fe1ec19edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:37.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:37.070+0000 7fe1eaa00640 1 -- 172.21.15.5:0/126178889 learned_addr learned my addr 172.21.15.5:0/126178889 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:38:37.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:37.070+0000 7fe1eb400640 1 -- 172.21.15.5:0/126178889 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe1ec074510 msgr2=0x7fe1ec19a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:38:37.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:37.070+0000 7fe1eb400640 1 --2- 172.21.15.5:0/126178889 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe1ec074510 0x7fe1ec19a4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:37.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:37.070+0000 7fe1eb400640 1 -- 172.21.15.5:0/126178889 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe1ec074e30 msgr2=0x7fe1ec19aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:38:37.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:37.070+0000 7fe1eb400640 1 --2- 172.21.15.5:0/126178889 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe1ec074e30 0x7fe1ec19aa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:37.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:37.070+0000 7fe1eb400640 1 -- 172.21.15.5:0/126178889 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1ec19f4a0 con 0x7fe1ec10e280 2024-09-10T09:38:37.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:37.070+0000 7fe1ea000640 1 --2- 172.21.15.5:0/126178889 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe1ec074e30 0x7fe1ec19aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:38:37.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:37.071+0000 7fe1eb400640 1 --2- 172.21.15.5:0/126178889 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe1ec10e280 0x7fe1ec19edc0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fe1e4005fd0 tx=0x7fe1e400edd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:38:37.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:37.071+0000 7fe1f0a00640 1 -- 172.21.15.5:0/126178889 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1e4004d20 con 0x7fe1ec10e280 2024-09-10T09:38:37.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:37.071+0000 7fe1ebe00640 1 -- 172.21.15.5:0/126178889 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1ec19f790 con 0x7fe1ec10e280 2024-09-10T09:38:37.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:37.071+0000 7fe1f0a00640 1 -- 172.21.15.5:0/126178889 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe1e4005020 con 0x7fe1ec10e280 2024-09-10T09:38:37.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:37.071+0000 7fe1f0a00640 1 -- 172.21.15.5:0/126178889 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1e40116d0 con 0x7fe1ec10e280 2024-09-10T09:38:37.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:37.071+0000 7fe1ebe00640 1 -- 172.21.15.5:0/126178889 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1ec1a7070 con 0x7fe1ec10e280 2024-09-10T09:38:37.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:37.072+0000 7fe1ebe00640 1 -- 172.21.15.5:0/126178889 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe1ec0748f0 con 0x7fe1ec10e280 2024-09-10T09:38:37.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:37.074+0000 7fe1f0a00640 1 -- 172.21.15.5:0/126178889 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 19) ==== 137883+0+0 (secure 0 0 0) 0x7fe1e4005540 con 0x7fe1ec10e280 2024-09-10T09:38:37.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:37.076+0000 7fe1f0a00640 1 --2- 172.21.15.5:0/126178889 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fe1d00a2c10 0x7fe1d00a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:37.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:37.076+0000 7fe1f0a00640 1 -- 172.21.15.5:0/126178889 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fe1e40cf750 con 0x7fe1ec10e280 2024-09-10T09:38:37.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:37.076+0000 7fe1eaa00640 1 --2- 172.21.15.5:0/126178889 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fe1d00a2c10 0x7fe1d00a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:37.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:37.077+0000 7fe1eaa00640 1 --2- 172.21.15.5:0/126178889 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fe1d00a2c10 0x7fe1d00a50d0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fe1dc005fd0 tx=0x7fe1dc00b4a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:38:37.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:37.078+0000 7fe1f0a00640 1 -- 172.21.15.5:0/126178889 <== mon.1 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe1e4016030 con 0x7fe1ec10e280 2024-09-10T09:38:37.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:37.300+0000 7fe1ebe00640 1 -- 172.21.15.5:0/126178889 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe1ec02d7c0 con 0x7fe1ec10e280 2024-09-10T09:38:37.301 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:38:37 smithi005 ceph-mon[30350]: pgmap v401: 129 pgs: 129 active+clean; 655 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 170 B/s wr, 12 op/s 2024-09-10T09:38:37.301 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:38:37 smithi005 ceph-mon[30350]: scrub summary: idle 2024-09-10T09:38:37.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:37.301+0000 7fe1f0a00640 1 -- 172.21.15.5:0/126178889 <== mon.1 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11969 (secure 0 0 0) 0x7fe1e409bab0 con 0x7fe1ec10e280 2024-09-10T09:38:37.302 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-10T09:38:37.302 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":30,"btime":"2024-09-10T09:38:15:834198+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14454,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/2970594311","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2970594311},{"type":"v1","addr":"172.21.15.5:6835","nonce":2970594311}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14514,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6839/3616502174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":3616502174},{"type":"v1","addr":"172.21.15.5:6839","nonce":3616502174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T09:35:29.626287+0000","modified":"2024-09-10T09:38:15.658058+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24542,"mds_1":24535,"mds_2":24511,"mds_3":24506,"mds_4":24443},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14520":{"gid":14520,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/2073599842","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":2073599842},{"type":"v1","addr":"172.21.15.5:6841","nonce":2073599842}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"k","rank":4,"incarnation":26,"state":"up:active","state_seq":24,"addr":"172.21.15.102:6833/2457007427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":2457007427},{"type":"v1","addr":"172.21.15.102:6833","nonce":2457007427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/356445159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":356445159},{"type":"v1","addr":"172.21.15.159:6833","nonce":356445159}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6837/2894314507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":2894314507},{"type":"v1","addr":"172.21.15.5:6837","nonce":2894314507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.102:6835/1647549111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":1647549111},{"type":"v1","addr":"172.21.15.102:6835","nonce":1647549111}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6835/952657242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":952657242},{"type":"v1","addr":"172.21.15.159:6835","nonce":952657242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":19,"addr":"172.21.15.102:6837/2689776764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":2689776764},{"type":"v1","addr":"172.21.15.102:6837","nonce":2689776764}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.159:6837/620627639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":620627639},{"type":"v1","addr":"172.21.15.159:6837","nonce":620627639}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24535":{"gid":24535,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":18,"addr":"172.21.15.159:6839/2073739599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":2073739599},{"type":"v1","addr":"172.21.15.159:6839","nonce":2073739599}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24542":{"gid":24542,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.102:6839/335409749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6838","nonce":335409749},{"type":"v1","addr":"172.21.15.102:6839","nonce":335409749}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24542,"qdb_cluster":[24542,24535,24511,24506,24443]},"id":1}]} 2024-09-10T09:38:37.302 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 30 2024-09-10T09:38:37.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:37.303+0000 7fe1ebe00640 1 -- 172.21.15.5:0/126178889 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fe1d00a2c10 msgr2=0x7fe1d00a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:38:37.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:37.303+0000 7fe1ebe00640 1 --2- 172.21.15.5:0/126178889 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fe1d00a2c10 0x7fe1d00a50d0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fe1dc005fd0 tx=0x7fe1dc00b4a0 comp rx=0 tx=0).stop 2024-09-10T09:38:37.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:37.303+0000 7fe1ebe00640 1 -- 172.21.15.5:0/126178889 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe1ec10e280 msgr2=0x7fe1ec19edc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:38:37.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:37.303+0000 7fe1ebe00640 1 --2- 172.21.15.5:0/126178889 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe1ec10e280 0x7fe1ec19edc0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fe1e4005fd0 tx=0x7fe1e400edd0 comp rx=0 tx=0).stop 2024-09-10T09:38:37.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:37.303+0000 7fe1ebe00640 1 -- 172.21.15.5:0/126178889 shutdown_connections 2024-09-10T09:38:37.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:37.303+0000 7fe1ebe00640 1 --2- 172.21.15.5:0/126178889 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fe1d00a2c10 0x7fe1d00a50d0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:37.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:37.303+0000 7fe1ebe00640 1 --2- 172.21.15.5:0/126178889 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe1ec10e280 0x7fe1ec19edc0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:37.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:37.303+0000 7fe1ebe00640 1 --2- 172.21.15.5:0/126178889 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe1ec074e30 0x7fe1ec19aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:37.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:37.303+0000 7fe1ebe00640 1 --2- 172.21.15.5:0/126178889 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe1ec074510 0x7fe1ec19a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:37.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:37.303+0000 7fe1ebe00640 1 -- 172.21.15.5:0/126178889 >> 172.21.15.5:0/126178889 conn(0x7fe1ec0fd740 msgr2=0x7fe1ec0fe480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:38:37.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:37.303+0000 7fe1ebe00640 1 -- 172.21.15.5:0/126178889 shutdown_connections 2024-09-10T09:38:37.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:37.303+0000 7fe1ebe00640 1 -- 172.21.15.5:0/126178889 wait complete. 2024-09-10T09:38:37.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:38:37 smithi159 ceph-mon[35513]: pgmap v401: 129 pgs: 129 active+clean; 655 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 170 B/s wr, 12 op/s 2024-09-10T09:38:37.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:38:37 smithi159 ceph-mon[35513]: scrub summary: idle 2024-09-10T09:38:37.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:38:37 smithi102 ceph-mon[35442]: pgmap v401: 129 pgs: 129 active+clean; 655 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 170 B/s wr, 12 op/s 2024-09-10T09:38:37.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:38:37 smithi102 ceph-mon[35442]: scrub summary: idle 2024-09-10T09:38:38.170 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-10T09:38:38.306 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:38:38 smithi005 ceph-mon[30350]: from='client.? 172.21.15.5:0/126178889' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:38:38.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:38:38 smithi159 ceph-mon[35513]: from='client.? 172.21.15.5:0/126178889' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:38:38.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:38:38 smithi102 ceph-mon[35442]: from='client.? 172.21.15.5:0/126178889' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:38:38.562 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:38:39.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:39.202+0000 7fa9dbc00640 1 -- 172.21.15.5:0/4258437177 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa9d4074eb0 msgr2=0x7fa9d4112650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:38:39.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:39.202+0000 7fa9dbc00640 1 --2- 172.21.15.5:0/4258437177 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa9d4074eb0 0x7fa9d4112650 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fa9c0009900 tx=0x7fa9c002a480 comp rx=0 tx=0).stop 2024-09-10T09:38:39.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:39.203+0000 7fa9dbc00640 1 -- 172.21.15.5:0/4258437177 shutdown_connections 2024-09-10T09:38:39.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:39.203+0000 7fa9dbc00640 1 --2- 172.21.15.5:0/4258437177 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa9d4112b90 0x7fa9d4114f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:39.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:39.203+0000 7fa9dbc00640 1 --2- 172.21.15.5:0/4258437177 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa9d4074eb0 0x7fa9d4112650 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:39.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:39.203+0000 7fa9dbc00640 1 --2- 172.21.15.5:0/4258437177 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa9d4074590 0x7fa9d4074970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:39.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:39.203+0000 7fa9dbc00640 1 -- 172.21.15.5:0/4258437177 >> 172.21.15.5:0/4258437177 conn(0x7fa9d40fd8c0 msgr2=0x7fa9d40ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:38:39.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:39.203+0000 7fa9dbc00640 1 -- 172.21.15.5:0/4258437177 shutdown_connections 2024-09-10T09:38:39.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:39.203+0000 7fa9dbc00640 1 -- 172.21.15.5:0/4258437177 wait complete. 2024-09-10T09:38:39.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:39.204+0000 7fa9dbc00640 1 Processor -- start 2024-09-10T09:38:39.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:39.204+0000 7fa9dbc00640 1 -- start start 2024-09-10T09:38:39.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:39.204+0000 7fa9dbc00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa9d4074590 0x7fa9d41a4730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:39.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:39.204+0000 7fa9dbc00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa9d4074eb0 0x7fa9d41a4c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:39.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:39.204+0000 7fa9dbc00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa9d4112b90 0x7fa9d419e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:39.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:39.204+0000 7fa9dbc00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa9d4118540 con 0x7fa9d4112b90 2024-09-10T09:38:39.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:39.204+0000 7fa9dbc00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fa9d41183c0 con 0x7fa9d4074590 2024-09-10T09:38:39.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:39.204+0000 7fa9dbc00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fa9d41186c0 con 0x7fa9d4074eb0 2024-09-10T09:38:39.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:39.204+0000 7fa9d9c00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa9d4112b90 0x7fa9d419e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:39.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:39.205+0000 7fa9d3e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa9d4074eb0 0x7fa9d41a4c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:39.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:39.205+0000 7fa9d9200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa9d4074590 0x7fa9d41a4730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:39.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:39.205+0000 7fa9d3e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa9d4074eb0 0x7fa9d41a4c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:38518/0 (socket says 172.21.15.5:38518) 2024-09-10T09:38:39.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:39.205+0000 7fa9d9200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa9d4074590 0x7fa9d41a4730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:38638/0 (socket says 172.21.15.5:38638) 2024-09-10T09:38:39.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:39.205+0000 7fa9d3e00640 1 -- 172.21.15.5:0/1804558232 learned_addr learned my addr 172.21.15.5:0/1804558232 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:38:39.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:39.205+0000 7fa9d9c00640 1 --2- 172.21.15.5:0/1804558232 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa9d4112b90 0x7fa9d419e8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53028/0 (socket says 172.21.15.5:53028) 2024-09-10T09:38:39.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:39.205+0000 7fa9d9200640 1 -- 172.21.15.5:0/1804558232 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa9d4074eb0 msgr2=0x7fa9d41a4c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:38:39.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:39.205+0000 7fa9d9200640 1 --2- 172.21.15.5:0/1804558232 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa9d4074eb0 0x7fa9d41a4c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:39.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:39.205+0000 7fa9d9200640 1 -- 172.21.15.5:0/1804558232 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa9d4112b90 msgr2=0x7fa9d419e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:38:39.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:39.205+0000 7fa9d9200640 1 --2- 172.21.15.5:0/1804558232 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa9d4112b90 0x7fa9d419e8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:39.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:39.205+0000 7fa9d9200640 1 -- 172.21.15.5:0/1804558232 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9d419f020 con 0x7fa9d4074590 2024-09-10T09:38:39.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:39.205+0000 7fa9d9c00640 1 --2- 172.21.15.5:0/1804558232 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa9d4112b90 0x7fa9d419e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:38:39.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:39.205+0000 7fa9d9200640 1 --2- 172.21.15.5:0/1804558232 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa9d4074590 0x7fa9d41a4730 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fa9c800d6d0 tx=0x7fa9c800db90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:38:39.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:39.206+0000 7fa9d1600640 1 -- 172.21.15.5:0/1804558232 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9c8017070 con 0x7fa9d4074590 2024-09-10T09:38:39.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:39.206+0000 7fa9d1600640 1 -- 172.21.15.5:0/1804558232 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa9c8009820 con 0x7fa9d4074590 2024-09-10T09:38:39.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:39.206+0000 7fa9d1600640 1 -- 172.21.15.5:0/1804558232 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9c8004d00 con 0x7fa9d4074590 2024-09-10T09:38:39.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:39.206+0000 7fa9dbc00640 1 -- 172.21.15.5:0/1804558232 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9d419f310 con 0x7fa9d4074590 2024-09-10T09:38:39.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:39.206+0000 7fa9dbc00640 1 -- 172.21.15.5:0/1804558232 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9d41ab600 con 0x7fa9d4074590 2024-09-10T09:38:39.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:39.207+0000 7fa9dbc00640 1 -- 172.21.15.5:0/1804558232 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa9d4074970 con 0x7fa9d4074590 2024-09-10T09:38:39.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:39.209+0000 7fa9d1600640 1 -- 172.21.15.5:0/1804558232 <== mon.2 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 19) ==== 137883+0+0 (secure 0 0 0) 0x7fa9c80099d0 con 0x7fa9d4074590 2024-09-10T09:38:39.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:39.210+0000 7fa9d1600640 1 --2- 172.21.15.5:0/1804558232 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fa9bc0a2c90 0x7fa9bc0a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:39.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:39.210+0000 7fa9d1600640 1 -- 172.21.15.5:0/1804558232 <== mon.2 v2:172.21.15.102:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fa9c80cf0a0 con 0x7fa9d4074590 2024-09-10T09:38:39.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:39.210+0000 7fa9d3e00640 1 --2- 172.21.15.5:0/1804558232 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fa9bc0a2c90 0x7fa9bc0a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:39.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:39.211+0000 7fa9d3e00640 1 --2- 172.21.15.5:0/1804558232 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fa9bc0a2c90 0x7fa9bc0a5150 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fa9d4100110 tx=0x7fa9c0005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:38:39.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:39.214+0000 7fa9d1600640 1 -- 172.21.15.5:0/1804558232 <== mon.2 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa9c809b400 con 0x7fa9d4074590 2024-09-10T09:38:39.353 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:38:39 smithi159 ceph-mon[35513]: pgmap v402: 129 pgs: 129 active+clean; 655 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 170 B/s wr, 9 op/s 2024-09-10T09:38:39.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:38:39 smithi102 ceph-mon[35442]: pgmap v402: 129 pgs: 129 active+clean; 655 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 170 B/s wr, 9 op/s 2024-09-10T09:38:39.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:39.403+0000 7fa9dbc00640 1 -- 172.21.15.5:0/1804558232 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa9d402d7c0 con 0x7fa9d4074590 2024-09-10T09:38:39.404 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:38:39 smithi005 ceph-mon[30350]: pgmap v402: 129 pgs: 129 active+clean; 655 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 170 B/s wr, 9 op/s 2024-09-10T09:38:39.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:39.405+0000 7fa9d1600640 1 -- 172.21.15.5:0/1804558232 <== mon.2 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v96) ==== 74+0+24846 (secure 0 0 0) 0x7fa9c809f700 con 0x7fa9d4074590 2024-09-10T09:38:39.406 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-10T09:38:39.406 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":96,"fsid":"bade4b50-6f56-11ef-bcea-c7b262605968","created":"2024-09-10T09:27:13.298498+0000","modified":"2024-09-10T09:37:07.520520+0000","last_up_change":"2024-09-10T09:34:30.730066+0000","last_in_change":"2024-09-10T09:34:07.340995+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T09:30:26.042255+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T09:35:24.461031+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T09:35:27.155349+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-10T09:35:35.121462+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"085749a8-e882-4377-92dd-34bb266c0f0b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":954535266},{"type":"v1","addr":"172.21.15.5:6803","nonce":954535266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":954535266},{"type":"v1","addr":"172.21.15.5:6805","nonce":954535266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":954535266},{"type":"v1","addr":"172.21.15.5:6809","nonce":954535266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":954535266},{"type":"v1","addr":"172.21.15.5:6807","nonce":954535266}]},"public_addr":"172.21.15.5:6803/954535266","cluster_addr":"172.21.15.5:6805/954535266","heartbeat_back_addr":"172.21.15.5:6809/954535266","heartbeat_front_addr":"172.21.15.5:6807/954535266","state":["exists","up"]},{"osd":1,"uuid":"0a76c795-2573-4a92-969a-2ec5dd65c094","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6800","nonce":733924375},{"type":"v1","addr":"172.21.15.102:6801","nonce":733924375}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6802","nonce":733924375},{"type":"v1","addr":"172.21.15.102:6803","nonce":733924375}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6806","nonce":733924375},{"type":"v1","addr":"172.21.15.102:6807","nonce":733924375}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6804","nonce":733924375},{"type":"v1","addr":"172.21.15.102:6805","nonce":733924375}]},"public_addr":"172.21.15.102:6801/733924375","cluster_addr":"172.21.15.102:6803/733924375","heartbeat_back_addr":"172.21.15.102:6807/733924375","heartbeat_front_addr":"172.21.15.102:6805/733924375","state":["exists","up"]},{"osd":2,"uuid":"0c98fcb8-60ab-4473-8995-1d9e97c577e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6800","nonce":2236204189},{"type":"v1","addr":"172.21.15.159:6801","nonce":2236204189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":2236204189},{"type":"v1","addr":"172.21.15.159:6803","nonce":2236204189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":2236204189},{"type":"v1","addr":"172.21.15.159:6807","nonce":2236204189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":2236204189},{"type":"v1","addr":"172.21.15.159:6805","nonce":2236204189}]},"public_addr":"172.21.15.159:6801/2236204189","cluster_addr":"172.21.15.159:6803/2236204189","heartbeat_back_addr":"172.21.15.159:6807/2236204189","heartbeat_front_addr":"172.21.15.159:6805/2236204189","state":["exists","up"]},{"osd":3,"uuid":"88eecdeb-5cee-4f78-a3f9-a66bf4bf772a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":2221468725},{"type":"v1","addr":"172.21.15.5:6811","nonce":2221468725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":2221468725},{"type":"v1","addr":"172.21.15.5:6813","nonce":2221468725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6816","nonce":2221468725},{"type":"v1","addr":"172.21.15.5:6817","nonce":2221468725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":2221468725},{"type":"v1","addr":"172.21.15.5:6815","nonce":2221468725}]},"public_addr":"172.21.15.5:6811/2221468725","cluster_addr":"172.21.15.5:6813/2221468725","heartbeat_back_addr":"172.21.15.5:6817/2221468725","heartbeat_front_addr":"172.21.15.5:6815/2221468725","state":["exists","up"]},{"osd":4,"uuid":"f2e7314d-7938-4668-9011-45c175f66b48","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6808","nonce":594465753},{"type":"v1","addr":"172.21.15.102:6809","nonce":594465753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6810","nonce":594465753},{"type":"v1","addr":"172.21.15.102:6811","nonce":594465753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6814","nonce":594465753},{"type":"v1","addr":"172.21.15.102:6815","nonce":594465753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6812","nonce":594465753},{"type":"v1","addr":"172.21.15.102:6813","nonce":594465753}]},"public_addr":"172.21.15.102:6809/594465753","cluster_addr":"172.21.15.102:6811/594465753","heartbeat_back_addr":"172.21.15.102:6815/594465753","heartbeat_front_addr":"172.21.15.102:6813/594465753","state":["exists","up"]},{"osd":5,"uuid":"0c0366d3-658b-4bf0-ba17-aff28a02e948","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":3230280402},{"type":"v1","addr":"172.21.15.159:6809","nonce":3230280402}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":3230280402},{"type":"v1","addr":"172.21.15.159:6811","nonce":3230280402}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":3230280402},{"type":"v1","addr":"172.21.15.159:6815","nonce":3230280402}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":3230280402},{"type":"v1","addr":"172.21.15.159:6813","nonce":3230280402}]},"public_addr":"172.21.15.159:6809/3230280402","cluster_addr":"172.21.15.159:6811/3230280402","heartbeat_back_addr":"172.21.15.159:6815/3230280402","heartbeat_front_addr":"172.21.15.159:6813/3230280402","state":["exists","up"]},{"osd":6,"uuid":"682670a6-51d5-46e8-af6e-dc2d8c1cf407","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":1907651140},{"type":"v1","addr":"172.21.15.5:6819","nonce":1907651140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":1907651140},{"type":"v1","addr":"172.21.15.5:6821","nonce":1907651140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":1907651140},{"type":"v1","addr":"172.21.15.5:6825","nonce":1907651140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":1907651140},{"type":"v1","addr":"172.21.15.5:6823","nonce":1907651140}]},"public_addr":"172.21.15.5:6819/1907651140","cluster_addr":"172.21.15.5:6821/1907651140","heartbeat_back_addr":"172.21.15.5:6825/1907651140","heartbeat_front_addr":"172.21.15.5:6823/1907651140","state":["exists","up"]},{"osd":7,"uuid":"8ae0f4a9-99e7-4563-aa2a-15169684a8b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6816","nonce":692040360},{"type":"v1","addr":"172.21.15.102:6817","nonce":692040360}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6818","nonce":692040360},{"type":"v1","addr":"172.21.15.102:6819","nonce":692040360}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6822","nonce":692040360},{"type":"v1","addr":"172.21.15.102:6823","nonce":692040360}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6820","nonce":692040360},{"type":"v1","addr":"172.21.15.102:6821","nonce":692040360}]},"public_addr":"172.21.15.102:6817/692040360","cluster_addr":"172.21.15.102:6819/692040360","heartbeat_back_addr":"172.21.15.102:6823/692040360","heartbeat_front_addr":"172.21.15.102:6821/692040360","state":["exists","up"]},{"osd":8,"uuid":"e37c845e-ced9-4413-9bbe-bc090537c2bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":2107291799},{"type":"v1","addr":"172.21.15.159:6817","nonce":2107291799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":2107291799},{"type":"v1","addr":"172.21.15.159:6819","nonce":2107291799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":2107291799},{"type":"v1","addr":"172.21.15.159:6823","nonce":2107291799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":2107291799},{"type":"v1","addr":"172.21.15.159:6821","nonce":2107291799}]},"public_addr":"172.21.15.159:6817/2107291799","cluster_addr":"172.21.15.159:6819/2107291799","heartbeat_back_addr":"172.21.15.159:6823/2107291799","heartbeat_front_addr":"172.21.15.159:6821/2107291799","state":["exists","up"]},{"osd":9,"uuid":"91a5b6cc-08da-47be-af12-d0896a849309","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":902431281},{"type":"v1","addr":"172.21.15.5:6827","nonce":902431281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":902431281},{"type":"v1","addr":"172.21.15.5:6829","nonce":902431281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":902431281},{"type":"v1","addr":"172.21.15.5:6833","nonce":902431281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":902431281},{"type":"v1","addr":"172.21.15.5:6831","nonce":902431281}]},"public_addr":"172.21.15.5:6827/902431281","cluster_addr":"172.21.15.5:6829/902431281","heartbeat_back_addr":"172.21.15.5:6833/902431281","heartbeat_front_addr":"172.21.15.5:6831/902431281","state":["exists","up"]},{"osd":10,"uuid":"2211030c-b00e-402b-b4d7-c587eb29a490","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6824","nonce":2853806211},{"type":"v1","addr":"172.21.15.102:6825","nonce":2853806211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6826","nonce":2853806211},{"type":"v1","addr":"172.21.15.102:6827","nonce":2853806211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6830","nonce":2853806211},{"type":"v1","addr":"172.21.15.102:6831","nonce":2853806211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6828","nonce":2853806211},{"type":"v1","addr":"172.21.15.102:6829","nonce":2853806211}]},"public_addr":"172.21.15.102:6825/2853806211","cluster_addr":"172.21.15.102:6827/2853806211","heartbeat_back_addr":"172.21.15.102:6831/2853806211","heartbeat_front_addr":"172.21.15.102:6829/2853806211","state":["exists","up"]},{"osd":11,"uuid":"4e9699ec-8ff2-4005-9d84-af5600b3a2ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6824","nonce":969941218},{"type":"v1","addr":"172.21.15.159:6825","nonce":969941218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":969941218},{"type":"v1","addr":"172.21.15.159:6827","nonce":969941218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6830","nonce":969941218},{"type":"v1","addr":"172.21.15.159:6831","nonce":969941218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":969941218},{"type":"v1","addr":"172.21.15.159:6829","nonce":969941218}]},"public_addr":"172.21.15.159:6825/969941218","cluster_addr":"172.21.15.159:6827/969941218","heartbeat_back_addr":"172.21.15.159:6831/969941218","heartbeat_front_addr":"172.21.15.159:6829/969941218","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:29:34.416490+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:30:00.246101+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:30:24.300464+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:30:54.663623+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:31:21.426292+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:31:46.738914+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:32:16.196571+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:32:42.549243+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:33:08.178651+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:33:37.477911+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:34:03.114026+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:34:28.891924+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4}]},{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"3.0","mappings":[{"from":4,"to":8}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":6}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9},{"from":5,"to":7}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]},{"pgid":"3.39","mappings":[{"from":5,"to":6}]},{"pgid":"4.8","mappings":[{"from":5,"to":8}]},{"pgid":"4.c","mappings":[{"from":9,"to":10}]},{"pgid":"4.18","mappings":[{"from":9,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.5:6801/1527422686":"2024-09-11T09:28:05.436742+0000","172.21.15.5:0/2276144345":"2024-09-11T09:28:05.436742+0000","172.21.15.5:0/412040104":"2024-09-11T09:28:05.436742+0000","172.21.15.5:6800/1527422686":"2024-09-11T09:28:05.436742+0000","172.21.15.5:0/1856581170":"2024-09-11T09:28:05.436742+0000","172.21.15.5:0/2847668298":"2024-09-11T09:27:37.778500+0000","172.21.15.5:6801/14425913":"2024-09-11T09:27:37.778500+0000","172.21.15.5:0/166910782":"2024-09-11T09:27:37.778500+0000","172.21.15.5:6800/14425913":"2024-09-11T09:27:37.778500+0000","172.21.15.5:0/1711858179":"2024-09-11T09:27:37.778500+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T09:38:39.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:39.407+0000 7fa9dbc00640 1 -- 172.21.15.5:0/1804558232 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fa9bc0a2c90 msgr2=0x7fa9bc0a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:38:39.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:39.407+0000 7fa9dbc00640 1 --2- 172.21.15.5:0/1804558232 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fa9bc0a2c90 0x7fa9bc0a5150 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fa9d4100110 tx=0x7fa9c0005a20 comp rx=0 tx=0).stop 2024-09-10T09:38:39.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:39.407+0000 7fa9dbc00640 1 -- 172.21.15.5:0/1804558232 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa9d4074590 msgr2=0x7fa9d41a4730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:38:39.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:39.407+0000 7fa9dbc00640 1 --2- 172.21.15.5:0/1804558232 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa9d4074590 0x7fa9d41a4730 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fa9c800d6d0 tx=0x7fa9c800db90 comp rx=0 tx=0).stop 2024-09-10T09:38:39.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:39.407+0000 7fa9dbc00640 1 -- 172.21.15.5:0/1804558232 shutdown_connections 2024-09-10T09:38:39.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:39.407+0000 7fa9dbc00640 1 --2- 172.21.15.5:0/1804558232 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fa9bc0a2c90 0x7fa9bc0a5150 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:39.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:39.407+0000 7fa9dbc00640 1 --2- 172.21.15.5:0/1804558232 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa9d4112b90 0x7fa9d419e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:39.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:39.407+0000 7fa9dbc00640 1 --2- 172.21.15.5:0/1804558232 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa9d4074eb0 0x7fa9d41a4c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:39.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:39.407+0000 7fa9dbc00640 1 --2- 172.21.15.5:0/1804558232 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa9d4074590 0x7fa9d41a4730 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:39.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:39.407+0000 7fa9dbc00640 1 -- 172.21.15.5:0/1804558232 >> 172.21.15.5:0/1804558232 conn(0x7fa9d40fd8c0 msgr2=0x7fa9d40ff230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:38:39.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:39.407+0000 7fa9dbc00640 1 -- 172.21.15.5:0/1804558232 shutdown_connections 2024-09-10T09:38:39.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:39.408+0000 7fa9dbc00640 1 -- 172.21.15.5:0/1804558232 wait complete. 2024-09-10T09:38:40.278 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph daemon mon.a --format=json config get run_dir 2024-09-10T09:38:40.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:38:40 smithi159 ceph-mon[35513]: from='client.? 172.21.15.5:0/1804558232' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T09:38:40.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:38:40 smithi102 ceph-mon[35442]: from='client.? 172.21.15.5:0/1804558232' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T09:38:40.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:38:40 smithi005 ceph-mon[30350]: from='client.? 172.21.15.5:0/1804558232' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T09:38:40.662 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:38:41.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:38:41 smithi159 ceph-mon[35513]: pgmap v403: 129 pgs: 129 active+clean; 655 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 170 B/s wr, 9 op/s 2024-09-10T09:38:41.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:38:41 smithi102 ceph-mon[35442]: pgmap v403: 129 pgs: 129 active+clean; 655 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 170 B/s wr, 9 op/s 2024-09-10T09:38:41.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:38:41 smithi005 ceph-mon[30350]: pgmap v403: 129 pgs: 129 active+clean; 655 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 170 B/s wr, 9 op/s 2024-09-10T09:38:42.043 INFO:teuthology.orchestra.run.smithi005.stdout:{"run_dir":"/var/run/ceph"} 2024-09-10T09:38:42.044 DEBUG:tasks.cephfs.filesystem:_json_asok output { "run_dir": "/var/run/ceph" } 2024-09-10T09:38:42.044 INFO:tasks.quiescer.fs.[cephfs]:Initializing Quiescer with seed 9893 2024-09-10T09:38:42.044 DEBUG:tasks.quiescer:Yielding 2024-09-10T09:38:42.044 INFO:teuthology.run_tasks:Running task workunit... 2024-09-10T09:38:42.059 INFO:tasks.workunit:Pulling workunits from ref 1cf819b7212a6048e92fc78c4eff963344f6d56b 2024-09-10T09:38:42.059 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-09-10T09:38:42.059 INFO:tasks.workunit:timeout=3h 2024-09-10T09:38:42.059 INFO:tasks.workunit:cleanup=True 2024-09-10T09:38:42.059 DEBUG:teuthology.orchestra.run.smithi005:> stat -- /home/ubuntu/cephtest/mnt.0 2024-09-10T09:38:42.060 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T09:38:42.097 INFO:teuthology.orchestra.run.smithi005.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-09-10T09:38:42.097 INFO:teuthology.orchestra.run.smithi005.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2024-09-10T09:38:42.097 INFO:teuthology.orchestra.run.smithi005.stdout:Device: 78h/120d Inode: 1099511628283 Links: 2 2024-09-10T09:38:42.098 INFO:teuthology.orchestra.run.smithi005.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T09:38:42.098 INFO:teuthology.orchestra.run.smithi005.stdout:Context: system_u:object_r:unlabeled_t:s0 2024-09-10T09:38:42.098 INFO:teuthology.orchestra.run.smithi005.stdout:Access: 2024-09-10 09:36:24.906508416 +0000 2024-09-10T09:38:42.098 INFO:teuthology.orchestra.run.smithi005.stdout:Modify: 2024-09-10 09:36:24.906508416 +0000 2024-09-10T09:38:42.098 INFO:teuthology.orchestra.run.smithi005.stdout:Change: 2024-09-10 09:38:14.782941573 +0000 2024-09-10T09:38:42.098 INFO:teuthology.orchestra.run.smithi005.stdout: Birth: 2024-09-10 09:36:24.906508416 +0000 2024-09-10T09:38:42.098 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-09-10T09:38:42.098 DEBUG:teuthology.orchestra.run.smithi005:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-09-10T09:38:42.207 DEBUG:teuthology.orchestra.run.smithi005:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 1cf819b7212a6048e92fc78c4eff963344f6d56b 2024-09-10T09:38:42.246 INFO:tasks.workunit.client.0.smithi005.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-09-10T09:38:42.472 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:38:43.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:43.040+0000 7fbcf6600640 1 -- 172.21.15.5:0/2925716008 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbcf0104010 msgr2=0x7fbcf01043f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:38:43.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:43.040+0000 7fbcf6600640 1 --2- 172.21.15.5:0/2925716008 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbcf0104010 0x7fbcf01043f0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fbce0009900 tx=0x7fbce002a410 comp rx=0 tx=0).stop 2024-09-10T09:38:43.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:43.040+0000 7fbcf6600640 1 -- 172.21.15.5:0/2925716008 shutdown_connections 2024-09-10T09:38:43.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:43.040+0000 7fbcf6600640 1 --2- 172.21.15.5:0/2925716008 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbcf0109140 0x7fbcf0110cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:43.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:43.040+0000 7fbcf6600640 1 --2- 172.21.15.5:0/2925716008 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbcf01049c0 0x7fbcf0108a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:43.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:43.040+0000 7fbcf6600640 1 --2- 172.21.15.5:0/2925716008 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbcf0104010 0x7fbcf01043f0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:43.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:43.040+0000 7fbcf6600640 1 -- 172.21.15.5:0/2925716008 >> 172.21.15.5:0/2925716008 conn(0x7fbcf00ffa50 msgr2=0x7fbcf0101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:38:43.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:43.040+0000 7fbcf6600640 1 -- 172.21.15.5:0/2925716008 shutdown_connections 2024-09-10T09:38:43.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:43.040+0000 7fbcf6600640 1 -- 172.21.15.5:0/2925716008 wait complete. 2024-09-10T09:38:43.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:43.041+0000 7fbcf6600640 1 Processor -- start 2024-09-10T09:38:43.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:43.041+0000 7fbcf6600640 1 -- start start 2024-09-10T09:38:43.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:43.041+0000 7fbcf6600640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbcf0104010 0x7fbcf019a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:43.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:43.041+0000 7fbcf6600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbcf01049c0 0x7fbcf019ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:43.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:43.041+0000 7fbcf6600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbcf0109140 0x7fbcf019eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:43.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:43.041+0000 7fbcf6600640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fbcf0114190 con 0x7fbcf01049c0 2024-09-10T09:38:43.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:43.041+0000 7fbcf6600640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fbcf0114010 con 0x7fbcf0109140 2024-09-10T09:38:43.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:43.041+0000 7fbcf6600640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fbcf0114310 con 0x7fbcf0104010 2024-09-10T09:38:43.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:43.042+0000 7fbceea00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbcf01049c0 0x7fbcf019ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:43.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:43.042+0000 7fbcefe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbcf0109140 0x7fbcf019eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:43.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:43.042+0000 7fbceea00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbcf01049c0 0x7fbcf019ab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58150/0 (socket says 172.21.15.5:58150) 2024-09-10T09:38:43.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:43.042+0000 7fbceea00640 1 -- 172.21.15.5:0/1308226531 learned_addr learned my addr 172.21.15.5:0/1308226531 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:38:43.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:43.042+0000 7fbcef400640 1 --2- 172.21.15.5:0/1308226531 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbcf0104010 0x7fbcf019a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:43.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:43.042+0000 7fbceea00640 1 -- 172.21.15.5:0/1308226531 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbcf0104010 msgr2=0x7fbcf019a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:38:43.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:43.042+0000 7fbceea00640 1 --2- 172.21.15.5:0/1308226531 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbcf0104010 0x7fbcf019a600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:43.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:43.042+0000 7fbceea00640 1 -- 172.21.15.5:0/1308226531 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbcf0109140 msgr2=0x7fbcf019eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:38:43.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:43.042+0000 7fbceea00640 1 --2- 172.21.15.5:0/1308226531 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbcf0109140 0x7fbcf019eed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:43.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:43.042+0000 7fbceea00640 1 -- 172.21.15.5:0/1308226531 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbcf019f5b0 con 0x7fbcf01049c0 2024-09-10T09:38:43.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:43.042+0000 7fbceea00640 1 --2- 172.21.15.5:0/1308226531 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbcf01049c0 0x7fbcf019ab40 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fbcdc00e9f0 tx=0x7fbcdc00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:38:43.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:43.043+0000 7fbce7e00640 1 -- 172.21.15.5:0/1308226531 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcdc00cd80 con 0x7fbcf01049c0 2024-09-10T09:38:43.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:43.043+0000 7fbce7e00640 1 -- 172.21.15.5:0/1308226531 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbcdc010910 con 0x7fbcf01049c0 2024-09-10T09:38:43.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:43.043+0000 7fbce7e00640 1 -- 172.21.15.5:0/1308226531 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcdc018990 con 0x7fbcf01049c0 2024-09-10T09:38:43.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:43.043+0000 7fbcf6600640 1 -- 172.21.15.5:0/1308226531 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbcf019f8a0 con 0x7fbcf01049c0 2024-09-10T09:38:43.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:43.043+0000 7fbcf6600640 1 -- 172.21.15.5:0/1308226531 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbcf01a7180 con 0x7fbcf01049c0 2024-09-10T09:38:43.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:43.044+0000 7fbcf6600640 1 -- 172.21.15.5:0/1308226531 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbcf01043f0 con 0x7fbcf01049c0 2024-09-10T09:38:43.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:43.046+0000 7fbce7e00640 1 -- 172.21.15.5:0/1308226531 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 19) ==== 137883+0+0 (secure 0 0 0) 0x7fbcdc010ac0 con 0x7fbcf01049c0 2024-09-10T09:38:43.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:43.047+0000 7fbce7e00640 1 --2- 172.21.15.5:0/1308226531 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fbcd40a2c10 0x7fbcd40a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:43.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:43.047+0000 7fbce7e00640 1 -- 172.21.15.5:0/1308226531 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fbcdc0cf930 con 0x7fbcf01049c0 2024-09-10T09:38:43.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:43.047+0000 7fbcef400640 1 --2- 172.21.15.5:0/1308226531 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fbcd40a2c10 0x7fbcd40a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:43.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:43.048+0000 7fbcef400640 1 --2- 172.21.15.5:0/1308226531 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fbcd40a2c10 0x7fbcd40a50d0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fbce0000c00 tx=0x7fbce0005b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:38:43.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:43.051+0000 7fbce7e00640 1 -- 172.21.15.5:0/1308226531 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fbcdc09bc90 con 0x7fbcf01049c0 2024-09-10T09:38:43.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:43.271+0000 7fbcf6600640 1 -- 172.21.15.5:0/1308226531 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbcf010bd90 con 0x7fbcf01049c0 2024-09-10T09:38:43.273 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:38:43 smithi005 ceph-mon[30350]: pgmap v404: 129 pgs: 129 active+clean; 655 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 170 B/s wr, 11 op/s 2024-09-10T09:38:43.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:43.272+0000 7fbce7e00640 1 -- 172.21.15.5:0/1308226531 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11969 (secure 0 0 0) 0x7fbcdc09ff90 con 0x7fbcf01049c0 2024-09-10T09:38:43.274 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-10T09:38:43.274 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":30,"btime":"2024-09-10T09:38:15:834198+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14454,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/2970594311","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2970594311},{"type":"v1","addr":"172.21.15.5:6835","nonce":2970594311}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14514,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6839/3616502174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":3616502174},{"type":"v1","addr":"172.21.15.5:6839","nonce":3616502174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T09:35:29.626287+0000","modified":"2024-09-10T09:38:15.658058+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24542,"mds_1":24535,"mds_2":24511,"mds_3":24506,"mds_4":24443},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14520":{"gid":14520,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/2073599842","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":2073599842},{"type":"v1","addr":"172.21.15.5:6841","nonce":2073599842}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"k","rank":4,"incarnation":26,"state":"up:active","state_seq":24,"addr":"172.21.15.102:6833/2457007427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":2457007427},{"type":"v1","addr":"172.21.15.102:6833","nonce":2457007427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/356445159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":356445159},{"type":"v1","addr":"172.21.15.159:6833","nonce":356445159}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6837/2894314507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":2894314507},{"type":"v1","addr":"172.21.15.5:6837","nonce":2894314507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.102:6835/1647549111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":1647549111},{"type":"v1","addr":"172.21.15.102:6835","nonce":1647549111}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6835/952657242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":952657242},{"type":"v1","addr":"172.21.15.159:6835","nonce":952657242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":19,"addr":"172.21.15.102:6837/2689776764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":2689776764},{"type":"v1","addr":"172.21.15.102:6837","nonce":2689776764}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.159:6837/620627639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":620627639},{"type":"v1","addr":"172.21.15.159:6837","nonce":620627639}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24535":{"gid":24535,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":18,"addr":"172.21.15.159:6839/2073739599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":2073739599},{"type":"v1","addr":"172.21.15.159:6839","nonce":2073739599}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24542":{"gid":24542,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.102:6839/335409749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6838","nonce":335409749},{"type":"v1","addr":"172.21.15.102:6839","nonce":335409749}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24542,"qdb_cluster":[24542,24535,24511,24506,24443]},"id":1}]} 2024-09-10T09:38:43.274 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 30 2024-09-10T09:38:43.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:43.275+0000 7fbcf6600640 1 -- 172.21.15.5:0/1308226531 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fbcd40a2c10 msgr2=0x7fbcd40a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:38:43.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:43.275+0000 7fbcf6600640 1 --2- 172.21.15.5:0/1308226531 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fbcd40a2c10 0x7fbcd40a50d0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fbce0000c00 tx=0x7fbce0005b50 comp rx=0 tx=0).stop 2024-09-10T09:38:43.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:43.275+0000 7fbcf6600640 1 -- 172.21.15.5:0/1308226531 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbcf01049c0 msgr2=0x7fbcf019ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:38:43.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:43.275+0000 7fbcf6600640 1 --2- 172.21.15.5:0/1308226531 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbcf01049c0 0x7fbcf019ab40 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fbcdc00e9f0 tx=0x7fbcdc00eeb0 comp rx=0 tx=0).stop 2024-09-10T09:38:43.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:43.275+0000 7fbcf6600640 1 -- 172.21.15.5:0/1308226531 shutdown_connections 2024-09-10T09:38:43.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:43.275+0000 7fbcf6600640 1 --2- 172.21.15.5:0/1308226531 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fbcd40a2c10 0x7fbcd40a50d0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:43.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:43.275+0000 7fbcf6600640 1 --2- 172.21.15.5:0/1308226531 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbcf0109140 0x7fbcf019eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:43.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:43.275+0000 7fbcf6600640 1 --2- 172.21.15.5:0/1308226531 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbcf01049c0 0x7fbcf019ab40 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:43.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:43.275+0000 7fbcf6600640 1 --2- 172.21.15.5:0/1308226531 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbcf0104010 0x7fbcf019a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:43.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:43.275+0000 7fbcf6600640 1 -- 172.21.15.5:0/1308226531 >> 172.21.15.5:0/1308226531 conn(0x7fbcf00ffa50 msgr2=0x7fbcf0100d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:38:43.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:43.275+0000 7fbcf6600640 1 -- 172.21.15.5:0/1308226531 shutdown_connections 2024-09-10T09:38:43.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:43.275+0000 7fbcf6600640 1 -- 172.21.15.5:0/1308226531 wait complete. 2024-09-10T09:38:43.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:38:43 smithi159 ceph-mon[35513]: pgmap v404: 129 pgs: 129 active+clean; 655 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 170 B/s wr, 11 op/s 2024-09-10T09:38:43.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:38:43 smithi102 ceph-mon[35442]: pgmap v404: 129 pgs: 129 active+clean; 655 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 170 B/s wr, 11 op/s 2024-09-10T09:38:44.141 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 30, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-09-10T09:35:29.626287+0000', 'modified': '2024-09-10T09:38:15.658058+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24542, 'mds_1': 24535, 'mds_2': 24511, 'mds_3': 24506, 'mds_4': 24443}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14520': {'gid': 14520, 'name': 'a', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.5:6841/2073599842', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.5:6840', 'nonce': 2073599842}, {'type': 'v1', 'addr': '172.21.15.5:6841', 'nonce': 2073599842}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24443': {'gid': 24443, 'name': 'k', 'rank': 4, 'incarnation': 26, 'state': 'up:active', 'state_seq': 24, 'addr': '172.21.15.102:6833/2457007427', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.102:6832', 'nonce': 2457007427}, {'type': 'v1', 'addr': '172.21.15.102:6833', 'nonce': 2457007427}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24460': {'gid': 24460, 'name': 'l', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.159:6833/356445159', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.159:6832', 'nonce': 356445159}, {'type': 'v1', 'addr': '172.21.15.159:6833', 'nonce': 356445159}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24467': {'gid': 24467, 'name': 'd', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.5:6837/2894314507', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.5:6836', 'nonce': 2894314507}, {'type': 'v1', 'addr': '172.21.15.5:6837', 'nonce': 2894314507}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24479': {'gid': 24479, 'name': 'e', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.102:6835/1647549111', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.102:6834', 'nonce': 1647549111}, {'type': 'v1', 'addr': '172.21.15.102:6835', 'nonce': 1647549111}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24505': {'gid': 24505, 'name': 'f', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.159:6835/952657242', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.159:6834', 'nonce': 952657242}, {'type': 'v1', 'addr': '172.21.15.159:6835', 'nonce': 952657242}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24506': {'gid': 24506, 'name': 'h', 'rank': 3, 'incarnation': 24, 'state': 'up:active', 'state_seq': 19, 'addr': '172.21.15.102:6837/2689776764', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.102:6836', 'nonce': 2689776764}, {'type': 'v1', 'addr': '172.21.15.102:6837', 'nonce': 2689776764}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24511': {'gid': 24511, 'name': 'c', 'rank': 2, 'incarnation': 22, 'state': 'up:active', 'state_seq': 19, 'addr': '172.21.15.159:6837/620627639', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.159:6836', 'nonce': 620627639}, {'type': 'v1', 'addr': '172.21.15.159:6837', 'nonce': 620627639}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24535': {'gid': 24535, 'name': 'i', 'rank': 1, 'incarnation': 20, 'state': 'up:active', 'state_seq': 18, 'addr': '172.21.15.159:6839/2073739599', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.159:6838', 'nonce': 2073739599}, {'type': 'v1', 'addr': '172.21.15.159:6839', 'nonce': 2073739599}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24542': {'gid': 24542, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.102:6839/335409749', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.102:6838', 'nonce': 335409749}, {'type': 'v1', 'addr': '172.21.15.102:6839', 'nonce': 335409749}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24542, 'qdb_cluster': [24542, 24535, 24511, 24506, 24443]} 2024-09-10T09:38:44.142 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-09-10T09:38:44.142 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph tell mds.k status 2024-09-10T09:38:44.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:38:44 smithi102 ceph-mon[35442]: from='client.? 172.21.15.5:0/1308226531' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:38:44.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:38:44 smithi005 ceph-mon[30350]: from='client.? 172.21.15.5:0/1308226531' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:38:44.559 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:38:44.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:38:44 smithi159 ceph-mon[35513]: from='client.? 172.21.15.5:0/1308226531' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:38:45.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:45.128+0000 7f7349400640 1 -- 172.21.15.5:0/1749417036 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f734410e450 msgr2=0x7f7344110840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:38:45.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:45.128+0000 7f7349400640 1 --2- 172.21.15.5:0/1749417036 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f734410e450 0x7f7344110840 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f733c009ca0 tx=0x7f733c02a950 comp rx=0 tx=0).stop 2024-09-10T09:38:45.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:45.128+0000 7f7349400640 1 -- 172.21.15.5:0/1749417036 shutdown_connections 2024-09-10T09:38:45.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:45.128+0000 7f7349400640 1 --2- 172.21.15.5:0/1749417036 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f734410e450 0x7f7344110840 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:45.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:45.128+0000 7f7349400640 1 --2- 172.21.15.5:0/1749417036 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7344074e30 0x7f734410df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:45.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:45.128+0000 7f7349400640 1 --2- 172.21.15.5:0/1749417036 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7344074510 0x7f73440748f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:45.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:45.128+0000 7f7349400640 1 -- 172.21.15.5:0/1749417036 >> 172.21.15.5:0/1749417036 conn(0x7f73440fd8b0 msgr2=0x7f73440ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:38:45.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:45.128+0000 7f7349400640 1 -- 172.21.15.5:0/1749417036 shutdown_connections 2024-09-10T09:38:45.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:45.128+0000 7f7349400640 1 -- 172.21.15.5:0/1749417036 wait complete. 2024-09-10T09:38:45.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:45.129+0000 7f7349400640 1 Processor -- start 2024-09-10T09:38:45.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:45.129+0000 7f7349400640 1 -- start start 2024-09-10T09:38:45.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:45.129+0000 7f7349400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7344074510 0x7f734419a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:45.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:45.130+0000 7f7349400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7344074e30 0x7f734419ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:45.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:45.130+0000 7f7349400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f734410e450 0x7f734419eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:45.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:45.130+0000 7f7349400640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f7344114180 con 0x7f7344074510 2024-09-10T09:38:45.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:45.130+0000 7f7349400640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f7344114000 con 0x7f7344074e30 2024-09-10T09:38:45.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:45.130+0000 7f7349400640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f7344114300 con 0x7f734410e450 2024-09-10T09:38:45.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:45.130+0000 7f7342800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7344074510 0x7f734419a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:45.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:45.130+0000 7f7342800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7344074510 0x7f734419a620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58158/0 (socket says 172.21.15.5:58158) 2024-09-10T09:38:45.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:45.130+0000 7f7342800640 1 -- 172.21.15.5:0/773087313 learned_addr learned my addr 172.21.15.5:0/773087313 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:38:45.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:45.130+0000 7f7343200640 1 --2- 172.21.15.5:0/773087313 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f734410e450 0x7f734419eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:45.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:45.130+0000 7f733be00640 1 --2- 172.21.15.5:0/773087313 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7344074e30 0x7f734419ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:45.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:45.130+0000 7f7343200640 1 -- 172.21.15.5:0/773087313 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7344074e30 msgr2=0x7f734419ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:38:45.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:45.130+0000 7f7343200640 1 --2- 172.21.15.5:0/773087313 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7344074e30 0x7f734419ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:45.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:45.130+0000 7f7343200640 1 -- 172.21.15.5:0/773087313 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7344074510 msgr2=0x7f734419a620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:38:45.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:45.130+0000 7f7343200640 1 --2- 172.21.15.5:0/773087313 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7344074510 0x7f734419a620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:45.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:45.130+0000 7f7343200640 1 -- 172.21.15.5:0/773087313 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f734419f5d0 con 0x7f734410e450 2024-09-10T09:38:45.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:45.131+0000 7f7343200640 1 --2- 172.21.15.5:0/773087313 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f734410e450 0x7f734419eef0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f733c02ae70 tx=0x7f733c02f3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:38:45.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:45.131+0000 7f733b400640 1 -- 172.21.15.5:0/773087313 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f733c034040 con 0x7f734410e450 2024-09-10T09:38:45.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:45.131+0000 7f733b400640 1 -- 172.21.15.5:0/773087313 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f733c030920 con 0x7f734410e450 2024-09-10T09:38:45.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:45.131+0000 7f733b400640 1 -- 172.21.15.5:0/773087313 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f733c0045b0 con 0x7f734410e450 2024-09-10T09:38:45.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:45.131+0000 7f7349400640 1 -- 172.21.15.5:0/773087313 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f734419f860 con 0x7f734410e450 2024-09-10T09:38:45.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:45.131+0000 7f7349400640 1 -- 172.21.15.5:0/773087313 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f734419fcc0 con 0x7f734410e450 2024-09-10T09:38:45.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:45.134+0000 7f734c00c740 1 Processor -- start 2024-09-10T09:38:45.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:45.135+0000 7f733b400640 1 -- 172.21.15.5:0/773087313 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 19) ==== 137883+0+0 (secure 0 0 0) 0x7f733c037020 con 0x7f734410e450 2024-09-10T09:38:45.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:45.135+0000 7f734c00c740 1 -- start start 2024-09-10T09:38:45.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:45.135+0000 7f734c00c740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562a04901330 0x562a04901710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:45.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:45.135+0000 7f734c00c740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562a04901c50 0x562a0490bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:45.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:45.135+0000 7f734c00c740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562a0490c510 0x562a0490e900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:45.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:45.135+0000 7f734c00c740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x562a048670b0 con 0x562a0490c510 2024-09-10T09:38:45.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:45.135+0000 7f734c00c740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x562a0480eab0 con 0x562a04901c50 2024-09-10T09:38:45.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:45.136+0000 7f734c00c740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x562a04863070 con 0x562a04901330 2024-09-10T09:38:45.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:45.136+0000 7f733be00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562a04901330 0x562a04901710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:45.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:45.136+0000 7f7342800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562a04901c50 0x562a0490bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:45.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:45.136+0000 7f733be00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562a0490c510 0x562a0490e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:45.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:45.136+0000 7f7342800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562a04901c50 0x562a0490bfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:49686/0 (socket says 172.21.15.5:49686) 2024-09-10T09:38:45.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:45.136+0000 7f7342800640 1 -- 172.21.15.5:0/335945517 learned_addr learned my addr 172.21.15.5:0/335945517 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:38:45.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:45.136+0000 7f7342800640 1 -- 172.21.15.5:0/335945517 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562a04901330 msgr2=0x562a04901710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:38:45.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:45.136+0000 7f7342800640 1 --2- 172.21.15.5:0/335945517 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562a04901330 0x562a04901710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:45.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:45.136+0000 7f7342800640 1 -- 172.21.15.5:0/335945517 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562a0490c510 msgr2=0x562a0490e900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:38:45.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:45.136+0000 7f7342800640 1 --2- 172.21.15.5:0/335945517 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562a0490c510 0x562a0490e900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:45.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:45.136+0000 7f7342800640 1 -- 172.21.15.5:0/335945517 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562a0481b4a0 con 0x562a04901c50 2024-09-10T09:38:45.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:45.136+0000 7f733be00640 1 --2- 172.21.15.5:0/335945517 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562a04901330 0x562a04901710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:38:45.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:45.137+0000 7f7342800640 1 --2- 172.21.15.5:0/335945517 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562a04901c50 0x562a0490bfd0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f7334009e00 tx=0x7f7334011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2ed1eb04f3f231c5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:38:45.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:45.137+0000 7f733b400640 1 --2- 172.21.15.5:0/773087313 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f73280a2c90 0x7f73280a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:45.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:45.137+0000 7f733b400640 1 -- 172.21.15.5:0/773087313 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f733c0f00c0 con 0x7f734410e450 2024-09-10T09:38:45.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:45.137+0000 7f7333400640 1 -- 172.21.15.5:0/335945517 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f733401e070 con 0x562a04901c50 2024-09-10T09:38:45.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:45.137+0000 7f733be00640 1 --2- 172.21.15.5:0/773087313 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f73280a2c90 0x7f73280a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:45.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:45.137+0000 7f7333400640 1 -- 172.21.15.5:0/335945517 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7334019b10 con 0x562a04901c50 2024-09-10T09:38:45.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:45.137+0000 7f7333400640 1 -- 172.21.15.5:0/335945517 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f733400fbc0 con 0x562a04901c50 2024-09-10T09:38:45.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:45.138+0000 7f734c00c740 1 -- 172.21.15.5:0/335945517 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562a04901c50 msgr2=0x562a0490bfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:38:45.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:45.138+0000 7f734c00c740 1 --2- 172.21.15.5:0/335945517 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562a04901c50 0x562a0490bfd0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f7334009e00 tx=0x7f7334011810 comp rx=0 tx=0).stop 2024-09-10T09:38:45.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:45.138+0000 7f733be00640 1 --2- 172.21.15.5:0/773087313 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f73280a2c90 0x7f73280a5150 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x562a048fec40 tx=0x7f732c004730 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:38:45.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:45.138+0000 7f734c00c740 1 -- 172.21.15.5:0/335945517 shutdown_connections 2024-09-10T09:38:45.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:45.138+0000 7f734c00c740 1 --2- 172.21.15.5:0/335945517 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562a0490c510 0x562a0490e900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:45.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:45.138+0000 7f734c00c740 1 --2- 172.21.15.5:0/335945517 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562a04901c50 0x562a0490bfd0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:45.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:45.138+0000 7f734c00c740 1 --2- 172.21.15.5:0/335945517 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562a04901330 0x562a04901710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:45.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:45.138+0000 7f734c00c740 1 -- 172.21.15.5:0/335945517 >> 172.21.15.5:0/335945517 conn(0x562a048fd8c0 msgr2=0x562a048fdca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:38:45.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:45.138+0000 7f734c00c740 1 -- 172.21.15.5:0/335945517 shutdown_connections 2024-09-10T09:38:45.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:45.138+0000 7f734c00c740 1 -- 172.21.15.5:0/335945517 wait complete. 2024-09-10T09:38:45.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:45.139+0000 7f734c00c740 1 -- start start 2024-09-10T09:38:45.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:45.139+0000 7f734c00c740 1 Processor -- start 2024-09-10T09:38:45.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:45.139+0000 7f734c00c740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562a04901330 0x562a04909670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:45.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:45.140+0000 7f734c00c740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562a04901c50 0x562a04909bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:45.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:45.140+0000 7f734c00c740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562a0490c510 0x562a0490a110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:45.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:45.140+0000 7f734c00c740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x562a0490f060 con 0x562a04901c50 2024-09-10T09:38:45.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:45.140+0000 7f734c00c740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x562a047ef920 con 0x562a0490c510 2024-09-10T09:38:45.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:45.140+0000 7f734c00c740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x562a0490f1e0 con 0x562a04901330 2024-09-10T09:38:45.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:45.140+0000 7f734c00c740 10 client.0 _ll_register_callbacks cb 0x562a048138a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:38:45.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:45.140+0000 7f7342800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562a04901330 0x562a04909670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:45.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:45.140+0000 7f7342800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562a04901330 0x562a04909670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:33022/0 (socket says 172.21.15.5:33022) 2024-09-10T09:38:45.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:45.140+0000 7f7342800640 1 -- 172.21.15.5:0/3843765583 learned_addr learned my addr 172.21.15.5:0/3843765583 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:38:45.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:45.140+0000 7f733be00640 1 --2- 172.21.15.5:0/3843765583 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562a04901c50 0x562a04909bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:45.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:45.140+0000 7f7343200640 1 --2- 172.21.15.5:0/3843765583 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562a0490c510 0x562a0490a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:45.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:45.140+0000 7f7342800640 1 -- 172.21.15.5:0/3843765583 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562a0490c510 msgr2=0x562a0490a110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:38:45.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:45.140+0000 7f7342800640 1 --2- 172.21.15.5:0/3843765583 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562a0490c510 0x562a0490a110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:45.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:45.140+0000 7f7342800640 1 -- 172.21.15.5:0/3843765583 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562a04901c50 msgr2=0x562a04909bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:38:45.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:45.140+0000 7f7342800640 1 --2- 172.21.15.5:0/3843765583 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562a04901c50 0x562a04909bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:45.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:45.140+0000 7f7342800640 1 -- 172.21.15.5:0/3843765583 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562a045d82b0 con 0x562a04901330 2024-09-10T09:38:45.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:45.140+0000 7f7343200640 1 --2- 172.21.15.5:0/3843765583 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562a0490c510 0x562a0490a110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:38:45.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:45.141+0000 7f7342800640 1 --2- 172.21.15.5:0/3843765583 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562a04901330 0x562a04909670 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f733401c040 tx=0x7f7334019ef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a8725f6b37e6934 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:38:45.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:45.141+0000 7f7327e00640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-09-10T09:38:45.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:45.141+0000 7f7327e00640 1 -- 172.21.15.5:0/3843765583 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f733401e050 con 0x562a04901330 2024-09-10T09:38:45.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:45.141+0000 7f734c00c740 1 -- 172.21.15.5:0/3843765583 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7334019b10 con 0x562a04901330 2024-09-10T09:38:45.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:45.141+0000 7f7327e00640 1 -- 172.21.15.5:0/3843765583 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f7334019840 con 0x562a04901330 2024-09-10T09:38:45.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:45.141+0000 7f7327e00640 1 -- 172.21.15.5:0/3843765583 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f7334009a90 con 0x562a04901330 2024-09-10T09:38:45.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:45.142+0000 7f734c00c740 10 client.25399 fetch_fsmap learned FSMap version 30 2024-09-10T09:38:45.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:45.142+0000 7f734c00c740 1 -- 172.21.15.5:0/3843765583 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x562a0491ae70 con 0x562a04901330 2024-09-10T09:38:45.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:45.142+0000 7f7327e00640 1 -- 172.21.15.5:0/3843765583 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7334019b10 con 0x562a04901330 2024-09-10T09:38:45.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:45.142+0000 7f7327e00640 1 -- 172.21.15.5:0/3843765583 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f73340037a0 con 0x562a04901330 2024-09-10T09:38:45.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:45.142+0000 7f734c00c740 10 client.25399 fetch_fsmap finished waiting for FSMap version 30 2024-09-10T09:38:45.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:45.143+0000 7f734c00c740 10 client.25399 resolve_mds: resolved name 'k' to daemon mds.k 2024-09-10T09:38:45.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:45.143+0000 7f734c00c740 20 client.25399 populate_metadata read hostname 'smithi005' 2024-09-10T09:38:45.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:45.143+0000 7f734c00c740 1 --2- 172.21.15.5:0/3843765583 >> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] conn(0x562a0491d2d0 0x562a0491e740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:45.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:45.143+0000 7f734c00c740 4 client.25399 mds_command: new command op to 24443 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:38:45.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:45.143+0000 7f734c00c740 1 -- 172.21.15.5:0/3843765583 --> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562a04874620 con 0x562a0491d2d0 2024-09-10T09:38:45.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:45.143+0000 7f733be00640 1 --2- 172.21.15.5:0/3843765583 >> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] conn(0x562a0491d2d0 0x562a0491e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:45.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:45.144+0000 7f733be00640 1 --2- 172.21.15.5:0/3843765583 >> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] conn(0x562a0491d2d0 0x562a0491e740 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=53842ab6c6115a2a server_cookie=1894ac1faf0e7c1e in_seq=0 out_seq=0 2024-09-10T09:38:45.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:45.144+0000 7f7327e00640 10 client.25399 ms_handle_connect on v2:172.21.15.102:6832/2457007427 2024-09-10T09:38:45.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:45.144+0000 7f7327e00640 1 -- 172.21.15.5:0/3843765583 <== mds.4 v2:172.21.15.102:6832/2457007427 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562a04874620 con 0x562a0491d2d0 2024-09-10T09:38:45.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:45.144+0000 7f7327e00640 10 client.25399 handle_command_reply: tid=0 2024-09-10T09:38:45.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:45.145+0000 7f734c00c740 1 client.25399 shutdown 2024-09-10T09:38:45.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:45.145+0000 7f734c00c740 1 -- 172.21.15.5:0/3843765583 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562a04901330 msgr2=0x562a04909670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:38:45.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:45.145+0000 7f734c00c740 1 --2- 172.21.15.5:0/3843765583 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562a04901330 0x562a04909670 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f733401c040 tx=0x7f7334019ef0 comp rx=0 tx=0).stop 2024-09-10T09:38:45.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:45.145+0000 7f734c00c740 1 -- 172.21.15.5:0/3843765583 shutdown_connections 2024-09-10T09:38:45.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:45.145+0000 7f734c00c740 1 --2- 172.21.15.5:0/3843765583 >> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] conn(0x562a0491d2d0 0x562a0491e740 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:45.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:45.145+0000 7f734c00c740 1 --2- 172.21.15.5:0/3843765583 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562a0490c510 0x562a0490a110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:45.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:45.145+0000 7f734c00c740 1 --2- 172.21.15.5:0/3843765583 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562a04901c50 0x562a04909bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:45.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:45.145+0000 7f734c00c740 1 --2- 172.21.15.5:0/3843765583 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562a04901330 0x562a04909670 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:45.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:45.145+0000 7f7327e00640 0 client.25399 ms_handle_reset on v2:172.21.15.102:6832/2457007427 2024-09-10T09:38:45.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:45.145+0000 7f734c00c740 1 -- 172.21.15.5:0/3843765583 >> 172.21.15.5:0/3843765583 conn(0x562a048fd8c0 msgr2=0x562a049035a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:38:45.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:45.145+0000 7f734c00c740 1 -- 172.21.15.5:0/3843765583 shutdown_connections 2024-09-10T09:38:45.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:45.145+0000 7f734c00c740 1 -- 172.21.15.5:0/3843765583 wait complete. 2024-09-10T09:38:45.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:45.145+0000 7f734c00c740 20 client.25399 trim_cache size 0 max 16384 2024-09-10T09:38:45.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:45.164+0000 7f734c00c740 1 Processor -- start 2024-09-10T09:38:45.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:45.164+0000 7f734c00c740 1 -- start start 2024-09-10T09:38:45.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:45.164+0000 7f734c00c740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562a04901330 0x562a04908f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:45.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:45.164+0000 7f734c00c740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562a04901c50 0x562a0491c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:45.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:45.164+0000 7f734c00c740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562a0490c510 0x562a04903f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:45.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:45.164+0000 7f734c00c740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x562a0491ca80 con 0x562a0490c510 2024-09-10T09:38:45.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:45.164+0000 7f734c00c740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x562a0491d150 con 0x562a04901330 2024-09-10T09:38:45.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:45.164+0000 7f734c00c740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x562a0491d820 con 0x562a04901c50 2024-09-10T09:38:45.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:45.165+0000 7f7342800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562a04901330 0x562a04908f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:45.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:45.165+0000 7f7343200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562a0490c510 0x562a04903f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:45.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:45.165+0000 7f7342800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562a04901330 0x562a04908f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:49712/0 (socket says 172.21.15.5:49712) 2024-09-10T09:38:45.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:45.165+0000 7f7342800640 1 -- 172.21.15.5:0/443284063 learned_addr learned my addr 172.21.15.5:0/443284063 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:38:45.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:45.165+0000 7f733be00640 1 --2- 172.21.15.5:0/443284063 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562a04901c50 0x562a0491c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:45.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:45.165+0000 7f7342800640 1 -- 172.21.15.5:0/443284063 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562a04901c50 msgr2=0x562a0491c2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:38:45.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:45.165+0000 7f7342800640 1 --2- 172.21.15.5:0/443284063 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562a04901c50 0x562a0491c2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:45.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:45.165+0000 7f7342800640 1 -- 172.21.15.5:0/443284063 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562a0490c510 msgr2=0x562a04903f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:38:45.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:45.165+0000 7f7342800640 1 --2- 172.21.15.5:0/443284063 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562a0490c510 0x562a04903f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:45.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:45.165+0000 7f7342800640 1 -- 172.21.15.5:0/443284063 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562a04901950 con 0x562a04901330 2024-09-10T09:38:45.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:45.165+0000 7f7342800640 1 --2- 172.21.15.5:0/443284063 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562a04901330 0x562a04908f30 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f7334009250 tx=0x7f7334011d10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a985fe2b11ce3482 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:38:45.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:45.166+0000 7f7332a00640 1 -- 172.21.15.5:0/443284063 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f733401e070 con 0x562a04901330 2024-09-10T09:38:45.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:45.166+0000 7f7332a00640 1 -- 172.21.15.5:0/443284063 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f733402a070 con 0x562a04901330 2024-09-10T09:38:45.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:45.166+0000 7f7332a00640 1 -- 172.21.15.5:0/443284063 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7334025720 con 0x562a04901330 2024-09-10T09:38:45.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:45.166+0000 7f734c00c740 1 -- 172.21.15.5:0/443284063 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562a04901330 msgr2=0x562a04908f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:38:45.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:45.166+0000 7f734c00c740 1 --2- 172.21.15.5:0/443284063 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562a04901330 0x562a04908f30 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f7334009250 tx=0x7f7334011d10 comp rx=0 tx=0).stop 2024-09-10T09:38:45.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:45.167+0000 7f734c00c740 1 -- 172.21.15.5:0/443284063 shutdown_connections 2024-09-10T09:38:45.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:45.167+0000 7f734c00c740 1 --2- 172.21.15.5:0/443284063 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562a0490c510 0x562a04903f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:45.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:45.167+0000 7f734c00c740 1 --2- 172.21.15.5:0/443284063 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562a04901c50 0x562a0491c2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:45.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:45.167+0000 7f734c00c740 1 --2- 172.21.15.5:0/443284063 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562a04901330 0x562a04908f30 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:45.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:45.167+0000 7f734c00c740 1 -- 172.21.15.5:0/443284063 >> 172.21.15.5:0/443284063 conn(0x562a048fd8c0 msgr2=0x562a048fdca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:38:45.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:45.167+0000 7f734c00c740 1 -- 172.21.15.5:0/443284063 shutdown_connections 2024-09-10T09:38:45.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:45.167+0000 7f734c00c740 1 -- 172.21.15.5:0/443284063 wait complete. 2024-09-10T09:38:45.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:45.167+0000 7f734c00c740 1 -- start start 2024-09-10T09:38:45.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:45.167+0000 7f734c00c740 1 Processor -- start 2024-09-10T09:38:45.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:45.168+0000 7f734c00c740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562a04901330 0x562a04906160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:45.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:45.168+0000 7f734c00c740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562a04901c50 0x562a049066a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:45.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:45.168+0000 7f734c00c740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562a0490c510 0x562a04925710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:45.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:45.168+0000 7f734c00c740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x562a0491e5c0 con 0x562a0490c510 2024-09-10T09:38:45.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:45.168+0000 7f734c00c740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x562a0491def0 con 0x562a04901c50 2024-09-10T09:38:45.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:45.168+0000 7f734c00c740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x562a0491ec90 con 0x562a04901330 2024-09-10T09:38:45.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:45.168+0000 7f734c00c740 10 client.0 _ll_register_callbacks cb 0x7f7334009c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:38:45.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:45.168+0000 7f7342800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562a04901330 0x562a04906160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:45.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:45.168+0000 7f7343200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562a0490c510 0x562a04925710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:45.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:45.168+0000 7f7342800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562a04901330 0x562a04906160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:33040/0 (socket says 172.21.15.5:33040) 2024-09-10T09:38:45.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:45.168+0000 7f7342800640 1 -- 172.21.15.5:0/2160912042 learned_addr learned my addr 172.21.15.5:0/2160912042 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:38:45.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:45.168+0000 7f733be00640 1 --2- 172.21.15.5:0/2160912042 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562a04901c50 0x562a049066a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:45.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:45.169+0000 7f7342800640 1 -- 172.21.15.5:0/2160912042 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562a04901c50 msgr2=0x562a049066a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:38:45.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:45.169+0000 7f7342800640 1 --2- 172.21.15.5:0/2160912042 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562a04901c50 0x562a049066a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:45.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:45.169+0000 7f7342800640 1 -- 172.21.15.5:0/2160912042 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562a0490c510 msgr2=0x562a04925710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:38:45.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:45.169+0000 7f7342800640 1 --2- 172.21.15.5:0/2160912042 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562a0490c510 0x562a04925710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:45.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:45.169+0000 7f7342800640 1 -- 172.21.15.5:0/2160912042 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562a04925d00 con 0x562a04901330 2024-09-10T09:38:45.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:45.169+0000 7f7342800640 1 --2- 172.21.15.5:0/2160912042 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562a04901330 0x562a04906160 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f7334003c40 tx=0x7f7334005660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=45e20a5f5ea6eabd server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:38:45.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:45.169+0000 7f7327e00640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-09-10T09:38:45.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:45.169+0000 7f7327e00640 1 -- 172.21.15.5:0/2160912042 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f733401e070 con 0x562a04901330 2024-09-10T09:38:45.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:45.169+0000 7f7327e00640 1 -- 172.21.15.5:0/2160912042 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f7334003800 con 0x562a04901330 2024-09-10T09:38:45.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:45.169+0000 7f734c00c740 1 -- 172.21.15.5:0/2160912042 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f733402a070 con 0x562a04901330 2024-09-10T09:38:45.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:45.169+0000 7f7327e00640 1 -- 172.21.15.5:0/2160912042 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f7334019640 con 0x562a04901330 2024-09-10T09:38:45.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:45.170+0000 7f734c00c740 10 client.25405 fetch_fsmap learned FSMap version 30 2024-09-10T09:38:45.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:45.170+0000 7f734c00c740 1 -- 172.21.15.5:0/2160912042 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x562a04927090 con 0x562a04901330 2024-09-10T09:38:45.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:45.170+0000 7f7327e00640 1 -- 172.21.15.5:0/2160912042 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f733402a070 con 0x562a04901330 2024-09-10T09:38:45.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:45.170+0000 7f7327e00640 1 -- 172.21.15.5:0/2160912042 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f7334037670 con 0x562a04901330 2024-09-10T09:38:45.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:45.170+0000 7f734c00c740 10 client.25405 fetch_fsmap finished waiting for FSMap version 30 2024-09-10T09:38:45.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:45.170+0000 7f734c00c740 10 client.25405 resolve_mds: resolved name 'k' to daemon mds.k 2024-09-10T09:38:45.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:45.170+0000 7f734c00c740 20 client.25405 populate_metadata read hostname 'smithi005' 2024-09-10T09:38:45.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:45.170+0000 7f734c00c740 1 --2- 172.21.15.5:0/2160912042 >> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] conn(0x562a0498b9b0 0x562a0498a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:45.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:45.170+0000 7f734c00c740 4 client.25405 mds_command: new command op to 24443 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T09:38:45.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:45.170+0000 7f734c00c740 1 -- 172.21.15.5:0/2160912042 --> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] -- command(tid 0: {"prefix": "status"}) -- 0x562a0486eb30 con 0x562a0498b9b0 2024-09-10T09:38:45.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:45.171+0000 7f733be00640 1 --2- 172.21.15.5:0/2160912042 >> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] conn(0x562a0498b9b0 0x562a0498a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:45.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:45.171+0000 7f733be00640 1 --2- 172.21.15.5:0/2160912042 >> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] conn(0x562a0498b9b0 0x562a0498a6c0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=4bc59ddc86e8cb8f server_cookie=77a04641984ba3a in_seq=0 out_seq=0 2024-09-10T09:38:45.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:45.171+0000 7f7327e00640 10 client.25405 ms_handle_connect on v2:172.21.15.102:6832/2457007427 2024-09-10T09:38:45.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:45.171+0000 7f7327e00640 1 -- 172.21.15.5:0/2160912042 <== mds.4 v2:172.21.15.102:6832/2457007427 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x562a0486eb30 con 0x562a0498b9b0 2024-09-10T09:38:45.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:45.171+0000 7f7327e00640 10 client.25405 handle_command_reply: tid=0 2024-09-10T09:38:45.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:45.171+0000 7f734c00c740 1 client.25405 shutdown 2024-09-10T09:38:45.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:45.171+0000 7f734c00c740 1 -- 172.21.15.5:0/2160912042 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562a04901330 msgr2=0x562a04906160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:38:45.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:45.171+0000 7f734c00c740 1 --2- 172.21.15.5:0/2160912042 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562a04901330 0x562a04906160 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f7334003c40 tx=0x7f7334005660 comp rx=0 tx=0).stop 2024-09-10T09:38:45.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:45.171+0000 7f734c00c740 1 -- 172.21.15.5:0/2160912042 shutdown_connections 2024-09-10T09:38:45.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:45.171+0000 7f734c00c740 1 --2- 172.21.15.5:0/2160912042 >> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] conn(0x562a0498b9b0 0x562a0498a6c0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:45.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:45.171+0000 7f734c00c740 1 --2- 172.21.15.5:0/2160912042 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562a0490c510 0x562a04925710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:45.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:45.171+0000 7f734c00c740 1 --2- 172.21.15.5:0/2160912042 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562a04901c50 0x562a049066a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:45.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:45.171+0000 7f734c00c740 1 --2- 172.21.15.5:0/2160912042 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562a04901330 0x562a04906160 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:45.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:45.171+0000 7f7327e00640 0 client.25405 ms_handle_reset on v2:172.21.15.102:6832/2457007427 2024-09-10T09:38:45.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:45.172+0000 7f734c00c740 1 -- 172.21.15.5:0/2160912042 >> 172.21.15.5:0/2160912042 conn(0x562a048fd8c0 msgr2=0x562a0491c2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:38:45.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:45.172+0000 7f734c00c740 1 -- 172.21.15.5:0/2160912042 shutdown_connections 2024-09-10T09:38:45.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:45.172+0000 7f734c00c740 1 -- 172.21.15.5:0/2160912042 wait complete. 2024-09-10T09:38:45.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:45.172+0000 7f734c00c740 20 client.25405 trim_cache size 0 max 16384 2024-09-10T09:38:45.177 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-10T09:38:45.177 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_fsid": "bade4b50-6f56-11ef-bcea-c7b262605968", 2024-09-10T09:38:45.177 INFO:teuthology.orchestra.run.smithi005.stdout: "whoami": 4, 2024-09-10T09:38:45.177 INFO:teuthology.orchestra.run.smithi005.stdout: "id": 24443, 2024-09-10T09:38:45.177 INFO:teuthology.orchestra.run.smithi005.stdout: "want_state": "up:active", 2024-09-10T09:38:45.177 INFO:teuthology.orchestra.run.smithi005.stdout: "state": "up:active", 2024-09-10T09:38:45.177 INFO:teuthology.orchestra.run.smithi005.stdout: "fs_name": "cephfs", 2024-09-10T09:38:45.177 INFO:teuthology.orchestra.run.smithi005.stdout: "rank_uptime": 150.68455485999999, 2024-09-10T09:38:45.177 INFO:teuthology.orchestra.run.smithi005.stdout: "mdsmap_epoch": 30, 2024-09-10T09:38:45.178 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch": 96, 2024-09-10T09:38:45.178 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch_barrier": 96, 2024-09-10T09:38:45.178 INFO:teuthology.orchestra.run.smithi005.stdout: "uptime": 240.218336314 2024-09-10T09:38:45.178 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-10T09:38:45.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:45.173+0000 7f7349400640 1 -- 172.21.15.5:0/773087313 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f73280a2c90 msgr2=0x7f73280a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:38:45.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:45.173+0000 7f7349400640 1 --2- 172.21.15.5:0/773087313 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f73280a2c90 0x7f73280a5150 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x562a048fec40 tx=0x7f732c004730 comp rx=0 tx=0).stop 2024-09-10T09:38:45.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:45.173+0000 7f7349400640 1 -- 172.21.15.5:0/773087313 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f734410e450 msgr2=0x7f734419eef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:38:45.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:45.173+0000 7f7349400640 1 --2- 172.21.15.5:0/773087313 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f734410e450 0x7f734419eef0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f733c02ae70 tx=0x7f733c02f3a0 comp rx=0 tx=0).stop 2024-09-10T09:38:45.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:45.173+0000 7f7349400640 1 -- 172.21.15.5:0/773087313 shutdown_connections 2024-09-10T09:38:45.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:45.173+0000 7f7349400640 1 --2- 172.21.15.5:0/773087313 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f73280a2c90 0x7f73280a5150 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:45.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:45.173+0000 7f7349400640 1 --2- 172.21.15.5:0/773087313 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f734410e450 0x7f734419eef0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:45.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:45.173+0000 7f7349400640 1 --2- 172.21.15.5:0/773087313 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7344074e30 0x7f734419ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:45.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:45.173+0000 7f7349400640 1 --2- 172.21.15.5:0/773087313 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7344074510 0x7f734419a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:45.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:45.173+0000 7f7349400640 1 -- 172.21.15.5:0/773087313 >> 172.21.15.5:0/773087313 conn(0x7f73440fd8b0 msgr2=0x7f73440fe4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:38:45.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:45.173+0000 7f7349400640 1 -- 172.21.15.5:0/773087313 shutdown_connections 2024-09-10T09:38:45.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:45.173+0000 7f7349400640 1 -- 172.21.15.5:0/773087313 wait complete. 2024-09-10T09:38:45.306 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:38:45 smithi005 ceph-mon[30350]: pgmap v405: 129 pgs: 129 active+clean; 655 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-10T09:38:45.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:38:45 smithi102 ceph-mon[35442]: pgmap v405: 129 pgs: 129 active+clean; 655 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-10T09:38:45.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:38:45 smithi159 ceph-mon[35513]: pgmap v405: 129 pgs: 129 active+clean; 655 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-10T09:38:45.978 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph tell mds.h status 2024-09-10T09:38:46.355 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:38:46.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:46.901+0000 7f1041400640 1 -- 172.21.15.5:0/2281366170 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f103c101de0 msgr2=0x7f103c1021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:38:46.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:46.901+0000 7f1041400640 1 --2- 172.21.15.5:0/2281366170 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f103c101de0 0x7f103c1021c0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f102c009900 tx=0x7f102c02a3f0 comp rx=0 tx=0).stop 2024-09-10T09:38:46.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:46.901+0000 7f1041400640 1 -- 172.21.15.5:0/2281366170 shutdown_connections 2024-09-10T09:38:46.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:46.901+0000 7f1041400640 1 --2- 172.21.15.5:0/2281366170 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f103c106f10 0x7f103c10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:46.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:46.901+0000 7f1041400640 1 --2- 172.21.15.5:0/2281366170 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f103c102790 0x7f103c1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:46.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:46.901+0000 7f1041400640 1 --2- 172.21.15.5:0/2281366170 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f103c101de0 0x7f103c1021c0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:46.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:46.901+0000 7f1041400640 1 -- 172.21.15.5:0/2281366170 >> 172.21.15.5:0/2281366170 conn(0x7f103c0fd8c0 msgr2=0x7f103c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:38:46.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:46.901+0000 7f1041400640 1 -- 172.21.15.5:0/2281366170 shutdown_connections 2024-09-10T09:38:46.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:46.901+0000 7f1041400640 1 -- 172.21.15.5:0/2281366170 wait complete. 2024-09-10T09:38:46.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:46.902+0000 7f1041400640 1 Processor -- start 2024-09-10T09:38:46.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:46.902+0000 7f1041400640 1 -- start start 2024-09-10T09:38:46.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:46.903+0000 7f1041400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f103c101de0 0x7f103c198450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:46.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:46.903+0000 7f1041400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f103c102790 0x7f103c198990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:46.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:46.903+0000 7f1041400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f103c106f10 0x7f103c19cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:46.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:46.903+0000 7f1041400640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f103c111f70 con 0x7f103c101de0 2024-09-10T09:38:46.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:46.903+0000 7f1041400640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f103c111df0 con 0x7f103c102790 2024-09-10T09:38:46.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:46.903+0000 7f1041400640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f103c1120f0 con 0x7f103c106f10 2024-09-10T09:38:46.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:46.903+0000 7f103a800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f103c101de0 0x7f103c198450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:46.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:46.903+0000 7f103a800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f103c101de0 0x7f103c198450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58192/0 (socket says 172.21.15.5:58192) 2024-09-10T09:38:46.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:46.903+0000 7f103a800640 1 -- 172.21.15.5:0/894401824 learned_addr learned my addr 172.21.15.5:0/894401824 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:38:46.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:46.903+0000 7f1031e00640 1 --2- 172.21.15.5:0/894401824 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f103c102790 0x7f103c198990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:46.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:46.903+0000 7f103b200640 1 --2- 172.21.15.5:0/894401824 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f103c106f10 0x7f103c19cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:46.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:46.903+0000 7f103a800640 1 -- 172.21.15.5:0/894401824 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f103c106f10 msgr2=0x7f103c19cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:38:46.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:46.903+0000 7f103a800640 1 --2- 172.21.15.5:0/894401824 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f103c106f10 0x7f103c19cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:46.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:46.903+0000 7f103a800640 1 -- 172.21.15.5:0/894401824 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f103c102790 msgr2=0x7f103c198990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:38:46.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:46.903+0000 7f103a800640 1 --2- 172.21.15.5:0/894401824 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f103c102790 0x7f103c198990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:46.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:46.903+0000 7f103a800640 1 -- 172.21.15.5:0/894401824 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f103c19d400 con 0x7f103c101de0 2024-09-10T09:38:46.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:46.904+0000 7f103a800640 1 --2- 172.21.15.5:0/894401824 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f103c101de0 0x7f103c198450 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f102c006010 tx=0x7f102c004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:38:46.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:46.904+0000 7f1033e00640 1 -- 172.21.15.5:0/894401824 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f102c03f030 con 0x7f103c101de0 2024-09-10T09:38:46.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:46.904+0000 7f1041400640 1 -- 172.21.15.5:0/894401824 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f103c19d690 con 0x7f103c101de0 2024-09-10T09:38:46.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:46.904+0000 7f1033e00640 1 -- 172.21.15.5:0/894401824 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f102c004440 con 0x7f103c101de0 2024-09-10T09:38:46.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:46.904+0000 7f1033e00640 1 -- 172.21.15.5:0/894401824 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f102c035070 con 0x7f103c101de0 2024-09-10T09:38:46.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:46.904+0000 7f1041400640 1 -- 172.21.15.5:0/894401824 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f103c1a4fd0 con 0x7f103c101de0 2024-09-10T09:38:46.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:46.908+0000 7f1043dea740 1 Processor -- start 2024-09-10T09:38:46.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:46.908+0000 7f1043dea740 1 -- start start 2024-09-10T09:38:46.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:46.909+0000 7f1033e00640 1 -- 172.21.15.5:0/894401824 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 19) ==== 137883+0+0 (secure 0 0 0) 0x7f102c042030 con 0x7f103c101de0 2024-09-10T09:38:46.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:46.909+0000 7f1043dea740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55601b787210 0x55601b7875f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:46.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:46.909+0000 7f1043dea740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55601b787b30 0x55601b7963e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:46.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:46.909+0000 7f1043dea740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55601b796920 0x55601b798d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:46.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:46.909+0000 7f1043dea740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55601b6ed0b0 con 0x55601b787210 2024-09-10T09:38:46.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:46.909+0000 7f1043dea740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55601b694ab0 con 0x55601b796920 2024-09-10T09:38:46.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:46.909+0000 7f1043dea740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55601b6e9070 con 0x55601b787b30 2024-09-10T09:38:46.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:46.909+0000 7f103b200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55601b787210 0x55601b7875f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:46.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:46.909+0000 7f103a800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55601b787b30 0x55601b7963e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:46.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:46.909+0000 7f103a800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55601b787b30 0x55601b7963e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:33084/0 (socket says 172.21.15.5:33084) 2024-09-10T09:38:46.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:46.909+0000 7f103a800640 1 -- 172.21.15.5:0/1351649759 learned_addr learned my addr 172.21.15.5:0/1351649759 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:38:46.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:46.909+0000 7f103b200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55601b787210 0x55601b7875f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58194/0 (socket says 172.21.15.5:58194) 2024-09-10T09:38:46.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:46.909+0000 7f1031e00640 1 --2- 172.21.15.5:0/1351649759 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55601b796920 0x55601b798d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:46.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:46.910+0000 7f103a800640 1 -- 172.21.15.5:0/1351649759 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55601b796920 msgr2=0x55601b798d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:38:46.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:46.910+0000 7f103a800640 1 --2- 172.21.15.5:0/1351649759 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55601b796920 0x55601b798d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:46.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:46.910+0000 7f103a800640 1 -- 172.21.15.5:0/1351649759 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55601b787210 msgr2=0x55601b7875f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:38:46.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:46.910+0000 7f103a800640 1 --2- 172.21.15.5:0/1351649759 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55601b787210 0x55601b7875f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:46.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:46.910+0000 7f103a800640 1 -- 172.21.15.5:0/1351649759 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55601b6a14a0 con 0x55601b787b30 2024-09-10T09:38:46.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:46.910+0000 7f1031e00640 1 --2- 172.21.15.5:0/1351649759 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55601b796920 0x55601b798d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:38:46.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:46.910+0000 7f103a800640 1 --2- 172.21.15.5:0/1351649759 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55601b787b30 0x55601b7963e0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f102c0ecd00 tx=0x7f102c0f5cb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=47657edf93445af0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:38:46.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:46.910+0000 7f1033e00640 1 --2- 172.21.15.5:0/894401824 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f10200a2c10 0x7f10200a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:46.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:46.911+0000 7f102b400640 1 -- 172.21.15.5:0/1351649759 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f102c0fe070 con 0x55601b787b30 2024-09-10T09:38:46.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:46.911+0000 7f1033e00640 1 -- 172.21.15.5:0/894401824 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f102c0ef800 con 0x7f103c101de0 2024-09-10T09:38:46.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:46.911+0000 7f103b200640 1 --2- 172.21.15.5:0/894401824 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f10200a2c10 0x7f10200a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:46.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:46.911+0000 7f102b400640 1 -- 172.21.15.5:0/1351649759 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f102c03ae50 con 0x55601b787b30 2024-09-10T09:38:46.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:46.911+0000 7f102b400640 1 -- 172.21.15.5:0/1351649759 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f102c0bb8e0 con 0x55601b787b30 2024-09-10T09:38:46.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:46.911+0000 7f1043dea740 1 -- 172.21.15.5:0/1351649759 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55601b787b30 msgr2=0x55601b7963e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:38:46.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:46.911+0000 7f103b200640 1 --2- 172.21.15.5:0/894401824 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f10200a2c10 0x7f10200a50d0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f1034005fd0 tx=0x7f1034009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:38:46.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:46.911+0000 7f1043dea740 1 --2- 172.21.15.5:0/1351649759 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55601b787b30 0x55601b7963e0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f102c0ecd00 tx=0x7f102c0f5cb0 comp rx=0 tx=0).stop 2024-09-10T09:38:46.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:46.912+0000 7f1043dea740 1 -- 172.21.15.5:0/1351649759 shutdown_connections 2024-09-10T09:38:46.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:46.912+0000 7f1043dea740 1 --2- 172.21.15.5:0/1351649759 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55601b796920 0x55601b798d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:46.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:46.912+0000 7f1043dea740 1 --2- 172.21.15.5:0/1351649759 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55601b787b30 0x55601b7963e0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:46.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:46.912+0000 7f1043dea740 1 --2- 172.21.15.5:0/1351649759 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55601b787210 0x55601b7875f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:46.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:46.912+0000 7f1043dea740 1 -- 172.21.15.5:0/1351649759 >> 172.21.15.5:0/1351649759 conn(0x55601b7838c0 msgr2=0x55601b783ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:38:46.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:46.912+0000 7f1043dea740 1 -- 172.21.15.5:0/1351649759 shutdown_connections 2024-09-10T09:38:46.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:46.912+0000 7f1043dea740 1 -- 172.21.15.5:0/1351649759 wait complete. 2024-09-10T09:38:46.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:46.912+0000 7f1043dea740 1 -- start start 2024-09-10T09:38:46.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:46.912+0000 7f1043dea740 1 Processor -- start 2024-09-10T09:38:46.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:46.913+0000 7f1043dea740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55601b787210 0x55601b793830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:46.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:46.913+0000 7f1043dea740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55601b787b30 0x55601b793d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:46.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:46.913+0000 7f1043dea740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55601b796920 0x55601b7942b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:46.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:46.913+0000 7f1043dea740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55601b6e9070 con 0x55601b787b30 2024-09-10T09:38:46.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:46.913+0000 7f1043dea740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55601b675920 con 0x55601b787210 2024-09-10T09:38:46.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:46.913+0000 7f1043dea740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55601b7992b0 con 0x55601b796920 2024-09-10T09:38:46.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:46.913+0000 7f1043dea740 10 client.0 _ll_register_callbacks cb 0x55601b6998a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:38:46.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:46.913+0000 7f1031e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55601b787b30 0x55601b793d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:46.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:46.913+0000 7f1031e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55601b787b30 0x55601b793d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58204/0 (socket says 172.21.15.5:58204) 2024-09-10T09:38:46.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:46.913+0000 7f1031e00640 1 -- 172.21.15.5:0/761920239 learned_addr learned my addr 172.21.15.5:0/761920239 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:38:46.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:46.913+0000 7f103b200640 1 --2- 172.21.15.5:0/761920239 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55601b796920 0x55601b7942b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:46.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:46.913+0000 7f103a800640 1 --2- 172.21.15.5:0/761920239 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55601b787210 0x55601b793830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:46.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:46.913+0000 7f1031e00640 1 -- 172.21.15.5:0/761920239 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55601b796920 msgr2=0x55601b7942b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:38:46.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:46.913+0000 7f1031e00640 1 --2- 172.21.15.5:0/761920239 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55601b796920 0x55601b7942b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:46.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:46.913+0000 7f1031e00640 1 -- 172.21.15.5:0/761920239 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55601b787210 msgr2=0x55601b793830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:38:46.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:46.913+0000 7f1031e00640 1 --2- 172.21.15.5:0/761920239 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55601b787210 0x55601b793830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:46.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:46.913+0000 7f1031e00640 1 -- 172.21.15.5:0/761920239 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55601b45e2b0 con 0x55601b787b30 2024-09-10T09:38:46.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:46.914+0000 7f1031e00640 1 --2- 172.21.15.5:0/761920239 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55601b787b30 0x55601b793d70 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f10240126e0 tx=0x7f1024012ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38b361afaed80a56 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:38:46.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:46.914+0000 7f101fe00640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-10T09:38:46.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:46.914+0000 7f101fe00640 1 -- 172.21.15.5:0/761920239 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1024012190 con 0x55601b787b30 2024-09-10T09:38:46.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:46.914+0000 7f101fe00640 1 -- 172.21.15.5:0/761920239 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f102400f920 con 0x55601b787b30 2024-09-10T09:38:46.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:46.914+0000 7f1043dea740 1 -- 172.21.15.5:0/761920239 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f102c03ae50 con 0x55601b787b30 2024-09-10T09:38:46.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:46.914+0000 7f101fe00640 1 -- 172.21.15.5:0/761920239 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f102400fb20 con 0x55601b787b30 2024-09-10T09:38:46.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:46.914+0000 7f1043dea740 10 client.15384 fetch_fsmap learned FSMap version 30 2024-09-10T09:38:46.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:46.914+0000 7f1043dea740 1 -- 172.21.15.5:0/761920239 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55601b7a4e80 con 0x55601b787b30 2024-09-10T09:38:46.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:46.914+0000 7f101fe00640 1 -- 172.21.15.5:0/761920239 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f102c03ae50 con 0x55601b787b30 2024-09-10T09:38:46.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:46.915+0000 7f101fe00640 1 -- 172.21.15.5:0/761920239 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f102401c420 con 0x55601b787b30 2024-09-10T09:38:46.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:46.915+0000 7f1043dea740 10 client.15384 fetch_fsmap finished waiting for FSMap version 30 2024-09-10T09:38:46.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:46.915+0000 7f1043dea740 10 client.15384 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-10T09:38:46.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:46.915+0000 7f1043dea740 20 client.15384 populate_metadata read hostname 'smithi005' 2024-09-10T09:38:46.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:46.915+0000 7f1043dea740 1 --2- 172.21.15.5:0/761920239 >> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] conn(0x55601b7a7520 0x55601b7a8990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:46.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:46.915+0000 7f1043dea740 4 client.15384 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:38:46.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:46.915+0000 7f1043dea740 1 -- 172.21.15.5:0/761920239 --> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55601b6fa620 con 0x55601b7a7520 2024-09-10T09:38:46.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:46.915+0000 7f103a800640 1 --2- 172.21.15.5:0/761920239 >> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] conn(0x55601b7a7520 0x55601b7a8990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:46.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:46.916+0000 7f103a800640 1 --2- 172.21.15.5:0/761920239 >> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] conn(0x55601b7a7520 0x55601b7a8990 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=a8422db2808646e1 server_cookie=291101af92cbd17 in_seq=0 out_seq=0 2024-09-10T09:38:46.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:46.916+0000 7f101fe00640 10 client.15384 ms_handle_connect on v2:172.21.15.102:6836/2689776764 2024-09-10T09:38:46.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:46.916+0000 7f101fe00640 1 -- 172.21.15.5:0/761920239 <== mds.3 v2:172.21.15.102:6836/2689776764 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55601b6fa620 con 0x55601b7a7520 2024-09-10T09:38:46.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:46.916+0000 7f101fe00640 10 client.15384 handle_command_reply: tid=0 2024-09-10T09:38:46.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:46.916+0000 7f1043dea740 1 client.15384 shutdown 2024-09-10T09:38:46.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:46.917+0000 7f1043dea740 1 -- 172.21.15.5:0/761920239 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55601b787b30 msgr2=0x55601b793d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:38:46.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:46.917+0000 7f1043dea740 1 --2- 172.21.15.5:0/761920239 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55601b787b30 0x55601b793d70 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f10240126e0 tx=0x7f1024012ba0 comp rx=0 tx=0).stop 2024-09-10T09:38:46.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:46.917+0000 7f1043dea740 1 -- 172.21.15.5:0/761920239 shutdown_connections 2024-09-10T09:38:46.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:46.917+0000 7f1043dea740 1 --2- 172.21.15.5:0/761920239 >> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] conn(0x55601b7a7520 0x55601b7a8990 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:46.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:46.917+0000 7f1043dea740 1 --2- 172.21.15.5:0/761920239 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55601b796920 0x55601b7942b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:46.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:46.917+0000 7f1043dea740 1 --2- 172.21.15.5:0/761920239 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55601b787b30 0x55601b793d70 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:46.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:46.917+0000 7f1043dea740 1 --2- 172.21.15.5:0/761920239 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55601b787210 0x55601b793830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:46.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:46.917+0000 7f101fe00640 0 client.15384 ms_handle_reset on v2:172.21.15.102:6836/2689776764 2024-09-10T09:38:46.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:46.917+0000 7f1043dea740 1 -- 172.21.15.5:0/761920239 >> 172.21.15.5:0/761920239 conn(0x55601b7838c0 msgr2=0x55601b789620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:38:46.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:46.917+0000 7f1043dea740 1 -- 172.21.15.5:0/761920239 shutdown_connections 2024-09-10T09:38:46.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:46.917+0000 7f1043dea740 1 -- 172.21.15.5:0/761920239 wait complete. 2024-09-10T09:38:46.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:46.917+0000 7f1043dea740 20 client.15384 trim_cache size 0 max 16384 2024-09-10T09:38:46.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:46.936+0000 7f1043dea740 1 Processor -- start 2024-09-10T09:38:46.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:46.937+0000 7f1043dea740 1 -- start start 2024-09-10T09:38:46.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:46.937+0000 7f1043dea740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55601b787210 0x55601b7a8990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:46.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:46.937+0000 7f1043dea740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55601b787b30 0x55601b782910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:46.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:46.937+0000 7f1043dea740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55601b796920 0x55601b794100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:46.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:46.937+0000 7f1043dea740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55601b7a6cd0 con 0x55601b796920 2024-09-10T09:38:46.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:46.937+0000 7f1043dea740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55601b7a73a0 con 0x55601b787b30 2024-09-10T09:38:46.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:46.937+0000 7f1043dea740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55601b7a7a70 con 0x55601b787210 2024-09-10T09:38:46.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:46.937+0000 7f103b200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55601b796920 0x55601b794100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:46.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:46.937+0000 7f103b200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55601b796920 0x55601b794100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58214/0 (socket says 172.21.15.5:58214) 2024-09-10T09:38:46.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:46.937+0000 7f103b200640 1 -- 172.21.15.5:0/1929482096 learned_addr learned my addr 172.21.15.5:0/1929482096 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:38:46.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:46.937+0000 7f1031e00640 1 --2- 172.21.15.5:0/1929482096 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55601b787b30 0x55601b782910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:46.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:46.937+0000 7f103a800640 1 --2- 172.21.15.5:0/1929482096 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55601b787210 0x55601b7a8990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:46.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:46.938+0000 7f103b200640 1 -- 172.21.15.5:0/1929482096 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55601b787210 msgr2=0x55601b7a8990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:38:46.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:46.938+0000 7f103b200640 1 --2- 172.21.15.5:0/1929482096 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55601b787210 0x55601b7a8990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:46.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:46.938+0000 7f103b200640 1 -- 172.21.15.5:0/1929482096 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55601b787b30 msgr2=0x55601b782910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:38:46.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:46.938+0000 7f103b200640 1 --2- 172.21.15.5:0/1929482096 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55601b787b30 0x55601b782910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:46.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:46.938+0000 7f103b200640 1 -- 172.21.15.5:0/1929482096 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55601b791d80 con 0x55601b796920 2024-09-10T09:38:46.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:46.938+0000 7f1031e00640 1 --2- 172.21.15.5:0/1929482096 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55601b787b30 0x55601b782910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:38:46.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:46.938+0000 7f103b200640 1 --2- 172.21.15.5:0/1929482096 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55601b796920 0x55601b794100 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f103400f9c0 tx=0x7f1034017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88bab74711c7798 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:38:46.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:46.938+0000 7f102aa00640 1 -- 172.21.15.5:0/1929482096 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f103401d4d0 con 0x55601b796920 2024-09-10T09:38:46.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:46.938+0000 7f102aa00640 1 -- 172.21.15.5:0/1929482096 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f103402e070 con 0x55601b796920 2024-09-10T09:38:46.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:46.938+0000 7f102aa00640 1 -- 172.21.15.5:0/1929482096 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1034029bd0 con 0x55601b796920 2024-09-10T09:38:46.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:46.939+0000 7f1043dea740 1 -- 172.21.15.5:0/1929482096 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55601b796920 msgr2=0x55601b794100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:38:46.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:46.939+0000 7f1043dea740 1 --2- 172.21.15.5:0/1929482096 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55601b796920 0x55601b794100 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f103400f9c0 tx=0x7f1034017000 comp rx=0 tx=0).stop 2024-09-10T09:38:46.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:46.939+0000 7f1043dea740 1 -- 172.21.15.5:0/1929482096 shutdown_connections 2024-09-10T09:38:46.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:46.939+0000 7f1043dea740 1 --2- 172.21.15.5:0/1929482096 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55601b796920 0x55601b794100 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:46.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:46.939+0000 7f1043dea740 1 --2- 172.21.15.5:0/1929482096 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55601b787b30 0x55601b782910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:46.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:46.939+0000 7f1043dea740 1 --2- 172.21.15.5:0/1929482096 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55601b787210 0x55601b7a8990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:46.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:46.939+0000 7f1043dea740 1 -- 172.21.15.5:0/1929482096 >> 172.21.15.5:0/1929482096 conn(0x55601b7838c0 msgr2=0x55601b782200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:38:46.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:46.939+0000 7f1043dea740 1 -- 172.21.15.5:0/1929482096 shutdown_connections 2024-09-10T09:38:46.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:46.939+0000 7f1043dea740 1 -- 172.21.15.5:0/1929482096 wait complete. 2024-09-10T09:38:46.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:46.940+0000 7f1043dea740 1 -- start start 2024-09-10T09:38:46.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:46.940+0000 7f1043dea740 1 Processor -- start 2024-09-10T09:38:46.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:46.940+0000 7f1043dea740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55601b787210 0x55601b7aa960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:46.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:46.940+0000 7f1043dea740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55601b787b30 0x55601b7aaea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:46.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:46.940+0000 7f1043dea740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55601b796920 0x55601b7ab3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:46.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:46.940+0000 7f1043dea740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55601b7a8810 con 0x55601b787b30 2024-09-10T09:38:46.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:46.940+0000 7f1043dea740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55601b7a8140 con 0x55601b787210 2024-09-10T09:38:46.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:46.940+0000 7f1043dea740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55601b7a8ee0 con 0x55601b796920 2024-09-10T09:38:46.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:46.940+0000 7f1043dea740 10 client.0 _ll_register_callbacks cb 0x7f1024009ea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:38:46.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:46.941+0000 7f1031e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55601b787b30 0x55601b7aaea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:46.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:46.941+0000 7f1031e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55601b787b30 0x55601b7aaea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58230/0 (socket says 172.21.15.5:58230) 2024-09-10T09:38:46.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:46.941+0000 7f1031e00640 1 -- 172.21.15.5:0/1038411865 learned_addr learned my addr 172.21.15.5:0/1038411865 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:38:46.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:46.941+0000 7f103a800640 1 --2- 172.21.15.5:0/1038411865 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55601b787210 0x55601b7aa960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:46.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:46.941+0000 7f103b200640 1 --2- 172.21.15.5:0/1038411865 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55601b796920 0x55601b7ab3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:46.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:46.941+0000 7f1031e00640 1 -- 172.21.15.5:0/1038411865 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55601b796920 msgr2=0x55601b7ab3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:38:46.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:46.941+0000 7f1031e00640 1 --2- 172.21.15.5:0/1038411865 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55601b796920 0x55601b7ab3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:46.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:46.941+0000 7f1031e00640 1 -- 172.21.15.5:0/1038411865 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55601b787210 msgr2=0x55601b7aa960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:38:46.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:46.941+0000 7f1031e00640 1 --2- 172.21.15.5:0/1038411865 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55601b787210 0x55601b7aa960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:46.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:46.941+0000 7f1031e00640 1 -- 172.21.15.5:0/1038411865 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55601b7af100 con 0x55601b787b30 2024-09-10T09:38:46.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:46.941+0000 7f103a800640 1 --2- 172.21.15.5:0/1038411865 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55601b787210 0x55601b7aa960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:38:46.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:46.942+0000 7f1031e00640 1 --2- 172.21.15.5:0/1038411865 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55601b787b30 0x55601b7aaea0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f10240126b0 tx=0x7f10240123c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d694f6e6417e7db server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:38:46.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:46.942+0000 7f101fe00640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-10T09:38:46.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:46.942+0000 7f101fe00640 1 -- 172.21.15.5:0/1038411865 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f102402f060 con 0x55601b787b30 2024-09-10T09:38:46.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:46.942+0000 7f1043dea740 1 -- 172.21.15.5:0/1038411865 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f103402e070 con 0x55601b787b30 2024-09-10T09:38:46.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:46.942+0000 7f101fe00640 1 -- 172.21.15.5:0/1038411865 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f102401c420 con 0x55601b787b30 2024-09-10T09:38:46.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:46.942+0000 7f101fe00640 1 -- 172.21.15.5:0/1038411865 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f102401cd10 con 0x55601b787b30 2024-09-10T09:38:46.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:46.942+0000 7f1043dea740 10 client.15390 fetch_fsmap learned FSMap version 30 2024-09-10T09:38:46.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:46.942+0000 7f1043dea740 1 -- 172.21.15.5:0/1038411865 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55601b811f70 con 0x55601b787b30 2024-09-10T09:38:46.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:46.942+0000 7f101fe00640 1 -- 172.21.15.5:0/1038411865 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f103402e070 con 0x55601b787b30 2024-09-10T09:38:46.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:46.943+0000 7f101fe00640 1 -- 172.21.15.5:0/1038411865 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f10240038f0 con 0x55601b787b30 2024-09-10T09:38:46.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:46.943+0000 7f1043dea740 10 client.15390 fetch_fsmap finished waiting for FSMap version 30 2024-09-10T09:38:46.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:46.943+0000 7f1043dea740 10 client.15390 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-10T09:38:46.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:46.943+0000 7f1043dea740 20 client.15390 populate_metadata read hostname 'smithi005' 2024-09-10T09:38:46.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:46.943+0000 7f1043dea740 1 --2- 172.21.15.5:0/1038411865 >> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] conn(0x55601b816ba0 0x55601b8158b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:46.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:46.943+0000 7f1043dea740 4 client.15390 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T09:38:46.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:46.943+0000 7f1043dea740 1 -- 172.21.15.5:0/1038411865 --> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] -- command(tid 0: {"prefix": "status"}) -- 0x55601b6f4b30 con 0x55601b816ba0 2024-09-10T09:38:46.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:46.943+0000 7f103a800640 1 --2- 172.21.15.5:0/1038411865 >> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] conn(0x55601b816ba0 0x55601b8158b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:46.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:46.944+0000 7f103a800640 1 --2- 172.21.15.5:0/1038411865 >> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] conn(0x55601b816ba0 0x55601b8158b0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=d53e1ab76ba30ac4 server_cookie=64cf6638c544257b in_seq=0 out_seq=0 2024-09-10T09:38:46.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:46.944+0000 7f101fe00640 10 client.15390 ms_handle_connect on v2:172.21.15.102:6836/2689776764 2024-09-10T09:38:46.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:46.944+0000 7f101fe00640 1 -- 172.21.15.5:0/1038411865 <== mds.3 v2:172.21.15.102:6836/2689776764 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55601b6f4b30 con 0x55601b816ba0 2024-09-10T09:38:46.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:46.944+0000 7f101fe00640 10 client.15390 handle_command_reply: tid=0 2024-09-10T09:38:46.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:46.944+0000 7f1043dea740 1 client.15390 shutdown 2024-09-10T09:38:46.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:46.944+0000 7f1043dea740 1 -- 172.21.15.5:0/1038411865 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55601b787b30 msgr2=0x55601b7aaea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:38:46.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:46.944+0000 7f1043dea740 1 --2- 172.21.15.5:0/1038411865 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55601b787b30 0x55601b7aaea0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f10240126b0 tx=0x7f10240123c0 comp rx=0 tx=0).stop 2024-09-10T09:38:46.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:46.944+0000 7f1043dea740 1 -- 172.21.15.5:0/1038411865 shutdown_connections 2024-09-10T09:38:46.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:46.944+0000 7f1043dea740 1 --2- 172.21.15.5:0/1038411865 >> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] conn(0x55601b816ba0 0x55601b8158b0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:46.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:46.944+0000 7f1043dea740 1 --2- 172.21.15.5:0/1038411865 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55601b796920 0x55601b7ab3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:46.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:46.944+0000 7f1043dea740 1 --2- 172.21.15.5:0/1038411865 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55601b787b30 0x55601b7aaea0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:46.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:46.944+0000 7f101fe00640 0 client.15390 ms_handle_reset on v2:172.21.15.102:6836/2689776764 2024-09-10T09:38:46.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:46.944+0000 7f1043dea740 1 --2- 172.21.15.5:0/1038411865 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55601b787210 0x55601b7aa960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:46.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:46.944+0000 7f1043dea740 1 -- 172.21.15.5:0/1038411865 >> 172.21.15.5:0/1038411865 conn(0x55601b7838c0 msgr2=0x55601b7a1d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:38:46.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:46.945+0000 7f1043dea740 1 -- 172.21.15.5:0/1038411865 shutdown_connections 2024-09-10T09:38:46.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:46.945+0000 7f1043dea740 1 -- 172.21.15.5:0/1038411865 wait complete. 2024-09-10T09:38:46.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:46.945+0000 7f1043dea740 20 client.15390 trim_cache size 0 max 16384 2024-09-10T09:38:46.956 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-10T09:38:46.956 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_fsid": "bade4b50-6f56-11ef-bcea-c7b262605968", 2024-09-10T09:38:46.956 INFO:teuthology.orchestra.run.smithi005.stdout: "whoami": 3, 2024-09-10T09:38:46.956 INFO:teuthology.orchestra.run.smithi005.stdout: "id": 24506, 2024-09-10T09:38:46.956 INFO:teuthology.orchestra.run.smithi005.stdout: "want_state": "up:active", 2024-09-10T09:38:46.956 INFO:teuthology.orchestra.run.smithi005.stdout: "state": "up:active", 2024-09-10T09:38:46.956 INFO:teuthology.orchestra.run.smithi005.stdout: "fs_name": "cephfs", 2024-09-10T09:38:46.956 INFO:teuthology.orchestra.run.smithi005.stdout: "rank_uptime": 153.13695999999999, 2024-09-10T09:38:46.957 INFO:teuthology.orchestra.run.smithi005.stdout: "mdsmap_epoch": 30, 2024-09-10T09:38:46.957 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch": 96, 2024-09-10T09:38:46.957 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch_barrier": 96, 2024-09-10T09:38:46.957 INFO:teuthology.orchestra.run.smithi005.stdout: "uptime": 222.860076163 2024-09-10T09:38:46.957 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-10T09:38:46.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:46.945+0000 7f1041400640 1 -- 172.21.15.5:0/894401824 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f10200a2c10 msgr2=0x7f10200a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:38:46.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:46.945+0000 7f1041400640 1 --2- 172.21.15.5:0/894401824 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f10200a2c10 0x7f10200a50d0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f1034005fd0 tx=0x7f1034009500 comp rx=0 tx=0).stop 2024-09-10T09:38:46.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:46.945+0000 7f1041400640 1 -- 172.21.15.5:0/894401824 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f103c101de0 msgr2=0x7f103c198450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:38:46.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:46.945+0000 7f1041400640 1 --2- 172.21.15.5:0/894401824 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f103c101de0 0x7f103c198450 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f102c006010 tx=0x7f102c004040 comp rx=0 tx=0).stop 2024-09-10T09:38:46.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:46.945+0000 7f1041400640 1 -- 172.21.15.5:0/894401824 shutdown_connections 2024-09-10T09:38:46.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:46.945+0000 7f1041400640 1 --2- 172.21.15.5:0/894401824 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f10200a2c10 0x7f10200a50d0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:46.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:46.945+0000 7f1041400640 1 --2- 172.21.15.5:0/894401824 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f103c106f10 0x7f103c19cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:46.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:46.945+0000 7f1041400640 1 --2- 172.21.15.5:0/894401824 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f103c102790 0x7f103c198990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:46.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:46.945+0000 7f1041400640 1 --2- 172.21.15.5:0/894401824 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f103c101de0 0x7f103c198450 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:46.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:46.945+0000 7f1041400640 1 -- 172.21.15.5:0/894401824 >> 172.21.15.5:0/894401824 conn(0x7f103c0fd8c0 msgr2=0x7f103c0feca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:38:46.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:46.946+0000 7f1041400640 1 -- 172.21.15.5:0/894401824 shutdown_connections 2024-09-10T09:38:46.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:46.946+0000 7f1041400640 1 -- 172.21.15.5:0/894401824 wait complete. 2024-09-10T09:38:47.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:38:47 smithi102 ceph-mon[35442]: pgmap v406: 129 pgs: 129 active+clean; 660 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 853 B/s wr, 12 op/s 2024-09-10T09:38:47.460 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:38:47 smithi005 ceph-mon[30350]: pgmap v406: 129 pgs: 129 active+clean; 660 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 853 B/s wr, 12 op/s 2024-09-10T09:38:47.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:38:47 smithi159 ceph-mon[35513]: pgmap v406: 129 pgs: 129 active+clean; 660 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 853 B/s wr, 12 op/s 2024-09-10T09:38:47.872 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph tell mds.c status 2024-09-10T09:38:48.247 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:38:48.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:48.812+0000 7fa9fd000640 1 -- 172.21.15.5:0/3797049903 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa9f8112a30 msgr2=0x7fa9f8114e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:38:48.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:48.812+0000 7fa9fd000640 1 --2- 172.21.15.5:0/3797049903 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa9f8112a30 0x7fa9f8114e20 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fa9f0009ca0 tx=0x7fa9f002a8c0 comp rx=0 tx=0).stop 2024-09-10T09:38:48.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:48.812+0000 7fa9fd000640 1 -- 172.21.15.5:0/3797049903 shutdown_connections 2024-09-10T09:38:48.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:48.812+0000 7fa9fd000640 1 --2- 172.21.15.5:0/3797049903 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa9f8112a30 0x7fa9f8114e20 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:48.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:48.812+0000 7fa9fd000640 1 --2- 172.21.15.5:0/3797049903 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa9f8077040 0x7fa9f81124f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:48.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:48.812+0000 7fa9fd000640 1 --2- 172.21.15.5:0/3797049903 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa9f8076720 0x7fa9f8076b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:48.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:48.812+0000 7fa9fd000640 1 -- 172.21.15.5:0/3797049903 >> 172.21.15.5:0/3797049903 conn(0x7fa9f80ffa90 msgr2=0x7fa9f8101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:38:48.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:48.812+0000 7fa9fd000640 1 -- 172.21.15.5:0/3797049903 shutdown_connections 2024-09-10T09:38:48.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:48.812+0000 7fa9fd000640 1 -- 172.21.15.5:0/3797049903 wait complete. 2024-09-10T09:38:48.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:48.813+0000 7fa9fd000640 1 Processor -- start 2024-09-10T09:38:48.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:48.814+0000 7fa9fd000640 1 -- start start 2024-09-10T09:38:48.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:48.814+0000 7fa9fd000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa9f8076720 0x7fa9f81a46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:48.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:48.814+0000 7fa9fd000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa9f8077040 0x7fa9f81a4be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:48.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:48.814+0000 7fa9fd000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa9f8112a30 0x7fa9f819e860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:48.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:48.814+0000 7fa9fd000640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa9f81185c0 con 0x7fa9f8112a30 2024-09-10T09:38:48.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:48.814+0000 7fa9fd000640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fa9f8118440 con 0x7fa9f8077040 2024-09-10T09:38:48.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:48.814+0000 7fa9fd000640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fa9f8118740 con 0x7fa9f8076720 2024-09-10T09:38:48.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:48.815+0000 7fa9f6800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa9f8112a30 0x7fa9f819e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:48.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:48.815+0000 7fa9f6800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa9f8112a30 0x7fa9f819e860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58244/0 (socket says 172.21.15.5:58244) 2024-09-10T09:38:48.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:48.815+0000 7fa9f6800640 1 -- 172.21.15.5:0/648737825 learned_addr learned my addr 172.21.15.5:0/648737825 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:38:48.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:48.815+0000 7fa9f5e00640 1 --2- 172.21.15.5:0/648737825 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa9f8076720 0x7fa9f81a46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:48.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:48.815+0000 7fa9f5400640 1 --2- 172.21.15.5:0/648737825 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa9f8077040 0x7fa9f81a4be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:48.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:48.815+0000 7fa9f5e00640 1 -- 172.21.15.5:0/648737825 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa9f8077040 msgr2=0x7fa9f81a4be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:38:48.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:48.815+0000 7fa9f5e00640 1 --2- 172.21.15.5:0/648737825 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa9f8077040 0x7fa9f81a4be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:48.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:48.815+0000 7fa9f5e00640 1 -- 172.21.15.5:0/648737825 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa9f8112a30 msgr2=0x7fa9f819e860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:38:48.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:48.815+0000 7fa9f5e00640 1 --2- 172.21.15.5:0/648737825 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa9f8112a30 0x7fa9f819e860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:48.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:48.815+0000 7fa9f5e00640 1 -- 172.21.15.5:0/648737825 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9f819ef90 con 0x7fa9f8076720 2024-09-10T09:38:48.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:48.815+0000 7fa9f6800640 1 --2- 172.21.15.5:0/648737825 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa9f8112a30 0x7fa9f819e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:38:48.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:48.815+0000 7fa9f5e00640 1 --2- 172.21.15.5:0/648737825 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa9f8076720 0x7fa9f81a46a0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fa9e800d6d0 tx=0x7fa9e800db90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:38:48.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:48.816+0000 7fa9eea00640 1 -- 172.21.15.5:0/648737825 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9e8017070 con 0x7fa9f8076720 2024-09-10T09:38:48.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:48.816+0000 7fa9fd000640 1 -- 172.21.15.5:0/648737825 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9f819f280 con 0x7fa9f8076720 2024-09-10T09:38:48.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:48.816+0000 7fa9eea00640 1 -- 172.21.15.5:0/648737825 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa9e8009820 con 0x7fa9f8076720 2024-09-10T09:38:48.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:48.816+0000 7fa9eea00640 1 -- 172.21.15.5:0/648737825 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9e8004cd0 con 0x7fa9f8076720 2024-09-10T09:38:48.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:48.816+0000 7fa9fd000640 1 -- 172.21.15.5:0/648737825 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9f81ab570 con 0x7fa9f8076720 2024-09-10T09:38:48.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:48.818+0000 7fa9ffa8c740 1 Processor -- start 2024-09-10T09:38:48.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:48.819+0000 7fa9ffa8c740 1 -- start start 2024-09-10T09:38:48.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:48.819+0000 7fa9ffa8c740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561399c12330 0x561399c12710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:48.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:48.819+0000 7fa9ffa8c740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561399c12c50 0x561399c1cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:48.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:48.819+0000 7fa9ffa8c740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561399c1d510 0x561399c1f900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:48.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:48.819+0000 7fa9ffa8c740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x561399b780b0 con 0x561399c12c50 2024-09-10T09:38:48.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:48.819+0000 7fa9ffa8c740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x561399b1fab0 con 0x561399c1d510 2024-09-10T09:38:48.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:48.819+0000 7fa9ffa8c740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x561399b74070 con 0x561399c12330 2024-09-10T09:38:48.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:48.819+0000 7fa9eea00640 1 -- 172.21.15.5:0/648737825 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 19) ==== 137883+0+0 (secure 0 0 0) 0x7fa9e80099d0 con 0x7fa9f8076720 2024-09-10T09:38:48.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:48.819+0000 7fa9f6800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561399c12330 0x561399c12710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:48.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:48.819+0000 7fa9f6800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561399c12330 0x561399c12710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:33154/0 (socket says 172.21.15.5:33154) 2024-09-10T09:38:48.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:48.819+0000 7fa9f6800640 1 -- 172.21.15.5:0/1163825696 learned_addr learned my addr 172.21.15.5:0/1163825696 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:38:48.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:48.819+0000 7fa9f5e00640 1 --2- 172.21.15.5:0/1163825696 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561399c12c50 0x561399c1cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:48.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:48.820+0000 7fa9f5400640 1 --2- 172.21.15.5:0/1163825696 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561399c1d510 0x561399c1f900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:48.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:48.820+0000 7fa9f6800640 1 -- 172.21.15.5:0/1163825696 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561399c1d510 msgr2=0x561399c1f900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:38:48.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:48.820+0000 7fa9f6800640 1 --2- 172.21.15.5:0/1163825696 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561399c1d510 0x561399c1f900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:48.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:48.820+0000 7fa9f6800640 1 -- 172.21.15.5:0/1163825696 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561399c12c50 msgr2=0x561399c1cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:38:48.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:48.820+0000 7fa9f6800640 1 --2- 172.21.15.5:0/1163825696 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561399c12c50 0x561399c1cfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:48.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:48.820+0000 7fa9f6800640 1 -- 172.21.15.5:0/1163825696 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561399b2c4a0 con 0x561399c12330 2024-09-10T09:38:48.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:48.820+0000 7fa9f5e00640 1 --2- 172.21.15.5:0/1163825696 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561399c12c50 0x561399c1cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:38:48.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:48.820+0000 7fa9f6800640 1 --2- 172.21.15.5:0/1163825696 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561399c12330 0x561399c12710 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fa9f002aeb0 tx=0x7fa9f002f3c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2472b063396a4e4d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:38:48.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:48.820+0000 7fa9e3400640 1 -- 172.21.15.5:0/1163825696 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9f0003bf0 con 0x561399c12330 2024-09-10T09:38:48.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:48.821+0000 7fa9e3400640 1 -- 172.21.15.5:0/1163825696 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa9f0036040 con 0x561399c12330 2024-09-10T09:38:48.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:48.821+0000 7fa9e3400640 1 -- 172.21.15.5:0/1163825696 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9f00046e0 con 0x561399c12330 2024-09-10T09:38:48.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:48.821+0000 7fa9eea00640 1 --2- 172.21.15.5:0/648737825 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fa9dc0a2c90 0x7fa9dc0a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:48.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:48.821+0000 7fa9eea00640 1 -- 172.21.15.5:0/648737825 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fa9e80cf0c0 con 0x7fa9f8076720 2024-09-10T09:38:48.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:48.821+0000 7fa9f5e00640 1 --2- 172.21.15.5:0/648737825 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fa9dc0a2c90 0x7fa9dc0a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:48.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:48.821+0000 7fa9ffa8c740 1 -- 172.21.15.5:0/1163825696 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561399c12330 msgr2=0x561399c12710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:38:48.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:48.821+0000 7fa9ffa8c740 1 --2- 172.21.15.5:0/1163825696 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561399c12330 0x561399c12710 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fa9f002aeb0 tx=0x7fa9f002f3c0 comp rx=0 tx=0).stop 2024-09-10T09:38:48.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:48.822+0000 7fa9ffa8c740 1 -- 172.21.15.5:0/1163825696 shutdown_connections 2024-09-10T09:38:48.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:48.822+0000 7fa9ffa8c740 1 --2- 172.21.15.5:0/1163825696 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561399c1d510 0x561399c1f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:48.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:48.822+0000 7fa9ffa8c740 1 --2- 172.21.15.5:0/1163825696 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561399c12c50 0x561399c1cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:48.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:48.822+0000 7fa9ffa8c740 1 --2- 172.21.15.5:0/1163825696 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561399c12330 0x561399c12710 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:48.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:48.822+0000 7fa9ffa8c740 1 -- 172.21.15.5:0/1163825696 >> 172.21.15.5:0/1163825696 conn(0x561399c0e8c0 msgr2=0x561399c0eca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:38:48.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:48.822+0000 7fa9ffa8c740 1 -- 172.21.15.5:0/1163825696 shutdown_connections 2024-09-10T09:38:48.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:48.822+0000 7fa9f5e00640 1 --2- 172.21.15.5:0/648737825 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fa9dc0a2c90 0x7fa9dc0a5150 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fa9e8097df0 tx=0x7fa9e8097040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:38:48.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:48.822+0000 7fa9ffa8c740 1 -- 172.21.15.5:0/1163825696 wait complete. 2024-09-10T09:38:48.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:48.822+0000 7fa9ffa8c740 1 -- start start 2024-09-10T09:38:48.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:48.822+0000 7fa9ffa8c740 1 Processor -- start 2024-09-10T09:38:48.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:48.823+0000 7fa9ffa8c740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561399c12330 0x561399c1a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:48.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:48.823+0000 7fa9ffa8c740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561399c12c50 0x561399c1a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:48.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:48.823+0000 7fa9ffa8c740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561399c1d510 0x561399c1af20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:48.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:48.823+0000 7fa9ffa8c740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x561399c20000 con 0x561399c1d510 2024-09-10T09:38:48.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:48.823+0000 7fa9ffa8c740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x561399b00920 con 0x561399c12c50 2024-09-10T09:38:48.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:48.823+0000 7fa9ffa8c740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x561399c20180 con 0x561399c12330 2024-09-10T09:38:48.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:48.823+0000 7fa9ffa8c740 10 client.0 _ll_register_callbacks cb 0x561399b248a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:38:48.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:48.823+0000 7fa9f6800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561399c12330 0x561399c1a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:48.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:48.823+0000 7fa9f6800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561399c1d510 0x561399c1af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:48.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:48.823+0000 7fa9f6800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561399c12330 0x561399c1a4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:33164/0 (socket says 172.21.15.5:33164) 2024-09-10T09:38:48.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:48.823+0000 7fa9f6800640 1 -- 172.21.15.5:0/4211732982 learned_addr learned my addr 172.21.15.5:0/4211732982 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:38:48.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:48.823+0000 7fa9f5400640 1 --2- 172.21.15.5:0/4211732982 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561399c12c50 0x561399c1a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:48.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:48.824+0000 7fa9f6800640 1 -- 172.21.15.5:0/4211732982 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561399c12c50 msgr2=0x561399c1a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:38:48.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:48.824+0000 7fa9f6800640 1 --2- 172.21.15.5:0/4211732982 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561399c12c50 0x561399c1a9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:48.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:48.824+0000 7fa9f6800640 1 -- 172.21.15.5:0/4211732982 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561399c1d510 msgr2=0x561399c1af20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:38:48.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:48.824+0000 7fa9f6800640 1 --2- 172.21.15.5:0/4211732982 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561399c1d510 0x561399c1af20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:48.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:48.824+0000 7fa9f6800640 1 -- 172.21.15.5:0/4211732982 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5613998e92b0 con 0x561399c12330 2024-09-10T09:38:48.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:48.824+0000 7fa9f6800640 1 --2- 172.21.15.5:0/4211732982 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561399c12330 0x561399c1a4a0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x561399c1d4e0 tx=0x7fa9f00043a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f814d3d7de70950b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:38:48.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:48.824+0000 7fa9d7e00640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-09-10T09:38:48.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:48.824+0000 7fa9d7e00640 1 -- 172.21.15.5:0/4211732982 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9f00046d0 con 0x561399c12330 2024-09-10T09:38:48.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:48.825+0000 7fa9d7e00640 1 -- 172.21.15.5:0/4211732982 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fa9f002f610 con 0x561399c12330 2024-09-10T09:38:48.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:48.825+0000 7fa9ffa8c740 1 -- 172.21.15.5:0/4211732982 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa9f0036040 con 0x561399c12330 2024-09-10T09:38:48.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:48.825+0000 7fa9d7e00640 1 -- 172.21.15.5:0/4211732982 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fa9f0004b40 con 0x561399c12330 2024-09-10T09:38:48.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:48.825+0000 7fa9ffa8c740 10 client.25435 fetch_fsmap learned FSMap version 30 2024-09-10T09:38:48.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:48.825+0000 7fa9ffa8c740 1 -- 172.21.15.5:0/4211732982 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x561399c20000 con 0x561399c12330 2024-09-10T09:38:48.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:48.825+0000 7fa9d7e00640 1 -- 172.21.15.5:0/4211732982 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa9f0036040 con 0x561399c12330 2024-09-10T09:38:48.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:48.825+0000 7fa9d7e00640 1 -- 172.21.15.5:0/4211732982 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fa9f0044070 con 0x561399c12330 2024-09-10T09:38:48.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:48.825+0000 7fa9ffa8c740 10 client.25435 fetch_fsmap finished waiting for FSMap version 30 2024-09-10T09:38:48.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:48.826+0000 7fa9ffa8c740 10 client.25435 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-10T09:38:48.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:48.826+0000 7fa9ffa8c740 20 client.25435 populate_metadata read hostname 'smithi005' 2024-09-10T09:38:48.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:48.826+0000 7fa9ffa8c740 1 --2- 172.21.15.5:0/4211732982 >> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] conn(0x561399c2df40 0x561399c2f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:48.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:48.826+0000 7fa9ffa8c740 4 client.25435 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:38:48.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:48.826+0000 7fa9ffa8c740 1 -- 172.21.15.5:0/4211732982 --> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561399b85620 con 0x561399c2df40 2024-09-10T09:38:48.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:48.826+0000 7fa9f5400640 1 --2- 172.21.15.5:0/4211732982 >> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] conn(0x561399c2df40 0x561399c2f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:48.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:48.826+0000 7fa9f5400640 1 --2- 172.21.15.5:0/4211732982 >> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] conn(0x561399c2df40 0x561399c2f3b0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1d7c457a1d7a24cf server_cookie=7357c2403fc6e6b4 in_seq=0 out_seq=0 2024-09-10T09:38:48.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:48.827+0000 7fa9d7e00640 10 client.25435 ms_handle_connect on v2:172.21.15.159:6836/620627639 2024-09-10T09:38:48.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:48.827+0000 7fa9d7e00640 1 -- 172.21.15.5:0/4211732982 <== mds.2 v2:172.21.15.159:6836/620627639 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561399b85620 con 0x561399c2df40 2024-09-10T09:38:48.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:48.827+0000 7fa9d7e00640 10 client.25435 handle_command_reply: tid=0 2024-09-10T09:38:48.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:48.827+0000 7fa9ffa8c740 1 client.25435 shutdown 2024-09-10T09:38:48.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:48.828+0000 7fa9ffa8c740 1 -- 172.21.15.5:0/4211732982 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561399c12330 msgr2=0x561399c1a4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:38:48.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:48.828+0000 7fa9ffa8c740 1 --2- 172.21.15.5:0/4211732982 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561399c12330 0x561399c1a4a0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x561399c1d4e0 tx=0x7fa9f00043a0 comp rx=0 tx=0).stop 2024-09-10T09:38:48.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:48.828+0000 7fa9ffa8c740 1 -- 172.21.15.5:0/4211732982 shutdown_connections 2024-09-10T09:38:48.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:48.828+0000 7fa9ffa8c740 1 --2- 172.21.15.5:0/4211732982 >> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] conn(0x561399c2df40 0x561399c2f3b0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:48.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:48.828+0000 7fa9ffa8c740 1 --2- 172.21.15.5:0/4211732982 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561399c1d510 0x561399c1af20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:48.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:48.828+0000 7fa9ffa8c740 1 --2- 172.21.15.5:0/4211732982 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561399c12c50 0x561399c1a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:48.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:48.828+0000 7fa9ffa8c740 1 --2- 172.21.15.5:0/4211732982 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561399c12330 0x561399c1a4a0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:48.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:48.828+0000 7fa9d7e00640 0 client.25435 ms_handle_reset on v2:172.21.15.159:6836/620627639 2024-09-10T09:38:48.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:48.828+0000 7fa9ffa8c740 1 -- 172.21.15.5:0/4211732982 >> 172.21.15.5:0/4211732982 conn(0x561399c0e8c0 msgr2=0x561399c145f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:38:48.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:48.828+0000 7fa9ffa8c740 1 -- 172.21.15.5:0/4211732982 shutdown_connections 2024-09-10T09:38:48.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:48.828+0000 7fa9ffa8c740 1 -- 172.21.15.5:0/4211732982 wait complete. 2024-09-10T09:38:48.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:48.828+0000 7fa9ffa8c740 20 client.25435 trim_cache size 0 max 16384 2024-09-10T09:38:48.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:48.847+0000 7fa9ffa8c740 1 Processor -- start 2024-09-10T09:38:48.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:48.847+0000 7fa9ffa8c740 1 -- start start 2024-09-10T09:38:48.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:48.848+0000 7fa9ffa8c740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561399c12330 0x561399c18c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:48.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:48.848+0000 7fa9ffa8c740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561399c12c50 0x561399c19180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:48.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:48.848+0000 7fa9ffa8c740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561399c1d510 0x561399c196c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:48.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:48.848+0000 7fa9ffa8c740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x561399c2d6f0 con 0x561399c1d510 2024-09-10T09:38:48.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:48.848+0000 7fa9ffa8c740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x561399c2ddc0 con 0x561399c12c50 2024-09-10T09:38:48.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:48.848+0000 7fa9ffa8c740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x561399c2e490 con 0x561399c12330 2024-09-10T09:38:48.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:48.848+0000 7fa9f6800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561399c1d510 0x561399c196c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:48.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:48.848+0000 7fa9f6800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561399c1d510 0x561399c196c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58264/0 (socket says 172.21.15.5:58264) 2024-09-10T09:38:48.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:48.848+0000 7fa9f6800640 1 -- 172.21.15.5:0/1661879465 learned_addr learned my addr 172.21.15.5:0/1661879465 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:38:48.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:48.848+0000 7fa9f5400640 1 --2- 172.21.15.5:0/1661879465 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561399c12c50 0x561399c19180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:48.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:48.848+0000 7fa9f6800640 1 --2- 172.21.15.5:0/1661879465 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561399c12330 0x561399c18c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:48.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:48.848+0000 7fa9f5400640 1 -- 172.21.15.5:0/1661879465 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561399c12330 msgr2=0x561399c18c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:38:48.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:48.848+0000 7fa9f5400640 1 --2- 172.21.15.5:0/1661879465 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561399c12330 0x561399c18c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:48.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:48.848+0000 7fa9f5400640 1 -- 172.21.15.5:0/1661879465 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561399c1d510 msgr2=0x561399c196c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:38:48.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:48.848+0000 7fa9f5400640 1 --2- 172.21.15.5:0/1661879465 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561399c1d510 0x561399c196c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:48.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:48.848+0000 7fa9f5400640 1 -- 172.21.15.5:0/1661879465 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561399c12160 con 0x561399c12c50 2024-09-10T09:38:48.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:48.848+0000 7fa9f6800640 1 --2- 172.21.15.5:0/1661879465 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561399c1d510 0x561399c196c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:38:48.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:48.849+0000 7fa9f5400640 1 --2- 172.21.15.5:0/1661879465 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561399c12c50 0x561399c19180 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x561399c0fc40 tx=0x7fa9e4013830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7797c9803dfec390 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:38:48.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:48.849+0000 7fa9e2a00640 1 -- 172.21.15.5:0/1661879465 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9e4005020 con 0x561399c12c50 2024-09-10T09:38:48.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:48.849+0000 7fa9e2a00640 1 -- 172.21.15.5:0/1661879465 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa9e40047a0 con 0x561399c12c50 2024-09-10T09:38:48.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:48.849+0000 7fa9e2a00640 1 -- 172.21.15.5:0/1661879465 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9e40116a0 con 0x561399c12c50 2024-09-10T09:38:48.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:48.850+0000 7fa9ffa8c740 1 -- 172.21.15.5:0/1661879465 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561399c12c50 msgr2=0x561399c19180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:38:48.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:48.850+0000 7fa9ffa8c740 1 --2- 172.21.15.5:0/1661879465 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561399c12c50 0x561399c19180 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x561399c0fc40 tx=0x7fa9e4013830 comp rx=0 tx=0).stop 2024-09-10T09:38:48.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:48.850+0000 7fa9ffa8c740 1 -- 172.21.15.5:0/1661879465 shutdown_connections 2024-09-10T09:38:48.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:48.850+0000 7fa9ffa8c740 1 --2- 172.21.15.5:0/1661879465 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561399c1d510 0x561399c196c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:48.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:48.850+0000 7fa9ffa8c740 1 --2- 172.21.15.5:0/1661879465 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561399c12c50 0x561399c19180 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:48.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:48.850+0000 7fa9ffa8c740 1 --2- 172.21.15.5:0/1661879465 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561399c12330 0x561399c18c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:48.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:48.850+0000 7fa9ffa8c740 1 -- 172.21.15.5:0/1661879465 >> 172.21.15.5:0/1661879465 conn(0x561399c0e8c0 msgr2=0x561399c0eca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:38:48.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:48.850+0000 7fa9ffa8c740 1 -- 172.21.15.5:0/1661879465 shutdown_connections 2024-09-10T09:38:48.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:48.850+0000 7fa9ffa8c740 1 -- 172.21.15.5:0/1661879465 wait complete. 2024-09-10T09:38:48.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:48.850+0000 7fa9ffa8c740 1 -- start start 2024-09-10T09:38:48.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:48.851+0000 7fa9ffa8c740 1 Processor -- start 2024-09-10T09:38:48.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:48.851+0000 7fa9ffa8c740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561399c12330 0x561399c37f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:48.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:48.851+0000 7fa9ffa8c740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561399c12c50 0x561399c17480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:48.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:48.851+0000 7fa9ffa8c740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561399c1d510 0x561399c17bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:48.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:48.851+0000 7fa9ffa8c740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x561399c2f230 con 0x561399c12c50 2024-09-10T09:38:48.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:48.851+0000 7fa9ffa8c740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x561399c2eb60 con 0x561399c12330 2024-09-10T09:38:48.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:48.851+0000 7fa9ffa8c740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x561399c2f900 con 0x561399c1d510 2024-09-10T09:38:48.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:48.851+0000 7fa9ffa8c740 10 client.0 _ll_register_callbacks cb 0x7fa9f0003dd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:38:48.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:48.851+0000 7fa9f5400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561399c12c50 0x561399c17480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:48.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:48.851+0000 7fa9f5400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561399c12c50 0x561399c17480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58276/0 (socket says 172.21.15.5:58276) 2024-09-10T09:38:48.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:48.852+0000 7fa9f5400640 1 -- 172.21.15.5:0/2719905325 learned_addr learned my addr 172.21.15.5:0/2719905325 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:38:48.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:48.852+0000 7fa9f6800640 1 --2- 172.21.15.5:0/2719905325 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561399c12330 0x561399c37f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:48.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:48.852+0000 7fa9f6800640 1 --2- 172.21.15.5:0/2719905325 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561399c1d510 0x561399c17bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:48.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:48.852+0000 7fa9f5400640 1 -- 172.21.15.5:0/2719905325 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561399c1d510 msgr2=0x561399c17bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:38:48.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:48.852+0000 7fa9f5400640 1 --2- 172.21.15.5:0/2719905325 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561399c1d510 0x561399c17bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:48.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:48.852+0000 7fa9f5400640 1 -- 172.21.15.5:0/2719905325 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561399c12330 msgr2=0x561399c37f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:38:48.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:48.852+0000 7fa9f5400640 1 --2- 172.21.15.5:0/2719905325 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561399c12330 0x561399c37f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:48.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:48.852+0000 7fa9f5400640 1 -- 172.21.15.5:0/2719905325 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561399c35a30 con 0x561399c12c50 2024-09-10T09:38:48.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:48.852+0000 7fa9f5400640 1 --2- 172.21.15.5:0/2719905325 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561399c12c50 0x561399c17480 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fa9e400bd40 tx=0x7fa9e40046f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=977097eee44d9ef8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:38:48.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:48.852+0000 7fa9d7e00640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-10T09:38:48.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:48.853+0000 7fa9d7e00640 1 -- 172.21.15.5:0/2719905325 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9e402b070 con 0x561399c12c50 2024-09-10T09:38:48.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:48.853+0000 7fa9d7e00640 1 -- 172.21.15.5:0/2719905325 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fa9e4004d60 con 0x561399c12c50 2024-09-10T09:38:48.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:48.853+0000 7fa9ffa8c740 1 -- 172.21.15.5:0/2719905325 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa9e40047a0 con 0x561399c12c50 2024-09-10T09:38:48.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:48.853+0000 7fa9d7e00640 1 -- 172.21.15.5:0/2719905325 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fa9e4037050 con 0x561399c12c50 2024-09-10T09:38:48.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:48.853+0000 7fa9ffa8c740 10 client.15408 fetch_fsmap learned FSMap version 30 2024-09-10T09:38:48.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:48.853+0000 7fa9ffa8c740 1 -- 172.21.15.5:0/2719905325 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x561399c98810 con 0x561399c12c50 2024-09-10T09:38:48.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:48.853+0000 7fa9d7e00640 1 -- 172.21.15.5:0/2719905325 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa9e40047a0 con 0x561399c12c50 2024-09-10T09:38:48.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:48.854+0000 7fa9d7e00640 1 -- 172.21.15.5:0/2719905325 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fa9e4027920 con 0x561399c12c50 2024-09-10T09:38:48.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:48.854+0000 7fa9ffa8c740 10 client.15408 fetch_fsmap finished waiting for FSMap version 30 2024-09-10T09:38:48.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:48.854+0000 7fa9ffa8c740 10 client.15408 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-10T09:38:48.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:48.854+0000 7fa9ffa8c740 20 client.15408 populate_metadata read hostname 'smithi005' 2024-09-10T09:38:48.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:48.854+0000 7fa9ffa8c740 1 --2- 172.21.15.5:0/2719905325 >> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] conn(0x561399c9d070 0x561399c9bd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:48.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:48.854+0000 7fa9ffa8c740 4 client.15408 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T09:38:48.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:48.854+0000 7fa9ffa8c740 1 -- 172.21.15.5:0/2719905325 --> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] -- command(tid 0: {"prefix": "status"}) -- 0x561399b7fb30 con 0x561399c9d070 2024-09-10T09:38:48.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:48.854+0000 7fa9f6800640 1 --2- 172.21.15.5:0/2719905325 >> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] conn(0x561399c9d070 0x561399c9bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:48.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:48.855+0000 7fa9f6800640 1 --2- 172.21.15.5:0/2719905325 >> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] conn(0x561399c9d070 0x561399c9bd80 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=dfc74377a7235eb8 server_cookie=90f503b247b9c955 in_seq=0 out_seq=0 2024-09-10T09:38:48.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:48.855+0000 7fa9d7e00640 10 client.15408 ms_handle_connect on v2:172.21.15.159:6836/620627639 2024-09-10T09:38:48.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:48.855+0000 7fa9d7e00640 1 -- 172.21.15.5:0/2719905325 <== mds.2 v2:172.21.15.159:6836/620627639 1 ==== command_reply(tid 0: 0 ) ==== 8+0+322 (crc 0 0 0) 0x561399b7fb30 con 0x561399c9d070 2024-09-10T09:38:48.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:48.855+0000 7fa9d7e00640 10 client.15408 handle_command_reply: tid=0 2024-09-10T09:38:48.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:48.855+0000 7fa9ffa8c740 1 client.15408 shutdown 2024-09-10T09:38:48.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:48.855+0000 7fa9ffa8c740 1 -- 172.21.15.5:0/2719905325 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561399c12c50 msgr2=0x561399c17480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:38:48.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:48.855+0000 7fa9ffa8c740 1 --2- 172.21.15.5:0/2719905325 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561399c12c50 0x561399c17480 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fa9e400bd40 tx=0x7fa9e40046f0 comp rx=0 tx=0).stop 2024-09-10T09:38:48.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:48.855+0000 7fa9ffa8c740 1 -- 172.21.15.5:0/2719905325 shutdown_connections 2024-09-10T09:38:48.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:48.855+0000 7fa9ffa8c740 1 --2- 172.21.15.5:0/2719905325 >> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] conn(0x561399c9d070 0x561399c9bd80 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:48.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:48.855+0000 7fa9ffa8c740 1 --2- 172.21.15.5:0/2719905325 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561399c1d510 0x561399c17bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:48.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:48.855+0000 7fa9ffa8c740 1 --2- 172.21.15.5:0/2719905325 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561399c12c50 0x561399c17480 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:48.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:48.855+0000 7fa9d7e00640 0 client.15408 ms_handle_reset on v2:172.21.15.159:6836/620627639 2024-09-10T09:38:48.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:48.855+0000 7fa9ffa8c740 1 --2- 172.21.15.5:0/2719905325 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561399c12330 0x561399c37f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:48.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:48.855+0000 7fa9ffa8c740 1 -- 172.21.15.5:0/2719905325 >> 172.21.15.5:0/2719905325 conn(0x561399c0e8c0 msgr2=0x561399c13cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:38:48.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:48.855+0000 7fa9ffa8c740 1 -- 172.21.15.5:0/2719905325 shutdown_connections 2024-09-10T09:38:48.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:48.855+0000 7fa9ffa8c740 1 -- 172.21.15.5:0/2719905325 wait complete. 2024-09-10T09:38:48.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:48.856+0000 7fa9ffa8c740 20 client.15408 trim_cache size 0 max 16384 2024-09-10T09:38:48.861 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-10T09:38:48.861 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_fsid": "bade4b50-6f56-11ef-bcea-c7b262605968", 2024-09-10T09:38:48.861 INFO:teuthology.orchestra.run.smithi005.stdout: "whoami": 2, 2024-09-10T09:38:48.861 INFO:teuthology.orchestra.run.smithi005.stdout: "id": 24511, 2024-09-10T09:38:48.861 INFO:teuthology.orchestra.run.smithi005.stdout: "want_state": "up:active", 2024-09-10T09:38:48.861 INFO:teuthology.orchestra.run.smithi005.stdout: "state": "up:active", 2024-09-10T09:38:48.861 INFO:teuthology.orchestra.run.smithi005.stdout: "fs_name": "cephfs", 2024-09-10T09:38:48.861 INFO:teuthology.orchestra.run.smithi005.stdout: "rank_uptime": 156.053673715, 2024-09-10T09:38:48.861 INFO:teuthology.orchestra.run.smithi005.stdout: "mdsmap_epoch": 30, 2024-09-10T09:38:48.862 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch": 96, 2024-09-10T09:38:48.862 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch_barrier": 96, 2024-09-10T09:38:48.862 INFO:teuthology.orchestra.run.smithi005.stdout: "uptime": 227.493881724 2024-09-10T09:38:48.862 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-10T09:38:48.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:48.856+0000 7fa9fd000640 1 -- 172.21.15.5:0/648737825 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fa9dc0a2c90 msgr2=0x7fa9dc0a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:38:48.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:48.856+0000 7fa9fd000640 1 --2- 172.21.15.5:0/648737825 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fa9dc0a2c90 0x7fa9dc0a5150 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fa9e8097df0 tx=0x7fa9e8097040 comp rx=0 tx=0).stop 2024-09-10T09:38:48.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:48.856+0000 7fa9fd000640 1 -- 172.21.15.5:0/648737825 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa9f8076720 msgr2=0x7fa9f81a46a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:38:48.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:48.856+0000 7fa9fd000640 1 --2- 172.21.15.5:0/648737825 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa9f8076720 0x7fa9f81a46a0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fa9e800d6d0 tx=0x7fa9e800db90 comp rx=0 tx=0).stop 2024-09-10T09:38:48.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:48.856+0000 7fa9fd000640 1 -- 172.21.15.5:0/648737825 shutdown_connections 2024-09-10T09:38:48.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:48.856+0000 7fa9fd000640 1 --2- 172.21.15.5:0/648737825 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fa9dc0a2c90 0x7fa9dc0a5150 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:48.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:48.856+0000 7fa9fd000640 1 --2- 172.21.15.5:0/648737825 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa9f8112a30 0x7fa9f819e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:48.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:48.856+0000 7fa9fd000640 1 --2- 172.21.15.5:0/648737825 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa9f8077040 0x7fa9f81a4be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:48.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:48.856+0000 7fa9fd000640 1 --2- 172.21.15.5:0/648737825 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa9f8076720 0x7fa9f81a46a0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:48.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:48.856+0000 7fa9fd000640 1 -- 172.21.15.5:0/648737825 >> 172.21.15.5:0/648737825 conn(0x7fa9f80ffa90 msgr2=0x7fa9f8101e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:38:48.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:48.856+0000 7fa9fd000640 1 -- 172.21.15.5:0/648737825 shutdown_connections 2024-09-10T09:38:48.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:48.856+0000 7fa9fd000640 1 -- 172.21.15.5:0/648737825 wait complete. 2024-09-10T09:38:49.306 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:38:49 smithi005 ceph-mon[30350]: pgmap v407: 129 pgs: 129 active+clean; 660 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 938 B/s wr, 9 op/s 2024-09-10T09:38:49.385 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:38:49 smithi102 ceph-mon[35442]: pgmap v407: 129 pgs: 129 active+clean; 660 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 938 B/s wr, 9 op/s 2024-09-10T09:38:49.402 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:38:49 smithi159 ceph-mon[35513]: pgmap v407: 129 pgs: 129 active+clean; 660 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 938 B/s wr, 9 op/s 2024-09-10T09:38:49.743 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph tell mds.i status 2024-09-10T09:38:50.120 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:38:50.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:50.680+0000 7fd372200640 1 -- 172.21.15.5:0/3595806162 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd36c077040 msgr2=0x7fd36c1124f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:38:50.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:50.680+0000 7fd372200640 1 --2- 172.21.15.5:0/3595806162 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd36c077040 0x7fd36c1124f0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fd358009900 tx=0x7fd35802a3f0 comp rx=0 tx=0).stop 2024-09-10T09:38:50.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:50.680+0000 7fd372200640 1 -- 172.21.15.5:0/3595806162 shutdown_connections 2024-09-10T09:38:50.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:50.680+0000 7fd372200640 1 --2- 172.21.15.5:0/3595806162 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd36c112a30 0x7fd36c114e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:50.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:50.680+0000 7fd372200640 1 --2- 172.21.15.5:0/3595806162 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd36c077040 0x7fd36c1124f0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:50.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:50.680+0000 7fd372200640 1 --2- 172.21.15.5:0/3595806162 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd36c076720 0x7fd36c076b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:50.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:50.680+0000 7fd372200640 1 -- 172.21.15.5:0/3595806162 >> 172.21.15.5:0/3595806162 conn(0x7fd36c0ffa90 msgr2=0x7fd36c101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:38:50.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:50.680+0000 7fd372200640 1 -- 172.21.15.5:0/3595806162 shutdown_connections 2024-09-10T09:38:50.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:50.680+0000 7fd372200640 1 -- 172.21.15.5:0/3595806162 wait complete. 2024-09-10T09:38:50.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:50.681+0000 7fd372200640 1 Processor -- start 2024-09-10T09:38:50.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:50.681+0000 7fd372200640 1 -- start start 2024-09-10T09:38:50.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:50.682+0000 7fd372200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd36c076720 0x7fd36c1a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:50.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:50.682+0000 7fd372200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd36c077040 0x7fd36c1a4c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:50.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:50.682+0000 7fd372200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd36c112a30 0x7fd36c19e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:50.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:50.682+0000 7fd372200640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd36c1185a0 con 0x7fd36c076720 2024-09-10T09:38:50.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:50.682+0000 7fd372200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fd36c118420 con 0x7fd36c112a30 2024-09-10T09:38:50.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:50.682+0000 7fd372200640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fd36c118720 con 0x7fd36c077040 2024-09-10T09:38:50.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:50.682+0000 7fd36b400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd36c076720 0x7fd36c1a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:50.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:50.682+0000 7fd36b400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd36c076720 0x7fd36c1a46f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58294/0 (socket says 172.21.15.5:58294) 2024-09-10T09:38:50.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:50.682+0000 7fd36b400640 1 -- 172.21.15.5:0/2790995569 learned_addr learned my addr 172.21.15.5:0/2790995569 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:38:50.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:50.682+0000 7fd36aa00640 1 --2- 172.21.15.5:0/2790995569 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd36c077040 0x7fd36c1a4c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:50.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:50.682+0000 7fd36be00640 1 --2- 172.21.15.5:0/2790995569 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd36c112a30 0x7fd36c19e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:50.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:50.683+0000 7fd36b400640 1 -- 172.21.15.5:0/2790995569 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd36c077040 msgr2=0x7fd36c1a4c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:38:50.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:50.683+0000 7fd36b400640 1 --2- 172.21.15.5:0/2790995569 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd36c077040 0x7fd36c1a4c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:50.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:50.683+0000 7fd36b400640 1 -- 172.21.15.5:0/2790995569 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd36c112a30 msgr2=0x7fd36c19e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:38:50.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:50.683+0000 7fd36b400640 1 --2- 172.21.15.5:0/2790995569 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd36c112a30 0x7fd36c19e8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:50.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:50.683+0000 7fd36b400640 1 -- 172.21.15.5:0/2790995569 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd36c19efe0 con 0x7fd36c076720 2024-09-10T09:38:50.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:50.683+0000 7fd36aa00640 1 --2- 172.21.15.5:0/2790995569 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd36c077040 0x7fd36c1a4c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:38:50.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:50.683+0000 7fd36b400640 1 --2- 172.21.15.5:0/2790995569 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd36c076720 0x7fd36c1a46f0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fd35c00ea60 tx=0x7fd35c00ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:38:50.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:50.683+0000 7fd363e00640 1 -- 172.21.15.5:0/2790995569 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd35c00cde0 con 0x7fd36c076720 2024-09-10T09:38:50.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:50.683+0000 7fd372200640 1 -- 172.21.15.5:0/2790995569 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd36c19f2d0 con 0x7fd36c076720 2024-09-10T09:38:50.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:50.683+0000 7fd372200640 1 -- 172.21.15.5:0/2790995569 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd36c1ab5c0 con 0x7fd36c076720 2024-09-10T09:38:50.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:50.683+0000 7fd363e00640 1 -- 172.21.15.5:0/2790995569 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd35c0035f0 con 0x7fd36c076720 2024-09-10T09:38:50.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:50.684+0000 7fd363e00640 1 -- 172.21.15.5:0/2790995569 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd35c018690 con 0x7fd36c076720 2024-09-10T09:38:50.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:50.687+0000 7fd374dd5740 1 Processor -- start 2024-09-10T09:38:50.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:50.687+0000 7fd363e00640 1 -- 172.21.15.5:0/2790995569 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 19) ==== 137883+0+0 (secure 0 0 0) 0x7fd35c003a80 con 0x7fd36c076720 2024-09-10T09:38:50.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:50.687+0000 7fd374dd5740 1 -- start start 2024-09-10T09:38:50.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:50.687+0000 7fd374dd5740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ae94b78330 0x55ae94b78710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:50.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:50.687+0000 7fd374dd5740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ae94b78c50 0x55ae94b82fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:50.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:50.687+0000 7fd374dd5740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ae94b83510 0x55ae94b85900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:50.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:50.687+0000 7fd374dd5740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55ae94ade0b0 con 0x55ae94b78330 2024-09-10T09:38:50.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:50.687+0000 7fd374dd5740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55ae94a85ab0 con 0x55ae94b78c50 2024-09-10T09:38:50.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:50.687+0000 7fd374dd5740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55ae94ada070 con 0x55ae94b83510 2024-09-10T09:38:50.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:50.688+0000 7fd36be00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ae94b78330 0x55ae94b78710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:50.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:50.688+0000 7fd36aa00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ae94b83510 0x55ae94b85900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:50.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:50.688+0000 7fd36aa00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ae94b83510 0x55ae94b85900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:33202/0 (socket says 172.21.15.5:33202) 2024-09-10T09:38:50.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:50.688+0000 7fd36b400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ae94b78c50 0x55ae94b82fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:50.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:50.688+0000 7fd36aa00640 1 -- 172.21.15.5:0/2272318360 learned_addr learned my addr 172.21.15.5:0/2272318360 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:38:50.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:50.688+0000 7fd36aa00640 1 -- 172.21.15.5:0/2272318360 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ae94b78c50 msgr2=0x55ae94b82fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:38:50.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:50.688+0000 7fd36aa00640 1 --2- 172.21.15.5:0/2272318360 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ae94b78c50 0x55ae94b82fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:50.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:50.688+0000 7fd36aa00640 1 -- 172.21.15.5:0/2272318360 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ae94b78330 msgr2=0x55ae94b78710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:38:50.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:50.688+0000 7fd36aa00640 1 --2- 172.21.15.5:0/2272318360 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ae94b78330 0x55ae94b78710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:50.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:50.688+0000 7fd36aa00640 1 -- 172.21.15.5:0/2272318360 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ae94a924a0 con 0x55ae94b83510 2024-09-10T09:38:50.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:50.688+0000 7fd36be00640 1 --2- 172.21.15.5:0/2272318360 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ae94b78330 0x55ae94b78710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:38:50.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:50.688+0000 7fd36aa00640 1 --2- 172.21.15.5:0/2272318360 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ae94b83510 0x55ae94b85900 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fd358009900 tx=0x7fd358005e80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2f6edb82705fef2d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:38:50.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:50.689+0000 7fd363e00640 1 --2- 172.21.15.5:0/2790995569 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fd3500a2c90 0x7fd3500a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:50.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:50.689+0000 7fd363e00640 1 -- 172.21.15.5:0/2790995569 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fd35c0cf900 con 0x7fd36c076720 2024-09-10T09:38:50.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:50.689+0000 7fd357e00640 1 -- 172.21.15.5:0/2272318360 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd358035040 con 0x55ae94b83510 2024-09-10T09:38:50.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:50.689+0000 7fd357e00640 1 -- 172.21.15.5:0/2272318360 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd3580318b0 con 0x55ae94b83510 2024-09-10T09:38:50.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:50.689+0000 7fd357e00640 1 -- 172.21.15.5:0/2272318360 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd358004a00 con 0x55ae94b83510 2024-09-10T09:38:50.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:50.689+0000 7fd36be00640 1 --2- 172.21.15.5:0/2790995569 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fd3500a2c90 0x7fd3500a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:50.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:50.690+0000 7fd374dd5740 1 -- 172.21.15.5:0/2272318360 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ae94b83510 msgr2=0x55ae94b85900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:38:50.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:50.690+0000 7fd374dd5740 1 --2- 172.21.15.5:0/2272318360 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ae94b83510 0x55ae94b85900 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fd358009900 tx=0x7fd358005e80 comp rx=0 tx=0).stop 2024-09-10T09:38:50.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:50.690+0000 7fd36be00640 1 --2- 172.21.15.5:0/2790995569 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fd3500a2c90 0x7fd3500a5150 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fd36c06b460 tx=0x7fd364009600 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:38:50.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:50.690+0000 7fd374dd5740 1 -- 172.21.15.5:0/2272318360 shutdown_connections 2024-09-10T09:38:50.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:50.690+0000 7fd374dd5740 1 --2- 172.21.15.5:0/2272318360 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ae94b83510 0x55ae94b85900 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:50.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:50.690+0000 7fd374dd5740 1 --2- 172.21.15.5:0/2272318360 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ae94b78c50 0x55ae94b82fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:50.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:50.690+0000 7fd374dd5740 1 --2- 172.21.15.5:0/2272318360 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ae94b78330 0x55ae94b78710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:50.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:50.690+0000 7fd374dd5740 1 -- 172.21.15.5:0/2272318360 >> 172.21.15.5:0/2272318360 conn(0x55ae94b748c0 msgr2=0x55ae94b74ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:38:50.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:50.690+0000 7fd374dd5740 1 -- 172.21.15.5:0/2272318360 shutdown_connections 2024-09-10T09:38:50.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:50.690+0000 7fd374dd5740 1 -- 172.21.15.5:0/2272318360 wait complete. 2024-09-10T09:38:50.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:50.690+0000 7fd374dd5740 1 -- start start 2024-09-10T09:38:50.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:50.691+0000 7fd374dd5740 1 Processor -- start 2024-09-10T09:38:50.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:50.691+0000 7fd374dd5740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ae94b78330 0x55ae94b80520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:50.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:50.691+0000 7fd374dd5740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ae94b78c50 0x55ae94b80a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:50.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:50.691+0000 7fd374dd5740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ae94b83510 0x55ae94b80fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:50.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:50.691+0000 7fd374dd5740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55ae94b86060 con 0x55ae94b78c50 2024-09-10T09:38:50.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:50.691+0000 7fd374dd5740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55ae94a66920 con 0x55ae94b78330 2024-09-10T09:38:50.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:50.691+0000 7fd374dd5740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55ae94b861e0 con 0x55ae94b83510 2024-09-10T09:38:50.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:50.691+0000 7fd374dd5740 10 client.0 _ll_register_callbacks cb 0x55ae94a8a8a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:38:50.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:50.691+0000 7fd36aa00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ae94b78c50 0x55ae94b80a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:50.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:50.691+0000 7fd36be00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ae94b83510 0x55ae94b80fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:50.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:50.691+0000 7fd36aa00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ae94b78c50 0x55ae94b80a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58308/0 (socket says 172.21.15.5:58308) 2024-09-10T09:38:50.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:50.691+0000 7fd36aa00640 1 -- 172.21.15.5:0/3841346971 learned_addr learned my addr 172.21.15.5:0/3841346971 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:38:50.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:50.691+0000 7fd36b400640 1 --2- 172.21.15.5:0/3841346971 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ae94b78330 0x55ae94b80520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:50.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:50.692+0000 7fd36aa00640 1 -- 172.21.15.5:0/3841346971 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ae94b83510 msgr2=0x55ae94b80fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:38:50.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:50.692+0000 7fd36aa00640 1 --2- 172.21.15.5:0/3841346971 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ae94b83510 0x55ae94b80fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:50.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:50.692+0000 7fd36aa00640 1 -- 172.21.15.5:0/3841346971 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ae94b78330 msgr2=0x55ae94b80520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:38:50.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:50.692+0000 7fd36aa00640 1 --2- 172.21.15.5:0/3841346971 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ae94b78330 0x55ae94b80520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:50.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:50.692+0000 7fd36aa00640 1 -- 172.21.15.5:0/3841346971 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ae9484f2b0 con 0x55ae94b78c50 2024-09-10T09:38:50.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:50.692+0000 7fd36b400640 1 --2- 172.21.15.5:0/3841346971 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ae94b78330 0x55ae94b80520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:38:50.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:50.692+0000 7fd36be00640 1 --2- 172.21.15.5:0/3841346971 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ae94b83510 0x55ae94b80fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:38:50.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:50.692+0000 7fd36aa00640 1 --2- 172.21.15.5:0/3841346971 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ae94b78c50 0x55ae94b80a60 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fd358000c00 tx=0x7fd358031a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87014401ac2cc2f5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:38:50.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:50.692+0000 7fd354c00640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-10T09:38:50.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:50.692+0000 7fd354c00640 1 -- 172.21.15.5:0/3841346971 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd35804e070 con 0x55ae94b78c50 2024-09-10T09:38:50.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:50.692+0000 7fd374dd5740 1 -- 172.21.15.5:0/3841346971 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd3580318b0 con 0x55ae94b78c50 2024-09-10T09:38:50.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:50.693+0000 7fd354c00640 1 -- 172.21.15.5:0/3841346971 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fd358004250 con 0x55ae94b78c50 2024-09-10T09:38:50.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:50.693+0000 7fd354c00640 1 -- 172.21.15.5:0/3841346971 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fd35805a050 con 0x55ae94b78c50 2024-09-10T09:38:50.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:50.693+0000 7fd374dd5740 10 client.15420 fetch_fsmap learned FSMap version 30 2024-09-10T09:38:50.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:50.693+0000 7fd374dd5740 1 -- 172.21.15.5:0/3841346971 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55ae94b91e30 con 0x55ae94b78c50 2024-09-10T09:38:50.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:50.693+0000 7fd354c00640 1 -- 172.21.15.5:0/3841346971 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd3580318b0 con 0x55ae94b78c50 2024-09-10T09:38:50.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:50.693+0000 7fd354c00640 1 -- 172.21.15.5:0/3841346971 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fd3580466b0 con 0x55ae94b78c50 2024-09-10T09:38:50.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:50.693+0000 7fd374dd5740 10 client.15420 fetch_fsmap finished waiting for FSMap version 30 2024-09-10T09:38:50.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:50.694+0000 7fd374dd5740 10 client.15420 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-10T09:38:50.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:50.694+0000 7fd374dd5740 20 client.15420 populate_metadata read hostname 'smithi005' 2024-09-10T09:38:50.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:50.694+0000 7fd374dd5740 1 --2- 172.21.15.5:0/3841346971 >> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] conn(0x55ae94b94060 0x55ae94b954d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:50.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:50.694+0000 7fd374dd5740 4 client.15420 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:38:50.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:50.694+0000 7fd374dd5740 1 -- 172.21.15.5:0/3841346971 --> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ae94aeb620 con 0x55ae94b94060 2024-09-10T09:38:50.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:50.694+0000 7fd36b400640 1 --2- 172.21.15.5:0/3841346971 >> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] conn(0x55ae94b94060 0x55ae94b954d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:50.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:50.694+0000 7fd36b400640 1 --2- 172.21.15.5:0/3841346971 >> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] conn(0x55ae94b94060 0x55ae94b954d0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=20d5c1275fa16f40 server_cookie=c27586a259e1ac2 in_seq=0 out_seq=0 2024-09-10T09:38:50.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:50.694+0000 7fd354c00640 10 client.15420 ms_handle_connect on v2:172.21.15.159:6838/2073739599 2024-09-10T09:38:50.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:50.695+0000 7fd354c00640 1 -- 172.21.15.5:0/3841346971 <== mds.1 v2:172.21.15.159:6838/2073739599 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ae94aeb620 con 0x55ae94b94060 2024-09-10T09:38:50.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:50.695+0000 7fd354c00640 10 client.15420 handle_command_reply: tid=0 2024-09-10T09:38:50.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:50.695+0000 7fd374dd5740 1 client.15420 shutdown 2024-09-10T09:38:50.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:50.695+0000 7fd374dd5740 1 -- 172.21.15.5:0/3841346971 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ae94b78c50 msgr2=0x55ae94b80a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:38:50.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:50.695+0000 7fd374dd5740 1 --2- 172.21.15.5:0/3841346971 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ae94b78c50 0x55ae94b80a60 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fd358000c00 tx=0x7fd358031a60 comp rx=0 tx=0).stop 2024-09-10T09:38:50.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:50.695+0000 7fd374dd5740 1 -- 172.21.15.5:0/3841346971 shutdown_connections 2024-09-10T09:38:50.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:50.695+0000 7fd374dd5740 1 --2- 172.21.15.5:0/3841346971 >> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] conn(0x55ae94b94060 0x55ae94b954d0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:50.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:50.695+0000 7fd374dd5740 1 --2- 172.21.15.5:0/3841346971 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ae94b83510 0x55ae94b80fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:50.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:50.695+0000 7fd374dd5740 1 --2- 172.21.15.5:0/3841346971 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ae94b78c50 0x55ae94b80a60 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:50.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:50.695+0000 7fd374dd5740 1 --2- 172.21.15.5:0/3841346971 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ae94b78330 0x55ae94b80520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:50.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:50.695+0000 7fd374dd5740 1 -- 172.21.15.5:0/3841346971 >> 172.21.15.5:0/3841346971 conn(0x55ae94b748c0 msgr2=0x55ae94b7a5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:38:50.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:50.695+0000 7fd354c00640 0 client.15420 ms_handle_reset on v2:172.21.15.159:6838/2073739599 2024-09-10T09:38:50.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:50.696+0000 7fd374dd5740 1 -- 172.21.15.5:0/3841346971 shutdown_connections 2024-09-10T09:38:50.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:50.696+0000 7fd374dd5740 1 -- 172.21.15.5:0/3841346971 wait complete. 2024-09-10T09:38:50.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:50.696+0000 7fd374dd5740 20 client.15420 trim_cache size 0 max 16384 2024-09-10T09:38:50.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:50.714+0000 7fd374dd5740 1 Processor -- start 2024-09-10T09:38:50.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:50.714+0000 7fd374dd5740 1 -- start start 2024-09-10T09:38:50.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:50.715+0000 7fd374dd5740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ae94b78330 0x55ae94b80df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:50.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:50.715+0000 7fd374dd5740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ae94b78c50 0x55ae94b7d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:50.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:50.715+0000 7fd374dd5740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ae94b83510 0x55ae94b7dd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:50.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:50.715+0000 7fd374dd5740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55ae94b93810 con 0x55ae94b83510 2024-09-10T09:38:50.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:50.715+0000 7fd374dd5740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55ae94b93ee0 con 0x55ae94b78330 2024-09-10T09:38:50.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:50.715+0000 7fd374dd5740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55ae94b945b0 con 0x55ae94b78c50 2024-09-10T09:38:50.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:50.715+0000 7fd36be00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ae94b83510 0x55ae94b7dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:50.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:50.715+0000 7fd36be00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ae94b83510 0x55ae94b7dd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58324/0 (socket says 172.21.15.5:58324) 2024-09-10T09:38:50.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:50.715+0000 7fd36be00640 1 -- 172.21.15.5:0/1327372728 learned_addr learned my addr 172.21.15.5:0/1327372728 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:38:50.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:50.715+0000 7fd36aa00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ae94b78c50 0x55ae94b7d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:50.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:50.715+0000 7fd36b400640 1 --2- 172.21.15.5:0/1327372728 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ae94b78330 0x55ae94b80df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:50.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:50.716+0000 7fd36b400640 1 -- 172.21.15.5:0/1327372728 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ae94b78c50 msgr2=0x55ae94b7d7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:38:50.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:50.716+0000 7fd36b400640 1 --2- 172.21.15.5:0/1327372728 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ae94b78c50 0x55ae94b7d7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:50.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:50.716+0000 7fd36b400640 1 -- 172.21.15.5:0/1327372728 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ae94b83510 msgr2=0x55ae94b7dd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:38:50.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:50.716+0000 7fd36b400640 1 --2- 172.21.15.5:0/1327372728 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ae94b83510 0x55ae94b7dd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:50.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:50.716+0000 7fd36b400640 1 -- 172.21.15.5:0/1327372728 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ae94b75a50 con 0x55ae94b78330 2024-09-10T09:38:50.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:50.716+0000 7fd36b400640 1 --2- 172.21.15.5:0/1327372728 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ae94b78330 0x55ae94b80df0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fd35c0cce60 tx=0x7fd35c0d5530 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9b9b7ba08b37490d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:38:50.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:50.716+0000 7fd357400640 1 -- 172.21.15.5:0/1327372728 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd35c09b510 con 0x55ae94b78330 2024-09-10T09:38:50.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:50.716+0000 7fd357400640 1 -- 172.21.15.5:0/1327372728 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd35c09c2b0 con 0x55ae94b78330 2024-09-10T09:38:50.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:50.716+0000 7fd357400640 1 -- 172.21.15.5:0/1327372728 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd35c098220 con 0x55ae94b78330 2024-09-10T09:38:50.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:50.717+0000 7fd374dd5740 1 -- 172.21.15.5:0/1327372728 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ae94b78330 msgr2=0x55ae94b80df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:38:50.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:50.717+0000 7fd374dd5740 1 --2- 172.21.15.5:0/1327372728 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ae94b78330 0x55ae94b80df0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fd35c0cce60 tx=0x7fd35c0d5530 comp rx=0 tx=0).stop 2024-09-10T09:38:50.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:50.717+0000 7fd374dd5740 1 -- 172.21.15.5:0/1327372728 shutdown_connections 2024-09-10T09:38:50.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:50.717+0000 7fd374dd5740 1 --2- 172.21.15.5:0/1327372728 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ae94b83510 0x55ae94b7dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:50.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:50.717+0000 7fd374dd5740 1 --2- 172.21.15.5:0/1327372728 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ae94b78c50 0x55ae94b7d7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:50.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:50.717+0000 7fd374dd5740 1 --2- 172.21.15.5:0/1327372728 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ae94b78330 0x55ae94b80df0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:50.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:50.717+0000 7fd374dd5740 1 -- 172.21.15.5:0/1327372728 >> 172.21.15.5:0/1327372728 conn(0x55ae94b748c0 msgr2=0x55ae94b7ee10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:38:50.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:50.717+0000 7fd374dd5740 1 -- 172.21.15.5:0/1327372728 shutdown_connections 2024-09-10T09:38:50.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:50.717+0000 7fd374dd5740 1 -- 172.21.15.5:0/1327372728 wait complete. 2024-09-10T09:38:50.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:50.718+0000 7fd374dd5740 1 -- start start 2024-09-10T09:38:50.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:50.718+0000 7fd374dd5740 1 Processor -- start 2024-09-10T09:38:50.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:50.718+0000 7fd374dd5740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ae94b78330 0x55ae94b9d640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:50.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:50.718+0000 7fd374dd5740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ae94b78c50 0x55ae94b9db80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:50.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:50.718+0000 7fd374dd5740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ae94b83510 0x55ae94b9e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:50.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:50.718+0000 7fd374dd5740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55ae94b95350 con 0x55ae94b78c50 2024-09-10T09:38:50.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:50.718+0000 7fd374dd5740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55ae94b94c80 con 0x55ae94b78330 2024-09-10T09:38:50.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:50.718+0000 7fd374dd5740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55ae94b95a20 con 0x55ae94b83510 2024-09-10T09:38:50.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:50.718+0000 7fd374dd5740 10 client.0 _ll_register_callbacks cb 0x7fd35c0d5400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:38:50.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:50.719+0000 7fd36be00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ae94b83510 0x55ae94b9e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:50.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:50.719+0000 7fd36be00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ae94b83510 0x55ae94b9e0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:33226/0 (socket says 172.21.15.5:33226) 2024-09-10T09:38:50.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:50.719+0000 7fd36be00640 1 -- 172.21.15.5:0/1173651874 learned_addr learned my addr 172.21.15.5:0/1173651874 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:38:50.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:50.719+0000 7fd36aa00640 1 --2- 172.21.15.5:0/1173651874 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ae94b78c50 0x55ae94b9db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:50.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:50.719+0000 7fd36b400640 1 --2- 172.21.15.5:0/1173651874 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ae94b78330 0x55ae94b9d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:50.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:50.719+0000 7fd36be00640 1 -- 172.21.15.5:0/1173651874 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ae94b78330 msgr2=0x55ae94b9d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:38:50.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:50.719+0000 7fd36be00640 1 --2- 172.21.15.5:0/1173651874 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ae94b78330 0x55ae94b9d640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:50.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:50.719+0000 7fd36be00640 1 -- 172.21.15.5:0/1173651874 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ae94b78c50 msgr2=0x55ae94b9db80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:38:50.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:50.719+0000 7fd36be00640 1 --2- 172.21.15.5:0/1173651874 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ae94b78c50 0x55ae94b9db80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:50.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:50.719+0000 7fd36be00640 1 -- 172.21.15.5:0/1173651874 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ae94b98820 con 0x55ae94b83510 2024-09-10T09:38:50.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:50.719+0000 7fd36be00640 1 --2- 172.21.15.5:0/1173651874 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ae94b83510 0x55ae94b9e0c0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fd364005f40 tx=0x7fd364004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ba886774386151b4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:38:50.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:50.719+0000 7fd354c00640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-09-10T09:38:50.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:50.720+0000 7fd354c00640 1 -- 172.21.15.5:0/1173651874 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd36402f060 con 0x55ae94b83510 2024-09-10T09:38:50.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:50.720+0000 7fd374dd5740 1 -- 172.21.15.5:0/1173651874 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd35c09c2b0 con 0x55ae94b83510 2024-09-10T09:38:50.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:50.720+0000 7fd354c00640 1 -- 172.21.15.5:0/1173651874 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fd36401d600 con 0x55ae94b83510 2024-09-10T09:38:50.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:50.720+0000 7fd354c00640 1 -- 172.21.15.5:0/1173651874 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fd364036050 con 0x55ae94b83510 2024-09-10T09:38:50.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:50.720+0000 7fd374dd5740 10 client.25459 fetch_fsmap learned FSMap version 30 2024-09-10T09:38:50.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:50.720+0000 7fd374dd5740 1 -- 172.21.15.5:0/1173651874 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55ae94bfed50 con 0x55ae94b83510 2024-09-10T09:38:50.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:50.720+0000 7fd354c00640 1 -- 172.21.15.5:0/1173651874 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd35c09c2b0 con 0x55ae94b83510 2024-09-10T09:38:50.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:50.721+0000 7fd354c00640 1 -- 172.21.15.5:0/1173651874 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fd364029070 con 0x55ae94b83510 2024-09-10T09:38:50.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:50.721+0000 7fd374dd5740 10 client.25459 fetch_fsmap finished waiting for FSMap version 30 2024-09-10T09:38:50.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:50.721+0000 7fd374dd5740 10 client.25459 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-10T09:38:50.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:50.721+0000 7fd374dd5740 20 client.25459 populate_metadata read hostname 'smithi005' 2024-09-10T09:38:50.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:50.721+0000 7fd374dd5740 1 --2- 172.21.15.5:0/1173651874 >> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] conn(0x55ae94c03920 0x55ae94c02630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:50.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:50.721+0000 7fd374dd5740 4 client.25459 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T09:38:50.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:50.721+0000 7fd374dd5740 1 -- 172.21.15.5:0/1173651874 --> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] -- command(tid 0: {"prefix": "status"}) -- 0x55ae94ae5b30 con 0x55ae94c03920 2024-09-10T09:38:50.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:50.721+0000 7fd36b400640 1 --2- 172.21.15.5:0/1173651874 >> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] conn(0x55ae94c03920 0x55ae94c02630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:50.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:50.721+0000 7fd36b400640 1 --2- 172.21.15.5:0/1173651874 >> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] conn(0x55ae94c03920 0x55ae94c02630 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=76f458186bb7c2d8 server_cookie=ea11f338d8aa3bb6 in_seq=0 out_seq=0 2024-09-10T09:38:50.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:50.721+0000 7fd354c00640 10 client.25459 ms_handle_connect on v2:172.21.15.159:6838/2073739599 2024-09-10T09:38:50.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:50.722+0000 7fd354c00640 1 -- 172.21.15.5:0/1173651874 <== mds.1 v2:172.21.15.159:6838/2073739599 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55ae94ae5b30 con 0x55ae94c03920 2024-09-10T09:38:50.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:50.722+0000 7fd354c00640 10 client.25459 handle_command_reply: tid=0 2024-09-10T09:38:50.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:50.722+0000 7fd374dd5740 1 client.25459 shutdown 2024-09-10T09:38:50.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:50.722+0000 7fd374dd5740 1 -- 172.21.15.5:0/1173651874 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ae94b83510 msgr2=0x55ae94b9e0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:38:50.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:50.722+0000 7fd374dd5740 1 --2- 172.21.15.5:0/1173651874 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ae94b83510 0x55ae94b9e0c0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fd364005f40 tx=0x7fd364004a40 comp rx=0 tx=0).stop 2024-09-10T09:38:50.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:50.722+0000 7fd374dd5740 1 -- 172.21.15.5:0/1173651874 shutdown_connections 2024-09-10T09:38:50.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:50.722+0000 7fd374dd5740 1 --2- 172.21.15.5:0/1173651874 >> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] conn(0x55ae94c03920 0x55ae94c02630 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:50.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:50.722+0000 7fd374dd5740 1 --2- 172.21.15.5:0/1173651874 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ae94b83510 0x55ae94b9e0c0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:50.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:50.722+0000 7fd374dd5740 1 --2- 172.21.15.5:0/1173651874 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ae94b78c50 0x55ae94b9db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:50.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:50.722+0000 7fd374dd5740 1 --2- 172.21.15.5:0/1173651874 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ae94b78330 0x55ae94b9d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:50.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:50.722+0000 7fd374dd5740 1 -- 172.21.15.5:0/1173651874 >> 172.21.15.5:0/1173651874 conn(0x55ae94b748c0 msgr2=0x55ae94b73200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:38:50.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:50.722+0000 7fd354c00640 0 client.25459 ms_handle_reset on v2:172.21.15.159:6838/2073739599 2024-09-10T09:38:50.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:50.722+0000 7fd374dd5740 1 -- 172.21.15.5:0/1173651874 shutdown_connections 2024-09-10T09:38:50.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:50.722+0000 7fd374dd5740 1 -- 172.21.15.5:0/1173651874 wait complete. 2024-09-10T09:38:50.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:50.722+0000 7fd374dd5740 20 client.25459 trim_cache size 0 max 16384 2024-09-10T09:38:50.731 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-10T09:38:50.731 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_fsid": "bade4b50-6f56-11ef-bcea-c7b262605968", 2024-09-10T09:38:50.731 INFO:teuthology.orchestra.run.smithi005.stdout: "whoami": 1, 2024-09-10T09:38:50.731 INFO:teuthology.orchestra.run.smithi005.stdout: "id": 24535, 2024-09-10T09:38:50.731 INFO:teuthology.orchestra.run.smithi005.stdout: "want_state": "up:active", 2024-09-10T09:38:50.731 INFO:teuthology.orchestra.run.smithi005.stdout: "state": "up:active", 2024-09-10T09:38:50.731 INFO:teuthology.orchestra.run.smithi005.stdout: "fs_name": "cephfs", 2024-09-10T09:38:50.731 INFO:teuthology.orchestra.run.smithi005.stdout: "rank_uptime": 158.92630708600001, 2024-09-10T09:38:50.731 INFO:teuthology.orchestra.run.smithi005.stdout: "mdsmap_epoch": 30, 2024-09-10T09:38:50.731 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch": 96, 2024-09-10T09:38:50.731 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch_barrier": 96, 2024-09-10T09:38:50.731 INFO:teuthology.orchestra.run.smithi005.stdout: "uptime": 223.87016505599999 2024-09-10T09:38:50.731 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-10T09:38:50.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:50.723+0000 7fd372200640 1 -- 172.21.15.5:0/2790995569 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fd3500a2c90 msgr2=0x7fd3500a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:38:50.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:50.723+0000 7fd372200640 1 --2- 172.21.15.5:0/2790995569 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fd3500a2c90 0x7fd3500a5150 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fd36c06b460 tx=0x7fd364009600 comp rx=0 tx=0).stop 2024-09-10T09:38:50.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:50.723+0000 7fd372200640 1 -- 172.21.15.5:0/2790995569 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd36c076720 msgr2=0x7fd36c1a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:38:50.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:50.723+0000 7fd372200640 1 --2- 172.21.15.5:0/2790995569 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd36c076720 0x7fd36c1a46f0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fd35c00ea60 tx=0x7fd35c00ef20 comp rx=0 tx=0).stop 2024-09-10T09:38:50.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:50.723+0000 7fd372200640 1 -- 172.21.15.5:0/2790995569 shutdown_connections 2024-09-10T09:38:50.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:50.723+0000 7fd372200640 1 --2- 172.21.15.5:0/2790995569 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fd3500a2c90 0x7fd3500a5150 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:50.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:50.723+0000 7fd372200640 1 --2- 172.21.15.5:0/2790995569 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd36c112a30 0x7fd36c19e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:50.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:50.723+0000 7fd372200640 1 --2- 172.21.15.5:0/2790995569 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd36c077040 0x7fd36c1a4c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:50.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:50.723+0000 7fd372200640 1 --2- 172.21.15.5:0/2790995569 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd36c076720 0x7fd36c1a46f0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:50.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:50.723+0000 7fd372200640 1 -- 172.21.15.5:0/2790995569 >> 172.21.15.5:0/2790995569 conn(0x7fd36c0ffa90 msgr2=0x7fd36c101e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:38:50.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:50.723+0000 7fd372200640 1 -- 172.21.15.5:0/2790995569 shutdown_connections 2024-09-10T09:38:50.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:50.723+0000 7fd372200640 1 -- 172.21.15.5:0/2790995569 wait complete. 2024-09-10T09:38:51.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:38:51 smithi102 ceph-mon[35442]: pgmap v408: 129 pgs: 129 active+clean; 660 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 938 B/s wr, 9 op/s 2024-09-10T09:38:51.431 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:38:51 smithi005 ceph-mon[30350]: pgmap v408: 129 pgs: 129 active+clean; 660 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 938 B/s wr, 9 op/s 2024-09-10T09:38:51.468 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph tell mds.b status 2024-09-10T09:38:51.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:38:51 smithi159 ceph-mon[35513]: pgmap v408: 129 pgs: 129 active+clean; 660 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 938 B/s wr, 9 op/s 2024-09-10T09:38:51.847 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:38:52.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:52.425+0000 7fde81800640 1 -- 172.21.15.5:0/656829648 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fde7c077040 msgr2=0x7fde7c1124f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:38:52.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:52.425+0000 7fde81800640 1 --2- 172.21.15.5:0/656829648 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fde7c077040 0x7fde7c1124f0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fde68009900 tx=0x7fde6802a410 comp rx=0 tx=0).stop 2024-09-10T09:38:52.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:52.425+0000 7fde81800640 1 -- 172.21.15.5:0/656829648 shutdown_connections 2024-09-10T09:38:52.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:52.425+0000 7fde81800640 1 --2- 172.21.15.5:0/656829648 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fde7c112a30 0x7fde7c114e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:52.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:52.425+0000 7fde81800640 1 --2- 172.21.15.5:0/656829648 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fde7c077040 0x7fde7c1124f0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:52.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:52.425+0000 7fde81800640 1 --2- 172.21.15.5:0/656829648 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fde7c076720 0x7fde7c076b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:52.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:52.425+0000 7fde81800640 1 -- 172.21.15.5:0/656829648 >> 172.21.15.5:0/656829648 conn(0x7fde7c0ffa90 msgr2=0x7fde7c101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:38:52.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:52.425+0000 7fde81800640 1 -- 172.21.15.5:0/656829648 shutdown_connections 2024-09-10T09:38:52.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:52.425+0000 7fde81800640 1 -- 172.21.15.5:0/656829648 wait complete. 2024-09-10T09:38:52.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:52.426+0000 7fde81800640 1 Processor -- start 2024-09-10T09:38:52.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:52.426+0000 7fde81800640 1 -- start start 2024-09-10T09:38:52.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:52.426+0000 7fde81800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fde7c076720 0x7fde7c1a4740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:52.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:52.426+0000 7fde81800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fde7c077040 0x7fde7c1a4c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:52.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:52.426+0000 7fde81800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fde7c112a30 0x7fde7c19e900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:52.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:52.426+0000 7fde81800640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fde7c118610 con 0x7fde7c077040 2024-09-10T09:38:52.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:52.426+0000 7fde81800640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fde7c118490 con 0x7fde7c112a30 2024-09-10T09:38:52.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:52.426+0000 7fde81800640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fde7c118790 con 0x7fde7c076720 2024-09-10T09:38:52.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:52.427+0000 7fde7a000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fde7c077040 0x7fde7c1a4c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:52.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:52.427+0000 7fde7aa00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fde7c076720 0x7fde7c1a4740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:52.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:52.427+0000 7fde7aa00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fde7c076720 0x7fde7c1a4740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:49420/0 (socket says 172.21.15.5:49420) 2024-09-10T09:38:52.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:52.427+0000 7fde7a000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fde7c077040 0x7fde7c1a4c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56792/0 (socket says 172.21.15.5:56792) 2024-09-10T09:38:52.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:52.427+0000 7fde7aa00640 1 -- 172.21.15.5:0/629383691 learned_addr learned my addr 172.21.15.5:0/629383691 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:38:52.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:52.427+0000 7fde7b400640 1 --2- 172.21.15.5:0/629383691 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fde7c112a30 0x7fde7c19e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:52.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:52.427+0000 7fde7aa00640 1 -- 172.21.15.5:0/629383691 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fde7c112a30 msgr2=0x7fde7c19e900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:38:52.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:52.427+0000 7fde7aa00640 1 --2- 172.21.15.5:0/629383691 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fde7c112a30 0x7fde7c19e900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:52.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:52.427+0000 7fde7aa00640 1 -- 172.21.15.5:0/629383691 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fde7c077040 msgr2=0x7fde7c1a4c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:38:52.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:52.427+0000 7fde7aa00640 1 --2- 172.21.15.5:0/629383691 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fde7c077040 0x7fde7c1a4c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:52.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:52.427+0000 7fde7aa00640 1 -- 172.21.15.5:0/629383691 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde7c19f030 con 0x7fde7c076720 2024-09-10T09:38:52.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:52.428+0000 7fde7aa00640 1 --2- 172.21.15.5:0/629383691 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fde7c076720 0x7fde7c1a4740 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fde6c00e9f0 tx=0x7fde6c00edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:38:52.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:52.428+0000 7fde73400640 1 -- 172.21.15.5:0/629383691 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde6c00ccc0 con 0x7fde7c076720 2024-09-10T09:38:52.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:52.428+0000 7fde81800640 1 -- 172.21.15.5:0/629383691 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde7c19f320 con 0x7fde7c076720 2024-09-10T09:38:52.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:52.428+0000 7fde73400640 1 -- 172.21.15.5:0/629383691 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fde6c00ce60 con 0x7fde7c076720 2024-09-10T09:38:52.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:52.428+0000 7fde73400640 1 -- 172.21.15.5:0/629383691 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde6c010690 con 0x7fde7c076720 2024-09-10T09:38:52.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:52.428+0000 7fde81800640 1 -- 172.21.15.5:0/629383691 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde7c1ab610 con 0x7fde7c076720 2024-09-10T09:38:52.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:52.431+0000 7fde73400640 1 -- 172.21.15.5:0/629383691 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 19) ==== 137883+0+0 (secure 0 0 0) 0x7fde6c0046c0 con 0x7fde7c076720 2024-09-10T09:38:52.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:52.432+0000 7fde843c6740 1 Processor -- start 2024-09-10T09:38:52.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:52.432+0000 7fde73400640 1 --2- 172.21.15.5:0/629383691 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fde600a2c10 0x7fde600a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:52.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:52.433+0000 7fde843c6740 1 -- start start 2024-09-10T09:38:52.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:52.433+0000 7fde73400640 1 -- 172.21.15.5:0/629383691 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fde6c0cfa40 con 0x7fde7c076720 2024-09-10T09:38:52.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:52.433+0000 7fde7b400640 1 --2- 172.21.15.5:0/629383691 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fde600a2c10 0x7fde600a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:52.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:52.433+0000 7fde843c6740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557dcea86330 0x557dcea86710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:52.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:52.433+0000 7fde843c6740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557dcea86c50 0x557dcea90fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:52.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:52.433+0000 7fde843c6740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557dcea91510 0x557dcea93900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:52.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:52.433+0000 7fde843c6740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x557dce9ec0b0 con 0x557dcea86c50 2024-09-10T09:38:52.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:52.433+0000 7fde843c6740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x557dce993ab0 con 0x557dcea86330 2024-09-10T09:38:52.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:52.433+0000 7fde843c6740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x557dce9e8070 con 0x557dcea91510 2024-09-10T09:38:52.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:52.434+0000 7fde7a000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557dcea86c50 0x557dcea90fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:52.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:52.434+0000 7fde7aa00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557dcea86330 0x557dcea86710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:52.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:52.434+0000 7fde7b400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557dcea91510 0x557dcea93900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:52.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:52.434+0000 7fde7aa00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557dcea86330 0x557dcea86710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:34384/0 (socket says 172.21.15.5:34384) 2024-09-10T09:38:52.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:52.434+0000 7fde7aa00640 1 -- 172.21.15.5:0/2035700120 learned_addr learned my addr 172.21.15.5:0/2035700120 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:38:52.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:52.434+0000 7fde7b400640 1 --2- 172.21.15.5:0/629383691 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fde600a2c10 0x7fde600a50d0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fde740099a0 tx=0x7fde74009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:38:52.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:52.434+0000 7fde7aa00640 1 -- 172.21.15.5:0/2035700120 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557dcea91510 msgr2=0x557dcea93900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:38:52.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:52.434+0000 7fde7aa00640 1 --2- 172.21.15.5:0/2035700120 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557dcea91510 0x557dcea93900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:52.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:52.434+0000 7fde7aa00640 1 -- 172.21.15.5:0/2035700120 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557dcea86c50 msgr2=0x557dcea90fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:38:52.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:52.434+0000 7fde7aa00640 1 --2- 172.21.15.5:0/2035700120 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557dcea86c50 0x557dcea90fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:52.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:52.434+0000 7fde7aa00640 1 -- 172.21.15.5:0/2035700120 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557dce9a04a0 con 0x557dcea86330 2024-09-10T09:38:52.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:52.434+0000 7fde7a000640 1 --2- 172.21.15.5:0/2035700120 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557dcea86c50 0x557dcea90fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:38:52.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:52.434+0000 7fde7aa00640 1 --2- 172.21.15.5:0/2035700120 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557dcea86330 0x557dcea86710 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fde6c09b9f0 tx=0x7fde6c09c4b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eee30519393f0b66 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:38:52.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:52.435+0000 7fde67e00640 1 -- 172.21.15.5:0/2035700120 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde6c0a4960 con 0x557dcea86330 2024-09-10T09:38:52.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:52.435+0000 7fde67e00640 1 -- 172.21.15.5:0/2035700120 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fde6c0cd1f0 con 0x557dcea86330 2024-09-10T09:38:52.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:52.435+0000 7fde67e00640 1 -- 172.21.15.5:0/2035700120 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde6c09abb0 con 0x557dcea86330 2024-09-10T09:38:52.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:52.435+0000 7fde843c6740 1 -- 172.21.15.5:0/2035700120 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557dcea86330 msgr2=0x557dcea86710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:38:52.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:52.435+0000 7fde843c6740 1 --2- 172.21.15.5:0/2035700120 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557dcea86330 0x557dcea86710 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fde6c09b9f0 tx=0x7fde6c09c4b0 comp rx=0 tx=0).stop 2024-09-10T09:38:52.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:52.436+0000 7fde843c6740 1 -- 172.21.15.5:0/2035700120 shutdown_connections 2024-09-10T09:38:52.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:52.436+0000 7fde843c6740 1 --2- 172.21.15.5:0/2035700120 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557dcea91510 0x557dcea93900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:52.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:52.436+0000 7fde843c6740 1 --2- 172.21.15.5:0/2035700120 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557dcea86c50 0x557dcea90fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:52.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:52.436+0000 7fde843c6740 1 --2- 172.21.15.5:0/2035700120 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557dcea86330 0x557dcea86710 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:52.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:52.436+0000 7fde843c6740 1 -- 172.21.15.5:0/2035700120 >> 172.21.15.5:0/2035700120 conn(0x557dcea828c0 msgr2=0x557dcea82ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:38:52.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:52.436+0000 7fde843c6740 1 -- 172.21.15.5:0/2035700120 shutdown_connections 2024-09-10T09:38:52.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:52.436+0000 7fde843c6740 1 -- 172.21.15.5:0/2035700120 wait complete. 2024-09-10T09:38:52.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:52.436+0000 7fde843c6740 1 -- start start 2024-09-10T09:38:52.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:52.436+0000 7fde843c6740 1 Processor -- start 2024-09-10T09:38:52.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:52.437+0000 7fde843c6740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557dcea86330 0x557dcea8e590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:52.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:52.437+0000 7fde843c6740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557dcea86c50 0x557dcea8ead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:52.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:52.437+0000 7fde843c6740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557dcea91510 0x557dcea8f010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:52.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:52.437+0000 7fde843c6740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x557dcea94060 con 0x557dcea86c50 2024-09-10T09:38:52.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:52.437+0000 7fde843c6740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x557dce974920 con 0x557dcea91510 2024-09-10T09:38:52.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:52.437+0000 7fde843c6740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x557dcea941e0 con 0x557dcea86330 2024-09-10T09:38:52.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:52.437+0000 7fde843c6740 10 client.0 _ll_register_callbacks cb 0x557dce9988a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:38:52.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:52.437+0000 7fde7aa00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557dcea86330 0x557dcea8e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:52.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:52.437+0000 7fde7a000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557dcea86c50 0x557dcea8ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:52.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:52.437+0000 7fde7aa00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557dcea86330 0x557dcea8e590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:49436/0 (socket says 172.21.15.5:49436) 2024-09-10T09:38:52.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:52.437+0000 7fde7aa00640 1 -- 172.21.15.5:0/1141249847 learned_addr learned my addr 172.21.15.5:0/1141249847 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:38:52.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:52.437+0000 7fde7b400640 1 --2- 172.21.15.5:0/1141249847 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557dcea91510 0x557dcea8f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:52.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:52.438+0000 7fde7aa00640 1 -- 172.21.15.5:0/1141249847 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557dcea91510 msgr2=0x557dcea8f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:38:52.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:52.438+0000 7fde7aa00640 1 --2- 172.21.15.5:0/1141249847 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557dcea91510 0x557dcea8f010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:52.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:52.438+0000 7fde7aa00640 1 -- 172.21.15.5:0/1141249847 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557dcea86c50 msgr2=0x557dcea8ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:38:52.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:52.438+0000 7fde7aa00640 1 --2- 172.21.15.5:0/1141249847 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557dcea86c50 0x557dcea8ead0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:52.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:52.438+0000 7fde7aa00640 1 -- 172.21.15.5:0/1141249847 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557dce75d2b0 con 0x557dcea86330 2024-09-10T09:38:52.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:52.438+0000 7fde7b400640 1 --2- 172.21.15.5:0/1141249847 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557dcea91510 0x557dcea8f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:38:52.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:52.438+0000 7fde7aa00640 1 --2- 172.21.15.5:0/1141249847 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557dcea86330 0x557dcea8e590 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fde6c09b9a0 tx=0x7fde6c09b830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dcbd6ce03d7bcd4f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:38:52.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:52.438+0000 7fde64c00640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-09-10T09:38:52.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:52.438+0000 7fde64c00640 1 -- 172.21.15.5:0/1141249847 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde6c09b3e0 con 0x557dcea86330 2024-09-10T09:38:52.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:52.438+0000 7fde843c6740 1 -- 172.21.15.5:0/1141249847 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fde6c0cd1f0 con 0x557dcea86330 2024-09-10T09:38:52.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:52.438+0000 7fde64c00640 1 -- 172.21.15.5:0/1141249847 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fde6c0a4bb0 con 0x557dcea86330 2024-09-10T09:38:52.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:52.439+0000 7fde64c00640 1 -- 172.21.15.5:0/1141249847 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fde6c0a4e30 con 0x557dcea86330 2024-09-10T09:38:52.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:52.439+0000 7fde843c6740 10 client.25471 fetch_fsmap learned FSMap version 30 2024-09-10T09:38:52.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:52.439+0000 7fde843c6740 1 -- 172.21.15.5:0/1141249847 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x557dcea94060 con 0x557dcea86330 2024-09-10T09:38:52.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:52.439+0000 7fde64c00640 1 -- 172.21.15.5:0/1141249847 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fde6c0cd1f0 con 0x557dcea86330 2024-09-10T09:38:52.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:52.439+0000 7fde64c00640 1 -- 172.21.15.5:0/1141249847 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fde6c0ba2c0 con 0x557dcea86330 2024-09-10T09:38:52.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:52.439+0000 7fde843c6740 10 client.25471 fetch_fsmap finished waiting for FSMap version 30 2024-09-10T09:38:52.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:52.439+0000 7fde843c6740 10 client.25471 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-10T09:38:52.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:52.439+0000 7fde843c6740 20 client.25471 populate_metadata read hostname 'smithi005' 2024-09-10T09:38:52.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:52.439+0000 7fde843c6740 1 --2- 172.21.15.5:0/1141249847 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x557dceaa1e70 0x557dceaa32e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:52.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:52.439+0000 7fde843c6740 4 client.25471 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:38:52.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:52.439+0000 7fde843c6740 1 -- 172.21.15.5:0/1141249847 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557dce9f9620 con 0x557dceaa1e70 2024-09-10T09:38:52.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:52.440+0000 7fde7a000640 1 --2- 172.21.15.5:0/1141249847 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x557dceaa1e70 0x557dceaa32e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:52.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:52.440+0000 7fde7a000640 1 --2- 172.21.15.5:0/1141249847 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x557dceaa1e70 0x557dceaa32e0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=230a8ac806111d0e server_cookie=ae0a98e6bb50e5d7 in_seq=0 out_seq=0 2024-09-10T09:38:52.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:52.440+0000 7fde64c00640 10 client.25471 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T09:38:52.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:52.441+0000 7fde64c00640 1 -- 172.21.15.5:0/1141249847 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557dce9f9620 con 0x557dceaa1e70 2024-09-10T09:38:52.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:52.441+0000 7fde64c00640 10 client.25471 handle_command_reply: tid=0 2024-09-10T09:38:52.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:52.441+0000 7fde843c6740 1 client.25471 shutdown 2024-09-10T09:38:52.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:52.442+0000 7fde843c6740 1 -- 172.21.15.5:0/1141249847 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557dcea86330 msgr2=0x557dcea8e590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:38:52.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:52.442+0000 7fde843c6740 1 --2- 172.21.15.5:0/1141249847 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557dcea86330 0x557dcea8e590 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fde6c09b9a0 tx=0x7fde6c09b830 comp rx=0 tx=0).stop 2024-09-10T09:38:52.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:52.442+0000 7fde843c6740 1 -- 172.21.15.5:0/1141249847 shutdown_connections 2024-09-10T09:38:52.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:52.442+0000 7fde843c6740 1 --2- 172.21.15.5:0/1141249847 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x557dceaa1e70 0x557dceaa32e0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:52.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:52.442+0000 7fde843c6740 1 --2- 172.21.15.5:0/1141249847 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557dcea91510 0x557dcea8f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:52.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:52.442+0000 7fde843c6740 1 --2- 172.21.15.5:0/1141249847 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557dcea86c50 0x557dcea8ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:52.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:52.442+0000 7fde843c6740 1 --2- 172.21.15.5:0/1141249847 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557dcea86330 0x557dcea8e590 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:52.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:52.442+0000 7fde64c00640 0 client.25471 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T09:38:52.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:52.442+0000 7fde843c6740 1 -- 172.21.15.5:0/1141249847 >> 172.21.15.5:0/1141249847 conn(0x557dcea828c0 msgr2=0x557dcea885f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:38:52.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:52.442+0000 7fde843c6740 1 -- 172.21.15.5:0/1141249847 shutdown_connections 2024-09-10T09:38:52.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:52.442+0000 7fde843c6740 1 -- 172.21.15.5:0/1141249847 wait complete. 2024-09-10T09:38:52.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:52.442+0000 7fde843c6740 20 client.25471 trim_cache size 0 max 16384 2024-09-10T09:38:52.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:52.479+0000 7fde843c6740 1 Processor -- start 2024-09-10T09:38:52.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:52.479+0000 7fde843c6740 1 -- start start 2024-09-10T09:38:52.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:52.479+0000 7fde843c6740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557dcea86330 0x557dceaa2540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:52.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:52.480+0000 7fde843c6740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557dcea86c50 0x557dcea88d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:52.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:52.480+0000 7fde843c6740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557dcea91510 0x557dcea81910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:52.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:52.480+0000 7fde843c6740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x557dceaa1620 con 0x557dcea86330 2024-09-10T09:38:52.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:52.480+0000 7fde843c6740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x557dceaa1cf0 con 0x557dcea91510 2024-09-10T09:38:52.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:52.480+0000 7fde843c6740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x557dceaa23c0 con 0x557dcea86c50 2024-09-10T09:38:52.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:52.480+0000 7fde7aa00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557dcea86330 0x557dceaa2540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:52.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:52.480+0000 7fde7aa00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557dcea86330 0x557dceaa2540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56806/0 (socket says 172.21.15.5:56806) 2024-09-10T09:38:52.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:52.480+0000 7fde7aa00640 1 -- 172.21.15.5:0/1115434390 learned_addr learned my addr 172.21.15.5:0/1115434390 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:38:52.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:52.480+0000 7fde7a000640 1 --2- 172.21.15.5:0/1115434390 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557dcea86c50 0x557dcea88d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:52.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:52.480+0000 7fde7b400640 1 --2- 172.21.15.5:0/1115434390 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557dcea91510 0x557dcea81910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:52.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:52.480+0000 7fde7aa00640 1 -- 172.21.15.5:0/1115434390 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557dcea86c50 msgr2=0x557dcea88d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:38:52.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:52.480+0000 7fde7aa00640 1 --2- 172.21.15.5:0/1115434390 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557dcea86c50 0x557dcea88d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:52.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:52.481+0000 7fde7aa00640 1 -- 172.21.15.5:0/1115434390 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557dcea91510 msgr2=0x557dcea81910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:38:52.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:52.481+0000 7fde7aa00640 1 --2- 172.21.15.5:0/1115434390 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557dcea91510 0x557dcea81910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:52.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:52.481+0000 7fde7aa00640 1 -- 172.21.15.5:0/1115434390 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557dcea8d540 con 0x557dcea86330 2024-09-10T09:38:52.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:52.481+0000 7fde7a000640 1 --2- 172.21.15.5:0/1115434390 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557dcea86c50 0x557dcea88d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:38:52.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:52.481+0000 7fde7aa00640 1 --2- 172.21.15.5:0/1115434390 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557dcea86330 0x557dceaa2540 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fde6c099fa0 tx=0x7fde6c004b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a864c61e3e99dc8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:38:52.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:52.481+0000 7fde67400640 1 -- 172.21.15.5:0/1115434390 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde6c0a15d0 con 0x557dcea86330 2024-09-10T09:38:52.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:52.481+0000 7fde67400640 1 -- 172.21.15.5:0/1115434390 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fde6c0cd1f0 con 0x557dcea86330 2024-09-10T09:38:52.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:52.481+0000 7fde67400640 1 -- 172.21.15.5:0/1115434390 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde6c0ba2c0 con 0x557dcea86330 2024-09-10T09:38:52.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:52.482+0000 7fde843c6740 1 -- 172.21.15.5:0/1115434390 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557dcea86330 msgr2=0x557dceaa2540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:38:52.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:52.482+0000 7fde843c6740 1 --2- 172.21.15.5:0/1115434390 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557dcea86330 0x557dceaa2540 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fde6c099fa0 tx=0x7fde6c004b00 comp rx=0 tx=0).stop 2024-09-10T09:38:52.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:52.482+0000 7fde843c6740 1 -- 172.21.15.5:0/1115434390 shutdown_connections 2024-09-10T09:38:52.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:52.482+0000 7fde843c6740 1 --2- 172.21.15.5:0/1115434390 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557dcea91510 0x557dcea81910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:52.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:52.482+0000 7fde843c6740 1 --2- 172.21.15.5:0/1115434390 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557dcea86c50 0x557dcea88d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:52.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:52.482+0000 7fde843c6740 1 --2- 172.21.15.5:0/1115434390 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557dcea86330 0x557dceaa2540 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:52.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:52.482+0000 7fde843c6740 1 -- 172.21.15.5:0/1115434390 >> 172.21.15.5:0/1115434390 conn(0x557dcea828c0 msgr2=0x557dcea81200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:38:52.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:52.482+0000 7fde843c6740 1 -- 172.21.15.5:0/1115434390 shutdown_connections 2024-09-10T09:38:52.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:52.482+0000 7fde843c6740 1 -- 172.21.15.5:0/1115434390 wait complete. 2024-09-10T09:38:52.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:52.483+0000 7fde843c6740 1 -- start start 2024-09-10T09:38:52.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:52.483+0000 7fde843c6740 1 Processor -- start 2024-09-10T09:38:52.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:52.483+0000 7fde843c6740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557dcea86330 0x557dceaabe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:52.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:52.483+0000 7fde843c6740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557dcea86c50 0x557dcea8b530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:52.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:52.483+0000 7fde843c6740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557dcea91510 0x557dcea8bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:52.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:52.483+0000 7fde843c6740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x557dceaa3160 con 0x557dcea91510 2024-09-10T09:38:52.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:52.483+0000 7fde843c6740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x557dceaa2a90 con 0x557dcea86c50 2024-09-10T09:38:52.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:52.483+0000 7fde843c6740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x557dceaa3830 con 0x557dcea86330 2024-09-10T09:38:52.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:52.483+0000 7fde843c6740 10 client.0 _ll_register_callbacks cb 0x7fde74009ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:38:52.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:52.484+0000 7fde7b400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557dcea91510 0x557dcea8bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:52.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:52.484+0000 7fde7b400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557dcea91510 0x557dcea8bbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56812/0 (socket says 172.21.15.5:56812) 2024-09-10T09:38:52.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:52.484+0000 7fde7b400640 1 -- 172.21.15.5:0/1741591987 learned_addr learned my addr 172.21.15.5:0/1741591987 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:38:52.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:52.484+0000 7fde7aa00640 1 --2- 172.21.15.5:0/1741591987 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557dcea86330 0x557dceaabe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:52.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:52.484+0000 7fde7a000640 1 --2- 172.21.15.5:0/1741591987 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557dcea86c50 0x557dcea8b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:52.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:52.484+0000 7fde7aa00640 1 -- 172.21.15.5:0/1741591987 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557dcea86c50 msgr2=0x557dcea8b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:38:52.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:52.484+0000 7fde7aa00640 1 --2- 172.21.15.5:0/1741591987 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557dcea86c50 0x557dcea8b530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:52.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:52.484+0000 7fde7aa00640 1 -- 172.21.15.5:0/1741591987 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557dcea91510 msgr2=0x557dcea8bbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:38:52.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:52.484+0000 7fde7aa00640 1 --2- 172.21.15.5:0/1741591987 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557dcea91510 0x557dcea8bbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:52.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:52.484+0000 7fde7aa00640 1 -- 172.21.15.5:0/1741591987 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557dceaa9920 con 0x557dcea86330 2024-09-10T09:38:52.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:52.485+0000 7fde7aa00640 1 --2- 172.21.15.5:0/1741591987 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557dcea86330 0x557dceaabe00 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fde6c0057a0 tx=0x7fde6c09c4b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=64230fd8e3400d0a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:38:52.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:52.485+0000 7fde64c00640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-09-10T09:38:52.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:52.485+0000 7fde64c00640 1 -- 172.21.15.5:0/1741591987 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde6c0a4bb0 con 0x557dcea86330 2024-09-10T09:38:52.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:52.485+0000 7fde843c6740 1 -- 172.21.15.5:0/1741591987 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fde6c0cd1f0 con 0x557dcea86330 2024-09-10T09:38:52.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:52.485+0000 7fde64c00640 1 -- 172.21.15.5:0/1741591987 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fde6c0ba2c0 con 0x557dcea86330 2024-09-10T09:38:52.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:52.485+0000 7fde64c00640 1 -- 172.21.15.5:0/1741591987 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fde6c097a40 con 0x557dcea86330 2024-09-10T09:38:52.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:52.485+0000 7fde843c6740 10 client.25477 fetch_fsmap learned FSMap version 30 2024-09-10T09:38:52.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:52.485+0000 7fde843c6740 1 -- 172.21.15.5:0/1741591987 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x557dceaa9920 con 0x557dcea86330 2024-09-10T09:38:52.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:52.485+0000 7fde64c00640 1 -- 172.21.15.5:0/1741591987 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fde6c0cd1f0 con 0x557dcea86330 2024-09-10T09:38:52.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:52.486+0000 7fde64c00640 1 -- 172.21.15.5:0/1741591987 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fde6c086370 con 0x557dcea86330 2024-09-10T09:38:52.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:52.486+0000 7fde843c6740 10 client.25477 fetch_fsmap finished waiting for FSMap version 30 2024-09-10T09:38:52.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:52.486+0000 7fde843c6740 10 client.25477 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-10T09:38:52.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:52.486+0000 7fde843c6740 20 client.25477 populate_metadata read hostname 'smithi005' 2024-09-10T09:38:52.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:52.486+0000 7fde843c6740 1 --2- 172.21.15.5:0/1741591987 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x557dceb10a30 0x557dceb0f740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:38:52.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:52.486+0000 7fde843c6740 4 client.25477 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T09:38:52.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:52.486+0000 7fde843c6740 1 -- 172.21.15.5:0/1741591987 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "status"}) -- 0x557dce9f3b30 con 0x557dceb10a30 2024-09-10T09:38:52.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:52.486+0000 7fde7a000640 1 --2- 172.21.15.5:0/1741591987 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x557dceb10a30 0x557dceb0f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:38:52.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:52.487+0000 7fde7a000640 1 --2- 172.21.15.5:0/1741591987 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x557dceb10a30 0x557dceb0f740 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=81fc12f7909472eb server_cookie=68dc5da0139da40e in_seq=0 out_seq=0 2024-09-10T09:38:52.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:52.487+0000 7fde64c00640 10 client.25477 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T09:38:52.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:52.487+0000 7fde64c00640 1 -- 172.21.15.5:0/1741591987 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+322 (crc 0 0 0) 0x557dce9f3b30 con 0x557dceb10a30 2024-09-10T09:38:52.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:52.487+0000 7fde64c00640 10 client.25477 handle_command_reply: tid=0 2024-09-10T09:38:52.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:52.487+0000 7fde843c6740 1 client.25477 shutdown 2024-09-10T09:38:52.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:52.487+0000 7fde843c6740 1 -- 172.21.15.5:0/1741591987 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557dcea86330 msgr2=0x557dceaabe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:38:52.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:52.487+0000 7fde843c6740 1 --2- 172.21.15.5:0/1741591987 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557dcea86330 0x557dceaabe00 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fde6c0057a0 tx=0x7fde6c09c4b0 comp rx=0 tx=0).stop 2024-09-10T09:38:52.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:52.487+0000 7fde843c6740 1 -- 172.21.15.5:0/1741591987 shutdown_connections 2024-09-10T09:38:52.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:52.487+0000 7fde843c6740 1 --2- 172.21.15.5:0/1741591987 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x557dceb10a30 0x557dceb0f740 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:52.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:52.487+0000 7fde843c6740 1 --2- 172.21.15.5:0/1741591987 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557dcea91510 0x557dcea8bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:52.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:52.487+0000 7fde843c6740 1 --2- 172.21.15.5:0/1741591987 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557dcea86c50 0x557dcea8b530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:52.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:52.487+0000 7fde843c6740 1 --2- 172.21.15.5:0/1741591987 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557dcea86330 0x557dceaabe00 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:52.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:52.487+0000 7fde843c6740 1 -- 172.21.15.5:0/1741591987 >> 172.21.15.5:0/1741591987 conn(0x557dcea828c0 msgr2=0x557dcea9cc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:38:52.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:52.487+0000 7fde64c00640 0 client.25477 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T09:38:52.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:52.488+0000 7fde843c6740 1 -- 172.21.15.5:0/1741591987 shutdown_connections 2024-09-10T09:38:52.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:52.488+0000 7fde843c6740 1 -- 172.21.15.5:0/1741591987 wait complete. 2024-09-10T09:38:52.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:52.488+0000 7fde843c6740 20 client.25477 trim_cache size 0 max 16384 2024-09-10T09:38:52.492 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-10T09:38:52.493 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_fsid": "bade4b50-6f56-11ef-bcea-c7b262605968", 2024-09-10T09:38:52.493 INFO:teuthology.orchestra.run.smithi005.stdout: "whoami": 0, 2024-09-10T09:38:52.493 INFO:teuthology.orchestra.run.smithi005.stdout: "id": 24542, 2024-09-10T09:38:52.493 INFO:teuthology.orchestra.run.smithi005.stdout: "want_state": "up:active", 2024-09-10T09:38:52.493 INFO:teuthology.orchestra.run.smithi005.stdout: "state": "up:active", 2024-09-10T09:38:52.493 INFO:teuthology.orchestra.run.smithi005.stdout: "fs_name": "cephfs", 2024-09-10T09:38:52.493 INFO:teuthology.orchestra.run.smithi005.stdout: "rank_uptime": 201.922520693, 2024-09-10T09:38:52.493 INFO:teuthology.orchestra.run.smithi005.stdout: "mdsmap_epoch": 30, 2024-09-10T09:38:52.493 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch": 96, 2024-09-10T09:38:52.493 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch_barrier": 96, 2024-09-10T09:38:52.493 INFO:teuthology.orchestra.run.smithi005.stdout: "uptime": 215.538467371 2024-09-10T09:38:52.493 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-10T09:38:52.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:52.488+0000 7fde81800640 1 -- 172.21.15.5:0/629383691 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fde600a2c10 msgr2=0x7fde600a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:38:52.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:52.488+0000 7fde81800640 1 --2- 172.21.15.5:0/629383691 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fde600a2c10 0x7fde600a50d0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fde740099a0 tx=0x7fde74009500 comp rx=0 tx=0).stop 2024-09-10T09:38:52.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:52.488+0000 7fde81800640 1 -- 172.21.15.5:0/629383691 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fde7c076720 msgr2=0x7fde7c1a4740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:38:52.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:52.488+0000 7fde81800640 1 --2- 172.21.15.5:0/629383691 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fde7c076720 0x7fde7c1a4740 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fde6c00e9f0 tx=0x7fde6c00edf0 comp rx=0 tx=0).stop 2024-09-10T09:38:52.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:52.489+0000 7fde81800640 1 -- 172.21.15.5:0/629383691 shutdown_connections 2024-09-10T09:38:52.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:52.489+0000 7fde81800640 1 --2- 172.21.15.5:0/629383691 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fde600a2c10 0x7fde600a50d0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:52.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:52.489+0000 7fde81800640 1 --2- 172.21.15.5:0/629383691 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fde7c112a30 0x7fde7c19e900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:52.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:52.489+0000 7fde81800640 1 --2- 172.21.15.5:0/629383691 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fde7c077040 0x7fde7c1a4c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:52.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:52.489+0000 7fde81800640 1 --2- 172.21.15.5:0/629383691 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fde7c076720 0x7fde7c1a4740 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:38:52.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:52.489+0000 7fde81800640 1 -- 172.21.15.5:0/629383691 >> 172.21.15.5:0/629383691 conn(0x7fde7c0ffa90 msgr2=0x7fde7c101e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:38:52.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:52.489+0000 7fde81800640 1 -- 172.21.15.5:0/629383691 shutdown_connections 2024-09-10T09:38:52.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:38:52.489+0000 7fde81800640 1 -- 172.21.15.5:0/629383691 wait complete. 2024-09-10T09:38:53.370 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:38:53 smithi005 ceph-mon[30350]: pgmap v409: 129 pgs: 129 active+clean; 660 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 938 B/s wr, 12 op/s 2024-09-10T09:38:53.373 INFO:tasks.quiescer:Ready to start quiesce thrashing; initial delay: 15 sec 2024-09-10T09:38:53.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:38:53 smithi102 ceph-mon[35442]: pgmap v409: 129 pgs: 129 active+clean; 660 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 938 B/s wr, 12 op/s 2024-09-10T09:38:53.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:38:53 smithi159 ceph-mon[35513]: pgmap v409: 129 pgs: 129 active+clean; 660 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 938 B/s wr, 12 op/s 2024-09-10T09:38:55.385 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:38:55 smithi102 ceph-mon[35442]: pgmap v410: 129 pgs: 129 active+clean; 660 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 938 B/s wr, 9 op/s 2024-09-10T09:38:55.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:38:55 smithi005 ceph-mon[30350]: pgmap v410: 129 pgs: 129 active+clean; 660 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 938 B/s wr, 9 op/s 2024-09-10T09:38:55.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:38:55 smithi159 ceph-mon[35513]: pgmap v410: 129 pgs: 129 active+clean; 660 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 938 B/s wr, 9 op/s 2024-09-10T09:38:57.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:38:57 smithi102 ceph-mon[35442]: pgmap v411: 129 pgs: 129 active+clean; 660 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 938 B/s wr, 12 op/s 2024-09-10T09:38:57.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:38:57 smithi005 ceph-mon[30350]: pgmap v411: 129 pgs: 129 active+clean; 660 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 938 B/s wr, 12 op/s 2024-09-10T09:38:57.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:38:57 smithi159 ceph-mon[35513]: pgmap v411: 129 pgs: 129 active+clean; 660 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 938 B/s wr, 12 op/s 2024-09-10T09:38:59.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:38:59 smithi102 ceph-mon[35442]: pgmap v412: 129 pgs: 129 active+clean; 660 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-09-10T09:38:59.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:38:59 smithi005 ceph-mon[30350]: pgmap v412: 129 pgs: 129 active+clean; 660 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-09-10T09:38:59.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:38:59 smithi159 ceph-mon[35513]: pgmap v412: 129 pgs: 129 active+clean; 660 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-09-10T09:39:00.393 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T09:39:00.793 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:39:01.303 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:39:01 smithi005 ceph-mon[30350]: pgmap v413: 129 pgs: 129 active+clean; 660 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-09-10T09:39:01.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:39:01 smithi102 ceph-mon[35442]: pgmap v413: 129 pgs: 129 active+clean; 660 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-09-10T09:39:01.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:01.420+0000 7f82dce00640 1 -- 172.21.15.5:0/2848306250 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f82d81049c0 msgr2=0x7f82d8108a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:39:01.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:01.420+0000 7f82dce00640 1 --2- 172.21.15.5:0/2848306250 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f82d81049c0 0x7f82d8108a10 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f82c4009900 tx=0x7f82c402a410 comp rx=0 tx=0).stop 2024-09-10T09:39:01.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:01.420+0000 7f82dce00640 1 -- 172.21.15.5:0/2848306250 shutdown_connections 2024-09-10T09:39:01.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:01.420+0000 7f82dce00640 1 --2- 172.21.15.5:0/2848306250 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f82d8109140 0x7f82d8110cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:01.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:01.420+0000 7f82dce00640 1 --2- 172.21.15.5:0/2848306250 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f82d81049c0 0x7f82d8108a10 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:01.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:01.420+0000 7f82dce00640 1 --2- 172.21.15.5:0/2848306250 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f82d8104010 0x7f82d81043f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:01.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:01.420+0000 7f82dce00640 1 -- 172.21.15.5:0/2848306250 >> 172.21.15.5:0/2848306250 conn(0x7f82d80ffa50 msgr2=0x7f82d8101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:39:01.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:01.421+0000 7f82dce00640 1 -- 172.21.15.5:0/2848306250 shutdown_connections 2024-09-10T09:39:01.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:01.421+0000 7f82dce00640 1 -- 172.21.15.5:0/2848306250 wait complete. 2024-09-10T09:39:01.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:01.422+0000 7f82dce00640 1 Processor -- start 2024-09-10T09:39:01.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:01.422+0000 7f82dce00640 1 -- start start 2024-09-10T09:39:01.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:01.423+0000 7f82dce00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f82d8104010 0x7f82d819a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:39:01.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:01.423+0000 7f82dce00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f82d81049c0 0x7f82d819abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:39:01.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:01.423+0000 7f82dce00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f82d8109140 0x7f82d819ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:39:01.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:01.423+0000 7f82dce00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f82d81141b0 con 0x7f82d8104010 2024-09-10T09:39:01.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:01.423+0000 7f82dce00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f82d8114030 con 0x7f82d8109140 2024-09-10T09:39:01.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:01.423+0000 7f82dce00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f82d8114330 con 0x7f82d81049c0 2024-09-10T09:39:01.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:01.423+0000 7f82d5400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f82d81049c0 0x7f82d819abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:39:01.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:01.423+0000 7f82d5400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f82d81049c0 0x7f82d819abb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:54264/0 (socket says 172.21.15.5:54264) 2024-09-10T09:39:01.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:01.423+0000 7f82d5400640 1 -- 172.21.15.5:0/1271395408 learned_addr learned my addr 172.21.15.5:0/1271395408 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:39:01.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:01.423+0000 7f82d5e00640 1 --2- 172.21.15.5:0/1271395408 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f82d8104010 0x7f82d819a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:39:01.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:01.423+0000 7f82d6800640 1 --2- 172.21.15.5:0/1271395408 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f82d8109140 0x7f82d819ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:39:01.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:01.424+0000 7f82d6800640 1 -- 172.21.15.5:0/1271395408 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f82d81049c0 msgr2=0x7f82d819abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:39:01.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:01.424+0000 7f82d6800640 1 --2- 172.21.15.5:0/1271395408 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f82d81049c0 0x7f82d819abb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:01.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:01.424+0000 7f82d6800640 1 -- 172.21.15.5:0/1271395408 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f82d8104010 msgr2=0x7f82d819a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:39:01.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:01.424+0000 7f82d6800640 1 --2- 172.21.15.5:0/1271395408 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f82d8104010 0x7f82d819a670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:01.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:01.424+0000 7f82d6800640 1 -- 172.21.15.5:0/1271395408 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82d819f620 con 0x7f82d8109140 2024-09-10T09:39:01.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:01.424+0000 7f82d5400640 1 --2- 172.21.15.5:0/1271395408 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f82d81049c0 0x7f82d819abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:39:01.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:01.424+0000 7f82d6800640 1 --2- 172.21.15.5:0/1271395408 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f82d8109140 0x7f82d819ef40 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f82d0009fd0 tx=0x7f82d000ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:39:01.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:01.424+0000 7f82cea00640 1 -- 172.21.15.5:0/1271395408 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82d000ce20 con 0x7f82d8109140 2024-09-10T09:39:01.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:01.424+0000 7f82cea00640 1 -- 172.21.15.5:0/1271395408 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f82d0005020 con 0x7f82d8109140 2024-09-10T09:39:01.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:01.424+0000 7f82dce00640 1 -- 172.21.15.5:0/1271395408 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82d819f910 con 0x7f82d8109140 2024-09-10T09:39:01.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:01.425+0000 7f82cea00640 1 -- 172.21.15.5:0/1271395408 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82d0010720 con 0x7f82d8109140 2024-09-10T09:39:01.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:01.425+0000 7f82dce00640 1 -- 172.21.15.5:0/1271395408 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82d81a71f0 con 0x7f82d8109140 2024-09-10T09:39:01.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:01.428+0000 7f82df9fa740 1 Processor -- start 2024-09-10T09:39:01.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:01.428+0000 7f82cea00640 1 -- 172.21.15.5:0/1271395408 <== mon.2 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 19) ==== 137883+0+0 (secure 0 0 0) 0x7f82d0005540 con 0x7f82d8109140 2024-09-10T09:39:01.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:01.428+0000 7f82df9fa740 1 -- start start 2024-09-10T09:39:01.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:01.429+0000 7f82df9fa740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bd75e08b90 0x55bd75e08f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:39:01.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:01.429+0000 7f82df9fa740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bd75e094b0 0x55bd75e13830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:39:01.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:01.429+0000 7f82df9fa740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bd75e13d70 0x55bd75e16160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:39:01.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:01.429+0000 7f82df9fa740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55bd75d6f3e0 con 0x55bd75e13d70 2024-09-10T09:39:01.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:01.429+0000 7f82df9fa740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55bd75d16d80 con 0x55bd75e08b90 2024-09-10T09:39:01.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:01.429+0000 7f82df9fa740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55bd75d6b3a0 con 0x55bd75e094b0 2024-09-10T09:39:01.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:01.429+0000 7f82d5400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bd75e08b90 0x55bd75e08f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:39:01.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:01.429+0000 7f82d5400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bd75e08b90 0x55bd75e08f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:56530/0 (socket says 172.21.15.5:56530) 2024-09-10T09:39:01.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:01.429+0000 7f82d5400640 1 -- 172.21.15.5:0/355432959 learned_addr learned my addr 172.21.15.5:0/355432959 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:39:01.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:01.429+0000 7f82d5400640 1 --2- 172.21.15.5:0/355432959 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bd75e13d70 0x55bd75e16160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:39:01.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:01.429+0000 7f82d5e00640 1 --2- 172.21.15.5:0/355432959 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bd75e094b0 0x55bd75e13830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:39:01.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:01.430+0000 7f82d5e00640 1 -- 172.21.15.5:0/355432959 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bd75e08b90 msgr2=0x55bd75e08f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:39:01.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:01.430+0000 7f82d5e00640 1 --2- 172.21.15.5:0/355432959 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bd75e08b90 0x55bd75e08f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:01.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:01.430+0000 7f82d5e00640 1 -- 172.21.15.5:0/355432959 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bd75e13d70 msgr2=0x55bd75e16160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:39:01.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:01.430+0000 7f82d5e00640 1 --2- 172.21.15.5:0/355432959 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bd75e13d70 0x55bd75e16160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:01.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:01.430+0000 7f82d5e00640 1 -- 172.21.15.5:0/355432959 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bd75d35ae0 con 0x55bd75e094b0 2024-09-10T09:39:01.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:01.430+0000 7f82cea00640 1 --2- 172.21.15.5:0/1271395408 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f82bc0a2c10 0x7f82bc0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:39:01.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:01.430+0000 7f82cea00640 1 -- 172.21.15.5:0/1271395408 <== mon.2 v2:172.21.15.102:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f82d00cfc60 con 0x7f82d8109140 2024-09-10T09:39:01.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:01.430+0000 7f82d5e00640 1 --2- 172.21.15.5:0/355432959 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bd75e094b0 0x55bd75e13830 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f82c8009c00 tx=0x7f82c8011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=893079687625e9f9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:39:01.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:01.430+0000 7f82d5400640 1 --2- 172.21.15.5:0/1271395408 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f82bc0a2c10 0x7f82bc0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:39:01.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:01.430+0000 7f82c3400640 1 -- 172.21.15.5:0/355432959 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82c8004230 con 0x55bd75e094b0 2024-09-10T09:39:01.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:01.431+0000 7f82c3400640 1 -- 172.21.15.5:0/355432959 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f82c80043d0 con 0x55bd75e094b0 2024-09-10T09:39:01.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:01.431+0000 7f82c3400640 1 -- 172.21.15.5:0/355432959 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82c8004d00 con 0x55bd75e094b0 2024-09-10T09:39:01.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:01.431+0000 7f82df9fa740 1 -- 172.21.15.5:0/355432959 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bd75e094b0 msgr2=0x55bd75e13830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:39:01.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:01.431+0000 7f82df9fa740 1 --2- 172.21.15.5:0/355432959 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bd75e094b0 0x55bd75e13830 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f82c8009c00 tx=0x7f82c8011810 comp rx=0 tx=0).stop 2024-09-10T09:39:01.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:01.431+0000 7f82d5400640 1 --2- 172.21.15.5:0/1271395408 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f82bc0a2c10 0x7f82bc0a50d0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x55bd75e064a0 tx=0x7f82c40046c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:39:01.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:01.432+0000 7f82df9fa740 1 -- 172.21.15.5:0/355432959 shutdown_connections 2024-09-10T09:39:01.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:01.432+0000 7f82df9fa740 1 --2- 172.21.15.5:0/355432959 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bd75e13d70 0x55bd75e16160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:01.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:01.432+0000 7f82df9fa740 1 --2- 172.21.15.5:0/355432959 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bd75e094b0 0x55bd75e13830 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:01.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:01.432+0000 7f82df9fa740 1 --2- 172.21.15.5:0/355432959 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bd75e08b90 0x55bd75e08f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:01.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:01.432+0000 7f82df9fa740 1 -- 172.21.15.5:0/355432959 >> 172.21.15.5:0/355432959 conn(0x55bd75e05120 msgr2=0x55bd75e05500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:39:01.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:01.432+0000 7f82df9fa740 1 -- 172.21.15.5:0/355432959 shutdown_connections 2024-09-10T09:39:01.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:01.432+0000 7f82df9fa740 1 -- 172.21.15.5:0/355432959 wait complete. 2024-09-10T09:39:01.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:01.432+0000 7f82df9fa740 1 -- start start 2024-09-10T09:39:01.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:01.432+0000 7f82df9fa740 1 Processor -- start 2024-09-10T09:39:01.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:01.433+0000 7f82df9fa740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bd75e08b90 0x55bd75e10bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:39:01.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:01.433+0000 7f82df9fa740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bd75e094b0 0x55bd75e11110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:39:01.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:01.433+0000 7f82df9fa740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bd75e13d70 0x55bd75e11650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:39:01.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:01.433+0000 7f82df9fa740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55bd75e169e0 con 0x55bd75e094b0 2024-09-10T09:39:01.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:01.433+0000 7f82df9fa740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55bd75d2a0f0 con 0x55bd75e08b90 2024-09-10T09:39:01.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:01.433+0000 7f82df9fa740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55bd75e16b60 con 0x55bd75e13d70 2024-09-10T09:39:01.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:01.433+0000 7f82df9fa740 10 client.0 _ll_register_callbacks cb 0x55bd75d1b9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:39:01.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:01.433+0000 7f82d5e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bd75e08b90 0x55bd75e10bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:39:01.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:01.433+0000 7f82d5e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bd75e08b90 0x55bd75e10bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:56536/0 (socket says 172.21.15.5:56536) 2024-09-10T09:39:01.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:01.433+0000 7f82d5e00640 1 -- 172.21.15.5:0/4279330168 learned_addr learned my addr 172.21.15.5:0/4279330168 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:39:01.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:01.433+0000 7f82d5400640 1 --2- 172.21.15.5:0/4279330168 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bd75e094b0 0x55bd75e11110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:39:01.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:01.434+0000 7f82d6800640 1 --2- 172.21.15.5:0/4279330168 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bd75e13d70 0x55bd75e11650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:39:01.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:01.434+0000 7f82d5e00640 1 -- 172.21.15.5:0/4279330168 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bd75e13d70 msgr2=0x55bd75e11650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:39:01.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:01.434+0000 7f82d5e00640 1 --2- 172.21.15.5:0/4279330168 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bd75e13d70 0x55bd75e11650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:01.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:01.434+0000 7f82d5e00640 1 -- 172.21.15.5:0/4279330168 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bd75e094b0 msgr2=0x55bd75e11110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:39:01.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:01.434+0000 7f82d5e00640 1 --2- 172.21.15.5:0/4279330168 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bd75e094b0 0x55bd75e11110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:01.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:01.434+0000 7f82d5e00640 1 -- 172.21.15.5:0/4279330168 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bd75ae03c0 con 0x55bd75e08b90 2024-09-10T09:39:01.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:01.434+0000 7f82d6800640 1 --2- 172.21.15.5:0/4279330168 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bd75e13d70 0x55bd75e11650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:39:01.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:01.434+0000 7f82d5e00640 1 --2- 172.21.15.5:0/4279330168 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bd75e08b90 0x55bd75e10bd0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f82c8005890 tx=0x7f82c8019af0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=551b522994f25a64 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:39:01.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:01.434+0000 7f82b7e00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-09-10T09:39:01.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:01.435+0000 7f82b7e00640 1 -- 172.21.15.5:0/4279330168 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82c8028020 con 0x55bd75e08b90 2024-09-10T09:39:01.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:01.435+0000 7f82b7e00640 1 -- 172.21.15.5:0/4279330168 <== mon.2 v2:172.21.15.102:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f82c8024620 con 0x55bd75e08b90 2024-09-10T09:39:01.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:01.435+0000 7f82df9fa740 1 -- 172.21.15.5:0/4279330168 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f82c80043d0 con 0x55bd75e08b90 2024-09-10T09:39:01.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:01.435+0000 7f82b7e00640 1 -- 172.21.15.5:0/4279330168 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f82c8033050 con 0x55bd75e08b90 2024-09-10T09:39:01.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:01.435+0000 7f82df9fa740 10 client.25472 fetch_fsmap learned FSMap version 30 2024-09-10T09:39:01.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:01.435+0000 7f82df9fa740 1 -- 172.21.15.5:0/4279330168 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55bd75e22610 con 0x55bd75e08b90 2024-09-10T09:39:01.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:01.435+0000 7f82b7e00640 1 -- 172.21.15.5:0/4279330168 <== mon.2 v2:172.21.15.102:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f82c80043d0 con 0x55bd75e08b90 2024-09-10T09:39:01.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:01.435+0000 7f82b7e00640 1 -- 172.21.15.5:0/4279330168 <== mon.2 v2:172.21.15.102:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f82c8018070 con 0x55bd75e08b90 2024-09-10T09:39:01.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:01.436+0000 7f82df9fa740 10 client.25472 fetch_fsmap finished waiting for FSMap version 30 2024-09-10T09:39:01.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:01.436+0000 7f82df9fa740 10 client.25472 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T09:39:01.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:01.436+0000 7f82df9fa740 20 client.25472 populate_metadata read hostname 'smithi005' 2024-09-10T09:39:01.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:01.436+0000 7f82df9fa740 1 --2- 172.21.15.5:0/4279330168 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55bd75e23a10 0x55bd75e25e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:39:01.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:01.436+0000 7f82df9fa740 4 client.25472 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:39:01.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:01.436+0000 7f82df9fa740 1 -- 172.21.15.5:0/4279330168 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bd75d7c950 con 0x55bd75e23a10 2024-09-10T09:39:01.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:01.436+0000 7f82d5400640 1 --2- 172.21.15.5:0/4279330168 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55bd75e23a10 0x55bd75e25e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:39:01.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:01.436+0000 7f82d5400640 1 --2- 172.21.15.5:0/4279330168 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55bd75e23a10 0x55bd75e25e00 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=167d572aa3d64a81 server_cookie=a019cb9bfaa81edc in_seq=0 out_seq=0 2024-09-10T09:39:01.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:01.436+0000 7f82b7e00640 10 client.25472 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T09:39:01.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:01.437+0000 7f82b7e00640 1 -- 172.21.15.5:0/4279330168 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55bd75d7c950 con 0x55bd75e23a10 2024-09-10T09:39:01.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:01.437+0000 7f82b7e00640 10 client.25472 handle_command_reply: tid=0 2024-09-10T09:39:01.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:01.437+0000 7f82df9fa740 1 client.25472 shutdown 2024-09-10T09:39:01.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:01.437+0000 7f82df9fa740 1 -- 172.21.15.5:0/4279330168 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bd75e08b90 msgr2=0x55bd75e10bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:39:01.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:01.437+0000 7f82df9fa740 1 --2- 172.21.15.5:0/4279330168 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bd75e08b90 0x55bd75e10bd0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f82c8005890 tx=0x7f82c8019af0 comp rx=0 tx=0).stop 2024-09-10T09:39:01.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:01.438+0000 7f82df9fa740 1 -- 172.21.15.5:0/4279330168 shutdown_connections 2024-09-10T09:39:01.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:01.438+0000 7f82df9fa740 1 --2- 172.21.15.5:0/4279330168 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55bd75e23a10 0x55bd75e25e00 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:01.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:01.438+0000 7f82df9fa740 1 --2- 172.21.15.5:0/4279330168 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bd75e13d70 0x55bd75e11650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:01.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:01.438+0000 7f82df9fa740 1 --2- 172.21.15.5:0/4279330168 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bd75e094b0 0x55bd75e11110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:01.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:01.438+0000 7f82df9fa740 1 --2- 172.21.15.5:0/4279330168 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bd75e08b90 0x55bd75e10bd0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:01.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:01.438+0000 7f82df9fa740 1 -- 172.21.15.5:0/4279330168 >> 172.21.15.5:0/4279330168 conn(0x55bd75e05120 msgr2=0x55bd75e0ae00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:39:01.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:01.438+0000 7f82b7e00640 0 client.25472 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T09:39:01.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:01.438+0000 7f82df9fa740 1 -- 172.21.15.5:0/4279330168 shutdown_connections 2024-09-10T09:39:01.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:01.438+0000 7f82df9fa740 1 -- 172.21.15.5:0/4279330168 wait complete. 2024-09-10T09:39:01.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:01.438+0000 7f82df9fa740 20 client.25472 trim_cache size 0 max 16384 2024-09-10T09:39:01.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:01.458+0000 7f82df9fa740 1 Processor -- start 2024-09-10T09:39:01.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:01.458+0000 7f82df9fa740 1 -- start start 2024-09-10T09:39:01.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:01.458+0000 7f82df9fa740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bd75e08b90 0x55bd75e2b600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:39:01.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:01.458+0000 7f82df9fa740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bd75e094b0 0x55bd75e2bb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:39:01.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:01.458+0000 7f82df9fa740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bd75e13d70 0x55bd75e12430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:39:01.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:01.458+0000 7f82df9fa740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f82d81b1090 con 0x55bd75e08b90 2024-09-10T09:39:01.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:01.458+0000 7f82df9fa740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f82d81b2bd0 con 0x55bd75e13d70 2024-09-10T09:39:01.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:01.458+0000 7f82df9fa740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f82d81b1760 con 0x55bd75e094b0 2024-09-10T09:39:01.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:01.459+0000 7f82d5e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bd75e08b90 0x55bd75e2b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:39:01.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:01.459+0000 7f82d5e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bd75e08b90 0x55bd75e2b600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36258/0 (socket says 172.21.15.5:36258) 2024-09-10T09:39:01.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:01.459+0000 7f82d5e00640 1 -- 172.21.15.5:0/3692543129 learned_addr learned my addr 172.21.15.5:0/3692543129 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:39:01.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:01.459+0000 7f82d5400640 1 --2- 172.21.15.5:0/3692543129 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bd75e094b0 0x55bd75e2bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:39:01.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:01.459+0000 7f82d6800640 1 --2- 172.21.15.5:0/3692543129 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bd75e13d70 0x55bd75e12430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:39:01.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:01.459+0000 7f82d5e00640 1 -- 172.21.15.5:0/3692543129 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bd75e094b0 msgr2=0x55bd75e2bb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:39:01.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:01.459+0000 7f82d5e00640 1 --2- 172.21.15.5:0/3692543129 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bd75e094b0 0x55bd75e2bb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:01.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:01.459+0000 7f82d5e00640 1 -- 172.21.15.5:0/3692543129 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bd75e13d70 msgr2=0x55bd75e12430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:39:01.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:01.459+0000 7f82d5e00640 1 --2- 172.21.15.5:0/3692543129 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bd75e13d70 0x55bd75e12430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:01.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:01.459+0000 7f82d5e00640 1 -- 172.21.15.5:0/3692543129 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bd75e0b950 con 0x55bd75e08b90 2024-09-10T09:39:01.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:01.459+0000 7f82d6800640 1 --2- 172.21.15.5:0/3692543129 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bd75e13d70 0x55bd75e12430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:39:01.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:01.459+0000 7f82d5e00640 1 --2- 172.21.15.5:0/3692543129 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bd75e08b90 0x55bd75e2b600 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f82c8004b30 tx=0x7f82c8019ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=beaf335ec5f1473f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:39:01.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:01.460+0000 7f82c2a00640 1 -- 172.21.15.5:0/3692543129 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82c8028050 con 0x55bd75e08b90 2024-09-10T09:39:01.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:01.460+0000 7f82c2a00640 1 -- 172.21.15.5:0/3692543129 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f82c801f040 con 0x55bd75e08b90 2024-09-10T09:39:01.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:01.460+0000 7f82c2a00640 1 -- 172.21.15.5:0/3692543129 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82c8003870 con 0x55bd75e08b90 2024-09-10T09:39:01.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:01.460+0000 7f82df9fa740 1 -- 172.21.15.5:0/3692543129 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bd75e08b90 msgr2=0x55bd75e2b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:39:01.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:01.460+0000 7f82df9fa740 1 --2- 172.21.15.5:0/3692543129 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bd75e08b90 0x55bd75e2b600 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f82c8004b30 tx=0x7f82c8019ef0 comp rx=0 tx=0).stop 2024-09-10T09:39:01.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:01.461+0000 7f82df9fa740 1 -- 172.21.15.5:0/3692543129 shutdown_connections 2024-09-10T09:39:01.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:01.461+0000 7f82df9fa740 1 --2- 172.21.15.5:0/3692543129 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bd75e13d70 0x55bd75e12430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:01.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:01.461+0000 7f82df9fa740 1 --2- 172.21.15.5:0/3692543129 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bd75e094b0 0x55bd75e2bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:01.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:01.461+0000 7f82df9fa740 1 --2- 172.21.15.5:0/3692543129 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bd75e08b90 0x55bd75e2b600 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:01.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:01.461+0000 7f82df9fa740 1 -- 172.21.15.5:0/3692543129 >> 172.21.15.5:0/3692543129 conn(0x55bd75e05120 msgr2=0x55bd75e0fcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:39:01.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:01.461+0000 7f82df9fa740 1 -- 172.21.15.5:0/3692543129 shutdown_connections 2024-09-10T09:39:01.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:01.461+0000 7f82df9fa740 1 -- 172.21.15.5:0/3692543129 wait complete. 2024-09-10T09:39:01.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:01.461+0000 7f82df9fa740 1 -- start start 2024-09-10T09:39:01.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:01.462+0000 7f82df9fa740 1 Processor -- start 2024-09-10T09:39:01.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:01.462+0000 7f82df9fa740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bd75e08b90 0x55bd75e148d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:39:01.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:01.462+0000 7f82df9fa740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bd75e094b0 0x55bd75e14e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:39:01.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:01.462+0000 7f82df9fa740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bd75e13d70 0x55bd75e1adb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:39:01.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:01.462+0000 7f82df9fa740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f82d81b2500 con 0x55bd75e08b90 2024-09-10T09:39:01.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:01.462+0000 7f82df9fa740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f82d81b1e30 con 0x55bd75e13d70 2024-09-10T09:39:01.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:01.462+0000 7f82df9fa740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f82d81af070 con 0x55bd75e094b0 2024-09-10T09:39:01.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:01.462+0000 7f82df9fa740 10 client.0 _ll_register_callbacks cb 0x7f82c8009a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:39:01.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:01.462+0000 7f82d5e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bd75e08b90 0x55bd75e148d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:39:01.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:01.462+0000 7f82d5e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bd75e08b90 0x55bd75e148d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36268/0 (socket says 172.21.15.5:36268) 2024-09-10T09:39:01.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:01.462+0000 7f82d5e00640 1 -- 172.21.15.5:0/3044839522 learned_addr learned my addr 172.21.15.5:0/3044839522 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:39:01.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:01.462+0000 7f82d5400640 1 --2- 172.21.15.5:0/3044839522 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bd75e094b0 0x55bd75e14e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:39:01.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:01.463+0000 7f82d5400640 1 -- 172.21.15.5:0/3044839522 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bd75e13d70 msgr2=0x55bd75e1adb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:39:01.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:01.463+0000 7f82d6800640 1 --2- 172.21.15.5:0/3044839522 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bd75e13d70 0x55bd75e1adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:39:01.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:01.463+0000 7f82d5400640 1 --2- 172.21.15.5:0/3044839522 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bd75e13d70 0x55bd75e1adb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:01.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:01.463+0000 7f82d5400640 1 -- 172.21.15.5:0/3044839522 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bd75e08b90 msgr2=0x55bd75e148d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:39:01.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:01.463+0000 7f82d5400640 1 --2- 172.21.15.5:0/3044839522 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bd75e08b90 0x55bd75e148d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:01.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:01.463+0000 7f82d5400640 1 -- 172.21.15.5:0/3044839522 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bd75e1b370 con 0x55bd75e094b0 2024-09-10T09:39:01.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:01.463+0000 7f82d6800640 1 --2- 172.21.15.5:0/3044839522 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bd75e13d70 0x55bd75e1adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T09:39:01.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:01.463+0000 7f82d5e00640 1 --2- 172.21.15.5:0/3044839522 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bd75e08b90 0x55bd75e148d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:39:01.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:01.463+0000 7f82d5400640 1 --2- 172.21.15.5:0/3044839522 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bd75e094b0 0x55bd75e14e10 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f82c40096c0 tx=0x7f82c402a410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d21556f3a5ad24de server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:39:01.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:01.463+0000 7f82b7e00640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-09-10T09:39:01.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:01.464+0000 7f82b7e00640 1 -- 172.21.15.5:0/3044839522 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82c40317f0 con 0x55bd75e094b0 2024-09-10T09:39:01.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:01.464+0000 7f82b7e00640 1 -- 172.21.15.5:0/3044839522 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f82c4031990 con 0x55bd75e094b0 2024-09-10T09:39:01.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:01.464+0000 7f82df9fa740 1 -- 172.21.15.5:0/3044839522 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f82c801f040 con 0x55bd75e094b0 2024-09-10T09:39:01.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:01.464+0000 7f82b7e00640 1 -- 172.21.15.5:0/3044839522 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f82c4063050 con 0x55bd75e094b0 2024-09-10T09:39:01.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:01.464+0000 7f82df9fa740 10 client.25495 fetch_fsmap learned FSMap version 30 2024-09-10T09:39:01.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:01.464+0000 7f82df9fa740 1 -- 172.21.15.5:0/3044839522 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55bd75e1c630 con 0x55bd75e094b0 2024-09-10T09:39:01.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:01.464+0000 7f82b7e00640 1 -- 172.21.15.5:0/3044839522 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f82c801f040 con 0x55bd75e094b0 2024-09-10T09:39:01.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:01.464+0000 7f82b7e00640 1 -- 172.21.15.5:0/3044839522 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f82c4054ad0 con 0x55bd75e094b0 2024-09-10T09:39:01.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:01.465+0000 7f82df9fa740 10 client.25495 fetch_fsmap finished waiting for FSMap version 30 2024-09-10T09:39:01.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:01.465+0000 7f82df9fa740 10 client.25495 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T09:39:01.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:01.465+0000 7f82df9fa740 20 client.25495 populate_metadata read hostname 'smithi005' 2024-09-10T09:39:01.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:01.465+0000 7f82df9fa740 1 --2- 172.21.15.5:0/3044839522 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55bd75e8e7b0 0x55bd75e90bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:39:01.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:01.465+0000 7f82df9fa740 4 client.25495 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T09:39:01.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:01.465+0000 7f82df9fa740 1 -- 172.21.15.5:0/3044839522 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55bd75d76e60 con 0x55bd75e8e7b0 2024-09-10T09:39:01.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:01.465+0000 7f82d5e00640 1 --2- 172.21.15.5:0/3044839522 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55bd75e8e7b0 0x55bd75e90bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:39:01.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:01.465+0000 7f82d5e00640 1 --2- 172.21.15.5:0/3044839522 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55bd75e8e7b0 0x55bd75e90bd0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9cb4fcb3e9a39451 server_cookie=62aa59fc314ba44b in_seq=0 out_seq=0 2024-09-10T09:39:01.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:01.466+0000 7f82b7e00640 10 client.25495 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T09:39:01.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:01.466+0000 7f82b7e00640 1 -- 172.21.15.5:0/3044839522 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55bd75d76e60 con 0x55bd75e8e7b0 2024-09-10T09:39:01.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:01.466+0000 7f82b7e00640 10 client.25495 handle_command_reply: tid=0 2024-09-10T09:39:01.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:01.466+0000 7f82df9fa740 1 client.25495 shutdown 2024-09-10T09:39:01.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:01.466+0000 7f82df9fa740 1 -- 172.21.15.5:0/3044839522 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bd75e094b0 msgr2=0x55bd75e14e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:39:01.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:01.466+0000 7f82df9fa740 1 --2- 172.21.15.5:0/3044839522 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bd75e094b0 0x55bd75e14e10 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f82c40096c0 tx=0x7f82c402a410 comp rx=0 tx=0).stop 2024-09-10T09:39:01.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:01.466+0000 7f82df9fa740 1 -- 172.21.15.5:0/3044839522 shutdown_connections 2024-09-10T09:39:01.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:01.466+0000 7f82df9fa740 1 --2- 172.21.15.5:0/3044839522 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55bd75e8e7b0 0x55bd75e90bd0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:01.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:01.466+0000 7f82df9fa740 1 --2- 172.21.15.5:0/3044839522 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bd75e13d70 0x55bd75e1adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:01.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:01.466+0000 7f82df9fa740 1 --2- 172.21.15.5:0/3044839522 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bd75e094b0 0x55bd75e14e10 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:01.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:01.466+0000 7f82df9fa740 1 --2- 172.21.15.5:0/3044839522 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bd75e08b90 0x55bd75e148d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:01.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:01.466+0000 7f82b7e00640 0 client.25495 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T09:39:01.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:01.466+0000 7f82df9fa740 1 -- 172.21.15.5:0/3044839522 >> 172.21.15.5:0/3044839522 conn(0x55bd75e05120 msgr2=0x55bd75e12430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:39:01.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:01.466+0000 7f82df9fa740 1 -- 172.21.15.5:0/3044839522 shutdown_connections 2024-09-10T09:39:01.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:01.466+0000 7f82df9fa740 1 -- 172.21.15.5:0/3044839522 wait complete. 2024-09-10T09:39:01.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:01.466+0000 7f82df9fa740 20 client.25495 trim_cache size 0 max 16384 2024-09-10T09:39:01.474 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-10T09:39:01.474 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "no active scrubs running", 2024-09-10T09:39:01.474 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": {} 2024-09-10T09:39:01.474 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-10T09:39:01.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:01.467+0000 7f82dce00640 1 -- 172.21.15.5:0/1271395408 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f82bc0a2c10 msgr2=0x7f82bc0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:39:01.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:01.467+0000 7f82dce00640 1 --2- 172.21.15.5:0/1271395408 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f82bc0a2c10 0x7f82bc0a50d0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x55bd75e064a0 tx=0x7f82c40046c0 comp rx=0 tx=0).stop 2024-09-10T09:39:01.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:01.467+0000 7f82dce00640 1 -- 172.21.15.5:0/1271395408 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f82d8109140 msgr2=0x7f82d819ef40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:39:01.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:01.467+0000 7f82dce00640 1 --2- 172.21.15.5:0/1271395408 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f82d8109140 0x7f82d819ef40 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f82d0009fd0 tx=0x7f82d000ef10 comp rx=0 tx=0).stop 2024-09-10T09:39:01.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:01.467+0000 7f82dce00640 1 -- 172.21.15.5:0/1271395408 shutdown_connections 2024-09-10T09:39:01.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:01.467+0000 7f82dce00640 1 --2- 172.21.15.5:0/1271395408 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f82bc0a2c10 0x7f82bc0a50d0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:01.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:01.467+0000 7f82dce00640 1 --2- 172.21.15.5:0/1271395408 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f82d8109140 0x7f82d819ef40 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:01.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:01.467+0000 7f82dce00640 1 --2- 172.21.15.5:0/1271395408 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f82d81049c0 0x7f82d819abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:01.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:01.467+0000 7f82dce00640 1 --2- 172.21.15.5:0/1271395408 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f82d8104010 0x7f82d819a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:01.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:01.467+0000 7f82dce00640 1 -- 172.21.15.5:0/1271395408 >> 172.21.15.5:0/1271395408 conn(0x7f82d80ffa50 msgr2=0x7f82d8100d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:39:01.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:01.467+0000 7f82dce00640 1 -- 172.21.15.5:0/1271395408 shutdown_connections 2024-09-10T09:39:01.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:01.467+0000 7f82dce00640 1 -- 172.21.15.5:0/1271395408 wait complete. 2024-09-10T09:39:01.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:39:01 smithi159 ceph-mon[35513]: pgmap v413: 129 pgs: 129 active+clean; 660 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-09-10T09:39:02.309 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-10T09:39:02.310 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T09:39:02.711 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:39:03.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:03.324+0000 7f7403e00640 1 -- 172.21.15.5:0/3797166745 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7404074510 msgr2=0x7f74040748f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:39:03.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:03.324+0000 7f7403e00640 1 --2- 172.21.15.5:0/3797166745 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7404074510 0x7f74040748f0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f73f4009900 tx=0x7f73f402a480 comp rx=0 tx=0).stop 2024-09-10T09:39:03.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:03.325+0000 7f7403e00640 1 -- 172.21.15.5:0/3797166745 shutdown_connections 2024-09-10T09:39:03.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:03.325+0000 7f7403e00640 1 --2- 172.21.15.5:0/3797166745 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f740410e280 0x7f7404110670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:03.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:03.325+0000 7f7403e00640 1 --2- 172.21.15.5:0/3797166745 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7404074e30 0x7f740410dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:03.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:03.325+0000 7f7403e00640 1 --2- 172.21.15.5:0/3797166745 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7404074510 0x7f74040748f0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:03.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:03.325+0000 7f7403e00640 1 -- 172.21.15.5:0/3797166745 >> 172.21.15.5:0/3797166745 conn(0x7f74040fd740 msgr2=0x7f74040ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:39:03.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:03.325+0000 7f7403e00640 1 -- 172.21.15.5:0/3797166745 shutdown_connections 2024-09-10T09:39:03.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:03.325+0000 7f7403e00640 1 -- 172.21.15.5:0/3797166745 wait complete. 2024-09-10T09:39:03.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:03.326+0000 7f7403e00640 1 Processor -- start 2024-09-10T09:39:03.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:03.326+0000 7f7403e00640 1 -- start start 2024-09-10T09:39:03.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:03.326+0000 7f7403e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7404074510 0x7f740419a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:39:03.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:03.326+0000 7f7403e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7404074e30 0x7f740419a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:39:03.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:03.326+0000 7f7403e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f740410e280 0x7f740419ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:39:03.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:03.326+0000 7f7403e00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f7404113f70 con 0x7f7404074510 2024-09-10T09:39:03.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:03.326+0000 7f7403e00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f7404113df0 con 0x7f740410e280 2024-09-10T09:39:03.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:03.326+0000 7f7403e00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f74041140f0 con 0x7f7404074e30 2024-09-10T09:39:03.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:03.327+0000 7f7402a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7404074510 0x7f740419a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:39:03.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:03.327+0000 7f7402a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7404074510 0x7f740419a470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36288/0 (socket says 172.21.15.5:36288) 2024-09-10T09:39:03.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:03.327+0000 7f7402a00640 1 -- 172.21.15.5:0/1014325321 learned_addr learned my addr 172.21.15.5:0/1014325321 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:39:03.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:03.327+0000 7f7402000640 1 --2- 172.21.15.5:0/1014325321 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7404074e30 0x7f740419a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:39:03.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:03.327+0000 7f7403400640 1 --2- 172.21.15.5:0/1014325321 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f740410e280 0x7f740419ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:39:03.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:03.327+0000 7f7402000640 1 -- 172.21.15.5:0/1014325321 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f740410e280 msgr2=0x7f740419ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:39:03.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:03.327+0000 7f7402000640 1 --2- 172.21.15.5:0/1014325321 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f740410e280 0x7f740419ed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:03.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:03.327+0000 7f7402000640 1 -- 172.21.15.5:0/1014325321 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7404074510 msgr2=0x7f740419a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:39:03.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:03.327+0000 7f7402000640 1 --2- 172.21.15.5:0/1014325321 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7404074510 0x7f740419a470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:03.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:03.327+0000 7f7402000640 1 -- 172.21.15.5:0/1014325321 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f740419f420 con 0x7f7404074e30 2024-09-10T09:39:03.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:03.328+0000 7f7402000640 1 --2- 172.21.15.5:0/1014325321 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7404074e30 0x7f740419a9b0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f73f000e9f0 tx=0x7f73f000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:39:03.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:03.328+0000 7f73fb400640 1 -- 172.21.15.5:0/1014325321 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73f000ccc0 con 0x7f7404074e30 2024-09-10T09:39:03.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:03.328+0000 7f73fb400640 1 -- 172.21.15.5:0/1014325321 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f73f0010910 con 0x7f7404074e30 2024-09-10T09:39:03.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:03.328+0000 7f7403e00640 1 -- 172.21.15.5:0/1014325321 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f740419f710 con 0x7f7404074e30 2024-09-10T09:39:03.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:03.329+0000 7f73fb400640 1 -- 172.21.15.5:0/1014325321 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73f0018960 con 0x7f7404074e30 2024-09-10T09:39:03.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:03.329+0000 7f7403e00640 1 -- 172.21.15.5:0/1014325321 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74041a6ff0 con 0x7f7404074e30 2024-09-10T09:39:03.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:03.330+0000 7f7403e00640 1 -- 172.21.15.5:0/1014325321 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f74040748f0 con 0x7f7404074e30 2024-09-10T09:39:03.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:03.332+0000 7f73fb400640 1 -- 172.21.15.5:0/1014325321 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 19) ==== 137883+0+0 (secure 0 0 0) 0x7f73f0010ac0 con 0x7f7404074e30 2024-09-10T09:39:03.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:03.335+0000 7f73fb400640 1 --2- 172.21.15.5:0/1014325321 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f73e80a2c10 0x7f73e80a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:39:03.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:03.335+0000 7f73fb400640 1 -- 172.21.15.5:0/1014325321 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f73f00cf9b0 con 0x7f7404074e30 2024-09-10T09:39:03.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:03.335+0000 7f7402a00640 1 --2- 172.21.15.5:0/1014325321 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f73e80a2c10 0x7f73e80a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:39:03.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:03.336+0000 7f7402a00640 1 --2- 172.21.15.5:0/1014325321 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f73e80a2c10 0x7f73e80a50d0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f73f40096c0 tx=0x7f73f4005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:39:03.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:03.338+0000 7f73fb400640 1 -- 172.21.15.5:0/1014325321 <== mon.1 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f73f009bd10 con 0x7f7404074e30 2024-09-10T09:39:03.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:39:03 smithi102 ceph-mon[35442]: pgmap v414: 129 pgs: 129 active+clean; 660 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-09-10T09:39:03.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:39:03 smithi005 ceph-mon[30350]: pgmap v414: 129 pgs: 129 active+clean; 660 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-09-10T09:39:03.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:39:03 smithi159 ceph-mon[35513]: pgmap v414: 129 pgs: 129 active+clean; 660 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-09-10T09:39:03.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:03.641+0000 7f7403e00640 1 -- 172.21.15.5:0/1014325321 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f740402d7c0 con 0x7f7404074e30 2024-09-10T09:39:03.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:03.642+0000 7f73fb400640 1 -- 172.21.15.5:0/1014325321 <== mon.1 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11969 (secure 0 0 0) 0x7f73f00a0010 con 0x7f7404074e30 2024-09-10T09:39:03.644 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-10T09:39:03.644 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":30,"btime":"2024-09-10T09:38:15:834198+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14454,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/2970594311","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2970594311},{"type":"v1","addr":"172.21.15.5:6835","nonce":2970594311}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14514,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6839/3616502174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":3616502174},{"type":"v1","addr":"172.21.15.5:6839","nonce":3616502174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T09:35:29.626287+0000","modified":"2024-09-10T09:38:15.658058+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24542,"mds_1":24535,"mds_2":24511,"mds_3":24506,"mds_4":24443},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14520":{"gid":14520,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/2073599842","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":2073599842},{"type":"v1","addr":"172.21.15.5:6841","nonce":2073599842}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"k","rank":4,"incarnation":26,"state":"up:active","state_seq":24,"addr":"172.21.15.102:6833/2457007427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":2457007427},{"type":"v1","addr":"172.21.15.102:6833","nonce":2457007427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/356445159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":356445159},{"type":"v1","addr":"172.21.15.159:6833","nonce":356445159}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6837/2894314507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":2894314507},{"type":"v1","addr":"172.21.15.5:6837","nonce":2894314507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.102:6835/1647549111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":1647549111},{"type":"v1","addr":"172.21.15.102:6835","nonce":1647549111}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6835/952657242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":952657242},{"type":"v1","addr":"172.21.15.159:6835","nonce":952657242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":19,"addr":"172.21.15.102:6837/2689776764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":2689776764},{"type":"v1","addr":"172.21.15.102:6837","nonce":2689776764}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.159:6837/620627639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":620627639},{"type":"v1","addr":"172.21.15.159:6837","nonce":620627639}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24535":{"gid":24535,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":18,"addr":"172.21.15.159:6839/2073739599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":2073739599},{"type":"v1","addr":"172.21.15.159:6839","nonce":2073739599}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24542":{"gid":24542,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.102:6839/335409749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6838","nonce":335409749},{"type":"v1","addr":"172.21.15.102:6839","nonce":335409749}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24542,"qdb_cluster":[24542,24535,24511,24506,24443]},"id":1}]} 2024-09-10T09:39:03.645 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 30 2024-09-10T09:39:03.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:03.644+0000 7f7403e00640 1 -- 172.21.15.5:0/1014325321 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f73e80a2c10 msgr2=0x7f73e80a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:39:03.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:03.644+0000 7f7403e00640 1 --2- 172.21.15.5:0/1014325321 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f73e80a2c10 0x7f73e80a50d0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f73f40096c0 tx=0x7f73f4005a20 comp rx=0 tx=0).stop 2024-09-10T09:39:03.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:03.644+0000 7f7403e00640 1 -- 172.21.15.5:0/1014325321 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7404074e30 msgr2=0x7f740419a9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:39:03.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:03.644+0000 7f7403e00640 1 --2- 172.21.15.5:0/1014325321 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7404074e30 0x7f740419a9b0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f73f000e9f0 tx=0x7f73f000eeb0 comp rx=0 tx=0).stop 2024-09-10T09:39:03.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:03.645+0000 7f7403e00640 1 -- 172.21.15.5:0/1014325321 shutdown_connections 2024-09-10T09:39:03.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:03.645+0000 7f7403e00640 1 --2- 172.21.15.5:0/1014325321 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f73e80a2c10 0x7f73e80a50d0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:03.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:03.645+0000 7f7403e00640 1 --2- 172.21.15.5:0/1014325321 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f740410e280 0x7f740419ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:03.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:03.645+0000 7f7403e00640 1 --2- 172.21.15.5:0/1014325321 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7404074e30 0x7f740419a9b0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:03.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:03.645+0000 7f7403e00640 1 --2- 172.21.15.5:0/1014325321 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7404074510 0x7f740419a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:03.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:03.645+0000 7f7403e00640 1 -- 172.21.15.5:0/1014325321 >> 172.21.15.5:0/1014325321 conn(0x7f74040fd740 msgr2=0x7f74040fee90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:39:03.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:03.645+0000 7f7403e00640 1 -- 172.21.15.5:0/1014325321 shutdown_connections 2024-09-10T09:39:03.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:03.645+0000 7f7403e00640 1 -- 172.21.15.5:0/1014325321 wait complete. 2024-09-10T09:39:04.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:39:04 smithi102 ceph-mon[35442]: from='client.? 172.21.15.5:0/1014325321' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:39:04.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:39:04 smithi005 ceph-mon[30350]: from='client.? 172.21.15.5:0/1014325321' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:39:04.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:39:04 smithi159 ceph-mon[35513]: from='client.? 172.21.15.5:0/1014325321' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:39:05.043 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-10T09:39:05.306 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:39:05 smithi005 ceph-mon[30350]: pgmap v415: 129 pgs: 129 active+clean; 660 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-10T09:39:05.445 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:39:05.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:39:05 smithi159 ceph-mon[35513]: pgmap v415: 129 pgs: 129 active+clean; 660 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-10T09:39:05.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:39:05 smithi102 ceph-mon[35442]: pgmap v415: 129 pgs: 129 active+clean; 660 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-10T09:39:06.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:06.035+0000 7f2ce6800640 1 -- 172.21.15.5:0/3418787812 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2ce01049c0 msgr2=0x7f2ce0108a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:39:06.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:06.035+0000 7f2ce6800640 1 --2- 172.21.15.5:0/3418787812 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2ce01049c0 0x7f2ce0108a10 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f2ccc009900 tx=0x7f2ccc02a3f0 comp rx=0 tx=0).stop 2024-09-10T09:39:06.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:06.035+0000 7f2ce6800640 1 -- 172.21.15.5:0/3418787812 shutdown_connections 2024-09-10T09:39:06.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:06.035+0000 7f2ce6800640 1 --2- 172.21.15.5:0/3418787812 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2ce0109140 0x7f2ce0110cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:06.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:06.035+0000 7f2ce6800640 1 --2- 172.21.15.5:0/3418787812 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2ce01049c0 0x7f2ce0108a10 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:06.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:06.035+0000 7f2ce6800640 1 --2- 172.21.15.5:0/3418787812 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2ce0104010 0x7f2ce01043f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:06.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:06.035+0000 7f2ce6800640 1 -- 172.21.15.5:0/3418787812 >> 172.21.15.5:0/3418787812 conn(0x7f2ce00ffa50 msgr2=0x7f2ce0101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:39:06.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:06.035+0000 7f2ce6800640 1 -- 172.21.15.5:0/3418787812 shutdown_connections 2024-09-10T09:39:06.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:06.035+0000 7f2ce6800640 1 -- 172.21.15.5:0/3418787812 wait complete. 2024-09-10T09:39:06.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:06.036+0000 7f2ce6800640 1 Processor -- start 2024-09-10T09:39:06.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:06.036+0000 7f2ce6800640 1 -- start start 2024-09-10T09:39:06.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:06.036+0000 7f2ce6800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2ce0104010 0x7f2ce019a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:39:06.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:06.036+0000 7f2ce6800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2ce01049c0 0x7f2ce019ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:39:06.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:06.037+0000 7f2ce6800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2ce0109140 0x7f2ce019efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:39:06.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:06.037+0000 7f2ce6800640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f2ce0114190 con 0x7f2ce01049c0 2024-09-10T09:39:06.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:06.037+0000 7f2ce6800640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f2ce0114010 con 0x7f2ce0104010 2024-09-10T09:39:06.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:06.037+0000 7f2ce6800640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f2ce0114310 con 0x7f2ce0109140 2024-09-10T09:39:06.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:06.037+0000 7f2cdea00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2ce01049c0 0x7f2ce019ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:39:06.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:06.037+0000 7f2cdea00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2ce01049c0 0x7f2ce019ac30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36302/0 (socket says 172.21.15.5:36302) 2024-09-10T09:39:06.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:06.037+0000 7f2cdea00640 1 -- 172.21.15.5:0/4012858505 learned_addr learned my addr 172.21.15.5:0/4012858505 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:39:06.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:06.037+0000 7f2cdf400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2ce0104010 0x7f2ce019a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:39:06.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:06.037+0000 7f2cdfe00640 1 --2- 172.21.15.5:0/4012858505 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2ce0109140 0x7f2ce019efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:39:06.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:06.037+0000 7f2cdea00640 1 -- 172.21.15.5:0/4012858505 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2ce0109140 msgr2=0x7f2ce019efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:39:06.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:06.037+0000 7f2cdea00640 1 --2- 172.21.15.5:0/4012858505 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2ce0109140 0x7f2ce019efc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:06.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:06.037+0000 7f2cdea00640 1 -- 172.21.15.5:0/4012858505 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2ce0104010 msgr2=0x7f2ce019a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:39:06.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:06.037+0000 7f2cdea00640 1 --2- 172.21.15.5:0/4012858505 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2ce0104010 0x7f2ce019a6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:06.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:06.037+0000 7f2cdea00640 1 -- 172.21.15.5:0/4012858505 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ce019f6a0 con 0x7f2ce01049c0 2024-09-10T09:39:06.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:06.037+0000 7f2cdfe00640 1 --2- 172.21.15.5:0/4012858505 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2ce0109140 0x7f2ce019efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:39:06.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:06.038+0000 7f2cdea00640 1 --2- 172.21.15.5:0/4012858505 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2ce01049c0 0x7f2ce019ac30 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f2ccc004770 tx=0x7f2ccc003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:39:06.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:06.038+0000 7f2cd7e00640 1 -- 172.21.15.5:0/4012858505 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ccc03f070 con 0x7f2ce01049c0 2024-09-10T09:39:06.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:06.038+0000 7f2ce6800640 1 -- 172.21.15.5:0/4012858505 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ce019f930 con 0x7f2ce01049c0 2024-09-10T09:39:06.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:06.038+0000 7f2cd7e00640 1 -- 172.21.15.5:0/4012858505 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2ccc031ca0 con 0x7f2ce01049c0 2024-09-10T09:39:06.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:06.038+0000 7f2cd7e00640 1 -- 172.21.15.5:0/4012858505 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ccc035030 con 0x7f2ce01049c0 2024-09-10T09:39:06.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:06.038+0000 7f2ce6800640 1 -- 172.21.15.5:0/4012858505 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ce01a7270 con 0x7f2ce01049c0 2024-09-10T09:39:06.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:06.042+0000 7f2ce93eb740 1 Processor -- start 2024-09-10T09:39:06.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:06.043+0000 7f2ce93eb740 1 -- start start 2024-09-10T09:39:06.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:06.043+0000 7f2ce93eb740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b747394c70 0x55b747395050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:39:06.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:06.043+0000 7f2ce93eb740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b747395590 0x55b74739f910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:39:06.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:06.043+0000 7f2ce93eb740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b74739fe50 0x55b7473a2240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:39:06.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:06.043+0000 7f2ce93eb740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55b7472fb3e0 con 0x55b747395590 2024-09-10T09:39:06.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:06.043+0000 7f2ce93eb740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55b7472a2d80 con 0x55b74739fe50 2024-09-10T09:39:06.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:06.043+0000 7f2ce93eb740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55b7472f73a0 con 0x55b747394c70 2024-09-10T09:39:06.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:06.043+0000 7f2cd7e00640 1 -- 172.21.15.5:0/4012858505 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 19) ==== 137883+0+0 (secure 0 0 0) 0x7f2ccc042070 con 0x7f2ce01049c0 2024-09-10T09:39:06.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:06.044+0000 7f2cdfe00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b747394c70 0x55b747395050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:39:06.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:06.044+0000 7f2cdfe00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b747394c70 0x55b747395050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:54318/0 (socket says 172.21.15.5:54318) 2024-09-10T09:39:06.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:06.044+0000 7f2cdfe00640 1 -- 172.21.15.5:0/312870709 learned_addr learned my addr 172.21.15.5:0/312870709 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:39:06.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:06.044+0000 7f2cdf400640 1 --2- 172.21.15.5:0/312870709 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b747395590 0x55b74739f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:39:06.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:06.044+0000 7f2cdea00640 1 --2- 172.21.15.5:0/312870709 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b74739fe50 0x55b7473a2240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:39:06.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:06.044+0000 7f2cdfe00640 1 -- 172.21.15.5:0/312870709 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b74739fe50 msgr2=0x55b7473a2240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:39:06.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:06.044+0000 7f2cdfe00640 1 --2- 172.21.15.5:0/312870709 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b74739fe50 0x55b7473a2240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:06.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:06.044+0000 7f2cdfe00640 1 -- 172.21.15.5:0/312870709 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b747395590 msgr2=0x55b74739f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:39:06.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:06.044+0000 7f2cdfe00640 1 --2- 172.21.15.5:0/312870709 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b747395590 0x55b74739f910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:06.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:06.044+0000 7f2cdfe00640 1 -- 172.21.15.5:0/312870709 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b7472c1ae0 con 0x55b747394c70 2024-09-10T09:39:06.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:06.044+0000 7f2cdea00640 1 --2- 172.21.15.5:0/312870709 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b74739fe50 0x55b7473a2240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T09:39:06.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:06.044+0000 7f2cdfe00640 1 --2- 172.21.15.5:0/312870709 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b747394c70 0x55b747395050 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f2cd8009f30 tx=0x7f2cd8010800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5c6e2a05975a9d25 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:39:06.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:06.045+0000 7f2cd4c00640 1 -- 172.21.15.5:0/312870709 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cd8016070 con 0x55b747394c70 2024-09-10T09:39:06.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:06.045+0000 7f2cd4c00640 1 -- 172.21.15.5:0/312870709 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2cd801bb40 con 0x55b747394c70 2024-09-10T09:39:06.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:06.045+0000 7f2cd4c00640 1 -- 172.21.15.5:0/312870709 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cd8012c60 con 0x55b747394c70 2024-09-10T09:39:06.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:06.045+0000 7f2cd7e00640 1 --2- 172.21.15.5:0/4012858505 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f2cc40a2c90 0x7f2cc40a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:39:06.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:06.045+0000 7f2cd7e00640 1 -- 172.21.15.5:0/4012858505 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f2ccc0ef9d0 con 0x7f2ce01049c0 2024-09-10T09:39:06.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:06.046+0000 7f2cdf400640 1 --2- 172.21.15.5:0/4012858505 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f2cc40a2c90 0x7f2cc40a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:39:06.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:06.046+0000 7f2ce93eb740 1 -- 172.21.15.5:0/312870709 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b747394c70 msgr2=0x55b747395050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:39:06.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:06.046+0000 7f2ce93eb740 1 --2- 172.21.15.5:0/312870709 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b747394c70 0x55b747395050 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f2cd8009f30 tx=0x7f2cd8010800 comp rx=0 tx=0).stop 2024-09-10T09:39:06.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:06.046+0000 7f2ce93eb740 1 -- 172.21.15.5:0/312870709 shutdown_connections 2024-09-10T09:39:06.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:06.046+0000 7f2ce93eb740 1 --2- 172.21.15.5:0/312870709 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b74739fe50 0x55b7473a2240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:06.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:06.046+0000 7f2ce93eb740 1 --2- 172.21.15.5:0/312870709 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b747395590 0x55b74739f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:06.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:06.046+0000 7f2ce93eb740 1 --2- 172.21.15.5:0/312870709 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b747394c70 0x55b747395050 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:06.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:06.046+0000 7f2ce93eb740 1 -- 172.21.15.5:0/312870709 >> 172.21.15.5:0/312870709 conn(0x55b747391200 msgr2=0x55b7473915e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:39:06.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:06.046+0000 7f2ce93eb740 1 -- 172.21.15.5:0/312870709 shutdown_connections 2024-09-10T09:39:06.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:06.046+0000 7f2ce93eb740 1 -- 172.21.15.5:0/312870709 wait complete. 2024-09-10T09:39:06.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:06.046+0000 7f2cdf400640 1 --2- 172.21.15.5:0/4012858505 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f2cc40a2c90 0x7f2cc40a5150 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x55b747392060 tx=0x7f2cd000b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:39:06.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:06.047+0000 7f2ce93eb740 1 -- start start 2024-09-10T09:39:06.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:06.047+0000 7f2ce93eb740 1 Processor -- start 2024-09-10T09:39:06.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:06.048+0000 7f2ce93eb740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b747394c70 0x55b74739cd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:39:06.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:06.048+0000 7f2ce93eb740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b747395590 0x55b74739d260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:39:06.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:06.048+0000 7f2ce93eb740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b74739fe50 0x55b74739d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:39:06.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:06.048+0000 7f2ce93eb740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55b7473a2a20 con 0x55b74739fe50 2024-09-10T09:39:06.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:06.048+0000 7f2ce93eb740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55b7472b60f0 con 0x55b747394c70 2024-09-10T09:39:06.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:06.048+0000 7f2ce93eb740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55b7473a2ba0 con 0x55b747395590 2024-09-10T09:39:06.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:06.048+0000 7f2ce93eb740 10 client.0 _ll_register_callbacks cb 0x55b74729dbd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:39:06.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:06.048+0000 7f2cdf400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b747394c70 0x55b74739cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:39:06.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:06.048+0000 7f2cdfe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b74739fe50 0x55b74739d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:39:06.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:06.048+0000 7f2cdea00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b747395590 0x55b74739d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:39:06.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:06.048+0000 7f2cdea00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b747395590 0x55b74739d260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:54324/0 (socket says 172.21.15.5:54324) 2024-09-10T09:39:06.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:06.048+0000 7f2cdf400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b747394c70 0x55b74739cd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:56598/0 (socket says 172.21.15.5:56598) 2024-09-10T09:39:06.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:06.048+0000 7f2cdfe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b74739fe50 0x55b74739d7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36322/0 (socket says 172.21.15.5:36322) 2024-09-10T09:39:06.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:06.048+0000 7f2cdea00640 1 -- 172.21.15.5:0/3516033153 learned_addr learned my addr 172.21.15.5:0/3516033153 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:39:06.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:06.049+0000 7f2cdea00640 1 -- 172.21.15.5:0/3516033153 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b747394c70 msgr2=0x55b74739cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:39:06.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:06.049+0000 7f2cdea00640 1 --2- 172.21.15.5:0/3516033153 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b747394c70 0x55b74739cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:06.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:06.049+0000 7f2cdea00640 1 -- 172.21.15.5:0/3516033153 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b74739fe50 msgr2=0x55b74739d7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:39:06.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:06.049+0000 7f2cdea00640 1 --2- 172.21.15.5:0/3516033153 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b74739fe50 0x55b74739d7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:06.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:06.049+0000 7f2cdea00640 1 -- 172.21.15.5:0/3516033153 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b74706c3c0 con 0x55b747395590 2024-09-10T09:39:06.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:06.049+0000 7f2cdea00640 1 --2- 172.21.15.5:0/3516033153 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b747395590 0x55b74739d260 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f2ccc031550 tx=0x7f2ccc0bbb10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c447fe1f93dced6e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:39:06.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:06.049+0000 7f2cc9600640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-09-10T09:39:06.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:06.049+0000 7f2cc9600640 1 -- 172.21.15.5:0/3516033153 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ccc049070 con 0x55b747395590 2024-09-10T09:39:06.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:06.049+0000 7f2cc9600640 1 -- 172.21.15.5:0/3516033153 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f2ccc03b5f0 con 0x55b747395590 2024-09-10T09:39:06.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:06.049+0000 7f2ce93eb740 1 -- 172.21.15.5:0/3516033153 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2cd801bb40 con 0x55b747395590 2024-09-10T09:39:06.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:06.050+0000 7f2cc9600640 1 -- 172.21.15.5:0/3516033153 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f2ccc0bc380 con 0x55b747395590 2024-09-10T09:39:06.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:06.050+0000 7f2ce93eb740 10 client.25513 fetch_fsmap learned FSMap version 30 2024-09-10T09:39:06.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:06.050+0000 7f2ce93eb740 1 -- 172.21.15.5:0/3516033153 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55b7473a2a20 con 0x55b747395590 2024-09-10T09:39:06.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:06.050+0000 7f2cc9600640 1 -- 172.21.15.5:0/3516033153 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2cd801bb40 con 0x55b747395590 2024-09-10T09:39:06.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:06.050+0000 7f2cc9600640 1 -- 172.21.15.5:0/3516033153 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f2ccc04c060 con 0x55b747395590 2024-09-10T09:39:06.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:06.050+0000 7f2ce93eb740 10 client.25513 fetch_fsmap finished waiting for FSMap version 30 2024-09-10T09:39:06.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:06.050+0000 7f2ce93eb740 10 client.25513 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-09-10T09:39:06.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:06.050+0000 7f2ce93eb740 20 client.25513 populate_metadata read hostname 'smithi005' 2024-09-10T09:39:06.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:06.050+0000 7f2ce93eb740 1 --2- 172.21.15.5:0/3516033153 >> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] conn(0x55b7473b85e0 0x55b7473baa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:39:06.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:06.050+0000 7f2ce93eb740 4 client.25513 mds_command: new command op to 24443 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:39:06.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:06.050+0000 7f2ce93eb740 1 -- 172.21.15.5:0/3516033153 --> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b747308950 con 0x55b7473b85e0 2024-09-10T09:39:06.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:06.051+0000 7f2cdf400640 1 --2- 172.21.15.5:0/3516033153 >> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] conn(0x55b7473b85e0 0x55b7473baa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:39:06.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:06.051+0000 7f2cdf400640 1 --2- 172.21.15.5:0/3516033153 >> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] conn(0x55b7473b85e0 0x55b7473baa20 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=e97fafbf55db4bdb server_cookie=aadefd017fefa365 in_seq=0 out_seq=0 2024-09-10T09:39:06.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:06.051+0000 7f2cc9600640 10 client.25513 ms_handle_connect on v2:172.21.15.102:6832/2457007427 2024-09-10T09:39:06.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:06.052+0000 7f2cc9600640 1 -- 172.21.15.5:0/3516033153 <== mds.4 v2:172.21.15.102:6832/2457007427 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b747308950 con 0x55b7473b85e0 2024-09-10T09:39:06.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:06.052+0000 7f2cc9600640 10 client.25513 handle_command_reply: tid=0 2024-09-10T09:39:06.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:06.052+0000 7f2ce93eb740 1 client.25513 shutdown 2024-09-10T09:39:06.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:06.052+0000 7f2ce93eb740 1 -- 172.21.15.5:0/3516033153 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b747395590 msgr2=0x55b74739d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:39:06.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:06.052+0000 7f2ce93eb740 1 --2- 172.21.15.5:0/3516033153 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b747395590 0x55b74739d260 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f2ccc031550 tx=0x7f2ccc0bbb10 comp rx=0 tx=0).stop 2024-09-10T09:39:06.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:06.053+0000 7f2ce93eb740 1 -- 172.21.15.5:0/3516033153 shutdown_connections 2024-09-10T09:39:06.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:06.053+0000 7f2ce93eb740 1 --2- 172.21.15.5:0/3516033153 >> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] conn(0x55b7473b85e0 0x55b7473baa20 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:06.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:06.053+0000 7f2ce93eb740 1 --2- 172.21.15.5:0/3516033153 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b74739fe50 0x55b74739d7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:06.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:06.053+0000 7f2ce93eb740 1 --2- 172.21.15.5:0/3516033153 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b747395590 0x55b74739d260 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:06.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:06.053+0000 7f2cc9600640 0 client.25513 ms_handle_reset on v2:172.21.15.102:6832/2457007427 2024-09-10T09:39:06.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:06.053+0000 7f2ce93eb740 1 --2- 172.21.15.5:0/3516033153 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b747394c70 0x55b74739cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:06.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:06.053+0000 7f2ce93eb740 1 -- 172.21.15.5:0/3516033153 >> 172.21.15.5:0/3516033153 conn(0x55b747391200 msgr2=0x55b747396f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:39:06.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:06.053+0000 7f2ce93eb740 1 -- 172.21.15.5:0/3516033153 shutdown_connections 2024-09-10T09:39:06.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:06.053+0000 7f2ce93eb740 1 -- 172.21.15.5:0/3516033153 wait complete. 2024-09-10T09:39:06.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:06.053+0000 7f2ce93eb740 20 client.25513 trim_cache size 0 max 16384 2024-09-10T09:39:06.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:06.068+0000 7f2ce93eb740 1 Processor -- start 2024-09-10T09:39:06.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:06.068+0000 7f2ce93eb740 1 -- start start 2024-09-10T09:39:06.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:06.069+0000 7f2ce93eb740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b747394c70 0x55b7473c02f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:39:06.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:06.069+0000 7f2ce93eb740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b747395590 0x55b7473c0830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:39:06.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:06.069+0000 7f2ce93eb740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b74739fe50 0x55b747399fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:39:06.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:06.069+0000 7f2ce93eb740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f2ce01acca0 con 0x55b747395590 2024-09-10T09:39:06.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:06.069+0000 7f2ce93eb740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f2ce01ae7e0 con 0x55b74739fe50 2024-09-10T09:39:06.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:06.069+0000 7f2ce93eb740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f2ce01ad370 con 0x55b747394c70 2024-09-10T09:39:06.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:06.069+0000 7f2cdea00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b747395590 0x55b7473c0830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:39:06.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:06.069+0000 7f2cdf400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b747394c70 0x55b7473c02f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:39:06.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:06.069+0000 7f2cdea00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b747395590 0x55b7473c0830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36326/0 (socket says 172.21.15.5:36326) 2024-09-10T09:39:06.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:06.069+0000 7f2cdea00640 1 -- 172.21.15.5:0/1905573114 learned_addr learned my addr 172.21.15.5:0/1905573114 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:39:06.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:06.069+0000 7f2cdfe00640 1 --2- 172.21.15.5:0/1905573114 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b74739fe50 0x55b747399fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:39:06.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:06.070+0000 7f2cdf400640 1 -- 172.21.15.5:0/1905573114 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b74739fe50 msgr2=0x55b747399fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:39:06.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:06.070+0000 7f2cdf400640 1 --2- 172.21.15.5:0/1905573114 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b74739fe50 0x55b747399fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:06.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:06.070+0000 7f2cdf400640 1 -- 172.21.15.5:0/1905573114 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b747395590 msgr2=0x55b7473c0830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:39:06.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:06.070+0000 7f2cdf400640 1 --2- 172.21.15.5:0/1905573114 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b747395590 0x55b7473c0830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:06.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:06.070+0000 7f2cdf400640 1 -- 172.21.15.5:0/1905573114 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b747391e90 con 0x55b747394c70 2024-09-10T09:39:06.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:06.070+0000 7f2cdea00640 1 --2- 172.21.15.5:0/1905573114 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b747395590 0x55b7473c0830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:39:06.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:06.070+0000 7f2cdf400640 1 --2- 172.21.15.5:0/1905573114 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b747394c70 0x55b7473c02f0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x55b7473bb230 tx=0x7f2cd0015000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f72a16067b73810b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:39:06.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:06.070+0000 7f2ccbe00640 1 -- 172.21.15.5:0/1905573114 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cd001eaa0 con 0x55b747394c70 2024-09-10T09:39:06.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:06.070+0000 7f2ccbe00640 1 -- 172.21.15.5:0/1905573114 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2cd001ec40 con 0x55b747394c70 2024-09-10T09:39:06.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:06.070+0000 7f2ccbe00640 1 -- 172.21.15.5:0/1905573114 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cd0003e50 con 0x55b747394c70 2024-09-10T09:39:06.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:06.071+0000 7f2ce93eb740 1 -- 172.21.15.5:0/1905573114 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b747394c70 msgr2=0x55b7473c02f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:39:06.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:06.071+0000 7f2ce93eb740 1 --2- 172.21.15.5:0/1905573114 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b747394c70 0x55b7473c02f0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x55b7473bb230 tx=0x7f2cd0015000 comp rx=0 tx=0).stop 2024-09-10T09:39:06.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:06.071+0000 7f2ce93eb740 1 -- 172.21.15.5:0/1905573114 shutdown_connections 2024-09-10T09:39:06.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:06.071+0000 7f2ce93eb740 1 --2- 172.21.15.5:0/1905573114 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b74739fe50 0x55b747399fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:06.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:06.071+0000 7f2ce93eb740 1 --2- 172.21.15.5:0/1905573114 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b747395590 0x55b7473c0830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:06.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:06.071+0000 7f2ce93eb740 1 --2- 172.21.15.5:0/1905573114 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b747394c70 0x55b7473c02f0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:06.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:06.071+0000 7f2ce93eb740 1 -- 172.21.15.5:0/1905573114 >> 172.21.15.5:0/1905573114 conn(0x55b747391200 msgr2=0x55b74739eda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:39:06.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:06.071+0000 7f2ce93eb740 1 -- 172.21.15.5:0/1905573114 shutdown_connections 2024-09-10T09:39:06.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:06.071+0000 7f2ce93eb740 1 -- 172.21.15.5:0/1905573114 wait complete. 2024-09-10T09:39:06.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:06.072+0000 7f2ce93eb740 1 -- start start 2024-09-10T09:39:06.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:06.072+0000 7f2ce93eb740 1 Processor -- start 2024-09-10T09:39:06.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:06.072+0000 7f2ce93eb740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b747394c70 0x55b7473b0680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:39:06.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:06.072+0000 7f2ce93eb740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b747395590 0x55b7473b0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:39:06.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:06.072+0000 7f2ce93eb740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b74739fe50 0x55b7473ad650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:39:06.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:06.072+0000 7f2ce93eb740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f2ce01ae110 con 0x55b74739fe50 2024-09-10T09:39:06.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:06.072+0000 7f2ce93eb740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f2ce01ada40 con 0x55b747394c70 2024-09-10T09:39:06.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:06.072+0000 7f2ce93eb740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f2ce01aac80 con 0x55b747395590 2024-09-10T09:39:06.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:06.072+0000 7f2ce93eb740 10 client.0 _ll_register_callbacks cb 0x7f2ccc005b70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:39:06.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:06.073+0000 7f2cdfe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b74739fe50 0x55b7473ad650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:39:06.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:06.073+0000 7f2cdfe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b74739fe50 0x55b7473ad650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36330/0 (socket says 172.21.15.5:36330) 2024-09-10T09:39:06.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:06.073+0000 7f2cdf400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b747394c70 0x55b7473b0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:39:06.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:06.073+0000 7f2cdfe00640 1 -- 172.21.15.5:0/4229930510 learned_addr learned my addr 172.21.15.5:0/4229930510 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:39:06.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:06.073+0000 7f2cdea00640 1 --2- 172.21.15.5:0/4229930510 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b747395590 0x55b7473b0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:39:06.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:06.073+0000 7f2cdfe00640 1 -- 172.21.15.5:0/4229930510 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b747395590 msgr2=0x55b7473b0bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:39:06.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:06.073+0000 7f2cdfe00640 1 --2- 172.21.15.5:0/4229930510 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b747395590 0x55b7473b0bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:06.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:06.073+0000 7f2cdfe00640 1 -- 172.21.15.5:0/4229930510 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b747394c70 msgr2=0x55b7473b0680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:39:06.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:06.073+0000 7f2cdfe00640 1 --2- 172.21.15.5:0/4229930510 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b747394c70 0x55b7473b0680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:06.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:06.073+0000 7f2cdfe00640 1 -- 172.21.15.5:0/4229930510 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b7473b1300 con 0x55b74739fe50 2024-09-10T09:39:06.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:06.074+0000 7f2cdfe00640 1 --2- 172.21.15.5:0/4229930510 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b74739fe50 0x55b7473ad650 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f2cd8012810 tx=0x7f2cd8004c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e50589b609296ca server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:39:06.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:06.074+0000 7f2cc9600640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-10T09:39:06.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:06.074+0000 7f2cc9600640 1 -- 172.21.15.5:0/4229930510 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cd802a020 con 0x55b74739fe50 2024-09-10T09:39:06.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:06.074+0000 7f2ce93eb740 1 -- 172.21.15.5:0/4229930510 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2cd001ec40 con 0x55b74739fe50 2024-09-10T09:39:06.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:06.074+0000 7f2cc9600640 1 -- 172.21.15.5:0/4229930510 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f2cd8012b70 con 0x55b74739fe50 2024-09-10T09:39:06.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:06.074+0000 7f2cc9600640 1 -- 172.21.15.5:0/4229930510 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f2cd8035050 con 0x55b74739fe50 2024-09-10T09:39:06.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:06.074+0000 7f2ce93eb740 10 client.15486 fetch_fsmap learned FSMap version 30 2024-09-10T09:39:06.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:06.074+0000 7f2ce93eb740 1 -- 172.21.15.5:0/4229930510 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55b747423970 con 0x55b74739fe50 2024-09-10T09:39:06.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:06.075+0000 7f2cc9600640 1 -- 172.21.15.5:0/4229930510 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2cd001ec40 con 0x55b74739fe50 2024-09-10T09:39:06.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:06.075+0000 7f2cc9600640 1 -- 172.21.15.5:0/4229930510 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f2cd800f040 con 0x55b74739fe50 2024-09-10T09:39:06.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:06.075+0000 7f2ce93eb740 10 client.15486 fetch_fsmap finished waiting for FSMap version 30 2024-09-10T09:39:06.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:06.075+0000 7f2ce93eb740 10 client.15486 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-09-10T09:39:06.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:06.075+0000 7f2ce93eb740 20 client.15486 populate_metadata read hostname 'smithi005' 2024-09-10T09:39:06.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:06.075+0000 7f2ce93eb740 1 --2- 172.21.15.5:0/4229930510 >> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] conn(0x55b747423e10 0x55b747426230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:39:06.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:06.075+0000 7f2ce93eb740 4 client.15486 mds_command: new command op to 24443 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T09:39:06.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:06.075+0000 7f2ce93eb740 1 -- 172.21.15.5:0/4229930510 --> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b747302e60 con 0x55b747423e10 2024-09-10T09:39:06.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:06.075+0000 7f2cdf400640 1 --2- 172.21.15.5:0/4229930510 >> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] conn(0x55b747423e10 0x55b747426230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:39:06.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:06.076+0000 7f2cdf400640 1 --2- 172.21.15.5:0/4229930510 >> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] conn(0x55b747423e10 0x55b747426230 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=3a0de2a062fc62be server_cookie=590f1af6f92b9259 in_seq=0 out_seq=0 2024-09-10T09:39:06.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:06.076+0000 7f2cc9600640 10 client.15486 ms_handle_connect on v2:172.21.15.102:6832/2457007427 2024-09-10T09:39:06.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:06.076+0000 7f2cc9600640 1 -- 172.21.15.5:0/4229930510 <== mds.4 v2:172.21.15.102:6832/2457007427 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b747302e60 con 0x55b747423e10 2024-09-10T09:39:06.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:06.076+0000 7f2cc9600640 10 client.15486 handle_command_reply: tid=0 2024-09-10T09:39:06.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:06.076+0000 7f2ce93eb740 1 client.15486 shutdown 2024-09-10T09:39:06.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:06.076+0000 7f2ce93eb740 1 -- 172.21.15.5:0/4229930510 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b74739fe50 msgr2=0x55b7473ad650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:39:06.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:06.076+0000 7f2ce93eb740 1 --2- 172.21.15.5:0/4229930510 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b74739fe50 0x55b7473ad650 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f2cd8012810 tx=0x7f2cd8004c30 comp rx=0 tx=0).stop 2024-09-10T09:39:06.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:06.077+0000 7f2ce93eb740 1 -- 172.21.15.5:0/4229930510 shutdown_connections 2024-09-10T09:39:06.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:06.077+0000 7f2ce93eb740 1 --2- 172.21.15.5:0/4229930510 >> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] conn(0x55b747423e10 0x55b747426230 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:06.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:06.077+0000 7f2ce93eb740 1 --2- 172.21.15.5:0/4229930510 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b74739fe50 0x55b7473ad650 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:06.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:06.077+0000 7f2ce93eb740 1 --2- 172.21.15.5:0/4229930510 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b747395590 0x55b7473b0bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:06.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:06.077+0000 7f2ce93eb740 1 --2- 172.21.15.5:0/4229930510 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b747394c70 0x55b7473b0680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:06.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:06.077+0000 7f2ce93eb740 1 -- 172.21.15.5:0/4229930510 >> 172.21.15.5:0/4229930510 conn(0x55b747391200 msgr2=0x55b74738fb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:39:06.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:06.077+0000 7f2cc9600640 0 client.15486 ms_handle_reset on v2:172.21.15.102:6832/2457007427 2024-09-10T09:39:06.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:06.077+0000 7f2ce93eb740 1 -- 172.21.15.5:0/4229930510 shutdown_connections 2024-09-10T09:39:06.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:06.077+0000 7f2ce93eb740 1 -- 172.21.15.5:0/4229930510 wait complete. 2024-09-10T09:39:06.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:06.077+0000 7f2ce93eb740 20 client.15486 trim_cache size 0 max 16384 2024-09-10T09:39:06.084 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-10T09:39:06.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:06.078+0000 7f2ce6800640 1 -- 172.21.15.5:0/4012858505 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f2cc40a2c90 msgr2=0x7f2cc40a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:39:06.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:06.078+0000 7f2ce6800640 1 --2- 172.21.15.5:0/4012858505 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f2cc40a2c90 0x7f2cc40a5150 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x55b747392060 tx=0x7f2cd000b410 comp rx=0 tx=0).stop 2024-09-10T09:39:06.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:06.078+0000 7f2ce6800640 1 -- 172.21.15.5:0/4012858505 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2ce01049c0 msgr2=0x7f2ce019ac30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:39:06.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:06.078+0000 7f2ce6800640 1 --2- 172.21.15.5:0/4012858505 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2ce01049c0 0x7f2ce019ac30 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f2ccc004770 tx=0x7f2ccc003c10 comp rx=0 tx=0).stop 2024-09-10T09:39:06.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:06.078+0000 7f2ce6800640 1 -- 172.21.15.5:0/4012858505 shutdown_connections 2024-09-10T09:39:06.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:06.078+0000 7f2ce6800640 1 --2- 172.21.15.5:0/4012858505 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f2cc40a2c90 0x7f2cc40a5150 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:06.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:06.078+0000 7f2ce6800640 1 --2- 172.21.15.5:0/4012858505 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2ce0109140 0x7f2ce019efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:06.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:06.078+0000 7f2ce6800640 1 --2- 172.21.15.5:0/4012858505 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2ce01049c0 0x7f2ce019ac30 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:06.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:06.078+0000 7f2ce6800640 1 --2- 172.21.15.5:0/4012858505 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2ce0104010 0x7f2ce019a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:06.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:06.078+0000 7f2ce6800640 1 -- 172.21.15.5:0/4012858505 >> 172.21.15.5:0/4012858505 conn(0x7f2ce00ffa50 msgr2=0x7f2ce0100dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:39:06.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:06.078+0000 7f2ce6800640 1 -- 172.21.15.5:0/4012858505 shutdown_connections 2024-09-10T09:39:06.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:06.078+0000 7f2ce6800640 1 -- 172.21.15.5:0/4012858505 wait complete. 2024-09-10T09:39:06.886 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-10T09:39:07.286 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:39:07.433 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:39:07 smithi005 ceph-mon[30350]: pgmap v416: 129 pgs: 129 active+clean; 660 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-10T09:39:07.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:39:07 smithi159 ceph-mon[35513]: pgmap v416: 129 pgs: 129 active+clean; 660 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-10T09:39:07.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:39:07 smithi102 ceph-mon[35442]: pgmap v416: 129 pgs: 129 active+clean; 660 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-09-10T09:39:07.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:07.871+0000 7f00f9800640 1 -- 172.21.15.5:0/4005587846 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f00f4102b90 msgr2=0x7f00f4102f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:39:07.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:07.871+0000 7f00f9800640 1 --2- 172.21.15.5:0/4005587846 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f00f4102b90 0x7f00f4102f70 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f00e4009860 tx=0x7f00e402a3d0 comp rx=0 tx=0).stop 2024-09-10T09:39:07.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:07.871+0000 7f00f9800640 1 -- 172.21.15.5:0/4005587846 shutdown_connections 2024-09-10T09:39:07.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:07.871+0000 7f00f9800640 1 --2- 172.21.15.5:0/4005587846 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f00f410bfc0 0x7f00f410e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:07.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:07.871+0000 7f00f9800640 1 --2- 172.21.15.5:0/4005587846 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f00f41034b0 0x7f00f410b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:07.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:07.871+0000 7f00f9800640 1 --2- 172.21.15.5:0/4005587846 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f00f4102b90 0x7f00f4102f70 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:07.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:07.871+0000 7f00f9800640 1 -- 172.21.15.5:0/4005587846 >> 172.21.15.5:0/4005587846 conn(0x7f00f40fc610 msgr2=0x7f00f40fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:39:07.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:07.871+0000 7f00f9800640 1 -- 172.21.15.5:0/4005587846 shutdown_connections 2024-09-10T09:39:07.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:07.871+0000 7f00f9800640 1 -- 172.21.15.5:0/4005587846 wait complete. 2024-09-10T09:39:07.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:07.872+0000 7f00f9800640 1 Processor -- start 2024-09-10T09:39:07.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:07.872+0000 7f00f9800640 1 -- start start 2024-09-10T09:39:07.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:07.873+0000 7f00f9800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f00f4102b90 0x7f00f419a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:39:07.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:07.873+0000 7f00f9800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f00f41034b0 0x7f00f419aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:39:07.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:07.873+0000 7f00f9800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f00f410bfc0 0x7f00f419eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:39:07.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:07.873+0000 7f00f9800640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f00f4111e10 con 0x7f00f41034b0 2024-09-10T09:39:07.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:07.873+0000 7f00f9800640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f00f4111c90 con 0x7f00f410bfc0 2024-09-10T09:39:07.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:07.873+0000 7f00f9800640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f00f4111f90 con 0x7f00f4102b90 2024-09-10T09:39:07.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:07.873+0000 7f00f3e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f00f4102b90 0x7f00f419a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:39:07.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:07.873+0000 7f00f3e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f00f4102b90 0x7f00f419a4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:54370/0 (socket says 172.21.15.5:54370) 2024-09-10T09:39:07.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:07.873+0000 7f00f3e00640 1 -- 172.21.15.5:0/4010756399 learned_addr learned my addr 172.21.15.5:0/4010756399 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:39:07.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:07.873+0000 7f00f3400640 1 --2- 172.21.15.5:0/4010756399 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f00f41034b0 0x7f00f419aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:39:07.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:07.873+0000 7f00f8e00640 1 --2- 172.21.15.5:0/4010756399 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f00f410bfc0 0x7f00f419eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:39:07.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:07.874+0000 7f00f3e00640 1 -- 172.21.15.5:0/4010756399 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f00f410bfc0 msgr2=0x7f00f419eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:39:07.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:07.874+0000 7f00f3e00640 1 --2- 172.21.15.5:0/4010756399 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f00f410bfc0 0x7f00f419eda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:07.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:07.874+0000 7f00f3e00640 1 -- 172.21.15.5:0/4010756399 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f00f41034b0 msgr2=0x7f00f419aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:39:07.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:07.874+0000 7f00f3e00640 1 --2- 172.21.15.5:0/4010756399 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f00f41034b0 0x7f00f419aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:07.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:07.874+0000 7f00f3e00640 1 -- 172.21.15.5:0/4010756399 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00f419f480 con 0x7f00f4102b90 2024-09-10T09:39:07.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:07.874+0000 7f00f3e00640 1 --2- 172.21.15.5:0/4010756399 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f00f4102b90 0x7f00f419a4d0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f00e4009980 tx=0x7f00e40309a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:39:07.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:07.874+0000 7f00f0c00640 1 -- 172.21.15.5:0/4010756399 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00e4040070 con 0x7f00f4102b90 2024-09-10T09:39:07.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:07.874+0000 7f00f9800640 1 -- 172.21.15.5:0/4010756399 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00f419f710 con 0x7f00f4102b90 2024-09-10T09:39:07.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:07.874+0000 7f00f0c00640 1 -- 172.21.15.5:0/4010756399 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f00e4005620 con 0x7f00f4102b90 2024-09-10T09:39:07.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:07.874+0000 7f00f0c00640 1 -- 172.21.15.5:0/4010756399 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00e40049b0 con 0x7f00f4102b90 2024-09-10T09:39:07.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:07.874+0000 7f00f9800640 1 -- 172.21.15.5:0/4010756399 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00f41a7050 con 0x7f00f4102b90 2024-09-10T09:39:07.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:07.877+0000 7f00fd84d740 1 Processor -- start 2024-09-10T09:39:07.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:07.878+0000 7f00fd84d740 1 -- start start 2024-09-10T09:39:07.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:07.878+0000 7f00fd84d740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562eb50cee40 0x562eb50cf220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:39:07.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:07.878+0000 7f00fd84d740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562eb50cf760 0x562eb50d9ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:39:07.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:07.878+0000 7f00fd84d740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562eb50da020 0x562eb50dc410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:39:07.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:07.878+0000 7f00fd84d740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x562eb50353e0 con 0x562eb50cee40 2024-09-10T09:39:07.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:07.878+0000 7f00fd84d740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x562eb4fdcd80 con 0x562eb50cf760 2024-09-10T09:39:07.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:07.878+0000 7f00fd84d740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x562eb50313a0 con 0x562eb50da020 2024-09-10T09:39:07.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:07.878+0000 7f00f8e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562eb50cee40 0x562eb50cf220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:39:07.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:07.878+0000 7f00f3400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562eb50da020 0x562eb50dc410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:39:07.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:07.878+0000 7f00f8e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562eb50cee40 0x562eb50cf220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36356/0 (socket says 172.21.15.5:36356) 2024-09-10T09:39:07.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:07.878+0000 7f00f8e00640 1 -- 172.21.15.5:0/4189414919 learned_addr learned my addr 172.21.15.5:0/4189414919 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:39:07.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:07.878+0000 7f00f0c00640 1 -- 172.21.15.5:0/4010756399 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 19) ==== 137883+0+0 (secure 0 0 0) 0x7f00e4043070 con 0x7f00f4102b90 2024-09-10T09:39:07.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:07.879+0000 7f00f3400640 1 -- 172.21.15.5:0/4189414919 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562eb50cf760 msgr2=0x562eb50d9ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:39:07.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:07.879+0000 7f00f3400640 1 --2- 172.21.15.5:0/4189414919 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562eb50cf760 0x562eb50d9ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:07.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:07.879+0000 7f00f3400640 1 -- 172.21.15.5:0/4189414919 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562eb50cee40 msgr2=0x562eb50cf220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:39:07.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:07.879+0000 7f00f3400640 1 --2- 172.21.15.5:0/4189414919 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562eb50cee40 0x562eb50cf220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:07.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:07.879+0000 7f00f3400640 1 -- 172.21.15.5:0/4189414919 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562eb4ffbae0 con 0x562eb50da020 2024-09-10T09:39:07.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:07.879+0000 7f00f8e00640 1 --2- 172.21.15.5:0/4189414919 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562eb50cee40 0x562eb50cf220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:39:07.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:07.879+0000 7f00f3400640 1 --2- 172.21.15.5:0/4189414919 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562eb50da020 0x562eb50dc410 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f00e0005fd0 tx=0x7f00e0011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=39be107cf36179a9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:39:07.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:07.880+0000 7f00e9600640 1 -- 172.21.15.5:0/4189414919 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00e001e070 con 0x562eb50da020 2024-09-10T09:39:07.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:07.880+0000 7f00e9600640 1 -- 172.21.15.5:0/4189414919 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f00e00035f0 con 0x562eb50da020 2024-09-10T09:39:07.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:07.880+0000 7f00e9600640 1 -- 172.21.15.5:0/4189414919 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00e0019690 con 0x562eb50da020 2024-09-10T09:39:07.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:07.880+0000 7f00f0c00640 1 --2- 172.21.15.5:0/4010756399 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f00d80a2c90 0x7f00d80a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:39:07.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:07.880+0000 7f00fd84d740 1 -- 172.21.15.5:0/4189414919 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562eb50da020 msgr2=0x562eb50dc410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:39:07.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:07.881+0000 7f00fd84d740 1 --2- 172.21.15.5:0/4189414919 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562eb50da020 0x562eb50dc410 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f00e0005fd0 tx=0x7f00e0011810 comp rx=0 tx=0).stop 2024-09-10T09:39:07.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:07.881+0000 7f00f0c00640 1 -- 172.21.15.5:0/4010756399 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f00e40f03c0 con 0x7f00f4102b90 2024-09-10T09:39:07.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:07.881+0000 7f00fd84d740 1 -- 172.21.15.5:0/4189414919 shutdown_connections 2024-09-10T09:39:07.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:07.881+0000 7f00fd84d740 1 --2- 172.21.15.5:0/4189414919 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562eb50da020 0x562eb50dc410 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:07.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:07.881+0000 7f00fd84d740 1 --2- 172.21.15.5:0/4189414919 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562eb50cf760 0x562eb50d9ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:07.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:07.881+0000 7f00fd84d740 1 --2- 172.21.15.5:0/4189414919 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562eb50cee40 0x562eb50cf220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:07.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:07.881+0000 7f00fd84d740 1 -- 172.21.15.5:0/4189414919 >> 172.21.15.5:0/4189414919 conn(0x562eb50cb3d0 msgr2=0x562eb50cb7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:39:07.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:07.881+0000 7f00f8e00640 1 --2- 172.21.15.5:0/4010756399 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f00d80a2c90 0x7f00d80a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:39:07.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:07.881+0000 7f00fd84d740 1 -- 172.21.15.5:0/4189414919 shutdown_connections 2024-09-10T09:39:07.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:07.881+0000 7f00fd84d740 1 -- 172.21.15.5:0/4189414919 wait complete. 2024-09-10T09:39:07.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:07.882+0000 7f00fd84d740 1 -- start start 2024-09-10T09:39:07.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:07.882+0000 7f00f8e00640 1 --2- 172.21.15.5:0/4010756399 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f00d80a2c90 0x7f00d80a5150 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f00ec005fd0 tx=0x7f00ec009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:39:07.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:07.882+0000 7f00fd84d740 1 Processor -- start 2024-09-10T09:39:07.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:07.882+0000 7f00fd84d740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562eb50cee40 0x562eb50d6fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:39:07.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:07.882+0000 7f00fd84d740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562eb50cf760 0x562eb50d7500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:39:07.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:07.883+0000 7f00fd84d740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562eb50da020 0x562eb50d7a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:39:07.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:07.883+0000 7f00fd84d740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x562eb50dcbf0 con 0x562eb50cf760 2024-09-10T09:39:07.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:07.883+0000 7f00fd84d740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x562eb4ff00f0 con 0x562eb50da020 2024-09-10T09:39:07.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:07.883+0000 7f00fd84d740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x562eb50dcd70 con 0x562eb50cee40 2024-09-10T09:39:07.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:07.883+0000 7f00fd84d740 10 client.0 _ll_register_callbacks cb 0x562eb4fd7bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:39:07.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:07.883+0000 7f00f3400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562eb50cf760 0x562eb50d7500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:39:07.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:07.883+0000 7f00f3e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562eb50cee40 0x562eb50d6fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:39:07.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:07.883+0000 7f00f8e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562eb50da020 0x562eb50d7a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:39:07.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:07.883+0000 7f00f3e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562eb50cee40 0x562eb50d6fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:54390/0 (socket says 172.21.15.5:54390) 2024-09-10T09:39:07.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:07.883+0000 7f00f3e00640 1 -- 172.21.15.5:0/1315140821 learned_addr learned my addr 172.21.15.5:0/1315140821 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:39:07.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:07.884+0000 7f00f8e00640 1 -- 172.21.15.5:0/1315140821 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562eb50cee40 msgr2=0x562eb50d6fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:39:07.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:07.884+0000 7f00f8e00640 1 --2- 172.21.15.5:0/1315140821 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562eb50cee40 0x562eb50d6fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:07.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:07.884+0000 7f00f8e00640 1 -- 172.21.15.5:0/1315140821 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562eb50cf760 msgr2=0x562eb50d7500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:39:07.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:07.884+0000 7f00f8e00640 1 --2- 172.21.15.5:0/1315140821 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562eb50cf760 0x562eb50d7500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:07.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:07.884+0000 7f00f8e00640 1 -- 172.21.15.5:0/1315140821 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562eb4da63c0 con 0x562eb50da020 2024-09-10T09:39:07.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:07.884+0000 7f00f3e00640 1 --2- 172.21.15.5:0/1315140821 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562eb50cee40 0x562eb50d6fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:39:07.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:07.884+0000 7f00f8e00640 1 --2- 172.21.15.5:0/1315140821 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562eb50da020 0x562eb50d7a40 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f00ec010960 tx=0x7f00ec004ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8add4ee7635b9f67 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:39:07.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:07.884+0000 7f00de000640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-09-10T09:39:07.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:07.884+0000 7f00de000640 1 -- 172.21.15.5:0/1315140821 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00ec01fbc0 con 0x562eb50da020 2024-09-10T09:39:07.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:07.884+0000 7f00fd84d740 1 -- 172.21.15.5:0/1315140821 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f00e00035f0 con 0x562eb50da020 2024-09-10T09:39:07.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:07.884+0000 7f00de000640 1 -- 172.21.15.5:0/1315140821 <== mon.2 v2:172.21.15.102:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f00ec017030 con 0x562eb50da020 2024-09-10T09:39:07.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:07.885+0000 7f00de000640 1 -- 172.21.15.5:0/1315140821 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f00ec032050 con 0x562eb50da020 2024-09-10T09:39:07.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:07.885+0000 7f00fd84d740 10 client.25505 fetch_fsmap learned FSMap version 30 2024-09-10T09:39:07.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:07.885+0000 7f00fd84d740 1 -- 172.21.15.5:0/1315140821 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x562eb50e8d30 con 0x562eb50da020 2024-09-10T09:39:07.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:07.885+0000 7f00de000640 1 -- 172.21.15.5:0/1315140821 <== mon.2 v2:172.21.15.102:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f00e00035f0 con 0x562eb50da020 2024-09-10T09:39:07.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:07.885+0000 7f00de000640 1 -- 172.21.15.5:0/1315140821 <== mon.2 v2:172.21.15.102:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f00ec005670 con 0x562eb50da020 2024-09-10T09:39:07.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:07.886+0000 7f00fd84d740 10 client.25505 fetch_fsmap finished waiting for FSMap version 30 2024-09-10T09:39:07.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:07.886+0000 7f00fd84d740 10 client.25505 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-09-10T09:39:07.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:07.886+0000 7f00fd84d740 20 client.25505 populate_metadata read hostname 'smithi005' 2024-09-10T09:39:07.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:07.886+0000 7f00fd84d740 1 --2- 172.21.15.5:0/1315140821 >> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] conn(0x562eb50ea1f0 0x562eb50ec610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:39:07.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:07.886+0000 7f00fd84d740 4 client.25505 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:39:07.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:07.886+0000 7f00fd84d740 1 -- 172.21.15.5:0/1315140821 --> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562eb5042950 con 0x562eb50ea1f0 2024-09-10T09:39:07.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:07.886+0000 7f00f3e00640 1 --2- 172.21.15.5:0/1315140821 >> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] conn(0x562eb50ea1f0 0x562eb50ec610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:39:07.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:07.887+0000 7f00f3e00640 1 --2- 172.21.15.5:0/1315140821 >> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] conn(0x562eb50ea1f0 0x562eb50ec610 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=b88c964fc9912d9e server_cookie=4cd68152fa303068 in_seq=0 out_seq=0 2024-09-10T09:39:07.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:07.887+0000 7f00de000640 10 client.25505 ms_handle_connect on v2:172.21.15.102:6836/2689776764 2024-09-10T09:39:07.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:07.887+0000 7f00de000640 1 -- 172.21.15.5:0/1315140821 <== mds.3 v2:172.21.15.102:6836/2689776764 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562eb5042950 con 0x562eb50ea1f0 2024-09-10T09:39:07.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:07.887+0000 7f00de000640 10 client.25505 handle_command_reply: tid=0 2024-09-10T09:39:07.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:07.887+0000 7f00fd84d740 1 client.25505 shutdown 2024-09-10T09:39:07.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:07.888+0000 7f00fd84d740 1 -- 172.21.15.5:0/1315140821 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562eb50da020 msgr2=0x562eb50d7a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:39:07.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:07.888+0000 7f00fd84d740 1 --2- 172.21.15.5:0/1315140821 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562eb50da020 0x562eb50d7a40 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f00ec010960 tx=0x7f00ec004ab0 comp rx=0 tx=0).stop 2024-09-10T09:39:07.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:07.888+0000 7f00fd84d740 1 -- 172.21.15.5:0/1315140821 shutdown_connections 2024-09-10T09:39:07.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:07.888+0000 7f00fd84d740 1 --2- 172.21.15.5:0/1315140821 >> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] conn(0x562eb50ea1f0 0x562eb50ec610 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:07.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:07.888+0000 7f00fd84d740 1 --2- 172.21.15.5:0/1315140821 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562eb50da020 0x562eb50d7a40 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:07.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:07.888+0000 7f00fd84d740 1 --2- 172.21.15.5:0/1315140821 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562eb50cf760 0x562eb50d7500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:07.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:07.888+0000 7f00fd84d740 1 --2- 172.21.15.5:0/1315140821 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562eb50cee40 0x562eb50d6fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:07.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:07.888+0000 7f00fd84d740 1 -- 172.21.15.5:0/1315140821 >> 172.21.15.5:0/1315140821 conn(0x562eb50cb3d0 msgr2=0x562eb50d1100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:39:07.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:07.888+0000 7f00de000640 0 client.25505 ms_handle_reset on v2:172.21.15.102:6836/2689776764 2024-09-10T09:39:07.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:07.888+0000 7f00fd84d740 1 -- 172.21.15.5:0/1315140821 shutdown_connections 2024-09-10T09:39:07.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:07.888+0000 7f00fd84d740 1 -- 172.21.15.5:0/1315140821 wait complete. 2024-09-10T09:39:07.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:07.888+0000 7f00fd84d740 20 client.25505 trim_cache size 0 max 16384 2024-09-10T09:39:07.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:07.902+0000 7f00fd84d740 1 Processor -- start 2024-09-10T09:39:07.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:07.902+0000 7f00fd84d740 1 -- start start 2024-09-10T09:39:07.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:07.903+0000 7f00fd84d740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562eb50cee40 0x562eb50f1f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:39:07.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:07.903+0000 7f00fd84d740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562eb50cf760 0x562eb50f2470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:39:07.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:07.903+0000 7f00fd84d740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562eb50da020 0x562eb50c9c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:39:07.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:07.903+0000 7f00fd84d740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f00f41b0ed0 con 0x562eb50da020 2024-09-10T09:39:07.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:07.903+0000 7f00fd84d740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f00f41b2a10 con 0x562eb50cee40 2024-09-10T09:39:07.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:07.903+0000 7f00fd84d740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f00f41b15a0 con 0x562eb50cf760 2024-09-10T09:39:07.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:07.903+0000 7f00f8e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562eb50da020 0x562eb50c9c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:39:07.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:07.903+0000 7f00f3e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562eb50cee40 0x562eb50f1f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:39:07.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:07.903+0000 7f00f3e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562eb50cee40 0x562eb50f1f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:56648/0 (socket says 172.21.15.5:56648) 2024-09-10T09:39:07.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:07.903+0000 7f00f3e00640 1 -- 172.21.15.5:0/488535996 learned_addr learned my addr 172.21.15.5:0/488535996 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:39:07.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:07.903+0000 7f00f3400640 1 --2- 172.21.15.5:0/488535996 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562eb50cf760 0x562eb50f2470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:39:07.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:07.904+0000 7f00f3e00640 1 -- 172.21.15.5:0/488535996 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562eb50cf760 msgr2=0x562eb50f2470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:39:07.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:07.904+0000 7f00f3e00640 1 --2- 172.21.15.5:0/488535996 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562eb50cf760 0x562eb50f2470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:07.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:07.904+0000 7f00f3e00640 1 -- 172.21.15.5:0/488535996 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562eb50da020 msgr2=0x562eb50c9c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:39:07.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:07.904+0000 7f00f3e00640 1 --2- 172.21.15.5:0/488535996 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562eb50da020 0x562eb50c9c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:07.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:07.904+0000 7f00f3e00640 1 -- 172.21.15.5:0/488535996 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562eb50cf590 con 0x562eb50cee40 2024-09-10T09:39:07.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:07.904+0000 7f00f3400640 1 --2- 172.21.15.5:0/488535996 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562eb50cf760 0x562eb50f2470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:39:07.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:07.904+0000 7f00f8e00640 1 --2- 172.21.15.5:0/488535996 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562eb50da020 0x562eb50c9c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:39:07.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:07.904+0000 7f00f3e00640 1 --2- 172.21.15.5:0/488535996 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562eb50cee40 0x562eb50f1f30 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f00e40045d0 tx=0x7f00e40a7040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c4399c8333c1a0a2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:39:07.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:07.904+0000 7f00e8c00640 1 -- 172.21.15.5:0/488535996 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00e404a040 con 0x562eb50cee40 2024-09-10T09:39:07.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:07.904+0000 7f00e8c00640 1 -- 172.21.15.5:0/488535996 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f00e40d5070 con 0x562eb50cee40 2024-09-10T09:39:07.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:07.904+0000 7f00e8c00640 1 -- 172.21.15.5:0/488535996 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00e40ba710 con 0x562eb50cee40 2024-09-10T09:39:07.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:07.905+0000 7f00fd84d740 1 -- 172.21.15.5:0/488535996 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562eb50cee40 msgr2=0x562eb50f1f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:39:07.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:07.905+0000 7f00fd84d740 1 --2- 172.21.15.5:0/488535996 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562eb50cee40 0x562eb50f1f30 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f00e40045d0 tx=0x7f00e40a7040 comp rx=0 tx=0).stop 2024-09-10T09:39:07.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:07.905+0000 7f00fd84d740 1 -- 172.21.15.5:0/488535996 shutdown_connections 2024-09-10T09:39:07.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:07.905+0000 7f00fd84d740 1 --2- 172.21.15.5:0/488535996 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562eb50da020 0x562eb50c9c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:07.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:07.905+0000 7f00fd84d740 1 --2- 172.21.15.5:0/488535996 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562eb50cf760 0x562eb50f2470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:07.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:07.905+0000 7f00fd84d740 1 --2- 172.21.15.5:0/488535996 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562eb50cee40 0x562eb50f1f30 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:07.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:07.905+0000 7f00fd84d740 1 -- 172.21.15.5:0/488535996 >> 172.21.15.5:0/488535996 conn(0x562eb50cb3d0 msgr2=0x562eb50cb7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:39:07.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:07.905+0000 7f00fd84d740 1 -- 172.21.15.5:0/488535996 shutdown_connections 2024-09-10T09:39:07.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:07.905+0000 7f00fd84d740 1 -- 172.21.15.5:0/488535996 wait complete. 2024-09-10T09:39:07.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:07.906+0000 7f00fd84d740 1 -- start start 2024-09-10T09:39:07.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:07.906+0000 7f00fd84d740 1 Processor -- start 2024-09-10T09:39:07.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:07.907+0000 7f00fd84d740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562eb50cee40 0x562eb50dd750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:39:07.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:07.907+0000 7f00fd84d740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562eb50cf760 0x562eb50ddc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:39:07.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:07.907+0000 7f00fd84d740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562eb50da020 0x562eb514cec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:39:07.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:07.907+0000 7f00fd84d740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f00f41b2340 con 0x562eb50cf760 2024-09-10T09:39:07.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:07.907+0000 7f00fd84d740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f00f41b1c70 con 0x562eb50da020 2024-09-10T09:39:07.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:07.907+0000 7f00fd84d740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f00f41aeeb0 con 0x562eb50cee40 2024-09-10T09:39:07.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:07.907+0000 7f00fd84d740 10 client.0 _ll_register_callbacks cb 0x7f00ec009db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:39:07.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:07.907+0000 7f00f3400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562eb50cf760 0x562eb50ddc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:39:07.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:07.907+0000 7f00f3e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562eb50cee40 0x562eb50dd750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:39:07.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:07.907+0000 7f00f3400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562eb50cf760 0x562eb50ddc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36390/0 (socket says 172.21.15.5:36390) 2024-09-10T09:39:07.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:07.907+0000 7f00f3400640 1 -- 172.21.15.5:0/4044624079 learned_addr learned my addr 172.21.15.5:0/4044624079 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:39:07.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:07.907+0000 7f00f8e00640 1 --2- 172.21.15.5:0/4044624079 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562eb50da020 0x562eb514cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:39:07.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:07.907+0000 7f00f3e00640 1 -- 172.21.15.5:0/4044624079 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562eb50da020 msgr2=0x562eb514cec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:39:07.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:07.907+0000 7f00f3e00640 1 --2- 172.21.15.5:0/4044624079 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562eb50da020 0x562eb514cec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:07.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:07.907+0000 7f00f3e00640 1 -- 172.21.15.5:0/4044624079 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562eb50cf760 msgr2=0x562eb50ddc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:39:07.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:07.907+0000 7f00f3e00640 1 --2- 172.21.15.5:0/4044624079 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562eb50cf760 0x562eb50ddc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:07.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:07.907+0000 7f00f3e00640 1 -- 172.21.15.5:0/4044624079 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562eb514d500 con 0x562eb50cee40 2024-09-10T09:39:07.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:07.908+0000 7f00f3e00640 1 --2- 172.21.15.5:0/4044624079 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562eb50cee40 0x562eb50dd750 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f00e40b8ad0 tx=0x7f00e40319c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f3ed369de5887004 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:39:07.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:07.908+0000 7f00de000640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-09-10T09:39:07.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:07.908+0000 7f00de000640 1 -- 172.21.15.5:0/4044624079 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00e40d9060 con 0x562eb50cee40 2024-09-10T09:39:07.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:07.908+0000 7f00de000640 1 -- 172.21.15.5:0/4044624079 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f00e4031400 con 0x562eb50cee40 2024-09-10T09:39:07.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:07.908+0000 7f00fd84d740 1 -- 172.21.15.5:0/4044624079 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f00e40d5070 con 0x562eb50cee40 2024-09-10T09:39:07.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:07.908+0000 7f00de000640 1 -- 172.21.15.5:0/4044624079 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f00e404a040 con 0x562eb50cee40 2024-09-10T09:39:07.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:07.908+0000 7f00fd84d740 10 client.25537 fetch_fsmap learned FSMap version 30 2024-09-10T09:39:07.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:07.908+0000 7f00fd84d740 1 -- 172.21.15.5:0/4044624079 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x562eb5154890 con 0x562eb50cee40 2024-09-10T09:39:07.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:07.908+0000 7f00de000640 1 -- 172.21.15.5:0/4044624079 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f00e40d5070 con 0x562eb50cee40 2024-09-10T09:39:07.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:07.909+0000 7f00de000640 1 -- 172.21.15.5:0/4044624079 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f00e4004b50 con 0x562eb50cee40 2024-09-10T09:39:07.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:07.909+0000 7f00fd84d740 10 client.25537 fetch_fsmap finished waiting for FSMap version 30 2024-09-10T09:39:07.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:07.909+0000 7f00fd84d740 10 client.25537 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-09-10T09:39:07.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:07.909+0000 7f00fd84d740 20 client.25537 populate_metadata read hostname 'smithi005' 2024-09-10T09:39:07.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:07.909+0000 7f00fd84d740 1 --2- 172.21.15.5:0/4044624079 >> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] conn(0x562eb5154d20 0x562eb5157140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:39:07.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:07.909+0000 7f00fd84d740 4 client.25537 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T09:39:07.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:07.909+0000 7f00fd84d740 1 -- 172.21.15.5:0/4044624079 --> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562eb503ce60 con 0x562eb5154d20 2024-09-10T09:39:07.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:07.909+0000 7f00f3400640 1 --2- 172.21.15.5:0/4044624079 >> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] conn(0x562eb5154d20 0x562eb5157140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:39:07.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:07.910+0000 7f00f3400640 1 --2- 172.21.15.5:0/4044624079 >> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] conn(0x562eb5154d20 0x562eb5157140 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=c19d8e9b1787e0e5 server_cookie=ce8ae0764504a875 in_seq=0 out_seq=0 2024-09-10T09:39:07.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:07.910+0000 7f00de000640 10 client.25537 ms_handle_connect on v2:172.21.15.102:6836/2689776764 2024-09-10T09:39:07.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:07.910+0000 7f00de000640 1 -- 172.21.15.5:0/4044624079 <== mds.3 v2:172.21.15.102:6836/2689776764 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562eb503ce60 con 0x562eb5154d20 2024-09-10T09:39:07.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:07.910+0000 7f00de000640 10 client.25537 handle_command_reply: tid=0 2024-09-10T09:39:07.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:07.910+0000 7f00fd84d740 1 client.25537 shutdown 2024-09-10T09:39:07.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:07.910+0000 7f00fd84d740 1 -- 172.21.15.5:0/4044624079 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562eb50cee40 msgr2=0x562eb50dd750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:39:07.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:07.910+0000 7f00fd84d740 1 --2- 172.21.15.5:0/4044624079 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562eb50cee40 0x562eb50dd750 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f00e40b8ad0 tx=0x7f00e40319c0 comp rx=0 tx=0).stop 2024-09-10T09:39:07.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:07.910+0000 7f00fd84d740 1 -- 172.21.15.5:0/4044624079 shutdown_connections 2024-09-10T09:39:07.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:07.910+0000 7f00fd84d740 1 --2- 172.21.15.5:0/4044624079 >> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] conn(0x562eb5154d20 0x562eb5157140 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:07.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:07.910+0000 7f00fd84d740 1 --2- 172.21.15.5:0/4044624079 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562eb50da020 0x562eb514cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:07.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:07.910+0000 7f00fd84d740 1 --2- 172.21.15.5:0/4044624079 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562eb50cf760 0x562eb50ddc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:07.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:07.910+0000 7f00fd84d740 1 --2- 172.21.15.5:0/4044624079 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562eb50cee40 0x562eb50dd750 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:07.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:07.910+0000 7f00fd84d740 1 -- 172.21.15.5:0/4044624079 >> 172.21.15.5:0/4044624079 conn(0x562eb50cb3d0 msgr2=0x562eb50d8820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:39:07.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:07.910+0000 7f00de000640 0 client.25537 ms_handle_reset on v2:172.21.15.102:6836/2689776764 2024-09-10T09:39:07.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:07.911+0000 7f00fd84d740 1 -- 172.21.15.5:0/4044624079 shutdown_connections 2024-09-10T09:39:07.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:07.911+0000 7f00fd84d740 1 -- 172.21.15.5:0/4044624079 wait complete. 2024-09-10T09:39:07.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:07.911+0000 7f00fd84d740 20 client.25537 trim_cache size 0 max 16384 2024-09-10T09:39:07.916 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-10T09:39:07.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:07.911+0000 7f00f9800640 1 -- 172.21.15.5:0/4010756399 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f00d80a2c90 msgr2=0x7f00d80a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:39:07.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:07.911+0000 7f00f9800640 1 --2- 172.21.15.5:0/4010756399 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f00d80a2c90 0x7f00d80a5150 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f00ec005fd0 tx=0x7f00ec009500 comp rx=0 tx=0).stop 2024-09-10T09:39:07.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:07.911+0000 7f00f9800640 1 -- 172.21.15.5:0/4010756399 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f00f4102b90 msgr2=0x7f00f419a4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:39:07.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:07.911+0000 7f00f9800640 1 --2- 172.21.15.5:0/4010756399 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f00f4102b90 0x7f00f419a4d0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f00e4009980 tx=0x7f00e40309a0 comp rx=0 tx=0).stop 2024-09-10T09:39:07.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:07.912+0000 7f00f9800640 1 -- 172.21.15.5:0/4010756399 shutdown_connections 2024-09-10T09:39:07.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:07.912+0000 7f00f9800640 1 --2- 172.21.15.5:0/4010756399 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f00d80a2c90 0x7f00d80a5150 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:07.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:07.912+0000 7f00f9800640 1 --2- 172.21.15.5:0/4010756399 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f00f410bfc0 0x7f00f419eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:07.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:07.912+0000 7f00f9800640 1 --2- 172.21.15.5:0/4010756399 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f00f41034b0 0x7f00f419aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:07.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:07.912+0000 7f00f9800640 1 --2- 172.21.15.5:0/4010756399 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f00f4102b90 0x7f00f419a4d0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:07.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:07.912+0000 7f00f9800640 1 -- 172.21.15.5:0/4010756399 >> 172.21.15.5:0/4010756399 conn(0x7f00f40fc610 msgr2=0x7f00f410d2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:39:07.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:07.912+0000 7f00f9800640 1 -- 172.21.15.5:0/4010756399 shutdown_connections 2024-09-10T09:39:07.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:07.912+0000 7f00f9800640 1 -- 172.21.15.5:0/4010756399 wait complete. 2024-09-10T09:39:08.374 INFO:tasks.quiescer.fs.[cephfs]:Total duration (60) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-09-10T09:39:08.374 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 30.0 2024-09-10T09:39:08.375 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T09:39:08.873 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-10T09:39:09.008 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:39:09.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:39:09 smithi005 ceph-mon[30350]: pgmap v417: 129 pgs: 129 active+clean; 660 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-10T09:39:09.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:39:09 smithi159 ceph-mon[35513]: pgmap v417: 129 pgs: 129 active+clean; 660 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-10T09:39:09.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:39:09 smithi102 ceph-mon[35442]: pgmap v417: 129 pgs: 129 active+clean; 660 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-10T09:39:09.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:09.761+0000 7f351ce00640 1 -- 172.21.15.5:0/258547736 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f35181079c0 msgr2=0x7f3518107da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:39:09.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:09.761+0000 7f351ce00640 1 --2- 172.21.15.5:0/258547736 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f35181079c0 0x7f3518107da0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f3508009900 tx=0x7f350802a3f0 comp rx=0 tx=0).stop 2024-09-10T09:39:09.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:09.761+0000 7f351ce00640 1 -- 172.21.15.5:0/258547736 shutdown_connections 2024-09-10T09:39:09.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:09.761+0000 7f351ce00640 1 --2- 172.21.15.5:0/258547736 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3518102360 0x7f351810e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:09.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:09.761+0000 7f351ce00640 1 --2- 172.21.15.5:0/258547736 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f35181019c0 0x7f3518101e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:09.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:09.761+0000 7f351ce00640 1 --2- 172.21.15.5:0/258547736 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f35181079c0 0x7f3518107da0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:09.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:09.761+0000 7f351ce00640 1 -- 172.21.15.5:0/258547736 >> 172.21.15.5:0/258547736 conn(0x7f35180fd740 msgr2=0x7f35180ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:39:09.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:09.761+0000 7f351ce00640 1 -- 172.21.15.5:0/258547736 shutdown_connections 2024-09-10T09:39:09.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:09.761+0000 7f351ce00640 1 -- 172.21.15.5:0/258547736 wait complete. 2024-09-10T09:39:09.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:09.762+0000 7f351ce00640 1 Processor -- start 2024-09-10T09:39:09.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:09.763+0000 7f351ce00640 1 -- start start 2024-09-10T09:39:09.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:09.763+0000 7f351ce00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f35181019c0 0x7f351819a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:39:09.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:09.763+0000 7f351ce00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3518102360 0x7f351819a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:39:09.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:09.763+0000 7f351ce00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f35181079c0 0x7f351819ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:39:09.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:09.763+0000 7f351ce00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f3518111e10 con 0x7f3518102360 2024-09-10T09:39:09.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:09.763+0000 7f351ce00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f3518111c90 con 0x7f35181019c0 2024-09-10T09:39:09.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:09.763+0000 7f351ce00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f3518111f90 con 0x7f35181079c0 2024-09-10T09:39:09.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:09.764+0000 7f3517400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f35181019c0 0x7f351819a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:39:09.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:09.764+0000 7f3516a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3518102360 0x7f351819a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:39:09.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:09.764+0000 7f3517400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f35181019c0 0x7f351819a480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:56668/0 (socket says 172.21.15.5:56668) 2024-09-10T09:39:09.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:09.764+0000 7f3517400640 1 -- 172.21.15.5:0/1829367775 learned_addr learned my addr 172.21.15.5:0/1829367775 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:39:09.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:09.764+0000 7f3517e00640 1 --2- 172.21.15.5:0/1829367775 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f35181079c0 0x7f351819ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:39:09.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:09.764+0000 7f3517400640 1 -- 172.21.15.5:0/1829367775 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f35181079c0 msgr2=0x7f351819ed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:39:09.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:09.764+0000 7f3517400640 1 --2- 172.21.15.5:0/1829367775 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f35181079c0 0x7f351819ed50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:09.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:09.764+0000 7f3517400640 1 -- 172.21.15.5:0/1829367775 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3518102360 msgr2=0x7f351819a9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:39:09.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:09.764+0000 7f3517400640 1 --2- 172.21.15.5:0/1829367775 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3518102360 0x7f351819a9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:09.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:09.764+0000 7f3517400640 1 -- 172.21.15.5:0/1829367775 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f351819f430 con 0x7f35181019c0 2024-09-10T09:39:09.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:09.765+0000 7f3517400640 1 --2- 172.21.15.5:0/1829367775 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f35181019c0 0x7f351819a480 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f350802a9e0 tx=0x7f3508003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:39:09.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:09.765+0000 7f350fe00640 1 -- 172.21.15.5:0/1829367775 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f350803f070 con 0x7f35181019c0 2024-09-10T09:39:09.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:09.765+0000 7f350fe00640 1 -- 172.21.15.5:0/1829367775 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3508031c30 con 0x7f35181019c0 2024-09-10T09:39:09.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:09.765+0000 7f350fe00640 1 -- 172.21.15.5:0/1829367775 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3508035030 con 0x7f35181019c0 2024-09-10T09:39:09.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:09.766+0000 7f351ce00640 1 -- 172.21.15.5:0/1829367775 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f351819f6c0 con 0x7f35181019c0 2024-09-10T09:39:09.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:09.766+0000 7f351ce00640 1 -- 172.21.15.5:0/1829367775 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f351819fb20 con 0x7f35181019c0 2024-09-10T09:39:09.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:09.767+0000 7f351ce00640 1 -- 172.21.15.5:0/1829367775 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3518101e20 con 0x7f35181019c0 2024-09-10T09:39:09.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:09.769+0000 7f350fe00640 1 -- 172.21.15.5:0/1829367775 <== mon.2 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 19) ==== 137883+0+0 (secure 0 0 0) 0x7f3508042070 con 0x7f35181019c0 2024-09-10T09:39:09.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:09.771+0000 7f350fe00640 1 --2- 172.21.15.5:0/1829367775 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f34fc0a2c90 0x7f34fc0a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:39:09.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:09.771+0000 7f350fe00640 1 -- 172.21.15.5:0/1829367775 <== mon.2 v2:172.21.15.102:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f35080f0300 con 0x7f35181019c0 2024-09-10T09:39:09.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:09.771+0000 7f3516a00640 1 --2- 172.21.15.5:0/1829367775 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f34fc0a2c90 0x7f34fc0a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:39:09.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:09.772+0000 7f3516a00640 1 --2- 172.21.15.5:0/1829367775 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f34fc0a2c90 0x7f34fc0a5150 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f351810ea80 tx=0x7f3504005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:39:09.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:09.773+0000 7f350fe00640 1 -- 172.21.15.5:0/1829367775 <== mon.2 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f35080bccb0 con 0x7f35181019c0 2024-09-10T09:39:09.825 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:39:10.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:10.016+0000 7f351ce00640 1 -- 172.21.15.5:0/1829367775 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f35181a71c0 con 0x7f35181019c0 2024-09-10T09:39:10.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:10.017+0000 7f350fe00640 1 -- 172.21.15.5:0/1829367775 <== mon.2 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11969 (secure 0 0 0) 0x7f35080bc5b0 con 0x7f35181019c0 2024-09-10T09:39:10.019 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-10T09:39:10.020 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":30,"btime":"2024-09-10T09:38:15:834198+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14454,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/2970594311","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2970594311},{"type":"v1","addr":"172.21.15.5:6835","nonce":2970594311}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14514,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6839/3616502174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":3616502174},{"type":"v1","addr":"172.21.15.5:6839","nonce":3616502174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T09:35:29.626287+0000","modified":"2024-09-10T09:38:15.658058+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24542,"mds_1":24535,"mds_2":24511,"mds_3":24506,"mds_4":24443},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14520":{"gid":14520,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/2073599842","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":2073599842},{"type":"v1","addr":"172.21.15.5:6841","nonce":2073599842}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"k","rank":4,"incarnation":26,"state":"up:active","state_seq":24,"addr":"172.21.15.102:6833/2457007427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":2457007427},{"type":"v1","addr":"172.21.15.102:6833","nonce":2457007427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/356445159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":356445159},{"type":"v1","addr":"172.21.15.159:6833","nonce":356445159}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6837/2894314507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":2894314507},{"type":"v1","addr":"172.21.15.5:6837","nonce":2894314507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.102:6835/1647549111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":1647549111},{"type":"v1","addr":"172.21.15.102:6835","nonce":1647549111}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6835/952657242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":952657242},{"type":"v1","addr":"172.21.15.159:6835","nonce":952657242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":19,"addr":"172.21.15.102:6837/2689776764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":2689776764},{"type":"v1","addr":"172.21.15.102:6837","nonce":2689776764}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.159:6837/620627639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":620627639},{"type":"v1","addr":"172.21.15.159:6837","nonce":620627639}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24535":{"gid":24535,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":18,"addr":"172.21.15.159:6839/2073739599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":2073739599},{"type":"v1","addr":"172.21.15.159:6839","nonce":2073739599}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24542":{"gid":24542,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.102:6839/335409749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6838","nonce":335409749},{"type":"v1","addr":"172.21.15.102:6839","nonce":335409749}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24542,"qdb_cluster":[24542,24535,24511,24506,24443]},"id":1}]} 2024-09-10T09:39:10.020 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 30 2024-09-10T09:39:10.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:10.021+0000 7f351ce00640 1 -- 172.21.15.5:0/1829367775 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f34fc0a2c90 msgr2=0x7f34fc0a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:39:10.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:10.021+0000 7f351ce00640 1 --2- 172.21.15.5:0/1829367775 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f34fc0a2c90 0x7f34fc0a5150 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f351810ea80 tx=0x7f3504005f90 comp rx=0 tx=0).stop 2024-09-10T09:39:10.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:10.021+0000 7f351ce00640 1 -- 172.21.15.5:0/1829367775 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f35181019c0 msgr2=0x7f351819a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:39:10.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:10.021+0000 7f351ce00640 1 --2- 172.21.15.5:0/1829367775 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f35181019c0 0x7f351819a480 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f350802a9e0 tx=0x7f3508003c10 comp rx=0 tx=0).stop 2024-09-10T09:39:10.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:10.021+0000 7f351ce00640 1 -- 172.21.15.5:0/1829367775 shutdown_connections 2024-09-10T09:39:10.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:10.021+0000 7f351ce00640 1 --2- 172.21.15.5:0/1829367775 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f34fc0a2c90 0x7f34fc0a5150 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:10.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:10.021+0000 7f351ce00640 1 --2- 172.21.15.5:0/1829367775 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f35181079c0 0x7f351819ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:10.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:10.021+0000 7f351ce00640 1 --2- 172.21.15.5:0/1829367775 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3518102360 0x7f351819a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:10.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:10.021+0000 7f351ce00640 1 --2- 172.21.15.5:0/1829367775 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f35181019c0 0x7f351819a480 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:10.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:10.022+0000 7f351ce00640 1 -- 172.21.15.5:0/1829367775 >> 172.21.15.5:0/1829367775 conn(0x7f35180fd740 msgr2=0x7f351810cfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:39:10.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:10.022+0000 7f351ce00640 1 -- 172.21.15.5:0/1829367775 shutdown_connections 2024-09-10T09:39:10.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:10.022+0000 7f351ce00640 1 -- 172.21.15.5:0/1829367775 wait complete. 2024-09-10T09:39:10.306 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:39:10 smithi005 ceph-mon[30350]: from='client.? 172.21.15.5:0/1829367775' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:39:10.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:39:10 smithi159 ceph-mon[35513]: from='client.? 172.21.15.5:0/1829367775' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:39:10.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:39:10 smithi102 ceph-mon[35442]: from='client.? 172.21.15.5:0/1829367775' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:39:10.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:10.679+0000 7fae44600640 1 -- 172.21.15.5:0/1711646005 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fae3c104010 msgr2=0x7fae3c1043f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:39:10.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:10.679+0000 7fae44600640 1 --2- 172.21.15.5:0/1711646005 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fae3c104010 0x7fae3c1043f0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fae30009900 tx=0x7fae3002a3f0 comp rx=0 tx=0).stop 2024-09-10T09:39:10.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:10.679+0000 7fae44600640 1 -- 172.21.15.5:0/1711646005 shutdown_connections 2024-09-10T09:39:10.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:10.679+0000 7fae44600640 1 --2- 172.21.15.5:0/1711646005 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fae3c109140 0x7fae3c110cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:10.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:10.679+0000 7fae44600640 1 --2- 172.21.15.5:0/1711646005 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fae3c1049c0 0x7fae3c108a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:10.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:10.679+0000 7fae44600640 1 --2- 172.21.15.5:0/1711646005 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fae3c104010 0x7fae3c1043f0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:10.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:10.679+0000 7fae44600640 1 -- 172.21.15.5:0/1711646005 >> 172.21.15.5:0/1711646005 conn(0x7fae3c0ffa50 msgr2=0x7fae3c101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:39:10.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:10.679+0000 7fae44600640 1 -- 172.21.15.5:0/1711646005 shutdown_connections 2024-09-10T09:39:10.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:10.680+0000 7fae44600640 1 -- 172.21.15.5:0/1711646005 wait complete. 2024-09-10T09:39:10.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:10.681+0000 7fae44600640 1 Processor -- start 2024-09-10T09:39:10.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:10.681+0000 7fae44600640 1 -- start start 2024-09-10T09:39:10.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:10.681+0000 7fae44600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fae3c104010 0x7fae3c19a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:39:10.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:10.681+0000 7fae44600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fae3c1049c0 0x7fae3c19ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:39:10.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:10.681+0000 7fae44600640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fae3c109140 0x7fae3c19ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:39:10.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:10.681+0000 7fae44600640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fae3c1141c0 con 0x7fae3c104010 2024-09-10T09:39:10.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:10.681+0000 7fae44600640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fae3c114040 con 0x7fae3c1049c0 2024-09-10T09:39:10.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:10.681+0000 7fae44600640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fae3c114340 con 0x7fae3c109140 2024-09-10T09:39:10.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:10.682+0000 7fae41c00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fae3c104010 0x7fae3c19a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:39:10.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:10.682+0000 7fae41200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fae3c1049c0 0x7fae3c19ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:39:10.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:10.682+0000 7fae41c00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fae3c104010 0x7fae3c19a640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36440/0 (socket says 172.21.15.5:36440) 2024-09-10T09:39:10.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:10.682+0000 7fae41c00640 1 -- 172.21.15.5:0/1960919457 learned_addr learned my addr 172.21.15.5:0/1960919457 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:39:10.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:10.682+0000 7fae42600640 1 --2- 172.21.15.5:0/1960919457 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fae3c109140 0x7fae3c19ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:39:10.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:10.682+0000 7fae41200640 1 -- 172.21.15.5:0/1960919457 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fae3c109140 msgr2=0x7fae3c19ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:39:10.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:10.682+0000 7fae41200640 1 --2- 172.21.15.5:0/1960919457 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fae3c109140 0x7fae3c19ef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:10.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:10.682+0000 7fae41200640 1 -- 172.21.15.5:0/1960919457 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fae3c104010 msgr2=0x7fae3c19a640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:39:10.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:10.682+0000 7fae41200640 1 --2- 172.21.15.5:0/1960919457 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fae3c104010 0x7fae3c19a640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:10.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:10.682+0000 7fae41200640 1 -- 172.21.15.5:0/1960919457 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae3c19f5f0 con 0x7fae3c1049c0 2024-09-10T09:39:10.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:10.683+0000 7fae41200640 1 --2- 172.21.15.5:0/1960919457 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fae3c1049c0 0x7fae3c19ab80 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fae2800e9f0 tx=0x7fae2800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:39:10.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:10.683+0000 7fae3a000640 1 -- 172.21.15.5:0/1960919457 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae2800ccc0 con 0x7fae3c1049c0 2024-09-10T09:39:10.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:10.683+0000 7fae44600640 1 -- 172.21.15.5:0/1960919457 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae3c19f8e0 con 0x7fae3c1049c0 2024-09-10T09:39:10.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:10.683+0000 7fae3a000640 1 -- 172.21.15.5:0/1960919457 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fae28010910 con 0x7fae3c1049c0 2024-09-10T09:39:10.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:10.683+0000 7fae44600640 1 -- 172.21.15.5:0/1960919457 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae3c1a71c0 con 0x7fae3c1049c0 2024-09-10T09:39:10.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:10.683+0000 7fae3a000640 1 -- 172.21.15.5:0/1960919457 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae28018960 con 0x7fae3c1049c0 2024-09-10T09:39:10.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:10.687+0000 7fae46ffb740 1 Processor -- start 2024-09-10T09:39:10.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:10.687+0000 7fae46ffb740 1 -- start start 2024-09-10T09:39:10.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:10.687+0000 7fae46ffb740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fedbcedc70 0x55fedbcee050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:39:10.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:10.687+0000 7fae46ffb740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fedbcee590 0x55fedbcf8910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:39:10.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:10.687+0000 7fae46ffb740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fedbcf8e50 0x55fedbcfb240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:39:10.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:10.687+0000 7fae46ffb740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55fedbc543e0 con 0x55fedbcedc70 2024-09-10T09:39:10.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:10.687+0000 7fae46ffb740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55fedbbfbd80 con 0x55fedbcee590 2024-09-10T09:39:10.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:10.688+0000 7fae46ffb740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55fedbc503a0 con 0x55fedbcf8e50 2024-09-10T09:39:10.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:10.688+0000 7fae42600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fedbcedc70 0x55fedbcee050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:39:10.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:10.688+0000 7fae42600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fedbcedc70 0x55fedbcee050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36450/0 (socket says 172.21.15.5:36450) 2024-09-10T09:39:10.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:10.688+0000 7fae42600640 1 -- 172.21.15.5:0/1655572682 learned_addr learned my addr 172.21.15.5:0/1655572682 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:39:10.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:10.689+0000 7fae3a000640 1 -- 172.21.15.5:0/1960919457 <== mon.2 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 19) ==== 137883+0+0 (secure 0 0 0) 0x7fae28018b00 con 0x7fae3c1049c0 2024-09-10T09:39:10.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:10.689+0000 7fae41c00640 1 --2- 172.21.15.5:0/1655572682 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fedbcee590 0x55fedbcf8910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:39:10.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:10.690+0000 7fae41200640 1 --2- 172.21.15.5:0/1655572682 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fedbcf8e50 0x55fedbcfb240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:39:10.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:10.690+0000 7fae42600640 1 -- 172.21.15.5:0/1655572682 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fedbcf8e50 msgr2=0x55fedbcfb240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:39:10.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:10.690+0000 7fae42600640 1 --2- 172.21.15.5:0/1655572682 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fedbcf8e50 0x55fedbcfb240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:10.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:10.690+0000 7fae42600640 1 -- 172.21.15.5:0/1655572682 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fedbcee590 msgr2=0x55fedbcf8910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:39:10.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:10.690+0000 7fae42600640 1 --2- 172.21.15.5:0/1655572682 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fedbcee590 0x55fedbcf8910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:10.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:10.690+0000 7fae42600640 1 -- 172.21.15.5:0/1655572682 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fedbc1aae0 con 0x55fedbcedc70 2024-09-10T09:39:10.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:10.690+0000 7fae41200640 1 --2- 172.21.15.5:0/1655572682 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fedbcf8e50 0x55fedbcfb240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T09:39:10.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:10.690+0000 7fae42600640 1 --2- 172.21.15.5:0/1655572682 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fedbcedc70 0x55fedbcee050 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fae34009fa0 tx=0x7fae340118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebb8cdcd34bf282f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:39:10.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:10.691+0000 7fae2ea00640 1 -- 172.21.15.5:0/1655572682 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae34004d20 con 0x55fedbcedc70 2024-09-10T09:39:10.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:10.691+0000 7fae2ea00640 1 -- 172.21.15.5:0/1655572682 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fae34004050 con 0x55fedbcedc70 2024-09-10T09:39:10.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:10.691+0000 7fae2ea00640 1 -- 172.21.15.5:0/1655572682 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae34005280 con 0x55fedbcedc70 2024-09-10T09:39:10.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:10.691+0000 7fae3a000640 1 --2- 172.21.15.5:0/1960919457 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fae240a2c90 0x7fae240a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:39:10.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:10.692+0000 7fae3a000640 1 -- 172.21.15.5:0/1960919457 <== mon.2 v2:172.21.15.102:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fae280cf8b0 con 0x7fae3c1049c0 2024-09-10T09:39:10.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:10.692+0000 7fae41c00640 1 --2- 172.21.15.5:0/1960919457 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fae240a2c90 0x7fae240a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:39:10.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:10.692+0000 7fae46ffb740 1 -- 172.21.15.5:0/1655572682 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fedbcedc70 msgr2=0x55fedbcee050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:39:10.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:10.692+0000 7fae46ffb740 1 --2- 172.21.15.5:0/1655572682 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fedbcedc70 0x55fedbcee050 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fae34009fa0 tx=0x7fae340118d0 comp rx=0 tx=0).stop 2024-09-10T09:39:10.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:10.692+0000 7fae46ffb740 1 -- 172.21.15.5:0/1655572682 shutdown_connections 2024-09-10T09:39:10.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:10.692+0000 7fae46ffb740 1 --2- 172.21.15.5:0/1655572682 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fedbcf8e50 0x55fedbcfb240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:10.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:10.692+0000 7fae46ffb740 1 --2- 172.21.15.5:0/1655572682 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fedbcee590 0x55fedbcf8910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:10.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:10.692+0000 7fae46ffb740 1 --2- 172.21.15.5:0/1655572682 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fedbcedc70 0x55fedbcee050 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:10.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:10.692+0000 7fae46ffb740 1 -- 172.21.15.5:0/1655572682 >> 172.21.15.5:0/1655572682 conn(0x55fedbcea200 msgr2=0x55fedbcea5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:39:10.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:10.693+0000 7fae46ffb740 1 -- 172.21.15.5:0/1655572682 shutdown_connections 2024-09-10T09:39:10.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:10.693+0000 7fae46ffb740 1 -- 172.21.15.5:0/1655572682 wait complete. 2024-09-10T09:39:10.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:10.693+0000 7fae41c00640 1 --2- 172.21.15.5:0/1960919457 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fae240a2c90 0x7fae240a5150 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fae300096c0 tx=0x7fae300046c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:39:10.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:10.693+0000 7fae46ffb740 1 -- start start 2024-09-10T09:39:10.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:10.694+0000 7fae46ffb740 1 Processor -- start 2024-09-10T09:39:10.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:10.694+0000 7fae46ffb740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fedbcedc70 0x55fedbcf5d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:39:10.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:10.694+0000 7fae46ffb740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fedbcee590 0x55fedbcf6250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:39:10.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:10.694+0000 7fae46ffb740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fedbcf8e50 0x55fedbcf6790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:39:10.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:10.694+0000 7fae46ffb740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55fedbd0ca30 con 0x55fedbcedc70 2024-09-10T09:39:10.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:10.694+0000 7fae46ffb740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55fedbc0f0f0 con 0x55fedbcf8e50 2024-09-10T09:39:10.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:10.694+0000 7fae46ffb740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55fedbd0cbb0 con 0x55fedbcee590 2024-09-10T09:39:10.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:10.694+0000 7fae46ffb740 10 client.0 _ll_register_callbacks cb 0x55fedbbf6bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:39:10.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:10.695+0000 7fae41c00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fedbcedc70 0x55fedbcf5d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:39:10.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:10.695+0000 7fae42600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fedbcf8e50 0x55fedbcf6790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:39:10.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:10.695+0000 7fae41200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fedbcee590 0x55fedbcf6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:39:10.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:10.695+0000 7fae42600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fedbcf8e50 0x55fedbcf6790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:56730/0 (socket says 172.21.15.5:56730) 2024-09-10T09:39:10.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:10.695+0000 7fae42600640 1 -- 172.21.15.5:0/1048501898 learned_addr learned my addr 172.21.15.5:0/1048501898 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:39:10.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:10.696+0000 7fae42600640 1 -- 172.21.15.5:0/1048501898 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fedbcee590 msgr2=0x55fedbcf6250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:39:10.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:10.696+0000 7fae42600640 1 --2- 172.21.15.5:0/1048501898 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fedbcee590 0x55fedbcf6250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:10.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:10.696+0000 7fae42600640 1 -- 172.21.15.5:0/1048501898 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fedbcedc70 msgr2=0x55fedbcf5d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:39:10.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:10.696+0000 7fae42600640 1 --2- 172.21.15.5:0/1048501898 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fedbcedc70 0x55fedbcf5d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:10.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:10.696+0000 7fae42600640 1 -- 172.21.15.5:0/1048501898 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fedb9c53c0 con 0x55fedbcf8e50 2024-09-10T09:39:10.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:10.696+0000 7fae42600640 1 --2- 172.21.15.5:0/1048501898 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fedbcf8e50 0x55fedbcf6790 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fae3401b040 tx=0x7fae34004280 comp rx=0 tx=0).ready entity=mon.2 client_cookie=94321a0679cf3cef server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:39:10.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:10.696+0000 7fae23400640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-09-10T09:39:10.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:10.696+0000 7fae23400640 1 -- 172.21.15.5:0/1048501898 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae34028070 con 0x55fedbcf8e50 2024-09-10T09:39:10.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:10.697+0000 7fae23400640 1 -- 172.21.15.5:0/1048501898 <== mon.2 v2:172.21.15.102:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fae34003c30 con 0x55fedbcf8e50 2024-09-10T09:39:10.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:10.697+0000 7fae46ffb740 1 -- 172.21.15.5:0/1048501898 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fae34004050 con 0x55fedbcf8e50 2024-09-10T09:39:10.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:10.697+0000 7fae23400640 1 -- 172.21.15.5:0/1048501898 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fae34034050 con 0x55fedbcf8e50 2024-09-10T09:39:10.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:10.697+0000 7fae46ffb740 10 client.25529 fetch_fsmap learned FSMap version 30 2024-09-10T09:39:10.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:10.697+0000 7fae46ffb740 1 -- 172.21.15.5:0/1048501898 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55fedbd18a40 con 0x55fedbcf8e50 2024-09-10T09:39:10.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:10.698+0000 7fae23400640 1 -- 172.21.15.5:0/1048501898 <== mon.2 v2:172.21.15.102:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fae34004050 con 0x55fedbcf8e50 2024-09-10T09:39:10.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:10.698+0000 7fae23400640 1 -- 172.21.15.5:0/1048501898 <== mon.2 v2:172.21.15.102:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fae34024460 con 0x55fedbcf8e50 2024-09-10T09:39:10.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:10.698+0000 7fae46ffb740 10 client.25529 fetch_fsmap finished waiting for FSMap version 30 2024-09-10T09:39:10.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:10.698+0000 7fae46ffb740 10 client.25529 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-10T09:39:10.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:10.698+0000 7fae46ffb740 20 client.25529 populate_metadata read hostname 'smithi005' 2024-09-10T09:39:10.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:10.698+0000 7fae46ffb740 1 --2- 172.21.15.5:0/1048501898 >> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] conn(0x55fedbd19df0 0x55fedbd1c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:39:10.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:10.698+0000 7fae46ffb740 4 client.25529 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:39:10.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:10.698+0000 7fae46ffb740 1 -- 172.21.15.5:0/1048501898 --> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fedbc61950 con 0x55fedbd19df0 2024-09-10T09:39:10.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:10.698+0000 7fae41c00640 1 --2- 172.21.15.5:0/1048501898 >> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] conn(0x55fedbd19df0 0x55fedbd1c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:39:10.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:10.699+0000 7fae41c00640 1 --2- 172.21.15.5:0/1048501898 >> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] conn(0x55fedbd19df0 0x55fedbd1c1e0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7f447e07564c0f75 server_cookie=12d03218f9d19b2d in_seq=0 out_seq=0 2024-09-10T09:39:10.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:10.699+0000 7fae23400640 10 client.25529 ms_handle_connect on v2:172.21.15.159:6836/620627639 2024-09-10T09:39:10.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:10.700+0000 7fae23400640 1 -- 172.21.15.5:0/1048501898 <== mds.2 v2:172.21.15.159:6836/620627639 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55fedbc61950 con 0x55fedbd19df0 2024-09-10T09:39:10.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:10.700+0000 7fae23400640 10 client.25529 handle_command_reply: tid=0 2024-09-10T09:39:10.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:10.700+0000 7fae46ffb740 1 client.25529 shutdown 2024-09-10T09:39:10.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:10.700+0000 7fae46ffb740 1 -- 172.21.15.5:0/1048501898 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fedbcf8e50 msgr2=0x55fedbcf6790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:39:10.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:10.700+0000 7fae46ffb740 1 --2- 172.21.15.5:0/1048501898 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fedbcf8e50 0x55fedbcf6790 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fae3401b040 tx=0x7fae34004280 comp rx=0 tx=0).stop 2024-09-10T09:39:10.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:10.700+0000 7fae46ffb740 1 -- 172.21.15.5:0/1048501898 shutdown_connections 2024-09-10T09:39:10.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:10.700+0000 7fae46ffb740 1 --2- 172.21.15.5:0/1048501898 >> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] conn(0x55fedbd19df0 0x55fedbd1c1e0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:10.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:10.700+0000 7fae46ffb740 1 --2- 172.21.15.5:0/1048501898 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fedbcf8e50 0x55fedbcf6790 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:10.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:10.700+0000 7fae46ffb740 1 --2- 172.21.15.5:0/1048501898 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fedbcee590 0x55fedbcf6250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:10.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:10.700+0000 7fae46ffb740 1 --2- 172.21.15.5:0/1048501898 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fedbcedc70 0x55fedbcf5d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:10.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:10.700+0000 7fae23400640 0 client.25529 ms_handle_reset on v2:172.21.15.159:6836/620627639 2024-09-10T09:39:10.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:10.700+0000 7fae46ffb740 1 -- 172.21.15.5:0/1048501898 >> 172.21.15.5:0/1048501898 conn(0x55fedbcea200 msgr2=0x55fedbceff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:39:10.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:10.701+0000 7fae46ffb740 1 -- 172.21.15.5:0/1048501898 shutdown_connections 2024-09-10T09:39:10.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:10.701+0000 7fae46ffb740 1 -- 172.21.15.5:0/1048501898 wait complete. 2024-09-10T09:39:10.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:10.701+0000 7fae46ffb740 20 client.25529 trim_cache size 0 max 16384 2024-09-10T09:39:10.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:10.715+0000 7fae46ffb740 1 Processor -- start 2024-09-10T09:39:10.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:10.715+0000 7fae46ffb740 1 -- start start 2024-09-10T09:39:10.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:10.715+0000 7fae46ffb740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fedbcedc70 0x55fedbcf4c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:39:10.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:10.715+0000 7fae46ffb740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fedbcee590 0x55fedbcf06d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:39:10.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:10.715+0000 7fae46ffb740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fedbcf8e50 0x55fedbcf65b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:39:10.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:10.715+0000 7fae46ffb740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fae1c00fcd0 con 0x55fedbcee590 2024-09-10T09:39:10.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:10.715+0000 7fae46ffb740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fae1c011810 con 0x55fedbcf8e50 2024-09-10T09:39:10.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:10.715+0000 7fae46ffb740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fae1c0103a0 con 0x55fedbcedc70 2024-09-10T09:39:10.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:10.716+0000 7fae41c00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fedbcedc70 0x55fedbcf4c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:39:10.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:10.716+0000 7fae41c00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fedbcedc70 0x55fedbcf4c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:54456/0 (socket says 172.21.15.5:54456) 2024-09-10T09:39:10.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:10.716+0000 7fae41c00640 1 -- 172.21.15.5:0/4185839099 learned_addr learned my addr 172.21.15.5:0/4185839099 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:39:10.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:10.716+0000 7fae41200640 1 --2- 172.21.15.5:0/4185839099 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fedbcee590 0x55fedbcf06d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:39:10.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:10.716+0000 7fae42600640 1 --2- 172.21.15.5:0/4185839099 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fedbcf8e50 0x55fedbcf65b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:39:10.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:10.716+0000 7fae41c00640 1 -- 172.21.15.5:0/4185839099 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fedbcf8e50 msgr2=0x55fedbcf65b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:39:10.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:10.716+0000 7fae41c00640 1 --2- 172.21.15.5:0/4185839099 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fedbcf8e50 0x55fedbcf65b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:10.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:10.716+0000 7fae41c00640 1 -- 172.21.15.5:0/4185839099 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fedbcee590 msgr2=0x55fedbcf06d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:39:10.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:10.716+0000 7fae41c00640 1 --2- 172.21.15.5:0/4185839099 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fedbcee590 0x55fedbcf06d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:10.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:10.716+0000 7fae41c00640 1 -- 172.21.15.5:0/4185839099 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fedbcedaa0 con 0x55fedbcedc70 2024-09-10T09:39:10.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:10.716+0000 7fae41200640 1 --2- 172.21.15.5:0/4185839099 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fedbcee590 0x55fedbcf06d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:39:10.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:10.716+0000 7fae41c00640 1 --2- 172.21.15.5:0/4185839099 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fedbcedc70 0x55fedbcf4c50 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fae30004770 tx=0x7fae3003e000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a8f0c81a8c0221a4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:39:10.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:10.717+0000 7fae2e000640 1 -- 172.21.15.5:0/4185839099 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae30035040 con 0x55fedbcedc70 2024-09-10T09:39:10.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:10.717+0000 7fae2e000640 1 -- 172.21.15.5:0/4185839099 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fae30058070 con 0x55fedbcedc70 2024-09-10T09:39:10.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:10.717+0000 7fae2e000640 1 -- 172.21.15.5:0/4185839099 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae300537c0 con 0x55fedbcedc70 2024-09-10T09:39:10.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:10.718+0000 7fae46ffb740 1 -- 172.21.15.5:0/4185839099 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fedbcedc70 msgr2=0x55fedbcf4c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:39:10.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:10.718+0000 7fae46ffb740 1 --2- 172.21.15.5:0/4185839099 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fedbcedc70 0x55fedbcf4c50 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fae30004770 tx=0x7fae3003e000 comp rx=0 tx=0).stop 2024-09-10T09:39:10.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:10.718+0000 7fae46ffb740 1 -- 172.21.15.5:0/4185839099 shutdown_connections 2024-09-10T09:39:10.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:10.718+0000 7fae46ffb740 1 --2- 172.21.15.5:0/4185839099 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fedbcf8e50 0x55fedbcf65b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:10.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:10.718+0000 7fae46ffb740 1 --2- 172.21.15.5:0/4185839099 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fedbcee590 0x55fedbcf06d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:10.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:10.718+0000 7fae46ffb740 1 --2- 172.21.15.5:0/4185839099 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fedbcedc70 0x55fedbcf4c50 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:10.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:10.718+0000 7fae46ffb740 1 -- 172.21.15.5:0/4185839099 >> 172.21.15.5:0/4185839099 conn(0x55fedbcea200 msgr2=0x55fedbcf7d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:39:10.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:10.718+0000 7fae46ffb740 1 -- 172.21.15.5:0/4185839099 shutdown_connections 2024-09-10T09:39:10.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:10.718+0000 7fae46ffb740 1 -- 172.21.15.5:0/4185839099 wait complete. 2024-09-10T09:39:10.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:10.718+0000 7fae46ffb740 1 -- start start 2024-09-10T09:39:10.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:10.719+0000 7fae46ffb740 1 Processor -- start 2024-09-10T09:39:10.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:10.719+0000 7fae46ffb740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fedbcedc70 0x55fedbd0d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:39:10.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:10.719+0000 7fae46ffb740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fedbcee590 0x55fedbd0d930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:39:10.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:10.719+0000 7fae46ffb740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fedbcf8e50 0x55fedbd82950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:39:10.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:10.719+0000 7fae46ffb740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fae1c011140 con 0x55fedbcee590 2024-09-10T09:39:10.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:10.719+0000 7fae46ffb740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fae1c010a70 con 0x55fedbcf8e50 2024-09-10T09:39:10.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:10.719+0000 7fae46ffb740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fae1c00dcb0 con 0x55fedbcedc70 2024-09-10T09:39:10.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:10.719+0000 7fae46ffb740 10 client.0 _ll_register_callbacks cb 0x7fae34009e20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:39:10.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:10.719+0000 7fae41200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fedbcee590 0x55fedbd0d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:39:10.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:10.719+0000 7fae41200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fedbcee590 0x55fedbd0d930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36476/0 (socket says 172.21.15.5:36476) 2024-09-10T09:39:10.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:10.719+0000 7fae41200640 1 -- 172.21.15.5:0/24953783 learned_addr learned my addr 172.21.15.5:0/24953783 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:39:10.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:10.720+0000 7fae42600640 1 --2- 172.21.15.5:0/24953783 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fedbcf8e50 0x55fedbd82950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:39:10.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:10.720+0000 7fae41c00640 1 --2- 172.21.15.5:0/24953783 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fedbcedc70 0x55fedbd0d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:39:10.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:10.720+0000 7fae41200640 1 -- 172.21.15.5:0/24953783 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fedbcedc70 msgr2=0x55fedbd0d3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:39:10.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:10.720+0000 7fae41200640 1 --2- 172.21.15.5:0/24953783 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fedbcedc70 0x55fedbd0d3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:10.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:10.720+0000 7fae41200640 1 -- 172.21.15.5:0/24953783 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fedbcf8e50 msgr2=0x55fedbd82950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:39:10.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:10.720+0000 7fae41200640 1 --2- 172.21.15.5:0/24953783 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fedbcf8e50 0x55fedbd82950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:10.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:10.720+0000 7fae41200640 1 -- 172.21.15.5:0/24953783 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fedbd82f90 con 0x55fedbcee590 2024-09-10T09:39:10.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:10.720+0000 7fae41c00640 1 --2- 172.21.15.5:0/24953783 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fedbcedc70 0x55fedbd0d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T09:39:10.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:10.720+0000 7fae41200640 1 --2- 172.21.15.5:0/24953783 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fedbcee590 0x55fedbd0d930 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fae2809bba0 tx=0x7fae2800cf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc56af494cc41620 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:39:10.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:10.720+0000 7fae23400640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-10T09:39:10.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:10.720+0000 7fae23400640 1 -- 172.21.15.5:0/24953783 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae280a47d0 con 0x55fedbcee590 2024-09-10T09:39:10.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:10.721+0000 7fae23400640 1 -- 172.21.15.5:0/24953783 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fae280cd060 con 0x55fedbcee590 2024-09-10T09:39:10.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:10.721+0000 7fae46ffb740 1 -- 172.21.15.5:0/24953783 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fae30058070 con 0x55fedbcee590 2024-09-10T09:39:10.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:10.721+0000 7fae23400640 1 -- 172.21.15.5:0/24953783 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fae280ba2e0 con 0x55fedbcee590 2024-09-10T09:39:10.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:10.721+0000 7fae46ffb740 10 client.15528 fetch_fsmap learned FSMap version 30 2024-09-10T09:39:10.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:10.721+0000 7fae46ffb740 1 -- 172.21.15.5:0/24953783 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55fedbd842d0 con 0x55fedbcee590 2024-09-10T09:39:10.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:10.721+0000 7fae23400640 1 -- 172.21.15.5:0/24953783 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fae30058070 con 0x55fedbcee590 2024-09-10T09:39:10.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:10.721+0000 7fae23400640 1 -- 172.21.15.5:0/24953783 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fae280cfce0 con 0x55fedbcee590 2024-09-10T09:39:10.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:10.722+0000 7fae46ffb740 10 client.15528 fetch_fsmap finished waiting for FSMap version 30 2024-09-10T09:39:10.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:10.722+0000 7fae46ffb740 10 client.15528 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-10T09:39:10.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:10.722+0000 7fae46ffb740 20 client.15528 populate_metadata read hostname 'smithi005' 2024-09-10T09:39:10.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:10.722+0000 7fae46ffb740 1 --2- 172.21.15.5:0/24953783 >> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] conn(0x55fedbd84a80 0x55fedbd86ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:39:10.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:10.722+0000 7fae46ffb740 4 client.15528 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T09:39:10.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:10.722+0000 7fae46ffb740 1 -- 172.21.15.5:0/24953783 --> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55fedbc5be60 con 0x55fedbd84a80 2024-09-10T09:39:10.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:10.722+0000 7fae41c00640 1 --2- 172.21.15.5:0/24953783 >> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] conn(0x55fedbd84a80 0x55fedbd86ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:39:10.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:10.723+0000 7fae41c00640 1 --2- 172.21.15.5:0/24953783 >> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] conn(0x55fedbd84a80 0x55fedbd86ea0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b802a24d09d31b32 server_cookie=82953e3b9dfec130 in_seq=0 out_seq=0 2024-09-10T09:39:10.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:10.723+0000 7fae23400640 10 client.15528 ms_handle_connect on v2:172.21.15.159:6836/620627639 2024-09-10T09:39:10.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:10.723+0000 7fae23400640 1 -- 172.21.15.5:0/24953783 <== mds.2 v2:172.21.15.159:6836/620627639 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55fedbc5be60 con 0x55fedbd84a80 2024-09-10T09:39:10.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:10.723+0000 7fae23400640 10 client.15528 handle_command_reply: tid=0 2024-09-10T09:39:10.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:10.723+0000 7fae46ffb740 1 client.15528 shutdown 2024-09-10T09:39:10.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:10.723+0000 7fae46ffb740 1 -- 172.21.15.5:0/24953783 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fedbcee590 msgr2=0x55fedbd0d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:39:10.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:10.723+0000 7fae46ffb740 1 --2- 172.21.15.5:0/24953783 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fedbcee590 0x55fedbd0d930 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fae2809bba0 tx=0x7fae2800cf20 comp rx=0 tx=0).stop 2024-09-10T09:39:10.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:10.723+0000 7fae46ffb740 1 -- 172.21.15.5:0/24953783 shutdown_connections 2024-09-10T09:39:10.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:10.723+0000 7fae46ffb740 1 --2- 172.21.15.5:0/24953783 >> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] conn(0x55fedbd84a80 0x55fedbd86ea0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:10.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:10.723+0000 7fae46ffb740 1 --2- 172.21.15.5:0/24953783 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fedbcf8e50 0x55fedbd82950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:10.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:10.723+0000 7fae46ffb740 1 --2- 172.21.15.5:0/24953783 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fedbcee590 0x55fedbd0d930 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:10.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:10.723+0000 7fae46ffb740 1 --2- 172.21.15.5:0/24953783 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fedbcedc70 0x55fedbd0d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:10.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:10.723+0000 7fae46ffb740 1 -- 172.21.15.5:0/24953783 >> 172.21.15.5:0/24953783 conn(0x55fedbcea200 msgr2=0x55fedbce8b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:39:10.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:10.723+0000 7fae23400640 0 client.15528 ms_handle_reset on v2:172.21.15.159:6836/620627639 2024-09-10T09:39:10.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:10.723+0000 7fae46ffb740 1 -- 172.21.15.5:0/24953783 shutdown_connections 2024-09-10T09:39:10.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:10.724+0000 7fae46ffb740 1 -- 172.21.15.5:0/24953783 wait complete. 2024-09-10T09:39:10.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:10.724+0000 7fae46ffb740 20 client.15528 trim_cache size 0 max 16384 2024-09-10T09:39:10.729 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-10T09:39:10.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:10.724+0000 7fae44600640 1 -- 172.21.15.5:0/1960919457 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fae240a2c90 msgr2=0x7fae240a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:39:10.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:10.724+0000 7fae44600640 1 --2- 172.21.15.5:0/1960919457 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fae240a2c90 0x7fae240a5150 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fae300096c0 tx=0x7fae300046c0 comp rx=0 tx=0).stop 2024-09-10T09:39:10.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:10.724+0000 7fae44600640 1 -- 172.21.15.5:0/1960919457 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fae3c1049c0 msgr2=0x7fae3c19ab80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:39:10.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:10.724+0000 7fae44600640 1 --2- 172.21.15.5:0/1960919457 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fae3c1049c0 0x7fae3c19ab80 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fae2800e9f0 tx=0x7fae2800eeb0 comp rx=0 tx=0).stop 2024-09-10T09:39:10.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:10.724+0000 7fae44600640 1 -- 172.21.15.5:0/1960919457 shutdown_connections 2024-09-10T09:39:10.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:10.724+0000 7fae44600640 1 --2- 172.21.15.5:0/1960919457 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fae240a2c90 0x7fae240a5150 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:10.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:10.724+0000 7fae44600640 1 --2- 172.21.15.5:0/1960919457 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fae3c109140 0x7fae3c19ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:10.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:10.724+0000 7fae44600640 1 --2- 172.21.15.5:0/1960919457 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fae3c1049c0 0x7fae3c19ab80 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:10.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:10.724+0000 7fae44600640 1 --2- 172.21.15.5:0/1960919457 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fae3c104010 0x7fae3c19a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:10.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:10.725+0000 7fae44600640 1 -- 172.21.15.5:0/1960919457 >> 172.21.15.5:0/1960919457 conn(0x7fae3c0ffa50 msgr2=0x7fae3c100dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:39:10.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:10.725+0000 7fae44600640 1 -- 172.21.15.5:0/1960919457 shutdown_connections 2024-09-10T09:39:10.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:10.725+0000 7fae44600640 1 -- 172.21.15.5:0/1960919457 wait complete. 2024-09-10T09:39:11.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:39:11 smithi005 ceph-mon[30350]: pgmap v418: 129 pgs: 129 active+clean; 660 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-10T09:39:11.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:39:11 smithi159 ceph-mon[35513]: pgmap v418: 129 pgs: 129 active+clean; 660 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-10T09:39:11.630 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24542 quiesce db / --timeout 90 --expiration 150.0 --await-for 45.0' 2024-09-10T09:39:11.631 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24542 quiesce db / --timeout 90 --expiration 150.0 --await-for 45.0 2024-09-10T09:39:11.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:39:11 smithi102 ceph-mon[35442]: pgmap v418: 129 pgs: 129 active+clean; 660 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-10T09:39:12.032 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-10T09:39:12.168 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:39:12.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:12.783+0000 7f72d7200640 1 -- 172.21.15.5:0/373902996 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f72d0074e30 msgr2=0x7f72d010dd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:39:12.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:12.783+0000 7f72d7200640 1 --2- 172.21.15.5:0/373902996 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f72d0074e30 0x7f72d010dd40 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f72c0009860 tx=0x7f72c002a3d0 comp rx=0 tx=0).stop 2024-09-10T09:39:12.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:12.783+0000 7f72d7200640 1 -- 172.21.15.5:0/373902996 shutdown_connections 2024-09-10T09:39:12.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:12.783+0000 7f72d7200640 1 --2- 172.21.15.5:0/373902996 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f72d010e280 0x7f72d0110670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:12.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:12.783+0000 7f72d7200640 1 --2- 172.21.15.5:0/373902996 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f72d0074e30 0x7f72d010dd40 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:12.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:12.783+0000 7f72d7200640 1 --2- 172.21.15.5:0/373902996 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f72d0074510 0x7f72d00748f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:12.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:12.783+0000 7f72d7200640 1 -- 172.21.15.5:0/373902996 >> 172.21.15.5:0/373902996 conn(0x7f72d00fd740 msgr2=0x7f72d00ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:39:12.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:12.784+0000 7f72d7200640 1 -- 172.21.15.5:0/373902996 shutdown_connections 2024-09-10T09:39:12.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:12.784+0000 7f72d7200640 1 -- 172.21.15.5:0/373902996 wait complete. 2024-09-10T09:39:12.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:12.787+0000 7f72d7200640 1 Processor -- start 2024-09-10T09:39:12.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:12.787+0000 7f72d7200640 1 -- start start 2024-09-10T09:39:12.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:12.787+0000 7f72d7200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f72d0074510 0x7f72d019a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:39:12.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:12.787+0000 7f72d7200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f72d0074e30 0x7f72d019aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:39:12.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:12.788+0000 7f72d7200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f72d010e280 0x7f72d019ee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:39:12.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:12.788+0000 7f72d7200640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f72d0114010 con 0x7f72d0074e30 2024-09-10T09:39:12.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:12.788+0000 7f72d7200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f72d0113e90 con 0x7f72d010e280 2024-09-10T09:39:12.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:12.788+0000 7f72d7200640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f72d0114190 con 0x7f72d0074510 2024-09-10T09:39:12.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:12.788+0000 7f72d5e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f72d0074510 0x7f72d019a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:39:12.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:12.788+0000 7f72d5e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f72d0074510 0x7f72d019a550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:44306/0 (socket says 172.21.15.5:44306) 2024-09-10T09:39:12.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:12.788+0000 7f72d5e00640 1 -- 172.21.15.5:0/2686625692 learned_addr learned my addr 172.21.15.5:0/2686625692 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:39:12.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:12.788+0000 7f72d5400640 1 --2- 172.21.15.5:0/2686625692 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f72d0074e30 0x7f72d019aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:39:12.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:12.789+0000 7f72d5e00640 1 -- 172.21.15.5:0/2686625692 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f72d010e280 msgr2=0x7f72d019ee20 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-10T09:39:12.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:12.789+0000 7f72d5e00640 1 --2- 172.21.15.5:0/2686625692 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f72d010e280 0x7f72d019ee20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:12.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:12.789+0000 7f72d5e00640 1 -- 172.21.15.5:0/2686625692 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f72d0074e30 msgr2=0x7f72d019aa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:39:12.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:12.789+0000 7f72d5e00640 1 --2- 172.21.15.5:0/2686625692 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f72d0074e30 0x7f72d019aa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:12.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:12.789+0000 7f72d5e00640 1 -- 172.21.15.5:0/2686625692 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72d019f500 con 0x7f72d0074510 2024-09-10T09:39:12.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:12.789+0000 7f72d5e00640 1 --2- 172.21.15.5:0/2686625692 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f72d0074510 0x7f72d019a550 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f72c400e9c0 tx=0x7f72c400ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:39:12.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:12.789+0000 7f72caa00640 1 -- 172.21.15.5:0/2686625692 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72c400ccc0 con 0x7f72d0074510 2024-09-10T09:39:12.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:12.789+0000 7f72caa00640 1 -- 172.21.15.5:0/2686625692 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f72c4005500 con 0x7f72d0074510 2024-09-10T09:39:12.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:12.789+0000 7f72caa00640 1 -- 172.21.15.5:0/2686625692 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72c4010430 con 0x7f72d0074510 2024-09-10T09:39:12.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:12.790+0000 7f72d7200640 1 -- 172.21.15.5:0/2686625692 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72d019f7f0 con 0x7f72d0074510 2024-09-10T09:39:12.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:12.790+0000 7f72d7200640 1 -- 172.21.15.5:0/2686625692 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72d01a70d0 con 0x7f72d0074510 2024-09-10T09:39:12.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:12.795+0000 7f72db203740 1 Processor -- start 2024-09-10T09:39:12.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:12.796+0000 7f72db203740 1 -- start start 2024-09-10T09:39:12.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:12.796+0000 7f72db203740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562132b8f560 0x562132b8f940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:39:12.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:12.796+0000 7f72db203740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562132b8fe80 0x562132b9a200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:39:12.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:12.796+0000 7f72db203740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562132b9a740 0x562132b9cb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:39:12.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:12.796+0000 7f72db203740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x562132af5530 con 0x562132b8fe80 2024-09-10T09:39:12.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:12.796+0000 7f72db203740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x562132a9c860 con 0x562132b8f560 2024-09-10T09:39:12.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:12.796+0000 7f72db203740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x562132af14f0 con 0x562132b9a740 2024-09-10T09:39:12.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:12.797+0000 7f72d6800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562132b8f560 0x562132b8f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:39:12.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:12.797+0000 7f72d6800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562132b8f560 0x562132b8f940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:38634/0 (socket says 172.21.15.5:38634) 2024-09-10T09:39:12.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:12.797+0000 7f72d6800640 1 -- 172.21.15.5:0/4030103978 learned_addr learned my addr 172.21.15.5:0/4030103978 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:39:12.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:12.797+0000 7f72d5400640 1 --2- 172.21.15.5:0/4030103978 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562132b9a740 0x562132b9cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:39:12.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:12.798+0000 7f72caa00640 1 -- 172.21.15.5:0/2686625692 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 19) ==== 137883+0+0 (secure 0 0 0) 0x7f72c40046c0 con 0x7f72d0074510 2024-09-10T09:39:12.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:12.798+0000 7f72d5400640 1 -- 172.21.15.5:0/4030103978 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562132b8f560 msgr2=0x562132b8f940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:39:12.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:12.798+0000 7f72d5400640 1 --2- 172.21.15.5:0/4030103978 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562132b8f560 0x562132b8f940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:12.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:12.798+0000 7f72d5400640 1 -- 172.21.15.5:0/4030103978 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562132b8fe80 msgr2=0x562132b9a200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:39:12.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:12.798+0000 7f72d5400640 1 --2- 172.21.15.5:0/4030103978 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562132b8fe80 0x562132b9a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:12.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:12.798+0000 7f72d5400640 1 -- 172.21.15.5:0/4030103978 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562132866b90 con 0x562132b9a740 2024-09-10T09:39:12.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:12.800+0000 7f72d5400640 1 --2- 172.21.15.5:0/4030103978 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562132b9a740 0x562132b9cb30 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f72c0009830 tx=0x7f72c0005d20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3bfd6db88667e1aa server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:39:12.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:12.801+0000 7f72caa00640 1 --2- 172.21.15.5:0/2686625692 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f72b80a2d10 0x7f72b80a51d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:39:12.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:12.801+0000 7f72caa00640 1 -- 172.21.15.5:0/2686625692 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f72c40cf9c0 con 0x7f72d0074510 2024-09-10T09:39:12.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:12.801+0000 7f72d6800640 1 --2- 172.21.15.5:0/2686625692 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f72b80a2d10 0x7f72b80a51d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:39:12.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:12.801+0000 7f72bf000640 1 -- 172.21.15.5:0/4030103978 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72c0030db0 con 0x562132b9a740 2024-09-10T09:39:12.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:12.801+0000 7f72bf000640 1 -- 172.21.15.5:0/4030103978 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f72c0041070 con 0x562132b9a740 2024-09-10T09:39:12.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:12.801+0000 7f72bf000640 1 -- 172.21.15.5:0/4030103978 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72c0004920 con 0x562132b9a740 2024-09-10T09:39:12.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:12.802+0000 7f72db203740 1 -- 172.21.15.5:0/4030103978 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562132b9a740 msgr2=0x562132b9cb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:39:12.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:12.802+0000 7f72db203740 1 --2- 172.21.15.5:0/4030103978 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562132b9a740 0x562132b9cb30 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f72c0009830 tx=0x7f72c0005d20 comp rx=0 tx=0).stop 2024-09-10T09:39:12.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:12.802+0000 7f72db203740 1 -- 172.21.15.5:0/4030103978 shutdown_connections 2024-09-10T09:39:12.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:12.802+0000 7f72db203740 1 --2- 172.21.15.5:0/4030103978 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562132b9a740 0x562132b9cb30 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:12.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:12.802+0000 7f72db203740 1 --2- 172.21.15.5:0/4030103978 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562132b8fe80 0x562132b9a200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:12.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:12.802+0000 7f72db203740 1 --2- 172.21.15.5:0/4030103978 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562132b8f560 0x562132b8f940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:12.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:12.802+0000 7f72db203740 1 -- 172.21.15.5:0/4030103978 >> 172.21.15.5:0/4030103978 conn(0x562132b8baf0 msgr2=0x562132b8bed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:39:12.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:12.803+0000 7f72d6800640 1 --2- 172.21.15.5:0/2686625692 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f72b80a2d10 0x7f72b80a51d0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f72cc009f70 tx=0x7f72cc00b040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:39:12.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:12.803+0000 7f72db203740 1 -- 172.21.15.5:0/4030103978 shutdown_connections 2024-09-10T09:39:12.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:12.804+0000 7f72db203740 1 -- 172.21.15.5:0/4030103978 wait complete. 2024-09-10T09:39:12.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:12.804+0000 7f72db203740 1 -- start start 2024-09-10T09:39:12.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:12.804+0000 7f72db203740 1 Processor -- start 2024-09-10T09:39:12.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:12.805+0000 7f72db203740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562132b8f560 0x562132b976b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:39:12.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:12.805+0000 7f72db203740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562132b8fe80 0x562132b97bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:39:12.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:12.805+0000 7f72db203740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562132b9a740 0x562132b98130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:39:12.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:12.805+0000 7f72db203740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x562132b9d310 con 0x562132b8f560 2024-09-10T09:39:12.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:12.805+0000 7f72db203740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x562132aafe60 con 0x562132b8fe80 2024-09-10T09:39:12.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:12.805+0000 7f72db203740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x562132b9d490 con 0x562132b9a740 2024-09-10T09:39:12.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:12.805+0000 7f72db203740 10 client.0 _ll_register_callbacks cb 0x562132ab4cc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:39:12.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:12.805+0000 7f72d5400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562132b8fe80 0x562132b97bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:39:12.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:12.806+0000 7f72d5400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562132b8fe80 0x562132b97bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:38646/0 (socket says 172.21.15.5:38646) 2024-09-10T09:39:12.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:12.806+0000 7f72d5400640 1 -- 172.21.15.5:0/1093840420 learned_addr learned my addr 172.21.15.5:0/1093840420 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:39:12.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:12.806+0000 7f72d6800640 1 --2- 172.21.15.5:0/1093840420 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562132b9a740 0x562132b98130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:39:12.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:12.806+0000 7f72d5e00640 1 --2- 172.21.15.5:0/1093840420 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562132b8f560 0x562132b976b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:39:12.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:12.806+0000 7f72d5400640 1 -- 172.21.15.5:0/1093840420 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562132b9a740 msgr2=0x562132b98130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:39:12.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:12.806+0000 7f72d5400640 1 --2- 172.21.15.5:0/1093840420 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562132b9a740 0x562132b98130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:12.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:12.806+0000 7f72d5400640 1 -- 172.21.15.5:0/1093840420 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562132b8f560 msgr2=0x562132b976b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:39:12.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:12.806+0000 7f72d5400640 1 --2- 172.21.15.5:0/1093840420 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562132b8f560 0x562132b976b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:12.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:12.807+0000 7f72d5400640 1 -- 172.21.15.5:0/1093840420 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562132abb990 con 0x562132b8fe80 2024-09-10T09:39:12.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:12.807+0000 7f72d5400640 1 --2- 172.21.15.5:0/1093840420 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562132b8fe80 0x562132b97bf0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f72c00054a0 tx=0x7f72c0030de0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cef781547abd04b6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:39:12.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:12.807+0000 7f72b3400640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-09-10T09:39:12.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:12.808+0000 7f72b3400640 1 -- 172.21.15.5:0/1093840420 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72c00305b0 con 0x562132b8fe80 2024-09-10T09:39:12.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:12.808+0000 7f72db203740 1 -- 172.21.15.5:0/1093840420 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f72c0041070 con 0x562132b8fe80 2024-09-10T09:39:12.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:12.808+0000 7f72b3400640 1 -- 172.21.15.5:0/1093840420 <== mon.2 v2:172.21.15.102:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f72c0030750 con 0x562132b8fe80 2024-09-10T09:39:12.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:12.808+0000 7f72b3400640 1 -- 172.21.15.5:0/1093840420 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f72c00046c0 con 0x562132b8fe80 2024-09-10T09:39:12.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:12.808+0000 7f72db203740 10 client.25538 fetch_fsmap learned FSMap version 30 2024-09-10T09:39:12.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:12.808+0000 7f72db203740 1 -- 172.21.15.5:0/1093840420 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x562132bb17b0 con 0x562132b8fe80 2024-09-10T09:39:12.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:12.809+0000 7f72b3400640 1 -- 172.21.15.5:0/1093840420 <== mon.2 v2:172.21.15.102:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f72c0041070 con 0x562132b8fe80 2024-09-10T09:39:12.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:12.809+0000 7f72b3400640 1 -- 172.21.15.5:0/1093840420 <== mon.2 v2:172.21.15.102:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f72c002f8d0 con 0x562132b8fe80 2024-09-10T09:39:12.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:12.809+0000 7f72db203740 10 client.25538 fetch_fsmap finished waiting for FSMap version 30 2024-09-10T09:39:12.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:12.809+0000 7f72db203740 10 client.25538 resolve_mds: validated gid 24542 aka daemon mds.b 2024-09-10T09:39:12.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:12.809+0000 7f72db203740 20 client.25538 populate_metadata read hostname 'smithi005' 2024-09-10T09:39:12.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:12.809+0000 7f72db203740 1 --2- 172.21.15.5:0/1093840420 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x562132bb2b00 0x562132bb4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:39:12.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:12.809+0000 7f72db203740 4 client.25538 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:39:12.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:12.809+0000 7f72db203740 1 -- 172.21.15.5:0/1093840420 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562132a98310 con 0x562132bb2b00 2024-09-10T09:39:12.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:12.810+0000 7f72d5e00640 1 --2- 172.21.15.5:0/1093840420 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x562132bb2b00 0x562132bb4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:39:12.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:12.810+0000 7f72d5e00640 1 --2- 172.21.15.5:0/1093840420 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x562132bb2b00 0x562132bb4ef0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=823e2338f1fe68f8 server_cookie=e6581efbad57659c in_seq=0 out_seq=0 2024-09-10T09:39:12.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:12.810+0000 7f72b3400640 10 client.25538 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T09:39:12.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:12.811+0000 7f72b3400640 1 -- 172.21.15.5:0/1093840420 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562132a98310 con 0x562132bb2b00 2024-09-10T09:39:12.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:12.811+0000 7f72b3400640 10 client.25538 handle_command_reply: tid=0 2024-09-10T09:39:12.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:12.811+0000 7f72db203740 1 client.25538 shutdown 2024-09-10T09:39:12.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:12.812+0000 7f72db203740 1 -- 172.21.15.5:0/1093840420 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562132b8fe80 msgr2=0x562132b97bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:39:12.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:12.812+0000 7f72db203740 1 --2- 172.21.15.5:0/1093840420 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562132b8fe80 0x562132b97bf0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f72c00054a0 tx=0x7f72c0030de0 comp rx=0 tx=0).stop 2024-09-10T09:39:12.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:12.812+0000 7f72db203740 1 -- 172.21.15.5:0/1093840420 shutdown_connections 2024-09-10T09:39:12.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:12.812+0000 7f72db203740 1 --2- 172.21.15.5:0/1093840420 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x562132bb2b00 0x562132bb4ef0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:12.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:12.812+0000 7f72db203740 1 --2- 172.21.15.5:0/1093840420 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562132b9a740 0x562132b98130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:12.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:12.812+0000 7f72db203740 1 --2- 172.21.15.5:0/1093840420 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562132b8fe80 0x562132b97bf0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:12.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:12.812+0000 7f72db203740 1 --2- 172.21.15.5:0/1093840420 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562132b8f560 0x562132b976b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:12.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:12.812+0000 7f72db203740 1 -- 172.21.15.5:0/1093840420 >> 172.21.15.5:0/1093840420 conn(0x562132b8baf0 msgr2=0x562132b91820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:39:12.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:12.812+0000 7f72b3400640 0 client.25538 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T09:39:12.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:12.812+0000 7f72db203740 1 -- 172.21.15.5:0/1093840420 shutdown_connections 2024-09-10T09:39:12.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:12.812+0000 7f72db203740 1 -- 172.21.15.5:0/1093840420 wait complete. 2024-09-10T09:39:12.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:12.812+0000 7f72db203740 20 client.25538 trim_cache size 0 max 16384 2024-09-10T09:39:12.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:12.832+0000 7f72db203740 1 Processor -- start 2024-09-10T09:39:12.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:12.833+0000 7f72db203740 1 -- start start 2024-09-10T09:39:12.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:12.833+0000 7f72db203740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562132b8f560 0x562132b97f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:39:12.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:12.833+0000 7f72db203740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562132b8fe80 0x562132b91710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:39:12.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:12.834+0000 7f72d5e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562132b8f560 0x562132b97f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:39:12.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:12.834+0000 7f72d5e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562132b8f560 0x562132b97f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:44332/0 (socket says 172.21.15.5:44332) 2024-09-10T09:39:12.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:12.833+0000 7f72db203740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562132b9a740 0x562132b94910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:39:12.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:12.834+0000 7f72db203740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f72d01acbc0 con 0x562132b8fe80 2024-09-10T09:39:12.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:12.834+0000 7f72db203740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f72d01ae700 con 0x562132b9a740 2024-09-10T09:39:12.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:12.834+0000 7f72db203740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f72d01ad290 con 0x562132b8f560 2024-09-10T09:39:12.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:12.834+0000 7f72d5e00640 1 -- 172.21.15.5:0/4162385429 learned_addr learned my addr 172.21.15.5:0/4162385429 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:39:12.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:12.834+0000 7f72d5400640 1 --2- 172.21.15.5:0/4162385429 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562132b8fe80 0x562132b91710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:39:12.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:12.835+0000 7f72d5e00640 1 -- 172.21.15.5:0/4162385429 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562132b9a740 msgr2=0x562132b94910 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T09:39:12.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:12.835+0000 7f72d5e00640 1 --2- 172.21.15.5:0/4162385429 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562132b9a740 0x562132b94910 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:12.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:12.835+0000 7f72d5e00640 1 -- 172.21.15.5:0/4162385429 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562132b8fe80 msgr2=0x562132b91710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:39:12.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:12.835+0000 7f72d5e00640 1 --2- 172.21.15.5:0/4162385429 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562132b8fe80 0x562132b91710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:12.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:12.835+0000 7f72d5e00640 1 -- 172.21.15.5:0/4162385429 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562132b87530 con 0x562132b8f560 2024-09-10T09:39:12.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:12.835+0000 7f72d5e00640 1 --2- 172.21.15.5:0/4162385429 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562132b8f560 0x562132b97f80 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f72c40980d0 tx=0x7f72c4009f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d56a31c5882ddefd server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:39:12.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:12.835+0000 7f72be600640 1 -- 172.21.15.5:0/4162385429 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72c409bc70 con 0x562132b8f560 2024-09-10T09:39:12.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:12.835+0000 7f72be600640 1 -- 172.21.15.5:0/4162385429 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f72c40a0020 con 0x562132b8f560 2024-09-10T09:39:12.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:12.835+0000 7f72be600640 1 -- 172.21.15.5:0/4162385429 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72c4016ed0 con 0x562132b8f560 2024-09-10T09:39:12.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:12.836+0000 7f72db203740 1 -- 172.21.15.5:0/4162385429 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562132b8f560 msgr2=0x562132b97f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:39:12.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:12.836+0000 7f72db203740 1 --2- 172.21.15.5:0/4162385429 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562132b8f560 0x562132b97f80 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f72c40980d0 tx=0x7f72c4009f90 comp rx=0 tx=0).stop 2024-09-10T09:39:12.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:12.836+0000 7f72db203740 1 -- 172.21.15.5:0/4162385429 shutdown_connections 2024-09-10T09:39:12.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:12.836+0000 7f72db203740 1 --2- 172.21.15.5:0/4162385429 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562132b9a740 0x562132b94910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:12.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:12.836+0000 7f72db203740 1 --2- 172.21.15.5:0/4162385429 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562132b8fe80 0x562132b91710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:12.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:12.836+0000 7f72db203740 1 --2- 172.21.15.5:0/4162385429 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562132b8f560 0x562132b97f80 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:12.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:12.836+0000 7f72db203740 1 -- 172.21.15.5:0/4162385429 >> 172.21.15.5:0/4162385429 conn(0x562132b8baf0 msgr2=0x562132b91000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:39:12.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:12.837+0000 7f72db203740 1 -- 172.21.15.5:0/4162385429 shutdown_connections 2024-09-10T09:39:12.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:12.837+0000 7f72db203740 1 -- 172.21.15.5:0/4162385429 wait complete. 2024-09-10T09:39:12.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:12.837+0000 7f72db203740 1 -- start start 2024-09-10T09:39:12.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:12.838+0000 7f72db203740 1 Processor -- start 2024-09-10T09:39:12.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:12.838+0000 7f72db203740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562132b8f560 0x562132bb8100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:39:12.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:12.838+0000 7f72db203740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562132b8fe80 0x562132bb8640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:39:12.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:12.838+0000 7f72db203740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562132b9a740 0x562132bb64f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:39:12.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:12.838+0000 7f72db203740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f72d01ae030 con 0x562132b9a740 2024-09-10T09:39:12.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:12.838+0000 7f72db203740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f72d01ad960 con 0x562132b8fe80 2024-09-10T09:39:12.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:12.838+0000 7f72db203740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f72d01aabc0 con 0x562132b8f560 2024-09-10T09:39:12.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:12.838+0000 7f72db203740 10 client.0 _ll_register_callbacks cb 0x7f72cc009500 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:39:12.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:12.839+0000 7f72d6800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562132b9a740 0x562132bb64f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:39:12.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:12.839+0000 7f72d5e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562132b8f560 0x562132bb8100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:39:12.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:12.839+0000 7f72d5e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562132b8f560 0x562132bb8100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:44346/0 (socket says 172.21.15.5:44346) 2024-09-10T09:39:12.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:12.839+0000 7f72d5e00640 1 -- 172.21.15.5:0/3123933983 learned_addr learned my addr 172.21.15.5:0/3123933983 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:39:12.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:12.839+0000 7f72d5400640 1 --2- 172.21.15.5:0/3123933983 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562132b8fe80 0x562132bb8640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:39:12.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:12.839+0000 7f72d5e00640 1 -- 172.21.15.5:0/3123933983 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562132b8fe80 msgr2=0x562132bb8640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:39:12.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:12.839+0000 7f72d5e00640 1 --2- 172.21.15.5:0/3123933983 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562132b8fe80 0x562132bb8640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:12.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:12.839+0000 7f72d5e00640 1 -- 172.21.15.5:0/3123933983 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562132b9a740 msgr2=0x562132bb64f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:39:12.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:12.839+0000 7f72d5e00640 1 --2- 172.21.15.5:0/3123933983 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562132b9a740 0x562132bb64f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:12.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:12.839+0000 7f72d5e00640 1 -- 172.21.15.5:0/3123933983 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562132bb6d20 con 0x562132b8f560 2024-09-10T09:39:12.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:12.840+0000 7f72d5e00640 1 --2- 172.21.15.5:0/3123933983 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562132b8f560 0x562132bb8100 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f72c40a43b0 tx=0x7f72c4004a00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=68d6ffd6531229a4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:39:12.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:12.840+0000 7f72b3400640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-09-10T09:39:12.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:12.840+0000 7f72b3400640 1 -- 172.21.15.5:0/3123933983 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72c407e030 con 0x562132b8f560 2024-09-10T09:39:12.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:12.840+0000 7f72b3400640 1 -- 172.21.15.5:0/3123933983 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f72c40cd170 con 0x562132b8f560 2024-09-10T09:39:12.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:12.840+0000 7f72db203740 1 -- 172.21.15.5:0/3123933983 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f72c40a0020 con 0x562132b8f560 2024-09-10T09:39:12.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:12.840+0000 7f72b3400640 1 -- 172.21.15.5:0/3123933983 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f72c409bc70 con 0x562132b8f560 2024-09-10T09:39:12.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:12.841+0000 7f72db203740 10 client.25573 fetch_fsmap learned FSMap version 30 2024-09-10T09:39:12.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:12.841+0000 7f72db203740 1 -- 172.21.15.5:0/3123933983 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x562132c1df60 con 0x562132b8f560 2024-09-10T09:39:12.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:12.841+0000 7f72b3400640 1 -- 172.21.15.5:0/3123933983 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f72c40a0020 con 0x562132b8f560 2024-09-10T09:39:12.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:12.842+0000 7f72b3400640 1 -- 172.21.15.5:0/3123933983 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f72c40a4450 con 0x562132b8f560 2024-09-10T09:39:12.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:12.842+0000 7f72db203740 10 client.25573 fetch_fsmap finished waiting for FSMap version 30 2024-09-10T09:39:12.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:12.842+0000 7f72db203740 10 client.25573 resolve_mds: validated gid 24542 aka daemon mds.b 2024-09-10T09:39:12.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:12.842+0000 7f72db203740 20 client.25573 populate_metadata read hostname 'smithi005' 2024-09-10T09:39:12.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:12.842+0000 7f72db203740 1 --2- 172.21.15.5:0/3123933983 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x562132c1e5f0 0x562132c209e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:39:12.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:12.843+0000 7f72db203740 4 client.25573 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 150.0, "await_for": 45.0}] 2024-09-10T09:39:12.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:12.843+0000 7f72db203740 1 -- 172.21.15.5:0/3123933983 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 150.0, "await_for": 45.0}) -- 0x562132b8c7a0 con 0x562132c1e5f0 2024-09-10T09:39:12.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:12.843+0000 7f72d5400640 1 --2- 172.21.15.5:0/3123933983 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x562132c1e5f0 0x562132c209e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:39:12.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:12.844+0000 7f72d5400640 1 --2- 172.21.15.5:0/3123933983 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x562132c1e5f0 0x562132c209e0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=665239eba769f1c6 server_cookie=ed4a941d415d6ceb in_seq=0 out_seq=0 2024-09-10T09:39:12.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:12.844+0000 7f72b3400640 10 client.25573 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T09:39:12.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:12.856+0000 7f72b3400640 1 -- 172.21.15.5:0/3123933983 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x562132b8c7a0 con 0x562132c1e5f0 2024-09-10T09:39:12.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:12.856+0000 7f72b3400640 10 client.25573 handle_command_reply: tid=0 2024-09-10T09:39:12.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:12.856+0000 7f72db203740 1 client.25573 shutdown 2024-09-10T09:39:12.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:12.856+0000 7f72db203740 1 -- 172.21.15.5:0/3123933983 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562132b8f560 msgr2=0x562132bb8100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:39:12.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:12.856+0000 7f72db203740 1 --2- 172.21.15.5:0/3123933983 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562132b8f560 0x562132bb8100 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f72c40a43b0 tx=0x7f72c4004a00 comp rx=0 tx=0).stop 2024-09-10T09:39:12.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:12.856+0000 7f72db203740 1 -- 172.21.15.5:0/3123933983 shutdown_connections 2024-09-10T09:39:12.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:12.856+0000 7f72db203740 1 --2- 172.21.15.5:0/3123933983 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x562132c1e5f0 0x562132c209e0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:12.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:12.856+0000 7f72db203740 1 --2- 172.21.15.5:0/3123933983 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562132b9a740 0x562132bb64f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:12.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:12.856+0000 7f72b3400640 0 client.25573 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T09:39:12.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:12.856+0000 7f72db203740 1 --2- 172.21.15.5:0/3123933983 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562132b8fe80 0x562132bb8640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:12.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:12.856+0000 7f72db203740 1 --2- 172.21.15.5:0/3123933983 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562132b8f560 0x562132bb8100 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:12.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:12.857+0000 7f72db203740 1 -- 172.21.15.5:0/3123933983 >> 172.21.15.5:0/3123933983 conn(0x562132b8baf0 msgr2=0x562132b8ac70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:39:12.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:12.857+0000 7f72db203740 1 -- 172.21.15.5:0/3123933983 shutdown_connections 2024-09-10T09:39:12.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:12.857+0000 7f72db203740 1 -- 172.21.15.5:0/3123933983 wait complete. 2024-09-10T09:39:12.860 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-10T09:39:12.860 INFO:teuthology.orchestra.run.smithi005.stdout: "epoch": 30, 2024-09-10T09:39:12.861 INFO:teuthology.orchestra.run.smithi005.stdout: "leader": 24542, 2024-09-10T09:39:12.861 INFO:teuthology.orchestra.run.smithi005.stdout: "set_version": 2, 2024-09-10T09:39:12.861 INFO:teuthology.orchestra.run.smithi005.stdout: "sets": { 2024-09-10T09:39:12.861 INFO:teuthology.orchestra.run.smithi005.stdout: "62d3ddc3": { 2024-09-10T09:39:12.861 INFO:teuthology.orchestra.run.smithi005.stdout: "version": 2, 2024-09-10T09:39:12.861 INFO:teuthology.orchestra.run.smithi005.stdout: "age_ref": 0.0, 2024-09-10T09:39:12.861 INFO:teuthology.orchestra.run.smithi005.stdout: "state": { 2024-09-10T09:39:12.861 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "QUIESCED", 2024-09-10T09:39:12.861 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0 2024-09-10T09:39:12.861 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T09:39:12.861 INFO:teuthology.orchestra.run.smithi005.stdout: "timeout": 90.0, 2024-09-10T09:39:12.861 INFO:teuthology.orchestra.run.smithi005.stdout: "expiration": 150.0, 2024-09-10T09:39:12.862 INFO:teuthology.orchestra.run.smithi005.stdout: "members": { 2024-09-10T09:39:12.862 INFO:teuthology.orchestra.run.smithi005.stdout: "file:/": { 2024-09-10T09:39:12.862 INFO:teuthology.orchestra.run.smithi005.stdout: "excluded": false, 2024-09-10T09:39:12.862 INFO:teuthology.orchestra.run.smithi005.stdout: "state": { 2024-09-10T09:39:12.862 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "QUIESCED", 2024-09-10T09:39:12.862 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0 2024-09-10T09:39:12.862 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T09:39:12.862 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T09:39:12.862 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T09:39:12.862 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T09:39:12.862 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T09:39:12.862 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-10T09:39:12.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:12.857+0000 7f72db203740 20 client.25573 trim_cache size 0 max 16384 2024-09-10T09:39:12.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:12.859+0000 7f72d7200640 1 -- 172.21.15.5:0/2686625692 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f72b80a2d10 msgr2=0x7f72b80a51d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:39:12.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:12.859+0000 7f72d7200640 1 --2- 172.21.15.5:0/2686625692 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f72b80a2d10 0x7f72b80a51d0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f72cc009f70 tx=0x7f72cc00b040 comp rx=0 tx=0).stop 2024-09-10T09:39:12.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:12.859+0000 7f72d7200640 1 -- 172.21.15.5:0/2686625692 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f72d0074510 msgr2=0x7f72d019a550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:39:12.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:12.859+0000 7f72d7200640 1 --2- 172.21.15.5:0/2686625692 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f72d0074510 0x7f72d019a550 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f72c400e9c0 tx=0x7f72c400ee80 comp rx=0 tx=0).stop 2024-09-10T09:39:12.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:12.859+0000 7f72d7200640 1 -- 172.21.15.5:0/2686625692 shutdown_connections 2024-09-10T09:39:12.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:12.859+0000 7f72d7200640 1 --2- 172.21.15.5:0/2686625692 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f72b80a2d10 0x7f72b80a51d0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:12.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:12.859+0000 7f72d7200640 1 --2- 172.21.15.5:0/2686625692 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f72d010e280 0x7f72d019ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:12.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:12.859+0000 7f72d7200640 1 --2- 172.21.15.5:0/2686625692 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f72d0074e30 0x7f72d019aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:12.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:12.859+0000 7f72d7200640 1 --2- 172.21.15.5:0/2686625692 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f72d0074510 0x7f72d019a550 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:12.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:12.859+0000 7f72d7200640 1 -- 172.21.15.5:0/2686625692 >> 172.21.15.5:0/2686625692 conn(0x7f72d00fd740 msgr2=0x7f72d00fee20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:39:12.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:12.859+0000 7f72d7200640 1 -- 172.21.15.5:0/2686625692 shutdown_connections 2024-09-10T09:39:12.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:12.859+0000 7f72d7200640 1 -- 172.21.15.5:0/2686625692 wait complete. 2024-09-10T09:39:12.866 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:39:13.418 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:39:13 smithi005 ceph-mon[30350]: pgmap v419: 129 pgs: 129 active+clean; 660 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-09-10T09:39:13.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:39:13 smithi159 ceph-mon[35513]: pgmap v419: 129 pgs: 129 active+clean; 660 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-09-10T09:39:13.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:39:13 smithi102 ceph-mon[35442]: pgmap v419: 129 pgs: 129 active+clean; 660 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-09-10T09:39:13.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:13.761+0000 7ff8ebe00640 1 -- 172.21.15.5:0/2399068138 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff8ec1019c0 msgr2=0x7ff8ec101e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:39:13.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:13.761+0000 7ff8ebe00640 1 --2- 172.21.15.5:0/2399068138 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff8ec1019c0 0x7ff8ec101e20 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7ff8d8009860 tx=0x7ff8d802a340 comp rx=0 tx=0).stop 2024-09-10T09:39:13.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:13.761+0000 7ff8ebe00640 1 -- 172.21.15.5:0/2399068138 shutdown_connections 2024-09-10T09:39:13.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:13.761+0000 7ff8ebe00640 1 --2- 172.21.15.5:0/2399068138 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff8ec102360 0x7ff8ec10e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:13.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:13.761+0000 7ff8ebe00640 1 --2- 172.21.15.5:0/2399068138 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff8ec1019c0 0x7ff8ec101e20 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:13.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:13.761+0000 7ff8ebe00640 1 --2- 172.21.15.5:0/2399068138 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff8ec1079c0 0x7ff8ec107da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:13.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:13.761+0000 7ff8ebe00640 1 -- 172.21.15.5:0/2399068138 >> 172.21.15.5:0/2399068138 conn(0x7ff8ec0fd740 msgr2=0x7ff8ec0ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:39:13.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:13.762+0000 7ff8ebe00640 1 -- 172.21.15.5:0/2399068138 shutdown_connections 2024-09-10T09:39:13.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:13.762+0000 7ff8ebe00640 1 -- 172.21.15.5:0/2399068138 wait complete. 2024-09-10T09:39:13.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:13.763+0000 7ff8ebe00640 1 Processor -- start 2024-09-10T09:39:13.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:13.763+0000 7ff8ebe00640 1 -- start start 2024-09-10T09:39:13.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:13.764+0000 7ff8ebe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff8ec1019c0 0x7ff8ec1a4660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:39:13.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:13.764+0000 7ff8ebe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff8ec102360 0x7ff8ec1a4ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:39:13.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:13.764+0000 7ff8ebe00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff8ec1079c0 0x7ff8ec10b920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:39:13.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:13.764+0000 7ff8ebe00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff8ec111dc0 con 0x7ff8ec1019c0 2024-09-10T09:39:13.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:13.764+0000 7ff8ebe00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7ff8ec111c40 con 0x7ff8ec102360 2024-09-10T09:39:13.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:13.764+0000 7ff8ebe00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7ff8ec111f40 con 0x7ff8ec1079c0 2024-09-10T09:39:13.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:13.764+0000 7ff8eaa00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff8ec1019c0 0x7ff8ec1a4660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:39:13.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:13.764+0000 7ff8ea000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff8ec102360 0x7ff8ec1a4ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:39:13.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:13.764+0000 7ff8ea000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff8ec102360 0x7ff8ec1a4ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:38690/0 (socket says 172.21.15.5:38690) 2024-09-10T09:39:13.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:13.764+0000 7ff8ea000640 1 -- 172.21.15.5:0/827591105 learned_addr learned my addr 172.21.15.5:0/827591105 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:39:13.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:13.765+0000 7ff8eaa00640 1 -- 172.21.15.5:0/827591105 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff8ec1079c0 msgr2=0x7ff8ec10b920 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T09:39:13.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:13.765+0000 7ff8eaa00640 1 --2- 172.21.15.5:0/827591105 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff8ec1079c0 0x7ff8ec10b920 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:13.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:13.765+0000 7ff8eaa00640 1 -- 172.21.15.5:0/827591105 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff8ec102360 msgr2=0x7ff8ec1a4ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:39:13.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:13.765+0000 7ff8eaa00640 1 --2- 172.21.15.5:0/827591105 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff8ec102360 0x7ff8ec1a4ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:13.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:13.765+0000 7ff8eaa00640 1 -- 172.21.15.5:0/827591105 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8ec10c1c0 con 0x7ff8ec1019c0 2024-09-10T09:39:13.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:13.765+0000 7ff8eaa00640 1 --2- 172.21.15.5:0/827591105 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff8ec1019c0 0x7ff8ec1a4660 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7ff8dc00e9f0 tx=0x7ff8dc00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:39:13.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:13.766+0000 7ff8e3400640 1 -- 172.21.15.5:0/827591105 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8dc00ccc0 con 0x7ff8ec1019c0 2024-09-10T09:39:13.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:13.766+0000 7ff8e3400640 1 -- 172.21.15.5:0/827591105 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff8dc010910 con 0x7ff8ec1019c0 2024-09-10T09:39:13.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:13.766+0000 7ff8e3400640 1 -- 172.21.15.5:0/827591105 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8dc018980 con 0x7ff8ec1019c0 2024-09-10T09:39:13.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:13.766+0000 7ff8ebe00640 1 -- 172.21.15.5:0/827591105 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8ec10c4b0 con 0x7ff8ec1019c0 2024-09-10T09:39:13.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:13.766+0000 7ff8ebe00640 1 -- 172.21.15.5:0/827591105 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8ec1ab5e0 con 0x7ff8ec1019c0 2024-09-10T09:39:13.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:13.770+0000 7ff8e3400640 1 -- 172.21.15.5:0/827591105 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 19) ==== 137883+0+0 (secure 0 0 0) 0x7ff8dc010ac0 con 0x7ff8ec1019c0 2024-09-10T09:39:13.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:13.771+0000 7ff8f41fa740 1 Processor -- start 2024-09-10T09:39:13.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:13.771+0000 7ff8e3400640 1 --2- 172.21.15.5:0/827591105 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7ff8d00a2c90 0x7ff8d00a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:39:13.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:13.772+0000 7ff8f41fa740 1 -- start start 2024-09-10T09:39:13.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:13.772+0000 7ff8e3400640 1 -- 172.21.15.5:0/827591105 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7ff8dc0cf940 con 0x7ff8ec1019c0 2024-09-10T09:39:13.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:13.772+0000 7ff8f41fa740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5575f8bece40 0x5575f8bed220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:39:13.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:13.772+0000 7ff8f41fa740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5575f8bed760 0x5575f8bf7ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:39:13.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:13.772+0000 7ff8f41fa740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5575f8bf8020 0x5575f8bfa410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:39:13.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:13.772+0000 7ff8f41fa740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5575f8b533e0 con 0x5575f8bf8020 2024-09-10T09:39:13.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:13.772+0000 7ff8f41fa740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x5575f8afad80 con 0x5575f8bece40 2024-09-10T09:39:13.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:13.772+0000 7ff8f41fa740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5575f8b4f3a0 con 0x5575f8bed760 2024-09-10T09:39:13.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:13.772+0000 7ff8eb400640 1 --2- 172.21.15.5:0/827591105 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7ff8d00a2c90 0x7ff8d00a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:39:13.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:13.773+0000 7ff8ea000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5575f8bed760 0x5575f8bf7ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:39:13.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:13.773+0000 7ff8eaa00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5575f8bece40 0x5575f8bed220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:39:13.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:13.773+0000 7ff8ea000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5575f8bed760 0x5575f8bf7ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:44356/0 (socket says 172.21.15.5:44356) 2024-09-10T09:39:13.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:13.773+0000 7ff8ea000640 1 -- 172.21.15.5:0/203245522 learned_addr learned my addr 172.21.15.5:0/203245522 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:39:13.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:13.773+0000 7ff8eaa00640 1 --2- 172.21.15.5:0/203245522 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5575f8bece40 0x5575f8bed220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:38706/0 (socket says 172.21.15.5:38706) 2024-09-10T09:39:13.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:13.774+0000 7ff8eaa00640 1 -- 172.21.15.5:0/203245522 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5575f8bed760 msgr2=0x5575f8bf7ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:39:13.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:13.774+0000 7ff8eaa00640 1 --2- 172.21.15.5:0/203245522 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5575f8bed760 0x5575f8bf7ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:13.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:13.774+0000 7ff8eaa00640 1 -- 172.21.15.5:0/203245522 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5575f8bf8020 msgr2=0x5575f8bfa410 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T09:39:13.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:13.774+0000 7ff8eaa00640 1 --2- 172.21.15.5:0/203245522 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5575f8bf8020 0x5575f8bfa410 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:13.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:13.774+0000 7ff8eaa00640 1 -- 172.21.15.5:0/203245522 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5575f8b19ae0 con 0x5575f8bece40 2024-09-10T09:39:13.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:13.774+0000 7ff8eb400640 1 --2- 172.21.15.5:0/827591105 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7ff8d00a2c90 0x7ff8d00a5150 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7ff8e4005fd0 tx=0x7ff8e40095c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:39:13.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:13.774+0000 7ff8eaa00640 1 --2- 172.21.15.5:0/203245522 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5575f8bece40 0x5575f8bed220 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7ff8dc0ccea0 tx=0x7ff8dc09c2f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f359b21b3e155a34 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:39:13.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:13.774+0000 7ff8d7400640 1 -- 172.21.15.5:0/203245522 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8dc0a4860 con 0x5575f8bece40 2024-09-10T09:39:13.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:13.774+0000 7ff8d7400640 1 -- 172.21.15.5:0/203245522 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff8dc086040 con 0x5575f8bece40 2024-09-10T09:39:13.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:13.774+0000 7ff8d7400640 1 -- 172.21.15.5:0/203245522 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8dc09b040 con 0x5575f8bece40 2024-09-10T09:39:13.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:13.775+0000 7ff8f41fa740 1 -- 172.21.15.5:0/203245522 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5575f8bece40 msgr2=0x5575f8bed220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:39:13.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:13.775+0000 7ff8f41fa740 1 --2- 172.21.15.5:0/203245522 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5575f8bece40 0x5575f8bed220 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7ff8dc0ccea0 tx=0x7ff8dc09c2f0 comp rx=0 tx=0).stop 2024-09-10T09:39:13.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:13.776+0000 7ff8f41fa740 1 -- 172.21.15.5:0/203245522 shutdown_connections 2024-09-10T09:39:13.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:13.776+0000 7ff8f41fa740 1 --2- 172.21.15.5:0/203245522 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5575f8bf8020 0x5575f8bfa410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:13.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:13.776+0000 7ff8f41fa740 1 --2- 172.21.15.5:0/203245522 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5575f8bed760 0x5575f8bf7ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:13.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:13.776+0000 7ff8f41fa740 1 --2- 172.21.15.5:0/203245522 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5575f8bece40 0x5575f8bed220 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:13.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:13.776+0000 7ff8f41fa740 1 -- 172.21.15.5:0/203245522 >> 172.21.15.5:0/203245522 conn(0x5575f8be93d0 msgr2=0x5575f8be97b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:39:13.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:13.776+0000 7ff8f41fa740 1 -- 172.21.15.5:0/203245522 shutdown_connections 2024-09-10T09:39:13.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:13.776+0000 7ff8f41fa740 1 -- 172.21.15.5:0/203245522 wait complete. 2024-09-10T09:39:13.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:13.777+0000 7ff8f41fa740 1 -- start start 2024-09-10T09:39:13.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:13.777+0000 7ff8f41fa740 1 Processor -- start 2024-09-10T09:39:13.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:13.778+0000 7ff8f41fa740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5575f8bece40 0x5575f8bf4f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:39:13.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:13.778+0000 7ff8f41fa740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5575f8bed760 0x5575f8bf5440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:39:13.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:13.778+0000 7ff8f41fa740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5575f8bf8020 0x5575f8bf5980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:39:13.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:13.778+0000 7ff8f41fa740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5575f8bfabf0 con 0x5575f8bed760 2024-09-10T09:39:13.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:13.778+0000 7ff8f41fa740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x5575f8b0e0f0 con 0x5575f8bf8020 2024-09-10T09:39:13.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:13.778+0000 7ff8f41fa740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5575f8bfad70 con 0x5575f8bece40 2024-09-10T09:39:13.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:13.778+0000 7ff8f41fa740 10 client.0 _ll_register_callbacks cb 0x5575f8af5bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:39:13.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:13.778+0000 7ff8eaa00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5575f8bece40 0x5575f8bf4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:39:13.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:13.778+0000 7ff8eaa00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5575f8bece40 0x5575f8bf4f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:44366/0 (socket says 172.21.15.5:44366) 2024-09-10T09:39:13.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:13.778+0000 7ff8eaa00640 1 -- 172.21.15.5:0/3343703353 learned_addr learned my addr 172.21.15.5:0/3343703353 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:39:13.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:13.779+0000 7ff8eb400640 1 --2- 172.21.15.5:0/3343703353 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5575f8bf8020 0x5575f8bf5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:39:13.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:13.779+0000 7ff8ea000640 1 --2- 172.21.15.5:0/3343703353 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5575f8bed760 0x5575f8bf5440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:39:13.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:13.779+0000 7ff8eaa00640 1 -- 172.21.15.5:0/3343703353 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5575f8bf8020 msgr2=0x5575f8bf5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:39:13.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:13.779+0000 7ff8eaa00640 1 --2- 172.21.15.5:0/3343703353 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5575f8bf8020 0x5575f8bf5980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:13.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:13.779+0000 7ff8eaa00640 1 -- 172.21.15.5:0/3343703353 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5575f8bed760 msgr2=0x5575f8bf5440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:39:13.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:13.779+0000 7ff8eaa00640 1 --2- 172.21.15.5:0/3343703353 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5575f8bed760 0x5575f8bf5440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:13.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:13.779+0000 7ff8eaa00640 1 -- 172.21.15.5:0/3343703353 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5575f88c43c0 con 0x5575f8bece40 2024-09-10T09:39:13.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:13.779+0000 7ff8eb400640 1 --2- 172.21.15.5:0/3343703353 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5575f8bf8020 0x5575f8bf5980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:39:13.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:13.780+0000 7ff8eaa00640 1 --2- 172.21.15.5:0/3343703353 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5575f8bece40 0x5575f8bf4f00 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7ff8dc00e9c0 tx=0x7ff8dc098500 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7dd1eae149e112fd server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:39:13.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:13.780+0000 7ff8cbe00640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-09-10T09:39:13.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:13.780+0000 7ff8cbe00640 1 -- 172.21.15.5:0/3343703353 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8dc0b8070 con 0x5575f8bece40 2024-09-10T09:39:13.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:13.780+0000 7ff8cbe00640 1 -- 172.21.15.5:0/3343703353 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7ff8dc0cd0f0 con 0x5575f8bece40 2024-09-10T09:39:13.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:13.781+0000 7ff8f41fa740 1 -- 172.21.15.5:0/3343703353 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff8dc086040 con 0x5575f8bece40 2024-09-10T09:39:13.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:13.781+0000 7ff8cbe00640 1 -- 172.21.15.5:0/3343703353 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7ff8dc09b040 con 0x5575f8bece40 2024-09-10T09:39:13.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:13.781+0000 7ff8f41fa740 10 client.25582 fetch_fsmap learned FSMap version 30 2024-09-10T09:39:13.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:13.781+0000 7ff8f41fa740 1 -- 172.21.15.5:0/3343703353 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5575f8bfabf0 con 0x5575f8bece40 2024-09-10T09:39:13.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:13.781+0000 7ff8cbe00640 1 -- 172.21.15.5:0/3343703353 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff8dc086040 con 0x5575f8bece40 2024-09-10T09:39:13.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:13.781+0000 7ff8cbe00640 1 -- 172.21.15.5:0/3343703353 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7ff8dc0979b0 con 0x5575f8bece40 2024-09-10T09:39:13.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:13.781+0000 7ff8f41fa740 10 client.25582 fetch_fsmap finished waiting for FSMap version 30 2024-09-10T09:39:13.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:13.781+0000 7ff8f41fa740 10 client.25582 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-10T09:39:13.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:13.781+0000 7ff8f41fa740 20 client.25582 populate_metadata read hostname 'smithi005' 2024-09-10T09:39:13.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:13.781+0000 7ff8f41fa740 1 --2- 172.21.15.5:0/3343703353 >> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] conn(0x5575f8c07de0 0x5575f8c0a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:39:13.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:13.782+0000 7ff8f41fa740 4 client.25582 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:39:13.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:13.782+0000 7ff8f41fa740 1 -- 172.21.15.5:0/3343703353 --> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5575f8b60950 con 0x5575f8c07de0 2024-09-10T09:39:13.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:13.782+0000 7ff8ea000640 1 --2- 172.21.15.5:0/3343703353 >> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] conn(0x5575f8c07de0 0x5575f8c0a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:39:13.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:13.783+0000 7ff8ea000640 1 --2- 172.21.15.5:0/3343703353 >> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] conn(0x5575f8c07de0 0x5575f8c0a1d0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8f64d09b89e63e14 server_cookie=3b300627af08ff93 in_seq=0 out_seq=0 2024-09-10T09:39:13.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:13.783+0000 7ff8cbe00640 10 client.25582 ms_handle_connect on v2:172.21.15.159:6838/2073739599 2024-09-10T09:39:13.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:13.783+0000 7ff8cbe00640 1 -- 172.21.15.5:0/3343703353 <== mds.1 v2:172.21.15.159:6838/2073739599 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5575f8b60950 con 0x5575f8c07de0 2024-09-10T09:39:13.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:13.783+0000 7ff8cbe00640 10 client.25582 handle_command_reply: tid=0 2024-09-10T09:39:13.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:13.783+0000 7ff8f41fa740 1 client.25582 shutdown 2024-09-10T09:39:13.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:13.784+0000 7ff8f41fa740 1 -- 172.21.15.5:0/3343703353 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5575f8bece40 msgr2=0x5575f8bf4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:39:13.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:13.784+0000 7ff8f41fa740 1 --2- 172.21.15.5:0/3343703353 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5575f8bece40 0x5575f8bf4f00 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7ff8dc00e9c0 tx=0x7ff8dc098500 comp rx=0 tx=0).stop 2024-09-10T09:39:13.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:13.784+0000 7ff8f41fa740 1 -- 172.21.15.5:0/3343703353 shutdown_connections 2024-09-10T09:39:13.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:13.784+0000 7ff8f41fa740 1 --2- 172.21.15.5:0/3343703353 >> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] conn(0x5575f8c07de0 0x5575f8c0a1d0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:13.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:13.784+0000 7ff8f41fa740 1 --2- 172.21.15.5:0/3343703353 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5575f8bf8020 0x5575f8bf5980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:13.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:13.784+0000 7ff8f41fa740 1 --2- 172.21.15.5:0/3343703353 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5575f8bed760 0x5575f8bf5440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:13.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:13.784+0000 7ff8f41fa740 1 --2- 172.21.15.5:0/3343703353 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5575f8bece40 0x5575f8bf4f00 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:13.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:13.784+0000 7ff8f41fa740 1 -- 172.21.15.5:0/3343703353 >> 172.21.15.5:0/3343703353 conn(0x5575f8be93d0 msgr2=0x5575f8bef0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:39:13.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:13.784+0000 7ff8cbe00640 0 client.25582 ms_handle_reset on v2:172.21.15.159:6838/2073739599 2024-09-10T09:39:13.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:13.784+0000 7ff8f41fa740 1 -- 172.21.15.5:0/3343703353 shutdown_connections 2024-09-10T09:39:13.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:13.784+0000 7ff8f41fa740 1 -- 172.21.15.5:0/3343703353 wait complete. 2024-09-10T09:39:13.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:13.785+0000 7ff8f41fa740 20 client.25582 trim_cache size 0 max 16384 2024-09-10T09:39:13.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:13.799+0000 7ff8f41fa740 1 Processor -- start 2024-09-10T09:39:13.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:13.799+0000 7ff8f41fa740 1 -- start start 2024-09-10T09:39:13.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:13.799+0000 7ff8f41fa740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5575f8bece40 0x5575f8c0f8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:39:13.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:13.799+0000 7ff8f41fa740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5575f8bed760 0x5575f8c0fe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:39:13.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:13.799+0000 7ff8f41fa740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5575f8bf8020 0x5575f8bee890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:39:13.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:13.799+0000 7ff8f41fa740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff8cc00fd50 con 0x5575f8bece40 2024-09-10T09:39:13.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:13.799+0000 7ff8f41fa740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7ff8cc011890 con 0x5575f8bf8020 2024-09-10T09:39:13.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:13.799+0000 7ff8f41fa740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7ff8cc010420 con 0x5575f8bed760 2024-09-10T09:39:13.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:13.800+0000 7ff8eaa00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5575f8bece40 0x5575f8c0f8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:39:13.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:13.800+0000 7ff8eaa00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5575f8bece40 0x5575f8c0f8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60740/0 (socket says 172.21.15.5:60740) 2024-09-10T09:39:13.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:13.800+0000 7ff8eaa00640 1 -- 172.21.15.5:0/121374449 learned_addr learned my addr 172.21.15.5:0/121374449 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:39:13.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:13.800+0000 7ff8eb400640 1 --2- 172.21.15.5:0/121374449 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5575f8bf8020 0x5575f8bee890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:39:13.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:13.800+0000 7ff8eaa00640 1 -- 172.21.15.5:0/121374449 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5575f8bed760 msgr2=0x5575f8c0fe00 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T09:39:13.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:13.800+0000 7ff8ea000640 1 --2- 172.21.15.5:0/121374449 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5575f8bed760 0x5575f8c0fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:39:13.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:13.800+0000 7ff8eaa00640 1 --2- 172.21.15.5:0/121374449 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5575f8bed760 0x5575f8c0fe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:13.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:13.800+0000 7ff8eaa00640 1 -- 172.21.15.5:0/121374449 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5575f8bf8020 msgr2=0x5575f8bee890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:39:13.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:13.800+0000 7ff8eaa00640 1 --2- 172.21.15.5:0/121374449 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5575f8bf8020 0x5575f8bee890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:13.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:13.800+0000 7ff8eaa00640 1 -- 172.21.15.5:0/121374449 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5575f8befcb0 con 0x5575f8bece40 2024-09-10T09:39:13.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:13.800+0000 7ff8eb400640 1 --2- 172.21.15.5:0/121374449 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5575f8bf8020 0x5575f8bee890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:39:13.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:13.801+0000 7ff8ea000640 1 --2- 172.21.15.5:0/121374449 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5575f8bed760 0x5575f8c0fe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T09:39:13.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:13.801+0000 7ff8eaa00640 1 --2- 172.21.15.5:0/121374449 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5575f8bece40 0x5575f8c0f8c0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7ff8dc099ea0 tx=0x7ff8dc09c2f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42e73f682c4bd749 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:39:13.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:13.801+0000 7ff8d6a00640 1 -- 172.21.15.5:0/121374449 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8dc0b8070 con 0x5575f8bece40 2024-09-10T09:39:13.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:13.801+0000 7ff8d6a00640 1 -- 172.21.15.5:0/121374449 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff8dc086040 con 0x5575f8bece40 2024-09-10T09:39:13.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:13.801+0000 7ff8d6a00640 1 -- 172.21.15.5:0/121374449 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8dc097940 con 0x5575f8bece40 2024-09-10T09:39:13.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:13.802+0000 7ff8f41fa740 1 -- 172.21.15.5:0/121374449 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5575f8bece40 msgr2=0x5575f8c0f8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:39:13.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:13.802+0000 7ff8f41fa740 1 --2- 172.21.15.5:0/121374449 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5575f8bece40 0x5575f8c0f8c0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7ff8dc099ea0 tx=0x7ff8dc09c2f0 comp rx=0 tx=0).stop 2024-09-10T09:39:13.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:13.802+0000 7ff8f41fa740 1 -- 172.21.15.5:0/121374449 shutdown_connections 2024-09-10T09:39:13.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:13.802+0000 7ff8f41fa740 1 --2- 172.21.15.5:0/121374449 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5575f8bf8020 0x5575f8bee890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:13.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:13.802+0000 7ff8f41fa740 1 --2- 172.21.15.5:0/121374449 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5575f8bed760 0x5575f8c0fe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:13.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:13.802+0000 7ff8f41fa740 1 --2- 172.21.15.5:0/121374449 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5575f8bece40 0x5575f8c0f8c0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:13.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:13.802+0000 7ff8f41fa740 1 -- 172.21.15.5:0/121374449 >> 172.21.15.5:0/121374449 conn(0x5575f8be93d0 msgr2=0x5575f8bf01c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:39:13.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:13.803+0000 7ff8f41fa740 1 -- 172.21.15.5:0/121374449 shutdown_connections 2024-09-10T09:39:13.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:13.803+0000 7ff8f41fa740 1 -- 172.21.15.5:0/121374449 wait complete. 2024-09-10T09:39:13.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:13.803+0000 7ff8f41fa740 1 -- start start 2024-09-10T09:39:13.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:13.803+0000 7ff8f41fa740 1 Processor -- start 2024-09-10T09:39:13.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:13.804+0000 7ff8f41fa740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5575f8bed760 0x5575f8c003a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:39:13.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:13.804+0000 7ff8f41fa740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5575f8bf8020 0x5575f8c008e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:39:13.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:13.804+0000 7ff8f41fa740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5575f8bf19e0 0x5575f8bf1e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:39:13.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:13.804+0000 7ff8f41fa740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff8cc0111c0 con 0x5575f8bf19e0 2024-09-10T09:39:13.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:13.804+0000 7ff8f41fa740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7ff8cc010af0 con 0x5575f8bf8020 2024-09-10T09:39:13.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:13.804+0000 7ff8f41fa740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7ff8cc00dd30 con 0x5575f8bed760 2024-09-10T09:39:13.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:13.804+0000 7ff8f41fa740 10 client.0 _ll_register_callbacks cb 0x7ff8dc005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:39:13.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:13.804+0000 7ff8eaa00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5575f8bed760 0x5575f8c003a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:39:13.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:13.804+0000 7ff8eaa00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5575f8bed760 0x5575f8c003a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:44390/0 (socket says 172.21.15.5:44390) 2024-09-10T09:39:13.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:13.804+0000 7ff8eaa00640 1 -- 172.21.15.5:0/2333963425 learned_addr learned my addr 172.21.15.5:0/2333963425 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:39:13.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:13.805+0000 7ff8eaa00640 1 -- 172.21.15.5:0/2333963425 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5575f8bf8020 msgr2=0x5575f8c008e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T09:39:13.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:13.805+0000 7ff8eaa00640 1 --2- 172.21.15.5:0/2333963425 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5575f8bf8020 0x5575f8c008e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:13.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:13.805+0000 7ff8eb400640 1 --2- 172.21.15.5:0/2333963425 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5575f8bf19e0 0x5575f8bf1e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:39:13.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:13.805+0000 7ff8eaa00640 1 -- 172.21.15.5:0/2333963425 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5575f8bf19e0 msgr2=0x5575f8bf1e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:39:13.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:13.805+0000 7ff8eaa00640 1 --2- 172.21.15.5:0/2333963425 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5575f8bf19e0 0x5575f8bf1e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:13.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:13.805+0000 7ff8eaa00640 1 -- 172.21.15.5:0/2333963425 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5575f8bf2690 con 0x5575f8bed760 2024-09-10T09:39:13.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:13.805+0000 7ff8eaa00640 1 --2- 172.21.15.5:0/2333963425 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5575f8bed760 0x5575f8c003a0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7ff8dc09b8f0 tx=0x7ff8dc097dd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e0fc7258b7800fff server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:39:13.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:13.806+0000 7ff8cbe00640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-09-10T09:39:13.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:13.806+0000 7ff8cbe00640 1 -- 172.21.15.5:0/2333963425 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8dc0b8050 con 0x5575f8bed760 2024-09-10T09:39:13.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:13.806+0000 7ff8cbe00640 1 -- 172.21.15.5:0/2333963425 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7ff8dc010430 con 0x5575f8bed760 2024-09-10T09:39:13.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:13.806+0000 7ff8f41fa740 1 -- 172.21.15.5:0/2333963425 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff8dc086040 con 0x5575f8bed760 2024-09-10T09:39:13.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:13.806+0000 7ff8cbe00640 1 -- 172.21.15.5:0/2333963425 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7ff8dc097b50 con 0x5575f8bed760 2024-09-10T09:39:13.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:13.806+0000 7ff8f41fa740 10 client.25585 fetch_fsmap learned FSMap version 30 2024-09-10T09:39:13.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:13.806+0000 7ff8f41fa740 1 -- 172.21.15.5:0/2333963425 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5575f8bf2690 con 0x5575f8bed760 2024-09-10T09:39:13.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:13.807+0000 7ff8cbe00640 1 -- 172.21.15.5:0/2333963425 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff8dc086040 con 0x5575f8bed760 2024-09-10T09:39:13.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:13.807+0000 7ff8cbe00640 1 -- 172.21.15.5:0/2333963425 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7ff8dc081070 con 0x5575f8bed760 2024-09-10T09:39:13.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:13.807+0000 7ff8f41fa740 10 client.25585 fetch_fsmap finished waiting for FSMap version 30 2024-09-10T09:39:13.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:13.807+0000 7ff8f41fa740 10 client.25585 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-10T09:39:13.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:13.807+0000 7ff8f41fa740 20 client.25585 populate_metadata read hostname 'smithi005' 2024-09-10T09:39:13.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:13.807+0000 7ff8f41fa740 1 --2- 172.21.15.5:0/2333963425 >> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] conn(0x5575f8c72b30 0x5575f8c74f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:39:13.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:13.807+0000 7ff8f41fa740 4 client.25585 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T09:39:13.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:13.807+0000 7ff8f41fa740 1 -- 172.21.15.5:0/2333963425 --> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5575f8b5ae60 con 0x5575f8c72b30 2024-09-10T09:39:13.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:13.807+0000 7ff8ea000640 1 --2- 172.21.15.5:0/2333963425 >> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] conn(0x5575f8c72b30 0x5575f8c74f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:39:13.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:13.808+0000 7ff8ea000640 1 --2- 172.21.15.5:0/2333963425 >> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] conn(0x5575f8c72b30 0x5575f8c74f20 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1927dabdaa74d061 server_cookie=63e0b7f2dd17f24a in_seq=0 out_seq=0 2024-09-10T09:39:13.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:13.808+0000 7ff8cbe00640 10 client.25585 ms_handle_connect on v2:172.21.15.159:6838/2073739599 2024-09-10T09:39:13.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:13.808+0000 7ff8cbe00640 1 -- 172.21.15.5:0/2333963425 <== mds.1 v2:172.21.15.159:6838/2073739599 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5575f8b5ae60 con 0x5575f8c72b30 2024-09-10T09:39:13.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:13.808+0000 7ff8cbe00640 10 client.25585 handle_command_reply: tid=0 2024-09-10T09:39:13.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:13.808+0000 7ff8f41fa740 1 client.25585 shutdown 2024-09-10T09:39:13.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:13.809+0000 7ff8f41fa740 1 -- 172.21.15.5:0/2333963425 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5575f8bed760 msgr2=0x5575f8c003a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:39:13.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:13.809+0000 7ff8f41fa740 1 --2- 172.21.15.5:0/2333963425 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5575f8bed760 0x5575f8c003a0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7ff8dc09b8f0 tx=0x7ff8dc097dd0 comp rx=0 tx=0).stop 2024-09-10T09:39:13.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:13.809+0000 7ff8f41fa740 1 -- 172.21.15.5:0/2333963425 shutdown_connections 2024-09-10T09:39:13.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:13.809+0000 7ff8f41fa740 1 --2- 172.21.15.5:0/2333963425 >> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] conn(0x5575f8c72b30 0x5575f8c74f20 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:13.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:13.809+0000 7ff8f41fa740 1 --2- 172.21.15.5:0/2333963425 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5575f8bf19e0 0x5575f8bf1e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:13.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:13.809+0000 7ff8f41fa740 1 --2- 172.21.15.5:0/2333963425 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5575f8bf8020 0x5575f8c008e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:13.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:13.809+0000 7ff8f41fa740 1 --2- 172.21.15.5:0/2333963425 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5575f8bed760 0x5575f8c003a0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:13.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:13.809+0000 7ff8f41fa740 1 -- 172.21.15.5:0/2333963425 >> 172.21.15.5:0/2333963425 conn(0x5575f8be93d0 msgr2=0x5575f8bee890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:39:13.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:13.809+0000 7ff8cbe00640 0 client.25585 ms_handle_reset on v2:172.21.15.159:6838/2073739599 2024-09-10T09:39:13.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:13.809+0000 7ff8f41fa740 1 -- 172.21.15.5:0/2333963425 shutdown_connections 2024-09-10T09:39:13.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:13.809+0000 7ff8f41fa740 1 -- 172.21.15.5:0/2333963425 wait complete. 2024-09-10T09:39:13.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:13.809+0000 7ff8f41fa740 20 client.25585 trim_cache size 0 max 16384 2024-09-10T09:39:13.813 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-10T09:39:13.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:13.810+0000 7ff8ebe00640 1 -- 172.21.15.5:0/827591105 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7ff8d00a2c90 msgr2=0x7ff8d00a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:39:13.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:13.810+0000 7ff8ebe00640 1 --2- 172.21.15.5:0/827591105 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7ff8d00a2c90 0x7ff8d00a5150 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7ff8e4005fd0 tx=0x7ff8e40095c0 comp rx=0 tx=0).stop 2024-09-10T09:39:13.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:13.810+0000 7ff8ebe00640 1 -- 172.21.15.5:0/827591105 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff8ec1019c0 msgr2=0x7ff8ec1a4660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:39:13.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:13.810+0000 7ff8ebe00640 1 --2- 172.21.15.5:0/827591105 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff8ec1019c0 0x7ff8ec1a4660 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7ff8dc00e9f0 tx=0x7ff8dc00eeb0 comp rx=0 tx=0).stop 2024-09-10T09:39:13.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:13.810+0000 7ff8ebe00640 1 -- 172.21.15.5:0/827591105 shutdown_connections 2024-09-10T09:39:13.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:13.810+0000 7ff8ebe00640 1 --2- 172.21.15.5:0/827591105 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7ff8d00a2c90 0x7ff8d00a5150 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:13.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:13.810+0000 7ff8ebe00640 1 --2- 172.21.15.5:0/827591105 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff8ec1079c0 0x7ff8ec10b920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:13.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:13.810+0000 7ff8ebe00640 1 --2- 172.21.15.5:0/827591105 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff8ec102360 0x7ff8ec1a4ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:13.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:13.810+0000 7ff8ebe00640 1 --2- 172.21.15.5:0/827591105 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff8ec1019c0 0x7ff8ec1a4660 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:13.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:13.810+0000 7ff8ebe00640 1 -- 172.21.15.5:0/827591105 >> 172.21.15.5:0/827591105 conn(0x7ff8ec0fd740 msgr2=0x7ff8ec10ccc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:39:13.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:13.811+0000 7ff8ebe00640 1 -- 172.21.15.5:0/827591105 shutdown_connections 2024-09-10T09:39:13.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:13.811+0000 7ff8ebe00640 1 -- 172.21.15.5:0/827591105 wait complete. 2024-09-10T09:39:14.590 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '62d3ddc3', quiesce took 6.2 seconds. Will release after: 23.8 2024-09-10T09:39:15.050 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-10T09:39:15.475 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:39:15.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:39:15 smithi005 ceph-mon[30350]: pgmap v420: 129 pgs: 129 active+clean; 660 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-10T09:39:15.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:39:15 smithi159 ceph-mon[35513]: pgmap v420: 129 pgs: 129 active+clean; 660 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-10T09:39:15.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:39:15 smithi102 ceph-mon[35442]: pgmap v420: 129 pgs: 129 active+clean; 660 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-10T09:39:16.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:16.116+0000 7f513d000640 1 -- 172.21.15.5:0/1468905972 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f51381010d0 msgr2=0x7f513810dd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:39:16.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:16.116+0000 7f513d000640 1 --2- 172.21.15.5:0/1468905972 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f51381010d0 0x7f513810dd90 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f5124009900 tx=0x7f512402a3f0 comp rx=0 tx=0).stop 2024-09-10T09:39:16.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:16.116+0000 7f513d000640 1 -- 172.21.15.5:0/1468905972 shutdown_connections 2024-09-10T09:39:16.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:16.116+0000 7f513d000640 1 --2- 172.21.15.5:0/1468905972 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f513810e2d0 0x7f51381106c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:16.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:16.116+0000 7f513d000640 1 --2- 172.21.15.5:0/1468905972 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f51381010d0 0x7f513810dd90 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:16.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:16.116+0000 7f513d000640 1 --2- 172.21.15.5:0/1468905972 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5138100720 0x7f5138100b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:16.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:16.116+0000 7f513d000640 1 -- 172.21.15.5:0/1468905972 >> 172.21.15.5:0/1468905972 conn(0x7f51380fc610 msgr2=0x7f51380fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:39:16.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:16.117+0000 7f513d000640 1 -- 172.21.15.5:0/1468905972 shutdown_connections 2024-09-10T09:39:16.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:16.117+0000 7f513d000640 1 -- 172.21.15.5:0/1468905972 wait complete. 2024-09-10T09:39:16.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:16.118+0000 7f513d000640 1 Processor -- start 2024-09-10T09:39:16.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:16.118+0000 7f513d000640 1 -- start start 2024-09-10T09:39:16.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:16.119+0000 7f513d000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5138100720 0x7f513819a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:39:16.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:16.119+0000 7f513d000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f51381010d0 0x7f513819aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:39:16.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:16.119+0000 7f513d000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f513810e2d0 0x7f513819edd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:39:16.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:16.119+0000 7f513d000640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f5138114020 con 0x7f513810e2d0 2024-09-10T09:39:16.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:16.119+0000 7f513d000640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f5138113ea0 con 0x7f51381010d0 2024-09-10T09:39:16.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:16.119+0000 7f513d000640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f51381141a0 con 0x7f5138100720 2024-09-10T09:39:16.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:16.119+0000 7f5137e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f513810e2d0 0x7f513819edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:39:16.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:16.119+0000 7f5136a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f51381010d0 0x7f513819aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:39:16.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:16.119+0000 7f5136a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f51381010d0 0x7f513819aa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:38750/0 (socket says 172.21.15.5:38750) 2024-09-10T09:39:16.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:16.119+0000 7f5136a00640 1 -- 172.21.15.5:0/843971433 learned_addr learned my addr 172.21.15.5:0/843971433 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:39:16.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:16.120+0000 7f5137400640 1 --2- 172.21.15.5:0/843971433 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5138100720 0x7f513819a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:39:16.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:16.120+0000 7f5136a00640 1 -- 172.21.15.5:0/843971433 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5138100720 msgr2=0x7f513819a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:39:16.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:16.120+0000 7f5136a00640 1 --2- 172.21.15.5:0/843971433 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5138100720 0x7f513819a500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:16.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:16.120+0000 7f5136a00640 1 -- 172.21.15.5:0/843971433 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f513810e2d0 msgr2=0x7f513819edd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:39:16.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:16.120+0000 7f5136a00640 1 --2- 172.21.15.5:0/843971433 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f513810e2d0 0x7f513819edd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:16.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:16.120+0000 7f5136a00640 1 -- 172.21.15.5:0/843971433 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f513819f4b0 con 0x7f51381010d0 2024-09-10T09:39:16.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:16.120+0000 7f5136a00640 1 --2- 172.21.15.5:0/843971433 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f51381010d0 0x7f513819aa40 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f5124004770 tx=0x7f5124003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:39:16.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:16.121+0000 7f512fe00640 1 -- 172.21.15.5:0/843971433 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f512403f070 con 0x7f51381010d0 2024-09-10T09:39:16.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:16.121+0000 7f512fe00640 1 -- 172.21.15.5:0/843971433 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5124031ca0 con 0x7f51381010d0 2024-09-10T09:39:16.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:16.121+0000 7f513d000640 1 -- 172.21.15.5:0/843971433 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f513819f740 con 0x7f51381010d0 2024-09-10T09:39:16.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:16.121+0000 7f512fe00640 1 -- 172.21.15.5:0/843971433 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5124035030 con 0x7f51381010d0 2024-09-10T09:39:16.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:16.121+0000 7f513d000640 1 -- 172.21.15.5:0/843971433 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f513819fbb0 con 0x7f51381010d0 2024-09-10T09:39:16.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:16.126+0000 7f5141084740 1 Processor -- start 2024-09-10T09:39:16.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:16.126+0000 7f5141084740 1 -- start start 2024-09-10T09:39:16.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:16.127+0000 7f512fe00640 1 -- 172.21.15.5:0/843971433 <== mon.2 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 19) ==== 137883+0+0 (secure 0 0 0) 0x7f5124042070 con 0x7f51381010d0 2024-09-10T09:39:16.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:16.127+0000 7f5141084740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56555aab4e40 0x56555aab5220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:39:16.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:16.127+0000 7f5141084740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56555aab5760 0x56555aabfae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:39:16.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:16.127+0000 7f5141084740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56555aac0020 0x56555aac2410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:39:16.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:16.127+0000 7f5141084740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x56555aa1b3e0 con 0x56555aab5760 2024-09-10T09:39:16.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:16.127+0000 7f5141084740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x56555a9c2d80 con 0x56555aac0020 2024-09-10T09:39:16.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:16.127+0000 7f5141084740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x56555aa173a0 con 0x56555aab4e40 2024-09-10T09:39:16.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:16.127+0000 7f5137e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56555aab4e40 0x56555aab5220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:39:16.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:16.127+0000 7f5137e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56555aab4e40 0x56555aab5220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:44424/0 (socket says 172.21.15.5:44424) 2024-09-10T09:39:16.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:16.127+0000 7f5137e00640 1 -- 172.21.15.5:0/1132893282 learned_addr learned my addr 172.21.15.5:0/1132893282 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:39:16.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:16.128+0000 7f5136a00640 1 --2- 172.21.15.5:0/1132893282 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56555aac0020 0x56555aac2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:39:16.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:16.128+0000 7f5137400640 1 --2- 172.21.15.5:0/1132893282 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56555aab5760 0x56555aabfae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:39:16.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:16.128+0000 7f5137e00640 1 -- 172.21.15.5:0/1132893282 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56555aac0020 msgr2=0x56555aac2410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:39:16.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:16.128+0000 7f5137e00640 1 --2- 172.21.15.5:0/1132893282 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56555aac0020 0x56555aac2410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:16.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:16.128+0000 7f5137e00640 1 -- 172.21.15.5:0/1132893282 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56555aab5760 msgr2=0x56555aabfae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:39:16.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:16.128+0000 7f5137e00640 1 --2- 172.21.15.5:0/1132893282 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56555aab5760 0x56555aabfae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:16.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:16.128+0000 7f5137e00640 1 -- 172.21.15.5:0/1132893282 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56555a9e1ae0 con 0x56555aab4e40 2024-09-10T09:39:16.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:16.128+0000 7f5137400640 1 --2- 172.21.15.5:0/1132893282 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56555aab5760 0x56555aabfae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:39:16.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:16.129+0000 7f5137e00640 1 --2- 172.21.15.5:0/1132893282 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56555aab4e40 0x56555aab5220 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f5130009ec0 tx=0x7f51300107a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e99b33cff2642be2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:39:16.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:16.129+0000 7f512fe00640 1 --2- 172.21.15.5:0/843971433 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f511c0a2c90 0x7f511c0a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:39:16.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:16.129+0000 7f512fe00640 1 -- 172.21.15.5:0/843971433 <== mon.2 v2:172.21.15.102:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f51240efca0 con 0x7f51381010d0 2024-09-10T09:39:16.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:16.129+0000 7f5137400640 1 --2- 172.21.15.5:0/843971433 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f511c0a2c90 0x7f511c0a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:39:16.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:16.130+0000 7f512cc00640 1 -- 172.21.15.5:0/1132893282 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f513001c680 con 0x56555aab4e40 2024-09-10T09:39:16.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:16.130+0000 7f512cc00640 1 -- 172.21.15.5:0/1132893282 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f513001cd00 con 0x56555aab4e40 2024-09-10T09:39:16.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:16.130+0000 7f512cc00640 1 -- 172.21.15.5:0/1132893282 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f513001a380 con 0x56555aab4e40 2024-09-10T09:39:16.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:16.131+0000 7f5141084740 1 -- 172.21.15.5:0/1132893282 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56555aab4e40 msgr2=0x56555aab5220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:39:16.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:16.131+0000 7f5141084740 1 --2- 172.21.15.5:0/1132893282 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56555aab4e40 0x56555aab5220 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f5130009ec0 tx=0x7f51300107a0 comp rx=0 tx=0).stop 2024-09-10T09:39:16.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:16.131+0000 7f5137400640 1 --2- 172.21.15.5:0/843971433 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f511c0a2c90 0x7f511c0a5150 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x56555aab2230 tx=0x7f5128005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:39:16.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:16.131+0000 7f5141084740 1 -- 172.21.15.5:0/1132893282 shutdown_connections 2024-09-10T09:39:16.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:16.131+0000 7f5141084740 1 --2- 172.21.15.5:0/1132893282 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56555aac0020 0x56555aac2410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:16.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:16.131+0000 7f5141084740 1 --2- 172.21.15.5:0/1132893282 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56555aab5760 0x56555aabfae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:16.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:16.131+0000 7f5141084740 1 --2- 172.21.15.5:0/1132893282 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56555aab4e40 0x56555aab5220 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:16.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:16.131+0000 7f5141084740 1 -- 172.21.15.5:0/1132893282 >> 172.21.15.5:0/1132893282 conn(0x56555aab13d0 msgr2=0x56555aab17b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:39:16.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:16.131+0000 7f5141084740 1 -- 172.21.15.5:0/1132893282 shutdown_connections 2024-09-10T09:39:16.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:16.131+0000 7f5141084740 1 -- 172.21.15.5:0/1132893282 wait complete. 2024-09-10T09:39:16.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:16.132+0000 7f5141084740 1 -- start start 2024-09-10T09:39:16.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:16.132+0000 7f5141084740 1 Processor -- start 2024-09-10T09:39:16.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:16.133+0000 7f5141084740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56555aab4e40 0x56555aabcfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:39:16.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:16.133+0000 7f5141084740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56555aab5760 0x56555aabd4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:39:16.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:16.133+0000 7f5141084740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56555aac0020 0x56555aabda20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:39:16.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:16.133+0000 7f5141084740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x56555aac2bf0 con 0x56555aac0020 2024-09-10T09:39:16.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:16.133+0000 7f5141084740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x56555a9d60f0 con 0x56555aab4e40 2024-09-10T09:39:16.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:16.133+0000 7f5141084740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x56555aac2d70 con 0x56555aab5760 2024-09-10T09:39:16.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:16.133+0000 7f5141084740 10 client.0 _ll_register_callbacks cb 0x56555a9bdbd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:39:16.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:16.133+0000 7f5136a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56555aab5760 0x56555aabd4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:39:16.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:16.133+0000 7f5136a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56555aab5760 0x56555aabd4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:44436/0 (socket says 172.21.15.5:44436) 2024-09-10T09:39:16.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:16.133+0000 7f5136a00640 1 -- 172.21.15.5:0/2012926951 learned_addr learned my addr 172.21.15.5:0/2012926951 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:39:16.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:16.133+0000 7f5137400640 1 --2- 172.21.15.5:0/2012926951 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56555aab4e40 0x56555aabcfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:39:16.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:16.134+0000 7f5137e00640 1 --2- 172.21.15.5:0/2012926951 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56555aac0020 0x56555aabda20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:39:16.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:16.134+0000 7f5136a00640 1 -- 172.21.15.5:0/2012926951 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56555aab4e40 msgr2=0x56555aabcfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:39:16.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:16.134+0000 7f5136a00640 1 --2- 172.21.15.5:0/2012926951 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56555aab4e40 0x56555aabcfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:16.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:16.134+0000 7f5136a00640 1 -- 172.21.15.5:0/2012926951 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56555aac0020 msgr2=0x56555aabda20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:39:16.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:16.134+0000 7f5136a00640 1 --2- 172.21.15.5:0/2012926951 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56555aac0020 0x56555aabda20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:16.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:16.134+0000 7f5136a00640 1 -- 172.21.15.5:0/2012926951 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56555a78c3c0 con 0x56555aab5760 2024-09-10T09:39:16.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:16.134+0000 7f5136a00640 1 --2- 172.21.15.5:0/2012926951 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56555aab5760 0x56555aabd4e0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f51240b89e0 tx=0x7f51240b8370 comp rx=0 tx=0).ready entity=mon.1 client_cookie=13304f1944713f35 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:39:16.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:16.134+0000 7f5121600640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-09-10T09:39:16.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:16.134+0000 7f5121600640 1 -- 172.21.15.5:0/2012926951 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5124049030 con 0x56555aab5760 2024-09-10T09:39:16.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:16.134+0000 7f5121600640 1 -- 172.21.15.5:0/2012926951 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f51240ed450 con 0x56555aab5760 2024-09-10T09:39:16.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:16.134+0000 7f5141084740 1 -- 172.21.15.5:0/2012926951 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f513001cd00 con 0x56555aab5760 2024-09-10T09:39:16.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:16.135+0000 7f5121600640 1 -- 172.21.15.5:0/2012926951 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f51240bb970 con 0x56555aab5760 2024-09-10T09:39:16.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:16.135+0000 7f5141084740 10 client.25594 fetch_fsmap learned FSMap version 30 2024-09-10T09:39:16.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:16.135+0000 7f5141084740 1 -- 172.21.15.5:0/2012926951 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x56555aac2c50 con 0x56555aab5760 2024-09-10T09:39:16.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:16.135+0000 7f5121600640 1 -- 172.21.15.5:0/2012926951 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f513001cd00 con 0x56555aab5760 2024-09-10T09:39:16.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:16.135+0000 7f5121600640 1 -- 172.21.15.5:0/2012926951 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f51240f5070 con 0x56555aab5760 2024-09-10T09:39:16.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:16.135+0000 7f5141084740 10 client.25594 fetch_fsmap finished waiting for FSMap version 30 2024-09-10T09:39:16.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:16.136+0000 7f5141084740 10 client.25594 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T09:39:16.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:16.136+0000 7f5141084740 20 client.25594 populate_metadata read hostname 'smithi005' 2024-09-10T09:39:16.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:16.136+0000 7f5141084740 1 --2- 172.21.15.5:0/2012926951 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x56555aad00b0 0x56555aad24d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:39:16.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:16.136+0000 7f5141084740 4 client.25594 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:39:16.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:16.136+0000 7f5141084740 1 -- 172.21.15.5:0/2012926951 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56555aa28950 con 0x56555aad00b0 2024-09-10T09:39:16.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:16.136+0000 7f5137400640 1 --2- 172.21.15.5:0/2012926951 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x56555aad00b0 0x56555aad24d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:39:16.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:16.137+0000 7f5137400640 1 --2- 172.21.15.5:0/2012926951 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x56555aad00b0 0x56555aad24d0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3039a0ab6de2b47 server_cookie=87b4af63affcf401 in_seq=0 out_seq=0 2024-09-10T09:39:16.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:16.137+0000 7f5121600640 10 client.25594 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T09:39:16.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:16.138+0000 7f5121600640 1 -- 172.21.15.5:0/2012926951 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56555aa28950 con 0x56555aad00b0 2024-09-10T09:39:16.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:16.138+0000 7f5121600640 10 client.25594 handle_command_reply: tid=0 2024-09-10T09:39:16.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:16.138+0000 7f5141084740 1 client.25594 shutdown 2024-09-10T09:39:16.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:16.138+0000 7f5141084740 1 -- 172.21.15.5:0/2012926951 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56555aab5760 msgr2=0x56555aabd4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:39:16.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:16.138+0000 7f5141084740 1 --2- 172.21.15.5:0/2012926951 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56555aab5760 0x56555aabd4e0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f51240b89e0 tx=0x7f51240b8370 comp rx=0 tx=0).stop 2024-09-10T09:39:16.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:16.138+0000 7f5141084740 1 -- 172.21.15.5:0/2012926951 shutdown_connections 2024-09-10T09:39:16.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:16.138+0000 7f5141084740 1 --2- 172.21.15.5:0/2012926951 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x56555aad00b0 0x56555aad24d0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:16.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:16.138+0000 7f5141084740 1 --2- 172.21.15.5:0/2012926951 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56555aac0020 0x56555aabda20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:16.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:16.138+0000 7f5141084740 1 --2- 172.21.15.5:0/2012926951 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56555aab5760 0x56555aabd4e0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:16.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:16.138+0000 7f5141084740 1 --2- 172.21.15.5:0/2012926951 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56555aab4e40 0x56555aabcfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:16.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:16.138+0000 7f5121600640 0 client.25594 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T09:39:16.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:16.138+0000 7f5141084740 1 -- 172.21.15.5:0/2012926951 >> 172.21.15.5:0/2012926951 conn(0x56555aab13d0 msgr2=0x56555aab7100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:39:16.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:16.139+0000 7f5141084740 1 -- 172.21.15.5:0/2012926951 shutdown_connections 2024-09-10T09:39:16.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:16.139+0000 7f5141084740 1 -- 172.21.15.5:0/2012926951 wait complete. 2024-09-10T09:39:16.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:16.139+0000 7f5141084740 20 client.25594 trim_cache size 0 max 16384 2024-09-10T09:39:16.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:16.156+0000 7f5141084740 1 Processor -- start 2024-09-10T09:39:16.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:16.157+0000 7f5141084740 1 -- start start 2024-09-10T09:39:16.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:16.157+0000 7f5141084740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56555aab4e40 0x56555aad7de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:39:16.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:16.157+0000 7f5141084740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56555aab5760 0x56555aad8320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:39:16.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:16.157+0000 7f5141084740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56555aac0020 0x56555aab68e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:39:16.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:16.157+0000 7f5141084740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f51381b0ee0 con 0x56555aac0020 2024-09-10T09:39:16.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:16.157+0000 7f5141084740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f51381b2a20 con 0x56555aab5760 2024-09-10T09:39:16.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:16.157+0000 7f5141084740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f51381b15b0 con 0x56555aab4e40 2024-09-10T09:39:16.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:16.157+0000 7f5137400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56555aab4e40 0x56555aad7de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:39:16.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:16.157+0000 7f5136a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56555aab5760 0x56555aad8320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:39:16.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:16.157+0000 7f5137400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56555aab4e40 0x56555aad7de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:44444/0 (socket says 172.21.15.5:44444) 2024-09-10T09:39:16.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:16.157+0000 7f5137400640 1 -- 172.21.15.5:0/2262937347 learned_addr learned my addr 172.21.15.5:0/2262937347 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:39:16.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:16.158+0000 7f5137e00640 1 --2- 172.21.15.5:0/2262937347 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56555aac0020 0x56555aab68e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:39:16.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:16.158+0000 7f5136a00640 1 -- 172.21.15.5:0/2262937347 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56555aab4e40 msgr2=0x56555aad7de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:39:16.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:16.158+0000 7f5136a00640 1 --2- 172.21.15.5:0/2262937347 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56555aab4e40 0x56555aad7de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:16.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:16.158+0000 7f5136a00640 1 -- 172.21.15.5:0/2262937347 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56555aac0020 msgr2=0x56555aab68e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:39:16.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:16.158+0000 7f5136a00640 1 --2- 172.21.15.5:0/2262937347 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56555aac0020 0x56555aab68e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:16.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:16.158+0000 7f5136a00640 1 -- 172.21.15.5:0/2262937347 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56555aad2b50 con 0x56555aab5760 2024-09-10T09:39:16.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:16.158+0000 7f5137400640 1 --2- 172.21.15.5:0/2262937347 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56555aab4e40 0x56555aad7de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:39:16.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:16.158+0000 7f5136a00640 1 --2- 172.21.15.5:0/2262937347 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56555aab5760 0x56555aad8320 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f51240ba2c0 tx=0x7f512402a3f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=97e988a8ea726338 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:39:16.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:16.159+0000 7f5123e00640 1 -- 172.21.15.5:0/2262937347 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5124049070 con 0x56555aab5760 2024-09-10T09:39:16.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:16.159+0000 7f5123e00640 1 -- 172.21.15.5:0/2262937347 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f51240bc710 con 0x56555aab5760 2024-09-10T09:39:16.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:16.159+0000 7f5123e00640 1 -- 172.21.15.5:0/2262937347 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5124005d50 con 0x56555aab5760 2024-09-10T09:39:16.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:16.160+0000 7f5141084740 1 -- 172.21.15.5:0/2262937347 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56555aab5760 msgr2=0x56555aad8320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:39:16.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:16.160+0000 7f5141084740 1 --2- 172.21.15.5:0/2262937347 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56555aab5760 0x56555aad8320 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f51240ba2c0 tx=0x7f512402a3f0 comp rx=0 tx=0).stop 2024-09-10T09:39:16.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:16.160+0000 7f5141084740 1 -- 172.21.15.5:0/2262937347 shutdown_connections 2024-09-10T09:39:16.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:16.160+0000 7f5141084740 1 --2- 172.21.15.5:0/2262937347 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56555aac0020 0x56555aab68e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:16.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:16.160+0000 7f5141084740 1 --2- 172.21.15.5:0/2262937347 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56555aab5760 0x56555aad8320 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:16.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:16.160+0000 7f5141084740 1 --2- 172.21.15.5:0/2262937347 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56555aab4e40 0x56555aad7de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:16.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:16.160+0000 7f5141084740 1 -- 172.21.15.5:0/2262937347 >> 172.21.15.5:0/2262937347 conn(0x56555aab13d0 msgr2=0x56555aabc090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:39:16.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:16.160+0000 7f5141084740 1 -- 172.21.15.5:0/2262937347 shutdown_connections 2024-09-10T09:39:16.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:16.160+0000 7f5141084740 1 -- 172.21.15.5:0/2262937347 wait complete. 2024-09-10T09:39:16.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:16.161+0000 7f5141084740 1 -- start start 2024-09-10T09:39:16.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:16.161+0000 7f5141084740 1 Processor -- start 2024-09-10T09:39:16.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:16.162+0000 7f5141084740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56555aab4e40 0x56555aac3d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:39:16.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:16.162+0000 7f5141084740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56555aab5760 0x56555aac4250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:39:16.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:16.162+0000 7f5141084740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56555aac0020 0x56555aac4790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:39:16.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:16.162+0000 7f5141084740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f51381b2350 con 0x56555aab5760 2024-09-10T09:39:16.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:16.162+0000 7f5141084740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f51381b1c80 con 0x56555aab4e40 2024-09-10T09:39:16.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:16.162+0000 7f5141084740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f51381aeec0 con 0x56555aac0020 2024-09-10T09:39:16.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:16.162+0000 7f5141084740 10 client.0 _ll_register_callbacks cb 0x7f51240f3440 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:39:16.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:16.162+0000 7f5137400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56555aab4e40 0x56555aac3d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:39:16.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:16.162+0000 7f5137400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56555aab4e40 0x56555aac3d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:38780/0 (socket says 172.21.15.5:38780) 2024-09-10T09:39:16.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:16.162+0000 7f5137400640 1 -- 172.21.15.5:0/428877508 learned_addr learned my addr 172.21.15.5:0/428877508 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:39:16.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:16.162+0000 7f5136a00640 1 --2- 172.21.15.5:0/428877508 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56555aab5760 0x56555aac4250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:39:16.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:16.162+0000 7f5137400640 1 -- 172.21.15.5:0/428877508 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56555aac0020 msgr2=0x56555aac4790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:39:16.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:16.162+0000 7f5137400640 1 --2- 172.21.15.5:0/428877508 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56555aac0020 0x56555aac4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:16.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:16.162+0000 7f5137400640 1 -- 172.21.15.5:0/428877508 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56555aab5760 msgr2=0x56555aac4250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:39:16.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:16.162+0000 7f5137400640 1 --2- 172.21.15.5:0/428877508 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56555aab5760 0x56555aac4250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:16.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:16.163+0000 7f5137400640 1 -- 172.21.15.5:0/428877508 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56555aab0c80 con 0x56555aab4e40 2024-09-10T09:39:16.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:16.163+0000 7f5136a00640 1 --2- 172.21.15.5:0/428877508 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56555aab5760 0x56555aac4250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T09:39:16.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:16.163+0000 7f5137400640 1 --2- 172.21.15.5:0/428877508 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56555aab4e40 0x56555aac3d10 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f5128012880 tx=0x7f51280046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9fb67b4e8bdb8135 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:39:16.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:16.163+0000 7f5121600640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-09-10T09:39:16.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:16.164+0000 7f5121600640 1 -- 172.21.15.5:0/428877508 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5128028970 con 0x56555aab4e40 2024-09-10T09:39:16.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:16.164+0000 7f5121600640 1 -- 172.21.15.5:0/428877508 <== mon.2 v2:172.21.15.102:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f5128028b10 con 0x56555aab4e40 2024-09-10T09:39:16.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:16.164+0000 7f5141084740 1 -- 172.21.15.5:0/428877508 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f51240bc710 con 0x56555aab4e40 2024-09-10T09:39:16.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:16.164+0000 7f5121600640 1 -- 172.21.15.5:0/428877508 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f5128005020 con 0x56555aab4e40 2024-09-10T09:39:16.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:16.164+0000 7f5141084740 10 client.25574 fetch_fsmap learned FSMap version 30 2024-09-10T09:39:16.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:16.164+0000 7f5141084740 1 -- 172.21.15.5:0/428877508 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x56555aab4c70 con 0x56555aab4e40 2024-09-10T09:39:16.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:16.165+0000 7f5121600640 1 -- 172.21.15.5:0/428877508 <== mon.2 v2:172.21.15.102:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5128039070 con 0x56555aab4e40 2024-09-10T09:39:16.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:16.165+0000 7f5121600640 1 -- 172.21.15.5:0/428877508 <== mon.2 v2:172.21.15.102:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f5128019040 con 0x56555aab4e40 2024-09-10T09:39:16.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:16.166+0000 7f5141084740 10 client.25574 fetch_fsmap finished waiting for FSMap version 30 2024-09-10T09:39:16.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:16.166+0000 7f5141084740 10 client.25574 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T09:39:16.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:16.166+0000 7f5141084740 20 client.25574 populate_metadata read hostname 'smithi005' 2024-09-10T09:39:16.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:16.166+0000 7f5141084740 1 --2- 172.21.15.5:0/428877508 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x56555ab43c80 0x56555ab460e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:39:16.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:16.166+0000 7f5141084740 4 client.25574 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T09:39:16.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:16.166+0000 7f5141084740 1 -- 172.21.15.5:0/428877508 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56555aa22e60 con 0x56555ab43c80 2024-09-10T09:39:16.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:16.166+0000 7f5136a00640 1 --2- 172.21.15.5:0/428877508 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x56555ab43c80 0x56555ab460e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:39:16.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:16.167+0000 7f5136a00640 1 --2- 172.21.15.5:0/428877508 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x56555ab43c80 0x56555ab460e0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2e163bf2b31d6d1 server_cookie=572d216dcb2853d1 in_seq=0 out_seq=0 2024-09-10T09:39:16.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:16.167+0000 7f5121600640 10 client.25574 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T09:39:16.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:16.167+0000 7f5121600640 1 -- 172.21.15.5:0/428877508 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56555aa22e60 con 0x56555ab43c80 2024-09-10T09:39:16.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:16.167+0000 7f5121600640 10 client.25574 handle_command_reply: tid=0 2024-09-10T09:39:16.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:16.167+0000 7f5141084740 1 client.25574 shutdown 2024-09-10T09:39:16.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:16.167+0000 7f5141084740 1 -- 172.21.15.5:0/428877508 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56555aab4e40 msgr2=0x56555aac3d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:39:16.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:16.167+0000 7f5141084740 1 --2- 172.21.15.5:0/428877508 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56555aab4e40 0x56555aac3d10 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f5128012880 tx=0x7f51280046c0 comp rx=0 tx=0).stop 2024-09-10T09:39:16.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:16.167+0000 7f5141084740 1 -- 172.21.15.5:0/428877508 shutdown_connections 2024-09-10T09:39:16.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:16.167+0000 7f5141084740 1 --2- 172.21.15.5:0/428877508 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x56555ab43c80 0x56555ab460e0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:16.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:16.167+0000 7f5121600640 0 client.25574 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T09:39:16.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:16.168+0000 7f5141084740 1 --2- 172.21.15.5:0/428877508 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56555aac0020 0x56555aac4790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:16.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:16.168+0000 7f5141084740 1 --2- 172.21.15.5:0/428877508 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56555aab5760 0x56555aac4250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:16.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:16.168+0000 7f5141084740 1 --2- 172.21.15.5:0/428877508 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56555aab4e40 0x56555aac3d10 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:16.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:16.168+0000 7f5141084740 1 -- 172.21.15.5:0/428877508 >> 172.21.15.5:0/428877508 conn(0x56555aab13d0 msgr2=0x56555aabe800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:39:16.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:16.168+0000 7f5141084740 1 -- 172.21.15.5:0/428877508 shutdown_connections 2024-09-10T09:39:16.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:16.168+0000 7f5141084740 1 -- 172.21.15.5:0/428877508 wait complete. 2024-09-10T09:39:16.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:16.168+0000 7f5141084740 20 client.25574 trim_cache size 0 max 16384 2024-09-10T09:39:16.170 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-10T09:39:16.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:16.169+0000 7f513d000640 1 -- 172.21.15.5:0/843971433 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f511c0a2c90 msgr2=0x7f511c0a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:39:16.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:16.169+0000 7f513d000640 1 --2- 172.21.15.5:0/843971433 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f511c0a2c90 0x7f511c0a5150 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x56555aab2230 tx=0x7f5128005f90 comp rx=0 tx=0).stop 2024-09-10T09:39:16.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:16.169+0000 7f513d000640 1 -- 172.21.15.5:0/843971433 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f51381010d0 msgr2=0x7f513819aa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:39:16.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:16.169+0000 7f513d000640 1 --2- 172.21.15.5:0/843971433 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f51381010d0 0x7f513819aa40 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f5124004770 tx=0x7f5124003c10 comp rx=0 tx=0).stop 2024-09-10T09:39:16.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:16.169+0000 7f513d000640 1 -- 172.21.15.5:0/843971433 shutdown_connections 2024-09-10T09:39:16.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:16.169+0000 7f513d000640 1 --2- 172.21.15.5:0/843971433 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f511c0a2c90 0x7f511c0a5150 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:16.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:16.169+0000 7f513d000640 1 --2- 172.21.15.5:0/843971433 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f513810e2d0 0x7f513819edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:16.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:16.169+0000 7f513d000640 1 --2- 172.21.15.5:0/843971433 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f51381010d0 0x7f513819aa40 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:16.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:16.169+0000 7f513d000640 1 --2- 172.21.15.5:0/843971433 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5138100720 0x7f513819a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:16.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:16.169+0000 7f513d000640 1 -- 172.21.15.5:0/843971433 >> 172.21.15.5:0/843971433 conn(0x7f51380fc610 msgr2=0x7f51380fea00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:39:16.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:16.170+0000 7f513d000640 1 -- 172.21.15.5:0/843971433 shutdown_connections 2024-09-10T09:39:16.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:16.170+0000 7f513d000640 1 -- 172.21.15.5:0/843971433 wait complete. 2024-09-10T09:39:17.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:39:17 smithi005 ceph-mon[30350]: pgmap v421: 129 pgs: 129 active+clean; 665 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 597 B/s wr, 12 op/s 2024-09-10T09:39:17.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:39:17 smithi159 ceph-mon[35513]: pgmap v421: 129 pgs: 129 active+clean; 665 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 597 B/s wr, 12 op/s 2024-09-10T09:39:17.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:39:17 smithi102 ceph-mon[35442]: pgmap v421: 129 pgs: 129 active+clean; 665 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 597 B/s wr, 12 op/s 2024-09-10T09:39:18.019 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-10T09:39:18.019 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-10T09:39:18.497 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:39:19.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:19.136+0000 7fce89600640 1 -- 172.21.15.5:0/2298270678 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fce84101dd0 msgr2=0x7fce841021b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:39:19.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:19.136+0000 7fce89600640 1 --2- 172.21.15.5:0/2298270678 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fce84101dd0 0x7fce841021b0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fce74009900 tx=0x7fce7402a410 comp rx=0 tx=0).stop 2024-09-10T09:39:19.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:19.136+0000 7fce89600640 1 -- 172.21.15.5:0/2298270678 shutdown_connections 2024-09-10T09:39:19.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:19.136+0000 7fce89600640 1 --2- 172.21.15.5:0/2298270678 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fce84106f00 0x7fce8410ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:19.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:19.136+0000 7fce89600640 1 --2- 172.21.15.5:0/2298270678 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fce84102780 0x7fce841067d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:19.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:19.136+0000 7fce89600640 1 --2- 172.21.15.5:0/2298270678 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fce84101dd0 0x7fce841021b0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:19.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:19.136+0000 7fce89600640 1 -- 172.21.15.5:0/2298270678 >> 172.21.15.5:0/2298270678 conn(0x7fce840fd8b0 msgr2=0x7fce840ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:39:19.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:19.137+0000 7fce89600640 1 -- 172.21.15.5:0/2298270678 shutdown_connections 2024-09-10T09:39:19.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:19.137+0000 7fce89600640 1 -- 172.21.15.5:0/2298270678 wait complete. 2024-09-10T09:39:19.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:19.138+0000 7fce89600640 1 Processor -- start 2024-09-10T09:39:19.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:19.138+0000 7fce89600640 1 -- start start 2024-09-10T09:39:19.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:19.139+0000 7fce89600640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fce84101dd0 0x7fce841a4780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:39:19.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:19.139+0000 7fce89600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fce84102780 0x7fce841a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:39:19.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:19.139+0000 7fce89600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fce84106f00 0x7fce8410dca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:39:19.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:19.139+0000 7fce89600640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fce84111fb0 con 0x7fce84102780 2024-09-10T09:39:19.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:19.139+0000 7fce89600640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fce84111e30 con 0x7fce84106f00 2024-09-10T09:39:19.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:19.139+0000 7fce89600640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fce84112130 con 0x7fce84101dd0 2024-09-10T09:39:19.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:19.139+0000 7fce82a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fce84101dd0 0x7fce841a4780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:39:19.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:19.139+0000 7fce82a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fce84101dd0 0x7fce841a4780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:44482/0 (socket says 172.21.15.5:44482) 2024-09-10T09:39:19.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:19.139+0000 7fce82a00640 1 -- 172.21.15.5:0/3929742432 learned_addr learned my addr 172.21.15.5:0/3929742432 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:39:19.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:19.139+0000 7fce82000640 1 --2- 172.21.15.5:0/3929742432 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fce84102780 0x7fce841a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:39:19.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:19.140+0000 7fce82a00640 1 -- 172.21.15.5:0/3929742432 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fce84106f00 msgr2=0x7fce8410dca0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-10T09:39:19.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:19.140+0000 7fce82a00640 1 --2- 172.21.15.5:0/3929742432 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fce84106f00 0x7fce8410dca0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:19.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:19.140+0000 7fce82a00640 1 -- 172.21.15.5:0/3929742432 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fce84102780 msgr2=0x7fce841a4cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:39:19.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:19.140+0000 7fce82a00640 1 --2- 172.21.15.5:0/3929742432 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fce84102780 0x7fce841a4cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:19.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:19.140+0000 7fce82a00640 1 -- 172.21.15.5:0/3929742432 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce8410e4d0 con 0x7fce84101dd0 2024-09-10T09:39:19.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:19.140+0000 7fce82a00640 1 --2- 172.21.15.5:0/3929742432 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fce84101dd0 0x7fce841a4780 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fce74004770 tx=0x7fce74003c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:39:19.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:19.141+0000 7fce7b400640 1 -- 172.21.15.5:0/3929742432 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce7403f070 con 0x7fce84101dd0 2024-09-10T09:39:19.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:19.141+0000 7fce7b400640 1 -- 172.21.15.5:0/3929742432 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fce74031ca0 con 0x7fce84101dd0 2024-09-10T09:39:19.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:19.141+0000 7fce7b400640 1 -- 172.21.15.5:0/3929742432 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce74035030 con 0x7fce84101dd0 2024-09-10T09:39:19.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:19.141+0000 7fce89600640 1 -- 172.21.15.5:0/3929742432 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce8410e760 con 0x7fce84101dd0 2024-09-10T09:39:19.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:19.141+0000 7fce89600640 1 -- 172.21.15.5:0/3929742432 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce841ab570 con 0x7fce84101dd0 2024-09-10T09:39:19.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:19.144+0000 7fce8c20e740 1 Processor -- start 2024-09-10T09:39:19.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:19.145+0000 7fce8c20e740 1 -- start start 2024-09-10T09:39:19.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:19.145+0000 7fce8c20e740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562a61de0ac0 0x562a61de0ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:39:19.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:19.145+0000 7fce8c20e740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562a61de13e0 0x562a61deb760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:39:19.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:19.145+0000 7fce8c20e740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562a61debca0 0x562a61dee090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:39:19.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:19.145+0000 7fce8c20e740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x562a61d47290 con 0x562a61de13e0 2024-09-10T09:39:19.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:19.145+0000 7fce8c20e740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x562a61ceeb40 con 0x562a61de0ac0 2024-09-10T09:39:19.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:19.145+0000 7fce8c20e740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x562a61d43250 con 0x562a61debca0 2024-09-10T09:39:19.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:19.146+0000 7fce83400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562a61de0ac0 0x562a61de0ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:39:19.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:19.146+0000 7fce82000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562a61debca0 0x562a61dee090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:39:19.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:19.146+0000 7fce83400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562a61de0ac0 0x562a61de0ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:38794/0 (socket says 172.21.15.5:38794) 2024-09-10T09:39:19.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:19.146+0000 7fce83400640 1 -- 172.21.15.5:0/1595940744 learned_addr learned my addr 172.21.15.5:0/1595940744 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:39:19.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:19.146+0000 7fce82000640 1 -- 172.21.15.5:0/1595940744 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562a61de0ac0 msgr2=0x562a61de0ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:39:19.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:19.146+0000 7fce82000640 1 --2- 172.21.15.5:0/1595940744 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562a61de0ac0 0x562a61de0ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:19.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:19.146+0000 7fce82000640 1 -- 172.21.15.5:0/1595940744 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562a61de13e0 msgr2=0x562a61deb760 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T09:39:19.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:19.146+0000 7fce82000640 1 --2- 172.21.15.5:0/1595940744 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562a61de13e0 0x562a61deb760 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:19.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:19.146+0000 7fce82000640 1 -- 172.21.15.5:0/1595940744 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562a61ab86a0 con 0x562a61debca0 2024-09-10T09:39:19.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:19.147+0000 7fce82000640 1 --2- 172.21.15.5:0/1595940744 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562a61debca0 0x562a61dee090 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fce8410ef50 tx=0x7fce70011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=52e50763a8fd68e3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:39:19.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:19.147+0000 7fce7b400640 1 -- 172.21.15.5:0/3929742432 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 19) ==== 137883+0+0 (secure 0 0 0) 0x7fce74042070 con 0x7fce84101dd0 2024-09-10T09:39:19.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:19.147+0000 7fce6f400640 1 -- 172.21.15.5:0/1595940744 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce70004230 con 0x562a61debca0 2024-09-10T09:39:19.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:19.147+0000 7fce6f400640 1 -- 172.21.15.5:0/1595940744 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fce70009e10 con 0x562a61debca0 2024-09-10T09:39:19.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:19.147+0000 7fce6f400640 1 -- 172.21.15.5:0/1595940744 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce70004d90 con 0x562a61debca0 2024-09-10T09:39:19.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:19.149+0000 7fce8c20e740 1 -- 172.21.15.5:0/1595940744 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562a61debca0 msgr2=0x562a61dee090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:39:19.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:19.149+0000 7fce8c20e740 1 --2- 172.21.15.5:0/1595940744 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562a61debca0 0x562a61dee090 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fce8410ef50 tx=0x7fce70011810 comp rx=0 tx=0).stop 2024-09-10T09:39:19.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:19.149+0000 7fce7b400640 1 --2- 172.21.15.5:0/3929742432 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fce680a2c90 0x7fce680a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:39:19.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:19.149+0000 7fce7b400640 1 -- 172.21.15.5:0/3929742432 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fce740ef8a0 con 0x7fce84101dd0 2024-09-10T09:39:19.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:19.150+0000 7fce8c20e740 1 -- 172.21.15.5:0/1595940744 shutdown_connections 2024-09-10T09:39:19.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:19.150+0000 7fce8c20e740 1 --2- 172.21.15.5:0/1595940744 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562a61debca0 0x562a61dee090 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:19.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:19.150+0000 7fce8c20e740 1 --2- 172.21.15.5:0/1595940744 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562a61de13e0 0x562a61deb760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:19.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:19.150+0000 7fce8c20e740 1 --2- 172.21.15.5:0/1595940744 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562a61de0ac0 0x562a61de0ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:19.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:19.150+0000 7fce8c20e740 1 -- 172.21.15.5:0/1595940744 >> 172.21.15.5:0/1595940744 conn(0x562a61ddd050 msgr2=0x562a61ddd430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:39:19.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:19.150+0000 7fce83400640 1 --2- 172.21.15.5:0/3929742432 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fce680a2c90 0x7fce680a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:39:19.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:19.150+0000 7fce8c20e740 1 -- 172.21.15.5:0/1595940744 shutdown_connections 2024-09-10T09:39:19.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:19.150+0000 7fce8c20e740 1 -- 172.21.15.5:0/1595940744 wait complete. 2024-09-10T09:39:19.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:19.151+0000 7fce8c20e740 1 -- start start 2024-09-10T09:39:19.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:19.151+0000 7fce8c20e740 1 Processor -- start 2024-09-10T09:39:19.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:19.151+0000 7fce83400640 1 --2- 172.21.15.5:0/3929742432 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fce680a2c90 0x7fce680a5150 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fce7c005fd0 tx=0x7fce7c009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:39:19.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:19.152+0000 7fce8c20e740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562a61de0ac0 0x562a61deb630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:39:19.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:19.152+0000 7fce8c20e740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562a61de13e0 0x562a61de88e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:39:19.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:19.152+0000 7fce8c20e740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562a61debca0 0x562a61de8e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:39:19.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:19.152+0000 7fce8c20e740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x562a61dee910 con 0x562a61debca0 2024-09-10T09:39:19.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:19.152+0000 7fce8c20e740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x562a61d01fa0 con 0x562a61de13e0 2024-09-10T09:39:19.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:19.152+0000 7fce8c20e740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x562a61deea90 con 0x562a61de0ac0 2024-09-10T09:39:19.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:19.152+0000 7fce8c20e740 10 client.0 _ll_register_callbacks cb 0x562a61a55ba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:39:19.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:19.153+0000 7fce82a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562a61de0ac0 0x562a61deb630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:39:19.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:19.153+0000 7fce82a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562a61de0ac0 0x562a61deb630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:44508/0 (socket says 172.21.15.5:44508) 2024-09-10T09:39:19.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:19.153+0000 7fce82a00640 1 -- 172.21.15.5:0/3881544656 learned_addr learned my addr 172.21.15.5:0/3881544656 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:39:19.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:19.153+0000 7fce83400640 1 --2- 172.21.15.5:0/3881544656 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562a61debca0 0x562a61de8e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:39:19.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:19.153+0000 7fce82000640 1 --2- 172.21.15.5:0/3881544656 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562a61de13e0 0x562a61de88e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:39:19.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:19.154+0000 7fce82000640 1 -- 172.21.15.5:0/3881544656 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562a61de0ac0 msgr2=0x562a61deb630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:39:19.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:19.154+0000 7fce82000640 1 --2- 172.21.15.5:0/3881544656 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562a61de0ac0 0x562a61deb630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:19.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:19.154+0000 7fce82000640 1 -- 172.21.15.5:0/3881544656 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562a61debca0 msgr2=0x562a61de8e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:39:19.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:19.154+0000 7fce82000640 1 --2- 172.21.15.5:0/3881544656 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562a61debca0 0x562a61de8e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:19.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:19.154+0000 7fce82000640 1 -- 172.21.15.5:0/3881544656 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562a61de9690 con 0x562a61de13e0 2024-09-10T09:39:19.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:19.154+0000 7fce83400640 1 --2- 172.21.15.5:0/3881544656 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562a61debca0 0x562a61de8e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:39:19.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:19.154+0000 7fce82a00640 1 --2- 172.21.15.5:0/3881544656 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562a61de0ac0 0x562a61deb630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:39:19.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:19.154+0000 7fce82000640 1 --2- 172.21.15.5:0/3881544656 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562a61de13e0 0x562a61de88e0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fce70011d10 tx=0x7fce70003820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7c01f49f58c01c0d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:39:19.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:19.154+0000 7fce63e00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-09-10T09:39:19.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:19.155+0000 7fce63e00640 1 -- 172.21.15.5:0/3881544656 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce70027020 con 0x562a61de13e0 2024-09-10T09:39:19.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:19.155+0000 7fce63e00640 1 -- 172.21.15.5:0/3881544656 <== mon.2 v2:172.21.15.102:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fce70003b90 con 0x562a61de13e0 2024-09-10T09:39:19.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:19.155+0000 7fce8c20e740 1 -- 172.21.15.5:0/3881544656 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fce70009e10 con 0x562a61de13e0 2024-09-10T09:39:19.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:19.155+0000 7fce63e00640 1 -- 172.21.15.5:0/3881544656 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fce70032050 con 0x562a61de13e0 2024-09-10T09:39:19.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:19.155+0000 7fce8c20e740 10 client.25583 fetch_fsmap learned FSMap version 30 2024-09-10T09:39:19.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:19.155+0000 7fce8c20e740 1 -- 172.21.15.5:0/3881544656 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x562a61e0b8a0 con 0x562a61de13e0 2024-09-10T09:39:19.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:19.156+0000 7fce63e00640 1 -- 172.21.15.5:0/3881544656 <== mon.2 v2:172.21.15.102:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fce70009e10 con 0x562a61de13e0 2024-09-10T09:39:19.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:19.156+0000 7fce63e00640 1 -- 172.21.15.5:0/3881544656 <== mon.2 v2:172.21.15.102:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fce70018070 con 0x562a61de13e0 2024-09-10T09:39:19.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:19.156+0000 7fce8c20e740 10 client.25583 fetch_fsmap finished waiting for FSMap version 30 2024-09-10T09:39:19.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:19.156+0000 7fce8c20e740 10 client.25583 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T09:39:19.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:19.156+0000 7fce8c20e740 20 client.25583 populate_metadata read hostname 'smithi005' 2024-09-10T09:39:19.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:19.156+0000 7fce8c20e740 1 --2- 172.21.15.5:0/3881544656 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x562a61e0cd60 0x562a61e0f150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:39:19.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:19.156+0000 7fce8c20e740 4 client.25583 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:39:19.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:19.156+0000 7fce8c20e740 1 -- 172.21.15.5:0/3881544656 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562a61d54800 con 0x562a61e0cd60 2024-09-10T09:39:19.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:19.156+0000 7fce82a00640 1 --2- 172.21.15.5:0/3881544656 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x562a61e0cd60 0x562a61e0f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:39:19.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:19.157+0000 7fce82a00640 1 --2- 172.21.15.5:0/3881544656 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x562a61e0cd60 0x562a61e0f150 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cab1a624ff625619 server_cookie=20a78941abaad63a in_seq=0 out_seq=0 2024-09-10T09:39:19.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:19.157+0000 7fce63e00640 10 client.25583 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T09:39:19.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:19.158+0000 7fce63e00640 1 -- 172.21.15.5:0/3881544656 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562a61d54800 con 0x562a61e0cd60 2024-09-10T09:39:19.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:19.158+0000 7fce63e00640 10 client.25583 handle_command_reply: tid=0 2024-09-10T09:39:19.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:19.158+0000 7fce8c20e740 1 client.25583 shutdown 2024-09-10T09:39:19.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:19.158+0000 7fce8c20e740 1 -- 172.21.15.5:0/3881544656 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562a61de13e0 msgr2=0x562a61de88e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:39:19.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:19.158+0000 7fce8c20e740 1 --2- 172.21.15.5:0/3881544656 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562a61de13e0 0x562a61de88e0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fce70011d10 tx=0x7fce70003820 comp rx=0 tx=0).stop 2024-09-10T09:39:19.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:19.159+0000 7fce8c20e740 1 -- 172.21.15.5:0/3881544656 shutdown_connections 2024-09-10T09:39:19.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:19.159+0000 7fce8c20e740 1 --2- 172.21.15.5:0/3881544656 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x562a61e0cd60 0x562a61e0f150 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:19.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:19.159+0000 7fce8c20e740 1 --2- 172.21.15.5:0/3881544656 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562a61debca0 0x562a61de8e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:19.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:19.159+0000 7fce8c20e740 1 --2- 172.21.15.5:0/3881544656 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562a61de13e0 0x562a61de88e0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:19.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:19.159+0000 7fce8c20e740 1 --2- 172.21.15.5:0/3881544656 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562a61de0ac0 0x562a61deb630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:19.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:19.159+0000 7fce8c20e740 1 -- 172.21.15.5:0/3881544656 >> 172.21.15.5:0/3881544656 conn(0x562a61ddd050 msgr2=0x562a61de2d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:39:19.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:19.159+0000 7fce63e00640 0 client.25583 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T09:39:19.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:19.159+0000 7fce8c20e740 1 -- 172.21.15.5:0/3881544656 shutdown_connections 2024-09-10T09:39:19.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:19.159+0000 7fce8c20e740 1 -- 172.21.15.5:0/3881544656 wait complete. 2024-09-10T09:39:19.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:19.159+0000 7fce8c20e740 20 client.25583 trim_cache size 0 max 16384 2024-09-10T09:39:19.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:19.195+0000 7fce8c20e740 1 Processor -- start 2024-09-10T09:39:19.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:19.195+0000 7fce8c20e740 1 -- start start 2024-09-10T09:39:19.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:19.195+0000 7fce8c20e740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562a61de0ac0 0x562a61dea850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:39:19.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:19.195+0000 7fce8c20e740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562a61de13e0 0x562a61e14870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:39:19.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:19.195+0000 7fce8c20e740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562a61debca0 0x562a61e14db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:39:19.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:19.195+0000 7fce8c20e740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fce841acd70 con 0x562a61de13e0 2024-09-10T09:39:19.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:19.195+0000 7fce8c20e740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fce841ae8b0 con 0x562a61debca0 2024-09-10T09:39:19.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:19.195+0000 7fce8c20e740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fce841ad440 con 0x562a61de0ac0 2024-09-10T09:39:19.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:19.196+0000 7fce82000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562a61de13e0 0x562a61e14870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:39:19.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:19.196+0000 7fce82000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562a61de13e0 0x562a61e14870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60826/0 (socket says 172.21.15.5:60826) 2024-09-10T09:39:19.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:19.196+0000 7fce82a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562a61de0ac0 0x562a61dea850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:39:19.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:19.196+0000 7fce82000640 1 -- 172.21.15.5:0/165806320 learned_addr learned my addr 172.21.15.5:0/165806320 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:39:19.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:19.196+0000 7fce83400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562a61debca0 0x562a61e14db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:39:19.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:19.196+0000 7fce82a00640 1 -- 172.21.15.5:0/165806320 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562a61debca0 msgr2=0x562a61e14db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:39:19.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:19.196+0000 7fce82a00640 1 --2- 172.21.15.5:0/165806320 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562a61debca0 0x562a61e14db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:19.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:19.196+0000 7fce82a00640 1 -- 172.21.15.5:0/165806320 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562a61de13e0 msgr2=0x562a61e14870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:39:19.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:19.196+0000 7fce82a00640 1 --2- 172.21.15.5:0/165806320 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562a61de13e0 0x562a61e14870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:19.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:19.196+0000 7fce82a00640 1 -- 172.21.15.5:0/165806320 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562a61de9860 con 0x562a61de0ac0 2024-09-10T09:39:19.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:19.196+0000 7fce82000640 1 --2- 172.21.15.5:0/165806320 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562a61de13e0 0x562a61e14870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:39:19.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:19.197+0000 7fce82a00640 1 --2- 172.21.15.5:0/165806320 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562a61de0ac0 0x562a61dea850 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fce740c4290 tx=0x7fce7402a410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4ba74bedfa3810ff server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:39:19.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:19.197+0000 7fce6ea00640 1 -- 172.21.15.5:0/165806320 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce740d3070 con 0x562a61de0ac0 2024-09-10T09:39:19.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:19.197+0000 7fce6ea00640 1 -- 172.21.15.5:0/165806320 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fce740bbb50 con 0x562a61de0ac0 2024-09-10T09:39:19.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:19.197+0000 7fce6ea00640 1 -- 172.21.15.5:0/165806320 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce74049040 con 0x562a61de0ac0 2024-09-10T09:39:19.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:19.198+0000 7fce8c20e740 1 -- 172.21.15.5:0/165806320 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562a61de0ac0 msgr2=0x562a61dea850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:39:19.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:19.198+0000 7fce8c20e740 1 --2- 172.21.15.5:0/165806320 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562a61de0ac0 0x562a61dea850 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fce740c4290 tx=0x7fce7402a410 comp rx=0 tx=0).stop 2024-09-10T09:39:19.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:19.198+0000 7fce8c20e740 1 -- 172.21.15.5:0/165806320 shutdown_connections 2024-09-10T09:39:19.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:19.198+0000 7fce8c20e740 1 --2- 172.21.15.5:0/165806320 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562a61debca0 0x562a61e14db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:19.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:19.198+0000 7fce8c20e740 1 --2- 172.21.15.5:0/165806320 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562a61de13e0 0x562a61e14870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:19.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:19.198+0000 7fce8c20e740 1 --2- 172.21.15.5:0/165806320 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562a61de0ac0 0x562a61dea850 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:19.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:19.198+0000 7fce8c20e740 1 -- 172.21.15.5:0/165806320 >> 172.21.15.5:0/165806320 conn(0x562a61ddd050 msgr2=0x562a61dea140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:39:19.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:19.198+0000 7fce8c20e740 1 -- 172.21.15.5:0/165806320 shutdown_connections 2024-09-10T09:39:19.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:19.199+0000 7fce8c20e740 1 -- 172.21.15.5:0/165806320 wait complete. 2024-09-10T09:39:19.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:19.199+0000 7fce8c20e740 1 -- start start 2024-09-10T09:39:19.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:19.200+0000 7fce8c20e740 1 Processor -- start 2024-09-10T09:39:19.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:19.200+0000 7fce8c20e740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562a61de0ac0 0x562a61df4420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:39:19.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:19.200+0000 7fce8c20e740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562a61de13e0 0x562a61de55f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:39:19.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:19.200+0000 7fce8c20e740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562a61debca0 0x562a61de5b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:39:19.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:19.200+0000 7fce8c20e740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fce841ae1e0 con 0x562a61de13e0 2024-09-10T09:39:19.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:19.200+0000 7fce8c20e740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fce841adb10 con 0x562a61de0ac0 2024-09-10T09:39:19.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:19.200+0000 7fce8c20e740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fce8419f250 con 0x562a61debca0 2024-09-10T09:39:19.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:19.200+0000 7fce8c20e740 10 client.0 _ll_register_callbacks cb 0x7fce7c009820 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:39:19.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:19.201+0000 7fce82000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562a61de13e0 0x562a61de55f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:39:19.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:19.201+0000 7fce82a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562a61de0ac0 0x562a61df4420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:39:19.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:19.201+0000 7fce82a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562a61de0ac0 0x562a61df4420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:38814/0 (socket says 172.21.15.5:38814) 2024-09-10T09:39:19.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:19.201+0000 7fce82a00640 1 -- 172.21.15.5:0/621738530 learned_addr learned my addr 172.21.15.5:0/621738530 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:39:19.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:19.201+0000 7fce83400640 1 --2- 172.21.15.5:0/621738530 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562a61debca0 0x562a61de5b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:39:19.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:19.201+0000 7fce82a00640 1 -- 172.21.15.5:0/621738530 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562a61debca0 msgr2=0x562a61de5b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:39:19.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:19.201+0000 7fce82a00640 1 --2- 172.21.15.5:0/621738530 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562a61debca0 0x562a61de5b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:19.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:19.201+0000 7fce82a00640 1 -- 172.21.15.5:0/621738530 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562a61de13e0 msgr2=0x562a61de55f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:39:19.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:19.201+0000 7fce82a00640 1 --2- 172.21.15.5:0/621738530 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562a61de13e0 0x562a61de55f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:19.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:19.201+0000 7fce82a00640 1 -- 172.21.15.5:0/621738530 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562a61de6330 con 0x562a61de0ac0 2024-09-10T09:39:19.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:19.202+0000 7fce82a00640 1 --2- 172.21.15.5:0/621738530 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562a61de0ac0 0x562a61df4420 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fce740b8520 tx=0x7fce74005e50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=32b48e5a0d09eb53 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:39:19.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:19.202+0000 7fce63e00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-09-10T09:39:19.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:19.202+0000 7fce63e00640 1 -- 172.21.15.5:0/621738530 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce740c9060 con 0x562a61de0ac0 2024-09-10T09:39:19.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:19.202+0000 7fce63e00640 1 -- 172.21.15.5:0/621738530 <== mon.2 v2:172.21.15.102:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fce7403b460 con 0x562a61de0ac0 2024-09-10T09:39:19.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:19.202+0000 7fce8c20e740 1 -- 172.21.15.5:0/621738530 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fce740bbb50 con 0x562a61de0ac0 2024-09-10T09:39:19.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:19.203+0000 7fce63e00640 1 -- 172.21.15.5:0/621738530 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fce74049070 con 0x562a61de0ac0 2024-09-10T09:39:19.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:19.203+0000 7fce8c20e740 10 client.25589 fetch_fsmap learned FSMap version 30 2024-09-10T09:39:19.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:19.203+0000 7fce8c20e740 1 -- 172.21.15.5:0/621738530 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x562a61e77820 con 0x562a61de0ac0 2024-09-10T09:39:19.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:19.203+0000 7fce63e00640 1 -- 172.21.15.5:0/621738530 <== mon.2 v2:172.21.15.102:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fce740bbb50 con 0x562a61de0ac0 2024-09-10T09:39:19.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:19.204+0000 7fce63e00640 1 -- 172.21.15.5:0/621738530 <== mon.2 v2:172.21.15.102:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fce7404c060 con 0x562a61de0ac0 2024-09-10T09:39:19.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:19.204+0000 7fce8c20e740 10 client.25589 fetch_fsmap finished waiting for FSMap version 30 2024-09-10T09:39:19.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:19.204+0000 7fce8c20e740 10 client.25589 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T09:39:19.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:19.204+0000 7fce8c20e740 20 client.25589 populate_metadata read hostname 'smithi005' 2024-09-10T09:39:19.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:19.204+0000 7fce8c20e740 1 --2- 172.21.15.5:0/621738530 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x562a61e77de0 0x562a61e7a200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:39:19.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:19.204+0000 7fce8c20e740 4 client.25589 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-10T09:39:19.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:19.204+0000 7fce8c20e740 1 -- 172.21.15.5:0/621738530 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x562a61d4ed10 con 0x562a61e77de0 2024-09-10T09:39:19.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:19.205+0000 7fce82000640 1 --2- 172.21.15.5:0/621738530 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x562a61e77de0 0x562a61e7a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:39:19.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:19.205+0000 7fce82000640 1 --2- 172.21.15.5:0/621738530 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x562a61e77de0 0x562a61e7a200 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=41e413ffc22ad08a server_cookie=a57209ea2e0c5289 in_seq=0 out_seq=0 2024-09-10T09:39:19.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:19.205+0000 7fce63e00640 10 client.25589 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T09:39:19.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:19.206+0000 7fce63e00640 1 -- 172.21.15.5:0/621738530 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x562a61d4ed10 con 0x562a61e77de0 2024-09-10T09:39:19.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:19.206+0000 7fce63e00640 10 client.25589 handle_command_reply: tid=0 2024-09-10T09:39:19.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:19.206+0000 7fce8c20e740 1 client.25589 shutdown 2024-09-10T09:39:19.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:19.207+0000 7fce8c20e740 1 -- 172.21.15.5:0/621738530 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562a61de0ac0 msgr2=0x562a61df4420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:39:19.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:19.207+0000 7fce8c20e740 1 --2- 172.21.15.5:0/621738530 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562a61de0ac0 0x562a61df4420 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fce740b8520 tx=0x7fce74005e50 comp rx=0 tx=0).stop 2024-09-10T09:39:19.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:19.207+0000 7fce8c20e740 1 -- 172.21.15.5:0/621738530 shutdown_connections 2024-09-10T09:39:19.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:19.207+0000 7fce8c20e740 1 --2- 172.21.15.5:0/621738530 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x562a61e77de0 0x562a61e7a200 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:19.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:19.207+0000 7fce8c20e740 1 --2- 172.21.15.5:0/621738530 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562a61debca0 0x562a61de5b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:19.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:19.207+0000 7fce8c20e740 1 --2- 172.21.15.5:0/621738530 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562a61de13e0 0x562a61de55f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:19.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:19.207+0000 7fce63e00640 0 client.25589 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T09:39:19.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:19.207+0000 7fce8c20e740 1 --2- 172.21.15.5:0/621738530 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562a61de0ac0 0x562a61df4420 secure :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fce740b8520 tx=0x7fce74005e50 comp rx=0 tx=0).stop 2024-09-10T09:39:19.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:19.207+0000 7fce8c20e740 1 -- 172.21.15.5:0/621738530 >> 172.21.15.5:0/621738530 conn(0x562a61ddd050 msgr2=0x562a61ddc1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:39:19.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:19.208+0000 7fce8c20e740 1 -- 172.21.15.5:0/621738530 shutdown_connections 2024-09-10T09:39:19.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:19.208+0000 7fce8c20e740 1 -- 172.21.15.5:0/621738530 wait complete. 2024-09-10T09:39:19.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:19.208+0000 7fce8c20e740 20 client.25589 trim_cache size 0 max 16384 2024-09-10T09:39:19.215 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-10T09:39:19.215 INFO:teuthology.orchestra.run.smithi005.stdout: "return_code": 0, 2024-09-10T09:39:19.215 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_tag": "e48c52b4-9086-4673-827f-22fe4a4ac7cb", 2024-09-10T09:39:19.215 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": "asynchronous" 2024-09-10T09:39:19.215 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-10T09:39:19.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:19.209+0000 7fce89600640 1 -- 172.21.15.5:0/3929742432 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fce680a2c90 msgr2=0x7fce680a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:39:19.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:19.209+0000 7fce89600640 1 --2- 172.21.15.5:0/3929742432 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fce680a2c90 0x7fce680a5150 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fce7c005fd0 tx=0x7fce7c009500 comp rx=0 tx=0).stop 2024-09-10T09:39:19.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:19.209+0000 7fce89600640 1 -- 172.21.15.5:0/3929742432 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fce84101dd0 msgr2=0x7fce841a4780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:39:19.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:19.209+0000 7fce89600640 1 --2- 172.21.15.5:0/3929742432 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fce84101dd0 0x7fce841a4780 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fce74004770 tx=0x7fce74003c10 comp rx=0 tx=0).stop 2024-09-10T09:39:19.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:19.209+0000 7fce89600640 1 -- 172.21.15.5:0/3929742432 shutdown_connections 2024-09-10T09:39:19.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:19.209+0000 7fce89600640 1 --2- 172.21.15.5:0/3929742432 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fce680a2c90 0x7fce680a5150 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:19.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:19.209+0000 7fce89600640 1 --2- 172.21.15.5:0/3929742432 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fce84106f00 0x7fce8410dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:19.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:19.209+0000 7fce89600640 1 --2- 172.21.15.5:0/3929742432 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fce84102780 0x7fce841a4cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:19.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:19.209+0000 7fce89600640 1 --2- 172.21.15.5:0/3929742432 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fce84101dd0 0x7fce841a4780 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:19.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:19.209+0000 7fce89600640 1 -- 172.21.15.5:0/3929742432 >> 172.21.15.5:0/3929742432 conn(0x7fce840fd8b0 msgr2=0x7fce840feb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:39:19.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:19.209+0000 7fce89600640 1 -- 172.21.15.5:0/3929742432 shutdown_connections 2024-09-10T09:39:19.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:19.209+0000 7fce89600640 1 -- 172.21.15.5:0/3929742432 wait complete. 2024-09-10T09:39:19.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:39:19 smithi005 ceph-mon[30350]: pgmap v422: 129 pgs: 129 active+clean; 665 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 597 B/s wr, 8 op/s 2024-09-10T09:39:19.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:39:19 smithi159 ceph-mon[35513]: pgmap v422: 129 pgs: 129 active+clean; 665 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 597 B/s wr, 8 op/s 2024-09-10T09:39:19.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:39:19 smithi102 ceph-mon[35442]: pgmap v422: 129 pgs: 129 active+clean; 665 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 597 B/s wr, 8 op/s 2024-09-10T09:39:20.067 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T09:39:20.520 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:39:20.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:39:20 smithi005 ceph-mon[30350]: scrub queued for path: / 2024-09-10T09:39:20.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:39:20 smithi005 ceph-mon[30350]: scrub summary: idle+waiting paths [/] 2024-09-10T09:39:20.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:39:20 smithi005 ceph-mon[30350]: scrub summary: active paths [/] 2024-09-10T09:39:20.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:39:20 smithi005 ceph-mon[30350]: scrub summary: idle+waiting paths [/] 2024-09-10T09:39:20.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:39:20 smithi159 ceph-mon[35513]: scrub queued for path: / 2024-09-10T09:39:20.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:39:20 smithi159 ceph-mon[35513]: scrub summary: idle+waiting paths [/] 2024-09-10T09:39:20.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:39:20 smithi159 ceph-mon[35513]: scrub summary: active paths [/] 2024-09-10T09:39:20.851 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:39:20 smithi159 ceph-mon[35513]: scrub summary: idle+waiting paths [/] 2024-09-10T09:39:20.885 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:39:20 smithi102 ceph-mon[35442]: scrub queued for path: / 2024-09-10T09:39:20.885 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:39:20 smithi102 ceph-mon[35442]: scrub summary: idle+waiting paths [/] 2024-09-10T09:39:20.885 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:39:20 smithi102 ceph-mon[35442]: scrub summary: active paths [/] 2024-09-10T09:39:20.885 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:39:20 smithi102 ceph-mon[35442]: scrub summary: idle+waiting paths [/] 2024-09-10T09:39:21.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:21.196+0000 7f5db1800640 1 -- 172.21.15.5:0/734822714 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5dac1047e0 msgr2=0x7f5dac108830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:39:21.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:21.196+0000 7f5db1800640 1 --2- 172.21.15.5:0/734822714 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5dac1047e0 0x7f5dac108830 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f5d98009900 tx=0x7f5d9802a3f0 comp rx=0 tx=0).stop 2024-09-10T09:39:21.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:21.196+0000 7f5db1800640 1 -- 172.21.15.5:0/734822714 shutdown_connections 2024-09-10T09:39:21.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:21.196+0000 7f5db1800640 1 --2- 172.21.15.5:0/734822714 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5dac108f60 0x7f5dac110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:21.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:21.196+0000 7f5db1800640 1 --2- 172.21.15.5:0/734822714 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5dac1047e0 0x7f5dac108830 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:21.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:21.196+0000 7f5db1800640 1 --2- 172.21.15.5:0/734822714 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5dac103e30 0x7f5dac104210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:21.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:21.196+0000 7f5db1800640 1 -- 172.21.15.5:0/734822714 >> 172.21.15.5:0/734822714 conn(0x7f5dac0ff910 msgr2=0x7f5dac101d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:39:21.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:21.197+0000 7f5db1800640 1 -- 172.21.15.5:0/734822714 shutdown_connections 2024-09-10T09:39:21.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:21.198+0000 7f5db1800640 1 -- 172.21.15.5:0/734822714 wait complete. 2024-09-10T09:39:21.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:21.199+0000 7f5db1800640 1 Processor -- start 2024-09-10T09:39:21.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:21.200+0000 7f5db1800640 1 -- start start 2024-09-10T09:39:21.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:21.200+0000 7f5db1800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5dac103e30 0x7f5dac076700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:39:21.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:21.200+0000 7f5db1800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5dac1047e0 0x7f5dac076c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:39:21.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:21.200+0000 7f5db1800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5dac108f60 0x7f5dac077180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:39:21.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:21.200+0000 7f5db1800640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f5dac114010 con 0x7f5dac108f60 2024-09-10T09:39:21.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:21.200+0000 7f5db1800640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f5dac113e90 con 0x7f5dac1047e0 2024-09-10T09:39:21.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:21.200+0000 7f5db1800640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f5dac114190 con 0x7f5dac103e30 2024-09-10T09:39:21.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:21.201+0000 7f5dabe00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5dac103e30 0x7f5dac076700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:39:21.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:21.201+0000 7f5dabe00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5dac103e30 0x7f5dac076700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:55686/0 (socket says 172.21.15.5:55686) 2024-09-10T09:39:21.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:21.201+0000 7f5dabe00640 1 -- 172.21.15.5:0/1696881768 learned_addr learned my addr 172.21.15.5:0/1696881768 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:39:21.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:21.201+0000 7f5dab400640 1 --2- 172.21.15.5:0/1696881768 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5dac1047e0 0x7f5dac076c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:39:21.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:21.201+0000 7f5db0e00640 1 --2- 172.21.15.5:0/1696881768 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5dac108f60 0x7f5dac077180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:39:21.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:21.202+0000 7f5dabe00640 1 -- 172.21.15.5:0/1696881768 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5dac1047e0 msgr2=0x7f5dac076c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:39:21.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:21.202+0000 7f5dabe00640 1 --2- 172.21.15.5:0/1696881768 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5dac1047e0 0x7f5dac076c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:21.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:21.202+0000 7f5dabe00640 1 -- 172.21.15.5:0/1696881768 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5dac108f60 msgr2=0x7f5dac077180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:39:21.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:21.202+0000 7f5dabe00640 1 --2- 172.21.15.5:0/1696881768 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5dac108f60 0x7f5dac077180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:21.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:21.202+0000 7f5dabe00640 1 -- 172.21.15.5:0/1696881768 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5dac19a250 con 0x7f5dac103e30 2024-09-10T09:39:21.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:21.202+0000 7f5dabe00640 1 --2- 172.21.15.5:0/1696881768 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5dac103e30 0x7f5dac076700 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f5d9c00d980 tx=0x7f5d9c00de40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:39:21.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:21.203+0000 7f5da8c00640 1 -- 172.21.15.5:0/1696881768 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d9c017030 con 0x7f5dac103e30 2024-09-10T09:39:21.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:21.203+0000 7f5da8c00640 1 -- 172.21.15.5:0/1696881768 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5d9c004020 con 0x7f5dac103e30 2024-09-10T09:39:21.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:21.203+0000 7f5da8c00640 1 -- 172.21.15.5:0/1696881768 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d9c0047a0 con 0x7f5dac103e30 2024-09-10T09:39:21.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:21.203+0000 7f5db1800640 1 -- 172.21.15.5:0/1696881768 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5dac19a540 con 0x7f5dac103e30 2024-09-10T09:39:21.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:21.203+0000 7f5db1800640 1 -- 172.21.15.5:0/1696881768 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5dac19aa00 con 0x7f5dac103e30 2024-09-10T09:39:21.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:21.208+0000 7f5db59bd740 1 Processor -- start 2024-09-10T09:39:21.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:21.209+0000 7f5db59bd740 1 -- start start 2024-09-10T09:39:21.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:21.209+0000 7f5da8c00640 1 -- 172.21.15.5:0/1696881768 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 19) ==== 137883+0+0 (secure 0 0 0) 0x7f5d9c009a30 con 0x7f5dac103e30 2024-09-10T09:39:21.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:21.209+0000 7f5db59bd740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557d5fac6d50 0x557d5fac7130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:39:21.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:21.209+0000 7f5db59bd740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557d5fac7670 0x557d5fad19f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:39:21.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:21.209+0000 7f5db59bd740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557d5fad1f30 0x557d5fad4320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:39:21.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:21.209+0000 7f5db59bd740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x557d5fa2d3e0 con 0x557d5fac6d50 2024-09-10T09:39:21.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:21.209+0000 7f5db59bd740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x557d5f9d4d80 con 0x557d5fac7670 2024-09-10T09:39:21.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:21.209+0000 7f5db59bd740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x557d5fa293a0 con 0x557d5fad1f30 2024-09-10T09:39:21.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:21.210+0000 7f5dabe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557d5fac7670 0x557d5fad19f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:39:21.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:21.210+0000 7f5dabe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557d5fac7670 0x557d5fad19f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:50106/0 (socket says 172.21.15.5:50106) 2024-09-10T09:39:21.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:21.210+0000 7f5dabe00640 1 -- 172.21.15.5:0/2881108504 learned_addr learned my addr 172.21.15.5:0/2881108504 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:39:21.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:21.210+0000 7f5db0e00640 1 --2- 172.21.15.5:0/2881108504 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557d5fac6d50 0x557d5fac7130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:39:21.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:21.211+0000 7f5dab400640 1 --2- 172.21.15.5:0/2881108504 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557d5fad1f30 0x557d5fad4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:39:21.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:21.211+0000 7f5db0e00640 1 -- 172.21.15.5:0/2881108504 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557d5fad1f30 msgr2=0x557d5fad4320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:39:21.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:21.211+0000 7f5db0e00640 1 --2- 172.21.15.5:0/2881108504 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557d5fad1f30 0x557d5fad4320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:21.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:21.211+0000 7f5db0e00640 1 -- 172.21.15.5:0/2881108504 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557d5fac7670 msgr2=0x557d5fad19f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:39:21.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:21.211+0000 7f5db0e00640 1 --2- 172.21.15.5:0/2881108504 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557d5fac7670 0x557d5fad19f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:21.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:21.211+0000 7f5db0e00640 1 -- 172.21.15.5:0/2881108504 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557d5f9f3ae0 con 0x557d5fac6d50 2024-09-10T09:39:21.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:21.212+0000 7f5db0e00640 1 --2- 172.21.15.5:0/2881108504 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557d5fac6d50 0x557d5fac7130 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f5da4009f00 tx=0x7f5da4011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8479038fb7cfc968 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:39:21.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:21.212+0000 7f5da8c00640 1 --2- 172.21.15.5:0/1696881768 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f5d900a2d10 0x7f5d900a51d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:39:21.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:21.212+0000 7f5da8c00640 1 -- 172.21.15.5:0/1696881768 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f5d9c0cefb0 con 0x7f5dac103e30 2024-09-10T09:39:21.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:21.212+0000 7f5da1600640 1 -- 172.21.15.5:0/2881108504 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5da4004d20 con 0x557d5fac6d50 2024-09-10T09:39:21.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:21.213+0000 7f5da1600640 1 -- 172.21.15.5:0/2881108504 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5da4004050 con 0x557d5fac6d50 2024-09-10T09:39:21.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:21.213+0000 7f5da1600640 1 -- 172.21.15.5:0/2881108504 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5da4005260 con 0x557d5fac6d50 2024-09-10T09:39:21.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:21.213+0000 7f5dabe00640 1 --2- 172.21.15.5:0/1696881768 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f5d900a2d10 0x7f5d900a51d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:39:21.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:21.214+0000 7f5db59bd740 1 -- 172.21.15.5:0/2881108504 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557d5fac6d50 msgr2=0x557d5fac7130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:39:21.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:21.214+0000 7f5db59bd740 1 --2- 172.21.15.5:0/2881108504 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557d5fac6d50 0x557d5fac7130 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f5da4009f00 tx=0x7f5da4011810 comp rx=0 tx=0).stop 2024-09-10T09:39:21.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:21.214+0000 7f5db59bd740 1 -- 172.21.15.5:0/2881108504 shutdown_connections 2024-09-10T09:39:21.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:21.214+0000 7f5db59bd740 1 --2- 172.21.15.5:0/2881108504 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557d5fad1f30 0x557d5fad4320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:21.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:21.214+0000 7f5db59bd740 1 --2- 172.21.15.5:0/2881108504 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557d5fac7670 0x557d5fad19f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:21.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:21.214+0000 7f5db59bd740 1 --2- 172.21.15.5:0/2881108504 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557d5fac6d50 0x557d5fac7130 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:21.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:21.214+0000 7f5db59bd740 1 -- 172.21.15.5:0/2881108504 >> 172.21.15.5:0/2881108504 conn(0x557d5fac32e0 msgr2=0x557d5fac36c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:39:21.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:21.215+0000 7f5db59bd740 1 -- 172.21.15.5:0/2881108504 shutdown_connections 2024-09-10T09:39:21.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:21.215+0000 7f5dabe00640 1 --2- 172.21.15.5:0/1696881768 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f5d900a2d10 0x7f5d900a51d0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f5d9c0976c0 tx=0x7f5d9c0cf220 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:39:21.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:21.215+0000 7f5db59bd740 1 -- 172.21.15.5:0/2881108504 wait complete. 2024-09-10T09:39:21.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:21.216+0000 7f5db59bd740 1 -- start start 2024-09-10T09:39:21.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:21.216+0000 7f5db59bd740 1 Processor -- start 2024-09-10T09:39:21.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:21.217+0000 7f5db59bd740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557d5fac6d50 0x557d5facee30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:39:21.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:21.217+0000 7f5db59bd740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557d5fac7670 0x557d5facf370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:39:21.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:21.217+0000 7f5db59bd740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557d5fad1f30 0x557d5facf8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:39:21.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:21.217+0000 7f5db59bd740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x557d5fad4bf0 con 0x557d5fac7670 2024-09-10T09:39:21.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:21.217+0000 7f5db59bd740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x557d5f9e80f0 con 0x557d5fad1f30 2024-09-10T09:39:21.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:21.217+0000 7f5db59bd740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x557d5fad4d70 con 0x557d5fac6d50 2024-09-10T09:39:21.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:21.217+0000 7f5db59bd740 10 client.0 _ll_register_callbacks cb 0x557d5f9d99e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:39:21.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:21.217+0000 7f5db0e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557d5fac6d50 0x557d5facee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:39:21.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:21.217+0000 7f5db0e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557d5fac6d50 0x557d5facee30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:55698/0 (socket says 172.21.15.5:55698) 2024-09-10T09:39:21.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:21.217+0000 7f5db0e00640 1 -- 172.21.15.5:0/2359332475 learned_addr learned my addr 172.21.15.5:0/2359332475 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:39:21.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:21.217+0000 7f5db0e00640 1 --2- 172.21.15.5:0/2359332475 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557d5fad1f30 0x557d5facf8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:39:21.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:21.218+0000 7f5dab400640 1 --2- 172.21.15.5:0/2359332475 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557d5fac7670 0x557d5facf370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:39:21.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:21.218+0000 7f5db0e00640 1 -- 172.21.15.5:0/2359332475 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557d5fac6d50 msgr2=0x557d5facee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:39:21.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:21.218+0000 7f5db0e00640 1 --2- 172.21.15.5:0/2359332475 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557d5fac6d50 0x557d5facee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:21.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:21.218+0000 7f5db0e00640 1 -- 172.21.15.5:0/2359332475 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557d5fac7670 msgr2=0x557d5facf370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:39:21.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:21.218+0000 7f5db0e00640 1 --2- 172.21.15.5:0/2359332475 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557d5fac7670 0x557d5facf370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:21.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:21.218+0000 7f5db0e00640 1 -- 172.21.15.5:0/2359332475 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557d5f79e3c0 con 0x557d5fad1f30 2024-09-10T09:39:21.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:21.218+0000 7f5db0e00640 1 --2- 172.21.15.5:0/2359332475 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557d5fad1f30 0x557d5facf8b0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f5da4009ed0 tx=0x7f5da4004990 comp rx=0 tx=0).ready entity=mon.2 client_cookie=83d9e9829f7d9a03 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:39:21.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:21.219+0000 7f5d96000640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-09-10T09:39:21.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:21.219+0000 7f5d96000640 1 -- 172.21.15.5:0/2359332475 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5da4004c40 con 0x557d5fad1f30 2024-09-10T09:39:21.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:21.219+0000 7f5db59bd740 1 -- 172.21.15.5:0/2359332475 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5da4004050 con 0x557d5fad1f30 2024-09-10T09:39:21.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:21.219+0000 7f5d96000640 1 -- 172.21.15.5:0/2359332475 <== mon.2 v2:172.21.15.102:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f5da4004de0 con 0x557d5fad1f30 2024-09-10T09:39:21.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:21.219+0000 7f5d96000640 1 -- 172.21.15.5:0/2359332475 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f5da4004fe0 con 0x557d5fad1f30 2024-09-10T09:39:21.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:21.219+0000 7f5db59bd740 10 client.25601 fetch_fsmap learned FSMap version 30 2024-09-10T09:39:21.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:21.219+0000 7f5db59bd740 1 -- 172.21.15.5:0/2359332475 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x557d5fae08b0 con 0x557d5fad1f30 2024-09-10T09:39:21.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:21.220+0000 7f5d96000640 1 -- 172.21.15.5:0/2359332475 <== mon.2 v2:172.21.15.102:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5da4004050 con 0x557d5fad1f30 2024-09-10T09:39:21.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:21.220+0000 7f5d96000640 1 -- 172.21.15.5:0/2359332475 <== mon.2 v2:172.21.15.102:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f5da4019430 con 0x557d5fad1f30 2024-09-10T09:39:21.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:21.220+0000 7f5db59bd740 10 client.25601 fetch_fsmap finished waiting for FSMap version 30 2024-09-10T09:39:21.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:21.220+0000 7f5db59bd740 10 client.25601 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T09:39:21.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:21.220+0000 7f5db59bd740 20 client.25601 populate_metadata read hostname 'smithi005' 2024-09-10T09:39:21.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:21.220+0000 7f5db59bd740 1 --2- 172.21.15.5:0/2359332475 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x557d5fae1c00 0x557d5fae3ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:39:21.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:21.220+0000 7f5db59bd740 4 client.25601 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:39:21.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:21.220+0000 7f5db59bd740 1 -- 172.21.15.5:0/2359332475 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557d5fa3a950 con 0x557d5fae1c00 2024-09-10T09:39:21.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:21.221+0000 7f5dab400640 1 --2- 172.21.15.5:0/2359332475 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x557d5fae1c00 0x557d5fae3ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:39:21.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:21.222+0000 7f5dab400640 1 --2- 172.21.15.5:0/2359332475 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x557d5fae1c00 0x557d5fae3ff0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=956373cc93038364 server_cookie=727b42a22abbe680 in_seq=0 out_seq=0 2024-09-10T09:39:21.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:21.222+0000 7f5d96000640 10 client.25601 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T09:39:21.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:21.223+0000 7f5d96000640 1 -- 172.21.15.5:0/2359332475 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557d5fa3a950 con 0x557d5fae1c00 2024-09-10T09:39:21.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:21.223+0000 7f5d96000640 10 client.25601 handle_command_reply: tid=0 2024-09-10T09:39:21.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:21.223+0000 7f5db59bd740 1 client.25601 shutdown 2024-09-10T09:39:21.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:21.224+0000 7f5db59bd740 1 -- 172.21.15.5:0/2359332475 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557d5fad1f30 msgr2=0x557d5facf8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:39:21.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:21.224+0000 7f5db59bd740 1 --2- 172.21.15.5:0/2359332475 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557d5fad1f30 0x557d5facf8b0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f5da4009ed0 tx=0x7f5da4004990 comp rx=0 tx=0).stop 2024-09-10T09:39:21.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:21.224+0000 7f5db59bd740 1 -- 172.21.15.5:0/2359332475 shutdown_connections 2024-09-10T09:39:21.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:21.224+0000 7f5db59bd740 1 --2- 172.21.15.5:0/2359332475 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x557d5fae1c00 0x557d5fae3ff0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:21.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:21.224+0000 7f5db59bd740 1 --2- 172.21.15.5:0/2359332475 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557d5fad1f30 0x557d5facf8b0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:21.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:21.224+0000 7f5db59bd740 1 --2- 172.21.15.5:0/2359332475 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557d5fac7670 0x557d5facf370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:21.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:21.224+0000 7f5db59bd740 1 --2- 172.21.15.5:0/2359332475 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557d5fac6d50 0x557d5facee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:21.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:21.224+0000 7f5db59bd740 1 -- 172.21.15.5:0/2359332475 >> 172.21.15.5:0/2359332475 conn(0x557d5fac32e0 msgr2=0x557d5fac9010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:39:21.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:21.224+0000 7f5d96000640 0 client.25601 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T09:39:21.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:21.224+0000 7f5db59bd740 1 -- 172.21.15.5:0/2359332475 shutdown_connections 2024-09-10T09:39:21.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:21.224+0000 7f5db59bd740 1 -- 172.21.15.5:0/2359332475 wait complete. 2024-09-10T09:39:21.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:21.225+0000 7f5db59bd740 20 client.25601 trim_cache size 0 max 16384 2024-09-10T09:39:21.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:21.261+0000 7f5db59bd740 1 Processor -- start 2024-09-10T09:39:21.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:21.261+0000 7f5db59bd740 1 -- start start 2024-09-10T09:39:21.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:21.261+0000 7f5db59bd740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557d5fac6d50 0x557d5fad1160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:39:21.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:21.261+0000 7f5db59bd740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557d5fac7670 0x557d5fac97b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:39:21.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:21.261+0000 7f5db59bd740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557d5fad1f30 0x557d5facf700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:39:21.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:21.261+0000 7f5db59bd740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f5dac1bdbb0 con 0x557d5fad1f30 2024-09-10T09:39:21.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:21.261+0000 7f5db59bd740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f5dac1bf6f0 con 0x557d5fac6d50 2024-09-10T09:39:21.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:21.261+0000 7f5db59bd740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f5dac1be280 con 0x557d5fac7670 2024-09-10T09:39:21.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:21.262+0000 7f5dab400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557d5fac7670 0x557d5fac97b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:39:21.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:21.262+0000 7f5db0e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557d5fad1f30 0x557d5facf700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:39:21.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:21.262+0000 7f5dab400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557d5fac7670 0x557d5fac97b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:55708/0 (socket says 172.21.15.5:55708) 2024-09-10T09:39:21.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:21.262+0000 7f5dab400640 1 -- 172.21.15.5:0/867472257 learned_addr learned my addr 172.21.15.5:0/867472257 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:39:21.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:21.262+0000 7f5db0e00640 1 --2- 172.21.15.5:0/867472257 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557d5fac6d50 0x557d5fad1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:39:21.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:21.263+0000 7f5dab400640 1 -- 172.21.15.5:0/867472257 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557d5fac6d50 msgr2=0x557d5fad1160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:39:21.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:21.263+0000 7f5dab400640 1 --2- 172.21.15.5:0/867472257 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557d5fac6d50 0x557d5fad1160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:21.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:21.263+0000 7f5dab400640 1 -- 172.21.15.5:0/867472257 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557d5fad1f30 msgr2=0x557d5facf700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:39:21.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:21.263+0000 7f5dab400640 1 --2- 172.21.15.5:0/867472257 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557d5fad1f30 0x557d5facf700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:21.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:21.263+0000 7f5dab400640 1 -- 172.21.15.5:0/867472257 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557d5fae44e0 con 0x557d5fac7670 2024-09-10T09:39:21.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:21.263+0000 7f5db0e00640 1 --2- 172.21.15.5:0/867472257 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557d5fad1f30 0x557d5facf700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:39:21.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:21.263+0000 7f5dab400640 1 --2- 172.21.15.5:0/867472257 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557d5fac7670 0x557d5fac97b0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f5d98009900 tx=0x7f5d98005b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fde75bce48df0bc5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:39:21.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:21.264+0000 7f5da0c00640 1 -- 172.21.15.5:0/867472257 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d98035040 con 0x557d5fac7670 2024-09-10T09:39:21.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:21.264+0000 7f5da0c00640 1 -- 172.21.15.5:0/867472257 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5d9804a070 con 0x557d5fac7670 2024-09-10T09:39:21.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:21.264+0000 7f5da0c00640 1 -- 172.21.15.5:0/867472257 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d980315b0 con 0x557d5fac7670 2024-09-10T09:39:21.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:21.265+0000 7f5db59bd740 1 -- 172.21.15.5:0/867472257 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557d5fac7670 msgr2=0x557d5fac97b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:39:21.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:21.265+0000 7f5db59bd740 1 --2- 172.21.15.5:0/867472257 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557d5fac7670 0x557d5fac97b0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f5d98009900 tx=0x7f5d98005b50 comp rx=0 tx=0).stop 2024-09-10T09:39:21.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:21.265+0000 7f5db59bd740 1 -- 172.21.15.5:0/867472257 shutdown_connections 2024-09-10T09:39:21.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:21.265+0000 7f5db59bd740 1 --2- 172.21.15.5:0/867472257 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557d5fad1f30 0x557d5facf700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:21.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:21.265+0000 7f5db59bd740 1 --2- 172.21.15.5:0/867472257 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557d5fac7670 0x557d5fac97b0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:21.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:21.265+0000 7f5db59bd740 1 --2- 172.21.15.5:0/867472257 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557d5fac6d50 0x557d5fad1160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:21.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:21.265+0000 7f5db59bd740 1 -- 172.21.15.5:0/867472257 >> 172.21.15.5:0/867472257 conn(0x557d5fac32e0 msgr2=0x557d5fac23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:39:21.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:21.266+0000 7f5db59bd740 1 -- 172.21.15.5:0/867472257 shutdown_connections 2024-09-10T09:39:21.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:21.266+0000 7f5db59bd740 1 -- 172.21.15.5:0/867472257 wait complete. 2024-09-10T09:39:21.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:21.266+0000 7f5db59bd740 1 -- start start 2024-09-10T09:39:21.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:21.267+0000 7f5db59bd740 1 Processor -- start 2024-09-10T09:39:21.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:21.267+0000 7f5db59bd740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557d5fac6d50 0x557d5fb44880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:39:21.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:21.267+0000 7f5db59bd740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557d5fac7670 0x557d5fb44dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:39:21.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:21.267+0000 7f5db59bd740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557d5fad1f30 0x557d5fad9c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:39:21.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:21.267+0000 7f5db59bd740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f5dac1bf020 con 0x557d5fad1f30 2024-09-10T09:39:21.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:21.267+0000 7f5db59bd740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f5dac1be950 con 0x557d5fac6d50 2024-09-10T09:39:21.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:21.267+0000 7f5db59bd740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f5dac1bbb50 con 0x557d5fac7670 2024-09-10T09:39:21.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:21.267+0000 7f5db59bd740 10 client.0 _ll_register_callbacks cb 0x7f5da40046b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:39:21.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:21.268+0000 7f5dab400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557d5fac7670 0x557d5fb44dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:39:21.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:21.268+0000 7f5dab400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557d5fac7670 0x557d5fb44dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:55720/0 (socket says 172.21.15.5:55720) 2024-09-10T09:39:21.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:21.268+0000 7f5dab400640 1 -- 172.21.15.5:0/1481086072 learned_addr learned my addr 172.21.15.5:0/1481086072 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:39:21.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:21.268+0000 7f5db0e00640 1 --2- 172.21.15.5:0/1481086072 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557d5fad1f30 0x557d5fad9c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:39:21.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:21.268+0000 7f5db0e00640 1 --2- 172.21.15.5:0/1481086072 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557d5fac6d50 0x557d5fb44880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:39:21.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:21.269+0000 7f5db0e00640 1 -- 172.21.15.5:0/1481086072 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557d5fac7670 msgr2=0x557d5fb44dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:39:21.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:21.269+0000 7f5db0e00640 1 --2- 172.21.15.5:0/1481086072 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557d5fac7670 0x557d5fb44dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:21.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:21.269+0000 7f5db0e00640 1 -- 172.21.15.5:0/1481086072 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557d5fac6d50 msgr2=0x557d5fb44880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:39:21.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:21.269+0000 7f5db0e00640 1 --2- 172.21.15.5:0/1481086072 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557d5fac6d50 0x557d5fb44880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:21.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:21.269+0000 7f5db0e00640 1 -- 172.21.15.5:0/1481086072 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557d5fada2f0 con 0x557d5fad1f30 2024-09-10T09:39:21.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:21.269+0000 7f5dab400640 1 --2- 172.21.15.5:0/1481086072 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557d5fac7670 0x557d5fb44dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:39:21.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:21.269+0000 7f5db0e00640 1 --2- 172.21.15.5:0/1481086072 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557d5fad1f30 0x557d5fad9c50 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f5da40199a0 tx=0x7f5da4009d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=111d4758edf97def server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:39:21.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:21.269+0000 7f5d96000640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-10T09:39:21.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:21.270+0000 7f5d96000640 1 -- 172.21.15.5:0/1481086072 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5da4019640 con 0x557d5fad1f30 2024-09-10T09:39:21.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:21.270+0000 7f5d96000640 1 -- 172.21.15.5:0/1481086072 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f5da4019a20 con 0x557d5fad1f30 2024-09-10T09:39:21.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:21.270+0000 7f5db59bd740 1 -- 172.21.15.5:0/1481086072 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5d9804a070 con 0x557d5fad1f30 2024-09-10T09:39:21.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:21.271+0000 7f5d96000640 1 -- 172.21.15.5:0/1481086072 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f5da4019c20 con 0x557d5fad1f30 2024-09-10T09:39:21.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:21.271+0000 7f5db59bd740 10 client.15597 fetch_fsmap learned FSMap version 30 2024-09-10T09:39:21.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:21.271+0000 7f5db59bd740 1 -- 172.21.15.5:0/1481086072 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x557d5fb4c0e0 con 0x557d5fad1f30 2024-09-10T09:39:21.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:21.271+0000 7f5d96000640 1 -- 172.21.15.5:0/1481086072 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5d9804a070 con 0x557d5fad1f30 2024-09-10T09:39:21.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:21.272+0000 7f5d96000640 1 -- 172.21.15.5:0/1481086072 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f5da4039970 con 0x557d5fad1f30 2024-09-10T09:39:21.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:21.272+0000 7f5db59bd740 10 client.15597 fetch_fsmap finished waiting for FSMap version 30 2024-09-10T09:39:21.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:21.272+0000 7f5db59bd740 10 client.15597 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T09:39:21.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:21.272+0000 7f5db59bd740 20 client.15597 populate_metadata read hostname 'smithi005' 2024-09-10T09:39:21.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:21.272+0000 7f5db59bd740 1 --2- 172.21.15.5:0/1481086072 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x557d5fb4c6f0 0x557d5fb4eb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:39:21.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:21.272+0000 7f5db59bd740 4 client.15597 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T09:39:21.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:21.272+0000 7f5db59bd740 1 -- 172.21.15.5:0/1481086072 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "scrub status"}) -- 0x557d5fa34e60 con 0x557d5fb4c6f0 2024-09-10T09:39:21.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:21.273+0000 7f5dab400640 1 --2- 172.21.15.5:0/1481086072 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x557d5fb4c6f0 0x557d5fb4eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:39:21.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:21.274+0000 7f5dab400640 1 --2- 172.21.15.5:0/1481086072 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x557d5fb4c6f0 0x557d5fb4eb10 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=96c955c39d183b39 server_cookie=efb5e1b95d18f424 in_seq=0 out_seq=0 2024-09-10T09:39:21.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:21.274+0000 7f5d96000640 10 client.15597 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T09:39:21.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:21.274+0000 7f5d96000640 1 -- 172.21.15.5:0/1481086072 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x557d5fa34e60 con 0x557d5fb4c6f0 2024-09-10T09:39:21.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:21.275+0000 7f5d96000640 10 client.15597 handle_command_reply: tid=0 2024-09-10T09:39:21.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:21.275+0000 7f5db59bd740 1 client.15597 shutdown 2024-09-10T09:39:21.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:21.275+0000 7f5db59bd740 1 -- 172.21.15.5:0/1481086072 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557d5fad1f30 msgr2=0x557d5fad9c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:39:21.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:21.275+0000 7f5db59bd740 1 --2- 172.21.15.5:0/1481086072 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557d5fad1f30 0x557d5fad9c50 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f5da40199a0 tx=0x7f5da4009d00 comp rx=0 tx=0).stop 2024-09-10T09:39:21.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:21.275+0000 7f5db59bd740 1 -- 172.21.15.5:0/1481086072 shutdown_connections 2024-09-10T09:39:21.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:21.275+0000 7f5db59bd740 1 --2- 172.21.15.5:0/1481086072 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x557d5fb4c6f0 0x557d5fb4eb10 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:21.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:21.275+0000 7f5db59bd740 1 --2- 172.21.15.5:0/1481086072 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557d5fad1f30 0x557d5fad9c50 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:21.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:21.275+0000 7f5db59bd740 1 --2- 172.21.15.5:0/1481086072 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557d5fac7670 0x557d5fb44dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:21.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:21.275+0000 7f5db59bd740 1 --2- 172.21.15.5:0/1481086072 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557d5fac6d50 0x557d5fb44880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:21.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:21.275+0000 7f5db59bd740 1 -- 172.21.15.5:0/1481086072 >> 172.21.15.5:0/1481086072 conn(0x557d5fac32e0 msgr2=0x557d5fadce40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:39:21.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:21.276+0000 7f5d96000640 0 client.15597 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T09:39:21.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:21.276+0000 7f5db59bd740 1 -- 172.21.15.5:0/1481086072 shutdown_connections 2024-09-10T09:39:21.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:21.277+0000 7f5db59bd740 1 -- 172.21.15.5:0/1481086072 wait complete. 2024-09-10T09:39:21.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:21.277+0000 7f5db59bd740 20 client.15597 trim_cache size 0 max 16384 2024-09-10T09:39:21.279 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-10T09:39:21.279 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-10T09:39:21.279 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2024-09-10T09:39:21.279 INFO:teuthology.orchestra.run.smithi005.stdout: "e48c52b4-9086-4673-827f-22fe4a4ac7cb": { 2024-09-10T09:39:21.279 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2024-09-10T09:39:21.279 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "e48c52b4-9086-4673-827f-22fe4a4ac7cb", 2024-09-10T09:39:21.279 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2024-09-10T09:39:21.280 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T09:39:21.280 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T09:39:21.280 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-10T09:39:21.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:21.279+0000 7f5db1800640 1 -- 172.21.15.5:0/1696881768 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f5d900a2d10 msgr2=0x7f5d900a51d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:39:21.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:21.279+0000 7f5db1800640 1 --2- 172.21.15.5:0/1696881768 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f5d900a2d10 0x7f5d900a51d0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f5d9c0976c0 tx=0x7f5d9c0cf220 comp rx=0 tx=0).stop 2024-09-10T09:39:21.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:21.279+0000 7f5db1800640 1 -- 172.21.15.5:0/1696881768 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5dac103e30 msgr2=0x7f5dac076700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:39:21.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:21.279+0000 7f5db1800640 1 --2- 172.21.15.5:0/1696881768 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5dac103e30 0x7f5dac076700 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f5d9c00d980 tx=0x7f5d9c00de40 comp rx=0 tx=0).stop 2024-09-10T09:39:21.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:21.279+0000 7f5db1800640 1 -- 172.21.15.5:0/1696881768 shutdown_connections 2024-09-10T09:39:21.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:21.279+0000 7f5db1800640 1 --2- 172.21.15.5:0/1696881768 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f5d900a2d10 0x7f5d900a51d0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:21.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:21.279+0000 7f5db1800640 1 --2- 172.21.15.5:0/1696881768 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5dac108f60 0x7f5dac077180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:21.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:21.279+0000 7f5db1800640 1 --2- 172.21.15.5:0/1696881768 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5dac1047e0 0x7f5dac076c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:21.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:21.279+0000 7f5db1800640 1 --2- 172.21.15.5:0/1696881768 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5dac103e30 0x7f5dac076700 secure :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f5d9c00d980 tx=0x7f5d9c00de40 comp rx=0 tx=0).stop 2024-09-10T09:39:21.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:21.279+0000 7f5db1800640 1 -- 172.21.15.5:0/1696881768 >> 172.21.15.5:0/1696881768 conn(0x7f5dac0ff910 msgr2=0x7f5dac100c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:39:21.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:21.283+0000 7f5db1800640 1 -- 172.21.15.5:0/1696881768 shutdown_connections 2024-09-10T09:39:21.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:21.284+0000 7f5db1800640 1 -- 172.21.15.5:0/1696881768 wait complete. 2024-09-10T09:39:21.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:39:21 smithi005 ceph-mon[30350]: pgmap v423: 129 pgs: 129 active+clean; 665 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 597 B/s wr, 9 op/s 2024-09-10T09:39:21.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:39:21 smithi159 ceph-mon[35513]: pgmap v423: 129 pgs: 129 active+clean; 665 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 597 B/s wr, 9 op/s 2024-09-10T09:39:21.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:39:21 smithi102 ceph-mon[35442]: pgmap v423: 129 pgs: 129 active+clean; 665 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 597 B/s wr, 9 op/s 2024-09-10T09:39:22.104 INFO:tasks.cephfs.filesystem:scrub status for tag:e48c52b4-9086-4673-827f-22fe4a4ac7cb - {'path': '/', 'tag': 'e48c52b4-9086-4673-827f-22fe4a4ac7cb', 'options': 'recursive,force'} 2024-09-10T09:39:22.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:39:22 smithi005 ceph-mon[30350]: scrub summary: idle 2024-09-10T09:39:22.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:39:22 smithi159 ceph-mon[35513]: scrub summary: idle 2024-09-10T09:39:22.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:39:22 smithi102 ceph-mon[35442]: scrub summary: idle 2024-09-10T09:39:23.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:39:23 smithi005 ceph-mon[30350]: pgmap v424: 129 pgs: 129 active+clean; 665 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 597 B/s wr, 12 op/s 2024-09-10T09:39:23.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:39:23 smithi159 ceph-mon[35513]: pgmap v424: 129 pgs: 129 active+clean; 665 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 597 B/s wr, 12 op/s 2024-09-10T09:39:23.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:39:23 smithi102 ceph-mon[35442]: pgmap v424: 129 pgs: 129 active+clean; 665 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 597 B/s wr, 12 op/s 2024-09-10T09:39:25.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:39:25 smithi005 ceph-mon[30350]: pgmap v425: 129 pgs: 129 active+clean; 665 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 597 B/s wr, 8 op/s 2024-09-10T09:39:25.853 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:39:25 smithi159 ceph-mon[35513]: pgmap v425: 129 pgs: 129 active+clean; 665 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 597 B/s wr, 8 op/s 2024-09-10T09:39:25.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:39:25 smithi102 ceph-mon[35442]: pgmap v425: 129 pgs: 129 active+clean; 665 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 597 B/s wr, 8 op/s 2024-09-10T09:39:27.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:39:27 smithi159 ceph-mon[35513]: pgmap v426: 129 pgs: 129 active+clean; 665 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 682 B/s wr, 12 op/s 2024-09-10T09:39:27.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:39:27 smithi102 ceph-mon[35442]: pgmap v426: 129 pgs: 129 active+clean; 665 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 682 B/s wr, 12 op/s 2024-09-10T09:39:28.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:39:27 smithi005 ceph-mon[30350]: pgmap v426: 129 pgs: 129 active+clean; 665 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 682 B/s wr, 12 op/s 2024-09-10T09:39:28.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:39:28 smithi102 ceph-mon[35442]: pgmap v427: 129 pgs: 129 active+clean; 665 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 85 B/s wr, 9 op/s 2024-09-10T09:39:29.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:39:28 smithi005 ceph-mon[30350]: pgmap v427: 129 pgs: 129 active+clean; 665 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 85 B/s wr, 9 op/s 2024-09-10T09:39:29.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:39:28 smithi159 ceph-mon[35513]: pgmap v427: 129 pgs: 129 active+clean; 665 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 85 B/s wr, 9 op/s 2024-09-10T09:39:30.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:39:30 smithi102 ceph-mon[35442]: pgmap v428: 129 pgs: 129 active+clean; 665 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 85 B/s wr, 9 op/s 2024-09-10T09:39:31.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:39:30 smithi005 ceph-mon[30350]: pgmap v428: 129 pgs: 129 active+clean; 665 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 85 B/s wr, 9 op/s 2024-09-10T09:39:31.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:39:30 smithi159 ceph-mon[35513]: pgmap v428: 129 pgs: 129 active+clean; 665 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 85 B/s wr, 9 op/s 2024-09-10T09:39:32.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:39:32 smithi102 ceph-mon[35442]: pgmap v429: 129 pgs: 129 active+clean; 665 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 85 B/s wr, 11 op/s 2024-09-10T09:39:33.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:39:32 smithi005 ceph-mon[30350]: pgmap v429: 129 pgs: 129 active+clean; 665 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 85 B/s wr, 11 op/s 2024-09-10T09:39:33.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:39:32 smithi159 ceph-mon[35513]: pgmap v429: 129 pgs: 129 active+clean; 665 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 85 B/s wr, 11 op/s 2024-09-10T09:39:34.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:39:34 smithi102 ceph-mon[35442]: pgmap v430: 129 pgs: 129 active+clean; 665 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-09-10T09:39:35.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:39:34 smithi005 ceph-mon[30350]: pgmap v430: 129 pgs: 129 active+clean; 665 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-09-10T09:39:35.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:39:34 smithi159 ceph-mon[35513]: pgmap v430: 129 pgs: 129 active+clean; 665 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-09-10T09:39:35.885 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:39:35 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:39:36.057 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:39:35 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:39:36.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:39:35 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:39:36.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:39:36 smithi102 ceph-mon[35442]: pgmap v431: 129 pgs: 129 active+clean; 665 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 85 B/s wr, 12 op/s 2024-09-10T09:39:36.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:39:36 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:39:36.885 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:39:36 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:39:36.885 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:39:36 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:39:37.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:39:36 smithi005 ceph-mon[30350]: pgmap v431: 129 pgs: 129 active+clean; 665 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 85 B/s wr, 12 op/s 2024-09-10T09:39:37.057 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:39:36 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:39:37.057 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:39:36 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:39:37.057 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:39:36 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:39:37.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:39:36 smithi159 ceph-mon[35513]: pgmap v431: 129 pgs: 129 active+clean; 665 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 85 B/s wr, 12 op/s 2024-09-10T09:39:37.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:39:36 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:39:37.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:39:36 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:39:37.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:39:36 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:39:38.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:39:37 smithi005 ceph-mon[30350]: mgrmap e20: x(active, since 11m), standbys: z, y 2024-09-10T09:39:38.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:39:37 smithi159 ceph-mon[35513]: mgrmap e20: x(active, since 11m), standbys: z, y 2024-09-10T09:39:38.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:39:37 smithi102 ceph-mon[35442]: mgrmap e20: x(active, since 11m), standbys: z, y 2024-09-10T09:39:38.392 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T09:39:38.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:39:38 smithi005 ceph-mon[30350]: pgmap v432: 129 pgs: 129 active+clean; 665 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-10T09:39:38.855 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:39:39.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:39:38 smithi159 ceph-mon[35513]: pgmap v432: 129 pgs: 129 active+clean; 665 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-10T09:39:39.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:39:38 smithi102 ceph-mon[35442]: pgmap v432: 129 pgs: 129 active+clean; 665 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-10T09:39:39.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:39.505+0000 7fccdd400640 1 -- 172.21.15.5:0/3673965662 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fccd80745c0 msgr2=0x7fccd80749a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:39:39.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:39.505+0000 7fccdd400640 1 --2- 172.21.15.5:0/3673965662 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fccd80745c0 0x7fccd80749a0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fccc8009860 tx=0x7fccc802a3d0 comp rx=0 tx=0).stop 2024-09-10T09:39:39.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:39.505+0000 7fccdd400640 1 -- 172.21.15.5:0/3673965662 shutdown_connections 2024-09-10T09:39:39.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:39.506+0000 7fccdd400640 1 --2- 172.21.15.5:0/3673965662 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fccd810e260 0x7fccd8110650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:39.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:39.506+0000 7fccdd400640 1 --2- 172.21.15.5:0/3673965662 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fccd8074ee0 0x7fccd810dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:39.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:39.506+0000 7fccdd400640 1 --2- 172.21.15.5:0/3673965662 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fccd80745c0 0x7fccd80749a0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:39.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:39.506+0000 7fccdd400640 1 -- 172.21.15.5:0/3673965662 >> 172.21.15.5:0/3673965662 conn(0x7fccd80fd740 msgr2=0x7fccd80ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:39:39.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:39.506+0000 7fccdd400640 1 -- 172.21.15.5:0/3673965662 shutdown_connections 2024-09-10T09:39:39.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:39.506+0000 7fccdd400640 1 -- 172.21.15.5:0/3673965662 wait complete. 2024-09-10T09:39:39.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:39.507+0000 7fccdd400640 1 Processor -- start 2024-09-10T09:39:39.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:39.507+0000 7fccdd400640 1 -- start start 2024-09-10T09:39:39.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:39.508+0000 7fccdd400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fccd80745c0 0x7fccd819a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:39:39.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:39.508+0000 7fccdd400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fccd8074ee0 0x7fccd819aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:39:39.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:39.508+0000 7fccdd400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fccd810e260 0x7fccd819edd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:39:39.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:39.508+0000 7fccdd400640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fccd8114000 con 0x7fccd80745c0 2024-09-10T09:39:39.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:39.508+0000 7fccdd400640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fccd8113e80 con 0x7fccd810e260 2024-09-10T09:39:39.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:39.508+0000 7fccdd400640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fccd8114180 con 0x7fccd8074ee0 2024-09-10T09:39:39.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:39.508+0000 7fccd7e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fccd80745c0 0x7fccd819a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:39:39.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:39.508+0000 7fccd7e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fccd80745c0 0x7fccd819a500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55690/0 (socket says 172.21.15.5:55690) 2024-09-10T09:39:39.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:39.508+0000 7fccd7e00640 1 -- 172.21.15.5:0/1555113101 learned_addr learned my addr 172.21.15.5:0/1555113101 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:39:39.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:39.509+0000 7fccdca00640 1 --2- 172.21.15.5:0/1555113101 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fccd810e260 0x7fccd819edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:39:39.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:39.509+0000 7fccd7400640 1 --2- 172.21.15.5:0/1555113101 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fccd8074ee0 0x7fccd819aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:39:39.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:39.509+0000 7fccd7e00640 1 -- 172.21.15.5:0/1555113101 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fccd8074ee0 msgr2=0x7fccd819aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:39:39.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:39.509+0000 7fccd7e00640 1 --2- 172.21.15.5:0/1555113101 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fccd8074ee0 0x7fccd819aa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:39.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:39.509+0000 7fccd7e00640 1 -- 172.21.15.5:0/1555113101 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fccd810e260 msgr2=0x7fccd819edd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:39:39.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:39.509+0000 7fccd7e00640 1 --2- 172.21.15.5:0/1555113101 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fccd810e260 0x7fccd819edd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:39.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:39.509+0000 7fccd7e00640 1 -- 172.21.15.5:0/1555113101 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fccd819f4b0 con 0x7fccd80745c0 2024-09-10T09:39:39.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:39.509+0000 7fccdca00640 1 --2- 172.21.15.5:0/1555113101 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fccd810e260 0x7fccd819edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:39:39.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:39.509+0000 7fccd7e00640 1 --2- 172.21.15.5:0/1555113101 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fccd80745c0 0x7fccd819a500 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fccc8009980 tx=0x7fccc80309a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:39:39.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:39.510+0000 7fccd4c00640 1 -- 172.21.15.5:0/1555113101 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccc8040070 con 0x7fccd80745c0 2024-09-10T09:39:39.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:39.510+0000 7fccd4c00640 1 -- 172.21.15.5:0/1555113101 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fccc8005620 con 0x7fccd80745c0 2024-09-10T09:39:39.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:39.510+0000 7fccd4c00640 1 -- 172.21.15.5:0/1555113101 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccc80049b0 con 0x7fccd80745c0 2024-09-10T09:39:39.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:39.510+0000 7fccdd400640 1 -- 172.21.15.5:0/1555113101 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fccd819f740 con 0x7fccd80745c0 2024-09-10T09:39:39.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:39.511+0000 7fccdd400640 1 -- 172.21.15.5:0/1555113101 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fccd819fbd0 con 0x7fccd80745c0 2024-09-10T09:39:39.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:39.512+0000 7fccdd400640 1 -- 172.21.15.5:0/1555113101 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fccd80749a0 con 0x7fccd80745c0 2024-09-10T09:39:39.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:39.514+0000 7fccd4c00640 1 -- 172.21.15.5:0/1555113101 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7fccc8043070 con 0x7fccd80745c0 2024-09-10T09:39:39.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:39.515+0000 7fccd4c00640 1 --2- 172.21.15.5:0/1555113101 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fccbc0a2c00 0x7fccbc0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:39:39.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:39.515+0000 7fccd4c00640 1 -- 172.21.15.5:0/1555113101 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fccc80efed0 con 0x7fccd80745c0 2024-09-10T09:39:39.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:39.516+0000 7fccd7400640 1 --2- 172.21.15.5:0/1555113101 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fccbc0a2c00 0x7fccbc0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:39:39.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:39.517+0000 7fccd7400640 1 --2- 172.21.15.5:0/1555113101 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fccbc0a2c00 0x7fccbc0a50c0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fccd8110b20 tx=0x7fccc4009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:39:39.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:39.524+0000 7fccd4c00640 1 -- 172.21.15.5:0/1555113101 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fccc80bc9e0 con 0x7fccd80745c0 2024-09-10T09:39:39.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:39.971+0000 7fccdd400640 1 -- 172.21.15.5:0/1555113101 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fccd8101aa0 con 0x7fccd80745c0 2024-09-10T09:39:39.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:39.972+0000 7fccd4c00640 1 -- 172.21.15.5:0/1555113101 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11969 (secure 0 0 0) 0x7fccc80bc2e0 con 0x7fccd80745c0 2024-09-10T09:39:39.974 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-10T09:39:39.974 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 30 2024-09-10T09:39:39.974 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":30,"btime":"2024-09-10T09:38:15:834198+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14454,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/2970594311","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2970594311},{"type":"v1","addr":"172.21.15.5:6835","nonce":2970594311}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14514,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6839/3616502174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":3616502174},{"type":"v1","addr":"172.21.15.5:6839","nonce":3616502174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T09:35:29.626287+0000","modified":"2024-09-10T09:38:15.658058+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24542,"mds_1":24535,"mds_2":24511,"mds_3":24506,"mds_4":24443},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14520":{"gid":14520,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/2073599842","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":2073599842},{"type":"v1","addr":"172.21.15.5:6841","nonce":2073599842}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"k","rank":4,"incarnation":26,"state":"up:active","state_seq":24,"addr":"172.21.15.102:6833/2457007427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":2457007427},{"type":"v1","addr":"172.21.15.102:6833","nonce":2457007427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/356445159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":356445159},{"type":"v1","addr":"172.21.15.159:6833","nonce":356445159}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6837/2894314507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":2894314507},{"type":"v1","addr":"172.21.15.5:6837","nonce":2894314507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.102:6835/1647549111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":1647549111},{"type":"v1","addr":"172.21.15.102:6835","nonce":1647549111}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6835/952657242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":952657242},{"type":"v1","addr":"172.21.15.159:6835","nonce":952657242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":19,"addr":"172.21.15.102:6837/2689776764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":2689776764},{"type":"v1","addr":"172.21.15.102:6837","nonce":2689776764}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.159:6837/620627639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":620627639},{"type":"v1","addr":"172.21.15.159:6837","nonce":620627639}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24535":{"gid":24535,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":18,"addr":"172.21.15.159:6839/2073739599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":2073739599},{"type":"v1","addr":"172.21.15.159:6839","nonce":2073739599}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24542":{"gid":24542,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.102:6839/335409749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6838","nonce":335409749},{"type":"v1","addr":"172.21.15.102:6839","nonce":335409749}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24542,"qdb_cluster":[24542,24535,24511,24506,24443]},"id":1}]} 2024-09-10T09:39:39.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:39.976+0000 7fccdd400640 1 -- 172.21.15.5:0/1555113101 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fccbc0a2c00 msgr2=0x7fccbc0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:39:39.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:39.976+0000 7fccdd400640 1 --2- 172.21.15.5:0/1555113101 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fccbc0a2c00 0x7fccbc0a50c0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fccd8110b20 tx=0x7fccc4009260 comp rx=0 tx=0).stop 2024-09-10T09:39:39.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:39.976+0000 7fccdd400640 1 -- 172.21.15.5:0/1555113101 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fccd80745c0 msgr2=0x7fccd819a500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:39:39.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:39.976+0000 7fccdd400640 1 --2- 172.21.15.5:0/1555113101 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fccd80745c0 0x7fccd819a500 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fccc8009980 tx=0x7fccc80309a0 comp rx=0 tx=0).stop 2024-09-10T09:39:39.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:39.976+0000 7fccdd400640 1 -- 172.21.15.5:0/1555113101 shutdown_connections 2024-09-10T09:39:39.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:39.976+0000 7fccdd400640 1 --2- 172.21.15.5:0/1555113101 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fccbc0a2c00 0x7fccbc0a50c0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:39.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:39.976+0000 7fccdd400640 1 --2- 172.21.15.5:0/1555113101 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fccd810e260 0x7fccd819edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:39.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:39.976+0000 7fccdd400640 1 --2- 172.21.15.5:0/1555113101 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fccd8074ee0 0x7fccd819aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:39.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:39.976+0000 7fccdd400640 1 --2- 172.21.15.5:0/1555113101 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fccd80745c0 0x7fccd819a500 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:39.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:39.976+0000 7fccdd400640 1 -- 172.21.15.5:0/1555113101 >> 172.21.15.5:0/1555113101 conn(0x7fccd80fd740 msgr2=0x7fccd80fe340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:39:39.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:39.977+0000 7fccdd400640 1 -- 172.21.15.5:0/1555113101 shutdown_connections 2024-09-10T09:39:39.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:39.977+0000 7fccdd400640 1 -- 172.21.15.5:0/1555113101 wait complete. 2024-09-10T09:39:41.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:39:40 smithi005 ceph-mon[30350]: pgmap v433: 129 pgs: 129 active+clean; 665 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-10T09:39:41.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:39:40 smithi005 ceph-mon[30350]: from='client.? 172.21.15.5:0/1555113101' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:39:41.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:39:40 smithi159 ceph-mon[35513]: pgmap v433: 129 pgs: 129 active+clean; 665 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-10T09:39:41.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:39:40 smithi159 ceph-mon[35513]: from='client.? 172.21.15.5:0/1555113101' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:39:41.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:39:40 smithi102 ceph-mon[35442]: pgmap v433: 129 pgs: 129 active+clean; 665 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-10T09:39:41.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:39:40 smithi102 ceph-mon[35442]: from='client.? 172.21.15.5:0/1555113101' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:39:42.919 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:39:42 smithi005 ceph-mon[30350]: pgmap v434: 129 pgs: 129 active+clean; 666 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 255 B/s wr, 12 op/s 2024-09-10T09:39:43.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:39:42 smithi159 ceph-mon[35513]: pgmap v434: 129 pgs: 129 active+clean; 666 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 255 B/s wr, 12 op/s 2024-09-10T09:39:43.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:39:42 smithi102 ceph-mon[35442]: pgmap v434: 129 pgs: 129 active+clean; 666 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 255 B/s wr, 12 op/s 2024-09-10T09:39:43.433 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24542 quiesce db --set-id 62d3ddc3 --release --await' 2024-09-10T09:39:43.433 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24542 quiesce db --set-id 62d3ddc3 --release --await 2024-09-10T09:39:43.981 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:39:44.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:44.632+0000 7fb107200640 1 -- 172.21.15.5:0/379214522 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb1081019c0 msgr2=0x7fb108101e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:39:44.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:44.632+0000 7fb107200640 1 --2- 172.21.15.5:0/379214522 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb1081019c0 0x7fb108101e20 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fb0f4009930 tx=0x7fb0f402a420 comp rx=0 tx=0).stop 2024-09-10T09:39:44.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:44.632+0000 7fb107200640 1 -- 172.21.15.5:0/379214522 shutdown_connections 2024-09-10T09:39:44.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:44.632+0000 7fb107200640 1 --2- 172.21.15.5:0/379214522 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb108102360 0x7fb10810e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:44.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:44.632+0000 7fb107200640 1 --2- 172.21.15.5:0/379214522 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb1081019c0 0x7fb108101e20 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:44.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:44.632+0000 7fb107200640 1 --2- 172.21.15.5:0/379214522 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb1081079c0 0x7fb108107da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:44.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:44.632+0000 7fb107200640 1 -- 172.21.15.5:0/379214522 >> 172.21.15.5:0/379214522 conn(0x7fb1080fd740 msgr2=0x7fb1080ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:39:44.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:44.633+0000 7fb107200640 1 -- 172.21.15.5:0/379214522 shutdown_connections 2024-09-10T09:39:44.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:44.633+0000 7fb107200640 1 -- 172.21.15.5:0/379214522 wait complete. 2024-09-10T09:39:44.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:44.635+0000 7fb107200640 1 Processor -- start 2024-09-10T09:39:44.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:44.635+0000 7fb107200640 1 -- start start 2024-09-10T09:39:44.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:44.636+0000 7fb107200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb1081019c0 0x7fb108108910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:39:44.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:44.636+0000 7fb107200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb108102360 0x7fb108108e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:39:44.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:44.636+0000 7fb107200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb1081079c0 0x7fb108109390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:39:44.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:44.636+0000 7fb107200640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fb108111e10 con 0x7fb108102360 2024-09-10T09:39:44.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:44.636+0000 7fb107200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fb108111c90 con 0x7fb1081019c0 2024-09-10T09:39:44.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:44.636+0000 7fb107200640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fb108111f90 con 0x7fb1081079c0 2024-09-10T09:39:44.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:44.637+0000 7fb105e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb1081019c0 0x7fb108108910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:39:44.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:44.637+0000 7fb105400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb108102360 0x7fb108108e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:39:44.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:44.637+0000 7fb106800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb1081079c0 0x7fb108109390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:39:44.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:44.637+0000 7fb106800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb1081079c0 0x7fb108109390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:56894/0 (socket says 172.21.15.5:56894) 2024-09-10T09:39:44.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:44.637+0000 7fb106800640 1 -- 172.21.15.5:0/2692893671 learned_addr learned my addr 172.21.15.5:0/2692893671 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:39:44.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:44.638+0000 7fb105400640 1 -- 172.21.15.5:0/2692893671 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb1081079c0 msgr2=0x7fb108109390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:39:44.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:44.638+0000 7fb105400640 1 --2- 172.21.15.5:0/2692893671 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb1081079c0 0x7fb108109390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:44.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:44.638+0000 7fb105400640 1 -- 172.21.15.5:0/2692893671 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb1081019c0 msgr2=0x7fb108108910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:39:44.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:44.638+0000 7fb105400640 1 --2- 172.21.15.5:0/2692893671 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb1081019c0 0x7fb108108910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:44.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:44.638+0000 7fb105400640 1 -- 172.21.15.5:0/2692893671 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1081aab40 con 0x7fb108102360 2024-09-10T09:39:44.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:44.639+0000 7fb105400640 1 --2- 172.21.15.5:0/2692893671 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb108102360 0x7fb108108e50 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fb0f4009850 tx=0x7fb0f4003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:39:44.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:44.639+0000 7fb0fea00640 1 -- 172.21.15.5:0/2692893671 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0f403f070 con 0x7fb108102360 2024-09-10T09:39:44.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:44.639+0000 7fb0fea00640 1 -- 172.21.15.5:0/2692893671 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb0f4031c90 con 0x7fb108102360 2024-09-10T09:39:44.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:44.639+0000 7fb0fea00640 1 -- 172.21.15.5:0/2692893671 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0f4035030 con 0x7fb108102360 2024-09-10T09:39:44.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:44.640+0000 7fb107200640 1 -- 172.21.15.5:0/2692893671 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1081aadd0 con 0x7fb108102360 2024-09-10T09:39:44.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:44.640+0000 7fb107200640 1 -- 172.21.15.5:0/2692893671 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1081ab200 con 0x7fb108102360 2024-09-10T09:39:44.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:44.645+0000 7fb10f6bd740 1 Processor -- start 2024-09-10T09:39:44.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:44.645+0000 7fb10f6bd740 1 -- start start 2024-09-10T09:39:44.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:44.646+0000 7fb10f6bd740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ed0787a520 0x55ed0787a900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:39:44.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:44.646+0000 7fb10f6bd740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ed0787ae40 0x55ed078851c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:39:44.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:44.646+0000 7fb10f6bd740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ed07885700 0x55ed07887af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:39:44.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:44.646+0000 7fb10f6bd740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55ed077e0650 con 0x55ed07885700 2024-09-10T09:39:44.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:44.646+0000 7fb10f6bd740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55ed07787f10 con 0x55ed0787ae40 2024-09-10T09:39:44.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:44.646+0000 7fb10f6bd740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55ed077dc610 con 0x55ed0787a520 2024-09-10T09:39:44.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:44.646+0000 7fb0fea00640 1 -- 172.21.15.5:0/2692893671 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7fb0f4042070 con 0x7fb108102360 2024-09-10T09:39:44.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:44.646+0000 7fb106800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ed0787a520 0x55ed0787a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:39:44.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:44.646+0000 7fb106800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ed0787a520 0x55ed0787a900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:56904/0 (socket says 172.21.15.5:56904) 2024-09-10T09:39:44.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:44.646+0000 7fb106800640 1 -- 172.21.15.5:0/2950772840 learned_addr learned my addr 172.21.15.5:0/2950772840 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:39:44.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:44.646+0000 7fb105e00640 1 --2- 172.21.15.5:0/2950772840 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ed0787ae40 0x55ed078851c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:39:44.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:44.647+0000 7fb105400640 1 --2- 172.21.15.5:0/2950772840 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ed07885700 0x55ed07887af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:39:44.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:44.647+0000 7fb106800640 1 -- 172.21.15.5:0/2950772840 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ed0787ae40 msgr2=0x55ed078851c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:39:44.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:44.648+0000 7fb106800640 1 --2- 172.21.15.5:0/2950772840 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ed0787ae40 0x55ed078851c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:44.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:44.648+0000 7fb106800640 1 -- 172.21.15.5:0/2950772840 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ed07885700 msgr2=0x55ed07887af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:39:44.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:44.648+0000 7fb106800640 1 --2- 172.21.15.5:0/2950772840 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ed07885700 0x55ed07887af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:44.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:44.648+0000 7fb106800640 1 -- 172.21.15.5:0/2950772840 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ed074ef580 con 0x55ed0787a520 2024-09-10T09:39:44.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:44.648+0000 7fb105400640 1 --2- 172.21.15.5:0/2950772840 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ed07885700 0x55ed07887af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:39:44.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:44.648+0000 7fb0fea00640 1 --2- 172.21.15.5:0/2692893671 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fb0ec0a2c40 0x7fb0ec0a5100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:39:44.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:44.648+0000 7fb0fea00640 1 -- 172.21.15.5:0/2692893671 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fb0f40ef890 con 0x7fb108102360 2024-09-10T09:39:44.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:44.649+0000 7fb105e00640 1 --2- 172.21.15.5:0/2692893671 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fb0ec0a2c40 0x7fb0ec0a5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:39:44.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:44.649+0000 7fb106800640 1 --2- 172.21.15.5:0/2950772840 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ed0787a520 0x55ed0787a900 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fb100011570 tx=0x7fb100011a30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=80286b8cbaba4b1d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:39:44.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:44.649+0000 7fb0f3000640 1 -- 172.21.15.5:0/2950772840 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb100004b20 con 0x55ed0787a520 2024-09-10T09:39:44.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:44.649+0000 7fb0f3000640 1 -- 172.21.15.5:0/2950772840 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb100004cc0 con 0x55ed0787a520 2024-09-10T09:39:44.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:44.649+0000 7fb0f3000640 1 -- 172.21.15.5:0/2950772840 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb100005560 con 0x55ed0787a520 2024-09-10T09:39:44.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:44.650+0000 7fb10f6bd740 1 -- 172.21.15.5:0/2950772840 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ed0787a520 msgr2=0x55ed0787a900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:39:44.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:44.650+0000 7fb10f6bd740 1 --2- 172.21.15.5:0/2950772840 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ed0787a520 0x55ed0787a900 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fb100011570 tx=0x7fb100011a30 comp rx=0 tx=0).stop 2024-09-10T09:39:44.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:44.650+0000 7fb10f6bd740 1 -- 172.21.15.5:0/2950772840 shutdown_connections 2024-09-10T09:39:44.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:44.650+0000 7fb10f6bd740 1 --2- 172.21.15.5:0/2950772840 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ed07885700 0x55ed07887af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:44.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:44.650+0000 7fb10f6bd740 1 --2- 172.21.15.5:0/2950772840 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ed0787ae40 0x55ed078851c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:44.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:44.650+0000 7fb10f6bd740 1 --2- 172.21.15.5:0/2950772840 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ed0787a520 0x55ed0787a900 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:44.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:44.650+0000 7fb10f6bd740 1 -- 172.21.15.5:0/2950772840 >> 172.21.15.5:0/2950772840 conn(0x55ed07876ab0 msgr2=0x55ed07876e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:39:44.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:44.651+0000 7fb10f6bd740 1 -- 172.21.15.5:0/2950772840 shutdown_connections 2024-09-10T09:39:44.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:44.651+0000 7fb10f6bd740 1 -- 172.21.15.5:0/2950772840 wait complete. 2024-09-10T09:39:44.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:44.651+0000 7fb105e00640 1 --2- 172.21.15.5:0/2692893671 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fb0ec0a2c40 0x7fb0ec0a5100 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x55ed07877910 tx=0x7fb0f8005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:39:44.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:44.651+0000 7fb10f6bd740 1 -- start start 2024-09-10T09:39:44.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:44.652+0000 7fb10f6bd740 1 Processor -- start 2024-09-10T09:39:44.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:44.652+0000 7fb10f6bd740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ed0787a520 0x55ed07882570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:39:44.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:44.652+0000 7fb10f6bd740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ed0787ae40 0x55ed07882ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:39:44.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:44.652+0000 7fb10f6bd740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ed07885700 0x55ed07882ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:39:44.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:44.652+0000 7fb10f6bd740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55ed078882d0 con 0x55ed0787a520 2024-09-10T09:39:44.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:44.652+0000 7fb10f6bd740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55ed0779b350 con 0x55ed0787ae40 2024-09-10T09:39:44.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:44.652+0000 7fb10f6bd740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55ed07888450 con 0x55ed07885700 2024-09-10T09:39:44.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:44.652+0000 7fb10f6bd740 10 client.0 _ll_register_callbacks cb 0x55ed077be290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:39:44.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:44.652+0000 7fb105400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ed0787ae40 0x55ed07882ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:39:44.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:44.653+0000 7fb105400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ed0787ae40 0x55ed07882ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:41982/0 (socket says 172.21.15.5:41982) 2024-09-10T09:39:44.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:44.653+0000 7fb105400640 1 -- 172.21.15.5:0/4158602115 learned_addr learned my addr 172.21.15.5:0/4158602115 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:39:44.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:44.652+0000 7fb105e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ed0787a520 0x55ed07882570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:39:44.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:44.653+0000 7fb105400640 1 -- 172.21.15.5:0/4158602115 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ed07885700 msgr2=0x55ed07882ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:39:44.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:44.653+0000 7fb105400640 1 --2- 172.21.15.5:0/4158602115 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ed07885700 0x55ed07882ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:44.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:44.653+0000 7fb105400640 1 -- 172.21.15.5:0/4158602115 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ed0787a520 msgr2=0x55ed07882570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:39:44.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:44.653+0000 7fb105400640 1 --2- 172.21.15.5:0/4158602115 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ed0787a520 0x55ed07882570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:44.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:44.653+0000 7fb105400640 1 -- 172.21.15.5:0/4158602115 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ed075b9960 con 0x55ed0787ae40 2024-09-10T09:39:44.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:44.653+0000 7fb105e00640 1 --2- 172.21.15.5:0/4158602115 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ed0787a520 0x55ed07882570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:39:44.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:44.653+0000 7fb105400640 1 --2- 172.21.15.5:0/4158602115 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ed0787ae40 0x55ed07882ab0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fb0f4000c00 tx=0x7fb0f40046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2fef5d3557e4cb7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:39:44.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:44.653+0000 7fb0e7400640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-09-10T09:39:44.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:44.654+0000 7fb0e7400640 1 -- 172.21.15.5:0/4158602115 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0f40bc400 con 0x55ed0787ae40 2024-09-10T09:39:44.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:44.654+0000 7fb10f6bd740 1 -- 172.21.15.5:0/4158602115 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb100004cc0 con 0x55ed0787ae40 2024-09-10T09:39:44.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:44.654+0000 7fb0e7400640 1 -- 172.21.15.5:0/4158602115 <== mon.2 v2:172.21.15.102:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fb0f40b8370 con 0x55ed0787ae40 2024-09-10T09:39:44.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:44.654+0000 7fb0e7400640 1 -- 172.21.15.5:0/4158602115 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fb0f40bbd00 con 0x55ed0787ae40 2024-09-10T09:39:44.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:44.654+0000 7fb10f6bd740 10 client.25625 fetch_fsmap learned FSMap version 30 2024-09-10T09:39:44.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:44.654+0000 7fb10f6bd740 1 -- 172.21.15.5:0/4158602115 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55ed078882d0 con 0x55ed0787ae40 2024-09-10T09:39:44.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:44.655+0000 7fb0e7400640 1 -- 172.21.15.5:0/4158602115 <== mon.2 v2:172.21.15.102:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb100004cc0 con 0x55ed0787ae40 2024-09-10T09:39:44.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:44.655+0000 7fb0e7400640 1 -- 172.21.15.5:0/4158602115 <== mon.2 v2:172.21.15.102:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fb0f40c44e0 con 0x55ed0787ae40 2024-09-10T09:39:44.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:44.655+0000 7fb10f6bd740 10 client.25625 fetch_fsmap finished waiting for FSMap version 30 2024-09-10T09:39:44.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:44.655+0000 7fb10f6bd740 10 client.25625 resolve_mds: validated gid 24542 aka daemon mds.b 2024-09-10T09:39:44.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:44.655+0000 7fb10f6bd740 20 client.25625 populate_metadata read hostname 'smithi005' 2024-09-10T09:39:44.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:44.655+0000 7fb10f6bd740 1 --2- 172.21.15.5:0/4158602115 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55ed078a6310 0x55ed078a8740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:39:44.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:44.655+0000 7fb10f6bd740 4 client.25625 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:39:44.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:44.655+0000 7fb10f6bd740 1 -- 172.21.15.5:0/4158602115 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ed077edbc0 con 0x55ed078a6310 2024-09-10T09:39:44.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:44.656+0000 7fb105e00640 1 --2- 172.21.15.5:0/4158602115 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55ed078a6310 0x55ed078a8740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:39:44.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:44.656+0000 7fb105e00640 1 --2- 172.21.15.5:0/4158602115 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55ed078a6310 0x55ed078a8740 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ff5b838ca4c905d4 server_cookie=860b9b7c2fedbae2 in_seq=0 out_seq=0 2024-09-10T09:39:44.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:44.656+0000 7fb0e7400640 10 client.25625 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T09:39:44.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:44.657+0000 7fb0e7400640 1 -- 172.21.15.5:0/4158602115 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ed077edbc0 con 0x55ed078a6310 2024-09-10T09:39:44.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:44.657+0000 7fb0e7400640 10 client.25625 handle_command_reply: tid=0 2024-09-10T09:39:44.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:44.657+0000 7fb10f6bd740 1 client.25625 shutdown 2024-09-10T09:39:44.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:44.657+0000 7fb10f6bd740 1 -- 172.21.15.5:0/4158602115 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ed0787ae40 msgr2=0x55ed07882ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:39:44.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:44.657+0000 7fb10f6bd740 1 --2- 172.21.15.5:0/4158602115 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ed0787ae40 0x55ed07882ab0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fb0f4000c00 tx=0x7fb0f40046c0 comp rx=0 tx=0).stop 2024-09-10T09:39:44.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:44.658+0000 7fb10f6bd740 1 -- 172.21.15.5:0/4158602115 shutdown_connections 2024-09-10T09:39:44.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:44.658+0000 7fb10f6bd740 1 --2- 172.21.15.5:0/4158602115 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55ed078a6310 0x55ed078a8740 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:44.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:44.658+0000 7fb10f6bd740 1 --2- 172.21.15.5:0/4158602115 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ed07885700 0x55ed07882ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:44.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:44.658+0000 7fb10f6bd740 1 --2- 172.21.15.5:0/4158602115 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ed0787ae40 0x55ed07882ab0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:44.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:44.658+0000 7fb10f6bd740 1 --2- 172.21.15.5:0/4158602115 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ed0787a520 0x55ed07882570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:44.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:44.658+0000 7fb10f6bd740 1 -- 172.21.15.5:0/4158602115 >> 172.21.15.5:0/4158602115 conn(0x55ed07876ab0 msgr2=0x55ed07875cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:39:44.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:44.658+0000 7fb0e7400640 0 client.25625 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T09:39:44.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:44.658+0000 7fb10f6bd740 1 -- 172.21.15.5:0/4158602115 shutdown_connections 2024-09-10T09:39:44.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:44.658+0000 7fb10f6bd740 1 -- 172.21.15.5:0/4158602115 wait complete. 2024-09-10T09:39:44.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:44.658+0000 7fb10f6bd740 20 client.25625 trim_cache size 0 max 16384 2024-09-10T09:39:44.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:44.680+0000 7fb10f6bd740 1 Processor -- start 2024-09-10T09:39:44.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:44.680+0000 7fb10f6bd740 1 -- start start 2024-09-10T09:39:44.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:44.680+0000 7fb10f6bd740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ed0787a520 0x55ed0787cf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:39:44.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:44.680+0000 7fb10f6bd740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ed0787ae40 0x55ed07882e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:39:44.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:44.680+0000 7fb10f6bd740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ed07885700 0x55ed078ae050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:39:44.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:44.680+0000 7fb10f6bd740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fb10806d280 con 0x55ed0787ae40 2024-09-10T09:39:44.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:44.680+0000 7fb10f6bd740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fb10806edc0 con 0x55ed07885700 2024-09-10T09:39:44.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:44.680+0000 7fb10f6bd740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fb10806d950 con 0x55ed0787a520 2024-09-10T09:39:44.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:44.681+0000 7fb105400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ed0787ae40 0x55ed07882e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:39:44.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:44.681+0000 7fb105400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ed0787ae40 0x55ed07882e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54154/0 (socket says 172.21.15.5:54154) 2024-09-10T09:39:44.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:44.681+0000 7fb105400640 1 -- 172.21.15.5:0/4071277815 learned_addr learned my addr 172.21.15.5:0/4071277815 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:39:44.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:44.681+0000 7fb106800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ed07885700 0x55ed078ae050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:39:44.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:44.681+0000 7fb105e00640 1 --2- 172.21.15.5:0/4071277815 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ed0787a520 0x55ed0787cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:39:44.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:44.682+0000 7fb106800640 1 -- 172.21.15.5:0/4071277815 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ed0787a520 msgr2=0x55ed0787cf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:39:44.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:44.682+0000 7fb106800640 1 --2- 172.21.15.5:0/4071277815 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ed0787a520 0x55ed0787cf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:44.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:44.682+0000 7fb106800640 1 -- 172.21.15.5:0/4071277815 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ed0787ae40 msgr2=0x55ed07882e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:39:44.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:44.682+0000 7fb106800640 1 --2- 172.21.15.5:0/4071277815 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ed0787ae40 0x55ed07882e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:44.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:44.682+0000 7fb106800640 1 -- 172.21.15.5:0/4071277815 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ed07884e40 con 0x55ed07885700 2024-09-10T09:39:44.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:44.682+0000 7fb106800640 1 --2- 172.21.15.5:0/4071277815 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ed07885700 0x55ed078ae050 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fb100011f30 tx=0x7fb100009a10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6c9c5cf668c6ba68 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:39:44.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:44.682+0000 7fb0f2600640 1 -- 172.21.15.5:0/4071277815 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb100028020 con 0x55ed07885700 2024-09-10T09:39:44.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:44.683+0000 7fb0f2600640 1 -- 172.21.15.5:0/4071277815 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb10000f5c0 con 0x55ed07885700 2024-09-10T09:39:44.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:44.683+0000 7fb0f2600640 1 -- 172.21.15.5:0/4071277815 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1000052a0 con 0x55ed07885700 2024-09-10T09:39:44.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:44.683+0000 7fb10f6bd740 1 -- 172.21.15.5:0/4071277815 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ed07885700 msgr2=0x55ed078ae050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:39:44.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:44.683+0000 7fb10f6bd740 1 --2- 172.21.15.5:0/4071277815 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ed07885700 0x55ed078ae050 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fb100011f30 tx=0x7fb100009a10 comp rx=0 tx=0).stop 2024-09-10T09:39:44.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:44.684+0000 7fb10f6bd740 1 -- 172.21.15.5:0/4071277815 shutdown_connections 2024-09-10T09:39:44.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:44.684+0000 7fb10f6bd740 1 --2- 172.21.15.5:0/4071277815 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ed07885700 0x55ed078ae050 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:44.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:44.684+0000 7fb10f6bd740 1 --2- 172.21.15.5:0/4071277815 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ed0787ae40 0x55ed07882e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:44.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:44.684+0000 7fb10f6bd740 1 --2- 172.21.15.5:0/4071277815 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ed0787a520 0x55ed0787cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:44.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:44.684+0000 7fb10f6bd740 1 -- 172.21.15.5:0/4071277815 >> 172.21.15.5:0/4071277815 conn(0x55ed07876ab0 msgr2=0x55ed07875cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:39:44.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:44.684+0000 7fb10f6bd740 1 -- 172.21.15.5:0/4071277815 shutdown_connections 2024-09-10T09:39:44.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:44.684+0000 7fb10f6bd740 1 -- 172.21.15.5:0/4071277815 wait complete. 2024-09-10T09:39:44.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:44.684+0000 7fb10f6bd740 1 -- start start 2024-09-10T09:39:44.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:44.685+0000 7fb10f6bd740 1 Processor -- start 2024-09-10T09:39:44.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:44.685+0000 7fb10f6bd740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ed0787a520 0x55ed0789ef00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:39:44.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:44.685+0000 7fb10f6bd740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ed0787ae40 0x55ed0787f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:39:44.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:44.685+0000 7fb10f6bd740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ed07885700 0x55ed0787f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:39:44.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:44.685+0000 7fb10f6bd740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fb10806e6f0 con 0x55ed0787ae40 2024-09-10T09:39:44.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:44.685+0000 7fb10f6bd740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fb10806e020 con 0x55ed07885700 2024-09-10T09:39:44.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:44.685+0000 7fb10f6bd740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fb108109e40 con 0x55ed0787a520 2024-09-10T09:39:44.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:44.685+0000 7fb10f6bd740 10 client.0 _ll_register_callbacks cb 0x7fb0f4031400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:39:44.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:44.686+0000 7fb106800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ed07885700 0x55ed0787f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:39:44.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:44.686+0000 7fb106800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ed07885700 0x55ed0787f7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:41994/0 (socket says 172.21.15.5:41994) 2024-09-10T09:39:44.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:44.686+0000 7fb106800640 1 -- 172.21.15.5:0/2104350020 learned_addr learned my addr 172.21.15.5:0/2104350020 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:39:44.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:44.686+0000 7fb105e00640 1 --2- 172.21.15.5:0/2104350020 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ed0787a520 0x55ed0789ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:39:44.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:44.686+0000 7fb106800640 1 -- 172.21.15.5:0/2104350020 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ed0787a520 msgr2=0x55ed0789ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:39:44.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:44.686+0000 7fb106800640 1 --2- 172.21.15.5:0/2104350020 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ed0787a520 0x55ed0789ef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:44.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:44.686+0000 7fb106800640 1 -- 172.21.15.5:0/2104350020 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ed0787ae40 msgr2=0x55ed0787f2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:39:44.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:44.686+0000 7fb106800640 1 --2- 172.21.15.5:0/2104350020 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ed0787ae40 0x55ed0787f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:44.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:44.686+0000 7fb106800640 1 -- 172.21.15.5:0/2104350020 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ed07880010 con 0x55ed07885700 2024-09-10T09:39:44.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:44.687+0000 7fb106800640 1 --2- 172.21.15.5:0/2104350020 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ed07885700 0x55ed0787f7e0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fb10000f9a0 tx=0x7fb10000f9d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=42f2b1d48b287032 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:39:44.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:44.687+0000 7fb0e7400640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-09-10T09:39:44.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:44.687+0000 7fb0e7400640 1 -- 172.21.15.5:0/2104350020 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb10000fd80 con 0x55ed07885700 2024-09-10T09:39:44.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:44.687+0000 7fb10f6bd740 1 -- 172.21.15.5:0/2104350020 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb10000f5c0 con 0x55ed07885700 2024-09-10T09:39:44.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:44.687+0000 7fb0e7400640 1 -- 172.21.15.5:0/2104350020 <== mon.2 v2:172.21.15.102:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fb100005040 con 0x55ed07885700 2024-09-10T09:39:44.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:44.687+0000 7fb0e7400640 1 -- 172.21.15.5:0/2104350020 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fb10003b050 con 0x55ed07885700 2024-09-10T09:39:44.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:44.687+0000 7fb10f6bd740 10 client.25631 fetch_fsmap learned FSMap version 30 2024-09-10T09:39:44.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:44.687+0000 7fb10f6bd740 1 -- 172.21.15.5:0/2104350020 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55ed07911030 con 0x55ed07885700 2024-09-10T09:39:44.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:44.688+0000 7fb0e7400640 1 -- 172.21.15.5:0/2104350020 <== mon.2 v2:172.21.15.102:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb10000f5c0 con 0x55ed07885700 2024-09-10T09:39:44.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:44.688+0000 7fb0e7400640 1 -- 172.21.15.5:0/2104350020 <== mon.2 v2:172.21.15.102:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fb100024630 con 0x55ed07885700 2024-09-10T09:39:44.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:44.688+0000 7fb10f6bd740 10 client.25631 fetch_fsmap finished waiting for FSMap version 30 2024-09-10T09:39:44.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:44.688+0000 7fb10f6bd740 10 client.25631 resolve_mds: validated gid 24542 aka daemon mds.b 2024-09-10T09:39:44.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:44.688+0000 7fb10f6bd740 20 client.25631 populate_metadata read hostname 'smithi005' 2024-09-10T09:39:44.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:44.688+0000 7fb10f6bd740 1 --2- 172.21.15.5:0/2104350020 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55ed07911530 0x55ed07913950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:39:44.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:44.688+0000 7fb10f6bd740 4 client.25631 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "62d3ddc3", "release": true, "await": true}] 2024-09-10T09:39:44.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:44.688+0000 7fb10f6bd740 1 -- 172.21.15.5:0/2104350020 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "quiesce db", "set_id": "62d3ddc3", "release": true, "await": true}) -- 0x55ed077e80d0 con 0x55ed07911530 2024-09-10T09:39:44.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:44.688+0000 7fb105e00640 1 --2- 172.21.15.5:0/2104350020 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55ed07911530 0x55ed07913950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:39:44.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:44.689+0000 7fb105e00640 1 --2- 172.21.15.5:0/2104350020 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55ed07911530 0x55ed07913950 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2ad39bea29f47ede server_cookie=49be074c8eab93da in_seq=0 out_seq=0 2024-09-10T09:39:44.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:44.689+0000 7fb0e7400640 10 client.25631 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T09:39:44.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:44.691+0000 7fb0e7400640 1 -- 172.21.15.5:0/2104350020 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x55ed077e80d0 con 0x55ed07911530 2024-09-10T09:39:44.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:44.691+0000 7fb0e7400640 10 client.25631 handle_command_reply: tid=0 2024-09-10T09:39:44.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:44.691+0000 7fb10f6bd740 1 client.25631 shutdown 2024-09-10T09:39:44.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:44.691+0000 7fb10f6bd740 1 -- 172.21.15.5:0/2104350020 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ed07885700 msgr2=0x55ed0787f7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:39:44.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:44.691+0000 7fb10f6bd740 1 --2- 172.21.15.5:0/2104350020 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ed07885700 0x55ed0787f7e0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fb10000f9a0 tx=0x7fb10000f9d0 comp rx=0 tx=0).stop 2024-09-10T09:39:44.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:44.691+0000 7fb10f6bd740 1 -- 172.21.15.5:0/2104350020 shutdown_connections 2024-09-10T09:39:44.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:44.692+0000 7fb10f6bd740 1 --2- 172.21.15.5:0/2104350020 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55ed07911530 0x55ed07913950 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:44.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:44.692+0000 7fb10f6bd740 1 --2- 172.21.15.5:0/2104350020 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ed07885700 0x55ed0787f7e0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:44.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:44.692+0000 7fb10f6bd740 1 --2- 172.21.15.5:0/2104350020 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ed0787ae40 0x55ed0787f2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:44.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:44.692+0000 7fb10f6bd740 1 --2- 172.21.15.5:0/2104350020 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ed0787a520 0x55ed0789ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:44.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:44.692+0000 7fb10f6bd740 1 -- 172.21.15.5:0/2104350020 >> 172.21.15.5:0/2104350020 conn(0x55ed07876ab0 msgr2=0x55ed07883bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:39:44.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:44.692+0000 7fb0e7400640 0 client.25631 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T09:39:44.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:44.692+0000 7fb10f6bd740 1 -- 172.21.15.5:0/2104350020 shutdown_connections 2024-09-10T09:39:44.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:44.692+0000 7fb10f6bd740 1 -- 172.21.15.5:0/2104350020 wait complete. 2024-09-10T09:39:44.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:44.692+0000 7fb10f6bd740 20 client.25631 trim_cache size 0 max 16384 2024-09-10T09:39:44.694 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-10T09:39:44.694 INFO:teuthology.orchestra.run.smithi005.stdout: "epoch": 30, 2024-09-10T09:39:44.694 INFO:teuthology.orchestra.run.smithi005.stdout: "leader": 24542, 2024-09-10T09:39:44.694 INFO:teuthology.orchestra.run.smithi005.stdout: "set_version": 4, 2024-09-10T09:39:44.694 INFO:teuthology.orchestra.run.smithi005.stdout: "sets": { 2024-09-10T09:39:44.695 INFO:teuthology.orchestra.run.smithi005.stdout: "62d3ddc3": { 2024-09-10T09:39:44.695 INFO:teuthology.orchestra.run.smithi005.stdout: "version": 4, 2024-09-10T09:39:44.695 INFO:teuthology.orchestra.run.smithi005.stdout: "age_ref": 0.0, 2024-09-10T09:39:44.695 INFO:teuthology.orchestra.run.smithi005.stdout: "state": { 2024-09-10T09:39:44.695 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "RELEASED", 2024-09-10T09:39:44.695 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0 2024-09-10T09:39:44.695 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T09:39:44.695 INFO:teuthology.orchestra.run.smithi005.stdout: "timeout": 90.0, 2024-09-10T09:39:44.695 INFO:teuthology.orchestra.run.smithi005.stdout: "expiration": 150.0, 2024-09-10T09:39:44.695 INFO:teuthology.orchestra.run.smithi005.stdout: "members": { 2024-09-10T09:39:44.695 INFO:teuthology.orchestra.run.smithi005.stdout: "file:/": { 2024-09-10T09:39:44.695 INFO:teuthology.orchestra.run.smithi005.stdout: "excluded": false, 2024-09-10T09:39:44.695 INFO:teuthology.orchestra.run.smithi005.stdout: "state": { 2024-09-10T09:39:44.695 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "RELEASED", 2024-09-10T09:39:44.695 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0 2024-09-10T09:39:44.696 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T09:39:44.696 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T09:39:44.696 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T09:39:44.696 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T09:39:44.696 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T09:39:44.696 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-10T09:39:44.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:44.693+0000 7fb107200640 1 -- 172.21.15.5:0/2692893671 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fb0ec0a2c40 msgr2=0x7fb0ec0a5100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:39:44.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:44.693+0000 7fb107200640 1 --2- 172.21.15.5:0/2692893671 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fb0ec0a2c40 0x7fb0ec0a5100 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x55ed07877910 tx=0x7fb0f8005f90 comp rx=0 tx=0).stop 2024-09-10T09:39:44.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:44.693+0000 7fb107200640 1 -- 172.21.15.5:0/2692893671 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb108102360 msgr2=0x7fb108108e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:39:44.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:44.693+0000 7fb107200640 1 --2- 172.21.15.5:0/2692893671 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb108102360 0x7fb108108e50 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fb0f4009850 tx=0x7fb0f4003b90 comp rx=0 tx=0).stop 2024-09-10T09:39:44.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:44.694+0000 7fb107200640 1 -- 172.21.15.5:0/2692893671 shutdown_connections 2024-09-10T09:39:44.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:44.694+0000 7fb107200640 1 --2- 172.21.15.5:0/2692893671 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fb0ec0a2c40 0x7fb0ec0a5100 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:44.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:44.694+0000 7fb107200640 1 --2- 172.21.15.5:0/2692893671 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb1081079c0 0x7fb108109390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:44.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:44.694+0000 7fb107200640 1 --2- 172.21.15.5:0/2692893671 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb108102360 0x7fb108108e50 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:44.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:44.694+0000 7fb107200640 1 --2- 172.21.15.5:0/2692893671 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb1081019c0 0x7fb108108910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:44.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:44.694+0000 7fb107200640 1 -- 172.21.15.5:0/2692893671 >> 172.21.15.5:0/2692893671 conn(0x7fb1080fd740 msgr2=0x7fb10810d090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:39:44.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:44.694+0000 7fb107200640 1 -- 172.21.15.5:0/2692893671 shutdown_connections 2024-09-10T09:39:44.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:44.694+0000 7fb107200640 1 -- 172.21.15.5:0/2692893671 wait complete. 2024-09-10T09:39:44.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:39:44 smithi005 ceph-mon[30350]: pgmap v435: 129 pgs: 129 active+clean; 666 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-09-10T09:39:45.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:39:44 smithi159 ceph-mon[35513]: pgmap v435: 129 pgs: 129 active+clean; 666 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-09-10T09:39:45.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:39:44 smithi102 ceph-mon[35442]: pgmap v435: 129 pgs: 129 active+clean; 666 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-09-10T09:39:45.641 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '62d3ddc3', total seconds elapsed: 37.3 2024-09-10T09:39:45.641 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 30.0 2024-09-10T09:39:45.641 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T09:39:46.050 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:39:46.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:46.644+0000 7ff863000640 1 -- 172.21.15.5:0/3348417374 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff85c1049c0 msgr2=0x7ff85c108a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:39:46.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:46.644+0000 7ff863000640 1 --2- 172.21.15.5:0/3348417374 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff85c1049c0 0x7ff85c108a10 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7ff848009860 tx=0x7ff84802a340 comp rx=0 tx=0).stop 2024-09-10T09:39:46.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:46.644+0000 7ff863000640 1 -- 172.21.15.5:0/3348417374 shutdown_connections 2024-09-10T09:39:46.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:46.644+0000 7ff863000640 1 --2- 172.21.15.5:0/3348417374 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff85c109140 0x7ff85c110cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:46.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:46.644+0000 7ff863000640 1 --2- 172.21.15.5:0/3348417374 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff85c1049c0 0x7ff85c108a10 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:46.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:46.644+0000 7ff863000640 1 --2- 172.21.15.5:0/3348417374 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff85c104010 0x7ff85c1043f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:46.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:46.644+0000 7ff863000640 1 -- 172.21.15.5:0/3348417374 >> 172.21.15.5:0/3348417374 conn(0x7ff85c0ffa50 msgr2=0x7ff85c101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:39:46.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:46.644+0000 7ff863000640 1 -- 172.21.15.5:0/3348417374 shutdown_connections 2024-09-10T09:39:46.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:46.644+0000 7ff863000640 1 -- 172.21.15.5:0/3348417374 wait complete. 2024-09-10T09:39:46.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:46.645+0000 7ff863000640 1 Processor -- start 2024-09-10T09:39:46.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:46.645+0000 7ff863000640 1 -- start start 2024-09-10T09:39:46.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:46.646+0000 7ff863000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff85c104010 0x7ff85c19a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:39:46.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:46.646+0000 7ff863000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff85c1049c0 0x7ff85c19abf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:39:46.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:46.646+0000 7ff863000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff85c109140 0x7ff85c19ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:39:46.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:46.646+0000 7ff863000640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff85c114140 con 0x7ff85c1049c0 2024-09-10T09:39:46.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:46.646+0000 7ff863000640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7ff85c113fc0 con 0x7ff85c104010 2024-09-10T09:39:46.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:46.646+0000 7ff863000640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7ff85c1142c0 con 0x7ff85c109140 2024-09-10T09:39:46.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:46.646+0000 7ff85b400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff85c1049c0 0x7ff85c19abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:39:46.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:46.646+0000 7ff85be00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff85c104010 0x7ff85c19a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:39:46.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:46.646+0000 7ff85be00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff85c104010 0x7ff85c19a6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:42026/0 (socket says 172.21.15.5:42026) 2024-09-10T09:39:46.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:46.646+0000 7ff85be00640 1 -- 172.21.15.5:0/1548803530 learned_addr learned my addr 172.21.15.5:0/1548803530 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:39:46.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:46.646+0000 7ff861000640 1 --2- 172.21.15.5:0/1548803530 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff85c109140 0x7ff85c19ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:39:46.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:46.647+0000 7ff85be00640 1 -- 172.21.15.5:0/1548803530 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff85c109140 msgr2=0x7ff85c19ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:39:46.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:46.647+0000 7ff85be00640 1 --2- 172.21.15.5:0/1548803530 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff85c109140 0x7ff85c19ef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:46.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:46.647+0000 7ff85be00640 1 -- 172.21.15.5:0/1548803530 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff85c1049c0 msgr2=0x7ff85c19abf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:39:46.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:46.647+0000 7ff85be00640 1 --2- 172.21.15.5:0/1548803530 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff85c1049c0 0x7ff85c19abf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:46.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:46.647+0000 7ff85be00640 1 -- 172.21.15.5:0/1548803530 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff85c19f660 con 0x7ff85c104010 2024-09-10T09:39:46.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:46.647+0000 7ff85be00640 1 --2- 172.21.15.5:0/1548803530 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff85c104010 0x7ff85c19a6b0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7ff84c00e9c0 tx=0x7ff84c00ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:39:46.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:46.647+0000 7ff858c00640 1 -- 172.21.15.5:0/1548803530 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff84c00ccc0 con 0x7ff85c104010 2024-09-10T09:39:46.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:46.647+0000 7ff863000640 1 -- 172.21.15.5:0/1548803530 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff85c19f950 con 0x7ff85c104010 2024-09-10T09:39:46.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:46.648+0000 7ff858c00640 1 -- 172.21.15.5:0/1548803530 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff84c005500 con 0x7ff85c104010 2024-09-10T09:39:46.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:46.648+0000 7ff858c00640 1 -- 172.21.15.5:0/1548803530 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff84c010430 con 0x7ff85c104010 2024-09-10T09:39:46.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:46.648+0000 7ff863000640 1 -- 172.21.15.5:0/1548803530 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff85c1a7230 con 0x7ff85c104010 2024-09-10T09:39:46.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:46.649+0000 7ff863000640 1 -- 172.21.15.5:0/1548803530 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff85c1043f0 con 0x7ff85c104010 2024-09-10T09:39:46.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:46.652+0000 7ff858c00640 1 -- 172.21.15.5:0/1548803530 <== mon.2 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7ff84c0046c0 con 0x7ff85c104010 2024-09-10T09:39:46.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:46.653+0000 7ff858c00640 1 --2- 172.21.15.5:0/1548803530 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7ff8400a2c00 0x7ff8400a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:39:46.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:46.653+0000 7ff858c00640 1 -- 172.21.15.5:0/1548803530 <== mon.2 v2:172.21.15.102:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7ff84c0cf680 con 0x7ff85c104010 2024-09-10T09:39:46.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:46.654+0000 7ff85b400640 1 --2- 172.21.15.5:0/1548803530 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7ff8400a2c00 0x7ff8400a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:39:46.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:46.654+0000 7ff858c00640 1 -- 172.21.15.5:0/1548803530 <== mon.2 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff84c09c190 con 0x7ff85c104010 2024-09-10T09:39:46.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:46.655+0000 7ff85b400640 1 --2- 172.21.15.5:0/1548803530 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7ff8400a2c00 0x7ff8400a50c0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7ff84802a930 tx=0x7ff8480056d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:39:46.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:39:46 smithi005 ceph-mon[30350]: pgmap v436: 129 pgs: 129 active+clean; 666 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-09-10T09:39:46.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:46.878+0000 7ff863000640 1 -- 172.21.15.5:0/1548803530 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff85c10c3b0 con 0x7ff85c104010 2024-09-10T09:39:46.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:46.879+0000 7ff858c00640 1 -- 172.21.15.5:0/1548803530 <== mon.2 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11969 (secure 0 0 0) 0x7ff84c09ba90 con 0x7ff85c104010 2024-09-10T09:39:46.881 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-10T09:39:46.881 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":30,"btime":"2024-09-10T09:38:15:834198+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14454,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/2970594311","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2970594311},{"type":"v1","addr":"172.21.15.5:6835","nonce":2970594311}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14514,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6839/3616502174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":3616502174},{"type":"v1","addr":"172.21.15.5:6839","nonce":3616502174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T09:35:29.626287+0000","modified":"2024-09-10T09:38:15.658058+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24542,"mds_1":24535,"mds_2":24511,"mds_3":24506,"mds_4":24443},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14520":{"gid":14520,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/2073599842","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":2073599842},{"type":"v1","addr":"172.21.15.5:6841","nonce":2073599842}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"k","rank":4,"incarnation":26,"state":"up:active","state_seq":24,"addr":"172.21.15.102:6833/2457007427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":2457007427},{"type":"v1","addr":"172.21.15.102:6833","nonce":2457007427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/356445159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":356445159},{"type":"v1","addr":"172.21.15.159:6833","nonce":356445159}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6837/2894314507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":2894314507},{"type":"v1","addr":"172.21.15.5:6837","nonce":2894314507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.102:6835/1647549111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":1647549111},{"type":"v1","addr":"172.21.15.102:6835","nonce":1647549111}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6835/952657242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":952657242},{"type":"v1","addr":"172.21.15.159:6835","nonce":952657242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":19,"addr":"172.21.15.102:6837/2689776764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":2689776764},{"type":"v1","addr":"172.21.15.102:6837","nonce":2689776764}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.159:6837/620627639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":620627639},{"type":"v1","addr":"172.21.15.159:6837","nonce":620627639}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24535":{"gid":24535,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":18,"addr":"172.21.15.159:6839/2073739599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":2073739599},{"type":"v1","addr":"172.21.15.159:6839","nonce":2073739599}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24542":{"gid":24542,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.102:6839/335409749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6838","nonce":335409749},{"type":"v1","addr":"172.21.15.102:6839","nonce":335409749}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24542,"qdb_cluster":[24542,24535,24511,24506,24443]},"id":1}]} 2024-09-10T09:39:46.881 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 30 2024-09-10T09:39:46.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:46.882+0000 7ff863000640 1 -- 172.21.15.5:0/1548803530 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7ff8400a2c00 msgr2=0x7ff8400a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:39:46.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:46.882+0000 7ff863000640 1 --2- 172.21.15.5:0/1548803530 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7ff8400a2c00 0x7ff8400a50c0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7ff84802a930 tx=0x7ff8480056d0 comp rx=0 tx=0).stop 2024-09-10T09:39:46.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:46.882+0000 7ff863000640 1 -- 172.21.15.5:0/1548803530 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff85c104010 msgr2=0x7ff85c19a6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:39:46.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:46.882+0000 7ff863000640 1 --2- 172.21.15.5:0/1548803530 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff85c104010 0x7ff85c19a6b0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7ff84c00e9c0 tx=0x7ff84c00ee80 comp rx=0 tx=0).stop 2024-09-10T09:39:46.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:46.882+0000 7ff863000640 1 -- 172.21.15.5:0/1548803530 shutdown_connections 2024-09-10T09:39:46.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:46.882+0000 7ff863000640 1 --2- 172.21.15.5:0/1548803530 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7ff8400a2c00 0x7ff8400a50c0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:46.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:46.882+0000 7ff863000640 1 --2- 172.21.15.5:0/1548803530 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff85c109140 0x7ff85c19ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:46.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:46.882+0000 7ff863000640 1 --2- 172.21.15.5:0/1548803530 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff85c1049c0 0x7ff85c19abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:46.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:46.882+0000 7ff863000640 1 --2- 172.21.15.5:0/1548803530 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff85c104010 0x7ff85c19a6b0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:46.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:46.882+0000 7ff863000640 1 -- 172.21.15.5:0/1548803530 >> 172.21.15.5:0/1548803530 conn(0x7ff85c0ffa50 msgr2=0x7ff85c100e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:39:46.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:46.882+0000 7ff863000640 1 -- 172.21.15.5:0/1548803530 shutdown_connections 2024-09-10T09:39:46.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:46.882+0000 7ff863000640 1 -- 172.21.15.5:0/1548803530 wait complete. 2024-09-10T09:39:47.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:39:46 smithi159 ceph-mon[35513]: pgmap v436: 129 pgs: 129 active+clean; 666 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-09-10T09:39:47.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:39:46 smithi102 ceph-mon[35442]: pgmap v436: 129 pgs: 129 active+clean; 666 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-09-10T09:39:47.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:39:47 smithi005 ceph-mon[30350]: from='client.? 172.21.15.5:0/1548803530' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:39:48.068 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24542 quiesce db / --timeout 90 --expiration 150.0 --await-for 45.0' 2024-09-10T09:39:48.068 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24542 quiesce db / --timeout 90 --expiration 150.0 --await-for 45.0 2024-09-10T09:39:48.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:39:47 smithi159 ceph-mon[35513]: from='client.? 172.21.15.5:0/1548803530' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:39:48.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:39:47 smithi102 ceph-mon[35442]: from='client.? 172.21.15.5:0/1548803530' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:39:48.461 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:39:48.957 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:39:48 smithi005 ceph-mon[30350]: pgmap v437: 129 pgs: 129 active+clean; 666 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 9 op/s 2024-09-10T09:39:49.001 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:39:48 smithi159 ceph-mon[35513]: pgmap v437: 129 pgs: 129 active+clean; 666 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 9 op/s 2024-09-10T09:39:49.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:49.081+0000 7f9947200640 1 -- 172.21.15.5:0/2540331795 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9948074e30 msgr2=0x7f994810dd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:39:49.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:49.081+0000 7f9947200640 1 --2- 172.21.15.5:0/2540331795 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9948074e30 0x7f994810dd40 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f9934009900 tx=0x7f993402a480 comp rx=0 tx=0).stop 2024-09-10T09:39:49.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:49.081+0000 7f9947200640 1 -- 172.21.15.5:0/2540331795 shutdown_connections 2024-09-10T09:39:49.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:49.081+0000 7f9947200640 1 --2- 172.21.15.5:0/2540331795 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f994810e280 0x7f9948110670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:49.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:49.081+0000 7f9947200640 1 --2- 172.21.15.5:0/2540331795 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9948074e30 0x7f994810dd40 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:49.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:49.081+0000 7f9947200640 1 --2- 172.21.15.5:0/2540331795 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9948074510 0x7f99480748f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:49.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:49.081+0000 7f9947200640 1 -- 172.21.15.5:0/2540331795 >> 172.21.15.5:0/2540331795 conn(0x7f99480fd740 msgr2=0x7f99480ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:39:49.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:49.081+0000 7f9947200640 1 -- 172.21.15.5:0/2540331795 shutdown_connections 2024-09-10T09:39:49.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:49.081+0000 7f9947200640 1 -- 172.21.15.5:0/2540331795 wait complete. 2024-09-10T09:39:49.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:49.083+0000 7f9947200640 1 Processor -- start 2024-09-10T09:39:49.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:49.083+0000 7f9947200640 1 -- start start 2024-09-10T09:39:49.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:49.083+0000 7f9947200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9948074510 0x7f994819a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:39:49.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:49.083+0000 7f9947200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9948074e30 0x7f994819aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:39:49.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:49.083+0000 7f9947200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f994810e280 0x7f994819edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:39:49.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:49.083+0000 7f9947200640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f9948114000 con 0x7f9948074510 2024-09-10T09:39:49.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:49.083+0000 7f9947200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f9948113e80 con 0x7f994810e280 2024-09-10T09:39:49.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:49.083+0000 7f9947200640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f9948114180 con 0x7f9948074e30 2024-09-10T09:39:49.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:49.084+0000 7f9945e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9948074510 0x7f994819a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:39:49.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:49.084+0000 7f9945e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9948074510 0x7f994819a4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54212/0 (socket says 172.21.15.5:54212) 2024-09-10T09:39:49.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:49.084+0000 7f9945e00640 1 -- 172.21.15.5:0/1857117217 learned_addr learned my addr 172.21.15.5:0/1857117217 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:39:49.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:49.084+0000 7f9946800640 1 --2- 172.21.15.5:0/1857117217 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f994810e280 0x7f994819edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:39:49.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:49.084+0000 7f9945400640 1 --2- 172.21.15.5:0/1857117217 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9948074e30 0x7f994819aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:39:49.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:49.084+0000 7f9945e00640 1 -- 172.21.15.5:0/1857117217 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9948074e30 msgr2=0x7f994819aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:39:49.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:49.084+0000 7f9945e00640 1 --2- 172.21.15.5:0/1857117217 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9948074e30 0x7f994819aa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:49.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:49.084+0000 7f9945e00640 1 -- 172.21.15.5:0/1857117217 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f994810e280 msgr2=0x7f994819edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:39:49.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:49.084+0000 7f9945e00640 1 --2- 172.21.15.5:0/1857117217 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f994810e280 0x7f994819edb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:49.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:49.084+0000 7f9945e00640 1 -- 172.21.15.5:0/1857117217 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f994819f490 con 0x7f9948074510 2024-09-10T09:39:49.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:49.084+0000 7f9945e00640 1 --2- 172.21.15.5:0/1857117217 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9948074510 0x7f994819a4e0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f993800e9f0 tx=0x7f993800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:39:49.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:49.085+0000 7f993ea00640 1 -- 172.21.15.5:0/1857117217 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f993800cd80 con 0x7f9948074510 2024-09-10T09:39:49.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:49.085+0000 7f993ea00640 1 -- 172.21.15.5:0/1857117217 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9938010910 con 0x7f9948074510 2024-09-10T09:39:49.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:49.085+0000 7f9947200640 1 -- 172.21.15.5:0/1857117217 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f994819f780 con 0x7f9948074510 2024-09-10T09:39:49.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:49.085+0000 7f993ea00640 1 -- 172.21.15.5:0/1857117217 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9938018990 con 0x7f9948074510 2024-09-10T09:39:49.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:49.085+0000 7f9947200640 1 -- 172.21.15.5:0/1857117217 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99481a7060 con 0x7f9948074510 2024-09-10T09:39:49.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:49.088+0000 7f993ea00640 1 -- 172.21.15.5:0/1857117217 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f9938010ac0 con 0x7f9948074510 2024-09-10T09:39:49.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:49.089+0000 7f994f3fa740 1 Processor -- start 2024-09-10T09:39:49.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:49.089+0000 7f994f3fa740 1 -- start start 2024-09-10T09:39:49.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:49.089+0000 7f994f3fa740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dacc0a0560 0x55dacc0a0940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:39:49.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:49.089+0000 7f994f3fa740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dacc0a0e80 0x55dacc0ab200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:39:49.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:49.089+0000 7f994f3fa740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dacc0ab740 0x55dacc0adb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:39:49.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:49.089+0000 7f994f3fa740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55dacc006530 con 0x55dacc0ab740 2024-09-10T09:39:49.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:49.089+0000 7f994f3fa740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55dacbfad860 con 0x55dacc0a0e80 2024-09-10T09:39:49.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:49.089+0000 7f994f3fa740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55dacc0024f0 con 0x55dacc0a0560 2024-09-10T09:39:49.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:49.090+0000 7f9945400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dacc0ab740 0x55dacc0adb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:39:49.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:49.090+0000 7f9945e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dacc0a0e80 0x55dacc0ab200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:39:49.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:49.090+0000 7f9946800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dacc0a0560 0x55dacc0a0940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:39:49.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:49.090+0000 7f9945400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dacc0ab740 0x55dacc0adb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54224/0 (socket says 172.21.15.5:54224) 2024-09-10T09:39:49.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:49.090+0000 7f9945400640 1 -- 172.21.15.5:0/17953684 learned_addr learned my addr 172.21.15.5:0/17953684 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:39:49.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:49.090+0000 7f9946800640 1 -- 172.21.15.5:0/17953684 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dacc0a0e80 msgr2=0x55dacc0ab200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:39:49.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:49.090+0000 7f9946800640 1 --2- 172.21.15.5:0/17953684 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dacc0a0e80 0x55dacc0ab200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:49.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:49.090+0000 7f9946800640 1 -- 172.21.15.5:0/17953684 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dacc0ab740 msgr2=0x55dacc0adb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:39:49.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:49.090+0000 7f9946800640 1 --2- 172.21.15.5:0/17953684 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dacc0ab740 0x55dacc0adb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:49.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:49.090+0000 7f9946800640 1 -- 172.21.15.5:0/17953684 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dacbd77b90 con 0x55dacc0a0560 2024-09-10T09:39:49.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:49.090+0000 7f993ea00640 1 --2- 172.21.15.5:0/1857117217 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f992c0a2c00 0x7f992c0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:39:49.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:49.091+0000 7f993ea00640 1 -- 172.21.15.5:0/1857117217 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f99380cf700 con 0x7f9948074510 2024-09-10T09:39:49.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:49.091+0000 7f9945e00640 1 --2- 172.21.15.5:0/1857117217 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f992c0a2c00 0x7f992c0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:39:49.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:49.091+0000 7f9946800640 1 --2- 172.21.15.5:0/17953684 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dacc0a0560 0x55dacc0a0940 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f9940005fd0 tx=0x7f9940010740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ab0f7e65e02b7824 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:39:49.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:49.091+0000 7f9933000640 1 -- 172.21.15.5:0/17953684 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9940016070 con 0x55dacc0a0560 2024-09-10T09:39:49.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:49.091+0000 7f9933000640 1 -- 172.21.15.5:0/17953684 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9940004d20 con 0x55dacc0a0560 2024-09-10T09:39:49.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:49.091+0000 7f9933000640 1 -- 172.21.15.5:0/17953684 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9940012780 con 0x55dacc0a0560 2024-09-10T09:39:49.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:49.092+0000 7f9945e00640 1 --2- 172.21.15.5:0/1857117217 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f992c0a2c00 0x7f992c0a50c0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f9938098020 tx=0x7f99380d5530 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:39:49.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:49.092+0000 7f994f3fa740 1 -- 172.21.15.5:0/17953684 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dacc0a0560 msgr2=0x55dacc0a0940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:39:49.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:49.092+0000 7f994f3fa740 1 --2- 172.21.15.5:0/17953684 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dacc0a0560 0x55dacc0a0940 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f9940005fd0 tx=0x7f9940010740 comp rx=0 tx=0).stop 2024-09-10T09:39:49.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:49.092+0000 7f994f3fa740 1 -- 172.21.15.5:0/17953684 shutdown_connections 2024-09-10T09:39:49.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:49.092+0000 7f994f3fa740 1 --2- 172.21.15.5:0/17953684 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dacc0ab740 0x55dacc0adb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:49.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:49.092+0000 7f994f3fa740 1 --2- 172.21.15.5:0/17953684 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dacc0a0e80 0x55dacc0ab200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:49.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:49.092+0000 7f994f3fa740 1 --2- 172.21.15.5:0/17953684 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dacc0a0560 0x55dacc0a0940 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:49.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:49.092+0000 7f994f3fa740 1 -- 172.21.15.5:0/17953684 >> 172.21.15.5:0/17953684 conn(0x55dacc09caf0 msgr2=0x55dacc09ced0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:39:49.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:49.093+0000 7f994f3fa740 1 -- 172.21.15.5:0/17953684 shutdown_connections 2024-09-10T09:39:49.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:49.093+0000 7f994f3fa740 1 -- 172.21.15.5:0/17953684 wait complete. 2024-09-10T09:39:49.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:49.093+0000 7f994f3fa740 1 -- start start 2024-09-10T09:39:49.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:49.093+0000 7f994f3fa740 1 Processor -- start 2024-09-10T09:39:49.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:49.094+0000 7f994f3fa740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dacc0a0560 0x55dacc0a8620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:39:49.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:49.094+0000 7f994f3fa740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dacc0a0e80 0x55dacc0a8b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:39:49.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:49.094+0000 7f994f3fa740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dacc0ab740 0x55dacc0a90a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:39:49.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:49.094+0000 7f994f3fa740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55dacc0ae310 con 0x55dacc0a0e80 2024-09-10T09:39:49.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:49.094+0000 7f994f3fa740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55dacbfc0e60 con 0x55dacc0ab740 2024-09-10T09:39:49.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:49.094+0000 7f994f3fa740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55dacc0ae490 con 0x55dacc0a0560 2024-09-10T09:39:49.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:49.094+0000 7f994f3fa740 10 client.0 _ll_register_callbacks cb 0x55dacbfc5cc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:39:49.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:49.094+0000 7f9945400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dacc0a0e80 0x55dacc0a8b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:39:49.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:49.094+0000 7f9945400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dacc0a0e80 0x55dacc0a8b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54226/0 (socket says 172.21.15.5:54226) 2024-09-10T09:39:49.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:49.094+0000 7f9945400640 1 -- 172.21.15.5:0/3932355937 learned_addr learned my addr 172.21.15.5:0/3932355937 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:39:49.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:49.094+0000 7f9946800640 1 --2- 172.21.15.5:0/3932355937 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dacc0a0560 0x55dacc0a8620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:39:49.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:49.095+0000 7f9946800640 1 --2- 172.21.15.5:0/3932355937 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dacc0ab740 0x55dacc0a90a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:39:49.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:49.095+0000 7f9945400640 1 -- 172.21.15.5:0/3932355937 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dacc0a0560 msgr2=0x55dacc0a8620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:39:49.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:49.095+0000 7f9945400640 1 --2- 172.21.15.5:0/3932355937 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dacc0a0560 0x55dacc0a8620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:49.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:49.095+0000 7f9945400640 1 -- 172.21.15.5:0/3932355937 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dacc0ab740 msgr2=0x55dacc0a90a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:39:49.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:49.095+0000 7f9945400640 1 --2- 172.21.15.5:0/3932355937 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dacc0ab740 0x55dacc0a90a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:49.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:49.095+0000 7f9945400640 1 -- 172.21.15.5:0/3932355937 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dacbfcc990 con 0x55dacc0a0e80 2024-09-10T09:39:49.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:49.095+0000 7f9946800640 1 --2- 172.21.15.5:0/3932355937 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dacc0ab740 0x55dacc0a90a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:39:49.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:49.095+0000 7f9945400640 1 --2- 172.21.15.5:0/3932355937 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dacc0a0e80 0x55dacc0a8b60 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f993402a9a0 tx=0x7f9934004440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=871b3910fc14880b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:39:49.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:49.095+0000 7f9927400640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-10T09:39:49.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:49.095+0000 7f9927400640 1 -- 172.21.15.5:0/3932355937 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9934035040 con 0x55dacc0a0e80 2024-09-10T09:39:49.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:49.096+0000 7f994f3fa740 1 -- 172.21.15.5:0/3932355937 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9940004d20 con 0x55dacc0a0e80 2024-09-10T09:39:49.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:49.096+0000 7f9927400640 1 -- 172.21.15.5:0/3932355937 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f9934005450 con 0x55dacc0a0e80 2024-09-10T09:39:49.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:49.096+0000 7f9927400640 1 -- 172.21.15.5:0/3932355937 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f9934005650 con 0x55dacc0a0e80 2024-09-10T09:39:49.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:49.096+0000 7f994f3fa740 10 client.15636 fetch_fsmap learned FSMap version 30 2024-09-10T09:39:49.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:49.096+0000 7f994f3fa740 1 -- 172.21.15.5:0/3932355937 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55dacc0b9f60 con 0x55dacc0a0e80 2024-09-10T09:39:49.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:49.096+0000 7f9927400640 1 -- 172.21.15.5:0/3932355937 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9940004d20 con 0x55dacc0a0e80 2024-09-10T09:39:49.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:49.096+0000 7f9927400640 1 -- 172.21.15.5:0/3932355937 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f9934003a30 con 0x55dacc0a0e80 2024-09-10T09:39:49.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:49.097+0000 7f994f3fa740 10 client.15636 fetch_fsmap finished waiting for FSMap version 30 2024-09-10T09:39:49.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:49.097+0000 7f994f3fa740 10 client.15636 resolve_mds: validated gid 24542 aka daemon mds.b 2024-09-10T09:39:49.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:49.097+0000 7f994f3fa740 20 client.15636 populate_metadata read hostname 'smithi005' 2024-09-10T09:39:49.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:49.097+0000 7f994f3fa740 1 --2- 172.21.15.5:0/3932355937 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55dacc0bb420 0x55dacc0bd840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:39:49.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:49.097+0000 7f994f3fa740 4 client.15636 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:39:49.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:49.097+0000 7f994f3fa740 1 -- 172.21.15.5:0/3932355937 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dacbfa9310 con 0x55dacc0bb420 2024-09-10T09:39:49.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:49.097+0000 7f9946800640 1 --2- 172.21.15.5:0/3932355937 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55dacc0bb420 0x55dacc0bd840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:39:49.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:49.098+0000 7f9946800640 1 --2- 172.21.15.5:0/3932355937 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55dacc0bb420 0x55dacc0bd840 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c9b1938cabd8eb7d server_cookie=fd2c647ee647ab2 in_seq=0 out_seq=0 2024-09-10T09:39:49.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:49.098+0000 7f9927400640 10 client.15636 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T09:39:49.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:49.099+0000 7f9927400640 1 -- 172.21.15.5:0/3932355937 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55dacbfa9310 con 0x55dacc0bb420 2024-09-10T09:39:49.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:49.099+0000 7f9927400640 10 client.15636 handle_command_reply: tid=0 2024-09-10T09:39:49.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:49.099+0000 7f994f3fa740 1 client.15636 shutdown 2024-09-10T09:39:49.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:49.099+0000 7f994f3fa740 1 -- 172.21.15.5:0/3932355937 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dacc0a0e80 msgr2=0x55dacc0a8b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:39:49.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:49.099+0000 7f994f3fa740 1 --2- 172.21.15.5:0/3932355937 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dacc0a0e80 0x55dacc0a8b60 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f993402a9a0 tx=0x7f9934004440 comp rx=0 tx=0).stop 2024-09-10T09:39:49.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:49.099+0000 7f994f3fa740 1 -- 172.21.15.5:0/3932355937 shutdown_connections 2024-09-10T09:39:49.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:49.099+0000 7f994f3fa740 1 --2- 172.21.15.5:0/3932355937 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55dacc0bb420 0x55dacc0bd840 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:49.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:49.099+0000 7f994f3fa740 1 --2- 172.21.15.5:0/3932355937 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dacc0ab740 0x55dacc0a90a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:49.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:49.099+0000 7f994f3fa740 1 --2- 172.21.15.5:0/3932355937 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dacc0a0e80 0x55dacc0a8b60 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:49.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:49.099+0000 7f994f3fa740 1 --2- 172.21.15.5:0/3932355937 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dacc0a0560 0x55dacc0a8620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:49.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:49.099+0000 7f994f3fa740 1 -- 172.21.15.5:0/3932355937 >> 172.21.15.5:0/3932355937 conn(0x55dacc09caf0 msgr2=0x55dacc0a27d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:39:49.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:49.099+0000 7f9927400640 0 client.15636 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T09:39:49.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:49.099+0000 7f994f3fa740 1 -- 172.21.15.5:0/3932355937 shutdown_connections 2024-09-10T09:39:49.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:49.099+0000 7f994f3fa740 1 -- 172.21.15.5:0/3932355937 wait complete. 2024-09-10T09:39:49.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:49.100+0000 7f994f3fa740 20 client.15636 trim_cache size 0 max 16384 2024-09-10T09:39:49.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:49.117+0000 7f994f3fa740 1 Processor -- start 2024-09-10T09:39:49.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:49.117+0000 7f994f3fa740 1 -- start start 2024-09-10T09:39:49.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:49.118+0000 7f994f3fa740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dacc0a0560 0x55dacc0a8ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:39:49.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:49.118+0000 7f994f3fa740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dacc0a0e80 0x55dacc0c3110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:39:49.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:49.118+0000 7f994f3fa740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dacc0ab740 0x55dacc0c3650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:39:49.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:49.118+0000 7f994f3fa740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f99481b0e70 con 0x55dacc0a0e80 2024-09-10T09:39:49.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:49.118+0000 7f994f3fa740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f99481b29b0 con 0x55dacc0a0560 2024-09-10T09:39:49.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:49.118+0000 7f994f3fa740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f99481b1540 con 0x55dacc0ab740 2024-09-10T09:39:49.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:49.118+0000 7f9946800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dacc0a0560 0x55dacc0a8ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:39:49.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:49.118+0000 7f9945400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dacc0a0e80 0x55dacc0c3110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:39:49.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:49.118+0000 7f9946800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dacc0a0560 0x55dacc0a8ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:42088/0 (socket says 172.21.15.5:42088) 2024-09-10T09:39:49.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:49.118+0000 7f9946800640 1 -- 172.21.15.5:0/1064337611 learned_addr learned my addr 172.21.15.5:0/1064337611 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:39:49.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:49.118+0000 7f9946800640 1 --2- 172.21.15.5:0/1064337611 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dacc0ab740 0x55dacc0c3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:39:49.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:49.119+0000 7f9945400640 1 -- 172.21.15.5:0/1064337611 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dacc0ab740 msgr2=0x55dacc0c3650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:39:49.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:49.119+0000 7f9945400640 1 --2- 172.21.15.5:0/1064337611 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dacc0ab740 0x55dacc0c3650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:49.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:49.119+0000 7f9945400640 1 -- 172.21.15.5:0/1064337611 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dacc0a0560 msgr2=0x55dacc0a8ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:39:49.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:49.119+0000 7f9945400640 1 --2- 172.21.15.5:0/1064337611 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dacc0a0560 0x55dacc0a8ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:49.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:49.119+0000 7f9945400640 1 -- 172.21.15.5:0/1064337611 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dacc09c920 con 0x55dacc0a0e80 2024-09-10T09:39:49.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:49.119+0000 7f9946800640 1 --2- 172.21.15.5:0/1064337611 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dacc0a0560 0x55dacc0a8ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:39:49.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:49.119+0000 7f9945400640 1 --2- 172.21.15.5:0/1064337611 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dacc0a0e80 0x55dacc0c3110 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f99340315b0 tx=0x7f9934009290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26feeba6b846fced server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:39:49.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:49.119+0000 7f9932600640 1 -- 172.21.15.5:0/1064337611 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9934035030 con 0x55dacc0a0e80 2024-09-10T09:39:49.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:49.119+0000 7f9932600640 1 -- 172.21.15.5:0/1064337611 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9934041070 con 0x55dacc0a0e80 2024-09-10T09:39:49.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:49.119+0000 7f9932600640 1 -- 172.21.15.5:0/1064337611 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9934003850 con 0x55dacc0a0e80 2024-09-10T09:39:49.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:49.120+0000 7f994f3fa740 1 -- 172.21.15.5:0/1064337611 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dacc0a0e80 msgr2=0x55dacc0c3110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:39:49.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:49.120+0000 7f994f3fa740 1 --2- 172.21.15.5:0/1064337611 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dacc0a0e80 0x55dacc0c3110 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f99340315b0 tx=0x7f9934009290 comp rx=0 tx=0).stop 2024-09-10T09:39:49.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:49.120+0000 7f994f3fa740 1 -- 172.21.15.5:0/1064337611 shutdown_connections 2024-09-10T09:39:49.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:49.120+0000 7f994f3fa740 1 --2- 172.21.15.5:0/1064337611 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dacc0ab740 0x55dacc0c3650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:49.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:49.120+0000 7f994f3fa740 1 --2- 172.21.15.5:0/1064337611 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dacc0a0e80 0x55dacc0c3110 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:49.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:49.120+0000 7f994f3fa740 1 --2- 172.21.15.5:0/1064337611 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dacc0a0560 0x55dacc0a8ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:49.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:49.120+0000 7f994f3fa740 1 -- 172.21.15.5:0/1064337611 >> 172.21.15.5:0/1064337611 conn(0x55dacc09caf0 msgr2=0x55dacc0aa590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:39:49.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:49.120+0000 7f994f3fa740 1 -- 172.21.15.5:0/1064337611 shutdown_connections 2024-09-10T09:39:49.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:49.120+0000 7f994f3fa740 1 -- 172.21.15.5:0/1064337611 wait complete. 2024-09-10T09:39:49.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:49.121+0000 7f994f3fa740 1 -- start start 2024-09-10T09:39:49.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:49.121+0000 7f994f3fa740 1 Processor -- start 2024-09-10T09:39:49.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:49.122+0000 7f994f3fa740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dacc0a0560 0x55dacc0b36e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:39:49.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:49.122+0000 7f994f3fa740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dacc0a0e80 0x55dacc0b3c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:39:49.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:49.122+0000 7f994f3fa740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dacc0ab740 0x55dacc0b06b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:39:49.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:49.122+0000 7f994f3fa740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f99481b22e0 con 0x55dacc0ab740 2024-09-10T09:39:49.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:49.122+0000 7f994f3fa740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f99481b1c10 con 0x55dacc0a0560 2024-09-10T09:39:49.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:49.122+0000 7f994f3fa740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f99481aee50 con 0x55dacc0a0e80 2024-09-10T09:39:49.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:49.122+0000 7f994f3fa740 10 client.0 _ll_register_callbacks cb 0x7f9934009640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:39:49.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:49.122+0000 7f9946800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dacc0a0560 0x55dacc0b36e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:39:49.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:49.122+0000 7f9946800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dacc0a0560 0x55dacc0b36e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:42090/0 (socket says 172.21.15.5:42090) 2024-09-10T09:39:49.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:49.122+0000 7f9946800640 1 -- 172.21.15.5:0/1419653666 learned_addr learned my addr 172.21.15.5:0/1419653666 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:39:49.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:49.122+0000 7f9946800640 1 --2- 172.21.15.5:0/1419653666 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dacc0ab740 0x55dacc0b06b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:39:49.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:49.122+0000 7f9945400640 1 --2- 172.21.15.5:0/1419653666 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dacc0a0e80 0x55dacc0b3c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:39:49.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:49.123+0000 7f9946800640 1 -- 172.21.15.5:0/1419653666 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dacc0a0e80 msgr2=0x55dacc0b3c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:39:49.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:49.123+0000 7f9946800640 1 --2- 172.21.15.5:0/1419653666 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dacc0a0e80 0x55dacc0b3c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:49.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:49.123+0000 7f9946800640 1 -- 172.21.15.5:0/1419653666 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dacc0ab740 msgr2=0x55dacc0b06b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:39:49.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:49.123+0000 7f9946800640 1 --2- 172.21.15.5:0/1419653666 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dacc0ab740 0x55dacc0b06b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:49.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:49.123+0000 7f9946800640 1 -- 172.21.15.5:0/1419653666 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dacc0b0c70 con 0x55dacc0a0560 2024-09-10T09:39:49.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:49.123+0000 7f9946800640 1 --2- 172.21.15.5:0/1419653666 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dacc0a0560 0x55dacc0b36e0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f99400127b0 tx=0x7f994001c040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=38599713a235a4e2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:39:49.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:49.123+0000 7f9927400640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-09-10T09:39:49.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:49.123+0000 7f9927400640 1 -- 172.21.15.5:0/1419653666 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9940016040 con 0x55dacc0a0560 2024-09-10T09:39:49.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:49.123+0000 7f994f3fa740 1 -- 172.21.15.5:0/1419653666 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9934041070 con 0x55dacc0a0560 2024-09-10T09:39:49.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:49.123+0000 7f9927400640 1 -- 172.21.15.5:0/1419653666 <== mon.2 v2:172.21.15.102:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f9940012d70 con 0x55dacc0a0560 2024-09-10T09:39:49.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:49.124+0000 7f9927400640 1 -- 172.21.15.5:0/1419653666 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f9940023040 con 0x55dacc0a0560 2024-09-10T09:39:49.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:49.124+0000 7f994f3fa740 10 client.25655 fetch_fsmap learned FSMap version 30 2024-09-10T09:39:49.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:49.124+0000 7f994f3fa740 1 -- 172.21.15.5:0/1419653666 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55dacc126990 con 0x55dacc0a0560 2024-09-10T09:39:49.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:49.124+0000 7f9927400640 1 -- 172.21.15.5:0/1419653666 <== mon.2 v2:172.21.15.102:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9934041070 con 0x55dacc0a0560 2024-09-10T09:39:49.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:49.124+0000 7f9927400640 1 -- 172.21.15.5:0/1419653666 <== mon.2 v2:172.21.15.102:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f994000f040 con 0x55dacc0a0560 2024-09-10T09:39:49.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:49.124+0000 7f994f3fa740 10 client.25655 fetch_fsmap finished waiting for FSMap version 30 2024-09-10T09:39:49.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:49.124+0000 7f994f3fa740 10 client.25655 resolve_mds: validated gid 24542 aka daemon mds.b 2024-09-10T09:39:49.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:49.124+0000 7f994f3fa740 20 client.25655 populate_metadata read hostname 'smithi005' 2024-09-10T09:39:49.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:49.124+0000 7f994f3fa740 1 --2- 172.21.15.5:0/1419653666 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55dacc126fa0 0x55dacc1293c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:39:49.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:49.124+0000 7f994f3fa740 4 client.25655 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 150.0, "await_for": 45.0}] 2024-09-10T09:39:49.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:49.124+0000 7f994f3fa740 1 -- 172.21.15.5:0/1419653666 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 150.0, "await_for": 45.0}) -- 0x55dacc013c60 con 0x55dacc126fa0 2024-09-10T09:39:49.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:49.125+0000 7f9945400640 1 --2- 172.21.15.5:0/1419653666 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55dacc126fa0 0x55dacc1293c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:39:49.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:49.125+0000 7f9945400640 1 --2- 172.21.15.5:0/1419653666 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55dacc126fa0 0x55dacc1293c0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5385411fd4d98f85 server_cookie=c0ab12da3c1c5722 in_seq=0 out_seq=0 2024-09-10T09:39:49.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:49.125+0000 7f9927400640 10 client.25655 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T09:39:49.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:49.132+0000 7f9927400640 1 -- 172.21.15.5:0/1419653666 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x55dacc013c60 con 0x55dacc126fa0 2024-09-10T09:39:49.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:49.132+0000 7f9927400640 10 client.25655 handle_command_reply: tid=0 2024-09-10T09:39:49.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:49.132+0000 7f994f3fa740 1 client.25655 shutdown 2024-09-10T09:39:49.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:49.132+0000 7f994f3fa740 1 -- 172.21.15.5:0/1419653666 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dacc0a0560 msgr2=0x55dacc0b36e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:39:49.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:49.132+0000 7f994f3fa740 1 --2- 172.21.15.5:0/1419653666 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dacc0a0560 0x55dacc0b36e0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f99400127b0 tx=0x7f994001c040 comp rx=0 tx=0).stop 2024-09-10T09:39:49.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:49.132+0000 7f994f3fa740 1 -- 172.21.15.5:0/1419653666 shutdown_connections 2024-09-10T09:39:49.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:49.132+0000 7f994f3fa740 1 --2- 172.21.15.5:0/1419653666 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55dacc126fa0 0x55dacc1293c0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:49.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:49.132+0000 7f994f3fa740 1 --2- 172.21.15.5:0/1419653666 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dacc0ab740 0x55dacc0b06b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:49.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:49.132+0000 7f994f3fa740 1 --2- 172.21.15.5:0/1419653666 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dacc0a0e80 0x55dacc0b3c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:49.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:49.132+0000 7f994f3fa740 1 --2- 172.21.15.5:0/1419653666 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dacc0a0560 0x55dacc0b36e0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:49.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:49.132+0000 7f9927400640 0 client.25655 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T09:39:49.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:49.132+0000 7f994f3fa740 1 -- 172.21.15.5:0/1419653666 >> 172.21.15.5:0/1419653666 conn(0x55dacc09caf0 msgr2=0x55dacc0a6dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:39:49.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:49.132+0000 7f994f3fa740 1 -- 172.21.15.5:0/1419653666 shutdown_connections 2024-09-10T09:39:49.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:49.132+0000 7f994f3fa740 1 -- 172.21.15.5:0/1419653666 wait complete. 2024-09-10T09:39:49.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:49.133+0000 7f994f3fa740 20 client.25655 trim_cache size 0 max 16384 2024-09-10T09:39:49.136 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:39:48 smithi102 ceph-mon[35442]: pgmap v437: 129 pgs: 129 active+clean; 666 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 9 op/s 2024-09-10T09:39:49.136 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-10T09:39:49.136 INFO:teuthology.orchestra.run.smithi005.stdout: "epoch": 30, 2024-09-10T09:39:49.136 INFO:teuthology.orchestra.run.smithi005.stdout: "leader": 24542, 2024-09-10T09:39:49.136 INFO:teuthology.orchestra.run.smithi005.stdout: "set_version": 6, 2024-09-10T09:39:49.136 INFO:teuthology.orchestra.run.smithi005.stdout: "sets": { 2024-09-10T09:39:49.137 INFO:teuthology.orchestra.run.smithi005.stdout: "f9cf197c": { 2024-09-10T09:39:49.137 INFO:teuthology.orchestra.run.smithi005.stdout: "version": 6, 2024-09-10T09:39:49.137 INFO:teuthology.orchestra.run.smithi005.stdout: "age_ref": 0.0, 2024-09-10T09:39:49.137 INFO:teuthology.orchestra.run.smithi005.stdout: "state": { 2024-09-10T09:39:49.137 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "QUIESCED", 2024-09-10T09:39:49.137 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0 2024-09-10T09:39:49.137 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T09:39:49.137 INFO:teuthology.orchestra.run.smithi005.stdout: "timeout": 90.0, 2024-09-10T09:39:49.137 INFO:teuthology.orchestra.run.smithi005.stdout: "expiration": 150.0, 2024-09-10T09:39:49.137 INFO:teuthology.orchestra.run.smithi005.stdout: "members": { 2024-09-10T09:39:49.137 INFO:teuthology.orchestra.run.smithi005.stdout: "file:/": { 2024-09-10T09:39:49.137 INFO:teuthology.orchestra.run.smithi005.stdout: "excluded": false, 2024-09-10T09:39:49.138 INFO:teuthology.orchestra.run.smithi005.stdout: "state": { 2024-09-10T09:39:49.138 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "QUIESCED", 2024-09-10T09:39:49.138 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0 2024-09-10T09:39:49.138 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T09:39:49.138 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T09:39:49.138 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T09:39:49.138 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T09:39:49.138 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T09:39:49.138 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-10T09:39:49.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:49.133+0000 7f9947200640 1 -- 172.21.15.5:0/1857117217 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f992c0a2c00 msgr2=0x7f992c0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:39:49.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:49.133+0000 7f9947200640 1 --2- 172.21.15.5:0/1857117217 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f992c0a2c00 0x7f992c0a50c0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f9938098020 tx=0x7f99380d5530 comp rx=0 tx=0).stop 2024-09-10T09:39:49.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:49.133+0000 7f9947200640 1 -- 172.21.15.5:0/1857117217 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9948074510 msgr2=0x7f994819a4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:39:49.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:49.133+0000 7f9947200640 1 --2- 172.21.15.5:0/1857117217 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9948074510 0x7f994819a4e0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f993800e9f0 tx=0x7f993800eeb0 comp rx=0 tx=0).stop 2024-09-10T09:39:49.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:49.134+0000 7f9947200640 1 -- 172.21.15.5:0/1857117217 shutdown_connections 2024-09-10T09:39:49.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:49.134+0000 7f9947200640 1 --2- 172.21.15.5:0/1857117217 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f992c0a2c00 0x7f992c0a50c0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:49.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:49.134+0000 7f9947200640 1 --2- 172.21.15.5:0/1857117217 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f994810e280 0x7f994819edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:49.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:49.134+0000 7f9947200640 1 --2- 172.21.15.5:0/1857117217 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9948074e30 0x7f994819aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:49.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:49.134+0000 7f9947200640 1 --2- 172.21.15.5:0/1857117217 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9948074510 0x7f994819a4e0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:49.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:49.134+0000 7f9947200640 1 -- 172.21.15.5:0/1857117217 >> 172.21.15.5:0/1857117217 conn(0x7f99480fd740 msgr2=0x7f99480fe3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:39:49.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:49.134+0000 7f9947200640 1 -- 172.21.15.5:0/1857117217 shutdown_connections 2024-09-10T09:39:49.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:49.134+0000 7f9947200640 1 -- 172.21.15.5:0/1857117217 wait complete. 2024-09-10T09:39:50.137 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'f9cf197c', quiesce took 4.5 seconds. Will release after: 25.5 2024-09-10T09:39:50.248 INFO:tasks.workunit.client.0.smithi005.stderr:Updating files: 56% (7177/12697) Updating files: 57% (7238/12697) Updating files: 58% (7365/12697) Updating files: 59% (7492/12697) Updating files: 60% (7619/12697) Updating files: 61% (7746/12697) Updating files: 62% (7873/12697) Updating files: 63% (8000/12697) Updating files: 64% (8127/12697) Updating files: 65% (8254/12697) Updating files: 66% (8381/12697) Updating files: 67% (8507/12697) Updating files: 68% (8634/12697) Updating files: 69% (8761/12697) Updating files: 70% (8888/12697) Updating files: 71% (9015/12697) Updating files: 72% (9142/12697) Updating files: 73% (9269/12697) Updating files: 74% (9396/12697) Updating files: 75% (9523/12697) Updating files: 76% (9650/12697) Updating files: 77% (9777/12697) Updating files: 78% (9904/12697) Updating files: 79% (10031/12697) Updating files: 80% (10158/12697) Updating files: 81% (10285/12697) Updating files: 82% (10412/12697) Updating files: 83% (10539/12697) Updating files: 84% (10666/12697) Updating files: 85% (10793/12697) Updating files: 86% (10920/12697) Updating files: 87% (11047/12697) Updating files: 88% (11174/12697) Updating files: 89% (11301/12697) Updating files: 90% (11428/12697) Updating files: 91% (11555/12697) Updating files: 92% (11682/12697) Updating files: 92% (11714/12697) Updating files: 93% (11809/12697) Updating files: 94% (11936/12697) Updating files: 95% (12063/12697) Updating files: 96% (12190/12697) Updating files: 97% (12317/12697) Updating files: 98% (12444/12697) Updating files: 99% (12571/12697) Updating files: 100% (12697/12697) Updating files: 100% (12697/12697), done. 2024-09-10T09:39:50.918 INFO:tasks.workunit.client.0.smithi005.stderr:Note: switching to '1cf819b7212a6048e92fc78c4eff963344f6d56b'. 2024-09-10T09:39:50.918 INFO:tasks.workunit.client.0.smithi005.stderr: 2024-09-10T09:39:50.918 INFO:tasks.workunit.client.0.smithi005.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-09-10T09:39:50.918 INFO:tasks.workunit.client.0.smithi005.stderr:changes and commit them, and you can discard any commits you make in this 2024-09-10T09:39:50.918 INFO:tasks.workunit.client.0.smithi005.stderr:state without impacting any branches by switching back to a branch. 2024-09-10T09:39:50.918 INFO:tasks.workunit.client.0.smithi005.stderr: 2024-09-10T09:39:50.919 INFO:tasks.workunit.client.0.smithi005.stderr:If you want to create a new branch to retain commits you create, you may 2024-09-10T09:39:50.919 INFO:tasks.workunit.client.0.smithi005.stderr:do so (now or later) by using -c with the switch command. Example: 2024-09-10T09:39:50.919 INFO:tasks.workunit.client.0.smithi005.stderr: 2024-09-10T09:39:50.919 INFO:tasks.workunit.client.0.smithi005.stderr: git switch -c 2024-09-10T09:39:50.919 INFO:tasks.workunit.client.0.smithi005.stderr: 2024-09-10T09:39:50.919 INFO:tasks.workunit.client.0.smithi005.stderr:Or undo this operation with: 2024-09-10T09:39:50.919 INFO:tasks.workunit.client.0.smithi005.stderr: 2024-09-10T09:39:50.919 INFO:tasks.workunit.client.0.smithi005.stderr: git switch - 2024-09-10T09:39:50.919 INFO:tasks.workunit.client.0.smithi005.stderr: 2024-09-10T09:39:50.919 INFO:tasks.workunit.client.0.smithi005.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-09-10T09:39:50.919 INFO:tasks.workunit.client.0.smithi005.stderr: 2024-09-10T09:39:50.919 INFO:tasks.workunit.client.0.smithi005.stderr:HEAD is now at 1cf819b7212 Merge PR #59672 into wip-rishabh-testing-20240909.165402-squid 2024-09-10T09:39:50.933 DEBUG:teuthology.orchestra.run.smithi005:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-09-10T09:39:51.021 INFO:tasks.workunit.client.0.smithi005.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-09-10T09:39:51.026 INFO:tasks.workunit.client.0.smithi005.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-09-10T09:39:51.026 INFO:tasks.workunit.client.0.smithi005.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-09-10T09:39:51.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:39:50 smithi005 ceph-mon[30350]: pgmap v438: 129 pgs: 129 active+clean; 666 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2024-09-10T09:39:51.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:39:50 smithi159 ceph-mon[35513]: pgmap v438: 129 pgs: 129 active+clean; 666 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2024-09-10T09:39:51.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:39:50 smithi102 ceph-mon[35442]: pgmap v438: 129 pgs: 129 active+clean; 666 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2024-09-10T09:39:51.984 INFO:tasks.workunit.client.0.smithi005.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-09-10T09:39:52.097 INFO:tasks.workunit.client.0.smithi005.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-09-10T09:39:52.105 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T09:39:52.165 INFO:tasks.workunit.client.0.smithi005.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-09-10T09:39:52.169 INFO:tasks.workunit.client.0.smithi005.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-09-10T09:39:52.169 INFO:tasks.workunit.client.0.smithi005.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-09-10T09:39:52.235 INFO:tasks.workunit.client.0.smithi005.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-09-10T09:39:52.242 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-09-10T09:39:52.242 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-09-10T09:39:52.278 INFO:tasks.workunit:Running workunits matching suites/dbench.sh on client.0... 2024-09-10T09:39:52.279 INFO:tasks.workunit:Running workunit suites/dbench.sh... 2024-09-10T09:39:52.280 DEBUG:teuthology.orchestra.run.smithi005:workunit test suites/dbench.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=1cf819b7212a6048e92fc78c4eff963344f6d56b TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/dbench.sh 2024-09-10T09:39:52.488 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:39:52.997 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:39:52 smithi005 ceph-mon[30350]: pgmap v439: 129 pgs: 129 active+clean; 666 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 255 B/s wr, 12 op/s 2024-09-10T09:39:53.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:39:52 smithi159 ceph-mon[35513]: pgmap v439: 129 pgs: 129 active+clean; 666 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 255 B/s wr, 12 op/s 2024-09-10T09:39:53.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:53.109+0000 7f7638400640 1 -- 172.21.15.5:0/1982674440 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7630112b90 msgr2=0x7f7630114f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:39:53.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:53.109+0000 7f7638400640 1 --2- 172.21.15.5:0/1982674440 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7630112b90 0x7f7630114f80 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f7628009ca0 tx=0x7f762802a950 comp rx=0 tx=0).stop 2024-09-10T09:39:53.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:53.110+0000 7f7638400640 1 -- 172.21.15.5:0/1982674440 shutdown_connections 2024-09-10T09:39:53.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:53.110+0000 7f7638400640 1 --2- 172.21.15.5:0/1982674440 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7630112b90 0x7f7630114f80 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:53.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:53.110+0000 7f7638400640 1 --2- 172.21.15.5:0/1982674440 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7630074eb0 0x7f7630112650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:53.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:53.110+0000 7f7638400640 1 --2- 172.21.15.5:0/1982674440 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7630074590 0x7f7630074970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:53.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:53.110+0000 7f7638400640 1 -- 172.21.15.5:0/1982674440 >> 172.21.15.5:0/1982674440 conn(0x7f76300fd8c0 msgr2=0x7f76300ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:39:53.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:53.110+0000 7f7638400640 1 -- 172.21.15.5:0/1982674440 shutdown_connections 2024-09-10T09:39:53.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:53.110+0000 7f7638400640 1 -- 172.21.15.5:0/1982674440 wait complete. 2024-09-10T09:39:53.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:53.111+0000 7f7638400640 1 Processor -- start 2024-09-10T09:39:53.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:53.111+0000 7f7638400640 1 -- start start 2024-09-10T09:39:53.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:53.111+0000 7f7638400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7630074590 0x7f76301a46b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:39:53.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:53.111+0000 7f7638400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7630074eb0 0x7f76301a4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:39:53.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:53.111+0000 7f7638400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7630112b90 0x7f763019e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:39:53.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:53.111+0000 7f7638400640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f76301185a0 con 0x7f7630112b90 2024-09-10T09:39:53.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:53.111+0000 7f7638400640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f7630118420 con 0x7f7630074eb0 2024-09-10T09:39:53.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:53.111+0000 7f7638400640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f7630118720 con 0x7f7630074590 2024-09-10T09:39:53.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:53.111+0000 7f7635000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7630074eb0 0x7f76301a4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:39:53.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:53.111+0000 7f7635a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7630074590 0x7f76301a46b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:39:53.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:53.111+0000 7f7635a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7630074590 0x7f76301a46b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:53812/0 (socket says 172.21.15.5:53812) 2024-09-10T09:39:53.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:53.111+0000 7f7635a00640 1 -- 172.21.15.5:0/1524664519 learned_addr learned my addr 172.21.15.5:0/1524664519 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:39:53.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:53.111+0000 7f7636400640 1 --2- 172.21.15.5:0/1524664519 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7630112b90 0x7f763019e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:39:53.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:53.112+0000 7f7635a00640 1 -- 172.21.15.5:0/1524664519 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7630074eb0 msgr2=0x7f76301a4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:39:53.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:53.112+0000 7f7635a00640 1 --2- 172.21.15.5:0/1524664519 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7630074eb0 0x7f76301a4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:53.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:53.112+0000 7f7635a00640 1 -- 172.21.15.5:0/1524664519 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7630112b90 msgr2=0x7f763019e870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:39:53.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:53.112+0000 7f7635a00640 1 --2- 172.21.15.5:0/1524664519 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7630112b90 0x7f763019e870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:53.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:53.112+0000 7f7635a00640 1 -- 172.21.15.5:0/1524664519 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f763019efa0 con 0x7f7630074590 2024-09-10T09:39:53.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:53.112+0000 7f7635000640 1 --2- 172.21.15.5:0/1524664519 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7630074eb0 0x7f76301a4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:39:53.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:53.112+0000 7f7636400640 1 --2- 172.21.15.5:0/1524664519 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7630112b90 0x7f763019e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:39:53.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:53.112+0000 7f7635a00640 1 --2- 172.21.15.5:0/1524664519 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7630074590 0x7f76301a46b0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f762400d950 tx=0x7f762400de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:39:53.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:53.112+0000 7f762e000640 1 -- 172.21.15.5:0/1524664519 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7624017070 con 0x7f7630074590 2024-09-10T09:39:53.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:53.112+0000 7f7638400640 1 -- 172.21.15.5:0/1524664519 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f763019f290 con 0x7f7630074590 2024-09-10T09:39:53.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:53.112+0000 7f762e000640 1 -- 172.21.15.5:0/1524664519 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7624009c30 con 0x7f7630074590 2024-09-10T09:39:53.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:53.112+0000 7f762e000640 1 -- 172.21.15.5:0/1524664519 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f762400f6b0 con 0x7f7630074590 2024-09-10T09:39:53.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:53.113+0000 7f7638400640 1 -- 172.21.15.5:0/1524664519 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76301ab580 con 0x7f7630074590 2024-09-10T09:39:53.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:53.115+0000 7f763ae2d740 1 Processor -- start 2024-09-10T09:39:53.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:53.116+0000 7f763ae2d740 1 -- start start 2024-09-10T09:39:53.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:53.116+0000 7f762e000640 1 -- 172.21.15.5:0/1524664519 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f76240109d0 con 0x7f7630074590 2024-09-10T09:39:53.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:53.116+0000 7f763ae2d740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fdaa21db90 0x55fdaa21df70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:39:53.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:53.116+0000 7f763ae2d740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fdaa21e4b0 0x55fdaa228830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:39:53.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:53.116+0000 7f763ae2d740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fdaa228d70 0x55fdaa22b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:39:53.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:53.116+0000 7f763ae2d740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55fdaa1843e0 con 0x55fdaa21db90 2024-09-10T09:39:53.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:53.116+0000 7f763ae2d740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55fdaa12bd80 con 0x55fdaa21e4b0 2024-09-10T09:39:53.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:53.116+0000 7f763ae2d740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55fdaa1803a0 con 0x55fdaa228d70 2024-09-10T09:39:53.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:53.116+0000 7f7636400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fdaa21db90 0x55fdaa21df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:39:53.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:53.116+0000 7f7636400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fdaa21db90 0x55fdaa21df70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33984/0 (socket says 172.21.15.5:33984) 2024-09-10T09:39:53.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:53.116+0000 7f7636400640 1 -- 172.21.15.5:0/3951533001 learned_addr learned my addr 172.21.15.5:0/3951533001 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:39:53.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:53.116+0000 7f7635a00640 1 --2- 172.21.15.5:0/3951533001 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fdaa21e4b0 0x55fdaa228830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:39:53.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:53.116+0000 7f7635000640 1 --2- 172.21.15.5:0/3951533001 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fdaa228d70 0x55fdaa22b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:39:53.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:53.117+0000 7f7636400640 1 -- 172.21.15.5:0/3951533001 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fdaa228d70 msgr2=0x55fdaa22b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:39:53.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:53.117+0000 7f7636400640 1 --2- 172.21.15.5:0/3951533001 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fdaa228d70 0x55fdaa22b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:53.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:53.117+0000 7f7636400640 1 -- 172.21.15.5:0/3951533001 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fdaa21e4b0 msgr2=0x55fdaa228830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:39:53.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:53.117+0000 7f7636400640 1 --2- 172.21.15.5:0/3951533001 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fdaa21e4b0 0x55fdaa228830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:53.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:53.117+0000 7f7636400640 1 -- 172.21.15.5:0/3951533001 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fdaa14aae0 con 0x55fdaa21db90 2024-09-10T09:39:53.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:53.117+0000 7f7636400640 1 --2- 172.21.15.5:0/3951533001 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fdaa21db90 0x55fdaa21df70 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f7630076600 tx=0x7f762802f750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cb244f4e4d3a9af server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:39:53.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:53.117+0000 7f7622a00640 1 -- 172.21.15.5:0/3951533001 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7628048070 con 0x55fdaa21db90 2024-09-10T09:39:53.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:53.117+0000 7f7622a00640 1 -- 172.21.15.5:0/3951533001 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7628003e50 con 0x55fdaa21db90 2024-09-10T09:39:53.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:53.117+0000 7f762e000640 1 --2- 172.21.15.5:0/1524664519 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f76180a2c00 0x7f76180a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:39:53.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:53.117+0000 7f7622a00640 1 -- 172.21.15.5:0/3951533001 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7628034040 con 0x55fdaa21db90 2024-09-10T09:39:53.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:53.118+0000 7f762e000640 1 -- 172.21.15.5:0/1524664519 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f76240cefe0 con 0x7f7630074590 2024-09-10T09:39:53.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:53.118+0000 7f7635a00640 1 --2- 172.21.15.5:0/1524664519 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f76180a2c00 0x7f76180a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:39:53.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:53.118+0000 7f763ae2d740 1 -- 172.21.15.5:0/3951533001 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fdaa21db90 msgr2=0x55fdaa21df70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:39:53.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:53.118+0000 7f763ae2d740 1 --2- 172.21.15.5:0/3951533001 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fdaa21db90 0x55fdaa21df70 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f7630076600 tx=0x7f762802f750 comp rx=0 tx=0).stop 2024-09-10T09:39:53.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:53.118+0000 7f763ae2d740 1 -- 172.21.15.5:0/3951533001 shutdown_connections 2024-09-10T09:39:53.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:53.118+0000 7f763ae2d740 1 --2- 172.21.15.5:0/3951533001 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fdaa228d70 0x55fdaa22b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:53.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:53.118+0000 7f763ae2d740 1 --2- 172.21.15.5:0/3951533001 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fdaa21e4b0 0x55fdaa228830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:53.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:53.118+0000 7f763ae2d740 1 --2- 172.21.15.5:0/3951533001 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fdaa21db90 0x55fdaa21df70 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:53.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:53.118+0000 7f763ae2d740 1 -- 172.21.15.5:0/3951533001 >> 172.21.15.5:0/3951533001 conn(0x55fdaa21a120 msgr2=0x55fdaa21a500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:39:53.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:53.118+0000 7f763ae2d740 1 -- 172.21.15.5:0/3951533001 shutdown_connections 2024-09-10T09:39:53.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:53.118+0000 7f7635a00640 1 --2- 172.21.15.5:0/1524664519 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f76180a2c00 0x7f76180a50c0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f7624097900 tx=0x7f76240d2a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:39:53.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:53.118+0000 7f763ae2d740 1 -- 172.21.15.5:0/3951533001 wait complete. 2024-09-10T09:39:53.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:53.119+0000 7f763ae2d740 1 -- start start 2024-09-10T09:39:53.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:53.119+0000 7f763ae2d740 1 Processor -- start 2024-09-10T09:39:53.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:53.119+0000 7f763ae2d740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fdaa21db90 0x55fdaa225cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:39:53.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:53.119+0000 7f763ae2d740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fdaa21e4b0 0x55fdaa226210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:39:53.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:53.119+0000 7f763ae2d740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fdaa228d70 0x55fdaa226750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:39:53.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:53.119+0000 7f763ae2d740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55fdaa22b9e0 con 0x55fdaa228d70 2024-09-10T09:39:53.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:53.119+0000 7f763ae2d740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55fdaa13f0f0 con 0x55fdaa21db90 2024-09-10T09:39:53.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:53.119+0000 7f763ae2d740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55fdaa22bb60 con 0x55fdaa21e4b0 2024-09-10T09:39:53.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:53.119+0000 7f763ae2d740 10 client.0 _ll_register_callbacks cb 0x55fdaa1309e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:39:53.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:53.120+0000 7f7636400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fdaa228d70 0x55fdaa226750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:39:53.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:53.120+0000 7f7636400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fdaa228d70 0x55fdaa226750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33986/0 (socket says 172.21.15.5:33986) 2024-09-10T09:39:53.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:53.120+0000 7f7636400640 1 -- 172.21.15.5:0/2781477530 learned_addr learned my addr 172.21.15.5:0/2781477530 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:39:53.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:53.120+0000 7f7635000640 1 --2- 172.21.15.5:0/2781477530 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fdaa21e4b0 0x55fdaa226210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:39:53.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:53.120+0000 7f7636400640 1 --2- 172.21.15.5:0/2781477530 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fdaa21db90 0x55fdaa225cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:39:53.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:53.120+0000 7f7636400640 1 -- 172.21.15.5:0/2781477530 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fdaa21e4b0 msgr2=0x55fdaa226210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:39:53.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:53.120+0000 7f7636400640 1 --2- 172.21.15.5:0/2781477530 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fdaa21e4b0 0x55fdaa226210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:53.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:53.120+0000 7f7636400640 1 -- 172.21.15.5:0/2781477530 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fdaa21db90 msgr2=0x55fdaa225cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:39:53.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:53.120+0000 7f7636400640 1 --2- 172.21.15.5:0/2781477530 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fdaa21db90 0x55fdaa225cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:53.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:53.120+0000 7f7636400640 1 -- 172.21.15.5:0/2781477530 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fda9ef53c0 con 0x55fdaa228d70 2024-09-10T09:39:53.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:53.120+0000 7f7636400640 1 --2- 172.21.15.5:0/2781477530 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fdaa228d70 0x55fdaa226750 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f762802af40 tx=0x7f7628004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfc6413c22b44704 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:39:53.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:53.121+0000 7f7617400640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-10T09:39:53.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:53.121+0000 7f7617400640 1 -- 172.21.15.5:0/2781477530 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7628048040 con 0x55fdaa228d70 2024-09-10T09:39:53.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:53.121+0000 7f763ae2d740 1 -- 172.21.15.5:0/2781477530 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7628003e50 con 0x55fdaa228d70 2024-09-10T09:39:53.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:53.121+0000 7f7617400640 1 -- 172.21.15.5:0/2781477530 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f7628004420 con 0x55fdaa228d70 2024-09-10T09:39:53.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:53.121+0000 7f7617400640 1 -- 172.21.15.5:0/2781477530 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f762805a050 con 0x55fdaa228d70 2024-09-10T09:39:53.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:53.121+0000 7f763ae2d740 10 client.15654 fetch_fsmap learned FSMap version 30 2024-09-10T09:39:53.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:53.121+0000 7f763ae2d740 1 -- 172.21.15.5:0/2781477530 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55fdaa237710 con 0x55fdaa228d70 2024-09-10T09:39:53.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:53.121+0000 7f7617400640 1 -- 172.21.15.5:0/2781477530 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7628003e50 con 0x55fdaa228d70 2024-09-10T09:39:53.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:53.122+0000 7f7617400640 1 -- 172.21.15.5:0/2781477530 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f762804e040 con 0x55fdaa228d70 2024-09-10T09:39:53.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:53.122+0000 7f763ae2d740 10 client.15654 fetch_fsmap finished waiting for FSMap version 30 2024-09-10T09:39:53.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:53.122+0000 7f763ae2d740 10 client.15654 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T09:39:53.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:53.122+0000 7f763ae2d740 20 client.15654 populate_metadata read hostname 'smithi005' 2024-09-10T09:39:53.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:53.122+0000 7f763ae2d740 1 --2- 172.21.15.5:0/2781477530 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55fdaa238a60 0x55fdaa23ae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:39:53.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:53.122+0000 7f763ae2d740 4 client.15654 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:39:53.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:53.122+0000 7f763ae2d740 1 -- 172.21.15.5:0/2781477530 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fdaa191950 con 0x55fdaa238a60 2024-09-10T09:39:53.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:53.122+0000 7f7635000640 1 --2- 172.21.15.5:0/2781477530 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55fdaa238a60 0x55fdaa23ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:39:53.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:53.123+0000 7f7635000640 1 --2- 172.21.15.5:0/2781477530 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55fdaa238a60 0x55fdaa23ae50 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8bead5699d1a920 server_cookie=f0ae99e96b3ec303 in_seq=0 out_seq=0 2024-09-10T09:39:53.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:53.123+0000 7f7617400640 10 client.15654 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T09:39:53.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:53.123+0000 7f7617400640 1 -- 172.21.15.5:0/2781477530 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55fdaa191950 con 0x55fdaa238a60 2024-09-10T09:39:53.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:53.123+0000 7f7617400640 10 client.15654 handle_command_reply: tid=0 2024-09-10T09:39:53.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:53.124+0000 7f763ae2d740 1 client.15654 shutdown 2024-09-10T09:39:53.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:53.124+0000 7f763ae2d740 1 -- 172.21.15.5:0/2781477530 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fdaa228d70 msgr2=0x55fdaa226750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:39:53.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:53.124+0000 7f763ae2d740 1 --2- 172.21.15.5:0/2781477530 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fdaa228d70 0x55fdaa226750 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f762802af40 tx=0x7f7628004040 comp rx=0 tx=0).stop 2024-09-10T09:39:53.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:53.124+0000 7f763ae2d740 1 -- 172.21.15.5:0/2781477530 shutdown_connections 2024-09-10T09:39:53.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:53.124+0000 7f763ae2d740 1 --2- 172.21.15.5:0/2781477530 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55fdaa238a60 0x55fdaa23ae50 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:53.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:53.124+0000 7f763ae2d740 1 --2- 172.21.15.5:0/2781477530 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fdaa228d70 0x55fdaa226750 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:53.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:53.124+0000 7f763ae2d740 1 --2- 172.21.15.5:0/2781477530 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fdaa21e4b0 0x55fdaa226210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:53.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:53.124+0000 7f763ae2d740 1 --2- 172.21.15.5:0/2781477530 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fdaa21db90 0x55fdaa225cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:53.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:53.124+0000 7f7617400640 0 client.15654 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T09:39:53.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:53.124+0000 7f763ae2d740 1 -- 172.21.15.5:0/2781477530 >> 172.21.15.5:0/2781477530 conn(0x55fdaa21a120 msgr2=0x55fdaa21fe50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:39:53.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:53.124+0000 7f763ae2d740 1 -- 172.21.15.5:0/2781477530 shutdown_connections 2024-09-10T09:39:53.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:53.124+0000 7f763ae2d740 1 -- 172.21.15.5:0/2781477530 wait complete. 2024-09-10T09:39:53.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:53.124+0000 7f763ae2d740 20 client.15654 trim_cache size 0 max 16384 2024-09-10T09:39:53.146 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:39:52 smithi102 ceph-mon[35442]: pgmap v439: 129 pgs: 129 active+clean; 666 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 255 B/s wr, 12 op/s 2024-09-10T09:39:53.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:53.142+0000 7f763ae2d740 1 Processor -- start 2024-09-10T09:39:53.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:53.142+0000 7f763ae2d740 1 -- start start 2024-09-10T09:39:53.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:53.142+0000 7f763ae2d740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fdaa21db90 0x55fdaa2403f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:39:53.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:53.142+0000 7f763ae2d740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fdaa21e4b0 0x55fdaa240930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:39:53.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:53.142+0000 7f763ae2d740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fdaa228d70 0x55fdaa222f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:39:53.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:53.142+0000 7f763ae2d740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f76301b5430 con 0x55fdaa21db90 2024-09-10T09:39:53.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:53.142+0000 7f763ae2d740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f76301b6f70 con 0x55fdaa228d70 2024-09-10T09:39:53.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:53.142+0000 7f763ae2d740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f76301b5b00 con 0x55fdaa21e4b0 2024-09-10T09:39:53.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:53.143+0000 7f7636400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fdaa21db90 0x55fdaa2403f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:39:53.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:53.143+0000 7f7636400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fdaa21db90 0x55fdaa2403f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33992/0 (socket says 172.21.15.5:33992) 2024-09-10T09:39:53.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:53.143+0000 7f7636400640 1 -- 172.21.15.5:0/2271711801 learned_addr learned my addr 172.21.15.5:0/2271711801 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:39:53.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:53.143+0000 7f7635000640 1 --2- 172.21.15.5:0/2271711801 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fdaa21e4b0 0x55fdaa240930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:39:53.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:53.143+0000 7f7636400640 1 -- 172.21.15.5:0/2271711801 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fdaa21e4b0 msgr2=0x55fdaa240930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:39:53.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:53.143+0000 7f7636400640 1 --2- 172.21.15.5:0/2271711801 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fdaa21e4b0 0x55fdaa240930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:53.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:53.143+0000 7f7636400640 1 -- 172.21.15.5:0/2271711801 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fdaa228d70 msgr2=0x55fdaa222f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:39:53.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:53.143+0000 7f7636400640 1 --2- 172.21.15.5:0/2271711801 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fdaa228d70 0x55fdaa222f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:53.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:53.143+0000 7f7636400640 1 -- 172.21.15.5:0/2271711801 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fdaa21d9c0 con 0x55fdaa21db90 2024-09-10T09:39:53.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:53.143+0000 7f7636400640 1 --2- 172.21.15.5:0/2271711801 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fdaa21db90 0x55fdaa2403f0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f7628004aa0 tx=0x7f7628005d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7812a7924153d0b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:39:53.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:53.144+0000 7f7622000640 1 -- 172.21.15.5:0/2271711801 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7628048070 con 0x55fdaa21db90 2024-09-10T09:39:53.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:53.144+0000 7f7622000640 1 -- 172.21.15.5:0/2271711801 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f762802f750 con 0x55fdaa21db90 2024-09-10T09:39:53.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:53.144+0000 7f7622000640 1 -- 172.21.15.5:0/2271711801 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7628034070 con 0x55fdaa21db90 2024-09-10T09:39:53.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:53.144+0000 7f763ae2d740 1 -- 172.21.15.5:0/2271711801 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fdaa21db90 msgr2=0x55fdaa2403f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:39:53.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:53.144+0000 7f763ae2d740 1 --2- 172.21.15.5:0/2271711801 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fdaa21db90 0x55fdaa2403f0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f7628004aa0 tx=0x7f7628005d80 comp rx=0 tx=0).stop 2024-09-10T09:39:53.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:53.144+0000 7f763ae2d740 1 -- 172.21.15.5:0/2271711801 shutdown_connections 2024-09-10T09:39:53.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:53.144+0000 7f763ae2d740 1 --2- 172.21.15.5:0/2271711801 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fdaa228d70 0x55fdaa222f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:53.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:53.144+0000 7f763ae2d740 1 --2- 172.21.15.5:0/2271711801 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fdaa21e4b0 0x55fdaa240930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:53.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:53.144+0000 7f763ae2d740 1 --2- 172.21.15.5:0/2271711801 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fdaa21db90 0x55fdaa2403f0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:53.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:53.144+0000 7f763ae2d740 1 -- 172.21.15.5:0/2271711801 >> 172.21.15.5:0/2271711801 conn(0x55fdaa21a120 msgr2=0x55fdaa2245a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:39:53.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:53.144+0000 7f763ae2d740 1 -- 172.21.15.5:0/2271711801 shutdown_connections 2024-09-10T09:39:53.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:53.144+0000 7f763ae2d740 1 -- 172.21.15.5:0/2271711801 wait complete. 2024-09-10T09:39:53.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:53.145+0000 7f763ae2d740 1 -- start start 2024-09-10T09:39:53.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:53.145+0000 7f763ae2d740 1 Processor -- start 2024-09-10T09:39:53.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:53.145+0000 7f763ae2d740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fdaa21db90 0x55fdaa230db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:39:53.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:53.145+0000 7f763ae2d740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fdaa21e4b0 0x55fdaa2312f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:39:53.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:53.145+0000 7f763ae2d740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fdaa228d70 0x55fdaa22dd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:39:53.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:53.145+0000 7f763ae2d740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f76301b68a0 con 0x55fdaa21db90 2024-09-10T09:39:53.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:53.145+0000 7f763ae2d740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f76301b61d0 con 0x55fdaa21e4b0 2024-09-10T09:39:53.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:53.145+0000 7f763ae2d740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f76301b3410 con 0x55fdaa228d70 2024-09-10T09:39:53.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:53.145+0000 7f763ae2d740 10 client.0 _ll_register_callbacks cb 0x7f761c009b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:39:53.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:53.146+0000 7f7636400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fdaa21db90 0x55fdaa230db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:39:53.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:53.146+0000 7f7636400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fdaa21db90 0x55fdaa230db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34008/0 (socket says 172.21.15.5:34008) 2024-09-10T09:39:53.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:53.146+0000 7f7636400640 1 -- 172.21.15.5:0/3339238104 learned_addr learned my addr 172.21.15.5:0/3339238104 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:39:53.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:53.146+0000 7f7635000640 1 --2- 172.21.15.5:0/3339238104 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fdaa21e4b0 0x55fdaa2312f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:39:53.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:53.146+0000 7f7636400640 1 --2- 172.21.15.5:0/3339238104 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fdaa228d70 0x55fdaa22dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:39:53.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:53.146+0000 7f7636400640 1 -- 172.21.15.5:0/3339238104 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fdaa228d70 msgr2=0x55fdaa22dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:39:53.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:53.146+0000 7f7636400640 1 --2- 172.21.15.5:0/3339238104 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fdaa228d70 0x55fdaa22dd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:53.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:53.146+0000 7f7636400640 1 -- 172.21.15.5:0/3339238104 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fdaa21e4b0 msgr2=0x55fdaa2312f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:39:53.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:53.146+0000 7f7636400640 1 --2- 172.21.15.5:0/3339238104 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fdaa21e4b0 0x55fdaa2312f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:53.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:53.146+0000 7f7636400640 1 -- 172.21.15.5:0/3339238104 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fdaa21adb0 con 0x55fdaa21db90 2024-09-10T09:39:53.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:53.147+0000 7f7636400640 1 --2- 172.21.15.5:0/3339238104 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fdaa21db90 0x55fdaa230db0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f7628009d90 tx=0x7f7628004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42f33b32519afdb9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:39:53.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:53.147+0000 7f7617400640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-10T09:39:53.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:53.147+0000 7f7617400640 1 -- 172.21.15.5:0/3339238104 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7628065060 con 0x55fdaa21db90 2024-09-10T09:39:53.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:53.147+0000 7f7617400640 1 -- 172.21.15.5:0/3339238104 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f7628004120 con 0x55fdaa21db90 2024-09-10T09:39:53.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:53.147+0000 7f763ae2d740 1 -- 172.21.15.5:0/3339238104 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55fdaa18be60 con 0x55fdaa21db90 2024-09-10T09:39:53.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:53.147+0000 7f7617400640 1 -- 172.21.15.5:0/3339238104 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f7628048070 con 0x55fdaa21db90 2024-09-10T09:39:53.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:53.147+0000 7f763ae2d740 10 client.15660 fetch_fsmap learned FSMap version 30 2024-09-10T09:39:53.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:53.147+0000 7f763ae2d740 1 -- 172.21.15.5:0/3339238104 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55fdaa2a39b0 con 0x55fdaa21db90 2024-09-10T09:39:53.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:53.147+0000 7f7617400640 1 -- 172.21.15.5:0/3339238104 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55fdaa18be60 con 0x55fdaa21db90 2024-09-10T09:39:53.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:53.148+0000 7f7617400640 1 -- 172.21.15.5:0/3339238104 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f762804e040 con 0x55fdaa21db90 2024-09-10T09:39:53.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:53.148+0000 7f763ae2d740 10 client.15660 fetch_fsmap finished waiting for FSMap version 30 2024-09-10T09:39:53.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:53.148+0000 7f763ae2d740 10 client.15660 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T09:39:53.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:53.148+0000 7f763ae2d740 20 client.15660 populate_metadata read hostname 'smithi005' 2024-09-10T09:39:53.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:53.148+0000 7f763ae2d740 1 --2- 172.21.15.5:0/3339238104 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55fdaa2a3e20 0x55fdaa2a6210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:39:53.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:53.148+0000 7f763ae2d740 4 client.15660 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T09:39:53.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:53.148+0000 7f763ae2d740 1 -- 172.21.15.5:0/3339238104 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55fdaa1080c0 con 0x55fdaa2a3e20 2024-09-10T09:39:53.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:53.148+0000 7f7635000640 1 --2- 172.21.15.5:0/3339238104 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55fdaa2a3e20 0x55fdaa2a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:39:53.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:53.149+0000 7f7635000640 1 --2- 172.21.15.5:0/3339238104 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55fdaa2a3e20 0x55fdaa2a6210 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4a87afc2fa2aea6a server_cookie=b32261a8dfec7aca in_seq=0 out_seq=0 2024-09-10T09:39:53.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:53.149+0000 7f7617400640 10 client.15660 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T09:39:53.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:53.149+0000 7f7617400640 1 -- 172.21.15.5:0/3339238104 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55fdaa1080c0 con 0x55fdaa2a3e20 2024-09-10T09:39:53.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:53.149+0000 7f7617400640 10 client.15660 handle_command_reply: tid=0 2024-09-10T09:39:53.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:53.149+0000 7f763ae2d740 1 client.15660 shutdown 2024-09-10T09:39:53.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:53.149+0000 7f763ae2d740 1 -- 172.21.15.5:0/3339238104 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fdaa21db90 msgr2=0x55fdaa230db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:39:53.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:53.149+0000 7f763ae2d740 1 --2- 172.21.15.5:0/3339238104 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fdaa21db90 0x55fdaa230db0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f7628009d90 tx=0x7f7628004b40 comp rx=0 tx=0).stop 2024-09-10T09:39:53.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:53.149+0000 7f763ae2d740 1 -- 172.21.15.5:0/3339238104 shutdown_connections 2024-09-10T09:39:53.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:53.149+0000 7f763ae2d740 1 --2- 172.21.15.5:0/3339238104 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55fdaa2a3e20 0x55fdaa2a6210 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:53.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:53.149+0000 7f763ae2d740 1 --2- 172.21.15.5:0/3339238104 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fdaa228d70 0x55fdaa22dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:53.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:53.149+0000 7f763ae2d740 1 --2- 172.21.15.5:0/3339238104 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fdaa21e4b0 0x55fdaa2312f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:53.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:53.149+0000 7f763ae2d740 1 --2- 172.21.15.5:0/3339238104 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fdaa21db90 0x55fdaa230db0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:53.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:53.149+0000 7f763ae2d740 1 -- 172.21.15.5:0/3339238104 >> 172.21.15.5:0/3339238104 conn(0x55fdaa21a120 msgr2=0x55fdaa21f550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:39:53.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:53.149+0000 7f7617400640 0 client.15660 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T09:39:53.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:53.149+0000 7f763ae2d740 1 -- 172.21.15.5:0/3339238104 shutdown_connections 2024-09-10T09:39:53.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:53.149+0000 7f763ae2d740 1 -- 172.21.15.5:0/3339238104 wait complete. 2024-09-10T09:39:53.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:53.149+0000 7f763ae2d740 20 client.15660 trim_cache size 0 max 16384 2024-09-10T09:39:53.162 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-10T09:39:53.162 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "no active scrubs running", 2024-09-10T09:39:53.162 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": {} 2024-09-10T09:39:53.162 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-10T09:39:53.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:53.150+0000 7f7638400640 1 -- 172.21.15.5:0/1524664519 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f76180a2c00 msgr2=0x7f76180a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:39:53.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:53.150+0000 7f7638400640 1 --2- 172.21.15.5:0/1524664519 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f76180a2c00 0x7f76180a50c0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f7624097900 tx=0x7f76240d2a20 comp rx=0 tx=0).stop 2024-09-10T09:39:53.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:53.150+0000 7f7638400640 1 -- 172.21.15.5:0/1524664519 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7630074590 msgr2=0x7f76301a46b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:39:53.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:53.150+0000 7f7638400640 1 --2- 172.21.15.5:0/1524664519 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7630074590 0x7f76301a46b0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f762400d950 tx=0x7f762400de10 comp rx=0 tx=0).stop 2024-09-10T09:39:53.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:53.150+0000 7f7638400640 1 -- 172.21.15.5:0/1524664519 shutdown_connections 2024-09-10T09:39:53.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:53.150+0000 7f7638400640 1 --2- 172.21.15.5:0/1524664519 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f76180a2c00 0x7f76180a50c0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:53.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:53.150+0000 7f7638400640 1 --2- 172.21.15.5:0/1524664519 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7630112b90 0x7f763019e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:53.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:53.150+0000 7f7638400640 1 --2- 172.21.15.5:0/1524664519 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7630074eb0 0x7f76301a4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:53.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:53.150+0000 7f7638400640 1 --2- 172.21.15.5:0/1524664519 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7630074590 0x7f76301a46b0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:53.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:53.150+0000 7f7638400640 1 -- 172.21.15.5:0/1524664519 >> 172.21.15.5:0/1524664519 conn(0x7f76300fd8c0 msgr2=0x7f76300ff010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:39:53.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:53.150+0000 7f7638400640 1 -- 172.21.15.5:0/1524664519 shutdown_connections 2024-09-10T09:39:53.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:53.151+0000 7f7638400640 1 -- 172.21.15.5:0/1524664519 wait complete. 2024-09-10T09:39:54.091 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-10T09:39:54.091 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T09:39:54.472 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:39:54.921 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:39:54 smithi005 ceph-mon[30350]: pgmap v440: 129 pgs: 129 active+clean; 666 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 0 B/s wr, 8 op/s 2024-09-10T09:39:54.936 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:39:54 smithi102 ceph-mon[35442]: pgmap v440: 129 pgs: 129 active+clean; 666 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 0 B/s wr, 8 op/s 2024-09-10T09:39:55.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:55.031+0000 7f3902c00640 1 -- 172.21.15.5:0/65238059 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f38fc1034b0 msgr2=0x7f38fc10b9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:39:55.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:55.031+0000 7f3902c00640 1 --2- 172.21.15.5:0/65238059 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f38fc1034b0 0x7f38fc10b9a0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f38e8009900 tx=0x7f38e802a480 comp rx=0 tx=0).stop 2024-09-10T09:39:55.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:55.031+0000 7f3902c00640 1 -- 172.21.15.5:0/65238059 shutdown_connections 2024-09-10T09:39:55.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:55.031+0000 7f3902c00640 1 --2- 172.21.15.5:0/65238059 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f38fc10bfc0 0x7f38fc10e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:55.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:55.031+0000 7f3902c00640 1 --2- 172.21.15.5:0/65238059 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f38fc1034b0 0x7f38fc10b9a0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:55.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:55.031+0000 7f3902c00640 1 --2- 172.21.15.5:0/65238059 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f38fc102b90 0x7f38fc102f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:55.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:55.031+0000 7f3902c00640 1 -- 172.21.15.5:0/65238059 >> 172.21.15.5:0/65238059 conn(0x7f38fc0fc630 msgr2=0x7f38fc0fea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:39:55.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:55.031+0000 7f3902c00640 1 -- 172.21.15.5:0/65238059 shutdown_connections 2024-09-10T09:39:55.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:55.031+0000 7f3902c00640 1 -- 172.21.15.5:0/65238059 wait complete. 2024-09-10T09:39:55.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:55.032+0000 7f3902c00640 1 Processor -- start 2024-09-10T09:39:55.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:55.032+0000 7f3902c00640 1 -- start start 2024-09-10T09:39:55.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:55.033+0000 7f3902c00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f38fc102b90 0x7f38fc1007f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:39:55.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:55.033+0000 7f3902c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f38fc1034b0 0x7f38fc100d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:39:55.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:55.033+0000 7f3902c00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f38fc10bfc0 0x7f38fc101270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:39:55.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:55.033+0000 7f3902c00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f38fc111dc0 con 0x7f38fc102b90 2024-09-10T09:39:55.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:55.033+0000 7f3902c00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f38fc111c40 con 0x7f38fc1034b0 2024-09-10T09:39:55.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:55.033+0000 7f3902c00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f38fc111f40 con 0x7f38fc10bfc0 2024-09-10T09:39:55.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:55.033+0000 7f3901800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f38fc102b90 0x7f38fc1007f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:39:55.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:55.033+0000 7f3901800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f38fc102b90 0x7f38fc1007f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34034/0 (socket says 172.21.15.5:34034) 2024-09-10T09:39:55.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:55.033+0000 7f3901800640 1 -- 172.21.15.5:0/24908828 learned_addr learned my addr 172.21.15.5:0/24908828 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:39:55.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:55.033+0000 7f3902200640 1 --2- 172.21.15.5:0/24908828 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f38fc10bfc0 0x7f38fc101270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:39:55.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:55.033+0000 7f3900e00640 1 --2- 172.21.15.5:0/24908828 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f38fc1034b0 0x7f38fc100d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:39:55.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:55.034+0000 7f3901800640 1 -- 172.21.15.5:0/24908828 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f38fc10bfc0 msgr2=0x7f38fc101270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:39:55.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:55.034+0000 7f3901800640 1 --2- 172.21.15.5:0/24908828 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f38fc10bfc0 0x7f38fc101270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:55.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:55.034+0000 7f3901800640 1 -- 172.21.15.5:0/24908828 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f38fc1034b0 msgr2=0x7f38fc100d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:39:55.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:55.034+0000 7f3901800640 1 --2- 172.21.15.5:0/24908828 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f38fc1034b0 0x7f38fc100d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:55.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:55.034+0000 7f3901800640 1 -- 172.21.15.5:0/24908828 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38fc1a67e0 con 0x7f38fc102b90 2024-09-10T09:39:55.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:55.034+0000 7f3902200640 1 --2- 172.21.15.5:0/24908828 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f38fc10bfc0 0x7f38fc101270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:39:55.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:55.034+0000 7f3901800640 1 --2- 172.21.15.5:0/24908828 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f38fc102b90 0x7f38fc1007f0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f38f000ea60 tx=0x7f38f000ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:39:55.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:55.034+0000 7f38fa000640 1 -- 172.21.15.5:0/24908828 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38f000cde0 con 0x7f38fc102b90 2024-09-10T09:39:55.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:55.034+0000 7f38fa000640 1 -- 172.21.15.5:0/24908828 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f38f00035f0 con 0x7f38fc102b90 2024-09-10T09:39:55.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:55.034+0000 7f38fa000640 1 -- 172.21.15.5:0/24908828 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38f0010670 con 0x7f38fc102b90 2024-09-10T09:39:55.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:55.034+0000 7f3902c00640 1 -- 172.21.15.5:0/24908828 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38fc1a6ad0 con 0x7f38fc102b90 2024-09-10T09:39:55.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:55.034+0000 7f3902c00640 1 -- 172.21.15.5:0/24908828 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38fc1a6f90 con 0x7f38fc102b90 2024-09-10T09:39:55.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:55.036+0000 7f3902c00640 1 -- 172.21.15.5:0/24908828 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f38fc102f70 con 0x7f38fc102b90 2024-09-10T09:39:55.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:55.037+0000 7f38fa000640 1 -- 172.21.15.5:0/24908828 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f38f00037a0 con 0x7f38fc102b90 2024-09-10T09:39:55.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:55.040+0000 7f38fa000640 1 --2- 172.21.15.5:0/24908828 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f38e40a2c00 0x7f38e40a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:39:55.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:55.040+0000 7f38fa000640 1 -- 172.21.15.5:0/24908828 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f38f00cf700 con 0x7f38fc102b90 2024-09-10T09:39:55.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:55.040+0000 7f3900e00640 1 --2- 172.21.15.5:0/24908828 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f38e40a2c00 0x7f38e40a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:39:55.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:55.041+0000 7f3900e00640 1 --2- 172.21.15.5:0/24908828 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f38e40a2c00 0x7f38e40a50c0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f38e8009230 tx=0x7f38e80058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:39:55.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:55.044+0000 7f38fa000640 1 -- 172.21.15.5:0/24908828 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f38f009c210 con 0x7f38fc102b90 2024-09-10T09:39:55.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:39:54 smithi159 ceph-mon[35513]: pgmap v440: 129 pgs: 129 active+clean; 666 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 0 B/s wr, 8 op/s 2024-09-10T09:39:55.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:55.266+0000 7f3902c00640 1 -- 172.21.15.5:0/24908828 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f38fc101d20 con 0x7f38fc102b90 2024-09-10T09:39:55.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:55.267+0000 7f38fa000640 1 -- 172.21.15.5:0/24908828 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11969 (secure 0 0 0) 0x7f38f009bb10 con 0x7f38fc102b90 2024-09-10T09:39:55.269 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-10T09:39:55.269 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":30,"btime":"2024-09-10T09:38:15:834198+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14454,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/2970594311","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2970594311},{"type":"v1","addr":"172.21.15.5:6835","nonce":2970594311}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14514,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6839/3616502174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":3616502174},{"type":"v1","addr":"172.21.15.5:6839","nonce":3616502174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T09:35:29.626287+0000","modified":"2024-09-10T09:38:15.658058+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24542,"mds_1":24535,"mds_2":24511,"mds_3":24506,"mds_4":24443},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14520":{"gid":14520,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/2073599842","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":2073599842},{"type":"v1","addr":"172.21.15.5:6841","nonce":2073599842}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"k","rank":4,"incarnation":26,"state":"up:active","state_seq":24,"addr":"172.21.15.102:6833/2457007427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":2457007427},{"type":"v1","addr":"172.21.15.102:6833","nonce":2457007427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/356445159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":356445159},{"type":"v1","addr":"172.21.15.159:6833","nonce":356445159}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6837/2894314507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":2894314507},{"type":"v1","addr":"172.21.15.5:6837","nonce":2894314507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.102:6835/1647549111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":1647549111},{"type":"v1","addr":"172.21.15.102:6835","nonce":1647549111}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6835/952657242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":952657242},{"type":"v1","addr":"172.21.15.159:6835","nonce":952657242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":19,"addr":"172.21.15.102:6837/2689776764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":2689776764},{"type":"v1","addr":"172.21.15.102:6837","nonce":2689776764}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.159:6837/620627639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":620627639},{"type":"v1","addr":"172.21.15.159:6837","nonce":620627639}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24535":{"gid":24535,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":18,"addr":"172.21.15.159:6839/2073739599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":2073739599},{"type":"v1","addr":"172.21.15.159:6839","nonce":2073739599}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24542":{"gid":24542,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.102:6839/335409749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6838","nonce":335409749},{"type":"v1","addr":"172.21.15.102:6839","nonce":335409749}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24542,"qdb_cluster":[24542,24535,24511,24506,24443]},"id":1}]} 2024-09-10T09:39:55.269 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 30 2024-09-10T09:39:55.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:55.270+0000 7f3902c00640 1 -- 172.21.15.5:0/24908828 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f38e40a2c00 msgr2=0x7f38e40a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:39:55.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:55.270+0000 7f3902c00640 1 --2- 172.21.15.5:0/24908828 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f38e40a2c00 0x7f38e40a50c0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f38e8009230 tx=0x7f38e80058d0 comp rx=0 tx=0).stop 2024-09-10T09:39:55.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:55.270+0000 7f3902c00640 1 -- 172.21.15.5:0/24908828 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f38fc102b90 msgr2=0x7f38fc1007f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:39:55.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:55.270+0000 7f3902c00640 1 --2- 172.21.15.5:0/24908828 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f38fc102b90 0x7f38fc1007f0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f38f000ea60 tx=0x7f38f000ef20 comp rx=0 tx=0).stop 2024-09-10T09:39:55.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:55.270+0000 7f3902c00640 1 -- 172.21.15.5:0/24908828 shutdown_connections 2024-09-10T09:39:55.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:55.270+0000 7f3902c00640 1 --2- 172.21.15.5:0/24908828 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f38e40a2c00 0x7f38e40a50c0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:55.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:55.270+0000 7f3902c00640 1 --2- 172.21.15.5:0/24908828 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f38fc10bfc0 0x7f38fc101270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:55.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:55.270+0000 7f3902c00640 1 --2- 172.21.15.5:0/24908828 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f38fc1034b0 0x7f38fc100d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:55.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:55.270+0000 7f3902c00640 1 --2- 172.21.15.5:0/24908828 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f38fc102b90 0x7f38fc1007f0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:55.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:55.270+0000 7f3902c00640 1 -- 172.21.15.5:0/24908828 >> 172.21.15.5:0/24908828 conn(0x7f38fc0fc630 msgr2=0x7f38fc0fd370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:39:55.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:55.271+0000 7f3902c00640 1 -- 172.21.15.5:0/24908828 shutdown_connections 2024-09-10T09:39:55.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:55.271+0000 7f3902c00640 1 -- 172.21.15.5:0/24908828 wait complete. 2024-09-10T09:39:56.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:39:55 smithi005 ceph-mon[30350]: from='client.? 172.21.15.5:0/24908828' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:39:56.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:39:55 smithi159 ceph-mon[35513]: from='client.? 172.21.15.5:0/24908828' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:39:56.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:39:55 smithi102 ceph-mon[35442]: from='client.? 172.21.15.5:0/24908828' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:39:56.148 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-10T09:39:56.524 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:39:56.973 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:39:56 smithi005 ceph-mon[30350]: pgmap v441: 129 pgs: 129 active+clean; 666 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 85 B/s wr, 12 op/s 2024-09-10T09:39:57.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:39:56 smithi159 ceph-mon[35513]: pgmap v441: 129 pgs: 129 active+clean; 666 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 85 B/s wr, 12 op/s 2024-09-10T09:39:57.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:57.087+0000 7f04d7000640 1 -- 172.21.15.5:0/1526450367 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f04d01049a0 msgr2=0x7f04d01089f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:39:57.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:57.087+0000 7f04d7000640 1 --2- 172.21.15.5:0/1526450367 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f04d01049a0 0x7f04d01089f0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f04bc009900 tx=0x7f04bc02a410 comp rx=0 tx=0).stop 2024-09-10T09:39:57.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:57.087+0000 7f04d7000640 1 -- 172.21.15.5:0/1526450367 shutdown_connections 2024-09-10T09:39:57.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:57.087+0000 7f04d7000640 1 --2- 172.21.15.5:0/1526450367 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f04d0109120 0x7f04d0110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:57.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:57.087+0000 7f04d7000640 1 --2- 172.21.15.5:0/1526450367 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f04d01049a0 0x7f04d01089f0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:57.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:57.087+0000 7f04d7000640 1 --2- 172.21.15.5:0/1526450367 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f04d0103ff0 0x7f04d01043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:57.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:57.087+0000 7f04d7000640 1 -- 172.21.15.5:0/1526450367 >> 172.21.15.5:0/1526450367 conn(0x7f04d00ffa90 msgr2=0x7f04d0101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:39:57.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:57.087+0000 7f04d7000640 1 -- 172.21.15.5:0/1526450367 shutdown_connections 2024-09-10T09:39:57.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:57.087+0000 7f04d7000640 1 -- 172.21.15.5:0/1526450367 wait complete. 2024-09-10T09:39:57.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:57.088+0000 7f04d7000640 1 Processor -- start 2024-09-10T09:39:57.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:57.088+0000 7f04d7000640 1 -- start start 2024-09-10T09:39:57.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:57.089+0000 7f04d7000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f04d0103ff0 0x7f04d019a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:39:57.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:57.089+0000 7f04d7000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f04d01049a0 0x7f04d019ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:39:57.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:57.089+0000 7f04d7000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f04d0109120 0x7f04d019ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:39:57.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:57.089+0000 7f04d7000640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f04d0114190 con 0x7f04d0103ff0 2024-09-10T09:39:57.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:57.089+0000 7f04d7000640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f04d0114010 con 0x7f04d0109120 2024-09-10T09:39:57.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:57.089+0000 7f04d7000640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f04d0114310 con 0x7f04d01049a0 2024-09-10T09:39:57.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:57.089+0000 7f04cfe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f04d0103ff0 0x7f04d019a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:39:57.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:57.089+0000 7f04cfe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f04d0103ff0 0x7f04d019a650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34050/0 (socket says 172.21.15.5:34050) 2024-09-10T09:39:57.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:57.089+0000 7f04cfe00640 1 -- 172.21.15.5:0/2324703590 learned_addr learned my addr 172.21.15.5:0/2324703590 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:39:57.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:57.089+0000 7f04cf400640 1 --2- 172.21.15.5:0/2324703590 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f04d01049a0 0x7f04d019ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:39:57.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:57.089+0000 7f04d5000640 1 --2- 172.21.15.5:0/2324703590 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f04d0109120 0x7f04d019ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:39:57.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:57.089+0000 7f04cfe00640 1 -- 172.21.15.5:0/2324703590 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f04d01049a0 msgr2=0x7f04d019ab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:39:57.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:57.089+0000 7f04cfe00640 1 --2- 172.21.15.5:0/2324703590 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f04d01049a0 0x7f04d019ab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:57.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:57.089+0000 7f04cfe00640 1 -- 172.21.15.5:0/2324703590 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f04d0109120 msgr2=0x7f04d019ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:39:57.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:57.089+0000 7f04cfe00640 1 --2- 172.21.15.5:0/2324703590 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f04d0109120 0x7f04d019ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:57.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:57.089+0000 7f04cfe00640 1 -- 172.21.15.5:0/2324703590 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04d019f600 con 0x7f04d0103ff0 2024-09-10T09:39:57.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:57.090+0000 7f04cfe00640 1 --2- 172.21.15.5:0/2324703590 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f04d0103ff0 0x7f04d019a650 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f04c000e9f0 tx=0x7f04c000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:39:57.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:57.090+0000 7f04ccc00640 1 -- 172.21.15.5:0/2324703590 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04c000cd80 con 0x7f04d0103ff0 2024-09-10T09:39:57.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:57.090+0000 7f04ccc00640 1 -- 172.21.15.5:0/2324703590 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f04c0005500 con 0x7f04d0103ff0 2024-09-10T09:39:57.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:57.090+0000 7f04d7000640 1 -- 172.21.15.5:0/2324703590 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04d019f8f0 con 0x7f04d0103ff0 2024-09-10T09:39:57.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:57.090+0000 7f04ccc00640 1 -- 172.21.15.5:0/2324703590 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04c0010690 con 0x7f04d0103ff0 2024-09-10T09:39:57.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:57.090+0000 7f04d7000640 1 -- 172.21.15.5:0/2324703590 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04d01a71d0 con 0x7f04d0103ff0 2024-09-10T09:39:57.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:57.093+0000 7f04d9a93740 1 Processor -- start 2024-09-10T09:39:57.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:57.093+0000 7f04ccc00640 1 -- 172.21.15.5:0/2324703590 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f04c00046c0 con 0x7f04d0103ff0 2024-09-10T09:39:57.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:57.094+0000 7f04d9a93740 1 -- start start 2024-09-10T09:39:57.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:57.094+0000 7f04d9a93740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a283342c70 0x55a283343050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:39:57.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:57.094+0000 7f04d9a93740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a283343590 0x55a28334d910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:39:57.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:57.094+0000 7f04d9a93740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a28334de50 0x55a283350240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:39:57.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:57.094+0000 7f04d9a93740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55a2832a93e0 con 0x55a283343590 2024-09-10T09:39:57.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:57.094+0000 7f04d9a93740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55a283250d80 con 0x55a283342c70 2024-09-10T09:39:57.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:57.094+0000 7f04d9a93740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55a2832a53a0 con 0x55a28334de50 2024-09-10T09:39:57.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:57.094+0000 7f04cfe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a283343590 0x55a28334d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:39:57.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:57.094+0000 7f04d5000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a283342c70 0x55a283343050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:39:57.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:57.094+0000 7f04d5000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a283342c70 0x55a283343050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:55220/0 (socket says 172.21.15.5:55220) 2024-09-10T09:39:57.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:57.094+0000 7f04d5000640 1 -- 172.21.15.5:0/1254985314 learned_addr learned my addr 172.21.15.5:0/1254985314 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:39:57.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:57.094+0000 7f04cf400640 1 --2- 172.21.15.5:0/1254985314 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a28334de50 0x55a283350240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:39:57.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:57.095+0000 7f04d5000640 1 -- 172.21.15.5:0/1254985314 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a28334de50 msgr2=0x55a283350240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:39:57.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:57.095+0000 7f04d5000640 1 --2- 172.21.15.5:0/1254985314 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a28334de50 0x55a283350240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:57.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:57.095+0000 7f04d5000640 1 -- 172.21.15.5:0/1254985314 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a283343590 msgr2=0x55a28334d910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:39:57.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:57.095+0000 7f04d5000640 1 --2- 172.21.15.5:0/1254985314 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a283343590 0x55a28334d910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:57.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:57.095+0000 7f04d5000640 1 -- 172.21.15.5:0/1254985314 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a28326fae0 con 0x55a283342c70 2024-09-10T09:39:57.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:57.095+0000 7f04d5000640 1 --2- 172.21.15.5:0/1254985314 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a283342c70 0x55a283343050 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f04c8007ec0 tx=0x7f04c800f790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6c83e6fe5d56b89f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:39:57.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:57.095+0000 7f04ccc00640 1 --2- 172.21.15.5:0/2324703590 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f04b40a2b80 0x7f04b40a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:39:57.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:57.095+0000 7f04c5600640 1 -- 172.21.15.5:0/1254985314 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04c8010840 con 0x55a283342c70 2024-09-10T09:39:57.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:57.095+0000 7f04c5600640 1 -- 172.21.15.5:0/1254985314 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f04c80183f0 con 0x55a283342c70 2024-09-10T09:39:57.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:57.096+0000 7f04ccc00640 1 -- 172.21.15.5:0/2324703590 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f04c00cf810 con 0x7f04d0103ff0 2024-09-10T09:39:57.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:57.096+0000 7f04cfe00640 1 --2- 172.21.15.5:0/2324703590 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f04b40a2b80 0x7f04b40a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:39:57.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:57.096+0000 7f04c5600640 1 -- 172.21.15.5:0/1254985314 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04c80175a0 con 0x55a283342c70 2024-09-10T09:39:57.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:57.096+0000 7f04d9a93740 1 -- 172.21.15.5:0/1254985314 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a283342c70 msgr2=0x55a283343050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:39:57.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:57.096+0000 7f04cfe00640 1 --2- 172.21.15.5:0/2324703590 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f04b40a2b80 0x7f04b40a5040 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f04c00cd290 tx=0x7f04c00d5550 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:39:57.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:57.096+0000 7f04d9a93740 1 --2- 172.21.15.5:0/1254985314 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a283342c70 0x55a283343050 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f04c8007ec0 tx=0x7f04c800f790 comp rx=0 tx=0).stop 2024-09-10T09:39:57.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:57.096+0000 7f04d9a93740 1 -- 172.21.15.5:0/1254985314 shutdown_connections 2024-09-10T09:39:57.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:57.096+0000 7f04d9a93740 1 --2- 172.21.15.5:0/1254985314 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a28334de50 0x55a283350240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:57.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:57.096+0000 7f04d9a93740 1 --2- 172.21.15.5:0/1254985314 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a283343590 0x55a28334d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:57.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:57.096+0000 7f04d9a93740 1 --2- 172.21.15.5:0/1254985314 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a283342c70 0x55a283343050 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:57.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:57.096+0000 7f04d9a93740 1 -- 172.21.15.5:0/1254985314 >> 172.21.15.5:0/1254985314 conn(0x55a28333f200 msgr2=0x55a28333f5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:39:57.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:57.097+0000 7f04d9a93740 1 -- 172.21.15.5:0/1254985314 shutdown_connections 2024-09-10T09:39:57.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:57.097+0000 7f04d9a93740 1 -- 172.21.15.5:0/1254985314 wait complete. 2024-09-10T09:39:57.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:57.097+0000 7f04d9a93740 1 -- start start 2024-09-10T09:39:57.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:57.097+0000 7f04d9a93740 1 Processor -- start 2024-09-10T09:39:57.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:57.098+0000 7f04d9a93740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a283342c70 0x55a28334ad20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:39:57.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:57.098+0000 7f04d9a93740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a283343590 0x55a28334b260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:39:57.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:57.098+0000 7f04d9a93740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a28334de50 0x55a28334b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:39:57.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:57.098+0000 7f04d9a93740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55a283350a70 con 0x55a283343590 2024-09-10T09:39:57.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:57.098+0000 7f04d9a93740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55a2832640f0 con 0x55a283342c70 2024-09-10T09:39:57.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:57.098+0000 7f04d9a93740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55a283350bf0 con 0x55a28334de50 2024-09-10T09:39:57.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:57.098+0000 7f04d9a93740 10 client.0 _ll_register_callbacks cb 0x55a28324bbd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:39:57.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:57.098+0000 7f04cf400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a283343590 0x55a28334b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:39:57.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:57.098+0000 7f04d5000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a283342c70 0x55a28334ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:39:57.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:57.098+0000 7f04d5000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a283342c70 0x55a28334ad20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:55228/0 (socket says 172.21.15.5:55228) 2024-09-10T09:39:57.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:57.098+0000 7f04d5000640 1 -- 172.21.15.5:0/2102282825 learned_addr learned my addr 172.21.15.5:0/2102282825 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:39:57.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:57.098+0000 7f04d5000640 1 --2- 172.21.15.5:0/2102282825 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a28334de50 0x55a28334b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:39:57.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:57.098+0000 7f04cf400640 1 -- 172.21.15.5:0/2102282825 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a28334de50 msgr2=0x55a28334b7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:39:57.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:57.098+0000 7f04cf400640 1 --2- 172.21.15.5:0/2102282825 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a28334de50 0x55a28334b7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:57.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:57.098+0000 7f04cf400640 1 -- 172.21.15.5:0/2102282825 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a283342c70 msgr2=0x55a28334ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:39:57.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:57.098+0000 7f04cf400640 1 --2- 172.21.15.5:0/2102282825 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a283342c70 0x55a28334ad20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:57.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:57.099+0000 7f04cf400640 1 -- 172.21.15.5:0/2102282825 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a28301a3c0 con 0x55a283343590 2024-09-10T09:39:57.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:57.099+0000 7f04cf400640 1 --2- 172.21.15.5:0/2102282825 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a283343590 0x55a28334b260 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f04bc02a930 tx=0x7f04bc004510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41985313e88d905 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:39:57.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:57.099+0000 7f04ba000640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-10T09:39:57.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:57.099+0000 7f04ba000640 1 -- 172.21.15.5:0/2102282825 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04bc035040 con 0x55a283343590 2024-09-10T09:39:57.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:57.099+0000 7f04d9a93740 1 -- 172.21.15.5:0/2102282825 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f04c80183f0 con 0x55a283343590 2024-09-10T09:39:57.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:57.099+0000 7f04ba000640 1 -- 172.21.15.5:0/2102282825 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f04bc005510 con 0x55a283343590 2024-09-10T09:39:57.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:57.099+0000 7f04ba000640 1 -- 172.21.15.5:0/2102282825 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f04bc04f050 con 0x55a283343590 2024-09-10T09:39:57.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:57.099+0000 7f04d9a93740 10 client.15678 fetch_fsmap learned FSMap version 30 2024-09-10T09:39:57.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:57.099+0000 7f04d9a93740 1 -- 172.21.15.5:0/2102282825 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55a28335cae0 con 0x55a283343590 2024-09-10T09:39:57.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:57.100+0000 7f04ba000640 1 -- 172.21.15.5:0/2102282825 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f04c80183f0 con 0x55a283343590 2024-09-10T09:39:57.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:57.100+0000 7f04ba000640 1 -- 172.21.15.5:0/2102282825 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f04bc031980 con 0x55a283343590 2024-09-10T09:39:57.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:57.100+0000 7f04d9a93740 10 client.15678 fetch_fsmap finished waiting for FSMap version 30 2024-09-10T09:39:57.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:57.100+0000 7f04d9a93740 10 client.15678 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-09-10T09:39:57.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:57.100+0000 7f04d9a93740 20 client.15678 populate_metadata read hostname 'smithi005' 2024-09-10T09:39:57.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:57.100+0000 7f04d9a93740 1 --2- 172.21.15.5:0/2102282825 >> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] conn(0x55a28335def0 0x55a2833602e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:39:57.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:57.100+0000 7f04d9a93740 4 client.15678 mds_command: new command op to 24443 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:39:57.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:57.100+0000 7f04d9a93740 1 -- 172.21.15.5:0/2102282825 --> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a2832b6950 con 0x55a28335def0 2024-09-10T09:39:57.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:57.101+0000 7f04d5000640 1 --2- 172.21.15.5:0/2102282825 >> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] conn(0x55a28335def0 0x55a2833602e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:39:57.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:57.101+0000 7f04d5000640 1 --2- 172.21.15.5:0/2102282825 >> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] conn(0x55a28335def0 0x55a2833602e0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=43986def8a52a1aa server_cookie=cf374921b06aad39 in_seq=0 out_seq=0 2024-09-10T09:39:57.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:57.101+0000 7f04ba000640 10 client.15678 ms_handle_connect on v2:172.21.15.102:6832/2457007427 2024-09-10T09:39:57.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:57.102+0000 7f04ba000640 1 -- 172.21.15.5:0/2102282825 <== mds.4 v2:172.21.15.102:6832/2457007427 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a2832b6950 con 0x55a28335def0 2024-09-10T09:39:57.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:57.102+0000 7f04ba000640 10 client.15678 handle_command_reply: tid=0 2024-09-10T09:39:57.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:57.102+0000 7f04d9a93740 1 client.15678 shutdown 2024-09-10T09:39:57.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:57.102+0000 7f04d9a93740 1 -- 172.21.15.5:0/2102282825 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a283343590 msgr2=0x55a28334b260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:39:57.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:57.102+0000 7f04d9a93740 1 --2- 172.21.15.5:0/2102282825 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a283343590 0x55a28334b260 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f04bc02a930 tx=0x7f04bc004510 comp rx=0 tx=0).stop 2024-09-10T09:39:57.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:57.102+0000 7f04d9a93740 1 -- 172.21.15.5:0/2102282825 shutdown_connections 2024-09-10T09:39:57.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:57.102+0000 7f04d9a93740 1 --2- 172.21.15.5:0/2102282825 >> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] conn(0x55a28335def0 0x55a2833602e0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:57.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:57.102+0000 7f04d9a93740 1 --2- 172.21.15.5:0/2102282825 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a28334de50 0x55a28334b7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:57.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:57.102+0000 7f04d9a93740 1 --2- 172.21.15.5:0/2102282825 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a283343590 0x55a28334b260 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:57.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:57.102+0000 7f04d9a93740 1 --2- 172.21.15.5:0/2102282825 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a283342c70 0x55a28334ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:57.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:57.102+0000 7f04ba000640 0 client.15678 ms_handle_reset on v2:172.21.15.102:6832/2457007427 2024-09-10T09:39:57.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:57.102+0000 7f04d9a93740 1 -- 172.21.15.5:0/2102282825 >> 172.21.15.5:0/2102282825 conn(0x55a28333f200 msgr2=0x55a28333e380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:39:57.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:57.102+0000 7f04d9a93740 1 -- 172.21.15.5:0/2102282825 shutdown_connections 2024-09-10T09:39:57.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:57.102+0000 7f04d9a93740 1 -- 172.21.15.5:0/2102282825 wait complete. 2024-09-10T09:39:57.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:57.102+0000 7f04d9a93740 20 client.15678 trim_cache size 0 max 16384 2024-09-10T09:39:57.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:57.116+0000 7f04d9a93740 1 Processor -- start 2024-09-10T09:39:57.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:57.116+0000 7f04d9a93740 1 -- start start 2024-09-10T09:39:57.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:57.117+0000 7f04d9a93740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a283342c70 0x55a283365b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:39:57.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:57.117+0000 7f04d9a93740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a283343590 0x55a2833660b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:39:57.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:57.117+0000 7f04d9a93740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a28334de50 0x55a283347f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:39:57.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:57.117+0000 7f04d9a93740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f04d01b0fe0 con 0x55a283342c70 2024-09-10T09:39:57.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:57.117+0000 7f04d9a93740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f04d01b2b20 con 0x55a283343590 2024-09-10T09:39:57.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:57.117+0000 7f04d9a93740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f04d01b16b0 con 0x55a28334de50 2024-09-10T09:39:57.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:57.117+0000 7f04d5000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a283342c70 0x55a283365b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:39:57.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:57.117+0000 7f04d5000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a283342c70 0x55a283365b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34072/0 (socket says 172.21.15.5:34072) 2024-09-10T09:39:57.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:57.117+0000 7f04d5000640 1 -- 172.21.15.5:0/2185174371 learned_addr learned my addr 172.21.15.5:0/2185174371 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:39:57.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:57.117+0000 7f04cf400640 1 --2- 172.21.15.5:0/2185174371 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a283343590 0x55a2833660b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:39:57.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:57.117+0000 7f04d5000640 1 --2- 172.21.15.5:0/2185174371 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a28334de50 0x55a283347f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:39:57.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:57.117+0000 7f04d5000640 1 -- 172.21.15.5:0/2185174371 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a28334de50 msgr2=0x55a283347f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:39:57.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:57.117+0000 7f04d5000640 1 --2- 172.21.15.5:0/2185174371 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a28334de50 0x55a283347f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:57.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:57.117+0000 7f04d5000640 1 -- 172.21.15.5:0/2185174371 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a283343590 msgr2=0x55a2833660b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:39:57.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:57.117+0000 7f04d5000640 1 --2- 172.21.15.5:0/2185174371 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a283343590 0x55a2833660b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:57.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:57.117+0000 7f04d5000640 1 -- 172.21.15.5:0/2185174371 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a28333a770 con 0x55a283342c70 2024-09-10T09:39:57.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:57.117+0000 7f04cf400640 1 --2- 172.21.15.5:0/2185174371 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a283343590 0x55a2833660b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:39:57.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:57.118+0000 7f04d5000640 1 --2- 172.21.15.5:0/2185174371 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a283342c70 0x55a283365b70 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f04c8007ec0 tx=0x7f04c8007520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d50570f05c612204 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:39:57.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:57.118+0000 7f04c4c00640 1 -- 172.21.15.5:0/2185174371 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04c802b020 con 0x55a283342c70 2024-09-10T09:39:57.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:57.118+0000 7f04c4c00640 1 -- 172.21.15.5:0/2185174371 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f04c80055b0 con 0x55a283342c70 2024-09-10T09:39:57.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:57.118+0000 7f04c4c00640 1 -- 172.21.15.5:0/2185174371 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04c8018a30 con 0x55a283342c70 2024-09-10T09:39:57.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:57.119+0000 7f04d9a93740 1 -- 172.21.15.5:0/2185174371 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a283342c70 msgr2=0x55a283365b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:39:57.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:57.119+0000 7f04d9a93740 1 --2- 172.21.15.5:0/2185174371 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a283342c70 0x55a283365b70 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f04c8007ec0 tx=0x7f04c8007520 comp rx=0 tx=0).stop 2024-09-10T09:39:57.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:57.119+0000 7f04d9a93740 1 -- 172.21.15.5:0/2185174371 shutdown_connections 2024-09-10T09:39:57.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:57.119+0000 7f04d9a93740 1 --2- 172.21.15.5:0/2185174371 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a28334de50 0x55a283347f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:57.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:57.119+0000 7f04d9a93740 1 --2- 172.21.15.5:0/2185174371 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a283343590 0x55a2833660b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:57.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:57.119+0000 7f04d9a93740 1 --2- 172.21.15.5:0/2185174371 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a283342c70 0x55a283365b70 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:57.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:57.119+0000 7f04d9a93740 1 -- 172.21.15.5:0/2185174371 >> 172.21.15.5:0/2185174371 conn(0x55a28333f200 msgr2=0x55a28334cda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:39:57.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:57.119+0000 7f04d9a93740 1 -- 172.21.15.5:0/2185174371 shutdown_connections 2024-09-10T09:39:57.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:57.119+0000 7f04d9a93740 1 -- 172.21.15.5:0/2185174371 wait complete. 2024-09-10T09:39:57.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:57.119+0000 7f04d9a93740 1 -- start start 2024-09-10T09:39:57.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:57.120+0000 7f04d9a93740 1 Processor -- start 2024-09-10T09:39:57.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:57.120+0000 7f04d9a93740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a283342c70 0x55a2833521f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:39:57.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:57.120+0000 7f04d9a93740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a283343590 0x55a283352730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:39:57.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:57.120+0000 7f04d9a93740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a28334de50 0x55a2833c7b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:39:57.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:57.120+0000 7f04d9a93740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f04d01b2450 con 0x55a283343590 2024-09-10T09:39:57.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:57.120+0000 7f04d9a93740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f04d01b1d80 con 0x55a283342c70 2024-09-10T09:39:57.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:57.120+0000 7f04d9a93740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f04d01aefc0 con 0x55a28334de50 2024-09-10T09:39:57.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:57.120+0000 7f04d9a93740 10 client.0 _ll_register_callbacks cb 0x7f04bc005d50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:39:57.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:57.120+0000 7f04cf400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a283343590 0x55a283352730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:39:57.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:57.120+0000 7f04d5000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a283342c70 0x55a2833521f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:39:57.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:57.120+0000 7f04cf400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a283343590 0x55a283352730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34078/0 (socket says 172.21.15.5:34078) 2024-09-10T09:39:57.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:57.120+0000 7f04cf400640 1 -- 172.21.15.5:0/2974196364 learned_addr learned my addr 172.21.15.5:0/2974196364 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:39:57.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:57.120+0000 7f04d5000640 1 --2- 172.21.15.5:0/2974196364 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a28334de50 0x55a2833c7b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:39:57.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:57.121+0000 7f04cf400640 1 -- 172.21.15.5:0/2974196364 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a28334de50 msgr2=0x55a2833c7b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:39:57.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:57.121+0000 7f04cf400640 1 --2- 172.21.15.5:0/2974196364 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a28334de50 0x55a2833c7b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:57.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:57.121+0000 7f04cf400640 1 -- 172.21.15.5:0/2974196364 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a283342c70 msgr2=0x55a2833521f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:39:57.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:57.121+0000 7f04cf400640 1 --2- 172.21.15.5:0/2974196364 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a283342c70 0x55a2833521f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:57.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:57.121+0000 7f04cf400640 1 -- 172.21.15.5:0/2974196364 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a2833c8080 con 0x55a283343590 2024-09-10T09:39:57.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:57.121+0000 7f04d5000640 1 --2- 172.21.15.5:0/2974196364 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a283342c70 0x55a2833521f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:39:57.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:57.121+0000 7f04cf400640 1 --2- 172.21.15.5:0/2974196364 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a283343590 0x55a283352730 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f04bc031460 tx=0x7f04bc005540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aeb5ef7c9a26afe3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:39:57.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:57.121+0000 7f04ba000640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-10T09:39:57.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:57.121+0000 7f04ba000640 1 -- 172.21.15.5:0/2974196364 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04bc058060 con 0x55a283343590 2024-09-10T09:39:57.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:57.121+0000 7f04ba000640 1 -- 172.21.15.5:0/2974196364 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f04bc0035f0 con 0x55a283343590 2024-09-10T09:39:57.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:57.121+0000 7f04d9a93740 1 -- 172.21.15.5:0/2974196364 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f04c80055b0 con 0x55a283343590 2024-09-10T09:39:57.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:57.122+0000 7f04ba000640 1 -- 172.21.15.5:0/2974196364 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f04bc05f050 con 0x55a283343590 2024-09-10T09:39:57.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:57.122+0000 7f04d9a93740 10 client.15684 fetch_fsmap learned FSMap version 30 2024-09-10T09:39:57.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:57.122+0000 7f04d9a93740 1 -- 172.21.15.5:0/2974196364 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55a2833c9290 con 0x55a283343590 2024-09-10T09:39:57.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:57.122+0000 7f04ba000640 1 -- 172.21.15.5:0/2974196364 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f04c80055b0 con 0x55a283343590 2024-09-10T09:39:57.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:57.122+0000 7f04ba000640 1 -- 172.21.15.5:0/2974196364 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f04bc004b30 con 0x55a283343590 2024-09-10T09:39:57.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:57.122+0000 7f04d9a93740 10 client.15684 fetch_fsmap finished waiting for FSMap version 30 2024-09-10T09:39:57.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:57.122+0000 7f04d9a93740 10 client.15684 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-09-10T09:39:57.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:57.122+0000 7f04d9a93740 20 client.15684 populate_metadata read hostname 'smithi005' 2024-09-10T09:39:57.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:57.122+0000 7f04d9a93740 1 --2- 172.21.15.5:0/2974196364 >> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] conn(0x55a2833c9a40 0x55a2833cbe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:39:57.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:57.122+0000 7f04d9a93740 4 client.15684 mds_command: new command op to 24443 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T09:39:57.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:57.122+0000 7f04d9a93740 1 -- 172.21.15.5:0/2974196364 --> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a2832b0e60 con 0x55a2833c9a40 2024-09-10T09:39:57.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:57.123+0000 7f04d5000640 1 --2- 172.21.15.5:0/2974196364 >> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] conn(0x55a2833c9a40 0x55a2833cbe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:39:57.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:57.123+0000 7f04d5000640 1 --2- 172.21.15.5:0/2974196364 >> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] conn(0x55a2833c9a40 0x55a2833cbe60 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=82e6819e4af7a37c server_cookie=221b2f5bc518293b in_seq=0 out_seq=0 2024-09-10T09:39:57.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:57.123+0000 7f04ba000640 10 client.15684 ms_handle_connect on v2:172.21.15.102:6832/2457007427 2024-09-10T09:39:57.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:57.123+0000 7f04ba000640 1 -- 172.21.15.5:0/2974196364 <== mds.4 v2:172.21.15.102:6832/2457007427 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a2832b0e60 con 0x55a2833c9a40 2024-09-10T09:39:57.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:57.123+0000 7f04ba000640 10 client.15684 handle_command_reply: tid=0 2024-09-10T09:39:57.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:57.123+0000 7f04d9a93740 1 client.15684 shutdown 2024-09-10T09:39:57.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:57.124+0000 7f04d9a93740 1 -- 172.21.15.5:0/2974196364 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a283343590 msgr2=0x55a283352730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:39:57.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:57.124+0000 7f04d9a93740 1 --2- 172.21.15.5:0/2974196364 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a283343590 0x55a283352730 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f04bc031460 tx=0x7f04bc005540 comp rx=0 tx=0).stop 2024-09-10T09:39:57.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:57.124+0000 7f04d9a93740 1 -- 172.21.15.5:0/2974196364 shutdown_connections 2024-09-10T09:39:57.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:57.124+0000 7f04d9a93740 1 --2- 172.21.15.5:0/2974196364 >> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] conn(0x55a2833c9a40 0x55a2833cbe60 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:57.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:57.124+0000 7f04d9a93740 1 --2- 172.21.15.5:0/2974196364 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a28334de50 0x55a2833c7b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:57.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:57.124+0000 7f04d9a93740 1 --2- 172.21.15.5:0/2974196364 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a283343590 0x55a283352730 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:57.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:57.124+0000 7f04d9a93740 1 --2- 172.21.15.5:0/2974196364 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a283342c70 0x55a2833521f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:57.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:57.124+0000 7f04d9a93740 1 -- 172.21.15.5:0/2974196364 >> 172.21.15.5:0/2974196364 conn(0x55a28333f200 msgr2=0x55a28334ac70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:39:57.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:57.124+0000 7f04ba000640 0 client.15684 ms_handle_reset on v2:172.21.15.102:6832/2457007427 2024-09-10T09:39:57.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:57.124+0000 7f04d9a93740 1 -- 172.21.15.5:0/2974196364 shutdown_connections 2024-09-10T09:39:57.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:57.124+0000 7f04d9a93740 1 -- 172.21.15.5:0/2974196364 wait complete. 2024-09-10T09:39:57.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:57.124+0000 7f04d9a93740 20 client.15684 trim_cache size 0 max 16384 2024-09-10T09:39:57.130 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-10T09:39:57.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:57.125+0000 7f04d7000640 1 -- 172.21.15.5:0/2324703590 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f04b40a2b80 msgr2=0x7f04b40a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:39:57.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:57.125+0000 7f04d7000640 1 --2- 172.21.15.5:0/2324703590 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f04b40a2b80 0x7f04b40a5040 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f04c00cd290 tx=0x7f04c00d5550 comp rx=0 tx=0).stop 2024-09-10T09:39:57.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:57.125+0000 7f04d7000640 1 -- 172.21.15.5:0/2324703590 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f04d0103ff0 msgr2=0x7f04d019a650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:39:57.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:57.125+0000 7f04d7000640 1 --2- 172.21.15.5:0/2324703590 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f04d0103ff0 0x7f04d019a650 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f04c000e9f0 tx=0x7f04c000edf0 comp rx=0 tx=0).stop 2024-09-10T09:39:57.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:57.125+0000 7f04d7000640 1 -- 172.21.15.5:0/2324703590 shutdown_connections 2024-09-10T09:39:57.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:57.125+0000 7f04d7000640 1 --2- 172.21.15.5:0/2324703590 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f04b40a2b80 0x7f04b40a5040 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:57.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:57.125+0000 7f04d7000640 1 --2- 172.21.15.5:0/2324703590 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f04d0109120 0x7f04d019ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:57.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:57.125+0000 7f04d7000640 1 --2- 172.21.15.5:0/2324703590 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f04d01049a0 0x7f04d019ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:57.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:57.125+0000 7f04d7000640 1 --2- 172.21.15.5:0/2324703590 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f04d0103ff0 0x7f04d019a650 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:57.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:57.125+0000 7f04d7000640 1 -- 172.21.15.5:0/2324703590 >> 172.21.15.5:0/2324703590 conn(0x7f04d00ffa90 msgr2=0x7f04d0100540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:39:57.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:57.125+0000 7f04d7000640 1 -- 172.21.15.5:0/2324703590 shutdown_connections 2024-09-10T09:39:57.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:57.125+0000 7f04d7000640 1 -- 172.21.15.5:0/2324703590 wait complete. 2024-09-10T09:39:57.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:39:56 smithi102 ceph-mon[35442]: pgmap v441: 129 pgs: 129 active+clean; 666 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 85 B/s wr, 12 op/s 2024-09-10T09:39:57.941 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-10T09:39:58.316 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:39:58.774 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:39:58 smithi005 ceph-mon[30350]: pgmap v442: 129 pgs: 129 active+clean; 666 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 9 op/s 2024-09-10T09:39:58.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:58.888+0000 7f73e3e00640 1 -- 172.21.15.5:0/206415296 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f73e41019c0 msgr2=0x7f73e4101e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:39:58.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:58.888+0000 7f73e3e00640 1 --2- 172.21.15.5:0/206415296 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f73e41019c0 0x7f73e4101e20 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f73d0009930 tx=0x7f73d002a420 comp rx=0 tx=0).stop 2024-09-10T09:39:58.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:58.889+0000 7f73e3e00640 1 -- 172.21.15.5:0/206415296 shutdown_connections 2024-09-10T09:39:58.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:58.889+0000 7f73e3e00640 1 --2- 172.21.15.5:0/206415296 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f73e4102360 0x7f73e410e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:58.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:58.889+0000 7f73e3e00640 1 --2- 172.21.15.5:0/206415296 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f73e41019c0 0x7f73e4101e20 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:58.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:58.889+0000 7f73e3e00640 1 --2- 172.21.15.5:0/206415296 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f73e41079c0 0x7f73e4107da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:58.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:58.889+0000 7f73e3e00640 1 -- 172.21.15.5:0/206415296 >> 172.21.15.5:0/206415296 conn(0x7f73e40fd6e0 msgr2=0x7f73e40ffb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:39:58.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:58.889+0000 7f73e3e00640 1 -- 172.21.15.5:0/206415296 shutdown_connections 2024-09-10T09:39:58.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:58.889+0000 7f73e3e00640 1 -- 172.21.15.5:0/206415296 wait complete. 2024-09-10T09:39:58.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:58.890+0000 7f73e3e00640 1 Processor -- start 2024-09-10T09:39:58.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:58.890+0000 7f73e3e00640 1 -- start start 2024-09-10T09:39:58.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:58.890+0000 7f73e3e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f73e41019c0 0x7f73e419a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:39:58.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:58.890+0000 7f73e3e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f73e4102360 0x7f73e419a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:39:58.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:58.890+0000 7f73e3e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f73e41079c0 0x7f73e419ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:39:58.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:58.890+0000 7f73e3e00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f73e4111dc0 con 0x7f73e41019c0 2024-09-10T09:39:58.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:58.890+0000 7f73e3e00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f73e4111c40 con 0x7f73e41079c0 2024-09-10T09:39:58.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:58.890+0000 7f73e3e00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f73e4111f40 con 0x7f73e4102360 2024-09-10T09:39:58.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:58.890+0000 7f73e2a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f73e41019c0 0x7f73e419a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:39:58.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:58.890+0000 7f73e2000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f73e4102360 0x7f73e419a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:39:58.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:58.890+0000 7f73e2a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f73e41019c0 0x7f73e419a460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34088/0 (socket says 172.21.15.5:34088) 2024-09-10T09:39:58.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:58.890+0000 7f73e2a00640 1 -- 172.21.15.5:0/1517049508 learned_addr learned my addr 172.21.15.5:0/1517049508 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:39:58.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:58.891+0000 7f73e3400640 1 --2- 172.21.15.5:0/1517049508 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f73e41079c0 0x7f73e419ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:39:58.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:58.891+0000 7f73e2a00640 1 -- 172.21.15.5:0/1517049508 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f73e4102360 msgr2=0x7f73e419a9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:39:58.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:58.891+0000 7f73e2a00640 1 --2- 172.21.15.5:0/1517049508 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f73e4102360 0x7f73e419a9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:58.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:58.891+0000 7f73e2a00640 1 -- 172.21.15.5:0/1517049508 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f73e41079c0 msgr2=0x7f73e419ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:39:58.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:58.891+0000 7f73e2a00640 1 --2- 172.21.15.5:0/1517049508 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f73e41079c0 0x7f73e419ed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:58.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:58.891+0000 7f73e2a00640 1 -- 172.21.15.5:0/1517049508 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73e419f410 con 0x7f73e41019c0 2024-09-10T09:39:58.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:58.891+0000 7f73e2a00640 1 --2- 172.21.15.5:0/1517049508 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f73e41019c0 0x7f73e419a460 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f73d400e9f0 tx=0x7f73d400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:39:58.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:58.891+0000 7f73db400640 1 -- 172.21.15.5:0/1517049508 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73d400cd80 con 0x7f73e41019c0 2024-09-10T09:39:58.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:58.892+0000 7f73e3e00640 1 -- 172.21.15.5:0/1517049508 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73e419f700 con 0x7f73e41019c0 2024-09-10T09:39:58.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:58.892+0000 7f73db400640 1 -- 172.21.15.5:0/1517049508 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f73d4010910 con 0x7f73e41019c0 2024-09-10T09:39:58.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:58.892+0000 7f73db400640 1 -- 172.21.15.5:0/1517049508 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73d40188f0 con 0x7f73e41019c0 2024-09-10T09:39:58.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:58.892+0000 7f73e3e00640 1 -- 172.21.15.5:0/1517049508 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73e41a6fe0 con 0x7f73e41019c0 2024-09-10T09:39:58.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:58.895+0000 7f73db400640 1 -- 172.21.15.5:0/1517049508 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f73d4010ac0 con 0x7f73e41019c0 2024-09-10T09:39:58.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:58.895+0000 7f73ec879740 1 Processor -- start 2024-09-10T09:39:58.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:58.895+0000 7f73ec879740 1 -- start start 2024-09-10T09:39:58.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:58.895+0000 7f73ec879740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562b9eebfe40 0x562b9eec0220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:39:58.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:58.895+0000 7f73ec879740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562b9eec0760 0x562b9eecaae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:39:58.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:58.895+0000 7f73ec879740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562b9eecb020 0x562b9eecd410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:39:58.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:58.895+0000 7f73ec879740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x562b9ee263e0 con 0x562b9eebfe40 2024-09-10T09:39:58.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:58.895+0000 7f73ec879740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x562b9edcdd80 con 0x562b9eecb020 2024-09-10T09:39:58.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:58.895+0000 7f73ec879740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x562b9ee223a0 con 0x562b9eec0760 2024-09-10T09:39:58.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:58.896+0000 7f73e3400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562b9eebfe40 0x562b9eec0220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:39:58.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:58.896+0000 7f73e3400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562b9eebfe40 0x562b9eec0220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34094/0 (socket says 172.21.15.5:34094) 2024-09-10T09:39:58.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:58.896+0000 7f73e3400640 1 -- 172.21.15.5:0/1196355990 learned_addr learned my addr 172.21.15.5:0/1196355990 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:39:58.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:58.896+0000 7f73e2a00640 1 --2- 172.21.15.5:0/1196355990 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562b9eec0760 0x562b9eecaae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:39:58.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:58.896+0000 7f73e2000640 1 --2- 172.21.15.5:0/1196355990 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562b9eecb020 0x562b9eecd410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:39:58.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:58.896+0000 7f73e3400640 1 -- 172.21.15.5:0/1196355990 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562b9eec0760 msgr2=0x562b9eecaae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:39:58.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:58.896+0000 7f73e3400640 1 --2- 172.21.15.5:0/1196355990 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562b9eec0760 0x562b9eecaae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:58.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:58.896+0000 7f73e3400640 1 -- 172.21.15.5:0/1196355990 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562b9eecb020 msgr2=0x562b9eecd410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:39:58.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:58.896+0000 7f73e3400640 1 --2- 172.21.15.5:0/1196355990 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562b9eecb020 0x562b9eecd410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:58.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:58.896+0000 7f73e3400640 1 -- 172.21.15.5:0/1196355990 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562b9edecae0 con 0x562b9eebfe40 2024-09-10T09:39:58.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:58.896+0000 7f73db400640 1 --2- 172.21.15.5:0/1517049508 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f73c80a2c00 0x7f73c80a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:39:58.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:58.896+0000 7f73e3400640 1 --2- 172.21.15.5:0/1196355990 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562b9eebfe40 0x562b9eec0220 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f73dc009fd0 tx=0x7f73dc0118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6af94b32fd8c8bb7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:39:58.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:58.896+0000 7f73db400640 1 -- 172.21.15.5:0/1517049508 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f73d40cf6f0 con 0x7f73e41019c0 2024-09-10T09:39:58.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:58.897+0000 7f73e2a00640 1 --2- 172.21.15.5:0/1517049508 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f73c80a2c00 0x7f73c80a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:39:58.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:58.897+0000 7f73cfe00640 1 -- 172.21.15.5:0/1196355990 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73dc004d20 con 0x562b9eebfe40 2024-09-10T09:39:58.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:58.897+0000 7f73cfe00640 1 -- 172.21.15.5:0/1196355990 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f73dc004050 con 0x562b9eebfe40 2024-09-10T09:39:58.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:58.897+0000 7f73cfe00640 1 -- 172.21.15.5:0/1196355990 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73dc005280 con 0x562b9eebfe40 2024-09-10T09:39:58.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:58.897+0000 7f73e2a00640 1 --2- 172.21.15.5:0/1517049508 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f73c80a2c00 0x7f73c80a50c0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f73d409b8b0 tx=0x7f73d40d3040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:39:58.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:58.897+0000 7f73ec879740 1 -- 172.21.15.5:0/1196355990 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562b9eebfe40 msgr2=0x562b9eec0220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:39:58.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:58.898+0000 7f73ec879740 1 --2- 172.21.15.5:0/1196355990 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562b9eebfe40 0x562b9eec0220 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f73dc009fd0 tx=0x7f73dc0118d0 comp rx=0 tx=0).stop 2024-09-10T09:39:58.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:58.898+0000 7f73ec879740 1 -- 172.21.15.5:0/1196355990 shutdown_connections 2024-09-10T09:39:58.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:58.898+0000 7f73ec879740 1 --2- 172.21.15.5:0/1196355990 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562b9eecb020 0x562b9eecd410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:58.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:58.898+0000 7f73ec879740 1 --2- 172.21.15.5:0/1196355990 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562b9eec0760 0x562b9eecaae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:58.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:58.898+0000 7f73ec879740 1 --2- 172.21.15.5:0/1196355990 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562b9eebfe40 0x562b9eec0220 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:58.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:58.898+0000 7f73ec879740 1 -- 172.21.15.5:0/1196355990 >> 172.21.15.5:0/1196355990 conn(0x562b9eebc3d0 msgr2=0x562b9eebc7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:39:58.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:58.898+0000 7f73ec879740 1 -- 172.21.15.5:0/1196355990 shutdown_connections 2024-09-10T09:39:58.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:58.898+0000 7f73ec879740 1 -- 172.21.15.5:0/1196355990 wait complete. 2024-09-10T09:39:58.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:58.898+0000 7f73ec879740 1 -- start start 2024-09-10T09:39:58.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:58.898+0000 7f73ec879740 1 Processor -- start 2024-09-10T09:39:58.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:58.899+0000 7f73ec879740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562b9eebfe40 0x562b9eec7ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:39:58.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:58.899+0000 7f73ec879740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562b9eec0760 0x562b9eec8400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:39:58.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:58.899+0000 7f73ec879740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562b9eecb020 0x562b9eec8940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:39:58.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:58.899+0000 7f73ec879740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x562b9eecdbf0 con 0x562b9eec0760 2024-09-10T09:39:58.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:58.899+0000 7f73ec879740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x562b9ede10f0 con 0x562b9eecb020 2024-09-10T09:39:58.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:58.899+0000 7f73ec879740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x562b9eecdd70 con 0x562b9eebfe40 2024-09-10T09:39:58.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:58.899+0000 7f73ec879740 10 client.0 _ll_register_callbacks cb 0x562b9edc8bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:39:58.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:58.899+0000 7f73e2000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562b9eec0760 0x562b9eec8400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:39:58.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:58.899+0000 7f73e2000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562b9eec0760 0x562b9eec8400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34108/0 (socket says 172.21.15.5:34108) 2024-09-10T09:39:58.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:58.899+0000 7f73e2000640 1 -- 172.21.15.5:0/1514906785 learned_addr learned my addr 172.21.15.5:0/1514906785 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:39:58.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:58.899+0000 7f73e3400640 1 --2- 172.21.15.5:0/1514906785 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562b9eecb020 0x562b9eec8940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:39:58.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:58.899+0000 7f73e3400640 1 --2- 172.21.15.5:0/1514906785 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562b9eebfe40 0x562b9eec7ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:39:58.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:58.900+0000 7f73e2000640 1 -- 172.21.15.5:0/1514906785 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562b9eebfe40 msgr2=0x562b9eec7ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:39:58.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:58.900+0000 7f73e2000640 1 --2- 172.21.15.5:0/1514906785 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562b9eebfe40 0x562b9eec7ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:58.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:58.900+0000 7f73e2000640 1 -- 172.21.15.5:0/1514906785 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562b9eecb020 msgr2=0x562b9eec8940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:39:58.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:58.900+0000 7f73e2000640 1 --2- 172.21.15.5:0/1514906785 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562b9eecb020 0x562b9eec8940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:58.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:58.900+0000 7f73e2000640 1 -- 172.21.15.5:0/1514906785 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562b9eb973c0 con 0x562b9eec0760 2024-09-10T09:39:58.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:58.900+0000 7f73e3400640 1 --2- 172.21.15.5:0/1514906785 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562b9eecb020 0x562b9eec8940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:39:58.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:58.900+0000 7f73e2000640 1 --2- 172.21.15.5:0/1514906785 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562b9eec0760 0x562b9eec8400 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f73d0009930 tx=0x7f73d0009680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c12973788c1560f5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:39:58.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:58.900+0000 7f73ccc00640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-10T09:39:58.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:58.900+0000 7f73ccc00640 1 -- 172.21.15.5:0/1514906785 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73d0035040 con 0x562b9eec0760 2024-09-10T09:39:58.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:58.900+0000 7f73ec879740 1 -- 172.21.15.5:0/1514906785 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f73dc004050 con 0x562b9eec0760 2024-09-10T09:39:58.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:58.900+0000 7f73ccc00640 1 -- 172.21.15.5:0/1514906785 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f73d0004250 con 0x562b9eec0760 2024-09-10T09:39:58.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:58.901+0000 7f73ccc00640 1 -- 172.21.15.5:0/1514906785 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f73d004d050 con 0x562b9eec0760 2024-09-10T09:39:58.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:58.901+0000 7f73ec879740 10 client.15696 fetch_fsmap learned FSMap version 30 2024-09-10T09:39:58.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:58.901+0000 7f73ec879740 1 -- 172.21.15.5:0/1514906785 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x562b9eed9c30 con 0x562b9eec0760 2024-09-10T09:39:58.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:58.901+0000 7f73ccc00640 1 -- 172.21.15.5:0/1514906785 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f73dc004050 con 0x562b9eec0760 2024-09-10T09:39:58.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:58.901+0000 7f73ccc00640 1 -- 172.21.15.5:0/1514906785 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f73d003c560 con 0x562b9eec0760 2024-09-10T09:39:58.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:58.901+0000 7f73ec879740 10 client.15696 fetch_fsmap finished waiting for FSMap version 30 2024-09-10T09:39:58.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:58.901+0000 7f73ec879740 10 client.15696 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-09-10T09:39:58.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:58.901+0000 7f73ec879740 20 client.15696 populate_metadata read hostname 'smithi005' 2024-09-10T09:39:58.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:58.901+0000 7f73ec879740 1 --2- 172.21.15.5:0/1514906785 >> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] conn(0x562b9eedb0f0 0x562b9eedd510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:39:58.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:58.901+0000 7f73ec879740 4 client.15696 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:39:58.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:58.901+0000 7f73ec879740 1 -- 172.21.15.5:0/1514906785 --> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562b9ee33950 con 0x562b9eedb0f0 2024-09-10T09:39:58.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:58.902+0000 7f73e3400640 1 --2- 172.21.15.5:0/1514906785 >> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] conn(0x562b9eedb0f0 0x562b9eedd510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:39:58.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:58.902+0000 7f73e3400640 1 --2- 172.21.15.5:0/1514906785 >> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] conn(0x562b9eedb0f0 0x562b9eedd510 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=4822f6c63e63f7d3 server_cookie=5688761357df7b7f in_seq=0 out_seq=0 2024-09-10T09:39:58.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:58.902+0000 7f73ccc00640 10 client.15696 ms_handle_connect on v2:172.21.15.102:6836/2689776764 2024-09-10T09:39:58.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:58.903+0000 7f73ccc00640 1 -- 172.21.15.5:0/1514906785 <== mds.3 v2:172.21.15.102:6836/2689776764 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562b9ee33950 con 0x562b9eedb0f0 2024-09-10T09:39:58.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:58.903+0000 7f73ccc00640 10 client.15696 handle_command_reply: tid=0 2024-09-10T09:39:58.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:58.903+0000 7f73ec879740 1 client.15696 shutdown 2024-09-10T09:39:58.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:58.904+0000 7f73ec879740 1 -- 172.21.15.5:0/1514906785 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562b9eec0760 msgr2=0x562b9eec8400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:39:58.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:58.904+0000 7f73ec879740 1 --2- 172.21.15.5:0/1514906785 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562b9eec0760 0x562b9eec8400 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f73d0009930 tx=0x7f73d0009680 comp rx=0 tx=0).stop 2024-09-10T09:39:58.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:58.904+0000 7f73ec879740 1 -- 172.21.15.5:0/1514906785 shutdown_connections 2024-09-10T09:39:58.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:58.904+0000 7f73ec879740 1 --2- 172.21.15.5:0/1514906785 >> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] conn(0x562b9eedb0f0 0x562b9eedd510 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:58.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:58.904+0000 7f73ec879740 1 --2- 172.21.15.5:0/1514906785 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562b9eecb020 0x562b9eec8940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:58.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:58.904+0000 7f73ec879740 1 --2- 172.21.15.5:0/1514906785 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562b9eec0760 0x562b9eec8400 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:58.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:58.904+0000 7f73ec879740 1 --2- 172.21.15.5:0/1514906785 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562b9eebfe40 0x562b9eec7ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:58.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:58.904+0000 7f73ccc00640 0 client.15696 ms_handle_reset on v2:172.21.15.102:6836/2689776764 2024-09-10T09:39:58.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:58.904+0000 7f73ec879740 1 -- 172.21.15.5:0/1514906785 >> 172.21.15.5:0/1514906785 conn(0x562b9eebc3d0 msgr2=0x562b9eebb4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:39:58.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:58.904+0000 7f73ec879740 1 -- 172.21.15.5:0/1514906785 shutdown_connections 2024-09-10T09:39:58.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:58.904+0000 7f73ec879740 1 -- 172.21.15.5:0/1514906785 wait complete. 2024-09-10T09:39:58.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:58.904+0000 7f73ec879740 20 client.15696 trim_cache size 0 max 16384 2024-09-10T09:39:58.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:58.918+0000 7f73ec879740 1 Processor -- start 2024-09-10T09:39:58.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:58.918+0000 7f73ec879740 1 -- start start 2024-09-10T09:39:58.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:58.918+0000 7f73ec879740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562b9eebfe40 0x562b9eec8790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:39:58.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:58.918+0000 7f73ec879740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562b9eec0760 0x562b9eee2de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:39:58.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:58.918+0000 7f73ec879740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562b9eecb020 0x562b9eee3320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:39:58.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:58.918+0000 7f73ec879740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f73e41b0ea0 con 0x562b9eecb020 2024-09-10T09:39:58.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:58.918+0000 7f73ec879740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f73e41b29e0 con 0x562b9eebfe40 2024-09-10T09:39:58.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:58.918+0000 7f73ec879740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f73e41b1570 con 0x562b9eec0760 2024-09-10T09:39:58.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:58.918+0000 7f73e3400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562b9eecb020 0x562b9eee3320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:39:58.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:58.918+0000 7f73e3400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562b9eebfe40 0x562b9eec8790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:39:58.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:58.918+0000 7f73e3400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562b9eebfe40 0x562b9eec8790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:55324/0 (socket says 172.21.15.5:55324) 2024-09-10T09:39:58.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:58.918+0000 7f73e3400640 1 -- 172.21.15.5:0/2284296377 learned_addr learned my addr 172.21.15.5:0/2284296377 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:39:58.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:58.918+0000 7f73e2000640 1 --2- 172.21.15.5:0/2284296377 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562b9eec0760 0x562b9eee2de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:39:58.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:58.919+0000 7f73e3400640 1 -- 172.21.15.5:0/2284296377 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562b9eec0760 msgr2=0x562b9eee2de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:39:58.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:58.919+0000 7f73e3400640 1 --2- 172.21.15.5:0/2284296377 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562b9eec0760 0x562b9eee2de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:58.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:58.919+0000 7f73e3400640 1 -- 172.21.15.5:0/2284296377 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562b9eecb020 msgr2=0x562b9eee3320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:39:58.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:58.919+0000 7f73e3400640 1 --2- 172.21.15.5:0/2284296377 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562b9eecb020 0x562b9eee3320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:58.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:58.919+0000 7f73e3400640 1 -- 172.21.15.5:0/2284296377 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562b9eebd060 con 0x562b9eebfe40 2024-09-10T09:39:58.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:58.919+0000 7f73e3400640 1 --2- 172.21.15.5:0/2284296377 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562b9eebfe40 0x562b9eec8790 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f73dc006010 tx=0x7f73dc005bd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ed4dd9226ea13106 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:39:58.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:58.919+0000 7f73cf400640 1 -- 172.21.15.5:0/2284296377 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73dc004490 con 0x562b9eebfe40 2024-09-10T09:39:58.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:58.919+0000 7f73cf400640 1 -- 172.21.15.5:0/2284296377 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f73dc004630 con 0x562b9eebfe40 2024-09-10T09:39:58.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:58.919+0000 7f73cf400640 1 -- 172.21.15.5:0/2284296377 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73dc0326b0 con 0x562b9eebfe40 2024-09-10T09:39:58.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:58.920+0000 7f73ec879740 1 -- 172.21.15.5:0/2284296377 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562b9eebfe40 msgr2=0x562b9eec8790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:39:58.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:58.920+0000 7f73ec879740 1 --2- 172.21.15.5:0/2284296377 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562b9eebfe40 0x562b9eec8790 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f73dc006010 tx=0x7f73dc005bd0 comp rx=0 tx=0).stop 2024-09-10T09:39:58.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:58.920+0000 7f73ec879740 1 -- 172.21.15.5:0/2284296377 shutdown_connections 2024-09-10T09:39:58.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:58.920+0000 7f73ec879740 1 --2- 172.21.15.5:0/2284296377 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562b9eecb020 0x562b9eee3320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:58.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:58.920+0000 7f73ec879740 1 --2- 172.21.15.5:0/2284296377 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562b9eec0760 0x562b9eee2de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:58.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:58.920+0000 7f73ec879740 1 --2- 172.21.15.5:0/2284296377 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562b9eebfe40 0x562b9eec8790 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:58.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:58.920+0000 7f73ec879740 1 -- 172.21.15.5:0/2284296377 >> 172.21.15.5:0/2284296377 conn(0x562b9eebc3d0 msgr2=0x562b9eec6ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:39:58.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:58.920+0000 7f73ec879740 1 -- 172.21.15.5:0/2284296377 shutdown_connections 2024-09-10T09:39:58.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:58.920+0000 7f73ec879740 1 -- 172.21.15.5:0/2284296377 wait complete. 2024-09-10T09:39:58.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:58.921+0000 7f73ec879740 1 -- start start 2024-09-10T09:39:58.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:58.921+0000 7f73ec879740 1 Processor -- start 2024-09-10T09:39:58.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:58.921+0000 7f73ec879740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562b9eebfe40 0x562b9eec50d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:39:58.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:58.921+0000 7f73ec879740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562b9eec0760 0x562b9eec5610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:39:58.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:58.921+0000 7f73ec879740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562b9eecb020 0x562b9eec5b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:39:58.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:58.921+0000 7f73ec879740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f73e41b2310 con 0x562b9eebfe40 2024-09-10T09:39:58.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:58.921+0000 7f73ec879740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f73e41b1c40 con 0x562b9eecb020 2024-09-10T09:39:58.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:58.921+0000 7f73ec879740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f73e41aee80 con 0x562b9eec0760 2024-09-10T09:39:58.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:58.921+0000 7f73ec879740 10 client.0 _ll_register_callbacks cb 0x7f73d4005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:39:58.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:58.922+0000 7f73e3400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562b9eebfe40 0x562b9eec50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:39:58.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:58.922+0000 7f73e2000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562b9eec0760 0x562b9eec5610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:39:58.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:58.922+0000 7f73e3400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562b9eebfe40 0x562b9eec50d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34124/0 (socket says 172.21.15.5:34124) 2024-09-10T09:39:58.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:58.922+0000 7f73e3400640 1 -- 172.21.15.5:0/1284092704 learned_addr learned my addr 172.21.15.5:0/1284092704 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:39:58.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:58.922+0000 7f73e3400640 1 --2- 172.21.15.5:0/1284092704 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562b9eecb020 0x562b9eec5b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:39:58.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:58.922+0000 7f73e2000640 1 -- 172.21.15.5:0/1284092704 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562b9eecb020 msgr2=0x562b9eec5b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:39:58.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:58.922+0000 7f73e2000640 1 --2- 172.21.15.5:0/1284092704 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562b9eecb020 0x562b9eec5b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:58.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:58.922+0000 7f73e2000640 1 -- 172.21.15.5:0/1284092704 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562b9eebfe40 msgr2=0x562b9eec50d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:39:58.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:58.922+0000 7f73e2000640 1 --2- 172.21.15.5:0/1284092704 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562b9eebfe40 0x562b9eec50d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:58.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:58.922+0000 7f73e2000640 1 -- 172.21.15.5:0/1284092704 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562b9ef44fb0 con 0x562b9eec0760 2024-09-10T09:39:58.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:58.922+0000 7f73e3400640 1 --2- 172.21.15.5:0/1284092704 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562b9eebfe40 0x562b9eec50d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:39:58.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:58.922+0000 7f73e2000640 1 --2- 172.21.15.5:0/1284092704 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562b9eec0760 0x562b9eec5610 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f73d0009900 tx=0x7f73d002a420 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dd59d4be73d86e1b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:39:58.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:58.922+0000 7f73ccc00640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-09-10T09:39:58.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:58.923+0000 7f73ccc00640 1 -- 172.21.15.5:0/1284092704 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73d0035040 con 0x562b9eec0760 2024-09-10T09:39:58.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:58.923+0000 7f73ccc00640 1 -- 172.21.15.5:0/1284092704 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f73d004a070 con 0x562b9eec0760 2024-09-10T09:39:58.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:58.923+0000 7f73ec879740 1 -- 172.21.15.5:0/1284092704 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f73dc004630 con 0x562b9eec0760 2024-09-10T09:39:58.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:58.923+0000 7f73ccc00640 1 -- 172.21.15.5:0/1284092704 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f73d005f050 con 0x562b9eec0760 2024-09-10T09:39:58.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:58.923+0000 7f73ec879740 10 client.25738 fetch_fsmap learned FSMap version 30 2024-09-10T09:39:58.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:58.923+0000 7f73ec879740 1 -- 172.21.15.5:0/1284092704 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x562b9ef46250 con 0x562b9eec0760 2024-09-10T09:39:58.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:58.923+0000 7f73ccc00640 1 -- 172.21.15.5:0/1284092704 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f73dc004630 con 0x562b9eec0760 2024-09-10T09:39:58.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:58.924+0000 7f73ccc00640 1 -- 172.21.15.5:0/1284092704 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f73d0053ab0 con 0x562b9eec0760 2024-09-10T09:39:58.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:58.924+0000 7f73ec879740 10 client.25738 fetch_fsmap finished waiting for FSMap version 30 2024-09-10T09:39:58.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:58.924+0000 7f73ec879740 10 client.25738 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-09-10T09:39:58.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:58.924+0000 7f73ec879740 20 client.25738 populate_metadata read hostname 'smithi005' 2024-09-10T09:39:58.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:58.924+0000 7f73ec879740 1 --2- 172.21.15.5:0/1284092704 >> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] conn(0x562b9ef46970 0x562b9ef48d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:39:58.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:58.924+0000 7f73ec879740 4 client.25738 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T09:39:58.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:58.924+0000 7f73ec879740 1 -- 172.21.15.5:0/1284092704 --> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562b9eeb79a0 con 0x562b9ef46970 2024-09-10T09:39:58.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:58.924+0000 7f73e3400640 1 --2- 172.21.15.5:0/1284092704 >> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] conn(0x562b9ef46970 0x562b9ef48d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:39:58.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:58.924+0000 7f73e3400640 1 --2- 172.21.15.5:0/1284092704 >> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] conn(0x562b9ef46970 0x562b9ef48d90 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=5aa373ffd79ceb92 server_cookie=bbecf06d1188561c in_seq=0 out_seq=0 2024-09-10T09:39:58.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:58.924+0000 7f73ccc00640 10 client.25738 ms_handle_connect on v2:172.21.15.102:6836/2689776764 2024-09-10T09:39:58.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:58.925+0000 7f73ccc00640 1 -- 172.21.15.5:0/1284092704 <== mds.3 v2:172.21.15.102:6836/2689776764 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562b9eeb79a0 con 0x562b9ef46970 2024-09-10T09:39:58.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:58.925+0000 7f73ccc00640 10 client.25738 handle_command_reply: tid=0 2024-09-10T09:39:58.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:58.925+0000 7f73ec879740 1 client.25738 shutdown 2024-09-10T09:39:58.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:58.925+0000 7f73ec879740 1 -- 172.21.15.5:0/1284092704 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562b9eec0760 msgr2=0x562b9eec5610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:39:58.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:58.925+0000 7f73ec879740 1 --2- 172.21.15.5:0/1284092704 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562b9eec0760 0x562b9eec5610 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f73d0009900 tx=0x7f73d002a420 comp rx=0 tx=0).stop 2024-09-10T09:39:58.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:58.925+0000 7f73ec879740 1 -- 172.21.15.5:0/1284092704 shutdown_connections 2024-09-10T09:39:58.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:58.925+0000 7f73ec879740 1 --2- 172.21.15.5:0/1284092704 >> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] conn(0x562b9ef46970 0x562b9ef48d90 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:58.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:58.925+0000 7f73ec879740 1 --2- 172.21.15.5:0/1284092704 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562b9eecb020 0x562b9eec5b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:58.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:58.925+0000 7f73ec879740 1 --2- 172.21.15.5:0/1284092704 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562b9eec0760 0x562b9eec5610 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:58.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:58.925+0000 7f73ec879740 1 --2- 172.21.15.5:0/1284092704 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562b9eebfe40 0x562b9eec50d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:58.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:58.925+0000 7f73ccc00640 0 client.25738 ms_handle_reset on v2:172.21.15.102:6836/2689776764 2024-09-10T09:39:58.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:58.925+0000 7f73ec879740 1 -- 172.21.15.5:0/1284092704 >> 172.21.15.5:0/1284092704 conn(0x562b9eebc3d0 msgr2=0x562b9eec1890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:39:58.950 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-10T09:39:58.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:58.925+0000 7f73ec879740 1 -- 172.21.15.5:0/1284092704 shutdown_connections 2024-09-10T09:39:58.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:58.925+0000 7f73ec879740 1 -- 172.21.15.5:0/1284092704 wait complete. 2024-09-10T09:39:58.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:58.925+0000 7f73ec879740 20 client.25738 trim_cache size 0 max 16384 2024-09-10T09:39:58.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:58.926+0000 7f73e3e00640 1 -- 172.21.15.5:0/1517049508 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f73c80a2c00 msgr2=0x7f73c80a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:39:58.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:58.926+0000 7f73e3e00640 1 --2- 172.21.15.5:0/1517049508 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f73c80a2c00 0x7f73c80a50c0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f73d409b8b0 tx=0x7f73d40d3040 comp rx=0 tx=0).stop 2024-09-10T09:39:58.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:58.926+0000 7f73e3e00640 1 -- 172.21.15.5:0/1517049508 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f73e41019c0 msgr2=0x7f73e419a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:39:58.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:58.926+0000 7f73e3e00640 1 --2- 172.21.15.5:0/1517049508 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f73e41019c0 0x7f73e419a460 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f73d400e9f0 tx=0x7f73d400eeb0 comp rx=0 tx=0).stop 2024-09-10T09:39:58.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:58.926+0000 7f73e3e00640 1 -- 172.21.15.5:0/1517049508 shutdown_connections 2024-09-10T09:39:58.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:58.926+0000 7f73e3e00640 1 --2- 172.21.15.5:0/1517049508 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f73c80a2c00 0x7f73c80a50c0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:58.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:58.926+0000 7f73e3e00640 1 --2- 172.21.15.5:0/1517049508 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f73e41079c0 0x7f73e419ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:58.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:58.926+0000 7f73e3e00640 1 --2- 172.21.15.5:0/1517049508 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f73e4102360 0x7f73e419a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:58.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:58.926+0000 7f73e3e00640 1 --2- 172.21.15.5:0/1517049508 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f73e41019c0 0x7f73e419a460 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:39:58.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:58.926+0000 7f73e3e00640 1 -- 172.21.15.5:0/1517049508 >> 172.21.15.5:0/1517049508 conn(0x7f73e40fd6e0 msgr2=0x7f73e410cce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:39:58.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:58.926+0000 7f73e3e00640 1 -- 172.21.15.5:0/1517049508 shutdown_connections 2024-09-10T09:39:58.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:39:58.926+0000 7f73e3e00640 1 -- 172.21.15.5:0/1517049508 wait complete. 2024-09-10T09:39:59.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:39:58 smithi159 ceph-mon[35513]: pgmap v442: 129 pgs: 129 active+clean; 666 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 9 op/s 2024-09-10T09:39:59.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:39:58 smithi102 ceph-mon[35442]: pgmap v442: 129 pgs: 129 active+clean; 666 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 9 op/s 2024-09-10T09:39:59.968 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-10T09:40:00.349 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:40:00.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:40:00 smithi005 ceph-mon[30350]: pgmap v443: 129 pgs: 129 active+clean; 666 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 85 B/s wr, 9 op/s 2024-09-10T09:40:00.807 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:40:00 smithi005 ceph-mon[30350]: overall HEALTH_OK 2024-09-10T09:40:00.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:00.937+0000 7f74a3600640 1 -- 172.21.15.5:0/2946323353 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f749c101df0 msgr2=0x7f749c1021d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:40:00.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:00.937+0000 7f74a3600640 1 --2- 172.21.15.5:0/2946323353 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f749c101df0 0x7f749c1021d0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f7490009ca0 tx=0x7f749002a8c0 comp rx=0 tx=0).stop 2024-09-10T09:40:00.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:00.937+0000 7f74a3600640 1 -- 172.21.15.5:0/2946323353 shutdown_connections 2024-09-10T09:40:00.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:00.937+0000 7f74a3600640 1 --2- 172.21.15.5:0/2946323353 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f749c106f20 0x7f749c10eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:00.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:00.937+0000 7f74a3600640 1 --2- 172.21.15.5:0/2946323353 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f749c1027a0 0x7f749c1067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:00.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:00.937+0000 7f74a3600640 1 --2- 172.21.15.5:0/2946323353 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f749c101df0 0x7f749c1021d0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:00.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:00.937+0000 7f74a3600640 1 -- 172.21.15.5:0/2946323353 >> 172.21.15.5:0/2946323353 conn(0x7f749c0fd8b0 msgr2=0x7f749c0ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:40:00.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:00.937+0000 7f74a3600640 1 -- 172.21.15.5:0/2946323353 shutdown_connections 2024-09-10T09:40:00.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:00.938+0000 7f74a3600640 1 -- 172.21.15.5:0/2946323353 wait complete. 2024-09-10T09:40:00.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:00.938+0000 7f74a3600640 1 Processor -- start 2024-09-10T09:40:00.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:00.939+0000 7f74a3600640 1 -- start start 2024-09-10T09:40:00.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:00.939+0000 7f74a3600640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f749c101df0 0x7f749c0744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:40:00.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:00.939+0000 7f74a3600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f749c1027a0 0x7f749c074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:40:00.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:00.939+0000 7f74a3600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f749c106f20 0x7f749c074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:40:00.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:00.939+0000 7f74a3600640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f749c111fd0 con 0x7f749c1027a0 2024-09-10T09:40:00.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:00.939+0000 7f74a3600640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f749c111e50 con 0x7f749c106f20 2024-09-10T09:40:00.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:00.939+0000 7f74a3600640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f749c112150 con 0x7f749c101df0 2024-09-10T09:40:00.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:00.939+0000 7f749be00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f749c1027a0 0x7f749c074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:40:00.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:00.939+0000 7f74a0c00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f749c101df0 0x7f749c0744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:40:00.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:00.939+0000 7f749be00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f749c1027a0 0x7f749c074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51124/0 (socket says 172.21.15.5:51124) 2024-09-10T09:40:00.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:00.939+0000 7f749be00640 1 -- 172.21.15.5:0/263404074 learned_addr learned my addr 172.21.15.5:0/263404074 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:40:00.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:00.940+0000 7f74a1600640 1 --2- 172.21.15.5:0/263404074 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f749c106f20 0x7f749c074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:40:00.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:00.940+0000 7f749be00640 1 -- 172.21.15.5:0/263404074 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f749c101df0 msgr2=0x7f749c0744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:40:00.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:00.940+0000 7f749be00640 1 --2- 172.21.15.5:0/263404074 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f749c101df0 0x7f749c0744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:00.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:00.940+0000 7f749be00640 1 -- 172.21.15.5:0/263404074 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f749c106f20 msgr2=0x7f749c074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:40:00.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:00.940+0000 7f749be00640 1 --2- 172.21.15.5:0/263404074 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f749c106f20 0x7f749c074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:00.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:00.940+0000 7f749be00640 1 -- 172.21.15.5:0/263404074 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f749c1a68b0 con 0x7f749c1027a0 2024-09-10T09:40:00.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:00.940+0000 7f74a1600640 1 --2- 172.21.15.5:0/263404074 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f749c106f20 0x7f749c074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:40:00.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:00.940+0000 7f749be00640 1 --2- 172.21.15.5:0/263404074 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f749c1027a0 0x7f749c074a30 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f748800e9f0 tx=0x7f748800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:40:00.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:00.940+0000 7f7499600640 1 -- 172.21.15.5:0/263404074 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f748800cd80 con 0x7f749c1027a0 2024-09-10T09:40:00.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:00.940+0000 7f7499600640 1 -- 172.21.15.5:0/263404074 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7488010910 con 0x7f749c1027a0 2024-09-10T09:40:00.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:00.941+0000 7f74a3600640 1 -- 172.21.15.5:0/263404074 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f749c1a6ba0 con 0x7f749c1027a0 2024-09-10T09:40:00.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:00.941+0000 7f7499600640 1 -- 172.21.15.5:0/263404074 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7488018990 con 0x7f749c1027a0 2024-09-10T09:40:00.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:00.941+0000 7f74a3600640 1 -- 172.21.15.5:0/263404074 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f749c1a70e0 con 0x7f749c1027a0 2024-09-10T09:40:00.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:00.944+0000 7f74a604e740 1 Processor -- start 2024-09-10T09:40:00.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:00.944+0000 7f74a604e740 1 -- start start 2024-09-10T09:40:00.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:00.944+0000 7f74a604e740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56355a4b6c70 0x56355a4b7050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:40:00.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:00.944+0000 7f74a604e740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56355a4b7590 0x56355a4c1910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:40:00.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:00.944+0000 7f7499600640 1 -- 172.21.15.5:0/263404074 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f7488010ac0 con 0x7f749c1027a0 2024-09-10T09:40:00.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:00.944+0000 7f74a604e740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56355a4c1e50 0x56355a4c4240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:40:00.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:00.944+0000 7f74a604e740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x56355a41d3e0 con 0x56355a4c1e50 2024-09-10T09:40:00.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:00.944+0000 7f74a604e740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x56355a3c4d80 con 0x56355a4b6c70 2024-09-10T09:40:00.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:00.944+0000 7f74a604e740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x56355a4193a0 con 0x56355a4b7590 2024-09-10T09:40:00.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:00.945+0000 7f749be00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56355a4c1e50 0x56355a4c4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:40:00.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:00.945+0000 7f74a1600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56355a4b6c70 0x56355a4b7050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:40:00.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:00.945+0000 7f749be00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56355a4c1e50 0x56355a4c4240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51130/0 (socket says 172.21.15.5:51130) 2024-09-10T09:40:00.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:00.945+0000 7f749be00640 1 -- 172.21.15.5:0/2237186326 learned_addr learned my addr 172.21.15.5:0/2237186326 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:40:00.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:00.945+0000 7f74a0c00640 1 --2- 172.21.15.5:0/2237186326 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56355a4b7590 0x56355a4c1910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:40:00.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:00.945+0000 7f749be00640 1 -- 172.21.15.5:0/2237186326 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56355a4b7590 msgr2=0x56355a4c1910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:40:00.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:00.945+0000 7f749be00640 1 --2- 172.21.15.5:0/2237186326 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56355a4b7590 0x56355a4c1910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:00.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:00.945+0000 7f749be00640 1 -- 172.21.15.5:0/2237186326 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56355a4b6c70 msgr2=0x56355a4b7050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:40:00.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:00.945+0000 7f749be00640 1 --2- 172.21.15.5:0/2237186326 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56355a4b6c70 0x56355a4b7050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:00.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:00.945+0000 7f749be00640 1 -- 172.21.15.5:0/2237186326 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56355a3e3ae0 con 0x56355a4c1e50 2024-09-10T09:40:00.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:00.946+0000 7f749be00640 1 --2- 172.21.15.5:0/2237186326 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56355a4c1e50 0x56355a4c4240 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f74880d5c20 tx=0x7f74880d7670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f812f6f96a02379 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:40:00.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:00.946+0000 7f748dc00640 1 -- 172.21.15.5:0/2237186326 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74880de070 con 0x56355a4c1e50 2024-09-10T09:40:00.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:00.946+0000 7f748dc00640 1 -- 172.21.15.5:0/2237186326 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f74880d17b0 con 0x56355a4c1e50 2024-09-10T09:40:00.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:00.946+0000 7f748dc00640 1 -- 172.21.15.5:0/2237186326 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74880dabd0 con 0x56355a4c1e50 2024-09-10T09:40:00.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:00.946+0000 7f7499600640 1 --2- 172.21.15.5:0/263404074 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f74840a2b80 0x7f74840a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:40:00.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:00.946+0000 7f7499600640 1 -- 172.21.15.5:0/263404074 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f74880cf830 con 0x7f749c1027a0 2024-09-10T09:40:00.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:00.946+0000 7f74a1600640 1 --2- 172.21.15.5:0/263404074 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f74840a2b80 0x7f74840a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:40:00.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:00.947+0000 7f74a604e740 1 -- 172.21.15.5:0/2237186326 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56355a4c1e50 msgr2=0x56355a4c4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:40:00.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:00.947+0000 7f74a604e740 1 --2- 172.21.15.5:0/2237186326 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56355a4c1e50 0x56355a4c4240 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f74880d5c20 tx=0x7f74880d7670 comp rx=0 tx=0).stop 2024-09-10T09:40:00.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:00.947+0000 7f74a604e740 1 -- 172.21.15.5:0/2237186326 shutdown_connections 2024-09-10T09:40:00.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:00.947+0000 7f74a604e740 1 --2- 172.21.15.5:0/2237186326 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56355a4c1e50 0x56355a4c4240 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:00.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:00.947+0000 7f74a604e740 1 --2- 172.21.15.5:0/2237186326 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56355a4b7590 0x56355a4c1910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:00.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:00.947+0000 7f74a604e740 1 --2- 172.21.15.5:0/2237186326 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56355a4b6c70 0x56355a4b7050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:00.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:00.947+0000 7f74a604e740 1 -- 172.21.15.5:0/2237186326 >> 172.21.15.5:0/2237186326 conn(0x56355a4b3200 msgr2=0x56355a4b35e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:40:00.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:00.947+0000 7f74a1600640 1 --2- 172.21.15.5:0/263404074 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f74840a2b80 0x7f74840a5040 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f749400bce0 tx=0x7f749400b4d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:40:00.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:00.947+0000 7f74a604e740 1 -- 172.21.15.5:0/2237186326 shutdown_connections 2024-09-10T09:40:00.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:00.947+0000 7f74a604e740 1 -- 172.21.15.5:0/2237186326 wait complete. 2024-09-10T09:40:00.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:00.948+0000 7f74a604e740 1 -- start start 2024-09-10T09:40:00.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:00.948+0000 7f74a604e740 1 Processor -- start 2024-09-10T09:40:00.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:00.948+0000 7f74a604e740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56355a4b6c70 0x56355a4becf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:40:00.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:00.948+0000 7f74a604e740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56355a4b7590 0x56355a4bf230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:40:00.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:00.948+0000 7f74a604e740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56355a4c1e50 0x56355a4bf770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:40:00.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:00.949+0000 7f74a604e740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x56355a4c4a20 con 0x56355a4c1e50 2024-09-10T09:40:00.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:00.949+0000 7f74a604e740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x56355a3d80f0 con 0x56355a4b6c70 2024-09-10T09:40:00.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:00.949+0000 7f74a604e740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x56355a4c4ba0 con 0x56355a4b7590 2024-09-10T09:40:00.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:00.949+0000 7f74a604e740 10 client.0 _ll_register_callbacks cb 0x56355a3bfbd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:40:00.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:00.949+0000 7f74a1600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56355a4c1e50 0x56355a4bf770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:40:00.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:00.949+0000 7f74a0c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56355a4b6c70 0x56355a4becf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:40:00.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:00.949+0000 7f74a1600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56355a4c1e50 0x56355a4bf770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51132/0 (socket says 172.21.15.5:51132) 2024-09-10T09:40:00.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:00.949+0000 7f74a1600640 1 -- 172.21.15.5:0/860575465 learned_addr learned my addr 172.21.15.5:0/860575465 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:40:00.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:00.949+0000 7f749be00640 1 --2- 172.21.15.5:0/860575465 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56355a4b7590 0x56355a4bf230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:40:00.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:00.949+0000 7f74a0c00640 1 -- 172.21.15.5:0/860575465 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56355a4b7590 msgr2=0x56355a4bf230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:40:00.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:00.949+0000 7f74a0c00640 1 --2- 172.21.15.5:0/860575465 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56355a4b7590 0x56355a4bf230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:00.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:00.949+0000 7f74a0c00640 1 -- 172.21.15.5:0/860575465 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56355a4c1e50 msgr2=0x56355a4bf770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:40:00.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:00.949+0000 7f74a0c00640 1 --2- 172.21.15.5:0/860575465 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56355a4c1e50 0x56355a4bf770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:00.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:00.949+0000 7f74a0c00640 1 -- 172.21.15.5:0/860575465 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56355a18e3c0 con 0x56355a4b6c70 2024-09-10T09:40:00.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:00.949+0000 7f74a1600640 1 --2- 172.21.15.5:0/860575465 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56355a4c1e50 0x56355a4bf770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:40:00.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:00.949+0000 7f749be00640 1 --2- 172.21.15.5:0/860575465 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56355a4b7590 0x56355a4bf230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:40:00.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:00.950+0000 7f74a0c00640 1 --2- 172.21.15.5:0/860575465 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56355a4b6c70 0x56355a4becf0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f7490000f80 tx=0x7f7490003ba0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ab9eee5c6fecf978 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:40:00.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:00.950+0000 7f7482000640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-09-10T09:40:00.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:00.950+0000 7f7482000640 1 -- 172.21.15.5:0/860575465 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7490031450 con 0x56355a4b6c70 2024-09-10T09:40:00.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:00.950+0000 7f74a604e740 1 -- 172.21.15.5:0/860575465 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f74880d17b0 con 0x56355a4b6c70 2024-09-10T09:40:00.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:00.950+0000 7f7482000640 1 -- 172.21.15.5:0/860575465 <== mon.2 v2:172.21.15.102:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f74900315f0 con 0x56355a4b6c70 2024-09-10T09:40:00.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:00.950+0000 7f7482000640 1 -- 172.21.15.5:0/860575465 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f74900317f0 con 0x56355a4b6c70 2024-09-10T09:40:00.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:00.950+0000 7f74a604e740 10 client.25721 fetch_fsmap learned FSMap version 30 2024-09-10T09:40:00.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:00.950+0000 7f74a604e740 1 -- 172.21.15.5:0/860575465 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x56355a4d0a60 con 0x56355a4b6c70 2024-09-10T09:40:00.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:00.951+0000 7f7482000640 1 -- 172.21.15.5:0/860575465 <== mon.2 v2:172.21.15.102:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f74880d17b0 con 0x56355a4b6c70 2024-09-10T09:40:00.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:00.951+0000 7f7482000640 1 -- 172.21.15.5:0/860575465 <== mon.2 v2:172.21.15.102:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f7490004640 con 0x56355a4b6c70 2024-09-10T09:40:00.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:00.951+0000 7f74a604e740 10 client.25721 fetch_fsmap finished waiting for FSMap version 30 2024-09-10T09:40:00.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:00.951+0000 7f74a604e740 10 client.25721 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-10T09:40:00.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:00.951+0000 7f74a604e740 20 client.25721 populate_metadata read hostname 'smithi005' 2024-09-10T09:40:00.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:00.951+0000 7f74a604e740 1 --2- 172.21.15.5:0/860575465 >> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] conn(0x56355a4d1e70 0x56355a4d4260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:40:00.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:00.951+0000 7f74a604e740 4 client.25721 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:40:00.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:00.951+0000 7f74a604e740 1 -- 172.21.15.5:0/860575465 --> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56355a42a950 con 0x56355a4d1e70 2024-09-10T09:40:00.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:00.951+0000 7f749be00640 1 --2- 172.21.15.5:0/860575465 >> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] conn(0x56355a4d1e70 0x56355a4d4260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:40:00.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:00.952+0000 7f749be00640 1 --2- 172.21.15.5:0/860575465 >> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] conn(0x56355a4d1e70 0x56355a4d4260 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=af72f18a88d672b1 server_cookie=380867ca7f8b5ccf in_seq=0 out_seq=0 2024-09-10T09:40:00.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:00.952+0000 7f7482000640 10 client.25721 ms_handle_connect on v2:172.21.15.159:6836/620627639 2024-09-10T09:40:00.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:00.953+0000 7f7482000640 1 -- 172.21.15.5:0/860575465 <== mds.2 v2:172.21.15.159:6836/620627639 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56355a42a950 con 0x56355a4d1e70 2024-09-10T09:40:00.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:00.953+0000 7f7482000640 10 client.25721 handle_command_reply: tid=0 2024-09-10T09:40:00.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:00.953+0000 7f74a604e740 1 client.25721 shutdown 2024-09-10T09:40:00.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:00.953+0000 7f74a604e740 1 -- 172.21.15.5:0/860575465 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56355a4b6c70 msgr2=0x56355a4becf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:40:00.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:00.953+0000 7f74a604e740 1 --2- 172.21.15.5:0/860575465 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56355a4b6c70 0x56355a4becf0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f7490000f80 tx=0x7f7490003ba0 comp rx=0 tx=0).stop 2024-09-10T09:40:00.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:00.953+0000 7f74a604e740 1 -- 172.21.15.5:0/860575465 shutdown_connections 2024-09-10T09:40:00.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:00.953+0000 7f74a604e740 1 --2- 172.21.15.5:0/860575465 >> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] conn(0x56355a4d1e70 0x56355a4d4260 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:00.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:00.953+0000 7f74a604e740 1 --2- 172.21.15.5:0/860575465 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56355a4c1e50 0x56355a4bf770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:00.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:00.953+0000 7f74a604e740 1 --2- 172.21.15.5:0/860575465 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56355a4b7590 0x56355a4bf230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:00.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:00.953+0000 7f74a604e740 1 --2- 172.21.15.5:0/860575465 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56355a4b6c70 0x56355a4becf0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:00.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:00.953+0000 7f7482000640 0 client.25721 ms_handle_reset on v2:172.21.15.159:6836/620627639 2024-09-10T09:40:00.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:00.953+0000 7f74a604e740 1 -- 172.21.15.5:0/860575465 >> 172.21.15.5:0/860575465 conn(0x56355a4b3200 msgr2=0x56355a4b2380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:40:00.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:00.953+0000 7f74a604e740 1 -- 172.21.15.5:0/860575465 shutdown_connections 2024-09-10T09:40:00.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:00.953+0000 7f74a604e740 1 -- 172.21.15.5:0/860575465 wait complete. 2024-09-10T09:40:00.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:00.953+0000 7f74a604e740 20 client.25721 trim_cache size 0 max 16384 2024-09-10T09:40:00.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:00.967+0000 7f74a604e740 1 Processor -- start 2024-09-10T09:40:00.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:00.967+0000 7f74a604e740 1 -- start start 2024-09-10T09:40:00.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:00.967+0000 7f74a604e740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56355a4b6c70 0x56355a4c0c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:40:00.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:00.967+0000 7f74a604e740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56355a4b7590 0x56355a4d9b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:40:00.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:00.967+0000 7f74a604e740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56355a4c1e50 0x56355a4da050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:40:00.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:00.967+0000 7f74a604e740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f749c1acc40 con 0x56355a4b7590 2024-09-10T09:40:00.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:00.967+0000 7f74a604e740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f749c1ae780 con 0x56355a4b6c70 2024-09-10T09:40:00.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:00.967+0000 7f74a604e740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f749c1ad310 con 0x56355a4c1e50 2024-09-10T09:40:00.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:00.968+0000 7f749be00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56355a4b7590 0x56355a4d9b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:40:00.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:00.968+0000 7f749be00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56355a4b7590 0x56355a4d9b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51140/0 (socket says 172.21.15.5:51140) 2024-09-10T09:40:00.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:00.968+0000 7f749be00640 1 -- 172.21.15.5:0/4184183595 learned_addr learned my addr 172.21.15.5:0/4184183595 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:40:00.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:00.968+0000 7f74a0c00640 1 --2- 172.21.15.5:0/4184183595 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56355a4b6c70 0x56355a4c0c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:40:00.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:00.968+0000 7f74a1600640 1 --2- 172.21.15.5:0/4184183595 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56355a4c1e50 0x56355a4da050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:40:00.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:00.968+0000 7f74a0c00640 1 -- 172.21.15.5:0/4184183595 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56355a4c1e50 msgr2=0x56355a4da050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:40:00.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:00.968+0000 7f74a0c00640 1 --2- 172.21.15.5:0/4184183595 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56355a4c1e50 0x56355a4da050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:00.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:00.968+0000 7f74a0c00640 1 -- 172.21.15.5:0/4184183595 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56355a4b7590 msgr2=0x56355a4d9b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:40:00.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:00.968+0000 7f74a0c00640 1 --2- 172.21.15.5:0/4184183595 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56355a4b7590 0x56355a4d9b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:00.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:00.968+0000 7f74a0c00640 1 -- 172.21.15.5:0/4184183595 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56355a4b73c0 con 0x56355a4b6c70 2024-09-10T09:40:00.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:00.968+0000 7f74a0c00640 1 --2- 172.21.15.5:0/4184183595 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56355a4b6c70 0x56355a4c0c60 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f749002ade0 tx=0x7f749002b060 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f9e6481d937bc37f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:40:00.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:00.969+0000 7f748d200640 1 -- 172.21.15.5:0/4184183595 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f749002f920 con 0x56355a4b6c70 2024-09-10T09:40:00.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:00.969+0000 7f748d200640 1 -- 172.21.15.5:0/4184183595 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7490036050 con 0x56355a4b6c70 2024-09-10T09:40:00.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:00.969+0000 7f748d200640 1 -- 172.21.15.5:0/4184183595 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7490030d00 con 0x56355a4b6c70 2024-09-10T09:40:00.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:00.969+0000 7f74a604e740 1 -- 172.21.15.5:0/4184183595 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56355a4b6c70 msgr2=0x56355a4c0c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:40:00.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:00.969+0000 7f74a604e740 1 --2- 172.21.15.5:0/4184183595 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56355a4b6c70 0x56355a4c0c60 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f749002ade0 tx=0x7f749002b060 comp rx=0 tx=0).stop 2024-09-10T09:40:00.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:00.969+0000 7f74a604e740 1 -- 172.21.15.5:0/4184183595 shutdown_connections 2024-09-10T09:40:00.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:00.969+0000 7f74a604e740 1 --2- 172.21.15.5:0/4184183595 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56355a4c1e50 0x56355a4da050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:00.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:00.969+0000 7f74a604e740 1 --2- 172.21.15.5:0/4184183595 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56355a4b7590 0x56355a4d9b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:00.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:00.969+0000 7f74a604e740 1 --2- 172.21.15.5:0/4184183595 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56355a4b6c70 0x56355a4c0c60 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:00.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:00.969+0000 7f74a604e740 1 -- 172.21.15.5:0/4184183595 >> 172.21.15.5:0/4184183595 conn(0x56355a4b3200 msgr2=0x56355a4c0550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:40:00.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:00.969+0000 7f74a604e740 1 -- 172.21.15.5:0/4184183595 shutdown_connections 2024-09-10T09:40:00.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:00.969+0000 7f74a604e740 1 -- 172.21.15.5:0/4184183595 wait complete. 2024-09-10T09:40:00.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:00.970+0000 7f74a604e740 1 -- start start 2024-09-10T09:40:00.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:00.970+0000 7f74a604e740 1 Processor -- start 2024-09-10T09:40:00.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:00.970+0000 7f74a604e740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56355a4b6c70 0x56355a4ca080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:40:00.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:00.970+0000 7f74a604e740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56355a4b7590 0x56355a4ca5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:40:00.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:00.970+0000 7f74a604e740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56355a4c1e50 0x56355a4bb620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:40:00.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:00.970+0000 7f74a604e740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f749c1ae0b0 con 0x56355a4b6c70 2024-09-10T09:40:00.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:00.970+0000 7f74a604e740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f749c1ad9e0 con 0x56355a4b7590 2024-09-10T09:40:00.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:00.970+0000 7f74a604e740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f749c1aac70 con 0x56355a4c1e50 2024-09-10T09:40:00.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:00.971+0000 7f74a604e740 10 client.0 _ll_register_callbacks cb 0x7f749002f310 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:40:00.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:00.971+0000 7f74a0c00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56355a4b6c70 0x56355a4ca080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:40:00.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:00.971+0000 7f74a0c00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56355a4b6c70 0x56355a4ca080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51156/0 (socket says 172.21.15.5:51156) 2024-09-10T09:40:00.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:00.971+0000 7f74a0c00640 1 -- 172.21.15.5:0/860552826 learned_addr learned my addr 172.21.15.5:0/860552826 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:40:00.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:00.971+0000 7f749be00640 1 --2- 172.21.15.5:0/860552826 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56355a4b7590 0x56355a4ca5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:40:00.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:00.971+0000 7f74a1600640 1 --2- 172.21.15.5:0/860552826 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56355a4c1e50 0x56355a4bb620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:40:00.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:00.971+0000 7f74a0c00640 1 -- 172.21.15.5:0/860552826 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56355a4c1e50 msgr2=0x56355a4bb620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:40:00.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:00.971+0000 7f74a0c00640 1 --2- 172.21.15.5:0/860552826 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56355a4c1e50 0x56355a4bb620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:00.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:00.971+0000 7f74a0c00640 1 -- 172.21.15.5:0/860552826 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56355a4b7590 msgr2=0x56355a4ca5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:40:00.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:00.971+0000 7f74a0c00640 1 --2- 172.21.15.5:0/860552826 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56355a4b7590 0x56355a4ca5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:00.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:00.971+0000 7f74a0c00640 1 -- 172.21.15.5:0/860552826 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56355a4bbe50 con 0x56355a4b6c70 2024-09-10T09:40:00.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:00.971+0000 7f74a1600640 1 --2- 172.21.15.5:0/860552826 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56355a4c1e50 0x56355a4bb620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:40:00.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:00.971+0000 7f749be00640 1 --2- 172.21.15.5:0/860552826 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56355a4b7590 0x56355a4ca5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:40:00.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:00.972+0000 7f74a0c00640 1 --2- 172.21.15.5:0/860552826 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56355a4b6c70 0x56355a4ca080 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f7490009620 tx=0x7f7490009820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90dfeb0976022d65 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:40:00.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:00.972+0000 7f7482000640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-10T09:40:00.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:00.972+0000 7f7482000640 1 -- 172.21.15.5:0/860552826 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f749002fe90 con 0x56355a4b6c70 2024-09-10T09:40:00.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:00.972+0000 7f7482000640 1 -- 172.21.15.5:0/860552826 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f749002f9b0 con 0x56355a4b6c70 2024-09-10T09:40:00.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:00.972+0000 7f74a604e740 1 -- 172.21.15.5:0/860552826 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7490036050 con 0x56355a4b6c70 2024-09-10T09:40:00.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:00.972+0000 7f7482000640 1 -- 172.21.15.5:0/860552826 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f7490031990 con 0x56355a4b6c70 2024-09-10T09:40:00.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:00.972+0000 7f74a604e740 10 client.15720 fetch_fsmap learned FSMap version 30 2024-09-10T09:40:00.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:00.972+0000 7f74a604e740 1 -- 172.21.15.5:0/860552826 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x56355a53c660 con 0x56355a4b6c70 2024-09-10T09:40:00.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:00.972+0000 7f7482000640 1 -- 172.21.15.5:0/860552826 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7490036050 con 0x56355a4b6c70 2024-09-10T09:40:00.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:00.973+0000 7f7482000640 1 -- 172.21.15.5:0/860552826 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f7490004140 con 0x56355a4b6c70 2024-09-10T09:40:00.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:00.973+0000 7f74a604e740 10 client.15720 fetch_fsmap finished waiting for FSMap version 30 2024-09-10T09:40:00.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:00.973+0000 7f74a604e740 10 client.15720 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-10T09:40:00.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:00.973+0000 7f74a604e740 20 client.15720 populate_metadata read hostname 'smithi005' 2024-09-10T09:40:00.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:00.973+0000 7f74a604e740 1 --2- 172.21.15.5:0/860552826 >> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] conn(0x56355a53ccc0 0x56355a53f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:40:00.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:00.973+0000 7f74a604e740 4 client.15720 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T09:40:00.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:00.973+0000 7f74a604e740 1 -- 172.21.15.5:0/860552826 --> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56355a424e60 con 0x56355a53ccc0 2024-09-10T09:40:00.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:00.973+0000 7f749be00640 1 --2- 172.21.15.5:0/860552826 >> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] conn(0x56355a53ccc0 0x56355a53f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:40:00.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:00.974+0000 7f749be00640 1 --2- 172.21.15.5:0/860552826 >> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] conn(0x56355a53ccc0 0x56355a53f0e0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f7b401f7b70e49e8 server_cookie=d1abeb0094f69734 in_seq=0 out_seq=0 2024-09-10T09:40:00.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:00.974+0000 7f7482000640 10 client.15720 ms_handle_connect on v2:172.21.15.159:6836/620627639 2024-09-10T09:40:00.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:00.974+0000 7f7482000640 1 -- 172.21.15.5:0/860552826 <== mds.2 v2:172.21.15.159:6836/620627639 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56355a424e60 con 0x56355a53ccc0 2024-09-10T09:40:00.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:00.974+0000 7f7482000640 10 client.15720 handle_command_reply: tid=0 2024-09-10T09:40:00.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:00.974+0000 7f74a604e740 1 client.15720 shutdown 2024-09-10T09:40:00.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:00.974+0000 7f74a604e740 1 -- 172.21.15.5:0/860552826 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56355a4b6c70 msgr2=0x56355a4ca080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:40:00.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:00.974+0000 7f74a604e740 1 --2- 172.21.15.5:0/860552826 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56355a4b6c70 0x56355a4ca080 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f7490009620 tx=0x7f7490009820 comp rx=0 tx=0).stop 2024-09-10T09:40:00.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:00.974+0000 7f74a604e740 1 -- 172.21.15.5:0/860552826 shutdown_connections 2024-09-10T09:40:00.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:00.974+0000 7f74a604e740 1 --2- 172.21.15.5:0/860552826 >> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] conn(0x56355a53ccc0 0x56355a53f0e0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:00.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:00.974+0000 7f74a604e740 1 --2- 172.21.15.5:0/860552826 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56355a4c1e50 0x56355a4bb620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:00.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:00.974+0000 7f74a604e740 1 --2- 172.21.15.5:0/860552826 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56355a4b7590 0x56355a4ca5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:00.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:00.974+0000 7f74a604e740 1 --2- 172.21.15.5:0/860552826 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56355a4b6c70 0x56355a4ca080 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:00.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:00.974+0000 7f74a604e740 1 -- 172.21.15.5:0/860552826 >> 172.21.15.5:0/860552826 conn(0x56355a4b3200 msgr2=0x56355a4b2380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:40:00.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:00.974+0000 7f7482000640 0 client.15720 ms_handle_reset on v2:172.21.15.159:6836/620627639 2024-09-10T09:40:00.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:00.974+0000 7f74a604e740 1 -- 172.21.15.5:0/860552826 shutdown_connections 2024-09-10T09:40:00.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:00.974+0000 7f74a604e740 1 -- 172.21.15.5:0/860552826 wait complete. 2024-09-10T09:40:00.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:00.975+0000 7f74a604e740 20 client.15720 trim_cache size 0 max 16384 2024-09-10T09:40:00.981 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-10T09:40:00.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:00.975+0000 7f74a3600640 1 -- 172.21.15.5:0/263404074 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f74840a2b80 msgr2=0x7f74840a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:40:00.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:00.975+0000 7f74a3600640 1 --2- 172.21.15.5:0/263404074 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f74840a2b80 0x7f74840a5040 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f749400bce0 tx=0x7f749400b4d0 comp rx=0 tx=0).stop 2024-09-10T09:40:00.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:00.975+0000 7f74a3600640 1 -- 172.21.15.5:0/263404074 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f749c1027a0 msgr2=0x7f749c074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:40:00.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:00.975+0000 7f74a3600640 1 --2- 172.21.15.5:0/263404074 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f749c1027a0 0x7f749c074a30 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f748800e9f0 tx=0x7f748800eeb0 comp rx=0 tx=0).stop 2024-09-10T09:40:00.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:00.975+0000 7f74a3600640 1 -- 172.21.15.5:0/263404074 shutdown_connections 2024-09-10T09:40:00.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:00.975+0000 7f74a3600640 1 --2- 172.21.15.5:0/263404074 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f74840a2b80 0x7f74840a5040 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:00.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:00.975+0000 7f74a3600640 1 --2- 172.21.15.5:0/263404074 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f749c106f20 0x7f749c074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:00.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:00.975+0000 7f74a3600640 1 --2- 172.21.15.5:0/263404074 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f749c1027a0 0x7f749c074a30 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:00.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:00.975+0000 7f74a3600640 1 --2- 172.21.15.5:0/263404074 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f749c101df0 0x7f749c0744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:00.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:00.975+0000 7f74a3600640 1 -- 172.21.15.5:0/263404074 >> 172.21.15.5:0/263404074 conn(0x7f749c0fd8b0 msgr2=0x7f749c0feb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:40:00.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:00.976+0000 7f74a3600640 1 -- 172.21.15.5:0/263404074 shutdown_connections 2024-09-10T09:40:00.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:00.976+0000 7f74a3600640 1 -- 172.21.15.5:0/263404074 wait complete. 2024-09-10T09:40:01.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:40:00 smithi159 ceph-mon[35513]: pgmap v443: 129 pgs: 129 active+clean; 666 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 85 B/s wr, 9 op/s 2024-09-10T09:40:01.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:40:00 smithi159 ceph-mon[35513]: overall HEALTH_OK 2024-09-10T09:40:01.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:40:00 smithi102 ceph-mon[35442]: pgmap v443: 129 pgs: 129 active+clean; 666 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 85 B/s wr, 9 op/s 2024-09-10T09:40:01.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:40:00 smithi102 ceph-mon[35442]: overall HEALTH_OK 2024-09-10T09:40:01.821 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-10T09:40:02.200 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:40:02.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:02.778+0000 7f5bafa00640 1 -- 172.21.15.5:0/213801254 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5ba8106f20 msgr2=0x7f5ba810eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:40:02.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:02.778+0000 7f5bafa00640 1 --2- 172.21.15.5:0/213801254 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5ba8106f20 0x7f5ba810eaa0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f5ba0009ca0 tx=0x7f5ba002a8c0 comp rx=0 tx=0).stop 2024-09-10T09:40:02.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:02.779+0000 7f5bafa00640 1 -- 172.21.15.5:0/213801254 shutdown_connections 2024-09-10T09:40:02.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:02.779+0000 7f5bafa00640 1 --2- 172.21.15.5:0/213801254 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5ba8106f20 0x7f5ba810eaa0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:02.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:02.779+0000 7f5bafa00640 1 --2- 172.21.15.5:0/213801254 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5ba81027a0 0x7f5ba81067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:02.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:02.779+0000 7f5bafa00640 1 --2- 172.21.15.5:0/213801254 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5ba8101df0 0x7f5ba81021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:02.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:02.779+0000 7f5bafa00640 1 -- 172.21.15.5:0/213801254 >> 172.21.15.5:0/213801254 conn(0x7f5ba80fd8b0 msgr2=0x7f5ba80ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:40:02.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:02.779+0000 7f5bafa00640 1 -- 172.21.15.5:0/213801254 shutdown_connections 2024-09-10T09:40:02.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:02.779+0000 7f5bafa00640 1 -- 172.21.15.5:0/213801254 wait complete. 2024-09-10T09:40:02.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:02.780+0000 7f5bafa00640 1 Processor -- start 2024-09-10T09:40:02.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:02.780+0000 7f5bafa00640 1 -- start start 2024-09-10T09:40:02.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:02.780+0000 7f5bafa00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5ba8101df0 0x7f5ba80744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:40:02.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:02.780+0000 7f5bafa00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5ba81027a0 0x7f5ba8074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:40:02.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:02.780+0000 7f5bafa00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5ba8106f20 0x7f5ba8074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:40:02.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:02.780+0000 7f5bafa00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f5ba8111fd0 con 0x7f5ba8101df0 2024-09-10T09:40:02.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:02.780+0000 7f5bafa00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f5ba8111e50 con 0x7f5ba8106f20 2024-09-10T09:40:02.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:02.780+0000 7f5bafa00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f5ba8112150 con 0x7f5ba81027a0 2024-09-10T09:40:02.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:02.780+0000 7f5bad000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5ba8101df0 0x7f5ba80744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:40:02.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:02.781+0000 7f5bad000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5ba8101df0 0x7f5ba80744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51182/0 (socket says 172.21.15.5:51182) 2024-09-10T09:40:02.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:02.781+0000 7f5bad000640 1 -- 172.21.15.5:0/3258171479 learned_addr learned my addr 172.21.15.5:0/3258171479 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:40:02.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:02.781+0000 7f5bada00640 1 --2- 172.21.15.5:0/3258171479 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5ba8106f20 0x7f5ba8074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:40:02.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:02.781+0000 7f5b9fe00640 1 --2- 172.21.15.5:0/3258171479 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5ba81027a0 0x7f5ba8074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:40:02.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:02.781+0000 7f5bad000640 1 -- 172.21.15.5:0/3258171479 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5ba81027a0 msgr2=0x7f5ba8074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:40:02.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:02.781+0000 7f5bad000640 1 --2- 172.21.15.5:0/3258171479 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5ba81027a0 0x7f5ba8074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:02.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:02.781+0000 7f5bad000640 1 -- 172.21.15.5:0/3258171479 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5ba8106f20 msgr2=0x7f5ba8074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:40:02.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:02.781+0000 7f5bad000640 1 --2- 172.21.15.5:0/3258171479 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5ba8106f20 0x7f5ba8074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:02.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:02.781+0000 7f5bad000640 1 -- 172.21.15.5:0/3258171479 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ba81a6880 con 0x7f5ba8101df0 2024-09-10T09:40:02.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:02.781+0000 7f5bada00640 1 --2- 172.21.15.5:0/3258171479 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5ba8106f20 0x7f5ba8074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:40:02.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:02.781+0000 7f5bad000640 1 --2- 172.21.15.5:0/3258171479 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5ba8101df0 0x7f5ba80744f0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f5b9800e9f0 tx=0x7f5b9800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:40:02.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:02.782+0000 7f5ba6000640 1 -- 172.21.15.5:0/3258171479 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b9800cd80 con 0x7f5ba8101df0 2024-09-10T09:40:02.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:02.782+0000 7f5ba6000640 1 -- 172.21.15.5:0/3258171479 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5b98005500 con 0x7f5ba8101df0 2024-09-10T09:40:02.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:02.782+0000 7f5ba6000640 1 -- 172.21.15.5:0/3258171479 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b98010690 con 0x7f5ba8101df0 2024-09-10T09:40:02.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:02.782+0000 7f5bafa00640 1 -- 172.21.15.5:0/3258171479 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ba81a6b70 con 0x7f5ba8101df0 2024-09-10T09:40:02.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:02.782+0000 7f5bafa00640 1 -- 172.21.15.5:0/3258171479 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ba81a7060 con 0x7f5ba8101df0 2024-09-10T09:40:02.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:02.785+0000 7f5bb25c9740 1 Processor -- start 2024-09-10T09:40:02.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:02.785+0000 7f5ba6000640 1 -- 172.21.15.5:0/3258171479 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f5b980046c0 con 0x7f5ba8101df0 2024-09-10T09:40:02.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:02.785+0000 7f5bb25c9740 1 -- start start 2024-09-10T09:40:02.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:02.786+0000 7f5bb25c9740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561e61f23c70 0x561e61f24050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:40:02.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:02.786+0000 7f5bb25c9740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561e61f24590 0x561e61f2e910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:40:02.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:02.786+0000 7f5bb25c9740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561e61f2ee50 0x561e61f31240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:40:02.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:02.786+0000 7f5bb25c9740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x561e61e8a3e0 con 0x561e61f23c70 2024-09-10T09:40:02.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:02.786+0000 7f5bb25c9740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x561e61e31d80 con 0x561e61f24590 2024-09-10T09:40:02.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:02.786+0000 7f5bb25c9740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x561e61e863a0 con 0x561e61f2ee50 2024-09-10T09:40:02.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:02.786+0000 7f5bada00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561e61f23c70 0x561e61f24050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:40:02.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:02.786+0000 7f5bad000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561e61f24590 0x561e61f2e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:40:02.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:02.786+0000 7f5bada00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561e61f23c70 0x561e61f24050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51198/0 (socket says 172.21.15.5:51198) 2024-09-10T09:40:02.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:02.786+0000 7f5bada00640 1 -- 172.21.15.5:0/4144883673 learned_addr learned my addr 172.21.15.5:0/4144883673 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:40:02.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:02.786+0000 7f5b9fe00640 1 --2- 172.21.15.5:0/4144883673 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561e61f2ee50 0x561e61f31240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:40:02.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:02.787+0000 7f5bad000640 1 -- 172.21.15.5:0/4144883673 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561e61f2ee50 msgr2=0x561e61f31240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:40:02.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:02.787+0000 7f5bad000640 1 --2- 172.21.15.5:0/4144883673 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561e61f2ee50 0x561e61f31240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:02.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:02.787+0000 7f5bad000640 1 -- 172.21.15.5:0/4144883673 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561e61f23c70 msgr2=0x561e61f24050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:40:02.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:02.787+0000 7f5bad000640 1 --2- 172.21.15.5:0/4144883673 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561e61f23c70 0x561e61f24050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:02.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:02.787+0000 7f5bad000640 1 -- 172.21.15.5:0/4144883673 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561e61e50ae0 con 0x561e61f24590 2024-09-10T09:40:02.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:02.787+0000 7f5ba6000640 1 --2- 172.21.15.5:0/3258171479 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f5b8c0a2b80 0x7f5b8c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:40:02.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:02.787+0000 7f5bad000640 1 --2- 172.21.15.5:0/4144883673 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561e61f24590 0x561e61f2e910 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f5b980ccf00 tx=0x7f5b980d5b20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b1234bd5c40530c0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:40:02.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:02.787+0000 7f5bada00640 1 --2- 172.21.15.5:0/3258171479 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f5b8c0a2b80 0x7f5b8c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:40:02.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:02.787+0000 7f5ba6000640 1 -- 172.21.15.5:0/3258171479 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f5b980cf8b0 con 0x7f5ba8101df0 2024-09-10T09:40:02.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:02.787+0000 7f5b9e000640 1 -- 172.21.15.5:0/4144883673 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b980de070 con 0x561e61f24590 2024-09-10T09:40:02.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:02.787+0000 7f5b9e000640 1 -- 172.21.15.5:0/4144883673 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5b9809c3c0 con 0x561e61f24590 2024-09-10T09:40:02.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:02.787+0000 7f5b9e000640 1 -- 172.21.15.5:0/4144883673 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b9809b800 con 0x561e61f24590 2024-09-10T09:40:02.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:02.788+0000 7f5bb25c9740 1 -- 172.21.15.5:0/4144883673 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561e61f24590 msgr2=0x561e61f2e910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:40:02.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:02.788+0000 7f5bada00640 1 --2- 172.21.15.5:0/3258171479 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f5b8c0a2b80 0x7f5b8c0a5040 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f5ba002aeb0 tx=0x7f5ba0005d80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:40:02.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:02.788+0000 7f5bb25c9740 1 --2- 172.21.15.5:0/4144883673 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561e61f24590 0x561e61f2e910 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f5b980ccf00 tx=0x7f5b980d5b20 comp rx=0 tx=0).stop 2024-09-10T09:40:02.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:02.788+0000 7f5bb25c9740 1 -- 172.21.15.5:0/4144883673 shutdown_connections 2024-09-10T09:40:02.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:02.788+0000 7f5bb25c9740 1 --2- 172.21.15.5:0/4144883673 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561e61f2ee50 0x561e61f31240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:02.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:02.788+0000 7f5bb25c9740 1 --2- 172.21.15.5:0/4144883673 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561e61f24590 0x561e61f2e910 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:02.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:02.788+0000 7f5bb25c9740 1 --2- 172.21.15.5:0/4144883673 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561e61f23c70 0x561e61f24050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:02.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:02.788+0000 7f5bb25c9740 1 -- 172.21.15.5:0/4144883673 >> 172.21.15.5:0/4144883673 conn(0x561e61f20200 msgr2=0x561e61f205e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:40:02.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:02.788+0000 7f5bb25c9740 1 -- 172.21.15.5:0/4144883673 shutdown_connections 2024-09-10T09:40:02.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:02.788+0000 7f5bb25c9740 1 -- 172.21.15.5:0/4144883673 wait complete. 2024-09-10T09:40:02.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:02.789+0000 7f5bb25c9740 1 -- start start 2024-09-10T09:40:02.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:02.789+0000 7f5bb25c9740 1 Processor -- start 2024-09-10T09:40:02.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:02.789+0000 7f5bb25c9740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561e61f23c70 0x561e61f2bce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:40:02.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:02.789+0000 7f5bb25c9740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561e61f24590 0x561e61f2c220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:40:02.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:02.790+0000 7f5bb25c9740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561e61f2ee50 0x561e61f2c760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:40:02.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:02.790+0000 7f5bb25c9740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x561e61f31a20 con 0x561e61f2ee50 2024-09-10T09:40:02.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:02.790+0000 7f5bb25c9740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x561e61e450f0 con 0x561e61f23c70 2024-09-10T09:40:02.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:02.790+0000 7f5bb25c9740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x561e61f31ba0 con 0x561e61f24590 2024-09-10T09:40:02.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:02.790+0000 7f5bb25c9740 10 client.0 _ll_register_callbacks cb 0x561e61e2cbd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:40:02.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:02.790+0000 7f5b9fe00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561e61f24590 0x561e61f2c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:40:02.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:02.790+0000 7f5b9fe00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561e61f24590 0x561e61f2c220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:36420/0 (socket says 172.21.15.5:36420) 2024-09-10T09:40:02.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:02.790+0000 7f5b9fe00640 1 -- 172.21.15.5:0/4010439486 learned_addr learned my addr 172.21.15.5:0/4010439486 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:40:02.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:02.790+0000 7f5bad000640 1 --2- 172.21.15.5:0/4010439486 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561e61f23c70 0x561e61f2bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:40:02.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:02.790+0000 7f5bada00640 1 --2- 172.21.15.5:0/4010439486 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561e61f2ee50 0x561e61f2c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:40:02.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:02.790+0000 7f5b9fe00640 1 -- 172.21.15.5:0/4010439486 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561e61f23c70 msgr2=0x561e61f2bce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:40:02.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:02.790+0000 7f5b9fe00640 1 --2- 172.21.15.5:0/4010439486 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561e61f23c70 0x561e61f2bce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:02.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:02.790+0000 7f5b9fe00640 1 -- 172.21.15.5:0/4010439486 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561e61f2ee50 msgr2=0x561e61f2c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:40:02.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:02.790+0000 7f5b9fe00640 1 --2- 172.21.15.5:0/4010439486 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561e61f2ee50 0x561e61f2c760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:02.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:02.790+0000 7f5b9fe00640 1 -- 172.21.15.5:0/4010439486 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561e61bfb3c0 con 0x561e61f24590 2024-09-10T09:40:02.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:02.790+0000 7f5bad000640 1 --2- 172.21.15.5:0/4010439486 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561e61f23c70 0x561e61f2bce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:40:02.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:02.791+0000 7f5b9fe00640 1 --2- 172.21.15.5:0/4010439486 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561e61f24590 0x561e61f2c220 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f5b90012700 tx=0x7f5b90012bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=16e3d2e9b2c90ec6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:40:02.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:02.791+0000 7f5b96a00640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-09-10T09:40:02.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:02.791+0000 7f5b96a00640 1 -- 172.21.15.5:0/4010439486 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b9000f8c0 con 0x561e61f24590 2024-09-10T09:40:02.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:02.791+0000 7f5bb25c9740 1 -- 172.21.15.5:0/4010439486 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5b9809c3c0 con 0x561e61f24590 2024-09-10T09:40:02.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:02.791+0000 7f5b96a00640 1 -- 172.21.15.5:0/4010439486 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f5b9000fa60 con 0x561e61f24590 2024-09-10T09:40:02.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:02.791+0000 7f5b96a00640 1 -- 172.21.15.5:0/4010439486 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f5b9000fc60 con 0x561e61f24590 2024-09-10T09:40:02.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:02.791+0000 7f5bb25c9740 10 client.25768 fetch_fsmap learned FSMap version 30 2024-09-10T09:40:02.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:02.791+0000 7f5bb25c9740 1 -- 172.21.15.5:0/4010439486 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x561e61f3dad0 con 0x561e61f24590 2024-09-10T09:40:02.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:02.792+0000 7f5b96a00640 1 -- 172.21.15.5:0/4010439486 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5b9809c3c0 con 0x561e61f24590 2024-09-10T09:40:02.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:02.792+0000 7f5b96a00640 1 -- 172.21.15.5:0/4010439486 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f5b90005460 con 0x561e61f24590 2024-09-10T09:40:02.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:02.792+0000 7f5bb25c9740 10 client.25768 fetch_fsmap finished waiting for FSMap version 30 2024-09-10T09:40:02.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:02.792+0000 7f5bb25c9740 10 client.25768 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-10T09:40:02.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:02.792+0000 7f5bb25c9740 20 client.25768 populate_metadata read hostname 'smithi005' 2024-09-10T09:40:02.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:02.792+0000 7f5bb25c9740 1 --2- 172.21.15.5:0/4010439486 >> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] conn(0x561e61f3eee0 0x561e61f412f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:40:02.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:02.792+0000 7f5bb25c9740 4 client.25768 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:40:02.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:02.792+0000 7f5bb25c9740 1 -- 172.21.15.5:0/4010439486 --> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561e61e97950 con 0x561e61f3eee0 2024-09-10T09:40:02.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:02.792+0000 7f5bad000640 1 --2- 172.21.15.5:0/4010439486 >> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] conn(0x561e61f3eee0 0x561e61f412f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:40:02.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:02.793+0000 7f5bad000640 1 --2- 172.21.15.5:0/4010439486 >> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] conn(0x561e61f3eee0 0x561e61f412f0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ccf29a36630dd9f8 server_cookie=50af66c9144dea76 in_seq=0 out_seq=0 2024-09-10T09:40:02.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:02.793+0000 7f5b96a00640 10 client.25768 ms_handle_connect on v2:172.21.15.159:6838/2073739599 2024-09-10T09:40:02.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:02.794+0000 7f5b96a00640 1 -- 172.21.15.5:0/4010439486 <== mds.1 v2:172.21.15.159:6838/2073739599 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561e61e97950 con 0x561e61f3eee0 2024-09-10T09:40:02.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:02.794+0000 7f5b96a00640 10 client.25768 handle_command_reply: tid=0 2024-09-10T09:40:02.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:02.794+0000 7f5bb25c9740 1 client.25768 shutdown 2024-09-10T09:40:02.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:02.794+0000 7f5bb25c9740 1 -- 172.21.15.5:0/4010439486 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561e61f24590 msgr2=0x561e61f2c220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:40:02.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:02.794+0000 7f5bb25c9740 1 --2- 172.21.15.5:0/4010439486 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561e61f24590 0x561e61f2c220 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f5b90012700 tx=0x7f5b90012bc0 comp rx=0 tx=0).stop 2024-09-10T09:40:02.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:02.794+0000 7f5bb25c9740 1 -- 172.21.15.5:0/4010439486 shutdown_connections 2024-09-10T09:40:02.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:02.794+0000 7f5bb25c9740 1 --2- 172.21.15.5:0/4010439486 >> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] conn(0x561e61f3eee0 0x561e61f412f0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:02.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:02.794+0000 7f5bb25c9740 1 --2- 172.21.15.5:0/4010439486 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561e61f2ee50 0x561e61f2c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:02.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:02.794+0000 7f5bb25c9740 1 --2- 172.21.15.5:0/4010439486 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561e61f24590 0x561e61f2c220 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:02.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:02.794+0000 7f5b96a00640 0 client.25768 ms_handle_reset on v2:172.21.15.159:6838/2073739599 2024-09-10T09:40:02.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:02.794+0000 7f5bb25c9740 1 --2- 172.21.15.5:0/4010439486 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561e61f23c70 0x561e61f2bce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:02.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:02.794+0000 7f5bb25c9740 1 -- 172.21.15.5:0/4010439486 >> 172.21.15.5:0/4010439486 conn(0x561e61f20200 msgr2=0x561e61f25f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:40:02.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:02.794+0000 7f5bb25c9740 1 -- 172.21.15.5:0/4010439486 shutdown_connections 2024-09-10T09:40:02.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:02.794+0000 7f5bb25c9740 1 -- 172.21.15.5:0/4010439486 wait complete. 2024-09-10T09:40:02.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:02.795+0000 7f5bb25c9740 20 client.25768 trim_cache size 0 max 16384 2024-09-10T09:40:02.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:02.808+0000 7f5bb25c9740 1 Processor -- start 2024-09-10T09:40:02.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:02.808+0000 7f5bb25c9740 1 -- start start 2024-09-10T09:40:02.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:02.809+0000 7f5bb25c9740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561e61f23c70 0x561e61f26720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:40:02.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:02.809+0000 7f5bb25c9740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561e61f24590 0x561e61f1f270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:40:02.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:02.809+0000 7f5bb25c9740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561e61f2ee50 0x561e61f46c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:40:02.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:02.809+0000 7f5bb25c9740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f5ba81b1090 con 0x561e61f24590 2024-09-10T09:40:02.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:02.809+0000 7f5bb25c9740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f5ba81b2bd0 con 0x561e61f23c70 2024-09-10T09:40:02.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:02.809+0000 7f5bb25c9740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f5ba81b1760 con 0x561e61f2ee50 2024-09-10T09:40:02.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:02.809+0000 7f5b9fe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561e61f24590 0x561e61f1f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:40:02.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:02.809+0000 7f5b9fe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561e61f24590 0x561e61f1f270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51220/0 (socket says 172.21.15.5:51220) 2024-09-10T09:40:02.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:02.809+0000 7f5bad000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561e61f23c70 0x561e61f26720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:40:02.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:02.809+0000 7f5b9fe00640 1 -- 172.21.15.5:0/4166745684 learned_addr learned my addr 172.21.15.5:0/4166745684 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:40:02.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:02.809+0000 7f5bada00640 1 --2- 172.21.15.5:0/4166745684 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561e61f2ee50 0x561e61f46c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:40:02.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:02.809+0000 7f5bad000640 1 -- 172.21.15.5:0/4166745684 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561e61f2ee50 msgr2=0x561e61f46c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:40:02.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:02.809+0000 7f5bad000640 1 --2- 172.21.15.5:0/4166745684 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561e61f2ee50 0x561e61f46c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:02.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:02.809+0000 7f5bad000640 1 -- 172.21.15.5:0/4166745684 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561e61f24590 msgr2=0x561e61f1f270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:40:02.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:02.809+0000 7f5bad000640 1 --2- 172.21.15.5:0/4166745684 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561e61f24590 0x561e61f1f270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:02.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:02.809+0000 7f5bad000640 1 -- 172.21.15.5:0/4166745684 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561e61f20030 con 0x561e61f23c70 2024-09-10T09:40:02.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:02.809+0000 7f5b9fe00640 1 --2- 172.21.15.5:0/4166745684 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561e61f24590 0x561e61f1f270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:40:02.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:02.810+0000 7f5bad000640 1 --2- 172.21.15.5:0/4166745684 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561e61f23c70 0x561e61f26720 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f5b9809bc00 tx=0x7f5b98009310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=12434e5db88fda9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:40:02.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:02.810+0000 7f5b9d600640 1 -- 172.21.15.5:0/4166745684 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b980de070 con 0x561e61f23c70 2024-09-10T09:40:02.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:02.810+0000 7f5b9d600640 1 -- 172.21.15.5:0/4166745684 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5b9809c3c0 con 0x561e61f23c70 2024-09-10T09:40:02.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:02.810+0000 7f5b9d600640 1 -- 172.21.15.5:0/4166745684 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b9809b620 con 0x561e61f23c70 2024-09-10T09:40:02.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:02.810+0000 7f5bb25c9740 1 -- 172.21.15.5:0/4166745684 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561e61f23c70 msgr2=0x561e61f26720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:40:02.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:02.810+0000 7f5bb25c9740 1 --2- 172.21.15.5:0/4166745684 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561e61f23c70 0x561e61f26720 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f5b9809bc00 tx=0x7f5b98009310 comp rx=0 tx=0).stop 2024-09-10T09:40:02.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:02.811+0000 7f5bb25c9740 1 -- 172.21.15.5:0/4166745684 shutdown_connections 2024-09-10T09:40:02.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:02.811+0000 7f5bb25c9740 1 --2- 172.21.15.5:0/4166745684 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561e61f2ee50 0x561e61f46c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:02.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:02.811+0000 7f5bb25c9740 1 --2- 172.21.15.5:0/4166745684 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561e61f24590 0x561e61f1f270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:02.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:02.811+0000 7f5bb25c9740 1 --2- 172.21.15.5:0/4166745684 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561e61f23c70 0x561e61f26720 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:02.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:02.811+0000 7f5bb25c9740 1 -- 172.21.15.5:0/4166745684 >> 172.21.15.5:0/4166745684 conn(0x561e61f20200 msgr2=0x561e61f1eb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:40:02.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:02.811+0000 7f5bb25c9740 1 -- 172.21.15.5:0/4166745684 shutdown_connections 2024-09-10T09:40:02.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:02.811+0000 7f5bb25c9740 1 -- 172.21.15.5:0/4166745684 wait complete. 2024-09-10T09:40:02.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:02.811+0000 7f5bb25c9740 1 -- start start 2024-09-10T09:40:02.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:02.811+0000 7f5bb25c9740 1 Processor -- start 2024-09-10T09:40:02.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:02.812+0000 7f5bb25c9740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561e61f23c70 0x561e61f32eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:40:02.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:02.812+0000 7f5bb25c9740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561e61f24590 0x561e61f333f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:40:02.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:02.812+0000 7f5bb25c9740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561e61f2ee50 0x561e61fa8bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:40:02.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:02.812+0000 7f5bb25c9740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f5ba81b2500 con 0x561e61f24590 2024-09-10T09:40:02.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:02.812+0000 7f5bb25c9740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f5ba81b1e30 con 0x561e61f23c70 2024-09-10T09:40:02.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:02.812+0000 7f5bb25c9740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f5ba81af070 con 0x561e61f2ee50 2024-09-10T09:40:02.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:02.812+0000 7f5bb25c9740 10 client.0 _ll_register_callbacks cb 0x7f5b90009850 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:40:02.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:02.812+0000 7f5b9fe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561e61f24590 0x561e61f333f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:40:02.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:02.812+0000 7f5bad000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561e61f23c70 0x561e61f32eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:40:02.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:02.812+0000 7f5b9fe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561e61f24590 0x561e61f333f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51226/0 (socket says 172.21.15.5:51226) 2024-09-10T09:40:02.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:02.812+0000 7f5b9fe00640 1 -- 172.21.15.5:0/1711046555 learned_addr learned my addr 172.21.15.5:0/1711046555 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:40:02.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:02.812+0000 7f5bada00640 1 --2- 172.21.15.5:0/1711046555 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561e61f2ee50 0x561e61fa8bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:40:02.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:02.813+0000 7f5bad000640 1 -- 172.21.15.5:0/1711046555 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561e61f2ee50 msgr2=0x561e61fa8bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:40:02.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:02.813+0000 7f5bad000640 1 --2- 172.21.15.5:0/1711046555 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561e61f2ee50 0x561e61fa8bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:02.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:02.813+0000 7f5bad000640 1 -- 172.21.15.5:0/1711046555 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561e61f24590 msgr2=0x561e61f333f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:40:02.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:02.813+0000 7f5bad000640 1 --2- 172.21.15.5:0/1711046555 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561e61f24590 0x561e61f333f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:02.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:02.813+0000 7f5bad000640 1 -- 172.21.15.5:0/1711046555 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561e61f1b770 con 0x561e61f23c70 2024-09-10T09:40:02.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:02.813+0000 7f5b9fe00640 1 --2- 172.21.15.5:0/1711046555 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561e61f24590 0x561e61f333f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:40:02.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:02.813+0000 7f5bad000640 1 --2- 172.21.15.5:0/1711046555 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561e61f23c70 0x561e61f32eb0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f5b9809bc00 tx=0x7f5b9809bb50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=def92ae174fb3ee3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:40:02.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:02.813+0000 7f5b96a00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-09-10T09:40:02.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:02.813+0000 7f5b96a00640 1 -- 172.21.15.5:0/1711046555 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b980de070 con 0x561e61f23c70 2024-09-10T09:40:02.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:02.813+0000 7f5bb25c9740 1 -- 172.21.15.5:0/1711046555 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5b9809c3c0 con 0x561e61f23c70 2024-09-10T09:40:02.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:02.813+0000 7f5b96a00640 1 -- 172.21.15.5:0/1711046555 <== mon.2 v2:172.21.15.102:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f5b980d5400 con 0x561e61f23c70 2024-09-10T09:40:02.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:02.813+0000 7f5b96a00640 1 -- 172.21.15.5:0/1711046555 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f5b9809b620 con 0x561e61f23c70 2024-09-10T09:40:02.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:02.814+0000 7f5bb25c9740 10 client.25745 fetch_fsmap learned FSMap version 30 2024-09-10T09:40:02.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:02.814+0000 7f5bb25c9740 1 -- 172.21.15.5:0/1711046555 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x561e61faa000 con 0x561e61f23c70 2024-09-10T09:40:02.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:02.814+0000 7f5b96a00640 1 -- 172.21.15.5:0/1711046555 <== mon.2 v2:172.21.15.102:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5b9809c3c0 con 0x561e61f23c70 2024-09-10T09:40:02.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:02.814+0000 7f5b96a00640 1 -- 172.21.15.5:0/1711046555 <== mon.2 v2:172.21.15.102:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f5b9809bf20 con 0x561e61f23c70 2024-09-10T09:40:02.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:02.814+0000 7f5bb25c9740 10 client.25745 fetch_fsmap finished waiting for FSMap version 30 2024-09-10T09:40:02.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:02.814+0000 7f5bb25c9740 10 client.25745 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-10T09:40:02.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:02.814+0000 7f5bb25c9740 20 client.25745 populate_metadata read hostname 'smithi005' 2024-09-10T09:40:02.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:02.814+0000 7f5bb25c9740 1 --2- 172.21.15.5:0/1711046555 >> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] conn(0x561e61faa280 0x561e61fac670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:40:02.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:02.814+0000 7f5bb25c9740 4 client.25745 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T09:40:02.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:02.814+0000 7f5bb25c9740 1 -- 172.21.15.5:0/1711046555 --> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561e61e91e60 con 0x561e61faa280 2024-09-10T09:40:02.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:02.814+0000 7f5b9fe00640 1 --2- 172.21.15.5:0/1711046555 >> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] conn(0x561e61faa280 0x561e61fac670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:40:02.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:02.815+0000 7f5b9fe00640 1 --2- 172.21.15.5:0/1711046555 >> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] conn(0x561e61faa280 0x561e61fac670 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e7eeb864b3607a97 server_cookie=95d2dae034f68b9f in_seq=0 out_seq=0 2024-09-10T09:40:02.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:02.815+0000 7f5b96a00640 10 client.25745 ms_handle_connect on v2:172.21.15.159:6838/2073739599 2024-09-10T09:40:02.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:02.815+0000 7f5b96a00640 1 -- 172.21.15.5:0/1711046555 <== mds.1 v2:172.21.15.159:6838/2073739599 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561e61e91e60 con 0x561e61faa280 2024-09-10T09:40:02.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:02.815+0000 7f5b96a00640 10 client.25745 handle_command_reply: tid=0 2024-09-10T09:40:02.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:02.815+0000 7f5bb25c9740 1 client.25745 shutdown 2024-09-10T09:40:02.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:02.815+0000 7f5bb25c9740 1 -- 172.21.15.5:0/1711046555 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561e61f23c70 msgr2=0x561e61f32eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:40:02.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:02.815+0000 7f5bb25c9740 1 --2- 172.21.15.5:0/1711046555 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561e61f23c70 0x561e61f32eb0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f5b9809bc00 tx=0x7f5b9809bb50 comp rx=0 tx=0).stop 2024-09-10T09:40:02.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:02.815+0000 7f5bb25c9740 1 -- 172.21.15.5:0/1711046555 shutdown_connections 2024-09-10T09:40:02.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:02.815+0000 7f5bb25c9740 1 --2- 172.21.15.5:0/1711046555 >> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] conn(0x561e61faa280 0x561e61fac670 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:02.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:02.816+0000 7f5bb25c9740 1 --2- 172.21.15.5:0/1711046555 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561e61f2ee50 0x561e61fa8bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:02.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:02.816+0000 7f5bb25c9740 1 --2- 172.21.15.5:0/1711046555 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561e61f24590 0x561e61f333f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:02.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:02.816+0000 7f5b96a00640 0 client.25745 ms_handle_reset on v2:172.21.15.159:6838/2073739599 2024-09-10T09:40:02.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:02.816+0000 7f5bb25c9740 1 --2- 172.21.15.5:0/1711046555 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561e61f23c70 0x561e61f32eb0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:02.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:02.816+0000 7f5bb25c9740 1 -- 172.21.15.5:0/1711046555 >> 172.21.15.5:0/1711046555 conn(0x561e61f20200 msgr2=0x561e61f2bc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:40:02.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:02.816+0000 7f5bb25c9740 1 -- 172.21.15.5:0/1711046555 shutdown_connections 2024-09-10T09:40:02.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:02.816+0000 7f5bb25c9740 1 -- 172.21.15.5:0/1711046555 wait complete. 2024-09-10T09:40:02.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:02.816+0000 7f5bb25c9740 20 client.25745 trim_cache size 0 max 16384 2024-09-10T09:40:02.821 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-10T09:40:02.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:02.816+0000 7f5bafa00640 1 -- 172.21.15.5:0/3258171479 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f5b8c0a2b80 msgr2=0x7f5b8c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:40:02.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:02.816+0000 7f5bafa00640 1 --2- 172.21.15.5:0/3258171479 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f5b8c0a2b80 0x7f5b8c0a5040 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f5ba002aeb0 tx=0x7f5ba0005d80 comp rx=0 tx=0).stop 2024-09-10T09:40:02.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:02.816+0000 7f5bafa00640 1 -- 172.21.15.5:0/3258171479 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5ba8101df0 msgr2=0x7f5ba80744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:40:02.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:02.816+0000 7f5bafa00640 1 --2- 172.21.15.5:0/3258171479 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5ba8101df0 0x7f5ba80744f0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f5b9800e9f0 tx=0x7f5b9800edf0 comp rx=0 tx=0).stop 2024-09-10T09:40:02.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:02.817+0000 7f5bafa00640 1 -- 172.21.15.5:0/3258171479 shutdown_connections 2024-09-10T09:40:02.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:02.817+0000 7f5bafa00640 1 --2- 172.21.15.5:0/3258171479 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f5b8c0a2b80 0x7f5b8c0a5040 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:02.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:02.817+0000 7f5bafa00640 1 --2- 172.21.15.5:0/3258171479 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5ba8106f20 0x7f5ba8074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:02.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:02.817+0000 7f5bafa00640 1 --2- 172.21.15.5:0/3258171479 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5ba81027a0 0x7f5ba8074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:02.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:02.817+0000 7f5bafa00640 1 --2- 172.21.15.5:0/3258171479 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5ba8101df0 0x7f5ba80744f0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:02.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:02.817+0000 7f5bafa00640 1 -- 172.21.15.5:0/3258171479 >> 172.21.15.5:0/3258171479 conn(0x7f5ba80fd8b0 msgr2=0x7f5ba80fe3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:40:02.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:02.817+0000 7f5bafa00640 1 -- 172.21.15.5:0/3258171479 shutdown_connections 2024-09-10T09:40:02.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:02.817+0000 7f5bafa00640 1 -- 172.21.15.5:0/3258171479 wait complete. 2024-09-10T09:40:03.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:40:02 smithi005 ceph-mon[30350]: pgmap v444: 129 pgs: 129 active+clean; 666 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 85 B/s wr, 12 op/s 2024-09-10T09:40:03.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:40:02 smithi159 ceph-mon[35513]: pgmap v444: 129 pgs: 129 active+clean; 666 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 85 B/s wr, 12 op/s 2024-09-10T09:40:03.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:40:02 smithi102 ceph-mon[35442]: pgmap v444: 129 pgs: 129 active+clean; 666 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 85 B/s wr, 12 op/s 2024-09-10T09:40:03.753 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-10T09:40:04.135 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:40:04.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:04.771+0000 7fbe66600640 1 -- 172.21.15.5:0/3180884166 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbe60102360 msgr2=0x7fbe6010e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:40:04.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:04.771+0000 7fbe66600640 1 --2- 172.21.15.5:0/3180884166 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbe60102360 0x7fbe6010e890 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fbe58009cd0 tx=0x7fbe5802a8f0 comp rx=0 tx=0).stop 2024-09-10T09:40:04.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:04.771+0000 7fbe66600640 1 -- 172.21.15.5:0/3180884166 shutdown_connections 2024-09-10T09:40:04.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:04.771+0000 7fbe66600640 1 --2- 172.21.15.5:0/3180884166 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbe60102360 0x7fbe6010e890 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:04.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:04.771+0000 7fbe66600640 1 --2- 172.21.15.5:0/3180884166 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbe601019c0 0x7fbe60101e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:04.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:04.771+0000 7fbe66600640 1 --2- 172.21.15.5:0/3180884166 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbe601079c0 0x7fbe60107da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:04.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:04.771+0000 7fbe66600640 1 -- 172.21.15.5:0/3180884166 >> 172.21.15.5:0/3180884166 conn(0x7fbe600fd740 msgr2=0x7fbe600ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:40:04.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:04.772+0000 7fbe66600640 1 -- 172.21.15.5:0/3180884166 shutdown_connections 2024-09-10T09:40:04.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:04.772+0000 7fbe66600640 1 -- 172.21.15.5:0/3180884166 wait complete. 2024-09-10T09:40:04.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:04.772+0000 7fbe66600640 1 Processor -- start 2024-09-10T09:40:04.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:04.773+0000 7fbe66600640 1 -- start start 2024-09-10T09:40:04.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:04.773+0000 7fbe66600640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbe601019c0 0x7fbe6019a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:40:04.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:04.773+0000 7fbe66600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbe60102360 0x7fbe6019a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:40:04.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:04.773+0000 7fbe66600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbe601079c0 0x7fbe6019ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:40:04.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:04.773+0000 7fbe66600640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fbe60111e10 con 0x7fbe601079c0 2024-09-10T09:40:04.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:04.773+0000 7fbe66600640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fbe60111c90 con 0x7fbe60102360 2024-09-10T09:40:04.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:04.773+0000 7fbe66600640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fbe60111f90 con 0x7fbe601019c0 2024-09-10T09:40:04.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:04.773+0000 7fbe65c00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbe601079c0 0x7fbe6019ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:40:04.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:04.773+0000 7fbe65c00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbe601079c0 0x7fbe6019ed70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51248/0 (socket says 172.21.15.5:51248) 2024-09-10T09:40:04.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:04.773+0000 7fbe65c00640 1 -- 172.21.15.5:0/3397056334 learned_addr learned my addr 172.21.15.5:0/3397056334 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:40:04.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:04.773+0000 7fbe65200640 1 --2- 172.21.15.5:0/3397056334 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbe601019c0 0x7fbe6019a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:40:04.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:04.773+0000 7fbe5fe00640 1 --2- 172.21.15.5:0/3397056334 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbe60102360 0x7fbe6019a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:40:04.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:04.774+0000 7fbe65200640 1 -- 172.21.15.5:0/3397056334 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbe60102360 msgr2=0x7fbe6019a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:40:04.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:04.774+0000 7fbe65200640 1 --2- 172.21.15.5:0/3397056334 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbe60102360 0x7fbe6019a9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:04.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:04.774+0000 7fbe65200640 1 -- 172.21.15.5:0/3397056334 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbe601079c0 msgr2=0x7fbe6019ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:40:04.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:04.774+0000 7fbe65200640 1 --2- 172.21.15.5:0/3397056334 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbe601079c0 0x7fbe6019ed70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:04.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:04.774+0000 7fbe65200640 1 -- 172.21.15.5:0/3397056334 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe6019f450 con 0x7fbe601019c0 2024-09-10T09:40:04.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:04.774+0000 7fbe65200640 1 --2- 172.21.15.5:0/3397056334 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbe601019c0 0x7fbe6019a4a0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fbe5400e9f0 tx=0x7fbe5400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:40:04.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:04.774+0000 7fbe5d600640 1 -- 172.21.15.5:0/3397056334 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe5400ccc0 con 0x7fbe601019c0 2024-09-10T09:40:04.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:04.774+0000 7fbe5d600640 1 -- 172.21.15.5:0/3397056334 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbe54010910 con 0x7fbe601019c0 2024-09-10T09:40:04.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:04.774+0000 7fbe66600640 1 -- 172.21.15.5:0/3397056334 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe6019f740 con 0x7fbe601019c0 2024-09-10T09:40:04.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:04.774+0000 7fbe5d600640 1 -- 172.21.15.5:0/3397056334 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe54018960 con 0x7fbe601019c0 2024-09-10T09:40:04.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:04.775+0000 7fbe66600640 1 -- 172.21.15.5:0/3397056334 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe601a7020 con 0x7fbe601019c0 2024-09-10T09:40:04.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:04.778+0000 7fbe6a6c7740 1 Processor -- start 2024-09-10T09:40:04.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:04.778+0000 7fbe6a6c7740 1 -- start start 2024-09-10T09:40:04.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:04.778+0000 7fbe6a6c7740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b27fc59e40 0x55b27fc5a220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:40:04.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:04.778+0000 7fbe6a6c7740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b27fc5a760 0x55b27fc64ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:40:04.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:04.778+0000 7fbe6a6c7740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b27fc65020 0x55b27fc67410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:40:04.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:04.778+0000 7fbe6a6c7740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55b27fbc03e0 con 0x55b27fc65020 2024-09-10T09:40:04.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:04.778+0000 7fbe6a6c7740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55b27fb67d80 con 0x55b27fc5a760 2024-09-10T09:40:04.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:04.778+0000 7fbe6a6c7740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55b27fbbc3a0 con 0x55b27fc59e40 2024-09-10T09:40:04.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:04.778+0000 7fbe5d600640 1 -- 172.21.15.5:0/3397056334 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7fbe54010ac0 con 0x7fbe601019c0 2024-09-10T09:40:04.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:04.779+0000 7fbe65c00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b27fc59e40 0x55b27fc5a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:40:04.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:04.779+0000 7fbe65c00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b27fc59e40 0x55b27fc5a220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:36480/0 (socket says 172.21.15.5:36480) 2024-09-10T09:40:04.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:04.779+0000 7fbe65c00640 1 -- 172.21.15.5:0/2581702870 learned_addr learned my addr 172.21.15.5:0/2581702870 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:40:04.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:04.779+0000 7fbe5fe00640 1 --2- 172.21.15.5:0/2581702870 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b27fc65020 0x55b27fc67410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:40:04.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:04.779+0000 7fbe65200640 1 --2- 172.21.15.5:0/2581702870 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b27fc5a760 0x55b27fc64ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:40:04.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:04.779+0000 7fbe65c00640 1 -- 172.21.15.5:0/2581702870 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b27fc5a760 msgr2=0x55b27fc64ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:40:04.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:04.779+0000 7fbe65c00640 1 --2- 172.21.15.5:0/2581702870 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b27fc5a760 0x55b27fc64ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:04.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:04.779+0000 7fbe65c00640 1 -- 172.21.15.5:0/2581702870 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b27fc65020 msgr2=0x55b27fc67410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:40:04.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:04.779+0000 7fbe65c00640 1 --2- 172.21.15.5:0/2581702870 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b27fc65020 0x55b27fc67410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:04.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:04.779+0000 7fbe65c00640 1 -- 172.21.15.5:0/2581702870 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b27fb86ae0 con 0x55b27fc59e40 2024-09-10T09:40:04.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:04.779+0000 7fbe65200640 1 --2- 172.21.15.5:0/2581702870 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b27fc5a760 0x55b27fc64ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:40:04.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:04.779+0000 7fbe65c00640 1 --2- 172.21.15.5:0/2581702870 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b27fc59e40 0x55b27fc5a220 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fbe58005fb0 tx=0x7fbe580093e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a592e84109531cd1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:40:04.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:04.780+0000 7fbe52000640 1 -- 172.21.15.5:0/2581702870 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe58003970 con 0x55b27fc59e40 2024-09-10T09:40:04.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:04.780+0000 7fbe52000640 1 -- 172.21.15.5:0/2581702870 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbe58036030 con 0x55b27fc59e40 2024-09-10T09:40:04.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:04.780+0000 7fbe52000640 1 -- 172.21.15.5:0/2581702870 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe58004460 con 0x55b27fc59e40 2024-09-10T09:40:04.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:04.780+0000 7fbe5d600640 1 --2- 172.21.15.5:0/3397056334 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fbe480a2b80 0x7fbe480a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:40:04.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:04.781+0000 7fbe5d600640 1 -- 172.21.15.5:0/3397056334 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fbe540cf7b0 con 0x7fbe601019c0 2024-09-10T09:40:04.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:04.781+0000 7fbe6a6c7740 1 -- 172.21.15.5:0/2581702870 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b27fc59e40 msgr2=0x55b27fc5a220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:40:04.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:04.781+0000 7fbe6a6c7740 1 --2- 172.21.15.5:0/2581702870 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b27fc59e40 0x55b27fc5a220 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fbe58005fb0 tx=0x7fbe580093e0 comp rx=0 tx=0).stop 2024-09-10T09:40:04.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:04.781+0000 7fbe65200640 1 --2- 172.21.15.5:0/3397056334 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fbe480a2b80 0x7fbe480a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:40:04.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:04.781+0000 7fbe6a6c7740 1 -- 172.21.15.5:0/2581702870 shutdown_connections 2024-09-10T09:40:04.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:04.781+0000 7fbe6a6c7740 1 --2- 172.21.15.5:0/2581702870 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b27fc65020 0x55b27fc67410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:04.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:04.781+0000 7fbe6a6c7740 1 --2- 172.21.15.5:0/2581702870 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b27fc5a760 0x55b27fc64ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:04.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:04.781+0000 7fbe6a6c7740 1 --2- 172.21.15.5:0/2581702870 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b27fc59e40 0x55b27fc5a220 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:04.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:04.781+0000 7fbe6a6c7740 1 -- 172.21.15.5:0/2581702870 >> 172.21.15.5:0/2581702870 conn(0x55b27fc563d0 msgr2=0x55b27fc567b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:40:04.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:04.781+0000 7fbe6a6c7740 1 -- 172.21.15.5:0/2581702870 shutdown_connections 2024-09-10T09:40:04.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:04.781+0000 7fbe6a6c7740 1 -- 172.21.15.5:0/2581702870 wait complete. 2024-09-10T09:40:04.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:04.782+0000 7fbe65200640 1 --2- 172.21.15.5:0/3397056334 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fbe480a2b80 0x7fbe480a5040 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fbe54098640 tx=0x7fbe54009700 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:40:04.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:04.782+0000 7fbe6a6c7740 1 -- start start 2024-09-10T09:40:04.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:04.782+0000 7fbe6a6c7740 1 Processor -- start 2024-09-10T09:40:04.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:04.782+0000 7fbe6a6c7740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b27fc59e40 0x55b27fc61f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:40:04.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:04.782+0000 7fbe6a6c7740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b27fc5a760 0x55b27fc62470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:40:04.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:04.782+0000 7fbe6a6c7740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b27fc65020 0x55b27fc629b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:40:04.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:04.782+0000 7fbe6a6c7740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55b27fc67bf0 con 0x55b27fc59e40 2024-09-10T09:40:04.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:04.782+0000 7fbe6a6c7740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55b27fb7b0f0 con 0x55b27fc5a760 2024-09-10T09:40:04.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:04.782+0000 7fbe6a6c7740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55b27fc67d70 con 0x55b27fc65020 2024-09-10T09:40:04.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:04.782+0000 7fbe6a6c7740 10 client.0 _ll_register_callbacks cb 0x55b27fb62bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:40:04.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:04.783+0000 7fbe65c00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b27fc59e40 0x55b27fc61f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:40:04.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:04.783+0000 7fbe65c00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b27fc59e40 0x55b27fc61f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51274/0 (socket says 172.21.15.5:51274) 2024-09-10T09:40:04.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:04.783+0000 7fbe65c00640 1 -- 172.21.15.5:0/2882221042 learned_addr learned my addr 172.21.15.5:0/2882221042 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:40:04.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:04.783+0000 7fbe5fe00640 1 --2- 172.21.15.5:0/2882221042 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b27fc5a760 0x55b27fc62470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:40:04.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:04.783+0000 7fbe65c00640 1 --2- 172.21.15.5:0/2882221042 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b27fc65020 0x55b27fc629b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:40:04.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:04.783+0000 7fbe5fe00640 1 -- 172.21.15.5:0/2882221042 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b27fc65020 msgr2=0x55b27fc629b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:40:04.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:04.783+0000 7fbe5fe00640 1 --2- 172.21.15.5:0/2882221042 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b27fc65020 0x55b27fc629b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:04.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:04.783+0000 7fbe5fe00640 1 -- 172.21.15.5:0/2882221042 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b27fc59e40 msgr2=0x55b27fc61f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:40:04.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:04.783+0000 7fbe5fe00640 1 --2- 172.21.15.5:0/2882221042 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b27fc59e40 0x55b27fc61f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:04.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:04.783+0000 7fbe5fe00640 1 -- 172.21.15.5:0/2882221042 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b27f9313c0 con 0x55b27fc5a760 2024-09-10T09:40:04.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:04.783+0000 7fbe5fe00640 1 --2- 172.21.15.5:0/2882221042 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b27fc5a760 0x55b27fc62470 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fbe4c0126d0 tx=0x7fbe4c012b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f1d807177418677 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:40:04.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:04.783+0000 7fbe46a00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-09-10T09:40:04.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:04.784+0000 7fbe46a00640 1 -- 172.21.15.5:0/2882221042 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe4c004020 con 0x55b27fc5a760 2024-09-10T09:40:04.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:04.784+0000 7fbe6a6c7740 1 -- 172.21.15.5:0/2882221042 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbe58036030 con 0x55b27fc5a760 2024-09-10T09:40:04.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:04.784+0000 7fbe46a00640 1 -- 172.21.15.5:0/2882221042 <== mon.2 v2:172.21.15.102:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fbe4c0041c0 con 0x55b27fc5a760 2024-09-10T09:40:04.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:04.784+0000 7fbe46a00640 1 -- 172.21.15.5:0/2882221042 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fbe4c025050 con 0x55b27fc5a760 2024-09-10T09:40:04.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:04.784+0000 7fbe6a6c7740 10 client.25757 fetch_fsmap learned FSMap version 30 2024-09-10T09:40:04.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:04.784+0000 7fbe6a6c7740 1 -- 172.21.15.5:0/2882221042 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55b27fc73d10 con 0x55b27fc5a760 2024-09-10T09:40:04.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:04.784+0000 7fbe46a00640 1 -- 172.21.15.5:0/2882221042 <== mon.2 v2:172.21.15.102:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbe58036030 con 0x55b27fc5a760 2024-09-10T09:40:04.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:04.785+0000 7fbe46a00640 1 -- 172.21.15.5:0/2882221042 <== mon.2 v2:172.21.15.102:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fbe4c003a60 con 0x55b27fc5a760 2024-09-10T09:40:04.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:04.785+0000 7fbe6a6c7740 10 client.25757 fetch_fsmap finished waiting for FSMap version 30 2024-09-10T09:40:04.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:04.785+0000 7fbe6a6c7740 10 client.25757 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T09:40:04.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:04.785+0000 7fbe6a6c7740 20 client.25757 populate_metadata read hostname 'smithi005' 2024-09-10T09:40:04.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:04.785+0000 7fbe6a6c7740 1 --2- 172.21.15.5:0/2882221042 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55b27fc75120 0x55b27fc77530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:40:04.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:04.785+0000 7fbe6a6c7740 4 client.25757 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:40:04.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:04.785+0000 7fbe6a6c7740 1 -- 172.21.15.5:0/2882221042 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b27fbcd950 con 0x55b27fc75120 2024-09-10T09:40:04.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:04.785+0000 7fbe65c00640 1 --2- 172.21.15.5:0/2882221042 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55b27fc75120 0x55b27fc77530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:40:04.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:04.785+0000 7fbe65c00640 1 --2- 172.21.15.5:0/2882221042 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55b27fc75120 0x55b27fc77530 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1d52eccfe92743db server_cookie=50d51cb66b4d09c4 in_seq=0 out_seq=0 2024-09-10T09:40:04.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:04.785+0000 7fbe46a00640 10 client.25757 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T09:40:04.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:04.786+0000 7fbe46a00640 1 -- 172.21.15.5:0/2882221042 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b27fbcd950 con 0x55b27fc75120 2024-09-10T09:40:04.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:04.786+0000 7fbe46a00640 10 client.25757 handle_command_reply: tid=0 2024-09-10T09:40:04.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:04.786+0000 7fbe6a6c7740 1 client.25757 shutdown 2024-09-10T09:40:04.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:04.786+0000 7fbe6a6c7740 1 -- 172.21.15.5:0/2882221042 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b27fc5a760 msgr2=0x55b27fc62470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:40:04.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:04.786+0000 7fbe6a6c7740 1 --2- 172.21.15.5:0/2882221042 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b27fc5a760 0x55b27fc62470 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fbe4c0126d0 tx=0x7fbe4c012b90 comp rx=0 tx=0).stop 2024-09-10T09:40:04.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:04.787+0000 7fbe6a6c7740 1 -- 172.21.15.5:0/2882221042 shutdown_connections 2024-09-10T09:40:04.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:04.787+0000 7fbe6a6c7740 1 --2- 172.21.15.5:0/2882221042 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55b27fc75120 0x55b27fc77530 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:04.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:04.787+0000 7fbe6a6c7740 1 --2- 172.21.15.5:0/2882221042 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b27fc65020 0x55b27fc629b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:04.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:04.787+0000 7fbe6a6c7740 1 --2- 172.21.15.5:0/2882221042 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b27fc5a760 0x55b27fc62470 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:04.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:04.787+0000 7fbe6a6c7740 1 --2- 172.21.15.5:0/2882221042 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b27fc59e40 0x55b27fc61f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:04.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:04.787+0000 7fbe46a00640 0 client.25757 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T09:40:04.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:04.787+0000 7fbe6a6c7740 1 -- 172.21.15.5:0/2882221042 >> 172.21.15.5:0/2882221042 conn(0x55b27fc563d0 msgr2=0x55b27fc5c0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:40:04.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:04.787+0000 7fbe6a6c7740 1 -- 172.21.15.5:0/2882221042 shutdown_connections 2024-09-10T09:40:04.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:04.787+0000 7fbe6a6c7740 1 -- 172.21.15.5:0/2882221042 wait complete. 2024-09-10T09:40:04.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:04.787+0000 7fbe6a6c7740 20 client.25757 trim_cache size 0 max 16384 2024-09-10T09:40:04.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:04.801+0000 7fbe6a6c7740 1 Processor -- start 2024-09-10T09:40:04.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:04.802+0000 7fbe6a6c7740 1 -- start start 2024-09-10T09:40:04.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:04.802+0000 7fbe6a6c7740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b27fc59e40 0x55b27fc5c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:40:04.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:04.802+0000 7fbe6a6c7740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b27fc5a760 0x55b27fc7ce00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:40:04.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:04.802+0000 7fbe6a6c7740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b27fc65020 0x55b27fc7d340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:40:04.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:04.802+0000 7fbe6a6c7740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fbe601b0ea0 con 0x55b27fc65020 2024-09-10T09:40:04.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:04.802+0000 7fbe6a6c7740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fbe601b29e0 con 0x55b27fc59e40 2024-09-10T09:40:04.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:04.802+0000 7fbe6a6c7740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fbe601b1570 con 0x55b27fc5a760 2024-09-10T09:40:04.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:04.802+0000 7fbe65c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b27fc59e40 0x55b27fc5c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:40:04.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:04.802+0000 7fbe65c00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b27fc65020 0x55b27fc7d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:40:04.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:04.802+0000 7fbe5fe00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b27fc5a760 0x55b27fc7ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:40:04.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:04.802+0000 7fbe5fe00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b27fc5a760 0x55b27fc7ce00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:36494/0 (socket says 172.21.15.5:36494) 2024-09-10T09:40:04.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:04.802+0000 7fbe65c00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b27fc65020 0x55b27fc7d340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51278/0 (socket says 172.21.15.5:51278) 2024-09-10T09:40:04.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:04.802+0000 7fbe5fe00640 1 -- 172.21.15.5:0/3364041196 learned_addr learned my addr 172.21.15.5:0/3364041196 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:40:04.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:04.803+0000 7fbe5fe00640 1 -- 172.21.15.5:0/3364041196 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b27fc59e40 msgr2=0x55b27fc5c8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:40:04.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:04.803+0000 7fbe5fe00640 1 --2- 172.21.15.5:0/3364041196 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b27fc59e40 0x55b27fc5c8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:04.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:04.803+0000 7fbe5fe00640 1 -- 172.21.15.5:0/3364041196 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b27fc65020 msgr2=0x55b27fc7d340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:40:04.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:04.803+0000 7fbe5fe00640 1 --2- 172.21.15.5:0/3364041196 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b27fc65020 0x55b27fc7d340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:04.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:04.803+0000 7fbe5fe00640 1 -- 172.21.15.5:0/3364041196 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b27fc5a590 con 0x55b27fc5a760 2024-09-10T09:40:04.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:04.803+0000 7fbe5fe00640 1 --2- 172.21.15.5:0/3364041196 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b27fc5a760 0x55b27fc7ce00 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fbe4c009de0 tx=0x7fbe4c005db0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ec68b05968848b01 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:40:04.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:04.804+0000 7fbe51600640 1 -- 172.21.15.5:0/3364041196 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe4c009e80 con 0x55b27fc5a760 2024-09-10T09:40:04.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:04.804+0000 7fbe51600640 1 -- 172.21.15.5:0/3364041196 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbe4c010dd0 con 0x55b27fc5a760 2024-09-10T09:40:04.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:04.804+0000 7fbe51600640 1 -- 172.21.15.5:0/3364041196 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe4c00fda0 con 0x55b27fc5a760 2024-09-10T09:40:04.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:04.804+0000 7fbe6a6c7740 1 -- 172.21.15.5:0/3364041196 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b27fc5a760 msgr2=0x55b27fc7ce00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:40:04.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:04.804+0000 7fbe6a6c7740 1 --2- 172.21.15.5:0/3364041196 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b27fc5a760 0x55b27fc7ce00 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fbe4c009de0 tx=0x7fbe4c005db0 comp rx=0 tx=0).stop 2024-09-10T09:40:04.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:04.805+0000 7fbe6a6c7740 1 -- 172.21.15.5:0/3364041196 shutdown_connections 2024-09-10T09:40:04.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:04.805+0000 7fbe6a6c7740 1 --2- 172.21.15.5:0/3364041196 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b27fc65020 0x55b27fc7d340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:04.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:04.805+0000 7fbe6a6c7740 1 --2- 172.21.15.5:0/3364041196 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b27fc5a760 0x55b27fc7ce00 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:04.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:04.805+0000 7fbe6a6c7740 1 --2- 172.21.15.5:0/3364041196 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b27fc59e40 0x55b27fc5c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:04.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:04.805+0000 7fbe6a6c7740 1 -- 172.21.15.5:0/3364041196 >> 172.21.15.5:0/3364041196 conn(0x55b27fc563d0 msgr2=0x55b27fc63fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:40:04.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:04.805+0000 7fbe6a6c7740 1 -- 172.21.15.5:0/3364041196 shutdown_connections 2024-09-10T09:40:04.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:04.805+0000 7fbe6a6c7740 1 -- 172.21.15.5:0/3364041196 wait complete. 2024-09-10T09:40:04.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:04.805+0000 7fbe6a6c7740 1 -- start start 2024-09-10T09:40:04.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:04.806+0000 7fbe6a6c7740 1 Processor -- start 2024-09-10T09:40:04.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:04.806+0000 7fbe6a6c7740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b27fc59e40 0x55b27fc68d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:40:04.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:04.806+0000 7fbe6a6c7740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b27fc5a760 0x55b27fc69260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:40:04.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:04.806+0000 7fbe6a6c7740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b27fc65020 0x55b27fc697a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:40:04.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:04.806+0000 7fbe6a6c7740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fbe601b2310 con 0x55b27fc65020 2024-09-10T09:40:04.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:04.806+0000 7fbe6a6c7740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fbe601b1c40 con 0x55b27fc59e40 2024-09-10T09:40:04.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:04.806+0000 7fbe6a6c7740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fbe601aee80 con 0x55b27fc5a760 2024-09-10T09:40:04.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:04.806+0000 7fbe6a6c7740 10 client.0 _ll_register_callbacks cb 0x7fbe4c009910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:40:04.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:04.806+0000 7fbe65c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b27fc59e40 0x55b27fc68d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:40:04.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:04.806+0000 7fbe5fe00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b27fc5a760 0x55b27fc69260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:40:04.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:04.806+0000 7fbe65c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b27fc59e40 0x55b27fc68d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:35760/0 (socket says 172.21.15.5:35760) 2024-09-10T09:40:04.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:04.806+0000 7fbe65c00640 1 -- 172.21.15.5:0/1580767756 learned_addr learned my addr 172.21.15.5:0/1580767756 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:40:04.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:04.807+0000 7fbe65c00640 1 --2- 172.21.15.5:0/1580767756 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b27fc65020 0x55b27fc697a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:40:04.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:04.807+0000 7fbe65c00640 1 -- 172.21.15.5:0/1580767756 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b27fc5a760 msgr2=0x55b27fc69260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:40:04.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:04.807+0000 7fbe65c00640 1 --2- 172.21.15.5:0/1580767756 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b27fc5a760 0x55b27fc69260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:04.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:04.807+0000 7fbe65c00640 1 -- 172.21.15.5:0/1580767756 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b27fc65020 msgr2=0x55b27fc697a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:40:04.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:04.807+0000 7fbe65c00640 1 --2- 172.21.15.5:0/1580767756 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b27fc65020 0x55b27fc697a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:04.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:04.807+0000 7fbe65c00640 1 -- 172.21.15.5:0/1580767756 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b27fcdedd0 con 0x55b27fc59e40 2024-09-10T09:40:04.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:04.807+0000 7fbe5fe00640 1 --2- 172.21.15.5:0/1580767756 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b27fc5a760 0x55b27fc69260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:40:04.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:04.807+0000 7fbe65c00640 1 --2- 172.21.15.5:0/1580767756 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b27fc59e40 0x55b27fc68d20 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fbe58009620 tx=0x7fbe58005d80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7ec4f54b6e76549a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:40:04.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:04.807+0000 7fbe46a00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-09-10T09:40:04.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:04.807+0000 7fbe46a00640 1 -- 172.21.15.5:0/1580767756 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe58003970 con 0x55b27fc59e40 2024-09-10T09:40:04.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:04.808+0000 7fbe6a6c7740 1 -- 172.21.15.5:0/1580767756 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbe4c010dd0 con 0x55b27fc59e40 2024-09-10T09:40:04.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:04.808+0000 7fbe46a00640 1 -- 172.21.15.5:0/1580767756 <== mon.2 v2:172.21.15.102:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fbe5804a040 con 0x55b27fc59e40 2024-09-10T09:40:04.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:04.808+0000 7fbe46a00640 1 -- 172.21.15.5:0/1580767756 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fbe58003b10 con 0x55b27fc59e40 2024-09-10T09:40:04.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:04.808+0000 7fbe6a6c7740 10 client.25763 fetch_fsmap learned FSMap version 30 2024-09-10T09:40:04.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:04.808+0000 7fbe6a6c7740 1 -- 172.21.15.5:0/1580767756 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55b27fce00b0 con 0x55b27fc59e40 2024-09-10T09:40:04.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:04.808+0000 7fbe46a00640 1 -- 172.21.15.5:0/1580767756 <== mon.2 v2:172.21.15.102:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbe4c010dd0 con 0x55b27fc59e40 2024-09-10T09:40:04.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:04.808+0000 7fbe46a00640 1 -- 172.21.15.5:0/1580767756 <== mon.2 v2:172.21.15.102:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fbe58043070 con 0x55b27fc59e40 2024-09-10T09:40:04.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:04.808+0000 7fbe6a6c7740 10 client.25763 fetch_fsmap finished waiting for FSMap version 30 2024-09-10T09:40:04.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:04.808+0000 7fbe6a6c7740 10 client.25763 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T09:40:04.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:04.809+0000 7fbe6a6c7740 20 client.25763 populate_metadata read hostname 'smithi005' 2024-09-10T09:40:04.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:04.809+0000 7fbe6a6c7740 1 --2- 172.21.15.5:0/1580767756 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55b27fce03f0 0x55b27fce27e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:40:04.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:04.809+0000 7fbe6a6c7740 4 client.25763 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T09:40:04.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:04.809+0000 7fbe6a6c7740 1 -- 172.21.15.5:0/1580767756 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b27fbc7e60 con 0x55b27fce03f0 2024-09-10T09:40:04.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:04.809+0000 7fbe5fe00640 1 --2- 172.21.15.5:0/1580767756 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55b27fce03f0 0x55b27fce27e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:40:04.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:04.809+0000 7fbe5fe00640 1 --2- 172.21.15.5:0/1580767756 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55b27fce03f0 0x55b27fce27e0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=91f83fc43a665d93 server_cookie=102195ea596ca77c in_seq=0 out_seq=0 2024-09-10T09:40:04.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:04.809+0000 7fbe46a00640 10 client.25763 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T09:40:04.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:04.810+0000 7fbe46a00640 1 -- 172.21.15.5:0/1580767756 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b27fbc7e60 con 0x55b27fce03f0 2024-09-10T09:40:04.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:04.810+0000 7fbe46a00640 10 client.25763 handle_command_reply: tid=0 2024-09-10T09:40:04.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:04.810+0000 7fbe6a6c7740 1 client.25763 shutdown 2024-09-10T09:40:04.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:04.810+0000 7fbe6a6c7740 1 -- 172.21.15.5:0/1580767756 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b27fc59e40 msgr2=0x55b27fc68d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:40:04.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:04.810+0000 7fbe6a6c7740 1 --2- 172.21.15.5:0/1580767756 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b27fc59e40 0x55b27fc68d20 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fbe58009620 tx=0x7fbe58005d80 comp rx=0 tx=0).stop 2024-09-10T09:40:04.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:04.810+0000 7fbe6a6c7740 1 -- 172.21.15.5:0/1580767756 shutdown_connections 2024-09-10T09:40:04.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:04.810+0000 7fbe6a6c7740 1 --2- 172.21.15.5:0/1580767756 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55b27fce03f0 0x55b27fce27e0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:04.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:04.810+0000 7fbe6a6c7740 1 --2- 172.21.15.5:0/1580767756 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b27fc65020 0x55b27fc697a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:04.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:04.810+0000 7fbe6a6c7740 1 --2- 172.21.15.5:0/1580767756 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b27fc5a760 0x55b27fc69260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:04.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:04.810+0000 7fbe6a6c7740 1 --2- 172.21.15.5:0/1580767756 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b27fc59e40 0x55b27fc68d20 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:04.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:04.810+0000 7fbe6a6c7740 1 -- 172.21.15.5:0/1580767756 >> 172.21.15.5:0/1580767756 conn(0x55b27fc563d0 msgr2=0x55b27fc61ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:40:04.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:04.810+0000 7fbe46a00640 0 client.25763 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T09:40:04.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:04.810+0000 7fbe6a6c7740 1 -- 172.21.15.5:0/1580767756 shutdown_connections 2024-09-10T09:40:04.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:04.810+0000 7fbe6a6c7740 1 -- 172.21.15.5:0/1580767756 wait complete. 2024-09-10T09:40:04.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:04.811+0000 7fbe6a6c7740 20 client.25763 trim_cache size 0 max 16384 2024-09-10T09:40:04.825 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-10T09:40:04.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:04.811+0000 7fbe66600640 1 -- 172.21.15.5:0/3397056334 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fbe480a2b80 msgr2=0x7fbe480a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:40:04.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:04.811+0000 7fbe66600640 1 --2- 172.21.15.5:0/3397056334 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fbe480a2b80 0x7fbe480a5040 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fbe54098640 tx=0x7fbe54009700 comp rx=0 tx=0).stop 2024-09-10T09:40:04.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:04.811+0000 7fbe66600640 1 -- 172.21.15.5:0/3397056334 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbe601019c0 msgr2=0x7fbe6019a4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:40:04.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:04.811+0000 7fbe66600640 1 --2- 172.21.15.5:0/3397056334 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbe601019c0 0x7fbe6019a4a0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fbe5400e9f0 tx=0x7fbe5400eeb0 comp rx=0 tx=0).stop 2024-09-10T09:40:04.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:04.811+0000 7fbe66600640 1 -- 172.21.15.5:0/3397056334 shutdown_connections 2024-09-10T09:40:04.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:04.811+0000 7fbe66600640 1 --2- 172.21.15.5:0/3397056334 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fbe480a2b80 0x7fbe480a5040 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:04.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:04.811+0000 7fbe66600640 1 --2- 172.21.15.5:0/3397056334 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbe601079c0 0x7fbe6019ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:04.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:04.811+0000 7fbe66600640 1 --2- 172.21.15.5:0/3397056334 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbe60102360 0x7fbe6019a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:04.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:04.811+0000 7fbe66600640 1 --2- 172.21.15.5:0/3397056334 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbe601019c0 0x7fbe6019a4a0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:04.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:04.811+0000 7fbe66600640 1 -- 172.21.15.5:0/3397056334 >> 172.21.15.5:0/3397056334 conn(0x7fbe600fd740 msgr2=0x7fbe600fdd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:40:04.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:04.812+0000 7fbe66600640 1 -- 172.21.15.5:0/3397056334 shutdown_connections 2024-09-10T09:40:04.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:04.812+0000 7fbe66600640 1 -- 172.21.15.5:0/3397056334 wait complete. 2024-09-10T09:40:05.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:40:04 smithi005 ceph-mon[30350]: pgmap v445: 129 pgs: 129 active+clean; 666 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-09-10T09:40:05.056 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:40:04 smithi159 ceph-mon[35513]: pgmap v445: 129 pgs: 129 active+clean; 666 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-09-10T09:40:05.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:40:04 smithi102 ceph-mon[35442]: pgmap v445: 129 pgs: 129 active+clean; 666 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-09-10T09:40:06.631 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-10T09:40:06.631 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-10T09:40:07.027 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:40:07.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:40:06 smithi005 ceph-mon[30350]: pgmap v446: 129 pgs: 129 active+clean; 666 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 85 B/s wr, 12 op/s 2024-09-10T09:40:07.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:40:06 smithi159 ceph-mon[35513]: pgmap v446: 129 pgs: 129 active+clean; 666 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 85 B/s wr, 12 op/s 2024-09-10T09:40:07.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:40:06 smithi102 ceph-mon[35442]: pgmap v446: 129 pgs: 129 active+clean; 666 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 85 B/s wr, 12 op/s 2024-09-10T09:40:07.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:07.620+0000 7fc7d3e00640 1 -- 172.21.15.5:0/183107415 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc7cc074620 msgr2=0x7fc7cc074a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:40:07.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:07.620+0000 7fc7d3e00640 1 --2- 172.21.15.5:0/183107415 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc7cc074620 0x7fc7cc074a00 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fc7c0009900 tx=0x7fc7c002a480 comp rx=0 tx=0).stop 2024-09-10T09:40:07.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:07.620+0000 7fc7d3e00640 1 -- 172.21.15.5:0/183107415 shutdown_connections 2024-09-10T09:40:07.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:07.620+0000 7fc7d3e00640 1 --2- 172.21.15.5:0/183107415 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc7cc10e3c0 0x7fc7cc110810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:07.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:07.620+0000 7fc7d3e00640 1 --2- 172.21.15.5:0/183107415 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc7cc074f40 0x7fc7cc10de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:07.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:07.620+0000 7fc7d3e00640 1 --2- 172.21.15.5:0/183107415 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc7cc074620 0x7fc7cc074a00 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:07.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:07.620+0000 7fc7d3e00640 1 -- 172.21.15.5:0/183107415 >> 172.21.15.5:0/183107415 conn(0x7fc7cc0fd8c0 msgr2=0x7fc7cc0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:40:07.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:07.620+0000 7fc7d3e00640 1 -- 172.21.15.5:0/183107415 shutdown_connections 2024-09-10T09:40:07.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:07.620+0000 7fc7d3e00640 1 -- 172.21.15.5:0/183107415 wait complete. 2024-09-10T09:40:07.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:07.621+0000 7fc7d3e00640 1 Processor -- start 2024-09-10T09:40:07.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:07.621+0000 7fc7d3e00640 1 -- start start 2024-09-10T09:40:07.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:07.621+0000 7fc7d3e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc7cc074620 0x7fc7cc19a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:40:07.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:07.621+0000 7fc7d3e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc7cc074f40 0x7fc7cc19ac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:40:07.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:07.621+0000 7fc7d3e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc7cc10e3c0 0x7fc7cc19efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:40:07.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:07.621+0000 7fc7d3e00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc7cc114170 con 0x7fc7cc074f40 2024-09-10T09:40:07.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:07.621+0000 7fc7d3e00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fc7cc113ff0 con 0x7fc7cc074620 2024-09-10T09:40:07.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:07.621+0000 7fc7d3e00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fc7cc1142f0 con 0x7fc7cc10e3c0 2024-09-10T09:40:07.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:07.622+0000 7fc7d0a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc7cc074f40 0x7fc7cc19ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:40:07.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:07.622+0000 7fc7d1400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc7cc074620 0x7fc7cc19a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:40:07.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:07.622+0000 7fc7d1400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc7cc074620 0x7fc7cc19a710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:35788/0 (socket says 172.21.15.5:35788) 2024-09-10T09:40:07.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:07.622+0000 7fc7d1400640 1 -- 172.21.15.5:0/1709749218 learned_addr learned my addr 172.21.15.5:0/1709749218 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:40:07.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:07.622+0000 7fc7d1e00640 1 --2- 172.21.15.5:0/1709749218 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc7cc10e3c0 0x7fc7cc19efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:40:07.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:07.622+0000 7fc7d1400640 1 -- 172.21.15.5:0/1709749218 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc7cc10e3c0 msgr2=0x7fc7cc19efe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:40:07.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:07.622+0000 7fc7d1400640 1 --2- 172.21.15.5:0/1709749218 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc7cc10e3c0 0x7fc7cc19efe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:07.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:07.622+0000 7fc7d1400640 1 -- 172.21.15.5:0/1709749218 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc7cc074f40 msgr2=0x7fc7cc19ac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:40:07.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:07.622+0000 7fc7d1400640 1 --2- 172.21.15.5:0/1709749218 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc7cc074f40 0x7fc7cc19ac50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:07.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:07.622+0000 7fc7d1400640 1 -- 172.21.15.5:0/1709749218 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7cc19f6c0 con 0x7fc7cc074620 2024-09-10T09:40:07.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:07.622+0000 7fc7d0a00640 1 --2- 172.21.15.5:0/1709749218 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc7cc074f40 0x7fc7cc19ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:40:07.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:07.623+0000 7fc7d1400640 1 --2- 172.21.15.5:0/1709749218 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc7cc074620 0x7fc7cc19a710 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fc7c0006010 tx=0x7fc7c0004040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:40:07.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:07.623+0000 7fc7ca000640 1 -- 172.21.15.5:0/1709749218 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7c003f030 con 0x7fc7cc074620 2024-09-10T09:40:07.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:07.623+0000 7fc7ca000640 1 -- 172.21.15.5:0/1709749218 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc7c0004440 con 0x7fc7cc074620 2024-09-10T09:40:07.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:07.623+0000 7fc7d3e00640 1 -- 172.21.15.5:0/1709749218 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7cc19f950 con 0x7fc7cc074620 2024-09-10T09:40:07.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:07.623+0000 7fc7ca000640 1 -- 172.21.15.5:0/1709749218 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7c0035070 con 0x7fc7cc074620 2024-09-10T09:40:07.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:07.623+0000 7fc7d3e00640 1 -- 172.21.15.5:0/1709749218 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7cc1a7290 con 0x7fc7cc074620 2024-09-10T09:40:07.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:07.626+0000 7fc7d6876740 1 Processor -- start 2024-09-10T09:40:07.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:07.626+0000 7fc7d6876740 1 -- start start 2024-09-10T09:40:07.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:07.626+0000 7fc7ca000640 1 -- 172.21.15.5:0/1709749218 <== mon.2 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7fc7c0042030 con 0x7fc7cc074620 2024-09-10T09:40:07.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:07.627+0000 7fc7d6876740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b4fc07dac0 0x55b4fc07dea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:40:07.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:07.627+0000 7fc7d6876740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b4fc07e3e0 0x55b4fc088760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:40:07.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:07.627+0000 7fc7d6876740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b4fc088ca0 0x55b4fc08b090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:40:07.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:07.627+0000 7fc7d6876740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55b4fbfe4290 con 0x55b4fc07dac0 2024-09-10T09:40:07.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:07.627+0000 7fc7d6876740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55b4fbf8bb40 con 0x55b4fc088ca0 2024-09-10T09:40:07.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:07.627+0000 7fc7d6876740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55b4fbfe0250 con 0x55b4fc07e3e0 2024-09-10T09:40:07.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:07.627+0000 7fc7d1400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b4fc07e3e0 0x55b4fc088760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:40:07.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:07.627+0000 7fc7d1400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b4fc07e3e0 0x55b4fc088760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:36538/0 (socket says 172.21.15.5:36538) 2024-09-10T09:40:07.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:07.627+0000 7fc7d1400640 1 -- 172.21.15.5:0/249804059 learned_addr learned my addr 172.21.15.5:0/249804059 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:40:07.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:07.627+0000 7fc7d1e00640 1 --2- 172.21.15.5:0/249804059 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b4fc07dac0 0x55b4fc07dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:40:07.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:07.627+0000 7fc7d0a00640 1 --2- 172.21.15.5:0/249804059 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b4fc088ca0 0x55b4fc08b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:40:07.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:07.627+0000 7fc7d1400640 1 -- 172.21.15.5:0/249804059 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b4fc088ca0 msgr2=0x55b4fc08b090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:40:07.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:07.627+0000 7fc7d1400640 1 --2- 172.21.15.5:0/249804059 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b4fc088ca0 0x55b4fc08b090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:07.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:07.628+0000 7fc7d1400640 1 -- 172.21.15.5:0/249804059 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b4fc07dac0 msgr2=0x55b4fc07dea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:40:07.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:07.628+0000 7fc7d1400640 1 --2- 172.21.15.5:0/249804059 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b4fc07dac0 0x55b4fc07dea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:07.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:07.628+0000 7fc7d1400640 1 -- 172.21.15.5:0/249804059 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b4fbd556a0 con 0x55b4fc07e3e0 2024-09-10T09:40:07.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:07.628+0000 7fc7d1400640 1 --2- 172.21.15.5:0/249804059 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b4fc07e3e0 0x55b4fc088760 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fc7c00f5c20 tx=0x7fc7c00f5d10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=886bd8b69f145fbb server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:40:07.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:07.628+0000 7fc7ca000640 1 --2- 172.21.15.5:0/1709749218 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fc7b40a2c00 0x7fc7b40a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:40:07.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:07.628+0000 7fc7ca000640 1 -- 172.21.15.5:0/1709749218 <== mon.2 v2:172.21.15.102:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fc7c00ef760 con 0x7fc7cc074620 2024-09-10T09:40:07.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:07.628+0000 7fc7d1e00640 1 --2- 172.21.15.5:0/1709749218 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fc7b40a2c00 0x7fc7b40a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:40:07.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:07.628+0000 7fc7be600640 1 -- 172.21.15.5:0/249804059 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7c00fe070 con 0x55b4fc07e3e0 2024-09-10T09:40:07.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:07.628+0000 7fc7be600640 1 -- 172.21.15.5:0/249804059 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc7c00c47e0 con 0x55b4fc07e3e0 2024-09-10T09:40:07.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:07.628+0000 7fc7be600640 1 -- 172.21.15.5:0/249804059 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7c00bb760 con 0x55b4fc07e3e0 2024-09-10T09:40:07.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:07.629+0000 7fc7d6876740 1 -- 172.21.15.5:0/249804059 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b4fc07e3e0 msgr2=0x55b4fc088760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:40:07.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:07.629+0000 7fc7d6876740 1 --2- 172.21.15.5:0/249804059 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b4fc07e3e0 0x55b4fc088760 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fc7c00f5c20 tx=0x7fc7c00f5d10 comp rx=0 tx=0).stop 2024-09-10T09:40:07.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:07.629+0000 7fc7d1e00640 1 --2- 172.21.15.5:0/1709749218 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fc7b40a2c00 0x7fc7b40a50c0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fc7c4005fd0 tx=0x7fc7c4009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:40:07.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:07.629+0000 7fc7d6876740 1 -- 172.21.15.5:0/249804059 shutdown_connections 2024-09-10T09:40:07.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:07.629+0000 7fc7d6876740 1 --2- 172.21.15.5:0/249804059 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b4fc088ca0 0x55b4fc08b090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:07.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:07.629+0000 7fc7d6876740 1 --2- 172.21.15.5:0/249804059 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b4fc07e3e0 0x55b4fc088760 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:07.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:07.629+0000 7fc7d6876740 1 --2- 172.21.15.5:0/249804059 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b4fc07dac0 0x55b4fc07dea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:07.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:07.629+0000 7fc7d6876740 1 -- 172.21.15.5:0/249804059 >> 172.21.15.5:0/249804059 conn(0x55b4fc07a050 msgr2=0x55b4fc07a430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:40:07.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:07.629+0000 7fc7d6876740 1 -- 172.21.15.5:0/249804059 shutdown_connections 2024-09-10T09:40:07.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:07.629+0000 7fc7d6876740 1 -- 172.21.15.5:0/249804059 wait complete. 2024-09-10T09:40:07.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:07.630+0000 7fc7d6876740 1 -- start start 2024-09-10T09:40:07.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:07.630+0000 7fc7d6876740 1 Processor -- start 2024-09-10T09:40:07.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:07.631+0000 7fc7d6876740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b4fc07dac0 0x55b4fc0885f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:40:07.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:07.631+0000 7fc7d6876740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b4fc07e3e0 0x55b4fc0858a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:40:07.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:07.631+0000 7fc7d6876740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b4fc088ca0 0x55b4fc085e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:40:07.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:07.631+0000 7fc7d6876740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55b4fc08b630 con 0x55b4fc07e3e0 2024-09-10T09:40:07.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:07.631+0000 7fc7d6876740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55b4fbf9efa0 con 0x55b4fc07dac0 2024-09-10T09:40:07.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:07.631+0000 7fc7d6876740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55b4fc08b7b0 con 0x55b4fc088ca0 2024-09-10T09:40:07.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:07.631+0000 7fc7d6876740 10 client.0 _ll_register_callbacks cb 0x55b4fbcf2ba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:40:07.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:07.631+0000 7fc7d0a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b4fc07e3e0 0x55b4fc0858a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:40:07.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:07.631+0000 7fc7d0a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b4fc07e3e0 0x55b4fc0858a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51330/0 (socket says 172.21.15.5:51330) 2024-09-10T09:40:07.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:07.631+0000 7fc7d0a00640 1 -- 172.21.15.5:0/1569202319 learned_addr learned my addr 172.21.15.5:0/1569202319 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:40:07.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:07.631+0000 7fc7d1e00640 1 --2- 172.21.15.5:0/1569202319 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b4fc088ca0 0x55b4fc085e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:40:07.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:07.631+0000 7fc7d1400640 1 --2- 172.21.15.5:0/1569202319 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b4fc07dac0 0x55b4fc0885f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:40:07.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:07.631+0000 7fc7d0a00640 1 -- 172.21.15.5:0/1569202319 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b4fc088ca0 msgr2=0x55b4fc085e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:40:07.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:07.631+0000 7fc7d0a00640 1 --2- 172.21.15.5:0/1569202319 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b4fc088ca0 0x55b4fc085e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:07.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:07.631+0000 7fc7d0a00640 1 -- 172.21.15.5:0/1569202319 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b4fc07dac0 msgr2=0x55b4fc0885f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:40:07.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:07.631+0000 7fc7d0a00640 1 --2- 172.21.15.5:0/1569202319 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b4fc07dac0 0x55b4fc0885f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:07.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:07.631+0000 7fc7d0a00640 1 -- 172.21.15.5:0/1569202319 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b4fc086630 con 0x55b4fc07e3e0 2024-09-10T09:40:07.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:07.632+0000 7fc7d0a00640 1 --2- 172.21.15.5:0/1569202319 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b4fc07e3e0 0x55b4fc0858a0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fc7b8012d20 tx=0x7fc7b800f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b544b0e88c64e3d5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:40:07.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:07.632+0000 7fc7b2a00640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-10T09:40:07.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:07.632+0000 7fc7b2a00640 1 -- 172.21.15.5:0/1569202319 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7b8012110 con 0x55b4fc07e3e0 2024-09-10T09:40:07.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:07.632+0000 7fc7b2a00640 1 -- 172.21.15.5:0/1569202319 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fc7b8004050 con 0x55b4fc07e3e0 2024-09-10T09:40:07.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:07.632+0000 7fc7d6876740 1 -- 172.21.15.5:0/1569202319 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc7c00c47e0 con 0x55b4fc07e3e0 2024-09-10T09:40:07.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:07.632+0000 7fc7b2a00640 1 -- 172.21.15.5:0/1569202319 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fc7b80122b0 con 0x55b4fc07e3e0 2024-09-10T09:40:07.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:07.632+0000 7fc7d6876740 10 client.15768 fetch_fsmap learned FSMap version 30 2024-09-10T09:40:07.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:07.632+0000 7fc7d6876740 1 -- 172.21.15.5:0/1569202319 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55b4fc0975d0 con 0x55b4fc07e3e0 2024-09-10T09:40:07.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:07.633+0000 7fc7b2a00640 1 -- 172.21.15.5:0/1569202319 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc7c00c47e0 con 0x55b4fc07e3e0 2024-09-10T09:40:07.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:07.633+0000 7fc7b2a00640 1 -- 172.21.15.5:0/1569202319 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fc7b8010430 con 0x55b4fc07e3e0 2024-09-10T09:40:07.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:07.633+0000 7fc7d6876740 10 client.15768 fetch_fsmap finished waiting for FSMap version 30 2024-09-10T09:40:07.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:07.633+0000 7fc7d6876740 10 client.15768 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T09:40:07.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:07.633+0000 7fc7d6876740 20 client.15768 populate_metadata read hostname 'smithi005' 2024-09-10T09:40:07.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:07.633+0000 7fc7d6876740 1 --2- 172.21.15.5:0/1569202319 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55b4fc098ae0 0x55b4fc09aed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:40:07.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:07.633+0000 7fc7d6876740 4 client.15768 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:40:07.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:07.633+0000 7fc7d6876740 1 -- 172.21.15.5:0/1569202319 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b4fbff1800 con 0x55b4fc098ae0 2024-09-10T09:40:07.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:07.633+0000 7fc7d1400640 1 --2- 172.21.15.5:0/1569202319 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55b4fc098ae0 0x55b4fc09aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:40:07.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:07.634+0000 7fc7d1400640 1 --2- 172.21.15.5:0/1569202319 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55b4fc098ae0 0x55b4fc09aed0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1c9e23c0b20277cc server_cookie=545f4e6da8b0114b in_seq=0 out_seq=0 2024-09-10T09:40:07.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:07.634+0000 7fc7b2a00640 10 client.15768 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T09:40:07.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:07.635+0000 7fc7b2a00640 1 -- 172.21.15.5:0/1569202319 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b4fbff1800 con 0x55b4fc098ae0 2024-09-10T09:40:07.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:07.635+0000 7fc7b2a00640 10 client.15768 handle_command_reply: tid=0 2024-09-10T09:40:07.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:07.635+0000 7fc7d6876740 1 client.15768 shutdown 2024-09-10T09:40:07.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:07.635+0000 7fc7d6876740 1 -- 172.21.15.5:0/1569202319 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b4fc07e3e0 msgr2=0x55b4fc0858a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:40:07.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:07.635+0000 7fc7d6876740 1 --2- 172.21.15.5:0/1569202319 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b4fc07e3e0 0x55b4fc0858a0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fc7b8012d20 tx=0x7fc7b800f790 comp rx=0 tx=0).stop 2024-09-10T09:40:07.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:07.635+0000 7fc7d6876740 1 -- 172.21.15.5:0/1569202319 shutdown_connections 2024-09-10T09:40:07.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:07.635+0000 7fc7d6876740 1 --2- 172.21.15.5:0/1569202319 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55b4fc098ae0 0x55b4fc09aed0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:07.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:07.635+0000 7fc7d6876740 1 --2- 172.21.15.5:0/1569202319 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b4fc088ca0 0x55b4fc085e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:07.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:07.635+0000 7fc7d6876740 1 --2- 172.21.15.5:0/1569202319 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b4fc07e3e0 0x55b4fc0858a0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:07.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:07.635+0000 7fc7d6876740 1 --2- 172.21.15.5:0/1569202319 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b4fc07dac0 0x55b4fc0885f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:07.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:07.635+0000 7fc7b2a00640 0 client.15768 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T09:40:07.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:07.635+0000 7fc7d6876740 1 -- 172.21.15.5:0/1569202319 >> 172.21.15.5:0/1569202319 conn(0x55b4fc07a050 msgr2=0x55b4fc07fd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:40:07.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:07.635+0000 7fc7d6876740 1 -- 172.21.15.5:0/1569202319 shutdown_connections 2024-09-10T09:40:07.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:07.635+0000 7fc7d6876740 1 -- 172.21.15.5:0/1569202319 wait complete. 2024-09-10T09:40:07.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:07.635+0000 7fc7d6876740 20 client.15768 trim_cache size 0 max 16384 2024-09-10T09:40:07.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:07.653+0000 7fc7d6876740 1 Processor -- start 2024-09-10T09:40:07.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:07.653+0000 7fc7d6876740 1 -- start start 2024-09-10T09:40:07.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:07.654+0000 7fc7d6876740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b4fc07a050 0x55b4fc0790b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:40:07.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:07.654+0000 7fc7d6876740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b4fc07dac0 0x55b4fc0a07c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:40:07.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:07.654+0000 7fc7d6876740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b4fc07e3e0 0x55b4fc0a0d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:40:07.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:07.654+0000 7fc7d6876740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc7cc1b10c0 con 0x55b4fc07e3e0 2024-09-10T09:40:07.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:07.654+0000 7fc7d6876740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fc7cc1b2c00 con 0x55b4fc07a050 2024-09-10T09:40:07.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:07.654+0000 7fc7d6876740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fc7cc1b1790 con 0x55b4fc07dac0 2024-09-10T09:40:07.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:07.654+0000 7fc7d1400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b4fc07a050 0x55b4fc0790b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:40:07.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:07.654+0000 7fc7d0a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b4fc07dac0 0x55b4fc0a07c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:40:07.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:07.654+0000 7fc7d1400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b4fc07a050 0x55b4fc0790b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:35824/0 (socket says 172.21.15.5:35824) 2024-09-10T09:40:07.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:07.654+0000 7fc7d1e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b4fc07e3e0 0x55b4fc0a0d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:40:07.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:07.654+0000 7fc7d1400640 1 -- 172.21.15.5:0/603733015 learned_addr learned my addr 172.21.15.5:0/603733015 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:40:07.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:07.655+0000 7fc7d0a00640 1 -- 172.21.15.5:0/603733015 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b4fc07a050 msgr2=0x55b4fc0790b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:40:07.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:07.655+0000 7fc7d0a00640 1 --2- 172.21.15.5:0/603733015 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b4fc07a050 0x55b4fc0790b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:07.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:07.655+0000 7fc7d0a00640 1 -- 172.21.15.5:0/603733015 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b4fc07e3e0 msgr2=0x55b4fc0a0d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:40:07.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:07.655+0000 7fc7d0a00640 1 --2- 172.21.15.5:0/603733015 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b4fc07e3e0 0x55b4fc0a0d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:07.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:07.655+0000 7fc7d0a00640 1 -- 172.21.15.5:0/603733015 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b4fc0868b0 con 0x55b4fc07dac0 2024-09-10T09:40:07.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:07.655+0000 7fc7d1400640 1 --2- 172.21.15.5:0/603733015 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b4fc07a050 0x55b4fc0790b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:40:07.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:07.655+0000 7fc7d0a00640 1 --2- 172.21.15.5:0/603733015 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b4fc07dac0 0x55b4fc0a07c0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fc7b8010850 tx=0x7fc7b800fc90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c8dd0bb32ce55944 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:40:07.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:07.655+0000 7fc7bdc00640 1 -- 172.21.15.5:0/603733015 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7b8012110 con 0x55b4fc07dac0 2024-09-10T09:40:07.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:07.655+0000 7fc7bdc00640 1 -- 172.21.15.5:0/603733015 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc7b802e070 con 0x55b4fc07dac0 2024-09-10T09:40:07.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:07.655+0000 7fc7bdc00640 1 -- 172.21.15.5:0/603733015 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7b8010660 con 0x55b4fc07dac0 2024-09-10T09:40:07.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:07.656+0000 7fc7d6876740 1 -- 172.21.15.5:0/603733015 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b4fc07dac0 msgr2=0x55b4fc0a07c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:40:07.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:07.656+0000 7fc7d6876740 1 --2- 172.21.15.5:0/603733015 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b4fc07dac0 0x55b4fc0a07c0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fc7b8010850 tx=0x7fc7b800fc90 comp rx=0 tx=0).stop 2024-09-10T09:40:07.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:07.656+0000 7fc7d6876740 1 -- 172.21.15.5:0/603733015 shutdown_connections 2024-09-10T09:40:07.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:07.656+0000 7fc7d6876740 1 --2- 172.21.15.5:0/603733015 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b4fc07e3e0 0x55b4fc0a0d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:07.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:07.656+0000 7fc7d6876740 1 --2- 172.21.15.5:0/603733015 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b4fc07dac0 0x55b4fc0a07c0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:07.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:07.656+0000 7fc7d6876740 1 --2- 172.21.15.5:0/603733015 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b4fc07a050 0x55b4fc0790b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:07.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:07.656+0000 7fc7d6876740 1 -- 172.21.15.5:0/603733015 >> 172.21.15.5:0/603733015 conn(0x55b4fc081330 msgr2=0x55b4fc0789a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:40:07.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:07.656+0000 7fc7d6876740 1 -- 172.21.15.5:0/603733015 shutdown_connections 2024-09-10T09:40:07.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:07.656+0000 7fc7d6876740 1 -- 172.21.15.5:0/603733015 wait complete. 2024-09-10T09:40:07.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:07.656+0000 7fc7d6876740 1 -- start start 2024-09-10T09:40:07.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:07.657+0000 7fc7d6876740 1 Processor -- start 2024-09-10T09:40:07.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:07.657+0000 7fc7d6876740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b4fc07a050 0x55b4fc0911d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:40:07.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:07.657+0000 7fc7d6876740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b4fc07dac0 0x55b4fc08d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:40:07.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:07.657+0000 7fc7d6876740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b4fc07e3e0 0x55b4fc08df70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:40:07.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:07.657+0000 7fc7d6876740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc7cc1b2530 con 0x55b4fc07e3e0 2024-09-10T09:40:07.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:07.657+0000 7fc7d6876740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fc7cc1b1e60 con 0x55b4fc07dac0 2024-09-10T09:40:07.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:07.657+0000 7fc7d6876740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fc7cc1af0a0 con 0x55b4fc07a050 2024-09-10T09:40:07.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:07.657+0000 7fc7d6876740 10 client.0 _ll_register_callbacks cb 0x55b4fc086800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:40:07.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:07.657+0000 7fc7d1400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b4fc07a050 0x55b4fc0911d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:40:07.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:07.657+0000 7fc7d1e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b4fc07e3e0 0x55b4fc08df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:40:07.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:07.657+0000 7fc7d0a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b4fc07dac0 0x55b4fc08d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:40:07.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:07.657+0000 7fc7d1400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b4fc07a050 0x55b4fc0911d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:36568/0 (socket says 172.21.15.5:36568) 2024-09-10T09:40:07.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:07.657+0000 7fc7d1400640 1 -- 172.21.15.5:0/1638101835 learned_addr learned my addr 172.21.15.5:0/1638101835 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:40:07.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:07.658+0000 7fc7d0a00640 1 -- 172.21.15.5:0/1638101835 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b4fc07a050 msgr2=0x55b4fc0911d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:40:07.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:07.658+0000 7fc7d0a00640 1 --2- 172.21.15.5:0/1638101835 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b4fc07a050 0x55b4fc0911d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:07.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:07.658+0000 7fc7d0a00640 1 -- 172.21.15.5:0/1638101835 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b4fc07e3e0 msgr2=0x55b4fc08df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:40:07.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:07.658+0000 7fc7d0a00640 1 --2- 172.21.15.5:0/1638101835 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b4fc07e3e0 0x55b4fc08df70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:07.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:07.658+0000 7fc7d0a00640 1 -- 172.21.15.5:0/1638101835 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b4fc08e7a0 con 0x55b4fc07dac0 2024-09-10T09:40:07.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:07.658+0000 7fc7d0a00640 1 --2- 172.21.15.5:0/1638101835 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b4fc07dac0 0x55b4fc08d9d0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fc7b802b040 tx=0x7fc7b8010770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7af6f3fbea3e4138 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:40:07.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:07.658+0000 7fc7b2a00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-09-10T09:40:07.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:07.658+0000 7fc7b2a00640 1 -- 172.21.15.5:0/1638101835 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7b8036030 con 0x55b4fc07dac0 2024-09-10T09:40:07.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:07.658+0000 7fc7d6876740 1 -- 172.21.15.5:0/1638101835 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc7b802e070 con 0x55b4fc07dac0 2024-09-10T09:40:07.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:07.658+0000 7fc7b2a00640 1 -- 172.21.15.5:0/1638101835 <== mon.2 v2:172.21.15.102:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fc7b8018070 con 0x55b4fc07dac0 2024-09-10T09:40:07.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:07.659+0000 7fc7b2a00640 1 -- 172.21.15.5:0/1638101835 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fc7b8043050 con 0x55b4fc07dac0 2024-09-10T09:40:07.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:07.659+0000 7fc7d6876740 10 client.25781 fetch_fsmap learned FSMap version 30 2024-09-10T09:40:07.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:07.659+0000 7fc7d6876740 1 -- 172.21.15.5:0/1638101835 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55b4fc1048b0 con 0x55b4fc07dac0 2024-09-10T09:40:07.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:07.659+0000 7fc7b2a00640 1 -- 172.21.15.5:0/1638101835 <== mon.2 v2:172.21.15.102:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc7b802e070 con 0x55b4fc07dac0 2024-09-10T09:40:07.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:07.659+0000 7fc7b2a00640 1 -- 172.21.15.5:0/1638101835 <== mon.2 v2:172.21.15.102:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7fc7b8003b40 con 0x55b4fc07dac0 2024-09-10T09:40:07.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:07.659+0000 7fc7d6876740 10 client.25781 fetch_fsmap finished waiting for FSMap version 30 2024-09-10T09:40:07.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:07.659+0000 7fc7d6876740 10 client.25781 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T09:40:07.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:07.659+0000 7fc7d6876740 20 client.25781 populate_metadata read hostname 'smithi005' 2024-09-10T09:40:07.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:07.659+0000 7fc7d6876740 1 --2- 172.21.15.5:0/1638101835 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55b4fc088ca0 0x55b4fc106ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:40:07.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:07.659+0000 7fc7d6876740 4 client.25781 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-10T09:40:07.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:07.659+0000 7fc7d6876740 1 -- 172.21.15.5:0/1638101835 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55b4fbfebd10 con 0x55b4fc088ca0 2024-09-10T09:40:07.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:07.660+0000 7fc7d1400640 1 --2- 172.21.15.5:0/1638101835 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55b4fc088ca0 0x55b4fc106ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:40:07.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:07.660+0000 7fc7d1400640 1 --2- 172.21.15.5:0/1638101835 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55b4fc088ca0 0x55b4fc106ec0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b1100f2e666ec3f server_cookie=aef173839fea6f26 in_seq=0 out_seq=0 2024-09-10T09:40:07.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:07.660+0000 7fc7b2a00640 10 client.25781 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T09:40:07.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:07.661+0000 7fc7b2a00640 1 -- 172.21.15.5:0/1638101835 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55b4fbfebd10 con 0x55b4fc088ca0 2024-09-10T09:40:07.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:07.661+0000 7fc7b2a00640 10 client.25781 handle_command_reply: tid=0 2024-09-10T09:40:07.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:07.661+0000 7fc7d6876740 1 client.25781 shutdown 2024-09-10T09:40:07.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:07.662+0000 7fc7d6876740 1 -- 172.21.15.5:0/1638101835 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b4fc07dac0 msgr2=0x55b4fc08d9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:40:07.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:07.662+0000 7fc7d6876740 1 --2- 172.21.15.5:0/1638101835 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b4fc07dac0 0x55b4fc08d9d0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fc7b802b040 tx=0x7fc7b8010770 comp rx=0 tx=0).stop 2024-09-10T09:40:07.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:07.662+0000 7fc7d6876740 1 -- 172.21.15.5:0/1638101835 shutdown_connections 2024-09-10T09:40:07.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:07.662+0000 7fc7d6876740 1 --2- 172.21.15.5:0/1638101835 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55b4fc088ca0 0x55b4fc106ec0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:07.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:07.662+0000 7fc7d6876740 1 --2- 172.21.15.5:0/1638101835 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b4fc07e3e0 0x55b4fc08df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:07.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:07.662+0000 7fc7d6876740 1 --2- 172.21.15.5:0/1638101835 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b4fc07dac0 0x55b4fc08d9d0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:07.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:07.662+0000 7fc7d6876740 1 --2- 172.21.15.5:0/1638101835 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b4fc07a050 0x55b4fc0911d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:07.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:07.662+0000 7fc7d6876740 1 -- 172.21.15.5:0/1638101835 >> 172.21.15.5:0/1638101835 conn(0x55b4fc081330 msgr2=0x55b4fc0791c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:40:07.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:07.662+0000 7fc7b2a00640 0 client.25781 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T09:40:07.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:07.662+0000 7fc7d6876740 1 -- 172.21.15.5:0/1638101835 shutdown_connections 2024-09-10T09:40:07.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:07.662+0000 7fc7d6876740 1 -- 172.21.15.5:0/1638101835 wait complete. 2024-09-10T09:40:07.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:07.662+0000 7fc7d6876740 20 client.25781 trim_cache size 0 max 16384 2024-09-10T09:40:07.665 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-10T09:40:07.665 INFO:teuthology.orchestra.run.smithi005.stdout: "return_code": 0, 2024-09-10T09:40:07.665 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_tag": "ce6f4f5d-2b80-422e-9fa0-6ff19dfb9c56", 2024-09-10T09:40:07.665 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": "asynchronous" 2024-09-10T09:40:07.665 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-10T09:40:07.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:07.663+0000 7fc7d3e00640 1 -- 172.21.15.5:0/1709749218 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fc7b40a2c00 msgr2=0x7fc7b40a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:40:07.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:07.663+0000 7fc7d3e00640 1 --2- 172.21.15.5:0/1709749218 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fc7b40a2c00 0x7fc7b40a50c0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fc7c4005fd0 tx=0x7fc7c4009500 comp rx=0 tx=0).stop 2024-09-10T09:40:07.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:07.663+0000 7fc7d3e00640 1 -- 172.21.15.5:0/1709749218 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc7cc074620 msgr2=0x7fc7cc19a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:40:07.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:07.663+0000 7fc7d3e00640 1 --2- 172.21.15.5:0/1709749218 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc7cc074620 0x7fc7cc19a710 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fc7c0006010 tx=0x7fc7c0004040 comp rx=0 tx=0).stop 2024-09-10T09:40:07.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:07.663+0000 7fc7d3e00640 1 -- 172.21.15.5:0/1709749218 shutdown_connections 2024-09-10T09:40:07.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:07.663+0000 7fc7d3e00640 1 --2- 172.21.15.5:0/1709749218 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fc7b40a2c00 0x7fc7b40a50c0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:07.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:07.663+0000 7fc7d3e00640 1 --2- 172.21.15.5:0/1709749218 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc7cc10e3c0 0x7fc7cc19efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:07.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:07.663+0000 7fc7d3e00640 1 --2- 172.21.15.5:0/1709749218 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc7cc074f40 0x7fc7cc19ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:07.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:07.663+0000 7fc7d3e00640 1 --2- 172.21.15.5:0/1709749218 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc7cc074620 0x7fc7cc19a710 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:07.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:07.663+0000 7fc7d3e00640 1 -- 172.21.15.5:0/1709749218 >> 172.21.15.5:0/1709749218 conn(0x7fc7cc0fd8c0 msgr2=0x7fc7cc0fe5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:40:07.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:07.663+0000 7fc7d3e00640 1 -- 172.21.15.5:0/1709749218 shutdown_connections 2024-09-10T09:40:07.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:07.663+0000 7fc7d3e00640 1 -- 172.21.15.5:0/1709749218 wait complete. 2024-09-10T09:40:08.625 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T09:40:09.009 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:40:09.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:40:08 smithi005 ceph-mon[30350]: pgmap v447: 129 pgs: 129 active+clean; 666 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-10T09:40:09.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:40:08 smithi005 ceph-mon[30350]: scrub queued for path: / 2024-09-10T09:40:09.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:40:08 smithi005 ceph-mon[30350]: scrub summary: idle+waiting paths [/] 2024-09-10T09:40:09.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:40:08 smithi005 ceph-mon[30350]: scrub summary: active paths [/] 2024-09-10T09:40:09.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:40:08 smithi005 ceph-mon[30350]: scrub summary: idle+waiting paths [/] 2024-09-10T09:40:09.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:40:08 smithi159 ceph-mon[35513]: pgmap v447: 129 pgs: 129 active+clean; 666 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-10T09:40:09.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:40:08 smithi159 ceph-mon[35513]: scrub queued for path: / 2024-09-10T09:40:09.101 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:40:08 smithi159 ceph-mon[35513]: scrub summary: idle+waiting paths [/] 2024-09-10T09:40:09.101 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:40:08 smithi159 ceph-mon[35513]: scrub summary: active paths [/] 2024-09-10T09:40:09.101 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:40:08 smithi159 ceph-mon[35513]: scrub summary: idle+waiting paths [/] 2024-09-10T09:40:09.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:40:08 smithi102 ceph-mon[35442]: pgmap v447: 129 pgs: 129 active+clean; 666 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-10T09:40:09.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:40:08 smithi102 ceph-mon[35442]: scrub queued for path: / 2024-09-10T09:40:09.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:40:08 smithi102 ceph-mon[35442]: scrub summary: idle+waiting paths [/] 2024-09-10T09:40:09.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:40:08 smithi102 ceph-mon[35442]: scrub summary: active paths [/] 2024-09-10T09:40:09.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:40:08 smithi102 ceph-mon[35442]: scrub summary: idle+waiting paths [/] 2024-09-10T09:40:09.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:09.593+0000 7f82ebe00640 1 -- 172.21.15.5:0/3171310215 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f82ec1047e0 msgr2=0x7f82ec108830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:40:09.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:09.593+0000 7f82ebe00640 1 --2- 172.21.15.5:0/3171310215 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f82ec1047e0 0x7f82ec108830 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f82d4009900 tx=0x7f82d402a3f0 comp rx=0 tx=0).stop 2024-09-10T09:40:09.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:09.593+0000 7f82ebe00640 1 -- 172.21.15.5:0/3171310215 shutdown_connections 2024-09-10T09:40:09.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:09.593+0000 7f82ebe00640 1 --2- 172.21.15.5:0/3171310215 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f82ec108f60 0x7f82ec110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:09.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:09.593+0000 7f82ebe00640 1 --2- 172.21.15.5:0/3171310215 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f82ec1047e0 0x7f82ec108830 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:09.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:09.593+0000 7f82ebe00640 1 --2- 172.21.15.5:0/3171310215 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f82ec103e30 0x7f82ec104210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:09.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:09.593+0000 7f82ebe00640 1 -- 172.21.15.5:0/3171310215 >> 172.21.15.5:0/3171310215 conn(0x7f82ec0ff8d0 msgr2=0x7f82ec101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:40:09.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:09.593+0000 7f82ebe00640 1 -- 172.21.15.5:0/3171310215 shutdown_connections 2024-09-10T09:40:09.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:09.593+0000 7f82ebe00640 1 -- 172.21.15.5:0/3171310215 wait complete. 2024-09-10T09:40:09.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:09.594+0000 7f82ebe00640 1 Processor -- start 2024-09-10T09:40:09.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:09.594+0000 7f82ebe00640 1 -- start start 2024-09-10T09:40:09.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:09.595+0000 7f82ebe00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f82ec103e30 0x7f82ec19a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:40:09.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:09.595+0000 7f82ebe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f82ec1047e0 0x7f82ec19a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:40:09.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:09.595+0000 7f82ebe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f82ec108f60 0x7f82ec19ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:40:09.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:09.595+0000 7f82ebe00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f82ec114010 con 0x7f82ec108f60 2024-09-10T09:40:09.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:09.595+0000 7f82ebe00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f82ec113e90 con 0x7f82ec1047e0 2024-09-10T09:40:09.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:09.595+0000 7f82ebe00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f82ec114190 con 0x7f82ec103e30 2024-09-10T09:40:09.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:09.595+0000 7f82eb400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f82ec108f60 0x7f82ec19ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:40:09.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:09.595+0000 7f82e3e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f82ec1047e0 0x7f82ec19a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:40:09.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:09.595+0000 7f82eaa00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f82ec103e30 0x7f82ec19a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:40:09.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:09.595+0000 7f82eb400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f82ec108f60 0x7f82ec19ed40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51348/0 (socket says 172.21.15.5:51348) 2024-09-10T09:40:09.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:09.595+0000 7f82e3e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f82ec1047e0 0x7f82ec19a9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:35848/0 (socket says 172.21.15.5:35848) 2024-09-10T09:40:09.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:09.595+0000 7f82eb400640 1 -- 172.21.15.5:0/777609855 learned_addr learned my addr 172.21.15.5:0/777609855 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:40:09.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:09.596+0000 7f82e3e00640 1 -- 172.21.15.5:0/777609855 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f82ec103e30 msgr2=0x7f82ec19a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:40:09.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:09.596+0000 7f82e3e00640 1 --2- 172.21.15.5:0/777609855 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f82ec103e30 0x7f82ec19a470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:09.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:09.596+0000 7f82e3e00640 1 -- 172.21.15.5:0/777609855 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f82ec108f60 msgr2=0x7f82ec19ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:40:09.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:09.596+0000 7f82e3e00640 1 --2- 172.21.15.5:0/777609855 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f82ec108f60 0x7f82ec19ed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:09.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:09.596+0000 7f82e3e00640 1 -- 172.21.15.5:0/777609855 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82ec19f420 con 0x7f82ec1047e0 2024-09-10T09:40:09.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:09.596+0000 7f82eaa00640 1 --2- 172.21.15.5:0/777609855 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f82ec103e30 0x7f82ec19a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:40:09.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:09.596+0000 7f82eb400640 1 --2- 172.21.15.5:0/777609855 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f82ec108f60 0x7f82ec19ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:40:09.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:09.596+0000 7f82e3e00640 1 --2- 172.21.15.5:0/777609855 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f82ec1047e0 0x7f82ec19a9b0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f82d402a910 tx=0x7f82d4003b20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:40:09.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:09.596+0000 7f82e3400640 1 -- 172.21.15.5:0/777609855 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82d403f070 con 0x7f82ec1047e0 2024-09-10T09:40:09.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:09.596+0000 7f82e3400640 1 -- 172.21.15.5:0/777609855 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f82d4031c90 con 0x7f82ec1047e0 2024-09-10T09:40:09.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:09.596+0000 7f82ebe00640 1 -- 172.21.15.5:0/777609855 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82ec19f6b0 con 0x7f82ec1047e0 2024-09-10T09:40:09.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:09.596+0000 7f82e3400640 1 -- 172.21.15.5:0/777609855 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82d4035030 con 0x7f82ec1047e0 2024-09-10T09:40:09.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:09.597+0000 7f82ebe00640 1 -- 172.21.15.5:0/777609855 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82ec1a6ff0 con 0x7f82ec1047e0 2024-09-10T09:40:09.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:09.600+0000 7f82f42b3740 1 Processor -- start 2024-09-10T09:40:09.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:09.601+0000 7f82f42b3740 1 -- start start 2024-09-10T09:40:09.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:09.601+0000 7f82f42b3740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5650ca07ed50 0x5650ca07f130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:40:09.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:09.601+0000 7f82f42b3740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5650ca07f670 0x5650ca0899f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:40:09.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:09.601+0000 7f82f42b3740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5650ca089f30 0x5650ca08c320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:40:09.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:09.601+0000 7f82f42b3740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5650c9fe53e0 con 0x5650ca07f670 2024-09-10T09:40:09.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:09.601+0000 7f82f42b3740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x5650c9f8cd80 con 0x5650ca07ed50 2024-09-10T09:40:09.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:09.601+0000 7f82f42b3740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5650c9fe13a0 con 0x5650ca089f30 2024-09-10T09:40:09.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:09.601+0000 7f82e3400640 1 -- 172.21.15.5:0/777609855 <== mon.2 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f82d4042070 con 0x7f82ec1047e0 2024-09-10T09:40:09.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:09.601+0000 7f82eaa00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5650ca07f670 0x5650ca0899f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:40:09.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:09.601+0000 7f82eb400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5650ca07ed50 0x5650ca07f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:40:09.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:09.601+0000 7f82eb400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5650ca07ed50 0x5650ca07f130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:35854/0 (socket says 172.21.15.5:35854) 2024-09-10T09:40:09.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:09.601+0000 7f82eaa00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5650ca07f670 0x5650ca0899f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51352/0 (socket says 172.21.15.5:51352) 2024-09-10T09:40:09.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:09.601+0000 7f82eb400640 1 -- 172.21.15.5:0/3997486231 learned_addr learned my addr 172.21.15.5:0/3997486231 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:40:09.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:09.602+0000 7f82e3e00640 1 --2- 172.21.15.5:0/3997486231 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5650ca089f30 0x5650ca08c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:40:09.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:09.602+0000 7f82eb400640 1 -- 172.21.15.5:0/3997486231 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5650ca089f30 msgr2=0x5650ca08c320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:40:09.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:09.602+0000 7f82eb400640 1 --2- 172.21.15.5:0/3997486231 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5650ca089f30 0x5650ca08c320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:09.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:09.602+0000 7f82eb400640 1 -- 172.21.15.5:0/3997486231 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5650ca07f670 msgr2=0x5650ca0899f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:40:09.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:09.602+0000 7f82eb400640 1 --2- 172.21.15.5:0/3997486231 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5650ca07f670 0x5650ca0899f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:09.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:09.602+0000 7f82eb400640 1 -- 172.21.15.5:0/3997486231 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5650c9fabae0 con 0x5650ca07ed50 2024-09-10T09:40:09.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:09.602+0000 7f82e3e00640 1 --2- 172.21.15.5:0/3997486231 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5650ca089f30 0x5650ca08c320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:40:09.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:09.602+0000 7f82eb400640 1 --2- 172.21.15.5:0/3997486231 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5650ca07ed50 0x5650ca07f130 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f82e4011850 tx=0x7f82e4011d10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3d3a7b65200d5df2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:40:09.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:09.603+0000 7f82dbe00640 1 -- 172.21.15.5:0/3997486231 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82e4011330 con 0x5650ca07ed50 2024-09-10T09:40:09.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:09.603+0000 7f82dbe00640 1 -- 172.21.15.5:0/3997486231 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f82e400fe80 con 0x5650ca07ed50 2024-09-10T09:40:09.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:09.603+0000 7f82dbe00640 1 -- 172.21.15.5:0/3997486231 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82e40193e0 con 0x5650ca07ed50 2024-09-10T09:40:09.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:09.603+0000 7f82e3400640 1 --2- 172.21.15.5:0/777609855 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f82d00a2c00 0x7f82d00a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:40:09.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:09.603+0000 7f82e3400640 1 -- 172.21.15.5:0/777609855 <== mon.2 v2:172.21.15.102:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f82d40efc90 con 0x7f82ec1047e0 2024-09-10T09:40:09.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:09.603+0000 7f82eaa00640 1 --2- 172.21.15.5:0/777609855 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f82d00a2c00 0x7f82d00a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:40:09.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:09.603+0000 7f82f42b3740 1 -- 172.21.15.5:0/3997486231 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5650ca07ed50 msgr2=0x5650ca07f130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:40:09.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:09.603+0000 7f82f42b3740 1 --2- 172.21.15.5:0/3997486231 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5650ca07ed50 0x5650ca07f130 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f82e4011850 tx=0x7f82e4011d10 comp rx=0 tx=0).stop 2024-09-10T09:40:09.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:09.604+0000 7f82f42b3740 1 -- 172.21.15.5:0/3997486231 shutdown_connections 2024-09-10T09:40:09.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:09.604+0000 7f82f42b3740 1 --2- 172.21.15.5:0/3997486231 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5650ca089f30 0x5650ca08c320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:09.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:09.604+0000 7f82f42b3740 1 --2- 172.21.15.5:0/3997486231 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5650ca07f670 0x5650ca0899f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:09.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:09.604+0000 7f82f42b3740 1 --2- 172.21.15.5:0/3997486231 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5650ca07ed50 0x5650ca07f130 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:09.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:09.604+0000 7f82f42b3740 1 -- 172.21.15.5:0/3997486231 >> 172.21.15.5:0/3997486231 conn(0x5650ca07b2e0 msgr2=0x5650ca07b6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:40:09.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:09.604+0000 7f82f42b3740 1 -- 172.21.15.5:0/3997486231 shutdown_connections 2024-09-10T09:40:09.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:09.604+0000 7f82f42b3740 1 -- 172.21.15.5:0/3997486231 wait complete. 2024-09-10T09:40:09.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:09.604+0000 7f82eaa00640 1 --2- 172.21.15.5:0/777609855 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f82d00a2c00 0x7f82d00a50c0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x5650ca07c140 tx=0x7f82dc005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:40:09.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:09.604+0000 7f82f42b3740 1 -- start start 2024-09-10T09:40:09.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:09.605+0000 7f82f42b3740 1 Processor -- start 2024-09-10T09:40:09.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:09.605+0000 7f82f42b3740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5650ca07ed50 0x5650ca086d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:40:09.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:09.605+0000 7f82f42b3740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5650ca07f670 0x5650ca087250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:40:09.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:09.605+0000 7f82f42b3740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5650ca089f30 0x5650ca087790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:40:09.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:09.605+0000 7f82f42b3740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5650ca08cba0 con 0x5650ca089f30 2024-09-10T09:40:09.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:09.605+0000 7f82f42b3740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x5650c9fa00f0 con 0x5650ca07f670 2024-09-10T09:40:09.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:09.605+0000 7f82f42b3740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5650ca08cd20 con 0x5650ca07ed50 2024-09-10T09:40:09.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:09.605+0000 7f82f42b3740 10 client.0 _ll_register_callbacks cb 0x5650c9f919e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:40:09.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:09.605+0000 7f82eb400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5650ca089f30 0x5650ca087790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:40:09.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:09.605+0000 7f82eb400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5650ca089f30 0x5650ca087790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51368/0 (socket says 172.21.15.5:51368) 2024-09-10T09:40:09.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:09.605+0000 7f82eb400640 1 -- 172.21.15.5:0/394558682 learned_addr learned my addr 172.21.15.5:0/394558682 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:40:09.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:09.605+0000 7f82eaa00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5650ca07ed50 0x5650ca086d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:40:09.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:09.605+0000 7f82e3e00640 1 --2- 172.21.15.5:0/394558682 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5650ca07f670 0x5650ca087250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:40:09.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:09.606+0000 7f82eb400640 1 -- 172.21.15.5:0/394558682 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5650ca07ed50 msgr2=0x5650ca086d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:40:09.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:09.606+0000 7f82eb400640 1 --2- 172.21.15.5:0/394558682 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5650ca07ed50 0x5650ca086d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:09.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:09.606+0000 7f82eb400640 1 -- 172.21.15.5:0/394558682 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5650ca07f670 msgr2=0x5650ca087250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:40:09.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:09.606+0000 7f82eb400640 1 --2- 172.21.15.5:0/394558682 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5650ca07f670 0x5650ca087250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:09.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:09.606+0000 7f82eb400640 1 -- 172.21.15.5:0/394558682 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5650c9d563c0 con 0x5650ca089f30 2024-09-10T09:40:09.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:09.606+0000 7f82eb400640 1 --2- 172.21.15.5:0/394558682 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5650ca089f30 0x5650ca087790 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f82e4009a70 tx=0x7f82e400f9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c99e647e5538e5b8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:40:09.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:09.606+0000 7f82d8c00640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-10T09:40:09.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:09.607+0000 7f82d8c00640 1 -- 172.21.15.5:0/394558682 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82e401f040 con 0x5650ca089f30 2024-09-10T09:40:09.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:09.607+0000 7f82d8c00640 1 -- 172.21.15.5:0/394558682 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f82e4004a40 con 0x5650ca089f30 2024-09-10T09:40:09.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:09.607+0000 7f82f42b3740 1 -- 172.21.15.5:0/394558682 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5650c9ff2950 con 0x5650ca089f30 2024-09-10T09:40:09.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:09.607+0000 7f82d8c00640 1 -- 172.21.15.5:0/394558682 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f82e4032050 con 0x5650ca089f30 2024-09-10T09:40:09.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:09.607+0000 7f82f42b3740 10 client.15786 fetch_fsmap learned FSMap version 30 2024-09-10T09:40:09.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:09.607+0000 7f82f42b3740 1 -- 172.21.15.5:0/394558682 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5650ca098750 con 0x5650ca089f30 2024-09-10T09:40:09.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:09.607+0000 7f82d8c00640 1 -- 172.21.15.5:0/394558682 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x5650c9ff2950 con 0x5650ca089f30 2024-09-10T09:40:09.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:09.608+0000 7f82d8c00640 1 -- 172.21.15.5:0/394558682 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f82e4018070 con 0x5650ca089f30 2024-09-10T09:40:09.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:09.608+0000 7f82f42b3740 10 client.15786 fetch_fsmap finished waiting for FSMap version 30 2024-09-10T09:40:09.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:09.608+0000 7f82f42b3740 10 client.15786 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T09:40:09.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:09.608+0000 7f82f42b3740 20 client.15786 populate_metadata read hostname 'smithi005' 2024-09-10T09:40:09.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:09.608+0000 7f82f42b3740 1 --2- 172.21.15.5:0/394558682 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x5650ca099bb0 0x5650ca09bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:40:09.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:09.608+0000 7f82f42b3740 4 client.15786 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:40:09.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:09.608+0000 7f82f42b3740 1 -- 172.21.15.5:0/394558682 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5650c9fece60 con 0x5650ca099bb0 2024-09-10T09:40:09.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:09.608+0000 7f82eaa00640 1 --2- 172.21.15.5:0/394558682 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x5650ca099bb0 0x5650ca09bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:40:09.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:09.609+0000 7f82eaa00640 1 --2- 172.21.15.5:0/394558682 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x5650ca099bb0 0x5650ca09bfa0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cbb6f4a98218d6f2 server_cookie=2e5697f134c8f17e in_seq=0 out_seq=0 2024-09-10T09:40:09.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:09.609+0000 7f82d8c00640 10 client.15786 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T09:40:09.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:09.610+0000 7f82d8c00640 1 -- 172.21.15.5:0/394558682 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5650c9fece60 con 0x5650ca099bb0 2024-09-10T09:40:09.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:09.610+0000 7f82d8c00640 10 client.15786 handle_command_reply: tid=0 2024-09-10T09:40:09.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:09.610+0000 7f82f42b3740 1 client.15786 shutdown 2024-09-10T09:40:09.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:09.611+0000 7f82f42b3740 1 -- 172.21.15.5:0/394558682 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5650ca089f30 msgr2=0x5650ca087790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:40:09.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:09.611+0000 7f82f42b3740 1 --2- 172.21.15.5:0/394558682 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5650ca089f30 0x5650ca087790 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f82e4009a70 tx=0x7f82e400f9a0 comp rx=0 tx=0).stop 2024-09-10T09:40:09.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:09.611+0000 7f82f42b3740 1 -- 172.21.15.5:0/394558682 shutdown_connections 2024-09-10T09:40:09.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:09.611+0000 7f82f42b3740 1 --2- 172.21.15.5:0/394558682 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x5650ca099bb0 0x5650ca09bfa0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:09.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:09.611+0000 7f82f42b3740 1 --2- 172.21.15.5:0/394558682 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5650ca089f30 0x5650ca087790 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:09.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:09.611+0000 7f82f42b3740 1 --2- 172.21.15.5:0/394558682 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5650ca07f670 0x5650ca087250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:09.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:09.611+0000 7f82f42b3740 1 --2- 172.21.15.5:0/394558682 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5650ca07ed50 0x5650ca086d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:09.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:09.611+0000 7f82f42b3740 1 -- 172.21.15.5:0/394558682 >> 172.21.15.5:0/394558682 conn(0x5650ca07b2e0 msgr2=0x5650ca081010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:40:09.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:09.611+0000 7f82d8c00640 0 client.15786 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T09:40:09.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:09.611+0000 7f82f42b3740 1 -- 172.21.15.5:0/394558682 shutdown_connections 2024-09-10T09:40:09.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:09.611+0000 7f82f42b3740 1 -- 172.21.15.5:0/394558682 wait complete. 2024-09-10T09:40:09.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:09.611+0000 7f82f42b3740 20 client.15786 trim_cache size 0 max 16384 2024-09-10T09:40:09.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:09.629+0000 7f82f42b3740 1 Processor -- start 2024-09-10T09:40:09.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:09.629+0000 7f82f42b3740 1 -- start start 2024-09-10T09:40:09.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:09.629+0000 7f82f42b3740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5650ca07ed50 0x5650ca0a1780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:40:09.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:09.629+0000 7f82f42b3740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5650ca07f670 0x5650ca0a1cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:40:09.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:09.629+0000 7f82f42b3740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5650ca089f30 0x5650ca079b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:40:09.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:09.629+0000 7f82f42b3740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f82ec1b0ef0 con 0x5650ca089f30 2024-09-10T09:40:09.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:09.629+0000 7f82f42b3740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f82ec1b2a30 con 0x5650ca07f670 2024-09-10T09:40:09.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:09.629+0000 7f82f42b3740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f82ec1b15c0 con 0x5650ca07ed50 2024-09-10T09:40:09.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:09.630+0000 7f82eb400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5650ca089f30 0x5650ca079b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:40:09.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:09.630+0000 7f82eaa00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5650ca07ed50 0x5650ca0a1780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:40:09.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:09.630+0000 7f82eaa00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5650ca07ed50 0x5650ca0a1780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:36626/0 (socket says 172.21.15.5:36626) 2024-09-10T09:40:09.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:09.630+0000 7f82eaa00640 1 -- 172.21.15.5:0/557123930 learned_addr learned my addr 172.21.15.5:0/557123930 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:40:09.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:09.630+0000 7f82e3e00640 1 --2- 172.21.15.5:0/557123930 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5650ca07f670 0x5650ca0a1cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:40:09.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:09.630+0000 7f82eaa00640 1 -- 172.21.15.5:0/557123930 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5650ca07f670 msgr2=0x5650ca0a1cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:40:09.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:09.630+0000 7f82eaa00640 1 --2- 172.21.15.5:0/557123930 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5650ca07f670 0x5650ca0a1cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:09.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:09.630+0000 7f82eaa00640 1 -- 172.21.15.5:0/557123930 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5650ca089f30 msgr2=0x5650ca079b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:40:09.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:09.630+0000 7f82eaa00640 1 --2- 172.21.15.5:0/557123930 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5650ca089f30 0x5650ca079b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:09.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:09.630+0000 7f82eaa00640 1 -- 172.21.15.5:0/557123930 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5650ca07b110 con 0x5650ca07ed50 2024-09-10T09:40:09.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:09.630+0000 7f82e3e00640 1 --2- 172.21.15.5:0/557123930 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5650ca07f670 0x5650ca0a1cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:40:09.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:09.630+0000 7f82eb400640 1 --2- 172.21.15.5:0/557123930 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5650ca089f30 0x5650ca079b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:40:09.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:09.630+0000 7f82eaa00640 1 --2- 172.21.15.5:0/557123930 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5650ca07ed50 0x5650ca0a1780 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f82dc00f8c0 tx=0x7f82dc01b000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=71fdaf9c443395e9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:40:09.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:09.631+0000 7f82db400640 1 -- 172.21.15.5:0/557123930 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82dc01f930 con 0x5650ca07ed50 2024-09-10T09:40:09.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:09.631+0000 7f82db400640 1 -- 172.21.15.5:0/557123930 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f82dc02b3b0 con 0x5650ca07ed50 2024-09-10T09:40:09.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:09.631+0000 7f82db400640 1 -- 172.21.15.5:0/557123930 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82dc02a6c0 con 0x5650ca07ed50 2024-09-10T09:40:09.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:09.631+0000 7f82f42b3740 1 -- 172.21.15.5:0/557123930 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5650ca07ed50 msgr2=0x5650ca0a1780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:40:09.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:09.631+0000 7f82f42b3740 1 --2- 172.21.15.5:0/557123930 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5650ca07ed50 0x5650ca0a1780 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f82dc00f8c0 tx=0x7f82dc01b000 comp rx=0 tx=0).stop 2024-09-10T09:40:09.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:09.631+0000 7f82f42b3740 1 -- 172.21.15.5:0/557123930 shutdown_connections 2024-09-10T09:40:09.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:09.631+0000 7f82f42b3740 1 --2- 172.21.15.5:0/557123930 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5650ca089f30 0x5650ca079b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:09.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:09.631+0000 7f82f42b3740 1 --2- 172.21.15.5:0/557123930 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5650ca07f670 0x5650ca0a1cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:09.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:09.631+0000 7f82f42b3740 1 --2- 172.21.15.5:0/557123930 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5650ca07ed50 0x5650ca0a1780 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:09.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:09.631+0000 7f82f42b3740 1 -- 172.21.15.5:0/557123930 >> 172.21.15.5:0/557123930 conn(0x5650ca07b2e0 msgr2=0x5650ca088d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:40:09.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:09.632+0000 7f82f42b3740 1 -- 172.21.15.5:0/557123930 shutdown_connections 2024-09-10T09:40:09.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:09.632+0000 7f82f42b3740 1 -- 172.21.15.5:0/557123930 wait complete. 2024-09-10T09:40:09.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:09.632+0000 7f82f42b3740 1 -- start start 2024-09-10T09:40:09.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:09.632+0000 7f82f42b3740 1 Processor -- start 2024-09-10T09:40:09.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:09.633+0000 7f82f42b3740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5650ca07ed50 0x5650ca082fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:40:09.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:09.633+0000 7f82f42b3740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5650ca07f670 0x5650ca0834e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:40:09.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:09.633+0000 7f82f42b3740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5650ca089f30 0x5650ca083a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:40:09.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:09.633+0000 7f82f42b3740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f82ec1b2360 con 0x5650ca089f30 2024-09-10T09:40:09.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:09.633+0000 7f82f42b3740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f82ec1b1c90 con 0x5650ca07f670 2024-09-10T09:40:09.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:09.633+0000 7f82f42b3740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f82ec1aeed0 con 0x5650ca07ed50 2024-09-10T09:40:09.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:09.633+0000 7f82f42b3740 10 client.0 _ll_register_callbacks cb 0x7f82e4011720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:40:09.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:09.633+0000 7f82eb400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5650ca089f30 0x5650ca083a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:40:09.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:09.633+0000 7f82eb400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5650ca089f30 0x5650ca083a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51392/0 (socket says 172.21.15.5:51392) 2024-09-10T09:40:09.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:09.633+0000 7f82e3e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5650ca07f670 0x5650ca0834e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:40:09.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:09.633+0000 7f82eb400640 1 -- 172.21.15.5:0/1180135704 learned_addr learned my addr 172.21.15.5:0/1180135704 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:40:09.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:09.633+0000 7f82eaa00640 1 --2- 172.21.15.5:0/1180135704 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5650ca07ed50 0x5650ca082fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:40:09.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:09.633+0000 7f82eb400640 1 -- 172.21.15.5:0/1180135704 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5650ca07ed50 msgr2=0x5650ca082fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:40:09.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:09.633+0000 7f82eb400640 1 --2- 172.21.15.5:0/1180135704 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5650ca07ed50 0x5650ca082fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:09.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:09.633+0000 7f82eb400640 1 -- 172.21.15.5:0/1180135704 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5650ca07f670 msgr2=0x5650ca0834e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:40:09.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:09.633+0000 7f82eb400640 1 --2- 172.21.15.5:0/1180135704 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5650ca07f670 0x5650ca0834e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:09.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:09.633+0000 7f82eb400640 1 -- 172.21.15.5:0/1180135704 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5650ca0910a0 con 0x5650ca089f30 2024-09-10T09:40:09.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:09.633+0000 7f82e3e00640 1 --2- 172.21.15.5:0/1180135704 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5650ca07f670 0x5650ca0834e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:40:09.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:09.634+0000 7f82eb400640 1 --2- 172.21.15.5:0/1180135704 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5650ca089f30 0x5650ca083a20 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f82e4000f80 tx=0x7f82e4010ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c36d3f41650754e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:40:09.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:09.634+0000 7f82d8c00640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-10T09:40:09.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:09.634+0000 7f82d8c00640 1 -- 172.21.15.5:0/1180135704 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82e40059a0 con 0x5650ca089f30 2024-09-10T09:40:09.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:09.634+0000 7f82d8c00640 1 -- 172.21.15.5:0/1180135704 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f82e4010430 con 0x5650ca089f30 2024-09-10T09:40:09.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:09.634+0000 7f82f42b3740 1 -- 172.21.15.5:0/1180135704 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f82dc02b3b0 con 0x5650ca089f30 2024-09-10T09:40:09.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:09.634+0000 7f82d8c00640 1 -- 172.21.15.5:0/1180135704 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f82e4010630 con 0x5650ca089f30 2024-09-10T09:40:09.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:09.634+0000 7f82f42b3740 10 client.15792 fetch_fsmap learned FSMap version 30 2024-09-10T09:40:09.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:09.634+0000 7f82f42b3740 1 -- 172.21.15.5:0/1180135704 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5650ca092350 con 0x5650ca089f30 2024-09-10T09:40:09.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:09.635+0000 7f82d8c00640 1 -- 172.21.15.5:0/1180135704 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f82dc02b3b0 con 0x5650ca089f30 2024-09-10T09:40:09.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:09.635+0000 7f82d8c00640 1 -- 172.21.15.5:0/1180135704 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f82e4018070 con 0x5650ca089f30 2024-09-10T09:40:09.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:09.635+0000 7f82f42b3740 10 client.15792 fetch_fsmap finished waiting for FSMap version 30 2024-09-10T09:40:09.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:09.635+0000 7f82f42b3740 10 client.15792 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T09:40:09.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:09.635+0000 7f82f42b3740 20 client.15792 populate_metadata read hostname 'smithi005' 2024-09-10T09:40:09.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:09.635+0000 7f82f42b3740 1 --2- 172.21.15.5:0/1180135704 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x5650ca104750 0x5650ca106b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:40:09.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:09.635+0000 7f82f42b3740 4 client.15792 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T09:40:09.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:09.635+0000 7f82f42b3740 1 -- 172.21.15.5:0/1180135704 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5650c9f690c0 con 0x5650ca104750 2024-09-10T09:40:09.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:09.635+0000 7f82eaa00640 1 --2- 172.21.15.5:0/1180135704 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x5650ca104750 0x5650ca106b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:40:09.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:09.636+0000 7f82eaa00640 1 --2- 172.21.15.5:0/1180135704 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x5650ca104750 0x5650ca106b40 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=706e3359e423a196 server_cookie=f67ed4702762c900 in_seq=0 out_seq=0 2024-09-10T09:40:09.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:09.636+0000 7f82d8c00640 10 client.15792 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T09:40:09.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:09.636+0000 7f82d8c00640 1 -- 172.21.15.5:0/1180135704 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5650c9f690c0 con 0x5650ca104750 2024-09-10T09:40:09.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:09.636+0000 7f82d8c00640 10 client.15792 handle_command_reply: tid=0 2024-09-10T09:40:09.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:09.636+0000 7f82f42b3740 1 client.15792 shutdown 2024-09-10T09:40:09.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:09.636+0000 7f82f42b3740 1 -- 172.21.15.5:0/1180135704 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5650ca089f30 msgr2=0x5650ca083a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:40:09.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:09.636+0000 7f82f42b3740 1 --2- 172.21.15.5:0/1180135704 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5650ca089f30 0x5650ca083a20 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f82e4000f80 tx=0x7f82e4010ac0 comp rx=0 tx=0).stop 2024-09-10T09:40:09.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:09.636+0000 7f82f42b3740 1 -- 172.21.15.5:0/1180135704 shutdown_connections 2024-09-10T09:40:09.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:09.636+0000 7f82f42b3740 1 --2- 172.21.15.5:0/1180135704 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x5650ca104750 0x5650ca106b40 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:09.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:09.636+0000 7f82f42b3740 1 --2- 172.21.15.5:0/1180135704 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5650ca089f30 0x5650ca083a20 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:09.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:09.636+0000 7f82f42b3740 1 --2- 172.21.15.5:0/1180135704 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5650ca07f670 0x5650ca0834e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:09.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:09.636+0000 7f82f42b3740 1 --2- 172.21.15.5:0/1180135704 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5650ca07ed50 0x5650ca082fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:09.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:09.636+0000 7f82f42b3740 1 -- 172.21.15.5:0/1180135704 >> 172.21.15.5:0/1180135704 conn(0x5650ca07b2e0 msgr2=0x5650ca0854d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:40:09.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:09.636+0000 7f82d8c00640 0 client.15792 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T09:40:09.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:09.637+0000 7f82f42b3740 1 -- 172.21.15.5:0/1180135704 shutdown_connections 2024-09-10T09:40:09.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:09.637+0000 7f82f42b3740 1 -- 172.21.15.5:0/1180135704 wait complete. 2024-09-10T09:40:09.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:09.637+0000 7f82f42b3740 20 client.15792 trim_cache size 0 max 16384 2024-09-10T09:40:09.641 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-10T09:40:09.641 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-10T09:40:09.641 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2024-09-10T09:40:09.641 INFO:teuthology.orchestra.run.smithi005.stdout: "ce6f4f5d-2b80-422e-9fa0-6ff19dfb9c56": { 2024-09-10T09:40:09.641 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2024-09-10T09:40:09.642 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "ce6f4f5d-2b80-422e-9fa0-6ff19dfb9c56", 2024-09-10T09:40:09.642 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2024-09-10T09:40:09.642 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T09:40:09.642 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T09:40:09.642 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-10T09:40:09.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:09.637+0000 7f82ebe00640 1 -- 172.21.15.5:0/777609855 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f82d00a2c00 msgr2=0x7f82d00a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:40:09.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:09.637+0000 7f82ebe00640 1 --2- 172.21.15.5:0/777609855 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f82d00a2c00 0x7f82d00a50c0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x5650ca07c140 tx=0x7f82dc005f90 comp rx=0 tx=0).stop 2024-09-10T09:40:09.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:09.637+0000 7f82ebe00640 1 -- 172.21.15.5:0/777609855 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f82ec1047e0 msgr2=0x7f82ec19a9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:40:09.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:09.637+0000 7f82ebe00640 1 --2- 172.21.15.5:0/777609855 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f82ec1047e0 0x7f82ec19a9b0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f82d402a910 tx=0x7f82d4003b20 comp rx=0 tx=0).stop 2024-09-10T09:40:09.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:09.637+0000 7f82ebe00640 1 -- 172.21.15.5:0/777609855 shutdown_connections 2024-09-10T09:40:09.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:09.637+0000 7f82ebe00640 1 --2- 172.21.15.5:0/777609855 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f82d00a2c00 0x7f82d00a50c0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:09.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:09.637+0000 7f82ebe00640 1 --2- 172.21.15.5:0/777609855 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f82ec108f60 0x7f82ec19ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:09.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:09.637+0000 7f82ebe00640 1 --2- 172.21.15.5:0/777609855 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f82ec1047e0 0x7f82ec19a9b0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:09.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:09.637+0000 7f82ebe00640 1 --2- 172.21.15.5:0/777609855 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f82ec103e30 0x7f82ec19a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:09.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:09.637+0000 7f82ebe00640 1 -- 172.21.15.5:0/777609855 >> 172.21.15.5:0/777609855 conn(0x7f82ec0ff8d0 msgr2=0x7f82ec100b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:40:09.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:09.638+0000 7f82ebe00640 1 -- 172.21.15.5:0/777609855 shutdown_connections 2024-09-10T09:40:09.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:09.638+0000 7f82ebe00640 1 -- 172.21.15.5:0/777609855 wait complete. 2024-09-10T09:40:10.576 INFO:tasks.cephfs.filesystem:scrub status for tag:ce6f4f5d-2b80-422e-9fa0-6ff19dfb9c56 - {'path': '/', 'tag': 'ce6f4f5d-2b80-422e-9fa0-6ff19dfb9c56', 'options': 'recursive,force'} 2024-09-10T09:40:11.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:40:10 smithi005 ceph-mon[30350]: pgmap v448: 129 pgs: 129 active+clean; 666 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-10T09:40:11.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:40:10 smithi159 ceph-mon[35513]: pgmap v448: 129 pgs: 129 active+clean; 666 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-10T09:40:11.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:40:10 smithi102 ceph-mon[35442]: pgmap v448: 129 pgs: 129 active+clean; 666 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-10T09:40:13.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:40:12 smithi005 ceph-mon[30350]: pgmap v449: 129 pgs: 129 active+clean; 666 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 11 op/s 2024-09-10T09:40:13.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:40:12 smithi005 ceph-mon[30350]: scrub summary: idle 2024-09-10T09:40:13.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:40:12 smithi159 ceph-mon[35513]: pgmap v449: 129 pgs: 129 active+clean; 666 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 11 op/s 2024-09-10T09:40:13.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:40:12 smithi159 ceph-mon[35513]: scrub summary: idle 2024-09-10T09:40:13.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:40:12 smithi102 ceph-mon[35442]: pgmap v449: 129 pgs: 129 active+clean; 666 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 11 op/s 2024-09-10T09:40:13.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:40:12 smithi102 ceph-mon[35442]: scrub summary: idle 2024-09-10T09:40:15.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:40:14 smithi005 ceph-mon[30350]: pgmap v450: 129 pgs: 129 active+clean; 666 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-09-10T09:40:15.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:40:14 smithi159 ceph-mon[35513]: pgmap v450: 129 pgs: 129 active+clean; 666 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-09-10T09:40:15.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:40:14 smithi102 ceph-mon[35442]: pgmap v450: 129 pgs: 129 active+clean; 666 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-09-10T09:40:15.638 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T09:40:16.035 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:40:16.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:16.605+0000 7f3d70c00640 1 -- 172.21.15.5:0/2262489415 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3d6c109140 msgr2=0x7f3d6c110cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:40:16.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:16.605+0000 7f3d70c00640 1 --2- 172.21.15.5:0/2262489415 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3d6c109140 0x7f3d6c110cc0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f3d64009900 tx=0x7f3d6402a410 comp rx=0 tx=0).stop 2024-09-10T09:40:16.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:16.605+0000 7f3d70c00640 1 -- 172.21.15.5:0/2262489415 shutdown_connections 2024-09-10T09:40:16.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:16.605+0000 7f3d70c00640 1 --2- 172.21.15.5:0/2262489415 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3d6c109140 0x7f3d6c110cc0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:16.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:16.605+0000 7f3d70c00640 1 --2- 172.21.15.5:0/2262489415 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3d6c1049c0 0x7f3d6c108a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:16.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:16.605+0000 7f3d70c00640 1 --2- 172.21.15.5:0/2262489415 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3d6c104010 0x7f3d6c1043f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:16.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:16.605+0000 7f3d70c00640 1 -- 172.21.15.5:0/2262489415 >> 172.21.15.5:0/2262489415 conn(0x7f3d6c0ffa50 msgr2=0x7f3d6c101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:40:16.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:16.605+0000 7f3d70c00640 1 -- 172.21.15.5:0/2262489415 shutdown_connections 2024-09-10T09:40:16.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:16.606+0000 7f3d70c00640 1 -- 172.21.15.5:0/2262489415 wait complete. 2024-09-10T09:40:16.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:16.607+0000 7f3d70c00640 1 Processor -- start 2024-09-10T09:40:16.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:16.607+0000 7f3d70c00640 1 -- start start 2024-09-10T09:40:16.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:16.607+0000 7f3d70c00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3d6c104010 0x7f3d6c0739d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:40:16.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:16.607+0000 7f3d70c00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3d6c1049c0 0x7f3d6c06ca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:40:16.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:16.607+0000 7f3d70c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3d6c109140 0x7f3d6c06cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:40:16.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:16.607+0000 7f3d70c00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f3d6c1141c0 con 0x7f3d6c104010 2024-09-10T09:40:16.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:16.607+0000 7f3d70c00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f3d6c114040 con 0x7f3d6c109140 2024-09-10T09:40:16.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:16.607+0000 7f3d70c00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f3d6c114340 con 0x7f3d6c1049c0 2024-09-10T09:40:16.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:16.608+0000 7f3d69e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3d6c104010 0x7f3d6c0739d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:40:16.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:16.608+0000 7f3d69e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3d6c104010 0x7f3d6c0739d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41722/0 (socket says 172.21.15.5:41722) 2024-09-10T09:40:16.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:16.608+0000 7f3d69e00640 1 -- 172.21.15.5:0/1825995284 learned_addr learned my addr 172.21.15.5:0/1825995284 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:40:16.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:16.608+0000 7f3d69400640 1 --2- 172.21.15.5:0/1825995284 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3d6c1049c0 0x7f3d6c06ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:40:16.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:16.608+0000 7f3d6a800640 1 --2- 172.21.15.5:0/1825995284 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3d6c109140 0x7f3d6c06cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:40:16.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:16.608+0000 7f3d69e00640 1 -- 172.21.15.5:0/1825995284 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3d6c1049c0 msgr2=0x7f3d6c06ca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:40:16.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:16.608+0000 7f3d69e00640 1 --2- 172.21.15.5:0/1825995284 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3d6c1049c0 0x7f3d6c06ca60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:16.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:16.608+0000 7f3d69e00640 1 -- 172.21.15.5:0/1825995284 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3d6c109140 msgr2=0x7f3d6c06cfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:40:16.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:16.608+0000 7f3d69e00640 1 --2- 172.21.15.5:0/1825995284 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3d6c109140 0x7f3d6c06cfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:16.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:16.608+0000 7f3d69e00640 1 -- 172.21.15.5:0/1825995284 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d6c06d530 con 0x7f3d6c104010 2024-09-10T09:40:16.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:16.608+0000 7f3d6a800640 1 --2- 172.21.15.5:0/1825995284 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3d6c109140 0x7f3d6c06cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:40:16.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:16.608+0000 7f3d69e00640 1 --2- 172.21.15.5:0/1825995284 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3d6c104010 0x7f3d6c0739d0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f3d5c00ebc0 tx=0x7f3d5c00c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:40:16.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:16.609+0000 7f3d62a00640 1 -- 172.21.15.5:0/1825995284 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d5c010630 con 0x7f3d6c104010 2024-09-10T09:40:16.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:16.609+0000 7f3d70c00640 1 -- 172.21.15.5:0/1825995284 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d6c06d820 con 0x7f3d6c104010 2024-09-10T09:40:16.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:16.609+0000 7f3d62a00640 1 -- 172.21.15.5:0/1825995284 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3d5c010cb0 con 0x7f3d6c104010 2024-09-10T09:40:16.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:16.609+0000 7f3d62a00640 1 -- 172.21.15.5:0/1825995284 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d5c018bd0 con 0x7f3d6c104010 2024-09-10T09:40:16.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:16.609+0000 7f3d70c00640 1 -- 172.21.15.5:0/1825995284 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d6c071810 con 0x7f3d6c104010 2024-09-10T09:40:16.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:16.610+0000 7f3d70c00640 1 -- 172.21.15.5:0/1825995284 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3d6c1054f0 con 0x7f3d6c104010 2024-09-10T09:40:16.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:16.612+0000 7f3d62a00640 1 -- 172.21.15.5:0/1825995284 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f3d5c0107d0 con 0x7f3d6c104010 2024-09-10T09:40:16.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:16.613+0000 7f3d62a00640 1 --2- 172.21.15.5:0/1825995284 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f3d500a2b80 0x7f3d500a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:40:16.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:16.613+0000 7f3d62a00640 1 -- 172.21.15.5:0/1825995284 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f3d5c0cfb60 con 0x7f3d6c104010 2024-09-10T09:40:16.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:16.613+0000 7f3d69400640 1 --2- 172.21.15.5:0/1825995284 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f3d500a2b80 0x7f3d500a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:40:16.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:16.614+0000 7f3d69400640 1 --2- 172.21.15.5:0/1825995284 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f3d500a2b80 0x7f3d500a5040 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f3d6c06e220 tx=0x7f3d58005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:40:16.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:16.617+0000 7f3d62a00640 1 -- 172.21.15.5:0/1825995284 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f3d5c09c670 con 0x7f3d6c104010 2024-09-10T09:40:16.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:40:16 smithi005 ceph-mon[30350]: pgmap v451: 129 pgs: 129 active+clean; 666 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-09-10T09:40:16.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:16.842+0000 7f3d70c00640 1 -- 172.21.15.5:0/1825995284 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3d6c0722d0 con 0x7f3d6c104010 2024-09-10T09:40:16.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:16.843+0000 7f3d62a00640 1 -- 172.21.15.5:0/1825995284 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11969 (secure 0 0 0) 0x7f3d5c09bf70 con 0x7f3d6c104010 2024-09-10T09:40:16.844 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-10T09:40:16.844 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":30,"btime":"2024-09-10T09:38:15:834198+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14454,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/2970594311","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2970594311},{"type":"v1","addr":"172.21.15.5:6835","nonce":2970594311}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14514,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6839/3616502174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":3616502174},{"type":"v1","addr":"172.21.15.5:6839","nonce":3616502174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T09:35:29.626287+0000","modified":"2024-09-10T09:38:15.658058+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24542,"mds_1":24535,"mds_2":24511,"mds_3":24506,"mds_4":24443},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14520":{"gid":14520,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/2073599842","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":2073599842},{"type":"v1","addr":"172.21.15.5:6841","nonce":2073599842}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"k","rank":4,"incarnation":26,"state":"up:active","state_seq":24,"addr":"172.21.15.102:6833/2457007427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":2457007427},{"type":"v1","addr":"172.21.15.102:6833","nonce":2457007427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/356445159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":356445159},{"type":"v1","addr":"172.21.15.159:6833","nonce":356445159}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6837/2894314507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":2894314507},{"type":"v1","addr":"172.21.15.5:6837","nonce":2894314507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.102:6835/1647549111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":1647549111},{"type":"v1","addr":"172.21.15.102:6835","nonce":1647549111}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6835/952657242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":952657242},{"type":"v1","addr":"172.21.15.159:6835","nonce":952657242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":19,"addr":"172.21.15.102:6837/2689776764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":2689776764},{"type":"v1","addr":"172.21.15.102:6837","nonce":2689776764}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.159:6837/620627639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":620627639},{"type":"v1","addr":"172.21.15.159:6837","nonce":620627639}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24535":{"gid":24535,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":18,"addr":"172.21.15.159:6839/2073739599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":2073739599},{"type":"v1","addr":"172.21.15.159:6839","nonce":2073739599}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24542":{"gid":24542,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.102:6839/335409749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6838","nonce":335409749},{"type":"v1","addr":"172.21.15.102:6839","nonce":335409749}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24542,"qdb_cluster":[24542,24535,24511,24506,24443]},"id":1}]} 2024-09-10T09:40:16.844 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 30 2024-09-10T09:40:16.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:16.845+0000 7f3d70c00640 1 -- 172.21.15.5:0/1825995284 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f3d500a2b80 msgr2=0x7f3d500a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:40:16.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:16.845+0000 7f3d70c00640 1 --2- 172.21.15.5:0/1825995284 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f3d500a2b80 0x7f3d500a5040 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f3d6c06e220 tx=0x7f3d58005f90 comp rx=0 tx=0).stop 2024-09-10T09:40:16.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:16.845+0000 7f3d70c00640 1 -- 172.21.15.5:0/1825995284 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3d6c104010 msgr2=0x7f3d6c0739d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:40:16.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:16.845+0000 7f3d70c00640 1 --2- 172.21.15.5:0/1825995284 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3d6c104010 0x7f3d6c0739d0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f3d5c00ebc0 tx=0x7f3d5c00c6a0 comp rx=0 tx=0).stop 2024-09-10T09:40:16.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:16.845+0000 7f3d70c00640 1 -- 172.21.15.5:0/1825995284 shutdown_connections 2024-09-10T09:40:16.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:16.845+0000 7f3d70c00640 1 --2- 172.21.15.5:0/1825995284 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f3d500a2b80 0x7f3d500a5040 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:16.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:16.845+0000 7f3d70c00640 1 --2- 172.21.15.5:0/1825995284 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3d6c109140 0x7f3d6c06cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:16.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:16.845+0000 7f3d70c00640 1 --2- 172.21.15.5:0/1825995284 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3d6c1049c0 0x7f3d6c06ca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:16.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:16.845+0000 7f3d70c00640 1 --2- 172.21.15.5:0/1825995284 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3d6c104010 0x7f3d6c0739d0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:16.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:16.845+0000 7f3d70c00640 1 -- 172.21.15.5:0/1825995284 >> 172.21.15.5:0/1825995284 conn(0x7f3d6c0ffa50 msgr2=0x7f3d6c100db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:40:16.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:16.845+0000 7f3d70c00640 1 -- 172.21.15.5:0/1825995284 shutdown_connections 2024-09-10T09:40:16.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:16.845+0000 7f3d70c00640 1 -- 172.21.15.5:0/1825995284 wait complete. 2024-09-10T09:40:17.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:40:16 smithi159 ceph-mon[35513]: pgmap v451: 129 pgs: 129 active+clean; 666 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-09-10T09:40:17.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:40:16 smithi102 ceph-mon[35442]: pgmap v451: 129 pgs: 129 active+clean; 666 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-09-10T09:40:17.740 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24542 quiesce db --set-id f9cf197c --release --await' 2024-09-10T09:40:17.740 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24542 quiesce db --set-id f9cf197c --release --await 2024-09-10T09:40:18.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:40:17 smithi005 ceph-mon[30350]: from='client.? 172.21.15.5:0/1825995284' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:40:18.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:40:17 smithi159 ceph-mon[35513]: from='client.? 172.21.15.5:0/1825995284' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:40:18.118 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:40:18.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:40:17 smithi102 ceph-mon[35442]: from='client.? 172.21.15.5:0/1825995284' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:40:18.860 INFO:tasks.workunit.client.0.smithi005.stderr:+ dbench 1 2024-09-10T09:40:18.874 INFO:tasks.workunit.client.0.smithi005.stdout:dbench version 4.00 - Copyright Andrew Tridgell 1999-2004 2024-09-10T09:40:18.874 INFO:tasks.workunit.client.0.smithi005.stdout: 2024-09-10T09:40:18.874 INFO:tasks.workunit.client.0.smithi005.stdout:Running for 600 seconds with load '/usr/share/dbench/client.txt' and minimum warmup 120 secs 2024-09-10T09:40:18.883 INFO:tasks.workunit.client.0.smithi005.stdout:0 of 1 processes prepared for launch 0 sec 2024-09-10T09:40:18.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:18.801+0000 7f8ce8000640 1 -- 172.21.15.5:0/4155506365 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8ce0106f10 msgr2=0x7f8ce010ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:40:18.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:18.801+0000 7f8ce8000640 1 --2- 172.21.15.5:0/4155506365 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8ce0106f10 0x7f8ce010ea90 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f8cd8009ca0 tx=0x7f8cd802a8c0 comp rx=0 tx=0).stop 2024-09-10T09:40:18.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:18.801+0000 7f8ce8000640 1 -- 172.21.15.5:0/4155506365 shutdown_connections 2024-09-10T09:40:18.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:18.801+0000 7f8ce8000640 1 --2- 172.21.15.5:0/4155506365 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8ce0106f10 0x7f8ce010ea90 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:18.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:18.801+0000 7f8ce8000640 1 --2- 172.21.15.5:0/4155506365 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8ce0102790 0x7f8ce01067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:18.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:18.801+0000 7f8ce8000640 1 --2- 172.21.15.5:0/4155506365 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8ce0101de0 0x7f8ce01021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:18.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:18.802+0000 7f8ce8000640 1 -- 172.21.15.5:0/4155506365 >> 172.21.15.5:0/4155506365 conn(0x7f8ce00fd8c0 msgr2=0x7f8ce00ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:40:18.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:18.802+0000 7f8ce8000640 1 -- 172.21.15.5:0/4155506365 shutdown_connections 2024-09-10T09:40:18.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:18.802+0000 7f8ce8000640 1 -- 172.21.15.5:0/4155506365 wait complete. 2024-09-10T09:40:18.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:18.803+0000 7f8ce8000640 1 Processor -- start 2024-09-10T09:40:18.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:18.803+0000 7f8ce8000640 1 -- start start 2024-09-10T09:40:18.981 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:40:18 smithi005 ceph-mon[30350]: pgmap v452: 129 pgs: 129 active+clean; 666 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-09-10T09:40:18.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:18.803+0000 7f8ce8000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8ce0101de0 0x7f8ce019a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:40:18.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:18.804+0000 7f8ce8000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8ce0102790 0x7f8ce019abf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:40:18.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:18.804+0000 7f8ce8000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8ce0106f10 0x7f8ce019ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:40:18.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:18.804+0000 7f8ce8000640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f8ce0111f70 con 0x7f8ce0102790 2024-09-10T09:40:18.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:18.804+0000 7f8ce8000640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f8ce0111df0 con 0x7f8ce0101de0 2024-09-10T09:40:18.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:18.804+0000 7f8ce8000640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f8ce01120f0 con 0x7f8ce0106f10 2024-09-10T09:40:18.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:18.804+0000 7f8ce4c00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8ce0102790 0x7f8ce019abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:40:18.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:18.804+0000 7f8ce5600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8ce0101de0 0x7f8ce019a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:40:18.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:18.804+0000 7f8ce4c00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8ce0102790 0x7f8ce019abf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41746/0 (socket says 172.21.15.5:41746) 2024-09-10T09:40:18.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:18.804+0000 7f8ce4c00640 1 -- 172.21.15.5:0/1581682871 learned_addr learned my addr 172.21.15.5:0/1581682871 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:40:18.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:18.804+0000 7f8ce6000640 1 --2- 172.21.15.5:0/1581682871 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8ce0106f10 0x7f8ce019ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:40:18.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:18.804+0000 7f8ce5600640 1 -- 172.21.15.5:0/1581682871 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8ce0106f10 msgr2=0x7f8ce019ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:40:18.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:18.804+0000 7f8ce5600640 1 --2- 172.21.15.5:0/1581682871 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8ce0106f10 0x7f8ce019ef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:18.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:18.804+0000 7f8ce5600640 1 -- 172.21.15.5:0/1581682871 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8ce0102790 msgr2=0x7f8ce019abf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:40:18.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:18.804+0000 7f8ce5600640 1 --2- 172.21.15.5:0/1581682871 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8ce0102790 0x7f8ce019abf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:18.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:18.804+0000 7f8ce5600640 1 -- 172.21.15.5:0/1581682871 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ce019f660 con 0x7f8ce0101de0 2024-09-10T09:40:18.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:18.805+0000 7f8ce5600640 1 --2- 172.21.15.5:0/1581682871 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8ce0101de0 0x7f8ce019a6b0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f8cd400e9f0 tx=0x7f8cd400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:40:18.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:18.805+0000 7f8cde000640 1 -- 172.21.15.5:0/1581682871 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cd400ccc0 con 0x7f8ce0101de0 2024-09-10T09:40:18.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:18.805+0000 7f8cde000640 1 -- 172.21.15.5:0/1581682871 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8cd4010910 con 0x7f8ce0101de0 2024-09-10T09:40:18.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:18.805+0000 7f8ce8000640 1 -- 172.21.15.5:0/1581682871 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ce019f950 con 0x7f8ce0101de0 2024-09-10T09:40:18.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:18.805+0000 7f8cde000640 1 -- 172.21.15.5:0/1581682871 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cd4018960 con 0x7f8ce0101de0 2024-09-10T09:40:18.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:18.805+0000 7f8ce8000640 1 -- 172.21.15.5:0/1581682871 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ce01a7230 con 0x7f8ce0101de0 2024-09-10T09:40:18.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:18.808+0000 7f8cde000640 1 -- 172.21.15.5:0/1581682871 <== mon.2 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f8cd4010ac0 con 0x7f8ce0101de0 2024-09-10T09:40:18.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:18.808+0000 7f8ceaa7f740 1 Processor -- start 2024-09-10T09:40:18.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:18.809+0000 7f8ceaa7f740 1 -- start start 2024-09-10T09:40:18.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:18.809+0000 7f8ceaa7f740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56359fa72310 0x56359fa726f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:40:18.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:18.809+0000 7f8ceaa7f740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56359fa72c30 0x56359fa7cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:40:18.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:18.809+0000 7f8ceaa7f740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56359fa7d4f0 0x56359fa7f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:40:18.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:18.809+0000 7f8ceaa7f740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x56359f9d8650 con 0x56359fa7d4f0 2024-09-10T09:40:18.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:18.809+0000 7f8ceaa7f740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x56359f97ff10 con 0x56359fa72c30 2024-09-10T09:40:18.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:18.809+0000 7f8ceaa7f740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x56359f9d4610 con 0x56359fa72310 2024-09-10T09:40:18.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:18.809+0000 7f8ce4c00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56359fa7d4f0 0x56359fa7f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:40:18.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:18.809+0000 7f8ce6000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56359fa72310 0x56359fa726f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:40:18.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:18.809+0000 7f8ce6000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56359fa72310 0x56359fa726f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:45984/0 (socket says 172.21.15.5:45984) 2024-09-10T09:40:18.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:18.809+0000 7f8ce6000640 1 -- 172.21.15.5:0/1778613640 learned_addr learned my addr 172.21.15.5:0/1778613640 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:40:18.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:18.809+0000 7f8ce5600640 1 --2- 172.21.15.5:0/1778613640 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56359fa72c30 0x56359fa7cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:40:18.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:18.810+0000 7f8ce6000640 1 -- 172.21.15.5:0/1778613640 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56359fa72c30 msgr2=0x56359fa7cfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:40:18.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:18.810+0000 7f8ce6000640 1 --2- 172.21.15.5:0/1778613640 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56359fa72c30 0x56359fa7cfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:18.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:18.810+0000 7f8ce6000640 1 -- 172.21.15.5:0/1778613640 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56359fa7d4f0 msgr2=0x56359fa7f8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:40:18.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:18.810+0000 7f8ce6000640 1 --2- 172.21.15.5:0/1778613640 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56359fa7d4f0 0x56359fa7f8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:18.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:18.810+0000 7f8ce6000640 1 -- 172.21.15.5:0/1778613640 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56359f6e7580 con 0x56359fa72310 2024-09-10T09:40:18.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:18.810+0000 7f8cde000640 1 --2- 172.21.15.5:0/1581682871 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f8cc80a2b80 0x7f8cc80a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:40:18.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:18.810+0000 7f8cde000640 1 -- 172.21.15.5:0/1581682871 <== mon.2 v2:172.21.15.102:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f8cd40cf820 con 0x7f8ce0101de0 2024-09-10T09:40:18.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:18.810+0000 7f8ce6000640 1 --2- 172.21.15.5:0/1778613640 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56359fa72310 0x56359fa726f0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f8cd8009ca0 tx=0x7f8cd8005f10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=48c3c6cbb752ea8f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:40:18.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:18.810+0000 7f8ce5600640 1 --2- 172.21.15.5:0/1581682871 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f8cc80a2b80 0x7f8cc80a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:40:18.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:18.810+0000 7f8cd2600640 1 -- 172.21.15.5:0/1778613640 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cd8031db0 con 0x56359fa72310 2024-09-10T09:40:18.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:18.810+0000 7f8cd2600640 1 -- 172.21.15.5:0/1778613640 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8cd8036040 con 0x56359fa72310 2024-09-10T09:40:18.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:18.811+0000 7f8cd2600640 1 -- 172.21.15.5:0/1778613640 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cd8004330 con 0x56359fa72310 2024-09-10T09:40:18.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:18.811+0000 7f8ceaa7f740 1 -- 172.21.15.5:0/1778613640 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56359fa72310 msgr2=0x56359fa726f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:40:18.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:18.811+0000 7f8ceaa7f740 1 --2- 172.21.15.5:0/1778613640 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56359fa72310 0x56359fa726f0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f8cd8009ca0 tx=0x7f8cd8005f10 comp rx=0 tx=0).stop 2024-09-10T09:40:18.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:18.811+0000 7f8ce5600640 1 --2- 172.21.15.5:0/1581682871 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f8cc80a2b80 0x7f8cc80a5040 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f8cd4098140 tx=0x7f8cd4009700 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:40:18.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:18.811+0000 7f8ceaa7f740 1 -- 172.21.15.5:0/1778613640 shutdown_connections 2024-09-10T09:40:18.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:18.811+0000 7f8ceaa7f740 1 --2- 172.21.15.5:0/1778613640 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56359fa7d4f0 0x56359fa7f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:18.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:18.811+0000 7f8ceaa7f740 1 --2- 172.21.15.5:0/1778613640 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56359fa72c30 0x56359fa7cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:18.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:18.811+0000 7f8ceaa7f740 1 --2- 172.21.15.5:0/1778613640 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56359fa72310 0x56359fa726f0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:18.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:18.811+0000 7f8ceaa7f740 1 -- 172.21.15.5:0/1778613640 >> 172.21.15.5:0/1778613640 conn(0x56359fa6e8a0 msgr2=0x56359fa6ec80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:40:18.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:18.811+0000 7f8ceaa7f740 1 -- 172.21.15.5:0/1778613640 shutdown_connections 2024-09-10T09:40:18.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:18.811+0000 7f8ceaa7f740 1 -- 172.21.15.5:0/1778613640 wait complete. 2024-09-10T09:40:18.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:18.812+0000 7f8ceaa7f740 1 -- start start 2024-09-10T09:40:18.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:18.812+0000 7f8ceaa7f740 1 Processor -- start 2024-09-10T09:40:18.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:18.813+0000 7f8ceaa7f740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56359fa72310 0x56359fa7a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:40:18.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:18.813+0000 7f8ceaa7f740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56359fa72c30 0x56359fa7a920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:40:18.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:18.813+0000 7f8ceaa7f740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56359fa7d4f0 0x56359fa7ae60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:40:18.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:18.813+0000 7f8ceaa7f740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x56359fa7ff40 con 0x56359fa7d4f0 2024-09-10T09:40:18.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:18.813+0000 7f8ceaa7f740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x56359f993350 con 0x56359fa72c30 2024-09-10T09:40:18.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:18.813+0000 7f8ceaa7f740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x56359fa800c0 con 0x56359fa72310 2024-09-10T09:40:18.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:18.813+0000 7f8ceaa7f740 10 client.0 _ll_register_callbacks cb 0x56359f9b6290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:40:18.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:18.813+0000 7f8ce6000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56359fa72310 0x56359fa7a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:40:18.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:18.813+0000 7f8ce6000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56359fa72310 0x56359fa7a3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:45988/0 (socket says 172.21.15.5:45988) 2024-09-10T09:40:18.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:18.813+0000 7f8ce4c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56359fa72c30 0x56359fa7a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:40:18.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:18.813+0000 7f8ce6000640 1 -- 172.21.15.5:0/2627325095 learned_addr learned my addr 172.21.15.5:0/2627325095 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:40:18.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:18.813+0000 7f8ce6000640 1 --2- 172.21.15.5:0/2627325095 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56359fa7d4f0 0x56359fa7ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:40:18.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:18.813+0000 7f8ce4c00640 1 -- 172.21.15.5:0/2627325095 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56359fa72310 msgr2=0x56359fa7a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:40:18.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:18.813+0000 7f8ce4c00640 1 --2- 172.21.15.5:0/2627325095 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56359fa72310 0x56359fa7a3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:18.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:18.813+0000 7f8ce4c00640 1 -- 172.21.15.5:0/2627325095 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56359fa7d4f0 msgr2=0x56359fa7ae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:40:18.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:18.813+0000 7f8ce4c00640 1 --2- 172.21.15.5:0/2627325095 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56359fa7d4f0 0x56359fa7ae60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:18.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:18.813+0000 7f8ce4c00640 1 -- 172.21.15.5:0/2627325095 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56359f7b1960 con 0x56359fa72c30 2024-09-10T09:40:18.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:18.814+0000 7f8ce4c00640 1 --2- 172.21.15.5:0/2627325095 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56359fa72c30 0x56359fa7a920 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x56359fa6fc20 tx=0x7f8ccc012b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9a0c5ac65a083a70 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:40:18.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:18.814+0000 7f8cc6a00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-09-10T09:40:18.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:18.814+0000 7f8cc6a00640 1 -- 172.21.15.5:0/2627325095 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ccc00f890 con 0x56359fa72c30 2024-09-10T09:40:18.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:18.814+0000 7f8ceaa7f740 1 -- 172.21.15.5:0/2627325095 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8cd8036040 con 0x56359fa72c30 2024-09-10T09:40:18.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:18.814+0000 7f8cc6a00640 1 -- 172.21.15.5:0/2627325095 <== mon.2 v2:172.21.15.102:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f8ccc00fa30 con 0x56359fa72c30 2024-09-10T09:40:18.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:18.814+0000 7f8cc6a00640 1 -- 172.21.15.5:0/2627325095 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f8ccc00fc30 con 0x56359fa72c30 2024-09-10T09:40:18.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:18.814+0000 7f8ceaa7f740 10 client.25817 fetch_fsmap learned FSMap version 30 2024-09-10T09:40:18.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:18.814+0000 7f8ceaa7f740 1 -- 172.21.15.5:0/2627325095 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x56359fa8bc00 con 0x56359fa72c30 2024-09-10T09:40:18.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:18.815+0000 7f8cc6a00640 1 -- 172.21.15.5:0/2627325095 <== mon.2 v2:172.21.15.102:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8cd8036040 con 0x56359fa72c30 2024-09-10T09:40:18.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:18.815+0000 7f8cc6a00640 1 -- 172.21.15.5:0/2627325095 <== mon.2 v2:172.21.15.102:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f8ccc004b00 con 0x56359fa72c30 2024-09-10T09:40:18.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:18.815+0000 7f8ceaa7f740 10 client.25817 fetch_fsmap finished waiting for FSMap version 30 2024-09-10T09:40:18.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:18.815+0000 7f8ceaa7f740 10 client.25817 resolve_mds: validated gid 24542 aka daemon mds.b 2024-09-10T09:40:18.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:18.815+0000 7f8ceaa7f740 20 client.25817 populate_metadata read hostname 'smithi005' 2024-09-10T09:40:18.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:18.815+0000 7f8ceaa7f740 1 --2- 172.21.15.5:0/2627325095 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x56359fa8d060 0x56359fa8f480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:40:18.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:18.815+0000 7f8ceaa7f740 4 client.25817 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:40:18.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:18.815+0000 7f8ceaa7f740 1 -- 172.21.15.5:0/2627325095 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56359f9e5bc0 con 0x56359fa8d060 2024-09-10T09:40:18.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:18.815+0000 7f8ce6000640 1 --2- 172.21.15.5:0/2627325095 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x56359fa8d060 0x56359fa8f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:40:18.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:18.816+0000 7f8ce6000640 1 --2- 172.21.15.5:0/2627325095 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x56359fa8d060 0x56359fa8f480 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ac3b0a5eff625a51 server_cookie=82dc358b83102e99 in_seq=0 out_seq=0 2024-09-10T09:40:18.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:18.816+0000 7f8cc6a00640 10 client.25817 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T09:40:18.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:18.817+0000 7f8cc6a00640 1 -- 172.21.15.5:0/2627325095 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56359f9e5bc0 con 0x56359fa8d060 2024-09-10T09:40:18.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:18.817+0000 7f8cc6a00640 10 client.25817 handle_command_reply: tid=0 2024-09-10T09:40:18.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:18.817+0000 7f8ceaa7f740 1 client.25817 shutdown 2024-09-10T09:40:18.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:18.817+0000 7f8ceaa7f740 1 -- 172.21.15.5:0/2627325095 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56359fa72c30 msgr2=0x56359fa7a920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:40:18.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:18.817+0000 7f8ceaa7f740 1 --2- 172.21.15.5:0/2627325095 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56359fa72c30 0x56359fa7a920 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x56359fa6fc20 tx=0x7f8ccc012b60 comp rx=0 tx=0).stop 2024-09-10T09:40:18.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:18.817+0000 7f8ceaa7f740 1 -- 172.21.15.5:0/2627325095 shutdown_connections 2024-09-10T09:40:18.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:18.817+0000 7f8ceaa7f740 1 --2- 172.21.15.5:0/2627325095 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x56359fa8d060 0x56359fa8f480 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:18.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:18.817+0000 7f8ceaa7f740 1 --2- 172.21.15.5:0/2627325095 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56359fa7d4f0 0x56359fa7ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:18.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:18.817+0000 7f8ceaa7f740 1 --2- 172.21.15.5:0/2627325095 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56359fa72c30 0x56359fa7a920 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:18.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:18.817+0000 7f8ceaa7f740 1 --2- 172.21.15.5:0/2627325095 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56359fa72310 0x56359fa7a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:18.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:18.817+0000 7f8ceaa7f740 1 -- 172.21.15.5:0/2627325095 >> 172.21.15.5:0/2627325095 conn(0x56359fa6e8a0 msgr2=0x56359fa6daa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:40:18.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:18.817+0000 7f8cc6a00640 0 client.25817 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T09:40:18.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:18.817+0000 7f8ceaa7f740 1 -- 172.21.15.5:0/2627325095 shutdown_connections 2024-09-10T09:40:18.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:18.817+0000 7f8ceaa7f740 1 -- 172.21.15.5:0/2627325095 wait complete. 2024-09-10T09:40:18.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:18.817+0000 7f8ceaa7f740 20 client.25817 trim_cache size 0 max 16384 2024-09-10T09:40:18.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:18.835+0000 7f8ceaa7f740 1 Processor -- start 2024-09-10T09:40:18.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:18.835+0000 7f8ceaa7f740 1 -- start start 2024-09-10T09:40:18.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:18.835+0000 7f8ceaa7f740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56359fa72310 0x56359fa94d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:40:18.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:18.835+0000 7f8ceaa7f740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56359fa72c30 0x56359fa95270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:40:18.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:18.835+0000 7f8ceaa7f740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56359fa7d4f0 0x56359fa77660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:40:18.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:18.835+0000 7f8ceaa7f740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f8ce01b0ff0 con 0x56359fa72c30 2024-09-10T09:40:18.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:18.835+0000 7f8ceaa7f740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f8ce01b2b30 con 0x56359fa7d4f0 2024-09-10T09:40:18.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:18.835+0000 7f8ceaa7f740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f8ce01b16c0 con 0x56359fa72310 2024-09-10T09:40:18.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:18.836+0000 7f8ce4c00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56359fa72c30 0x56359fa95270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:40:18.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:18.836+0000 7f8ce6000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56359fa7d4f0 0x56359fa77660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:40:18.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:18.836+0000 7f8ce6000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56359fa7d4f0 0x56359fa77660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:40476/0 (socket says 172.21.15.5:40476) 2024-09-10T09:40:18.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:18.836+0000 7f8ce6000640 1 -- 172.21.15.5:0/4023801767 learned_addr learned my addr 172.21.15.5:0/4023801767 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:40:18.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:18.836+0000 7f8ce6000640 1 --2- 172.21.15.5:0/4023801767 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56359fa72310 0x56359fa94d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:40:18.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:18.836+0000 7f8ce6000640 1 -- 172.21.15.5:0/4023801767 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56359fa72310 msgr2=0x56359fa94d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:40:18.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:18.836+0000 7f8ce6000640 1 --2- 172.21.15.5:0/4023801767 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56359fa72310 0x56359fa94d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:18.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:18.836+0000 7f8ce6000640 1 -- 172.21.15.5:0/4023801767 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56359fa72c30 msgr2=0x56359fa95270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:40:18.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:18.836+0000 7f8ce6000640 1 --2- 172.21.15.5:0/4023801767 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56359fa72c30 0x56359fa95270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:18.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:18.836+0000 7f8ce6000640 1 -- 172.21.15.5:0/4023801767 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56359fa7b3b0 con 0x56359fa7d4f0 2024-09-10T09:40:18.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:18.837+0000 7f8ce6000640 1 --2- 172.21.15.5:0/4023801767 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56359fa7d4f0 0x56359fa77660 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f8cd8031950 tx=0x7f8cd802a8c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2ac7f439afc6b44d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:40:18.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:18.837+0000 7f8cd1c00640 1 -- 172.21.15.5:0/4023801767 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cd8031bb0 con 0x56359fa7d4f0 2024-09-10T09:40:18.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:18.837+0000 7f8cd1c00640 1 -- 172.21.15.5:0/4023801767 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8cd8036030 con 0x56359fa7d4f0 2024-09-10T09:40:18.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:18.837+0000 7f8cd1c00640 1 -- 172.21.15.5:0/4023801767 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cd80305c0 con 0x56359fa7d4f0 2024-09-10T09:40:18.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:18.838+0000 7f8ceaa7f740 1 -- 172.21.15.5:0/4023801767 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56359fa7d4f0 msgr2=0x56359fa77660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:40:18.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:18.838+0000 7f8ceaa7f740 1 --2- 172.21.15.5:0/4023801767 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56359fa7d4f0 0x56359fa77660 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f8cd8031950 tx=0x7f8cd802a8c0 comp rx=0 tx=0).stop 2024-09-10T09:40:18.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:18.838+0000 7f8ceaa7f740 1 -- 172.21.15.5:0/4023801767 shutdown_connections 2024-09-10T09:40:18.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:18.838+0000 7f8ceaa7f740 1 --2- 172.21.15.5:0/4023801767 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56359fa7d4f0 0x56359fa77660 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:18.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:18.838+0000 7f8ceaa7f740 1 --2- 172.21.15.5:0/4023801767 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56359fa72c30 0x56359fa95270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:18.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:18.838+0000 7f8ceaa7f740 1 --2- 172.21.15.5:0/4023801767 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56359fa72310 0x56359fa94d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:18.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:18.838+0000 7f8ceaa7f740 1 -- 172.21.15.5:0/4023801767 >> 172.21.15.5:0/4023801767 conn(0x56359fa6e8a0 msgr2=0x56359fa6daa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:40:18.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:18.838+0000 7f8ceaa7f740 1 -- 172.21.15.5:0/4023801767 shutdown_connections 2024-09-10T09:40:18.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:18.838+0000 7f8ceaa7f740 1 -- 172.21.15.5:0/4023801767 wait complete. 2024-09-10T09:40:18.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:18.838+0000 7f8ceaa7f740 1 -- start start 2024-09-10T09:40:18.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:18.839+0000 7f8ceaa7f740 1 Processor -- start 2024-09-10T09:40:18.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:18.839+0000 7f8ceaa7f740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56359fa72310 0x56359fa85310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:40:18.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:18.839+0000 7f8ceaa7f740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56359fa72c30 0x56359fa85850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:40:18.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:18.839+0000 7f8ceaa7f740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56359fa7d4f0 0x56359fa822e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:40:18.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:18.839+0000 7f8ceaa7f740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f8ce01b2460 con 0x56359fa72c30 2024-09-10T09:40:18.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:18.839+0000 7f8ceaa7f740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f8ce01b1d90 con 0x56359fa72310 2024-09-10T09:40:18.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:18.839+0000 7f8ceaa7f740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f8ce01aefd0 con 0x56359fa7d4f0 2024-09-10T09:40:18.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:18.839+0000 7f8ceaa7f740 10 client.0 _ll_register_callbacks cb 0x7f8ccc009720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:40:18.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:18.839+0000 7f8ce4c00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56359fa72c30 0x56359fa85850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:40:18.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:18.839+0000 7f8ce6000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56359fa72310 0x56359fa85310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:40:18.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:18.839+0000 7f8ce6000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56359fa72310 0x56359fa85310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:40484/0 (socket says 172.21.15.5:40484) 2024-09-10T09:40:18.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:18.839+0000 7f8ce6000640 1 -- 172.21.15.5:0/577456608 learned_addr learned my addr 172.21.15.5:0/577456608 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:40:18.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:18.839+0000 7f8ce6000640 1 --2- 172.21.15.5:0/577456608 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56359fa7d4f0 0x56359fa822e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:40:18.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:18.840+0000 7f8ce6000640 1 -- 172.21.15.5:0/577456608 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56359fa7d4f0 msgr2=0x56359fa822e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:40:18.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:18.840+0000 7f8ce6000640 1 --2- 172.21.15.5:0/577456608 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56359fa7d4f0 0x56359fa822e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:18.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:18.840+0000 7f8ce6000640 1 -- 172.21.15.5:0/577456608 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56359fa72c30 msgr2=0x56359fa85850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:40:18.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:18.840+0000 7f8ce6000640 1 --2- 172.21.15.5:0/577456608 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56359fa72c30 0x56359fa85850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:18.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:18.840+0000 7f8ce6000640 1 -- 172.21.15.5:0/577456608 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56359fa82820 con 0x56359fa72310 2024-09-10T09:40:18.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:18.840+0000 7f8ce6000640 1 --2- 172.21.15.5:0/577456608 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56359fa72310 0x56359fa85310 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f8cd802f470 tx=0x7f8cd802b040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5faad79593b57c17 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:40:18.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:18.840+0000 7f8cc6a00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-09-10T09:40:18.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:18.840+0000 7f8cc6a00640 1 -- 172.21.15.5:0/577456608 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cd8031db0 con 0x56359fa72310 2024-09-10T09:40:18.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:18.840+0000 7f8ceaa7f740 1 -- 172.21.15.5:0/577456608 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8cd8036030 con 0x56359fa72310 2024-09-10T09:40:18.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:18.841+0000 7f8cc6a00640 1 -- 172.21.15.5:0/577456608 <== mon.2 v2:172.21.15.102:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f8cd802f610 con 0x56359fa72310 2024-09-10T09:40:18.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:18.841+0000 7f8cc6a00640 1 -- 172.21.15.5:0/577456608 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f8cd802f810 con 0x56359fa72310 2024-09-10T09:40:18.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:18.841+0000 7f8ceaa7f740 10 client.25823 fetch_fsmap learned FSMap version 30 2024-09-10T09:40:18.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:18.841+0000 7f8ceaa7f740 1 -- 172.21.15.5:0/577456608 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x56359faf8400 con 0x56359fa72310 2024-09-10T09:40:18.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:18.841+0000 7f8cc6a00640 1 -- 172.21.15.5:0/577456608 <== mon.2 v2:172.21.15.102:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8cd8036030 con 0x56359fa72310 2024-09-10T09:40:18.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:18.841+0000 7f8cc6a00640 1 -- 172.21.15.5:0/577456608 <== mon.2 v2:172.21.15.102:3300/0 5 ==== fsmap(e 30) ==== 8370+0+0 (secure 0 0 0) 0x7f8cd8050040 con 0x56359fa72310 2024-09-10T09:40:18.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:18.841+0000 7f8ceaa7f740 10 client.25823 fetch_fsmap finished waiting for FSMap version 30 2024-09-10T09:40:18.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:18.841+0000 7f8ceaa7f740 10 client.25823 resolve_mds: validated gid 24542 aka daemon mds.b 2024-09-10T09:40:18.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:18.841+0000 7f8ceaa7f740 20 client.25823 populate_metadata read hostname 'smithi005' 2024-09-10T09:40:18.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:18.841+0000 7f8ceaa7f740 1 --2- 172.21.15.5:0/577456608 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x56359faf8730 0x56359fafab20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:40:18.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:18.841+0000 7f8ceaa7f740 4 client.25823 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "f9cf197c", "release": true, "await": true}] 2024-09-10T09:40:18.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:18.841+0000 7f8ceaa7f740 1 -- 172.21.15.5:0/577456608 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "quiesce db", "set_id": "f9cf197c", "release": true, "await": true}) -- 0x56359f9e00d0 con 0x56359faf8730 2024-09-10T09:40:18.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:18.842+0000 7f8ce4c00640 1 --2- 172.21.15.5:0/577456608 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x56359faf8730 0x56359fafab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:40:18.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:18.842+0000 7f8ce4c00640 1 --2- 172.21.15.5:0/577456608 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x56359faf8730 0x56359fafab20 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=849d233abe7678a server_cookie=acbf69e7e2891373 in_seq=0 out_seq=0 2024-09-10T09:40:18.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:18.842+0000 7f8cc6a00640 10 client.25823 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T09:40:18.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:18.845+0000 7f8cc6a00640 1 -- 172.21.15.5:0/577456608 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x56359f9e00d0 con 0x56359faf8730 2024-09-10T09:40:18.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:18.845+0000 7f8cc6a00640 10 client.25823 handle_command_reply: tid=0 2024-09-10T09:40:18.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:18.845+0000 7f8ceaa7f740 1 client.25823 shutdown 2024-09-10T09:40:18.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:18.845+0000 7f8ceaa7f740 1 -- 172.21.15.5:0/577456608 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56359fa72310 msgr2=0x56359fa85310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:40:18.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:18.845+0000 7f8ceaa7f740 1 --2- 172.21.15.5:0/577456608 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56359fa72310 0x56359fa85310 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f8cd802f470 tx=0x7f8cd802b040 comp rx=0 tx=0).stop 2024-09-10T09:40:18.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:18.846+0000 7f8ceaa7f740 1 -- 172.21.15.5:0/577456608 shutdown_connections 2024-09-10T09:40:18.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:18.846+0000 7f8ceaa7f740 1 --2- 172.21.15.5:0/577456608 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x56359faf8730 0x56359fafab20 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:18.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:18.846+0000 7f8ceaa7f740 1 --2- 172.21.15.5:0/577456608 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56359fa7d4f0 0x56359fa822e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:18.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:18.846+0000 7f8ceaa7f740 1 --2- 172.21.15.5:0/577456608 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56359fa72c30 0x56359fa85850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:18.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:18.846+0000 7f8cc6a00640 0 client.25823 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T09:40:18.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:18.846+0000 7f8ceaa7f740 1 --2- 172.21.15.5:0/577456608 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56359fa72310 0x56359fa85310 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:18.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:18.846+0000 7f8ceaa7f740 1 -- 172.21.15.5:0/577456608 >> 172.21.15.5:0/577456608 conn(0x56359fa6e8a0 msgr2=0x56359fa73d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:40:18.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:18.846+0000 7f8ceaa7f740 1 -- 172.21.15.5:0/577456608 shutdown_connections 2024-09-10T09:40:18.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:18.846+0000 7f8ceaa7f740 1 -- 172.21.15.5:0/577456608 wait complete. 2024-09-10T09:40:18.999 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-10T09:40:19.000 INFO:teuthology.orchestra.run.smithi005.stdout: "epoch": 30, 2024-09-10T09:40:19.000 INFO:teuthology.orchestra.run.smithi005.stdout: "leader": 24542, 2024-09-10T09:40:19.000 INFO:teuthology.orchestra.run.smithi005.stdout: "set_version": 8, 2024-09-10T09:40:19.000 INFO:teuthology.orchestra.run.smithi005.stdout: "sets": { 2024-09-10T09:40:19.000 INFO:teuthology.orchestra.run.smithi005.stdout: "f9cf197c": { 2024-09-10T09:40:19.000 INFO:teuthology.orchestra.run.smithi005.stdout: "version": 8, 2024-09-10T09:40:19.000 INFO:teuthology.orchestra.run.smithi005.stdout: "age_ref": 0.0, 2024-09-10T09:40:19.000 INFO:teuthology.orchestra.run.smithi005.stdout: "state": { 2024-09-10T09:40:19.000 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "RELEASED", 2024-09-10T09:40:19.000 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0 2024-09-10T09:40:19.000 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T09:40:19.000 INFO:teuthology.orchestra.run.smithi005.stdout: "timeout": 90.0, 2024-09-10T09:40:19.000 INFO:teuthology.orchestra.run.smithi005.stdout: "expiration": 150.0, 2024-09-10T09:40:19.001 INFO:teuthology.orchestra.run.smithi005.stdout: "members": { 2024-09-10T09:40:19.001 INFO:teuthology.orchestra.run.smithi005.stdout: "file:/": { 2024-09-10T09:40:19.001 INFO:teuthology.orchestra.run.smithi005.stdout: "excluded": false, 2024-09-10T09:40:19.001 INFO:teuthology.orchestra.run.smithi005.stdout: "state": { 2024-09-10T09:40:19.001 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "RELEASED", 2024-09-10T09:40:19.001 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0 2024-09-10T09:40:19.001 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T09:40:19.001 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T09:40:19.001 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T09:40:19.001 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T09:40:19.001 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T09:40:19.001 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-10T09:40:19.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:18.846+0000 7f8ceaa7f740 20 client.25823 trim_cache size 0 max 16384 2024-09-10T09:40:19.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:18.847+0000 7f8ce8000640 1 -- 172.21.15.5:0/1581682871 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f8cc80a2b80 msgr2=0x7f8cc80a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:40:19.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:18.847+0000 7f8ce8000640 1 --2- 172.21.15.5:0/1581682871 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f8cc80a2b80 0x7f8cc80a5040 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f8cd4098140 tx=0x7f8cd4009700 comp rx=0 tx=0).stop 2024-09-10T09:40:19.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:18.847+0000 7f8ce8000640 1 -- 172.21.15.5:0/1581682871 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8ce0101de0 msgr2=0x7f8ce019a6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:40:19.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:18.847+0000 7f8ce8000640 1 --2- 172.21.15.5:0/1581682871 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8ce0101de0 0x7f8ce019a6b0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f8cd400e9f0 tx=0x7f8cd400eeb0 comp rx=0 tx=0).stop 2024-09-10T09:40:19.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:18.847+0000 7f8ce8000640 1 -- 172.21.15.5:0/1581682871 shutdown_connections 2024-09-10T09:40:19.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:18.847+0000 7f8ce8000640 1 --2- 172.21.15.5:0/1581682871 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f8cc80a2b80 0x7f8cc80a5040 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:19.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:18.847+0000 7f8ce8000640 1 --2- 172.21.15.5:0/1581682871 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8ce0106f10 0x7f8ce019ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:19.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:18.847+0000 7f8ce8000640 1 --2- 172.21.15.5:0/1581682871 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8ce0102790 0x7f8ce019abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:19.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:18.847+0000 7f8ce8000640 1 --2- 172.21.15.5:0/1581682871 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8ce0101de0 0x7f8ce019a6b0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:19.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:18.847+0000 7f8ce8000640 1 -- 172.21.15.5:0/1581682871 >> 172.21.15.5:0/1581682871 conn(0x7f8ce00fd8c0 msgr2=0x7f8ce00febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:40:19.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:18.847+0000 7f8ce8000640 1 -- 172.21.15.5:0/1581682871 shutdown_connections 2024-09-10T09:40:19.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:18.847+0000 7f8ce8000640 1 -- 172.21.15.5:0/1581682871 wait complete. 2024-09-10T09:40:19.003 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:40:18 smithi159 ceph-mon[35513]: pgmap v452: 129 pgs: 129 active+clean; 666 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-09-10T09:40:19.003 INFO:tasks.workunit.client.0.smithi005.stdout:1 of 1 processes prepared for launch 0 sec 2024-09-10T09:40:19.003 INFO:tasks.workunit.client.0.smithi005.stdout:releasing clients 2024-09-10T09:40:19.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:40:18 smithi102 ceph-mon[35442]: pgmap v452: 129 pgs: 129 active+clean; 666 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-09-10T09:40:19.984 INFO:tasks.workunit.client.0.smithi005.stdout: 1 3339 35.38 MB/sec warmup 1 sec latency 6.848 ms 2024-09-10T09:40:20.000 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'f9cf197c', total seconds elapsed: 34.4 2024-09-10T09:40:20.984 INFO:tasks.workunit.client.0.smithi005.stdout: 1 6324 25.54 MB/sec warmup 2 sec latency 14.962 ms 2024-09-10T09:40:21.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:40:20 smithi005 ceph-mon[30350]: pgmap v453: 129 pgs: 129 active+clean; 820 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 15 KiB/s wr, 10 op/s 2024-09-10T09:40:21.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:40:20 smithi159 ceph-mon[35513]: pgmap v453: 129 pgs: 129 active+clean; 820 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 15 KiB/s wr, 10 op/s 2024-09-10T09:40:21.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:40:20 smithi102 ceph-mon[35442]: pgmap v453: 129 pgs: 129 active+clean; 820 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 15 KiB/s wr, 10 op/s 2024-09-10T09:40:21.984 INFO:tasks.workunit.client.0.smithi005.stdout: 1 9244 21.64 MB/sec warmup 3 sec latency 10.659 ms 2024-09-10T09:40:22.984 INFO:tasks.workunit.client.0.smithi005.stdout: 1 11641 19.82 MB/sec warmup 4 sec latency 56.986 ms 2024-09-10T09:40:23.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:40:22 smithi005 ceph-mon[30350]: pgmap v454: 129 pgs: 129 active+clean; 1.6 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 97 KiB/s wr, 23 op/s 2024-09-10T09:40:23.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:40:22 smithi159 ceph-mon[35513]: pgmap v454: 129 pgs: 129 active+clean; 1.6 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 97 KiB/s wr, 23 op/s 2024-09-10T09:40:23.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:40:22 smithi102 ceph-mon[35442]: pgmap v454: 129 pgs: 129 active+clean; 1.6 MiB data, 380 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 97 KiB/s wr, 23 op/s 2024-09-10T09:40:23.984 INFO:tasks.workunit.client.0.smithi005.stdout: 1 14014 18.56 MB/sec warmup 5 sec latency 293.130 ms 2024-09-10T09:40:24.035 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:40:23 smithi102 ceph-mon[35442]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T09:40:24.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:40:23 smithi005 ceph-mon[30350]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T09:40:24.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:40:23 smithi159 ceph-mon[35513]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T09:40:24.984 INFO:tasks.workunit.client.0.smithi005.stdout: 1 14014 15.47 MB/sec warmup 6 sec latency 1293.282 ms 2024-09-10T09:40:25.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:40:24 smithi005 ceph-mon[30350]: pgmap v455: 129 pgs: 129 active+clean; 3.2 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 281 KiB/s wr, 37 op/s 2024-09-10T09:40:25.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:40:24 smithi159 ceph-mon[35513]: pgmap v455: 129 pgs: 129 active+clean; 3.2 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 281 KiB/s wr, 37 op/s 2024-09-10T09:40:25.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:40:24 smithi102 ceph-mon[35442]: pgmap v455: 129 pgs: 129 active+clean; 3.2 MiB data, 387 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 281 KiB/s wr, 37 op/s 2024-09-10T09:40:25.984 INFO:tasks.workunit.client.0.smithi005.stdout: 1 14014 13.26 MB/sec warmup 7 sec latency 2293.428 ms 2024-09-10T09:40:26.984 INFO:tasks.workunit.client.0.smithi005.stdout: 1 14027 11.60 MB/sec warmup 8 sec latency 2684.972 ms 2024-09-10T09:40:27.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:40:26 smithi005 ceph-mon[30350]: pgmap v456: 129 pgs: 129 active+clean; 7.0 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 297 KiB/s rd, 661 KiB/s wr, 92 op/s 2024-09-10T09:40:27.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:40:26 smithi159 ceph-mon[35513]: pgmap v456: 129 pgs: 129 active+clean; 7.0 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 297 KiB/s rd, 661 KiB/s wr, 92 op/s 2024-09-10T09:40:27.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:40:26 smithi102 ceph-mon[35442]: pgmap v456: 129 pgs: 129 active+clean; 7.0 MiB data, 399 MiB used, 1.0 TiB / 1.0 TiB avail; 297 KiB/s rd, 661 KiB/s wr, 92 op/s 2024-09-10T09:40:27.984 INFO:tasks.workunit.client.0.smithi005.stdout: 1 14027 10.31 MB/sec warmup 9 sec latency 1596.089 ms 2024-09-10T09:40:28.985 INFO:tasks.workunit.client.0.smithi005.stdout: 1 14027 9.28 MB/sec warmup 10 sec latency 2596.234 ms 2024-09-10T09:40:29.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:40:28 smithi005 ceph-mon[30350]: pgmap v457: 129 pgs: 129 active+clean; 7.2 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 295 KiB/s rd, 695 KiB/s wr, 91 op/s 2024-09-10T09:40:29.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:40:28 smithi005 ceph-mon[30350]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T09:40:29.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:40:28 smithi159 ceph-mon[35513]: pgmap v457: 129 pgs: 129 active+clean; 7.2 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 295 KiB/s rd, 695 KiB/s wr, 91 op/s 2024-09-10T09:40:29.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:40:28 smithi159 ceph-mon[35513]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T09:40:29.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:40:28 smithi102 ceph-mon[35442]: pgmap v457: 129 pgs: 129 active+clean; 7.2 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 295 KiB/s rd, 695 KiB/s wr, 91 op/s 2024-09-10T09:40:29.135 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:40:28 smithi102 ceph-mon[35442]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T09:40:29.985 INFO:tasks.workunit.client.0.smithi005.stdout: 1 14027 8.44 MB/sec warmup 11 sec latency 3596.382 ms 2024-09-10T09:40:30.985 INFO:tasks.workunit.client.0.smithi005.stdout: 1 14027 7.73 MB/sec warmup 12 sec latency 4596.485 ms 2024-09-10T09:40:31.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:40:30 smithi005 ceph-mon[30350]: pgmap v458: 129 pgs: 129 active+clean; 8.0 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 329 KiB/s rd, 771 KiB/s wr, 95 op/s 2024-09-10T09:40:31.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:40:30 smithi159 ceph-mon[35513]: pgmap v458: 129 pgs: 129 active+clean; 8.0 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 329 KiB/s rd, 771 KiB/s wr, 95 op/s 2024-09-10T09:40:31.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:40:30 smithi102 ceph-mon[35442]: pgmap v458: 129 pgs: 129 active+clean; 8.0 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 329 KiB/s rd, 771 KiB/s wr, 95 op/s 2024-09-10T09:40:31.985 INFO:tasks.workunit.client.0.smithi005.stdout: 1 14879 7.51 MB/sec warmup 13 sec latency 4991.413 ms 2024-09-10T09:40:32.985 INFO:tasks.workunit.client.0.smithi005.stdout: 1 17701 8.20 MB/sec warmup 14 sec latency 8.930 ms 2024-09-10T09:40:33.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:40:32 smithi005 ceph-mon[30350]: pgmap v459: 129 pgs: 129 active+clean; 17 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 353 KiB/s rd, 1.5 MiB/s wr, 100 op/s 2024-09-10T09:40:33.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:40:32 smithi159 ceph-mon[35513]: pgmap v459: 129 pgs: 129 active+clean; 17 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 353 KiB/s rd, 1.5 MiB/s wr, 100 op/s 2024-09-10T09:40:33.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:40:32 smithi102 ceph-mon[35442]: pgmap v459: 129 pgs: 129 active+clean; 17 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 353 KiB/s rd, 1.5 MiB/s wr, 100 op/s 2024-09-10T09:40:33.985 INFO:tasks.workunit.client.0.smithi005.stdout: 1 19366 8.12 MB/sec warmup 15 sec latency 23.811 ms 2024-09-10T09:40:34.992 INFO:tasks.workunit.client.0.smithi005.stdout: 1 21004 8.12 MB/sec warmup 16 sec latency 221.088 ms 2024-09-10T09:40:35.015 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:40:34 smithi005 ceph-mon[30350]: pgmap v460: 129 pgs: 129 active+clean; 24 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 525 KiB/s rd, 2.0 MiB/s wr, 115 op/s 2024-09-10T09:40:35.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:40:34 smithi159 ceph-mon[35513]: pgmap v460: 129 pgs: 129 active+clean; 24 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 525 KiB/s rd, 2.0 MiB/s wr, 115 op/s 2024-09-10T09:40:35.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:40:34 smithi102 ceph-mon[35442]: pgmap v460: 129 pgs: 129 active+clean; 24 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 525 KiB/s rd, 2.0 MiB/s wr, 115 op/s 2024-09-10T09:40:35.990 INFO:tasks.workunit.client.0.smithi005.stdout: 1 21004 7.64 MB/sec warmup 17 sec latency 1188.970 ms 2024-09-10T09:40:36.990 INFO:tasks.workunit.client.0.smithi005.stdout: 1 22445 7.65 MB/sec warmup 18 sec latency 1311.768 ms 2024-09-10T09:40:37.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:40:36 smithi005 ceph-mon[30350]: pgmap v461: 129 pgs: 129 active+clean; 32 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 757 KiB/s rd, 2.6 MiB/s wr, 122 op/s 2024-09-10T09:40:37.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:40:36 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:40:37.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:40:36 smithi159 ceph-mon[35513]: pgmap v461: 129 pgs: 129 active+clean; 32 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 757 KiB/s rd, 2.6 MiB/s wr, 122 op/s 2024-09-10T09:40:37.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:40:36 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:40:37.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:40:36 smithi102 ceph-mon[35442]: pgmap v461: 129 pgs: 129 active+clean; 32 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 757 KiB/s rd, 2.6 MiB/s wr, 122 op/s 2024-09-10T09:40:37.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:40:36 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:40:37.990 INFO:tasks.workunit.client.0.smithi005.stdout: 1 25241 8.15 MB/sec warmup 19 sec latency 8.671 ms 2024-09-10T09:40:38.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:40:37 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:40:38.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:40:37 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:40:38.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:40:37 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:40:38.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:40:37 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:40:38.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:40:37 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:40:38.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:40:37 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:40:38.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:40:37 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:40:38.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:40:37 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:40:38.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:40:37 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:40:38.990 INFO:tasks.workunit.client.0.smithi005.stdout: 1 27767 8.33 MB/sec warmup 20 sec latency 21.571 ms 2024-09-10T09:40:39.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:40:39 smithi159 ceph-mon[35513]: pgmap v462: 129 pgs: 129 active+clean; 33 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 667 KiB/s rd, 2.6 MiB/s wr, 78 op/s 2024-09-10T09:40:39.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:40:39 smithi102 ceph-mon[35442]: pgmap v462: 129 pgs: 129 active+clean; 33 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 667 KiB/s rd, 2.6 MiB/s wr, 78 op/s 2024-09-10T09:40:39.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:40:39 smithi005 ceph-mon[30350]: pgmap v462: 129 pgs: 129 active+clean; 33 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 667 KiB/s rd, 2.6 MiB/s wr, 78 op/s 2024-09-10T09:40:39.991 INFO:tasks.workunit.client.0.smithi005.stdout: 1 29853 8.52 MB/sec warmup 21 sec latency 23.821 ms 2024-09-10T09:40:40.577 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T09:40:40.991 INFO:tasks.workunit.client.0.smithi005.stdout: 1 32405 8.77 MB/sec warmup 22 sec latency 13.535 ms 2024-09-10T09:40:41.011 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:40:41.477 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:40:41 smithi005 ceph-mon[30350]: pgmap v463: 129 pgs: 129 active+clean; 35 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.1 MiB/s wr, 138 op/s 2024-09-10T09:40:41.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:40:41 smithi159 ceph-mon[35513]: pgmap v463: 129 pgs: 129 active+clean; 35 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.1 MiB/s wr, 138 op/s 2024-09-10T09:40:41.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:41.601+0000 7f8d80e00640 1 -- 172.21.15.5:0/3605250283 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8d7c101270 msgr2=0x7f8d7c10d7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:40:41.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:41.601+0000 7f8d80e00640 1 --2- 172.21.15.5:0/3605250283 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8d7c101270 0x7f8d7c10d7a0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f8d74009be0 tx=0x7f8d7402a6c0 comp rx=0 tx=0).stop 2024-09-10T09:40:41.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:41.601+0000 7f8d80e00640 1 -- 172.21.15.5:0/3605250283 shutdown_connections 2024-09-10T09:40:41.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:41.601+0000 7f8d80e00640 1 --2- 172.21.15.5:0/3605250283 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8d7c101270 0x7f8d7c10d7a0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:41.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:41.601+0000 7f8d80e00640 1 --2- 172.21.15.5:0/3605250283 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8d7c1008d0 0x7f8d7c100d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:41.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:41.601+0000 7f8d80e00640 1 --2- 172.21.15.5:0/3605250283 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8d7c1068d0 0x7f8d7c106cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:41.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:41.601+0000 7f8d80e00640 1 -- 172.21.15.5:0/3605250283 >> 172.21.15.5:0/3605250283 conn(0x7f8d7c0fc630 msgr2=0x7f8d7c0fea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:40:41.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:41.601+0000 7f8d80e00640 1 -- 172.21.15.5:0/3605250283 shutdown_connections 2024-09-10T09:40:41.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:41.601+0000 7f8d80e00640 1 -- 172.21.15.5:0/3605250283 wait complete. 2024-09-10T09:40:41.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:41.603+0000 7f8d80e00640 1 Processor -- start 2024-09-10T09:40:41.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:41.603+0000 7f8d80e00640 1 -- start start 2024-09-10T09:40:41.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:41.603+0000 7f8d80e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8d7c1008d0 0x7f8d7c19a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:40:41.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:41.603+0000 7f8d80e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8d7c101270 0x7f8d7c19a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:40:41.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:41.603+0000 7f8d80e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8d7c1068d0 0x7f8d7c19ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:40:41.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:41.603+0000 7f8d80e00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f8d7c110c80 con 0x7f8d7c1068d0 2024-09-10T09:40:41.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:41.603+0000 7f8d80e00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f8d7c110b00 con 0x7f8d7c1008d0 2024-09-10T09:40:41.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:41.603+0000 7f8d80e00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f8d7c110e00 con 0x7f8d7c101270 2024-09-10T09:40:41.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:41.604+0000 7f8d7b400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8d7c1008d0 0x7f8d7c19a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:40:41.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:41.604+0000 7f8d7b400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8d7c1008d0 0x7f8d7c19a480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:37548/0 (socket says 172.21.15.5:37548) 2024-09-10T09:40:41.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:41.604+0000 7f8d7b400640 1 -- 172.21.15.5:0/2938382943 learned_addr learned my addr 172.21.15.5:0/2938382943 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:40:41.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:41.604+0000 7f8d7aa00640 1 --2- 172.21.15.5:0/2938382943 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8d7c101270 0x7f8d7c19a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:40:41.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:41.604+0000 7f8d7be00640 1 --2- 172.21.15.5:0/2938382943 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8d7c1068d0 0x7f8d7c19ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:40:41.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:41.604+0000 7f8d7b400640 1 -- 172.21.15.5:0/2938382943 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8d7c101270 msgr2=0x7f8d7c19a9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:40:41.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:41.604+0000 7f8d7b400640 1 --2- 172.21.15.5:0/2938382943 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8d7c101270 0x7f8d7c19a9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:41.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:41.604+0000 7f8d7b400640 1 -- 172.21.15.5:0/2938382943 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8d7c1068d0 msgr2=0x7f8d7c19ed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:40:41.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:41.604+0000 7f8d7b400640 1 --2- 172.21.15.5:0/2938382943 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8d7c1068d0 0x7f8d7c19ed50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:41.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:41.604+0000 7f8d7b400640 1 -- 172.21.15.5:0/2938382943 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d7c19f430 con 0x7f8d7c1008d0 2024-09-10T09:40:41.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:41.604+0000 7f8d7b400640 1 --2- 172.21.15.5:0/2938382943 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8d7c1008d0 0x7f8d7c19a480 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f8d6c00e9c0 tx=0x7f8d6c00ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:40:41.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:41.605+0000 7f8d73e00640 1 -- 172.21.15.5:0/2938382943 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d6c00ccc0 con 0x7f8d7c1008d0 2024-09-10T09:40:41.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:41.605+0000 7f8d73e00640 1 -- 172.21.15.5:0/2938382943 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8d6c005500 con 0x7f8d7c1008d0 2024-09-10T09:40:41.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:41.605+0000 7f8d73e00640 1 -- 172.21.15.5:0/2938382943 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d6c010430 con 0x7f8d7c1008d0 2024-09-10T09:40:41.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:41.605+0000 7f8d80e00640 1 -- 172.21.15.5:0/2938382943 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d7c19f720 con 0x7f8d7c1008d0 2024-09-10T09:40:41.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:41.605+0000 7f8d80e00640 1 -- 172.21.15.5:0/2938382943 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d7c1a7000 con 0x7f8d7c1008d0 2024-09-10T09:40:41.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:41.609+0000 7f8d73e00640 1 -- 172.21.15.5:0/2938382943 <== mon.2 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f8d6c0046c0 con 0x7f8d7c1008d0 2024-09-10T09:40:41.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:41.609+0000 7f8d84e1d740 1 Processor -- start 2024-09-10T09:40:41.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:41.610+0000 7f8d84e1d740 1 -- start start 2024-09-10T09:40:41.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:41.610+0000 7f8d84e1d740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5607f6e78d50 0x5607f6e79130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:40:41.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:41.610+0000 7f8d84e1d740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5607f6e79670 0x5607f6e839f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:40:41.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:41.610+0000 7f8d84e1d740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5607f6e83f30 0x5607f6e86320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:40:41.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:41.610+0000 7f8d84e1d740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5607f6ddf3e0 con 0x5607f6e78d50 2024-09-10T09:40:41.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:41.610+0000 7f8d84e1d740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x5607f6d86d80 con 0x5607f6e83f30 2024-09-10T09:40:41.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:41.610+0000 7f8d84e1d740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5607f6ddb3a0 con 0x5607f6e79670 2024-09-10T09:40:41.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:41.610+0000 7f8d7b400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5607f6e79670 0x5607f6e839f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:40:41.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:41.610+0000 7f8d7be00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5607f6e78d50 0x5607f6e79130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:40:41.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:41.610+0000 7f8d7aa00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5607f6e83f30 0x5607f6e86320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:40:41.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:41.610+0000 7f8d7b400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5607f6e79670 0x5607f6e839f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:44792/0 (socket says 172.21.15.5:44792) 2024-09-10T09:40:41.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:41.610+0000 7f8d7aa00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5607f6e83f30 0x5607f6e86320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:37564/0 (socket says 172.21.15.5:37564) 2024-09-10T09:40:41.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:41.610+0000 7f8d7b400640 1 -- 172.21.15.5:0/2120447306 learned_addr learned my addr 172.21.15.5:0/2120447306 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:40:41.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:41.611+0000 7f8d7b400640 1 -- 172.21.15.5:0/2120447306 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5607f6e83f30 msgr2=0x5607f6e86320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:40:41.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:41.611+0000 7f8d7b400640 1 --2- 172.21.15.5:0/2120447306 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5607f6e83f30 0x5607f6e86320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:41.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:41.611+0000 7f8d7b400640 1 -- 172.21.15.5:0/2120447306 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5607f6e78d50 msgr2=0x5607f6e79130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:40:41.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:41.611+0000 7f8d7b400640 1 --2- 172.21.15.5:0/2120447306 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5607f6e78d50 0x5607f6e79130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:41.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:41.611+0000 7f8d7b400640 1 -- 172.21.15.5:0/2120447306 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5607f6da5ae0 con 0x5607f6e79670 2024-09-10T09:40:41.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:41.611+0000 7f8d7b400640 1 --2- 172.21.15.5:0/2120447306 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5607f6e79670 0x5607f6e839f0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f8d6c0d55d0 tx=0x7f8d6c0d56c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1c941b718cdb6246 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:40:41.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:41.611+0000 7f8d73e00640 1 --2- 172.21.15.5:0/2938382943 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f8d600a2c00 0x7f8d600a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:40:41.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:41.611+0000 7f8d70c00640 1 -- 172.21.15.5:0/2120447306 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d6c0e3070 con 0x5607f6e79670 2024-09-10T09:40:41.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:41.612+0000 7f8d70c00640 1 -- 172.21.15.5:0/2120447306 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8d6c087cd0 con 0x5607f6e79670 2024-09-10T09:40:41.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:41.612+0000 7f8d73e00640 1 -- 172.21.15.5:0/2938382943 <== mon.2 v2:172.21.15.102:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f8d6c0cf5f0 con 0x7f8d7c1008d0 2024-09-10T09:40:41.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:41.612+0000 7f8d7be00640 1 --2- 172.21.15.5:0/2938382943 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f8d600a2c00 0x7f8d600a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:40:41.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:41.613+0000 7f8d70c00640 1 -- 172.21.15.5:0/2120447306 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d6c09b830 con 0x5607f6e79670 2024-09-10T09:40:41.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:41.613+0000 7f8d84e1d740 1 -- 172.21.15.5:0/2120447306 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5607f6e79670 msgr2=0x5607f6e839f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:40:41.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:41.613+0000 7f8d84e1d740 1 --2- 172.21.15.5:0/2120447306 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5607f6e79670 0x5607f6e839f0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f8d6c0d55d0 tx=0x7f8d6c0d56c0 comp rx=0 tx=0).stop 2024-09-10T09:40:41.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:41.613+0000 7f8d7be00640 1 --2- 172.21.15.5:0/2938382943 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f8d600a2c00 0x7f8d600a50c0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f8d74009370 tx=0x7f8d740098a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:40:41.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:41.613+0000 7f8d84e1d740 1 -- 172.21.15.5:0/2120447306 shutdown_connections 2024-09-10T09:40:41.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:41.613+0000 7f8d84e1d740 1 --2- 172.21.15.5:0/2120447306 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5607f6e83f30 0x5607f6e86320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:41.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:41.613+0000 7f8d84e1d740 1 --2- 172.21.15.5:0/2120447306 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5607f6e79670 0x5607f6e839f0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:41.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:41.613+0000 7f8d84e1d740 1 --2- 172.21.15.5:0/2120447306 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5607f6e78d50 0x5607f6e79130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:41.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:41.613+0000 7f8d84e1d740 1 -- 172.21.15.5:0/2120447306 >> 172.21.15.5:0/2120447306 conn(0x5607f6e752e0 msgr2=0x5607f6e756c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:40:41.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:41.613+0000 7f8d84e1d740 1 -- 172.21.15.5:0/2120447306 shutdown_connections 2024-09-10T09:40:41.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:41.613+0000 7f8d84e1d740 1 -- 172.21.15.5:0/2120447306 wait complete. 2024-09-10T09:40:41.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:41.614+0000 7f8d84e1d740 1 -- start start 2024-09-10T09:40:41.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:41.614+0000 7f8d84e1d740 1 Processor -- start 2024-09-10T09:40:41.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:41.614+0000 7f8d84e1d740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5607f6e78d50 0x5607f6e80d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:40:41.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:41.614+0000 7f8d84e1d740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5607f6e79670 0x5607f6e812b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:40:41.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:41.614+0000 7f8d84e1d740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5607f6e83f30 0x5607f6e817f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:40:41.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:41.615+0000 7f8d84e1d740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5607f6e86bf0 con 0x5607f6e79670 2024-09-10T09:40:41.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:41.615+0000 7f8d84e1d740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x5607f6d9a0f0 con 0x5607f6e83f30 2024-09-10T09:40:41.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:41.615+0000 7f8d84e1d740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5607f6e86d70 con 0x5607f6e78d50 2024-09-10T09:40:41.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:41.615+0000 7f8d84e1d740 10 client.0 _ll_register_callbacks cb 0x5607f6d8b9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:40:41.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:41.615+0000 7f8d7be00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5607f6e83f30 0x5607f6e817f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:40:41.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:41.615+0000 7f8d7b400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5607f6e78d50 0x5607f6e80d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:40:41.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:41.615+0000 7f8d7be00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5607f6e83f30 0x5607f6e817f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:37566/0 (socket says 172.21.15.5:37566) 2024-09-10T09:40:41.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:41.615+0000 7f8d7be00640 1 -- 172.21.15.5:0/2773833744 learned_addr learned my addr 172.21.15.5:0/2773833744 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:40:41.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:41.615+0000 7f8d7aa00640 1 --2- 172.21.15.5:0/2773833744 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5607f6e79670 0x5607f6e812b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:40:41.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:41.615+0000 7f8d7b400640 1 -- 172.21.15.5:0/2773833744 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5607f6e83f30 msgr2=0x5607f6e817f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:40:41.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:41.616+0000 7f8d7b400640 1 --2- 172.21.15.5:0/2773833744 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5607f6e83f30 0x5607f6e817f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:41.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:41.616+0000 7f8d7b400640 1 -- 172.21.15.5:0/2773833744 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5607f6e79670 msgr2=0x5607f6e812b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:40:41.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:41.616+0000 7f8d7b400640 1 --2- 172.21.15.5:0/2773833744 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5607f6e79670 0x5607f6e812b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:41.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:41.616+0000 7f8d7b400640 1 -- 172.21.15.5:0/2773833744 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5607f6b503c0 con 0x5607f6e78d50 2024-09-10T09:40:41.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:41.616+0000 7f8d7be00640 1 --2- 172.21.15.5:0/2773833744 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5607f6e83f30 0x5607f6e817f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:40:41.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:41.616+0000 7f8d7aa00640 1 --2- 172.21.15.5:0/2773833744 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5607f6e79670 0x5607f6e812b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:40:41.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:41.616+0000 7f8d7b400640 1 --2- 172.21.15.5:0/2773833744 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5607f6e78d50 0x5607f6e80d70 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x5607f6e76140 tx=0x7f8d6c00af70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=56b85984671a0d5a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:40:41.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:41.616+0000 7f8d65600640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-09-10T09:40:41.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:41.616+0000 7f8d65600640 1 -- 172.21.15.5:0/2773833744 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d6c008410 con 0x5607f6e78d50 2024-09-10T09:40:41.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:41.616+0000 7f8d65600640 1 -- 172.21.15.5:0/2773833744 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f8d6c099d60 con 0x5607f6e78d50 2024-09-10T09:40:41.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:41.616+0000 7f8d84e1d740 1 -- 172.21.15.5:0/2773833744 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8d6c087cd0 con 0x5607f6e78d50 2024-09-10T09:40:41.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:41.617+0000 7f8d65600640 1 -- 172.21.15.5:0/2773833744 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f8d6c008600 con 0x5607f6e78d50 2024-09-10T09:40:41.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:41.617+0000 7f8d84e1d740 10 client.25864 fetch_fsmap learned FSMap version 32 2024-09-10T09:40:41.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:41.617+0000 7f8d84e1d740 1 -- 172.21.15.5:0/2773833744 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5607f6e86bf0 con 0x5607f6e78d50 2024-09-10T09:40:41.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:41.617+0000 7f8d65600640 1 -- 172.21.15.5:0/2773833744 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8d6c087cd0 con 0x5607f6e78d50 2024-09-10T09:40:41.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:41.617+0000 7f8d65600640 1 -- 172.21.15.5:0/2773833744 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7f8d6c0980d0 con 0x5607f6e78d50 2024-09-10T09:40:41.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:41.617+0000 7f8d84e1d740 10 client.25864 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T09:40:41.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:41.617+0000 7f8d84e1d740 10 client.25864 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T09:40:41.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:41.617+0000 7f8d84e1d740 20 client.25864 populate_metadata read hostname 'smithi005' 2024-09-10T09:40:41.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:41.617+0000 7f8d84e1d740 1 --2- 172.21.15.5:0/2773833744 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x5607f6e9c1d0 0x5607f6e9e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:40:41.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:41.617+0000 7f8d84e1d740 4 client.25864 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:40:41.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:41.617+0000 7f8d84e1d740 1 -- 172.21.15.5:0/2773833744 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5607f6dec950 con 0x5607f6e9c1d0 2024-09-10T09:40:41.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:41.618+0000 7f8d7aa00640 1 --2- 172.21.15.5:0/2773833744 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x5607f6e9c1d0 0x5607f6e9e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:40:41.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:41.618+0000 7f8d7aa00640 1 --2- 172.21.15.5:0/2773833744 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x5607f6e9c1d0 0x5607f6e9e5c0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1123ca4544da5bd3 server_cookie=83a6cb69684fd9ef in_seq=0 out_seq=0 2024-09-10T09:40:41.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:41.618+0000 7f8d65600640 10 client.25864 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T09:40:41.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:41.619+0000 7f8d65600640 1 -- 172.21.15.5:0/2773833744 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5607f6dec950 con 0x5607f6e9c1d0 2024-09-10T09:40:41.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:41.619+0000 7f8d65600640 10 client.25864 handle_command_reply: tid=0 2024-09-10T09:40:41.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:41.619+0000 7f8d84e1d740 1 client.25864 shutdown 2024-09-10T09:40:41.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:41.619+0000 7f8d84e1d740 1 -- 172.21.15.5:0/2773833744 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5607f6e78d50 msgr2=0x5607f6e80d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:40:41.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:41.619+0000 7f8d84e1d740 1 --2- 172.21.15.5:0/2773833744 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5607f6e78d50 0x5607f6e80d70 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x5607f6e76140 tx=0x7f8d6c00af70 comp rx=0 tx=0).stop 2024-09-10T09:40:41.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:41.619+0000 7f8d84e1d740 1 -- 172.21.15.5:0/2773833744 shutdown_connections 2024-09-10T09:40:41.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:41.619+0000 7f8d84e1d740 1 --2- 172.21.15.5:0/2773833744 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x5607f6e9c1d0 0x5607f6e9e5c0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:41.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:41.619+0000 7f8d84e1d740 1 --2- 172.21.15.5:0/2773833744 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5607f6e83f30 0x5607f6e817f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:41.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:41.619+0000 7f8d84e1d740 1 --2- 172.21.15.5:0/2773833744 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5607f6e79670 0x5607f6e812b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:41.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:41.619+0000 7f8d84e1d740 1 --2- 172.21.15.5:0/2773833744 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5607f6e78d50 0x5607f6e80d70 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:41.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:41.619+0000 7f8d65600640 0 client.25864 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T09:40:41.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:41.619+0000 7f8d84e1d740 1 -- 172.21.15.5:0/2773833744 >> 172.21.15.5:0/2773833744 conn(0x5607f6e752e0 msgr2=0x5607f6e7afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:40:41.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:41.619+0000 7f8d84e1d740 1 -- 172.21.15.5:0/2773833744 shutdown_connections 2024-09-10T09:40:41.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:41.619+0000 7f8d84e1d740 1 -- 172.21.15.5:0/2773833744 wait complete. 2024-09-10T09:40:41.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:41.620+0000 7f8d84e1d740 20 client.25864 trim_cache size 0 max 16384 2024-09-10T09:40:41.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:40:41 smithi102 ceph-mon[35442]: pgmap v463: 129 pgs: 129 active+clean; 35 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.1 MiB/s wr, 138 op/s 2024-09-10T09:40:41.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:41.638+0000 7f8d84e1d740 1 Processor -- start 2024-09-10T09:40:41.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:41.639+0000 7f8d84e1d740 1 -- start start 2024-09-10T09:40:41.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:41.639+0000 7f8d84e1d740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5607f6e78d50 0x5607f6e82ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:40:41.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:41.639+0000 7f8d84e1d740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5607f6e79670 0x5607f6e81640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:40:41.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:41.639+0000 7f8d84e1d740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5607f6e83f30 0x5607f6ea3cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:40:41.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:41.639+0000 7f8d84e1d740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f8d7c1acb40 con 0x5607f6e83f30 2024-09-10T09:40:41.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:41.639+0000 7f8d84e1d740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f8d7c1ae740 con 0x5607f6e79670 2024-09-10T09:40:41.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:41.639+0000 7f8d84e1d740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f8d7c1ad210 con 0x5607f6e78d50 2024-09-10T09:40:41.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:41.639+0000 7f8d7b400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5607f6e78d50 0x5607f6e82ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:40:41.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:41.639+0000 7f8d7b400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5607f6e78d50 0x5607f6e82ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:44814/0 (socket says 172.21.15.5:44814) 2024-09-10T09:40:41.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:41.639+0000 7f8d7b400640 1 -- 172.21.15.5:0/1253613656 learned_addr learned my addr 172.21.15.5:0/1253613656 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:40:41.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:41.640+0000 7f8d7aa00640 1 --2- 172.21.15.5:0/1253613656 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5607f6e79670 0x5607f6e81640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:40:41.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:41.640+0000 7f8d7be00640 1 --2- 172.21.15.5:0/1253613656 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5607f6e83f30 0x5607f6ea3cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:40:41.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:41.640+0000 7f8d7b400640 1 -- 172.21.15.5:0/1253613656 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5607f6e79670 msgr2=0x5607f6e81640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:40:41.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:41.640+0000 7f8d7b400640 1 --2- 172.21.15.5:0/1253613656 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5607f6e79670 0x5607f6e81640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:41.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:41.640+0000 7f8d7b400640 1 -- 172.21.15.5:0/1253613656 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5607f6e83f30 msgr2=0x5607f6ea3cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:40:41.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:41.640+0000 7f8d7b400640 1 --2- 172.21.15.5:0/1253613656 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5607f6e83f30 0x5607f6ea3cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:41.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:41.640+0000 7f8d7b400640 1 -- 172.21.15.5:0/1253613656 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5607f6e79ef0 con 0x5607f6e78d50 2024-09-10T09:40:41.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:41.640+0000 7f8d7aa00640 1 --2- 172.21.15.5:0/1253613656 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5607f6e79670 0x5607f6e81640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:40:41.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:41.640+0000 7f8d7b400640 1 --2- 172.21.15.5:0/1253613656 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5607f6e78d50 0x5607f6e82ce0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f8d6c0d55d0 tx=0x7f8d6c0d56c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9e66728236384f3f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:40:41.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:41.640+0000 7f8d67e00640 1 -- 172.21.15.5:0/1253613656 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d6c008800 con 0x5607f6e78d50 2024-09-10T09:40:41.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:41.641+0000 7f8d67e00640 1 -- 172.21.15.5:0/1253613656 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8d6c0d5bc0 con 0x5607f6e78d50 2024-09-10T09:40:41.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:41.641+0000 7f8d67e00640 1 -- 172.21.15.5:0/1253613656 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d6c008a00 con 0x5607f6e78d50 2024-09-10T09:40:41.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:41.641+0000 7f8d84e1d740 1 -- 172.21.15.5:0/1253613656 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5607f6e78d50 msgr2=0x5607f6e82ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:40:41.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:41.641+0000 7f8d84e1d740 1 --2- 172.21.15.5:0/1253613656 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5607f6e78d50 0x5607f6e82ce0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f8d6c0d55d0 tx=0x7f8d6c0d56c0 comp rx=0 tx=0).stop 2024-09-10T09:40:41.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:41.642+0000 7f8d84e1d740 1 -- 172.21.15.5:0/1253613656 shutdown_connections 2024-09-10T09:40:41.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:41.642+0000 7f8d84e1d740 1 --2- 172.21.15.5:0/1253613656 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5607f6e83f30 0x5607f6ea3cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:41.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:41.642+0000 7f8d84e1d740 1 --2- 172.21.15.5:0/1253613656 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5607f6e79670 0x5607f6e81640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:41.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:41.642+0000 7f8d84e1d740 1 --2- 172.21.15.5:0/1253613656 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5607f6e78d50 0x5607f6e82ce0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:41.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:41.642+0000 7f8d84e1d740 1 -- 172.21.15.5:0/1253613656 >> 172.21.15.5:0/1253613656 conn(0x5607f6e752e0 msgr2=0x5607f6e825d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:40:41.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:41.642+0000 7f8d84e1d740 1 -- 172.21.15.5:0/1253613656 shutdown_connections 2024-09-10T09:40:41.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:41.642+0000 7f8d84e1d740 1 -- 172.21.15.5:0/1253613656 wait complete. 2024-09-10T09:40:41.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:41.642+0000 7f8d84e1d740 1 -- start start 2024-09-10T09:40:41.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:41.642+0000 7f8d84e1d740 1 Processor -- start 2024-09-10T09:40:41.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:41.643+0000 7f8d84e1d740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5607f6e78d50 0x5607f6e85020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:40:41.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:41.643+0000 7f8d84e1d740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5607f6e79670 0x5607f6e85560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:40:41.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:41.643+0000 7f8d84e1d740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5607f6e83f30 0x5607f6e85aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:40:41.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:41.643+0000 7f8d84e1d740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f8d7c1adfb0 con 0x5607f6e83f30 2024-09-10T09:40:41.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:41.643+0000 7f8d84e1d740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f8d7c1ad8e0 con 0x5607f6e78d50 2024-09-10T09:40:41.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:41.643+0000 7f8d84e1d740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f8d7c1aab20 con 0x5607f6e79670 2024-09-10T09:40:41.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:41.643+0000 7f8d84e1d740 10 client.0 _ll_register_callbacks cb 0x7f8d6800b770 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:40:41.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:41.643+0000 7f8d7aa00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5607f6e79670 0x5607f6e85560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:40:41.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:41.643+0000 7f8d7b400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5607f6e78d50 0x5607f6e85020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:40:41.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:41.643+0000 7f8d7aa00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5607f6e79670 0x5607f6e85560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:44828/0 (socket says 172.21.15.5:44828) 2024-09-10T09:40:41.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:41.643+0000 7f8d7aa00640 1 -- 172.21.15.5:0/4163461649 learned_addr learned my addr 172.21.15.5:0/4163461649 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:40:41.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:41.643+0000 7f8d7be00640 1 --2- 172.21.15.5:0/4163461649 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5607f6e83f30 0x5607f6e85aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:40:41.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:41.644+0000 7f8d7aa00640 1 -- 172.21.15.5:0/4163461649 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5607f6e78d50 msgr2=0x5607f6e85020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:40:41.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:41.644+0000 7f8d7aa00640 1 --2- 172.21.15.5:0/4163461649 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5607f6e78d50 0x5607f6e85020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:41.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:41.644+0000 7f8d7aa00640 1 -- 172.21.15.5:0/4163461649 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5607f6e83f30 msgr2=0x5607f6e85aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:40:41.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:41.644+0000 7f8d7aa00640 1 --2- 172.21.15.5:0/4163461649 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5607f6e83f30 0x5607f6e85aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:41.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:41.644+0000 7f8d7aa00640 1 -- 172.21.15.5:0/4163461649 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5607f6e92d80 con 0x5607f6e79670 2024-09-10T09:40:41.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:41.644+0000 7f8d7b400640 1 --2- 172.21.15.5:0/4163461649 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5607f6e78d50 0x5607f6e85020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:40:41.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:41.644+0000 7f8d7be00640 1 --2- 172.21.15.5:0/4163461649 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5607f6e83f30 0x5607f6e85aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:40:41.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:41.644+0000 7f8d7aa00640 1 --2- 172.21.15.5:0/4163461649 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5607f6e79670 0x5607f6e85560 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x5607f6e76660 tx=0x7f8d6800bf40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bbf0638ed671c24e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:40:41.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:41.644+0000 7f8d65600640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-09-10T09:40:41.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:41.644+0000 7f8d65600640 1 -- 172.21.15.5:0/4163461649 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d68005220 con 0x5607f6e79670 2024-09-10T09:40:41.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:41.644+0000 7f8d84e1d740 1 -- 172.21.15.5:0/4163461649 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8d6c0d5bc0 con 0x5607f6e79670 2024-09-10T09:40:41.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:41.644+0000 7f8d65600640 1 -- 172.21.15.5:0/4163461649 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f8d680053c0 con 0x5607f6e79670 2024-09-10T09:40:41.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:41.645+0000 7f8d65600640 1 -- 172.21.15.5:0/4163461649 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f8d68031050 con 0x5607f6e79670 2024-09-10T09:40:41.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:41.645+0000 7f8d84e1d740 10 client.25870 fetch_fsmap learned FSMap version 32 2024-09-10T09:40:41.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:41.645+0000 7f8d84e1d740 1 -- 172.21.15.5:0/4163461649 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5607f6f06f10 con 0x5607f6e79670 2024-09-10T09:40:41.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:41.645+0000 7f8d65600640 1 -- 172.21.15.5:0/4163461649 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8d6c0d5bc0 con 0x5607f6e79670 2024-09-10T09:40:41.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:41.645+0000 7f8d65600640 1 -- 172.21.15.5:0/4163461649 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7f8d68026660 con 0x5607f6e79670 2024-09-10T09:40:41.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:41.645+0000 7f8d84e1d740 10 client.25870 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T09:40:41.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:41.645+0000 7f8d84e1d740 10 client.25870 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T09:40:41.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:41.645+0000 7f8d84e1d740 20 client.25870 populate_metadata read hostname 'smithi005' 2024-09-10T09:40:41.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:41.645+0000 7f8d84e1d740 1 --2- 172.21.15.5:0/4163461649 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x5607f6f077b0 0x5607f6f09bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:40:41.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:41.645+0000 7f8d84e1d740 4 client.25870 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T09:40:41.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:41.645+0000 7f8d84e1d740 1 -- 172.21.15.5:0/4163461649 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5607f6de6e60 con 0x5607f6f077b0 2024-09-10T09:40:41.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:41.646+0000 7f8d7b400640 1 --2- 172.21.15.5:0/4163461649 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x5607f6f077b0 0x5607f6f09bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:40:41.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:41.646+0000 7f8d7b400640 1 --2- 172.21.15.5:0/4163461649 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x5607f6f077b0 0x5607f6f09bd0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3762794766876147 server_cookie=a718d7c3839a219a in_seq=0 out_seq=0 2024-09-10T09:40:41.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:41.646+0000 7f8d65600640 10 client.25870 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T09:40:41.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:41.647+0000 7f8d65600640 1 -- 172.21.15.5:0/4163461649 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5607f6de6e60 con 0x5607f6f077b0 2024-09-10T09:40:41.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:41.647+0000 7f8d65600640 10 client.25870 handle_command_reply: tid=0 2024-09-10T09:40:41.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:41.647+0000 7f8d84e1d740 1 client.25870 shutdown 2024-09-10T09:40:41.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:41.647+0000 7f8d84e1d740 1 -- 172.21.15.5:0/4163461649 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5607f6e79670 msgr2=0x5607f6e85560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:40:41.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:41.647+0000 7f8d84e1d740 1 --2- 172.21.15.5:0/4163461649 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5607f6e79670 0x5607f6e85560 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x5607f6e76660 tx=0x7f8d6800bf40 comp rx=0 tx=0).stop 2024-09-10T09:40:41.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:41.647+0000 7f8d84e1d740 1 -- 172.21.15.5:0/4163461649 shutdown_connections 2024-09-10T09:40:41.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:41.647+0000 7f8d84e1d740 1 --2- 172.21.15.5:0/4163461649 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x5607f6f077b0 0x5607f6f09bd0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:41.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:41.647+0000 7f8d84e1d740 1 --2- 172.21.15.5:0/4163461649 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5607f6e83f30 0x5607f6e85aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:41.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:41.647+0000 7f8d84e1d740 1 --2- 172.21.15.5:0/4163461649 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5607f6e79670 0x5607f6e85560 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:41.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:41.647+0000 7f8d84e1d740 1 --2- 172.21.15.5:0/4163461649 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5607f6e78d50 0x5607f6e85020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:41.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:41.647+0000 7f8d65600640 0 client.25870 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T09:40:41.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:41.647+0000 7f8d84e1d740 1 -- 172.21.15.5:0/4163461649 >> 172.21.15.5:0/4163461649 conn(0x5607f6e752e0 msgr2=0x5607f6e97f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:40:41.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:41.648+0000 7f8d84e1d740 1 -- 172.21.15.5:0/4163461649 shutdown_connections 2024-09-10T09:40:41.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:41.648+0000 7f8d84e1d740 1 -- 172.21.15.5:0/4163461649 wait complete. 2024-09-10T09:40:41.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:41.648+0000 7f8d84e1d740 20 client.25870 trim_cache size 0 max 16384 2024-09-10T09:40:41.654 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-10T09:40:41.654 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "no active scrubs running", 2024-09-10T09:40:41.654 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": {} 2024-09-10T09:40:41.654 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-10T09:40:41.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:41.649+0000 7f8d80e00640 1 -- 172.21.15.5:0/2938382943 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f8d600a2c00 msgr2=0x7f8d600a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:40:41.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:41.649+0000 7f8d80e00640 1 --2- 172.21.15.5:0/2938382943 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f8d600a2c00 0x7f8d600a50c0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f8d74009370 tx=0x7f8d740098a0 comp rx=0 tx=0).stop 2024-09-10T09:40:41.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:41.649+0000 7f8d80e00640 1 -- 172.21.15.5:0/2938382943 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8d7c1008d0 msgr2=0x7f8d7c19a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:40:41.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:41.649+0000 7f8d80e00640 1 --2- 172.21.15.5:0/2938382943 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8d7c1008d0 0x7f8d7c19a480 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f8d6c00e9c0 tx=0x7f8d6c00ee80 comp rx=0 tx=0).stop 2024-09-10T09:40:41.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:41.649+0000 7f8d80e00640 1 -- 172.21.15.5:0/2938382943 shutdown_connections 2024-09-10T09:40:41.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:41.649+0000 7f8d80e00640 1 --2- 172.21.15.5:0/2938382943 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f8d600a2c00 0x7f8d600a50c0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:41.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:41.649+0000 7f8d80e00640 1 --2- 172.21.15.5:0/2938382943 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8d7c1068d0 0x7f8d7c19ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:41.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:41.649+0000 7f8d80e00640 1 --2- 172.21.15.5:0/2938382943 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8d7c101270 0x7f8d7c19a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:41.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:41.649+0000 7f8d80e00640 1 --2- 172.21.15.5:0/2938382943 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8d7c1008d0 0x7f8d7c19a480 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:41.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:41.649+0000 7f8d80e00640 1 -- 172.21.15.5:0/2938382943 >> 172.21.15.5:0/2938382943 conn(0x7f8d7c0fc630 msgr2=0x7f8d7c10c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:40:41.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:41.649+0000 7f8d80e00640 1 -- 172.21.15.5:0/2938382943 shutdown_connections 2024-09-10T09:40:41.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:41.649+0000 7f8d80e00640 1 -- 172.21.15.5:0/2938382943 wait complete. 2024-09-10T09:40:41.991 INFO:tasks.workunit.client.0.smithi005.stdout: 1 34896 8.94 MB/sec warmup 23 sec latency 24.855 ms 2024-09-10T09:40:42.566 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-10T09:40:42.566 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T09:40:42.991 INFO:tasks.workunit.client.0.smithi005.stdout: 1 36766 8.93 MB/sec warmup 24 sec latency 20.063 ms 2024-09-10T09:40:43.018 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:40:43.489 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:40:43 smithi005 ceph-mon[30350]: pgmap v464: 129 pgs: 129 active+clean; 33 MiB data, 452 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.3 MiB/s wr, 180 op/s 2024-09-10T09:40:43.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:40:43 smithi159 ceph-mon[35513]: pgmap v464: 129 pgs: 129 active+clean; 33 MiB data, 452 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.3 MiB/s wr, 180 op/s 2024-09-10T09:40:43.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:43.614+0000 7fe7a7000640 1 -- 172.21.15.5:0/4101113893 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe7a0074ee0 msgr2=0x7fe7a010dd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:40:43.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:43.614+0000 7fe7a7000640 1 --2- 172.21.15.5:0/4101113893 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe7a0074ee0 0x7fe7a010dd20 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fe78c009900 tx=0x7fe78c02a480 comp rx=0 tx=0).stop 2024-09-10T09:40:43.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:43.615+0000 7fe7a7000640 1 -- 172.21.15.5:0/4101113893 shutdown_connections 2024-09-10T09:40:43.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:43.615+0000 7fe7a7000640 1 --2- 172.21.15.5:0/4101113893 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe7a010e260 0x7fe7a0110650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:43.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:43.615+0000 7fe7a7000640 1 --2- 172.21.15.5:0/4101113893 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe7a0074ee0 0x7fe7a010dd20 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:43.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:43.615+0000 7fe7a7000640 1 --2- 172.21.15.5:0/4101113893 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe7a00745c0 0x7fe7a00749a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:43.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:43.615+0000 7fe7a7000640 1 -- 172.21.15.5:0/4101113893 >> 172.21.15.5:0/4101113893 conn(0x7fe7a00fd740 msgr2=0x7fe7a00ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:40:43.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:43.615+0000 7fe7a7000640 1 -- 172.21.15.5:0/4101113893 shutdown_connections 2024-09-10T09:40:43.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:43.615+0000 7fe7a7000640 1 -- 172.21.15.5:0/4101113893 wait complete. 2024-09-10T09:40:43.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:43.617+0000 7fe7a7000640 1 Processor -- start 2024-09-10T09:40:43.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:43.617+0000 7fe7a7000640 1 -- start start 2024-09-10T09:40:43.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:43.617+0000 7fe7a7000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe7a00745c0 0x7fe7a019a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:40:43.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:43.617+0000 7fe7a7000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe7a0074ee0 0x7fe7a019aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:40:43.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:43.617+0000 7fe7a7000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe7a010e260 0x7fe7a019edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:40:43.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:43.617+0000 7fe7a7000640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe7a0114000 con 0x7fe7a010e260 2024-09-10T09:40:43.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:43.617+0000 7fe7a7000640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fe7a0113e80 con 0x7fe7a0074ee0 2024-09-10T09:40:43.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:43.617+0000 7fe7a7000640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fe7a0114180 con 0x7fe7a00745c0 2024-09-10T09:40:43.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:43.618+0000 7fe7a5200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe7a0074ee0 0x7fe7a019aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:40:43.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:43.618+0000 7fe7a5200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe7a0074ee0 0x7fe7a019aa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:37610/0 (socket says 172.21.15.5:37610) 2024-09-10T09:40:43.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:43.618+0000 7fe7a5200640 1 -- 172.21.15.5:0/1758294774 learned_addr learned my addr 172.21.15.5:0/1758294774 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:40:43.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:43.618+0000 7fe7a6600640 1 --2- 172.21.15.5:0/1758294774 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe7a010e260 0x7fe7a019edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:40:43.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:43.618+0000 7fe7a5c00640 1 --2- 172.21.15.5:0/1758294774 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe7a00745c0 0x7fe7a019a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:40:43.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:43.618+0000 7fe7a5200640 1 -- 172.21.15.5:0/1758294774 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe7a00745c0 msgr2=0x7fe7a019a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:40:43.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:43.618+0000 7fe7a5200640 1 --2- 172.21.15.5:0/1758294774 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe7a00745c0 0x7fe7a019a4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:43.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:43.618+0000 7fe7a5200640 1 -- 172.21.15.5:0/1758294774 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe7a010e260 msgr2=0x7fe7a019edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:40:43.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:43.618+0000 7fe7a5200640 1 --2- 172.21.15.5:0/1758294774 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe7a010e260 0x7fe7a019edb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:43.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:43.618+0000 7fe7a5200640 1 -- 172.21.15.5:0/1758294774 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7a019f490 con 0x7fe7a0074ee0 2024-09-10T09:40:43.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:43.618+0000 7fe7a5c00640 1 --2- 172.21.15.5:0/1758294774 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe7a00745c0 0x7fe7a019a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T09:40:43.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:43.619+0000 7fe7a5200640 1 --2- 172.21.15.5:0/1758294774 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe7a0074ee0 0x7fe7a019aa20 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fe78c004770 tx=0x7fe78c003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:40:43.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:43.619+0000 7fe79e000640 1 -- 172.21.15.5:0/1758294774 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe78c03f070 con 0x7fe7a0074ee0 2024-09-10T09:40:43.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:43.619+0000 7fe7a7000640 1 -- 172.21.15.5:0/1758294774 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7a019f720 con 0x7fe7a0074ee0 2024-09-10T09:40:43.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:43.619+0000 7fe79e000640 1 -- 172.21.15.5:0/1758294774 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe78c031ca0 con 0x7fe7a0074ee0 2024-09-10T09:40:43.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:43.619+0000 7fe79e000640 1 -- 172.21.15.5:0/1758294774 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe78c035030 con 0x7fe7a0074ee0 2024-09-10T09:40:43.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:43.619+0000 7fe7a7000640 1 -- 172.21.15.5:0/1758294774 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7a01a7060 con 0x7fe7a0074ee0 2024-09-10T09:40:43.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:43.621+0000 7fe7a7000640 1 -- 172.21.15.5:0/1758294774 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe7a00749a0 con 0x7fe7a0074ee0 2024-09-10T09:40:43.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:43.622+0000 7fe79e000640 1 -- 172.21.15.5:0/1758294774 <== mon.2 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7fe78c042070 con 0x7fe7a0074ee0 2024-09-10T09:40:43.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:43.624+0000 7fe79e000640 1 --2- 172.21.15.5:0/1758294774 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fe7880a2c00 0x7fe7880a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:40:43.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:43.624+0000 7fe79e000640 1 -- 172.21.15.5:0/1758294774 <== mon.2 v2:172.21.15.102:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fe78c0ef7b0 con 0x7fe7a0074ee0 2024-09-10T09:40:43.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:43.624+0000 7fe7a5c00640 1 --2- 172.21.15.5:0/1758294774 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fe7880a2c00 0x7fe7880a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:40:43.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:43.625+0000 7fe7a5c00640 1 --2- 172.21.15.5:0/1758294774 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fe7880a2c00 0x7fe7880a50c0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fe794009b50 tx=0x7fe794005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:40:43.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:43.627+0000 7fe79e000640 1 -- 172.21.15.5:0/1758294774 <== mon.2 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe78c0bc380 con 0x7fe7a0074ee0 2024-09-10T09:40:43.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:40:43 smithi102 ceph-mon[35442]: pgmap v464: 129 pgs: 129 active+clean; 33 MiB data, 452 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.3 MiB/s wr, 180 op/s 2024-09-10T09:40:43.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:43.900+0000 7fe7a7000640 1 -- 172.21.15.5:0/1758294774 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe7a01093e0 con 0x7fe7a0074ee0 2024-09-10T09:40:43.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:43.901+0000 7fe79e000640 1 -- 172.21.15.5:0/1758294774 <== mon.2 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+11976 (secure 0 0 0) 0x7fe78c0bbc80 con 0x7fe7a0074ee0 2024-09-10T09:40:43.903 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-10T09:40:43.904 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":32,"btime":"2024-09-10T09:40:27:751081+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14454,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/2970594311","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2970594311},{"type":"v1","addr":"172.21.15.5:6835","nonce":2970594311}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14514,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6839/3616502174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":3616502174},{"type":"v1","addr":"172.21.15.5:6839","nonce":3616502174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":32,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T09:35:29.626287+0000","modified":"2024-09-10T09:40:26.959994+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24542,"mds_1":24535,"mds_2":24511,"mds_3":24506,"mds_4":24443},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14520":{"gid":14520,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/2073599842","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":2073599842},{"type":"v1","addr":"172.21.15.5:6841","nonce":2073599842}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"k","rank":4,"incarnation":26,"state":"up:active","state_seq":24,"addr":"172.21.15.102:6833/2457007427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":2457007427},{"type":"v1","addr":"172.21.15.102:6833","nonce":2457007427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/356445159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":356445159},{"type":"v1","addr":"172.21.15.159:6833","nonce":356445159}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6837/2894314507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":2894314507},{"type":"v1","addr":"172.21.15.5:6837","nonce":2894314507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.102:6835/1647549111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":1647549111},{"type":"v1","addr":"172.21.15.102:6835","nonce":1647549111}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6835/952657242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":952657242},{"type":"v1","addr":"172.21.15.159:6835","nonce":952657242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":19,"addr":"172.21.15.102:6837/2689776764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":2689776764},{"type":"v1","addr":"172.21.15.102:6837","nonce":2689776764}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.159:6837/620627639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":620627639},{"type":"v1","addr":"172.21.15.159:6837","nonce":620627639}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24535":{"gid":24535,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":18,"addr":"172.21.15.159:6839/2073739599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":2073739599},{"type":"v1","addr":"172.21.15.159:6839","nonce":2073739599}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24542":{"gid":24542,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.102:6839/335409749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6838","nonce":335409749},{"type":"v1","addr":"172.21.15.102:6839","nonce":335409749}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24542,"qdb_cluster":[24542,24535,24511,24506,24443]},"id":1}]} 2024-09-10T09:40:43.904 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 32 2024-09-10T09:40:43.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:43.904+0000 7fe7a7000640 1 -- 172.21.15.5:0/1758294774 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fe7880a2c00 msgr2=0x7fe7880a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:40:43.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:43.904+0000 7fe7a7000640 1 --2- 172.21.15.5:0/1758294774 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fe7880a2c00 0x7fe7880a50c0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fe794009b50 tx=0x7fe794005f90 comp rx=0 tx=0).stop 2024-09-10T09:40:43.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:43.904+0000 7fe7a7000640 1 -- 172.21.15.5:0/1758294774 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe7a0074ee0 msgr2=0x7fe7a019aa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:40:43.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:43.904+0000 7fe7a7000640 1 --2- 172.21.15.5:0/1758294774 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe7a0074ee0 0x7fe7a019aa20 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fe78c004770 tx=0x7fe78c003c10 comp rx=0 tx=0).stop 2024-09-10T09:40:43.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:43.904+0000 7fe7a7000640 1 -- 172.21.15.5:0/1758294774 shutdown_connections 2024-09-10T09:40:43.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:43.904+0000 7fe7a7000640 1 --2- 172.21.15.5:0/1758294774 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fe7880a2c00 0x7fe7880a50c0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:43.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:43.904+0000 7fe7a7000640 1 --2- 172.21.15.5:0/1758294774 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe7a010e260 0x7fe7a019edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:43.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:43.904+0000 7fe7a7000640 1 --2- 172.21.15.5:0/1758294774 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe7a0074ee0 0x7fe7a019aa20 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:43.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:43.904+0000 7fe7a7000640 1 --2- 172.21.15.5:0/1758294774 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe7a00745c0 0x7fe7a019a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:43.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:43.904+0000 7fe7a7000640 1 -- 172.21.15.5:0/1758294774 >> 172.21.15.5:0/1758294774 conn(0x7fe7a00fd740 msgr2=0x7fe7a00fe400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:40:43.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:43.904+0000 7fe7a7000640 1 -- 172.21.15.5:0/1758294774 shutdown_connections 2024-09-10T09:40:43.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:43.904+0000 7fe7a7000640 1 -- 172.21.15.5:0/1758294774 wait complete. 2024-09-10T09:40:43.991 INFO:tasks.workunit.client.0.smithi005.stdout: 1 39414 9.25 MB/sec warmup 25 sec latency 13.675 ms 2024-09-10T09:40:44.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:40:44 smithi005 ceph-mon[30350]: from='client.? 172.21.15.5:0/1758294774' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:40:44.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:40:44 smithi159 ceph-mon[35513]: from='client.? 172.21.15.5:0/1758294774' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:40:44.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:40:44 smithi102 ceph-mon[35442]: from='client.? 172.21.15.5:0/1758294774' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:40:44.752 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-10T09:40:44.991 INFO:tasks.workunit.client.0.smithi005.stdout: 1 41616 9.20 MB/sec warmup 26 sec latency 24.225 ms 2024-09-10T09:40:45.178 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:40:45.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:40:45 smithi159 ceph-mon[35513]: pgmap v465: 129 pgs: 129 active+clean; 34 MiB data, 458 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.6 MiB/s wr, 180 op/s 2024-09-10T09:40:45.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:40:45 smithi102 ceph-mon[35442]: pgmap v465: 129 pgs: 129 active+clean; 34 MiB data, 458 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.6 MiB/s wr, 180 op/s 2024-09-10T09:40:45.642 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:40:45 smithi005 ceph-mon[30350]: pgmap v465: 129 pgs: 129 active+clean; 34 MiB data, 458 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.6 MiB/s wr, 180 op/s 2024-09-10T09:40:45.991 INFO:tasks.workunit.client.0.smithi005.stdout: 1 43398 9.29 MB/sec warmup 27 sec latency 70.466 ms 2024-09-10T09:40:46.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:45.763+0000 7f112ca00640 1 -- 172.21.15.5:0/2716998518 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f11281079c0 msgr2=0x7f1128107da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:40:46.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:45.763+0000 7f112ca00640 1 --2- 172.21.15.5:0/2716998518 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f11281079c0 0x7f1128107da0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f1118009860 tx=0x7f111802a340 comp rx=0 tx=0).stop 2024-09-10T09:40:46.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:45.763+0000 7f112ca00640 1 -- 172.21.15.5:0/2716998518 shutdown_connections 2024-09-10T09:40:46.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:45.763+0000 7f112ca00640 1 --2- 172.21.15.5:0/2716998518 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1128102360 0x7f112810e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:46.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:45.763+0000 7f112ca00640 1 --2- 172.21.15.5:0/2716998518 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f11281019c0 0x7f1128101e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:46.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:45.763+0000 7f112ca00640 1 --2- 172.21.15.5:0/2716998518 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f11281079c0 0x7f1128107da0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:46.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:45.763+0000 7f112ca00640 1 -- 172.21.15.5:0/2716998518 >> 172.21.15.5:0/2716998518 conn(0x7f11280fd740 msgr2=0x7f11280ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:40:46.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:45.764+0000 7f112ca00640 1 -- 172.21.15.5:0/2716998518 shutdown_connections 2024-09-10T09:40:46.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:45.764+0000 7f112ca00640 1 -- 172.21.15.5:0/2716998518 wait complete. 2024-09-10T09:40:46.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:45.765+0000 7f112ca00640 1 Processor -- start 2024-09-10T09:40:46.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:45.765+0000 7f112ca00640 1 -- start start 2024-09-10T09:40:46.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:45.765+0000 7f112ca00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f11281019c0 0x7f11280744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:40:46.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:45.765+0000 7f112ca00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1128102360 0x7f1128074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:40:46.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:45.765+0000 7f112ca00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f11281079c0 0x7f1128074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:40:46.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:45.765+0000 7f112ca00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1128111dc0 con 0x7f11281019c0 2024-09-10T09:40:46.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:45.765+0000 7f112ca00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f1128111c40 con 0x7f11281079c0 2024-09-10T09:40:46.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:45.765+0000 7f112ca00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f1128111f40 con 0x7f1128102360 2024-09-10T09:40:46.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:45.765+0000 7f1126800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f11281019c0 0x7f11280744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:40:46.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:45.765+0000 7f1125e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1128102360 0x7f1128074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:40:46.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:45.765+0000 7f1127200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f11281079c0 0x7f1128074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:40:46.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:45.765+0000 7f1125e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1128102360 0x7f1128074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:44860/0 (socket says 172.21.15.5:44860) 2024-09-10T09:40:46.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:45.765+0000 7f1125e00640 1 -- 172.21.15.5:0/3332841390 learned_addr learned my addr 172.21.15.5:0/3332841390 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:40:46.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:45.766+0000 7f1126800640 1 -- 172.21.15.5:0/3332841390 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1128102360 msgr2=0x7f1128074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:40:46.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:45.766+0000 7f1126800640 1 --2- 172.21.15.5:0/3332841390 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1128102360 0x7f1128074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:46.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:45.766+0000 7f1126800640 1 -- 172.21.15.5:0/3332841390 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f11281079c0 msgr2=0x7f1128074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:40:46.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:45.766+0000 7f1126800640 1 --2- 172.21.15.5:0/3332841390 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f11281079c0 0x7f1128074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:46.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:45.766+0000 7f1126800640 1 -- 172.21.15.5:0/3332841390 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11281a6740 con 0x7f11281019c0 2024-09-10T09:40:46.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:45.766+0000 7f1125e00640 1 --2- 172.21.15.5:0/3332841390 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1128102360 0x7f1128074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:40:46.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:45.766+0000 7f1126800640 1 --2- 172.21.15.5:0/3332841390 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f11281019c0 0x7f11280744f0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f1118035040 tx=0x7f1118033b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:40:46.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:45.766+0000 7f111f400640 1 -- 172.21.15.5:0/3332841390 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f111803f020 con 0x7f11281019c0 2024-09-10T09:40:46.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:45.767+0000 7f111f400640 1 -- 172.21.15.5:0/3332841390 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1118004040 con 0x7f11281019c0 2024-09-10T09:40:46.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:45.767+0000 7f112ca00640 1 -- 172.21.15.5:0/3332841390 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11281a69d0 con 0x7f11281019c0 2024-09-10T09:40:46.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:45.767+0000 7f111f400640 1 -- 172.21.15.5:0/3332841390 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1118037040 con 0x7f11281019c0 2024-09-10T09:40:46.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:45.767+0000 7f112ca00640 1 -- 172.21.15.5:0/3332841390 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11281a6eb0 con 0x7f11281019c0 2024-09-10T09:40:46.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:45.770+0000 7f111f400640 1 -- 172.21.15.5:0/3332841390 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f1118042020 con 0x7f11281019c0 2024-09-10T09:40:46.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:45.770+0000 7f112fe8f740 1 Processor -- start 2024-09-10T09:40:46.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:45.770+0000 7f112fe8f740 1 -- start start 2024-09-10T09:40:46.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:45.770+0000 7f112fe8f740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558851cb5e40 0x558851cb6220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:40:46.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:45.770+0000 7f112fe8f740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558851cb6760 0x558851cc0ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:40:46.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:45.770+0000 7f112fe8f740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558851cc1020 0x558851cc3410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:40:46.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:45.770+0000 7f112fe8f740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x558851c1c3e0 con 0x558851cb5e40 2024-09-10T09:40:46.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:45.770+0000 7f112fe8f740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x558851bc3d80 con 0x558851cc1020 2024-09-10T09:40:46.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:45.770+0000 7f112fe8f740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x558851c183a0 con 0x558851cb6760 2024-09-10T09:40:46.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:45.771+0000 7f1127200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558851cb5e40 0x558851cb6220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:40:46.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:45.771+0000 7f1126800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558851cb6760 0x558851cc0ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:40:46.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:45.771+0000 7f1126800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558851cb6760 0x558851cc0ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:44862/0 (socket says 172.21.15.5:44862) 2024-09-10T09:40:46.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:45.771+0000 7f1127200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558851cb5e40 0x558851cb6220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38680/0 (socket says 172.21.15.5:38680) 2024-09-10T09:40:46.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:45.771+0000 7f1126800640 1 -- 172.21.15.5:0/681801671 learned_addr learned my addr 172.21.15.5:0/681801671 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:40:46.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:45.771+0000 7f1125e00640 1 --2- 172.21.15.5:0/681801671 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558851cc1020 0x558851cc3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:40:46.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:45.771+0000 7f1126800640 1 -- 172.21.15.5:0/681801671 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558851cc1020 msgr2=0x558851cc3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:40:46.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:45.771+0000 7f1126800640 1 --2- 172.21.15.5:0/681801671 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558851cc1020 0x558851cc3410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:46.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:45.771+0000 7f1126800640 1 -- 172.21.15.5:0/681801671 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558851cb5e40 msgr2=0x558851cb6220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:40:46.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:45.771+0000 7f1126800640 1 --2- 172.21.15.5:0/681801671 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558851cb5e40 0x558851cb6220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:46.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:45.771+0000 7f1126800640 1 -- 172.21.15.5:0/681801671 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558851be2ae0 con 0x558851cb6760 2024-09-10T09:40:46.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:45.771+0000 7f1127200640 1 --2- 172.21.15.5:0/681801671 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558851cb5e40 0x558851cb6220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:40:46.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:45.772+0000 7f1126800640 1 --2- 172.21.15.5:0/681801671 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558851cb6760 0x558851cc0ae0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f11180eca90 tx=0x7f11180f54f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8e17e689cde39f30 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:40:46.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:45.772+0000 7f111f400640 1 --2- 172.21.15.5:0/3332841390 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f110c0a2b80 0x7f110c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:40:46.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:45.772+0000 7f1113400640 1 -- 172.21.15.5:0/681801671 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11180b7b50 con 0x558851cb6760 2024-09-10T09:40:46.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:45.772+0000 7f111f400640 1 -- 172.21.15.5:0/3332841390 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f11180ef440 con 0x7f11281019c0 2024-09-10T09:40:46.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:45.772+0000 7f1113400640 1 -- 172.21.15.5:0/681801671 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f11180fc030 con 0x558851cb6760 2024-09-10T09:40:46.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:45.772+0000 7f1113400640 1 -- 172.21.15.5:0/681801671 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11180bad00 con 0x558851cb6760 2024-09-10T09:40:46.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:45.772+0000 7f1127200640 1 --2- 172.21.15.5:0/3332841390 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f110c0a2b80 0x7f110c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:40:46.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:45.773+0000 7f112fe8f740 1 -- 172.21.15.5:0/681801671 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558851cb6760 msgr2=0x558851cc0ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:40:46.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:45.773+0000 7f112fe8f740 1 --2- 172.21.15.5:0/681801671 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558851cb6760 0x558851cc0ae0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f11180eca90 tx=0x7f11180f54f0 comp rx=0 tx=0).stop 2024-09-10T09:40:46.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:45.773+0000 7f1127200640 1 --2- 172.21.15.5:0/3332841390 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f110c0a2b80 0x7f110c0a5040 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f1120009fd0 tx=0x7f1120009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:40:46.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:45.773+0000 7f112fe8f740 1 -- 172.21.15.5:0/681801671 shutdown_connections 2024-09-10T09:40:46.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:45.773+0000 7f112fe8f740 1 --2- 172.21.15.5:0/681801671 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558851cc1020 0x558851cc3410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:46.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:45.773+0000 7f112fe8f740 1 --2- 172.21.15.5:0/681801671 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558851cb6760 0x558851cc0ae0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:46.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:45.773+0000 7f112fe8f740 1 --2- 172.21.15.5:0/681801671 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558851cb5e40 0x558851cb6220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:46.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:45.773+0000 7f112fe8f740 1 -- 172.21.15.5:0/681801671 >> 172.21.15.5:0/681801671 conn(0x558851cb23d0 msgr2=0x558851cb27b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:40:46.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:45.773+0000 7f112fe8f740 1 -- 172.21.15.5:0/681801671 shutdown_connections 2024-09-10T09:40:46.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:45.773+0000 7f112fe8f740 1 -- 172.21.15.5:0/681801671 wait complete. 2024-09-10T09:40:46.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:45.773+0000 7f112fe8f740 1 -- start start 2024-09-10T09:40:46.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:45.774+0000 7f112fe8f740 1 Processor -- start 2024-09-10T09:40:46.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:45.774+0000 7f112fe8f740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558851cb5e40 0x558851cbdf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:40:46.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:45.774+0000 7f112fe8f740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558851cb6760 0x558851cbe450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:40:46.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:45.774+0000 7f112fe8f740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558851cc1020 0x558851cbe990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:40:46.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:45.774+0000 7f112fe8f740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x558851cc39b0 con 0x558851cb5e40 2024-09-10T09:40:46.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:45.774+0000 7f112fe8f740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x558851bd70f0 con 0x558851cc1020 2024-09-10T09:40:46.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:45.774+0000 7f112fe8f740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x558851cc3b30 con 0x558851cb6760 2024-09-10T09:40:46.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:45.774+0000 7f112fe8f740 10 client.0 _ll_register_callbacks cb 0x558851bbebd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:40:46.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:45.774+0000 7f1126800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558851cb5e40 0x558851cbdf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:40:46.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:45.775+0000 7f1125e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558851cb6760 0x558851cbe450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:40:46.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:45.775+0000 7f1127200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558851cc1020 0x558851cbe990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:40:46.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:45.775+0000 7f1125e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558851cb6760 0x558851cbe450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:44870/0 (socket says 172.21.15.5:44870) 2024-09-10T09:40:46.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:45.775+0000 7f1125e00640 1 -- 172.21.15.5:0/1622284549 learned_addr learned my addr 172.21.15.5:0/1622284549 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:40:46.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:45.775+0000 7f1125e00640 1 -- 172.21.15.5:0/1622284549 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558851cc1020 msgr2=0x558851cbe990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:40:46.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:45.775+0000 7f1125e00640 1 --2- 172.21.15.5:0/1622284549 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558851cc1020 0x558851cbe990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:46.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:45.775+0000 7f1125e00640 1 -- 172.21.15.5:0/1622284549 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558851cb5e40 msgr2=0x558851cbdf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:40:46.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:45.775+0000 7f1125e00640 1 --2- 172.21.15.5:0/1622284549 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558851cb5e40 0x558851cbdf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:46.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:45.775+0000 7f1125e00640 1 -- 172.21.15.5:0/1622284549 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55885198d3c0 con 0x558851cb6760 2024-09-10T09:40:46.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:45.775+0000 7f1126800640 1 --2- 172.21.15.5:0/1622284549 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558851cb5e40 0x558851cbdf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:40:46.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:45.775+0000 7f1127200640 1 --2- 172.21.15.5:0/1622284549 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558851cc1020 0x558851cbe990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:40:46.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:45.775+0000 7f1125e00640 1 --2- 172.21.15.5:0/1622284549 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558851cb6760 0x558851cbe450 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f1114011d40 tx=0x7f111400f790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=10dc8464377dc3da server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:40:46.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:45.775+0000 7f1107e00640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-09-10T09:40:46.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:45.775+0000 7f1107e00640 1 -- 172.21.15.5:0/1622284549 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f111401f060 con 0x558851cb6760 2024-09-10T09:40:46.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:45.775+0000 7f112fe8f740 1 -- 172.21.15.5:0/1622284549 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f11180fc030 con 0x558851cb6760 2024-09-10T09:40:46.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:45.775+0000 7f1107e00640 1 -- 172.21.15.5:0/1622284549 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f1114004240 con 0x558851cb6760 2024-09-10T09:40:46.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:45.776+0000 7f1107e00640 1 -- 172.21.15.5:0/1622284549 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f1114026050 con 0x558851cb6760 2024-09-10T09:40:46.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:45.776+0000 7f112fe8f740 10 client.25885 fetch_fsmap learned FSMap version 32 2024-09-10T09:40:46.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:45.776+0000 7f112fe8f740 1 -- 172.21.15.5:0/1622284549 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x558851ccf9c0 con 0x558851cb6760 2024-09-10T09:40:46.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:45.776+0000 7f1107e00640 1 -- 172.21.15.5:0/1622284549 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f11180fc030 con 0x558851cb6760 2024-09-10T09:40:46.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:45.776+0000 7f1107e00640 1 -- 172.21.15.5:0/1622284549 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7f1114019030 con 0x558851cb6760 2024-09-10T09:40:46.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:45.776+0000 7f112fe8f740 10 client.25885 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T09:40:46.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:45.776+0000 7f112fe8f740 10 client.25885 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-09-10T09:40:46.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:45.776+0000 7f112fe8f740 20 client.25885 populate_metadata read hostname 'smithi005' 2024-09-10T09:40:46.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:45.776+0000 7f112fe8f740 1 --2- 172.21.15.5:0/1622284549 >> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] conn(0x558851cd0e80 0x558851cd32d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:40:46.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:45.776+0000 7f112fe8f740 4 client.25885 mds_command: new command op to 24443 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:40:46.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:45.776+0000 7f112fe8f740 1 -- 172.21.15.5:0/1622284549 --> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558851c29950 con 0x558851cd0e80 2024-09-10T09:40:46.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:45.777+0000 7f1126800640 1 --2- 172.21.15.5:0/1622284549 >> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] conn(0x558851cd0e80 0x558851cd32d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:40:46.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:45.777+0000 7f1126800640 1 --2- 172.21.15.5:0/1622284549 >> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] conn(0x558851cd0e80 0x558851cd32d0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=1d6e73f80745cd87 server_cookie=4c2527bd06150bb in_seq=0 out_seq=0 2024-09-10T09:40:46.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:45.777+0000 7f1107e00640 10 client.25885 ms_handle_connect on v2:172.21.15.102:6832/2457007427 2024-09-10T09:40:46.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:45.778+0000 7f1107e00640 1 -- 172.21.15.5:0/1622284549 <== mds.4 v2:172.21.15.102:6832/2457007427 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558851c29950 con 0x558851cd0e80 2024-09-10T09:40:46.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:45.778+0000 7f1107e00640 10 client.25885 handle_command_reply: tid=0 2024-09-10T09:40:46.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:45.778+0000 7f112fe8f740 1 client.25885 shutdown 2024-09-10T09:40:46.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:45.778+0000 7f112fe8f740 1 -- 172.21.15.5:0/1622284549 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558851cb6760 msgr2=0x558851cbe450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:40:46.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:45.778+0000 7f112fe8f740 1 --2- 172.21.15.5:0/1622284549 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558851cb6760 0x558851cbe450 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f1114011d40 tx=0x7f111400f790 comp rx=0 tx=0).stop 2024-09-10T09:40:46.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:45.778+0000 7f112fe8f740 1 -- 172.21.15.5:0/1622284549 shutdown_connections 2024-09-10T09:40:46.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:45.778+0000 7f112fe8f740 1 --2- 172.21.15.5:0/1622284549 >> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] conn(0x558851cd0e80 0x558851cd32d0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:46.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:45.778+0000 7f112fe8f740 1 --2- 172.21.15.5:0/1622284549 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558851cc1020 0x558851cbe990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:46.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:45.778+0000 7f112fe8f740 1 --2- 172.21.15.5:0/1622284549 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558851cb6760 0x558851cbe450 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:46.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:45.778+0000 7f1107e00640 0 client.25885 ms_handle_reset on v2:172.21.15.102:6832/2457007427 2024-09-10T09:40:46.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:45.778+0000 7f112fe8f740 1 --2- 172.21.15.5:0/1622284549 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558851cb5e40 0x558851cbdf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:46.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:45.778+0000 7f112fe8f740 1 -- 172.21.15.5:0/1622284549 >> 172.21.15.5:0/1622284549 conn(0x558851cb23d0 msgr2=0x558851cb8100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:40:46.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:45.778+0000 7f112fe8f740 1 -- 172.21.15.5:0/1622284549 shutdown_connections 2024-09-10T09:40:46.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:45.778+0000 7f112fe8f740 1 -- 172.21.15.5:0/1622284549 wait complete. 2024-09-10T09:40:46.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:45.778+0000 7f112fe8f740 20 client.25885 trim_cache size 0 max 16384 2024-09-10T09:40:46.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:45.793+0000 7f112fe8f740 1 Processor -- start 2024-09-10T09:40:46.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:45.793+0000 7f112fe8f740 1 -- start start 2024-09-10T09:40:46.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:45.793+0000 7f112fe8f740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558851cb5e40 0x558851cb1380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:40:46.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:45.793+0000 7f112fe8f740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558851cb6760 0x558851cd8be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:40:46.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:45.793+0000 7f112fe8f740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558851cc1020 0x558851cd9120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:40:46.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:45.793+0000 7f112fe8f740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f11281b0ed0 con 0x558851cc1020 2024-09-10T09:40:46.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:45.793+0000 7f112fe8f740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f11281b2ad0 con 0x558851cb6760 2024-09-10T09:40:46.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:45.793+0000 7f112fe8f740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f11281b15a0 con 0x558851cb5e40 2024-09-10T09:40:46.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:45.794+0000 7f1126800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558851cb5e40 0x558851cb1380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:40:46.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:45.794+0000 7f1125e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558851cb6760 0x558851cd8be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:40:46.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:45.794+0000 7f1126800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558851cb5e40 0x558851cb1380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:44872/0 (socket says 172.21.15.5:44872) 2024-09-10T09:40:46.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:45.794+0000 7f1126800640 1 -- 172.21.15.5:0/581052062 learned_addr learned my addr 172.21.15.5:0/581052062 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:40:46.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:45.794+0000 7f1127200640 1 --2- 172.21.15.5:0/581052062 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558851cc1020 0x558851cd9120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:40:46.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:45.794+0000 7f1125e00640 1 -- 172.21.15.5:0/581052062 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558851cb5e40 msgr2=0x558851cb1380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:40:46.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:45.794+0000 7f1125e00640 1 --2- 172.21.15.5:0/581052062 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558851cb5e40 0x558851cb1380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:46.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:45.794+0000 7f1125e00640 1 -- 172.21.15.5:0/581052062 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558851cc1020 msgr2=0x558851cd9120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:40:46.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:45.794+0000 7f1125e00640 1 --2- 172.21.15.5:0/581052062 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558851cc1020 0x558851cd9120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:46.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:45.794+0000 7f1125e00640 1 -- 172.21.15.5:0/581052062 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558851cb8c90 con 0x558851cb6760 2024-09-10T09:40:46.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:45.794+0000 7f1126800640 1 --2- 172.21.15.5:0/581052062 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558851cb5e40 0x558851cb1380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:40:46.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:45.794+0000 7f1125e00640 1 --2- 172.21.15.5:0/581052062 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558851cb6760 0x558851cd8be0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x558851cbf4c0 tx=0x7f1114011820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=111127a1b63d6c57 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:40:46.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:45.794+0000 7f1112a00640 1 -- 172.21.15.5:0/581052062 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f111402f070 con 0x558851cb6760 2024-09-10T09:40:46.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:45.794+0000 7f1112a00640 1 -- 172.21.15.5:0/581052062 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f111400e780 con 0x558851cb6760 2024-09-10T09:40:46.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:45.794+0000 7f1112a00640 1 -- 172.21.15.5:0/581052062 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1114003650 con 0x558851cb6760 2024-09-10T09:40:46.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:45.795+0000 7f112fe8f740 1 -- 172.21.15.5:0/581052062 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558851cb6760 msgr2=0x558851cd8be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:40:46.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:45.795+0000 7f112fe8f740 1 --2- 172.21.15.5:0/581052062 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558851cb6760 0x558851cd8be0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x558851cbf4c0 tx=0x7f1114011820 comp rx=0 tx=0).stop 2024-09-10T09:40:46.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:45.795+0000 7f112fe8f740 1 -- 172.21.15.5:0/581052062 shutdown_connections 2024-09-10T09:40:46.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:45.795+0000 7f112fe8f740 1 --2- 172.21.15.5:0/581052062 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558851cc1020 0x558851cd9120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:46.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:45.795+0000 7f112fe8f740 1 --2- 172.21.15.5:0/581052062 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558851cb6760 0x558851cd8be0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:46.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:45.795+0000 7f112fe8f740 1 --2- 172.21.15.5:0/581052062 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558851cb5e40 0x558851cb1380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:46.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:45.795+0000 7f112fe8f740 1 -- 172.21.15.5:0/581052062 >> 172.21.15.5:0/581052062 conn(0x558851cb23d0 msgr2=0x558851cb0c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:40:46.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:45.795+0000 7f112fe8f740 1 -- 172.21.15.5:0/581052062 shutdown_connections 2024-09-10T09:40:46.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:45.795+0000 7f112fe8f740 1 -- 172.21.15.5:0/581052062 wait complete. 2024-09-10T09:40:46.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:45.796+0000 7f112fe8f740 1 -- start start 2024-09-10T09:40:46.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:45.796+0000 7f112fe8f740 1 Processor -- start 2024-09-10T09:40:46.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:45.796+0000 7f112fe8f740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558851cb5e40 0x558851cc43a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:40:46.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:45.796+0000 7f112fe8f740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558851cb6760 0x558851cc48e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:40:46.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:45.796+0000 7f112fe8f740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558851cc1020 0x558851cc8960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:40:46.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:45.796+0000 7f112fe8f740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f11281b2340 con 0x558851cb5e40 2024-09-10T09:40:46.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:45.796+0000 7f112fe8f740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f11281b1c70 con 0x558851cc1020 2024-09-10T09:40:46.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:45.796+0000 7f112fe8f740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f11281aeeb0 con 0x558851cb6760 2024-09-10T09:40:46.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:45.796+0000 7f112fe8f740 10 client.0 _ll_register_callbacks cb 0x7f1114009cd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:40:46.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:45.797+0000 7f1125e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558851cb6760 0x558851cc48e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:40:46.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:45.797+0000 7f1126800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558851cb5e40 0x558851cc43a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:40:46.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:45.797+0000 7f1125e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558851cb6760 0x558851cc48e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:44876/0 (socket says 172.21.15.5:44876) 2024-09-10T09:40:46.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:45.797+0000 7f1125e00640 1 -- 172.21.15.5:0/4216681221 learned_addr learned my addr 172.21.15.5:0/4216681221 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:40:46.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:45.797+0000 7f1125e00640 1 -- 172.21.15.5:0/4216681221 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558851cc1020 msgr2=0x558851cc8960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:40:46.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:45.797+0000 7f1127200640 1 --2- 172.21.15.5:0/4216681221 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558851cc1020 0x558851cc8960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:40:46.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:45.797+0000 7f1125e00640 1 --2- 172.21.15.5:0/4216681221 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558851cc1020 0x558851cc8960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:46.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:45.797+0000 7f1125e00640 1 -- 172.21.15.5:0/4216681221 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558851cb5e40 msgr2=0x558851cc43a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:40:46.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:45.797+0000 7f1125e00640 1 --2- 172.21.15.5:0/4216681221 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558851cb5e40 0x558851cc43a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:46.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:45.797+0000 7f1125e00640 1 -- 172.21.15.5:0/4216681221 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558851cc9000 con 0x558851cb6760 2024-09-10T09:40:46.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:45.797+0000 7f1126800640 1 --2- 172.21.15.5:0/4216681221 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558851cb5e40 0x558851cc43a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:40:46.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:45.797+0000 7f1127200640 1 --2- 172.21.15.5:0/4216681221 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558851cc1020 0x558851cc8960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T09:40:46.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:45.797+0000 7f1125e00640 1 --2- 172.21.15.5:0/4216681221 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558851cb6760 0x558851cc48e0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f111400e590 tx=0x7f1114005a00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2f5189e1e721352a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:40:46.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:45.797+0000 7f1107e00640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-09-10T09:40:46.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:45.798+0000 7f1107e00640 1 -- 172.21.15.5:0/4216681221 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f111401f070 con 0x558851cb6760 2024-09-10T09:40:46.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:45.798+0000 7f1107e00640 1 -- 172.21.15.5:0/4216681221 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f111400e930 con 0x558851cb6760 2024-09-10T09:40:46.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:45.798+0000 7f112fe8f740 1 -- 172.21.15.5:0/4216681221 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f111400e780 con 0x558851cb6760 2024-09-10T09:40:46.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:45.798+0000 7f1107e00640 1 -- 172.21.15.5:0/4216681221 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f111402f040 con 0x558851cb6760 2024-09-10T09:40:46.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:45.798+0000 7f112fe8f740 10 client.25891 fetch_fsmap learned FSMap version 32 2024-09-10T09:40:46.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:45.798+0000 7f112fe8f740 1 -- 172.21.15.5:0/4216681221 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x558851d3b600 con 0x558851cb6760 2024-09-10T09:40:46.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:45.798+0000 7f1107e00640 1 -- 172.21.15.5:0/4216681221 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f111400e780 con 0x558851cb6760 2024-09-10T09:40:46.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:45.798+0000 7f1107e00640 1 -- 172.21.15.5:0/4216681221 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7f1114019070 con 0x558851cb6760 2024-09-10T09:40:46.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:45.798+0000 7f112fe8f740 10 client.25891 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T09:40:46.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:45.798+0000 7f112fe8f740 10 client.25891 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-09-10T09:40:46.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:45.798+0000 7f112fe8f740 20 client.25891 populate_metadata read hostname 'smithi005' 2024-09-10T09:40:46.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:45.798+0000 7f112fe8f740 1 --2- 172.21.15.5:0/4216681221 >> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] conn(0x558851d3bc00 0x558851d3e020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:40:46.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:45.798+0000 7f112fe8f740 4 client.25891 mds_command: new command op to 24443 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T09:40:46.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:45.798+0000 7f112fe8f740 1 -- 172.21.15.5:0/4216681221 --> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558851c23e60 con 0x558851d3bc00 2024-09-10T09:40:46.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:45.799+0000 7f1126800640 1 --2- 172.21.15.5:0/4216681221 >> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] conn(0x558851d3bc00 0x558851d3e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:40:46.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:45.799+0000 7f1126800640 1 --2- 172.21.15.5:0/4216681221 >> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] conn(0x558851d3bc00 0x558851d3e020 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=68b2e9f9d1760f8 server_cookie=4fe99bfda45b2858 in_seq=0 out_seq=0 2024-09-10T09:40:46.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:45.799+0000 7f1107e00640 10 client.25891 ms_handle_connect on v2:172.21.15.102:6832/2457007427 2024-09-10T09:40:46.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:45.799+0000 7f1107e00640 1 -- 172.21.15.5:0/4216681221 <== mds.4 v2:172.21.15.102:6832/2457007427 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558851c23e60 con 0x558851d3bc00 2024-09-10T09:40:46.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:45.799+0000 7f1107e00640 10 client.25891 handle_command_reply: tid=0 2024-09-10T09:40:46.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:45.799+0000 7f112fe8f740 1 client.25891 shutdown 2024-09-10T09:40:46.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:45.799+0000 7f112fe8f740 1 -- 172.21.15.5:0/4216681221 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558851cb6760 msgr2=0x558851cc48e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:40:46.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:45.799+0000 7f112fe8f740 1 --2- 172.21.15.5:0/4216681221 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558851cb6760 0x558851cc48e0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f111400e590 tx=0x7f1114005a00 comp rx=0 tx=0).stop 2024-09-10T09:40:46.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:45.800+0000 7f112fe8f740 1 -- 172.21.15.5:0/4216681221 shutdown_connections 2024-09-10T09:40:46.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:45.800+0000 7f112fe8f740 1 --2- 172.21.15.5:0/4216681221 >> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] conn(0x558851d3bc00 0x558851d3e020 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:46.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:45.800+0000 7f112fe8f740 1 --2- 172.21.15.5:0/4216681221 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558851cc1020 0x558851cc8960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:46.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:45.800+0000 7f112fe8f740 1 --2- 172.21.15.5:0/4216681221 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558851cb6760 0x558851cc48e0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:46.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:45.800+0000 7f1107e00640 0 client.25891 ms_handle_reset on v2:172.21.15.102:6832/2457007427 2024-09-10T09:40:46.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:45.800+0000 7f112fe8f740 1 --2- 172.21.15.5:0/4216681221 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558851cb5e40 0x558851cc43a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:46.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:45.800+0000 7f112fe8f740 1 -- 172.21.15.5:0/4216681221 >> 172.21.15.5:0/4216681221 conn(0x558851cb23d0 msgr2=0x558851cb80b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:40:46.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:45.800+0000 7f112fe8f740 1 -- 172.21.15.5:0/4216681221 shutdown_connections 2024-09-10T09:40:46.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:45.800+0000 7f112fe8f740 1 -- 172.21.15.5:0/4216681221 wait complete. 2024-09-10T09:40:46.146 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-10T09:40:46.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:45.800+0000 7f112fe8f740 20 client.25891 trim_cache size 0 max 16384 2024-09-10T09:40:46.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:45.800+0000 7f112ca00640 1 -- 172.21.15.5:0/3332841390 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f110c0a2b80 msgr2=0x7f110c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:40:46.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:45.800+0000 7f112ca00640 1 --2- 172.21.15.5:0/3332841390 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f110c0a2b80 0x7f110c0a5040 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f1120009fd0 tx=0x7f1120009500 comp rx=0 tx=0).stop 2024-09-10T09:40:46.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:45.800+0000 7f112ca00640 1 -- 172.21.15.5:0/3332841390 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f11281019c0 msgr2=0x7f11280744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:40:46.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:45.800+0000 7f112ca00640 1 --2- 172.21.15.5:0/3332841390 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f11281019c0 0x7f11280744f0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f1118035040 tx=0x7f1118033b60 comp rx=0 tx=0).stop 2024-09-10T09:40:46.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:45.801+0000 7f112ca00640 1 -- 172.21.15.5:0/3332841390 shutdown_connections 2024-09-10T09:40:46.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:45.801+0000 7f112ca00640 1 --2- 172.21.15.5:0/3332841390 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f110c0a2b80 0x7f110c0a5040 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:46.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:45.801+0000 7f112ca00640 1 --2- 172.21.15.5:0/3332841390 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f11281079c0 0x7f1128074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:46.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:45.801+0000 7f112ca00640 1 --2- 172.21.15.5:0/3332841390 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1128102360 0x7f1128074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:46.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:45.801+0000 7f112ca00640 1 --2- 172.21.15.5:0/3332841390 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f11281019c0 0x7f11280744f0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:46.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:45.801+0000 7f112ca00640 1 -- 172.21.15.5:0/3332841390 >> 172.21.15.5:0/3332841390 conn(0x7f11280fd740 msgr2=0x7f11280fdf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:40:46.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:45.801+0000 7f112ca00640 1 -- 172.21.15.5:0/3332841390 shutdown_connections 2024-09-10T09:40:46.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:45.801+0000 7f112ca00640 1 -- 172.21.15.5:0/3332841390 wait complete. 2024-09-10T09:40:46.991 INFO:tasks.workunit.client.0.smithi005.stdout: 1 46290 9.58 MB/sec warmup 28 sec latency 10.477 ms 2024-09-10T09:40:47.120 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-10T09:40:47.577 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:40:47.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:40:47 smithi159 ceph-mon[35513]: pgmap v466: 129 pgs: 129 active+clean; 35 MiB data, 474 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.2 MiB/s wr, 179 op/s 2024-09-10T09:40:47.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:40:47 smithi102 ceph-mon[35442]: pgmap v466: 129 pgs: 129 active+clean; 35 MiB data, 474 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.2 MiB/s wr, 179 op/s 2024-09-10T09:40:47.723 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:40:47 smithi005 ceph-mon[30350]: pgmap v466: 129 pgs: 129 active+clean; 35 MiB data, 474 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.2 MiB/s wr, 179 op/s 2024-09-10T09:40:47.991 INFO:tasks.workunit.client.0.smithi005.stdout: 1 48125 9.48 MB/sec warmup 29 sec latency 19.093 ms 2024-09-10T09:40:48.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:48.226+0000 7f15e8a00640 1 -- 172.21.15.5:0/3099206650 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f15e41079c0 msgr2=0x7f15e4107da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:40:48.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:48.226+0000 7f15e8a00640 1 --2- 172.21.15.5:0/3099206650 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f15e41079c0 0x7f15e4107da0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f15d4009900 tx=0x7f15d402a3f0 comp rx=0 tx=0).stop 2024-09-10T09:40:48.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:48.227+0000 7f15e8a00640 1 -- 172.21.15.5:0/3099206650 shutdown_connections 2024-09-10T09:40:48.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:48.227+0000 7f15e8a00640 1 --2- 172.21.15.5:0/3099206650 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f15e4102360 0x7f15e410e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:48.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:48.227+0000 7f15e8a00640 1 --2- 172.21.15.5:0/3099206650 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f15e41019c0 0x7f15e4101e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:48.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:48.227+0000 7f15e8a00640 1 --2- 172.21.15.5:0/3099206650 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f15e41079c0 0x7f15e4107da0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:48.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:48.227+0000 7f15e8a00640 1 -- 172.21.15.5:0/3099206650 >> 172.21.15.5:0/3099206650 conn(0x7f15e40fd740 msgr2=0x7f15e40ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:40:48.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:48.227+0000 7f15e8a00640 1 -- 172.21.15.5:0/3099206650 shutdown_connections 2024-09-10T09:40:48.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:48.227+0000 7f15e8a00640 1 -- 172.21.15.5:0/3099206650 wait complete. 2024-09-10T09:40:48.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:48.228+0000 7f15e8a00640 1 Processor -- start 2024-09-10T09:40:48.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:48.228+0000 7f15e8a00640 1 -- start start 2024-09-10T09:40:48.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:48.228+0000 7f15e8a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f15e41019c0 0x7f15e40744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:40:48.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:48.228+0000 7f15e8a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f15e4102360 0x7f15e4074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:40:48.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:48.228+0000 7f15e8a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f15e41079c0 0x7f15e4074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:40:48.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:48.228+0000 7f15e8a00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f15e4111dd0 con 0x7f15e4102360 2024-09-10T09:40:48.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:48.228+0000 7f15e8a00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f15e4111c50 con 0x7f15e41079c0 2024-09-10T09:40:48.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:48.228+0000 7f15e8a00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f15e4111f50 con 0x7f15e41019c0 2024-09-10T09:40:48.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:48.229+0000 7f15e2800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f15e41019c0 0x7f15e40744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:40:48.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:48.229+0000 7f15e1e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f15e4102360 0x7f15e4074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:40:48.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:48.229+0000 7f15e2800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f15e41019c0 0x7f15e40744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:44894/0 (socket says 172.21.15.5:44894) 2024-09-10T09:40:48.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:48.229+0000 7f15e2800640 1 -- 172.21.15.5:0/1693522015 learned_addr learned my addr 172.21.15.5:0/1693522015 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:40:48.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:48.229+0000 7f15e3200640 1 --2- 172.21.15.5:0/1693522015 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f15e41079c0 0x7f15e4074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:40:48.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:48.229+0000 7f15e2800640 1 -- 172.21.15.5:0/1693522015 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f15e41079c0 msgr2=0x7f15e4074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:40:48.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:48.229+0000 7f15e2800640 1 --2- 172.21.15.5:0/1693522015 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f15e41079c0 0x7f15e4074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:48.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:48.229+0000 7f15e2800640 1 -- 172.21.15.5:0/1693522015 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f15e4102360 msgr2=0x7f15e4074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:40:48.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:48.229+0000 7f15e2800640 1 --2- 172.21.15.5:0/1693522015 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f15e4102360 0x7f15e4074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:48.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:48.229+0000 7f15e2800640 1 -- 172.21.15.5:0/1693522015 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15e41aac20 con 0x7f15e41019c0 2024-09-10T09:40:48.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:48.230+0000 7f15e2800640 1 --2- 172.21.15.5:0/1693522015 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f15e41019c0 0x7f15e40744f0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f15d402a9e0 tx=0x7f15d4003c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:40:48.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:48.230+0000 7f15db400640 1 -- 172.21.15.5:0/1693522015 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15d403f070 con 0x7f15e41019c0 2024-09-10T09:40:48.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:48.230+0000 7f15db400640 1 -- 172.21.15.5:0/1693522015 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f15d4031c30 con 0x7f15e41019c0 2024-09-10T09:40:48.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:48.230+0000 7f15e8a00640 1 -- 172.21.15.5:0/1693522015 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15e41aadf0 con 0x7f15e41019c0 2024-09-10T09:40:48.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:48.230+0000 7f15db400640 1 -- 172.21.15.5:0/1693522015 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15d4035030 con 0x7f15e41019c0 2024-09-10T09:40:48.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:48.231+0000 7f15e8a00640 1 -- 172.21.15.5:0/1693522015 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15e41ab2a0 con 0x7f15e41019c0 2024-09-10T09:40:48.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:48.234+0000 7f15ec022740 1 Processor -- start 2024-09-10T09:40:48.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:48.234+0000 7f15ec022740 1 -- start start 2024-09-10T09:40:48.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:48.234+0000 7f15db400640 1 -- 172.21.15.5:0/1693522015 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f15d4042070 con 0x7f15e41019c0 2024-09-10T09:40:48.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:48.234+0000 7f15ec022740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5587a0f04e40 0x5587a0f05220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:40:48.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:48.234+0000 7f15ec022740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5587a0f05760 0x5587a0f0fae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:40:48.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:48.234+0000 7f15ec022740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5587a0f10020 0x5587a0f12410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:40:48.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:48.234+0000 7f15ec022740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5587a0e6b3e0 con 0x5587a0f10020 2024-09-10T09:40:48.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:48.235+0000 7f15ec022740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x5587a0e12d80 con 0x5587a0f05760 2024-09-10T09:40:48.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:48.235+0000 7f15ec022740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5587a0e673a0 con 0x5587a0f04e40 2024-09-10T09:40:48.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:48.235+0000 7f15e3200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5587a0f04e40 0x5587a0f05220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:40:48.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:48.235+0000 7f15e3200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5587a0f04e40 0x5587a0f05220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:44904/0 (socket says 172.21.15.5:44904) 2024-09-10T09:40:48.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:48.235+0000 7f15e3200640 1 -- 172.21.15.5:0/702738049 learned_addr learned my addr 172.21.15.5:0/702738049 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:40:48.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:48.235+0000 7f15e1e00640 1 --2- 172.21.15.5:0/702738049 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5587a0f10020 0x5587a0f12410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:40:48.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:48.235+0000 7f15e2800640 1 --2- 172.21.15.5:0/702738049 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5587a0f05760 0x5587a0f0fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:40:48.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:48.235+0000 7f15e3200640 1 -- 172.21.15.5:0/702738049 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5587a0f05760 msgr2=0x5587a0f0fae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:40:48.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:48.235+0000 7f15e3200640 1 --2- 172.21.15.5:0/702738049 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5587a0f05760 0x5587a0f0fae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:48.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:48.235+0000 7f15e3200640 1 -- 172.21.15.5:0/702738049 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5587a0f10020 msgr2=0x5587a0f12410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:40:48.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:48.235+0000 7f15e3200640 1 --2- 172.21.15.5:0/702738049 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5587a0f10020 0x5587a0f12410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:48.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:48.235+0000 7f15e3200640 1 -- 172.21.15.5:0/702738049 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5587a0e31ae0 con 0x5587a0f04e40 2024-09-10T09:40:48.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:48.236+0000 7f15e3200640 1 --2- 172.21.15.5:0/702738049 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5587a0f04e40 0x5587a0f05220 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f15dc009f50 tx=0x7f15dc0118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bda5175fccb1b178 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:40:48.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:48.236+0000 7f15cf400640 1 -- 172.21.15.5:0/702738049 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15dc004d20 con 0x5587a0f04e40 2024-09-10T09:40:48.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:48.236+0000 7f15cf400640 1 -- 172.21.15.5:0/702738049 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f15dc004050 con 0x5587a0f04e40 2024-09-10T09:40:48.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:48.236+0000 7f15cf400640 1 -- 172.21.15.5:0/702738049 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15dc005260 con 0x5587a0f04e40 2024-09-10T09:40:48.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:48.236+0000 7f15db400640 1 --2- 172.21.15.5:0/1693522015 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f15c80a2c00 0x7f15c80a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:40:48.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:48.236+0000 7f15db400640 1 -- 172.21.15.5:0/1693522015 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f15d40ef610 con 0x7f15e41019c0 2024-09-10T09:40:48.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:48.236+0000 7f15e2800640 1 --2- 172.21.15.5:0/1693522015 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f15c80a2c00 0x7f15c80a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:40:48.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:48.237+0000 7f15ec022740 1 -- 172.21.15.5:0/702738049 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5587a0f04e40 msgr2=0x5587a0f05220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:40:48.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:48.237+0000 7f15ec022740 1 --2- 172.21.15.5:0/702738049 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5587a0f04e40 0x5587a0f05220 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f15dc009f50 tx=0x7f15dc0118d0 comp rx=0 tx=0).stop 2024-09-10T09:40:48.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:48.237+0000 7f15ec022740 1 -- 172.21.15.5:0/702738049 shutdown_connections 2024-09-10T09:40:48.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:48.237+0000 7f15ec022740 1 --2- 172.21.15.5:0/702738049 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5587a0f10020 0x5587a0f12410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:48.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:48.237+0000 7f15ec022740 1 --2- 172.21.15.5:0/702738049 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5587a0f05760 0x5587a0f0fae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:48.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:48.237+0000 7f15ec022740 1 --2- 172.21.15.5:0/702738049 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5587a0f04e40 0x5587a0f05220 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:48.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:48.237+0000 7f15ec022740 1 -- 172.21.15.5:0/702738049 >> 172.21.15.5:0/702738049 conn(0x5587a0f013d0 msgr2=0x5587a0f017b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:40:48.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:48.237+0000 7f15e2800640 1 --2- 172.21.15.5:0/1693522015 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f15c80a2c00 0x7f15c80a50c0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f15d403d040 tx=0x7f15d403b720 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:40:48.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:48.237+0000 7f15ec022740 1 -- 172.21.15.5:0/702738049 shutdown_connections 2024-09-10T09:40:48.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:48.237+0000 7f15ec022740 1 -- 172.21.15.5:0/702738049 wait complete. 2024-09-10T09:40:48.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:48.238+0000 7f15ec022740 1 -- start start 2024-09-10T09:40:48.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:48.238+0000 7f15ec022740 1 Processor -- start 2024-09-10T09:40:48.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:48.239+0000 7f15ec022740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5587a0f04e40 0x5587a0f0cf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:40:48.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:48.239+0000 7f15ec022740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5587a0f05760 0x5587a0f0d460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:40:48.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:48.239+0000 7f15ec022740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5587a0f10020 0x5587a0f0d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:40:48.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:48.239+0000 7f15ec022740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5587a0f12bf0 con 0x5587a0f10020 2024-09-10T09:40:48.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:48.239+0000 7f15ec022740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x5587a0e260f0 con 0x5587a0f04e40 2024-09-10T09:40:48.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:48.239+0000 7f15ec022740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5587a0f12d70 con 0x5587a0f05760 2024-09-10T09:40:48.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:48.239+0000 7f15ec022740 10 client.0 _ll_register_callbacks cb 0x5587a0e0dbd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:40:48.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:48.239+0000 7f15e1e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5587a0f05760 0x5587a0f0d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:40:48.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:48.239+0000 7f15e3200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5587a0f04e40 0x5587a0f0cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:40:48.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:48.239+0000 7f15e3200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5587a0f04e40 0x5587a0f0cf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:37702/0 (socket says 172.21.15.5:37702) 2024-09-10T09:40:48.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:48.239+0000 7f15e3200640 1 -- 172.21.15.5:0/285413793 learned_addr learned my addr 172.21.15.5:0/285413793 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:40:48.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:48.239+0000 7f15e1e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5587a0f05760 0x5587a0f0d460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:44906/0 (socket says 172.21.15.5:44906) 2024-09-10T09:40:48.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:48.239+0000 7f15e3200640 1 --2- 172.21.15.5:0/285413793 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5587a0f10020 0x5587a0f0d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:40:48.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:48.240+0000 7f15e3200640 1 -- 172.21.15.5:0/285413793 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5587a0f05760 msgr2=0x5587a0f0d460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:40:48.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:48.240+0000 7f15e3200640 1 --2- 172.21.15.5:0/285413793 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5587a0f05760 0x5587a0f0d460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:48.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:48.240+0000 7f15e3200640 1 -- 172.21.15.5:0/285413793 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5587a0f10020 msgr2=0x5587a0f0d9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:40:48.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:48.240+0000 7f15e3200640 1 --2- 172.21.15.5:0/285413793 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5587a0f10020 0x5587a0f0d9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:48.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:48.240+0000 7f15e3200640 1 -- 172.21.15.5:0/285413793 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5587a0bdc3c0 con 0x5587a0f04e40 2024-09-10T09:40:48.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:48.240+0000 7f15e3200640 1 --2- 172.21.15.5:0/285413793 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5587a0f04e40 0x5587a0f0cf20 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f15dc004e40 tx=0x7f15dc005bd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f3bf0c9938b0806 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:40:48.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:48.240+0000 7f15c3e00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-09-10T09:40:48.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:48.240+0000 7f15c3e00640 1 -- 172.21.15.5:0/285413793 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15dc02b070 con 0x5587a0f04e40 2024-09-10T09:40:48.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:48.240+0000 7f15c3e00640 1 -- 172.21.15.5:0/285413793 <== mon.2 v2:172.21.15.102:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f15dc027640 con 0x5587a0f04e40 2024-09-10T09:40:48.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:48.240+0000 7f15ec022740 1 -- 172.21.15.5:0/285413793 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f15dc004050 con 0x5587a0f04e40 2024-09-10T09:40:48.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:48.241+0000 7f15c3e00640 1 -- 172.21.15.5:0/285413793 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f15dc037050 con 0x5587a0f04e40 2024-09-10T09:40:48.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:48.241+0000 7f15ec022740 10 client.25871 fetch_fsmap learned FSMap version 32 2024-09-10T09:40:48.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:48.241+0000 7f15ec022740 1 -- 172.21.15.5:0/285413793 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5587a0f1ec70 con 0x5587a0f04e40 2024-09-10T09:40:48.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:48.241+0000 7f15c3e00640 1 -- 172.21.15.5:0/285413793 <== mon.2 v2:172.21.15.102:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f15dc004050 con 0x5587a0f04e40 2024-09-10T09:40:48.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:48.241+0000 7f15c3e00640 1 -- 172.21.15.5:0/285413793 <== mon.2 v2:172.21.15.102:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7f15dc00f9a0 con 0x5587a0f04e40 2024-09-10T09:40:48.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:48.241+0000 7f15ec022740 10 client.25871 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T09:40:48.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:48.241+0000 7f15ec022740 10 client.25871 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-09-10T09:40:48.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:48.241+0000 7f15ec022740 20 client.25871 populate_metadata read hostname 'smithi005' 2024-09-10T09:40:48.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:48.241+0000 7f15ec022740 1 --2- 172.21.15.5:0/285413793 >> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] conn(0x5587a0f20020 0x5587a0f22410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:40:48.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:48.241+0000 7f15ec022740 4 client.25871 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:40:48.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:48.241+0000 7f15ec022740 1 -- 172.21.15.5:0/285413793 --> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5587a0e78950 con 0x5587a0f20020 2024-09-10T09:40:48.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:48.242+0000 7f15e1e00640 1 --2- 172.21.15.5:0/285413793 >> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] conn(0x5587a0f20020 0x5587a0f22410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:40:48.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:48.242+0000 7f15c3e00640 10 client.25871 ms_handle_connect on v2:172.21.15.102:6836/2689776764 2024-09-10T09:40:48.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:48.242+0000 7f15e1e00640 1 --2- 172.21.15.5:0/285413793 >> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] conn(0x5587a0f20020 0x5587a0f22410 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=4c121bb1c1692a6f server_cookie=1b4aaa2f18abddf in_seq=0 out_seq=0 2024-09-10T09:40:48.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:48.243+0000 7f15c3e00640 1 -- 172.21.15.5:0/285413793 <== mds.3 v2:172.21.15.102:6836/2689776764 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5587a0e78950 con 0x5587a0f20020 2024-09-10T09:40:48.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:48.243+0000 7f15c3e00640 10 client.25871 handle_command_reply: tid=0 2024-09-10T09:40:48.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:48.243+0000 7f15ec022740 1 client.25871 shutdown 2024-09-10T09:40:48.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:48.243+0000 7f15ec022740 1 -- 172.21.15.5:0/285413793 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5587a0f04e40 msgr2=0x5587a0f0cf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:40:48.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:48.243+0000 7f15ec022740 1 --2- 172.21.15.5:0/285413793 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5587a0f04e40 0x5587a0f0cf20 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f15dc004e40 tx=0x7f15dc005bd0 comp rx=0 tx=0).stop 2024-09-10T09:40:48.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:48.244+0000 7f15ec022740 1 -- 172.21.15.5:0/285413793 shutdown_connections 2024-09-10T09:40:48.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:48.244+0000 7f15ec022740 1 --2- 172.21.15.5:0/285413793 >> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] conn(0x5587a0f20020 0x5587a0f22410 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:48.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:48.244+0000 7f15ec022740 1 --2- 172.21.15.5:0/285413793 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5587a0f10020 0x5587a0f0d9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:48.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:48.244+0000 7f15ec022740 1 --2- 172.21.15.5:0/285413793 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5587a0f05760 0x5587a0f0d460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:48.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:48.244+0000 7f15ec022740 1 --2- 172.21.15.5:0/285413793 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5587a0f04e40 0x5587a0f0cf20 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:48.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:48.244+0000 7f15c3e00640 0 client.25871 ms_handle_reset on v2:172.21.15.102:6836/2689776764 2024-09-10T09:40:48.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:48.244+0000 7f15ec022740 1 -- 172.21.15.5:0/285413793 >> 172.21.15.5:0/285413793 conn(0x5587a0f013d0 msgr2=0x5587a0f070b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:40:48.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:48.244+0000 7f15ec022740 1 -- 172.21.15.5:0/285413793 shutdown_connections 2024-09-10T09:40:48.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:48.244+0000 7f15ec022740 1 -- 172.21.15.5:0/285413793 wait complete. 2024-09-10T09:40:48.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:48.244+0000 7f15ec022740 20 client.25871 trim_cache size 0 max 16384 2024-09-10T09:40:48.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:48.258+0000 7f15ec022740 1 Processor -- start 2024-09-10T09:40:48.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:48.258+0000 7f15ec022740 1 -- start start 2024-09-10T09:40:48.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:48.259+0000 7f15ec022740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5587a0f04e40 0x5587a0f0d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:40:48.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:48.259+0000 7f15ec022740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5587a0f05760 0x5587a0f0ee90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:40:48.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:48.259+0000 7f15ec022740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5587a0f10020 0x5587a0f27b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:40:48.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:48.259+0000 7f15ec022740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f15e41b52a0 con 0x5587a0f05760 2024-09-10T09:40:48.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:48.259+0000 7f15ec022740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f15e41b6ea0 con 0x5587a0f10020 2024-09-10T09:40:48.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:48.259+0000 7f15ec022740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f15e41b5970 con 0x5587a0f04e40 2024-09-10T09:40:48.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:48.259+0000 7f15e3200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5587a0f04e40 0x5587a0f0d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:40:48.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:48.259+0000 7f15e3200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5587a0f04e40 0x5587a0f0d7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:44912/0 (socket says 172.21.15.5:44912) 2024-09-10T09:40:48.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:48.259+0000 7f15e3200640 1 -- 172.21.15.5:0/945315879 learned_addr learned my addr 172.21.15.5:0/945315879 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:40:48.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:48.259+0000 7f15e1e00640 1 --2- 172.21.15.5:0/945315879 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5587a0f05760 0x5587a0f0ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:40:48.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:48.259+0000 7f15e3200640 1 --2- 172.21.15.5:0/945315879 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5587a0f10020 0x5587a0f27b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:40:48.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:48.259+0000 7f15e3200640 1 -- 172.21.15.5:0/945315879 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5587a0f10020 msgr2=0x5587a0f27b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:40:48.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:48.259+0000 7f15e3200640 1 --2- 172.21.15.5:0/945315879 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5587a0f10020 0x5587a0f27b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:48.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:48.259+0000 7f15e3200640 1 -- 172.21.15.5:0/945315879 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5587a0f05760 msgr2=0x5587a0f0ee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:40:48.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:48.259+0000 7f15e3200640 1 --2- 172.21.15.5:0/945315879 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5587a0f05760 0x5587a0f0ee90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:48.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:48.260+0000 7f15e3200640 1 -- 172.21.15.5:0/945315879 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5587a0f04c70 con 0x5587a0f04e40 2024-09-10T09:40:48.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:48.260+0000 7f15e1e00640 1 --2- 172.21.15.5:0/945315879 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5587a0f05760 0x5587a0f0ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T09:40:48.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:48.260+0000 7f15e3200640 1 --2- 172.21.15.5:0/945315879 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5587a0f04e40 0x5587a0f0d7f0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f15dc010010 tx=0x7f15dc00f480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b1f20f4fcdd120d8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:40:48.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:48.260+0000 7f15cea00640 1 -- 172.21.15.5:0/945315879 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15dc02b070 con 0x5587a0f04e40 2024-09-10T09:40:48.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:48.260+0000 7f15cea00640 1 -- 172.21.15.5:0/945315879 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f15dc019d00 con 0x5587a0f04e40 2024-09-10T09:40:48.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:48.260+0000 7f15cea00640 1 -- 172.21.15.5:0/945315879 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15dc00f750 con 0x5587a0f04e40 2024-09-10T09:40:48.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:48.261+0000 7f15ec022740 1 -- 172.21.15.5:0/945315879 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5587a0f04e40 msgr2=0x5587a0f0d7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:40:48.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:48.261+0000 7f15ec022740 1 --2- 172.21.15.5:0/945315879 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5587a0f04e40 0x5587a0f0d7f0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f15dc010010 tx=0x7f15dc00f480 comp rx=0 tx=0).stop 2024-09-10T09:40:48.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:48.261+0000 7f15ec022740 1 -- 172.21.15.5:0/945315879 shutdown_connections 2024-09-10T09:40:48.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:48.261+0000 7f15ec022740 1 --2- 172.21.15.5:0/945315879 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5587a0f10020 0x5587a0f27b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:48.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:48.261+0000 7f15ec022740 1 --2- 172.21.15.5:0/945315879 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5587a0f05760 0x5587a0f0ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:48.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:48.261+0000 7f15ec022740 1 --2- 172.21.15.5:0/945315879 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5587a0f04e40 0x5587a0f0d7f0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:48.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:48.261+0000 7f15ec022740 1 -- 172.21.15.5:0/945315879 >> 172.21.15.5:0/945315879 conn(0x5587a0f013d0 msgr2=0x5587a0f0e780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:40:48.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:48.261+0000 7f15ec022740 1 -- 172.21.15.5:0/945315879 shutdown_connections 2024-09-10T09:40:48.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:48.261+0000 7f15ec022740 1 -- 172.21.15.5:0/945315879 wait complete. 2024-09-10T09:40:48.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:48.261+0000 7f15ec022740 1 -- start start 2024-09-10T09:40:48.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:48.262+0000 7f15ec022740 1 Processor -- start 2024-09-10T09:40:48.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:48.262+0000 7f15ec022740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5587a0f04e40 0x5587a0f18240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:40:48.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:48.262+0000 7f15ec022740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5587a0f05760 0x5587a0f18780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:40:48.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:48.262+0000 7f15ec022740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5587a0f10020 0x5587a0f09790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:40:48.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:48.262+0000 7f15ec022740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f15e41b6710 con 0x5587a0f04e40 2024-09-10T09:40:48.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:48.262+0000 7f15ec022740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f15e41b6040 con 0x5587a0f05760 2024-09-10T09:40:48.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:48.262+0000 7f15ec022740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f15e41b3280 con 0x5587a0f10020 2024-09-10T09:40:48.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:48.262+0000 7f15ec022740 10 client.0 _ll_register_callbacks cb 0x7f15d00097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:40:48.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:48.262+0000 7f15e1e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5587a0f05760 0x5587a0f18780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:40:48.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:48.262+0000 7f15e3200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5587a0f04e40 0x5587a0f18240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:40:48.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:48.262+0000 7f15e1e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5587a0f05760 0x5587a0f18780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:37728/0 (socket says 172.21.15.5:37728) 2024-09-10T09:40:48.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:48.262+0000 7f15e1e00640 1 -- 172.21.15.5:0/3901259246 learned_addr learned my addr 172.21.15.5:0/3901259246 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:40:48.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:48.263+0000 7f15e3200640 1 --2- 172.21.15.5:0/3901259246 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5587a0f10020 0x5587a0f09790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:40:48.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:48.263+0000 7f15e1e00640 1 -- 172.21.15.5:0/3901259246 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5587a0f10020 msgr2=0x5587a0f09790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:40:48.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:48.263+0000 7f15e1e00640 1 --2- 172.21.15.5:0/3901259246 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5587a0f10020 0x5587a0f09790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:48.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:48.263+0000 7f15e1e00640 1 -- 172.21.15.5:0/3901259246 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5587a0f04e40 msgr2=0x5587a0f18240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:40:48.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:48.263+0000 7f15e1e00640 1 --2- 172.21.15.5:0/3901259246 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5587a0f04e40 0x5587a0f18240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:48.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:48.263+0000 7f15e1e00640 1 -- 172.21.15.5:0/3901259246 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5587a0f09f90 con 0x5587a0f05760 2024-09-10T09:40:48.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:48.263+0000 7f15e3200640 1 --2- 172.21.15.5:0/3901259246 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5587a0f10020 0x5587a0f09790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:40:48.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:48.263+0000 7f15e1e00640 1 --2- 172.21.15.5:0/3901259246 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5587a0f05760 0x5587a0f18780 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f15e4075cb0 tx=0x7f15d00108c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2079727f88b2fbf7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:40:48.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:48.263+0000 7f15c3e00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-09-10T09:40:48.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:48.264+0000 7f15c3e00640 1 -- 172.21.15.5:0/3901259246 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15d0022070 con 0x5587a0f05760 2024-09-10T09:40:48.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:48.264+0000 7f15c3e00640 1 -- 172.21.15.5:0/3901259246 <== mon.2 v2:172.21.15.102:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f15d0005220 con 0x5587a0f05760 2024-09-10T09:40:48.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:48.264+0000 7f15ec022740 1 -- 172.21.15.5:0/3901259246 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f15dc019d00 con 0x5587a0f05760 2024-09-10T09:40:48.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:48.264+0000 7f15c3e00640 1 -- 172.21.15.5:0/3901259246 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f15d002f050 con 0x5587a0f05760 2024-09-10T09:40:48.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:48.264+0000 7f15ec022740 10 client.25877 fetch_fsmap learned FSMap version 32 2024-09-10T09:40:48.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:48.264+0000 7f15ec022740 1 -- 172.21.15.5:0/3901259246 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5587a0f8a600 con 0x5587a0f05760 2024-09-10T09:40:48.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:48.264+0000 7f15c3e00640 1 -- 172.21.15.5:0/3901259246 <== mon.2 v2:172.21.15.102:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f15dc019d00 con 0x5587a0f05760 2024-09-10T09:40:48.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:48.264+0000 7f15c3e00640 1 -- 172.21.15.5:0/3901259246 <== mon.2 v2:172.21.15.102:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7f15d0004aa0 con 0x5587a0f05760 2024-09-10T09:40:48.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:48.264+0000 7f15ec022740 10 client.25877 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T09:40:48.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:48.264+0000 7f15ec022740 10 client.25877 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-09-10T09:40:48.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:48.265+0000 7f15ec022740 20 client.25877 populate_metadata read hostname 'smithi005' 2024-09-10T09:40:48.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:48.265+0000 7f15ec022740 1 --2- 172.21.15.5:0/3901259246 >> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] conn(0x5587a0f8ad60 0x5587a0f8d180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:40:48.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:48.265+0000 7f15ec022740 4 client.25877 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T09:40:48.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:48.265+0000 7f15ec022740 1 -- 172.21.15.5:0/3901259246 --> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5587a0e72e60 con 0x5587a0f8ad60 2024-09-10T09:40:48.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:48.265+0000 7f15e3200640 1 --2- 172.21.15.5:0/3901259246 >> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] conn(0x5587a0f8ad60 0x5587a0f8d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:40:48.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:48.265+0000 7f15e3200640 1 --2- 172.21.15.5:0/3901259246 >> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] conn(0x5587a0f8ad60 0x5587a0f8d180 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=1917fc790721415e server_cookie=b0991ab964f16875 in_seq=0 out_seq=0 2024-09-10T09:40:48.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:48.265+0000 7f15c3e00640 10 client.25877 ms_handle_connect on v2:172.21.15.102:6836/2689776764 2024-09-10T09:40:48.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:48.266+0000 7f15c3e00640 1 -- 172.21.15.5:0/3901259246 <== mds.3 v2:172.21.15.102:6836/2689776764 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5587a0e72e60 con 0x5587a0f8ad60 2024-09-10T09:40:48.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:48.266+0000 7f15c3e00640 10 client.25877 handle_command_reply: tid=0 2024-09-10T09:40:48.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:48.266+0000 7f15ec022740 1 client.25877 shutdown 2024-09-10T09:40:48.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:48.266+0000 7f15ec022740 1 -- 172.21.15.5:0/3901259246 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5587a0f05760 msgr2=0x5587a0f18780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:40:48.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:48.266+0000 7f15ec022740 1 --2- 172.21.15.5:0/3901259246 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5587a0f05760 0x5587a0f18780 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f15e4075cb0 tx=0x7f15d00108c0 comp rx=0 tx=0).stop 2024-09-10T09:40:48.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:48.266+0000 7f15ec022740 1 -- 172.21.15.5:0/3901259246 shutdown_connections 2024-09-10T09:40:48.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:48.266+0000 7f15ec022740 1 --2- 172.21.15.5:0/3901259246 >> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] conn(0x5587a0f8ad60 0x5587a0f8d180 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:48.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:48.266+0000 7f15ec022740 1 --2- 172.21.15.5:0/3901259246 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5587a0f10020 0x5587a0f09790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:48.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:48.266+0000 7f15ec022740 1 --2- 172.21.15.5:0/3901259246 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5587a0f05760 0x5587a0f18780 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:48.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:48.266+0000 7f15ec022740 1 --2- 172.21.15.5:0/3901259246 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5587a0f04e40 0x5587a0f18240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:48.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:48.266+0000 7f15ec022740 1 -- 172.21.15.5:0/3901259246 >> 172.21.15.5:0/3901259246 conn(0x5587a0f013d0 msgr2=0x5587a0f06ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:40:48.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:48.266+0000 7f15c3e00640 0 client.25877 ms_handle_reset on v2:172.21.15.102:6836/2689776764 2024-09-10T09:40:48.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:48.266+0000 7f15ec022740 1 -- 172.21.15.5:0/3901259246 shutdown_connections 2024-09-10T09:40:48.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:48.266+0000 7f15ec022740 1 -- 172.21.15.5:0/3901259246 wait complete. 2024-09-10T09:40:48.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:48.266+0000 7f15ec022740 20 client.25877 trim_cache size 0 max 16384 2024-09-10T09:40:48.272 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-10T09:40:48.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:48.267+0000 7f15e8a00640 1 -- 172.21.15.5:0/1693522015 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f15c80a2c00 msgr2=0x7f15c80a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:40:48.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:48.267+0000 7f15e8a00640 1 --2- 172.21.15.5:0/1693522015 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f15c80a2c00 0x7f15c80a50c0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f15d403d040 tx=0x7f15d403b720 comp rx=0 tx=0).stop 2024-09-10T09:40:48.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:48.267+0000 7f15e8a00640 1 -- 172.21.15.5:0/1693522015 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f15e41019c0 msgr2=0x7f15e40744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:40:48.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:48.267+0000 7f15e8a00640 1 --2- 172.21.15.5:0/1693522015 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f15e41019c0 0x7f15e40744f0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f15d402a9e0 tx=0x7f15d4003c10 comp rx=0 tx=0).stop 2024-09-10T09:40:48.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:48.267+0000 7f15e8a00640 1 -- 172.21.15.5:0/1693522015 shutdown_connections 2024-09-10T09:40:48.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:48.267+0000 7f15e8a00640 1 --2- 172.21.15.5:0/1693522015 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f15c80a2c00 0x7f15c80a50c0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:48.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:48.267+0000 7f15e8a00640 1 --2- 172.21.15.5:0/1693522015 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f15e41079c0 0x7f15e4074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:48.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:48.267+0000 7f15e8a00640 1 --2- 172.21.15.5:0/1693522015 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f15e4102360 0x7f15e4074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:48.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:48.267+0000 7f15e8a00640 1 --2- 172.21.15.5:0/1693522015 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f15e41019c0 0x7f15e40744f0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:48.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:48.267+0000 7f15e8a00640 1 -- 172.21.15.5:0/1693522015 >> 172.21.15.5:0/1693522015 conn(0x7f15e40fd740 msgr2=0x7f15e40fdf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:40:48.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:48.267+0000 7f15e8a00640 1 -- 172.21.15.5:0/1693522015 shutdown_connections 2024-09-10T09:40:48.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:48.267+0000 7f15e8a00640 1 -- 172.21.15.5:0/1693522015 wait complete. 2024-09-10T09:40:48.991 INFO:tasks.workunit.client.0.smithi005.stdout: 1 50411 9.63 MB/sec warmup 30 sec latency 34.842 ms 2024-09-10T09:40:49.238 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-10T09:40:49.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:40:49 smithi005 ceph-mon[30350]: pgmap v467: 129 pgs: 129 active+clean; 38 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 1.8 MiB/s wr, 207 op/s 2024-09-10T09:40:49.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:40:49 smithi159 ceph-mon[35513]: pgmap v467: 129 pgs: 129 active+clean; 38 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 1.8 MiB/s wr, 207 op/s 2024-09-10T09:40:49.637 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:40:49 smithi102 ceph-mon[35442]: pgmap v467: 129 pgs: 129 active+clean; 38 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 1.8 MiB/s wr, 207 op/s 2024-09-10T09:40:49.652 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:40:49.992 INFO:tasks.workunit.client.0.smithi005.stdout: 1 53242 9.87 MB/sec warmup 31 sec latency 13.641 ms 2024-09-10T09:40:50.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:50.267+0000 7fd618c00640 1 -- 172.21.15.5:0/221696730 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd6141049c0 msgr2=0x7fd614108a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:40:50.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:50.267+0000 7fd618c00640 1 --2- 172.21.15.5:0/221696730 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd6141049c0 0x7fd614108a10 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fd600009900 tx=0x7fd60002a400 comp rx=0 tx=0).stop 2024-09-10T09:40:50.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:50.267+0000 7fd618c00640 1 -- 172.21.15.5:0/221696730 shutdown_connections 2024-09-10T09:40:50.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:50.267+0000 7fd618c00640 1 --2- 172.21.15.5:0/221696730 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd614109140 0x7fd614110cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:50.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:50.267+0000 7fd618c00640 1 --2- 172.21.15.5:0/221696730 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd6141049c0 0x7fd614108a10 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:50.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:50.267+0000 7fd618c00640 1 --2- 172.21.15.5:0/221696730 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd614104010 0x7fd6141043f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:50.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:50.267+0000 7fd618c00640 1 -- 172.21.15.5:0/221696730 >> 172.21.15.5:0/221696730 conn(0x7fd6140ffa50 msgr2=0x7fd614101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:40:50.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:50.267+0000 7fd618c00640 1 -- 172.21.15.5:0/221696730 shutdown_connections 2024-09-10T09:40:50.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:50.267+0000 7fd618c00640 1 -- 172.21.15.5:0/221696730 wait complete. 2024-09-10T09:40:50.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:50.268+0000 7fd618c00640 1 Processor -- start 2024-09-10T09:40:50.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:50.269+0000 7fd618c00640 1 -- start start 2024-09-10T09:40:50.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:50.269+0000 7fd618c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd614104010 0x7fd6141a0400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:40:50.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:50.269+0000 7fd618c00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd6141049c0 0x7fd6141a0940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:40:50.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:50.269+0000 7fd618c00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd614109140 0x7fd61419a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:40:50.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:50.269+0000 7fd618c00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd614114190 con 0x7fd614109140 2024-09-10T09:40:50.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:50.269+0000 7fd618c00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fd614114010 con 0x7fd614104010 2024-09-10T09:40:50.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:50.269+0000 7fd618c00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fd614114310 con 0x7fd6141049c0 2024-09-10T09:40:50.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:50.269+0000 7fd611e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd614104010 0x7fd6141a0400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:40:50.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:50.269+0000 7fd611400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd6141049c0 0x7fd6141a0940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:40:50.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:50.269+0000 7fd612800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd614109140 0x7fd61419a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:40:50.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:50.269+0000 7fd611e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd614104010 0x7fd6141a0400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:37742/0 (socket says 172.21.15.5:37742) 2024-09-10T09:40:50.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:50.269+0000 7fd611e00640 1 -- 172.21.15.5:0/1549571359 learned_addr learned my addr 172.21.15.5:0/1549571359 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:40:50.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:50.270+0000 7fd611e00640 1 -- 172.21.15.5:0/1549571359 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd6141049c0 msgr2=0x7fd6141a0940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:40:50.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:50.270+0000 7fd611e00640 1 --2- 172.21.15.5:0/1549571359 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd6141049c0 0x7fd6141a0940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:50.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:50.270+0000 7fd611e00640 1 -- 172.21.15.5:0/1549571359 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd614109140 msgr2=0x7fd61419a5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:40:50.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:50.270+0000 7fd611e00640 1 --2- 172.21.15.5:0/1549571359 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd614109140 0x7fd61419a5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:50.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:50.270+0000 7fd611e00640 1 -- 172.21.15.5:0/1549571359 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd61419acf0 con 0x7fd614104010 2024-09-10T09:40:50.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:50.270+0000 7fd611e00640 1 --2- 172.21.15.5:0/1549571359 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd614104010 0x7fd6141a0400 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fd60400e9f0 tx=0x7fd60400edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:40:50.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:50.270+0000 7fd60aa00640 1 -- 172.21.15.5:0/1549571359 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd60400ccc0 con 0x7fd614104010 2024-09-10T09:40:50.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:50.270+0000 7fd60aa00640 1 -- 172.21.15.5:0/1549571359 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd604010910 con 0x7fd614104010 2024-09-10T09:40:50.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:50.270+0000 7fd60aa00640 1 -- 172.21.15.5:0/1549571359 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6040188a0 con 0x7fd614104010 2024-09-10T09:40:50.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:50.270+0000 7fd618c00640 1 -- 172.21.15.5:0/1549571359 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd61419afe0 con 0x7fd614104010 2024-09-10T09:40:50.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:50.271+0000 7fd618c00640 1 -- 172.21.15.5:0/1549571359 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6141a72d0 con 0x7fd614104010 2024-09-10T09:40:50.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:50.274+0000 7fd60aa00640 1 -- 172.21.15.5:0/1549571359 <== mon.2 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7fd604010ac0 con 0x7fd614104010 2024-09-10T09:40:50.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:50.274+0000 7fd61b6da740 1 Processor -- start 2024-09-10T09:40:50.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:50.274+0000 7fd61b6da740 1 -- start start 2024-09-10T09:40:50.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:50.275+0000 7fd61b6da740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b88f34fc70 0x55b88f350050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:40:50.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:50.275+0000 7fd61b6da740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b88f350590 0x55b88f35a910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:40:50.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:50.275+0000 7fd61b6da740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b88f35ae50 0x55b88f35d240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:40:50.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:50.275+0000 7fd61b6da740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55b88f2b63e0 con 0x55b88f34fc70 2024-09-10T09:40:50.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:50.275+0000 7fd61b6da740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55b88f25dd80 con 0x55b88f35ae50 2024-09-10T09:40:50.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:50.275+0000 7fd61b6da740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55b88f2b23a0 con 0x55b88f350590 2024-09-10T09:40:50.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:50.275+0000 7fd612800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b88f34fc70 0x55b88f350050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:40:50.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:50.275+0000 7fd612800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b88f34fc70 0x55b88f350050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38790/0 (socket says 172.21.15.5:38790) 2024-09-10T09:40:50.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:50.275+0000 7fd612800640 1 -- 172.21.15.5:0/2555744716 learned_addr learned my addr 172.21.15.5:0/2555744716 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:40:50.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:50.275+0000 7fd611400640 1 --2- 172.21.15.5:0/2555744716 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b88f35ae50 0x55b88f35d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:40:50.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:50.275+0000 7fd611e00640 1 --2- 172.21.15.5:0/2555744716 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b88f350590 0x55b88f35a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:40:50.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:50.276+0000 7fd612800640 1 -- 172.21.15.5:0/2555744716 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b88f350590 msgr2=0x55b88f35a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:40:50.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:50.276+0000 7fd612800640 1 --2- 172.21.15.5:0/2555744716 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b88f350590 0x55b88f35a910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:50.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:50.276+0000 7fd612800640 1 -- 172.21.15.5:0/2555744716 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b88f35ae50 msgr2=0x55b88f35d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:40:50.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:50.276+0000 7fd612800640 1 --2- 172.21.15.5:0/2555744716 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b88f35ae50 0x55b88f35d240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:50.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:50.276+0000 7fd612800640 1 -- 172.21.15.5:0/2555744716 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b88f27cae0 con 0x55b88f34fc70 2024-09-10T09:40:50.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:50.276+0000 7fd60aa00640 1 --2- 172.21.15.5:0/1549571359 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fd5f80a2c00 0x7fd5f80a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:40:50.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:50.276+0000 7fd612800640 1 --2- 172.21.15.5:0/2555744716 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b88f34fc70 0x55b88f350050 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fd60c009f60 tx=0x7fd60c009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a89c54a83c852ab4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:40:50.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:50.276+0000 7fd60aa00640 1 -- 172.21.15.5:0/1549571359 <== mon.2 v2:172.21.15.102:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fd6040cf7e0 con 0x7fd614104010 2024-09-10T09:40:50.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:50.276+0000 7fd611e00640 1 --2- 172.21.15.5:0/1549571359 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fd5f80a2c00 0x7fd5f80a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:40:50.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:50.277+0000 7fd5ff000640 1 -- 172.21.15.5:0/2555744716 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd60c01e070 con 0x55b88f34fc70 2024-09-10T09:40:50.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:50.277+0000 7fd5ff000640 1 -- 172.21.15.5:0/2555744716 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd60c003c30 con 0x55b88f34fc70 2024-09-10T09:40:50.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:50.277+0000 7fd5ff000640 1 -- 172.21.15.5:0/2555744716 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd60c00fa20 con 0x55b88f34fc70 2024-09-10T09:40:50.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:50.278+0000 7fd61b6da740 1 -- 172.21.15.5:0/2555744716 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b88f34fc70 msgr2=0x55b88f350050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:40:50.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:50.278+0000 7fd611e00640 1 --2- 172.21.15.5:0/1549571359 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fd5f80a2c00 0x7fd5f80a50c0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fd6040cd260 tx=0x7fd604009700 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:40:50.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:50.278+0000 7fd61b6da740 1 --2- 172.21.15.5:0/2555744716 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b88f34fc70 0x55b88f350050 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fd60c009f60 tx=0x7fd60c009f90 comp rx=0 tx=0).stop 2024-09-10T09:40:50.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:50.278+0000 7fd61b6da740 1 -- 172.21.15.5:0/2555744716 shutdown_connections 2024-09-10T09:40:50.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:50.278+0000 7fd61b6da740 1 --2- 172.21.15.5:0/2555744716 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b88f35ae50 0x55b88f35d240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:50.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:50.278+0000 7fd61b6da740 1 --2- 172.21.15.5:0/2555744716 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b88f350590 0x55b88f35a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:50.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:50.278+0000 7fd61b6da740 1 --2- 172.21.15.5:0/2555744716 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b88f34fc70 0x55b88f350050 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:50.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:50.278+0000 7fd61b6da740 1 -- 172.21.15.5:0/2555744716 >> 172.21.15.5:0/2555744716 conn(0x55b88f34c200 msgr2=0x55b88f34c5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:40:50.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:50.278+0000 7fd61b6da740 1 -- 172.21.15.5:0/2555744716 shutdown_connections 2024-09-10T09:40:50.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:50.278+0000 7fd61b6da740 1 -- 172.21.15.5:0/2555744716 wait complete. 2024-09-10T09:40:50.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:50.279+0000 7fd61b6da740 1 -- start start 2024-09-10T09:40:50.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:50.279+0000 7fd61b6da740 1 Processor -- start 2024-09-10T09:40:50.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:50.279+0000 7fd61b6da740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b88f34fc70 0x55b88f357d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:40:50.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:50.279+0000 7fd61b6da740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b88f350590 0x55b88f358280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:40:50.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:50.279+0000 7fd61b6da740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b88f35ae50 0x55b88f3587c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:40:50.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:50.279+0000 7fd61b6da740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55b88f35da20 con 0x55b88f34fc70 2024-09-10T09:40:50.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:50.279+0000 7fd61b6da740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55b88f2710f0 con 0x55b88f35ae50 2024-09-10T09:40:50.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:50.279+0000 7fd61b6da740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55b88f35dba0 con 0x55b88f350590 2024-09-10T09:40:50.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:50.279+0000 7fd61b6da740 10 client.0 _ll_register_callbacks cb 0x55b88f258bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:40:50.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:50.280+0000 7fd612800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b88f34fc70 0x55b88f357d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:40:50.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:50.280+0000 7fd611400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b88f350590 0x55b88f358280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:40:50.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:50.280+0000 7fd612800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b88f34fc70 0x55b88f357d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38798/0 (socket says 172.21.15.5:38798) 2024-09-10T09:40:50.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:50.280+0000 7fd612800640 1 -- 172.21.15.5:0/2774970719 learned_addr learned my addr 172.21.15.5:0/2774970719 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:40:50.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:50.280+0000 7fd612800640 1 --2- 172.21.15.5:0/2774970719 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b88f35ae50 0x55b88f3587c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:40:50.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:50.280+0000 7fd611400640 1 -- 172.21.15.5:0/2774970719 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b88f35ae50 msgr2=0x55b88f3587c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:40:50.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:50.280+0000 7fd611400640 1 --2- 172.21.15.5:0/2774970719 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b88f35ae50 0x55b88f3587c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:50.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:50.280+0000 7fd611400640 1 -- 172.21.15.5:0/2774970719 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b88f34fc70 msgr2=0x55b88f357d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:40:50.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:50.280+0000 7fd611400640 1 --2- 172.21.15.5:0/2774970719 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b88f34fc70 0x55b88f357d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:50.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:50.280+0000 7fd611400640 1 -- 172.21.15.5:0/2774970719 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b88f0273c0 con 0x55b88f350590 2024-09-10T09:40:50.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:50.280+0000 7fd612800640 1 --2- 172.21.15.5:0/2774970719 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b88f35ae50 0x55b88f3587c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:40:50.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:50.281+0000 7fd611400640 1 --2- 172.21.15.5:0/2774970719 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b88f350590 0x55b88f358280 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fd60002a9f0 tx=0x7fd600004390 comp rx=0 tx=0).ready entity=mon.1 client_cookie=11758bd63289416b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:40:50.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:50.281+0000 7fd5f3400640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-09-10T09:40:50.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:50.281+0000 7fd5f3400640 1 -- 172.21.15.5:0/2774970719 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd600035040 con 0x55b88f350590 2024-09-10T09:40:50.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:50.281+0000 7fd5f3400640 1 -- 172.21.15.5:0/2774970719 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fd600005450 con 0x55b88f350590 2024-09-10T09:40:50.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:50.281+0000 7fd61b6da740 1 -- 172.21.15.5:0/2774970719 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd60c003c30 con 0x55b88f350590 2024-09-10T09:40:50.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:50.281+0000 7fd5f3400640 1 -- 172.21.15.5:0/2774970719 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fd60004f050 con 0x55b88f350590 2024-09-10T09:40:50.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:50.281+0000 7fd61b6da740 10 client.25921 fetch_fsmap learned FSMap version 32 2024-09-10T09:40:50.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:50.281+0000 7fd61b6da740 1 -- 172.21.15.5:0/2774970719 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55b88f372270 con 0x55b88f350590 2024-09-10T09:40:50.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:50.282+0000 7fd5f3400640 1 -- 172.21.15.5:0/2774970719 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd60c003c30 con 0x55b88f350590 2024-09-10T09:40:50.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:50.282+0000 7fd5f3400640 1 -- 172.21.15.5:0/2774970719 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7fd6000317f0 con 0x55b88f350590 2024-09-10T09:40:50.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:50.282+0000 7fd61b6da740 10 client.25921 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T09:40:50.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:50.282+0000 7fd61b6da740 10 client.25921 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-10T09:40:50.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:50.282+0000 7fd61b6da740 20 client.25921 populate_metadata read hostname 'smithi005' 2024-09-10T09:40:50.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:50.282+0000 7fd61b6da740 1 --2- 172.21.15.5:0/2774970719 >> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] conn(0x55b88f373730 0x55b88f375b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:40:50.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:50.282+0000 7fd61b6da740 4 client.25921 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:40:50.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:50.282+0000 7fd61b6da740 1 -- 172.21.15.5:0/2774970719 --> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b88f2c3950 con 0x55b88f373730 2024-09-10T09:40:50.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:50.282+0000 7fd612800640 1 --2- 172.21.15.5:0/2774970719 >> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] conn(0x55b88f373730 0x55b88f375b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:40:50.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:50.283+0000 7fd612800640 1 --2- 172.21.15.5:0/2774970719 >> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] conn(0x55b88f373730 0x55b88f375b20 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8b2fd86737939d16 server_cookie=28f936fbdf2d862c in_seq=0 out_seq=0 2024-09-10T09:40:50.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:50.283+0000 7fd5f3400640 10 client.25921 ms_handle_connect on v2:172.21.15.159:6836/620627639 2024-09-10T09:40:50.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:50.284+0000 7fd5f3400640 1 -- 172.21.15.5:0/2774970719 <== mds.2 v2:172.21.15.159:6836/620627639 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b88f2c3950 con 0x55b88f373730 2024-09-10T09:40:50.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:50.284+0000 7fd5f3400640 10 client.25921 handle_command_reply: tid=0 2024-09-10T09:40:50.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:50.284+0000 7fd61b6da740 1 client.25921 shutdown 2024-09-10T09:40:50.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:50.284+0000 7fd61b6da740 1 -- 172.21.15.5:0/2774970719 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b88f350590 msgr2=0x55b88f358280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:40:50.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:50.284+0000 7fd61b6da740 1 --2- 172.21.15.5:0/2774970719 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b88f350590 0x55b88f358280 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fd60002a9f0 tx=0x7fd600004390 comp rx=0 tx=0).stop 2024-09-10T09:40:50.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:50.284+0000 7fd61b6da740 1 -- 172.21.15.5:0/2774970719 shutdown_connections 2024-09-10T09:40:50.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:50.284+0000 7fd61b6da740 1 --2- 172.21.15.5:0/2774970719 >> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] conn(0x55b88f373730 0x55b88f375b20 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:50.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:50.284+0000 7fd61b6da740 1 --2- 172.21.15.5:0/2774970719 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b88f35ae50 0x55b88f3587c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:50.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:50.284+0000 7fd61b6da740 1 --2- 172.21.15.5:0/2774970719 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b88f350590 0x55b88f358280 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:50.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:50.284+0000 7fd61b6da740 1 --2- 172.21.15.5:0/2774970719 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b88f34fc70 0x55b88f357d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:50.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:50.284+0000 7fd61b6da740 1 -- 172.21.15.5:0/2774970719 >> 172.21.15.5:0/2774970719 conn(0x55b88f34c200 msgr2=0x55b88f34b380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:40:50.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:50.284+0000 7fd5f3400640 0 client.25921 ms_handle_reset on v2:172.21.15.159:6836/620627639 2024-09-10T09:40:50.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:50.285+0000 7fd61b6da740 1 -- 172.21.15.5:0/2774970719 shutdown_connections 2024-09-10T09:40:50.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:50.285+0000 7fd61b6da740 1 -- 172.21.15.5:0/2774970719 wait complete. 2024-09-10T09:40:50.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:50.285+0000 7fd61b6da740 20 client.25921 trim_cache size 0 max 16384 2024-09-10T09:40:50.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:50.300+0000 7fd61b6da740 1 Processor -- start 2024-09-10T09:40:50.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:50.300+0000 7fd61b6da740 1 -- start start 2024-09-10T09:40:50.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:50.300+0000 7fd61b6da740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b88f34fc70 0x55b88f3526d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:40:50.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:50.300+0000 7fd61b6da740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b88f350590 0x55b88f359cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:40:50.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:50.300+0000 7fd61b6da740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b88f35ae50 0x55b88f37b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:40:50.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:50.300+0000 7fd61b6da740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd6141accb0 con 0x55b88f35ae50 2024-09-10T09:40:50.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:50.300+0000 7fd61b6da740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fd6141ae8b0 con 0x55b88f350590 2024-09-10T09:40:50.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:50.300+0000 7fd61b6da740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fd6141ad380 con 0x55b88f34fc70 2024-09-10T09:40:50.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:50.300+0000 7fd612800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b88f34fc70 0x55b88f3526d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:40:50.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:50.300+0000 7fd611400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b88f350590 0x55b88f359cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:40:50.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:50.300+0000 7fd612800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b88f35ae50 0x55b88f37b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:40:50.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:50.300+0000 7fd611400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b88f350590 0x55b88f359cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:37754/0 (socket says 172.21.15.5:37754) 2024-09-10T09:40:50.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:50.300+0000 7fd611400640 1 -- 172.21.15.5:0/4176774196 learned_addr learned my addr 172.21.15.5:0/4176774196 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:40:50.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:50.301+0000 7fd611400640 1 -- 172.21.15.5:0/4176774196 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b88f34fc70 msgr2=0x55b88f3526d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:40:50.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:50.301+0000 7fd611400640 1 --2- 172.21.15.5:0/4176774196 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b88f34fc70 0x55b88f3526d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:50.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:50.301+0000 7fd611400640 1 -- 172.21.15.5:0/4176774196 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b88f35ae50 msgr2=0x55b88f37b3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:40:50.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:50.301+0000 7fd611400640 1 --2- 172.21.15.5:0/4176774196 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b88f35ae50 0x55b88f37b3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:50.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:50.301+0000 7fd611400640 1 -- 172.21.15.5:0/4176774196 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b88f347770 con 0x55b88f350590 2024-09-10T09:40:50.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:50.301+0000 7fd611400640 1 --2- 172.21.15.5:0/4176774196 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b88f350590 0x55b88f359cb0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fd600031d60 tx=0x7fd600005fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fdc38e7d6c753f36 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:40:50.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:50.301+0000 7fd5fe600640 1 -- 172.21.15.5:0/4176774196 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd600035030 con 0x55b88f350590 2024-09-10T09:40:50.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:50.301+0000 7fd5fe600640 1 -- 172.21.15.5:0/4176774196 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd600041070 con 0x55b88f350590 2024-09-10T09:40:50.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:50.301+0000 7fd5fe600640 1 -- 172.21.15.5:0/4176774196 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd600004ae0 con 0x55b88f350590 2024-09-10T09:40:50.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:50.302+0000 7fd61b6da740 1 -- 172.21.15.5:0/4176774196 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b88f350590 msgr2=0x55b88f359cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:40:50.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:50.302+0000 7fd61b6da740 1 --2- 172.21.15.5:0/4176774196 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b88f350590 0x55b88f359cb0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fd600031d60 tx=0x7fd600005fd0 comp rx=0 tx=0).stop 2024-09-10T09:40:50.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:50.302+0000 7fd61b6da740 1 -- 172.21.15.5:0/4176774196 shutdown_connections 2024-09-10T09:40:50.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:50.302+0000 7fd61b6da740 1 --2- 172.21.15.5:0/4176774196 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b88f35ae50 0x55b88f37b3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:50.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:50.302+0000 7fd61b6da740 1 --2- 172.21.15.5:0/4176774196 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b88f350590 0x55b88f359cb0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:50.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:50.302+0000 7fd61b6da740 1 --2- 172.21.15.5:0/4176774196 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b88f34fc70 0x55b88f3526d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:50.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:50.302+0000 7fd61b6da740 1 -- 172.21.15.5:0/4176774196 >> 172.21.15.5:0/4176774196 conn(0x55b88f34c200 msgr2=0x55b88f3595a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:40:50.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:50.302+0000 7fd61b6da740 1 -- 172.21.15.5:0/4176774196 shutdown_connections 2024-09-10T09:40:50.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:50.302+0000 7fd61b6da740 1 -- 172.21.15.5:0/4176774196 wait complete. 2024-09-10T09:40:50.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:50.303+0000 7fd61b6da740 1 -- start start 2024-09-10T09:40:50.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:50.303+0000 7fd61b6da740 1 Processor -- start 2024-09-10T09:40:50.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:50.303+0000 7fd61b6da740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b88f34fc70 0x55b88f36bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:40:50.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:50.303+0000 7fd61b6da740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b88f350590 0x55b88f367010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:40:50.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:50.303+0000 7fd61b6da740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b88f35ae50 0x55b88f367550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:40:50.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:50.303+0000 7fd61b6da740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd6141ae120 con 0x55b88f34fc70 2024-09-10T09:40:50.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:50.303+0000 7fd61b6da740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fd6141ada50 con 0x55b88f350590 2024-09-10T09:40:50.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:50.303+0000 7fd61b6da740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fd6141aac90 con 0x55b88f35ae50 2024-09-10T09:40:50.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:50.303+0000 7fd61b6da740 10 client.0 _ll_register_callbacks cb 0x7fd600009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:40:50.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:50.304+0000 7fd612800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b88f34fc70 0x55b88f36bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:40:50.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:50.304+0000 7fd611400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b88f350590 0x55b88f367010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:40:50.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:50.304+0000 7fd612800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b88f34fc70 0x55b88f36bc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38820/0 (socket says 172.21.15.5:38820) 2024-09-10T09:40:50.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:50.304+0000 7fd612800640 1 -- 172.21.15.5:0/128072247 learned_addr learned my addr 172.21.15.5:0/128072247 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:40:50.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:50.304+0000 7fd612800640 1 --2- 172.21.15.5:0/128072247 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b88f35ae50 0x55b88f367550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:40:50.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:50.304+0000 7fd611400640 1 -- 172.21.15.5:0/128072247 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b88f35ae50 msgr2=0x55b88f367550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:40:50.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:50.304+0000 7fd611400640 1 --2- 172.21.15.5:0/128072247 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b88f35ae50 0x55b88f367550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:50.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:50.304+0000 7fd611400640 1 -- 172.21.15.5:0/128072247 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b88f34fc70 msgr2=0x55b88f36bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:40:50.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:50.304+0000 7fd611400640 1 --2- 172.21.15.5:0/128072247 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b88f34fc70 0x55b88f36bc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:50.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:50.304+0000 7fd611400640 1 -- 172.21.15.5:0/128072247 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b88f367c40 con 0x55b88f350590 2024-09-10T09:40:50.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:50.305+0000 7fd611400640 1 --2- 172.21.15.5:0/128072247 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b88f350590 0x55b88f367010 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fd600031ce0 tx=0x7fd600005480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2ac5426c4e032663 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:40:50.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:50.305+0000 7fd5f3400640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-09-10T09:40:50.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:50.305+0000 7fd5f3400640 1 -- 172.21.15.5:0/128072247 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd600035070 con 0x55b88f350590 2024-09-10T09:40:50.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:50.305+0000 7fd61b6da740 1 -- 172.21.15.5:0/128072247 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd600041070 con 0x55b88f350590 2024-09-10T09:40:50.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:50.305+0000 7fd5f3400640 1 -- 172.21.15.5:0/128072247 <== mon.2 v2:172.21.15.102:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fd6000317c0 con 0x55b88f350590 2024-09-10T09:40:50.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:50.305+0000 7fd5f3400640 1 -- 172.21.15.5:0/128072247 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fd600064050 con 0x55b88f350590 2024-09-10T09:40:50.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:50.305+0000 7fd61b6da740 10 client.25895 fetch_fsmap learned FSMap version 32 2024-09-10T09:40:50.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:50.305+0000 7fd61b6da740 1 -- 172.21.15.5:0/128072247 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55b88f3de240 con 0x55b88f350590 2024-09-10T09:40:50.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:50.305+0000 7fd5f3400640 1 -- 172.21.15.5:0/128072247 <== mon.2 v2:172.21.15.102:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd600041070 con 0x55b88f350590 2024-09-10T09:40:50.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:50.306+0000 7fd5f3400640 1 -- 172.21.15.5:0/128072247 <== mon.2 v2:172.21.15.102:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7fd600003870 con 0x55b88f350590 2024-09-10T09:40:50.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:50.306+0000 7fd61b6da740 10 client.25895 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T09:40:50.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:50.306+0000 7fd61b6da740 10 client.25895 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-10T09:40:50.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:50.306+0000 7fd61b6da740 20 client.25895 populate_metadata read hostname 'smithi005' 2024-09-10T09:40:50.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:50.306+0000 7fd61b6da740 1 --2- 172.21.15.5:0/128072247 >> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] conn(0x55b88f3de8d0 0x55b88f3e0cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:40:50.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:50.306+0000 7fd61b6da740 4 client.25895 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T09:40:50.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:50.306+0000 7fd61b6da740 1 -- 172.21.15.5:0/128072247 --> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b88f353960 con 0x55b88f3de8d0 2024-09-10T09:40:50.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:50.306+0000 7fd612800640 1 --2- 172.21.15.5:0/128072247 >> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] conn(0x55b88f3de8d0 0x55b88f3e0cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:40:50.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:50.306+0000 7fd612800640 1 --2- 172.21.15.5:0/128072247 >> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] conn(0x55b88f3de8d0 0x55b88f3e0cf0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=42ea72935067a2ba server_cookie=7a024d50723c502a in_seq=0 out_seq=0 2024-09-10T09:40:50.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:50.306+0000 7fd5f3400640 10 client.25895 ms_handle_connect on v2:172.21.15.159:6836/620627639 2024-09-10T09:40:50.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:50.307+0000 7fd5f3400640 1 -- 172.21.15.5:0/128072247 <== mds.2 v2:172.21.15.159:6836/620627639 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b88f353960 con 0x55b88f3de8d0 2024-09-10T09:40:50.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:50.307+0000 7fd5f3400640 10 client.25895 handle_command_reply: tid=0 2024-09-10T09:40:50.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:50.307+0000 7fd61b6da740 1 client.25895 shutdown 2024-09-10T09:40:50.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:50.307+0000 7fd61b6da740 1 -- 172.21.15.5:0/128072247 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b88f350590 msgr2=0x55b88f367010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:40:50.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:50.307+0000 7fd61b6da740 1 --2- 172.21.15.5:0/128072247 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b88f350590 0x55b88f367010 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fd600031ce0 tx=0x7fd600005480 comp rx=0 tx=0).stop 2024-09-10T09:40:50.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:50.307+0000 7fd61b6da740 1 -- 172.21.15.5:0/128072247 shutdown_connections 2024-09-10T09:40:50.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:50.307+0000 7fd61b6da740 1 --2- 172.21.15.5:0/128072247 >> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] conn(0x55b88f3de8d0 0x55b88f3e0cf0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:50.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:50.307+0000 7fd61b6da740 1 --2- 172.21.15.5:0/128072247 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b88f35ae50 0x55b88f367550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:50.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:50.307+0000 7fd61b6da740 1 --2- 172.21.15.5:0/128072247 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b88f350590 0x55b88f367010 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:50.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:50.307+0000 7fd5f3400640 0 client.25895 ms_handle_reset on v2:172.21.15.159:6836/620627639 2024-09-10T09:40:50.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:50.307+0000 7fd61b6da740 1 --2- 172.21.15.5:0/128072247 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b88f34fc70 0x55b88f36bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:50.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:50.307+0000 7fd61b6da740 1 -- 172.21.15.5:0/128072247 >> 172.21.15.5:0/128072247 conn(0x55b88f34c200 msgr2=0x55b88f357cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:40:50.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:50.308+0000 7fd61b6da740 1 -- 172.21.15.5:0/128072247 shutdown_connections 2024-09-10T09:40:50.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:50.308+0000 7fd61b6da740 1 -- 172.21.15.5:0/128072247 wait complete. 2024-09-10T09:40:50.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:50.308+0000 7fd61b6da740 20 client.25895 trim_cache size 0 max 16384 2024-09-10T09:40:50.312 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-10T09:40:50.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:50.309+0000 7fd618c00640 1 -- 172.21.15.5:0/1549571359 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fd5f80a2c00 msgr2=0x7fd5f80a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:40:50.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:50.309+0000 7fd618c00640 1 --2- 172.21.15.5:0/1549571359 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fd5f80a2c00 0x7fd5f80a50c0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fd6040cd260 tx=0x7fd604009700 comp rx=0 tx=0).stop 2024-09-10T09:40:50.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:50.309+0000 7fd618c00640 1 -- 172.21.15.5:0/1549571359 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd614104010 msgr2=0x7fd6141a0400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:40:50.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:50.309+0000 7fd618c00640 1 --2- 172.21.15.5:0/1549571359 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd614104010 0x7fd6141a0400 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fd60400e9f0 tx=0x7fd60400edf0 comp rx=0 tx=0).stop 2024-09-10T09:40:50.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:50.309+0000 7fd618c00640 1 -- 172.21.15.5:0/1549571359 shutdown_connections 2024-09-10T09:40:50.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:50.309+0000 7fd618c00640 1 --2- 172.21.15.5:0/1549571359 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fd5f80a2c00 0x7fd5f80a50c0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:50.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:50.309+0000 7fd618c00640 1 --2- 172.21.15.5:0/1549571359 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd614109140 0x7fd61419a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:50.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:50.309+0000 7fd618c00640 1 --2- 172.21.15.5:0/1549571359 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd6141049c0 0x7fd6141a0940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:50.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:50.309+0000 7fd618c00640 1 --2- 172.21.15.5:0/1549571359 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd614104010 0x7fd6141a0400 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:50.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:50.309+0000 7fd618c00640 1 -- 172.21.15.5:0/1549571359 >> 172.21.15.5:0/1549571359 conn(0x7fd6140ffa50 msgr2=0x7fd614100e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:40:50.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:50.309+0000 7fd618c00640 1 -- 172.21.15.5:0/1549571359 shutdown_connections 2024-09-10T09:40:50.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:50.309+0000 7fd618c00640 1 -- 172.21.15.5:0/1549571359 wait complete. 2024-09-10T09:40:50.992 INFO:tasks.workunit.client.0.smithi005.stdout: 1 55012 9.78 MB/sec warmup 32 sec latency 25.123 ms 2024-09-10T09:40:51.529 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-10T09:40:51.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:40:51 smithi102 ceph-mon[35442]: pgmap v468: 129 pgs: 129 active+clean; 40 MiB data, 494 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 1.6 MiB/s wr, 220 op/s 2024-09-10T09:40:51.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:40:51 smithi005 ceph-mon[30350]: pgmap v468: 129 pgs: 129 active+clean; 40 MiB data, 494 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 1.6 MiB/s wr, 220 op/s 2024-09-10T09:40:51.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:40:51 smithi159 ceph-mon[35513]: pgmap v468: 129 pgs: 129 active+clean; 40 MiB data, 494 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 1.6 MiB/s wr, 220 op/s 2024-09-10T09:40:51.924 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:40:51.992 INFO:tasks.workunit.client.0.smithi005.stdout: 1 57469 9.91 MB/sec warmup 33 sec latency 14.386 ms 2024-09-10T09:40:52.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:52.531+0000 7f024a600640 1 -- 172.21.15.5:0/2939451967 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0244108f60 msgr2=0x7f0244110ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:40:52.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:52.531+0000 7f024a600640 1 --2- 172.21.15.5:0/2939451967 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0244108f60 0x7f0244110ae0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f023c009ca0 tx=0x7f023c02a8c0 comp rx=0 tx=0).stop 2024-09-10T09:40:52.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:52.531+0000 7f024a600640 1 -- 172.21.15.5:0/2939451967 shutdown_connections 2024-09-10T09:40:52.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:52.531+0000 7f024a600640 1 --2- 172.21.15.5:0/2939451967 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0244108f60 0x7f0244110ae0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:52.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:52.531+0000 7f024a600640 1 --2- 172.21.15.5:0/2939451967 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f02441047e0 0x7f0244108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:52.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:52.531+0000 7f024a600640 1 --2- 172.21.15.5:0/2939451967 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0244103e30 0x7f0244104210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:52.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:52.531+0000 7f024a600640 1 -- 172.21.15.5:0/2939451967 >> 172.21.15.5:0/2939451967 conn(0x7f02440ff910 msgr2=0x7f0244101d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:40:52.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:52.531+0000 7f024a600640 1 -- 172.21.15.5:0/2939451967 shutdown_connections 2024-09-10T09:40:52.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:52.532+0000 7f024a600640 1 -- 172.21.15.5:0/2939451967 wait complete. 2024-09-10T09:40:52.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:52.533+0000 7f024a600640 1 Processor -- start 2024-09-10T09:40:52.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:52.533+0000 7f024a600640 1 -- start start 2024-09-10T09:40:52.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:52.533+0000 7f024a600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0244103e30 0x7f024419a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:40:52.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:52.533+0000 7f024a600640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f02441047e0 0x7f024419a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:40:52.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:52.533+0000 7f024a600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0244108f60 0x7f024419ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:40:52.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:52.533+0000 7f024a600640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f0244113fc0 con 0x7f0244103e30 2024-09-10T09:40:52.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:52.533+0000 7f024a600640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f0244113e40 con 0x7f0244108f60 2024-09-10T09:40:52.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:52.533+0000 7f024a600640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f0244114140 con 0x7f02441047e0 2024-09-10T09:40:52.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:52.534+0000 7f0249200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0244103e30 0x7f024419a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:40:52.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:52.534+0000 7f0249c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0244108f60 0x7f024419ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:40:52.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:52.534+0000 7f0249200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0244103e30 0x7f024419a460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43292/0 (socket says 172.21.15.5:43292) 2024-09-10T09:40:52.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:52.534+0000 7f0249200640 1 -- 172.21.15.5:0/98367550 learned_addr learned my addr 172.21.15.5:0/98367550 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:40:52.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:52.534+0000 7f0243e00640 1 --2- 172.21.15.5:0/98367550 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f02441047e0 0x7f024419a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:40:52.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:52.534+0000 7f0249200640 1 -- 172.21.15.5:0/98367550 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f02441047e0 msgr2=0x7f024419a9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:40:52.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:52.534+0000 7f0249200640 1 --2- 172.21.15.5:0/98367550 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f02441047e0 0x7f024419a9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:52.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:52.534+0000 7f0249200640 1 -- 172.21.15.5:0/98367550 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0244108f60 msgr2=0x7f024419ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:40:52.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:52.534+0000 7f0249200640 1 --2- 172.21.15.5:0/98367550 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0244108f60 0x7f024419ed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:52.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:52.534+0000 7f0249200640 1 -- 172.21.15.5:0/98367550 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f024419f410 con 0x7f0244103e30 2024-09-10T09:40:52.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:52.535+0000 7f0249200640 1 --2- 172.21.15.5:0/98367550 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0244103e30 0x7f024419a460 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f023800e9f0 tx=0x7f023800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:40:52.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:52.535+0000 7f0241600640 1 -- 172.21.15.5:0/98367550 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f023800cd80 con 0x7f0244103e30 2024-09-10T09:40:52.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:52.535+0000 7f0241600640 1 -- 172.21.15.5:0/98367550 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0238005500 con 0x7f0244103e30 2024-09-10T09:40:52.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:52.535+0000 7f024a600640 1 -- 172.21.15.5:0/98367550 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f024419f700 con 0x7f0244103e30 2024-09-10T09:40:52.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:52.535+0000 7f0241600640 1 -- 172.21.15.5:0/98367550 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0238018630 con 0x7f0244103e30 2024-09-10T09:40:52.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:52.535+0000 7f024a600640 1 -- 172.21.15.5:0/98367550 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02441a6fe0 con 0x7f0244103e30 2024-09-10T09:40:52.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:52.539+0000 7f024e81f740 1 Processor -- start 2024-09-10T09:40:52.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:52.539+0000 7f024e81f740 1 -- start start 2024-09-10T09:40:52.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:52.539+0000 7f0241600640 1 -- 172.21.15.5:0/98367550 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f02380046c0 con 0x7f0244103e30 2024-09-10T09:40:52.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:52.539+0000 7f024e81f740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562a1091ce40 0x562a1091d220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:40:52.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:52.539+0000 7f024e81f740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562a1091d760 0x562a10927ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:40:52.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:52.539+0000 7f024e81f740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562a10928020 0x562a1092a410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:40:52.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:52.539+0000 7f024e81f740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x562a108833e0 con 0x562a10928020 2024-09-10T09:40:52.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:52.539+0000 7f024e81f740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x562a1082ad80 con 0x562a1091d760 2024-09-10T09:40:52.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:52.539+0000 7f024e81f740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x562a1087f3a0 con 0x562a1091ce40 2024-09-10T09:40:52.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:52.539+0000 7f0249c00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562a1091ce40 0x562a1091d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:40:52.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:52.539+0000 7f0249c00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562a1091ce40 0x562a1091d220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:56908/0 (socket says 172.21.15.5:56908) 2024-09-10T09:40:52.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:52.540+0000 7f0249c00640 1 -- 172.21.15.5:0/619520936 learned_addr learned my addr 172.21.15.5:0/619520936 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:40:52.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:52.540+0000 7f0249200640 1 --2- 172.21.15.5:0/619520936 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562a1091d760 0x562a10927ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:40:52.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:52.540+0000 7f0243e00640 1 --2- 172.21.15.5:0/619520936 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562a10928020 0x562a1092a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:40:52.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:52.540+0000 7f0249c00640 1 -- 172.21.15.5:0/619520936 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562a1091d760 msgr2=0x562a10927ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:40:52.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:52.540+0000 7f0249c00640 1 --2- 172.21.15.5:0/619520936 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562a1091d760 0x562a10927ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:52.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:52.540+0000 7f0249c00640 1 -- 172.21.15.5:0/619520936 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562a10928020 msgr2=0x562a1092a410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:40:52.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:52.541+0000 7f0249c00640 1 --2- 172.21.15.5:0/619520936 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562a10928020 0x562a1092a410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:52.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:52.541+0000 7f0249c00640 1 -- 172.21.15.5:0/619520936 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562a10849ae0 con 0x562a1091ce40 2024-09-10T09:40:52.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:52.541+0000 7f0243e00640 1 --2- 172.21.15.5:0/619520936 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562a10928020 0x562a1092a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:40:52.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:52.541+0000 7f0241600640 1 --2- 172.21.15.5:0/98367550 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f022c0a2b80 0x7f022c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:40:52.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:52.541+0000 7f0249c00640 1 --2- 172.21.15.5:0/619520936 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562a1091ce40 0x562a1091d220 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f023c009ca0 tx=0x7f023c005f10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=29083c841359b78b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:40:52.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:52.541+0000 7f0241600640 1 -- 172.21.15.5:0/98367550 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f02380cf970 con 0x7f0244103e30 2024-09-10T09:40:52.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:52.541+0000 7f0249200640 1 --2- 172.21.15.5:0/98367550 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f022c0a2b80 0x7f022c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:40:52.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:52.541+0000 7f0236000640 1 -- 172.21.15.5:0/619520936 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f023c031db0 con 0x562a1091ce40 2024-09-10T09:40:52.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:52.541+0000 7f0236000640 1 -- 172.21.15.5:0/619520936 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f023c036040 con 0x562a1091ce40 2024-09-10T09:40:52.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:52.541+0000 7f0236000640 1 -- 172.21.15.5:0/619520936 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f023c004330 con 0x562a1091ce40 2024-09-10T09:40:52.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:52.542+0000 7f024e81f740 1 -- 172.21.15.5:0/619520936 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562a1091ce40 msgr2=0x562a1091d220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:40:52.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:52.542+0000 7f024e81f740 1 --2- 172.21.15.5:0/619520936 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562a1091ce40 0x562a1091d220 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f023c009ca0 tx=0x7f023c005f10 comp rx=0 tx=0).stop 2024-09-10T09:40:52.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:52.542+0000 7f0249200640 1 --2- 172.21.15.5:0/98367550 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f022c0a2b80 0x7f022c0a5040 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f0238098750 tx=0x7f0238004a40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:40:52.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:52.542+0000 7f024e81f740 1 -- 172.21.15.5:0/619520936 shutdown_connections 2024-09-10T09:40:52.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:52.542+0000 7f024e81f740 1 --2- 172.21.15.5:0/619520936 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562a10928020 0x562a1092a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:52.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:52.542+0000 7f024e81f740 1 --2- 172.21.15.5:0/619520936 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562a1091d760 0x562a10927ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:52.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:52.542+0000 7f024e81f740 1 --2- 172.21.15.5:0/619520936 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562a1091ce40 0x562a1091d220 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:52.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:52.542+0000 7f024e81f740 1 -- 172.21.15.5:0/619520936 >> 172.21.15.5:0/619520936 conn(0x562a109193d0 msgr2=0x562a109197b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:40:52.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:52.543+0000 7f024e81f740 1 -- 172.21.15.5:0/619520936 shutdown_connections 2024-09-10T09:40:52.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:52.543+0000 7f024e81f740 1 -- 172.21.15.5:0/619520936 wait complete. 2024-09-10T09:40:52.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:52.543+0000 7f024e81f740 1 -- start start 2024-09-10T09:40:52.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:52.543+0000 7f024e81f740 1 Processor -- start 2024-09-10T09:40:52.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:52.544+0000 7f024e81f740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562a1091ce40 0x562a10924f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:40:52.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:52.544+0000 7f024e81f740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562a1091d760 0x562a10925440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:40:52.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:52.544+0000 7f024e81f740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562a10928020 0x562a10925980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:40:52.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:52.544+0000 7f024e81f740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x562a1092aa70 con 0x562a10928020 2024-09-10T09:40:52.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:52.544+0000 7f024e81f740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x562a1083e0f0 con 0x562a1091d760 2024-09-10T09:40:52.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:52.544+0000 7f024e81f740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x562a1092abf0 con 0x562a1091ce40 2024-09-10T09:40:52.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:52.544+0000 7f024e81f740 10 client.0 _ll_register_callbacks cb 0x562a10825bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:40:52.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:52.544+0000 7f0243e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562a1091d760 0x562a10925440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:40:52.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:52.544+0000 7f0243e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562a1091d760 0x562a10925440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:41048/0 (socket says 172.21.15.5:41048) 2024-09-10T09:40:52.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:52.544+0000 7f0243e00640 1 -- 172.21.15.5:0/2022152027 learned_addr learned my addr 172.21.15.5:0/2022152027 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:40:52.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:52.544+0000 7f0249c00640 1 --2- 172.21.15.5:0/2022152027 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562a1091ce40 0x562a10924f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:40:52.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:52.544+0000 7f0249c00640 1 --2- 172.21.15.5:0/2022152027 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562a10928020 0x562a10925980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:40:52.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:52.545+0000 7f0243e00640 1 -- 172.21.15.5:0/2022152027 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562a1091ce40 msgr2=0x562a10924f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:40:52.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:52.545+0000 7f0243e00640 1 --2- 172.21.15.5:0/2022152027 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562a1091ce40 0x562a10924f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:52.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:52.545+0000 7f0243e00640 1 -- 172.21.15.5:0/2022152027 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562a10928020 msgr2=0x562a10925980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:40:52.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:52.545+0000 7f0243e00640 1 --2- 172.21.15.5:0/2022152027 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562a10928020 0x562a10925980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:52.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:52.545+0000 7f0243e00640 1 -- 172.21.15.5:0/2022152027 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562a105f43c0 con 0x562a1091d760 2024-09-10T09:40:52.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:52.545+0000 7f0249c00640 1 --2- 172.21.15.5:0/2022152027 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562a1091ce40 0x562a10924f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:40:52.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:52.545+0000 7f0243e00640 1 --2- 172.21.15.5:0/2022152027 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562a1091d760 0x562a10925440 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f0230012700 tx=0x7f0230012bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=74f91c0dc95f0647 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:40:52.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:52.545+0000 7f022aa00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-09-10T09:40:52.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:52.545+0000 7f022aa00640 1 -- 172.21.15.5:0/2022152027 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f023000f8c0 con 0x562a1091d760 2024-09-10T09:40:52.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:52.545+0000 7f022aa00640 1 -- 172.21.15.5:0/2022152027 <== mon.2 v2:172.21.15.102:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f023000fa60 con 0x562a1091d760 2024-09-10T09:40:52.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:52.545+0000 7f024e81f740 1 -- 172.21.15.5:0/2022152027 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f023c036040 con 0x562a1091d760 2024-09-10T09:40:52.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:52.545+0000 7f022aa00640 1 -- 172.21.15.5:0/2022152027 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f0230025050 con 0x562a1091d760 2024-09-10T09:40:52.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:52.546+0000 7f024e81f740 10 client.25907 fetch_fsmap learned FSMap version 32 2024-09-10T09:40:52.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:52.546+0000 7f024e81f740 1 -- 172.21.15.5:0/2022152027 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x562a10936b10 con 0x562a1091d760 2024-09-10T09:40:52.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:52.546+0000 7f022aa00640 1 -- 172.21.15.5:0/2022152027 <== mon.2 v2:172.21.15.102:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f023c036040 con 0x562a1091d760 2024-09-10T09:40:52.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:52.546+0000 7f022aa00640 1 -- 172.21.15.5:0/2022152027 <== mon.2 v2:172.21.15.102:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7f023001c470 con 0x562a1091d760 2024-09-10T09:40:52.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:52.547+0000 7f024e81f740 10 client.25907 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T09:40:52.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:52.547+0000 7f024e81f740 10 client.25907 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-10T09:40:52.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:52.547+0000 7f024e81f740 20 client.25907 populate_metadata read hostname 'smithi005' 2024-09-10T09:40:52.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:52.547+0000 7f024e81f740 1 --2- 172.21.15.5:0/2022152027 >> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] conn(0x562a10938020 0x562a1093a440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:40:52.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:52.547+0000 7f024e81f740 4 client.25907 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:40:52.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:52.547+0000 7f024e81f740 1 -- 172.21.15.5:0/2022152027 --> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562a10890950 con 0x562a10938020 2024-09-10T09:40:52.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:52.547+0000 7f0249c00640 1 --2- 172.21.15.5:0/2022152027 >> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] conn(0x562a10938020 0x562a1093a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:40:52.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:52.547+0000 7f0249c00640 1 --2- 172.21.15.5:0/2022152027 >> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] conn(0x562a10938020 0x562a1093a440 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d2b8b7496f330062 server_cookie=94fddd317994a92e in_seq=0 out_seq=0 2024-09-10T09:40:52.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:52.547+0000 7f022aa00640 10 client.25907 ms_handle_connect on v2:172.21.15.159:6838/2073739599 2024-09-10T09:40:52.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:52.548+0000 7f022aa00640 1 -- 172.21.15.5:0/2022152027 <== mds.1 v2:172.21.15.159:6838/2073739599 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562a10890950 con 0x562a10938020 2024-09-10T09:40:52.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:52.548+0000 7f022aa00640 10 client.25907 handle_command_reply: tid=0 2024-09-10T09:40:52.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:52.548+0000 7f024e81f740 1 client.25907 shutdown 2024-09-10T09:40:52.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:52.549+0000 7f024e81f740 1 -- 172.21.15.5:0/2022152027 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562a1091d760 msgr2=0x562a10925440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:40:52.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:52.549+0000 7f024e81f740 1 --2- 172.21.15.5:0/2022152027 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562a1091d760 0x562a10925440 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f0230012700 tx=0x7f0230012bc0 comp rx=0 tx=0).stop 2024-09-10T09:40:52.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:52.549+0000 7f024e81f740 1 -- 172.21.15.5:0/2022152027 shutdown_connections 2024-09-10T09:40:52.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:52.549+0000 7f024e81f740 1 --2- 172.21.15.5:0/2022152027 >> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] conn(0x562a10938020 0x562a1093a440 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:52.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:52.549+0000 7f024e81f740 1 --2- 172.21.15.5:0/2022152027 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562a10928020 0x562a10925980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:52.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:52.549+0000 7f024e81f740 1 --2- 172.21.15.5:0/2022152027 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562a1091d760 0x562a10925440 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:52.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:52.549+0000 7f024e81f740 1 --2- 172.21.15.5:0/2022152027 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562a1091ce40 0x562a10924f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:52.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:52.549+0000 7f022aa00640 0 client.25907 ms_handle_reset on v2:172.21.15.159:6838/2073739599 2024-09-10T09:40:52.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:52.549+0000 7f024e81f740 1 -- 172.21.15.5:0/2022152027 >> 172.21.15.5:0/2022152027 conn(0x562a109193d0 msgr2=0x562a1091f0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:40:52.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:52.549+0000 7f024e81f740 1 -- 172.21.15.5:0/2022152027 shutdown_connections 2024-09-10T09:40:52.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:52.549+0000 7f024e81f740 1 -- 172.21.15.5:0/2022152027 wait complete. 2024-09-10T09:40:52.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:52.549+0000 7f024e81f740 20 client.25907 trim_cache size 0 max 16384 2024-09-10T09:40:52.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:52.564+0000 7f024e81f740 1 Processor -- start 2024-09-10T09:40:52.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:52.564+0000 7f024e81f740 1 -- start start 2024-09-10T09:40:52.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:52.564+0000 7f024e81f740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562a1091ce40 0x562a1091f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:40:52.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:52.564+0000 7f024e81f740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562a1091d760 0x562a1093fd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:40:52.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:52.564+0000 7f024e81f740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562a10928020 0x562a10940250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:40:52.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:52.564+0000 7f024e81f740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f02441b0e80 con 0x562a10928020 2024-09-10T09:40:52.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:52.564+0000 7f024e81f740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f02441b2a80 con 0x562a1091ce40 2024-09-10T09:40:52.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:52.564+0000 7f024e81f740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f02441b1550 con 0x562a1091d760 2024-09-10T09:40:52.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:52.564+0000 7f0243e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562a1091d760 0x562a1093fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:40:52.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:52.564+0000 7f0243e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562a1091d760 0x562a1093fd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:56926/0 (socket says 172.21.15.5:56926) 2024-09-10T09:40:52.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:52.564+0000 7f0243e00640 1 -- 172.21.15.5:0/1335431100 learned_addr learned my addr 172.21.15.5:0/1335431100 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:40:52.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:52.564+0000 7f0249c00640 1 --2- 172.21.15.5:0/1335431100 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562a1091ce40 0x562a1091f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:40:52.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:52.565+0000 7f0249c00640 1 --2- 172.21.15.5:0/1335431100 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562a10928020 0x562a10940250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:40:52.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:52.565+0000 7f0243e00640 1 -- 172.21.15.5:0/1335431100 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562a1091ce40 msgr2=0x562a1091f8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:40:52.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:52.565+0000 7f0243e00640 1 --2- 172.21.15.5:0/1335431100 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562a1091ce40 0x562a1091f8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:52.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:52.565+0000 7f0243e00640 1 -- 172.21.15.5:0/1335431100 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562a10928020 msgr2=0x562a10940250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:40:52.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:52.565+0000 7f0243e00640 1 --2- 172.21.15.5:0/1335431100 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562a10928020 0x562a10940250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:52.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:52.565+0000 7f0243e00640 1 -- 172.21.15.5:0/1335431100 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562a10918e70 con 0x562a1091d760 2024-09-10T09:40:52.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:52.565+0000 7f0249c00640 1 --2- 172.21.15.5:0/1335431100 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562a1091ce40 0x562a1091f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:40:52.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:52.565+0000 7f0243e00640 1 --2- 172.21.15.5:0/1335431100 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562a1091d760 0x562a1093fd10 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f023001c810 tx=0x7f023001cb00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=46519a8615aa8b76 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:40:52.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:52.565+0000 7f0235600640 1 -- 172.21.15.5:0/1335431100 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f023001c570 con 0x562a1091d760 2024-09-10T09:40:52.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:52.566+0000 7f0235600640 1 -- 172.21.15.5:0/1335431100 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f023000fcb0 con 0x562a1091d760 2024-09-10T09:40:52.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:52.566+0000 7f0235600640 1 -- 172.21.15.5:0/1335431100 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f023000f9b0 con 0x562a1091d760 2024-09-10T09:40:52.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:52.566+0000 7f024e81f740 1 -- 172.21.15.5:0/1335431100 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562a1091d760 msgr2=0x562a1093fd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:40:52.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:52.566+0000 7f024e81f740 1 --2- 172.21.15.5:0/1335431100 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562a1091d760 0x562a1093fd10 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f023001c810 tx=0x7f023001cb00 comp rx=0 tx=0).stop 2024-09-10T09:40:52.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:52.567+0000 7f024e81f740 1 -- 172.21.15.5:0/1335431100 shutdown_connections 2024-09-10T09:40:52.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:52.567+0000 7f024e81f740 1 --2- 172.21.15.5:0/1335431100 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562a10928020 0x562a10940250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:52.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:52.567+0000 7f024e81f740 1 --2- 172.21.15.5:0/1335431100 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562a1091d760 0x562a1093fd10 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:52.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:52.567+0000 7f024e81f740 1 --2- 172.21.15.5:0/1335431100 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562a1091ce40 0x562a1091f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:52.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:52.567+0000 7f024e81f740 1 -- 172.21.15.5:0/1335431100 >> 172.21.15.5:0/1335431100 conn(0x562a109193d0 msgr2=0x562a10923ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:40:52.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:52.567+0000 7f024e81f740 1 -- 172.21.15.5:0/1335431100 shutdown_connections 2024-09-10T09:40:52.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:52.567+0000 7f024e81f740 1 -- 172.21.15.5:0/1335431100 wait complete. 2024-09-10T09:40:52.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:52.567+0000 7f024e81f740 1 -- start start 2024-09-10T09:40:52.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:52.567+0000 7f024e81f740 1 Processor -- start 2024-09-10T09:40:52.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:52.568+0000 7f024e81f740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562a1091ce40 0x562a10930020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:40:52.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:52.568+0000 7f024e81f740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562a1091d760 0x562a10930560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:40:52.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:52.568+0000 7f024e81f740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562a10928020 0x562a10921710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:40:52.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:52.568+0000 7f024e81f740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f02441b22f0 con 0x562a1091d760 2024-09-10T09:40:52.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:52.568+0000 7f024e81f740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f02441b1c20 con 0x562a1091ce40 2024-09-10T09:40:52.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:52.568+0000 7f024e81f740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f02441aee60 con 0x562a10928020 2024-09-10T09:40:52.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:52.568+0000 7f024e81f740 10 client.0 _ll_register_callbacks cb 0x7f0230009930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:40:52.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:52.568+0000 7f0249c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562a1091ce40 0x562a10930020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:40:52.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:52.568+0000 7f0249c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562a1091ce40 0x562a10930020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:41074/0 (socket says 172.21.15.5:41074) 2024-09-10T09:40:52.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:52.568+0000 7f0243e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562a1091d760 0x562a10930560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:40:52.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:52.568+0000 7f0249c00640 1 -- 172.21.15.5:0/1732463171 learned_addr learned my addr 172.21.15.5:0/1732463171 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:40:52.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:52.568+0000 7f0249c00640 1 --2- 172.21.15.5:0/1732463171 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562a10928020 0x562a10921710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:40:52.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:52.569+0000 7f0249c00640 1 -- 172.21.15.5:0/1732463171 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562a10928020 msgr2=0x562a10921710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:40:52.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:52.569+0000 7f0249c00640 1 --2- 172.21.15.5:0/1732463171 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562a10928020 0x562a10921710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:52.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:52.569+0000 7f0249c00640 1 -- 172.21.15.5:0/1732463171 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562a1091d760 msgr2=0x562a10930560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:40:52.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:52.569+0000 7f0249c00640 1 --2- 172.21.15.5:0/1732463171 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562a1091d760 0x562a10930560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:52.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:52.569+0000 7f0249c00640 1 -- 172.21.15.5:0/1732463171 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562a10921e20 con 0x562a1091ce40 2024-09-10T09:40:52.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:52.569+0000 7f0249c00640 1 --2- 172.21.15.5:0/1732463171 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562a1091ce40 0x562a10930020 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f023c005f50 tx=0x7f023c03d040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a9dcb1638893ecb0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:40:52.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:52.569+0000 7f022aa00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-09-10T09:40:52.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:52.569+0000 7f022aa00640 1 -- 172.21.15.5:0/1732463171 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f023c063060 con 0x562a1091ce40 2024-09-10T09:40:52.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:52.569+0000 7f024e81f740 1 -- 172.21.15.5:0/1732463171 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f023000fcb0 con 0x562a1091ce40 2024-09-10T09:40:52.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:52.569+0000 7f022aa00640 1 -- 172.21.15.5:0/1732463171 <== mon.2 v2:172.21.15.102:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f023c003b70 con 0x562a1091ce40 2024-09-10T09:40:52.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:52.569+0000 7f022aa00640 1 -- 172.21.15.5:0/1732463171 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f023c06a050 con 0x562a1091ce40 2024-09-10T09:40:52.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:52.569+0000 7f024e81f740 10 client.25913 fetch_fsmap learned FSMap version 32 2024-09-10T09:40:52.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:52.569+0000 7f024e81f740 1 -- 172.21.15.5:0/1732463171 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x562a109a26e0 con 0x562a1091ce40 2024-09-10T09:40:52.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:52.570+0000 7f022aa00640 1 -- 172.21.15.5:0/1732463171 <== mon.2 v2:172.21.15.102:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f023000fcb0 con 0x562a1091ce40 2024-09-10T09:40:52.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:52.570+0000 7f022aa00640 1 -- 172.21.15.5:0/1732463171 <== mon.2 v2:172.21.15.102:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7f023c043070 con 0x562a1091ce40 2024-09-10T09:40:52.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:52.570+0000 7f024e81f740 10 client.25913 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T09:40:52.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:52.570+0000 7f024e81f740 10 client.25913 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-10T09:40:52.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:52.570+0000 7f024e81f740 20 client.25913 populate_metadata read hostname 'smithi005' 2024-09-10T09:40:52.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:52.570+0000 7f024e81f740 1 --2- 172.21.15.5:0/1732463171 >> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] conn(0x562a109a2ad0 0x562a109a4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:40:52.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:52.570+0000 7f024e81f740 4 client.25913 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T09:40:52.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:52.570+0000 7f024e81f740 1 -- 172.21.15.5:0/1732463171 --> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562a1088ae60 con 0x562a109a2ad0 2024-09-10T09:40:52.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:52.570+0000 7f0243e00640 1 --2- 172.21.15.5:0/1732463171 >> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] conn(0x562a109a2ad0 0x562a109a4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:40:52.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:52.571+0000 7f0243e00640 1 --2- 172.21.15.5:0/1732463171 >> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] conn(0x562a109a2ad0 0x562a109a4ec0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=bbee280c3158c713 server_cookie=4d474ff6d489d844 in_seq=0 out_seq=0 2024-09-10T09:40:52.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:52.571+0000 7f022aa00640 10 client.25913 ms_handle_connect on v2:172.21.15.159:6838/2073739599 2024-09-10T09:40:52.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:52.571+0000 7f022aa00640 1 -- 172.21.15.5:0/1732463171 <== mds.1 v2:172.21.15.159:6838/2073739599 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562a1088ae60 con 0x562a109a2ad0 2024-09-10T09:40:52.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:52.571+0000 7f022aa00640 10 client.25913 handle_command_reply: tid=0 2024-09-10T09:40:52.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:52.571+0000 7f024e81f740 1 client.25913 shutdown 2024-09-10T09:40:52.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:52.571+0000 7f024e81f740 1 -- 172.21.15.5:0/1732463171 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562a1091ce40 msgr2=0x562a10930020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:40:52.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:52.571+0000 7f024e81f740 1 --2- 172.21.15.5:0/1732463171 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562a1091ce40 0x562a10930020 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f023c005f50 tx=0x7f023c03d040 comp rx=0 tx=0).stop 2024-09-10T09:40:52.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:52.571+0000 7f024e81f740 1 -- 172.21.15.5:0/1732463171 shutdown_connections 2024-09-10T09:40:52.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:52.571+0000 7f024e81f740 1 --2- 172.21.15.5:0/1732463171 >> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] conn(0x562a109a2ad0 0x562a109a4ec0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:52.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:52.571+0000 7f024e81f740 1 --2- 172.21.15.5:0/1732463171 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562a10928020 0x562a10921710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:52.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:52.571+0000 7f024e81f740 1 --2- 172.21.15.5:0/1732463171 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562a1091d760 0x562a10930560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:52.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:52.571+0000 7f024e81f740 1 --2- 172.21.15.5:0/1732463171 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562a1091ce40 0x562a10930020 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:52.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:52.571+0000 7f022aa00640 0 client.25913 ms_handle_reset on v2:172.21.15.159:6838/2073739599 2024-09-10T09:40:52.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:52.571+0000 7f024e81f740 1 -- 172.21.15.5:0/1732463171 >> 172.21.15.5:0/1732463171 conn(0x562a109193d0 msgr2=0x562a10917c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:40:52.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:52.572+0000 7f024e81f740 1 -- 172.21.15.5:0/1732463171 shutdown_connections 2024-09-10T09:40:52.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:52.572+0000 7f024e81f740 1 -- 172.21.15.5:0/1732463171 wait complete. 2024-09-10T09:40:52.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:52.572+0000 7f024e81f740 20 client.25913 trim_cache size 0 max 16384 2024-09-10T09:40:52.576 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-10T09:40:52.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:52.572+0000 7f024a600640 1 -- 172.21.15.5:0/98367550 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f022c0a2b80 msgr2=0x7f022c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:40:52.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:52.572+0000 7f024a600640 1 --2- 172.21.15.5:0/98367550 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f022c0a2b80 0x7f022c0a5040 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f0238098750 tx=0x7f0238004a40 comp rx=0 tx=0).stop 2024-09-10T09:40:52.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:52.573+0000 7f024a600640 1 -- 172.21.15.5:0/98367550 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0244103e30 msgr2=0x7f024419a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:40:52.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:52.573+0000 7f024a600640 1 --2- 172.21.15.5:0/98367550 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0244103e30 0x7f024419a460 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f023800e9f0 tx=0x7f023800edf0 comp rx=0 tx=0).stop 2024-09-10T09:40:52.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:52.573+0000 7f024a600640 1 -- 172.21.15.5:0/98367550 shutdown_connections 2024-09-10T09:40:52.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:52.573+0000 7f024a600640 1 --2- 172.21.15.5:0/98367550 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f022c0a2b80 0x7f022c0a5040 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:52.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:52.573+0000 7f024a600640 1 --2- 172.21.15.5:0/98367550 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0244108f60 0x7f024419ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:52.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:52.573+0000 7f024a600640 1 --2- 172.21.15.5:0/98367550 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f02441047e0 0x7f024419a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:52.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:52.573+0000 7f024a600640 1 --2- 172.21.15.5:0/98367550 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0244103e30 0x7f024419a460 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:52.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:52.573+0000 7f024a600640 1 -- 172.21.15.5:0/98367550 >> 172.21.15.5:0/98367550 conn(0x7f02440ff910 msgr2=0x7f0244100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:40:52.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:52.573+0000 7f024a600640 1 -- 172.21.15.5:0/98367550 shutdown_connections 2024-09-10T09:40:52.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:52.573+0000 7f024a600640 1 -- 172.21.15.5:0/98367550 wait complete. 2024-09-10T09:40:52.992 INFO:tasks.workunit.client.0.smithi005.stdout: 1 60306 10.13 MB/sec warmup 34 sec latency 11.761 ms 2024-09-10T09:40:53.448 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-10T09:40:53.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:40:53 smithi102 ceph-mon[35442]: pgmap v469: 129 pgs: 129 active+clean; 43 MiB data, 501 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 1.3 MiB/s wr, 215 op/s 2024-09-10T09:40:53.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:40:53 smithi005 ceph-mon[30350]: pgmap v469: 129 pgs: 129 active+clean; 43 MiB data, 501 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 1.3 MiB/s wr, 215 op/s 2024-09-10T09:40:53.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:40:53 smithi159 ceph-mon[35513]: pgmap v469: 129 pgs: 129 active+clean; 43 MiB data, 501 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 1.3 MiB/s wr, 215 op/s 2024-09-10T09:40:53.859 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:40:53.992 INFO:tasks.workunit.client.0.smithi005.stdout: 1 61997 10.04 MB/sec warmup 35 sec latency 24.578 ms 2024-09-10T09:40:54.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:54.403+0000 7fbd17c00640 1 -- 172.21.15.5:0/357137525 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbd1010e450 msgr2=0x7fbd10110840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:40:54.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:54.403+0000 7fbd17c00640 1 --2- 172.21.15.5:0/357137525 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbd1010e450 0x7fbd10110840 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fbd08009ca0 tx=0x7fbd0802a950 comp rx=0 tx=0).stop 2024-09-10T09:40:54.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:54.403+0000 7fbd17c00640 1 -- 172.21.15.5:0/357137525 shutdown_connections 2024-09-10T09:40:54.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:54.403+0000 7fbd17c00640 1 --2- 172.21.15.5:0/357137525 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbd1010e450 0x7fbd10110840 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:54.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:54.403+0000 7fbd17c00640 1 --2- 172.21.15.5:0/357137525 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbd10074e30 0x7fbd1010df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:54.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:54.403+0000 7fbd17c00640 1 --2- 172.21.15.5:0/357137525 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbd10074510 0x7fbd100748f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:54.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:54.403+0000 7fbd17c00640 1 -- 172.21.15.5:0/357137525 >> 172.21.15.5:0/357137525 conn(0x7fbd100fd8b0 msgr2=0x7fbd100ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:40:54.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:54.403+0000 7fbd17c00640 1 -- 172.21.15.5:0/357137525 shutdown_connections 2024-09-10T09:40:54.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:54.403+0000 7fbd17c00640 1 -- 172.21.15.5:0/357137525 wait complete. 2024-09-10T09:40:54.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:54.405+0000 7fbd17c00640 1 Processor -- start 2024-09-10T09:40:54.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:54.405+0000 7fbd17c00640 1 -- start start 2024-09-10T09:40:54.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:54.405+0000 7fbd17c00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbd10074510 0x7fbd1019a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:40:54.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:54.405+0000 7fbd17c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbd10074e30 0x7fbd1019ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:40:54.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:54.405+0000 7fbd17c00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbd1010e450 0x7fbd1019ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:40:54.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:54.405+0000 7fbd17c00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fbd10114160 con 0x7fbd10074510 2024-09-10T09:40:54.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:54.405+0000 7fbd17c00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fbd10113fe0 con 0x7fbd10074e30 2024-09-10T09:40:54.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:54.405+0000 7fbd17c00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fbd101142e0 con 0x7fbd1010e450 2024-09-10T09:40:54.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:54.406+0000 7fbd15200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbd10074510 0x7fbd1019a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:40:54.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:54.406+0000 7fbd0fe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbd10074e30 0x7fbd1019ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:40:54.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:54.406+0000 7fbd0fe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbd10074e30 0x7fbd1019ac00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:41098/0 (socket says 172.21.15.5:41098) 2024-09-10T09:40:54.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:54.406+0000 7fbd15200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbd10074510 0x7fbd1019a6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43344/0 (socket says 172.21.15.5:43344) 2024-09-10T09:40:54.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:54.406+0000 7fbd0fe00640 1 -- 172.21.15.5:0/807485851 learned_addr learned my addr 172.21.15.5:0/807485851 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:40:54.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:54.406+0000 7fbd15c00640 1 --2- 172.21.15.5:0/807485851 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbd1010e450 0x7fbd1019ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:40:54.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:54.406+0000 7fbd15200640 1 -- 172.21.15.5:0/807485851 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbd1010e450 msgr2=0x7fbd1019ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:40:54.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:54.406+0000 7fbd15200640 1 --2- 172.21.15.5:0/807485851 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbd1010e450 0x7fbd1019ef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:54.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:54.406+0000 7fbd15200640 1 -- 172.21.15.5:0/807485851 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbd10074e30 msgr2=0x7fbd1019ac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:40:54.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:54.406+0000 7fbd15200640 1 --2- 172.21.15.5:0/807485851 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbd10074e30 0x7fbd1019ac00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:54.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:54.406+0000 7fbd15200640 1 -- 172.21.15.5:0/807485851 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd1019f670 con 0x7fbd10074510 2024-09-10T09:40:54.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:54.406+0000 7fbd0fe00640 1 --2- 172.21.15.5:0/807485851 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbd10074e30 0x7fbd1019ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:40:54.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:54.407+0000 7fbd15200640 1 --2- 172.21.15.5:0/807485851 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbd10074510 0x7fbd1019a6c0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fbd0400e9f0 tx=0x7fbd0400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:40:54.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:54.407+0000 7fbd0d600640 1 -- 172.21.15.5:0/807485851 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd0400ccc0 con 0x7fbd10074510 2024-09-10T09:40:54.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:54.407+0000 7fbd17c00640 1 -- 172.21.15.5:0/807485851 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd1019f960 con 0x7fbd10074510 2024-09-10T09:40:54.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:54.407+0000 7fbd17c00640 1 -- 172.21.15.5:0/807485851 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd101a7240 con 0x7fbd10074510 2024-09-10T09:40:54.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:54.407+0000 7fbd0d600640 1 -- 172.21.15.5:0/807485851 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbd04010910 con 0x7fbd10074510 2024-09-10T09:40:54.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:54.407+0000 7fbd0d600640 1 -- 172.21.15.5:0/807485851 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd04018960 con 0x7fbd10074510 2024-09-10T09:40:54.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:54.410+0000 7fbd1a658740 1 Processor -- start 2024-09-10T09:40:54.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:54.410+0000 7fbd0d600640 1 -- 172.21.15.5:0/807485851 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7fbd04018b00 con 0x7fbd10074510 2024-09-10T09:40:54.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:54.410+0000 7fbd1a658740 1 -- start start 2024-09-10T09:40:54.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:54.411+0000 7fbd1a658740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563197198c70 0x563197199050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:40:54.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:54.411+0000 7fbd1a658740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563197199590 0x5631971a3910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:40:54.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:54.411+0000 7fbd1a658740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5631971a3e50 0x5631971a6240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:40:54.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:54.411+0000 7fbd1a658740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5631970ff3e0 con 0x563197198c70 2024-09-10T09:40:54.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:54.411+0000 7fbd1a658740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x5631970a6d80 con 0x5631971a3e50 2024-09-10T09:40:54.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:54.411+0000 7fbd1a658740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5631970fb3a0 con 0x563197199590 2024-09-10T09:40:54.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:54.411+0000 7fbd15200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563197199590 0x5631971a3910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:40:54.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:54.411+0000 7fbd15200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563197199590 0x5631971a3910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:56970/0 (socket says 172.21.15.5:56970) 2024-09-10T09:40:54.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:54.411+0000 7fbd15200640 1 -- 172.21.15.5:0/2938045320 learned_addr learned my addr 172.21.15.5:0/2938045320 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:40:54.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:54.411+0000 7fbd15c00640 1 --2- 172.21.15.5:0/2938045320 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563197198c70 0x563197199050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:40:54.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:54.412+0000 7fbd0fe00640 1 --2- 172.21.15.5:0/2938045320 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5631971a3e50 0x5631971a6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:40:54.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:54.412+0000 7fbd15200640 1 -- 172.21.15.5:0/2938045320 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5631971a3e50 msgr2=0x5631971a6240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:40:54.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:54.412+0000 7fbd15200640 1 --2- 172.21.15.5:0/2938045320 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5631971a3e50 0x5631971a6240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:54.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:54.412+0000 7fbd15200640 1 -- 172.21.15.5:0/2938045320 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563197198c70 msgr2=0x563197199050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:40:54.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:54.412+0000 7fbd15200640 1 --2- 172.21.15.5:0/2938045320 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563197198c70 0x563197199050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:54.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:54.412+0000 7fbd15200640 1 -- 172.21.15.5:0/2938045320 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5631970c5ae0 con 0x563197199590 2024-09-10T09:40:54.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:54.412+0000 7fbd0fe00640 1 --2- 172.21.15.5:0/2938045320 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5631971a3e50 0x5631971a6240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:40:54.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:54.412+0000 7fbd0d600640 1 --2- 172.21.15.5:0/807485851 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fbcf80a2c00 0x7fbcf80a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:40:54.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:54.412+0000 7fbd15200640 1 --2- 172.21.15.5:0/2938045320 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563197199590 0x5631971a3910 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fbd040d5c20 tx=0x7fbd040d7670 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4ff8df28f823d57b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:40:54.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:54.412+0000 7fbd0d600640 1 -- 172.21.15.5:0/807485851 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fbd040cf720 con 0x7fbd10074510 2024-09-10T09:40:54.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:54.413+0000 7fbd15c00640 1 --2- 172.21.15.5:0/807485851 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fbcf80a2c00 0x7fbcf80a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:40:54.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:54.413+0000 7fbd02000640 1 -- 172.21.15.5:0/2938045320 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd0409ce60 con 0x563197199590 2024-09-10T09:40:54.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:54.413+0000 7fbd02000640 1 -- 172.21.15.5:0/2938045320 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbd0409fee0 con 0x563197199590 2024-09-10T09:40:54.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:54.413+0000 7fbd02000640 1 -- 172.21.15.5:0/2938045320 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd0400ce60 con 0x563197199590 2024-09-10T09:40:54.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:54.413+0000 7fbd1a658740 1 -- 172.21.15.5:0/2938045320 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563197199590 msgr2=0x5631971a3910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:40:54.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:54.413+0000 7fbd1a658740 1 --2- 172.21.15.5:0/2938045320 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563197199590 0x5631971a3910 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fbd040d5c20 tx=0x7fbd040d7670 comp rx=0 tx=0).stop 2024-09-10T09:40:54.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:54.413+0000 7fbd15c00640 1 --2- 172.21.15.5:0/807485851 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fbcf80a2c00 0x7fbcf80a50c0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fbd08005e30 tx=0x7fbd08005c50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:40:54.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:54.413+0000 7fbd1a658740 1 -- 172.21.15.5:0/2938045320 shutdown_connections 2024-09-10T09:40:54.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:54.413+0000 7fbd1a658740 1 --2- 172.21.15.5:0/2938045320 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5631971a3e50 0x5631971a6240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:54.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:54.413+0000 7fbd1a658740 1 --2- 172.21.15.5:0/2938045320 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563197199590 0x5631971a3910 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:54.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:54.413+0000 7fbd1a658740 1 --2- 172.21.15.5:0/2938045320 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563197198c70 0x563197199050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:54.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:54.413+0000 7fbd1a658740 1 -- 172.21.15.5:0/2938045320 >> 172.21.15.5:0/2938045320 conn(0x563197195200 msgr2=0x5631971955e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:40:54.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:54.414+0000 7fbd1a658740 1 -- 172.21.15.5:0/2938045320 shutdown_connections 2024-09-10T09:40:54.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:54.414+0000 7fbd1a658740 1 -- 172.21.15.5:0/2938045320 wait complete. 2024-09-10T09:40:54.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:54.414+0000 7fbd1a658740 1 -- start start 2024-09-10T09:40:54.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:54.414+0000 7fbd1a658740 1 Processor -- start 2024-09-10T09:40:54.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:54.415+0000 7fbd1a658740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563197198c70 0x5631971a0cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:40:54.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:54.415+0000 7fbd1a658740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563197199590 0x5631971a1230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:40:54.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:54.415+0000 7fbd1a658740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5631971a3e50 0x5631971a1770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:40:54.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:54.415+0000 7fbd1a658740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5631971a69c0 con 0x5631971a3e50 2024-09-10T09:40:54.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:54.415+0000 7fbd1a658740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x5631970ba0f0 con 0x563197199590 2024-09-10T09:40:54.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:54.415+0000 7fbd1a658740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5631971a6b40 con 0x563197198c70 2024-09-10T09:40:54.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:54.415+0000 7fbd1a658740 10 client.0 _ll_register_callbacks cb 0x5631970a1bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:40:54.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:54.415+0000 7fbd15200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563197198c70 0x5631971a0cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:40:54.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:54.415+0000 7fbd15200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563197198c70 0x5631971a0cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:56978/0 (socket says 172.21.15.5:56978) 2024-09-10T09:40:54.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:54.415+0000 7fbd15200640 1 -- 172.21.15.5:0/806151488 learned_addr learned my addr 172.21.15.5:0/806151488 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:40:54.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:54.415+0000 7fbd0fe00640 1 --2- 172.21.15.5:0/806151488 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563197199590 0x5631971a1230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:40:54.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:54.415+0000 7fbd15c00640 1 --2- 172.21.15.5:0/806151488 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5631971a3e50 0x5631971a1770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:40:54.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:54.416+0000 7fbd15200640 1 -- 172.21.15.5:0/806151488 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563197199590 msgr2=0x5631971a1230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:40:54.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:54.416+0000 7fbd15200640 1 --2- 172.21.15.5:0/806151488 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563197199590 0x5631971a1230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:54.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:54.416+0000 7fbd15200640 1 -- 172.21.15.5:0/806151488 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5631971a3e50 msgr2=0x5631971a1770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:40:54.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:54.416+0000 7fbd15200640 1 --2- 172.21.15.5:0/806151488 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5631971a3e50 0x5631971a1770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:54.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:54.416+0000 7fbd15200640 1 -- 172.21.15.5:0/806151488 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563196e703c0 con 0x563197198c70 2024-09-10T09:40:54.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:54.416+0000 7fbd15c00640 1 --2- 172.21.15.5:0/806151488 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5631971a3e50 0x5631971a1770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T09:40:54.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:54.416+0000 7fbd15200640 1 --2- 172.21.15.5:0/806151488 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563197198c70 0x5631971a0cf0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fbd04000c00 tx=0x7fbd0409b540 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b0a3a4f1ccb84655 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:40:54.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:54.416+0000 7fbcf6a00640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-09-10T09:40:54.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:54.416+0000 7fbcf6a00640 1 -- 172.21.15.5:0/806151488 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd0409ce60 con 0x563197198c70 2024-09-10T09:40:54.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:54.416+0000 7fbd1a658740 1 -- 172.21.15.5:0/806151488 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbd0409fee0 con 0x563197198c70 2024-09-10T09:40:54.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:54.416+0000 7fbcf6a00640 1 -- 172.21.15.5:0/806151488 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fbd040cf390 con 0x563197198c70 2024-09-10T09:40:54.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:54.416+0000 7fbcf6a00640 1 -- 172.21.15.5:0/806151488 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fbd04086070 con 0x563197198c70 2024-09-10T09:40:54.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:54.416+0000 7fbd1a658740 10 client.25957 fetch_fsmap learned FSMap version 32 2024-09-10T09:40:54.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:54.416+0000 7fbd1a658740 1 -- 172.21.15.5:0/806151488 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5631971a69c0 con 0x563197198c70 2024-09-10T09:40:54.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:54.417+0000 7fbcf6a00640 1 -- 172.21.15.5:0/806151488 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbd0409fee0 con 0x563197198c70 2024-09-10T09:40:54.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:54.417+0000 7fbcf6a00640 1 -- 172.21.15.5:0/806151488 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7fbd040cf070 con 0x563197198c70 2024-09-10T09:40:54.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:54.417+0000 7fbd1a658740 10 client.25957 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T09:40:54.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:54.417+0000 7fbd1a658740 10 client.25957 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T09:40:54.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:54.417+0000 7fbd1a658740 20 client.25957 populate_metadata read hostname 'smithi005' 2024-09-10T09:40:54.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:54.417+0000 7fbd1a658740 1 --2- 172.21.15.5:0/806151488 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x5631971b3bc0 0x5631971b5fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:40:54.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:54.417+0000 7fbd1a658740 4 client.25957 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:40:54.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:54.417+0000 7fbd1a658740 1 -- 172.21.15.5:0/806151488 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56319710c950 con 0x5631971b3bc0 2024-09-10T09:40:54.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:54.418+0000 7fbd0fe00640 1 --2- 172.21.15.5:0/806151488 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x5631971b3bc0 0x5631971b5fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:40:54.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:54.418+0000 7fbd0fe00640 1 --2- 172.21.15.5:0/806151488 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x5631971b3bc0 0x5631971b5fb0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=95909426379ceb97 server_cookie=a2e7db163c244aa6 in_seq=0 out_seq=0 2024-09-10T09:40:54.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:54.418+0000 7fbcf6a00640 10 client.25957 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T09:40:54.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:54.419+0000 7fbcf6a00640 1 -- 172.21.15.5:0/806151488 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56319710c950 con 0x5631971b3bc0 2024-09-10T09:40:54.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:54.419+0000 7fbcf6a00640 10 client.25957 handle_command_reply: tid=0 2024-09-10T09:40:54.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:54.419+0000 7fbd1a658740 1 client.25957 shutdown 2024-09-10T09:40:54.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:54.420+0000 7fbd1a658740 1 -- 172.21.15.5:0/806151488 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563197198c70 msgr2=0x5631971a0cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:40:54.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:54.420+0000 7fbd1a658740 1 --2- 172.21.15.5:0/806151488 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563197198c70 0x5631971a0cf0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fbd04000c00 tx=0x7fbd0409b540 comp rx=0 tx=0).stop 2024-09-10T09:40:54.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:54.420+0000 7fbd1a658740 1 -- 172.21.15.5:0/806151488 shutdown_connections 2024-09-10T09:40:54.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:54.420+0000 7fbd1a658740 1 --2- 172.21.15.5:0/806151488 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x5631971b3bc0 0x5631971b5fb0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:54.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:54.420+0000 7fbd1a658740 1 --2- 172.21.15.5:0/806151488 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5631971a3e50 0x5631971a1770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:54.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:54.420+0000 7fbd1a658740 1 --2- 172.21.15.5:0/806151488 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563197199590 0x5631971a1230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:54.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:54.420+0000 7fbd1a658740 1 --2- 172.21.15.5:0/806151488 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563197198c70 0x5631971a0cf0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:54.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:54.420+0000 7fbcf6a00640 0 client.25957 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T09:40:54.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:54.420+0000 7fbd1a658740 1 -- 172.21.15.5:0/806151488 >> 172.21.15.5:0/806151488 conn(0x563197195200 msgr2=0x56319719af30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:40:54.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:54.420+0000 7fbd1a658740 1 -- 172.21.15.5:0/806151488 shutdown_connections 2024-09-10T09:40:54.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:54.420+0000 7fbd1a658740 1 -- 172.21.15.5:0/806151488 wait complete. 2024-09-10T09:40:54.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:54.420+0000 7fbd1a658740 20 client.25957 trim_cache size 0 max 16384 2024-09-10T09:40:54.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:54.436+0000 7fbd1a658740 1 Processor -- start 2024-09-10T09:40:54.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:54.436+0000 7fbd1a658740 1 -- start start 2024-09-10T09:40:54.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:54.436+0000 7fbd1a658740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563197198c70 0x5631971bb830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:40:54.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:54.436+0000 7fbd1a658740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563197199590 0x5631971bbd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:40:54.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:54.436+0000 7fbd1a658740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5631971a3e50 0x56319719a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:40:54.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:54.436+0000 7fbd1a658740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fbcf000fd20 con 0x563197199590 2024-09-10T09:40:54.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:54.436+0000 7fbd1a658740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fbcf0011920 con 0x5631971a3e50 2024-09-10T09:40:54.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:54.436+0000 7fbd1a658740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fbcf00103f0 con 0x563197198c70 2024-09-10T09:40:54.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:54.436+0000 7fbd15200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563197198c70 0x5631971bb830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:40:54.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:54.436+0000 7fbd0fe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563197199590 0x5631971bbd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:40:54.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:54.437+0000 7fbd15c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5631971a3e50 0x56319719a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:40:54.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:54.437+0000 7fbd0fe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563197199590 0x5631971bbd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43376/0 (socket says 172.21.15.5:43376) 2024-09-10T09:40:54.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:54.437+0000 7fbd0fe00640 1 -- 172.21.15.5:0/11545465 learned_addr learned my addr 172.21.15.5:0/11545465 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:40:54.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:54.436+0000 7fbd15200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563197198c70 0x5631971bb830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:56992/0 (socket says 172.21.15.5:56992) 2024-09-10T09:40:54.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:54.437+0000 7fbd15c00640 1 -- 172.21.15.5:0/11545465 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563197198c70 msgr2=0x5631971bb830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:40:54.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:54.437+0000 7fbd15c00640 1 --2- 172.21.15.5:0/11545465 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563197198c70 0x5631971bb830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:54.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:54.437+0000 7fbd15c00640 1 -- 172.21.15.5:0/11545465 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563197199590 msgr2=0x5631971bbd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:40:54.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:54.437+0000 7fbd15c00640 1 --2- 172.21.15.5:0/11545465 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563197199590 0x5631971bbd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:54.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:54.437+0000 7fbd15c00640 1 -- 172.21.15.5:0/11545465 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563197195030 con 0x5631971a3e50 2024-09-10T09:40:54.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:54.437+0000 7fbd15200640 1 --2- 172.21.15.5:0/11545465 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563197198c70 0x5631971bb830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T09:40:54.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:54.437+0000 7fbd15c00640 1 --2- 172.21.15.5:0/11545465 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5631971a3e50 0x56319719a6c0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fbd0802ae70 tx=0x7fbd08042000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=770ad309f320e13e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:40:54.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:54.438+0000 7fbd01600640 1 -- 172.21.15.5:0/11545465 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd08034040 con 0x5631971a3e50 2024-09-10T09:40:54.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:54.438+0000 7fbd01600640 1 -- 172.21.15.5:0/11545465 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbd08003e50 con 0x5631971a3e50 2024-09-10T09:40:54.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:54.438+0000 7fbd01600640 1 -- 172.21.15.5:0/11545465 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd080306a0 con 0x5631971a3e50 2024-09-10T09:40:54.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:54.438+0000 7fbd1a658740 1 -- 172.21.15.5:0/11545465 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5631971a3e50 msgr2=0x56319719a6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:40:54.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:54.438+0000 7fbd1a658740 1 --2- 172.21.15.5:0/11545465 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5631971a3e50 0x56319719a6c0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fbd0802ae70 tx=0x7fbd08042000 comp rx=0 tx=0).stop 2024-09-10T09:40:54.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:54.438+0000 7fbd1a658740 1 -- 172.21.15.5:0/11545465 shutdown_connections 2024-09-10T09:40:54.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:54.438+0000 7fbd1a658740 1 --2- 172.21.15.5:0/11545465 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5631971a3e50 0x56319719a6c0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:54.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:54.438+0000 7fbd1a658740 1 --2- 172.21.15.5:0/11545465 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563197199590 0x5631971bbd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:54.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:54.438+0000 7fbd1a658740 1 --2- 172.21.15.5:0/11545465 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563197198c70 0x5631971bb830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:54.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:54.438+0000 7fbd1a658740 1 -- 172.21.15.5:0/11545465 >> 172.21.15.5:0/11545465 conn(0x563197195200 msgr2=0x5631971a2d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:40:54.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:54.439+0000 7fbd1a658740 1 -- 172.21.15.5:0/11545465 shutdown_connections 2024-09-10T09:40:54.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:54.439+0000 7fbd1a658740 1 -- 172.21.15.5:0/11545465 wait complete. 2024-09-10T09:40:54.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:54.439+0000 7fbd1a658740 1 -- start start 2024-09-10T09:40:54.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:54.439+0000 7fbd1a658740 1 Processor -- start 2024-09-10T09:40:54.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:54.440+0000 7fbd1a658740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563197198c70 0x5631971ac590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:40:54.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:54.440+0000 7fbd1a658740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563197199590 0x5631971a78c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:40:54.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:54.440+0000 7fbd1a658740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5631971a3e50 0x5631971a7e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:40:54.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:54.440+0000 7fbd1a658740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fbcf0011190 con 0x563197198c70 2024-09-10T09:40:54.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:54.440+0000 7fbd1a658740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fbcf0010ac0 con 0x563197199590 2024-09-10T09:40:54.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:54.440+0000 7fbd1a658740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fbcf000dd00 con 0x5631971a3e50 2024-09-10T09:40:54.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:54.440+0000 7fbd1a658740 10 client.0 _ll_register_callbacks cb 0x7fbcfc005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:40:54.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:54.440+0000 7fbd15200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563197198c70 0x5631971ac590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:40:54.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:54.440+0000 7fbd0fe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563197199590 0x5631971a78c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:40:54.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:54.440+0000 7fbd0fe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563197199590 0x5631971a78c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:41118/0 (socket says 172.21.15.5:41118) 2024-09-10T09:40:54.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:54.440+0000 7fbd0fe00640 1 -- 172.21.15.5:0/2197283512 learned_addr learned my addr 172.21.15.5:0/2197283512 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:40:54.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:54.440+0000 7fbd15200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563197198c70 0x5631971ac590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43378/0 (socket says 172.21.15.5:43378) 2024-09-10T09:40:54.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:54.440+0000 7fbd15c00640 1 --2- 172.21.15.5:0/2197283512 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5631971a3e50 0x5631971a7e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:40:54.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:54.440+0000 7fbd0fe00640 1 -- 172.21.15.5:0/2197283512 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5631971a3e50 msgr2=0x5631971a7e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:40:54.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:54.440+0000 7fbd0fe00640 1 --2- 172.21.15.5:0/2197283512 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5631971a3e50 0x5631971a7e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:54.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:54.440+0000 7fbd0fe00640 1 -- 172.21.15.5:0/2197283512 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563197198c70 msgr2=0x5631971ac590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:40:54.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:54.440+0000 7fbd0fe00640 1 --2- 172.21.15.5:0/2197283512 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563197198c70 0x5631971ac590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:54.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:54.441+0000 7fbd0fe00640 1 -- 172.21.15.5:0/2197283512 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56319719bae0 con 0x563197199590 2024-09-10T09:40:54.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:54.441+0000 7fbd0fe00640 1 --2- 172.21.15.5:0/2197283512 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563197199590 0x5631971a78c0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x5631971a0580 tx=0x7fbcfc010da0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1581c83d20fb7058 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:40:54.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:54.441+0000 7fbcf6a00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-09-10T09:40:54.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:54.441+0000 7fbcf6a00640 1 -- 172.21.15.5:0/2197283512 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcfc00fd80 con 0x563197199590 2024-09-10T09:40:54.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:54.441+0000 7fbcf6a00640 1 -- 172.21.15.5:0/2197283512 <== mon.2 v2:172.21.15.102:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fbcfc005020 con 0x563197199590 2024-09-10T09:40:54.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:54.441+0000 7fbd1a658740 1 -- 172.21.15.5:0/2197283512 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbd08003e50 con 0x563197199590 2024-09-10T09:40:54.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:54.441+0000 7fbcf6a00640 1 -- 172.21.15.5:0/2197283512 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fbcfc02f050 con 0x563197199590 2024-09-10T09:40:54.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:54.441+0000 7fbd1a658740 10 client.25931 fetch_fsmap learned FSMap version 32 2024-09-10T09:40:54.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:54.442+0000 7fbd1a658740 1 -- 172.21.15.5:0/2197283512 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x563197190770 con 0x563197199590 2024-09-10T09:40:54.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:54.442+0000 7fbcf6a00640 1 -- 172.21.15.5:0/2197283512 <== mon.2 v2:172.21.15.102:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbd08003e50 con 0x563197199590 2024-09-10T09:40:54.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:54.442+0000 7fbcf6a00640 1 -- 172.21.15.5:0/2197283512 <== mon.2 v2:172.21.15.102:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7fbcfc025030 con 0x563197199590 2024-09-10T09:40:54.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:54.442+0000 7fbd1a658740 10 client.25931 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T09:40:54.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:54.442+0000 7fbd1a658740 10 client.25931 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T09:40:54.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:54.442+0000 7fbd1a658740 20 client.25931 populate_metadata read hostname 'smithi005' 2024-09-10T09:40:54.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:54.442+0000 7fbd1a658740 1 --2- 172.21.15.5:0/2197283512 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x563197227500 0x563197229920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:40:54.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:54.442+0000 7fbd1a658740 4 client.25931 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T09:40:54.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:54.442+0000 7fbd1a658740 1 -- 172.21.15.5:0/2197283512 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563197106e60 con 0x563197227500 2024-09-10T09:40:54.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:54.443+0000 7fbd15200640 1 --2- 172.21.15.5:0/2197283512 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x563197227500 0x563197229920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:40:54.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:54.443+0000 7fbd15200640 1 --2- 172.21.15.5:0/2197283512 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x563197227500 0x563197229920 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6f22577632b06459 server_cookie=386bd8d3ff21e674 in_seq=0 out_seq=0 2024-09-10T09:40:54.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:54.443+0000 7fbcf6a00640 10 client.25931 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T09:40:54.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:54.444+0000 7fbcf6a00640 1 -- 172.21.15.5:0/2197283512 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563197106e60 con 0x563197227500 2024-09-10T09:40:54.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:54.444+0000 7fbcf6a00640 10 client.25931 handle_command_reply: tid=0 2024-09-10T09:40:54.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:54.444+0000 7fbd1a658740 1 client.25931 shutdown 2024-09-10T09:40:54.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:54.444+0000 7fbd1a658740 1 -- 172.21.15.5:0/2197283512 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563197199590 msgr2=0x5631971a78c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:40:54.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:54.444+0000 7fbd1a658740 1 --2- 172.21.15.5:0/2197283512 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563197199590 0x5631971a78c0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x5631971a0580 tx=0x7fbcfc010da0 comp rx=0 tx=0).stop 2024-09-10T09:40:54.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:54.444+0000 7fbd1a658740 1 -- 172.21.15.5:0/2197283512 shutdown_connections 2024-09-10T09:40:54.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:54.444+0000 7fbd1a658740 1 --2- 172.21.15.5:0/2197283512 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x563197227500 0x563197229920 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:54.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:54.444+0000 7fbd1a658740 1 --2- 172.21.15.5:0/2197283512 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5631971a3e50 0x5631971a7e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:54.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:54.444+0000 7fbd1a658740 1 --2- 172.21.15.5:0/2197283512 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563197199590 0x5631971a78c0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:54.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:54.444+0000 7fbd1a658740 1 --2- 172.21.15.5:0/2197283512 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563197198c70 0x5631971ac590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:54.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:54.444+0000 7fbd1a658740 1 -- 172.21.15.5:0/2197283512 >> 172.21.15.5:0/2197283512 conn(0x563197195200 msgr2=0x56319719f490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:40:54.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:54.444+0000 7fbcf6a00640 0 client.25931 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T09:40:54.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:54.444+0000 7fbd1a658740 1 -- 172.21.15.5:0/2197283512 shutdown_connections 2024-09-10T09:40:54.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:54.444+0000 7fbd1a658740 1 -- 172.21.15.5:0/2197283512 wait complete. 2024-09-10T09:40:54.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:54.444+0000 7fbd1a658740 20 client.25931 trim_cache size 0 max 16384 2024-09-10T09:40:54.449 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-10T09:40:54.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:54.445+0000 7fbd17c00640 1 -- 172.21.15.5:0/807485851 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fbcf80a2c00 msgr2=0x7fbcf80a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:40:54.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:54.445+0000 7fbd17c00640 1 --2- 172.21.15.5:0/807485851 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fbcf80a2c00 0x7fbcf80a50c0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fbd08005e30 tx=0x7fbd08005c50 comp rx=0 tx=0).stop 2024-09-10T09:40:54.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:54.445+0000 7fbd17c00640 1 -- 172.21.15.5:0/807485851 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbd10074510 msgr2=0x7fbd1019a6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:40:54.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:54.445+0000 7fbd17c00640 1 --2- 172.21.15.5:0/807485851 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbd10074510 0x7fbd1019a6c0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fbd0400e9f0 tx=0x7fbd0400eeb0 comp rx=0 tx=0).stop 2024-09-10T09:40:54.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:54.445+0000 7fbd17c00640 1 -- 172.21.15.5:0/807485851 shutdown_connections 2024-09-10T09:40:54.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:54.445+0000 7fbd17c00640 1 --2- 172.21.15.5:0/807485851 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fbcf80a2c00 0x7fbcf80a50c0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:54.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:54.445+0000 7fbd17c00640 1 --2- 172.21.15.5:0/807485851 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbd1010e450 0x7fbd1019ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:54.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:54.445+0000 7fbd17c00640 1 --2- 172.21.15.5:0/807485851 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbd10074e30 0x7fbd1019ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:54.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:54.445+0000 7fbd17c00640 1 --2- 172.21.15.5:0/807485851 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbd10074510 0x7fbd1019a6c0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:54.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:54.445+0000 7fbd17c00640 1 -- 172.21.15.5:0/807485851 >> 172.21.15.5:0/807485851 conn(0x7fbd100fd8b0 msgr2=0x7fbd100fef90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:40:54.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:54.445+0000 7fbd17c00640 1 -- 172.21.15.5:0/807485851 shutdown_connections 2024-09-10T09:40:54.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:54.445+0000 7fbd17c00640 1 -- 172.21.15.5:0/807485851 wait complete. 2024-09-10T09:40:54.992 INFO:tasks.workunit.client.0.smithi005.stdout: 1 64536 10.14 MB/sec warmup 36 sec latency 22.963 ms 2024-09-10T09:40:55.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:40:55 smithi005 ceph-mon[30350]: pgmap v470: 129 pgs: 129 active+clean; 44 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 1.2 MiB/s wr, 178 op/s 2024-09-10T09:40:55.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:40:55 smithi102 ceph-mon[35442]: pgmap v470: 129 pgs: 129 active+clean; 44 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 1.2 MiB/s wr, 178 op/s 2024-09-10T09:40:55.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:40:55 smithi159 ceph-mon[35513]: pgmap v470: 129 pgs: 129 active+clean; 44 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 1.2 MiB/s wr, 178 op/s 2024-09-10T09:40:55.992 INFO:tasks.workunit.client.0.smithi005.stdout: 1 67425 10.34 MB/sec warmup 37 sec latency 21.693 ms 2024-09-10T09:40:56.775 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-10T09:40:56.775 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-10T09:40:56.992 INFO:tasks.workunit.client.0.smithi005.stdout: 1 69387 10.26 MB/sec warmup 38 sec latency 20.641 ms 2024-09-10T09:40:57.189 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:40:57.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:40:57 smithi102 ceph-mon[35442]: pgmap v471: 129 pgs: 129 active+clean; 48 MiB data, 469 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 1.5 MiB/s wr, 245 op/s 2024-09-10T09:40:57.645 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:40:57 smithi005 ceph-mon[30350]: pgmap v471: 129 pgs: 129 active+clean; 48 MiB data, 469 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 1.5 MiB/s wr, 245 op/s 2024-09-10T09:40:57.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:57.763+0000 7f9413800640 1 -- 172.21.15.5:0/2030252605 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f940c1049a0 msgr2=0x7f940c1089f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:40:57.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:57.763+0000 7f9413800640 1 --2- 172.21.15.5:0/2030252605 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f940c1049a0 0x7f940c1089f0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f93f8009900 tx=0x7f93f802a400 comp rx=0 tx=0).stop 2024-09-10T09:40:57.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:57.763+0000 7f9413800640 1 -- 172.21.15.5:0/2030252605 shutdown_connections 2024-09-10T09:40:57.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:57.763+0000 7f9413800640 1 --2- 172.21.15.5:0/2030252605 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f940c109120 0x7f940c110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:57.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:57.763+0000 7f9413800640 1 --2- 172.21.15.5:0/2030252605 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f940c1049a0 0x7f940c1089f0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:57.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:57.763+0000 7f9413800640 1 --2- 172.21.15.5:0/2030252605 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f940c103ff0 0x7f940c1043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:57.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:57.763+0000 7f9413800640 1 -- 172.21.15.5:0/2030252605 >> 172.21.15.5:0/2030252605 conn(0x7f940c0ffa90 msgr2=0x7f940c101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:40:57.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:57.763+0000 7f9413800640 1 -- 172.21.15.5:0/2030252605 shutdown_connections 2024-09-10T09:40:57.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:57.763+0000 7f9413800640 1 -- 172.21.15.5:0/2030252605 wait complete. 2024-09-10T09:40:57.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:57.764+0000 7f9413800640 1 Processor -- start 2024-09-10T09:40:57.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:57.764+0000 7f9413800640 1 -- start start 2024-09-10T09:40:57.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:57.765+0000 7f9413800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f940c103ff0 0x7f940c19a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:40:57.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:57.765+0000 7f9413800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f940c1049a0 0x7f940c19abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:40:57.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:57.765+0000 7f9413800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f940c109120 0x7f940c19ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:40:57.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:57.765+0000 7f9413800640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f940c114120 con 0x7f940c109120 2024-09-10T09:40:57.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:57.765+0000 7f9413800640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f940c113fa0 con 0x7f940c1049a0 2024-09-10T09:40:57.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:57.765+0000 7f9413800640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f940c1142a0 con 0x7f940c103ff0 2024-09-10T09:40:57.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:57.765+0000 7f9410e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f940c103ff0 0x7f940c19a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:40:57.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:57.765+0000 7f940be00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f940c1049a0 0x7f940c19abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:40:57.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:57.765+0000 7f9410e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f940c103ff0 0x7f940c19a6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:57018/0 (socket says 172.21.15.5:57018) 2024-09-10T09:40:57.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:57.765+0000 7f9410e00640 1 -- 172.21.15.5:0/343288191 learned_addr learned my addr 172.21.15.5:0/343288191 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:40:57.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:57.765+0000 7f9411800640 1 --2- 172.21.15.5:0/343288191 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f940c109120 0x7f940c19ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:40:57.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:57.765+0000 7f940be00640 1 -- 172.21.15.5:0/343288191 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f940c103ff0 msgr2=0x7f940c19a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:40:57.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:57.765+0000 7f940be00640 1 --2- 172.21.15.5:0/343288191 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f940c103ff0 0x7f940c19a6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:57.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:57.766+0000 7f940be00640 1 -- 172.21.15.5:0/343288191 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f940c109120 msgr2=0x7f940c19ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:40:57.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:57.766+0000 7f940be00640 1 --2- 172.21.15.5:0/343288191 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f940c109120 0x7f940c19ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:57.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:57.766+0000 7f940be00640 1 -- 172.21.15.5:0/343288191 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f940c19f650 con 0x7f940c1049a0 2024-09-10T09:40:57.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:57.766+0000 7f9410e00640 1 --2- 172.21.15.5:0/343288191 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f940c103ff0 0x7f940c19a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:40:57.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:57.766+0000 7f940be00640 1 --2- 172.21.15.5:0/343288191 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f940c1049a0 0x7f940c19abe0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f93f8005940 tx=0x7f93f80042f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:40:57.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:57.766+0000 7f9409600640 1 -- 172.21.15.5:0/343288191 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93f803f030 con 0x7f940c1049a0 2024-09-10T09:40:57.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:57.766+0000 7f9409600640 1 -- 172.21.15.5:0/343288191 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f93f8003ce0 con 0x7f940c1049a0 2024-09-10T09:40:57.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:57.766+0000 7f9413800640 1 -- 172.21.15.5:0/343288191 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f940c19f8e0 con 0x7f940c1049a0 2024-09-10T09:40:57.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:57.766+0000 7f9409600640 1 -- 172.21.15.5:0/343288191 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93f8035070 con 0x7f940c1049a0 2024-09-10T09:40:57.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:57.766+0000 7f9413800640 1 -- 172.21.15.5:0/343288191 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f940c1a7220 con 0x7f940c1049a0 2024-09-10T09:40:57.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:57.770+0000 7f94162b0740 1 Processor -- start 2024-09-10T09:40:57.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:57.771+0000 7f94162b0740 1 -- start start 2024-09-10T09:40:57.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:57.771+0000 7f94162b0740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ab778c5ac0 0x55ab778c5ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:40:57.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:57.771+0000 7f94162b0740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ab778c63e0 0x55ab778d0760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:40:57.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:57.771+0000 7f9409600640 1 -- 172.21.15.5:0/343288191 <== mon.2 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f93f8042030 con 0x7f940c1049a0 2024-09-10T09:40:57.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:57.771+0000 7f94162b0740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ab778d0ca0 0x55ab778d3090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:40:57.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:57.771+0000 7f94162b0740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55ab7782c290 con 0x55ab778c5ac0 2024-09-10T09:40:57.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:57.771+0000 7f94162b0740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55ab777d3b40 con 0x55ab778d0ca0 2024-09-10T09:40:57.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:57.771+0000 7f94162b0740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55ab77828250 con 0x55ab778c63e0 2024-09-10T09:40:57.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:57.771+0000 7f9410e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ab778c63e0 0x55ab778d0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:40:57.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:57.771+0000 7f9410e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ab778c63e0 0x55ab778d0760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:57026/0 (socket says 172.21.15.5:57026) 2024-09-10T09:40:57.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:57.772+0000 7f9410e00640 1 -- 172.21.15.5:0/2828466863 learned_addr learned my addr 172.21.15.5:0/2828466863 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:40:57.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:57.772+0000 7f9411800640 1 --2- 172.21.15.5:0/2828466863 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ab778c5ac0 0x55ab778c5ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:40:57.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:57.772+0000 7f940be00640 1 --2- 172.21.15.5:0/2828466863 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ab778d0ca0 0x55ab778d3090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:40:57.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:57.772+0000 7f9410e00640 1 -- 172.21.15.5:0/2828466863 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ab778d0ca0 msgr2=0x55ab778d3090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:40:57.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:57.772+0000 7f9410e00640 1 --2- 172.21.15.5:0/2828466863 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ab778d0ca0 0x55ab778d3090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:57.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:57.772+0000 7f9410e00640 1 -- 172.21.15.5:0/2828466863 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ab778c5ac0 msgr2=0x55ab778c5ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:40:57.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:57.772+0000 7f9410e00640 1 --2- 172.21.15.5:0/2828466863 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ab778c5ac0 0x55ab778c5ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:57.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:57.772+0000 7f9410e00640 1 -- 172.21.15.5:0/2828466863 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ab7759d6a0 con 0x55ab778c63e0 2024-09-10T09:40:57.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:57.772+0000 7f940be00640 1 --2- 172.21.15.5:0/2828466863 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ab778d0ca0 0x55ab778d3090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T09:40:57.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:57.772+0000 7f9410e00640 1 --2- 172.21.15.5:0/2828466863 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ab778c63e0 0x55ab778d0760 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f94000118b0 tx=0x7f9400011d70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=24df1e8e5522282a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:40:57.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:57.773+0000 7f93fe000640 1 -- 172.21.15.5:0/2828466863 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f940000fc00 con 0x55ab778c63e0 2024-09-10T09:40:57.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:57.773+0000 7f93fe000640 1 -- 172.21.15.5:0/2828466863 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9400016070 con 0x55ab778c63e0 2024-09-10T09:40:57.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:57.773+0000 7f93fe000640 1 -- 172.21.15.5:0/2828466863 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94000126b0 con 0x55ab778c63e0 2024-09-10T09:40:57.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:57.773+0000 7f9409600640 1 --2- 172.21.15.5:0/343288191 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f93f40a2b80 0x7f93f40a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:40:57.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:57.773+0000 7f94162b0740 1 -- 172.21.15.5:0/2828466863 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ab778c63e0 msgr2=0x55ab778d0760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:40:57.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:57.773+0000 7f94162b0740 1 --2- 172.21.15.5:0/2828466863 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ab778c63e0 0x55ab778d0760 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f94000118b0 tx=0x7f9400011d70 comp rx=0 tx=0).stop 2024-09-10T09:40:57.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:57.773+0000 7f9409600640 1 -- 172.21.15.5:0/343288191 <== mon.2 v2:172.21.15.102:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f93f80ef8b0 con 0x7f940c1049a0 2024-09-10T09:40:57.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:57.773+0000 7f94162b0740 1 -- 172.21.15.5:0/2828466863 shutdown_connections 2024-09-10T09:40:57.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:57.773+0000 7f94162b0740 1 --2- 172.21.15.5:0/2828466863 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ab778d0ca0 0x55ab778d3090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:57.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:57.773+0000 7f94162b0740 1 --2- 172.21.15.5:0/2828466863 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ab778c63e0 0x55ab778d0760 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:57.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:57.773+0000 7f94162b0740 1 --2- 172.21.15.5:0/2828466863 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ab778c5ac0 0x55ab778c5ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:57.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:57.773+0000 7f94162b0740 1 -- 172.21.15.5:0/2828466863 >> 172.21.15.5:0/2828466863 conn(0x55ab778c2050 msgr2=0x55ab778c2430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:40:57.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:57.773+0000 7f9411800640 1 --2- 172.21.15.5:0/343288191 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f93f40a2b80 0x7f93f40a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:40:57.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:57.774+0000 7f94162b0740 1 -- 172.21.15.5:0/2828466863 shutdown_connections 2024-09-10T09:40:57.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:57.774+0000 7f94162b0740 1 -- 172.21.15.5:0/2828466863 wait complete. 2024-09-10T09:40:57.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:57.774+0000 7f9411800640 1 --2- 172.21.15.5:0/343288191 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f93f40a2b80 0x7f93f40a5040 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f940400b710 tx=0x7f940400b2e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:40:57.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:57.774+0000 7f94162b0740 1 -- start start 2024-09-10T09:40:57.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:57.774+0000 7f94162b0740 1 Processor -- start 2024-09-10T09:40:57.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:57.775+0000 7f94162b0740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ab778c5ac0 0x55ab778d05a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:40:57.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:57.775+0000 7f94162b0740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ab778c63e0 0x55ab778cd850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:40:57.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:57.775+0000 7f94162b0740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ab778d0ca0 0x55ab778cdd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:40:57.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:57.775+0000 7f94162b0740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55ab778d3910 con 0x55ab778c5ac0 2024-09-10T09:40:57.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:57.775+0000 7f94162b0740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55ab777e6fa0 con 0x55ab778c63e0 2024-09-10T09:40:57.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:57.775+0000 7f94162b0740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55ab778d3a90 con 0x55ab778d0ca0 2024-09-10T09:40:57.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:57.775+0000 7f94162b0740 10 client.0 _ll_register_callbacks cb 0x55ab7753aba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:40:57.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:57.775+0000 7f9410e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ab778c5ac0 0x55ab778d05a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:40:57.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:57.775+0000 7f9411800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ab778d0ca0 0x55ab778cdd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:40:57.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:57.776+0000 7f9410e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ab778c5ac0 0x55ab778d05a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43430/0 (socket says 172.21.15.5:43430) 2024-09-10T09:40:57.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:57.776+0000 7f9411800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ab778d0ca0 0x55ab778cdd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:57034/0 (socket says 172.21.15.5:57034) 2024-09-10T09:40:57.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:57.776+0000 7f9410e00640 1 -- 172.21.15.5:0/2197802124 learned_addr learned my addr 172.21.15.5:0/2197802124 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:40:57.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:57.776+0000 7f940be00640 1 --2- 172.21.15.5:0/2197802124 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ab778c63e0 0x55ab778cd850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:40:57.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:57.776+0000 7f9411800640 1 -- 172.21.15.5:0/2197802124 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ab778c63e0 msgr2=0x55ab778cd850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:40:57.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:57.776+0000 7f9411800640 1 --2- 172.21.15.5:0/2197802124 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ab778c63e0 0x55ab778cd850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:57.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:57.776+0000 7f9411800640 1 -- 172.21.15.5:0/2197802124 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ab778c5ac0 msgr2=0x55ab778d05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:40:57.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:57.776+0000 7f9411800640 1 --2- 172.21.15.5:0/2197802124 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ab778c5ac0 0x55ab778d05a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:57.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:57.776+0000 7f9411800640 1 -- 172.21.15.5:0/2197802124 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ab778ce5e0 con 0x55ab778d0ca0 2024-09-10T09:40:57.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:57.776+0000 7f9411800640 1 --2- 172.21.15.5:0/2197802124 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ab778d0ca0 0x55ab778cdd90 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f940400bfa0 tx=0x7f9404004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=55ec6757cf39462c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:40:57.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:57.776+0000 7f93f2a00640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-09-10T09:40:57.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:57.776+0000 7f93f2a00640 1 -- 172.21.15.5:0/2197802124 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f940401fc50 con 0x55ab778d0ca0 2024-09-10T09:40:57.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:57.777+0000 7f93f2a00640 1 -- 172.21.15.5:0/2197802124 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f9404017030 con 0x55ab778d0ca0 2024-09-10T09:40:57.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:57.777+0000 7f94162b0740 1 -- 172.21.15.5:0/2197802124 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9400016070 con 0x55ab778d0ca0 2024-09-10T09:40:57.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:57.777+0000 7f93f2a00640 1 -- 172.21.15.5:0/2197802124 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f9404032050 con 0x55ab778d0ca0 2024-09-10T09:40:57.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:57.777+0000 7f94162b0740 10 client.25972 fetch_fsmap learned FSMap version 32 2024-09-10T09:40:57.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:57.777+0000 7f94162b0740 1 -- 172.21.15.5:0/2197802124 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55ab778df820 con 0x55ab778d0ca0 2024-09-10T09:40:57.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:57.777+0000 7f93f2a00640 1 -- 172.21.15.5:0/2197802124 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9400016070 con 0x55ab778d0ca0 2024-09-10T09:40:57.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:57.777+0000 7f93f2a00640 1 -- 172.21.15.5:0/2197802124 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7f94040054e0 con 0x55ab778d0ca0 2024-09-10T09:40:57.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:57.777+0000 7f94162b0740 10 client.25972 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T09:40:57.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:57.777+0000 7f94162b0740 10 client.25972 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T09:40:57.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:57.777+0000 7f94162b0740 20 client.25972 populate_metadata read hostname 'smithi005' 2024-09-10T09:40:57.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:57.777+0000 7f94162b0740 1 --2- 172.21.15.5:0/2197802124 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55ab778e0db0 0x55ab778e3230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:40:57.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:57.777+0000 7f94162b0740 4 client.25972 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:40:57.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:57.777+0000 7f94162b0740 1 -- 172.21.15.5:0/2197802124 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ab77839800 con 0x55ab778e0db0 2024-09-10T09:40:57.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:57.778+0000 7f9410e00640 1 --2- 172.21.15.5:0/2197802124 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55ab778e0db0 0x55ab778e3230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:40:57.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:57.778+0000 7f9410e00640 1 --2- 172.21.15.5:0/2197802124 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55ab778e0db0 0x55ab778e3230 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b1c26ddce06e9c69 server_cookie=93fec18924af6ff3 in_seq=0 out_seq=0 2024-09-10T09:40:57.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:57.778+0000 7f93f2a00640 10 client.25972 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T09:40:57.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:57.779+0000 7f93f2a00640 1 -- 172.21.15.5:0/2197802124 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ab77839800 con 0x55ab778e0db0 2024-09-10T09:40:57.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:57.779+0000 7f93f2a00640 10 client.25972 handle_command_reply: tid=0 2024-09-10T09:40:57.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:57.779+0000 7f94162b0740 1 client.25972 shutdown 2024-09-10T09:40:57.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:57.779+0000 7f94162b0740 1 -- 172.21.15.5:0/2197802124 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ab778d0ca0 msgr2=0x55ab778cdd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:40:57.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:57.779+0000 7f94162b0740 1 --2- 172.21.15.5:0/2197802124 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ab778d0ca0 0x55ab778cdd90 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f940400bfa0 tx=0x7f9404004730 comp rx=0 tx=0).stop 2024-09-10T09:40:57.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:57.780+0000 7f94162b0740 1 -- 172.21.15.5:0/2197802124 shutdown_connections 2024-09-10T09:40:57.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:57.780+0000 7f94162b0740 1 --2- 172.21.15.5:0/2197802124 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55ab778e0db0 0x55ab778e3230 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:57.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:57.780+0000 7f94162b0740 1 --2- 172.21.15.5:0/2197802124 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ab778d0ca0 0x55ab778cdd90 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:57.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:57.780+0000 7f94162b0740 1 --2- 172.21.15.5:0/2197802124 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ab778c63e0 0x55ab778cd850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:57.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:57.780+0000 7f94162b0740 1 --2- 172.21.15.5:0/2197802124 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ab778c5ac0 0x55ab778d05a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:57.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:57.780+0000 7f93f2a00640 0 client.25972 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T09:40:57.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:57.780+0000 7f94162b0740 1 -- 172.21.15.5:0/2197802124 >> 172.21.15.5:0/2197802124 conn(0x55ab778c2050 msgr2=0x55ab778c11c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:40:57.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:57.780+0000 7f94162b0740 1 -- 172.21.15.5:0/2197802124 shutdown_connections 2024-09-10T09:40:57.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:57.780+0000 7f94162b0740 1 -- 172.21.15.5:0/2197802124 wait complete. 2024-09-10T09:40:57.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:57.780+0000 7f94162b0740 20 client.25972 trim_cache size 0 max 16384 2024-09-10T09:40:57.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:57.798+0000 7f94162b0740 1 Processor -- start 2024-09-10T09:40:57.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:57.798+0000 7f94162b0740 1 -- start start 2024-09-10T09:40:57.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:57.799+0000 7f94162b0740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ab778c2050 0x55ab778c8ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:40:57.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:57.799+0000 7f94162b0740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ab778c5ac0 0x55ab778cf7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:40:57.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:57.799+0000 7f94162b0740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ab778c63e0 0x55ab778e8b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:40:57.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:57.799+0000 7f94162b0740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f940c1b1010 con 0x55ab778c5ac0 2024-09-10T09:40:57.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:57.799+0000 7f94162b0740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f940c1b2c10 con 0x55ab778c63e0 2024-09-10T09:40:57.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:57.799+0000 7f94162b0740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f940c1b16e0 con 0x55ab778c2050 2024-09-10T09:40:57.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:57.799+0000 7f9411800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ab778c63e0 0x55ab778e8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:40:57.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:57.799+0000 7f940be00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ab778c5ac0 0x55ab778cf7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:40:57.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:57.799+0000 7f9410e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ab778c2050 0x55ab778c8ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:40:57.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:57.799+0000 7f9411800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ab778c63e0 0x55ab778e8b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:41158/0 (socket says 172.21.15.5:41158) 2024-09-10T09:40:57.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:57.799+0000 7f9411800640 1 -- 172.21.15.5:0/1009092158 learned_addr learned my addr 172.21.15.5:0/1009092158 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:40:57.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:57.799+0000 7f9411800640 1 -- 172.21.15.5:0/1009092158 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ab778c2050 msgr2=0x55ab778c8ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:40:57.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:57.799+0000 7f9411800640 1 --2- 172.21.15.5:0/1009092158 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ab778c2050 0x55ab778c8ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:57.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:57.799+0000 7f9411800640 1 -- 172.21.15.5:0/1009092158 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ab778c5ac0 msgr2=0x55ab778cf7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:40:57.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:57.799+0000 7f9411800640 1 --2- 172.21.15.5:0/1009092158 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ab778c5ac0 0x55ab778cf7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:57.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:57.799+0000 7f9411800640 1 -- 172.21.15.5:0/1009092158 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ab778dd510 con 0x55ab778c63e0 2024-09-10T09:40:57.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:57.800+0000 7f9411800640 1 --2- 172.21.15.5:0/1009092158 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ab778c63e0 0x55ab778e8b00 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f9404004010 tx=0x7f94040044c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3a074a61812c2381 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:40:57.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:57.800+0000 7f93fd600640 1 -- 172.21.15.5:0/1009092158 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f940403b070 con 0x55ab778c63e0 2024-09-10T09:40:57.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:57.800+0000 7f93fd600640 1 -- 172.21.15.5:0/1009092158 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9404027820 con 0x55ab778c63e0 2024-09-10T09:40:57.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:57.800+0000 7f93fd600640 1 -- 172.21.15.5:0/1009092158 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94040052f0 con 0x55ab778c63e0 2024-09-10T09:40:57.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:57.801+0000 7f94162b0740 1 -- 172.21.15.5:0/1009092158 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ab778c63e0 msgr2=0x55ab778e8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:40:57.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:57.801+0000 7f94162b0740 1 --2- 172.21.15.5:0/1009092158 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ab778c63e0 0x55ab778e8b00 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f9404004010 tx=0x7f94040044c0 comp rx=0 tx=0).stop 2024-09-10T09:40:57.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:57.801+0000 7f94162b0740 1 -- 172.21.15.5:0/1009092158 shutdown_connections 2024-09-10T09:40:57.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:57.801+0000 7f94162b0740 1 --2- 172.21.15.5:0/1009092158 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ab778c63e0 0x55ab778e8b00 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:57.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:57.801+0000 7f94162b0740 1 --2- 172.21.15.5:0/1009092158 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ab778c5ac0 0x55ab778cf7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:57.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:57.801+0000 7f94162b0740 1 --2- 172.21.15.5:0/1009092158 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ab778c2050 0x55ab778c8ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:57.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:57.801+0000 7f94162b0740 1 -- 172.21.15.5:0/1009092158 >> 172.21.15.5:0/1009092158 conn(0x55ab778cdef0 msgr2=0x55ab778cf0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:40:57.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:57.801+0000 7f94162b0740 1 -- 172.21.15.5:0/1009092158 shutdown_connections 2024-09-10T09:40:57.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:57.801+0000 7f94162b0740 1 -- 172.21.15.5:0/1009092158 wait complete. 2024-09-10T09:40:57.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:57.801+0000 7f94162b0740 1 -- start start 2024-09-10T09:40:57.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:57.802+0000 7f94162b0740 1 Processor -- start 2024-09-10T09:40:57.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:57.802+0000 7f94162b0740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ab778c2050 0x55ab77944df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:40:57.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:57.802+0000 7f94162b0740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ab778c5ac0 0x55ab77945330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:40:57.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:57.802+0000 7f94162b0740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ab778c63e0 0x55ab778d5220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:40:57.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:57.802+0000 7f94162b0740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f940c1b2480 con 0x55ab778c5ac0 2024-09-10T09:40:57.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:57.802+0000 7f94162b0740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f940c1b1db0 con 0x55ab778c63e0 2024-09-10T09:40:57.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:57.802+0000 7f94162b0740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f940c1aeff0 con 0x55ab778c2050 2024-09-10T09:40:57.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:57.802+0000 7f94162b0740 10 client.0 _ll_register_callbacks cb 0x55ab778ce7b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:40:57.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:57.802+0000 7f9410e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ab778c2050 0x55ab77944df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:40:57.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:57.802+0000 7f940be00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ab778c5ac0 0x55ab77945330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:40:57.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:57.802+0000 7f9411800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ab778c63e0 0x55ab778d5220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:40:57.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:57.802+0000 7f9410e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ab778c2050 0x55ab77944df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:57048/0 (socket says 172.21.15.5:57048) 2024-09-10T09:40:57.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:57.802+0000 7f940be00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ab778c5ac0 0x55ab77945330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43438/0 (socket says 172.21.15.5:43438) 2024-09-10T09:40:57.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:57.803+0000 7f9410e00640 1 -- 172.21.15.5:0/1270444605 learned_addr learned my addr 172.21.15.5:0/1270444605 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:40:57.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:57.803+0000 7f940be00640 1 -- 172.21.15.5:0/1270444605 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ab778c2050 msgr2=0x55ab77944df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:40:57.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:57.803+0000 7f940be00640 1 --2- 172.21.15.5:0/1270444605 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ab778c2050 0x55ab77944df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:57.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:57.803+0000 7f940be00640 1 -- 172.21.15.5:0/1270444605 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ab778c63e0 msgr2=0x55ab778d5220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:40:57.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:57.803+0000 7f940be00640 1 --2- 172.21.15.5:0/1270444605 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ab778c63e0 0x55ab778d5220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:57.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:57.803+0000 7f940be00640 1 -- 172.21.15.5:0/1270444605 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ab778d5a20 con 0x55ab778c5ac0 2024-09-10T09:40:57.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:57.803+0000 7f9411800640 1 --2- 172.21.15.5:0/1270444605 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ab778c63e0 0x55ab778d5220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:40:57.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:57.803+0000 7f9410e00640 1 --2- 172.21.15.5:0/1270444605 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ab778c2050 0x55ab77944df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:40:57.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:57.803+0000 7f940be00640 1 --2- 172.21.15.5:0/1270444605 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ab778c5ac0 0x55ab77945330 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x55ab778c33d0 tx=0x7f93f80bbc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aeba916c7a5b4f48 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:40:57.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:57.803+0000 7f93f2a00640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-10T09:40:57.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:57.803+0000 7f93f2a00640 1 -- 172.21.15.5:0/1270444605 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93f8049050 con 0x55ab778c5ac0 2024-09-10T09:40:57.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:57.804+0000 7f93f2a00640 1 -- 172.21.15.5:0/1270444605 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f93f80b83e0 con 0x55ab778c5ac0 2024-09-10T09:40:57.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:57.804+0000 7f94162b0740 1 -- 172.21.15.5:0/1270444605 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9404027820 con 0x55ab778c5ac0 2024-09-10T09:40:57.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:57.804+0000 7f93f2a00640 1 -- 172.21.15.5:0/1270444605 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f93f80bbd70 con 0x55ab778c5ac0 2024-09-10T09:40:57.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:57.804+0000 7f94162b0740 10 client.15942 fetch_fsmap learned FSMap version 32 2024-09-10T09:40:57.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:57.804+0000 7f94162b0740 1 -- 172.21.15.5:0/1270444605 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55ab7794c630 con 0x55ab778c5ac0 2024-09-10T09:40:57.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:57.804+0000 7f93f2a00640 1 -- 172.21.15.5:0/1270444605 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9404027820 con 0x55ab778c5ac0 2024-09-10T09:40:57.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:57.804+0000 7f93f2a00640 1 -- 172.21.15.5:0/1270444605 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7f93f809abe0 con 0x55ab778c5ac0 2024-09-10T09:40:57.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:57.804+0000 7f94162b0740 10 client.15942 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T09:40:57.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:57.804+0000 7f94162b0740 10 client.15942 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T09:40:57.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:57.804+0000 7f94162b0740 20 client.15942 populate_metadata read hostname 'smithi005' 2024-09-10T09:40:57.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:57.804+0000 7f94162b0740 1 --2- 172.21.15.5:0/1270444605 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55ab778d0ca0 0x55ab7794eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:40:57.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:57.804+0000 7f94162b0740 4 client.15942 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-10T09:40:57.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:57.804+0000 7f94162b0740 1 -- 172.21.15.5:0/1270444605 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55ab77833d10 con 0x55ab778d0ca0 2024-09-10T09:40:57.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:57.805+0000 7f9410e00640 1 --2- 172.21.15.5:0/1270444605 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55ab778d0ca0 0x55ab7794eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:40:57.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:57.805+0000 7f9410e00640 1 --2- 172.21.15.5:0/1270444605 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55ab778d0ca0 0x55ab7794eeb0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=53626b7377708604 server_cookie=d83f654df32da0da in_seq=0 out_seq=0 2024-09-10T09:40:57.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:57.805+0000 7f93f2a00640 10 client.15942 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T09:40:57.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:57.806+0000 7f93f2a00640 1 -- 172.21.15.5:0/1270444605 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x7f9400016080 con 0x55ab778d0ca0 2024-09-10T09:40:57.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:57.806+0000 7f93f2a00640 10 client.15942 handle_command_reply: tid=0 2024-09-10T09:40:57.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:57.806+0000 7f94162b0740 1 client.15942 shutdown 2024-09-10T09:40:57.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:57.806+0000 7f94162b0740 1 -- 172.21.15.5:0/1270444605 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ab778c5ac0 msgr2=0x55ab77945330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:40:57.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:57.806+0000 7f94162b0740 1 --2- 172.21.15.5:0/1270444605 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ab778c5ac0 0x55ab77945330 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x55ab778c33d0 tx=0x7f93f80bbc00 comp rx=0 tx=0).stop 2024-09-10T09:40:57.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:57.807+0000 7f94162b0740 1 -- 172.21.15.5:0/1270444605 shutdown_connections 2024-09-10T09:40:57.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:57.807+0000 7f94162b0740 1 --2- 172.21.15.5:0/1270444605 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55ab778d0ca0 0x55ab7794eeb0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:57.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:57.807+0000 7f94162b0740 1 --2- 172.21.15.5:0/1270444605 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ab778c63e0 0x55ab778d5220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:57.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:57.807+0000 7f94162b0740 1 --2- 172.21.15.5:0/1270444605 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ab778c5ac0 0x55ab77945330 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:57.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:57.807+0000 7f94162b0740 1 --2- 172.21.15.5:0/1270444605 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ab778c2050 0x55ab77944df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:57.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:57.807+0000 7f93f2a00640 0 client.15942 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T09:40:57.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:57.807+0000 7f94162b0740 1 -- 172.21.15.5:0/1270444605 >> 172.21.15.5:0/1270444605 conn(0x55ab778cdef0 msgr2=0x55ab778c7560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:40:57.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:57.807+0000 7f94162b0740 1 -- 172.21.15.5:0/1270444605 shutdown_connections 2024-09-10T09:40:57.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:57.807+0000 7f94162b0740 1 -- 172.21.15.5:0/1270444605 wait complete. 2024-09-10T09:40:57.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:57.807+0000 7f94162b0740 20 client.15942 trim_cache size 0 max 16384 2024-09-10T09:40:57.811 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-10T09:40:57.811 INFO:teuthology.orchestra.run.smithi005.stdout: "return_code": 0, 2024-09-10T09:40:57.811 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_tag": "5eeb0cee-4bfb-4c8b-8b78-241139cb3771", 2024-09-10T09:40:57.811 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": "asynchronous" 2024-09-10T09:40:57.811 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-10T09:40:57.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:57.808+0000 7f9413800640 1 -- 172.21.15.5:0/343288191 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f93f40a2b80 msgr2=0x7f93f40a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:40:57.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:57.808+0000 7f9413800640 1 --2- 172.21.15.5:0/343288191 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f93f40a2b80 0x7f93f40a5040 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f940400b710 tx=0x7f940400b2e0 comp rx=0 tx=0).stop 2024-09-10T09:40:57.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:57.808+0000 7f9413800640 1 -- 172.21.15.5:0/343288191 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f940c1049a0 msgr2=0x7f940c19abe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:40:57.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:57.808+0000 7f9413800640 1 --2- 172.21.15.5:0/343288191 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f940c1049a0 0x7f940c19abe0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f93f8005940 tx=0x7f93f80042f0 comp rx=0 tx=0).stop 2024-09-10T09:40:57.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:57.808+0000 7f9413800640 1 -- 172.21.15.5:0/343288191 shutdown_connections 2024-09-10T09:40:57.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:57.808+0000 7f9413800640 1 --2- 172.21.15.5:0/343288191 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f93f40a2b80 0x7f93f40a5040 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:57.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:57.808+0000 7f9413800640 1 --2- 172.21.15.5:0/343288191 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f940c109120 0x7f940c19ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:57.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:57.808+0000 7f9413800640 1 --2- 172.21.15.5:0/343288191 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f940c1049a0 0x7f940c19abe0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:57.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:57.808+0000 7f9413800640 1 --2- 172.21.15.5:0/343288191 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f940c103ff0 0x7f940c19a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:57.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:57.808+0000 7f9413800640 1 -- 172.21.15.5:0/343288191 >> 172.21.15.5:0/343288191 conn(0x7f940c0ffa90 msgr2=0x7f940c100d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:40:57.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:57.808+0000 7f9413800640 1 -- 172.21.15.5:0/343288191 shutdown_connections 2024-09-10T09:40:57.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:57.808+0000 7f9413800640 1 -- 172.21.15.5:0/343288191 wait complete. 2024-09-10T09:40:57.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:40:57 smithi159 ceph-mon[35513]: pgmap v471: 129 pgs: 129 active+clean; 48 MiB data, 469 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 1.5 MiB/s wr, 245 op/s 2024-09-10T09:40:57.992 INFO:tasks.workunit.client.0.smithi005.stdout: 1 71684 10.35 MB/sec warmup 39 sec latency 11.797 ms 2024-09-10T09:40:58.542 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:40:58 smithi005 ceph-mon[30350]: scrub queued for path: / 2024-09-10T09:40:58.542 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:40:58 smithi005 ceph-mon[30350]: scrub summary: idle+waiting paths [/] 2024-09-10T09:40:58.543 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:40:58 smithi005 ceph-mon[30350]: scrub summary: active paths [/] 2024-09-10T09:40:58.543 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:40:58 smithi005 ceph-mon[30350]: scrub summary: idle+waiting paths [/] 2024-09-10T09:40:58.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:40:58 smithi102 ceph-mon[35442]: scrub queued for path: / 2024-09-10T09:40:58.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:40:58 smithi102 ceph-mon[35442]: scrub summary: idle+waiting paths [/] 2024-09-10T09:40:58.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:40:58 smithi102 ceph-mon[35442]: scrub summary: active paths [/] 2024-09-10T09:40:58.635 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:40:58 smithi102 ceph-mon[35442]: scrub summary: idle+waiting paths [/] 2024-09-10T09:40:58.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:40:58 smithi159 ceph-mon[35513]: scrub queued for path: / 2024-09-10T09:40:58.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:40:58 smithi159 ceph-mon[35513]: scrub summary: idle+waiting paths [/] 2024-09-10T09:40:58.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:40:58 smithi159 ceph-mon[35513]: scrub summary: active paths [/] 2024-09-10T09:40:58.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:40:58 smithi159 ceph-mon[35513]: scrub summary: idle+waiting paths [/] 2024-09-10T09:40:58.959 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T09:40:58.992 INFO:tasks.workunit.client.0.smithi005.stdout: 1 74538 10.52 MB/sec warmup 40 sec latency 14.632 ms 2024-09-10T09:40:59.359 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:40:59.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:40:59 smithi102 ceph-mon[35442]: pgmap v472: 129 pgs: 129 active+clean; 48 MiB data, 470 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 1.4 MiB/s wr, 225 op/s 2024-09-10T09:40:59.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:40:59 smithi102 ceph-mon[35442]: scrub summary: active paths [/] 2024-09-10T09:40:59.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:40:59 smithi102 ceph-mon[35442]: scrub summary: idle+waiting paths [/] 2024-09-10T09:40:59.694 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:40:59 smithi005 ceph-mon[30350]: pgmap v472: 129 pgs: 129 active+clean; 48 MiB data, 470 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 1.4 MiB/s wr, 225 op/s 2024-09-10T09:40:59.694 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:40:59 smithi005 ceph-mon[30350]: scrub summary: active paths [/] 2024-09-10T09:40:59.694 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:40:59 smithi005 ceph-mon[30350]: scrub summary: idle+waiting paths [/] 2024-09-10T09:40:59.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:40:59 smithi159 ceph-mon[35513]: pgmap v472: 129 pgs: 129 active+clean; 48 MiB data, 470 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 1.4 MiB/s wr, 225 op/s 2024-09-10T09:40:59.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:40:59 smithi159 ceph-mon[35513]: scrub summary: active paths [/] 2024-09-10T09:40:59.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:40:59 smithi159 ceph-mon[35513]: scrub summary: idle+waiting paths [/] 2024-09-10T09:40:59.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:59.959+0000 7f33c6200640 1 -- 172.21.15.5:0/3057625710 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f33c0077040 msgr2=0x7f33c0112510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:40:59.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:59.959+0000 7f33c6200640 1 --2- 172.21.15.5:0/3057625710 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f33c0077040 0x7f33c0112510 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f33ac009900 tx=0x7f33ac02a3f0 comp rx=0 tx=0).stop 2024-09-10T09:40:59.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:59.959+0000 7f33c6200640 1 -- 172.21.15.5:0/3057625710 shutdown_connections 2024-09-10T09:40:59.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:59.959+0000 7f33c6200640 1 --2- 172.21.15.5:0/3057625710 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f33c0112a50 0x7f33c0114e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:59.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:59.959+0000 7f33c6200640 1 --2- 172.21.15.5:0/3057625710 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f33c0077040 0x7f33c0112510 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:59.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:59.959+0000 7f33c6200640 1 --2- 172.21.15.5:0/3057625710 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f33c0076720 0x7f33c0076b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:59.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:59.959+0000 7f33c6200640 1 -- 172.21.15.5:0/3057625710 >> 172.21.15.5:0/3057625710 conn(0x7f33c00ffa50 msgr2=0x7f33c0101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:40:59.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:59.959+0000 7f33c6200640 1 -- 172.21.15.5:0/3057625710 shutdown_connections 2024-09-10T09:40:59.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:59.959+0000 7f33c6200640 1 -- 172.21.15.5:0/3057625710 wait complete. 2024-09-10T09:40:59.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:59.960+0000 7f33c6200640 1 Processor -- start 2024-09-10T09:40:59.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:59.961+0000 7f33c6200640 1 -- start start 2024-09-10T09:40:59.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:59.961+0000 7f33c6200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f33c0076720 0x7f33c01a4780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:40:59.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:59.961+0000 7f33c6200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f33c0077040 0x7f33c01a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:40:59.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:59.961+0000 7f33c6200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f33c0112a50 0x7f33c019e940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:40:59.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:59.961+0000 7f33c6200640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f33c0118630 con 0x7f33c0076720 2024-09-10T09:40:59.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:59.961+0000 7f33c6200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f33c01184b0 con 0x7f33c0077040 2024-09-10T09:40:59.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:59.961+0000 7f33c6200640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f33c01187b0 con 0x7f33c0112a50 2024-09-10T09:40:59.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:59.961+0000 7f33bf400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f33c0076720 0x7f33c01a4780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:40:59.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:59.961+0000 7f33bea00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f33c0077040 0x7f33c01a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:40:59.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:59.961+0000 7f33bf400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f33c0076720 0x7f33c01a4780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43448/0 (socket says 172.21.15.5:43448) 2024-09-10T09:40:59.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:59.961+0000 7f33bf400640 1 -- 172.21.15.5:0/4206535572 learned_addr learned my addr 172.21.15.5:0/4206535572 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:40:59.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:59.962+0000 7f33bfe00640 1 --2- 172.21.15.5:0/4206535572 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f33c0112a50 0x7f33c019e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:40:59.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:59.962+0000 7f33bf400640 1 -- 172.21.15.5:0/4206535572 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f33c0112a50 msgr2=0x7f33c019e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:40:59.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:59.962+0000 7f33bf400640 1 --2- 172.21.15.5:0/4206535572 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f33c0112a50 0x7f33c019e940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:59.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:59.962+0000 7f33bf400640 1 -- 172.21.15.5:0/4206535572 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f33c0077040 msgr2=0x7f33c01a4cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:40:59.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:59.962+0000 7f33bf400640 1 --2- 172.21.15.5:0/4206535572 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f33c0077040 0x7f33c01a4cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:59.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:59.962+0000 7f33bf400640 1 -- 172.21.15.5:0/4206535572 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33c019f070 con 0x7f33c0076720 2024-09-10T09:40:59.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:59.962+0000 7f33bf400640 1 --2- 172.21.15.5:0/4206535572 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f33c0076720 0x7f33c01a4780 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f33b000e9f0 tx=0x7f33b000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:40:59.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:59.962+0000 7f33b7e00640 1 -- 172.21.15.5:0/4206535572 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33b000cd80 con 0x7f33c0076720 2024-09-10T09:40:59.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:59.963+0000 7f33c6200640 1 -- 172.21.15.5:0/4206535572 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33c019f360 con 0x7f33c0076720 2024-09-10T09:40:59.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:59.963+0000 7f33b7e00640 1 -- 172.21.15.5:0/4206535572 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f33b0005500 con 0x7f33c0076720 2024-09-10T09:40:59.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:59.963+0000 7f33b7e00640 1 -- 172.21.15.5:0/4206535572 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33b0010690 con 0x7f33c0076720 2024-09-10T09:40:59.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:59.963+0000 7f33c6200640 1 -- 172.21.15.5:0/4206535572 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33c01ab650 con 0x7f33c0076720 2024-09-10T09:40:59.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:59.966+0000 7f33b7e00640 1 -- 172.21.15.5:0/4206535572 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f33b00046c0 con 0x7f33c0076720 2024-09-10T09:40:59.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:59.966+0000 7f33c8c6e740 1 Processor -- start 2024-09-10T09:40:59.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:59.966+0000 7f33c8c6e740 1 -- start start 2024-09-10T09:40:59.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:59.967+0000 7f33c8c6e740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558cce891b90 0x558cce891f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:40:59.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:59.967+0000 7f33c8c6e740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558cce8924b0 0x558cce89c830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:40:59.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:59.967+0000 7f33c8c6e740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558cce89cd70 0x558cce89f160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:40:59.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:59.967+0000 7f33c8c6e740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x558cce7f83e0 con 0x558cce89cd70 2024-09-10T09:40:59.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:59.967+0000 7f33c8c6e740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x558cce79fd80 con 0x558cce891b90 2024-09-10T09:40:59.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:59.967+0000 7f33c8c6e740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x558cce7f43a0 con 0x558cce8924b0 2024-09-10T09:40:59.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:59.967+0000 7f33bf400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558cce8924b0 0x558cce89c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:40:59.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:59.967+0000 7f33bfe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558cce891b90 0x558cce891f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:40:59.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:59.967+0000 7f33bfe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558cce891b90 0x558cce891f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:41196/0 (socket says 172.21.15.5:41196) 2024-09-10T09:40:59.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:59.967+0000 7f33bfe00640 1 -- 172.21.15.5:0/1929609964 learned_addr learned my addr 172.21.15.5:0/1929609964 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:40:59.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:59.967+0000 7f33bf400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558cce8924b0 0x558cce89c830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:57066/0 (socket says 172.21.15.5:57066) 2024-09-10T09:40:59.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:59.967+0000 7f33bea00640 1 --2- 172.21.15.5:0/1929609964 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558cce89cd70 0x558cce89f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:40:59.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:59.968+0000 7f33bf400640 1 -- 172.21.15.5:0/1929609964 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558cce891b90 msgr2=0x558cce891f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:40:59.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:59.968+0000 7f33bf400640 1 --2- 172.21.15.5:0/1929609964 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558cce891b90 0x558cce891f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:59.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:59.968+0000 7f33bf400640 1 -- 172.21.15.5:0/1929609964 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558cce89cd70 msgr2=0x558cce89f160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:40:59.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:59.968+0000 7f33bf400640 1 --2- 172.21.15.5:0/1929609964 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558cce89cd70 0x558cce89f160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:59.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:59.968+0000 7f33bf400640 1 -- 172.21.15.5:0/1929609964 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558cce7beae0 con 0x558cce8924b0 2024-09-10T09:40:59.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:59.968+0000 7f33b7e00640 1 --2- 172.21.15.5:0/4206535572 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f33a40a2c00 0x7f33a40a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:40:59.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:59.968+0000 7f33bf400640 1 --2- 172.21.15.5:0/1929609964 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558cce8924b0 0x558cce89c830 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f33b00cce60 tx=0x7f33b00d5b20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1aa219fea6295daf server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:40:59.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:59.968+0000 7f33bfe00640 1 --2- 172.21.15.5:0/4206535572 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f33a40a2c00 0x7f33a40a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:40:59.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:59.968+0000 7f33b7e00640 1 -- 172.21.15.5:0/4206535572 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f33b00cf810 con 0x7f33c0076720 2024-09-10T09:40:59.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:59.969+0000 7f33abe00640 1 -- 172.21.15.5:0/1929609964 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33b0097ec0 con 0x558cce8924b0 2024-09-10T09:40:59.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:59.969+0000 7f33abe00640 1 -- 172.21.15.5:0/1929609964 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f33b009c320 con 0x558cce8924b0 2024-09-10T09:40:59.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:59.969+0000 7f33abe00640 1 -- 172.21.15.5:0/1929609964 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33b009b580 con 0x558cce8924b0 2024-09-10T09:40:59.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:59.969+0000 7f33bfe00640 1 --2- 172.21.15.5:0/4206535572 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f33a40a2c00 0x7f33a40a50c0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f33b8011630 tx=0x7f33b8009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:40:59.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:59.970+0000 7f33c8c6e740 1 -- 172.21.15.5:0/1929609964 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558cce8924b0 msgr2=0x558cce89c830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:40:59.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:59.970+0000 7f33c8c6e740 1 --2- 172.21.15.5:0/1929609964 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558cce8924b0 0x558cce89c830 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f33b00cce60 tx=0x7f33b00d5b20 comp rx=0 tx=0).stop 2024-09-10T09:40:59.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:59.970+0000 7f33c8c6e740 1 -- 172.21.15.5:0/1929609964 shutdown_connections 2024-09-10T09:40:59.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:59.970+0000 7f33c8c6e740 1 --2- 172.21.15.5:0/1929609964 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558cce89cd70 0x558cce89f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:59.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:59.970+0000 7f33c8c6e740 1 --2- 172.21.15.5:0/1929609964 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558cce8924b0 0x558cce89c830 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:59.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:59.970+0000 7f33c8c6e740 1 --2- 172.21.15.5:0/1929609964 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558cce891b90 0x558cce891f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:59.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:59.970+0000 7f33c8c6e740 1 -- 172.21.15.5:0/1929609964 >> 172.21.15.5:0/1929609964 conn(0x558cce88e120 msgr2=0x558cce88e500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:40:59.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:59.970+0000 7f33c8c6e740 1 -- 172.21.15.5:0/1929609964 shutdown_connections 2024-09-10T09:40:59.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:59.970+0000 7f33c8c6e740 1 -- 172.21.15.5:0/1929609964 wait complete. 2024-09-10T09:40:59.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:59.970+0000 7f33c8c6e740 1 -- start start 2024-09-10T09:40:59.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:59.971+0000 7f33c8c6e740 1 Processor -- start 2024-09-10T09:40:59.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:59.971+0000 7f33c8c6e740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558cce891b90 0x558cce899c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:40:59.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:59.971+0000 7f33c8c6e740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558cce8924b0 0x558cce89a160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:40:59.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:59.971+0000 7f33c8c6e740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558cce89cd70 0x558cce89a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:40:59.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:59.971+0000 7f33c8c6e740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x558cce89f9e0 con 0x558cce8924b0 2024-09-10T09:40:59.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:59.971+0000 7f33c8c6e740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x558cce7b30f0 con 0x558cce89cd70 2024-09-10T09:40:59.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:59.971+0000 7f33c8c6e740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x558cce89fb60 con 0x558cce891b90 2024-09-10T09:40:59.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:59.971+0000 7f33c8c6e740 10 client.0 _ll_register_callbacks cb 0x558cce7a49e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:40:59.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:59.971+0000 7f33bf400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558cce891b90 0x558cce899c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:40:59.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:59.971+0000 7f33bfe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558cce89cd70 0x558cce89a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:40:59.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:59.971+0000 7f33bf400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558cce891b90 0x558cce899c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:57070/0 (socket says 172.21.15.5:57070) 2024-09-10T09:40:59.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:59.971+0000 7f33bf400640 1 -- 172.21.15.5:0/54405502 learned_addr learned my addr 172.21.15.5:0/54405502 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:40:59.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:59.972+0000 7f33bea00640 1 --2- 172.21.15.5:0/54405502 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558cce8924b0 0x558cce89a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:40:59.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:59.972+0000 7f33bf400640 1 -- 172.21.15.5:0/54405502 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558cce89cd70 msgr2=0x558cce89a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:40:59.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:59.972+0000 7f33bf400640 1 --2- 172.21.15.5:0/54405502 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558cce89cd70 0x558cce89a6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:59.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:59.972+0000 7f33bf400640 1 -- 172.21.15.5:0/54405502 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558cce8924b0 msgr2=0x558cce89a160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:40:59.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:59.972+0000 7f33bf400640 1 --2- 172.21.15.5:0/54405502 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558cce8924b0 0x558cce89a160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:59.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:59.972+0000 7f33bf400640 1 -- 172.21.15.5:0/54405502 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558cce5693c0 con 0x558cce891b90 2024-09-10T09:40:59.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:59.972+0000 7f33bfe00640 1 --2- 172.21.15.5:0/54405502 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558cce89cd70 0x558cce89a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:40:59.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:59.972+0000 7f33bf400640 1 --2- 172.21.15.5:0/54405502 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558cce891b90 0x558cce899c20 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f33b009b920 tx=0x7f33b000cf20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6c99c5679478be0f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:40:59.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:59.972+0000 7f33a8c00640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-09-10T09:40:59.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:59.973+0000 7f33a8c00640 1 -- 172.21.15.5:0/54405502 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33b009b760 con 0x558cce891b90 2024-09-10T09:40:59.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:59.973+0000 7f33a8c00640 1 -- 172.21.15.5:0/54405502 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f33b00cf480 con 0x558cce891b90 2024-09-10T09:40:59.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:59.973+0000 7f33c8c6e740 1 -- 172.21.15.5:0/54405502 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f33b009c320 con 0x558cce891b90 2024-09-10T09:40:59.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:59.974+0000 7f33a8c00640 1 -- 172.21.15.5:0/54405502 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f33b009b9f0 con 0x558cce891b90 2024-09-10T09:40:59.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:59.974+0000 7f33c8c6e740 10 client.25990 fetch_fsmap learned FSMap version 32 2024-09-10T09:40:59.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:59.974+0000 7f33c8c6e740 1 -- 172.21.15.5:0/54405502 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x558cce89f9e0 con 0x558cce891b90 2024-09-10T09:40:59.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:59.974+0000 7f33a8c00640 1 -- 172.21.15.5:0/54405502 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f33b009c320 con 0x558cce891b90 2024-09-10T09:40:59.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:59.975+0000 7f33a8c00640 1 -- 172.21.15.5:0/54405502 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7f33b009be80 con 0x558cce891b90 2024-09-10T09:40:59.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:59.975+0000 7f33c8c6e740 10 client.25990 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T09:40:59.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:59.975+0000 7f33c8c6e740 10 client.25990 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T09:40:59.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:59.975+0000 7f33c8c6e740 20 client.25990 populate_metadata read hostname 'smithi005' 2024-09-10T09:40:59.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:59.975+0000 7f33c8c6e740 1 --2- 172.21.15.5:0/54405502 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x558cce8ac7e0 0x558cce8aebd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:40:59.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:59.975+0000 7f33c8c6e740 4 client.25990 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:40:59.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:59.975+0000 7f33c8c6e740 1 -- 172.21.15.5:0/54405502 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558cce805950 con 0x558cce8ac7e0 2024-09-10T09:40:59.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:59.976+0000 7f33bea00640 1 --2- 172.21.15.5:0/54405502 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x558cce8ac7e0 0x558cce8aebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:40:59.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:59.977+0000 7f33bea00640 1 --2- 172.21.15.5:0/54405502 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x558cce8ac7e0 0x558cce8aebd0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e52b35175aabdada server_cookie=d78b81fa31da2f7f in_seq=0 out_seq=0 2024-09-10T09:40:59.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:59.977+0000 7f33a8c00640 10 client.25990 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T09:40:59.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:59.978+0000 7f33a8c00640 1 -- 172.21.15.5:0/54405502 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558cce805950 con 0x558cce8ac7e0 2024-09-10T09:40:59.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:59.978+0000 7f33a8c00640 10 client.25990 handle_command_reply: tid=0 2024-09-10T09:40:59.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:59.978+0000 7f33c8c6e740 1 client.25990 shutdown 2024-09-10T09:40:59.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:59.979+0000 7f33c8c6e740 1 -- 172.21.15.5:0/54405502 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558cce891b90 msgr2=0x558cce899c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:40:59.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:59.979+0000 7f33c8c6e740 1 --2- 172.21.15.5:0/54405502 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558cce891b90 0x558cce899c20 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f33b009b920 tx=0x7f33b000cf20 comp rx=0 tx=0).stop 2024-09-10T09:40:59.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:59.980+0000 7f33c8c6e740 1 -- 172.21.15.5:0/54405502 shutdown_connections 2024-09-10T09:40:59.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:59.980+0000 7f33c8c6e740 1 --2- 172.21.15.5:0/54405502 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x558cce8ac7e0 0x558cce8aebd0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:59.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:59.980+0000 7f33c8c6e740 1 --2- 172.21.15.5:0/54405502 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558cce89cd70 0x558cce89a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:59.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:59.980+0000 7f33c8c6e740 1 --2- 172.21.15.5:0/54405502 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558cce8924b0 0x558cce89a160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:59.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:59.980+0000 7f33c8c6e740 1 --2- 172.21.15.5:0/54405502 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558cce891b90 0x558cce899c20 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:40:59.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:59.980+0000 7f33c8c6e740 1 -- 172.21.15.5:0/54405502 >> 172.21.15.5:0/54405502 conn(0x558cce88e120 msgr2=0x558cce893e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:40:59.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:59.980+0000 7f33a8c00640 0 client.25990 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T09:40:59.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:59.981+0000 7f33c8c6e740 1 -- 172.21.15.5:0/54405502 shutdown_connections 2024-09-10T09:40:59.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:59.981+0000 7f33c8c6e740 1 -- 172.21.15.5:0/54405502 wait complete. 2024-09-10T09:40:59.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:40:59.981+0000 7f33c8c6e740 20 client.25990 trim_cache size 0 max 16384 2024-09-10T09:40:59.992 INFO:tasks.workunit.client.0.smithi005.stdout: 1 76247 10.43 MB/sec warmup 41 sec latency 26.066 ms 2024-09-10T09:41:00.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:00.016+0000 7f33c8c6e740 1 Processor -- start 2024-09-10T09:41:00.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:00.016+0000 7f33c8c6e740 1 -- start start 2024-09-10T09:41:00.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:00.017+0000 7f33c8c6e740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558cce891b90 0x558cce8945f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:41:00.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:00.017+0000 7f33c8c6e740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558cce8924b0 0x558cce8b4260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:41:00.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:00.017+0000 7f33c8c6e740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558cce89cd70 0x558cce8b47a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:41:00.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:00.017+0000 7f33c8c6e740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f33c01b55c0 con 0x558cce891b90 2024-09-10T09:41:00.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:00.017+0000 7f33c8c6e740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f33c01b71c0 con 0x558cce89cd70 2024-09-10T09:41:00.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:00.017+0000 7f33c8c6e740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f33c01b5c90 con 0x558cce8924b0 2024-09-10T09:41:00.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:00.017+0000 7f33bfe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558cce89cd70 0x558cce8b47a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:41:00.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:00.017+0000 7f33bfe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558cce89cd70 0x558cce8b47a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:41200/0 (socket says 172.21.15.5:41200) 2024-09-10T09:41:00.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:00.017+0000 7f33bfe00640 1 -- 172.21.15.5:0/717819159 learned_addr learned my addr 172.21.15.5:0/717819159 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:41:00.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:00.017+0000 7f33bf400640 1 --2- 172.21.15.5:0/717819159 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558cce891b90 0x558cce8945f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:41:00.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:00.018+0000 7f33bea00640 1 --2- 172.21.15.5:0/717819159 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558cce8924b0 0x558cce8b4260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:41:00.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:00.018+0000 7f33bf400640 1 -- 172.21.15.5:0/717819159 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558cce8924b0 msgr2=0x558cce8b4260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:41:00.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:00.018+0000 7f33bf400640 1 --2- 172.21.15.5:0/717819159 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558cce8924b0 0x558cce8b4260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:41:00.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:00.018+0000 7f33bf400640 1 -- 172.21.15.5:0/717819159 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558cce89cd70 msgr2=0x558cce8b47a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:41:00.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:00.018+0000 7f33bf400640 1 --2- 172.21.15.5:0/717819159 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558cce89cd70 0x558cce8b47a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:41:00.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:00.018+0000 7f33bf400640 1 -- 172.21.15.5:0/717819159 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558cce898bd0 con 0x558cce891b90 2024-09-10T09:41:00.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:00.018+0000 7f33bea00640 1 --2- 172.21.15.5:0/717819159 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558cce8924b0 0x558cce8b4260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T09:41:00.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:00.019+0000 7f33bf400640 1 --2- 172.21.15.5:0/717819159 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558cce891b90 0x558cce8945f0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f33b00d5210 tx=0x7f33b009a130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=260e3d644988953d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:41:00.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:00.019+0000 7f33ab400640 1 -- 172.21.15.5:0/717819159 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33b009b760 con 0x558cce891b90 2024-09-10T09:41:00.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:00.019+0000 7f33ab400640 1 -- 172.21.15.5:0/717819159 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f33b00ab070 con 0x558cce891b90 2024-09-10T09:41:00.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:00.019+0000 7f33ab400640 1 -- 172.21.15.5:0/717819159 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33b009b760 con 0x558cce891b90 2024-09-10T09:41:00.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:00.020+0000 7f33c8c6e740 1 -- 172.21.15.5:0/717819159 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558cce891b90 msgr2=0x558cce8945f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:41:00.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:00.020+0000 7f33c8c6e740 1 --2- 172.21.15.5:0/717819159 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558cce891b90 0x558cce8945f0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f33b00d5210 tx=0x7f33b009a130 comp rx=0 tx=0).stop 2024-09-10T09:41:00.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:00.020+0000 7f33c8c6e740 1 -- 172.21.15.5:0/717819159 shutdown_connections 2024-09-10T09:41:00.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:00.020+0000 7f33c8c6e740 1 --2- 172.21.15.5:0/717819159 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558cce89cd70 0x558cce8b47a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:41:00.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:00.020+0000 7f33c8c6e740 1 --2- 172.21.15.5:0/717819159 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558cce8924b0 0x558cce8b4260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:41:00.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:00.020+0000 7f33c8c6e740 1 --2- 172.21.15.5:0/717819159 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558cce891b90 0x558cce8945f0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:41:00.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:00.020+0000 7f33c8c6e740 1 -- 172.21.15.5:0/717819159 >> 172.21.15.5:0/717819159 conn(0x558cce88e120 msgr2=0x558cce89b480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:41:00.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:00.021+0000 7f33c8c6e740 1 -- 172.21.15.5:0/717819159 shutdown_connections 2024-09-10T09:41:00.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:00.021+0000 7f33c8c6e740 1 -- 172.21.15.5:0/717819159 wait complete. 2024-09-10T09:41:00.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:00.021+0000 7f33c8c6e740 1 -- start start 2024-09-10T09:41:00.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:00.021+0000 7f33c8c6e740 1 Processor -- start 2024-09-10T09:41:00.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:00.022+0000 7f33c8c6e740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558cce891b90 0x558cce8a56c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:41:00.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:00.022+0000 7f33c8c6e740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558cce8924b0 0x558cce896ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:41:00.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:00.022+0000 7f33c8c6e740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558cce89cd70 0x558cce8971f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:41:00.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:00.022+0000 7f33bf400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558cce891b90 0x558cce8a56c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:41:00.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:00.022+0000 7f33bf400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558cce891b90 0x558cce8a56c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:41214/0 (socket says 172.21.15.5:41214) 2024-09-10T09:41:00.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:00.022+0000 7f33bf400640 1 -- 172.21.15.5:0/291920724 learned_addr learned my addr 172.21.15.5:0/291920724 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:41:00.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:00.022+0000 7f33c8c6e740 1 -- 172.21.15.5:0/291920724 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f33c01b6a30 con 0x558cce8924b0 2024-09-10T09:41:00.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:00.022+0000 7f33c8c6e740 1 -- 172.21.15.5:0/291920724 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f33c01b6360 con 0x558cce891b90 2024-09-10T09:41:00.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:00.022+0000 7f33c8c6e740 1 -- 172.21.15.5:0/291920724 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f33c01b35a0 con 0x558cce89cd70 2024-09-10T09:41:00.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:00.022+0000 7f33c8c6e740 10 client.0 _ll_register_callbacks cb 0x7f33b00d58e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:41:00.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:00.023+0000 7f33bea00640 1 --2- 172.21.15.5:0/291920724 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558cce8924b0 0x558cce896ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:41:00.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:00.023+0000 7f33bf400640 1 -- 172.21.15.5:0/291920724 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558cce89cd70 msgr2=0x558cce8971f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:41:00.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:00.023+0000 7f33bfe00640 1 --2- 172.21.15.5:0/291920724 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558cce89cd70 0x558cce8971f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:41:00.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:00.023+0000 7f33bf400640 1 --2- 172.21.15.5:0/291920724 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558cce89cd70 0x558cce8971f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:41:00.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:00.023+0000 7f33bf400640 1 -- 172.21.15.5:0/291920724 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558cce8924b0 msgr2=0x558cce896ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:41:00.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:00.023+0000 7f33bf400640 1 --2- 172.21.15.5:0/291920724 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558cce8924b0 0x558cce896ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:41:00.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:00.023+0000 7f33bf400640 1 -- 172.21.15.5:0/291920724 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558cce916230 con 0x558cce891b90 2024-09-10T09:41:00.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:00.023+0000 7f33bea00640 1 --2- 172.21.15.5:0/291920724 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558cce8924b0 0x558cce896ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:41:00.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:00.023+0000 7f33bfe00640 1 --2- 172.21.15.5:0/291920724 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558cce89cd70 0x558cce8971f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T09:41:00.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:00.024+0000 7f33bf400640 1 --2- 172.21.15.5:0/291920724 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558cce891b90 0x558cce8a56c0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f33b00985a0 tx=0x7f33b00d5b20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=757820e4ed9071ef server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:41:00.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:00.024+0000 7f33a8c00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-09-10T09:41:00.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:00.024+0000 7f33a8c00640 1 -- 172.21.15.5:0/291920724 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33b0097ec0 con 0x558cce891b90 2024-09-10T09:41:00.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:00.024+0000 7f33a8c00640 1 -- 172.21.15.5:0/291920724 <== mon.2 v2:172.21.15.102:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f33b009be80 con 0x558cce891b90 2024-09-10T09:41:00.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:00.024+0000 7f33c8c6e740 1 -- 172.21.15.5:0/291920724 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f33b00ab070 con 0x558cce891b90 2024-09-10T09:41:00.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:00.024+0000 7f33a8c00640 1 -- 172.21.15.5:0/291920724 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f33b009b580 con 0x558cce891b90 2024-09-10T09:41:00.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:00.024+0000 7f33c8c6e740 10 client.25964 fetch_fsmap learned FSMap version 32 2024-09-10T09:41:00.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:00.024+0000 7f33c8c6e740 1 -- 172.21.15.5:0/291920724 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x558cce916230 con 0x558cce891b90 2024-09-10T09:41:00.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:00.025+0000 7f33a8c00640 1 -- 172.21.15.5:0/291920724 <== mon.2 v2:172.21.15.102:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f33b00ab070 con 0x558cce891b90 2024-09-10T09:41:00.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:00.025+0000 7f33a8c00640 1 -- 172.21.15.5:0/291920724 <== mon.2 v2:172.21.15.102:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7f33b009b070 con 0x558cce891b90 2024-09-10T09:41:00.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:00.026+0000 7f33c8c6e740 10 client.25964 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T09:41:00.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:00.026+0000 7f33c8c6e740 10 client.25964 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T09:41:00.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:00.026+0000 7f33c8c6e740 20 client.25964 populate_metadata read hostname 'smithi005' 2024-09-10T09:41:00.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:00.026+0000 7f33c8c6e740 1 --2- 172.21.15.5:0/291920724 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x558cce917650 0x558cce919a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:41:00.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:00.026+0000 7f33c8c6e740 4 client.25964 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T09:41:00.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:00.026+0000 7f33c8c6e740 1 -- 172.21.15.5:0/291920724 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "scrub status"}) -- 0x558cce7ffe60 con 0x558cce917650 2024-09-10T09:41:00.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:00.026+0000 7f33bea00640 1 --2- 172.21.15.5:0/291920724 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x558cce917650 0x558cce919a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:41:00.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:00.026+0000 7f33bea00640 1 --2- 172.21.15.5:0/291920724 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x558cce917650 0x558cce919a70 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5dd7ec389e4cda56 server_cookie=111b4998688ca2c8 in_seq=0 out_seq=0 2024-09-10T09:41:00.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:00.026+0000 7f33a8c00640 10 client.25964 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T09:41:00.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:00.027+0000 7f33a8c00640 1 -- 172.21.15.5:0/291920724 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x558cce7ffe60 con 0x558cce917650 2024-09-10T09:41:00.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:00.027+0000 7f33a8c00640 10 client.25964 handle_command_reply: tid=0 2024-09-10T09:41:00.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:00.027+0000 7f33c8c6e740 1 client.25964 shutdown 2024-09-10T09:41:00.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:00.027+0000 7f33c8c6e740 1 -- 172.21.15.5:0/291920724 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558cce891b90 msgr2=0x558cce8a56c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:41:00.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:00.027+0000 7f33c8c6e740 1 --2- 172.21.15.5:0/291920724 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558cce891b90 0x558cce8a56c0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f33b00985a0 tx=0x7f33b00d5b20 comp rx=0 tx=0).stop 2024-09-10T09:41:00.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:00.027+0000 7f33c8c6e740 1 -- 172.21.15.5:0/291920724 shutdown_connections 2024-09-10T09:41:00.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:00.027+0000 7f33c8c6e740 1 --2- 172.21.15.5:0/291920724 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x558cce917650 0x558cce919a70 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:41:00.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:00.027+0000 7f33c8c6e740 1 --2- 172.21.15.5:0/291920724 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558cce89cd70 0x558cce8971f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:41:00.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:00.027+0000 7f33c8c6e740 1 --2- 172.21.15.5:0/291920724 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558cce8924b0 0x558cce896ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:41:00.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:00.027+0000 7f33c8c6e740 1 --2- 172.21.15.5:0/291920724 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558cce891b90 0x558cce8a56c0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:41:00.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:00.027+0000 7f33c8c6e740 1 -- 172.21.15.5:0/291920724 >> 172.21.15.5:0/291920724 conn(0x558cce88e120 msgr2=0x558cce88c9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:41:00.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:00.027+0000 7f33a8c00640 0 client.25964 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T09:41:00.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:00.027+0000 7f33c8c6e740 1 -- 172.21.15.5:0/291920724 shutdown_connections 2024-09-10T09:41:00.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:00.027+0000 7f33c8c6e740 1 -- 172.21.15.5:0/291920724 wait complete. 2024-09-10T09:41:00.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:00.027+0000 7f33c8c6e740 20 client.25964 trim_cache size 0 max 16384 2024-09-10T09:41:00.030 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-10T09:41:00.031 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-10T09:41:00.031 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2024-09-10T09:41:00.031 INFO:teuthology.orchestra.run.smithi005.stdout: "5eeb0cee-4bfb-4c8b-8b78-241139cb3771": { 2024-09-10T09:41:00.031 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2024-09-10T09:41:00.031 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "5eeb0cee-4bfb-4c8b-8b78-241139cb3771", 2024-09-10T09:41:00.031 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2024-09-10T09:41:00.031 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T09:41:00.031 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T09:41:00.031 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-10T09:41:00.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:00.028+0000 7f33c6200640 1 -- 172.21.15.5:0/4206535572 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f33a40a2c00 msgr2=0x7f33a40a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:41:00.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:00.028+0000 7f33c6200640 1 --2- 172.21.15.5:0/4206535572 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f33a40a2c00 0x7f33a40a50c0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f33b8011630 tx=0x7f33b8009500 comp rx=0 tx=0).stop 2024-09-10T09:41:00.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:00.028+0000 7f33c6200640 1 -- 172.21.15.5:0/4206535572 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f33c0076720 msgr2=0x7f33c01a4780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:41:00.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:00.028+0000 7f33c6200640 1 --2- 172.21.15.5:0/4206535572 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f33c0076720 0x7f33c01a4780 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f33b000e9f0 tx=0x7f33b000edf0 comp rx=0 tx=0).stop 2024-09-10T09:41:00.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:00.028+0000 7f33c6200640 1 -- 172.21.15.5:0/4206535572 shutdown_connections 2024-09-10T09:41:00.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:00.028+0000 7f33c6200640 1 --2- 172.21.15.5:0/4206535572 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f33a40a2c00 0x7f33a40a50c0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:41:00.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:00.028+0000 7f33c6200640 1 --2- 172.21.15.5:0/4206535572 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f33c0112a50 0x7f33c019e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:41:00.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:00.028+0000 7f33c6200640 1 --2- 172.21.15.5:0/4206535572 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f33c0077040 0x7f33c01a4cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:41:00.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:00.028+0000 7f33c6200640 1 --2- 172.21.15.5:0/4206535572 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f33c0076720 0x7f33c01a4780 secure :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f33b000e9f0 tx=0x7f33b000edf0 comp rx=0 tx=0).stop 2024-09-10T09:41:00.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:00.028+0000 7f33c6200640 1 -- 172.21.15.5:0/4206535572 >> 172.21.15.5:0/4206535572 conn(0x7f33c00ffa50 msgr2=0x7f33c0101e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:41:00.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:00.028+0000 7f33c6200640 1 -- 172.21.15.5:0/4206535572 shutdown_connections 2024-09-10T09:41:00.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:00.029+0000 7f33c6200640 1 -- 172.21.15.5:0/4206535572 wait complete. 2024-09-10T09:41:00.992 INFO:tasks.workunit.client.0.smithi005.stdout: 1 78679 10.52 MB/sec warmup 42 sec latency 20.396 ms 2024-09-10T09:41:01.060 INFO:tasks.cephfs.filesystem:scrub status for tag:5eeb0cee-4bfb-4c8b-8b78-241139cb3771 - {'path': '/', 'tag': '5eeb0cee-4bfb-4c8b-8b78-241139cb3771', 'options': 'recursive,force'} 2024-09-10T09:41:01.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:41:01 smithi102 ceph-mon[35442]: pgmap v473: 129 pgs: 129 active+clean; 53 MiB data, 461 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 1.5 MiB/s wr, 245 op/s 2024-09-10T09:41:01.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:41:01 smithi005 ceph-mon[30350]: pgmap v473: 129 pgs: 129 active+clean; 53 MiB data, 461 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 1.5 MiB/s wr, 245 op/s 2024-09-10T09:41:01.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:41:01 smithi159 ceph-mon[35513]: pgmap v473: 129 pgs: 129 active+clean; 53 MiB data, 461 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 1.5 MiB/s wr, 245 op/s 2024-09-10T09:41:01.993 INFO:tasks.workunit.client.0.smithi005.stdout: 1 81280 10.58 MB/sec warmup 43 sec latency 41.898 ms 2024-09-10T09:41:02.993 INFO:tasks.workunit.client.0.smithi005.stdout: 1 83166 10.59 MB/sec warmup 44 sec latency 21.314 ms 2024-09-10T09:41:03.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:41:03 smithi102 ceph-mon[35442]: pgmap v474: 129 pgs: 129 active+clean; 54 MiB data, 468 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 1.5 MiB/s wr, 243 op/s 2024-09-10T09:41:03.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:41:03 smithi005 ceph-mon[30350]: pgmap v474: 129 pgs: 129 active+clean; 54 MiB data, 468 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 1.5 MiB/s wr, 243 op/s 2024-09-10T09:41:03.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:41:03 smithi159 ceph-mon[35513]: pgmap v474: 129 pgs: 129 active+clean; 54 MiB data, 468 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 1.5 MiB/s wr, 243 op/s 2024-09-10T09:41:03.993 INFO:tasks.workunit.client.0.smithi005.stdout: 1 85655 10.66 MB/sec warmup 45 sec latency 16.748 ms 2024-09-10T09:41:04.993 INFO:tasks.workunit.client.0.smithi005.stdout: 1 88148 10.70 MB/sec warmup 46 sec latency 26.296 ms 2024-09-10T09:41:05.438 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:41:05 smithi005 ceph-mon[30350]: pgmap v475: 129 pgs: 129 active+clean; 54 MiB data, 473 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 1.3 MiB/s wr, 192 op/s 2024-09-10T09:41:05.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:41:05 smithi102 ceph-mon[35442]: pgmap v475: 129 pgs: 129 active+clean; 54 MiB data, 473 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 1.3 MiB/s wr, 192 op/s 2024-09-10T09:41:05.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:41:05 smithi159 ceph-mon[35513]: pgmap v475: 129 pgs: 129 active+clean; 54 MiB data, 473 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 1.3 MiB/s wr, 192 op/s 2024-09-10T09:41:05.993 INFO:tasks.workunit.client.0.smithi005.stdout: 1 88698 10.58 MB/sec warmup 47 sec latency 532.007 ms 2024-09-10T09:41:06.993 INFO:tasks.workunit.client.0.smithi005.stdout: 1 89515 10.40 MB/sec warmup 48 sec latency 844.612 ms 2024-09-10T09:41:07.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:41:07 smithi005 ceph-mon[30350]: pgmap v476: 129 pgs: 129 active+clean; 59 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 1.6 MiB/s wr, 263 op/s 2024-09-10T09:41:07.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:41:07 smithi102 ceph-mon[35442]: pgmap v476: 129 pgs: 129 active+clean; 59 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 1.6 MiB/s wr, 263 op/s 2024-09-10T09:41:07.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:41:07 smithi159 ceph-mon[35513]: pgmap v476: 129 pgs: 129 active+clean; 59 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 1.6 MiB/s wr, 263 op/s 2024-09-10T09:41:07.993 INFO:tasks.workunit.client.0.smithi005.stdout: 1 92281 10.54 MB/sec warmup 49 sec latency 13.688 ms 2024-09-10T09:41:08.994 INFO:tasks.workunit.client.0.smithi005.stdout: 1 93937 10.47 MB/sec warmup 50 sec latency 24.066 ms 2024-09-10T09:41:09.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:41:09 smithi005 ceph-mon[30350]: pgmap v477: 129 pgs: 129 active+clean; 62 MiB data, 484 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 1.5 MiB/s wr, 199 op/s 2024-09-10T09:41:09.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:41:09 smithi159 ceph-mon[35513]: pgmap v477: 129 pgs: 129 active+clean; 62 MiB data, 484 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 1.5 MiB/s wr, 199 op/s 2024-09-10T09:41:09.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:41:09 smithi102 ceph-mon[35442]: pgmap v477: 129 pgs: 129 active+clean; 62 MiB data, 484 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 1.5 MiB/s wr, 199 op/s 2024-09-10T09:41:09.994 INFO:tasks.workunit.client.0.smithi005.stdout: 1 96408 10.53 MB/sec warmup 51 sec latency 15.005 ms 2024-09-10T09:41:10.994 INFO:tasks.workunit.client.0.smithi005.stdout: 1 96785 10.35 MB/sec warmup 52 sec latency 548.757 ms 2024-09-10T09:41:11.640 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:41:11 smithi005 ceph-mon[30350]: pgmap v478: 129 pgs: 129 active+clean; 64 MiB data, 499 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 1.9 MiB/s wr, 251 op/s 2024-09-10T09:41:11.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:41:11 smithi159 ceph-mon[35513]: pgmap v478: 129 pgs: 129 active+clean; 64 MiB data, 499 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 1.9 MiB/s wr, 251 op/s 2024-09-10T09:41:11.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:41:11 smithi102 ceph-mon[35442]: pgmap v478: 129 pgs: 129 active+clean; 64 MiB data, 499 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 1.9 MiB/s wr, 251 op/s 2024-09-10T09:41:11.994 INFO:tasks.workunit.client.0.smithi005.stdout: 1 96785 10.15 MB/sec warmup 53 sec latency 1548.815 ms 2024-09-10T09:41:12.463 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:41:12 smithi005 ceph-mon[30350]: scrub summary: idle 2024-09-10T09:41:12.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:41:12 smithi159 ceph-mon[35513]: scrub summary: idle 2024-09-10T09:41:12.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:41:12 smithi102 ceph-mon[35442]: scrub summary: idle 2024-09-10T09:41:12.994 INFO:tasks.workunit.client.0.smithi005.stdout: 1 97731 10.05 MB/sec warmup 54 sec latency 2142.129 ms 2024-09-10T09:41:13.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:41:13 smithi005 ceph-mon[30350]: pgmap v479: 129 pgs: 129 active+clean; 64 MiB data, 499 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 1.8 MiB/s wr, 203 op/s 2024-09-10T09:41:13.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:41:13 smithi005 ceph-mon[30350]: scrub summary: idle 2024-09-10T09:41:13.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:41:13 smithi159 ceph-mon[35513]: pgmap v479: 129 pgs: 129 active+clean; 64 MiB data, 499 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 1.8 MiB/s wr, 203 op/s 2024-09-10T09:41:13.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:41:13 smithi159 ceph-mon[35513]: scrub summary: idle 2024-09-10T09:41:13.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:41:13 smithi102 ceph-mon[35442]: pgmap v479: 129 pgs: 129 active+clean; 64 MiB data, 499 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 1.8 MiB/s wr, 203 op/s 2024-09-10T09:41:13.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:41:13 smithi102 ceph-mon[35442]: scrub summary: idle 2024-09-10T09:41:13.994 INFO:tasks.workunit.client.0.smithi005.stdout: 1 100047 10.12 MB/sec warmup 55 sec latency 24.386 ms 2024-09-10T09:41:14.994 INFO:tasks.workunit.client.0.smithi005.stdout: 1 102902 10.25 MB/sec warmup 56 sec latency 12.403 ms 2024-09-10T09:41:15.657 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:41:15 smithi005 ceph-mon[30350]: pgmap v480: 129 pgs: 129 active+clean; 64 MiB data, 500 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 1.9 MiB/s wr, 188 op/s 2024-09-10T09:41:15.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:41:15 smithi159 ceph-mon[35513]: pgmap v480: 129 pgs: 129 active+clean; 64 MiB data, 500 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 1.9 MiB/s wr, 188 op/s 2024-09-10T09:41:15.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:41:15 smithi102 ceph-mon[35442]: pgmap v480: 129 pgs: 129 active+clean; 64 MiB data, 500 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 1.9 MiB/s wr, 188 op/s 2024-09-10T09:41:15.998 INFO:tasks.workunit.client.0.smithi005.stdout: 1 103371 10.10 MB/sec warmup 57 sec latency 462.372 ms 2024-09-10T09:41:16.998 INFO:tasks.workunit.client.0.smithi005.stdout: 1 103371 9.92 MB/sec warmup 58 sec latency 1117.686 ms 2024-09-10T09:41:17.444 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:41:17 smithi005 ceph-mon[30350]: pgmap v481: 129 pgs: 129 active+clean; 66 MiB data, 542 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 2.3 MiB/s wr, 238 op/s 2024-09-10T09:41:17.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:41:17 smithi159 ceph-mon[35513]: pgmap v481: 129 pgs: 129 active+clean; 66 MiB data, 542 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 2.3 MiB/s wr, 238 op/s 2024-09-10T09:41:17.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:41:17 smithi102 ceph-mon[35442]: pgmap v481: 129 pgs: 129 active+clean; 66 MiB data, 542 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 2.3 MiB/s wr, 238 op/s 2024-09-10T09:41:17.998 INFO:tasks.workunit.client.0.smithi005.stdout: 1 104064 9.79 MB/sec warmup 59 sec latency 1742.295 ms 2024-09-10T09:41:18.998 INFO:tasks.workunit.client.0.smithi005.stdout: 1 106810 9.91 MB/sec warmup 60 sec latency 11.627 ms 2024-09-10T09:41:19.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:41:19 smithi005 ceph-mon[30350]: pgmap v482: 129 pgs: 129 active+clean; 68 MiB data, 573 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.2 MiB/s wr, 173 op/s 2024-09-10T09:41:19.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:41:19 smithi159 ceph-mon[35513]: pgmap v482: 129 pgs: 129 active+clean; 68 MiB data, 573 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.2 MiB/s wr, 173 op/s 2024-09-10T09:41:19.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:41:19 smithi102 ceph-mon[35442]: pgmap v482: 129 pgs: 129 active+clean; 68 MiB data, 573 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.2 MiB/s wr, 173 op/s 2024-09-10T09:41:19.998 INFO:tasks.workunit.client.0.smithi005.stdout: 1 108900 9.86 MB/sec warmup 61 sec latency 25.859 ms 2024-09-10T09:41:20.998 INFO:tasks.workunit.client.0.smithi005.stdout: 1 110262 9.89 MB/sec warmup 62 sec latency 467.847 ms 2024-09-10T09:41:21.625 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:41:21 smithi005 ceph-mon[30350]: pgmap v483: 129 pgs: 129 active+clean; 69 MiB data, 587 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.5 MiB/s wr, 210 op/s 2024-09-10T09:41:21.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:41:21 smithi159 ceph-mon[35513]: pgmap v483: 129 pgs: 129 active+clean; 69 MiB data, 587 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.5 MiB/s wr, 210 op/s 2024-09-10T09:41:21.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:41:21 smithi102 ceph-mon[35442]: pgmap v483: 129 pgs: 129 active+clean; 69 MiB data, 587 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.5 MiB/s wr, 210 op/s 2024-09-10T09:41:21.998 INFO:tasks.workunit.client.0.smithi005.stdout: 1 110262 9.74 MB/sec warmup 63 sec latency 1467.891 ms 2024-09-10T09:41:22.998 INFO:tasks.workunit.client.0.smithi005.stdout: 1 111385 9.67 MB/sec warmup 64 sec latency 1818.350 ms 2024-09-10T09:41:23.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:41:23 smithi005 ceph-mon[30350]: pgmap v484: 129 pgs: 129 active+clean; 66 MiB data, 585 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.4 MiB/s wr, 173 op/s 2024-09-10T09:41:23.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:41:23 smithi159 ceph-mon[35513]: pgmap v484: 129 pgs: 129 active+clean; 66 MiB data, 585 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.4 MiB/s wr, 173 op/s 2024-09-10T09:41:23.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:41:23 smithi102 ceph-mon[35442]: pgmap v484: 129 pgs: 129 active+clean; 66 MiB data, 585 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.4 MiB/s wr, 173 op/s 2024-09-10T09:41:23.999 INFO:tasks.workunit.client.0.smithi005.stdout: 1 113800 9.73 MB/sec warmup 65 sec latency 10.866 ms 2024-09-10T09:41:24.999 INFO:tasks.workunit.client.0.smithi005.stdout: 1 115961 9.70 MB/sec warmup 66 sec latency 24.453 ms 2024-09-10T09:41:25.684 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:41:25 smithi005 ceph-mon[30350]: pgmap v485: 129 pgs: 129 active+clean; 69 MiB data, 626 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.5 MiB/s wr, 194 op/s 2024-09-10T09:41:25.685 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:41:25 smithi102 ceph-mon[35442]: pgmap v485: 129 pgs: 129 active+clean; 69 MiB data, 626 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.5 MiB/s wr, 194 op/s 2024-09-10T09:41:25.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:41:25 smithi159 ceph-mon[35513]: pgmap v485: 129 pgs: 129 active+clean; 69 MiB data, 626 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.5 MiB/s wr, 194 op/s 2024-09-10T09:41:25.999 INFO:tasks.workunit.client.0.smithi005.stdout: 1 117482 9.73 MB/sec warmup 67 sec latency 299.152 ms 2024-09-10T09:41:26.999 INFO:tasks.workunit.client.0.smithi005.stdout: 1 117482 9.59 MB/sec warmup 68 sec latency 1299.216 ms 2024-09-10T09:41:27.645 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:41:27 smithi005 ceph-mon[30350]: pgmap v486: 129 pgs: 129 active+clean; 69 MiB data, 625 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.7 MiB/s wr, 216 op/s 2024-09-10T09:41:27.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:41:27 smithi159 ceph-mon[35513]: pgmap v486: 129 pgs: 129 active+clean; 69 MiB data, 625 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.7 MiB/s wr, 216 op/s 2024-09-10T09:41:27.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:41:27 smithi102 ceph-mon[35442]: pgmap v486: 129 pgs: 129 active+clean; 69 MiB data, 625 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.7 MiB/s wr, 216 op/s 2024-09-10T09:41:27.999 INFO:tasks.workunit.client.0.smithi005.stdout: 1 117775 9.45 MB/sec warmup 69 sec latency 1720.162 ms 2024-09-10T09:41:28.999 INFO:tasks.workunit.client.0.smithi005.stdout: 1 120598 9.56 MB/sec warmup 70 sec latency 12.486 ms 2024-09-10T09:41:29.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:41:29 smithi005 ceph-mon[30350]: pgmap v487: 129 pgs: 129 active+clean; 68 MiB data, 598 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.4 MiB/s wr, 172 op/s 2024-09-10T09:41:29.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:41:29 smithi159 ceph-mon[35513]: pgmap v487: 129 pgs: 129 active+clean; 68 MiB data, 598 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.4 MiB/s wr, 172 op/s 2024-09-10T09:41:29.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:41:29 smithi102 ceph-mon[35442]: pgmap v487: 129 pgs: 129 active+clean; 68 MiB data, 598 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.4 MiB/s wr, 172 op/s 2024-09-10T09:41:29.999 INFO:tasks.workunit.client.0.smithi005.stdout: 1 122206 9.52 MB/sec warmup 71 sec latency 23.006 ms 2024-09-10T09:41:31.000 INFO:tasks.workunit.client.0.smithi005.stdout: 1 124406 9.58 MB/sec warmup 72 sec latency 248.474 ms 2024-09-10T09:41:31.061 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T09:41:31.500 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:41:31.639 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:41:31 smithi005 ceph-mon[30350]: pgmap v488: 129 pgs: 129 active+clean; 70 MiB data, 604 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.7 MiB/s wr, 221 op/s 2024-09-10T09:41:31.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:41:31 smithi159 ceph-mon[35513]: pgmap v488: 129 pgs: 129 active+clean; 70 MiB data, 604 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.7 MiB/s wr, 221 op/s 2024-09-10T09:41:31.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:41:31 smithi102 ceph-mon[35442]: pgmap v488: 129 pgs: 129 active+clean; 70 MiB data, 604 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.7 MiB/s wr, 221 op/s 2024-09-10T09:41:32.000 INFO:tasks.workunit.client.0.smithi005.stdout: 1 124409 9.45 MB/sec warmup 73 sec latency 952.644 ms 2024-09-10T09:41:32.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:32.408+0000 7f5629000640 1 -- 172.21.15.5:0/4229876286 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5624104010 msgr2=0x7f56241043f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:41:32.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:32.408+0000 7f5629000640 1 --2- 172.21.15.5:0/4229876286 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5624104010 0x7f56241043f0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f5614009860 tx=0x7f561402a340 comp rx=0 tx=0).stop 2024-09-10T09:41:32.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:32.408+0000 7f5629000640 1 -- 172.21.15.5:0/4229876286 shutdown_connections 2024-09-10T09:41:32.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:32.408+0000 7f5629000640 1 --2- 172.21.15.5:0/4229876286 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5624109140 0x7f5624110cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:41:32.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:32.408+0000 7f5629000640 1 --2- 172.21.15.5:0/4229876286 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f56241049c0 0x7f5624108a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:41:32.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:32.408+0000 7f5629000640 1 --2- 172.21.15.5:0/4229876286 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5624104010 0x7f56241043f0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:41:32.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:32.408+0000 7f5629000640 1 -- 172.21.15.5:0/4229876286 >> 172.21.15.5:0/4229876286 conn(0x7f56240ffa50 msgr2=0x7f5624101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:41:32.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:32.408+0000 7f5629000640 1 -- 172.21.15.5:0/4229876286 shutdown_connections 2024-09-10T09:41:32.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:32.408+0000 7f5629000640 1 -- 172.21.15.5:0/4229876286 wait complete. 2024-09-10T09:41:32.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:32.408+0000 7f5629000640 1 Processor -- start 2024-09-10T09:41:32.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:32.408+0000 7f5629000640 1 -- start start 2024-09-10T09:41:32.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:32.408+0000 7f5629000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5624104010 0x7f562419a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:41:32.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:32.408+0000 7f5629000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f56241049c0 0x7f562419ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:41:32.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:32.408+0000 7f5629000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5624109140 0x7f562419ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:41:32.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:32.408+0000 7f5629000640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f5624114140 con 0x7f5624109140 2024-09-10T09:41:32.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:32.408+0000 7f5629000640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f5624113fc0 con 0x7f56241049c0 2024-09-10T09:41:32.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:32.408+0000 7f5629000640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f56241142c0 con 0x7f5624104010 2024-09-10T09:41:32.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:32.408+0000 7f5621e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5624104010 0x7f562419a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:41:32.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:32.408+0000 7f5621e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5624104010 0x7f562419a640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:42064/0 (socket says 172.21.15.5:42064) 2024-09-10T09:41:32.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:32.408+0000 7f5621e00640 1 -- 172.21.15.5:0/1275778555 learned_addr learned my addr 172.21.15.5:0/1275778555 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:41:32.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:32.408+0000 7f5621400640 1 --2- 172.21.15.5:0/1275778555 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f56241049c0 0x7f562419ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:41:32.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:32.408+0000 7f5622800640 1 --2- 172.21.15.5:0/1275778555 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5624109140 0x7f562419ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:41:32.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:32.408+0000 7f5621e00640 1 -- 172.21.15.5:0/1275778555 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f56241049c0 msgr2=0x7f562419ab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:41:32.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:32.408+0000 7f5621e00640 1 --2- 172.21.15.5:0/1275778555 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f56241049c0 0x7f562419ab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:41:32.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:32.413+0000 7f5621e00640 1 -- 172.21.15.5:0/1275778555 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5624109140 msgr2=0x7f562419ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:41:32.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:32.413+0000 7f5621e00640 1 --2- 172.21.15.5:0/1275778555 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5624109140 0x7f562419ef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:41:32.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:32.413+0000 7f5621e00640 1 -- 172.21.15.5:0/1275778555 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f562419f5f0 con 0x7f5624104010 2024-09-10T09:41:32.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:32.413+0000 7f5621e00640 1 --2- 172.21.15.5:0/1275778555 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5624104010 0x7f562419a640 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f5614035040 tx=0x7f5614033b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:41:32.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:32.413+0000 7f561aa00640 1 -- 172.21.15.5:0/1275778555 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5614037040 con 0x7f5624104010 2024-09-10T09:41:32.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:32.413+0000 7f561aa00640 1 -- 172.21.15.5:0/1275778555 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5614040070 con 0x7f5624104010 2024-09-10T09:41:32.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:32.413+0000 7f5629000640 1 -- 172.21.15.5:0/1275778555 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f562419f880 con 0x7f5624104010 2024-09-10T09:41:32.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:32.413+0000 7f561aa00640 1 -- 172.21.15.5:0/1275778555 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56140039b0 con 0x7f5624104010 2024-09-10T09:41:32.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:32.413+0000 7f5629000640 1 -- 172.21.15.5:0/1275778555 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56241a71c0 con 0x7f5624104010 2024-09-10T09:41:32.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:32.416+0000 7f562bac0740 1 Processor -- start 2024-09-10T09:41:32.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:32.416+0000 7f562bac0740 1 -- start start 2024-09-10T09:41:32.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:32.416+0000 7f562bac0740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5576291bfb90 0x5576291bff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:41:32.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:32.416+0000 7f562bac0740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5576291c04b0 0x5576291ca830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:41:32.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:32.416+0000 7f562bac0740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5576291cad70 0x5576291cd160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:41:32.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:32.416+0000 7f562bac0740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5576291263e0 con 0x5576291cad70 2024-09-10T09:41:32.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:32.416+0000 7f562bac0740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x5576290cdd80 con 0x5576291bfb90 2024-09-10T09:41:32.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:32.416+0000 7f562bac0740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5576291223a0 con 0x5576291c04b0 2024-09-10T09:41:32.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:32.416+0000 7f5621400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5576291cad70 0x5576291cd160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:41:32.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:32.416+0000 7f5621400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5576291cad70 0x5576291cd160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58386/0 (socket says 172.21.15.5:58386) 2024-09-10T09:41:32.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:32.416+0000 7f5621400640 1 -- 172.21.15.5:0/2140560938 learned_addr learned my addr 172.21.15.5:0/2140560938 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:41:32.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:32.416+0000 7f5622800640 1 --2- 172.21.15.5:0/2140560938 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5576291bfb90 0x5576291bff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:41:32.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:32.416+0000 7f561aa00640 1 -- 172.21.15.5:0/1275778555 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f56140305e0 con 0x7f5624104010 2024-09-10T09:41:32.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:32.416+0000 7f5621400640 1 -- 172.21.15.5:0/2140560938 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5576291c04b0 msgr2=0x5576291ca830 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T09:41:32.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:32.416+0000 7f5621400640 1 --2- 172.21.15.5:0/2140560938 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5576291c04b0 0x5576291ca830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:41:32.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:32.416+0000 7f5621400640 1 -- 172.21.15.5:0/2140560938 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5576291bfb90 msgr2=0x5576291bff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:41:32.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:32.416+0000 7f5621400640 1 --2- 172.21.15.5:0/2140560938 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5576291bfb90 0x5576291bff70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:41:32.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:32.416+0000 7f5621400640 1 -- 172.21.15.5:0/2140560938 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5576290ecae0 con 0x5576291cad70 2024-09-10T09:41:32.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:32.416+0000 7f5621400640 1 --2- 172.21.15.5:0/2140560938 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5576291cad70 0x5576291cd160 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f5610009c90 tx=0x7f5610011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64958e161af203c7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:41:32.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:32.416+0000 7f560f400640 1 -- 172.21.15.5:0/2140560938 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f561001e070 con 0x5576291cad70 2024-09-10T09:41:32.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:32.416+0000 7f560f400640 1 -- 172.21.15.5:0/2140560938 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5610019b10 con 0x5576291cad70 2024-09-10T09:41:32.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:32.416+0000 7f560f400640 1 -- 172.21.15.5:0/2140560938 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f561000ea80 con 0x5576291cad70 2024-09-10T09:41:32.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:32.416+0000 7f561aa00640 1 --2- 172.21.15.5:0/1275778555 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f56080a2b80 0x7f56080a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:41:32.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:32.416+0000 7f562bac0740 1 -- 172.21.15.5:0/2140560938 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5576291cad70 msgr2=0x5576291cd160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:41:32.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:32.416+0000 7f562bac0740 1 --2- 172.21.15.5:0/2140560938 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5576291cad70 0x5576291cd160 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f5610009c90 tx=0x7f5610011810 comp rx=0 tx=0).stop 2024-09-10T09:41:32.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:32.416+0000 7f561aa00640 1 -- 172.21.15.5:0/1275778555 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f56140ef450 con 0x7f5624104010 2024-09-10T09:41:32.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:32.416+0000 7f562bac0740 1 -- 172.21.15.5:0/2140560938 shutdown_connections 2024-09-10T09:41:32.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:32.416+0000 7f562bac0740 1 --2- 172.21.15.5:0/2140560938 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5576291cad70 0x5576291cd160 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:41:32.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:32.416+0000 7f562bac0740 1 --2- 172.21.15.5:0/2140560938 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5576291c04b0 0x5576291ca830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:41:32.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:32.416+0000 7f562bac0740 1 --2- 172.21.15.5:0/2140560938 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5576291bfb90 0x5576291bff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:41:32.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:32.416+0000 7f562bac0740 1 -- 172.21.15.5:0/2140560938 >> 172.21.15.5:0/2140560938 conn(0x5576291bc120 msgr2=0x5576291bc500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:41:32.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:32.416+0000 7f562bac0740 1 -- 172.21.15.5:0/2140560938 shutdown_connections 2024-09-10T09:41:32.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:32.416+0000 7f562bac0740 1 -- 172.21.15.5:0/2140560938 wait complete. 2024-09-10T09:41:32.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:32.416+0000 7f562bac0740 1 -- start start 2024-09-10T09:41:32.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:32.416+0000 7f5622800640 1 --2- 172.21.15.5:0/1275778555 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f56080a2b80 0x7f56080a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:41:32.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:32.416+0000 7f562bac0740 1 Processor -- start 2024-09-10T09:41:32.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:32.421+0000 7f5622800640 1 --2- 172.21.15.5:0/1275778555 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f56080a2b80 0x7f56080a5040 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f561c009a60 tx=0x7f561c010040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:41:32.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:32.421+0000 7f562bac0740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5576291bfb90 0x5576291c7d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:41:32.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:32.421+0000 7f562bac0740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5576291c04b0 0x5576291c8290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:41:32.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:32.421+0000 7f562bac0740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5576291cad70 0x5576291c87d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:41:32.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:32.421+0000 7f562bac0740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5576291cd9e0 con 0x5576291bfb90 2024-09-10T09:41:32.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:32.421+0000 7f562bac0740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x5576290e10f0 con 0x5576291c04b0 2024-09-10T09:41:32.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:32.421+0000 7f562bac0740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5576291cdb60 con 0x5576291cad70 2024-09-10T09:41:32.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:32.421+0000 7f562bac0740 10 client.0 _ll_register_callbacks cb 0x5576290d29e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:41:32.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:32.421+0000 7f5621e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5576291bfb90 0x5576291c7d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:41:32.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:32.421+0000 7f5621e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5576291bfb90 0x5576291c7d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58394/0 (socket says 172.21.15.5:58394) 2024-09-10T09:41:32.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:32.421+0000 7f5621e00640 1 -- 172.21.15.5:0/3654932587 learned_addr learned my addr 172.21.15.5:0/3654932587 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:41:32.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:32.421+0000 7f5622800640 1 --2- 172.21.15.5:0/3654932587 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5576291cad70 0x5576291c87d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:41:32.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:32.421+0000 7f5621400640 1 --2- 172.21.15.5:0/3654932587 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5576291c04b0 0x5576291c8290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:41:32.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:32.421+0000 7f5621e00640 1 -- 172.21.15.5:0/3654932587 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5576291cad70 msgr2=0x5576291c87d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:41:32.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:32.421+0000 7f5621e00640 1 --2- 172.21.15.5:0/3654932587 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5576291cad70 0x5576291c87d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:41:32.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:32.421+0000 7f5621e00640 1 -- 172.21.15.5:0/3654932587 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5576291c04b0 msgr2=0x5576291c8290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:41:32.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:32.421+0000 7f5621e00640 1 --2- 172.21.15.5:0/3654932587 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5576291c04b0 0x5576291c8290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:41:32.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:32.421+0000 7f5621e00640 1 -- 172.21.15.5:0/3654932587 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557628e973c0 con 0x5576291bfb90 2024-09-10T09:41:32.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:32.421+0000 7f5621400640 1 --2- 172.21.15.5:0/3654932587 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5576291c04b0 0x5576291c8290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T09:41:32.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:32.421+0000 7f5621e00640 1 --2- 172.21.15.5:0/3654932587 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5576291bfb90 0x5576291c7d50 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f5614000c00 tx=0x7f5614033f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa3735537441e06d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:41:32.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:32.421+0000 7f5603e00640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-10T09:41:32.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:32.421+0000 7f5603e00640 1 -- 172.21.15.5:0/3654932587 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56140bb860 con 0x5576291bfb90 2024-09-10T09:41:32.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:32.421+0000 7f5603e00640 1 -- 172.21.15.5:0/3654932587 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f56140ecc10 con 0x5576291bfb90 2024-09-10T09:41:32.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:32.421+0000 7f562bac0740 1 -- 172.21.15.5:0/3654932587 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5610019b10 con 0x5576291bfb90 2024-09-10T09:41:32.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:32.421+0000 7f5603e00640 1 -- 172.21.15.5:0/3654932587 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f56140bfc10 con 0x5576291bfb90 2024-09-10T09:41:32.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:32.421+0000 7f562bac0740 10 client.15972 fetch_fsmap learned FSMap version 32 2024-09-10T09:41:32.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:32.421+0000 7f562bac0740 1 -- 172.21.15.5:0/3654932587 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5576291e1f80 con 0x5576291bfb90 2024-09-10T09:41:32.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:32.421+0000 7f5603e00640 1 -- 172.21.15.5:0/3654932587 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5610019b10 con 0x5576291bfb90 2024-09-10T09:41:32.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:32.421+0000 7f5603e00640 1 -- 172.21.15.5:0/3654932587 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7f561403a070 con 0x5576291bfb90 2024-09-10T09:41:32.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:32.421+0000 7f562bac0740 10 client.15972 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T09:41:32.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:32.421+0000 7f562bac0740 10 client.15972 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T09:41:32.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:32.421+0000 7f562bac0740 20 client.15972 populate_metadata read hostname 'smithi005' 2024-09-10T09:41:32.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:32.421+0000 7f562bac0740 1 --2- 172.21.15.5:0/3654932587 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x5576291e3410 0x5576291e5850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:41:32.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:32.421+0000 7f562bac0740 4 client.15972 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:41:32.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:32.421+0000 7f562bac0740 1 -- 172.21.15.5:0/3654932587 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557629133950 con 0x5576291e3410 2024-09-10T09:41:32.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:32.421+0000 7f5621400640 1 --2- 172.21.15.5:0/3654932587 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x5576291e3410 0x5576291e5850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:41:32.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:32.421+0000 7f5621400640 1 --2- 172.21.15.5:0/3654932587 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x5576291e3410 0x5576291e5850 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cdc1e8f5b4465b02 server_cookie=9ee031fba664d55b in_seq=0 out_seq=0 2024-09-10T09:41:32.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:32.421+0000 7f5603e00640 10 client.15972 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T09:41:32.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:32.421+0000 7f5603e00640 1 -- 172.21.15.5:0/3654932587 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557629133950 con 0x5576291e3410 2024-09-10T09:41:32.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:32.421+0000 7f5603e00640 10 client.15972 handle_command_reply: tid=0 2024-09-10T09:41:32.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:32.426+0000 7f562bac0740 1 client.15972 shutdown 2024-09-10T09:41:32.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:32.426+0000 7f562bac0740 1 -- 172.21.15.5:0/3654932587 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5576291bfb90 msgr2=0x5576291c7d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:41:32.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:32.426+0000 7f562bac0740 1 --2- 172.21.15.5:0/3654932587 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5576291bfb90 0x5576291c7d50 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f5614000c00 tx=0x7f5614033f50 comp rx=0 tx=0).stop 2024-09-10T09:41:32.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:32.426+0000 7f562bac0740 1 -- 172.21.15.5:0/3654932587 shutdown_connections 2024-09-10T09:41:32.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:32.426+0000 7f562bac0740 1 --2- 172.21.15.5:0/3654932587 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x5576291e3410 0x5576291e5850 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:41:32.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:32.426+0000 7f562bac0740 1 --2- 172.21.15.5:0/3654932587 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5576291cad70 0x5576291c87d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:41:32.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:32.426+0000 7f562bac0740 1 --2- 172.21.15.5:0/3654932587 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5576291c04b0 0x5576291c8290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:41:32.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:32.426+0000 7f562bac0740 1 --2- 172.21.15.5:0/3654932587 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5576291bfb90 0x5576291c7d50 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:41:32.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:32.426+0000 7f5603e00640 0 client.15972 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T09:41:32.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:32.426+0000 7f562bac0740 1 -- 172.21.15.5:0/3654932587 >> 172.21.15.5:0/3654932587 conn(0x5576291bc120 msgr2=0x5576291c1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:41:32.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:32.426+0000 7f562bac0740 1 -- 172.21.15.5:0/3654932587 shutdown_connections 2024-09-10T09:41:32.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:32.426+0000 7f562bac0740 1 -- 172.21.15.5:0/3654932587 wait complete. 2024-09-10T09:41:32.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:32.426+0000 7f562bac0740 20 client.15972 trim_cache size 0 max 16384 2024-09-10T09:41:32.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:32.445+0000 7f562bac0740 1 Processor -- start 2024-09-10T09:41:32.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:32.445+0000 7f562bac0740 1 -- start start 2024-09-10T09:41:32.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:32.445+0000 7f562bac0740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5576291bfb90 0x5576291eb160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:41:32.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:32.450+0000 7f562bac0740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5576291c04b0 0x5576291eb6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:41:32.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:32.450+0000 7f562bac0740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5576291cad70 0x5576291c4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:41:32.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:32.450+0000 7f562bac0740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f56241acc30 con 0x5576291cad70 2024-09-10T09:41:32.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:32.450+0000 7f562bac0740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f56241ae830 con 0x5576291bfb90 2024-09-10T09:41:32.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:32.450+0000 7f562bac0740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f56241ad300 con 0x5576291c04b0 2024-09-10T09:41:32.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:32.450+0000 7f5621400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5576291c04b0 0x5576291eb6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:41:32.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:32.450+0000 7f5621e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5576291bfb90 0x5576291eb160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:41:32.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:32.450+0000 7f5621400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5576291c04b0 0x5576291eb6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:42084/0 (socket says 172.21.15.5:42084) 2024-09-10T09:41:32.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:32.450+0000 7f5621400640 1 -- 172.21.15.5:0/1006905686 learned_addr learned my addr 172.21.15.5:0/1006905686 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:41:32.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:32.450+0000 7f5622800640 1 --2- 172.21.15.5:0/1006905686 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5576291cad70 0x5576291c4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:41:32.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:32.450+0000 7f5621400640 1 -- 172.21.15.5:0/1006905686 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5576291bfb90 msgr2=0x5576291eb160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:41:32.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:32.450+0000 7f5621400640 1 --2- 172.21.15.5:0/1006905686 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5576291bfb90 0x5576291eb160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:41:32.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:32.450+0000 7f5621400640 1 -- 172.21.15.5:0/1006905686 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5576291cad70 msgr2=0x5576291c4fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:41:32.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:32.450+0000 7f5621400640 1 --2- 172.21.15.5:0/1006905686 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5576291cad70 0x5576291c4fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:41:32.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:32.450+0000 7f5621400640 1 -- 172.21.15.5:0/1006905686 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5576291ca6c0 con 0x5576291c04b0 2024-09-10T09:41:32.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:32.450+0000 7f5621e00640 1 --2- 172.21.15.5:0/1006905686 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5576291bfb90 0x5576291eb160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:41:32.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:32.450+0000 7f5622800640 1 --2- 172.21.15.5:0/1006905686 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5576291cad70 0x5576291c4fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:41:32.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:32.450+0000 7f5621400640 1 --2- 172.21.15.5:0/1006905686 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5576291c04b0 0x5576291eb6a0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f561001c040 tx=0x7f561000e6e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e8cfc46a9d95073a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:41:32.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:32.452+0000 7f560ea00640 1 -- 172.21.15.5:0/1006905686 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f561001e040 con 0x5576291c04b0 2024-09-10T09:41:32.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:32.452+0000 7f560ea00640 1 -- 172.21.15.5:0/1006905686 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5610005020 con 0x5576291c04b0 2024-09-10T09:41:32.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:32.452+0000 7f560ea00640 1 -- 172.21.15.5:0/1006905686 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f561000eb20 con 0x5576291c04b0 2024-09-10T09:41:32.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:32.452+0000 7f562bac0740 1 -- 172.21.15.5:0/1006905686 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5576291c04b0 msgr2=0x5576291eb6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:41:32.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:32.452+0000 7f562bac0740 1 --2- 172.21.15.5:0/1006905686 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5576291c04b0 0x5576291eb6a0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f561001c040 tx=0x7f561000e6e0 comp rx=0 tx=0).stop 2024-09-10T09:41:32.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:32.452+0000 7f562bac0740 1 -- 172.21.15.5:0/1006905686 shutdown_connections 2024-09-10T09:41:32.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:32.452+0000 7f562bac0740 1 --2- 172.21.15.5:0/1006905686 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5576291cad70 0x5576291c4fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:41:32.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:32.452+0000 7f562bac0740 1 --2- 172.21.15.5:0/1006905686 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5576291c04b0 0x5576291eb6a0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:41:32.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:32.452+0000 7f562bac0740 1 --2- 172.21.15.5:0/1006905686 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5576291bfb90 0x5576291eb160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:41:32.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:32.452+0000 7f562bac0740 1 -- 172.21.15.5:0/1006905686 >> 172.21.15.5:0/1006905686 conn(0x5576291bc120 msgr2=0x5576291c8530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:41:32.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:32.452+0000 7f562bac0740 1 -- 172.21.15.5:0/1006905686 shutdown_connections 2024-09-10T09:41:32.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:32.452+0000 7f562bac0740 1 -- 172.21.15.5:0/1006905686 wait complete. 2024-09-10T09:41:32.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:32.452+0000 7f562bac0740 1 -- start start 2024-09-10T09:41:32.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:32.452+0000 7f562bac0740 1 Processor -- start 2024-09-10T09:41:32.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:32.452+0000 7f562bac0740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5576291bfb90 0x5576291db820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:41:32.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:32.452+0000 7f562bac0740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5576291c04b0 0x5576291dbd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:41:32.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:32.452+0000 7f562bac0740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5576291cad70 0x5576291d8610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:41:32.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:32.452+0000 7f562bac0740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f56241ae0a0 con 0x5576291c04b0 2024-09-10T09:41:32.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:32.452+0000 7f562bac0740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f56241ad9d0 con 0x5576291bfb90 2024-09-10T09:41:32.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:32.452+0000 7f562bac0740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f56241aac30 con 0x5576291cad70 2024-09-10T09:41:32.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:32.452+0000 7f562bac0740 10 client.0 _ll_register_callbacks cb 0x7f56140bb630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:41:32.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:32.452+0000 7f5621e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5576291bfb90 0x5576291db820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:41:32.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:32.452+0000 7f5622800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5576291cad70 0x5576291d8610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:41:32.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:32.452+0000 7f5621e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5576291bfb90 0x5576291db820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:35820/0 (socket says 172.21.15.5:35820) 2024-09-10T09:41:32.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:32.452+0000 7f5621e00640 1 -- 172.21.15.5:0/1576226662 learned_addr learned my addr 172.21.15.5:0/1576226662 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:41:32.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:32.452+0000 7f5622800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5576291cad70 0x5576291d8610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:42090/0 (socket says 172.21.15.5:42090) 2024-09-10T09:41:32.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:32.452+0000 7f5621400640 1 --2- 172.21.15.5:0/1576226662 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5576291c04b0 0x5576291dbd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:41:32.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:32.452+0000 7f5621e00640 1 -- 172.21.15.5:0/1576226662 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5576291cad70 msgr2=0x5576291d8610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:41:32.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:32.452+0000 7f5621e00640 1 --2- 172.21.15.5:0/1576226662 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5576291cad70 0x5576291d8610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:41:32.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:32.452+0000 7f5621e00640 1 -- 172.21.15.5:0/1576226662 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5576291c04b0 msgr2=0x5576291dbd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:41:32.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:32.452+0000 7f5621e00640 1 --2- 172.21.15.5:0/1576226662 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5576291c04b0 0x5576291dbd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:41:32.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:32.452+0000 7f5621e00640 1 -- 172.21.15.5:0/1576226662 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5576291d8df0 con 0x5576291bfb90 2024-09-10T09:41:32.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:32.452+0000 7f5621e00640 1 --2- 172.21.15.5:0/1576226662 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5576291bfb90 0x5576291db820 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f5614009830 tx=0x7f561402a340 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2ebfcec7aa428f6b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:41:32.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:32.452+0000 7f5603e00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-09-10T09:41:32.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:32.452+0000 7f5603e00640 1 -- 172.21.15.5:0/1576226662 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56140a62b0 con 0x5576291bfb90 2024-09-10T09:41:32.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:32.452+0000 7f562bac0740 1 -- 172.21.15.5:0/1576226662 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5610005020 con 0x5576291bfb90 2024-09-10T09:41:32.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:32.452+0000 7f5603e00640 1 -- 172.21.15.5:0/1576226662 <== mon.2 v2:172.21.15.102:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f56140bacb0 con 0x5576291bfb90 2024-09-10T09:41:32.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:32.452+0000 7f5603e00640 1 -- 172.21.15.5:0/1576226662 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f56140a2050 con 0x5576291bfb90 2024-09-10T09:41:32.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:32.452+0000 7f562bac0740 10 client.25979 fetch_fsmap learned FSMap version 32 2024-09-10T09:41:32.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:32.452+0000 7f562bac0740 1 -- 172.21.15.5:0/1576226662 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55762924ec30 con 0x5576291bfb90 2024-09-10T09:41:32.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:32.452+0000 7f5603e00640 1 -- 172.21.15.5:0/1576226662 <== mon.2 v2:172.21.15.102:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5610005020 con 0x5576291bfb90 2024-09-10T09:41:32.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:32.452+0000 7f5603e00640 1 -- 172.21.15.5:0/1576226662 <== mon.2 v2:172.21.15.102:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7f561403a070 con 0x5576291bfb90 2024-09-10T09:41:32.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:32.452+0000 7f562bac0740 10 client.25979 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T09:41:32.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:32.452+0000 7f562bac0740 10 client.25979 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T09:41:32.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:32.452+0000 7f562bac0740 20 client.25979 populate_metadata read hostname 'smithi005' 2024-09-10T09:41:32.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:32.452+0000 7f562bac0740 1 --2- 172.21.15.5:0/1576226662 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55762924f2f0 0x557629251730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:41:32.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:32.452+0000 7f562bac0740 4 client.25979 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T09:41:32.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:32.452+0000 7f562bac0740 1 -- 172.21.15.5:0/1576226662 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55762912de60 con 0x55762924f2f0 2024-09-10T09:41:32.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:32.452+0000 7f5621400640 1 --2- 172.21.15.5:0/1576226662 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55762924f2f0 0x557629251730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:41:32.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:32.452+0000 7f5621400640 1 --2- 172.21.15.5:0/1576226662 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55762924f2f0 0x557629251730 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5d32c220ea0466bc server_cookie=76235864bc74fd9f in_seq=0 out_seq=0 2024-09-10T09:41:32.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:32.452+0000 7f5603e00640 10 client.25979 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T09:41:32.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:32.457+0000 7f5603e00640 1 -- 172.21.15.5:0/1576226662 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55762912de60 con 0x55762924f2f0 2024-09-10T09:41:32.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:32.457+0000 7f5603e00640 10 client.25979 handle_command_reply: tid=0 2024-09-10T09:41:32.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:32.457+0000 7f562bac0740 1 client.25979 shutdown 2024-09-10T09:41:32.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:32.457+0000 7f562bac0740 1 -- 172.21.15.5:0/1576226662 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5576291bfb90 msgr2=0x5576291db820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:41:32.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:32.457+0000 7f562bac0740 1 --2- 172.21.15.5:0/1576226662 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5576291bfb90 0x5576291db820 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f5614009830 tx=0x7f561402a340 comp rx=0 tx=0).stop 2024-09-10T09:41:32.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:32.457+0000 7f562bac0740 1 -- 172.21.15.5:0/1576226662 shutdown_connections 2024-09-10T09:41:32.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:32.457+0000 7f562bac0740 1 --2- 172.21.15.5:0/1576226662 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55762924f2f0 0x557629251730 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:41:32.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:32.457+0000 7f562bac0740 1 --2- 172.21.15.5:0/1576226662 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5576291cad70 0x5576291d8610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:41:32.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:32.457+0000 7f562bac0740 1 --2- 172.21.15.5:0/1576226662 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5576291c04b0 0x5576291dbd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:41:32.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:32.457+0000 7f562bac0740 1 --2- 172.21.15.5:0/1576226662 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5576291bfb90 0x5576291db820 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:41:32.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:32.457+0000 7f5603e00640 0 client.25979 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T09:41:32.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:32.457+0000 7f562bac0740 1 -- 172.21.15.5:0/1576226662 >> 172.21.15.5:0/1576226662 conn(0x5576291bc120 msgr2=0x5576291c95b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:41:32.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:32.457+0000 7f562bac0740 1 -- 172.21.15.5:0/1576226662 shutdown_connections 2024-09-10T09:41:32.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:32.457+0000 7f562bac0740 1 -- 172.21.15.5:0/1576226662 wait complete. 2024-09-10T09:41:32.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:32.457+0000 7f562bac0740 20 client.25979 trim_cache size 0 max 16384 2024-09-10T09:41:32.463 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-10T09:41:32.463 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "no active scrubs running", 2024-09-10T09:41:32.463 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": {} 2024-09-10T09:41:32.464 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-10T09:41:32.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:32.457+0000 7f5629000640 1 -- 172.21.15.5:0/1275778555 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f56080a2b80 msgr2=0x7f56080a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:41:32.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:32.457+0000 7f5629000640 1 --2- 172.21.15.5:0/1275778555 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f56080a2b80 0x7f56080a5040 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f561c009a60 tx=0x7f561c010040 comp rx=0 tx=0).stop 2024-09-10T09:41:32.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:32.457+0000 7f5629000640 1 -- 172.21.15.5:0/1275778555 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5624104010 msgr2=0x7f562419a640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:41:32.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:32.457+0000 7f5629000640 1 --2- 172.21.15.5:0/1275778555 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5624104010 0x7f562419a640 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f5614035040 tx=0x7f5614033b60 comp rx=0 tx=0).stop 2024-09-10T09:41:32.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:32.457+0000 7f5629000640 1 -- 172.21.15.5:0/1275778555 shutdown_connections 2024-09-10T09:41:32.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:32.457+0000 7f5629000640 1 --2- 172.21.15.5:0/1275778555 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f56080a2b80 0x7f56080a5040 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:41:32.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:32.457+0000 7f5629000640 1 --2- 172.21.15.5:0/1275778555 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5624109140 0x7f562419ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:41:32.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:32.457+0000 7f5629000640 1 --2- 172.21.15.5:0/1275778555 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f56241049c0 0x7f562419ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:41:32.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:32.457+0000 7f5629000640 1 --2- 172.21.15.5:0/1275778555 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5624104010 0x7f562419a640 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:41:32.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:32.457+0000 7f5629000640 1 -- 172.21.15.5:0/1275778555 >> 172.21.15.5:0/1275778555 conn(0x7f56240ffa50 msgr2=0x7f5624100e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:41:32.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:32.457+0000 7f5629000640 1 -- 172.21.15.5:0/1275778555 shutdown_connections 2024-09-10T09:41:32.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:32.457+0000 7f5629000640 1 -- 172.21.15.5:0/1275778555 wait complete. 2024-09-10T09:41:33.000 INFO:tasks.workunit.client.0.smithi005.stdout: 1 124409 9.32 MB/sec warmup 74 sec latency 1952.689 ms 2024-09-10T09:41:33.614 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-10T09:41:33.614 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T09:41:33.679 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:41:33 smithi005 ceph-mon[30350]: pgmap v489: 129 pgs: 129 active+clean; 71 MiB data, 619 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.6 MiB/s wr, 195 op/s 2024-09-10T09:41:33.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:41:33 smithi159 ceph-mon[35513]: pgmap v489: 129 pgs: 129 active+clean; 71 MiB data, 619 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.6 MiB/s wr, 195 op/s 2024-09-10T09:41:33.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:41:33 smithi102 ceph-mon[35442]: pgmap v489: 129 pgs: 129 active+clean; 71 MiB data, 619 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.6 MiB/s wr, 195 op/s 2024-09-10T09:41:34.000 INFO:tasks.workunit.client.0.smithi005.stdout: 1 125755 9.27 MB/sec warmup 75 sec latency 2108.296 ms 2024-09-10T09:41:34.068 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:41:34.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:34.660+0000 7f9d8d600640 1 -- 172.21.15.5:0/4196067234 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9d88109140 msgr2=0x7f9d88110cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:41:34.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:34.660+0000 7f9d8d600640 1 --2- 172.21.15.5:0/4196067234 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9d88109140 0x7f9d88110cc0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f9d80009ca0 tx=0x7f9d8002a8b0 comp rx=0 tx=0).stop 2024-09-10T09:41:34.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:34.660+0000 7f9d8d600640 1 -- 172.21.15.5:0/4196067234 shutdown_connections 2024-09-10T09:41:34.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:34.660+0000 7f9d8d600640 1 --2- 172.21.15.5:0/4196067234 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9d88109140 0x7f9d88110cc0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:41:34.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:34.660+0000 7f9d8d600640 1 --2- 172.21.15.5:0/4196067234 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9d881049c0 0x7f9d88108a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:41:34.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:34.660+0000 7f9d8d600640 1 --2- 172.21.15.5:0/4196067234 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9d88104010 0x7f9d881043f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:41:34.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:34.660+0000 7f9d8d600640 1 -- 172.21.15.5:0/4196067234 >> 172.21.15.5:0/4196067234 conn(0x7f9d880ffa50 msgr2=0x7f9d88101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:41:34.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:34.661+0000 7f9d8d600640 1 -- 172.21.15.5:0/4196067234 shutdown_connections 2024-09-10T09:41:34.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:34.661+0000 7f9d8d600640 1 -- 172.21.15.5:0/4196067234 wait complete. 2024-09-10T09:41:34.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:34.662+0000 7f9d8d600640 1 Processor -- start 2024-09-10T09:41:34.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:34.662+0000 7f9d8d600640 1 -- start start 2024-09-10T09:41:34.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:34.663+0000 7f9d8d600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9d88104010 0x7f9d8819a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:41:34.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:34.663+0000 7f9d8d600640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9d881049c0 0x7f9d8819abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:41:34.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:34.663+0000 7f9d8d600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9d88109140 0x7f9d8819ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:41:34.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:34.663+0000 7f9d8d600640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f9d881141a0 con 0x7f9d88109140 2024-09-10T09:41:34.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:34.663+0000 7f9d8d600640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f9d88114020 con 0x7f9d88104010 2024-09-10T09:41:34.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:34.663+0000 7f9d8d600640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f9d88114320 con 0x7f9d881049c0 2024-09-10T09:41:34.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:34.663+0000 7f9d86a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9d88104010 0x7f9d8819a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:41:34.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:34.663+0000 7f9d86000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9d881049c0 0x7f9d8819abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:41:34.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:34.663+0000 7f9d86a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9d88104010 0x7f9d8819a6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:35834/0 (socket says 172.21.15.5:35834) 2024-09-10T09:41:34.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:34.663+0000 7f9d86a00640 1 -- 172.21.15.5:0/593071838 learned_addr learned my addr 172.21.15.5:0/593071838 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:41:34.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:34.663+0000 7f9d87400640 1 --2- 172.21.15.5:0/593071838 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9d88109140 0x7f9d8819ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:41:34.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:34.663+0000 7f9d86a00640 1 -- 172.21.15.5:0/593071838 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9d881049c0 msgr2=0x7f9d8819abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:41:34.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:34.663+0000 7f9d86a00640 1 --2- 172.21.15.5:0/593071838 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9d881049c0 0x7f9d8819abe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:41:34.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:34.663+0000 7f9d86a00640 1 -- 172.21.15.5:0/593071838 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9d88109140 msgr2=0x7f9d8819ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:41:34.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:34.663+0000 7f9d86a00640 1 --2- 172.21.15.5:0/593071838 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9d88109140 0x7f9d8819ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:41:34.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:34.663+0000 7f9d86a00640 1 -- 172.21.15.5:0/593071838 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d8819f650 con 0x7f9d88104010 2024-09-10T09:41:34.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:34.664+0000 7f9d86a00640 1 --2- 172.21.15.5:0/593071838 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9d88104010 0x7f9d8819a6a0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f9d7800e9f0 tx=0x7f9d7800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:41:34.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:34.664+0000 7f9d7f400640 1 -- 172.21.15.5:0/593071838 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d7800cd80 con 0x7f9d88104010 2024-09-10T09:41:34.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:34.664+0000 7f9d7f400640 1 -- 172.21.15.5:0/593071838 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9d78010910 con 0x7f9d88104010 2024-09-10T09:41:34.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:34.664+0000 7f9d7f400640 1 -- 172.21.15.5:0/593071838 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d78018990 con 0x7f9d88104010 2024-09-10T09:41:34.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:34.664+0000 7f9d8d600640 1 -- 172.21.15.5:0/593071838 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d8819f940 con 0x7f9d88104010 2024-09-10T09:41:34.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:34.664+0000 7f9d8d600640 1 -- 172.21.15.5:0/593071838 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d881a7220 con 0x7f9d88104010 2024-09-10T09:41:34.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:34.666+0000 7f9d8d600640 1 -- 172.21.15.5:0/593071838 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9d881043f0 con 0x7f9d88104010 2024-09-10T09:41:34.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:34.668+0000 7f9d7f400640 1 -- 172.21.15.5:0/593071838 <== mon.2 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f9d78010ac0 con 0x7f9d88104010 2024-09-10T09:41:34.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:34.669+0000 7f9d7f400640 1 --2- 172.21.15.5:0/593071838 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f9d6c0a2cd0 0x7f9d6c0a5190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:41:34.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:34.669+0000 7f9d7f400640 1 -- 172.21.15.5:0/593071838 <== mon.2 v2:172.21.15.102:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f9d780cfcc0 con 0x7f9d88104010 2024-09-10T09:41:34.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:34.669+0000 7f9d86000640 1 --2- 172.21.15.5:0/593071838 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f9d6c0a2cd0 0x7f9d6c0a5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:41:34.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:34.670+0000 7f9d86000640 1 --2- 172.21.15.5:0/593071838 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f9d6c0a2cd0 0x7f9d6c0a5190 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f9d8810ab10 tx=0x7f9d74005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:41:34.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:34.671+0000 7f9d7f400640 1 -- 172.21.15.5:0/593071838 <== mon.2 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f9d7809c850 con 0x7f9d88104010 2024-09-10T09:41:34.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:34.922+0000 7f9d8d600640 1 -- 172.21.15.5:0/593071838 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9d8819ba00 con 0x7f9d88104010 2024-09-10T09:41:34.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:34.923+0000 7f9d7f400640 1 -- 172.21.15.5:0/593071838 <== mon.2 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+11976 (secure 0 0 0) 0x7f9d7809c150 con 0x7f9d88104010 2024-09-10T09:41:34.926 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-10T09:41:34.926 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":32,"btime":"2024-09-10T09:40:27:751081+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14454,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/2970594311","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2970594311},{"type":"v1","addr":"172.21.15.5:6835","nonce":2970594311}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14514,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6839/3616502174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":3616502174},{"type":"v1","addr":"172.21.15.5:6839","nonce":3616502174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":32,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T09:35:29.626287+0000","modified":"2024-09-10T09:40:26.959994+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24542,"mds_1":24535,"mds_2":24511,"mds_3":24506,"mds_4":24443},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14520":{"gid":14520,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/2073599842","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":2073599842},{"type":"v1","addr":"172.21.15.5:6841","nonce":2073599842}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"k","rank":4,"incarnation":26,"state":"up:active","state_seq":24,"addr":"172.21.15.102:6833/2457007427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":2457007427},{"type":"v1","addr":"172.21.15.102:6833","nonce":2457007427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/356445159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":356445159},{"type":"v1","addr":"172.21.15.159:6833","nonce":356445159}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6837/2894314507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":2894314507},{"type":"v1","addr":"172.21.15.5:6837","nonce":2894314507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.102:6835/1647549111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":1647549111},{"type":"v1","addr":"172.21.15.102:6835","nonce":1647549111}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6835/952657242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":952657242},{"type":"v1","addr":"172.21.15.159:6835","nonce":952657242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":19,"addr":"172.21.15.102:6837/2689776764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":2689776764},{"type":"v1","addr":"172.21.15.102:6837","nonce":2689776764}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.159:6837/620627639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":620627639},{"type":"v1","addr":"172.21.15.159:6837","nonce":620627639}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24535":{"gid":24535,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":18,"addr":"172.21.15.159:6839/2073739599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":2073739599},{"type":"v1","addr":"172.21.15.159:6839","nonce":2073739599}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24542":{"gid":24542,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.102:6839/335409749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6838","nonce":335409749},{"type":"v1","addr":"172.21.15.102:6839","nonce":335409749}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24542,"qdb_cluster":[24542,24535,24511,24506,24443]},"id":1}]} 2024-09-10T09:41:34.926 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 32 2024-09-10T09:41:34.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:34.926+0000 7f9d8d600640 1 -- 172.21.15.5:0/593071838 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f9d6c0a2cd0 msgr2=0x7f9d6c0a5190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:41:34.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:34.926+0000 7f9d8d600640 1 --2- 172.21.15.5:0/593071838 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f9d6c0a2cd0 0x7f9d6c0a5190 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f9d8810ab10 tx=0x7f9d74005f90 comp rx=0 tx=0).stop 2024-09-10T09:41:34.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:34.926+0000 7f9d8d600640 1 -- 172.21.15.5:0/593071838 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9d88104010 msgr2=0x7f9d8819a6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:41:34.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:34.926+0000 7f9d8d600640 1 --2- 172.21.15.5:0/593071838 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9d88104010 0x7f9d8819a6a0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f9d7800e9f0 tx=0x7f9d7800eeb0 comp rx=0 tx=0).stop 2024-09-10T09:41:34.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:34.926+0000 7f9d8d600640 1 -- 172.21.15.5:0/593071838 shutdown_connections 2024-09-10T09:41:34.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:34.926+0000 7f9d8d600640 1 --2- 172.21.15.5:0/593071838 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f9d6c0a2cd0 0x7f9d6c0a5190 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:41:34.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:34.926+0000 7f9d8d600640 1 --2- 172.21.15.5:0/593071838 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9d88109140 0x7f9d8819ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:41:34.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:34.926+0000 7f9d8d600640 1 --2- 172.21.15.5:0/593071838 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9d881049c0 0x7f9d8819abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:41:34.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:34.926+0000 7f9d8d600640 1 --2- 172.21.15.5:0/593071838 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9d88104010 0x7f9d8819a6a0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:41:34.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:34.926+0000 7f9d8d600640 1 -- 172.21.15.5:0/593071838 >> 172.21.15.5:0/593071838 conn(0x7f9d880ffa50 msgr2=0x7f9d88100db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:41:34.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:34.926+0000 7f9d8d600640 1 -- 172.21.15.5:0/593071838 shutdown_connections 2024-09-10T09:41:34.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:34.927+0000 7f9d8d600640 1 -- 172.21.15.5:0/593071838 wait complete. 2024-09-10T09:41:35.000 INFO:tasks.workunit.client.0.smithi005.stdout: 1 128123 9.33 MB/sec warmup 76 sec latency 24.243 ms 2024-09-10T09:41:35.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:41:35 smithi005 ceph-mon[30350]: pgmap v490: 129 pgs: 129 active+clean; 72 MiB data, 650 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.6 MiB/s wr, 184 op/s 2024-09-10T09:41:35.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:41:35 smithi005 ceph-mon[30350]: from='client.? 172.21.15.5:0/593071838' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:41:35.766 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-10T09:41:35.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:41:35 smithi159 ceph-mon[35513]: pgmap v490: 129 pgs: 129 active+clean; 72 MiB data, 650 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.6 MiB/s wr, 184 op/s 2024-09-10T09:41:35.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:41:35 smithi159 ceph-mon[35513]: from='client.? 172.21.15.5:0/593071838' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:41:35.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:41:35 smithi102 ceph-mon[35442]: pgmap v490: 129 pgs: 129 active+clean; 72 MiB data, 650 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.6 MiB/s wr, 184 op/s 2024-09-10T09:41:35.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:41:35 smithi102 ceph-mon[35442]: from='client.? 172.21.15.5:0/593071838' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:41:36.004 INFO:tasks.workunit.client.0.smithi005.stdout: 1 130028 9.30 MB/sec warmup 77 sec latency 96.821 ms 2024-09-10T09:41:36.201 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:41:36.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:36.987+0000 7f9fc6a00640 1 -- 172.21.15.5:0/726276828 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9fc0103ff0 msgr2=0x7f9fc01043d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:41:36.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:36.987+0000 7f9fc6a00640 1 --2- 172.21.15.5:0/726276828 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9fc0103ff0 0x7f9fc01043d0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f9fb0009900 tx=0x7f9fb002a3f0 comp rx=0 tx=0).stop 2024-09-10T09:41:36.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:36.988+0000 7f9fc6a00640 1 -- 172.21.15.5:0/726276828 shutdown_connections 2024-09-10T09:41:36.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:36.988+0000 7f9fc6a00640 1 --2- 172.21.15.5:0/726276828 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9fc0109120 0x7f9fc0110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:41:36.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:36.988+0000 7f9fc6a00640 1 --2- 172.21.15.5:0/726276828 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9fc01049a0 0x7f9fc01089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:41:36.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:36.988+0000 7f9fc6a00640 1 --2- 172.21.15.5:0/726276828 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9fc0103ff0 0x7f9fc01043d0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:41:36.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:36.988+0000 7f9fc6a00640 1 -- 172.21.15.5:0/726276828 >> 172.21.15.5:0/726276828 conn(0x7f9fc00ffa90 msgr2=0x7f9fc0101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:41:36.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:36.988+0000 7f9fc6a00640 1 -- 172.21.15.5:0/726276828 shutdown_connections 2024-09-10T09:41:36.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:36.988+0000 7f9fc6a00640 1 -- 172.21.15.5:0/726276828 wait complete. 2024-09-10T09:41:36.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:36.989+0000 7f9fc6a00640 1 Processor -- start 2024-09-10T09:41:36.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:36.989+0000 7f9fc6a00640 1 -- start start 2024-09-10T09:41:36.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:36.989+0000 7f9fc6a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9fc0103ff0 0x7f9fc019a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:41:36.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:36.989+0000 7f9fc6a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9fc01049a0 0x7f9fc019ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:41:36.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:36.989+0000 7f9fc6a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9fc0109120 0x7f9fc019eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:41:36.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:36.989+0000 7f9fc6a00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f9fc0114120 con 0x7f9fc01049a0 2024-09-10T09:41:36.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:36.989+0000 7f9fc6a00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f9fc0113fa0 con 0x7f9fc0109120 2024-09-10T09:41:36.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:36.989+0000 7f9fc6a00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f9fc01142a0 con 0x7f9fc0103ff0 2024-09-10T09:41:36.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:36.990+0000 7f9fbf400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9fc01049a0 0x7f9fc019ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:41:36.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:36.990+0000 7f9fbf400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9fc01049a0 0x7f9fc019ab20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58436/0 (socket says 172.21.15.5:58436) 2024-09-10T09:41:36.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:36.990+0000 7f9fbf400640 1 -- 172.21.15.5:0/2913943689 learned_addr learned my addr 172.21.15.5:0/2913943689 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:41:36.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:36.990+0000 7f9fbfe00640 1 --2- 172.21.15.5:0/2913943689 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9fc0103ff0 0x7f9fc019a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:41:36.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:36.990+0000 7f9fc4a00640 1 --2- 172.21.15.5:0/2913943689 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9fc0109120 0x7f9fc019eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:41:36.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:36.990+0000 7f9fbf400640 1 -- 172.21.15.5:0/2913943689 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9fc0103ff0 msgr2=0x7f9fc019a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:41:36.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:36.990+0000 7f9fbf400640 1 --2- 172.21.15.5:0/2913943689 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9fc0103ff0 0x7f9fc019a5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:41:36.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:36.990+0000 7f9fbf400640 1 -- 172.21.15.5:0/2913943689 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9fc0109120 msgr2=0x7f9fc019eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:41:36.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:36.990+0000 7f9fbf400640 1 --2- 172.21.15.5:0/2913943689 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9fc0109120 0x7f9fc019eeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:41:36.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:36.990+0000 7f9fbf400640 1 -- 172.21.15.5:0/2913943689 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9fc019f590 con 0x7f9fc01049a0 2024-09-10T09:41:36.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:36.990+0000 7f9fc4a00640 1 --2- 172.21.15.5:0/2913943689 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9fc0109120 0x7f9fc019eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:41:36.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:36.990+0000 7f9fbfe00640 1 --2- 172.21.15.5:0/2913943689 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9fc0103ff0 0x7f9fc019a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:41:36.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:36.991+0000 7f9fbf400640 1 --2- 172.21.15.5:0/2913943689 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9fc01049a0 0x7f9fc019ab20 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f9fac00e9f0 tx=0x7f9fac00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:41:36.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:36.991+0000 7f9fbcc00640 1 -- 172.21.15.5:0/2913943689 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fac00cd80 con 0x7f9fc01049a0 2024-09-10T09:41:36.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:36.991+0000 7f9fbcc00640 1 -- 172.21.15.5:0/2913943689 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9fac010910 con 0x7f9fc01049a0 2024-09-10T09:41:36.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:36.991+0000 7f9fbcc00640 1 -- 172.21.15.5:0/2913943689 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fac018990 con 0x7f9fc01049a0 2024-09-10T09:41:36.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:36.991+0000 7f9fc6a00640 1 -- 172.21.15.5:0/2913943689 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9fc019f880 con 0x7f9fc01049a0 2024-09-10T09:41:36.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:36.991+0000 7f9fc6a00640 1 -- 172.21.15.5:0/2913943689 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9fc01a7160 con 0x7f9fc01049a0 2024-09-10T09:41:36.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:36.996+0000 7f9fbcc00640 1 -- 172.21.15.5:0/2913943689 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f9fac010ac0 con 0x7f9fc01049a0 2024-09-10T09:41:36.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:36.996+0000 7f9fc946c740 1 Processor -- start 2024-09-10T09:41:36.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:36.996+0000 7f9fc946c740 1 -- start start 2024-09-10T09:41:37.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:36.997+0000 7f9fc946c740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562fdc513c70 0x562fdc514050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:41:37.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:36.997+0000 7f9fc946c740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562fdc514590 0x562fdc51e910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:41:37.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:36.997+0000 7f9fc946c740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562fdc51ee50 0x562fdc521240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:41:37.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:36.997+0000 7f9fc946c740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x562fdc47a3e0 con 0x562fdc51ee50 2024-09-10T09:41:37.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:36.997+0000 7f9fc946c740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x562fdc421d80 con 0x562fdc513c70 2024-09-10T09:41:37.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:36.997+0000 7f9fc946c740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x562fdc4763a0 con 0x562fdc514590 2024-09-10T09:41:37.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:36.997+0000 7f9fc4a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562fdc513c70 0x562fdc514050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:41:37.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:36.997+0000 7f9fc4a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562fdc513c70 0x562fdc514050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:35858/0 (socket says 172.21.15.5:35858) 2024-09-10T09:41:37.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:36.997+0000 7f9fc4a00640 1 -- 172.21.15.5:0/3036468021 learned_addr learned my addr 172.21.15.5:0/3036468021 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:41:37.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:36.997+0000 7f9fbf400640 1 --2- 172.21.15.5:0/3036468021 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562fdc51ee50 0x562fdc521240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:41:37.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:36.997+0000 7f9fbcc00640 1 --2- 172.21.15.5:0/2913943689 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f9fa40a2c00 0x7f9fa40a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:41:37.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:36.998+0000 7f9fbcc00640 1 -- 172.21.15.5:0/2913943689 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f9fac0cf7b0 con 0x7f9fc01049a0 2024-09-10T09:41:37.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:36.998+0000 7f9fc4a00640 1 -- 172.21.15.5:0/3036468021 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562fdc514590 msgr2=0x562fdc51e910 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T09:41:37.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:36.998+0000 7f9fc4a00640 1 --2- 172.21.15.5:0/3036468021 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562fdc514590 0x562fdc51e910 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:41:37.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:36.998+0000 7f9fc4a00640 1 -- 172.21.15.5:0/3036468021 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562fdc51ee50 msgr2=0x562fdc521240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:41:37.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:36.998+0000 7f9fc4a00640 1 --2- 172.21.15.5:0/3036468021 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562fdc51ee50 0x562fdc521240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:41:37.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:36.998+0000 7f9fc4a00640 1 -- 172.21.15.5:0/3036468021 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562fdc440ae0 con 0x562fdc513c70 2024-09-10T09:41:37.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:36.998+0000 7f9fbf400640 1 --2- 172.21.15.5:0/3036468021 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562fdc51ee50 0x562fdc521240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:41:37.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:36.998+0000 7f9fc4a00640 1 --2- 172.21.15.5:0/2913943689 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f9fa40a2c00 0x7f9fa40a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:41:37.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:36.998+0000 7f9fc4a00640 1 --2- 172.21.15.5:0/3036468021 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562fdc513c70 0x562fdc514050 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f9fb8009fd0 tx=0x7f9fb8010940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7546ddb729f27c70 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:41:37.003 INFO:tasks.workunit.client.0.smithi005.stdout: 1 130029 9.18 MB/sec warmup 78 sec latency 860.212 ms 2024-09-10T09:41:37.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:36.999+0000 7f9fb5200640 1 -- 172.21.15.5:0/3036468021 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fb8019070 con 0x562fdc513c70 2024-09-10T09:41:37.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:36.999+0000 7f9fb5200640 1 -- 172.21.15.5:0/3036468021 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9fb8022070 con 0x562fdc513c70 2024-09-10T09:41:37.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:36.999+0000 7f9fb5200640 1 -- 172.21.15.5:0/3036468021 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fb8026c50 con 0x562fdc513c70 2024-09-10T09:41:37.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:37.000+0000 7f9fc946c740 1 -- 172.21.15.5:0/3036468021 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562fdc513c70 msgr2=0x562fdc514050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:41:37.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:37.000+0000 7f9fc4a00640 1 --2- 172.21.15.5:0/2913943689 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f9fa40a2c00 0x7f9fa40a50c0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f9fb8021000 tx=0x7f9fb801f000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:41:37.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:37.000+0000 7f9fc946c740 1 --2- 172.21.15.5:0/3036468021 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562fdc513c70 0x562fdc514050 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f9fb8009fd0 tx=0x7f9fb8010940 comp rx=0 tx=0).stop 2024-09-10T09:41:37.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:37.000+0000 7f9fc946c740 1 -- 172.21.15.5:0/3036468021 shutdown_connections 2024-09-10T09:41:37.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:37.000+0000 7f9fc946c740 1 --2- 172.21.15.5:0/3036468021 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562fdc51ee50 0x562fdc521240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:41:37.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:37.000+0000 7f9fc946c740 1 --2- 172.21.15.5:0/3036468021 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562fdc514590 0x562fdc51e910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:41:37.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:37.000+0000 7f9fc946c740 1 --2- 172.21.15.5:0/3036468021 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562fdc513c70 0x562fdc514050 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:41:37.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:37.000+0000 7f9fc946c740 1 -- 172.21.15.5:0/3036468021 >> 172.21.15.5:0/3036468021 conn(0x562fdc510200 msgr2=0x562fdc5105e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:41:37.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:37.000+0000 7f9fc946c740 1 -- 172.21.15.5:0/3036468021 shutdown_connections 2024-09-10T09:41:37.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:37.000+0000 7f9fc946c740 1 -- 172.21.15.5:0/3036468021 wait complete. 2024-09-10T09:41:37.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:37.001+0000 7f9fc946c740 1 -- start start 2024-09-10T09:41:37.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:37.001+0000 7f9fc946c740 1 Processor -- start 2024-09-10T09:41:37.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:37.001+0000 7f9fc946c740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562fdc513c70 0x562fdc51bd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:41:37.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:37.001+0000 7f9fc946c740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562fdc514590 0x562fdc51c260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:41:37.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:37.001+0000 7f9fc946c740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562fdc51ee50 0x562fdc51c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:41:37.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:37.002+0000 7f9fc946c740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x562fdc521a20 con 0x562fdc51ee50 2024-09-10T09:41:37.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:37.002+0000 7f9fc946c740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x562fdc4350f0 con 0x562fdc514590 2024-09-10T09:41:37.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:37.002+0000 7f9fc946c740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x562fdc521ba0 con 0x562fdc513c70 2024-09-10T09:41:37.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:37.002+0000 7f9fc946c740 10 client.0 _ll_register_callbacks cb 0x562fdc41cbd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:41:37.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:37.002+0000 7f9fc4a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562fdc51ee50 0x562fdc51c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:41:37.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:37.002+0000 7f9fbfe00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562fdc513c70 0x562fdc51bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:41:37.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:37.002+0000 7f9fc4a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562fdc51ee50 0x562fdc51c7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58454/0 (socket says 172.21.15.5:58454) 2024-09-10T09:41:37.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:37.002+0000 7f9fc4a00640 1 -- 172.21.15.5:0/3092945375 learned_addr learned my addr 172.21.15.5:0/3092945375 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:41:37.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:37.002+0000 7f9fbfe00640 1 -- 172.21.15.5:0/3092945375 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562fdc514590 msgr2=0x562fdc51c260 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T09:41:37.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:37.002+0000 7f9fbfe00640 1 --2- 172.21.15.5:0/3092945375 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562fdc514590 0x562fdc51c260 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:41:37.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:37.002+0000 7f9fbfe00640 1 -- 172.21.15.5:0/3092945375 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562fdc51ee50 msgr2=0x562fdc51c7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:41:37.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:37.002+0000 7f9fbfe00640 1 --2- 172.21.15.5:0/3092945375 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562fdc51ee50 0x562fdc51c7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:41:37.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:37.002+0000 7f9fbfe00640 1 -- 172.21.15.5:0/3092945375 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562fdc1eb3c0 con 0x562fdc513c70 2024-09-10T09:41:37.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:37.002+0000 7f9fc4a00640 1 --2- 172.21.15.5:0/3092945375 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562fdc51ee50 0x562fdc51c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:41:37.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:37.003+0000 7f9fbfe00640 1 --2- 172.21.15.5:0/3092945375 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562fdc513c70 0x562fdc51bd20 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f9fb0005940 tx=0x7f9fb00317e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=508b1744f51e6229 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:41:37.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:37.003+0000 7f9fa9600640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-09-10T09:41:37.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:37.003+0000 7f9fa9600640 1 -- 172.21.15.5:0/3092945375 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fb0035040 con 0x562fdc513c70 2024-09-10T09:41:37.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:37.003+0000 7f9fc946c740 1 -- 172.21.15.5:0/3092945375 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9fb8022070 con 0x562fdc513c70 2024-09-10T09:41:37.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:37.003+0000 7f9fa9600640 1 -- 172.21.15.5:0/3092945375 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f9fb0004240 con 0x562fdc513c70 2024-09-10T09:41:37.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:37.003+0000 7f9fa9600640 1 -- 172.21.15.5:0/3092945375 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f9fb0004440 con 0x562fdc513c70 2024-09-10T09:41:37.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:37.004+0000 7f9fc946c740 10 client.26017 fetch_fsmap learned FSMap version 32 2024-09-10T09:41:37.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:37.004+0000 7f9fc946c740 1 -- 172.21.15.5:0/3092945375 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x562fdc52da90 con 0x562fdc513c70 2024-09-10T09:41:37.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:37.004+0000 7f9fa9600640 1 -- 172.21.15.5:0/3092945375 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9fb8022070 con 0x562fdc513c70 2024-09-10T09:41:37.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:37.004+0000 7f9fa9600640 1 -- 172.21.15.5:0/3092945375 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7f9fb003c6a0 con 0x562fdc513c70 2024-09-10T09:41:37.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:37.004+0000 7f9fc946c740 10 client.26017 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T09:41:37.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:37.004+0000 7f9fc946c740 10 client.26017 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-09-10T09:41:37.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:37.004+0000 7f9fc946c740 20 client.26017 populate_metadata read hostname 'smithi005' 2024-09-10T09:41:37.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:37.004+0000 7f9fc946c740 1 --2- 172.21.15.5:0/3092945375 >> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] conn(0x562fdc52eea0 0x562fdc5312b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:41:37.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:37.004+0000 7f9fc946c740 4 client.26017 mds_command: new command op to 24443 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:41:37.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:37.004+0000 7f9fc946c740 1 -- 172.21.15.5:0/3092945375 --> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562fdc487950 con 0x562fdc52eea0 2024-09-10T09:41:37.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:37.028+0000 7f9fbf400640 1 --2- 172.21.15.5:0/3092945375 >> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] conn(0x562fdc52eea0 0x562fdc5312b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:41:37.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:37.076+0000 7f9fbf400640 1 --2- 172.21.15.5:0/3092945375 >> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] conn(0x562fdc52eea0 0x562fdc5312b0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=746a644b18151e56 server_cookie=41db4677be72cf31 in_seq=0 out_seq=0 2024-09-10T09:41:37.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:37.076+0000 7f9fa9600640 10 client.26017 ms_handle_connect on v2:172.21.15.102:6832/2457007427 2024-09-10T09:41:37.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:37.088+0000 7f9fa9600640 1 -- 172.21.15.5:0/3092945375 <== mds.4 v2:172.21.15.102:6832/2457007427 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562fdc487950 con 0x562fdc52eea0 2024-09-10T09:41:37.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:37.088+0000 7f9fa9600640 10 client.26017 handle_command_reply: tid=0 2024-09-10T09:41:37.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:37.088+0000 7f9fc946c740 1 client.26017 shutdown 2024-09-10T09:41:37.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:37.089+0000 7f9fc946c740 1 -- 172.21.15.5:0/3092945375 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562fdc513c70 msgr2=0x562fdc51bd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:41:37.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:37.089+0000 7f9fc946c740 1 --2- 172.21.15.5:0/3092945375 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562fdc513c70 0x562fdc51bd20 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f9fb0005940 tx=0x7f9fb00317e0 comp rx=0 tx=0).stop 2024-09-10T09:41:37.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:37.089+0000 7f9fc946c740 1 -- 172.21.15.5:0/3092945375 shutdown_connections 2024-09-10T09:41:37.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:37.089+0000 7f9fc946c740 1 --2- 172.21.15.5:0/3092945375 >> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] conn(0x562fdc52eea0 0x562fdc5312b0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:41:37.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:37.089+0000 7f9fc946c740 1 --2- 172.21.15.5:0/3092945375 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562fdc51ee50 0x562fdc51c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:41:37.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:37.089+0000 7f9fc946c740 1 --2- 172.21.15.5:0/3092945375 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562fdc514590 0x562fdc51c260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:41:37.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:37.089+0000 7f9fc946c740 1 --2- 172.21.15.5:0/3092945375 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562fdc513c70 0x562fdc51bd20 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:41:37.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:37.089+0000 7f9fa9600640 0 client.26017 ms_handle_reset on v2:172.21.15.102:6832/2457007427 2024-09-10T09:41:37.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:37.090+0000 7f9fc946c740 1 -- 172.21.15.5:0/3092945375 >> 172.21.15.5:0/3092945375 conn(0x562fdc510200 msgr2=0x562fdc50f380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:41:37.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:37.090+0000 7f9fc946c740 1 -- 172.21.15.5:0/3092945375 shutdown_connections 2024-09-10T09:41:37.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:37.090+0000 7f9fc946c740 1 -- 172.21.15.5:0/3092945375 wait complete. 2024-09-10T09:41:37.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:37.090+0000 7f9fc946c740 20 client.26017 trim_cache size 0 max 16384 2024-09-10T09:41:37.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:37.131+0000 7f9fc946c740 1 Processor -- start 2024-09-10T09:41:37.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:37.132+0000 7f9fc946c740 1 -- start start 2024-09-10T09:41:37.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:37.132+0000 7f9fc946c740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562fdc513c70 0x562fdc5166d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:41:37.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:37.132+0000 7f9fc946c740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562fdc514590 0x562fdc536b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:41:37.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:37.132+0000 7f9fc946c740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562fdc51ee50 0x562fdc5370c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:41:37.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:37.132+0000 7f9fc946c740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f9fc01b1060 con 0x562fdc51ee50 2024-09-10T09:41:37.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:37.132+0000 7f9fc946c740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f9fc01b2c60 con 0x562fdc513c70 2024-09-10T09:41:37.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:37.132+0000 7f9fc946c740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f9fc01b1730 con 0x562fdc514590 2024-09-10T09:41:37.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:37.133+0000 7f9fbfe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562fdc513c70 0x562fdc5166d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:41:37.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:37.133+0000 7f9fbfe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562fdc513c70 0x562fdc5166d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:35868/0 (socket says 172.21.15.5:35868) 2024-09-10T09:41:37.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:37.134+0000 7f9fbfe00640 1 -- 172.21.15.5:0/685639162 learned_addr learned my addr 172.21.15.5:0/685639162 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:41:37.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:37.134+0000 7f9fbf400640 1 --2- 172.21.15.5:0/685639162 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562fdc514590 0x562fdc536b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:41:37.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:37.135+0000 7f9fbfe00640 1 -- 172.21.15.5:0/685639162 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562fdc514590 msgr2=0x562fdc536b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:41:37.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:37.135+0000 7f9fbfe00640 1 --2- 172.21.15.5:0/685639162 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562fdc514590 0x562fdc536b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:41:37.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:37.135+0000 7f9fbfe00640 1 -- 172.21.15.5:0/685639162 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562fdc51ee50 msgr2=0x562fdc5370c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:41:37.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:37.135+0000 7f9fbfe00640 1 --2- 172.21.15.5:0/685639162 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562fdc51ee50 0x562fdc5370c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:41:37.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:37.135+0000 7f9fbfe00640 1 -- 172.21.15.5:0/685639162 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562fdc50fca0 con 0x562fdc513c70 2024-09-10T09:41:37.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:37.136+0000 7f9fbfe00640 1 --2- 172.21.15.5:0/685639162 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562fdc513c70 0x562fdc5166d0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f9fb00463e0 tx=0x7f9fb0000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7fc962bbfdbe89aa server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:41:37.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:37.137+0000 7f9fabe00640 1 -- 172.21.15.5:0/685639162 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fb0035070 con 0x562fdc513c70 2024-09-10T09:41:37.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:37.137+0000 7f9fabe00640 1 -- 172.21.15.5:0/685639162 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9fb0009520 con 0x562fdc513c70 2024-09-10T09:41:37.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:37.137+0000 7f9fabe00640 1 -- 172.21.15.5:0/685639162 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fb0005360 con 0x562fdc513c70 2024-09-10T09:41:37.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:37.138+0000 7f9fc946c740 1 -- 172.21.15.5:0/685639162 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562fdc513c70 msgr2=0x562fdc5166d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:41:37.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:37.138+0000 7f9fc946c740 1 --2- 172.21.15.5:0/685639162 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562fdc513c70 0x562fdc5166d0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f9fb00463e0 tx=0x7f9fb0000bc0 comp rx=0 tx=0).stop 2024-09-10T09:41:37.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:37.139+0000 7f9fc946c740 1 -- 172.21.15.5:0/685639162 shutdown_connections 2024-09-10T09:41:37.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:37.139+0000 7f9fc946c740 1 --2- 172.21.15.5:0/685639162 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562fdc51ee50 0x562fdc5370c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:41:37.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:37.139+0000 7f9fc946c740 1 --2- 172.21.15.5:0/685639162 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562fdc514590 0x562fdc536b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:41:37.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:37.139+0000 7f9fc946c740 1 --2- 172.21.15.5:0/685639162 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562fdc513c70 0x562fdc5166d0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:41:37.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:37.139+0000 7f9fc946c740 1 -- 172.21.15.5:0/685639162 >> 172.21.15.5:0/685639162 conn(0x562fdc510200 msgr2=0x562fdc51dda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:41:37.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:37.139+0000 7f9fc946c740 1 -- 172.21.15.5:0/685639162 shutdown_connections 2024-09-10T09:41:37.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:37.139+0000 7f9fc946c740 1 -- 172.21.15.5:0/685639162 wait complete. 2024-09-10T09:41:37.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:37.140+0000 7f9fc946c740 1 -- start start 2024-09-10T09:41:37.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:37.141+0000 7f9fc946c740 1 Processor -- start 2024-09-10T09:41:37.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:37.141+0000 7f9fc946c740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562fdc513c70 0x562fdc522510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:41:37.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:37.141+0000 7f9fc946c740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562fdc514590 0x562fdc522a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:41:37.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:37.141+0000 7f9fc946c740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562fdc51ee50 0x562fdc526b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:41:37.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:37.141+0000 7f9fc946c740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f9fc01b24d0 con 0x562fdc513c70 2024-09-10T09:41:37.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:37.141+0000 7f9fc946c740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f9fc01b1e00 con 0x562fdc51ee50 2024-09-10T09:41:37.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:37.141+0000 7f9fc946c740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f9fc01af040 con 0x562fdc514590 2024-09-10T09:41:37.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:37.141+0000 7f9fc946c740 10 client.0 _ll_register_callbacks cb 0x7f9fac005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:41:37.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:37.142+0000 7f9fc4a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562fdc51ee50 0x562fdc526b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:41:37.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:37.142+0000 7f9fc4a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562fdc51ee50 0x562fdc526b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:35876/0 (socket says 172.21.15.5:35876) 2024-09-10T09:41:37.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:37.142+0000 7f9fc4a00640 1 -- 172.21.15.5:0/288870863 learned_addr learned my addr 172.21.15.5:0/288870863 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:41:37.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:37.142+0000 7f9fc4a00640 1 -- 172.21.15.5:0/288870863 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562fdc514590 msgr2=0x562fdc522a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:41:37.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:37.142+0000 7f9fc4a00640 1 --2- 172.21.15.5:0/288870863 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562fdc514590 0x562fdc522a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:41:37.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:37.142+0000 7f9fc4a00640 1 -- 172.21.15.5:0/288870863 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562fdc513c70 msgr2=0x562fdc522510 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T09:41:37.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:37.143+0000 7f9fbfe00640 1 --2- 172.21.15.5:0/288870863 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562fdc513c70 0x562fdc522510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:41:37.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:37.143+0000 7f9fc4a00640 1 --2- 172.21.15.5:0/288870863 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562fdc513c70 0x562fdc522510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:41:37.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:37.143+0000 7f9fc4a00640 1 -- 172.21.15.5:0/288870863 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562fdc527280 con 0x562fdc51ee50 2024-09-10T09:41:37.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:37.143+0000 7f9fa9600640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-09-10T09:41:37.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:37.143+0000 7f9fc4a00640 1 --2- 172.21.15.5:0/288870863 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562fdc51ee50 0x562fdc526b90 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f9fb80269e0 tx=0x7f9fb8026650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1b2fc28acc5bbab8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:41:37.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:37.144+0000 7f9fa9600640 1 -- 172.21.15.5:0/288870863 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fb8033020 con 0x562fdc51ee50 2024-09-10T09:41:37.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:37.144+0000 7f9fa9600640 1 -- 172.21.15.5:0/288870863 <== mon.2 v2:172.21.15.102:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f9fb800e880 con 0x562fdc51ee50 2024-09-10T09:41:37.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:37.144+0000 7f9fc946c740 1 -- 172.21.15.5:0/288870863 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9fb0009520 con 0x562fdc51ee50 2024-09-10T09:41:37.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:37.145+0000 7f9fa9600640 1 -- 172.21.15.5:0/288870863 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f9fb8033020 con 0x562fdc51ee50 2024-09-10T09:41:37.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:37.145+0000 7f9fc946c740 10 client.26000 fetch_fsmap learned FSMap version 32 2024-09-10T09:41:37.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:37.145+0000 7f9fc946c740 1 -- 172.21.15.5:0/288870863 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x562fdc5a1e90 con 0x562fdc51ee50 2024-09-10T09:41:37.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:37.145+0000 7f9fa9600640 1 -- 172.21.15.5:0/288870863 <== mon.2 v2:172.21.15.102:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9fb8022040 con 0x562fdc51ee50 2024-09-10T09:41:37.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:37.146+0000 7f9fa9600640 1 -- 172.21.15.5:0/288870863 <== mon.2 v2:172.21.15.102:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7f9fb802e030 con 0x562fdc51ee50 2024-09-10T09:41:37.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:37.146+0000 7f9fc946c740 10 client.26000 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T09:41:37.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:37.146+0000 7f9fc946c740 10 client.26000 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-09-10T09:41:37.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:37.146+0000 7f9fc946c740 20 client.26000 populate_metadata read hostname 'smithi005' 2024-09-10T09:41:37.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:37.146+0000 7f9fc946c740 1 --2- 172.21.15.5:0/288870863 >> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] conn(0x562fdc5a2590 0x562fdc5a49b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:41:37.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:37.146+0000 7f9fc946c740 4 client.26000 mds_command: new command op to 24443 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T09:41:37.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:37.146+0000 7f9fc946c740 1 -- 172.21.15.5:0/288870863 --> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f9fb0009520 con 0x562fdc5a2590 2024-09-10T09:41:37.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:37.146+0000 7f9fbfe00640 1 --2- 172.21.15.5:0/288870863 >> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] conn(0x562fdc5a2590 0x562fdc5a49b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:41:37.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:37.147+0000 7f9fa9600640 10 client.26000 ms_handle_connect on v2:172.21.15.102:6832/2457007427 2024-09-10T09:41:37.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:37.147+0000 7f9fbfe00640 1 --2- 172.21.15.5:0/288870863 >> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] conn(0x562fdc5a2590 0x562fdc5a49b0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=c6c899fe1cccd9d0 server_cookie=571235fc7ce53066 in_seq=0 out_seq=0 2024-09-10T09:41:37.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:37.148+0000 7f9fa9600640 1 -- 172.21.15.5:0/288870863 <== mds.4 v2:172.21.15.102:6832/2457007427 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f9fb0009520 con 0x562fdc5a2590 2024-09-10T09:41:37.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:37.148+0000 7f9fa9600640 10 client.26000 handle_command_reply: tid=0 2024-09-10T09:41:37.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:37.148+0000 7f9fc946c740 1 client.26000 shutdown 2024-09-10T09:41:37.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:37.148+0000 7f9fc946c740 1 -- 172.21.15.5:0/288870863 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562fdc51ee50 msgr2=0x562fdc526b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:41:37.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:37.148+0000 7f9fc946c740 1 --2- 172.21.15.5:0/288870863 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562fdc51ee50 0x562fdc526b90 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f9fb80269e0 tx=0x7f9fb8026650 comp rx=0 tx=0).stop 2024-09-10T09:41:37.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:37.148+0000 7f9fc946c740 1 -- 172.21.15.5:0/288870863 shutdown_connections 2024-09-10T09:41:37.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:37.148+0000 7f9fc946c740 1 --2- 172.21.15.5:0/288870863 >> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] conn(0x562fdc5a2590 0x562fdc5a49b0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:41:37.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:37.148+0000 7f9fc946c740 1 --2- 172.21.15.5:0/288870863 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562fdc51ee50 0x562fdc526b90 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:41:37.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:37.148+0000 7f9fc946c740 1 --2- 172.21.15.5:0/288870863 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562fdc514590 0x562fdc522a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:41:37.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:37.148+0000 7f9fc946c740 1 --2- 172.21.15.5:0/288870863 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562fdc513c70 0x562fdc522510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:41:37.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:37.148+0000 7f9fa9600640 0 client.26000 ms_handle_reset on v2:172.21.15.102:6832/2457007427 2024-09-10T09:41:37.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:37.148+0000 7f9fc946c740 1 -- 172.21.15.5:0/288870863 >> 172.21.15.5:0/288870863 conn(0x562fdc510200 msgr2=0x562fdc529c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:41:37.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:37.148+0000 7f9fc946c740 1 -- 172.21.15.5:0/288870863 shutdown_connections 2024-09-10T09:41:37.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:37.148+0000 7f9fc946c740 1 -- 172.21.15.5:0/288870863 wait complete. 2024-09-10T09:41:37.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:37.149+0000 7f9fc946c740 20 client.26000 trim_cache size 0 max 16384 2024-09-10T09:41:37.154 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-10T09:41:37.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:37.149+0000 7f9fc6a00640 1 -- 172.21.15.5:0/2913943689 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f9fa40a2c00 msgr2=0x7f9fa40a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:41:37.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:37.149+0000 7f9fc6a00640 1 --2- 172.21.15.5:0/2913943689 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f9fa40a2c00 0x7f9fa40a50c0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f9fb8021000 tx=0x7f9fb801f000 comp rx=0 tx=0).stop 2024-09-10T09:41:37.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:37.149+0000 7f9fc6a00640 1 -- 172.21.15.5:0/2913943689 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9fc01049a0 msgr2=0x7f9fc019ab20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:41:37.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:37.149+0000 7f9fc6a00640 1 --2- 172.21.15.5:0/2913943689 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9fc01049a0 0x7f9fc019ab20 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f9fac00e9f0 tx=0x7f9fac00eeb0 comp rx=0 tx=0).stop 2024-09-10T09:41:37.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:37.150+0000 7f9fc6a00640 1 -- 172.21.15.5:0/2913943689 shutdown_connections 2024-09-10T09:41:37.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:37.150+0000 7f9fc6a00640 1 --2- 172.21.15.5:0/2913943689 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f9fa40a2c00 0x7f9fa40a50c0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:41:37.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:37.150+0000 7f9fc6a00640 1 --2- 172.21.15.5:0/2913943689 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9fc0109120 0x7f9fc019eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:41:37.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:37.150+0000 7f9fc6a00640 1 --2- 172.21.15.5:0/2913943689 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9fc01049a0 0x7f9fc019ab20 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:41:37.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:37.150+0000 7f9fc6a00640 1 --2- 172.21.15.5:0/2913943689 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9fc0103ff0 0x7f9fc019a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:41:37.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:37.150+0000 7f9fc6a00640 1 -- 172.21.15.5:0/2913943689 >> 172.21.15.5:0/2913943689 conn(0x7f9fc00ffa90 msgr2=0x7f9fc0100e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:41:37.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:37.151+0000 7f9fc6a00640 1 -- 172.21.15.5:0/2913943689 shutdown_connections 2024-09-10T09:41:37.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:37.151+0000 7f9fc6a00640 1 -- 172.21.15.5:0/2913943689 wait complete. 2024-09-10T09:41:37.695 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:41:37 smithi005 ceph-mon[30350]: pgmap v491: 129 pgs: 129 active+clean; 75 MiB data, 691 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.8 MiB/s wr, 197 op/s 2024-09-10T09:41:37.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:41:37 smithi159 ceph-mon[35513]: pgmap v491: 129 pgs: 129 active+clean; 75 MiB data, 691 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.8 MiB/s wr, 197 op/s 2024-09-10T09:41:37.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:41:37 smithi102 ceph-mon[35442]: pgmap v491: 129 pgs: 129 active+clean; 75 MiB data, 691 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.8 MiB/s wr, 197 op/s 2024-09-10T09:41:38.001 INFO:tasks.workunit.client.0.smithi005.stdout: 1 130029 9.06 MB/sec warmup 79 sec latency 1860.250 ms 2024-09-10T09:41:38.347 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-10T09:41:38.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:41:38 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:41:38.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:41:38 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:41:38.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:41:38 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:41:38.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:41:38 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:41:38.757 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:41:38.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:41:38 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:41:38.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:41:38 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:41:38.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:41:38 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:41:38.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:41:38 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:41:38.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:41:38 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:41:38.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:41:38 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:41:38.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:41:38 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:41:38.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:41:38 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:41:39.001 INFO:tasks.workunit.client.0.smithi005.stdout: 1 131869 9.11 MB/sec warmup 80 sec latency 1896.785 ms 2024-09-10T09:41:39.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:39.512+0000 7fdcb5c00640 1 -- 172.21.15.5:0/2138189204 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdcb0104800 msgr2=0x7fdcb0108960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:41:39.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:39.512+0000 7fdcb5c00640 1 --2- 172.21.15.5:0/2138189204 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdcb0104800 0x7fdcb0108960 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fdc9c009900 tx=0x7fdc9c02a3f0 comp rx=0 tx=0).stop 2024-09-10T09:41:39.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:39.512+0000 7fdcb5c00640 1 -- 172.21.15.5:0/2138189204 shutdown_connections 2024-09-10T09:41:39.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:39.512+0000 7fdcb5c00640 1 --2- 172.21.15.5:0/2138189204 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdcb0109090 0x7fdcb0110c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:41:39.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:39.512+0000 7fdcb5c00640 1 --2- 172.21.15.5:0/2138189204 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdcb0104800 0x7fdcb0108960 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:41:39.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:39.512+0000 7fdcb5c00640 1 --2- 172.21.15.5:0/2138189204 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdcb0103e50 0x7fdcb0104230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:41:39.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:39.512+0000 7fdcb5c00640 1 -- 172.21.15.5:0/2138189204 >> 172.21.15.5:0/2138189204 conn(0x7fdcb00ff8b0 msgr2=0x7fdcb0101cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:41:39.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:39.512+0000 7fdcb5c00640 1 -- 172.21.15.5:0/2138189204 shutdown_connections 2024-09-10T09:41:39.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:39.513+0000 7fdcb5c00640 1 -- 172.21.15.5:0/2138189204 wait complete. 2024-09-10T09:41:39.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:39.514+0000 7fdcb5c00640 1 Processor -- start 2024-09-10T09:41:39.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:39.514+0000 7fdcb5c00640 1 -- start start 2024-09-10T09:41:39.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:39.514+0000 7fdcb5c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdcb0103e50 0x7fdcb019a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:41:39.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:39.514+0000 7fdcb5c00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdcb0104800 0x7fdcb019ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:41:39.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:39.514+0000 7fdcb5c00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdcb0109090 0x7fdcb019eea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:41:39.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:39.514+0000 7fdcb5c00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fdcb0114090 con 0x7fdcb0109090 2024-09-10T09:41:39.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:39.514+0000 7fdcb5c00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fdcb0113f10 con 0x7fdcb0103e50 2024-09-10T09:41:39.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:39.514+0000 7fdcb5c00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fdcb0114210 con 0x7fdcb0104800 2024-09-10T09:41:39.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:39.514+0000 7fdcafe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdcb0103e50 0x7fdcb019a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:41:39.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:39.514+0000 7fdcafe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdcb0103e50 0x7fdcb019a5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:35888/0 (socket says 172.21.15.5:35888) 2024-09-10T09:41:39.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:39.514+0000 7fdcafe00640 1 -- 172.21.15.5:0/1697517840 learned_addr learned my addr 172.21.15.5:0/1697517840 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:41:39.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:39.514+0000 7fdcaf400640 1 --2- 172.21.15.5:0/1697517840 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdcb0104800 0x7fdcb019ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:41:39.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:39.515+0000 7fdcb5200640 1 --2- 172.21.15.5:0/1697517840 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdcb0109090 0x7fdcb019eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:41:39.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:39.515+0000 7fdcafe00640 1 -- 172.21.15.5:0/1697517840 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdcb0104800 msgr2=0x7fdcb019ab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:41:39.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:39.515+0000 7fdcafe00640 1 --2- 172.21.15.5:0/1697517840 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdcb0104800 0x7fdcb019ab10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:41:39.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:39.515+0000 7fdcafe00640 1 -- 172.21.15.5:0/1697517840 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdcb0109090 msgr2=0x7fdcb019eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:41:39.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:39.515+0000 7fdcafe00640 1 --2- 172.21.15.5:0/1697517840 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdcb0109090 0x7fdcb019eea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:41:39.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:39.515+0000 7fdcafe00640 1 -- 172.21.15.5:0/1697517840 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdcb019f580 con 0x7fdcb0103e50 2024-09-10T09:41:39.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:39.515+0000 7fdcb5200640 1 --2- 172.21.15.5:0/1697517840 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdcb0109090 0x7fdcb019eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:41:39.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:39.515+0000 7fdcafe00640 1 --2- 172.21.15.5:0/1697517840 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdcb0103e50 0x7fdcb019a5d0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fdca000e9f0 tx=0x7fdca000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:41:39.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:39.516+0000 7fdcacc00640 1 -- 172.21.15.5:0/1697517840 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdca000cd80 con 0x7fdcb0103e50 2024-09-10T09:41:39.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:39.516+0000 7fdcacc00640 1 -- 172.21.15.5:0/1697517840 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdca0010910 con 0x7fdcb0103e50 2024-09-10T09:41:39.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:39.516+0000 7fdcb5c00640 1 -- 172.21.15.5:0/1697517840 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdcb019f870 con 0x7fdcb0103e50 2024-09-10T09:41:39.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:39.516+0000 7fdcacc00640 1 -- 172.21.15.5:0/1697517840 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdca0018990 con 0x7fdcb0103e50 2024-09-10T09:41:39.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:39.516+0000 7fdcb5c00640 1 -- 172.21.15.5:0/1697517840 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdcb01a7150 con 0x7fdcb0103e50 2024-09-10T09:41:39.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:39.519+0000 7fdcacc00640 1 -- 172.21.15.5:0/1697517840 <== mon.2 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7fdca0010ac0 con 0x7fdcb0103e50 2024-09-10T09:41:39.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:39.519+0000 7fdcb9be2740 1 Processor -- start 2024-09-10T09:41:39.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:39.519+0000 7fdcb9be2740 1 -- start start 2024-09-10T09:41:39.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:39.520+0000 7fdcb9be2740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564d12226e40 0x564d12227220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:41:39.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:39.520+0000 7fdcb9be2740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564d12227760 0x564d12231ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:41:39.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:39.520+0000 7fdcb9be2740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564d12232020 0x564d12234410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:41:39.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:39.520+0000 7fdcb9be2740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x564d1218d3e0 con 0x564d12226e40 2024-09-10T09:41:39.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:39.520+0000 7fdcb9be2740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x564d12134d80 con 0x564d12227760 2024-09-10T09:41:39.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:39.520+0000 7fdcb9be2740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x564d121893a0 con 0x564d12232020 2024-09-10T09:41:39.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:39.520+0000 7fdcafe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564d12227760 0x564d12231ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:41:39.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:39.520+0000 7fdcaf400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564d12232020 0x564d12234410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:41:39.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:39.520+0000 7fdcaf400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564d12232020 0x564d12234410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:42194/0 (socket says 172.21.15.5:42194) 2024-09-10T09:41:39.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:39.520+0000 7fdcaf400640 1 -- 172.21.15.5:0/66619638 learned_addr learned my addr 172.21.15.5:0/66619638 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:41:39.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:39.520+0000 7fdcb5200640 1 --2- 172.21.15.5:0/66619638 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564d12226e40 0x564d12227220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:41:39.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:39.520+0000 7fdcafe00640 1 -- 172.21.15.5:0/66619638 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564d12232020 msgr2=0x564d12234410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:41:39.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:39.520+0000 7fdcafe00640 1 --2- 172.21.15.5:0/66619638 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564d12232020 0x564d12234410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:41:39.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:39.520+0000 7fdcafe00640 1 -- 172.21.15.5:0/66619638 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564d12226e40 msgr2=0x564d12227220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:41:39.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:39.520+0000 7fdcafe00640 1 --2- 172.21.15.5:0/66619638 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564d12226e40 0x564d12227220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:41:39.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:39.520+0000 7fdcafe00640 1 -- 172.21.15.5:0/66619638 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564d12153ae0 con 0x564d12227760 2024-09-10T09:41:39.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:39.521+0000 7fdcacc00640 1 --2- 172.21.15.5:0/1697517840 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fdc940a2c00 0x7fdc940a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:41:39.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:39.521+0000 7fdcafe00640 1 --2- 172.21.15.5:0/66619638 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564d12227760 0x564d12231ae0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fdca00cce00 tx=0x7fdca00d7670 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2c90d312fde27ac server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:41:39.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:39.521+0000 7fdcacc00640 1 -- 172.21.15.5:0/1697517840 <== mon.2 v2:172.21.15.102:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fdca00cf7b0 con 0x7fdcb0103e50 2024-09-10T09:41:39.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:39.521+0000 7fdca5600640 1 -- 172.21.15.5:0/66619638 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdca0097e60 con 0x564d12227760 2024-09-10T09:41:39.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:39.521+0000 7fdca5600640 1 -- 172.21.15.5:0/66619638 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdca009c2c0 con 0x564d12227760 2024-09-10T09:41:39.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:39.521+0000 7fdca5600640 1 -- 172.21.15.5:0/66619638 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdca009b520 con 0x564d12227760 2024-09-10T09:41:39.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:39.521+0000 7fdcb5200640 1 --2- 172.21.15.5:0/1697517840 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fdc940a2c00 0x7fdc940a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:41:39.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:39.522+0000 7fdcb9be2740 1 -- 172.21.15.5:0/66619638 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564d12227760 msgr2=0x564d12231ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:41:39.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:39.522+0000 7fdcb9be2740 1 --2- 172.21.15.5:0/66619638 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564d12227760 0x564d12231ae0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fdca00cce00 tx=0x7fdca00d7670 comp rx=0 tx=0).stop 2024-09-10T09:41:39.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:39.522+0000 7fdcb5200640 1 --2- 172.21.15.5:0/1697517840 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fdc940a2c00 0x7fdc940a50c0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fdca8005fd0 tx=0x7fdca8009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:41:39.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:39.522+0000 7fdcb9be2740 1 -- 172.21.15.5:0/66619638 shutdown_connections 2024-09-10T09:41:39.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:39.522+0000 7fdcb9be2740 1 --2- 172.21.15.5:0/66619638 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564d12232020 0x564d12234410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:41:39.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:39.522+0000 7fdcb9be2740 1 --2- 172.21.15.5:0/66619638 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564d12227760 0x564d12231ae0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:41:39.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:39.522+0000 7fdcb9be2740 1 --2- 172.21.15.5:0/66619638 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564d12226e40 0x564d12227220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:41:39.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:39.522+0000 7fdcb9be2740 1 -- 172.21.15.5:0/66619638 >> 172.21.15.5:0/66619638 conn(0x564d122233d0 msgr2=0x564d122237b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:41:39.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:39.522+0000 7fdcb9be2740 1 -- 172.21.15.5:0/66619638 shutdown_connections 2024-09-10T09:41:39.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:39.522+0000 7fdcb9be2740 1 -- 172.21.15.5:0/66619638 wait complete. 2024-09-10T09:41:39.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:39.523+0000 7fdcb9be2740 1 -- start start 2024-09-10T09:41:39.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:39.523+0000 7fdcb9be2740 1 Processor -- start 2024-09-10T09:41:39.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:39.524+0000 7fdcb9be2740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564d12226e40 0x564d1222ef00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:41:39.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:39.524+0000 7fdcb9be2740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564d12227760 0x564d1222f440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:41:39.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:39.524+0000 7fdcb9be2740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564d12232020 0x564d1222f980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:41:39.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:39.524+0000 7fdcb9be2740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x564d12234bf0 con 0x564d12232020 2024-09-10T09:41:39.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:39.524+0000 7fdcb9be2740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x564d121480f0 con 0x564d12226e40 2024-09-10T09:41:39.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:39.524+0000 7fdcb9be2740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x564d12234d70 con 0x564d12227760 2024-09-10T09:41:39.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:39.524+0000 7fdcb9be2740 10 client.0 _ll_register_callbacks cb 0x564d1212fbd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:41:39.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:39.524+0000 7fdcafe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564d12226e40 0x564d1222ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:41:39.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:39.524+0000 7fdcafe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564d12226e40 0x564d1222ef00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:35908/0 (socket says 172.21.15.5:35908) 2024-09-10T09:41:39.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:39.524+0000 7fdcafe00640 1 -- 172.21.15.5:0/456635115 learned_addr learned my addr 172.21.15.5:0/456635115 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:41:39.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:39.524+0000 7fdcb5200640 1 --2- 172.21.15.5:0/456635115 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564d12232020 0x564d1222f980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:41:39.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:39.524+0000 7fdcaf400640 1 --2- 172.21.15.5:0/456635115 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564d12227760 0x564d1222f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:41:39.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:39.524+0000 7fdcafe00640 1 -- 172.21.15.5:0/456635115 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564d12227760 msgr2=0x564d1222f440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:41:39.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:39.524+0000 7fdcafe00640 1 --2- 172.21.15.5:0/456635115 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564d12227760 0x564d1222f440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:41:39.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:39.524+0000 7fdcafe00640 1 -- 172.21.15.5:0/456635115 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564d12232020 msgr2=0x564d1222f980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:41:39.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:39.524+0000 7fdcafe00640 1 --2- 172.21.15.5:0/456635115 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564d12232020 0x564d1222f980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:41:39.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:39.524+0000 7fdcafe00640 1 -- 172.21.15.5:0/456635115 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564d11efe3c0 con 0x564d12226e40 2024-09-10T09:41:39.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:39.524+0000 7fdcaf400640 1 --2- 172.21.15.5:0/456635115 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564d12227760 0x564d1222f440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:41:39.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:39.525+0000 7fdcafe00640 1 --2- 172.21.15.5:0/456635115 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564d12226e40 0x564d1222ef00 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fdca00cce30 tx=0x7fdca00d5300 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5c1396bb23cce9bf server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:41:39.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:39.525+0000 7fdc9a000640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-09-10T09:41:39.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:39.525+0000 7fdc9a000640 1 -- 172.21.15.5:0/456635115 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdca009b700 con 0x564d12226e40 2024-09-10T09:41:39.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:39.525+0000 7fdc9a000640 1 -- 172.21.15.5:0/456635115 <== mon.2 v2:172.21.15.102:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fdca00cf460 con 0x564d12226e40 2024-09-10T09:41:39.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:39.525+0000 7fdcb9be2740 1 -- 172.21.15.5:0/456635115 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdca009c2c0 con 0x564d12226e40 2024-09-10T09:41:39.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:39.525+0000 7fdc9a000640 1 -- 172.21.15.5:0/456635115 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fdca009b990 con 0x564d12226e40 2024-09-10T09:41:39.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:39.525+0000 7fdcb9be2740 10 client.26012 fetch_fsmap learned FSMap version 32 2024-09-10T09:41:39.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:39.525+0000 7fdcb9be2740 1 -- 172.21.15.5:0/456635115 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x564d12234bf0 con 0x564d12226e40 2024-09-10T09:41:39.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:39.525+0000 7fdc9a000640 1 -- 172.21.15.5:0/456635115 <== mon.2 v2:172.21.15.102:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdca009c2c0 con 0x564d12226e40 2024-09-10T09:41:39.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:39.526+0000 7fdc9a000640 1 -- 172.21.15.5:0/456635115 <== mon.2 v2:172.21.15.102:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7fdca00d5cb0 con 0x564d12226e40 2024-09-10T09:41:39.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:39.526+0000 7fdcb9be2740 10 client.26012 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T09:41:39.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:39.526+0000 7fdcb9be2740 10 client.26012 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-09-10T09:41:39.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:39.526+0000 7fdcb9be2740 20 client.26012 populate_metadata read hostname 'smithi005' 2024-09-10T09:41:39.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:39.526+0000 7fdcb9be2740 1 --2- 172.21.15.5:0/456635115 >> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] conn(0x564d12241de0 0x564d122441d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:41:39.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:39.526+0000 7fdcb9be2740 4 client.26012 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:41:39.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:39.526+0000 7fdcb9be2740 1 -- 172.21.15.5:0/456635115 --> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564d1219a950 con 0x564d12241de0 2024-09-10T09:41:39.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:39.526+0000 7fdcaf400640 1 --2- 172.21.15.5:0/456635115 >> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] conn(0x564d12241de0 0x564d122441d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:41:39.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:39.527+0000 7fdcaf400640 1 --2- 172.21.15.5:0/456635115 >> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] conn(0x564d12241de0 0x564d122441d0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=38c301cbb24aa67e server_cookie=723878b76a1a88f3 in_seq=0 out_seq=0 2024-09-10T09:41:39.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:39.527+0000 7fdc9a000640 10 client.26012 ms_handle_connect on v2:172.21.15.102:6836/2689776764 2024-09-10T09:41:39.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:39.528+0000 7fdc9a000640 1 -- 172.21.15.5:0/456635115 <== mds.3 v2:172.21.15.102:6836/2689776764 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564d1219a950 con 0x564d12241de0 2024-09-10T09:41:39.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:39.528+0000 7fdc9a000640 10 client.26012 handle_command_reply: tid=0 2024-09-10T09:41:39.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:39.528+0000 7fdcb9be2740 1 client.26012 shutdown 2024-09-10T09:41:39.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:39.528+0000 7fdcb9be2740 1 -- 172.21.15.5:0/456635115 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564d12226e40 msgr2=0x564d1222ef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:41:39.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:39.528+0000 7fdcb9be2740 1 --2- 172.21.15.5:0/456635115 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564d12226e40 0x564d1222ef00 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fdca00cce30 tx=0x7fdca00d5300 comp rx=0 tx=0).stop 2024-09-10T09:41:39.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:39.528+0000 7fdcb9be2740 1 -- 172.21.15.5:0/456635115 shutdown_connections 2024-09-10T09:41:39.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:39.528+0000 7fdcb9be2740 1 --2- 172.21.15.5:0/456635115 >> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] conn(0x564d12241de0 0x564d122441d0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:41:39.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:39.528+0000 7fdcb9be2740 1 --2- 172.21.15.5:0/456635115 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564d12232020 0x564d1222f980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:41:39.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:39.528+0000 7fdcb9be2740 1 --2- 172.21.15.5:0/456635115 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564d12227760 0x564d1222f440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:41:39.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:39.528+0000 7fdc9a000640 0 client.26012 ms_handle_reset on v2:172.21.15.102:6836/2689776764 2024-09-10T09:41:39.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:39.528+0000 7fdcb9be2740 1 --2- 172.21.15.5:0/456635115 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564d12226e40 0x564d1222ef00 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:41:39.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:39.528+0000 7fdcb9be2740 1 -- 172.21.15.5:0/456635115 >> 172.21.15.5:0/456635115 conn(0x564d122233d0 msgr2=0x564d12229100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:41:39.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:39.528+0000 7fdcb9be2740 1 -- 172.21.15.5:0/456635115 shutdown_connections 2024-09-10T09:41:39.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:39.528+0000 7fdcb9be2740 1 -- 172.21.15.5:0/456635115 wait complete. 2024-09-10T09:41:39.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:39.528+0000 7fdcb9be2740 20 client.26012 trim_cache size 0 max 16384 2024-09-10T09:41:39.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:39.542+0000 7fdcb9be2740 1 Processor -- start 2024-09-10T09:41:39.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:39.543+0000 7fdcb9be2740 1 -- start start 2024-09-10T09:41:39.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:39.543+0000 7fdcb9be2740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564d12226e40 0x564d12231230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:41:39.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:39.543+0000 7fdcb9be2740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564d12227760 0x564d1222e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:41:39.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:39.543+0000 7fdcb9be2740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564d12232020 0x564d12249960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:41:39.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:39.543+0000 7fdcb9be2740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fdcb01b0f10 con 0x564d12227760 2024-09-10T09:41:39.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:39.543+0000 7fdcb9be2740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fdcb01b2b10 con 0x564d12226e40 2024-09-10T09:41:39.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:39.543+0000 7fdcb9be2740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fdcb01b15e0 con 0x564d12232020 2024-09-10T09:41:39.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:39.543+0000 7fdcafe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564d12226e40 0x564d12231230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:41:39.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:39.543+0000 7fdcaf400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564d12227760 0x564d1222e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:41:39.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:39.543+0000 7fdcafe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564d12226e40 0x564d12231230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:35910/0 (socket says 172.21.15.5:35910) 2024-09-10T09:41:39.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:39.543+0000 7fdcafe00640 1 -- 172.21.15.5:0/4136112795 learned_addr learned my addr 172.21.15.5:0/4136112795 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:41:39.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:39.543+0000 7fdcb5200640 1 --2- 172.21.15.5:0/4136112795 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564d12232020 0x564d12249960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:41:39.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:39.544+0000 7fdcafe00640 1 -- 172.21.15.5:0/4136112795 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564d12232020 msgr2=0x564d12249960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:41:39.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:39.544+0000 7fdcafe00640 1 --2- 172.21.15.5:0/4136112795 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564d12232020 0x564d12249960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:41:39.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:39.544+0000 7fdcafe00640 1 -- 172.21.15.5:0/4136112795 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564d12227760 msgr2=0x564d1222e7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:41:39.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:39.544+0000 7fdcafe00640 1 --2- 172.21.15.5:0/4136112795 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564d12227760 0x564d1222e7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:41:39.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:39.544+0000 7fdcafe00640 1 -- 172.21.15.5:0/4136112795 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564d12223200 con 0x564d12226e40 2024-09-10T09:41:39.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:39.544+0000 7fdcafe00640 1 --2- 172.21.15.5:0/4136112795 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564d12226e40 0x564d12231230 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fdca0098590 tx=0x7fdca00d5370 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b4455a4c994405b7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:41:39.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:39.544+0000 7fdca4c00640 1 -- 172.21.15.5:0/4136112795 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdca00a8020 con 0x564d12226e40 2024-09-10T09:41:39.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:39.544+0000 7fdca4c00640 1 -- 172.21.15.5:0/4136112795 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdca00d7e70 con 0x564d12226e40 2024-09-10T09:41:39.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:39.544+0000 7fdca4c00640 1 -- 172.21.15.5:0/4136112795 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdca00b2020 con 0x564d12226e40 2024-09-10T09:41:39.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:39.545+0000 7fdcb9be2740 1 -- 172.21.15.5:0/4136112795 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564d12226e40 msgr2=0x564d12231230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:41:39.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:39.545+0000 7fdcb9be2740 1 --2- 172.21.15.5:0/4136112795 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564d12226e40 0x564d12231230 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fdca0098590 tx=0x7fdca00d5370 comp rx=0 tx=0).stop 2024-09-10T09:41:39.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:39.545+0000 7fdcb9be2740 1 -- 172.21.15.5:0/4136112795 shutdown_connections 2024-09-10T09:41:39.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:39.545+0000 7fdcb9be2740 1 --2- 172.21.15.5:0/4136112795 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564d12232020 0x564d12249960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:41:39.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:39.545+0000 7fdcb9be2740 1 --2- 172.21.15.5:0/4136112795 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564d12227760 0x564d1222e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:41:39.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:39.545+0000 7fdcb9be2740 1 --2- 172.21.15.5:0/4136112795 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564d12226e40 0x564d12231230 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:41:39.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:39.545+0000 7fdcb9be2740 1 -- 172.21.15.5:0/4136112795 >> 172.21.15.5:0/4136112795 conn(0x564d122233d0 msgr2=0x564d12222490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:41:39.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:39.545+0000 7fdcb9be2740 1 -- 172.21.15.5:0/4136112795 shutdown_connections 2024-09-10T09:41:39.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:39.545+0000 7fdcb9be2740 1 -- 172.21.15.5:0/4136112795 wait complete. 2024-09-10T09:41:39.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:39.545+0000 7fdcb9be2740 1 -- start start 2024-09-10T09:41:39.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:39.545+0000 7fdcb9be2740 1 Processor -- start 2024-09-10T09:41:39.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:39.546+0000 7fdcb9be2740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564d12226e40 0x564d1223a560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:41:39.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:39.546+0000 7fdcb9be2740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564d12227760 0x564d12235890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:41:39.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:39.546+0000 7fdcb9be2740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564d12232020 0x564d12235dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:41:39.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:39.546+0000 7fdcb9be2740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fdcb01b2380 con 0x564d12232020 2024-09-10T09:41:39.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:39.546+0000 7fdcb9be2740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fdcb01b1cb0 con 0x564d12226e40 2024-09-10T09:41:39.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:39.546+0000 7fdcb9be2740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fdcb01aeef0 con 0x564d12227760 2024-09-10T09:41:39.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:39.546+0000 7fdcb9be2740 10 client.0 _ll_register_callbacks cb 0x7fdca80099c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:41:39.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:39.546+0000 7fdcafe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564d12226e40 0x564d1223a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:41:39.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:39.546+0000 7fdcb5200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564d12232020 0x564d12235dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:41:39.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:39.546+0000 7fdcafe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564d12226e40 0x564d1223a560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:35922/0 (socket says 172.21.15.5:35922) 2024-09-10T09:41:39.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:39.546+0000 7fdcafe00640 1 -- 172.21.15.5:0/2316175218 learned_addr learned my addr 172.21.15.5:0/2316175218 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:41:39.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:39.546+0000 7fdcaf400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564d12227760 0x564d12235890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:41:39.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:39.547+0000 7fdcafe00640 1 -- 172.21.15.5:0/2316175218 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564d12227760 msgr2=0x564d12235890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:41:39.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:39.547+0000 7fdcafe00640 1 --2- 172.21.15.5:0/2316175218 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564d12227760 0x564d12235890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:41:39.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:39.547+0000 7fdcafe00640 1 -- 172.21.15.5:0/2316175218 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564d12232020 msgr2=0x564d12235dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:41:39.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:39.547+0000 7fdcafe00640 1 --2- 172.21.15.5:0/2316175218 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564d12232020 0x564d12235dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:41:39.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:39.547+0000 7fdcafe00640 1 -- 172.21.15.5:0/2316175218 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564d12244730 con 0x564d12226e40 2024-09-10T09:41:39.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:39.547+0000 7fdcafe00640 1 --2- 172.21.15.5:0/2316175218 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564d12226e40 0x564d1223a560 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fdca00d5b40 tx=0x7fdca00a4830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f872eefe2189220d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:41:39.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:39.547+0000 7fdc9a000640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-09-10T09:41:39.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:39.547+0000 7fdc9a000640 1 -- 172.21.15.5:0/2316175218 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdca00a8070 con 0x564d12226e40 2024-09-10T09:41:39.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:39.547+0000 7fdc9a000640 1 -- 172.21.15.5:0/2316175218 <== mon.2 v2:172.21.15.102:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fdca00cf460 con 0x564d12226e40 2024-09-10T09:41:39.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:39.547+0000 7fdcb9be2740 1 -- 172.21.15.5:0/2316175218 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdca00d7e70 con 0x564d12226e40 2024-09-10T09:41:39.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:39.547+0000 7fdc9a000640 1 -- 172.21.15.5:0/2316175218 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fdca00b2070 con 0x564d12226e40 2024-09-10T09:41:39.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:39.547+0000 7fdcb9be2740 10 client.26018 fetch_fsmap learned FSMap version 32 2024-09-10T09:41:39.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:39.547+0000 7fdcb9be2740 1 -- 172.21.15.5:0/2316175218 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x564d12244730 con 0x564d12226e40 2024-09-10T09:41:39.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:39.548+0000 7fdc9a000640 1 -- 172.21.15.5:0/2316175218 <== mon.2 v2:172.21.15.102:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdca00d7e70 con 0x564d12226e40 2024-09-10T09:41:39.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:39.548+0000 7fdc9a000640 1 -- 172.21.15.5:0/2316175218 <== mon.2 v2:172.21.15.102:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7fdca00a43a0 con 0x564d12226e40 2024-09-10T09:41:39.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:39.548+0000 7fdcb9be2740 10 client.26018 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T09:41:39.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:39.548+0000 7fdcb9be2740 10 client.26018 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-09-10T09:41:39.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:39.548+0000 7fdcb9be2740 20 client.26018 populate_metadata read hostname 'smithi005' 2024-09-10T09:41:39.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:39.548+0000 7fdcb9be2740 1 --2- 172.21.15.5:0/2316175218 >> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] conn(0x564d122ac8c0 0x564d122aecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:41:39.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:39.548+0000 7fdcb9be2740 4 client.26018 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T09:41:39.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:39.548+0000 7fdcb9be2740 1 -- 172.21.15.5:0/2316175218 --> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564d12194e60 con 0x564d122ac8c0 2024-09-10T09:41:39.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:39.548+0000 7fdcaf400640 1 --2- 172.21.15.5:0/2316175218 >> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] conn(0x564d122ac8c0 0x564d122aecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:41:39.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:39.549+0000 7fdcaf400640 1 --2- 172.21.15.5:0/2316175218 >> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] conn(0x564d122ac8c0 0x564d122aecb0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=373d75966af0bcf7 server_cookie=e63f8f2914a51b2a in_seq=0 out_seq=0 2024-09-10T09:41:39.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:39.549+0000 7fdc9a000640 10 client.26018 ms_handle_connect on v2:172.21.15.102:6836/2689776764 2024-09-10T09:41:39.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:39.549+0000 7fdc9a000640 1 -- 172.21.15.5:0/2316175218 <== mds.3 v2:172.21.15.102:6836/2689776764 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564d12194e60 con 0x564d122ac8c0 2024-09-10T09:41:39.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:39.549+0000 7fdc9a000640 10 client.26018 handle_command_reply: tid=0 2024-09-10T09:41:39.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:39.549+0000 7fdcb9be2740 1 client.26018 shutdown 2024-09-10T09:41:39.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:39.549+0000 7fdcb9be2740 1 -- 172.21.15.5:0/2316175218 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564d12226e40 msgr2=0x564d1223a560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:41:39.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:39.549+0000 7fdcb9be2740 1 --2- 172.21.15.5:0/2316175218 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564d12226e40 0x564d1223a560 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fdca00d5b40 tx=0x7fdca00a4830 comp rx=0 tx=0).stop 2024-09-10T09:41:39.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:39.549+0000 7fdcb9be2740 1 -- 172.21.15.5:0/2316175218 shutdown_connections 2024-09-10T09:41:39.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:39.549+0000 7fdcb9be2740 1 --2- 172.21.15.5:0/2316175218 >> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] conn(0x564d122ac8c0 0x564d122aecb0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:41:39.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:39.549+0000 7fdcb9be2740 1 --2- 172.21.15.5:0/2316175218 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564d12232020 0x564d12235dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:41:39.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:39.549+0000 7fdcb9be2740 1 --2- 172.21.15.5:0/2316175218 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564d12227760 0x564d12235890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:41:39.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:39.549+0000 7fdcb9be2740 1 --2- 172.21.15.5:0/2316175218 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564d12226e40 0x564d1223a560 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:41:39.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:39.549+0000 7fdc9a000640 0 client.26018 ms_handle_reset on v2:172.21.15.102:6836/2689776764 2024-09-10T09:41:39.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:39.549+0000 7fdcb9be2740 1 -- 172.21.15.5:0/2316175218 >> 172.21.15.5:0/2316175218 conn(0x564d122233d0 msgr2=0x564d1222f690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:41:39.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:39.549+0000 7fdcb9be2740 1 -- 172.21.15.5:0/2316175218 shutdown_connections 2024-09-10T09:41:39.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:39.550+0000 7fdcb9be2740 1 -- 172.21.15.5:0/2316175218 wait complete. 2024-09-10T09:41:39.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:39.550+0000 7fdcb9be2740 20 client.26018 trim_cache size 0 max 16384 2024-09-10T09:41:39.559 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-10T09:41:39.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:39.550+0000 7fdcb5c00640 1 -- 172.21.15.5:0/1697517840 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fdc940a2c00 msgr2=0x7fdc940a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:41:39.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:39.550+0000 7fdcb5c00640 1 --2- 172.21.15.5:0/1697517840 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fdc940a2c00 0x7fdc940a50c0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fdca8005fd0 tx=0x7fdca8009500 comp rx=0 tx=0).stop 2024-09-10T09:41:39.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:39.550+0000 7fdcb5c00640 1 -- 172.21.15.5:0/1697517840 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdcb0103e50 msgr2=0x7fdcb019a5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:41:39.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:39.550+0000 7fdcb5c00640 1 --2- 172.21.15.5:0/1697517840 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdcb0103e50 0x7fdcb019a5d0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fdca000e9f0 tx=0x7fdca000eeb0 comp rx=0 tx=0).stop 2024-09-10T09:41:39.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:39.551+0000 7fdcb5c00640 1 -- 172.21.15.5:0/1697517840 shutdown_connections 2024-09-10T09:41:39.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:39.551+0000 7fdcb5c00640 1 --2- 172.21.15.5:0/1697517840 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fdc940a2c00 0x7fdc940a50c0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:41:39.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:39.551+0000 7fdcb5c00640 1 --2- 172.21.15.5:0/1697517840 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdcb0109090 0x7fdcb019eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:41:39.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:39.551+0000 7fdcb5c00640 1 --2- 172.21.15.5:0/1697517840 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdcb0104800 0x7fdcb019ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:41:39.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:39.551+0000 7fdcb5c00640 1 --2- 172.21.15.5:0/1697517840 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdcb0103e50 0x7fdcb019a5d0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:41:39.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:39.551+0000 7fdcb5c00640 1 -- 172.21.15.5:0/1697517840 >> 172.21.15.5:0/1697517840 conn(0x7fdcb00ff8b0 msgr2=0x7fdcb0100bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:41:39.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:39.551+0000 7fdcb5c00640 1 -- 172.21.15.5:0/1697517840 shutdown_connections 2024-09-10T09:41:39.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:39.551+0000 7fdcb5c00640 1 -- 172.21.15.5:0/1697517840 wait complete. 2024-09-10T09:41:39.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:41:39 smithi005 ceph-mon[30350]: pgmap v492: 129 pgs: 129 active+clean; 74 MiB data, 694 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.6 MiB/s wr, 180 op/s 2024-09-10T09:41:39.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:41:39 smithi159 ceph-mon[35513]: pgmap v492: 129 pgs: 129 active+clean; 74 MiB data, 694 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.6 MiB/s wr, 180 op/s 2024-09-10T09:41:39.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:41:39 smithi102 ceph-mon[35442]: pgmap v492: 129 pgs: 129 active+clean; 74 MiB data, 694 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.6 MiB/s wr, 180 op/s 2024-09-10T09:41:40.001 INFO:tasks.workunit.client.0.smithi005.stdout: 1 134274 9.15 MB/sec warmup 81 sec latency 88.633 ms 2024-09-10T09:41:40.460 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-10T09:41:40.865 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:41:41.001 INFO:tasks.workunit.client.0.smithi005.stdout: 1 136308 9.18 MB/sec warmup 82 sec latency 23.526 ms 2024-09-10T09:41:41.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:41:41 smithi005 ceph-mon[30350]: pgmap v493: 129 pgs: 129 active+clean; 75 MiB data, 702 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.0 MiB/s wr, 228 op/s 2024-09-10T09:41:41.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:41:41 smithi159 ceph-mon[35513]: pgmap v493: 129 pgs: 129 active+clean; 75 MiB data, 702 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.0 MiB/s wr, 228 op/s 2024-09-10T09:41:41.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:41:41 smithi102 ceph-mon[35442]: pgmap v493: 129 pgs: 129 active+clean; 75 MiB data, 702 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.0 MiB/s wr, 228 op/s 2024-09-10T09:41:42.012 INFO:tasks.workunit.client.0.smithi005.stdout: 1 136313 9.07 MB/sec warmup 83 sec latency 782.216 ms 2024-09-10T09:41:42.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:42.848+0000 7f41bd400640 1 -- 172.21.15.5:0/29087711 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f41b8104010 msgr2=0x7f41b81043f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:41:42.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:42.848+0000 7f41bd400640 1 --2- 172.21.15.5:0/29087711 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f41b8104010 0x7f41b81043f0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f41a8009ca0 tx=0x7f41a802a8c0 comp rx=0 tx=0).stop 2024-09-10T09:41:42.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:42.848+0000 7f41bd400640 1 -- 172.21.15.5:0/29087711 shutdown_connections 2024-09-10T09:41:42.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:42.848+0000 7f41bd400640 1 --2- 172.21.15.5:0/29087711 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f41b8109140 0x7f41b8110cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:41:42.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:42.848+0000 7f41bd400640 1 --2- 172.21.15.5:0/29087711 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f41b81049c0 0x7f41b8108a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:41:42.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:42.848+0000 7f41bd400640 1 --2- 172.21.15.5:0/29087711 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f41b8104010 0x7f41b81043f0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:41:42.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:42.848+0000 7f41bd400640 1 -- 172.21.15.5:0/29087711 >> 172.21.15.5:0/29087711 conn(0x7f41b80ffa50 msgr2=0x7f41b8101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:41:42.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:42.848+0000 7f41bd400640 1 -- 172.21.15.5:0/29087711 shutdown_connections 2024-09-10T09:41:42.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:42.848+0000 7f41bd400640 1 -- 172.21.15.5:0/29087711 wait complete. 2024-09-10T09:41:42.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:42.849+0000 7f41bd400640 1 Processor -- start 2024-09-10T09:41:42.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:42.849+0000 7f41bd400640 1 -- start start 2024-09-10T09:41:42.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:42.850+0000 7f41bd400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f41b8104010 0x7f41b819a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:41:42.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:42.850+0000 7f41bd400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f41b81049c0 0x7f41b819ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:41:42.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:42.850+0000 7f41bd400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f41b8109140 0x7f41b819ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:41:42.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:42.850+0000 7f41bd400640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f41b81141b0 con 0x7f41b8104010 2024-09-10T09:41:42.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:42.850+0000 7f41bd400640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f41b8114030 con 0x7f41b8109140 2024-09-10T09:41:42.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:42.850+0000 7f41bd400640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f41b8114330 con 0x7f41b81049c0 2024-09-10T09:41:42.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:42.850+0000 7f41b6800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f41b8104010 0x7f41b819a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:41:42.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:42.850+0000 7f41ade00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f41b81049c0 0x7f41b819ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:41:42.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:42.850+0000 7f41b6800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f41b8104010 0x7f41b819a630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33634/0 (socket says 172.21.15.5:33634) 2024-09-10T09:41:42.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:42.850+0000 7f41b6800640 1 -- 172.21.15.5:0/900229068 learned_addr learned my addr 172.21.15.5:0/900229068 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:41:42.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:42.850+0000 7f41b7200640 1 --2- 172.21.15.5:0/900229068 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f41b8109140 0x7f41b819ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:41:42.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:42.851+0000 7f41ade00640 1 -- 172.21.15.5:0/900229068 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f41b8109140 msgr2=0x7f41b819ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:41:42.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:42.851+0000 7f41ade00640 1 --2- 172.21.15.5:0/900229068 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f41b8109140 0x7f41b819ef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:41:42.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:42.851+0000 7f41ade00640 1 -- 172.21.15.5:0/900229068 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f41b8104010 msgr2=0x7f41b819a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:41:42.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:42.851+0000 7f41ade00640 1 --2- 172.21.15.5:0/900229068 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f41b8104010 0x7f41b819a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:41:42.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:42.851+0000 7f41ade00640 1 -- 172.21.15.5:0/900229068 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41b819f5e0 con 0x7f41b81049c0 2024-09-10T09:41:42.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:42.851+0000 7f41ade00640 1 --2- 172.21.15.5:0/900229068 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f41b81049c0 0x7f41b819ab70 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f41a000e9f0 tx=0x7f41a000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:41:42.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:42.851+0000 7f41afe00640 1 -- 172.21.15.5:0/900229068 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41a000ccc0 con 0x7f41b81049c0 2024-09-10T09:41:42.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:42.851+0000 7f41afe00640 1 -- 172.21.15.5:0/900229068 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f41a0005020 con 0x7f41b81049c0 2024-09-10T09:41:42.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:42.851+0000 7f41afe00640 1 -- 172.21.15.5:0/900229068 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41a00049e0 con 0x7f41b81049c0 2024-09-10T09:41:42.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:42.851+0000 7f41bd400640 1 -- 172.21.15.5:0/900229068 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41b819f8d0 con 0x7f41b81049c0 2024-09-10T09:41:42.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:42.852+0000 7f41bd400640 1 -- 172.21.15.5:0/900229068 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41b81a71b0 con 0x7f41b81049c0 2024-09-10T09:41:42.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:42.855+0000 7f41bfe7a740 1 Processor -- start 2024-09-10T09:41:42.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:42.855+0000 7f41bfe7a740 1 -- start start 2024-09-10T09:41:42.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:42.855+0000 7f41bfe7a740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e015a3fc70 0x55e015a40050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:41:42.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:42.855+0000 7f41bfe7a740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e015a40590 0x55e015a4a910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:41:42.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:42.855+0000 7f41bfe7a740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e015a4ae50 0x55e015a4d240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:41:42.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:42.855+0000 7f41bfe7a740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55e0159a63e0 con 0x55e015a4ae50 2024-09-10T09:41:42.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:42.855+0000 7f41bfe7a740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55e01594dd80 con 0x55e015a40590 2024-09-10T09:41:42.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:42.855+0000 7f41bfe7a740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55e0159a23a0 con 0x55e015a3fc70 2024-09-10T09:41:42.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:42.855+0000 7f41b7200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e015a3fc70 0x55e015a40050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:41:42.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:42.855+0000 7f41b7200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e015a3fc70 0x55e015a40050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:50598/0 (socket says 172.21.15.5:50598) 2024-09-10T09:41:42.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:42.855+0000 7f41b7200640 1 -- 172.21.15.5:0/4160805326 learned_addr learned my addr 172.21.15.5:0/4160805326 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:41:42.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:42.855+0000 7f41b6800640 1 --2- 172.21.15.5:0/4160805326 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e015a40590 0x55e015a4a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:41:42.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:42.856+0000 7f41afe00640 1 -- 172.21.15.5:0/900229068 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f41a0005560 con 0x7f41b81049c0 2024-09-10T09:41:42.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:42.856+0000 7f41b7200640 1 -- 172.21.15.5:0/4160805326 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e015a40590 msgr2=0x55e015a4a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:41:42.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:42.856+0000 7f41b7200640 1 --2- 172.21.15.5:0/4160805326 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e015a40590 0x55e015a4a910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:41:42.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:42.856+0000 7f41b7200640 1 -- 172.21.15.5:0/4160805326 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e015a4ae50 msgr2=0x55e015a4d240 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T09:41:42.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:42.856+0000 7f41b7200640 1 --2- 172.21.15.5:0/4160805326 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e015a4ae50 0x55e015a4d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:41:42.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:42.856+0000 7f41b7200640 1 -- 172.21.15.5:0/4160805326 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e01596cae0 con 0x55e015a3fc70 2024-09-10T09:41:42.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:42.856+0000 7f41b7200640 1 --2- 172.21.15.5:0/4160805326 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e015a3fc70 0x55e015a40050 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f41b0009c20 tx=0x7f41b0011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c752f30d105d45b4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:41:42.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:42.857+0000 7f41a7400640 1 -- 172.21.15.5:0/4160805326 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41b001e070 con 0x55e015a3fc70 2024-09-10T09:41:42.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:42.857+0000 7f41a7400640 1 -- 172.21.15.5:0/4160805326 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f41b0019b10 con 0x55e015a3fc70 2024-09-10T09:41:42.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:42.857+0000 7f41a7400640 1 -- 172.21.15.5:0/4160805326 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41b000eae0 con 0x55e015a3fc70 2024-09-10T09:41:42.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:42.857+0000 7f41afe00640 1 --2- 172.21.15.5:0/900229068 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f419c0a2c00 0x7f419c0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:41:42.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:42.857+0000 7f41bfe7a740 1 -- 172.21.15.5:0/4160805326 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e015a3fc70 msgr2=0x55e015a40050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:41:42.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:42.857+0000 7f41bfe7a740 1 --2- 172.21.15.5:0/4160805326 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e015a3fc70 0x55e015a40050 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f41b0009c20 tx=0x7f41b0011810 comp rx=0 tx=0).stop 2024-09-10T09:41:42.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:42.857+0000 7f41afe00640 1 -- 172.21.15.5:0/900229068 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f41a00cf3a0 con 0x7f41b81049c0 2024-09-10T09:41:42.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:42.857+0000 7f41bfe7a740 1 -- 172.21.15.5:0/4160805326 shutdown_connections 2024-09-10T09:41:42.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:42.857+0000 7f41bfe7a740 1 --2- 172.21.15.5:0/4160805326 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e015a4ae50 0x55e015a4d240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:41:42.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:42.857+0000 7f41bfe7a740 1 --2- 172.21.15.5:0/4160805326 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e015a40590 0x55e015a4a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:41:42.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:42.857+0000 7f41bfe7a740 1 --2- 172.21.15.5:0/4160805326 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e015a3fc70 0x55e015a40050 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:41:42.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:42.857+0000 7f41bfe7a740 1 -- 172.21.15.5:0/4160805326 >> 172.21.15.5:0/4160805326 conn(0x55e015a3c200 msgr2=0x55e015a3c5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:41:42.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:42.858+0000 7f41bfe7a740 1 -- 172.21.15.5:0/4160805326 shutdown_connections 2024-09-10T09:41:42.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:42.858+0000 7f41b6800640 1 --2- 172.21.15.5:0/900229068 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f419c0a2c00 0x7f419c0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:41:42.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:42.858+0000 7f41bfe7a740 1 -- 172.21.15.5:0/4160805326 wait complete. 2024-09-10T09:41:42.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:42.858+0000 7f41bfe7a740 1 -- start start 2024-09-10T09:41:42.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:42.859+0000 7f41b6800640 1 --2- 172.21.15.5:0/900229068 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f419c0a2c00 0x7f419c0a50c0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x55e015a3d060 tx=0x7f41a8005d80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:41:42.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:42.859+0000 7f41bfe7a740 1 Processor -- start 2024-09-10T09:41:42.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:42.859+0000 7f41bfe7a740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e015a3fc70 0x55e015a47df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:41:42.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:42.859+0000 7f41bfe7a740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e015a40590 0x55e015a48330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:41:42.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:42.859+0000 7f41bfe7a740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e015a4ae50 0x55e015a48870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:41:42.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:42.859+0000 7f41bfe7a740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55e015a4da20 con 0x55e015a40590 2024-09-10T09:41:42.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:42.859+0000 7f41bfe7a740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55e0159610f0 con 0x55e015a3fc70 2024-09-10T09:41:42.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:42.859+0000 7f41bfe7a740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55e015a4dba0 con 0x55e015a4ae50 2024-09-10T09:41:42.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:42.859+0000 7f41bfe7a740 10 client.0 _ll_register_callbacks cb 0x55e015948bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:41:42.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:42.859+0000 7f41b6800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e015a3fc70 0x55e015a47df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:41:42.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:42.859+0000 7f41b7200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e015a4ae50 0x55e015a48870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:41:42.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:42.860+0000 7f41ade00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e015a40590 0x55e015a48330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:41:42.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:42.860+0000 7f41b7200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e015a4ae50 0x55e015a48870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:50610/0 (socket says 172.21.15.5:50610) 2024-09-10T09:41:42.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:42.860+0000 7f41b7200640 1 -- 172.21.15.5:0/3980049801 learned_addr learned my addr 172.21.15.5:0/3980049801 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:41:42.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:42.860+0000 7f41b6800640 1 -- 172.21.15.5:0/3980049801 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e015a4ae50 msgr2=0x55e015a48870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:41:42.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:42.860+0000 7f41b6800640 1 --2- 172.21.15.5:0/3980049801 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e015a4ae50 0x55e015a48870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:41:42.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:42.860+0000 7f41b6800640 1 -- 172.21.15.5:0/3980049801 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e015a40590 msgr2=0x55e015a48330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:41:42.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:42.860+0000 7f41b6800640 1 --2- 172.21.15.5:0/3980049801 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e015a40590 0x55e015a48330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:41:42.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:42.860+0000 7f41b6800640 1 -- 172.21.15.5:0/3980049801 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e0157173c0 con 0x55e015a3fc70 2024-09-10T09:41:42.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:42.860+0000 7f41b6800640 1 --2- 172.21.15.5:0/3980049801 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e015a3fc70 0x55e015a47df0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f41a8006010 tx=0x7f41a802a8c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cef4ccba1789c08a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:41:42.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:42.860+0000 7f419be00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-09-10T09:41:42.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:42.860+0000 7f419be00640 1 -- 172.21.15.5:0/3980049801 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41a80317e0 con 0x55e015a3fc70 2024-09-10T09:41:42.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:42.861+0000 7f41bfe7a740 1 -- 172.21.15.5:0/3980049801 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f41b0019b10 con 0x55e015a3fc70 2024-09-10T09:41:42.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:42.861+0000 7f419be00640 1 -- 172.21.15.5:0/3980049801 <== mon.2 v2:172.21.15.102:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f41a8031980 con 0x55e015a3fc70 2024-09-10T09:41:42.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:42.861+0000 7f419be00640 1 -- 172.21.15.5:0/3980049801 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f41a8003fd0 con 0x55e015a3fc70 2024-09-10T09:41:42.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:42.861+0000 7f41bfe7a740 10 client.26030 fetch_fsmap learned FSMap version 32 2024-09-10T09:41:42.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:42.861+0000 7f41bfe7a740 1 -- 172.21.15.5:0/3980049801 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55e015a59b00 con 0x55e015a3fc70 2024-09-10T09:41:42.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:42.861+0000 7f419be00640 1 -- 172.21.15.5:0/3980049801 <== mon.2 v2:172.21.15.102:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f41b0019b10 con 0x55e015a3fc70 2024-09-10T09:41:42.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:42.861+0000 7f419be00640 1 -- 172.21.15.5:0/3980049801 <== mon.2 v2:172.21.15.102:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7f41a8051070 con 0x55e015a3fc70 2024-09-10T09:41:42.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:42.862+0000 7f41bfe7a740 10 client.26030 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T09:41:42.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:42.862+0000 7f41bfe7a740 10 client.26030 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-10T09:41:42.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:42.862+0000 7f41bfe7a740 20 client.26030 populate_metadata read hostname 'smithi005' 2024-09-10T09:41:42.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:42.862+0000 7f41bfe7a740 1 --2- 172.21.15.5:0/3980049801 >> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] conn(0x55e015a5aef0 0x55e015a5d360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:41:42.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:42.862+0000 7f41bfe7a740 4 client.26030 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:41:42.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:42.862+0000 7f41bfe7a740 1 -- 172.21.15.5:0/3980049801 --> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e0159b3950 con 0x55e015a5aef0 2024-09-10T09:41:42.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:42.862+0000 7f41ade00640 1 --2- 172.21.15.5:0/3980049801 >> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] conn(0x55e015a5aef0 0x55e015a5d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:41:42.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:42.863+0000 7f41ade00640 1 --2- 172.21.15.5:0/3980049801 >> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] conn(0x55e015a5aef0 0x55e015a5d360 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=dd1971e56ea6d01f server_cookie=3a6d1d64b995ed6d in_seq=0 out_seq=0 2024-09-10T09:41:42.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:42.863+0000 7f419be00640 10 client.26030 ms_handle_connect on v2:172.21.15.159:6836/620627639 2024-09-10T09:41:42.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:42.863+0000 7f419be00640 1 -- 172.21.15.5:0/3980049801 <== mds.2 v2:172.21.15.159:6836/620627639 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e0159b3950 con 0x55e015a5aef0 2024-09-10T09:41:42.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:42.863+0000 7f419be00640 10 client.26030 handle_command_reply: tid=0 2024-09-10T09:41:42.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:42.863+0000 7f41bfe7a740 1 client.26030 shutdown 2024-09-10T09:41:42.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:42.863+0000 7f41bfe7a740 1 -- 172.21.15.5:0/3980049801 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e015a3fc70 msgr2=0x55e015a47df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:41:42.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:42.863+0000 7f41bfe7a740 1 --2- 172.21.15.5:0/3980049801 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e015a3fc70 0x55e015a47df0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f41a8006010 tx=0x7f41a802a8c0 comp rx=0 tx=0).stop 2024-09-10T09:41:42.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:42.864+0000 7f41bfe7a740 1 -- 172.21.15.5:0/3980049801 shutdown_connections 2024-09-10T09:41:42.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:42.864+0000 7f41bfe7a740 1 --2- 172.21.15.5:0/3980049801 >> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] conn(0x55e015a5aef0 0x55e015a5d360 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:41:42.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:42.864+0000 7f41bfe7a740 1 --2- 172.21.15.5:0/3980049801 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e015a4ae50 0x55e015a48870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:41:42.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:42.864+0000 7f41bfe7a740 1 --2- 172.21.15.5:0/3980049801 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e015a40590 0x55e015a48330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:41:42.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:42.864+0000 7f41bfe7a740 1 --2- 172.21.15.5:0/3980049801 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e015a3fc70 0x55e015a47df0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:41:42.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:42.864+0000 7f419be00640 0 client.26030 ms_handle_reset on v2:172.21.15.159:6836/620627639 2024-09-10T09:41:42.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:42.864+0000 7f41bfe7a740 1 -- 172.21.15.5:0/3980049801 >> 172.21.15.5:0/3980049801 conn(0x55e015a3c200 msgr2=0x55e015a41f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:41:42.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:42.864+0000 7f41bfe7a740 1 -- 172.21.15.5:0/3980049801 shutdown_connections 2024-09-10T09:41:42.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:42.864+0000 7f41bfe7a740 1 -- 172.21.15.5:0/3980049801 wait complete. 2024-09-10T09:41:42.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:42.864+0000 7f41bfe7a740 20 client.26030 trim_cache size 0 max 16384 2024-09-10T09:41:42.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:42.882+0000 7f41bfe7a740 1 Processor -- start 2024-09-10T09:41:42.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:42.882+0000 7f41bfe7a740 1 -- start start 2024-09-10T09:41:42.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:42.882+0000 7f41bfe7a740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e015a3fc70 0x55e015a42770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:41:42.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:42.883+0000 7f41bfe7a740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e015a40590 0x55e015a62c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:41:42.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:42.883+0000 7f41bfe7a740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e015a4ae50 0x55e015a63170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:41:42.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:42.883+0000 7f41bfe7a740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f41b81b1080 con 0x55e015a40590 2024-09-10T09:41:42.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:42.883+0000 7f41bfe7a740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f41b81b2c80 con 0x55e015a4ae50 2024-09-10T09:41:42.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:42.883+0000 7f41bfe7a740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f41b81b1750 con 0x55e015a3fc70 2024-09-10T09:41:42.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:42.883+0000 7f41b6800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e015a3fc70 0x55e015a42770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:41:42.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:42.883+0000 7f41b6800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e015a3fc70 0x55e015a42770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:50620/0 (socket says 172.21.15.5:50620) 2024-09-10T09:41:42.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:42.883+0000 7f41b6800640 1 -- 172.21.15.5:0/316104714 learned_addr learned my addr 172.21.15.5:0/316104714 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:41:42.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:42.883+0000 7f41ade00640 1 --2- 172.21.15.5:0/316104714 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e015a40590 0x55e015a62c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:41:42.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:42.883+0000 7f41b7200640 1 --2- 172.21.15.5:0/316104714 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e015a4ae50 0x55e015a63170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:41:42.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:42.883+0000 7f41b6800640 1 -- 172.21.15.5:0/316104714 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e015a4ae50 msgr2=0x55e015a63170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:41:42.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:42.883+0000 7f41b6800640 1 --2- 172.21.15.5:0/316104714 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e015a4ae50 0x55e015a63170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:41:42.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:42.883+0000 7f41b6800640 1 -- 172.21.15.5:0/316104714 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e015a40590 msgr2=0x55e015a62c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:41:42.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:42.883+0000 7f41b6800640 1 --2- 172.21.15.5:0/316104714 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e015a40590 0x55e015a62c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:41:42.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:42.883+0000 7f41b6800640 1 -- 172.21.15.5:0/316104714 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e015a4a760 con 0x55e015a3fc70 2024-09-10T09:41:42.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:42.884+0000 7f41b6800640 1 --2- 172.21.15.5:0/316104714 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e015a3fc70 0x55e015a42770 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f41a802fc60 tx=0x7f41a8032fe0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cf6fa82b195802b2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:41:42.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:42.884+0000 7f41a6a00640 1 -- 172.21.15.5:0/316104714 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41a80319e0 con 0x55e015a3fc70 2024-09-10T09:41:42.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:42.884+0000 7f41a6a00640 1 -- 172.21.15.5:0/316104714 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f41a8036050 con 0x55e015a3fc70 2024-09-10T09:41:42.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:42.884+0000 7f41a6a00640 1 -- 172.21.15.5:0/316104714 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41a80305c0 con 0x55e015a3fc70 2024-09-10T09:41:42.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:42.885+0000 7f41bfe7a740 1 -- 172.21.15.5:0/316104714 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e015a3fc70 msgr2=0x55e015a42770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:41:42.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:42.885+0000 7f41bfe7a740 1 --2- 172.21.15.5:0/316104714 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e015a3fc70 0x55e015a42770 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f41a802fc60 tx=0x7f41a8032fe0 comp rx=0 tx=0).stop 2024-09-10T09:41:42.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:42.885+0000 7f41bfe7a740 1 -- 172.21.15.5:0/316104714 shutdown_connections 2024-09-10T09:41:42.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:42.885+0000 7f41bfe7a740 1 --2- 172.21.15.5:0/316104714 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e015a4ae50 0x55e015a63170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:41:42.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:42.885+0000 7f41bfe7a740 1 --2- 172.21.15.5:0/316104714 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e015a40590 0x55e015a62c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:41:42.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:42.885+0000 7f41bfe7a740 1 --2- 172.21.15.5:0/316104714 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e015a3fc70 0x55e015a42770 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:41:42.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:42.885+0000 7f41bfe7a740 1 -- 172.21.15.5:0/316104714 >> 172.21.15.5:0/316104714 conn(0x55e015a3c200 msgr2=0x55e015a49e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:41:42.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:42.885+0000 7f41bfe7a740 1 -- 172.21.15.5:0/316104714 shutdown_connections 2024-09-10T09:41:42.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:42.885+0000 7f41bfe7a740 1 -- 172.21.15.5:0/316104714 wait complete. 2024-09-10T09:41:42.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:42.886+0000 7f41bfe7a740 1 -- start start 2024-09-10T09:41:42.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:42.886+0000 7f41bfe7a740 1 Processor -- start 2024-09-10T09:41:42.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:42.886+0000 7f41bfe7a740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e015a3fc70 0x55e015a52df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:41:42.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:42.886+0000 7f41bfe7a740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e015a40590 0x55e015a53330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:41:42.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:42.886+0000 7f41bfe7a740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e015a4ae50 0x55e015a4fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:41:42.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:42.886+0000 7f41bfe7a740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f41b81b24f0 con 0x55e015a40590 2024-09-10T09:41:42.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:42.886+0000 7f41bfe7a740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f41b81b1e20 con 0x55e015a4ae50 2024-09-10T09:41:42.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:42.886+0000 7f41bfe7a740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f41b81af060 con 0x55e015a3fc70 2024-09-10T09:41:42.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:42.886+0000 7f41bfe7a740 10 client.0 _ll_register_callbacks cb 0x7f41a8009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:41:42.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:42.887+0000 7f41b6800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e015a3fc70 0x55e015a52df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:41:42.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:42.887+0000 7f41b6800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e015a3fc70 0x55e015a52df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:50624/0 (socket says 172.21.15.5:50624) 2024-09-10T09:41:42.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:42.887+0000 7f41b6800640 1 -- 172.21.15.5:0/3154419267 learned_addr learned my addr 172.21.15.5:0/3154419267 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:41:42.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:42.887+0000 7f41ade00640 1 --2- 172.21.15.5:0/3154419267 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e015a40590 0x55e015a53330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:41:42.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:42.887+0000 7f41b7200640 1 --2- 172.21.15.5:0/3154419267 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e015a4ae50 0x55e015a4fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:41:42.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:42.887+0000 7f41b6800640 1 -- 172.21.15.5:0/3154419267 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e015a4ae50 msgr2=0x55e015a4fdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:41:42.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:42.887+0000 7f41b6800640 1 --2- 172.21.15.5:0/3154419267 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e015a4ae50 0x55e015a4fdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:41:42.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:42.887+0000 7f41b6800640 1 -- 172.21.15.5:0/3154419267 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e015a40590 msgr2=0x55e015a53330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:41:42.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:42.887+0000 7f41b6800640 1 --2- 172.21.15.5:0/3154419267 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e015a40590 0x55e015a53330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:41:42.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:42.887+0000 7f41b6800640 1 -- 172.21.15.5:0/3154419267 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e015a45d60 con 0x55e015a3fc70 2024-09-10T09:41:42.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:42.887+0000 7f41ade00640 1 --2- 172.21.15.5:0/3154419267 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e015a40590 0x55e015a53330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:41:42.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:42.887+0000 7f41b6800640 1 --2- 172.21.15.5:0/3154419267 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e015a3fc70 0x55e015a52df0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f41a8004510 tx=0x7f41a8031a10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2ffce14c4665427 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:41:42.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:42.887+0000 7f419be00640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-09-10T09:41:42.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:42.887+0000 7f419be00640 1 -- 172.21.15.5:0/3154419267 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41a802f810 con 0x55e015a3fc70 2024-09-10T09:41:42.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:42.888+0000 7f41bfe7a740 1 -- 172.21.15.5:0/3154419267 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f41a8036050 con 0x55e015a3fc70 2024-09-10T09:41:42.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:42.888+0000 7f419be00640 1 -- 172.21.15.5:0/3154419267 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f41a802f9b0 con 0x55e015a3fc70 2024-09-10T09:41:42.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:42.888+0000 7f419be00640 1 -- 172.21.15.5:0/3154419267 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f41a8004820 con 0x55e015a3fc70 2024-09-10T09:41:42.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:42.888+0000 7f41bfe7a740 10 client.26056 fetch_fsmap learned FSMap version 32 2024-09-10T09:41:42.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:42.888+0000 7f41bfe7a740 1 -- 172.21.15.5:0/3154419267 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55e015ac6140 con 0x55e015a3fc70 2024-09-10T09:41:42.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:42.888+0000 7f419be00640 1 -- 172.21.15.5:0/3154419267 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f41a8036050 con 0x55e015a3fc70 2024-09-10T09:41:42.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:42.888+0000 7f419be00640 1 -- 172.21.15.5:0/3154419267 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7f41a8060030 con 0x55e015a3fc70 2024-09-10T09:41:42.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:42.888+0000 7f41bfe7a740 10 client.26056 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T09:41:42.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:42.888+0000 7f41bfe7a740 10 client.26056 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-10T09:41:42.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:42.888+0000 7f41bfe7a740 20 client.26056 populate_metadata read hostname 'smithi005' 2024-09-10T09:41:42.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:42.888+0000 7f41bfe7a740 1 --2- 172.21.15.5:0/3154419267 >> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] conn(0x55e015ac65c0 0x55e015ac89b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:41:42.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:42.888+0000 7f41bfe7a740 4 client.26056 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T09:41:42.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:42.888+0000 7f41bfe7a740 1 -- 172.21.15.5:0/3154419267 --> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e0159ade60 con 0x55e015ac65c0 2024-09-10T09:41:42.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:42.889+0000 7f41ade00640 1 --2- 172.21.15.5:0/3154419267 >> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] conn(0x55e015ac65c0 0x55e015ac89b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:41:42.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:42.889+0000 7f41ade00640 1 --2- 172.21.15.5:0/3154419267 >> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] conn(0x55e015ac65c0 0x55e015ac89b0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=28cb1838b81a96f9 server_cookie=c31380ecde9fa273 in_seq=0 out_seq=0 2024-09-10T09:41:42.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:42.889+0000 7f419be00640 10 client.26056 ms_handle_connect on v2:172.21.15.159:6836/620627639 2024-09-10T09:41:42.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:42.890+0000 7f419be00640 1 -- 172.21.15.5:0/3154419267 <== mds.2 v2:172.21.15.159:6836/620627639 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e0159ade60 con 0x55e015ac65c0 2024-09-10T09:41:42.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:42.890+0000 7f419be00640 10 client.26056 handle_command_reply: tid=0 2024-09-10T09:41:42.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:42.890+0000 7f41bfe7a740 1 client.26056 shutdown 2024-09-10T09:41:42.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:42.890+0000 7f41bfe7a740 1 -- 172.21.15.5:0/3154419267 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e015a3fc70 msgr2=0x55e015a52df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:41:42.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:42.890+0000 7f41bfe7a740 1 --2- 172.21.15.5:0/3154419267 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e015a3fc70 0x55e015a52df0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f41a8004510 tx=0x7f41a8031a10 comp rx=0 tx=0).stop 2024-09-10T09:41:42.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:42.890+0000 7f41bfe7a740 1 -- 172.21.15.5:0/3154419267 shutdown_connections 2024-09-10T09:41:42.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:42.890+0000 7f41bfe7a740 1 --2- 172.21.15.5:0/3154419267 >> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] conn(0x55e015ac65c0 0x55e015ac89b0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:41:42.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:42.890+0000 7f41bfe7a740 1 --2- 172.21.15.5:0/3154419267 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e015a4ae50 0x55e015a4fdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:41:42.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:42.890+0000 7f41bfe7a740 1 --2- 172.21.15.5:0/3154419267 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e015a40590 0x55e015a53330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:41:42.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:42.890+0000 7f41bfe7a740 1 --2- 172.21.15.5:0/3154419267 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e015a3fc70 0x55e015a52df0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:41:42.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:42.890+0000 7f41bfe7a740 1 -- 172.21.15.5:0/3154419267 >> 172.21.15.5:0/3154419267 conn(0x55e015a3c200 msgr2=0x55e015a3ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:41:42.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:42.890+0000 7f419be00640 0 client.26056 ms_handle_reset on v2:172.21.15.159:6836/620627639 2024-09-10T09:41:42.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:42.890+0000 7f41bfe7a740 1 -- 172.21.15.5:0/3154419267 shutdown_connections 2024-09-10T09:41:42.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:42.890+0000 7f41bfe7a740 1 -- 172.21.15.5:0/3154419267 wait complete. 2024-09-10T09:41:42.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:42.890+0000 7f41bfe7a740 20 client.26056 trim_cache size 0 max 16384 2024-09-10T09:41:42.895 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-10T09:41:42.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:42.891+0000 7f41bd400640 1 -- 172.21.15.5:0/900229068 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f419c0a2c00 msgr2=0x7f419c0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:41:42.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:42.891+0000 7f41bd400640 1 --2- 172.21.15.5:0/900229068 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f419c0a2c00 0x7f419c0a50c0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x55e015a3d060 tx=0x7f41a8005d80 comp rx=0 tx=0).stop 2024-09-10T09:41:42.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:42.891+0000 7f41bd400640 1 -- 172.21.15.5:0/900229068 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f41b81049c0 msgr2=0x7f41b819ab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:41:42.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:42.891+0000 7f41bd400640 1 --2- 172.21.15.5:0/900229068 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f41b81049c0 0x7f41b819ab70 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f41a000e9f0 tx=0x7f41a000eeb0 comp rx=0 tx=0).stop 2024-09-10T09:41:42.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:42.891+0000 7f41bd400640 1 -- 172.21.15.5:0/900229068 shutdown_connections 2024-09-10T09:41:42.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:42.891+0000 7f41bd400640 1 --2- 172.21.15.5:0/900229068 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f419c0a2c00 0x7f419c0a50c0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:41:42.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:42.891+0000 7f41bd400640 1 --2- 172.21.15.5:0/900229068 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f41b8109140 0x7f41b819ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:41:42.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:42.891+0000 7f41bd400640 1 --2- 172.21.15.5:0/900229068 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f41b81049c0 0x7f41b819ab70 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:41:42.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:42.891+0000 7f41bd400640 1 --2- 172.21.15.5:0/900229068 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f41b8104010 0x7f41b819a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:41:42.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:42.891+0000 7f41bd400640 1 -- 172.21.15.5:0/900229068 >> 172.21.15.5:0/900229068 conn(0x7f41b80ffa50 msgr2=0x7f41b8100dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:41:42.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:42.891+0000 7f41bd400640 1 -- 172.21.15.5:0/900229068 shutdown_connections 2024-09-10T09:41:42.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:42.891+0000 7f41bd400640 1 -- 172.21.15.5:0/900229068 wait complete. 2024-09-10T09:41:43.012 INFO:tasks.workunit.client.0.smithi005.stdout: 1 137810 9.05 MB/sec warmup 84 sec latency 763.476 ms 2024-09-10T09:41:43.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:41:43 smithi005 ceph-mon[30350]: pgmap v494: 129 pgs: 129 active+clean; 73 MiB data, 713 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.6 MiB/s wr, 182 op/s 2024-09-10T09:41:43.783 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-10T09:41:43.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:41:43 smithi159 ceph-mon[35513]: pgmap v494: 129 pgs: 129 active+clean; 73 MiB data, 713 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.6 MiB/s wr, 182 op/s 2024-09-10T09:41:43.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:41:43 smithi102 ceph-mon[35442]: pgmap v494: 129 pgs: 129 active+clean; 73 MiB data, 713 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.6 MiB/s wr, 182 op/s 2024-09-10T09:41:44.012 INFO:tasks.workunit.client.0.smithi005.stdout: 1 139823 9.08 MB/sec warmup 85 sec latency 23.560 ms 2024-09-10T09:41:44.195 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:41:44.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:44.786+0000 7faaf8e00640 1 -- 172.21.15.5:0/4132838001 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7faaf4076720 msgr2=0x7faaf4076b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:41:44.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:44.786+0000 7faaf8e00640 1 --2- 172.21.15.5:0/4132838001 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7faaf4076720 0x7faaf4076b00 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7faae4009900 tx=0x7faae402a3f0 comp rx=0 tx=0).stop 2024-09-10T09:41:44.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:44.786+0000 7faaf8e00640 1 -- 172.21.15.5:0/4132838001 shutdown_connections 2024-09-10T09:41:44.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:44.786+0000 7faaf8e00640 1 --2- 172.21.15.5:0/4132838001 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7faaf4112870 0x7faaf4114c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:41:44.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:44.786+0000 7faaf8e00640 1 --2- 172.21.15.5:0/4132838001 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faaf4077040 0x7faaf4112330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:41:44.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:44.786+0000 7faaf8e00640 1 --2- 172.21.15.5:0/4132838001 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7faaf4076720 0x7faaf4076b00 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:41:44.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:44.786+0000 7faaf8e00640 1 -- 172.21.15.5:0/4132838001 >> 172.21.15.5:0/4132838001 conn(0x7faaf40ff910 msgr2=0x7faaf4101d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:41:44.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:44.786+0000 7faaf8e00640 1 -- 172.21.15.5:0/4132838001 shutdown_connections 2024-09-10T09:41:44.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:44.786+0000 7faaf8e00640 1 -- 172.21.15.5:0/4132838001 wait complete. 2024-09-10T09:41:44.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:44.787+0000 7faaf8e00640 1 Processor -- start 2024-09-10T09:41:44.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:44.787+0000 7faaf8e00640 1 -- start start 2024-09-10T09:41:44.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:44.787+0000 7faaf8e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faaf4076720 0x7faaf41a4640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:41:44.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:44.787+0000 7faaf8e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7faaf4077040 0x7faaf41a4b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:41:44.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:44.787+0000 7faaf8e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7faaf4112870 0x7faaf419e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:41:44.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:44.787+0000 7faaf8e00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7faaf4118450 con 0x7faaf4076720 2024-09-10T09:41:44.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:44.787+0000 7faaf8e00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7faaf41182d0 con 0x7faaf4077040 2024-09-10T09:41:44.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:44.787+0000 7faaf8e00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7faaf41185d0 con 0x7faaf4112870 2024-09-10T09:41:44.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:44.788+0000 7faaf3e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7faaf4112870 0x7faaf419e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:41:44.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:44.788+0000 7faaebe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7faaf4077040 0x7faaf41a4b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:41:44.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:44.788+0000 7faaf3e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7faaf4112870 0x7faaf419e7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:50644/0 (socket says 172.21.15.5:50644) 2024-09-10T09:41:44.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:44.788+0000 7faaf3e00640 1 -- 172.21.15.5:0/2177169412 learned_addr learned my addr 172.21.15.5:0/2177169412 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:41:44.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:44.788+0000 7faaf3400640 1 --2- 172.21.15.5:0/2177169412 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faaf4076720 0x7faaf41a4640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:41:44.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:44.788+0000 7faaf3e00640 1 -- 172.21.15.5:0/2177169412 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7faaf4077040 msgr2=0x7faaf41a4b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:41:44.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:44.788+0000 7faaf3e00640 1 --2- 172.21.15.5:0/2177169412 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7faaf4077040 0x7faaf41a4b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:41:44.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:44.788+0000 7faaf3e00640 1 -- 172.21.15.5:0/2177169412 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faaf4076720 msgr2=0x7faaf41a4640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:41:44.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:44.788+0000 7faaf3e00640 1 --2- 172.21.15.5:0/2177169412 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faaf4076720 0x7faaf41a4640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:41:44.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:44.788+0000 7faaf3e00640 1 -- 172.21.15.5:0/2177169412 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faaf419ef60 con 0x7faaf4112870 2024-09-10T09:41:44.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:44.788+0000 7faaebe00640 1 --2- 172.21.15.5:0/2177169412 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7faaf4077040 0x7faaf41a4b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:41:44.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:44.788+0000 7faaf3e00640 1 --2- 172.21.15.5:0/2177169412 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7faaf4112870 0x7faaf419e7c0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7faaec00eb10 tx=0x7faaec00c2d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:41:44.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:44.789+0000 7faaf0c00640 1 -- 172.21.15.5:0/2177169412 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaec004b20 con 0x7faaf4112870 2024-09-10T09:41:44.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:44.789+0000 7faaf0c00640 1 -- 172.21.15.5:0/2177169412 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faaec004cc0 con 0x7faaf4112870 2024-09-10T09:41:44.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:44.789+0000 7faaf8e00640 1 -- 172.21.15.5:0/2177169412 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faaf419f250 con 0x7faaf4112870 2024-09-10T09:41:44.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:44.789+0000 7faaf0c00640 1 -- 172.21.15.5:0/2177169412 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaec010790 con 0x7faaf4112870 2024-09-10T09:41:44.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:44.789+0000 7faaf8e00640 1 -- 172.21.15.5:0/2177169412 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faaf41ab4a0 con 0x7faaf4112870 2024-09-10T09:41:44.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:44.792+0000 7faaf0c00640 1 -- 172.21.15.5:0/2177169412 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7faaec005020 con 0x7faaf4112870 2024-09-10T09:41:44.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:44.792+0000 7faafce2b740 1 Processor -- start 2024-09-10T09:41:44.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:44.792+0000 7faafce2b740 1 -- start start 2024-09-10T09:41:44.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:44.792+0000 7faafce2b740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f7bbe2ee40 0x55f7bbe2f220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:41:44.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:44.792+0000 7faafce2b740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f7bbe2f760 0x55f7bbe39ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:41:44.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:44.792+0000 7faafce2b740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f7bbe3a020 0x55f7bbe3c410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:41:44.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:44.792+0000 7faafce2b740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55f7bbd953e0 con 0x55f7bbe2f760 2024-09-10T09:41:44.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:44.792+0000 7faafce2b740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55f7bbd3cd80 con 0x55f7bbe2ee40 2024-09-10T09:41:44.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:44.792+0000 7faafce2b740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55f7bbd913a0 con 0x55f7bbe3a020 2024-09-10T09:41:44.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:44.793+0000 7faaebe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f7bbe2ee40 0x55f7bbe2f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:41:44.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:44.793+0000 7faaf3400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f7bbe2f760 0x55f7bbe39ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:41:44.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:44.793+0000 7faaebe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f7bbe2ee40 0x55f7bbe2f220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:57300/0 (socket says 172.21.15.5:57300) 2024-09-10T09:41:44.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:44.793+0000 7faaebe00640 1 -- 172.21.15.5:0/2984855380 learned_addr learned my addr 172.21.15.5:0/2984855380 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:41:44.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:44.793+0000 7faaebe00640 1 --2- 172.21.15.5:0/2984855380 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f7bbe3a020 0x55f7bbe3c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:41:44.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:44.793+0000 7faaf3400640 1 -- 172.21.15.5:0/2984855380 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f7bbe3a020 msgr2=0x55f7bbe3c410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:41:44.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:44.793+0000 7faaf3400640 1 --2- 172.21.15.5:0/2984855380 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f7bbe3a020 0x55f7bbe3c410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:41:44.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:44.793+0000 7faaf3400640 1 -- 172.21.15.5:0/2984855380 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f7bbe2ee40 msgr2=0x55f7bbe2f220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:41:44.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:44.793+0000 7faaf3400640 1 --2- 172.21.15.5:0/2984855380 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f7bbe2ee40 0x55f7bbe2f220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:41:44.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:44.793+0000 7faaf3400640 1 -- 172.21.15.5:0/2984855380 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f7bbd5bae0 con 0x55f7bbe2f760 2024-09-10T09:41:44.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:44.793+0000 7faaebe00640 1 --2- 172.21.15.5:0/2984855380 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f7bbe2ee40 0x55f7bbe2f220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:41:44.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:44.793+0000 7faaf3400640 1 --2- 172.21.15.5:0/2984855380 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f7bbe2f760 0x55f7bbe39ae0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7faae402a9e0 tx=0x7faae4005de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6f91652ba706a67 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:41:44.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:44.794+0000 7faaf0c00640 1 --2- 172.21.15.5:0/2177169412 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7faad80a2b80 0x7faad80a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:41:44.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:44.794+0000 7faae8c00640 1 -- 172.21.15.5:0/2984855380 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faae4035040 con 0x55f7bbe2f760 2024-09-10T09:41:44.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:44.794+0000 7faaf0c00640 1 -- 172.21.15.5:0/2177169412 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7faaec0cfb40 con 0x7faaf4112870 2024-09-10T09:41:44.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:44.794+0000 7faae8c00640 1 -- 172.21.15.5:0/2984855380 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faae4004400 con 0x55f7bbe2f760 2024-09-10T09:41:44.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:44.794+0000 7faae8c00640 1 -- 172.21.15.5:0/2984855380 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faae4004e60 con 0x55f7bbe2f760 2024-09-10T09:41:44.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:44.794+0000 7faaebe00640 1 --2- 172.21.15.5:0/2177169412 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7faad80a2b80 0x7faad80a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:41:44.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:44.795+0000 7faafce2b740 1 -- 172.21.15.5:0/2984855380 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f7bbe2f760 msgr2=0x55f7bbe39ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:41:44.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:44.795+0000 7faaebe00640 1 --2- 172.21.15.5:0/2177169412 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7faad80a2b80 0x7faad80a5040 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x55f7bbe2c750 tx=0x7faadc004730 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:41:44.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:44.795+0000 7faafce2b740 1 --2- 172.21.15.5:0/2984855380 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f7bbe2f760 0x55f7bbe39ae0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7faae402a9e0 tx=0x7faae4005de0 comp rx=0 tx=0).stop 2024-09-10T09:41:44.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:44.795+0000 7faafce2b740 1 -- 172.21.15.5:0/2984855380 shutdown_connections 2024-09-10T09:41:44.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:44.795+0000 7faafce2b740 1 --2- 172.21.15.5:0/2984855380 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f7bbe3a020 0x55f7bbe3c410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:41:44.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:44.795+0000 7faafce2b740 1 --2- 172.21.15.5:0/2984855380 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f7bbe2f760 0x55f7bbe39ae0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:41:44.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:44.795+0000 7faafce2b740 1 --2- 172.21.15.5:0/2984855380 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f7bbe2ee40 0x55f7bbe2f220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:41:44.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:44.795+0000 7faafce2b740 1 -- 172.21.15.5:0/2984855380 >> 172.21.15.5:0/2984855380 conn(0x55f7bbe2b3d0 msgr2=0x55f7bbe2b7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:41:44.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:44.795+0000 7faafce2b740 1 -- 172.21.15.5:0/2984855380 shutdown_connections 2024-09-10T09:41:44.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:44.795+0000 7faafce2b740 1 -- 172.21.15.5:0/2984855380 wait complete. 2024-09-10T09:41:44.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:44.795+0000 7faafce2b740 1 -- start start 2024-09-10T09:41:44.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:44.796+0000 7faafce2b740 1 Processor -- start 2024-09-10T09:41:44.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:44.796+0000 7faafce2b740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f7bbe2ee40 0x55f7bbe36fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:41:44.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:44.796+0000 7faafce2b740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f7bbe2f760 0x55f7bbe374e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:41:44.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:44.796+0000 7faafce2b740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f7bbe3a020 0x55f7bbe37a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:41:44.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:44.796+0000 7faafce2b740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55f7bbe3cbf0 con 0x55f7bbe3a020 2024-09-10T09:41:44.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:44.796+0000 7faafce2b740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55f7bbd500f0 con 0x55f7bbe2ee40 2024-09-10T09:41:44.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:44.796+0000 7faafce2b740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55f7bbe3cd70 con 0x55f7bbe2f760 2024-09-10T09:41:44.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:44.796+0000 7faafce2b740 10 client.0 _ll_register_callbacks cb 0x55f7bbd37bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:41:44.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:44.797+0000 7faaebe00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f7bbe2f760 0x55f7bbe374e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:41:44.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:44.797+0000 7faaebe00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f7bbe2f760 0x55f7bbe374e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:50666/0 (socket says 172.21.15.5:50666) 2024-09-10T09:41:44.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:44.797+0000 7faaebe00640 1 -- 172.21.15.5:0/1190618876 learned_addr learned my addr 172.21.15.5:0/1190618876 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:41:44.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:44.797+0000 7faaf3e00640 1 --2- 172.21.15.5:0/1190618876 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f7bbe3a020 0x55f7bbe37a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:41:44.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:44.797+0000 7faaf3400640 1 --2- 172.21.15.5:0/1190618876 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f7bbe2ee40 0x55f7bbe36fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:41:44.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:44.797+0000 7faaebe00640 1 -- 172.21.15.5:0/1190618876 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f7bbe2ee40 msgr2=0x55f7bbe36fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:41:44.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:44.797+0000 7faaebe00640 1 --2- 172.21.15.5:0/1190618876 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f7bbe2ee40 0x55f7bbe36fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:41:44.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:44.797+0000 7faaebe00640 1 -- 172.21.15.5:0/1190618876 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f7bbe3a020 msgr2=0x55f7bbe37a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:41:44.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:44.797+0000 7faaebe00640 1 --2- 172.21.15.5:0/1190618876 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f7bbe3a020 0x55f7bbe37a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:41:44.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:44.797+0000 7faaebe00640 1 -- 172.21.15.5:0/1190618876 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f7bbb063c0 con 0x55f7bbe2f760 2024-09-10T09:41:44.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:44.797+0000 7faaf3e00640 1 --2- 172.21.15.5:0/1190618876 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f7bbe3a020 0x55f7bbe37a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:41:44.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:44.797+0000 7faaf3400640 1 --2- 172.21.15.5:0/1190618876 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f7bbe2ee40 0x55f7bbe36fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T09:41:44.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:44.798+0000 7faaebe00640 1 --2- 172.21.15.5:0/1190618876 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f7bbe2f760 0x55f7bbe374e0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7faadc0096f0 tx=0x7faadc005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2c3bf14ff5d061cc server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:41:44.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:44.798+0000 7faae1600640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-09-10T09:41:44.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:44.798+0000 7faae1600640 1 -- 172.21.15.5:0/1190618876 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faadc005020 con 0x55f7bbe2f760 2024-09-10T09:41:44.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:44.798+0000 7faafce2b740 1 -- 172.21.15.5:0/1190618876 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faae4004400 con 0x55f7bbe2f760 2024-09-10T09:41:44.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:44.798+0000 7faae1600640 1 -- 172.21.15.5:0/1190618876 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7faadc009040 con 0x55f7bbe2f760 2024-09-10T09:41:44.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:44.798+0000 7faae1600640 1 -- 172.21.15.5:0/1190618876 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7faadc034050 con 0x55f7bbe2f760 2024-09-10T09:41:44.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:44.798+0000 7faafce2b740 10 client.26068 fetch_fsmap learned FSMap version 32 2024-09-10T09:41:44.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:44.798+0000 7faafce2b740 1 -- 172.21.15.5:0/1190618876 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55f7bbe48d00 con 0x55f7bbe2f760 2024-09-10T09:41:44.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:44.799+0000 7faae1600640 1 -- 172.21.15.5:0/1190618876 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faae4004400 con 0x55f7bbe2f760 2024-09-10T09:41:44.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:44.799+0000 7faae1600640 1 -- 172.21.15.5:0/1190618876 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7faadc00e5c0 con 0x55f7bbe2f760 2024-09-10T09:41:44.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:44.799+0000 7faafce2b740 10 client.26068 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T09:41:44.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:44.799+0000 7faafce2b740 10 client.26068 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-10T09:41:44.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:44.799+0000 7faafce2b740 20 client.26068 populate_metadata read hostname 'smithi005' 2024-09-10T09:41:44.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:44.799+0000 7faafce2b740 1 --2- 172.21.15.5:0/1190618876 >> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] conn(0x55f7bbe4a110 0x55f7bbe4c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:41:44.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:44.799+0000 7faafce2b740 4 client.26068 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:41:44.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:44.799+0000 7faafce2b740 1 -- 172.21.15.5:0/1190618876 --> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f7bbda2950 con 0x55f7bbe4a110 2024-09-10T09:41:44.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:44.799+0000 7faaf3400640 1 --2- 172.21.15.5:0/1190618876 >> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] conn(0x55f7bbe4a110 0x55f7bbe4c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:41:44.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:44.800+0000 7faaf3400640 1 --2- 172.21.15.5:0/1190618876 >> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] conn(0x55f7bbe4a110 0x55f7bbe4c5d0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=cb95515941cd700a server_cookie=b0917655f99cf92a in_seq=0 out_seq=0 2024-09-10T09:41:44.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:44.800+0000 7faae1600640 10 client.26068 ms_handle_connect on v2:172.21.15.159:6838/2073739599 2024-09-10T09:41:44.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:44.800+0000 7faae1600640 1 -- 172.21.15.5:0/1190618876 <== mds.1 v2:172.21.15.159:6838/2073739599 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f7bbda2950 con 0x55f7bbe4a110 2024-09-10T09:41:44.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:44.800+0000 7faae1600640 10 client.26068 handle_command_reply: tid=0 2024-09-10T09:41:44.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:44.801+0000 7faafce2b740 1 client.26068 shutdown 2024-09-10T09:41:44.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:44.801+0000 7faafce2b740 1 -- 172.21.15.5:0/1190618876 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f7bbe2f760 msgr2=0x55f7bbe374e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:41:44.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:44.801+0000 7faafce2b740 1 --2- 172.21.15.5:0/1190618876 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f7bbe2f760 0x55f7bbe374e0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7faadc0096f0 tx=0x7faadc005f90 comp rx=0 tx=0).stop 2024-09-10T09:41:44.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:44.801+0000 7faafce2b740 1 -- 172.21.15.5:0/1190618876 shutdown_connections 2024-09-10T09:41:44.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:44.801+0000 7faafce2b740 1 --2- 172.21.15.5:0/1190618876 >> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] conn(0x55f7bbe4a110 0x55f7bbe4c5d0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:41:44.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:44.801+0000 7faafce2b740 1 --2- 172.21.15.5:0/1190618876 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f7bbe3a020 0x55f7bbe37a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:41:44.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:44.801+0000 7faafce2b740 1 --2- 172.21.15.5:0/1190618876 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f7bbe2f760 0x55f7bbe374e0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:41:44.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:44.801+0000 7faafce2b740 1 --2- 172.21.15.5:0/1190618876 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f7bbe2ee40 0x55f7bbe36fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:41:44.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:44.801+0000 7faafce2b740 1 -- 172.21.15.5:0/1190618876 >> 172.21.15.5:0/1190618876 conn(0x55f7bbe2b3d0 msgr2=0x55f7bbe31100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:41:44.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:44.801+0000 7faae1600640 0 client.26068 ms_handle_reset on v2:172.21.15.159:6838/2073739599 2024-09-10T09:41:44.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:44.801+0000 7faafce2b740 1 -- 172.21.15.5:0/1190618876 shutdown_connections 2024-09-10T09:41:44.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:44.801+0000 7faafce2b740 1 -- 172.21.15.5:0/1190618876 wait complete. 2024-09-10T09:41:44.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:44.801+0000 7faafce2b740 20 client.26068 trim_cache size 0 max 16384 2024-09-10T09:41:44.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:44.815+0000 7faafce2b740 1 Processor -- start 2024-09-10T09:41:44.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:44.815+0000 7faafce2b740 1 -- start start 2024-09-10T09:41:44.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:44.816+0000 7faafce2b740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f7bbe2ee40 0x55f7bbe26300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:41:44.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:44.816+0000 7faafce2b740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f7bbe2f760 0x55f7bbe51ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:41:44.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:44.816+0000 7faafce2b740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f7bbe3a020 0x55f7bbe523e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:41:44.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:44.816+0000 7faafce2b740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7faaf41b5330 con 0x55f7bbe2ee40 2024-09-10T09:41:44.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:44.816+0000 7faafce2b740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7faaf41b6f30 con 0x55f7bbe3a020 2024-09-10T09:41:44.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:44.816+0000 7faafce2b740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7faaf41b5a00 con 0x55f7bbe2f760 2024-09-10T09:41:44.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:44.816+0000 7faaebe00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f7bbe2f760 0x55f7bbe51ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:41:44.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:44.816+0000 7faaf3400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f7bbe2ee40 0x55f7bbe26300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:41:44.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:44.816+0000 7faaebe00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f7bbe2f760 0x55f7bbe51ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:50676/0 (socket says 172.21.15.5:50676) 2024-09-10T09:41:44.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:44.816+0000 7faaebe00640 1 -- 172.21.15.5:0/262489209 learned_addr learned my addr 172.21.15.5:0/262489209 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:41:44.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:44.816+0000 7faaf3e00640 1 --2- 172.21.15.5:0/262489209 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f7bbe3a020 0x55f7bbe523e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:41:44.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:44.816+0000 7faaf3e00640 1 -- 172.21.15.5:0/262489209 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f7bbe2f760 msgr2=0x55f7bbe51ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:41:44.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:44.816+0000 7faaf3e00640 1 --2- 172.21.15.5:0/262489209 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f7bbe2f760 0x55f7bbe51ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:41:44.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:44.816+0000 7faaf3e00640 1 -- 172.21.15.5:0/262489209 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f7bbe2ee40 msgr2=0x55f7bbe26300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:41:44.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:44.816+0000 7faaf3e00640 1 --2- 172.21.15.5:0/262489209 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f7bbe2ee40 0x55f7bbe26300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:41:44.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:44.816+0000 7faaf3e00640 1 -- 172.21.15.5:0/262489209 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f7bbe39910 con 0x55f7bbe3a020 2024-09-10T09:41:44.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:44.816+0000 7faaebe00640 1 --2- 172.21.15.5:0/262489209 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f7bbe2f760 0x55f7bbe51ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:41:44.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:44.817+0000 7faaf3e00640 1 --2- 172.21.15.5:0/262489209 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f7bbe3a020 0x55f7bbe523e0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7faaec00efd0 tx=0x7faaec00ce80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5d74de19d170e7fa server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:41:44.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:44.817+0000 7faae3e00640 1 -- 172.21.15.5:0/262489209 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaec0b4070 con 0x55f7bbe3a020 2024-09-10T09:41:44.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:44.817+0000 7faae3e00640 1 -- 172.21.15.5:0/262489209 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faaec09c650 con 0x55f7bbe3a020 2024-09-10T09:41:44.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:44.817+0000 7faae3e00640 1 -- 172.21.15.5:0/262489209 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaec0bab90 con 0x55f7bbe3a020 2024-09-10T09:41:44.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:44.817+0000 7faafce2b740 1 -- 172.21.15.5:0/262489209 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f7bbe3a020 msgr2=0x55f7bbe523e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:41:44.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:44.817+0000 7faafce2b740 1 --2- 172.21.15.5:0/262489209 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f7bbe3a020 0x55f7bbe523e0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7faaec00efd0 tx=0x7faaec00ce80 comp rx=0 tx=0).stop 2024-09-10T09:41:44.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:44.818+0000 7faafce2b740 1 -- 172.21.15.5:0/262489209 shutdown_connections 2024-09-10T09:41:44.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:44.818+0000 7faafce2b740 1 --2- 172.21.15.5:0/262489209 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f7bbe3a020 0x55f7bbe523e0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:41:44.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:44.818+0000 7faafce2b740 1 --2- 172.21.15.5:0/262489209 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f7bbe2f760 0x55f7bbe51ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:41:44.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:44.818+0000 7faafce2b740 1 --2- 172.21.15.5:0/262489209 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f7bbe2ee40 0x55f7bbe26300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:41:44.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:44.818+0000 7faafce2b740 1 -- 172.21.15.5:0/262489209 >> 172.21.15.5:0/262489209 conn(0x55f7bbe2b3d0 msgr2=0x55f7bbe358b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:41:44.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:44.818+0000 7faafce2b740 1 -- 172.21.15.5:0/262489209 shutdown_connections 2024-09-10T09:41:44.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:44.818+0000 7faafce2b740 1 -- 172.21.15.5:0/262489209 wait complete. 2024-09-10T09:41:44.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:44.818+0000 7faafce2b740 1 -- start start 2024-09-10T09:41:44.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:44.818+0000 7faafce2b740 1 Processor -- start 2024-09-10T09:41:44.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:44.819+0000 7faafce2b740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f7bbe2ee40 0x55f7bbe42350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:41:44.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:44.819+0000 7faafce2b740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f7bbe2f760 0x55f7bbe42890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:41:44.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:44.819+0000 7faafce2b740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f7bbe3a020 0x55f7bbe33be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:41:44.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:44.819+0000 7faafce2b740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7faaf41b67a0 con 0x55f7bbe2f760 2024-09-10T09:41:44.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:44.819+0000 7faafce2b740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7faaf41b60d0 con 0x55f7bbe2ee40 2024-09-10T09:41:44.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:44.819+0000 7faafce2b740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7faaf41b3310 con 0x55f7bbe3a020 2024-09-10T09:41:44.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:44.819+0000 7faafce2b740 10 client.0 _ll_register_callbacks cb 0x7faadc00faa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:41:44.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:44.819+0000 7faaf3400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f7bbe2ee40 0x55f7bbe42350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:41:44.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:44.819+0000 7faaf3400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f7bbe2ee40 0x55f7bbe42350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:57330/0 (socket says 172.21.15.5:57330) 2024-09-10T09:41:44.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:44.819+0000 7faaf3400640 1 -- 172.21.15.5:0/1681718492 learned_addr learned my addr 172.21.15.5:0/1681718492 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:41:44.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:44.819+0000 7faaf3e00640 1 --2- 172.21.15.5:0/1681718492 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f7bbe3a020 0x55f7bbe33be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:41:44.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:44.819+0000 7faaebe00640 1 --2- 172.21.15.5:0/1681718492 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f7bbe2f760 0x55f7bbe42890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:41:44.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:44.819+0000 7faaf3400640 1 -- 172.21.15.5:0/1681718492 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f7bbe3a020 msgr2=0x55f7bbe33be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:41:44.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:44.819+0000 7faaf3400640 1 --2- 172.21.15.5:0/1681718492 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f7bbe3a020 0x55f7bbe33be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:41:44.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:44.819+0000 7faaf3400640 1 -- 172.21.15.5:0/1681718492 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f7bbe2f760 msgr2=0x55f7bbe42890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:41:44.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:44.819+0000 7faaf3400640 1 --2- 172.21.15.5:0/1681718492 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f7bbe2f760 0x55f7bbe42890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:41:44.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:44.819+0000 7faaf3400640 1 -- 172.21.15.5:0/1681718492 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f7bbe34410 con 0x55f7bbe2ee40 2024-09-10T09:41:44.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:44.819+0000 7faaebe00640 1 --2- 172.21.15.5:0/1681718492 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f7bbe2f760 0x55f7bbe42890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:41:44.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:44.820+0000 7faaf3400640 1 --2- 172.21.15.5:0/1681718492 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f7bbe2ee40 0x55f7bbe42350 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7faae402a9e0 tx=0x7faae402a3f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=931f394f015c604a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:41:44.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:44.820+0000 7faae1600640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-09-10T09:41:44.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:44.820+0000 7faae1600640 1 -- 172.21.15.5:0/1681718492 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faae4035040 con 0x55f7bbe2ee40 2024-09-10T09:41:44.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:44.820+0000 7faafce2b740 1 -- 172.21.15.5:0/1681718492 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faaec09c650 con 0x55f7bbe2ee40 2024-09-10T09:41:44.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:44.820+0000 7faae1600640 1 -- 172.21.15.5:0/1681718492 <== mon.2 v2:172.21.15.102:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7faae4004480 con 0x55f7bbe2ee40 2024-09-10T09:41:44.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:44.820+0000 7faae1600640 1 -- 172.21.15.5:0/1681718492 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7faae40048d0 con 0x55f7bbe2ee40 2024-09-10T09:41:44.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:44.820+0000 7faafce2b740 10 client.26051 fetch_fsmap learned FSMap version 32 2024-09-10T09:41:44.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:44.820+0000 7faafce2b740 1 -- 172.21.15.5:0/1681718492 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55f7bbeb4ad0 con 0x55f7bbe2ee40 2024-09-10T09:41:44.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:44.821+0000 7faae1600640 1 -- 172.21.15.5:0/1681718492 <== mon.2 v2:172.21.15.102:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faaec09c650 con 0x55f7bbe2ee40 2024-09-10T09:41:44.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:44.821+0000 7faae1600640 1 -- 172.21.15.5:0/1681718492 <== mon.2 v2:172.21.15.102:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7faae4058050 con 0x55f7bbe2ee40 2024-09-10T09:41:44.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:44.821+0000 7faafce2b740 10 client.26051 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T09:41:44.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:44.821+0000 7faafce2b740 10 client.26051 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-10T09:41:44.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:44.821+0000 7faafce2b740 20 client.26051 populate_metadata read hostname 'smithi005' 2024-09-10T09:41:44.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:44.821+0000 7faafce2b740 1 --2- 172.21.15.5:0/1681718492 >> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] conn(0x55f7bbeb4e10 0x55f7bbeb7200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:41:44.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:44.821+0000 7faafce2b740 4 client.26051 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T09:41:44.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:44.821+0000 7faafce2b740 1 -- 172.21.15.5:0/1681718492 --> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f7bbd9ce60 con 0x55f7bbeb4e10 2024-09-10T09:41:44.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:44.821+0000 7faaebe00640 1 --2- 172.21.15.5:0/1681718492 >> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] conn(0x55f7bbeb4e10 0x55f7bbeb7200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:41:44.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:44.822+0000 7faaebe00640 1 --2- 172.21.15.5:0/1681718492 >> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] conn(0x55f7bbeb4e10 0x55f7bbeb7200 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a7e8fc77428c98d2 server_cookie=469241723dc58ff0 in_seq=0 out_seq=0 2024-09-10T09:41:44.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:44.822+0000 7faae1600640 10 client.26051 ms_handle_connect on v2:172.21.15.159:6838/2073739599 2024-09-10T09:41:44.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:44.822+0000 7faae1600640 1 -- 172.21.15.5:0/1681718492 <== mds.1 v2:172.21.15.159:6838/2073739599 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f7bbd9ce60 con 0x55f7bbeb4e10 2024-09-10T09:41:44.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:44.822+0000 7faae1600640 10 client.26051 handle_command_reply: tid=0 2024-09-10T09:41:44.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:44.822+0000 7faafce2b740 1 client.26051 shutdown 2024-09-10T09:41:44.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:44.822+0000 7faafce2b740 1 -- 172.21.15.5:0/1681718492 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f7bbe2ee40 msgr2=0x55f7bbe42350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:41:44.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:44.822+0000 7faafce2b740 1 --2- 172.21.15.5:0/1681718492 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f7bbe2ee40 0x55f7bbe42350 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7faae402a9e0 tx=0x7faae402a3f0 comp rx=0 tx=0).stop 2024-09-10T09:41:44.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:44.822+0000 7faafce2b740 1 -- 172.21.15.5:0/1681718492 shutdown_connections 2024-09-10T09:41:44.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:44.822+0000 7faafce2b740 1 --2- 172.21.15.5:0/1681718492 >> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] conn(0x55f7bbeb4e10 0x55f7bbeb7200 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:41:44.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:44.822+0000 7faafce2b740 1 --2- 172.21.15.5:0/1681718492 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f7bbe3a020 0x55f7bbe33be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:41:44.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:44.822+0000 7faafce2b740 1 --2- 172.21.15.5:0/1681718492 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f7bbe2f760 0x55f7bbe42890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:41:44.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:44.822+0000 7faafce2b740 1 --2- 172.21.15.5:0/1681718492 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f7bbe2ee40 0x55f7bbe42350 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:41:44.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:44.822+0000 7faae1600640 0 client.26051 ms_handle_reset on v2:172.21.15.159:6838/2073739599 2024-09-10T09:41:44.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:44.822+0000 7faafce2b740 1 -- 172.21.15.5:0/1681718492 >> 172.21.15.5:0/1681718492 conn(0x55f7bbe2b3d0 msgr2=0x55f7bbe36f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:41:44.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:44.822+0000 7faafce2b740 1 -- 172.21.15.5:0/1681718492 shutdown_connections 2024-09-10T09:41:44.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:44.823+0000 7faafce2b740 1 -- 172.21.15.5:0/1681718492 wait complete. 2024-09-10T09:41:44.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:44.823+0000 7faafce2b740 20 client.26051 trim_cache size 0 max 16384 2024-09-10T09:41:44.831 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-10T09:41:44.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:44.824+0000 7faaf8e00640 1 -- 172.21.15.5:0/2177169412 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7faad80a2b80 msgr2=0x7faad80a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:41:44.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:44.824+0000 7faaf8e00640 1 --2- 172.21.15.5:0/2177169412 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7faad80a2b80 0x7faad80a5040 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x55f7bbe2c750 tx=0x7faadc004730 comp rx=0 tx=0).stop 2024-09-10T09:41:44.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:44.824+0000 7faaf8e00640 1 -- 172.21.15.5:0/2177169412 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7faaf4112870 msgr2=0x7faaf419e7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:41:44.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:44.824+0000 7faaf8e00640 1 --2- 172.21.15.5:0/2177169412 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7faaf4112870 0x7faaf419e7c0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7faaec00eb10 tx=0x7faaec00c2d0 comp rx=0 tx=0).stop 2024-09-10T09:41:44.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:44.824+0000 7faaf8e00640 1 -- 172.21.15.5:0/2177169412 shutdown_connections 2024-09-10T09:41:44.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:44.824+0000 7faaf8e00640 1 --2- 172.21.15.5:0/2177169412 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7faad80a2b80 0x7faad80a5040 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:41:44.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:44.824+0000 7faaf8e00640 1 --2- 172.21.15.5:0/2177169412 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7faaf4112870 0x7faaf419e7c0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:41:44.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:44.824+0000 7faaf8e00640 1 --2- 172.21.15.5:0/2177169412 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7faaf4077040 0x7faaf41a4b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:41:44.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:44.824+0000 7faaf8e00640 1 --2- 172.21.15.5:0/2177169412 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faaf4076720 0x7faaf41a4640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:41:44.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:44.824+0000 7faaf8e00640 1 -- 172.21.15.5:0/2177169412 >> 172.21.15.5:0/2177169412 conn(0x7faaf40ff910 msgr2=0x7faaf4101d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:41:44.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:44.824+0000 7faaf8e00640 1 -- 172.21.15.5:0/2177169412 shutdown_connections 2024-09-10T09:41:44.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:44.824+0000 7faaf8e00640 1 -- 172.21.15.5:0/2177169412 wait complete. 2024-09-10T09:41:45.012 INFO:tasks.workunit.client.0.smithi005.stdout: 1 142375 9.14 MB/sec warmup 86 sec latency 14.882 ms 2024-09-10T09:41:45.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:41:45 smithi005 ceph-mon[30350]: pgmap v495: 129 pgs: 129 active+clean; 73 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.3 MiB/s wr, 171 op/s 2024-09-10T09:41:45.614 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-10T09:41:45.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:41:45 smithi159 ceph-mon[35513]: pgmap v495: 129 pgs: 129 active+clean; 73 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.3 MiB/s wr, 171 op/s 2024-09-10T09:41:45.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:41:45 smithi102 ceph-mon[35442]: pgmap v495: 129 pgs: 129 active+clean; 73 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.3 MiB/s wr, 171 op/s 2024-09-10T09:41:46.012 INFO:tasks.workunit.client.0.smithi005.stdout: 1 144672 9.16 MB/sec warmup 87 sec latency 30.417 ms 2024-09-10T09:41:46.025 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:41:46.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:46.603+0000 7fc45d400640 1 -- 172.21.15.5:0/3381515487 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4500319b0 con 0x7fc460101de0 2024-09-10T09:41:46.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:46.603+0000 7fc465000640 1 -- 172.21.15.5:0/3381515487 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc460101de0 msgr2=0x7fc4601021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:41:46.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:46.603+0000 7fc465000640 1 --2- 172.21.15.5:0/3381515487 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc460101de0 0x7fc4601021c0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fc450009900 tx=0x7fc45002a400 comp rx=0 tx=0).stop 2024-09-10T09:41:46.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:46.603+0000 7fc465000640 1 -- 172.21.15.5:0/3381515487 shutdown_connections 2024-09-10T09:41:46.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:46.603+0000 7fc465000640 1 --2- 172.21.15.5:0/3381515487 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc460106f10 0x7fc46010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:41:46.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:46.603+0000 7fc465000640 1 --2- 172.21.15.5:0/3381515487 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc460102790 0x7fc4601067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:41:46.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:46.603+0000 7fc465000640 1 --2- 172.21.15.5:0/3381515487 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc460101de0 0x7fc4601021c0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:41:46.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:46.603+0000 7fc465000640 1 -- 172.21.15.5:0/3381515487 >> 172.21.15.5:0/3381515487 conn(0x7fc4600fd8c0 msgr2=0x7fc4600ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:41:46.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:46.603+0000 7fc465000640 1 -- 172.21.15.5:0/3381515487 shutdown_connections 2024-09-10T09:41:46.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:46.603+0000 7fc465000640 1 -- 172.21.15.5:0/3381515487 wait complete. 2024-09-10T09:41:46.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:46.604+0000 7fc465000640 1 Processor -- start 2024-09-10T09:41:46.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:46.604+0000 7fc465000640 1 -- start start 2024-09-10T09:41:46.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:46.605+0000 7fc465000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc460101de0 0x7fc46019a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:41:46.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:46.605+0000 7fc465000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc460102790 0x7fc46019ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:41:46.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:46.605+0000 7fc465000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc460106f10 0x7fc46019eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:41:46.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:46.605+0000 7fc465000640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc460111f30 con 0x7fc460102790 2024-09-10T09:41:46.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:46.605+0000 7fc465000640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fc460111db0 con 0x7fc460101de0 2024-09-10T09:41:46.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:46.605+0000 7fc465000640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fc4601120b0 con 0x7fc460106f10 2024-09-10T09:41:46.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:46.605+0000 7fc45de00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc460101de0 0x7fc46019a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:41:46.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:46.605+0000 7fc457e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc460102790 0x7fc46019ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:41:46.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:46.605+0000 7fc45de00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc460101de0 0x7fc46019a5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:57342/0 (socket says 172.21.15.5:57342) 2024-09-10T09:41:46.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:46.605+0000 7fc45de00640 1 -- 172.21.15.5:0/1086790951 learned_addr learned my addr 172.21.15.5:0/1086790951 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:41:46.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:46.605+0000 7fc45e800640 1 --2- 172.21.15.5:0/1086790951 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc460106f10 0x7fc46019eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:41:46.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:46.606+0000 7fc45de00640 1 -- 172.21.15.5:0/1086790951 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc460106f10 msgr2=0x7fc46019eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:41:46.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:46.606+0000 7fc45de00640 1 --2- 172.21.15.5:0/1086790951 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc460106f10 0x7fc46019eec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:41:46.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:46.606+0000 7fc45de00640 1 -- 172.21.15.5:0/1086790951 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc460102790 msgr2=0x7fc46019ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:41:46.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:46.606+0000 7fc45de00640 1 --2- 172.21.15.5:0/1086790951 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc460102790 0x7fc46019ab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:41:46.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:46.606+0000 7fc45de00640 1 -- 172.21.15.5:0/1086790951 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc46019f5a0 con 0x7fc460101de0 2024-09-10T09:41:46.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:46.606+0000 7fc45de00640 1 --2- 172.21.15.5:0/1086790951 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc460101de0 0x7fc46019a5f0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fc450005ab0 tx=0x7fc4500042f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:41:46.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:46.606+0000 7fc456a00640 1 -- 172.21.15.5:0/1086790951 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc45003f030 con 0x7fc460101de0 2024-09-10T09:41:46.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:46.606+0000 7fc465000640 1 -- 172.21.15.5:0/1086790951 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc46019f830 con 0x7fc460101de0 2024-09-10T09:41:46.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:46.606+0000 7fc465000640 1 -- 172.21.15.5:0/1086790951 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4601a7170 con 0x7fc460101de0 2024-09-10T09:41:46.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:46.606+0000 7fc456a00640 1 -- 172.21.15.5:0/1086790951 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc450003ce0 con 0x7fc460101de0 2024-09-10T09:41:46.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:46.606+0000 7fc456a00640 1 -- 172.21.15.5:0/1086790951 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc450049060 con 0x7fc460101de0 2024-09-10T09:41:46.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:46.611+0000 7fc4679cf740 1 Processor -- start 2024-09-10T09:41:46.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:46.611+0000 7fc4679cf740 1 -- start start 2024-09-10T09:41:46.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:46.611+0000 7fc456a00640 1 -- 172.21.15.5:0/1086790951 <== mon.2 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7fc450042030 con 0x7fc460101de0 2024-09-10T09:41:46.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:46.611+0000 7fc4679cf740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555aabb28c70 0x555aabb29050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:41:46.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:46.611+0000 7fc4679cf740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555aabb29590 0x555aabb33910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:41:46.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:46.611+0000 7fc4679cf740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555aabb33e50 0x555aabb36240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:41:46.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:46.611+0000 7fc4679cf740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x555aaba8f3e0 con 0x555aabb28c70 2024-09-10T09:41:46.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:46.611+0000 7fc4679cf740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x555aaba36d80 con 0x555aabb29590 2024-09-10T09:41:46.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:46.611+0000 7fc4679cf740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x555aaba8b3a0 con 0x555aabb33e50 2024-09-10T09:41:46.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:46.611+0000 7fc45e800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555aabb28c70 0x555aabb29050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:41:46.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:46.612+0000 7fc45e800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555aabb28c70 0x555aabb29050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33738/0 (socket says 172.21.15.5:33738) 2024-09-10T09:41:46.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:46.612+0000 7fc457e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555aabb33e50 0x555aabb36240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:41:46.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:46.612+0000 7fc45e800640 1 -- 172.21.15.5:0/1325989917 learned_addr learned my addr 172.21.15.5:0/1325989917 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:41:46.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:46.612+0000 7fc45de00640 1 --2- 172.21.15.5:0/1325989917 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555aabb29590 0x555aabb33910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:41:46.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:46.612+0000 7fc457e00640 1 -- 172.21.15.5:0/1325989917 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555aabb29590 msgr2=0x555aabb33910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:41:46.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:46.612+0000 7fc457e00640 1 --2- 172.21.15.5:0/1325989917 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555aabb29590 0x555aabb33910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:41:46.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:46.612+0000 7fc457e00640 1 -- 172.21.15.5:0/1325989917 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555aabb28c70 msgr2=0x555aabb29050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:41:46.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:46.612+0000 7fc457e00640 1 --2- 172.21.15.5:0/1325989917 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555aabb28c70 0x555aabb29050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:41:46.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:46.612+0000 7fc457e00640 1 -- 172.21.15.5:0/1325989917 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555aaba55ae0 con 0x555aabb33e50 2024-09-10T09:41:46.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:46.612+0000 7fc457e00640 1 --2- 172.21.15.5:0/1325989917 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555aabb33e50 0x555aabb36240 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fc44800abc0 tx=0x7fc44800f790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=11482704ead329d3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:41:46.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:46.613+0000 7fc456a00640 1 --2- 172.21.15.5:0/1086790951 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fc4440a2c00 0x7fc4440a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:41:46.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:46.613+0000 7fc44f400640 1 -- 172.21.15.5:0/1325989917 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc448005220 con 0x555aabb33e50 2024-09-10T09:41:46.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:46.613+0000 7fc44f400640 1 -- 172.21.15.5:0/1325989917 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc448004c80 con 0x555aabb33e50 2024-09-10T09:41:46.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:46.613+0000 7fc44f400640 1 -- 172.21.15.5:0/1325989917 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc448018900 con 0x555aabb33e50 2024-09-10T09:41:46.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:46.613+0000 7fc456a00640 1 -- 172.21.15.5:0/1086790951 <== mon.2 v2:172.21.15.102:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fc4500ef500 con 0x7fc460101de0 2024-09-10T09:41:46.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:46.613+0000 7fc45e800640 1 --2- 172.21.15.5:0/1086790951 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fc4440a2c00 0x7fc4440a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:41:46.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:46.613+0000 7fc4679cf740 1 -- 172.21.15.5:0/1325989917 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555aabb33e50 msgr2=0x555aabb36240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:41:46.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:46.613+0000 7fc4679cf740 1 --2- 172.21.15.5:0/1325989917 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555aabb33e50 0x555aabb36240 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fc44800abc0 tx=0x7fc44800f790 comp rx=0 tx=0).stop 2024-09-10T09:41:46.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:46.614+0000 7fc45e800640 1 --2- 172.21.15.5:0/1086790951 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fc4440a2c00 0x7fc4440a50c0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fc458010430 tx=0x7fc458007520 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:41:46.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:46.614+0000 7fc4679cf740 1 -- 172.21.15.5:0/1325989917 shutdown_connections 2024-09-10T09:41:46.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:46.614+0000 7fc4679cf740 1 --2- 172.21.15.5:0/1325989917 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555aabb33e50 0x555aabb36240 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:41:46.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:46.614+0000 7fc4679cf740 1 --2- 172.21.15.5:0/1325989917 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555aabb29590 0x555aabb33910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:41:46.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:46.614+0000 7fc4679cf740 1 --2- 172.21.15.5:0/1325989917 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555aabb28c70 0x555aabb29050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:41:46.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:46.614+0000 7fc4679cf740 1 -- 172.21.15.5:0/1325989917 >> 172.21.15.5:0/1325989917 conn(0x555aabb25200 msgr2=0x555aabb255e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:41:46.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:46.614+0000 7fc4679cf740 1 -- 172.21.15.5:0/1325989917 shutdown_connections 2024-09-10T09:41:46.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:46.614+0000 7fc4679cf740 1 -- 172.21.15.5:0/1325989917 wait complete. 2024-09-10T09:41:46.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:46.615+0000 7fc4679cf740 1 -- start start 2024-09-10T09:41:46.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:46.615+0000 7fc4679cf740 1 Processor -- start 2024-09-10T09:41:46.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:46.615+0000 7fc4679cf740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555aabb28c70 0x555aabb30c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:41:46.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:46.615+0000 7fc4679cf740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555aabb29590 0x555aabb311c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:41:46.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:46.615+0000 7fc4679cf740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555aabb33e50 0x555aabb31700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:41:46.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:46.615+0000 7fc4679cf740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x555aabb36a20 con 0x555aabb29590 2024-09-10T09:41:46.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:46.615+0000 7fc4679cf740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x555aaba4a0f0 con 0x555aabb28c70 2024-09-10T09:41:46.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:46.615+0000 7fc4679cf740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x555aabb36ba0 con 0x555aabb33e50 2024-09-10T09:41:46.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:46.616+0000 7fc4679cf740 10 client.0 _ll_register_callbacks cb 0x555aaba31bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:41:46.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:46.616+0000 7fc45de00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555aabb28c70 0x555aabb30c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:41:46.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:46.616+0000 7fc45de00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555aabb28c70 0x555aabb30c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:57354/0 (socket says 172.21.15.5:57354) 2024-09-10T09:41:46.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:46.616+0000 7fc45de00640 1 -- 172.21.15.5:0/3568478890 learned_addr learned my addr 172.21.15.5:0/3568478890 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:41:46.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:46.616+0000 7fc457e00640 1 --2- 172.21.15.5:0/3568478890 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555aabb29590 0x555aabb311c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:41:46.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:46.616+0000 7fc45e800640 1 --2- 172.21.15.5:0/3568478890 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555aabb33e50 0x555aabb31700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:41:46.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:46.616+0000 7fc457e00640 1 -- 172.21.15.5:0/3568478890 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555aabb33e50 msgr2=0x555aabb31700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:41:46.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:46.616+0000 7fc457e00640 1 --2- 172.21.15.5:0/3568478890 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555aabb33e50 0x555aabb31700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:41:46.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:46.616+0000 7fc457e00640 1 -- 172.21.15.5:0/3568478890 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555aabb28c70 msgr2=0x555aabb30c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:41:46.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:46.616+0000 7fc457e00640 1 --2- 172.21.15.5:0/3568478890 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555aabb28c70 0x555aabb30c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:41:46.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:46.616+0000 7fc457e00640 1 -- 172.21.15.5:0/3568478890 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555aab8003c0 con 0x555aabb29590 2024-09-10T09:41:46.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:46.616+0000 7fc45e800640 1 --2- 172.21.15.5:0/3568478890 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555aabb33e50 0x555aabb31700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:41:46.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:46.617+0000 7fc457e00640 1 --2- 172.21.15.5:0/3568478890 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555aabb29590 0x555aabb311c0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fc44800fc90 tx=0x7fc448005060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7982e0bd98d7b429 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:41:46.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:46.617+0000 7fc443e00640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-10T09:41:46.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:46.617+0000 7fc443e00640 1 -- 172.21.15.5:0/3568478890 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc448027070 con 0x555aabb29590 2024-09-10T09:41:46.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:46.617+0000 7fc4679cf740 1 -- 172.21.15.5:0/3568478890 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc448004c80 con 0x555aabb29590 2024-09-10T09:41:46.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:46.617+0000 7fc443e00640 1 -- 172.21.15.5:0/3568478890 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fc448018680 con 0x555aabb29590 2024-09-10T09:41:46.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:46.617+0000 7fc443e00640 1 -- 172.21.15.5:0/3568478890 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fc448033050 con 0x555aabb29590 2024-09-10T09:41:46.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:46.617+0000 7fc4679cf740 10 client.16059 fetch_fsmap learned FSMap version 32 2024-09-10T09:41:46.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:46.617+0000 7fc4679cf740 1 -- 172.21.15.5:0/3568478890 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x555aabb429e0 con 0x555aabb29590 2024-09-10T09:41:46.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:46.618+0000 7fc443e00640 1 -- 172.21.15.5:0/3568478890 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc448004c80 con 0x555aabb29590 2024-09-10T09:41:46.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:46.618+0000 7fc443e00640 1 -- 172.21.15.5:0/3568478890 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7fc448023430 con 0x555aabb29590 2024-09-10T09:41:46.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:46.618+0000 7fc4679cf740 10 client.16059 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T09:41:46.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:46.618+0000 7fc4679cf740 10 client.16059 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T09:41:46.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:46.618+0000 7fc4679cf740 20 client.16059 populate_metadata read hostname 'smithi005' 2024-09-10T09:41:46.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:46.618+0000 7fc4679cf740 1 --2- 172.21.15.5:0/3568478890 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x555aabb43df0 0x555aabb461e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:41:46.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:46.618+0000 7fc4679cf740 4 client.16059 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:41:46.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:46.618+0000 7fc4679cf740 1 -- 172.21.15.5:0/3568478890 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555aaba9c950 con 0x555aabb43df0 2024-09-10T09:41:46.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:46.619+0000 7fc45de00640 1 --2- 172.21.15.5:0/3568478890 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x555aabb43df0 0x555aabb461e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:41:46.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:46.619+0000 7fc45de00640 1 --2- 172.21.15.5:0/3568478890 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x555aabb43df0 0x555aabb461e0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7993e2b851c5b7c9 server_cookie=fe9c02126e2be826 in_seq=0 out_seq=0 2024-09-10T09:41:46.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:46.619+0000 7fc443e00640 10 client.16059 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T09:41:46.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:46.620+0000 7fc443e00640 1 -- 172.21.15.5:0/3568478890 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555aaba9c950 con 0x555aabb43df0 2024-09-10T09:41:46.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:46.620+0000 7fc443e00640 10 client.16059 handle_command_reply: tid=0 2024-09-10T09:41:46.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:46.620+0000 7fc4679cf740 1 client.16059 shutdown 2024-09-10T09:41:46.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:46.620+0000 7fc4679cf740 1 -- 172.21.15.5:0/3568478890 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555aabb29590 msgr2=0x555aabb311c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:41:46.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:46.620+0000 7fc4679cf740 1 --2- 172.21.15.5:0/3568478890 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555aabb29590 0x555aabb311c0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fc44800fc90 tx=0x7fc448005060 comp rx=0 tx=0).stop 2024-09-10T09:41:46.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:46.620+0000 7fc4679cf740 1 -- 172.21.15.5:0/3568478890 shutdown_connections 2024-09-10T09:41:46.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:46.620+0000 7fc4679cf740 1 --2- 172.21.15.5:0/3568478890 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x555aabb43df0 0x555aabb461e0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:41:46.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:46.620+0000 7fc4679cf740 1 --2- 172.21.15.5:0/3568478890 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555aabb33e50 0x555aabb31700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:41:46.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:46.620+0000 7fc4679cf740 1 --2- 172.21.15.5:0/3568478890 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555aabb29590 0x555aabb311c0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:41:46.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:46.620+0000 7fc4679cf740 1 --2- 172.21.15.5:0/3568478890 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555aabb28c70 0x555aabb30c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:41:46.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:46.620+0000 7fc443e00640 0 client.16059 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T09:41:46.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:46.620+0000 7fc4679cf740 1 -- 172.21.15.5:0/3568478890 >> 172.21.15.5:0/3568478890 conn(0x555aabb25200 msgr2=0x555aabb2af30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:41:46.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:46.620+0000 7fc4679cf740 1 -- 172.21.15.5:0/3568478890 shutdown_connections 2024-09-10T09:41:46.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:46.620+0000 7fc4679cf740 1 -- 172.21.15.5:0/3568478890 wait complete. 2024-09-10T09:41:46.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:46.620+0000 7fc4679cf740 20 client.16059 trim_cache size 0 max 16384 2024-09-10T09:41:46.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:46.634+0000 7fc4679cf740 1 Processor -- start 2024-09-10T09:41:46.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:46.634+0000 7fc4679cf740 1 -- start start 2024-09-10T09:41:46.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:46.635+0000 7fc4679cf740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555aabb28c70 0x555aabb4b890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:41:46.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:46.635+0000 7fc4679cf740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555aabb29590 0x555aabb4bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:41:46.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:46.635+0000 7fc4679cf740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555aabb33e50 0x555aabb324e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:41:46.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:46.635+0000 7fc4679cf740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc4601b0f50 con 0x555aabb28c70 2024-09-10T09:41:46.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:46.635+0000 7fc4679cf740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fc4601b2b50 con 0x555aabb33e50 2024-09-10T09:41:46.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:46.635+0000 7fc4679cf740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fc4601b1620 con 0x555aabb29590 2024-09-10T09:41:46.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:46.635+0000 7fc45e800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555aabb33e50 0x555aabb324e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:41:46.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:46.635+0000 7fc45e800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555aabb33e50 0x555aabb324e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:57368/0 (socket says 172.21.15.5:57368) 2024-09-10T09:41:46.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:46.635+0000 7fc45e800640 1 -- 172.21.15.5:0/997867565 learned_addr learned my addr 172.21.15.5:0/997867565 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:41:46.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:46.635+0000 7fc45de00640 1 --2- 172.21.15.5:0/997867565 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555aabb28c70 0x555aabb4b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:41:46.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:46.635+0000 7fc457e00640 1 --2- 172.21.15.5:0/997867565 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555aabb29590 0x555aabb4bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:41:46.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:46.635+0000 7fc45e800640 1 -- 172.21.15.5:0/997867565 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555aabb29590 msgr2=0x555aabb4bdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:41:46.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:46.635+0000 7fc45e800640 1 --2- 172.21.15.5:0/997867565 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555aabb29590 0x555aabb4bdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:41:46.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:46.635+0000 7fc45e800640 1 -- 172.21.15.5:0/997867565 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555aabb28c70 msgr2=0x555aabb4b890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:41:46.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:46.635+0000 7fc45e800640 1 --2- 172.21.15.5:0/997867565 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555aabb28c70 0x555aabb4b890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:41:46.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:46.636+0000 7fc45e800640 1 -- 172.21.15.5:0/997867565 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555aabb20770 con 0x555aabb33e50 2024-09-10T09:41:46.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:46.636+0000 7fc45e800640 1 --2- 172.21.15.5:0/997867565 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555aabb33e50 0x555aabb324e0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fc45800fbe0 tx=0x7fc458018000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6ebb39add0a47020 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:41:46.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:46.636+0000 7fc44ea00640 1 -- 172.21.15.5:0/997867565 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc45801da60 con 0x555aabb33e50 2024-09-10T09:41:46.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:46.636+0000 7fc44ea00640 1 -- 172.21.15.5:0/997867565 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc45801dc00 con 0x555aabb33e50 2024-09-10T09:41:46.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:46.636+0000 7fc44ea00640 1 -- 172.21.15.5:0/997867565 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc45801b760 con 0x555aabb33e50 2024-09-10T09:41:46.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:46.637+0000 7fc4679cf740 1 -- 172.21.15.5:0/997867565 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555aabb33e50 msgr2=0x555aabb324e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:41:46.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:46.637+0000 7fc4679cf740 1 --2- 172.21.15.5:0/997867565 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555aabb33e50 0x555aabb324e0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fc45800fbe0 tx=0x7fc458018000 comp rx=0 tx=0).stop 2024-09-10T09:41:46.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:46.637+0000 7fc4679cf740 1 -- 172.21.15.5:0/997867565 shutdown_connections 2024-09-10T09:41:46.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:46.637+0000 7fc4679cf740 1 --2- 172.21.15.5:0/997867565 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555aabb33e50 0x555aabb324e0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:41:46.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:46.637+0000 7fc4679cf740 1 --2- 172.21.15.5:0/997867565 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555aabb29590 0x555aabb4bdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:41:46.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:46.637+0000 7fc4679cf740 1 --2- 172.21.15.5:0/997867565 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555aabb28c70 0x555aabb4b890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:41:46.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:46.637+0000 7fc4679cf740 1 -- 172.21.15.5:0/997867565 >> 172.21.15.5:0/997867565 conn(0x555aabb25200 msgr2=0x555aabb24380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:41:46.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:46.637+0000 7fc4679cf740 1 -- 172.21.15.5:0/997867565 shutdown_connections 2024-09-10T09:41:46.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:46.637+0000 7fc4679cf740 1 -- 172.21.15.5:0/997867565 wait complete. 2024-09-10T09:41:46.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:46.638+0000 7fc4679cf740 1 -- start start 2024-09-10T09:41:46.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:46.638+0000 7fc4679cf740 1 Processor -- start 2024-09-10T09:41:46.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:46.638+0000 7fc4679cf740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555aabb28c70 0x555aabb2d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:41:46.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:46.638+0000 7fc4679cf740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555aabb29590 0x555aabb2dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:41:46.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:46.638+0000 7fc4679cf740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555aabb33e50 0x555aabb3ae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:41:46.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:46.638+0000 7fc4679cf740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc4601b23c0 con 0x555aabb33e50 2024-09-10T09:41:46.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:46.638+0000 7fc4679cf740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fc4601b1cf0 con 0x555aabb29590 2024-09-10T09:41:46.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:46.638+0000 7fc4679cf740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fc4601aef30 con 0x555aabb28c70 2024-09-10T09:41:46.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:46.638+0000 7fc4679cf740 10 client.0 _ll_register_callbacks cb 0x7fc4580078e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:41:46.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:46.639+0000 7fc45e800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555aabb33e50 0x555aabb3ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:41:46.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:46.639+0000 7fc457e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555aabb29590 0x555aabb2dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:41:46.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:46.639+0000 7fc457e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555aabb29590 0x555aabb2dbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:57378/0 (socket says 172.21.15.5:57378) 2024-09-10T09:41:46.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:46.639+0000 7fc457e00640 1 -- 172.21.15.5:0/3110818226 learned_addr learned my addr 172.21.15.5:0/3110818226 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:41:46.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:46.639+0000 7fc45de00640 1 --2- 172.21.15.5:0/3110818226 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555aabb28c70 0x555aabb2d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:41:46.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:46.639+0000 7fc45e800640 1 -- 172.21.15.5:0/3110818226 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555aabb28c70 msgr2=0x555aabb2d6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:41:46.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:46.639+0000 7fc45e800640 1 --2- 172.21.15.5:0/3110818226 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555aabb28c70 0x555aabb2d6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:41:46.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:46.639+0000 7fc45e800640 1 -- 172.21.15.5:0/3110818226 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555aabb29590 msgr2=0x555aabb2dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:41:46.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:46.639+0000 7fc45e800640 1 --2- 172.21.15.5:0/3110818226 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555aabb29590 0x555aabb2dbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:41:46.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:46.639+0000 7fc45e800640 1 -- 172.21.15.5:0/3110818226 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555aabb3b460 con 0x555aabb33e50 2024-09-10T09:41:46.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:46.639+0000 7fc45e800640 1 --2- 172.21.15.5:0/3110818226 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555aabb33e50 0x555aabb3ae20 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fc45800fbb0 tx=0x7fc458004cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63d8dff9574ed7a3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:41:46.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:46.639+0000 7fc443e00640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-10T09:41:46.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:46.640+0000 7fc443e00640 1 -- 172.21.15.5:0/3110818226 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc45801da60 con 0x555aabb33e50 2024-09-10T09:41:46.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:46.640+0000 7fc443e00640 1 -- 172.21.15.5:0/3110818226 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fc458003d80 con 0x555aabb33e50 2024-09-10T09:41:46.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:46.640+0000 7fc4679cf740 1 -- 172.21.15.5:0/3110818226 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc45801dc00 con 0x555aabb33e50 2024-09-10T09:41:46.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:46.640+0000 7fc443e00640 1 -- 172.21.15.5:0/3110818226 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fc458039050 con 0x555aabb33e50 2024-09-10T09:41:46.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:46.640+0000 7fc4679cf740 10 client.16065 fetch_fsmap learned FSMap version 32 2024-09-10T09:41:46.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:46.640+0000 7fc4679cf740 1 -- 172.21.15.5:0/3110818226 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x555aabb3c720 con 0x555aabb33e50 2024-09-10T09:41:46.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:46.640+0000 7fc443e00640 1 -- 172.21.15.5:0/3110818226 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc45801dc00 con 0x555aabb33e50 2024-09-10T09:41:46.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:46.641+0000 7fc443e00640 1 -- 172.21.15.5:0/3110818226 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7fc458004370 con 0x555aabb33e50 2024-09-10T09:41:46.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:46.641+0000 7fc4679cf740 10 client.16065 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T09:41:46.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:46.641+0000 7fc4679cf740 10 client.16065 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T09:41:46.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:46.641+0000 7fc4679cf740 20 client.16065 populate_metadata read hostname 'smithi005' 2024-09-10T09:41:46.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:46.641+0000 7fc4679cf740 1 --2- 172.21.15.5:0/3110818226 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x555aabbae9a0 0x555aabbb0dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:41:46.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:46.641+0000 7fc4679cf740 4 client.16065 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T09:41:46.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:46.641+0000 7fc4679cf740 1 -- 172.21.15.5:0/3110818226 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555aaba96e60 con 0x555aabbae9a0 2024-09-10T09:41:46.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:46.641+0000 7fc45de00640 1 --2- 172.21.15.5:0/3110818226 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x555aabbae9a0 0x555aabbb0dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:41:46.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:46.641+0000 7fc45de00640 1 --2- 172.21.15.5:0/3110818226 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x555aabbae9a0 0x555aabbb0dc0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=53c703158e1c413f server_cookie=e33042c0de1f8851 in_seq=0 out_seq=0 2024-09-10T09:41:46.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:46.641+0000 7fc443e00640 10 client.16065 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T09:41:46.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:46.642+0000 7fc443e00640 1 -- 172.21.15.5:0/3110818226 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555aaba96e60 con 0x555aabbae9a0 2024-09-10T09:41:46.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:46.642+0000 7fc443e00640 10 client.16065 handle_command_reply: tid=0 2024-09-10T09:41:46.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:46.642+0000 7fc4679cf740 1 client.16065 shutdown 2024-09-10T09:41:46.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:46.642+0000 7fc4679cf740 1 -- 172.21.15.5:0/3110818226 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555aabb33e50 msgr2=0x555aabb3ae20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:41:46.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:46.642+0000 7fc4679cf740 1 --2- 172.21.15.5:0/3110818226 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555aabb33e50 0x555aabb3ae20 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fc45800fbb0 tx=0x7fc458004cf0 comp rx=0 tx=0).stop 2024-09-10T09:41:46.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:46.642+0000 7fc4679cf740 1 -- 172.21.15.5:0/3110818226 shutdown_connections 2024-09-10T09:41:46.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:46.642+0000 7fc4679cf740 1 --2- 172.21.15.5:0/3110818226 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x555aabbae9a0 0x555aabbb0dc0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:41:46.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:46.642+0000 7fc4679cf740 1 --2- 172.21.15.5:0/3110818226 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555aabb33e50 0x555aabb3ae20 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:41:46.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:46.642+0000 7fc4679cf740 1 --2- 172.21.15.5:0/3110818226 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555aabb29590 0x555aabb2dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:41:46.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:46.642+0000 7fc4679cf740 1 --2- 172.21.15.5:0/3110818226 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555aabb28c70 0x555aabb2d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:41:46.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:46.642+0000 7fc4679cf740 1 -- 172.21.15.5:0/3110818226 >> 172.21.15.5:0/3110818226 conn(0x555aabb25200 msgr2=0x555aabb324e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:41:46.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:46.642+0000 7fc443e00640 0 client.16065 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T09:41:46.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:46.642+0000 7fc4679cf740 1 -- 172.21.15.5:0/3110818226 shutdown_connections 2024-09-10T09:41:46.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:46.642+0000 7fc4679cf740 1 -- 172.21.15.5:0/3110818226 wait complete. 2024-09-10T09:41:46.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:46.643+0000 7fc4679cf740 20 client.16065 trim_cache size 0 max 16384 2024-09-10T09:41:46.647 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-10T09:41:46.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:46.643+0000 7fc465000640 1 -- 172.21.15.5:0/1086790951 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fc4440a2c00 msgr2=0x7fc4440a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:41:46.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:46.643+0000 7fc465000640 1 --2- 172.21.15.5:0/1086790951 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fc4440a2c00 0x7fc4440a50c0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fc458010430 tx=0x7fc458007520 comp rx=0 tx=0).stop 2024-09-10T09:41:46.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:46.643+0000 7fc465000640 1 -- 172.21.15.5:0/1086790951 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc460101de0 msgr2=0x7fc46019a5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:41:46.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:46.643+0000 7fc465000640 1 --2- 172.21.15.5:0/1086790951 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc460101de0 0x7fc46019a5f0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fc450005ab0 tx=0x7fc4500042f0 comp rx=0 tx=0).stop 2024-09-10T09:41:46.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:46.643+0000 7fc465000640 1 -- 172.21.15.5:0/1086790951 shutdown_connections 2024-09-10T09:41:46.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:46.643+0000 7fc465000640 1 --2- 172.21.15.5:0/1086790951 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fc4440a2c00 0x7fc4440a50c0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:41:46.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:46.643+0000 7fc465000640 1 --2- 172.21.15.5:0/1086790951 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc460106f10 0x7fc46019eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:41:46.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:46.643+0000 7fc465000640 1 --2- 172.21.15.5:0/1086790951 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc460102790 0x7fc46019ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:41:46.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:46.643+0000 7fc465000640 1 --2- 172.21.15.5:0/1086790951 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc460101de0 0x7fc46019a5f0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:41:46.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:46.643+0000 7fc465000640 1 -- 172.21.15.5:0/1086790951 >> 172.21.15.5:0/1086790951 conn(0x7fc4600fd8c0 msgr2=0x7fc4600fe5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:41:46.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:46.644+0000 7fc465000640 1 -- 172.21.15.5:0/1086790951 shutdown_connections 2024-09-10T09:41:46.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:46.644+0000 7fc465000640 1 -- 172.21.15.5:0/1086790951 wait complete. 2024-09-10T09:41:47.012 INFO:tasks.workunit.client.0.smithi005.stdout: 1 145076 9.08 MB/sec warmup 88 sec latency 840.861 ms 2024-09-10T09:41:47.507 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:41:47 smithi005 ceph-mon[30350]: pgmap v496: 129 pgs: 129 active+clean; 75 MiB data, 698 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.6 MiB/s wr, 228 op/s 2024-09-10T09:41:47.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:41:47 smithi159 ceph-mon[35513]: pgmap v496: 129 pgs: 129 active+clean; 75 MiB data, 698 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.6 MiB/s wr, 228 op/s 2024-09-10T09:41:47.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:41:47 smithi102 ceph-mon[35442]: pgmap v496: 129 pgs: 129 active+clean; 75 MiB data, 698 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.6 MiB/s wr, 228 op/s 2024-09-10T09:41:48.012 INFO:tasks.workunit.client.0.smithi005.stdout: 1 145076 8.98 MB/sec warmup 89 sec latency 1840.901 ms 2024-09-10T09:41:48.893 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-10T09:41:48.893 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-10T09:41:49.013 INFO:tasks.workunit.client.0.smithi005.stdout: 1 146121 8.94 MB/sec warmup 90 sec latency 2233.539 ms 2024-09-10T09:41:49.366 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:41:49.692 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:41:49 smithi005 ceph-mon[30350]: pgmap v497: 129 pgs: 129 active+clean; 75 MiB data, 679 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.2 MiB/s wr, 182 op/s 2024-09-10T09:41:49.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:41:49 smithi159 ceph-mon[35513]: pgmap v497: 129 pgs: 129 active+clean; 75 MiB data, 679 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.2 MiB/s wr, 182 op/s 2024-09-10T09:41:49.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:41:49 smithi102 ceph-mon[35442]: pgmap v497: 129 pgs: 129 active+clean; 75 MiB data, 679 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.2 MiB/s wr, 182 op/s 2024-09-10T09:41:49.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:49.988+0000 7f5871000640 1 -- 172.21.15.5:0/1597203471 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f586c1049a0 msgr2=0x7f586c1089f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:41:49.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:49.988+0000 7f5871000640 1 --2- 172.21.15.5:0/1597203471 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f586c1049a0 0x7f586c1089f0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f5854009900 tx=0x7f585402a3f0 comp rx=0 tx=0).stop 2024-09-10T09:41:49.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:49.988+0000 7f5871000640 1 -- 172.21.15.5:0/1597203471 shutdown_connections 2024-09-10T09:41:49.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:49.988+0000 7f5871000640 1 --2- 172.21.15.5:0/1597203471 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f586c109120 0x7f586c110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:41:49.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:49.988+0000 7f5871000640 1 --2- 172.21.15.5:0/1597203471 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f586c1049a0 0x7f586c1089f0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:41:49.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:49.988+0000 7f5871000640 1 --2- 172.21.15.5:0/1597203471 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f586c103ff0 0x7f586c1043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:41:49.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:49.988+0000 7f5871000640 1 -- 172.21.15.5:0/1597203471 >> 172.21.15.5:0/1597203471 conn(0x7f586c0ffa90 msgr2=0x7f586c101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:41:49.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:49.988+0000 7f5871000640 1 -- 172.21.15.5:0/1597203471 shutdown_connections 2024-09-10T09:41:49.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:49.988+0000 7f5871000640 1 -- 172.21.15.5:0/1597203471 wait complete. 2024-09-10T09:41:49.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:49.989+0000 7f5871000640 1 Processor -- start 2024-09-10T09:41:49.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:49.989+0000 7f5871000640 1 -- start start 2024-09-10T09:41:49.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:49.990+0000 7f5871000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f586c103ff0 0x7f586c19a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:41:49.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:49.990+0000 7f5871000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f586c1049a0 0x7f586c19ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:41:49.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:49.990+0000 7f5871000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f586c109120 0x7f586c19ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:41:49.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:49.990+0000 7f5871000640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f586c114170 con 0x7f586c1049a0 2024-09-10T09:41:49.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:49.990+0000 7f5871000640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f586c113ff0 con 0x7f586c103ff0 2024-09-10T09:41:49.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:49.990+0000 7f5871000640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f586c1142f0 con 0x7f586c109120 2024-09-10T09:41:49.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:49.990+0000 7f5869e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f586c103ff0 0x7f586c19a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:41:49.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:49.990+0000 7f5863e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f586c1049a0 0x7f586c19ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:41:49.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:49.990+0000 7f586a800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f586c109120 0x7f586c19ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:41:49.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:49.990+0000 7f5863e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f586c1049a0 0x7f586c19ab90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33788/0 (socket says 172.21.15.5:33788) 2024-09-10T09:41:49.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:49.990+0000 7f5863e00640 1 -- 172.21.15.5:0/1583781864 learned_addr learned my addr 172.21.15.5:0/1583781864 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:41:49.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:49.991+0000 7f586a800640 1 -- 172.21.15.5:0/1583781864 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f586c103ff0 msgr2=0x7f586c19a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:41:49.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:49.991+0000 7f586a800640 1 --2- 172.21.15.5:0/1583781864 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f586c103ff0 0x7f586c19a650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:41:49.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:49.991+0000 7f586a800640 1 -- 172.21.15.5:0/1583781864 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f586c1049a0 msgr2=0x7f586c19ab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:41:49.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:49.991+0000 7f586a800640 1 --2- 172.21.15.5:0/1583781864 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f586c1049a0 0x7f586c19ab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:41:49.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:49.991+0000 7f586a800640 1 -- 172.21.15.5:0/1583781864 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f586c19f600 con 0x7f586c109120 2024-09-10T09:41:49.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:49.991+0000 7f5863e00640 1 --2- 172.21.15.5:0/1583781864 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f586c1049a0 0x7f586c19ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:41:49.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:49.991+0000 7f586a800640 1 --2- 172.21.15.5:0/1583781864 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f586c109120 0x7f586c19ef20 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f5864009fd0 tx=0x7f586400ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:41:49.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:49.991+0000 7f5862a00640 1 -- 172.21.15.5:0/1583781864 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f586400ce20 con 0x7f586c109120 2024-09-10T09:41:49.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:49.991+0000 7f5862a00640 1 -- 172.21.15.5:0/1583781864 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5864005020 con 0x7f586c109120 2024-09-10T09:41:49.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:49.991+0000 7f5871000640 1 -- 172.21.15.5:0/1583781864 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f586c19f8f0 con 0x7f586c109120 2024-09-10T09:41:49.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:49.992+0000 7f5871000640 1 -- 172.21.15.5:0/1583781864 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f586c1a71d0 con 0x7f586c109120 2024-09-10T09:41:49.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:49.991+0000 7f5862a00640 1 -- 172.21.15.5:0/1583781864 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5864010720 con 0x7f586c109120 2024-09-10T09:41:49.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:49.995+0000 7f5873a9e740 1 Processor -- start 2024-09-10T09:41:49.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:49.995+0000 7f5862a00640 1 -- 172.21.15.5:0/1583781864 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f5864005540 con 0x7f586c109120 2024-09-10T09:41:49.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:49.995+0000 7f5873a9e740 1 -- start start 2024-09-10T09:41:49.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:49.995+0000 7f5873a9e740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559e62ee4ac0 0x559e62ee4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:41:49.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:49.995+0000 7f5873a9e740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559e62ee53e0 0x559e62eef760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:41:49.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:49.995+0000 7f5873a9e740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559e62eefca0 0x559e62ef2090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:41:49.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:49.995+0000 7f5873a9e740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x559e62e4b290 con 0x559e62ee53e0 2024-09-10T09:41:49.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:49.995+0000 7f5873a9e740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x559e62df2b40 con 0x559e62eefca0 2024-09-10T09:41:49.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:49.995+0000 7f5873a9e740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x559e62e47250 con 0x559e62ee4ac0 2024-09-10T09:41:49.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:49.996+0000 7f5869e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559e62ee53e0 0x559e62eef760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:41:49.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:49.996+0000 7f5863e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559e62ee4ac0 0x559e62ee4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:41:49.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:49.996+0000 7f5863e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559e62ee4ac0 0x559e62ee4ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:50766/0 (socket says 172.21.15.5:50766) 2024-09-10T09:41:49.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:49.996+0000 7f5869e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559e62ee53e0 0x559e62eef760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33800/0 (socket says 172.21.15.5:33800) 2024-09-10T09:41:49.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:49.996+0000 7f5863e00640 1 -- 172.21.15.5:0/400069029 learned_addr learned my addr 172.21.15.5:0/400069029 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:41:49.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:49.996+0000 7f5863e00640 1 --2- 172.21.15.5:0/400069029 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559e62eefca0 0x559e62ef2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:41:49.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:49.996+0000 7f5869e00640 1 -- 172.21.15.5:0/400069029 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559e62ee4ac0 msgr2=0x559e62ee4ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:41:49.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:49.996+0000 7f5869e00640 1 --2- 172.21.15.5:0/400069029 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559e62ee4ac0 0x559e62ee4ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:41:49.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:49.996+0000 7f5869e00640 1 -- 172.21.15.5:0/400069029 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559e62eefca0 msgr2=0x559e62ef2090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:41:49.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:49.996+0000 7f5869e00640 1 --2- 172.21.15.5:0/400069029 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559e62eefca0 0x559e62ef2090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:41:49.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:49.996+0000 7f5869e00640 1 -- 172.21.15.5:0/400069029 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559e62bbc6a0 con 0x559e62ee53e0 2024-09-10T09:41:49.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:49.996+0000 7f5863e00640 1 --2- 172.21.15.5:0/400069029 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559e62eefca0 0x559e62ef2090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:41:49.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:49.997+0000 7f5869e00640 1 --2- 172.21.15.5:0/400069029 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559e62ee53e0 0x559e62eef760 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f585c009c20 tx=0x7f585c011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2159ef4ea99c39f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:41:49.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:49.997+0000 7f5862a00640 1 --2- 172.21.15.5:0/1583781864 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f58500a2c00 0x7f58500a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:41:49.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:49.997+0000 7f5862a00640 1 -- 172.21.15.5:0/1583781864 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f58640cfa50 con 0x7f586c109120 2024-09-10T09:41:49.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:49.997+0000 7f5863e00640 1 --2- 172.21.15.5:0/1583781864 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f58500a2c00 0x7f58500a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:41:49.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:49.997+0000 7f585b400640 1 -- 172.21.15.5:0/400069029 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f585c009dd0 con 0x559e62ee53e0 2024-09-10T09:41:50.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:49.997+0000 7f585b400640 1 -- 172.21.15.5:0/400069029 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f585c005020 con 0x559e62ee53e0 2024-09-10T09:41:50.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:49.997+0000 7f585b400640 1 -- 172.21.15.5:0/400069029 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f585c004c40 con 0x559e62ee53e0 2024-09-10T09:41:50.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:49.998+0000 7f5873a9e740 1 -- 172.21.15.5:0/400069029 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559e62ee53e0 msgr2=0x559e62eef760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:41:50.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:49.998+0000 7f5863e00640 1 --2- 172.21.15.5:0/1583781864 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f58500a2c00 0x7f58500a50c0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x559e62ee23d0 tx=0x7f58540046c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:41:50.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:49.998+0000 7f5873a9e740 1 --2- 172.21.15.5:0/400069029 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559e62ee53e0 0x559e62eef760 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f585c009c20 tx=0x7f585c011810 comp rx=0 tx=0).stop 2024-09-10T09:41:50.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:49.998+0000 7f5873a9e740 1 -- 172.21.15.5:0/400069029 shutdown_connections 2024-09-10T09:41:50.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:49.998+0000 7f5873a9e740 1 --2- 172.21.15.5:0/400069029 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559e62eefca0 0x559e62ef2090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:41:50.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:49.998+0000 7f5873a9e740 1 --2- 172.21.15.5:0/400069029 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559e62ee53e0 0x559e62eef760 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:41:50.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:49.998+0000 7f5873a9e740 1 --2- 172.21.15.5:0/400069029 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559e62ee4ac0 0x559e62ee4ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:41:50.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:49.998+0000 7f5873a9e740 1 -- 172.21.15.5:0/400069029 >> 172.21.15.5:0/400069029 conn(0x559e62ee1050 msgr2=0x559e62ee1430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:41:50.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:49.998+0000 7f5873a9e740 1 -- 172.21.15.5:0/400069029 shutdown_connections 2024-09-10T09:41:50.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:49.998+0000 7f5873a9e740 1 -- 172.21.15.5:0/400069029 wait complete. 2024-09-10T09:41:50.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:49.999+0000 7f5873a9e740 1 -- start start 2024-09-10T09:41:50.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:49.999+0000 7f5873a9e740 1 Processor -- start 2024-09-10T09:41:50.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:49.999+0000 7f5873a9e740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559e62ee4ac0 0x559e62eef520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:41:50.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:49.999+0000 7f5873a9e740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559e62ee53e0 0x559e62eec7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:41:50.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:49.999+0000 7f5873a9e740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559e62eefca0 0x559e62eecd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:41:50.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:50.000+0000 7f5873a9e740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x559e62ef2910 con 0x559e62ee4ac0 2024-09-10T09:41:50.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:50.000+0000 7f5873a9e740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x559e62e05fa0 con 0x559e62eefca0 2024-09-10T09:41:50.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:50.000+0000 7f5873a9e740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x559e62ef2a90 con 0x559e62ee53e0 2024-09-10T09:41:50.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:50.000+0000 7f5873a9e740 10 client.0 _ll_register_callbacks cb 0x559e62b59ba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:41:50.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:50.000+0000 7f5863e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559e62ee53e0 0x559e62eec7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:41:50.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:50.000+0000 7f5863e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559e62ee53e0 0x559e62eec7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:50770/0 (socket says 172.21.15.5:50770) 2024-09-10T09:41:50.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:50.000+0000 7f5863e00640 1 -- 172.21.15.5:0/1578249959 learned_addr learned my addr 172.21.15.5:0/1578249959 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:41:50.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:50.000+0000 7f5869e00640 1 --2- 172.21.15.5:0/1578249959 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559e62ee4ac0 0x559e62eef520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:41:50.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:50.000+0000 7f5863e00640 1 -- 172.21.15.5:0/1578249959 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559e62eefca0 msgr2=0x559e62eecd10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T09:41:50.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:50.000+0000 7f5863e00640 1 --2- 172.21.15.5:0/1578249959 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559e62eefca0 0x559e62eecd10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:41:50.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:50.000+0000 7f5863e00640 1 -- 172.21.15.5:0/1578249959 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559e62ee4ac0 msgr2=0x559e62eef520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:41:50.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:50.000+0000 7f5863e00640 1 --2- 172.21.15.5:0/1578249959 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559e62ee4ac0 0x559e62eef520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:41:50.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:50.000+0000 7f5863e00640 1 -- 172.21.15.5:0/1578249959 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559e62eed4f0 con 0x559e62ee53e0 2024-09-10T09:41:50.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:50.000+0000 7f5869e00640 1 --2- 172.21.15.5:0/1578249959 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559e62ee4ac0 0x559e62eef520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T09:41:50.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:50.001+0000 7f5863e00640 1 --2- 172.21.15.5:0/1578249959 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559e62ee53e0 0x559e62eec7d0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f5854006010 tx=0x7f5854009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=32b654c376154254 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:41:50.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:50.001+0000 7f584fe00640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-09-10T09:41:50.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:50.001+0000 7f584fe00640 1 -- 172.21.15.5:0/1578249959 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f585404e070 con 0x559e62ee53e0 2024-09-10T09:41:50.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:50.001+0000 7f5873a9e740 1 -- 172.21.15.5:0/1578249959 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f585c005020 con 0x559e62ee53e0 2024-09-10T09:41:50.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:50.001+0000 7f584fe00640 1 -- 172.21.15.5:0/1578249959 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f5854003d00 con 0x559e62ee53e0 2024-09-10T09:41:50.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:50.001+0000 7f584fe00640 1 -- 172.21.15.5:0/1578249959 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f585405a050 con 0x559e62ee53e0 2024-09-10T09:41:50.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:50.001+0000 7f5873a9e740 10 client.26101 fetch_fsmap learned FSMap version 32 2024-09-10T09:41:50.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:50.001+0000 7f5873a9e740 1 -- 172.21.15.5:0/1578249959 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x559e62efe7a0 con 0x559e62ee53e0 2024-09-10T09:41:50.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:50.002+0000 7f584fe00640 1 -- 172.21.15.5:0/1578249959 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f585c005020 con 0x559e62ee53e0 2024-09-10T09:41:50.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:50.002+0000 7f584fe00640 1 -- 172.21.15.5:0/1578249959 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7f585403c500 con 0x559e62ee53e0 2024-09-10T09:41:50.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:50.002+0000 7f5873a9e740 10 client.26101 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T09:41:50.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:50.002+0000 7f5873a9e740 10 client.26101 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T09:41:50.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:50.002+0000 7f5873a9e740 20 client.26101 populate_metadata read hostname 'smithi005' 2024-09-10T09:41:50.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:50.002+0000 7f5873a9e740 1 --2- 172.21.15.5:0/1578249959 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x559e62effd20 0x559e62f021c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:41:50.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:50.002+0000 7f5873a9e740 4 client.26101 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:41:50.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:50.002+0000 7f5873a9e740 1 -- 172.21.15.5:0/1578249959 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559e62e58800 con 0x559e62effd20 2024-09-10T09:41:50.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:50.002+0000 7f5869e00640 1 --2- 172.21.15.5:0/1578249959 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x559e62effd20 0x559e62f021c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:41:50.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:50.003+0000 7f5869e00640 1 --2- 172.21.15.5:0/1578249959 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x559e62effd20 0x559e62f021c0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=49894ec9850334c6 server_cookie=a7f9d5afe6938e5e in_seq=0 out_seq=0 2024-09-10T09:41:50.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:50.003+0000 7f584fe00640 10 client.26101 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T09:41:50.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:50.004+0000 7f584fe00640 1 -- 172.21.15.5:0/1578249959 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559e62e58800 con 0x559e62effd20 2024-09-10T09:41:50.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:50.004+0000 7f584fe00640 10 client.26101 handle_command_reply: tid=0 2024-09-10T09:41:50.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:50.004+0000 7f5873a9e740 1 client.26101 shutdown 2024-09-10T09:41:50.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:50.004+0000 7f5873a9e740 1 -- 172.21.15.5:0/1578249959 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559e62ee53e0 msgr2=0x559e62eec7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:41:50.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:50.004+0000 7f5873a9e740 1 --2- 172.21.15.5:0/1578249959 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559e62ee53e0 0x559e62eec7d0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f5854006010 tx=0x7f5854009260 comp rx=0 tx=0).stop 2024-09-10T09:41:50.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:50.004+0000 7f5873a9e740 1 -- 172.21.15.5:0/1578249959 shutdown_connections 2024-09-10T09:41:50.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:50.004+0000 7f5873a9e740 1 --2- 172.21.15.5:0/1578249959 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x559e62effd20 0x559e62f021c0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:41:50.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:50.004+0000 7f5873a9e740 1 --2- 172.21.15.5:0/1578249959 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559e62eefca0 0x559e62eecd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:41:50.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:50.004+0000 7f584fe00640 0 client.26101 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T09:41:50.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:50.004+0000 7f5873a9e740 1 --2- 172.21.15.5:0/1578249959 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559e62ee53e0 0x559e62eec7d0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:41:50.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:50.004+0000 7f5873a9e740 1 --2- 172.21.15.5:0/1578249959 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559e62ee4ac0 0x559e62eef520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:41:50.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:50.004+0000 7f5873a9e740 1 -- 172.21.15.5:0/1578249959 >> 172.21.15.5:0/1578249959 conn(0x559e62ee1050 msgr2=0x559e62ee6d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:41:50.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:50.004+0000 7f5873a9e740 1 -- 172.21.15.5:0/1578249959 shutdown_connections 2024-09-10T09:41:50.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:50.004+0000 7f5873a9e740 1 -- 172.21.15.5:0/1578249959 wait complete. 2024-09-10T09:41:50.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:50.004+0000 7f5873a9e740 20 client.26101 trim_cache size 0 max 16384 2024-09-10T09:41:50.012 INFO:tasks.workunit.client.0.smithi005.stdout: 1 148633 8.99 MB/sec warmup 91 sec latency 20.595 ms 2024-09-10T09:41:50.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:50.023+0000 7f5873a9e740 1 Processor -- start 2024-09-10T09:41:50.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:50.023+0000 7f5873a9e740 1 -- start start 2024-09-10T09:41:50.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:50.023+0000 7f5873a9e740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559e62ee4ac0 0x559e62ee7d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:41:50.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:50.023+0000 7f5873a9e740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559e62ee53e0 0x559e62f07ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:41:50.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:50.023+0000 7f5873a9e740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559e62eefca0 0x559e62f08010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:41:50.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:50.023+0000 7f5873a9e740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f586c1b0fc0 con 0x559e62ee4ac0 2024-09-10T09:41:50.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:50.023+0000 7f5873a9e740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f586c1b2bc0 con 0x559e62eefca0 2024-09-10T09:41:50.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:50.023+0000 7f5873a9e740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f586c1b1690 con 0x559e62ee53e0 2024-09-10T09:41:50.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:50.024+0000 7f5863e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559e62ee53e0 0x559e62f07ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:41:50.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:50.024+0000 7f5869e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559e62ee4ac0 0x559e62ee7d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:41:50.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:50.024+0000 7f586a800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559e62eefca0 0x559e62f08010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:41:50.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:50.024+0000 7f5863e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559e62ee53e0 0x559e62f07ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:50772/0 (socket says 172.21.15.5:50772) 2024-09-10T09:41:50.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:50.024+0000 7f586a800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559e62eefca0 0x559e62f08010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:57416/0 (socket says 172.21.15.5:57416) 2024-09-10T09:41:50.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:50.024+0000 7f5863e00640 1 -- 172.21.15.5:0/2616579468 learned_addr learned my addr 172.21.15.5:0/2616579468 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:41:50.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:50.024+0000 7f5863e00640 1 -- 172.21.15.5:0/2616579468 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559e62eefca0 msgr2=0x559e62f08010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:41:50.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:50.024+0000 7f5863e00640 1 --2- 172.21.15.5:0/2616579468 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559e62eefca0 0x559e62f08010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:41:50.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:50.024+0000 7f5863e00640 1 -- 172.21.15.5:0/2616579468 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559e62ee4ac0 msgr2=0x559e62ee7d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:41:50.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:50.024+0000 7f5863e00640 1 --2- 172.21.15.5:0/2616579468 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559e62ee4ac0 0x559e62ee7d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:41:50.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:50.024+0000 7f5863e00640 1 -- 172.21.15.5:0/2616579468 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559e62ee0e80 con 0x559e62ee53e0 2024-09-10T09:41:50.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:50.024+0000 7f5863e00640 1 --2- 172.21.15.5:0/2616579468 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559e62ee53e0 0x559e62f07ad0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f585403c8d0 tx=0x7f585403cfd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f316c28cea262f3e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:41:50.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:50.025+0000 7f585aa00640 1 -- 172.21.15.5:0/2616579468 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f585404e070 con 0x559e62ee53e0 2024-09-10T09:41:50.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:50.025+0000 7f585aa00640 1 -- 172.21.15.5:0/2616579468 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5854031670 con 0x559e62ee53e0 2024-09-10T09:41:50.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:50.025+0000 7f585aa00640 1 -- 172.21.15.5:0/2616579468 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f585403d820 con 0x559e62ee53e0 2024-09-10T09:41:50.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:50.026+0000 7f5873a9e740 1 -- 172.21.15.5:0/2616579468 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559e62ee53e0 msgr2=0x559e62f07ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:41:50.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:50.026+0000 7f5873a9e740 1 --2- 172.21.15.5:0/2616579468 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559e62ee53e0 0x559e62f07ad0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f585403c8d0 tx=0x7f585403cfd0 comp rx=0 tx=0).stop 2024-09-10T09:41:50.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:50.026+0000 7f5873a9e740 1 -- 172.21.15.5:0/2616579468 shutdown_connections 2024-09-10T09:41:50.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:50.026+0000 7f5873a9e740 1 --2- 172.21.15.5:0/2616579468 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559e62eefca0 0x559e62f08010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:41:50.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:50.026+0000 7f5873a9e740 1 --2- 172.21.15.5:0/2616579468 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559e62ee53e0 0x559e62f07ad0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:41:50.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:50.026+0000 7f5873a9e740 1 --2- 172.21.15.5:0/2616579468 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559e62ee4ac0 0x559e62ee7d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:41:50.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:50.026+0000 7f5873a9e740 1 -- 172.21.15.5:0/2616579468 >> 172.21.15.5:0/2616579468 conn(0x559e62ee1050 msgr2=0x559e62ee7650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:41:50.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:50.026+0000 7f5873a9e740 1 -- 172.21.15.5:0/2616579468 shutdown_connections 2024-09-10T09:41:50.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:50.026+0000 7f5873a9e740 1 -- 172.21.15.5:0/2616579468 wait complete. 2024-09-10T09:41:50.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:50.026+0000 7f5873a9e740 1 -- start start 2024-09-10T09:41:50.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:50.027+0000 7f5873a9e740 1 Processor -- start 2024-09-10T09:41:50.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:50.027+0000 7f5873a9e740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559e62ee4ac0 0x559e62ef3de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:41:50.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:50.027+0000 7f5873a9e740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559e62ee53e0 0x559e62ef4320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:41:50.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:50.027+0000 7f5873a9e740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559e62eefca0 0x559e62f69aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:41:50.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:50.027+0000 7f5873a9e740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f586c1b2430 con 0x559e62ee53e0 2024-09-10T09:41:50.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:50.027+0000 7f5873a9e740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f586c1b1d60 con 0x559e62ee4ac0 2024-09-10T09:41:50.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:50.027+0000 7f5873a9e740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f586c1aefa0 con 0x559e62eefca0 2024-09-10T09:41:50.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:50.027+0000 7f5873a9e740 10 client.0 _ll_register_callbacks cb 0x7f5854009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:41:50.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:50.028+0000 7f586a800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559e62eefca0 0x559e62f69aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:41:50.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:50.028+0000 7f586a800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559e62eefca0 0x559e62f69aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:50780/0 (socket says 172.21.15.5:50780) 2024-09-10T09:41:50.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:50.028+0000 7f586a800640 1 -- 172.21.15.5:0/3554923153 learned_addr learned my addr 172.21.15.5:0/3554923153 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:41:50.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:50.028+0000 7f5869e00640 1 --2- 172.21.15.5:0/3554923153 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559e62ee4ac0 0x559e62ef3de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:41:50.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:50.028+0000 7f5863e00640 1 --2- 172.21.15.5:0/3554923153 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559e62ee53e0 0x559e62ef4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:41:50.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:50.028+0000 7f586a800640 1 -- 172.21.15.5:0/3554923153 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559e62ee4ac0 msgr2=0x559e62ef3de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:41:50.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:50.028+0000 7f586a800640 1 --2- 172.21.15.5:0/3554923153 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559e62ee4ac0 0x559e62ef3de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:41:50.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:50.028+0000 7f586a800640 1 -- 172.21.15.5:0/3554923153 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559e62ee53e0 msgr2=0x559e62ef4320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:41:50.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:50.028+0000 7f586a800640 1 --2- 172.21.15.5:0/3554923153 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559e62ee53e0 0x559e62ef4320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:41:50.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:50.028+0000 7f586a800640 1 -- 172.21.15.5:0/3554923153 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559e62ef4c00 con 0x559e62eefca0 2024-09-10T09:41:50.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:50.028+0000 7f5863e00640 1 --2- 172.21.15.5:0/3554923153 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559e62ee53e0 0x559e62ef4320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:41:50.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:50.028+0000 7f586a800640 1 --2- 172.21.15.5:0/3554923153 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559e62eefca0 0x559e62f69aa0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f5864098830 tx=0x7f586409c620 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f18b72b0ae03426b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:41:50.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:50.028+0000 7f584fe00640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-09-10T09:41:50.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:50.028+0000 7f584fe00640 1 -- 172.21.15.5:0/3554923153 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58640ba060 con 0x559e62eefca0 2024-09-10T09:41:50.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:50.029+0000 7f5873a9e740 1 -- 172.21.15.5:0/3554923153 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5854031670 con 0x559e62eefca0 2024-09-10T09:41:50.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:50.029+0000 7f584fe00640 1 -- 172.21.15.5:0/3554923153 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f586409c0c0 con 0x559e62eefca0 2024-09-10T09:41:50.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:50.029+0000 7f584fe00640 1 -- 172.21.15.5:0/3554923153 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f58640a4ad0 con 0x559e62eefca0 2024-09-10T09:41:50.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:50.029+0000 7f5873a9e740 10 client.26107 fetch_fsmap learned FSMap version 32 2024-09-10T09:41:50.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:50.029+0000 7f5873a9e740 1 -- 172.21.15.5:0/3554923153 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x559e62f6b1e0 con 0x559e62eefca0 2024-09-10T09:41:50.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:50.029+0000 7f584fe00640 1 -- 172.21.15.5:0/3554923153 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5854031670 con 0x559e62eefca0 2024-09-10T09:41:50.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:50.029+0000 7f584fe00640 1 -- 172.21.15.5:0/3554923153 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7f586409b420 con 0x559e62eefca0 2024-09-10T09:41:50.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:50.029+0000 7f5873a9e740 10 client.26107 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T09:41:50.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:50.029+0000 7f5873a9e740 10 client.26107 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T09:41:50.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:50.029+0000 7f5873a9e740 20 client.26107 populate_metadata read hostname 'smithi005' 2024-09-10T09:41:50.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:50.029+0000 7f5873a9e740 1 --2- 172.21.15.5:0/3554923153 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x559e62effd20 0x559e62f6d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:41:50.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:50.029+0000 7f5873a9e740 4 client.26107 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-10T09:41:50.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:50.029+0000 7f5873a9e740 1 -- 172.21.15.5:0/3554923153 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x559e62e52d10 con 0x559e62effd20 2024-09-10T09:41:50.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:50.030+0000 7f5869e00640 1 --2- 172.21.15.5:0/3554923153 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x559e62effd20 0x559e62f6d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:41:50.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:50.030+0000 7f5869e00640 1 --2- 172.21.15.5:0/3554923153 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x559e62effd20 0x559e62f6d8f0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8580283daba56ec2 server_cookie=f594295e4e031edb in_seq=0 out_seq=0 2024-09-10T09:41:50.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:50.030+0000 7f584fe00640 10 client.26107 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T09:41:50.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:50.031+0000 7f584fe00640 1 -- 172.21.15.5:0/3554923153 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x559e62e52d10 con 0x559e62effd20 2024-09-10T09:41:50.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:50.031+0000 7f584fe00640 10 client.26107 handle_command_reply: tid=0 2024-09-10T09:41:50.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:50.031+0000 7f5873a9e740 1 client.26107 shutdown 2024-09-10T09:41:50.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:50.031+0000 7f5873a9e740 1 -- 172.21.15.5:0/3554923153 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559e62eefca0 msgr2=0x559e62f69aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:41:50.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:50.031+0000 7f5873a9e740 1 --2- 172.21.15.5:0/3554923153 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559e62eefca0 0x559e62f69aa0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f5864098830 tx=0x7f586409c620 comp rx=0 tx=0).stop 2024-09-10T09:41:50.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:50.031+0000 7f5873a9e740 1 -- 172.21.15.5:0/3554923153 shutdown_connections 2024-09-10T09:41:50.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:50.031+0000 7f5873a9e740 1 --2- 172.21.15.5:0/3554923153 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x559e62effd20 0x559e62f6d8f0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:41:50.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:50.031+0000 7f5873a9e740 1 --2- 172.21.15.5:0/3554923153 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559e62eefca0 0x559e62f69aa0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:41:50.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:50.031+0000 7f5873a9e740 1 --2- 172.21.15.5:0/3554923153 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559e62ee53e0 0x559e62ef4320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:41:50.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:50.031+0000 7f584fe00640 0 client.26107 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T09:41:50.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:50.031+0000 7f5873a9e740 1 --2- 172.21.15.5:0/3554923153 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559e62ee4ac0 0x559e62ef3de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:41:50.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:50.031+0000 7f5873a9e740 1 -- 172.21.15.5:0/3554923153 >> 172.21.15.5:0/3554923153 conn(0x559e62ee1050 msgr2=0x559e62ee7e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:41:50.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:50.032+0000 7f5873a9e740 1 -- 172.21.15.5:0/3554923153 shutdown_connections 2024-09-10T09:41:50.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:50.032+0000 7f5873a9e740 1 -- 172.21.15.5:0/3554923153 wait complete. 2024-09-10T09:41:50.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:50.032+0000 7f5873a9e740 20 client.26107 trim_cache size 0 max 16384 2024-09-10T09:41:50.036 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-10T09:41:50.036 INFO:teuthology.orchestra.run.smithi005.stdout: "return_code": 0, 2024-09-10T09:41:50.036 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_tag": "392d4a34-883b-4ee8-8c88-d99e7a494fb9", 2024-09-10T09:41:50.036 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": "asynchronous" 2024-09-10T09:41:50.036 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-10T09:41:50.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:50.032+0000 7f5871000640 1 -- 172.21.15.5:0/1583781864 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f58500a2c00 msgr2=0x7f58500a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:41:50.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:50.032+0000 7f5871000640 1 --2- 172.21.15.5:0/1583781864 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f58500a2c00 0x7f58500a50c0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x559e62ee23d0 tx=0x7f58540046c0 comp rx=0 tx=0).stop 2024-09-10T09:41:50.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:50.032+0000 7f5871000640 1 -- 172.21.15.5:0/1583781864 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f586c109120 msgr2=0x7f586c19ef20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:41:50.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:50.032+0000 7f5871000640 1 --2- 172.21.15.5:0/1583781864 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f586c109120 0x7f586c19ef20 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f5864009fd0 tx=0x7f586400ef10 comp rx=0 tx=0).stop 2024-09-10T09:41:50.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:50.032+0000 7f5871000640 1 -- 172.21.15.5:0/1583781864 shutdown_connections 2024-09-10T09:41:50.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:50.032+0000 7f5871000640 1 --2- 172.21.15.5:0/1583781864 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f58500a2c00 0x7f58500a50c0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:41:50.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:50.032+0000 7f5871000640 1 --2- 172.21.15.5:0/1583781864 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f586c109120 0x7f586c19ef20 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:41:50.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:50.032+0000 7f5871000640 1 --2- 172.21.15.5:0/1583781864 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f586c1049a0 0x7f586c19ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:41:50.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:50.032+0000 7f5871000640 1 --2- 172.21.15.5:0/1583781864 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f586c103ff0 0x7f586c19a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:41:50.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:50.032+0000 7f5871000640 1 -- 172.21.15.5:0/1583781864 >> 172.21.15.5:0/1583781864 conn(0x7f586c0ffa90 msgr2=0x7f586c100da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:41:50.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:50.033+0000 7f5871000640 1 -- 172.21.15.5:0/1583781864 shutdown_connections 2024-09-10T09:41:50.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:50.033+0000 7f5871000640 1 -- 172.21.15.5:0/1583781864 wait complete. 2024-09-10T09:41:50.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:41:50 smithi005 ceph-mon[30350]: scrub summary: active paths [/] 2024-09-10T09:41:50.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:41:50 smithi005 ceph-mon[30350]: scrub summary: idle+waiting paths [/] 2024-09-10T09:41:50.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:41:50 smithi159 ceph-mon[35513]: scrub summary: active paths [/] 2024-09-10T09:41:50.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:41:50 smithi159 ceph-mon[35513]: scrub summary: idle+waiting paths [/] 2024-09-10T09:41:50.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:41:50 smithi102 ceph-mon[35442]: scrub summary: active paths [/] 2024-09-10T09:41:50.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:41:50 smithi102 ceph-mon[35442]: scrub summary: idle+waiting paths [/] 2024-09-10T09:41:51.013 INFO:tasks.workunit.client.0.smithi005.stdout: 1 150425 9.01 MB/sec warmup 92 sec latency 16.703 ms 2024-09-10T09:41:51.022 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T09:41:51.460 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:41:51.539 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:41:51 smithi005 ceph-mon[30350]: pgmap v498: 129 pgs: 129 active+clean; 80 MiB data, 695 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.3 MiB/s wr, 219 op/s 2024-09-10T09:41:51.539 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:41:51 smithi005 ceph-mon[30350]: scrub queued for path: / 2024-09-10T09:41:51.539 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:41:51 smithi005 ceph-mon[30350]: scrub summary: idle+waiting paths [/] 2024-09-10T09:41:51.539 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:41:51 smithi005 ceph-mon[30350]: scrub summary: active paths [/] 2024-09-10T09:41:51.540 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:41:51 smithi005 ceph-mon[30350]: scrub summary: idle+waiting paths [/] 2024-09-10T09:41:51.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:41:51 smithi159 ceph-mon[35513]: pgmap v498: 129 pgs: 129 active+clean; 80 MiB data, 695 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.3 MiB/s wr, 219 op/s 2024-09-10T09:41:51.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:41:51 smithi159 ceph-mon[35513]: scrub queued for path: / 2024-09-10T09:41:51.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:41:51 smithi159 ceph-mon[35513]: scrub summary: idle+waiting paths [/] 2024-09-10T09:41:51.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:41:51 smithi159 ceph-mon[35513]: scrub summary: active paths [/] 2024-09-10T09:41:51.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:41:51 smithi159 ceph-mon[35513]: scrub summary: idle+waiting paths [/] 2024-09-10T09:41:51.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:41:51 smithi102 ceph-mon[35442]: pgmap v498: 129 pgs: 129 active+clean; 80 MiB data, 695 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.3 MiB/s wr, 219 op/s 2024-09-10T09:41:51.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:41:51 smithi102 ceph-mon[35442]: scrub queued for path: / 2024-09-10T09:41:51.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:41:51 smithi102 ceph-mon[35442]: scrub summary: idle+waiting paths [/] 2024-09-10T09:41:51.885 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:41:51 smithi102 ceph-mon[35442]: scrub summary: active paths [/] 2024-09-10T09:41:51.885 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:41:51 smithi102 ceph-mon[35442]: scrub summary: idle+waiting paths [/] 2024-09-10T09:41:52.013 INFO:tasks.workunit.client.0.smithi005.stdout: 1 150915 8.92 MB/sec warmup 93 sec latency 720.790 ms 2024-09-10T09:41:52.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:52.774+0000 7fafdb200640 1 -- 172.21.15.5:0/2074362680 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fafdc10e260 msgr2=0x7fafdc110650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:41:52.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:52.774+0000 7fafdb200640 1 --2- 172.21.15.5:0/2074362680 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fafdc10e260 0x7fafdc110650 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fafd4009be0 tx=0x7fafd402a750 comp rx=0 tx=0).stop 2024-09-10T09:41:52.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:52.774+0000 7fafdb200640 1 -- 172.21.15.5:0/2074362680 shutdown_connections 2024-09-10T09:41:52.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:52.774+0000 7fafdb200640 1 --2- 172.21.15.5:0/2074362680 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fafdc10e260 0x7fafdc110650 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:41:52.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:52.774+0000 7fafdb200640 1 --2- 172.21.15.5:0/2074362680 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fafdc074ee0 0x7fafdc10dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:41:52.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:52.774+0000 7fafdb200640 1 --2- 172.21.15.5:0/2074362680 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fafdc0745c0 0x7fafdc0749a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:41:52.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:52.774+0000 7fafdb200640 1 -- 172.21.15.5:0/2074362680 >> 172.21.15.5:0/2074362680 conn(0x7fafdc0fd740 msgr2=0x7fafdc0ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:41:52.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:52.775+0000 7fafdb200640 1 -- 172.21.15.5:0/2074362680 shutdown_connections 2024-09-10T09:41:52.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:52.775+0000 7fafdb200640 1 -- 172.21.15.5:0/2074362680 wait complete. 2024-09-10T09:41:52.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:52.776+0000 7fafdb200640 1 Processor -- start 2024-09-10T09:41:52.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:52.776+0000 7fafdb200640 1 -- start start 2024-09-10T09:41:52.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:52.776+0000 7fafdb200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fafdc0745c0 0x7fafdc19a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:41:52.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:52.776+0000 7fafdb200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fafdc074ee0 0x7fafdc19a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:41:52.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:52.776+0000 7fafdb200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fafdc10e260 0x7fafdc19ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:41:52.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:52.776+0000 7fafdb200640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fafdc113fb0 con 0x7fafdc0745c0 2024-09-10T09:41:52.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:52.776+0000 7fafdb200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fafdc113e30 con 0x7fafdc10e260 2024-09-10T09:41:52.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:52.776+0000 7fafdb200640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fafdc114130 con 0x7fafdc074ee0 2024-09-10T09:41:52.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:52.777+0000 7fafd9400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fafdc074ee0 0x7fafdc19a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:41:52.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:52.777+0000 7fafd9400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fafdc074ee0 0x7fafdc19a9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:47582/0 (socket says 172.21.15.5:47582) 2024-09-10T09:41:52.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:52.777+0000 7fafd9e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fafdc0745c0 0x7fafdc19a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:41:52.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:52.777+0000 7fafd9400640 1 -- 172.21.15.5:0/908374961 learned_addr learned my addr 172.21.15.5:0/908374961 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:41:52.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:52.777+0000 7fafda800640 1 --2- 172.21.15.5:0/908374961 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fafdc10e260 0x7fafdc19ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:41:52.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:52.777+0000 7fafd9400640 1 -- 172.21.15.5:0/908374961 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fafdc10e260 msgr2=0x7fafdc19ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:41:52.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:52.777+0000 7fafd9400640 1 --2- 172.21.15.5:0/908374961 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fafdc10e260 0x7fafdc19ed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:41:52.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:52.777+0000 7fafd9400640 1 -- 172.21.15.5:0/908374961 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fafdc0745c0 msgr2=0x7fafdc19a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:41:52.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:52.777+0000 7fafd9400640 1 --2- 172.21.15.5:0/908374961 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fafdc0745c0 0x7fafdc19a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:41:52.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:52.777+0000 7fafd9400640 1 -- 172.21.15.5:0/908374961 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafdc19f440 con 0x7fafdc074ee0 2024-09-10T09:41:52.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:52.778+0000 7fafd9400640 1 --2- 172.21.15.5:0/908374961 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fafdc074ee0 0x7fafdc19a9d0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fafc800e9f0 tx=0x7fafc800edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:41:52.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:52.778+0000 7fafd2a00640 1 -- 172.21.15.5:0/908374961 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafc800ccc0 con 0x7fafdc074ee0 2024-09-10T09:41:52.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:52.778+0000 7fafd2a00640 1 -- 172.21.15.5:0/908374961 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fafc8010910 con 0x7fafdc074ee0 2024-09-10T09:41:52.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:52.778+0000 7fafdb200640 1 -- 172.21.15.5:0/908374961 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fafdc19f730 con 0x7fafdc074ee0 2024-09-10T09:41:52.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:52.778+0000 7fafd2a00640 1 -- 172.21.15.5:0/908374961 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafc80188a0 con 0x7fafdc074ee0 2024-09-10T09:41:52.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:52.778+0000 7fafdb200640 1 -- 172.21.15.5:0/908374961 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fafdc1a7010 con 0x7fafdc074ee0 2024-09-10T09:41:52.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:52.782+0000 7fafe362c740 1 Processor -- start 2024-09-10T09:41:52.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:52.782+0000 7fafe362c740 1 -- start start 2024-09-10T09:41:52.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:52.783+0000 7fafe362c740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c4a5200d50 0x55c4a5201130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:41:52.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:52.783+0000 7fafe362c740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c4a5201670 0x55c4a520b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:41:52.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:52.783+0000 7fafe362c740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c4a520bf30 0x55c4a520e320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:41:52.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:52.783+0000 7fafe362c740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55c4a51673e0 con 0x55c4a5201670 2024-09-10T09:41:52.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:52.783+0000 7fafe362c740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55c4a510ed80 con 0x55c4a5200d50 2024-09-10T09:41:52.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:52.783+0000 7fafe362c740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55c4a51633a0 con 0x55c4a520bf30 2024-09-10T09:41:52.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:52.783+0000 7fafda800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c4a5200d50 0x55c4a5201130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:41:52.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:52.783+0000 7fafda800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c4a5200d50 0x55c4a5201130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:47368/0 (socket says 172.21.15.5:47368) 2024-09-10T09:41:52.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:52.783+0000 7fafda800640 1 -- 172.21.15.5:0/2594915537 learned_addr learned my addr 172.21.15.5:0/2594915537 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:41:52.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:52.783+0000 7fafd9e00640 1 --2- 172.21.15.5:0/2594915537 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c4a5201670 0x55c4a520b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:41:52.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:52.783+0000 7fafda800640 1 -- 172.21.15.5:0/2594915537 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c4a520bf30 msgr2=0x55c4a520e320 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T09:41:52.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:52.784+0000 7fafda800640 1 --2- 172.21.15.5:0/2594915537 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c4a520bf30 0x55c4a520e320 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:41:52.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:52.784+0000 7fafda800640 1 -- 172.21.15.5:0/2594915537 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c4a5201670 msgr2=0x55c4a520b9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:41:52.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:52.784+0000 7fafda800640 1 --2- 172.21.15.5:0/2594915537 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c4a5201670 0x55c4a520b9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:41:52.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:52.784+0000 7fafda800640 1 -- 172.21.15.5:0/2594915537 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c4a512dae0 con 0x55c4a5200d50 2024-09-10T09:41:52.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:52.784+0000 7fafd2a00640 1 -- 172.21.15.5:0/908374961 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7fafc8010ac0 con 0x7fafdc074ee0 2024-09-10T09:41:52.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:52.784+0000 7fafda800640 1 --2- 172.21.15.5:0/2594915537 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c4a5200d50 0x55c4a5201130 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fafd4009be0 tx=0x7fafd4005ec0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b03dcd2591da3f4f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:41:52.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:52.784+0000 7fafc7000640 1 -- 172.21.15.5:0/2594915537 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafd4041030 con 0x55c4a5200d50 2024-09-10T09:41:52.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:52.784+0000 7fafc7000640 1 -- 172.21.15.5:0/2594915537 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fafd4030bf0 con 0x55c4a5200d50 2024-09-10T09:41:52.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:52.784+0000 7fafc7000640 1 -- 172.21.15.5:0/2594915537 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafd4036040 con 0x55c4a5200d50 2024-09-10T09:41:52.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:52.785+0000 7fafe362c740 1 -- 172.21.15.5:0/2594915537 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c4a5200d50 msgr2=0x55c4a5201130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:41:52.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:52.785+0000 7fafe362c740 1 --2- 172.21.15.5:0/2594915537 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c4a5200d50 0x55c4a5201130 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fafd4009be0 tx=0x7fafd4005ec0 comp rx=0 tx=0).stop 2024-09-10T09:41:52.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:52.786+0000 7fafe362c740 1 -- 172.21.15.5:0/2594915537 shutdown_connections 2024-09-10T09:41:52.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:52.786+0000 7fafe362c740 1 --2- 172.21.15.5:0/2594915537 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c4a520bf30 0x55c4a520e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:41:52.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:52.786+0000 7fafe362c740 1 --2- 172.21.15.5:0/2594915537 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c4a5201670 0x55c4a520b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:41:52.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:52.786+0000 7fafe362c740 1 --2- 172.21.15.5:0/2594915537 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c4a5200d50 0x55c4a5201130 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:41:52.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:52.786+0000 7fafe362c740 1 -- 172.21.15.5:0/2594915537 >> 172.21.15.5:0/2594915537 conn(0x55c4a51fd2e0 msgr2=0x55c4a51fd6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:41:52.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:52.786+0000 7fafe362c740 1 -- 172.21.15.5:0/2594915537 shutdown_connections 2024-09-10T09:41:52.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:52.786+0000 7fafe362c740 1 -- 172.21.15.5:0/2594915537 wait complete. 2024-09-10T09:41:52.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:52.786+0000 7fafd2a00640 1 --2- 172.21.15.5:0/908374961 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fafc00a2b10 0x7fafc00a4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:41:52.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:52.786+0000 7fafd2a00640 1 -- 172.21.15.5:0/908374961 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fafc80cf7f0 con 0x7fafdc074ee0 2024-09-10T09:41:52.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:52.786+0000 7fafd9e00640 1 --2- 172.21.15.5:0/908374961 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fafc00a2b10 0x7fafc00a4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:41:52.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:52.786+0000 7fafe362c740 1 -- start start 2024-09-10T09:41:52.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:52.787+0000 7fafe362c740 1 Processor -- start 2024-09-10T09:41:52.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:52.787+0000 7fafd9e00640 1 --2- 172.21.15.5:0/908374961 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fafc00a2b10 0x7fafc00a4fd0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x55c4a51fe140 tx=0x7fafcc009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:41:52.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:52.787+0000 7fafe362c740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c4a5200d50 0x55c4a5208e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:41:52.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:52.787+0000 7fafe362c740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c4a5201670 0x55c4a52093d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:41:52.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:52.787+0000 7fafe362c740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c4a520bf30 0x55c4a5209910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:41:52.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:52.787+0000 7fafe362c740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55c4a520eba0 con 0x55c4a5200d50 2024-09-10T09:41:52.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:52.787+0000 7fafe362c740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55c4a51220f0 con 0x55c4a520bf30 2024-09-10T09:41:52.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:52.787+0000 7fafe362c740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55c4a520ed20 con 0x55c4a5201670 2024-09-10T09:41:52.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:52.787+0000 7fafe362c740 10 client.0 _ll_register_callbacks cb 0x55c4a51139e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:41:52.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:52.787+0000 7fafd9e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c4a5200d50 0x55c4a5208e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:41:52.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:52.788+0000 7fafda800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c4a520bf30 0x55c4a5209910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:41:52.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:52.788+0000 7fafda800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c4a520bf30 0x55c4a5209910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:47382/0 (socket says 172.21.15.5:47382) 2024-09-10T09:41:52.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:52.788+0000 7fafda800640 1 -- 172.21.15.5:0/1772335780 learned_addr learned my addr 172.21.15.5:0/1772335780 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:41:52.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:52.788+0000 7fafd9400640 1 --2- 172.21.15.5:0/1772335780 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c4a5201670 0x55c4a52093d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:41:52.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:52.788+0000 7fafda800640 1 -- 172.21.15.5:0/1772335780 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c4a5201670 msgr2=0x55c4a52093d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:41:52.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:52.788+0000 7fafda800640 1 --2- 172.21.15.5:0/1772335780 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c4a5201670 0x55c4a52093d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:41:52.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:52.788+0000 7fafda800640 1 -- 172.21.15.5:0/1772335780 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c4a5200d50 msgr2=0x55c4a5208e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:41:52.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:52.788+0000 7fafda800640 1 --2- 172.21.15.5:0/1772335780 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c4a5200d50 0x55c4a5208e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:41:52.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:52.788+0000 7fafda800640 1 -- 172.21.15.5:0/1772335780 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c4a4ed83c0 con 0x55c4a520bf30 2024-09-10T09:41:52.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:52.788+0000 7fafda800640 1 --2- 172.21.15.5:0/1772335780 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c4a520bf30 0x55c4a5209910 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fafd4030e80 tx=0x7fafd4030eb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=508d946d3be207bb server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:41:52.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:52.788+0000 7fafbb400640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-09-10T09:41:52.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:52.789+0000 7fafbb400640 1 -- 172.21.15.5:0/1772335780 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafd4041040 con 0x55c4a520bf30 2024-09-10T09:41:52.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:52.789+0000 7fafe362c740 1 -- 172.21.15.5:0/1772335780 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fafd4030bf0 con 0x55c4a520bf30 2024-09-10T09:41:52.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:52.789+0000 7fafbb400640 1 -- 172.21.15.5:0/1772335780 <== mon.2 v2:172.21.15.102:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fafd40305a0 con 0x55c4a520bf30 2024-09-10T09:41:52.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:52.789+0000 7fafbb400640 1 -- 172.21.15.5:0/1772335780 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fafd4057050 con 0x55c4a520bf30 2024-09-10T09:41:52.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:52.789+0000 7fafe362c740 10 client.26093 fetch_fsmap learned FSMap version 32 2024-09-10T09:41:52.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:52.789+0000 7fafe362c740 1 -- 172.21.15.5:0/1772335780 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55c4a521a8d0 con 0x55c4a520bf30 2024-09-10T09:41:52.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:52.789+0000 7fafbb400640 1 -- 172.21.15.5:0/1772335780 <== mon.2 v2:172.21.15.102:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fafd4030bf0 con 0x55c4a520bf30 2024-09-10T09:41:52.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:52.789+0000 7fafbb400640 1 -- 172.21.15.5:0/1772335780 <== mon.2 v2:172.21.15.102:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7fafd4031440 con 0x55c4a520bf30 2024-09-10T09:41:52.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:52.790+0000 7fafe362c740 10 client.26093 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T09:41:52.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:52.790+0000 7fafe362c740 10 client.26093 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T09:41:52.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:52.790+0000 7fafe362c740 20 client.26093 populate_metadata read hostname 'smithi005' 2024-09-10T09:41:52.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:52.790+0000 7fafe362c740 1 --2- 172.21.15.5:0/1772335780 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55c4a521bc20 0x55c4a521e010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:41:52.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:52.790+0000 7fafe362c740 4 client.26093 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:41:52.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:52.790+0000 7fafe362c740 1 -- 172.21.15.5:0/1772335780 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c4a5174950 con 0x55c4a521bc20 2024-09-10T09:41:52.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:52.790+0000 7fafd9e00640 1 --2- 172.21.15.5:0/1772335780 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55c4a521bc20 0x55c4a521e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:41:52.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:52.791+0000 7fafd9e00640 1 --2- 172.21.15.5:0/1772335780 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55c4a521bc20 0x55c4a521e010 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4a8486a75b3b4b7 server_cookie=5dcd998e5303b582 in_seq=0 out_seq=0 2024-09-10T09:41:52.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:52.791+0000 7fafbb400640 10 client.26093 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T09:41:52.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:52.791+0000 7fafbb400640 1 -- 172.21.15.5:0/1772335780 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c4a5174950 con 0x55c4a521bc20 2024-09-10T09:41:52.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:52.791+0000 7fafbb400640 10 client.26093 handle_command_reply: tid=0 2024-09-10T09:41:52.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:52.791+0000 7fafe362c740 1 client.26093 shutdown 2024-09-10T09:41:52.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:52.792+0000 7fafe362c740 1 -- 172.21.15.5:0/1772335780 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c4a520bf30 msgr2=0x55c4a5209910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:41:52.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:52.792+0000 7fafe362c740 1 --2- 172.21.15.5:0/1772335780 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c4a520bf30 0x55c4a5209910 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fafd4030e80 tx=0x7fafd4030eb0 comp rx=0 tx=0).stop 2024-09-10T09:41:52.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:52.792+0000 7fafe362c740 1 -- 172.21.15.5:0/1772335780 shutdown_connections 2024-09-10T09:41:52.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:52.792+0000 7fafe362c740 1 --2- 172.21.15.5:0/1772335780 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55c4a521bc20 0x55c4a521e010 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:41:52.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:52.792+0000 7fafe362c740 1 --2- 172.21.15.5:0/1772335780 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c4a520bf30 0x55c4a5209910 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:41:52.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:52.792+0000 7fafe362c740 1 --2- 172.21.15.5:0/1772335780 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c4a5201670 0x55c4a52093d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:41:52.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:52.792+0000 7fafe362c740 1 --2- 172.21.15.5:0/1772335780 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c4a5200d50 0x55c4a5208e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:41:52.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:52.792+0000 7fafe362c740 1 -- 172.21.15.5:0/1772335780 >> 172.21.15.5:0/1772335780 conn(0x55c4a51fd2e0 msgr2=0x55c4a5203010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:41:52.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:52.792+0000 7fafbb400640 0 client.26093 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T09:41:52.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:52.792+0000 7fafe362c740 1 -- 172.21.15.5:0/1772335780 shutdown_connections 2024-09-10T09:41:52.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:52.792+0000 7fafe362c740 1 -- 172.21.15.5:0/1772335780 wait complete. 2024-09-10T09:41:52.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:52.792+0000 7fafe362c740 20 client.26093 trim_cache size 0 max 16384 2024-09-10T09:41:52.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:52.810+0000 7fafe362c740 1 Processor -- start 2024-09-10T09:41:52.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:52.810+0000 7fafe362c740 1 -- start start 2024-09-10T09:41:52.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:52.810+0000 7fafe362c740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c4a5200d50 0x55c4a5209760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:41:52.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:52.810+0000 7fafe362c740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c4a5201670 0x55c4a5223610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:41:52.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:52.810+0000 7fafe362c740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c4a520bf30 0x55c4a5223b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:41:52.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:52.810+0000 7fafe362c740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fafdc1b0eb0 con 0x55c4a520bf30 2024-09-10T09:41:52.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:52.810+0000 7fafe362c740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fafdc1b2ab0 con 0x55c4a5201670 2024-09-10T09:41:52.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:52.810+0000 7fafe362c740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fafdc1b1580 con 0x55c4a5200d50 2024-09-10T09:41:52.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:52.811+0000 7fafd9400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c4a5201670 0x55c4a5223610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:41:52.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:52.811+0000 7fafd9400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c4a5201670 0x55c4a5223610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:47392/0 (socket says 172.21.15.5:47392) 2024-09-10T09:41:52.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:52.811+0000 7fafd9400640 1 -- 172.21.15.5:0/1087079894 learned_addr learned my addr 172.21.15.5:0/1087079894 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:41:52.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:52.811+0000 7fafda800640 1 --2- 172.21.15.5:0/1087079894 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c4a520bf30 0x55c4a5223b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:41:52.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:52.811+0000 7fafd9e00640 1 --2- 172.21.15.5:0/1087079894 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c4a5200d50 0x55c4a5209760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:41:52.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:52.812+0000 7fafda800640 1 -- 172.21.15.5:0/1087079894 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c4a5200d50 msgr2=0x55c4a5209760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:41:52.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:52.812+0000 7fafda800640 1 --2- 172.21.15.5:0/1087079894 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c4a5200d50 0x55c4a5209760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:41:52.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:52.812+0000 7fafda800640 1 -- 172.21.15.5:0/1087079894 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c4a5201670 msgr2=0x55c4a5223610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:41:52.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:52.812+0000 7fafda800640 1 --2- 172.21.15.5:0/1087079894 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c4a5201670 0x55c4a5223610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:41:52.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:52.812+0000 7fafda800640 1 -- 172.21.15.5:0/1087079894 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c4a51fde90 con 0x55c4a520bf30 2024-09-10T09:41:52.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:52.812+0000 7fafd9e00640 1 --2- 172.21.15.5:0/1087079894 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c4a5200d50 0x55c4a5209760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:41:52.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:52.812+0000 7fafda800640 1 --2- 172.21.15.5:0/1087079894 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c4a520bf30 0x55c4a5223b50 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fafd402ac70 tx=0x7fafd4005ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2eab7071381906d5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:41:52.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:52.812+0000 7fafc6600640 1 -- 172.21.15.5:0/1087079894 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafd4041070 con 0x55c4a520bf30 2024-09-10T09:41:52.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:52.812+0000 7fafc6600640 1 -- 172.21.15.5:0/1087079894 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fafd4031440 con 0x55c4a520bf30 2024-09-10T09:41:52.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:52.812+0000 7fafc6600640 1 -- 172.21.15.5:0/1087079894 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafd4036040 con 0x55c4a520bf30 2024-09-10T09:41:52.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:52.813+0000 7fafe362c740 1 -- 172.21.15.5:0/1087079894 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c4a520bf30 msgr2=0x55c4a5223b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:41:52.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:52.813+0000 7fafe362c740 1 --2- 172.21.15.5:0/1087079894 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c4a520bf30 0x55c4a5223b50 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fafd402ac70 tx=0x7fafd4005ec0 comp rx=0 tx=0).stop 2024-09-10T09:41:52.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:52.813+0000 7fafe362c740 1 -- 172.21.15.5:0/1087079894 shutdown_connections 2024-09-10T09:41:52.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:52.813+0000 7fafe362c740 1 --2- 172.21.15.5:0/1087079894 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c4a520bf30 0x55c4a5223b50 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:41:52.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:52.813+0000 7fafe362c740 1 --2- 172.21.15.5:0/1087079894 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c4a5201670 0x55c4a5223610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:41:52.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:52.813+0000 7fafe362c740 1 --2- 172.21.15.5:0/1087079894 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c4a5200d50 0x55c4a5209760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:41:52.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:52.813+0000 7fafe362c740 1 -- 172.21.15.5:0/1087079894 >> 172.21.15.5:0/1087079894 conn(0x55c4a51fd2e0 msgr2=0x55c4a520af10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:41:52.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:52.813+0000 7fafe362c740 1 -- 172.21.15.5:0/1087079894 shutdown_connections 2024-09-10T09:41:52.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:52.813+0000 7fafe362c740 1 -- 172.21.15.5:0/1087079894 wait complete. 2024-09-10T09:41:52.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:52.814+0000 7fafe362c740 1 -- start start 2024-09-10T09:41:52.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:52.814+0000 7fafe362c740 1 Processor -- start 2024-09-10T09:41:52.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:52.815+0000 7fafe362c740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c4a5200d50 0x55c4a5214460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:41:52.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:52.815+0000 7fafe362c740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c4a5201670 0x55c4a5205a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:41:52.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:52.815+0000 7fafe362c740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c4a520bf30 0x55c4a5205f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:41:52.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:52.815+0000 7fafe362c740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fafdc1b2320 con 0x55c4a520bf30 2024-09-10T09:41:52.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:52.815+0000 7fafe362c740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fafdc1b1c50 con 0x55c4a5200d50 2024-09-10T09:41:52.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:52.815+0000 7fafe362c740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fafdc1aee90 con 0x55c4a5201670 2024-09-10T09:41:52.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:52.815+0000 7fafe362c740 10 client.0 _ll_register_callbacks cb 0x7fafd4009a60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:41:52.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:52.815+0000 7fafd9400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c4a5201670 0x55c4a5205a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:41:52.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:52.815+0000 7fafd9e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c4a5200d50 0x55c4a5214460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:41:52.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:52.815+0000 7fafd9400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c4a5201670 0x55c4a5205a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:47600/0 (socket says 172.21.15.5:47600) 2024-09-10T09:41:52.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:52.815+0000 7fafd9400640 1 -- 172.21.15.5:0/480032906 learned_addr learned my addr 172.21.15.5:0/480032906 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:41:52.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:52.815+0000 7fafda800640 1 --2- 172.21.15.5:0/480032906 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c4a520bf30 0x55c4a5205f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:41:52.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:52.815+0000 7fafd9e00640 1 -- 172.21.15.5:0/480032906 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c4a5201670 msgr2=0x55c4a5205a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:41:52.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:52.815+0000 7fafd9e00640 1 --2- 172.21.15.5:0/480032906 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c4a5201670 0x55c4a5205a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:41:52.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:52.815+0000 7fafd9e00640 1 -- 172.21.15.5:0/480032906 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c4a520bf30 msgr2=0x55c4a5205f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:41:52.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:52.815+0000 7fafd9e00640 1 --2- 172.21.15.5:0/480032906 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c4a520bf30 0x55c4a5205f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:41:52.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:52.815+0000 7fafd9e00640 1 -- 172.21.15.5:0/480032906 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c4a5206520 con 0x55c4a5200d50 2024-09-10T09:41:52.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:52.816+0000 7fafd9e00640 1 --2- 172.21.15.5:0/480032906 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c4a5200d50 0x55c4a5214460 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x55c4a521e580 tx=0x7fafcc0046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c664033a0692f8c1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:41:52.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:52.816+0000 7fafbb400640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-09-10T09:41:52.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:52.816+0000 7fafbb400640 1 -- 172.21.15.5:0/480032906 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafcc0042d0 con 0x55c4a5200d50 2024-09-10T09:41:52.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:52.816+0000 7fafbb400640 1 -- 172.21.15.5:0/480032906 <== mon.2 v2:172.21.15.102:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fafcc004470 con 0x55c4a5200d50 2024-09-10T09:41:52.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:52.816+0000 7fafe362c740 1 -- 172.21.15.5:0/480032906 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fafd4031440 con 0x55c4a5200d50 2024-09-10T09:41:52.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:52.816+0000 7fafbb400640 1 -- 172.21.15.5:0/480032906 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fafcc03a050 con 0x55c4a5200d50 2024-09-10T09:41:52.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:52.816+0000 7fafe362c740 10 client.26099 fetch_fsmap learned FSMap version 32 2024-09-10T09:41:52.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:52.816+0000 7fafe362c740 1 -- 172.21.15.5:0/480032906 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55c4a5286420 con 0x55c4a5200d50 2024-09-10T09:41:52.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:52.817+0000 7fafbb400640 1 -- 172.21.15.5:0/480032906 <== mon.2 v2:172.21.15.102:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fafd4031440 con 0x55c4a5200d50 2024-09-10T09:41:52.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:52.817+0000 7fafbb400640 1 -- 172.21.15.5:0/480032906 <== mon.2 v2:172.21.15.102:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7fafcc01e670 con 0x55c4a5200d50 2024-09-10T09:41:52.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:52.817+0000 7fafe362c740 10 client.26099 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T09:41:52.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:52.817+0000 7fafe362c740 10 client.26099 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T09:41:52.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:52.817+0000 7fafe362c740 20 client.26099 populate_metadata read hostname 'smithi005' 2024-09-10T09:41:52.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:52.817+0000 7fafe362c740 1 --2- 172.21.15.5:0/480032906 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55c4a5286c90 0x55c4a52890b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:41:52.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:52.817+0000 7fafe362c740 4 client.26099 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T09:41:52.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:52.817+0000 7fafe362c740 1 -- 172.21.15.5:0/480032906 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55c4a516ee60 con 0x55c4a5286c90 2024-09-10T09:41:52.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:52.817+0000 7fafd9400640 1 --2- 172.21.15.5:0/480032906 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55c4a5286c90 0x55c4a52890b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:41:52.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:52.818+0000 7fafbb400640 10 client.26099 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T09:41:52.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:52.818+0000 7fafd9400640 1 --2- 172.21.15.5:0/480032906 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55c4a5286c90 0x55c4a52890b0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9c97b838e722d7a5 server_cookie=63a08f42f5ba60ce in_seq=0 out_seq=0 2024-09-10T09:41:52.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:52.818+0000 7fafbb400640 1 -- 172.21.15.5:0/480032906 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55c4a516ee60 con 0x55c4a5286c90 2024-09-10T09:41:52.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:52.818+0000 7fafbb400640 10 client.26099 handle_command_reply: tid=0 2024-09-10T09:41:52.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:52.818+0000 7fafe362c740 1 client.26099 shutdown 2024-09-10T09:41:52.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:52.818+0000 7fafe362c740 1 -- 172.21.15.5:0/480032906 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c4a5200d50 msgr2=0x55c4a5214460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:41:52.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:52.818+0000 7fafe362c740 1 --2- 172.21.15.5:0/480032906 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c4a5200d50 0x55c4a5214460 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x55c4a521e580 tx=0x7fafcc0046c0 comp rx=0 tx=0).stop 2024-09-10T09:41:52.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:52.819+0000 7fafe362c740 1 -- 172.21.15.5:0/480032906 shutdown_connections 2024-09-10T09:41:52.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:52.819+0000 7fafe362c740 1 --2- 172.21.15.5:0/480032906 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55c4a5286c90 0x55c4a52890b0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:41:52.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:52.819+0000 7fafe362c740 1 --2- 172.21.15.5:0/480032906 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c4a520bf30 0x55c4a5205f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:41:52.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:52.819+0000 7fafe362c740 1 --2- 172.21.15.5:0/480032906 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c4a5201670 0x55c4a5205a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:41:52.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:52.819+0000 7fafe362c740 1 --2- 172.21.15.5:0/480032906 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c4a5200d50 0x55c4a5214460 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:41:52.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:52.819+0000 7fafbb400640 0 client.26099 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T09:41:52.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:52.819+0000 7fafe362c740 1 -- 172.21.15.5:0/480032906 >> 172.21.15.5:0/480032906 conn(0x55c4a51fd2e0 msgr2=0x55c4a51fbb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:41:52.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:52.819+0000 7fafe362c740 1 -- 172.21.15.5:0/480032906 shutdown_connections 2024-09-10T09:41:52.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:52.819+0000 7fafe362c740 1 -- 172.21.15.5:0/480032906 wait complete. 2024-09-10T09:41:52.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:52.819+0000 7fafe362c740 20 client.26099 trim_cache size 0 max 16384 2024-09-10T09:41:52.826 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-10T09:41:52.826 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-10T09:41:52.826 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2024-09-10T09:41:52.826 INFO:teuthology.orchestra.run.smithi005.stdout: "392d4a34-883b-4ee8-8c88-d99e7a494fb9": { 2024-09-10T09:41:52.826 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2024-09-10T09:41:52.827 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "392d4a34-883b-4ee8-8c88-d99e7a494fb9", 2024-09-10T09:41:52.827 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2024-09-10T09:41:52.827 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T09:41:52.827 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T09:41:52.827 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-10T09:41:52.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:52.820+0000 7fafdb200640 1 -- 172.21.15.5:0/908374961 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fafc00a2b10 msgr2=0x7fafc00a4fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:41:52.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:52.820+0000 7fafdb200640 1 --2- 172.21.15.5:0/908374961 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fafc00a2b10 0x7fafc00a4fd0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x55c4a51fe140 tx=0x7fafcc009260 comp rx=0 tx=0).stop 2024-09-10T09:41:52.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:52.820+0000 7fafdb200640 1 -- 172.21.15.5:0/908374961 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fafdc074ee0 msgr2=0x7fafdc19a9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:41:52.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:52.820+0000 7fafdb200640 1 --2- 172.21.15.5:0/908374961 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fafdc074ee0 0x7fafdc19a9d0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fafc800e9f0 tx=0x7fafc800edf0 comp rx=0 tx=0).stop 2024-09-10T09:41:52.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:52.820+0000 7fafdb200640 1 -- 172.21.15.5:0/908374961 shutdown_connections 2024-09-10T09:41:52.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:52.820+0000 7fafdb200640 1 --2- 172.21.15.5:0/908374961 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fafc00a2b10 0x7fafc00a4fd0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:41:52.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:52.820+0000 7fafdb200640 1 --2- 172.21.15.5:0/908374961 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fafdc10e260 0x7fafdc19ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:41:52.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:52.820+0000 7fafdb200640 1 --2- 172.21.15.5:0/908374961 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fafdc074ee0 0x7fafdc19a9d0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:41:52.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:52.820+0000 7fafdb200640 1 --2- 172.21.15.5:0/908374961 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fafdc0745c0 0x7fafdc19a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:41:52.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:52.820+0000 7fafdb200640 1 -- 172.21.15.5:0/908374961 >> 172.21.15.5:0/908374961 conn(0x7fafdc0fd740 msgr2=0x7fafdc0fe3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:41:52.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:52.820+0000 7fafdb200640 1 -- 172.21.15.5:0/908374961 shutdown_connections 2024-09-10T09:41:52.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:41:52.820+0000 7fafdb200640 1 -- 172.21.15.5:0/908374961 wait complete. 2024-09-10T09:41:53.013 INFO:tasks.workunit.client.0.smithi005.stdout: 1 151469 8.85 MB/sec warmup 94 sec latency 1549.982 ms 2024-09-10T09:41:53.697 INFO:tasks.cephfs.filesystem:scrub status for tag:392d4a34-883b-4ee8-8c88-d99e7a494fb9 - {'path': '/', 'tag': '392d4a34-883b-4ee8-8c88-d99e7a494fb9', 'options': 'recursive,force'} 2024-09-10T09:41:53.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:41:53 smithi005 ceph-mon[30350]: pgmap v499: 129 pgs: 129 active+clean; 80 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.0 MiB/s wr, 170 op/s 2024-09-10T09:41:53.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:41:53 smithi159 ceph-mon[35513]: pgmap v499: 129 pgs: 129 active+clean; 80 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.0 MiB/s wr, 170 op/s 2024-09-10T09:41:53.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:41:53 smithi102 ceph-mon[35442]: pgmap v499: 129 pgs: 129 active+clean; 80 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.0 MiB/s wr, 170 op/s 2024-09-10T09:41:54.013 INFO:tasks.workunit.client.0.smithi005.stdout: 1 153487 8.88 MB/sec warmup 95 sec latency 32.201 ms 2024-09-10T09:41:55.013 INFO:tasks.workunit.client.0.smithi005.stdout: 1 156314 8.98 MB/sec warmup 96 sec latency 12.451 ms 2024-09-10T09:41:55.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:41:55 smithi005 ceph-mon[30350]: pgmap v500: 129 pgs: 129 active+clean; 78 MiB data, 723 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 1.9 MiB/s wr, 164 op/s 2024-09-10T09:41:55.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:41:55 smithi159 ceph-mon[35513]: pgmap v500: 129 pgs: 129 active+clean; 78 MiB data, 723 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 1.9 MiB/s wr, 164 op/s 2024-09-10T09:41:55.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:41:55 smithi102 ceph-mon[35442]: pgmap v500: 129 pgs: 129 active+clean; 78 MiB data, 723 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 1.9 MiB/s wr, 164 op/s 2024-09-10T09:41:56.013 INFO:tasks.workunit.client.0.smithi005.stdout: 1 158463 8.96 MB/sec warmup 97 sec latency 25.164 ms 2024-09-10T09:41:57.013 INFO:tasks.workunit.client.0.smithi005.stdout: 1 159480 8.95 MB/sec warmup 98 sec latency 615.464 ms 2024-09-10T09:41:57.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:41:57 smithi005 ceph-mon[30350]: pgmap v501: 129 pgs: 129 active+clean; 79 MiB data, 740 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.3 MiB/s wr, 221 op/s 2024-09-10T09:41:57.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:41:57 smithi159 ceph-mon[35513]: pgmap v501: 129 pgs: 129 active+clean; 79 MiB data, 740 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.3 MiB/s wr, 221 op/s 2024-09-10T09:41:57.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:41:57 smithi102 ceph-mon[35442]: pgmap v501: 129 pgs: 129 active+clean; 79 MiB data, 740 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.3 MiB/s wr, 221 op/s 2024-09-10T09:41:58.013 INFO:tasks.workunit.client.0.smithi005.stdout: 1 160195 8.90 MB/sec warmup 99 sec latency 975.800 ms 2024-09-10T09:41:59.013 INFO:tasks.workunit.client.0.smithi005.stdout: 1 162848 8.95 MB/sec warmup 100 sec latency 34.519 ms 2024-09-10T09:41:59.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:41:59 smithi005 ceph-mon[30350]: pgmap v502: 129 pgs: 129 active+clean; 79 MiB data, 751 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.8 MiB/s wr, 163 op/s 2024-09-10T09:41:59.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:41:59 smithi159 ceph-mon[35513]: pgmap v502: 129 pgs: 129 active+clean; 79 MiB data, 751 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.8 MiB/s wr, 163 op/s 2024-09-10T09:41:59.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:41:59 smithi102 ceph-mon[35442]: pgmap v502: 129 pgs: 129 active+clean; 79 MiB data, 751 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.8 MiB/s wr, 163 op/s 2024-09-10T09:42:00.013 INFO:tasks.workunit.client.0.smithi005.stdout: 1 164760 8.97 MB/sec warmup 101 sec latency 21.611 ms 2024-09-10T09:42:01.013 INFO:tasks.workunit.client.0.smithi005.stdout: 1 167288 9.01 MB/sec warmup 102 sec latency 13.758 ms 2024-09-10T09:42:01.756 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:42:01 smithi005 ceph-mon[30350]: pgmap v503: 129 pgs: 129 active+clean; 82 MiB data, 753 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.2 MiB/s wr, 214 op/s 2024-09-10T09:42:01.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:42:01 smithi159 ceph-mon[35513]: pgmap v503: 129 pgs: 129 active+clean; 82 MiB data, 753 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.2 MiB/s wr, 214 op/s 2024-09-10T09:42:01.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:42:01 smithi102 ceph-mon[35442]: pgmap v503: 129 pgs: 129 active+clean; 82 MiB data, 753 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.2 MiB/s wr, 214 op/s 2024-09-10T09:42:02.013 INFO:tasks.workunit.client.0.smithi005.stdout: 1 168117 8.98 MB/sec warmup 103 sec latency 543.687 ms 2024-09-10T09:42:02.547 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:42:02 smithi005 ceph-mon[30350]: scrub summary: idle 2024-09-10T09:42:02.547 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:42:02 smithi005 ceph-mon[30350]: scrub summary: idle 2024-09-10T09:42:02.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:42:02 smithi159 ceph-mon[35513]: scrub summary: idle 2024-09-10T09:42:02.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:42:02 smithi159 ceph-mon[35513]: scrub summary: idle 2024-09-10T09:42:02.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:42:02 smithi102 ceph-mon[35442]: scrub summary: idle 2024-09-10T09:42:02.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:42:02 smithi102 ceph-mon[35442]: scrub summary: idle 2024-09-10T09:42:03.014 INFO:tasks.workunit.client.0.smithi005.stdout: 1 169128 8.91 MB/sec warmup 104 sec latency 592.461 ms 2024-09-10T09:42:03.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:42:03 smithi005 ceph-mon[30350]: pgmap v504: 129 pgs: 129 active+clean; 82 MiB data, 755 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 1.9 MiB/s wr, 177 op/s 2024-09-10T09:42:03.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:42:03 smithi159 ceph-mon[35513]: pgmap v504: 129 pgs: 129 active+clean; 82 MiB data, 755 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 1.9 MiB/s wr, 177 op/s 2024-09-10T09:42:03.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:42:03 smithi102 ceph-mon[35442]: pgmap v504: 129 pgs: 129 active+clean; 82 MiB data, 755 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 1.9 MiB/s wr, 177 op/s 2024-09-10T09:42:04.013 INFO:tasks.workunit.client.0.smithi005.stdout: 1 171078 8.94 MB/sec warmup 105 sec latency 31.246 ms 2024-09-10T09:42:05.014 INFO:tasks.workunit.client.0.smithi005.stdout: 1 173929 9.02 MB/sec warmup 106 sec latency 10.695 ms 2024-09-10T09:42:05.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:42:05 smithi005 ceph-mon[30350]: pgmap v505: 129 pgs: 129 active+clean; 81 MiB data, 759 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 1.9 MiB/s wr, 175 op/s 2024-09-10T09:42:05.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:42:05 smithi159 ceph-mon[35513]: pgmap v505: 129 pgs: 129 active+clean; 81 MiB data, 759 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 1.9 MiB/s wr, 175 op/s 2024-09-10T09:42:05.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:42:05 smithi102 ceph-mon[35442]: pgmap v505: 129 pgs: 129 active+clean; 81 MiB data, 759 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 1.9 MiB/s wr, 175 op/s 2024-09-10T09:42:06.014 INFO:tasks.workunit.client.0.smithi005.stdout: 1 175537 9.00 MB/sec warmup 107 sec latency 22.816 ms 2024-09-10T09:42:07.014 INFO:tasks.workunit.client.0.smithi005.stdout: 1 177453 9.03 MB/sec warmup 108 sec latency 361.247 ms 2024-09-10T09:42:07.807 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:42:07 smithi005 ceph-mon[30350]: pgmap v506: 129 pgs: 129 active+clean; 85 MiB data, 757 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.2 MiB/s wr, 235 op/s 2024-09-10T09:42:07.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:42:07 smithi159 ceph-mon[35513]: pgmap v506: 129 pgs: 129 active+clean; 85 MiB data, 757 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.2 MiB/s wr, 235 op/s 2024-09-10T09:42:07.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:42:07 smithi102 ceph-mon[35442]: pgmap v506: 129 pgs: 129 active+clean; 85 MiB data, 757 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.2 MiB/s wr, 235 op/s 2024-09-10T09:42:08.014 INFO:tasks.workunit.client.0.smithi005.stdout: 1 178098 8.97 MB/sec warmup 109 sec latency 871.101 ms 2024-09-10T09:42:09.014 INFO:tasks.workunit.client.0.smithi005.stdout: 1 180889 9.04 MB/sec warmup 110 sec latency 16.298 ms 2024-09-10T09:42:09.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:42:09 smithi005 ceph-mon[30350]: pgmap v507: 129 pgs: 129 active+clean; 85 MiB data, 770 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 1.9 MiB/s wr, 180 op/s 2024-09-10T09:42:09.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:42:09 smithi159 ceph-mon[35513]: pgmap v507: 129 pgs: 129 active+clean; 85 MiB data, 770 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 1.9 MiB/s wr, 180 op/s 2024-09-10T09:42:09.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:42:09 smithi102 ceph-mon[35442]: pgmap v507: 129 pgs: 129 active+clean; 85 MiB data, 770 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 1.9 MiB/s wr, 180 op/s 2024-09-10T09:42:10.014 INFO:tasks.workunit.client.0.smithi005.stdout: 1 182501 9.02 MB/sec warmup 111 sec latency 35.666 ms 2024-09-10T09:42:11.014 INFO:tasks.workunit.client.0.smithi005.stdout: 1 185035 9.06 MB/sec warmup 112 sec latency 9.693 ms 2024-09-10T09:42:11.429 INFO:tasks.quiescer.fs.[cephfs]:Total duration (45.5) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-09-10T09:42:11.430 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 22.75 2024-09-10T09:42:11.430 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T09:42:11.747 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:42:11 smithi005 ceph-mon[30350]: pgmap v508: 129 pgs: 129 active+clean; 89 MiB data, 782 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 2.3 MiB/s wr, 247 op/s 2024-09-10T09:42:11.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:42:11 smithi159 ceph-mon[35513]: pgmap v508: 129 pgs: 129 active+clean; 89 MiB data, 782 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 2.3 MiB/s wr, 247 op/s 2024-09-10T09:42:11.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:42:11 smithi102 ceph-mon[35442]: pgmap v508: 129 pgs: 129 active+clean; 89 MiB data, 782 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 2.3 MiB/s wr, 247 op/s 2024-09-10T09:42:11.892 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:42:12.014 INFO:tasks.workunit.client.0.smithi005.stdout: 1 186640 9.03 MB/sec warmup 113 sec latency 297.539 ms 2024-09-10T09:42:12.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:12.824+0000 7fbcdbe00640 1 -- 172.21.15.5:0/480840572 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbcd4102790 msgr2=0x7fbcd41067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:42:12.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:12.824+0000 7fbcdbe00640 1 --2- 172.21.15.5:0/480840572 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbcd4102790 0x7fbcd41067e0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fbcc0009900 tx=0x7fbcc002a3f0 comp rx=0 tx=0).stop 2024-09-10T09:42:12.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:12.824+0000 7fbcdbe00640 1 -- 172.21.15.5:0/480840572 shutdown_connections 2024-09-10T09:42:12.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:12.824+0000 7fbcdbe00640 1 --2- 172.21.15.5:0/480840572 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbcd4106f10 0x7fbcd410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:12.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:12.824+0000 7fbcdbe00640 1 --2- 172.21.15.5:0/480840572 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbcd4102790 0x7fbcd41067e0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:12.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:12.824+0000 7fbcdbe00640 1 --2- 172.21.15.5:0/480840572 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbcd4101de0 0x7fbcd41021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:12.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:12.824+0000 7fbcdbe00640 1 -- 172.21.15.5:0/480840572 >> 172.21.15.5:0/480840572 conn(0x7fbcd40fd8c0 msgr2=0x7fbcd40ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:42:12.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:12.824+0000 7fbcdbe00640 1 -- 172.21.15.5:0/480840572 shutdown_connections 2024-09-10T09:42:12.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:12.824+0000 7fbcdbe00640 1 -- 172.21.15.5:0/480840572 wait complete. 2024-09-10T09:42:12.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:12.825+0000 7fbcdbe00640 1 Processor -- start 2024-09-10T09:42:12.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:12.825+0000 7fbcdbe00640 1 -- start start 2024-09-10T09:42:12.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:12.825+0000 7fbcdbe00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbcd4101de0 0x7fbcd4108b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:42:12.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:12.825+0000 7fbcdbe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbcd4102790 0x7fbcd4109050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:42:12.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:12.825+0000 7fbcdbe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbcd4106f10 0x7fbcd4109590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:42:12.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:12.825+0000 7fbcdbe00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fbcd4111fc0 con 0x7fbcd4106f10 2024-09-10T09:42:12.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:12.825+0000 7fbcdbe00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fbcd4111e40 con 0x7fbcd4102790 2024-09-10T09:42:12.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:12.825+0000 7fbcdbe00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fbcd4112140 con 0x7fbcd4101de0 2024-09-10T09:42:12.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:12.825+0000 7fbcd9400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbcd4101de0 0x7fbcd4108b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:42:12.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:12.826+0000 7fbcd8a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbcd4102790 0x7fbcd4109050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:42:12.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:12.826+0000 7fbcd9400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbcd4101de0 0x7fbcd4108b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:48230/0 (socket says 172.21.15.5:48230) 2024-09-10T09:42:12.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:12.826+0000 7fbcd9400640 1 -- 172.21.15.5:0/40470469 learned_addr learned my addr 172.21.15.5:0/40470469 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:42:12.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:12.826+0000 7fbcd9e00640 1 --2- 172.21.15.5:0/40470469 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbcd4106f10 0x7fbcd4109590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:42:12.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:12.826+0000 7fbcd9400640 1 -- 172.21.15.5:0/40470469 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbcd4102790 msgr2=0x7fbcd4109050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:42:12.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:12.826+0000 7fbcd9400640 1 --2- 172.21.15.5:0/40470469 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbcd4102790 0x7fbcd4109050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:12.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:12.826+0000 7fbcd9400640 1 -- 172.21.15.5:0/40470469 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbcd4106f10 msgr2=0x7fbcd4109590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:42:12.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:12.826+0000 7fbcd9400640 1 --2- 172.21.15.5:0/40470469 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbcd4106f10 0x7fbcd4109590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:12.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:12.826+0000 7fbcd9400640 1 -- 172.21.15.5:0/40470469 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbcd41aad30 con 0x7fbcd4101de0 2024-09-10T09:42:12.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:12.826+0000 7fbcd8a00640 1 --2- 172.21.15.5:0/40470469 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbcd4102790 0x7fbcd4109050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:42:12.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:12.826+0000 7fbcd9400640 1 --2- 172.21.15.5:0/40470469 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbcd4101de0 0x7fbcd4108b10 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fbcc800d980 tx=0x7fbcc800dd80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:42:12.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:12.827+0000 7fbcd2000640 1 -- 172.21.15.5:0/40470469 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcc8017070 con 0x7fbcd4101de0 2024-09-10T09:42:12.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:12.827+0000 7fbcd2000640 1 -- 172.21.15.5:0/40470469 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbcc8009c30 con 0x7fbcd4101de0 2024-09-10T09:42:12.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:12.827+0000 7fbcdbe00640 1 -- 172.21.15.5:0/40470469 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbcd41aafc0 con 0x7fbcd4101de0 2024-09-10T09:42:12.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:12.827+0000 7fbcd2000640 1 -- 172.21.15.5:0/40470469 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcc800f620 con 0x7fbcd4101de0 2024-09-10T09:42:12.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:12.827+0000 7fbcdbe00640 1 -- 172.21.15.5:0/40470469 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbcd41ab480 con 0x7fbcd4101de0 2024-09-10T09:42:12.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:12.828+0000 7fbcdbe00640 1 -- 172.21.15.5:0/40470469 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbcd41032c0 con 0x7fbcd4101de0 2024-09-10T09:42:12.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:12.830+0000 7fbcd2000640 1 -- 172.21.15.5:0/40470469 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7fbcc8010460 con 0x7fbcd4101de0 2024-09-10T09:42:12.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:12.832+0000 7fbcd2000640 1 --2- 172.21.15.5:0/40470469 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fbcbc0a2c00 0x7fbcbc0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:42:12.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:12.832+0000 7fbcd2000640 1 -- 172.21.15.5:0/40470469 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fbcc80ceee0 con 0x7fbcd4101de0 2024-09-10T09:42:12.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:12.832+0000 7fbcd8a00640 1 --2- 172.21.15.5:0/40470469 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fbcbc0a2c00 0x7fbcbc0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:42:12.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:12.833+0000 7fbcd8a00640 1 --2- 172.21.15.5:0/40470469 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fbcbc0a2c00 0x7fbcbc0a50c0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fbcc0000c00 tx=0x7fbcc0031510 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:42:12.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:12.835+0000 7fbcd2000640 1 -- 172.21.15.5:0/40470469 <== mon.1 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fbcc809ba70 con 0x7fbcd4101de0 2024-09-10T09:42:13.014 INFO:tasks.workunit.client.0.smithi005.stdout: 1 187465 9.01 MB/sec warmup 114 sec latency 545.953 ms 2024-09-10T09:42:13.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:13.079+0000 7fbcdbe00640 1 -- 172.21.15.5:0/40470469 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbcd410a040 con 0x7fbcd4101de0 2024-09-10T09:42:13.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:13.079+0000 7fbcd2000640 1 -- 172.21.15.5:0/40470469 <== mon.1 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+11976 (secure 0 0 0) 0x7fbcc809b370 con 0x7fbcd4101de0 2024-09-10T09:42:13.082 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-10T09:42:13.082 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":32,"btime":"2024-09-10T09:40:27:751081+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14454,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/2970594311","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2970594311},{"type":"v1","addr":"172.21.15.5:6835","nonce":2970594311}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14514,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6839/3616502174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":3616502174},{"type":"v1","addr":"172.21.15.5:6839","nonce":3616502174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":32,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T09:35:29.626287+0000","modified":"2024-09-10T09:40:26.959994+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24542,"mds_1":24535,"mds_2":24511,"mds_3":24506,"mds_4":24443},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14520":{"gid":14520,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/2073599842","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":2073599842},{"type":"v1","addr":"172.21.15.5:6841","nonce":2073599842}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"k","rank":4,"incarnation":26,"state":"up:active","state_seq":24,"addr":"172.21.15.102:6833/2457007427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":2457007427},{"type":"v1","addr":"172.21.15.102:6833","nonce":2457007427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/356445159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":356445159},{"type":"v1","addr":"172.21.15.159:6833","nonce":356445159}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6837/2894314507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":2894314507},{"type":"v1","addr":"172.21.15.5:6837","nonce":2894314507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.102:6835/1647549111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":1647549111},{"type":"v1","addr":"172.21.15.102:6835","nonce":1647549111}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6835/952657242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":952657242},{"type":"v1","addr":"172.21.15.159:6835","nonce":952657242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":19,"addr":"172.21.15.102:6837/2689776764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":2689776764},{"type":"v1","addr":"172.21.15.102:6837","nonce":2689776764}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.159:6837/620627639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":620627639},{"type":"v1","addr":"172.21.15.159:6837","nonce":620627639}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24535":{"gid":24535,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":18,"addr":"172.21.15.159:6839/2073739599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":2073739599},{"type":"v1","addr":"172.21.15.159:6839","nonce":2073739599}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24542":{"gid":24542,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.102:6839/335409749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6838","nonce":335409749},{"type":"v1","addr":"172.21.15.102:6839","nonce":335409749}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24542,"qdb_cluster":[24542,24535,24511,24506,24443]},"id":1}]} 2024-09-10T09:42:13.082 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 32 2024-09-10T09:42:13.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:13.082+0000 7fbcdbe00640 1 -- 172.21.15.5:0/40470469 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fbcbc0a2c00 msgr2=0x7fbcbc0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:42:13.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:13.082+0000 7fbcdbe00640 1 --2- 172.21.15.5:0/40470469 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fbcbc0a2c00 0x7fbcbc0a50c0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fbcc0000c00 tx=0x7fbcc0031510 comp rx=0 tx=0).stop 2024-09-10T09:42:13.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:13.082+0000 7fbcdbe00640 1 -- 172.21.15.5:0/40470469 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbcd4101de0 msgr2=0x7fbcd4108b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:42:13.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:13.082+0000 7fbcdbe00640 1 --2- 172.21.15.5:0/40470469 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbcd4101de0 0x7fbcd4108b10 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fbcc800d980 tx=0x7fbcc800dd80 comp rx=0 tx=0).stop 2024-09-10T09:42:13.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:13.083+0000 7fbcdbe00640 1 -- 172.21.15.5:0/40470469 shutdown_connections 2024-09-10T09:42:13.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:13.083+0000 7fbcdbe00640 1 --2- 172.21.15.5:0/40470469 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fbcbc0a2c00 0x7fbcbc0a50c0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:13.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:13.083+0000 7fbcdbe00640 1 --2- 172.21.15.5:0/40470469 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbcd4106f10 0x7fbcd4109590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:13.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:13.083+0000 7fbcdbe00640 1 --2- 172.21.15.5:0/40470469 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbcd4102790 0x7fbcd4109050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:13.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:13.083+0000 7fbcdbe00640 1 --2- 172.21.15.5:0/40470469 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbcd4101de0 0x7fbcd4108b10 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:13.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:13.083+0000 7fbcdbe00640 1 -- 172.21.15.5:0/40470469 >> 172.21.15.5:0/40470469 conn(0x7fbcd40fd8c0 msgr2=0x7fbcd40feb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:42:13.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:13.083+0000 7fbcdbe00640 1 -- 172.21.15.5:0/40470469 shutdown_connections 2024-09-10T09:42:13.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:13.083+0000 7fbcdbe00640 1 -- 172.21.15.5:0/40470469 wait complete. 2024-09-10T09:42:13.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:42:13 smithi005 ceph-mon[30350]: pgmap v509: 129 pgs: 129 active+clean; 89 MiB data, 806 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 1.8 MiB/s wr, 197 op/s 2024-09-10T09:42:13.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:42:13 smithi005 ceph-mon[30350]: from='client.? 172.21.15.5:0/40470469' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:42:13.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:42:13 smithi159 ceph-mon[35513]: pgmap v509: 129 pgs: 129 active+clean; 89 MiB data, 806 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 1.8 MiB/s wr, 197 op/s 2024-09-10T09:42:13.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:42:13 smithi159 ceph-mon[35513]: from='client.? 172.21.15.5:0/40470469' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:42:13.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:42:13 smithi102 ceph-mon[35442]: pgmap v509: 129 pgs: 129 active+clean; 89 MiB data, 806 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 1.8 MiB/s wr, 197 op/s 2024-09-10T09:42:13.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:42:13 smithi102 ceph-mon[35442]: from='client.? 172.21.15.5:0/40470469' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:42:14.014 INFO:tasks.workunit.client.0.smithi005.stdout: 1 189268 9.01 MB/sec warmup 115 sec latency 24.270 ms 2024-09-10T09:42:14.033 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24542 quiesce db / --timeout 90 --expiration 142.75 --await-for 45.0' 2024-09-10T09:42:14.033 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24542 quiesce db / --timeout 90 --expiration 142.75 --await-for 45.0 2024-09-10T09:42:14.468 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:42:15.014 INFO:tasks.workunit.client.0.smithi005.stdout: 1 191842 9.08 MB/sec warmup 116 sec latency 17.013 ms 2024-09-10T09:42:15.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:15.049+0000 7f6a97a00640 1 -- 172.21.15.5:0/1930799894 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6a90112a30 msgr2=0x7f6a90114e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:42:15.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:15.049+0000 7f6a97a00640 1 --2- 172.21.15.5:0/1930799894 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6a90112a30 0x7f6a90114e20 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f6a88009ca0 tx=0x7f6a8802a8c0 comp rx=0 tx=0).stop 2024-09-10T09:42:15.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:15.049+0000 7f6a97a00640 1 -- 172.21.15.5:0/1930799894 shutdown_connections 2024-09-10T09:42:15.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:15.049+0000 7f6a97a00640 1 --2- 172.21.15.5:0/1930799894 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6a90112a30 0x7f6a90114e20 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:15.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:15.049+0000 7f6a97a00640 1 --2- 172.21.15.5:0/1930799894 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6a90077040 0x7f6a901124f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:15.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:15.049+0000 7f6a97a00640 1 --2- 172.21.15.5:0/1930799894 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6a90076720 0x7f6a90076b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:15.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:15.049+0000 7f6a97a00640 1 -- 172.21.15.5:0/1930799894 >> 172.21.15.5:0/1930799894 conn(0x7f6a900ffa90 msgr2=0x7f6a90101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:42:15.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:15.049+0000 7f6a97a00640 1 -- 172.21.15.5:0/1930799894 shutdown_connections 2024-09-10T09:42:15.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:15.049+0000 7f6a97a00640 1 -- 172.21.15.5:0/1930799894 wait complete. 2024-09-10T09:42:15.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:15.050+0000 7f6a97a00640 1 Processor -- start 2024-09-10T09:42:15.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:15.050+0000 7f6a97a00640 1 -- start start 2024-09-10T09:42:15.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:15.051+0000 7f6a97a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6a90076720 0x7f6a901a46d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:42:15.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:15.051+0000 7f6a97a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6a90077040 0x7f6a901a4c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:42:15.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:15.051+0000 7f6a97a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6a90112a30 0x7f6a9019e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:42:15.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:15.051+0000 7f6a97a00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f6a901185c0 con 0x7f6a90077040 2024-09-10T09:42:15.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:15.051+0000 7f6a97a00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f6a90118440 con 0x7f6a90076720 2024-09-10T09:42:15.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:15.051+0000 7f6a97a00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f6a90118740 con 0x7f6a90112a30 2024-09-10T09:42:15.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:15.051+0000 7f6a95a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6a90112a30 0x7f6a9019e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:42:15.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:15.051+0000 7f6a95a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6a90112a30 0x7f6a9019e890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:48240/0 (socket says 172.21.15.5:48240) 2024-09-10T09:42:15.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:15.051+0000 7f6a87e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6a90077040 0x7f6a901a4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:42:15.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:15.051+0000 7f6a95a00640 1 -- 172.21.15.5:0/1983818463 learned_addr learned my addr 172.21.15.5:0/1983818463 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:42:15.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:15.051+0000 7f6a95000640 1 --2- 172.21.15.5:0/1983818463 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6a90076720 0x7f6a901a46d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:42:15.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:15.052+0000 7f6a95a00640 1 -- 172.21.15.5:0/1983818463 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6a90076720 msgr2=0x7f6a901a46d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:42:15.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:15.052+0000 7f6a95a00640 1 --2- 172.21.15.5:0/1983818463 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6a90076720 0x7f6a901a46d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:15.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:15.052+0000 7f6a95a00640 1 -- 172.21.15.5:0/1983818463 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6a90077040 msgr2=0x7f6a901a4c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:42:15.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:15.052+0000 7f6a95a00640 1 --2- 172.21.15.5:0/1983818463 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6a90077040 0x7f6a901a4c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:15.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:15.052+0000 7f6a95a00640 1 -- 172.21.15.5:0/1983818463 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a9019efc0 con 0x7f6a90112a30 2024-09-10T09:42:15.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:15.052+0000 7f6a95a00640 1 --2- 172.21.15.5:0/1983818463 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6a90112a30 0x7f6a9019e890 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f6a88005e30 tx=0x7f6a88003f10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:42:15.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:15.052+0000 7f6a8e000640 1 -- 172.21.15.5:0/1983818463 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a88031650 con 0x7f6a90112a30 2024-09-10T09:42:15.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:15.052+0000 7f6a8e000640 1 -- 172.21.15.5:0/1983818463 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6a880317f0 con 0x7f6a90112a30 2024-09-10T09:42:15.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:15.052+0000 7f6a97a00640 1 -- 172.21.15.5:0/1983818463 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a9019f250 con 0x7f6a90112a30 2024-09-10T09:42:15.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:15.052+0000 7f6a8e000640 1 -- 172.21.15.5:0/1983818463 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a8802fa10 con 0x7f6a90112a30 2024-09-10T09:42:15.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:15.053+0000 7f6a97a00640 1 -- 172.21.15.5:0/1983818463 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a901ab5a0 con 0x7f6a90112a30 2024-09-10T09:42:15.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:15.056+0000 7f6a8e000640 1 -- 172.21.15.5:0/1983818463 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f6a8802fbd0 con 0x7f6a90112a30 2024-09-10T09:42:15.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:15.056+0000 7f6a9a5db740 1 Processor -- start 2024-09-10T09:42:15.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:15.056+0000 7f6a9a5db740 1 -- start start 2024-09-10T09:42:15.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:15.057+0000 7f6a9a5db740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f4ee4b04e0 0x55f4ee4b08c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:42:15.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:15.057+0000 7f6a9a5db740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f4ee4b0e00 0x55f4ee4bb180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:42:15.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:15.057+0000 7f6a9a5db740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f4ee4bb6c0 0x55f4ee4bdab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:42:15.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:15.057+0000 7f6a9a5db740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55f4ee415260 con 0x55f4ee4b0e00 2024-09-10T09:42:15.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:15.057+0000 7f6a9a5db740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55f4ee3bc500 con 0x55f4ee4bb6c0 2024-09-10T09:42:15.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:15.057+0000 7f6a9a5db740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55f4ee411220 con 0x55f4ee4b04e0 2024-09-10T09:42:15.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:15.057+0000 7f6a87e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f4ee4b04e0 0x55f4ee4b08c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:42:15.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:15.057+0000 7f6a87e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f4ee4b04e0 0x55f4ee4b08c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:48248/0 (socket says 172.21.15.5:48248) 2024-09-10T09:42:15.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:15.057+0000 7f6a87e00640 1 -- 172.21.15.5:0/3948725858 learned_addr learned my addr 172.21.15.5:0/3948725858 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:42:15.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:15.057+0000 7f6a95000640 1 --2- 172.21.15.5:0/3948725858 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f4ee4b0e00 0x55f4ee4bb180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:42:15.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:15.057+0000 7f6a87e00640 1 --2- 172.21.15.5:0/3948725858 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f4ee4bb6c0 0x55f4ee4bdab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:42:15.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:15.058+0000 7f6a87e00640 1 -- 172.21.15.5:0/3948725858 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f4ee4bb6c0 msgr2=0x55f4ee4bdab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:42:15.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:15.058+0000 7f6a87e00640 1 --2- 172.21.15.5:0/3948725858 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f4ee4bb6c0 0x55f4ee4bdab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:15.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:15.058+0000 7f6a87e00640 1 -- 172.21.15.5:0/3948725858 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f4ee4b0e00 msgr2=0x55f4ee4bb180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:42:15.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:15.058+0000 7f6a87e00640 1 --2- 172.21.15.5:0/3948725858 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f4ee4b0e00 0x55f4ee4bb180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:15.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:15.058+0000 7f6a87e00640 1 -- 172.21.15.5:0/3948725858 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f4ee186ba0 con 0x55f4ee4b04e0 2024-09-10T09:42:15.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:15.058+0000 7f6a87e00640 1 --2- 172.21.15.5:0/3948725858 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f4ee4b04e0 0x55f4ee4b08c0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f6a78009560 tx=0x7f6a78009960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1007346451e4e4a5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:42:15.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:15.058+0000 7f6a86000640 1 -- 172.21.15.5:0/3948725858 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a78004050 con 0x55f4ee4b04e0 2024-09-10T09:42:15.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:15.058+0000 7f6a86000640 1 -- 172.21.15.5:0/3948725858 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6a78005500 con 0x55f4ee4b04e0 2024-09-10T09:42:15.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:15.058+0000 7f6a86000640 1 -- 172.21.15.5:0/3948725858 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a78004d60 con 0x55f4ee4b04e0 2024-09-10T09:42:15.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:15.058+0000 7f6a8e000640 1 --2- 172.21.15.5:0/1983818463 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f6a740a2b80 0x7f6a740a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:42:15.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:15.058+0000 7f6a8e000640 1 -- 172.21.15.5:0/1983818463 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f6a880f0910 con 0x7f6a90112a30 2024-09-10T09:42:15.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:15.059+0000 7f6a95000640 1 --2- 172.21.15.5:0/1983818463 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f6a740a2b80 0x7f6a740a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:42:15.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:15.059+0000 7f6a9a5db740 1 -- 172.21.15.5:0/3948725858 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f4ee4b04e0 msgr2=0x55f4ee4b08c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:42:15.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:15.059+0000 7f6a9a5db740 1 --2- 172.21.15.5:0/3948725858 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f4ee4b04e0 0x55f4ee4b08c0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f6a78009560 tx=0x7f6a78009960 comp rx=0 tx=0).stop 2024-09-10T09:42:15.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:15.059+0000 7f6a9a5db740 1 -- 172.21.15.5:0/3948725858 shutdown_connections 2024-09-10T09:42:15.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:15.059+0000 7f6a9a5db740 1 --2- 172.21.15.5:0/3948725858 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f4ee4bb6c0 0x55f4ee4bdab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:15.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:15.059+0000 7f6a9a5db740 1 --2- 172.21.15.5:0/3948725858 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f4ee4b0e00 0x55f4ee4bb180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:15.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:15.059+0000 7f6a9a5db740 1 --2- 172.21.15.5:0/3948725858 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f4ee4b04e0 0x55f4ee4b08c0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:15.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:15.059+0000 7f6a9a5db740 1 -- 172.21.15.5:0/3948725858 >> 172.21.15.5:0/3948725858 conn(0x55f4ee4aca70 msgr2=0x55f4ee4ace50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:42:15.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:15.059+0000 7f6a9a5db740 1 -- 172.21.15.5:0/3948725858 shutdown_connections 2024-09-10T09:42:15.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:15.059+0000 7f6a9a5db740 1 -- 172.21.15.5:0/3948725858 wait complete. 2024-09-10T09:42:15.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:15.059+0000 7f6a95000640 1 --2- 172.21.15.5:0/1983818463 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f6a740a2b80 0x7f6a740a5040 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x55f4ee4ad8d0 tx=0x7f6a8000b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:42:15.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:15.060+0000 7f6a9a5db740 1 -- start start 2024-09-10T09:42:15.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:15.060+0000 7f6a9a5db740 1 Processor -- start 2024-09-10T09:42:15.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:15.060+0000 7f6a9a5db740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f4ee4b04e0 0x55f4ee4b8580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:42:15.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:15.060+0000 7f6a9a5db740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f4ee4b0e00 0x55f4ee4b8ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:42:15.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:15.060+0000 7f6a9a5db740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f4ee4bb6c0 0x55f4ee4b9000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:42:15.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:15.060+0000 7f6a9a5db740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55f4ee4be290 con 0x55f4ee4b0e00 2024-09-10T09:42:15.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:15.060+0000 7f6a9a5db740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55f4ee3cfbd0 con 0x55f4ee4b04e0 2024-09-10T09:42:15.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:15.060+0000 7f6a9a5db740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55f4ee4be410 con 0x55f4ee4bb6c0 2024-09-10T09:42:15.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:15.060+0000 7f6a9a5db740 10 client.0 _ll_register_callbacks cb 0x55f4ee1240a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:42:15.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:15.061+0000 7f6a95a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f4ee4bb6c0 0x55f4ee4b9000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:42:15.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:15.061+0000 7f6a87e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f4ee4b0e00 0x55f4ee4b8ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:42:15.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:15.061+0000 7f6a95a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f4ee4bb6c0 0x55f4ee4b9000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:48252/0 (socket says 172.21.15.5:48252) 2024-09-10T09:42:15.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:15.061+0000 7f6a95a00640 1 -- 172.21.15.5:0/582936069 learned_addr learned my addr 172.21.15.5:0/582936069 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:42:15.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:15.061+0000 7f6a95a00640 1 -- 172.21.15.5:0/582936069 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f4ee4b04e0 msgr2=0x55f4ee4b8580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:42:15.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:15.061+0000 7f6a95a00640 1 --2- 172.21.15.5:0/582936069 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f4ee4b04e0 0x55f4ee4b8580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:15.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:15.061+0000 7f6a95a00640 1 -- 172.21.15.5:0/582936069 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f4ee4b0e00 msgr2=0x55f4ee4b8ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:42:15.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:15.061+0000 7f6a95a00640 1 --2- 172.21.15.5:0/582936069 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f4ee4b0e00 0x55f4ee4b8ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:15.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:15.061+0000 7f6a95a00640 1 -- 172.21.15.5:0/582936069 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f4ee4c7700 con 0x55f4ee4bb6c0 2024-09-10T09:42:15.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:15.061+0000 7f6a95a00640 1 --2- 172.21.15.5:0/582936069 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f4ee4bb6c0 0x55f4ee4b9000 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f6a8802ae80 tx=0x7f6a88031ae0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bc654a6b4a5f5403 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:42:15.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:15.061+0000 7f6a7ea00640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-09-10T09:42:15.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:15.062+0000 7f6a7ea00640 1 -- 172.21.15.5:0/582936069 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a880c5990 con 0x55f4ee4bb6c0 2024-09-10T09:42:15.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:15.062+0000 7f6a7ea00640 1 -- 172.21.15.5:0/582936069 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f6a880bcf80 con 0x55f4ee4bb6c0 2024-09-10T09:42:15.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:15.062+0000 7f6a9a5db740 1 -- 172.21.15.5:0/582936069 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6a78005500 con 0x55f4ee4bb6c0 2024-09-10T09:42:15.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:15.062+0000 7f6a7ea00640 1 -- 172.21.15.5:0/582936069 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f6a880bd180 con 0x55f4ee4bb6c0 2024-09-10T09:42:15.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:15.062+0000 7f6a9a5db740 10 client.26140 fetch_fsmap learned FSMap version 32 2024-09-10T09:42:15.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:15.062+0000 7f6a9a5db740 1 -- 172.21.15.5:0/582936069 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55f4ee4be290 con 0x55f4ee4bb6c0 2024-09-10T09:42:15.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:15.063+0000 7f6a7ea00640 1 -- 172.21.15.5:0/582936069 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6a78005500 con 0x55f4ee4bb6c0 2024-09-10T09:42:15.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:15.063+0000 7f6a7ea00640 1 -- 172.21.15.5:0/582936069 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7f6a8809bc90 con 0x55f4ee4bb6c0 2024-09-10T09:42:15.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:15.063+0000 7f6a9a5db740 10 client.26140 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T09:42:15.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:15.063+0000 7f6a9a5db740 10 client.26140 resolve_mds: validated gid 24542 aka daemon mds.b 2024-09-10T09:42:15.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:15.063+0000 7f6a9a5db740 20 client.26140 populate_metadata read hostname 'smithi005' 2024-09-10T09:42:15.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:15.063+0000 7f6a9a5db740 1 --2- 172.21.15.5:0/582936069 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55f4ee4cb4b0 0x55f4ee4cd950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:42:15.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:15.063+0000 7f6a9a5db740 4 client.26140 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:42:15.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:15.063+0000 7f6a9a5db740 1 -- 172.21.15.5:0/582936069 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f4ee422990 con 0x55f4ee4cb4b0 2024-09-10T09:42:15.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:15.064+0000 7f6a95000640 1 --2- 172.21.15.5:0/582936069 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55f4ee4cb4b0 0x55f4ee4cd950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:42:15.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:15.064+0000 7f6a95000640 1 --2- 172.21.15.5:0/582936069 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55f4ee4cb4b0 0x55f4ee4cd950 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d33bd302ac48af62 server_cookie=59520570ca7fc7e2 in_seq=0 out_seq=0 2024-09-10T09:42:15.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:15.064+0000 7f6a7ea00640 10 client.26140 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T09:42:15.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:15.065+0000 7f6a7ea00640 1 -- 172.21.15.5:0/582936069 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f4ee422990 con 0x55f4ee4cb4b0 2024-09-10T09:42:15.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:15.065+0000 7f6a7ea00640 10 client.26140 handle_command_reply: tid=0 2024-09-10T09:42:15.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:15.065+0000 7f6a9a5db740 1 client.26140 shutdown 2024-09-10T09:42:15.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:15.065+0000 7f6a9a5db740 1 -- 172.21.15.5:0/582936069 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f4ee4bb6c0 msgr2=0x55f4ee4b9000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:42:15.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:15.065+0000 7f6a9a5db740 1 --2- 172.21.15.5:0/582936069 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f4ee4bb6c0 0x55f4ee4b9000 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f6a8802ae80 tx=0x7f6a88031ae0 comp rx=0 tx=0).stop 2024-09-10T09:42:15.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:15.065+0000 7f6a9a5db740 1 -- 172.21.15.5:0/582936069 shutdown_connections 2024-09-10T09:42:15.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:15.065+0000 7f6a9a5db740 1 --2- 172.21.15.5:0/582936069 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55f4ee4cb4b0 0x55f4ee4cd950 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:15.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:15.065+0000 7f6a9a5db740 1 --2- 172.21.15.5:0/582936069 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f4ee4bb6c0 0x55f4ee4b9000 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:15.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:15.065+0000 7f6a9a5db740 1 --2- 172.21.15.5:0/582936069 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f4ee4b0e00 0x55f4ee4b8ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:15.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:15.065+0000 7f6a9a5db740 1 --2- 172.21.15.5:0/582936069 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f4ee4b04e0 0x55f4ee4b8580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:15.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:15.065+0000 7f6a9a5db740 1 -- 172.21.15.5:0/582936069 >> 172.21.15.5:0/582936069 conn(0x55f4ee4aca70 msgr2=0x55f4ee4abbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:42:15.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:15.065+0000 7f6a7ea00640 0 client.26140 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T09:42:15.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:15.066+0000 7f6a9a5db740 1 -- 172.21.15.5:0/582936069 shutdown_connections 2024-09-10T09:42:15.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:15.066+0000 7f6a9a5db740 1 -- 172.21.15.5:0/582936069 wait complete. 2024-09-10T09:42:15.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:15.066+0000 7f6a9a5db740 20 client.26140 trim_cache size 0 max 16384 2024-09-10T09:42:15.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:15.085+0000 7f6a9a5db740 1 Processor -- start 2024-09-10T09:42:15.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:15.085+0000 7f6a9a5db740 1 -- start start 2024-09-10T09:42:15.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:15.085+0000 7f6a9a5db740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f4ee4b04e0 0x55f4ee4b2f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:42:15.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:15.085+0000 7f6a9a5db740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f4ee4b0e00 0x55f4ee4b74c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:42:15.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:15.085+0000 7f6a9a5db740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f4ee4bb6c0 0x55f4ee4d3260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:42:15.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:15.085+0000 7f6a9a5db740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f6a901b54a0 con 0x55f4ee4b0e00 2024-09-10T09:42:15.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:15.085+0000 7f6a9a5db740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f6a901b70a0 con 0x55f4ee4b04e0 2024-09-10T09:42:15.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:15.085+0000 7f6a9a5db740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f6a901b5b70 con 0x55f4ee4bb6c0 2024-09-10T09:42:15.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:15.086+0000 7f6a95000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f4ee4b04e0 0x55f4ee4b2f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:42:15.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:15.086+0000 7f6a87e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f4ee4b0e00 0x55f4ee4b74c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:42:15.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:15.086+0000 7f6a95a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f4ee4bb6c0 0x55f4ee4d3260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:42:15.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:15.086+0000 7f6a95000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f4ee4b04e0 0x55f4ee4b2f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:43632/0 (socket says 172.21.15.5:43632) 2024-09-10T09:42:15.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:15.086+0000 7f6a95000640 1 -- 172.21.15.5:0/473057709 learned_addr learned my addr 172.21.15.5:0/473057709 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:42:15.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:15.086+0000 7f6a95000640 1 -- 172.21.15.5:0/473057709 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f4ee4bb6c0 msgr2=0x55f4ee4d3260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:42:15.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:15.086+0000 7f6a95000640 1 --2- 172.21.15.5:0/473057709 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f4ee4bb6c0 0x55f4ee4d3260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:15.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:15.086+0000 7f6a95000640 1 -- 172.21.15.5:0/473057709 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f4ee4b0e00 msgr2=0x55f4ee4b74c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:42:15.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:15.086+0000 7f6a95000640 1 --2- 172.21.15.5:0/473057709 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f4ee4b0e00 0x55f4ee4b74c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:15.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:15.086+0000 7f6a95000640 1 -- 172.21.15.5:0/473057709 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f4ee4a6fe0 con 0x55f4ee4b04e0 2024-09-10T09:42:15.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:15.086+0000 7f6a95000640 1 --2- 172.21.15.5:0/473057709 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f4ee4b04e0 0x55f4ee4b2f40 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f6a80005d00 tx=0x7f6a80015000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aadea6dad3c0c71e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:42:15.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:15.087+0000 7f6a85600640 1 -- 172.21.15.5:0/473057709 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a8002c070 con 0x55f4ee4b04e0 2024-09-10T09:42:15.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:15.087+0000 7f6a85600640 1 -- 172.21.15.5:0/473057709 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6a8001f8d0 con 0x55f4ee4b04e0 2024-09-10T09:42:15.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:15.087+0000 7f6a85600640 1 -- 172.21.15.5:0/473057709 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a80026840 con 0x55f4ee4b04e0 2024-09-10T09:42:15.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:15.087+0000 7f6a9a5db740 1 -- 172.21.15.5:0/473057709 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f4ee4b04e0 msgr2=0x55f4ee4b2f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:42:15.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:15.087+0000 7f6a9a5db740 1 --2- 172.21.15.5:0/473057709 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f4ee4b04e0 0x55f4ee4b2f40 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f6a80005d00 tx=0x7f6a80015000 comp rx=0 tx=0).stop 2024-09-10T09:42:15.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:15.088+0000 7f6a9a5db740 1 -- 172.21.15.5:0/473057709 shutdown_connections 2024-09-10T09:42:15.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:15.088+0000 7f6a9a5db740 1 --2- 172.21.15.5:0/473057709 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f4ee4bb6c0 0x55f4ee4d3260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:15.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:15.088+0000 7f6a9a5db740 1 --2- 172.21.15.5:0/473057709 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f4ee4b0e00 0x55f4ee4b74c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:15.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:15.088+0000 7f6a9a5db740 1 --2- 172.21.15.5:0/473057709 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f4ee4b04e0 0x55f4ee4b2f40 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:15.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:15.088+0000 7f6a9a5db740 1 -- 172.21.15.5:0/473057709 >> 172.21.15.5:0/473057709 conn(0x55f4ee4aca70 msgr2=0x55f4ee4ba4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:42:15.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:15.088+0000 7f6a9a5db740 1 -- 172.21.15.5:0/473057709 shutdown_connections 2024-09-10T09:42:15.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:15.088+0000 7f6a9a5db740 1 -- 172.21.15.5:0/473057709 wait complete. 2024-09-10T09:42:15.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:15.088+0000 7f6a9a5db740 1 -- start start 2024-09-10T09:42:15.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:15.089+0000 7f6a9a5db740 1 Processor -- start 2024-09-10T09:42:15.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:15.089+0000 7f6a9a5db740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f4ee4b04e0 0x55f4ee4b5550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:42:15.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:15.089+0000 7f6a9a5db740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f4ee4b0e00 0x55f4ee4b5a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:42:15.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:15.089+0000 7f6a9a5db740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f4ee4bb6c0 0x55f4ee4b5fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:42:15.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:15.089+0000 7f6a9a5db740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f6a901b6910 con 0x55f4ee4b0e00 2024-09-10T09:42:15.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:15.089+0000 7f6a9a5db740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f6a901b6240 con 0x55f4ee4bb6c0 2024-09-10T09:42:15.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:15.089+0000 7f6a9a5db740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f6a901b3480 con 0x55f4ee4b04e0 2024-09-10T09:42:15.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:15.089+0000 7f6a9a5db740 10 client.0 _ll_register_callbacks cb 0x7f6a88009900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:42:15.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:15.089+0000 7f6a95000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f4ee4b04e0 0x55f4ee4b5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:42:15.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:15.089+0000 7f6a95000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f4ee4b04e0 0x55f4ee4b5550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:48270/0 (socket says 172.21.15.5:48270) 2024-09-10T09:42:15.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:15.089+0000 7f6a95000640 1 -- 172.21.15.5:0/2297240684 learned_addr learned my addr 172.21.15.5:0/2297240684 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:42:15.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:15.089+0000 7f6a87e00640 1 --2- 172.21.15.5:0/2297240684 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f4ee4b0e00 0x55f4ee4b5a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:42:15.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:15.090+0000 7f6a95a00640 1 --2- 172.21.15.5:0/2297240684 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f4ee4bb6c0 0x55f4ee4b5fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:42:15.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:15.090+0000 7f6a95000640 1 -- 172.21.15.5:0/2297240684 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f4ee4bb6c0 msgr2=0x55f4ee4b5fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:42:15.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:15.090+0000 7f6a95000640 1 --2- 172.21.15.5:0/2297240684 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f4ee4bb6c0 0x55f4ee4b5fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:15.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:15.090+0000 7f6a95000640 1 -- 172.21.15.5:0/2297240684 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f4ee4b0e00 msgr2=0x55f4ee4b5a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:42:15.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:15.090+0000 7f6a95000640 1 --2- 172.21.15.5:0/2297240684 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f4ee4b0e00 0x55f4ee4b5a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:15.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:15.090+0000 7f6a95000640 1 -- 172.21.15.5:0/2297240684 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f4ee4adb20 con 0x55f4ee4b04e0 2024-09-10T09:42:15.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:15.090+0000 7f6a95000640 1 --2- 172.21.15.5:0/2297240684 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f4ee4b04e0 0x55f4ee4b5550 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f6a80026460 tx=0x7f6a8000baa0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7cacb1cf68ea5c9c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:42:15.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:15.090+0000 7f6a7ea00640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-09-10T09:42:15.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:15.091+0000 7f6a7ea00640 1 -- 172.21.15.5:0/2297240684 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a8002c040 con 0x55f4ee4b04e0 2024-09-10T09:42:15.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:15.091+0000 7f6a9a5db740 1 -- 172.21.15.5:0/2297240684 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6a8001f8d0 con 0x55f4ee4b04e0 2024-09-10T09:42:15.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:15.091+0000 7f6a7ea00640 1 -- 172.21.15.5:0/2297240684 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f6a8001f5f0 con 0x55f4ee4b04e0 2024-09-10T09:42:15.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:15.091+0000 7f6a7ea00640 1 -- 172.21.15.5:0/2297240684 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f6a8001fa80 con 0x55f4ee4b04e0 2024-09-10T09:42:15.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:15.091+0000 7f6a9a5db740 10 client.26146 fetch_fsmap learned FSMap version 32 2024-09-10T09:42:15.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:15.091+0000 7f6a9a5db740 1 -- 172.21.15.5:0/2297240684 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55f4ee5362f0 con 0x55f4ee4b04e0 2024-09-10T09:42:15.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:15.091+0000 7f6a7ea00640 1 -- 172.21.15.5:0/2297240684 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6a8001f8d0 con 0x55f4ee4b04e0 2024-09-10T09:42:15.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:15.091+0000 7f6a7ea00640 1 -- 172.21.15.5:0/2297240684 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7f6a80038030 con 0x55f4ee4b04e0 2024-09-10T09:42:15.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:15.091+0000 7f6a9a5db740 10 client.26146 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T09:42:15.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:15.091+0000 7f6a9a5db740 10 client.26146 resolve_mds: validated gid 24542 aka daemon mds.b 2024-09-10T09:42:15.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:15.091+0000 7f6a9a5db740 20 client.26146 populate_metadata read hostname 'smithi005' 2024-09-10T09:42:15.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:15.091+0000 7f6a9a5db740 1 --2- 172.21.15.5:0/2297240684 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55f4ee536b10 0x55f4ee538f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:42:15.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:15.091+0000 7f6a9a5db740 4 client.26146 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 142.75, "await_for": 45.0}] 2024-09-10T09:42:15.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:15.091+0000 7f6a9a5db740 1 -- 172.21.15.5:0/2297240684 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 142.75, "await_for": 45.0}) -- 0x55f4ee41cce0 con 0x55f4ee536b10 2024-09-10T09:42:15.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:15.092+0000 7f6a87e00640 1 --2- 172.21.15.5:0/2297240684 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55f4ee536b10 0x55f4ee538f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:42:15.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:15.092+0000 7f6a87e00640 1 --2- 172.21.15.5:0/2297240684 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55f4ee536b10 0x55f4ee538f30 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ff5883621ad53a08 server_cookie=44253dbd415cdc21 in_seq=0 out_seq=0 2024-09-10T09:42:15.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:15.092+0000 7f6a7ea00640 10 client.26146 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T09:42:15.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:15.287+0000 7f6a7ea00640 1 -- 172.21.15.5:0/2297240684 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55f4ee41cce0 con 0x55f4ee536b10 2024-09-10T09:42:15.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:15.287+0000 7f6a7ea00640 10 client.26146 handle_command_reply: tid=0 2024-09-10T09:42:15.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:15.287+0000 7f6a9a5db740 1 client.26146 shutdown 2024-09-10T09:42:15.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:15.287+0000 7f6a9a5db740 1 -- 172.21.15.5:0/2297240684 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f4ee4b04e0 msgr2=0x55f4ee4b5550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:42:15.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:15.287+0000 7f6a9a5db740 1 --2- 172.21.15.5:0/2297240684 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f4ee4b04e0 0x55f4ee4b5550 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f6a80026460 tx=0x7f6a8000baa0 comp rx=0 tx=0).stop 2024-09-10T09:42:15.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:15.287+0000 7f6a9a5db740 1 -- 172.21.15.5:0/2297240684 shutdown_connections 2024-09-10T09:42:15.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:15.287+0000 7f6a9a5db740 1 --2- 172.21.15.5:0/2297240684 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55f4ee536b10 0x55f4ee538f30 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:15.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:15.287+0000 7f6a9a5db740 1 --2- 172.21.15.5:0/2297240684 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f4ee4bb6c0 0x55f4ee4b5fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:15.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:15.287+0000 7f6a9a5db740 1 --2- 172.21.15.5:0/2297240684 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f4ee4b0e00 0x55f4ee4b5a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:15.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:15.287+0000 7f6a9a5db740 1 --2- 172.21.15.5:0/2297240684 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f4ee4b04e0 0x55f4ee4b5550 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:15.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:15.287+0000 7f6a7ea00640 0 client.26146 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T09:42:15.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:15.287+0000 7f6a9a5db740 1 -- 172.21.15.5:0/2297240684 >> 172.21.15.5:0/2297240684 conn(0x55f4ee4aca70 msgr2=0x55f4ee4ab3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:42:15.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:15.287+0000 7f6a9a5db740 1 -- 172.21.15.5:0/2297240684 shutdown_connections 2024-09-10T09:42:15.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:15.288+0000 7f6a9a5db740 1 -- 172.21.15.5:0/2297240684 wait complete. 2024-09-10T09:42:15.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:15.288+0000 7f6a9a5db740 20 client.26146 trim_cache size 0 max 16384 2024-09-10T09:42:15.291 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-10T09:42:15.291 INFO:teuthology.orchestra.run.smithi005.stdout: "epoch": 32, 2024-09-10T09:42:15.291 INFO:teuthology.orchestra.run.smithi005.stdout: "leader": 24542, 2024-09-10T09:42:15.291 INFO:teuthology.orchestra.run.smithi005.stdout: "set_version": 10, 2024-09-10T09:42:15.292 INFO:teuthology.orchestra.run.smithi005.stdout: "sets": { 2024-09-10T09:42:15.292 INFO:teuthology.orchestra.run.smithi005.stdout: "ba5a0a38": { 2024-09-10T09:42:15.292 INFO:teuthology.orchestra.run.smithi005.stdout: "version": 10, 2024-09-10T09:42:15.292 INFO:teuthology.orchestra.run.smithi005.stdout: "age_ref": 0.0, 2024-09-10T09:42:15.292 INFO:teuthology.orchestra.run.smithi005.stdout: "state": { 2024-09-10T09:42:15.292 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "QUIESCED", 2024-09-10T09:42:15.292 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0 2024-09-10T09:42:15.292 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T09:42:15.292 INFO:teuthology.orchestra.run.smithi005.stdout: "timeout": 90.0, 2024-09-10T09:42:15.292 INFO:teuthology.orchestra.run.smithi005.stdout: "expiration": 142.8, 2024-09-10T09:42:15.292 INFO:teuthology.orchestra.run.smithi005.stdout: "members": { 2024-09-10T09:42:15.292 INFO:teuthology.orchestra.run.smithi005.stdout: "file:/": { 2024-09-10T09:42:15.292 INFO:teuthology.orchestra.run.smithi005.stdout: "excluded": false, 2024-09-10T09:42:15.293 INFO:teuthology.orchestra.run.smithi005.stdout: "state": { 2024-09-10T09:42:15.293 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "QUIESCED", 2024-09-10T09:42:15.293 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0 2024-09-10T09:42:15.293 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T09:42:15.293 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T09:42:15.293 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T09:42:15.293 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T09:42:15.293 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T09:42:15.293 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-10T09:42:15.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:15.288+0000 7f6a97a00640 1 -- 172.21.15.5:0/1983818463 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f6a740a2b80 msgr2=0x7f6a740a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:42:15.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:15.288+0000 7f6a97a00640 1 --2- 172.21.15.5:0/1983818463 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f6a740a2b80 0x7f6a740a5040 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x55f4ee4ad8d0 tx=0x7f6a8000b390 comp rx=0 tx=0).stop 2024-09-10T09:42:15.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:15.288+0000 7f6a97a00640 1 -- 172.21.15.5:0/1983818463 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6a90112a30 msgr2=0x7f6a9019e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:42:15.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:15.288+0000 7f6a97a00640 1 --2- 172.21.15.5:0/1983818463 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6a90112a30 0x7f6a9019e890 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f6a88005e30 tx=0x7f6a88003f10 comp rx=0 tx=0).stop 2024-09-10T09:42:15.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:15.288+0000 7f6a97a00640 1 -- 172.21.15.5:0/1983818463 shutdown_connections 2024-09-10T09:42:15.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:15.288+0000 7f6a97a00640 1 --2- 172.21.15.5:0/1983818463 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f6a740a2b80 0x7f6a740a5040 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:15.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:15.289+0000 7f6a97a00640 1 --2- 172.21.15.5:0/1983818463 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6a90112a30 0x7f6a9019e890 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:15.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:15.289+0000 7f6a97a00640 1 --2- 172.21.15.5:0/1983818463 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6a90077040 0x7f6a901a4c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:15.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:15.289+0000 7f6a97a00640 1 --2- 172.21.15.5:0/1983818463 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6a90076720 0x7f6a901a46d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:15.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:15.289+0000 7f6a97a00640 1 -- 172.21.15.5:0/1983818463 >> 172.21.15.5:0/1983818463 conn(0x7f6a900ffa90 msgr2=0x7f6a90101e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:42:15.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:15.289+0000 7f6a97a00640 1 -- 172.21.15.5:0/1983818463 shutdown_connections 2024-09-10T09:42:15.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:15.289+0000 7f6a97a00640 1 -- 172.21.15.5:0/1983818463 wait complete. 2024-09-10T09:42:15.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:42:15 smithi005 ceph-mon[30350]: pgmap v510: 129 pgs: 129 active+clean; 90 MiB data, 812 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.0 MiB/s wr, 195 op/s 2024-09-10T09:42:15.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:42:15 smithi159 ceph-mon[35513]: pgmap v510: 129 pgs: 129 active+clean; 90 MiB data, 812 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.0 MiB/s wr, 195 op/s 2024-09-10T09:42:15.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:42:15 smithi102 ceph-mon[35442]: pgmap v510: 129 pgs: 129 active+clean; 90 MiB data, 812 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.0 MiB/s wr, 195 op/s 2024-09-10T09:42:16.014 INFO:tasks.workunit.client.0.smithi005.stdout: 1 191939 9.00 MB/sec warmup 117 sec latency 919.631 ms 2024-09-10T09:42:16.180 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'ba5a0a38', quiesce took 4.8 seconds. Will release after: 17.95 2024-09-10T09:42:17.014 INFO:tasks.workunit.client.0.smithi005.stdout: 1 191939 8.93 MB/sec warmup 118 sec latency 1919.714 ms 2024-09-10T09:42:17.686 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:42:17 smithi005 ceph-mon[30350]: pgmap v511: 129 pgs: 129 active+clean; 93 MiB data, 833 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 2.3 MiB/s wr, 255 op/s 2024-09-10T09:42:17.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:42:17 smithi159 ceph-mon[35513]: pgmap v511: 129 pgs: 129 active+clean; 93 MiB data, 833 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 2.3 MiB/s wr, 255 op/s 2024-09-10T09:42:17.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:42:17 smithi102 ceph-mon[35442]: pgmap v511: 129 pgs: 129 active+clean; 93 MiB data, 833 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 2.3 MiB/s wr, 255 op/s 2024-09-10T09:42:18.014 INFO:tasks.workunit.client.0.smithi005.stdout: 1 191939 8.85 MB/sec warmup 119 sec latency 2919.780 ms 2024-09-10T09:42:18.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:42:18 smithi005 ceph-mon[30350]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T09:42:18.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:42:18 smithi159 ceph-mon[35513]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T09:42:18.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:42:18 smithi102 ceph-mon[35442]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T09:42:19.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:42:19 smithi159 ceph-mon[35513]: pgmap v512: 129 pgs: 129 active+clean; 93 MiB data, 843 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 1.9 MiB/s wr, 193 op/s 2024-09-10T09:42:19.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:42:19 smithi102 ceph-mon[35442]: pgmap v512: 129 pgs: 129 active+clean; 93 MiB data, 843 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 1.9 MiB/s wr, 193 op/s 2024-09-10T09:42:20.015 INFO:tasks.workunit.client.0.smithi005.stdout: 1 191939 0.00 MB/sec execute 1 sec latency 4919.916 ms 2024-09-10T09:42:20.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:42:19 smithi005 ceph-mon[30350]: pgmap v512: 129 pgs: 129 active+clean; 93 MiB data, 843 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 1.9 MiB/s wr, 193 op/s 2024-09-10T09:42:21.015 INFO:tasks.workunit.client.0.smithi005.stdout: 1 191939 0.00 MB/sec execute 2 sec latency 5919.991 ms 2024-09-10T09:42:21.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:42:21 smithi159 ceph-mon[35513]: pgmap v513: 129 pgs: 129 active+clean; 95 MiB data, 852 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.1 MiB/s wr, 209 op/s 2024-09-10T09:42:21.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:42:21 smithi102 ceph-mon[35442]: pgmap v513: 129 pgs: 129 active+clean; 95 MiB data, 852 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.1 MiB/s wr, 209 op/s 2024-09-10T09:42:21.973 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:42:21 smithi005 ceph-mon[30350]: pgmap v513: 129 pgs: 129 active+clean; 95 MiB data, 852 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.1 MiB/s wr, 209 op/s 2024-09-10T09:42:22.015 INFO:tasks.workunit.client.0.smithi005.stdout: 1 191939 0.00 MB/sec execute 3 sec latency 6920.043 ms 2024-09-10T09:42:23.015 INFO:tasks.workunit.client.0.smithi005.stdout: 1 191939 0.00 MB/sec execute 4 sec latency 7920.152 ms 2024-09-10T09:42:23.698 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T09:42:23.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:42:23 smithi159 ceph-mon[35513]: pgmap v514: 129 pgs: 129 active+clean; 98 MiB data, 853 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.9 MiB/s wr, 141 op/s 2024-09-10T09:42:23.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:42:23 smithi102 ceph-mon[35442]: pgmap v514: 129 pgs: 129 active+clean; 98 MiB data, 853 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.9 MiB/s wr, 141 op/s 2024-09-10T09:42:24.015 INFO:tasks.workunit.client.0.smithi005.stdout: 1 191939 0.00 MB/sec execute 5 sec latency 8920.210 ms 2024-09-10T09:42:24.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:42:23 smithi005 ceph-mon[30350]: pgmap v514: 129 pgs: 129 active+clean; 98 MiB data, 853 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.9 MiB/s wr, 141 op/s 2024-09-10T09:42:24.089 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:42:24.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:24.713+0000 7f49cbe00640 1 -- 172.21.15.5:0/3553334691 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f49c4102790 msgr2=0x7f49c41067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:42:24.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:24.713+0000 7f49cbe00640 1 --2- 172.21.15.5:0/3553334691 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f49c4102790 0x7f49c41067e0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f49b0009900 tx=0x7f49b002a410 comp rx=0 tx=0).stop 2024-09-10T09:42:24.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:24.714+0000 7f49cbe00640 1 -- 172.21.15.5:0/3553334691 shutdown_connections 2024-09-10T09:42:24.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:24.714+0000 7f49cbe00640 1 --2- 172.21.15.5:0/3553334691 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f49c4106f10 0x7f49c410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:24.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:24.714+0000 7f49cbe00640 1 --2- 172.21.15.5:0/3553334691 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f49c4102790 0x7f49c41067e0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:24.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:24.714+0000 7f49cbe00640 1 --2- 172.21.15.5:0/3553334691 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f49c4101de0 0x7f49c41021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:24.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:24.714+0000 7f49cbe00640 1 -- 172.21.15.5:0/3553334691 >> 172.21.15.5:0/3553334691 conn(0x7f49c40fd8c0 msgr2=0x7f49c40ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:42:24.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:24.714+0000 7f49cbe00640 1 -- 172.21.15.5:0/3553334691 shutdown_connections 2024-09-10T09:42:24.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:24.714+0000 7f49cbe00640 1 -- 172.21.15.5:0/3553334691 wait complete. 2024-09-10T09:42:24.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:24.715+0000 7f49cbe00640 1 Processor -- start 2024-09-10T09:42:24.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:24.715+0000 7f49cbe00640 1 -- start start 2024-09-10T09:42:24.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:24.715+0000 7f49cbe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f49c4101de0 0x7f49c419a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:42:24.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:24.715+0000 7f49cbe00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f49c4102790 0x7f49c419ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:42:24.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:24.715+0000 7f49cbe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f49c4106f10 0x7f49c419ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:42:24.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:24.715+0000 7f49cbe00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f49c4111f30 con 0x7f49c4101de0 2024-09-10T09:42:24.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:24.715+0000 7f49cbe00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f49c4111db0 con 0x7f49c4106f10 2024-09-10T09:42:24.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:24.715+0000 7f49cbe00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f49c41120b0 con 0x7f49c4102790 2024-09-10T09:42:24.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:24.716+0000 7f49c9400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f49c4101de0 0x7f49c419a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:42:24.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:24.716+0000 7f49c9400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f49c4101de0 0x7f49c419a630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35546/0 (socket says 172.21.15.5:35546) 2024-09-10T09:42:24.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:24.716+0000 7f49c9400640 1 -- 172.21.15.5:0/3277515101 learned_addr learned my addr 172.21.15.5:0/3277515101 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:42:24.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:24.716+0000 7f49c9e00640 1 --2- 172.21.15.5:0/3277515101 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f49c4106f10 0x7f49c419ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:42:24.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:24.716+0000 7f49c8a00640 1 --2- 172.21.15.5:0/3277515101 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f49c4102790 0x7f49c419ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:42:24.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:24.716+0000 7f49c9400640 1 -- 172.21.15.5:0/3277515101 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f49c4102790 msgr2=0x7f49c419ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:42:24.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:24.716+0000 7f49c9400640 1 --2- 172.21.15.5:0/3277515101 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f49c4102790 0x7f49c419ab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:24.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:24.716+0000 7f49c9400640 1 -- 172.21.15.5:0/3277515101 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f49c4106f10 msgr2=0x7f49c419ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:42:24.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:24.716+0000 7f49c9400640 1 --2- 172.21.15.5:0/3277515101 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f49c4106f10 0x7f49c419ef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:24.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:24.716+0000 7f49c9400640 1 -- 172.21.15.5:0/3277515101 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49c419f5e0 con 0x7f49c4101de0 2024-09-10T09:42:24.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:24.716+0000 7f49c8a00640 1 --2- 172.21.15.5:0/3277515101 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f49c4102790 0x7f49c419ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:42:24.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:24.716+0000 7f49c9e00640 1 --2- 172.21.15.5:0/3277515101 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f49c4106f10 0x7f49c419ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:42:24.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:24.716+0000 7f49c9400640 1 --2- 172.21.15.5:0/3277515101 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f49c4101de0 0x7f49c419a630 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f49b800d950 tx=0x7f49b800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:42:24.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:24.717+0000 7f49c2000640 1 -- 172.21.15.5:0/3277515101 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49b8017030 con 0x7f49c4101de0 2024-09-10T09:42:24.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:24.717+0000 7f49c2000640 1 -- 172.21.15.5:0/3277515101 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f49b8004020 con 0x7f49c4101de0 2024-09-10T09:42:24.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:24.717+0000 7f49c2000640 1 -- 172.21.15.5:0/3277515101 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49b80047a0 con 0x7f49c4101de0 2024-09-10T09:42:24.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:24.717+0000 7f49cbe00640 1 -- 172.21.15.5:0/3277515101 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49c419f8d0 con 0x7f49c4101de0 2024-09-10T09:42:24.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:24.717+0000 7f49cbe00640 1 -- 172.21.15.5:0/3277515101 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49c41a71b0 con 0x7f49c4101de0 2024-09-10T09:42:24.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:24.720+0000 7f49ce7d3740 1 Processor -- start 2024-09-10T09:42:24.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:24.720+0000 7f49c2000640 1 -- 172.21.15.5:0/3277515101 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f49b80041d0 con 0x7f49c4101de0 2024-09-10T09:42:24.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:24.720+0000 7f49ce7d3740 1 -- start start 2024-09-10T09:42:24.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:24.720+0000 7f49ce7d3740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b4679c4b90 0x55b4679c4f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:42:24.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:24.720+0000 7f49ce7d3740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b4679c54b0 0x55b4679cf830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:42:24.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:24.720+0000 7f49ce7d3740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b4679cfd70 0x55b4679d2160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:42:24.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:24.721+0000 7f49ce7d3740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55b46792b3e0 con 0x55b4679c54b0 2024-09-10T09:42:24.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:24.721+0000 7f49ce7d3740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55b4678d2d80 con 0x55b4679cfd70 2024-09-10T09:42:24.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:24.721+0000 7f49ce7d3740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55b4679273a0 con 0x55b4679c4b90 2024-09-10T09:42:24.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:24.721+0000 7f49c9400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b4679c54b0 0x55b4679cf830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:42:24.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:24.721+0000 7f49c8a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b4679cfd70 0x55b4679d2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:42:24.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:24.721+0000 7f49c9400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b4679c54b0 0x55b4679cf830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35562/0 (socket says 172.21.15.5:35562) 2024-09-10T09:42:24.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:24.721+0000 7f49c9e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b4679c4b90 0x55b4679c4f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:42:24.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:24.721+0000 7f49c9400640 1 -- 172.21.15.5:0/1924864123 learned_addr learned my addr 172.21.15.5:0/1924864123 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:42:24.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:24.721+0000 7f49c8a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b4679cfd70 0x55b4679d2160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:40424/0 (socket says 172.21.15.5:40424) 2024-09-10T09:42:24.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:24.721+0000 7f49c8a00640 1 -- 172.21.15.5:0/1924864123 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b4679c4b90 msgr2=0x55b4679c4f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:42:24.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:24.721+0000 7f49c8a00640 1 --2- 172.21.15.5:0/1924864123 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b4679c4b90 0x55b4679c4f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:24.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:24.721+0000 7f49c8a00640 1 -- 172.21.15.5:0/1924864123 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b4679c54b0 msgr2=0x55b4679cf830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:42:24.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:24.721+0000 7f49c8a00640 1 --2- 172.21.15.5:0/1924864123 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b4679c54b0 0x55b4679cf830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:24.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:24.721+0000 7f49c8a00640 1 -- 172.21.15.5:0/1924864123 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b4678f1ae0 con 0x55b4679cfd70 2024-09-10T09:42:24.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:24.722+0000 7f49c8a00640 1 --2- 172.21.15.5:0/1924864123 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b4679cfd70 0x55b4679d2160 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f49b0009230 tx=0x7f49b0005e60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d1711f926dde7f2c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:42:24.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:24.722+0000 7f49c2000640 1 --2- 172.21.15.5:0/3277515101 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f49ac0a2c00 0x7f49ac0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:42:24.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:24.722+0000 7f49b6600640 1 -- 172.21.15.5:0/1924864123 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49b003f070 con 0x55b4679cfd70 2024-09-10T09:42:24.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:24.722+0000 7f49b6600640 1 -- 172.21.15.5:0/1924864123 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f49b0004030 con 0x55b4679cfd70 2024-09-10T09:42:24.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:24.722+0000 7f49b6600640 1 -- 172.21.15.5:0/1924864123 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49b0035070 con 0x55b4679cfd70 2024-09-10T09:42:24.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:24.722+0000 7f49c2000640 1 -- 172.21.15.5:0/3277515101 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f49b80ceeb0 con 0x7f49c4101de0 2024-09-10T09:42:24.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:24.722+0000 7f49c9e00640 1 --2- 172.21.15.5:0/3277515101 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f49ac0a2c00 0x7f49ac0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:42:24.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:24.723+0000 7f49ce7d3740 1 -- 172.21.15.5:0/1924864123 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b4679cfd70 msgr2=0x55b4679d2160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:42:24.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:24.723+0000 7f49c9e00640 1 --2- 172.21.15.5:0/3277515101 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f49ac0a2c00 0x7f49ac0a50c0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f49bc005fd0 tx=0x7f49bc009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:42:24.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:24.723+0000 7f49ce7d3740 1 --2- 172.21.15.5:0/1924864123 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b4679cfd70 0x55b4679d2160 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f49b0009230 tx=0x7f49b0005e60 comp rx=0 tx=0).stop 2024-09-10T09:42:24.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:24.723+0000 7f49ce7d3740 1 -- 172.21.15.5:0/1924864123 shutdown_connections 2024-09-10T09:42:24.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:24.723+0000 7f49ce7d3740 1 --2- 172.21.15.5:0/1924864123 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b4679cfd70 0x55b4679d2160 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:24.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:24.723+0000 7f49ce7d3740 1 --2- 172.21.15.5:0/1924864123 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b4679c54b0 0x55b4679cf830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:24.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:24.723+0000 7f49ce7d3740 1 --2- 172.21.15.5:0/1924864123 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b4679c4b90 0x55b4679c4f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:24.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:24.723+0000 7f49ce7d3740 1 -- 172.21.15.5:0/1924864123 >> 172.21.15.5:0/1924864123 conn(0x55b4679c1120 msgr2=0x55b4679c1500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:42:24.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:24.723+0000 7f49ce7d3740 1 -- 172.21.15.5:0/1924864123 shutdown_connections 2024-09-10T09:42:24.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:24.723+0000 7f49ce7d3740 1 -- 172.21.15.5:0/1924864123 wait complete. 2024-09-10T09:42:24.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:24.724+0000 7f49ce7d3740 1 -- start start 2024-09-10T09:42:24.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:24.724+0000 7f49ce7d3740 1 Processor -- start 2024-09-10T09:42:24.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:24.724+0000 7f49ce7d3740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b4679c4b90 0x55b4679ccc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:42:24.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:24.724+0000 7f49ce7d3740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b4679c54b0 0x55b4679cd160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:42:24.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:24.724+0000 7f49ce7d3740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b4679cfd70 0x55b4679cd6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:42:24.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:24.724+0000 7f49ce7d3740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55b4679d29e0 con 0x55b4679c4b90 2024-09-10T09:42:24.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:24.724+0000 7f49ce7d3740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55b4678e60f0 con 0x55b4679c54b0 2024-09-10T09:42:24.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:24.724+0000 7f49ce7d3740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55b4679d2b60 con 0x55b4679cfd70 2024-09-10T09:42:24.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:24.725+0000 7f49ce7d3740 10 client.0 _ll_register_callbacks cb 0x55b4678d79e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:42:24.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:24.725+0000 7f49c9400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b4679c4b90 0x55b4679ccc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:42:24.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:24.725+0000 7f49c9400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b4679c4b90 0x55b4679ccc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35576/0 (socket says 172.21.15.5:35576) 2024-09-10T09:42:24.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:24.725+0000 7f49c9400640 1 -- 172.21.15.5:0/2331028542 learned_addr learned my addr 172.21.15.5:0/2331028542 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:42:24.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:24.725+0000 7f49c9e00640 1 --2- 172.21.15.5:0/2331028542 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b4679cfd70 0x55b4679cd6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:42:24.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:24.725+0000 7f49c8a00640 1 --2- 172.21.15.5:0/2331028542 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b4679c54b0 0x55b4679cd160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:42:24.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:24.725+0000 7f49c9400640 1 -- 172.21.15.5:0/2331028542 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b4679cfd70 msgr2=0x55b4679cd6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:42:24.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:24.725+0000 7f49c9400640 1 --2- 172.21.15.5:0/2331028542 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b4679cfd70 0x55b4679cd6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:24.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:24.725+0000 7f49c9400640 1 -- 172.21.15.5:0/2331028542 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b4679c54b0 msgr2=0x55b4679cd160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:42:24.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:24.725+0000 7f49c9400640 1 --2- 172.21.15.5:0/2331028542 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b4679c54b0 0x55b4679cd160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:24.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:24.725+0000 7f49c9400640 1 -- 172.21.15.5:0/2331028542 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b46769c3c0 con 0x55b4679c4b90 2024-09-10T09:42:24.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:24.726+0000 7f49c9400640 1 --2- 172.21.15.5:0/2331028542 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b4679c4b90 0x55b4679ccc20 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f49b8005fd0 tx=0x7f49b80cc4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a24eba55341e0f3f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:42:24.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:24.726+0000 7f49aaa00640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-10T09:42:24.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:24.726+0000 7f49aaa00640 1 -- 172.21.15.5:0/2331028542 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49b809ae00 con 0x55b4679c4b90 2024-09-10T09:42:24.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:24.726+0000 7f49ce7d3740 1 -- 172.21.15.5:0/2331028542 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f49b0004030 con 0x55b4679c4b90 2024-09-10T09:42:24.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:24.726+0000 7f49aaa00640 1 -- 172.21.15.5:0/2331028542 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f49b80a3aa0 con 0x55b4679c4b90 2024-09-10T09:42:24.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:24.726+0000 7f49aaa00640 1 -- 172.21.15.5:0/2331028542 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f49b809ac20 con 0x55b4679c4b90 2024-09-10T09:42:24.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:24.726+0000 7f49ce7d3740 10 client.16134 fetch_fsmap learned FSMap version 33 2024-09-10T09:42:24.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:24.726+0000 7f49ce7d3740 1 -- 172.21.15.5:0/2331028542 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55b4679de710 con 0x55b4679c4b90 2024-09-10T09:42:24.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:24.726+0000 7f49aaa00640 1 -- 172.21.15.5:0/2331028542 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f49b0004030 con 0x55b4679c4b90 2024-09-10T09:42:24.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:24.726+0000 7f49aaa00640 1 -- 172.21.15.5:0/2331028542 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 33) ==== 8374+0+0 (secure 0 0 0) 0x7f49b80bb0f0 con 0x55b4679c4b90 2024-09-10T09:42:24.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:24.727+0000 7f49ce7d3740 10 client.16134 fetch_fsmap finished waiting for FSMap version 33 2024-09-10T09:42:24.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:24.727+0000 7f49ce7d3740 10 client.16134 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T09:42:24.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:24.727+0000 7f49ce7d3740 20 client.16134 populate_metadata read hostname 'smithi005' 2024-09-10T09:42:24.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:24.727+0000 7f49ce7d3740 1 --2- 172.21.15.5:0/2331028542 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55b4679dfb20 0x55b4679e1f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:42:24.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:24.727+0000 7f49ce7d3740 4 client.16134 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:42:24.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:24.727+0000 7f49ce7d3740 1 -- 172.21.15.5:0/2331028542 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b467938950 con 0x55b4679dfb20 2024-09-10T09:42:24.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:24.727+0000 7f49c8a00640 1 --2- 172.21.15.5:0/2331028542 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55b4679dfb20 0x55b4679e1f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:42:24.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:24.728+0000 7f49c8a00640 1 --2- 172.21.15.5:0/2331028542 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55b4679dfb20 0x55b4679e1f70 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f2b0a7878cce8ebe server_cookie=c01462747449270b in_seq=0 out_seq=0 2024-09-10T09:42:24.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:24.728+0000 7f49aaa00640 10 client.16134 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T09:42:24.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:24.729+0000 7f49aaa00640 1 -- 172.21.15.5:0/2331028542 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b467938950 con 0x55b4679dfb20 2024-09-10T09:42:24.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:24.729+0000 7f49aaa00640 10 client.16134 handle_command_reply: tid=0 2024-09-10T09:42:24.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:24.729+0000 7f49ce7d3740 1 client.16134 shutdown 2024-09-10T09:42:24.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:24.729+0000 7f49ce7d3740 1 -- 172.21.15.5:0/2331028542 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b4679c4b90 msgr2=0x55b4679ccc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:42:24.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:24.729+0000 7f49ce7d3740 1 --2- 172.21.15.5:0/2331028542 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b4679c4b90 0x55b4679ccc20 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f49b8005fd0 tx=0x7f49b80cc4a0 comp rx=0 tx=0).stop 2024-09-10T09:42:24.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:24.729+0000 7f49ce7d3740 1 -- 172.21.15.5:0/2331028542 shutdown_connections 2024-09-10T09:42:24.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:24.729+0000 7f49ce7d3740 1 --2- 172.21.15.5:0/2331028542 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55b4679dfb20 0x55b4679e1f70 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:24.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:24.729+0000 7f49ce7d3740 1 --2- 172.21.15.5:0/2331028542 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b4679cfd70 0x55b4679cd6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:24.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:24.729+0000 7f49ce7d3740 1 --2- 172.21.15.5:0/2331028542 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b4679c54b0 0x55b4679cd160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:24.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:24.729+0000 7f49ce7d3740 1 --2- 172.21.15.5:0/2331028542 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b4679c4b90 0x55b4679ccc20 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:24.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:24.729+0000 7f49ce7d3740 1 -- 172.21.15.5:0/2331028542 >> 172.21.15.5:0/2331028542 conn(0x55b4679c1120 msgr2=0x55b4679c6e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:42:24.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:24.729+0000 7f49aaa00640 0 client.16134 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T09:42:24.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:24.729+0000 7f49ce7d3740 1 -- 172.21.15.5:0/2331028542 shutdown_connections 2024-09-10T09:42:24.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:24.729+0000 7f49ce7d3740 1 -- 172.21.15.5:0/2331028542 wait complete. 2024-09-10T09:42:24.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:24.729+0000 7f49ce7d3740 20 client.16134 trim_cache size 0 max 16384 2024-09-10T09:42:24.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:24.747+0000 7f49ce7d3740 1 Processor -- start 2024-09-10T09:42:24.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:24.747+0000 7f49ce7d3740 1 -- start start 2024-09-10T09:42:24.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:24.748+0000 7f49ce7d3740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b4679c4b90 0x55b4679e7840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:42:24.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:24.748+0000 7f49ce7d3740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b4679c54b0 0x55b4679e7d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:42:24.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:24.748+0000 7f49ce7d3740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b4679cfd70 0x55b4679c9e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:42:24.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:24.748+0000 7f49ce7d3740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f49c41b1030 con 0x55b4679cfd70 2024-09-10T09:42:24.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:24.748+0000 7f49ce7d3740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f49c41b2ba0 con 0x55b4679c54b0 2024-09-10T09:42:24.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:24.748+0000 7f49ce7d3740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f49c41b1700 con 0x55b4679c4b90 2024-09-10T09:42:24.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:24.748+0000 7f49c9e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b4679cfd70 0x55b4679c9e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:42:24.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:24.748+0000 7f49c9e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b4679cfd70 0x55b4679c9e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35588/0 (socket says 172.21.15.5:35588) 2024-09-10T09:42:24.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:24.748+0000 7f49c9e00640 1 -- 172.21.15.5:0/4177401231 learned_addr learned my addr 172.21.15.5:0/4177401231 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:42:24.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:24.748+0000 7f49c8a00640 1 --2- 172.21.15.5:0/4177401231 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b4679c54b0 0x55b4679e7d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:42:24.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:24.748+0000 7f49c9400640 1 --2- 172.21.15.5:0/4177401231 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b4679c4b90 0x55b4679e7840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:42:24.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:24.749+0000 7f49c9e00640 1 -- 172.21.15.5:0/4177401231 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b4679c4b90 msgr2=0x55b4679e7840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:42:24.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:24.749+0000 7f49c9e00640 1 --2- 172.21.15.5:0/4177401231 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b4679c4b90 0x55b4679e7840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:24.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:24.749+0000 7f49c9e00640 1 -- 172.21.15.5:0/4177401231 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b4679c54b0 msgr2=0x55b4679e7d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:42:24.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:24.749+0000 7f49c9e00640 1 --2- 172.21.15.5:0/4177401231 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b4679c54b0 0x55b4679e7d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:24.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:24.749+0000 7f49c9e00640 1 -- 172.21.15.5:0/4177401231 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b4679c52e0 con 0x55b4679cfd70 2024-09-10T09:42:24.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:24.749+0000 7f49c9400640 1 --2- 172.21.15.5:0/4177401231 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b4679c4b90 0x55b4679e7840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:42:24.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:24.749+0000 7f49c8a00640 1 --2- 172.21.15.5:0/4177401231 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b4679c54b0 0x55b4679e7d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:42:24.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:24.749+0000 7f49c9e00640 1 --2- 172.21.15.5:0/4177401231 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b4679cfd70 0x55b4679c9e80 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f49bc010940 tx=0x7f49bc019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=499f829fe7f4f45e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:42:24.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:24.749+0000 7f49b5c00640 1 -- 172.21.15.5:0/4177401231 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49bc01e9a0 con 0x55b4679cfd70 2024-09-10T09:42:24.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:24.749+0000 7f49b5c00640 1 -- 172.21.15.5:0/4177401231 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f49bc01eb40 con 0x55b4679cfd70 2024-09-10T09:42:24.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:24.749+0000 7f49b5c00640 1 -- 172.21.15.5:0/4177401231 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49bc01c650 con 0x55b4679cfd70 2024-09-10T09:42:24.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:24.750+0000 7f49ce7d3740 1 -- 172.21.15.5:0/4177401231 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b4679cfd70 msgr2=0x55b4679c9e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:42:24.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:24.750+0000 7f49ce7d3740 1 --2- 172.21.15.5:0/4177401231 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b4679cfd70 0x55b4679c9e80 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f49bc010940 tx=0x7f49bc019000 comp rx=0 tx=0).stop 2024-09-10T09:42:24.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:24.750+0000 7f49ce7d3740 1 -- 172.21.15.5:0/4177401231 shutdown_connections 2024-09-10T09:42:24.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:24.750+0000 7f49ce7d3740 1 --2- 172.21.15.5:0/4177401231 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b4679cfd70 0x55b4679c9e80 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:24.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:24.750+0000 7f49ce7d3740 1 --2- 172.21.15.5:0/4177401231 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b4679c54b0 0x55b4679e7d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:24.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:24.750+0000 7f49ce7d3740 1 --2- 172.21.15.5:0/4177401231 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b4679c4b90 0x55b4679e7840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:24.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:24.750+0000 7f49ce7d3740 1 -- 172.21.15.5:0/4177401231 >> 172.21.15.5:0/4177401231 conn(0x55b4679c1120 msgr2=0x55b4679cb4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:42:24.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:24.750+0000 7f49ce7d3740 1 -- 172.21.15.5:0/4177401231 shutdown_connections 2024-09-10T09:42:24.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:24.750+0000 7f49ce7d3740 1 -- 172.21.15.5:0/4177401231 wait complete. 2024-09-10T09:42:24.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:24.751+0000 7f49ce7d3740 1 -- start start 2024-09-10T09:42:24.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:24.751+0000 7f49ce7d3740 1 Processor -- start 2024-09-10T09:42:24.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:24.751+0000 7f49ce7d3740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b4679c4b90 0x55b4679d3af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:42:24.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:24.751+0000 7f49ce7d3740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b4679c54b0 0x55b4679d4030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:42:24.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:24.751+0000 7f49ce7d3740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b4679cfd70 0x55b4679d4570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:42:24.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:24.751+0000 7f49ce7d3740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f49c41b24a0 con 0x55b4679c4b90 2024-09-10T09:42:24.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:24.751+0000 7f49ce7d3740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f49c41b1dd0 con 0x55b4679c54b0 2024-09-10T09:42:24.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:24.751+0000 7f49ce7d3740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f49c41af010 con 0x55b4679cfd70 2024-09-10T09:42:24.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:24.751+0000 7f49ce7d3740 10 client.0 _ll_register_callbacks cb 0x7f49b80d25e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:42:24.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:24.752+0000 7f49c9400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b4679c4b90 0x55b4679d3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:42:24.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:24.752+0000 7f49c9400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b4679c4b90 0x55b4679d3af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35596/0 (socket says 172.21.15.5:35596) 2024-09-10T09:42:24.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:24.752+0000 7f49c9400640 1 -- 172.21.15.5:0/3862133756 learned_addr learned my addr 172.21.15.5:0/3862133756 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:42:24.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:24.752+0000 7f49c8a00640 1 --2- 172.21.15.5:0/3862133756 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b4679c54b0 0x55b4679d4030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:42:24.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:24.752+0000 7f49c9e00640 1 --2- 172.21.15.5:0/3862133756 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b4679cfd70 0x55b4679d4570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:42:24.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:24.752+0000 7f49c9400640 1 -- 172.21.15.5:0/3862133756 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b4679cfd70 msgr2=0x55b4679d4570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:42:24.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:24.752+0000 7f49c9400640 1 --2- 172.21.15.5:0/3862133756 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b4679cfd70 0x55b4679d4570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:24.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:24.752+0000 7f49c9400640 1 -- 172.21.15.5:0/3862133756 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b4679c54b0 msgr2=0x55b4679d4030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:42:24.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:24.752+0000 7f49c9400640 1 --2- 172.21.15.5:0/3862133756 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b4679c54b0 0x55b4679d4030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:24.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:24.752+0000 7f49c9400640 1 -- 172.21.15.5:0/3862133756 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b467a49810 con 0x55b4679c4b90 2024-09-10T09:42:24.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:24.752+0000 7f49c8a00640 1 --2- 172.21.15.5:0/3862133756 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b4679c54b0 0x55b4679d4030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:42:24.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:24.752+0000 7f49c9400640 1 --2- 172.21.15.5:0/3862133756 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b4679c4b90 0x55b4679d3af0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f49b809b250 tx=0x7f49b80975d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=658d5ee0326bbd29 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:42:24.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:24.752+0000 7f49aaa00640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-10T09:42:24.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:24.753+0000 7f49aaa00640 1 -- 172.21.15.5:0/3862133756 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49b809ae00 con 0x55b4679c4b90 2024-09-10T09:42:24.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:24.753+0000 7f49aaa00640 1 -- 172.21.15.5:0/3862133756 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f49b809f160 con 0x55b4679c4b90 2024-09-10T09:42:24.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:24.753+0000 7f49ce7d3740 1 -- 172.21.15.5:0/3862133756 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f49bc01eb40 con 0x55b4679c4b90 2024-09-10T09:42:24.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:24.753+0000 7f49aaa00640 1 -- 172.21.15.5:0/3862133756 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f49b80a3f30 con 0x55b4679c4b90 2024-09-10T09:42:24.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:24.753+0000 7f49ce7d3740 10 client.16140 fetch_fsmap learned FSMap version 33 2024-09-10T09:42:24.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:24.753+0000 7f49ce7d3740 1 -- 172.21.15.5:0/3862133756 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55b467a4aae0 con 0x55b4679c4b90 2024-09-10T09:42:24.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:24.753+0000 7f49aaa00640 1 -- 172.21.15.5:0/3862133756 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f49bc01eb40 con 0x55b4679c4b90 2024-09-10T09:42:24.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:24.753+0000 7f49aaa00640 1 -- 172.21.15.5:0/3862133756 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 33) ==== 8374+0+0 (secure 0 0 0) 0x7f49b809a710 con 0x55b4679c4b90 2024-09-10T09:42:24.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:24.753+0000 7f49ce7d3740 10 client.16140 fetch_fsmap finished waiting for FSMap version 33 2024-09-10T09:42:24.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:24.753+0000 7f49ce7d3740 10 client.16140 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T09:42:24.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:24.753+0000 7f49ce7d3740 20 client.16140 populate_metadata read hostname 'smithi005' 2024-09-10T09:42:24.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:24.754+0000 7f49ce7d3740 1 --2- 172.21.15.5:0/3862133756 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55b467a4b0a0 0x55b467a4d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:42:24.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:24.754+0000 7f49ce7d3740 4 client.16140 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T09:42:24.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:24.754+0000 7f49ce7d3740 1 -- 172.21.15.5:0/3862133756 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55b467932e60 con 0x55b467a4b0a0 2024-09-10T09:42:24.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:24.754+0000 7f49c8a00640 1 --2- 172.21.15.5:0/3862133756 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55b467a4b0a0 0x55b467a4d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:42:24.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:24.755+0000 7f49c8a00640 1 --2- 172.21.15.5:0/3862133756 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55b467a4b0a0 0x55b467a4d4c0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3c8fab8a064ff54f server_cookie=d912aeebceecc35f in_seq=0 out_seq=0 2024-09-10T09:42:24.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:24.755+0000 7f49aaa00640 10 client.16140 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T09:42:24.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:24.755+0000 7f49aaa00640 1 -- 172.21.15.5:0/3862133756 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55b467932e60 con 0x55b467a4b0a0 2024-09-10T09:42:24.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:24.755+0000 7f49aaa00640 10 client.16140 handle_command_reply: tid=0 2024-09-10T09:42:24.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:24.755+0000 7f49ce7d3740 1 client.16140 shutdown 2024-09-10T09:42:24.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:24.755+0000 7f49ce7d3740 1 -- 172.21.15.5:0/3862133756 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b4679c4b90 msgr2=0x55b4679d3af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:42:24.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:24.755+0000 7f49ce7d3740 1 --2- 172.21.15.5:0/3862133756 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b4679c4b90 0x55b4679d3af0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f49b809b250 tx=0x7f49b80975d0 comp rx=0 tx=0).stop 2024-09-10T09:42:24.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:24.755+0000 7f49ce7d3740 1 -- 172.21.15.5:0/3862133756 shutdown_connections 2024-09-10T09:42:24.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:24.755+0000 7f49ce7d3740 1 --2- 172.21.15.5:0/3862133756 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55b467a4b0a0 0x55b467a4d4c0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:24.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:24.755+0000 7f49ce7d3740 1 --2- 172.21.15.5:0/3862133756 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b4679cfd70 0x55b4679d4570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:24.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:24.755+0000 7f49ce7d3740 1 --2- 172.21.15.5:0/3862133756 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b4679c54b0 0x55b4679d4030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:24.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:24.755+0000 7f49aaa00640 0 client.16140 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T09:42:24.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:24.755+0000 7f49ce7d3740 1 --2- 172.21.15.5:0/3862133756 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b4679c4b90 0x55b4679d3af0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:24.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:24.755+0000 7f49ce7d3740 1 -- 172.21.15.5:0/3862133756 >> 172.21.15.5:0/3862133756 conn(0x55b4679c1120 msgr2=0x55b4679c65e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:42:24.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:24.755+0000 7f49ce7d3740 1 -- 172.21.15.5:0/3862133756 shutdown_connections 2024-09-10T09:42:24.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:24.755+0000 7f49ce7d3740 1 -- 172.21.15.5:0/3862133756 wait complete. 2024-09-10T09:42:24.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:24.756+0000 7f49ce7d3740 20 client.16140 trim_cache size 0 max 16384 2024-09-10T09:42:24.761 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-10T09:42:24.761 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "no active scrubs running", 2024-09-10T09:42:24.761 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": {} 2024-09-10T09:42:24.762 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-10T09:42:24.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:24.756+0000 7f49cbe00640 1 -- 172.21.15.5:0/3277515101 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f49ac0a2c00 msgr2=0x7f49ac0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:42:24.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:24.756+0000 7f49cbe00640 1 --2- 172.21.15.5:0/3277515101 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f49ac0a2c00 0x7f49ac0a50c0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f49bc005fd0 tx=0x7f49bc009500 comp rx=0 tx=0).stop 2024-09-10T09:42:24.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:24.756+0000 7f49cbe00640 1 -- 172.21.15.5:0/3277515101 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f49c4101de0 msgr2=0x7f49c419a630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:42:24.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:24.756+0000 7f49cbe00640 1 --2- 172.21.15.5:0/3277515101 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f49c4101de0 0x7f49c419a630 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f49b800d950 tx=0x7f49b800de10 comp rx=0 tx=0).stop 2024-09-10T09:42:24.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:24.756+0000 7f49cbe00640 1 -- 172.21.15.5:0/3277515101 shutdown_connections 2024-09-10T09:42:24.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:24.756+0000 7f49cbe00640 1 --2- 172.21.15.5:0/3277515101 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f49ac0a2c00 0x7f49ac0a50c0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:24.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:24.756+0000 7f49cbe00640 1 --2- 172.21.15.5:0/3277515101 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f49c4106f10 0x7f49c419ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:24.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:24.756+0000 7f49cbe00640 1 --2- 172.21.15.5:0/3277515101 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f49c4102790 0x7f49c419ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:24.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:24.756+0000 7f49cbe00640 1 --2- 172.21.15.5:0/3277515101 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f49c4101de0 0x7f49c419a630 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:24.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:24.756+0000 7f49cbe00640 1 -- 172.21.15.5:0/3277515101 >> 172.21.15.5:0/3277515101 conn(0x7f49c40fd8c0 msgr2=0x7f49c40fec40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:42:24.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:24.757+0000 7f49cbe00640 1 -- 172.21.15.5:0/3277515101 shutdown_connections 2024-09-10T09:42:24.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:24.757+0000 7f49cbe00640 1 -- 172.21.15.5:0/3277515101 wait complete. 2024-09-10T09:42:25.015 INFO:tasks.workunit.client.0.smithi005.stdout: 1 191939 0.00 MB/sec execute 6 sec latency 9920.308 ms 2024-09-10T09:42:25.720 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-10T09:42:25.720 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T09:42:25.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:42:25 smithi159 ceph-mon[35513]: pgmap v515: 129 pgs: 129 active+clean; 99 MiB data, 854 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.9 MiB/s wr, 132 op/s 2024-09-10T09:42:25.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:42:25 smithi102 ceph-mon[35442]: pgmap v515: 129 pgs: 129 active+clean; 99 MiB data, 854 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.9 MiB/s wr, 132 op/s 2024-09-10T09:42:26.017 INFO:tasks.workunit.client.0.smithi005.stdout: 1 191939 0.00 MB/sec execute 7 sec latency 10920.389 ms 2024-09-10T09:42:26.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:42:25 smithi005 ceph-mon[30350]: pgmap v515: 129 pgs: 129 active+clean; 99 MiB data, 854 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.9 MiB/s wr, 132 op/s 2024-09-10T09:42:26.101 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:42:26.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:26.887+0000 7fecc5400640 1 -- 172.21.15.5:0/1058620634 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fecc0109120 msgr2=0x7fecc0110ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:42:26.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:26.887+0000 7fecc5400640 1 --2- 172.21.15.5:0/1058620634 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fecc0109120 0x7fecc0110ca0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fecb8009ca0 tx=0x7fecb802a8c0 comp rx=0 tx=0).stop 2024-09-10T09:42:26.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:26.887+0000 7fecc5400640 1 -- 172.21.15.5:0/1058620634 shutdown_connections 2024-09-10T09:42:26.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:26.887+0000 7fecc5400640 1 --2- 172.21.15.5:0/1058620634 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fecc0109120 0x7fecc0110ca0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:26.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:26.887+0000 7fecc5400640 1 --2- 172.21.15.5:0/1058620634 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fecc01049a0 0x7fecc01089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:26.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:26.887+0000 7fecc5400640 1 --2- 172.21.15.5:0/1058620634 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fecc0103ff0 0x7fecc01043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:26.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:26.887+0000 7fecc5400640 1 -- 172.21.15.5:0/1058620634 >> 172.21.15.5:0/1058620634 conn(0x7fecc00ffa90 msgr2=0x7fecc0101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:42:26.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:26.887+0000 7fecc5400640 1 -- 172.21.15.5:0/1058620634 shutdown_connections 2024-09-10T09:42:26.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:26.887+0000 7fecc5400640 1 -- 172.21.15.5:0/1058620634 wait complete. 2024-09-10T09:42:26.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:26.888+0000 7fecc5400640 1 Processor -- start 2024-09-10T09:42:26.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:26.889+0000 7fecc5400640 1 -- start start 2024-09-10T09:42:26.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:26.889+0000 7fecc5400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fecc0103ff0 0x7fecc019a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:42:26.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:26.889+0000 7fecc5400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fecc01049a0 0x7fecc019aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:42:26.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:26.889+0000 7fecc5400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fecc0109120 0x7fecc019ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:42:26.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:26.889+0000 7fecc5400640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fecc0114150 con 0x7fecc0103ff0 2024-09-10T09:42:26.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:26.889+0000 7fecc5400640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fecc0113fd0 con 0x7fecc0109120 2024-09-10T09:42:26.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:26.889+0000 7fecc5400640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fecc01142d0 con 0x7fecc01049a0 2024-09-10T09:42:26.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:26.889+0000 7fecbe800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fecc0103ff0 0x7fecc019a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:42:26.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:26.889+0000 7fecbe800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fecc0103ff0 0x7fecc019a5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35618/0 (socket says 172.21.15.5:35618) 2024-09-10T09:42:26.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:26.889+0000 7fecbe800640 1 -- 172.21.15.5:0/3107614305 learned_addr learned my addr 172.21.15.5:0/3107614305 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:42:26.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:26.889+0000 7fecbf200640 1 --2- 172.21.15.5:0/3107614305 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fecc0109120 0x7fecc019ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:42:26.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:26.889+0000 7fecbde00640 1 --2- 172.21.15.5:0/3107614305 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fecc01049a0 0x7fecc019aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:42:26.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:26.890+0000 7fecbe800640 1 -- 172.21.15.5:0/3107614305 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fecc01049a0 msgr2=0x7fecc019aaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:42:26.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:26.890+0000 7fecbe800640 1 --2- 172.21.15.5:0/3107614305 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fecc01049a0 0x7fecc019aaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:26.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:26.890+0000 7fecbe800640 1 -- 172.21.15.5:0/3107614305 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fecc0109120 msgr2=0x7fecc019ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:42:26.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:26.890+0000 7fecbe800640 1 --2- 172.21.15.5:0/3107614305 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fecc0109120 0x7fecc019ee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:26.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:26.890+0000 7fecbe800640 1 -- 172.21.15.5:0/3107614305 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fecc019f560 con 0x7fecc0103ff0 2024-09-10T09:42:26.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:26.890+0000 7fecbf200640 1 --2- 172.21.15.5:0/3107614305 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fecc0109120 0x7fecc019ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:42:26.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:26.890+0000 7fecbe800640 1 --2- 172.21.15.5:0/3107614305 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fecc0103ff0 0x7fecc019a5b0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fecb000e9f0 tx=0x7fecb000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:42:26.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:26.890+0000 7fecb7400640 1 -- 172.21.15.5:0/3107614305 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecb000cd80 con 0x7fecc0103ff0 2024-09-10T09:42:26.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:26.890+0000 7fecb7400640 1 -- 172.21.15.5:0/3107614305 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fecb0005500 con 0x7fecc0103ff0 2024-09-10T09:42:26.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:26.890+0000 7fecc5400640 1 -- 172.21.15.5:0/3107614305 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fecc019f850 con 0x7fecc0103ff0 2024-09-10T09:42:26.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:26.890+0000 7fecb7400640 1 -- 172.21.15.5:0/3107614305 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecb0010690 con 0x7fecc0103ff0 2024-09-10T09:42:26.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:26.890+0000 7fecc5400640 1 -- 172.21.15.5:0/3107614305 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fecc01a7130 con 0x7fecc0103ff0 2024-09-10T09:42:26.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:26.891+0000 7fecc5400640 1 -- 172.21.15.5:0/3107614305 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fecc01043d0 con 0x7fecc0103ff0 2024-09-10T09:42:26.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:26.893+0000 7fecb7400640 1 -- 172.21.15.5:0/3107614305 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7fecb00046c0 con 0x7fecc0103ff0 2024-09-10T09:42:26.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:26.895+0000 7fecb7400640 1 --2- 172.21.15.5:0/3107614305 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7feca40a2b80 0x7feca40a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:42:26.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:26.895+0000 7fecb7400640 1 -- 172.21.15.5:0/3107614305 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fecb00cf9f0 con 0x7fecc0103ff0 2024-09-10T09:42:26.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:26.895+0000 7fecbde00640 1 --2- 172.21.15.5:0/3107614305 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7feca40a2b80 0x7feca40a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:42:26.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:26.896+0000 7fecbde00640 1 --2- 172.21.15.5:0/3107614305 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7feca40a2b80 0x7feca40a5040 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fecc010aa90 tx=0x7fecac005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:42:26.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:26.899+0000 7fecb7400640 1 -- 172.21.15.5:0/3107614305 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fecb009c500 con 0x7fecc0103ff0 2024-09-10T09:42:27.015 INFO:tasks.workunit.client.0.smithi005.stdout: 1 191939 0.00 MB/sec execute 8 sec latency 11920.454 ms 2024-09-10T09:42:27.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:27.180+0000 7fecc5400640 1 -- 172.21.15.5:0/3107614305 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fecc019b910 con 0x7fecc0103ff0 2024-09-10T09:42:27.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:27.181+0000 7fecb7400640 1 -- 172.21.15.5:0/3107614305 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) ==== 94+0+11970 (secure 0 0 0) 0x7fecb009be00 con 0x7fecc0103ff0 2024-09-10T09:42:27.185 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-10T09:42:27.185 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":33,"btime":"2024-09-10T09:42:17:528412+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14454,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/2970594311","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2970594311},{"type":"v1","addr":"172.21.15.5:6835","nonce":2970594311}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14514,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6839/3616502174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":3616502174},{"type":"v1","addr":"172.21.15.5:6839","nonce":3616502174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":33,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T09:35:29.626287+0000","modified":"2024-09-10T09:42:16.961192+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24542,"mds_1":24535,"mds_2":24511,"mds_3":24506,"mds_4":24443},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14520":{"gid":14520,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/2073599842","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":2073599842},{"type":"v1","addr":"172.21.15.5:6841","nonce":2073599842}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"k","rank":4,"incarnation":26,"state":"up:active","state_seq":24,"addr":"172.21.15.102:6833/2457007427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":2457007427},{"type":"v1","addr":"172.21.15.102:6833","nonce":2457007427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/356445159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":356445159},{"type":"v1","addr":"172.21.15.159:6833","nonce":356445159}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6837/2894314507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":2894314507},{"type":"v1","addr":"172.21.15.5:6837","nonce":2894314507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.102:6835/1647549111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":1647549111},{"type":"v1","addr":"172.21.15.102:6835","nonce":1647549111}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6835/952657242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":952657242},{"type":"v1","addr":"172.21.15.159:6835","nonce":952657242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":19,"addr":"172.21.15.102:6837/2689776764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":2689776764},{"type":"v1","addr":"172.21.15.102:6837","nonce":2689776764}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.159:6837/620627639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":620627639},{"type":"v1","addr":"172.21.15.159:6837","nonce":620627639}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24535":{"gid":24535,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":18,"addr":"172.21.15.159:6839/2073739599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":2073739599},{"type":"v1","addr":"172.21.15.159:6839","nonce":2073739599}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24542":{"gid":24542,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.102:6839/335409749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6838","nonce":335409749},{"type":"v1","addr":"172.21.15.102:6839","nonce":335409749}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24542,"qdb_cluster":[24443,24506,24511,24535,24542]},"id":1}]} 2024-09-10T09:42:27.186 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 33 2024-09-10T09:42:27.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:27.186+0000 7fecc5400640 1 -- 172.21.15.5:0/3107614305 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7feca40a2b80 msgr2=0x7feca40a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:42:27.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:27.186+0000 7fecc5400640 1 --2- 172.21.15.5:0/3107614305 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7feca40a2b80 0x7feca40a5040 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fecc010aa90 tx=0x7fecac005f90 comp rx=0 tx=0).stop 2024-09-10T09:42:27.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:27.186+0000 7fecc5400640 1 -- 172.21.15.5:0/3107614305 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fecc0103ff0 msgr2=0x7fecc019a5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:42:27.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:27.186+0000 7fecc5400640 1 --2- 172.21.15.5:0/3107614305 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fecc0103ff0 0x7fecc019a5b0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fecb000e9f0 tx=0x7fecb000edf0 comp rx=0 tx=0).stop 2024-09-10T09:42:27.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:27.187+0000 7fecc5400640 1 -- 172.21.15.5:0/3107614305 shutdown_connections 2024-09-10T09:42:27.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:27.187+0000 7fecc5400640 1 --2- 172.21.15.5:0/3107614305 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7feca40a2b80 0x7feca40a5040 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:27.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:27.187+0000 7fecc5400640 1 --2- 172.21.15.5:0/3107614305 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fecc0109120 0x7fecc019ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:27.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:27.187+0000 7fecc5400640 1 --2- 172.21.15.5:0/3107614305 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fecc01049a0 0x7fecc019aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:27.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:27.187+0000 7fecc5400640 1 --2- 172.21.15.5:0/3107614305 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fecc0103ff0 0x7fecc019a5b0 secure :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fecb000e9f0 tx=0x7fecb000edf0 comp rx=0 tx=0).stop 2024-09-10T09:42:27.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:27.187+0000 7fecc5400640 1 -- 172.21.15.5:0/3107614305 >> 172.21.15.5:0/3107614305 conn(0x7fecc00ffa90 msgr2=0x7fecc0101ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:42:27.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:27.189+0000 7fecc5400640 1 -- 172.21.15.5:0/3107614305 shutdown_connections 2024-09-10T09:42:27.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:27.189+0000 7fecc5400640 1 -- 172.21.15.5:0/3107614305 wait complete. 2024-09-10T09:42:27.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:42:27 smithi005 ceph-mon[30350]: pgmap v516: 129 pgs: 129 active+clean; 96 MiB data, 853 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.6 MiB/s wr, 135 op/s 2024-09-10T09:42:27.629 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:42:27 smithi005 ceph-mon[30350]: from='client.? 172.21.15.5:0/3107614305' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:42:27.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:42:27 smithi159 ceph-mon[35513]: pgmap v516: 129 pgs: 129 active+clean; 96 MiB data, 853 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.6 MiB/s wr, 135 op/s 2024-09-10T09:42:27.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:42:27 smithi159 ceph-mon[35513]: from='client.? 172.21.15.5:0/3107614305' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:42:27.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:42:27 smithi102 ceph-mon[35442]: pgmap v516: 129 pgs: 129 active+clean; 96 MiB data, 853 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.6 MiB/s wr, 135 op/s 2024-09-10T09:42:27.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:42:27 smithi102 ceph-mon[35442]: from='client.? 172.21.15.5:0/3107614305' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:42:28.015 INFO:tasks.workunit.client.0.smithi005.stdout: 1 191939 0.00 MB/sec execute 9 sec latency 12920.502 ms 2024-09-10T09:42:28.240 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-10T09:42:28.619 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:42:29.015 INFO:tasks.workunit.client.0.smithi005.stdout: 1 191939 0.00 MB/sec execute 10 sec latency 13920.648 ms 2024-09-10T09:42:29.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:29.201+0000 7f2fc6e00640 1 -- 172.21.15.5:0/1999081407 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2fc0102790 msgr2=0x7f2fc01067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:42:29.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:29.201+0000 7f2fc6e00640 1 --2- 172.21.15.5:0/1999081407 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2fc0102790 0x7f2fc01067e0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f2fa8009900 tx=0x7f2fa802a410 comp rx=0 tx=0).stop 2024-09-10T09:42:29.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:29.201+0000 7f2fc6e00640 1 -- 172.21.15.5:0/1999081407 shutdown_connections 2024-09-10T09:42:29.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:29.201+0000 7f2fc6e00640 1 --2- 172.21.15.5:0/1999081407 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2fc0106f10 0x7f2fc010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:29.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:29.201+0000 7f2fc6e00640 1 --2- 172.21.15.5:0/1999081407 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2fc0102790 0x7f2fc01067e0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:29.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:29.201+0000 7f2fc6e00640 1 --2- 172.21.15.5:0/1999081407 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2fc0101de0 0x7f2fc01021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:29.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:29.201+0000 7f2fc6e00640 1 -- 172.21.15.5:0/1999081407 >> 172.21.15.5:0/1999081407 conn(0x7f2fc00fd8c0 msgr2=0x7f2fc00ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:42:29.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:29.201+0000 7f2fc6e00640 1 -- 172.21.15.5:0/1999081407 shutdown_connections 2024-09-10T09:42:29.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:29.201+0000 7f2fc6e00640 1 -- 172.21.15.5:0/1999081407 wait complete. 2024-09-10T09:42:29.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:29.202+0000 7f2fc6e00640 1 Processor -- start 2024-09-10T09:42:29.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:29.202+0000 7f2fc6e00640 1 -- start start 2024-09-10T09:42:29.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:29.202+0000 7f2fc6e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2fc0101de0 0x7f2fc019a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:42:29.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:29.202+0000 7f2fc6e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2fc0102790 0x7f2fc019ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:42:29.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:29.202+0000 7f2fc6e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2fc0106f10 0x7f2fc019ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:42:29.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:29.202+0000 7f2fc6e00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f2fc0111fc0 con 0x7f2fc0102790 2024-09-10T09:42:29.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:29.202+0000 7f2fc6e00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f2fc0111e40 con 0x7f2fc0106f10 2024-09-10T09:42:29.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:29.202+0000 7f2fc6e00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f2fc0112140 con 0x7f2fc0101de0 2024-09-10T09:42:29.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:29.203+0000 7f2fb7e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2fc0102790 0x7f2fc019ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:42:29.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:29.203+0000 7f2fb7e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2fc0102790 0x7f2fc019ab90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35636/0 (socket says 172.21.15.5:35636) 2024-09-10T09:42:29.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:29.203+0000 7f2fb7e00640 1 -- 172.21.15.5:0/684353248 learned_addr learned my addr 172.21.15.5:0/684353248 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:42:29.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:29.203+0000 7f2fbfe00640 1 --2- 172.21.15.5:0/684353248 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2fc0101de0 0x7f2fc019a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:42:29.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:29.203+0000 7f2fc4e00640 1 --2- 172.21.15.5:0/684353248 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2fc0106f10 0x7f2fc019ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:42:29.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:29.203+0000 7f2fb7e00640 1 -- 172.21.15.5:0/684353248 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2fc0101de0 msgr2=0x7f2fc019a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:42:29.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:29.203+0000 7f2fb7e00640 1 --2- 172.21.15.5:0/684353248 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2fc0101de0 0x7f2fc019a650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:29.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:29.203+0000 7f2fb7e00640 1 -- 172.21.15.5:0/684353248 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2fc0106f10 msgr2=0x7f2fc019ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:42:29.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:29.203+0000 7f2fb7e00640 1 --2- 172.21.15.5:0/684353248 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2fc0106f10 0x7f2fc019ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:29.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:29.203+0000 7f2fb7e00640 1 -- 172.21.15.5:0/684353248 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2fc019f600 con 0x7f2fc0102790 2024-09-10T09:42:29.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:29.203+0000 7f2fb7e00640 1 --2- 172.21.15.5:0/684353248 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2fc0102790 0x7f2fc019ab90 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f2fa8004770 tx=0x7f2fa8003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:42:29.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:29.204+0000 7f2fbd600640 1 -- 172.21.15.5:0/684353248 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fa803f070 con 0x7f2fc0102790 2024-09-10T09:42:29.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:29.204+0000 7f2fbd600640 1 -- 172.21.15.5:0/684353248 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2fa8031ca0 con 0x7f2fc0102790 2024-09-10T09:42:29.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:29.204+0000 7f2fc6e00640 1 -- 172.21.15.5:0/684353248 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2fc019f890 con 0x7f2fc0102790 2024-09-10T09:42:29.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:29.204+0000 7f2fc6e00640 1 -- 172.21.15.5:0/684353248 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2fc01a71d0 con 0x7f2fc0102790 2024-09-10T09:42:29.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:29.204+0000 7f2fbd600640 1 -- 172.21.15.5:0/684353248 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fa8035030 con 0x7f2fc0102790 2024-09-10T09:42:29.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:29.207+0000 7f2fc9813740 1 Processor -- start 2024-09-10T09:42:29.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:29.207+0000 7f2fbd600640 1 -- 172.21.15.5:0/684353248 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f2fa8042070 con 0x7f2fc0102790 2024-09-10T09:42:29.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:29.207+0000 7f2fc9813740 1 -- start start 2024-09-10T09:42:29.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:29.208+0000 7f2fc9813740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560542552c70 0x560542553050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:42:29.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:29.208+0000 7f2fc9813740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560542553590 0x56054255d910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:42:29.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:29.208+0000 7f2fc9813740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56054255de50 0x560542560240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:42:29.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:29.208+0000 7f2fc9813740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5605424b93e0 con 0x560542552c70 2024-09-10T09:42:29.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:29.208+0000 7f2fc9813740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x560542460d80 con 0x56054255de50 2024-09-10T09:42:29.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:29.208+0000 7f2fc9813740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5605424b53a0 con 0x560542553590 2024-09-10T09:42:29.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:29.208+0000 7f2fc4e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560542552c70 0x560542553050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:42:29.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:29.208+0000 7f2fc4e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560542552c70 0x560542553050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35640/0 (socket says 172.21.15.5:35640) 2024-09-10T09:42:29.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:29.208+0000 7f2fc4e00640 1 -- 172.21.15.5:0/3271250315 learned_addr learned my addr 172.21.15.5:0/3271250315 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:42:29.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:29.208+0000 7f2fbfe00640 1 --2- 172.21.15.5:0/3271250315 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560542553590 0x56054255d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:42:29.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:29.208+0000 7f2fb7e00640 1 --2- 172.21.15.5:0/3271250315 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56054255de50 0x560542560240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:42:29.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:29.209+0000 7f2fc4e00640 1 -- 172.21.15.5:0/3271250315 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560542553590 msgr2=0x56054255d910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:42:29.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:29.209+0000 7f2fc4e00640 1 --2- 172.21.15.5:0/3271250315 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560542553590 0x56054255d910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:29.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:29.209+0000 7f2fc4e00640 1 -- 172.21.15.5:0/3271250315 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56054255de50 msgr2=0x560542560240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:42:29.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:29.209+0000 7f2fc4e00640 1 --2- 172.21.15.5:0/3271250315 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56054255de50 0x560542560240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:29.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:29.209+0000 7f2fc4e00640 1 -- 172.21.15.5:0/3271250315 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56054247fae0 con 0x560542552c70 2024-09-10T09:42:29.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:29.209+0000 7f2fbfe00640 1 --2- 172.21.15.5:0/3271250315 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560542553590 0x56054255d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:42:29.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:29.209+0000 7f2fc4e00640 1 --2- 172.21.15.5:0/3271250315 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560542552c70 0x560542553050 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f2fb8009f70 tx=0x7f2fb8010800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f6510450a8b08eb server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:42:29.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:29.209+0000 7f2fbd600640 1 --2- 172.21.15.5:0/684353248 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f2fa40a2b80 0x7f2fa40a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:42:29.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:29.209+0000 7f2fb5600640 1 -- 172.21.15.5:0/3271250315 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fb8016070 con 0x560542552c70 2024-09-10T09:42:29.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:29.209+0000 7f2fb5600640 1 -- 172.21.15.5:0/3271250315 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2fb8004d20 con 0x560542552c70 2024-09-10T09:42:29.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:29.209+0000 7f2fb5600640 1 -- 172.21.15.5:0/3271250315 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fb8012850 con 0x560542552c70 2024-09-10T09:42:29.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:29.209+0000 7f2fbd600640 1 -- 172.21.15.5:0/684353248 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f2fa80efb10 con 0x7f2fc0102790 2024-09-10T09:42:29.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:29.209+0000 7f2fbfe00640 1 --2- 172.21.15.5:0/684353248 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f2fa40a2b80 0x7f2fa40a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:42:29.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:29.210+0000 7f2fc9813740 1 -- 172.21.15.5:0/3271250315 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560542552c70 msgr2=0x560542553050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:42:29.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:29.210+0000 7f2fbfe00640 1 --2- 172.21.15.5:0/684353248 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f2fa40a2b80 0x7f2fa40a5040 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x560542550060 tx=0x7f2fb0005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:42:29.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:29.210+0000 7f2fc9813740 1 --2- 172.21.15.5:0/3271250315 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560542552c70 0x560542553050 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f2fb8009f70 tx=0x7f2fb8010800 comp rx=0 tx=0).stop 2024-09-10T09:42:29.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:29.210+0000 7f2fc9813740 1 -- 172.21.15.5:0/3271250315 shutdown_connections 2024-09-10T09:42:29.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:29.210+0000 7f2fc9813740 1 --2- 172.21.15.5:0/3271250315 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56054255de50 0x560542560240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:29.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:29.210+0000 7f2fc9813740 1 --2- 172.21.15.5:0/3271250315 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560542553590 0x56054255d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:29.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:29.210+0000 7f2fc9813740 1 --2- 172.21.15.5:0/3271250315 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560542552c70 0x560542553050 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:29.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:29.210+0000 7f2fc9813740 1 -- 172.21.15.5:0/3271250315 >> 172.21.15.5:0/3271250315 conn(0x56054254f200 msgr2=0x56054254f5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:42:29.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:29.211+0000 7f2fc9813740 1 -- 172.21.15.5:0/3271250315 shutdown_connections 2024-09-10T09:42:29.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:29.211+0000 7f2fc9813740 1 -- 172.21.15.5:0/3271250315 wait complete. 2024-09-10T09:42:29.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:29.211+0000 7f2fc9813740 1 -- start start 2024-09-10T09:42:29.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:29.211+0000 7f2fc9813740 1 Processor -- start 2024-09-10T09:42:29.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:29.212+0000 7f2fc9813740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560542552c70 0x56054255acf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:42:29.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:29.212+0000 7f2fc9813740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560542553590 0x56054255b230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:42:29.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:29.212+0000 7f2fc9813740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56054255de50 0x56054255b770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:42:29.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:29.212+0000 7f2fc9813740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x560542560a20 con 0x560542553590 2024-09-10T09:42:29.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:29.212+0000 7f2fc9813740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x5605424740f0 con 0x56054255de50 2024-09-10T09:42:29.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:29.212+0000 7f2fc9813740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x560542560ba0 con 0x560542552c70 2024-09-10T09:42:29.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:29.212+0000 7f2fc9813740 10 client.0 _ll_register_callbacks cb 0x56054245bbd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:42:29.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:29.212+0000 7f2fbfe00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560542552c70 0x56054255acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:42:29.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:29.212+0000 7f2fb7e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560542553590 0x56054255b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:42:29.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:29.212+0000 7f2fbfe00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560542552c70 0x56054255acf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:33192/0 (socket says 172.21.15.5:33192) 2024-09-10T09:42:29.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:29.212+0000 7f2fbfe00640 1 -- 172.21.15.5:0/4154350922 learned_addr learned my addr 172.21.15.5:0/4154350922 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:42:29.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:29.212+0000 7f2fc4e00640 1 --2- 172.21.15.5:0/4154350922 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56054255de50 0x56054255b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:42:29.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:29.212+0000 7f2fb7e00640 1 -- 172.21.15.5:0/4154350922 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560542552c70 msgr2=0x56054255acf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:42:29.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:29.212+0000 7f2fb7e00640 1 --2- 172.21.15.5:0/4154350922 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560542552c70 0x56054255acf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:29.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:29.212+0000 7f2fb7e00640 1 -- 172.21.15.5:0/4154350922 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56054255de50 msgr2=0x56054255b770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:42:29.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:29.212+0000 7f2fb7e00640 1 --2- 172.21.15.5:0/4154350922 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56054255de50 0x56054255b770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:29.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:29.212+0000 7f2fb7e00640 1 -- 172.21.15.5:0/4154350922 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56054222a3c0 con 0x560542553590 2024-09-10T09:42:29.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:29.213+0000 7f2fb7e00640 1 --2- 172.21.15.5:0/4154350922 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560542553590 0x56054255b230 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x560542550580 tx=0x7f2fa8009560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd7c72c2b4b6690c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:42:29.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:29.213+0000 7f2fae000640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-10T09:42:29.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:29.213+0000 7f2fae000640 1 -- 172.21.15.5:0/4154350922 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fa8049070 con 0x560542553590 2024-09-10T09:42:29.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:29.213+0000 7f2fae000640 1 -- 172.21.15.5:0/4154350922 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f2fa80b8640 con 0x560542553590 2024-09-10T09:42:29.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:29.213+0000 7f2fc9813740 1 -- 172.21.15.5:0/4154350922 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2fb8004d20 con 0x560542553590 2024-09-10T09:42:29.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:29.213+0000 7f2fae000640 1 -- 172.21.15.5:0/4154350922 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f2fa80bbda0 con 0x560542553590 2024-09-10T09:42:29.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:29.213+0000 7f2fc9813740 10 client.16158 fetch_fsmap learned FSMap version 33 2024-09-10T09:42:29.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:29.213+0000 7f2fc9813740 1 -- 172.21.15.5:0/4154350922 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x56054256ca40 con 0x560542553590 2024-09-10T09:42:29.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:29.213+0000 7f2fae000640 1 -- 172.21.15.5:0/4154350922 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2fb8004d20 con 0x560542553590 2024-09-10T09:42:29.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:29.214+0000 7f2fae000640 1 -- 172.21.15.5:0/4154350922 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 33) ==== 8374+0+0 (secure 0 0 0) 0x7f2fa80bc230 con 0x560542553590 2024-09-10T09:42:29.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:29.214+0000 7f2fc9813740 10 client.16158 fetch_fsmap finished waiting for FSMap version 33 2024-09-10T09:42:29.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:29.214+0000 7f2fc9813740 10 client.16158 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-09-10T09:42:29.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:29.214+0000 7f2fc9813740 20 client.16158 populate_metadata read hostname 'smithi005' 2024-09-10T09:42:29.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:29.214+0000 7f2fc9813740 1 --2- 172.21.15.5:0/4154350922 >> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] conn(0x56054256def0 0x560542570330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:42:29.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:29.214+0000 7f2fc9813740 4 client.16158 mds_command: new command op to 24443 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:42:29.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:29.214+0000 7f2fc9813740 1 -- 172.21.15.5:0/4154350922 --> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5605424c6950 con 0x56054256def0 2024-09-10T09:42:29.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:29.214+0000 7f2fbfe00640 1 --2- 172.21.15.5:0/4154350922 >> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] conn(0x56054256def0 0x560542570330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:42:29.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:29.215+0000 7f2fbfe00640 1 --2- 172.21.15.5:0/4154350922 >> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] conn(0x56054256def0 0x560542570330 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=96927b93fb96dac6 server_cookie=d56edcd06d0df130 in_seq=0 out_seq=0 2024-09-10T09:42:29.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:29.215+0000 7f2fae000640 10 client.16158 ms_handle_connect on v2:172.21.15.102:6832/2457007427 2024-09-10T09:42:29.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:29.215+0000 7f2fae000640 1 -- 172.21.15.5:0/4154350922 <== mds.4 v2:172.21.15.102:6832/2457007427 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5605424c6950 con 0x56054256def0 2024-09-10T09:42:29.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:29.215+0000 7f2fae000640 10 client.16158 handle_command_reply: tid=0 2024-09-10T09:42:29.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:29.215+0000 7f2fc9813740 1 client.16158 shutdown 2024-09-10T09:42:29.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:29.216+0000 7f2fc9813740 1 -- 172.21.15.5:0/4154350922 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560542553590 msgr2=0x56054255b230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:42:29.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:29.216+0000 7f2fc9813740 1 --2- 172.21.15.5:0/4154350922 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560542553590 0x56054255b230 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x560542550580 tx=0x7f2fa8009560 comp rx=0 tx=0).stop 2024-09-10T09:42:29.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:29.216+0000 7f2fc9813740 1 -- 172.21.15.5:0/4154350922 shutdown_connections 2024-09-10T09:42:29.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:29.216+0000 7f2fc9813740 1 --2- 172.21.15.5:0/4154350922 >> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] conn(0x56054256def0 0x560542570330 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:29.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:29.216+0000 7f2fc9813740 1 --2- 172.21.15.5:0/4154350922 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56054255de50 0x56054255b770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:29.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:29.216+0000 7f2fc9813740 1 --2- 172.21.15.5:0/4154350922 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560542553590 0x56054255b230 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:29.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:29.216+0000 7f2fc9813740 1 --2- 172.21.15.5:0/4154350922 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560542552c70 0x56054255acf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:29.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:29.216+0000 7f2fae000640 0 client.16158 ms_handle_reset on v2:172.21.15.102:6832/2457007427 2024-09-10T09:42:29.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:29.216+0000 7f2fc9813740 1 -- 172.21.15.5:0/4154350922 >> 172.21.15.5:0/4154350922 conn(0x56054254f200 msgr2=0x56054254e380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:42:29.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:29.216+0000 7f2fc9813740 1 -- 172.21.15.5:0/4154350922 shutdown_connections 2024-09-10T09:42:29.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:29.216+0000 7f2fc9813740 1 -- 172.21.15.5:0/4154350922 wait complete. 2024-09-10T09:42:29.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:29.216+0000 7f2fc9813740 20 client.16158 trim_cache size 0 max 16384 2024-09-10T09:42:29.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:29.230+0000 7f2fc9813740 1 Processor -- start 2024-09-10T09:42:29.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:29.230+0000 7f2fc9813740 1 -- start start 2024-09-10T09:42:29.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:29.230+0000 7f2fc9813740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560542552c70 0x560542575c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:42:29.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:29.230+0000 7f2fc9813740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560542553590 0x560542576140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:42:29.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:29.230+0000 7f2fc9813740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56054255de50 0x560542557f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:42:29.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:29.230+0000 7f2fc9813740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f2fc01b10b0 con 0x56054255de50 2024-09-10T09:42:29.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:29.230+0000 7f2fc9813740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f2fc01b2c20 con 0x560542553590 2024-09-10T09:42:29.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:29.230+0000 7f2fc9813740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f2fc01b1780 con 0x560542552c70 2024-09-10T09:42:29.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:29.230+0000 7f2fc4e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56054255de50 0x560542557f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:42:29.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:29.230+0000 7f2fc4e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56054255de50 0x560542557f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35664/0 (socket says 172.21.15.5:35664) 2024-09-10T09:42:29.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:29.230+0000 7f2fc4e00640 1 -- 172.21.15.5:0/3949991650 learned_addr learned my addr 172.21.15.5:0/3949991650 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:42:29.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:29.230+0000 7f2fb7e00640 1 --2- 172.21.15.5:0/3949991650 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560542553590 0x560542576140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:42:29.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:29.230+0000 7f2fbfe00640 1 --2- 172.21.15.5:0/3949991650 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560542552c70 0x560542575c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:42:29.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:29.231+0000 7f2fc4e00640 1 -- 172.21.15.5:0/3949991650 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560542552c70 msgr2=0x560542575c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:42:29.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:29.231+0000 7f2fc4e00640 1 --2- 172.21.15.5:0/3949991650 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560542552c70 0x560542575c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:29.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:29.231+0000 7f2fc4e00640 1 -- 172.21.15.5:0/3949991650 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560542553590 msgr2=0x560542576140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:42:29.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:29.231+0000 7f2fc4e00640 1 --2- 172.21.15.5:0/3949991650 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560542553590 0x560542576140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:29.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:29.231+0000 7f2fc4e00640 1 -- 172.21.15.5:0/3949991650 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56054254eca0 con 0x56054255de50 2024-09-10T09:42:29.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:29.231+0000 7f2fb7e00640 1 --2- 172.21.15.5:0/3949991650 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560542553590 0x560542576140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:42:29.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:29.231+0000 7f2fc4e00640 1 --2- 172.21.15.5:0/3949991650 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56054255de50 0x560542557f50 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f2fb8000f80 tx=0x7f2fb80124f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b26c0c22dd742f53 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:42:29.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:29.231+0000 7f2fb4c00640 1 -- 172.21.15.5:0/3949991650 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fb802a070 con 0x56054255de50 2024-09-10T09:42:29.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:29.231+0000 7f2fb4c00640 1 -- 172.21.15.5:0/3949991650 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2fb8012a40 con 0x56054255de50 2024-09-10T09:42:29.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:29.231+0000 7f2fb4c00640 1 -- 172.21.15.5:0/3949991650 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fb8016050 con 0x56054255de50 2024-09-10T09:42:29.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:29.232+0000 7f2fc9813740 1 -- 172.21.15.5:0/3949991650 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56054255de50 msgr2=0x560542557f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:42:29.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:29.232+0000 7f2fc9813740 1 --2- 172.21.15.5:0/3949991650 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56054255de50 0x560542557f50 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f2fb8000f80 tx=0x7f2fb80124f0 comp rx=0 tx=0).stop 2024-09-10T09:42:29.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:29.232+0000 7f2fc9813740 1 -- 172.21.15.5:0/3949991650 shutdown_connections 2024-09-10T09:42:29.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:29.232+0000 7f2fc9813740 1 --2- 172.21.15.5:0/3949991650 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56054255de50 0x560542557f50 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:29.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:29.232+0000 7f2fc9813740 1 --2- 172.21.15.5:0/3949991650 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560542553590 0x560542576140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:29.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:29.232+0000 7f2fc9813740 1 --2- 172.21.15.5:0/3949991650 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560542552c70 0x560542575c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:29.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:29.232+0000 7f2fc9813740 1 -- 172.21.15.5:0/3949991650 >> 172.21.15.5:0/3949991650 conn(0x56054254f200 msgr2=0x56054255cd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:42:29.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:29.232+0000 7f2fc9813740 1 -- 172.21.15.5:0/3949991650 shutdown_connections 2024-09-10T09:42:29.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:29.232+0000 7f2fc9813740 1 -- 172.21.15.5:0/3949991650 wait complete. 2024-09-10T09:42:29.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:29.232+0000 7f2fc9813740 1 -- start start 2024-09-10T09:42:29.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:29.233+0000 7f2fc9813740 1 Processor -- start 2024-09-10T09:42:29.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:29.233+0000 7f2fc9813740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560542552c70 0x560542562330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:42:29.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:29.233+0000 7f2fc9813740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560542553590 0x560542562870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:42:29.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:29.233+0000 7f2fc9813740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56054255de50 0x5605425d7bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:42:29.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:29.233+0000 7f2fc9813740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f2fc01b2520 con 0x560542553590 2024-09-10T09:42:29.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:29.233+0000 7f2fc9813740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f2fc01b1e50 con 0x56054255de50 2024-09-10T09:42:29.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:29.233+0000 7f2fc9813740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f2fc01af090 con 0x560542552c70 2024-09-10T09:42:29.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:29.233+0000 7f2fc9813740 10 client.0 _ll_register_callbacks cb 0x7f2fa80f3240 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:42:29.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:29.233+0000 7f2fb7e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560542553590 0x560542562870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:42:29.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:29.233+0000 7f2fb7e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560542553590 0x560542562870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35678/0 (socket says 172.21.15.5:35678) 2024-09-10T09:42:29.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:29.234+0000 7f2fb7e00640 1 -- 172.21.15.5:0/2360352864 learned_addr learned my addr 172.21.15.5:0/2360352864 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:42:29.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:29.234+0000 7f2fc4e00640 1 --2- 172.21.15.5:0/2360352864 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56054255de50 0x5605425d7bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:42:29.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:29.234+0000 7f2fbfe00640 1 --2- 172.21.15.5:0/2360352864 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560542552c70 0x560542562330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:42:29.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:29.234+0000 7f2fb7e00640 1 -- 172.21.15.5:0/2360352864 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560542552c70 msgr2=0x560542562330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:42:29.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:29.234+0000 7f2fb7e00640 1 --2- 172.21.15.5:0/2360352864 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560542552c70 0x560542562330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:29.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:29.234+0000 7f2fb7e00640 1 -- 172.21.15.5:0/2360352864 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56054255de50 msgr2=0x5605425d7bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:42:29.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:29.234+0000 7f2fb7e00640 1 --2- 172.21.15.5:0/2360352864 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56054255de50 0x5605425d7bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:29.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:29.234+0000 7f2fb7e00640 1 -- 172.21.15.5:0/2360352864 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5605425d81c0 con 0x560542553590 2024-09-10T09:42:29.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:29.234+0000 7f2fb7e00640 1 --2- 172.21.15.5:0/2360352864 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560542553590 0x560542562870 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f2fa80bbf60 tx=0x7f2fa803b460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a02ba41aa199b0a5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:42:29.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:29.234+0000 7f2fae000640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-10T09:42:29.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:29.235+0000 7f2fae000640 1 -- 172.21.15.5:0/2360352864 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fa8049040 con 0x560542553590 2024-09-10T09:42:29.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:29.235+0000 7f2fae000640 1 -- 172.21.15.5:0/2360352864 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f2fa80dab60 con 0x560542553590 2024-09-10T09:42:29.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:29.235+0000 7f2fc9813740 1 -- 172.21.15.5:0/2360352864 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2fb8012a40 con 0x560542553590 2024-09-10T09:42:29.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:29.235+0000 7f2fae000640 1 -- 172.21.15.5:0/2360352864 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f2fa80ce050 con 0x560542553590 2024-09-10T09:42:29.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:29.235+0000 7f2fc9813740 10 client.16164 fetch_fsmap learned FSMap version 33 2024-09-10T09:42:29.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:29.235+0000 7f2fc9813740 1 -- 172.21.15.5:0/2360352864 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5605425d9480 con 0x560542553590 2024-09-10T09:42:29.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:29.235+0000 7f2fae000640 1 -- 172.21.15.5:0/2360352864 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2fb8012a40 con 0x560542553590 2024-09-10T09:42:29.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:29.235+0000 7f2fae000640 1 -- 172.21.15.5:0/2360352864 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 33) ==== 8374+0+0 (secure 0 0 0) 0x7f2fa80b6b40 con 0x560542553590 2024-09-10T09:42:29.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:29.235+0000 7f2fc9813740 10 client.16164 fetch_fsmap finished waiting for FSMap version 33 2024-09-10T09:42:29.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:29.235+0000 7f2fc9813740 10 client.16164 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-09-10T09:42:29.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:29.235+0000 7f2fc9813740 20 client.16164 populate_metadata read hostname 'smithi005' 2024-09-10T09:42:29.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:29.235+0000 7f2fc9813740 1 --2- 172.21.15.5:0/2360352864 >> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] conn(0x5605425d99b0 0x5605425dbda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:42:29.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:29.235+0000 7f2fc9813740 4 client.16164 mds_command: new command op to 24443 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T09:42:29.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:29.235+0000 7f2fc9813740 1 -- 172.21.15.5:0/2360352864 --> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5605424c0e60 con 0x5605425d99b0 2024-09-10T09:42:29.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:29.236+0000 7f2fbfe00640 1 --2- 172.21.15.5:0/2360352864 >> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] conn(0x5605425d99b0 0x5605425dbda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:42:29.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:29.236+0000 7f2fbfe00640 1 --2- 172.21.15.5:0/2360352864 >> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] conn(0x5605425d99b0 0x5605425dbda0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=a91e25db5018bcf7 server_cookie=b53c8d4bdfd18310 in_seq=0 out_seq=0 2024-09-10T09:42:29.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:29.236+0000 7f2fae000640 10 client.16164 ms_handle_connect on v2:172.21.15.102:6832/2457007427 2024-09-10T09:42:29.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:29.236+0000 7f2fae000640 1 -- 172.21.15.5:0/2360352864 <== mds.4 v2:172.21.15.102:6832/2457007427 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5605424c0e60 con 0x5605425d99b0 2024-09-10T09:42:29.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:29.236+0000 7f2fae000640 10 client.16164 handle_command_reply: tid=0 2024-09-10T09:42:29.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:29.236+0000 7f2fc9813740 1 client.16164 shutdown 2024-09-10T09:42:29.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:29.237+0000 7f2fc9813740 1 -- 172.21.15.5:0/2360352864 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560542553590 msgr2=0x560542562870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:42:29.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:29.237+0000 7f2fc9813740 1 --2- 172.21.15.5:0/2360352864 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560542553590 0x560542562870 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f2fa80bbf60 tx=0x7f2fa803b460 comp rx=0 tx=0).stop 2024-09-10T09:42:29.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:29.237+0000 7f2fc9813740 1 -- 172.21.15.5:0/2360352864 shutdown_connections 2024-09-10T09:42:29.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:29.237+0000 7f2fc9813740 1 --2- 172.21.15.5:0/2360352864 >> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] conn(0x5605425d99b0 0x5605425dbda0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:29.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:29.237+0000 7f2fc9813740 1 --2- 172.21.15.5:0/2360352864 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56054255de50 0x5605425d7bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:29.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:29.237+0000 7f2fc9813740 1 --2- 172.21.15.5:0/2360352864 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560542553590 0x560542562870 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:29.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:29.237+0000 7f2fc9813740 1 --2- 172.21.15.5:0/2360352864 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560542552c70 0x560542562330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:29.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:29.237+0000 7f2fc9813740 1 -- 172.21.15.5:0/2360352864 >> 172.21.15.5:0/2360352864 conn(0x56054254f200 msgr2=0x56054254db60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:42:29.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:29.237+0000 7f2fae000640 0 client.16164 ms_handle_reset on v2:172.21.15.102:6832/2457007427 2024-09-10T09:42:29.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:29.237+0000 7f2fc9813740 1 -- 172.21.15.5:0/2360352864 shutdown_connections 2024-09-10T09:42:29.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:29.237+0000 7f2fc9813740 1 -- 172.21.15.5:0/2360352864 wait complete. 2024-09-10T09:42:29.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:29.237+0000 7f2fc9813740 20 client.16164 trim_cache size 0 max 16384 2024-09-10T09:42:29.256 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-10T09:42:29.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:29.238+0000 7f2fc6e00640 1 -- 172.21.15.5:0/684353248 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f2fa40a2b80 msgr2=0x7f2fa40a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:42:29.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:29.238+0000 7f2fc6e00640 1 --2- 172.21.15.5:0/684353248 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f2fa40a2b80 0x7f2fa40a5040 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x560542550060 tx=0x7f2fb0005f90 comp rx=0 tx=0).stop 2024-09-10T09:42:29.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:29.238+0000 7f2fc6e00640 1 -- 172.21.15.5:0/684353248 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2fc0102790 msgr2=0x7f2fc019ab90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:42:29.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:29.238+0000 7f2fc6e00640 1 --2- 172.21.15.5:0/684353248 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2fc0102790 0x7f2fc019ab90 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f2fa8004770 tx=0x7f2fa8003c10 comp rx=0 tx=0).stop 2024-09-10T09:42:29.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:29.238+0000 7f2fc6e00640 1 -- 172.21.15.5:0/684353248 shutdown_connections 2024-09-10T09:42:29.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:29.238+0000 7f2fc6e00640 1 --2- 172.21.15.5:0/684353248 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f2fa40a2b80 0x7f2fa40a5040 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:29.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:29.238+0000 7f2fc6e00640 1 --2- 172.21.15.5:0/684353248 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2fc0106f10 0x7f2fc019ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:29.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:29.238+0000 7f2fc6e00640 1 --2- 172.21.15.5:0/684353248 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2fc0102790 0x7f2fc019ab90 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:29.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:29.238+0000 7f2fc6e00640 1 --2- 172.21.15.5:0/684353248 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2fc0101de0 0x7f2fc019a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:29.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:29.238+0000 7f2fc6e00640 1 -- 172.21.15.5:0/684353248 >> 172.21.15.5:0/684353248 conn(0x7f2fc00fd8c0 msgr2=0x7f2fc00ffc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:42:29.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:29.238+0000 7f2fc6e00640 1 -- 172.21.15.5:0/684353248 shutdown_connections 2024-09-10T09:42:29.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:29.238+0000 7f2fc6e00640 1 -- 172.21.15.5:0/684353248 wait complete. 2024-09-10T09:42:29.856 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:42:29 smithi159 ceph-mon[35513]: pgmap v517: 129 pgs: 129 active+clean; 95 MiB data, 853 MiB used, 1.0 TiB / 1.0 TiB avail; 555 KiB/s rd, 1.2 MiB/s wr, 69 op/s 2024-09-10T09:42:29.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:42:29 smithi102 ceph-mon[35442]: pgmap v517: 129 pgs: 129 active+clean; 95 MiB data, 853 MiB used, 1.0 TiB / 1.0 TiB avail; 555 KiB/s rd, 1.2 MiB/s wr, 69 op/s 2024-09-10T09:42:29.918 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:42:29 smithi005 ceph-mon[30350]: pgmap v517: 129 pgs: 129 active+clean; 95 MiB data, 853 MiB used, 1.0 TiB / 1.0 TiB avail; 555 KiB/s rd, 1.2 MiB/s wr, 69 op/s 2024-09-10T09:42:29.957 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-10T09:42:30.015 INFO:tasks.workunit.client.0.smithi005.stdout: 1 191939 0.00 MB/sec execute 11 sec latency 14920.715 ms 2024-09-10T09:42:30.336 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:42:30.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:30.902+0000 7fcb16000640 1 -- 172.21.15.5:0/2860518378 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcb1010e3c0 msgr2=0x7fcb10110810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:42:30.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:30.902+0000 7fcb16000640 1 --2- 172.21.15.5:0/2860518378 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcb1010e3c0 0x7fcb10110810 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fcb08009ca0 tx=0x7fcb0802a950 comp rx=0 tx=0).stop 2024-09-10T09:42:30.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:30.903+0000 7fcb16000640 1 -- 172.21.15.5:0/2860518378 shutdown_connections 2024-09-10T09:42:30.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:30.903+0000 7fcb16000640 1 --2- 172.21.15.5:0/2860518378 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcb1010e3c0 0x7fcb10110810 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:30.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:30.903+0000 7fcb16000640 1 --2- 172.21.15.5:0/2860518378 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcb10074f40 0x7fcb1010de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:30.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:30.903+0000 7fcb16000640 1 --2- 172.21.15.5:0/2860518378 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcb10074620 0x7fcb10074a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:30.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:30.903+0000 7fcb16000640 1 -- 172.21.15.5:0/2860518378 >> 172.21.15.5:0/2860518378 conn(0x7fcb100fd8c0 msgr2=0x7fcb100ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:42:30.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:30.903+0000 7fcb16000640 1 -- 172.21.15.5:0/2860518378 shutdown_connections 2024-09-10T09:42:30.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:30.903+0000 7fcb16000640 1 -- 172.21.15.5:0/2860518378 wait complete. 2024-09-10T09:42:30.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:30.904+0000 7fcb16000640 1 Processor -- start 2024-09-10T09:42:30.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:30.904+0000 7fcb16000640 1 -- start start 2024-09-10T09:42:30.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:30.904+0000 7fcb16000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcb10074620 0x7fcb1019a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:42:30.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:30.904+0000 7fcb16000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcb10074f40 0x7fcb1019ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:42:30.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:30.904+0000 7fcb16000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcb1010e3c0 0x7fcb1019ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:42:30.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:30.904+0000 7fcb16000640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fcb101141b0 con 0x7fcb1010e3c0 2024-09-10T09:42:30.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:30.904+0000 7fcb16000640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fcb10114030 con 0x7fcb10074f40 2024-09-10T09:42:30.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:30.904+0000 7fcb16000640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fcb10114330 con 0x7fcb10074620 2024-09-10T09:42:30.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:30.904+0000 7fcb0fe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcb1010e3c0 0x7fcb1019ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:42:30.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:30.905+0000 7fcb06a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcb10074f40 0x7fcb1019ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:42:30.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:30.905+0000 7fcb0fe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcb1010e3c0 0x7fcb1019ef20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46042/0 (socket says 172.21.15.5:46042) 2024-09-10T09:42:30.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:30.905+0000 7fcb0fe00640 1 -- 172.21.15.5:0/2833441503 learned_addr learned my addr 172.21.15.5:0/2833441503 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:42:30.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:30.905+0000 7fcb06a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcb10074f40 0x7fcb1019ab90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:34352/0 (socket says 172.21.15.5:34352) 2024-09-10T09:42:30.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:30.905+0000 7fcb0f400640 1 --2- 172.21.15.5:0/2833441503 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcb10074620 0x7fcb1019a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:42:30.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:30.905+0000 7fcb0fe00640 1 -- 172.21.15.5:0/2833441503 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcb10074620 msgr2=0x7fcb1019a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:42:30.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:30.905+0000 7fcb0fe00640 1 --2- 172.21.15.5:0/2833441503 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcb10074620 0x7fcb1019a650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:30.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:30.905+0000 7fcb0fe00640 1 -- 172.21.15.5:0/2833441503 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcb10074f40 msgr2=0x7fcb1019ab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:42:30.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:30.905+0000 7fcb0fe00640 1 --2- 172.21.15.5:0/2833441503 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcb10074f40 0x7fcb1019ab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:30.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:30.905+0000 7fcb0fe00640 1 -- 172.21.15.5:0/2833441503 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb1019f600 con 0x7fcb1010e3c0 2024-09-10T09:42:30.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:30.905+0000 7fcb0fe00640 1 --2- 172.21.15.5:0/2833441503 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcb1010e3c0 0x7fcb1019ef20 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fcb08009dc0 tx=0x7fcb08003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:42:30.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:30.906+0000 7fcb0cc00640 1 -- 172.21.15.5:0/2833441503 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb08034040 con 0x7fcb1010e3c0 2024-09-10T09:42:30.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:30.906+0000 7fcb0cc00640 1 -- 172.21.15.5:0/2833441503 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcb0802f940 con 0x7fcb1010e3c0 2024-09-10T09:42:30.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:30.906+0000 7fcb0cc00640 1 -- 172.21.15.5:0/2833441503 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb080046a0 con 0x7fcb1010e3c0 2024-09-10T09:42:30.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:30.906+0000 7fcb16000640 1 -- 172.21.15.5:0/2833441503 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb1019f890 con 0x7fcb1010e3c0 2024-09-10T09:42:30.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:30.906+0000 7fcb16000640 1 -- 172.21.15.5:0/2833441503 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb101a71d0 con 0x7fcb1010e3c0 2024-09-10T09:42:30.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:30.909+0000 7fcb18a3a740 1 Processor -- start 2024-09-10T09:42:30.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:30.909+0000 7fcb0cc00640 1 -- 172.21.15.5:0/2833441503 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7fcb08037020 con 0x7fcb1010e3c0 2024-09-10T09:42:30.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:30.909+0000 7fcb18a3a740 1 -- start start 2024-09-10T09:42:30.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:30.910+0000 7fcb18a3a740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564f64babc70 0x564f64bac050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:42:30.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:30.910+0000 7fcb18a3a740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564f64bac590 0x564f64bb6910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:42:30.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:30.910+0000 7fcb18a3a740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564f64bb6e50 0x564f64bb9240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:42:30.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:30.910+0000 7fcb18a3a740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x564f64b123e0 con 0x564f64babc70 2024-09-10T09:42:30.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:30.910+0000 7fcb18a3a740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x564f64ab9d80 con 0x564f64bac590 2024-09-10T09:42:30.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:30.910+0000 7fcb18a3a740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x564f64b0e3a0 con 0x564f64bb6e50 2024-09-10T09:42:30.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:30.910+0000 7fcb06a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564f64babc70 0x564f64bac050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:42:30.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:30.910+0000 7fcb0f400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564f64bac590 0x564f64bb6910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:42:30.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:30.910+0000 7fcb06a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564f64babc70 0x564f64bac050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46050/0 (socket says 172.21.15.5:46050) 2024-09-10T09:42:30.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:30.910+0000 7fcb06a00640 1 -- 172.21.15.5:0/1353422932 learned_addr learned my addr 172.21.15.5:0/1353422932 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:42:30.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:30.910+0000 7fcb06a00640 1 --2- 172.21.15.5:0/1353422932 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564f64bb6e50 0x564f64bb9240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:42:30.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:30.911+0000 7fcb0f400640 1 -- 172.21.15.5:0/1353422932 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564f64bb6e50 msgr2=0x564f64bb9240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:42:30.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:30.911+0000 7fcb0f400640 1 --2- 172.21.15.5:0/1353422932 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564f64bb6e50 0x564f64bb9240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:30.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:30.911+0000 7fcb0f400640 1 -- 172.21.15.5:0/1353422932 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564f64babc70 msgr2=0x564f64bac050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:42:30.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:30.911+0000 7fcb0f400640 1 --2- 172.21.15.5:0/1353422932 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564f64babc70 0x564f64bac050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:30.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:30.911+0000 7fcb0f400640 1 -- 172.21.15.5:0/1353422932 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564f64ad8ae0 con 0x564f64bac590 2024-09-10T09:42:30.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:30.911+0000 7fcb06a00640 1 --2- 172.21.15.5:0/1353422932 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564f64bb6e50 0x564f64bb9240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:42:30.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:30.911+0000 7fcb0f400640 1 --2- 172.21.15.5:0/1353422932 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564f64bac590 0x564f64bb6910 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fcb00009c00 tx=0x7fcb00011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c6a0327be427266d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:42:30.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:30.911+0000 7fcb0cc00640 1 --2- 172.21.15.5:0/2833441503 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fcaf40a2b80 0x7fcaf40a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:42:30.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:30.911+0000 7fcb0cc00640 1 -- 172.21.15.5:0/2833441503 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fcb080f01c0 con 0x7fcb1010e3c0 2024-09-10T09:42:30.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:30.911+0000 7fcaffe00640 1 -- 172.21.15.5:0/1353422932 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb00004230 con 0x564f64bac590 2024-09-10T09:42:30.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:30.911+0000 7fcb06a00640 1 --2- 172.21.15.5:0/2833441503 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fcaf40a2b80 0x7fcaf40a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:42:30.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:30.911+0000 7fcaffe00640 1 -- 172.21.15.5:0/1353422932 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcb000043d0 con 0x564f64bac590 2024-09-10T09:42:30.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:30.911+0000 7fcaffe00640 1 -- 172.21.15.5:0/1353422932 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb00004d30 con 0x564f64bac590 2024-09-10T09:42:30.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:30.912+0000 7fcb18a3a740 1 -- 172.21.15.5:0/1353422932 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564f64bac590 msgr2=0x564f64bb6910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:42:30.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:30.912+0000 7fcb18a3a740 1 --2- 172.21.15.5:0/1353422932 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564f64bac590 0x564f64bb6910 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fcb00009c00 tx=0x7fcb00011810 comp rx=0 tx=0).stop 2024-09-10T09:42:30.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:30.912+0000 7fcb06a00640 1 --2- 172.21.15.5:0/2833441503 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fcaf40a2b80 0x7fcaf40a5040 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x564f64ba9580 tx=0x7fcaf8005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:42:30.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:30.912+0000 7fcb18a3a740 1 -- 172.21.15.5:0/1353422932 shutdown_connections 2024-09-10T09:42:30.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:30.912+0000 7fcb18a3a740 1 --2- 172.21.15.5:0/1353422932 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564f64bb6e50 0x564f64bb9240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:30.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:30.912+0000 7fcb18a3a740 1 --2- 172.21.15.5:0/1353422932 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564f64bac590 0x564f64bb6910 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:30.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:30.912+0000 7fcb18a3a740 1 --2- 172.21.15.5:0/1353422932 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564f64babc70 0x564f64bac050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:30.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:30.912+0000 7fcb18a3a740 1 -- 172.21.15.5:0/1353422932 >> 172.21.15.5:0/1353422932 conn(0x564f64ba8200 msgr2=0x564f64ba85e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:42:30.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:30.912+0000 7fcb18a3a740 1 -- 172.21.15.5:0/1353422932 shutdown_connections 2024-09-10T09:42:30.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:30.913+0000 7fcb18a3a740 1 -- 172.21.15.5:0/1353422932 wait complete. 2024-09-10T09:42:30.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:30.913+0000 7fcb18a3a740 1 -- start start 2024-09-10T09:42:30.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:30.913+0000 7fcb18a3a740 1 Processor -- start 2024-09-10T09:42:30.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:30.914+0000 7fcb18a3a740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564f64babc70 0x564f64bb3ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:42:30.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:30.914+0000 7fcb18a3a740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564f64bac590 0x564f64bb4220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:42:30.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:30.914+0000 7fcb18a3a740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564f64bb6e50 0x564f64bb4760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:42:30.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:30.914+0000 7fcb18a3a740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x564f64bb9a20 con 0x564f64bac590 2024-09-10T09:42:30.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:30.914+0000 7fcb18a3a740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x564f64acd0f0 con 0x564f64babc70 2024-09-10T09:42:30.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:30.914+0000 7fcb18a3a740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x564f64bb9ba0 con 0x564f64bb6e50 2024-09-10T09:42:30.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:30.914+0000 7fcb18a3a740 10 client.0 _ll_register_callbacks cb 0x564f64ab4bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:42:30.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:30.914+0000 7fcb06a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564f64bac590 0x564f64bb4220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:42:30.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:30.914+0000 7fcb0f400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564f64babc70 0x564f64bb3ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:42:30.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:30.914+0000 7fcb0fe00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564f64bb6e50 0x564f64bb4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:42:30.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:30.914+0000 7fcb06a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564f64bac590 0x564f64bb4220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46054/0 (socket says 172.21.15.5:46054) 2024-09-10T09:42:30.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:30.914+0000 7fcb06a00640 1 -- 172.21.15.5:0/2241872698 learned_addr learned my addr 172.21.15.5:0/2241872698 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:42:30.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:30.914+0000 7fcb06a00640 1 -- 172.21.15.5:0/2241872698 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564f64bb6e50 msgr2=0x564f64bb4760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:42:30.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:30.914+0000 7fcb06a00640 1 --2- 172.21.15.5:0/2241872698 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564f64bb6e50 0x564f64bb4760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:30.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:30.914+0000 7fcb06a00640 1 -- 172.21.15.5:0/2241872698 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564f64babc70 msgr2=0x564f64bb3ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:42:30.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:30.915+0000 7fcb06a00640 1 --2- 172.21.15.5:0/2241872698 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564f64babc70 0x564f64bb3ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:30.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:30.915+0000 7fcb06a00640 1 -- 172.21.15.5:0/2241872698 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564f648833c0 con 0x564f64bac590 2024-09-10T09:42:30.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:30.915+0000 7fcb0f400640 1 --2- 172.21.15.5:0/2241872698 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564f64babc70 0x564f64bb3ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:42:30.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:30.915+0000 7fcb06a00640 1 --2- 172.21.15.5:0/2241872698 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564f64bac590 0x564f64bb4220 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fcaf8005f40 tx=0x7fcaf8005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c8c38d5219e1d90 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:42:30.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:30.915+0000 7fcafcc00640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-10T09:42:30.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:30.915+0000 7fcafcc00640 1 -- 172.21.15.5:0/2241872698 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaf8021b20 con 0x564f64bac590 2024-09-10T09:42:30.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:30.915+0000 7fcafcc00640 1 -- 172.21.15.5:0/2241872698 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fcaf8021cc0 con 0x564f64bac590 2024-09-10T09:42:30.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:30.915+0000 7fcb18a3a740 1 -- 172.21.15.5:0/2241872698 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcb000043d0 con 0x564f64bac590 2024-09-10T09:42:30.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:30.916+0000 7fcafcc00640 1 -- 172.21.15.5:0/2241872698 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fcaf8005020 con 0x564f64bac590 2024-09-10T09:42:30.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:30.916+0000 7fcb18a3a740 10 client.16176 fetch_fsmap learned FSMap version 33 2024-09-10T09:42:30.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:30.916+0000 7fcb18a3a740 1 -- 172.21.15.5:0/2241872698 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x564f64bc59f0 con 0x564f64bac590 2024-09-10T09:42:30.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:30.916+0000 7fcafcc00640 1 -- 172.21.15.5:0/2241872698 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcb000043d0 con 0x564f64bac590 2024-09-10T09:42:30.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:30.916+0000 7fcafcc00640 1 -- 172.21.15.5:0/2241872698 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 33) ==== 8374+0+0 (secure 0 0 0) 0x7fcaf8022410 con 0x564f64bac590 2024-09-10T09:42:30.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:30.916+0000 7fcb18a3a740 10 client.16176 fetch_fsmap finished waiting for FSMap version 33 2024-09-10T09:42:30.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:30.916+0000 7fcb18a3a740 10 client.16176 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-09-10T09:42:30.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:30.916+0000 7fcb18a3a740 20 client.16176 populate_metadata read hostname 'smithi005' 2024-09-10T09:42:30.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:30.916+0000 7fcb18a3a740 1 --2- 172.21.15.5:0/2241872698 >> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] conn(0x564f64bc6eb0 0x564f64bc9350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:42:30.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:30.916+0000 7fcb18a3a740 4 client.16176 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:42:30.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:30.916+0000 7fcb18a3a740 1 -- 172.21.15.5:0/2241872698 --> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564f64b1f950 con 0x564f64bc6eb0 2024-09-10T09:42:30.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:30.917+0000 7fcb0f400640 1 --2- 172.21.15.5:0/2241872698 >> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] conn(0x564f64bc6eb0 0x564f64bc9350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:42:30.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:30.918+0000 7fcb0f400640 1 --2- 172.21.15.5:0/2241872698 >> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] conn(0x564f64bc6eb0 0x564f64bc9350 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=40a98f6ff56acda7 server_cookie=984b2d947e10922f in_seq=0 out_seq=0 2024-09-10T09:42:30.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:30.918+0000 7fcafcc00640 10 client.16176 ms_handle_connect on v2:172.21.15.102:6836/2689776764 2024-09-10T09:42:30.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:30.918+0000 7fcafcc00640 1 -- 172.21.15.5:0/2241872698 <== mds.3 v2:172.21.15.102:6836/2689776764 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564f64b1f950 con 0x564f64bc6eb0 2024-09-10T09:42:30.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:30.918+0000 7fcafcc00640 10 client.16176 handle_command_reply: tid=0 2024-09-10T09:42:30.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:30.919+0000 7fcb18a3a740 1 client.16176 shutdown 2024-09-10T09:42:30.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:30.919+0000 7fcb18a3a740 1 -- 172.21.15.5:0/2241872698 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564f64bac590 msgr2=0x564f64bb4220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:42:30.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:30.919+0000 7fcb18a3a740 1 --2- 172.21.15.5:0/2241872698 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564f64bac590 0x564f64bb4220 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fcaf8005f40 tx=0x7fcaf8005f90 comp rx=0 tx=0).stop 2024-09-10T09:42:30.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:30.919+0000 7fcb18a3a740 1 -- 172.21.15.5:0/2241872698 shutdown_connections 2024-09-10T09:42:30.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:30.919+0000 7fcb18a3a740 1 --2- 172.21.15.5:0/2241872698 >> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] conn(0x564f64bc6eb0 0x564f64bc9350 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:30.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:30.919+0000 7fcb18a3a740 1 --2- 172.21.15.5:0/2241872698 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564f64bb6e50 0x564f64bb4760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:30.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:30.919+0000 7fcb18a3a740 1 --2- 172.21.15.5:0/2241872698 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564f64bac590 0x564f64bb4220 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:30.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:30.919+0000 7fcb18a3a740 1 --2- 172.21.15.5:0/2241872698 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564f64babc70 0x564f64bb3ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:30.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:30.919+0000 7fcafcc00640 0 client.16176 ms_handle_reset on v2:172.21.15.102:6836/2689776764 2024-09-10T09:42:30.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:30.919+0000 7fcb18a3a740 1 -- 172.21.15.5:0/2241872698 >> 172.21.15.5:0/2241872698 conn(0x564f64ba8200 msgr2=0x564f64ba7380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:42:30.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:30.919+0000 7fcb18a3a740 1 -- 172.21.15.5:0/2241872698 shutdown_connections 2024-09-10T09:42:30.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:30.919+0000 7fcb18a3a740 1 -- 172.21.15.5:0/2241872698 wait complete. 2024-09-10T09:42:30.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:30.919+0000 7fcb18a3a740 20 client.16176 trim_cache size 0 max 16384 2024-09-10T09:42:30.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:30.933+0000 7fcb18a3a740 1 Processor -- start 2024-09-10T09:42:30.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:30.933+0000 7fcb18a3a740 1 -- start start 2024-09-10T09:42:30.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:30.933+0000 7fcb18a3a740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564f64ba8200 0x564f64bb5c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:42:30.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:30.933+0000 7fcb18a3a740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564f64babc70 0x564f64bcec20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:42:30.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:30.933+0000 7fcb18a3a740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564f64bac590 0x564f64bcf160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:42:30.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:30.933+0000 7fcb18a3a740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fcb101b1010 con 0x564f64babc70 2024-09-10T09:42:30.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:30.933+0000 7fcb18a3a740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fcb101b2b80 con 0x564f64ba8200 2024-09-10T09:42:30.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:30.933+0000 7fcb18a3a740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fcb101b16e0 con 0x564f64bac590 2024-09-10T09:42:30.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:30.934+0000 7fcb06a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564f64babc70 0x564f64bcec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:42:30.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:30.934+0000 7fcb0f400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564f64ba8200 0x564f64bb5c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:42:30.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:30.934+0000 7fcb06a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564f64babc70 0x564f64bcec20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46058/0 (socket says 172.21.15.5:46058) 2024-09-10T09:42:30.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:30.934+0000 7fcb06a00640 1 -- 172.21.15.5:0/1629739284 learned_addr learned my addr 172.21.15.5:0/1629739284 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:42:30.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:30.934+0000 7fcb0fe00640 1 --2- 172.21.15.5:0/1629739284 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564f64bac590 0x564f64bcf160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:42:30.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:30.934+0000 7fcb06a00640 1 -- 172.21.15.5:0/1629739284 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564f64bac590 msgr2=0x564f64bcf160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:42:30.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:30.934+0000 7fcb06a00640 1 --2- 172.21.15.5:0/1629739284 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564f64bac590 0x564f64bcf160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:30.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:30.934+0000 7fcb06a00640 1 -- 172.21.15.5:0/1629739284 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564f64ba8200 msgr2=0x564f64bb5c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:42:30.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:30.934+0000 7fcb06a00640 1 --2- 172.21.15.5:0/1629739284 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564f64ba8200 0x564f64bb5c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:30.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:30.934+0000 7fcb06a00640 1 -- 172.21.15.5:0/1629739284 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564f64ba3770 con 0x564f64babc70 2024-09-10T09:42:30.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:30.934+0000 7fcb0f400640 1 --2- 172.21.15.5:0/1629739284 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564f64ba8200 0x564f64bb5c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:42:30.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:30.934+0000 7fcb06a00640 1 --2- 172.21.15.5:0/1629739284 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564f64babc70 0x564f64bcec20 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fcaf801b040 tx=0x7fcaf8005db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcb7b24e0d5b8b8e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:42:30.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:30.935+0000 7fcaff400640 1 -- 172.21.15.5:0/1629739284 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaf8022cf0 con 0x564f64babc70 2024-09-10T09:42:30.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:30.935+0000 7fcaff400640 1 -- 172.21.15.5:0/1629739284 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcaf8022410 con 0x564f64babc70 2024-09-10T09:42:30.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:30.935+0000 7fcaff400640 1 -- 172.21.15.5:0/1629739284 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaf80219d0 con 0x564f64babc70 2024-09-10T09:42:30.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:30.935+0000 7fcb18a3a740 1 -- 172.21.15.5:0/1629739284 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564f64babc70 msgr2=0x564f64bcec20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:42:30.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:30.935+0000 7fcb18a3a740 1 --2- 172.21.15.5:0/1629739284 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564f64babc70 0x564f64bcec20 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fcaf801b040 tx=0x7fcaf8005db0 comp rx=0 tx=0).stop 2024-09-10T09:42:30.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:30.935+0000 7fcb18a3a740 1 -- 172.21.15.5:0/1629739284 shutdown_connections 2024-09-10T09:42:30.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:30.935+0000 7fcb18a3a740 1 --2- 172.21.15.5:0/1629739284 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564f64bac590 0x564f64bcf160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:30.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:30.935+0000 7fcb18a3a740 1 --2- 172.21.15.5:0/1629739284 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564f64babc70 0x564f64bcec20 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:30.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:30.935+0000 7fcb18a3a740 1 --2- 172.21.15.5:0/1629739284 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564f64ba8200 0x564f64bb5c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:30.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:30.935+0000 7fcb18a3a740 1 -- 172.21.15.5:0/1629739284 >> 172.21.15.5:0/1629739284 conn(0x564f64baf6d0 msgr2=0x564f64bb5540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:42:30.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:30.936+0000 7fcb18a3a740 1 -- 172.21.15.5:0/1629739284 shutdown_connections 2024-09-10T09:42:30.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:30.936+0000 7fcb18a3a740 1 -- 172.21.15.5:0/1629739284 wait complete. 2024-09-10T09:42:30.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:30.936+0000 7fcb18a3a740 1 -- start start 2024-09-10T09:42:30.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:30.936+0000 7fcb18a3a740 1 Processor -- start 2024-09-10T09:42:30.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:30.937+0000 7fcb18a3a740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564f64ba8200 0x564f64bbac80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:42:30.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:30.937+0000 7fcb18a3a740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564f64babc70 0x564f64bbb1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:42:30.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:30.937+0000 7fcb18a3a740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564f64bac590 0x564f64bbb700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:42:30.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:30.937+0000 7fcb18a3a740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fcb101b2480 con 0x564f64bac590 2024-09-10T09:42:30.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:30.937+0000 7fcb18a3a740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fcb101b1db0 con 0x564f64babc70 2024-09-10T09:42:30.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:30.937+0000 7fcb18a3a740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fcb101aeff0 con 0x564f64ba8200 2024-09-10T09:42:30.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:30.937+0000 7fcb18a3a740 10 client.0 _ll_register_callbacks cb 0x7fcaf8009940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:42:30.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:30.937+0000 7fcb0f400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564f64ba8200 0x564f64bbac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:42:30.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:30.937+0000 7fcb06a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564f64babc70 0x564f64bbb1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:42:30.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:30.937+0000 7fcb0f400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564f64ba8200 0x564f64bbac80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:40262/0 (socket says 172.21.15.5:40262) 2024-09-10T09:42:30.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:30.937+0000 7fcb0fe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564f64bac590 0x564f64bbb700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:42:30.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:30.937+0000 7fcb0f400640 1 -- 172.21.15.5:0/664422891 learned_addr learned my addr 172.21.15.5:0/664422891 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:42:30.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:30.937+0000 7fcb0f400640 1 -- 172.21.15.5:0/664422891 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564f64babc70 msgr2=0x564f64bbb1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:42:30.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:30.937+0000 7fcb0f400640 1 --2- 172.21.15.5:0/664422891 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564f64babc70 0x564f64bbb1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:30.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:30.937+0000 7fcb0f400640 1 -- 172.21.15.5:0/664422891 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564f64bac590 msgr2=0x564f64bbb700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:42:30.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:30.937+0000 7fcb0f400640 1 --2- 172.21.15.5:0/664422891 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564f64bac590 0x564f64bbb700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:30.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:30.937+0000 7fcb0f400640 1 -- 172.21.15.5:0/664422891 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564f64baeae0 con 0x564f64ba8200 2024-09-10T09:42:30.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:30.937+0000 7fcb06a00640 1 --2- 172.21.15.5:0/664422891 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564f64babc70 0x564f64bbb1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:42:30.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:30.938+0000 7fcb0f400640 1 --2- 172.21.15.5:0/664422891 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564f64ba8200 0x564f64bbac80 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fcb00009c00 tx=0x7fcb00005dc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=59c75c6a77ec98f9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:42:30.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:30.938+0000 7fcafcc00640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-09-10T09:42:30.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:30.938+0000 7fcafcc00640 1 -- 172.21.15.5:0/664422891 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb0001e040 con 0x564f64ba8200 2024-09-10T09:42:30.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:30.938+0000 7fcafcc00640 1 -- 172.21.15.5:0/664422891 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fcb00004450 con 0x564f64ba8200 2024-09-10T09:42:30.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:30.938+0000 7fcb18a3a740 1 -- 172.21.15.5:0/664422891 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcaf8022410 con 0x564f64ba8200 2024-09-10T09:42:30.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:30.938+0000 7fcafcc00640 1 -- 172.21.15.5:0/664422891 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fcb00037050 con 0x564f64ba8200 2024-09-10T09:42:30.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:30.938+0000 7fcb18a3a740 10 client.26206 fetch_fsmap learned FSMap version 33 2024-09-10T09:42:30.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:30.938+0000 7fcb18a3a740 1 -- 172.21.15.5:0/664422891 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x564f64ba8e90 con 0x564f64ba8200 2024-09-10T09:42:30.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:30.939+0000 7fcafcc00640 1 -- 172.21.15.5:0/664422891 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcaf8022410 con 0x564f64ba8200 2024-09-10T09:42:30.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:30.939+0000 7fcafcc00640 1 -- 172.21.15.5:0/664422891 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 33) ==== 8374+0+0 (secure 0 0 0) 0x7fcb00024040 con 0x564f64ba8200 2024-09-10T09:42:30.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:30.939+0000 7fcb18a3a740 10 client.26206 fetch_fsmap finished waiting for FSMap version 33 2024-09-10T09:42:30.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:30.939+0000 7fcb18a3a740 10 client.26206 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-09-10T09:42:30.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:30.939+0000 7fcb18a3a740 20 client.26206 populate_metadata read hostname 'smithi005' 2024-09-10T09:42:30.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:30.939+0000 7fcb18a3a740 1 --2- 172.21.15.5:0/664422891 >> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] conn(0x564f64bb6e50 0x564f64c34210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:42:30.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:30.939+0000 7fcb18a3a740 4 client.26206 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T09:42:30.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:30.939+0000 7fcb18a3a740 1 -- 172.21.15.5:0/664422891 --> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564f64b19e60 con 0x564f64bb6e50 2024-09-10T09:42:30.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:30.940+0000 7fcb06a00640 1 --2- 172.21.15.5:0/664422891 >> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] conn(0x564f64bb6e50 0x564f64c34210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:42:30.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:30.940+0000 7fcb06a00640 1 --2- 172.21.15.5:0/664422891 >> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] conn(0x564f64bb6e50 0x564f64c34210 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=9c3b83841d833d9a server_cookie=6516b12a55396413 in_seq=0 out_seq=0 2024-09-10T09:42:30.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:30.940+0000 7fcafcc00640 10 client.26206 ms_handle_connect on v2:172.21.15.102:6836/2689776764 2024-09-10T09:42:30.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:30.940+0000 7fcafcc00640 1 -- 172.21.15.5:0/664422891 <== mds.3 v2:172.21.15.102:6836/2689776764 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564f64b19e60 con 0x564f64bb6e50 2024-09-10T09:42:30.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:30.940+0000 7fcafcc00640 10 client.26206 handle_command_reply: tid=0 2024-09-10T09:42:30.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:30.940+0000 7fcb18a3a740 1 client.26206 shutdown 2024-09-10T09:42:30.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:30.940+0000 7fcb18a3a740 1 -- 172.21.15.5:0/664422891 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564f64ba8200 msgr2=0x564f64bbac80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:42:30.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:30.940+0000 7fcb18a3a740 1 --2- 172.21.15.5:0/664422891 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564f64ba8200 0x564f64bbac80 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fcb00009c00 tx=0x7fcb00005dc0 comp rx=0 tx=0).stop 2024-09-10T09:42:30.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:30.941+0000 7fcb18a3a740 1 -- 172.21.15.5:0/664422891 shutdown_connections 2024-09-10T09:42:30.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:30.941+0000 7fcb18a3a740 1 --2- 172.21.15.5:0/664422891 >> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] conn(0x564f64bb6e50 0x564f64c34210 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:30.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:30.941+0000 7fcb18a3a740 1 --2- 172.21.15.5:0/664422891 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564f64bac590 0x564f64bbb700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:30.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:30.941+0000 7fcb18a3a740 1 --2- 172.21.15.5:0/664422891 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564f64babc70 0x564f64bbb1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:30.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:30.941+0000 7fcb18a3a740 1 --2- 172.21.15.5:0/664422891 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564f64ba8200 0x564f64bbac80 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:30.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:30.941+0000 7fcafcc00640 0 client.26206 ms_handle_reset on v2:172.21.15.102:6836/2689776764 2024-09-10T09:42:30.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:30.941+0000 7fcb18a3a740 1 -- 172.21.15.5:0/664422891 >> 172.21.15.5:0/664422891 conn(0x564f64baf6d0 msgr2=0x564f64bb5540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:42:30.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:30.941+0000 7fcb18a3a740 1 -- 172.21.15.5:0/664422891 shutdown_connections 2024-09-10T09:42:30.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:30.941+0000 7fcb18a3a740 1 -- 172.21.15.5:0/664422891 wait complete. 2024-09-10T09:42:30.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:30.941+0000 7fcb18a3a740 20 client.26206 trim_cache size 0 max 16384 2024-09-10T09:42:30.951 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-10T09:42:30.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:30.942+0000 7fcb16000640 1 -- 172.21.15.5:0/2833441503 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fcaf40a2b80 msgr2=0x7fcaf40a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:42:30.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:30.942+0000 7fcb16000640 1 --2- 172.21.15.5:0/2833441503 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fcaf40a2b80 0x7fcaf40a5040 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x564f64ba9580 tx=0x7fcaf8005850 comp rx=0 tx=0).stop 2024-09-10T09:42:30.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:30.942+0000 7fcb16000640 1 -- 172.21.15.5:0/2833441503 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcb1010e3c0 msgr2=0x7fcb1019ef20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:42:30.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:30.942+0000 7fcb16000640 1 --2- 172.21.15.5:0/2833441503 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcb1010e3c0 0x7fcb1019ef20 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fcb08009dc0 tx=0x7fcb08003c10 comp rx=0 tx=0).stop 2024-09-10T09:42:30.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:30.942+0000 7fcb16000640 1 -- 172.21.15.5:0/2833441503 shutdown_connections 2024-09-10T09:42:30.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:30.942+0000 7fcb16000640 1 --2- 172.21.15.5:0/2833441503 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fcaf40a2b80 0x7fcaf40a5040 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:30.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:30.942+0000 7fcb16000640 1 --2- 172.21.15.5:0/2833441503 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcb1010e3c0 0x7fcb1019ef20 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:30.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:30.942+0000 7fcb16000640 1 --2- 172.21.15.5:0/2833441503 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcb10074f40 0x7fcb1019ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:30.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:30.942+0000 7fcb16000640 1 --2- 172.21.15.5:0/2833441503 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcb10074620 0x7fcb1019a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:30.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:30.942+0000 7fcb16000640 1 -- 172.21.15.5:0/2833441503 >> 172.21.15.5:0/2833441503 conn(0x7fcb100fd8c0 msgr2=0x7fcb100fe520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:42:30.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:30.942+0000 7fcb16000640 1 -- 172.21.15.5:0/2833441503 shutdown_connections 2024-09-10T09:42:30.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:30.942+0000 7fcb16000640 1 -- 172.21.15.5:0/2833441503 wait complete. 2024-09-10T09:42:31.016 INFO:tasks.workunit.client.0.smithi005.stdout: 1 191939 0.00 MB/sec execute 12 sec latency 15920.819 ms 2024-09-10T09:42:31.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:42:31 smithi005 ceph-mon[30350]: pgmap v518: 129 pgs: 129 active+clean; 98 MiB data, 841 MiB used, 1.0 TiB / 1.0 TiB avail; 360 KiB/s rd, 1.5 MiB/s wr, 73 op/s 2024-09-10T09:42:31.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:42:31 smithi159 ceph-mon[35513]: pgmap v518: 129 pgs: 129 active+clean; 98 MiB data, 841 MiB used, 1.0 TiB / 1.0 TiB avail; 360 KiB/s rd, 1.5 MiB/s wr, 73 op/s 2024-09-10T09:42:31.882 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-10T09:42:31.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:42:31 smithi102 ceph-mon[35442]: pgmap v518: 129 pgs: 129 active+clean; 98 MiB data, 841 MiB used, 1.0 TiB / 1.0 TiB avail; 360 KiB/s rd, 1.5 MiB/s wr, 73 op/s 2024-09-10T09:42:32.016 INFO:tasks.workunit.client.0.smithi005.stdout: 1 191939 0.00 MB/sec execute 13 sec latency 16920.908 ms 2024-09-10T09:42:32.332 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:42:32.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:32.981+0000 7fbf1aa00640 1 -- 172.21.15.5:0/1265844295 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbf14102360 msgr2=0x7fbf1410e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:42:32.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:32.981+0000 7fbf1aa00640 1 --2- 172.21.15.5:0/1265844295 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbf14102360 0x7fbf1410e890 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fbf0c009cd0 tx=0x7fbf0c02a910 comp rx=0 tx=0).stop 2024-09-10T09:42:32.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:32.981+0000 7fbf1aa00640 1 -- 172.21.15.5:0/1265844295 shutdown_connections 2024-09-10T09:42:32.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:32.981+0000 7fbf1aa00640 1 --2- 172.21.15.5:0/1265844295 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbf14102360 0x7fbf1410e890 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:32.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:32.981+0000 7fbf1aa00640 1 --2- 172.21.15.5:0/1265844295 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbf141019c0 0x7fbf14101e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:32.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:32.981+0000 7fbf1aa00640 1 --2- 172.21.15.5:0/1265844295 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbf141079c0 0x7fbf14107da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:32.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:32.981+0000 7fbf1aa00640 1 -- 172.21.15.5:0/1265844295 >> 172.21.15.5:0/1265844295 conn(0x7fbf140fd740 msgr2=0x7fbf140ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:42:32.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:32.981+0000 7fbf1aa00640 1 -- 172.21.15.5:0/1265844295 shutdown_connections 2024-09-10T09:42:32.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:32.981+0000 7fbf1aa00640 1 -- 172.21.15.5:0/1265844295 wait complete. 2024-09-10T09:42:32.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:32.981+0000 7fbf1aa00640 1 Processor -- start 2024-09-10T09:42:32.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:32.981+0000 7fbf1aa00640 1 -- start start 2024-09-10T09:42:32.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:32.981+0000 7fbf1aa00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbf141019c0 0x7fbf1419a510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:42:32.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:32.981+0000 7fbf1aa00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbf14102360 0x7fbf1419aa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:42:32.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:32.981+0000 7fbf1aa00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbf141079c0 0x7fbf1419ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:42:32.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:32.981+0000 7fbf1aa00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fbf14111e00 con 0x7fbf141019c0 2024-09-10T09:42:32.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:32.981+0000 7fbf1aa00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fbf14111c80 con 0x7fbf14102360 2024-09-10T09:42:32.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:32.981+0000 7fbf1aa00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fbf14111f80 con 0x7fbf141079c0 2024-09-10T09:42:32.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:32.981+0000 7fbf19600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbf141019c0 0x7fbf1419a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:42:32.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:32.981+0000 7fbf19600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbf141019c0 0x7fbf1419a510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46088/0 (socket says 172.21.15.5:46088) 2024-09-10T09:42:32.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:32.981+0000 7fbf19600640 1 -- 172.21.15.5:0/1324169272 learned_addr learned my addr 172.21.15.5:0/1324169272 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:42:32.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:32.981+0000 7fbf1a000640 1 --2- 172.21.15.5:0/1324169272 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbf141079c0 0x7fbf1419ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:42:32.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:32.981+0000 7fbf18c00640 1 --2- 172.21.15.5:0/1324169272 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbf14102360 0x7fbf1419aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:42:32.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:32.981+0000 7fbf19600640 1 -- 172.21.15.5:0/1324169272 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbf141079c0 msgr2=0x7fbf1419ede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:42:32.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:32.981+0000 7fbf19600640 1 --2- 172.21.15.5:0/1324169272 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbf141079c0 0x7fbf1419ede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:32.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:32.981+0000 7fbf19600640 1 -- 172.21.15.5:0/1324169272 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbf14102360 msgr2=0x7fbf1419aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:42:32.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:32.981+0000 7fbf19600640 1 --2- 172.21.15.5:0/1324169272 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbf14102360 0x7fbf1419aa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:32.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:32.981+0000 7fbf19600640 1 -- 172.21.15.5:0/1324169272 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf1419f4c0 con 0x7fbf141019c0 2024-09-10T09:42:32.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:32.981+0000 7fbf18c00640 1 --2- 172.21.15.5:0/1324169272 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbf14102360 0x7fbf1419aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:42:32.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:32.981+0000 7fbf19600640 1 --2- 172.21.15.5:0/1324169272 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbf141019c0 0x7fbf1419a510 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fbf0800ea60 tx=0x7fbf0800ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:42:32.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:32.987+0000 7fbf12000640 1 -- 172.21.15.5:0/1324169272 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf0800cde0 con 0x7fbf141019c0 2024-09-10T09:42:32.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:32.987+0000 7fbf12000640 1 -- 172.21.15.5:0/1324169272 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbf080035f0 con 0x7fbf141019c0 2024-09-10T09:42:32.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:32.987+0000 7fbf1aa00640 1 -- 172.21.15.5:0/1324169272 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf1419f7b0 con 0x7fbf141019c0 2024-09-10T09:42:32.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:32.987+0000 7fbf1aa00640 1 -- 172.21.15.5:0/1324169272 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf141a7090 con 0x7fbf141019c0 2024-09-10T09:42:32.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:32.987+0000 7fbf12000640 1 -- 172.21.15.5:0/1324169272 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf08010670 con 0x7fbf141019c0 2024-09-10T09:42:32.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:32.990+0000 7fbf12000640 1 -- 172.21.15.5:0/1324169272 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7fbf08004000 con 0x7fbf141019c0 2024-09-10T09:42:32.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:32.990+0000 7fbf1ea8f740 1 Processor -- start 2024-09-10T09:42:32.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:32.991+0000 7fbf1ea8f740 1 -- start start 2024-09-10T09:42:32.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:32.991+0000 7fbf1ea8f740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56091ae88e40 0x56091ae89220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:42:32.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:32.991+0000 7fbf1ea8f740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56091ae89760 0x56091ae93ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:42:32.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:32.991+0000 7fbf1ea8f740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56091ae94020 0x56091ae96410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:42:32.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:32.991+0000 7fbf1ea8f740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x56091adef3e0 con 0x56091ae89760 2024-09-10T09:42:32.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:32.991+0000 7fbf1ea8f740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x56091ad96d80 con 0x56091ae88e40 2024-09-10T09:42:32.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:32.991+0000 7fbf1ea8f740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x56091adeb3a0 con 0x56091ae94020 2024-09-10T09:42:32.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:32.991+0000 7fbf1a000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56091ae88e40 0x56091ae89220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:42:32.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:32.991+0000 7fbf19600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56091ae89760 0x56091ae93ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:42:32.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:32.991+0000 7fbf1a000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56091ae88e40 0x56091ae89220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:34416/0 (socket says 172.21.15.5:34416) 2024-09-10T09:42:32.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:32.991+0000 7fbf1a000640 1 -- 172.21.15.5:0/1022580043 learned_addr learned my addr 172.21.15.5:0/1022580043 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:42:32.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:32.991+0000 7fbf18c00640 1 --2- 172.21.15.5:0/1022580043 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56091ae94020 0x56091ae96410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:42:32.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:32.991+0000 7fbf1a000640 1 -- 172.21.15.5:0/1022580043 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56091ae94020 msgr2=0x56091ae96410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:42:32.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:32.991+0000 7fbf1a000640 1 --2- 172.21.15.5:0/1022580043 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56091ae94020 0x56091ae96410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:33.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:32.991+0000 7fbf1a000640 1 -- 172.21.15.5:0/1022580043 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56091ae89760 msgr2=0x56091ae93ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:42:33.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:32.991+0000 7fbf1a000640 1 --2- 172.21.15.5:0/1022580043 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56091ae89760 0x56091ae93ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:33.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:32.991+0000 7fbf1a000640 1 -- 172.21.15.5:0/1022580043 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56091adb5ae0 con 0x56091ae88e40 2024-09-10T09:42:33.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:32.991+0000 7fbf12000640 1 --2- 172.21.15.5:0/1324169272 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fbefc0a2b80 0x7fbefc0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:42:33.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:32.991+0000 7fbf1a000640 1 --2- 172.21.15.5:0/1022580043 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56091ae88e40 0x56091ae89220 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fbf0c035000 tx=0x7fbf0c02f440 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4bcefe9c56210319 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:42:33.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:32.991+0000 7fbf12000640 1 -- 172.21.15.5:0/1324169272 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fbf080cf730 con 0x7fbf141019c0 2024-09-10T09:42:33.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:32.991+0000 7fbf19600640 1 --2- 172.21.15.5:0/1324169272 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fbefc0a2b80 0x7fbefc0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:42:33.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:32.991+0000 7fbf06600640 1 -- 172.21.15.5:0/1022580043 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf0c031d80 con 0x56091ae88e40 2024-09-10T09:42:33.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:32.991+0000 7fbf06600640 1 -- 172.21.15.5:0/1022580043 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbf0c036030 con 0x56091ae88e40 2024-09-10T09:42:33.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:32.991+0000 7fbf06600640 1 -- 172.21.15.5:0/1022580043 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf0c0042a0 con 0x56091ae88e40 2024-09-10T09:42:33.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:32.991+0000 7fbf1ea8f740 1 -- 172.21.15.5:0/1022580043 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56091ae88e40 msgr2=0x56091ae89220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:42:33.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:32.991+0000 7fbf1ea8f740 1 --2- 172.21.15.5:0/1022580043 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56091ae88e40 0x56091ae89220 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fbf0c035000 tx=0x7fbf0c02f440 comp rx=0 tx=0).stop 2024-09-10T09:42:33.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:32.991+0000 7fbf19600640 1 --2- 172.21.15.5:0/1324169272 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fbefc0a2b80 0x7fbefc0a5040 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fbf080a4280 tx=0x7fbf08009490 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:42:33.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:32.991+0000 7fbf1ea8f740 1 -- 172.21.15.5:0/1022580043 shutdown_connections 2024-09-10T09:42:33.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:32.991+0000 7fbf1ea8f740 1 --2- 172.21.15.5:0/1022580043 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56091ae94020 0x56091ae96410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:33.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:32.991+0000 7fbf1ea8f740 1 --2- 172.21.15.5:0/1022580043 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56091ae89760 0x56091ae93ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:33.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:32.991+0000 7fbf1ea8f740 1 --2- 172.21.15.5:0/1022580043 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56091ae88e40 0x56091ae89220 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:33.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:32.991+0000 7fbf1ea8f740 1 -- 172.21.15.5:0/1022580043 >> 172.21.15.5:0/1022580043 conn(0x56091ae853d0 msgr2=0x56091ae857b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:42:33.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:32.991+0000 7fbf1ea8f740 1 -- 172.21.15.5:0/1022580043 shutdown_connections 2024-09-10T09:42:33.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:32.991+0000 7fbf1ea8f740 1 -- 172.21.15.5:0/1022580043 wait complete. 2024-09-10T09:42:33.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:32.991+0000 7fbf1ea8f740 1 -- start start 2024-09-10T09:42:33.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:32.991+0000 7fbf1ea8f740 1 Processor -- start 2024-09-10T09:42:33.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:32.991+0000 7fbf1ea8f740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56091ae88e40 0x56091ae90ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:42:33.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:32.991+0000 7fbf1ea8f740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56091ae89760 0x56091ae913e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:42:33.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:32.991+0000 7fbf1ea8f740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56091ae94020 0x56091ae91920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:42:33.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:32.991+0000 7fbf1ea8f740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x56091ae96bf0 con 0x56091ae88e40 2024-09-10T09:42:33.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:32.991+0000 7fbf1ea8f740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x56091adaa0f0 con 0x56091ae89760 2024-09-10T09:42:33.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:32.991+0000 7fbf1ea8f740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x56091ae96d70 con 0x56091ae94020 2024-09-10T09:42:33.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:32.991+0000 7fbf1ea8f740 10 client.0 _ll_register_callbacks cb 0x56091ad91bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:42:33.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:32.991+0000 7fbf1a000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56091ae88e40 0x56091ae90ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:42:33.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:32.991+0000 7fbf1a000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56091ae88e40 0x56091ae90ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46110/0 (socket says 172.21.15.5:46110) 2024-09-10T09:42:33.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:32.991+0000 7fbf18c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56091ae89760 0x56091ae913e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:42:33.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:32.991+0000 7fbf1a000640 1 -- 172.21.15.5:0/1180186988 learned_addr learned my addr 172.21.15.5:0/1180186988 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:42:33.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:32.991+0000 7fbf1a000640 1 --2- 172.21.15.5:0/1180186988 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56091ae94020 0x56091ae91920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:42:33.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:32.991+0000 7fbf18c00640 1 -- 172.21.15.5:0/1180186988 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56091ae94020 msgr2=0x56091ae91920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:42:33.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:32.991+0000 7fbf18c00640 1 --2- 172.21.15.5:0/1180186988 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56091ae94020 0x56091ae91920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:33.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:32.991+0000 7fbf18c00640 1 -- 172.21.15.5:0/1180186988 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56091ae88e40 msgr2=0x56091ae90ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:42:33.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:32.991+0000 7fbf18c00640 1 --2- 172.21.15.5:0/1180186988 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56091ae88e40 0x56091ae90ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:33.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:32.991+0000 7fbf18c00640 1 -- 172.21.15.5:0/1180186988 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56091ab603c0 con 0x56091ae89760 2024-09-10T09:42:33.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:32.991+0000 7fbf18c00640 1 --2- 172.21.15.5:0/1180186988 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56091ae89760 0x56091ae913e0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fbf00006fd0 tx=0x7fbf0000f5b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6e2ffad6673bfec6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:42:33.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:32.991+0000 7fbefaa00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-09-10T09:42:33.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:32.997+0000 7fbefaa00640 1 -- 172.21.15.5:0/1180186988 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf00014660 con 0x56091ae89760 2024-09-10T09:42:33.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:32.997+0000 7fbefaa00640 1 -- 172.21.15.5:0/1180186988 <== mon.2 v2:172.21.15.102:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fbf00014800 con 0x56091ae89760 2024-09-10T09:42:33.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:32.997+0000 7fbf1ea8f740 1 -- 172.21.15.5:0/1180186988 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbf0c036030 con 0x56091ae89760 2024-09-10T09:42:33.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:32.997+0000 7fbefaa00640 1 -- 172.21.15.5:0/1180186988 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fbf00024050 con 0x56091ae89760 2024-09-10T09:42:33.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:32.997+0000 7fbf1ea8f740 10 client.26192 fetch_fsmap learned FSMap version 33 2024-09-10T09:42:33.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:32.997+0000 7fbf1ea8f740 1 -- 172.21.15.5:0/1180186988 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x56091aea2ce0 con 0x56091ae89760 2024-09-10T09:42:33.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:32.997+0000 7fbefaa00640 1 -- 172.21.15.5:0/1180186988 <== mon.2 v2:172.21.15.102:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbf0c036030 con 0x56091ae89760 2024-09-10T09:42:33.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:32.997+0000 7fbefaa00640 1 -- 172.21.15.5:0/1180186988 <== mon.2 v2:172.21.15.102:3300/0 5 ==== fsmap(e 33) ==== 8374+0+0 (secure 0 0 0) 0x7fbf00015b10 con 0x56091ae89760 2024-09-10T09:42:33.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:32.997+0000 7fbf1ea8f740 10 client.26192 fetch_fsmap finished waiting for FSMap version 33 2024-09-10T09:42:33.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:32.997+0000 7fbf1ea8f740 10 client.26192 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-10T09:42:33.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:32.997+0000 7fbf1ea8f740 20 client.26192 populate_metadata read hostname 'smithi005' 2024-09-10T09:42:33.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:32.997+0000 7fbf1ea8f740 1 --2- 172.21.15.5:0/1180186988 >> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] conn(0x56091aea4240 0x56091aea6690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:42:33.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:32.997+0000 7fbf1ea8f740 4 client.26192 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:42:33.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:32.997+0000 7fbf1ea8f740 1 -- 172.21.15.5:0/1180186988 --> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56091adfc950 con 0x56091aea4240 2024-09-10T09:42:33.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:32.997+0000 7fbf1a000640 1 --2- 172.21.15.5:0/1180186988 >> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] conn(0x56091aea4240 0x56091aea6690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:42:33.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:32.997+0000 7fbf1a000640 1 --2- 172.21.15.5:0/1180186988 >> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] conn(0x56091aea4240 0x56091aea6690 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=394f28cf12c476eb server_cookie=6339cbab41f056c in_seq=0 out_seq=0 2024-09-10T09:42:33.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:32.997+0000 7fbefaa00640 10 client.26192 ms_handle_connect on v2:172.21.15.159:6836/620627639 2024-09-10T09:42:33.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:32.997+0000 7fbefaa00640 1 -- 172.21.15.5:0/1180186988 <== mds.2 v2:172.21.15.159:6836/620627639 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56091adfc950 con 0x56091aea4240 2024-09-10T09:42:33.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:32.997+0000 7fbefaa00640 10 client.26192 handle_command_reply: tid=0 2024-09-10T09:42:33.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:32.997+0000 7fbf1ea8f740 1 client.26192 shutdown 2024-09-10T09:42:33.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:32.997+0000 7fbf1ea8f740 1 -- 172.21.15.5:0/1180186988 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56091ae89760 msgr2=0x56091ae913e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:42:33.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:32.997+0000 7fbf1ea8f740 1 --2- 172.21.15.5:0/1180186988 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56091ae89760 0x56091ae913e0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fbf00006fd0 tx=0x7fbf0000f5b0 comp rx=0 tx=0).stop 2024-09-10T09:42:33.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:32.997+0000 7fbf1ea8f740 1 -- 172.21.15.5:0/1180186988 shutdown_connections 2024-09-10T09:42:33.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:32.997+0000 7fbf1ea8f740 1 --2- 172.21.15.5:0/1180186988 >> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] conn(0x56091aea4240 0x56091aea6690 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:33.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:32.997+0000 7fbf1ea8f740 1 --2- 172.21.15.5:0/1180186988 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56091ae94020 0x56091ae91920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:33.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:32.997+0000 7fbf1ea8f740 1 --2- 172.21.15.5:0/1180186988 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56091ae89760 0x56091ae913e0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:33.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:32.997+0000 7fbf1ea8f740 1 --2- 172.21.15.5:0/1180186988 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56091ae88e40 0x56091ae90ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:33.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:32.997+0000 7fbefaa00640 0 client.26192 ms_handle_reset on v2:172.21.15.159:6836/620627639 2024-09-10T09:42:33.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:32.997+0000 7fbf1ea8f740 1 -- 172.21.15.5:0/1180186988 >> 172.21.15.5:0/1180186988 conn(0x56091ae853d0 msgr2=0x56091ae8b100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:42:33.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:32.997+0000 7fbf1ea8f740 1 -- 172.21.15.5:0/1180186988 shutdown_connections 2024-09-10T09:42:33.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:32.997+0000 7fbf1ea8f740 1 -- 172.21.15.5:0/1180186988 wait complete. 2024-09-10T09:42:33.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:32.997+0000 7fbf1ea8f740 20 client.26192 trim_cache size 0 max 16384 2024-09-10T09:42:33.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:33.013+0000 7fbf1ea8f740 1 Processor -- start 2024-09-10T09:42:33.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:33.013+0000 7fbf1ea8f740 1 -- start start 2024-09-10T09:42:33.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:33.013+0000 7fbf1ea8f740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56091ae88e40 0x56091ae8b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:42:33.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:33.013+0000 7fbf1ea8f740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56091ae89760 0x56091aeabfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:42:33.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:33.013+0000 7fbf1ea8f740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56091ae94020 0x56091aeac4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:42:33.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:33.013+0000 7fbf1ea8f740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fbef400fcd0 con 0x56091ae94020 2024-09-10T09:42:33.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:33.013+0000 7fbf1ea8f740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fbef4011840 con 0x56091ae89760 2024-09-10T09:42:33.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:33.013+0000 7fbf1ea8f740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fbef40103a0 con 0x56091ae88e40 2024-09-10T09:42:33.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:33.013+0000 7fbf18c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56091ae89760 0x56091aeabfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:42:33.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:33.013+0000 7fbf1a000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56091ae94020 0x56091aeac4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:42:33.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:33.013+0000 7fbf18c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56091ae89760 0x56091aeabfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:34438/0 (socket says 172.21.15.5:34438) 2024-09-10T09:42:33.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:33.013+0000 7fbf18c00640 1 -- 172.21.15.5:0/1636538441 learned_addr learned my addr 172.21.15.5:0/1636538441 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:42:33.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:33.013+0000 7fbf1a000640 1 --2- 172.21.15.5:0/1636538441 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56091ae88e40 0x56091ae8b8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:42:33.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:33.013+0000 7fbf18c00640 1 -- 172.21.15.5:0/1636538441 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56091ae88e40 msgr2=0x56091ae8b8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:42:33.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:33.013+0000 7fbf18c00640 1 --2- 172.21.15.5:0/1636538441 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56091ae88e40 0x56091ae8b8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:33.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:33.013+0000 7fbf18c00640 1 -- 172.21.15.5:0/1636538441 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56091ae94020 msgr2=0x56091aeac4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:42:33.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:33.013+0000 7fbf18c00640 1 --2- 172.21.15.5:0/1636538441 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56091ae94020 0x56091aeac4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:33.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:33.013+0000 7fbf18c00640 1 -- 172.21.15.5:0/1636538441 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56091aea6d10 con 0x56091ae89760 2024-09-10T09:42:33.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:33.013+0000 7fbf1a000640 1 --2- 172.21.15.5:0/1636538441 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56091ae94020 0x56091aeac4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:42:33.017 INFO:tasks.workunit.client.0.smithi005.stdout: 1 191939 0.00 MB/sec execute 14 sec latency 17920.958 ms 2024-09-10T09:42:33.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:33.013+0000 7fbf18c00640 1 --2- 172.21.15.5:0/1636538441 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56091ae89760 0x56091aeabfa0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fbf00006dc0 tx=0x7fbf0000fb30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a64f28f814db7778 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:42:33.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:33.013+0000 7fbf05c00640 1 -- 172.21.15.5:0/1636538441 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf00004240 con 0x56091ae89760 2024-09-10T09:42:33.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:33.013+0000 7fbf05c00640 1 -- 172.21.15.5:0/1636538441 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbf0002d070 con 0x56091ae89760 2024-09-10T09:42:33.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:33.013+0000 7fbf05c00640 1 -- 172.21.15.5:0/1636538441 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf0001d800 con 0x56091ae89760 2024-09-10T09:42:33.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:33.013+0000 7fbf1ea8f740 1 -- 172.21.15.5:0/1636538441 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56091ae89760 msgr2=0x56091aeabfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:42:33.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:33.013+0000 7fbf1ea8f740 1 --2- 172.21.15.5:0/1636538441 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56091ae89760 0x56091aeabfa0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fbf00006dc0 tx=0x7fbf0000fb30 comp rx=0 tx=0).stop 2024-09-10T09:42:33.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:33.013+0000 7fbf1ea8f740 1 -- 172.21.15.5:0/1636538441 shutdown_connections 2024-09-10T09:42:33.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:33.013+0000 7fbf1ea8f740 1 --2- 172.21.15.5:0/1636538441 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56091ae94020 0x56091aeac4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:33.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:33.013+0000 7fbf1ea8f740 1 --2- 172.21.15.5:0/1636538441 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56091ae89760 0x56091aeabfa0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:33.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:33.013+0000 7fbf1ea8f740 1 --2- 172.21.15.5:0/1636538441 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56091ae88e40 0x56091ae8b8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:33.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:33.013+0000 7fbf1ea8f740 1 -- 172.21.15.5:0/1636538441 >> 172.21.15.5:0/1636538441 conn(0x56091ae853d0 msgr2=0x56091ae92f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:42:33.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:33.013+0000 7fbf1ea8f740 1 -- 172.21.15.5:0/1636538441 shutdown_connections 2024-09-10T09:42:33.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:33.013+0000 7fbf1ea8f740 1 -- 172.21.15.5:0/1636538441 wait complete. 2024-09-10T09:42:33.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:33.013+0000 7fbf1ea8f740 1 -- start start 2024-09-10T09:42:33.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:33.013+0000 7fbf1ea8f740 1 Processor -- start 2024-09-10T09:42:33.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:33.013+0000 7fbf1ea8f740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56091ae88e40 0x56091ae8df40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:42:33.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:33.013+0000 7fbf1ea8f740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56091ae89760 0x56091ae8e480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:42:33.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:33.013+0000 7fbf1ea8f740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56091ae94020 0x56091ae8e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:42:33.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:33.013+0000 7fbf1ea8f740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fbef4011140 con 0x56091ae89760 2024-09-10T09:42:33.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:33.013+0000 7fbf1ea8f740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fbef4010a70 con 0x56091ae94020 2024-09-10T09:42:33.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:33.013+0000 7fbf1ea8f740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fbef400dcb0 con 0x56091ae88e40 2024-09-10T09:42:33.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:33.013+0000 7fbf1ea8f740 10 client.0 _ll_register_callbacks cb 0x7fbf0000d240 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:42:33.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:33.013+0000 7fbf18c00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56091ae89760 0x56091ae8e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:42:33.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:33.013+0000 7fbf1a000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56091ae88e40 0x56091ae8df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:42:33.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:33.013+0000 7fbf1a000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56091ae88e40 0x56091ae8df40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:40296/0 (socket says 172.21.15.5:40296) 2024-09-10T09:42:33.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:33.013+0000 7fbf18c00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56091ae89760 0x56091ae8e480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46122/0 (socket says 172.21.15.5:46122) 2024-09-10T09:42:33.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:33.013+0000 7fbf1a000640 1 -- 172.21.15.5:0/3044891015 learned_addr learned my addr 172.21.15.5:0/3044891015 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:42:33.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:33.013+0000 7fbf1a000640 1 --2- 172.21.15.5:0/3044891015 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56091ae94020 0x56091ae8e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:42:33.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:33.013+0000 7fbf1a000640 1 -- 172.21.15.5:0/3044891015 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56091ae94020 msgr2=0x56091ae8e9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:42:33.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:33.013+0000 7fbf1a000640 1 --2- 172.21.15.5:0/3044891015 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56091ae94020 0x56091ae8e9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:33.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:33.013+0000 7fbf1a000640 1 -- 172.21.15.5:0/3044891015 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56091ae89760 msgr2=0x56091ae8e480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:42:33.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:33.013+0000 7fbf1a000640 1 --2- 172.21.15.5:0/3044891015 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56091ae89760 0x56091ae8e480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:33.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:33.013+0000 7fbf1a000640 1 -- 172.21.15.5:0/3044891015 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56091ae86060 con 0x56091ae88e40 2024-09-10T09:42:33.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:33.013+0000 7fbf18c00640 1 --2- 172.21.15.5:0/3044891015 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56091ae89760 0x56091ae8e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:42:33.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:33.013+0000 7fbf1a000640 1 --2- 172.21.15.5:0/3044891015 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56091ae88e40 0x56091ae8df40 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fbf0c031820 tx=0x7fbf0c02f440 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d75dbdc739395f92 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:42:33.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:33.013+0000 7fbefaa00640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-09-10T09:42:33.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:33.019+0000 7fbefaa00640 1 -- 172.21.15.5:0/3044891015 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf0c04a040 con 0x56091ae88e40 2024-09-10T09:42:33.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:33.019+0000 7fbf1ea8f740 1 -- 172.21.15.5:0/3044891015 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbf0002d070 con 0x56091ae88e40 2024-09-10T09:42:33.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:33.019+0000 7fbefaa00640 1 -- 172.21.15.5:0/3044891015 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fbf0c02fb40 con 0x56091ae88e40 2024-09-10T09:42:33.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:33.019+0000 7fbefaa00640 1 -- 172.21.15.5:0/3044891015 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fbf0c02fd40 con 0x56091ae88e40 2024-09-10T09:42:33.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:33.019+0000 7fbf1ea8f740 10 client.26221 fetch_fsmap learned FSMap version 33 2024-09-10T09:42:33.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:33.019+0000 7fbf1ea8f740 1 -- 172.21.15.5:0/3044891015 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x56091ae85200 con 0x56091ae88e40 2024-09-10T09:42:33.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:33.019+0000 7fbefaa00640 1 -- 172.21.15.5:0/3044891015 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbf0002d070 con 0x56091ae88e40 2024-09-10T09:42:33.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:33.019+0000 7fbefaa00640 1 -- 172.21.15.5:0/3044891015 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 33) ==== 8374+0+0 (secure 0 0 0) 0x7fbf0c043070 con 0x56091ae88e40 2024-09-10T09:42:33.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:33.019+0000 7fbf1ea8f740 10 client.26221 fetch_fsmap finished waiting for FSMap version 33 2024-09-10T09:42:33.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:33.019+0000 7fbf1ea8f740 10 client.26221 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-10T09:42:33.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:33.019+0000 7fbf1ea8f740 20 client.26221 populate_metadata read hostname 'smithi005' 2024-09-10T09:42:33.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:33.019+0000 7fbf1ea8f740 1 --2- 172.21.15.5:0/3044891015 >> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] conn(0x56091af0f250 0x56091af11640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:42:33.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:33.019+0000 7fbf1ea8f740 4 client.26221 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T09:42:33.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:33.019+0000 7fbf1ea8f740 1 -- 172.21.15.5:0/3044891015 --> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56091adf6e60 con 0x56091af0f250 2024-09-10T09:42:33.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:33.019+0000 7fbf18c00640 1 --2- 172.21.15.5:0/3044891015 >> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] conn(0x56091af0f250 0x56091af11640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:42:33.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:33.019+0000 7fbf18c00640 1 --2- 172.21.15.5:0/3044891015 >> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] conn(0x56091af0f250 0x56091af11640 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c110d5c9ec473f45 server_cookie=ba5817d9f1ae3f59 in_seq=0 out_seq=0 2024-09-10T09:42:33.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:33.019+0000 7fbefaa00640 10 client.26221 ms_handle_connect on v2:172.21.15.159:6836/620627639 2024-09-10T09:42:33.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:33.019+0000 7fbefaa00640 1 -- 172.21.15.5:0/3044891015 <== mds.2 v2:172.21.15.159:6836/620627639 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56091adf6e60 con 0x56091af0f250 2024-09-10T09:42:33.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:33.019+0000 7fbefaa00640 10 client.26221 handle_command_reply: tid=0 2024-09-10T09:42:33.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:33.019+0000 7fbf1ea8f740 1 client.26221 shutdown 2024-09-10T09:42:33.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:33.019+0000 7fbf1ea8f740 1 -- 172.21.15.5:0/3044891015 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56091ae88e40 msgr2=0x56091ae8df40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:42:33.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:33.019+0000 7fbf1ea8f740 1 --2- 172.21.15.5:0/3044891015 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56091ae88e40 0x56091ae8df40 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fbf0c031820 tx=0x7fbf0c02f440 comp rx=0 tx=0).stop 2024-09-10T09:42:33.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:33.019+0000 7fbf1ea8f740 1 -- 172.21.15.5:0/3044891015 shutdown_connections 2024-09-10T09:42:33.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:33.019+0000 7fbf1ea8f740 1 --2- 172.21.15.5:0/3044891015 >> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] conn(0x56091af0f250 0x56091af11640 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:33.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:33.019+0000 7fbf1ea8f740 1 --2- 172.21.15.5:0/3044891015 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56091ae94020 0x56091ae8e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:33.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:33.019+0000 7fbf1ea8f740 1 --2- 172.21.15.5:0/3044891015 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56091ae89760 0x56091ae8e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:33.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:33.019+0000 7fbefaa00640 0 client.26221 ms_handle_reset on v2:172.21.15.159:6836/620627639 2024-09-10T09:42:33.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:33.019+0000 7fbf1ea8f740 1 --2- 172.21.15.5:0/3044891015 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56091ae88e40 0x56091ae8df40 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:33.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:33.019+0000 7fbf1ea8f740 1 -- 172.21.15.5:0/3044891015 >> 172.21.15.5:0/3044891015 conn(0x56091ae853d0 msgr2=0x56091ae8f370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:42:33.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:33.019+0000 7fbf1ea8f740 1 -- 172.21.15.5:0/3044891015 shutdown_connections 2024-09-10T09:42:33.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:33.019+0000 7fbf1ea8f740 1 -- 172.21.15.5:0/3044891015 wait complete. 2024-09-10T09:42:33.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:33.019+0000 7fbf1ea8f740 20 client.26221 trim_cache size 0 max 16384 2024-09-10T09:42:33.027 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-10T09:42:33.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:33.019+0000 7fbf1aa00640 1 -- 172.21.15.5:0/1324169272 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fbefc0a2b80 msgr2=0x7fbefc0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:42:33.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:33.019+0000 7fbf1aa00640 1 --2- 172.21.15.5:0/1324169272 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fbefc0a2b80 0x7fbefc0a5040 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fbf080a4280 tx=0x7fbf08009490 comp rx=0 tx=0).stop 2024-09-10T09:42:33.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:33.019+0000 7fbf1aa00640 1 -- 172.21.15.5:0/1324169272 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbf141019c0 msgr2=0x7fbf1419a510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:42:33.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:33.019+0000 7fbf1aa00640 1 --2- 172.21.15.5:0/1324169272 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbf141019c0 0x7fbf1419a510 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fbf0800ea60 tx=0x7fbf0800ef20 comp rx=0 tx=0).stop 2024-09-10T09:42:33.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:33.019+0000 7fbf1aa00640 1 -- 172.21.15.5:0/1324169272 shutdown_connections 2024-09-10T09:42:33.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:33.019+0000 7fbf1aa00640 1 --2- 172.21.15.5:0/1324169272 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fbefc0a2b80 0x7fbefc0a5040 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:33.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:33.019+0000 7fbf1aa00640 1 --2- 172.21.15.5:0/1324169272 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbf141079c0 0x7fbf1419ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:33.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:33.019+0000 7fbf1aa00640 1 --2- 172.21.15.5:0/1324169272 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbf14102360 0x7fbf1419aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:33.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:33.019+0000 7fbf1aa00640 1 --2- 172.21.15.5:0/1324169272 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbf141019c0 0x7fbf1419a510 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:33.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:33.019+0000 7fbf1aa00640 1 -- 172.21.15.5:0/1324169272 >> 172.21.15.5:0/1324169272 conn(0x7fbf140fd740 msgr2=0x7fbf1410d090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:42:33.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:33.019+0000 7fbf1aa00640 1 -- 172.21.15.5:0/1324169272 shutdown_connections 2024-09-10T09:42:33.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:33.019+0000 7fbf1aa00640 1 -- 172.21.15.5:0/1324169272 wait complete. 2024-09-10T09:42:33.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:42:33 smithi159 ceph-mon[35513]: pgmap v519: 129 pgs: 129 active+clean; 99 MiB data, 845 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 1.3 MiB/s wr, 54 op/s 2024-09-10T09:42:33.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:42:33 smithi102 ceph-mon[35442]: pgmap v519: 129 pgs: 129 active+clean; 99 MiB data, 845 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 1.3 MiB/s wr, 54 op/s 2024-09-10T09:42:34.016 INFO:tasks.workunit.client.0.smithi005.stdout: 1 191939 0.00 MB/sec execute 15 sec latency 18921.084 ms 2024-09-10T09:42:34.041 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:42:33 smithi005 ceph-mon[30350]: pgmap v519: 129 pgs: 129 active+clean; 99 MiB data, 845 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 1.3 MiB/s wr, 54 op/s 2024-09-10T09:42:34.078 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-10T09:42:34.131 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T09:42:34.474 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:42:34.523 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:42:35.016 INFO:tasks.workunit.client.0.smithi005.stdout: 1 191939 0.00 MB/sec execute 16 sec latency 19921.171 ms 2024-09-10T09:42:35.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:35.111+0000 7ff062200640 1 -- 172.21.15.5:0/1485478165 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff05c1049c0 msgr2=0x7ff05c108a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:42:35.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:35.111+0000 7ff062200640 1 --2- 172.21.15.5:0/1485478165 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff05c1049c0 0x7ff05c108a10 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7ff044009900 tx=0x7ff04402a3f0 comp rx=0 tx=0).stop 2024-09-10T09:42:35.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:35.111+0000 7ff062200640 1 -- 172.21.15.5:0/1485478165 shutdown_connections 2024-09-10T09:42:35.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:35.111+0000 7ff062200640 1 --2- 172.21.15.5:0/1485478165 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff05c109140 0x7ff05c110cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:35.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:35.111+0000 7ff062200640 1 --2- 172.21.15.5:0/1485478165 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff05c1049c0 0x7ff05c108a10 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:35.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:35.111+0000 7ff062200640 1 --2- 172.21.15.5:0/1485478165 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff05c104010 0x7ff05c1043f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:35.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:35.111+0000 7ff062200640 1 -- 172.21.15.5:0/1485478165 >> 172.21.15.5:0/1485478165 conn(0x7ff05c0ffa50 msgr2=0x7ff05c101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:42:35.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:35.111+0000 7ff062200640 1 -- 172.21.15.5:0/1485478165 shutdown_connections 2024-09-10T09:42:35.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:35.111+0000 7ff062200640 1 -- 172.21.15.5:0/1485478165 wait complete. 2024-09-10T09:42:35.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:35.113+0000 7ff062200640 1 Processor -- start 2024-09-10T09:42:35.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:35.113+0000 7ff062200640 1 -- start start 2024-09-10T09:42:35.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:35.113+0000 7ff062200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff05c104010 0x7ff05c19a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:42:35.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:35.113+0000 7ff062200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff05c1049c0 0x7ff05c19ac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:42:35.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:35.113+0000 7ff062200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff05c109140 0x7ff05c19efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:42:35.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:35.113+0000 7ff062200640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff05c1141b0 con 0x7ff05c104010 2024-09-10T09:42:35.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:35.113+0000 7ff062200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7ff05c114030 con 0x7ff05c109140 2024-09-10T09:42:35.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:35.113+0000 7ff062200640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7ff05c114330 con 0x7ff05c1049c0 2024-09-10T09:42:35.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:35.114+0000 7ff05b400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff05c104010 0x7ff05c19a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:42:35.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:35.114+0000 7ff052a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff05c1049c0 0x7ff05c19ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:42:35.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:35.114+0000 7ff052a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff05c1049c0 0x7ff05c19ac50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:40320/0 (socket says 172.21.15.5:40320) 2024-09-10T09:42:35.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:35.114+0000 7ff052a00640 1 -- 172.21.15.5:0/4176597564 learned_addr learned my addr 172.21.15.5:0/4176597564 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:42:35.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:35.114+0000 7ff05be00640 1 --2- 172.21.15.5:0/4176597564 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff05c109140 0x7ff05c19efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:42:35.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:35.114+0000 7ff052a00640 1 -- 172.21.15.5:0/4176597564 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff05c109140 msgr2=0x7ff05c19efe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:42:35.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:35.114+0000 7ff052a00640 1 --2- 172.21.15.5:0/4176597564 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff05c109140 0x7ff05c19efe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:35.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:35.114+0000 7ff052a00640 1 -- 172.21.15.5:0/4176597564 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff05c104010 msgr2=0x7ff05c19a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:42:35.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:35.114+0000 7ff052a00640 1 --2- 172.21.15.5:0/4176597564 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff05c104010 0x7ff05c19a710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:35.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:35.114+0000 7ff052a00640 1 -- 172.21.15.5:0/4176597564 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff05c19f6c0 con 0x7ff05c1049c0 2024-09-10T09:42:35.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:35.114+0000 7ff052a00640 1 --2- 172.21.15.5:0/4176597564 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff05c1049c0 0x7ff05c19ac50 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7ff044004770 tx=0x7ff044003c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:42:35.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:35.115+0000 7ff058c00640 1 -- 172.21.15.5:0/4176597564 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff04403f070 con 0x7ff05c1049c0 2024-09-10T09:42:35.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:35.115+0000 7ff062200640 1 -- 172.21.15.5:0/4176597564 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff05c19f950 con 0x7ff05c1049c0 2024-09-10T09:42:35.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:35.115+0000 7ff058c00640 1 -- 172.21.15.5:0/4176597564 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff044031ca0 con 0x7ff05c1049c0 2024-09-10T09:42:35.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:35.115+0000 7ff058c00640 1 -- 172.21.15.5:0/4176597564 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff044035030 con 0x7ff05c1049c0 2024-09-10T09:42:35.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:35.115+0000 7ff062200640 1 -- 172.21.15.5:0/4176597564 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff05c1a7290 con 0x7ff05c1049c0 2024-09-10T09:42:35.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:35.118+0000 7ff064e22740 1 Processor -- start 2024-09-10T09:42:35.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:35.118+0000 7ff064e22740 1 -- start start 2024-09-10T09:42:35.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:35.118+0000 7ff058c00640 1 -- 172.21.15.5:0/4176597564 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7ff044042070 con 0x7ff05c1049c0 2024-09-10T09:42:35.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:35.118+0000 7ff064e22740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5571c285fc70 0x5571c2860050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:42:35.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:35.118+0000 7ff064e22740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5571c2860590 0x5571c286a910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:42:35.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:35.118+0000 7ff064e22740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5571c286ae50 0x5571c286d240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:42:35.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:35.118+0000 7ff064e22740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5571c27c63e0 con 0x5571c286ae50 2024-09-10T09:42:35.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:35.118+0000 7ff064e22740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x5571c276dd80 con 0x5571c2860590 2024-09-10T09:42:35.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:35.119+0000 7ff064e22740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5571c27c23a0 con 0x5571c285fc70 2024-09-10T09:42:35.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:35.119+0000 7ff05b400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5571c2860590 0x5571c286a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:42:35.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:35.119+0000 7ff05b400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5571c2860590 0x5571c286a910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:34484/0 (socket says 172.21.15.5:34484) 2024-09-10T09:42:35.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:35.119+0000 7ff05b400640 1 -- 172.21.15.5:0/387059413 learned_addr learned my addr 172.21.15.5:0/387059413 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:42:35.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:35.119+0000 7ff052a00640 1 --2- 172.21.15.5:0/387059413 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5571c286ae50 0x5571c286d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:42:35.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:35.119+0000 7ff05be00640 1 --2- 172.21.15.5:0/387059413 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5571c285fc70 0x5571c2860050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:42:35.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:35.119+0000 7ff05be00640 1 -- 172.21.15.5:0/387059413 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5571c2860590 msgr2=0x5571c286a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:42:35.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:35.119+0000 7ff05be00640 1 --2- 172.21.15.5:0/387059413 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5571c2860590 0x5571c286a910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:35.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:35.119+0000 7ff05be00640 1 -- 172.21.15.5:0/387059413 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5571c286ae50 msgr2=0x5571c286d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:42:35.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:35.119+0000 7ff05be00640 1 --2- 172.21.15.5:0/387059413 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5571c286ae50 0x5571c286d240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:35.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:35.119+0000 7ff05be00640 1 -- 172.21.15.5:0/387059413 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5571c278cae0 con 0x5571c285fc70 2024-09-10T09:42:35.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:35.120+0000 7ff05be00640 1 --2- 172.21.15.5:0/387059413 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5571c285fc70 0x5571c2860050 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7ff054009f00 tx=0x7ff054011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7f2d5dbafa1593c0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:42:35.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:35.120+0000 7ff04be00640 1 -- 172.21.15.5:0/387059413 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff054004d20 con 0x5571c285fc70 2024-09-10T09:42:35.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:35.120+0000 7ff058c00640 1 --2- 172.21.15.5:0/4176597564 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7ff0400a2b80 0x7ff0400a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:42:35.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:35.120+0000 7ff04be00640 1 -- 172.21.15.5:0/387059413 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff054004050 con 0x5571c285fc70 2024-09-10T09:42:35.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:35.120+0000 7ff04be00640 1 -- 172.21.15.5:0/387059413 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff054005280 con 0x5571c285fc70 2024-09-10T09:42:35.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:35.120+0000 7ff058c00640 1 -- 172.21.15.5:0/4176597564 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7ff0440ef6f0 con 0x7ff05c1049c0 2024-09-10T09:42:35.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:35.121+0000 7ff05b400640 1 --2- 172.21.15.5:0/4176597564 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7ff0400a2b80 0x7ff0400a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:42:35.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:35.121+0000 7ff064e22740 1 -- 172.21.15.5:0/387059413 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5571c285fc70 msgr2=0x5571c2860050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:42:35.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:35.121+0000 7ff064e22740 1 --2- 172.21.15.5:0/387059413 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5571c285fc70 0x5571c2860050 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7ff054009f00 tx=0x7ff054011810 comp rx=0 tx=0).stop 2024-09-10T09:42:35.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:35.121+0000 7ff064e22740 1 -- 172.21.15.5:0/387059413 shutdown_connections 2024-09-10T09:42:35.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:35.121+0000 7ff064e22740 1 --2- 172.21.15.5:0/387059413 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5571c286ae50 0x5571c286d240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:35.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:35.121+0000 7ff064e22740 1 --2- 172.21.15.5:0/387059413 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5571c2860590 0x5571c286a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:35.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:35.121+0000 7ff064e22740 1 --2- 172.21.15.5:0/387059413 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5571c285fc70 0x5571c2860050 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:35.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:35.121+0000 7ff064e22740 1 -- 172.21.15.5:0/387059413 >> 172.21.15.5:0/387059413 conn(0x5571c285c200 msgr2=0x5571c285c5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:42:35.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:35.121+0000 7ff064e22740 1 -- 172.21.15.5:0/387059413 shutdown_connections 2024-09-10T09:42:35.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:35.121+0000 7ff05b400640 1 --2- 172.21.15.5:0/4176597564 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7ff0400a2b80 0x7ff0400a5040 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x5571c285d060 tx=0x7ff04c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:42:35.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:35.121+0000 7ff064e22740 1 -- 172.21.15.5:0/387059413 wait complete. 2024-09-10T09:42:35.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:35.122+0000 7ff064e22740 1 -- start start 2024-09-10T09:42:35.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:35.122+0000 7ff064e22740 1 Processor -- start 2024-09-10T09:42:35.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:35.122+0000 7ff064e22740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5571c285fc70 0x5571c2867d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:42:35.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:35.122+0000 7ff064e22740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5571c2860590 0x5571c2868250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:42:35.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:35.122+0000 7ff064e22740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5571c286ae50 0x5571c2868790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:42:35.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:35.122+0000 7ff064e22740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5571c286da20 con 0x5571c2860590 2024-09-10T09:42:35.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:35.122+0000 7ff064e22740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x5571c27810f0 con 0x5571c285fc70 2024-09-10T09:42:35.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:35.122+0000 7ff064e22740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5571c286dba0 con 0x5571c286ae50 2024-09-10T09:42:35.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:35.122+0000 7ff064e22740 10 client.0 _ll_register_callbacks cb 0x5571c2768bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:42:35.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:35.123+0000 7ff05b400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5571c285fc70 0x5571c2867d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:42:35.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:35.123+0000 7ff05b400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5571c285fc70 0x5571c2867d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:34488/0 (socket says 172.21.15.5:34488) 2024-09-10T09:42:35.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:35.123+0000 7ff05b400640 1 -- 172.21.15.5:0/2484266831 learned_addr learned my addr 172.21.15.5:0/2484266831 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:42:35.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:35.123+0000 7ff052a00640 1 --2- 172.21.15.5:0/2484266831 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5571c2860590 0x5571c2868250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:42:35.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:35.123+0000 7ff05be00640 1 --2- 172.21.15.5:0/2484266831 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5571c286ae50 0x5571c2868790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:42:35.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:35.123+0000 7ff05b400640 1 -- 172.21.15.5:0/2484266831 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5571c286ae50 msgr2=0x5571c2868790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:42:35.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:35.123+0000 7ff05b400640 1 --2- 172.21.15.5:0/2484266831 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5571c286ae50 0x5571c2868790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:35.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:35.123+0000 7ff05b400640 1 -- 172.21.15.5:0/2484266831 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5571c2860590 msgr2=0x5571c2868250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:42:35.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:35.123+0000 7ff05b400640 1 --2- 172.21.15.5:0/2484266831 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5571c2860590 0x5571c2868250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:35.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:35.123+0000 7ff05b400640 1 -- 172.21.15.5:0/2484266831 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5571c25373c0 con 0x5571c285fc70 2024-09-10T09:42:35.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:35.124+0000 7ff05b400640 1 --2- 172.21.15.5:0/2484266831 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5571c285fc70 0x5571c2867d10 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7ff04c01ea20 tx=0x7ff04c004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ede536428f097ce1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:42:35.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:35.124+0000 7ff048c00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-09-10T09:42:35.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:35.124+0000 7ff048c00640 1 -- 172.21.15.5:0/2484266831 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff04c02e060 con 0x5571c285fc70 2024-09-10T09:42:35.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:35.124+0000 7ff064e22740 1 -- 172.21.15.5:0/2484266831 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff054004050 con 0x5571c285fc70 2024-09-10T09:42:35.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:35.124+0000 7ff048c00640 1 -- 172.21.15.5:0/2484266831 <== mon.2 v2:172.21.15.102:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7ff04c018030 con 0x5571c285fc70 2024-09-10T09:42:35.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:35.124+0000 7ff048c00640 1 -- 172.21.15.5:0/2484266831 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7ff04c01c460 con 0x5571c285fc70 2024-09-10T09:42:35.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:35.124+0000 7ff064e22740 10 client.26210 fetch_fsmap learned FSMap version 33 2024-09-10T09:42:35.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:35.124+0000 7ff064e22740 1 -- 172.21.15.5:0/2484266831 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5571c2879ae0 con 0x5571c285fc70 2024-09-10T09:42:35.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:35.125+0000 7ff048c00640 1 -- 172.21.15.5:0/2484266831 <== mon.2 v2:172.21.15.102:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff054004050 con 0x5571c285fc70 2024-09-10T09:42:35.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:35.125+0000 7ff048c00640 1 -- 172.21.15.5:0/2484266831 <== mon.2 v2:172.21.15.102:3300/0 5 ==== fsmap(e 33) ==== 8374+0+0 (secure 0 0 0) 0x7ff04c00f690 con 0x5571c285fc70 2024-09-10T09:42:35.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:35.125+0000 7ff064e22740 10 client.26210 fetch_fsmap finished waiting for FSMap version 33 2024-09-10T09:42:35.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:35.125+0000 7ff064e22740 10 client.26210 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-10T09:42:35.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:35.125+0000 7ff064e22740 20 client.26210 populate_metadata read hostname 'smithi005' 2024-09-10T09:42:35.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:35.125+0000 7ff064e22740 1 --2- 172.21.15.5:0/2484266831 >> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] conn(0x5571c287afa0 0x5571c287d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:42:35.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:35.125+0000 7ff064e22740 4 client.26210 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:42:35.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:35.125+0000 7ff064e22740 1 -- 172.21.15.5:0/2484266831 --> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5571c27d3950 con 0x5571c287afa0 2024-09-10T09:42:35.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:35.126+0000 7ff052a00640 1 --2- 172.21.15.5:0/2484266831 >> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] conn(0x5571c287afa0 0x5571c287d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:42:35.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:35.126+0000 7ff052a00640 1 --2- 172.21.15.5:0/2484266831 >> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] conn(0x5571c287afa0 0x5571c287d3d0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f556aa95445b9d82 server_cookie=553260ce681b052e in_seq=0 out_seq=0 2024-09-10T09:42:35.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:35.126+0000 7ff048c00640 10 client.26210 ms_handle_connect on v2:172.21.15.159:6838/2073739599 2024-09-10T09:42:35.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:35.127+0000 7ff048c00640 1 -- 172.21.15.5:0/2484266831 <== mds.1 v2:172.21.15.159:6838/2073739599 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5571c27d3950 con 0x5571c287afa0 2024-09-10T09:42:35.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:35.127+0000 7ff048c00640 10 client.26210 handle_command_reply: tid=0 2024-09-10T09:42:35.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:35.127+0000 7ff064e22740 1 client.26210 shutdown 2024-09-10T09:42:35.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:35.127+0000 7ff064e22740 1 -- 172.21.15.5:0/2484266831 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5571c285fc70 msgr2=0x5571c2867d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:42:35.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:35.127+0000 7ff064e22740 1 --2- 172.21.15.5:0/2484266831 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5571c285fc70 0x5571c2867d10 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7ff04c01ea20 tx=0x7ff04c004730 comp rx=0 tx=0).stop 2024-09-10T09:42:35.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:35.127+0000 7ff064e22740 1 -- 172.21.15.5:0/2484266831 shutdown_connections 2024-09-10T09:42:35.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:35.127+0000 7ff064e22740 1 --2- 172.21.15.5:0/2484266831 >> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] conn(0x5571c287afa0 0x5571c287d3d0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:35.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:35.127+0000 7ff064e22740 1 --2- 172.21.15.5:0/2484266831 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5571c286ae50 0x5571c2868790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:35.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:35.127+0000 7ff064e22740 1 --2- 172.21.15.5:0/2484266831 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5571c2860590 0x5571c2868250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:35.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:35.127+0000 7ff064e22740 1 --2- 172.21.15.5:0/2484266831 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5571c285fc70 0x5571c2867d10 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:35.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:35.127+0000 7ff064e22740 1 -- 172.21.15.5:0/2484266831 >> 172.21.15.5:0/2484266831 conn(0x5571c285c200 msgr2=0x5571c285b380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:42:35.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:35.127+0000 7ff048c00640 0 client.26210 ms_handle_reset on v2:172.21.15.159:6838/2073739599 2024-09-10T09:42:35.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:35.127+0000 7ff064e22740 1 -- 172.21.15.5:0/2484266831 shutdown_connections 2024-09-10T09:42:35.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:35.127+0000 7ff064e22740 1 -- 172.21.15.5:0/2484266831 wait complete. 2024-09-10T09:42:35.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:35.128+0000 7ff064e22740 20 client.26210 trim_cache size 0 max 16384 2024-09-10T09:42:35.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:35.141+0000 7ff064e22740 1 Processor -- start 2024-09-10T09:42:35.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:35.142+0000 7ff064e22740 1 -- start start 2024-09-10T09:42:35.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:35.142+0000 7ff064e22740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5571c285fc70 0x5571c28626d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:42:35.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:35.142+0000 7ff064e22740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5571c2860590 0x5571c2869c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:42:35.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:35.142+0000 7ff064e22740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5571c286ae50 0x5571c2882ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:42:35.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:35.142+0000 7ff064e22740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff05c1b10f0 con 0x5571c285fc70 2024-09-10T09:42:35.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:35.142+0000 7ff064e22740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7ff05c1b2c60 con 0x5571c2860590 2024-09-10T09:42:35.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:35.142+0000 7ff064e22740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7ff05c1b17c0 con 0x5571c286ae50 2024-09-10T09:42:35.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:35.142+0000 7ff052a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5571c2860590 0x5571c2869c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:42:35.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:35.142+0000 7ff052a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5571c2860590 0x5571c2869c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:34500/0 (socket says 172.21.15.5:34500) 2024-09-10T09:42:35.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:35.142+0000 7ff052a00640 1 -- 172.21.15.5:0/1551679829 learned_addr learned my addr 172.21.15.5:0/1551679829 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:42:35.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:35.142+0000 7ff05b400640 1 --2- 172.21.15.5:0/1551679829 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5571c285fc70 0x5571c28626d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:42:35.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:35.142+0000 7ff05be00640 1 --2- 172.21.15.5:0/1551679829 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5571c286ae50 0x5571c2882ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:42:35.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:35.143+0000 7ff052a00640 1 -- 172.21.15.5:0/1551679829 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5571c286ae50 msgr2=0x5571c2882ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:42:35.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:35.143+0000 7ff052a00640 1 --2- 172.21.15.5:0/1551679829 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5571c286ae50 0x5571c2882ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:35.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:35.143+0000 7ff052a00640 1 -- 172.21.15.5:0/1551679829 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5571c285fc70 msgr2=0x5571c28626d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:42:35.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:35.143+0000 7ff052a00640 1 --2- 172.21.15.5:0/1551679829 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5571c285fc70 0x5571c28626d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:35.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:35.143+0000 7ff052a00640 1 -- 172.21.15.5:0/1551679829 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5571c2857770 con 0x5571c2860590 2024-09-10T09:42:35.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:35.143+0000 7ff05b400640 1 --2- 172.21.15.5:0/1551679829 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5571c285fc70 0x5571c28626d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:42:35.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:35.143+0000 7ff052a00640 1 --2- 172.21.15.5:0/1551679829 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5571c2860590 0x5571c2869c80 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7ff0440b80d0 tx=0x7ff0440ecc70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a2575fcc2349c8ae server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:42:35.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:35.143+0000 7ff04b400640 1 -- 172.21.15.5:0/1551679829 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff044049040 con 0x5571c2860590 2024-09-10T09:42:35.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:35.143+0000 7ff04b400640 1 -- 172.21.15.5:0/1551679829 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff0440bbc70 con 0x5571c2860590 2024-09-10T09:42:35.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:35.143+0000 7ff04b400640 1 -- 172.21.15.5:0/1551679829 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0440ef370 con 0x5571c2860590 2024-09-10T09:42:35.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:35.144+0000 7ff064e22740 1 -- 172.21.15.5:0/1551679829 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5571c2860590 msgr2=0x5571c2869c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:42:35.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:35.144+0000 7ff064e22740 1 --2- 172.21.15.5:0/1551679829 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5571c2860590 0x5571c2869c80 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7ff0440b80d0 tx=0x7ff0440ecc70 comp rx=0 tx=0).stop 2024-09-10T09:42:35.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:35.144+0000 7ff064e22740 1 -- 172.21.15.5:0/1551679829 shutdown_connections 2024-09-10T09:42:35.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:35.144+0000 7ff064e22740 1 --2- 172.21.15.5:0/1551679829 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5571c286ae50 0x5571c2882ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:35.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:35.144+0000 7ff064e22740 1 --2- 172.21.15.5:0/1551679829 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5571c2860590 0x5571c2869c80 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:35.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:35.144+0000 7ff064e22740 1 --2- 172.21.15.5:0/1551679829 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5571c285fc70 0x5571c28626d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:35.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:35.144+0000 7ff064e22740 1 -- 172.21.15.5:0/1551679829 >> 172.21.15.5:0/1551679829 conn(0x5571c285c200 msgr2=0x5571c2869570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:42:35.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:35.144+0000 7ff064e22740 1 -- 172.21.15.5:0/1551679829 shutdown_connections 2024-09-10T09:42:35.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:35.144+0000 7ff064e22740 1 -- 172.21.15.5:0/1551679829 wait complete. 2024-09-10T09:42:35.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:35.144+0000 7ff064e22740 1 -- start start 2024-09-10T09:42:35.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:35.144+0000 7ff064e22740 1 Processor -- start 2024-09-10T09:42:35.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:35.145+0000 7ff064e22740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5571c285fc70 0x5571c286ec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:42:35.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:35.145+0000 7ff064e22740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5571c2860590 0x5571c286f180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:42:35.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:35.145+0000 7ff064e22740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5571c286ae50 0x5571c286f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:42:35.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:35.145+0000 7ff064e22740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff05c1b2560 con 0x5571c286ae50 2024-09-10T09:42:35.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:35.145+0000 7ff064e22740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7ff05c1b1e90 con 0x5571c2860590 2024-09-10T09:42:35.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:35.145+0000 7ff064e22740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7ff05c1af0d0 con 0x5571c285fc70 2024-09-10T09:42:35.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:35.145+0000 7ff064e22740 10 client.0 _ll_register_callbacks cb 0x7ff04c009830 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:42:35.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:35.145+0000 7ff052a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5571c2860590 0x5571c286f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:42:35.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:35.145+0000 7ff05be00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5571c286ae50 0x5571c286f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:42:35.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:35.145+0000 7ff052a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5571c2860590 0x5571c286f180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:34506/0 (socket says 172.21.15.5:34506) 2024-09-10T09:42:35.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:35.145+0000 7ff052a00640 1 -- 172.21.15.5:0/1672852539 learned_addr learned my addr 172.21.15.5:0/1672852539 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:42:35.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:35.145+0000 7ff05b400640 1 --2- 172.21.15.5:0/1672852539 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5571c285fc70 0x5571c286ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:42:35.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:35.146+0000 7ff052a00640 1 -- 172.21.15.5:0/1672852539 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5571c285fc70 msgr2=0x5571c286ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:42:35.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:35.146+0000 7ff052a00640 1 --2- 172.21.15.5:0/1672852539 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5571c285fc70 0x5571c286ec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:35.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:35.146+0000 7ff052a00640 1 -- 172.21.15.5:0/1672852539 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5571c286ae50 msgr2=0x5571c286f6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:42:35.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:35.146+0000 7ff052a00640 1 --2- 172.21.15.5:0/1672852539 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5571c286ae50 0x5571c286f6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:35.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:35.146+0000 7ff052a00640 1 -- 172.21.15.5:0/1672852539 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5571c28e4d70 con 0x5571c2860590 2024-09-10T09:42:35.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:35.146+0000 7ff052a00640 1 --2- 172.21.15.5:0/1672852539 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5571c2860590 0x5571c286f180 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7ff0440bbc00 tx=0x7ff044009560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5e45c905f60b9508 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:42:35.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:35.146+0000 7ff048c00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-09-10T09:42:35.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:35.146+0000 7ff048c00640 1 -- 172.21.15.5:0/1672852539 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0440d8030 con 0x5571c2860590 2024-09-10T09:42:35.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:35.146+0000 7ff064e22740 1 -- 172.21.15.5:0/1672852539 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff0440bbc70 con 0x5571c2860590 2024-09-10T09:42:35.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:35.146+0000 7ff048c00640 1 -- 172.21.15.5:0/1672852539 <== mon.2 v2:172.21.15.102:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7ff0440b9f70 con 0x5571c2860590 2024-09-10T09:42:35.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:35.146+0000 7ff048c00640 1 -- 172.21.15.5:0/1672852539 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7ff044049070 con 0x5571c2860590 2024-09-10T09:42:35.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:35.146+0000 7ff064e22740 10 client.26216 fetch_fsmap learned FSMap version 33 2024-09-10T09:42:35.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:35.146+0000 7ff064e22740 1 -- 172.21.15.5:0/1672852539 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5571c28e60a0 con 0x5571c2860590 2024-09-10T09:42:35.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:35.147+0000 7ff048c00640 1 -- 172.21.15.5:0/1672852539 <== mon.2 v2:172.21.15.102:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff0440bbc70 con 0x5571c2860590 2024-09-10T09:42:35.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:35.147+0000 7ff048c00640 1 -- 172.21.15.5:0/1672852539 <== mon.2 v2:172.21.15.102:3300/0 5 ==== fsmap(e 33) ==== 8374+0+0 (secure 0 0 0) 0x7ff04404c060 con 0x5571c2860590 2024-09-10T09:42:35.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:35.147+0000 7ff064e22740 10 client.26216 fetch_fsmap finished waiting for FSMap version 33 2024-09-10T09:42:35.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:35.147+0000 7ff064e22740 10 client.26216 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-10T09:42:35.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:35.147+0000 7ff064e22740 20 client.26216 populate_metadata read hostname 'smithi005' 2024-09-10T09:42:35.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:35.147+0000 7ff064e22740 1 --2- 172.21.15.5:0/1672852539 >> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] conn(0x5571c28e65e0 0x5571c28e8a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:42:35.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:35.147+0000 7ff064e22740 4 client.26216 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T09:42:35.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:35.147+0000 7ff064e22740 1 -- 172.21.15.5:0/1672852539 --> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5571c27cde60 con 0x5571c28e65e0 2024-09-10T09:42:35.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:35.147+0000 7ff05b400640 1 --2- 172.21.15.5:0/1672852539 >> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] conn(0x5571c28e65e0 0x5571c28e8a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:42:35.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:35.148+0000 7ff05b400640 1 --2- 172.21.15.5:0/1672852539 >> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] conn(0x5571c28e65e0 0x5571c28e8a00 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e5129975bbeb3992 server_cookie=b62bb06683e8392e in_seq=0 out_seq=0 2024-09-10T09:42:35.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:35.148+0000 7ff048c00640 10 client.26216 ms_handle_connect on v2:172.21.15.159:6838/2073739599 2024-09-10T09:42:35.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:35.148+0000 7ff048c00640 1 -- 172.21.15.5:0/1672852539 <== mds.1 v2:172.21.15.159:6838/2073739599 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5571c27cde60 con 0x5571c28e65e0 2024-09-10T09:42:35.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:35.148+0000 7ff048c00640 10 client.26216 handle_command_reply: tid=0 2024-09-10T09:42:35.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:35.148+0000 7ff064e22740 1 client.26216 shutdown 2024-09-10T09:42:35.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:35.148+0000 7ff064e22740 1 -- 172.21.15.5:0/1672852539 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5571c2860590 msgr2=0x5571c286f180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:42:35.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:35.148+0000 7ff064e22740 1 --2- 172.21.15.5:0/1672852539 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5571c2860590 0x5571c286f180 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7ff0440bbc00 tx=0x7ff044009560 comp rx=0 tx=0).stop 2024-09-10T09:42:35.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:35.148+0000 7ff064e22740 1 -- 172.21.15.5:0/1672852539 shutdown_connections 2024-09-10T09:42:35.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:35.148+0000 7ff064e22740 1 --2- 172.21.15.5:0/1672852539 >> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] conn(0x5571c28e65e0 0x5571c28e8a00 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:35.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:35.148+0000 7ff064e22740 1 --2- 172.21.15.5:0/1672852539 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5571c286ae50 0x5571c286f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:35.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:35.148+0000 7ff064e22740 1 --2- 172.21.15.5:0/1672852539 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5571c2860590 0x5571c286f180 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:35.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:35.148+0000 7ff064e22740 1 --2- 172.21.15.5:0/1672852539 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5571c285fc70 0x5571c286ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:35.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:35.148+0000 7ff048c00640 0 client.26216 ms_handle_reset on v2:172.21.15.159:6838/2073739599 2024-09-10T09:42:35.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:35.148+0000 7ff064e22740 1 -- 172.21.15.5:0/1672852539 >> 172.21.15.5:0/1672852539 conn(0x5571c285c200 msgr2=0x5571c285b380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:42:35.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:35.149+0000 7ff064e22740 1 -- 172.21.15.5:0/1672852539 shutdown_connections 2024-09-10T09:42:35.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:35.149+0000 7ff064e22740 1 -- 172.21.15.5:0/1672852539 wait complete. 2024-09-10T09:42:35.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:35.149+0000 7ff064e22740 20 client.26216 trim_cache size 0 max 16384 2024-09-10T09:42:35.153 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-10T09:42:35.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:35.149+0000 7ff062200640 1 -- 172.21.15.5:0/4176597564 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7ff0400a2b80 msgr2=0x7ff0400a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:42:35.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:35.149+0000 7ff062200640 1 --2- 172.21.15.5:0/4176597564 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7ff0400a2b80 0x7ff0400a5040 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x5571c285d060 tx=0x7ff04c005f90 comp rx=0 tx=0).stop 2024-09-10T09:42:35.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:35.149+0000 7ff062200640 1 -- 172.21.15.5:0/4176597564 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff05c1049c0 msgr2=0x7ff05c19ac50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:42:35.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:35.149+0000 7ff062200640 1 --2- 172.21.15.5:0/4176597564 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff05c1049c0 0x7ff05c19ac50 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7ff044004770 tx=0x7ff044003c10 comp rx=0 tx=0).stop 2024-09-10T09:42:35.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:35.149+0000 7ff062200640 1 -- 172.21.15.5:0/4176597564 shutdown_connections 2024-09-10T09:42:35.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:35.149+0000 7ff062200640 1 --2- 172.21.15.5:0/4176597564 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7ff0400a2b80 0x7ff0400a5040 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:35.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:35.149+0000 7ff062200640 1 --2- 172.21.15.5:0/4176597564 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff05c109140 0x7ff05c19efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:35.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:35.149+0000 7ff062200640 1 --2- 172.21.15.5:0/4176597564 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff05c1049c0 0x7ff05c19ac50 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:35.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:35.149+0000 7ff062200640 1 --2- 172.21.15.5:0/4176597564 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff05c104010 0x7ff05c19a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:35.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:35.149+0000 7ff062200640 1 -- 172.21.15.5:0/4176597564 >> 172.21.15.5:0/4176597564 conn(0x7ff05c0ffa50 msgr2=0x7ff05c100d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:42:35.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:35.150+0000 7ff062200640 1 -- 172.21.15.5:0/4176597564 shutdown_connections 2024-09-10T09:42:35.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:35.150+0000 7ff062200640 1 -- 172.21.15.5:0/4176597564 wait complete. 2024-09-10T09:42:35.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:35.832+0000 7fbde7800640 1 -- 172.21.15.5:0/2310608595 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbde01049a0 msgr2=0x7fbde01089f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:42:35.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:35.832+0000 7fbde7800640 1 --2- 172.21.15.5:0/2310608595 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbde01049a0 0x7fbde01089f0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fbdcc009900 tx=0x7fbdcc02a410 comp rx=0 tx=0).stop 2024-09-10T09:42:35.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:35.832+0000 7fbde7800640 1 -- 172.21.15.5:0/2310608595 shutdown_connections 2024-09-10T09:42:35.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:35.832+0000 7fbde7800640 1 --2- 172.21.15.5:0/2310608595 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbde0109120 0x7fbde0110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:35.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:35.832+0000 7fbde7800640 1 --2- 172.21.15.5:0/2310608595 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbde01049a0 0x7fbde01089f0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:35.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:35.832+0000 7fbde7800640 1 --2- 172.21.15.5:0/2310608595 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbde0103ff0 0x7fbde01043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:35.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:35.832+0000 7fbde7800640 1 -- 172.21.15.5:0/2310608595 >> 172.21.15.5:0/2310608595 conn(0x7fbde00ffa90 msgr2=0x7fbde0101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:42:35.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:35.832+0000 7fbde7800640 1 -- 172.21.15.5:0/2310608595 shutdown_connections 2024-09-10T09:42:35.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:35.833+0000 7fbde7800640 1 -- 172.21.15.5:0/2310608595 wait complete. 2024-09-10T09:42:35.835 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:42:35 smithi005 ceph-mon[30350]: pgmap v520: 129 pgs: 129 active+clean; 95 MiB data, 849 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 1.1 MiB/s wr, 51 op/s 2024-09-10T09:42:35.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:35.833+0000 7fbde7800640 1 Processor -- start 2024-09-10T09:42:35.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:35.834+0000 7fbde7800640 1 -- start start 2024-09-10T09:42:35.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:35.834+0000 7fbde7800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbde0103ff0 0x7fbde019a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:42:35.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:35.834+0000 7fbde7800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbde01049a0 0x7fbde019abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:42:35.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:35.834+0000 7fbde7800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbde0109120 0x7fbde019ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:42:35.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:35.834+0000 7fbde7800640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fbde0114140 con 0x7fbde01049a0 2024-09-10T09:42:35.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:35.834+0000 7fbde7800640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fbde0113fc0 con 0x7fbde0109120 2024-09-10T09:42:35.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:35.834+0000 7fbde7800640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fbde01142c0 con 0x7fbde0103ff0 2024-09-10T09:42:35.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:35.834+0000 7fbddfe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbde01049a0 0x7fbde019abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:42:35.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:35.834+0000 7fbddfe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbde01049a0 0x7fbde019abe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46202/0 (socket says 172.21.15.5:46202) 2024-09-10T09:42:35.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:35.834+0000 7fbddfe00640 1 -- 172.21.15.5:0/1789775760 learned_addr learned my addr 172.21.15.5:0/1789775760 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:42:35.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:35.834+0000 7fbde5800640 1 --2- 172.21.15.5:0/1789775760 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbde0109120 0x7fbde019ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:42:35.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:35.835+0000 7fbde4e00640 1 --2- 172.21.15.5:0/1789775760 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbde0103ff0 0x7fbde019a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:42:35.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:35.835+0000 7fbddfe00640 1 -- 172.21.15.5:0/1789775760 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbde0103ff0 msgr2=0x7fbde019a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:42:35.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:35.835+0000 7fbddfe00640 1 --2- 172.21.15.5:0/1789775760 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbde0103ff0 0x7fbde019a6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:35.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:35.835+0000 7fbddfe00640 1 -- 172.21.15.5:0/1789775760 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbde0109120 msgr2=0x7fbde019ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:42:35.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:35.835+0000 7fbddfe00640 1 --2- 172.21.15.5:0/1789775760 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbde0109120 0x7fbde019ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:35.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:35.835+0000 7fbddfe00640 1 -- 172.21.15.5:0/1789775760 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbde019f650 con 0x7fbde01049a0 2024-09-10T09:42:35.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:35.835+0000 7fbddfe00640 1 --2- 172.21.15.5:0/1789775760 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbde01049a0 0x7fbde019abe0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fbdcc006010 tx=0x7fbdcc0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:42:35.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:35.836+0000 7fbddd600640 1 -- 172.21.15.5:0/1789775760 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdcc03f030 con 0x7fbde01049a0 2024-09-10T09:42:35.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:35.836+0000 7fbde7800640 1 -- 172.21.15.5:0/1789775760 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbde019f8e0 con 0x7fbde01049a0 2024-09-10T09:42:35.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:35.836+0000 7fbddd600640 1 -- 172.21.15.5:0/1789775760 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbdcc003ce0 con 0x7fbde01049a0 2024-09-10T09:42:35.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:35.836+0000 7fbddd600640 1 -- 172.21.15.5:0/1789775760 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdcc035070 con 0x7fbde01049a0 2024-09-10T09:42:35.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:35.836+0000 7fbde7800640 1 -- 172.21.15.5:0/1789775760 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbde01a7220 con 0x7fbde01049a0 2024-09-10T09:42:35.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:35.837+0000 7fbde7800640 1 -- 172.21.15.5:0/1789775760 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbde01043d0 con 0x7fbde01049a0 2024-09-10T09:42:35.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:35.839+0000 7fbddd600640 1 -- 172.21.15.5:0/1789775760 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7fbdcc042030 con 0x7fbde01049a0 2024-09-10T09:42:35.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:35.840+0000 7fbddd600640 1 --2- 172.21.15.5:0/1789775760 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fbdc80a2c00 0x7fbdc80a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:42:35.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:35.841+0000 7fbddd600640 1 -- 172.21.15.5:0/1789775760 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fbdcc0ef360 con 0x7fbde01049a0 2024-09-10T09:42:35.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:35.841+0000 7fbde4e00640 1 --2- 172.21.15.5:0/1789775760 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fbdc80a2c00 0x7fbdc80a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:42:35.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:35.841+0000 7fbde4e00640 1 --2- 172.21.15.5:0/1789775760 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fbdc80a2c00 0x7fbdc80a50c0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fbdd4009bd0 tx=0x7fbdd4005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:42:35.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:35.845+0000 7fbddd600640 1 -- 172.21.15.5:0/1789775760 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fbdcc0bb8e0 con 0x7fbde01049a0 2024-09-10T09:42:35.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:42:35 smithi159 ceph-mon[35513]: pgmap v520: 129 pgs: 129 active+clean; 95 MiB data, 849 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 1.1 MiB/s wr, 51 op/s 2024-09-10T09:42:35.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:42:35 smithi102 ceph-mon[35442]: pgmap v520: 129 pgs: 129 active+clean; 95 MiB data, 849 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 1.1 MiB/s wr, 51 op/s 2024-09-10T09:42:36.016 INFO:tasks.workunit.client.0.smithi005.stdout: 1 191939 0.00 MB/sec execute 17 sec latency 20921.233 ms 2024-09-10T09:42:36.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:36.067+0000 7fbde7800640 1 -- 172.21.15.5:0/1789775760 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbde010c360 con 0x7fbde01049a0 2024-09-10T09:42:36.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:36.068+0000 7fbddd600640 1 -- 172.21.15.5:0/1789775760 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) ==== 94+0+11970 (secure 0 0 0) 0x7fbdcc0bfbe0 con 0x7fbde01049a0 2024-09-10T09:42:36.069 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-10T09:42:36.069 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":33,"btime":"2024-09-10T09:42:17:528412+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14454,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/2970594311","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2970594311},{"type":"v1","addr":"172.21.15.5:6835","nonce":2970594311}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14514,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6839/3616502174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":3616502174},{"type":"v1","addr":"172.21.15.5:6839","nonce":3616502174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":33,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T09:35:29.626287+0000","modified":"2024-09-10T09:42:16.961192+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24542,"mds_1":24535,"mds_2":24511,"mds_3":24506,"mds_4":24443},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14520":{"gid":14520,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/2073599842","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":2073599842},{"type":"v1","addr":"172.21.15.5:6841","nonce":2073599842}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"k","rank":4,"incarnation":26,"state":"up:active","state_seq":24,"addr":"172.21.15.102:6833/2457007427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":2457007427},{"type":"v1","addr":"172.21.15.102:6833","nonce":2457007427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/356445159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":356445159},{"type":"v1","addr":"172.21.15.159:6833","nonce":356445159}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6837/2894314507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":2894314507},{"type":"v1","addr":"172.21.15.5:6837","nonce":2894314507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.102:6835/1647549111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":1647549111},{"type":"v1","addr":"172.21.15.102:6835","nonce":1647549111}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6835/952657242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":952657242},{"type":"v1","addr":"172.21.15.159:6835","nonce":952657242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":19,"addr":"172.21.15.102:6837/2689776764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":2689776764},{"type":"v1","addr":"172.21.15.102:6837","nonce":2689776764}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.159:6837/620627639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":620627639},{"type":"v1","addr":"172.21.15.159:6837","nonce":620627639}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24535":{"gid":24535,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":18,"addr":"172.21.15.159:6839/2073739599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":2073739599},{"type":"v1","addr":"172.21.15.159:6839","nonce":2073739599}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24542":{"gid":24542,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.102:6839/335409749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6838","nonce":335409749},{"type":"v1","addr":"172.21.15.102:6839","nonce":335409749}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24542,"qdb_cluster":[24443,24506,24511,24535,24542]},"id":1}]} 2024-09-10T09:42:36.069 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 33 2024-09-10T09:42:36.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:36.070+0000 7fbde7800640 1 -- 172.21.15.5:0/1789775760 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fbdc80a2c00 msgr2=0x7fbdc80a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:42:36.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:36.070+0000 7fbde7800640 1 --2- 172.21.15.5:0/1789775760 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fbdc80a2c00 0x7fbdc80a50c0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fbdd4009bd0 tx=0x7fbdd4005f90 comp rx=0 tx=0).stop 2024-09-10T09:42:36.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:36.070+0000 7fbde7800640 1 -- 172.21.15.5:0/1789775760 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbde01049a0 msgr2=0x7fbde019abe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:42:36.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:36.070+0000 7fbde7800640 1 --2- 172.21.15.5:0/1789775760 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbde01049a0 0x7fbde019abe0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fbdcc006010 tx=0x7fbdcc0042f0 comp rx=0 tx=0).stop 2024-09-10T09:42:36.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:36.070+0000 7fbde7800640 1 -- 172.21.15.5:0/1789775760 shutdown_connections 2024-09-10T09:42:36.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:36.070+0000 7fbde7800640 1 --2- 172.21.15.5:0/1789775760 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fbdc80a2c00 0x7fbdc80a50c0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:36.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:36.070+0000 7fbde7800640 1 --2- 172.21.15.5:0/1789775760 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbde0109120 0x7fbde019ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:36.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:36.070+0000 7fbde7800640 1 --2- 172.21.15.5:0/1789775760 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbde01049a0 0x7fbde019abe0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:36.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:36.070+0000 7fbde7800640 1 --2- 172.21.15.5:0/1789775760 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbde0103ff0 0x7fbde019a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:36.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:36.070+0000 7fbde7800640 1 -- 172.21.15.5:0/1789775760 >> 172.21.15.5:0/1789775760 conn(0x7fbde00ffa90 msgr2=0x7fbde0100d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:42:36.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:36.070+0000 7fbde7800640 1 -- 172.21.15.5:0/1789775760 shutdown_connections 2024-09-10T09:42:36.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:36.070+0000 7fbde7800640 1 -- 172.21.15.5:0/1789775760 wait complete. 2024-09-10T09:42:36.344 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-10T09:42:36.672 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:42:36 smithi005 ceph-mon[30350]: from='client.? 172.21.15.5:0/1789775760' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:42:36.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:42:36 smithi159 ceph-mon[35513]: from='client.? 172.21.15.5:0/1789775760' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:42:36.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:42:36 smithi102 ceph-mon[35442]: from='client.? 172.21.15.5:0/1789775760' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:42:37.016 INFO:tasks.workunit.client.0.smithi005.stdout: 1 191939 0.00 MB/sec execute 18 sec latency 21921.366 ms 2024-09-10T09:42:37.236 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24542 quiesce db --set-id ba5a0a38 --release --await' 2024-09-10T09:42:37.237 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24542 quiesce db --set-id ba5a0a38 --release --await 2024-09-10T09:42:37.274 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:42:37.684 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:42:37 smithi005 ceph-mon[30350]: pgmap v521: 129 pgs: 129 active+clean; 96 MiB data, 833 MiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 1.2 MiB/s wr, 60 op/s 2024-09-10T09:42:37.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:42:37 smithi159 ceph-mon[35513]: pgmap v521: 129 pgs: 129 active+clean; 96 MiB data, 833 MiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 1.2 MiB/s wr, 60 op/s 2024-09-10T09:42:37.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:42:37 smithi102 ceph-mon[35442]: pgmap v521: 129 pgs: 129 active+clean; 96 MiB data, 833 MiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 1.2 MiB/s wr, 60 op/s 2024-09-10T09:42:37.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:37.955+0000 7f54fb200640 1 -- 172.21.15.5:0/3033790623 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f54fc10e330 msgr2=0x7f54fc110720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:42:37.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:37.955+0000 7f54fb200640 1 --2- 172.21.15.5:0/3033790623 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f54fc10e330 0x7f54fc110720 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f54f4009be0 tx=0x7f54f402a6f0 comp rx=0 tx=0).stop 2024-09-10T09:42:37.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:37.955+0000 7f54fb200640 1 -- 172.21.15.5:0/3033790623 shutdown_connections 2024-09-10T09:42:37.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:37.955+0000 7f54fb200640 1 --2- 172.21.15.5:0/3033790623 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f54fc10e330 0x7f54fc110720 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:37.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:37.955+0000 7f54fb200640 1 --2- 172.21.15.5:0/3033790623 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f54fc101130 0x7f54fc10ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:37.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:37.955+0000 7f54fb200640 1 --2- 172.21.15.5:0/3033790623 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f54fc100780 0x7f54fc100b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:37.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:37.955+0000 7f54fb200640 1 -- 172.21.15.5:0/3033790623 >> 172.21.15.5:0/3033790623 conn(0x7f54fc0fc610 msgr2=0x7f54fc0fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:42:37.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:37.955+0000 7f54fb200640 1 -- 172.21.15.5:0/3033790623 shutdown_connections 2024-09-10T09:42:37.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:37.955+0000 7f54fb200640 1 -- 172.21.15.5:0/3033790623 wait complete. 2024-09-10T09:42:37.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:37.955+0000 7f54fb200640 1 Processor -- start 2024-09-10T09:42:37.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:37.955+0000 7f54fb200640 1 -- start start 2024-09-10T09:42:37.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:37.955+0000 7f54fb200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f54fc100780 0x7f54fc19a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:42:37.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:37.955+0000 7f54fb200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f54fc101130 0x7f54fc19aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:42:37.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:37.955+0000 7f54fb200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f54fc10e330 0x7f54fc19edd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:42:37.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:37.955+0000 7f54fb200640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f54fc114020 con 0x7f54fc100780 2024-09-10T09:42:37.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:37.955+0000 7f54fb200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f54fc113ea0 con 0x7f54fc10e330 2024-09-10T09:42:37.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:37.955+0000 7f54fb200640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f54fc1141a0 con 0x7f54fc101130 2024-09-10T09:42:37.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:37.955+0000 7f54f9e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f54fc100780 0x7f54fc19a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:42:37.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:37.955+0000 7f54fa800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f54fc10e330 0x7f54fc19edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:42:37.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:37.955+0000 7f54f9e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f54fc100780 0x7f54fc19a500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46212/0 (socket says 172.21.15.5:46212) 2024-09-10T09:42:37.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:37.955+0000 7f54f9e00640 1 -- 172.21.15.5:0/1354103664 learned_addr learned my addr 172.21.15.5:0/1354103664 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:42:37.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:37.955+0000 7f54f9e00640 1 -- 172.21.15.5:0/1354103664 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f54fc101130 msgr2=0x7f54fc19aa40 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-10T09:42:37.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:37.955+0000 7f54f9e00640 1 --2- 172.21.15.5:0/1354103664 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f54fc101130 0x7f54fc19aa40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:37.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:37.955+0000 7f54f9e00640 1 -- 172.21.15.5:0/1354103664 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f54fc10e330 msgr2=0x7f54fc19edd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:42:37.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:37.955+0000 7f54f9e00640 1 --2- 172.21.15.5:0/1354103664 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f54fc10e330 0x7f54fc19edd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:37.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:37.955+0000 7f54f9e00640 1 -- 172.21.15.5:0/1354103664 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54fc19f4b0 con 0x7f54fc100780 2024-09-10T09:42:37.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:37.955+0000 7f54fa800640 1 --2- 172.21.15.5:0/1354103664 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f54fc10e330 0x7f54fc19edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:42:37.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:37.960+0000 7f54f9e00640 1 --2- 172.21.15.5:0/1354103664 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f54fc100780 0x7f54fc19a500 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f54ec00e9c0 tx=0x7f54ec00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:42:37.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:37.960+0000 7f54f2a00640 1 -- 172.21.15.5:0/1354103664 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54ec00cde0 con 0x7f54fc100780 2024-09-10T09:42:37.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:37.960+0000 7f54f2a00640 1 -- 172.21.15.5:0/1354103664 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f54ec0035f0 con 0x7f54fc100780 2024-09-10T09:42:37.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:37.960+0000 7f54f2a00640 1 -- 172.21.15.5:0/1354103664 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54ec010690 con 0x7f54fc100780 2024-09-10T09:42:37.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:37.960+0000 7f54fb200640 1 -- 172.21.15.5:0/1354103664 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54fc19f7a0 con 0x7f54fc100780 2024-09-10T09:42:37.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:37.960+0000 7f54fb200640 1 -- 172.21.15.5:0/1354103664 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54fc1a7080 con 0x7f54fc100780 2024-09-10T09:42:37.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:37.965+0000 7f54f2a00640 1 -- 172.21.15.5:0/1354103664 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f54ec0037a0 con 0x7f54fc100780 2024-09-10T09:42:37.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:37.965+0000 7f5503c4d740 1 Processor -- start 2024-09-10T09:42:37.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:37.965+0000 7f5503c4d740 1 -- start start 2024-09-10T09:42:37.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:37.965+0000 7f5503c4d740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557c5f24ae40 0x557c5f24b220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:42:37.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:37.965+0000 7f5503c4d740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557c5f24b760 0x557c5f255ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:42:37.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:37.965+0000 7f5503c4d740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557c5f256020 0x557c5f258410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:42:37.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:37.965+0000 7f5503c4d740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x557c5f1b13e0 con 0x557c5f24b760 2024-09-10T09:42:37.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:37.965+0000 7f5503c4d740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x557c5f158d80 con 0x557c5f256020 2024-09-10T09:42:37.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:37.965+0000 7f5503c4d740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x557c5f1ad3a0 con 0x557c5f24ae40 2024-09-10T09:42:37.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:37.966+0000 7f54fa800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557c5f24ae40 0x557c5f24b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:42:37.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:37.966+0000 7f54f9400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557c5f256020 0x557c5f258410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:42:37.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:37.966+0000 7f54f9e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557c5f24b760 0x557c5f255ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:42:37.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:37.966+0000 7f54f2a00640 1 --2- 172.21.15.5:0/1354103664 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f54e00a2cd0 0x7f54e00a5190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:42:37.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:37.966+0000 7f54f9400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557c5f256020 0x557c5f258410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:34552/0 (socket says 172.21.15.5:34552) 2024-09-10T09:42:37.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:37.966+0000 7f54f9400640 1 -- 172.21.15.5:0/1469246750 learned_addr learned my addr 172.21.15.5:0/1469246750 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:42:37.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:37.966+0000 7f54f2a00640 1 -- 172.21.15.5:0/1354103664 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f54ec0cf760 con 0x7f54fc100780 2024-09-10T09:42:37.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:37.966+0000 7f54f9400640 1 -- 172.21.15.5:0/1469246750 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557c5f24ae40 msgr2=0x557c5f24b220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:42:37.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:37.966+0000 7f54f9400640 1 --2- 172.21.15.5:0/1469246750 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557c5f24ae40 0x557c5f24b220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:37.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:37.966+0000 7f54f9400640 1 -- 172.21.15.5:0/1469246750 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557c5f24b760 msgr2=0x557c5f255ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:42:37.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:37.966+0000 7f54f9400640 1 --2- 172.21.15.5:0/1469246750 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557c5f24b760 0x557c5f255ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:37.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:37.966+0000 7f54f9400640 1 -- 172.21.15.5:0/1469246750 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557c5f177ae0 con 0x557c5f256020 2024-09-10T09:42:37.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:37.966+0000 7f54fa800640 1 --2- 172.21.15.5:0/1469246750 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557c5f24ae40 0x557c5f24b220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:42:37.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:37.966+0000 7f54fa800640 1 --2- 172.21.15.5:0/1354103664 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f54e00a2cd0 0x7f54e00a5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:42:37.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:37.966+0000 7f54f9400640 1 --2- 172.21.15.5:0/1469246750 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557c5f256020 0x557c5f258410 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f54e8013de0 tx=0x7f54e8009620 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8dd74ac6ab818dea server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:42:37.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:37.966+0000 7f54e7400640 1 -- 172.21.15.5:0/1469246750 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54e8020070 con 0x557c5f256020 2024-09-10T09:42:37.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:37.966+0000 7f54e7400640 1 -- 172.21.15.5:0/1469246750 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f54e8012cf0 con 0x557c5f256020 2024-09-10T09:42:37.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:37.966+0000 7f54e7400640 1 -- 172.21.15.5:0/1469246750 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54e801b9d0 con 0x557c5f256020 2024-09-10T09:42:37.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:37.966+0000 7f54f9e00640 1 --2- 172.21.15.5:0/1469246750 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557c5f24b760 0x557c5f255ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:42:37.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:37.966+0000 7f5503c4d740 1 -- 172.21.15.5:0/1469246750 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557c5f256020 msgr2=0x557c5f258410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:42:37.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:37.966+0000 7f54fa800640 1 --2- 172.21.15.5:0/1354103664 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f54e00a2cd0 0x7f54e00a5190 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f54f402ac10 tx=0x7f54f4009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:42:37.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:37.966+0000 7f5503c4d740 1 --2- 172.21.15.5:0/1469246750 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557c5f256020 0x557c5f258410 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f54e8013de0 tx=0x7f54e8009620 comp rx=0 tx=0).stop 2024-09-10T09:42:37.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:37.966+0000 7f5503c4d740 1 -- 172.21.15.5:0/1469246750 shutdown_connections 2024-09-10T09:42:37.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:37.966+0000 7f5503c4d740 1 --2- 172.21.15.5:0/1469246750 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557c5f256020 0x557c5f258410 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:37.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:37.966+0000 7f5503c4d740 1 --2- 172.21.15.5:0/1469246750 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557c5f24b760 0x557c5f255ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:37.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:37.966+0000 7f5503c4d740 1 --2- 172.21.15.5:0/1469246750 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557c5f24ae40 0x557c5f24b220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:37.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:37.966+0000 7f5503c4d740 1 -- 172.21.15.5:0/1469246750 >> 172.21.15.5:0/1469246750 conn(0x557c5f2473d0 msgr2=0x557c5f2477b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:42:37.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:37.966+0000 7f5503c4d740 1 -- 172.21.15.5:0/1469246750 shutdown_connections 2024-09-10T09:42:37.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:37.966+0000 7f5503c4d740 1 -- 172.21.15.5:0/1469246750 wait complete. 2024-09-10T09:42:37.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:37.966+0000 7f5503c4d740 1 -- start start 2024-09-10T09:42:37.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:37.966+0000 7f5503c4d740 1 Processor -- start 2024-09-10T09:42:37.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:37.966+0000 7f5503c4d740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557c5f24ae40 0x557c5f252ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:42:37.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:37.966+0000 7f5503c4d740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557c5f24b760 0x557c5f253410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:42:37.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:37.966+0000 7f5503c4d740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557c5f256020 0x557c5f253950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:42:37.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:37.966+0000 7f5503c4d740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x557c5f258bf0 con 0x557c5f24b760 2024-09-10T09:42:37.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:37.966+0000 7f5503c4d740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x557c5f16c0f0 con 0x557c5f24ae40 2024-09-10T09:42:37.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:37.966+0000 7f5503c4d740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x557c5f258d70 con 0x557c5f256020 2024-09-10T09:42:37.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:37.966+0000 7f5503c4d740 10 client.0 _ll_register_callbacks cb 0x557c5f153bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:42:37.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:37.966+0000 7f54f9400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557c5f24b760 0x557c5f253410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:42:37.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:37.966+0000 7f54f9400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557c5f24b760 0x557c5f253410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46242/0 (socket says 172.21.15.5:46242) 2024-09-10T09:42:37.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:37.966+0000 7f54f9400640 1 -- 172.21.15.5:0/728600243 learned_addr learned my addr 172.21.15.5:0/728600243 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:42:37.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:37.966+0000 7f54f9e00640 1 --2- 172.21.15.5:0/728600243 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557c5f24ae40 0x557c5f252ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:42:37.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:37.966+0000 7f54fa800640 1 --2- 172.21.15.5:0/728600243 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557c5f256020 0x557c5f253950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:42:37.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:37.966+0000 7f54f9400640 1 -- 172.21.15.5:0/728600243 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557c5f256020 msgr2=0x557c5f253950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:42:37.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:37.966+0000 7f54f9400640 1 --2- 172.21.15.5:0/728600243 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557c5f256020 0x557c5f253950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:37.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:37.966+0000 7f54f9400640 1 -- 172.21.15.5:0/728600243 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557c5f24ae40 msgr2=0x557c5f252ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:42:37.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:37.966+0000 7f54f9400640 1 --2- 172.21.15.5:0/728600243 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557c5f24ae40 0x557c5f252ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:37.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:37.966+0000 7f54f9400640 1 -- 172.21.15.5:0/728600243 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557c5ef223c0 con 0x557c5f24b760 2024-09-10T09:42:37.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:37.972+0000 7f54f9400640 1 --2- 172.21.15.5:0/728600243 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557c5f24b760 0x557c5f253410 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f54e8016040 tx=0x7f54e8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddad22ad2cce4287 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:42:37.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:37.972+0000 7f54dbe00640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-10T09:42:37.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:37.972+0000 7f54dbe00640 1 -- 172.21.15.5:0/728600243 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54e8029060 con 0x557c5f24b760 2024-09-10T09:42:37.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:37.972+0000 7f5503c4d740 1 -- 172.21.15.5:0/728600243 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f54e8012cf0 con 0x557c5f24b760 2024-09-10T09:42:37.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:37.972+0000 7f54dbe00640 1 -- 172.21.15.5:0/728600243 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f54e80138d0 con 0x557c5f24b760 2024-09-10T09:42:37.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:37.972+0000 7f54dbe00640 1 -- 172.21.15.5:0/728600243 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f54e8030050 con 0x557c5f24b760 2024-09-10T09:42:37.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:37.972+0000 7f5503c4d740 10 client.16236 fetch_fsmap learned FSMap version 33 2024-09-10T09:42:37.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:37.972+0000 7f5503c4d740 1 -- 172.21.15.5:0/728600243 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x557c5f264c00 con 0x557c5f24b760 2024-09-10T09:42:37.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:37.972+0000 7f54dbe00640 1 -- 172.21.15.5:0/728600243 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f54e8012cf0 con 0x557c5f24b760 2024-09-10T09:42:37.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:37.972+0000 7f54dbe00640 1 -- 172.21.15.5:0/728600243 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 33) ==== 8374+0+0 (secure 0 0 0) 0x7f54e8023070 con 0x557c5f24b760 2024-09-10T09:42:37.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:37.972+0000 7f5503c4d740 10 client.16236 fetch_fsmap finished waiting for FSMap version 33 2024-09-10T09:42:37.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:37.972+0000 7f5503c4d740 10 client.16236 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T09:42:37.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:37.972+0000 7f5503c4d740 20 client.16236 populate_metadata read hostname 'smithi005' 2024-09-10T09:42:37.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:37.972+0000 7f5503c4d740 1 --2- 172.21.15.5:0/728600243 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x557c5f265fb0 0x557c5f2683a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:42:37.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:37.972+0000 7f5503c4d740 4 client.16236 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:42:37.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:37.972+0000 7f5503c4d740 1 -- 172.21.15.5:0/728600243 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557c5f1be950 con 0x557c5f265fb0 2024-09-10T09:42:37.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:37.972+0000 7f54f9e00640 1 --2- 172.21.15.5:0/728600243 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x557c5f265fb0 0x557c5f2683a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:42:37.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:37.972+0000 7f54f9e00640 1 --2- 172.21.15.5:0/728600243 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x557c5f265fb0 0x557c5f2683a0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cbd079fd18b31baf server_cookie=20acb75553be2262 in_seq=0 out_seq=0 2024-09-10T09:42:37.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:37.972+0000 7f54dbe00640 10 client.16236 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T09:42:37.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:37.972+0000 7f54dbe00640 1 -- 172.21.15.5:0/728600243 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557c5f1be950 con 0x557c5f265fb0 2024-09-10T09:42:37.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:37.972+0000 7f54dbe00640 10 client.16236 handle_command_reply: tid=0 2024-09-10T09:42:37.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:37.972+0000 7f5503c4d740 1 client.16236 shutdown 2024-09-10T09:42:37.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:37.972+0000 7f5503c4d740 1 -- 172.21.15.5:0/728600243 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557c5f24b760 msgr2=0x557c5f253410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:42:37.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:37.972+0000 7f5503c4d740 1 --2- 172.21.15.5:0/728600243 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557c5f24b760 0x557c5f253410 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f54e8016040 tx=0x7f54e8004730 comp rx=0 tx=0).stop 2024-09-10T09:42:37.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:37.972+0000 7f5503c4d740 1 -- 172.21.15.5:0/728600243 shutdown_connections 2024-09-10T09:42:37.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:37.972+0000 7f5503c4d740 1 --2- 172.21.15.5:0/728600243 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x557c5f265fb0 0x557c5f2683a0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:37.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:37.972+0000 7f5503c4d740 1 --2- 172.21.15.5:0/728600243 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557c5f256020 0x557c5f253950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:37.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:37.972+0000 7f5503c4d740 1 --2- 172.21.15.5:0/728600243 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557c5f24b760 0x557c5f253410 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:37.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:37.972+0000 7f5503c4d740 1 --2- 172.21.15.5:0/728600243 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557c5f24ae40 0x557c5f252ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:37.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:37.972+0000 7f5503c4d740 1 -- 172.21.15.5:0/728600243 >> 172.21.15.5:0/728600243 conn(0x557c5f2473d0 msgr2=0x557c5f24d100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:42:37.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:37.972+0000 7f54dbe00640 0 client.16236 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T09:42:37.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:37.972+0000 7f5503c4d740 1 -- 172.21.15.5:0/728600243 shutdown_connections 2024-09-10T09:42:37.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:37.972+0000 7f5503c4d740 1 -- 172.21.15.5:0/728600243 wait complete. 2024-09-10T09:42:37.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:37.972+0000 7f5503c4d740 20 client.16236 trim_cache size 0 max 16384 2024-09-10T09:42:37.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:37.990+0000 7f5503c4d740 1 Processor -- start 2024-09-10T09:42:37.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:37.990+0000 7f5503c4d740 1 -- start start 2024-09-10T09:42:37.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:37.990+0000 7f5503c4d740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557c5f24ae40 0x557c5f242300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:42:37.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:37.990+0000 7f5503c4d740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557c5f24b760 0x557c5f26dae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:42:37.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:37.990+0000 7f5503c4d740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557c5f256020 0x557c5f26e020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:42:37.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:37.990+0000 7f5503c4d740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f54fc1acb00 con 0x557c5f24b760 2024-09-10T09:42:37.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:37.990+0000 7f5503c4d740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f54fc1ae670 con 0x557c5f256020 2024-09-10T09:42:37.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:37.990+0000 7f5503c4d740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f54fc1ad1d0 con 0x557c5f24ae40 2024-09-10T09:42:37.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:37.990+0000 7f54fa800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557c5f256020 0x557c5f26e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:42:37.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:37.990+0000 7f54fa800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557c5f256020 0x557c5f26e020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:34576/0 (socket says 172.21.15.5:34576) 2024-09-10T09:42:37.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:37.990+0000 7f54fa800640 1 -- 172.21.15.5:0/3101700326 learned_addr learned my addr 172.21.15.5:0/3101700326 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:42:37.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:37.990+0000 7f54fa800640 1 -- 172.21.15.5:0/3101700326 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557c5f24ae40 msgr2=0x557c5f242300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:42:37.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:37.990+0000 7f54f9e00640 1 --2- 172.21.15.5:0/3101700326 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557c5f24ae40 0x557c5f242300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:42:37.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:37.990+0000 7f54fa800640 1 --2- 172.21.15.5:0/3101700326 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557c5f24ae40 0x557c5f242300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:37.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:37.990+0000 7f54fa800640 1 -- 172.21.15.5:0/3101700326 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557c5f24b760 msgr2=0x557c5f26dae0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T09:42:37.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:37.990+0000 7f54fa800640 1 --2- 172.21.15.5:0/3101700326 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557c5f24b760 0x557c5f26dae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:37.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:37.990+0000 7f54fa800640 1 -- 172.21.15.5:0/3101700326 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557c5f251e80 con 0x557c5f256020 2024-09-10T09:42:37.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:37.990+0000 7f54fa800640 1 --2- 172.21.15.5:0/3101700326 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557c5f256020 0x557c5f26e020 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f54f4033000 tx=0x7f54f403f000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6a8b16f6f8f6bdee server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:42:37.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:37.990+0000 7f54f9e00640 1 --2- 172.21.15.5:0/3101700326 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557c5f24ae40 0x557c5f242300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T09:42:37.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:37.990+0000 7f54e6a00640 1 -- 172.21.15.5:0/3101700326 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54f4036040 con 0x557c5f256020 2024-09-10T09:42:37.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:37.990+0000 7f54e6a00640 1 -- 172.21.15.5:0/3101700326 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f54f404f030 con 0x557c5f256020 2024-09-10T09:42:37.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:37.990+0000 7f54e6a00640 1 -- 172.21.15.5:0/3101700326 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54f404bc90 con 0x557c5f256020 2024-09-10T09:42:37.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:37.990+0000 7f5503c4d740 1 -- 172.21.15.5:0/3101700326 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557c5f256020 msgr2=0x557c5f26e020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:42:37.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:37.990+0000 7f5503c4d740 1 --2- 172.21.15.5:0/3101700326 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557c5f256020 0x557c5f26e020 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f54f4033000 tx=0x7f54f403f000 comp rx=0 tx=0).stop 2024-09-10T09:42:37.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:37.990+0000 7f5503c4d740 1 -- 172.21.15.5:0/3101700326 shutdown_connections 2024-09-10T09:42:37.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:37.990+0000 7f5503c4d740 1 --2- 172.21.15.5:0/3101700326 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557c5f256020 0x557c5f26e020 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:37.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:37.990+0000 7f5503c4d740 1 --2- 172.21.15.5:0/3101700326 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557c5f24b760 0x557c5f26dae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:37.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:37.990+0000 7f5503c4d740 1 --2- 172.21.15.5:0/3101700326 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557c5f24ae40 0x557c5f242300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:37.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:37.990+0000 7f5503c4d740 1 -- 172.21.15.5:0/3101700326 >> 172.21.15.5:0/3101700326 conn(0x557c5f2473d0 msgr2=0x557c5f254f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:42:37.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:37.990+0000 7f5503c4d740 1 -- 172.21.15.5:0/3101700326 shutdown_connections 2024-09-10T09:42:37.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:37.990+0000 7f5503c4d740 1 -- 172.21.15.5:0/3101700326 wait complete. 2024-09-10T09:42:37.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:37.990+0000 7f5503c4d740 1 -- start start 2024-09-10T09:42:37.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:37.990+0000 7f5503c4d740 1 Processor -- start 2024-09-10T09:42:37.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:37.990+0000 7f5503c4d740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557c5f24ae40 0x557c5f259e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:42:37.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:37.990+0000 7f5503c4d740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557c5f24b760 0x557c5f25a390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:42:37.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:37.990+0000 7f5503c4d740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557c5f256020 0x557c5f25a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:42:37.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:37.990+0000 7f5503c4d740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f54fc1adf70 con 0x557c5f24b760 2024-09-10T09:42:37.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:37.990+0000 7f5503c4d740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f54fc1ad8a0 con 0x557c5f256020 2024-09-10T09:42:37.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:37.990+0000 7f5503c4d740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f54fc1aaae0 con 0x557c5f24ae40 2024-09-10T09:42:38.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:37.990+0000 7f5503c4d740 10 client.0 _ll_register_callbacks cb 0x7f54e801e070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:42:38.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:37.996+0000 7f54f9400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557c5f24b760 0x557c5f25a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:42:38.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:37.996+0000 7f54f9400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557c5f24b760 0x557c5f25a390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46258/0 (socket says 172.21.15.5:46258) 2024-09-10T09:42:38.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:37.996+0000 7f54f9400640 1 -- 172.21.15.5:0/3438540273 learned_addr learned my addr 172.21.15.5:0/3438540273 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:42:38.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:37.996+0000 7f54f9e00640 1 --2- 172.21.15.5:0/3438540273 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557c5f24ae40 0x557c5f259e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:42:38.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:37.996+0000 7f54f9400640 1 -- 172.21.15.5:0/3438540273 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557c5f24ae40 msgr2=0x557c5f259e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:42:38.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:37.996+0000 7f54f9400640 1 --2- 172.21.15.5:0/3438540273 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557c5f24ae40 0x557c5f259e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:38.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:37.996+0000 7f54f9400640 1 -- 172.21.15.5:0/3438540273 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557c5f256020 msgr2=0x557c5f25a8d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T09:42:38.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:37.996+0000 7f54f9400640 1 --2- 172.21.15.5:0/3438540273 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557c5f256020 0x557c5f25a8d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:38.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:37.996+0000 7f54f9400640 1 -- 172.21.15.5:0/3438540273 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557c5f2cfc10 con 0x557c5f24b760 2024-09-10T09:42:38.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:37.996+0000 7f54f9400640 1 --2- 172.21.15.5:0/3438540273 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557c5f24b760 0x557c5f25a390 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x557c5f254510 tx=0x7f54e8013c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a587db095e5da295 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:42:38.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:37.996+0000 7f54dbe00640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-10T09:42:38.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:37.996+0000 7f54dbe00640 1 -- 172.21.15.5:0/3438540273 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54e8029070 con 0x557c5f24b760 2024-09-10T09:42:38.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:37.996+0000 7f54dbe00640 1 -- 172.21.15.5:0/3438540273 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f54e8005970 con 0x557c5f24b760 2024-09-10T09:42:38.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:37.996+0000 7f5503c4d740 1 -- 172.21.15.5:0/3438540273 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f54f404f030 con 0x557c5f24b760 2024-09-10T09:42:38.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:37.996+0000 7f54dbe00640 1 -- 172.21.15.5:0/3438540273 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f54e8020040 con 0x557c5f24b760 2024-09-10T09:42:38.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:37.996+0000 7f5503c4d740 10 client.16239 fetch_fsmap learned FSMap version 33 2024-09-10T09:42:38.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:37.996+0000 7f5503c4d740 1 -- 172.21.15.5:0/3438540273 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x557c5f2d0e20 con 0x557c5f24b760 2024-09-10T09:42:38.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:37.996+0000 7f54dbe00640 1 -- 172.21.15.5:0/3438540273 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f54f404f030 con 0x557c5f24b760 2024-09-10T09:42:38.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:37.996+0000 7f54dbe00640 1 -- 172.21.15.5:0/3438540273 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 33) ==== 8374+0+0 (secure 0 0 0) 0x7f54e8023040 con 0x557c5f24b760 2024-09-10T09:42:38.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:37.996+0000 7f5503c4d740 10 client.16239 fetch_fsmap finished waiting for FSMap version 33 2024-09-10T09:42:38.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:37.996+0000 7f5503c4d740 10 client.16239 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T09:42:38.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:37.996+0000 7f5503c4d740 20 client.16239 populate_metadata read hostname 'smithi005' 2024-09-10T09:42:38.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:37.996+0000 7f5503c4d740 1 --2- 172.21.15.5:0/3438540273 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x557c5f2d15a0 0x557c5f2d39c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:42:38.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:37.996+0000 7f5503c4d740 4 client.16239 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T09:42:38.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:37.996+0000 7f5503c4d740 1 -- 172.21.15.5:0/3438540273 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557c5f1b8e60 con 0x557c5f2d15a0 2024-09-10T09:42:38.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:37.996+0000 7f54f9e00640 1 --2- 172.21.15.5:0/3438540273 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x557c5f2d15a0 0x557c5f2d39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:42:38.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:37.996+0000 7f54f9e00640 1 --2- 172.21.15.5:0/3438540273 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x557c5f2d15a0 0x557c5f2d39c0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=671f967979235da5 server_cookie=24bf9c732c72bb3c in_seq=0 out_seq=0 2024-09-10T09:42:38.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:37.996+0000 7f54dbe00640 10 client.16239 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T09:42:38.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:37.996+0000 7f54dbe00640 1 -- 172.21.15.5:0/3438540273 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557c5f1b8e60 con 0x557c5f2d15a0 2024-09-10T09:42:38.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:37.996+0000 7f54dbe00640 10 client.16239 handle_command_reply: tid=0 2024-09-10T09:42:38.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:37.996+0000 7f5503c4d740 1 client.16239 shutdown 2024-09-10T09:42:38.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:37.996+0000 7f5503c4d740 1 -- 172.21.15.5:0/3438540273 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557c5f24b760 msgr2=0x557c5f25a390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:42:38.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:37.996+0000 7f5503c4d740 1 --2- 172.21.15.5:0/3438540273 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557c5f24b760 0x557c5f25a390 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x557c5f254510 tx=0x7f54e8013c30 comp rx=0 tx=0).stop 2024-09-10T09:42:38.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:37.996+0000 7f5503c4d740 1 -- 172.21.15.5:0/3438540273 shutdown_connections 2024-09-10T09:42:38.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:37.996+0000 7f5503c4d740 1 --2- 172.21.15.5:0/3438540273 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x557c5f2d15a0 0x557c5f2d39c0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:38.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:37.996+0000 7f5503c4d740 1 --2- 172.21.15.5:0/3438540273 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557c5f256020 0x557c5f25a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:38.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:37.996+0000 7f54dbe00640 0 client.16239 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T09:42:38.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:37.996+0000 7f5503c4d740 1 --2- 172.21.15.5:0/3438540273 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557c5f24b760 0x557c5f25a390 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:38.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:37.996+0000 7f5503c4d740 1 --2- 172.21.15.5:0/3438540273 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557c5f24ae40 0x557c5f259e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:38.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:37.996+0000 7f5503c4d740 1 -- 172.21.15.5:0/3438540273 >> 172.21.15.5:0/3438540273 conn(0x557c5f2473d0 msgr2=0x557c5f24c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:42:38.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:37.996+0000 7f5503c4d740 1 -- 172.21.15.5:0/3438540273 shutdown_connections 2024-09-10T09:42:38.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:37.996+0000 7f5503c4d740 1 -- 172.21.15.5:0/3438540273 wait complete. 2024-09-10T09:42:38.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:37.996+0000 7f5503c4d740 20 client.16239 trim_cache size 0 max 16384 2024-09-10T09:42:38.007 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-10T09:42:38.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:38.001+0000 7f54fb200640 1 -- 172.21.15.5:0/1354103664 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f54e00a2cd0 msgr2=0x7f54e00a5190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:42:38.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:38.002+0000 7f54fb200640 1 --2- 172.21.15.5:0/1354103664 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f54e00a2cd0 0x7f54e00a5190 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f54f402ac10 tx=0x7f54f4009500 comp rx=0 tx=0).stop 2024-09-10T09:42:38.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:38.002+0000 7f54fb200640 1 -- 172.21.15.5:0/1354103664 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f54fc100780 msgr2=0x7f54fc19a500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:42:38.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:38.002+0000 7f54fb200640 1 --2- 172.21.15.5:0/1354103664 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f54fc100780 0x7f54fc19a500 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f54ec00e9c0 tx=0x7f54ec00ee80 comp rx=0 tx=0).stop 2024-09-10T09:42:38.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:38.002+0000 7f54fb200640 1 -- 172.21.15.5:0/1354103664 shutdown_connections 2024-09-10T09:42:38.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:38.002+0000 7f54fb200640 1 --2- 172.21.15.5:0/1354103664 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f54e00a2cd0 0x7f54e00a5190 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:38.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:38.002+0000 7f54fb200640 1 --2- 172.21.15.5:0/1354103664 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f54fc10e330 0x7f54fc19edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:38.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:38.002+0000 7f54fb200640 1 --2- 172.21.15.5:0/1354103664 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f54fc101130 0x7f54fc19aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:38.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:38.002+0000 7f54fb200640 1 --2- 172.21.15.5:0/1354103664 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f54fc100780 0x7f54fc19a500 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:38.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:38.002+0000 7f54fb200640 1 -- 172.21.15.5:0/1354103664 >> 172.21.15.5:0/1354103664 conn(0x7f54fc0fc610 msgr2=0x7f54fc0fea00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:42:38.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:38.002+0000 7f54fb200640 1 -- 172.21.15.5:0/1354103664 shutdown_connections 2024-09-10T09:42:38.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:38.002+0000 7f54fb200640 1 -- 172.21.15.5:0/1354103664 wait complete. 2024-09-10T09:42:38.025 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:42:38.026 INFO:tasks.workunit.client.0.smithi005.stdout: 1 191939 0.00 MB/sec execute 19 sec latency 22931.528 ms 2024-09-10T09:42:38.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:42:38 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:42:38.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:42:38 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:42:39.026 INFO:tasks.workunit.client.0.smithi005.stdout: 1 191939 0.00 MB/sec execute 20 sec latency 23931.604 ms 2024-09-10T09:42:39.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:42:38 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:42:39.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:39.524+0000 7f34ffa00640 1 -- 172.21.15.5:0/2395548196 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f34f8074590 msgr2=0x7f34f8074970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:42:39.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:39.524+0000 7f34ffa00640 1 --2- 172.21.15.5:0/2395548196 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f34f8074590 0x7f34f8074970 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f34ec009900 tx=0x7f34ec02a480 comp rx=0 tx=0).stop 2024-09-10T09:42:39.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:39.525+0000 7f34ffa00640 1 -- 172.21.15.5:0/2395548196 shutdown_connections 2024-09-10T09:42:39.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:39.525+0000 7f34ffa00640 1 --2- 172.21.15.5:0/2395548196 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f34f8112b80 0x7f34f8114f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:39.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:39.525+0000 7f34ffa00640 1 --2- 172.21.15.5:0/2395548196 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f34f8074eb0 0x7f34f8112640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:39.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:39.525+0000 7f34ffa00640 1 --2- 172.21.15.5:0/2395548196 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f34f8074590 0x7f34f8074970 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:39.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:39.525+0000 7f34ffa00640 1 -- 172.21.15.5:0/2395548196 >> 172.21.15.5:0/2395548196 conn(0x7f34f80fd8b0 msgr2=0x7f34f80ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:42:39.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:39.525+0000 7f34ffa00640 1 -- 172.21.15.5:0/2395548196 shutdown_connections 2024-09-10T09:42:39.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:39.525+0000 7f34ffa00640 1 -- 172.21.15.5:0/2395548196 wait complete. 2024-09-10T09:42:39.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:39.526+0000 7f34ffa00640 1 Processor -- start 2024-09-10T09:42:39.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:39.526+0000 7f34ffa00640 1 -- start start 2024-09-10T09:42:39.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:39.526+0000 7f34ffa00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f34f8074590 0x7f34f81a46d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:42:39.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:39.526+0000 7f34ffa00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f34f8074eb0 0x7f34f81a4c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:42:39.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:39.526+0000 7f34ffa00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f34f8112b80 0x7f34f819e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:42:39.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:39.526+0000 7f34ffa00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f34f8118540 con 0x7f34f8112b80 2024-09-10T09:42:39.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:39.526+0000 7f34ffa00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f34f81183c0 con 0x7f34f8074590 2024-09-10T09:42:39.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:39.526+0000 7f34ffa00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f34f81186c0 con 0x7f34f8074eb0 2024-09-10T09:42:39.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:39.526+0000 7f34fda00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f34f8112b80 0x7f34f819e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:42:39.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:39.527+0000 7f34f7e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f34f8074eb0 0x7f34f81a4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:42:39.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:39.527+0000 7f34fda00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f34f8112b80 0x7f34f819e890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46290/0 (socket says 172.21.15.5:46290) 2024-09-10T09:42:39.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:39.527+0000 7f34fda00640 1 -- 172.21.15.5:0/441296391 learned_addr learned my addr 172.21.15.5:0/441296391 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:42:39.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:39.527+0000 7f34fd000640 1 --2- 172.21.15.5:0/441296391 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f34f8074590 0x7f34f81a46d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:42:39.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:39.527+0000 7f34f7e00640 1 -- 172.21.15.5:0/441296391 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f34f8074590 msgr2=0x7f34f81a46d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:42:39.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:39.527+0000 7f34f7e00640 1 --2- 172.21.15.5:0/441296391 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f34f8074590 0x7f34f81a46d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:39.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:39.527+0000 7f34f7e00640 1 -- 172.21.15.5:0/441296391 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f34f8112b80 msgr2=0x7f34f819e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:42:39.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:39.527+0000 7f34f7e00640 1 --2- 172.21.15.5:0/441296391 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f34f8112b80 0x7f34f819e890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:39.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:39.527+0000 7f34f7e00640 1 -- 172.21.15.5:0/441296391 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34f819efc0 con 0x7f34f8074eb0 2024-09-10T09:42:39.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:39.527+0000 7f34fda00640 1 --2- 172.21.15.5:0/441296391 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f34f8112b80 0x7f34f819e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:42:39.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:39.527+0000 7f34f7e00640 1 --2- 172.21.15.5:0/441296391 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f34f8074eb0 0x7f34f81a4c10 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f34e400e9f0 tx=0x7f34e400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:42:39.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:39.528+0000 7f34f5600640 1 -- 172.21.15.5:0/441296391 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34e400ccc0 con 0x7f34f8074eb0 2024-09-10T09:42:39.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:39.528+0000 7f34ffa00640 1 -- 172.21.15.5:0/441296391 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34f819f2b0 con 0x7f34f8074eb0 2024-09-10T09:42:39.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:39.528+0000 7f34f5600640 1 -- 172.21.15.5:0/441296391 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f34e4010910 con 0x7f34f8074eb0 2024-09-10T09:42:39.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:39.528+0000 7f34f5600640 1 -- 172.21.15.5:0/441296391 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34e4018960 con 0x7f34f8074eb0 2024-09-10T09:42:39.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:39.528+0000 7f34ffa00640 1 -- 172.21.15.5:0/441296391 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34f81ab5a0 con 0x7f34f8074eb0 2024-09-10T09:42:39.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:39.531+0000 7f35025f4740 1 Processor -- start 2024-09-10T09:42:39.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:39.531+0000 7f35025f4740 1 -- start start 2024-09-10T09:42:39.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:39.532+0000 7f35025f4740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557ae3351310 0x557ae33516f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:42:39.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:39.532+0000 7f35025f4740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557ae3351c30 0x557ae335bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:42:39.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:39.532+0000 7f35025f4740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557ae335c4f0 0x557ae335e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:42:39.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:39.532+0000 7f35025f4740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x557ae32b7650 con 0x557ae3351c30 2024-09-10T09:42:39.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:39.532+0000 7f35025f4740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x557ae325ef10 con 0x557ae335c4f0 2024-09-10T09:42:39.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:39.532+0000 7f35025f4740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x557ae32b3610 con 0x557ae3351310 2024-09-10T09:42:39.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:39.532+0000 7f34fd000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557ae3351c30 0x557ae335bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:42:39.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:39.532+0000 7f34fd000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557ae3351c30 0x557ae335bfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46304/0 (socket says 172.21.15.5:46304) 2024-09-10T09:42:39.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:39.532+0000 7f34fda00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557ae3351310 0x557ae33516f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:42:39.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:39.532+0000 7f34fd000640 1 -- 172.21.15.5:0/1787350397 learned_addr learned my addr 172.21.15.5:0/1787350397 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:42:39.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:39.532+0000 7f34fd000640 1 -- 172.21.15.5:0/1787350397 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557ae3351310 msgr2=0x557ae33516f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:42:39.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:39.532+0000 7f34fd000640 1 --2- 172.21.15.5:0/1787350397 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557ae3351310 0x557ae33516f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:39.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:39.532+0000 7f34fd000640 1 -- 172.21.15.5:0/1787350397 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557ae335c4f0 msgr2=0x557ae335e8e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T09:42:39.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:39.532+0000 7f34fd000640 1 --2- 172.21.15.5:0/1787350397 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557ae335c4f0 0x557ae335e8e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:39.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:39.532+0000 7f34fd000640 1 -- 172.21.15.5:0/1787350397 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557ae2fc6580 con 0x557ae3351c30 2024-09-10T09:42:39.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:39.532+0000 7f34fda00640 1 --2- 172.21.15.5:0/1787350397 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557ae3351310 0x557ae33516f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:42:39.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:39.533+0000 7f34fd000640 1 --2- 172.21.15.5:0/1787350397 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557ae3351c30 0x557ae335bfb0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f34ec009900 tx=0x7f34ec005be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79d0f964260849c0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:42:39.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:39.533+0000 7f34f5600640 1 -- 172.21.15.5:0/441296391 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f34e4010ac0 con 0x7f34f8074eb0 2024-09-10T09:42:39.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:39.533+0000 7f34ea000640 1 -- 172.21.15.5:0/1787350397 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34ec040070 con 0x557ae3351c30 2024-09-10T09:42:39.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:39.533+0000 7f34ea000640 1 -- 172.21.15.5:0/1787350397 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f34ec031870 con 0x557ae3351c30 2024-09-10T09:42:39.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:39.533+0000 7f34ea000640 1 -- 172.21.15.5:0/1787350397 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34ec035040 con 0x557ae3351c30 2024-09-10T09:42:39.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:39.534+0000 7f35025f4740 1 -- 172.21.15.5:0/1787350397 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557ae3351c30 msgr2=0x557ae335bfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:42:39.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:39.534+0000 7f35025f4740 1 --2- 172.21.15.5:0/1787350397 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557ae3351c30 0x557ae335bfb0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f34ec009900 tx=0x7f34ec005be0 comp rx=0 tx=0).stop 2024-09-10T09:42:39.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:39.534+0000 7f35025f4740 1 -- 172.21.15.5:0/1787350397 shutdown_connections 2024-09-10T09:42:39.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:39.534+0000 7f35025f4740 1 --2- 172.21.15.5:0/1787350397 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557ae335c4f0 0x557ae335e8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:39.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:39.534+0000 7f35025f4740 1 --2- 172.21.15.5:0/1787350397 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557ae3351c30 0x557ae335bfb0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:39.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:39.534+0000 7f35025f4740 1 --2- 172.21.15.5:0/1787350397 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557ae3351310 0x557ae33516f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:39.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:39.534+0000 7f35025f4740 1 -- 172.21.15.5:0/1787350397 >> 172.21.15.5:0/1787350397 conn(0x557ae334d8a0 msgr2=0x557ae334dc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:42:39.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:39.534+0000 7f35025f4740 1 -- 172.21.15.5:0/1787350397 shutdown_connections 2024-09-10T09:42:39.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:39.534+0000 7f35025f4740 1 -- 172.21.15.5:0/1787350397 wait complete. 2024-09-10T09:42:39.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:39.535+0000 7f35025f4740 1 -- start start 2024-09-10T09:42:39.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:39.535+0000 7f34f5600640 1 --2- 172.21.15.5:0/441296391 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f34e00a2a90 0x7f34e00a4f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:42:39.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:39.535+0000 7f35025f4740 1 Processor -- start 2024-09-10T09:42:39.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:39.535+0000 7f34f5600640 1 -- 172.21.15.5:0/441296391 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f34e40cf820 con 0x7f34f8074eb0 2024-09-10T09:42:39.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:39.536+0000 7f34fda00640 1 --2- 172.21.15.5:0/441296391 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f34e00a2a90 0x7f34e00a4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:42:39.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:39.536+0000 7f35025f4740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557ae3351310 0x557ae335ac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:42:39.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:39.536+0000 7f35025f4740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557ae3351c30 0x557ae335b170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:42:39.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:39.536+0000 7f35025f4740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557ae335c4f0 0x557ae335b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:42:39.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:39.536+0000 7f35025f4740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x557ae335f0c0 con 0x557ae335c4f0 2024-09-10T09:42:39.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:39.536+0000 7f35025f4740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x557ae3272350 con 0x557ae3351c30 2024-09-10T09:42:39.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:39.536+0000 7f35025f4740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x557ae335f240 con 0x557ae3351310 2024-09-10T09:42:39.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:39.536+0000 7f35025f4740 10 client.0 _ll_register_callbacks cb 0x557ae3295290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:42:39.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:39.536+0000 7f34f7e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557ae3351c30 0x557ae335b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:42:39.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:39.536+0000 7f34f7e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557ae3351c30 0x557ae335b170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:34624/0 (socket says 172.21.15.5:34624) 2024-09-10T09:42:39.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:39.536+0000 7f34f7e00640 1 -- 172.21.15.5:0/620508975 learned_addr learned my addr 172.21.15.5:0/620508975 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:42:39.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:39.536+0000 7f34fd000640 1 --2- 172.21.15.5:0/620508975 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557ae3351310 0x557ae335ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:42:39.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:39.536+0000 7f34fda00640 1 --2- 172.21.15.5:0/620508975 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557ae335c4f0 0x557ae335b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:42:39.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:39.537+0000 7f34fda00640 1 --2- 172.21.15.5:0/441296391 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f34e00a2a90 0x7f34e00a4f50 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f34f000e910 tx=0x7f34f0009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:42:39.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:39.537+0000 7f34f7e00640 1 -- 172.21.15.5:0/620508975 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557ae3351310 msgr2=0x557ae335ac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:42:39.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:39.537+0000 7f34f7e00640 1 --2- 172.21.15.5:0/620508975 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557ae3351310 0x557ae335ac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:39.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:39.537+0000 7f34f7e00640 1 -- 172.21.15.5:0/620508975 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557ae335c4f0 msgr2=0x557ae335b6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:42:39.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:39.537+0000 7f34f7e00640 1 --2- 172.21.15.5:0/620508975 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557ae335c4f0 0x557ae335b6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:39.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:39.537+0000 7f34f7e00640 1 -- 172.21.15.5:0/620508975 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557ae3090960 con 0x557ae3351c30 2024-09-10T09:42:39.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:39.537+0000 7f34f7e00640 1 --2- 172.21.15.5:0/620508975 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557ae3351c30 0x557ae335b170 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f34e409bc70 tx=0x7f34e409bda0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=91d89ea981d0faa4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:42:39.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:39.537+0000 7f34dea00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-09-10T09:42:39.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:39.537+0000 7f34dea00640 1 -- 172.21.15.5:0/620508975 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34e409b330 con 0x557ae3351c30 2024-09-10T09:42:39.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:39.537+0000 7f35025f4740 1 -- 172.21.15.5:0/620508975 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f34ec031870 con 0x557ae3351c30 2024-09-10T09:42:39.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:39.538+0000 7f34dea00640 1 -- 172.21.15.5:0/620508975 <== mon.2 v2:172.21.15.102:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f34e4010430 con 0x557ae3351c30 2024-09-10T09:42:39.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:39.538+0000 7f34dea00640 1 -- 172.21.15.5:0/620508975 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f34e4010630 con 0x557ae3351c30 2024-09-10T09:42:39.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:39.538+0000 7f35025f4740 10 client.26246 fetch_fsmap learned FSMap version 33 2024-09-10T09:42:39.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:39.538+0000 7f35025f4740 1 -- 172.21.15.5:0/620508975 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x557ae336ad90 con 0x557ae3351c30 2024-09-10T09:42:39.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:39.538+0000 7f34dea00640 1 -- 172.21.15.5:0/620508975 <== mon.2 v2:172.21.15.102:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f34ec031870 con 0x557ae3351c30 2024-09-10T09:42:39.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:39.538+0000 7f34dea00640 1 -- 172.21.15.5:0/620508975 <== mon.2 v2:172.21.15.102:3300/0 5 ==== fsmap(e 33) ==== 8374+0+0 (secure 0 0 0) 0x7f34e407ace0 con 0x557ae3351c30 2024-09-10T09:42:39.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:39.538+0000 7f35025f4740 10 client.26246 fetch_fsmap finished waiting for FSMap version 33 2024-09-10T09:42:39.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:39.538+0000 7f35025f4740 10 client.26246 resolve_mds: validated gid 24542 aka daemon mds.b 2024-09-10T09:42:39.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:39.538+0000 7f35025f4740 20 client.26246 populate_metadata read hostname 'smithi005' 2024-09-10T09:42:39.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:39.538+0000 7f35025f4740 1 --2- 172.21.15.5:0/620508975 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x557ae336c1a0 0x557ae336e590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:42:39.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:39.538+0000 7f35025f4740 4 client.26246 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:42:39.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:39.538+0000 7f35025f4740 1 -- 172.21.15.5:0/620508975 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557ae32c4bc0 con 0x557ae336c1a0 2024-09-10T09:42:39.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:39.539+0000 7f34fd000640 1 --2- 172.21.15.5:0/620508975 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x557ae336c1a0 0x557ae336e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:42:39.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:39.539+0000 7f34fd000640 1 --2- 172.21.15.5:0/620508975 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x557ae336c1a0 0x557ae336e590 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=56bacb05a7e4214c server_cookie=5639ff3a1a5fd3cf in_seq=0 out_seq=0 2024-09-10T09:42:39.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:39.539+0000 7f34dea00640 10 client.26246 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T09:42:39.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:39.540+0000 7f34dea00640 1 -- 172.21.15.5:0/620508975 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557ae32c4bc0 con 0x557ae336c1a0 2024-09-10T09:42:39.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:39.540+0000 7f34dea00640 10 client.26246 handle_command_reply: tid=0 2024-09-10T09:42:39.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:39.540+0000 7f35025f4740 1 client.26246 shutdown 2024-09-10T09:42:39.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:39.541+0000 7f35025f4740 1 -- 172.21.15.5:0/620508975 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557ae3351c30 msgr2=0x557ae335b170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:42:39.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:39.541+0000 7f35025f4740 1 --2- 172.21.15.5:0/620508975 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557ae3351c30 0x557ae335b170 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f34e409bc70 tx=0x7f34e409bda0 comp rx=0 tx=0).stop 2024-09-10T09:42:39.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:39.541+0000 7f35025f4740 1 -- 172.21.15.5:0/620508975 shutdown_connections 2024-09-10T09:42:39.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:39.541+0000 7f35025f4740 1 --2- 172.21.15.5:0/620508975 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x557ae336c1a0 0x557ae336e590 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:39.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:39.541+0000 7f35025f4740 1 --2- 172.21.15.5:0/620508975 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557ae335c4f0 0x557ae335b6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:39.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:39.541+0000 7f35025f4740 1 --2- 172.21.15.5:0/620508975 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557ae3351c30 0x557ae335b170 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:39.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:39.541+0000 7f35025f4740 1 --2- 172.21.15.5:0/620508975 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557ae3351310 0x557ae335ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:39.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:39.541+0000 7f35025f4740 1 -- 172.21.15.5:0/620508975 >> 172.21.15.5:0/620508975 conn(0x557ae334d8a0 msgr2=0x557ae334caa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:42:39.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:39.541+0000 7f34dea00640 0 client.26246 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T09:42:39.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:39.541+0000 7f35025f4740 1 -- 172.21.15.5:0/620508975 shutdown_connections 2024-09-10T09:42:39.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:39.541+0000 7f35025f4740 1 -- 172.21.15.5:0/620508975 wait complete. 2024-09-10T09:42:39.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:39.541+0000 7f35025f4740 20 client.26246 trim_cache size 0 max 16384 2024-09-10T09:42:39.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:39.559+0000 7f35025f4740 1 Processor -- start 2024-09-10T09:42:39.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:39.559+0000 7f35025f4740 1 -- start start 2024-09-10T09:42:39.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:39.560+0000 7f35025f4740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557ae3351310 0x557ae3357d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:42:39.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:39.560+0000 7f35025f4740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557ae3351c30 0x557ae3373e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:42:39.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:39.560+0000 7f35025f4740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557ae335c4f0 0x557ae3374350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:42:39.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:39.560+0000 7f35025f4740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f34f81b5510 con 0x557ae3351310 2024-09-10T09:42:39.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:39.560+0000 7f35025f4740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f34f81b7080 con 0x557ae335c4f0 2024-09-10T09:42:39.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:39.560+0000 7f35025f4740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f34f81b5be0 con 0x557ae3351c30 2024-09-10T09:42:39.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:39.560+0000 7f34fd000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557ae3351310 0x557ae3357d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:42:39.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:39.560+0000 7f34fd000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557ae3351310 0x557ae3357d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46332/0 (socket says 172.21.15.5:46332) 2024-09-10T09:42:39.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:39.560+0000 7f34fd000640 1 -- 172.21.15.5:0/2580756103 learned_addr learned my addr 172.21.15.5:0/2580756103 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:42:39.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:39.560+0000 7f34f7e00640 1 --2- 172.21.15.5:0/2580756103 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557ae3351c30 0x557ae3373e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:42:39.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:39.560+0000 7f34fda00640 1 --2- 172.21.15.5:0/2580756103 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557ae335c4f0 0x557ae3374350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:42:39.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:39.560+0000 7f34fd000640 1 -- 172.21.15.5:0/2580756103 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557ae3351c30 msgr2=0x557ae3373e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:42:39.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:39.560+0000 7f34fd000640 1 --2- 172.21.15.5:0/2580756103 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557ae3351c30 0x557ae3373e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:39.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:39.560+0000 7f34fd000640 1 -- 172.21.15.5:0/2580756103 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557ae335c4f0 msgr2=0x557ae3374350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:42:39.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:39.560+0000 7f34fd000640 1 --2- 172.21.15.5:0/2580756103 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557ae335c4f0 0x557ae3374350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:39.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:39.560+0000 7f34fd000640 1 -- 172.21.15.5:0/2580756103 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557ae33521a0 con 0x557ae3351310 2024-09-10T09:42:39.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:39.561+0000 7f34fd000640 1 --2- 172.21.15.5:0/2580756103 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557ae3351310 0x557ae3357d40 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f34ec009230 tx=0x7f34ec009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b8bfc6318a04a5c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:42:39.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:39.561+0000 7f34e9600640 1 -- 172.21.15.5:0/2580756103 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34ec040040 con 0x557ae3351310 2024-09-10T09:42:39.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:39.561+0000 7f34e9600640 1 -- 172.21.15.5:0/2580756103 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f34ec0037f0 con 0x557ae3351310 2024-09-10T09:42:39.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:39.561+0000 7f34e9600640 1 -- 172.21.15.5:0/2580756103 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34ec035040 con 0x557ae3351310 2024-09-10T09:42:39.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:39.562+0000 7f35025f4740 1 -- 172.21.15.5:0/2580756103 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557ae3351310 msgr2=0x557ae3357d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:42:39.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:39.562+0000 7f35025f4740 1 --2- 172.21.15.5:0/2580756103 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557ae3351310 0x557ae3357d40 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f34ec009230 tx=0x7f34ec009260 comp rx=0 tx=0).stop 2024-09-10T09:42:39.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:39.562+0000 7f35025f4740 1 -- 172.21.15.5:0/2580756103 shutdown_connections 2024-09-10T09:42:39.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:39.562+0000 7f35025f4740 1 --2- 172.21.15.5:0/2580756103 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557ae335c4f0 0x557ae3374350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:39.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:39.562+0000 7f35025f4740 1 --2- 172.21.15.5:0/2580756103 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557ae3351c30 0x557ae3373e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:39.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:39.562+0000 7f35025f4740 1 --2- 172.21.15.5:0/2580756103 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557ae3351310 0x557ae3357d40 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:39.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:39.562+0000 7f35025f4740 1 -- 172.21.15.5:0/2580756103 >> 172.21.15.5:0/2580756103 conn(0x557ae334d8a0 msgr2=0x557ae3353580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:42:39.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:39.562+0000 7f35025f4740 1 -- 172.21.15.5:0/2580756103 shutdown_connections 2024-09-10T09:42:39.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:39.562+0000 7f35025f4740 1 -- 172.21.15.5:0/2580756103 wait complete. 2024-09-10T09:42:39.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:39.563+0000 7f35025f4740 1 -- start start 2024-09-10T09:42:39.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:39.563+0000 7f35025f4740 1 Processor -- start 2024-09-10T09:42:39.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:39.563+0000 7f35025f4740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557ae3351310 0x557ae3360310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:42:39.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:39.563+0000 7f35025f4740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557ae3351c30 0x557ae3360850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:42:39.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:39.563+0000 7f35025f4740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557ae335c4f0 0x557ae3360d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:42:39.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:39.563+0000 7f35025f4740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f34f81b6980 con 0x557ae3351c30 2024-09-10T09:42:39.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:39.563+0000 7f35025f4740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f34f81b62b0 con 0x557ae335c4f0 2024-09-10T09:42:39.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:39.563+0000 7f35025f4740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f34f81b34f0 con 0x557ae3351310 2024-09-10T09:42:39.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:39.563+0000 7f35025f4740 10 client.0 _ll_register_callbacks cb 0x7f34e400ce60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:42:39.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:39.564+0000 7f34fda00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557ae335c4f0 0x557ae3360d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:42:39.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:39.564+0000 7f34fda00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557ae335c4f0 0x557ae3360d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:34636/0 (socket says 172.21.15.5:34636) 2024-09-10T09:42:39.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:39.564+0000 7f34fda00640 1 -- 172.21.15.5:0/501012700 learned_addr learned my addr 172.21.15.5:0/501012700 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:42:39.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:39.564+0000 7f34f7e00640 1 --2- 172.21.15.5:0/501012700 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557ae3351c30 0x557ae3360850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:42:39.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:39.564+0000 7f34fd000640 1 --2- 172.21.15.5:0/501012700 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557ae3351310 0x557ae3360310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:42:39.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:39.564+0000 7f34fd000640 1 -- 172.21.15.5:0/501012700 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557ae335c4f0 msgr2=0x557ae3360d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:42:39.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:39.564+0000 7f34fd000640 1 --2- 172.21.15.5:0/501012700 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557ae335c4f0 0x557ae3360d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:39.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:39.564+0000 7f34fd000640 1 -- 172.21.15.5:0/501012700 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557ae3351c30 msgr2=0x557ae3360850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:42:39.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:39.564+0000 7f34fd000640 1 --2- 172.21.15.5:0/501012700 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557ae3351c30 0x557ae3360850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:39.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:39.564+0000 7f34fd000640 1 -- 172.21.15.5:0/501012700 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557ae33d5f40 con 0x557ae3351310 2024-09-10T09:42:39.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:39.564+0000 7f34fd000640 1 --2- 172.21.15.5:0/501012700 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557ae3351310 0x557ae3360310 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f34ec02a9a0 tx=0x7f34ec031960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4ac5da195bcd6b2b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:42:39.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:39.564+0000 7f34dea00640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-09-10T09:42:39.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:39.565+0000 7f34dea00640 1 -- 172.21.15.5:0/501012700 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34ec035070 con 0x557ae3351310 2024-09-10T09:42:39.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:39.565+0000 7f34dea00640 1 -- 172.21.15.5:0/501012700 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f34ec0046c0 con 0x557ae3351310 2024-09-10T09:42:39.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:39.565+0000 7f35025f4740 1 -- 172.21.15.5:0/501012700 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f34ec0037f0 con 0x557ae3351310 2024-09-10T09:42:39.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:39.565+0000 7f34dea00640 1 -- 172.21.15.5:0/501012700 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f34ec040040 con 0x557ae3351310 2024-09-10T09:42:39.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:39.565+0000 7f35025f4740 10 client.26272 fetch_fsmap learned FSMap version 33 2024-09-10T09:42:39.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:39.565+0000 7f35025f4740 1 -- 172.21.15.5:0/501012700 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x557ae33d7220 con 0x557ae3351310 2024-09-10T09:42:39.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:39.565+0000 7f34dea00640 1 -- 172.21.15.5:0/501012700 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f34ec0037f0 con 0x557ae3351310 2024-09-10T09:42:39.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:39.565+0000 7f34dea00640 1 -- 172.21.15.5:0/501012700 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 33) ==== 8374+0+0 (secure 0 0 0) 0x7f34ec03c5e0 con 0x557ae3351310 2024-09-10T09:42:39.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:39.566+0000 7f35025f4740 10 client.26272 fetch_fsmap finished waiting for FSMap version 33 2024-09-10T09:42:39.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:39.566+0000 7f35025f4740 10 client.26272 resolve_mds: validated gid 24542 aka daemon mds.b 2024-09-10T09:42:39.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:39.566+0000 7f35025f4740 20 client.26272 populate_metadata read hostname 'smithi005' 2024-09-10T09:42:39.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:39.566+0000 7f35025f4740 1 --2- 172.21.15.5:0/501012700 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x557ae33d74a0 0x557ae33d9890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:42:39.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:39.566+0000 7f35025f4740 4 client.26272 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "ba5a0a38", "release": true, "await": true}] 2024-09-10T09:42:39.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:39.566+0000 7f35025f4740 1 -- 172.21.15.5:0/501012700 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "quiesce db", "set_id": "ba5a0a38", "release": true, "await": true}) -- 0x557ae32bf0d0 con 0x557ae33d74a0 2024-09-10T09:42:39.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:39.566+0000 7f34f7e00640 1 --2- 172.21.15.5:0/501012700 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x557ae33d74a0 0x557ae33d9890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:42:39.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:39.566+0000 7f34f7e00640 1 --2- 172.21.15.5:0/501012700 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x557ae33d74a0 0x557ae33d9890 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=101cebfa751db817 server_cookie=7753cddee094621b in_seq=0 out_seq=0 2024-09-10T09:42:39.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:39.567+0000 7f34dea00640 10 client.26272 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T09:42:39.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:39.575+0000 7f34dea00640 1 -- 172.21.15.5:0/501012700 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x557ae32bf0d0 con 0x557ae33d74a0 2024-09-10T09:42:39.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:39.575+0000 7f34dea00640 10 client.26272 handle_command_reply: tid=0 2024-09-10T09:42:39.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:39.575+0000 7f35025f4740 1 client.26272 shutdown 2024-09-10T09:42:39.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:39.576+0000 7f35025f4740 1 -- 172.21.15.5:0/501012700 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557ae3351310 msgr2=0x557ae3360310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:42:39.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:39.576+0000 7f35025f4740 1 --2- 172.21.15.5:0/501012700 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557ae3351310 0x557ae3360310 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f34ec02a9a0 tx=0x7f34ec031960 comp rx=0 tx=0).stop 2024-09-10T09:42:39.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:39.576+0000 7f35025f4740 1 -- 172.21.15.5:0/501012700 shutdown_connections 2024-09-10T09:42:39.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:39.576+0000 7f35025f4740 1 --2- 172.21.15.5:0/501012700 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x557ae33d74a0 0x557ae33d9890 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:39.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:39.576+0000 7f35025f4740 1 --2- 172.21.15.5:0/501012700 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557ae335c4f0 0x557ae3360d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:39.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:39.576+0000 7f35025f4740 1 --2- 172.21.15.5:0/501012700 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557ae3351c30 0x557ae3360850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:39.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:39.576+0000 7f34dea00640 0 client.26272 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T09:42:39.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:39.576+0000 7f35025f4740 1 --2- 172.21.15.5:0/501012700 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557ae3351310 0x557ae3360310 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:39.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:39.576+0000 7f35025f4740 1 -- 172.21.15.5:0/501012700 >> 172.21.15.5:0/501012700 conn(0x557ae334d8a0 msgr2=0x557ae3359490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:42:39.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:39.576+0000 7f35025f4740 1 -- 172.21.15.5:0/501012700 shutdown_connections 2024-09-10T09:42:39.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:39.576+0000 7f35025f4740 1 -- 172.21.15.5:0/501012700 wait complete. 2024-09-10T09:42:39.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:39.576+0000 7f35025f4740 20 client.26272 trim_cache size 0 max 16384 2024-09-10T09:42:39.578 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-10T09:42:39.578 INFO:teuthology.orchestra.run.smithi005.stdout: "epoch": 33, 2024-09-10T09:42:39.578 INFO:teuthology.orchestra.run.smithi005.stdout: "leader": 24542, 2024-09-10T09:42:39.579 INFO:teuthology.orchestra.run.smithi005.stdout: "set_version": 12, 2024-09-10T09:42:39.579 INFO:teuthology.orchestra.run.smithi005.stdout: "sets": { 2024-09-10T09:42:39.579 INFO:teuthology.orchestra.run.smithi005.stdout: "ba5a0a38": { 2024-09-10T09:42:39.579 INFO:teuthology.orchestra.run.smithi005.stdout: "version": 12, 2024-09-10T09:42:39.579 INFO:teuthology.orchestra.run.smithi005.stdout: "age_ref": 0.0, 2024-09-10T09:42:39.579 INFO:teuthology.orchestra.run.smithi005.stdout: "state": { 2024-09-10T09:42:39.579 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "RELEASED", 2024-09-10T09:42:39.579 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0 2024-09-10T09:42:39.579 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T09:42:39.579 INFO:teuthology.orchestra.run.smithi005.stdout: "timeout": 90.0, 2024-09-10T09:42:39.579 INFO:teuthology.orchestra.run.smithi005.stdout: "expiration": 142.8, 2024-09-10T09:42:39.579 INFO:teuthology.orchestra.run.smithi005.stdout: "members": { 2024-09-10T09:42:39.579 INFO:teuthology.orchestra.run.smithi005.stdout: "file:/": { 2024-09-10T09:42:39.579 INFO:teuthology.orchestra.run.smithi005.stdout: "excluded": false, 2024-09-10T09:42:39.580 INFO:teuthology.orchestra.run.smithi005.stdout: "state": { 2024-09-10T09:42:39.580 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "RELEASED", 2024-09-10T09:42:39.580 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0 2024-09-10T09:42:39.580 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T09:42:39.580 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T09:42:39.580 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T09:42:39.580 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T09:42:39.580 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T09:42:39.580 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-10T09:42:39.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:39.577+0000 7f34ffa00640 1 -- 172.21.15.5:0/441296391 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f34e00a2a90 msgr2=0x7f34e00a4f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:42:39.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:39.577+0000 7f34ffa00640 1 --2- 172.21.15.5:0/441296391 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f34e00a2a90 0x7f34e00a4f50 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f34f000e910 tx=0x7f34f0009500 comp rx=0 tx=0).stop 2024-09-10T09:42:39.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:39.577+0000 7f34ffa00640 1 -- 172.21.15.5:0/441296391 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f34f8074eb0 msgr2=0x7f34f81a4c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:42:39.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:39.577+0000 7f34ffa00640 1 --2- 172.21.15.5:0/441296391 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f34f8074eb0 0x7f34f81a4c10 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f34e400e9f0 tx=0x7f34e400eeb0 comp rx=0 tx=0).stop 2024-09-10T09:42:39.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:39.577+0000 7f34ffa00640 1 -- 172.21.15.5:0/441296391 shutdown_connections 2024-09-10T09:42:39.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:39.577+0000 7f34ffa00640 1 --2- 172.21.15.5:0/441296391 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f34e00a2a90 0x7f34e00a4f50 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:39.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:39.577+0000 7f34ffa00640 1 --2- 172.21.15.5:0/441296391 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f34f8112b80 0x7f34f819e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:39.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:39.577+0000 7f34ffa00640 1 --2- 172.21.15.5:0/441296391 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f34f8074eb0 0x7f34f81a4c10 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:39.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:39.577+0000 7f34ffa00640 1 --2- 172.21.15.5:0/441296391 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f34f8074590 0x7f34f81a46d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:39.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:39.577+0000 7f34ffa00640 1 -- 172.21.15.5:0/441296391 >> 172.21.15.5:0/441296391 conn(0x7f34f80fd8b0 msgr2=0x7f34f80ff470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:42:39.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:39.577+0000 7f34ffa00640 1 -- 172.21.15.5:0/441296391 shutdown_connections 2024-09-10T09:42:39.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:39.577+0000 7f34ffa00640 1 -- 172.21.15.5:0/441296391 wait complete. 2024-09-10T09:42:39.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:42:39 smithi005 ceph-mon[30350]: pgmap v522: 129 pgs: 129 active+clean; 95 MiB data, 832 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 1.3 MiB/s wr, 53 op/s 2024-09-10T09:42:39.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:42:39 smithi102 ceph-mon[35442]: pgmap v522: 129 pgs: 129 active+clean; 95 MiB data, 832 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 1.3 MiB/s wr, 53 op/s 2024-09-10T09:42:40.026 INFO:tasks.workunit.client.0.smithi005.stdout: 1 192490 0.04 MB/sec execute 21 sec latency 24482.563 ms 2024-09-10T09:42:40.041 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-10T09:42:40.041 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-10T09:42:40.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:42:39 smithi159 ceph-mon[35513]: pgmap v522: 129 pgs: 129 active+clean; 95 MiB data, 832 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 1.3 MiB/s wr, 53 op/s 2024-09-10T09:42:40.517 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'ba5a0a38', total seconds elapsed: 29.1 2024-09-10T09:42:40.517 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 22.75 2024-09-10T09:42:40.517 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T09:42:40.635 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:42:40.787 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:42:40 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:42:40.787 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:42:40 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:42:40.787 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:42:40 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:42:40.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:42:40 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:42:40.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:42:40 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:42:40.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:42:40 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:42:41.026 INFO:tasks.workunit.client.0.smithi005.stdout: 1 194912 0.66 MB/sec execute 22 sec latency 17.639 ms 2024-09-10T09:42:41.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:42:40 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:42:41.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:42:40 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:42:41.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:42:40 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:42:41.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:41.239+0000 7fea8ce00640 1 -- 172.21.15.5:0/4042383148 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fea88103e30 msgr2=0x7fea88104210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:42:41.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:41.239+0000 7fea8ce00640 1 --2- 172.21.15.5:0/4042383148 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fea88103e30 0x7fea88104210 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fea78009900 tx=0x7fea7802a3f0 comp rx=0 tx=0).stop 2024-09-10T09:42:41.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:41.239+0000 7fea8ce00640 1 -- 172.21.15.5:0/4042383148 shutdown_connections 2024-09-10T09:42:41.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:41.239+0000 7fea8ce00640 1 --2- 172.21.15.5:0/4042383148 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fea88108f60 0x7fea88110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:41.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:41.239+0000 7fea8ce00640 1 --2- 172.21.15.5:0/4042383148 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fea881047e0 0x7fea88108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:41.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:41.239+0000 7fea8ce00640 1 --2- 172.21.15.5:0/4042383148 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fea88103e30 0x7fea88104210 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:41.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:41.239+0000 7fea8ce00640 1 -- 172.21.15.5:0/4042383148 >> 172.21.15.5:0/4042383148 conn(0x7fea880ff910 msgr2=0x7fea88101d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:42:41.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:41.239+0000 7fea8ce00640 1 -- 172.21.15.5:0/4042383148 shutdown_connections 2024-09-10T09:42:41.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:41.239+0000 7fea8ce00640 1 -- 172.21.15.5:0/4042383148 wait complete. 2024-09-10T09:42:41.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:41.240+0000 7fea8ce00640 1 Processor -- start 2024-09-10T09:42:41.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:41.240+0000 7fea8ce00640 1 -- start start 2024-09-10T09:42:41.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:41.241+0000 7fea8ce00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fea88103e30 0x7fea88076700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:42:41.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:41.241+0000 7fea8ce00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fea881047e0 0x7fea88076c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:42:41.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:41.241+0000 7fea8ce00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fea88108f60 0x7fea88077180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:42:41.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:41.241+0000 7fea8ce00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fea88113fd0 con 0x7fea88103e30 2024-09-10T09:42:41.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:41.241+0000 7fea8ce00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fea88113e50 con 0x7fea88108f60 2024-09-10T09:42:41.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:41.241+0000 7fea8ce00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fea88114150 con 0x7fea881047e0 2024-09-10T09:42:41.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:41.241+0000 7fea86a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fea881047e0 0x7fea88076c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:42:41.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:41.241+0000 7fea87e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fea88108f60 0x7fea88077180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:42:41.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:41.241+0000 7fea87e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fea88108f60 0x7fea88077180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:34666/0 (socket says 172.21.15.5:34666) 2024-09-10T09:42:41.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:41.241+0000 7fea87e00640 1 -- 172.21.15.5:0/1960757426 learned_addr learned my addr 172.21.15.5:0/1960757426 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:42:41.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:41.241+0000 7fea87400640 1 --2- 172.21.15.5:0/1960757426 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fea88103e30 0x7fea88076700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:42:41.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:41.242+0000 7fea87e00640 1 -- 172.21.15.5:0/1960757426 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fea881047e0 msgr2=0x7fea88076c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:42:41.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:41.242+0000 7fea87e00640 1 --2- 172.21.15.5:0/1960757426 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fea881047e0 0x7fea88076c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:41.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:41.242+0000 7fea87e00640 1 -- 172.21.15.5:0/1960757426 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fea88103e30 msgr2=0x7fea88076700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:42:41.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:41.242+0000 7fea87e00640 1 --2- 172.21.15.5:0/1960757426 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fea88103e30 0x7fea88076700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:41.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:41.242+0000 7fea87e00640 1 -- 172.21.15.5:0/1960757426 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea881aaad0 con 0x7fea88108f60 2024-09-10T09:42:41.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:41.242+0000 7fea87e00640 1 --2- 172.21.15.5:0/1960757426 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fea88108f60 0x7fea88077180 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fea8000eb70 tx=0x7fea8000c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:42:41.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:41.242+0000 7fea7fe00640 1 -- 172.21.15.5:0/1960757426 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea80004d20 con 0x7fea88108f60 2024-09-10T09:42:41.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:41.242+0000 7fea7fe00640 1 -- 172.21.15.5:0/1960757426 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea8000ee70 con 0x7fea88108f60 2024-09-10T09:42:41.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:41.242+0000 7fea7fe00640 1 -- 172.21.15.5:0/1960757426 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea80010760 con 0x7fea88108f60 2024-09-10T09:42:41.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:41.243+0000 7fea8ce00640 1 -- 172.21.15.5:0/1960757426 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea881aadc0 con 0x7fea88108f60 2024-09-10T09:42:41.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:41.243+0000 7fea8ce00640 1 -- 172.21.15.5:0/1960757426 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea881ab2d0 con 0x7fea88108f60 2024-09-10T09:42:41.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:41.246+0000 7fea9100e740 1 Processor -- start 2024-09-10T09:42:41.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:41.246+0000 7fea7fe00640 1 -- 172.21.15.5:0/1960757426 <== mon.2 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7fea80005020 con 0x7fea88108f60 2024-09-10T09:42:41.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:41.246+0000 7fea9100e740 1 -- start start 2024-09-10T09:42:41.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:41.246+0000 7fea9100e740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55820da80cb0 0x55820da81090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:42:41.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:41.247+0000 7fea9100e740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55820da815d0 0x55820da8b950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:42:41.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:41.247+0000 7fea9100e740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55820da8be90 0x55820da8e280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:42:41.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:41.247+0000 7fea9100e740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55820d9e7290 con 0x55820da80cb0 2024-09-10T09:42:41.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:41.247+0000 7fea9100e740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55820d98eb40 con 0x55820da8be90 2024-09-10T09:42:41.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:41.247+0000 7fea9100e740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55820d9e3250 con 0x55820da815d0 2024-09-10T09:42:41.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:41.247+0000 7fea87400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55820da815d0 0x55820da8b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:42:41.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:41.247+0000 7fea86a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55820da80cb0 0x55820da81090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:42:41.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:41.247+0000 7fea87400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55820da815d0 0x55820da8b950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:58420/0 (socket says 172.21.15.5:58420) 2024-09-10T09:42:41.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:41.247+0000 7fea87400640 1 -- 172.21.15.5:0/1474144829 learned_addr learned my addr 172.21.15.5:0/1474144829 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:42:41.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:41.247+0000 7fea86a00640 1 --2- 172.21.15.5:0/1474144829 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55820da8be90 0x55820da8e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:42:41.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:41.247+0000 7fea87400640 1 -- 172.21.15.5:0/1474144829 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55820da8be90 msgr2=0x55820da8e280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:42:41.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:41.247+0000 7fea87400640 1 --2- 172.21.15.5:0/1474144829 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55820da8be90 0x55820da8e280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:41.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:41.247+0000 7fea87400640 1 -- 172.21.15.5:0/1474144829 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55820da80cb0 msgr2=0x55820da81090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:42:41.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:41.247+0000 7fea87400640 1 --2- 172.21.15.5:0/1474144829 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55820da80cb0 0x55820da81090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:41.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:41.247+0000 7fea87400640 1 -- 172.21.15.5:0/1474144829 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55820d7586a0 con 0x55820da815d0 2024-09-10T09:42:41.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:41.247+0000 7fea86a00640 1 --2- 172.21.15.5:0/1474144829 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55820da8be90 0x55820da8e280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T09:42:41.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:41.248+0000 7fea7fe00640 1 --2- 172.21.15.5:0/1960757426 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fea6c0a2c00 0x7fea6c0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:42:41.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:41.248+0000 7fea87400640 1 --2- 172.21.15.5:0/1474144829 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55820da815d0 0x55820da8b950 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fea78009900 tx=0x7fea78005e60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eb121a5db6bd3ac9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:42:41.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:41.248+0000 7fea7fe00640 1 -- 172.21.15.5:0/1960757426 <== mon.2 v2:172.21.15.102:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fea800cfac0 con 0x7fea88108f60 2024-09-10T09:42:41.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:41.248+0000 7fea86a00640 1 --2- 172.21.15.5:0/1960757426 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fea6c0a2c00 0x7fea6c0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:42:41.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:41.248+0000 7fea7cc00640 1 -- 172.21.15.5:0/1474144829 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea78035030 con 0x55820da815d0 2024-09-10T09:42:41.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:41.248+0000 7fea7cc00640 1 -- 172.21.15.5:0/1474144829 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea780043c0 con 0x55820da815d0 2024-09-10T09:42:41.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:41.248+0000 7fea7cc00640 1 -- 172.21.15.5:0/1474144829 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea780035f0 con 0x55820da815d0 2024-09-10T09:42:41.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:41.249+0000 7fea9100e740 1 -- 172.21.15.5:0/1474144829 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55820da815d0 msgr2=0x55820da8b950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:42:41.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:41.249+0000 7fea9100e740 1 --2- 172.21.15.5:0/1474144829 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55820da815d0 0x55820da8b950 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fea78009900 tx=0x7fea78005e60 comp rx=0 tx=0).stop 2024-09-10T09:42:41.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:41.249+0000 7fea86a00640 1 --2- 172.21.15.5:0/1960757426 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fea6c0a2c00 0x7fea6c0a50c0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fea740095f0 tx=0x7fea74015040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:42:41.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:41.249+0000 7fea9100e740 1 -- 172.21.15.5:0/1474144829 shutdown_connections 2024-09-10T09:42:41.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:41.249+0000 7fea9100e740 1 --2- 172.21.15.5:0/1474144829 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55820da8be90 0x55820da8e280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:41.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:41.249+0000 7fea9100e740 1 --2- 172.21.15.5:0/1474144829 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55820da815d0 0x55820da8b950 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:41.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:41.249+0000 7fea9100e740 1 --2- 172.21.15.5:0/1474144829 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55820da80cb0 0x55820da81090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:41.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:41.249+0000 7fea9100e740 1 -- 172.21.15.5:0/1474144829 >> 172.21.15.5:0/1474144829 conn(0x55820da7d240 msgr2=0x55820da7d620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:42:41.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:41.250+0000 7fea9100e740 1 -- 172.21.15.5:0/1474144829 shutdown_connections 2024-09-10T09:42:41.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:41.250+0000 7fea9100e740 1 -- 172.21.15.5:0/1474144829 wait complete. 2024-09-10T09:42:41.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:41.250+0000 7fea9100e740 1 -- start start 2024-09-10T09:42:41.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:41.250+0000 7fea9100e740 1 Processor -- start 2024-09-10T09:42:41.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:41.251+0000 7fea9100e740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55820da80cb0 0x55820da8b820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:42:41.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:41.251+0000 7fea9100e740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55820da815d0 0x55820da88ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:42:41.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:41.251+0000 7fea9100e740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55820da8be90 0x55820da89050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:42:41.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:41.251+0000 7fea9100e740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55820da8eb00 con 0x55820da8be90 2024-09-10T09:42:41.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:41.251+0000 7fea9100e740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55820d9a1fa0 con 0x55820da80cb0 2024-09-10T09:42:41.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:41.251+0000 7fea9100e740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55820da8ec80 con 0x55820da815d0 2024-09-10T09:42:41.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:41.251+0000 7fea9100e740 10 client.0 _ll_register_callbacks cb 0x55820d6f5ba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:42:41.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:41.251+0000 7fea87400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55820da80cb0 0x55820da8b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:42:41.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:41.251+0000 7fea87400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55820da80cb0 0x55820da8b820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:34692/0 (socket says 172.21.15.5:34692) 2024-09-10T09:42:41.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:41.251+0000 7fea87400640 1 -- 172.21.15.5:0/310541879 learned_addr learned my addr 172.21.15.5:0/310541879 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:42:41.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:41.251+0000 7fea86a00640 1 --2- 172.21.15.5:0/310541879 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55820da815d0 0x55820da88ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:42:41.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:41.252+0000 7fea87400640 1 -- 172.21.15.5:0/310541879 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55820da815d0 msgr2=0x55820da88ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:42:41.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:41.252+0000 7fea87400640 1 --2- 172.21.15.5:0/310541879 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55820da815d0 0x55820da88ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:41.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:41.252+0000 7fea87400640 1 -- 172.21.15.5:0/310541879 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55820da8be90 msgr2=0x55820da89050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:42:41.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:41.252+0000 7fea87400640 1 --2- 172.21.15.5:0/310541879 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55820da8be90 0x55820da89050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:41.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:41.252+0000 7fea87400640 1 -- 172.21.15.5:0/310541879 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55820da89880 con 0x55820da80cb0 2024-09-10T09:42:41.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:41.252+0000 7fea87400640 1 --2- 172.21.15.5:0/310541879 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55820da80cb0 0x55820da8b820 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fea78004b40 tx=0x7fea78003820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b66777bbac3098a3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:42:41.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:41.252+0000 7fea71600640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-09-10T09:42:41.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:41.252+0000 7fea71600640 1 -- 172.21.15.5:0/310541879 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea78051060 con 0x55820da80cb0 2024-09-10T09:42:41.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:41.252+0000 7fea71600640 1 -- 172.21.15.5:0/310541879 <== mon.2 v2:172.21.15.102:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fea78003b20 con 0x55820da80cb0 2024-09-10T09:42:41.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:41.253+0000 7fea9100e740 1 -- 172.21.15.5:0/310541879 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fea780043c0 con 0x55820da80cb0 2024-09-10T09:42:41.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:41.253+0000 7fea71600640 1 -- 172.21.15.5:0/310541879 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fea78035070 con 0x55820da80cb0 2024-09-10T09:42:41.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:41.253+0000 7fea9100e740 10 client.26261 fetch_fsmap learned FSMap version 33 2024-09-10T09:42:41.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:41.253+0000 7fea9100e740 1 -- 172.21.15.5:0/310541879 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55820da9a9c0 con 0x55820da80cb0 2024-09-10T09:42:41.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:41.253+0000 7fea71600640 1 -- 172.21.15.5:0/310541879 <== mon.2 v2:172.21.15.102:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea780043c0 con 0x55820da80cb0 2024-09-10T09:42:41.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:41.253+0000 7fea71600640 1 -- 172.21.15.5:0/310541879 <== mon.2 v2:172.21.15.102:3300/0 5 ==== fsmap(e 33) ==== 8374+0+0 (secure 0 0 0) 0x7fea7803b430 con 0x55820da80cb0 2024-09-10T09:42:41.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:41.253+0000 7fea9100e740 10 client.26261 fetch_fsmap finished waiting for FSMap version 33 2024-09-10T09:42:41.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:41.253+0000 7fea9100e740 10 client.26261 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T09:42:41.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:41.253+0000 7fea9100e740 20 client.26261 populate_metadata read hostname 'smithi005' 2024-09-10T09:42:41.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:41.254+0000 7fea9100e740 1 --2- 172.21.15.5:0/310541879 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55820da9bd10 0x55820da9e100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:42:41.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:41.254+0000 7fea9100e740 4 client.26261 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:42:41.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:41.254+0000 7fea9100e740 1 -- 172.21.15.5:0/310541879 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55820d9f4800 con 0x55820da9bd10 2024-09-10T09:42:41.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:41.254+0000 7fea86a00640 1 --2- 172.21.15.5:0/310541879 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55820da9bd10 0x55820da9e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:42:41.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:41.255+0000 7fea86a00640 1 --2- 172.21.15.5:0/310541879 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55820da9bd10 0x55820da9e100 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=143e3d41b356d17f server_cookie=4ab99b80143ac63f in_seq=0 out_seq=0 2024-09-10T09:42:41.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:41.255+0000 7fea71600640 10 client.26261 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T09:42:41.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:41.255+0000 7fea71600640 1 -- 172.21.15.5:0/310541879 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55820d9f4800 con 0x55820da9bd10 2024-09-10T09:42:41.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:41.255+0000 7fea71600640 10 client.26261 handle_command_reply: tid=0 2024-09-10T09:42:41.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:41.255+0000 7fea9100e740 1 client.26261 shutdown 2024-09-10T09:42:41.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:41.256+0000 7fea9100e740 1 -- 172.21.15.5:0/310541879 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55820da80cb0 msgr2=0x55820da8b820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:42:41.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:41.256+0000 7fea9100e740 1 --2- 172.21.15.5:0/310541879 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55820da80cb0 0x55820da8b820 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fea78004b40 tx=0x7fea78003820 comp rx=0 tx=0).stop 2024-09-10T09:42:41.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:41.256+0000 7fea9100e740 1 -- 172.21.15.5:0/310541879 shutdown_connections 2024-09-10T09:42:41.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:41.256+0000 7fea9100e740 1 --2- 172.21.15.5:0/310541879 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55820da9bd10 0x55820da9e100 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:41.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:41.256+0000 7fea9100e740 1 --2- 172.21.15.5:0/310541879 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55820da8be90 0x55820da89050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:41.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:41.256+0000 7fea9100e740 1 --2- 172.21.15.5:0/310541879 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55820da815d0 0x55820da88ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:41.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:41.256+0000 7fea71600640 0 client.26261 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T09:42:41.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:41.256+0000 7fea9100e740 1 --2- 172.21.15.5:0/310541879 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55820da80cb0 0x55820da8b820 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:41.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:41.256+0000 7fea9100e740 1 -- 172.21.15.5:0/310541879 >> 172.21.15.5:0/310541879 conn(0x55820da7d240 msgr2=0x55820da82f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:42:41.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:41.256+0000 7fea9100e740 1 -- 172.21.15.5:0/310541879 shutdown_connections 2024-09-10T09:42:41.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:41.256+0000 7fea9100e740 1 -- 172.21.15.5:0/310541879 wait complete. 2024-09-10T09:42:41.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:41.256+0000 7fea9100e740 20 client.26261 trim_cache size 0 max 16384 2024-09-10T09:42:41.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:41.277+0000 7fea9100e740 1 Processor -- start 2024-09-10T09:42:41.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:41.278+0000 7fea9100e740 1 -- start start 2024-09-10T09:42:41.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:41.278+0000 7fea9100e740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55820da80cb0 0x55820da82d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:42:41.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:41.278+0000 7fea9100e740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55820da815d0 0x55820da84000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:42:41.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:41.278+0000 7fea9100e740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55820da8be90 0x55820da85d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:42:41.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:41.278+0000 7fea9100e740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fea881b52e0 con 0x55820da8be90 2024-09-10T09:42:41.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:41.278+0000 7fea9100e740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fea881b6e50 con 0x55820da80cb0 2024-09-10T09:42:41.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:41.278+0000 7fea9100e740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fea881b59b0 con 0x55820da815d0 2024-09-10T09:42:41.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:41.278+0000 7fea87400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55820da80cb0 0x55820da82d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:42:41.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:41.278+0000 7fea87400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55820da80cb0 0x55820da82d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:34702/0 (socket says 172.21.15.5:34702) 2024-09-10T09:42:41.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:41.278+0000 7fea87400640 1 -- 172.21.15.5:0/6902211 learned_addr learned my addr 172.21.15.5:0/6902211 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:42:41.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:41.279+0000 7fea86a00640 1 --2- 172.21.15.5:0/6902211 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55820da815d0 0x55820da84000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:42:41.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:41.279+0000 7fea87e00640 1 --2- 172.21.15.5:0/6902211 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55820da8be90 0x55820da85d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:42:41.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:41.279+0000 7fea87400640 1 -- 172.21.15.5:0/6902211 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55820da815d0 msgr2=0x55820da84000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:42:41.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:41.279+0000 7fea87400640 1 --2- 172.21.15.5:0/6902211 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55820da815d0 0x55820da84000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:41.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:41.279+0000 7fea87400640 1 -- 172.21.15.5:0/6902211 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55820da8be90 msgr2=0x55820da85d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:42:41.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:41.279+0000 7fea87400640 1 --2- 172.21.15.5:0/6902211 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55820da8be90 0x55820da85d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:41.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:41.279+0000 7fea87400640 1 -- 172.21.15.5:0/6902211 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55820da7e3b0 con 0x55820da80cb0 2024-09-10T09:42:41.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:41.279+0000 7fea87e00640 1 --2- 172.21.15.5:0/6902211 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55820da8be90 0x55820da85d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T09:42:41.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:41.279+0000 7fea87400640 1 --2- 172.21.15.5:0/6902211 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55820da80cb0 0x55820da82d70 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fea7802a430 tx=0x7fea7803bf90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=933f74cc891ecb32 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:42:41.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:41.280+0000 7fea73e00640 1 -- 172.21.15.5:0/6902211 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea78051030 con 0x55820da80cb0 2024-09-10T09:42:41.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:41.280+0000 7fea73e00640 1 -- 172.21.15.5:0/6902211 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea78049070 con 0x55820da80cb0 2024-09-10T09:42:41.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:41.280+0000 7fea73e00640 1 -- 172.21.15.5:0/6902211 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea78031cd0 con 0x55820da80cb0 2024-09-10T09:42:41.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:41.281+0000 7fea9100e740 1 -- 172.21.15.5:0/6902211 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55820da80cb0 msgr2=0x55820da82d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:42:41.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:41.281+0000 7fea9100e740 1 --2- 172.21.15.5:0/6902211 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55820da80cb0 0x55820da82d70 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fea7802a430 tx=0x7fea7803bf90 comp rx=0 tx=0).stop 2024-09-10T09:42:41.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:41.281+0000 7fea9100e740 1 -- 172.21.15.5:0/6902211 shutdown_connections 2024-09-10T09:42:41.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:41.281+0000 7fea9100e740 1 --2- 172.21.15.5:0/6902211 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55820da8be90 0x55820da85d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:41.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:41.281+0000 7fea9100e740 1 --2- 172.21.15.5:0/6902211 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55820da815d0 0x55820da84000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:41.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:41.281+0000 7fea9100e740 1 --2- 172.21.15.5:0/6902211 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55820da80cb0 0x55820da82d70 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:41.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:41.281+0000 7fea9100e740 1 -- 172.21.15.5:0/6902211 >> 172.21.15.5:0/6902211 conn(0x55820da7d240 msgr2=0x55820da838f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:42:41.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:41.282+0000 7fea9100e740 1 -- 172.21.15.5:0/6902211 shutdown_connections 2024-09-10T09:42:41.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:41.282+0000 7fea9100e740 1 -- 172.21.15.5:0/6902211 wait complete. 2024-09-10T09:42:41.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:41.282+0000 7fea9100e740 1 -- start start 2024-09-10T09:42:41.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:41.283+0000 7fea9100e740 1 Processor -- start 2024-09-10T09:42:41.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:41.283+0000 7fea9100e740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55820da80cb0 0x55820daa1800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:42:41.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:41.283+0000 7fea9100e740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55820da815d0 0x55820da9fc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:42:41.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:41.283+0000 7fea9100e740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55820da8be90 0x55820daa0140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:42:41.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:41.283+0000 7fea9100e740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fea881b6750 con 0x55820da80cb0 2024-09-10T09:42:41.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:41.283+0000 7fea9100e740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fea881b6080 con 0x55820da815d0 2024-09-10T09:42:41.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:41.283+0000 7fea9100e740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fea881b32c0 con 0x55820da8be90 2024-09-10T09:42:41.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:41.283+0000 7fea9100e740 10 client.0 _ll_register_callbacks cb 0x7fea800093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:42:41.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:41.283+0000 7fea87e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55820da8be90 0x55820daa0140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:42:41.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:41.284+0000 7fea87e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55820da8be90 0x55820daa0140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:58448/0 (socket says 172.21.15.5:58448) 2024-09-10T09:42:41.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:41.284+0000 7fea86a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55820da815d0 0x55820da9fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:42:41.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:41.284+0000 7fea87e00640 1 -- 172.21.15.5:0/1788598694 learned_addr learned my addr 172.21.15.5:0/1788598694 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:42:41.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:41.284+0000 7fea87400640 1 --2- 172.21.15.5:0/1788598694 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55820da80cb0 0x55820daa1800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:42:41.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:41.284+0000 7fea87e00640 1 -- 172.21.15.5:0/1788598694 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55820da815d0 msgr2=0x55820da9fc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:42:41.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:41.284+0000 7fea87e00640 1 --2- 172.21.15.5:0/1788598694 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55820da815d0 0x55820da9fc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:41.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:41.284+0000 7fea87e00640 1 -- 172.21.15.5:0/1788598694 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55820da80cb0 msgr2=0x55820daa1800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:42:41.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:41.284+0000 7fea87e00640 1 --2- 172.21.15.5:0/1788598694 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55820da80cb0 0x55820daa1800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:41.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:41.284+0000 7fea87e00640 1 -- 172.21.15.5:0/1788598694 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55820daa0940 con 0x55820da8be90 2024-09-10T09:42:41.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:41.284+0000 7fea86a00640 1 --2- 172.21.15.5:0/1788598694 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55820da815d0 0x55820da9fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:42:41.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:41.284+0000 7fea87400640 1 --2- 172.21.15.5:0/1788598694 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55820da80cb0 0x55820daa1800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T09:42:41.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:41.284+0000 7fea87e00640 1 --2- 172.21.15.5:0/1788598694 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55820da8be90 0x55820daa0140 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fea80098330 tx=0x7fea8009b830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8b380d09ece23d3d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:42:41.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:41.284+0000 7fea71600640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-09-10T09:42:41.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:41.285+0000 7fea71600640 1 -- 172.21.15.5:0/1788598694 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea800b4070 con 0x55820da8be90 2024-09-10T09:42:41.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:41.285+0000 7fea9100e740 1 -- 172.21.15.5:0/1788598694 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fea78049070 con 0x55820da8be90 2024-09-10T09:42:41.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:41.285+0000 7fea71600640 1 -- 172.21.15.5:0/1788598694 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fea80016df0 con 0x55820da8be90 2024-09-10T09:42:41.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:41.285+0000 7fea71600640 1 -- 172.21.15.5:0/1788598694 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fea8007e030 con 0x55820da8be90 2024-09-10T09:42:41.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:41.285+0000 7fea9100e740 10 client.26290 fetch_fsmap learned FSMap version 33 2024-09-10T09:42:41.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:41.285+0000 7fea9100e740 1 -- 172.21.15.5:0/1788598694 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55820db076f0 con 0x55820da8be90 2024-09-10T09:42:41.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:41.286+0000 7fea71600640 1 -- 172.21.15.5:0/1788598694 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea78049070 con 0x55820da8be90 2024-09-10T09:42:41.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:41.286+0000 7fea71600640 1 -- 172.21.15.5:0/1788598694 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 33) ==== 8374+0+0 (secure 0 0 0) 0x7fea800a46b0 con 0x55820da8be90 2024-09-10T09:42:41.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:41.286+0000 7fea9100e740 10 client.26290 fetch_fsmap finished waiting for FSMap version 33 2024-09-10T09:42:41.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:41.286+0000 7fea9100e740 10 client.26290 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T09:42:41.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:41.286+0000 7fea9100e740 20 client.26290 populate_metadata read hostname 'smithi005' 2024-09-10T09:42:41.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:41.286+0000 7fea9100e740 1 --2- 172.21.15.5:0/1788598694 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55820db07e20 0x55820db0a240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:42:41.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:41.286+0000 7fea9100e740 4 client.26290 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-10T09:42:41.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:41.286+0000 7fea9100e740 1 -- 172.21.15.5:0/1788598694 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55820d9eed10 con 0x55820db07e20 2024-09-10T09:42:41.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:41.287+0000 7fea87400640 1 --2- 172.21.15.5:0/1788598694 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55820db07e20 0x55820db0a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:42:41.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:41.287+0000 7fea87400640 1 --2- 172.21.15.5:0/1788598694 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55820db07e20 0x55820db0a240 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8cfce90fac60a924 server_cookie=6e34981545086e5b in_seq=0 out_seq=0 2024-09-10T09:42:41.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:41.287+0000 7fea71600640 10 client.26290 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T09:42:41.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:41.288+0000 7fea71600640 1 -- 172.21.15.5:0/1788598694 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55820d9eed10 con 0x55820db07e20 2024-09-10T09:42:41.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:41.288+0000 7fea71600640 10 client.26290 handle_command_reply: tid=0 2024-09-10T09:42:41.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:41.289+0000 7fea9100e740 1 client.26290 shutdown 2024-09-10T09:42:41.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:41.289+0000 7fea9100e740 1 -- 172.21.15.5:0/1788598694 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55820da8be90 msgr2=0x55820daa0140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:42:41.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:41.289+0000 7fea9100e740 1 --2- 172.21.15.5:0/1788598694 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55820da8be90 0x55820daa0140 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fea80098330 tx=0x7fea8009b830 comp rx=0 tx=0).stop 2024-09-10T09:42:41.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:41.289+0000 7fea9100e740 1 -- 172.21.15.5:0/1788598694 shutdown_connections 2024-09-10T09:42:41.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:41.289+0000 7fea9100e740 1 --2- 172.21.15.5:0/1788598694 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55820db07e20 0x55820db0a240 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:41.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:41.289+0000 7fea9100e740 1 --2- 172.21.15.5:0/1788598694 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55820da8be90 0x55820daa0140 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:41.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:41.289+0000 7fea9100e740 1 --2- 172.21.15.5:0/1788598694 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55820da815d0 0x55820da9fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:41.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:41.289+0000 7fea9100e740 1 --2- 172.21.15.5:0/1788598694 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55820da80cb0 0x55820daa1800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:41.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:41.289+0000 7fea71600640 0 client.26290 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T09:42:41.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:41.289+0000 7fea9100e740 1 -- 172.21.15.5:0/1788598694 >> 172.21.15.5:0/1788598694 conn(0x55820da7d240 msgr2=0x55820da84110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:42:41.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:41.289+0000 7fea9100e740 1 -- 172.21.15.5:0/1788598694 shutdown_connections 2024-09-10T09:42:41.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:41.289+0000 7fea9100e740 1 -- 172.21.15.5:0/1788598694 wait complete. 2024-09-10T09:42:41.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:41.290+0000 7fea9100e740 20 client.26290 trim_cache size 0 max 16384 2024-09-10T09:42:41.293 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-10T09:42:41.293 INFO:teuthology.orchestra.run.smithi005.stdout: "return_code": 0, 2024-09-10T09:42:41.294 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_tag": "e9d447d4-791d-4f12-905a-fd737c5e40a1", 2024-09-10T09:42:41.294 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": "asynchronous" 2024-09-10T09:42:41.294 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-10T09:42:41.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:41.290+0000 7fea8ce00640 1 -- 172.21.15.5:0/1960757426 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fea6c0a2c00 msgr2=0x7fea6c0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:42:41.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:41.290+0000 7fea8ce00640 1 --2- 172.21.15.5:0/1960757426 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fea6c0a2c00 0x7fea6c0a50c0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fea740095f0 tx=0x7fea74015040 comp rx=0 tx=0).stop 2024-09-10T09:42:41.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:41.290+0000 7fea8ce00640 1 -- 172.21.15.5:0/1960757426 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fea88108f60 msgr2=0x7fea88077180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:42:41.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:41.290+0000 7fea8ce00640 1 --2- 172.21.15.5:0/1960757426 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fea88108f60 0x7fea88077180 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fea8000eb70 tx=0x7fea8000c6a0 comp rx=0 tx=0).stop 2024-09-10T09:42:41.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:41.291+0000 7fea8ce00640 1 -- 172.21.15.5:0/1960757426 shutdown_connections 2024-09-10T09:42:41.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:41.291+0000 7fea8ce00640 1 --2- 172.21.15.5:0/1960757426 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fea6c0a2c00 0x7fea6c0a50c0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:41.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:41.291+0000 7fea8ce00640 1 --2- 172.21.15.5:0/1960757426 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fea88108f60 0x7fea88077180 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:41.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:41.291+0000 7fea8ce00640 1 --2- 172.21.15.5:0/1960757426 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fea881047e0 0x7fea88076c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:41.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:41.291+0000 7fea8ce00640 1 --2- 172.21.15.5:0/1960757426 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fea88103e30 0x7fea88076700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:41.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:41.291+0000 7fea8ce00640 1 -- 172.21.15.5:0/1960757426 >> 172.21.15.5:0/1960757426 conn(0x7fea880ff910 msgr2=0x7fea88100be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:42:41.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:41.291+0000 7fea8ce00640 1 -- 172.21.15.5:0/1960757426 shutdown_connections 2024-09-10T09:42:41.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:41.291+0000 7fea8ce00640 1 -- 172.21.15.5:0/1960757426 wait complete. 2024-09-10T09:42:41.309 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:42:41.882 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:42:41 smithi005 ceph-mon[30350]: pgmap v523: 129 pgs: 129 active+clean; 96 MiB data, 836 MiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 1.6 MiB/s wr, 64 op/s 2024-09-10T09:42:41.882 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:42:41 smithi005 ceph-mon[30350]: scrub summary: active paths [/] 2024-09-10T09:42:41.882 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:42:41 smithi005 ceph-mon[30350]: scrub summary: idle+waiting paths [/] 2024-09-10T09:42:41.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:42:41 smithi102 ceph-mon[35442]: pgmap v523: 129 pgs: 129 active+clean; 96 MiB data, 836 MiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 1.6 MiB/s wr, 64 op/s 2024-09-10T09:42:41.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:42:41 smithi102 ceph-mon[35442]: scrub summary: active paths [/] 2024-09-10T09:42:41.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:42:41 smithi102 ceph-mon[35442]: scrub summary: idle+waiting paths [/] 2024-09-10T09:42:42.026 INFO:tasks.workunit.client.0.smithi005.stdout: 1 196602 1.05 MB/sec execute 23 sec latency 11.714 ms 2024-09-10T09:42:42.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:42:41 smithi159 ceph-mon[35513]: pgmap v523: 129 pgs: 129 active+clean; 96 MiB data, 836 MiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 1.6 MiB/s wr, 64 op/s 2024-09-10T09:42:42.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:42:41 smithi159 ceph-mon[35513]: scrub summary: active paths [/] 2024-09-10T09:42:42.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:42:41 smithi159 ceph-mon[35513]: scrub summary: idle+waiting paths [/] 2024-09-10T09:42:42.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:42.259+0000 7f4236800640 1 -- 172.21.15.5:0/2404372492 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f42301019c0 msgr2=0x7f4230101e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:42:42.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:42.259+0000 7f4236800640 1 --2- 172.21.15.5:0/2404372492 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f42301019c0 0x7f4230101e20 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f421c009930 tx=0x7f421c02a420 comp rx=0 tx=0).stop 2024-09-10T09:42:42.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:42.259+0000 7f4236800640 1 -- 172.21.15.5:0/2404372492 shutdown_connections 2024-09-10T09:42:42.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:42.259+0000 7f4236800640 1 --2- 172.21.15.5:0/2404372492 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4230102360 0x7f423010e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:42.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:42.259+0000 7f4236800640 1 --2- 172.21.15.5:0/2404372492 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f42301019c0 0x7f4230101e20 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:42.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:42.259+0000 7f4236800640 1 --2- 172.21.15.5:0/2404372492 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f42301079c0 0x7f4230107da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:42.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:42.259+0000 7f4236800640 1 -- 172.21.15.5:0/2404372492 >> 172.21.15.5:0/2404372492 conn(0x7f42300fd740 msgr2=0x7f42300ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:42:42.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:42.259+0000 7f4236800640 1 -- 172.21.15.5:0/2404372492 shutdown_connections 2024-09-10T09:42:42.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:42.259+0000 7f4236800640 1 -- 172.21.15.5:0/2404372492 wait complete. 2024-09-10T09:42:42.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:42.260+0000 7f4236800640 1 Processor -- start 2024-09-10T09:42:42.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:42.260+0000 7f4236800640 1 -- start start 2024-09-10T09:42:42.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:42.261+0000 7f4236800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f42301019c0 0x7f42301982b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:42:42.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:42.261+0000 7f4236800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4230102360 0x7f42301987f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:42:42.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:42.261+0000 7f4236800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f42301079c0 0x7f423019cb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:42:42.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:42.261+0000 7f4236800640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f4230111e10 con 0x7f42301079c0 2024-09-10T09:42:42.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:42.261+0000 7f4236800640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f4230111c90 con 0x7f4230102360 2024-09-10T09:42:42.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:42.261+0000 7f4236800640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f4230111f90 con 0x7f42301019c0 2024-09-10T09:42:42.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:42.261+0000 7f4234a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4230102360 0x7f42301987f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:42:42.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:42.261+0000 7f4234a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4230102360 0x7f42301987f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:34724/0 (socket says 172.21.15.5:34724) 2024-09-10T09:42:42.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:42.261+0000 7f4234a00640 1 -- 172.21.15.5:0/4162857820 learned_addr learned my addr 172.21.15.5:0/4162857820 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:42:42.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:42.261+0000 7f4235e00640 1 --2- 172.21.15.5:0/4162857820 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f42301079c0 0x7f423019cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:42:42.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:42.261+0000 7f4235400640 1 --2- 172.21.15.5:0/4162857820 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f42301019c0 0x7f42301982b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:42:42.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:42.261+0000 7f4234a00640 1 -- 172.21.15.5:0/4162857820 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f42301019c0 msgr2=0x7f42301982b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:42:42.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:42.261+0000 7f4234a00640 1 --2- 172.21.15.5:0/4162857820 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f42301019c0 0x7f42301982b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:42.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:42.261+0000 7f4234a00640 1 -- 172.21.15.5:0/4162857820 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f42301079c0 msgr2=0x7f423019cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:42:42.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:42.261+0000 7f4234a00640 1 --2- 172.21.15.5:0/4162857820 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f42301079c0 0x7f423019cb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:42.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:42.261+0000 7f4234a00640 1 -- 172.21.15.5:0/4162857820 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f423019d260 con 0x7f4230102360 2024-09-10T09:42:42.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:42.262+0000 7f4234a00640 1 --2- 172.21.15.5:0/4162857820 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4230102360 0x7f42301987f0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f421c02a940 tx=0x7f421c003b20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:42:42.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:42.262+0000 7f422e000640 1 -- 172.21.15.5:0/4162857820 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f421c03f070 con 0x7f4230102360 2024-09-10T09:42:42.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:42.262+0000 7f4236800640 1 -- 172.21.15.5:0/4162857820 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f423019d4f0 con 0x7f4230102360 2024-09-10T09:42:42.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:42.262+0000 7f422e000640 1 -- 172.21.15.5:0/4162857820 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f421c031c90 con 0x7f4230102360 2024-09-10T09:42:42.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:42.262+0000 7f422e000640 1 -- 172.21.15.5:0/4162857820 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f421c035030 con 0x7f4230102360 2024-09-10T09:42:42.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:42.262+0000 7f4236800640 1 -- 172.21.15.5:0/4162857820 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42301a4e30 con 0x7f4230102360 2024-09-10T09:42:42.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:42.264+0000 7f4236800640 1 -- 172.21.15.5:0/4162857820 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4230101e20 con 0x7f4230102360 2024-09-10T09:42:42.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:42.266+0000 7f422e000640 1 -- 172.21.15.5:0/4162857820 <== mon.2 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f421c042070 con 0x7f4230102360 2024-09-10T09:42:42.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:42.267+0000 7f422e000640 1 --2- 172.21.15.5:0/4162857820 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f42180a2c00 0x7f42180a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:42:42.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:42.267+0000 7f422e000640 1 -- 172.21.15.5:0/4162857820 <== mon.2 v2:172.21.15.102:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f421c0efa40 con 0x7f4230102360 2024-09-10T09:42:42.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:42.268+0000 7f4235400640 1 --2- 172.21.15.5:0/4162857820 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f42180a2c00 0x7f42180a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:42:42.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:42.268+0000 7f4235400640 1 --2- 172.21.15.5:0/4162857820 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f42180a2c00 0x7f42180a50c0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f4224009b50 tx=0x7f4224005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:42:42.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:42.271+0000 7f422e000640 1 -- 172.21.15.5:0/4162857820 <== mon.2 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f421c0bbf00 con 0x7f4230102360 2024-09-10T09:42:42.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:42.554+0000 7f4236800640 1 -- 172.21.15.5:0/4162857820 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4230109f50 con 0x7f4230102360 2024-09-10T09:42:42.556 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-10T09:42:42.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:42.555+0000 7f422e000640 1 -- 172.21.15.5:0/4162857820 <== mon.2 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) ==== 94+0+11970 (secure 0 0 0) 0x7f421c0c0200 con 0x7f4230102360 2024-09-10T09:42:42.556 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 33 2024-09-10T09:42:42.556 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":33,"btime":"2024-09-10T09:42:17:528412+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14454,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/2970594311","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2970594311},{"type":"v1","addr":"172.21.15.5:6835","nonce":2970594311}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14514,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6839/3616502174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":3616502174},{"type":"v1","addr":"172.21.15.5:6839","nonce":3616502174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":33,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T09:35:29.626287+0000","modified":"2024-09-10T09:42:16.961192+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24542,"mds_1":24535,"mds_2":24511,"mds_3":24506,"mds_4":24443},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14520":{"gid":14520,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/2073599842","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":2073599842},{"type":"v1","addr":"172.21.15.5:6841","nonce":2073599842}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"k","rank":4,"incarnation":26,"state":"up:active","state_seq":24,"addr":"172.21.15.102:6833/2457007427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":2457007427},{"type":"v1","addr":"172.21.15.102:6833","nonce":2457007427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/356445159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":356445159},{"type":"v1","addr":"172.21.15.159:6833","nonce":356445159}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6837/2894314507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":2894314507},{"type":"v1","addr":"172.21.15.5:6837","nonce":2894314507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.102:6835/1647549111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":1647549111},{"type":"v1","addr":"172.21.15.102:6835","nonce":1647549111}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6835/952657242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":952657242},{"type":"v1","addr":"172.21.15.159:6835","nonce":952657242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":19,"addr":"172.21.15.102:6837/2689776764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":2689776764},{"type":"v1","addr":"172.21.15.102:6837","nonce":2689776764}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.159:6837/620627639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":620627639},{"type":"v1","addr":"172.21.15.159:6837","nonce":620627639}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24535":{"gid":24535,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":18,"addr":"172.21.15.159:6839/2073739599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":2073739599},{"type":"v1","addr":"172.21.15.159:6839","nonce":2073739599}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24542":{"gid":24542,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.102:6839/335409749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6838","nonce":335409749},{"type":"v1","addr":"172.21.15.102:6839","nonce":335409749}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24542,"qdb_cluster":[24443,24506,24511,24535,24542]},"id":1}]} 2024-09-10T09:42:42.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:42.557+0000 7f4236800640 1 -- 172.21.15.5:0/4162857820 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f42180a2c00 msgr2=0x7f42180a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:42:42.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:42.557+0000 7f4236800640 1 --2- 172.21.15.5:0/4162857820 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f42180a2c00 0x7f42180a50c0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f4224009b50 tx=0x7f4224005f90 comp rx=0 tx=0).stop 2024-09-10T09:42:42.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:42.557+0000 7f4236800640 1 -- 172.21.15.5:0/4162857820 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4230102360 msgr2=0x7f42301987f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:42:42.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:42.557+0000 7f4236800640 1 --2- 172.21.15.5:0/4162857820 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4230102360 0x7f42301987f0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f421c02a940 tx=0x7f421c003b20 comp rx=0 tx=0).stop 2024-09-10T09:42:42.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:42.557+0000 7f4236800640 1 -- 172.21.15.5:0/4162857820 shutdown_connections 2024-09-10T09:42:42.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:42.557+0000 7f4236800640 1 --2- 172.21.15.5:0/4162857820 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f42180a2c00 0x7f42180a50c0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:42.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:42.557+0000 7f4236800640 1 --2- 172.21.15.5:0/4162857820 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f42301079c0 0x7f423019cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:42.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:42.557+0000 7f4236800640 1 --2- 172.21.15.5:0/4162857820 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4230102360 0x7f42301987f0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:42.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:42.557+0000 7f4236800640 1 --2- 172.21.15.5:0/4162857820 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f42301019c0 0x7f42301982b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:42.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:42.557+0000 7f4236800640 1 -- 172.21.15.5:0/4162857820 >> 172.21.15.5:0/4162857820 conn(0x7f42300fd740 msgr2=0x7f42300fdec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:42:42.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:42.558+0000 7f4236800640 1 -- 172.21.15.5:0/4162857820 shutdown_connections 2024-09-10T09:42:42.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:42.558+0000 7f4236800640 1 -- 172.21.15.5:0/4162857820 wait complete. 2024-09-10T09:42:42.732 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:42:42 smithi005 ceph-mon[30350]: scrub queued for path: / 2024-09-10T09:42:42.732 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:42:42 smithi005 ceph-mon[30350]: scrub summary: idle+waiting paths [/] 2024-09-10T09:42:42.732 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:42:42 smithi005 ceph-mon[30350]: scrub summary: active paths [/] 2024-09-10T09:42:42.732 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:42:42 smithi005 ceph-mon[30350]: scrub summary: idle+waiting paths [/] 2024-09-10T09:42:42.732 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:42:42 smithi005 ceph-mon[30350]: from='client.? 172.21.15.5:0/4162857820' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:42:42.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:42:42 smithi102 ceph-mon[35442]: scrub queued for path: / 2024-09-10T09:42:42.885 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:42:42 smithi102 ceph-mon[35442]: scrub summary: idle+waiting paths [/] 2024-09-10T09:42:42.885 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:42:42 smithi102 ceph-mon[35442]: scrub summary: active paths [/] 2024-09-10T09:42:42.885 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:42:42 smithi102 ceph-mon[35442]: scrub summary: idle+waiting paths [/] 2024-09-10T09:42:42.885 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:42:42 smithi102 ceph-mon[35442]: from='client.? 172.21.15.5:0/4162857820' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:42:43.026 INFO:tasks.workunit.client.0.smithi005.stdout: 1 197697 1.13 MB/sec execute 24 sec latency 495.176 ms 2024-09-10T09:42:43.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:42:42 smithi159 ceph-mon[35513]: scrub queued for path: / 2024-09-10T09:42:43.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:42:42 smithi159 ceph-mon[35513]: scrub summary: idle+waiting paths [/] 2024-09-10T09:42:43.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:42:42 smithi159 ceph-mon[35513]: scrub summary: active paths [/] 2024-09-10T09:42:43.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:42:42 smithi159 ceph-mon[35513]: scrub summary: idle+waiting paths [/] 2024-09-10T09:42:43.101 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:42:42 smithi159 ceph-mon[35513]: from='client.? 172.21.15.5:0/4162857820' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:42:43.669 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T09:42:43.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:42:43 smithi102 ceph-mon[35442]: pgmap v524: 129 pgs: 129 active+clean; 94 MiB data, 840 MiB used, 1.0 TiB / 1.0 TiB avail; 326 KiB/s rd, 1.5 MiB/s wr, 66 op/s 2024-09-10T09:42:43.889 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:42:43 smithi005 ceph-mon[30350]: pgmap v524: 129 pgs: 129 active+clean; 94 MiB data, 840 MiB used, 1.0 TiB / 1.0 TiB avail; 326 KiB/s rd, 1.5 MiB/s wr, 66 op/s 2024-09-10T09:42:44.026 INFO:tasks.workunit.client.0.smithi005.stdout: 1 197971 1.21 MB/sec execute 25 sec latency 1428.720 ms 2024-09-10T09:42:44.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:42:43 smithi159 ceph-mon[35513]: pgmap v524: 129 pgs: 129 active+clean; 94 MiB data, 840 MiB used, 1.0 TiB / 1.0 TiB avail; 326 KiB/s rd, 1.5 MiB/s wr, 66 op/s 2024-09-10T09:42:44.727 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24542 quiesce db / --timeout 90 --expiration 142.75 --await-for 45.0' 2024-09-10T09:42:44.728 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24542 quiesce db / --timeout 90 --expiration 142.75 --await-for 45.0 2024-09-10T09:42:44.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:42:44 smithi102 ceph-mon[35442]: pgmap v525: 129 pgs: 129 active+clean; 93 MiB data, 859 MiB used, 1.0 TiB / 1.0 TiB avail; 875 KiB/s rd, 1.4 MiB/s wr, 71 op/s 2024-09-10T09:42:44.927 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:42:45.027 INFO:tasks.workunit.client.0.smithi005.stdout: 1 199551 1.50 MB/sec execute 26 sec latency 27.023 ms 2024-09-10T09:42:45.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:42:44 smithi005 ceph-mon[30350]: pgmap v525: 129 pgs: 129 active+clean; 93 MiB data, 859 MiB used, 1.0 TiB / 1.0 TiB avail; 875 KiB/s rd, 1.4 MiB/s wr, 71 op/s 2024-09-10T09:42:45.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:42:44 smithi159 ceph-mon[35513]: pgmap v525: 129 pgs: 129 active+clean; 93 MiB data, 859 MiB used, 1.0 TiB / 1.0 TiB avail; 875 KiB/s rd, 1.4 MiB/s wr, 71 op/s 2024-09-10T09:42:45.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:45.517+0000 7fe6de000640 1 -- 172.21.15.5:0/1515488843 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe6d8104010 msgr2=0x7fe6d81043f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:42:45.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:45.517+0000 7fe6de000640 1 --2- 172.21.15.5:0/1515488843 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe6d8104010 0x7fe6d81043f0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fe6c8009900 tx=0x7fe6c802a400 comp rx=0 tx=0).stop 2024-09-10T09:42:45.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:45.517+0000 7fe6de000640 1 -- 172.21.15.5:0/1515488843 shutdown_connections 2024-09-10T09:42:45.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:45.517+0000 7fe6de000640 1 --2- 172.21.15.5:0/1515488843 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe6d8109140 0x7fe6d8110cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:45.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:45.517+0000 7fe6de000640 1 --2- 172.21.15.5:0/1515488843 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe6d81049c0 0x7fe6d8108a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:45.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:45.517+0000 7fe6de000640 1 --2- 172.21.15.5:0/1515488843 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe6d8104010 0x7fe6d81043f0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:45.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:45.517+0000 7fe6de000640 1 -- 172.21.15.5:0/1515488843 >> 172.21.15.5:0/1515488843 conn(0x7fe6d80ffa50 msgr2=0x7fe6d8101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:42:45.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:45.518+0000 7fe6de000640 1 -- 172.21.15.5:0/1515488843 shutdown_connections 2024-09-10T09:42:45.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:45.518+0000 7fe6de000640 1 -- 172.21.15.5:0/1515488843 wait complete. 2024-09-10T09:42:45.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:45.519+0000 7fe6de000640 1 Processor -- start 2024-09-10T09:42:45.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:45.519+0000 7fe6de000640 1 -- start start 2024-09-10T09:42:45.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:45.520+0000 7fe6de000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe6d8104010 0x7fe6d819a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:42:45.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:45.520+0000 7fe6de000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe6d81049c0 0x7fe6d819ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:42:45.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:45.520+0000 7fe6de000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe6d8109140 0x7fe6d819ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:42:45.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:45.520+0000 7fe6de000640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe6d8114170 con 0x7fe6d8104010 2024-09-10T09:42:45.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:45.520+0000 7fe6de000640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fe6d8113ff0 con 0x7fe6d81049c0 2024-09-10T09:42:45.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:45.520+0000 7fe6de000640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fe6d81142f0 con 0x7fe6d8109140 2024-09-10T09:42:45.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:45.520+0000 7fe6d7400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe6d8104010 0x7fe6d819a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:42:45.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:45.520+0000 7fe6d7400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe6d8104010 0x7fe6d819a630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50112/0 (socket says 172.21.15.5:50112) 2024-09-10T09:42:45.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:45.520+0000 7fe6d7400640 1 -- 172.21.15.5:0/4266582905 learned_addr learned my addr 172.21.15.5:0/4266582905 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:42:45.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:45.520+0000 7fe6d7e00640 1 --2- 172.21.15.5:0/4266582905 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe6d8109140 0x7fe6d819ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:42:45.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:45.520+0000 7fe6cfe00640 1 --2- 172.21.15.5:0/4266582905 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe6d81049c0 0x7fe6d819ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:42:45.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:45.521+0000 7fe6d7e00640 1 -- 172.21.15.5:0/4266582905 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe6d81049c0 msgr2=0x7fe6d819ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:42:45.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:45.521+0000 7fe6d7e00640 1 --2- 172.21.15.5:0/4266582905 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe6d81049c0 0x7fe6d819ab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:45.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:45.521+0000 7fe6d7e00640 1 -- 172.21.15.5:0/4266582905 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe6d8104010 msgr2=0x7fe6d819a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:42:45.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:45.521+0000 7fe6d7e00640 1 --2- 172.21.15.5:0/4266582905 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe6d8104010 0x7fe6d819a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:45.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:45.521+0000 7fe6d7e00640 1 -- 172.21.15.5:0/4266582905 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6d819f5e0 con 0x7fe6d8109140 2024-09-10T09:42:45.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:45.521+0000 7fe6cfe00640 1 --2- 172.21.15.5:0/4266582905 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe6d81049c0 0x7fe6d819ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:42:45.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:45.521+0000 7fe6d7e00640 1 --2- 172.21.15.5:0/4266582905 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe6d8109140 0x7fe6d819ef00 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fe6d0009fd0 tx=0x7fe6d000ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:42:45.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:45.522+0000 7fe6d4c00640 1 -- 172.21.15.5:0/4266582905 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6d000ce20 con 0x7fe6d8109140 2024-09-10T09:42:45.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:45.522+0000 7fe6d4c00640 1 -- 172.21.15.5:0/4266582905 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe6d0005020 con 0x7fe6d8109140 2024-09-10T09:42:45.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:45.522+0000 7fe6d4c00640 1 -- 172.21.15.5:0/4266582905 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6d0010720 con 0x7fe6d8109140 2024-09-10T09:42:45.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:45.522+0000 7fe6de000640 1 -- 172.21.15.5:0/4266582905 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6d819f8d0 con 0x7fe6d8109140 2024-09-10T09:42:45.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:45.522+0000 7fe6de000640 1 -- 172.21.15.5:0/4266582905 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6d81a71b0 con 0x7fe6d8109140 2024-09-10T09:42:45.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:45.526+0000 7fe6d4c00640 1 -- 172.21.15.5:0/4266582905 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7fe6d0005540 con 0x7fe6d8109140 2024-09-10T09:42:45.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:45.526+0000 7fe6e0c5c740 1 Processor -- start 2024-09-10T09:42:45.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:45.526+0000 7fe6e0c5c740 1 -- start start 2024-09-10T09:42:45.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:45.527+0000 7fe6e0c5c740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a5e9f91b90 0x55a5e9f91f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:42:45.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:45.527+0000 7fe6e0c5c740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a5e9f924b0 0x55a5e9f9c830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:42:45.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:45.527+0000 7fe6e0c5c740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a5e9f9cd70 0x55a5e9f9f160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:42:45.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:45.527+0000 7fe6e0c5c740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55a5e9ef83e0 con 0x55a5e9f91b90 2024-09-10T09:42:45.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:45.527+0000 7fe6e0c5c740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55a5e9e9fd80 con 0x55a5e9f9cd70 2024-09-10T09:42:45.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:45.527+0000 7fe6e0c5c740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55a5e9ef43a0 con 0x55a5e9f924b0 2024-09-10T09:42:45.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:45.527+0000 7fe6d4c00640 1 --2- 172.21.15.5:0/4266582905 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fe6bc0a2c00 0x7fe6bc0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:42:45.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:45.527+0000 7fe6d7400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a5e9f924b0 0x55a5e9f9c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:42:45.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:45.527+0000 7fe6d7400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a5e9f924b0 0x55a5e9f9c830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:58484/0 (socket says 172.21.15.5:58484) 2024-09-10T09:42:45.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:45.527+0000 7fe6d7400640 1 -- 172.21.15.5:0/2563163801 learned_addr learned my addr 172.21.15.5:0/2563163801 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:42:45.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:45.527+0000 7fe6d4c00640 1 -- 172.21.15.5:0/4266582905 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fe6d00cf9b0 con 0x7fe6d8109140 2024-09-10T09:42:45.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:45.528+0000 7fe6d7400640 1 -- 172.21.15.5:0/2563163801 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a5e9f9cd70 msgr2=0x55a5e9f9f160 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T09:42:45.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:45.528+0000 7fe6d7400640 1 --2- 172.21.15.5:0/2563163801 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a5e9f9cd70 0x55a5e9f9f160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:45.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:45.528+0000 7fe6d7400640 1 -- 172.21.15.5:0/2563163801 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a5e9f91b90 msgr2=0x55a5e9f91f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:42:45.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:45.528+0000 7fe6d7400640 1 --2- 172.21.15.5:0/2563163801 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a5e9f91b90 0x55a5e9f91f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:45.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:45.528+0000 7fe6d7400640 1 -- 172.21.15.5:0/2563163801 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a5e9ebeae0 con 0x55a5e9f924b0 2024-09-10T09:42:45.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:45.528+0000 7fe6d7e00640 1 --2- 172.21.15.5:0/4266582905 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fe6bc0a2c00 0x7fe6bc0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:42:45.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:45.528+0000 7fe6d7400640 1 --2- 172.21.15.5:0/2563163801 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a5e9f924b0 0x55a5e9f9c830 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fe6c8009a20 tx=0x7fe6c8005dd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a94684af844ae40e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:42:45.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:45.529+0000 7fe6c7e00640 1 -- 172.21.15.5:0/2563163801 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6c8040070 con 0x55a5e9f924b0 2024-09-10T09:42:45.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:45.529+0000 7fe6d7e00640 1 --2- 172.21.15.5:0/4266582905 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fe6bc0a2c00 0x7fe6bc0a50c0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fe6d0000f80 tx=0x7fe6d0005940 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:42:45.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:45.529+0000 7fe6c7e00640 1 -- 172.21.15.5:0/2563163801 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe6c8004030 con 0x55a5e9f924b0 2024-09-10T09:42:45.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:45.529+0000 7fe6c7e00640 1 -- 172.21.15.5:0/2563163801 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6c8035040 con 0x55a5e9f924b0 2024-09-10T09:42:45.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:45.529+0000 7fe6e0c5c740 1 -- 172.21.15.5:0/2563163801 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a5e9f924b0 msgr2=0x55a5e9f9c830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:42:45.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:45.529+0000 7fe6e0c5c740 1 --2- 172.21.15.5:0/2563163801 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a5e9f924b0 0x55a5e9f9c830 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fe6c8009a20 tx=0x7fe6c8005dd0 comp rx=0 tx=0).stop 2024-09-10T09:42:45.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:45.530+0000 7fe6e0c5c740 1 -- 172.21.15.5:0/2563163801 shutdown_connections 2024-09-10T09:42:45.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:45.530+0000 7fe6e0c5c740 1 --2- 172.21.15.5:0/2563163801 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a5e9f9cd70 0x55a5e9f9f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:45.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:45.530+0000 7fe6e0c5c740 1 --2- 172.21.15.5:0/2563163801 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a5e9f924b0 0x55a5e9f9c830 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:45.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:45.530+0000 7fe6e0c5c740 1 --2- 172.21.15.5:0/2563163801 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a5e9f91b90 0x55a5e9f91f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:45.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:45.530+0000 7fe6e0c5c740 1 -- 172.21.15.5:0/2563163801 >> 172.21.15.5:0/2563163801 conn(0x55a5e9f8e120 msgr2=0x55a5e9f8e500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:42:45.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:45.530+0000 7fe6e0c5c740 1 -- 172.21.15.5:0/2563163801 shutdown_connections 2024-09-10T09:42:45.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:45.530+0000 7fe6e0c5c740 1 -- 172.21.15.5:0/2563163801 wait complete. 2024-09-10T09:42:45.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:45.530+0000 7fe6e0c5c740 1 -- start start 2024-09-10T09:42:45.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:45.531+0000 7fe6e0c5c740 1 Processor -- start 2024-09-10T09:42:45.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:45.531+0000 7fe6e0c5c740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a5e9f91b90 0x55a5e9f99cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:42:45.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:45.531+0000 7fe6e0c5c740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a5e9f924b0 0x55a5e9f9a210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:42:45.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:45.531+0000 7fe6e0c5c740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a5e9f9cd70 0x55a5e9f9a750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:42:45.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:45.531+0000 7fe6e0c5c740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55a5e9f9f9e0 con 0x55a5e9f924b0 2024-09-10T09:42:45.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:45.531+0000 7fe6e0c5c740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55a5e9eb30f0 con 0x55a5e9f9cd70 2024-09-10T09:42:45.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:45.531+0000 7fe6e0c5c740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55a5e9f9fb60 con 0x55a5e9f91b90 2024-09-10T09:42:45.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:45.531+0000 7fe6e0c5c740 10 client.0 _ll_register_callbacks cb 0x55a5e9ea49e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:42:45.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:45.531+0000 7fe6cfe00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a5e9f91b90 0x55a5e9f99cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:42:45.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:45.532+0000 7fe6cfe00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a5e9f91b90 0x55a5e9f99cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:58492/0 (socket says 172.21.15.5:58492) 2024-09-10T09:42:45.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:45.532+0000 7fe6cfe00640 1 -- 172.21.15.5:0/3015096636 learned_addr learned my addr 172.21.15.5:0/3015096636 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:42:45.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:45.532+0000 7fe6d7400640 1 --2- 172.21.15.5:0/3015096636 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a5e9f924b0 0x55a5e9f9a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:42:45.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:45.532+0000 7fe6cfe00640 1 --2- 172.21.15.5:0/3015096636 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a5e9f9cd70 0x55a5e9f9a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:42:45.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:45.532+0000 7fe6cfe00640 1 -- 172.21.15.5:0/3015096636 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a5e9f9cd70 msgr2=0x55a5e9f9a750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:42:45.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:45.532+0000 7fe6cfe00640 1 --2- 172.21.15.5:0/3015096636 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a5e9f9cd70 0x55a5e9f9a750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:45.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:45.532+0000 7fe6cfe00640 1 -- 172.21.15.5:0/3015096636 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a5e9f924b0 msgr2=0x55a5e9f9a210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:42:45.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:45.532+0000 7fe6cfe00640 1 --2- 172.21.15.5:0/3015096636 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a5e9f924b0 0x55a5e9f9a210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:45.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:45.532+0000 7fe6cfe00640 1 -- 172.21.15.5:0/3015096636 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a5e9c693c0 con 0x55a5e9f91b90 2024-09-10T09:42:45.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:45.532+0000 7fe6cfe00640 1 --2- 172.21.15.5:0/3015096636 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a5e9f91b90 0x55a5e9f99cd0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fe6c0011f80 tx=0x7fe6c0009570 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b4469b533de766bf server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:42:45.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:45.532+0000 7fe6c4c00640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-09-10T09:42:45.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:45.533+0000 7fe6c4c00640 1 -- 172.21.15.5:0/3015096636 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6c0004030 con 0x55a5e9f91b90 2024-09-10T09:42:45.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:45.533+0000 7fe6e0c5c740 1 -- 172.21.15.5:0/3015096636 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe6c8004030 con 0x55a5e9f91b90 2024-09-10T09:42:45.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:45.533+0000 7fe6c4c00640 1 -- 172.21.15.5:0/3015096636 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fe6c00041d0 con 0x55a5e9f91b90 2024-09-10T09:42:45.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:45.533+0000 7fe6c4c00640 1 -- 172.21.15.5:0/3015096636 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fe6c0027050 con 0x55a5e9f91b90 2024-09-10T09:42:45.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:45.533+0000 7fe6e0c5c740 10 client.26308 fetch_fsmap learned FSMap version 33 2024-09-10T09:42:45.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:45.533+0000 7fe6e0c5c740 1 -- 172.21.15.5:0/3015096636 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55a5e9fab790 con 0x55a5e9f91b90 2024-09-10T09:42:45.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:45.533+0000 7fe6c4c00640 1 -- 172.21.15.5:0/3015096636 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe6c8004030 con 0x55a5e9f91b90 2024-09-10T09:42:45.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:45.533+0000 7fe6c4c00640 1 -- 172.21.15.5:0/3015096636 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 33) ==== 8374+0+0 (secure 0 0 0) 0x7fe6c00214c0 con 0x55a5e9f91b90 2024-09-10T09:42:45.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:45.534+0000 7fe6e0c5c740 10 client.26308 fetch_fsmap finished waiting for FSMap version 33 2024-09-10T09:42:45.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:45.534+0000 7fe6e0c5c740 10 client.26308 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T09:42:45.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:45.534+0000 7fe6e0c5c740 20 client.26308 populate_metadata read hostname 'smithi005' 2024-09-10T09:42:45.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:45.534+0000 7fe6e0c5c740 1 --2- 172.21.15.5:0/3015096636 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55a5e9facba0 0x55a5e9faef90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:42:45.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:45.534+0000 7fe6e0c5c740 4 client.26308 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:42:45.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:45.534+0000 7fe6e0c5c740 1 -- 172.21.15.5:0/3015096636 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a5e9f05950 con 0x55a5e9facba0 2024-09-10T09:42:45.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:45.534+0000 7fe6d7400640 1 --2- 172.21.15.5:0/3015096636 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55a5e9facba0 0x55a5e9faef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:42:45.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:45.535+0000 7fe6d7400640 1 --2- 172.21.15.5:0/3015096636 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55a5e9facba0 0x55a5e9faef90 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6f5e04e2cdaf4d19 server_cookie=5cd6cbfdc07b5740 in_seq=0 out_seq=0 2024-09-10T09:42:45.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:45.535+0000 7fe6c4c00640 10 client.26308 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T09:42:45.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:45.535+0000 7fe6c4c00640 1 -- 172.21.15.5:0/3015096636 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a5e9f05950 con 0x55a5e9facba0 2024-09-10T09:42:45.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:45.535+0000 7fe6c4c00640 10 client.26308 handle_command_reply: tid=0 2024-09-10T09:42:45.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:45.535+0000 7fe6e0c5c740 1 client.26308 shutdown 2024-09-10T09:42:45.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:45.536+0000 7fe6e0c5c740 1 -- 172.21.15.5:0/3015096636 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a5e9f91b90 msgr2=0x55a5e9f99cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:42:45.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:45.536+0000 7fe6e0c5c740 1 --2- 172.21.15.5:0/3015096636 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a5e9f91b90 0x55a5e9f99cd0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fe6c0011f80 tx=0x7fe6c0009570 comp rx=0 tx=0).stop 2024-09-10T09:42:45.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:45.536+0000 7fe6e0c5c740 1 -- 172.21.15.5:0/3015096636 shutdown_connections 2024-09-10T09:42:45.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:45.536+0000 7fe6e0c5c740 1 --2- 172.21.15.5:0/3015096636 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55a5e9facba0 0x55a5e9faef90 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:45.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:45.536+0000 7fe6e0c5c740 1 --2- 172.21.15.5:0/3015096636 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a5e9f9cd70 0x55a5e9f9a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:45.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:45.536+0000 7fe6e0c5c740 1 --2- 172.21.15.5:0/3015096636 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a5e9f924b0 0x55a5e9f9a210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:45.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:45.536+0000 7fe6e0c5c740 1 --2- 172.21.15.5:0/3015096636 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a5e9f91b90 0x55a5e9f99cd0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:45.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:45.536+0000 7fe6c4c00640 0 client.26308 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T09:42:45.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:45.536+0000 7fe6e0c5c740 1 -- 172.21.15.5:0/3015096636 >> 172.21.15.5:0/3015096636 conn(0x55a5e9f8e120 msgr2=0x55a5e9f93e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:42:45.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:45.536+0000 7fe6e0c5c740 1 -- 172.21.15.5:0/3015096636 shutdown_connections 2024-09-10T09:42:45.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:45.536+0000 7fe6e0c5c740 1 -- 172.21.15.5:0/3015096636 wait complete. 2024-09-10T09:42:45.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:45.536+0000 7fe6e0c5c740 20 client.26308 trim_cache size 0 max 16384 2024-09-10T09:42:45.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:45.558+0000 7fe6e0c5c740 1 Processor -- start 2024-09-10T09:42:45.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:45.558+0000 7fe6e0c5c740 1 -- start start 2024-09-10T09:42:45.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:45.558+0000 7fe6e0c5c740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a5e9f91b90 0x55a5e9f8d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:42:45.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:45.559+0000 7fe6e0c5c740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a5e9f924b0 0x55a5e9f9a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:42:45.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:45.559+0000 7fe6e0c5c740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a5e9f9cd70 0x55a5e9fb47f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:42:45.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:45.559+0000 7fe6e0c5c740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe6d81b10d0 con 0x55a5e9f9cd70 2024-09-10T09:42:45.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:45.559+0000 7fe6e0c5c740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fe6d81b2c40 con 0x55a5e9f924b0 2024-09-10T09:42:45.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:45.559+0000 7fe6e0c5c740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fe6d81b17a0 con 0x55a5e9f91b90 2024-09-10T09:42:45.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:45.559+0000 7fe6d7400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a5e9f924b0 0x55a5e9f9a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:42:45.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:45.559+0000 7fe6d7400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a5e9f924b0 0x55a5e9f9a5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:34778/0 (socket says 172.21.15.5:34778) 2024-09-10T09:42:45.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:45.559+0000 7fe6d7400640 1 -- 172.21.15.5:0/1631789048 learned_addr learned my addr 172.21.15.5:0/1631789048 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:42:45.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:45.559+0000 7fe6cfe00640 1 --2- 172.21.15.5:0/1631789048 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a5e9f91b90 0x55a5e9f8d0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:42:45.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:45.559+0000 7fe6cfe00640 1 --2- 172.21.15.5:0/1631789048 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a5e9f9cd70 0x55a5e9fb47f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:42:45.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:45.559+0000 7fe6d7400640 1 -- 172.21.15.5:0/1631789048 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a5e9f91b90 msgr2=0x55a5e9f8d0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:42:45.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:45.559+0000 7fe6d7400640 1 --2- 172.21.15.5:0/1631789048 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a5e9f91b90 0x55a5e9f8d0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:45.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:45.559+0000 7fe6d7400640 1 -- 172.21.15.5:0/1631789048 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a5e9f9cd70 msgr2=0x55a5e9fb47f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:42:45.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:45.559+0000 7fe6d7400640 1 --2- 172.21.15.5:0/1631789048 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a5e9f9cd70 0x55a5e9fb47f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:45.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:45.559+0000 7fe6d7400640 1 -- 172.21.15.5:0/1631789048 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a5e9faf5c0 con 0x55a5e9f924b0 2024-09-10T09:42:45.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:45.560+0000 7fe6d7400640 1 --2- 172.21.15.5:0/1631789048 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a5e9f924b0 0x55a5e9f9a5a0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fe6c8009a20 tx=0x7fe6c8005180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a162ca1bb3fd312e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:42:45.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:45.560+0000 7fe6c7400640 1 -- 172.21.15.5:0/1631789048 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6c8040040 con 0x55a5e9f924b0 2024-09-10T09:42:45.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:45.560+0000 7fe6c7400640 1 -- 172.21.15.5:0/1631789048 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe6c8009520 con 0x55a5e9f924b0 2024-09-10T09:42:45.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:45.560+0000 7fe6c7400640 1 -- 172.21.15.5:0/1631789048 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6c8035040 con 0x55a5e9f924b0 2024-09-10T09:42:45.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:45.561+0000 7fe6e0c5c740 1 -- 172.21.15.5:0/1631789048 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a5e9f924b0 msgr2=0x55a5e9f9a5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:42:45.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:45.561+0000 7fe6e0c5c740 1 --2- 172.21.15.5:0/1631789048 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a5e9f924b0 0x55a5e9f9a5a0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fe6c8009a20 tx=0x7fe6c8005180 comp rx=0 tx=0).stop 2024-09-10T09:42:45.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:45.561+0000 7fe6e0c5c740 1 -- 172.21.15.5:0/1631789048 shutdown_connections 2024-09-10T09:42:45.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:45.561+0000 7fe6e0c5c740 1 --2- 172.21.15.5:0/1631789048 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a5e9f9cd70 0x55a5e9fb47f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:45.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:45.561+0000 7fe6e0c5c740 1 --2- 172.21.15.5:0/1631789048 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a5e9f924b0 0x55a5e9f9a5a0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:45.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:45.561+0000 7fe6e0c5c740 1 --2- 172.21.15.5:0/1631789048 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a5e9f91b90 0x55a5e9f8d0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:45.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:45.561+0000 7fe6e0c5c740 1 -- 172.21.15.5:0/1631789048 >> 172.21.15.5:0/1631789048 conn(0x55a5e9f8e120 msgr2=0x55a5e9f8c9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:42:45.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:45.561+0000 7fe6e0c5c740 1 -- 172.21.15.5:0/1631789048 shutdown_connections 2024-09-10T09:42:45.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:45.561+0000 7fe6e0c5c740 1 -- 172.21.15.5:0/1631789048 wait complete. 2024-09-10T09:42:45.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:45.561+0000 7fe6e0c5c740 1 -- start start 2024-09-10T09:42:45.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:45.562+0000 7fe6e0c5c740 1 Processor -- start 2024-09-10T09:42:45.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:45.562+0000 7fe6e0c5c740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a5e9f91b90 0x55a5e9fa0ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:42:45.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:45.562+0000 7fe6e0c5c740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a5e9f924b0 0x55a5e9fa1410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:42:45.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:45.562+0000 7fe6e0c5c740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a5e9f9cd70 0x55a5ea0167c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:42:45.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:45.562+0000 7fe6e0c5c740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe6d81b2540 con 0x55a5e9f9cd70 2024-09-10T09:42:45.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:45.562+0000 7fe6e0c5c740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fe6d81b1e70 con 0x55a5e9f924b0 2024-09-10T09:42:45.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:45.562+0000 7fe6e0c5c740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fe6d81af0b0 con 0x55a5e9f91b90 2024-09-10T09:42:45.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:45.562+0000 7fe6e0c5c740 10 client.0 _ll_register_callbacks cb 0x7fe6c0011da0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:42:45.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:45.563+0000 7fe6d7400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a5e9f924b0 0x55a5e9fa1410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:42:45.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:45.563+0000 7fe6d7400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a5e9f924b0 0x55a5e9fa1410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:34782/0 (socket says 172.21.15.5:34782) 2024-09-10T09:42:45.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:45.563+0000 7fe6d7400640 1 -- 172.21.15.5:0/3940062571 learned_addr learned my addr 172.21.15.5:0/3940062571 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:42:45.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:45.563+0000 7fe6cfe00640 1 --2- 172.21.15.5:0/3940062571 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a5e9f9cd70 0x55a5ea0167c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:42:45.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:45.563+0000 7fe6cfe00640 1 --2- 172.21.15.5:0/3940062571 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a5e9f91b90 0x55a5e9fa0ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:42:45.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:45.563+0000 7fe6d7400640 1 -- 172.21.15.5:0/3940062571 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a5e9f91b90 msgr2=0x55a5e9fa0ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:42:45.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:45.563+0000 7fe6d7400640 1 --2- 172.21.15.5:0/3940062571 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a5e9f91b90 0x55a5e9fa0ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:45.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:45.563+0000 7fe6d7400640 1 -- 172.21.15.5:0/3940062571 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a5e9f9cd70 msgr2=0x55a5ea0167c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:42:45.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:45.563+0000 7fe6d7400640 1 --2- 172.21.15.5:0/3940062571 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a5e9f9cd70 0x55a5ea0167c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:45.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:45.563+0000 7fe6d7400640 1 -- 172.21.15.5:0/3940062571 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a5e9f8edb0 con 0x55a5e9f924b0 2024-09-10T09:42:45.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:45.563+0000 7fe6cfe00640 1 --2- 172.21.15.5:0/3940062571 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a5e9f91b90 0x55a5e9fa0ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T09:42:45.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:45.563+0000 7fe6d7400640 1 --2- 172.21.15.5:0/3940062571 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a5e9f924b0 0x55a5e9fa1410 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fe6c8000c00 tx=0x7fe6c802a9a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e9dc4dded58b1fd5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:42:45.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:45.564+0000 7fe6c4c00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-09-10T09:42:45.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:45.564+0000 7fe6c4c00640 1 -- 172.21.15.5:0/3940062571 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6c8035040 con 0x55a5e9f924b0 2024-09-10T09:42:45.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:45.564+0000 7fe6c4c00640 1 -- 172.21.15.5:0/3940062571 <== mon.2 v2:172.21.15.102:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fe6c80315b0 con 0x55a5e9f924b0 2024-09-10T09:42:45.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:45.564+0000 7fe6e0c5c740 1 -- 172.21.15.5:0/3940062571 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe6c8009520 con 0x55a5e9f924b0 2024-09-10T09:42:45.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:45.564+0000 7fe6c4c00640 1 -- 172.21.15.5:0/3940062571 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fe6c80317f0 con 0x55a5e9f924b0 2024-09-10T09:42:45.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:45.564+0000 7fe6e0c5c740 10 client.26288 fetch_fsmap learned FSMap version 33 2024-09-10T09:42:45.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:45.564+0000 7fe6e0c5c740 1 -- 172.21.15.5:0/3940062571 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55a5ea017c10 con 0x55a5e9f924b0 2024-09-10T09:42:45.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:45.565+0000 7fe6c4c00640 1 -- 172.21.15.5:0/3940062571 <== mon.2 v2:172.21.15.102:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe6c8009520 con 0x55a5e9f924b0 2024-09-10T09:42:45.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:45.566+0000 7fe6c4c00640 1 -- 172.21.15.5:0/3940062571 <== mon.2 v2:172.21.15.102:3300/0 5 ==== fsmap(e 33) ==== 8374+0+0 (secure 0 0 0) 0x7fe6c8065020 con 0x55a5e9f924b0 2024-09-10T09:42:45.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:45.566+0000 7fe6e0c5c740 10 client.26288 fetch_fsmap finished waiting for FSMap version 33 2024-09-10T09:42:45.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:45.566+0000 7fe6e0c5c740 10 client.26288 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T09:42:45.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:45.566+0000 7fe6e0c5c740 20 client.26288 populate_metadata read hostname 'smithi005' 2024-09-10T09:42:45.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:45.566+0000 7fe6e0c5c740 1 --2- 172.21.15.5:0/3940062571 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55a5ea018030 0x55a5ea01a420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:42:45.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:45.566+0000 7fe6e0c5c740 4 client.26288 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T09:42:45.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:45.566+0000 7fe6e0c5c740 1 -- 172.21.15.5:0/3940062571 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55a5e9effe60 con 0x55a5ea018030 2024-09-10T09:42:45.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:45.566+0000 7fe6cfe00640 1 --2- 172.21.15.5:0/3940062571 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55a5ea018030 0x55a5ea01a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:42:45.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:45.567+0000 7fe6cfe00640 1 --2- 172.21.15.5:0/3940062571 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55a5ea018030 0x55a5ea01a420 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a3fc9c0baae1b339 server_cookie=9d493d71719f4430 in_seq=0 out_seq=0 2024-09-10T09:42:45.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:45.567+0000 7fe6c4c00640 10 client.26288 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T09:42:45.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:45.567+0000 7fe6c4c00640 1 -- 172.21.15.5:0/3940062571 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55a5e9effe60 con 0x55a5ea018030 2024-09-10T09:42:45.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:45.567+0000 7fe6c4c00640 10 client.26288 handle_command_reply: tid=0 2024-09-10T09:42:45.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:45.567+0000 7fe6e0c5c740 1 client.26288 shutdown 2024-09-10T09:42:45.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:45.567+0000 7fe6e0c5c740 1 -- 172.21.15.5:0/3940062571 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a5e9f924b0 msgr2=0x55a5e9fa1410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:42:45.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:45.567+0000 7fe6e0c5c740 1 --2- 172.21.15.5:0/3940062571 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a5e9f924b0 0x55a5e9fa1410 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fe6c8000c00 tx=0x7fe6c802a9a0 comp rx=0 tx=0).stop 2024-09-10T09:42:45.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:45.568+0000 7fe6e0c5c740 1 -- 172.21.15.5:0/3940062571 shutdown_connections 2024-09-10T09:42:45.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:45.568+0000 7fe6e0c5c740 1 --2- 172.21.15.5:0/3940062571 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55a5ea018030 0x55a5ea01a420 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:45.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:45.568+0000 7fe6e0c5c740 1 --2- 172.21.15.5:0/3940062571 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a5e9f9cd70 0x55a5ea0167c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:45.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:45.568+0000 7fe6e0c5c740 1 --2- 172.21.15.5:0/3940062571 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a5e9f924b0 0x55a5e9fa1410 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:45.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:45.568+0000 7fe6e0c5c740 1 --2- 172.21.15.5:0/3940062571 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a5e9f91b90 0x55a5e9fa0ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:45.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:45.568+0000 7fe6c4c00640 0 client.26288 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T09:42:45.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:45.568+0000 7fe6e0c5c740 1 -- 172.21.15.5:0/3940062571 >> 172.21.15.5:0/3940062571 conn(0x55a5e9f8e120 msgr2=0x55a5e9f8c9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:42:45.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:45.568+0000 7fe6e0c5c740 1 -- 172.21.15.5:0/3940062571 shutdown_connections 2024-09-10T09:42:45.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:45.568+0000 7fe6e0c5c740 1 -- 172.21.15.5:0/3940062571 wait complete. 2024-09-10T09:42:45.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:45.568+0000 7fe6e0c5c740 20 client.26288 trim_cache size 0 max 16384 2024-09-10T09:42:45.570 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-10T09:42:45.570 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-10T09:42:45.570 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2024-09-10T09:42:45.571 INFO:teuthology.orchestra.run.smithi005.stdout: "e9d447d4-791d-4f12-905a-fd737c5e40a1": { 2024-09-10T09:42:45.571 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2024-09-10T09:42:45.571 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "e9d447d4-791d-4f12-905a-fd737c5e40a1", 2024-09-10T09:42:45.571 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2024-09-10T09:42:45.571 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T09:42:45.571 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T09:42:45.571 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-10T09:42:45.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:45.569+0000 7fe6de000640 1 -- 172.21.15.5:0/4266582905 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fe6bc0a2c00 msgr2=0x7fe6bc0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:42:45.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:45.569+0000 7fe6de000640 1 --2- 172.21.15.5:0/4266582905 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fe6bc0a2c00 0x7fe6bc0a50c0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fe6d0000f80 tx=0x7fe6d0005940 comp rx=0 tx=0).stop 2024-09-10T09:42:45.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:45.569+0000 7fe6de000640 1 -- 172.21.15.5:0/4266582905 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe6d8109140 msgr2=0x7fe6d819ef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:42:45.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:45.569+0000 7fe6de000640 1 --2- 172.21.15.5:0/4266582905 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe6d8109140 0x7fe6d819ef00 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fe6d0009fd0 tx=0x7fe6d000ef10 comp rx=0 tx=0).stop 2024-09-10T09:42:45.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:45.569+0000 7fe6de000640 1 -- 172.21.15.5:0/4266582905 shutdown_connections 2024-09-10T09:42:45.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:45.569+0000 7fe6de000640 1 --2- 172.21.15.5:0/4266582905 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fe6bc0a2c00 0x7fe6bc0a50c0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:45.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:45.569+0000 7fe6de000640 1 --2- 172.21.15.5:0/4266582905 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe6d8109140 0x7fe6d819ef00 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:45.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:45.569+0000 7fe6de000640 1 --2- 172.21.15.5:0/4266582905 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe6d81049c0 0x7fe6d819ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:45.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:45.569+0000 7fe6de000640 1 --2- 172.21.15.5:0/4266582905 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe6d8104010 0x7fe6d819a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:45.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:45.569+0000 7fe6de000640 1 -- 172.21.15.5:0/4266582905 >> 172.21.15.5:0/4266582905 conn(0x7fe6d80ffa50 msgr2=0x7fe6d8100e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:42:45.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:45.569+0000 7fe6de000640 1 -- 172.21.15.5:0/4266582905 shutdown_connections 2024-09-10T09:42:45.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:45.569+0000 7fe6de000640 1 -- 172.21.15.5:0/4266582905 wait complete. 2024-09-10T09:42:46.027 INFO:tasks.workunit.client.0.smithi005.stdout: 1 202280 2.07 MB/sec execute 27 sec latency 14.671 ms 2024-09-10T09:42:46.412 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:42:46.442 INFO:tasks.cephfs.filesystem:scrub status for tag:e9d447d4-791d-4f12-905a-fd737c5e40a1 - {'path': '/', 'tag': 'e9d447d4-791d-4f12-905a-fd737c5e40a1', 'options': 'recursive,force'} 2024-09-10T09:42:46.963 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:42:46 smithi005 ceph-mon[30350]: pgmap v526: 129 pgs: 129 active+clean; 90 MiB data, 878 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.9 MiB/s wr, 131 op/s 2024-09-10T09:42:46.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:46.994+0000 7f6141000640 1 -- 172.21.15.5:0/717977847 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f613c074510 msgr2=0x7f613c0748f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:42:46.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:46.994+0000 7f6141000640 1 --2- 172.21.15.5:0/717977847 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f613c074510 0x7f613c0748f0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f612c009900 tx=0x7f612c02a480 comp rx=0 tx=0).stop 2024-09-10T09:42:46.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:46.994+0000 7f6141000640 1 -- 172.21.15.5:0/717977847 shutdown_connections 2024-09-10T09:42:46.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:46.994+0000 7f6141000640 1 --2- 172.21.15.5:0/717977847 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f613c10e280 0x7f613c110670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:46.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:46.994+0000 7f6141000640 1 --2- 172.21.15.5:0/717977847 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f613c074e30 0x7f613c10dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:46.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:46.994+0000 7f6141000640 1 --2- 172.21.15.5:0/717977847 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f613c074510 0x7f613c0748f0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:46.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:46.994+0000 7f6141000640 1 -- 172.21.15.5:0/717977847 >> 172.21.15.5:0/717977847 conn(0x7f613c0fd740 msgr2=0x7f613c0ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:42:46.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:46.995+0000 7f6141000640 1 -- 172.21.15.5:0/717977847 shutdown_connections 2024-09-10T09:42:46.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:46.995+0000 7f6141000640 1 -- 172.21.15.5:0/717977847 wait complete. 2024-09-10T09:42:46.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:46.996+0000 7f6141000640 1 Processor -- start 2024-09-10T09:42:46.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:46.996+0000 7f6141000640 1 -- start start 2024-09-10T09:42:46.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:46.997+0000 7f6141000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f613c074510 0x7f613c19a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:42:46.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:46.997+0000 7f6141000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f613c074e30 0x7f613c19aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:42:46.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:46.997+0000 7f6141000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f613c10e280 0x7f613c19eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:42:46.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:46.997+0000 7f6141000640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f613c114000 con 0x7f613c074e30 2024-09-10T09:42:46.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:46.997+0000 7f6141000640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f613c113e80 con 0x7f613c074510 2024-09-10T09:42:46.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:46.997+0000 7f6141000640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f613c114180 con 0x7f613c10e280 2024-09-10T09:42:46.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:46.997+0000 7f613b400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f613c074510 0x7f613c19a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:42:46.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:46.997+0000 7f613b400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f613c074510 0x7f613c19a4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:34802/0 (socket says 172.21.15.5:34802) 2024-09-10T09:42:46.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:46.997+0000 7f613b400640 1 -- 172.21.15.5:0/3459345700 learned_addr learned my addr 172.21.15.5:0/3459345700 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:42:46.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:46.997+0000 7f613aa00640 1 --2- 172.21.15.5:0/3459345700 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f613c074e30 0x7f613c19aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:42:46.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:46.997+0000 7f613be00640 1 --2- 172.21.15.5:0/3459345700 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f613c10e280 0x7f613c19eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:42:46.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:46.997+0000 7f613b400640 1 -- 172.21.15.5:0/3459345700 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f613c10e280 msgr2=0x7f613c19eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:42:46.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:46.997+0000 7f613b400640 1 --2- 172.21.15.5:0/3459345700 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f613c10e280 0x7f613c19eda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:46.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:46.997+0000 7f613b400640 1 -- 172.21.15.5:0/3459345700 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f613c074e30 msgr2=0x7f613c19aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:42:47.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:46.997+0000 7f613b400640 1 --2- 172.21.15.5:0/3459345700 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f613c074e30 0x7f613c19aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:47.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:46.997+0000 7f613b400640 1 -- 172.21.15.5:0/3459345700 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f613c19f480 con 0x7f613c074510 2024-09-10T09:42:47.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:46.998+0000 7f613b400640 1 --2- 172.21.15.5:0/3459345700 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f613c074510 0x7f613c19a4d0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f612c004770 tx=0x7f612c003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:42:47.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:46.998+0000 7f6133e00640 1 -- 172.21.15.5:0/3459345700 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f612c03f070 con 0x7f613c074510 2024-09-10T09:42:47.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:46.998+0000 7f6133e00640 1 -- 172.21.15.5:0/3459345700 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f612c031ca0 con 0x7f613c074510 2024-09-10T09:42:47.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:46.998+0000 7f6133e00640 1 -- 172.21.15.5:0/3459345700 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f612c035030 con 0x7f613c074510 2024-09-10T09:42:47.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:46.998+0000 7f6141000640 1 -- 172.21.15.5:0/3459345700 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f613c19f710 con 0x7f613c074510 2024-09-10T09:42:47.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:46.998+0000 7f6141000640 1 -- 172.21.15.5:0/3459345700 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f613c1a7050 con 0x7f613c074510 2024-09-10T09:42:47.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:47.002+0000 7f614520c740 1 Processor -- start 2024-09-10T09:42:47.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:47.002+0000 7f6133e00640 1 -- 172.21.15.5:0/3459345700 <== mon.2 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f612c042070 con 0x7f613c074510 2024-09-10T09:42:47.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:47.002+0000 7f614520c740 1 -- start start 2024-09-10T09:42:47.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:47.002+0000 7f614520c740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5608b11b06c0 0x5608b11b0aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:42:47.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:47.002+0000 7f614520c740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5608b11b0fe0 0x5608b11bb360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:42:47.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:47.002+0000 7f614520c740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5608b11bb8a0 0x5608b11bdc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:42:47.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:47.002+0000 7f614520c740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5608b1115260 con 0x5608b11bb8a0 2024-09-10T09:42:47.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:47.002+0000 7f614520c740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x5608b10bc500 con 0x5608b11b0fe0 2024-09-10T09:42:47.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:47.002+0000 7f614520c740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5608b1111220 con 0x5608b11b06c0 2024-09-10T09:42:47.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:47.003+0000 7f613be00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5608b11b06c0 0x5608b11b0aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:42:47.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:47.003+0000 7f613b400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5608b11b0fe0 0x5608b11bb360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:42:47.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:47.003+0000 7f613be00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5608b11b06c0 0x5608b11b0aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:58520/0 (socket says 172.21.15.5:58520) 2024-09-10T09:42:47.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:47.003+0000 7f613be00640 1 -- 172.21.15.5:0/3713958942 learned_addr learned my addr 172.21.15.5:0/3713958942 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:42:47.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:47.003+0000 7f613aa00640 1 --2- 172.21.15.5:0/3713958942 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5608b11bb8a0 0x5608b11bdc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:42:47.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:47.003+0000 7f613be00640 1 -- 172.21.15.5:0/3713958942 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5608b11b0fe0 msgr2=0x5608b11bb360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:42:47.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:47.003+0000 7f613be00640 1 --2- 172.21.15.5:0/3713958942 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5608b11b0fe0 0x5608b11bb360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:47.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:47.003+0000 7f613be00640 1 -- 172.21.15.5:0/3713958942 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5608b11bb8a0 msgr2=0x5608b11bdc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:42:47.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:47.003+0000 7f613be00640 1 --2- 172.21.15.5:0/3713958942 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5608b11bb8a0 0x5608b11bdc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:47.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:47.003+0000 7f613be00640 1 -- 172.21.15.5:0/3713958942 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5608b0e86ba0 con 0x5608b11b06c0 2024-09-10T09:42:47.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:47.003+0000 7f613be00640 1 --2- 172.21.15.5:0/3713958942 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5608b11b06c0 0x5608b11b0aa0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f6134009f30 tx=0x7f61340118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d091cfb8e7751ce8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:42:47.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:47.004+0000 7f6130c00640 1 -- 172.21.15.5:0/3713958942 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6134004d20 con 0x5608b11b06c0 2024-09-10T09:42:47.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:47.004+0000 7f6130c00640 1 -- 172.21.15.5:0/3713958942 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6134004050 con 0x5608b11b06c0 2024-09-10T09:42:47.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:47.004+0000 7f6130c00640 1 -- 172.21.15.5:0/3713958942 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6134005280 con 0x5608b11b06c0 2024-09-10T09:42:47.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:47.004+0000 7f6133e00640 1 --2- 172.21.15.5:0/3459345700 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f61200a2c00 0x7f61200a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:42:47.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:47.004+0000 7f6133e00640 1 -- 172.21.15.5:0/3459345700 <== mon.2 v2:172.21.15.102:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f612c0ef5e0 con 0x7f613c074510 2024-09-10T09:42:47.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:47.004+0000 7f613b400640 1 --2- 172.21.15.5:0/3459345700 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f61200a2c00 0x7f61200a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:42:47.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:47.004+0000 7f614520c740 1 -- 172.21.15.5:0/3713958942 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5608b11b06c0 msgr2=0x5608b11b0aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:42:47.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:47.005+0000 7f614520c740 1 --2- 172.21.15.5:0/3713958942 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5608b11b06c0 0x5608b11b0aa0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f6134009f30 tx=0x7f61340118d0 comp rx=0 tx=0).stop 2024-09-10T09:42:47.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:47.005+0000 7f614520c740 1 -- 172.21.15.5:0/3713958942 shutdown_connections 2024-09-10T09:42:47.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:47.005+0000 7f614520c740 1 --2- 172.21.15.5:0/3713958942 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5608b11bb8a0 0x5608b11bdc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:47.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:47.005+0000 7f614520c740 1 --2- 172.21.15.5:0/3713958942 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5608b11b0fe0 0x5608b11bb360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:47.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:47.005+0000 7f614520c740 1 --2- 172.21.15.5:0/3713958942 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5608b11b06c0 0x5608b11b0aa0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:47.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:47.005+0000 7f614520c740 1 -- 172.21.15.5:0/3713958942 >> 172.21.15.5:0/3713958942 conn(0x5608b11acc50 msgr2=0x5608b11ad030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:42:47.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:47.005+0000 7f614520c740 1 -- 172.21.15.5:0/3713958942 shutdown_connections 2024-09-10T09:42:47.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:47.005+0000 7f614520c740 1 -- 172.21.15.5:0/3713958942 wait complete. 2024-09-10T09:42:47.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:47.005+0000 7f613b400640 1 --2- 172.21.15.5:0/3459345700 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f61200a2c00 0x7f61200a50c0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f612c0ed060 tx=0x7f612c0f2f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:42:47.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:47.006+0000 7f614520c740 1 -- start start 2024-09-10T09:42:47.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:47.006+0000 7f614520c740 1 Processor -- start 2024-09-10T09:42:47.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:47.006+0000 7f614520c740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5608b11b06c0 0x5608b11b87c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:42:47.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:47.006+0000 7f614520c740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5608b11b0fe0 0x5608b11b8d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:42:47.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:47.006+0000 7f614520c740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5608b11bb8a0 0x5608b11b9240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:42:47.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:47.006+0000 7f614520c740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5608b11be470 con 0x5608b11b0fe0 2024-09-10T09:42:47.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:47.006+0000 7f614520c740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x5608b10cfbd0 con 0x5608b11bb8a0 2024-09-10T09:42:47.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:47.006+0000 7f614520c740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5608b11be5f0 con 0x5608b11b06c0 2024-09-10T09:42:47.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:47.006+0000 7f614520c740 10 client.0 _ll_register_callbacks cb 0x5608b0e240a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:42:47.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:47.007+0000 7f613aa00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5608b11b0fe0 0x5608b11b8d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:42:47.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:47.007+0000 7f613aa00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5608b11b0fe0 0x5608b11b8d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50190/0 (socket says 172.21.15.5:50190) 2024-09-10T09:42:47.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:47.007+0000 7f613aa00640 1 -- 172.21.15.5:0/1979487850 learned_addr learned my addr 172.21.15.5:0/1979487850 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:42:47.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:47.007+0000 7f613be00640 1 --2- 172.21.15.5:0/1979487850 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5608b11b06c0 0x5608b11b87c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:42:47.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:47.007+0000 7f613be00640 1 --2- 172.21.15.5:0/1979487850 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5608b11bb8a0 0x5608b11b9240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:42:47.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:47.007+0000 7f613aa00640 1 -- 172.21.15.5:0/1979487850 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5608b11b06c0 msgr2=0x5608b11b87c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:42:47.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:47.007+0000 7f613aa00640 1 --2- 172.21.15.5:0/1979487850 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5608b11b06c0 0x5608b11b87c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:47.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:47.007+0000 7f613aa00640 1 -- 172.21.15.5:0/1979487850 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5608b11bb8a0 msgr2=0x5608b11b9240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:42:47.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:47.007+0000 7f613aa00640 1 --2- 172.21.15.5:0/1979487850 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5608b11bb8a0 0x5608b11b9240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:47.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:47.007+0000 7f613aa00640 1 -- 172.21.15.5:0/1979487850 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5608b11c7910 con 0x5608b11b0fe0 2024-09-10T09:42:47.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:47.007+0000 7f613be00640 1 --2- 172.21.15.5:0/1979487850 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5608b11b06c0 0x5608b11b87c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:42:47.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:47.007+0000 7f613aa00640 1 --2- 172.21.15.5:0/1979487850 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5608b11b0fe0 0x5608b11b8d00 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f6128013740 tx=0x7f6128013c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9f7d5b8f405349b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:42:47.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:47.007+0000 7f6125600640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-10T09:42:47.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:47.008+0000 7f6125600640 1 -- 172.21.15.5:0/1979487850 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6128011960 con 0x5608b11b0fe0 2024-09-10T09:42:47.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:47.008+0000 7f6125600640 1 -- 172.21.15.5:0/1979487850 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f6128011b00 con 0x5608b11b0fe0 2024-09-10T09:42:47.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:47.008+0000 7f614520c740 1 -- 172.21.15.5:0/1979487850 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6134004050 con 0x5608b11b0fe0 2024-09-10T09:42:47.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:47.008+0000 7f6125600640 1 -- 172.21.15.5:0/1979487850 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f6128011d00 con 0x5608b11b0fe0 2024-09-10T09:42:47.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:47.008+0000 7f614520c740 10 client.16293 fetch_fsmap learned FSMap version 33 2024-09-10T09:42:47.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:47.008+0000 7f614520c740 1 -- 172.21.15.5:0/1979487850 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5608b11ca2e0 con 0x5608b11b0fe0 2024-09-10T09:42:47.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:47.008+0000 7f6125600640 1 -- 172.21.15.5:0/1979487850 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6134004050 con 0x5608b11b0fe0 2024-09-10T09:42:47.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:47.009+0000 7f6125600640 1 -- 172.21.15.5:0/1979487850 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 33) ==== 8374+0+0 (secure 0 0 0) 0x7f612801a070 con 0x5608b11b0fe0 2024-09-10T09:42:47.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:47.009+0000 7f614520c740 10 client.16293 fetch_fsmap finished waiting for FSMap version 33 2024-09-10T09:42:47.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:47.009+0000 7f614520c740 10 client.16293 resolve_mds: validated gid 24542 aka daemon mds.b 2024-09-10T09:42:47.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:47.009+0000 7f614520c740 20 client.16293 populate_metadata read hostname 'smithi005' 2024-09-10T09:42:47.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:47.009+0000 7f614520c740 1 --2- 172.21.15.5:0/1979487850 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x5608b11cb890 0x5608b11cdcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:42:47.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:47.009+0000 7f614520c740 4 client.16293 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:42:47.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:47.009+0000 7f614520c740 1 -- 172.21.15.5:0/1979487850 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5608b1122990 con 0x5608b11cb890 2024-09-10T09:42:47.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:47.009+0000 7f613be00640 1 --2- 172.21.15.5:0/1979487850 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x5608b11cb890 0x5608b11cdcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:42:47.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:47.010+0000 7f6125600640 10 client.16293 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T09:42:47.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:47.010+0000 7f613be00640 1 --2- 172.21.15.5:0/1979487850 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x5608b11cb890 0x5608b11cdcb0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=219765f92ef28abb server_cookie=cddb555a3e171da9 in_seq=0 out_seq=0 2024-09-10T09:42:47.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:47.010+0000 7f6125600640 1 -- 172.21.15.5:0/1979487850 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5608b1122990 con 0x5608b11cb890 2024-09-10T09:42:47.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:47.010+0000 7f6125600640 10 client.16293 handle_command_reply: tid=0 2024-09-10T09:42:47.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:47.011+0000 7f614520c740 1 client.16293 shutdown 2024-09-10T09:42:47.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:47.011+0000 7f614520c740 1 -- 172.21.15.5:0/1979487850 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5608b11b0fe0 msgr2=0x5608b11b8d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:42:47.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:47.011+0000 7f614520c740 1 --2- 172.21.15.5:0/1979487850 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5608b11b0fe0 0x5608b11b8d00 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f6128013740 tx=0x7f6128013c00 comp rx=0 tx=0).stop 2024-09-10T09:42:47.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:47.011+0000 7f614520c740 1 -- 172.21.15.5:0/1979487850 shutdown_connections 2024-09-10T09:42:47.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:47.011+0000 7f614520c740 1 --2- 172.21.15.5:0/1979487850 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x5608b11cb890 0x5608b11cdcb0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:47.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:47.011+0000 7f614520c740 1 --2- 172.21.15.5:0/1979487850 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5608b11bb8a0 0x5608b11b9240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:47.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:47.011+0000 7f614520c740 1 --2- 172.21.15.5:0/1979487850 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5608b11b0fe0 0x5608b11b8d00 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:47.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:47.011+0000 7f614520c740 1 --2- 172.21.15.5:0/1979487850 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5608b11b06c0 0x5608b11b87c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:47.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:47.011+0000 7f614520c740 1 -- 172.21.15.5:0/1979487850 >> 172.21.15.5:0/1979487850 conn(0x5608b11acc50 msgr2=0x5608b11abe70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:42:47.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:47.011+0000 7f6125600640 0 client.16293 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T09:42:47.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:47.011+0000 7f614520c740 1 -- 172.21.15.5:0/1979487850 shutdown_connections 2024-09-10T09:42:47.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:47.011+0000 7f614520c740 1 -- 172.21.15.5:0/1979487850 wait complete. 2024-09-10T09:42:47.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:47.011+0000 7f614520c740 20 client.16293 trim_cache size 0 max 16384 2024-09-10T09:42:47.026 INFO:tasks.workunit.client.0.smithi005.stdout: 1 204012 2.24 MB/sec execute 28 sec latency 20.013 ms 2024-09-10T09:42:47.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:47.034+0000 7f614520c740 1 Processor -- start 2024-09-10T09:42:47.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:47.034+0000 7f614520c740 1 -- start start 2024-09-10T09:42:47.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:47.034+0000 7f614520c740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5608b11b06c0 0x5608b11ad030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:42:47.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:47.034+0000 7f614520c740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5608b11b0fe0 0x5608b11ba620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:42:47.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:47.034+0000 7f614520c740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5608b11bb8a0 0x5608b11d3560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:42:47.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:47.034+0000 7f614520c740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f613c1b0f20 con 0x5608b11bb8a0 2024-09-10T09:42:47.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:47.034+0000 7f614520c740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f613c1b2a90 con 0x5608b11b0fe0 2024-09-10T09:42:47.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:47.034+0000 7f614520c740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f613c1b15f0 con 0x5608b11b06c0 2024-09-10T09:42:47.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:47.034+0000 7f613aa00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5608b11b0fe0 0x5608b11ba620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:42:47.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:47.034+0000 7f613aa00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5608b11b0fe0 0x5608b11ba620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:34844/0 (socket says 172.21.15.5:34844) 2024-09-10T09:42:47.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:47.034+0000 7f613aa00640 1 -- 172.21.15.5:0/791873986 learned_addr learned my addr 172.21.15.5:0/791873986 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:42:47.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:47.034+0000 7f613be00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5608b11b06c0 0x5608b11ad030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:42:47.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:47.035+0000 7f613be00640 1 --2- 172.21.15.5:0/791873986 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5608b11bb8a0 0x5608b11d3560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:42:47.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:47.035+0000 7f613aa00640 1 -- 172.21.15.5:0/791873986 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5608b11b06c0 msgr2=0x5608b11ad030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:42:47.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:47.035+0000 7f613aa00640 1 --2- 172.21.15.5:0/791873986 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5608b11b06c0 0x5608b11ad030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:47.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:47.035+0000 7f613aa00640 1 -- 172.21.15.5:0/791873986 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5608b11bb8a0 msgr2=0x5608b11d3560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:42:47.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:47.035+0000 7f613aa00640 1 --2- 172.21.15.5:0/791873986 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5608b11bb8a0 0x5608b11d3560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:47.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:47.035+0000 7f613aa00640 1 -- 172.21.15.5:0/791873986 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5608b11b7770 con 0x5608b11b0fe0 2024-09-10T09:42:47.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:47.035+0000 7f613be00640 1 --2- 172.21.15.5:0/791873986 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5608b11bb8a0 0x5608b11d3560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:42:47.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:47.035+0000 7f613aa00640 1 --2- 172.21.15.5:0/791873986 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5608b11b0fe0 0x5608b11ba620 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f6128001eb0 tx=0x7f6128005c50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6ba770c54d1d1e9d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:42:47.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:47.036+0000 7f6127e00640 1 -- 172.21.15.5:0/791873986 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6128030070 con 0x5608b11b0fe0 2024-09-10T09:42:47.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:47.036+0000 7f6127e00640 1 -- 172.21.15.5:0/791873986 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f612801cb20 con 0x5608b11b0fe0 2024-09-10T09:42:47.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:47.036+0000 7f6127e00640 1 -- 172.21.15.5:0/791873986 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6128003bd0 con 0x5608b11b0fe0 2024-09-10T09:42:47.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:47.036+0000 7f614520c740 1 -- 172.21.15.5:0/791873986 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5608b11b0fe0 msgr2=0x5608b11ba620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:42:47.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:47.036+0000 7f614520c740 1 --2- 172.21.15.5:0/791873986 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5608b11b0fe0 0x5608b11ba620 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f6128001eb0 tx=0x7f6128005c50 comp rx=0 tx=0).stop 2024-09-10T09:42:47.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:47.037+0000 7f614520c740 1 -- 172.21.15.5:0/791873986 shutdown_connections 2024-09-10T09:42:47.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:47.037+0000 7f614520c740 1 --2- 172.21.15.5:0/791873986 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5608b11bb8a0 0x5608b11d3560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:47.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:47.037+0000 7f614520c740 1 --2- 172.21.15.5:0/791873986 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5608b11b0fe0 0x5608b11ba620 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:47.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:47.037+0000 7f614520c740 1 --2- 172.21.15.5:0/791873986 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5608b11b06c0 0x5608b11ad030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:47.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:47.037+0000 7f614520c740 1 -- 172.21.15.5:0/791873986 >> 172.21.15.5:0/791873986 conn(0x5608b11acc50 msgr2=0x5608b11b9f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:42:47.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:47.037+0000 7f614520c740 1 -- 172.21.15.5:0/791873986 shutdown_connections 2024-09-10T09:42:47.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:47.037+0000 7f614520c740 1 -- 172.21.15.5:0/791873986 wait complete. 2024-09-10T09:42:47.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:47.037+0000 7f614520c740 1 -- start start 2024-09-10T09:42:47.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:47.037+0000 7f614520c740 1 Processor -- start 2024-09-10T09:42:47.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:47.038+0000 7f614520c740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5608b11b06c0 0x5608b11c3840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:42:47.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:47.038+0000 7f614520c740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5608b11b0fe0 0x5608b11c3d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:42:47.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:47.038+0000 7f614520c740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5608b11bb8a0 0x5608b11c0810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:42:47.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:47.038+0000 7f614520c740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f613c1b2390 con 0x5608b11b0fe0 2024-09-10T09:42:47.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:47.038+0000 7f614520c740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f613c1b1cc0 con 0x5608b11bb8a0 2024-09-10T09:42:47.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:47.038+0000 7f614520c740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f613c1aef00 con 0x5608b11b06c0 2024-09-10T09:42:47.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:47.038+0000 7f614520c740 10 client.0 _ll_register_callbacks cb 0x7f61280097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:42:47.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:47.038+0000 7f613be00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5608b11b06c0 0x5608b11c3840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:42:47.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:47.038+0000 7f613aa00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5608b11b0fe0 0x5608b11c3d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:42:47.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:47.038+0000 7f613be00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5608b11b06c0 0x5608b11c3840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:58548/0 (socket says 172.21.15.5:58548) 2024-09-10T09:42:47.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:47.038+0000 7f613be00640 1 -- 172.21.15.5:0/788159851 learned_addr learned my addr 172.21.15.5:0/788159851 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:42:47.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:47.038+0000 7f613aa00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5608b11b0fe0 0x5608b11c3d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50214/0 (socket says 172.21.15.5:50214) 2024-09-10T09:42:47.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:47.038+0000 7f613be00640 1 --2- 172.21.15.5:0/788159851 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5608b11bb8a0 0x5608b11c0810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:42:47.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:47.039+0000 7f613aa00640 1 -- 172.21.15.5:0/788159851 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5608b11b06c0 msgr2=0x5608b11c3840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:42:47.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:47.039+0000 7f613aa00640 1 --2- 172.21.15.5:0/788159851 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5608b11b06c0 0x5608b11c3840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:47.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:47.039+0000 7f613aa00640 1 -- 172.21.15.5:0/788159851 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5608b11bb8a0 msgr2=0x5608b11c0810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:42:47.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:47.039+0000 7f613aa00640 1 --2- 172.21.15.5:0/788159851 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5608b11bb8a0 0x5608b11c0810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:47.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:47.039+0000 7f613aa00640 1 -- 172.21.15.5:0/788159851 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5608b11b3670 con 0x5608b11b0fe0 2024-09-10T09:42:47.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:47.039+0000 7f613be00640 1 --2- 172.21.15.5:0/788159851 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5608b11b06c0 0x5608b11c3840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:42:47.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:47.039+0000 7f613aa00640 1 --2- 172.21.15.5:0/788159851 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5608b11b0fe0 0x5608b11c3d80 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f612802e040 tx=0x7f61280035f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff19635abb32211c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:42:47.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:47.039+0000 7f6125600640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-10T09:42:47.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:47.040+0000 7f6125600640 1 -- 172.21.15.5:0/788159851 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6128030070 con 0x5608b11b0fe0 2024-09-10T09:42:47.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:47.040+0000 7f6125600640 1 -- 172.21.15.5:0/788159851 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f61280037d0 con 0x5608b11b0fe0 2024-09-10T09:42:47.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:47.040+0000 7f614520c740 1 -- 172.21.15.5:0/788159851 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f612801cb20 con 0x5608b11b0fe0 2024-09-10T09:42:47.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:47.040+0000 7f6125600640 1 -- 172.21.15.5:0/788159851 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f61280039f0 con 0x5608b11b0fe0 2024-09-10T09:42:47.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:47.040+0000 7f614520c740 10 client.16299 fetch_fsmap learned FSMap version 33 2024-09-10T09:42:47.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:47.040+0000 7f614520c740 1 -- 172.21.15.5:0/788159851 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5608b1236870 con 0x5608b11b0fe0 2024-09-10T09:42:47.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:47.041+0000 7f6125600640 1 -- 172.21.15.5:0/788159851 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f612801cb20 con 0x5608b11b0fe0 2024-09-10T09:42:47.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:47.041+0000 7f6125600640 1 -- 172.21.15.5:0/788159851 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 33) ==== 8374+0+0 (secure 0 0 0) 0x7f612801a070 con 0x5608b11b0fe0 2024-09-10T09:42:47.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:47.041+0000 7f614520c740 10 client.16299 fetch_fsmap finished waiting for FSMap version 33 2024-09-10T09:42:47.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:47.041+0000 7f614520c740 10 client.16299 resolve_mds: validated gid 24542 aka daemon mds.b 2024-09-10T09:42:47.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:47.041+0000 7f614520c740 20 client.16299 populate_metadata read hostname 'smithi005' 2024-09-10T09:42:47.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:47.041+0000 7f614520c740 1 --2- 172.21.15.5:0/788159851 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x5608b1236fc0 0x5608b12393e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:42:47.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:47.041+0000 7f614520c740 4 client.16299 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 142.75, "await_for": 45.0}] 2024-09-10T09:42:47.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:47.041+0000 7f614520c740 1 -- 172.21.15.5:0/788159851 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 142.75, "await_for": 45.0}) -- 0x5608b111cce0 con 0x5608b1236fc0 2024-09-10T09:42:47.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:47.041+0000 7f613be00640 1 --2- 172.21.15.5:0/788159851 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x5608b1236fc0 0x5608b12393e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:42:47.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:47.042+0000 7f613be00640 1 --2- 172.21.15.5:0/788159851 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x5608b1236fc0 0x5608b12393e0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=db61f16a88dcfb8 server_cookie=9ebc171a2233ab87 in_seq=0 out_seq=0 2024-09-10T09:42:47.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:47.042+0000 7f6125600640 10 client.16299 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T09:42:47.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:42:46 smithi159 ceph-mon[35513]: pgmap v526: 129 pgs: 129 active+clean; 90 MiB data, 878 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.9 MiB/s wr, 131 op/s 2024-09-10T09:42:47.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:47.127+0000 7f6125600640 1 -- 172.21.15.5:0/788159851 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x5608b111cce0 con 0x5608b1236fc0 2024-09-10T09:42:47.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:47.127+0000 7f6125600640 10 client.16299 handle_command_reply: tid=0 2024-09-10T09:42:47.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:47.127+0000 7f614520c740 1 client.16299 shutdown 2024-09-10T09:42:47.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:47.127+0000 7f614520c740 1 -- 172.21.15.5:0/788159851 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5608b11b0fe0 msgr2=0x5608b11c3d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:42:47.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:47.127+0000 7f614520c740 1 --2- 172.21.15.5:0/788159851 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5608b11b0fe0 0x5608b11c3d80 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f612802e040 tx=0x7f61280035f0 comp rx=0 tx=0).stop 2024-09-10T09:42:47.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:47.127+0000 7f614520c740 1 -- 172.21.15.5:0/788159851 shutdown_connections 2024-09-10T09:42:47.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:47.127+0000 7f614520c740 1 --2- 172.21.15.5:0/788159851 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x5608b1236fc0 0x5608b12393e0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:47.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:47.127+0000 7f614520c740 1 --2- 172.21.15.5:0/788159851 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5608b11bb8a0 0x5608b11c0810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:47.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:47.127+0000 7f614520c740 1 --2- 172.21.15.5:0/788159851 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5608b11b0fe0 0x5608b11c3d80 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:47.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:47.127+0000 7f614520c740 1 --2- 172.21.15.5:0/788159851 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5608b11b06c0 0x5608b11c3840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:47.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:47.127+0000 7f614520c740 1 -- 172.21.15.5:0/788159851 >> 172.21.15.5:0/788159851 conn(0x5608b11acc50 msgr2=0x5608b11abe70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:42:47.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:47.127+0000 7f6125600640 0 client.16299 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T09:42:47.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:47.128+0000 7f614520c740 1 -- 172.21.15.5:0/788159851 shutdown_connections 2024-09-10T09:42:47.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:47.128+0000 7f614520c740 1 -- 172.21.15.5:0/788159851 wait complete. 2024-09-10T09:42:47.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:47.128+0000 7f614520c740 20 client.16299 trim_cache size 0 max 16384 2024-09-10T09:42:47.130 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-10T09:42:47.131 INFO:teuthology.orchestra.run.smithi005.stdout: "epoch": 33, 2024-09-10T09:42:47.131 INFO:teuthology.orchestra.run.smithi005.stdout: "leader": 24542, 2024-09-10T09:42:47.131 INFO:teuthology.orchestra.run.smithi005.stdout: "set_version": 14, 2024-09-10T09:42:47.131 INFO:teuthology.orchestra.run.smithi005.stdout: "sets": { 2024-09-10T09:42:47.131 INFO:teuthology.orchestra.run.smithi005.stdout: "6609e6f": { 2024-09-10T09:42:47.131 INFO:teuthology.orchestra.run.smithi005.stdout: "version": 14, 2024-09-10T09:42:47.131 INFO:teuthology.orchestra.run.smithi005.stdout: "age_ref": 0.0, 2024-09-10T09:42:47.131 INFO:teuthology.orchestra.run.smithi005.stdout: "state": { 2024-09-10T09:42:47.131 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "QUIESCED", 2024-09-10T09:42:47.131 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0 2024-09-10T09:42:47.131 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T09:42:47.131 INFO:teuthology.orchestra.run.smithi005.stdout: "timeout": 90.0, 2024-09-10T09:42:47.132 INFO:teuthology.orchestra.run.smithi005.stdout: "expiration": 142.8, 2024-09-10T09:42:47.132 INFO:teuthology.orchestra.run.smithi005.stdout: "members": { 2024-09-10T09:42:47.132 INFO:teuthology.orchestra.run.smithi005.stdout: "file:/": { 2024-09-10T09:42:47.132 INFO:teuthology.orchestra.run.smithi005.stdout: "excluded": false, 2024-09-10T09:42:47.132 INFO:teuthology.orchestra.run.smithi005.stdout: "state": { 2024-09-10T09:42:47.132 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "QUIESCED", 2024-09-10T09:42:47.132 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0 2024-09-10T09:42:47.132 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T09:42:47.132 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T09:42:47.132 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T09:42:47.132 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T09:42:47.132 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T09:42:47.132 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-10T09:42:47.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:47.128+0000 7f6141000640 1 -- 172.21.15.5:0/3459345700 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f61200a2c00 msgr2=0x7f61200a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:42:47.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:47.128+0000 7f6141000640 1 --2- 172.21.15.5:0/3459345700 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f61200a2c00 0x7f61200a50c0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f612c0ed060 tx=0x7f612c0f2f90 comp rx=0 tx=0).stop 2024-09-10T09:42:47.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:47.128+0000 7f6141000640 1 -- 172.21.15.5:0/3459345700 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f613c074510 msgr2=0x7f613c19a4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:42:47.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:47.128+0000 7f6141000640 1 --2- 172.21.15.5:0/3459345700 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f613c074510 0x7f613c19a4d0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f612c004770 tx=0x7f612c003c10 comp rx=0 tx=0).stop 2024-09-10T09:42:47.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:47.129+0000 7f6141000640 1 -- 172.21.15.5:0/3459345700 shutdown_connections 2024-09-10T09:42:47.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:47.129+0000 7f6141000640 1 --2- 172.21.15.5:0/3459345700 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f61200a2c00 0x7f61200a50c0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:47.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:47.129+0000 7f6141000640 1 --2- 172.21.15.5:0/3459345700 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f613c10e280 0x7f613c19eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:47.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:47.129+0000 7f6141000640 1 --2- 172.21.15.5:0/3459345700 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f613c074e30 0x7f613c19aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:47.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:47.129+0000 7f6141000640 1 --2- 172.21.15.5:0/3459345700 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f613c074510 0x7f613c19a4d0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:42:47.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:47.129+0000 7f6141000640 1 -- 172.21.15.5:0/3459345700 >> 172.21.15.5:0/3459345700 conn(0x7f613c0fd740 msgr2=0x7f613c0feed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:42:47.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:47.129+0000 7f6141000640 1 -- 172.21.15.5:0/3459345700 shutdown_connections 2024-09-10T09:42:47.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:42:47.129+0000 7f6141000640 1 -- 172.21.15.5:0/3459345700 wait complete. 2024-09-10T09:42:47.135 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:42:46 smithi102 ceph-mon[35442]: pgmap v526: 129 pgs: 129 active+clean; 90 MiB data, 878 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.9 MiB/s wr, 131 op/s 2024-09-10T09:42:48.027 INFO:tasks.workunit.client.0.smithi005.stdout: 1 204065 2.16 MB/sec execute 29 sec latency 898.664 ms 2024-09-10T09:42:48.059 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '6609e6f', quiesce took 7.5 seconds. Will release after: 15.25 2024-09-10T09:42:49.016 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:42:48 smithi005 ceph-mon[30350]: pgmap v527: 129 pgs: 129 active+clean; 90 MiB data, 885 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.8 MiB/s wr, 132 op/s 2024-09-10T09:42:49.027 INFO:tasks.workunit.client.0.smithi005.stdout: 1 204065 2.09 MB/sec execute 30 sec latency 1898.779 ms 2024-09-10T09:42:49.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:42:48 smithi159 ceph-mon[35513]: pgmap v527: 129 pgs: 129 active+clean; 90 MiB data, 885 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.8 MiB/s wr, 132 op/s 2024-09-10T09:42:49.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:42:48 smithi102 ceph-mon[35442]: pgmap v527: 129 pgs: 129 active+clean; 90 MiB data, 885 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.8 MiB/s wr, 132 op/s 2024-09-10T09:42:50.027 INFO:tasks.workunit.client.0.smithi005.stdout: 1 204065 2.02 MB/sec execute 31 sec latency 2898.860 ms 2024-09-10T09:42:50.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:42:50 smithi005 ceph-mon[30350]: pgmap v528: 129 pgs: 129 active+clean; 93 MiB data, 914 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.9 MiB/s wr, 142 op/s 2024-09-10T09:42:51.027 INFO:tasks.workunit.client.0.smithi005.stdout: 1 204065 1.96 MB/sec execute 32 sec latency 3898.952 ms 2024-09-10T09:42:51.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:42:50 smithi159 ceph-mon[35513]: pgmap v528: 129 pgs: 129 active+clean; 93 MiB data, 914 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.9 MiB/s wr, 142 op/s 2024-09-10T09:42:51.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:42:50 smithi102 ceph-mon[35442]: pgmap v528: 129 pgs: 129 active+clean; 93 MiB data, 914 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.9 MiB/s wr, 142 op/s 2024-09-10T09:42:52.027 INFO:tasks.workunit.client.0.smithi005.stdout: 1 204065 1.90 MB/sec execute 33 sec latency 4899.072 ms 2024-09-10T09:42:53.027 INFO:tasks.workunit.client.0.smithi005.stdout: 1 204065 1.85 MB/sec execute 34 sec latency 5899.198 ms 2024-09-10T09:42:53.057 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:42:52 smithi005 ceph-mon[30350]: pgmap v529: 129 pgs: 129 active+clean; 97 MiB data, 904 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 1.9 MiB/s wr, 165 op/s 2024-09-10T09:42:53.057 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:42:52 smithi005 ceph-mon[30350]: scrub summary: idle 2024-09-10T09:42:53.057 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:42:52 smithi005 ceph-mon[30350]: scrub summary: idle 2024-09-10T09:42:53.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:42:52 smithi159 ceph-mon[35513]: pgmap v529: 129 pgs: 129 active+clean; 97 MiB data, 904 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 1.9 MiB/s wr, 165 op/s 2024-09-10T09:42:53.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:42:52 smithi159 ceph-mon[35513]: scrub summary: idle 2024-09-10T09:42:53.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:42:52 smithi159 ceph-mon[35513]: scrub summary: idle 2024-09-10T09:42:53.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:42:52 smithi102 ceph-mon[35442]: pgmap v529: 129 pgs: 129 active+clean; 97 MiB data, 904 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 1.9 MiB/s wr, 165 op/s 2024-09-10T09:42:53.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:42:52 smithi102 ceph-mon[35442]: scrub summary: idle 2024-09-10T09:42:53.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:42:52 smithi102 ceph-mon[35442]: scrub summary: idle 2024-09-10T09:42:54.027 INFO:tasks.workunit.client.0.smithi005.stdout: 1 204065 1.79 MB/sec execute 35 sec latency 6899.339 ms 2024-09-10T09:42:55.027 INFO:tasks.workunit.client.0.smithi005.stdout: 1 204065 1.74 MB/sec execute 36 sec latency 7899.445 ms 2024-09-10T09:42:55.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:42:54 smithi005 ceph-mon[30350]: pgmap v530: 129 pgs: 129 active+clean; 96 MiB data, 908 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.6 MiB/s wr, 151 op/s 2024-09-10T09:42:55.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:42:54 smithi159 ceph-mon[35513]: pgmap v530: 129 pgs: 129 active+clean; 96 MiB data, 908 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.6 MiB/s wr, 151 op/s 2024-09-10T09:42:55.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:42:54 smithi102 ceph-mon[35442]: pgmap v530: 129 pgs: 129 active+clean; 96 MiB data, 908 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.6 MiB/s wr, 151 op/s 2024-09-10T09:42:56.028 INFO:tasks.workunit.client.0.smithi005.stdout: 1 204065 1.70 MB/sec execute 37 sec latency 8899.568 ms 2024-09-10T09:42:57.028 INFO:tasks.workunit.client.0.smithi005.stdout: 1 204065 1.65 MB/sec execute 38 sec latency 9899.710 ms 2024-09-10T09:42:57.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:42:56 smithi005 ceph-mon[30350]: pgmap v531: 129 pgs: 129 active+clean; 94 MiB data, 908 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.4 MiB/s wr, 146 op/s 2024-09-10T09:42:57.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:42:56 smithi159 ceph-mon[35513]: pgmap v531: 129 pgs: 129 active+clean; 94 MiB data, 908 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.4 MiB/s wr, 146 op/s 2024-09-10T09:42:57.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:42:56 smithi102 ceph-mon[35442]: pgmap v531: 129 pgs: 129 active+clean; 94 MiB data, 908 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.4 MiB/s wr, 146 op/s 2024-09-10T09:42:58.028 INFO:tasks.workunit.client.0.smithi005.stdout: 1 204065 1.61 MB/sec execute 39 sec latency 10899.869 ms 2024-09-10T09:42:59.029 INFO:tasks.workunit.client.0.smithi005.stdout: 1 204065 1.57 MB/sec execute 40 sec latency 11900.033 ms 2024-09-10T09:42:59.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:42:58 smithi005 ceph-mon[30350]: pgmap v532: 129 pgs: 129 active+clean; 92 MiB data, 907 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 877 KiB/s wr, 82 op/s 2024-09-10T09:42:59.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:42:58 smithi005 ceph-mon[30350]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T09:42:59.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:42:58 smithi159 ceph-mon[35513]: pgmap v532: 129 pgs: 129 active+clean; 92 MiB data, 907 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 877 KiB/s wr, 82 op/s 2024-09-10T09:42:59.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:42:58 smithi159 ceph-mon[35513]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T09:42:59.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:42:58 smithi102 ceph-mon[35442]: pgmap v532: 129 pgs: 129 active+clean; 92 MiB data, 907 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 877 KiB/s wr, 82 op/s 2024-09-10T09:42:59.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:42:58 smithi102 ceph-mon[35442]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T09:43:00.028 INFO:tasks.workunit.client.0.smithi005.stdout: 1 204065 1.53 MB/sec execute 41 sec latency 12900.156 ms 2024-09-10T09:43:01.028 INFO:tasks.workunit.client.0.smithi005.stdout: 1 204065 1.49 MB/sec execute 42 sec latency 13900.348 ms 2024-09-10T09:43:01.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:43:00 smithi005 ceph-mon[30350]: pgmap v533: 129 pgs: 129 active+clean; 92 MiB data, 907 MiB used, 1.0 TiB / 1.0 TiB avail; 958 KiB/s rd, 727 KiB/s wr, 70 op/s 2024-09-10T09:43:01.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:43:00 smithi159 ceph-mon[35513]: pgmap v533: 129 pgs: 129 active+clean; 92 MiB data, 907 MiB used, 1.0 TiB / 1.0 TiB avail; 958 KiB/s rd, 727 KiB/s wr, 70 op/s 2024-09-10T09:43:01.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:43:00 smithi102 ceph-mon[35442]: pgmap v533: 129 pgs: 129 active+clean; 92 MiB data, 907 MiB used, 1.0 TiB / 1.0 TiB avail; 958 KiB/s rd, 727 KiB/s wr, 70 op/s 2024-09-10T09:43:02.029 INFO:tasks.workunit.client.0.smithi005.stdout: 1 204065 1.46 MB/sec execute 43 sec latency 14900.421 ms 2024-09-10T09:43:03.029 INFO:tasks.workunit.client.0.smithi005.stdout: 1 204065 1.43 MB/sec execute 44 sec latency 15900.540 ms 2024-09-10T09:43:03.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:43:02 smithi005 ceph-mon[30350]: pgmap v534: 129 pgs: 129 active+clean; 92 MiB data, 907 MiB used, 1.0 TiB / 1.0 TiB avail; 260 KiB/s rd, 470 KiB/s wr, 55 op/s 2024-09-10T09:43:03.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:43:02 smithi159 ceph-mon[35513]: pgmap v534: 129 pgs: 129 active+clean; 92 MiB data, 907 MiB used, 1.0 TiB / 1.0 TiB avail; 260 KiB/s rd, 470 KiB/s wr, 55 op/s 2024-09-10T09:43:03.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:43:02 smithi102 ceph-mon[35442]: pgmap v534: 129 pgs: 129 active+clean; 92 MiB data, 907 MiB used, 1.0 TiB / 1.0 TiB avail; 260 KiB/s rd, 470 KiB/s wr, 55 op/s 2024-09-10T09:43:03.310 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T09:43:03.702 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:43:04.029 INFO:tasks.workunit.client.0.smithi005.stdout: 1 204065 1.39 MB/sec execute 45 sec latency 16900.600 ms 2024-09-10T09:43:04.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:04.288+0000 7fa71b000640 1 -- 172.21.15.5:0/2204806812 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa714109120 msgr2=0x7fa714110ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:43:04.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:04.288+0000 7fa71b000640 1 --2- 172.21.15.5:0/2204806812 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa714109120 0x7fa714110ca0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fa70c009ca0 tx=0x7fa70c02a8c0 comp rx=0 tx=0).stop 2024-09-10T09:43:04.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:04.288+0000 7fa71b000640 1 -- 172.21.15.5:0/2204806812 shutdown_connections 2024-09-10T09:43:04.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:04.288+0000 7fa71b000640 1 --2- 172.21.15.5:0/2204806812 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa714109120 0x7fa714110ca0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:04.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:04.288+0000 7fa71b000640 1 --2- 172.21.15.5:0/2204806812 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa7141049a0 0x7fa7141089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:04.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:04.288+0000 7fa71b000640 1 --2- 172.21.15.5:0/2204806812 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa714103ff0 0x7fa7141043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:04.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:04.288+0000 7fa71b000640 1 -- 172.21.15.5:0/2204806812 >> 172.21.15.5:0/2204806812 conn(0x7fa7140ffa90 msgr2=0x7fa714101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:43:04.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:04.288+0000 7fa71b000640 1 -- 172.21.15.5:0/2204806812 shutdown_connections 2024-09-10T09:43:04.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:04.288+0000 7fa71b000640 1 -- 172.21.15.5:0/2204806812 wait complete. 2024-09-10T09:43:04.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:04.289+0000 7fa71b000640 1 Processor -- start 2024-09-10T09:43:04.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:04.289+0000 7fa71b000640 1 -- start start 2024-09-10T09:43:04.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:04.290+0000 7fa71b000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa714103ff0 0x7fa71419a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:43:04.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:04.290+0000 7fa71b000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa7141049a0 0x7fa71419aae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:43:04.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:04.290+0000 7fa71b000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa714109120 0x7fa71419ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:43:04.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:04.290+0000 7fa71b000640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa714114180 con 0x7fa7141049a0 2024-09-10T09:43:04.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:04.290+0000 7fa71b000640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fa714114000 con 0x7fa714109120 2024-09-10T09:43:04.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:04.290+0000 7fa71b000640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fa714114300 con 0x7fa714103ff0 2024-09-10T09:43:04.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:04.290+0000 7fa713e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa714103ff0 0x7fa71419a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:43:04.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:04.290+0000 7fa70be00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa7141049a0 0x7fa71419aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:43:04.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:04.290+0000 7fa713e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa714103ff0 0x7fa71419a5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:59250/0 (socket says 172.21.15.5:59250) 2024-09-10T09:43:04.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:04.290+0000 7fa713e00640 1 -- 172.21.15.5:0/2678782940 learned_addr learned my addr 172.21.15.5:0/2678782940 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:43:04.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:04.290+0000 7fa719000640 1 --2- 172.21.15.5:0/2678782940 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa714109120 0x7fa71419ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:43:04.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:04.291+0000 7fa70be00640 1 -- 172.21.15.5:0/2678782940 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa714103ff0 msgr2=0x7fa71419a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:43:04.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:04.291+0000 7fa70be00640 1 --2- 172.21.15.5:0/2678782940 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa714103ff0 0x7fa71419a5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:04.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:04.291+0000 7fa70be00640 1 -- 172.21.15.5:0/2678782940 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa714109120 msgr2=0x7fa71419ee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:43:04.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:04.291+0000 7fa70be00640 1 --2- 172.21.15.5:0/2678782940 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa714109120 0x7fa71419ee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:04.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:04.291+0000 7fa70be00640 1 -- 172.21.15.5:0/2678782940 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa71419f550 con 0x7fa7141049a0 2024-09-10T09:43:04.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:04.291+0000 7fa713e00640 1 --2- 172.21.15.5:0/2678782940 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa714103ff0 0x7fa71419a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:43:04.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:04.291+0000 7fa70be00640 1 --2- 172.21.15.5:0/2678782940 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa7141049a0 0x7fa71419aae0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fa6fc00d6d0 tx=0x7fa6fc00db90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:43:04.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:04.291+0000 7fa711600640 1 -- 172.21.15.5:0/2678782940 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6fc017070 con 0x7fa7141049a0 2024-09-10T09:43:04.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:04.291+0000 7fa711600640 1 -- 172.21.15.5:0/2678782940 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa6fc009820 con 0x7fa7141049a0 2024-09-10T09:43:04.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:04.291+0000 7fa71b000640 1 -- 172.21.15.5:0/2678782940 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa71419f840 con 0x7fa7141049a0 2024-09-10T09:43:04.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:04.291+0000 7fa711600640 1 -- 172.21.15.5:0/2678782940 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6fc004d00 con 0x7fa7141049a0 2024-09-10T09:43:04.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:04.292+0000 7fa71b000640 1 -- 172.21.15.5:0/2678782940 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7141a7120 con 0x7fa7141049a0 2024-09-10T09:43:04.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:04.293+0000 7fa71b000640 1 -- 172.21.15.5:0/2678782940 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa7141043d0 con 0x7fa7141049a0 2024-09-10T09:43:04.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:04.295+0000 7fa711600640 1 -- 172.21.15.5:0/2678782940 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7fa6fc0099d0 con 0x7fa7141049a0 2024-09-10T09:43:04.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:04.296+0000 7fa711600640 1 --2- 172.21.15.5:0/2678782940 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fa6f80a2b80 0x7fa6f80a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:43:04.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:04.296+0000 7fa711600640 1 -- 172.21.15.5:0/2678782940 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fa6fc0ceab0 con 0x7fa7141049a0 2024-09-10T09:43:04.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:04.296+0000 7fa713e00640 1 --2- 172.21.15.5:0/2678782940 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fa6f80a2b80 0x7fa6f80a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:43:04.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:04.297+0000 7fa713e00640 1 --2- 172.21.15.5:0/2678782940 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fa6f80a2b80 0x7fa6f80a5040 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fa704005fd0 tx=0x7fa70400b4a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:43:04.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:04.300+0000 7fa711600640 1 -- 172.21.15.5:0/2678782940 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa6fc09afd0 con 0x7fa7141049a0 2024-09-10T09:43:04.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:04.522+0000 7fa71b000640 1 -- 172.21.15.5:0/2678782940 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa7141a7450 con 0x7fa7141049a0 2024-09-10T09:43:04.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:04.523+0000 7fa711600640 1 -- 172.21.15.5:0/2678782940 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+11969 (secure 0 0 0) 0x7fa6fc09f2d0 con 0x7fa7141049a0 2024-09-10T09:43:04.526 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 34 2024-09-10T09:43:04.526 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-10T09:43:04.526 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":34,"btime":"2024-09-10T09:42:57:737230+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14454,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/2970594311","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2970594311},{"type":"v1","addr":"172.21.15.5:6835","nonce":2970594311}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14514,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6839/3616502174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":3616502174},{"type":"v1","addr":"172.21.15.5:6839","nonce":3616502174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":34,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T09:35:29.626287+0000","modified":"2024-09-10T09:42:56.961831+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24542,"mds_1":24535,"mds_2":24511,"mds_3":24506,"mds_4":24443},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14520":{"gid":14520,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/2073599842","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":2073599842},{"type":"v1","addr":"172.21.15.5:6841","nonce":2073599842}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"k","rank":4,"incarnation":26,"state":"up:active","state_seq":24,"addr":"172.21.15.102:6833/2457007427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":2457007427},{"type":"v1","addr":"172.21.15.102:6833","nonce":2457007427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/356445159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":356445159},{"type":"v1","addr":"172.21.15.159:6833","nonce":356445159}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6837/2894314507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":2894314507},{"type":"v1","addr":"172.21.15.5:6837","nonce":2894314507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.102:6835/1647549111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":1647549111},{"type":"v1","addr":"172.21.15.102:6835","nonce":1647549111}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6835/952657242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":952657242},{"type":"v1","addr":"172.21.15.159:6835","nonce":952657242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":19,"addr":"172.21.15.102:6837/2689776764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":2689776764},{"type":"v1","addr":"172.21.15.102:6837","nonce":2689776764}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.159:6837/620627639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":620627639},{"type":"v1","addr":"172.21.15.159:6837","nonce":620627639}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24535":{"gid":24535,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":18,"addr":"172.21.15.159:6839/2073739599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":2073739599},{"type":"v1","addr":"172.21.15.159:6839","nonce":2073739599}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24542":{"gid":24542,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.102:6839/335409749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6838","nonce":335409749},{"type":"v1","addr":"172.21.15.102:6839","nonce":335409749}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24542,"qdb_cluster":[24542,24535,24511,24506,24443]},"id":1}]} 2024-09-10T09:43:04.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:04.527+0000 7fa71b000640 1 -- 172.21.15.5:0/2678782940 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fa6f80a2b80 msgr2=0x7fa6f80a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:43:04.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:04.527+0000 7fa71b000640 1 --2- 172.21.15.5:0/2678782940 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fa6f80a2b80 0x7fa6f80a5040 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fa704005fd0 tx=0x7fa70400b4a0 comp rx=0 tx=0).stop 2024-09-10T09:43:04.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:04.527+0000 7fa71b000640 1 -- 172.21.15.5:0/2678782940 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa7141049a0 msgr2=0x7fa71419aae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:43:04.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:04.527+0000 7fa71b000640 1 --2- 172.21.15.5:0/2678782940 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa7141049a0 0x7fa71419aae0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fa6fc00d6d0 tx=0x7fa6fc00db90 comp rx=0 tx=0).stop 2024-09-10T09:43:04.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:04.527+0000 7fa71b000640 1 -- 172.21.15.5:0/2678782940 shutdown_connections 2024-09-10T09:43:04.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:04.527+0000 7fa71b000640 1 --2- 172.21.15.5:0/2678782940 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fa6f80a2b80 0x7fa6f80a5040 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:04.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:04.527+0000 7fa71b000640 1 --2- 172.21.15.5:0/2678782940 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa714109120 0x7fa71419ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:04.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:04.527+0000 7fa71b000640 1 --2- 172.21.15.5:0/2678782940 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa7141049a0 0x7fa71419aae0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:04.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:04.527+0000 7fa71b000640 1 --2- 172.21.15.5:0/2678782940 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa714103ff0 0x7fa71419a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:04.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:04.527+0000 7fa71b000640 1 -- 172.21.15.5:0/2678782940 >> 172.21.15.5:0/2678782940 conn(0x7fa7140ffa90 msgr2=0x7fa7141004b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:43:04.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:04.527+0000 7fa71b000640 1 -- 172.21.15.5:0/2678782940 shutdown_connections 2024-09-10T09:43:04.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:04.528+0000 7fa71b000640 1 -- 172.21.15.5:0/2678782940 wait complete. 2024-09-10T09:43:04.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:43:04 smithi005 ceph-mon[30350]: pgmap v535: 129 pgs: 129 active+clean; 92 MiB data, 907 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 38 KiB/s wr, 18 op/s 2024-09-10T09:43:04.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:43:04 smithi005 ceph-mon[30350]: from='client.? 172.21.15.5:0/2678782940' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:43:05.029 INFO:tasks.workunit.client.0.smithi005.stdout: 1 204065 1.36 MB/sec execute 46 sec latency 17900.718 ms 2024-09-10T09:43:05.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:43:04 smithi159 ceph-mon[35513]: pgmap v535: 129 pgs: 129 active+clean; 92 MiB data, 907 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 38 KiB/s wr, 18 op/s 2024-09-10T09:43:05.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:43:04 smithi159 ceph-mon[35513]: from='client.? 172.21.15.5:0/2678782940' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:43:05.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:43:04 smithi102 ceph-mon[35442]: pgmap v535: 129 pgs: 129 active+clean; 92 MiB data, 907 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 38 KiB/s wr, 18 op/s 2024-09-10T09:43:05.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:43:04 smithi102 ceph-mon[35442]: from='client.? 172.21.15.5:0/2678782940' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:43:05.473 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24542 quiesce db --set-id 6609e6f --release --await' 2024-09-10T09:43:05.473 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24542 quiesce db --set-id 6609e6f --release --await 2024-09-10T09:43:05.855 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:43:06.029 INFO:tasks.workunit.client.0.smithi005.stdout: 1 204065 1.34 MB/sec execute 47 sec latency 18900.838 ms 2024-09-10T09:43:06.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:06.424+0000 7f6574a00640 1 -- 172.21.15.5:0/1028573422 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6570102790 msgr2=0x7f65701067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:43:06.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:06.424+0000 7f6574a00640 1 --2- 172.21.15.5:0/1028573422 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6570102790 0x7f65701067e0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f655c009900 tx=0x7f655c02a410 comp rx=0 tx=0).stop 2024-09-10T09:43:06.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:06.425+0000 7f6574a00640 1 -- 172.21.15.5:0/1028573422 shutdown_connections 2024-09-10T09:43:06.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:06.425+0000 7f6574a00640 1 --2- 172.21.15.5:0/1028573422 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6570106f10 0x7f657010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:06.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:06.425+0000 7f6574a00640 1 --2- 172.21.15.5:0/1028573422 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6570102790 0x7f65701067e0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:06.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:06.425+0000 7f6574a00640 1 --2- 172.21.15.5:0/1028573422 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6570101de0 0x7f65701021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:06.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:06.425+0000 7f6574a00640 1 -- 172.21.15.5:0/1028573422 >> 172.21.15.5:0/1028573422 conn(0x7f65700fd8c0 msgr2=0x7f65700ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:43:06.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:06.425+0000 7f6574a00640 1 -- 172.21.15.5:0/1028573422 shutdown_connections 2024-09-10T09:43:06.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:06.425+0000 7f6574a00640 1 -- 172.21.15.5:0/1028573422 wait complete. 2024-09-10T09:43:06.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:06.426+0000 7f6574a00640 1 Processor -- start 2024-09-10T09:43:06.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:06.426+0000 7f6574a00640 1 -- start start 2024-09-10T09:43:06.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:06.427+0000 7f6574a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6570101de0 0x7f65700744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:43:06.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:06.427+0000 7f6574a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6570102790 0x7f6570074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:43:06.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:06.427+0000 7f6574a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6570106f10 0x7f6570074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:43:06.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:06.427+0000 7f6574a00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f6570111fc0 con 0x7f6570102790 2024-09-10T09:43:06.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:06.427+0000 7f6574a00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f6570111e40 con 0x7f6570106f10 2024-09-10T09:43:06.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:06.427+0000 7f6574a00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f6570112140 con 0x7f6570101de0 2024-09-10T09:43:06.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:06.427+0000 7f656d400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6570102790 0x7f6570074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:43:06.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:06.427+0000 7f656d400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6570102790 0x7f6570074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33210/0 (socket says 172.21.15.5:33210) 2024-09-10T09:43:06.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:06.427+0000 7f656d400640 1 -- 172.21.15.5:0/2953873758 learned_addr learned my addr 172.21.15.5:0/2953873758 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:43:06.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:06.427+0000 7f656de00640 1 --2- 172.21.15.5:0/2953873758 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6570101de0 0x7f65700744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:43:06.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:06.427+0000 7f656e800640 1 --2- 172.21.15.5:0/2953873758 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6570106f10 0x7f6570074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:43:06.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:06.428+0000 7f656d400640 1 -- 172.21.15.5:0/2953873758 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6570101de0 msgr2=0x7f65700744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:43:06.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:06.428+0000 7f656d400640 1 --2- 172.21.15.5:0/2953873758 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6570101de0 0x7f65700744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:06.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:06.428+0000 7f656d400640 1 -- 172.21.15.5:0/2953873758 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6570106f10 msgr2=0x7f6570074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:43:06.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:06.428+0000 7f656d400640 1 --2- 172.21.15.5:0/2953873758 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6570106f10 0x7f6570074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:06.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:06.428+0000 7f656d400640 1 -- 172.21.15.5:0/2953873758 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65701a6960 con 0x7f6570102790 2024-09-10T09:43:06.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:06.428+0000 7f656de00640 1 --2- 172.21.15.5:0/2953873758 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6570101de0 0x7f65700744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:43:06.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:06.428+0000 7f656d400640 1 --2- 172.21.15.5:0/2953873758 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6570102790 0x7f6570074a30 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f655c004770 tx=0x7f655c003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:43:06.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:06.428+0000 7f6566a00640 1 -- 172.21.15.5:0/2953873758 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f655c03f070 con 0x7f6570102790 2024-09-10T09:43:06.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:06.428+0000 7f6566a00640 1 -- 172.21.15.5:0/2953873758 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f655c031ca0 con 0x7f6570102790 2024-09-10T09:43:06.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:06.428+0000 7f6566a00640 1 -- 172.21.15.5:0/2953873758 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f655c035030 con 0x7f6570102790 2024-09-10T09:43:06.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:06.428+0000 7f6574a00640 1 -- 172.21.15.5:0/2953873758 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65701a6bf0 con 0x7f6570102790 2024-09-10T09:43:06.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:06.429+0000 7f6574a00640 1 -- 172.21.15.5:0/2953873758 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65701a7120 con 0x7f6570102790 2024-09-10T09:43:06.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:06.432+0000 7f65774d3740 1 Processor -- start 2024-09-10T09:43:06.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:06.432+0000 7f6566a00640 1 -- 172.21.15.5:0/2953873758 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f655c042070 con 0x7f6570102790 2024-09-10T09:43:06.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:06.432+0000 7f65774d3740 1 -- start start 2024-09-10T09:43:06.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:06.432+0000 7f65774d3740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557b4235c310 0x557b4235c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:43:06.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:06.432+0000 7f65774d3740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557b4235cc30 0x557b42366fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:43:06.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:06.432+0000 7f65774d3740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557b423674f0 0x557b423698e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:43:06.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:06.432+0000 7f65774d3740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x557b422c2650 con 0x557b423674f0 2024-09-10T09:43:06.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:06.432+0000 7f65774d3740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x557b42269f10 con 0x557b4235c310 2024-09-10T09:43:06.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:06.432+0000 7f65774d3740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x557b422be610 con 0x557b4235cc30 2024-09-10T09:43:06.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:06.433+0000 7f656d400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557b423674f0 0x557b423698e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:43:06.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:06.433+0000 7f656e800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557b4235c310 0x557b4235c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:43:06.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:06.433+0000 7f656de00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557b4235cc30 0x557b42366fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:43:06.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:06.433+0000 7f656de00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557b4235cc30 0x557b42366fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:59266/0 (socket says 172.21.15.5:59266) 2024-09-10T09:43:06.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:06.433+0000 7f656e800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557b4235c310 0x557b4235c6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:50592/0 (socket says 172.21.15.5:50592) 2024-09-10T09:43:06.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:06.433+0000 7f656de00640 1 -- 172.21.15.5:0/982253626 learned_addr learned my addr 172.21.15.5:0/982253626 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:43:06.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:06.433+0000 7f656de00640 1 -- 172.21.15.5:0/982253626 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557b4235c310 msgr2=0x557b4235c6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:43:06.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:06.433+0000 7f656de00640 1 --2- 172.21.15.5:0/982253626 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557b4235c310 0x557b4235c6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:06.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:06.433+0000 7f656de00640 1 -- 172.21.15.5:0/982253626 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557b423674f0 msgr2=0x557b423698e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:43:06.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:06.433+0000 7f656de00640 1 --2- 172.21.15.5:0/982253626 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557b423674f0 0x557b423698e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:06.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:06.433+0000 7f656de00640 1 -- 172.21.15.5:0/982253626 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557b41fd1580 con 0x557b4235cc30 2024-09-10T09:43:06.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:06.434+0000 7f6566a00640 1 --2- 172.21.15.5:0/2953873758 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f65540a2c00 0x7f65540a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:43:06.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:06.434+0000 7f656de00640 1 --2- 172.21.15.5:0/982253626 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557b4235cc30 0x557b42366fb0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f65600115c0 tx=0x7f6560011a80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1b5356108b21aff3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:43:06.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:06.434+0000 7f6566a00640 1 -- 172.21.15.5:0/2953873758 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f655c0efa90 con 0x7f6570102790 2024-09-10T09:43:06.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:06.434+0000 7f656e800640 1 --2- 172.21.15.5:0/2953873758 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f65540a2c00 0x7f65540a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:43:06.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:06.434+0000 7f655b000640 1 -- 172.21.15.5:0/982253626 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f656000f640 con 0x557b4235cc30 2024-09-10T09:43:06.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:06.434+0000 7f655b000640 1 -- 172.21.15.5:0/982253626 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f656000fcc0 con 0x557b4235cc30 2024-09-10T09:43:06.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:06.434+0000 7f655b000640 1 -- 172.21.15.5:0/982253626 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6560019e10 con 0x557b4235cc30 2024-09-10T09:43:06.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:06.435+0000 7f65774d3740 1 -- 172.21.15.5:0/982253626 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557b4235cc30 msgr2=0x557b42366fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:43:06.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:06.435+0000 7f65774d3740 1 --2- 172.21.15.5:0/982253626 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557b4235cc30 0x557b42366fb0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f65600115c0 tx=0x7f6560011a80 comp rx=0 tx=0).stop 2024-09-10T09:43:06.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:06.435+0000 7f656e800640 1 --2- 172.21.15.5:0/2953873758 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f65540a2c00 0x7f65540a50c0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f6568005fd0 tx=0x7f65680096a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:43:06.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:06.435+0000 7f65774d3740 1 -- 172.21.15.5:0/982253626 shutdown_connections 2024-09-10T09:43:06.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:06.435+0000 7f65774d3740 1 --2- 172.21.15.5:0/982253626 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557b423674f0 0x557b423698e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:06.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:06.435+0000 7f65774d3740 1 --2- 172.21.15.5:0/982253626 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557b4235cc30 0x557b42366fb0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:06.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:06.435+0000 7f65774d3740 1 --2- 172.21.15.5:0/982253626 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557b4235c310 0x557b4235c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:06.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:06.435+0000 7f65774d3740 1 -- 172.21.15.5:0/982253626 >> 172.21.15.5:0/982253626 conn(0x557b423588a0 msgr2=0x557b42358c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:43:06.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:06.436+0000 7f65774d3740 1 -- 172.21.15.5:0/982253626 shutdown_connections 2024-09-10T09:43:06.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:06.436+0000 7f65774d3740 1 -- 172.21.15.5:0/982253626 wait complete. 2024-09-10T09:43:06.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:06.436+0000 7f65774d3740 1 -- start start 2024-09-10T09:43:06.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:06.436+0000 7f65774d3740 1 Processor -- start 2024-09-10T09:43:06.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:06.437+0000 7f65774d3740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557b4235c310 0x557b423643b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:43:06.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:06.437+0000 7f65774d3740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557b4235cc30 0x557b423648f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:43:06.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:06.437+0000 7f65774d3740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557b423674f0 0x557b42364e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:43:06.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:06.437+0000 7f65774d3740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x557b4236a0c0 con 0x557b423674f0 2024-09-10T09:43:06.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:06.437+0000 7f65774d3740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x557b4227d350 con 0x557b4235cc30 2024-09-10T09:43:06.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:06.437+0000 7f65774d3740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x557b4236a240 con 0x557b4235c310 2024-09-10T09:43:06.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:06.437+0000 7f65774d3740 10 client.0 _ll_register_callbacks cb 0x557b422a0290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:43:06.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:06.437+0000 7f656d400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557b4235cc30 0x557b423648f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:43:06.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:06.437+0000 7f656de00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557b4235c310 0x557b423643b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:43:06.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:06.437+0000 7f656e800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557b423674f0 0x557b42364e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:43:06.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:06.437+0000 7f656d400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557b4235cc30 0x557b423648f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:50604/0 (socket says 172.21.15.5:50604) 2024-09-10T09:43:06.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:06.437+0000 7f656de00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557b4235c310 0x557b423643b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:59270/0 (socket says 172.21.15.5:59270) 2024-09-10T09:43:06.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:06.437+0000 7f656d400640 1 -- 172.21.15.5:0/562066080 learned_addr learned my addr 172.21.15.5:0/562066080 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:43:06.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:06.438+0000 7f656d400640 1 -- 172.21.15.5:0/562066080 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557b4235c310 msgr2=0x557b423643b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:43:06.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:06.438+0000 7f656d400640 1 --2- 172.21.15.5:0/562066080 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557b4235c310 0x557b423643b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:06.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:06.438+0000 7f656d400640 1 -- 172.21.15.5:0/562066080 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557b423674f0 msgr2=0x557b42364e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:43:06.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:06.438+0000 7f656d400640 1 --2- 172.21.15.5:0/562066080 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557b423674f0 0x557b42364e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:06.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:06.438+0000 7f656d400640 1 -- 172.21.15.5:0/562066080 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557b4209b960 con 0x557b4235cc30 2024-09-10T09:43:06.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:06.438+0000 7f656d400640 1 --2- 172.21.15.5:0/562066080 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557b4235cc30 0x557b423648f0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f655c0bbc50 tx=0x7f655c031f20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=756923e6bdc8abf server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:43:06.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:06.438+0000 7f654f400640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-09-10T09:43:06.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:06.438+0000 7f654f400640 1 -- 172.21.15.5:0/562066080 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f655c049070 con 0x557b4235cc30 2024-09-10T09:43:06.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:06.438+0000 7f654f400640 1 -- 172.21.15.5:0/562066080 <== mon.2 v2:172.21.15.102:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f655c0ed250 con 0x557b4235cc30 2024-09-10T09:43:06.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:06.438+0000 7f65774d3740 1 -- 172.21.15.5:0/562066080 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f656000fcc0 con 0x557b4235cc30 2024-09-10T09:43:06.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:06.438+0000 7f654f400640 1 -- 172.21.15.5:0/562066080 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f655c0bba90 con 0x557b4235cc30 2024-09-10T09:43:06.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:06.438+0000 7f65774d3740 10 client.26324 fetch_fsmap learned FSMap version 34 2024-09-10T09:43:06.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:06.438+0000 7f65774d3740 1 -- 172.21.15.5:0/562066080 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x557b42375cd0 con 0x557b4235cc30 2024-09-10T09:43:06.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:06.439+0000 7f654f400640 1 -- 172.21.15.5:0/562066080 <== mon.2 v2:172.21.15.102:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f656000fcc0 con 0x557b4235cc30 2024-09-10T09:43:06.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:06.439+0000 7f654f400640 1 -- 172.21.15.5:0/562066080 <== mon.2 v2:172.21.15.102:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f655c03b4f0 con 0x557b4235cc30 2024-09-10T09:43:06.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:06.439+0000 7f65774d3740 10 client.26324 fetch_fsmap finished waiting for FSMap version 34 2024-09-10T09:43:06.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:06.439+0000 7f65774d3740 10 client.26324 resolve_mds: validated gid 24542 aka daemon mds.b 2024-09-10T09:43:06.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:06.439+0000 7f65774d3740 20 client.26324 populate_metadata read hostname 'smithi005' 2024-09-10T09:43:06.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:06.439+0000 7f65774d3740 1 --2- 172.21.15.5:0/562066080 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x557b42377130 0x557b42379540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:43:06.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:06.439+0000 7f65774d3740 4 client.26324 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:43:06.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:06.439+0000 7f65774d3740 1 -- 172.21.15.5:0/562066080 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557b422cfbc0 con 0x557b42377130 2024-09-10T09:43:06.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:06.440+0000 7f656de00640 1 --2- 172.21.15.5:0/562066080 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x557b42377130 0x557b42379540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:43:06.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:06.440+0000 7f656de00640 1 --2- 172.21.15.5:0/562066080 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x557b42377130 0x557b42379540 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9077e0e71abf9781 server_cookie=6b5a9904e47f80cf in_seq=0 out_seq=0 2024-09-10T09:43:06.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:06.440+0000 7f654f400640 10 client.26324 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T09:43:06.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:06.441+0000 7f654f400640 1 -- 172.21.15.5:0/562066080 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557b422cfbc0 con 0x557b42377130 2024-09-10T09:43:06.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:06.441+0000 7f654f400640 10 client.26324 handle_command_reply: tid=0 2024-09-10T09:43:06.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:06.441+0000 7f65774d3740 1 client.26324 shutdown 2024-09-10T09:43:06.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:06.441+0000 7f65774d3740 1 -- 172.21.15.5:0/562066080 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557b4235cc30 msgr2=0x557b423648f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:43:06.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:06.441+0000 7f65774d3740 1 --2- 172.21.15.5:0/562066080 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557b4235cc30 0x557b423648f0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f655c0bbc50 tx=0x7f655c031f20 comp rx=0 tx=0).stop 2024-09-10T09:43:06.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:06.441+0000 7f65774d3740 1 -- 172.21.15.5:0/562066080 shutdown_connections 2024-09-10T09:43:06.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:06.441+0000 7f65774d3740 1 --2- 172.21.15.5:0/562066080 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x557b42377130 0x557b42379540 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:06.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:06.441+0000 7f65774d3740 1 --2- 172.21.15.5:0/562066080 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557b423674f0 0x557b42364e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:06.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:06.441+0000 7f65774d3740 1 --2- 172.21.15.5:0/562066080 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557b4235cc30 0x557b423648f0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:06.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:06.441+0000 7f65774d3740 1 --2- 172.21.15.5:0/562066080 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557b4235c310 0x557b423643b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:06.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:06.441+0000 7f654f400640 0 client.26324 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T09:43:06.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:06.441+0000 7f65774d3740 1 -- 172.21.15.5:0/562066080 >> 172.21.15.5:0/562066080 conn(0x557b423588a0 msgr2=0x557b42357aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:43:06.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:06.441+0000 7f65774d3740 1 -- 172.21.15.5:0/562066080 shutdown_connections 2024-09-10T09:43:06.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:06.441+0000 7f65774d3740 1 -- 172.21.15.5:0/562066080 wait complete. 2024-09-10T09:43:06.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:06.442+0000 7f65774d3740 20 client.26324 trim_cache size 0 max 16384 2024-09-10T09:43:06.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:06.460+0000 7f65774d3740 1 Processor -- start 2024-09-10T09:43:06.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:06.460+0000 7f65774d3740 1 -- start start 2024-09-10T09:43:06.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:06.460+0000 7f65774d3740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557b423588a0 0x557b4237ee50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:43:06.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:06.460+0000 7f65774d3740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557b4235c310 0x557b4237f390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:43:06.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:06.460+0000 7f65774d3740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557b4235cc30 0x557b42361630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:43:06.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:06.460+0000 7f65774d3740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f65701acc00 con 0x557b4235cc30 2024-09-10T09:43:06.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:06.460+0000 7f65774d3740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f65701ae740 con 0x557b4235c310 2024-09-10T09:43:06.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:06.460+0000 7f65774d3740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f65701ad2d0 con 0x557b423588a0 2024-09-10T09:43:06.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:06.461+0000 7f656d400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557b4235c310 0x557b4237f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:43:06.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:06.461+0000 7f656e800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557b4235cc30 0x557b42361630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:43:06.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:06.461+0000 7f656d400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557b4235c310 0x557b4237f390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:50620/0 (socket says 172.21.15.5:50620) 2024-09-10T09:43:06.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:06.461+0000 7f656d400640 1 -- 172.21.15.5:0/395270278 learned_addr learned my addr 172.21.15.5:0/395270278 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:43:06.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:06.461+0000 7f656de00640 1 --2- 172.21.15.5:0/395270278 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557b423588a0 0x557b4237ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:43:06.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:06.461+0000 7f656de00640 1 -- 172.21.15.5:0/395270278 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557b4235c310 msgr2=0x557b4237f390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:43:06.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:06.461+0000 7f656de00640 1 --2- 172.21.15.5:0/395270278 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557b4235c310 0x557b4237f390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:06.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:06.461+0000 7f656de00640 1 -- 172.21.15.5:0/395270278 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557b4235cc30 msgr2=0x557b42361630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:43:06.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:06.461+0000 7f656de00640 1 --2- 172.21.15.5:0/395270278 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557b4235cc30 0x557b42361630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:06.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:06.461+0000 7f656de00640 1 -- 172.21.15.5:0/395270278 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557b42369fa0 con 0x557b423588a0 2024-09-10T09:43:06.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:06.461+0000 7f656d400640 1 --2- 172.21.15.5:0/395270278 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557b4235c310 0x557b4237f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:43:06.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:06.462+0000 7f656de00640 1 --2- 172.21.15.5:0/395270278 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557b423588a0 0x557b4237ee50 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f656001b040 tx=0x7f6560019910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fe505dec234f02dd server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:43:06.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:06.462+0000 7f655a600640 1 -- 172.21.15.5:0/395270278 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6560030070 con 0x557b423588a0 2024-09-10T09:43:06.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:06.462+0000 7f655a600640 1 -- 172.21.15.5:0/395270278 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6560019410 con 0x557b423588a0 2024-09-10T09:43:06.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:06.462+0000 7f655a600640 1 -- 172.21.15.5:0/395270278 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6560005590 con 0x557b423588a0 2024-09-10T09:43:06.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:06.463+0000 7f65774d3740 1 -- 172.21.15.5:0/395270278 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557b423588a0 msgr2=0x557b4237ee50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:43:06.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:06.463+0000 7f65774d3740 1 --2- 172.21.15.5:0/395270278 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557b423588a0 0x557b4237ee50 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f656001b040 tx=0x7f6560019910 comp rx=0 tx=0).stop 2024-09-10T09:43:06.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:06.463+0000 7f65774d3740 1 -- 172.21.15.5:0/395270278 shutdown_connections 2024-09-10T09:43:06.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:06.463+0000 7f65774d3740 1 --2- 172.21.15.5:0/395270278 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557b4235cc30 0x557b42361630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:06.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:06.463+0000 7f65774d3740 1 --2- 172.21.15.5:0/395270278 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557b4235c310 0x557b4237f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:06.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:06.463+0000 7f65774d3740 1 --2- 172.21.15.5:0/395270278 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557b423588a0 0x557b4237ee50 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:06.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:06.463+0000 7f65774d3740 1 -- 172.21.15.5:0/395270278 >> 172.21.15.5:0/395270278 conn(0x557b42353e60 msgr2=0x557b42357aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:43:06.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:06.463+0000 7f65774d3740 1 -- 172.21.15.5:0/395270278 shutdown_connections 2024-09-10T09:43:06.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:06.463+0000 7f65774d3740 1 -- 172.21.15.5:0/395270278 wait complete. 2024-09-10T09:43:06.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:06.463+0000 7f65774d3740 1 -- start start 2024-09-10T09:43:06.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:06.464+0000 7f65774d3740 1 Processor -- start 2024-09-10T09:43:06.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:06.464+0000 7f65774d3740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557b423588a0 0x557b4236f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:43:06.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:06.464+0000 7f65774d3740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557b4235c310 0x557b4236fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:43:06.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:06.464+0000 7f65774d3740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557b4235cc30 0x557b4236c490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:43:06.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:06.464+0000 7f65774d3740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f65701ae070 con 0x557b4235cc30 2024-09-10T09:43:06.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:06.464+0000 7f65774d3740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f65701ad9a0 con 0x557b423588a0 2024-09-10T09:43:06.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:06.464+0000 7f65774d3740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f65701aac00 con 0x557b4235c310 2024-09-10T09:43:06.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:06.464+0000 7f65774d3740 10 client.0 _ll_register_callbacks cb 0x7f655c0f5220 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:43:06.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:06.464+0000 7f656de00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557b423588a0 0x557b4236f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:43:06.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:06.464+0000 7f656d400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557b4235c310 0x557b4236fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:43:06.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:06.464+0000 7f656e800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557b4235cc30 0x557b4236c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:43:06.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:06.464+0000 7f656de00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557b423588a0 0x557b4236f6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:50632/0 (socket says 172.21.15.5:50632) 2024-09-10T09:43:06.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:06.465+0000 7f656de00640 1 -- 172.21.15.5:0/2161882607 learned_addr learned my addr 172.21.15.5:0/2161882607 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:43:06.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:06.465+0000 7f656d400640 1 --2- 172.21.15.5:0/2161882607 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557b4235c310 0x557b4236fbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:59282/0 (socket says 172.21.15.5:59282) 2024-09-10T09:43:06.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:06.465+0000 7f656de00640 1 -- 172.21.15.5:0/2161882607 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557b4235c310 msgr2=0x557b4236fbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:43:06.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:06.465+0000 7f656de00640 1 --2- 172.21.15.5:0/2161882607 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557b4235c310 0x557b4236fbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:06.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:06.465+0000 7f656de00640 1 -- 172.21.15.5:0/2161882607 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557b4235cc30 msgr2=0x557b4236c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:43:06.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:06.465+0000 7f656de00640 1 --2- 172.21.15.5:0/2161882607 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557b4235cc30 0x557b4236c490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:06.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:06.465+0000 7f656de00640 1 -- 172.21.15.5:0/2161882607 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557b4236ccc0 con 0x557b423588a0 2024-09-10T09:43:06.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:06.465+0000 7f656de00640 1 --2- 172.21.15.5:0/2161882607 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557b423588a0 0x557b4236f6a0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f65600115c0 tx=0x7f65600195c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8224ac0be2a5eedc server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:43:06.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:06.465+0000 7f654f400640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-09-10T09:43:06.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:06.466+0000 7f654f400640 1 -- 172.21.15.5:0/2161882607 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6560030040 con 0x557b423588a0 2024-09-10T09:43:06.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:06.466+0000 7f654f400640 1 -- 172.21.15.5:0/2161882607 <== mon.2 v2:172.21.15.102:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f6560009ca0 con 0x557b423588a0 2024-09-10T09:43:06.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:06.466+0000 7f65774d3740 1 -- 172.21.15.5:0/2161882607 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6560019410 con 0x557b423588a0 2024-09-10T09:43:06.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:06.466+0000 7f654f400640 1 -- 172.21.15.5:0/2161882607 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f656003e050 con 0x557b423588a0 2024-09-10T09:43:06.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:06.466+0000 7f65774d3740 10 client.26330 fetch_fsmap learned FSMap version 34 2024-09-10T09:43:06.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:06.466+0000 7f65774d3740 1 -- 172.21.15.5:0/2161882607 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x557b423e2950 con 0x557b423588a0 2024-09-10T09:43:06.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:06.466+0000 7f654f400640 1 -- 172.21.15.5:0/2161882607 <== mon.2 v2:172.21.15.102:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6560019410 con 0x557b423588a0 2024-09-10T09:43:06.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:06.467+0000 7f654f400640 1 -- 172.21.15.5:0/2161882607 <== mon.2 v2:172.21.15.102:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f656000f800 con 0x557b423588a0 2024-09-10T09:43:06.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:06.467+0000 7f65774d3740 10 client.26330 fetch_fsmap finished waiting for FSMap version 34 2024-09-10T09:43:06.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:06.467+0000 7f65774d3740 10 client.26330 resolve_mds: validated gid 24542 aka daemon mds.b 2024-09-10T09:43:06.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:06.467+0000 7f65774d3740 20 client.26330 populate_metadata read hostname 'smithi005' 2024-09-10T09:43:06.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:06.467+0000 7f65774d3740 1 --2- 172.21.15.5:0/2161882607 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x557b423674f0 0x557b423e4d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:43:06.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:06.467+0000 7f65774d3740 4 client.26330 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "6609e6f", "release": true, "await": true}] 2024-09-10T09:43:06.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:06.467+0000 7f65774d3740 1 -- 172.21.15.5:0/2161882607 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "quiesce db", "set_id": "6609e6f", "release": true, "await": true}) -- 0x557b422ca0d0 con 0x557b423674f0 2024-09-10T09:43:06.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:06.467+0000 7f656d400640 1 --2- 172.21.15.5:0/2161882607 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x557b423674f0 0x557b423e4d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:43:06.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:06.467+0000 7f656d400640 1 --2- 172.21.15.5:0/2161882607 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x557b423674f0 0x557b423e4d80 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6576ad018ed77741 server_cookie=2417409f54cab347 in_seq=0 out_seq=0 2024-09-10T09:43:06.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:06.467+0000 7f654f400640 10 client.26330 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T09:43:06.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:06.476+0000 7f654f400640 1 -- 172.21.15.5:0/2161882607 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x557b422ca0d0 con 0x557b423674f0 2024-09-10T09:43:06.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:06.476+0000 7f654f400640 10 client.26330 handle_command_reply: tid=0 2024-09-10T09:43:06.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:06.476+0000 7f65774d3740 1 client.26330 shutdown 2024-09-10T09:43:06.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:06.476+0000 7f65774d3740 1 -- 172.21.15.5:0/2161882607 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557b423588a0 msgr2=0x557b4236f6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:43:06.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:06.476+0000 7f65774d3740 1 --2- 172.21.15.5:0/2161882607 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557b423588a0 0x557b4236f6a0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f65600115c0 tx=0x7f65600195c0 comp rx=0 tx=0).stop 2024-09-10T09:43:06.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:06.476+0000 7f65774d3740 1 -- 172.21.15.5:0/2161882607 shutdown_connections 2024-09-10T09:43:06.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:06.476+0000 7f65774d3740 1 --2- 172.21.15.5:0/2161882607 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x557b423674f0 0x557b423e4d80 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:06.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:06.476+0000 7f65774d3740 1 --2- 172.21.15.5:0/2161882607 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557b4235cc30 0x557b4236c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:06.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:06.476+0000 7f65774d3740 1 --2- 172.21.15.5:0/2161882607 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557b4235c310 0x557b4236fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:06.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:06.476+0000 7f65774d3740 1 --2- 172.21.15.5:0/2161882607 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557b423588a0 0x557b4236f6a0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:06.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:06.476+0000 7f654f400640 0 client.26330 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T09:43:06.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:06.476+0000 7f65774d3740 1 -- 172.21.15.5:0/2161882607 >> 172.21.15.5:0/2161882607 conn(0x557b42353e60 msgr2=0x557b4235dd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:43:06.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:06.476+0000 7f65774d3740 1 -- 172.21.15.5:0/2161882607 shutdown_connections 2024-09-10T09:43:06.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:06.476+0000 7f65774d3740 1 -- 172.21.15.5:0/2161882607 wait complete. 2024-09-10T09:43:06.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:06.476+0000 7f65774d3740 20 client.26330 trim_cache size 0 max 16384 2024-09-10T09:43:06.479 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-10T09:43:06.480 INFO:teuthology.orchestra.run.smithi005.stdout: "epoch": 34, 2024-09-10T09:43:06.480 INFO:teuthology.orchestra.run.smithi005.stdout: "leader": 24542, 2024-09-10T09:43:06.480 INFO:teuthology.orchestra.run.smithi005.stdout: "set_version": 16, 2024-09-10T09:43:06.480 INFO:teuthology.orchestra.run.smithi005.stdout: "sets": { 2024-09-10T09:43:06.480 INFO:teuthology.orchestra.run.smithi005.stdout: "6609e6f": { 2024-09-10T09:43:06.480 INFO:teuthology.orchestra.run.smithi005.stdout: "version": 16, 2024-09-10T09:43:06.480 INFO:teuthology.orchestra.run.smithi005.stdout: "age_ref": 0.0, 2024-09-10T09:43:06.480 INFO:teuthology.orchestra.run.smithi005.stdout: "state": { 2024-09-10T09:43:06.480 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "RELEASED", 2024-09-10T09:43:06.480 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0 2024-09-10T09:43:06.480 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T09:43:06.480 INFO:teuthology.orchestra.run.smithi005.stdout: "timeout": 90.0, 2024-09-10T09:43:06.480 INFO:teuthology.orchestra.run.smithi005.stdout: "expiration": 142.8, 2024-09-10T09:43:06.481 INFO:teuthology.orchestra.run.smithi005.stdout: "members": { 2024-09-10T09:43:06.481 INFO:teuthology.orchestra.run.smithi005.stdout: "file:/": { 2024-09-10T09:43:06.481 INFO:teuthology.orchestra.run.smithi005.stdout: "excluded": false, 2024-09-10T09:43:06.481 INFO:teuthology.orchestra.run.smithi005.stdout: "state": { 2024-09-10T09:43:06.481 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "RELEASED", 2024-09-10T09:43:06.481 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0 2024-09-10T09:43:06.481 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T09:43:06.481 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T09:43:06.481 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T09:43:06.481 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T09:43:06.481 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T09:43:06.481 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-10T09:43:06.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:06.477+0000 7f6574a00640 1 -- 172.21.15.5:0/2953873758 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f65540a2c00 msgr2=0x7f65540a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:43:06.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:06.477+0000 7f6574a00640 1 --2- 172.21.15.5:0/2953873758 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f65540a2c00 0x7f65540a50c0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f6568005fd0 tx=0x7f65680096a0 comp rx=0 tx=0).stop 2024-09-10T09:43:06.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:06.477+0000 7f6574a00640 1 -- 172.21.15.5:0/2953873758 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6570102790 msgr2=0x7f6570074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:43:06.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:06.477+0000 7f6574a00640 1 --2- 172.21.15.5:0/2953873758 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6570102790 0x7f6570074a30 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f655c004770 tx=0x7f655c003c10 comp rx=0 tx=0).stop 2024-09-10T09:43:06.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:06.477+0000 7f6574a00640 1 -- 172.21.15.5:0/2953873758 shutdown_connections 2024-09-10T09:43:06.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:06.477+0000 7f6574a00640 1 --2- 172.21.15.5:0/2953873758 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f65540a2c00 0x7f65540a50c0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:06.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:06.477+0000 7f6574a00640 1 --2- 172.21.15.5:0/2953873758 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6570106f10 0x7f6570074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:06.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:06.477+0000 7f6574a00640 1 --2- 172.21.15.5:0/2953873758 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6570102790 0x7f6570074a30 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:06.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:06.477+0000 7f6574a00640 1 --2- 172.21.15.5:0/2953873758 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6570101de0 0x7f65700744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:06.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:06.477+0000 7f6574a00640 1 -- 172.21.15.5:0/2953873758 >> 172.21.15.5:0/2953873758 conn(0x7f65700fd8c0 msgr2=0x7f65700fec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:43:06.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:06.478+0000 7f6574a00640 1 -- 172.21.15.5:0/2953873758 shutdown_connections 2024-09-10T09:43:06.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:06.478+0000 7f6574a00640 1 -- 172.21.15.5:0/2953873758 wait complete. 2024-09-10T09:43:07.021 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:43:06 smithi005 ceph-mon[30350]: pgmap v536: 129 pgs: 129 active+clean; 92 MiB data, 907 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 24 KiB/s wr, 19 op/s 2024-09-10T09:43:07.029 INFO:tasks.workunit.client.0.smithi005.stdout: 1 205728 1.56 MB/sec execute 48 sec latency 19346.645 ms 2024-09-10T09:43:07.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:43:06 smithi159 ceph-mon[35513]: pgmap v536: 129 pgs: 129 active+clean; 92 MiB data, 907 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 24 KiB/s wr, 19 op/s 2024-09-10T09:43:07.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:43:06 smithi102 ceph-mon[35442]: pgmap v536: 129 pgs: 129 active+clean; 92 MiB data, 907 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 24 KiB/s wr, 19 op/s 2024-09-10T09:43:07.511 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '6609e6f', total seconds elapsed: 27.0 2024-09-10T09:43:08.029 INFO:tasks.workunit.client.0.smithi005.stdout: 1 207391 1.67 MB/sec execute 49 sec latency 31.124 ms 2024-09-10T09:43:09.029 INFO:tasks.workunit.client.0.smithi005.stdout: 1 209746 1.91 MB/sec execute 50 sec latency 9.383 ms 2024-09-10T09:43:09.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:43:08 smithi005 ceph-mon[30350]: pgmap v537: 129 pgs: 129 active+clean; 92 MiB data, 907 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 6.8 KiB/s wr, 12 op/s 2024-09-10T09:43:09.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:43:08 smithi159 ceph-mon[35513]: pgmap v537: 129 pgs: 129 active+clean; 92 MiB data, 907 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 6.8 KiB/s wr, 12 op/s 2024-09-10T09:43:09.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:43:08 smithi102 ceph-mon[35442]: pgmap v537: 129 pgs: 129 active+clean; 92 MiB data, 907 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 6.8 KiB/s wr, 12 op/s 2024-09-10T09:43:10.029 INFO:tasks.workunit.client.0.smithi005.stdout: 1 212156 2.09 MB/sec execute 51 sec latency 25.630 ms 2024-09-10T09:43:11.029 INFO:tasks.workunit.client.0.smithi005.stdout: 1 214230 2.30 MB/sec execute 52 sec latency 24.916 ms 2024-09-10T09:43:11.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:43:10 smithi005 ceph-mon[30350]: pgmap v538: 129 pgs: 129 active+clean; 92 MiB data, 886 MiB used, 1.0 TiB / 1.0 TiB avail; 625 KiB/s rd, 69 KiB/s wr, 19 op/s 2024-09-10T09:43:11.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:43:10 smithi159 ceph-mon[35513]: pgmap v538: 129 pgs: 129 active+clean; 92 MiB data, 886 MiB used, 1.0 TiB / 1.0 TiB avail; 625 KiB/s rd, 69 KiB/s wr, 19 op/s 2024-09-10T09:43:11.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:43:10 smithi102 ceph-mon[35442]: pgmap v538: 129 pgs: 129 active+clean; 92 MiB data, 886 MiB used, 1.0 TiB / 1.0 TiB avail; 625 KiB/s rd, 69 KiB/s wr, 19 op/s 2024-09-10T09:43:12.029 INFO:tasks.workunit.client.0.smithi005.stdout: 1 216802 2.53 MB/sec execute 53 sec latency 19.020 ms 2024-09-10T09:43:13.029 INFO:tasks.workunit.client.0.smithi005.stdout: 1 219205 2.68 MB/sec execute 54 sec latency 42.678 ms 2024-09-10T09:43:13.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:43:12 smithi005 ceph-mon[30350]: pgmap v539: 129 pgs: 129 active+clean; 96 MiB data, 889 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 416 KiB/s wr, 79 op/s 2024-09-10T09:43:13.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:43:12 smithi159 ceph-mon[35513]: pgmap v539: 129 pgs: 129 active+clean; 96 MiB data, 889 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 416 KiB/s wr, 79 op/s 2024-09-10T09:43:13.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:43:12 smithi102 ceph-mon[35442]: pgmap v539: 129 pgs: 129 active+clean; 96 MiB data, 889 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 416 KiB/s wr, 79 op/s 2024-09-10T09:43:14.029 INFO:tasks.workunit.client.0.smithi005.stdout: 1 219690 2.70 MB/sec execute 55 sec latency 834.883 ms 2024-09-10T09:43:15.030 INFO:tasks.workunit.client.0.smithi005.stdout: 1 220955 2.75 MB/sec execute 56 sec latency 991.689 ms 2024-09-10T09:43:15.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:43:15 smithi159 ceph-mon[35513]: pgmap v540: 129 pgs: 129 active+clean; 98 MiB data, 873 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 651 KiB/s wr, 89 op/s 2024-09-10T09:43:15.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:43:15 smithi102 ceph-mon[35442]: pgmap v540: 129 pgs: 129 active+clean; 98 MiB data, 873 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 651 KiB/s wr, 89 op/s 2024-09-10T09:43:16.030 INFO:tasks.workunit.client.0.smithi005.stdout: 1 223766 3.01 MB/sec execute 57 sec latency 13.592 ms 2024-09-10T09:43:16.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:43:15 smithi005 ceph-mon[30350]: pgmap v540: 129 pgs: 129 active+clean; 98 MiB data, 873 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 651 KiB/s wr, 89 op/s 2024-09-10T09:43:16.443 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T09:43:16.844 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:43:17.030 INFO:tasks.workunit.client.0.smithi005.stdout: 1 225713 3.07 MB/sec execute 58 sec latency 25.947 ms 2024-09-10T09:43:17.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:17.469+0000 7f883dc00640 1 -- 172.21.15.5:0/370918270 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8838108f50 msgr2=0x7f8838110ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:43:17.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:17.469+0000 7f883dc00640 1 --2- 172.21.15.5:0/370918270 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8838108f50 0x7f8838110ad0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f8830009ca0 tx=0x7f883002a8c0 comp rx=0 tx=0).stop 2024-09-10T09:43:17.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:17.469+0000 7f883dc00640 1 -- 172.21.15.5:0/370918270 shutdown_connections 2024-09-10T09:43:17.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:17.469+0000 7f883dc00640 1 --2- 172.21.15.5:0/370918270 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8838108f50 0x7f8838110ad0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:17.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:17.469+0000 7f883dc00640 1 --2- 172.21.15.5:0/370918270 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f88381047d0 0x7f8838108820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:17.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:17.469+0000 7f883dc00640 1 --2- 172.21.15.5:0/370918270 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8838103e20 0x7f8838104200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:17.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:17.469+0000 7f883dc00640 1 -- 172.21.15.5:0/370918270 >> 172.21.15.5:0/370918270 conn(0x7f88380ff860 msgr2=0x7f8838101c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:43:17.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:17.469+0000 7f883dc00640 1 -- 172.21.15.5:0/370918270 shutdown_connections 2024-09-10T09:43:17.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:17.469+0000 7f883dc00640 1 -- 172.21.15.5:0/370918270 wait complete. 2024-09-10T09:43:17.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:17.470+0000 7f883dc00640 1 Processor -- start 2024-09-10T09:43:17.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:17.470+0000 7f883dc00640 1 -- start start 2024-09-10T09:43:17.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:17.471+0000 7f883dc00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8838103e20 0x7f883819a400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:43:17.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:17.471+0000 7f883dc00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f88381047d0 0x7f883819a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:43:17.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:17.471+0000 7f883dc00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8838108f50 0x7f883819ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:43:17.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:17.471+0000 7f883dc00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f8838114000 con 0x7f8838103e20 2024-09-10T09:43:17.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:17.471+0000 7f883dc00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f8838113e80 con 0x7f8838108f50 2024-09-10T09:43:17.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:17.471+0000 7f883dc00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f8838114180 con 0x7f88381047d0 2024-09-10T09:43:17.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:17.471+0000 7f8837e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8838103e20 0x7f883819a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:43:17.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:17.471+0000 7f8837400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f88381047d0 0x7f883819a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:43:17.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:17.471+0000 7f8837e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8838103e20 0x7f883819a400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46456/0 (socket says 172.21.15.5:46456) 2024-09-10T09:43:17.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:17.471+0000 7f8837e00640 1 -- 172.21.15.5:0/859344980 learned_addr learned my addr 172.21.15.5:0/859344980 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:43:17.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:17.471+0000 7f883d200640 1 --2- 172.21.15.5:0/859344980 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8838108f50 0x7f883819ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:43:17.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:17.472+0000 7f8837e00640 1 -- 172.21.15.5:0/859344980 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f88381047d0 msgr2=0x7f883819a940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:43:17.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:17.472+0000 7f8837e00640 1 --2- 172.21.15.5:0/859344980 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f88381047d0 0x7f883819a940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:17.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:17.472+0000 7f8837e00640 1 -- 172.21.15.5:0/859344980 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8838108f50 msgr2=0x7f883819ecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:43:17.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:17.472+0000 7f8837e00640 1 --2- 172.21.15.5:0/859344980 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8838108f50 0x7f883819ecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:17.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:17.472+0000 7f8837e00640 1 -- 172.21.15.5:0/859344980 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f883819f3b0 con 0x7f8838103e20 2024-09-10T09:43:17.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:17.472+0000 7f8837e00640 1 --2- 172.21.15.5:0/859344980 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8838103e20 0x7f883819a400 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f882800e9f0 tx=0x7f882800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:43:17.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:17.472+0000 7f8834c00640 1 -- 172.21.15.5:0/859344980 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f882800cd80 con 0x7f8838103e20 2024-09-10T09:43:17.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:17.472+0000 7f8834c00640 1 -- 172.21.15.5:0/859344980 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8828010910 con 0x7f8838103e20 2024-09-10T09:43:17.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:17.472+0000 7f8834c00640 1 -- 172.21.15.5:0/859344980 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8828018990 con 0x7f8838103e20 2024-09-10T09:43:17.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:17.472+0000 7f883dc00640 1 -- 172.21.15.5:0/859344980 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f883819f6a0 con 0x7f8838103e20 2024-09-10T09:43:17.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:17.473+0000 7f883dc00640 1 -- 172.21.15.5:0/859344980 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88381a6f80 con 0x7f8838103e20 2024-09-10T09:43:17.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:17.476+0000 7f8834c00640 1 -- 172.21.15.5:0/859344980 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f8828010ac0 con 0x7f8838103e20 2024-09-10T09:43:17.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:17.476+0000 7f8841e01740 1 Processor -- start 2024-09-10T09:43:17.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:17.476+0000 7f8841e01740 1 -- start start 2024-09-10T09:43:17.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:17.476+0000 7f8841e01740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e79e705d50 0x55e79e706130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:43:17.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:17.476+0000 7f8841e01740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e79e706670 0x55e79e7109f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:43:17.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:17.477+0000 7f8841e01740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e79e710f30 0x55e79e713320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:43:17.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:17.477+0000 7f8841e01740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55e79e66c3e0 con 0x55e79e710f30 2024-09-10T09:43:17.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:17.477+0000 7f8841e01740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55e79e613d80 con 0x55e79e706670 2024-09-10T09:43:17.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:17.477+0000 7f8841e01740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55e79e6683a0 con 0x55e79e705d50 2024-09-10T09:43:17.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:17.477+0000 7f8837e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e79e706670 0x55e79e7109f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:43:17.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:17.477+0000 7f8837e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e79e706670 0x55e79e7109f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:41994/0 (socket says 172.21.15.5:41994) 2024-09-10T09:43:17.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:17.477+0000 7f883d200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e79e705d50 0x55e79e706130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:43:17.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:17.477+0000 7f8837e00640 1 -- 172.21.15.5:0/4261360934 learned_addr learned my addr 172.21.15.5:0/4261360934 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:43:17.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:17.477+0000 7f8837400640 1 --2- 172.21.15.5:0/4261360934 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e79e710f30 0x55e79e713320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:43:17.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:17.477+0000 7f8837e00640 1 -- 172.21.15.5:0/4261360934 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e79e705d50 msgr2=0x55e79e706130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:43:17.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:17.477+0000 7f8837e00640 1 --2- 172.21.15.5:0/4261360934 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e79e705d50 0x55e79e706130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:17.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:17.477+0000 7f8837e00640 1 -- 172.21.15.5:0/4261360934 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e79e710f30 msgr2=0x55e79e713320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:43:17.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:17.477+0000 7f8837e00640 1 --2- 172.21.15.5:0/4261360934 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e79e710f30 0x55e79e713320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:17.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:17.477+0000 7f8837e00640 1 -- 172.21.15.5:0/4261360934 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e79e632ae0 con 0x55e79e706670 2024-09-10T09:43:17.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:17.477+0000 7f8837400640 1 --2- 172.21.15.5:0/4261360934 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e79e710f30 0x55e79e713320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T09:43:17.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:17.478+0000 7f883d200640 1 --2- 172.21.15.5:0/4261360934 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e79e705d50 0x55e79e706130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:43:17.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:17.478+0000 7f8834c00640 1 --2- 172.21.15.5:0/859344980 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f881c0a2b80 0x7f881c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:43:17.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:17.478+0000 7f8834c00640 1 -- 172.21.15.5:0/859344980 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f88280cf830 con 0x7f8838103e20 2024-09-10T09:43:17.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:17.478+0000 7f8837e00640 1 --2- 172.21.15.5:0/4261360934 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e79e706670 0x55e79e7109f0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f88280d5c20 tx=0x7f88280d7670 comp rx=0 tx=0).ready entity=mon.2 client_cookie=85810e78a1b82cd1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:43:17.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:17.478+0000 7f883d200640 1 --2- 172.21.15.5:0/859344980 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f881c0a2b80 0x7f881c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:43:17.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:17.478+0000 7f882d600640 1 -- 172.21.15.5:0/4261360934 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8828097ee0 con 0x55e79e706670 2024-09-10T09:43:17.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:17.479+0000 7f882d600640 1 -- 172.21.15.5:0/4261360934 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f882809c340 con 0x55e79e706670 2024-09-10T09:43:17.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:17.479+0000 7f882d600640 1 -- 172.21.15.5:0/4261360934 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f882809b5a0 con 0x55e79e706670 2024-09-10T09:43:17.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:17.479+0000 7f8841e01740 1 -- 172.21.15.5:0/4261360934 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e79e706670 msgr2=0x55e79e7109f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:43:17.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:17.479+0000 7f8841e01740 1 --2- 172.21.15.5:0/4261360934 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e79e706670 0x55e79e7109f0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f88280d5c20 tx=0x7f88280d7670 comp rx=0 tx=0).stop 2024-09-10T09:43:17.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:17.479+0000 7f883d200640 1 --2- 172.21.15.5:0/859344980 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f881c0a2b80 0x7f881c0a5040 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f8830009430 tx=0x7f8830009820 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:43:17.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:17.479+0000 7f8841e01740 1 -- 172.21.15.5:0/4261360934 shutdown_connections 2024-09-10T09:43:17.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:17.479+0000 7f8841e01740 1 --2- 172.21.15.5:0/4261360934 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e79e710f30 0x55e79e713320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:17.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:17.479+0000 7f8841e01740 1 --2- 172.21.15.5:0/4261360934 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e79e706670 0x55e79e7109f0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:17.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:17.479+0000 7f8841e01740 1 --2- 172.21.15.5:0/4261360934 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e79e705d50 0x55e79e706130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:17.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:17.479+0000 7f8841e01740 1 -- 172.21.15.5:0/4261360934 >> 172.21.15.5:0/4261360934 conn(0x55e79e7022e0 msgr2=0x55e79e7026c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:43:17.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:17.480+0000 7f8841e01740 1 -- 172.21.15.5:0/4261360934 shutdown_connections 2024-09-10T09:43:17.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:17.480+0000 7f8841e01740 1 -- 172.21.15.5:0/4261360934 wait complete. 2024-09-10T09:43:17.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:17.480+0000 7f8841e01740 1 -- start start 2024-09-10T09:43:17.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:17.480+0000 7f8841e01740 1 Processor -- start 2024-09-10T09:43:17.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:17.481+0000 7f8841e01740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e79e705d50 0x55e79e70de20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:43:17.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:17.481+0000 7f8841e01740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e79e706670 0x55e79e70e360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:43:17.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:17.481+0000 7f8841e01740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e79e710f30 0x55e79e70e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:43:17.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:17.481+0000 7f8841e01740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55e79e713ba0 con 0x55e79e705d50 2024-09-10T09:43:17.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:17.481+0000 7f8841e01740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55e79e6270f0 con 0x55e79e710f30 2024-09-10T09:43:17.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:17.481+0000 7f8841e01740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55e79e713d20 con 0x55e79e706670 2024-09-10T09:43:17.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:17.481+0000 7f8841e01740 10 client.0 _ll_register_callbacks cb 0x55e79e6189e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:43:17.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:17.481+0000 7f8837e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e79e705d50 0x55e79e70de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:43:17.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:17.481+0000 7f8837e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e79e705d50 0x55e79e70de20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46472/0 (socket says 172.21.15.5:46472) 2024-09-10T09:43:17.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:17.481+0000 7f8837400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e79e706670 0x55e79e70e360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:43:17.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:17.481+0000 7f8837e00640 1 -- 172.21.15.5:0/927310532 learned_addr learned my addr 172.21.15.5:0/927310532 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:43:17.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:17.481+0000 7f883d200640 1 --2- 172.21.15.5:0/927310532 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e79e710f30 0x55e79e70e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:43:17.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:17.482+0000 7f8837400640 1 -- 172.21.15.5:0/927310532 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e79e710f30 msgr2=0x55e79e70e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:43:17.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:17.482+0000 7f8837400640 1 --2- 172.21.15.5:0/927310532 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e79e710f30 0x55e79e70e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:17.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:17.482+0000 7f8837400640 1 -- 172.21.15.5:0/927310532 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e79e705d50 msgr2=0x55e79e70de20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:43:17.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:17.482+0000 7f8837400640 1 --2- 172.21.15.5:0/927310532 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e79e705d50 0x55e79e70de20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:17.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:17.482+0000 7f8837400640 1 -- 172.21.15.5:0/927310532 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e79e3dd3c0 con 0x55e79e706670 2024-09-10T09:43:17.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:17.482+0000 7f8837e00640 1 --2- 172.21.15.5:0/927310532 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e79e705d50 0x55e79e70de20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:43:17.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:17.482+0000 7f8837400640 1 --2- 172.21.15.5:0/927310532 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e79e706670 0x55e79e70e360 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f88380ffc80 tx=0x7f8824013b70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=326bb27e2f78c970 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:43:17.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:17.482+0000 7f8822000640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-09-10T09:43:17.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:17.482+0000 7f8822000640 1 -- 172.21.15.5:0/927310532 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88240119c0 con 0x55e79e706670 2024-09-10T09:43:17.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:17.482+0000 7f8822000640 1 -- 172.21.15.5:0/927310532 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f8824011b60 con 0x55e79e706670 2024-09-10T09:43:17.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:17.482+0000 7f8841e01740 1 -- 172.21.15.5:0/927310532 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f882809c340 con 0x55e79e706670 2024-09-10T09:43:17.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:17.484+0000 7f8822000640 1 -- 172.21.15.5:0/927310532 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f8824011d60 con 0x55e79e706670 2024-09-10T09:43:17.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:17.484+0000 7f8841e01740 10 client.26365 fetch_fsmap learned FSMap version 34 2024-09-10T09:43:17.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:17.484+0000 7f8841e01740 1 -- 172.21.15.5:0/927310532 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55e79e71f820 con 0x55e79e706670 2024-09-10T09:43:17.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:17.484+0000 7f8822000640 1 -- 172.21.15.5:0/927310532 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f882809c340 con 0x55e79e706670 2024-09-10T09:43:17.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:17.484+0000 7f8822000640 1 -- 172.21.15.5:0/927310532 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f882401a070 con 0x55e79e706670 2024-09-10T09:43:17.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:17.484+0000 7f8841e01740 10 client.26365 fetch_fsmap finished waiting for FSMap version 34 2024-09-10T09:43:17.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:17.484+0000 7f8841e01740 10 client.26365 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T09:43:17.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:17.485+0000 7f8841e01740 20 client.26365 populate_metadata read hostname 'smithi005' 2024-09-10T09:43:17.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:17.485+0000 7f8841e01740 1 --2- 172.21.15.5:0/927310532 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55e79e720ce0 0x55e79e723180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:43:17.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:17.485+0000 7f8841e01740 4 client.26365 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:43:17.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:17.485+0000 7f8841e01740 1 -- 172.21.15.5:0/927310532 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e79e679950 con 0x55e79e720ce0 2024-09-10T09:43:17.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:17.485+0000 7f8837e00640 1 --2- 172.21.15.5:0/927310532 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55e79e720ce0 0x55e79e723180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:43:17.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:17.485+0000 7f8837e00640 1 --2- 172.21.15.5:0/927310532 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55e79e720ce0 0x55e79e723180 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=374c4a470610898b server_cookie=4c6821275f15766 in_seq=0 out_seq=0 2024-09-10T09:43:17.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:17.485+0000 7f8822000640 10 client.26365 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T09:43:17.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:17.486+0000 7f8822000640 1 -- 172.21.15.5:0/927310532 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e79e679950 con 0x55e79e720ce0 2024-09-10T09:43:17.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:17.486+0000 7f8822000640 10 client.26365 handle_command_reply: tid=0 2024-09-10T09:43:17.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:17.486+0000 7f8841e01740 1 client.26365 shutdown 2024-09-10T09:43:17.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:17.486+0000 7f8841e01740 1 -- 172.21.15.5:0/927310532 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e79e706670 msgr2=0x55e79e70e360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:43:17.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:17.486+0000 7f8841e01740 1 --2- 172.21.15.5:0/927310532 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e79e706670 0x55e79e70e360 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f88380ffc80 tx=0x7f8824013b70 comp rx=0 tx=0).stop 2024-09-10T09:43:17.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:17.486+0000 7f8841e01740 1 -- 172.21.15.5:0/927310532 shutdown_connections 2024-09-10T09:43:17.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:17.486+0000 7f8841e01740 1 --2- 172.21.15.5:0/927310532 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55e79e720ce0 0x55e79e723180 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:17.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:17.486+0000 7f8841e01740 1 --2- 172.21.15.5:0/927310532 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e79e710f30 0x55e79e70e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:17.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:17.486+0000 7f8841e01740 1 --2- 172.21.15.5:0/927310532 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e79e706670 0x55e79e70e360 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:17.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:17.486+0000 7f8841e01740 1 --2- 172.21.15.5:0/927310532 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e79e705d50 0x55e79e70de20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:17.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:17.486+0000 7f8822000640 0 client.26365 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T09:43:17.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:17.486+0000 7f8841e01740 1 -- 172.21.15.5:0/927310532 >> 172.21.15.5:0/927310532 conn(0x55e79e7022e0 msgr2=0x55e79e707fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:43:17.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:17.487+0000 7f8841e01740 1 -- 172.21.15.5:0/927310532 shutdown_connections 2024-09-10T09:43:17.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:17.487+0000 7f8841e01740 1 -- 172.21.15.5:0/927310532 wait complete. 2024-09-10T09:43:17.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:17.487+0000 7f8841e01740 20 client.26365 trim_cache size 0 max 16384 2024-09-10T09:43:17.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:17.507+0000 7f8841e01740 1 Processor -- start 2024-09-10T09:43:17.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:17.507+0000 7f8841e01740 1 -- start start 2024-09-10T09:43:17.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:17.507+0000 7f8841e01740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e79e705d50 0x55e79e6fd060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:43:17.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:17.507+0000 7f8841e01740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e79e706670 0x55e79e70fd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:43:17.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:17.507+0000 7f8841e01740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e79e710f30 0x55e79e728a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:43:17.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:17.507+0000 7f8841e01740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f88381b0f10 con 0x55e79e705d50 2024-09-10T09:43:17.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:17.507+0000 7f8841e01740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f88381b2a50 con 0x55e79e710f30 2024-09-10T09:43:17.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:17.507+0000 7f8841e01740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f88381b15e0 con 0x55e79e706670 2024-09-10T09:43:17.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:17.507+0000 7f8837400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e79e706670 0x55e79e70fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:43:17.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:17.507+0000 7f883d200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e79e710f30 0x55e79e728a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:43:17.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:17.507+0000 7f8837e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e79e705d50 0x55e79e6fd060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:43:17.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:17.507+0000 7f8837400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e79e706670 0x55e79e70fd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:51186/0 (socket says 172.21.15.5:51186) 2024-09-10T09:43:17.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:17.507+0000 7f8837400640 1 -- 172.21.15.5:0/1612018161 learned_addr learned my addr 172.21.15.5:0/1612018161 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:43:17.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:17.507+0000 7f883d200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e79e710f30 0x55e79e728a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:42012/0 (socket says 172.21.15.5:42012) 2024-09-10T09:43:17.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:17.508+0000 7f8837400640 1 -- 172.21.15.5:0/1612018161 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e79e710f30 msgr2=0x55e79e728a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:43:17.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:17.508+0000 7f8837400640 1 --2- 172.21.15.5:0/1612018161 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e79e710f30 0x55e79e728a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:17.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:17.508+0000 7f8837400640 1 -- 172.21.15.5:0/1612018161 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e79e705d50 msgr2=0x55e79e6fd060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:43:17.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:17.508+0000 7f8837400640 1 --2- 172.21.15.5:0/1612018161 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e79e705d50 0x55e79e6fd060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:17.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:17.508+0000 7f8837400640 1 -- 172.21.15.5:0/1612018161 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e79e70cdd0 con 0x55e79e706670 2024-09-10T09:43:17.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:17.508+0000 7f883d200640 1 --2- 172.21.15.5:0/1612018161 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e79e710f30 0x55e79e728a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:43:17.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:17.508+0000 7f8837400640 1 --2- 172.21.15.5:0/1612018161 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e79e706670 0x55e79e70fd90 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f882401c780 tx=0x7f8824009fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7827db7aa0d7260d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:43:17.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:17.508+0000 7f882cc00640 1 -- 172.21.15.5:0/1612018161 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f882402f070 con 0x55e79e706670 2024-09-10T09:43:17.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:17.508+0000 7f882cc00640 1 -- 172.21.15.5:0/1612018161 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f88240119c0 con 0x55e79e706670 2024-09-10T09:43:17.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:17.509+0000 7f882cc00640 1 -- 172.21.15.5:0/1612018161 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8824003ad0 con 0x55e79e706670 2024-09-10T09:43:17.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:17.509+0000 7f8841e01740 1 -- 172.21.15.5:0/1612018161 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e79e706670 msgr2=0x55e79e70fd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:43:17.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:17.509+0000 7f8841e01740 1 --2- 172.21.15.5:0/1612018161 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e79e706670 0x55e79e70fd90 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f882401c780 tx=0x7f8824009fd0 comp rx=0 tx=0).stop 2024-09-10T09:43:17.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:17.509+0000 7f8841e01740 1 -- 172.21.15.5:0/1612018161 shutdown_connections 2024-09-10T09:43:17.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:17.509+0000 7f8841e01740 1 --2- 172.21.15.5:0/1612018161 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e79e710f30 0x55e79e728a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:17.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:17.509+0000 7f8841e01740 1 --2- 172.21.15.5:0/1612018161 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e79e706670 0x55e79e70fd90 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:17.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:17.509+0000 7f8841e01740 1 --2- 172.21.15.5:0/1612018161 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e79e705d50 0x55e79e6fd060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:17.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:17.509+0000 7f8841e01740 1 -- 172.21.15.5:0/1612018161 >> 172.21.15.5:0/1612018161 conn(0x55e79e7022e0 msgr2=0x55e79e70f680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:43:17.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:17.509+0000 7f8841e01740 1 -- 172.21.15.5:0/1612018161 shutdown_connections 2024-09-10T09:43:17.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:17.510+0000 7f8841e01740 1 -- 172.21.15.5:0/1612018161 wait complete. 2024-09-10T09:43:17.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:17.510+0000 7f8841e01740 1 -- start start 2024-09-10T09:43:17.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:17.510+0000 7f8841e01740 1 Processor -- start 2024-09-10T09:43:17.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:17.511+0000 7f8841e01740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e79e705d50 0x55e79e7197d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:43:17.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:17.511+0000 7f8841e01740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e79e706670 0x55e79e714b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:43:17.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:17.511+0000 7f8841e01740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e79e710f30 0x55e79e715140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:43:17.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:17.511+0000 7f8841e01740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f88381b2380 con 0x55e79e710f30 2024-09-10T09:43:17.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:17.511+0000 7f8841e01740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f88381b1cb0 con 0x55e79e706670 2024-09-10T09:43:17.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:17.511+0000 7f8841e01740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f88381aeef0 con 0x55e79e705d50 2024-09-10T09:43:17.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:17.511+0000 7f8841e01740 10 client.0 _ll_register_callbacks cb 0x7f8824009db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:43:17.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:17.511+0000 7f8837400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e79e706670 0x55e79e714b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:43:17.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:17.511+0000 7f8837e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e79e705d50 0x55e79e7197d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:43:17.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:17.511+0000 7f8837400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e79e706670 0x55e79e714b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:42026/0 (socket says 172.21.15.5:42026) 2024-09-10T09:43:17.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:17.511+0000 7f8837400640 1 -- 172.21.15.5:0/2943334059 learned_addr learned my addr 172.21.15.5:0/2943334059 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:43:17.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:17.511+0000 7f883d200640 1 --2- 172.21.15.5:0/2943334059 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e79e710f30 0x55e79e715140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:43:17.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:17.511+0000 7f8837e00640 1 -- 172.21.15.5:0/2943334059 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e79e706670 msgr2=0x55e79e714b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:43:17.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:17.511+0000 7f8837e00640 1 --2- 172.21.15.5:0/2943334059 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e79e706670 0x55e79e714b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:17.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:17.511+0000 7f8837e00640 1 -- 172.21.15.5:0/2943334059 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e79e710f30 msgr2=0x55e79e715140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:43:17.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:17.511+0000 7f8837e00640 1 --2- 172.21.15.5:0/2943334059 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e79e710f30 0x55e79e715140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:17.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:17.511+0000 7f8837e00640 1 -- 172.21.15.5:0/2943334059 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e79e715970 con 0x55e79e705d50 2024-09-10T09:43:17.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:17.512+0000 7f8837e00640 1 --2- 172.21.15.5:0/2943334059 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e79e705d50 0x55e79e7197d0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f88280a4380 tx=0x7f88280d7f80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=60cbd235c77dcf7f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:43:17.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:17.512+0000 7f8822000640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-09-10T09:43:17.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:17.512+0000 7f8822000640 1 -- 172.21.15.5:0/2943334059 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f882809b780 con 0x55e79e705d50 2024-09-10T09:43:17.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:17.512+0000 7f8822000640 1 -- 172.21.15.5:0/2943334059 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f8828099f40 con 0x55e79e705d50 2024-09-10T09:43:17.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:17.512+0000 7f8841e01740 1 -- 172.21.15.5:0/2943334059 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f88240119c0 con 0x55e79e705d50 2024-09-10T09:43:17.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:17.512+0000 7f8822000640 1 -- 172.21.15.5:0/2943334059 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f882809ba10 con 0x55e79e705d50 2024-09-10T09:43:17.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:17.512+0000 7f8841e01740 10 client.26371 fetch_fsmap learned FSMap version 34 2024-09-10T09:43:17.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:17.512+0000 7f8841e01740 1 -- 172.21.15.5:0/2943334059 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55e79e78bbe0 con 0x55e79e705d50 2024-09-10T09:43:17.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:17.513+0000 7f8822000640 1 -- 172.21.15.5:0/2943334059 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f88240119c0 con 0x55e79e705d50 2024-09-10T09:43:17.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:17.513+0000 7f8822000640 1 -- 172.21.15.5:0/2943334059 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f88280982b0 con 0x55e79e705d50 2024-09-10T09:43:17.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:17.513+0000 7f8841e01740 10 client.26371 fetch_fsmap finished waiting for FSMap version 34 2024-09-10T09:43:17.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:17.513+0000 7f8841e01740 10 client.26371 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T09:43:17.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:17.513+0000 7f8841e01740 20 client.26371 populate_metadata read hostname 'smithi005' 2024-09-10T09:43:17.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:17.513+0000 7f8841e01740 1 --2- 172.21.15.5:0/2943334059 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55e79e78bed0 0x55e79e78e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:43:17.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:17.513+0000 7f8841e01740 4 client.26371 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T09:43:17.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:17.513+0000 7f8841e01740 1 -- 172.21.15.5:0/2943334059 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55e79e673e60 con 0x55e79e78bed0 2024-09-10T09:43:17.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:17.514+0000 7f8837400640 1 --2- 172.21.15.5:0/2943334059 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55e79e78bed0 0x55e79e78e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:43:17.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:17.514+0000 7f8837400640 1 --2- 172.21.15.5:0/2943334059 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55e79e78bed0 0x55e79e78e2c0 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=894c804955c4c87a server_cookie=8ad9c1e5912cd6cf in_seq=0 out_seq=0 2024-09-10T09:43:17.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:17.514+0000 7f8822000640 10 client.26371 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T09:43:17.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:17.514+0000 7f8822000640 1 -- 172.21.15.5:0/2943334059 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55e79e673e60 con 0x55e79e78bed0 2024-09-10T09:43:17.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:17.514+0000 7f8822000640 10 client.26371 handle_command_reply: tid=0 2024-09-10T09:43:17.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:17.514+0000 7f8841e01740 1 client.26371 shutdown 2024-09-10T09:43:17.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:17.514+0000 7f8841e01740 1 -- 172.21.15.5:0/2943334059 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e79e705d50 msgr2=0x55e79e7197d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:43:17.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:17.514+0000 7f8841e01740 1 --2- 172.21.15.5:0/2943334059 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e79e705d50 0x55e79e7197d0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f88280a4380 tx=0x7f88280d7f80 comp rx=0 tx=0).stop 2024-09-10T09:43:17.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:17.514+0000 7f8841e01740 1 -- 172.21.15.5:0/2943334059 shutdown_connections 2024-09-10T09:43:17.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:17.514+0000 7f8841e01740 1 --2- 172.21.15.5:0/2943334059 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55e79e78bed0 0x55e79e78e2c0 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:17.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:17.515+0000 7f8841e01740 1 --2- 172.21.15.5:0/2943334059 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e79e710f30 0x55e79e715140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:17.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:17.515+0000 7f8822000640 0 client.26371 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T09:43:17.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:17.515+0000 7f8841e01740 1 --2- 172.21.15.5:0/2943334059 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e79e706670 0x55e79e714b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:17.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:17.515+0000 7f8841e01740 1 --2- 172.21.15.5:0/2943334059 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e79e705d50 0x55e79e7197d0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:17.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:17.515+0000 7f8841e01740 1 -- 172.21.15.5:0/2943334059 >> 172.21.15.5:0/2943334059 conn(0x55e79e7022e0 msgr2=0x55e79e70e600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:43:17.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:17.515+0000 7f8841e01740 1 -- 172.21.15.5:0/2943334059 shutdown_connections 2024-09-10T09:43:17.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:17.515+0000 7f8841e01740 1 -- 172.21.15.5:0/2943334059 wait complete. 2024-09-10T09:43:17.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:17.515+0000 7f8841e01740 20 client.26371 trim_cache size 0 max 16384 2024-09-10T09:43:17.520 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-10T09:43:17.521 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "no active scrubs running", 2024-09-10T09:43:17.521 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": {} 2024-09-10T09:43:17.521 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-10T09:43:17.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:17.516+0000 7f883dc00640 1 -- 172.21.15.5:0/859344980 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f881c0a2b80 msgr2=0x7f881c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:43:17.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:17.516+0000 7f883dc00640 1 --2- 172.21.15.5:0/859344980 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f881c0a2b80 0x7f881c0a5040 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f8830009430 tx=0x7f8830009820 comp rx=0 tx=0).stop 2024-09-10T09:43:17.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:17.516+0000 7f883dc00640 1 -- 172.21.15.5:0/859344980 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8838103e20 msgr2=0x7f883819a400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:43:17.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:17.516+0000 7f883dc00640 1 --2- 172.21.15.5:0/859344980 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8838103e20 0x7f883819a400 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f882800e9f0 tx=0x7f882800eeb0 comp rx=0 tx=0).stop 2024-09-10T09:43:17.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:17.516+0000 7f883dc00640 1 -- 172.21.15.5:0/859344980 shutdown_connections 2024-09-10T09:43:17.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:17.516+0000 7f883dc00640 1 --2- 172.21.15.5:0/859344980 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f881c0a2b80 0x7f881c0a5040 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:17.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:17.516+0000 7f883dc00640 1 --2- 172.21.15.5:0/859344980 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8838108f50 0x7f883819ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:17.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:17.516+0000 7f883dc00640 1 --2- 172.21.15.5:0/859344980 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f88381047d0 0x7f883819a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:17.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:17.516+0000 7f883dc00640 1 --2- 172.21.15.5:0/859344980 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8838103e20 0x7f883819a400 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:17.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:17.516+0000 7f883dc00640 1 -- 172.21.15.5:0/859344980 >> 172.21.15.5:0/859344980 conn(0x7f88380ff860 msgr2=0x7f8838100ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:43:17.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:17.516+0000 7f883dc00640 1 -- 172.21.15.5:0/859344980 shutdown_connections 2024-09-10T09:43:17.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:17.516+0000 7f883dc00640 1 -- 172.21.15.5:0/859344980 wait complete. 2024-09-10T09:43:17.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:43:17 smithi005 ceph-mon[30350]: pgmap v541: 129 pgs: 129 active+clean; 99 MiB data, 903 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 845 KiB/s wr, 110 op/s 2024-09-10T09:43:17.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:43:17 smithi159 ceph-mon[35513]: pgmap v541: 129 pgs: 129 active+clean; 99 MiB data, 903 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 845 KiB/s wr, 110 op/s 2024-09-10T09:43:17.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:43:17 smithi102 ceph-mon[35442]: pgmap v541: 129 pgs: 129 active+clean; 99 MiB data, 903 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 845 KiB/s wr, 110 op/s 2024-09-10T09:43:18.030 INFO:tasks.workunit.client.0.smithi005.stdout: 1 227778 3.25 MB/sec execute 59 sec latency 31.832 ms 2024-09-10T09:43:18.633 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-10T09:43:18.634 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T09:43:19.020 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:43:19.039 INFO:tasks.workunit.client.0.smithi005.stdout: 1 228123 3.22 MB/sec execute 60 sec latency 658.079 ms 2024-09-10T09:43:19.760 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:43:19 smithi005 ceph-mon[30350]: pgmap v542: 129 pgs: 129 active+clean; 102 MiB data, 903 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 1.3 MiB/s wr, 160 op/s 2024-09-10T09:43:19.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:19.785+0000 7f8a36c00640 1 -- 172.21.15.5:0/1123220786 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8a301079c0 msgr2=0x7f8a30107da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:43:19.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:19.785+0000 7f8a36c00640 1 --2- 172.21.15.5:0/1123220786 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8a301079c0 0x7f8a30107da0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f8a24009ca0 tx=0x7f8a2402a8c0 comp rx=0 tx=0).stop 2024-09-10T09:43:19.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:19.785+0000 7f8a36c00640 1 -- 172.21.15.5:0/1123220786 shutdown_connections 2024-09-10T09:43:19.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:19.785+0000 7f8a36c00640 1 --2- 172.21.15.5:0/1123220786 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8a30102360 0x7f8a3010e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:19.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:19.785+0000 7f8a36c00640 1 --2- 172.21.15.5:0/1123220786 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8a301019c0 0x7f8a30101e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:19.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:19.785+0000 7f8a36c00640 1 --2- 172.21.15.5:0/1123220786 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8a301079c0 0x7f8a30107da0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:19.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:19.785+0000 7f8a36c00640 1 -- 172.21.15.5:0/1123220786 >> 172.21.15.5:0/1123220786 conn(0x7f8a300fd740 msgr2=0x7f8a300ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:43:19.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:19.785+0000 7f8a36c00640 1 -- 172.21.15.5:0/1123220786 shutdown_connections 2024-09-10T09:43:19.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:19.785+0000 7f8a36c00640 1 -- 172.21.15.5:0/1123220786 wait complete. 2024-09-10T09:43:19.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:19.786+0000 7f8a36c00640 1 Processor -- start 2024-09-10T09:43:19.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:19.787+0000 7f8a36c00640 1 -- start start 2024-09-10T09:43:19.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:19.787+0000 7f8a36c00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8a301019c0 0x7f8a300744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:43:19.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:19.787+0000 7f8a36c00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8a30102360 0x7f8a30074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:43:19.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:19.787+0000 7f8a36c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8a301079c0 0x7f8a30074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:43:19.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:19.787+0000 7f8a36c00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f8a30111e10 con 0x7f8a30102360 2024-09-10T09:43:19.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:19.787+0000 7f8a36c00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f8a30111c90 con 0x7f8a301079c0 2024-09-10T09:43:19.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:19.787+0000 7f8a36c00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f8a30111f90 con 0x7f8a301019c0 2024-09-10T09:43:19.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:19.787+0000 7f8a35800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8a301019c0 0x7f8a300744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:43:19.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:19.787+0000 7f8a35800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8a301019c0 0x7f8a300744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:51198/0 (socket says 172.21.15.5:51198) 2024-09-10T09:43:19.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:19.788+0000 7f8a35800640 1 -- 172.21.15.5:0/4218821597 learned_addr learned my addr 172.21.15.5:0/4218821597 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:43:19.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:19.788+0000 7f8a34e00640 1 --2- 172.21.15.5:0/4218821597 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8a30102360 0x7f8a30074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:43:19.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:19.788+0000 7f8a36200640 1 --2- 172.21.15.5:0/4218821597 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8a301079c0 0x7f8a30074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:43:19.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:19.788+0000 7f8a35800640 1 -- 172.21.15.5:0/4218821597 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8a301079c0 msgr2=0x7f8a30074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:43:19.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:19.788+0000 7f8a35800640 1 --2- 172.21.15.5:0/4218821597 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8a301079c0 0x7f8a30074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:19.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:19.788+0000 7f8a35800640 1 -- 172.21.15.5:0/4218821597 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8a30102360 msgr2=0x7f8a30074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:43:19.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:19.788+0000 7f8a35800640 1 --2- 172.21.15.5:0/4218821597 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8a30102360 0x7f8a30074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:19.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:19.788+0000 7f8a35800640 1 -- 172.21.15.5:0/4218821597 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a301a6730 con 0x7f8a301019c0 2024-09-10T09:43:19.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:19.788+0000 7f8a36200640 1 --2- 172.21.15.5:0/4218821597 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8a301079c0 0x7f8a30074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:43:19.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:19.788+0000 7f8a34e00640 1 --2- 172.21.15.5:0/4218821597 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8a30102360 0x7f8a30074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:43:19.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:19.789+0000 7f8a35800640 1 --2- 172.21.15.5:0/4218821597 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8a301019c0 0x7f8a300744f0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f8a24005e30 tx=0x7f8a24003f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:43:19.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:19.789+0000 7f8a2e000640 1 -- 172.21.15.5:0/4218821597 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a24031650 con 0x7f8a301019c0 2024-09-10T09:43:19.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:19.789+0000 7f8a2e000640 1 -- 172.21.15.5:0/4218821597 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8a240317f0 con 0x7f8a301019c0 2024-09-10T09:43:19.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:19.789+0000 7f8a36c00640 1 -- 172.21.15.5:0/4218821597 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a301a69c0 con 0x7f8a301019c0 2024-09-10T09:43:19.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:19.790+0000 7f8a2e000640 1 -- 172.21.15.5:0/4218821597 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a2402faa0 con 0x7f8a301019c0 2024-09-10T09:43:19.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:19.790+0000 7f8a36c00640 1 -- 172.21.15.5:0/4218821597 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a301a6f50 con 0x7f8a301019c0 2024-09-10T09:43:19.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:19.791+0000 7f8a36c00640 1 -- 172.21.15.5:0/4218821597 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a30101e20 con 0x7f8a301019c0 2024-09-10T09:43:19.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:19.793+0000 7f8a2e000640 1 -- 172.21.15.5:0/4218821597 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f8a2402fc40 con 0x7f8a301019c0 2024-09-10T09:43:19.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:19.794+0000 7f8a2e000640 1 --2- 172.21.15.5:0/4218821597 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f8a180a2c00 0x7f8a180a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:43:19.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:19.794+0000 7f8a2e000640 1 -- 172.21.15.5:0/4218821597 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f8a240f0a40 con 0x7f8a301019c0 2024-09-10T09:43:19.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:19.795+0000 7f8a34e00640 1 --2- 172.21.15.5:0/4218821597 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f8a180a2c00 0x7f8a180a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:43:19.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:19.795+0000 7f8a34e00640 1 --2- 172.21.15.5:0/4218821597 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f8a180a2c00 0x7f8a180a50c0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f8a30075f80 tx=0x7f8a1c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:43:19.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:19.796+0000 7f8a2e000640 1 -- 172.21.15.5:0/4218821597 <== mon.1 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f8a240bd550 con 0x7f8a301019c0 2024-09-10T09:43:19.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:43:19 smithi159 ceph-mon[35513]: pgmap v542: 129 pgs: 129 active+clean; 102 MiB data, 903 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 1.3 MiB/s wr, 160 op/s 2024-09-10T09:43:19.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:43:19 smithi102 ceph-mon[35442]: pgmap v542: 129 pgs: 129 active+clean; 102 MiB data, 903 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 1.3 MiB/s wr, 160 op/s 2024-09-10T09:43:20.039 INFO:tasks.workunit.client.0.smithi005.stdout: 1 230277 3.35 MB/sec execute 61 sec latency 872.982 ms 2024-09-10T09:43:20.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:20.079+0000 7f8a36c00640 1 -- 172.21.15.5:0/4218821597 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8a30075a20 con 0x7f8a301019c0 2024-09-10T09:43:20.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:20.079+0000 7f8a2e000640 1 -- 172.21.15.5:0/4218821597 <== mon.1 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+11969 (secure 0 0 0) 0x7f8a240bce50 con 0x7f8a301019c0 2024-09-10T09:43:20.081 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-10T09:43:20.081 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":34,"btime":"2024-09-10T09:42:57:737230+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14454,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/2970594311","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2970594311},{"type":"v1","addr":"172.21.15.5:6835","nonce":2970594311}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14514,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6839/3616502174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":3616502174},{"type":"v1","addr":"172.21.15.5:6839","nonce":3616502174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":34,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T09:35:29.626287+0000","modified":"2024-09-10T09:42:56.961831+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24542,"mds_1":24535,"mds_2":24511,"mds_3":24506,"mds_4":24443},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14520":{"gid":14520,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/2073599842","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":2073599842},{"type":"v1","addr":"172.21.15.5:6841","nonce":2073599842}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"k","rank":4,"incarnation":26,"state":"up:active","state_seq":24,"addr":"172.21.15.102:6833/2457007427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":2457007427},{"type":"v1","addr":"172.21.15.102:6833","nonce":2457007427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/356445159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":356445159},{"type":"v1","addr":"172.21.15.159:6833","nonce":356445159}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6837/2894314507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":2894314507},{"type":"v1","addr":"172.21.15.5:6837","nonce":2894314507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.102:6835/1647549111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":1647549111},{"type":"v1","addr":"172.21.15.102:6835","nonce":1647549111}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6835/952657242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":952657242},{"type":"v1","addr":"172.21.15.159:6835","nonce":952657242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":19,"addr":"172.21.15.102:6837/2689776764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":2689776764},{"type":"v1","addr":"172.21.15.102:6837","nonce":2689776764}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.159:6837/620627639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":620627639},{"type":"v1","addr":"172.21.15.159:6837","nonce":620627639}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24535":{"gid":24535,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":18,"addr":"172.21.15.159:6839/2073739599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":2073739599},{"type":"v1","addr":"172.21.15.159:6839","nonce":2073739599}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24542":{"gid":24542,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.102:6839/335409749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6838","nonce":335409749},{"type":"v1","addr":"172.21.15.102:6839","nonce":335409749}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24542,"qdb_cluster":[24542,24535,24511,24506,24443]},"id":1}]} 2024-09-10T09:43:20.081 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 34 2024-09-10T09:43:20.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:20.083+0000 7f8a36c00640 1 -- 172.21.15.5:0/4218821597 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f8a180a2c00 msgr2=0x7f8a180a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:43:20.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:20.083+0000 7f8a36c00640 1 --2- 172.21.15.5:0/4218821597 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f8a180a2c00 0x7f8a180a50c0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f8a30075f80 tx=0x7f8a1c005f90 comp rx=0 tx=0).stop 2024-09-10T09:43:20.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:20.083+0000 7f8a36c00640 1 -- 172.21.15.5:0/4218821597 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8a301019c0 msgr2=0x7f8a300744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:43:20.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:20.083+0000 7f8a36c00640 1 --2- 172.21.15.5:0/4218821597 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8a301019c0 0x7f8a300744f0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f8a24005e30 tx=0x7f8a24003f90 comp rx=0 tx=0).stop 2024-09-10T09:43:20.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:20.083+0000 7f8a36c00640 1 -- 172.21.15.5:0/4218821597 shutdown_connections 2024-09-10T09:43:20.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:20.083+0000 7f8a36c00640 1 --2- 172.21.15.5:0/4218821597 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f8a180a2c00 0x7f8a180a50c0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:20.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:20.083+0000 7f8a36c00640 1 --2- 172.21.15.5:0/4218821597 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8a301079c0 0x7f8a30074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:20.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:20.083+0000 7f8a36c00640 1 --2- 172.21.15.5:0/4218821597 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8a30102360 0x7f8a30074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:20.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:20.083+0000 7f8a36c00640 1 --2- 172.21.15.5:0/4218821597 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8a301019c0 0x7f8a300744f0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:20.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:20.083+0000 7f8a36c00640 1 -- 172.21.15.5:0/4218821597 >> 172.21.15.5:0/4218821597 conn(0x7f8a300fd740 msgr2=0x7f8a300fef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:43:20.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:20.083+0000 7f8a36c00640 1 -- 172.21.15.5:0/4218821597 shutdown_connections 2024-09-10T09:43:20.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:20.083+0000 7f8a36c00640 1 -- 172.21.15.5:0/4218821597 wait complete. 2024-09-10T09:43:20.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:43:20 smithi005 ceph-mon[30350]: from='client.? 172.21.15.5:0/4218821597' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:43:20.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:43:20 smithi159 ceph-mon[35513]: from='client.? 172.21.15.5:0/4218821597' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:43:20.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:43:20 smithi102 ceph-mon[35442]: from='client.? 172.21.15.5:0/4218821597' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:43:21.033 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-10T09:43:21.039 INFO:tasks.workunit.client.0.smithi005.stdout: 1 232039 3.47 MB/sec execute 62 sec latency 32.873 ms 2024-09-10T09:43:21.473 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:43:21.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:43:21 smithi102 ceph-mon[35442]: pgmap v543: 129 pgs: 129 active+clean; 104 MiB data, 940 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 1.7 MiB/s wr, 189 op/s 2024-09-10T09:43:21.914 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:43:21 smithi005 ceph-mon[30350]: pgmap v543: 129 pgs: 129 active+clean; 104 MiB data, 940 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 1.7 MiB/s wr, 189 op/s 2024-09-10T09:43:22.039 INFO:tasks.workunit.client.0.smithi005.stdout: 1 234392 3.64 MB/sec execute 63 sec latency 22.794 ms 2024-09-10T09:43:22.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:22.039+0000 7f1d48a00640 1 -- 172.21.15.5:0/1126599877 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1d441034b0 msgr2=0x7f1d4410b9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:43:22.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:22.039+0000 7f1d48a00640 1 --2- 172.21.15.5:0/1126599877 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1d441034b0 0x7f1d4410b9a0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f1d30009900 tx=0x7f1d3002a490 comp rx=0 tx=0).stop 2024-09-10T09:43:22.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:22.039+0000 7f1d48a00640 1 -- 172.21.15.5:0/1126599877 shutdown_connections 2024-09-10T09:43:22.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:22.039+0000 7f1d48a00640 1 --2- 172.21.15.5:0/1126599877 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1d4410bfc0 0x7f1d4410e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:22.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:22.039+0000 7f1d48a00640 1 --2- 172.21.15.5:0/1126599877 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1d441034b0 0x7f1d4410b9a0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:22.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:22.039+0000 7f1d48a00640 1 --2- 172.21.15.5:0/1126599877 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1d44102b90 0x7f1d44102f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:22.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:22.039+0000 7f1d48a00640 1 -- 172.21.15.5:0/1126599877 >> 172.21.15.5:0/1126599877 conn(0x7f1d440fc610 msgr2=0x7f1d440fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:43:22.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:22.039+0000 7f1d48a00640 1 -- 172.21.15.5:0/1126599877 shutdown_connections 2024-09-10T09:43:22.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:22.039+0000 7f1d48a00640 1 -- 172.21.15.5:0/1126599877 wait complete. 2024-09-10T09:43:22.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:22.040+0000 7f1d48a00640 1 Processor -- start 2024-09-10T09:43:22.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:22.040+0000 7f1d48a00640 1 -- start start 2024-09-10T09:43:22.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:22.041+0000 7f1d48a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1d44102b90 0x7f1d4419a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:43:22.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:22.041+0000 7f1d48a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1d441034b0 0x7f1d4419aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:43:22.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:22.041+0000 7f1d48a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1d4410bfc0 0x7f1d4419edd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:43:22.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:22.041+0000 7f1d48a00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1d44111e10 con 0x7f1d4410bfc0 2024-09-10T09:43:22.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:22.041+0000 7f1d48a00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f1d44111c90 con 0x7f1d441034b0 2024-09-10T09:43:22.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:22.041+0000 7f1d48a00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f1d44111f90 con 0x7f1d44102b90 2024-09-10T09:43:22.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:22.041+0000 7f1d42800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1d44102b90 0x7f1d4419a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:43:22.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:22.041+0000 7f1d42800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1d44102b90 0x7f1d4419a500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:40154/0 (socket says 172.21.15.5:40154) 2024-09-10T09:43:22.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:22.041+0000 7f1d42800640 1 -- 172.21.15.5:0/3723331341 learned_addr learned my addr 172.21.15.5:0/3723331341 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:43:22.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:22.041+0000 7f1d41e00640 1 --2- 172.21.15.5:0/3723331341 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1d441034b0 0x7f1d4419aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:43:22.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:22.041+0000 7f1d43200640 1 --2- 172.21.15.5:0/3723331341 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1d4410bfc0 0x7f1d4419edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:43:22.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:22.042+0000 7f1d42800640 1 -- 172.21.15.5:0/3723331341 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1d441034b0 msgr2=0x7f1d4419aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:43:22.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:22.042+0000 7f1d42800640 1 --2- 172.21.15.5:0/3723331341 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1d441034b0 0x7f1d4419aa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:22.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:22.042+0000 7f1d42800640 1 -- 172.21.15.5:0/3723331341 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1d4410bfc0 msgr2=0x7f1d4419edd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:43:22.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:22.042+0000 7f1d42800640 1 --2- 172.21.15.5:0/3723331341 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1d4410bfc0 0x7f1d4419edd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:22.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:22.042+0000 7f1d42800640 1 -- 172.21.15.5:0/3723331341 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d4419f4b0 con 0x7f1d44102b90 2024-09-10T09:43:22.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:22.042+0000 7f1d41e00640 1 --2- 172.21.15.5:0/3723331341 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1d441034b0 0x7f1d4419aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:43:22.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:22.042+0000 7f1d42800640 1 --2- 172.21.15.5:0/3723331341 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1d44102b90 0x7f1d4419a500 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f1d3400ea60 tx=0x7f1d3400ee60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:43:22.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:22.042+0000 7f1d3b400640 1 -- 172.21.15.5:0/3723331341 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d3400cd20 con 0x7f1d44102b90 2024-09-10T09:43:22.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:22.042+0000 7f1d3b400640 1 -- 172.21.15.5:0/3723331341 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1d340035f0 con 0x7f1d44102b90 2024-09-10T09:43:22.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:22.042+0000 7f1d48a00640 1 -- 172.21.15.5:0/3723331341 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d4419f7a0 con 0x7f1d44102b90 2024-09-10T09:43:22.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:22.042+0000 7f1d3b400640 1 -- 172.21.15.5:0/3723331341 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d34010690 con 0x7f1d44102b90 2024-09-10T09:43:22.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:22.042+0000 7f1d48a00640 1 -- 172.21.15.5:0/3723331341 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d441a7080 con 0x7f1d44102b90 2024-09-10T09:43:22.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:22.046+0000 7f1d3b400640 1 -- 172.21.15.5:0/3723331341 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f1d340037a0 con 0x7f1d44102b90 2024-09-10T09:43:22.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:22.046+0000 7f1d4c04e740 1 Processor -- start 2024-09-10T09:43:22.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:22.046+0000 7f1d4c04e740 1 -- start start 2024-09-10T09:43:22.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:22.047+0000 7f1d4c04e740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561e626eee40 0x561e626ef220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:43:22.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:22.047+0000 7f1d4c04e740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561e626ef760 0x561e626f9ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:43:22.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:22.047+0000 7f1d4c04e740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561e626fa020 0x561e626fc410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:43:22.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:22.047+0000 7f1d4c04e740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x561e626553e0 con 0x561e626fa020 2024-09-10T09:43:22.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:22.047+0000 7f1d4c04e740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x561e625fcd80 con 0x561e626ef760 2024-09-10T09:43:22.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:22.047+0000 7f1d4c04e740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x561e626513a0 con 0x561e626eee40 2024-09-10T09:43:22.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:22.047+0000 7f1d43200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561e626eee40 0x561e626ef220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:43:22.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:22.047+0000 7f1d43200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561e626eee40 0x561e626ef220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:40160/0 (socket says 172.21.15.5:40160) 2024-09-10T09:43:22.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:22.047+0000 7f1d43200640 1 -- 172.21.15.5:0/1452870757 learned_addr learned my addr 172.21.15.5:0/1452870757 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:43:22.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:22.047+0000 7f1d42800640 1 --2- 172.21.15.5:0/1452870757 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561e626ef760 0x561e626f9ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:43:22.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:22.047+0000 7f1d41e00640 1 --2- 172.21.15.5:0/1452870757 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561e626fa020 0x561e626fc410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:43:22.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:22.048+0000 7f1d43200640 1 -- 172.21.15.5:0/1452870757 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561e626ef760 msgr2=0x561e626f9ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:43:22.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:22.048+0000 7f1d43200640 1 --2- 172.21.15.5:0/1452870757 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561e626ef760 0x561e626f9ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:22.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:22.048+0000 7f1d43200640 1 -- 172.21.15.5:0/1452870757 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561e626fa020 msgr2=0x561e626fc410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:43:22.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:22.048+0000 7f1d43200640 1 --2- 172.21.15.5:0/1452870757 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561e626fa020 0x561e626fc410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:22.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:22.048+0000 7f1d43200640 1 -- 172.21.15.5:0/1452870757 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561e6261bae0 con 0x561e626eee40 2024-09-10T09:43:22.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:22.048+0000 7f1d42800640 1 --2- 172.21.15.5:0/1452870757 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561e626ef760 0x561e626f9ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:43:22.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:22.048+0000 7f1d41e00640 1 --2- 172.21.15.5:0/1452870757 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561e626fa020 0x561e626fc410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:43:22.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:22.048+0000 7f1d3b400640 1 --2- 172.21.15.5:0/3723331341 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f1d280a2c00 0x7f1d280a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:43:22.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:22.048+0000 7f1d43200640 1 --2- 172.21.15.5:0/1452870757 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561e626eee40 0x561e626ef220 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f1d3c009f60 tx=0x7f1d3c009f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e263f24a15e3a1bf server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:43:22.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:22.048+0000 7f1d3b400640 1 -- 172.21.15.5:0/3723331341 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f1d340cf6f0 con 0x7f1d44102b90 2024-09-10T09:43:22.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:22.048+0000 7f1d2f400640 1 -- 172.21.15.5:0/1452870757 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d3c01e070 con 0x561e626eee40 2024-09-10T09:43:22.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:22.048+0000 7f1d2f400640 1 -- 172.21.15.5:0/1452870757 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1d3c004030 con 0x561e626eee40 2024-09-10T09:43:22.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:22.048+0000 7f1d2f400640 1 -- 172.21.15.5:0/1452870757 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d3c00f720 con 0x561e626eee40 2024-09-10T09:43:22.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:22.048+0000 7f1d42800640 1 --2- 172.21.15.5:0/3723331341 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f1d280a2c00 0x7f1d280a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:43:22.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:22.049+0000 7f1d4c04e740 1 -- 172.21.15.5:0/1452870757 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561e626eee40 msgr2=0x561e626ef220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:43:22.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:22.049+0000 7f1d42800640 1 --2- 172.21.15.5:0/3723331341 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f1d280a2c00 0x7f1d280a50c0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f1d34097f60 tx=0x7f1d340d5530 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:43:22.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:22.049+0000 7f1d4c04e740 1 --2- 172.21.15.5:0/1452870757 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561e626eee40 0x561e626ef220 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f1d3c009f60 tx=0x7f1d3c009f90 comp rx=0 tx=0).stop 2024-09-10T09:43:22.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:22.049+0000 7f1d4c04e740 1 -- 172.21.15.5:0/1452870757 shutdown_connections 2024-09-10T09:43:22.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:22.049+0000 7f1d4c04e740 1 --2- 172.21.15.5:0/1452870757 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561e626fa020 0x561e626fc410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:22.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:22.049+0000 7f1d4c04e740 1 --2- 172.21.15.5:0/1452870757 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561e626ef760 0x561e626f9ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:22.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:22.049+0000 7f1d4c04e740 1 --2- 172.21.15.5:0/1452870757 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561e626eee40 0x561e626ef220 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:22.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:22.049+0000 7f1d4c04e740 1 -- 172.21.15.5:0/1452870757 >> 172.21.15.5:0/1452870757 conn(0x561e626eb3d0 msgr2=0x561e626eb7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:43:22.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:22.049+0000 7f1d4c04e740 1 -- 172.21.15.5:0/1452870757 shutdown_connections 2024-09-10T09:43:22.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:22.050+0000 7f1d4c04e740 1 -- 172.21.15.5:0/1452870757 wait complete. 2024-09-10T09:43:22.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:22.050+0000 7f1d4c04e740 1 -- start start 2024-09-10T09:43:22.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:22.050+0000 7f1d4c04e740 1 Processor -- start 2024-09-10T09:43:22.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:22.051+0000 7f1d4c04e740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561e626eee40 0x561e626f6f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:43:22.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:22.051+0000 7f1d4c04e740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561e626ef760 0x561e626f7470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:43:22.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:22.051+0000 7f1d4c04e740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561e626fa020 0x561e626f79b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:43:22.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:22.051+0000 7f1d4c04e740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x561e626fcbf0 con 0x561e626eee40 2024-09-10T09:43:22.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:22.051+0000 7f1d4c04e740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x561e626100f0 con 0x561e626fa020 2024-09-10T09:43:22.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:22.051+0000 7f1d4c04e740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x561e626fcd70 con 0x561e626ef760 2024-09-10T09:43:22.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:22.051+0000 7f1d4c04e740 10 client.0 _ll_register_callbacks cb 0x561e625f7bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:43:22.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:22.051+0000 7f1d41e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561e626ef760 0x561e626f7470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:43:22.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:22.051+0000 7f1d41e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561e626ef760 0x561e626f7470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:40170/0 (socket says 172.21.15.5:40170) 2024-09-10T09:43:22.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:22.051+0000 7f1d41e00640 1 -- 172.21.15.5:0/3640245716 learned_addr learned my addr 172.21.15.5:0/3640245716 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:43:22.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:22.051+0000 7f1d43200640 1 --2- 172.21.15.5:0/3640245716 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561e626eee40 0x561e626f6f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:43:22.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:22.051+0000 7f1d41e00640 1 -- 172.21.15.5:0/3640245716 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561e626fa020 msgr2=0x561e626f79b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:43:22.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:22.051+0000 7f1d43200640 1 --2- 172.21.15.5:0/3640245716 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561e626fa020 0x561e626f79b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:43:22.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:22.052+0000 7f1d41e00640 1 --2- 172.21.15.5:0/3640245716 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561e626fa020 0x561e626f79b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:22.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:22.052+0000 7f1d41e00640 1 -- 172.21.15.5:0/3640245716 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561e626eee40 msgr2=0x561e626f6f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:43:22.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:22.052+0000 7f1d41e00640 1 --2- 172.21.15.5:0/3640245716 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561e626eee40 0x561e626f6f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:22.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:22.052+0000 7f1d41e00640 1 -- 172.21.15.5:0/3640245716 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561e623c63c0 con 0x561e626ef760 2024-09-10T09:43:22.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:22.052+0000 7f1d43200640 1 --2- 172.21.15.5:0/3640245716 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561e626fa020 0x561e626f79b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T09:43:22.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:22.052+0000 7f1d41e00640 1 --2- 172.21.15.5:0/3640245716 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561e626ef760 0x561e626f7470 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f1d30033000 tx=0x7f1d30005dc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6fdc34f504c268ce server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:43:22.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:22.052+0000 7f1d23e00640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-09-10T09:43:22.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:22.052+0000 7f1d23e00640 1 -- 172.21.15.5:0/3640245716 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d30035040 con 0x561e626ef760 2024-09-10T09:43:22.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:22.052+0000 7f1d23e00640 1 -- 172.21.15.5:0/3640245716 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f1d30005550 con 0x561e626ef760 2024-09-10T09:43:22.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:22.052+0000 7f1d4c04e740 1 -- 172.21.15.5:0/3640245716 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1d3c004030 con 0x561e626ef760 2024-09-10T09:43:22.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:22.053+0000 7f1d23e00640 1 -- 172.21.15.5:0/3640245716 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f1d30004760 con 0x561e626ef760 2024-09-10T09:43:22.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:22.053+0000 7f1d4c04e740 10 client.26389 fetch_fsmap learned FSMap version 34 2024-09-10T09:43:22.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:22.053+0000 7f1d4c04e740 1 -- 172.21.15.5:0/3640245716 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x561e62708c60 con 0x561e626ef760 2024-09-10T09:43:22.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:22.053+0000 7f1d23e00640 1 -- 172.21.15.5:0/3640245716 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1d3c004030 con 0x561e626ef760 2024-09-10T09:43:22.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:22.054+0000 7f1d23e00640 1 -- 172.21.15.5:0/3640245716 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f1d300035f0 con 0x561e626ef760 2024-09-10T09:43:22.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:22.054+0000 7f1d4c04e740 10 client.26389 fetch_fsmap finished waiting for FSMap version 34 2024-09-10T09:43:22.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:22.054+0000 7f1d4c04e740 10 client.26389 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-09-10T09:43:22.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:22.054+0000 7f1d4c04e740 20 client.26389 populate_metadata read hostname 'smithi005' 2024-09-10T09:43:22.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:22.054+0000 7f1d4c04e740 1 --2- 172.21.15.5:0/3640245716 >> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] conn(0x561e6270a120 0x561e6270c580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:43:22.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:22.054+0000 7f1d4c04e740 4 client.26389 mds_command: new command op to 24443 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:43:22.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:22.054+0000 7f1d4c04e740 1 -- 172.21.15.5:0/3640245716 --> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561e62662950 con 0x561e6270a120 2024-09-10T09:43:22.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:22.054+0000 7f1d43200640 1 --2- 172.21.15.5:0/3640245716 >> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] conn(0x561e6270a120 0x561e6270c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:43:22.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:22.055+0000 7f1d43200640 1 --2- 172.21.15.5:0/3640245716 >> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] conn(0x561e6270a120 0x561e6270c580 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=8bdbfdbc807128fe server_cookie=5c670418ece15f9b in_seq=0 out_seq=0 2024-09-10T09:43:22.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:22.055+0000 7f1d23e00640 10 client.26389 ms_handle_connect on v2:172.21.15.102:6832/2457007427 2024-09-10T09:43:22.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:22.056+0000 7f1d23e00640 1 -- 172.21.15.5:0/3640245716 <== mds.4 v2:172.21.15.102:6832/2457007427 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561e62662950 con 0x561e6270a120 2024-09-10T09:43:22.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:22.056+0000 7f1d23e00640 10 client.26389 handle_command_reply: tid=0 2024-09-10T09:43:22.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:22.056+0000 7f1d4c04e740 1 client.26389 shutdown 2024-09-10T09:43:22.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:22.056+0000 7f1d4c04e740 1 -- 172.21.15.5:0/3640245716 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561e626ef760 msgr2=0x561e626f7470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:43:22.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:22.056+0000 7f1d4c04e740 1 --2- 172.21.15.5:0/3640245716 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561e626ef760 0x561e626f7470 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f1d30033000 tx=0x7f1d30005dc0 comp rx=0 tx=0).stop 2024-09-10T09:43:22.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:22.056+0000 7f1d4c04e740 1 -- 172.21.15.5:0/3640245716 shutdown_connections 2024-09-10T09:43:22.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:22.056+0000 7f1d4c04e740 1 --2- 172.21.15.5:0/3640245716 >> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] conn(0x561e6270a120 0x561e6270c580 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:22.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:22.056+0000 7f1d4c04e740 1 --2- 172.21.15.5:0/3640245716 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561e626fa020 0x561e626f79b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:22.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:22.056+0000 7f1d4c04e740 1 --2- 172.21.15.5:0/3640245716 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561e626ef760 0x561e626f7470 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:22.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:22.056+0000 7f1d4c04e740 1 --2- 172.21.15.5:0/3640245716 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561e626eee40 0x561e626f6f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:22.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:22.056+0000 7f1d23e00640 0 client.26389 ms_handle_reset on v2:172.21.15.102:6832/2457007427 2024-09-10T09:43:22.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:22.056+0000 7f1d4c04e740 1 -- 172.21.15.5:0/3640245716 >> 172.21.15.5:0/3640245716 conn(0x561e626eb3d0 msgr2=0x561e626f10b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:43:22.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:22.056+0000 7f1d4c04e740 1 -- 172.21.15.5:0/3640245716 shutdown_connections 2024-09-10T09:43:22.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:22.056+0000 7f1d4c04e740 1 -- 172.21.15.5:0/3640245716 wait complete. 2024-09-10T09:43:22.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:22.057+0000 7f1d4c04e740 20 client.26389 trim_cache size 0 max 16384 2024-09-10T09:43:22.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:22.074+0000 7f1d4c04e740 1 Processor -- start 2024-09-10T09:43:22.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:22.074+0000 7f1d4c04e740 1 -- start start 2024-09-10T09:43:22.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:22.074+0000 7f1d4c04e740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561e626eee40 0x561e626f18a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:43:22.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:22.074+0000 7f1d4c04e740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561e626ef760 0x561e626f8ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:43:22.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:22.074+0000 7f1d4c04e740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561e626fa020 0x561e62711e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:43:22.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:22.074+0000 7f1d4c04e740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1d441b0fd0 con 0x561e626fa020 2024-09-10T09:43:22.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:22.074+0000 7f1d4c04e740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f1d441b2b10 con 0x561e626eee40 2024-09-10T09:43:22.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:22.074+0000 7f1d4c04e740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f1d441b16a0 con 0x561e626ef760 2024-09-10T09:43:22.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:22.075+0000 7f1d41e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561e626ef760 0x561e626f8ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:43:22.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:22.075+0000 7f1d43200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561e626eee40 0x561e626f18a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:43:22.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:22.075+0000 7f1d41e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561e626ef760 0x561e626f8ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:40186/0 (socket says 172.21.15.5:40186) 2024-09-10T09:43:22.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:22.075+0000 7f1d41e00640 1 -- 172.21.15.5:0/2906360477 learned_addr learned my addr 172.21.15.5:0/2906360477 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:43:22.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:22.075+0000 7f1d43200640 1 --2- 172.21.15.5:0/2906360477 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561e626fa020 0x561e62711e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:43:22.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:22.075+0000 7f1d41e00640 1 -- 172.21.15.5:0/2906360477 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561e626eee40 msgr2=0x561e626f18a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:43:22.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:22.075+0000 7f1d41e00640 1 --2- 172.21.15.5:0/2906360477 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561e626eee40 0x561e626f18a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:22.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:22.075+0000 7f1d41e00640 1 -- 172.21.15.5:0/2906360477 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561e626fa020 msgr2=0x561e62711e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:43:22.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:22.075+0000 7f1d41e00640 1 --2- 172.21.15.5:0/2906360477 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561e626fa020 0x561e62711e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:22.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:22.075+0000 7f1d41e00640 1 -- 172.21.15.5:0/2906360477 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561e626f98a0 con 0x561e626ef760 2024-09-10T09:43:22.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:22.075+0000 7f1d43200640 1 --2- 172.21.15.5:0/2906360477 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561e626eee40 0x561e626f18a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:43:22.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:22.076+0000 7f1d41e00640 1 --2- 172.21.15.5:0/2906360477 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561e626ef760 0x561e626f8ea0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f1d30009900 tx=0x7f1d3002a490 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8caedd55a5d11710 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:43:22.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:22.076+0000 7f1d2ea00640 1 -- 172.21.15.5:0/2906360477 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d30035030 con 0x561e626ef760 2024-09-10T09:43:22.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:22.076+0000 7f1d2ea00640 1 -- 172.21.15.5:0/2906360477 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1d30041030 con 0x561e626ef760 2024-09-10T09:43:22.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:22.076+0000 7f1d2ea00640 1 -- 172.21.15.5:0/2906360477 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d3004e030 con 0x561e626ef760 2024-09-10T09:43:22.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:22.077+0000 7f1d4c04e740 1 -- 172.21.15.5:0/2906360477 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561e626ef760 msgr2=0x561e626f8ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:43:22.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:22.077+0000 7f1d4c04e740 1 --2- 172.21.15.5:0/2906360477 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561e626ef760 0x561e626f8ea0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f1d30009900 tx=0x7f1d3002a490 comp rx=0 tx=0).stop 2024-09-10T09:43:22.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:22.077+0000 7f1d4c04e740 1 -- 172.21.15.5:0/2906360477 shutdown_connections 2024-09-10T09:43:22.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:22.077+0000 7f1d4c04e740 1 --2- 172.21.15.5:0/2906360477 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561e626fa020 0x561e62711e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:22.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:22.077+0000 7f1d4c04e740 1 --2- 172.21.15.5:0/2906360477 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561e626ef760 0x561e626f8ea0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:22.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:22.077+0000 7f1d4c04e740 1 --2- 172.21.15.5:0/2906360477 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561e626eee40 0x561e626f18a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:22.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:22.077+0000 7f1d4c04e740 1 -- 172.21.15.5:0/2906360477 >> 172.21.15.5:0/2906360477 conn(0x561e626eb3d0 msgr2=0x561e626f8790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:43:22.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:22.077+0000 7f1d4c04e740 1 -- 172.21.15.5:0/2906360477 shutdown_connections 2024-09-10T09:43:22.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:22.077+0000 7f1d4c04e740 1 -- 172.21.15.5:0/2906360477 wait complete. 2024-09-10T09:43:22.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:22.078+0000 7f1d4c04e740 1 -- start start 2024-09-10T09:43:22.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:22.078+0000 7f1d4c04e740 1 Processor -- start 2024-09-10T09:43:22.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:22.078+0000 7f1d4c04e740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561e626eee40 0x561e626fdeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:43:22.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:22.078+0000 7f1d4c04e740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561e626ef760 0x561e626fe3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:43:22.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:22.078+0000 7f1d4c04e740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561e626fa020 0x561e626fe930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:43:22.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:22.078+0000 7f1d4c04e740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1d441b2440 con 0x561e626eee40 2024-09-10T09:43:22.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:22.078+0000 7f1d4c04e740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f1d441b1d70 con 0x561e626fa020 2024-09-10T09:43:22.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:22.078+0000 7f1d4c04e740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f1d441aefb0 con 0x561e626ef760 2024-09-10T09:43:22.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:22.078+0000 7f1d4c04e740 10 client.0 _ll_register_callbacks cb 0x7f1d34005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:43:22.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:22.079+0000 7f1d41e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561e626ef760 0x561e626fe3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:43:22.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:22.079+0000 7f1d41e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561e626ef760 0x561e626fe3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:40196/0 (socket says 172.21.15.5:40196) 2024-09-10T09:43:22.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:22.079+0000 7f1d41e00640 1 -- 172.21.15.5:0/1323009216 learned_addr learned my addr 172.21.15.5:0/1323009216 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:43:22.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:22.079+0000 7f1d43200640 1 --2- 172.21.15.5:0/1323009216 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561e626eee40 0x561e626fdeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:43:22.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:22.079+0000 7f1d43200640 1 --2- 172.21.15.5:0/1323009216 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561e626fa020 0x561e626fe930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:43:22.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:22.079+0000 7f1d41e00640 1 -- 172.21.15.5:0/1323009216 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561e626fa020 msgr2=0x561e626fe930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:43:22.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:22.079+0000 7f1d41e00640 1 --2- 172.21.15.5:0/1323009216 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561e626fa020 0x561e626fe930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:22.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:22.079+0000 7f1d41e00640 1 -- 172.21.15.5:0/1323009216 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561e626eee40 msgr2=0x561e626fdeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:43:22.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:22.079+0000 7f1d41e00640 1 --2- 172.21.15.5:0/1323009216 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561e626eee40 0x561e626fdeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:22.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:22.079+0000 7f1d41e00640 1 -- 172.21.15.5:0/1323009216 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561e62773fc0 con 0x561e626ef760 2024-09-10T09:43:22.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:22.079+0000 7f1d41e00640 1 --2- 172.21.15.5:0/1323009216 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561e626ef760 0x561e626fe3f0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f1d3002a9b0 tx=0x7f1d30003f60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8165f7d5954e347d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:43:22.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:22.079+0000 7f1d23e00640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-09-10T09:43:22.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:22.080+0000 7f1d23e00640 1 -- 172.21.15.5:0/1323009216 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d30035070 con 0x561e626ef760 2024-09-10T09:43:22.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:22.080+0000 7f1d23e00640 1 -- 172.21.15.5:0/1323009216 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f1d30004900 con 0x561e626ef760 2024-09-10T09:43:22.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:22.080+0000 7f1d4c04e740 1 -- 172.21.15.5:0/1323009216 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1d30041030 con 0x561e626ef760 2024-09-10T09:43:22.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:22.080+0000 7f1d23e00640 1 -- 172.21.15.5:0/1323009216 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f1d30061050 con 0x561e626ef760 2024-09-10T09:43:22.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:22.080+0000 7f1d4c04e740 10 client.26395 fetch_fsmap learned FSMap version 34 2024-09-10T09:43:22.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:22.080+0000 7f1d4c04e740 1 -- 172.21.15.5:0/1323009216 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x561e627752a0 con 0x561e626ef760 2024-09-10T09:43:22.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:22.080+0000 7f1d23e00640 1 -- 172.21.15.5:0/1323009216 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1d30041030 con 0x561e626ef760 2024-09-10T09:43:22.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:22.081+0000 7f1d23e00640 1 -- 172.21.15.5:0/1323009216 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f1d30031880 con 0x561e626ef760 2024-09-10T09:43:22.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:22.081+0000 7f1d4c04e740 10 client.26395 fetch_fsmap finished waiting for FSMap version 34 2024-09-10T09:43:22.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:22.081+0000 7f1d4c04e740 10 client.26395 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-09-10T09:43:22.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:22.081+0000 7f1d4c04e740 20 client.26395 populate_metadata read hostname 'smithi005' 2024-09-10T09:43:22.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:22.081+0000 7f1d4c04e740 1 --2- 172.21.15.5:0/1323009216 >> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] conn(0x561e627759a0 0x561e62777dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:43:22.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:22.081+0000 7f1d4c04e740 4 client.26395 mds_command: new command op to 24443 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T09:43:22.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:22.081+0000 7f1d4c04e740 1 -- 172.21.15.5:0/1323009216 --> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561e6265ce60 con 0x561e627759a0 2024-09-10T09:43:22.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:22.081+0000 7f1d43200640 1 --2- 172.21.15.5:0/1323009216 >> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] conn(0x561e627759a0 0x561e62777dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:43:22.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:22.081+0000 7f1d43200640 1 --2- 172.21.15.5:0/1323009216 >> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] conn(0x561e627759a0 0x561e62777dc0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=20215fde5db4f46d server_cookie=f6a500c5bb1b5a5e in_seq=0 out_seq=0 2024-09-10T09:43:22.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:22.081+0000 7f1d23e00640 10 client.26395 ms_handle_connect on v2:172.21.15.102:6832/2457007427 2024-09-10T09:43:22.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:22.082+0000 7f1d23e00640 1 -- 172.21.15.5:0/1323009216 <== mds.4 v2:172.21.15.102:6832/2457007427 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561e6265ce60 con 0x561e627759a0 2024-09-10T09:43:22.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:22.082+0000 7f1d23e00640 10 client.26395 handle_command_reply: tid=0 2024-09-10T09:43:22.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:22.082+0000 7f1d4c04e740 1 client.26395 shutdown 2024-09-10T09:43:22.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:22.082+0000 7f1d4c04e740 1 -- 172.21.15.5:0/1323009216 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561e626ef760 msgr2=0x561e626fe3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:43:22.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:22.082+0000 7f1d4c04e740 1 --2- 172.21.15.5:0/1323009216 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561e626ef760 0x561e626fe3f0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f1d3002a9b0 tx=0x7f1d30003f60 comp rx=0 tx=0).stop 2024-09-10T09:43:22.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:22.082+0000 7f1d4c04e740 1 -- 172.21.15.5:0/1323009216 shutdown_connections 2024-09-10T09:43:22.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:22.082+0000 7f1d4c04e740 1 --2- 172.21.15.5:0/1323009216 >> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] conn(0x561e627759a0 0x561e62777dc0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:22.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:22.082+0000 7f1d4c04e740 1 --2- 172.21.15.5:0/1323009216 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561e626fa020 0x561e626fe930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:22.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:22.082+0000 7f1d4c04e740 1 --2- 172.21.15.5:0/1323009216 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561e626ef760 0x561e626fe3f0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:22.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:22.082+0000 7f1d4c04e740 1 --2- 172.21.15.5:0/1323009216 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561e626eee40 0x561e626fdeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:22.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:22.082+0000 7f1d4c04e740 1 -- 172.21.15.5:0/1323009216 >> 172.21.15.5:0/1323009216 conn(0x561e626eb3d0 msgr2=0x561e626f8fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:43:22.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:22.082+0000 7f1d23e00640 0 client.26395 ms_handle_reset on v2:172.21.15.102:6832/2457007427 2024-09-10T09:43:22.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:22.082+0000 7f1d4c04e740 1 -- 172.21.15.5:0/1323009216 shutdown_connections 2024-09-10T09:43:22.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:22.082+0000 7f1d4c04e740 1 -- 172.21.15.5:0/1323009216 wait complete. 2024-09-10T09:43:22.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:22.082+0000 7f1d4c04e740 20 client.26395 trim_cache size 0 max 16384 2024-09-10T09:43:22.087 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-10T09:43:22.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:22.083+0000 7f1d48a00640 1 -- 172.21.15.5:0/3723331341 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f1d280a2c00 msgr2=0x7f1d280a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:43:22.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:22.083+0000 7f1d48a00640 1 --2- 172.21.15.5:0/3723331341 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f1d280a2c00 0x7f1d280a50c0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f1d34097f60 tx=0x7f1d340d5530 comp rx=0 tx=0).stop 2024-09-10T09:43:22.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:22.083+0000 7f1d48a00640 1 -- 172.21.15.5:0/3723331341 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1d44102b90 msgr2=0x7f1d4419a500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:43:22.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:22.083+0000 7f1d48a00640 1 --2- 172.21.15.5:0/3723331341 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1d44102b90 0x7f1d4419a500 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f1d3400ea60 tx=0x7f1d3400ee60 comp rx=0 tx=0).stop 2024-09-10T09:43:22.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:22.084+0000 7f1d48a00640 1 -- 172.21.15.5:0/3723331341 shutdown_connections 2024-09-10T09:43:22.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:22.084+0000 7f1d48a00640 1 --2- 172.21.15.5:0/3723331341 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f1d280a2c00 0x7f1d280a50c0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:22.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:22.084+0000 7f1d48a00640 1 --2- 172.21.15.5:0/3723331341 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1d4410bfc0 0x7f1d4419edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:22.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:22.084+0000 7f1d48a00640 1 --2- 172.21.15.5:0/3723331341 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1d441034b0 0x7f1d4419aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:22.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:22.084+0000 7f1d48a00640 1 --2- 172.21.15.5:0/3723331341 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1d44102b90 0x7f1d4419a500 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:22.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:22.084+0000 7f1d48a00640 1 -- 172.21.15.5:0/3723331341 >> 172.21.15.5:0/3723331341 conn(0x7f1d440fc610 msgr2=0x7f1d4410d2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:43:22.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:22.084+0000 7f1d48a00640 1 -- 172.21.15.5:0/3723331341 shutdown_connections 2024-09-10T09:43:22.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:22.084+0000 7f1d48a00640 1 -- 172.21.15.5:0/3723331341 wait complete. 2024-09-10T09:43:22.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:43:21 smithi159 ceph-mon[35513]: pgmap v543: 129 pgs: 129 active+clean; 104 MiB data, 940 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 1.7 MiB/s wr, 189 op/s 2024-09-10T09:43:23.040 INFO:tasks.workunit.client.0.smithi005.stdout: 1 235806 3.67 MB/sec execute 64 sec latency 85.485 ms 2024-09-10T09:43:23.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:43:23 smithi102 ceph-mon[35442]: pgmap v544: 129 pgs: 129 active+clean; 105 MiB data, 982 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 1.9 MiB/s wr, 210 op/s 2024-09-10T09:43:23.946 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:43:23 smithi005 ceph-mon[30350]: pgmap v544: 129 pgs: 129 active+clean; 105 MiB data, 982 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 1.9 MiB/s wr, 210 op/s 2024-09-10T09:43:23.989 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-10T09:43:24.039 INFO:tasks.workunit.client.0.smithi005.stdout: 1 237800 3.80 MB/sec execute 65 sec latency 220.093 ms 2024-09-10T09:43:24.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:43:23 smithi159 ceph-mon[35513]: pgmap v544: 129 pgs: 129 active+clean; 105 MiB data, 982 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 1.9 MiB/s wr, 210 op/s 2024-09-10T09:43:24.465 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:43:25.040 INFO:tasks.workunit.client.0.smithi005.stdout: 1 239401 3.85 MB/sec execute 66 sec latency 35.194 ms 2024-09-10T09:43:25.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:25.230+0000 7f7650000640 1 -- 172.21.15.5:0/2669531630 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f76480ffc00 msgr2=0x7f7648100060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:43:25.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:25.230+0000 7f7650000640 1 --2- 172.21.15.5:0/2669531630 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f76480ffc00 0x7f7648100060 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f763400fb40 tx=0x7f76340329b0 comp rx=0 tx=0).stop 2024-09-10T09:43:25.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:25.231+0000 7f7650000640 1 -- 172.21.15.5:0/2669531630 shutdown_connections 2024-09-10T09:43:25.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:25.231+0000 7f7650000640 1 --2- 172.21.15.5:0/2669531630 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f76481005a0 0x7f764810ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:25.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:25.231+0000 7f7650000640 1 --2- 172.21.15.5:0/2669531630 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f76480ffc00 0x7f7648100060 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:25.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:25.231+0000 7f7650000640 1 --2- 172.21.15.5:0/2669531630 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7648106b80 0x7f7648106f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:25.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:25.231+0000 7f7650000640 1 -- 172.21.15.5:0/2669531630 >> 172.21.15.5:0/2669531630 conn(0x7f76480fba90 msgr2=0x7f76480fdeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:43:25.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:25.232+0000 7f7650000640 1 -- 172.21.15.5:0/2669531630 shutdown_connections 2024-09-10T09:43:25.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:25.233+0000 7f7650000640 1 -- 172.21.15.5:0/2669531630 wait complete. 2024-09-10T09:43:25.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:25.234+0000 7f7650000640 1 Processor -- start 2024-09-10T09:43:25.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:25.234+0000 7f7650000640 1 -- start start 2024-09-10T09:43:25.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:25.235+0000 7f7650000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f76480ffc00 0x7f76481a7200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:43:25.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:25.235+0000 7f7650000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f76481005a0 0x7f76481a7740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:43:25.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:25.235+0000 7f7650000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7648106b80 0x7f76481abad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:43:25.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:25.235+0000 7f7650000640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f764810fe00 con 0x7f76480ffc00 2024-09-10T09:43:25.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:25.235+0000 7f7650000640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f764810fc80 con 0x7f76481005a0 2024-09-10T09:43:25.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:25.235+0000 7f7650000640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f764810ff80 con 0x7f7648106b80 2024-09-10T09:43:25.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:25.236+0000 7f764cc00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f76481005a0 0x7f76481a7740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:43:25.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:25.236+0000 7f764cc00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f76481005a0 0x7f76481a7740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:33560/0 (socket says 172.21.15.5:33560) 2024-09-10T09:43:25.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:25.236+0000 7f764cc00640 1 -- 172.21.15.5:0/132934384 learned_addr learned my addr 172.21.15.5:0/132934384 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:43:25.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:25.236+0000 7f764cc00640 1 -- 172.21.15.5:0/132934384 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7648106b80 msgr2=0x7f76481abad0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T09:43:25.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:25.236+0000 7f764cc00640 1 --2- 172.21.15.5:0/132934384 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7648106b80 0x7f76481abad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:25.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:25.236+0000 7f764cc00640 1 -- 172.21.15.5:0/132934384 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f76480ffc00 msgr2=0x7f76481a7200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:43:25.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:25.236+0000 7f764cc00640 1 --2- 172.21.15.5:0/132934384 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f76480ffc00 0x7f76481a7200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:25.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:25.236+0000 7f764cc00640 1 -- 172.21.15.5:0/132934384 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76481ac1b0 con 0x7f76481005a0 2024-09-10T09:43:25.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:25.237+0000 7f764cc00640 1 --2- 172.21.15.5:0/132934384 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f76481005a0 0x7f76481a7740 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f7634032fa0 tx=0x7f76340052d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:43:25.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:25.237+0000 7f7646000640 1 -- 172.21.15.5:0/132934384 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f763400f3e0 con 0x7f76481005a0 2024-09-10T09:43:25.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:25.237+0000 7f7650000640 1 -- 172.21.15.5:0/132934384 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76481ac440 con 0x7f76481005a0 2024-09-10T09:43:25.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:25.237+0000 7f7646000640 1 -- 172.21.15.5:0/132934384 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f763400f580 con 0x7f76481005a0 2024-09-10T09:43:25.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:25.238+0000 7f7650000640 1 -- 172.21.15.5:0/132934384 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76481b3d80 con 0x7f76481005a0 2024-09-10T09:43:25.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:25.238+0000 7f7646000640 1 -- 172.21.15.5:0/132934384 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f763400e9f0 con 0x7f76481005a0 2024-09-10T09:43:25.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:25.243+0000 7f7652aac740 1 Processor -- start 2024-09-10T09:43:25.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:25.245+0000 7f7652aac740 1 -- start start 2024-09-10T09:43:25.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:25.246+0000 7f7646000640 1 -- 172.21.15.5:0/132934384 <== mon.2 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f763403f020 con 0x7f76481005a0 2024-09-10T09:43:25.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:25.246+0000 7f7652aac740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562b119cdc70 0x562b119ce050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:43:25.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:25.246+0000 7f7652aac740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562b119ce590 0x562b119d8910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:43:25.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:25.246+0000 7f7652aac740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562b119d8e50 0x562b119db240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:43:25.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:25.246+0000 7f764e000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562b119cdc70 0x562b119ce050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:43:25.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:25.246+0000 7f764e000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562b119cdc70 0x562b119ce050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:33562/0 (socket says 172.21.15.5:33562) 2024-09-10T09:43:25.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:25.246+0000 7f7652aac740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x562b119343e0 con 0x562b119d8e50 2024-09-10T09:43:25.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:25.247+0000 7f764e000640 1 -- 172.21.15.5:0/1319486771 learned_addr learned my addr 172.21.15.5:0/1319486771 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:43:25.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:25.247+0000 7f7652aac740 1 -- 172.21.15.5:0/1319486771 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x562b118dbd80 con 0x562b119cdc70 2024-09-10T09:43:25.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:25.247+0000 7f7652aac740 1 -- 172.21.15.5:0/1319486771 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x562b119303a0 con 0x562b119ce590 2024-09-10T09:43:25.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:25.247+0000 7f764d600640 1 --2- 172.21.15.5:0/1319486771 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562b119ce590 0x562b119d8910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:43:25.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:25.247+0000 7f764d600640 1 -- 172.21.15.5:0/1319486771 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562b119cdc70 msgr2=0x562b119ce050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:43:25.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:25.248+0000 7f764d600640 1 --2- 172.21.15.5:0/1319486771 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562b119cdc70 0x562b119ce050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:25.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:25.248+0000 7f764d600640 1 -- 172.21.15.5:0/1319486771 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562b119d8e50 msgr2=0x562b119db240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:43:25.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:25.248+0000 7f764d600640 1 --2- 172.21.15.5:0/1319486771 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562b119d8e50 0x562b119db240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:25.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:25.248+0000 7f764d600640 1 -- 172.21.15.5:0/1319486771 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562b118faae0 con 0x562b119ce590 2024-09-10T09:43:25.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:25.248+0000 7f764e000640 1 --2- 172.21.15.5:0/1319486771 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562b119cdc70 0x562b119ce050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T09:43:25.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:25.248+0000 7f764d600640 1 --2- 172.21.15.5:0/1319486771 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562b119ce590 0x562b119d8910 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f763c00ba40 tx=0x7f763c00bf00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2f342b185722b24d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:43:25.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:25.249+0000 7f763a600640 1 -- 172.21.15.5:0/1319486771 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f763c00dc40 con 0x562b119ce590 2024-09-10T09:43:25.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:25.249+0000 7f763a600640 1 -- 172.21.15.5:0/1319486771 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f763c00dde0 con 0x562b119ce590 2024-09-10T09:43:25.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:25.249+0000 7f763a600640 1 -- 172.21.15.5:0/1319486771 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f763c004a20 con 0x562b119ce590 2024-09-10T09:43:25.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:25.249+0000 7f7646000640 1 --2- 172.21.15.5:0/132934384 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f76300a2c80 0x7f76300a5140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:43:25.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:25.250+0000 7f764e000640 1 --2- 172.21.15.5:0/132934384 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f76300a2c80 0x7f76300a5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:43:25.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:25.250+0000 7f7646000640 1 -- 172.21.15.5:0/132934384 <== mon.2 v2:172.21.15.102:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f76340eeeb0 con 0x7f76481005a0 2024-09-10T09:43:25.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:25.251+0000 7f7652aac740 1 -- 172.21.15.5:0/1319486771 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562b119ce590 msgr2=0x562b119d8910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:43:25.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:25.251+0000 7f764e000640 1 --2- 172.21.15.5:0/132934384 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f76300a2c80 0x7f76300a5140 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f7640009fc0 tx=0x7f7640009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:43:25.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:25.251+0000 7f7652aac740 1 --2- 172.21.15.5:0/1319486771 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562b119ce590 0x562b119d8910 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f763c00ba40 tx=0x7f763c00bf00 comp rx=0 tx=0).stop 2024-09-10T09:43:25.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:25.251+0000 7f7652aac740 1 -- 172.21.15.5:0/1319486771 shutdown_connections 2024-09-10T09:43:25.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:25.251+0000 7f7652aac740 1 --2- 172.21.15.5:0/1319486771 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562b119d8e50 0x562b119db240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:25.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:25.251+0000 7f7652aac740 1 --2- 172.21.15.5:0/1319486771 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562b119ce590 0x562b119d8910 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:25.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:25.251+0000 7f7652aac740 1 --2- 172.21.15.5:0/1319486771 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562b119cdc70 0x562b119ce050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:25.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:25.251+0000 7f7652aac740 1 -- 172.21.15.5:0/1319486771 >> 172.21.15.5:0/1319486771 conn(0x562b119ca200 msgr2=0x562b119ca5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:43:25.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:25.252+0000 7f7652aac740 1 -- 172.21.15.5:0/1319486771 shutdown_connections 2024-09-10T09:43:25.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:25.252+0000 7f7652aac740 1 -- 172.21.15.5:0/1319486771 wait complete. 2024-09-10T09:43:25.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:25.252+0000 7f7652aac740 1 -- start start 2024-09-10T09:43:25.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:25.253+0000 7f7652aac740 1 Processor -- start 2024-09-10T09:43:25.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:25.253+0000 7f7652aac740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562b119cdc70 0x562b119d5d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:43:25.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:25.253+0000 7f7652aac740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562b119ce590 0x562b119d6290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:43:25.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:25.253+0000 7f7652aac740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562b119d8e50 0x562b119d67d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:43:25.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:25.253+0000 7f7652aac740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x562b119dba20 con 0x562b119cdc70 2024-09-10T09:43:25.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:25.253+0000 7f7652aac740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x562b118ef0f0 con 0x562b119ce590 2024-09-10T09:43:25.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:25.253+0000 7f7652aac740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x562b119dbba0 con 0x562b119d8e50 2024-09-10T09:43:25.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:25.253+0000 7f7652aac740 10 client.0 _ll_register_callbacks cb 0x562b118d6bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:43:25.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:25.254+0000 7f764e000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562b119d8e50 0x562b119d67d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:43:25.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:25.254+0000 7f764e000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562b119d8e50 0x562b119d67d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:40226/0 (socket says 172.21.15.5:40226) 2024-09-10T09:43:25.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:25.254+0000 7f764e000640 1 -- 172.21.15.5:0/477446568 learned_addr learned my addr 172.21.15.5:0/477446568 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:43:25.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:25.254+0000 7f764cc00640 1 --2- 172.21.15.5:0/477446568 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562b119ce590 0x562b119d6290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:43:25.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:25.254+0000 7f764d600640 1 --2- 172.21.15.5:0/477446568 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562b119cdc70 0x562b119d5d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:43:25.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:25.254+0000 7f764e000640 1 -- 172.21.15.5:0/477446568 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562b119ce590 msgr2=0x562b119d6290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:43:25.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:25.254+0000 7f764e000640 1 --2- 172.21.15.5:0/477446568 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562b119ce590 0x562b119d6290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:25.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:25.254+0000 7f764e000640 1 -- 172.21.15.5:0/477446568 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562b119cdc70 msgr2=0x562b119d5d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:43:25.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:25.254+0000 7f764e000640 1 --2- 172.21.15.5:0/477446568 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562b119cdc70 0x562b119d5d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:25.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:25.254+0000 7f764e000640 1 -- 172.21.15.5:0/477446568 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562b116a53c0 con 0x562b119d8e50 2024-09-10T09:43:25.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:25.255+0000 7f764e000640 1 --2- 172.21.15.5:0/477446568 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562b119d8e50 0x562b119d67d0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f7640010010 tx=0x7f7640004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f418b5ec34406862 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:43:25.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:25.255+0000 7f762ea00640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-09-10T09:43:25.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:25.255+0000 7f762ea00640 1 -- 172.21.15.5:0/477446568 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7640022070 con 0x562b119d8e50 2024-09-10T09:43:25.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:25.255+0000 7f762ea00640 1 -- 172.21.15.5:0/477446568 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f76400109a0 con 0x562b119d8e50 2024-09-10T09:43:25.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:25.255+0000 7f7652aac740 1 -- 172.21.15.5:0/477446568 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f763c00dde0 con 0x562b119d8e50 2024-09-10T09:43:25.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:25.255+0000 7f762ea00640 1 -- 172.21.15.5:0/477446568 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f7640010ba0 con 0x562b119d8e50 2024-09-10T09:43:25.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:25.255+0000 7f7652aac740 10 client.26404 fetch_fsmap learned FSMap version 34 2024-09-10T09:43:25.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:25.255+0000 7f7652aac740 1 -- 172.21.15.5:0/477446568 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x562b119e7ad0 con 0x562b119d8e50 2024-09-10T09:43:25.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:25.255+0000 7f762ea00640 1 -- 172.21.15.5:0/477446568 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f763c00dde0 con 0x562b119d8e50 2024-09-10T09:43:25.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:25.256+0000 7f762ea00640 1 -- 172.21.15.5:0/477446568 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f76400045c0 con 0x562b119d8e50 2024-09-10T09:43:25.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:25.256+0000 7f7652aac740 10 client.26404 fetch_fsmap finished waiting for FSMap version 34 2024-09-10T09:43:25.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:25.256+0000 7f7652aac740 10 client.26404 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-09-10T09:43:25.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:25.256+0000 7f7652aac740 20 client.26404 populate_metadata read hostname 'smithi005' 2024-09-10T09:43:25.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:25.256+0000 7f7652aac740 1 --2- 172.21.15.5:0/477446568 >> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] conn(0x562b119e8f80 0x562b119eb3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:43:25.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:25.256+0000 7f7652aac740 4 client.26404 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:43:25.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:25.256+0000 7f7652aac740 1 -- 172.21.15.5:0/477446568 --> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562b11941950 con 0x562b119e8f80 2024-09-10T09:43:25.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:25.256+0000 7f764d600640 1 --2- 172.21.15.5:0/477446568 >> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] conn(0x562b119e8f80 0x562b119eb3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:43:25.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:25.257+0000 7f764d600640 1 --2- 172.21.15.5:0/477446568 >> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] conn(0x562b119e8f80 0x562b119eb3a0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=a46453755bb7b4a2 server_cookie=ff8c99ce515cf975 in_seq=0 out_seq=0 2024-09-10T09:43:25.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:25.257+0000 7f762ea00640 10 client.26404 ms_handle_connect on v2:172.21.15.102:6836/2689776764 2024-09-10T09:43:25.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:25.258+0000 7f762ea00640 1 -- 172.21.15.5:0/477446568 <== mds.3 v2:172.21.15.102:6836/2689776764 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562b11941950 con 0x562b119e8f80 2024-09-10T09:43:25.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:25.258+0000 7f762ea00640 10 client.26404 handle_command_reply: tid=0 2024-09-10T09:43:25.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:25.258+0000 7f7652aac740 1 client.26404 shutdown 2024-09-10T09:43:25.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:25.258+0000 7f7652aac740 1 -- 172.21.15.5:0/477446568 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562b119d8e50 msgr2=0x562b119d67d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:43:25.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:25.258+0000 7f7652aac740 1 --2- 172.21.15.5:0/477446568 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562b119d8e50 0x562b119d67d0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f7640010010 tx=0x7f7640004a40 comp rx=0 tx=0).stop 2024-09-10T09:43:25.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:25.258+0000 7f7652aac740 1 -- 172.21.15.5:0/477446568 shutdown_connections 2024-09-10T09:43:25.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:25.258+0000 7f7652aac740 1 --2- 172.21.15.5:0/477446568 >> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] conn(0x562b119e8f80 0x562b119eb3a0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:25.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:25.258+0000 7f7652aac740 1 --2- 172.21.15.5:0/477446568 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562b119d8e50 0x562b119d67d0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:25.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:25.258+0000 7f7652aac740 1 --2- 172.21.15.5:0/477446568 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562b119ce590 0x562b119d6290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:25.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:25.258+0000 7f7652aac740 1 --2- 172.21.15.5:0/477446568 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562b119cdc70 0x562b119d5d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:25.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:25.258+0000 7f7652aac740 1 -- 172.21.15.5:0/477446568 >> 172.21.15.5:0/477446568 conn(0x562b119ca200 msgr2=0x562b119c9380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:43:25.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:25.258+0000 7f762ea00640 0 client.26404 ms_handle_reset on v2:172.21.15.102:6836/2689776764 2024-09-10T09:43:25.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:25.259+0000 7f7652aac740 1 -- 172.21.15.5:0/477446568 shutdown_connections 2024-09-10T09:43:25.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:25.259+0000 7f7652aac740 1 -- 172.21.15.5:0/477446568 wait complete. 2024-09-10T09:43:25.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:25.259+0000 7f7652aac740 20 client.26404 trim_cache size 0 max 16384 2024-09-10T09:43:25.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:25.277+0000 7f7652aac740 1 Processor -- start 2024-09-10T09:43:25.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:25.278+0000 7f7652aac740 1 -- start start 2024-09-10T09:43:25.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:25.278+0000 7f7652aac740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562b119cdc70 0x562b119f0c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:43:25.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:25.278+0000 7f7652aac740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562b119ce590 0x562b119f11b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:43:25.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:25.278+0000 7f7652aac740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562b119d8e50 0x562b119d2ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:43:25.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:25.278+0000 7f7652aac740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f762800fd20 con 0x562b119ce590 2024-09-10T09:43:25.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:25.278+0000 7f7652aac740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f7628011860 con 0x562b119d8e50 2024-09-10T09:43:25.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:25.278+0000 7f7652aac740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f76280103f0 con 0x562b119cdc70 2024-09-10T09:43:25.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:25.278+0000 7f764d600640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562b119cdc70 0x562b119f0c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:43:25.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:25.278+0000 7f764e000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562b119d8e50 0x562b119d2ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:43:25.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:25.278+0000 7f764e000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562b119d8e50 0x562b119d2ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:33576/0 (socket says 172.21.15.5:33576) 2024-09-10T09:43:25.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:25.278+0000 7f764e000640 1 -- 172.21.15.5:0/2815181226 learned_addr learned my addr 172.21.15.5:0/2815181226 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:43:25.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:25.278+0000 7f764d600640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562b119cdc70 0x562b119f0c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:40240/0 (socket says 172.21.15.5:40240) 2024-09-10T09:43:25.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:25.278+0000 7f764cc00640 1 --2- 172.21.15.5:0/2815181226 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562b119ce590 0x562b119f11b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:43:25.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:25.279+0000 7f764e000640 1 -- 172.21.15.5:0/2815181226 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562b119cdc70 msgr2=0x562b119f0c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:43:25.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:25.279+0000 7f764e000640 1 --2- 172.21.15.5:0/2815181226 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562b119cdc70 0x562b119f0c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:25.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:25.279+0000 7f764e000640 1 -- 172.21.15.5:0/2815181226 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562b119ce590 msgr2=0x562b119f11b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:43:25.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:25.279+0000 7f764e000640 1 --2- 172.21.15.5:0/2815181226 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562b119ce590 0x562b119f11b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:25.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:25.279+0000 7f764e000640 1 -- 172.21.15.5:0/2815181226 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562b119cb390 con 0x562b119d8e50 2024-09-10T09:43:25.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:25.279+0000 7f764e000640 1 --2- 172.21.15.5:0/2815181226 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562b119d8e50 0x562b119d2ff0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f7640010f50 tx=0x7f7640010f80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c21969991c6528d0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:43:25.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:25.279+0000 7f7639c00640 1 -- 172.21.15.5:0/2815181226 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7640037030 con 0x562b119d8e50 2024-09-10T09:43:25.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:25.279+0000 7f7639c00640 1 -- 172.21.15.5:0/2815181226 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f76400107a0 con 0x562b119d8e50 2024-09-10T09:43:25.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:25.279+0000 7f7639c00640 1 -- 172.21.15.5:0/2815181226 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7640022040 con 0x562b119d8e50 2024-09-10T09:43:25.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:25.280+0000 7f7652aac740 1 -- 172.21.15.5:0/2815181226 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562b119d8e50 msgr2=0x562b119d2ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:43:25.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:25.280+0000 7f7652aac740 1 --2- 172.21.15.5:0/2815181226 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562b119d8e50 0x562b119d2ff0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f7640010f50 tx=0x7f7640010f80 comp rx=0 tx=0).stop 2024-09-10T09:43:25.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:25.280+0000 7f7652aac740 1 -- 172.21.15.5:0/2815181226 shutdown_connections 2024-09-10T09:43:25.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:25.280+0000 7f7652aac740 1 --2- 172.21.15.5:0/2815181226 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562b119d8e50 0x562b119d2ff0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:25.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:25.280+0000 7f7652aac740 1 --2- 172.21.15.5:0/2815181226 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562b119ce590 0x562b119f11b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:25.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:25.280+0000 7f7652aac740 1 --2- 172.21.15.5:0/2815181226 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562b119cdc70 0x562b119f0c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:25.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:25.280+0000 7f7652aac740 1 -- 172.21.15.5:0/2815181226 >> 172.21.15.5:0/2815181226 conn(0x562b119ca200 msgr2=0x562b119d7dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:43:25.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:25.280+0000 7f7652aac740 1 -- 172.21.15.5:0/2815181226 shutdown_connections 2024-09-10T09:43:25.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:25.280+0000 7f7652aac740 1 -- 172.21.15.5:0/2815181226 wait complete. 2024-09-10T09:43:25.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:25.281+0000 7f7652aac740 1 -- start start 2024-09-10T09:43:25.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:25.281+0000 7f7652aac740 1 Processor -- start 2024-09-10T09:43:25.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:25.282+0000 7f7652aac740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562b119cdc70 0x562b119e0e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:43:25.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:25.282+0000 7f7652aac740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562b119ce590 0x562b119e13c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:43:25.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:25.282+0000 7f7652aac740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562b119d8e50 0x562b119dde50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:43:25.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:25.282+0000 7f7652aac740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f7628011190 con 0x562b119cdc70 2024-09-10T09:43:25.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:25.282+0000 7f7652aac740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f7628010ac0 con 0x562b119ce590 2024-09-10T09:43:25.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:25.282+0000 7f7652aac740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f762800dd00 con 0x562b119d8e50 2024-09-10T09:43:25.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:25.282+0000 7f7652aac740 10 client.0 _ll_register_callbacks cb 0x7f7640015070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:43:25.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:25.282+0000 7f764cc00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562b119ce590 0x562b119e13c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:43:25.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:25.282+0000 7f764cc00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562b119ce590 0x562b119e13c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:33590/0 (socket says 172.21.15.5:33590) 2024-09-10T09:43:25.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:25.282+0000 7f764cc00640 1 -- 172.21.15.5:0/3716572953 learned_addr learned my addr 172.21.15.5:0/3716572953 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:43:25.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:25.283+0000 7f764e000640 1 --2- 172.21.15.5:0/3716572953 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562b119d8e50 0x562b119dde50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:43:25.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:25.283+0000 7f764d600640 1 --2- 172.21.15.5:0/3716572953 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562b119cdc70 0x562b119e0e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:43:25.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:25.283+0000 7f764cc00640 1 -- 172.21.15.5:0/3716572953 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562b119d8e50 msgr2=0x562b119dde50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:43:25.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:25.283+0000 7f764cc00640 1 --2- 172.21.15.5:0/3716572953 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562b119d8e50 0x562b119dde50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:25.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:25.283+0000 7f764cc00640 1 -- 172.21.15.5:0/3716572953 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562b119cdc70 msgr2=0x562b119e0e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:43:25.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:25.283+0000 7f764cc00640 1 --2- 172.21.15.5:0/3716572953 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562b119cdc70 0x562b119e0e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:25.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:25.283+0000 7f764cc00640 1 -- 172.21.15.5:0/3716572953 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562b119e1a30 con 0x562b119ce590 2024-09-10T09:43:25.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:25.283+0000 7f764cc00640 1 --2- 172.21.15.5:0/3716572953 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562b119ce590 0x562b119e13c0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f76340bb2e0 tx=0x7f76340bad70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a02b3751e9cf638a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:43:25.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:25.283+0000 7f762ea00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-09-10T09:43:25.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:25.283+0000 7f762ea00640 1 -- 172.21.15.5:0/3716572953 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76340bb350 con 0x562b119ce590 2024-09-10T09:43:25.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:25.283+0000 7f762ea00640 1 -- 172.21.15.5:0/3716572953 <== mon.2 v2:172.21.15.102:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f76340c3aa0 con 0x562b119ce590 2024-09-10T09:43:25.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:25.283+0000 7f7652aac740 1 -- 172.21.15.5:0/3716572953 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f76400107a0 con 0x562b119ce590 2024-09-10T09:43:25.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:25.284+0000 7f762ea00640 1 -- 172.21.15.5:0/3716572953 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f76340bf700 con 0x562b119ce590 2024-09-10T09:43:25.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:25.284+0000 7f7652aac740 10 client.26384 fetch_fsmap learned FSMap version 34 2024-09-10T09:43:25.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:25.284+0000 7f7652aac740 1 -- 172.21.15.5:0/3716572953 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x562b11a54220 con 0x562b119ce590 2024-09-10T09:43:25.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:25.284+0000 7f762ea00640 1 -- 172.21.15.5:0/3716572953 <== mon.2 v2:172.21.15.102:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f76400107a0 con 0x562b119ce590 2024-09-10T09:43:25.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:25.284+0000 7f762ea00640 1 -- 172.21.15.5:0/3716572953 <== mon.2 v2:172.21.15.102:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f7634018530 con 0x562b119ce590 2024-09-10T09:43:25.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:25.284+0000 7f7652aac740 10 client.26384 fetch_fsmap finished waiting for FSMap version 34 2024-09-10T09:43:25.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:25.284+0000 7f7652aac740 10 client.26384 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-09-10T09:43:25.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:25.284+0000 7f7652aac740 20 client.26384 populate_metadata read hostname 'smithi005' 2024-09-10T09:43:25.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:25.284+0000 7f7652aac740 1 --2- 172.21.15.5:0/3716572953 >> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] conn(0x562b11a549a0 0x562b11a56dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:43:25.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:25.284+0000 7f7652aac740 4 client.26384 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T09:43:25.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:25.284+0000 7f7652aac740 1 -- 172.21.15.5:0/3716572953 --> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562b1193be60 con 0x562b11a549a0 2024-09-10T09:43:25.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:25.285+0000 7f764d600640 1 --2- 172.21.15.5:0/3716572953 >> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] conn(0x562b11a549a0 0x562b11a56dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:43:25.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:25.285+0000 7f764d600640 1 --2- 172.21.15.5:0/3716572953 >> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] conn(0x562b11a549a0 0x562b11a56dc0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=f24b0072eaecc4e9 server_cookie=cd2cfc27d40cbe83 in_seq=0 out_seq=0 2024-09-10T09:43:25.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:25.285+0000 7f762ea00640 10 client.26384 ms_handle_connect on v2:172.21.15.102:6836/2689776764 2024-09-10T09:43:25.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:25.285+0000 7f762ea00640 1 -- 172.21.15.5:0/3716572953 <== mds.3 v2:172.21.15.102:6836/2689776764 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562b1193be60 con 0x562b11a549a0 2024-09-10T09:43:25.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:25.285+0000 7f762ea00640 10 client.26384 handle_command_reply: tid=0 2024-09-10T09:43:25.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:25.285+0000 7f7652aac740 1 client.26384 shutdown 2024-09-10T09:43:25.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:25.286+0000 7f7652aac740 1 -- 172.21.15.5:0/3716572953 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562b119ce590 msgr2=0x562b119e13c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:43:25.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:25.286+0000 7f7652aac740 1 --2- 172.21.15.5:0/3716572953 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562b119ce590 0x562b119e13c0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f76340bb2e0 tx=0x7f76340bad70 comp rx=0 tx=0).stop 2024-09-10T09:43:25.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:25.286+0000 7f7652aac740 1 -- 172.21.15.5:0/3716572953 shutdown_connections 2024-09-10T09:43:25.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:25.286+0000 7f7652aac740 1 --2- 172.21.15.5:0/3716572953 >> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] conn(0x562b11a549a0 0x562b11a56dc0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:25.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:25.286+0000 7f7652aac740 1 --2- 172.21.15.5:0/3716572953 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562b119d8e50 0x562b119dde50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:25.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:25.286+0000 7f7652aac740 1 --2- 172.21.15.5:0/3716572953 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562b119ce590 0x562b119e13c0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:25.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:25.286+0000 7f7652aac740 1 --2- 172.21.15.5:0/3716572953 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562b119cdc70 0x562b119e0e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:25.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:25.286+0000 7f762ea00640 0 client.26384 ms_handle_reset on v2:172.21.15.102:6836/2689776764 2024-09-10T09:43:25.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:25.286+0000 7f7652aac740 1 -- 172.21.15.5:0/3716572953 >> 172.21.15.5:0/3716572953 conn(0x562b119ca200 msgr2=0x562b119cf6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:43:25.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:25.286+0000 7f7652aac740 1 -- 172.21.15.5:0/3716572953 shutdown_connections 2024-09-10T09:43:25.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:25.286+0000 7f7652aac740 1 -- 172.21.15.5:0/3716572953 wait complete. 2024-09-10T09:43:25.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:25.286+0000 7f7652aac740 20 client.26384 trim_cache size 0 max 16384 2024-09-10T09:43:25.289 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-10T09:43:25.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:25.288+0000 7f7650000640 1 -- 172.21.15.5:0/132934384 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f76300a2c80 msgr2=0x7f76300a5140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:43:25.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:25.288+0000 7f7650000640 1 --2- 172.21.15.5:0/132934384 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f76300a2c80 0x7f76300a5140 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f7640009fc0 tx=0x7f7640009500 comp rx=0 tx=0).stop 2024-09-10T09:43:25.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:25.288+0000 7f7650000640 1 -- 172.21.15.5:0/132934384 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f76481005a0 msgr2=0x7f76481a7740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:43:25.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:25.288+0000 7f7650000640 1 --2- 172.21.15.5:0/132934384 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f76481005a0 0x7f76481a7740 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f7634032fa0 tx=0x7f76340052d0 comp rx=0 tx=0).stop 2024-09-10T09:43:25.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:25.288+0000 7f7650000640 1 -- 172.21.15.5:0/132934384 shutdown_connections 2024-09-10T09:43:25.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:25.288+0000 7f7650000640 1 --2- 172.21.15.5:0/132934384 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f76300a2c80 0x7f76300a5140 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:25.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:25.288+0000 7f7650000640 1 --2- 172.21.15.5:0/132934384 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7648106b80 0x7f76481abad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:25.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:25.288+0000 7f7650000640 1 --2- 172.21.15.5:0/132934384 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f76481005a0 0x7f76481a7740 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:25.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:25.288+0000 7f7650000640 1 --2- 172.21.15.5:0/132934384 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f76480ffc00 0x7f76481a7200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:25.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:25.288+0000 7f7650000640 1 -- 172.21.15.5:0/132934384 >> 172.21.15.5:0/132934384 conn(0x7f76480fba90 msgr2=0x7f76480fc2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:43:25.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:25.288+0000 7f7650000640 1 -- 172.21.15.5:0/132934384 shutdown_connections 2024-09-10T09:43:25.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:25.289+0000 7f7650000640 1 -- 172.21.15.5:0/132934384 wait complete. 2024-09-10T09:43:25.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:43:25 smithi102 ceph-mon[35442]: pgmap v545: 129 pgs: 129 active+clean; 104 MiB data, 971 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 1.7 MiB/s wr, 155 op/s 2024-09-10T09:43:26.040 INFO:tasks.workunit.client.0.smithi005.stdout: 1 241691 3.99 MB/sec execute 67 sec latency 19.011 ms 2024-09-10T09:43:26.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:43:25 smithi005 ceph-mon[30350]: pgmap v545: 129 pgs: 129 active+clean; 104 MiB data, 971 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 1.7 MiB/s wr, 155 op/s 2024-09-10T09:43:26.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:43:25 smithi159 ceph-mon[35513]: pgmap v545: 129 pgs: 129 active+clean; 104 MiB data, 971 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 1.7 MiB/s wr, 155 op/s 2024-09-10T09:43:26.215 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-10T09:43:26.678 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:43:27.040 INFO:tasks.workunit.client.0.smithi005.stdout: 1 244000 4.10 MB/sec execute 68 sec latency 25.483 ms 2024-09-10T09:43:27.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:27.297+0000 7fa3dfe00640 1 -- 172.21.15.5:0/4255182379 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa3e0074f40 msgr2=0x7fa3e010dcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:43:27.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:27.297+0000 7fa3dfe00640 1 --2- 172.21.15.5:0/4255182379 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa3e0074f40 0x7fa3e010dcd0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fa3cc009900 tx=0x7fa3cc02a480 comp rx=0 tx=0).stop 2024-09-10T09:43:27.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:27.297+0000 7fa3dfe00640 1 -- 172.21.15.5:0/4255182379 shutdown_connections 2024-09-10T09:43:27.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:27.297+0000 7fa3dfe00640 1 --2- 172.21.15.5:0/4255182379 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa3e010e210 0x7fa3e0110660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:27.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:27.297+0000 7fa3dfe00640 1 --2- 172.21.15.5:0/4255182379 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa3e0074f40 0x7fa3e010dcd0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:27.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:27.297+0000 7fa3dfe00640 1 --2- 172.21.15.5:0/4255182379 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa3e0074620 0x7fa3e0074a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:27.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:27.297+0000 7fa3dfe00640 1 -- 172.21.15.5:0/4255182379 >> 172.21.15.5:0/4255182379 conn(0x7fa3e00fd710 msgr2=0x7fa3e00ffb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:43:27.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:27.298+0000 7fa3dfe00640 1 -- 172.21.15.5:0/4255182379 shutdown_connections 2024-09-10T09:43:27.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:27.298+0000 7fa3dfe00640 1 -- 172.21.15.5:0/4255182379 wait complete. 2024-09-10T09:43:27.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:27.299+0000 7fa3dfe00640 1 Processor -- start 2024-09-10T09:43:27.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:27.299+0000 7fa3dfe00640 1 -- start start 2024-09-10T09:43:27.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:27.299+0000 7fa3dfe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa3e0074620 0x7fa3e019a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:43:27.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:27.300+0000 7fa3dfe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa3e0074f40 0x7fa3e019aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:43:27.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:27.300+0000 7fa3dfe00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa3e010e210 0x7fa3e019eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:43:27.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:27.300+0000 7fa3dfe00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa3e0114010 con 0x7fa3e0074620 2024-09-10T09:43:27.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:27.300+0000 7fa3dfe00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fa3e0113e90 con 0x7fa3e0074f40 2024-09-10T09:43:27.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:27.300+0000 7fa3dfe00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fa3e0114190 con 0x7fa3e010e210 2024-09-10T09:43:27.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:27.300+0000 7fa3de000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa3e0074f40 0x7fa3e019aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:43:27.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:27.300+0000 7fa3df400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa3e010e210 0x7fa3e019eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:43:27.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:27.300+0000 7fa3de000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa3e0074f40 0x7fa3e019aa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:33608/0 (socket says 172.21.15.5:33608) 2024-09-10T09:43:27.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:27.300+0000 7fa3dea00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa3e0074620 0x7fa3e019a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:43:27.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:27.300+0000 7fa3de000640 1 -- 172.21.15.5:0/314542613 learned_addr learned my addr 172.21.15.5:0/314542613 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:43:27.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:27.300+0000 7fa3df400640 1 -- 172.21.15.5:0/314542613 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa3e0074f40 msgr2=0x7fa3e019aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:43:27.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:27.300+0000 7fa3df400640 1 --2- 172.21.15.5:0/314542613 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa3e0074f40 0x7fa3e019aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:27.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:27.300+0000 7fa3df400640 1 -- 172.21.15.5:0/314542613 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa3e0074620 msgr2=0x7fa3e019a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:43:27.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:27.300+0000 7fa3df400640 1 --2- 172.21.15.5:0/314542613 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa3e0074620 0x7fa3e019a4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:27.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:27.300+0000 7fa3df400640 1 -- 172.21.15.5:0/314542613 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3e019f480 con 0x7fa3e010e210 2024-09-10T09:43:27.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:27.300+0000 7fa3de000640 1 --2- 172.21.15.5:0/314542613 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa3e0074f40 0x7fa3e019aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:43:27.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:27.301+0000 7fa3df400640 1 --2- 172.21.15.5:0/314542613 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa3e010e210 0x7fa3e019eda0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fa3d8009fd0 tx=0x7fa3d800ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:43:27.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:27.301+0000 7fa3d7400640 1 -- 172.21.15.5:0/314542613 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3d800ce20 con 0x7fa3e010e210 2024-09-10T09:43:27.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:27.301+0000 7fa3d7400640 1 -- 172.21.15.5:0/314542613 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa3d8005020 con 0x7fa3e010e210 2024-09-10T09:43:27.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:27.301+0000 7fa3d7400640 1 -- 172.21.15.5:0/314542613 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3d8010720 con 0x7fa3e010e210 2024-09-10T09:43:27.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:27.301+0000 7fa3dfe00640 1 -- 172.21.15.5:0/314542613 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3e019f770 con 0x7fa3e010e210 2024-09-10T09:43:27.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:27.301+0000 7fa3dfe00640 1 -- 172.21.15.5:0/314542613 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3e01a7050 con 0x7fa3e010e210 2024-09-10T09:43:27.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:27.304+0000 7fa3e86ad740 1 Processor -- start 2024-09-10T09:43:27.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:27.305+0000 7fa3d7400640 1 -- 172.21.15.5:0/314542613 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7fa3d8005540 con 0x7fa3e010e210 2024-09-10T09:43:27.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:27.305+0000 7fa3e86ad740 1 -- start start 2024-09-10T09:43:27.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:27.305+0000 7fa3e86ad740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556cf6511e40 0x556cf6512220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:43:27.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:27.305+0000 7fa3e86ad740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556cf6512760 0x556cf651cae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:43:27.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:27.305+0000 7fa3e86ad740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556cf651d020 0x556cf651f410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:43:27.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:27.305+0000 7fa3e86ad740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x556cf64783e0 con 0x556cf651d020 2024-09-10T09:43:27.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:27.305+0000 7fa3e86ad740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x556cf641fd80 con 0x556cf6512760 2024-09-10T09:43:27.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:27.305+0000 7fa3e86ad740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x556cf64743a0 con 0x556cf6511e40 2024-09-10T09:43:27.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:27.306+0000 7fa3de000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556cf6511e40 0x556cf6512220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:43:27.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:27.306+0000 7fa3de000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556cf6511e40 0x556cf6512220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:40288/0 (socket says 172.21.15.5:40288) 2024-09-10T09:43:27.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:27.306+0000 7fa3de000640 1 -- 172.21.15.5:0/1123460713 learned_addr learned my addr 172.21.15.5:0/1123460713 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:43:27.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:27.306+0000 7fa3dea00640 1 --2- 172.21.15.5:0/1123460713 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556cf6512760 0x556cf651cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:43:27.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:27.306+0000 7fa3de000640 1 --2- 172.21.15.5:0/1123460713 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556cf651d020 0x556cf651f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:43:27.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:27.306+0000 7fa3dea00640 1 -- 172.21.15.5:0/1123460713 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556cf6511e40 msgr2=0x556cf6512220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:43:27.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:27.306+0000 7fa3dea00640 1 --2- 172.21.15.5:0/1123460713 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556cf6511e40 0x556cf6512220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:27.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:27.306+0000 7fa3dea00640 1 -- 172.21.15.5:0/1123460713 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556cf651d020 msgr2=0x556cf651f410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:43:27.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:27.306+0000 7fa3dea00640 1 --2- 172.21.15.5:0/1123460713 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556cf651d020 0x556cf651f410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:27.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:27.306+0000 7fa3dea00640 1 -- 172.21.15.5:0/1123460713 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556cf643eae0 con 0x556cf6512760 2024-09-10T09:43:27.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:27.306+0000 7fa3d7400640 1 --2- 172.21.15.5:0/314542613 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fa3c40a2c40 0x7fa3c40a5100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:43:27.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:27.307+0000 7fa3dea00640 1 --2- 172.21.15.5:0/1123460713 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556cf6512760 0x556cf651cae0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fa3d0009c20 tx=0x7fa3d0011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=31a05c65e7a39db0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:43:27.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:27.307+0000 7fa3d7400640 1 -- 172.21.15.5:0/314542613 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fa3d80cfac0 con 0x7fa3e010e210 2024-09-10T09:43:27.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:27.307+0000 7fa3de000640 1 --2- 172.21.15.5:0/314542613 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fa3c40a2c40 0x7fa3c40a5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:43:27.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:27.307+0000 7fa3cbe00640 1 -- 172.21.15.5:0/1123460713 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3d0004050 con 0x556cf6512760 2024-09-10T09:43:27.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:27.307+0000 7fa3cbe00640 1 -- 172.21.15.5:0/1123460713 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa3d0005500 con 0x556cf6512760 2024-09-10T09:43:27.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:27.307+0000 7fa3cbe00640 1 -- 172.21.15.5:0/1123460713 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3d0004da0 con 0x556cf6512760 2024-09-10T09:43:27.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:27.308+0000 7fa3e86ad740 1 -- 172.21.15.5:0/1123460713 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556cf6512760 msgr2=0x556cf651cae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:43:27.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:27.308+0000 7fa3e86ad740 1 --2- 172.21.15.5:0/1123460713 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556cf6512760 0x556cf651cae0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fa3d0009c20 tx=0x7fa3d0011810 comp rx=0 tx=0).stop 2024-09-10T09:43:27.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:27.308+0000 7fa3de000640 1 --2- 172.21.15.5:0/314542613 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fa3c40a2c40 0x7fa3c40a5100 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x556cf650f750 tx=0x7fa3cc02a980 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:43:27.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:27.308+0000 7fa3e86ad740 1 -- 172.21.15.5:0/1123460713 shutdown_connections 2024-09-10T09:43:27.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:27.308+0000 7fa3e86ad740 1 --2- 172.21.15.5:0/1123460713 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556cf651d020 0x556cf651f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:27.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:27.308+0000 7fa3e86ad740 1 --2- 172.21.15.5:0/1123460713 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556cf6512760 0x556cf651cae0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:27.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:27.308+0000 7fa3e86ad740 1 --2- 172.21.15.5:0/1123460713 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556cf6511e40 0x556cf6512220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:27.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:27.308+0000 7fa3e86ad740 1 -- 172.21.15.5:0/1123460713 >> 172.21.15.5:0/1123460713 conn(0x556cf650e3d0 msgr2=0x556cf650e7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:43:27.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:27.308+0000 7fa3e86ad740 1 -- 172.21.15.5:0/1123460713 shutdown_connections 2024-09-10T09:43:27.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:27.308+0000 7fa3e86ad740 1 -- 172.21.15.5:0/1123460713 wait complete. 2024-09-10T09:43:27.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:27.309+0000 7fa3e86ad740 1 -- start start 2024-09-10T09:43:27.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:27.309+0000 7fa3e86ad740 1 Processor -- start 2024-09-10T09:43:27.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:27.309+0000 7fa3e86ad740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556cf6511e40 0x556cf6519e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:43:27.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:27.309+0000 7fa3e86ad740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556cf6512760 0x556cf651a390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:43:27.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:27.309+0000 7fa3e86ad740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556cf651d020 0x556cf651a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:43:27.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:27.309+0000 7fa3e86ad740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x556cf651fbf0 con 0x556cf6511e40 2024-09-10T09:43:27.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:27.309+0000 7fa3e86ad740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x556cf64330f0 con 0x556cf6512760 2024-09-10T09:43:27.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:27.309+0000 7fa3e86ad740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x556cf651fd70 con 0x556cf651d020 2024-09-10T09:43:27.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:27.309+0000 7fa3e86ad740 10 client.0 _ll_register_callbacks cb 0x556cf641abd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:43:27.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:27.310+0000 7fa3df400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556cf651d020 0x556cf651a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:43:27.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:27.310+0000 7fa3df400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556cf651d020 0x556cf651a8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:40300/0 (socket says 172.21.15.5:40300) 2024-09-10T09:43:27.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:27.310+0000 7fa3df400640 1 -- 172.21.15.5:0/2618596165 learned_addr learned my addr 172.21.15.5:0/2618596165 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:43:27.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:27.310+0000 7fa3dea00640 1 --2- 172.21.15.5:0/2618596165 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556cf6511e40 0x556cf6519e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:43:27.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:27.310+0000 7fa3de000640 1 --2- 172.21.15.5:0/2618596165 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556cf6512760 0x556cf651a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:43:27.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:27.310+0000 7fa3df400640 1 -- 172.21.15.5:0/2618596165 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556cf6512760 msgr2=0x556cf651a390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:43:27.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:27.310+0000 7fa3df400640 1 --2- 172.21.15.5:0/2618596165 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556cf6512760 0x556cf651a390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:27.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:27.310+0000 7fa3df400640 1 -- 172.21.15.5:0/2618596165 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556cf6511e40 msgr2=0x556cf6519e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:43:27.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:27.310+0000 7fa3df400640 1 --2- 172.21.15.5:0/2618596165 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556cf6511e40 0x556cf6519e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:27.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:27.310+0000 7fa3df400640 1 -- 172.21.15.5:0/2618596165 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556cf61e93c0 con 0x556cf651d020 2024-09-10T09:43:27.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:27.310+0000 7fa3df400640 1 --2- 172.21.15.5:0/2618596165 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556cf651d020 0x556cf651a8d0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fa3d809bbd0 tx=0x7fa3d8005310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e91f807481da8070 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:43:27.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:27.310+0000 7fa3c8c00640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-09-10T09:43:27.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:27.311+0000 7fa3c8c00640 1 -- 172.21.15.5:0/2618596165 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3d80a48b0 con 0x556cf651d020 2024-09-10T09:43:27.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:27.311+0000 7fa3c8c00640 1 -- 172.21.15.5:0/2618596165 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fa3d809b320 con 0x556cf651d020 2024-09-10T09:43:27.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:27.311+0000 7fa3e86ad740 1 -- 172.21.15.5:0/2618596165 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa3d0005500 con 0x556cf651d020 2024-09-10T09:43:27.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:27.311+0000 7fa3c8c00640 1 -- 172.21.15.5:0/2618596165 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fa3d80a0280 con 0x556cf651d020 2024-09-10T09:43:27.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:27.311+0000 7fa3e86ad740 10 client.26422 fetch_fsmap learned FSMap version 34 2024-09-10T09:43:27.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:27.311+0000 7fa3e86ad740 1 -- 172.21.15.5:0/2618596165 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x556cf652bbe0 con 0x556cf651d020 2024-09-10T09:43:27.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:27.311+0000 7fa3c8c00640 1 -- 172.21.15.5:0/2618596165 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa3d0005500 con 0x556cf651d020 2024-09-10T09:43:27.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:27.311+0000 7fa3c8c00640 1 -- 172.21.15.5:0/2618596165 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7fa3d809ad90 con 0x556cf651d020 2024-09-10T09:43:27.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:27.311+0000 7fa3e86ad740 10 client.26422 fetch_fsmap finished waiting for FSMap version 34 2024-09-10T09:43:27.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:27.311+0000 7fa3e86ad740 10 client.26422 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-10T09:43:27.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:27.311+0000 7fa3e86ad740 20 client.26422 populate_metadata read hostname 'smithi005' 2024-09-10T09:43:27.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:27.311+0000 7fa3e86ad740 1 --2- 172.21.15.5:0/2618596165 >> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] conn(0x556cf652d0a0 0x556cf652f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:43:27.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:27.311+0000 7fa3e86ad740 4 client.26422 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:43:27.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:27.311+0000 7fa3e86ad740 1 -- 172.21.15.5:0/2618596165 --> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556cf6485950 con 0x556cf652d0a0 2024-09-10T09:43:27.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:27.312+0000 7fa3dea00640 1 --2- 172.21.15.5:0/2618596165 >> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] conn(0x556cf652d0a0 0x556cf652f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:43:27.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:27.312+0000 7fa3dea00640 1 --2- 172.21.15.5:0/2618596165 >> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] conn(0x556cf652d0a0 0x556cf652f4b0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=442ed80a2e18dd6d server_cookie=4134daa7e52c3466 in_seq=0 out_seq=0 2024-09-10T09:43:27.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:27.312+0000 7fa3c8c00640 10 client.26422 ms_handle_connect on v2:172.21.15.159:6836/620627639 2024-09-10T09:43:27.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:27.313+0000 7fa3c8c00640 1 -- 172.21.15.5:0/2618596165 <== mds.2 v2:172.21.15.159:6836/620627639 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556cf6485950 con 0x556cf652d0a0 2024-09-10T09:43:27.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:27.313+0000 7fa3c8c00640 10 client.26422 handle_command_reply: tid=0 2024-09-10T09:43:27.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:27.313+0000 7fa3e86ad740 1 client.26422 shutdown 2024-09-10T09:43:27.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:27.313+0000 7fa3e86ad740 1 -- 172.21.15.5:0/2618596165 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556cf651d020 msgr2=0x556cf651a8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:43:27.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:27.313+0000 7fa3e86ad740 1 --2- 172.21.15.5:0/2618596165 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556cf651d020 0x556cf651a8d0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fa3d809bbd0 tx=0x7fa3d8005310 comp rx=0 tx=0).stop 2024-09-10T09:43:27.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:27.313+0000 7fa3e86ad740 1 -- 172.21.15.5:0/2618596165 shutdown_connections 2024-09-10T09:43:27.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:27.313+0000 7fa3e86ad740 1 --2- 172.21.15.5:0/2618596165 >> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] conn(0x556cf652d0a0 0x556cf652f4b0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:27.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:27.313+0000 7fa3e86ad740 1 --2- 172.21.15.5:0/2618596165 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556cf651d020 0x556cf651a8d0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:27.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:27.313+0000 7fa3e86ad740 1 --2- 172.21.15.5:0/2618596165 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556cf6512760 0x556cf651a390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:27.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:27.313+0000 7fa3e86ad740 1 --2- 172.21.15.5:0/2618596165 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556cf6511e40 0x556cf6519e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:27.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:27.313+0000 7fa3c8c00640 0 client.26422 ms_handle_reset on v2:172.21.15.159:6836/620627639 2024-09-10T09:43:27.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:27.313+0000 7fa3e86ad740 1 -- 172.21.15.5:0/2618596165 >> 172.21.15.5:0/2618596165 conn(0x556cf650e3d0 msgr2=0x556cf65140b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:43:27.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:27.314+0000 7fa3e86ad740 1 -- 172.21.15.5:0/2618596165 shutdown_connections 2024-09-10T09:43:27.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:27.314+0000 7fa3e86ad740 1 -- 172.21.15.5:0/2618596165 wait complete. 2024-09-10T09:43:27.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:27.314+0000 7fa3e86ad740 20 client.26422 trim_cache size 0 max 16384 2024-09-10T09:43:27.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:27.328+0000 7fa3e86ad740 1 Processor -- start 2024-09-10T09:43:27.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:27.328+0000 7fa3e86ad740 1 -- start start 2024-09-10T09:43:27.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:27.328+0000 7fa3e86ad740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556cf6511e40 0x556cf6534d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:43:27.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:27.328+0000 7fa3e86ad740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556cf6512760 0x556cf65352c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:43:27.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:27.328+0000 7fa3e86ad740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556cf651d020 0x556cf6519e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:43:27.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:27.328+0000 7fa3e86ad740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa3e01b0f00 con 0x556cf6512760 2024-09-10T09:43:27.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:27.328+0000 7fa3e86ad740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fa3e01b2a40 con 0x556cf651d020 2024-09-10T09:43:27.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:27.328+0000 7fa3e86ad740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fa3e01b15d0 con 0x556cf6511e40 2024-09-10T09:43:27.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:27.328+0000 7fa3dea00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556cf6511e40 0x556cf6534d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:43:27.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:27.328+0000 7fa3df400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556cf651d020 0x556cf6519e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:43:27.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:27.328+0000 7fa3dea00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556cf6511e40 0x556cf6534d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:40312/0 (socket says 172.21.15.5:40312) 2024-09-10T09:43:27.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:27.328+0000 7fa3dea00640 1 -- 172.21.15.5:0/1575484811 learned_addr learned my addr 172.21.15.5:0/1575484811 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:43:27.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:27.328+0000 7fa3df400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556cf651d020 0x556cf6519e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:33634/0 (socket says 172.21.15.5:33634) 2024-09-10T09:43:27.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:27.328+0000 7fa3de000640 1 --2- 172.21.15.5:0/1575484811 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556cf6512760 0x556cf65352c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:43:27.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:27.329+0000 7fa3df400640 1 -- 172.21.15.5:0/1575484811 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556cf6511e40 msgr2=0x556cf6534d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:43:27.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:27.329+0000 7fa3df400640 1 --2- 172.21.15.5:0/1575484811 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556cf6511e40 0x556cf6534d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:27.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:27.329+0000 7fa3df400640 1 -- 172.21.15.5:0/1575484811 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556cf6512760 msgr2=0x556cf65352c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:43:27.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:27.329+0000 7fa3df400640 1 --2- 172.21.15.5:0/1575484811 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556cf6512760 0x556cf65352c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:27.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:27.329+0000 7fa3df400640 1 -- 172.21.15.5:0/1575484811 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556cf650e200 con 0x556cf651d020 2024-09-10T09:43:27.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:27.329+0000 7fa3de000640 1 --2- 172.21.15.5:0/1575484811 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556cf6512760 0x556cf65352c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:43:27.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:27.329+0000 7fa3dea00640 1 --2- 172.21.15.5:0/1575484811 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556cf6511e40 0x556cf6534d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:43:27.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:27.329+0000 7fa3df400640 1 --2- 172.21.15.5:0/1575484811 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556cf651d020 0x556cf6519e10 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fa3d8098330 tx=0x7fa3d8005830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=702a11ea7355e895 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:43:27.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:27.329+0000 7fa3cb400640 1 -- 172.21.15.5:0/1575484811 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3d809b830 con 0x556cf651d020 2024-09-10T09:43:27.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:27.329+0000 7fa3cb400640 1 -- 172.21.15.5:0/1575484811 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa3d80a4b40 con 0x556cf651d020 2024-09-10T09:43:27.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:27.329+0000 7fa3cb400640 1 -- 172.21.15.5:0/1575484811 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3d8097c20 con 0x556cf651d020 2024-09-10T09:43:27.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:27.330+0000 7fa3e86ad740 1 -- 172.21.15.5:0/1575484811 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556cf651d020 msgr2=0x556cf6519e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:43:27.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:27.330+0000 7fa3e86ad740 1 --2- 172.21.15.5:0/1575484811 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556cf651d020 0x556cf6519e10 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fa3d8098330 tx=0x7fa3d8005830 comp rx=0 tx=0).stop 2024-09-10T09:43:27.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:27.330+0000 7fa3e86ad740 1 -- 172.21.15.5:0/1575484811 shutdown_connections 2024-09-10T09:43:27.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:27.330+0000 7fa3e86ad740 1 --2- 172.21.15.5:0/1575484811 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556cf651d020 0x556cf6519e10 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:27.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:27.330+0000 7fa3e86ad740 1 --2- 172.21.15.5:0/1575484811 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556cf6512760 0x556cf65352c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:27.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:27.330+0000 7fa3e86ad740 1 --2- 172.21.15.5:0/1575484811 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556cf6511e40 0x556cf6534d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:27.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:27.330+0000 7fa3e86ad740 1 -- 172.21.15.5:0/1575484811 >> 172.21.15.5:0/1575484811 conn(0x556cf650e3d0 msgr2=0x556cf6518f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:43:27.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:27.330+0000 7fa3e86ad740 1 -- 172.21.15.5:0/1575484811 shutdown_connections 2024-09-10T09:43:27.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:27.330+0000 7fa3e86ad740 1 -- 172.21.15.5:0/1575484811 wait complete. 2024-09-10T09:43:27.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:27.331+0000 7fa3e86ad740 1 -- start start 2024-09-10T09:43:27.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:27.331+0000 7fa3e86ad740 1 Processor -- start 2024-09-10T09:43:27.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:27.331+0000 7fa3e86ad740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556cf6511e40 0x556cf65257a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:43:27.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:27.331+0000 7fa3e86ad740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556cf6512760 0x556cf6520ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:43:27.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:27.331+0000 7fa3e86ad740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556cf651d020 0x556cf6521030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:43:27.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:27.331+0000 7fa3e86ad740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa3e01b2370 con 0x556cf6511e40 2024-09-10T09:43:27.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:27.331+0000 7fa3e86ad740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fa3e01b1ca0 con 0x556cf651d020 2024-09-10T09:43:27.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:27.331+0000 7fa3e86ad740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fa3e01aeee0 con 0x556cf6512760 2024-09-10T09:43:27.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:27.332+0000 7fa3e86ad740 10 client.0 _ll_register_callbacks cb 0x7fa3d80053b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:43:27.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:27.332+0000 7fa3de000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556cf6512760 0x556cf6520ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:43:27.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:27.332+0000 7fa3de000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556cf6512760 0x556cf6520ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:40322/0 (socket says 172.21.15.5:40322) 2024-09-10T09:43:27.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:27.332+0000 7fa3de000640 1 -- 172.21.15.5:0/2066351863 learned_addr learned my addr 172.21.15.5:0/2066351863 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:43:27.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:27.332+0000 7fa3dea00640 1 --2- 172.21.15.5:0/2066351863 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556cf6511e40 0x556cf65257a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:43:27.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:27.332+0000 7fa3df400640 1 --2- 172.21.15.5:0/2066351863 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556cf651d020 0x556cf6521030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:43:27.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:27.332+0000 7fa3de000640 1 -- 172.21.15.5:0/2066351863 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556cf651d020 msgr2=0x556cf6521030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:43:27.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:27.332+0000 7fa3de000640 1 --2- 172.21.15.5:0/2066351863 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556cf651d020 0x556cf6521030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:27.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:27.332+0000 7fa3de000640 1 -- 172.21.15.5:0/2066351863 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556cf6511e40 msgr2=0x556cf65257a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:43:27.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:27.332+0000 7fa3de000640 1 --2- 172.21.15.5:0/2066351863 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556cf6511e40 0x556cf65257a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:27.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:27.332+0000 7fa3de000640 1 -- 172.21.15.5:0/2066351863 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556cf650f060 con 0x556cf6512760 2024-09-10T09:43:27.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:27.332+0000 7fa3df400640 1 --2- 172.21.15.5:0/2066351863 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556cf651d020 0x556cf6521030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:43:27.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:27.332+0000 7fa3dea00640 1 --2- 172.21.15.5:0/2066351863 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556cf6511e40 0x556cf65257a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:43:27.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:27.333+0000 7fa3de000640 1 --2- 172.21.15.5:0/2066351863 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556cf6512760 0x556cf6520ad0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fa3cc02aa70 tx=0x7fa3cc02a480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ac9d8180fecf308a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:43:27.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:27.333+0000 7fa3c8c00640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-09-10T09:43:27.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:27.333+0000 7fa3c8c00640 1 -- 172.21.15.5:0/2066351863 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3cc031d00 con 0x556cf6512760 2024-09-10T09:43:27.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:27.333+0000 7fa3c8c00640 1 -- 172.21.15.5:0/2066351863 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fa3cc03d450 con 0x556cf6512760 2024-09-10T09:43:27.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:27.333+0000 7fa3e86ad740 1 -- 172.21.15.5:0/2066351863 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa3d80a4b40 con 0x556cf6512760 2024-09-10T09:43:27.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:27.333+0000 7fa3c8c00640 1 -- 172.21.15.5:0/2066351863 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fa3cc05f050 con 0x556cf6512760 2024-09-10T09:43:27.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:27.333+0000 7fa3e86ad740 10 client.26428 fetch_fsmap learned FSMap version 34 2024-09-10T09:43:27.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:27.333+0000 7fa3e86ad740 1 -- 172.21.15.5:0/2066351863 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x556cf6597c30 con 0x556cf6512760 2024-09-10T09:43:27.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:27.333+0000 7fa3c8c00640 1 -- 172.21.15.5:0/2066351863 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa3d80a4b40 con 0x556cf6512760 2024-09-10T09:43:27.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:27.334+0000 7fa3c8c00640 1 -- 172.21.15.5:0/2066351863 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7fa3cc054020 con 0x556cf6512760 2024-09-10T09:43:27.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:27.334+0000 7fa3e86ad740 10 client.26428 fetch_fsmap finished waiting for FSMap version 34 2024-09-10T09:43:27.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:27.334+0000 7fa3e86ad740 10 client.26428 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-10T09:43:27.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:27.334+0000 7fa3e86ad740 20 client.26428 populate_metadata read hostname 'smithi005' 2024-09-10T09:43:27.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:27.334+0000 7fa3e86ad740 1 --2- 172.21.15.5:0/2066351863 >> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] conn(0x556cf65983e0 0x556cf659a800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:43:27.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:27.334+0000 7fa3e86ad740 4 client.26428 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T09:43:27.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:27.334+0000 7fa3e86ad740 1 -- 172.21.15.5:0/2066351863 --> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556cf647fe60 con 0x556cf65983e0 2024-09-10T09:43:27.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:27.334+0000 7fa3dea00640 1 --2- 172.21.15.5:0/2066351863 >> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] conn(0x556cf65983e0 0x556cf659a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:43:27.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:27.335+0000 7fa3dea00640 1 --2- 172.21.15.5:0/2066351863 >> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] conn(0x556cf65983e0 0x556cf659a800 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3ba1c266033d23f3 server_cookie=1926f14a32bb3a7d in_seq=0 out_seq=0 2024-09-10T09:43:27.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:27.335+0000 7fa3c8c00640 10 client.26428 ms_handle_connect on v2:172.21.15.159:6836/620627639 2024-09-10T09:43:27.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:27.335+0000 7fa3c8c00640 1 -- 172.21.15.5:0/2066351863 <== mds.2 v2:172.21.15.159:6836/620627639 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556cf647fe60 con 0x556cf65983e0 2024-09-10T09:43:27.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:27.335+0000 7fa3c8c00640 10 client.26428 handle_command_reply: tid=0 2024-09-10T09:43:27.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:27.335+0000 7fa3e86ad740 1 client.26428 shutdown 2024-09-10T09:43:27.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:27.335+0000 7fa3e86ad740 1 -- 172.21.15.5:0/2066351863 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556cf6512760 msgr2=0x556cf6520ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:43:27.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:27.335+0000 7fa3e86ad740 1 --2- 172.21.15.5:0/2066351863 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556cf6512760 0x556cf6520ad0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fa3cc02aa70 tx=0x7fa3cc02a480 comp rx=0 tx=0).stop 2024-09-10T09:43:27.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:27.335+0000 7fa3e86ad740 1 -- 172.21.15.5:0/2066351863 shutdown_connections 2024-09-10T09:43:27.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:27.335+0000 7fa3e86ad740 1 --2- 172.21.15.5:0/2066351863 >> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] conn(0x556cf65983e0 0x556cf659a800 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:27.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:27.335+0000 7fa3e86ad740 1 --2- 172.21.15.5:0/2066351863 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556cf651d020 0x556cf6521030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:27.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:27.335+0000 7fa3e86ad740 1 --2- 172.21.15.5:0/2066351863 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556cf6512760 0x556cf6520ad0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:27.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:27.335+0000 7fa3e86ad740 1 --2- 172.21.15.5:0/2066351863 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556cf6511e40 0x556cf65257a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:27.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:27.335+0000 7fa3e86ad740 1 -- 172.21.15.5:0/2066351863 >> 172.21.15.5:0/2066351863 conn(0x556cf650e3d0 msgr2=0x556cf6519e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:43:27.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:27.335+0000 7fa3c8c00640 0 client.26428 ms_handle_reset on v2:172.21.15.159:6836/620627639 2024-09-10T09:43:27.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:27.335+0000 7fa3e86ad740 1 -- 172.21.15.5:0/2066351863 shutdown_connections 2024-09-10T09:43:27.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:27.335+0000 7fa3e86ad740 1 -- 172.21.15.5:0/2066351863 wait complete. 2024-09-10T09:43:27.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:27.336+0000 7fa3e86ad740 20 client.26428 trim_cache size 0 max 16384 2024-09-10T09:43:27.340 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-10T09:43:27.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:27.336+0000 7fa3dfe00640 1 -- 172.21.15.5:0/314542613 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fa3c40a2c40 msgr2=0x7fa3c40a5100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:43:27.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:27.336+0000 7fa3dfe00640 1 --2- 172.21.15.5:0/314542613 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fa3c40a2c40 0x7fa3c40a5100 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x556cf650f750 tx=0x7fa3cc02a980 comp rx=0 tx=0).stop 2024-09-10T09:43:27.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:27.336+0000 7fa3dfe00640 1 -- 172.21.15.5:0/314542613 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa3e010e210 msgr2=0x7fa3e019eda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:43:27.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:27.336+0000 7fa3dfe00640 1 --2- 172.21.15.5:0/314542613 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa3e010e210 0x7fa3e019eda0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fa3d8009fd0 tx=0x7fa3d800ef10 comp rx=0 tx=0).stop 2024-09-10T09:43:27.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:27.336+0000 7fa3dfe00640 1 -- 172.21.15.5:0/314542613 shutdown_connections 2024-09-10T09:43:27.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:27.336+0000 7fa3dfe00640 1 --2- 172.21.15.5:0/314542613 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fa3c40a2c40 0x7fa3c40a5100 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:27.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:27.336+0000 7fa3dfe00640 1 --2- 172.21.15.5:0/314542613 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa3e010e210 0x7fa3e019eda0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:27.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:27.336+0000 7fa3dfe00640 1 --2- 172.21.15.5:0/314542613 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa3e0074f40 0x7fa3e019aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:27.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:27.336+0000 7fa3dfe00640 1 --2- 172.21.15.5:0/314542613 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa3e0074620 0x7fa3e019a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:27.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:27.336+0000 7fa3dfe00640 1 -- 172.21.15.5:0/314542613 >> 172.21.15.5:0/314542613 conn(0x7fa3e00fd710 msgr2=0x7fa3e00fe3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:43:27.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:27.337+0000 7fa3dfe00640 1 -- 172.21.15.5:0/314542613 shutdown_connections 2024-09-10T09:43:27.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:27.337+0000 7fa3dfe00640 1 -- 172.21.15.5:0/314542613 wait complete. 2024-09-10T09:43:27.867 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:43:27 smithi005 ceph-mon[30350]: pgmap v546: 129 pgs: 129 active+clean; 105 MiB data, 961 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.7 MiB/s wr, 200 op/s 2024-09-10T09:43:27.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:43:27 smithi102 ceph-mon[35442]: pgmap v546: 129 pgs: 129 active+clean; 105 MiB data, 961 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.7 MiB/s wr, 200 op/s 2024-09-10T09:43:28.040 INFO:tasks.workunit.client.0.smithi005.stdout: 1 245966 4.19 MB/sec execute 69 sec latency 22.082 ms 2024-09-10T09:43:28.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:43:27 smithi159 ceph-mon[35513]: pgmap v546: 129 pgs: 129 active+clean; 105 MiB data, 961 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.7 MiB/s wr, 200 op/s 2024-09-10T09:43:28.475 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-10T09:43:28.873 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:43:29.040 INFO:tasks.workunit.client.0.smithi005.stdout: 1 248554 4.37 MB/sec execute 70 sec latency 14.192 ms 2024-09-10T09:43:29.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:29.500+0000 7f07e7400640 1 -- 172.21.15.5:0/2109052406 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f07e010e430 msgr2=0x7f07e0110820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:43:29.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:29.500+0000 7f07e7400640 1 --2- 172.21.15.5:0/2109052406 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f07e010e430 0x7f07e0110820 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f07d8009ca0 tx=0x7f07d802a950 comp rx=0 tx=0).stop 2024-09-10T09:43:29.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:29.500+0000 7f07e7400640 1 -- 172.21.15.5:0/2109052406 shutdown_connections 2024-09-10T09:43:29.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:29.500+0000 7f07e7400640 1 --2- 172.21.15.5:0/2109052406 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f07e010e430 0x7f07e0110820 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:29.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:29.500+0000 7f07e7400640 1 --2- 172.21.15.5:0/2109052406 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f07e0074ee0 0x7f07e010def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:29.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:29.500+0000 7f07e7400640 1 --2- 172.21.15.5:0/2109052406 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f07e00745c0 0x7f07e00749a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:29.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:29.500+0000 7f07e7400640 1 -- 172.21.15.5:0/2109052406 >> 172.21.15.5:0/2109052406 conn(0x7f07e00fd8b0 msgr2=0x7f07e00ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:43:29.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:29.500+0000 7f07e7400640 1 -- 172.21.15.5:0/2109052406 shutdown_connections 2024-09-10T09:43:29.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:29.500+0000 7f07e7400640 1 -- 172.21.15.5:0/2109052406 wait complete. 2024-09-10T09:43:29.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:29.501+0000 7f07e7400640 1 Processor -- start 2024-09-10T09:43:29.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:29.502+0000 7f07e7400640 1 -- start start 2024-09-10T09:43:29.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:29.502+0000 7f07e7400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f07e00745c0 0x7f07e019a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:43:29.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:29.502+0000 7f07e7400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f07e0074ee0 0x7f07e019aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:43:29.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:29.502+0000 7f07e7400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f07e010e430 0x7f07e019ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:43:29.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:29.502+0000 7f07e7400640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f07e01141c0 con 0x7f07e010e430 2024-09-10T09:43:29.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:29.502+0000 7f07e7400640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f07e0114040 con 0x7f07e00745c0 2024-09-10T09:43:29.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:29.502+0000 7f07e7400640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f07e0114340 con 0x7f07e0074ee0 2024-09-10T09:43:29.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:29.502+0000 7f07dfe00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f07e0074ee0 0x7f07e019aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:43:29.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:29.502+0000 7f07e4a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f07e00745c0 0x7f07e019a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:43:29.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:29.502+0000 7f07dfe00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f07e0074ee0 0x7f07e019aba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:40348/0 (socket says 172.21.15.5:40348) 2024-09-10T09:43:29.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:29.502+0000 7f07dfe00640 1 -- 172.21.15.5:0/538222507 learned_addr learned my addr 172.21.15.5:0/538222507 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:43:29.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:29.503+0000 7f07e5400640 1 --2- 172.21.15.5:0/538222507 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f07e010e430 0x7f07e019ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:43:29.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:29.503+0000 7f07dfe00640 1 -- 172.21.15.5:0/538222507 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f07e00745c0 msgr2=0x7f07e019a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:43:29.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:29.503+0000 7f07dfe00640 1 --2- 172.21.15.5:0/538222507 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f07e00745c0 0x7f07e019a660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:29.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:29.503+0000 7f07dfe00640 1 -- 172.21.15.5:0/538222507 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f07e010e430 msgr2=0x7f07e019ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:43:29.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:29.503+0000 7f07dfe00640 1 --2- 172.21.15.5:0/538222507 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f07e010e430 0x7f07e019ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:29.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:29.503+0000 7f07dfe00640 1 -- 172.21.15.5:0/538222507 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07e019f610 con 0x7f07e0074ee0 2024-09-10T09:43:29.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:29.503+0000 7f07e4a00640 1 --2- 172.21.15.5:0/538222507 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f07e00745c0 0x7f07e019a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:43:29.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:29.503+0000 7f07e5400640 1 --2- 172.21.15.5:0/538222507 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f07e010e430 0x7f07e019ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T09:43:29.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:29.503+0000 7f07dfe00640 1 --2- 172.21.15.5:0/538222507 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f07e0074ee0 0x7f07e019aba0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f07cc00e9f0 tx=0x7f07cc00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:43:29.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:29.504+0000 7f07dd600640 1 -- 172.21.15.5:0/538222507 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07cc00cd80 con 0x7f07e0074ee0 2024-09-10T09:43:29.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:29.504+0000 7f07dd600640 1 -- 172.21.15.5:0/538222507 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f07cc010910 con 0x7f07e0074ee0 2024-09-10T09:43:29.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:29.504+0000 7f07e7400640 1 -- 172.21.15.5:0/538222507 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07e019f900 con 0x7f07e0074ee0 2024-09-10T09:43:29.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:29.504+0000 7f07dd600640 1 -- 172.21.15.5:0/538222507 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07cc018990 con 0x7f07e0074ee0 2024-09-10T09:43:29.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:29.504+0000 7f07e7400640 1 -- 172.21.15.5:0/538222507 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07e01a71e0 con 0x7f07e0074ee0 2024-09-10T09:43:29.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:29.507+0000 7f07ea02b740 1 Processor -- start 2024-09-10T09:43:29.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:29.507+0000 7f07ea02b740 1 -- start start 2024-09-10T09:43:29.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:29.508+0000 7f07ea02b740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555c0ac1ac70 0x555c0ac1b050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:43:29.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:29.508+0000 7f07ea02b740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555c0ac1b590 0x555c0ac25910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:43:29.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:29.508+0000 7f07dd600640 1 -- 172.21.15.5:0/538222507 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f07cc010ac0 con 0x7f07e0074ee0 2024-09-10T09:43:29.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:29.508+0000 7f07ea02b740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555c0ac25e50 0x555c0ac28240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:43:29.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:29.508+0000 7f07ea02b740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x555c0ab813e0 con 0x555c0ac1ac70 2024-09-10T09:43:29.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:29.508+0000 7f07ea02b740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x555c0ab28d80 con 0x555c0ac25e50 2024-09-10T09:43:29.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:29.508+0000 7f07ea02b740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x555c0ab7d3a0 con 0x555c0ac1b590 2024-09-10T09:43:29.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:29.508+0000 7f07e4a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555c0ac1b590 0x555c0ac25910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:43:29.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:29.508+0000 7f07e4a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555c0ac1b590 0x555c0ac25910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:40362/0 (socket says 172.21.15.5:40362) 2024-09-10T09:43:29.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:29.508+0000 7f07e5400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555c0ac1ac70 0x555c0ac1b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:43:29.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:29.508+0000 7f07e4a00640 1 -- 172.21.15.5:0/3356959604 learned_addr learned my addr 172.21.15.5:0/3356959604 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:43:29.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:29.508+0000 7f07dfe00640 1 --2- 172.21.15.5:0/3356959604 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555c0ac25e50 0x555c0ac28240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:43:29.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:29.508+0000 7f07e4a00640 1 -- 172.21.15.5:0/3356959604 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555c0ac25e50 msgr2=0x555c0ac28240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:43:29.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:29.508+0000 7f07e4a00640 1 --2- 172.21.15.5:0/3356959604 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555c0ac25e50 0x555c0ac28240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:29.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:29.508+0000 7f07e4a00640 1 -- 172.21.15.5:0/3356959604 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555c0ac1ac70 msgr2=0x555c0ac1b050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:43:29.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:29.508+0000 7f07e4a00640 1 --2- 172.21.15.5:0/3356959604 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555c0ac1ac70 0x555c0ac1b050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:29.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:29.508+0000 7f07e4a00640 1 -- 172.21.15.5:0/3356959604 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555c0ab47ae0 con 0x555c0ac1b590 2024-09-10T09:43:29.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:29.509+0000 7f07e4a00640 1 --2- 172.21.15.5:0/3356959604 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555c0ac1b590 0x555c0ac25910 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f07d40115c0 tx=0x7f07d4011a80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5e0e078a5deb749c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:43:29.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:29.509+0000 7f07d1c00640 1 -- 172.21.15.5:0/3356959604 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07d400f640 con 0x555c0ac1b590 2024-09-10T09:43:29.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:29.509+0000 7f07d1c00640 1 -- 172.21.15.5:0/3356959604 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f07d400fcc0 con 0x555c0ac1b590 2024-09-10T09:43:29.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:29.509+0000 7f07d1c00640 1 -- 172.21.15.5:0/3356959604 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07d4019e10 con 0x555c0ac1b590 2024-09-10T09:43:29.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:29.510+0000 7f07ea02b740 1 -- 172.21.15.5:0/3356959604 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555c0ac1b590 msgr2=0x555c0ac25910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:43:29.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:29.510+0000 7f07ea02b740 1 --2- 172.21.15.5:0/3356959604 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555c0ac1b590 0x555c0ac25910 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f07d40115c0 tx=0x7f07d4011a80 comp rx=0 tx=0).stop 2024-09-10T09:43:29.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:29.510+0000 7f07ea02b740 1 -- 172.21.15.5:0/3356959604 shutdown_connections 2024-09-10T09:43:29.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:29.510+0000 7f07ea02b740 1 --2- 172.21.15.5:0/3356959604 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555c0ac25e50 0x555c0ac28240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:29.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:29.510+0000 7f07ea02b740 1 --2- 172.21.15.5:0/3356959604 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555c0ac1b590 0x555c0ac25910 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:29.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:29.510+0000 7f07ea02b740 1 --2- 172.21.15.5:0/3356959604 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555c0ac1ac70 0x555c0ac1b050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:29.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:29.510+0000 7f07ea02b740 1 -- 172.21.15.5:0/3356959604 >> 172.21.15.5:0/3356959604 conn(0x555c0ac17200 msgr2=0x555c0ac175e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:43:29.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:29.510+0000 7f07ea02b740 1 -- 172.21.15.5:0/3356959604 shutdown_connections 2024-09-10T09:43:29.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:29.510+0000 7f07ea02b740 1 -- 172.21.15.5:0/3356959604 wait complete. 2024-09-10T09:43:29.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:29.511+0000 7f07dd600640 1 --2- 172.21.15.5:0/538222507 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f07c80a2b10 0x7f07c80a4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:43:29.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:29.511+0000 7f07dd600640 1 -- 172.21.15.5:0/538222507 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f07cc0cf710 con 0x7f07e0074ee0 2024-09-10T09:43:29.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:29.511+0000 7f07ea02b740 1 -- start start 2024-09-10T09:43:29.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:29.511+0000 7f07e4a00640 1 --2- 172.21.15.5:0/538222507 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f07c80a2b10 0x7f07c80a4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:43:29.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:29.511+0000 7f07ea02b740 1 Processor -- start 2024-09-10T09:43:29.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:29.512+0000 7f07ea02b740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555c0ac1ac70 0x555c0ac22d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:43:29.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:29.512+0000 7f07ea02b740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555c0ac1b590 0x555c0ac23250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:43:29.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:29.512+0000 7f07ea02b740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555c0ac25e50 0x555c0ac23790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:43:29.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:29.512+0000 7f07ea02b740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x555c0ac28a20 con 0x555c0ac1ac70 2024-09-10T09:43:29.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:29.512+0000 7f07ea02b740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x555c0ab3c0f0 con 0x555c0ac1b590 2024-09-10T09:43:29.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:29.512+0000 7f07ea02b740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x555c0ac28ba0 con 0x555c0ac25e50 2024-09-10T09:43:29.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:29.512+0000 7f07ea02b740 10 client.0 _ll_register_callbacks cb 0x555c0ab23bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:43:29.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:29.512+0000 7f07dfe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555c0ac1b590 0x555c0ac23250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:43:29.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:29.512+0000 7f07dfe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555c0ac1b590 0x555c0ac23250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:33676/0 (socket says 172.21.15.5:33676) 2024-09-10T09:43:29.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:29.512+0000 7f07dfe00640 1 -- 172.21.15.5:0/281940071 learned_addr learned my addr 172.21.15.5:0/281940071 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:43:29.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:29.512+0000 7f07e4a00640 1 --2- 172.21.15.5:0/538222507 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f07c80a2b10 0x7f07c80a4fd0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f07d400ff90 tx=0x7f07d4009ca0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:43:29.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:29.512+0000 7f07e4a00640 1 --2- 172.21.15.5:0/281940071 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555c0ac1ac70 0x555c0ac22d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:43:29.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:29.512+0000 7f07e5400640 1 --2- 172.21.15.5:0/281940071 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555c0ac25e50 0x555c0ac23790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:43:29.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:29.512+0000 7f07dfe00640 1 -- 172.21.15.5:0/281940071 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555c0ac25e50 msgr2=0x555c0ac23790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:43:29.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:29.512+0000 7f07dfe00640 1 --2- 172.21.15.5:0/281940071 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555c0ac25e50 0x555c0ac23790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:29.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:29.513+0000 7f07dfe00640 1 -- 172.21.15.5:0/281940071 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555c0ac1ac70 msgr2=0x555c0ac22d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:43:29.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:29.513+0000 7f07dfe00640 1 --2- 172.21.15.5:0/281940071 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555c0ac1ac70 0x555c0ac22d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:29.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:29.513+0000 7f07dfe00640 1 -- 172.21.15.5:0/281940071 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555c0a8f23c0 con 0x555c0ac1b590 2024-09-10T09:43:29.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:29.513+0000 7f07dfe00640 1 --2- 172.21.15.5:0/281940071 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555c0ac1b590 0x555c0ac23250 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f07cc09b820 tx=0x7f07cc09b9b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=31b7247f969e81a6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:43:29.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:29.513+0000 7f07c6000640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-09-10T09:43:29.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:29.513+0000 7f07c6000640 1 -- 172.21.15.5:0/281940071 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07cc09bb20 con 0x555c0ac1b590 2024-09-10T09:43:29.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:29.513+0000 7f07c6000640 1 -- 172.21.15.5:0/281940071 <== mon.2 v2:172.21.15.102:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f07cc018b30 con 0x555c0ac1b590 2024-09-10T09:43:29.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:29.513+0000 7f07ea02b740 1 -- 172.21.15.5:0/281940071 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f07d400fcc0 con 0x555c0ac1b590 2024-09-10T09:43:29.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:29.513+0000 7f07c6000640 1 -- 172.21.15.5:0/281940071 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f07cc09fed0 con 0x555c0ac1b590 2024-09-10T09:43:29.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:29.513+0000 7f07ea02b740 10 client.26414 fetch_fsmap learned FSMap version 34 2024-09-10T09:43:29.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:29.513+0000 7f07ea02b740 1 -- 172.21.15.5:0/281940071 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x555c0ac349f0 con 0x555c0ac1b590 2024-09-10T09:43:29.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:29.514+0000 7f07c6000640 1 -- 172.21.15.5:0/281940071 <== mon.2 v2:172.21.15.102:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f07d400fcc0 con 0x555c0ac1b590 2024-09-10T09:43:29.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:29.514+0000 7f07c6000640 1 -- 172.21.15.5:0/281940071 <== mon.2 v2:172.21.15.102:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f07cc09b090 con 0x555c0ac1b590 2024-09-10T09:43:29.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:29.514+0000 7f07ea02b740 10 client.26414 fetch_fsmap finished waiting for FSMap version 34 2024-09-10T09:43:29.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:29.514+0000 7f07ea02b740 10 client.26414 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-10T09:43:29.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:29.514+0000 7f07ea02b740 20 client.26414 populate_metadata read hostname 'smithi005' 2024-09-10T09:43:29.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:29.514+0000 7f07ea02b740 1 --2- 172.21.15.5:0/281940071 >> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] conn(0x555c0ac35eb0 0x555c0ac38340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:43:29.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:29.514+0000 7f07ea02b740 4 client.26414 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:43:29.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:29.514+0000 7f07ea02b740 1 -- 172.21.15.5:0/281940071 --> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555c0ab8e950 con 0x555c0ac35eb0 2024-09-10T09:43:29.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:29.514+0000 7f07e4a00640 1 --2- 172.21.15.5:0/281940071 >> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] conn(0x555c0ac35eb0 0x555c0ac38340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:43:29.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:29.515+0000 7f07e4a00640 1 --2- 172.21.15.5:0/281940071 >> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] conn(0x555c0ac35eb0 0x555c0ac38340 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d46a5d57d487f7c server_cookie=9b57c5085134b8d8 in_seq=0 out_seq=0 2024-09-10T09:43:29.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:29.515+0000 7f07c6000640 10 client.26414 ms_handle_connect on v2:172.21.15.159:6838/2073739599 2024-09-10T09:43:29.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:29.516+0000 7f07c6000640 1 -- 172.21.15.5:0/281940071 <== mds.1 v2:172.21.15.159:6838/2073739599 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555c0ab8e950 con 0x555c0ac35eb0 2024-09-10T09:43:29.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:29.516+0000 7f07c6000640 10 client.26414 handle_command_reply: tid=0 2024-09-10T09:43:29.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:29.516+0000 7f07ea02b740 1 client.26414 shutdown 2024-09-10T09:43:29.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:29.516+0000 7f07ea02b740 1 -- 172.21.15.5:0/281940071 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555c0ac1b590 msgr2=0x555c0ac23250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:43:29.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:29.516+0000 7f07ea02b740 1 --2- 172.21.15.5:0/281940071 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555c0ac1b590 0x555c0ac23250 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f07cc09b820 tx=0x7f07cc09b9b0 comp rx=0 tx=0).stop 2024-09-10T09:43:29.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:29.516+0000 7f07ea02b740 1 -- 172.21.15.5:0/281940071 shutdown_connections 2024-09-10T09:43:29.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:29.516+0000 7f07ea02b740 1 --2- 172.21.15.5:0/281940071 >> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] conn(0x555c0ac35eb0 0x555c0ac38340 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:29.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:29.516+0000 7f07ea02b740 1 --2- 172.21.15.5:0/281940071 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555c0ac25e50 0x555c0ac23790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:29.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:29.516+0000 7f07ea02b740 1 --2- 172.21.15.5:0/281940071 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555c0ac1b590 0x555c0ac23250 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:29.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:29.516+0000 7f07ea02b740 1 --2- 172.21.15.5:0/281940071 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555c0ac1ac70 0x555c0ac22d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:29.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:29.516+0000 7f07ea02b740 1 -- 172.21.15.5:0/281940071 >> 172.21.15.5:0/281940071 conn(0x555c0ac17200 msgr2=0x555c0ac1cf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:43:29.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:29.517+0000 7f07c6000640 0 client.26414 ms_handle_reset on v2:172.21.15.159:6838/2073739599 2024-09-10T09:43:29.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:29.517+0000 7f07ea02b740 1 -- 172.21.15.5:0/281940071 shutdown_connections 2024-09-10T09:43:29.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:29.517+0000 7f07ea02b740 1 -- 172.21.15.5:0/281940071 wait complete. 2024-09-10T09:43:29.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:29.517+0000 7f07ea02b740 20 client.26414 trim_cache size 0 max 16384 2024-09-10T09:43:29.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:29.532+0000 7f07ea02b740 1 Processor -- start 2024-09-10T09:43:29.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:29.532+0000 7f07ea02b740 1 -- start start 2024-09-10T09:43:29.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:29.532+0000 7f07ea02b740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555c0ac1ac70 0x555c0ac1d720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:43:29.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:29.532+0000 7f07ea02b740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555c0ac1b590 0x555c0ac3dc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:43:29.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:29.532+0000 7f07ea02b740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555c0ac25e50 0x555c0ac3e150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:43:29.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:29.532+0000 7f07ea02b740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f07e01b1180 con 0x555c0ac25e50 2024-09-10T09:43:29.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:29.532+0000 7f07ea02b740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f07e01b2cc0 con 0x555c0ac1ac70 2024-09-10T09:43:29.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:29.532+0000 7f07ea02b740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f07e01b1850 con 0x555c0ac1b590 2024-09-10T09:43:29.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:29.533+0000 7f07dfe00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555c0ac1b590 0x555c0ac3dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:43:29.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:29.533+0000 7f07e4a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555c0ac1ac70 0x555c0ac1d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:43:29.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:29.533+0000 7f07dfe00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555c0ac1b590 0x555c0ac3dc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:40372/0 (socket says 172.21.15.5:40372) 2024-09-10T09:43:29.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:29.533+0000 7f07dfe00640 1 -- 172.21.15.5:0/1453688326 learned_addr learned my addr 172.21.15.5:0/1453688326 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:43:29.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:29.533+0000 7f07e5400640 1 --2- 172.21.15.5:0/1453688326 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555c0ac25e50 0x555c0ac3e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:43:29.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:29.533+0000 7f07dfe00640 1 -- 172.21.15.5:0/1453688326 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555c0ac1ac70 msgr2=0x555c0ac1d720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:43:29.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:29.533+0000 7f07dfe00640 1 --2- 172.21.15.5:0/1453688326 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555c0ac1ac70 0x555c0ac1d720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:29.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:29.533+0000 7f07dfe00640 1 -- 172.21.15.5:0/1453688326 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555c0ac25e50 msgr2=0x555c0ac3e150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:43:29.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:29.533+0000 7f07dfe00640 1 --2- 172.21.15.5:0/1453688326 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555c0ac25e50 0x555c0ac3e150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:29.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:29.533+0000 7f07dfe00640 1 -- 172.21.15.5:0/1453688326 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555c0ac12770 con 0x555c0ac1b590 2024-09-10T09:43:29.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:29.533+0000 7f07e4a00640 1 --2- 172.21.15.5:0/1453688326 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555c0ac1ac70 0x555c0ac1d720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:43:29.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:29.533+0000 7f07e5400640 1 --2- 172.21.15.5:0/1453688326 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555c0ac25e50 0x555c0ac3e150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:43:29.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:29.533+0000 7f07dfe00640 1 --2- 172.21.15.5:0/1453688326 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555c0ac1b590 0x555c0ac3dc10 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f07cc00e9c0 tx=0x7f07cc000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=800faaeb08fff7d5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:43:29.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:29.534+0000 7f07d1200640 1 -- 172.21.15.5:0/1453688326 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07cc09bb20 con 0x555c0ac1b590 2024-09-10T09:43:29.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:29.534+0000 7f07d1200640 1 -- 172.21.15.5:0/1453688326 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f07cc09ce50 con 0x555c0ac1b590 2024-09-10T09:43:29.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:29.534+0000 7f07d1200640 1 -- 172.21.15.5:0/1453688326 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07cc097940 con 0x555c0ac1b590 2024-09-10T09:43:29.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:29.534+0000 7f07ea02b740 1 -- 172.21.15.5:0/1453688326 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555c0ac1b590 msgr2=0x555c0ac3dc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:43:29.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:29.534+0000 7f07ea02b740 1 --2- 172.21.15.5:0/1453688326 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555c0ac1b590 0x555c0ac3dc10 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f07cc00e9c0 tx=0x7f07cc000bc0 comp rx=0 tx=0).stop 2024-09-10T09:43:29.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:29.534+0000 7f07ea02b740 1 -- 172.21.15.5:0/1453688326 shutdown_connections 2024-09-10T09:43:29.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:29.534+0000 7f07ea02b740 1 --2- 172.21.15.5:0/1453688326 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555c0ac25e50 0x555c0ac3e150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:29.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:29.534+0000 7f07ea02b740 1 --2- 172.21.15.5:0/1453688326 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555c0ac1b590 0x555c0ac3dc10 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:29.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:29.534+0000 7f07ea02b740 1 --2- 172.21.15.5:0/1453688326 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555c0ac1ac70 0x555c0ac1d720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:29.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:29.534+0000 7f07ea02b740 1 -- 172.21.15.5:0/1453688326 >> 172.21.15.5:0/1453688326 conn(0x555c0ac17200 msgr2=0x555c0ac24d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:43:29.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:29.535+0000 7f07ea02b740 1 -- 172.21.15.5:0/1453688326 shutdown_connections 2024-09-10T09:43:29.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:29.535+0000 7f07ea02b740 1 -- 172.21.15.5:0/1453688326 wait complete. 2024-09-10T09:43:29.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:29.535+0000 7f07ea02b740 1 -- start start 2024-09-10T09:43:29.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:29.536+0000 7f07ea02b740 1 Processor -- start 2024-09-10T09:43:29.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:29.536+0000 7f07ea02b740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555c0ac1ac70 0x555c0ac1fea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:43:29.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:29.536+0000 7f07ea02b740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555c0ac1b590 0x555c0ac203e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:43:29.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:29.536+0000 7f07ea02b740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555c0ac25e50 0x555c0ac20920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:43:29.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:29.536+0000 7f07ea02b740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f07e01b25f0 con 0x555c0ac1b590 2024-09-10T09:43:29.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:29.536+0000 7f07ea02b740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f07e01b1f20 con 0x555c0ac25e50 2024-09-10T09:43:29.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:29.536+0000 7f07ea02b740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f07e01af160 con 0x555c0ac1ac70 2024-09-10T09:43:29.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:29.536+0000 7f07ea02b740 10 client.0 _ll_register_callbacks cb 0x7f07cc0d5220 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:43:29.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:29.537+0000 7f07e4a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555c0ac1ac70 0x555c0ac1fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:43:29.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:29.537+0000 7f07dfe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555c0ac1b590 0x555c0ac203e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:43:29.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:29.537+0000 7f07e5400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555c0ac25e50 0x555c0ac20920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:43:29.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:29.537+0000 7f07e4a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555c0ac1ac70 0x555c0ac1fea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:40382/0 (socket says 172.21.15.5:40382) 2024-09-10T09:43:29.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:29.537+0000 7f07e4a00640 1 -- 172.21.15.5:0/2395559489 learned_addr learned my addr 172.21.15.5:0/2395559489 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:43:29.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:29.537+0000 7f07e4a00640 1 -- 172.21.15.5:0/2395559489 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555c0ac25e50 msgr2=0x555c0ac20920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:43:29.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:29.537+0000 7f07e4a00640 1 --2- 172.21.15.5:0/2395559489 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555c0ac25e50 0x555c0ac20920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:29.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:29.537+0000 7f07e4a00640 1 -- 172.21.15.5:0/2395559489 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555c0ac1b590 msgr2=0x555c0ac203e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:43:29.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:29.537+0000 7f07e4a00640 1 --2- 172.21.15.5:0/2395559489 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555c0ac1b590 0x555c0ac203e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:29.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:29.537+0000 7f07e4a00640 1 -- 172.21.15.5:0/2395559489 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555c0ac9fd10 con 0x555c0ac1ac70 2024-09-10T09:43:29.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:29.537+0000 7f07e5400640 1 --2- 172.21.15.5:0/2395559489 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555c0ac25e50 0x555c0ac20920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:43:29.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:29.537+0000 7f07e4a00640 1 --2- 172.21.15.5:0/2395559489 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555c0ac1ac70 0x555c0ac1fea0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f07d4019a20 tx=0x7f07d40199b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c916616d14564f43 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:43:29.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:29.537+0000 7f07c6000640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-09-10T09:43:29.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:29.538+0000 7f07c6000640 1 -- 172.21.15.5:0/2395559489 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07d4048070 con 0x555c0ac1ac70 2024-09-10T09:43:29.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:29.538+0000 7f07ea02b740 1 -- 172.21.15.5:0/2395559489 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f07cc09ce50 con 0x555c0ac1ac70 2024-09-10T09:43:29.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:29.538+0000 7f07c6000640 1 -- 172.21.15.5:0/2395559489 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f07d4004250 con 0x555c0ac1ac70 2024-09-10T09:43:29.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:29.538+0000 7f07c6000640 1 -- 172.21.15.5:0/2395559489 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f07d4053050 con 0x555c0ac1ac70 2024-09-10T09:43:29.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:29.538+0000 7f07ea02b740 10 client.26446 fetch_fsmap learned FSMap version 34 2024-09-10T09:43:29.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:29.538+0000 7f07ea02b740 1 -- 172.21.15.5:0/2395559489 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x555c0aca1050 con 0x555c0ac1ac70 2024-09-10T09:43:29.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:29.538+0000 7f07c6000640 1 -- 172.21.15.5:0/2395559489 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f07cc09ce50 con 0x555c0ac1ac70 2024-09-10T09:43:29.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:29.539+0000 7f07c6000640 1 -- 172.21.15.5:0/2395559489 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f07d4003950 con 0x555c0ac1ac70 2024-09-10T09:43:29.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:29.539+0000 7f07ea02b740 10 client.26446 fetch_fsmap finished waiting for FSMap version 34 2024-09-10T09:43:29.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:29.539+0000 7f07ea02b740 10 client.26446 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-10T09:43:29.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:29.539+0000 7f07ea02b740 20 client.26446 populate_metadata read hostname 'smithi005' 2024-09-10T09:43:29.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:29.539+0000 7f07ea02b740 1 --2- 172.21.15.5:0/2395559489 >> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] conn(0x555c0aca1590 0x555c0aca39b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:43:29.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:29.539+0000 7f07ea02b740 4 client.26446 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T09:43:29.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:29.539+0000 7f07ea02b740 1 -- 172.21.15.5:0/2395559489 --> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555c0ab88e60 con 0x555c0aca1590 2024-09-10T09:43:29.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:29.539+0000 7f07dfe00640 1 --2- 172.21.15.5:0/2395559489 >> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] conn(0x555c0aca1590 0x555c0aca39b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:43:29.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:29.539+0000 7f07dfe00640 1 --2- 172.21.15.5:0/2395559489 >> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] conn(0x555c0aca1590 0x555c0aca39b0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f9d627fb4e61139b server_cookie=f6bed42f79c17377 in_seq=0 out_seq=0 2024-09-10T09:43:29.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:29.539+0000 7f07c6000640 10 client.26446 ms_handle_connect on v2:172.21.15.159:6838/2073739599 2024-09-10T09:43:29.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:29.540+0000 7f07c6000640 1 -- 172.21.15.5:0/2395559489 <== mds.1 v2:172.21.15.159:6838/2073739599 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555c0ab88e60 con 0x555c0aca1590 2024-09-10T09:43:29.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:29.540+0000 7f07c6000640 10 client.26446 handle_command_reply: tid=0 2024-09-10T09:43:29.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:29.540+0000 7f07ea02b740 1 client.26446 shutdown 2024-09-10T09:43:29.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:29.540+0000 7f07ea02b740 1 -- 172.21.15.5:0/2395559489 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555c0ac1ac70 msgr2=0x555c0ac1fea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:43:29.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:29.540+0000 7f07ea02b740 1 --2- 172.21.15.5:0/2395559489 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555c0ac1ac70 0x555c0ac1fea0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f07d4019a20 tx=0x7f07d40199b0 comp rx=0 tx=0).stop 2024-09-10T09:43:29.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:29.540+0000 7f07ea02b740 1 -- 172.21.15.5:0/2395559489 shutdown_connections 2024-09-10T09:43:29.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:29.540+0000 7f07ea02b740 1 --2- 172.21.15.5:0/2395559489 >> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] conn(0x555c0aca1590 0x555c0aca39b0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:29.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:29.540+0000 7f07ea02b740 1 --2- 172.21.15.5:0/2395559489 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555c0ac25e50 0x555c0ac20920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:29.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:29.540+0000 7f07ea02b740 1 --2- 172.21.15.5:0/2395559489 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555c0ac1b590 0x555c0ac203e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:29.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:29.540+0000 7f07c6000640 0 client.26446 ms_handle_reset on v2:172.21.15.159:6838/2073739599 2024-09-10T09:43:29.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:29.540+0000 7f07ea02b740 1 --2- 172.21.15.5:0/2395559489 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555c0ac1ac70 0x555c0ac1fea0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:29.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:29.540+0000 7f07ea02b740 1 -- 172.21.15.5:0/2395559489 >> 172.21.15.5:0/2395559489 conn(0x555c0ac17200 msgr2=0x555c0ac24d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:43:29.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:29.540+0000 7f07ea02b740 1 -- 172.21.15.5:0/2395559489 shutdown_connections 2024-09-10T09:43:29.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:29.540+0000 7f07ea02b740 1 -- 172.21.15.5:0/2395559489 wait complete. 2024-09-10T09:43:29.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:29.540+0000 7f07ea02b740 20 client.26446 trim_cache size 0 max 16384 2024-09-10T09:43:29.548 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-10T09:43:29.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:29.541+0000 7f07e7400640 1 -- 172.21.15.5:0/538222507 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f07c80a2b10 msgr2=0x7f07c80a4fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:43:29.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:29.541+0000 7f07e7400640 1 --2- 172.21.15.5:0/538222507 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f07c80a2b10 0x7f07c80a4fd0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f07d400ff90 tx=0x7f07d4009ca0 comp rx=0 tx=0).stop 2024-09-10T09:43:29.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:29.541+0000 7f07e7400640 1 -- 172.21.15.5:0/538222507 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f07e0074ee0 msgr2=0x7f07e019aba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:43:29.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:29.541+0000 7f07e7400640 1 --2- 172.21.15.5:0/538222507 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f07e0074ee0 0x7f07e019aba0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f07cc00e9f0 tx=0x7f07cc00eeb0 comp rx=0 tx=0).stop 2024-09-10T09:43:29.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:29.541+0000 7f07e7400640 1 -- 172.21.15.5:0/538222507 shutdown_connections 2024-09-10T09:43:29.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:29.541+0000 7f07e7400640 1 --2- 172.21.15.5:0/538222507 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f07c80a2b10 0x7f07c80a4fd0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:29.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:29.541+0000 7f07e7400640 1 --2- 172.21.15.5:0/538222507 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f07e010e430 0x7f07e019ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:29.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:29.541+0000 7f07e7400640 1 --2- 172.21.15.5:0/538222507 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f07e0074ee0 0x7f07e019aba0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:29.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:29.541+0000 7f07e7400640 1 --2- 172.21.15.5:0/538222507 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f07e00745c0 0x7f07e019a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:29.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:29.541+0000 7f07e7400640 1 -- 172.21.15.5:0/538222507 >> 172.21.15.5:0/538222507 conn(0x7f07e00fd8b0 msgr2=0x7f07e00fe510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:43:29.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:29.541+0000 7f07e7400640 1 -- 172.21.15.5:0/538222507 shutdown_connections 2024-09-10T09:43:29.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:29.541+0000 7f07e7400640 1 -- 172.21.15.5:0/538222507 wait complete. 2024-09-10T09:43:29.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:43:29 smithi005 ceph-mon[30350]: pgmap v547: 129 pgs: 129 active+clean; 105 MiB data, 936 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.6 MiB/s wr, 186 op/s 2024-09-10T09:43:29.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:43:29 smithi102 ceph-mon[35442]: pgmap v547: 129 pgs: 129 active+clean; 105 MiB data, 936 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.6 MiB/s wr, 186 op/s 2024-09-10T09:43:30.040 INFO:tasks.workunit.client.0.smithi005.stdout: 1 250155 4.38 MB/sec execute 71 sec latency 31.418 ms 2024-09-10T09:43:30.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:43:29 smithi159 ceph-mon[35513]: pgmap v547: 129 pgs: 129 active+clean; 105 MiB data, 936 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.6 MiB/s wr, 186 op/s 2024-09-10T09:43:30.585 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-10T09:43:30.986 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:43:31.040 INFO:tasks.workunit.client.0.smithi005.stdout: 1 252573 4.52 MB/sec execute 72 sec latency 25.645 ms 2024-09-10T09:43:31.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:31.597+0000 7f1ff0a00640 1 -- 172.21.15.5:0/1209224330 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1fec1019c0 msgr2=0x7f1fec101e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:43:31.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:31.597+0000 7f1ff0a00640 1 --2- 172.21.15.5:0/1209224330 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1fec1019c0 0x7f1fec101e20 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f1fd4009890 tx=0x7f1fd402a370 comp rx=0 tx=0).stop 2024-09-10T09:43:31.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:31.597+0000 7f1ff0a00640 1 -- 172.21.15.5:0/1209224330 shutdown_connections 2024-09-10T09:43:31.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:31.597+0000 7f1ff0a00640 1 --2- 172.21.15.5:0/1209224330 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1fec102360 0x7f1fec10e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:31.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:31.597+0000 7f1ff0a00640 1 --2- 172.21.15.5:0/1209224330 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1fec1019c0 0x7f1fec101e20 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:31.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:31.597+0000 7f1ff0a00640 1 --2- 172.21.15.5:0/1209224330 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1fec1079c0 0x7f1fec107da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:31.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:31.597+0000 7f1ff0a00640 1 -- 172.21.15.5:0/1209224330 >> 172.21.15.5:0/1209224330 conn(0x7f1fec0fd740 msgr2=0x7f1fec0ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:43:31.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:31.597+0000 7f1ff0a00640 1 -- 172.21.15.5:0/1209224330 shutdown_connections 2024-09-10T09:43:31.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:31.597+0000 7f1ff0a00640 1 -- 172.21.15.5:0/1209224330 wait complete. 2024-09-10T09:43:31.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:31.599+0000 7f1ff0a00640 1 Processor -- start 2024-09-10T09:43:31.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:31.599+0000 7f1ff0a00640 1 -- start start 2024-09-10T09:43:31.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:31.599+0000 7f1ff0a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1fec1019c0 0x7f1fec1982d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:43:31.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:31.599+0000 7f1ff0a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1fec102360 0x7f1fec198810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:43:31.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:31.599+0000 7f1ff0a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1fec1079c0 0x7f1fec19cba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:43:31.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:31.599+0000 7f1ff0a00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1fec111dc0 con 0x7f1fec1079c0 2024-09-10T09:43:31.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:31.599+0000 7f1ff0a00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f1fec111c40 con 0x7f1fec102360 2024-09-10T09:43:31.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:31.599+0000 7f1ff0a00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f1fec111f40 con 0x7f1fec1019c0 2024-09-10T09:43:31.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:31.600+0000 7f1fe3e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1fec102360 0x7f1fec198810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:43:31.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:31.600+0000 7f1feb400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1fec1019c0 0x7f1fec1982d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:43:31.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:31.600+0000 7f1fe3e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1fec102360 0x7f1fec198810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:37236/0 (socket says 172.21.15.5:37236) 2024-09-10T09:43:31.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:31.600+0000 7f1fe3e00640 1 -- 172.21.15.5:0/3522188415 learned_addr learned my addr 172.21.15.5:0/3522188415 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:43:31.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:31.600+0000 7f1febe00640 1 --2- 172.21.15.5:0/3522188415 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1fec1079c0 0x7f1fec19cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:43:31.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:31.600+0000 7f1feb400640 1 -- 172.21.15.5:0/3522188415 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1fec102360 msgr2=0x7f1fec198810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:43:31.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:31.600+0000 7f1feb400640 1 --2- 172.21.15.5:0/3522188415 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1fec102360 0x7f1fec198810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:31.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:31.600+0000 7f1feb400640 1 -- 172.21.15.5:0/3522188415 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1fec1079c0 msgr2=0x7f1fec19cba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:43:31.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:31.600+0000 7f1feb400640 1 --2- 172.21.15.5:0/3522188415 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1fec1079c0 0x7f1fec19cba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:31.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:31.600+0000 7f1feb400640 1 -- 172.21.15.5:0/3522188415 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fec19d280 con 0x7f1fec1019c0 2024-09-10T09:43:31.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:31.600+0000 7f1fe3e00640 1 --2- 172.21.15.5:0/3522188415 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1fec102360 0x7f1fec198810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:43:31.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:31.600+0000 7f1feb400640 1 --2- 172.21.15.5:0/3522188415 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1fec1019c0 0x7f1fec1982d0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f1fdc00e790 tx=0x7f1fdc00ec50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:43:31.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:31.601+0000 7f1fe8c00640 1 -- 172.21.15.5:0/3522188415 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fdc00eea0 con 0x7f1fec1019c0 2024-09-10T09:43:31.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:31.601+0000 7f1ff0a00640 1 -- 172.21.15.5:0/3522188415 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1fec19d570 con 0x7f1fec1019c0 2024-09-10T09:43:31.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:31.601+0000 7f1fe8c00640 1 -- 172.21.15.5:0/3522188415 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1fdc00ccc0 con 0x7f1fec1019c0 2024-09-10T09:43:31.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:31.601+0000 7f1fe8c00640 1 -- 172.21.15.5:0/3522188415 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fdc018660 con 0x7f1fec1019c0 2024-09-10T09:43:31.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:31.601+0000 7f1ff0a00640 1 -- 172.21.15.5:0/3522188415 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1fec1a4e50 con 0x7f1fec1019c0 2024-09-10T09:43:31.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:31.604+0000 7f1fe8c00640 1 -- 172.21.15.5:0/3522188415 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f1fdc010430 con 0x7f1fec1019c0 2024-09-10T09:43:31.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:31.604+0000 7f1ff4a02740 1 Processor -- start 2024-09-10T09:43:31.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:31.604+0000 7f1ff4a02740 1 -- start start 2024-09-10T09:43:31.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:31.604+0000 7f1ff4a02740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558dc3ccb250 0x558dc3ccb630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:43:31.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:31.605+0000 7f1ff4a02740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558dc3ccbb70 0x558dc3cd5ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:43:31.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:31.605+0000 7f1ff4a02740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558dc3cd6430 0x558dc3cd8820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:43:31.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:31.605+0000 7f1ff4a02740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x558dc3c2d3e0 con 0x558dc3cd6430 2024-09-10T09:43:31.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:31.605+0000 7f1ff4a02740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x558dc3bd4d80 con 0x558dc3ccb250 2024-09-10T09:43:31.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:31.605+0000 7f1ff4a02740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x558dc3c293a0 con 0x558dc3ccbb70 2024-09-10T09:43:31.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:31.605+0000 7f1febe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558dc3ccb250 0x558dc3ccb630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:43:31.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:31.605+0000 7f1febe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558dc3ccb250 0x558dc3ccb630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:37242/0 (socket says 172.21.15.5:37242) 2024-09-10T09:43:31.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:31.605+0000 7f1febe00640 1 -- 172.21.15.5:0/579647676 learned_addr learned my addr 172.21.15.5:0/579647676 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:43:31.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:31.605+0000 7f1feb400640 1 --2- 172.21.15.5:0/579647676 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558dc3ccbb70 0x558dc3cd5ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:43:31.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:31.605+0000 7f1fe3e00640 1 --2- 172.21.15.5:0/579647676 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558dc3cd6430 0x558dc3cd8820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:43:31.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:31.605+0000 7f1febe00640 1 -- 172.21.15.5:0/579647676 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558dc3ccbb70 msgr2=0x558dc3cd5ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:43:31.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:31.605+0000 7f1febe00640 1 --2- 172.21.15.5:0/579647676 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558dc3ccbb70 0x558dc3cd5ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:31.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:31.605+0000 7f1febe00640 1 -- 172.21.15.5:0/579647676 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558dc3cd6430 msgr2=0x558dc3cd8820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:43:31.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:31.605+0000 7f1febe00640 1 --2- 172.21.15.5:0/579647676 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558dc3cd6430 0x558dc3cd8820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:31.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:31.605+0000 7f1febe00640 1 -- 172.21.15.5:0/579647676 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558dc3bf3ae0 con 0x558dc3ccb250 2024-09-10T09:43:31.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:31.605+0000 7f1feb400640 1 --2- 172.21.15.5:0/579647676 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558dc3ccbb70 0x558dc3cd5ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:43:31.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:31.606+0000 7f1febe00640 1 --2- 172.21.15.5:0/579647676 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558dc3ccb250 0x558dc3ccb630 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f1fe4009fc0 tx=0x7f1fe4010860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1ef753410318b534 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:43:31.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:31.606+0000 7f1fdbe00640 1 -- 172.21.15.5:0/579647676 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fe4003b70 con 0x558dc3ccb250 2024-09-10T09:43:31.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:31.606+0000 7f1fdbe00640 1 -- 172.21.15.5:0/579647676 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1fe4003d10 con 0x558dc3ccb250 2024-09-10T09:43:31.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:31.606+0000 7f1fdbe00640 1 -- 172.21.15.5:0/579647676 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fe4011ab0 con 0x558dc3ccb250 2024-09-10T09:43:31.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:31.607+0000 7f1ff4a02740 1 -- 172.21.15.5:0/579647676 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558dc3ccb250 msgr2=0x558dc3ccb630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:43:31.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:31.607+0000 7f1ff4a02740 1 --2- 172.21.15.5:0/579647676 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558dc3ccb250 0x558dc3ccb630 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f1fe4009fc0 tx=0x7f1fe4010860 comp rx=0 tx=0).stop 2024-09-10T09:43:31.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:31.607+0000 7f1fe8c00640 1 --2- 172.21.15.5:0/3522188415 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f1fd00a2c00 0x7f1fd00a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:43:31.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:31.607+0000 7f1ff4a02740 1 -- 172.21.15.5:0/579647676 shutdown_connections 2024-09-10T09:43:31.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:31.607+0000 7f1ff4a02740 1 --2- 172.21.15.5:0/579647676 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558dc3cd6430 0x558dc3cd8820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:31.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:31.607+0000 7f1ff4a02740 1 --2- 172.21.15.5:0/579647676 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558dc3ccbb70 0x558dc3cd5ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:31.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:31.607+0000 7f1ff4a02740 1 --2- 172.21.15.5:0/579647676 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558dc3ccb250 0x558dc3ccb630 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:31.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:31.607+0000 7f1ff4a02740 1 -- 172.21.15.5:0/579647676 >> 172.21.15.5:0/579647676 conn(0x558dc3cc77e0 msgr2=0x558dc3cc7bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:43:31.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:31.607+0000 7f1fe8c00640 1 -- 172.21.15.5:0/3522188415 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f1fdc0cf6a0 con 0x7f1fec1019c0 2024-09-10T09:43:31.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:31.607+0000 7f1febe00640 1 --2- 172.21.15.5:0/3522188415 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f1fd00a2c00 0x7f1fd00a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:43:31.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:31.607+0000 7f1ff4a02740 1 -- 172.21.15.5:0/579647676 shutdown_connections 2024-09-10T09:43:31.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:31.607+0000 7f1ff4a02740 1 -- 172.21.15.5:0/579647676 wait complete. 2024-09-10T09:43:31.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:31.608+0000 7f1ff4a02740 1 -- start start 2024-09-10T09:43:31.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:31.608+0000 7f1febe00640 1 --2- 172.21.15.5:0/3522188415 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f1fd00a2c00 0x7f1fd00a50c0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f1fe4009f90 tx=0x7f1fe4010fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:43:31.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:31.608+0000 7f1ff4a02740 1 Processor -- start 2024-09-10T09:43:31.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:31.608+0000 7f1ff4a02740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558dc3ccb250 0x558dc3cd3330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:43:31.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:31.608+0000 7f1ff4a02740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558dc3ccbb70 0x558dc3cd3870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:43:31.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:31.608+0000 7f1ff4a02740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558dc3cd6430 0x558dc3cd3db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:43:31.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:31.608+0000 7f1ff4a02740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x558dc3cd9000 con 0x558dc3ccb250 2024-09-10T09:43:31.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:31.608+0000 7f1ff4a02740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x558dc3be80f0 con 0x558dc3cd6430 2024-09-10T09:43:31.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:31.608+0000 7f1ff4a02740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x558dc3cd9180 con 0x558dc3ccbb70 2024-09-10T09:43:31.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:31.608+0000 7f1ff4a02740 10 client.0 _ll_register_callbacks cb 0x558dc3bcfbd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:43:31.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:31.609+0000 7f1fe3e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558dc3ccbb70 0x558dc3cd3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:43:31.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:31.609+0000 7f1febe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558dc3cd6430 0x558dc3cd3db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:43:31.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:31.609+0000 7f1fe3e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558dc3ccbb70 0x558dc3cd3870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:38058/0 (socket says 172.21.15.5:38058) 2024-09-10T09:43:31.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:31.609+0000 7f1fe3e00640 1 -- 172.21.15.5:0/2619927926 learned_addr learned my addr 172.21.15.5:0/2619927926 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:43:31.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:31.609+0000 7f1feb400640 1 --2- 172.21.15.5:0/2619927926 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558dc3ccb250 0x558dc3cd3330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:43:31.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:31.609+0000 7f1fe3e00640 1 -- 172.21.15.5:0/2619927926 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558dc3cd6430 msgr2=0x558dc3cd3db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:43:31.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:31.609+0000 7f1fe3e00640 1 --2- 172.21.15.5:0/2619927926 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558dc3cd6430 0x558dc3cd3db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:31.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:31.609+0000 7f1fe3e00640 1 -- 172.21.15.5:0/2619927926 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558dc3ccb250 msgr2=0x558dc3cd3330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:43:31.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:31.609+0000 7f1fe3e00640 1 --2- 172.21.15.5:0/2619927926 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558dc3ccb250 0x558dc3cd3330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:31.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:31.609+0000 7f1fe3e00640 1 -- 172.21.15.5:0/2619927926 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558dc399e3c0 con 0x558dc3ccbb70 2024-09-10T09:43:31.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:31.609+0000 7f1febe00640 1 --2- 172.21.15.5:0/2619927926 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558dc3cd6430 0x558dc3cd3db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:43:31.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:31.609+0000 7f1fe3e00640 1 --2- 172.21.15.5:0/2619927926 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558dc3ccbb70 0x558dc3cd3870 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f1fd40309f0 tx=0x7f1fd4030a20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=92cbf0944b11ea8c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:43:31.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:31.609+0000 7f1fd8c00640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-09-10T09:43:31.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:31.610+0000 7f1fd8c00640 1 -- 172.21.15.5:0/2619927926 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fd4037040 con 0x558dc3ccbb70 2024-09-10T09:43:31.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:31.610+0000 7f1ff4a02740 1 -- 172.21.15.5:0/2619927926 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1fe4003d10 con 0x558dc3ccbb70 2024-09-10T09:43:31.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:31.610+0000 7f1fd8c00640 1 -- 172.21.15.5:0/2619927926 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f1fd4004460 con 0x558dc3ccbb70 2024-09-10T09:43:31.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:31.610+0000 7f1fd8c00640 1 -- 172.21.15.5:0/2619927926 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f1fd404b050 con 0x558dc3ccbb70 2024-09-10T09:43:31.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:31.610+0000 7f1ff4a02740 10 client.26458 fetch_fsmap learned FSMap version 34 2024-09-10T09:43:31.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:31.610+0000 7f1ff4a02740 1 -- 172.21.15.5:0/2619927926 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x558dc3ce50a0 con 0x558dc3ccbb70 2024-09-10T09:43:31.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:31.610+0000 7f1fd8c00640 1 -- 172.21.15.5:0/2619927926 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1fe4003d10 con 0x558dc3ccbb70 2024-09-10T09:43:31.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:31.610+0000 7f1fd8c00640 1 -- 172.21.15.5:0/2619927926 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f1fd403d750 con 0x558dc3ccbb70 2024-09-10T09:43:31.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:31.611+0000 7f1ff4a02740 10 client.26458 fetch_fsmap finished waiting for FSMap version 34 2024-09-10T09:43:31.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:31.611+0000 7f1ff4a02740 10 client.26458 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T09:43:31.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:31.611+0000 7f1ff4a02740 20 client.26458 populate_metadata read hostname 'smithi005' 2024-09-10T09:43:31.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:31.611+0000 7f1ff4a02740 1 --2- 172.21.15.5:0/2619927926 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x558dc3ce64b0 0x558dc3ce8950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:43:31.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:31.611+0000 7f1ff4a02740 4 client.26458 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:43:31.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:31.611+0000 7f1ff4a02740 1 -- 172.21.15.5:0/2619927926 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558dc3c3a950 con 0x558dc3ce64b0 2024-09-10T09:43:31.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:31.611+0000 7f1feb400640 1 --2- 172.21.15.5:0/2619927926 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x558dc3ce64b0 0x558dc3ce8950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:43:31.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:31.611+0000 7f1feb400640 1 --2- 172.21.15.5:0/2619927926 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x558dc3ce64b0 0x558dc3ce8950 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f586a200ddd52546 server_cookie=27577ac2a31baf60 in_seq=0 out_seq=0 2024-09-10T09:43:31.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:31.611+0000 7f1fd8c00640 10 client.26458 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T09:43:31.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:31.612+0000 7f1fd8c00640 1 -- 172.21.15.5:0/2619927926 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558dc3c3a950 con 0x558dc3ce64b0 2024-09-10T09:43:31.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:31.612+0000 7f1fd8c00640 10 client.26458 handle_command_reply: tid=0 2024-09-10T09:43:31.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:31.612+0000 7f1ff4a02740 1 client.26458 shutdown 2024-09-10T09:43:31.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:31.612+0000 7f1ff4a02740 1 -- 172.21.15.5:0/2619927926 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558dc3ccbb70 msgr2=0x558dc3cd3870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:43:31.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:31.612+0000 7f1ff4a02740 1 --2- 172.21.15.5:0/2619927926 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558dc3ccbb70 0x558dc3cd3870 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f1fd40309f0 tx=0x7f1fd4030a20 comp rx=0 tx=0).stop 2024-09-10T09:43:31.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:31.613+0000 7f1ff4a02740 1 -- 172.21.15.5:0/2619927926 shutdown_connections 2024-09-10T09:43:31.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:31.613+0000 7f1ff4a02740 1 --2- 172.21.15.5:0/2619927926 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x558dc3ce64b0 0x558dc3ce8950 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:31.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:31.613+0000 7f1ff4a02740 1 --2- 172.21.15.5:0/2619927926 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558dc3cd6430 0x558dc3cd3db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:31.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:31.613+0000 7f1ff4a02740 1 --2- 172.21.15.5:0/2619927926 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558dc3ccbb70 0x558dc3cd3870 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:31.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:31.613+0000 7f1ff4a02740 1 --2- 172.21.15.5:0/2619927926 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558dc3ccb250 0x558dc3cd3330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:31.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:31.613+0000 7f1fd8c00640 0 client.26458 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T09:43:31.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:31.613+0000 7f1ff4a02740 1 -- 172.21.15.5:0/2619927926 >> 172.21.15.5:0/2619927926 conn(0x558dc3cc77e0 msgr2=0x558dc3ccd4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:43:31.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:31.613+0000 7f1ff4a02740 1 -- 172.21.15.5:0/2619927926 shutdown_connections 2024-09-10T09:43:31.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:31.613+0000 7f1ff4a02740 1 -- 172.21.15.5:0/2619927926 wait complete. 2024-09-10T09:43:31.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:31.613+0000 7f1ff4a02740 20 client.26458 trim_cache size 0 max 16384 2024-09-10T09:43:31.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:31.629+0000 7f1ff4a02740 1 Processor -- start 2024-09-10T09:43:31.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:31.629+0000 7f1ff4a02740 1 -- start start 2024-09-10T09:43:31.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:31.629+0000 7f1ff4a02740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558dc3ccb250 0x558dc3cd2bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:43:31.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:31.629+0000 7f1ff4a02740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558dc3ccbb70 0x558dc3cd52a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:43:31.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:31.629+0000 7f1ff4a02740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558dc3cd6430 0x558dc3cee220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:43:31.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:31.629+0000 7f1ff4a02740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1fec1acbc0 con 0x558dc3ccb250 2024-09-10T09:43:31.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:31.629+0000 7f1ff4a02740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f1fec1ae700 con 0x558dc3cd6430 2024-09-10T09:43:31.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:31.629+0000 7f1ff4a02740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f1fec1ad290 con 0x558dc3ccbb70 2024-09-10T09:43:31.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:31.629+0000 7f1feb400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558dc3ccb250 0x558dc3cd2bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:43:31.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:31.629+0000 7f1fe3e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558dc3ccbb70 0x558dc3cd52a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:43:31.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:31.629+0000 7f1fe3e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558dc3ccbb70 0x558dc3cd52a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:38074/0 (socket says 172.21.15.5:38074) 2024-09-10T09:43:31.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:31.629+0000 7f1feb400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558dc3ccb250 0x558dc3cd2bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59546/0 (socket says 172.21.15.5:59546) 2024-09-10T09:43:31.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:31.629+0000 7f1fe3e00640 1 -- 172.21.15.5:0/2880498472 learned_addr learned my addr 172.21.15.5:0/2880498472 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:43:31.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:31.629+0000 7f1febe00640 1 --2- 172.21.15.5:0/2880498472 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558dc3cd6430 0x558dc3cee220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:43:31.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:31.630+0000 7f1fe3e00640 1 -- 172.21.15.5:0/2880498472 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558dc3cd6430 msgr2=0x558dc3cee220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:43:31.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:31.630+0000 7f1fe3e00640 1 --2- 172.21.15.5:0/2880498472 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558dc3cd6430 0x558dc3cee220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:31.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:31.630+0000 7f1fe3e00640 1 -- 172.21.15.5:0/2880498472 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558dc3ccb250 msgr2=0x558dc3cd2bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:43:31.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:31.630+0000 7f1fe3e00640 1 --2- 172.21.15.5:0/2880498472 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558dc3ccb250 0x558dc3cd2bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:31.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:31.630+0000 7f1fe3e00640 1 -- 172.21.15.5:0/2880498472 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558dc3cc8470 con 0x558dc3ccbb70 2024-09-10T09:43:31.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:31.630+0000 7f1fe3e00640 1 --2- 172.21.15.5:0/2880498472 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558dc3ccbb70 0x558dc3cd52a0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f1fd4035040 tx=0x7f1fd402a370 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7de9dfd59326ff43 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:43:31.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:31.630+0000 7f1fdb400640 1 -- 172.21.15.5:0/2880498472 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fd4054070 con 0x558dc3ccbb70 2024-09-10T09:43:31.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:31.630+0000 7f1fdb400640 1 -- 172.21.15.5:0/2880498472 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1fd4009450 con 0x558dc3ccbb70 2024-09-10T09:43:31.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:31.630+0000 7f1fdb400640 1 -- 172.21.15.5:0/2880498472 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fd4037040 con 0x558dc3ccbb70 2024-09-10T09:43:31.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:31.631+0000 7f1ff4a02740 1 -- 172.21.15.5:0/2880498472 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558dc3ccbb70 msgr2=0x558dc3cd52a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:43:31.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:31.631+0000 7f1ff4a02740 1 --2- 172.21.15.5:0/2880498472 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558dc3ccbb70 0x558dc3cd52a0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f1fd4035040 tx=0x7f1fd402a370 comp rx=0 tx=0).stop 2024-09-10T09:43:31.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:31.631+0000 7f1ff4a02740 1 -- 172.21.15.5:0/2880498472 shutdown_connections 2024-09-10T09:43:31.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:31.631+0000 7f1ff4a02740 1 --2- 172.21.15.5:0/2880498472 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558dc3cd6430 0x558dc3cee220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:31.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:31.631+0000 7f1ff4a02740 1 --2- 172.21.15.5:0/2880498472 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558dc3ccbb70 0x558dc3cd52a0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:31.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:31.631+0000 7f1ff4a02740 1 --2- 172.21.15.5:0/2880498472 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558dc3ccb250 0x558dc3cd2bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:31.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:31.631+0000 7f1ff4a02740 1 -- 172.21.15.5:0/2880498472 >> 172.21.15.5:0/2880498472 conn(0x558dc3cc77e0 msgr2=0x558dc3cd4b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:43:31.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:31.631+0000 7f1ff4a02740 1 -- 172.21.15.5:0/2880498472 shutdown_connections 2024-09-10T09:43:31.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:31.632+0000 7f1ff4a02740 1 -- 172.21.15.5:0/2880498472 wait complete. 2024-09-10T09:43:31.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:31.632+0000 7f1ff4a02740 1 -- start start 2024-09-10T09:43:31.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:31.632+0000 7f1ff4a02740 1 Processor -- start 2024-09-10T09:43:31.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:31.633+0000 7f1ff4a02740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558dc3ccb250 0x558dc3cda240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:43:31.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:31.633+0000 7f1ff4a02740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558dc3ccbb70 0x558dc3cda780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:43:31.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:31.633+0000 7f1ff4a02740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558dc3cd6430 0x558dc3cdacc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:43:31.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:31.633+0000 7f1ff4a02740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1fec1ae030 con 0x558dc3ccb250 2024-09-10T09:43:31.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:31.633+0000 7f1ff4a02740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f1fec1ad960 con 0x558dc3ccbb70 2024-09-10T09:43:31.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:31.633+0000 7f1ff4a02740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f1fec1aaba0 con 0x558dc3cd6430 2024-09-10T09:43:31.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:31.633+0000 7f1ff4a02740 10 client.0 _ll_register_callbacks cb 0x7f1fdc005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:43:31.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:31.633+0000 7f1feb400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558dc3ccb250 0x558dc3cda240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:43:31.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:31.633+0000 7f1fe3e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558dc3ccbb70 0x558dc3cda780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:43:31.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:31.633+0000 7f1feb400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558dc3ccb250 0x558dc3cda240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59556/0 (socket says 172.21.15.5:59556) 2024-09-10T09:43:31.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:31.633+0000 7f1feb400640 1 -- 172.21.15.5:0/3235506795 learned_addr learned my addr 172.21.15.5:0/3235506795 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:43:31.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:31.633+0000 7f1fe3e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558dc3ccbb70 0x558dc3cda780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:37270/0 (socket says 172.21.15.5:37270) 2024-09-10T09:43:31.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:31.633+0000 7f1febe00640 1 --2- 172.21.15.5:0/3235506795 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558dc3cd6430 0x558dc3cdacc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:43:31.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:31.633+0000 7f1fe3e00640 1 -- 172.21.15.5:0/3235506795 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558dc3cd6430 msgr2=0x558dc3cdacc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:43:31.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:31.633+0000 7f1fe3e00640 1 --2- 172.21.15.5:0/3235506795 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558dc3cd6430 0x558dc3cdacc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:31.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:31.633+0000 7f1fe3e00640 1 -- 172.21.15.5:0/3235506795 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558dc3ccb250 msgr2=0x558dc3cda240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:43:31.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:31.633+0000 7f1fe3e00640 1 --2- 172.21.15.5:0/3235506795 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558dc3ccb250 0x558dc3cda240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:31.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:31.633+0000 7f1fe3e00640 1 -- 172.21.15.5:0/3235506795 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558dc3d50350 con 0x558dc3ccbb70 2024-09-10T09:43:31.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:31.633+0000 7f1febe00640 1 --2- 172.21.15.5:0/3235506795 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558dc3cd6430 0x558dc3cdacc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:43:31.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:31.633+0000 7f1feb400640 1 --2- 172.21.15.5:0/3235506795 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558dc3ccb250 0x558dc3cda240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:43:31.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:31.634+0000 7f1fe3e00640 1 --2- 172.21.15.5:0/3235506795 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558dc3ccbb70 0x558dc3cda780 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x558dc3ce2cc0 tx=0x7f1fd4004490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f40fa5ec8e71c858 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:43:31.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:31.634+0000 7f1fd8c00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-09-10T09:43:31.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:31.634+0000 7f1fd8c00640 1 -- 172.21.15.5:0/3235506795 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fd4054070 con 0x558dc3ccbb70 2024-09-10T09:43:31.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:31.634+0000 7f1fd8c00640 1 -- 172.21.15.5:0/3235506795 <== mon.2 v2:172.21.15.102:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f1fd403d860 con 0x558dc3ccbb70 2024-09-10T09:43:31.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:31.634+0000 7f1ff4a02740 1 -- 172.21.15.5:0/3235506795 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1fd4009450 con 0x558dc3ccbb70 2024-09-10T09:43:31.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:31.634+0000 7f1fd8c00640 1 -- 172.21.15.5:0/3235506795 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f1fd405f050 con 0x558dc3ccbb70 2024-09-10T09:43:31.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:31.634+0000 7f1ff4a02740 10 client.26438 fetch_fsmap learned FSMap version 34 2024-09-10T09:43:31.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:31.634+0000 7f1ff4a02740 1 -- 172.21.15.5:0/3235506795 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x558dc3d515d0 con 0x558dc3ccbb70 2024-09-10T09:43:31.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:31.634+0000 7f1fd8c00640 1 -- 172.21.15.5:0/3235506795 <== mon.2 v2:172.21.15.102:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1fd4009450 con 0x558dc3ccbb70 2024-09-10T09:43:31.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:31.635+0000 7f1fd8c00640 1 -- 172.21.15.5:0/3235506795 <== mon.2 v2:172.21.15.102:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f1fd4004a60 con 0x558dc3ccbb70 2024-09-10T09:43:31.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:31.635+0000 7f1ff4a02740 10 client.26438 fetch_fsmap finished waiting for FSMap version 34 2024-09-10T09:43:31.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:31.635+0000 7f1ff4a02740 10 client.26438 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T09:43:31.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:31.635+0000 7f1ff4a02740 20 client.26438 populate_metadata read hostname 'smithi005' 2024-09-10T09:43:31.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:31.635+0000 7f1ff4a02740 1 --2- 172.21.15.5:0/3235506795 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x558dc3d51bd0 0x558dc3d53ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:43:31.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:31.635+0000 7f1ff4a02740 4 client.26438 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T09:43:31.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:31.635+0000 7f1ff4a02740 1 -- 172.21.15.5:0/3235506795 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558dc3c34e60 con 0x558dc3d51bd0 2024-09-10T09:43:31.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:31.635+0000 7f1feb400640 1 --2- 172.21.15.5:0/3235506795 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x558dc3d51bd0 0x558dc3d53ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:43:31.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:31.636+0000 7f1feb400640 1 --2- 172.21.15.5:0/3235506795 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x558dc3d51bd0 0x558dc3d53ff0 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9bf0064be69d2177 server_cookie=3493550617195ef6 in_seq=0 out_seq=0 2024-09-10T09:43:31.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:31.636+0000 7f1fd8c00640 10 client.26438 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T09:43:31.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:31.636+0000 7f1fd8c00640 1 -- 172.21.15.5:0/3235506795 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558dc3c34e60 con 0x558dc3d51bd0 2024-09-10T09:43:31.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:31.636+0000 7f1fd8c00640 10 client.26438 handle_command_reply: tid=0 2024-09-10T09:43:31.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:31.636+0000 7f1ff4a02740 1 client.26438 shutdown 2024-09-10T09:43:31.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:31.636+0000 7f1ff4a02740 1 -- 172.21.15.5:0/3235506795 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558dc3ccbb70 msgr2=0x558dc3cda780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:43:31.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:31.636+0000 7f1ff4a02740 1 --2- 172.21.15.5:0/3235506795 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558dc3ccbb70 0x558dc3cda780 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x558dc3ce2cc0 tx=0x7f1fd4004490 comp rx=0 tx=0).stop 2024-09-10T09:43:31.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:31.636+0000 7f1ff4a02740 1 -- 172.21.15.5:0/3235506795 shutdown_connections 2024-09-10T09:43:31.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:31.636+0000 7f1ff4a02740 1 --2- 172.21.15.5:0/3235506795 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x558dc3d51bd0 0x558dc3d53ff0 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:31.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:31.636+0000 7f1ff4a02740 1 --2- 172.21.15.5:0/3235506795 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558dc3cd6430 0x558dc3cdacc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:31.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:31.636+0000 7f1ff4a02740 1 --2- 172.21.15.5:0/3235506795 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558dc3ccbb70 0x558dc3cda780 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:31.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:31.636+0000 7f1ff4a02740 1 --2- 172.21.15.5:0/3235506795 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558dc3ccb250 0x558dc3cda240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:31.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:31.636+0000 7f1fd8c00640 0 client.26438 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T09:43:31.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:31.636+0000 7f1ff4a02740 1 -- 172.21.15.5:0/3235506795 >> 172.21.15.5:0/3235506795 conn(0x558dc3cc77e0 msgr2=0x558dc3cc2490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:43:31.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:31.636+0000 7f1ff4a02740 1 -- 172.21.15.5:0/3235506795 shutdown_connections 2024-09-10T09:43:31.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:31.636+0000 7f1ff4a02740 1 -- 172.21.15.5:0/3235506795 wait complete. 2024-09-10T09:43:31.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:31.636+0000 7f1ff4a02740 20 client.26438 trim_cache size 0 max 16384 2024-09-10T09:43:31.643 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-10T09:43:31.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:31.637+0000 7f1ff0a00640 1 -- 172.21.15.5:0/3522188415 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f1fd00a2c00 msgr2=0x7f1fd00a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:43:31.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:31.637+0000 7f1ff0a00640 1 --2- 172.21.15.5:0/3522188415 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f1fd00a2c00 0x7f1fd00a50c0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f1fe4009f90 tx=0x7f1fe4010fd0 comp rx=0 tx=0).stop 2024-09-10T09:43:31.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:31.637+0000 7f1ff0a00640 1 -- 172.21.15.5:0/3522188415 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1fec1019c0 msgr2=0x7f1fec1982d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:43:31.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:31.637+0000 7f1ff0a00640 1 --2- 172.21.15.5:0/3522188415 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1fec1019c0 0x7f1fec1982d0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f1fdc00e790 tx=0x7f1fdc00ec50 comp rx=0 tx=0).stop 2024-09-10T09:43:31.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:31.637+0000 7f1ff0a00640 1 -- 172.21.15.5:0/3522188415 shutdown_connections 2024-09-10T09:43:31.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:31.637+0000 7f1ff0a00640 1 --2- 172.21.15.5:0/3522188415 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f1fd00a2c00 0x7f1fd00a50c0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:31.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:31.637+0000 7f1ff0a00640 1 --2- 172.21.15.5:0/3522188415 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1fec1079c0 0x7f1fec19cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:31.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:31.637+0000 7f1ff0a00640 1 --2- 172.21.15.5:0/3522188415 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1fec102360 0x7f1fec198810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:31.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:31.637+0000 7f1ff0a00640 1 --2- 172.21.15.5:0/3522188415 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1fec1019c0 0x7f1fec1982d0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:31.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:31.637+0000 7f1ff0a00640 1 -- 172.21.15.5:0/3522188415 >> 172.21.15.5:0/3522188415 conn(0x7f1fec0fd740 msgr2=0x7f1fec0fe090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:43:31.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:31.637+0000 7f1ff0a00640 1 -- 172.21.15.5:0/3522188415 shutdown_connections 2024-09-10T09:43:31.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:31.637+0000 7f1ff0a00640 1 -- 172.21.15.5:0/3522188415 wait complete. 2024-09-10T09:43:31.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:43:31 smithi005 ceph-mon[30350]: pgmap v548: 129 pgs: 129 active+clean; 109 MiB data, 963 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.6 MiB/s wr, 204 op/s 2024-09-10T09:43:31.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:43:31 smithi102 ceph-mon[35442]: pgmap v548: 129 pgs: 129 active+clean; 109 MiB data, 963 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.6 MiB/s wr, 204 op/s 2024-09-10T09:43:32.040 INFO:tasks.workunit.client.0.smithi005.stdout: 1 255065 4.63 MB/sec execute 73 sec latency 25.780 ms 2024-09-10T09:43:32.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:43:31 smithi159 ceph-mon[35513]: pgmap v548: 129 pgs: 129 active+clean; 109 MiB data, 963 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.6 MiB/s wr, 204 op/s 2024-09-10T09:43:33.040 INFO:tasks.workunit.client.0.smithi005.stdout: 1 256893 4.72 MB/sec execute 74 sec latency 22.450 ms 2024-09-10T09:43:33.397 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-10T09:43:33.398 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-10T09:43:33.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:43:33 smithi005 ceph-mon[30350]: pgmap v549: 129 pgs: 129 active+clean; 111 MiB data, 968 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.3 MiB/s wr, 190 op/s 2024-09-10T09:43:33.819 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:43:33.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:43:33 smithi102 ceph-mon[35442]: pgmap v549: 129 pgs: 129 active+clean; 111 MiB data, 968 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.3 MiB/s wr, 190 op/s 2024-09-10T09:43:34.040 INFO:tasks.workunit.client.0.smithi005.stdout: 1 259342 4.84 MB/sec execute 75 sec latency 16.121 ms 2024-09-10T09:43:34.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:43:33 smithi159 ceph-mon[35513]: pgmap v549: 129 pgs: 129 active+clean; 111 MiB data, 968 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.3 MiB/s wr, 190 op/s 2024-09-10T09:43:34.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:34.587+0000 7fcfdb400640 1 -- 172.21.15.5:0/1139998017 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcfd4104010 msgr2=0x7fcfd41043f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:43:34.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:34.587+0000 7fcfdb400640 1 --2- 172.21.15.5:0/1139998017 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcfd4104010 0x7fcfd41043f0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fcfc8009900 tx=0x7fcfc802a400 comp rx=0 tx=0).stop 2024-09-10T09:43:34.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:34.588+0000 7fcfdb400640 1 -- 172.21.15.5:0/1139998017 shutdown_connections 2024-09-10T09:43:34.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:34.588+0000 7fcfdb400640 1 --2- 172.21.15.5:0/1139998017 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcfd4109140 0x7fcfd4110cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:34.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:34.588+0000 7fcfdb400640 1 --2- 172.21.15.5:0/1139998017 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcfd41049c0 0x7fcfd4108a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:34.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:34.588+0000 7fcfdb400640 1 --2- 172.21.15.5:0/1139998017 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcfd4104010 0x7fcfd41043f0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:34.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:34.588+0000 7fcfdb400640 1 -- 172.21.15.5:0/1139998017 >> 172.21.15.5:0/1139998017 conn(0x7fcfd40ffa50 msgr2=0x7fcfd4101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:43:34.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:34.588+0000 7fcfdb400640 1 -- 172.21.15.5:0/1139998017 shutdown_connections 2024-09-10T09:43:34.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:34.588+0000 7fcfdb400640 1 -- 172.21.15.5:0/1139998017 wait complete. 2024-09-10T09:43:34.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:34.589+0000 7fcfdb400640 1 Processor -- start 2024-09-10T09:43:34.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:34.589+0000 7fcfdb400640 1 -- start start 2024-09-10T09:43:34.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:34.590+0000 7fcfdb400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcfd4104010 0x7fcfd4076700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:43:34.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:34.590+0000 7fcfdb400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcfd41049c0 0x7fcfd4076c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:43:34.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:34.590+0000 7fcfdb400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcfd4109140 0x7fcfd4077180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:43:34.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:34.590+0000 7fcfdb400640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fcfd41141c0 con 0x7fcfd4104010 2024-09-10T09:43:34.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:34.590+0000 7fcfdb400640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fcfd4114040 con 0x7fcfd41049c0 2024-09-10T09:43:34.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:34.590+0000 7fcfdb400640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fcfd4114340 con 0x7fcfd4109140 2024-09-10T09:43:34.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:34.590+0000 7fcfd8a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcfd4104010 0x7fcfd4076700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:43:34.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:34.590+0000 7fcfd9400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcfd4109140 0x7fcfd4077180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:43:34.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:34.590+0000 7fcfd8a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcfd4104010 0x7fcfd4076700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59566/0 (socket says 172.21.15.5:59566) 2024-09-10T09:43:34.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:34.590+0000 7fcfd8a00640 1 -- 172.21.15.5:0/2958236018 learned_addr learned my addr 172.21.15.5:0/2958236018 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:43:34.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:34.590+0000 7fcfd3e00640 1 --2- 172.21.15.5:0/2958236018 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcfd41049c0 0x7fcfd4076c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:43:34.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:34.590+0000 7fcfd9400640 1 -- 172.21.15.5:0/2958236018 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcfd41049c0 msgr2=0x7fcfd4076c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:43:34.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:34.590+0000 7fcfd9400640 1 --2- 172.21.15.5:0/2958236018 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcfd41049c0 0x7fcfd4076c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:34.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:34.591+0000 7fcfd9400640 1 -- 172.21.15.5:0/2958236018 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcfd4104010 msgr2=0x7fcfd4076700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:43:34.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:34.591+0000 7fcfd9400640 1 --2- 172.21.15.5:0/2958236018 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcfd4104010 0x7fcfd4076700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:34.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:34.591+0000 7fcfd9400640 1 -- 172.21.15.5:0/2958236018 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcfd41aad80 con 0x7fcfd4109140 2024-09-10T09:43:34.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:34.591+0000 7fcfd3e00640 1 --2- 172.21.15.5:0/2958236018 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcfd41049c0 0x7fcfd4076c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:43:34.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:34.591+0000 7fcfd9400640 1 --2- 172.21.15.5:0/2958236018 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcfd4109140 0x7fcfd4077180 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fcfcc00ea80 tx=0x7fcfcc00ef40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:43:34.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:34.591+0000 7fcfd1600640 1 -- 172.21.15.5:0/2958236018 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfcc00cea0 con 0x7fcfd4109140 2024-09-10T09:43:34.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:34.591+0000 7fcfdb400640 1 -- 172.21.15.5:0/2958236018 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcfd41ab070 con 0x7fcfd4109140 2024-09-10T09:43:34.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:34.591+0000 7fcfd1600640 1 -- 172.21.15.5:0/2958236018 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcfcc005020 con 0x7fcfd4109140 2024-09-10T09:43:34.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:34.591+0000 7fcfd1600640 1 -- 172.21.15.5:0/2958236018 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfcc010690 con 0x7fcfd4109140 2024-09-10T09:43:34.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:34.591+0000 7fcfdb400640 1 -- 172.21.15.5:0/2958236018 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcfd41ab600 con 0x7fcfd4109140 2024-09-10T09:43:34.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:34.594+0000 7fcfdde90740 1 Processor -- start 2024-09-10T09:43:34.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:34.594+0000 7fcfd1600640 1 -- 172.21.15.5:0/2958236018 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7fcfcc005540 con 0x7fcfd4109140 2024-09-10T09:43:34.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:34.595+0000 7fcfdde90740 1 -- start start 2024-09-10T09:43:34.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:34.595+0000 7fcfdde90740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f99553cac0 0x55f99553cea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:43:34.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:34.595+0000 7fcfdde90740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f99553d3e0 0x55f995547760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:43:34.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:34.595+0000 7fcfdde90740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f995547ca0 0x55f99554a090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:43:34.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:34.595+0000 7fcfdde90740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55f9954a3290 con 0x55f995547ca0 2024-09-10T09:43:34.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:34.595+0000 7fcfdde90740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55f99544ab40 con 0x55f99553cac0 2024-09-10T09:43:34.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:34.595+0000 7fcfdde90740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55f99549f250 con 0x55f99553d3e0 2024-09-10T09:43:34.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:34.595+0000 7fcfd8a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f99553d3e0 0x55f995547760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:43:34.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:34.595+0000 7fcfd8a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f99553d3e0 0x55f995547760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:38110/0 (socket says 172.21.15.5:38110) 2024-09-10T09:43:34.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:34.595+0000 7fcfd8a00640 1 -- 172.21.15.5:0/245844201 learned_addr learned my addr 172.21.15.5:0/245844201 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:43:34.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:34.596+0000 7fcfd8a00640 1 -- 172.21.15.5:0/245844201 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f99553cac0 msgr2=0x55f99553cea0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T09:43:34.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:34.596+0000 7fcfd8a00640 1 --2- 172.21.15.5:0/245844201 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f99553cac0 0x55f99553cea0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:34.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:34.596+0000 7fcfd8a00640 1 -- 172.21.15.5:0/245844201 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f995547ca0 msgr2=0x55f99554a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:43:34.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:34.596+0000 7fcfd3e00640 1 --2- 172.21.15.5:0/245844201 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f995547ca0 0x55f99554a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:43:34.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:34.596+0000 7fcfd8a00640 1 --2- 172.21.15.5:0/245844201 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f995547ca0 0x55f99554a090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:34.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:34.596+0000 7fcfd8a00640 1 -- 172.21.15.5:0/245844201 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f9952146a0 con 0x55f99553d3e0 2024-09-10T09:43:34.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:34.596+0000 7fcfd3e00640 1 --2- 172.21.15.5:0/245844201 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f995547ca0 0x55f99554a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T09:43:34.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:34.596+0000 7fcfd8a00640 1 --2- 172.21.15.5:0/245844201 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f99553d3e0 0x55f995547760 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fcfc8005dd0 tx=0x7fcfc8005c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=17c23239bdd68cca server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:43:34.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:34.597+0000 7fcfd1600640 1 --2- 172.21.15.5:0/2958236018 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fcfbc0a2b80 0x7fcfbc0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:43:34.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:34.597+0000 7fcfc5c00640 1 -- 172.21.15.5:0/245844201 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfc8040070 con 0x55f99553d3e0 2024-09-10T09:43:34.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:34.597+0000 7fcfc5c00640 1 -- 172.21.15.5:0/245844201 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcfc8004400 con 0x55f99553d3e0 2024-09-10T09:43:34.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:34.597+0000 7fcfc5c00640 1 -- 172.21.15.5:0/245844201 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfc8035040 con 0x55f99553d3e0 2024-09-10T09:43:34.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:34.597+0000 7fcfd1600640 1 -- 172.21.15.5:0/2958236018 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fcfcc0cfae0 con 0x7fcfd4109140 2024-09-10T09:43:34.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:34.597+0000 7fcfd3e00640 1 --2- 172.21.15.5:0/2958236018 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fcfbc0a2b80 0x7fcfbc0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:43:34.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:34.597+0000 7fcfdde90740 1 -- 172.21.15.5:0/245844201 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f99553d3e0 msgr2=0x55f995547760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:43:34.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:34.597+0000 7fcfdde90740 1 --2- 172.21.15.5:0/245844201 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f99553d3e0 0x55f995547760 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fcfc8005dd0 tx=0x7fcfc8005c00 comp rx=0 tx=0).stop 2024-09-10T09:43:34.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:34.598+0000 7fcfdde90740 1 -- 172.21.15.5:0/245844201 shutdown_connections 2024-09-10T09:43:34.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:34.598+0000 7fcfdde90740 1 --2- 172.21.15.5:0/245844201 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f995547ca0 0x55f99554a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:34.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:34.598+0000 7fcfdde90740 1 --2- 172.21.15.5:0/245844201 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f99553d3e0 0x55f995547760 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:34.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:34.598+0000 7fcfdde90740 1 --2- 172.21.15.5:0/245844201 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f99553cac0 0x55f99553cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:34.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:34.598+0000 7fcfdde90740 1 -- 172.21.15.5:0/245844201 >> 172.21.15.5:0/245844201 conn(0x55f995539050 msgr2=0x55f995539430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:43:34.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:34.598+0000 7fcfd3e00640 1 --2- 172.21.15.5:0/2958236018 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fcfbc0a2b80 0x7fcfbc0a5040 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x55f99553a3d0 tx=0x7fcfc0005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:43:34.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:34.598+0000 7fcfdde90740 1 -- 172.21.15.5:0/245844201 shutdown_connections 2024-09-10T09:43:34.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:34.598+0000 7fcfdde90740 1 -- 172.21.15.5:0/245844201 wait complete. 2024-09-10T09:43:34.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:34.598+0000 7fcfdde90740 1 -- start start 2024-09-10T09:43:34.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:34.598+0000 7fcfdde90740 1 Processor -- start 2024-09-10T09:43:34.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:34.599+0000 7fcfdde90740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f99553cac0 0x55f995547590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:43:34.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:34.599+0000 7fcfdde90740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f99553d3e0 0x55f995544840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:43:34.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:34.599+0000 7fcfdde90740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f995547ca0 0x55f995544d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:43:34.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:34.599+0000 7fcfdde90740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55f99554a910 con 0x55f995547ca0 2024-09-10T09:43:34.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:34.599+0000 7fcfdde90740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55f99545dfa0 con 0x55f99553d3e0 2024-09-10T09:43:34.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:34.599+0000 7fcfdde90740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55f99554aa90 con 0x55f99553cac0 2024-09-10T09:43:34.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:34.599+0000 7fcfdde90740 10 client.0 _ll_register_callbacks cb 0x55f9951b1ba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:43:34.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:34.599+0000 7fcfd9400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f995547ca0 0x55f995544d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:43:34.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:34.599+0000 7fcfd3e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f99553d3e0 0x55f995544840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:43:34.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:34.599+0000 7fcfd9400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f995547ca0 0x55f995544d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59580/0 (socket says 172.21.15.5:59580) 2024-09-10T09:43:34.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:34.599+0000 7fcfd9400640 1 -- 172.21.15.5:0/1945486213 learned_addr learned my addr 172.21.15.5:0/1945486213 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:43:34.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:34.599+0000 7fcfd8a00640 1 --2- 172.21.15.5:0/1945486213 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f99553cac0 0x55f995547590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:43:34.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:34.600+0000 7fcfd9400640 1 -- 172.21.15.5:0/1945486213 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f99553cac0 msgr2=0x55f995547590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:43:34.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:34.600+0000 7fcfd9400640 1 --2- 172.21.15.5:0/1945486213 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f99553cac0 0x55f995547590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:34.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:34.600+0000 7fcfd9400640 1 -- 172.21.15.5:0/1945486213 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f99553d3e0 msgr2=0x55f995544840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:43:34.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:34.600+0000 7fcfd9400640 1 --2- 172.21.15.5:0/1945486213 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f99553d3e0 0x55f995544840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:34.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:34.600+0000 7fcfd9400640 1 -- 172.21.15.5:0/1945486213 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f995545560 con 0x55f995547ca0 2024-09-10T09:43:34.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:34.600+0000 7fcfd3e00640 1 --2- 172.21.15.5:0/1945486213 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f99553d3e0 0x55f995544840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:43:34.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:34.600+0000 7fcfd8a00640 1 --2- 172.21.15.5:0/1945486213 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f99553cac0 0x55f995547590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:43:34.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:34.600+0000 7fcfd9400640 1 --2- 172.21.15.5:0/1945486213 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f995547ca0 0x55f995544d80 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fcfcc0988c0 tx=0x7fcfcc09c730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=532115b5f5adc47a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:43:34.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:34.600+0000 7fcfba000640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-10T09:43:34.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:34.600+0000 7fcfba000640 1 -- 172.21.15.5:0/1945486213 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfcc0a4b60 con 0x55f995547ca0 2024-09-10T09:43:34.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:34.601+0000 7fcfdde90740 1 -- 172.21.15.5:0/1945486213 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcfc8004400 con 0x55f995547ca0 2024-09-10T09:43:34.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:34.601+0000 7fcfba000640 1 -- 172.21.15.5:0/1945486213 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fcfcc0a46d0 con 0x55f995547ca0 2024-09-10T09:43:34.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:34.601+0000 7fcfba000640 1 -- 172.21.15.5:0/1945486213 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fcfcc0a48d0 con 0x55f995547ca0 2024-09-10T09:43:34.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:34.601+0000 7fcfdde90740 10 client.16434 fetch_fsmap learned FSMap version 34 2024-09-10T09:43:34.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:34.601+0000 7fcfdde90740 1 -- 172.21.15.5:0/1945486213 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55f995556850 con 0x55f995547ca0 2024-09-10T09:43:34.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:34.601+0000 7fcfba000640 1 -- 172.21.15.5:0/1945486213 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcfc8004400 con 0x55f995547ca0 2024-09-10T09:43:34.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:34.602+0000 7fcfba000640 1 -- 172.21.15.5:0/1945486213 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7fcfcc09adb0 con 0x55f995547ca0 2024-09-10T09:43:34.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:34.602+0000 7fcfdde90740 10 client.16434 fetch_fsmap finished waiting for FSMap version 34 2024-09-10T09:43:34.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:34.602+0000 7fcfdde90740 10 client.16434 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T09:43:34.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:34.602+0000 7fcfdde90740 20 client.16434 populate_metadata read hostname 'smithi005' 2024-09-10T09:43:34.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:34.602+0000 7fcfdde90740 1 --2- 172.21.15.5:0/1945486213 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55f995557cb0 0x55f99555a150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:43:34.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:34.602+0000 7fcfdde90740 4 client.16434 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:43:34.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:34.602+0000 7fcfdde90740 1 -- 172.21.15.5:0/1945486213 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f9954b0800 con 0x55f995557cb0 2024-09-10T09:43:34.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:34.602+0000 7fcfd8a00640 1 --2- 172.21.15.5:0/1945486213 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55f995557cb0 0x55f99555a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:43:34.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:34.603+0000 7fcfd8a00640 1 --2- 172.21.15.5:0/1945486213 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55f995557cb0 0x55f99555a150 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7211efb026dae85 server_cookie=ddbeb8503175ecb7 in_seq=0 out_seq=0 2024-09-10T09:43:34.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:34.603+0000 7fcfba000640 10 client.16434 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T09:43:34.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:34.603+0000 7fcfba000640 1 -- 172.21.15.5:0/1945486213 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f9954b0800 con 0x55f995557cb0 2024-09-10T09:43:34.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:34.603+0000 7fcfba000640 10 client.16434 handle_command_reply: tid=0 2024-09-10T09:43:34.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:34.604+0000 7fcfdde90740 1 client.16434 shutdown 2024-09-10T09:43:34.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:34.604+0000 7fcfdde90740 1 -- 172.21.15.5:0/1945486213 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f995547ca0 msgr2=0x55f995544d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:43:34.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:34.604+0000 7fcfdde90740 1 --2- 172.21.15.5:0/1945486213 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f995547ca0 0x55f995544d80 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fcfcc0988c0 tx=0x7fcfcc09c730 comp rx=0 tx=0).stop 2024-09-10T09:43:34.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:34.604+0000 7fcfdde90740 1 -- 172.21.15.5:0/1945486213 shutdown_connections 2024-09-10T09:43:34.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:34.604+0000 7fcfdde90740 1 --2- 172.21.15.5:0/1945486213 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55f995557cb0 0x55f99555a150 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:34.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:34.604+0000 7fcfdde90740 1 --2- 172.21.15.5:0/1945486213 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f995547ca0 0x55f995544d80 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:34.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:34.604+0000 7fcfdde90740 1 --2- 172.21.15.5:0/1945486213 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f99553d3e0 0x55f995544840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:34.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:34.604+0000 7fcfdde90740 1 --2- 172.21.15.5:0/1945486213 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f99553cac0 0x55f995547590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:34.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:34.604+0000 7fcfdde90740 1 -- 172.21.15.5:0/1945486213 >> 172.21.15.5:0/1945486213 conn(0x55f995539050 msgr2=0x55f9955381c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:43:34.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:34.604+0000 7fcfba000640 0 client.16434 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T09:43:34.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:34.604+0000 7fcfdde90740 1 -- 172.21.15.5:0/1945486213 shutdown_connections 2024-09-10T09:43:34.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:34.604+0000 7fcfdde90740 1 -- 172.21.15.5:0/1945486213 wait complete. 2024-09-10T09:43:34.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:34.604+0000 7fcfdde90740 20 client.16434 trim_cache size 0 max 16384 2024-09-10T09:43:34.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:34.625+0000 7fcfdde90740 1 Processor -- start 2024-09-10T09:43:34.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:34.625+0000 7fcfdde90740 1 -- start start 2024-09-10T09:43:34.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:34.625+0000 7fcfdde90740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f99553cac0 0x55f9955467b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:43:34.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:34.625+0000 7fcfdde90740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f99553d3e0 0x55f99555fa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:43:34.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:34.625+0000 7fcfdde90740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f995547ca0 0x55f99555ff60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:43:34.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:34.625+0000 7fcfdde90740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fcfd41b5450 con 0x55f99553d3e0 2024-09-10T09:43:34.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:34.625+0000 7fcfdde90740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fcfd41b6f90 con 0x55f99553cac0 2024-09-10T09:43:34.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:34.625+0000 7fcfdde90740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fcfd41b5b20 con 0x55f995547ca0 2024-09-10T09:43:34.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:34.626+0000 7fcfd8a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f99553cac0 0x55f9955467b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:43:34.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:34.626+0000 7fcfd8a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f99553cac0 0x55f9955467b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:37306/0 (socket says 172.21.15.5:37306) 2024-09-10T09:43:34.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:34.626+0000 7fcfd9400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f995547ca0 0x55f99555ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:43:34.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:34.626+0000 7fcfd8a00640 1 -- 172.21.15.5:0/523571157 learned_addr learned my addr 172.21.15.5:0/523571157 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:43:34.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:34.626+0000 7fcfd3e00640 1 --2- 172.21.15.5:0/523571157 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f99553d3e0 0x55f99555fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:43:34.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:34.626+0000 7fcfd9400640 1 -- 172.21.15.5:0/523571157 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f99553cac0 msgr2=0x55f9955467b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:43:34.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:34.626+0000 7fcfd9400640 1 --2- 172.21.15.5:0/523571157 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f99553cac0 0x55f9955467b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:34.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:34.626+0000 7fcfd9400640 1 -- 172.21.15.5:0/523571157 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f99553d3e0 msgr2=0x55f99555fa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:43:34.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:34.626+0000 7fcfd9400640 1 --2- 172.21.15.5:0/523571157 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f99553d3e0 0x55f99555fa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:34.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:34.626+0000 7fcfd9400640 1 -- 172.21.15.5:0/523571157 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f9955437f0 con 0x55f995547ca0 2024-09-10T09:43:34.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:34.626+0000 7fcfd8a00640 1 --2- 172.21.15.5:0/523571157 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f99553cac0 0x55f9955467b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:43:34.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:34.626+0000 7fcfd9400640 1 --2- 172.21.15.5:0/523571157 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f995547ca0 0x55f99555ff60 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fcfcc09bbf0 tx=0x7fcfcc09a400 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bd494fcfbb0b517a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:43:34.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:34.627+0000 7fcfc5200640 1 -- 172.21.15.5:0/523571157 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfcc0a4b60 con 0x55f995547ca0 2024-09-10T09:43:34.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:34.627+0000 7fcfc5200640 1 -- 172.21.15.5:0/523571157 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcfcc07f070 con 0x55f995547ca0 2024-09-10T09:43:34.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:34.627+0000 7fcfc5200640 1 -- 172.21.15.5:0/523571157 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfcc0bab30 con 0x55f995547ca0 2024-09-10T09:43:34.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:34.628+0000 7fcfdde90740 1 -- 172.21.15.5:0/523571157 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f995547ca0 msgr2=0x55f99555ff60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:43:34.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:34.628+0000 7fcfdde90740 1 --2- 172.21.15.5:0/523571157 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f995547ca0 0x55f99555ff60 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fcfcc09bbf0 tx=0x7fcfcc09a400 comp rx=0 tx=0).stop 2024-09-10T09:43:34.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:34.628+0000 7fcfdde90740 1 -- 172.21.15.5:0/523571157 shutdown_connections 2024-09-10T09:43:34.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:34.628+0000 7fcfdde90740 1 --2- 172.21.15.5:0/523571157 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f995547ca0 0x55f99555ff60 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:34.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:34.628+0000 7fcfdde90740 1 --2- 172.21.15.5:0/523571157 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f99553d3e0 0x55f99555fa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:34.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:34.628+0000 7fcfdde90740 1 --2- 172.21.15.5:0/523571157 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f99553cac0 0x55f9955467b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:34.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:34.628+0000 7fcfdde90740 1 -- 172.21.15.5:0/523571157 >> 172.21.15.5:0/523571157 conn(0x55f995539050 msgr2=0x55f9955460a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:43:34.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:34.628+0000 7fcfdde90740 1 -- 172.21.15.5:0/523571157 shutdown_connections 2024-09-10T09:43:34.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:34.628+0000 7fcfdde90740 1 -- 172.21.15.5:0/523571157 wait complete. 2024-09-10T09:43:34.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:34.628+0000 7fcfdde90740 1 -- start start 2024-09-10T09:43:34.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:34.629+0000 7fcfdde90740 1 Processor -- start 2024-09-10T09:43:34.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:34.629+0000 7fcfdde90740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f99553cac0 0x55f99554de10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:43:34.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:34.629+0000 7fcfdde90740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f99553d3e0 0x55f99554e350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:43:34.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:34.629+0000 7fcfdde90740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f995547ca0 0x55f9955c1a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:43:34.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:34.629+0000 7fcfdde90740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fcfd41b68c0 con 0x55f99553cac0 2024-09-10T09:43:34.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:34.629+0000 7fcfdde90740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fcfd41b61f0 con 0x55f995547ca0 2024-09-10T09:43:34.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:34.629+0000 7fcfdde90740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fcfd41b3430 con 0x55f99553d3e0 2024-09-10T09:43:34.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:34.629+0000 7fcfdde90740 10 client.0 _ll_register_callbacks cb 0x7fcfcc005870 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:43:34.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:34.629+0000 7fcfd8a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f99553cac0 0x55f99554de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:43:34.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:34.629+0000 7fcfd3e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f99553d3e0 0x55f99554e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:43:34.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:34.629+0000 7fcfd8a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f99553cac0 0x55f99554de10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59584/0 (socket says 172.21.15.5:59584) 2024-09-10T09:43:34.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:34.629+0000 7fcfd8a00640 1 -- 172.21.15.5:0/45969054 learned_addr learned my addr 172.21.15.5:0/45969054 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:43:34.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:34.629+0000 7fcfd3e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f99553d3e0 0x55f99554e350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:38140/0 (socket says 172.21.15.5:38140) 2024-09-10T09:43:34.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:34.630+0000 7fcfd9400640 1 --2- 172.21.15.5:0/45969054 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f995547ca0 0x55f9955c1a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:43:34.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:34.630+0000 7fcfd3e00640 1 -- 172.21.15.5:0/45969054 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f995547ca0 msgr2=0x55f9955c1a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:43:34.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:34.630+0000 7fcfd3e00640 1 --2- 172.21.15.5:0/45969054 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f995547ca0 0x55f9955c1a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:34.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:34.630+0000 7fcfd3e00640 1 -- 172.21.15.5:0/45969054 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f99553cac0 msgr2=0x55f99554de10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:43:34.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:34.630+0000 7fcfd3e00640 1 --2- 172.21.15.5:0/45969054 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f99553cac0 0x55f99554de10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:34.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:34.630+0000 7fcfd3e00640 1 -- 172.21.15.5:0/45969054 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f99554ec00 con 0x55f99553d3e0 2024-09-10T09:43:34.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:34.630+0000 7fcfd9400640 1 --2- 172.21.15.5:0/45969054 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f995547ca0 0x55f9955c1a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:43:34.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:34.630+0000 7fcfd8a00640 1 --2- 172.21.15.5:0/45969054 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f99553cac0 0x55f99554de10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:43:34.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:34.630+0000 7fcfd3e00640 1 --2- 172.21.15.5:0/45969054 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f99553d3e0 0x55f99554e350 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x55f995545e80 tx=0x7fcfc00046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a18aed7bc9b18b17 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:43:34.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:34.630+0000 7fcfba000640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-09-10T09:43:34.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:34.631+0000 7fcfba000640 1 -- 172.21.15.5:0/45969054 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfc001fc40 con 0x55f99553d3e0 2024-09-10T09:43:34.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:34.631+0000 7fcfdde90740 1 -- 172.21.15.5:0/45969054 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcfcc07f070 con 0x55f99553d3e0 2024-09-10T09:43:34.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:34.631+0000 7fcfba000640 1 -- 172.21.15.5:0/45969054 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fcfc001fde0 con 0x55f99553d3e0 2024-09-10T09:43:34.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:34.631+0000 7fcfba000640 1 -- 172.21.15.5:0/45969054 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fcfc0036050 con 0x55f99553d3e0 2024-09-10T09:43:34.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:34.631+0000 7fcfdde90740 10 client.26482 fetch_fsmap learned FSMap version 34 2024-09-10T09:43:34.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:34.631+0000 7fcfdde90740 1 -- 172.21.15.5:0/45969054 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55f9955c3110 con 0x55f99553d3e0 2024-09-10T09:43:34.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:34.631+0000 7fcfba000640 1 -- 172.21.15.5:0/45969054 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcfcc07f070 con 0x55f99553d3e0 2024-09-10T09:43:34.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:34.631+0000 7fcfba000640 1 -- 172.21.15.5:0/45969054 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7fcfc00293f0 con 0x55f99553d3e0 2024-09-10T09:43:34.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:34.632+0000 7fcfdde90740 10 client.26482 fetch_fsmap finished waiting for FSMap version 34 2024-09-10T09:43:34.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:34.632+0000 7fcfdde90740 10 client.26482 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T09:43:34.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:34.632+0000 7fcfdde90740 20 client.26482 populate_metadata read hostname 'smithi005' 2024-09-10T09:43:34.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:34.632+0000 7fcfdde90740 1 --2- 172.21.15.5:0/45969054 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55f9955c3890 0x55f9955c5cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:43:34.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:34.632+0000 7fcfdde90740 4 client.26482 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-10T09:43:34.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:34.632+0000 7fcfdde90740 1 -- 172.21.15.5:0/45969054 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55f9954aad10 con 0x55f9955c3890 2024-09-10T09:43:34.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:34.632+0000 7fcfd8a00640 1 --2- 172.21.15.5:0/45969054 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55f9955c3890 0x55f9955c5cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:43:34.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:34.632+0000 7fcfd8a00640 1 --2- 172.21.15.5:0/45969054 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55f9955c3890 0x55f9955c5cb0 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f5e8a95a23edd064 server_cookie=172f6ed5f96de54b in_seq=0 out_seq=0 2024-09-10T09:43:34.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:34.632+0000 7fcfba000640 10 client.26482 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T09:43:34.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:34.633+0000 7fcfba000640 1 -- 172.21.15.5:0/45969054 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55f9954aad10 con 0x55f9955c3890 2024-09-10T09:43:34.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:34.633+0000 7fcfba000640 10 client.26482 handle_command_reply: tid=0 2024-09-10T09:43:34.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:34.633+0000 7fcfdde90740 1 client.26482 shutdown 2024-09-10T09:43:34.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:34.633+0000 7fcfdde90740 1 -- 172.21.15.5:0/45969054 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f99553d3e0 msgr2=0x55f99554e350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:43:34.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:34.633+0000 7fcfdde90740 1 --2- 172.21.15.5:0/45969054 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f99553d3e0 0x55f99554e350 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x55f995545e80 tx=0x7fcfc00046c0 comp rx=0 tx=0).stop 2024-09-10T09:43:34.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:34.633+0000 7fcfdde90740 1 -- 172.21.15.5:0/45969054 shutdown_connections 2024-09-10T09:43:34.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:34.633+0000 7fcfdde90740 1 --2- 172.21.15.5:0/45969054 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55f9955c3890 0x55f9955c5cb0 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:34.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:34.633+0000 7fcfdde90740 1 --2- 172.21.15.5:0/45969054 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f995547ca0 0x55f9955c1a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:34.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:34.633+0000 7fcfdde90740 1 --2- 172.21.15.5:0/45969054 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f99553d3e0 0x55f99554e350 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:34.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:34.633+0000 7fcfdde90740 1 --2- 172.21.15.5:0/45969054 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f99553cac0 0x55f99554de10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:34.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:34.633+0000 7fcfdde90740 1 -- 172.21.15.5:0/45969054 >> 172.21.15.5:0/45969054 conn(0x55f995539050 msgr2=0x55f9955381c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:43:34.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:34.634+0000 7fcfba000640 0 client.26482 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T09:43:34.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:34.634+0000 7fcfdde90740 1 -- 172.21.15.5:0/45969054 shutdown_connections 2024-09-10T09:43:34.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:34.634+0000 7fcfdde90740 1 -- 172.21.15.5:0/45969054 wait complete. 2024-09-10T09:43:34.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:34.634+0000 7fcfdde90740 20 client.26482 trim_cache size 0 max 16384 2024-09-10T09:43:34.638 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-10T09:43:34.638 INFO:teuthology.orchestra.run.smithi005.stdout: "return_code": 0, 2024-09-10T09:43:34.638 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_tag": "5c923e45-ce30-445f-9933-9923624272dc", 2024-09-10T09:43:34.638 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": "asynchronous" 2024-09-10T09:43:34.638 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-10T09:43:34.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:34.635+0000 7fcfdb400640 1 -- 172.21.15.5:0/2958236018 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fcfbc0a2b80 msgr2=0x7fcfbc0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:43:34.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:34.635+0000 7fcfdb400640 1 --2- 172.21.15.5:0/2958236018 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fcfbc0a2b80 0x7fcfbc0a5040 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x55f99553a3d0 tx=0x7fcfc0005f90 comp rx=0 tx=0).stop 2024-09-10T09:43:34.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:34.635+0000 7fcfdb400640 1 -- 172.21.15.5:0/2958236018 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcfd4109140 msgr2=0x7fcfd4077180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:43:34.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:34.635+0000 7fcfdb400640 1 --2- 172.21.15.5:0/2958236018 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcfd4109140 0x7fcfd4077180 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fcfcc00ea80 tx=0x7fcfcc00ef40 comp rx=0 tx=0).stop 2024-09-10T09:43:34.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:34.635+0000 7fcfdb400640 1 -- 172.21.15.5:0/2958236018 shutdown_connections 2024-09-10T09:43:34.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:34.635+0000 7fcfdb400640 1 --2- 172.21.15.5:0/2958236018 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fcfbc0a2b80 0x7fcfbc0a5040 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:34.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:34.635+0000 7fcfdb400640 1 --2- 172.21.15.5:0/2958236018 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcfd4109140 0x7fcfd4077180 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:34.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:34.635+0000 7fcfdb400640 1 --2- 172.21.15.5:0/2958236018 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcfd41049c0 0x7fcfd4076c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:34.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:34.635+0000 7fcfdb400640 1 --2- 172.21.15.5:0/2958236018 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcfd4104010 0x7fcfd4076700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:34.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:34.635+0000 7fcfdb400640 1 -- 172.21.15.5:0/2958236018 >> 172.21.15.5:0/2958236018 conn(0x7fcfd40ffa50 msgr2=0x7fcfd4100de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:43:34.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:34.635+0000 7fcfdb400640 1 -- 172.21.15.5:0/2958236018 shutdown_connections 2024-09-10T09:43:34.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:34.635+0000 7fcfdb400640 1 -- 172.21.15.5:0/2958236018 wait complete. 2024-09-10T09:43:35.040 INFO:tasks.workunit.client.0.smithi005.stdout: 1 261544 4.88 MB/sec execute 76 sec latency 47.942 ms 2024-09-10T09:43:35.640 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T09:43:35.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:43:35 smithi102 ceph-mon[35442]: pgmap v550: 129 pgs: 129 active+clean; 111 MiB data, 975 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.1 MiB/s wr, 166 op/s 2024-09-10T09:43:35.885 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:43:35 smithi102 ceph-mon[35442]: scrub queued for path: / 2024-09-10T09:43:35.885 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:43:35 smithi102 ceph-mon[35442]: scrub summary: idle+waiting paths [/] 2024-09-10T09:43:35.885 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:43:35 smithi102 ceph-mon[35442]: scrub summary: active paths [/] 2024-09-10T09:43:35.885 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:43:35 smithi102 ceph-mon[35442]: scrub summary: active paths [/] 2024-09-10T09:43:35.885 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:43:35 smithi102 ceph-mon[35442]: scrub summary: idle+waiting paths [/] 2024-09-10T09:43:35.885 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:43:35 smithi102 ceph-mon[35442]: scrub summary: idle+waiting paths [/] 2024-09-10T09:43:36.040 INFO:tasks.workunit.client.0.smithi005.stdout: 1 263605 4.99 MB/sec execute 77 sec latency 25.849 ms 2024-09-10T09:43:36.054 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:43:36.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:43:35 smithi005 ceph-mon[30350]: pgmap v550: 129 pgs: 129 active+clean; 111 MiB data, 975 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.1 MiB/s wr, 166 op/s 2024-09-10T09:43:36.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:43:35 smithi005 ceph-mon[30350]: scrub queued for path: / 2024-09-10T09:43:36.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:43:35 smithi005 ceph-mon[30350]: scrub summary: idle+waiting paths [/] 2024-09-10T09:43:36.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:43:35 smithi005 ceph-mon[30350]: scrub summary: active paths [/] 2024-09-10T09:43:36.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:43:35 smithi005 ceph-mon[30350]: scrub summary: active paths [/] 2024-09-10T09:43:36.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:43:35 smithi005 ceph-mon[30350]: scrub summary: idle+waiting paths [/] 2024-09-10T09:43:36.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:43:35 smithi005 ceph-mon[30350]: scrub summary: idle+waiting paths [/] 2024-09-10T09:43:36.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:43:35 smithi159 ceph-mon[35513]: pgmap v550: 129 pgs: 129 active+clean; 111 MiB data, 975 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.1 MiB/s wr, 166 op/s 2024-09-10T09:43:36.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:43:35 smithi159 ceph-mon[35513]: scrub queued for path: / 2024-09-10T09:43:36.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:43:35 smithi159 ceph-mon[35513]: scrub summary: idle+waiting paths [/] 2024-09-10T09:43:36.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:43:35 smithi159 ceph-mon[35513]: scrub summary: active paths [/] 2024-09-10T09:43:36.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:43:35 smithi159 ceph-mon[35513]: scrub summary: active paths [/] 2024-09-10T09:43:36.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:43:35 smithi159 ceph-mon[35513]: scrub summary: idle+waiting paths [/] 2024-09-10T09:43:36.101 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:43:35 smithi159 ceph-mon[35513]: scrub summary: idle+waiting paths [/] 2024-09-10T09:43:36.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:36.660+0000 7f90a5c00640 1 -- 172.21.15.5:0/2824996147 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f90a0100720 msgr2=0x7f90a0100b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:43:36.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:36.660+0000 7f90a5c00640 1 --2- 172.21.15.5:0/2824996147 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f90a0100720 0x7f90a0100b00 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f9090009900 tx=0x7f909002a3f0 comp rx=0 tx=0).stop 2024-09-10T09:43:36.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:36.661+0000 7f90a5c00640 1 -- 172.21.15.5:0/2824996147 shutdown_connections 2024-09-10T09:43:36.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:36.661+0000 7f90a5c00640 1 --2- 172.21.15.5:0/2824996147 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f90a010e2d0 0x7f90a01106c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:36.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:36.661+0000 7f90a5c00640 1 --2- 172.21.15.5:0/2824996147 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f90a01010d0 0x7f90a010dd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:36.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:36.661+0000 7f90a5c00640 1 --2- 172.21.15.5:0/2824996147 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f90a0100720 0x7f90a0100b00 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:36.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:36.661+0000 7f90a5c00640 1 -- 172.21.15.5:0/2824996147 >> 172.21.15.5:0/2824996147 conn(0x7f90a00fc610 msgr2=0x7f90a00fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:43:36.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:36.661+0000 7f90a5c00640 1 -- 172.21.15.5:0/2824996147 shutdown_connections 2024-09-10T09:43:36.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:36.661+0000 7f90a5c00640 1 -- 172.21.15.5:0/2824996147 wait complete. 2024-09-10T09:43:36.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:36.662+0000 7f90a5c00640 1 Processor -- start 2024-09-10T09:43:36.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:36.662+0000 7f90a5c00640 1 -- start start 2024-09-10T09:43:36.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:36.663+0000 7f90a5c00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f90a0100720 0x7f90a019a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:43:36.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:36.663+0000 7f90a5c00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f90a01010d0 0x7f90a019aac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:43:36.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:36.663+0000 7f90a5c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f90a010e2d0 0x7f90a019ee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:43:36.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:36.663+0000 7f90a5c00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f90a0114040 con 0x7f90a01010d0 2024-09-10T09:43:36.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:36.663+0000 7f90a5c00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f90a0113ec0 con 0x7f90a010e2d0 2024-09-10T09:43:36.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:36.663+0000 7f90a5c00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f90a01141c0 con 0x7f90a0100720 2024-09-10T09:43:36.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:36.664+0000 7f909fe00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f90a0100720 0x7f90a019a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:43:36.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:36.664+0000 7f9097e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f90a01010d0 0x7f90a019aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:43:36.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:36.664+0000 7f9097e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f90a01010d0 0x7f90a019aac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59606/0 (socket says 172.21.15.5:59606) 2024-09-10T09:43:36.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:36.664+0000 7f90a5200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f90a010e2d0 0x7f90a019ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:43:36.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:36.664+0000 7f9097e00640 1 -- 172.21.15.5:0/3932560449 learned_addr learned my addr 172.21.15.5:0/3932560449 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:43:36.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:36.664+0000 7f909fe00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f90a0100720 0x7f90a019a580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:38154/0 (socket says 172.21.15.5:38154) 2024-09-10T09:43:36.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:36.664+0000 7f90a5200640 1 -- 172.21.15.5:0/3932560449 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f90a0100720 msgr2=0x7f90a019a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:43:36.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:36.664+0000 7f90a5200640 1 --2- 172.21.15.5:0/3932560449 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f90a0100720 0x7f90a019a580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:36.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:36.664+0000 7f90a5200640 1 -- 172.21.15.5:0/3932560449 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f90a01010d0 msgr2=0x7f90a019aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:43:36.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:36.664+0000 7f90a5200640 1 --2- 172.21.15.5:0/3932560449 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f90a01010d0 0x7f90a019aac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:36.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:36.664+0000 7f90a5200640 1 -- 172.21.15.5:0/3932560449 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90a019f530 con 0x7f90a010e2d0 2024-09-10T09:43:36.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:36.664+0000 7f909fe00640 1 --2- 172.21.15.5:0/3932560449 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f90a0100720 0x7f90a019a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:43:36.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:36.664+0000 7f90a5200640 1 --2- 172.21.15.5:0/3932560449 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f90a010e2d0 0x7f90a019ee50 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f90a0102900 tx=0x7f909800ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:43:36.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:36.665+0000 7f909d600640 1 -- 172.21.15.5:0/3932560449 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f909800cdf0 con 0x7f90a010e2d0 2024-09-10T09:43:36.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:36.665+0000 7f90a5c00640 1 -- 172.21.15.5:0/3932560449 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90a019f820 con 0x7f90a010e2d0 2024-09-10T09:43:36.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:36.665+0000 7f909d600640 1 -- 172.21.15.5:0/3932560449 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9098005020 con 0x7f90a010e2d0 2024-09-10T09:43:36.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:36.665+0000 7f909d600640 1 -- 172.21.15.5:0/3932560449 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9098010690 con 0x7f90a010e2d0 2024-09-10T09:43:36.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:36.665+0000 7f90a5c00640 1 -- 172.21.15.5:0/3932560449 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90a01a7100 con 0x7f90a010e2d0 2024-09-10T09:43:36.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:36.669+0000 7f90a9cb3740 1 Processor -- start 2024-09-10T09:43:36.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:36.669+0000 7f909d600640 1 -- 172.21.15.5:0/3932560449 <== mon.2 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f90980054d0 con 0x7f90a010e2d0 2024-09-10T09:43:36.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:36.670+0000 7f90a9cb3740 1 -- start start 2024-09-10T09:43:36.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:36.670+0000 7f90a9cb3740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d9eced6d50 0x55d9eced7130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:43:36.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:36.670+0000 7f90a9cb3740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d9eced7670 0x55d9ecee19f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:43:36.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:36.670+0000 7f90a9cb3740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d9ecee1f30 0x55d9ecee4320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:43:36.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:36.670+0000 7f90a9cb3740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55d9ece3d3e0 con 0x55d9eced7670 2024-09-10T09:43:36.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:36.670+0000 7f90a9cb3740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55d9ecde4d80 con 0x55d9eced6d50 2024-09-10T09:43:36.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:36.670+0000 7f90a9cb3740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55d9ece393a0 con 0x55d9ecee1f30 2024-09-10T09:43:36.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:36.671+0000 7f9097e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d9eced6d50 0x55d9eced7130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:43:36.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:36.671+0000 7f9097e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d9eced6d50 0x55d9eced7130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:37344/0 (socket says 172.21.15.5:37344) 2024-09-10T09:43:36.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:36.671+0000 7f9097e00640 1 -- 172.21.15.5:0/1220043007 learned_addr learned my addr 172.21.15.5:0/1220043007 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:43:36.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:36.671+0000 7f9097e00640 1 --2- 172.21.15.5:0/1220043007 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d9ecee1f30 0x55d9ecee4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:43:36.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:36.671+0000 7f909fe00640 1 --2- 172.21.15.5:0/1220043007 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d9eced7670 0x55d9ecee19f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:43:36.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:36.671+0000 7f9097e00640 1 -- 172.21.15.5:0/1220043007 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d9ecee1f30 msgr2=0x55d9ecee4320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:43:36.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:36.671+0000 7f9097e00640 1 --2- 172.21.15.5:0/1220043007 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d9ecee1f30 0x55d9ecee4320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:36.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:36.671+0000 7f9097e00640 1 -- 172.21.15.5:0/1220043007 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d9eced7670 msgr2=0x55d9ecee19f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:43:36.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:36.671+0000 7f9097e00640 1 --2- 172.21.15.5:0/1220043007 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d9eced7670 0x55d9ecee19f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:36.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:36.671+0000 7f9097e00640 1 -- 172.21.15.5:0/1220043007 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d9ece03ae0 con 0x55d9eced6d50 2024-09-10T09:43:36.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:36.671+0000 7f9097e00640 1 --2- 172.21.15.5:0/1220043007 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d9eced6d50 0x55d9eced7130 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f9088004650 tx=0x7f9088004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ab1cc9a83185ff45 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:43:36.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:36.672+0000 7f909d600640 1 --2- 172.21.15.5:0/3932560449 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f90840a2b80 0x7f90840a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:43:36.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:36.672+0000 7f9095600640 1 -- 172.21.15.5:0/1220043007 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9088005020 con 0x55d9eced6d50 2024-09-10T09:43:36.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:36.672+0000 7f9095600640 1 -- 172.21.15.5:0/1220043007 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f90880047a0 con 0x55d9eced6d50 2024-09-10T09:43:36.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:36.672+0000 7f9095600640 1 -- 172.21.15.5:0/1220043007 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f908800fa20 con 0x55d9eced6d50 2024-09-10T09:43:36.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:36.672+0000 7f909fe00640 1 --2- 172.21.15.5:0/3932560449 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f90840a2b80 0x7f90840a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:43:36.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:36.672+0000 7f909d600640 1 -- 172.21.15.5:0/3932560449 <== mon.2 v2:172.21.15.102:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f90980cfa40 con 0x7f90a010e2d0 2024-09-10T09:43:36.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:36.673+0000 7f909fe00640 1 --2- 172.21.15.5:0/3932560449 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f90840a2b80 0x7f90840a5040 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f9090000c00 tx=0x7f9090031510 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:43:36.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:36.673+0000 7f90a9cb3740 1 -- 172.21.15.5:0/1220043007 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d9eced6d50 msgr2=0x55d9eced7130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:43:36.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:36.673+0000 7f90a9cb3740 1 --2- 172.21.15.5:0/1220043007 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d9eced6d50 0x55d9eced7130 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f9088004650 tx=0x7f9088004730 comp rx=0 tx=0).stop 2024-09-10T09:43:36.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:36.673+0000 7f90a9cb3740 1 -- 172.21.15.5:0/1220043007 shutdown_connections 2024-09-10T09:43:36.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:36.673+0000 7f90a9cb3740 1 --2- 172.21.15.5:0/1220043007 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d9ecee1f30 0x55d9ecee4320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:36.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:36.673+0000 7f90a9cb3740 1 --2- 172.21.15.5:0/1220043007 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d9eced7670 0x55d9ecee19f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:36.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:36.673+0000 7f90a9cb3740 1 --2- 172.21.15.5:0/1220043007 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d9eced6d50 0x55d9eced7130 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:36.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:36.673+0000 7f90a9cb3740 1 -- 172.21.15.5:0/1220043007 >> 172.21.15.5:0/1220043007 conn(0x55d9eced32e0 msgr2=0x55d9eced36c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:43:36.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:36.673+0000 7f90a9cb3740 1 -- 172.21.15.5:0/1220043007 shutdown_connections 2024-09-10T09:43:36.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:36.673+0000 7f90a9cb3740 1 -- 172.21.15.5:0/1220043007 wait complete. 2024-09-10T09:43:36.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:36.674+0000 7f90a9cb3740 1 -- start start 2024-09-10T09:43:36.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:36.674+0000 7f90a9cb3740 1 Processor -- start 2024-09-10T09:43:36.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:36.675+0000 7f90a9cb3740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d9eced6d50 0x55d9ecedef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:43:36.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:36.675+0000 7f90a9cb3740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d9eced7670 0x55d9ecedf470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:43:36.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:36.675+0000 7f90a9cb3740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d9ecee1f30 0x55d9ecedf9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:43:36.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:36.675+0000 7f90a9cb3740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55d9ecee4ba0 con 0x55d9ecee1f30 2024-09-10T09:43:36.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:36.675+0000 7f90a9cb3740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55d9ecdf80f0 con 0x55d9eced7670 2024-09-10T09:43:36.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:36.675+0000 7f90a9cb3740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55d9ecee4d20 con 0x55d9eced6d50 2024-09-10T09:43:36.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:36.675+0000 7f90a9cb3740 10 client.0 _ll_register_callbacks cb 0x55d9ecde99e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:43:36.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:36.675+0000 7f9097e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d9eced7670 0x55d9ecedf470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:43:36.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:36.675+0000 7f909fe00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d9eced6d50 0x55d9ecedef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:43:36.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:36.675+0000 7f9097e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d9eced7670 0x55d9ecedf470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:37348/0 (socket says 172.21.15.5:37348) 2024-09-10T09:43:36.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:36.675+0000 7f9097e00640 1 -- 172.21.15.5:0/1171268163 learned_addr learned my addr 172.21.15.5:0/1171268163 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:43:36.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:36.676+0000 7f9097e00640 1 -- 172.21.15.5:0/1171268163 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d9eced6d50 msgr2=0x55d9ecedef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:43:36.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:36.676+0000 7f90a5200640 1 --2- 172.21.15.5:0/1171268163 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d9ecee1f30 0x55d9ecedf9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:43:36.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:36.676+0000 7f9097e00640 1 --2- 172.21.15.5:0/1171268163 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d9eced6d50 0x55d9ecedef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:36.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:36.676+0000 7f9097e00640 1 -- 172.21.15.5:0/1171268163 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d9ecee1f30 msgr2=0x55d9ecedf9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:43:36.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:36.676+0000 7f9097e00640 1 --2- 172.21.15.5:0/1171268163 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d9ecee1f30 0x55d9ecedf9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:36.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:36.676+0000 7f9097e00640 1 -- 172.21.15.5:0/1171268163 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d9ecbae3c0 con 0x55d9eced7670 2024-09-10T09:43:36.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:36.676+0000 7f90a5200640 1 --2- 172.21.15.5:0/1171268163 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d9ecee1f30 0x55d9ecedf9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T09:43:36.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:36.676+0000 7f9097e00640 1 --2- 172.21.15.5:0/1171268163 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d9eced7670 0x55d9ecedf470 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f9088004d40 tx=0x7f9088009530 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a1b97c4aaa3ac01d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:43:36.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:36.676+0000 7f908e000640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-09-10T09:43:36.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:36.676+0000 7f908e000640 1 -- 172.21.15.5:0/1171268163 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9088005230 con 0x55d9eced7670 2024-09-10T09:43:36.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:36.676+0000 7f90a9cb3740 1 -- 172.21.15.5:0/1171268163 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f90880047a0 con 0x55d9eced7670 2024-09-10T09:43:36.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:36.676+0000 7f908e000640 1 -- 172.21.15.5:0/1171268163 <== mon.2 v2:172.21.15.102:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f90880053d0 con 0x55d9eced7670 2024-09-10T09:43:36.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:36.677+0000 7f908e000640 1 -- 172.21.15.5:0/1171268163 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f9088035050 con 0x55d9eced7670 2024-09-10T09:43:36.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:36.677+0000 7f90a9cb3740 10 client.26465 fetch_fsmap learned FSMap version 34 2024-09-10T09:43:36.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:36.677+0000 7f90a9cb3740 1 -- 172.21.15.5:0/1171268163 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55d9ecef0970 con 0x55d9eced7670 2024-09-10T09:43:36.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:36.677+0000 7f908e000640 1 -- 172.21.15.5:0/1171268163 <== mon.2 v2:172.21.15.102:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f90880047a0 con 0x55d9eced7670 2024-09-10T09:43:36.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:36.677+0000 7f908e000640 1 -- 172.21.15.5:0/1171268163 <== mon.2 v2:172.21.15.102:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f908801d030 con 0x55d9eced7670 2024-09-10T09:43:36.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:36.677+0000 7f90a9cb3740 10 client.26465 fetch_fsmap finished waiting for FSMap version 34 2024-09-10T09:43:36.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:36.677+0000 7f90a9cb3740 10 client.26465 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T09:43:36.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:36.677+0000 7f90a9cb3740 20 client.26465 populate_metadata read hostname 'smithi005' 2024-09-10T09:43:36.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:36.678+0000 7f90a9cb3740 1 --2- 172.21.15.5:0/1171268163 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55d9ecef1cc0 0x55d9ecef40b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:43:36.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:36.678+0000 7f90a9cb3740 4 client.26465 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:43:36.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:36.678+0000 7f90a9cb3740 1 -- 172.21.15.5:0/1171268163 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d9ece4a950 con 0x55d9ecef1cc0 2024-09-10T09:43:36.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:36.678+0000 7f909fe00640 1 --2- 172.21.15.5:0/1171268163 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55d9ecef1cc0 0x55d9ecef40b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:43:36.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:36.679+0000 7f909fe00640 1 --2- 172.21.15.5:0/1171268163 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55d9ecef1cc0 0x55d9ecef40b0 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c9a03a16c60dad4f server_cookie=82876a0e866c6520 in_seq=0 out_seq=0 2024-09-10T09:43:36.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:36.679+0000 7f908e000640 10 client.26465 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T09:43:36.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:36.679+0000 7f908e000640 1 -- 172.21.15.5:0/1171268163 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d9ece4a950 con 0x55d9ecef1cc0 2024-09-10T09:43:36.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:36.679+0000 7f908e000640 10 client.26465 handle_command_reply: tid=0 2024-09-10T09:43:36.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:36.679+0000 7f90a9cb3740 1 client.26465 shutdown 2024-09-10T09:43:36.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:36.680+0000 7f90a9cb3740 1 -- 172.21.15.5:0/1171268163 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d9eced7670 msgr2=0x55d9ecedf470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:43:36.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:36.680+0000 7f90a9cb3740 1 --2- 172.21.15.5:0/1171268163 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d9eced7670 0x55d9ecedf470 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f9088004d40 tx=0x7f9088009530 comp rx=0 tx=0).stop 2024-09-10T09:43:36.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:36.680+0000 7f90a9cb3740 1 -- 172.21.15.5:0/1171268163 shutdown_connections 2024-09-10T09:43:36.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:36.680+0000 7f90a9cb3740 1 --2- 172.21.15.5:0/1171268163 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55d9ecef1cc0 0x55d9ecef40b0 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:36.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:36.680+0000 7f90a9cb3740 1 --2- 172.21.15.5:0/1171268163 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d9ecee1f30 0x55d9ecedf9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:36.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:36.680+0000 7f90a9cb3740 1 --2- 172.21.15.5:0/1171268163 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d9eced7670 0x55d9ecedf470 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:36.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:36.680+0000 7f90a9cb3740 1 --2- 172.21.15.5:0/1171268163 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d9eced6d50 0x55d9ecedef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:36.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:36.680+0000 7f908e000640 0 client.26465 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T09:43:36.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:36.680+0000 7f90a9cb3740 1 -- 172.21.15.5:0/1171268163 >> 172.21.15.5:0/1171268163 conn(0x55d9eced32e0 msgr2=0x55d9eced9010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:43:36.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:36.680+0000 7f90a9cb3740 1 -- 172.21.15.5:0/1171268163 shutdown_connections 2024-09-10T09:43:36.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:36.680+0000 7f90a9cb3740 1 -- 172.21.15.5:0/1171268163 wait complete. 2024-09-10T09:43:36.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:36.680+0000 7f90a9cb3740 20 client.26465 trim_cache size 0 max 16384 2024-09-10T09:43:36.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:36.717+0000 7f90a9cb3740 1 Processor -- start 2024-09-10T09:43:36.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:36.717+0000 7f90a9cb3740 1 -- start start 2024-09-10T09:43:36.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:36.718+0000 7f90a9cb3740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d9eced6d50 0x55d9ecedf800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:43:36.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:36.718+0000 7f90a9cb3740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d9ecee1f30 0x55d9eced8f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:43:36.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:36.718+0000 7f90a9cb3740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d9ecee0fb0 0x55d9ecef97a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:43:36.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:36.718+0000 7f90a9cb3740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f90a01b0fa0 con 0x55d9eced6d50 2024-09-10T09:43:36.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:36.718+0000 7f90a9cb3740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f90a01b2ae0 con 0x55d9ecee1f30 2024-09-10T09:43:36.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:36.718+0000 7f90a9cb3740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f90a01b1670 con 0x55d9ecee0fb0 2024-09-10T09:43:36.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:36.718+0000 7f9097e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d9ecee1f30 0x55d9eced8f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:43:36.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:36.718+0000 7f909fe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d9eced6d50 0x55d9ecedf800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:43:36.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:36.718+0000 7f9097e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d9ecee1f30 0x55d9eced8f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:37356/0 (socket says 172.21.15.5:37356) 2024-09-10T09:43:36.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:36.718+0000 7f9097e00640 1 -- 172.21.15.5:0/743716255 learned_addr learned my addr 172.21.15.5:0/743716255 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:43:36.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:36.718+0000 7f90a5200640 1 --2- 172.21.15.5:0/743716255 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d9ecee0fb0 0x55d9ecef97a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:43:36.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:36.718+0000 7f9097e00640 1 -- 172.21.15.5:0/743716255 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d9ecee0fb0 msgr2=0x55d9ecef97a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:43:36.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:36.719+0000 7f9097e00640 1 --2- 172.21.15.5:0/743716255 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d9ecee0fb0 0x55d9ecef97a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:36.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:36.719+0000 7f9097e00640 1 -- 172.21.15.5:0/743716255 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d9eced6d50 msgr2=0x55d9ecedf800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:43:36.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:36.719+0000 7f9097e00640 1 --2- 172.21.15.5:0/743716255 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d9eced6d50 0x55d9ecedf800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:36.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:36.719+0000 7f9097e00640 1 -- 172.21.15.5:0/743716255 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d9eced4390 con 0x55d9ecee1f30 2024-09-10T09:43:36.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:36.719+0000 7f909fe00640 1 --2- 172.21.15.5:0/743716255 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d9eced6d50 0x55d9ecedf800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:43:36.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:36.719+0000 7f9097e00640 1 --2- 172.21.15.5:0/743716255 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d9ecee1f30 0x55d9eced8f00 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f9088009620 tx=0x7f90880116d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ba6a9a5b6fd03d5b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:43:36.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:36.719+0000 7f9094c00640 1 -- 172.21.15.5:0/743716255 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9088005430 con 0x55d9ecee1f30 2024-09-10T09:43:36.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:36.719+0000 7f9094c00640 1 -- 172.21.15.5:0/743716255 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f908802c070 con 0x55d9ecee1f30 2024-09-10T09:43:36.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:36.719+0000 7f9094c00640 1 -- 172.21.15.5:0/743716255 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90880273e0 con 0x55d9ecee1f30 2024-09-10T09:43:36.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:36.720+0000 7f90a9cb3740 1 -- 172.21.15.5:0/743716255 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d9ecee1f30 msgr2=0x55d9eced8f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:43:36.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:36.720+0000 7f90a9cb3740 1 --2- 172.21.15.5:0/743716255 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d9ecee1f30 0x55d9eced8f00 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f9088009620 tx=0x7f90880116d0 comp rx=0 tx=0).stop 2024-09-10T09:43:36.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:36.720+0000 7f90a9cb3740 1 -- 172.21.15.5:0/743716255 shutdown_connections 2024-09-10T09:43:36.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:36.720+0000 7f90a9cb3740 1 --2- 172.21.15.5:0/743716255 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d9ecee0fb0 0x55d9ecef97a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:36.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:36.720+0000 7f90a9cb3740 1 --2- 172.21.15.5:0/743716255 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d9ecee1f30 0x55d9eced8f00 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:36.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:36.720+0000 7f90a9cb3740 1 --2- 172.21.15.5:0/743716255 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d9eced6d50 0x55d9ecedf800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:36.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:36.720+0000 7f90a9cb3740 1 -- 172.21.15.5:0/743716255 >> 172.21.15.5:0/743716255 conn(0x55d9eced32e0 msgr2=0x55d9eced87f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:43:36.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:36.721+0000 7f90a9cb3740 1 -- 172.21.15.5:0/743716255 shutdown_connections 2024-09-10T09:43:36.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:36.721+0000 7f90a9cb3740 1 -- 172.21.15.5:0/743716255 wait complete. 2024-09-10T09:43:36.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:36.721+0000 7f90a9cb3740 1 -- start start 2024-09-10T09:43:36.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:36.721+0000 7f90a9cb3740 1 Processor -- start 2024-09-10T09:43:36.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:36.722+0000 7f90a9cb3740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d9eced6d50 0x55d9ecee9f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:43:36.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:36.722+0000 7f90a9cb3740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d9ecee0fb0 0x55d9eceea4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:43:36.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:36.722+0000 7f90a9cb3740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d9ecee1f30 0x55d9ecee6f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:43:36.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:36.722+0000 7f90a9cb3740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f90a01b2410 con 0x55d9eced6d50 2024-09-10T09:43:36.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:36.722+0000 7f90a9cb3740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f90a01b1d40 con 0x55d9ecee1f30 2024-09-10T09:43:36.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:36.722+0000 7f90a9cb3740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f90a01aef80 con 0x55d9ecee0fb0 2024-09-10T09:43:36.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:36.722+0000 7f90a9cb3740 10 client.0 _ll_register_callbacks cb 0x7f9088005f90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:43:36.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:36.722+0000 7f9097e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d9ecee0fb0 0x55d9eceea4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:43:36.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:36.722+0000 7f909fe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d9eced6d50 0x55d9ecee9f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:43:36.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:36.722+0000 7f9097e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d9ecee0fb0 0x55d9eceea4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:38198/0 (socket says 172.21.15.5:38198) 2024-09-10T09:43:36.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:36.722+0000 7f9097e00640 1 -- 172.21.15.5:0/3560719679 learned_addr learned my addr 172.21.15.5:0/3560719679 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:43:36.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:36.722+0000 7f90a5200640 1 --2- 172.21.15.5:0/3560719679 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d9ecee1f30 0x55d9ecee6f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:43:36.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:36.723+0000 7f9097e00640 1 -- 172.21.15.5:0/3560719679 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d9ecee1f30 msgr2=0x55d9ecee6f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:43:36.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:36.723+0000 7f9097e00640 1 --2- 172.21.15.5:0/3560719679 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d9ecee1f30 0x55d9ecee6f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:36.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:36.723+0000 7f9097e00640 1 -- 172.21.15.5:0/3560719679 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d9eced6d50 msgr2=0x55d9ecee9f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:43:36.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:36.723+0000 7f9097e00640 1 --2- 172.21.15.5:0/3560719679 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d9eced6d50 0x55d9ecee9f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:36.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:36.723+0000 7f9097e00640 1 -- 172.21.15.5:0/3560719679 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d9eceeabf0 con 0x55d9ecee0fb0 2024-09-10T09:43:36.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:36.723+0000 7f9097e00640 1 --2- 172.21.15.5:0/3560719679 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d9ecee0fb0 0x55d9eceea4b0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f9088004000 tx=0x7f9088005730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e3cc3afa5b3ab610 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:43:36.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:36.723+0000 7f908e000640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-09-10T09:43:36.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:36.723+0000 7f908e000640 1 -- 172.21.15.5:0/3560719679 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9088005430 con 0x55d9ecee0fb0 2024-09-10T09:43:36.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:36.723+0000 7f90a9cb3740 1 -- 172.21.15.5:0/3560719679 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f908802c070 con 0x55d9ecee0fb0 2024-09-10T09:43:36.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:36.723+0000 7f908e000640 1 -- 172.21.15.5:0/3560719679 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f90880273e0 con 0x55d9ecee0fb0 2024-09-10T09:43:36.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:36.724+0000 7f908e000640 1 -- 172.21.15.5:0/3560719679 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f9088004d70 con 0x55d9ecee0fb0 2024-09-10T09:43:36.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:36.724+0000 7f90a9cb3740 10 client.26500 fetch_fsmap learned FSMap version 34 2024-09-10T09:43:36.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:36.724+0000 7f90a9cb3740 1 -- 172.21.15.5:0/3560719679 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55d9ecf5cdd0 con 0x55d9ecee0fb0 2024-09-10T09:43:36.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:36.724+0000 7f908e000640 1 -- 172.21.15.5:0/3560719679 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f908802c070 con 0x55d9ecee0fb0 2024-09-10T09:43:36.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:36.724+0000 7f908e000640 1 -- 172.21.15.5:0/3560719679 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f908801d040 con 0x55d9ecee0fb0 2024-09-10T09:43:36.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:36.724+0000 7f90a9cb3740 10 client.26500 fetch_fsmap finished waiting for FSMap version 34 2024-09-10T09:43:36.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:36.724+0000 7f90a9cb3740 10 client.26500 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T09:43:36.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:36.724+0000 7f90a9cb3740 20 client.26500 populate_metadata read hostname 'smithi005' 2024-09-10T09:43:36.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:36.724+0000 7f90a9cb3740 1 --2- 172.21.15.5:0/3560719679 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55d9ecf5d400 0x55d9ecf5f820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:43:36.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:36.724+0000 7f90a9cb3740 4 client.26500 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T09:43:36.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:36.724+0000 7f90a9cb3740 1 -- 172.21.15.5:0/3560719679 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55d9ece44e60 con 0x55d9ecf5d400 2024-09-10T09:43:36.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:36.725+0000 7f909fe00640 1 --2- 172.21.15.5:0/3560719679 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55d9ecf5d400 0x55d9ecf5f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:43:36.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:36.725+0000 7f909fe00640 1 --2- 172.21.15.5:0/3560719679 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55d9ecf5d400 0x55d9ecf5f820 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=173a9549b157311 server_cookie=d58bdea64d60fb2a in_seq=0 out_seq=0 2024-09-10T09:43:36.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:36.725+0000 7f908e000640 10 client.26500 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T09:43:36.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:36.725+0000 7f908e000640 1 -- 172.21.15.5:0/3560719679 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55d9ece44e60 con 0x55d9ecf5d400 2024-09-10T09:43:36.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:36.725+0000 7f908e000640 10 client.26500 handle_command_reply: tid=0 2024-09-10T09:43:36.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:36.725+0000 7f90a9cb3740 1 client.26500 shutdown 2024-09-10T09:43:36.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:36.726+0000 7f90a9cb3740 1 -- 172.21.15.5:0/3560719679 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d9ecee0fb0 msgr2=0x55d9eceea4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:43:36.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:36.726+0000 7f90a9cb3740 1 --2- 172.21.15.5:0/3560719679 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d9ecee0fb0 0x55d9eceea4b0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f9088004000 tx=0x7f9088005730 comp rx=0 tx=0).stop 2024-09-10T09:43:36.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:36.726+0000 7f90a9cb3740 1 -- 172.21.15.5:0/3560719679 shutdown_connections 2024-09-10T09:43:36.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:36.726+0000 7f90a9cb3740 1 --2- 172.21.15.5:0/3560719679 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55d9ecf5d400 0x55d9ecf5f820 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:36.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:36.726+0000 7f90a9cb3740 1 --2- 172.21.15.5:0/3560719679 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d9ecee1f30 0x55d9ecee6f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:36.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:36.726+0000 7f90a9cb3740 1 --2- 172.21.15.5:0/3560719679 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d9ecee0fb0 0x55d9eceea4b0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:36.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:36.726+0000 7f90a9cb3740 1 --2- 172.21.15.5:0/3560719679 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d9eced6d50 0x55d9ecee9f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:36.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:36.726+0000 7f908e000640 0 client.26500 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T09:43:36.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:36.726+0000 7f90a9cb3740 1 -- 172.21.15.5:0/3560719679 >> 172.21.15.5:0/3560719679 conn(0x55d9eced32e0 msgr2=0x55d9eceed610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:43:36.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:36.726+0000 7f90a9cb3740 1 -- 172.21.15.5:0/3560719679 shutdown_connections 2024-09-10T09:43:36.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:36.726+0000 7f90a9cb3740 1 -- 172.21.15.5:0/3560719679 wait complete. 2024-09-10T09:43:36.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:36.726+0000 7f90a9cb3740 20 client.26500 trim_cache size 0 max 16384 2024-09-10T09:43:36.730 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-10T09:43:36.730 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-10T09:43:36.730 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2024-09-10T09:43:36.730 INFO:teuthology.orchestra.run.smithi005.stdout: "5c923e45-ce30-445f-9933-9923624272dc": { 2024-09-10T09:43:36.731 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2024-09-10T09:43:36.731 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "5c923e45-ce30-445f-9933-9923624272dc", 2024-09-10T09:43:36.731 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2024-09-10T09:43:36.731 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T09:43:36.731 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T09:43:36.731 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-10T09:43:36.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:36.727+0000 7f90a5c00640 1 -- 172.21.15.5:0/3932560449 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f90840a2b80 msgr2=0x7f90840a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:43:36.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:36.727+0000 7f90a5c00640 1 --2- 172.21.15.5:0/3932560449 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f90840a2b80 0x7f90840a5040 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f9090000c00 tx=0x7f9090031510 comp rx=0 tx=0).stop 2024-09-10T09:43:36.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:36.727+0000 7f90a5c00640 1 -- 172.21.15.5:0/3932560449 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f90a010e2d0 msgr2=0x7f90a019ee50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:43:36.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:36.727+0000 7f90a5c00640 1 --2- 172.21.15.5:0/3932560449 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f90a010e2d0 0x7f90a019ee50 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f90a0102900 tx=0x7f909800ef10 comp rx=0 tx=0).stop 2024-09-10T09:43:36.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:36.727+0000 7f90a5c00640 1 -- 172.21.15.5:0/3932560449 shutdown_connections 2024-09-10T09:43:36.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:36.727+0000 7f90a5c00640 1 --2- 172.21.15.5:0/3932560449 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f90840a2b80 0x7f90840a5040 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:36.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:36.727+0000 7f90a5c00640 1 --2- 172.21.15.5:0/3932560449 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f90a010e2d0 0x7f90a019ee50 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:36.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:36.727+0000 7f90a5c00640 1 --2- 172.21.15.5:0/3932560449 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f90a01010d0 0x7f90a019aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:36.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:36.727+0000 7f90a5c00640 1 --2- 172.21.15.5:0/3932560449 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f90a0100720 0x7f90a019a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:43:36.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:36.727+0000 7f90a5c00640 1 -- 172.21.15.5:0/3932560449 >> 172.21.15.5:0/3932560449 conn(0x7f90a00fc610 msgr2=0x7f90a00fea00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:43:36.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:36.728+0000 7f90a5c00640 1 -- 172.21.15.5:0/3932560449 shutdown_connections 2024-09-10T09:43:36.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:43:36.728+0000 7f90a5c00640 1 -- 172.21.15.5:0/3932560449 wait complete. 2024-09-10T09:43:37.040 INFO:tasks.workunit.client.0.smithi005.stdout: 1 266176 5.13 MB/sec execute 78 sec latency 17.070 ms 2024-09-10T09:43:37.774 INFO:tasks.cephfs.filesystem:scrub status for tag:5c923e45-ce30-445f-9933-9923624272dc - {'path': '/', 'tag': '5c923e45-ce30-445f-9933-9923624272dc', 'options': 'recursive,force'} 2024-09-10T09:43:37.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:43:37 smithi102 ceph-mon[35442]: pgmap v551: 129 pgs: 129 active+clean; 116 MiB data, 1016 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 1.4 MiB/s wr, 231 op/s 2024-09-10T09:43:38.041 INFO:tasks.workunit.client.0.smithi005.stdout: 1 267921 5.15 MB/sec execute 79 sec latency 42.134 ms 2024-09-10T09:43:38.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:43:37 smithi005 ceph-mon[30350]: pgmap v551: 129 pgs: 129 active+clean; 116 MiB data, 1016 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 1.4 MiB/s wr, 231 op/s 2024-09-10T09:43:38.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:43:37 smithi159 ceph-mon[35513]: pgmap v551: 129 pgs: 129 active+clean; 116 MiB data, 1016 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 1.4 MiB/s wr, 231 op/s 2024-09-10T09:43:39.040 INFO:tasks.workunit.client.0.smithi005.stdout: 1 269907 5.25 MB/sec execute 80 sec latency 246.967 ms 2024-09-10T09:43:39.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:43:39 smithi102 ceph-mon[35442]: pgmap v552: 129 pgs: 129 active+clean; 116 MiB data, 1015 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.2 MiB/s wr, 180 op/s 2024-09-10T09:43:40.041 INFO:tasks.workunit.client.0.smithi005.stdout: 1 270408 5.20 MB/sec execute 81 sec latency 629.560 ms 2024-09-10T09:43:40.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:43:39 smithi005 ceph-mon[30350]: pgmap v552: 129 pgs: 129 active+clean; 116 MiB data, 1015 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.2 MiB/s wr, 180 op/s 2024-09-10T09:43:40.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:43:39 smithi159 ceph-mon[35513]: pgmap v552: 129 pgs: 129 active+clean; 116 MiB data, 1015 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.2 MiB/s wr, 180 op/s 2024-09-10T09:43:40.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:43:40 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:43:41.041 INFO:tasks.workunit.client.0.smithi005.stdout: 1 272809 5.29 MB/sec execute 82 sec latency 14.881 ms 2024-09-10T09:43:41.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:43:40 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:43:41.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:43:40 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:43:41.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:43:41 smithi102 ceph-mon[35442]: pgmap v553: 129 pgs: 129 active+clean; 118 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.3 MiB/s wr, 190 op/s 2024-09-10T09:43:41.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:43:41 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:43:41.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:43:41 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:43:41.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:43:41 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:43:41.885 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:43:41 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:43:41.885 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:43:41 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:43:41.885 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:43:41 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:43:41.885 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:43:41 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:43:42.041 INFO:tasks.workunit.client.0.smithi005.stdout: 1 274623 5.36 MB/sec execute 83 sec latency 20.337 ms 2024-09-10T09:43:42.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:43:41 smithi005 ceph-mon[30350]: pgmap v553: 129 pgs: 129 active+clean; 118 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.3 MiB/s wr, 190 op/s 2024-09-10T09:43:42.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:43:41 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:43:42.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:43:41 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:43:42.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:43:41 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:43:42.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:43:41 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:43:42.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:43:41 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:43:42.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:43:41 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:43:42.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:43:41 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:43:42.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:43:41 smithi159 ceph-mon[35513]: pgmap v553: 129 pgs: 129 active+clean; 118 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.3 MiB/s wr, 190 op/s 2024-09-10T09:43:42.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:43:41 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:43:42.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:43:41 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:43:42.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:43:41 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:43:42.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:43:41 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:43:42.101 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:43:41 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:43:42.101 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:43:41 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:43:42.101 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:43:41 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:43:43.041 INFO:tasks.workunit.client.0.smithi005.stdout: 1 277136 5.46 MB/sec execute 84 sec latency 8.876 ms 2024-09-10T09:43:43.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:43:42 smithi005 ceph-mon[30350]: Adjusting osd_memory_target on smithi005 to 177.5M 2024-09-10T09:43:43.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:43:42 smithi005 ceph-mon[30350]: Unable to set osd_memory_target on smithi005 to 186147225: error parsing value: Value '186147225' is below minimum 939524096 2024-09-10T09:43:43.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:43:42 smithi159 ceph-mon[35513]: Adjusting osd_memory_target on smithi005 to 177.5M 2024-09-10T09:43:43.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:43:42 smithi159 ceph-mon[35513]: Unable to set osd_memory_target on smithi005 to 186147225: error parsing value: Value '186147225' is below minimum 939524096 2024-09-10T09:43:43.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:43:42 smithi102 ceph-mon[35442]: Adjusting osd_memory_target on smithi005 to 177.5M 2024-09-10T09:43:43.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:43:42 smithi102 ceph-mon[35442]: Unable to set osd_memory_target on smithi005 to 186147225: error parsing value: Value '186147225' is below minimum 939524096 2024-09-10T09:43:43.912 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:43:43 smithi005 ceph-mon[30350]: pgmap v554: 129 pgs: 129 active+clean; 121 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.4 MiB/s wr, 179 op/s 2024-09-10T09:43:44.041 INFO:tasks.workunit.client.0.smithi005.stdout: 1 278953 5.47 MB/sec execute 85 sec latency 129.052 ms 2024-09-10T09:43:44.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:43:43 smithi159 ceph-mon[35513]: pgmap v554: 129 pgs: 129 active+clean; 121 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.4 MiB/s wr, 179 op/s 2024-09-10T09:43:44.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:43:43 smithi102 ceph-mon[35442]: pgmap v554: 129 pgs: 129 active+clean; 121 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.4 MiB/s wr, 179 op/s 2024-09-10T09:43:44.769 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:43:44 smithi005 ceph-mon[30350]: pgmap v555: 129 pgs: 129 active+clean; 121 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.4 MiB/s wr, 170 op/s 2024-09-10T09:43:45.041 INFO:tasks.workunit.client.0.smithi005.stdout: 1 278987 5.41 MB/sec execute 86 sec latency 745.499 ms 2024-09-10T09:43:45.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:43:44 smithi159 ceph-mon[35513]: pgmap v555: 129 pgs: 129 active+clean; 121 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.4 MiB/s wr, 170 op/s 2024-09-10T09:43:45.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:43:44 smithi102 ceph-mon[35442]: pgmap v555: 129 pgs: 129 active+clean; 121 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.4 MiB/s wr, 170 op/s 2024-09-10T09:43:46.041 INFO:tasks.workunit.client.0.smithi005.stdout: 1 280250 5.47 MB/sec execute 87 sec latency 1284.807 ms 2024-09-10T09:43:47.041 INFO:tasks.workunit.client.0.smithi005.stdout: 1 281851 5.49 MB/sec execute 88 sec latency 27.465 ms 2024-09-10T09:43:47.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:43:46 smithi005 ceph-mon[30350]: pgmap v556: 129 pgs: 129 active+clean; 127 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.1 MiB/s wr, 230 op/s 2024-09-10T09:43:47.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:43:46 smithi159 ceph-mon[35513]: pgmap v556: 129 pgs: 129 active+clean; 127 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.1 MiB/s wr, 230 op/s 2024-09-10T09:43:47.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:43:46 smithi102 ceph-mon[35442]: pgmap v556: 129 pgs: 129 active+clean; 127 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.1 MiB/s wr, 230 op/s 2024-09-10T09:43:47.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:43:47 smithi005 ceph-mon[30350]: scrub summary: idle 2024-09-10T09:43:47.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:43:47 smithi005 ceph-mon[30350]: scrub summary: idle 2024-09-10T09:43:48.041 INFO:tasks.workunit.client.0.smithi005.stdout: 1 284378 5.59 MB/sec execute 89 sec latency 8.820 ms 2024-09-10T09:43:48.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:43:47 smithi159 ceph-mon[35513]: scrub summary: idle 2024-09-10T09:43:48.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:43:47 smithi159 ceph-mon[35513]: scrub summary: idle 2024-09-10T09:43:48.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:43:47 smithi102 ceph-mon[35442]: scrub summary: idle 2024-09-10T09:43:48.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:43:47 smithi102 ceph-mon[35442]: scrub summary: idle 2024-09-10T09:43:49.004 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:43:48 smithi005 ceph-mon[30350]: pgmap v557: 129 pgs: 129 active+clean; 128 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.0 MiB/s wr, 169 op/s 2024-09-10T09:43:49.041 INFO:tasks.workunit.client.0.smithi005.stdout: 1 286725 5.66 MB/sec execute 90 sec latency 50.445 ms 2024-09-10T09:43:49.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:43:48 smithi159 ceph-mon[35513]: pgmap v557: 129 pgs: 129 active+clean; 128 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.0 MiB/s wr, 169 op/s 2024-09-10T09:43:49.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:43:48 smithi102 ceph-mon[35442]: pgmap v557: 129 pgs: 129 active+clean; 128 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.0 MiB/s wr, 169 op/s 2024-09-10T09:43:50.041 INFO:tasks.workunit.client.0.smithi005.stdout: 1 286751 5.60 MB/sec execute 91 sec latency 669.219 ms 2024-09-10T09:43:51.041 INFO:tasks.workunit.client.0.smithi005.stdout: 1 287672 5.61 MB/sec execute 92 sec latency 1216.289 ms 2024-09-10T09:43:51.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:43:50 smithi005 ceph-mon[30350]: pgmap v558: 129 pgs: 129 active+clean; 128 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.5 MiB/s wr, 228 op/s 2024-09-10T09:43:51.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:43:50 smithi159 ceph-mon[35513]: pgmap v558: 129 pgs: 129 active+clean; 128 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.5 MiB/s wr, 228 op/s 2024-09-10T09:43:51.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:43:50 smithi102 ceph-mon[35442]: pgmap v558: 129 pgs: 129 active+clean; 128 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.5 MiB/s wr, 228 op/s 2024-09-10T09:43:52.041 INFO:tasks.workunit.client.0.smithi005.stdout: 1 289941 5.64 MB/sec execute 93 sec latency 28.204 ms 2024-09-10T09:43:53.041 INFO:tasks.workunit.client.0.smithi005.stdout: 1 292042 5.72 MB/sec execute 94 sec latency 18.488 ms 2024-09-10T09:43:53.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:43:52 smithi005 ceph-mon[30350]: pgmap v559: 129 pgs: 129 active+clean; 127 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.6 MiB/s wr, 227 op/s 2024-09-10T09:43:53.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:43:52 smithi159 ceph-mon[35513]: pgmap v559: 129 pgs: 129 active+clean; 127 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.6 MiB/s wr, 227 op/s 2024-09-10T09:43:53.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:43:52 smithi102 ceph-mon[35442]: pgmap v559: 129 pgs: 129 active+clean; 127 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.6 MiB/s wr, 227 op/s 2024-09-10T09:43:54.041 INFO:tasks.workunit.client.0.smithi005.stdout: 1 294655 5.83 MB/sec execute 95 sec latency 19.691 ms 2024-09-10T09:43:54.964 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:43:54 smithi005 ceph-mon[30350]: pgmap v560: 129 pgs: 129 active+clean; 127 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.3 MiB/s wr, 178 op/s 2024-09-10T09:43:55.041 INFO:tasks.workunit.client.0.smithi005.stdout: 1 294746 5.77 MB/sec execute 96 sec latency 808.804 ms 2024-09-10T09:43:55.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:43:54 smithi159 ceph-mon[35513]: pgmap v560: 129 pgs: 129 active+clean; 127 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.3 MiB/s wr, 178 op/s 2024-09-10T09:43:55.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:43:54 smithi102 ceph-mon[35442]: pgmap v560: 129 pgs: 129 active+clean; 127 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.3 MiB/s wr, 178 op/s 2024-09-10T09:43:56.042 INFO:tasks.workunit.client.0.smithi005.stdout: 1 294993 5.72 MB/sec execute 97 sec latency 1592.096 ms 2024-09-10T09:43:57.042 INFO:tasks.workunit.client.0.smithi005.stdout: 1 297444 5.79 MB/sec execute 98 sec latency 36.341 ms 2024-09-10T09:43:57.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:43:56 smithi005 ceph-mon[30350]: pgmap v561: 129 pgs: 129 active+clean; 132 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.7 MiB/s wr, 231 op/s 2024-09-10T09:43:57.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:43:56 smithi159 ceph-mon[35513]: pgmap v561: 129 pgs: 129 active+clean; 132 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.7 MiB/s wr, 231 op/s 2024-09-10T09:43:57.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:43:56 smithi102 ceph-mon[35442]: pgmap v561: 129 pgs: 129 active+clean; 132 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.7 MiB/s wr, 231 op/s 2024-09-10T09:43:58.042 INFO:tasks.workunit.client.0.smithi005.stdout: 1 299426 5.84 MB/sec execute 99 sec latency 20.888 ms 2024-09-10T09:43:59.042 INFO:tasks.workunit.client.0.smithi005.stdout: 1 301911 5.93 MB/sec execute 100 sec latency 10.765 ms 2024-09-10T09:43:59.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:43:58 smithi005 ceph-mon[30350]: pgmap v562: 129 pgs: 129 active+clean; 132 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.3 MiB/s wr, 178 op/s 2024-09-10T09:43:59.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:43:58 smithi159 ceph-mon[35513]: pgmap v562: 129 pgs: 129 active+clean; 132 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.3 MiB/s wr, 178 op/s 2024-09-10T09:43:59.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:43:58 smithi102 ceph-mon[35442]: pgmap v562: 129 pgs: 129 active+clean; 132 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.3 MiB/s wr, 178 op/s 2024-09-10T09:44:00.042 INFO:tasks.workunit.client.0.smithi005.stdout: 1 302184 5.87 MB/sec execute 101 sec latency 575.366 ms 2024-09-10T09:44:00.957 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:44:00 smithi005 ceph-mon[30350]: pgmap v563: 129 pgs: 129 active+clean; 132 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.5 MiB/s wr, 221 op/s 2024-09-10T09:44:01.042 INFO:tasks.workunit.client.0.smithi005.stdout: 1 302184 5.81 MB/sec execute 102 sec latency 1575.424 ms 2024-09-10T09:44:01.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:44:00 smithi159 ceph-mon[35513]: pgmap v563: 129 pgs: 129 active+clean; 132 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.5 MiB/s wr, 221 op/s 2024-09-10T09:44:01.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:44:00 smithi102 ceph-mon[35442]: pgmap v563: 129 pgs: 129 active+clean; 132 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.5 MiB/s wr, 221 op/s 2024-09-10T09:44:02.042 INFO:tasks.workunit.client.0.smithi005.stdout: 1 304542 5.88 MB/sec execute 103 sec latency 1579.213 ms 2024-09-10T09:44:03.042 INFO:tasks.workunit.client.0.smithi005.stdout: 1 306553 5.93 MB/sec execute 104 sec latency 20.754 ms 2024-09-10T09:44:03.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:44:02 smithi005 ceph-mon[30350]: pgmap v564: 129 pgs: 129 active+clean; 133 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.4 MiB/s wr, 182 op/s 2024-09-10T09:44:03.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:44:02 smithi159 ceph-mon[35513]: pgmap v564: 129 pgs: 129 active+clean; 133 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.4 MiB/s wr, 182 op/s 2024-09-10T09:44:03.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:44:02 smithi102 ceph-mon[35442]: pgmap v564: 129 pgs: 129 active+clean; 133 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.4 MiB/s wr, 182 op/s 2024-09-10T09:44:04.042 INFO:tasks.workunit.client.0.smithi005.stdout: 1 309138 6.01 MB/sec execute 105 sec latency 10.639 ms 2024-09-10T09:44:04.915 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:44:04 smithi005 ceph-mon[30350]: pgmap v565: 129 pgs: 129 active+clean; 133 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.3 MiB/s wr, 176 op/s 2024-09-10T09:44:05.042 INFO:tasks.workunit.client.0.smithi005.stdout: 1 309525 5.96 MB/sec execute 106 sec latency 681.510 ms 2024-09-10T09:44:05.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:44:04 smithi159 ceph-mon[35513]: pgmap v565: 129 pgs: 129 active+clean; 133 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.3 MiB/s wr, 176 op/s 2024-09-10T09:44:05.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:44:04 smithi102 ceph-mon[35442]: pgmap v565: 129 pgs: 129 active+clean; 133 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.3 MiB/s wr, 176 op/s 2024-09-10T09:44:06.042 INFO:tasks.workunit.client.0.smithi005.stdout: 1 309525 5.90 MB/sec execute 107 sec latency 1681.573 ms 2024-09-10T09:44:07.042 INFO:tasks.workunit.client.0.smithi005.stdout: 1 311815 5.97 MB/sec execute 108 sec latency 1832.131 ms 2024-09-10T09:44:07.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:44:06 smithi005 ceph-mon[30350]: pgmap v566: 129 pgs: 129 active+clean; 133 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.5 MiB/s wr, 195 op/s 2024-09-10T09:44:07.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:44:06 smithi159 ceph-mon[35513]: pgmap v566: 129 pgs: 129 active+clean; 133 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.5 MiB/s wr, 195 op/s 2024-09-10T09:44:07.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:44:06 smithi102 ceph-mon[35442]: pgmap v566: 129 pgs: 129 active+clean; 133 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.5 MiB/s wr, 195 op/s 2024-09-10T09:44:07.776 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T09:44:08.042 INFO:tasks.workunit.client.0.smithi005.stdout: 1 313558 6.01 MB/sec execute 109 sec latency 19.657 ms 2024-09-10T09:44:08.201 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:44:08.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:08.849+0000 7fcc00c00640 1 -- 172.21.15.5:0/2638689760 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcbfc103ff0 msgr2=0x7fcbfc1043d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:44:08.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:08.849+0000 7fcc00c00640 1 --2- 172.21.15.5:0/2638689760 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcbfc103ff0 0x7fcbfc1043d0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fcbec009900 tx=0x7fcbec02a410 comp rx=0 tx=0).stop 2024-09-10T09:44:08.851 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:44:08 smithi005 ceph-mon[30350]: pgmap v567: 129 pgs: 129 active+clean; 136 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.4 MiB/s wr, 180 op/s 2024-09-10T09:44:08.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:08.849+0000 7fcc00c00640 1 -- 172.21.15.5:0/2638689760 shutdown_connections 2024-09-10T09:44:08.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:08.849+0000 7fcc00c00640 1 --2- 172.21.15.5:0/2638689760 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcbfc109120 0x7fcbfc110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:08.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:08.849+0000 7fcc00c00640 1 --2- 172.21.15.5:0/2638689760 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcbfc1049a0 0x7fcbfc1089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:08.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:08.849+0000 7fcc00c00640 1 --2- 172.21.15.5:0/2638689760 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcbfc103ff0 0x7fcbfc1043d0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:08.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:08.849+0000 7fcc00c00640 1 -- 172.21.15.5:0/2638689760 >> 172.21.15.5:0/2638689760 conn(0x7fcbfc0ffa90 msgr2=0x7fcbfc101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:44:08.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:08.849+0000 7fcc00c00640 1 -- 172.21.15.5:0/2638689760 shutdown_connections 2024-09-10T09:44:08.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:08.849+0000 7fcc00c00640 1 -- 172.21.15.5:0/2638689760 wait complete. 2024-09-10T09:44:08.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:08.851+0000 7fcc00c00640 1 Processor -- start 2024-09-10T09:44:08.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:08.851+0000 7fcc00c00640 1 -- start start 2024-09-10T09:44:08.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:08.851+0000 7fcc00c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcbfc103ff0 0x7fcbfc19a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:44:08.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:08.851+0000 7fcc00c00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcbfc1049a0 0x7fcbfc19ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:44:08.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:08.851+0000 7fcc00c00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcbfc109120 0x7fcbfc19eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:44:08.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:08.851+0000 7fcc00c00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fcbfc114120 con 0x7fcbfc1049a0 2024-09-10T09:44:08.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:08.851+0000 7fcc00c00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fcbfc113fa0 con 0x7fcbfc103ff0 2024-09-10T09:44:08.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:08.851+0000 7fcc00c00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fcbfc1142a0 con 0x7fcbfc109120 2024-09-10T09:44:08.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:08.851+0000 7fcbf9e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcbfc103ff0 0x7fcbfc19a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:44:08.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:08.851+0000 7fcbf9e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcbfc103ff0 0x7fcbfc19a5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:53264/0 (socket says 172.21.15.5:53264) 2024-09-10T09:44:08.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:08.851+0000 7fcbf9e00640 1 -- 172.21.15.5:0/1345133073 learned_addr learned my addr 172.21.15.5:0/1345133073 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:44:08.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:08.851+0000 7fcbfa800640 1 --2- 172.21.15.5:0/1345133073 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcbfc109120 0x7fcbfc19eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:44:08.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:08.852+0000 7fcbf9400640 1 --2- 172.21.15.5:0/1345133073 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcbfc1049a0 0x7fcbfc19ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:44:08.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:08.852+0000 7fcbf9e00640 1 -- 172.21.15.5:0/1345133073 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcbfc109120 msgr2=0x7fcbfc19eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:44:08.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:08.852+0000 7fcbf9e00640 1 --2- 172.21.15.5:0/1345133073 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcbfc109120 0x7fcbfc19eeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:08.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:08.852+0000 7fcbf9e00640 1 -- 172.21.15.5:0/1345133073 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcbfc1049a0 msgr2=0x7fcbfc19ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:44:08.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:08.852+0000 7fcbf9e00640 1 --2- 172.21.15.5:0/1345133073 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcbfc1049a0 0x7fcbfc19ab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:08.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:08.852+0000 7fcbf9e00640 1 -- 172.21.15.5:0/1345133073 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcbfc19f590 con 0x7fcbfc103ff0 2024-09-10T09:44:08.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:08.852+0000 7fcbf9e00640 1 --2- 172.21.15.5:0/1345133073 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcbfc103ff0 0x7fcbfc19a5e0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fcbec006010 tx=0x7fcbec004040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:44:08.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:08.853+0000 7fcbf2a00640 1 -- 172.21.15.5:0/1345133073 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbec03f030 con 0x7fcbfc103ff0 2024-09-10T09:44:08.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:08.853+0000 7fcbf2a00640 1 -- 172.21.15.5:0/1345133073 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcbec004440 con 0x7fcbfc103ff0 2024-09-10T09:44:08.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:08.853+0000 7fcc00c00640 1 -- 172.21.15.5:0/1345133073 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcbfc19f820 con 0x7fcbfc103ff0 2024-09-10T09:44:08.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:08.853+0000 7fcbf2a00640 1 -- 172.21.15.5:0/1345133073 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbec035070 con 0x7fcbfc103ff0 2024-09-10T09:44:08.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:08.853+0000 7fcc00c00640 1 -- 172.21.15.5:0/1345133073 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcbfc1a7160 con 0x7fcbfc103ff0 2024-09-10T09:44:08.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:08.856+0000 7fcbf2a00640 1 -- 172.21.15.5:0/1345133073 <== mon.2 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7fcbec042030 con 0x7fcbfc103ff0 2024-09-10T09:44:08.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:08.856+0000 7fcc03831740 1 Processor -- start 2024-09-10T09:44:08.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:08.856+0000 7fcc03831740 1 -- start start 2024-09-10T09:44:08.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:08.857+0000 7fcc03831740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b371252b90 0x55b371252f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:44:08.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:08.857+0000 7fcc03831740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b3712534b0 0x55b37125d830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:44:08.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:08.857+0000 7fcc03831740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b37125dd70 0x55b371260160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:44:08.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:08.857+0000 7fcc03831740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55b3711b93e0 con 0x55b37125dd70 2024-09-10T09:44:08.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:08.857+0000 7fcc03831740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55b371160d80 con 0x55b3712534b0 2024-09-10T09:44:08.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:08.857+0000 7fcc03831740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55b3711b53a0 con 0x55b371252b90 2024-09-10T09:44:08.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:08.857+0000 7fcbf9e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b3712534b0 0x55b37125d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:44:08.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:08.857+0000 7fcbf9400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b37125dd70 0x55b371260160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:44:08.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:08.857+0000 7fcbf9e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b3712534b0 0x55b37125d830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:53272/0 (socket says 172.21.15.5:53272) 2024-09-10T09:44:08.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:08.857+0000 7fcbf9e00640 1 -- 172.21.15.5:0/3066210230 learned_addr learned my addr 172.21.15.5:0/3066210230 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:44:08.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:08.857+0000 7fcbfa800640 1 --2- 172.21.15.5:0/3066210230 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b371252b90 0x55b371252f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:44:08.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:08.858+0000 7fcbf9400640 1 -- 172.21.15.5:0/3066210230 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b371252b90 msgr2=0x55b371252f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:44:08.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:08.858+0000 7fcbf9400640 1 --2- 172.21.15.5:0/3066210230 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b371252b90 0x55b371252f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:08.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:08.858+0000 7fcbf9400640 1 -- 172.21.15.5:0/3066210230 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b3712534b0 msgr2=0x55b37125d830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:44:08.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:08.858+0000 7fcbf9400640 1 --2- 172.21.15.5:0/3066210230 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b3712534b0 0x55b37125d830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:08.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:08.858+0000 7fcbf9400640 1 -- 172.21.15.5:0/3066210230 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b37117fae0 con 0x55b37125dd70 2024-09-10T09:44:08.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:08.858+0000 7fcbf9e00640 1 --2- 172.21.15.5:0/3066210230 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b3712534b0 0x55b37125d830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:44:08.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:08.858+0000 7fcbf2a00640 1 --2- 172.21.15.5:0/1345133073 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fcbe00a2c00 0x7fcbe00a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:44:08.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:08.858+0000 7fcbf9400640 1 --2- 172.21.15.5:0/3066210230 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b37125dd70 0x55b371260160 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fcbe8009bf0 tx=0x7fcbe8011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37b7294955134832 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:44:08.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:08.858+0000 7fcbf2a00640 1 -- 172.21.15.5:0/1345133073 <== mon.2 v2:172.21.15.102:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fcbec0ef7c0 con 0x7fcbfc103ff0 2024-09-10T09:44:08.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:08.858+0000 7fcbfa800640 1 --2- 172.21.15.5:0/1345133073 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fcbe00a2c00 0x7fcbe00a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:44:08.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:08.859+0000 7fcbe7000640 1 -- 172.21.15.5:0/3066210230 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbe8004050 con 0x55b37125dd70 2024-09-10T09:44:08.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:08.859+0000 7fcbe7000640 1 -- 172.21.15.5:0/3066210230 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcbe8005500 con 0x55b37125dd70 2024-09-10T09:44:08.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:08.859+0000 7fcbe7000640 1 -- 172.21.15.5:0/3066210230 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbe80035f0 con 0x55b37125dd70 2024-09-10T09:44:08.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:08.859+0000 7fcbfa800640 1 --2- 172.21.15.5:0/1345133073 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fcbe00a2c00 0x7fcbe00a50c0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fcbf4005fd0 tx=0x7fcbf4009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:44:08.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:08.859+0000 7fcc03831740 1 -- 172.21.15.5:0/3066210230 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b37125dd70 msgr2=0x55b371260160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:44:08.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:08.859+0000 7fcc03831740 1 --2- 172.21.15.5:0/3066210230 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b37125dd70 0x55b371260160 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fcbe8009bf0 tx=0x7fcbe8011810 comp rx=0 tx=0).stop 2024-09-10T09:44:08.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:08.860+0000 7fcc03831740 1 -- 172.21.15.5:0/3066210230 shutdown_connections 2024-09-10T09:44:08.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:08.860+0000 7fcc03831740 1 --2- 172.21.15.5:0/3066210230 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b37125dd70 0x55b371260160 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:08.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:08.860+0000 7fcc03831740 1 --2- 172.21.15.5:0/3066210230 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b3712534b0 0x55b37125d830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:08.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:08.860+0000 7fcc03831740 1 --2- 172.21.15.5:0/3066210230 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b371252b90 0x55b371252f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:08.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:08.860+0000 7fcc03831740 1 -- 172.21.15.5:0/3066210230 >> 172.21.15.5:0/3066210230 conn(0x55b37124f120 msgr2=0x55b37124f500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:44:08.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:08.860+0000 7fcc03831740 1 -- 172.21.15.5:0/3066210230 shutdown_connections 2024-09-10T09:44:08.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:08.860+0000 7fcc03831740 1 -- 172.21.15.5:0/3066210230 wait complete. 2024-09-10T09:44:08.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:08.860+0000 7fcc03831740 1 -- start start 2024-09-10T09:44:08.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:08.861+0000 7fcc03831740 1 Processor -- start 2024-09-10T09:44:08.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:08.861+0000 7fcc03831740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b371252b90 0x55b37125ac50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:44:08.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:08.861+0000 7fcc03831740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b3712534b0 0x55b37125b190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:44:08.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:08.861+0000 7fcc03831740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b37125dd70 0x55b37125b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:44:08.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:08.861+0000 7fcc03831740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55b3712609e0 con 0x55b37125dd70 2024-09-10T09:44:08.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:08.861+0000 7fcc03831740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55b3711740f0 con 0x55b371252b90 2024-09-10T09:44:08.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:08.861+0000 7fcc03831740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55b371260b60 con 0x55b3712534b0 2024-09-10T09:44:08.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:08.861+0000 7fcc03831740 10 client.0 _ll_register_callbacks cb 0x55b3711659e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:44:08.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:08.861+0000 7fcbf9e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b371252b90 0x55b37125ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:44:08.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:08.861+0000 7fcbf9400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b3712534b0 0x55b37125b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:44:08.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:08.861+0000 7fcbf9e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b371252b90 0x55b37125ac50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:53280/0 (socket says 172.21.15.5:53280) 2024-09-10T09:44:08.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:08.861+0000 7fcbf9e00640 1 -- 172.21.15.5:0/3055390577 learned_addr learned my addr 172.21.15.5:0/3055390577 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:44:08.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:08.861+0000 7fcbfa800640 1 --2- 172.21.15.5:0/3055390577 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b37125dd70 0x55b37125b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:44:08.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:08.862+0000 7fcbf9e00640 1 -- 172.21.15.5:0/3055390577 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b3712534b0 msgr2=0x55b37125b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:44:08.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:08.862+0000 7fcbf9e00640 1 --2- 172.21.15.5:0/3055390577 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b3712534b0 0x55b37125b190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:08.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:08.862+0000 7fcbf9e00640 1 -- 172.21.15.5:0/3055390577 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b37125dd70 msgr2=0x55b37125b6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:44:08.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:08.862+0000 7fcbf9e00640 1 --2- 172.21.15.5:0/3055390577 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b37125dd70 0x55b37125b6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:08.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:08.862+0000 7fcbf9e00640 1 -- 172.21.15.5:0/3055390577 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b370f2a3c0 con 0x55b371252b90 2024-09-10T09:44:08.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:08.862+0000 7fcbf9e00640 1 --2- 172.21.15.5:0/3055390577 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b371252b90 0x55b37125ac50 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fcbec0f52d0 tx=0x7fcbec03b4d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6e8a98115fe2a331 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:44:08.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:08.862+0000 7fcbdb400640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-09-10T09:44:08.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:08.862+0000 7fcbdb400640 1 -- 172.21.15.5:0/3055390577 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbec049070 con 0x55b371252b90 2024-09-10T09:44:08.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:08.862+0000 7fcbdb400640 1 -- 172.21.15.5:0/3055390577 <== mon.2 v2:172.21.15.102:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fcbec0b82f0 con 0x55b371252b90 2024-09-10T09:44:08.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:08.862+0000 7fcc03831740 1 -- 172.21.15.5:0/3055390577 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcbe8005500 con 0x55b371252b90 2024-09-10T09:44:08.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:08.862+0000 7fcbdb400640 1 -- 172.21.15.5:0/3055390577 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fcbec0bba50 con 0x55b371252b90 2024-09-10T09:44:08.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:08.862+0000 7fcc03831740 10 client.26483 fetch_fsmap learned FSMap version 34 2024-09-10T09:44:08.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:08.862+0000 7fcc03831740 1 -- 172.21.15.5:0/3055390577 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55b37126c690 con 0x55b371252b90 2024-09-10T09:44:08.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:08.863+0000 7fcbdb400640 1 -- 172.21.15.5:0/3055390577 <== mon.2 v2:172.21.15.102:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcbe8005500 con 0x55b371252b90 2024-09-10T09:44:08.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:08.863+0000 7fcbdb400640 1 -- 172.21.15.5:0/3055390577 <== mon.2 v2:172.21.15.102:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7fcbec0bbee0 con 0x55b371252b90 2024-09-10T09:44:08.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:08.863+0000 7fcc03831740 10 client.26483 fetch_fsmap finished waiting for FSMap version 34 2024-09-10T09:44:08.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:08.863+0000 7fcc03831740 10 client.26483 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T09:44:08.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:08.863+0000 7fcc03831740 20 client.26483 populate_metadata read hostname 'smithi005' 2024-09-10T09:44:08.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:08.863+0000 7fcc03831740 1 --2- 172.21.15.5:0/3055390577 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55b37126da80 0x55b37126fef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:44:08.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:08.863+0000 7fcc03831740 4 client.26483 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:44:08.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:08.863+0000 7fcc03831740 1 -- 172.21.15.5:0/3055390577 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b3711c6950 con 0x55b37126da80 2024-09-10T09:44:08.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:08.863+0000 7fcbf9400640 1 --2- 172.21.15.5:0/3055390577 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55b37126da80 0x55b37126fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:44:08.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:08.864+0000 7fcbf9400640 1 --2- 172.21.15.5:0/3055390577 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55b37126da80 0x55b37126fef0 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2a33c00c2e3c560e server_cookie=db1eceb09aafc36b in_seq=0 out_seq=0 2024-09-10T09:44:08.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:08.864+0000 7fcbdb400640 10 client.26483 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T09:44:08.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:08.865+0000 7fcbdb400640 1 -- 172.21.15.5:0/3055390577 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b3711c6950 con 0x55b37126da80 2024-09-10T09:44:08.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:08.865+0000 7fcbdb400640 10 client.26483 handle_command_reply: tid=0 2024-09-10T09:44:08.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:08.865+0000 7fcc03831740 1 client.26483 shutdown 2024-09-10T09:44:08.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:08.865+0000 7fcc03831740 1 -- 172.21.15.5:0/3055390577 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b371252b90 msgr2=0x55b37125ac50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:44:08.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:08.865+0000 7fcc03831740 1 --2- 172.21.15.5:0/3055390577 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b371252b90 0x55b37125ac50 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fcbec0f52d0 tx=0x7fcbec03b4d0 comp rx=0 tx=0).stop 2024-09-10T09:44:08.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:08.865+0000 7fcc03831740 1 -- 172.21.15.5:0/3055390577 shutdown_connections 2024-09-10T09:44:08.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:08.865+0000 7fcc03831740 1 --2- 172.21.15.5:0/3055390577 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55b37126da80 0x55b37126fef0 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:08.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:08.865+0000 7fcc03831740 1 --2- 172.21.15.5:0/3055390577 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b37125dd70 0x55b37125b6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:08.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:08.865+0000 7fcc03831740 1 --2- 172.21.15.5:0/3055390577 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b3712534b0 0x55b37125b190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:08.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:08.865+0000 7fcc03831740 1 --2- 172.21.15.5:0/3055390577 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b371252b90 0x55b37125ac50 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:08.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:08.865+0000 7fcbdb400640 0 client.26483 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T09:44:08.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:08.865+0000 7fcc03831740 1 -- 172.21.15.5:0/3055390577 >> 172.21.15.5:0/3055390577 conn(0x55b37124f120 msgr2=0x55b371254e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:44:08.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:08.865+0000 7fcc03831740 1 -- 172.21.15.5:0/3055390577 shutdown_connections 2024-09-10T09:44:08.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:08.866+0000 7fcc03831740 1 -- 172.21.15.5:0/3055390577 wait complete. 2024-09-10T09:44:08.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:08.866+0000 7fcc03831740 20 client.26483 trim_cache size 0 max 16384 2024-09-10T09:44:08.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:08.884+0000 7fcc03831740 1 Processor -- start 2024-09-10T09:44:08.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:08.884+0000 7fcc03831740 1 -- start start 2024-09-10T09:44:08.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:08.884+0000 7fcc03831740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b371252b90 0x55b3712757c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:44:08.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:08.884+0000 7fcc03831740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b3712534b0 0x55b371275d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:44:08.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:08.884+0000 7fcc03831740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b37125dd70 0x55b371257eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:44:08.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:08.884+0000 7fcc03831740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fcbfc1b0ff0 con 0x55b3712534b0 2024-09-10T09:44:08.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:08.884+0000 7fcc03831740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fcbfc1b2b30 con 0x55b37125dd70 2024-09-10T09:44:08.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:08.884+0000 7fcc03831740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fcbfc1b16c0 con 0x55b371252b90 2024-09-10T09:44:08.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:08.884+0000 7fcbf9400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b3712534b0 0x55b371275d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:44:08.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:08.884+0000 7fcbf9e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b371252b90 0x55b3712757c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:44:08.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:08.884+0000 7fcbf9400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b3712534b0 0x55b371275d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44060/0 (socket says 172.21.15.5:44060) 2024-09-10T09:44:08.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:08.885+0000 7fcbf9400640 1 -- 172.21.15.5:0/1690668712 learned_addr learned my addr 172.21.15.5:0/1690668712 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:44:08.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:08.885+0000 7fcbfa800640 1 --2- 172.21.15.5:0/1690668712 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b37125dd70 0x55b371257eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:44:08.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:08.885+0000 7fcbf9400640 1 -- 172.21.15.5:0/1690668712 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b371252b90 msgr2=0x55b3712757c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:44:08.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:08.885+0000 7fcbf9400640 1 --2- 172.21.15.5:0/1690668712 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b371252b90 0x55b3712757c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:08.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:08.885+0000 7fcbf9400640 1 -- 172.21.15.5:0/1690668712 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b37125dd70 msgr2=0x55b371257eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:44:08.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:08.885+0000 7fcbf9400640 1 --2- 172.21.15.5:0/1690668712 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b37125dd70 0x55b371257eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:08.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:08.885+0000 7fcbf9400640 1 -- 172.21.15.5:0/1690668712 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b371270570 con 0x55b3712534b0 2024-09-10T09:44:08.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:08.885+0000 7fcbfa800640 1 --2- 172.21.15.5:0/1690668712 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b37125dd70 0x55b371257eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:44:08.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:08.885+0000 7fcbf9e00640 1 --2- 172.21.15.5:0/1690668712 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b371252b90 0x55b3712757c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:44:08.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:08.885+0000 7fcbf9400640 1 --2- 172.21.15.5:0/1690668712 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b3712534b0 0x55b371275d00 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fcbe8009bf0 tx=0x7fcbe8004b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7604aca481e0753 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:44:08.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:08.886+0000 7fcbe6600640 1 -- 172.21.15.5:0/1690668712 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbe8033070 con 0x55b3712534b0 2024-09-10T09:44:08.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:08.886+0000 7fcbe6600640 1 -- 172.21.15.5:0/1690668712 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcbe8004250 con 0x55b3712534b0 2024-09-10T09:44:08.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:08.886+0000 7fcbe6600640 1 -- 172.21.15.5:0/1690668712 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbe802e3e0 con 0x55b3712534b0 2024-09-10T09:44:08.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:08.887+0000 7fcc03831740 1 -- 172.21.15.5:0/1690668712 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b3712534b0 msgr2=0x55b371275d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:44:08.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:08.887+0000 7fcc03831740 1 --2- 172.21.15.5:0/1690668712 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b3712534b0 0x55b371275d00 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fcbe8009bf0 tx=0x7fcbe8004b20 comp rx=0 tx=0).stop 2024-09-10T09:44:08.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:08.887+0000 7fcc03831740 1 -- 172.21.15.5:0/1690668712 shutdown_connections 2024-09-10T09:44:08.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:08.887+0000 7fcc03831740 1 --2- 172.21.15.5:0/1690668712 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b37125dd70 0x55b371257eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:08.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:08.887+0000 7fcc03831740 1 --2- 172.21.15.5:0/1690668712 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b3712534b0 0x55b371275d00 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:08.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:08.887+0000 7fcc03831740 1 --2- 172.21.15.5:0/1690668712 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b371252b90 0x55b3712757c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:08.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:08.887+0000 7fcc03831740 1 -- 172.21.15.5:0/1690668712 >> 172.21.15.5:0/1690668712 conn(0x55b37124f120 msgr2=0x55b371254e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:44:08.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:08.887+0000 7fcc03831740 1 -- 172.21.15.5:0/1690668712 shutdown_connections 2024-09-10T09:44:08.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:08.887+0000 7fcc03831740 1 -- 172.21.15.5:0/1690668712 wait complete. 2024-09-10T09:44:08.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:08.887+0000 7fcc03831740 1 -- start start 2024-09-10T09:44:08.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:08.887+0000 7fcc03831740 1 Processor -- start 2024-09-10T09:44:08.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:08.888+0000 7fcc03831740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b371252b90 0x55b371261ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:44:08.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:08.888+0000 7fcc03831740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b3712534b0 0x55b3712621e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:44:08.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:08.888+0000 7fcc03831740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b37125dd70 0x55b371262720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:44:08.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:08.888+0000 7fcc03831740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fcbfc1b2460 con 0x55b3712534b0 2024-09-10T09:44:08.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:08.888+0000 7fcc03831740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fcbfc1b1d90 con 0x55b371252b90 2024-09-10T09:44:08.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:08.888+0000 7fcc03831740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fcbfc1aefd0 con 0x55b37125dd70 2024-09-10T09:44:08.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:08.888+0000 7fcc03831740 10 client.0 _ll_register_callbacks cb 0x7fcbec005890 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:44:08.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:08.888+0000 7fcbf9400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b3712534b0 0x55b3712621e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:44:08.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:08.888+0000 7fcbfa800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b37125dd70 0x55b371262720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:44:08.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:08.888+0000 7fcbf9e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b371252b90 0x55b371261ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:44:08.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:08.888+0000 7fcbfa800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b37125dd70 0x55b371262720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:42552/0 (socket says 172.21.15.5:42552) 2024-09-10T09:44:08.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:08.888+0000 7fcbfa800640 1 -- 172.21.15.5:0/3412364129 learned_addr learned my addr 172.21.15.5:0/3412364129 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:44:08.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:08.888+0000 7fcbf9e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b371252b90 0x55b371261ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:53302/0 (socket says 172.21.15.5:53302) 2024-09-10T09:44:08.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:08.889+0000 7fcbfa800640 1 -- 172.21.15.5:0/3412364129 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b371252b90 msgr2=0x55b371261ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:44:08.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:08.889+0000 7fcbfa800640 1 --2- 172.21.15.5:0/3412364129 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b371252b90 0x55b371261ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:08.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:08.889+0000 7fcbfa800640 1 -- 172.21.15.5:0/3412364129 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b3712534b0 msgr2=0x55b3712621e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:44:08.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:08.889+0000 7fcbfa800640 1 --2- 172.21.15.5:0/3412364129 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b3712534b0 0x55b3712621e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:08.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:08.889+0000 7fcbfa800640 1 -- 172.21.15.5:0/3412364129 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b3712d7930 con 0x55b37125dd70 2024-09-10T09:44:08.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:08.889+0000 7fcbf9e00640 1 --2- 172.21.15.5:0/3412364129 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b371252b90 0x55b371261ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:44:08.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:08.889+0000 7fcbf9400640 1 --2- 172.21.15.5:0/3412364129 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b3712534b0 0x55b3712621e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:44:08.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:08.889+0000 7fcbfa800640 1 --2- 172.21.15.5:0/3412364129 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b37125dd70 0x55b371262720 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fcbf40042b0 tx=0x7fcbf4004ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d463fced8f580f80 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:44:08.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:08.889+0000 7fcbdb400640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-09-10T09:44:08.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:08.889+0000 7fcbdb400640 1 -- 172.21.15.5:0/3412364129 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbf4004b20 con 0x55b37125dd70 2024-09-10T09:44:08.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:08.890+0000 7fcc03831740 1 -- 172.21.15.5:0/3412364129 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcbe8004250 con 0x55b37125dd70 2024-09-10T09:44:08.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:08.890+0000 7fcbdb400640 1 -- 172.21.15.5:0/3412364129 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fcbf4004cc0 con 0x55b37125dd70 2024-09-10T09:44:08.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:08.890+0000 7fcbdb400640 1 -- 172.21.15.5:0/3412364129 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fcbf4034050 con 0x55b37125dd70 2024-09-10T09:44:08.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:08.890+0000 7fcc03831740 10 client.26518 fetch_fsmap learned FSMap version 34 2024-09-10T09:44:08.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:08.890+0000 7fcc03831740 1 -- 172.21.15.5:0/3412364129 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55b3712d8cd0 con 0x55b37125dd70 2024-09-10T09:44:08.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:08.890+0000 7fcbdb400640 1 -- 172.21.15.5:0/3412364129 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcbe8004250 con 0x55b37125dd70 2024-09-10T09:44:08.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:08.890+0000 7fcbdb400640 1 -- 172.21.15.5:0/3412364129 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7fcbf4028030 con 0x55b37125dd70 2024-09-10T09:44:08.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:08.890+0000 7fcc03831740 10 client.26518 fetch_fsmap finished waiting for FSMap version 34 2024-09-10T09:44:08.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:08.890+0000 7fcc03831740 10 client.26518 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T09:44:08.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:08.890+0000 7fcc03831740 20 client.26518 populate_metadata read hostname 'smithi005' 2024-09-10T09:44:08.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:08.890+0000 7fcc03831740 1 --2- 172.21.15.5:0/3412364129 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55b3712d9450 0x55b3712db870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:44:08.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:08.891+0000 7fcc03831740 4 client.26518 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T09:44:08.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:08.891+0000 7fcc03831740 1 -- 172.21.15.5:0/3412364129 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55b3711c0e60 con 0x55b3712d9450 2024-09-10T09:44:08.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:08.891+0000 7fcbf9e00640 1 --2- 172.21.15.5:0/3412364129 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55b3712d9450 0x55b3712db870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:44:08.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:08.891+0000 7fcbf9e00640 1 --2- 172.21.15.5:0/3412364129 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55b3712d9450 0x55b3712db870 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=be449b519b52ac10 server_cookie=19ab093f712c86d6 in_seq=0 out_seq=0 2024-09-10T09:44:08.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:08.891+0000 7fcbdb400640 10 client.26518 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T09:44:08.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:08.891+0000 7fcbdb400640 1 -- 172.21.15.5:0/3412364129 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55b3711c0e60 con 0x55b3712d9450 2024-09-10T09:44:08.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:08.891+0000 7fcbdb400640 10 client.26518 handle_command_reply: tid=0 2024-09-10T09:44:08.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:08.891+0000 7fcc03831740 1 client.26518 shutdown 2024-09-10T09:44:08.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:08.892+0000 7fcc03831740 1 -- 172.21.15.5:0/3412364129 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b37125dd70 msgr2=0x55b371262720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:44:08.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:08.892+0000 7fcc03831740 1 --2- 172.21.15.5:0/3412364129 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b37125dd70 0x55b371262720 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fcbf40042b0 tx=0x7fcbf4004ab0 comp rx=0 tx=0).stop 2024-09-10T09:44:08.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:08.892+0000 7fcc03831740 1 -- 172.21.15.5:0/3412364129 shutdown_connections 2024-09-10T09:44:08.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:08.892+0000 7fcc03831740 1 --2- 172.21.15.5:0/3412364129 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55b3712d9450 0x55b3712db870 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:08.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:08.892+0000 7fcc03831740 1 --2- 172.21.15.5:0/3412364129 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b37125dd70 0x55b371262720 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:08.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:08.892+0000 7fcc03831740 1 --2- 172.21.15.5:0/3412364129 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b3712534b0 0x55b3712621e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:08.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:08.892+0000 7fcc03831740 1 --2- 172.21.15.5:0/3412364129 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b371252b90 0x55b371261ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:08.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:08.892+0000 7fcbdb400640 0 client.26518 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T09:44:08.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:08.892+0000 7fcc03831740 1 -- 172.21.15.5:0/3412364129 >> 172.21.15.5:0/3412364129 conn(0x55b37124f120 msgr2=0x55b37125abf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:44:08.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:08.892+0000 7fcc03831740 1 -- 172.21.15.5:0/3412364129 shutdown_connections 2024-09-10T09:44:08.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:08.892+0000 7fcc03831740 1 -- 172.21.15.5:0/3412364129 wait complete. 2024-09-10T09:44:08.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:08.892+0000 7fcc03831740 20 client.26518 trim_cache size 0 max 16384 2024-09-10T09:44:08.907 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-10T09:44:08.907 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "no active scrubs running", 2024-09-10T09:44:08.907 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": {} 2024-09-10T09:44:08.907 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-10T09:44:08.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:08.893+0000 7fcc00c00640 1 -- 172.21.15.5:0/1345133073 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fcbe00a2c00 msgr2=0x7fcbe00a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:44:08.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:08.893+0000 7fcc00c00640 1 --2- 172.21.15.5:0/1345133073 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fcbe00a2c00 0x7fcbe00a50c0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fcbf4005fd0 tx=0x7fcbf4009500 comp rx=0 tx=0).stop 2024-09-10T09:44:08.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:08.893+0000 7fcc00c00640 1 -- 172.21.15.5:0/1345133073 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcbfc103ff0 msgr2=0x7fcbfc19a5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:44:08.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:08.893+0000 7fcc00c00640 1 --2- 172.21.15.5:0/1345133073 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcbfc103ff0 0x7fcbfc19a5e0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fcbec006010 tx=0x7fcbec004040 comp rx=0 tx=0).stop 2024-09-10T09:44:08.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:08.893+0000 7fcc00c00640 1 -- 172.21.15.5:0/1345133073 shutdown_connections 2024-09-10T09:44:08.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:08.893+0000 7fcc00c00640 1 --2- 172.21.15.5:0/1345133073 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fcbe00a2c00 0x7fcbe00a50c0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:08.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:08.893+0000 7fcc00c00640 1 --2- 172.21.15.5:0/1345133073 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcbfc109120 0x7fcbfc19eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:08.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:08.893+0000 7fcc00c00640 1 --2- 172.21.15.5:0/1345133073 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcbfc1049a0 0x7fcbfc19ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:08.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:08.893+0000 7fcc00c00640 1 --2- 172.21.15.5:0/1345133073 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcbfc103ff0 0x7fcbfc19a5e0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:08.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:08.893+0000 7fcc00c00640 1 -- 172.21.15.5:0/1345133073 >> 172.21.15.5:0/1345133073 conn(0x7fcbfc0ffa90 msgr2=0x7fcbfc100e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:44:08.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:08.893+0000 7fcc00c00640 1 -- 172.21.15.5:0/1345133073 shutdown_connections 2024-09-10T09:44:08.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:08.893+0000 7fcc00c00640 1 -- 172.21.15.5:0/1345133073 wait complete. 2024-09-10T09:44:09.043 INFO:tasks.workunit.client.0.smithi005.stdout: 1 316092 6.08 MB/sec execute 110 sec latency 21.116 ms 2024-09-10T09:44:09.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:44:08 smithi159 ceph-mon[35513]: pgmap v567: 129 pgs: 129 active+clean; 136 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.4 MiB/s wr, 180 op/s 2024-09-10T09:44:09.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:44:08 smithi102 ceph-mon[35442]: pgmap v567: 129 pgs: 129 active+clean; 136 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.4 MiB/s wr, 180 op/s 2024-09-10T09:44:09.791 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-10T09:44:09.792 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T09:44:10.042 INFO:tasks.workunit.client.0.smithi005.stdout: 1 316704 6.04 MB/sec execute 111 sec latency 369.660 ms 2024-09-10T09:44:10.236 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:44:11.042 INFO:tasks.workunit.client.0.smithi005.stdout: 1 316704 5.98 MB/sec execute 112 sec latency 1195.408 ms 2024-09-10T09:44:11.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:44:10 smithi005 ceph-mon[30350]: pgmap v568: 129 pgs: 129 active+clean; 135 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.5 MiB/s wr, 195 op/s 2024-09-10T09:44:11.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:44:10 smithi159 ceph-mon[35513]: pgmap v568: 129 pgs: 129 active+clean; 135 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.5 MiB/s wr, 195 op/s 2024-09-10T09:44:11.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:44:10 smithi102 ceph-mon[35442]: pgmap v568: 129 pgs: 129 active+clean; 135 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.5 MiB/s wr, 195 op/s 2024-09-10T09:44:12.043 INFO:tasks.workunit.client.0.smithi005.stdout: 1 316704 5.93 MB/sec execute 113 sec latency 2195.450 ms 2024-09-10T09:44:12.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:12.529+0000 7f1381800640 1 -- 172.21.15.5:0/540791164 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f137c074620 msgr2=0x7f137c074a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:44:12.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:12.529+0000 7f1381800640 1 --2- 172.21.15.5:0/540791164 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f137c074620 0x7f137c074a00 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f136c009900 tx=0x7f136c02a480 comp rx=0 tx=0).stop 2024-09-10T09:44:12.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:12.529+0000 7f1381800640 1 -- 172.21.15.5:0/540791164 shutdown_connections 2024-09-10T09:44:12.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:12.529+0000 7f1381800640 1 --2- 172.21.15.5:0/540791164 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f137c10e210 0x7f137c110660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:12.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:12.529+0000 7f1381800640 1 --2- 172.21.15.5:0/540791164 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f137c074f40 0x7f137c10dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:12.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:12.529+0000 7f1381800640 1 --2- 172.21.15.5:0/540791164 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f137c074620 0x7f137c074a00 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:12.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:12.529+0000 7f1381800640 1 -- 172.21.15.5:0/540791164 >> 172.21.15.5:0/540791164 conn(0x7f137c0fd710 msgr2=0x7f137c0ffb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:44:12.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:12.530+0000 7f1381800640 1 -- 172.21.15.5:0/540791164 shutdown_connections 2024-09-10T09:44:12.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:12.530+0000 7f1381800640 1 -- 172.21.15.5:0/540791164 wait complete. 2024-09-10T09:44:12.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:12.531+0000 7f1381800640 1 Processor -- start 2024-09-10T09:44:12.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:12.531+0000 7f1381800640 1 -- start start 2024-09-10T09:44:12.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:12.532+0000 7f1381800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f137c074620 0x7f137c19a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:44:12.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:12.532+0000 7f1381800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f137c074f40 0x7f137c19aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:44:12.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:12.532+0000 7f1381800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f137c10e210 0x7f137c19ee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:44:12.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:12.532+0000 7f1381800640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f137c113fc0 con 0x7f137c074620 2024-09-10T09:44:12.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:12.532+0000 7f1381800640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f137c113e40 con 0x7f137c10e210 2024-09-10T09:44:12.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:12.532+0000 7f1381800640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f137c114140 con 0x7f137c074f40 2024-09-10T09:44:12.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:12.532+0000 7f137be00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f137c074620 0x7f137c19a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:44:12.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:12.532+0000 7f1373e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f137c074f40 0x7f137c19aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:44:12.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:12.532+0000 7f137be00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f137c074620 0x7f137c19a540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37610/0 (socket says 172.21.15.5:37610) 2024-09-10T09:44:12.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:12.532+0000 7f137be00640 1 -- 172.21.15.5:0/568198236 learned_addr learned my addr 172.21.15.5:0/568198236 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:44:12.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:12.532+0000 7f1380e00640 1 --2- 172.21.15.5:0/568198236 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f137c10e210 0x7f137c19ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:44:12.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:12.533+0000 7f1380e00640 1 -- 172.21.15.5:0/568198236 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f137c074f40 msgr2=0x7f137c19aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:44:12.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:12.533+0000 7f1380e00640 1 --2- 172.21.15.5:0/568198236 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f137c074f40 0x7f137c19aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:12.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:12.533+0000 7f1380e00640 1 -- 172.21.15.5:0/568198236 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f137c074620 msgr2=0x7f137c19a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:44:12.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:12.533+0000 7f1380e00640 1 --2- 172.21.15.5:0/568198236 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f137c074620 0x7f137c19a540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:12.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:12.533+0000 7f1380e00640 1 -- 172.21.15.5:0/568198236 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f137c19f4f0 con 0x7f137c10e210 2024-09-10T09:44:12.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:12.533+0000 7f1373e00640 1 --2- 172.21.15.5:0/568198236 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f137c074f40 0x7f137c19aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:44:12.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:12.533+0000 7f137be00640 1 --2- 172.21.15.5:0/568198236 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f137c074620 0x7f137c19a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:44:12.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:12.533+0000 7f1380e00640 1 --2- 172.21.15.5:0/568198236 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f137c10e210 0x7f137c19ee10 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f1374009fd0 tx=0x7f137400ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:44:12.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:12.533+0000 7f1379600640 1 -- 172.21.15.5:0/568198236 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f137400ce20 con 0x7f137c10e210 2024-09-10T09:44:12.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:12.533+0000 7f1381800640 1 -- 172.21.15.5:0/568198236 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f137c19f7e0 con 0x7f137c10e210 2024-09-10T09:44:12.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:12.533+0000 7f1379600640 1 -- 172.21.15.5:0/568198236 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1374005020 con 0x7f137c10e210 2024-09-10T09:44:12.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:12.533+0000 7f1379600640 1 -- 172.21.15.5:0/568198236 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1374010720 con 0x7f137c10e210 2024-09-10T09:44:12.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:12.534+0000 7f1381800640 1 -- 172.21.15.5:0/568198236 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f137c1a70c0 con 0x7f137c10e210 2024-09-10T09:44:12.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:12.535+0000 7f1381800640 1 -- 172.21.15.5:0/568198236 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f137c074a00 con 0x7f137c10e210 2024-09-10T09:44:12.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:12.538+0000 7f1379600640 1 -- 172.21.15.5:0/568198236 <== mon.2 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f1374005540 con 0x7f137c10e210 2024-09-10T09:44:12.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:12.539+0000 7f1379600640 1 --2- 172.21.15.5:0/568198236 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f13600a2c00 0x7f13600a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:44:12.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:12.539+0000 7f1379600640 1 -- 172.21.15.5:0/568198236 <== mon.2 v2:172.21.15.102:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f13740cfab0 con 0x7f137c10e210 2024-09-10T09:44:12.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:12.539+0000 7f137be00640 1 --2- 172.21.15.5:0/568198236 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f13600a2c00 0x7f13600a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:44:12.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:12.540+0000 7f137be00640 1 --2- 172.21.15.5:0/568198236 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f13600a2c00 0x7f13600a50c0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f136c0096c0 tx=0x7f136c005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:44:12.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:12.542+0000 7f1379600640 1 -- 172.21.15.5:0/568198236 <== mon.2 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f137409c650 con 0x7f137c10e210 2024-09-10T09:44:12.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:12.784+0000 7f1381800640 1 -- 172.21.15.5:0/568198236 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f137c19b8a0 con 0x7f137c10e210 2024-09-10T09:44:12.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:12.784+0000 7f1379600640 1 -- 172.21.15.5:0/568198236 <== mon.2 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+11969 (secure 0 0 0) 0x7f137c19b8a0 con 0x7f137c10e210 2024-09-10T09:44:12.786 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-10T09:44:12.786 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":34,"btime":"2024-09-10T09:42:57:737230+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14454,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/2970594311","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2970594311},{"type":"v1","addr":"172.21.15.5:6835","nonce":2970594311}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14514,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6839/3616502174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":3616502174},{"type":"v1","addr":"172.21.15.5:6839","nonce":3616502174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":34,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T09:35:29.626287+0000","modified":"2024-09-10T09:42:56.961831+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24542,"mds_1":24535,"mds_2":24511,"mds_3":24506,"mds_4":24443},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14520":{"gid":14520,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/2073599842","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":2073599842},{"type":"v1","addr":"172.21.15.5:6841","nonce":2073599842}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"k","rank":4,"incarnation":26,"state":"up:active","state_seq":24,"addr":"172.21.15.102:6833/2457007427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":2457007427},{"type":"v1","addr":"172.21.15.102:6833","nonce":2457007427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/356445159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":356445159},{"type":"v1","addr":"172.21.15.159:6833","nonce":356445159}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6837/2894314507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":2894314507},{"type":"v1","addr":"172.21.15.5:6837","nonce":2894314507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.102:6835/1647549111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":1647549111},{"type":"v1","addr":"172.21.15.102:6835","nonce":1647549111}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6835/952657242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":952657242},{"type":"v1","addr":"172.21.15.159:6835","nonce":952657242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":19,"addr":"172.21.15.102:6837/2689776764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":2689776764},{"type":"v1","addr":"172.21.15.102:6837","nonce":2689776764}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.159:6837/620627639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":620627639},{"type":"v1","addr":"172.21.15.159:6837","nonce":620627639}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24535":{"gid":24535,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":18,"addr":"172.21.15.159:6839/2073739599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":2073739599},{"type":"v1","addr":"172.21.15.159:6839","nonce":2073739599}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24542":{"gid":24542,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.102:6839/335409749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6838","nonce":335409749},{"type":"v1","addr":"172.21.15.102:6839","nonce":335409749}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24542,"qdb_cluster":[24542,24535,24511,24506,24443]},"id":1}]} 2024-09-10T09:44:12.786 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 34 2024-09-10T09:44:12.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:12.787+0000 7f1381800640 1 -- 172.21.15.5:0/568198236 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f13600a2c00 msgr2=0x7f13600a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:44:12.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:12.787+0000 7f1381800640 1 --2- 172.21.15.5:0/568198236 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f13600a2c00 0x7f13600a50c0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f136c0096c0 tx=0x7f136c005a20 comp rx=0 tx=0).stop 2024-09-10T09:44:12.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:12.787+0000 7f1381800640 1 -- 172.21.15.5:0/568198236 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f137c10e210 msgr2=0x7f137c19ee10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:44:12.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:12.787+0000 7f1381800640 1 --2- 172.21.15.5:0/568198236 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f137c10e210 0x7f137c19ee10 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f1374009fd0 tx=0x7f137400ef10 comp rx=0 tx=0).stop 2024-09-10T09:44:12.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:12.787+0000 7f1381800640 1 -- 172.21.15.5:0/568198236 shutdown_connections 2024-09-10T09:44:12.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:12.787+0000 7f1381800640 1 --2- 172.21.15.5:0/568198236 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f13600a2c00 0x7f13600a50c0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:12.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:12.787+0000 7f1381800640 1 --2- 172.21.15.5:0/568198236 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f137c10e210 0x7f137c19ee10 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:12.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:12.787+0000 7f1381800640 1 --2- 172.21.15.5:0/568198236 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f137c074f40 0x7f137c19aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:12.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:12.787+0000 7f1381800640 1 --2- 172.21.15.5:0/568198236 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f137c074620 0x7f137c19a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:12.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:12.787+0000 7f1381800640 1 -- 172.21.15.5:0/568198236 >> 172.21.15.5:0/568198236 conn(0x7f137c0fd710 msgr2=0x7f137c0fe410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:44:12.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:12.787+0000 7f1381800640 1 -- 172.21.15.5:0/568198236 shutdown_connections 2024-09-10T09:44:12.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:12.787+0000 7f1381800640 1 -- 172.21.15.5:0/568198236 wait complete. 2024-09-10T09:44:12.803 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:44:12 smithi005 ceph-mon[30350]: pgmap v569: 129 pgs: 129 active+clean; 138 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.5 MiB/s wr, 188 op/s 2024-09-10T09:44:13.043 INFO:tasks.workunit.client.0.smithi005.stdout: 1 318870 5.99 MB/sec execute 114 sec latency 2531.638 ms 2024-09-10T09:44:13.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:44:12 smithi159 ceph-mon[35513]: pgmap v569: 129 pgs: 129 active+clean; 138 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.5 MiB/s wr, 188 op/s 2024-09-10T09:44:13.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:44:12 smithi102 ceph-mon[35442]: pgmap v569: 129 pgs: 129 active+clean; 138 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.5 MiB/s wr, 188 op/s 2024-09-10T09:44:13.714 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-10T09:44:14.043 INFO:tasks.workunit.client.0.smithi005.stdout: 1 320497 6.00 MB/sec execute 115 sec latency 25.395 ms 2024-09-10T09:44:14.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:44:13 smithi005 ceph-mon[30350]: from='client.? 172.21.15.5:0/568198236' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:44:14.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:44:13 smithi159 ceph-mon[35513]: from='client.? 172.21.15.5:0/568198236' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:44:14.132 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:44:14.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:44:13 smithi102 ceph-mon[35442]: from='client.? 172.21.15.5:0/568198236' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:44:14.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:14.782+0000 7fcdf7200640 1 -- 172.21.15.5:0/3140359855 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcdf81079c0 msgr2=0x7fcdf8107da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:44:14.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:14.782+0000 7fcdf7200640 1 --2- 172.21.15.5:0/3140359855 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcdf81079c0 0x7fcdf8107da0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fcde8009890 tx=0x7fcde802a370 comp rx=0 tx=0).stop 2024-09-10T09:44:14.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:14.782+0000 7fcdf7200640 1 -- 172.21.15.5:0/3140359855 shutdown_connections 2024-09-10T09:44:14.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:14.782+0000 7fcdf7200640 1 --2- 172.21.15.5:0/3140359855 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcdf8102360 0x7fcdf810e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:14.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:14.782+0000 7fcdf7200640 1 --2- 172.21.15.5:0/3140359855 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcdf81019c0 0x7fcdf8101e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:14.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:14.782+0000 7fcdf7200640 1 --2- 172.21.15.5:0/3140359855 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcdf81079c0 0x7fcdf8107da0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:14.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:14.782+0000 7fcdf7200640 1 -- 172.21.15.5:0/3140359855 >> 172.21.15.5:0/3140359855 conn(0x7fcdf80fd740 msgr2=0x7fcdf80ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:44:14.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:14.782+0000 7fcdf7200640 1 -- 172.21.15.5:0/3140359855 shutdown_connections 2024-09-10T09:44:14.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:14.782+0000 7fcdf7200640 1 -- 172.21.15.5:0/3140359855 wait complete. 2024-09-10T09:44:14.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:14.783+0000 7fcdf7200640 1 Processor -- start 2024-09-10T09:44:14.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:14.783+0000 7fcdf7200640 1 -- start start 2024-09-10T09:44:14.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:14.784+0000 7fcdf7200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcdf81019c0 0x7fcdf819a450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:44:14.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:14.784+0000 7fcdf7200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcdf8102360 0x7fcdf819a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:44:14.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:14.784+0000 7fcdf7200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcdf81079c0 0x7fcdf819ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:44:14.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:14.784+0000 7fcdf7200640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fcdf8111d80 con 0x7fcdf8102360 2024-09-10T09:44:14.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:14.784+0000 7fcdf7200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fcdf8111c00 con 0x7fcdf81079c0 2024-09-10T09:44:14.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:14.784+0000 7fcdf7200640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fcdf8111f00 con 0x7fcdf81019c0 2024-09-10T09:44:14.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:14.784+0000 7fcdf5e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcdf81019c0 0x7fcdf819a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:44:14.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:14.784+0000 7fcdf5e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcdf81019c0 0x7fcdf819a450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:39094/0 (socket says 172.21.15.5:39094) 2024-09-10T09:44:14.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:14.784+0000 7fcdf5e00640 1 -- 172.21.15.5:0/2082391245 learned_addr learned my addr 172.21.15.5:0/2082391245 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:44:14.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:14.784+0000 7fcdf6800640 1 --2- 172.21.15.5:0/2082391245 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcdf81079c0 0x7fcdf819ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:44:14.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:14.784+0000 7fcdf5e00640 1 -- 172.21.15.5:0/2082391245 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcdf81079c0 msgr2=0x7fcdf819ed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:44:14.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:14.785+0000 7fcdf5e00640 1 --2- 172.21.15.5:0/2082391245 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcdf81079c0 0x7fcdf819ed20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:14.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:14.785+0000 7fcdf5e00640 1 -- 172.21.15.5:0/2082391245 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcdf8102360 msgr2=0x7fcdf819a990 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T09:44:14.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:14.785+0000 7fcdf5400640 1 --2- 172.21.15.5:0/2082391245 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcdf8102360 0x7fcdf819a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:44:14.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:14.785+0000 7fcdf5e00640 1 --2- 172.21.15.5:0/2082391245 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcdf8102360 0x7fcdf819a990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:14.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:14.785+0000 7fcdf5e00640 1 -- 172.21.15.5:0/2082391245 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcdf819f400 con 0x7fcdf81019c0 2024-09-10T09:44:14.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:14.785+0000 7fcdf5400640 1 --2- 172.21.15.5:0/2082391245 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcdf8102360 0x7fcdf819a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T09:44:14.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:14.785+0000 7fcdf6800640 1 --2- 172.21.15.5:0/2082391245 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcdf81079c0 0x7fcdf819ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:44:14.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:14.785+0000 7fcdf5e00640 1 --2- 172.21.15.5:0/2082391245 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcdf81019c0 0x7fcdf819a450 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fcde8035040 tx=0x7fcde8033b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:44:14.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:14.786+0000 7fcdeea00640 1 -- 172.21.15.5:0/2082391245 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcde8037040 con 0x7fcdf81019c0 2024-09-10T09:44:14.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:14.786+0000 7fcdeea00640 1 -- 172.21.15.5:0/2082391245 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcde8040070 con 0x7fcdf81019c0 2024-09-10T09:44:14.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:14.786+0000 7fcdeea00640 1 -- 172.21.15.5:0/2082391245 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcde8003a10 con 0x7fcdf81019c0 2024-09-10T09:44:14.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:14.786+0000 7fcdf7200640 1 -- 172.21.15.5:0/2082391245 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcdf819f690 con 0x7fcdf81019c0 2024-09-10T09:44:14.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:14.786+0000 7fcdf7200640 1 -- 172.21.15.5:0/2082391245 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcdf81a6fd0 con 0x7fcdf81019c0 2024-09-10T09:44:14.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:14.790+0000 7fcdeea00640 1 -- 172.21.15.5:0/2082391245 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7fcde8003bb0 con 0x7fcdf81019c0 2024-09-10T09:44:14.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:14.790+0000 7fcdff8c5740 1 Processor -- start 2024-09-10T09:44:14.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:14.790+0000 7fcdff8c5740 1 -- start start 2024-09-10T09:44:14.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:14.791+0000 7fcdff8c5740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559fbc3a8d60 0x559fbc3a9140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:44:14.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:14.791+0000 7fcdff8c5740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559fbc3a9680 0x559fbc3bc2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:44:14.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:14.791+0000 7fcdff8c5740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559fbc3bc830 0x559fbc3bec20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:44:14.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:14.791+0000 7fcdff8c5740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x559fbc30f3e0 con 0x559fbc3bc830 2024-09-10T09:44:14.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:14.791+0000 7fcdff8c5740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x559fbc2b6d80 con 0x559fbc3a8d60 2024-09-10T09:44:14.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:14.791+0000 7fcdff8c5740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x559fbc30b3a0 con 0x559fbc3a9680 2024-09-10T09:44:14.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:14.791+0000 7fcdf6800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559fbc3a8d60 0x559fbc3a9140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:44:14.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:14.791+0000 7fcdf5e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559fbc3a9680 0x559fbc3bc2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:44:14.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:14.791+0000 7fcdf6800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559fbc3a8d60 0x559fbc3a9140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:38736/0 (socket says 172.21.15.5:38736) 2024-09-10T09:44:14.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:14.791+0000 7fcdf6800640 1 -- 172.21.15.5:0/501012319 learned_addr learned my addr 172.21.15.5:0/501012319 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:44:14.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:14.791+0000 7fcdf5400640 1 --2- 172.21.15.5:0/501012319 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559fbc3bc830 0x559fbc3bec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:44:14.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:14.792+0000 7fcdf5e00640 1 -- 172.21.15.5:0/501012319 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559fbc3a8d60 msgr2=0x559fbc3a9140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:44:14.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:14.792+0000 7fcdf5e00640 1 --2- 172.21.15.5:0/501012319 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559fbc3a8d60 0x559fbc3a9140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:14.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:14.792+0000 7fcdf5e00640 1 -- 172.21.15.5:0/501012319 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559fbc3bc830 msgr2=0x559fbc3bec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:44:14.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:14.792+0000 7fcdf5e00640 1 --2- 172.21.15.5:0/501012319 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559fbc3bc830 0x559fbc3bec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:14.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:14.792+0000 7fcdf5e00640 1 -- 172.21.15.5:0/501012319 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559fbc2d5ae0 con 0x559fbc3a9680 2024-09-10T09:44:14.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:14.792+0000 7fcdf5400640 1 --2- 172.21.15.5:0/501012319 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559fbc3bc830 0x559fbc3bec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:44:14.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:14.792+0000 7fcdeea00640 1 --2- 172.21.15.5:0/2082391245 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fcddc0a2c00 0x7fcddc0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:44:14.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:14.792+0000 7fcdf5e00640 1 --2- 172.21.15.5:0/501012319 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559fbc3a9680 0x559fbc3bc2f0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fcde80ec840 tx=0x7fcde80f7320 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fc64284f35982a82 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:44:14.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:14.792+0000 7fcdeea00640 1 -- 172.21.15.5:0/2082391245 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fcde80ef160 con 0x7fcdf81019c0 2024-09-10T09:44:14.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:14.793+0000 7fcde3400640 1 -- 172.21.15.5:0/501012319 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcde80b7870 con 0x559fbc3a9680 2024-09-10T09:44:14.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:14.793+0000 7fcdf6800640 1 --2- 172.21.15.5:0/2082391245 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fcddc0a2c00 0x7fcddc0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:44:14.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:14.793+0000 7fcde3400640 1 -- 172.21.15.5:0/501012319 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcde80bbcd0 con 0x559fbc3a9680 2024-09-10T09:44:14.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:14.793+0000 7fcde3400640 1 -- 172.21.15.5:0/501012319 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcde80baf30 con 0x559fbc3a9680 2024-09-10T09:44:14.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:14.793+0000 7fcdff8c5740 1 -- 172.21.15.5:0/501012319 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559fbc3a9680 msgr2=0x559fbc3bc2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:44:14.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:14.793+0000 7fcdff8c5740 1 --2- 172.21.15.5:0/501012319 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559fbc3a9680 0x559fbc3bc2f0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fcde80ec840 tx=0x7fcde80f7320 comp rx=0 tx=0).stop 2024-09-10T09:44:14.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:14.794+0000 7fcdff8c5740 1 -- 172.21.15.5:0/501012319 shutdown_connections 2024-09-10T09:44:14.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:14.794+0000 7fcdff8c5740 1 --2- 172.21.15.5:0/501012319 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559fbc3bc830 0x559fbc3bec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:14.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:14.794+0000 7fcdff8c5740 1 --2- 172.21.15.5:0/501012319 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559fbc3a9680 0x559fbc3bc2f0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:14.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:14.794+0000 7fcdff8c5740 1 --2- 172.21.15.5:0/501012319 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559fbc3a8d60 0x559fbc3a9140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:14.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:14.794+0000 7fcdff8c5740 1 -- 172.21.15.5:0/501012319 >> 172.21.15.5:0/501012319 conn(0x559fbc3a53d0 msgr2=0x559fbc3a57b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:44:14.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:14.794+0000 7fcdff8c5740 1 -- 172.21.15.5:0/501012319 shutdown_connections 2024-09-10T09:44:14.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:14.794+0000 7fcdf6800640 1 --2- 172.21.15.5:0/2082391245 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fcddc0a2c00 0x7fcddc0a50c0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fcdf0005c40 tx=0x7fcdf00095c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:44:14.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:14.794+0000 7fcdff8c5740 1 -- 172.21.15.5:0/501012319 wait complete. 2024-09-10T09:44:14.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:14.794+0000 7fcdff8c5740 1 -- start start 2024-09-10T09:44:14.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:14.795+0000 7fcdff8c5740 1 Processor -- start 2024-09-10T09:44:14.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:14.795+0000 7fcdff8c5740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559fbc3a8d60 0x559fbc3b9780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:44:14.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:14.795+0000 7fcdff8c5740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559fbc3a9680 0x559fbc3b9cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:44:14.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:14.795+0000 7fcdff8c5740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559fbc3bc830 0x559fbc3ba200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:44:14.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:14.795+0000 7fcdff8c5740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x559fbc3bf160 con 0x559fbc3bc830 2024-09-10T09:44:14.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:14.795+0000 7fcdff8c5740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x559fbc2ca0f0 con 0x559fbc3a8d60 2024-09-10T09:44:14.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:14.795+0000 7fcdff8c5740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x559fbc3bf2e0 con 0x559fbc3a9680 2024-09-10T09:44:14.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:14.795+0000 7fcdff8c5740 10 client.0 _ll_register_callbacks cb 0x559fbc2b1bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:44:14.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:14.796+0000 7fcdf5e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559fbc3a8d60 0x559fbc3b9780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:44:14.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:14.796+0000 7fcdf5e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559fbc3a8d60 0x559fbc3b9780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:38742/0 (socket says 172.21.15.5:38742) 2024-09-10T09:44:14.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:14.796+0000 7fcdf5e00640 1 -- 172.21.15.5:0/2026471135 learned_addr learned my addr 172.21.15.5:0/2026471135 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:44:14.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:14.796+0000 7fcdf5400640 1 --2- 172.21.15.5:0/2026471135 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559fbc3a9680 0x559fbc3b9cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:44:14.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:14.796+0000 7fcdf6800640 1 --2- 172.21.15.5:0/2026471135 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559fbc3bc830 0x559fbc3ba200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:44:14.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:14.796+0000 7fcdf5e00640 1 -- 172.21.15.5:0/2026471135 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559fbc3a9680 msgr2=0x559fbc3b9cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:44:14.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:14.796+0000 7fcdf5e00640 1 --2- 172.21.15.5:0/2026471135 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559fbc3a9680 0x559fbc3b9cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:14.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:14.796+0000 7fcdf5e00640 1 -- 172.21.15.5:0/2026471135 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559fbc3bc830 msgr2=0x559fbc3ba200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:44:14.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:14.796+0000 7fcdf5e00640 1 --2- 172.21.15.5:0/2026471135 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559fbc3bc830 0x559fbc3ba200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:14.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:14.796+0000 7fcdf5e00640 1 -- 172.21.15.5:0/2026471135 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559fbc0803c0 con 0x559fbc3a8d60 2024-09-10T09:44:14.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:14.796+0000 7fcdf6800640 1 --2- 172.21.15.5:0/2026471135 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559fbc3bc830 0x559fbc3ba200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T09:44:14.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:14.797+0000 7fcdf5e00640 1 --2- 172.21.15.5:0/2026471135 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559fbc3a8d60 0x559fbc3b9780 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fcde80ec810 tx=0x7fcde8004510 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7e2aeb50eed44213 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:44:14.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:14.797+0000 7fcdd7e00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-09-10T09:44:14.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:14.797+0000 7fcdd7e00640 1 -- 172.21.15.5:0/2026471135 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcde80bb110 con 0x559fbc3a8d60 2024-09-10T09:44:14.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:14.797+0000 7fcdff8c5740 1 -- 172.21.15.5:0/2026471135 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcde80bbcd0 con 0x559fbc3a8d60 2024-09-10T09:44:14.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:14.797+0000 7fcdd7e00640 1 -- 172.21.15.5:0/2026471135 <== mon.2 v2:172.21.15.102:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fcde8004170 con 0x559fbc3a8d60 2024-09-10T09:44:14.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:14.797+0000 7fcdd7e00640 1 -- 172.21.15.5:0/2026471135 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fcde80bb3a0 con 0x559fbc3a8d60 2024-09-10T09:44:14.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:14.797+0000 7fcdff8c5740 10 client.26507 fetch_fsmap learned FSMap version 34 2024-09-10T09:44:14.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:14.797+0000 7fcdff8c5740 1 -- 172.21.15.5:0/2026471135 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x559fbc3bf160 con 0x559fbc3a8d60 2024-09-10T09:44:14.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:14.798+0000 7fcdd7e00640 1 -- 172.21.15.5:0/2026471135 <== mon.2 v2:172.21.15.102:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcde80bbcd0 con 0x559fbc3a8d60 2024-09-10T09:44:14.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:14.798+0000 7fcdd7e00640 1 -- 172.21.15.5:0/2026471135 <== mon.2 v2:172.21.15.102:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7fcde803a070 con 0x559fbc3a8d60 2024-09-10T09:44:14.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:14.798+0000 7fcdff8c5740 10 client.26507 fetch_fsmap finished waiting for FSMap version 34 2024-09-10T09:44:14.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:14.798+0000 7fcdff8c5740 10 client.26507 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-09-10T09:44:14.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:14.798+0000 7fcdff8c5740 20 client.26507 populate_metadata read hostname 'smithi005' 2024-09-10T09:44:14.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:14.798+0000 7fcdff8c5740 1 --2- 172.21.15.5:0/2026471135 >> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] conn(0x559fbc3cc3c0 0x559fbc3ce7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:44:14.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:14.798+0000 7fcdff8c5740 4 client.26507 mds_command: new command op to 24443 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:44:14.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:14.798+0000 7fcdff8c5740 1 -- 172.21.15.5:0/2026471135 --> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559fbc31c950 con 0x559fbc3cc3c0 2024-09-10T09:44:14.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:14.798+0000 7fcdf5400640 1 --2- 172.21.15.5:0/2026471135 >> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] conn(0x559fbc3cc3c0 0x559fbc3ce7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:44:14.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:14.799+0000 7fcdf5400640 1 --2- 172.21.15.5:0/2026471135 >> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] conn(0x559fbc3cc3c0 0x559fbc3ce7d0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=2d69f864dabe2e14 server_cookie=4a3c940df54748cb in_seq=0 out_seq=0 2024-09-10T09:44:14.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:14.799+0000 7fcdd7e00640 10 client.26507 ms_handle_connect on v2:172.21.15.102:6832/2457007427 2024-09-10T09:44:14.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:14.800+0000 7fcdd7e00640 1 -- 172.21.15.5:0/2026471135 <== mds.4 v2:172.21.15.102:6832/2457007427 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559fbc31c950 con 0x559fbc3cc3c0 2024-09-10T09:44:14.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:14.800+0000 7fcdd7e00640 10 client.26507 handle_command_reply: tid=0 2024-09-10T09:44:14.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:14.800+0000 7fcdff8c5740 1 client.26507 shutdown 2024-09-10T09:44:14.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:14.800+0000 7fcdff8c5740 1 -- 172.21.15.5:0/2026471135 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559fbc3a8d60 msgr2=0x559fbc3b9780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:44:14.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:14.800+0000 7fcdff8c5740 1 --2- 172.21.15.5:0/2026471135 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559fbc3a8d60 0x559fbc3b9780 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fcde80ec810 tx=0x7fcde8004510 comp rx=0 tx=0).stop 2024-09-10T09:44:14.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:14.800+0000 7fcdff8c5740 1 -- 172.21.15.5:0/2026471135 shutdown_connections 2024-09-10T09:44:14.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:14.800+0000 7fcdff8c5740 1 --2- 172.21.15.5:0/2026471135 >> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] conn(0x559fbc3cc3c0 0x559fbc3ce7d0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:14.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:14.800+0000 7fcdff8c5740 1 --2- 172.21.15.5:0/2026471135 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559fbc3bc830 0x559fbc3ba200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:14.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:14.800+0000 7fcdff8c5740 1 --2- 172.21.15.5:0/2026471135 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559fbc3a9680 0x559fbc3b9cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:14.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:14.800+0000 7fcdff8c5740 1 --2- 172.21.15.5:0/2026471135 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559fbc3a8d60 0x559fbc3b9780 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:14.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:14.800+0000 7fcdff8c5740 1 -- 172.21.15.5:0/2026471135 >> 172.21.15.5:0/2026471135 conn(0x559fbc3a53d0 msgr2=0x559fbc3ab060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:44:14.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:14.800+0000 7fcdd7e00640 0 client.26507 ms_handle_reset on v2:172.21.15.102:6832/2457007427 2024-09-10T09:44:14.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:14.800+0000 7fcdff8c5740 1 -- 172.21.15.5:0/2026471135 shutdown_connections 2024-09-10T09:44:14.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:14.800+0000 7fcdff8c5740 1 -- 172.21.15.5:0/2026471135 wait complete. 2024-09-10T09:44:14.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:14.800+0000 7fcdff8c5740 20 client.26507 trim_cache size 0 max 16384 2024-09-10T09:44:14.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:14.814+0000 7fcdff8c5740 1 Processor -- start 2024-09-10T09:44:14.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:14.814+0000 7fcdff8c5740 1 -- start start 2024-09-10T09:44:14.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:14.814+0000 7fcdff8c5740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559fbc3a8d60 0x559fbc3d4070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:44:14.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:14.814+0000 7fcdff8c5740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559fbc3a9680 0x559fbc3d45b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:44:14.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:14.815+0000 7fcdff8c5740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559fbc3bc830 0x559fbc3b6a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:44:14.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:14.815+0000 7fcdff8c5740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fcdf81acaa0 con 0x559fbc3a8d60 2024-09-10T09:44:14.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:14.815+0000 7fcdff8c5740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fcdf81ae5e0 con 0x559fbc3bc830 2024-09-10T09:44:14.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:14.815+0000 7fcdff8c5740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fcdf81ad170 con 0x559fbc3a9680 2024-09-10T09:44:14.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:14.815+0000 7fcdf5e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559fbc3a8d60 0x559fbc3d4070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:44:14.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:14.815+0000 7fcdf5e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559fbc3a8d60 0x559fbc3d4070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37666/0 (socket says 172.21.15.5:37666) 2024-09-10T09:44:14.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:14.815+0000 7fcdf5e00640 1 -- 172.21.15.5:0/3465595610 learned_addr learned my addr 172.21.15.5:0/3465595610 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:44:14.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:14.815+0000 7fcdf5400640 1 --2- 172.21.15.5:0/3465595610 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559fbc3a9680 0x559fbc3d45b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:44:14.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:14.815+0000 7fcdf6800640 1 --2- 172.21.15.5:0/3465595610 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559fbc3bc830 0x559fbc3b6a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:44:14.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:14.815+0000 7fcdf5e00640 1 -- 172.21.15.5:0/3465595610 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559fbc3a9680 msgr2=0x559fbc3d45b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:44:14.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:14.815+0000 7fcdf5e00640 1 --2- 172.21.15.5:0/3465595610 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559fbc3a9680 0x559fbc3d45b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:14.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:14.815+0000 7fcdf5e00640 1 -- 172.21.15.5:0/3465595610 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559fbc3bc830 msgr2=0x559fbc3b6a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:44:14.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:14.815+0000 7fcdf5e00640 1 --2- 172.21.15.5:0/3465595610 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559fbc3bc830 0x559fbc3b6a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:14.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:14.815+0000 7fcdf5e00640 1 -- 172.21.15.5:0/3465595610 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559fbc3a4c80 con 0x559fbc3a8d60 2024-09-10T09:44:14.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:14.815+0000 7fcdf6800640 1 --2- 172.21.15.5:0/3465595610 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559fbc3bc830 0x559fbc3b6a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:44:14.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:14.816+0000 7fcdf5e00640 1 --2- 172.21.15.5:0/3465595610 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559fbc3a8d60 0x559fbc3d4070 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fcde80bb2d0 tx=0x7fcde80b9ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=553385157dffd6b5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:44:14.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:14.816+0000 7fcde2a00640 1 -- 172.21.15.5:0/3465595610 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcde80bb110 con 0x559fbc3a8d60 2024-09-10T09:44:14.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:14.816+0000 7fcde2a00640 1 -- 172.21.15.5:0/3465595610 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcde80bbcd0 con 0x559fbc3a8d60 2024-09-10T09:44:14.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:14.816+0000 7fcde2a00640 1 -- 172.21.15.5:0/3465595610 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcde80baf30 con 0x559fbc3a8d60 2024-09-10T09:44:14.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:14.817+0000 7fcdff8c5740 1 -- 172.21.15.5:0/3465595610 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559fbc3a8d60 msgr2=0x559fbc3d4070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:44:14.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:14.817+0000 7fcdff8c5740 1 --2- 172.21.15.5:0/3465595610 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559fbc3a8d60 0x559fbc3d4070 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fcde80bb2d0 tx=0x7fcde80b9ae0 comp rx=0 tx=0).stop 2024-09-10T09:44:14.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:14.817+0000 7fcdff8c5740 1 -- 172.21.15.5:0/3465595610 shutdown_connections 2024-09-10T09:44:14.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:14.817+0000 7fcdff8c5740 1 --2- 172.21.15.5:0/3465595610 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559fbc3bc830 0x559fbc3b6a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:14.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:14.817+0000 7fcdff8c5740 1 --2- 172.21.15.5:0/3465595610 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559fbc3a9680 0x559fbc3d45b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:14.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:14.817+0000 7fcdff8c5740 1 --2- 172.21.15.5:0/3465595610 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559fbc3a8d60 0x559fbc3d4070 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:14.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:14.817+0000 7fcdff8c5740 1 -- 172.21.15.5:0/3465595610 >> 172.21.15.5:0/3465595610 conn(0x559fbc3a53d0 msgr2=0x559fbc3aafd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:44:14.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:14.817+0000 7fcdff8c5740 1 -- 172.21.15.5:0/3465595610 shutdown_connections 2024-09-10T09:44:14.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:14.817+0000 7fcdff8c5740 1 -- 172.21.15.5:0/3465595610 wait complete. 2024-09-10T09:44:14.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:14.818+0000 7fcdff8c5740 1 -- start start 2024-09-10T09:44:14.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:14.818+0000 7fcdff8c5740 1 Processor -- start 2024-09-10T09:44:14.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:14.818+0000 7fcdff8c5740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559fbc3a9680 0x559fbc3b6790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:44:14.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:14.818+0000 7fcdff8c5740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559fbc3bc830 0x559fbc3b6cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:44:14.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:14.818+0000 7fcdff8c5740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559fbc3b7210 0x559fbc436040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:44:14.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:14.818+0000 7fcdff8c5740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fcdf81adf10 con 0x559fbc3b7210 2024-09-10T09:44:14.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:14.818+0000 7fcdff8c5740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fcdf81ad840 con 0x559fbc3a9680 2024-09-10T09:44:14.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:14.818+0000 7fcdff8c5740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fcdf81aaa80 con 0x559fbc3bc830 2024-09-10T09:44:14.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:14.818+0000 7fcdff8c5740 10 client.0 _ll_register_callbacks cb 0x7fcde4009470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:44:14.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:14.819+0000 7fcdf5400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559fbc3bc830 0x559fbc3b6cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:44:14.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:14.819+0000 7fcdf5e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559fbc3a9680 0x559fbc3b6790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:44:14.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:14.819+0000 7fcdf5400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559fbc3bc830 0x559fbc3b6cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:39118/0 (socket says 172.21.15.5:39118) 2024-09-10T09:44:14.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:14.819+0000 7fcdf5e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559fbc3a9680 0x559fbc3b6790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:38768/0 (socket says 172.21.15.5:38768) 2024-09-10T09:44:14.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:14.819+0000 7fcdf5400640 1 -- 172.21.15.5:0/2534211465 learned_addr learned my addr 172.21.15.5:0/2534211465 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:44:14.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:14.819+0000 7fcdf6800640 1 --2- 172.21.15.5:0/2534211465 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559fbc3b7210 0x559fbc436040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:44:14.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:14.819+0000 7fcdf5e00640 1 -- 172.21.15.5:0/2534211465 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559fbc3bc830 msgr2=0x559fbc3b6cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:44:14.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:14.819+0000 7fcdf5e00640 1 --2- 172.21.15.5:0/2534211465 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559fbc3bc830 0x559fbc3b6cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:14.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:14.819+0000 7fcdf5e00640 1 -- 172.21.15.5:0/2534211465 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559fbc3b7210 msgr2=0x559fbc436040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:44:14.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:14.819+0000 7fcdf5e00640 1 --2- 172.21.15.5:0/2534211465 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559fbc3b7210 0x559fbc436040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:14.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:14.819+0000 7fcdf5e00640 1 -- 172.21.15.5:0/2534211465 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559fbc436580 con 0x559fbc3a9680 2024-09-10T09:44:14.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:14.819+0000 7fcdf5e00640 1 --2- 172.21.15.5:0/2534211465 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559fbc3a9680 0x559fbc3b6790 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fcde80bd000 tx=0x7fcde8004510 comp rx=0 tx=0).ready entity=mon.2 client_cookie=85a8730bd5927877 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:44:14.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:14.820+0000 7fcdd7e00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-09-10T09:44:14.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:14.820+0000 7fcdd7e00640 1 -- 172.21.15.5:0/2534211465 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcde80b7870 con 0x559fbc3a9680 2024-09-10T09:44:14.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:14.820+0000 7fcdd7e00640 1 -- 172.21.15.5:0/2534211465 <== mon.2 v2:172.21.15.102:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fcde80bba30 con 0x559fbc3a9680 2024-09-10T09:44:14.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:14.820+0000 7fcdff8c5740 1 -- 172.21.15.5:0/2534211465 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcde80bbcd0 con 0x559fbc3a9680 2024-09-10T09:44:14.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:14.820+0000 7fcdd7e00640 1 -- 172.21.15.5:0/2534211465 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fcde80baf30 con 0x559fbc3a9680 2024-09-10T09:44:14.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:14.820+0000 7fcdff8c5740 10 client.26513 fetch_fsmap learned FSMap version 34 2024-09-10T09:44:14.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:14.820+0000 7fcdff8c5740 1 -- 172.21.15.5:0/2534211465 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x559fbc436580 con 0x559fbc3a9680 2024-09-10T09:44:14.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:14.820+0000 7fcdd7e00640 1 -- 172.21.15.5:0/2534211465 <== mon.2 v2:172.21.15.102:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcde80bbcd0 con 0x559fbc3a9680 2024-09-10T09:44:14.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:14.820+0000 7fcdd7e00640 1 -- 172.21.15.5:0/2534211465 <== mon.2 v2:172.21.15.102:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7fcde803a030 con 0x559fbc3a9680 2024-09-10T09:44:14.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:14.820+0000 7fcdff8c5740 10 client.26513 fetch_fsmap finished waiting for FSMap version 34 2024-09-10T09:44:14.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:14.820+0000 7fcdff8c5740 10 client.26513 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-09-10T09:44:14.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:14.820+0000 7fcdff8c5740 20 client.26513 populate_metadata read hostname 'smithi005' 2024-09-10T09:44:14.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:14.821+0000 7fcdff8c5740 1 --2- 172.21.15.5:0/2534211465 >> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] conn(0x559fbc437a60 0x559fbc439e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:44:14.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:14.821+0000 7fcdff8c5740 4 client.26513 mds_command: new command op to 24443 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T09:44:14.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:14.821+0000 7fcdff8c5740 1 -- 172.21.15.5:0/2534211465 --> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559fbc316e60 con 0x559fbc437a60 2024-09-10T09:44:14.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:14.821+0000 7fcdf5400640 1 --2- 172.21.15.5:0/2534211465 >> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] conn(0x559fbc437a60 0x559fbc439e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:44:14.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:14.821+0000 7fcdf5400640 1 --2- 172.21.15.5:0/2534211465 >> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] conn(0x559fbc437a60 0x559fbc439e50 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=83023784810f2bba server_cookie=f05c37702b0a1f0f in_seq=0 out_seq=0 2024-09-10T09:44:14.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:14.821+0000 7fcdd7e00640 10 client.26513 ms_handle_connect on v2:172.21.15.102:6832/2457007427 2024-09-10T09:44:14.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:14.821+0000 7fcdd7e00640 1 -- 172.21.15.5:0/2534211465 <== mds.4 v2:172.21.15.102:6832/2457007427 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559fbc316e60 con 0x559fbc437a60 2024-09-10T09:44:14.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:14.822+0000 7fcdd7e00640 10 client.26513 handle_command_reply: tid=0 2024-09-10T09:44:14.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:14.822+0000 7fcdff8c5740 1 client.26513 shutdown 2024-09-10T09:44:14.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:14.822+0000 7fcdff8c5740 1 -- 172.21.15.5:0/2534211465 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559fbc3a9680 msgr2=0x559fbc3b6790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:44:14.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:14.822+0000 7fcdff8c5740 1 --2- 172.21.15.5:0/2534211465 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559fbc3a9680 0x559fbc3b6790 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fcde80bd000 tx=0x7fcde8004510 comp rx=0 tx=0).stop 2024-09-10T09:44:14.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:14.822+0000 7fcdff8c5740 1 -- 172.21.15.5:0/2534211465 shutdown_connections 2024-09-10T09:44:14.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:14.822+0000 7fcdff8c5740 1 --2- 172.21.15.5:0/2534211465 >> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] conn(0x559fbc437a60 0x559fbc439e50 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:14.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:14.822+0000 7fcdff8c5740 1 --2- 172.21.15.5:0/2534211465 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559fbc3b7210 0x559fbc436040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:14.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:14.822+0000 7fcdff8c5740 1 --2- 172.21.15.5:0/2534211465 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559fbc3bc830 0x559fbc3b6cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:14.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:14.822+0000 7fcdff8c5740 1 --2- 172.21.15.5:0/2534211465 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559fbc3a9680 0x559fbc3b6790 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:14.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:14.822+0000 7fcdd7e00640 0 client.26513 ms_handle_reset on v2:172.21.15.102:6832/2457007427 2024-09-10T09:44:14.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:14.822+0000 7fcdff8c5740 1 -- 172.21.15.5:0/2534211465 >> 172.21.15.5:0/2534211465 conn(0x559fbc3a53d0 msgr2=0x559fbc3b8050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:44:14.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:14.822+0000 7fcdff8c5740 1 -- 172.21.15.5:0/2534211465 shutdown_connections 2024-09-10T09:44:14.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:14.822+0000 7fcdff8c5740 1 -- 172.21.15.5:0/2534211465 wait complete. 2024-09-10T09:44:14.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:14.822+0000 7fcdff8c5740 20 client.26513 trim_cache size 0 max 16384 2024-09-10T09:44:14.828 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-10T09:44:14.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:14.823+0000 7fcdf7200640 1 -- 172.21.15.5:0/2082391245 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fcddc0a2c00 msgr2=0x7fcddc0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:44:14.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:14.823+0000 7fcdf7200640 1 --2- 172.21.15.5:0/2082391245 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fcddc0a2c00 0x7fcddc0a50c0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fcdf0005c40 tx=0x7fcdf00095c0 comp rx=0 tx=0).stop 2024-09-10T09:44:14.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:14.823+0000 7fcdf7200640 1 -- 172.21.15.5:0/2082391245 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcdf81019c0 msgr2=0x7fcdf819a450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:44:14.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:14.823+0000 7fcdf7200640 1 --2- 172.21.15.5:0/2082391245 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcdf81019c0 0x7fcdf819a450 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fcde8035040 tx=0x7fcde8033b60 comp rx=0 tx=0).stop 2024-09-10T09:44:14.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:14.823+0000 7fcdf7200640 1 -- 172.21.15.5:0/2082391245 shutdown_connections 2024-09-10T09:44:14.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:14.823+0000 7fcdf7200640 1 --2- 172.21.15.5:0/2082391245 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fcddc0a2c00 0x7fcddc0a50c0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:14.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:14.823+0000 7fcdf7200640 1 --2- 172.21.15.5:0/2082391245 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcdf81079c0 0x7fcdf819ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:14.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:14.823+0000 7fcdf7200640 1 --2- 172.21.15.5:0/2082391245 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcdf8102360 0x7fcdf819a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:14.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:14.823+0000 7fcdf7200640 1 --2- 172.21.15.5:0/2082391245 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcdf81019c0 0x7fcdf819a450 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:14.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:14.823+0000 7fcdf7200640 1 -- 172.21.15.5:0/2082391245 >> 172.21.15.5:0/2082391245 conn(0x7fcdf80fd740 msgr2=0x7fcdf810cc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:44:14.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:14.824+0000 7fcdf7200640 1 -- 172.21.15.5:0/2082391245 shutdown_connections 2024-09-10T09:44:14.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:14.824+0000 7fcdf7200640 1 -- 172.21.15.5:0/2082391245 wait complete. 2024-09-10T09:44:14.867 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:44:14 smithi005 ceph-mon[30350]: pgmap v570: 129 pgs: 129 active+clean; 139 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.4 MiB/s wr, 171 op/s 2024-09-10T09:44:15.043 INFO:tasks.workunit.client.0.smithi005.stdout: 1 322222 6.04 MB/sec execute 116 sec latency 345.048 ms 2024-09-10T09:44:15.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:44:14 smithi159 ceph-mon[35513]: pgmap v570: 129 pgs: 129 active+clean; 139 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.4 MiB/s wr, 171 op/s 2024-09-10T09:44:15.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:44:14 smithi102 ceph-mon[35442]: pgmap v570: 129 pgs: 129 active+clean; 139 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.4 MiB/s wr, 171 op/s 2024-09-10T09:44:16.043 INFO:tasks.workunit.client.0.smithi005.stdout: 1 322222 5.99 MB/sec execute 117 sec latency 1082.479 ms 2024-09-10T09:44:16.438 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-10T09:44:16.836 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:44:17.043 INFO:tasks.workunit.client.0.smithi005.stdout: 1 323478 6.00 MB/sec execute 118 sec latency 1331.925 ms 2024-09-10T09:44:17.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:44:16 smithi005 ceph-mon[30350]: pgmap v571: 129 pgs: 129 active+clean; 138 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.4 MiB/s wr, 186 op/s 2024-09-10T09:44:17.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:44:16 smithi159 ceph-mon[35513]: pgmap v571: 129 pgs: 129 active+clean; 138 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.4 MiB/s wr, 186 op/s 2024-09-10T09:44:17.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:44:16 smithi102 ceph-mon[35442]: pgmap v571: 129 pgs: 129 active+clean; 138 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.4 MiB/s wr, 186 op/s 2024-09-10T09:44:17.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:17.473+0000 7fe0fb200640 1 -- 172.21.15.5:0/3885433410 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe0fc103e30 msgr2=0x7fe0fc104210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:44:17.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:17.473+0000 7fe0fb200640 1 --2- 172.21.15.5:0/3885433410 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe0fc103e30 0x7fe0fc104210 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fe0ec009900 tx=0x7fe0ec02a3f0 comp rx=0 tx=0).stop 2024-09-10T09:44:17.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:17.474+0000 7fe0fb200640 1 -- 172.21.15.5:0/3885433410 shutdown_connections 2024-09-10T09:44:17.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:17.474+0000 7fe0fb200640 1 --2- 172.21.15.5:0/3885433410 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe0fc108f60 0x7fe0fc110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:17.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:17.474+0000 7fe0fb200640 1 --2- 172.21.15.5:0/3885433410 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe0fc1047e0 0x7fe0fc108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:17.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:17.474+0000 7fe0fb200640 1 --2- 172.21.15.5:0/3885433410 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe0fc103e30 0x7fe0fc104210 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:17.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:17.474+0000 7fe0fb200640 1 -- 172.21.15.5:0/3885433410 >> 172.21.15.5:0/3885433410 conn(0x7fe0fc0ff8d0 msgr2=0x7fe0fc101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:44:17.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:17.474+0000 7fe0fb200640 1 -- 172.21.15.5:0/3885433410 shutdown_connections 2024-09-10T09:44:17.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:17.474+0000 7fe0fb200640 1 -- 172.21.15.5:0/3885433410 wait complete. 2024-09-10T09:44:17.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:17.475+0000 7fe0fb200640 1 Processor -- start 2024-09-10T09:44:17.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:17.475+0000 7fe0fb200640 1 -- start start 2024-09-10T09:44:17.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:17.475+0000 7fe0fb200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe0fc103e30 0x7fe0fc19a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:44:17.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:17.475+0000 7fe0fb200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe0fc1047e0 0x7fe0fc19a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:44:17.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:17.475+0000 7fe0fb200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe0fc108f60 0x7fe0fc19ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:44:17.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:17.475+0000 7fe0fb200640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe0fc114010 con 0x7fe0fc1047e0 2024-09-10T09:44:17.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:17.475+0000 7fe0fb200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fe0fc113e90 con 0x7fe0fc103e30 2024-09-10T09:44:17.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:17.475+0000 7fe0fb200640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fe0fc114190 con 0x7fe0fc108f60 2024-09-10T09:44:17.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:17.476+0000 7fe0fa800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe0fc108f60 0x7fe0fc19ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:44:17.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:17.476+0000 7fe0fa800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe0fc108f60 0x7fe0fc19ed80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:39146/0 (socket says 172.21.15.5:39146) 2024-09-10T09:44:17.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:17.476+0000 7fe0fa800640 1 -- 172.21.15.5:0/2998622734 learned_addr learned my addr 172.21.15.5:0/2998622734 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:44:17.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:17.476+0000 7fe0f9e00640 1 --2- 172.21.15.5:0/2998622734 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe0fc103e30 0x7fe0fc19a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:44:17.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:17.476+0000 7fe0f3e00640 1 --2- 172.21.15.5:0/2998622734 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe0fc1047e0 0x7fe0fc19a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:44:17.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:17.476+0000 7fe0fa800640 1 -- 172.21.15.5:0/2998622734 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe0fc103e30 msgr2=0x7fe0fc19a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:44:17.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:17.476+0000 7fe0fa800640 1 --2- 172.21.15.5:0/2998622734 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe0fc103e30 0x7fe0fc19a4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:17.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:17.476+0000 7fe0fa800640 1 -- 172.21.15.5:0/2998622734 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe0fc1047e0 msgr2=0x7fe0fc19a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:44:17.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:17.476+0000 7fe0fa800640 1 --2- 172.21.15.5:0/2998622734 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe0fc1047e0 0x7fe0fc19a9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:17.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:17.476+0000 7fe0fa800640 1 -- 172.21.15.5:0/2998622734 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0fc19f460 con 0x7fe0fc108f60 2024-09-10T09:44:17.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:17.476+0000 7fe0f3e00640 1 --2- 172.21.15.5:0/2998622734 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe0fc1047e0 0x7fe0fc19a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:44:17.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:17.476+0000 7fe0fa800640 1 --2- 172.21.15.5:0/2998622734 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe0fc108f60 0x7fe0fc19ed80 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fe0f400eb10 tx=0x7fe0f400c2d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:44:17.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:17.477+0000 7fe0f2a00640 1 -- 172.21.15.5:0/2998622734 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0f4004b20 con 0x7fe0fc108f60 2024-09-10T09:44:17.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:17.477+0000 7fe0fb200640 1 -- 172.21.15.5:0/2998622734 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0fc19f750 con 0x7fe0fc108f60 2024-09-10T09:44:17.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:17.477+0000 7fe0f2a00640 1 -- 172.21.15.5:0/2998622734 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe0f4004cc0 con 0x7fe0fc108f60 2024-09-10T09:44:17.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:17.477+0000 7fe0f2a00640 1 -- 172.21.15.5:0/2998622734 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0f4010760 con 0x7fe0fc108f60 2024-09-10T09:44:17.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:17.477+0000 7fe0fb200640 1 -- 172.21.15.5:0/2998622734 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0fc1a7030 con 0x7fe0fc108f60 2024-09-10T09:44:17.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:17.481+0000 7fe103a2b740 1 Processor -- start 2024-09-10T09:44:17.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:17.481+0000 7fe0f2a00640 1 -- 172.21.15.5:0/2998622734 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7fe0f4005020 con 0x7fe0fc108f60 2024-09-10T09:44:17.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:17.482+0000 7fe103a2b740 1 -- start start 2024-09-10T09:44:17.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:17.482+0000 7fe103a2b740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a580cfce40 0x55a580cfd220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:44:17.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:17.482+0000 7fe103a2b740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a580cfd760 0x55a580d07ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:44:17.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:17.482+0000 7fe103a2b740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a580d08020 0x55a580d0a410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:44:17.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:17.482+0000 7fe103a2b740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55a580c633e0 con 0x55a580d08020 2024-09-10T09:44:17.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:17.482+0000 7fe103a2b740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55a580c0ad80 con 0x55a580cfce40 2024-09-10T09:44:17.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:17.482+0000 7fe103a2b740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55a580c5f3a0 con 0x55a580cfd760 2024-09-10T09:44:17.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:17.482+0000 7fe0f9e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a580cfd760 0x55a580d07ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:44:17.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:17.482+0000 7fe0f3e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a580cfce40 0x55a580cfd220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:44:17.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:17.482+0000 7fe0f9e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a580cfd760 0x55a580d07ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:39152/0 (socket says 172.21.15.5:39152) 2024-09-10T09:44:17.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:17.482+0000 7fe0f9e00640 1 -- 172.21.15.5:0/2475616665 learned_addr learned my addr 172.21.15.5:0/2475616665 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:44:17.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:17.483+0000 7fe0f9e00640 1 -- 172.21.15.5:0/2475616665 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a580cfce40 msgr2=0x55a580cfd220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:44:17.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:17.483+0000 7fe0f9e00640 1 --2- 172.21.15.5:0/2475616665 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a580cfce40 0x55a580cfd220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:17.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:17.483+0000 7fe0f9e00640 1 -- 172.21.15.5:0/2475616665 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a580d08020 msgr2=0x55a580d0a410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:44:17.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:17.483+0000 7fe0f9e00640 1 --2- 172.21.15.5:0/2475616665 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a580d08020 0x55a580d0a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:17.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:17.483+0000 7fe0f9e00640 1 -- 172.21.15.5:0/2475616665 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a580c29ae0 con 0x55a580cfd760 2024-09-10T09:44:17.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:17.483+0000 7fe0f9e00640 1 --2- 172.21.15.5:0/2475616665 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a580cfd760 0x55a580d07ae0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fe0ec009900 tx=0x7fe0ec005de0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3b9d48dab7bfe0c1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:44:17.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:17.483+0000 7fe0f2a00640 1 --2- 172.21.15.5:0/2998622734 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fe0e00a2b80 0x7fe0e00a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:44:17.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:17.483+0000 7fe0eb400640 1 -- 172.21.15.5:0/2475616665 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0ec035040 con 0x55a580cfd760 2024-09-10T09:44:17.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:17.484+0000 7fe0f2a00640 1 -- 172.21.15.5:0/2998622734 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fe0f40cfa30 con 0x7fe0fc108f60 2024-09-10T09:44:17.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:17.484+0000 7fe0eb400640 1 -- 172.21.15.5:0/2475616665 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe0ec004400 con 0x55a580cfd760 2024-09-10T09:44:17.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:17.484+0000 7fe0eb400640 1 -- 172.21.15.5:0/2475616665 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0ec0035f0 con 0x55a580cfd760 2024-09-10T09:44:17.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:17.484+0000 7fe0f3e00640 1 --2- 172.21.15.5:0/2998622734 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fe0e00a2b80 0x7fe0e00a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:44:17.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:17.484+0000 7fe103a2b740 1 -- 172.21.15.5:0/2475616665 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a580cfd760 msgr2=0x55a580d07ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:44:17.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:17.484+0000 7fe103a2b740 1 --2- 172.21.15.5:0/2475616665 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a580cfd760 0x55a580d07ae0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fe0ec009900 tx=0x7fe0ec005de0 comp rx=0 tx=0).stop 2024-09-10T09:44:17.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:17.485+0000 7fe103a2b740 1 -- 172.21.15.5:0/2475616665 shutdown_connections 2024-09-10T09:44:17.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:17.485+0000 7fe103a2b740 1 --2- 172.21.15.5:0/2475616665 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a580d08020 0x55a580d0a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:17.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:17.485+0000 7fe103a2b740 1 --2- 172.21.15.5:0/2475616665 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a580cfd760 0x55a580d07ae0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:17.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:17.485+0000 7fe103a2b740 1 --2- 172.21.15.5:0/2475616665 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a580cfce40 0x55a580cfd220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:17.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:17.485+0000 7fe103a2b740 1 -- 172.21.15.5:0/2475616665 >> 172.21.15.5:0/2475616665 conn(0x55a580cf93d0 msgr2=0x55a580cf97b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:44:17.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:17.485+0000 7fe103a2b740 1 -- 172.21.15.5:0/2475616665 shutdown_connections 2024-09-10T09:44:17.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:17.485+0000 7fe103a2b740 1 -- 172.21.15.5:0/2475616665 wait complete. 2024-09-10T09:44:17.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:17.485+0000 7fe0f3e00640 1 --2- 172.21.15.5:0/2998622734 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fe0e00a2b80 0x7fe0e00a5040 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x55a580cfa750 tx=0x7fe0e40095e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:44:17.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:17.485+0000 7fe103a2b740 1 -- start start 2024-09-10T09:44:17.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:17.486+0000 7fe103a2b740 1 Processor -- start 2024-09-10T09:44:17.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:17.486+0000 7fe103a2b740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a580cfce40 0x55a580d04ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:44:17.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:17.486+0000 7fe103a2b740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a580cfd760 0x55a580d05430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:44:17.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:17.486+0000 7fe103a2b740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a580d08020 0x55a580d05970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:44:17.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:17.486+0000 7fe103a2b740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55a580d0abf0 con 0x55a580cfd760 2024-09-10T09:44:17.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:17.486+0000 7fe103a2b740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55a580c1e0f0 con 0x55a580cfce40 2024-09-10T09:44:17.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:17.486+0000 7fe103a2b740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55a580d0ad70 con 0x55a580d08020 2024-09-10T09:44:17.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:17.486+0000 7fe103a2b740 10 client.0 _ll_register_callbacks cb 0x55a580c05bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:44:17.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:17.486+0000 7fe0f9e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a580cfce40 0x55a580d04ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:44:17.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:17.486+0000 7fe0f9e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a580cfce40 0x55a580d04ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:38816/0 (socket says 172.21.15.5:38816) 2024-09-10T09:44:17.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:17.487+0000 7fe0f9e00640 1 -- 172.21.15.5:0/499645188 learned_addr learned my addr 172.21.15.5:0/499645188 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:44:17.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:17.487+0000 7fe0f3e00640 1 --2- 172.21.15.5:0/499645188 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a580cfd760 0x55a580d05430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:44:17.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:17.487+0000 7fe0fa800640 1 --2- 172.21.15.5:0/499645188 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a580d08020 0x55a580d05970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:44:17.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:17.487+0000 7fe0f9e00640 1 -- 172.21.15.5:0/499645188 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a580d08020 msgr2=0x55a580d05970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:44:17.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:17.487+0000 7fe0f9e00640 1 --2- 172.21.15.5:0/499645188 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a580d08020 0x55a580d05970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:17.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:17.487+0000 7fe0f9e00640 1 -- 172.21.15.5:0/499645188 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a580cfd760 msgr2=0x55a580d05430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:44:17.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:17.487+0000 7fe0f9e00640 1 --2- 172.21.15.5:0/499645188 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a580cfd760 0x55a580d05430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:17.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:17.487+0000 7fe0f9e00640 1 -- 172.21.15.5:0/499645188 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a5809d43c0 con 0x55a580cfce40 2024-09-10T09:44:17.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:17.487+0000 7fe0f3e00640 1 --2- 172.21.15.5:0/499645188 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a580cfd760 0x55a580d05430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:44:17.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:17.487+0000 7fe0f9e00640 1 --2- 172.21.15.5:0/499645188 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a580cfce40 0x55a580d04ef0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fe0ec0048f0 tx=0x7fe0ec02a3f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aae25f42bc314d32 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:44:17.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:17.487+0000 7fe0dfe00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-09-10T09:44:17.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:17.488+0000 7fe0dfe00640 1 -- 172.21.15.5:0/499645188 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0ec052060 con 0x55a580cfce40 2024-09-10T09:44:17.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:17.488+0000 7fe103a2b740 1 -- 172.21.15.5:0/499645188 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe0ec004400 con 0x55a580cfce40 2024-09-10T09:44:17.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:17.488+0000 7fe0dfe00640 1 -- 172.21.15.5:0/499645188 <== mon.2 v2:172.21.15.102:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fe0ec004200 con 0x55a580cfce40 2024-09-10T09:44:17.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:17.488+0000 7fe0dfe00640 1 -- 172.21.15.5:0/499645188 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fe0ec035070 con 0x55a580cfce40 2024-09-10T09:44:17.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:17.488+0000 7fe103a2b740 10 client.26525 fetch_fsmap learned FSMap version 34 2024-09-10T09:44:17.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:17.488+0000 7fe103a2b740 1 -- 172.21.15.5:0/499645188 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55a580d1f420 con 0x55a580cfce40 2024-09-10T09:44:17.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:17.489+0000 7fe0dfe00640 1 -- 172.21.15.5:0/499645188 <== mon.2 v2:172.21.15.102:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe0ec004400 con 0x55a580cfce40 2024-09-10T09:44:17.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:17.489+0000 7fe0dfe00640 1 -- 172.21.15.5:0/499645188 <== mon.2 v2:172.21.15.102:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7fe0ec0039d0 con 0x55a580cfce40 2024-09-10T09:44:17.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:17.489+0000 7fe103a2b740 10 client.26525 fetch_fsmap finished waiting for FSMap version 34 2024-09-10T09:44:17.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:17.489+0000 7fe103a2b740 10 client.26525 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-09-10T09:44:17.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:17.489+0000 7fe103a2b740 20 client.26525 populate_metadata read hostname 'smithi005' 2024-09-10T09:44:17.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:17.489+0000 7fe103a2b740 1 --2- 172.21.15.5:0/499645188 >> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] conn(0x55a580d20770 0x55a580d22b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:44:17.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:17.489+0000 7fe103a2b740 4 client.26525 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:44:17.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:17.489+0000 7fe103a2b740 1 -- 172.21.15.5:0/499645188 --> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a580c70950 con 0x55a580d20770 2024-09-10T09:44:17.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:17.489+0000 7fe0f3e00640 1 --2- 172.21.15.5:0/499645188 >> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] conn(0x55a580d20770 0x55a580d22b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:44:17.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:17.490+0000 7fe0f3e00640 1 --2- 172.21.15.5:0/499645188 >> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] conn(0x55a580d20770 0x55a580d22b60 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=8e97ca2664548070 server_cookie=63c326a5b3318aea in_seq=0 out_seq=0 2024-09-10T09:44:17.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:17.490+0000 7fe0dfe00640 10 client.26525 ms_handle_connect on v2:172.21.15.102:6836/2689776764 2024-09-10T09:44:17.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:17.490+0000 7fe0dfe00640 1 -- 172.21.15.5:0/499645188 <== mds.3 v2:172.21.15.102:6836/2689776764 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a580c70950 con 0x55a580d20770 2024-09-10T09:44:17.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:17.490+0000 7fe0dfe00640 10 client.26525 handle_command_reply: tid=0 2024-09-10T09:44:17.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:17.490+0000 7fe103a2b740 1 client.26525 shutdown 2024-09-10T09:44:17.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:17.491+0000 7fe103a2b740 1 -- 172.21.15.5:0/499645188 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a580cfce40 msgr2=0x55a580d04ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:44:17.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:17.491+0000 7fe103a2b740 1 --2- 172.21.15.5:0/499645188 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a580cfce40 0x55a580d04ef0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fe0ec0048f0 tx=0x7fe0ec02a3f0 comp rx=0 tx=0).stop 2024-09-10T09:44:17.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:17.491+0000 7fe103a2b740 1 -- 172.21.15.5:0/499645188 shutdown_connections 2024-09-10T09:44:17.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:17.491+0000 7fe103a2b740 1 --2- 172.21.15.5:0/499645188 >> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] conn(0x55a580d20770 0x55a580d22b60 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:17.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:17.491+0000 7fe103a2b740 1 --2- 172.21.15.5:0/499645188 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a580d08020 0x55a580d05970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:17.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:17.491+0000 7fe103a2b740 1 --2- 172.21.15.5:0/499645188 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a580cfd760 0x55a580d05430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:17.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:17.491+0000 7fe103a2b740 1 --2- 172.21.15.5:0/499645188 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a580cfce40 0x55a580d04ef0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:17.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:17.491+0000 7fe103a2b740 1 -- 172.21.15.5:0/499645188 >> 172.21.15.5:0/499645188 conn(0x55a580cf93d0 msgr2=0x55a580cff0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:44:17.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:17.491+0000 7fe0dfe00640 0 client.26525 ms_handle_reset on v2:172.21.15.102:6836/2689776764 2024-09-10T09:44:17.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:17.491+0000 7fe103a2b740 1 -- 172.21.15.5:0/499645188 shutdown_connections 2024-09-10T09:44:17.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:17.491+0000 7fe103a2b740 1 -- 172.21.15.5:0/499645188 wait complete. 2024-09-10T09:44:17.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:17.491+0000 7fe103a2b740 20 client.26525 trim_cache size 0 max 16384 2024-09-10T09:44:17.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:17.505+0000 7fe103a2b740 1 Processor -- start 2024-09-10T09:44:17.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:17.505+0000 7fe103a2b740 1 -- start start 2024-09-10T09:44:17.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:17.506+0000 7fe103a2b740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a580cfce40 0x55a580d25b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:44:17.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:17.506+0000 7fe103a2b740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a580cfd760 0x55a580d260c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:44:17.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:17.506+0000 7fe103a2b740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a580d08020 0x55a580d02150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:44:17.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:17.506+0000 7fe103a2b740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe0fc1acac0 con 0x55a580d08020 2024-09-10T09:44:17.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:17.506+0000 7fe103a2b740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fe0fc1ae600 con 0x55a580cfd760 2024-09-10T09:44:17.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:17.506+0000 7fe103a2b740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fe0fc1ad190 con 0x55a580cfce40 2024-09-10T09:44:17.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:17.506+0000 7fe0f3e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a580cfd760 0x55a580d260c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:44:17.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:17.506+0000 7fe0f9e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a580cfce40 0x55a580d25b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:44:17.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:17.506+0000 7fe0f3e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a580cfd760 0x55a580d260c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:38828/0 (socket says 172.21.15.5:38828) 2024-09-10T09:44:17.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:17.506+0000 7fe0f3e00640 1 -- 172.21.15.5:0/942892990 learned_addr learned my addr 172.21.15.5:0/942892990 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:44:17.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:17.506+0000 7fe0f9e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a580cfce40 0x55a580d25b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:39180/0 (socket says 172.21.15.5:39180) 2024-09-10T09:44:17.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:17.506+0000 7fe0fa800640 1 --2- 172.21.15.5:0/942892990 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a580d08020 0x55a580d02150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:44:17.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:17.506+0000 7fe0f3e00640 1 -- 172.21.15.5:0/942892990 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a580cfce40 msgr2=0x55a580d25b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:44:17.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:17.507+0000 7fe0f3e00640 1 --2- 172.21.15.5:0/942892990 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a580cfce40 0x55a580d25b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:17.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:17.507+0000 7fe0f3e00640 1 -- 172.21.15.5:0/942892990 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a580d08020 msgr2=0x55a580d02150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:44:17.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:17.507+0000 7fe0f3e00640 1 --2- 172.21.15.5:0/942892990 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a580d08020 0x55a580d02150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:17.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:17.507+0000 7fe0f3e00640 1 -- 172.21.15.5:0/942892990 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a580cffcb0 con 0x55a580cfd760 2024-09-10T09:44:17.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:17.507+0000 7fe0f3e00640 1 --2- 172.21.15.5:0/942892990 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a580cfd760 0x55a580d260c0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fe0e4004700 tx=0x7fe0e4016000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5cf28bdd7bb02c77 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:44:17.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:17.507+0000 7fe0eaa00640 1 -- 172.21.15.5:0/942892990 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0e4020190 con 0x55a580cfd760 2024-09-10T09:44:17.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:17.507+0000 7fe0eaa00640 1 -- 172.21.15.5:0/942892990 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe0e4005500 con 0x55a580cfd760 2024-09-10T09:44:17.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:17.507+0000 7fe0eaa00640 1 -- 172.21.15.5:0/942892990 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0e4020500 con 0x55a580cfd760 2024-09-10T09:44:17.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:17.508+0000 7fe103a2b740 1 -- 172.21.15.5:0/942892990 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a580cfd760 msgr2=0x55a580d260c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:44:17.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:17.508+0000 7fe103a2b740 1 --2- 172.21.15.5:0/942892990 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a580cfd760 0x55a580d260c0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fe0e4004700 tx=0x7fe0e4016000 comp rx=0 tx=0).stop 2024-09-10T09:44:17.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:17.508+0000 7fe103a2b740 1 -- 172.21.15.5:0/942892990 shutdown_connections 2024-09-10T09:44:17.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:17.508+0000 7fe103a2b740 1 --2- 172.21.15.5:0/942892990 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a580d08020 0x55a580d02150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:17.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:17.508+0000 7fe103a2b740 1 --2- 172.21.15.5:0/942892990 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a580cfd760 0x55a580d260c0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:17.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:17.508+0000 7fe103a2b740 1 --2- 172.21.15.5:0/942892990 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a580cfce40 0x55a580d25b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:17.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:17.508+0000 7fe103a2b740 1 -- 172.21.15.5:0/942892990 >> 172.21.15.5:0/942892990 conn(0x55a580cf93d0 msgr2=0x55a580d06f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:44:17.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:17.508+0000 7fe103a2b740 1 -- 172.21.15.5:0/942892990 shutdown_connections 2024-09-10T09:44:17.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:17.508+0000 7fe103a2b740 1 -- 172.21.15.5:0/942892990 wait complete. 2024-09-10T09:44:17.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:17.508+0000 7fe103a2b740 1 -- start start 2024-09-10T09:44:17.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:17.509+0000 7fe103a2b740 1 Processor -- start 2024-09-10T09:44:17.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:17.509+0000 7fe103a2b740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a580cfce40 0x55a580d239c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:44:17.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:17.509+0000 7fe103a2b740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a580cfd760 0x55a580d23f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:44:17.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:17.509+0000 7fe103a2b740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a580d08020 0x55a580d24440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:44:17.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:17.509+0000 7fe103a2b740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe0fc1adf30 con 0x55a580cfd760 2024-09-10T09:44:17.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:17.509+0000 7fe103a2b740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fe0fc1ad860 con 0x55a580d08020 2024-09-10T09:44:17.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:17.509+0000 7fe103a2b740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fe0fc1aaa90 con 0x55a580cfce40 2024-09-10T09:44:17.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:17.509+0000 7fe103a2b740 10 client.0 _ll_register_callbacks cb 0x7fe0ec009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:44:17.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:17.509+0000 7fe0f9e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a580cfce40 0x55a580d239c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:44:17.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:17.510+0000 7fe0f9e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a580cfce40 0x55a580d239c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:39192/0 (socket says 172.21.15.5:39192) 2024-09-10T09:44:17.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:17.510+0000 7fe0f3e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a580cfd760 0x55a580d23f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:44:17.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:17.510+0000 7fe0f9e00640 1 -- 172.21.15.5:0/2084683306 learned_addr learned my addr 172.21.15.5:0/2084683306 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:44:17.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:17.510+0000 7fe0fa800640 1 --2- 172.21.15.5:0/2084683306 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a580d08020 0x55a580d24440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:44:17.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:17.510+0000 7fe0f9e00640 1 -- 172.21.15.5:0/2084683306 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a580d08020 msgr2=0x55a580d24440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:44:17.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:17.510+0000 7fe0f9e00640 1 --2- 172.21.15.5:0/2084683306 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a580d08020 0x55a580d24440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:17.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:17.510+0000 7fe0f9e00640 1 -- 172.21.15.5:0/2084683306 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a580cfd760 msgr2=0x55a580d23f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:44:17.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:17.510+0000 7fe0f9e00640 1 --2- 172.21.15.5:0/2084683306 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a580cfd760 0x55a580d23f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:17.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:17.510+0000 7fe0f9e00640 1 -- 172.21.15.5:0/2084683306 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a580cf8c80 con 0x55a580cfce40 2024-09-10T09:44:17.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:17.510+0000 7fe0f9e00640 1 --2- 172.21.15.5:0/2084683306 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a580cfce40 0x55a580d239c0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fe0ec0048f0 tx=0x7fe0ec001e50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9bbd21cc4cf9d5e8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:44:17.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:17.510+0000 7fe0dfe00640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-09-10T09:44:17.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:17.511+0000 7fe0dfe00640 1 -- 172.21.15.5:0/2084683306 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0ec052040 con 0x55a580cfce40 2024-09-10T09:44:17.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:17.511+0000 7fe0dfe00640 1 -- 172.21.15.5:0/2084683306 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fe0ec04a070 con 0x55a580cfce40 2024-09-10T09:44:17.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:17.511+0000 7fe103a2b740 1 -- 172.21.15.5:0/2084683306 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe0e4005500 con 0x55a580cfce40 2024-09-10T09:44:17.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:17.511+0000 7fe0dfe00640 1 -- 172.21.15.5:0/2084683306 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fe0ec065050 con 0x55a580cfce40 2024-09-10T09:44:17.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:17.511+0000 7fe103a2b740 10 client.26560 fetch_fsmap learned FSMap version 34 2024-09-10T09:44:17.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:17.511+0000 7fe103a2b740 1 -- 172.21.15.5:0/2084683306 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55a580d8b1d0 con 0x55a580cfce40 2024-09-10T09:44:17.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:17.511+0000 7fe0dfe00640 1 -- 172.21.15.5:0/2084683306 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe0e4005500 con 0x55a580cfce40 2024-09-10T09:44:17.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:17.511+0000 7fe0dfe00640 1 -- 172.21.15.5:0/2084683306 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7fe0ec0315b0 con 0x55a580cfce40 2024-09-10T09:44:17.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:17.511+0000 7fe103a2b740 10 client.26560 fetch_fsmap finished waiting for FSMap version 34 2024-09-10T09:44:17.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:17.511+0000 7fe103a2b740 10 client.26560 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-09-10T09:44:17.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:17.511+0000 7fe103a2b740 20 client.26560 populate_metadata read hostname 'smithi005' 2024-09-10T09:44:17.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:17.511+0000 7fe103a2b740 1 --2- 172.21.15.5:0/2084683306 >> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] conn(0x55a580d8b890 0x55a580d8dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:44:17.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:17.511+0000 7fe103a2b740 4 client.26560 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T09:44:17.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:17.511+0000 7fe103a2b740 1 -- 172.21.15.5:0/2084683306 --> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a580c6ae60 con 0x55a580d8b890 2024-09-10T09:44:17.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:17.512+0000 7fe0f3e00640 1 --2- 172.21.15.5:0/2084683306 >> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] conn(0x55a580d8b890 0x55a580d8dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:44:17.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:17.512+0000 7fe0f3e00640 1 --2- 172.21.15.5:0/2084683306 >> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] conn(0x55a580d8b890 0x55a580d8dcb0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=f630a26a8d2f40f3 server_cookie=650a2591e62a0071 in_seq=0 out_seq=0 2024-09-10T09:44:17.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:17.512+0000 7fe0dfe00640 10 client.26560 ms_handle_connect on v2:172.21.15.102:6836/2689776764 2024-09-10T09:44:17.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:17.512+0000 7fe0dfe00640 1 -- 172.21.15.5:0/2084683306 <== mds.3 v2:172.21.15.102:6836/2689776764 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a580c6ae60 con 0x55a580d8b890 2024-09-10T09:44:17.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:17.512+0000 7fe0dfe00640 10 client.26560 handle_command_reply: tid=0 2024-09-10T09:44:17.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:17.512+0000 7fe103a2b740 1 client.26560 shutdown 2024-09-10T09:44:17.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:17.513+0000 7fe103a2b740 1 -- 172.21.15.5:0/2084683306 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a580cfce40 msgr2=0x55a580d239c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:44:17.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:17.513+0000 7fe103a2b740 1 --2- 172.21.15.5:0/2084683306 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a580cfce40 0x55a580d239c0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fe0ec0048f0 tx=0x7fe0ec001e50 comp rx=0 tx=0).stop 2024-09-10T09:44:17.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:17.513+0000 7fe103a2b740 1 -- 172.21.15.5:0/2084683306 shutdown_connections 2024-09-10T09:44:17.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:17.513+0000 7fe103a2b740 1 --2- 172.21.15.5:0/2084683306 >> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] conn(0x55a580d8b890 0x55a580d8dcb0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:17.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:17.513+0000 7fe103a2b740 1 --2- 172.21.15.5:0/2084683306 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a580d08020 0x55a580d24440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:17.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:17.513+0000 7fe103a2b740 1 --2- 172.21.15.5:0/2084683306 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a580cfd760 0x55a580d23f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:17.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:17.513+0000 7fe103a2b740 1 --2- 172.21.15.5:0/2084683306 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a580cfce40 0x55a580d239c0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:17.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:17.513+0000 7fe0dfe00640 0 client.26560 ms_handle_reset on v2:172.21.15.102:6836/2689776764 2024-09-10T09:44:17.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:17.513+0000 7fe103a2b740 1 -- 172.21.15.5:0/2084683306 >> 172.21.15.5:0/2084683306 conn(0x55a580cf93d0 msgr2=0x55a580d1b850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:44:17.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:17.513+0000 7fe103a2b740 1 -- 172.21.15.5:0/2084683306 shutdown_connections 2024-09-10T09:44:17.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:17.513+0000 7fe103a2b740 1 -- 172.21.15.5:0/2084683306 wait complete. 2024-09-10T09:44:17.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:17.513+0000 7fe103a2b740 20 client.26560 trim_cache size 0 max 16384 2024-09-10T09:44:17.518 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-10T09:44:17.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:17.514+0000 7fe0fb200640 1 -- 172.21.15.5:0/2998622734 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fe0e00a2b80 msgr2=0x7fe0e00a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:44:17.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:17.514+0000 7fe0fb200640 1 --2- 172.21.15.5:0/2998622734 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fe0e00a2b80 0x7fe0e00a5040 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x55a580cfa750 tx=0x7fe0e40095e0 comp rx=0 tx=0).stop 2024-09-10T09:44:17.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:17.514+0000 7fe0fb200640 1 -- 172.21.15.5:0/2998622734 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe0fc108f60 msgr2=0x7fe0fc19ed80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:44:17.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:17.514+0000 7fe0fb200640 1 --2- 172.21.15.5:0/2998622734 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe0fc108f60 0x7fe0fc19ed80 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fe0f400eb10 tx=0x7fe0f400c2d0 comp rx=0 tx=0).stop 2024-09-10T09:44:17.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:17.514+0000 7fe0fb200640 1 -- 172.21.15.5:0/2998622734 shutdown_connections 2024-09-10T09:44:17.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:17.514+0000 7fe0fb200640 1 --2- 172.21.15.5:0/2998622734 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fe0e00a2b80 0x7fe0e00a5040 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:17.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:17.514+0000 7fe0fb200640 1 --2- 172.21.15.5:0/2998622734 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe0fc108f60 0x7fe0fc19ed80 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:17.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:17.514+0000 7fe0fb200640 1 --2- 172.21.15.5:0/2998622734 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe0fc1047e0 0x7fe0fc19a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:17.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:17.514+0000 7fe0fb200640 1 --2- 172.21.15.5:0/2998622734 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe0fc103e30 0x7fe0fc19a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:17.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:17.514+0000 7fe0fb200640 1 -- 172.21.15.5:0/2998622734 >> 172.21.15.5:0/2998622734 conn(0x7fe0fc0ff8d0 msgr2=0x7fe0fc100c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:44:17.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:17.514+0000 7fe0fb200640 1 -- 172.21.15.5:0/2998622734 shutdown_connections 2024-09-10T09:44:17.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:17.514+0000 7fe0fb200640 1 -- 172.21.15.5:0/2998622734 wait complete. 2024-09-10T09:44:18.043 INFO:tasks.workunit.client.0.smithi005.stdout: 1 326004 6.06 MB/sec execute 119 sec latency 13.638 ms 2024-09-10T09:44:18.411 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-10T09:44:18.820 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:44:19.043 INFO:tasks.workunit.client.0.smithi005.stdout: 1 327830 6.10 MB/sec execute 120 sec latency 26.619 ms 2024-09-10T09:44:19.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:44:18 smithi005 ceph-mon[30350]: pgmap v572: 129 pgs: 129 active+clean; 139 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.4 MiB/s wr, 191 op/s 2024-09-10T09:44:19.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:44:18 smithi159 ceph-mon[35513]: pgmap v572: 129 pgs: 129 active+clean; 139 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.4 MiB/s wr, 191 op/s 2024-09-10T09:44:19.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:44:18 smithi102 ceph-mon[35442]: pgmap v572: 129 pgs: 129 active+clean; 139 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.4 MiB/s wr, 191 op/s 2024-09-10T09:44:19.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:19.752+0000 7fd0b6a00640 1 -- 172.21.15.5:0/354747913 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd0b0100c80 msgr2=0x7fd0b0110890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:44:19.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:19.752+0000 7fd0aea00640 1 -- 172.21.15.5:0/354747913 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0a0035020 con 0x7fd0b0100c80 2024-09-10T09:44:19.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:19.752+0000 7fd0b6a00640 1 --2- 172.21.15.5:0/354747913 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd0b0100c80 0x7fd0b0110890 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fd0a0009900 tx=0x7fd0a002a400 comp rx=0 tx=0).stop 2024-09-10T09:44:19.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:19.752+0000 7fd0b6a00640 1 -- 172.21.15.5:0/354747913 shutdown_connections 2024-09-10T09:44:19.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:19.752+0000 7fd0b6a00640 1 --2- 172.21.15.5:0/354747913 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd0b0100c80 0x7fd0b0110890 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:19.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:19.752+0000 7fd0b6a00640 1 --2- 172.21.15.5:0/354747913 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd0b0100100 0x7fd0b0100520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:19.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:19.752+0000 7fd0b6a00640 1 --2- 172.21.15.5:0/354747913 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd0b0102a50 0x7fd0b0102e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:19.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:19.752+0000 7fd0b6a00640 1 -- 172.21.15.5:0/354747913 >> 172.21.15.5:0/354747913 conn(0x7fd0b0077230 msgr2=0x7fd0b00fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:44:19.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:19.753+0000 7fd0b6a00640 1 -- 172.21.15.5:0/354747913 shutdown_connections 2024-09-10T09:44:19.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:19.753+0000 7fd0b6a00640 1 -- 172.21.15.5:0/354747913 wait complete. 2024-09-10T09:44:19.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:19.757+0000 7fd0b6a00640 1 Processor -- start 2024-09-10T09:44:19.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:19.757+0000 7fd0b6a00640 1 -- start start 2024-09-10T09:44:19.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:19.757+0000 7fd0b6a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd0b0100100 0x7fd0b019a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:44:19.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:19.758+0000 7fd0b6a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd0b0100c80 0x7fd0b019a930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:44:19.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:19.758+0000 7fd0b6a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd0b0102a50 0x7fd0b019ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:44:19.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:19.758+0000 7fd0b6a00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd0b0114150 con 0x7fd0b0102a50 2024-09-10T09:44:19.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:19.758+0000 7fd0b6a00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fd0b0113fd0 con 0x7fd0b0100c80 2024-09-10T09:44:19.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:19.758+0000 7fd0b6a00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fd0b01142d0 con 0x7fd0b0100100 2024-09-10T09:44:19.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:19.758+0000 7fd0afe00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd0b0100100 0x7fd0b019a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:44:19.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:19.758+0000 7fd0afe00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd0b0100100 0x7fd0b019a3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:39202/0 (socket says 172.21.15.5:39202) 2024-09-10T09:44:19.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:19.758+0000 7fd0afe00640 1 -- 172.21.15.5:0/4129409976 learned_addr learned my addr 172.21.15.5:0/4129409976 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:44:19.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:19.759+0000 7fd0b4a00640 1 --2- 172.21.15.5:0/4129409976 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd0b0102a50 0x7fd0b019ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:44:19.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:19.759+0000 7fd0afe00640 1 -- 172.21.15.5:0/4129409976 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd0b0100c80 msgr2=0x7fd0b019a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:44:19.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:19.759+0000 7fd0afe00640 1 --2- 172.21.15.5:0/4129409976 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd0b0100c80 0x7fd0b019a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:19.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:19.759+0000 7fd0afe00640 1 -- 172.21.15.5:0/4129409976 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd0b0102a50 msgr2=0x7fd0b019ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:44:19.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:19.759+0000 7fd0afe00640 1 --2- 172.21.15.5:0/4129409976 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd0b0102a50 0x7fd0b019ecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:19.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:19.759+0000 7fd0afe00640 1 -- 172.21.15.5:0/4129409976 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0b019f3a0 con 0x7fd0b0100100 2024-09-10T09:44:19.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:19.761+0000 7fd0afe00640 1 --2- 172.21.15.5:0/4129409976 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd0b0100100 0x7fd0b019a3f0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fd0a800db80 tx=0x7fd0a800b630 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:44:19.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:19.762+0000 7fd0acc00640 1 -- 172.21.15.5:0/4129409976 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0a8017070 con 0x7fd0b0100100 2024-09-10T09:44:19.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:19.762+0000 7fd0b6a00640 1 -- 172.21.15.5:0/4129409976 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0b019f690 con 0x7fd0b0100100 2024-09-10T09:44:19.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:19.762+0000 7fd0b6a00640 1 -- 172.21.15.5:0/4129409976 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0b01a6f70 con 0x7fd0b0100100 2024-09-10T09:44:19.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:19.762+0000 7fd0acc00640 1 -- 172.21.15.5:0/4129409976 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd0a8010940 con 0x7fd0b0100100 2024-09-10T09:44:19.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:19.762+0000 7fd0acc00640 1 -- 172.21.15.5:0/4129409976 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0a8017070 con 0x7fd0b0100100 2024-09-10T09:44:19.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:19.769+0000 7fd0acc00640 1 -- 172.21.15.5:0/4129409976 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7fd0a8010460 con 0x7fd0b0100100 2024-09-10T09:44:19.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:19.775+0000 7fd0acc00640 1 --2- 172.21.15.5:0/4129409976 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fd0940a2e40 0x7fd0940a5300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:44:19.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:19.775+0000 7fd0acc00640 1 -- 172.21.15.5:0/4129409976 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fd0a80ceb90 con 0x7fd0b0100100 2024-09-10T09:44:19.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:19.778+0000 7fd0af400640 1 --2- 172.21.15.5:0/4129409976 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fd0940a2e40 0x7fd0940a5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:44:19.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:19.780+0000 7fd0af400640 1 --2- 172.21.15.5:0/4129409976 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fd0940a2e40 0x7fd0940a5300 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fd0b00fe8a0 tx=0x7fd09c005bf0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:44:19.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:19.782+0000 7fd0b94c5740 1 Processor -- start 2024-09-10T09:44:19.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:19.783+0000 7fd0b94c5740 1 -- start start 2024-09-10T09:44:19.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:19.784+0000 7fd0b94c5740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55633f4f1480 0x55633f4f1860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:44:19.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:19.784+0000 7fd0b94c5740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55633f4f1da0 0x55633f4fc120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:44:19.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:19.784+0000 7fd0b94c5740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55633f4fc660 0x55633f4fea50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:44:19.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:19.784+0000 7fd0b94c5740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55633f44f3e0 con 0x55633f4fc660 2024-09-10T09:44:19.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:19.784+0000 7fd0b94c5740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55633f3f6d80 con 0x55633f4f1480 2024-09-10T09:44:19.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:19.784+0000 7fd0b94c5740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55633f44b3a0 con 0x55633f4f1da0 2024-09-10T09:44:19.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:19.785+0000 7fd0afe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55633f4f1480 0x55633f4f1860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:44:19.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:19.785+0000 7fd0afe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55633f4f1480 0x55633f4f1860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:38860/0 (socket says 172.21.15.5:38860) 2024-09-10T09:44:19.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:19.785+0000 7fd0afe00640 1 -- 172.21.15.5:0/698461739 learned_addr learned my addr 172.21.15.5:0/698461739 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:44:19.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:19.785+0000 7fd0b4a00640 1 --2- 172.21.15.5:0/698461739 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55633f4fc660 0x55633f4fea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:44:19.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:19.785+0000 7fd0af400640 1 --2- 172.21.15.5:0/698461739 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55633f4f1da0 0x55633f4fc120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:44:19.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:19.786+0000 7fd0af400640 1 -- 172.21.15.5:0/698461739 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55633f4f1480 msgr2=0x55633f4f1860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:44:19.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:19.786+0000 7fd0af400640 1 --2- 172.21.15.5:0/698461739 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55633f4f1480 0x55633f4f1860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:19.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:19.786+0000 7fd0af400640 1 -- 172.21.15.5:0/698461739 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55633f4fc660 msgr2=0x55633f4fea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:44:19.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:19.786+0000 7fd0af400640 1 --2- 172.21.15.5:0/698461739 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55633f4fc660 0x55633f4fea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:19.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:19.786+0000 7fd0af400640 1 -- 172.21.15.5:0/698461739 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55633f415ae0 con 0x55633f4f1da0 2024-09-10T09:44:19.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:19.786+0000 7fd0afe00640 1 --2- 172.21.15.5:0/698461739 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55633f4f1480 0x55633f4f1860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:44:19.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:19.786+0000 7fd0b4a00640 1 --2- 172.21.15.5:0/698461739 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55633f4fc660 0x55633f4fea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:44:19.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:19.787+0000 7fd0af400640 1 --2- 172.21.15.5:0/698461739 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55633f4f1da0 0x55633f4fc120 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fd09c00cc70 tx=0x7fd09c00f000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=590e7799256380d2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:44:19.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:19.787+0000 7fd0a5200640 1 -- 172.21.15.5:0/698461739 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd09c020070 con 0x55633f4f1da0 2024-09-10T09:44:19.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:19.787+0000 7fd0a5200640 1 -- 172.21.15.5:0/698461739 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd09c016070 con 0x55633f4f1da0 2024-09-10T09:44:19.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:19.787+0000 7fd0a5200640 1 -- 172.21.15.5:0/698461739 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd09c01c650 con 0x55633f4f1da0 2024-09-10T09:44:19.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:19.789+0000 7fd0b94c5740 1 -- 172.21.15.5:0/698461739 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55633f4f1da0 msgr2=0x55633f4fc120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:44:19.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:19.789+0000 7fd0b94c5740 1 --2- 172.21.15.5:0/698461739 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55633f4f1da0 0x55633f4fc120 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fd09c00cc70 tx=0x7fd09c00f000 comp rx=0 tx=0).stop 2024-09-10T09:44:19.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:19.790+0000 7fd0b94c5740 1 -- 172.21.15.5:0/698461739 shutdown_connections 2024-09-10T09:44:19.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:19.790+0000 7fd0b94c5740 1 --2- 172.21.15.5:0/698461739 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55633f4fc660 0x55633f4fea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:19.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:19.790+0000 7fd0b94c5740 1 --2- 172.21.15.5:0/698461739 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55633f4f1da0 0x55633f4fc120 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:19.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:19.790+0000 7fd0b94c5740 1 --2- 172.21.15.5:0/698461739 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55633f4f1480 0x55633f4f1860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:19.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:19.790+0000 7fd0b94c5740 1 -- 172.21.15.5:0/698461739 >> 172.21.15.5:0/698461739 conn(0x55633f4eda10 msgr2=0x55633f4eddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:44:19.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:19.791+0000 7fd0b94c5740 1 -- 172.21.15.5:0/698461739 shutdown_connections 2024-09-10T09:44:19.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:19.791+0000 7fd0b94c5740 1 -- 172.21.15.5:0/698461739 wait complete. 2024-09-10T09:44:19.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:19.792+0000 7fd0b94c5740 1 -- start start 2024-09-10T09:44:19.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:19.792+0000 7fd0b94c5740 1 Processor -- start 2024-09-10T09:44:19.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:19.795+0000 7fd0b94c5740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55633f4f1480 0x55633f4f9540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:44:19.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:19.795+0000 7fd0b94c5740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55633f4f1da0 0x55633f4f9a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:44:19.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:19.795+0000 7fd0b94c5740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55633f4fc660 0x55633f4f9fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:44:19.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:19.795+0000 7fd0b94c5740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55633f4ff230 con 0x55633f4f1da0 2024-09-10T09:44:19.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:19.795+0000 7fd0b94c5740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55633f40a0f0 con 0x55633f4f1480 2024-09-10T09:44:19.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:19.795+0000 7fd0b94c5740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55633f4ff3b0 con 0x55633f4fc660 2024-09-10T09:44:19.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:19.795+0000 7fd0b94c5740 10 client.0 _ll_register_callbacks cb 0x55633f3f1bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:44:19.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:19.796+0000 7fd0af400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55633f4f1da0 0x55633f4f9a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:44:19.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:19.796+0000 7fd0af400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55633f4f1da0 0x55633f4f9a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37770/0 (socket says 172.21.15.5:37770) 2024-09-10T09:44:19.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:19.796+0000 7fd0af400640 1 -- 172.21.15.5:0/1154380497 learned_addr learned my addr 172.21.15.5:0/1154380497 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:44:19.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:19.796+0000 7fd0b4a00640 1 --2- 172.21.15.5:0/1154380497 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55633f4fc660 0x55633f4f9fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:44:19.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:19.796+0000 7fd0afe00640 1 --2- 172.21.15.5:0/1154380497 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55633f4f1480 0x55633f4f9540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:44:19.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:19.796+0000 7fd0afe00640 1 -- 172.21.15.5:0/1154380497 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55633f4fc660 msgr2=0x55633f4f9fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:44:19.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:19.796+0000 7fd0afe00640 1 --2- 172.21.15.5:0/1154380497 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55633f4fc660 0x55633f4f9fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:19.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:19.796+0000 7fd0afe00640 1 -- 172.21.15.5:0/1154380497 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55633f4f1da0 msgr2=0x55633f4f9a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:44:19.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:19.796+0000 7fd0afe00640 1 --2- 172.21.15.5:0/1154380497 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55633f4f1da0 0x55633f4f9a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:19.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:19.796+0000 7fd0afe00640 1 -- 172.21.15.5:0/1154380497 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55633f1c03c0 con 0x55633f4f1480 2024-09-10T09:44:19.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:19.797+0000 7fd0af400640 1 --2- 172.21.15.5:0/1154380497 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55633f4f1da0 0x55633f4f9a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:44:19.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:19.797+0000 7fd0afe00640 1 --2- 172.21.15.5:0/1154380497 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55633f4f1480 0x55633f4f9540 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fd0a800db50 tx=0x7fd0a8010af0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4b2ba1080e04f804 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:44:19.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:19.797+0000 7fd099600640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-09-10T09:44:19.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:19.797+0000 7fd099600640 1 -- 172.21.15.5:0/1154380497 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0a809f3b0 con 0x55633f4f1480 2024-09-10T09:44:19.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:19.797+0000 7fd099600640 1 -- 172.21.15.5:0/1154380497 <== mon.2 v2:172.21.15.102:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fd0a8097670 con 0x55633f4f1480 2024-09-10T09:44:19.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:19.797+0000 7fd0b94c5740 1 -- 172.21.15.5:0/1154380497 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd09c016070 con 0x55633f4f1480 2024-09-10T09:44:19.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:19.806+0000 7fd099600640 1 -- 172.21.15.5:0/1154380497 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fd0a80a3c70 con 0x55633f4f1480 2024-09-10T09:44:19.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:19.806+0000 7fd0b94c5740 10 client.26537 fetch_fsmap learned FSMap version 34 2024-09-10T09:44:19.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:19.806+0000 7fd0b94c5740 1 -- 172.21.15.5:0/1154380497 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55633f51c310 con 0x55633f4f1480 2024-09-10T09:44:19.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:19.807+0000 7fd099600640 1 -- 172.21.15.5:0/1154380497 <== mon.2 v2:172.21.15.102:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd09c016070 con 0x55633f4f1480 2024-09-10T09:44:19.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:19.807+0000 7fd099600640 1 -- 172.21.15.5:0/1154380497 <== mon.2 v2:172.21.15.102:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7fd0a80b7060 con 0x55633f4f1480 2024-09-10T09:44:19.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:19.807+0000 7fd0b94c5740 10 client.26537 fetch_fsmap finished waiting for FSMap version 34 2024-09-10T09:44:19.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:19.807+0000 7fd0b94c5740 10 client.26537 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-10T09:44:19.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:19.807+0000 7fd0b94c5740 20 client.26537 populate_metadata read hostname 'smithi005' 2024-09-10T09:44:19.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:19.807+0000 7fd0b94c5740 1 --2- 172.21.15.5:0/1154380497 >> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] conn(0x55633f51d770 0x55633f51fb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:44:19.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:19.808+0000 7fd0b94c5740 4 client.26537 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:44:19.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:19.808+0000 7fd0b94c5740 1 -- 172.21.15.5:0/1154380497 --> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55633f45c950 con 0x55633f51d770 2024-09-10T09:44:19.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:19.808+0000 7fd0af400640 1 --2- 172.21.15.5:0/1154380497 >> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] conn(0x55633f51d770 0x55633f51fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:44:19.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:19.808+0000 7fd0af400640 1 --2- 172.21.15.5:0/1154380497 >> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] conn(0x55633f51d770 0x55633f51fb90 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=89130171d93dd774 server_cookie=ebbf3927ff89c184 in_seq=0 out_seq=0 2024-09-10T09:44:19.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:19.808+0000 7fd099600640 10 client.26537 ms_handle_connect on v2:172.21.15.159:6836/620627639 2024-09-10T09:44:19.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:19.809+0000 7fd099600640 1 -- 172.21.15.5:0/1154380497 <== mds.2 v2:172.21.15.159:6836/620627639 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55633f45c950 con 0x55633f51d770 2024-09-10T09:44:19.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:19.809+0000 7fd099600640 10 client.26537 handle_command_reply: tid=0 2024-09-10T09:44:19.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:19.809+0000 7fd0b94c5740 1 client.26537 shutdown 2024-09-10T09:44:19.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:19.810+0000 7fd0b94c5740 1 -- 172.21.15.5:0/1154380497 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55633f4f1480 msgr2=0x55633f4f9540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:44:19.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:19.810+0000 7fd0b94c5740 1 --2- 172.21.15.5:0/1154380497 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55633f4f1480 0x55633f4f9540 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fd0a800db50 tx=0x7fd0a8010af0 comp rx=0 tx=0).stop 2024-09-10T09:44:19.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:19.810+0000 7fd0b94c5740 1 -- 172.21.15.5:0/1154380497 shutdown_connections 2024-09-10T09:44:19.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:19.810+0000 7fd0b94c5740 1 --2- 172.21.15.5:0/1154380497 >> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] conn(0x55633f51d770 0x55633f51fb90 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:19.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:19.810+0000 7fd0b94c5740 1 --2- 172.21.15.5:0/1154380497 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55633f4fc660 0x55633f4f9fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:19.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:19.810+0000 7fd0b94c5740 1 --2- 172.21.15.5:0/1154380497 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55633f4f1da0 0x55633f4f9a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:19.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:19.810+0000 7fd0b94c5740 1 --2- 172.21.15.5:0/1154380497 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55633f4f1480 0x55633f4f9540 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:19.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:19.810+0000 7fd0b94c5740 1 -- 172.21.15.5:0/1154380497 >> 172.21.15.5:0/1154380497 conn(0x55633f4eda10 msgr2=0x55633f4e4380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:44:19.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:19.810+0000 7fd099600640 0 client.26537 ms_handle_reset on v2:172.21.15.159:6836/620627639 2024-09-10T09:44:19.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:19.810+0000 7fd0b94c5740 1 -- 172.21.15.5:0/1154380497 shutdown_connections 2024-09-10T09:44:19.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:19.810+0000 7fd0b94c5740 1 -- 172.21.15.5:0/1154380497 wait complete. 2024-09-10T09:44:19.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:19.810+0000 7fd0b94c5740 20 client.26537 trim_cache size 0 max 16384 2024-09-10T09:44:19.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:19.826+0000 7fd0b94c5740 1 Processor -- start 2024-09-10T09:44:19.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:19.826+0000 7fd0b94c5740 1 -- start start 2024-09-10T09:44:19.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:19.826+0000 7fd0b94c5740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55633f4f1480 0x55633f4f9e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:44:19.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:19.826+0000 7fd0b94c5740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55633f4f1da0 0x55633f5254a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:44:19.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:19.826+0000 7fd0b94c5740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55633f4fc660 0x55633f5259e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:44:19.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:19.826+0000 7fd0b94c5740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd09000fe20 con 0x55633f4fc660 2024-09-10T09:44:19.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:19.826+0000 7fd0b94c5740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fd090011960 con 0x55633f4f1da0 2024-09-10T09:44:19.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:19.826+0000 7fd0b94c5740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fd0900104f0 con 0x55633f4f1480 2024-09-10T09:44:19.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:19.827+0000 7fd0af400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55633f4f1da0 0x55633f5254a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:44:19.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:19.827+0000 7fd0af400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55633f4f1da0 0x55633f5254a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:38872/0 (socket says 172.21.15.5:38872) 2024-09-10T09:44:19.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:19.827+0000 7fd0af400640 1 -- 172.21.15.5:0/1245124421 learned_addr learned my addr 172.21.15.5:0/1245124421 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:44:19.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:19.827+0000 7fd0afe00640 1 --2- 172.21.15.5:0/1245124421 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55633f4f1480 0x55633f4f9e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:44:19.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:19.827+0000 7fd0b4a00640 1 --2- 172.21.15.5:0/1245124421 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55633f4fc660 0x55633f5259e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:44:19.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:19.827+0000 7fd0af400640 1 -- 172.21.15.5:0/1245124421 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55633f4f1480 msgr2=0x55633f4f9e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:44:19.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:19.827+0000 7fd0af400640 1 --2- 172.21.15.5:0/1245124421 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55633f4f1480 0x55633f4f9e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:19.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:19.827+0000 7fd0af400640 1 -- 172.21.15.5:0/1245124421 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55633f4fc660 msgr2=0x55633f5259e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:44:19.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:19.827+0000 7fd0af400640 1 --2- 172.21.15.5:0/1245124421 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55633f4fc660 0x55633f5259e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:19.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:19.827+0000 7fd0af400640 1 -- 172.21.15.5:0/1245124421 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55633f4ee350 con 0x55633f4f1da0 2024-09-10T09:44:19.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:19.827+0000 7fd0af400640 1 --2- 172.21.15.5:0/1245124421 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55633f4f1da0 0x55633f5254a0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fd09c004700 tx=0x7fd09c00c7f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=78d844e519c6dd65 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:44:19.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:19.828+0000 7fd09be00640 1 -- 172.21.15.5:0/1245124421 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd09c020040 con 0x55633f4f1da0 2024-09-10T09:44:19.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:19.828+0000 7fd09be00640 1 -- 172.21.15.5:0/1245124421 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd09c016040 con 0x55633f4f1da0 2024-09-10T09:44:19.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:19.828+0000 7fd09be00640 1 -- 172.21.15.5:0/1245124421 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd09c024400 con 0x55633f4f1da0 2024-09-10T09:44:19.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:19.829+0000 7fd0b94c5740 1 -- 172.21.15.5:0/1245124421 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55633f4f1da0 msgr2=0x55633f5254a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:44:19.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:19.829+0000 7fd0b94c5740 1 --2- 172.21.15.5:0/1245124421 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55633f4f1da0 0x55633f5254a0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fd09c004700 tx=0x7fd09c00c7f0 comp rx=0 tx=0).stop 2024-09-10T09:44:19.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:19.829+0000 7fd0b94c5740 1 -- 172.21.15.5:0/1245124421 shutdown_connections 2024-09-10T09:44:19.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:19.829+0000 7fd0b94c5740 1 --2- 172.21.15.5:0/1245124421 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55633f4fc660 0x55633f5259e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:19.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:19.829+0000 7fd0b94c5740 1 --2- 172.21.15.5:0/1245124421 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55633f4f1da0 0x55633f5254a0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:19.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:19.829+0000 7fd0b94c5740 1 --2- 172.21.15.5:0/1245124421 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55633f4f1480 0x55633f4f9e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:19.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:19.829+0000 7fd0b94c5740 1 -- 172.21.15.5:0/1245124421 >> 172.21.15.5:0/1245124421 conn(0x55633f4eda10 msgr2=0x55633f4f36f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:44:19.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:19.829+0000 7fd0b94c5740 1 -- 172.21.15.5:0/1245124421 shutdown_connections 2024-09-10T09:44:19.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:19.829+0000 7fd0b94c5740 1 -- 172.21.15.5:0/1245124421 wait complete. 2024-09-10T09:44:19.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:19.829+0000 7fd0b94c5740 1 -- start start 2024-09-10T09:44:19.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:19.830+0000 7fd0b94c5740 1 Processor -- start 2024-09-10T09:44:19.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:19.830+0000 7fd0b94c5740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55633f4f1480 0x55633f504950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:44:19.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:19.830+0000 7fd0b94c5740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55633f4f1da0 0x55633f504e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:44:19.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:19.830+0000 7fd0b94c5740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55633f4fc660 0x55633f4ffd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:44:19.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:19.830+0000 7fd0b94c5740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd090011290 con 0x55633f4fc660 2024-09-10T09:44:19.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:19.830+0000 7fd0b94c5740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fd090010bc0 con 0x55633f4f1da0 2024-09-10T09:44:19.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:19.830+0000 7fd0b94c5740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fd09000de00 con 0x55633f4f1480 2024-09-10T09:44:19.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:19.830+0000 7fd0b94c5740 10 client.0 _ll_register_callbacks cb 0x7fd0a8096be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:44:19.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:19.830+0000 7fd0af400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55633f4f1da0 0x55633f504e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:44:19.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:19.830+0000 7fd0afe00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55633f4f1480 0x55633f504950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:44:19.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:19.830+0000 7fd0af400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55633f4f1da0 0x55633f504e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:38874/0 (socket says 172.21.15.5:38874) 2024-09-10T09:44:19.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:19.830+0000 7fd0af400640 1 -- 172.21.15.5:0/1078383186 learned_addr learned my addr 172.21.15.5:0/1078383186 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:44:19.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:19.831+0000 7fd0b4a00640 1 --2- 172.21.15.5:0/1078383186 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55633f4fc660 0x55633f4ffd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:44:19.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:19.831+0000 7fd0afe00640 1 -- 172.21.15.5:0/1078383186 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55633f4f1da0 msgr2=0x55633f504e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:44:19.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:19.831+0000 7fd0afe00640 1 --2- 172.21.15.5:0/1078383186 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55633f4f1da0 0x55633f504e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:19.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:19.831+0000 7fd0afe00640 1 -- 172.21.15.5:0/1078383186 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55633f4fc660 msgr2=0x55633f4ffd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:44:19.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:19.831+0000 7fd0afe00640 1 --2- 172.21.15.5:0/1078383186 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55633f4fc660 0x55633f4ffd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:19.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:19.831+0000 7fd0afe00640 1 -- 172.21.15.5:0/1078383186 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55633f500540 con 0x55633f4f1480 2024-09-10T09:44:19.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:19.831+0000 7fd0afe00640 1 --2- 172.21.15.5:0/1078383186 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55633f4f1480 0x55633f504950 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fd0a800dfd0 tx=0x7fd0a8009310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a73586adb9666f5b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:44:19.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:19.831+0000 7fd099600640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-09-10T09:44:19.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:19.831+0000 7fd099600640 1 -- 172.21.15.5:0/1078383186 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0a809f3b0 con 0x55633f4f1480 2024-09-10T09:44:19.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:19.831+0000 7fd0b94c5740 1 -- 172.21.15.5:0/1078383186 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd09c016040 con 0x55633f4f1480 2024-09-10T09:44:19.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:19.831+0000 7fd099600640 1 -- 172.21.15.5:0/1078383186 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fd0a8099300 con 0x55633f4f1480 2024-09-10T09:44:19.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:19.832+0000 7fd099600640 1 -- 172.21.15.5:0/1078383186 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fd0a809ab40 con 0x55633f4f1480 2024-09-10T09:44:19.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:19.832+0000 7fd0b94c5740 10 client.26578 fetch_fsmap learned FSMap version 34 2024-09-10T09:44:19.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:19.832+0000 7fd0b94c5740 1 -- 172.21.15.5:0/1078383186 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55633f587fd0 con 0x55633f4f1480 2024-09-10T09:44:19.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:19.832+0000 7fd099600640 1 -- 172.21.15.5:0/1078383186 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd09c016040 con 0x55633f4f1480 2024-09-10T09:44:19.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:19.832+0000 7fd099600640 1 -- 172.21.15.5:0/1078383186 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7fd0a80b7040 con 0x55633f4f1480 2024-09-10T09:44:19.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:19.832+0000 7fd0b94c5740 10 client.26578 fetch_fsmap finished waiting for FSMap version 34 2024-09-10T09:44:19.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:19.832+0000 7fd0b94c5740 10 client.26578 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-10T09:44:19.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:19.832+0000 7fd0b94c5740 20 client.26578 populate_metadata read hostname 'smithi005' 2024-09-10T09:44:19.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:19.832+0000 7fd0b94c5740 1 --2- 172.21.15.5:0/1078383186 >> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] conn(0x55633f5887a0 0x55633f58abc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:44:19.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:19.832+0000 7fd0b94c5740 4 client.26578 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T09:44:19.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:19.832+0000 7fd0b94c5740 1 -- 172.21.15.5:0/1078383186 --> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55633f456e60 con 0x55633f5887a0 2024-09-10T09:44:19.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:19.833+0000 7fd0af400640 1 --2- 172.21.15.5:0/1078383186 >> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] conn(0x55633f5887a0 0x55633f58abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:44:19.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:19.833+0000 7fd0af400640 1 --2- 172.21.15.5:0/1078383186 >> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] conn(0x55633f5887a0 0x55633f58abc0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9d85bf9a79b7df48 server_cookie=312a3e2fd4093f9b in_seq=0 out_seq=0 2024-09-10T09:44:19.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:19.833+0000 7fd099600640 10 client.26578 ms_handle_connect on v2:172.21.15.159:6836/620627639 2024-09-10T09:44:19.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:19.833+0000 7fd099600640 1 -- 172.21.15.5:0/1078383186 <== mds.2 v2:172.21.15.159:6836/620627639 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55633f456e60 con 0x55633f5887a0 2024-09-10T09:44:19.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:19.833+0000 7fd099600640 10 client.26578 handle_command_reply: tid=0 2024-09-10T09:44:19.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:19.833+0000 7fd0b94c5740 1 client.26578 shutdown 2024-09-10T09:44:19.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:19.834+0000 7fd0b94c5740 1 -- 172.21.15.5:0/1078383186 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55633f4f1480 msgr2=0x55633f504950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:44:19.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:19.834+0000 7fd0b94c5740 1 --2- 172.21.15.5:0/1078383186 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55633f4f1480 0x55633f504950 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fd0a800dfd0 tx=0x7fd0a8009310 comp rx=0 tx=0).stop 2024-09-10T09:44:19.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:19.834+0000 7fd0b94c5740 1 -- 172.21.15.5:0/1078383186 shutdown_connections 2024-09-10T09:44:19.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:19.834+0000 7fd0b94c5740 1 --2- 172.21.15.5:0/1078383186 >> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] conn(0x55633f5887a0 0x55633f58abc0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:19.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:19.834+0000 7fd0b94c5740 1 --2- 172.21.15.5:0/1078383186 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55633f4fc660 0x55633f4ffd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:19.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:19.834+0000 7fd0b94c5740 1 --2- 172.21.15.5:0/1078383186 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55633f4f1da0 0x55633f504e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:19.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:19.834+0000 7fd0b94c5740 1 --2- 172.21.15.5:0/1078383186 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55633f4f1480 0x55633f504950 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:19.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:19.834+0000 7fd0b94c5740 1 -- 172.21.15.5:0/1078383186 >> 172.21.15.5:0/1078383186 conn(0x55633f4eda10 msgr2=0x55633f4e3b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:44:19.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:19.835+0000 7fd099600640 0 client.26578 ms_handle_reset on v2:172.21.15.159:6836/620627639 2024-09-10T09:44:19.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:19.835+0000 7fd0b94c5740 1 -- 172.21.15.5:0/1078383186 shutdown_connections 2024-09-10T09:44:19.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:19.835+0000 7fd0b94c5740 1 -- 172.21.15.5:0/1078383186 wait complete. 2024-09-10T09:44:19.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:19.835+0000 7fd0b94c5740 20 client.26578 trim_cache size 0 max 16384 2024-09-10T09:44:19.839 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-10T09:44:19.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:19.836+0000 7fd0b6a00640 1 -- 172.21.15.5:0/4129409976 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fd0940a2e40 msgr2=0x7fd0940a5300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:44:19.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:19.836+0000 7fd0b6a00640 1 --2- 172.21.15.5:0/4129409976 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fd0940a2e40 0x7fd0940a5300 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fd0b00fe8a0 tx=0x7fd09c005bf0 comp rx=0 tx=0).stop 2024-09-10T09:44:19.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:19.836+0000 7fd0b6a00640 1 -- 172.21.15.5:0/4129409976 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd0b0100100 msgr2=0x7fd0b019a3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:44:19.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:19.836+0000 7fd0b6a00640 1 --2- 172.21.15.5:0/4129409976 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd0b0100100 0x7fd0b019a3f0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fd0a800db80 tx=0x7fd0a800b630 comp rx=0 tx=0).stop 2024-09-10T09:44:19.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:19.836+0000 7fd0b6a00640 1 -- 172.21.15.5:0/4129409976 shutdown_connections 2024-09-10T09:44:19.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:19.836+0000 7fd0b6a00640 1 --2- 172.21.15.5:0/4129409976 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fd0940a2e40 0x7fd0940a5300 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:19.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:19.836+0000 7fd0b6a00640 1 --2- 172.21.15.5:0/4129409976 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd0b0102a50 0x7fd0b019ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:19.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:19.836+0000 7fd0b6a00640 1 --2- 172.21.15.5:0/4129409976 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd0b0100c80 0x7fd0b019a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:19.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:19.836+0000 7fd0b6a00640 1 --2- 172.21.15.5:0/4129409976 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd0b0100100 0x7fd0b019a3f0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:19.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:19.836+0000 7fd0b6a00640 1 -- 172.21.15.5:0/4129409976 >> 172.21.15.5:0/4129409976 conn(0x7fd0b0077230 msgr2=0x7fd0b00fd4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:44:19.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:19.836+0000 7fd0b6a00640 1 -- 172.21.15.5:0/4129409976 shutdown_connections 2024-09-10T09:44:19.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:19.837+0000 7fd0b6a00640 1 -- 172.21.15.5:0/4129409976 wait complete. 2024-09-10T09:44:20.043 INFO:tasks.workunit.client.0.smithi005.stdout: 1 329676 6.12 MB/sec execute 121 sec latency 244.607 ms 2024-09-10T09:44:20.904 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:44:20 smithi005 ceph-mon[30350]: pgmap v573: 129 pgs: 129 active+clean; 139 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.2 MiB/s wr, 174 op/s 2024-09-10T09:44:21.043 INFO:tasks.workunit.client.0.smithi005.stdout: 1 329676 6.07 MB/sec execute 122 sec latency 1244.653 ms 2024-09-10T09:44:21.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:44:20 smithi159 ceph-mon[35513]: pgmap v573: 129 pgs: 129 active+clean; 139 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.2 MiB/s wr, 174 op/s 2024-09-10T09:44:21.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:44:20 smithi102 ceph-mon[35442]: pgmap v573: 129 pgs: 129 active+clean; 139 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.2 MiB/s wr, 174 op/s 2024-09-10T09:44:21.622 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-10T09:44:22.043 INFO:tasks.workunit.client.0.smithi005.stdout: 1 330630 6.06 MB/sec execute 123 sec latency 1513.867 ms 2024-09-10T09:44:22.065 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:44:22.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:22.806+0000 7f7889600640 1 -- 172.21.15.5:0/3692777744 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7884100720 msgr2=0x7f7884100b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:44:22.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:22.806+0000 7f7889600640 1 --2- 172.21.15.5:0/3692777744 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7884100720 0x7f7884100b00 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f7874009900 tx=0x7f787402a3f0 comp rx=0 tx=0).stop 2024-09-10T09:44:22.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:22.806+0000 7f7889600640 1 -- 172.21.15.5:0/3692777744 shutdown_connections 2024-09-10T09:44:22.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:22.806+0000 7f7889600640 1 --2- 172.21.15.5:0/3692777744 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f788410e2d0 0x7f78841106c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:22.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:22.806+0000 7f7889600640 1 --2- 172.21.15.5:0/3692777744 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f78841010d0 0x7f788410dd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:22.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:22.806+0000 7f7889600640 1 --2- 172.21.15.5:0/3692777744 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7884100720 0x7f7884100b00 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:22.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:22.806+0000 7f7889600640 1 -- 172.21.15.5:0/3692777744 >> 172.21.15.5:0/3692777744 conn(0x7f78840fc630 msgr2=0x7f78840fea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:44:22.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:22.806+0000 7f7889600640 1 -- 172.21.15.5:0/3692777744 shutdown_connections 2024-09-10T09:44:22.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:22.806+0000 7f7889600640 1 -- 172.21.15.5:0/3692777744 wait complete. 2024-09-10T09:44:22.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:22.807+0000 7f7889600640 1 Processor -- start 2024-09-10T09:44:22.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:22.807+0000 7f7889600640 1 -- start start 2024-09-10T09:44:22.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:22.808+0000 7f7889600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7884100720 0x7f788419a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:44:22.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:22.808+0000 7f7889600640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f78841010d0 0x7f788419a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:44:22.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:22.808+0000 7f7889600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f788410e2d0 0x7f788419ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:44:22.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:22.808+0000 7f7889600640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f7884114040 con 0x7f7884100720 2024-09-10T09:44:22.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:22.808+0000 7f7889600640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f7884113ec0 con 0x7f788410e2d0 2024-09-10T09:44:22.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:22.808+0000 7f7889600640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f78841141c0 con 0x7f78841010d0 2024-09-10T09:44:22.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:22.808+0000 7f7883400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f78841010d0 0x7f788419a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:44:22.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:22.808+0000 7f7883400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f78841010d0 0x7f788419a9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:60182/0 (socket says 172.21.15.5:60182) 2024-09-10T09:44:22.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:22.808+0000 7f7883400640 1 -- 172.21.15.5:0/3159578559 learned_addr learned my addr 172.21.15.5:0/3159578559 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:44:22.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:22.808+0000 7f7883e00640 1 --2- 172.21.15.5:0/3159578559 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7884100720 0x7f788419a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:44:22.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:22.808+0000 7f7888c00640 1 --2- 172.21.15.5:0/3159578559 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f788410e2d0 0x7f788419ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:44:22.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:22.809+0000 7f7883400640 1 -- 172.21.15.5:0/3159578559 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f788410e2d0 msgr2=0x7f788419ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:44:22.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:22.809+0000 7f7883400640 1 --2- 172.21.15.5:0/3159578559 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f788410e2d0 0x7f788419ed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:22.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:22.809+0000 7f7883400640 1 -- 172.21.15.5:0/3159578559 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7884100720 msgr2=0x7f788419a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:44:22.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:22.809+0000 7f7883400640 1 --2- 172.21.15.5:0/3159578559 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7884100720 0x7f788419a460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:22.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:22.809+0000 7f7883400640 1 -- 172.21.15.5:0/3159578559 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f788419f410 con 0x7f78841010d0 2024-09-10T09:44:22.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:22.809+0000 7f7888c00640 1 --2- 172.21.15.5:0/3159578559 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f788410e2d0 0x7f788419ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:44:22.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:22.809+0000 7f7883400640 1 --2- 172.21.15.5:0/3159578559 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f78841010d0 0x7f788419a9a0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f787000e9f0 tx=0x7f787000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:44:22.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:22.809+0000 7f7880c00640 1 -- 172.21.15.5:0/3159578559 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f787000cd80 con 0x7f78841010d0 2024-09-10T09:44:22.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:22.810+0000 7f7880c00640 1 -- 172.21.15.5:0/3159578559 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7870010910 con 0x7f78841010d0 2024-09-10T09:44:22.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:22.810+0000 7f7880c00640 1 -- 172.21.15.5:0/3159578559 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7870018990 con 0x7f78841010d0 2024-09-10T09:44:22.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:22.810+0000 7f7889600640 1 -- 172.21.15.5:0/3159578559 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f788419f700 con 0x7f78841010d0 2024-09-10T09:44:22.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:22.810+0000 7f7889600640 1 -- 172.21.15.5:0/3159578559 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78841a6fe0 con 0x7f78841010d0 2024-09-10T09:44:22.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:22.813+0000 7f788d7e6740 1 Processor -- start 2024-09-10T09:44:22.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:22.813+0000 7f788d7e6740 1 -- start start 2024-09-10T09:44:22.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:22.813+0000 7f7880c00640 1 -- 172.21.15.5:0/3159578559 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f7870010ac0 con 0x7f78841010d0 2024-09-10T09:44:22.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:22.813+0000 7f788d7e6740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f9fc950e40 0x55f9fc951220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:44:22.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:22.814+0000 7f788d7e6740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f9fc951760 0x55f9fc95bae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:44:22.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:22.814+0000 7f788d7e6740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f9fc95c020 0x55f9fc95e410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:44:22.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:22.814+0000 7f788d7e6740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55f9fc8b73e0 con 0x55f9fc950e40 2024-09-10T09:44:22.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:22.814+0000 7f788d7e6740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55f9fc85ed80 con 0x55f9fc95c020 2024-09-10T09:44:22.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:22.814+0000 7f788d7e6740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55f9fc8b33a0 con 0x55f9fc951760 2024-09-10T09:44:22.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:22.814+0000 7f7888c00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f9fc950e40 0x55f9fc951220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:44:22.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:22.814+0000 7f7888c00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f9fc950e40 0x55f9fc951220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49746/0 (socket says 172.21.15.5:49746) 2024-09-10T09:44:22.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:22.814+0000 7f7888c00640 1 -- 172.21.15.5:0/2011379847 learned_addr learned my addr 172.21.15.5:0/2011379847 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:44:22.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:22.814+0000 7f7883400640 1 --2- 172.21.15.5:0/2011379847 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f9fc95c020 0x55f9fc95e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:44:22.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:22.815+0000 7f7883e00640 1 --2- 172.21.15.5:0/2011379847 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f9fc951760 0x55f9fc95bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:44:22.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:22.815+0000 7f7883400640 1 -- 172.21.15.5:0/2011379847 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f9fc951760 msgr2=0x55f9fc95bae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:44:22.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:22.815+0000 7f7883400640 1 --2- 172.21.15.5:0/2011379847 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f9fc951760 0x55f9fc95bae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:22.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:22.815+0000 7f7883400640 1 -- 172.21.15.5:0/2011379847 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f9fc950e40 msgr2=0x55f9fc951220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:44:22.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:22.815+0000 7f7883400640 1 --2- 172.21.15.5:0/2011379847 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f9fc950e40 0x55f9fc951220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:22.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:22.815+0000 7f7883400640 1 -- 172.21.15.5:0/2011379847 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f9fc87dae0 con 0x55f9fc95c020 2024-09-10T09:44:22.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:22.815+0000 7f7888c00640 1 --2- 172.21.15.5:0/2011379847 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f9fc950e40 0x55f9fc951220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:44:22.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:22.815+0000 7f7883400640 1 --2- 172.21.15.5:0/2011379847 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f9fc95c020 0x55f9fc95e410 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f78700cce00 tx=0x7f78700d7670 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8d1650343380274b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:44:22.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:22.815+0000 7f7880c00640 1 --2- 172.21.15.5:0/3159578559 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f78680a2c00 0x7f78680a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:44:22.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:22.815+0000 7f7879200640 1 -- 172.21.15.5:0/2011379847 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78700de070 con 0x55f9fc95c020 2024-09-10T09:44:22.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:22.816+0000 7f7879200640 1 -- 172.21.15.5:0/2011379847 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f787009c2c0 con 0x55f9fc95c020 2024-09-10T09:44:22.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:22.816+0000 7f7879200640 1 -- 172.21.15.5:0/2011379847 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f787009b520 con 0x55f9fc95c020 2024-09-10T09:44:22.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:22.816+0000 7f7880c00640 1 -- 172.21.15.5:0/3159578559 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f78700cf7b0 con 0x7f78841010d0 2024-09-10T09:44:22.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:22.816+0000 7f7888c00640 1 --2- 172.21.15.5:0/3159578559 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f78680a2c00 0x7f78680a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:44:22.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:22.816+0000 7f788d7e6740 1 -- 172.21.15.5:0/2011379847 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f9fc95c020 msgr2=0x55f9fc95e410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:44:22.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:22.816+0000 7f788d7e6740 1 --2- 172.21.15.5:0/2011379847 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f9fc95c020 0x55f9fc95e410 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f78700cce00 tx=0x7f78700d7670 comp rx=0 tx=0).stop 2024-09-10T09:44:22.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:22.817+0000 7f788d7e6740 1 -- 172.21.15.5:0/2011379847 shutdown_connections 2024-09-10T09:44:22.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:22.817+0000 7f788d7e6740 1 --2- 172.21.15.5:0/2011379847 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f9fc95c020 0x55f9fc95e410 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:22.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:22.817+0000 7f788d7e6740 1 --2- 172.21.15.5:0/2011379847 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f9fc951760 0x55f9fc95bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:22.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:22.817+0000 7f788d7e6740 1 --2- 172.21.15.5:0/2011379847 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f9fc950e40 0x55f9fc951220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:22.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:22.817+0000 7f788d7e6740 1 -- 172.21.15.5:0/2011379847 >> 172.21.15.5:0/2011379847 conn(0x55f9fc94d3d0 msgr2=0x55f9fc94d7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:44:22.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:22.817+0000 7f7888c00640 1 --2- 172.21.15.5:0/3159578559 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f78680a2c00 0x7f78680a50c0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f787c011770 tx=0x7f787c009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:44:22.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:22.817+0000 7f788d7e6740 1 -- 172.21.15.5:0/2011379847 shutdown_connections 2024-09-10T09:44:22.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:22.817+0000 7f788d7e6740 1 -- 172.21.15.5:0/2011379847 wait complete. 2024-09-10T09:44:22.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:22.817+0000 7f788d7e6740 1 -- start start 2024-09-10T09:44:22.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:22.818+0000 7f788d7e6740 1 Processor -- start 2024-09-10T09:44:22.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:22.818+0000 7f788d7e6740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f9fc950e40 0x55f9fc958ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:44:22.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:22.818+0000 7f788d7e6740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f9fc951760 0x55f9fc959400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:44:22.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:22.818+0000 7f788d7e6740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f9fc95c020 0x55f9fc959940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:44:22.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:22.818+0000 7f788d7e6740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55f9fc95ebf0 con 0x55f9fc95c020 2024-09-10T09:44:22.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:22.818+0000 7f788d7e6740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55f9fc8720f0 con 0x55f9fc951760 2024-09-10T09:44:22.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:22.818+0000 7f788d7e6740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55f9fc95ed70 con 0x55f9fc950e40 2024-09-10T09:44:22.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:22.818+0000 7f788d7e6740 10 client.0 _ll_register_callbacks cb 0x55f9fc859bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:44:22.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:22.818+0000 7f7883e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f9fc950e40 0x55f9fc958ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:44:22.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:22.818+0000 7f7883400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f9fc951760 0x55f9fc959400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:44:22.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:22.818+0000 7f7883e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f9fc950e40 0x55f9fc958ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:60200/0 (socket says 172.21.15.5:60200) 2024-09-10T09:44:22.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:22.818+0000 7f7883e00640 1 -- 172.21.15.5:0/32108763 learned_addr learned my addr 172.21.15.5:0/32108763 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:44:22.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:22.819+0000 7f7888c00640 1 --2- 172.21.15.5:0/32108763 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f9fc95c020 0x55f9fc959940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:44:22.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:22.819+0000 7f7883e00640 1 -- 172.21.15.5:0/32108763 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f9fc951760 msgr2=0x55f9fc959400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:44:22.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:22.819+0000 7f7883e00640 1 --2- 172.21.15.5:0/32108763 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f9fc951760 0x55f9fc959400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:22.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:22.819+0000 7f7883e00640 1 -- 172.21.15.5:0/32108763 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f9fc95c020 msgr2=0x55f9fc959940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:44:22.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:22.819+0000 7f7883e00640 1 --2- 172.21.15.5:0/32108763 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f9fc95c020 0x55f9fc959940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:22.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:22.819+0000 7f7883e00640 1 -- 172.21.15.5:0/32108763 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f9fc6283c0 con 0x55f9fc950e40 2024-09-10T09:44:22.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:22.819+0000 7f7888c00640 1 --2- 172.21.15.5:0/32108763 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f9fc95c020 0x55f9fc959940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:44:22.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:22.819+0000 7f7883e00640 1 --2- 172.21.15.5:0/32108763 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f9fc950e40 0x55f9fc958ec0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f7874009900 tx=0x7f78740043b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4d17c0647052603a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:44:22.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:22.819+0000 7f786d600640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-09-10T09:44:22.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:22.819+0000 7f786d600640 1 -- 172.21.15.5:0/32108763 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7874035040 con 0x55f9fc950e40 2024-09-10T09:44:22.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:22.819+0000 7f786d600640 1 -- 172.21.15.5:0/32108763 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f7874005450 con 0x55f9fc950e40 2024-09-10T09:44:22.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:22.820+0000 7f788d7e6740 1 -- 172.21.15.5:0/32108763 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f787009c2c0 con 0x55f9fc950e40 2024-09-10T09:44:22.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:22.820+0000 7f786d600640 1 -- 172.21.15.5:0/32108763 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f787404e050 con 0x55f9fc950e40 2024-09-10T09:44:22.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:22.820+0000 7f788d7e6740 10 client.26590 fetch_fsmap learned FSMap version 34 2024-09-10T09:44:22.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:22.820+0000 7f788d7e6740 1 -- 172.21.15.5:0/32108763 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55f9fc96ac30 con 0x55f9fc950e40 2024-09-10T09:44:22.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:22.820+0000 7f786d600640 1 -- 172.21.15.5:0/32108763 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f787009c2c0 con 0x55f9fc950e40 2024-09-10T09:44:22.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:22.820+0000 7f786d600640 1 -- 172.21.15.5:0/32108763 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f78740038c0 con 0x55f9fc950e40 2024-09-10T09:44:22.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:22.820+0000 7f788d7e6740 10 client.26590 fetch_fsmap finished waiting for FSMap version 34 2024-09-10T09:44:22.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:22.820+0000 7f788d7e6740 10 client.26590 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-10T09:44:22.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:22.821+0000 7f788d7e6740 20 client.26590 populate_metadata read hostname 'smithi005' 2024-09-10T09:44:22.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:22.821+0000 7f788d7e6740 1 --2- 172.21.15.5:0/32108763 >> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] conn(0x55f9fc96c0f0 0x55f9fc96e520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:44:22.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:22.821+0000 7f788d7e6740 4 client.26590 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:44:22.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:22.821+0000 7f788d7e6740 1 -- 172.21.15.5:0/32108763 --> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f9fc8c4950 con 0x55f9fc96c0f0 2024-09-10T09:44:22.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:22.821+0000 7f7883400640 1 --2- 172.21.15.5:0/32108763 >> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] conn(0x55f9fc96c0f0 0x55f9fc96e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:44:22.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:22.821+0000 7f786d600640 10 client.26590 ms_handle_connect on v2:172.21.15.159:6838/2073739599 2024-09-10T09:44:22.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:22.822+0000 7f7883400640 1 --2- 172.21.15.5:0/32108763 >> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] conn(0x55f9fc96c0f0 0x55f9fc96e520 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=fce486930aeff508 server_cookie=aaa9ae172924d3e5 in_seq=0 out_seq=0 2024-09-10T09:44:22.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:22.822+0000 7f786d600640 1 -- 172.21.15.5:0/32108763 <== mds.1 v2:172.21.15.159:6838/2073739599 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f9fc8c4950 con 0x55f9fc96c0f0 2024-09-10T09:44:22.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:22.822+0000 7f786d600640 10 client.26590 handle_command_reply: tid=0 2024-09-10T09:44:22.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:22.822+0000 7f788d7e6740 1 client.26590 shutdown 2024-09-10T09:44:22.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:22.823+0000 7f788d7e6740 1 -- 172.21.15.5:0/32108763 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f9fc950e40 msgr2=0x55f9fc958ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:44:22.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:22.823+0000 7f788d7e6740 1 --2- 172.21.15.5:0/32108763 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f9fc950e40 0x55f9fc958ec0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f7874009900 tx=0x7f78740043b0 comp rx=0 tx=0).stop 2024-09-10T09:44:22.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:22.823+0000 7f788d7e6740 1 -- 172.21.15.5:0/32108763 shutdown_connections 2024-09-10T09:44:22.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:22.823+0000 7f788d7e6740 1 --2- 172.21.15.5:0/32108763 >> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] conn(0x55f9fc96c0f0 0x55f9fc96e520 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:22.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:22.823+0000 7f788d7e6740 1 --2- 172.21.15.5:0/32108763 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f9fc95c020 0x55f9fc959940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:22.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:22.823+0000 7f788d7e6740 1 --2- 172.21.15.5:0/32108763 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f9fc951760 0x55f9fc959400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:22.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:22.823+0000 7f788d7e6740 1 --2- 172.21.15.5:0/32108763 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f9fc950e40 0x55f9fc958ec0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:22.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:22.823+0000 7f788d7e6740 1 -- 172.21.15.5:0/32108763 >> 172.21.15.5:0/32108763 conn(0x55f9fc94d3d0 msgr2=0x55f9fc953100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:44:22.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:22.823+0000 7f786d600640 0 client.26590 ms_handle_reset on v2:172.21.15.159:6838/2073739599 2024-09-10T09:44:22.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:22.823+0000 7f788d7e6740 1 -- 172.21.15.5:0/32108763 shutdown_connections 2024-09-10T09:44:22.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:22.823+0000 7f788d7e6740 1 -- 172.21.15.5:0/32108763 wait complete. 2024-09-10T09:44:22.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:22.823+0000 7f788d7e6740 20 client.26590 trim_cache size 0 max 16384 2024-09-10T09:44:22.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:22.839+0000 7f788d7e6740 1 Processor -- start 2024-09-10T09:44:22.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:22.839+0000 7f788d7e6740 1 -- start start 2024-09-10T09:44:22.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:22.839+0000 7f788d7e6740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f9fc950e40 0x55f9fc973df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:44:22.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:22.839+0000 7f788d7e6740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f9fc951760 0x55f9fc974330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:44:22.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:22.839+0000 7f788d7e6740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f9fc95c020 0x55f9fc956120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:44:22.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:22.839+0000 7f788d7e6740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f78841b0e70 con 0x55f9fc95c020 2024-09-10T09:44:22.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:22.839+0000 7f788d7e6740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f78841b29b0 con 0x55f9fc950e40 2024-09-10T09:44:22.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:22.839+0000 7f788d7e6740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f78841b1540 con 0x55f9fc951760 2024-09-10T09:44:22.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:22.840+0000 7f7883400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f9fc951760 0x55f9fc974330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:44:22.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:22.840+0000 7f7888c00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f9fc95c020 0x55f9fc956120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:44:22.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:22.840+0000 7f7883400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f9fc951760 0x55f9fc974330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:60208/0 (socket says 172.21.15.5:60208) 2024-09-10T09:44:22.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:22.840+0000 7f7883400640 1 -- 172.21.15.5:0/3031068085 learned_addr learned my addr 172.21.15.5:0/3031068085 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:44:22.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:22.840+0000 7f7883e00640 1 --2- 172.21.15.5:0/3031068085 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f9fc950e40 0x55f9fc973df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:44:22.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:22.840+0000 7f7883400640 1 -- 172.21.15.5:0/3031068085 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f9fc950e40 msgr2=0x55f9fc973df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:44:22.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:22.840+0000 7f7883400640 1 --2- 172.21.15.5:0/3031068085 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f9fc950e40 0x55f9fc973df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:22.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:22.840+0000 7f7883400640 1 -- 172.21.15.5:0/3031068085 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f9fc95c020 msgr2=0x55f9fc956120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:44:22.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:22.840+0000 7f7883400640 1 --2- 172.21.15.5:0/3031068085 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f9fc95c020 0x55f9fc956120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:22.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:22.840+0000 7f7883400640 1 -- 172.21.15.5:0/3031068085 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f9fc94d200 con 0x55f9fc951760 2024-09-10T09:44:22.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:22.840+0000 7f7883e00640 1 --2- 172.21.15.5:0/3031068085 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f9fc950e40 0x55f9fc973df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:44:22.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:22.841+0000 7f7883400640 1 --2- 172.21.15.5:0/3031068085 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f9fc951760 0x55f9fc974330 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f7870098590 tx=0x7f7870018420 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7a198c88de4e8017 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:44:22.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:22.841+0000 7f786fe00640 1 -- 172.21.15.5:0/3031068085 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7870091300 con 0x55f9fc951760 2024-09-10T09:44:22.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:22.841+0000 7f786fe00640 1 -- 172.21.15.5:0/3031068085 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f78700d7e70 con 0x55f9fc951760 2024-09-10T09:44:22.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:22.841+0000 7f786fe00640 1 -- 172.21.15.5:0/3031068085 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7870091d30 con 0x55f9fc951760 2024-09-10T09:44:22.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:22.842+0000 7f788d7e6740 1 -- 172.21.15.5:0/3031068085 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f9fc951760 msgr2=0x55f9fc974330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:44:22.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:22.842+0000 7f788d7e6740 1 --2- 172.21.15.5:0/3031068085 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f9fc951760 0x55f9fc974330 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f7870098590 tx=0x7f7870018420 comp rx=0 tx=0).stop 2024-09-10T09:44:22.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:22.842+0000 7f788d7e6740 1 -- 172.21.15.5:0/3031068085 shutdown_connections 2024-09-10T09:44:22.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:22.842+0000 7f788d7e6740 1 --2- 172.21.15.5:0/3031068085 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f9fc95c020 0x55f9fc956120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:22.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:22.842+0000 7f788d7e6740 1 --2- 172.21.15.5:0/3031068085 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f9fc951760 0x55f9fc974330 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:22.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:22.842+0000 7f788d7e6740 1 --2- 172.21.15.5:0/3031068085 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f9fc950e40 0x55f9fc973df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:22.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:22.842+0000 7f788d7e6740 1 -- 172.21.15.5:0/3031068085 >> 172.21.15.5:0/3031068085 conn(0x55f9fc94d3d0 msgr2=0x55f9fc95af40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:44:22.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:22.842+0000 7f788d7e6740 1 -- 172.21.15.5:0/3031068085 shutdown_connections 2024-09-10T09:44:22.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:22.842+0000 7f788d7e6740 1 -- 172.21.15.5:0/3031068085 wait complete. 2024-09-10T09:44:22.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:22.843+0000 7f788d7e6740 1 -- start start 2024-09-10T09:44:22.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:22.843+0000 7f788d7e6740 1 Processor -- start 2024-09-10T09:44:22.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:22.843+0000 7f788d7e6740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f9fc950e40 0x55f9fc95fe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:44:22.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:22.843+0000 7f788d7e6740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f9fc951760 0x55f9fc9603d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:44:22.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:22.843+0000 7f788d7e6740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f9fc95c020 0x55f9fc960910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:44:22.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:22.843+0000 7f788d7e6740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f78841b22e0 con 0x55f9fc951760 2024-09-10T09:44:22.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:22.843+0000 7f788d7e6740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f78841b1c10 con 0x55f9fc950e40 2024-09-10T09:44:22.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:22.843+0000 7f788d7e6740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f78841aee50 con 0x55f9fc95c020 2024-09-10T09:44:22.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:22.843+0000 7f788d7e6740 10 client.0 _ll_register_callbacks cb 0x7f787c0093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:44:22.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:22.844+0000 7f7883e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f9fc950e40 0x55f9fc95fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:44:22.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:22.844+0000 7f7883e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f9fc950e40 0x55f9fc95fe90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:43836/0 (socket says 172.21.15.5:43836) 2024-09-10T09:44:22.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:22.844+0000 7f7883e00640 1 -- 172.21.15.5:0/414903525 learned_addr learned my addr 172.21.15.5:0/414903525 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:44:22.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:22.844+0000 7f7888c00640 1 --2- 172.21.15.5:0/414903525 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f9fc95c020 0x55f9fc960910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:44:22.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:22.844+0000 7f7883400640 1 --2- 172.21.15.5:0/414903525 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f9fc951760 0x55f9fc9603d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:44:22.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:22.844+0000 7f7883e00640 1 -- 172.21.15.5:0/414903525 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f9fc95c020 msgr2=0x55f9fc960910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:44:22.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:22.844+0000 7f7883e00640 1 --2- 172.21.15.5:0/414903525 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f9fc95c020 0x55f9fc960910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:22.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:22.844+0000 7f7883e00640 1 -- 172.21.15.5:0/414903525 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f9fc951760 msgr2=0x55f9fc9603d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:44:22.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:22.844+0000 7f7883e00640 1 --2- 172.21.15.5:0/414903525 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f9fc951760 0x55f9fc9603d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:22.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:22.844+0000 7f7883e00640 1 -- 172.21.15.5:0/414903525 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f9fc9de770 con 0x55f9fc950e40 2024-09-10T09:44:22.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:22.844+0000 7f7883e00640 1 --2- 172.21.15.5:0/414903525 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f9fc950e40 0x55f9fc95fe90 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f7874003c90 tx=0x7f7874009800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ead0a6c9a5ad15be server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:44:22.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:22.844+0000 7f786d600640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-09-10T09:44:22.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:22.845+0000 7f786d600640 1 -- 172.21.15.5:0/414903525 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7874035070 con 0x55f9fc950e40 2024-09-10T09:44:22.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:22.845+0000 7f786d600640 1 -- 172.21.15.5:0/414903525 <== mon.2 v2:172.21.15.102:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f7874005250 con 0x55f9fc950e40 2024-09-10T09:44:22.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:22.845+0000 7f788d7e6740 1 -- 172.21.15.5:0/414903525 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f78700d7e70 con 0x55f9fc950e40 2024-09-10T09:44:22.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:22.845+0000 7f786d600640 1 -- 172.21.15.5:0/414903525 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f787405f050 con 0x55f9fc950e40 2024-09-10T09:44:22.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:22.845+0000 7f788d7e6740 10 client.26561 fetch_fsmap learned FSMap version 34 2024-09-10T09:44:22.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:22.845+0000 7f788d7e6740 1 -- 172.21.15.5:0/414903525 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55f9fc9dfa10 con 0x55f9fc950e40 2024-09-10T09:44:22.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:22.845+0000 7f786d600640 1 -- 172.21.15.5:0/414903525 <== mon.2 v2:172.21.15.102:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f78700d7e70 con 0x55f9fc950e40 2024-09-10T09:44:22.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:22.845+0000 7f786d600640 1 -- 172.21.15.5:0/414903525 <== mon.2 v2:172.21.15.102:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f787405a030 con 0x55f9fc950e40 2024-09-10T09:44:22.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:22.846+0000 7f788d7e6740 10 client.26561 fetch_fsmap finished waiting for FSMap version 34 2024-09-10T09:44:22.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:22.846+0000 7f788d7e6740 10 client.26561 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-10T09:44:22.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:22.846+0000 7f788d7e6740 20 client.26561 populate_metadata read hostname 'smithi005' 2024-09-10T09:44:22.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:22.846+0000 7f788d7e6740 1 --2- 172.21.15.5:0/414903525 >> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] conn(0x55f9fc9e01b0 0x55f9fc9e25d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:44:22.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:22.846+0000 7f788d7e6740 4 client.26561 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T09:44:22.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:22.846+0000 7f788d7e6740 1 -- 172.21.15.5:0/414903525 --> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f9fc8bee60 con 0x55f9fc9e01b0 2024-09-10T09:44:22.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:22.846+0000 7f7883400640 1 --2- 172.21.15.5:0/414903525 >> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] conn(0x55f9fc9e01b0 0x55f9fc9e25d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:44:22.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:22.847+0000 7f786d600640 10 client.26561 ms_handle_connect on v2:172.21.15.159:6838/2073739599 2024-09-10T09:44:22.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:22.847+0000 7f7883400640 1 --2- 172.21.15.5:0/414903525 >> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] conn(0x55f9fc9e01b0 0x55f9fc9e25d0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=916a0b124c540b8d server_cookie=c10fe07e6e7ab4f1 in_seq=0 out_seq=0 2024-09-10T09:44:22.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:22.847+0000 7f786d600640 1 -- 172.21.15.5:0/414903525 <== mds.1 v2:172.21.15.159:6838/2073739599 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f9fc8bee60 con 0x55f9fc9e01b0 2024-09-10T09:44:22.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:22.847+0000 7f786d600640 10 client.26561 handle_command_reply: tid=0 2024-09-10T09:44:22.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:22.847+0000 7f788d7e6740 1 client.26561 shutdown 2024-09-10T09:44:22.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:22.848+0000 7f788d7e6740 1 -- 172.21.15.5:0/414903525 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f9fc950e40 msgr2=0x55f9fc95fe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:44:22.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:22.848+0000 7f788d7e6740 1 --2- 172.21.15.5:0/414903525 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f9fc950e40 0x55f9fc95fe90 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f7874003c90 tx=0x7f7874009800 comp rx=0 tx=0).stop 2024-09-10T09:44:22.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:22.848+0000 7f788d7e6740 1 -- 172.21.15.5:0/414903525 shutdown_connections 2024-09-10T09:44:22.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:22.848+0000 7f788d7e6740 1 --2- 172.21.15.5:0/414903525 >> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] conn(0x55f9fc9e01b0 0x55f9fc9e25d0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:22.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:22.848+0000 7f788d7e6740 1 --2- 172.21.15.5:0/414903525 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f9fc95c020 0x55f9fc960910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:22.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:22.848+0000 7f788d7e6740 1 --2- 172.21.15.5:0/414903525 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f9fc951760 0x55f9fc9603d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:22.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:22.848+0000 7f788d7e6740 1 --2- 172.21.15.5:0/414903525 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f9fc950e40 0x55f9fc95fe90 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:22.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:22.848+0000 7f788d7e6740 1 -- 172.21.15.5:0/414903525 >> 172.21.15.5:0/414903525 conn(0x55f9fc94d3d0 msgr2=0x55f9fc95af40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:44:22.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:22.848+0000 7f786d600640 0 client.26561 ms_handle_reset on v2:172.21.15.159:6838/2073739599 2024-09-10T09:44:22.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:22.848+0000 7f788d7e6740 1 -- 172.21.15.5:0/414903525 shutdown_connections 2024-09-10T09:44:22.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:22.848+0000 7f788d7e6740 1 -- 172.21.15.5:0/414903525 wait complete. 2024-09-10T09:44:22.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:22.848+0000 7f788d7e6740 20 client.26561 trim_cache size 0 max 16384 2024-09-10T09:44:22.853 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-10T09:44:22.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:22.849+0000 7f7889600640 1 -- 172.21.15.5:0/3159578559 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f78680a2c00 msgr2=0x7f78680a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:44:22.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:22.849+0000 7f7889600640 1 --2- 172.21.15.5:0/3159578559 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f78680a2c00 0x7f78680a50c0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f787c011770 tx=0x7f787c009500 comp rx=0 tx=0).stop 2024-09-10T09:44:22.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:22.849+0000 7f7889600640 1 -- 172.21.15.5:0/3159578559 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f78841010d0 msgr2=0x7f788419a9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:44:22.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:22.849+0000 7f7889600640 1 --2- 172.21.15.5:0/3159578559 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f78841010d0 0x7f788419a9a0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f787000e9f0 tx=0x7f787000eeb0 comp rx=0 tx=0).stop 2024-09-10T09:44:22.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:22.849+0000 7f7889600640 1 -- 172.21.15.5:0/3159578559 shutdown_connections 2024-09-10T09:44:22.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:22.849+0000 7f7889600640 1 --2- 172.21.15.5:0/3159578559 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f78680a2c00 0x7f78680a50c0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:22.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:22.849+0000 7f7889600640 1 --2- 172.21.15.5:0/3159578559 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f788410e2d0 0x7f788419ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:22.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:22.849+0000 7f7889600640 1 --2- 172.21.15.5:0/3159578559 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f78841010d0 0x7f788419a9a0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:22.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:22.849+0000 7f7889600640 1 --2- 172.21.15.5:0/3159578559 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7884100720 0x7f788419a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:22.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:22.849+0000 7f7889600640 1 -- 172.21.15.5:0/3159578559 >> 172.21.15.5:0/3159578559 conn(0x7f78840fc630 msgr2=0x7f78840fea20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:44:22.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:22.849+0000 7f7889600640 1 -- 172.21.15.5:0/3159578559 shutdown_connections 2024-09-10T09:44:22.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:22.849+0000 7f7889600640 1 -- 172.21.15.5:0/3159578559 wait complete. 2024-09-10T09:44:23.044 INFO:tasks.workunit.client.0.smithi005.stdout: 1 333438 6.15 MB/sec execute 124 sec latency 27.049 ms 2024-09-10T09:44:23.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:44:22 smithi005 ceph-mon[30350]: pgmap v574: 129 pgs: 129 active+clean; 138 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.2 MiB/s wr, 190 op/s 2024-09-10T09:44:23.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:44:22 smithi159 ceph-mon[35513]: pgmap v574: 129 pgs: 129 active+clean; 138 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.2 MiB/s wr, 190 op/s 2024-09-10T09:44:23.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:44:22 smithi102 ceph-mon[35442]: pgmap v574: 129 pgs: 129 active+clean; 138 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.2 MiB/s wr, 190 op/s 2024-09-10T09:44:23.767 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-10T09:44:24.044 INFO:tasks.workunit.client.0.smithi005.stdout: 1 335087 6.16 MB/sec execute 125 sec latency 22.827 ms 2024-09-10T09:44:24.189 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:44:24.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:24.758+0000 7f90a5400640 1 -- 172.21.15.5:0/4094592424 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f90a0076720 msgr2=0x7f90a0076b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:44:24.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:24.758+0000 7f90a5400640 1 --2- 172.21.15.5:0/4094592424 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f90a0076720 0x7f90a0076b00 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f9090009900 tx=0x7f909002a400 comp rx=0 tx=0).stop 2024-09-10T09:44:24.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:24.758+0000 7f90a5400640 1 -- 172.21.15.5:0/4094592424 shutdown_connections 2024-09-10T09:44:24.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:24.758+0000 7f90a5400640 1 --2- 172.21.15.5:0/4094592424 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f90a0112a50 0x7f90a0114e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:24.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:24.758+0000 7f90a5400640 1 --2- 172.21.15.5:0/4094592424 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f90a0077040 0x7f90a0112510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:24.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:24.758+0000 7f90a5400640 1 --2- 172.21.15.5:0/4094592424 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f90a0076720 0x7f90a0076b00 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:24.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:24.758+0000 7f90a5400640 1 -- 172.21.15.5:0/4094592424 >> 172.21.15.5:0/4094592424 conn(0x7f90a00ffa50 msgr2=0x7f90a0101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:44:24.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:24.758+0000 7f90a5400640 1 -- 172.21.15.5:0/4094592424 shutdown_connections 2024-09-10T09:44:24.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:24.758+0000 7f90a5400640 1 -- 172.21.15.5:0/4094592424 wait complete. 2024-09-10T09:44:24.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:24.759+0000 7f90a5400640 1 Processor -- start 2024-09-10T09:44:24.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:24.759+0000 7f90a5400640 1 -- start start 2024-09-10T09:44:24.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:24.760+0000 7f90a5400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f90a0076720 0x7f90a01a4750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:44:24.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:24.760+0000 7f90a5400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f90a0077040 0x7f90a01a4c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:44:24.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:24.760+0000 7f90a5400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f90a0112a50 0x7f90a019e910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:44:24.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:24.760+0000 7f90a5400640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f90a01185e0 con 0x7f90a0076720 2024-09-10T09:44:24.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:24.760+0000 7f90a5400640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f90a0118460 con 0x7f90a0112a50 2024-09-10T09:44:24.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:24.760+0000 7f90a5400640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f90a0118760 con 0x7f90a0077040 2024-09-10T09:44:24.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:24.760+0000 7f909e800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f90a0076720 0x7f90a01a4750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:44:24.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:24.760+0000 7f909de00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f90a0077040 0x7f90a01a4c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:44:24.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:24.760+0000 7f909f200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f90a0112a50 0x7f90a019e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:44:24.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:24.760+0000 7f909e800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f90a0076720 0x7f90a01a4750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49780/0 (socket says 172.21.15.5:49780) 2024-09-10T09:44:24.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:24.760+0000 7f909e800640 1 -- 172.21.15.5:0/3659692133 learned_addr learned my addr 172.21.15.5:0/3659692133 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:44:24.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:24.760+0000 7f909e800640 1 -- 172.21.15.5:0/3659692133 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f90a0077040 msgr2=0x7f90a01a4c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:44:24.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:24.760+0000 7f909e800640 1 --2- 172.21.15.5:0/3659692133 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f90a0077040 0x7f90a01a4c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:24.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:24.760+0000 7f909e800640 1 -- 172.21.15.5:0/3659692133 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f90a0112a50 msgr2=0x7f90a019e910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:44:24.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:24.760+0000 7f909e800640 1 --2- 172.21.15.5:0/3659692133 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f90a0112a50 0x7f90a019e910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:24.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:24.760+0000 7f909e800640 1 -- 172.21.15.5:0/3659692133 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90a019f040 con 0x7f90a0076720 2024-09-10T09:44:24.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:24.761+0000 7f909e800640 1 --2- 172.21.15.5:0/3659692133 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f90a0076720 0x7f90a01a4750 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f9090006010 tx=0x7f9090004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:44:24.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:24.761+0000 7f9097400640 1 -- 172.21.15.5:0/3659692133 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f909003f030 con 0x7f90a0076720 2024-09-10T09:44:24.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:24.761+0000 7f90a5400640 1 -- 172.21.15.5:0/3659692133 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90a019f2d0 con 0x7f90a0076720 2024-09-10T09:44:24.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:24.761+0000 7f9097400640 1 -- 172.21.15.5:0/3659692133 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9090004440 con 0x7f90a0076720 2024-09-10T09:44:24.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:24.761+0000 7f9097400640 1 -- 172.21.15.5:0/3659692133 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9090035070 con 0x7f90a0076720 2024-09-10T09:44:24.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:24.761+0000 7f90a5400640 1 -- 172.21.15.5:0/3659692133 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90a01ab620 con 0x7f90a0076720 2024-09-10T09:44:24.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:24.764+0000 7f90a7dfd740 1 Processor -- start 2024-09-10T09:44:24.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:24.764+0000 7f90a7dfd740 1 -- start start 2024-09-10T09:44:24.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:24.765+0000 7f90a7dfd740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559019b9dc70 0x559019b9e050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:44:24.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:24.765+0000 7f90a7dfd740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559019b9e590 0x559019ba8910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:44:24.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:24.765+0000 7f90a7dfd740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559019ba8e50 0x559019bab240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:44:24.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:24.765+0000 7f90a7dfd740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x559019b043e0 con 0x559019b9dc70 2024-09-10T09:44:24.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:24.765+0000 7f90a7dfd740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x559019aabd80 con 0x559019b9e590 2024-09-10T09:44:24.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:24.765+0000 7f90a7dfd740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x559019b003a0 con 0x559019ba8e50 2024-09-10T09:44:24.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:24.765+0000 7f9097400640 1 -- 172.21.15.5:0/3659692133 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f9090042030 con 0x7f90a0076720 2024-09-10T09:44:24.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:24.765+0000 7f909de00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559019ba8e50 0x559019bab240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:44:24.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:24.765+0000 7f909de00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559019ba8e50 0x559019bab240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:60254/0 (socket says 172.21.15.5:60254) 2024-09-10T09:44:24.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:24.765+0000 7f909de00640 1 -- 172.21.15.5:0/3483279305 learned_addr learned my addr 172.21.15.5:0/3483279305 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:44:24.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:24.765+0000 7f909f200640 1 --2- 172.21.15.5:0/3483279305 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559019b9dc70 0x559019b9e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:44:24.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:24.766+0000 7f909e800640 1 --2- 172.21.15.5:0/3483279305 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559019b9e590 0x559019ba8910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:44:24.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:24.766+0000 7f909de00640 1 -- 172.21.15.5:0/3483279305 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559019b9e590 msgr2=0x559019ba8910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:44:24.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:24.766+0000 7f909de00640 1 --2- 172.21.15.5:0/3483279305 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559019b9e590 0x559019ba8910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:24.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:24.766+0000 7f909de00640 1 -- 172.21.15.5:0/3483279305 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559019b9dc70 msgr2=0x559019b9e050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:44:24.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:24.766+0000 7f909de00640 1 --2- 172.21.15.5:0/3483279305 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559019b9dc70 0x559019b9e050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:24.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:24.766+0000 7f909de00640 1 -- 172.21.15.5:0/3483279305 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559019acaae0 con 0x559019ba8e50 2024-09-10T09:44:24.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:24.766+0000 7f909de00640 1 --2- 172.21.15.5:0/3483279305 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559019ba8e50 0x559019bab240 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f908c009c60 tx=0x7f908c011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=76f4f6bfbd1555c4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:44:24.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:24.767+0000 7f908b400640 1 -- 172.21.15.5:0/3483279305 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f908c004230 con 0x559019ba8e50 2024-09-10T09:44:24.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:24.767+0000 7f908b400640 1 -- 172.21.15.5:0/3483279305 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f908c0043d0 con 0x559019ba8e50 2024-09-10T09:44:24.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:24.767+0000 7f908b400640 1 -- 172.21.15.5:0/3483279305 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f908c019880 con 0x559019ba8e50 2024-09-10T09:44:24.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:24.767+0000 7f9097400640 1 --2- 172.21.15.5:0/3659692133 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f90840a2c00 0x7f90840a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:44:24.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:24.767+0000 7f9097400640 1 -- 172.21.15.5:0/3659692133 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f90900ef440 con 0x7f90a0076720 2024-09-10T09:44:24.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:24.767+0000 7f90a7dfd740 1 -- 172.21.15.5:0/3483279305 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559019ba8e50 msgr2=0x559019bab240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:44:24.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:24.767+0000 7f90a7dfd740 1 --2- 172.21.15.5:0/3483279305 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559019ba8e50 0x559019bab240 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f908c009c60 tx=0x7f908c011810 comp rx=0 tx=0).stop 2024-09-10T09:44:24.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:24.768+0000 7f909f200640 1 --2- 172.21.15.5:0/3659692133 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f90840a2c00 0x7f90840a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:44:24.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:24.768+0000 7f90a7dfd740 1 -- 172.21.15.5:0/3483279305 shutdown_connections 2024-09-10T09:44:24.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:24.768+0000 7f90a7dfd740 1 --2- 172.21.15.5:0/3483279305 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559019ba8e50 0x559019bab240 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:24.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:24.768+0000 7f90a7dfd740 1 --2- 172.21.15.5:0/3483279305 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559019b9e590 0x559019ba8910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:24.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:24.768+0000 7f90a7dfd740 1 --2- 172.21.15.5:0/3483279305 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559019b9dc70 0x559019b9e050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:24.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:24.768+0000 7f90a7dfd740 1 -- 172.21.15.5:0/3483279305 >> 172.21.15.5:0/3483279305 conn(0x559019b9a200 msgr2=0x559019b9a5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:44:24.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:24.768+0000 7f90a7dfd740 1 -- 172.21.15.5:0/3483279305 shutdown_connections 2024-09-10T09:44:24.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:24.768+0000 7f90a7dfd740 1 -- 172.21.15.5:0/3483279305 wait complete. 2024-09-10T09:44:24.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:24.768+0000 7f90a7dfd740 1 -- start start 2024-09-10T09:44:24.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:24.768+0000 7f909f200640 1 --2- 172.21.15.5:0/3659692133 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f90840a2c00 0x7f90840a50c0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f90a006b460 tx=0x7f9098009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:44:24.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:24.769+0000 7f90a7dfd740 1 Processor -- start 2024-09-10T09:44:24.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:24.769+0000 7f90a7dfd740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559019b9dc70 0x559019ba5cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:44:24.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:24.769+0000 7f90a7dfd740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559019b9e590 0x559019ba6200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:44:24.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:24.769+0000 7f90a7dfd740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559019ba8e50 0x559019ba6740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:44:24.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:24.769+0000 7f90a7dfd740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x559019baba20 con 0x559019ba8e50 2024-09-10T09:44:24.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:24.769+0000 7f90a7dfd740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x559019abf0f0 con 0x559019b9e590 2024-09-10T09:44:24.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:24.769+0000 7f90a7dfd740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x559019babba0 con 0x559019b9dc70 2024-09-10T09:44:24.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:24.769+0000 7f90a7dfd740 10 client.0 _ll_register_callbacks cb 0x559019aa6bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:44:24.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:24.769+0000 7f909e800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559019b9dc70 0x559019ba5cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:44:24.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:24.769+0000 7f909de00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559019b9e590 0x559019ba6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:44:24.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:24.769+0000 7f909e800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559019b9dc70 0x559019ba5cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:60270/0 (socket says 172.21.15.5:60270) 2024-09-10T09:44:24.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:24.769+0000 7f909de00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559019b9e590 0x559019ba6200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:43874/0 (socket says 172.21.15.5:43874) 2024-09-10T09:44:24.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:24.769+0000 7f909e800640 1 -- 172.21.15.5:0/3135608009 learned_addr learned my addr 172.21.15.5:0/3135608009 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:44:24.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:24.769+0000 7f909f200640 1 --2- 172.21.15.5:0/3135608009 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559019ba8e50 0x559019ba6740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:44:24.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:24.770+0000 7f909e800640 1 -- 172.21.15.5:0/3135608009 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559019b9e590 msgr2=0x559019ba6200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:44:24.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:24.770+0000 7f909e800640 1 --2- 172.21.15.5:0/3135608009 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559019b9e590 0x559019ba6200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:24.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:24.770+0000 7f909e800640 1 -- 172.21.15.5:0/3135608009 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559019ba8e50 msgr2=0x559019ba6740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:44:24.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:24.770+0000 7f909e800640 1 --2- 172.21.15.5:0/3135608009 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559019ba8e50 0x559019ba6740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:24.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:24.770+0000 7f909e800640 1 -- 172.21.15.5:0/3135608009 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5590198753c0 con 0x559019b9dc70 2024-09-10T09:44:24.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:24.770+0000 7f909de00640 1 --2- 172.21.15.5:0/3135608009 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559019b9e590 0x559019ba6200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:44:24.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:24.770+0000 7f909e800640 1 --2- 172.21.15.5:0/3135608009 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559019b9dc70 0x559019ba5cc0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f90900bb890 tx=0x7f9090009620 comp rx=0 tx=0).ready entity=mon.1 client_cookie=46248343f5bb0c34 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:44:24.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:24.770+0000 7f907fe00640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-09-10T09:44:24.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:24.770+0000 7f907fe00640 1 -- 172.21.15.5:0/3135608009 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9090049030 con 0x559019b9dc70 2024-09-10T09:44:24.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:24.770+0000 7f90a7dfd740 1 -- 172.21.15.5:0/3135608009 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f908c0043d0 con 0x559019b9dc70 2024-09-10T09:44:24.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:24.770+0000 7f907fe00640 1 -- 172.21.15.5:0/3135608009 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f9090003cd0 con 0x559019b9dc70 2024-09-10T09:44:24.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:24.770+0000 7f907fe00640 1 -- 172.21.15.5:0/3135608009 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f90900bb260 con 0x559019b9dc70 2024-09-10T09:44:24.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:24.771+0000 7f90a7dfd740 10 client.26608 fetch_fsmap learned FSMap version 34 2024-09-10T09:44:24.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:24.771+0000 7f90a7dfd740 1 -- 172.21.15.5:0/3135608009 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x559019baba70 con 0x559019b9dc70 2024-09-10T09:44:24.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:24.771+0000 7f907fe00640 1 -- 172.21.15.5:0/3135608009 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f908c0043d0 con 0x559019b9dc70 2024-09-10T09:44:24.896 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:44:24 smithi005 ceph-mon[30350]: pgmap v575: 129 pgs: 129 active+clean; 139 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.0 MiB/s wr, 153 op/s 2024-09-10T09:44:24.897 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-10T09:44:24.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:24.771+0000 7f907fe00640 1 -- 172.21.15.5:0/3135608009 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f909004c060 con 0x559019b9dc70 2024-09-10T09:44:24.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:24.771+0000 7f90a7dfd740 10 client.26608 fetch_fsmap finished waiting for FSMap version 34 2024-09-10T09:44:24.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:24.771+0000 7f90a7dfd740 10 client.26608 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T09:44:24.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:24.771+0000 7f90a7dfd740 20 client.26608 populate_metadata read hostname 'smithi005' 2024-09-10T09:44:24.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:24.771+0000 7f90a7dfd740 1 --2- 172.21.15.5:0/3135608009 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x559019bb8cc0 0x559019bbb130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:44:24.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:24.771+0000 7f90a7dfd740 4 client.26608 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:44:24.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:24.771+0000 7f90a7dfd740 1 -- 172.21.15.5:0/3135608009 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559019b11950 con 0x559019bb8cc0 2024-09-10T09:44:24.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:24.771+0000 7f909de00640 1 --2- 172.21.15.5:0/3135608009 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x559019bb8cc0 0x559019bbb130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:44:24.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:24.772+0000 7f909de00640 1 --2- 172.21.15.5:0/3135608009 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x559019bb8cc0 0x559019bbb130 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d22b492b66f7d125 server_cookie=dc56f359afacd885 in_seq=0 out_seq=0 2024-09-10T09:44:24.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:24.772+0000 7f907fe00640 10 client.26608 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T09:44:24.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:24.773+0000 7f907fe00640 1 -- 172.21.15.5:0/3135608009 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559019b11950 con 0x559019bb8cc0 2024-09-10T09:44:24.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:24.773+0000 7f907fe00640 10 client.26608 handle_command_reply: tid=0 2024-09-10T09:44:24.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:24.773+0000 7f90a7dfd740 1 client.26608 shutdown 2024-09-10T09:44:24.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:24.773+0000 7f90a7dfd740 1 -- 172.21.15.5:0/3135608009 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559019b9dc70 msgr2=0x559019ba5cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:44:24.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:24.773+0000 7f90a7dfd740 1 --2- 172.21.15.5:0/3135608009 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559019b9dc70 0x559019ba5cc0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f90900bb890 tx=0x7f9090009620 comp rx=0 tx=0).stop 2024-09-10T09:44:24.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:24.773+0000 7f90a7dfd740 1 -- 172.21.15.5:0/3135608009 shutdown_connections 2024-09-10T09:44:24.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:24.773+0000 7f90a7dfd740 1 --2- 172.21.15.5:0/3135608009 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x559019bb8cc0 0x559019bbb130 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:24.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:24.773+0000 7f90a7dfd740 1 --2- 172.21.15.5:0/3135608009 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559019ba8e50 0x559019ba6740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:24.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:24.773+0000 7f90a7dfd740 1 --2- 172.21.15.5:0/3135608009 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559019b9e590 0x559019ba6200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:24.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:24.773+0000 7f90a7dfd740 1 --2- 172.21.15.5:0/3135608009 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559019b9dc70 0x559019ba5cc0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:24.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:24.773+0000 7f907fe00640 0 client.26608 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T09:44:24.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:24.773+0000 7f90a7dfd740 1 -- 172.21.15.5:0/3135608009 >> 172.21.15.5:0/3135608009 conn(0x559019b9a200 msgr2=0x559019b9ff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:44:24.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:24.773+0000 7f90a7dfd740 1 -- 172.21.15.5:0/3135608009 shutdown_connections 2024-09-10T09:44:24.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:24.773+0000 7f90a7dfd740 1 -- 172.21.15.5:0/3135608009 wait complete. 2024-09-10T09:44:24.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:24.774+0000 7f90a7dfd740 20 client.26608 trim_cache size 0 max 16384 2024-09-10T09:44:24.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:24.789+0000 7f90a7dfd740 1 Processor -- start 2024-09-10T09:44:24.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:24.789+0000 7f90a7dfd740 1 -- start start 2024-09-10T09:44:24.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:24.790+0000 7f90a7dfd740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559019b9dc70 0x559019ba06d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:44:24.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:24.790+0000 7f90a7dfd740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559019b9e590 0x559019bc09d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:44:24.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:24.790+0000 7f90a7dfd740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559019ba8e50 0x559019bc0f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:44:24.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:24.790+0000 7f90a7dfd740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f90a01b5460 con 0x559019b9dc70 2024-09-10T09:44:24.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:24.790+0000 7f90a7dfd740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f90a01b6fa0 con 0x559019b9e590 2024-09-10T09:44:24.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:24.790+0000 7f90a7dfd740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f90a01b5b30 con 0x559019ba8e50 2024-09-10T09:44:24.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:24.790+0000 7f909de00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559019b9e590 0x559019bc09d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:44:24.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:24.790+0000 7f909f200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559019ba8e50 0x559019bc0f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:44:24.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:24.790+0000 7f909de00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559019b9e590 0x559019bc09d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:43880/0 (socket says 172.21.15.5:43880) 2024-09-10T09:44:24.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:24.790+0000 7f909e800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559019b9dc70 0x559019ba06d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:44:24.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:24.790+0000 7f909de00640 1 -- 172.21.15.5:0/1874600904 learned_addr learned my addr 172.21.15.5:0/1874600904 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:44:24.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:24.790+0000 7f909de00640 1 -- 172.21.15.5:0/1874600904 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559019ba8e50 msgr2=0x559019bc0f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:44:24.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:24.790+0000 7f909de00640 1 --2- 172.21.15.5:0/1874600904 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559019ba8e50 0x559019bc0f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:24.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:24.790+0000 7f909de00640 1 -- 172.21.15.5:0/1874600904 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559019b9dc70 msgr2=0x559019ba06d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:44:24.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:24.790+0000 7f909de00640 1 --2- 172.21.15.5:0/1874600904 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559019b9dc70 0x559019ba06d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:24.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:24.791+0000 7f909de00640 1 -- 172.21.15.5:0/1874600904 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559019b9a030 con 0x559019b9e590 2024-09-10T09:44:24.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:24.791+0000 7f909de00640 1 --2- 172.21.15.5:0/1874600904 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559019b9e590 0x559019bc09d0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f908c011d10 tx=0x7f908c005850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=de94f99e92cff9c4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:44:24.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:24.791+0000 7f908aa00640 1 -- 172.21.15.5:0/1874600904 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f908c005140 con 0x559019b9e590 2024-09-10T09:44:24.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:24.791+0000 7f908aa00640 1 -- 172.21.15.5:0/1874600904 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f908c0035f0 con 0x559019b9e590 2024-09-10T09:44:24.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:24.791+0000 7f908aa00640 1 -- 172.21.15.5:0/1874600904 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f908c0045a0 con 0x559019b9e590 2024-09-10T09:44:24.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:24.792+0000 7f90a7dfd740 1 -- 172.21.15.5:0/1874600904 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559019b9e590 msgr2=0x559019bc09d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:44:24.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:24.792+0000 7f90a7dfd740 1 --2- 172.21.15.5:0/1874600904 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559019b9e590 0x559019bc09d0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f908c011d10 tx=0x7f908c005850 comp rx=0 tx=0).stop 2024-09-10T09:44:24.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:24.792+0000 7f90a7dfd740 1 -- 172.21.15.5:0/1874600904 shutdown_connections 2024-09-10T09:44:24.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:24.792+0000 7f90a7dfd740 1 --2- 172.21.15.5:0/1874600904 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559019ba8e50 0x559019bc0f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:24.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:24.792+0000 7f90a7dfd740 1 --2- 172.21.15.5:0/1874600904 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559019b9e590 0x559019bc09d0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:24.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:24.792+0000 7f90a7dfd740 1 --2- 172.21.15.5:0/1874600904 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559019b9dc70 0x559019ba06d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:24.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:24.792+0000 7f90a7dfd740 1 -- 172.21.15.5:0/1874600904 >> 172.21.15.5:0/1874600904 conn(0x559019b9a200 msgr2=0x559019ba4db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:44:24.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:24.792+0000 7f90a7dfd740 1 -- 172.21.15.5:0/1874600904 shutdown_connections 2024-09-10T09:44:24.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:24.792+0000 7f90a7dfd740 1 -- 172.21.15.5:0/1874600904 wait complete. 2024-09-10T09:44:24.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:24.792+0000 7f90a7dfd740 1 -- start start 2024-09-10T09:44:24.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:24.793+0000 7f90a7dfd740 1 Processor -- start 2024-09-10T09:44:24.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:24.793+0000 7f90a7dfd740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559019b9dc70 0x559019bb1440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:44:24.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:24.793+0000 7f90a7dfd740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559019b9e590 0x559019bac770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:44:24.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:24.793+0000 7f90a7dfd740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559019ba8e50 0x559019baccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:44:24.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:24.793+0000 7f90a7dfd740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f90a01b68d0 con 0x559019b9e590 2024-09-10T09:44:24.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:24.793+0000 7f90a7dfd740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f90a01b6200 con 0x559019ba8e50 2024-09-10T09:44:24.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:24.793+0000 7f90a7dfd740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f90a01b3440 con 0x559019b9dc70 2024-09-10T09:44:24.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:24.794+0000 7f90a7dfd740 10 client.0 _ll_register_callbacks cb 0x7f909003b460 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:44:24.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:24.794+0000 7f909e800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559019b9dc70 0x559019bb1440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:44:24.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:24.794+0000 7f909e800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559019b9dc70 0x559019bb1440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:60286/0 (socket says 172.21.15.5:60286) 2024-09-10T09:44:24.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:24.794+0000 7f909e800640 1 -- 172.21.15.5:0/3313159062 learned_addr learned my addr 172.21.15.5:0/3313159062 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:44:24.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:24.794+0000 7f909de00640 1 --2- 172.21.15.5:0/3313159062 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559019b9e590 0x559019bac770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:44:24.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:24.794+0000 7f909f200640 1 --2- 172.21.15.5:0/3313159062 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559019ba8e50 0x559019baccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:44:24.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:24.794+0000 7f909e800640 1 -- 172.21.15.5:0/3313159062 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559019ba8e50 msgr2=0x559019baccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:44:24.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:24.794+0000 7f909e800640 1 --2- 172.21.15.5:0/3313159062 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559019ba8e50 0x559019baccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:24.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:24.794+0000 7f909e800640 1 -- 172.21.15.5:0/3313159062 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559019b9e590 msgr2=0x559019bac770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:44:24.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:24.794+0000 7f909e800640 1 --2- 172.21.15.5:0/3313159062 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559019b9e590 0x559019bac770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:24.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:24.794+0000 7f909e800640 1 -- 172.21.15.5:0/3313159062 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559019b95770 con 0x559019b9dc70 2024-09-10T09:44:24.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:24.794+0000 7f909f200640 1 --2- 172.21.15.5:0/3313159062 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559019ba8e50 0x559019baccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:44:24.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:24.794+0000 7f909de00640 1 --2- 172.21.15.5:0/3313159062 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559019b9e590 0x559019bac770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:44:24.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:24.795+0000 7f909e800640 1 --2- 172.21.15.5:0/3313159062 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559019b9dc70 0x559019bb1440 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f90900c3f90 tx=0x7f90900bb570 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5c89ddb34c4e55e8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:44:24.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:24.795+0000 7f907fe00640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-09-10T09:44:24.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:24.795+0000 7f907fe00640 1 -- 172.21.15.5:0/3313159062 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9090049070 con 0x559019b9dc70 2024-09-10T09:44:24.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:24.795+0000 7f90a7dfd740 1 -- 172.21.15.5:0/3313159062 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f908c0035f0 con 0x559019b9dc70 2024-09-10T09:44:24.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:24.795+0000 7f907fe00640 1 -- 172.21.15.5:0/3313159062 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f90900f2b10 con 0x559019b9dc70 2024-09-10T09:44:24.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:24.795+0000 7f907fe00640 1 -- 172.21.15.5:0/3313159062 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f90900bb900 con 0x559019b9dc70 2024-09-10T09:44:24.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:24.795+0000 7f90a7dfd740 10 client.26614 fetch_fsmap learned FSMap version 34 2024-09-10T09:44:24.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:24.795+0000 7f90a7dfd740 1 -- 172.21.15.5:0/3313159062 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x559019c236e0 con 0x559019b9dc70 2024-09-10T09:44:24.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:24.795+0000 7f907fe00640 1 -- 172.21.15.5:0/3313159062 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f908c0035f0 con 0x559019b9dc70 2024-09-10T09:44:24.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:24.795+0000 7f907fe00640 1 -- 172.21.15.5:0/3313159062 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f90900bad50 con 0x559019b9dc70 2024-09-10T09:44:24.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:24.795+0000 7f90a7dfd740 10 client.26614 fetch_fsmap finished waiting for FSMap version 34 2024-09-10T09:44:24.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:24.795+0000 7f90a7dfd740 10 client.26614 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T09:44:24.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:24.795+0000 7f90a7dfd740 20 client.26614 populate_metadata read hostname 'smithi005' 2024-09-10T09:44:24.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:24.795+0000 7f90a7dfd740 1 --2- 172.21.15.5:0/3313159062 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x559019c23c70 0x559019c26060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:44:24.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:24.795+0000 7f90a7dfd740 4 client.26614 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T09:44:24.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:24.795+0000 7f90a7dfd740 1 -- 172.21.15.5:0/3313159062 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559019b0be60 con 0x559019c23c70 2024-09-10T09:44:24.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:24.796+0000 7f909de00640 1 --2- 172.21.15.5:0/3313159062 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x559019c23c70 0x559019c26060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:44:24.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:24.796+0000 7f909de00640 1 --2- 172.21.15.5:0/3313159062 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x559019c23c70 0x559019c26060 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7b8d3225f77bb7f6 server_cookie=b09e83e4ab52cfe8 in_seq=0 out_seq=0 2024-09-10T09:44:24.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:24.796+0000 7f907fe00640 10 client.26614 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T09:44:24.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:24.797+0000 7f907fe00640 1 -- 172.21.15.5:0/3313159062 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559019b0be60 con 0x559019c23c70 2024-09-10T09:44:24.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:24.797+0000 7f907fe00640 10 client.26614 handle_command_reply: tid=0 2024-09-10T09:44:24.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:24.797+0000 7f90a7dfd740 1 client.26614 shutdown 2024-09-10T09:44:24.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:24.797+0000 7f90a7dfd740 1 -- 172.21.15.5:0/3313159062 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559019b9dc70 msgr2=0x559019bb1440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:44:24.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:24.797+0000 7f90a7dfd740 1 --2- 172.21.15.5:0/3313159062 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559019b9dc70 0x559019bb1440 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f90900c3f90 tx=0x7f90900bb570 comp rx=0 tx=0).stop 2024-09-10T09:44:24.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:24.797+0000 7f90a7dfd740 1 -- 172.21.15.5:0/3313159062 shutdown_connections 2024-09-10T09:44:24.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:24.797+0000 7f90a7dfd740 1 --2- 172.21.15.5:0/3313159062 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x559019c23c70 0x559019c26060 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:24.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:24.797+0000 7f90a7dfd740 1 --2- 172.21.15.5:0/3313159062 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559019ba8e50 0x559019baccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:24.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:24.797+0000 7f90a7dfd740 1 --2- 172.21.15.5:0/3313159062 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559019b9e590 0x559019bac770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:24.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:24.797+0000 7f90a7dfd740 1 --2- 172.21.15.5:0/3313159062 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559019b9dc70 0x559019bb1440 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:24.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:24.797+0000 7f907fe00640 0 client.26614 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T09:44:24.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:24.797+0000 7f90a7dfd740 1 -- 172.21.15.5:0/3313159062 >> 172.21.15.5:0/3313159062 conn(0x559019b9a200 msgr2=0x559019b98b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:44:24.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:24.797+0000 7f90a7dfd740 1 -- 172.21.15.5:0/3313159062 shutdown_connections 2024-09-10T09:44:24.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:24.797+0000 7f90a7dfd740 1 -- 172.21.15.5:0/3313159062 wait complete. 2024-09-10T09:44:24.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:24.797+0000 7f90a7dfd740 20 client.26614 trim_cache size 0 max 16384 2024-09-10T09:44:24.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:24.798+0000 7f90a5400640 1 -- 172.21.15.5:0/3659692133 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f90840a2c00 msgr2=0x7f90840a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:44:24.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:24.798+0000 7f90a5400640 1 --2- 172.21.15.5:0/3659692133 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f90840a2c00 0x7f90840a50c0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f90a006b460 tx=0x7f9098009500 comp rx=0 tx=0).stop 2024-09-10T09:44:24.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:24.798+0000 7f90a5400640 1 -- 172.21.15.5:0/3659692133 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f90a0076720 msgr2=0x7f90a01a4750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:44:24.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:24.798+0000 7f90a5400640 1 --2- 172.21.15.5:0/3659692133 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f90a0076720 0x7f90a01a4750 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f9090006010 tx=0x7f9090004040 comp rx=0 tx=0).stop 2024-09-10T09:44:24.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:24.798+0000 7f90a5400640 1 -- 172.21.15.5:0/3659692133 shutdown_connections 2024-09-10T09:44:24.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:24.798+0000 7f90a5400640 1 --2- 172.21.15.5:0/3659692133 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f90840a2c00 0x7f90840a50c0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:24.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:24.798+0000 7f90a5400640 1 --2- 172.21.15.5:0/3659692133 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f90a0112a50 0x7f90a019e910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:24.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:24.798+0000 7f90a5400640 1 --2- 172.21.15.5:0/3659692133 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f90a0077040 0x7f90a01a4c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:24.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:24.798+0000 7f90a5400640 1 --2- 172.21.15.5:0/3659692133 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f90a0076720 0x7f90a01a4750 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:24.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:24.798+0000 7f90a5400640 1 -- 172.21.15.5:0/3659692133 >> 172.21.15.5:0/3659692133 conn(0x7f90a00ffa50 msgr2=0x7f90a0101e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:44:24.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:24.798+0000 7f90a5400640 1 -- 172.21.15.5:0/3659692133 shutdown_connections 2024-09-10T09:44:24.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:24.798+0000 7f90a5400640 1 -- 172.21.15.5:0/3659692133 wait complete. 2024-09-10T09:44:25.044 INFO:tasks.workunit.client.0.smithi005.stdout: 1 337208 6.22 MB/sec execute 126 sec latency 135.989 ms 2024-09-10T09:44:25.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:44:24 smithi159 ceph-mon[35513]: pgmap v575: 129 pgs: 129 active+clean; 139 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.0 MiB/s wr, 153 op/s 2024-09-10T09:44:25.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:44:24 smithi102 ceph-mon[35442]: pgmap v575: 129 pgs: 129 active+clean; 139 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.0 MiB/s wr, 153 op/s 2024-09-10T09:44:26.044 INFO:tasks.workunit.client.0.smithi005.stdout: 1 337208 6.17 MB/sec execute 127 sec latency 1136.048 ms 2024-09-10T09:44:26.821 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-10T09:44:26.821 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-10T09:44:27.044 INFO:tasks.workunit.client.0.smithi005.stdout: 1 338587 6.16 MB/sec execute 128 sec latency 1267.746 ms 2024-09-10T09:44:27.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:44:26 smithi159 ceph-mon[35513]: pgmap v576: 129 pgs: 129 active+clean; 141 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.2 MiB/s wr, 202 op/s 2024-09-10T09:44:27.136 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:44:26 smithi102 ceph-mon[35442]: pgmap v576: 129 pgs: 129 active+clean; 141 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.2 MiB/s wr, 202 op/s 2024-09-10T09:44:27.249 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:44:27.306 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:44:26 smithi005 ceph-mon[30350]: pgmap v576: 129 pgs: 129 active+clean; 141 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.2 MiB/s wr, 202 op/s 2024-09-10T09:44:27.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:27.840+0000 7fe613e00640 1 -- 172.21.15.5:0/489323969 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe60c112a50 msgr2=0x7fe60c114e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:44:27.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:27.840+0000 7fe613e00640 1 --2- 172.21.15.5:0/489323969 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe60c112a50 0x7fe60c114e40 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fe604009ca0 tx=0x7fe60402a8c0 comp rx=0 tx=0).stop 2024-09-10T09:44:27.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:27.840+0000 7fe613e00640 1 -- 172.21.15.5:0/489323969 shutdown_connections 2024-09-10T09:44:27.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:27.840+0000 7fe613e00640 1 --2- 172.21.15.5:0/489323969 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe60c112a50 0x7fe60c114e40 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:27.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:27.840+0000 7fe613e00640 1 --2- 172.21.15.5:0/489323969 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe60c077040 0x7fe60c112510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:27.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:27.840+0000 7fe613e00640 1 --2- 172.21.15.5:0/489323969 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe60c076720 0x7fe60c076b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:27.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:27.840+0000 7fe613e00640 1 -- 172.21.15.5:0/489323969 >> 172.21.15.5:0/489323969 conn(0x7fe60c0ffa50 msgr2=0x7fe60c101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:44:27.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:27.840+0000 7fe613e00640 1 -- 172.21.15.5:0/489323969 shutdown_connections 2024-09-10T09:44:27.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:27.840+0000 7fe613e00640 1 -- 172.21.15.5:0/489323969 wait complete. 2024-09-10T09:44:27.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:27.842+0000 7fe613e00640 1 Processor -- start 2024-09-10T09:44:27.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:27.842+0000 7fe613e00640 1 -- start start 2024-09-10T09:44:27.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:27.842+0000 7fe613e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe60c076720 0x7fe60c19eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:44:27.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:27.842+0000 7fe613e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe60c077040 0x7fe60c19f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:44:27.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:27.842+0000 7fe613e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe60c112a50 0x7fe60c1a3480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:44:27.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:27.842+0000 7fe613e00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe60c1185e0 con 0x7fe60c077040 2024-09-10T09:44:27.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:27.842+0000 7fe613e00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fe60c118460 con 0x7fe60c112a50 2024-09-10T09:44:27.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:27.842+0000 7fe613e00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fe60c118760 con 0x7fe60c076720 2024-09-10T09:44:27.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:27.842+0000 7fe611400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe60c076720 0x7fe60c19eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:44:27.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:27.842+0000 7fe611400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe60c076720 0x7fe60c19eb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:60302/0 (socket says 172.21.15.5:60302) 2024-09-10T09:44:27.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:27.842+0000 7fe611400640 1 -- 172.21.15.5:0/369752396 learned_addr learned my addr 172.21.15.5:0/369752396 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:44:27.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:27.843+0000 7fe611e00640 1 --2- 172.21.15.5:0/369752396 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe60c112a50 0x7fe60c1a3480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:44:27.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:27.843+0000 7fe610a00640 1 --2- 172.21.15.5:0/369752396 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe60c077040 0x7fe60c19f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:44:27.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:27.843+0000 7fe611400640 1 -- 172.21.15.5:0/369752396 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe60c112a50 msgr2=0x7fe60c1a3480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:44:27.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:27.843+0000 7fe611400640 1 --2- 172.21.15.5:0/369752396 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe60c112a50 0x7fe60c1a3480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:27.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:27.843+0000 7fe611400640 1 -- 172.21.15.5:0/369752396 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe60c077040 msgr2=0x7fe60c19f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:44:27.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:27.843+0000 7fe611400640 1 --2- 172.21.15.5:0/369752396 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe60c077040 0x7fe60c19f0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:27.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:27.843+0000 7fe611400640 1 -- 172.21.15.5:0/369752396 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe60c1a3bb0 con 0x7fe60c076720 2024-09-10T09:44:27.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:27.843+0000 7fe610a00640 1 --2- 172.21.15.5:0/369752396 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe60c077040 0x7fe60c19f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:44:27.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:27.843+0000 7fe611400640 1 --2- 172.21.15.5:0/369752396 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe60c076720 0x7fe60c19eb80 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fe60000e9f0 tx=0x7fe60000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:44:27.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:27.843+0000 7fe60a000640 1 -- 172.21.15.5:0/369752396 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe60000cd80 con 0x7fe60c076720 2024-09-10T09:44:27.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:27.843+0000 7fe613e00640 1 -- 172.21.15.5:0/369752396 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe60c1a3ea0 con 0x7fe60c076720 2024-09-10T09:44:27.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:27.843+0000 7fe60a000640 1 -- 172.21.15.5:0/369752396 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe600010910 con 0x7fe60c076720 2024-09-10T09:44:27.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:27.843+0000 7fe60a000640 1 -- 172.21.15.5:0/369752396 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6000188f0 con 0x7fe60c076720 2024-09-10T09:44:27.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:27.844+0000 7fe613e00640 1 -- 172.21.15.5:0/369752396 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe60c1ab730 con 0x7fe60c076720 2024-09-10T09:44:27.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:27.847+0000 7fe60a000640 1 -- 172.21.15.5:0/369752396 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7fe600010ac0 con 0x7fe60c076720 2024-09-10T09:44:27.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:27.847+0000 7fe61686e740 1 Processor -- start 2024-09-10T09:44:27.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:27.847+0000 7fe61686e740 1 -- start start 2024-09-10T09:44:27.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:27.848+0000 7fe61686e740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561c45eb6ac0 0x561c45eb6ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:44:27.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:27.848+0000 7fe61686e740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561c45eb73e0 0x561c45ec1760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:44:27.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:27.848+0000 7fe61686e740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561c45ec1ca0 0x561c45ec4090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:44:27.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:27.848+0000 7fe61686e740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x561c45e1d290 con 0x561c45eb6ac0 2024-09-10T09:44:27.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:27.848+0000 7fe61686e740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x561c45dc4b40 con 0x561c45ec1ca0 2024-09-10T09:44:27.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:27.848+0000 7fe61686e740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x561c45e19250 con 0x561c45eb73e0 2024-09-10T09:44:27.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:27.848+0000 7fe611400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561c45eb73e0 0x561c45ec1760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:44:27.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:27.848+0000 7fe611400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561c45eb73e0 0x561c45ec1760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:60316/0 (socket says 172.21.15.5:60316) 2024-09-10T09:44:27.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:27.848+0000 7fe611400640 1 -- 172.21.15.5:0/2686968459 learned_addr learned my addr 172.21.15.5:0/2686968459 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:44:27.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:27.848+0000 7fe610a00640 1 --2- 172.21.15.5:0/2686968459 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561c45ec1ca0 0x561c45ec4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:44:27.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:27.848+0000 7fe611e00640 1 --2- 172.21.15.5:0/2686968459 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561c45eb6ac0 0x561c45eb6ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:44:27.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:27.849+0000 7fe611400640 1 -- 172.21.15.5:0/2686968459 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561c45ec1ca0 msgr2=0x561c45ec4090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:44:27.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:27.849+0000 7fe611400640 1 --2- 172.21.15.5:0/2686968459 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561c45ec1ca0 0x561c45ec4090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:27.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:27.849+0000 7fe611400640 1 -- 172.21.15.5:0/2686968459 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561c45eb6ac0 msgr2=0x561c45eb6ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:44:27.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:27.849+0000 7fe611400640 1 --2- 172.21.15.5:0/2686968459 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561c45eb6ac0 0x561c45eb6ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:27.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:27.849+0000 7fe611400640 1 -- 172.21.15.5:0/2686968459 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561c45b8e6a0 con 0x561c45eb73e0 2024-09-10T09:44:27.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:27.849+0000 7fe60a000640 1 --2- 172.21.15.5:0/369752396 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fe5f40a2c00 0x7fe5f40a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:44:27.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:27.849+0000 7fe611400640 1 --2- 172.21.15.5:0/2686968459 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561c45eb73e0 0x561c45ec1760 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fe6000d5c20 tx=0x7fe6000d7670 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f162a61ec189abb3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:44:27.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:27.849+0000 7fe60a000640 1 -- 172.21.15.5:0/369752396 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fe6000cf870 con 0x7fe60c076720 2024-09-10T09:44:27.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:27.849+0000 7fe5fe600640 1 -- 172.21.15.5:0/2686968459 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe600097f20 con 0x561c45eb73e0 2024-09-10T09:44:27.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:27.850+0000 7fe5fe600640 1 -- 172.21.15.5:0/2686968459 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe60009c380 con 0x561c45eb73e0 2024-09-10T09:44:27.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:27.850+0000 7fe5fe600640 1 -- 172.21.15.5:0/2686968459 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe60009b5e0 con 0x561c45eb73e0 2024-09-10T09:44:27.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:27.850+0000 7fe611e00640 1 --2- 172.21.15.5:0/369752396 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fe5f40a2c00 0x7fe5f40a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:44:27.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:27.850+0000 7fe611e00640 1 --2- 172.21.15.5:0/369752396 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fe5f40a2c00 0x7fe5f40a50c0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fe604005e30 tx=0x7fe604005d80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:44:27.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:27.850+0000 7fe61686e740 1 -- 172.21.15.5:0/2686968459 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561c45eb73e0 msgr2=0x561c45ec1760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:44:27.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:27.850+0000 7fe61686e740 1 --2- 172.21.15.5:0/2686968459 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561c45eb73e0 0x561c45ec1760 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fe6000d5c20 tx=0x7fe6000d7670 comp rx=0 tx=0).stop 2024-09-10T09:44:27.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:27.851+0000 7fe61686e740 1 -- 172.21.15.5:0/2686968459 shutdown_connections 2024-09-10T09:44:27.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:27.851+0000 7fe61686e740 1 --2- 172.21.15.5:0/2686968459 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561c45ec1ca0 0x561c45ec4090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:27.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:27.851+0000 7fe61686e740 1 --2- 172.21.15.5:0/2686968459 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561c45eb73e0 0x561c45ec1760 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:27.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:27.851+0000 7fe61686e740 1 --2- 172.21.15.5:0/2686968459 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561c45eb6ac0 0x561c45eb6ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:27.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:27.851+0000 7fe61686e740 1 -- 172.21.15.5:0/2686968459 >> 172.21.15.5:0/2686968459 conn(0x561c45eb3050 msgr2=0x561c45eb3430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:44:27.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:27.851+0000 7fe61686e740 1 -- 172.21.15.5:0/2686968459 shutdown_connections 2024-09-10T09:44:27.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:27.851+0000 7fe61686e740 1 -- 172.21.15.5:0/2686968459 wait complete. 2024-09-10T09:44:27.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:27.851+0000 7fe61686e740 1 -- start start 2024-09-10T09:44:27.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:27.851+0000 7fe61686e740 1 Processor -- start 2024-09-10T09:44:27.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:27.852+0000 7fe61686e740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561c45eb6ac0 0x561c45ec1670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:44:27.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:27.852+0000 7fe61686e740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561c45eb73e0 0x561c45ebe920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:44:27.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:27.852+0000 7fe61686e740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561c45ec1ca0 0x561c45ebef10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:44:27.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:27.852+0000 7fe61686e740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x561c45ec4910 con 0x561c45eb73e0 2024-09-10T09:44:27.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:27.852+0000 7fe61686e740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x561c45dd7fa0 con 0x561c45eb6ac0 2024-09-10T09:44:27.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:27.852+0000 7fe61686e740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x561c45ec4a90 con 0x561c45ec1ca0 2024-09-10T09:44:27.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:27.852+0000 7fe61686e740 10 client.0 _ll_register_callbacks cb 0x561c45b2bba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:44:27.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:27.852+0000 7fe610a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561c45eb73e0 0x561c45ebe920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:44:27.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:27.852+0000 7fe611e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561c45ec1ca0 0x561c45ebef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:44:27.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:27.852+0000 7fe611400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561c45eb6ac0 0x561c45ec1670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:44:27.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:27.852+0000 7fe610a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561c45eb73e0 0x561c45ebe920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49826/0 (socket says 172.21.15.5:49826) 2024-09-10T09:44:27.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:27.852+0000 7fe611400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561c45eb6ac0 0x561c45ec1670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:43920/0 (socket says 172.21.15.5:43920) 2024-09-10T09:44:27.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:27.852+0000 7fe610a00640 1 -- 172.21.15.5:0/1935633696 learned_addr learned my addr 172.21.15.5:0/1935633696 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:44:27.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:27.853+0000 7fe611400640 1 -- 172.21.15.5:0/1935633696 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561c45ec1ca0 msgr2=0x561c45ebef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:44:27.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:27.853+0000 7fe611400640 1 --2- 172.21.15.5:0/1935633696 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561c45ec1ca0 0x561c45ebef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:27.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:27.853+0000 7fe611400640 1 -- 172.21.15.5:0/1935633696 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561c45eb73e0 msgr2=0x561c45ebe920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:44:27.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:27.853+0000 7fe611400640 1 --2- 172.21.15.5:0/1935633696 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561c45eb73e0 0x561c45ebe920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:27.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:27.853+0000 7fe611400640 1 -- 172.21.15.5:0/1935633696 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561c45ebf710 con 0x561c45eb6ac0 2024-09-10T09:44:27.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:27.853+0000 7fe611400640 1 --2- 172.21.15.5:0/1935633696 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561c45eb6ac0 0x561c45ec1670 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x561c45eb3eb0 tx=0x7fe6000d7f10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6e42630dd7e28564 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:44:27.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:27.853+0000 7fe5f2a00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-09-10T09:44:27.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:27.853+0000 7fe5f2a00640 1 -- 172.21.15.5:0/1935633696 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe60009b7c0 con 0x561c45eb6ac0 2024-09-10T09:44:27.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:27.853+0000 7fe5f2a00640 1 -- 172.21.15.5:0/1935633696 <== mon.2 v2:172.21.15.102:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fe6000982f0 con 0x561c45eb6ac0 2024-09-10T09:44:27.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:27.853+0000 7fe61686e740 1 -- 172.21.15.5:0/1935633696 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe60009c380 con 0x561c45eb6ac0 2024-09-10T09:44:27.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:27.853+0000 7fe5f2a00640 1 -- 172.21.15.5:0/1935633696 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fe60009ba50 con 0x561c45eb6ac0 2024-09-10T09:44:27.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:27.853+0000 7fe61686e740 10 client.26591 fetch_fsmap learned FSMap version 34 2024-09-10T09:44:27.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:27.853+0000 7fe61686e740 1 -- 172.21.15.5:0/1935633696 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x561c45ed08a0 con 0x561c45eb6ac0 2024-09-10T09:44:27.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:27.854+0000 7fe5f2a00640 1 -- 172.21.15.5:0/1935633696 <== mon.2 v2:172.21.15.102:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe60009c380 con 0x561c45eb6ac0 2024-09-10T09:44:27.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:27.854+0000 7fe5f2a00640 1 -- 172.21.15.5:0/1935633696 <== mon.2 v2:172.21.15.102:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7fe6000a4460 con 0x561c45eb6ac0 2024-09-10T09:44:27.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:27.854+0000 7fe61686e740 10 client.26591 fetch_fsmap finished waiting for FSMap version 34 2024-09-10T09:44:27.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:27.854+0000 7fe61686e740 10 client.26591 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T09:44:27.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:27.854+0000 7fe61686e740 20 client.26591 populate_metadata read hostname 'smithi005' 2024-09-10T09:44:27.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:27.854+0000 7fe61686e740 1 --2- 172.21.15.5:0/1935633696 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x561c45ed1bf0 0x561c45ed3fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:44:27.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:27.854+0000 7fe61686e740 4 client.26591 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:44:27.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:27.854+0000 7fe61686e740 1 -- 172.21.15.5:0/1935633696 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561c45e2a800 con 0x561c45ed1bf0 2024-09-10T09:44:27.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:27.855+0000 7fe610a00640 1 --2- 172.21.15.5:0/1935633696 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x561c45ed1bf0 0x561c45ed3fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:44:27.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:27.855+0000 7fe610a00640 1 --2- 172.21.15.5:0/1935633696 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x561c45ed1bf0 0x561c45ed3fe0 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f34bb0a98f4b9931 server_cookie=77a5da217491f528 in_seq=0 out_seq=0 2024-09-10T09:44:27.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:27.855+0000 7fe5f2a00640 10 client.26591 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T09:44:27.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:27.856+0000 7fe5f2a00640 1 -- 172.21.15.5:0/1935633696 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561c45e2a800 con 0x561c45ed1bf0 2024-09-10T09:44:27.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:27.856+0000 7fe5f2a00640 10 client.26591 handle_command_reply: tid=0 2024-09-10T09:44:27.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:27.856+0000 7fe61686e740 1 client.26591 shutdown 2024-09-10T09:44:27.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:27.856+0000 7fe61686e740 1 -- 172.21.15.5:0/1935633696 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561c45eb6ac0 msgr2=0x561c45ec1670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:44:27.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:27.856+0000 7fe61686e740 1 --2- 172.21.15.5:0/1935633696 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561c45eb6ac0 0x561c45ec1670 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x561c45eb3eb0 tx=0x7fe6000d7f10 comp rx=0 tx=0).stop 2024-09-10T09:44:27.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:27.856+0000 7fe61686e740 1 -- 172.21.15.5:0/1935633696 shutdown_connections 2024-09-10T09:44:27.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:27.856+0000 7fe61686e740 1 --2- 172.21.15.5:0/1935633696 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x561c45ed1bf0 0x561c45ed3fe0 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:27.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:27.856+0000 7fe61686e740 1 --2- 172.21.15.5:0/1935633696 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561c45ec1ca0 0x561c45ebef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:27.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:27.856+0000 7fe61686e740 1 --2- 172.21.15.5:0/1935633696 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561c45eb73e0 0x561c45ebe920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:27.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:27.856+0000 7fe61686e740 1 --2- 172.21.15.5:0/1935633696 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561c45eb6ac0 0x561c45ec1670 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:27.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:27.856+0000 7fe5f2a00640 0 client.26591 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T09:44:27.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:27.856+0000 7fe61686e740 1 -- 172.21.15.5:0/1935633696 >> 172.21.15.5:0/1935633696 conn(0x561c45eb3050 msgr2=0x561c45eb8d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:44:27.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:27.856+0000 7fe61686e740 1 -- 172.21.15.5:0/1935633696 shutdown_connections 2024-09-10T09:44:27.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:27.856+0000 7fe61686e740 1 -- 172.21.15.5:0/1935633696 wait complete. 2024-09-10T09:44:27.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:27.857+0000 7fe61686e740 20 client.26591 trim_cache size 0 max 16384 2024-09-10T09:44:27.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:27.877+0000 7fe61686e740 1 Processor -- start 2024-09-10T09:44:27.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:27.877+0000 7fe61686e740 1 -- start start 2024-09-10T09:44:27.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:27.878+0000 7fe61686e740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561c45eb6ac0 0x561c45ed7010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:44:27.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:27.878+0000 7fe61686e740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561c45eb73e0 0x561c45ed7550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:44:27.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:27.878+0000 7fe61686e740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561c45ec1ca0 0x561c45ebbb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:44:27.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:27.878+0000 7fe61686e740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe60c1b5520 con 0x561c45eb6ac0 2024-09-10T09:44:27.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:27.878+0000 7fe61686e740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fe60c1b7060 con 0x561c45eb73e0 2024-09-10T09:44:27.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:27.878+0000 7fe61686e740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fe60c1b5bf0 con 0x561c45ec1ca0 2024-09-10T09:44:27.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:27.878+0000 7fe611400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561c45eb6ac0 0x561c45ed7010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:44:27.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:27.878+0000 7fe611e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561c45ec1ca0 0x561c45ebbb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:44:27.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:27.878+0000 7fe611e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561c45ec1ca0 0x561c45ebbb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:60336/0 (socket says 172.21.15.5:60336) 2024-09-10T09:44:27.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:27.878+0000 7fe610a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561c45eb73e0 0x561c45ed7550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:44:27.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:27.878+0000 7fe611e00640 1 -- 172.21.15.5:0/4164332632 learned_addr learned my addr 172.21.15.5:0/4164332632 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:44:27.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:27.879+0000 7fe610a00640 1 -- 172.21.15.5:0/4164332632 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561c45ec1ca0 msgr2=0x561c45ebbb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:44:27.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:27.879+0000 7fe610a00640 1 --2- 172.21.15.5:0/4164332632 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561c45ec1ca0 0x561c45ebbb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:27.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:27.879+0000 7fe610a00640 1 -- 172.21.15.5:0/4164332632 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561c45eb6ac0 msgr2=0x561c45ed7010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:44:27.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:27.879+0000 7fe610a00640 1 --2- 172.21.15.5:0/4164332632 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561c45eb6ac0 0x561c45ed7010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:27.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:27.879+0000 7fe610a00640 1 -- 172.21.15.5:0/4164332632 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561c45ebf280 con 0x561c45eb73e0 2024-09-10T09:44:27.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:27.879+0000 7fe611e00640 1 --2- 172.21.15.5:0/4164332632 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561c45ec1ca0 0x561c45ebbb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:44:27.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:27.879+0000 7fe610a00640 1 --2- 172.21.15.5:0/4164332632 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561c45eb73e0 0x561c45ed7550 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x561c45ec4b60 tx=0x7fe5f8009e80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8a04a7bcc7304b1a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:44:27.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:27.879+0000 7fe5fdc00640 1 -- 172.21.15.5:0/4164332632 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5f8016070 con 0x561c45eb73e0 2024-09-10T09:44:27.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:27.879+0000 7fe5fdc00640 1 -- 172.21.15.5:0/4164332632 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe5f801e890 con 0x561c45eb73e0 2024-09-10T09:44:27.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:27.879+0000 7fe5fdc00640 1 -- 172.21.15.5:0/4164332632 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5f8012b20 con 0x561c45eb73e0 2024-09-10T09:44:27.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:27.880+0000 7fe61686e740 1 -- 172.21.15.5:0/4164332632 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561c45eb73e0 msgr2=0x561c45ed7550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:44:27.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:27.880+0000 7fe61686e740 1 --2- 172.21.15.5:0/4164332632 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561c45eb73e0 0x561c45ed7550 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x561c45ec4b60 tx=0x7fe5f8009e80 comp rx=0 tx=0).stop 2024-09-10T09:44:27.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:27.880+0000 7fe61686e740 1 -- 172.21.15.5:0/4164332632 shutdown_connections 2024-09-10T09:44:27.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:27.880+0000 7fe61686e740 1 --2- 172.21.15.5:0/4164332632 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561c45ec1ca0 0x561c45ebbb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:27.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:27.880+0000 7fe61686e740 1 --2- 172.21.15.5:0/4164332632 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561c45eb73e0 0x561c45ed7550 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:27.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:27.880+0000 7fe61686e740 1 --2- 172.21.15.5:0/4164332632 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561c45eb6ac0 0x561c45ed7010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:27.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:27.880+0000 7fe61686e740 1 -- 172.21.15.5:0/4164332632 >> 172.21.15.5:0/4164332632 conn(0x561c45eb3050 msgr2=0x561c45ec09a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:44:27.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:27.880+0000 7fe61686e740 1 -- 172.21.15.5:0/4164332632 shutdown_connections 2024-09-10T09:44:27.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:27.880+0000 7fe61686e740 1 -- 172.21.15.5:0/4164332632 wait complete. 2024-09-10T09:44:27.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:27.881+0000 7fe61686e740 1 -- start start 2024-09-10T09:44:27.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:27.881+0000 7fe61686e740 1 Processor -- start 2024-09-10T09:44:27.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:27.882+0000 7fe61686e740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561c45eb6ac0 0x561c45f35760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:44:27.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:27.882+0000 7fe61686e740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561c45eb73e0 0x561c45f35ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:44:27.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:27.882+0000 7fe61686e740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561c45ec1ca0 0x561c45f34500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:44:27.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:27.882+0000 7fe61686e740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe60c1b6990 con 0x561c45eb73e0 2024-09-10T09:44:27.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:27.882+0000 7fe61686e740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fe60c1b62c0 con 0x561c45eb6ac0 2024-09-10T09:44:27.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:27.882+0000 7fe61686e740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fe60c1b3500 con 0x561c45ec1ca0 2024-09-10T09:44:27.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:27.882+0000 7fe61686e740 10 client.0 _ll_register_callbacks cb 0x7fe5f8009550 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:44:27.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:27.882+0000 7fe611e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561c45ec1ca0 0x561c45f34500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:44:27.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:27.882+0000 7fe611400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561c45eb6ac0 0x561c45f35760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:44:27.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:27.882+0000 7fe611e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561c45ec1ca0 0x561c45f34500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:60350/0 (socket says 172.21.15.5:60350) 2024-09-10T09:44:27.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:27.882+0000 7fe611e00640 1 -- 172.21.15.5:0/2233309082 learned_addr learned my addr 172.21.15.5:0/2233309082 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:44:27.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:27.882+0000 7fe610a00640 1 --2- 172.21.15.5:0/2233309082 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561c45eb73e0 0x561c45f35ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:44:27.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:27.882+0000 7fe611400640 1 -- 172.21.15.5:0/2233309082 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561c45ec1ca0 msgr2=0x561c45f34500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:44:27.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:27.882+0000 7fe611400640 1 --2- 172.21.15.5:0/2233309082 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561c45ec1ca0 0x561c45f34500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:27.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:27.882+0000 7fe611400640 1 -- 172.21.15.5:0/2233309082 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561c45eb73e0 msgr2=0x561c45f35ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:44:27.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:27.882+0000 7fe611400640 1 --2- 172.21.15.5:0/2233309082 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561c45eb73e0 0x561c45f35ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:27.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:27.882+0000 7fe611400640 1 -- 172.21.15.5:0/2233309082 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561c45f34b40 con 0x561c45eb6ac0 2024-09-10T09:44:27.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:27.882+0000 7fe610a00640 1 --2- 172.21.15.5:0/2233309082 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561c45eb73e0 0x561c45f35ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:44:27.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:27.883+0000 7fe611400640 1 --2- 172.21.15.5:0/2233309082 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561c45eb6ac0 0x561c45f35760 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fe60000e9c0 tx=0x7fe60009a190 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a4e2fc4150b4e41c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:44:27.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:27.883+0000 7fe5f2a00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-09-10T09:44:27.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:27.883+0000 7fe5f2a00640 1 -- 172.21.15.5:0/2233309082 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe60009b7c0 con 0x561c45eb6ac0 2024-09-10T09:44:27.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:27.883+0000 7fe5f2a00640 1 -- 172.21.15.5:0/2233309082 <== mon.2 v2:172.21.15.102:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fe6000d5300 con 0x561c45eb6ac0 2024-09-10T09:44:27.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:27.883+0000 7fe61686e740 1 -- 172.21.15.5:0/2233309082 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe5f801e890 con 0x561c45eb6ac0 2024-09-10T09:44:27.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:27.883+0000 7fe5f2a00640 1 -- 172.21.15.5:0/2233309082 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fe6000ad050 con 0x561c45eb6ac0 2024-09-10T09:44:27.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:27.883+0000 7fe61686e740 10 client.26597 fetch_fsmap learned FSMap version 34 2024-09-10T09:44:27.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:27.883+0000 7fe61686e740 1 -- 172.21.15.5:0/2233309082 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x561c45f34b40 con 0x561c45eb6ac0 2024-09-10T09:44:27.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:27.883+0000 7fe5f2a00640 1 -- 172.21.15.5:0/2233309082 <== mon.2 v2:172.21.15.102:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe5f801e890 con 0x561c45eb6ac0 2024-09-10T09:44:27.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:27.884+0000 7fe5f2a00640 1 -- 172.21.15.5:0/2233309082 <== mon.2 v2:172.21.15.102:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7fe60009bee0 con 0x561c45eb6ac0 2024-09-10T09:44:27.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:27.884+0000 7fe61686e740 10 client.26597 fetch_fsmap finished waiting for FSMap version 34 2024-09-10T09:44:27.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:27.884+0000 7fe61686e740 10 client.26597 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T09:44:27.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:27.884+0000 7fe61686e740 20 client.26597 populate_metadata read hostname 'smithi005' 2024-09-10T09:44:27.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:27.884+0000 7fe61686e740 1 --2- 172.21.15.5:0/2233309082 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x561c45f3d3d0 0x561c45f3f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:44:27.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:27.884+0000 7fe61686e740 4 client.26597 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-10T09:44:27.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:27.884+0000 7fe61686e740 1 -- 172.21.15.5:0/2233309082 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x561c45e24d10 con 0x561c45f3d3d0 2024-09-10T09:44:27.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:27.884+0000 7fe610a00640 1 --2- 172.21.15.5:0/2233309082 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x561c45f3d3d0 0x561c45f3f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:44:27.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:27.885+0000 7fe610a00640 1 --2- 172.21.15.5:0/2233309082 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x561c45f3d3d0 0x561c45f3f7c0 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3130bc97a898c00c server_cookie=a44a2cc9f25f061a in_seq=0 out_seq=0 2024-09-10T09:44:27.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:27.885+0000 7fe5f2a00640 10 client.26597 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T09:44:27.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:27.885+0000 7fe5f2a00640 1 -- 172.21.15.5:0/2233309082 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x561c45e24d10 con 0x561c45f3d3d0 2024-09-10T09:44:27.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:27.885+0000 7fe5f2a00640 10 client.26597 handle_command_reply: tid=0 2024-09-10T09:44:27.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:27.885+0000 7fe61686e740 1 client.26597 shutdown 2024-09-10T09:44:27.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:27.886+0000 7fe61686e740 1 -- 172.21.15.5:0/2233309082 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561c45eb6ac0 msgr2=0x561c45f35760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:44:27.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:27.886+0000 7fe61686e740 1 --2- 172.21.15.5:0/2233309082 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561c45eb6ac0 0x561c45f35760 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fe60000e9c0 tx=0x7fe60009a190 comp rx=0 tx=0).stop 2024-09-10T09:44:27.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:27.886+0000 7fe61686e740 1 -- 172.21.15.5:0/2233309082 shutdown_connections 2024-09-10T09:44:27.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:27.886+0000 7fe61686e740 1 --2- 172.21.15.5:0/2233309082 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x561c45f3d3d0 0x561c45f3f7c0 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:27.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:27.886+0000 7fe61686e740 1 --2- 172.21.15.5:0/2233309082 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561c45ec1ca0 0x561c45f34500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:27.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:27.886+0000 7fe61686e740 1 --2- 172.21.15.5:0/2233309082 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561c45eb73e0 0x561c45f35ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:27.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:27.886+0000 7fe5f2a00640 0 client.26597 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T09:44:27.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:27.886+0000 7fe61686e740 1 --2- 172.21.15.5:0/2233309082 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561c45eb6ac0 0x561c45f35760 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:27.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:27.886+0000 7fe61686e740 1 -- 172.21.15.5:0/2233309082 >> 172.21.15.5:0/2233309082 conn(0x561c45eb3050 msgr2=0x561c45eccce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:44:27.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:27.886+0000 7fe61686e740 1 -- 172.21.15.5:0/2233309082 shutdown_connections 2024-09-10T09:44:27.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:27.886+0000 7fe61686e740 1 -- 172.21.15.5:0/2233309082 wait complete. 2024-09-10T09:44:27.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:27.886+0000 7fe61686e740 20 client.26597 trim_cache size 0 max 16384 2024-09-10T09:44:27.890 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-10T09:44:27.890 INFO:teuthology.orchestra.run.smithi005.stdout: "return_code": 0, 2024-09-10T09:44:27.891 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_tag": "542eb324-daa1-4175-b9c0-0acfdeca248f", 2024-09-10T09:44:27.891 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": "asynchronous" 2024-09-10T09:44:27.891 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-10T09:44:27.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:27.887+0000 7fe613e00640 1 -- 172.21.15.5:0/369752396 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fe5f40a2c00 msgr2=0x7fe5f40a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:44:27.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:27.887+0000 7fe613e00640 1 --2- 172.21.15.5:0/369752396 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fe5f40a2c00 0x7fe5f40a50c0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fe604005e30 tx=0x7fe604005d80 comp rx=0 tx=0).stop 2024-09-10T09:44:27.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:27.887+0000 7fe613e00640 1 -- 172.21.15.5:0/369752396 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe60c076720 msgr2=0x7fe60c19eb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:44:27.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:27.887+0000 7fe613e00640 1 --2- 172.21.15.5:0/369752396 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe60c076720 0x7fe60c19eb80 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fe60000e9f0 tx=0x7fe60000eeb0 comp rx=0 tx=0).stop 2024-09-10T09:44:27.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:27.887+0000 7fe613e00640 1 -- 172.21.15.5:0/369752396 shutdown_connections 2024-09-10T09:44:27.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:27.887+0000 7fe613e00640 1 --2- 172.21.15.5:0/369752396 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fe5f40a2c00 0x7fe5f40a50c0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:27.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:27.887+0000 7fe613e00640 1 --2- 172.21.15.5:0/369752396 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe60c112a50 0x7fe60c1a3480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:27.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:27.887+0000 7fe613e00640 1 --2- 172.21.15.5:0/369752396 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe60c077040 0x7fe60c19f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:27.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:27.887+0000 7fe613e00640 1 --2- 172.21.15.5:0/369752396 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe60c076720 0x7fe60c19eb80 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:27.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:27.887+0000 7fe613e00640 1 -- 172.21.15.5:0/369752396 >> 172.21.15.5:0/369752396 conn(0x7fe60c0ffa50 msgr2=0x7fe60c101e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:44:27.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:27.887+0000 7fe613e00640 1 -- 172.21.15.5:0/369752396 shutdown_connections 2024-09-10T09:44:27.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:27.888+0000 7fe613e00640 1 -- 172.21.15.5:0/369752396 wait complete. 2024-09-10T09:44:28.044 INFO:tasks.workunit.client.0.smithi005.stdout: 1 340958 6.22 MB/sec execute 129 sec latency 10.448 ms 2024-09-10T09:44:29.037 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T09:44:29.044 INFO:tasks.workunit.client.0.smithi005.stdout: 1 343017 6.24 MB/sec execute 130 sec latency 34.669 ms 2024-09-10T09:44:29.087 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:44:28 smithi005 ceph-mon[30350]: pgmap v577: 129 pgs: 129 active+clean; 140 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.3 MiB/s wr, 193 op/s 2024-09-10T09:44:29.088 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:44:28 smithi005 ceph-mon[30350]: scrub queued for path: / 2024-09-10T09:44:29.088 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:44:28 smithi005 ceph-mon[30350]: scrub summary: idle+waiting paths [/] 2024-09-10T09:44:29.088 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:44:28 smithi005 ceph-mon[30350]: scrub summary: active paths [/] 2024-09-10T09:44:29.088 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:44:28 smithi005 ceph-mon[30350]: scrub summary: active paths [/] 2024-09-10T09:44:29.088 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:44:28 smithi005 ceph-mon[30350]: scrub summary: idle+waiting paths [/] 2024-09-10T09:44:29.088 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:44:28 smithi005 ceph-mon[30350]: scrub summary: idle+waiting paths [/] 2024-09-10T09:44:29.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:44:28 smithi159 ceph-mon[35513]: pgmap v577: 129 pgs: 129 active+clean; 140 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.3 MiB/s wr, 193 op/s 2024-09-10T09:44:29.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:44:28 smithi159 ceph-mon[35513]: scrub queued for path: / 2024-09-10T09:44:29.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:44:28 smithi159 ceph-mon[35513]: scrub summary: idle+waiting paths [/] 2024-09-10T09:44:29.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:44:28 smithi159 ceph-mon[35513]: scrub summary: active paths [/] 2024-09-10T09:44:29.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:44:28 smithi159 ceph-mon[35513]: scrub summary: active paths [/] 2024-09-10T09:44:29.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:44:28 smithi159 ceph-mon[35513]: scrub summary: idle+waiting paths [/] 2024-09-10T09:44:29.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:44:28 smithi159 ceph-mon[35513]: scrub summary: idle+waiting paths [/] 2024-09-10T09:44:29.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:44:28 smithi102 ceph-mon[35442]: pgmap v577: 129 pgs: 129 active+clean; 140 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.3 MiB/s wr, 193 op/s 2024-09-10T09:44:29.135 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:44:28 smithi102 ceph-mon[35442]: scrub queued for path: / 2024-09-10T09:44:29.135 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:44:28 smithi102 ceph-mon[35442]: scrub summary: idle+waiting paths [/] 2024-09-10T09:44:29.135 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:44:28 smithi102 ceph-mon[35442]: scrub summary: active paths [/] 2024-09-10T09:44:29.135 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:44:28 smithi102 ceph-mon[35442]: scrub summary: active paths [/] 2024-09-10T09:44:29.135 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:44:28 smithi102 ceph-mon[35442]: scrub summary: idle+waiting paths [/] 2024-09-10T09:44:29.135 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:44:28 smithi102 ceph-mon[35442]: scrub summary: idle+waiting paths [/] 2024-09-10T09:44:29.437 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:44:30.046 INFO:tasks.workunit.client.0.smithi005.stdout: 1 344870 6.28 MB/sec execute 131 sec latency 99.936 ms 2024-09-10T09:44:30.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:30.070+0000 7f2bd7200640 1 -- 172.21.15.5:0/1100373420 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2bd810e2d0 msgr2=0x7f2bd81106c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:44:30.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:30.070+0000 7f2bd7200640 1 --2- 172.21.15.5:0/1100373420 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2bd810e2d0 0x7f2bd81106c0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f2bd0009ca0 tx=0x7f2bd002a8c0 comp rx=0 tx=0).stop 2024-09-10T09:44:30.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:30.071+0000 7f2bd7200640 1 -- 172.21.15.5:0/1100373420 shutdown_connections 2024-09-10T09:44:30.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:30.071+0000 7f2bd7200640 1 --2- 172.21.15.5:0/1100373420 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2bd810e2d0 0x7f2bd81106c0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:30.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:30.071+0000 7f2bd7200640 1 --2- 172.21.15.5:0/1100373420 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2bd81010d0 0x7f2bd810dd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:30.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:30.071+0000 7f2bd7200640 1 --2- 172.21.15.5:0/1100373420 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2bd8100720 0x7f2bd8100b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:30.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:30.071+0000 7f2bd7200640 1 -- 172.21.15.5:0/1100373420 >> 172.21.15.5:0/1100373420 conn(0x7f2bd80fc610 msgr2=0x7f2bd80fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:44:30.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:30.071+0000 7f2bd7200640 1 -- 172.21.15.5:0/1100373420 shutdown_connections 2024-09-10T09:44:30.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:30.071+0000 7f2bd7200640 1 -- 172.21.15.5:0/1100373420 wait complete. 2024-09-10T09:44:30.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:30.072+0000 7f2bd7200640 1 Processor -- start 2024-09-10T09:44:30.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:30.072+0000 7f2bd7200640 1 -- start start 2024-09-10T09:44:30.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:30.072+0000 7f2bd7200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2bd8100720 0x7f2bd810dc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:44:30.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:30.072+0000 7f2bd7200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2bd81010d0 0x7f2bd810acf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:44:30.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:30.072+0000 7f2bd7200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2bd810e2d0 0x7f2bd810b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:44:30.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:30.073+0000 7f2bd7200640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f2bd8114020 con 0x7f2bd810e2d0 2024-09-10T09:44:30.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:30.073+0000 7f2bd7200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f2bd8113ea0 con 0x7f2bd8100720 2024-09-10T09:44:30.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:30.073+0000 7f2bd7200640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f2bd81141a0 con 0x7f2bd81010d0 2024-09-10T09:44:30.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:30.073+0000 7f2bcfe00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2bd81010d0 0x7f2bd810acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:44:30.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:30.073+0000 7f2bcfe00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2bd81010d0 0x7f2bd810acf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:60370/0 (socket says 172.21.15.5:60370) 2024-09-10T09:44:30.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:30.073+0000 7f2bcfe00640 1 -- 172.21.15.5:0/1645134387 learned_addr learned my addr 172.21.15.5:0/1645134387 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:44:30.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:30.073+0000 7f2bd5e00640 1 --2- 172.21.15.5:0/1645134387 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2bd8100720 0x7f2bd810dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:44:30.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:30.073+0000 7f2bd6800640 1 --2- 172.21.15.5:0/1645134387 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2bd810e2d0 0x7f2bd810b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:44:30.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:30.073+0000 7f2bcfe00640 1 -- 172.21.15.5:0/1645134387 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2bd8100720 msgr2=0x7f2bd810dc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:44:30.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:30.073+0000 7f2bcfe00640 1 --2- 172.21.15.5:0/1645134387 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2bd8100720 0x7f2bd810dc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:30.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:30.073+0000 7f2bcfe00640 1 -- 172.21.15.5:0/1645134387 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2bd810e2d0 msgr2=0x7f2bd810b250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:44:30.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:30.073+0000 7f2bcfe00640 1 --2- 172.21.15.5:0/1645134387 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2bd810e2d0 0x7f2bd810b250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:30.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:30.073+0000 7f2bcfe00640 1 -- 172.21.15.5:0/1645134387 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2bd810bb10 con 0x7f2bd81010d0 2024-09-10T09:44:30.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:30.073+0000 7f2bd5e00640 1 --2- 172.21.15.5:0/1645134387 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2bd8100720 0x7f2bd810dc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:44:30.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:30.073+0000 7f2bd6800640 1 --2- 172.21.15.5:0/1645134387 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2bd810e2d0 0x7f2bd810b250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T09:44:30.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:30.074+0000 7f2bcfe00640 1 --2- 172.21.15.5:0/1645134387 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2bd81010d0 0x7f2bd810acf0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f2bc000e9f0 tx=0x7f2bc000edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:44:30.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:30.074+0000 7f2bcea00640 1 -- 172.21.15.5:0/1645134387 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bc000cd80 con 0x7f2bd81010d0 2024-09-10T09:44:30.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:30.074+0000 7f2bcea00640 1 -- 172.21.15.5:0/1645134387 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2bc0005500 con 0x7f2bd81010d0 2024-09-10T09:44:30.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:30.074+0000 7f2bcea00640 1 -- 172.21.15.5:0/1645134387 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bc0010690 con 0x7f2bd81010d0 2024-09-10T09:44:30.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:30.074+0000 7f2bd7200640 1 -- 172.21.15.5:0/1645134387 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2bd81aae60 con 0x7f2bd81010d0 2024-09-10T09:44:30.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:30.074+0000 7f2bd7200640 1 -- 172.21.15.5:0/1645134387 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2bd81ab320 con 0x7f2bd81010d0 2024-09-10T09:44:30.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:30.078+0000 7f2bdf3e2740 1 Processor -- start 2024-09-10T09:44:30.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:30.078+0000 7f2bdf3e2740 1 -- start start 2024-09-10T09:44:30.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:30.079+0000 7f2bcea00640 1 -- 172.21.15.5:0/1645134387 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f2bc00046c0 con 0x7f2bd81010d0 2024-09-10T09:44:30.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:30.079+0000 7f2bdf3e2740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5607fe30fd50 0x5607fe310130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:44:30.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:30.079+0000 7f2bdf3e2740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5607fe310670 0x5607fe31a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:44:30.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:30.080+0000 7f2bdf3e2740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5607fe31af30 0x5607fe31d320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:44:30.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:30.080+0000 7f2bdf3e2740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5607fe2763e0 con 0x5607fe310670 2024-09-10T09:44:30.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:30.080+0000 7f2bdf3e2740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x5607fe21dd80 con 0x5607fe31af30 2024-09-10T09:44:30.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:30.080+0000 7f2bdf3e2740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5607fe2723a0 con 0x5607fe30fd50 2024-09-10T09:44:30.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:30.080+0000 7f2bd5e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5607fe310670 0x5607fe31a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:44:30.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:30.080+0000 7f2bd5e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5607fe310670 0x5607fe31a9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49856/0 (socket says 172.21.15.5:49856) 2024-09-10T09:44:30.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:30.080+0000 7f2bd5e00640 1 -- 172.21.15.5:0/529128178 learned_addr learned my addr 172.21.15.5:0/529128178 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:44:30.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:30.080+0000 7f2bd6800640 1 --2- 172.21.15.5:0/529128178 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5607fe30fd50 0x5607fe310130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:44:30.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:30.080+0000 7f2bcfe00640 1 --2- 172.21.15.5:0/529128178 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5607fe31af30 0x5607fe31d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:44:30.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:30.080+0000 7f2bd5e00640 1 -- 172.21.15.5:0/529128178 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5607fe30fd50 msgr2=0x5607fe310130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:44:30.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:30.081+0000 7f2bd5e00640 1 --2- 172.21.15.5:0/529128178 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5607fe30fd50 0x5607fe310130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:30.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:30.081+0000 7f2bd5e00640 1 -- 172.21.15.5:0/529128178 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5607fe31af30 msgr2=0x5607fe31d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:44:30.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:30.081+0000 7f2bd5e00640 1 --2- 172.21.15.5:0/529128178 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5607fe31af30 0x5607fe31d320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:30.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:30.081+0000 7f2bd5e00640 1 -- 172.21.15.5:0/529128178 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5607fe23cae0 con 0x5607fe310670 2024-09-10T09:44:30.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:30.081+0000 7f2bd6800640 1 --2- 172.21.15.5:0/529128178 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5607fe30fd50 0x5607fe310130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T09:44:30.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:30.081+0000 7f2bd5e00640 1 --2- 172.21.15.5:0/529128178 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5607fe310670 0x5607fe31a9f0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f2bc8009d60 tx=0x7f2bc80117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=392f4b883b536cea server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:44:30.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:30.082+0000 7f2bc7000640 1 -- 172.21.15.5:0/529128178 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bc8012650 con 0x5607fe310670 2024-09-10T09:44:30.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:30.082+0000 7f2bc7000640 1 -- 172.21.15.5:0/529128178 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2bc8009e50 con 0x5607fe310670 2024-09-10T09:44:30.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:30.082+0000 7f2bc7000640 1 -- 172.21.15.5:0/529128178 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bc8012650 con 0x5607fe310670 2024-09-10T09:44:30.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:30.082+0000 7f2bcea00640 1 --2- 172.21.15.5:0/1645134387 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f2bbc0a2c00 0x7f2bbc0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:44:30.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:30.082+0000 7f2bcea00640 1 -- 172.21.15.5:0/1645134387 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f2bc00cf8c0 con 0x7f2bd81010d0 2024-09-10T09:44:30.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:30.083+0000 7f2bd6800640 1 --2- 172.21.15.5:0/1645134387 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f2bbc0a2c00 0x7f2bbc0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:44:30.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:30.083+0000 7f2bdf3e2740 1 -- 172.21.15.5:0/529128178 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5607fe310670 msgr2=0x5607fe31a9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:44:30.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:30.083+0000 7f2bdf3e2740 1 --2- 172.21.15.5:0/529128178 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5607fe310670 0x5607fe31a9f0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f2bc8009d60 tx=0x7f2bc80117b0 comp rx=0 tx=0).stop 2024-09-10T09:44:30.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:30.083+0000 7f2bdf3e2740 1 -- 172.21.15.5:0/529128178 shutdown_connections 2024-09-10T09:44:30.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:30.083+0000 7f2bdf3e2740 1 --2- 172.21.15.5:0/529128178 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5607fe31af30 0x5607fe31d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:30.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:30.083+0000 7f2bdf3e2740 1 --2- 172.21.15.5:0/529128178 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5607fe310670 0x5607fe31a9f0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:30.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:30.083+0000 7f2bdf3e2740 1 --2- 172.21.15.5:0/529128178 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5607fe30fd50 0x5607fe310130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:30.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:30.083+0000 7f2bdf3e2740 1 -- 172.21.15.5:0/529128178 >> 172.21.15.5:0/529128178 conn(0x5607fe30c2e0 msgr2=0x5607fe30c6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:44:30.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:30.084+0000 7f2bd6800640 1 --2- 172.21.15.5:0/1645134387 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f2bbc0a2c00 0x7f2bbc0a50c0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f2bd002ade0 tx=0x7f2bd002f400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:44:30.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:30.084+0000 7f2bdf3e2740 1 -- 172.21.15.5:0/529128178 shutdown_connections 2024-09-10T09:44:30.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:30.084+0000 7f2bdf3e2740 1 -- 172.21.15.5:0/529128178 wait complete. 2024-09-10T09:44:30.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:30.085+0000 7f2bdf3e2740 1 -- start start 2024-09-10T09:44:30.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:30.085+0000 7f2bdf3e2740 1 Processor -- start 2024-09-10T09:44:30.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:30.085+0000 7f2bdf3e2740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5607fe30fd50 0x5607fe317e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:44:30.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:30.085+0000 7f2bdf3e2740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5607fe310670 0x5607fe318380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:44:30.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:30.085+0000 7f2bdf3e2740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5607fe31af30 0x5607fe3188c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:44:30.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:30.085+0000 7f2bdf3e2740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5607fe31dba0 con 0x5607fe31af30 2024-09-10T09:44:30.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:30.085+0000 7f2bdf3e2740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x5607fe2310f0 con 0x5607fe30fd50 2024-09-10T09:44:30.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:30.085+0000 7f2bdf3e2740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5607fe31dd20 con 0x5607fe310670 2024-09-10T09:44:30.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:30.085+0000 7f2bdf3e2740 10 client.0 _ll_register_callbacks cb 0x5607fe2229e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:44:30.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:30.086+0000 7f2bd6800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5607fe31af30 0x5607fe3188c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:44:30.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:30.086+0000 7f2bd5e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5607fe30fd50 0x5607fe317e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:44:30.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:30.086+0000 7f2bd6800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5607fe31af30 0x5607fe3188c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49862/0 (socket says 172.21.15.5:49862) 2024-09-10T09:44:30.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:30.086+0000 7f2bd6800640 1 -- 172.21.15.5:0/1807507755 learned_addr learned my addr 172.21.15.5:0/1807507755 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:44:30.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:30.086+0000 7f2bd6800640 1 -- 172.21.15.5:0/1807507755 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5607fe310670 msgr2=0x5607fe318380 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T09:44:30.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:30.086+0000 7f2bd6800640 1 --2- 172.21.15.5:0/1807507755 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5607fe310670 0x5607fe318380 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:30.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:30.086+0000 7f2bd6800640 1 -- 172.21.15.5:0/1807507755 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5607fe30fd50 msgr2=0x5607fe317e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:44:30.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:30.086+0000 7f2bd6800640 1 --2- 172.21.15.5:0/1807507755 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5607fe30fd50 0x5607fe317e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:30.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:30.086+0000 7f2bd6800640 1 -- 172.21.15.5:0/1807507755 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5607fdfe73c0 con 0x5607fe31af30 2024-09-10T09:44:30.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:30.086+0000 7f2bd5e00640 1 --2- 172.21.15.5:0/1807507755 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5607fe30fd50 0x5607fe317e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T09:44:30.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:30.087+0000 7f2bd6800640 1 --2- 172.21.15.5:0/1807507755 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5607fe31af30 0x5607fe3188c0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f2bd0009ca0 tx=0x7f2bd00095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75cb113df68776fe server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:44:30.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:30.087+0000 7f2bbb400640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-10T09:44:30.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:30.087+0000 7f2bbb400640 1 -- 172.21.15.5:0/1807507755 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bd002f810 con 0x5607fe31af30 2024-09-10T09:44:30.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:30.087+0000 7f2bbb400640 1 -- 172.21.15.5:0/1807507755 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f2bd002f9b0 con 0x5607fe31af30 2024-09-10T09:44:30.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:30.087+0000 7f2bdf3e2740 1 -- 172.21.15.5:0/1807507755 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2bc8009e50 con 0x5607fe31af30 2024-09-10T09:44:30.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:30.088+0000 7f2bbb400640 1 -- 172.21.15.5:0/1807507755 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f2bd002fbb0 con 0x5607fe31af30 2024-09-10T09:44:30.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:30.088+0000 7f2bdf3e2740 10 client.16587 fetch_fsmap learned FSMap version 34 2024-09-10T09:44:30.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:30.088+0000 7f2bdf3e2740 1 -- 172.21.15.5:0/1807507755 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5607fe3297f0 con 0x5607fe31af30 2024-09-10T09:44:30.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:30.088+0000 7f2bbb400640 1 -- 172.21.15.5:0/1807507755 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2bc8009e50 con 0x5607fe31af30 2024-09-10T09:44:30.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:30.089+0000 7f2bbb400640 1 -- 172.21.15.5:0/1807507755 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f2bd00305c0 con 0x5607fe31af30 2024-09-10T09:44:30.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:30.089+0000 7f2bdf3e2740 10 client.16587 fetch_fsmap finished waiting for FSMap version 34 2024-09-10T09:44:30.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:30.089+0000 7f2bdf3e2740 10 client.16587 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T09:44:30.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:30.089+0000 7f2bdf3e2740 20 client.16587 populate_metadata read hostname 'smithi005' 2024-09-10T09:44:30.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:30.089+0000 7f2bdf3e2740 1 --2- 172.21.15.5:0/1807507755 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x5607fe32acb0 0x5607fe32d100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:44:30.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:30.089+0000 7f2bdf3e2740 4 client.16587 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:44:30.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:30.089+0000 7f2bdf3e2740 1 -- 172.21.15.5:0/1807507755 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5607fe283950 con 0x5607fe32acb0 2024-09-10T09:44:30.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:30.089+0000 7f2bd5e00640 1 --2- 172.21.15.5:0/1807507755 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x5607fe32acb0 0x5607fe32d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:44:30.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:30.090+0000 7f2bd5e00640 1 --2- 172.21.15.5:0/1807507755 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x5607fe32acb0 0x5607fe32d100 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7a5771079c1f8a0d server_cookie=9c5fc41ab50aecc0 in_seq=0 out_seq=0 2024-09-10T09:44:30.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:30.090+0000 7f2bbb400640 10 client.16587 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T09:44:30.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:30.091+0000 7f2bbb400640 1 -- 172.21.15.5:0/1807507755 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5607fe283950 con 0x5607fe32acb0 2024-09-10T09:44:30.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:30.091+0000 7f2bbb400640 10 client.16587 handle_command_reply: tid=0 2024-09-10T09:44:30.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:30.091+0000 7f2bdf3e2740 1 client.16587 shutdown 2024-09-10T09:44:30.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:30.091+0000 7f2bdf3e2740 1 -- 172.21.15.5:0/1807507755 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5607fe31af30 msgr2=0x5607fe3188c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:44:30.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:30.091+0000 7f2bdf3e2740 1 --2- 172.21.15.5:0/1807507755 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5607fe31af30 0x5607fe3188c0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f2bd0009ca0 tx=0x7f2bd00095c0 comp rx=0 tx=0).stop 2024-09-10T09:44:30.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:30.091+0000 7f2bdf3e2740 1 -- 172.21.15.5:0/1807507755 shutdown_connections 2024-09-10T09:44:30.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:30.091+0000 7f2bdf3e2740 1 --2- 172.21.15.5:0/1807507755 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x5607fe32acb0 0x5607fe32d100 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:30.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:30.091+0000 7f2bdf3e2740 1 --2- 172.21.15.5:0/1807507755 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5607fe31af30 0x5607fe3188c0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:30.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:30.091+0000 7f2bdf3e2740 1 --2- 172.21.15.5:0/1807507755 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5607fe310670 0x5607fe318380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:30.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:30.091+0000 7f2bdf3e2740 1 --2- 172.21.15.5:0/1807507755 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5607fe30fd50 0x5607fe317e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:30.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:30.091+0000 7f2bdf3e2740 1 -- 172.21.15.5:0/1807507755 >> 172.21.15.5:0/1807507755 conn(0x5607fe30c2e0 msgr2=0x5607fe311fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:44:30.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:30.091+0000 7f2bbb400640 0 client.16587 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T09:44:30.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:30.091+0000 7f2bdf3e2740 1 -- 172.21.15.5:0/1807507755 shutdown_connections 2024-09-10T09:44:30.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:30.092+0000 7f2bdf3e2740 1 -- 172.21.15.5:0/1807507755 wait complete. 2024-09-10T09:44:30.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:30.092+0000 7f2bdf3e2740 20 client.16587 trim_cache size 0 max 16384 2024-09-10T09:44:30.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:30.115+0000 7f2bdf3e2740 1 Processor -- start 2024-09-10T09:44:30.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:30.115+0000 7f2bdf3e2740 1 -- start start 2024-09-10T09:44:30.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:30.115+0000 7f2bdf3e2740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5607fe30fd50 0x5607fe3127b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:44:30.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:30.115+0000 7f2bdf3e2740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5607fe310670 0x5607fe316710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:44:30.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:30.115+0000 7f2bdf3e2740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5607fe31af30 0x5607fe3329d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:44:30.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:30.115+0000 7f2bdf3e2740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f2bd81b5340 con 0x5607fe31af30 2024-09-10T09:44:30.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:30.115+0000 7f2bdf3e2740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f2bd81b6e80 con 0x5607fe310670 2024-09-10T09:44:30.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:30.115+0000 7f2bdf3e2740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f2bd81b5a10 con 0x5607fe30fd50 2024-09-10T09:44:30.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:30.116+0000 7f2bd6800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5607fe31af30 0x5607fe3329d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:44:30.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:30.116+0000 7f2bd6800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5607fe31af30 0x5607fe3329d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49876/0 (socket says 172.21.15.5:49876) 2024-09-10T09:44:30.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:30.116+0000 7f2bd6800640 1 -- 172.21.15.5:0/2690187886 learned_addr learned my addr 172.21.15.5:0/2690187886 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:44:30.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:30.116+0000 7f2bcfe00640 1 --2- 172.21.15.5:0/2690187886 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5607fe310670 0x5607fe316710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:44:30.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:30.116+0000 7f2bd6800640 1 -- 172.21.15.5:0/2690187886 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5607fe30fd50 msgr2=0x5607fe3127b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T09:44:30.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:30.116+0000 7f2bd6800640 1 --2- 172.21.15.5:0/2690187886 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5607fe30fd50 0x5607fe3127b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:30.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:30.116+0000 7f2bd6800640 1 -- 172.21.15.5:0/2690187886 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5607fe310670 msgr2=0x5607fe316710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:44:30.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:30.116+0000 7f2bd6800640 1 --2- 172.21.15.5:0/2690187886 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5607fe310670 0x5607fe316710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:30.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:30.116+0000 7f2bd6800640 1 -- 172.21.15.5:0/2690187886 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5607fe316df0 con 0x5607fe31af30 2024-09-10T09:44:30.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:30.116+0000 7f2bd6800640 1 --2- 172.21.15.5:0/2690187886 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5607fe31af30 0x5607fe3329d0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f2bd0031fd0 tx=0x7f2bd0009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2a421fec0da6735 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:44:30.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:30.117+0000 7f2bc6600640 1 -- 172.21.15.5:0/2690187886 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bd0069070 con 0x5607fe31af30 2024-09-10T09:44:30.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:30.117+0000 7f2bc6600640 1 -- 172.21.15.5:0/2690187886 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2bd0036040 con 0x5607fe31af30 2024-09-10T09:44:30.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:30.117+0000 7f2bc6600640 1 -- 172.21.15.5:0/2690187886 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bd0060070 con 0x5607fe31af30 2024-09-10T09:44:30.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:30.118+0000 7f2bdf3e2740 1 -- 172.21.15.5:0/2690187886 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5607fe31af30 msgr2=0x5607fe3329d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:44:30.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:30.118+0000 7f2bdf3e2740 1 --2- 172.21.15.5:0/2690187886 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5607fe31af30 0x5607fe3329d0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f2bd0031fd0 tx=0x7f2bd0009c30 comp rx=0 tx=0).stop 2024-09-10T09:44:30.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:30.118+0000 7f2bdf3e2740 1 -- 172.21.15.5:0/2690187886 shutdown_connections 2024-09-10T09:44:30.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:30.118+0000 7f2bdf3e2740 1 --2- 172.21.15.5:0/2690187886 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5607fe31af30 0x5607fe3329d0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:30.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:30.118+0000 7f2bdf3e2740 1 --2- 172.21.15.5:0/2690187886 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5607fe310670 0x5607fe316710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:30.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:30.118+0000 7f2bdf3e2740 1 --2- 172.21.15.5:0/2690187886 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5607fe30fd50 0x5607fe3127b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:30.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:30.118+0000 7f2bdf3e2740 1 -- 172.21.15.5:0/2690187886 >> 172.21.15.5:0/2690187886 conn(0x5607fe30c2e0 msgr2=0x5607fe319ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:44:30.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:30.118+0000 7f2bdf3e2740 1 -- 172.21.15.5:0/2690187886 shutdown_connections 2024-09-10T09:44:30.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:30.118+0000 7f2bdf3e2740 1 -- 172.21.15.5:0/2690187886 wait complete. 2024-09-10T09:44:30.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:30.119+0000 7f2bdf3e2740 1 -- start start 2024-09-10T09:44:30.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:30.119+0000 7f2bdf3e2740 1 Processor -- start 2024-09-10T09:44:30.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:30.119+0000 7f2bdf3e2740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5607fe30fd50 0x5607fe323810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:44:30.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:30.119+0000 7f2bdf3e2740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5607fe310670 0x5607fe31eb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:44:30.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:30.120+0000 7f2bdf3e2740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5607fe31af30 0x5607fe31f2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:44:30.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:30.120+0000 7f2bdf3e2740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f2bd81b67b0 con 0x5607fe31af30 2024-09-10T09:44:30.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:30.120+0000 7f2bdf3e2740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f2bd81b60e0 con 0x5607fe310670 2024-09-10T09:44:30.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:30.120+0000 7f2bdf3e2740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f2bd81b3320 con 0x5607fe30fd50 2024-09-10T09:44:30.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:30.120+0000 7f2bdf3e2740 10 client.0 _ll_register_callbacks cb 0x7f2bd0009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:44:30.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:30.120+0000 7f2bd6800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5607fe31af30 0x5607fe31f2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:44:30.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:30.120+0000 7f2bcfe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5607fe310670 0x5607fe31eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:44:30.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:30.120+0000 7f2bcfe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5607fe310670 0x5607fe31eb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:43994/0 (socket says 172.21.15.5:43994) 2024-09-10T09:44:30.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:30.120+0000 7f2bcfe00640 1 -- 172.21.15.5:0/2216472609 learned_addr learned my addr 172.21.15.5:0/2216472609 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:44:30.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:30.120+0000 7f2bd5e00640 1 --2- 172.21.15.5:0/2216472609 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5607fe30fd50 0x5607fe323810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:44:30.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:30.121+0000 7f2bcfe00640 1 -- 172.21.15.5:0/2216472609 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5607fe30fd50 msgr2=0x5607fe323810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:44:30.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:30.121+0000 7f2bcfe00640 1 --2- 172.21.15.5:0/2216472609 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5607fe30fd50 0x5607fe323810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:30.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:30.121+0000 7f2bcfe00640 1 -- 172.21.15.5:0/2216472609 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5607fe31af30 msgr2=0x5607fe31f2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:44:30.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:30.121+0000 7f2bcfe00640 1 --2- 172.21.15.5:0/2216472609 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5607fe31af30 0x5607fe31f2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:30.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:30.121+0000 7f2bcfe00640 1 -- 172.21.15.5:0/2216472609 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5607fe3949a0 con 0x5607fe310670 2024-09-10T09:44:30.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:30.121+0000 7f2bd5e00640 1 --2- 172.21.15.5:0/2216472609 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5607fe30fd50 0x5607fe323810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:44:30.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:30.121+0000 7f2bd6800640 1 --2- 172.21.15.5:0/2216472609 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5607fe31af30 0x5607fe31f2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:44:30.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:30.121+0000 7f2bcfe00640 1 --2- 172.21.15.5:0/2216472609 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5607fe310670 0x5607fe31eb40 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f2bc009ba30 tx=0x7f2bc009b690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=21530ee7a2a36ae6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:44:30.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:30.121+0000 7f2bbb400640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-09-10T09:44:30.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:30.121+0000 7f2bbb400640 1 -- 172.21.15.5:0/2216472609 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bc009b380 con 0x5607fe310670 2024-09-10T09:44:30.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:30.121+0000 7f2bbb400640 1 -- 172.21.15.5:0/2216472609 <== mon.2 v2:172.21.15.102:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f2bc0097a80 con 0x5607fe310670 2024-09-10T09:44:30.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:30.121+0000 7f2bdf3e2740 1 -- 172.21.15.5:0/2216472609 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2bd0036040 con 0x5607fe310670 2024-09-10T09:44:30.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:30.122+0000 7f2bbb400640 1 -- 172.21.15.5:0/2216472609 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f2bc009c430 con 0x5607fe310670 2024-09-10T09:44:30.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:30.122+0000 7f2bdf3e2740 10 client.26612 fetch_fsmap learned FSMap version 34 2024-09-10T09:44:30.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:30.122+0000 7f2bdf3e2740 1 -- 172.21.15.5:0/2216472609 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5607fe395b00 con 0x5607fe310670 2024-09-10T09:44:30.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:30.122+0000 7f2bbb400640 1 -- 172.21.15.5:0/2216472609 <== mon.2 v2:172.21.15.102:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2bc009bd30 con 0x5607fe310670 2024-09-10T09:44:30.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:30.122+0000 7f2bbb400640 1 -- 172.21.15.5:0/2216472609 <== mon.2 v2:172.21.15.102:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f2bc00a1270 con 0x5607fe310670 2024-09-10T09:44:30.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:30.122+0000 7f2bdf3e2740 10 client.26612 fetch_fsmap finished waiting for FSMap version 34 2024-09-10T09:44:30.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:30.122+0000 7f2bdf3e2740 10 client.26612 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T09:44:30.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:30.122+0000 7f2bdf3e2740 20 client.26612 populate_metadata read hostname 'smithi005' 2024-09-10T09:44:30.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:30.122+0000 7f2bdf3e2740 1 --2- 172.21.15.5:0/2216472609 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x5607fe396230 0x5607fe398650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:44:30.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:30.122+0000 7f2bdf3e2740 4 client.26612 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T09:44:30.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:30.122+0000 7f2bdf3e2740 1 -- 172.21.15.5:0/2216472609 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7f2bd0036040 con 0x5607fe396230 2024-09-10T09:44:30.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:30.123+0000 7f2bd5e00640 1 --2- 172.21.15.5:0/2216472609 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x5607fe396230 0x5607fe398650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:44:30.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:30.123+0000 7f2bd5e00640 1 --2- 172.21.15.5:0/2216472609 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x5607fe396230 0x5607fe398650 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cf0482e1264a5ce0 server_cookie=7ce773030ac3903b in_seq=0 out_seq=0 2024-09-10T09:44:30.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:30.123+0000 7f2bbb400640 10 client.26612 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T09:44:30.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:30.123+0000 7f2bbb400640 1 -- 172.21.15.5:0/2216472609 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x7f2bd0036040 con 0x5607fe396230 2024-09-10T09:44:30.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:30.123+0000 7f2bbb400640 10 client.26612 handle_command_reply: tid=0 2024-09-10T09:44:30.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:30.123+0000 7f2bdf3e2740 1 client.26612 shutdown 2024-09-10T09:44:30.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:30.124+0000 7f2bdf3e2740 1 -- 172.21.15.5:0/2216472609 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5607fe310670 msgr2=0x5607fe31eb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:44:30.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:30.124+0000 7f2bdf3e2740 1 --2- 172.21.15.5:0/2216472609 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5607fe310670 0x5607fe31eb40 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f2bc009ba30 tx=0x7f2bc009b690 comp rx=0 tx=0).stop 2024-09-10T09:44:30.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:30.124+0000 7f2bdf3e2740 1 -- 172.21.15.5:0/2216472609 shutdown_connections 2024-09-10T09:44:30.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:30.124+0000 7f2bdf3e2740 1 --2- 172.21.15.5:0/2216472609 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x5607fe396230 0x5607fe398650 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:30.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:30.124+0000 7f2bdf3e2740 1 --2- 172.21.15.5:0/2216472609 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5607fe31af30 0x5607fe31f2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:30.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:30.124+0000 7f2bdf3e2740 1 --2- 172.21.15.5:0/2216472609 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5607fe310670 0x5607fe31eb40 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:30.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:30.124+0000 7f2bdf3e2740 1 --2- 172.21.15.5:0/2216472609 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5607fe30fd50 0x5607fe323810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:30.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:30.124+0000 7f2bdf3e2740 1 -- 172.21.15.5:0/2216472609 >> 172.21.15.5:0/2216472609 conn(0x5607fe30c2e0 msgr2=0x5607fe319ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:44:30.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:30.124+0000 7f2bbb400640 0 client.26612 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T09:44:30.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:30.124+0000 7f2bdf3e2740 1 -- 172.21.15.5:0/2216472609 shutdown_connections 2024-09-10T09:44:30.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:30.124+0000 7f2bdf3e2740 1 -- 172.21.15.5:0/2216472609 wait complete. 2024-09-10T09:44:30.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:30.124+0000 7f2bdf3e2740 20 client.26612 trim_cache size 0 max 16384 2024-09-10T09:44:30.128 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-10T09:44:30.128 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-10T09:44:30.128 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2024-09-10T09:44:30.128 INFO:teuthology.orchestra.run.smithi005.stdout: "542eb324-daa1-4175-b9c0-0acfdeca248f": { 2024-09-10T09:44:30.128 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2024-09-10T09:44:30.128 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "542eb324-daa1-4175-b9c0-0acfdeca248f", 2024-09-10T09:44:30.128 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2024-09-10T09:44:30.129 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T09:44:30.129 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T09:44:30.129 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-10T09:44:30.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:30.125+0000 7f2bd7200640 1 -- 172.21.15.5:0/1645134387 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f2bbc0a2c00 msgr2=0x7f2bbc0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:44:30.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:30.125+0000 7f2bd7200640 1 --2- 172.21.15.5:0/1645134387 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f2bbc0a2c00 0x7f2bbc0a50c0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f2bd002ade0 tx=0x7f2bd002f400 comp rx=0 tx=0).stop 2024-09-10T09:44:30.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:30.125+0000 7f2bd7200640 1 -- 172.21.15.5:0/1645134387 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2bd81010d0 msgr2=0x7f2bd810acf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:44:30.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:30.125+0000 7f2bd7200640 1 --2- 172.21.15.5:0/1645134387 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2bd81010d0 0x7f2bd810acf0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f2bc000e9f0 tx=0x7f2bc000edf0 comp rx=0 tx=0).stop 2024-09-10T09:44:30.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:30.125+0000 7f2bd7200640 1 -- 172.21.15.5:0/1645134387 shutdown_connections 2024-09-10T09:44:30.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:30.125+0000 7f2bd7200640 1 --2- 172.21.15.5:0/1645134387 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f2bbc0a2c00 0x7f2bbc0a50c0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:30.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:30.125+0000 7f2bd7200640 1 --2- 172.21.15.5:0/1645134387 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2bd810e2d0 0x7f2bd810b250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:30.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:30.125+0000 7f2bd7200640 1 --2- 172.21.15.5:0/1645134387 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2bd81010d0 0x7f2bd810acf0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:30.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:30.125+0000 7f2bd7200640 1 --2- 172.21.15.5:0/1645134387 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2bd8100720 0x7f2bd810dc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:30.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:30.125+0000 7f2bd7200640 1 -- 172.21.15.5:0/1645134387 >> 172.21.15.5:0/1645134387 conn(0x7f2bd80fc610 msgr2=0x7f2bd80fea00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:44:30.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:30.125+0000 7f2bd7200640 1 -- 172.21.15.5:0/1645134387 shutdown_connections 2024-09-10T09:44:30.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:30.126+0000 7f2bd7200640 1 -- 172.21.15.5:0/1645134387 wait complete. 2024-09-10T09:44:31.046 INFO:tasks.workunit.client.0.smithi005.stdout: 1 344887 6.23 MB/sec execute 132 sec latency 938.810 ms 2024-09-10T09:44:31.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:44:30 smithi005 ceph-mon[30350]: pgmap v578: 129 pgs: 129 active+clean; 140 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.1 MiB/s wr, 183 op/s 2024-09-10T09:44:31.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:44:30 smithi159 ceph-mon[35513]: pgmap v578: 129 pgs: 129 active+clean; 140 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.1 MiB/s wr, 183 op/s 2024-09-10T09:44:31.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:44:30 smithi102 ceph-mon[35442]: pgmap v578: 129 pgs: 129 active+clean; 140 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.1 MiB/s wr, 183 op/s 2024-09-10T09:44:31.431 INFO:tasks.cephfs.filesystem:scrub status for tag:542eb324-daa1-4175-b9c0-0acfdeca248f - {'path': '/', 'tag': '542eb324-daa1-4175-b9c0-0acfdeca248f', 'options': 'recursive,force'} 2024-09-10T09:44:32.012 INFO:tasks.quiescer.fs.[cephfs]:Total duration (40.2) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-09-10T09:44:32.012 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 20.1 2024-09-10T09:44:32.012 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T09:44:32.046 INFO:tasks.workunit.client.0.smithi005.stdout: 1 347042 6.28 MB/sec execute 133 sec latency 1084.431 ms 2024-09-10T09:44:32.430 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:44:33.046 INFO:tasks.workunit.client.0.smithi005.stdout: 1 348831 6.29 MB/sec execute 134 sec latency 21.638 ms 2024-09-10T09:44:33.053 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:44:32 smithi005 ceph-mon[30350]: pgmap v579: 129 pgs: 129 active+clean; 142 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.2 MiB/s wr, 203 op/s 2024-09-10T09:44:33.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:33.078+0000 7fe8b2400640 1 -- 172.21.15.5:0/4162490828 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe8ac1019c0 msgr2=0x7fe8ac101e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:44:33.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:33.078+0000 7fe8b2400640 1 --2- 172.21.15.5:0/4162490828 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe8ac1019c0 0x7fe8ac101e20 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fe894009900 tx=0x7fe89402a3f0 comp rx=0 tx=0).stop 2024-09-10T09:44:33.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:33.078+0000 7fe8b2400640 1 -- 172.21.15.5:0/4162490828 shutdown_connections 2024-09-10T09:44:33.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:33.078+0000 7fe8b2400640 1 --2- 172.21.15.5:0/4162490828 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe8ac102360 0x7fe8ac10e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:33.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:33.078+0000 7fe8b2400640 1 --2- 172.21.15.5:0/4162490828 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe8ac1019c0 0x7fe8ac101e20 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:33.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:33.078+0000 7fe8b2400640 1 --2- 172.21.15.5:0/4162490828 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe8ac1079c0 0x7fe8ac107da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:33.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:33.078+0000 7fe8b2400640 1 -- 172.21.15.5:0/4162490828 >> 172.21.15.5:0/4162490828 conn(0x7fe8ac0fd740 msgr2=0x7fe8ac0ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:44:33.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:33.078+0000 7fe8b2400640 1 -- 172.21.15.5:0/4162490828 shutdown_connections 2024-09-10T09:44:33.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:33.078+0000 7fe8b2400640 1 -- 172.21.15.5:0/4162490828 wait complete. 2024-09-10T09:44:33.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:33.079+0000 7fe8b2400640 1 Processor -- start 2024-09-10T09:44:33.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:33.079+0000 7fe8b2400640 1 -- start start 2024-09-10T09:44:33.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:33.079+0000 7fe8b2400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe8ac1019c0 0x7fe8ac198260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:44:33.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:33.079+0000 7fe8b2400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe8ac102360 0x7fe8ac1987a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:44:33.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:33.080+0000 7fe8b2400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe8ac1079c0 0x7fe8ac19cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:44:33.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:33.080+0000 7fe8b2400640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe8ac111dc0 con 0x7fe8ac1019c0 2024-09-10T09:44:33.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:33.080+0000 7fe8b2400640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fe8ac111c40 con 0x7fe8ac102360 2024-09-10T09:44:33.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:33.080+0000 7fe8b2400640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fe8ac111f40 con 0x7fe8ac1079c0 2024-09-10T09:44:33.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:33.080+0000 7fe8b1000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe8ac1019c0 0x7fe8ac198260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:44:33.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:33.080+0000 7fe8a3e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe8ac102360 0x7fe8ac1987a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:44:33.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:33.080+0000 7fe8b1000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe8ac1019c0 0x7fe8ac198260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47446/0 (socket says 172.21.15.5:47446) 2024-09-10T09:44:33.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:33.080+0000 7fe8b1000640 1 -- 172.21.15.5:0/419928135 learned_addr learned my addr 172.21.15.5:0/419928135 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:44:33.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:33.080+0000 7fe8b1a00640 1 --2- 172.21.15.5:0/419928135 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe8ac1079c0 0x7fe8ac19cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:44:33.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:33.080+0000 7fe8a3e00640 1 -- 172.21.15.5:0/419928135 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe8ac1079c0 msgr2=0x7fe8ac19cb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:44:33.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:33.080+0000 7fe8a3e00640 1 --2- 172.21.15.5:0/419928135 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe8ac1079c0 0x7fe8ac19cb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:33.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:33.080+0000 7fe8a3e00640 1 -- 172.21.15.5:0/419928135 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe8ac1019c0 msgr2=0x7fe8ac198260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:44:33.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:33.080+0000 7fe8a3e00640 1 --2- 172.21.15.5:0/419928135 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe8ac1019c0 0x7fe8ac198260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:33.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:33.080+0000 7fe8a3e00640 1 -- 172.21.15.5:0/419928135 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8ac19d210 con 0x7fe8ac102360 2024-09-10T09:44:33.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:33.080+0000 7fe8b1000640 1 --2- 172.21.15.5:0/419928135 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe8ac1019c0 0x7fe8ac198260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:44:33.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:33.081+0000 7fe8a3e00640 1 --2- 172.21.15.5:0/419928135 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe8ac102360 0x7fe8ac1987a0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fe894005940 tx=0x7fe894003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:44:33.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:33.081+0000 7fe8aa000640 1 -- 172.21.15.5:0/419928135 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe89403f070 con 0x7fe8ac102360 2024-09-10T09:44:33.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:33.081+0000 7fe8aa000640 1 -- 172.21.15.5:0/419928135 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe894031c30 con 0x7fe8ac102360 2024-09-10T09:44:33.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:33.081+0000 7fe8b2400640 1 -- 172.21.15.5:0/419928135 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8ac19d4a0 con 0x7fe8ac102360 2024-09-10T09:44:33.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:33.081+0000 7fe8aa000640 1 -- 172.21.15.5:0/419928135 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe894035030 con 0x7fe8ac102360 2024-09-10T09:44:33.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:33.081+0000 7fe8b2400640 1 -- 172.21.15.5:0/419928135 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8ac1a4de0 con 0x7fe8ac102360 2024-09-10T09:44:33.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:33.082+0000 7fe8b2400640 1 -- 172.21.15.5:0/419928135 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe8ac101e20 con 0x7fe8ac102360 2024-09-10T09:44:33.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:33.084+0000 7fe8aa000640 1 -- 172.21.15.5:0/419928135 <== mon.2 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7fe894042070 con 0x7fe8ac102360 2024-09-10T09:44:33.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:33.085+0000 7fe8aa000640 1 --2- 172.21.15.5:0/419928135 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fe8900a2c00 0x7fe8900a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:44:33.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:33.086+0000 7fe8aa000640 1 -- 172.21.15.5:0/419928135 <== mon.2 v2:172.21.15.102:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fe8940ef6a0 con 0x7fe8ac102360 2024-09-10T09:44:33.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:33.086+0000 7fe8b1000640 1 --2- 172.21.15.5:0/419928135 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fe8900a2c00 0x7fe8900a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:44:33.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:33.086+0000 7fe8b1000640 1 --2- 172.21.15.5:0/419928135 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fe8900a2c00 0x7fe8900a50c0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fe89c005fd0 tx=0x7fe89c00b480 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:44:33.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:33.090+0000 7fe8aa000640 1 -- 172.21.15.5:0/419928135 <== mon.2 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe894031de0 con 0x7fe8ac102360 2024-09-10T09:44:33.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:44:32 smithi159 ceph-mon[35513]: pgmap v579: 129 pgs: 129 active+clean; 142 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.2 MiB/s wr, 203 op/s 2024-09-10T09:44:33.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:44:32 smithi102 ceph-mon[35442]: pgmap v579: 129 pgs: 129 active+clean; 142 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.2 MiB/s wr, 203 op/s 2024-09-10T09:44:33.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:33.359+0000 7fe8b2400640 1 -- 172.21.15.5:0/419928135 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe8ac1995c0 con 0x7fe8ac102360 2024-09-10T09:44:33.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:33.360+0000 7fe8aa000640 1 -- 172.21.15.5:0/419928135 <== mon.2 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+11969 (secure 0 0 0) 0x7fe8940bbb60 con 0x7fe8ac102360 2024-09-10T09:44:33.361 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-10T09:44:33.362 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":34,"btime":"2024-09-10T09:42:57:737230+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14454,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/2970594311","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2970594311},{"type":"v1","addr":"172.21.15.5:6835","nonce":2970594311}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14514,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6839/3616502174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":3616502174},{"type":"v1","addr":"172.21.15.5:6839","nonce":3616502174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":34,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T09:35:29.626287+0000","modified":"2024-09-10T09:42:56.961831+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24542,"mds_1":24535,"mds_2":24511,"mds_3":24506,"mds_4":24443},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14520":{"gid":14520,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/2073599842","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":2073599842},{"type":"v1","addr":"172.21.15.5:6841","nonce":2073599842}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"k","rank":4,"incarnation":26,"state":"up:active","state_seq":24,"addr":"172.21.15.102:6833/2457007427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":2457007427},{"type":"v1","addr":"172.21.15.102:6833","nonce":2457007427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/356445159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":356445159},{"type":"v1","addr":"172.21.15.159:6833","nonce":356445159}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6837/2894314507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":2894314507},{"type":"v1","addr":"172.21.15.5:6837","nonce":2894314507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.102:6835/1647549111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":1647549111},{"type":"v1","addr":"172.21.15.102:6835","nonce":1647549111}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6835/952657242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":952657242},{"type":"v1","addr":"172.21.15.159:6835","nonce":952657242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":19,"addr":"172.21.15.102:6837/2689776764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":2689776764},{"type":"v1","addr":"172.21.15.102:6837","nonce":2689776764}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.159:6837/620627639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":620627639},{"type":"v1","addr":"172.21.15.159:6837","nonce":620627639}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24535":{"gid":24535,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":18,"addr":"172.21.15.159:6839/2073739599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":2073739599},{"type":"v1","addr":"172.21.15.159:6839","nonce":2073739599}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24542":{"gid":24542,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.102:6839/335409749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6838","nonce":335409749},{"type":"v1","addr":"172.21.15.102:6839","nonce":335409749}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24542,"qdb_cluster":[24542,24535,24511,24506,24443]},"id":1}]} 2024-09-10T09:44:33.362 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 34 2024-09-10T09:44:33.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:33.363+0000 7fe8b2400640 1 -- 172.21.15.5:0/419928135 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fe8900a2c00 msgr2=0x7fe8900a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:44:33.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:33.363+0000 7fe8b2400640 1 --2- 172.21.15.5:0/419928135 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fe8900a2c00 0x7fe8900a50c0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fe89c005fd0 tx=0x7fe89c00b480 comp rx=0 tx=0).stop 2024-09-10T09:44:33.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:33.363+0000 7fe8b2400640 1 -- 172.21.15.5:0/419928135 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe8ac102360 msgr2=0x7fe8ac1987a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:44:33.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:33.363+0000 7fe8b2400640 1 --2- 172.21.15.5:0/419928135 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe8ac102360 0x7fe8ac1987a0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fe894005940 tx=0x7fe894003c10 comp rx=0 tx=0).stop 2024-09-10T09:44:33.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:33.363+0000 7fe8b2400640 1 -- 172.21.15.5:0/419928135 shutdown_connections 2024-09-10T09:44:33.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:33.363+0000 7fe8b2400640 1 --2- 172.21.15.5:0/419928135 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fe8900a2c00 0x7fe8900a50c0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:33.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:33.363+0000 7fe8b2400640 1 --2- 172.21.15.5:0/419928135 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe8ac1079c0 0x7fe8ac19cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:33.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:33.363+0000 7fe8b2400640 1 --2- 172.21.15.5:0/419928135 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe8ac102360 0x7fe8ac1987a0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:33.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:33.363+0000 7fe8b2400640 1 --2- 172.21.15.5:0/419928135 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe8ac1019c0 0x7fe8ac198260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:33.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:33.363+0000 7fe8b2400640 1 -- 172.21.15.5:0/419928135 >> 172.21.15.5:0/419928135 conn(0x7fe8ac0fd740 msgr2=0x7fe8ac0fde10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:44:33.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:33.363+0000 7fe8b2400640 1 -- 172.21.15.5:0/419928135 shutdown_connections 2024-09-10T09:44:33.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:33.363+0000 7fe8b2400640 1 -- 172.21.15.5:0/419928135 wait complete. 2024-09-10T09:44:34.046 INFO:tasks.workunit.client.0.smithi005.stdout: 1 351386 6.37 MB/sec execute 135 sec latency 17.507 ms 2024-09-10T09:44:34.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:44:33 smithi159 ceph-mon[35513]: from='client.? 172.21.15.5:0/419928135' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:44:34.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:44:33 smithi102 ceph-mon[35442]: from='client.? 172.21.15.5:0/419928135' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:44:34.306 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:44:33 smithi005 ceph-mon[30350]: from='client.? 172.21.15.5:0/419928135' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:44:34.355 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24542 quiesce db / --timeout 90 --expiration 140.1 --await-for 45.0' 2024-09-10T09:44:34.355 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24542 quiesce db / --timeout 90 --expiration 140.1 --await-for 45.0 2024-09-10T09:44:34.826 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:44:35.046 INFO:tasks.workunit.client.0.smithi005.stdout: 1 353002 6.37 MB/sec execute 136 sec latency 48.401 ms 2024-09-10T09:44:35.086 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:44:34 smithi005 ceph-mon[30350]: pgmap v580: 129 pgs: 129 active+clean; 143 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.1 MiB/s wr, 168 op/s 2024-09-10T09:44:35.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:44:34 smithi159 ceph-mon[35513]: pgmap v580: 129 pgs: 129 active+clean; 143 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.1 MiB/s wr, 168 op/s 2024-09-10T09:44:35.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:44:34 smithi102 ceph-mon[35442]: pgmap v580: 129 pgs: 129 active+clean; 143 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.1 MiB/s wr, 168 op/s 2024-09-10T09:44:35.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:35.677+0000 7f9b06200640 1 -- 172.21.15.5:0/1150551339 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9b001034b0 msgr2=0x7f9b0010b9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:44:35.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:35.677+0000 7f9b06200640 1 --2- 172.21.15.5:0/1150551339 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9b001034b0 0x7f9b0010b9a0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f9aec009900 tx=0x7f9aec02a4b0 comp rx=0 tx=0).stop 2024-09-10T09:44:35.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:35.677+0000 7f9b06200640 1 -- 172.21.15.5:0/1150551339 shutdown_connections 2024-09-10T09:44:35.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:35.677+0000 7f9b06200640 1 --2- 172.21.15.5:0/1150551339 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9b0010bfc0 0x7f9b0010e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:35.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:35.677+0000 7f9b06200640 1 --2- 172.21.15.5:0/1150551339 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9b001034b0 0x7f9b0010b9a0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:35.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:35.677+0000 7f9b06200640 1 --2- 172.21.15.5:0/1150551339 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9b00102b90 0x7f9b00102f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:35.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:35.677+0000 7f9b06200640 1 -- 172.21.15.5:0/1150551339 >> 172.21.15.5:0/1150551339 conn(0x7f9b000fc610 msgr2=0x7f9b000fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:44:35.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:35.677+0000 7f9b06200640 1 -- 172.21.15.5:0/1150551339 shutdown_connections 2024-09-10T09:44:35.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:35.677+0000 7f9b06200640 1 -- 172.21.15.5:0/1150551339 wait complete. 2024-09-10T09:44:35.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:35.686+0000 7f9b06200640 1 Processor -- start 2024-09-10T09:44:35.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:35.686+0000 7f9b06200640 1 -- start start 2024-09-10T09:44:35.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:35.686+0000 7f9b06200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9b00102b90 0x7f9b00198350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:44:35.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:35.686+0000 7f9b06200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9b001034b0 0x7f9b00198890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:44:35.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:35.686+0000 7f9b06200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9b0010bfc0 0x7f9b0019cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:44:35.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:35.686+0000 7f9b06200640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f9b00111e70 con 0x7f9b001034b0 2024-09-10T09:44:35.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:35.686+0000 7f9b06200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f9b00111cf0 con 0x7f9b0010bfc0 2024-09-10T09:44:35.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:35.686+0000 7f9b06200640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f9b00111ff0 con 0x7f9b00102b90 2024-09-10T09:44:35.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:35.686+0000 7f9b04e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9b00102b90 0x7f9b00198350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:44:35.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:35.686+0000 7f9b04e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9b00102b90 0x7f9b00198350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:50602/0 (socket says 172.21.15.5:50602) 2024-09-10T09:44:35.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:35.686+0000 7f9b04e00640 1 -- 172.21.15.5:0/1397944590 learned_addr learned my addr 172.21.15.5:0/1397944590 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:44:35.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:35.686+0000 7f9affe00640 1 --2- 172.21.15.5:0/1397944590 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9b001034b0 0x7f9b00198890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:44:35.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:35.686+0000 7f9b05800640 1 --2- 172.21.15.5:0/1397944590 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9b0010bfc0 0x7f9b0019cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:44:35.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:35.686+0000 7f9b04e00640 1 -- 172.21.15.5:0/1397944590 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9b0010bfc0 msgr2=0x7f9b0019cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:44:35.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:35.686+0000 7f9b04e00640 1 --2- 172.21.15.5:0/1397944590 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9b0010bfc0 0x7f9b0019cc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:35.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:35.686+0000 7f9b04e00640 1 -- 172.21.15.5:0/1397944590 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9b001034b0 msgr2=0x7f9b00198890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:44:35.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:35.686+0000 7f9b04e00640 1 --2- 172.21.15.5:0/1397944590 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9b001034b0 0x7f9b00198890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:35.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:35.686+0000 7f9b04e00640 1 -- 172.21.15.5:0/1397944590 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b0019d300 con 0x7f9b00102b90 2024-09-10T09:44:35.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:35.686+0000 7f9affe00640 1 --2- 172.21.15.5:0/1397944590 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9b001034b0 0x7f9b00198890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:44:35.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:35.686+0000 7f9b04e00640 1 --2- 172.21.15.5:0/1397944590 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9b00102b90 0x7f9b00198350 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f9af400b450 tx=0x7f9af400b910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:44:35.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:35.686+0000 7f9afd600640 1 -- 172.21.15.5:0/1397944590 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9af4015020 con 0x7f9b00102b90 2024-09-10T09:44:35.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:35.686+0000 7f9afd600640 1 -- 172.21.15.5:0/1397944590 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9af4011d70 con 0x7f9b00102b90 2024-09-10T09:44:35.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:35.686+0000 7f9afd600640 1 -- 172.21.15.5:0/1397944590 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9af400f340 con 0x7f9b00102b90 2024-09-10T09:44:35.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:35.686+0000 7f9b06200640 1 -- 172.21.15.5:0/1397944590 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b0019d5f0 con 0x7f9b00102b90 2024-09-10T09:44:35.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:35.686+0000 7f9b06200640 1 -- 172.21.15.5:0/1397944590 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b00100720 con 0x7f9b00102b90 2024-09-10T09:44:35.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:35.691+0000 7f9b0a3cd740 1 Processor -- start 2024-09-10T09:44:35.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:35.691+0000 7f9b0a3cd740 1 -- start start 2024-09-10T09:44:35.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:35.691+0000 7f9b0a3cd740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5635fae01560 0x5635fae01940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:44:35.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:35.691+0000 7f9b0a3cd740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5635fae01e80 0x5635fae0c200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:44:35.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:35.691+0000 7f9b0a3cd740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5635fae0c740 0x5635fae0eb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:44:35.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:35.691+0000 7f9b0a3cd740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5635fad67530 con 0x5635fae0c740 2024-09-10T09:44:35.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:35.691+0000 7f9b0a3cd740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x5635fad0e860 con 0x5635fae01560 2024-09-10T09:44:35.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:35.691+0000 7f9b0a3cd740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5635fad634f0 con 0x5635fae01e80 2024-09-10T09:44:35.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:35.691+0000 7f9b05800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5635fae01560 0x5635fae01940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:44:35.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:35.691+0000 7f9b05800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5635fae01560 0x5635fae01940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:51924/0 (socket says 172.21.15.5:51924) 2024-09-10T09:44:35.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:35.691+0000 7f9b05800640 1 -- 172.21.15.5:0/4109675288 learned_addr learned my addr 172.21.15.5:0/4109675288 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:44:35.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:35.691+0000 7f9affe00640 1 --2- 172.21.15.5:0/4109675288 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5635fae0c740 0x5635fae0eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:44:35.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:35.691+0000 7f9afd600640 1 -- 172.21.15.5:0/1397944590 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f9af4018020 con 0x7f9b00102b90 2024-09-10T09:44:35.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:35.691+0000 7f9affe00640 1 -- 172.21.15.5:0/4109675288 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5635fae01e80 msgr2=0x5635fae0c200 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T09:44:35.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:35.691+0000 7f9affe00640 1 --2- 172.21.15.5:0/4109675288 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5635fae01e80 0x5635fae0c200 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:35.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:35.691+0000 7f9affe00640 1 -- 172.21.15.5:0/4109675288 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5635fae01560 msgr2=0x5635fae01940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:44:35.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:35.691+0000 7f9affe00640 1 --2- 172.21.15.5:0/4109675288 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5635fae01560 0x5635fae01940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:35.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:35.691+0000 7f9affe00640 1 -- 172.21.15.5:0/4109675288 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5635faad8b90 con 0x5635fae0c740 2024-09-10T09:44:35.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:35.691+0000 7f9b05800640 1 --2- 172.21.15.5:0/4109675288 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5635fae01560 0x5635fae01940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:44:35.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:35.691+0000 7f9affe00640 1 --2- 172.21.15.5:0/4109675288 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5635fae0c740 0x5635fae0eb30 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f9aec02aaa0 tx=0x7f9aec004150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6c2b4a2d6bb1d4f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:44:35.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:35.691+0000 7f9af2000640 1 -- 172.21.15.5:0/4109675288 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9aec035030 con 0x5635fae0c740 2024-09-10T09:44:35.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:35.691+0000 7f9af2000640 1 -- 172.21.15.5:0/4109675288 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9aec005520 con 0x5635fae0c740 2024-09-10T09:44:35.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:35.691+0000 7f9af2000640 1 -- 172.21.15.5:0/4109675288 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9aec0035f0 con 0x5635fae0c740 2024-09-10T09:44:35.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:35.691+0000 7f9afd600640 1 --2- 172.21.15.5:0/1397944590 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f9ae80a2c00 0x7f9ae80a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:44:35.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:35.691+0000 7f9b0a3cd740 1 -- 172.21.15.5:0/4109675288 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5635fae0c740 msgr2=0x5635fae0eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:44:35.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:35.691+0000 7f9b0a3cd740 1 --2- 172.21.15.5:0/4109675288 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5635fae0c740 0x5635fae0eb30 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f9aec02aaa0 tx=0x7f9aec004150 comp rx=0 tx=0).stop 2024-09-10T09:44:35.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:35.691+0000 7f9afd600640 1 -- 172.21.15.5:0/1397944590 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f9af40cef30 con 0x7f9b00102b90 2024-09-10T09:44:35.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:35.691+0000 7f9b05800640 1 --2- 172.21.15.5:0/1397944590 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f9ae80a2c00 0x7f9ae80a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:44:35.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:35.691+0000 7f9b0a3cd740 1 -- 172.21.15.5:0/4109675288 shutdown_connections 2024-09-10T09:44:35.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:35.691+0000 7f9b0a3cd740 1 --2- 172.21.15.5:0/4109675288 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5635fae0c740 0x5635fae0eb30 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:35.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:35.691+0000 7f9b0a3cd740 1 --2- 172.21.15.5:0/4109675288 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5635fae01e80 0x5635fae0c200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:35.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:35.691+0000 7f9b0a3cd740 1 --2- 172.21.15.5:0/4109675288 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5635fae01560 0x5635fae01940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:35.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:35.691+0000 7f9b0a3cd740 1 -- 172.21.15.5:0/4109675288 >> 172.21.15.5:0/4109675288 conn(0x5635fadfdaf0 msgr2=0x5635fadfded0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:44:35.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:35.691+0000 7f9b0a3cd740 1 -- 172.21.15.5:0/4109675288 shutdown_connections 2024-09-10T09:44:35.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:35.691+0000 7f9b0a3cd740 1 -- 172.21.15.5:0/4109675288 wait complete. 2024-09-10T09:44:35.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:35.696+0000 7f9b05800640 1 --2- 172.21.15.5:0/1397944590 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f9ae80a2c00 0x7f9ae80a50c0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f9af8005fd0 tx=0x7f9af8009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:44:35.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:35.696+0000 7f9b0a3cd740 1 -- start start 2024-09-10T09:44:35.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:35.696+0000 7f9b0a3cd740 1 Processor -- start 2024-09-10T09:44:35.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:35.696+0000 7f9b0a3cd740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5635fae01560 0x5635fae096f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:44:35.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:35.696+0000 7f9b0a3cd740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5635fae01e80 0x5635fae09c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:44:35.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:35.696+0000 7f9b0a3cd740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5635fae0c740 0x5635fae0a170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:44:35.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:35.696+0000 7f9b0a3cd740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5635fae0f310 con 0x5635fae01e80 2024-09-10T09:44:35.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:35.696+0000 7f9b0a3cd740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x5635fad21e60 con 0x5635fae01560 2024-09-10T09:44:35.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:35.696+0000 7f9b0a3cd740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5635fae0f490 con 0x5635fae0c740 2024-09-10T09:44:35.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:35.696+0000 7f9b0a3cd740 10 client.0 _ll_register_callbacks cb 0x5635fad26cc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:44:35.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:35.697+0000 7f9affe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5635fae01e80 0x5635fae09c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:44:35.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:35.697+0000 7f9affe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5635fae01e80 0x5635fae09c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47478/0 (socket says 172.21.15.5:47478) 2024-09-10T09:44:35.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:35.697+0000 7f9affe00640 1 -- 172.21.15.5:0/1706277615 learned_addr learned my addr 172.21.15.5:0/1706277615 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:44:35.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:35.697+0000 7f9b05800640 1 --2- 172.21.15.5:0/1706277615 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5635fae0c740 0x5635fae0a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:44:35.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:35.697+0000 7f9b04e00640 1 --2- 172.21.15.5:0/1706277615 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5635fae01560 0x5635fae096f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:44:35.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:35.697+0000 7f9b05800640 1 -- 172.21.15.5:0/1706277615 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5635fae01560 msgr2=0x5635fae096f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:44:35.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:35.697+0000 7f9b05800640 1 --2- 172.21.15.5:0/1706277615 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5635fae01560 0x5635fae096f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:35.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:35.697+0000 7f9b05800640 1 -- 172.21.15.5:0/1706277615 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5635fae01e80 msgr2=0x5635fae09c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:44:35.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:35.697+0000 7f9b05800640 1 --2- 172.21.15.5:0/1706277615 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5635fae01e80 0x5635fae09c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:35.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:35.697+0000 7f9b05800640 1 -- 172.21.15.5:0/1706277615 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5635fad2d990 con 0x5635fae0c740 2024-09-10T09:44:35.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:35.697+0000 7f9b05800640 1 --2- 172.21.15.5:0/1706277615 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5635fae0c740 0x5635fae0a170 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f9af8003f50 tx=0x7f9af8004ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6bab7577dcb0e59 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:44:35.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:35.697+0000 7f9ae6a00640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-09-10T09:44:35.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:35.697+0000 7f9ae6a00640 1 -- 172.21.15.5:0/1706277615 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9af8004b20 con 0x5635fae0c740 2024-09-10T09:44:35.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:35.697+0000 7f9b0a3cd740 1 -- 172.21.15.5:0/1706277615 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9aec005520 con 0x5635fae0c740 2024-09-10T09:44:35.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:35.697+0000 7f9ae6a00640 1 -- 172.21.15.5:0/1706277615 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f9af8004cc0 con 0x5635fae0c740 2024-09-10T09:44:35.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:35.697+0000 7f9ae6a00640 1 -- 172.21.15.5:0/1706277615 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f9af8030050 con 0x5635fae0c740 2024-09-10T09:44:35.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:35.697+0000 7f9b0a3cd740 10 client.26656 fetch_fsmap learned FSMap version 34 2024-09-10T09:44:35.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:35.697+0000 7f9b0a3cd740 1 -- 172.21.15.5:0/1706277615 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5635fae238b0 con 0x5635fae0c740 2024-09-10T09:44:35.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:35.697+0000 7f9ae6a00640 1 -- 172.21.15.5:0/1706277615 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9aec005520 con 0x5635fae0c740 2024-09-10T09:44:35.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:35.697+0000 7f9ae6a00640 1 -- 172.21.15.5:0/1706277615 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f9af8026030 con 0x5635fae0c740 2024-09-10T09:44:35.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:35.697+0000 7f9b0a3cd740 10 client.26656 fetch_fsmap finished waiting for FSMap version 34 2024-09-10T09:44:35.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:35.697+0000 7f9b0a3cd740 10 client.26656 resolve_mds: validated gid 24542 aka daemon mds.b 2024-09-10T09:44:35.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:35.697+0000 7f9b0a3cd740 20 client.26656 populate_metadata read hostname 'smithi005' 2024-09-10T09:44:35.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:35.697+0000 7f9b0a3cd740 1 --2- 172.21.15.5:0/1706277615 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x5635fae24dc0 0x5635fae27200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:44:35.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:35.697+0000 7f9b0a3cd740 4 client.26656 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:44:35.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:35.697+0000 7f9b0a3cd740 1 -- 172.21.15.5:0/1706277615 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5635fad0a310 con 0x5635fae24dc0 2024-09-10T09:44:35.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:35.697+0000 7f9b04e00640 1 --2- 172.21.15.5:0/1706277615 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x5635fae24dc0 0x5635fae27200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:44:35.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:35.697+0000 7f9b04e00640 1 --2- 172.21.15.5:0/1706277615 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x5635fae24dc0 0x5635fae27200 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a807e1df1adf2102 server_cookie=48c7837d21f64a62 in_seq=0 out_seq=0 2024-09-10T09:44:35.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:35.697+0000 7f9ae6a00640 10 client.26656 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T09:44:35.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:35.697+0000 7f9ae6a00640 1 -- 172.21.15.5:0/1706277615 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5635fad0a310 con 0x5635fae24dc0 2024-09-10T09:44:35.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:35.697+0000 7f9ae6a00640 10 client.26656 handle_command_reply: tid=0 2024-09-10T09:44:35.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:35.697+0000 7f9b0a3cd740 1 client.26656 shutdown 2024-09-10T09:44:35.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:35.697+0000 7f9b0a3cd740 1 -- 172.21.15.5:0/1706277615 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5635fae0c740 msgr2=0x5635fae0a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:44:35.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:35.697+0000 7f9b0a3cd740 1 --2- 172.21.15.5:0/1706277615 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5635fae0c740 0x5635fae0a170 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f9af8003f50 tx=0x7f9af8004ab0 comp rx=0 tx=0).stop 2024-09-10T09:44:35.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:35.697+0000 7f9b0a3cd740 1 -- 172.21.15.5:0/1706277615 shutdown_connections 2024-09-10T09:44:35.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:35.697+0000 7f9b0a3cd740 1 --2- 172.21.15.5:0/1706277615 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x5635fae24dc0 0x5635fae27200 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:35.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:35.697+0000 7f9b0a3cd740 1 --2- 172.21.15.5:0/1706277615 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5635fae0c740 0x5635fae0a170 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:35.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:35.697+0000 7f9b0a3cd740 1 --2- 172.21.15.5:0/1706277615 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5635fae01e80 0x5635fae09c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:35.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:35.697+0000 7f9b0a3cd740 1 --2- 172.21.15.5:0/1706277615 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5635fae01560 0x5635fae096f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:35.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:35.697+0000 7f9b0a3cd740 1 -- 172.21.15.5:0/1706277615 >> 172.21.15.5:0/1706277615 conn(0x5635fadfdaf0 msgr2=0x5635fae037d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:44:35.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:35.697+0000 7f9ae6a00640 0 client.26656 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T09:44:35.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:35.697+0000 7f9b0a3cd740 1 -- 172.21.15.5:0/1706277615 shutdown_connections 2024-09-10T09:44:35.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:35.697+0000 7f9b0a3cd740 1 -- 172.21.15.5:0/1706277615 wait complete. 2024-09-10T09:44:35.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:35.697+0000 7f9b0a3cd740 20 client.26656 trim_cache size 0 max 16384 2024-09-10T09:44:35.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:35.721+0000 7f9b0a3cd740 1 Processor -- start 2024-09-10T09:44:35.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:35.721+0000 7f9b0a3cd740 1 -- start start 2024-09-10T09:44:35.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:35.721+0000 7f9b0a3cd740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5635fae01560 0x5635fae0b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:44:35.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:35.721+0000 7f9b0a3cd740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5635fae01e80 0x5635fae2cad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:44:35.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:35.721+0000 7f9b0a3cd740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5635fae0c740 0x5635fae2d010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:44:35.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:35.721+0000 7f9b0a3cd740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f9b001aca10 con 0x5635fae0c740 2024-09-10T09:44:35.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:35.721+0000 7f9b0a3cd740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f9b001ae550 con 0x5635fae01e80 2024-09-10T09:44:35.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:35.721+0000 7f9b0a3cd740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f9b001ad0e0 con 0x5635fae01560 2024-09-10T09:44:35.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:35.721+0000 7f9b04e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5635fae01560 0x5635fae0b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:44:35.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:35.721+0000 7f9b04e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5635fae01560 0x5635fae0b550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:50628/0 (socket says 172.21.15.5:50628) 2024-09-10T09:44:35.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:35.721+0000 7f9b04e00640 1 -- 172.21.15.5:0/2880495534 learned_addr learned my addr 172.21.15.5:0/2880495534 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:44:35.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:35.721+0000 7f9b05800640 1 --2- 172.21.15.5:0/2880495534 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5635fae0c740 0x5635fae2d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:44:35.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:35.721+0000 7f9affe00640 1 --2- 172.21.15.5:0/2880495534 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5635fae01e80 0x5635fae2cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:44:35.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:35.721+0000 7f9b04e00640 1 -- 172.21.15.5:0/2880495534 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5635fae01e80 msgr2=0x5635fae2cad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:44:35.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:35.721+0000 7f9b04e00640 1 --2- 172.21.15.5:0/2880495534 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5635fae01e80 0x5635fae2cad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:35.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:35.721+0000 7f9b04e00640 1 -- 172.21.15.5:0/2880495534 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5635fae0c740 msgr2=0x5635fae2d010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:44:35.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:35.721+0000 7f9b04e00640 1 --2- 172.21.15.5:0/2880495534 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5635fae0c740 0x5635fae2d010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:35.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:35.721+0000 7f9b04e00640 1 -- 172.21.15.5:0/2880495534 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5635fadfe780 con 0x5635fae01560 2024-09-10T09:44:35.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:35.721+0000 7f9b04e00640 1 --2- 172.21.15.5:0/2880495534 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5635fae01560 0x5635fae0b550 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f9af40977a0 tx=0x7f9af40107f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=40a0805250105697 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:44:35.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:35.721+0000 7f9af1600640 1 -- 172.21.15.5:0/2880495534 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9af409f6f0 con 0x5635fae01560 2024-09-10T09:44:35.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:35.721+0000 7f9af1600640 1 -- 172.21.15.5:0/2880495534 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9af40a3fb0 con 0x5635fae01560 2024-09-10T09:44:35.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:35.721+0000 7f9af1600640 1 -- 172.21.15.5:0/2880495534 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9af409a790 con 0x5635fae01560 2024-09-10T09:44:35.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:35.721+0000 7f9b0a3cd740 1 -- 172.21.15.5:0/2880495534 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5635fae01560 msgr2=0x5635fae0b550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:44:35.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:35.721+0000 7f9b0a3cd740 1 --2- 172.21.15.5:0/2880495534 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5635fae01560 0x5635fae0b550 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f9af40977a0 tx=0x7f9af40107f0 comp rx=0 tx=0).stop 2024-09-10T09:44:35.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:35.721+0000 7f9b0a3cd740 1 -- 172.21.15.5:0/2880495534 shutdown_connections 2024-09-10T09:44:35.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:35.721+0000 7f9b0a3cd740 1 --2- 172.21.15.5:0/2880495534 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5635fae0c740 0x5635fae2d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:35.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:35.721+0000 7f9b0a3cd740 1 --2- 172.21.15.5:0/2880495534 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5635fae01e80 0x5635fae2cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:35.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:35.721+0000 7f9b0a3cd740 1 --2- 172.21.15.5:0/2880495534 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5635fae01560 0x5635fae0b550 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:35.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:35.721+0000 7f9b0a3cd740 1 -- 172.21.15.5:0/2880495534 >> 172.21.15.5:0/2880495534 conn(0x5635fadfdaf0 msgr2=0x5635fae0ae40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:44:35.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:35.721+0000 7f9b0a3cd740 1 -- 172.21.15.5:0/2880495534 shutdown_connections 2024-09-10T09:44:35.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:35.721+0000 7f9b0a3cd740 1 -- 172.21.15.5:0/2880495534 wait complete. 2024-09-10T09:44:35.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:35.721+0000 7f9b0a3cd740 1 -- start start 2024-09-10T09:44:35.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:35.721+0000 7f9b0a3cd740 1 Processor -- start 2024-09-10T09:44:35.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:35.721+0000 7f9b0a3cd740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5635fae01560 0x5635fae1d390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:44:35.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:35.721+0000 7f9b0a3cd740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5635fae01e80 0x5635fae19f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:44:35.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:35.721+0000 7f9b0a3cd740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5635fae0c740 0x5635fae1a480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:44:35.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:35.721+0000 7f9b0a3cd740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f9b001ade80 con 0x5635fae01560 2024-09-10T09:44:35.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:35.721+0000 7f9b0a3cd740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f9b001ad7b0 con 0x5635fae0c740 2024-09-10T09:44:35.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:35.721+0000 7f9b0a3cd740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f9b001aa9f0 con 0x5635fae01e80 2024-09-10T09:44:35.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:35.721+0000 7f9b0a3cd740 10 client.0 _ll_register_callbacks cb 0x7f9af8009e10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:44:35.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:35.721+0000 7f9affe00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5635fae01e80 0x5635fae19f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:44:35.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:35.721+0000 7f9affe00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5635fae01e80 0x5635fae19f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:50638/0 (socket says 172.21.15.5:50638) 2024-09-10T09:44:35.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:35.721+0000 7f9affe00640 1 -- 172.21.15.5:0/3303288652 learned_addr learned my addr 172.21.15.5:0/3303288652 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:44:35.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:35.721+0000 7f9b05800640 1 --2- 172.21.15.5:0/3303288652 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5635fae0c740 0x5635fae1a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:44:35.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:35.721+0000 7f9b04e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5635fae01560 0x5635fae1d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:44:35.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:35.721+0000 7f9b05800640 1 -- 172.21.15.5:0/3303288652 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5635fae01e80 msgr2=0x5635fae19f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:44:35.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:35.721+0000 7f9b05800640 1 --2- 172.21.15.5:0/3303288652 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5635fae01e80 0x5635fae19f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:35.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:35.721+0000 7f9b05800640 1 -- 172.21.15.5:0/3303288652 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5635fae01560 msgr2=0x5635fae1d390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:44:35.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:35.721+0000 7f9b05800640 1 --2- 172.21.15.5:0/3303288652 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5635fae01560 0x5635fae1d390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:35.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:35.721+0000 7f9b05800640 1 -- 172.21.15.5:0/3303288652 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5635fae1a9c0 con 0x5635fae0c740 2024-09-10T09:44:35.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:35.733+0000 7f9b05800640 1 --2- 172.21.15.5:0/3303288652 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5635fae0c740 0x5635fae1a480 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f9af8018040 tx=0x7f9af8003da0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=af8961170a6796e5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:44:35.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:35.733+0000 7f9ae6a00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-09-10T09:44:35.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:35.733+0000 7f9ae6a00640 1 -- 172.21.15.5:0/3303288652 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9af8038070 con 0x5635fae0c740 2024-09-10T09:44:35.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:35.733+0000 7f9b0a3cd740 1 -- 172.21.15.5:0/3303288652 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9af40a3fb0 con 0x5635fae0c740 2024-09-10T09:44:35.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:35.733+0000 7f9ae6a00640 1 -- 172.21.15.5:0/3303288652 <== mon.2 v2:172.21.15.102:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f9af80059e0 con 0x5635fae0c740 2024-09-10T09:44:35.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:35.733+0000 7f9ae6a00640 1 -- 172.21.15.5:0/3303288652 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f9af8044050 con 0x5635fae0c740 2024-09-10T09:44:35.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:35.733+0000 7f9b0a3cd740 10 client.26633 fetch_fsmap learned FSMap version 34 2024-09-10T09:44:35.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:35.733+0000 7f9b0a3cd740 1 -- 172.21.15.5:0/3303288652 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5635fae90780 con 0x5635fae0c740 2024-09-10T09:44:35.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:35.733+0000 7f9ae6a00640 1 -- 172.21.15.5:0/3303288652 <== mon.2 v2:172.21.15.102:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9af40a3fb0 con 0x5635fae0c740 2024-09-10T09:44:35.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:35.733+0000 7f9ae6a00640 1 -- 172.21.15.5:0/3303288652 <== mon.2 v2:172.21.15.102:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f9af8026050 con 0x5635fae0c740 2024-09-10T09:44:35.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:35.733+0000 7f9b0a3cd740 10 client.26633 fetch_fsmap finished waiting for FSMap version 34 2024-09-10T09:44:35.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:35.733+0000 7f9b0a3cd740 10 client.26633 resolve_mds: validated gid 24542 aka daemon mds.b 2024-09-10T09:44:35.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:35.733+0000 7f9b0a3cd740 20 client.26633 populate_metadata read hostname 'smithi005' 2024-09-10T09:44:35.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:35.733+0000 7f9b0a3cd740 1 --2- 172.21.15.5:0/3303288652 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x5635fae90f10 0x5635fae93300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:44:35.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:35.733+0000 7f9b0a3cd740 4 client.26633 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 140.1, "await_for": 45.0}] 2024-09-10T09:44:35.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:35.733+0000 7f9b0a3cd740 1 -- 172.21.15.5:0/3303288652 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 140.1, "await_for": 45.0}) -- 0x5635fad74c60 con 0x5635fae90f10 2024-09-10T09:44:35.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:35.744+0000 7f9b04e00640 1 --2- 172.21.15.5:0/3303288652 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x5635fae90f10 0x5635fae93300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:44:35.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:35.744+0000 7f9b04e00640 1 --2- 172.21.15.5:0/3303288652 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x5635fae90f10 0x5635fae93300 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6aca5bab6b68f7dc server_cookie=943edcc785284bc6 in_seq=0 out_seq=0 2024-09-10T09:44:35.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:35.744+0000 7f9ae6a00640 10 client.26633 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T09:44:36.046 INFO:tasks.workunit.client.0.smithi005.stdout: 1 353073 6.32 MB/sec execute 137 sec latency 648.668 ms 2024-09-10T09:44:36.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:36.363+0000 7f9ae6a00640 1 -- 172.21.15.5:0/3303288652 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x5635fad74c60 con 0x5635fae90f10 2024-09-10T09:44:36.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:36.363+0000 7f9ae6a00640 10 client.26633 handle_command_reply: tid=0 2024-09-10T09:44:36.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:36.364+0000 7f9b0a3cd740 1 client.26633 shutdown 2024-09-10T09:44:36.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:36.364+0000 7f9b0a3cd740 1 -- 172.21.15.5:0/3303288652 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5635fae0c740 msgr2=0x5635fae1a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:44:36.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:36.364+0000 7f9b0a3cd740 1 --2- 172.21.15.5:0/3303288652 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5635fae0c740 0x5635fae1a480 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f9af8018040 tx=0x7f9af8003da0 comp rx=0 tx=0).stop 2024-09-10T09:44:36.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:36.364+0000 7f9b0a3cd740 1 -- 172.21.15.5:0/3303288652 shutdown_connections 2024-09-10T09:44:36.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:36.364+0000 7f9b0a3cd740 1 --2- 172.21.15.5:0/3303288652 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x5635fae90f10 0x5635fae93300 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:36.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:36.364+0000 7f9b0a3cd740 1 --2- 172.21.15.5:0/3303288652 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5635fae0c740 0x5635fae1a480 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:36.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:36.364+0000 7f9b0a3cd740 1 --2- 172.21.15.5:0/3303288652 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5635fae01e80 0x5635fae19f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:36.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:36.364+0000 7f9b0a3cd740 1 --2- 172.21.15.5:0/3303288652 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5635fae01560 0x5635fae1d390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:36.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:36.364+0000 7f9ae6a00640 0 client.26633 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T09:44:36.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:36.364+0000 7f9b0a3cd740 1 -- 172.21.15.5:0/3303288652 >> 172.21.15.5:0/3303288652 conn(0x5635fadfdaf0 msgr2=0x5635fae09ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:44:36.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:36.364+0000 7f9b0a3cd740 1 -- 172.21.15.5:0/3303288652 shutdown_connections 2024-09-10T09:44:36.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:36.364+0000 7f9b0a3cd740 1 -- 172.21.15.5:0/3303288652 wait complete. 2024-09-10T09:44:36.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:36.364+0000 7f9b0a3cd740 20 client.26633 trim_cache size 0 max 16384 2024-09-10T09:44:36.367 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-10T09:44:36.367 INFO:teuthology.orchestra.run.smithi005.stdout: "epoch": 34, 2024-09-10T09:44:36.367 INFO:teuthology.orchestra.run.smithi005.stdout: "leader": 24542, 2024-09-10T09:44:36.367 INFO:teuthology.orchestra.run.smithi005.stdout: "set_version": 18, 2024-09-10T09:44:36.367 INFO:teuthology.orchestra.run.smithi005.stdout: "sets": { 2024-09-10T09:44:36.367 INFO:teuthology.orchestra.run.smithi005.stdout: "14ee912": { 2024-09-10T09:44:36.367 INFO:teuthology.orchestra.run.smithi005.stdout: "version": 18, 2024-09-10T09:44:36.367 INFO:teuthology.orchestra.run.smithi005.stdout: "age_ref": 0.0, 2024-09-10T09:44:36.368 INFO:teuthology.orchestra.run.smithi005.stdout: "state": { 2024-09-10T09:44:36.368 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "QUIESCED", 2024-09-10T09:44:36.368 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0 2024-09-10T09:44:36.368 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T09:44:36.368 INFO:teuthology.orchestra.run.smithi005.stdout: "timeout": 90.0, 2024-09-10T09:44:36.368 INFO:teuthology.orchestra.run.smithi005.stdout: "expiration": 140.1, 2024-09-10T09:44:36.368 INFO:teuthology.orchestra.run.smithi005.stdout: "members": { 2024-09-10T09:44:36.368 INFO:teuthology.orchestra.run.smithi005.stdout: "file:/": { 2024-09-10T09:44:36.368 INFO:teuthology.orchestra.run.smithi005.stdout: "excluded": false, 2024-09-10T09:44:36.368 INFO:teuthology.orchestra.run.smithi005.stdout: "state": { 2024-09-10T09:44:36.368 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "QUIESCED", 2024-09-10T09:44:36.368 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0 2024-09-10T09:44:36.368 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T09:44:36.368 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T09:44:36.369 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T09:44:36.369 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T09:44:36.369 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T09:44:36.369 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-10T09:44:36.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:36.365+0000 7f9b06200640 1 -- 172.21.15.5:0/1397944590 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f9ae80a2c00 msgr2=0x7f9ae80a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:44:36.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:36.365+0000 7f9b06200640 1 --2- 172.21.15.5:0/1397944590 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f9ae80a2c00 0x7f9ae80a50c0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f9af8005fd0 tx=0x7f9af8009500 comp rx=0 tx=0).stop 2024-09-10T09:44:36.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:36.365+0000 7f9b06200640 1 -- 172.21.15.5:0/1397944590 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9b00102b90 msgr2=0x7f9b00198350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:44:36.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:36.365+0000 7f9b06200640 1 --2- 172.21.15.5:0/1397944590 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9b00102b90 0x7f9b00198350 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f9af400b450 tx=0x7f9af400b910 comp rx=0 tx=0).stop 2024-09-10T09:44:36.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:36.365+0000 7f9b06200640 1 -- 172.21.15.5:0/1397944590 shutdown_connections 2024-09-10T09:44:36.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:36.365+0000 7f9b06200640 1 --2- 172.21.15.5:0/1397944590 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f9ae80a2c00 0x7f9ae80a50c0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:36.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:36.365+0000 7f9b06200640 1 --2- 172.21.15.5:0/1397944590 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9b0010bfc0 0x7f9b0019cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:36.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:36.365+0000 7f9b06200640 1 --2- 172.21.15.5:0/1397944590 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9b001034b0 0x7f9b00198890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:36.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:36.365+0000 7f9b06200640 1 --2- 172.21.15.5:0/1397944590 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9b00102b90 0x7f9b00198350 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:36.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:36.365+0000 7f9b06200640 1 -- 172.21.15.5:0/1397944590 >> 172.21.15.5:0/1397944590 conn(0x7f9b000fc610 msgr2=0x7f9b000fd2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:44:36.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:36.366+0000 7f9b06200640 1 -- 172.21.15.5:0/1397944590 shutdown_connections 2024-09-10T09:44:36.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:36.366+0000 7f9b06200640 1 -- 172.21.15.5:0/1397944590 wait complete. 2024-09-10T09:44:37.046 INFO:tasks.workunit.client.0.smithi005.stdout: 1 353114 6.27 MB/sec execute 138 sec latency 680.267 ms 2024-09-10T09:44:37.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:44:36 smithi005 ceph-mon[30350]: pgmap v581: 129 pgs: 129 active+clean; 146 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.3 MiB/s wr, 219 op/s 2024-09-10T09:44:37.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:44:36 smithi159 ceph-mon[35513]: pgmap v581: 129 pgs: 129 active+clean; 146 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.3 MiB/s wr, 219 op/s 2024-09-10T09:44:37.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:44:36 smithi102 ceph-mon[35442]: pgmap v581: 129 pgs: 129 active+clean; 146 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.3 MiB/s wr, 219 op/s 2024-09-10T09:44:37.142 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '14ee912', quiesce took 5.1 seconds. Will release after: 15.000000000000002 2024-09-10T09:44:38.046 INFO:tasks.workunit.client.0.smithi005.stdout: 1 353114 6.23 MB/sec execute 139 sec latency 1680.417 ms 2024-09-10T09:44:39.047 INFO:tasks.workunit.client.0.smithi005.stdout: 1 353114 6.18 MB/sec execute 140 sec latency 2680.609 ms 2024-09-10T09:44:39.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:44:38 smithi159 ceph-mon[35513]: pgmap v582: 129 pgs: 129 active+clean; 147 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.0 MiB/s wr, 171 op/s 2024-09-10T09:44:39.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:44:38 smithi102 ceph-mon[35442]: pgmap v582: 129 pgs: 129 active+clean; 147 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.0 MiB/s wr, 171 op/s 2024-09-10T09:44:39.306 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:44:38 smithi005 ceph-mon[30350]: pgmap v582: 129 pgs: 129 active+clean; 147 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.0 MiB/s wr, 171 op/s 2024-09-10T09:44:40.047 INFO:tasks.workunit.client.0.smithi005.stdout: 1 353114 6.14 MB/sec execute 141 sec latency 3680.709 ms 2024-09-10T09:44:41.047 INFO:tasks.workunit.client.0.smithi005.stdout: 1 353114 6.10 MB/sec execute 142 sec latency 4680.750 ms 2024-09-10T09:44:41.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:44:40 smithi005 ceph-mon[30350]: pgmap v583: 129 pgs: 129 active+clean; 146 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.0 MiB/s wr, 174 op/s 2024-09-10T09:44:41.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:44:40 smithi159 ceph-mon[35513]: pgmap v583: 129 pgs: 129 active+clean; 146 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.0 MiB/s wr, 174 op/s 2024-09-10T09:44:41.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:44:40 smithi102 ceph-mon[35442]: pgmap v583: 129 pgs: 129 active+clean; 146 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.0 MiB/s wr, 174 op/s 2024-09-10T09:44:42.046 INFO:tasks.workunit.client.0.smithi005.stdout: 1 353114 6.05 MB/sec execute 143 sec latency 5680.788 ms 2024-09-10T09:44:42.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:44:41 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:44:42.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:44:41 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:44:42.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:44:41 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:44:42.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:44:41 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:44:42.101 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:44:41 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:44:42.101 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:44:41 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:44:42.101 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:44:41 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:44:42.101 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:44:41 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:44:42.101 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:44:41 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:44:42.101 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:44:41 smithi159 ceph-mon[35513]: Adjusting osd_memory_target on smithi159 to 176.0M 2024-09-10T09:44:42.101 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:44:41 smithi159 ceph-mon[35513]: Adjusting osd_memory_target on smithi102 to 176.0M 2024-09-10T09:44:42.101 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:44:41 smithi159 ceph-mon[35513]: Unable to set osd_memory_target on smithi159 to 184591052: error parsing value: Value '184591052' is below minimum 939524096 2024-09-10T09:44:42.101 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:44:41 smithi159 ceph-mon[35513]: Unable to set osd_memory_target on smithi102 to 184589619: error parsing value: Value '184589619' is below minimum 939524096 2024-09-10T09:44:42.101 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:44:41 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:44:42.101 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:44:41 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:44:42.101 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:44:41 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:44:42.135 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:44:41 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:44:42.135 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:44:41 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:44:42.135 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:44:41 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:44:42.135 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:44:41 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:44:42.135 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:44:41 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:44:42.135 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:44:41 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:44:42.135 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:44:41 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:44:42.135 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:44:41 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:44:42.135 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:44:41 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:44:42.135 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:44:41 smithi102 ceph-mon[35442]: Adjusting osd_memory_target on smithi159 to 176.0M 2024-09-10T09:44:42.136 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:44:41 smithi102 ceph-mon[35442]: Adjusting osd_memory_target on smithi102 to 176.0M 2024-09-10T09:44:42.136 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:44:41 smithi102 ceph-mon[35442]: Unable to set osd_memory_target on smithi159 to 184591052: error parsing value: Value '184591052' is below minimum 939524096 2024-09-10T09:44:42.136 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:44:41 smithi102 ceph-mon[35442]: Unable to set osd_memory_target on smithi102 to 184589619: error parsing value: Value '184589619' is below minimum 939524096 2024-09-10T09:44:42.136 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:44:41 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:44:42.136 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:44:41 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:44:42.136 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:44:41 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:44:42.306 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:44:41 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:44:42.307 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:44:41 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:44:42.307 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:44:41 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:44:42.307 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:44:41 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:44:42.307 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:44:41 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:44:42.307 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:44:41 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:44:42.307 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:44:41 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:44:42.307 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:44:41 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:44:42.307 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:44:41 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:44:42.307 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:44:41 smithi005 ceph-mon[30350]: Adjusting osd_memory_target on smithi159 to 176.0M 2024-09-10T09:44:42.307 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:44:41 smithi005 ceph-mon[30350]: Adjusting osd_memory_target on smithi102 to 176.0M 2024-09-10T09:44:42.307 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:44:41 smithi005 ceph-mon[30350]: Unable to set osd_memory_target on smithi159 to 184591052: error parsing value: Value '184591052' is below minimum 939524096 2024-09-10T09:44:42.307 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:44:41 smithi005 ceph-mon[30350]: Unable to set osd_memory_target on smithi102 to 184589619: error parsing value: Value '184589619' is below minimum 939524096 2024-09-10T09:44:42.307 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:44:41 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:44:42.308 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:44:41 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:44:42.308 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:44:41 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:44:43.047 INFO:tasks.workunit.client.0.smithi005.stdout: 1 353114 6.01 MB/sec execute 144 sec latency 6680.837 ms 2024-09-10T09:44:43.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:44:42 smithi159 ceph-mon[35513]: pgmap v584: 129 pgs: 129 active+clean; 145 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.9 MiB/s wr, 160 op/s 2024-09-10T09:44:43.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:44:42 smithi159 ceph-mon[35513]: scrub summary: idle 2024-09-10T09:44:43.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:44:42 smithi159 ceph-mon[35513]: scrub summary: idle 2024-09-10T09:44:43.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:44:42 smithi102 ceph-mon[35442]: pgmap v584: 129 pgs: 129 active+clean; 145 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.9 MiB/s wr, 160 op/s 2024-09-10T09:44:43.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:44:42 smithi102 ceph-mon[35442]: scrub summary: idle 2024-09-10T09:44:43.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:44:42 smithi102 ceph-mon[35442]: scrub summary: idle 2024-09-10T09:44:43.306 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:44:42 smithi005 ceph-mon[30350]: pgmap v584: 129 pgs: 129 active+clean; 145 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.9 MiB/s wr, 160 op/s 2024-09-10T09:44:43.306 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:44:42 smithi005 ceph-mon[30350]: scrub summary: idle 2024-09-10T09:44:43.307 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:44:42 smithi005 ceph-mon[30350]: scrub summary: idle 2024-09-10T09:44:44.047 INFO:tasks.workunit.client.0.smithi005.stdout: 1 353114 5.97 MB/sec execute 145 sec latency 7680.935 ms 2024-09-10T09:44:45.047 INFO:tasks.workunit.client.0.smithi005.stdout: 1 353114 5.93 MB/sec execute 146 sec latency 8680.998 ms 2024-09-10T09:44:45.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:44:44 smithi102 ceph-mon[35442]: pgmap v585: 129 pgs: 129 active+clean; 145 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 974 KiB/s rd, 1.5 MiB/s wr, 117 op/s 2024-09-10T09:44:45.306 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:44:44 smithi005 ceph-mon[30350]: pgmap v585: 129 pgs: 129 active+clean; 145 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 974 KiB/s rd, 1.5 MiB/s wr, 117 op/s 2024-09-10T09:44:45.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:44:44 smithi159 ceph-mon[35513]: pgmap v585: 129 pgs: 129 active+clean; 145 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 974 KiB/s rd, 1.5 MiB/s wr, 117 op/s 2024-09-10T09:44:46.047 INFO:tasks.workunit.client.0.smithi005.stdout: 1 353114 5.89 MB/sec execute 147 sec latency 9681.072 ms 2024-09-10T09:44:47.047 INFO:tasks.workunit.client.0.smithi005.stdout: 1 353114 5.85 MB/sec execute 148 sec latency 10681.145 ms 2024-09-10T09:44:47.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:44:46 smithi102 ceph-mon[35442]: pgmap v586: 129 pgs: 129 active+clean; 145 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 743 KiB/s rd, 1.6 MiB/s wr, 122 op/s 2024-09-10T09:44:47.306 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:44:46 smithi005 ceph-mon[30350]: pgmap v586: 129 pgs: 129 active+clean; 145 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 743 KiB/s rd, 1.6 MiB/s wr, 122 op/s 2024-09-10T09:44:47.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:44:46 smithi159 ceph-mon[35513]: pgmap v586: 129 pgs: 129 active+clean; 145 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 743 KiB/s rd, 1.6 MiB/s wr, 122 op/s 2024-09-10T09:44:48.047 INFO:tasks.workunit.client.0.smithi005.stdout: 1 353114 5.81 MB/sec execute 149 sec latency 11681.274 ms 2024-09-10T09:44:49.047 INFO:tasks.workunit.client.0.smithi005.stdout: 1 353114 5.77 MB/sec execute 150 sec latency 12681.383 ms 2024-09-10T09:44:49.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:44:48 smithi102 ceph-mon[35442]: pgmap v587: 129 pgs: 129 active+clean; 146 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 443 KiB/s rd, 1.3 MiB/s wr, 65 op/s 2024-09-10T09:44:49.306 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:44:48 smithi005 ceph-mon[30350]: pgmap v587: 129 pgs: 129 active+clean; 146 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 443 KiB/s rd, 1.3 MiB/s wr, 65 op/s 2024-09-10T09:44:49.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:44:48 smithi159 ceph-mon[35513]: pgmap v587: 129 pgs: 129 active+clean; 146 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 443 KiB/s rd, 1.3 MiB/s wr, 65 op/s 2024-09-10T09:44:50.047 INFO:tasks.workunit.client.0.smithi005.stdout: 1 353114 5.73 MB/sec execute 151 sec latency 13681.559 ms 2024-09-10T09:44:51.000 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:44:50 smithi005 ceph-mon[30350]: pgmap v588: 129 pgs: 129 active+clean; 145 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 276 KiB/s rd, 1.2 MiB/s wr, 67 op/s 2024-09-10T09:44:51.047 INFO:tasks.workunit.client.0.smithi005.stdout: 1 353114 5.70 MB/sec execute 152 sec latency 14681.613 ms 2024-09-10T09:44:51.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:44:50 smithi102 ceph-mon[35442]: pgmap v588: 129 pgs: 129 active+clean; 145 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 276 KiB/s rd, 1.2 MiB/s wr, 67 op/s 2024-09-10T09:44:51.152 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:44:50 smithi159 ceph-mon[35513]: pgmap v588: 129 pgs: 129 active+clean; 145 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 276 KiB/s rd, 1.2 MiB/s wr, 67 op/s 2024-09-10T09:44:52.048 INFO:tasks.workunit.client.0.smithi005.stdout: 1 353114 5.66 MB/sec execute 153 sec latency 15681.731 ms 2024-09-10T09:44:52.143 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T09:44:52.549 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:44:53.015 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:44:52 smithi005 ceph-mon[30350]: pgmap v589: 129 pgs: 129 active+clean; 144 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 1.1 MiB/s wr, 57 op/s 2024-09-10T09:44:53.048 INFO:tasks.workunit.client.0.smithi005.stdout: 1 353114 5.62 MB/sec execute 154 sec latency 16681.805 ms 2024-09-10T09:44:53.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:53.127+0000 7fad62000640 1 -- 172.21.15.5:0/3841027453 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fad5c1079c0 msgr2=0x7fad5c107da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:44:53.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:53.127+0000 7fad62000640 1 --2- 172.21.15.5:0/3841027453 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fad5c1079c0 0x7fad5c107da0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fad50009930 tx=0x7fad5002a420 comp rx=0 tx=0).stop 2024-09-10T09:44:53.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:53.127+0000 7fad62000640 1 -- 172.21.15.5:0/3841027453 shutdown_connections 2024-09-10T09:44:53.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:53.127+0000 7fad62000640 1 --2- 172.21.15.5:0/3841027453 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fad5c102360 0x7fad5c10e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:53.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:53.127+0000 7fad62000640 1 --2- 172.21.15.5:0/3841027453 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fad5c1019c0 0x7fad5c101e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:53.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:53.127+0000 7fad62000640 1 --2- 172.21.15.5:0/3841027453 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fad5c1079c0 0x7fad5c107da0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:53.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:53.127+0000 7fad62000640 1 -- 172.21.15.5:0/3841027453 >> 172.21.15.5:0/3841027453 conn(0x7fad5c0fd740 msgr2=0x7fad5c0ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:44:53.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:53.127+0000 7fad62000640 1 -- 172.21.15.5:0/3841027453 shutdown_connections 2024-09-10T09:44:53.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:53.127+0000 7fad62000640 1 -- 172.21.15.5:0/3841027453 wait complete. 2024-09-10T09:44:53.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:53.128+0000 7fad62000640 1 Processor -- start 2024-09-10T09:44:53.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:53.129+0000 7fad62000640 1 -- start start 2024-09-10T09:44:53.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:53.129+0000 7fad62000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fad5c1019c0 0x7fad5c19a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:44:53.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:53.129+0000 7fad62000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fad5c102360 0x7fad5c19aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:44:53.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:53.129+0000 7fad62000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fad5c1079c0 0x7fad5c19eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:44:53.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:53.129+0000 7fad62000640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fad5c111e10 con 0x7fad5c102360 2024-09-10T09:44:53.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:53.129+0000 7fad62000640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fad5c111c90 con 0x7fad5c1019c0 2024-09-10T09:44:53.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:53.129+0000 7fad62000640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fad5c111f90 con 0x7fad5c1079c0 2024-09-10T09:44:53.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:53.130+0000 7fad5be00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fad5c102360 0x7fad5c19aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:44:53.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:53.130+0000 7fad5be00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fad5c102360 0x7fad5c19aa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39014/0 (socket says 172.21.15.5:39014) 2024-09-10T09:44:53.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:53.130+0000 7fad5be00640 1 -- 172.21.15.5:0/4020893206 learned_addr learned my addr 172.21.15.5:0/4020893206 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:44:53.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:53.130+0000 7fad60c00640 1 --2- 172.21.15.5:0/4020893206 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fad5c1019c0 0x7fad5c19a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:44:53.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:53.130+0000 7fad61600640 1 --2- 172.21.15.5:0/4020893206 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fad5c1079c0 0x7fad5c19eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:44:53.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:53.130+0000 7fad60c00640 1 -- 172.21.15.5:0/4020893206 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fad5c1079c0 msgr2=0x7fad5c19eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:44:53.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:53.130+0000 7fad60c00640 1 --2- 172.21.15.5:0/4020893206 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fad5c1079c0 0x7fad5c19eda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:53.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:53.130+0000 7fad60c00640 1 -- 172.21.15.5:0/4020893206 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fad5c102360 msgr2=0x7fad5c19aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:44:53.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:53.130+0000 7fad60c00640 1 --2- 172.21.15.5:0/4020893206 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fad5c102360 0x7fad5c19aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:53.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:53.130+0000 7fad60c00640 1 -- 172.21.15.5:0/4020893206 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad5c19f480 con 0x7fad5c1019c0 2024-09-10T09:44:53.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:53.130+0000 7fad5be00640 1 --2- 172.21.15.5:0/4020893206 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fad5c102360 0x7fad5c19aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:44:53.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:53.130+0000 7fad60c00640 1 --2- 172.21.15.5:0/4020893206 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fad5c1019c0 0x7fad5c19a4d0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fad5002a940 tx=0x7fad50003b20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:44:53.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:53.131+0000 7fad59600640 1 -- 172.21.15.5:0/4020893206 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad5003f070 con 0x7fad5c1019c0 2024-09-10T09:44:53.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:53.131+0000 7fad59600640 1 -- 172.21.15.5:0/4020893206 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fad50031c90 con 0x7fad5c1019c0 2024-09-10T09:44:53.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:53.131+0000 7fad62000640 1 -- 172.21.15.5:0/4020893206 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad5c19f710 con 0x7fad5c1019c0 2024-09-10T09:44:53.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:53.131+0000 7fad59600640 1 -- 172.21.15.5:0/4020893206 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad50035030 con 0x7fad5c1019c0 2024-09-10T09:44:53.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:53.131+0000 7fad62000640 1 -- 172.21.15.5:0/4020893206 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad5c1a7050 con 0x7fad5c1019c0 2024-09-10T09:44:53.135 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:44:52 smithi102 ceph-mon[35442]: pgmap v589: 129 pgs: 129 active+clean; 144 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 1.1 MiB/s wr, 57 op/s 2024-09-10T09:44:53.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:53.132+0000 7fad62000640 1 -- 172.21.15.5:0/4020893206 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad5c103100 con 0x7fad5c1019c0 2024-09-10T09:44:53.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:53.134+0000 7fad59600640 1 -- 172.21.15.5:0/4020893206 <== mon.2 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7fad50042070 con 0x7fad5c1019c0 2024-09-10T09:44:53.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:53.135+0000 7fad59600640 1 --2- 172.21.15.5:0/4020893206 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fad440a2b80 0x7fad440a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:44:53.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:53.135+0000 7fad59600640 1 -- 172.21.15.5:0/4020893206 <== mon.2 v2:172.21.15.102:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fad500f0360 con 0x7fad5c1019c0 2024-09-10T09:44:53.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:53.135+0000 7fad5be00640 1 --2- 172.21.15.5:0/4020893206 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fad440a2b80 0x7fad440a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:44:53.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:53.136+0000 7fad5be00640 1 --2- 172.21.15.5:0/4020893206 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fad440a2b80 0x7fad440a5040 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fad5c10eac0 tx=0x7fad48005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:44:53.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:53.138+0000 7fad59600640 1 -- 172.21.15.5:0/4020893206 <== mon.2 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fad500bc820 con 0x7fad5c1019c0 2024-09-10T09:44:53.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:44:52 smithi159 ceph-mon[35513]: pgmap v589: 129 pgs: 129 active+clean; 144 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 1.1 MiB/s wr, 57 op/s 2024-09-10T09:44:53.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:53.359+0000 7fad62000640 1 -- 172.21.15.5:0/4020893206 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fad5c101e20 con 0x7fad5c1019c0 2024-09-10T09:44:53.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:53.360+0000 7fad59600640 1 -- 172.21.15.5:0/4020893206 <== mon.2 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+11969 (secure 0 0 0) 0x7fad5c101e20 con 0x7fad5c1019c0 2024-09-10T09:44:53.361 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-10T09:44:53.361 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":34,"btime":"2024-09-10T09:42:57:737230+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14454,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/2970594311","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2970594311},{"type":"v1","addr":"172.21.15.5:6835","nonce":2970594311}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14514,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6839/3616502174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":3616502174},{"type":"v1","addr":"172.21.15.5:6839","nonce":3616502174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":34,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T09:35:29.626287+0000","modified":"2024-09-10T09:42:56.961831+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24542,"mds_1":24535,"mds_2":24511,"mds_3":24506,"mds_4":24443},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14520":{"gid":14520,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/2073599842","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":2073599842},{"type":"v1","addr":"172.21.15.5:6841","nonce":2073599842}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"k","rank":4,"incarnation":26,"state":"up:active","state_seq":24,"addr":"172.21.15.102:6833/2457007427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":2457007427},{"type":"v1","addr":"172.21.15.102:6833","nonce":2457007427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/356445159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":356445159},{"type":"v1","addr":"172.21.15.159:6833","nonce":356445159}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6837/2894314507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":2894314507},{"type":"v1","addr":"172.21.15.5:6837","nonce":2894314507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.102:6835/1647549111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":1647549111},{"type":"v1","addr":"172.21.15.102:6835","nonce":1647549111}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6835/952657242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":952657242},{"type":"v1","addr":"172.21.15.159:6835","nonce":952657242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":19,"addr":"172.21.15.102:6837/2689776764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":2689776764},{"type":"v1","addr":"172.21.15.102:6837","nonce":2689776764}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.159:6837/620627639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":620627639},{"type":"v1","addr":"172.21.15.159:6837","nonce":620627639}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24535":{"gid":24535,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":18,"addr":"172.21.15.159:6839/2073739599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":2073739599},{"type":"v1","addr":"172.21.15.159:6839","nonce":2073739599}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24542":{"gid":24542,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.102:6839/335409749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6838","nonce":335409749},{"type":"v1","addr":"172.21.15.102:6839","nonce":335409749}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24542,"qdb_cluster":[24542,24535,24511,24506,24443]},"id":1}]} 2024-09-10T09:44:53.361 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 34 2024-09-10T09:44:53.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:53.362+0000 7fad62000640 1 -- 172.21.15.5:0/4020893206 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fad440a2b80 msgr2=0x7fad440a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:44:53.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:53.362+0000 7fad62000640 1 --2- 172.21.15.5:0/4020893206 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fad440a2b80 0x7fad440a5040 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fad5c10eac0 tx=0x7fad48005f90 comp rx=0 tx=0).stop 2024-09-10T09:44:53.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:53.362+0000 7fad62000640 1 -- 172.21.15.5:0/4020893206 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fad5c1019c0 msgr2=0x7fad5c19a4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:44:53.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:53.362+0000 7fad62000640 1 --2- 172.21.15.5:0/4020893206 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fad5c1019c0 0x7fad5c19a4d0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fad5002a940 tx=0x7fad50003b20 comp rx=0 tx=0).stop 2024-09-10T09:44:53.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:53.362+0000 7fad62000640 1 -- 172.21.15.5:0/4020893206 shutdown_connections 2024-09-10T09:44:53.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:53.362+0000 7fad62000640 1 --2- 172.21.15.5:0/4020893206 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fad440a2b80 0x7fad440a5040 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:53.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:53.362+0000 7fad62000640 1 --2- 172.21.15.5:0/4020893206 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fad5c1079c0 0x7fad5c19eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:53.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:53.362+0000 7fad62000640 1 --2- 172.21.15.5:0/4020893206 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fad5c102360 0x7fad5c19aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:53.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:53.362+0000 7fad62000640 1 --2- 172.21.15.5:0/4020893206 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fad5c1019c0 0x7fad5c19a4d0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:53.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:53.362+0000 7fad62000640 1 -- 172.21.15.5:0/4020893206 >> 172.21.15.5:0/4020893206 conn(0x7fad5c0fd740 msgr2=0x7fad5c10cf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:44:53.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:53.363+0000 7fad62000640 1 -- 172.21.15.5:0/4020893206 shutdown_connections 2024-09-10T09:44:53.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:53.363+0000 7fad62000640 1 -- 172.21.15.5:0/4020893206 wait complete. 2024-09-10T09:44:54.048 INFO:tasks.workunit.client.0.smithi005.stdout: 1 353114 5.59 MB/sec execute 155 sec latency 17681.928 ms 2024-09-10T09:44:54.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:44:53 smithi005 ceph-mon[30350]: from='client.? 172.21.15.5:0/4020893206' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:44:54.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:44:53 smithi102 ceph-mon[35442]: from='client.? 172.21.15.5:0/4020893206' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:44:54.242 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24542 quiesce db --set-id 14ee912 --release --await' 2024-09-10T09:44:54.242 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24542 quiesce db --set-id 14ee912 --release --await 2024-09-10T09:44:54.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:44:53 smithi159 ceph-mon[35513]: from='client.? 172.21.15.5:0/4020893206' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:44:54.620 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:44:55.048 INFO:tasks.workunit.client.0.smithi005.stdout: 1 353114 5.55 MB/sec execute 156 sec latency 18682.057 ms 2024-09-10T09:44:55.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:44:54 smithi005 ceph-mon[30350]: pgmap v590: 129 pgs: 129 active+clean; 144 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 1.1 MiB/s wr, 51 op/s 2024-09-10T09:44:55.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:44:54 smithi102 ceph-mon[35442]: pgmap v590: 129 pgs: 129 active+clean; 144 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 1.1 MiB/s wr, 51 op/s 2024-09-10T09:44:55.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:55.230+0000 7fe08bc00640 1 -- 172.21.15.5:0/1764140887 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe084109140 msgr2=0x7fe084110cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:44:55.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:55.230+0000 7fe08bc00640 1 --2- 172.21.15.5:0/1764140887 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe084109140 0x7fe084110cc0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fe07c009ca0 tx=0x7fe07c02a8c0 comp rx=0 tx=0).stop 2024-09-10T09:44:55.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:55.230+0000 7fe08bc00640 1 -- 172.21.15.5:0/1764140887 shutdown_connections 2024-09-10T09:44:55.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:55.230+0000 7fe08bc00640 1 --2- 172.21.15.5:0/1764140887 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe084109140 0x7fe084110cc0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:55.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:55.230+0000 7fe08bc00640 1 --2- 172.21.15.5:0/1764140887 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe0841049c0 0x7fe084108a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:55.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:55.230+0000 7fe08bc00640 1 --2- 172.21.15.5:0/1764140887 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe084104010 0x7fe0841043f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:55.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:55.230+0000 7fe08bc00640 1 -- 172.21.15.5:0/1764140887 >> 172.21.15.5:0/1764140887 conn(0x7fe0840ffa50 msgr2=0x7fe084101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:44:55.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:55.230+0000 7fe08bc00640 1 -- 172.21.15.5:0/1764140887 shutdown_connections 2024-09-10T09:44:55.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:55.230+0000 7fe08bc00640 1 -- 172.21.15.5:0/1764140887 wait complete. 2024-09-10T09:44:55.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:55.232+0000 7fe08bc00640 1 Processor -- start 2024-09-10T09:44:55.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:55.232+0000 7fe08bc00640 1 -- start start 2024-09-10T09:44:55.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:55.232+0000 7fe08bc00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe084104010 0x7fe08419a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:44:55.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:55.232+0000 7fe08bc00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe0841049c0 0x7fe08419ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:44:55.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:55.232+0000 7fe08bc00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe084109140 0x7fe08419ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:44:55.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:55.232+0000 7fe08bc00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe0841141c0 con 0x7fe084109140 2024-09-10T09:44:55.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:55.232+0000 7fe08bc00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fe084114040 con 0x7fe0841049c0 2024-09-10T09:44:55.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:55.232+0000 7fe08bc00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fe084114340 con 0x7fe084104010 2024-09-10T09:44:55.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:55.233+0000 7fe089c00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe084109140 0x7fe08419ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:44:55.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:55.233+0000 7fe089c00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe084109140 0x7fe08419ef90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39038/0 (socket says 172.21.15.5:39038) 2024-09-10T09:44:55.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:55.233+0000 7fe089c00640 1 -- 172.21.15.5:0/3737893288 learned_addr learned my addr 172.21.15.5:0/3737893288 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:44:55.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:55.233+0000 7fe083e00640 1 --2- 172.21.15.5:0/3737893288 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe0841049c0 0x7fe08419ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:44:55.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:55.233+0000 7fe089200640 1 --2- 172.21.15.5:0/3737893288 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe084104010 0x7fe08419a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:44:55.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:55.233+0000 7fe083e00640 1 -- 172.21.15.5:0/3737893288 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe084104010 msgr2=0x7fe08419a6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:44:55.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:55.233+0000 7fe083e00640 1 --2- 172.21.15.5:0/3737893288 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe084104010 0x7fe08419a6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:55.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:55.233+0000 7fe083e00640 1 -- 172.21.15.5:0/3737893288 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe084109140 msgr2=0x7fe08419ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:44:55.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:55.233+0000 7fe083e00640 1 --2- 172.21.15.5:0/3737893288 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe084109140 0x7fe08419ef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:55.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:55.233+0000 7fe083e00640 1 -- 172.21.15.5:0/3737893288 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe08419f670 con 0x7fe0841049c0 2024-09-10T09:44:55.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:55.233+0000 7fe083e00640 1 --2- 172.21.15.5:0/3737893288 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe0841049c0 0x7fe08419ac00 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fe07000e9f0 tx=0x7fe07000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:44:55.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:55.234+0000 7fe081600640 1 -- 172.21.15.5:0/3737893288 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe07000ccc0 con 0x7fe0841049c0 2024-09-10T09:44:55.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:55.234+0000 7fe081600640 1 -- 172.21.15.5:0/3737893288 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe070005020 con 0x7fe0841049c0 2024-09-10T09:44:55.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:55.234+0000 7fe08bc00640 1 -- 172.21.15.5:0/3737893288 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe08419f960 con 0x7fe0841049c0 2024-09-10T09:44:55.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:55.234+0000 7fe081600640 1 -- 172.21.15.5:0/3737893288 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0700049e0 con 0x7fe0841049c0 2024-09-10T09:44:55.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:55.234+0000 7fe08bc00640 1 -- 172.21.15.5:0/3737893288 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0841a7240 con 0x7fe0841049c0 2024-09-10T09:44:55.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:55.237+0000 7fe08e867740 1 Processor -- start 2024-09-10T09:44:55.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:55.237+0000 7fe08e867740 1 -- start start 2024-09-10T09:44:55.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:55.237+0000 7fe081600640 1 -- 172.21.15.5:0/3737893288 <== mon.2 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7fe070005560 con 0x7fe0841049c0 2024-09-10T09:44:55.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:55.237+0000 7fe08e867740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55788290f310 0x55788290f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:44:55.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:55.237+0000 7fe08e867740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55788290fc30 0x557882919fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:44:55.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:55.237+0000 7fe08e867740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55788291a4f0 0x55788291c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:44:55.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:55.237+0000 7fe08e867740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x557882875650 con 0x55788291a4f0 2024-09-10T09:44:55.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:55.237+0000 7fe08e867740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55788281cf10 con 0x55788290f310 2024-09-10T09:44:55.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:55.237+0000 7fe08e867740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x557882871610 con 0x55788290fc30 2024-09-10T09:44:55.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:55.238+0000 7fe089200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55788290fc30 0x557882919fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:44:55.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:55.238+0000 7fe089c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55788290f310 0x55788290f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:44:55.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:55.238+0000 7fe089c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55788290f310 0x55788290f6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:59246/0 (socket says 172.21.15.5:59246) 2024-09-10T09:44:55.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:55.238+0000 7fe089c00640 1 -- 172.21.15.5:0/2438328643 learned_addr learned my addr 172.21.15.5:0/2438328643 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:44:55.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:55.238+0000 7fe083e00640 1 --2- 172.21.15.5:0/2438328643 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55788291a4f0 0x55788291c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:44:55.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:55.238+0000 7fe089200640 1 -- 172.21.15.5:0/2438328643 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55788290f310 msgr2=0x55788290f6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:44:55.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:55.238+0000 7fe089200640 1 --2- 172.21.15.5:0/2438328643 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55788290f310 0x55788290f6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:55.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:55.238+0000 7fe089200640 1 -- 172.21.15.5:0/2438328643 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55788291a4f0 msgr2=0x55788291c8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:44:55.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:55.238+0000 7fe089200640 1 --2- 172.21.15.5:0/2438328643 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55788291a4f0 0x55788291c8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:55.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:55.238+0000 7fe089200640 1 -- 172.21.15.5:0/2438328643 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557882584580 con 0x55788290fc30 2024-09-10T09:44:55.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:55.238+0000 7fe089c00640 1 --2- 172.21.15.5:0/2438328643 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55788290f310 0x55788290f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:44:55.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:55.238+0000 7fe083e00640 1 --2- 172.21.15.5:0/2438328643 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55788291a4f0 0x55788291c8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:44:55.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:55.238+0000 7fe089200640 1 --2- 172.21.15.5:0/2438328643 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55788290fc30 0x557882919fb0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fe078009c20 tx=0x7fe078011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3ee6f7335f439335 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:44:55.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:55.239+0000 7fe081600640 1 --2- 172.21.15.5:0/3737893288 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fe06c0a2b80 0x7fe06c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:44:55.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:55.239+0000 7fe076000640 1 -- 172.21.15.5:0/2438328643 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe078004050 con 0x55788290fc30 2024-09-10T09:44:55.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:55.239+0000 7fe081600640 1 -- 172.21.15.5:0/3737893288 <== mon.2 v2:172.21.15.102:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fe0700cf360 con 0x7fe0841049c0 2024-09-10T09:44:55.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:55.239+0000 7fe076000640 1 -- 172.21.15.5:0/2438328643 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe078005500 con 0x55788290fc30 2024-09-10T09:44:55.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:55.239+0000 7fe076000640 1 -- 172.21.15.5:0/2438328643 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0780035f0 con 0x55788290fc30 2024-09-10T09:44:55.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:55.239+0000 7fe089c00640 1 --2- 172.21.15.5:0/3737893288 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fe06c0a2b80 0x7fe06c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:44:55.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:55.240+0000 7fe08e867740 1 -- 172.21.15.5:0/2438328643 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55788290fc30 msgr2=0x557882919fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:44:55.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:55.240+0000 7fe08e867740 1 --2- 172.21.15.5:0/2438328643 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55788290fc30 0x557882919fb0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fe078009c20 tx=0x7fe078011810 comp rx=0 tx=0).stop 2024-09-10T09:44:55.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:55.240+0000 7fe089c00640 1 --2- 172.21.15.5:0/3737893288 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fe06c0a2b80 0x7fe06c0a5040 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fe07c031c90 tx=0x7fe07c005d80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:44:55.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:55.240+0000 7fe08e867740 1 -- 172.21.15.5:0/2438328643 shutdown_connections 2024-09-10T09:44:55.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:55.240+0000 7fe08e867740 1 --2- 172.21.15.5:0/2438328643 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55788291a4f0 0x55788291c8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:55.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:55.240+0000 7fe08e867740 1 --2- 172.21.15.5:0/2438328643 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55788290fc30 0x557882919fb0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:55.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:55.240+0000 7fe08e867740 1 --2- 172.21.15.5:0/2438328643 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55788290f310 0x55788290f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:55.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:55.240+0000 7fe08e867740 1 -- 172.21.15.5:0/2438328643 >> 172.21.15.5:0/2438328643 conn(0x55788290b8a0 msgr2=0x55788290bc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:44:55.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:55.240+0000 7fe08e867740 1 -- 172.21.15.5:0/2438328643 shutdown_connections 2024-09-10T09:44:55.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:55.240+0000 7fe08e867740 1 -- 172.21.15.5:0/2438328643 wait complete. 2024-09-10T09:44:55.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:55.240+0000 7fe08e867740 1 -- start start 2024-09-10T09:44:55.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:55.241+0000 7fe08e867740 1 Processor -- start 2024-09-10T09:44:55.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:55.241+0000 7fe08e867740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55788290f310 0x557882917390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:44:55.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:55.241+0000 7fe08e867740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55788290fc30 0x5578829178d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:44:55.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:55.241+0000 7fe08e867740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55788291a4f0 0x557882917e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:44:55.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:55.241+0000 7fe08e867740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55788291d0c0 con 0x55788290f310 2024-09-10T09:44:55.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:55.241+0000 7fe08e867740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x557882830350 con 0x55788290fc30 2024-09-10T09:44:55.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:55.241+0000 7fe08e867740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55788291d240 con 0x55788291a4f0 2024-09-10T09:44:55.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:55.241+0000 7fe08e867740 10 client.0 _ll_register_callbacks cb 0x557882853290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:44:55.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:55.242+0000 7fe089200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55788290f310 0x557882917390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:44:55.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:55.242+0000 7fe089200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55788290f310 0x557882917390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39062/0 (socket says 172.21.15.5:39062) 2024-09-10T09:44:55.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:55.242+0000 7fe089200640 1 -- 172.21.15.5:0/2910570334 learned_addr learned my addr 172.21.15.5:0/2910570334 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:44:55.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:55.242+0000 7fe083e00640 1 --2- 172.21.15.5:0/2910570334 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55788290fc30 0x5578829178d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:44:55.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:55.242+0000 7fe089c00640 1 --2- 172.21.15.5:0/2910570334 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55788291a4f0 0x557882917e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:44:55.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:55.242+0000 7fe083e00640 1 -- 172.21.15.5:0/2910570334 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55788291a4f0 msgr2=0x557882917e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:44:55.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:55.242+0000 7fe083e00640 1 --2- 172.21.15.5:0/2910570334 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55788291a4f0 0x557882917e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:55.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:55.242+0000 7fe083e00640 1 -- 172.21.15.5:0/2910570334 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55788290f310 msgr2=0x557882917390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:44:55.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:55.242+0000 7fe083e00640 1 --2- 172.21.15.5:0/2910570334 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55788290f310 0x557882917390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:55.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:55.242+0000 7fe083e00640 1 -- 172.21.15.5:0/2910570334 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55788264e960 con 0x55788290fc30 2024-09-10T09:44:55.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:55.242+0000 7fe083e00640 1 --2- 172.21.15.5:0/2910570334 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55788290fc30 0x5578829178d0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x55788290cc20 tx=0x7fe07009b600 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8107f56a5ee47e89 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:44:55.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:55.242+0000 7fe06aa00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-09-10T09:44:55.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:55.243+0000 7fe06aa00640 1 -- 172.21.15.5:0/2910570334 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe07001c070 con 0x55788290fc30 2024-09-10T09:44:55.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:55.243+0000 7fe06aa00640 1 -- 172.21.15.5:0/2910570334 <== mon.2 v2:172.21.15.102:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fe070097de0 con 0x55788290fc30 2024-09-10T09:44:55.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:55.243+0000 7fe08e867740 1 -- 172.21.15.5:0/2910570334 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe078005500 con 0x55788290fc30 2024-09-10T09:44:55.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:55.243+0000 7fe06aa00640 1 -- 172.21.15.5:0/2910570334 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fe07009be70 con 0x55788290fc30 2024-09-10T09:44:55.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:55.243+0000 7fe08e867740 10 client.26651 fetch_fsmap learned FSMap version 34 2024-09-10T09:44:55.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:55.243+0000 7fe08e867740 1 -- 172.21.15.5:0/2910570334 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x557882928cf0 con 0x55788290fc30 2024-09-10T09:44:55.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:55.243+0000 7fe06aa00640 1 -- 172.21.15.5:0/2910570334 <== mon.2 v2:172.21.15.102:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe078005500 con 0x55788290fc30 2024-09-10T09:44:55.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:55.243+0000 7fe06aa00640 1 -- 172.21.15.5:0/2910570334 <== mon.2 v2:172.21.15.102:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7fe0700d2c70 con 0x55788290fc30 2024-09-10T09:44:55.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:55.243+0000 7fe08e867740 10 client.26651 fetch_fsmap finished waiting for FSMap version 34 2024-09-10T09:44:55.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:55.243+0000 7fe08e867740 10 client.26651 resolve_mds: validated gid 24542 aka daemon mds.b 2024-09-10T09:44:55.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:55.244+0000 7fe08e867740 20 client.26651 populate_metadata read hostname 'smithi005' 2024-09-10T09:44:55.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:55.244+0000 7fe08e867740 1 --2- 172.21.15.5:0/2910570334 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55788292a1b0 0x55788292c620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:44:55.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:55.244+0000 7fe08e867740 4 client.26651 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:44:55.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:55.244+0000 7fe08e867740 1 -- 172.21.15.5:0/2910570334 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557882882bc0 con 0x55788292a1b0 2024-09-10T09:44:55.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:55.244+0000 7fe089200640 1 --2- 172.21.15.5:0/2910570334 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55788292a1b0 0x55788292c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:44:55.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:55.245+0000 7fe089200640 1 --2- 172.21.15.5:0/2910570334 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55788292a1b0 0x55788292c620 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e9f4ba675c249a89 server_cookie=8ac26bb7195421c6 in_seq=0 out_seq=0 2024-09-10T09:44:55.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:55.245+0000 7fe06aa00640 10 client.26651 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T09:44:55.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:55.245+0000 7fe06aa00640 1 -- 172.21.15.5:0/2910570334 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557882882bc0 con 0x55788292a1b0 2024-09-10T09:44:55.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:55.245+0000 7fe06aa00640 10 client.26651 handle_command_reply: tid=0 2024-09-10T09:44:55.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:55.246+0000 7fe08e867740 1 client.26651 shutdown 2024-09-10T09:44:55.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:55.246+0000 7fe08e867740 1 -- 172.21.15.5:0/2910570334 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55788290fc30 msgr2=0x5578829178d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:44:55.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:55.246+0000 7fe08e867740 1 --2- 172.21.15.5:0/2910570334 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55788290fc30 0x5578829178d0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x55788290cc20 tx=0x7fe07009b600 comp rx=0 tx=0).stop 2024-09-10T09:44:55.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:55.246+0000 7fe08e867740 1 -- 172.21.15.5:0/2910570334 shutdown_connections 2024-09-10T09:44:55.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:55.246+0000 7fe08e867740 1 --2- 172.21.15.5:0/2910570334 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55788292a1b0 0x55788292c620 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:55.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:55.246+0000 7fe08e867740 1 --2- 172.21.15.5:0/2910570334 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55788291a4f0 0x557882917e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:55.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:55.246+0000 7fe08e867740 1 --2- 172.21.15.5:0/2910570334 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55788290fc30 0x5578829178d0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:55.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:55.246+0000 7fe08e867740 1 --2- 172.21.15.5:0/2910570334 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55788290f310 0x557882917390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:55.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:55.246+0000 7fe08e867740 1 -- 172.21.15.5:0/2910570334 >> 172.21.15.5:0/2910570334 conn(0x55788290b8a0 msgr2=0x55788290aaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:44:55.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:55.246+0000 7fe06aa00640 0 client.26651 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T09:44:55.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:55.246+0000 7fe08e867740 1 -- 172.21.15.5:0/2910570334 shutdown_connections 2024-09-10T09:44:55.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:55.246+0000 7fe08e867740 1 -- 172.21.15.5:0/2910570334 wait complete. 2024-09-10T09:44:55.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:55.246+0000 7fe08e867740 20 client.26651 trim_cache size 0 max 16384 2024-09-10T09:44:55.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:55.264+0000 7fe08e867740 1 Processor -- start 2024-09-10T09:44:55.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:55.264+0000 7fe08e867740 1 -- start start 2024-09-10T09:44:55.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:55.264+0000 7fe08e867740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55788290f310 0x5578829162d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:44:55.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:55.265+0000 7fe08e867740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55788290fc30 0x557882931ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:44:55.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:55.265+0000 7fe08e867740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55788291a4f0 0x557882932430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:44:55.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:55.265+0000 7fe08e867740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe0841b1120 con 0x55788290f310 2024-09-10T09:44:55.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:55.265+0000 7fe08e867740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fe0841b2c60 con 0x55788290fc30 2024-09-10T09:44:55.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:55.265+0000 7fe08e867740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fe0841b17f0 con 0x55788291a4f0 2024-09-10T09:44:55.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:55.265+0000 7fe089200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55788290f310 0x5578829162d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:44:55.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:55.265+0000 7fe089200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55788290f310 0x5578829162d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39070/0 (socket says 172.21.15.5:39070) 2024-09-10T09:44:55.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:55.265+0000 7fe089200640 1 -- 172.21.15.5:0/2305238216 learned_addr learned my addr 172.21.15.5:0/2305238216 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:44:55.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:55.265+0000 7fe083e00640 1 --2- 172.21.15.5:0/2305238216 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55788290fc30 0x557882931ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:44:55.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:55.265+0000 7fe089c00640 1 --2- 172.21.15.5:0/2305238216 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55788291a4f0 0x557882932430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:44:55.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:55.265+0000 7fe089200640 1 -- 172.21.15.5:0/2305238216 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55788291a4f0 msgr2=0x557882932430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:44:55.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:55.265+0000 7fe089200640 1 --2- 172.21.15.5:0/2305238216 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55788291a4f0 0x557882932430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:55.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:55.265+0000 7fe089200640 1 -- 172.21.15.5:0/2305238216 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55788290fc30 msgr2=0x557882931ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:44:55.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:55.265+0000 7fe089200640 1 --2- 172.21.15.5:0/2305238216 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55788290fc30 0x557882931ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:55.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:55.265+0000 7fe089200640 1 -- 172.21.15.5:0/2305238216 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55788290b6d0 con 0x55788290f310 2024-09-10T09:44:55.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:55.266+0000 7fe089200640 1 --2- 172.21.15.5:0/2305238216 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55788290f310 0x5578829162d0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fe078014040 tx=0x7fe078004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbb2dc7bc889fd91 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:44:55.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:55.266+0000 7fe075600640 1 -- 172.21.15.5:0/2305238216 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe078004250 con 0x55788290f310 2024-09-10T09:44:55.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:55.266+0000 7fe075600640 1 -- 172.21.15.5:0/2305238216 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe0780043f0 con 0x55788290f310 2024-09-10T09:44:55.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:55.266+0000 7fe075600640 1 -- 172.21.15.5:0/2305238216 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe078003d30 con 0x55788290f310 2024-09-10T09:44:55.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:55.266+0000 7fe08e867740 1 -- 172.21.15.5:0/2305238216 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55788290f310 msgr2=0x5578829162d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:44:55.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:55.266+0000 7fe08e867740 1 --2- 172.21.15.5:0/2305238216 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55788290f310 0x5578829162d0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fe078014040 tx=0x7fe078004b40 comp rx=0 tx=0).stop 2024-09-10T09:44:55.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:55.267+0000 7fe08e867740 1 -- 172.21.15.5:0/2305238216 shutdown_connections 2024-09-10T09:44:55.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:55.267+0000 7fe08e867740 1 --2- 172.21.15.5:0/2305238216 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55788291a4f0 0x557882932430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:55.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:55.267+0000 7fe08e867740 1 --2- 172.21.15.5:0/2305238216 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55788290fc30 0x557882931ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:55.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:55.267+0000 7fe08e867740 1 --2- 172.21.15.5:0/2305238216 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55788290f310 0x5578829162d0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:55.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:55.267+0000 7fe08e867740 1 -- 172.21.15.5:0/2305238216 >> 172.21.15.5:0/2305238216 conn(0x55788290b8a0 msgr2=0x557882919300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:44:55.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:55.267+0000 7fe08e867740 1 -- 172.21.15.5:0/2305238216 shutdown_connections 2024-09-10T09:44:55.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:55.267+0000 7fe08e867740 1 -- 172.21.15.5:0/2305238216 wait complete. 2024-09-10T09:44:55.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:55.267+0000 7fe08e867740 1 -- start start 2024-09-10T09:44:55.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:55.267+0000 7fe08e867740 1 Processor -- start 2024-09-10T09:44:55.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:55.268+0000 7fe08e867740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55788290f310 0x55788291e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:44:55.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:55.268+0000 7fe08e867740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55788290fc30 0x55788291e870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:44:55.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:55.268+0000 7fe08e867740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55788291a4f0 0x55788291edb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:44:55.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:55.268+0000 7fe08e867740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe0841b2590 con 0x55788290f310 2024-09-10T09:44:55.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:55.268+0000 7fe08e867740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fe0841b1ec0 con 0x55788290fc30 2024-09-10T09:44:55.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:55.268+0000 7fe08e867740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fe0841af100 con 0x55788291a4f0 2024-09-10T09:44:55.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:55.268+0000 7fe08e867740 10 client.0 _ll_register_callbacks cb 0x7fe070097a10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:44:55.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:55.268+0000 7fe089200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55788290f310 0x55788291e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:44:55.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:55.268+0000 7fe083e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55788290fc30 0x55788291e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:44:55.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:55.268+0000 7fe089c00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55788291a4f0 0x55788291edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:44:55.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:55.268+0000 7fe089200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55788290f310 0x55788291e330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39084/0 (socket says 172.21.15.5:39084) 2024-09-10T09:44:55.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:55.268+0000 7fe089200640 1 -- 172.21.15.5:0/3127887474 learned_addr learned my addr 172.21.15.5:0/3127887474 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:44:55.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:55.268+0000 7fe089200640 1 -- 172.21.15.5:0/3127887474 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55788291a4f0 msgr2=0x55788291edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:44:55.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:55.268+0000 7fe089200640 1 --2- 172.21.15.5:0/3127887474 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55788291a4f0 0x55788291edb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:55.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:55.268+0000 7fe089200640 1 -- 172.21.15.5:0/3127887474 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55788290fc30 msgr2=0x55788291e870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:44:55.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:55.268+0000 7fe089200640 1 --2- 172.21.15.5:0/3127887474 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55788290fc30 0x55788291e870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:55.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:55.268+0000 7fe089200640 1 -- 172.21.15.5:0/3127887474 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557882994020 con 0x55788290f310 2024-09-10T09:44:55.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:55.269+0000 7fe083e00640 1 --2- 172.21.15.5:0/3127887474 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55788290fc30 0x55788291e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:44:55.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:55.269+0000 7fe089200640 1 --2- 172.21.15.5:0/3127887474 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55788290f310 0x55788291e330 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fe078009800 tx=0x7fe078011e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79a71703ab163d3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:44:55.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:55.269+0000 7fe06aa00640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-10T09:44:55.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:55.269+0000 7fe06aa00640 1 -- 172.21.15.5:0/3127887474 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe078004250 con 0x55788290f310 2024-09-10T09:44:55.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:55.269+0000 7fe08e867740 1 -- 172.21.15.5:0/3127887474 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe0780043f0 con 0x55788290f310 2024-09-10T09:44:55.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:55.269+0000 7fe06aa00640 1 -- 172.21.15.5:0/3127887474 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fe0780035f0 con 0x55788290f310 2024-09-10T09:44:55.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:55.269+0000 7fe06aa00640 1 -- 172.21.15.5:0/3127887474 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fe07803f050 con 0x55788290f310 2024-09-10T09:44:55.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:55.269+0000 7fe08e867740 10 client.16638 fetch_fsmap learned FSMap version 34 2024-09-10T09:44:55.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:55.269+0000 7fe08e867740 1 -- 172.21.15.5:0/3127887474 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x557882995320 con 0x55788290f310 2024-09-10T09:44:55.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:55.270+0000 7fe06aa00640 1 -- 172.21.15.5:0/3127887474 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe0780043f0 con 0x55788290f310 2024-09-10T09:44:55.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:55.270+0000 7fe06aa00640 1 -- 172.21.15.5:0/3127887474 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7fe07802e6b0 con 0x55788290f310 2024-09-10T09:44:55.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:55.270+0000 7fe08e867740 10 client.16638 fetch_fsmap finished waiting for FSMap version 34 2024-09-10T09:44:55.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:55.270+0000 7fe08e867740 10 client.16638 resolve_mds: validated gid 24542 aka daemon mds.b 2024-09-10T09:44:55.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:55.270+0000 7fe08e867740 20 client.16638 populate_metadata read hostname 'smithi005' 2024-09-10T09:44:55.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:55.270+0000 7fe08e867740 1 --2- 172.21.15.5:0/3127887474 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x5578829957e0 0x557882997bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:44:55.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:55.270+0000 7fe08e867740 4 client.16638 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "14ee912", "release": true, "await": true}] 2024-09-10T09:44:55.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:55.270+0000 7fe08e867740 1 -- 172.21.15.5:0/3127887474 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "quiesce db", "set_id": "14ee912", "release": true, "await": true}) -- 0x55788287d0d0 con 0x5578829957e0 2024-09-10T09:44:55.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:55.270+0000 7fe083e00640 1 --2- 172.21.15.5:0/3127887474 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x5578829957e0 0x557882997bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:44:55.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:55.271+0000 7fe083e00640 1 --2- 172.21.15.5:0/3127887474 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x5578829957e0 0x557882997bd0 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=14d790285514bc3d server_cookie=245cb4cb73510dec in_seq=0 out_seq=0 2024-09-10T09:44:55.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:55.271+0000 7fe06aa00640 10 client.16638 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T09:44:55.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:55.279+0000 7fe06aa00640 1 -- 172.21.15.5:0/3127887474 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x55788287d0d0 con 0x5578829957e0 2024-09-10T09:44:55.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:55.279+0000 7fe06aa00640 10 client.16638 handle_command_reply: tid=0 2024-09-10T09:44:55.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:55.279+0000 7fe08e867740 1 client.16638 shutdown 2024-09-10T09:44:55.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:55.279+0000 7fe08e867740 1 -- 172.21.15.5:0/3127887474 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55788290f310 msgr2=0x55788291e330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:44:55.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:55.279+0000 7fe08e867740 1 --2- 172.21.15.5:0/3127887474 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55788290f310 0x55788291e330 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fe078009800 tx=0x7fe078011e50 comp rx=0 tx=0).stop 2024-09-10T09:44:55.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:55.279+0000 7fe08e867740 1 -- 172.21.15.5:0/3127887474 shutdown_connections 2024-09-10T09:44:55.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:55.279+0000 7fe08e867740 1 --2- 172.21.15.5:0/3127887474 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x5578829957e0 0x557882997bd0 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:55.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:55.279+0000 7fe08e867740 1 --2- 172.21.15.5:0/3127887474 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55788291a4f0 0x55788291edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:55.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:55.279+0000 7fe08e867740 1 --2- 172.21.15.5:0/3127887474 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55788290fc30 0x55788291e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:55.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:55.279+0000 7fe08e867740 1 --2- 172.21.15.5:0/3127887474 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55788290f310 0x55788291e330 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:55.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:55.279+0000 7fe06aa00640 0 client.16638 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T09:44:55.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:55.279+0000 7fe08e867740 1 -- 172.21.15.5:0/3127887474 >> 172.21.15.5:0/3127887474 conn(0x55788290b8a0 msgr2=0x557882910d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:44:55.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:55.279+0000 7fe08e867740 1 -- 172.21.15.5:0/3127887474 shutdown_connections 2024-09-10T09:44:55.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:55.279+0000 7fe08e867740 1 -- 172.21.15.5:0/3127887474 wait complete. 2024-09-10T09:44:55.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:55.279+0000 7fe08e867740 20 client.16638 trim_cache size 0 max 16384 2024-09-10T09:44:55.281 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-10T09:44:55.282 INFO:teuthology.orchestra.run.smithi005.stdout: "epoch": 34, 2024-09-10T09:44:55.282 INFO:teuthology.orchestra.run.smithi005.stdout: "leader": 24542, 2024-09-10T09:44:55.282 INFO:teuthology.orchestra.run.smithi005.stdout: "set_version": 20, 2024-09-10T09:44:55.282 INFO:teuthology.orchestra.run.smithi005.stdout: "sets": { 2024-09-10T09:44:55.282 INFO:teuthology.orchestra.run.smithi005.stdout: "14ee912": { 2024-09-10T09:44:55.282 INFO:teuthology.orchestra.run.smithi005.stdout: "version": 20, 2024-09-10T09:44:55.282 INFO:teuthology.orchestra.run.smithi005.stdout: "age_ref": 0.0, 2024-09-10T09:44:55.282 INFO:teuthology.orchestra.run.smithi005.stdout: "state": { 2024-09-10T09:44:55.282 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "RELEASED", 2024-09-10T09:44:55.283 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0 2024-09-10T09:44:55.283 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T09:44:55.283 INFO:teuthology.orchestra.run.smithi005.stdout: "timeout": 90.0, 2024-09-10T09:44:55.283 INFO:teuthology.orchestra.run.smithi005.stdout: "expiration": 140.1, 2024-09-10T09:44:55.283 INFO:teuthology.orchestra.run.smithi005.stdout: "members": { 2024-09-10T09:44:55.283 INFO:teuthology.orchestra.run.smithi005.stdout: "file:/": { 2024-09-10T09:44:55.283 INFO:teuthology.orchestra.run.smithi005.stdout: "excluded": false, 2024-09-10T09:44:55.283 INFO:teuthology.orchestra.run.smithi005.stdout: "state": { 2024-09-10T09:44:55.283 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "RELEASED", 2024-09-10T09:44:55.283 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0 2024-09-10T09:44:55.283 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T09:44:55.284 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T09:44:55.284 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T09:44:55.284 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T09:44:55.284 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T09:44:55.284 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-10T09:44:55.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:55.280+0000 7fe08bc00640 1 -- 172.21.15.5:0/3737893288 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fe06c0a2b80 msgr2=0x7fe06c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:44:55.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:55.280+0000 7fe08bc00640 1 --2- 172.21.15.5:0/3737893288 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fe06c0a2b80 0x7fe06c0a5040 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fe07c031c90 tx=0x7fe07c005d80 comp rx=0 tx=0).stop 2024-09-10T09:44:55.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:55.280+0000 7fe08bc00640 1 -- 172.21.15.5:0/3737893288 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe0841049c0 msgr2=0x7fe08419ac00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:44:55.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:55.280+0000 7fe08bc00640 1 --2- 172.21.15.5:0/3737893288 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe0841049c0 0x7fe08419ac00 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fe07000e9f0 tx=0x7fe07000eeb0 comp rx=0 tx=0).stop 2024-09-10T09:44:55.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:55.280+0000 7fe08bc00640 1 -- 172.21.15.5:0/3737893288 shutdown_connections 2024-09-10T09:44:55.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:55.280+0000 7fe08bc00640 1 --2- 172.21.15.5:0/3737893288 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fe06c0a2b80 0x7fe06c0a5040 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:55.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:55.280+0000 7fe08bc00640 1 --2- 172.21.15.5:0/3737893288 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe084109140 0x7fe08419ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:55.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:55.280+0000 7fe08bc00640 1 --2- 172.21.15.5:0/3737893288 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe0841049c0 0x7fe08419ac00 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:55.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:55.280+0000 7fe08bc00640 1 --2- 172.21.15.5:0/3737893288 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe084104010 0x7fe08419a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:55.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:55.280+0000 7fe08bc00640 1 -- 172.21.15.5:0/3737893288 >> 172.21.15.5:0/3737893288 conn(0x7fe0840ffa50 msgr2=0x7fe084100d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:44:55.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:55.280+0000 7fe08bc00640 1 -- 172.21.15.5:0/3737893288 shutdown_connections 2024-09-10T09:44:55.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:55.280+0000 7fe08bc00640 1 -- 172.21.15.5:0/3737893288 wait complete. 2024-09-10T09:44:55.351 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:44:54 smithi159 ceph-mon[35513]: pgmap v590: 129 pgs: 129 active+clean; 144 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 1.1 MiB/s wr, 51 op/s 2024-09-10T09:44:56.053 INFO:tasks.workunit.client.0.smithi005.stdout: 1 353740 5.53 MB/sec execute 157 sec latency 18915.668 ms 2024-09-10T09:44:56.367 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '14ee912', total seconds elapsed: 24.4 2024-09-10T09:44:56.367 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 20.1 2024-09-10T09:44:56.367 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T09:44:56.757 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:44:57.053 INFO:tasks.workunit.client.0.smithi005.stdout: 1 354430 5.54 MB/sec execute 158 sec latency 667.647 ms 2024-09-10T09:44:57.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:44:56 smithi005 ceph-mon[30350]: pgmap v591: 129 pgs: 129 active+clean; 144 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 1.3 MiB/s wr, 58 op/s 2024-09-10T09:44:57.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:44:56 smithi102 ceph-mon[35442]: pgmap v591: 129 pgs: 129 active+clean; 144 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 1.3 MiB/s wr, 58 op/s 2024-09-10T09:44:57.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:57.309+0000 7f3085e00640 1 -- 172.21.15.5:0/3509595863 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3080103ff0 msgr2=0x7f30801043d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:44:57.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:57.309+0000 7f3085e00640 1 --2- 172.21.15.5:0/3509595863 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3080103ff0 0x7f30801043d0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f3070009900 tx=0x7f307002a410 comp rx=0 tx=0).stop 2024-09-10T09:44:57.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:57.309+0000 7f3085e00640 1 -- 172.21.15.5:0/3509595863 shutdown_connections 2024-09-10T09:44:57.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:57.309+0000 7f3085e00640 1 --2- 172.21.15.5:0/3509595863 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3080109120 0x7f3080110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:57.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:57.309+0000 7f3085e00640 1 --2- 172.21.15.5:0/3509595863 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f30801049a0 0x7f30801089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:57.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:57.309+0000 7f3085e00640 1 --2- 172.21.15.5:0/3509595863 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3080103ff0 0x7f30801043d0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:57.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:57.309+0000 7f3085e00640 1 -- 172.21.15.5:0/3509595863 >> 172.21.15.5:0/3509595863 conn(0x7f30800ffa90 msgr2=0x7f3080101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:44:57.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:57.309+0000 7f3085e00640 1 -- 172.21.15.5:0/3509595863 shutdown_connections 2024-09-10T09:44:57.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:57.309+0000 7f3085e00640 1 -- 172.21.15.5:0/3509595863 wait complete. 2024-09-10T09:44:57.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:57.310+0000 7f3085e00640 1 Processor -- start 2024-09-10T09:44:57.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:57.311+0000 7f3085e00640 1 -- start start 2024-09-10T09:44:57.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:57.311+0000 7f3085e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3080103ff0 0x7f308019a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:44:57.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:57.311+0000 7f3085e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f30801049a0 0x7f308019abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:44:57.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:57.311+0000 7f3085e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3080109120 0x7f308019ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:44:57.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:57.311+0000 7f3085e00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f3080114170 con 0x7f30801049a0 2024-09-10T09:44:57.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:57.311+0000 7f3085e00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f3080113ff0 con 0x7f3080109120 2024-09-10T09:44:57.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:57.311+0000 7f3085e00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f30801142f0 con 0x7f3080103ff0 2024-09-10T09:44:57.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:57.311+0000 7f307ea00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3080103ff0 0x7f308019a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:44:57.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:57.311+0000 7f307ea00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3080103ff0 0x7f308019a670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:47548/0 (socket says 172.21.15.5:47548) 2024-09-10T09:44:57.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:57.311+0000 7f307ea00640 1 -- 172.21.15.5:0/2797824939 learned_addr learned my addr 172.21.15.5:0/2797824939 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:44:57.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:57.312+0000 7f307e000640 1 --2- 172.21.15.5:0/2797824939 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f30801049a0 0x7f308019abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:44:57.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:57.312+0000 7f307f400640 1 --2- 172.21.15.5:0/2797824939 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3080109120 0x7f308019ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:44:57.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:57.312+0000 7f307ea00640 1 -- 172.21.15.5:0/2797824939 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3080109120 msgr2=0x7f308019ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:44:57.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:57.312+0000 7f307ea00640 1 --2- 172.21.15.5:0/2797824939 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3080109120 0x7f308019ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:57.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:57.312+0000 7f307ea00640 1 -- 172.21.15.5:0/2797824939 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f30801049a0 msgr2=0x7f308019abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:44:57.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:57.312+0000 7f307ea00640 1 --2- 172.21.15.5:0/2797824939 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f30801049a0 0x7f308019abb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:57.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:57.312+0000 7f307ea00640 1 -- 172.21.15.5:0/2797824939 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f308019f620 con 0x7f3080103ff0 2024-09-10T09:44:57.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:57.312+0000 7f307e000640 1 --2- 172.21.15.5:0/2797824939 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f30801049a0 0x7f308019abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:44:57.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:57.312+0000 7f307ea00640 1 --2- 172.21.15.5:0/2797824939 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3080103ff0 0x7f308019a670 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f3070004770 tx=0x7f3070003c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:44:57.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:57.312+0000 7f3084a00640 1 -- 172.21.15.5:0/2797824939 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f307003f070 con 0x7f3080103ff0 2024-09-10T09:44:57.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:57.313+0000 7f3084a00640 1 -- 172.21.15.5:0/2797824939 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3070031ca0 con 0x7f3080103ff0 2024-09-10T09:44:57.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:57.313+0000 7f3085e00640 1 -- 172.21.15.5:0/2797824939 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f308019f8b0 con 0x7f3080103ff0 2024-09-10T09:44:57.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:57.313+0000 7f3085e00640 1 -- 172.21.15.5:0/2797824939 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30801a71f0 con 0x7f3080103ff0 2024-09-10T09:44:57.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:57.313+0000 7f3084a00640 1 -- 172.21.15.5:0/2797824939 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3070035030 con 0x7f3080103ff0 2024-09-10T09:44:57.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:57.314+0000 7f3085e00640 1 -- 172.21.15.5:0/2797824939 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f30801043d0 con 0x7f3080103ff0 2024-09-10T09:44:57.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:57.316+0000 7f3084a00640 1 -- 172.21.15.5:0/2797824939 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f3070042070 con 0x7f3080103ff0 2024-09-10T09:44:57.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:57.317+0000 7f3084a00640 1 --2- 172.21.15.5:0/2797824939 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f30640a2c00 0x7f30640a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:44:57.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:57.317+0000 7f3084a00640 1 -- 172.21.15.5:0/2797824939 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f30700efb00 con 0x7f3080103ff0 2024-09-10T09:44:57.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:57.317+0000 7f307e000640 1 --2- 172.21.15.5:0/2797824939 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f30640a2c00 0x7f30640a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:44:57.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:57.318+0000 7f307e000640 1 --2- 172.21.15.5:0/2797824939 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f30640a2c00 0x7f30640a50c0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f30800fff40 tx=0x7f306c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:44:57.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:57.321+0000 7f3084a00640 1 -- 172.21.15.5:0/2797824939 <== mon.1 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f30700bc6d0 con 0x7f3080103ff0 2024-09-10T09:44:57.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:44:56 smithi159 ceph-mon[35513]: pgmap v591: 129 pgs: 129 active+clean; 144 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 1.3 MiB/s wr, 58 op/s 2024-09-10T09:44:57.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:57.582+0000 7f3085e00640 1 -- 172.21.15.5:0/2797824939 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f308010c300 con 0x7f3080103ff0 2024-09-10T09:44:57.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:57.583+0000 7f3084a00640 1 -- 172.21.15.5:0/2797824939 <== mon.1 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+11969 (secure 0 0 0) 0x7f30700bbfd0 con 0x7f3080103ff0 2024-09-10T09:44:57.584 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-10T09:44:57.584 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":34,"btime":"2024-09-10T09:42:57:737230+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14454,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/2970594311","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2970594311},{"type":"v1","addr":"172.21.15.5:6835","nonce":2970594311}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14514,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6839/3616502174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":3616502174},{"type":"v1","addr":"172.21.15.5:6839","nonce":3616502174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":34,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T09:35:29.626287+0000","modified":"2024-09-10T09:42:56.961831+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24542,"mds_1":24535,"mds_2":24511,"mds_3":24506,"mds_4":24443},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14520":{"gid":14520,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/2073599842","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":2073599842},{"type":"v1","addr":"172.21.15.5:6841","nonce":2073599842}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"k","rank":4,"incarnation":26,"state":"up:active","state_seq":24,"addr":"172.21.15.102:6833/2457007427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":2457007427},{"type":"v1","addr":"172.21.15.102:6833","nonce":2457007427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/356445159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":356445159},{"type":"v1","addr":"172.21.15.159:6833","nonce":356445159}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6837/2894314507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":2894314507},{"type":"v1","addr":"172.21.15.5:6837","nonce":2894314507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.102:6835/1647549111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":1647549111},{"type":"v1","addr":"172.21.15.102:6835","nonce":1647549111}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6835/952657242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":952657242},{"type":"v1","addr":"172.21.15.159:6835","nonce":952657242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":19,"addr":"172.21.15.102:6837/2689776764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":2689776764},{"type":"v1","addr":"172.21.15.102:6837","nonce":2689776764}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.159:6837/620627639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":620627639},{"type":"v1","addr":"172.21.15.159:6837","nonce":620627639}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24535":{"gid":24535,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":18,"addr":"172.21.15.159:6839/2073739599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":2073739599},{"type":"v1","addr":"172.21.15.159:6839","nonce":2073739599}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24542":{"gid":24542,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.102:6839/335409749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6838","nonce":335409749},{"type":"v1","addr":"172.21.15.102:6839","nonce":335409749}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24542,"qdb_cluster":[24542,24535,24511,24506,24443]},"id":1}]} 2024-09-10T09:44:57.584 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 34 2024-09-10T09:44:57.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:57.585+0000 7f3085e00640 1 -- 172.21.15.5:0/2797824939 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f30640a2c00 msgr2=0x7f30640a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:44:57.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:57.585+0000 7f3085e00640 1 --2- 172.21.15.5:0/2797824939 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f30640a2c00 0x7f30640a50c0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f30800fff40 tx=0x7f306c005f90 comp rx=0 tx=0).stop 2024-09-10T09:44:57.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:57.585+0000 7f3085e00640 1 -- 172.21.15.5:0/2797824939 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3080103ff0 msgr2=0x7f308019a670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:44:57.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:57.585+0000 7f3085e00640 1 --2- 172.21.15.5:0/2797824939 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3080103ff0 0x7f308019a670 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f3070004770 tx=0x7f3070003c10 comp rx=0 tx=0).stop 2024-09-10T09:44:57.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:57.585+0000 7f3085e00640 1 -- 172.21.15.5:0/2797824939 shutdown_connections 2024-09-10T09:44:57.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:57.585+0000 7f3085e00640 1 --2- 172.21.15.5:0/2797824939 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f30640a2c00 0x7f30640a50c0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:57.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:57.585+0000 7f3085e00640 1 --2- 172.21.15.5:0/2797824939 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3080109120 0x7f308019ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:57.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:57.585+0000 7f3085e00640 1 --2- 172.21.15.5:0/2797824939 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f30801049a0 0x7f308019abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:57.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:57.585+0000 7f3085e00640 1 --2- 172.21.15.5:0/2797824939 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3080103ff0 0x7f308019a670 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:57.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:57.585+0000 7f3085e00640 1 -- 172.21.15.5:0/2797824939 >> 172.21.15.5:0/2797824939 conn(0x7f30800ffa90 msgr2=0x7f3080100d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:44:57.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:57.586+0000 7f3085e00640 1 -- 172.21.15.5:0/2797824939 shutdown_connections 2024-09-10T09:44:57.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:57.586+0000 7f3085e00640 1 -- 172.21.15.5:0/2797824939 wait complete. 2024-09-10T09:44:58.054 INFO:tasks.workunit.client.0.smithi005.stdout: 1 356135 5.56 MB/sec execute 159 sec latency 18.529 ms 2024-09-10T09:44:58.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:44:57 smithi102 ceph-mon[35442]: from='client.? 172.21.15.5:0/2797824939' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:44:58.306 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:44:57 smithi005 ceph-mon[30350]: from='client.? 172.21.15.5:0/2797824939' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:44:58.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:44:57 smithi159 ceph-mon[35513]: from='client.? 172.21.15.5:0/2797824939' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:44:58.583 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24542 quiesce db / --timeout 90 --expiration 140.1 --await-for 45.0' 2024-09-10T09:44:58.583 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24542 quiesce db / --timeout 90 --expiration 140.1 --await-for 45.0 2024-09-10T09:44:58.972 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:44:59.054 INFO:tasks.workunit.client.0.smithi005.stdout: 1 358723 5.62 MB/sec execute 160 sec latency 12.027 ms 2024-09-10T09:44:59.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:44:58 smithi102 ceph-mon[35442]: pgmap v592: 129 pgs: 129 active+clean; 144 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 1.2 MiB/s wr, 52 op/s 2024-09-10T09:44:59.241 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:44:58 smithi005 ceph-mon[30350]: pgmap v592: 129 pgs: 129 active+clean; 144 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 1.2 MiB/s wr, 52 op/s 2024-09-10T09:44:59.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:44:58 smithi159 ceph-mon[35513]: pgmap v592: 129 pgs: 129 active+clean; 144 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 1.2 MiB/s wr, 52 op/s 2024-09-10T09:44:59.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:59.522+0000 7fef73e00640 1 -- 172.21.15.5:0/1388301116 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fef6c102da0 msgr2=0x7fef6c103180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:44:59.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:59.522+0000 7fef73e00640 1 --2- 172.21.15.5:0/1388301116 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fef6c102da0 0x7fef6c103180 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fef60009900 tx=0x7fef6002a480 comp rx=0 tx=0).stop 2024-09-10T09:44:59.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:59.522+0000 7fef73e00640 1 -- 172.21.15.5:0/1388301116 shutdown_connections 2024-09-10T09:44:59.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:59.522+0000 7fef73e00640 1 --2- 172.21.15.5:0/1388301116 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fef6c10c180 0x7fef6c10e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:59.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:59.522+0000 7fef73e00640 1 --2- 172.21.15.5:0/1388301116 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fef6c1036c0 0x7fef6c10bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:59.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:59.522+0000 7fef73e00640 1 --2- 172.21.15.5:0/1388301116 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fef6c102da0 0x7fef6c103180 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:59.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:59.522+0000 7fef73e00640 1 -- 172.21.15.5:0/1388301116 >> 172.21.15.5:0/1388301116 conn(0x7fef6c0fc7a0 msgr2=0x7fef6c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:44:59.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:59.522+0000 7fef73e00640 1 -- 172.21.15.5:0/1388301116 shutdown_connections 2024-09-10T09:44:59.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:59.522+0000 7fef73e00640 1 -- 172.21.15.5:0/1388301116 wait complete. 2024-09-10T09:44:59.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:59.523+0000 7fef73e00640 1 Processor -- start 2024-09-10T09:44:59.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:59.524+0000 7fef73e00640 1 -- start start 2024-09-10T09:44:59.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:59.524+0000 7fef73e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fef6c102da0 0x7fef6c1984b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:44:59.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:59.524+0000 7fef73e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fef6c1036c0 0x7fef6c1989f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:44:59.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:59.524+0000 7fef73e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fef6c10c180 0x7fef6c19cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:44:59.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:59.524+0000 7fef73e00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fef6c111f60 con 0x7fef6c102da0 2024-09-10T09:44:59.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:59.524+0000 7fef73e00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fef6c111de0 con 0x7fef6c1036c0 2024-09-10T09:44:59.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:59.524+0000 7fef73e00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fef6c1120e0 con 0x7fef6c10c180 2024-09-10T09:44:59.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:59.524+0000 7fef70a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fef6c1036c0 0x7fef6c1989f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:44:59.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:59.524+0000 7fef71400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fef6c102da0 0x7fef6c1984b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:44:59.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:59.524+0000 7fef70a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fef6c1036c0 0x7fef6c1989f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:59322/0 (socket says 172.21.15.5:59322) 2024-09-10T09:44:59.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:59.524+0000 7fef70a00640 1 -- 172.21.15.5:0/1128013611 learned_addr learned my addr 172.21.15.5:0/1128013611 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:44:59.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:59.524+0000 7fef71e00640 1 --2- 172.21.15.5:0/1128013611 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fef6c10c180 0x7fef6c19cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:44:59.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:59.525+0000 7fef70a00640 1 -- 172.21.15.5:0/1128013611 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fef6c10c180 msgr2=0x7fef6c19cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:44:59.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:59.525+0000 7fef70a00640 1 --2- 172.21.15.5:0/1128013611 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fef6c10c180 0x7fef6c19cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:59.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:59.525+0000 7fef70a00640 1 -- 172.21.15.5:0/1128013611 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fef6c102da0 msgr2=0x7fef6c1984b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:44:59.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:59.525+0000 7fef70a00640 1 --2- 172.21.15.5:0/1128013611 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fef6c102da0 0x7fef6c1984b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:59.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:59.525+0000 7fef70a00640 1 -- 172.21.15.5:0/1128013611 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef6c19d460 con 0x7fef6c1036c0 2024-09-10T09:44:59.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:59.525+0000 7fef70a00640 1 --2- 172.21.15.5:0/1128013611 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fef6c1036c0 0x7fef6c1989f0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fef5800e9f0 tx=0x7fef5800edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:44:59.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:59.525+0000 7fef6a000640 1 -- 172.21.15.5:0/1128013611 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef5800cd80 con 0x7fef6c1036c0 2024-09-10T09:44:59.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:59.525+0000 7fef73e00640 1 -- 172.21.15.5:0/1128013611 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef6c19d750 con 0x7fef6c1036c0 2024-09-10T09:44:59.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:59.525+0000 7fef6a000640 1 -- 172.21.15.5:0/1128013611 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fef58005500 con 0x7fef6c1036c0 2024-09-10T09:44:59.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:59.525+0000 7fef6a000640 1 -- 172.21.15.5:0/1128013611 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef58010690 con 0x7fef6c1036c0 2024-09-10T09:44:59.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:59.526+0000 7fef73e00640 1 -- 172.21.15.5:0/1128013611 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef6c1a5030 con 0x7fef6c1036c0 2024-09-10T09:44:59.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:59.529+0000 7fef6a000640 1 -- 172.21.15.5:0/1128013611 <== mon.2 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7fef580046c0 con 0x7fef6c1036c0 2024-09-10T09:44:59.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:59.529+0000 7fef76865740 1 Processor -- start 2024-09-10T09:44:59.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:59.529+0000 7fef76865740 1 -- start start 2024-09-10T09:44:59.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:59.530+0000 7fef76865740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559a8726a3b0 0x559a8726a790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:44:59.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:59.530+0000 7fef76865740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559a8726acd0 0x559a87275050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:44:59.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:59.530+0000 7fef76865740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559a87275590 0x559a87277980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:44:59.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:59.530+0000 7fef76865740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x559a871d0530 con 0x559a8726a3b0 2024-09-10T09:44:59.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:59.530+0000 7fef76865740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x559a87177860 con 0x559a87275590 2024-09-10T09:44:59.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:59.530+0000 7fef76865740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x559a871cc4f0 con 0x559a8726acd0 2024-09-10T09:44:59.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:59.530+0000 7fef71400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559a8726acd0 0x559a87275050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:44:59.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:59.530+0000 7fef71e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559a8726a3b0 0x559a8726a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:44:59.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:59.530+0000 7fef71400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559a8726acd0 0x559a87275050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:47576/0 (socket says 172.21.15.5:47576) 2024-09-10T09:44:59.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:59.530+0000 7fef71400640 1 -- 172.21.15.5:0/3341458703 learned_addr learned my addr 172.21.15.5:0/3341458703 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:44:59.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:59.530+0000 7fef71e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559a8726a3b0 0x559a8726a790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39130/0 (socket says 172.21.15.5:39130) 2024-09-10T09:44:59.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:59.530+0000 7fef70a00640 1 --2- 172.21.15.5:0/3341458703 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559a87275590 0x559a87277980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:44:59.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:59.530+0000 7fef71400640 1 -- 172.21.15.5:0/3341458703 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559a87275590 msgr2=0x559a87277980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:44:59.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:59.530+0000 7fef71400640 1 --2- 172.21.15.5:0/3341458703 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559a87275590 0x559a87277980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:59.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:59.530+0000 7fef71400640 1 -- 172.21.15.5:0/3341458703 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559a8726a3b0 msgr2=0x559a8726a790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:44:59.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:59.530+0000 7fef71400640 1 --2- 172.21.15.5:0/3341458703 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559a8726a3b0 0x559a8726a790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:59.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:59.530+0000 7fef71400640 1 -- 172.21.15.5:0/3341458703 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559a86f41b90 con 0x559a8726acd0 2024-09-10T09:44:59.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:59.531+0000 7fef6a000640 1 --2- 172.21.15.5:0/1128013611 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fef540a2c00 0x7fef540a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:44:59.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:59.531+0000 7fef71400640 1 --2- 172.21.15.5:0/3341458703 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559a8726acd0 0x559a87275050 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fef60009900 tx=0x7fef60005be0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8a8a1fc33a305104 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:44:59.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:59.531+0000 7fef6a000640 1 -- 172.21.15.5:0/1128013611 <== mon.2 v2:172.21.15.102:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fef580cf810 con 0x7fef6c1036c0 2024-09-10T09:44:59.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:59.531+0000 7fef71e00640 1 --2- 172.21.15.5:0/1128013611 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fef540a2c00 0x7fef540a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:44:59.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:59.531+0000 7fef5e600640 1 -- 172.21.15.5:0/3341458703 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef60040070 con 0x559a8726acd0 2024-09-10T09:44:59.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:59.531+0000 7fef5e600640 1 -- 172.21.15.5:0/3341458703 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fef60031870 con 0x559a8726acd0 2024-09-10T09:44:59.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:59.531+0000 7fef5e600640 1 -- 172.21.15.5:0/3341458703 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef60035040 con 0x559a8726acd0 2024-09-10T09:44:59.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:59.532+0000 7fef76865740 1 -- 172.21.15.5:0/3341458703 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559a8726acd0 msgr2=0x559a87275050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:44:59.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:59.532+0000 7fef71e00640 1 --2- 172.21.15.5:0/1128013611 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fef540a2c00 0x7fef540a50c0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fef64005fd0 tx=0x7fef64009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:44:59.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:59.532+0000 7fef76865740 1 --2- 172.21.15.5:0/3341458703 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559a8726acd0 0x559a87275050 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fef60009900 tx=0x7fef60005be0 comp rx=0 tx=0).stop 2024-09-10T09:44:59.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:59.532+0000 7fef76865740 1 -- 172.21.15.5:0/3341458703 shutdown_connections 2024-09-10T09:44:59.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:59.532+0000 7fef76865740 1 --2- 172.21.15.5:0/3341458703 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559a87275590 0x559a87277980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:59.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:59.532+0000 7fef76865740 1 --2- 172.21.15.5:0/3341458703 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559a8726acd0 0x559a87275050 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:59.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:59.532+0000 7fef76865740 1 --2- 172.21.15.5:0/3341458703 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559a8726a3b0 0x559a8726a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:59.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:59.532+0000 7fef76865740 1 -- 172.21.15.5:0/3341458703 >> 172.21.15.5:0/3341458703 conn(0x559a87266940 msgr2=0x559a87266d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:44:59.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:59.532+0000 7fef76865740 1 -- 172.21.15.5:0/3341458703 shutdown_connections 2024-09-10T09:44:59.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:59.533+0000 7fef76865740 1 -- 172.21.15.5:0/3341458703 wait complete. 2024-09-10T09:44:59.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:59.533+0000 7fef76865740 1 -- start start 2024-09-10T09:44:59.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:59.533+0000 7fef76865740 1 Processor -- start 2024-09-10T09:44:59.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:59.534+0000 7fef76865740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559a8726a3b0 0x559a87272700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:44:59.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:59.534+0000 7fef76865740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559a8726acd0 0x559a87272c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:44:59.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:59.534+0000 7fef76865740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559a87275590 0x559a87273180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:44:59.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:59.534+0000 7fef76865740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x559a87277fb0 con 0x559a8726acd0 2024-09-10T09:44:59.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:59.534+0000 7fef76865740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x559a8718ae60 con 0x559a8726a3b0 2024-09-10T09:44:59.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:59.534+0000 7fef76865740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x559a87278130 con 0x559a87275590 2024-09-10T09:44:59.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:59.534+0000 7fef76865740 10 client.0 _ll_register_callbacks cb 0x559a8718fcc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:44:59.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:59.534+0000 7fef71400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559a8726a3b0 0x559a87272700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:44:59.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:59.534+0000 7fef71400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559a8726a3b0 0x559a87272700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:59344/0 (socket says 172.21.15.5:59344) 2024-09-10T09:44:59.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:59.534+0000 7fef71e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559a87275590 0x559a87273180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:44:59.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:59.534+0000 7fef71400640 1 -- 172.21.15.5:0/1700866508 learned_addr learned my addr 172.21.15.5:0/1700866508 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:44:59.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:59.534+0000 7fef70a00640 1 --2- 172.21.15.5:0/1700866508 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559a8726acd0 0x559a87272c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:44:59.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:59.534+0000 7fef71400640 1 -- 172.21.15.5:0/1700866508 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559a87275590 msgr2=0x559a87273180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:44:59.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:59.534+0000 7fef71400640 1 --2- 172.21.15.5:0/1700866508 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559a87275590 0x559a87273180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:59.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:59.534+0000 7fef71400640 1 -- 172.21.15.5:0/1700866508 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559a8726acd0 msgr2=0x559a87272c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:44:59.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:59.534+0000 7fef71400640 1 --2- 172.21.15.5:0/1700866508 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559a8726acd0 0x559a87272c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:59.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:59.534+0000 7fef71400640 1 -- 172.21.15.5:0/1700866508 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559a87196990 con 0x559a8726a3b0 2024-09-10T09:44:59.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:59.534+0000 7fef70a00640 1 --2- 172.21.15.5:0/1700866508 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559a8726acd0 0x559a87272c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:44:59.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:59.535+0000 7fef71400640 1 --2- 172.21.15.5:0/1700866508 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559a8726a3b0 0x559a87272700 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fef60000c00 tx=0x7fef600052f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=edf47e45654dd6cf server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:44:59.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:59.535+0000 7fef52a00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-09-10T09:44:59.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:59.535+0000 7fef52a00640 1 -- 172.21.15.5:0/1700866508 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef60040070 con 0x559a8726a3b0 2024-09-10T09:44:59.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:59.535+0000 7fef76865740 1 -- 172.21.15.5:0/1700866508 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fef60031870 con 0x559a8726a3b0 2024-09-10T09:44:59.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:59.535+0000 7fef52a00640 1 -- 172.21.15.5:0/1700866508 <== mon.2 v2:172.21.15.102:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fef60004030 con 0x559a8726a3b0 2024-09-10T09:44:59.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:59.535+0000 7fef52a00640 1 -- 172.21.15.5:0/1700866508 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fef60056050 con 0x559a8726a3b0 2024-09-10T09:44:59.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:59.535+0000 7fef76865740 10 client.26675 fetch_fsmap learned FSMap version 34 2024-09-10T09:44:59.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:59.535+0000 7fef76865740 1 -- 172.21.15.5:0/1700866508 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x559a87283dd0 con 0x559a8726a3b0 2024-09-10T09:44:59.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:59.536+0000 7fef52a00640 1 -- 172.21.15.5:0/1700866508 <== mon.2 v2:172.21.15.102:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fef60031870 con 0x559a8726a3b0 2024-09-10T09:44:59.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:59.536+0000 7fef52a00640 1 -- 172.21.15.5:0/1700866508 <== mon.2 v2:172.21.15.102:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7fef600035f0 con 0x559a8726a3b0 2024-09-10T09:44:59.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:59.536+0000 7fef76865740 10 client.26675 fetch_fsmap finished waiting for FSMap version 34 2024-09-10T09:44:59.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:59.536+0000 7fef76865740 10 client.26675 resolve_mds: validated gid 24542 aka daemon mds.b 2024-09-10T09:44:59.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:59.536+0000 7fef76865740 20 client.26675 populate_metadata read hostname 'smithi005' 2024-09-10T09:44:59.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:59.536+0000 7fef76865740 1 --2- 172.21.15.5:0/1700866508 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x559a87285120 0x559a87287510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:44:59.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:59.536+0000 7fef76865740 4 client.26675 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:44:59.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:59.536+0000 7fef76865740 1 -- 172.21.15.5:0/1700866508 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559a87173310 con 0x559a87285120 2024-09-10T09:44:59.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:59.536+0000 7fef70a00640 1 --2- 172.21.15.5:0/1700866508 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x559a87285120 0x559a87287510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:44:59.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:59.537+0000 7fef70a00640 1 --2- 172.21.15.5:0/1700866508 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x559a87285120 0x559a87287510 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=78967e4c5e236b8d server_cookie=e5b7dc8f7d06edc0 in_seq=0 out_seq=0 2024-09-10T09:44:59.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:59.537+0000 7fef52a00640 10 client.26675 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T09:44:59.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:59.537+0000 7fef52a00640 1 -- 172.21.15.5:0/1700866508 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559a87173310 con 0x559a87285120 2024-09-10T09:44:59.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:59.537+0000 7fef52a00640 10 client.26675 handle_command_reply: tid=0 2024-09-10T09:44:59.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:59.537+0000 7fef76865740 1 client.26675 shutdown 2024-09-10T09:44:59.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:59.538+0000 7fef76865740 1 -- 172.21.15.5:0/1700866508 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559a8726a3b0 msgr2=0x559a87272700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:44:59.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:59.538+0000 7fef76865740 1 --2- 172.21.15.5:0/1700866508 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559a8726a3b0 0x559a87272700 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fef60000c00 tx=0x7fef600052f0 comp rx=0 tx=0).stop 2024-09-10T09:44:59.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:59.538+0000 7fef76865740 1 -- 172.21.15.5:0/1700866508 shutdown_connections 2024-09-10T09:44:59.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:59.538+0000 7fef76865740 1 --2- 172.21.15.5:0/1700866508 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x559a87285120 0x559a87287510 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:59.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:59.538+0000 7fef76865740 1 --2- 172.21.15.5:0/1700866508 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559a87275590 0x559a87273180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:59.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:59.538+0000 7fef76865740 1 --2- 172.21.15.5:0/1700866508 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559a8726acd0 0x559a87272c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:59.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:59.538+0000 7fef76865740 1 --2- 172.21.15.5:0/1700866508 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559a8726a3b0 0x559a87272700 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:59.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:59.538+0000 7fef52a00640 0 client.26675 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T09:44:59.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:59.538+0000 7fef76865740 1 -- 172.21.15.5:0/1700866508 >> 172.21.15.5:0/1700866508 conn(0x559a87266940 msgr2=0x559a87265c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:44:59.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:59.538+0000 7fef76865740 1 -- 172.21.15.5:0/1700866508 shutdown_connections 2024-09-10T09:44:59.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:59.538+0000 7fef76865740 1 -- 172.21.15.5:0/1700866508 wait complete. 2024-09-10T09:44:59.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:59.538+0000 7fef76865740 20 client.26675 trim_cache size 0 max 16384 2024-09-10T09:44:59.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:59.557+0000 7fef76865740 1 Processor -- start 2024-09-10T09:44:59.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:59.558+0000 7fef76865740 1 -- start start 2024-09-10T09:44:59.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:59.558+0000 7fef76865740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559a8726a3b0 0x559a87265a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:44:59.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:59.558+0000 7fef76865740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559a8726acd0 0x559a8728cab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:44:59.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:59.558+0000 7fef76865740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559a87275590 0x559a8728cff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:44:59.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:59.558+0000 7fef76865740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fef6c1aee00 con 0x559a87275590 2024-09-10T09:44:59.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:59.558+0000 7fef76865740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fef6c1b0940 con 0x559a8726acd0 2024-09-10T09:44:59.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:59.558+0000 7fef76865740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fef6c1af4d0 con 0x559a8726a3b0 2024-09-10T09:44:59.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:59.558+0000 7fef71400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559a8726a3b0 0x559a87265a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:44:59.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:59.558+0000 7fef70a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559a8726acd0 0x559a8728cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:44:59.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:59.558+0000 7fef71400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559a8726a3b0 0x559a87265a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:47598/0 (socket says 172.21.15.5:47598) 2024-09-10T09:44:59.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:59.558+0000 7fef71400640 1 -- 172.21.15.5:0/3172777814 learned_addr learned my addr 172.21.15.5:0/3172777814 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:44:59.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:59.558+0000 7fef71e00640 1 --2- 172.21.15.5:0/3172777814 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559a87275590 0x559a8728cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:44:59.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:59.559+0000 7fef71400640 1 -- 172.21.15.5:0/3172777814 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559a8726acd0 msgr2=0x559a8728cab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:44:59.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:59.559+0000 7fef71400640 1 --2- 172.21.15.5:0/3172777814 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559a8726acd0 0x559a8728cab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:59.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:59.559+0000 7fef71400640 1 -- 172.21.15.5:0/3172777814 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559a87275590 msgr2=0x559a8728cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:44:59.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:59.559+0000 7fef71400640 1 --2- 172.21.15.5:0/3172777814 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559a87275590 0x559a8728cff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:59.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:59.559+0000 7fef71400640 1 -- 172.21.15.5:0/3172777814 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559a872780b0 con 0x559a8726a3b0 2024-09-10T09:44:59.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:59.559+0000 7fef71e00640 1 --2- 172.21.15.5:0/3172777814 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559a87275590 0x559a8728cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:44:59.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:59.559+0000 7fef71400640 1 --2- 172.21.15.5:0/3172777814 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559a8726a3b0 0x559a87265a90 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fef60005490 tx=0x7fef60005be0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fda7003b47685a05 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:44:59.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:59.559+0000 7fef5dc00640 1 -- 172.21.15.5:0/3172777814 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef6005f070 con 0x559a8726a3b0 2024-09-10T09:44:59.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:59.559+0000 7fef5dc00640 1 -- 172.21.15.5:0/3172777814 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fef6004d070 con 0x559a8726a3b0 2024-09-10T09:44:59.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:59.559+0000 7fef5dc00640 1 -- 172.21.15.5:0/3172777814 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef60040040 con 0x559a8726a3b0 2024-09-10T09:44:59.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:59.560+0000 7fef76865740 1 -- 172.21.15.5:0/3172777814 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559a8726a3b0 msgr2=0x559a87265a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:44:59.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:59.560+0000 7fef76865740 1 --2- 172.21.15.5:0/3172777814 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559a8726a3b0 0x559a87265a90 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fef60005490 tx=0x7fef60005be0 comp rx=0 tx=0).stop 2024-09-10T09:44:59.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:59.560+0000 7fef76865740 1 -- 172.21.15.5:0/3172777814 shutdown_connections 2024-09-10T09:44:59.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:59.560+0000 7fef76865740 1 --2- 172.21.15.5:0/3172777814 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559a87275590 0x559a8728cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:59.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:59.560+0000 7fef76865740 1 --2- 172.21.15.5:0/3172777814 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559a8726acd0 0x559a8728cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:59.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:59.560+0000 7fef76865740 1 --2- 172.21.15.5:0/3172777814 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559a8726a3b0 0x559a87265a90 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:59.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:59.560+0000 7fef76865740 1 -- 172.21.15.5:0/3172777814 >> 172.21.15.5:0/3172777814 conn(0x559a87266940 msgr2=0x559a87274670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:44:59.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:59.560+0000 7fef76865740 1 -- 172.21.15.5:0/3172777814 shutdown_connections 2024-09-10T09:44:59.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:59.560+0000 7fef76865740 1 -- 172.21.15.5:0/3172777814 wait complete. 2024-09-10T09:44:59.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:59.561+0000 7fef76865740 1 -- start start 2024-09-10T09:44:59.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:59.561+0000 7fef76865740 1 Processor -- start 2024-09-10T09:44:59.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:59.561+0000 7fef76865740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559a8726a3b0 0x559a8727d320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:44:59.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:59.561+0000 7fef76865740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559a8726acd0 0x559a8727d860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:44:59.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:59.561+0000 7fef76865740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559a87275590 0x559a8727a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:44:59.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:59.561+0000 7fef76865740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fef6c1b0270 con 0x559a8726a3b0 2024-09-10T09:44:59.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:59.561+0000 7fef76865740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fef6c1afba0 con 0x559a8726acd0 2024-09-10T09:44:59.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:59.561+0000 7fef76865740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fef6c1acde0 con 0x559a87275590 2024-09-10T09:44:59.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:59.562+0000 7fef76865740 10 client.0 _ll_register_callbacks cb 0x7fef64009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:44:59.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:59.562+0000 7fef70a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559a8726acd0 0x559a8727d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:44:59.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:59.562+0000 7fef71400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559a8726a3b0 0x559a8727d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:44:59.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:59.562+0000 7fef71e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559a87275590 0x559a8727a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:44:59.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:59.562+0000 7fef70a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559a8726acd0 0x559a8727d860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:59354/0 (socket says 172.21.15.5:59354) 2024-09-10T09:44:59.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:59.562+0000 7fef70a00640 1 -- 172.21.15.5:0/779806036 learned_addr learned my addr 172.21.15.5:0/779806036 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:44:59.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:59.562+0000 7fef71e00640 1 -- 172.21.15.5:0/779806036 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559a8726acd0 msgr2=0x559a8727d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:44:59.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:59.562+0000 7fef71e00640 1 --2- 172.21.15.5:0/779806036 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559a8726acd0 0x559a8727d860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:59.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:59.562+0000 7fef71e00640 1 -- 172.21.15.5:0/779806036 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559a8726a3b0 msgr2=0x559a8727d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:44:59.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:59.562+0000 7fef71e00640 1 --2- 172.21.15.5:0/779806036 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559a8726a3b0 0x559a8727d320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:59.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:59.562+0000 7fef71e00640 1 -- 172.21.15.5:0/779806036 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559a8727a830 con 0x559a87275590 2024-09-10T09:44:59.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:59.563+0000 7fef71e00640 1 --2- 172.21.15.5:0/779806036 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559a87275590 0x559a8727a2f0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x559a87267670 tx=0x7fef64004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5f23a01a3b3b3f59 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:44:59.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:59.563+0000 7fef52a00640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-09-10T09:44:59.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:59.563+0000 7fef52a00640 1 -- 172.21.15.5:0/779806036 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef64004860 con 0x559a87275590 2024-09-10T09:44:59.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:59.563+0000 7fef52a00640 1 -- 172.21.15.5:0/779806036 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fef6400fbe0 con 0x559a87275590 2024-09-10T09:44:59.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:59.563+0000 7fef76865740 1 -- 172.21.15.5:0/779806036 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fef6004d070 con 0x559a87275590 2024-09-10T09:44:59.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:59.563+0000 7fef52a00640 1 -- 172.21.15.5:0/779806036 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fef64033050 con 0x559a87275590 2024-09-10T09:44:59.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:59.563+0000 7fef76865740 10 client.26710 fetch_fsmap learned FSMap version 34 2024-09-10T09:44:59.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:59.563+0000 7fef76865740 1 -- 172.21.15.5:0/779806036 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x559a872f4530 con 0x559a87275590 2024-09-10T09:44:59.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:59.564+0000 7fef52a00640 1 -- 172.21.15.5:0/779806036 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fef6004d070 con 0x559a87275590 2024-09-10T09:44:59.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:59.564+0000 7fef52a00640 1 -- 172.21.15.5:0/779806036 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7fef64005670 con 0x559a87275590 2024-09-10T09:44:59.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:59.564+0000 7fef76865740 10 client.26710 fetch_fsmap finished waiting for FSMap version 34 2024-09-10T09:44:59.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:59.564+0000 7fef76865740 10 client.26710 resolve_mds: validated gid 24542 aka daemon mds.b 2024-09-10T09:44:59.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:59.564+0000 7fef76865740 20 client.26710 populate_metadata read hostname 'smithi005' 2024-09-10T09:44:59.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:59.564+0000 7fef76865740 1 --2- 172.21.15.5:0/779806036 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x559a872f4d10 0x559a872f7100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:44:59.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:59.564+0000 7fef76865740 4 client.26710 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 140.1, "await_for": 45.0}] 2024-09-10T09:44:59.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:59.564+0000 7fef76865740 1 -- 172.21.15.5:0/779806036 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 140.1, "await_for": 45.0}) -- 0x559a871ddc60 con 0x559a872f4d10 2024-09-10T09:44:59.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:59.564+0000 7fef71400640 1 --2- 172.21.15.5:0/779806036 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x559a872f4d10 0x559a872f7100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:44:59.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:59.565+0000 7fef71400640 1 --2- 172.21.15.5:0/779806036 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x559a872f4d10 0x559a872f7100 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=35bfaa361d81900a server_cookie=dcf46d8bf2b9c19a in_seq=0 out_seq=0 2024-09-10T09:44:59.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:59.565+0000 7fef52a00640 10 client.26710 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T09:44:59.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:59.642+0000 7fef52a00640 1 -- 172.21.15.5:0/779806036 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x559a871ddc60 con 0x559a872f4d10 2024-09-10T09:44:59.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:59.642+0000 7fef52a00640 10 client.26710 handle_command_reply: tid=0 2024-09-10T09:44:59.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:59.642+0000 7fef76865740 1 client.26710 shutdown 2024-09-10T09:44:59.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:59.643+0000 7fef76865740 1 -- 172.21.15.5:0/779806036 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559a87275590 msgr2=0x559a8727a2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:44:59.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:59.643+0000 7fef76865740 1 --2- 172.21.15.5:0/779806036 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559a87275590 0x559a8727a2f0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x559a87267670 tx=0x7fef64004a40 comp rx=0 tx=0).stop 2024-09-10T09:44:59.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:59.643+0000 7fef76865740 1 -- 172.21.15.5:0/779806036 shutdown_connections 2024-09-10T09:44:59.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:59.643+0000 7fef76865740 1 --2- 172.21.15.5:0/779806036 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x559a872f4d10 0x559a872f7100 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:59.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:59.643+0000 7fef76865740 1 --2- 172.21.15.5:0/779806036 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559a87275590 0x559a8727a2f0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:59.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:59.643+0000 7fef76865740 1 --2- 172.21.15.5:0/779806036 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559a8726acd0 0x559a8727d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:59.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:59.643+0000 7fef76865740 1 --2- 172.21.15.5:0/779806036 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559a8726a3b0 0x559a8727d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:59.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:59.643+0000 7fef52a00640 0 client.26710 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T09:44:59.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:59.643+0000 7fef76865740 1 -- 172.21.15.5:0/779806036 >> 172.21.15.5:0/779806036 conn(0x559a87266940 msgr2=0x559a8726be00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:44:59.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:59.643+0000 7fef76865740 1 -- 172.21.15.5:0/779806036 shutdown_connections 2024-09-10T09:44:59.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:59.643+0000 7fef76865740 1 -- 172.21.15.5:0/779806036 wait complete. 2024-09-10T09:44:59.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:59.643+0000 7fef76865740 20 client.26710 trim_cache size 0 max 16384 2024-09-10T09:44:59.646 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-10T09:44:59.646 INFO:teuthology.orchestra.run.smithi005.stdout: "epoch": 34, 2024-09-10T09:44:59.646 INFO:teuthology.orchestra.run.smithi005.stdout: "leader": 24542, 2024-09-10T09:44:59.646 INFO:teuthology.orchestra.run.smithi005.stdout: "set_version": 22, 2024-09-10T09:44:59.646 INFO:teuthology.orchestra.run.smithi005.stdout: "sets": { 2024-09-10T09:44:59.646 INFO:teuthology.orchestra.run.smithi005.stdout: "36f005be": { 2024-09-10T09:44:59.646 INFO:teuthology.orchestra.run.smithi005.stdout: "version": 22, 2024-09-10T09:44:59.646 INFO:teuthology.orchestra.run.smithi005.stdout: "age_ref": 0.0, 2024-09-10T09:44:59.646 INFO:teuthology.orchestra.run.smithi005.stdout: "state": { 2024-09-10T09:44:59.646 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "QUIESCED", 2024-09-10T09:44:59.647 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0 2024-09-10T09:44:59.647 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T09:44:59.647 INFO:teuthology.orchestra.run.smithi005.stdout: "timeout": 90.0, 2024-09-10T09:44:59.647 INFO:teuthology.orchestra.run.smithi005.stdout: "expiration": 140.1, 2024-09-10T09:44:59.647 INFO:teuthology.orchestra.run.smithi005.stdout: "members": { 2024-09-10T09:44:59.647 INFO:teuthology.orchestra.run.smithi005.stdout: "file:/": { 2024-09-10T09:44:59.647 INFO:teuthology.orchestra.run.smithi005.stdout: "excluded": false, 2024-09-10T09:44:59.647 INFO:teuthology.orchestra.run.smithi005.stdout: "state": { 2024-09-10T09:44:59.647 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "QUIESCED", 2024-09-10T09:44:59.647 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0 2024-09-10T09:44:59.647 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T09:44:59.647 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T09:44:59.647 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T09:44:59.648 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T09:44:59.648 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T09:44:59.648 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-10T09:44:59.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:59.644+0000 7fef73e00640 1 -- 172.21.15.5:0/1128013611 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fef540a2c00 msgr2=0x7fef540a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:44:59.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:59.644+0000 7fef73e00640 1 --2- 172.21.15.5:0/1128013611 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fef540a2c00 0x7fef540a50c0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fef64005fd0 tx=0x7fef64009500 comp rx=0 tx=0).stop 2024-09-10T09:44:59.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:59.644+0000 7fef73e00640 1 -- 172.21.15.5:0/1128013611 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fef6c1036c0 msgr2=0x7fef6c1989f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:44:59.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:59.644+0000 7fef73e00640 1 --2- 172.21.15.5:0/1128013611 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fef6c1036c0 0x7fef6c1989f0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fef5800e9f0 tx=0x7fef5800edf0 comp rx=0 tx=0).stop 2024-09-10T09:44:59.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:59.644+0000 7fef73e00640 1 -- 172.21.15.5:0/1128013611 shutdown_connections 2024-09-10T09:44:59.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:59.644+0000 7fef73e00640 1 --2- 172.21.15.5:0/1128013611 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fef540a2c00 0x7fef540a50c0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:59.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:59.644+0000 7fef73e00640 1 --2- 172.21.15.5:0/1128013611 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fef6c10c180 0x7fef6c19cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:59.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:59.644+0000 7fef73e00640 1 --2- 172.21.15.5:0/1128013611 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fef6c1036c0 0x7fef6c1989f0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:59.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:59.644+0000 7fef73e00640 1 --2- 172.21.15.5:0/1128013611 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fef6c102da0 0x7fef6c1984b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:44:59.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:59.644+0000 7fef73e00640 1 -- 172.21.15.5:0/1128013611 >> 172.21.15.5:0/1128013611 conn(0x7fef6c0fc7a0 msgr2=0x7fef6c0fdfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:44:59.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:59.644+0000 7fef73e00640 1 -- 172.21.15.5:0/1128013611 shutdown_connections 2024-09-10T09:44:59.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:44:59.644+0000 7fef73e00640 1 -- 172.21.15.5:0/1128013611 wait complete. 2024-09-10T09:45:00.054 INFO:tasks.workunit.client.0.smithi005.stdout: 1 359745 5.61 MB/sec execute 161 sec latency 413.122 ms 2024-09-10T09:45:00.498 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '36f005be', quiesce took 4.1 seconds. Will release after: 16.0 2024-09-10T09:45:00.988 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:45:00 smithi005 ceph-mon[30350]: pgmap v593: 129 pgs: 129 active+clean; 146 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 720 KiB/s rd, 1.5 MiB/s wr, 95 op/s 2024-09-10T09:45:01.054 INFO:tasks.workunit.client.0.smithi005.stdout: 1 359745 5.58 MB/sec execute 162 sec latency 1413.184 ms 2024-09-10T09:45:01.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:45:00 smithi102 ceph-mon[35442]: pgmap v593: 129 pgs: 129 active+clean; 146 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 720 KiB/s rd, 1.5 MiB/s wr, 95 op/s 2024-09-10T09:45:01.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:45:00 smithi159 ceph-mon[35513]: pgmap v593: 129 pgs: 129 active+clean; 146 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 720 KiB/s rd, 1.5 MiB/s wr, 95 op/s 2024-09-10T09:45:01.432 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T09:45:01.830 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:45:02.054 INFO:tasks.workunit.client.0.smithi005.stdout: 1 359745 5.54 MB/sec execute 163 sec latency 2413.289 ms 2024-09-10T09:45:02.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:02.426+0000 7f945aa00640 1 -- 172.21.15.5:0/2078371365 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9454104980 msgr2=0x7f94541089d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:45:02.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:02.426+0000 7f945aa00640 1 --2- 172.21.15.5:0/2078371365 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9454104980 0x7f94541089d0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f9444009900 tx=0x7f944402a3f0 comp rx=0 tx=0).stop 2024-09-10T09:45:02.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:02.426+0000 7f945aa00640 1 -- 172.21.15.5:0/2078371365 shutdown_connections 2024-09-10T09:45:02.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:02.426+0000 7f945aa00640 1 --2- 172.21.15.5:0/2078371365 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9454109100 0x7f9454110c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:02.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:02.426+0000 7f945aa00640 1 --2- 172.21.15.5:0/2078371365 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9454104980 0x7f94541089d0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:02.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:02.426+0000 7f945aa00640 1 --2- 172.21.15.5:0/2078371365 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9454103fd0 0x7f94541043b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:02.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:02.426+0000 7f945aa00640 1 -- 172.21.15.5:0/2078371365 >> 172.21.15.5:0/2078371365 conn(0x7f94540ffa30 msgr2=0x7f9454101e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:45:02.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:02.426+0000 7f945aa00640 1 -- 172.21.15.5:0/2078371365 shutdown_connections 2024-09-10T09:45:02.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:02.427+0000 7f945aa00640 1 -- 172.21.15.5:0/2078371365 wait complete. 2024-09-10T09:45:02.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:02.428+0000 7f945aa00640 1 Processor -- start 2024-09-10T09:45:02.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:02.428+0000 7f945aa00640 1 -- start start 2024-09-10T09:45:02.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:02.428+0000 7f945aa00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9454103fd0 0x7f945410fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:45:02.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:02.428+0000 7f945aa00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9454104980 0x7f9454110120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:45:02.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:02.428+0000 7f945aa00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9454109100 0x7f94541a3370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:45:02.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:02.428+0000 7f945aa00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f9454077be0 con 0x7f9454104980 2024-09-10T09:45:02.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:02.428+0000 7f945aa00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f9454077a60 con 0x7f9454109100 2024-09-10T09:45:02.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:02.428+0000 7f945aa00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f9454077d60 con 0x7f9454103fd0 2024-09-10T09:45:02.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:02.429+0000 7f9453e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9454103fd0 0x7f945410fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:45:02.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:02.429+0000 7f9453e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9454103fd0 0x7f945410fbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:54196/0 (socket says 172.21.15.5:54196) 2024-09-10T09:45:02.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:02.429+0000 7f9453e00640 1 -- 172.21.15.5:0/3393233461 learned_addr learned my addr 172.21.15.5:0/3393233461 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:45:02.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:02.429+0000 7f9453400640 1 --2- 172.21.15.5:0/3393233461 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9454104980 0x7f9454110120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:45:02.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:02.429+0000 7f9458a00640 1 --2- 172.21.15.5:0/3393233461 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9454109100 0x7f94541a3370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:45:02.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:02.429+0000 7f9453e00640 1 -- 172.21.15.5:0/3393233461 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9454109100 msgr2=0x7f94541a3370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:45:02.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:02.429+0000 7f9453e00640 1 --2- 172.21.15.5:0/3393233461 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9454109100 0x7f94541a3370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:02.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:02.429+0000 7f9453e00640 1 -- 172.21.15.5:0/3393233461 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9454104980 msgr2=0x7f9454110120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:45:02.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:02.429+0000 7f9453e00640 1 --2- 172.21.15.5:0/3393233461 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9454104980 0x7f9454110120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:02.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:02.429+0000 7f9453e00640 1 -- 172.21.15.5:0/3393233461 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94541a3a70 con 0x7f9454103fd0 2024-09-10T09:45:02.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:02.430+0000 7f9453e00640 1 --2- 172.21.15.5:0/3393233461 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9454103fd0 0x7f945410fbe0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f944000e9f0 tx=0x7f944000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:45:02.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:02.430+0000 7f9450c00640 1 -- 172.21.15.5:0/3393233461 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f944000cd80 con 0x7f9454103fd0 2024-09-10T09:45:02.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:02.430+0000 7f9450c00640 1 -- 172.21.15.5:0/3393233461 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9440010910 con 0x7f9454103fd0 2024-09-10T09:45:02.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:02.430+0000 7f9450c00640 1 -- 172.21.15.5:0/3393233461 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9440018990 con 0x7f9454103fd0 2024-09-10T09:45:02.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:02.430+0000 7f945aa00640 1 -- 172.21.15.5:0/3393233461 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94541a3d60 con 0x7f9454103fd0 2024-09-10T09:45:02.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:02.430+0000 7f945aa00640 1 -- 172.21.15.5:0/3393233461 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94541ab600 con 0x7f9454103fd0 2024-09-10T09:45:02.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:02.433+0000 7f945d5fc740 1 Processor -- start 2024-09-10T09:45:02.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:02.433+0000 7f945d5fc740 1 -- start start 2024-09-10T09:45:02.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:02.434+0000 7f945d5fc740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563c28161b90 0x563c28161f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:45:02.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:02.434+0000 7f945d5fc740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563c281624b0 0x563c2816c830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:45:02.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:02.434+0000 7f9450c00640 1 -- 172.21.15.5:0/3393233461 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f9440010ac0 con 0x7f9454103fd0 2024-09-10T09:45:02.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:02.434+0000 7f945d5fc740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563c2816cd70 0x563c2816f160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:45:02.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:02.434+0000 7f945d5fc740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x563c280c83e0 con 0x563c2816cd70 2024-09-10T09:45:02.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:02.434+0000 7f945d5fc740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x563c2806fd80 con 0x563c281624b0 2024-09-10T09:45:02.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:02.434+0000 7f945d5fc740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x563c280c43a0 con 0x563c28161b90 2024-09-10T09:45:02.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:02.434+0000 7f9458a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563c28161b90 0x563c28161f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:45:02.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:02.434+0000 7f9458a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563c28161b90 0x563c28161f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:54212/0 (socket says 172.21.15.5:54212) 2024-09-10T09:45:02.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:02.434+0000 7f9458a00640 1 -- 172.21.15.5:0/1393714319 learned_addr learned my addr 172.21.15.5:0/1393714319 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:45:02.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:02.434+0000 7f9453e00640 1 --2- 172.21.15.5:0/1393714319 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563c281624b0 0x563c2816c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:45:02.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:02.434+0000 7f9453400640 1 --2- 172.21.15.5:0/1393714319 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563c2816cd70 0x563c2816f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:45:02.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:02.434+0000 7f9458a00640 1 -- 172.21.15.5:0/1393714319 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563c281624b0 msgr2=0x563c2816c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:45:02.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:02.434+0000 7f9458a00640 1 --2- 172.21.15.5:0/1393714319 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563c281624b0 0x563c2816c830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:02.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:02.435+0000 7f9458a00640 1 -- 172.21.15.5:0/1393714319 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563c2816cd70 msgr2=0x563c2816f160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:45:02.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:02.435+0000 7f9458a00640 1 --2- 172.21.15.5:0/1393714319 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563c2816cd70 0x563c2816f160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:02.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:02.435+0000 7f9458a00640 1 -- 172.21.15.5:0/1393714319 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563c2808eae0 con 0x563c28161b90 2024-09-10T09:45:02.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:02.435+0000 7f9453e00640 1 --2- 172.21.15.5:0/1393714319 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563c281624b0 0x563c2816c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:45:02.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:02.435+0000 7f9458a00640 1 --2- 172.21.15.5:0/1393714319 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563c28161b90 0x563c28161f70 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f944c009f70 tx=0x7f944c010800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a8e0a5f445de8ef2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:45:02.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:02.435+0000 7f9449200640 1 -- 172.21.15.5:0/1393714319 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f944c016070 con 0x563c28161b90 2024-09-10T09:45:02.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:02.435+0000 7f9449200640 1 -- 172.21.15.5:0/1393714319 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f944c004d20 con 0x563c28161b90 2024-09-10T09:45:02.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:02.435+0000 7f9449200640 1 -- 172.21.15.5:0/1393714319 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f944c012950 con 0x563c28161b90 2024-09-10T09:45:02.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:02.435+0000 7f9450c00640 1 --2- 172.21.15.5:0/3393233461 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f94380a2c00 0x7f94380a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:45:02.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:02.436+0000 7f9450c00640 1 -- 172.21.15.5:0/3393233461 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f94400cf740 con 0x7f9454103fd0 2024-09-10T09:45:02.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:02.436+0000 7f9453e00640 1 --2- 172.21.15.5:0/3393233461 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f94380a2c00 0x7f94380a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:45:02.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:02.436+0000 7f945d5fc740 1 -- 172.21.15.5:0/1393714319 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563c28161b90 msgr2=0x563c28161f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:45:02.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:02.436+0000 7f945d5fc740 1 --2- 172.21.15.5:0/1393714319 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563c28161b90 0x563c28161f70 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f944c009f70 tx=0x7f944c010800 comp rx=0 tx=0).stop 2024-09-10T09:45:02.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:02.436+0000 7f945d5fc740 1 -- 172.21.15.5:0/1393714319 shutdown_connections 2024-09-10T09:45:02.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:02.436+0000 7f945d5fc740 1 --2- 172.21.15.5:0/1393714319 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563c2816cd70 0x563c2816f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:02.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:02.436+0000 7f945d5fc740 1 --2- 172.21.15.5:0/1393714319 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563c281624b0 0x563c2816c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:02.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:02.436+0000 7f9453e00640 1 --2- 172.21.15.5:0/3393233461 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f94380a2c00 0x7f94380a50c0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f944009b900 tx=0x7f9440018420 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:45:02.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:02.436+0000 7f945d5fc740 1 --2- 172.21.15.5:0/1393714319 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563c28161b90 0x563c28161f70 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:02.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:02.436+0000 7f945d5fc740 1 -- 172.21.15.5:0/1393714319 >> 172.21.15.5:0/1393714319 conn(0x563c2815e120 msgr2=0x563c2815e500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:45:02.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:02.436+0000 7f945d5fc740 1 -- 172.21.15.5:0/1393714319 shutdown_connections 2024-09-10T09:45:02.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:02.437+0000 7f945d5fc740 1 -- 172.21.15.5:0/1393714319 wait complete. 2024-09-10T09:45:02.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:02.437+0000 7f945d5fc740 1 -- start start 2024-09-10T09:45:02.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:02.437+0000 7f945d5fc740 1 Processor -- start 2024-09-10T09:45:02.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:02.438+0000 7f945d5fc740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563c28161b90 0x563c28169d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:45:02.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:02.438+0000 7f945d5fc740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563c281624b0 0x563c2816a270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:45:02.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:02.438+0000 7f945d5fc740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563c2816cd70 0x563c2816a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:45:02.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:02.438+0000 7f945d5fc740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x563c2816f9e0 con 0x563c2816cd70 2024-09-10T09:45:02.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:02.438+0000 7f945d5fc740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x563c280830f0 con 0x563c28161b90 2024-09-10T09:45:02.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:02.438+0000 7f945d5fc740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x563c2816fb60 con 0x563c281624b0 2024-09-10T09:45:02.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:02.438+0000 7f945d5fc740 10 client.0 _ll_register_callbacks cb 0x563c280749e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:45:02.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:02.438+0000 7f9453400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563c281624b0 0x563c2816a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:45:02.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:02.438+0000 7f9458a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563c28161b90 0x563c28169d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:45:02.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:02.438+0000 7f9453400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563c281624b0 0x563c2816a270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:54218/0 (socket says 172.21.15.5:54218) 2024-09-10T09:45:02.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:02.438+0000 7f9453400640 1 -- 172.21.15.5:0/3027690538 learned_addr learned my addr 172.21.15.5:0/3027690538 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:45:02.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:02.438+0000 7f9458a00640 1 --2- 172.21.15.5:0/3027690538 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563c2816cd70 0x563c2816a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:45:02.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:02.438+0000 7f9453400640 1 -- 172.21.15.5:0/3027690538 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563c28161b90 msgr2=0x563c28169d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:45:02.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:02.438+0000 7f9453400640 1 --2- 172.21.15.5:0/3027690538 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563c28161b90 0x563c28169d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:02.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:02.438+0000 7f9453400640 1 -- 172.21.15.5:0/3027690538 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563c2816cd70 msgr2=0x563c2816a7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:45:02.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:02.438+0000 7f9453400640 1 --2- 172.21.15.5:0/3027690538 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563c2816cd70 0x563c2816a7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:02.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:02.438+0000 7f9453400640 1 -- 172.21.15.5:0/3027690538 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563c27e393c0 con 0x563c281624b0 2024-09-10T09:45:02.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:02.438+0000 7f9458a00640 1 --2- 172.21.15.5:0/3027690538 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563c28161b90 0x563c28169d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:45:02.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:02.439+0000 7f9453400640 1 --2- 172.21.15.5:0/3027690538 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563c281624b0 0x563c2816a270 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f944402a910 tx=0x7f94440317e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=91428fff0d2fdc server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:45:02.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:02.439+0000 7f943d600640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-09-10T09:45:02.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:02.439+0000 7f943d600640 1 -- 172.21.15.5:0/3027690538 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9444035040 con 0x563c281624b0 2024-09-10T09:45:02.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:02.439+0000 7f945d5fc740 1 -- 172.21.15.5:0/3027690538 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f944c004d20 con 0x563c281624b0 2024-09-10T09:45:02.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:02.439+0000 7f943d600640 1 -- 172.21.15.5:0/3027690538 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f9444004240 con 0x563c281624b0 2024-09-10T09:45:02.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:02.439+0000 7f943d600640 1 -- 172.21.15.5:0/3027690538 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f9444004440 con 0x563c281624b0 2024-09-10T09:45:02.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:02.439+0000 7f945d5fc740 10 client.26722 fetch_fsmap learned FSMap version 34 2024-09-10T09:45:02.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:02.439+0000 7f945d5fc740 1 -- 172.21.15.5:0/3027690538 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x563c2817b7a0 con 0x563c281624b0 2024-09-10T09:45:02.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:02.440+0000 7f943d600640 1 -- 172.21.15.5:0/3027690538 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f944c004d20 con 0x563c281624b0 2024-09-10T09:45:02.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:02.440+0000 7f943d600640 1 -- 172.21.15.5:0/3027690538 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f944403c670 con 0x563c281624b0 2024-09-10T09:45:02.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:02.440+0000 7f945d5fc740 10 client.26722 fetch_fsmap finished waiting for FSMap version 34 2024-09-10T09:45:02.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:02.440+0000 7f945d5fc740 10 client.26722 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T09:45:02.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:02.440+0000 7f945d5fc740 20 client.26722 populate_metadata read hostname 'smithi005' 2024-09-10T09:45:02.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:02.440+0000 7f945d5fc740 1 --2- 172.21.15.5:0/3027690538 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x563c2817cc00 0x563c2817f040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:45:02.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:02.440+0000 7f945d5fc740 4 client.26722 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:45:02.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:02.440+0000 7f945d5fc740 1 -- 172.21.15.5:0/3027690538 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563c280d5950 con 0x563c2817cc00 2024-09-10T09:45:02.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:02.440+0000 7f9458a00640 1 --2- 172.21.15.5:0/3027690538 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x563c2817cc00 0x563c2817f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:45:02.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:02.441+0000 7f9458a00640 1 --2- 172.21.15.5:0/3027690538 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x563c2817cc00 0x563c2817f040 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=57298035ee9c119d server_cookie=de69ade3de8b6489 in_seq=0 out_seq=0 2024-09-10T09:45:02.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:02.441+0000 7f943d600640 10 client.26722 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T09:45:02.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:02.441+0000 7f943d600640 1 -- 172.21.15.5:0/3027690538 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563c280d5950 con 0x563c2817cc00 2024-09-10T09:45:02.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:02.441+0000 7f943d600640 10 client.26722 handle_command_reply: tid=0 2024-09-10T09:45:02.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:02.442+0000 7f945d5fc740 1 client.26722 shutdown 2024-09-10T09:45:02.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:02.442+0000 7f945d5fc740 1 -- 172.21.15.5:0/3027690538 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563c281624b0 msgr2=0x563c2816a270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:45:02.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:02.442+0000 7f945d5fc740 1 --2- 172.21.15.5:0/3027690538 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563c281624b0 0x563c2816a270 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f944402a910 tx=0x7f94440317e0 comp rx=0 tx=0).stop 2024-09-10T09:45:02.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:02.442+0000 7f945d5fc740 1 -- 172.21.15.5:0/3027690538 shutdown_connections 2024-09-10T09:45:02.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:02.442+0000 7f945d5fc740 1 --2- 172.21.15.5:0/3027690538 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x563c2817cc00 0x563c2817f040 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:02.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:02.442+0000 7f945d5fc740 1 --2- 172.21.15.5:0/3027690538 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563c2816cd70 0x563c2816a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:02.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:02.442+0000 7f945d5fc740 1 --2- 172.21.15.5:0/3027690538 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563c281624b0 0x563c2816a270 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:02.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:02.442+0000 7f945d5fc740 1 --2- 172.21.15.5:0/3027690538 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563c28161b90 0x563c28169d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:02.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:02.442+0000 7f943d600640 0 client.26722 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T09:45:02.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:02.442+0000 7f945d5fc740 1 -- 172.21.15.5:0/3027690538 >> 172.21.15.5:0/3027690538 conn(0x563c2815e120 msgr2=0x563c28163e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:45:02.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:02.442+0000 7f945d5fc740 1 -- 172.21.15.5:0/3027690538 shutdown_connections 2024-09-10T09:45:02.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:02.442+0000 7f945d5fc740 1 -- 172.21.15.5:0/3027690538 wait complete. 2024-09-10T09:45:02.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:02.443+0000 7f945d5fc740 20 client.26722 trim_cache size 0 max 16384 2024-09-10T09:45:02.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:02.460+0000 7f945d5fc740 1 Processor -- start 2024-09-10T09:45:02.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:02.461+0000 7f945d5fc740 1 -- start start 2024-09-10T09:45:02.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:02.461+0000 7f945d5fc740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563c28161b90 0x563c2816bca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:45:02.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:02.461+0000 7f945d5fc740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563c281624b0 0x563c28184910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:45:02.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:02.461+0000 7f945d5fc740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563c2816cd70 0x563c28184e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:45:02.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:02.461+0000 7f945d5fc740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f94541b55a0 con 0x563c281624b0 2024-09-10T09:45:02.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:02.461+0000 7f945d5fc740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f94541b70e0 con 0x563c2816cd70 2024-09-10T09:45:02.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:02.461+0000 7f945d5fc740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f94541b5c70 con 0x563c28161b90 2024-09-10T09:45:02.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:02.461+0000 7f9453400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563c281624b0 0x563c28184910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:45:02.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:02.461+0000 7f9458a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563c28161b90 0x563c2816bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:45:02.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:02.461+0000 7f9453400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563c281624b0 0x563c28184910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60036/0 (socket says 172.21.15.5:60036) 2024-09-10T09:45:02.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:02.461+0000 7f9453400640 1 -- 172.21.15.5:0/2588850655 learned_addr learned my addr 172.21.15.5:0/2588850655 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:45:02.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:02.461+0000 7f9458a00640 1 --2- 172.21.15.5:0/2588850655 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563c2816cd70 0x563c28184e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:45:02.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:02.462+0000 7f9458a00640 1 -- 172.21.15.5:0/2588850655 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563c2816cd70 msgr2=0x563c28184e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:45:02.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:02.462+0000 7f9458a00640 1 --2- 172.21.15.5:0/2588850655 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563c2816cd70 0x563c28184e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:02.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:02.462+0000 7f9458a00640 1 -- 172.21.15.5:0/2588850655 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563c281624b0 msgr2=0x563c28184910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:45:02.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:02.462+0000 7f9458a00640 1 --2- 172.21.15.5:0/2588850655 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563c281624b0 0x563c28184910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:02.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:02.462+0000 7f9458a00640 1 -- 172.21.15.5:0/2588850655 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563c2816c6a0 con 0x563c28161b90 2024-09-10T09:45:02.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:02.462+0000 7f9458a00640 1 --2- 172.21.15.5:0/2588850655 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563c28161b90 0x563c2816bca0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f944c000f80 tx=0x7f944c009f00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eec451befa982543 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:45:02.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:02.462+0000 7f943fe00640 1 -- 172.21.15.5:0/2588850655 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f944c016070 con 0x563c28161b90 2024-09-10T09:45:02.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:02.462+0000 7f943fe00640 1 -- 172.21.15.5:0/2588850655 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f944c02e070 con 0x563c28161b90 2024-09-10T09:45:02.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:02.462+0000 7f943fe00640 1 -- 172.21.15.5:0/2588850655 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f944c023040 con 0x563c28161b90 2024-09-10T09:45:02.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:02.463+0000 7f945d5fc740 1 -- 172.21.15.5:0/2588850655 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563c28161b90 msgr2=0x563c2816bca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:45:02.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:02.463+0000 7f945d5fc740 1 --2- 172.21.15.5:0/2588850655 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563c28161b90 0x563c2816bca0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f944c000f80 tx=0x7f944c009f00 comp rx=0 tx=0).stop 2024-09-10T09:45:02.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:02.463+0000 7f945d5fc740 1 -- 172.21.15.5:0/2588850655 shutdown_connections 2024-09-10T09:45:02.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:02.463+0000 7f945d5fc740 1 --2- 172.21.15.5:0/2588850655 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563c2816cd70 0x563c28184e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:02.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:02.463+0000 7f945d5fc740 1 --2- 172.21.15.5:0/2588850655 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563c281624b0 0x563c28184910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:02.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:02.463+0000 7f945d5fc740 1 --2- 172.21.15.5:0/2588850655 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563c28161b90 0x563c2816bca0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:02.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:02.463+0000 7f945d5fc740 1 -- 172.21.15.5:0/2588850655 >> 172.21.15.5:0/2588850655 conn(0x563c2815e120 msgr2=0x563c2816b590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:45:02.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:02.463+0000 7f945d5fc740 1 -- 172.21.15.5:0/2588850655 shutdown_connections 2024-09-10T09:45:02.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:02.463+0000 7f945d5fc740 1 -- 172.21.15.5:0/2588850655 wait complete. 2024-09-10T09:45:02.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:02.464+0000 7f945d5fc740 1 -- start start 2024-09-10T09:45:02.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:02.464+0000 7f945d5fc740 1 Processor -- start 2024-09-10T09:45:02.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:02.464+0000 7f945d5fc740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563c28161b90 0x563c28166eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:45:02.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:02.464+0000 7f945d5fc740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563c281624b0 0x563c281673f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:45:02.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:02.464+0000 7f945d5fc740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563c2816cd70 0x563c28167930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:45:02.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:02.464+0000 7f945d5fc740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f94541b6a10 con 0x563c2816cd70 2024-09-10T09:45:02.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:02.464+0000 7f945d5fc740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f94541b6340 con 0x563c281624b0 2024-09-10T09:45:02.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:02.464+0000 7f945d5fc740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f94541b3580 con 0x563c28161b90 2024-09-10T09:45:02.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:02.464+0000 7f945d5fc740 10 client.0 _ll_register_callbacks cb 0x7f9444005d50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:45:02.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:02.465+0000 7f9453400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563c281624b0 0x563c281673f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:45:02.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:02.465+0000 7f9453400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563c281624b0 0x563c281673f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:45222/0 (socket says 172.21.15.5:45222) 2024-09-10T09:45:02.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:02.465+0000 7f9453400640 1 -- 172.21.15.5:0/2140741737 learned_addr learned my addr 172.21.15.5:0/2140741737 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:45:02.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:02.465+0000 7f9458a00640 1 --2- 172.21.15.5:0/2140741737 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563c28161b90 0x563c28166eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:45:02.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:02.465+0000 7f9458a00640 1 --2- 172.21.15.5:0/2140741737 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563c2816cd70 0x563c28167930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:45:02.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:02.465+0000 7f9453400640 1 -- 172.21.15.5:0/2140741737 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563c28161b90 msgr2=0x563c28166eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:45:02.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:02.465+0000 7f9453400640 1 --2- 172.21.15.5:0/2140741737 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563c28161b90 0x563c28166eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:02.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:02.465+0000 7f9453400640 1 -- 172.21.15.5:0/2140741737 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563c2816cd70 msgr2=0x563c28167930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:45:02.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:02.465+0000 7f9453400640 1 --2- 172.21.15.5:0/2140741737 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563c2816cd70 0x563c28167930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:02.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:02.465+0000 7f9453400640 1 -- 172.21.15.5:0/2140741737 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563c281e6a40 con 0x563c281624b0 2024-09-10T09:45:02.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:02.465+0000 7f9458a00640 1 --2- 172.21.15.5:0/2140741737 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563c2816cd70 0x563c28167930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:45:02.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:02.465+0000 7f9453400640 1 --2- 172.21.15.5:0/2140741737 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563c281624b0 0x563c281673f0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f944403ca90 tx=0x7f9444000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bc92003d3859ae6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:45:02.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:02.465+0000 7f943d600640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-09-10T09:45:02.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:02.466+0000 7f943d600640 1 -- 172.21.15.5:0/2140741737 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9444035040 con 0x563c281624b0 2024-09-10T09:45:02.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:02.466+0000 7f945d5fc740 1 -- 172.21.15.5:0/2140741737 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f944c02e070 con 0x563c281624b0 2024-09-10T09:45:02.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:02.466+0000 7f943d600640 1 -- 172.21.15.5:0/2140741737 <== mon.2 v2:172.21.15.102:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f9444005030 con 0x563c281624b0 2024-09-10T09:45:02.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:02.466+0000 7f943d600640 1 -- 172.21.15.5:0/2140741737 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f9444063050 con 0x563c281624b0 2024-09-10T09:45:02.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:02.466+0000 7f945d5fc740 10 client.26699 fetch_fsmap learned FSMap version 34 2024-09-10T09:45:02.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:02.466+0000 7f945d5fc740 1 -- 172.21.15.5:0/2140741737 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x563c281e7d30 con 0x563c281624b0 2024-09-10T09:45:02.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:02.466+0000 7f943d600640 1 -- 172.21.15.5:0/2140741737 <== mon.2 v2:172.21.15.102:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f944c02e070 con 0x563c281624b0 2024-09-10T09:45:02.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:02.466+0000 7f943d600640 1 -- 172.21.15.5:0/2140741737 <== mon.2 v2:172.21.15.102:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f9444004aa0 con 0x563c281624b0 2024-09-10T09:45:02.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:02.467+0000 7f945d5fc740 10 client.26699 fetch_fsmap finished waiting for FSMap version 34 2024-09-10T09:45:02.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:02.467+0000 7f945d5fc740 10 client.26699 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T09:45:02.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:02.467+0000 7f945d5fc740 20 client.26699 populate_metadata read hostname 'smithi005' 2024-09-10T09:45:02.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:02.467+0000 7f945d5fc740 1 --2- 172.21.15.5:0/2140741737 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x563c281e84a0 0x563c281ea8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:45:02.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:02.467+0000 7f945d5fc740 4 client.26699 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T09:45:02.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:02.467+0000 7f945d5fc740 1 -- 172.21.15.5:0/2140741737 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "scrub status"}) -- 0x563c280cfe60 con 0x563c281e84a0 2024-09-10T09:45:02.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:02.467+0000 7f9458a00640 1 --2- 172.21.15.5:0/2140741737 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x563c281e84a0 0x563c281ea8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:45:02.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:02.467+0000 7f9458a00640 1 --2- 172.21.15.5:0/2140741737 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x563c281e84a0 0x563c281ea8c0 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=233ab1fdb80cd336 server_cookie=a20a6059a65059c6 in_seq=0 out_seq=0 2024-09-10T09:45:02.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:02.467+0000 7f943d600640 10 client.26699 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T09:45:02.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:02.468+0000 7f943d600640 1 -- 172.21.15.5:0/2140741737 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x563c280cfe60 con 0x563c281e84a0 2024-09-10T09:45:02.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:02.468+0000 7f943d600640 10 client.26699 handle_command_reply: tid=0 2024-09-10T09:45:02.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:02.468+0000 7f945d5fc740 1 client.26699 shutdown 2024-09-10T09:45:02.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:02.468+0000 7f945d5fc740 1 -- 172.21.15.5:0/2140741737 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563c281624b0 msgr2=0x563c281673f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:45:02.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:02.468+0000 7f945d5fc740 1 --2- 172.21.15.5:0/2140741737 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563c281624b0 0x563c281673f0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f944403ca90 tx=0x7f9444000bc0 comp rx=0 tx=0).stop 2024-09-10T09:45:02.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:02.468+0000 7f945d5fc740 1 -- 172.21.15.5:0/2140741737 shutdown_connections 2024-09-10T09:45:02.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:02.468+0000 7f945d5fc740 1 --2- 172.21.15.5:0/2140741737 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x563c281e84a0 0x563c281ea8c0 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:02.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:02.468+0000 7f945d5fc740 1 --2- 172.21.15.5:0/2140741737 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563c2816cd70 0x563c28167930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:02.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:02.468+0000 7f945d5fc740 1 --2- 172.21.15.5:0/2140741737 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563c281624b0 0x563c281673f0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:02.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:02.468+0000 7f945d5fc740 1 --2- 172.21.15.5:0/2140741737 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563c28161b90 0x563c28166eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:02.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:02.468+0000 7f943d600640 0 client.26699 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T09:45:02.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:02.468+0000 7f945d5fc740 1 -- 172.21.15.5:0/2140741737 >> 172.21.15.5:0/2140741737 conn(0x563c2815e120 msgr2=0x563c2816a510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:45:02.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:02.468+0000 7f945d5fc740 1 -- 172.21.15.5:0/2140741737 shutdown_connections 2024-09-10T09:45:02.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:02.468+0000 7f945d5fc740 1 -- 172.21.15.5:0/2140741737 wait complete. 2024-09-10T09:45:02.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:02.468+0000 7f945d5fc740 20 client.26699 trim_cache size 0 max 16384 2024-09-10T09:45:02.473 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-10T09:45:02.473 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "no active scrubs running", 2024-09-10T09:45:02.473 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": {} 2024-09-10T09:45:02.473 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-10T09:45:02.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:02.469+0000 7f945aa00640 1 -- 172.21.15.5:0/3393233461 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f94380a2c00 msgr2=0x7f94380a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:45:02.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:02.469+0000 7f945aa00640 1 --2- 172.21.15.5:0/3393233461 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f94380a2c00 0x7f94380a50c0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f944009b900 tx=0x7f9440018420 comp rx=0 tx=0).stop 2024-09-10T09:45:02.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:02.469+0000 7f945aa00640 1 -- 172.21.15.5:0/3393233461 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9454103fd0 msgr2=0x7f945410fbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:45:02.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:02.469+0000 7f945aa00640 1 --2- 172.21.15.5:0/3393233461 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9454103fd0 0x7f945410fbe0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f944000e9f0 tx=0x7f944000eeb0 comp rx=0 tx=0).stop 2024-09-10T09:45:02.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:02.469+0000 7f945aa00640 1 -- 172.21.15.5:0/3393233461 shutdown_connections 2024-09-10T09:45:02.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:02.469+0000 7f945aa00640 1 --2- 172.21.15.5:0/3393233461 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f94380a2c00 0x7f94380a50c0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:02.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:02.469+0000 7f945aa00640 1 --2- 172.21.15.5:0/3393233461 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9454109100 0x7f94541a3370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:02.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:02.469+0000 7f945aa00640 1 --2- 172.21.15.5:0/3393233461 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9454104980 0x7f9454110120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:02.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:02.469+0000 7f945aa00640 1 --2- 172.21.15.5:0/3393233461 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9454103fd0 0x7f945410fbe0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:02.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:02.469+0000 7f945aa00640 1 -- 172.21.15.5:0/3393233461 >> 172.21.15.5:0/3393233461 conn(0x7f94540ffa30 msgr2=0x7f9454101e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:45:02.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:02.469+0000 7f945aa00640 1 -- 172.21.15.5:0/3393233461 shutdown_connections 2024-09-10T09:45:02.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:02.469+0000 7f945aa00640 1 -- 172.21.15.5:0/3393233461 wait complete. 2024-09-10T09:45:03.054 INFO:tasks.workunit.client.0.smithi005.stdout: 1 359745 5.51 MB/sec execute 164 sec latency 3413.411 ms 2024-09-10T09:45:03.306 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:45:02 smithi005 ceph-mon[30350]: pgmap v594: 129 pgs: 129 active+clean; 145 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1006 KiB/s rd, 1.5 MiB/s wr, 102 op/s 2024-09-10T09:45:03.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:45:02 smithi159 ceph-mon[35513]: pgmap v594: 129 pgs: 129 active+clean; 145 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1006 KiB/s rd, 1.5 MiB/s wr, 102 op/s 2024-09-10T09:45:03.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:45:02 smithi102 ceph-mon[35442]: pgmap v594: 129 pgs: 129 active+clean; 145 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1006 KiB/s rd, 1.5 MiB/s wr, 102 op/s 2024-09-10T09:45:03.610 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-10T09:45:03.610 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T09:45:03.990 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:45:04.054 INFO:tasks.workunit.client.0.smithi005.stdout: 1 359745 5.48 MB/sec execute 165 sec latency 4413.537 ms 2024-09-10T09:45:04.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:04.522+0000 7f0c45e00640 1 -- 172.21.15.5:0/3816227558 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0c4010e3c0 msgr2=0x7f0c40110810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:45:04.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:04.522+0000 7f0c45e00640 1 --2- 172.21.15.5:0/3816227558 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0c4010e3c0 0x7f0c40110810 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f0c38009ca0 tx=0x7f0c3802a950 comp rx=0 tx=0).stop 2024-09-10T09:45:04.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:04.522+0000 7f0c45e00640 1 -- 172.21.15.5:0/3816227558 shutdown_connections 2024-09-10T09:45:04.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:04.522+0000 7f0c45e00640 1 --2- 172.21.15.5:0/3816227558 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0c4010e3c0 0x7f0c40110810 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:04.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:04.522+0000 7f0c45e00640 1 --2- 172.21.15.5:0/3816227558 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0c40074f40 0x7f0c4010de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:04.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:04.522+0000 7f0c45e00640 1 --2- 172.21.15.5:0/3816227558 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0c40074620 0x7f0c40074a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:04.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:04.522+0000 7f0c45e00640 1 -- 172.21.15.5:0/3816227558 >> 172.21.15.5:0/3816227558 conn(0x7f0c400fd8c0 msgr2=0x7f0c400ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:45:04.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:04.522+0000 7f0c45e00640 1 -- 172.21.15.5:0/3816227558 shutdown_connections 2024-09-10T09:45:04.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:04.522+0000 7f0c45e00640 1 -- 172.21.15.5:0/3816227558 wait complete. 2024-09-10T09:45:04.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:04.523+0000 7f0c45e00640 1 Processor -- start 2024-09-10T09:45:04.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:04.523+0000 7f0c45e00640 1 -- start start 2024-09-10T09:45:04.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:04.524+0000 7f0c45e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0c40074620 0x7f0c4019a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:45:04.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:04.524+0000 7f0c45e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0c40074f40 0x7f0c4019ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:45:04.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:04.524+0000 7f0c45e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0c4010e3c0 0x7f0c4019ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:45:04.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:04.524+0000 7f0c45e00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f0c401141b0 con 0x7f0c40074f40 2024-09-10T09:45:04.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:04.524+0000 7f0c45e00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f0c40114030 con 0x7f0c40074620 2024-09-10T09:45:04.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:04.524+0000 7f0c45e00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f0c40114330 con 0x7f0c4010e3c0 2024-09-10T09:45:04.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:04.524+0000 7f0c3e000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0c40074f40 0x7f0c4019ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:45:04.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:04.524+0000 7f0c3e000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0c40074f40 0x7f0c4019ab90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60058/0 (socket says 172.21.15.5:60058) 2024-09-10T09:45:04.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:04.524+0000 7f0c3e000640 1 -- 172.21.15.5:0/2620995869 learned_addr learned my addr 172.21.15.5:0/2620995869 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:45:04.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:04.524+0000 7f0c3ea00640 1 --2- 172.21.15.5:0/2620995869 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0c40074620 0x7f0c4019a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:45:04.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:04.524+0000 7f0c3f400640 1 --2- 172.21.15.5:0/2620995869 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0c4010e3c0 0x7f0c4019ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:45:04.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:04.524+0000 7f0c3e000640 1 -- 172.21.15.5:0/2620995869 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0c4010e3c0 msgr2=0x7f0c4019ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:45:04.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:04.524+0000 7f0c3e000640 1 --2- 172.21.15.5:0/2620995869 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0c4010e3c0 0x7f0c4019ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:04.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:04.525+0000 7f0c3e000640 1 -- 172.21.15.5:0/2620995869 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0c40074620 msgr2=0x7f0c4019a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:45:04.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:04.525+0000 7f0c3e000640 1 --2- 172.21.15.5:0/2620995869 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0c40074620 0x7f0c4019a650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:04.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:04.525+0000 7f0c3e000640 1 -- 172.21.15.5:0/2620995869 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c4019f600 con 0x7f0c40074f40 2024-09-10T09:45:04.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:04.525+0000 7f0c3ea00640 1 --2- 172.21.15.5:0/2620995869 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0c40074620 0x7f0c4019a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:45:04.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:04.525+0000 7f0c3e000640 1 --2- 172.21.15.5:0/2620995869 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0c40074f40 0x7f0c4019ab90 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f0c2c00e9f0 tx=0x7f0c2c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:45:04.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:04.525+0000 7f0c44a00640 1 -- 172.21.15.5:0/2620995869 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c2c00cd80 con 0x7f0c40074f40 2024-09-10T09:45:04.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:04.525+0000 7f0c44a00640 1 -- 172.21.15.5:0/2620995869 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0c2c010910 con 0x7f0c40074f40 2024-09-10T09:45:04.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:04.525+0000 7f0c44a00640 1 -- 172.21.15.5:0/2620995869 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c2c018990 con 0x7f0c40074f40 2024-09-10T09:45:04.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:04.525+0000 7f0c45e00640 1 -- 172.21.15.5:0/2620995869 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c4019f8f0 con 0x7f0c40074f40 2024-09-10T09:45:04.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:04.525+0000 7f0c45e00640 1 -- 172.21.15.5:0/2620995869 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c401a71d0 con 0x7f0c40074f40 2024-09-10T09:45:04.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:04.526+0000 7f0c45e00640 1 -- 172.21.15.5:0/2620995869 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0c40074a00 con 0x7f0c40074f40 2024-09-10T09:45:04.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:04.529+0000 7f0c44a00640 1 -- 172.21.15.5:0/2620995869 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f0c2c010ac0 con 0x7f0c40074f40 2024-09-10T09:45:04.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:04.530+0000 7f0c44a00640 1 --2- 172.21.15.5:0/2620995869 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f0c240a2b80 0x7f0c240a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:45:04.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:04.530+0000 7f0c44a00640 1 -- 172.21.15.5:0/2620995869 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f0c2c0cf720 con 0x7f0c40074f40 2024-09-10T09:45:04.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:04.530+0000 7f0c3ea00640 1 --2- 172.21.15.5:0/2620995869 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f0c240a2b80 0x7f0c240a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:45:04.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:04.531+0000 7f0c3ea00640 1 --2- 172.21.15.5:0/2620995869 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f0c240a2b80 0x7f0c240a5040 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f0c30009fc0 tx=0x7f0c30005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:45:04.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:04.535+0000 7f0c44a00640 1 -- 172.21.15.5:0/2620995869 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f0c2c09bbe0 con 0x7f0c40074f40 2024-09-10T09:45:04.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:04.763+0000 7f0c45e00640 1 -- 172.21.15.5:0/2620995869 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0c40109570 con 0x7f0c40074f40 2024-09-10T09:45:04.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:04.763+0000 7f0c44a00640 1 -- 172.21.15.5:0/2620995869 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+11969 (secure 0 0 0) 0x7f0c2c09fee0 con 0x7f0c40074f40 2024-09-10T09:45:04.765 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-10T09:45:04.766 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":34,"btime":"2024-09-10T09:42:57:737230+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14454,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/2970594311","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2970594311},{"type":"v1","addr":"172.21.15.5:6835","nonce":2970594311}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14514,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6839/3616502174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":3616502174},{"type":"v1","addr":"172.21.15.5:6839","nonce":3616502174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":34,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T09:35:29.626287+0000","modified":"2024-09-10T09:42:56.961831+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24542,"mds_1":24535,"mds_2":24511,"mds_3":24506,"mds_4":24443},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14520":{"gid":14520,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/2073599842","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":2073599842},{"type":"v1","addr":"172.21.15.5:6841","nonce":2073599842}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"k","rank":4,"incarnation":26,"state":"up:active","state_seq":24,"addr":"172.21.15.102:6833/2457007427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":2457007427},{"type":"v1","addr":"172.21.15.102:6833","nonce":2457007427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/356445159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":356445159},{"type":"v1","addr":"172.21.15.159:6833","nonce":356445159}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6837/2894314507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":2894314507},{"type":"v1","addr":"172.21.15.5:6837","nonce":2894314507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.102:6835/1647549111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":1647549111},{"type":"v1","addr":"172.21.15.102:6835","nonce":1647549111}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6835/952657242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":952657242},{"type":"v1","addr":"172.21.15.159:6835","nonce":952657242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":19,"addr":"172.21.15.102:6837/2689776764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":2689776764},{"type":"v1","addr":"172.21.15.102:6837","nonce":2689776764}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.159:6837/620627639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":620627639},{"type":"v1","addr":"172.21.15.159:6837","nonce":620627639}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24535":{"gid":24535,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":18,"addr":"172.21.15.159:6839/2073739599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":2073739599},{"type":"v1","addr":"172.21.15.159:6839","nonce":2073739599}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24542":{"gid":24542,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.102:6839/335409749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6838","nonce":335409749},{"type":"v1","addr":"172.21.15.102:6839","nonce":335409749}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24542,"qdb_cluster":[24542,24535,24511,24506,24443]},"id":1}]} 2024-09-10T09:45:04.766 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 34 2024-09-10T09:45:04.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:04.766+0000 7f0c45e00640 1 -- 172.21.15.5:0/2620995869 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f0c240a2b80 msgr2=0x7f0c240a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:45:04.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:04.766+0000 7f0c45e00640 1 --2- 172.21.15.5:0/2620995869 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f0c240a2b80 0x7f0c240a5040 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f0c30009fc0 tx=0x7f0c30005f90 comp rx=0 tx=0).stop 2024-09-10T09:45:04.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:04.766+0000 7f0c45e00640 1 -- 172.21.15.5:0/2620995869 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0c40074f40 msgr2=0x7f0c4019ab90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:45:04.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:04.766+0000 7f0c45e00640 1 --2- 172.21.15.5:0/2620995869 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0c40074f40 0x7f0c4019ab90 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f0c2c00e9f0 tx=0x7f0c2c00eeb0 comp rx=0 tx=0).stop 2024-09-10T09:45:04.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:04.766+0000 7f0c45e00640 1 -- 172.21.15.5:0/2620995869 shutdown_connections 2024-09-10T09:45:04.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:04.766+0000 7f0c45e00640 1 --2- 172.21.15.5:0/2620995869 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f0c240a2b80 0x7f0c240a5040 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:04.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:04.766+0000 7f0c45e00640 1 --2- 172.21.15.5:0/2620995869 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0c4010e3c0 0x7f0c4019ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:04.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:04.766+0000 7f0c45e00640 1 --2- 172.21.15.5:0/2620995869 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0c40074f40 0x7f0c4019ab90 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:04.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:04.766+0000 7f0c45e00640 1 --2- 172.21.15.5:0/2620995869 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0c40074620 0x7f0c4019a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:04.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:04.766+0000 7f0c45e00640 1 -- 172.21.15.5:0/2620995869 >> 172.21.15.5:0/2620995869 conn(0x7f0c400fd8c0 msgr2=0x7f0c400fe520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:45:04.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:04.766+0000 7f0c45e00640 1 -- 172.21.15.5:0/2620995869 shutdown_connections 2024-09-10T09:45:04.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:04.766+0000 7f0c45e00640 1 -- 172.21.15.5:0/2620995869 wait complete. 2024-09-10T09:45:05.054 INFO:tasks.workunit.client.0.smithi005.stdout: 1 359745 5.44 MB/sec execute 166 sec latency 5413.673 ms 2024-09-10T09:45:05.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:45:04 smithi005 ceph-mon[30350]: pgmap v595: 129 pgs: 129 active+clean; 145 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.5 MiB/s wr, 103 op/s 2024-09-10T09:45:05.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:45:04 smithi005 ceph-mon[30350]: from='client.? 172.21.15.5:0/2620995869' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:45:05.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:45:04 smithi159 ceph-mon[35513]: pgmap v595: 129 pgs: 129 active+clean; 145 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.5 MiB/s wr, 103 op/s 2024-09-10T09:45:05.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:45:04 smithi159 ceph-mon[35513]: from='client.? 172.21.15.5:0/2620995869' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:45:05.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:45:04 smithi102 ceph-mon[35442]: pgmap v595: 129 pgs: 129 active+clean; 145 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.5 MiB/s wr, 103 op/s 2024-09-10T09:45:05.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:45:04 smithi102 ceph-mon[35442]: from='client.? 172.21.15.5:0/2620995869' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:45:05.614 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-10T09:45:06.031 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:45:06.054 INFO:tasks.workunit.client.0.smithi005.stdout: 1 359745 5.41 MB/sec execute 167 sec latency 6413.989 ms 2024-09-10T09:45:06.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:06.657+0000 7f4647200640 1 -- 172.21.15.5:0/4276995944 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4640104010 msgr2=0x7f46401043f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:45:06.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:06.657+0000 7f4647200640 1 --2- 172.21.15.5:0/4276995944 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4640104010 0x7f46401043f0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f4630009900 tx=0x7f463002a3f0 comp rx=0 tx=0).stop 2024-09-10T09:45:06.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:06.657+0000 7f4647200640 1 -- 172.21.15.5:0/4276995944 shutdown_connections 2024-09-10T09:45:06.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:06.657+0000 7f4647200640 1 --2- 172.21.15.5:0/4276995944 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4640109140 0x7f4640110cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:06.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:06.657+0000 7f4647200640 1 --2- 172.21.15.5:0/4276995944 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f46401049c0 0x7f4640108a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:06.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:06.657+0000 7f4647200640 1 --2- 172.21.15.5:0/4276995944 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4640104010 0x7f46401043f0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:06.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:06.657+0000 7f4647200640 1 -- 172.21.15.5:0/4276995944 >> 172.21.15.5:0/4276995944 conn(0x7f46400ffa50 msgr2=0x7f4640101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:45:06.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:06.657+0000 7f4647200640 1 -- 172.21.15.5:0/4276995944 shutdown_connections 2024-09-10T09:45:06.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:06.657+0000 7f4647200640 1 -- 172.21.15.5:0/4276995944 wait complete. 2024-09-10T09:45:06.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:06.657+0000 7f4647200640 1 Processor -- start 2024-09-10T09:45:06.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:06.657+0000 7f4647200640 1 -- start start 2024-09-10T09:45:06.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:06.657+0000 7f4647200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4640104010 0x7f464019a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:45:06.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:06.657+0000 7f4647200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f46401049c0 0x7f464019ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:45:06.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:06.657+0000 7f4647200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4640109140 0x7f464019ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:45:06.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:06.657+0000 7f4647200640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f4640114140 con 0x7f4640109140 2024-09-10T09:45:06.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:06.657+0000 7f4647200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f4640113fc0 con 0x7f4640104010 2024-09-10T09:45:06.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:06.657+0000 7f4647200640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f46401142c0 con 0x7f46401049c0 2024-09-10T09:45:06.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:06.657+0000 7f4645200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4640109140 0x7f464019ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:45:06.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:06.657+0000 7f463f400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f46401049c0 0x7f464019ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:45:06.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:06.657+0000 7f463fe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4640104010 0x7f464019a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:45:06.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:06.657+0000 7f4645200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4640109140 0x7f464019ef10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60082/0 (socket says 172.21.15.5:60082) 2024-09-10T09:45:06.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:06.657+0000 7f4645200640 1 -- 172.21.15.5:0/2107350932 learned_addr learned my addr 172.21.15.5:0/2107350932 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:45:06.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:06.657+0000 7f463f400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f46401049c0 0x7f464019ab80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:54298/0 (socket says 172.21.15.5:54298) 2024-09-10T09:45:06.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:06.657+0000 7f463fe00640 1 -- 172.21.15.5:0/2107350932 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f46401049c0 msgr2=0x7f464019ab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:45:06.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:06.657+0000 7f463fe00640 1 --2- 172.21.15.5:0/2107350932 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f46401049c0 0x7f464019ab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:06.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:06.657+0000 7f463fe00640 1 -- 172.21.15.5:0/2107350932 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4640109140 msgr2=0x7f464019ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:45:06.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:06.657+0000 7f463fe00640 1 --2- 172.21.15.5:0/2107350932 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4640109140 0x7f464019ef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:06.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:06.657+0000 7f463fe00640 1 -- 172.21.15.5:0/2107350932 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f464019f5f0 con 0x7f4640104010 2024-09-10T09:45:06.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:06.657+0000 7f4645200640 1 --2- 172.21.15.5:0/2107350932 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4640109140 0x7f464019ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:45:06.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:06.663+0000 7f463fe00640 1 --2- 172.21.15.5:0/2107350932 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4640104010 0x7f464019a640 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f4630005940 tx=0x7f4630004270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:45:06.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:06.663+0000 7f463cc00640 1 -- 172.21.15.5:0/2107350932 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f463003f030 con 0x7f4640104010 2024-09-10T09:45:06.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:06.663+0000 7f4647200640 1 -- 172.21.15.5:0/2107350932 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f464019f880 con 0x7f4640104010 2024-09-10T09:45:06.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:06.663+0000 7f463cc00640 1 -- 172.21.15.5:0/2107350932 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4630003ad0 con 0x7f4640104010 2024-09-10T09:45:06.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:06.663+0000 7f463cc00640 1 -- 172.21.15.5:0/2107350932 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4630035070 con 0x7f4640104010 2024-09-10T09:45:06.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:06.663+0000 7f4647200640 1 -- 172.21.15.5:0/2107350932 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46401a71c0 con 0x7f4640104010 2024-09-10T09:45:06.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:06.667+0000 7f4649e38740 1 Processor -- start 2024-09-10T09:45:06.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:06.667+0000 7f4649e38740 1 -- start start 2024-09-10T09:45:06.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:06.667+0000 7f4649e38740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ccad9b3c70 0x55ccad9b4050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:45:06.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:06.667+0000 7f4649e38740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ccad9b4590 0x55ccad9be910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:45:06.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:06.667+0000 7f4649e38740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ccad9bee50 0x55ccad9c1240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:45:06.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:06.667+0000 7f4649e38740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55ccad91a3e0 con 0x55ccad9bee50 2024-09-10T09:45:06.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:06.667+0000 7f4649e38740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55ccad8c1d80 con 0x55ccad9b4590 2024-09-10T09:45:06.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:06.667+0000 7f4649e38740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55ccad9163a0 con 0x55ccad9b3c70 2024-09-10T09:45:06.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:06.667+0000 7f4645200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ccad9b3c70 0x55ccad9b4050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:45:06.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:06.667+0000 7f4645200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ccad9b3c70 0x55ccad9b4050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:54304/0 (socket says 172.21.15.5:54304) 2024-09-10T09:45:06.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:06.667+0000 7f463cc00640 1 -- 172.21.15.5:0/2107350932 <== mon.2 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f4630042030 con 0x7f4640104010 2024-09-10T09:45:06.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:06.667+0000 7f4645200640 1 -- 172.21.15.5:0/4281293406 learned_addr learned my addr 172.21.15.5:0/4281293406 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:45:06.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:06.667+0000 7f463f400640 1 --2- 172.21.15.5:0/4281293406 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ccad9bee50 0x55ccad9c1240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:45:06.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:06.667+0000 7f4645200640 1 -- 172.21.15.5:0/4281293406 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ccad9b4590 msgr2=0x55ccad9be910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:45:06.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:06.667+0000 7f4645200640 1 --2- 172.21.15.5:0/4281293406 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ccad9b4590 0x55ccad9be910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:06.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:06.667+0000 7f4645200640 1 -- 172.21.15.5:0/4281293406 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ccad9bee50 msgr2=0x55ccad9c1240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:45:06.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:06.667+0000 7f4645200640 1 --2- 172.21.15.5:0/4281293406 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ccad9bee50 0x55ccad9c1240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:06.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:06.667+0000 7f4645200640 1 -- 172.21.15.5:0/4281293406 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ccad8e0ae0 con 0x55ccad9b3c70 2024-09-10T09:45:06.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:06.667+0000 7f4645200640 1 --2- 172.21.15.5:0/4281293406 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ccad9b3c70 0x55ccad9b4050 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f4638010810 tx=0x7f4638010cd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8095e580dfc76815 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:45:06.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:06.667+0000 7f4635600640 1 -- 172.21.15.5:0/4281293406 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f463800eb30 con 0x55ccad9b3c70 2024-09-10T09:45:06.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:06.667+0000 7f4635600640 1 -- 172.21.15.5:0/4281293406 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4638015070 con 0x55ccad9b3c70 2024-09-10T09:45:06.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:06.667+0000 7f4635600640 1 -- 172.21.15.5:0/4281293406 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46380116a0 con 0x55ccad9b3c70 2024-09-10T09:45:06.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:06.667+0000 7f463cc00640 1 --2- 172.21.15.5:0/2107350932 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f46240a2c00 0x7f46240a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:45:06.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:06.667+0000 7f463cc00640 1 -- 172.21.15.5:0/2107350932 <== mon.2 v2:172.21.15.102:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f46300ef990 con 0x7f4640104010 2024-09-10T09:45:06.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:06.667+0000 7f4649e38740 1 -- 172.21.15.5:0/4281293406 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ccad9b3c70 msgr2=0x55ccad9b4050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:45:06.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:06.667+0000 7f4649e38740 1 --2- 172.21.15.5:0/4281293406 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ccad9b3c70 0x55ccad9b4050 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f4638010810 tx=0x7f4638010cd0 comp rx=0 tx=0).stop 2024-09-10T09:45:06.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:06.667+0000 7f463fe00640 1 --2- 172.21.15.5:0/2107350932 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f46240a2c00 0x7f46240a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:45:06.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:06.667+0000 7f4649e38740 1 -- 172.21.15.5:0/4281293406 shutdown_connections 2024-09-10T09:45:06.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:06.667+0000 7f4649e38740 1 --2- 172.21.15.5:0/4281293406 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ccad9bee50 0x55ccad9c1240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:06.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:06.667+0000 7f4649e38740 1 --2- 172.21.15.5:0/4281293406 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ccad9b4590 0x55ccad9be910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:06.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:06.667+0000 7f4649e38740 1 --2- 172.21.15.5:0/4281293406 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ccad9b3c70 0x55ccad9b4050 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:06.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:06.667+0000 7f4649e38740 1 -- 172.21.15.5:0/4281293406 >> 172.21.15.5:0/4281293406 conn(0x55ccad9b0200 msgr2=0x55ccad9b05e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:45:06.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:06.667+0000 7f4649e38740 1 -- 172.21.15.5:0/4281293406 shutdown_connections 2024-09-10T09:45:06.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:06.667+0000 7f463fe00640 1 --2- 172.21.15.5:0/2107350932 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f46240a2c00 0x7f46240a50c0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f4630006010 tx=0x7f46300efc60 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:45:06.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:06.667+0000 7f4649e38740 1 -- 172.21.15.5:0/4281293406 wait complete. 2024-09-10T09:45:06.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:06.672+0000 7f4649e38740 1 -- start start 2024-09-10T09:45:06.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:06.672+0000 7f4649e38740 1 Processor -- start 2024-09-10T09:45:06.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:06.672+0000 7f4649e38740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ccad9b3c70 0x55ccad9bbda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:45:06.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:06.672+0000 7f4649e38740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ccad9b4590 0x55ccad9bc2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:45:06.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:06.672+0000 7f4649e38740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ccad9bee50 0x55ccad9bc820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:45:06.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:06.672+0000 7f4649e38740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55ccad9c1a20 con 0x55ccad9b3c70 2024-09-10T09:45:06.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:06.672+0000 7f4649e38740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55ccad8d50f0 con 0x55ccad9b4590 2024-09-10T09:45:06.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:06.672+0000 7f4649e38740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55ccad9c1ba0 con 0x55ccad9bee50 2024-09-10T09:45:06.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:06.672+0000 7f4649e38740 10 client.0 _ll_register_callbacks cb 0x55ccad8bcbd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:45:06.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:06.672+0000 7f4645200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ccad9b3c70 0x55ccad9bbda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:45:06.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:06.672+0000 7f463f400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ccad9b4590 0x55ccad9bc2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:45:06.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:06.672+0000 7f463f400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ccad9b4590 0x55ccad9bc2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:45276/0 (socket says 172.21.15.5:45276) 2024-09-10T09:45:06.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:06.672+0000 7f463f400640 1 -- 172.21.15.5:0/2545380675 learned_addr learned my addr 172.21.15.5:0/2545380675 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:45:06.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:06.672+0000 7f4645200640 1 --2- 172.21.15.5:0/2545380675 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ccad9bee50 0x55ccad9bc820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:45:06.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:06.672+0000 7f463f400640 1 -- 172.21.15.5:0/2545380675 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ccad9bee50 msgr2=0x55ccad9bc820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:45:06.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:06.672+0000 7f463f400640 1 --2- 172.21.15.5:0/2545380675 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ccad9bee50 0x55ccad9bc820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:06.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:06.672+0000 7f463f400640 1 -- 172.21.15.5:0/2545380675 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ccad9b3c70 msgr2=0x55ccad9bbda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:45:06.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:06.672+0000 7f463f400640 1 --2- 172.21.15.5:0/2545380675 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ccad9b3c70 0x55ccad9bbda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:06.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:06.672+0000 7f463f400640 1 -- 172.21.15.5:0/2545380675 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ccad68b3c0 con 0x55ccad9b4590 2024-09-10T09:45:06.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:06.672+0000 7f4645200640 1 --2- 172.21.15.5:0/2545380675 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ccad9bee50 0x55ccad9bc820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T09:45:06.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:06.672+0000 7f463f400640 1 --2- 172.21.15.5:0/2545380675 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ccad9b4590 0x55ccad9bc2e0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f462c012730 tx=0x7f462c012bf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=644af8cc5afdfe16 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:45:06.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:06.672+0000 7f462a000640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-09-10T09:45:06.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:06.672+0000 7f462a000640 1 -- 172.21.15.5:0/2545380675 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f462c004050 con 0x55ccad9b4590 2024-09-10T09:45:06.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:06.672+0000 7f4649e38740 1 -- 172.21.15.5:0/2545380675 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4638015070 con 0x55ccad9b4590 2024-09-10T09:45:06.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:06.672+0000 7f462a000640 1 -- 172.21.15.5:0/2545380675 <== mon.2 v2:172.21.15.102:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f462c0041f0 con 0x55ccad9b4590 2024-09-10T09:45:06.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:06.672+0000 7f462a000640 1 -- 172.21.15.5:0/2545380675 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f462c025050 con 0x55ccad9b4590 2024-09-10T09:45:06.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:06.672+0000 7f4649e38740 10 client.26714 fetch_fsmap learned FSMap version 34 2024-09-10T09:45:06.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:06.672+0000 7f4649e38740 1 -- 172.21.15.5:0/2545380675 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55ccad9cdba0 con 0x55ccad9b4590 2024-09-10T09:45:06.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:06.672+0000 7f462a000640 1 -- 172.21.15.5:0/2545380675 <== mon.2 v2:172.21.15.102:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4638015070 con 0x55ccad9b4590 2024-09-10T09:45:06.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:06.672+0000 7f462a000640 1 -- 172.21.15.5:0/2545380675 <== mon.2 v2:172.21.15.102:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f462c0039f0 con 0x55ccad9b4590 2024-09-10T09:45:06.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:06.672+0000 7f4649e38740 10 client.26714 fetch_fsmap finished waiting for FSMap version 34 2024-09-10T09:45:06.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:06.672+0000 7f4649e38740 10 client.26714 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-09-10T09:45:06.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:06.672+0000 7f4649e38740 20 client.26714 populate_metadata read hostname 'smithi005' 2024-09-10T09:45:06.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:06.672+0000 7f4649e38740 1 --2- 172.21.15.5:0/2545380675 >> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] conn(0x55ccad9cefb0 0x55ccad9d13a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:45:06.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:06.672+0000 7f4649e38740 4 client.26714 mds_command: new command op to 24443 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:45:06.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:06.672+0000 7f4649e38740 1 -- 172.21.15.5:0/2545380675 --> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ccad927950 con 0x55ccad9cefb0 2024-09-10T09:45:06.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:06.672+0000 7f4645200640 1 --2- 172.21.15.5:0/2545380675 >> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] conn(0x55ccad9cefb0 0x55ccad9d13a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:45:06.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:06.672+0000 7f4645200640 1 --2- 172.21.15.5:0/2545380675 >> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] conn(0x55ccad9cefb0 0x55ccad9d13a0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=a4a3cfdf6abe7a1 server_cookie=e0501871d3ad60e in_seq=0 out_seq=0 2024-09-10T09:45:06.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:06.672+0000 7f462a000640 10 client.26714 ms_handle_connect on v2:172.21.15.102:6832/2457007427 2024-09-10T09:45:06.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:06.677+0000 7f462a000640 1 -- 172.21.15.5:0/2545380675 <== mds.4 v2:172.21.15.102:6832/2457007427 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ccad927950 con 0x55ccad9cefb0 2024-09-10T09:45:06.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:06.677+0000 7f462a000640 10 client.26714 handle_command_reply: tid=0 2024-09-10T09:45:06.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:06.677+0000 7f4649e38740 1 client.26714 shutdown 2024-09-10T09:45:06.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:06.677+0000 7f4649e38740 1 -- 172.21.15.5:0/2545380675 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ccad9b4590 msgr2=0x55ccad9bc2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:45:06.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:06.677+0000 7f4649e38740 1 --2- 172.21.15.5:0/2545380675 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ccad9b4590 0x55ccad9bc2e0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f462c012730 tx=0x7f462c012bf0 comp rx=0 tx=0).stop 2024-09-10T09:45:06.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:06.677+0000 7f4649e38740 1 -- 172.21.15.5:0/2545380675 shutdown_connections 2024-09-10T09:45:06.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:06.677+0000 7f4649e38740 1 --2- 172.21.15.5:0/2545380675 >> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] conn(0x55ccad9cefb0 0x55ccad9d13a0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:06.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:06.677+0000 7f4649e38740 1 --2- 172.21.15.5:0/2545380675 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ccad9bee50 0x55ccad9bc820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:06.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:06.677+0000 7f4649e38740 1 --2- 172.21.15.5:0/2545380675 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ccad9b4590 0x55ccad9bc2e0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:06.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:06.677+0000 7f4649e38740 1 --2- 172.21.15.5:0/2545380675 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ccad9b3c70 0x55ccad9bbda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:06.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:06.677+0000 7f462a000640 0 client.26714 ms_handle_reset on v2:172.21.15.102:6832/2457007427 2024-09-10T09:45:06.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:06.677+0000 7f4649e38740 1 -- 172.21.15.5:0/2545380675 >> 172.21.15.5:0/2545380675 conn(0x55ccad9b0200 msgr2=0x55ccad9b5ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:45:06.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:06.677+0000 7f4649e38740 1 -- 172.21.15.5:0/2545380675 shutdown_connections 2024-09-10T09:45:06.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:06.677+0000 7f4649e38740 1 -- 172.21.15.5:0/2545380675 wait complete. 2024-09-10T09:45:06.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:06.677+0000 7f4649e38740 20 client.26714 trim_cache size 0 max 16384 2024-09-10T09:45:06.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:06.690+0000 7f4649e38740 1 Processor -- start 2024-09-10T09:45:06.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:06.690+0000 7f4649e38740 1 -- start start 2024-09-10T09:45:06.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:06.690+0000 7f4649e38740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ccad9b3c70 0x55ccad9b66d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:45:06.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:06.690+0000 7f4649e38740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ccad9b4590 0x55ccad9af270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:45:06.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:06.690+0000 7f4649e38740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ccad9bee50 0x55ccad9d6cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:45:06.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:06.690+0000 7f4649e38740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f46401b1080 con 0x55ccad9b3c70 2024-09-10T09:45:06.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:06.690+0000 7f4649e38740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f46401b2bc0 con 0x55ccad9b4590 2024-09-10T09:45:06.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:06.690+0000 7f4649e38740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f46401b1750 con 0x55ccad9bee50 2024-09-10T09:45:06.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:06.690+0000 7f4645200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ccad9b3c70 0x55ccad9b66d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:45:06.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:06.690+0000 7f4645200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ccad9b3c70 0x55ccad9b66d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60098/0 (socket says 172.21.15.5:60098) 2024-09-10T09:45:06.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:06.690+0000 7f4645200640 1 -- 172.21.15.5:0/4193393477 learned_addr learned my addr 172.21.15.5:0/4193393477 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:45:06.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:06.690+0000 7f463f400640 1 --2- 172.21.15.5:0/4193393477 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ccad9b4590 0x55ccad9af270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:45:06.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:06.690+0000 7f4645200640 1 --2- 172.21.15.5:0/4193393477 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ccad9bee50 0x55ccad9d6cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:45:06.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:06.690+0000 7f4645200640 1 -- 172.21.15.5:0/4193393477 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ccad9bee50 msgr2=0x55ccad9d6cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:45:06.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:06.690+0000 7f4645200640 1 --2- 172.21.15.5:0/4193393477 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ccad9bee50 0x55ccad9d6cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:06.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:06.690+0000 7f4645200640 1 -- 172.21.15.5:0/4193393477 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ccad9b4590 msgr2=0x55ccad9af270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:45:06.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:06.690+0000 7f4645200640 1 --2- 172.21.15.5:0/4193393477 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ccad9b4590 0x55ccad9af270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:06.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:06.690+0000 7f4645200640 1 -- 172.21.15.5:0/4193393477 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ccad9b3aa0 con 0x55ccad9b3c70 2024-09-10T09:45:06.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:06.690+0000 7f4645200640 1 --2- 172.21.15.5:0/4193393477 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ccad9b3c70 0x55ccad9b66d0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f463800e720 tx=0x7f463801e040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48282a3a69397506 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:45:06.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:06.690+0000 7f4634c00640 1 -- 172.21.15.5:0/4193393477 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f463800eb90 con 0x55ccad9b3c70 2024-09-10T09:45:06.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:06.690+0000 7f4634c00640 1 -- 172.21.15.5:0/4193393477 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4638015040 con 0x55ccad9b3c70 2024-09-10T09:45:06.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:06.690+0000 7f4634c00640 1 -- 172.21.15.5:0/4193393477 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f463801a840 con 0x55ccad9b3c70 2024-09-10T09:45:06.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:06.690+0000 7f4649e38740 1 -- 172.21.15.5:0/4193393477 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ccad9b3c70 msgr2=0x55ccad9b66d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:45:06.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:06.690+0000 7f4649e38740 1 --2- 172.21.15.5:0/4193393477 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ccad9b3c70 0x55ccad9b66d0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f463800e720 tx=0x7f463801e040 comp rx=0 tx=0).stop 2024-09-10T09:45:06.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:06.690+0000 7f4649e38740 1 -- 172.21.15.5:0/4193393477 shutdown_connections 2024-09-10T09:45:06.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:06.690+0000 7f4649e38740 1 --2- 172.21.15.5:0/4193393477 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ccad9bee50 0x55ccad9d6cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:06.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:06.690+0000 7f4649e38740 1 --2- 172.21.15.5:0/4193393477 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ccad9b4590 0x55ccad9af270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:06.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:06.690+0000 7f4649e38740 1 --2- 172.21.15.5:0/4193393477 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ccad9b3c70 0x55ccad9b66d0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:06.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:06.690+0000 7f4649e38740 1 -- 172.21.15.5:0/4193393477 >> 172.21.15.5:0/4193393477 conn(0x55ccad9b0200 msgr2=0x55ccad9aeb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:45:06.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:06.690+0000 7f4649e38740 1 -- 172.21.15.5:0/4193393477 shutdown_connections 2024-09-10T09:45:06.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:06.690+0000 7f4649e38740 1 -- 172.21.15.5:0/4193393477 wait complete. 2024-09-10T09:45:06.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:06.690+0000 7f4649e38740 1 -- start start 2024-09-10T09:45:06.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:06.690+0000 7f4649e38740 1 Processor -- start 2024-09-10T09:45:06.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:06.690+0000 7f4649e38740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ccad9b3c70 0x55ccad9c7000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:45:06.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:06.690+0000 7f4649e38740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ccad9b4590 0x55ccad9c7540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:45:06.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:06.690+0000 7f4649e38740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ccad9bee50 0x55ccad9c3df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:45:06.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:06.690+0000 7f4649e38740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f46401b24f0 con 0x55ccad9b3c70 2024-09-10T09:45:06.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:06.690+0000 7f4649e38740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f46401b1e20 con 0x55ccad9b4590 2024-09-10T09:45:06.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:06.690+0000 7f4649e38740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f46401af060 con 0x55ccad9bee50 2024-09-10T09:45:06.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:06.690+0000 7f4649e38740 10 client.0 _ll_register_callbacks cb 0x7f462c0097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:45:06.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:06.696+0000 7f4645200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ccad9b3c70 0x55ccad9c7000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:45:06.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:06.696+0000 7f4645200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ccad9b3c70 0x55ccad9c7000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60112/0 (socket says 172.21.15.5:60112) 2024-09-10T09:45:06.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:06.696+0000 7f4645200640 1 -- 172.21.15.5:0/2125790632 learned_addr learned my addr 172.21.15.5:0/2125790632 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:45:06.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:06.696+0000 7f463f400640 1 --2- 172.21.15.5:0/2125790632 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ccad9b4590 0x55ccad9c7540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:45:06.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:06.696+0000 7f4645200640 1 -- 172.21.15.5:0/2125790632 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ccad9bee50 msgr2=0x55ccad9c3df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:45:06.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:06.696+0000 7f4645200640 1 --2- 172.21.15.5:0/2125790632 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ccad9bee50 0x55ccad9c3df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:06.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:06.696+0000 7f4645200640 1 -- 172.21.15.5:0/2125790632 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ccad9b4590 msgr2=0x55ccad9c7540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:45:06.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:06.696+0000 7f4645200640 1 --2- 172.21.15.5:0/2125790632 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ccad9b4590 0x55ccad9c7540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:06.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:06.696+0000 7f4645200640 1 -- 172.21.15.5:0/2125790632 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ccad9ab770 con 0x55ccad9b3c70 2024-09-10T09:45:06.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:06.696+0000 7f4645200640 1 --2- 172.21.15.5:0/2125790632 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ccad9b3c70 0x55ccad9c7000 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f4638009f10 tx=0x7f4638009330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe902ab0db88c16b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:45:06.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:06.696+0000 7f462a000640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-10T09:45:06.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:06.696+0000 7f462a000640 1 -- 172.21.15.5:0/2125790632 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4638037030 con 0x55ccad9b3c70 2024-09-10T09:45:06.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:06.696+0000 7f462a000640 1 -- 172.21.15.5:0/2125790632 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f4638024040 con 0x55ccad9b3c70 2024-09-10T09:45:06.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:06.696+0000 7f4649e38740 1 -- 172.21.15.5:0/2125790632 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4638015040 con 0x55ccad9b3c70 2024-09-10T09:45:06.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:06.696+0000 7f462a000640 1 -- 172.21.15.5:0/2125790632 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f463800ee80 con 0x55ccad9b3c70 2024-09-10T09:45:06.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:06.696+0000 7f4649e38740 10 client.16704 fetch_fsmap learned FSMap version 34 2024-09-10T09:45:06.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:06.696+0000 7f4649e38740 1 -- 172.21.15.5:0/2125790632 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55ccada3a450 con 0x55ccad9b3c70 2024-09-10T09:45:06.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:06.696+0000 7f462a000640 1 -- 172.21.15.5:0/2125790632 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4638015040 con 0x55ccad9b3c70 2024-09-10T09:45:06.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:06.696+0000 7f462a000640 1 -- 172.21.15.5:0/2125790632 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f4638011a50 con 0x55ccad9b3c70 2024-09-10T09:45:06.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:06.696+0000 7f4649e38740 10 client.16704 fetch_fsmap finished waiting for FSMap version 34 2024-09-10T09:45:06.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:06.696+0000 7f4649e38740 10 client.16704 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-09-10T09:45:06.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:06.696+0000 7f4649e38740 20 client.16704 populate_metadata read hostname 'smithi005' 2024-09-10T09:45:06.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:06.696+0000 7f4649e38740 1 --2- 172.21.15.5:0/2125790632 >> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] conn(0x55ccada3aa00 0x55ccada3ce20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:45:06.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:06.696+0000 7f4649e38740 4 client.16704 mds_command: new command op to 24443 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T09:45:06.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:06.696+0000 7f4649e38740 1 -- 172.21.15.5:0/2125790632 --> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ccad921e60 con 0x55ccada3aa00 2024-09-10T09:45:06.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:06.696+0000 7f463f400640 1 --2- 172.21.15.5:0/2125790632 >> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] conn(0x55ccada3aa00 0x55ccada3ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:45:06.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:06.696+0000 7f463f400640 1 --2- 172.21.15.5:0/2125790632 >> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] conn(0x55ccada3aa00 0x55ccada3ce20 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=5ef3be4b2cd70c52 server_cookie=71b9b51467599253 in_seq=0 out_seq=0 2024-09-10T09:45:06.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:06.696+0000 7f462a000640 10 client.16704 ms_handle_connect on v2:172.21.15.102:6832/2457007427 2024-09-10T09:45:06.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:06.696+0000 7f462a000640 1 -- 172.21.15.5:0/2125790632 <== mds.4 v2:172.21.15.102:6832/2457007427 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ccad921e60 con 0x55ccada3aa00 2024-09-10T09:45:06.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:06.696+0000 7f462a000640 10 client.16704 handle_command_reply: tid=0 2024-09-10T09:45:06.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:06.696+0000 7f4649e38740 1 client.16704 shutdown 2024-09-10T09:45:06.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:06.696+0000 7f4649e38740 1 -- 172.21.15.5:0/2125790632 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ccad9b3c70 msgr2=0x55ccad9c7000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:45:06.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:06.696+0000 7f4649e38740 1 --2- 172.21.15.5:0/2125790632 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ccad9b3c70 0x55ccad9c7000 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f4638009f10 tx=0x7f4638009330 comp rx=0 tx=0).stop 2024-09-10T09:45:06.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:06.696+0000 7f4649e38740 1 -- 172.21.15.5:0/2125790632 shutdown_connections 2024-09-10T09:45:06.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:06.696+0000 7f4649e38740 1 --2- 172.21.15.5:0/2125790632 >> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] conn(0x55ccada3aa00 0x55ccada3ce20 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:06.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:06.696+0000 7f4649e38740 1 --2- 172.21.15.5:0/2125790632 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ccad9bee50 0x55ccad9c3df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:06.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:06.696+0000 7f4649e38740 1 --2- 172.21.15.5:0/2125790632 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ccad9b4590 0x55ccad9c7540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:06.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:06.696+0000 7f4649e38740 1 --2- 172.21.15.5:0/2125790632 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ccad9b3c70 0x55ccad9c7000 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:06.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:06.696+0000 7f4649e38740 1 -- 172.21.15.5:0/2125790632 >> 172.21.15.5:0/2125790632 conn(0x55ccad9b0200 msgr2=0x55ccad9af380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:45:06.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:06.696+0000 7f462a000640 0 client.16704 ms_handle_reset on v2:172.21.15.102:6832/2457007427 2024-09-10T09:45:06.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:06.696+0000 7f4649e38740 1 -- 172.21.15.5:0/2125790632 shutdown_connections 2024-09-10T09:45:06.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:06.696+0000 7f4649e38740 1 -- 172.21.15.5:0/2125790632 wait complete. 2024-09-10T09:45:06.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:06.696+0000 7f4649e38740 20 client.16704 trim_cache size 0 max 16384 2024-09-10T09:45:06.704 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-10T09:45:06.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:06.701+0000 7f4647200640 1 -- 172.21.15.5:0/2107350932 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f46240a2c00 msgr2=0x7f46240a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:45:06.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:06.701+0000 7f4647200640 1 --2- 172.21.15.5:0/2107350932 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f46240a2c00 0x7f46240a50c0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f4630006010 tx=0x7f46300efc60 comp rx=0 tx=0).stop 2024-09-10T09:45:06.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:06.701+0000 7f4647200640 1 -- 172.21.15.5:0/2107350932 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4640104010 msgr2=0x7f464019a640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:45:06.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:06.701+0000 7f4647200640 1 --2- 172.21.15.5:0/2107350932 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4640104010 0x7f464019a640 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f4630005940 tx=0x7f4630004270 comp rx=0 tx=0).stop 2024-09-10T09:45:06.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:06.701+0000 7f4647200640 1 -- 172.21.15.5:0/2107350932 shutdown_connections 2024-09-10T09:45:06.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:06.701+0000 7f4647200640 1 --2- 172.21.15.5:0/2107350932 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f46240a2c00 0x7f46240a50c0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:06.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:06.701+0000 7f4647200640 1 --2- 172.21.15.5:0/2107350932 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4640109140 0x7f464019ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:06.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:06.701+0000 7f4647200640 1 --2- 172.21.15.5:0/2107350932 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f46401049c0 0x7f464019ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:06.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:06.701+0000 7f4647200640 1 --2- 172.21.15.5:0/2107350932 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4640104010 0x7f464019a640 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:06.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:06.701+0000 7f4647200640 1 -- 172.21.15.5:0/2107350932 >> 172.21.15.5:0/2107350932 conn(0x7f46400ffa50 msgr2=0x7f4640100e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:45:06.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:06.701+0000 7f4647200640 1 -- 172.21.15.5:0/2107350932 shutdown_connections 2024-09-10T09:45:06.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:06.701+0000 7f4647200640 1 -- 172.21.15.5:0/2107350932 wait complete. 2024-09-10T09:45:07.055 INFO:tasks.workunit.client.0.smithi005.stdout: 1 359745 5.38 MB/sec execute 168 sec latency 7414.031 ms 2024-09-10T09:45:07.165 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:45:06 smithi005 ceph-mon[30350]: pgmap v596: 129 pgs: 129 active+clean; 145 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.6 MiB/s wr, 113 op/s 2024-09-10T09:45:07.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:45:06 smithi159 ceph-mon[35513]: pgmap v596: 129 pgs: 129 active+clean; 145 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.6 MiB/s wr, 113 op/s 2024-09-10T09:45:07.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:45:06 smithi102 ceph-mon[35442]: pgmap v596: 129 pgs: 129 active+clean; 145 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.6 MiB/s wr, 113 op/s 2024-09-10T09:45:07.966 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-10T09:45:08.055 INFO:tasks.workunit.client.0.smithi005.stdout: 1 359745 5.35 MB/sec execute 169 sec latency 8414.093 ms 2024-09-10T09:45:08.344 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:45:08.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:08.871+0000 7fba5e200640 1 -- 172.21.15.5:0/2225659333 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fba58103e30 msgr2=0x7fba58104210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:45:08.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:08.871+0000 7fba5e200640 1 --2- 172.21.15.5:0/2225659333 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fba58103e30 0x7fba58104210 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fba4c009860 tx=0x7fba4c02a340 comp rx=0 tx=0).stop 2024-09-10T09:45:08.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:08.871+0000 7fba5e200640 1 -- 172.21.15.5:0/2225659333 shutdown_connections 2024-09-10T09:45:08.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:08.871+0000 7fba5e200640 1 --2- 172.21.15.5:0/2225659333 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fba58108f60 0x7fba58110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:08.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:08.871+0000 7fba5e200640 1 --2- 172.21.15.5:0/2225659333 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fba581047e0 0x7fba58108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:08.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:08.871+0000 7fba5e200640 1 --2- 172.21.15.5:0/2225659333 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fba58103e30 0x7fba58104210 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:08.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:08.871+0000 7fba5e200640 1 -- 172.21.15.5:0/2225659333 >> 172.21.15.5:0/2225659333 conn(0x7fba580ff8d0 msgr2=0x7fba58101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:45:08.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:08.871+0000 7fba5e200640 1 -- 172.21.15.5:0/2225659333 shutdown_connections 2024-09-10T09:45:08.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:08.871+0000 7fba5e200640 1 -- 172.21.15.5:0/2225659333 wait complete. 2024-09-10T09:45:08.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:08.873+0000 7fba5e200640 1 Processor -- start 2024-09-10T09:45:08.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:08.873+0000 7fba5e200640 1 -- start start 2024-09-10T09:45:08.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:08.873+0000 7fba5e200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fba58103e30 0x7fba5819a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:45:08.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:08.873+0000 7fba5e200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fba581047e0 0x7fba5819a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:45:08.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:08.873+0000 7fba5e200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fba58108f60 0x7fba5819ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:45:08.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:08.873+0000 7fba5e200640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fba58113fc0 con 0x7fba58103e30 2024-09-10T09:45:08.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:08.873+0000 7fba5e200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fba58113e40 con 0x7fba58108f60 2024-09-10T09:45:08.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:08.873+0000 7fba5e200640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fba58114140 con 0x7fba581047e0 2024-09-10T09:45:08.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:08.874+0000 7fba5ce00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fba58103e30 0x7fba5819a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:45:08.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:08.874+0000 7fba5ce00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fba58103e30 0x7fba5819a460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60144/0 (socket says 172.21.15.5:60144) 2024-09-10T09:45:08.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:08.874+0000 7fba5d800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fba58108f60 0x7fba5819ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:45:08.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:08.874+0000 7fba5ce00640 1 -- 172.21.15.5:0/1427474226 learned_addr learned my addr 172.21.15.5:0/1427474226 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:45:08.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:08.874+0000 7fba57e00640 1 --2- 172.21.15.5:0/1427474226 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fba581047e0 0x7fba5819a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:45:08.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:08.874+0000 7fba5ce00640 1 -- 172.21.15.5:0/1427474226 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fba581047e0 msgr2=0x7fba5819a9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:45:08.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:08.874+0000 7fba5ce00640 1 --2- 172.21.15.5:0/1427474226 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fba581047e0 0x7fba5819a9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:08.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:08.874+0000 7fba5ce00640 1 -- 172.21.15.5:0/1427474226 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fba58108f60 msgr2=0x7fba5819ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:45:08.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:08.874+0000 7fba5ce00640 1 --2- 172.21.15.5:0/1427474226 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fba58108f60 0x7fba5819ed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:08.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:08.874+0000 7fba5ce00640 1 -- 172.21.15.5:0/1427474226 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba5819f410 con 0x7fba58103e30 2024-09-10T09:45:08.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:08.874+0000 7fba5ce00640 1 --2- 172.21.15.5:0/1427474226 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fba58103e30 0x7fba5819a460 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fba4c035040 tx=0x7fba4c033b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:45:08.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:08.875+0000 7fba55600640 1 -- 172.21.15.5:0/1427474226 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba4c037040 con 0x7fba58103e30 2024-09-10T09:45:08.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:08.875+0000 7fba55600640 1 -- 172.21.15.5:0/1427474226 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fba4c040070 con 0x7fba58103e30 2024-09-10T09:45:08.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:08.875+0000 7fba55600640 1 -- 172.21.15.5:0/1427474226 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba4c0039b0 con 0x7fba58103e30 2024-09-10T09:45:08.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:08.875+0000 7fba5e200640 1 -- 172.21.15.5:0/1427474226 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba5819f6a0 con 0x7fba58103e30 2024-09-10T09:45:08.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:08.875+0000 7fba5e200640 1 -- 172.21.15.5:0/1427474226 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba5819fb30 con 0x7fba58103e30 2024-09-10T09:45:08.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:08.879+0000 7fba62466740 1 Processor -- start 2024-09-10T09:45:08.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:08.879+0000 7fba62466740 1 -- start start 2024-09-10T09:45:08.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:08.879+0000 7fba55600640 1 -- 172.21.15.5:0/1427474226 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7fba4c0305e0 con 0x7fba58103e30 2024-09-10T09:45:08.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:08.880+0000 7fba62466740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c272b60e40 0x55c272b61220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:45:08.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:08.880+0000 7fba62466740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c272b61760 0x55c272b6bae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:45:08.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:08.880+0000 7fba62466740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c272b6c020 0x55c272b6e410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:45:08.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:08.880+0000 7fba62466740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55c272ac73e0 con 0x55c272b6c020 2024-09-10T09:45:08.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:08.880+0000 7fba62466740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55c272a6ed80 con 0x55c272b60e40 2024-09-10T09:45:08.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:08.880+0000 7fba62466740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55c272ac33a0 con 0x55c272b61760 2024-09-10T09:45:08.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:08.880+0000 7fba5d800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c272b60e40 0x55c272b61220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:45:08.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:08.880+0000 7fba5d800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c272b60e40 0x55c272b61220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:45338/0 (socket says 172.21.15.5:45338) 2024-09-10T09:45:08.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:08.880+0000 7fba5d800640 1 -- 172.21.15.5:0/3308274968 learned_addr learned my addr 172.21.15.5:0/3308274968 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:45:08.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:08.880+0000 7fba5ce00640 1 --2- 172.21.15.5:0/3308274968 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c272b61760 0x55c272b6bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:45:08.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:08.880+0000 7fba57e00640 1 --2- 172.21.15.5:0/3308274968 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c272b6c020 0x55c272b6e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:45:08.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:08.880+0000 7fba5d800640 1 -- 172.21.15.5:0/3308274968 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c272b61760 msgr2=0x55c272b6bae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:45:08.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:08.880+0000 7fba5d800640 1 --2- 172.21.15.5:0/3308274968 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c272b61760 0x55c272b6bae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:08.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:08.880+0000 7fba5d800640 1 -- 172.21.15.5:0/3308274968 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c272b6c020 msgr2=0x55c272b6e410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:45:08.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:08.880+0000 7fba5d800640 1 --2- 172.21.15.5:0/3308274968 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c272b6c020 0x55c272b6e410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:08.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:08.880+0000 7fba5d800640 1 -- 172.21.15.5:0/3308274968 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c272a8dae0 con 0x55c272b60e40 2024-09-10T09:45:08.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:08.880+0000 7fba57e00640 1 --2- 172.21.15.5:0/3308274968 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c272b6c020 0x55c272b6e410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:45:08.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:08.881+0000 7fba5d800640 1 --2- 172.21.15.5:0/3308274968 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c272b60e40 0x55c272b61220 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fba50009f50 tx=0x7fba500118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e781077772220ea5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:45:08.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:08.881+0000 7fba4a000640 1 -- 172.21.15.5:0/3308274968 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba50004d20 con 0x55c272b60e40 2024-09-10T09:45:08.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:08.881+0000 7fba4a000640 1 -- 172.21.15.5:0/3308274968 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fba50004050 con 0x55c272b60e40 2024-09-10T09:45:08.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:08.881+0000 7fba4a000640 1 -- 172.21.15.5:0/3308274968 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba50005280 con 0x55c272b60e40 2024-09-10T09:45:08.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:08.881+0000 7fba55600640 1 --2- 172.21.15.5:0/1427474226 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fba400a2c00 0x7fba400a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:45:08.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:08.882+0000 7fba55600640 1 -- 172.21.15.5:0/1427474226 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fba4c0ef350 con 0x7fba58103e30 2024-09-10T09:45:08.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:08.882+0000 7fba5ce00640 1 --2- 172.21.15.5:0/1427474226 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fba400a2c00 0x7fba400a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:45:08.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:08.882+0000 7fba62466740 1 -- 172.21.15.5:0/3308274968 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c272b60e40 msgr2=0x55c272b61220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:45:08.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:08.882+0000 7fba62466740 1 --2- 172.21.15.5:0/3308274968 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c272b60e40 0x55c272b61220 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fba50009f50 tx=0x7fba500118d0 comp rx=0 tx=0).stop 2024-09-10T09:45:08.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:08.882+0000 7fba62466740 1 -- 172.21.15.5:0/3308274968 shutdown_connections 2024-09-10T09:45:08.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:08.882+0000 7fba62466740 1 --2- 172.21.15.5:0/3308274968 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c272b6c020 0x55c272b6e410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:08.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:08.882+0000 7fba62466740 1 --2- 172.21.15.5:0/3308274968 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c272b61760 0x55c272b6bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:08.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:08.882+0000 7fba62466740 1 --2- 172.21.15.5:0/3308274968 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c272b60e40 0x55c272b61220 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:08.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:08.882+0000 7fba62466740 1 -- 172.21.15.5:0/3308274968 >> 172.21.15.5:0/3308274968 conn(0x55c272b5d3d0 msgr2=0x55c272b5d7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:45:08.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:08.883+0000 7fba62466740 1 -- 172.21.15.5:0/3308274968 shutdown_connections 2024-09-10T09:45:08.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:08.883+0000 7fba62466740 1 -- 172.21.15.5:0/3308274968 wait complete. 2024-09-10T09:45:08.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:08.883+0000 7fba5ce00640 1 --2- 172.21.15.5:0/1427474226 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fba400a2c00 0x7fba400a50c0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fba4c0ecdd0 tx=0x7fba4c0f2e30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:45:08.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:08.883+0000 7fba62466740 1 -- start start 2024-09-10T09:45:08.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:08.883+0000 7fba62466740 1 Processor -- start 2024-09-10T09:45:08.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:08.884+0000 7fba62466740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c272b60e40 0x55c272b68eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:45:08.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:08.884+0000 7fba62466740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c272b61760 0x55c272b693f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:45:08.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:08.884+0000 7fba62466740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c272b6c020 0x55c272b69930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:45:08.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:08.884+0000 7fba62466740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55c272b6ebf0 con 0x55c272b61760 2024-09-10T09:45:08.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:08.884+0000 7fba62466740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55c272a820f0 con 0x55c272b60e40 2024-09-10T09:45:08.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:08.884+0000 7fba62466740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55c272b6ed70 con 0x55c272b6c020 2024-09-10T09:45:08.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:08.884+0000 7fba62466740 10 client.0 _ll_register_callbacks cb 0x55c272a69bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:45:08.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:08.884+0000 7fba57e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c272b61760 0x55c272b693f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:45:08.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:08.884+0000 7fba57e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c272b61760 0x55c272b693f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60174/0 (socket says 172.21.15.5:60174) 2024-09-10T09:45:08.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:08.884+0000 7fba57e00640 1 -- 172.21.15.5:0/1509446553 learned_addr learned my addr 172.21.15.5:0/1509446553 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:45:08.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:08.884+0000 7fba5d800640 1 --2- 172.21.15.5:0/1509446553 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c272b60e40 0x55c272b68eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:45:08.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:08.884+0000 7fba5d800640 1 --2- 172.21.15.5:0/1509446553 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c272b6c020 0x55c272b69930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:45:08.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:08.885+0000 7fba57e00640 1 -- 172.21.15.5:0/1509446553 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c272b6c020 msgr2=0x55c272b69930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:45:08.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:08.885+0000 7fba57e00640 1 --2- 172.21.15.5:0/1509446553 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c272b6c020 0x55c272b69930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:08.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:08.885+0000 7fba57e00640 1 -- 172.21.15.5:0/1509446553 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c272b60e40 msgr2=0x55c272b68eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:45:08.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:08.885+0000 7fba57e00640 1 --2- 172.21.15.5:0/1509446553 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c272b60e40 0x55c272b68eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:08.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:08.885+0000 7fba57e00640 1 -- 172.21.15.5:0/1509446553 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c2728383c0 con 0x55c272b61760 2024-09-10T09:45:08.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:08.885+0000 7fba57e00640 1 --2- 172.21.15.5:0/1509446553 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c272b61760 0x55c272b693f0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fba440116f0 tx=0x7fba44011bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc2cc3c9ed245850 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:45:08.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:08.885+0000 7fba3ea00640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-10T09:45:08.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:08.885+0000 7fba3ea00640 1 -- 172.21.15.5:0/1509446553 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba4400fbb0 con 0x55c272b61760 2024-09-10T09:45:08.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:08.885+0000 7fba3ea00640 1 -- 172.21.15.5:0/1509446553 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fba4400fd50 con 0x55c272b61760 2024-09-10T09:45:08.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:08.885+0000 7fba62466740 1 -- 172.21.15.5:0/1509446553 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fba50004050 con 0x55c272b61760 2024-09-10T09:45:08.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:08.886+0000 7fba3ea00640 1 -- 172.21.15.5:0/1509446553 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fba44027050 con 0x55c272b61760 2024-09-10T09:45:08.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:08.886+0000 7fba62466740 10 client.16716 fetch_fsmap learned FSMap version 34 2024-09-10T09:45:08.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:08.886+0000 7fba62466740 1 -- 172.21.15.5:0/1509446553 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55c272b83460 con 0x55c272b61760 2024-09-10T09:45:08.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:08.886+0000 7fba3ea00640 1 -- 172.21.15.5:0/1509446553 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fba50004050 con 0x55c272b61760 2024-09-10T09:45:08.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:08.886+0000 7fba3ea00640 1 -- 172.21.15.5:0/1509446553 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7fba44022030 con 0x55c272b61760 2024-09-10T09:45:08.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:08.886+0000 7fba62466740 10 client.16716 fetch_fsmap finished waiting for FSMap version 34 2024-09-10T09:45:08.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:08.886+0000 7fba62466740 10 client.16716 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-09-10T09:45:08.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:08.886+0000 7fba62466740 20 client.16716 populate_metadata read hostname 'smithi005' 2024-09-10T09:45:08.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:08.886+0000 7fba62466740 1 --2- 172.21.15.5:0/1509446553 >> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] conn(0x55c272b84920 0x55c272b86d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:45:08.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:08.886+0000 7fba62466740 4 client.16716 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:45:08.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:08.886+0000 7fba62466740 1 -- 172.21.15.5:0/1509446553 --> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c272ad4950 con 0x55c272b84920 2024-09-10T09:45:08.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:08.887+0000 7fba5d800640 1 --2- 172.21.15.5:0/1509446553 >> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] conn(0x55c272b84920 0x55c272b86d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:45:08.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:08.887+0000 7fba5d800640 1 --2- 172.21.15.5:0/1509446553 >> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] conn(0x55c272b84920 0x55c272b86d80 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=3e0e6e2d528de49b server_cookie=9ffde3912ad9ac37 in_seq=0 out_seq=0 2024-09-10T09:45:08.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:08.887+0000 7fba3ea00640 10 client.16716 ms_handle_connect on v2:172.21.15.102:6836/2689776764 2024-09-10T09:45:08.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:08.888+0000 7fba3ea00640 1 -- 172.21.15.5:0/1509446553 <== mds.3 v2:172.21.15.102:6836/2689776764 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c272ad4950 con 0x55c272b84920 2024-09-10T09:45:08.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:08.888+0000 7fba3ea00640 10 client.16716 handle_command_reply: tid=0 2024-09-10T09:45:08.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:08.888+0000 7fba62466740 1 client.16716 shutdown 2024-09-10T09:45:08.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:08.888+0000 7fba62466740 1 -- 172.21.15.5:0/1509446553 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c272b61760 msgr2=0x55c272b693f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:45:08.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:08.888+0000 7fba62466740 1 --2- 172.21.15.5:0/1509446553 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c272b61760 0x55c272b693f0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fba440116f0 tx=0x7fba44011bb0 comp rx=0 tx=0).stop 2024-09-10T09:45:08.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:08.889+0000 7fba62466740 1 -- 172.21.15.5:0/1509446553 shutdown_connections 2024-09-10T09:45:08.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:08.889+0000 7fba62466740 1 --2- 172.21.15.5:0/1509446553 >> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] conn(0x55c272b84920 0x55c272b86d80 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:08.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:08.889+0000 7fba62466740 1 --2- 172.21.15.5:0/1509446553 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c272b6c020 0x55c272b69930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:08.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:08.889+0000 7fba62466740 1 --2- 172.21.15.5:0/1509446553 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c272b61760 0x55c272b693f0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:08.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:08.889+0000 7fba62466740 1 --2- 172.21.15.5:0/1509446553 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c272b60e40 0x55c272b68eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:08.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:08.889+0000 7fba62466740 1 -- 172.21.15.5:0/1509446553 >> 172.21.15.5:0/1509446553 conn(0x55c272b5d3d0 msgr2=0x55c272b630b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:45:08.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:08.889+0000 7fba3ea00640 0 client.16716 ms_handle_reset on v2:172.21.15.102:6836/2689776764 2024-09-10T09:45:08.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:08.889+0000 7fba62466740 1 -- 172.21.15.5:0/1509446553 shutdown_connections 2024-09-10T09:45:08.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:08.889+0000 7fba62466740 1 -- 172.21.15.5:0/1509446553 wait complete. 2024-09-10T09:45:08.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:08.889+0000 7fba62466740 20 client.16716 trim_cache size 0 max 16384 2024-09-10T09:45:08.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:08.903+0000 7fba62466740 1 Processor -- start 2024-09-10T09:45:08.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:08.903+0000 7fba62466740 1 -- start start 2024-09-10T09:45:08.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:08.903+0000 7fba62466740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c272b60e40 0x55c272b638a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:45:08.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:08.903+0000 7fba62466740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c272b61760 0x55c272b58300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:45:08.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:08.903+0000 7fba62466740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c272b6c020 0x55c272b8c650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:45:08.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:08.903+0000 7fba62466740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fba581aca50 con 0x55c272b6c020 2024-09-10T09:45:08.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:08.903+0000 7fba62466740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fba581ae590 con 0x55c272b60e40 2024-09-10T09:45:08.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:08.903+0000 7fba62466740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fba581ad120 con 0x55c272b61760 2024-09-10T09:45:08.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:08.904+0000 7fba5d800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c272b6c020 0x55c272b8c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:45:08.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:08.904+0000 7fba57e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c272b61760 0x55c272b58300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:45:08.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:08.904+0000 7fba5d800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c272b6c020 0x55c272b8c650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60188/0 (socket says 172.21.15.5:60188) 2024-09-10T09:45:08.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:08.904+0000 7fba57e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c272b61760 0x55c272b58300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:54394/0 (socket says 172.21.15.5:54394) 2024-09-10T09:45:08.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:08.904+0000 7fba5d800640 1 -- 172.21.15.5:0/3445401611 learned_addr learned my addr 172.21.15.5:0/3445401611 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:45:08.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:08.904+0000 7fba5d800640 1 --2- 172.21.15.5:0/3445401611 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c272b60e40 0x55c272b638a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:45:08.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:08.904+0000 7fba57e00640 1 -- 172.21.15.5:0/3445401611 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c272b60e40 msgr2=0x55c272b638a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:45:08.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:08.904+0000 7fba57e00640 1 --2- 172.21.15.5:0/3445401611 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c272b60e40 0x55c272b638a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:08.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:08.904+0000 7fba57e00640 1 -- 172.21.15.5:0/3445401611 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c272b6c020 msgr2=0x55c272b8c650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:45:08.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:08.904+0000 7fba57e00640 1 --2- 172.21.15.5:0/3445401611 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c272b6c020 0x55c272b8c650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:08.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:08.904+0000 7fba57e00640 1 -- 172.21.15.5:0/3445401611 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c272b67e60 con 0x55c272b61760 2024-09-10T09:45:08.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:08.904+0000 7fba5d800640 1 --2- 172.21.15.5:0/3445401611 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c272b6c020 0x55c272b8c650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:45:08.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:08.904+0000 7fba57e00640 1 --2- 172.21.15.5:0/3445401611 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c272b61760 0x55c272b58300 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fba440195b0 tx=0x7fba44019390 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4945f99f7cf8185d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:45:08.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:08.905+0000 7fba49600640 1 -- 172.21.15.5:0/3445401611 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba4400f790 con 0x55c272b61760 2024-09-10T09:45:08.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:08.905+0000 7fba49600640 1 -- 172.21.15.5:0/3445401611 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fba440197a0 con 0x55c272b61760 2024-09-10T09:45:08.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:08.905+0000 7fba49600640 1 -- 172.21.15.5:0/3445401611 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba44019dd0 con 0x55c272b61760 2024-09-10T09:45:08.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:08.905+0000 7fba62466740 1 -- 172.21.15.5:0/3445401611 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c272b61760 msgr2=0x55c272b58300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:45:08.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:08.905+0000 7fba62466740 1 --2- 172.21.15.5:0/3445401611 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c272b61760 0x55c272b58300 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fba440195b0 tx=0x7fba44019390 comp rx=0 tx=0).stop 2024-09-10T09:45:08.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:08.905+0000 7fba62466740 1 -- 172.21.15.5:0/3445401611 shutdown_connections 2024-09-10T09:45:08.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:08.905+0000 7fba62466740 1 --2- 172.21.15.5:0/3445401611 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c272b6c020 0x55c272b8c650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:08.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:08.905+0000 7fba62466740 1 --2- 172.21.15.5:0/3445401611 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c272b61760 0x55c272b58300 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:08.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:08.905+0000 7fba62466740 1 --2- 172.21.15.5:0/3445401611 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c272b60e40 0x55c272b638a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:08.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:08.905+0000 7fba62466740 1 -- 172.21.15.5:0/3445401611 >> 172.21.15.5:0/3445401611 conn(0x55c272b5d3d0 msgr2=0x55c272b6af30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:45:08.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:08.905+0000 7fba62466740 1 -- 172.21.15.5:0/3445401611 shutdown_connections 2024-09-10T09:45:08.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:08.905+0000 7fba62466740 1 -- 172.21.15.5:0/3445401611 wait complete. 2024-09-10T09:45:08.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:08.906+0000 7fba62466740 1 -- start start 2024-09-10T09:45:08.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:08.906+0000 7fba62466740 1 Processor -- start 2024-09-10T09:45:08.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:08.906+0000 7fba62466740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c272b60e40 0x55c272b786b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:45:08.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:08.906+0000 7fba62466740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c272b61760 0x55c272b78bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:45:08.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:08.907+0000 7fba62466740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c272b6c020 0x55c272b79130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:45:08.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:08.907+0000 7fba62466740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fba581adec0 con 0x55c272b60e40 2024-09-10T09:45:08.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:08.907+0000 7fba62466740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fba581ad7f0 con 0x55c272b61760 2024-09-10T09:45:08.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:08.907+0000 7fba62466740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fba581aaa30 con 0x55c272b6c020 2024-09-10T09:45:08.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:08.907+0000 7fba62466740 10 client.0 _ll_register_callbacks cb 0x7fba44009640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:45:08.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:08.907+0000 7fba5d800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c272b60e40 0x55c272b786b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:45:08.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:08.907+0000 7fba57e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c272b61760 0x55c272b78bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:45:08.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:08.907+0000 7fba5d800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c272b60e40 0x55c272b786b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60194/0 (socket says 172.21.15.5:60194) 2024-09-10T09:45:08.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:08.907+0000 7fba5d800640 1 -- 172.21.15.5:0/700292184 learned_addr learned my addr 172.21.15.5:0/700292184 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:45:08.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:08.907+0000 7fba57e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c272b61760 0x55c272b78bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:45364/0 (socket says 172.21.15.5:45364) 2024-09-10T09:45:08.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:08.907+0000 7fba5d800640 1 --2- 172.21.15.5:0/700292184 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c272b6c020 0x55c272b79130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:45:08.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:08.907+0000 7fba57e00640 1 -- 172.21.15.5:0/700292184 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c272b6c020 msgr2=0x55c272b79130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:45:08.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:08.907+0000 7fba57e00640 1 --2- 172.21.15.5:0/700292184 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c272b6c020 0x55c272b79130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:08.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:08.907+0000 7fba57e00640 1 -- 172.21.15.5:0/700292184 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c272b60e40 msgr2=0x55c272b786b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:45:08.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:08.907+0000 7fba57e00640 1 --2- 172.21.15.5:0/700292184 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c272b60e40 0x55c272b786b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:08.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:08.907+0000 7fba57e00640 1 -- 172.21.15.5:0/700292184 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c272bee6f0 con 0x55c272b61760 2024-09-10T09:45:08.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:08.908+0000 7fba57e00640 1 --2- 172.21.15.5:0/700292184 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c272b61760 0x55c272b78bf0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x55c272b68690 tx=0x7fba44005a30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6912c8497bed35c5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:45:08.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:08.908+0000 7fba3ea00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-09-10T09:45:08.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:08.911+0000 7fba3ea00640 1 -- 172.21.15.5:0/700292184 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba4400fdc0 con 0x55c272b61760 2024-09-10T09:45:08.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:08.911+0000 7fba3ea00640 1 -- 172.21.15.5:0/700292184 <== mon.2 v2:172.21.15.102:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fba44004020 con 0x55c272b61760 2024-09-10T09:45:08.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:08.912+0000 7fba62466740 1 -- 172.21.15.5:0/700292184 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fba440197a0 con 0x55c272b61760 2024-09-10T09:45:08.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:08.912+0000 7fba3ea00640 1 -- 172.21.15.5:0/700292184 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fba44004220 con 0x55c272b61760 2024-09-10T09:45:08.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:08.912+0000 7fba62466740 10 client.26738 fetch_fsmap learned FSMap version 34 2024-09-10T09:45:08.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:08.912+0000 7fba62466740 1 -- 172.21.15.5:0/700292184 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55c272bef9d0 con 0x55c272b61760 2024-09-10T09:45:08.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:08.912+0000 7fba3ea00640 1 -- 172.21.15.5:0/700292184 <== mon.2 v2:172.21.15.102:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fba440197a0 con 0x55c272b61760 2024-09-10T09:45:08.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:08.912+0000 7fba3ea00640 1 -- 172.21.15.5:0/700292184 <== mon.2 v2:172.21.15.102:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7fba44022070 con 0x55c272b61760 2024-09-10T09:45:08.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:08.912+0000 7fba62466740 10 client.26738 fetch_fsmap finished waiting for FSMap version 34 2024-09-10T09:45:08.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:08.912+0000 7fba62466740 10 client.26738 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-09-10T09:45:08.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:08.912+0000 7fba62466740 20 client.26738 populate_metadata read hostname 'smithi005' 2024-09-10T09:45:08.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:08.912+0000 7fba62466740 1 --2- 172.21.15.5:0/700292184 >> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] conn(0x55c272bf0020 0x55c272bf2410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:45:08.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:08.913+0000 7fba62466740 4 client.26738 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T09:45:08.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:08.913+0000 7fba62466740 1 -- 172.21.15.5:0/700292184 --> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c272acee60 con 0x55c272bf0020 2024-09-10T09:45:08.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:08.913+0000 7fba5d800640 1 --2- 172.21.15.5:0/700292184 >> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] conn(0x55c272bf0020 0x55c272bf2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:45:08.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:08.913+0000 7fba5d800640 1 --2- 172.21.15.5:0/700292184 >> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] conn(0x55c272bf0020 0x55c272bf2410 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=7e1e6baa85e52865 server_cookie=3b850503c6053f82 in_seq=0 out_seq=0 2024-09-10T09:45:08.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:08.913+0000 7fba3ea00640 10 client.26738 ms_handle_connect on v2:172.21.15.102:6836/2689776764 2024-09-10T09:45:08.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:08.914+0000 7fba3ea00640 1 -- 172.21.15.5:0/700292184 <== mds.3 v2:172.21.15.102:6836/2689776764 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c272acee60 con 0x55c272bf0020 2024-09-10T09:45:08.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:08.914+0000 7fba3ea00640 10 client.26738 handle_command_reply: tid=0 2024-09-10T09:45:08.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:08.914+0000 7fba62466740 1 client.26738 shutdown 2024-09-10T09:45:08.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:08.914+0000 7fba62466740 1 -- 172.21.15.5:0/700292184 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c272b61760 msgr2=0x55c272b78bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:45:08.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:08.914+0000 7fba62466740 1 --2- 172.21.15.5:0/700292184 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c272b61760 0x55c272b78bf0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x55c272b68690 tx=0x7fba44005a30 comp rx=0 tx=0).stop 2024-09-10T09:45:08.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:08.914+0000 7fba62466740 1 -- 172.21.15.5:0/700292184 shutdown_connections 2024-09-10T09:45:08.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:08.914+0000 7fba62466740 1 --2- 172.21.15.5:0/700292184 >> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] conn(0x55c272bf0020 0x55c272bf2410 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:08.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:08.914+0000 7fba62466740 1 --2- 172.21.15.5:0/700292184 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c272b6c020 0x55c272b79130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:08.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:08.914+0000 7fba62466740 1 --2- 172.21.15.5:0/700292184 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c272b61760 0x55c272b78bf0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:08.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:08.914+0000 7fba62466740 1 --2- 172.21.15.5:0/700292184 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c272b60e40 0x55c272b786b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:08.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:08.914+0000 7fba3ea00640 0 client.26738 ms_handle_reset on v2:172.21.15.102:6836/2689776764 2024-09-10T09:45:08.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:08.914+0000 7fba62466740 1 -- 172.21.15.5:0/700292184 >> 172.21.15.5:0/700292184 conn(0x55c272b5d3d0 msgr2=0x55c272b68e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:45:08.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:08.914+0000 7fba62466740 1 -- 172.21.15.5:0/700292184 shutdown_connections 2024-09-10T09:45:08.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:08.914+0000 7fba62466740 1 -- 172.21.15.5:0/700292184 wait complete. 2024-09-10T09:45:08.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:08.914+0000 7fba62466740 20 client.26738 trim_cache size 0 max 16384 2024-09-10T09:45:08.919 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-10T09:45:08.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:08.915+0000 7fba5e200640 1 -- 172.21.15.5:0/1427474226 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fba400a2c00 msgr2=0x7fba400a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:45:08.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:08.915+0000 7fba5e200640 1 --2- 172.21.15.5:0/1427474226 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fba400a2c00 0x7fba400a50c0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fba4c0ecdd0 tx=0x7fba4c0f2e30 comp rx=0 tx=0).stop 2024-09-10T09:45:08.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:08.915+0000 7fba5e200640 1 -- 172.21.15.5:0/1427474226 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fba58103e30 msgr2=0x7fba5819a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:45:08.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:08.915+0000 7fba5e200640 1 --2- 172.21.15.5:0/1427474226 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fba58103e30 0x7fba5819a460 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fba4c035040 tx=0x7fba4c033b60 comp rx=0 tx=0).stop 2024-09-10T09:45:08.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:08.915+0000 7fba5e200640 1 -- 172.21.15.5:0/1427474226 shutdown_connections 2024-09-10T09:45:08.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:08.915+0000 7fba5e200640 1 --2- 172.21.15.5:0/1427474226 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fba400a2c00 0x7fba400a50c0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:08.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:08.915+0000 7fba5e200640 1 --2- 172.21.15.5:0/1427474226 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fba58108f60 0x7fba5819ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:08.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:08.915+0000 7fba5e200640 1 --2- 172.21.15.5:0/1427474226 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fba581047e0 0x7fba5819a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:08.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:08.915+0000 7fba5e200640 1 --2- 172.21.15.5:0/1427474226 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fba58103e30 0x7fba5819a460 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:08.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:08.915+0000 7fba5e200640 1 -- 172.21.15.5:0/1427474226 >> 172.21.15.5:0/1427474226 conn(0x7fba580ff8d0 msgr2=0x7fba58100c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:45:08.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:08.915+0000 7fba5e200640 1 -- 172.21.15.5:0/1427474226 shutdown_connections 2024-09-10T09:45:08.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:08.915+0000 7fba5e200640 1 -- 172.21.15.5:0/1427474226 wait complete. 2024-09-10T09:45:09.055 INFO:tasks.workunit.client.0.smithi005.stdout: 1 359745 5.32 MB/sec execute 170 sec latency 9414.223 ms 2024-09-10T09:45:09.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:45:08 smithi005 ceph-mon[30350]: pgmap v597: 129 pgs: 129 active+clean; 145 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.5 MiB/s wr, 108 op/s 2024-09-10T09:45:09.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:45:08 smithi159 ceph-mon[35513]: pgmap v597: 129 pgs: 129 active+clean; 145 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.5 MiB/s wr, 108 op/s 2024-09-10T09:45:09.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:45:08 smithi102 ceph-mon[35442]: pgmap v597: 129 pgs: 129 active+clean; 145 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.5 MiB/s wr, 108 op/s 2024-09-10T09:45:09.902 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-10T09:45:10.055 INFO:tasks.workunit.client.0.smithi005.stdout: 1 359745 5.28 MB/sec execute 171 sec latency 10414.305 ms 2024-09-10T09:45:10.281 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:45:10.976 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:45:10 smithi005 ceph-mon[30350]: pgmap v598: 129 pgs: 129 active+clean; 144 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.6 MiB/s wr, 113 op/s 2024-09-10T09:45:10.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:10.981+0000 7f6bc9e00640 1 -- 172.21.15.5:0/850079009 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6bc41049a0 msgr2=0x7f6bc41089f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:45:10.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:10.981+0000 7f6bc9e00640 1 --2- 172.21.15.5:0/850079009 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6bc41049a0 0x7f6bc41089f0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f6bac009900 tx=0x7f6bac02a400 comp rx=0 tx=0).stop 2024-09-10T09:45:10.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:10.981+0000 7f6bc9e00640 1 -- 172.21.15.5:0/850079009 shutdown_connections 2024-09-10T09:45:10.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:10.981+0000 7f6bc9e00640 1 --2- 172.21.15.5:0/850079009 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6bc4109120 0x7f6bc4110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:10.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:10.981+0000 7f6bc9e00640 1 --2- 172.21.15.5:0/850079009 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6bc41049a0 0x7f6bc41089f0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:10.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:10.981+0000 7f6bc9e00640 1 --2- 172.21.15.5:0/850079009 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6bc4103ff0 0x7f6bc41043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:10.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:10.981+0000 7f6bc9e00640 1 -- 172.21.15.5:0/850079009 >> 172.21.15.5:0/850079009 conn(0x7f6bc40ffa90 msgr2=0x7f6bc4101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:45:10.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:10.981+0000 7f6bc9e00640 1 -- 172.21.15.5:0/850079009 shutdown_connections 2024-09-10T09:45:10.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:10.981+0000 7f6bc9e00640 1 -- 172.21.15.5:0/850079009 wait complete. 2024-09-10T09:45:10.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:10.982+0000 7f6bc9e00640 1 Processor -- start 2024-09-10T09:45:10.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:10.983+0000 7f6bc9e00640 1 -- start start 2024-09-10T09:45:10.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:10.983+0000 7f6bc9e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6bc4103ff0 0x7f6bc419a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:45:10.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:10.983+0000 7f6bc9e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6bc41049a0 0x7f6bc419ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:45:10.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:10.983+0000 7f6bc9e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6bc4109120 0x7f6bc419ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:45:10.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:10.983+0000 7f6bc9e00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f6bc4114170 con 0x7f6bc41049a0 2024-09-10T09:45:10.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:10.983+0000 7f6bc9e00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f6bc4113ff0 con 0x7f6bc4109120 2024-09-10T09:45:10.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:10.983+0000 7f6bc9e00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f6bc41142f0 con 0x7f6bc4103ff0 2024-09-10T09:45:10.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:10.983+0000 7f6bba000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6bc41049a0 0x7f6bc419ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:45:10.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:10.983+0000 7f6bba000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6bc41049a0 0x7f6bc419ab70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34472/0 (socket says 172.21.15.5:34472) 2024-09-10T09:45:10.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:10.983+0000 7f6bba000640 1 -- 172.21.15.5:0/4145988473 learned_addr learned my addr 172.21.15.5:0/4145988473 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:45:10.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:10.983+0000 7f6bc2a00640 1 --2- 172.21.15.5:0/4145988473 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6bc4103ff0 0x7f6bc419a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:45:10.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:10.983+0000 7f6bc3400640 1 --2- 172.21.15.5:0/4145988473 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6bc4109120 0x7f6bc419ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:45:10.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:10.984+0000 7f6bba000640 1 -- 172.21.15.5:0/4145988473 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6bc4103ff0 msgr2=0x7f6bc419a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:45:10.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:10.984+0000 7f6bba000640 1 --2- 172.21.15.5:0/4145988473 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6bc4103ff0 0x7f6bc419a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:10.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:10.984+0000 7f6bba000640 1 -- 172.21.15.5:0/4145988473 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6bc4109120 msgr2=0x7f6bc419ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:45:10.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:10.984+0000 7f6bba000640 1 --2- 172.21.15.5:0/4145988473 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6bc4109120 0x7f6bc419ef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:10.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:10.984+0000 7f6bba000640 1 -- 172.21.15.5:0/4145988473 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6bc419f5e0 con 0x7f6bc41049a0 2024-09-10T09:45:10.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:10.984+0000 7f6bc3400640 1 --2- 172.21.15.5:0/4145988473 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6bc4109120 0x7f6bc419ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:45:10.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:10.984+0000 7f6bba000640 1 --2- 172.21.15.5:0/4145988473 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6bc41049a0 0x7f6bc419ab70 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f6bac004770 tx=0x7f6bac003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:45:10.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:10.984+0000 7f6bc8a00640 1 -- 172.21.15.5:0/4145988473 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bac03f070 con 0x7f6bc41049a0 2024-09-10T09:45:10.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:10.984+0000 7f6bc8a00640 1 -- 172.21.15.5:0/4145988473 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6bac031ca0 con 0x7f6bc41049a0 2024-09-10T09:45:10.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:10.984+0000 7f6bc9e00640 1 -- 172.21.15.5:0/4145988473 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6bc419f870 con 0x7f6bc41049a0 2024-09-10T09:45:10.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:10.984+0000 7f6bc8a00640 1 -- 172.21.15.5:0/4145988473 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bac035030 con 0x7f6bc41049a0 2024-09-10T09:45:10.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:10.985+0000 7f6bc9e00640 1 -- 172.21.15.5:0/4145988473 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6bc41a71b0 con 0x7f6bc41049a0 2024-09-10T09:45:10.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:10.988+0000 7f6bc8a00640 1 -- 172.21.15.5:0/4145988473 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f6bac042070 con 0x7f6bc41049a0 2024-09-10T09:45:10.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:10.988+0000 7f6bcc868740 1 Processor -- start 2024-09-10T09:45:10.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:10.988+0000 7f6bcc868740 1 -- start start 2024-09-10T09:45:10.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:10.988+0000 7f6bcc868740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d376f73c70 0x55d376f74050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:45:10.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:10.988+0000 7f6bcc868740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d376f74590 0x55d376f7e910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:45:10.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:10.988+0000 7f6bcc868740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d376f7ee50 0x55d376f81240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:45:10.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:10.988+0000 7f6bcc868740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55d376eda3e0 con 0x55d376f74590 2024-09-10T09:45:10.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:10.988+0000 7f6bcc868740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55d376e81d80 con 0x55d376f73c70 2024-09-10T09:45:10.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:10.988+0000 7f6bcc868740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55d376ed63a0 con 0x55d376f7ee50 2024-09-10T09:45:10.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:10.989+0000 7f6bc2a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d376f74590 0x55d376f7e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:45:10.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:10.989+0000 7f6bc3400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d376f73c70 0x55d376f74050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:45:10.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:10.989+0000 7f6bba000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d376f7ee50 0x55d376f81240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:45:10.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:10.989+0000 7f6bc3400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d376f73c70 0x55d376f74050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:33884/0 (socket says 172.21.15.5:33884) 2024-09-10T09:45:10.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:10.989+0000 7f6bc2a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d376f74590 0x55d376f7e910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34480/0 (socket says 172.21.15.5:34480) 2024-09-10T09:45:10.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:10.989+0000 7f6bc3400640 1 -- 172.21.15.5:0/300080077 learned_addr learned my addr 172.21.15.5:0/300080077 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:45:10.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:10.989+0000 7f6bc3400640 1 -- 172.21.15.5:0/300080077 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d376f7ee50 msgr2=0x55d376f81240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:45:10.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:10.989+0000 7f6bc3400640 1 --2- 172.21.15.5:0/300080077 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d376f7ee50 0x55d376f81240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:10.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:10.989+0000 7f6bc3400640 1 -- 172.21.15.5:0/300080077 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d376f74590 msgr2=0x55d376f7e910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:45:10.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:10.989+0000 7f6bc3400640 1 --2- 172.21.15.5:0/300080077 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d376f74590 0x55d376f7e910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:10.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:10.989+0000 7f6bc3400640 1 -- 172.21.15.5:0/300080077 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d376ea0ae0 con 0x55d376f73c70 2024-09-10T09:45:10.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:10.989+0000 7f6bc8a00640 1 --2- 172.21.15.5:0/4145988473 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f6ba80a2c00 0x7f6ba80a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:45:10.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:10.990+0000 7f6bc3400640 1 --2- 172.21.15.5:0/300080077 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d376f73c70 0x55d376f74050 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f6bbc009fd0 tx=0x7f6bbc010860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c002711d4099ded3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:45:10.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:10.990+0000 7f6bc8a00640 1 -- 172.21.15.5:0/4145988473 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f6bac0efa50 con 0x7f6bc41049a0 2024-09-10T09:45:10.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:10.990+0000 7f6bc2a00640 1 --2- 172.21.15.5:0/4145988473 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f6ba80a2c00 0x7f6ba80a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:45:10.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:10.990+0000 7f6bb3e00640 1 -- 172.21.15.5:0/300080077 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bbc011700 con 0x55d376f73c70 2024-09-10T09:45:10.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:10.990+0000 7f6bb3e00640 1 -- 172.21.15.5:0/300080077 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6bbc011d80 con 0x55d376f73c70 2024-09-10T09:45:10.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:10.990+0000 7f6bb3e00640 1 -- 172.21.15.5:0/300080077 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bbc00e3c0 con 0x55d376f73c70 2024-09-10T09:45:10.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:10.991+0000 7f6bcc868740 1 -- 172.21.15.5:0/300080077 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d376f73c70 msgr2=0x55d376f74050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:45:10.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:10.991+0000 7f6bcc868740 1 --2- 172.21.15.5:0/300080077 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d376f73c70 0x55d376f74050 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f6bbc009fd0 tx=0x7f6bbc010860 comp rx=0 tx=0).stop 2024-09-10T09:45:10.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:10.991+0000 7f6bc2a00640 1 --2- 172.21.15.5:0/4145988473 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f6ba80a2c00 0x7f6ba80a50c0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x55d376f71060 tx=0x7f6bb4005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:45:10.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:10.991+0000 7f6bcc868740 1 -- 172.21.15.5:0/300080077 shutdown_connections 2024-09-10T09:45:10.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:10.991+0000 7f6bcc868740 1 --2- 172.21.15.5:0/300080077 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d376f7ee50 0x55d376f81240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:10.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:10.991+0000 7f6bcc868740 1 --2- 172.21.15.5:0/300080077 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d376f74590 0x55d376f7e910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:10.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:10.991+0000 7f6bcc868740 1 --2- 172.21.15.5:0/300080077 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d376f73c70 0x55d376f74050 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:10.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:10.991+0000 7f6bcc868740 1 -- 172.21.15.5:0/300080077 >> 172.21.15.5:0/300080077 conn(0x55d376f70200 msgr2=0x55d376f705e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:45:10.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:10.991+0000 7f6bcc868740 1 -- 172.21.15.5:0/300080077 shutdown_connections 2024-09-10T09:45:10.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:10.991+0000 7f6bcc868740 1 -- 172.21.15.5:0/300080077 wait complete. 2024-09-10T09:45:10.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:10.991+0000 7f6bcc868740 1 -- start start 2024-09-10T09:45:10.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:10.992+0000 7f6bcc868740 1 Processor -- start 2024-09-10T09:45:10.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:10.992+0000 7f6bcc868740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d376f73c70 0x55d376f7bd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:45:10.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:10.992+0000 7f6bcc868740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d376f74590 0x55d376f7c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:45:10.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:10.992+0000 7f6bcc868740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d376f7ee50 0x55d376f7c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:45:10.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:10.992+0000 7f6bcc868740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55d376f81a20 con 0x55d376f74590 2024-09-10T09:45:10.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:10.992+0000 7f6bcc868740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55d376e950f0 con 0x55d376f73c70 2024-09-10T09:45:10.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:10.992+0000 7f6bcc868740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55d376f81ba0 con 0x55d376f7ee50 2024-09-10T09:45:10.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:10.992+0000 7f6bcc868740 10 client.0 _ll_register_callbacks cb 0x55d376e7cbd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:45:10.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:10.992+0000 7f6bc2a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d376f73c70 0x55d376f7bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:45:10.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:10.992+0000 7f6bc2a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d376f73c70 0x55d376f7bd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:33898/0 (socket says 172.21.15.5:33898) 2024-09-10T09:45:10.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:10.992+0000 7f6bc2a00640 1 -- 172.21.15.5:0/1047690214 learned_addr learned my addr 172.21.15.5:0/1047690214 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:45:10.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:10.993+0000 7f6bba000640 1 --2- 172.21.15.5:0/1047690214 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d376f74590 0x55d376f7c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:45:10.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:10.993+0000 7f6bc3400640 1 --2- 172.21.15.5:0/1047690214 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d376f7ee50 0x55d376f7c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:45:10.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:10.993+0000 7f6bc2a00640 1 -- 172.21.15.5:0/1047690214 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d376f7ee50 msgr2=0x55d376f7c7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:45:10.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:10.993+0000 7f6bc2a00640 1 --2- 172.21.15.5:0/1047690214 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d376f7ee50 0x55d376f7c7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:10.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:10.993+0000 7f6bc2a00640 1 -- 172.21.15.5:0/1047690214 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d376f74590 msgr2=0x55d376f7c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:45:10.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:10.993+0000 7f6bc2a00640 1 --2- 172.21.15.5:0/1047690214 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d376f74590 0x55d376f7c2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:10.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:10.993+0000 7f6bc2a00640 1 -- 172.21.15.5:0/1047690214 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d376c4b3c0 con 0x55d376f73c70 2024-09-10T09:45:10.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:10.993+0000 7f6bba000640 1 --2- 172.21.15.5:0/1047690214 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d376f74590 0x55d376f7c2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:45:10.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:10.993+0000 7f6bc2a00640 1 --2- 172.21.15.5:0/1047690214 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d376f73c70 0x55d376f7bd60 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f6bb40045f0 tx=0x7f6bb4004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=67f86b142c9be2ac server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:45:10.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:10.993+0000 7f6bb0c00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-09-10T09:45:10.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:10.994+0000 7f6bb0c00640 1 -- 172.21.15.5:0/1047690214 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bb401f640 con 0x55d376f73c70 2024-09-10T09:45:10.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:10.994+0000 7f6bb0c00640 1 -- 172.21.15.5:0/1047690214 <== mon.2 v2:172.21.15.102:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f6bb401f7e0 con 0x55d376f73c70 2024-09-10T09:45:10.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:10.994+0000 7f6bcc868740 1 -- 172.21.15.5:0/1047690214 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6bbc011d80 con 0x55d376f73c70 2024-09-10T09:45:11.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:10.994+0000 7f6bb0c00640 1 -- 172.21.15.5:0/1047690214 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f6bb401f9e0 con 0x55d376f73c70 2024-09-10T09:45:11.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:10.994+0000 7f6bcc868740 10 client.26750 fetch_fsmap learned FSMap version 34 2024-09-10T09:45:11.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:10.994+0000 7f6bcc868740 1 -- 172.21.15.5:0/1047690214 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55d376f8daf0 con 0x55d376f73c70 2024-09-10T09:45:11.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:10.994+0000 7f6bb0c00640 1 -- 172.21.15.5:0/1047690214 <== mon.2 v2:172.21.15.102:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6bbc011d80 con 0x55d376f73c70 2024-09-10T09:45:11.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:10.994+0000 7f6bb0c00640 1 -- 172.21.15.5:0/1047690214 <== mon.2 v2:172.21.15.102:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f6bb4027020 con 0x55d376f73c70 2024-09-10T09:45:11.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:10.995+0000 7f6bcc868740 10 client.26750 fetch_fsmap finished waiting for FSMap version 34 2024-09-10T09:45:11.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:10.995+0000 7f6bcc868740 10 client.26750 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-10T09:45:11.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:10.995+0000 7f6bcc868740 20 client.26750 populate_metadata read hostname 'smithi005' 2024-09-10T09:45:11.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:10.995+0000 7f6bcc868740 1 --2- 172.21.15.5:0/1047690214 >> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] conn(0x55d376f8efb0 0x55d376f91400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:45:11.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:10.995+0000 7f6bcc868740 4 client.26750 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:45:11.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:10.995+0000 7f6bcc868740 1 -- 172.21.15.5:0/1047690214 --> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d376ee7950 con 0x55d376f8efb0 2024-09-10T09:45:11.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:10.995+0000 7f6bba000640 1 --2- 172.21.15.5:0/1047690214 >> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] conn(0x55d376f8efb0 0x55d376f91400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:45:11.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:10.995+0000 7f6bba000640 1 --2- 172.21.15.5:0/1047690214 >> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] conn(0x55d376f8efb0 0x55d376f91400 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f8ce3703f1c6c481 server_cookie=c8c4def0c700e172 in_seq=0 out_seq=0 2024-09-10T09:45:11.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:10.995+0000 7f6bb0c00640 10 client.26750 ms_handle_connect on v2:172.21.15.159:6836/620627639 2024-09-10T09:45:11.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:10.996+0000 7f6bb0c00640 1 -- 172.21.15.5:0/1047690214 <== mds.2 v2:172.21.15.159:6836/620627639 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d376ee7950 con 0x55d376f8efb0 2024-09-10T09:45:11.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:10.996+0000 7f6bb0c00640 10 client.26750 handle_command_reply: tid=0 2024-09-10T09:45:11.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:10.996+0000 7f6bcc868740 1 client.26750 shutdown 2024-09-10T09:45:11.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:10.997+0000 7f6bcc868740 1 -- 172.21.15.5:0/1047690214 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d376f73c70 msgr2=0x55d376f7bd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:45:11.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:10.997+0000 7f6bcc868740 1 --2- 172.21.15.5:0/1047690214 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d376f73c70 0x55d376f7bd60 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f6bb40045f0 tx=0x7f6bb4004730 comp rx=0 tx=0).stop 2024-09-10T09:45:11.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:10.997+0000 7f6bcc868740 1 -- 172.21.15.5:0/1047690214 shutdown_connections 2024-09-10T09:45:11.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:10.997+0000 7f6bcc868740 1 --2- 172.21.15.5:0/1047690214 >> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] conn(0x55d376f8efb0 0x55d376f91400 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:11.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:10.997+0000 7f6bcc868740 1 --2- 172.21.15.5:0/1047690214 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d376f7ee50 0x55d376f7c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:11.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:10.997+0000 7f6bcc868740 1 --2- 172.21.15.5:0/1047690214 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d376f74590 0x55d376f7c2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:11.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:10.997+0000 7f6bb0c00640 0 client.26750 ms_handle_reset on v2:172.21.15.159:6836/620627639 2024-09-10T09:45:11.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:10.997+0000 7f6bcc868740 1 --2- 172.21.15.5:0/1047690214 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d376f73c70 0x55d376f7bd60 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:11.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:10.997+0000 7f6bcc868740 1 -- 172.21.15.5:0/1047690214 >> 172.21.15.5:0/1047690214 conn(0x55d376f70200 msgr2=0x55d376f75f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:45:11.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:10.997+0000 7f6bcc868740 1 -- 172.21.15.5:0/1047690214 shutdown_connections 2024-09-10T09:45:11.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:10.997+0000 7f6bcc868740 1 -- 172.21.15.5:0/1047690214 wait complete. 2024-09-10T09:45:11.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:10.997+0000 7f6bcc868740 20 client.26750 trim_cache size 0 max 16384 2024-09-10T09:45:11.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:11.011+0000 7f6bcc868740 1 Processor -- start 2024-09-10T09:45:11.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:11.011+0000 7f6bcc868740 1 -- start start 2024-09-10T09:45:11.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:11.011+0000 7f6bcc868740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d376f73c70 0x55d376f96cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:45:11.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:11.011+0000 7f6bcc868740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d376f74590 0x55d376f97210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:45:11.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:11.011+0000 7f6bcc868740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d376f7ee50 0x55d376f75710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:45:11.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:11.011+0000 7f6bcc868740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f6bc41b1010 con 0x55d376f73c70 2024-09-10T09:45:11.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:11.011+0000 7f6bcc868740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f6bc41b2b50 con 0x55d376f7ee50 2024-09-10T09:45:11.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:11.011+0000 7f6bcc868740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f6bc41b16e0 con 0x55d376f74590 2024-09-10T09:45:11.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:11.011+0000 7f6bc2a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d376f73c70 0x55d376f96cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:45:11.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:11.011+0000 7f6bc2a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d376f73c70 0x55d376f96cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34498/0 (socket says 172.21.15.5:34498) 2024-09-10T09:45:11.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:11.011+0000 7f6bc2a00640 1 -- 172.21.15.5:0/855546262 learned_addr learned my addr 172.21.15.5:0/855546262 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:45:11.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:11.011+0000 7f6bba000640 1 --2- 172.21.15.5:0/855546262 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d376f74590 0x55d376f97210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:45:11.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:11.012+0000 7f6bc3400640 1 --2- 172.21.15.5:0/855546262 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d376f7ee50 0x55d376f75710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:45:11.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:11.012+0000 7f6bc2a00640 1 -- 172.21.15.5:0/855546262 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d376f74590 msgr2=0x55d376f97210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:45:11.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:11.012+0000 7f6bc2a00640 1 --2- 172.21.15.5:0/855546262 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d376f74590 0x55d376f97210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:11.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:11.012+0000 7f6bc2a00640 1 -- 172.21.15.5:0/855546262 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d376f7ee50 msgr2=0x55d376f75710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:45:11.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:11.012+0000 7f6bc2a00640 1 --2- 172.21.15.5:0/855546262 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d376f7ee50 0x55d376f75710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:11.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:11.012+0000 7f6bc2a00640 1 -- 172.21.15.5:0/855546262 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d376f6b770 con 0x55d376f73c70 2024-09-10T09:45:11.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:11.012+0000 7f6bc3400640 1 --2- 172.21.15.5:0/855546262 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d376f7ee50 0x55d376f75710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:45:11.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:11.012+0000 7f6bc2a00640 1 --2- 172.21.15.5:0/855546262 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d376f73c70 0x55d376f96cd0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f6bb4029d90 tx=0x7f6bb4009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=548fbb58fe6c1010 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:45:11.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:11.012+0000 7f6bb3400640 1 -- 172.21.15.5:0/855546262 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bb403b070 con 0x55d376f73c70 2024-09-10T09:45:11.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:11.012+0000 7f6bb3400640 1 -- 172.21.15.5:0/855546262 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6bb4029920 con 0x55d376f73c70 2024-09-10T09:45:11.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:11.012+0000 7f6bb3400640 1 -- 172.21.15.5:0/855546262 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bb4036730 con 0x55d376f73c70 2024-09-10T09:45:11.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:11.013+0000 7f6bcc868740 1 -- 172.21.15.5:0/855546262 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d376f73c70 msgr2=0x55d376f96cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:45:11.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:11.013+0000 7f6bcc868740 1 --2- 172.21.15.5:0/855546262 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d376f73c70 0x55d376f96cd0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f6bb4029d90 tx=0x7f6bb4009c00 comp rx=0 tx=0).stop 2024-09-10T09:45:11.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:11.013+0000 7f6bcc868740 1 -- 172.21.15.5:0/855546262 shutdown_connections 2024-09-10T09:45:11.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:11.013+0000 7f6bcc868740 1 --2- 172.21.15.5:0/855546262 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d376f7ee50 0x55d376f75710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:11.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:11.013+0000 7f6bcc868740 1 --2- 172.21.15.5:0/855546262 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d376f74590 0x55d376f97210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:11.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:11.013+0000 7f6bcc868740 1 --2- 172.21.15.5:0/855546262 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d376f73c70 0x55d376f96cd0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:11.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:11.013+0000 7f6bcc868740 1 -- 172.21.15.5:0/855546262 >> 172.21.15.5:0/855546262 conn(0x55d376f70200 msgr2=0x55d376f7ae50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:45:11.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:11.013+0000 7f6bcc868740 1 -- 172.21.15.5:0/855546262 shutdown_connections 2024-09-10T09:45:11.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:11.013+0000 7f6bcc868740 1 -- 172.21.15.5:0/855546262 wait complete. 2024-09-10T09:45:11.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:11.014+0000 7f6bcc868740 1 -- start start 2024-09-10T09:45:11.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:11.014+0000 7f6bcc868740 1 Processor -- start 2024-09-10T09:45:11.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:11.014+0000 7f6bcc868740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d376f73c70 0x55d376f82350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:45:11.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:11.014+0000 7f6bcc868740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d376f74590 0x55d376f82890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:45:11.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:11.014+0000 7f6bcc868740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d376f7ee50 0x55d376ff7c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:45:11.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:11.014+0000 7f6bcc868740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f6bc41b2480 con 0x55d376f74590 2024-09-10T09:45:11.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:11.014+0000 7f6bcc868740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f6bc41b1db0 con 0x55d376f7ee50 2024-09-10T09:45:11.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:11.014+0000 7f6bcc868740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f6bc41aeff0 con 0x55d376f73c70 2024-09-10T09:45:11.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:11.014+0000 7f6bcc868740 10 client.0 _ll_register_callbacks cb 0x7f6bb4009ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:45:11.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:11.015+0000 7f6bc2a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d376f73c70 0x55d376f82350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:45:11.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:11.015+0000 7f6bba000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d376f74590 0x55d376f82890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:45:11.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:11.015+0000 7f6bc2a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d376f73c70 0x55d376f82350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:49420/0 (socket says 172.21.15.5:49420) 2024-09-10T09:45:11.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:11.015+0000 7f6bc2a00640 1 -- 172.21.15.5:0/800564305 learned_addr learned my addr 172.21.15.5:0/800564305 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:45:11.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:11.015+0000 7f6bc3400640 1 --2- 172.21.15.5:0/800564305 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d376f7ee50 0x55d376ff7c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:45:11.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:11.015+0000 7f6bc2a00640 1 -- 172.21.15.5:0/800564305 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d376f7ee50 msgr2=0x55d376ff7c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:45:11.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:11.015+0000 7f6bc2a00640 1 --2- 172.21.15.5:0/800564305 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d376f7ee50 0x55d376ff7c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:11.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:11.015+0000 7f6bc2a00640 1 -- 172.21.15.5:0/800564305 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d376f74590 msgr2=0x55d376f82890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:45:11.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:11.015+0000 7f6bc2a00640 1 --2- 172.21.15.5:0/800564305 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d376f74590 0x55d376f82890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:11.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:11.015+0000 7f6bc2a00640 1 -- 172.21.15.5:0/800564305 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d376f712b0 con 0x55d376f73c70 2024-09-10T09:45:11.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:11.015+0000 7f6bc2a00640 1 --2- 172.21.15.5:0/800564305 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d376f73c70 0x55d376f82350 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f6bb4019040 tx=0x7f6bb4020f50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a07bd554c1f899bb server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:45:11.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:11.015+0000 7f6bb0c00640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-09-10T09:45:11.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:11.016+0000 7f6bb0c00640 1 -- 172.21.15.5:0/800564305 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bb4044060 con 0x55d376f73c70 2024-09-10T09:45:11.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:11.016+0000 7f6bb0c00640 1 -- 172.21.15.5:0/800564305 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f6bb4029640 con 0x55d376f73c70 2024-09-10T09:45:11.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:11.016+0000 7f6bcc868740 1 -- 172.21.15.5:0/800564305 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6bb4029920 con 0x55d376f73c70 2024-09-10T09:45:11.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:11.016+0000 7f6bb0c00640 1 -- 172.21.15.5:0/800564305 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f6bb403b070 con 0x55d376f73c70 2024-09-10T09:45:11.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:11.016+0000 7f6bcc868740 10 client.26782 fetch_fsmap learned FSMap version 34 2024-09-10T09:45:11.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:11.016+0000 7f6bcc868740 1 -- 172.21.15.5:0/800564305 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55d376ff9390 con 0x55d376f73c70 2024-09-10T09:45:11.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:11.016+0000 7f6bb0c00640 1 -- 172.21.15.5:0/800564305 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6bb4029920 con 0x55d376f73c70 2024-09-10T09:45:11.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:11.017+0000 7f6bb0c00640 1 -- 172.21.15.5:0/800564305 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f6bb4027070 con 0x55d376f73c70 2024-09-10T09:45:11.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:11.017+0000 7f6bcc868740 10 client.26782 fetch_fsmap finished waiting for FSMap version 34 2024-09-10T09:45:11.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:11.017+0000 7f6bcc868740 10 client.26782 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-10T09:45:11.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:11.017+0000 7f6bcc868740 20 client.26782 populate_metadata read hostname 'smithi005' 2024-09-10T09:45:11.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:11.017+0000 7f6bcc868740 1 --2- 172.21.15.5:0/800564305 >> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] conn(0x55d376ff99f0 0x55d376ffbe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:45:11.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:11.017+0000 7f6bcc868740 4 client.26782 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T09:45:11.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:11.017+0000 7f6bcc868740 1 -- 172.21.15.5:0/800564305 --> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d376ee1e60 con 0x55d376ff99f0 2024-09-10T09:45:11.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:11.017+0000 7f6bba000640 1 --2- 172.21.15.5:0/800564305 >> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] conn(0x55d376ff99f0 0x55d376ffbe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:45:11.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:11.018+0000 7f6bba000640 1 --2- 172.21.15.5:0/800564305 >> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] conn(0x55d376ff99f0 0x55d376ffbe10 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d21ae64bffa2ddf9 server_cookie=fde02fde9b50ca03 in_seq=0 out_seq=0 2024-09-10T09:45:11.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:11.018+0000 7f6bb0c00640 10 client.26782 ms_handle_connect on v2:172.21.15.159:6836/620627639 2024-09-10T09:45:11.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:11.018+0000 7f6bb0c00640 1 -- 172.21.15.5:0/800564305 <== mds.2 v2:172.21.15.159:6836/620627639 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d376ee1e60 con 0x55d376ff99f0 2024-09-10T09:45:11.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:11.018+0000 7f6bb0c00640 10 client.26782 handle_command_reply: tid=0 2024-09-10T09:45:11.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:11.018+0000 7f6bcc868740 1 client.26782 shutdown 2024-09-10T09:45:11.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:11.018+0000 7f6bcc868740 1 -- 172.21.15.5:0/800564305 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d376f73c70 msgr2=0x55d376f82350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:45:11.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:11.018+0000 7f6bcc868740 1 --2- 172.21.15.5:0/800564305 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d376f73c70 0x55d376f82350 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f6bb4019040 tx=0x7f6bb4020f50 comp rx=0 tx=0).stop 2024-09-10T09:45:11.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:11.018+0000 7f6bcc868740 1 -- 172.21.15.5:0/800564305 shutdown_connections 2024-09-10T09:45:11.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:11.018+0000 7f6bcc868740 1 --2- 172.21.15.5:0/800564305 >> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] conn(0x55d376ff99f0 0x55d376ffbe10 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:11.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:11.018+0000 7f6bcc868740 1 --2- 172.21.15.5:0/800564305 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d376f7ee50 0x55d376ff7c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:11.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:11.018+0000 7f6bcc868740 1 --2- 172.21.15.5:0/800564305 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d376f74590 0x55d376f82890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:11.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:11.018+0000 7f6bcc868740 1 --2- 172.21.15.5:0/800564305 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d376f73c70 0x55d376f82350 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:11.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:11.018+0000 7f6bb0c00640 0 client.26782 ms_handle_reset on v2:172.21.15.159:6836/620627639 2024-09-10T09:45:11.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:11.018+0000 7f6bcc868740 1 -- 172.21.15.5:0/800564305 >> 172.21.15.5:0/800564305 conn(0x55d376f70200 msgr2=0x55d376f75710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:45:11.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:11.018+0000 7f6bcc868740 1 -- 172.21.15.5:0/800564305 shutdown_connections 2024-09-10T09:45:11.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:11.018+0000 7f6bcc868740 1 -- 172.21.15.5:0/800564305 wait complete. 2024-09-10T09:45:11.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:11.018+0000 7f6bcc868740 20 client.26782 trim_cache size 0 max 16384 2024-09-10T09:45:11.024 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-10T09:45:11.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:11.019+0000 7f6bc9e00640 1 -- 172.21.15.5:0/4145988473 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f6ba80a2c00 msgr2=0x7f6ba80a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:45:11.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:11.019+0000 7f6bc9e00640 1 --2- 172.21.15.5:0/4145988473 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f6ba80a2c00 0x7f6ba80a50c0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x55d376f71060 tx=0x7f6bb4005f90 comp rx=0 tx=0).stop 2024-09-10T09:45:11.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:11.019+0000 7f6bc9e00640 1 -- 172.21.15.5:0/4145988473 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6bc41049a0 msgr2=0x7f6bc419ab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:45:11.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:11.019+0000 7f6bc9e00640 1 --2- 172.21.15.5:0/4145988473 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6bc41049a0 0x7f6bc419ab70 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f6bac004770 tx=0x7f6bac003c10 comp rx=0 tx=0).stop 2024-09-10T09:45:11.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:11.019+0000 7f6bc9e00640 1 -- 172.21.15.5:0/4145988473 shutdown_connections 2024-09-10T09:45:11.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:11.019+0000 7f6bc9e00640 1 --2- 172.21.15.5:0/4145988473 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f6ba80a2c00 0x7f6ba80a50c0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:11.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:11.019+0000 7f6bc9e00640 1 --2- 172.21.15.5:0/4145988473 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6bc4109120 0x7f6bc419ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:11.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:11.019+0000 7f6bc9e00640 1 --2- 172.21.15.5:0/4145988473 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6bc41049a0 0x7f6bc419ab70 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:11.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:11.019+0000 7f6bc9e00640 1 --2- 172.21.15.5:0/4145988473 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6bc4103ff0 0x7f6bc419a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:11.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:11.019+0000 7f6bc9e00640 1 -- 172.21.15.5:0/4145988473 >> 172.21.15.5:0/4145988473 conn(0x7f6bc40ffa90 msgr2=0x7f6bc4100da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:45:11.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:11.019+0000 7f6bc9e00640 1 -- 172.21.15.5:0/4145988473 shutdown_connections 2024-09-10T09:45:11.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:11.019+0000 7f6bc9e00640 1 -- 172.21.15.5:0/4145988473 wait complete. 2024-09-10T09:45:11.055 INFO:tasks.workunit.client.0.smithi005.stdout: 1 359745 5.25 MB/sec execute 172 sec latency 11414.364 ms 2024-09-10T09:45:11.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:45:10 smithi159 ceph-mon[35513]: pgmap v598: 129 pgs: 129 active+clean; 144 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.6 MiB/s wr, 113 op/s 2024-09-10T09:45:11.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:45:10 smithi102 ceph-mon[35442]: pgmap v598: 129 pgs: 129 active+clean; 144 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.6 MiB/s wr, 113 op/s 2024-09-10T09:45:11.855 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-10T09:45:12.055 INFO:tasks.workunit.client.0.smithi005.stdout: 1 359745 5.22 MB/sec execute 173 sec latency 12414.418 ms 2024-09-10T09:45:12.234 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:45:12.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:12.883+0000 7fe562e00640 1 -- 172.21.15.5:0/90247167 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe55c1049a0 msgr2=0x7fe55c1089f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:45:12.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:12.883+0000 7fe562e00640 1 --2- 172.21.15.5:0/90247167 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe55c1049a0 0x7fe55c1089f0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fe544009900 tx=0x7fe54402a410 comp rx=0 tx=0).stop 2024-09-10T09:45:12.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:12.883+0000 7fe562e00640 1 -- 172.21.15.5:0/90247167 shutdown_connections 2024-09-10T09:45:12.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:12.883+0000 7fe562e00640 1 --2- 172.21.15.5:0/90247167 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe55c109120 0x7fe55c110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:12.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:12.883+0000 7fe562e00640 1 --2- 172.21.15.5:0/90247167 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe55c1049a0 0x7fe55c1089f0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:12.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:12.883+0000 7fe562e00640 1 --2- 172.21.15.5:0/90247167 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe55c103ff0 0x7fe55c1043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:12.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:12.883+0000 7fe562e00640 1 -- 172.21.15.5:0/90247167 >> 172.21.15.5:0/90247167 conn(0x7fe55c0ffa90 msgr2=0x7fe55c101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:45:12.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:12.883+0000 7fe562e00640 1 -- 172.21.15.5:0/90247167 shutdown_connections 2024-09-10T09:45:12.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:12.883+0000 7fe562e00640 1 -- 172.21.15.5:0/90247167 wait complete. 2024-09-10T09:45:12.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:12.884+0000 7fe562e00640 1 Processor -- start 2024-09-10T09:45:12.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:12.884+0000 7fe562e00640 1 -- start start 2024-09-10T09:45:12.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:12.884+0000 7fe562e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe55c103ff0 0x7fe55c19a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:45:12.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:12.884+0000 7fe562e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe55c1049a0 0x7fe55c19ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:45:12.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:12.884+0000 7fe562e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe55c109120 0x7fe55c19ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:45:12.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:12.884+0000 7fe562e00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe55c114170 con 0x7fe55c103ff0 2024-09-10T09:45:12.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:12.884+0000 7fe562e00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fe55c113ff0 con 0x7fe55c109120 2024-09-10T09:45:12.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:12.884+0000 7fe562e00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fe55c1142f0 con 0x7fe55c1049a0 2024-09-10T09:45:12.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:12.885+0000 7fe55be00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe55c103ff0 0x7fe55c19a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:45:12.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:12.885+0000 7fe55be00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe55c103ff0 0x7fe55c19a650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34522/0 (socket says 172.21.15.5:34522) 2024-09-10T09:45:12.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:12.885+0000 7fe55be00640 1 -- 172.21.15.5:0/926441638 learned_addr learned my addr 172.21.15.5:0/926441638 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:45:12.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:12.885+0000 7fe553e00640 1 --2- 172.21.15.5:0/926441638 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe55c1049a0 0x7fe55c19ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:45:12.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:12.885+0000 7fe560e00640 1 --2- 172.21.15.5:0/926441638 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe55c109120 0x7fe55c19ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:45:12.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:12.885+0000 7fe55be00640 1 -- 172.21.15.5:0/926441638 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe55c1049a0 msgr2=0x7fe55c19ab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:45:12.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:12.885+0000 7fe55be00640 1 --2- 172.21.15.5:0/926441638 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe55c1049a0 0x7fe55c19ab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:12.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:12.885+0000 7fe55be00640 1 -- 172.21.15.5:0/926441638 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe55c109120 msgr2=0x7fe55c19ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:45:12.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:12.885+0000 7fe55be00640 1 --2- 172.21.15.5:0/926441638 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe55c109120 0x7fe55c19ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:12.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:12.885+0000 7fe55be00640 1 -- 172.21.15.5:0/926441638 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe55c19f600 con 0x7fe55c103ff0 2024-09-10T09:45:12.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:12.885+0000 7fe553e00640 1 --2- 172.21.15.5:0/926441638 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe55c1049a0 0x7fe55c19ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:45:12.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:12.886+0000 7fe55be00640 1 --2- 172.21.15.5:0/926441638 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe55c103ff0 0x7fe55c19a650 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fe54c00e9f0 tx=0x7fe54c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:45:12.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:12.886+0000 7fe559600640 1 -- 172.21.15.5:0/926441638 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe54c00cd80 con 0x7fe55c103ff0 2024-09-10T09:45:12.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:12.886+0000 7fe562e00640 1 -- 172.21.15.5:0/926441638 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe55c19f8f0 con 0x7fe55c103ff0 2024-09-10T09:45:12.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:12.886+0000 7fe559600640 1 -- 172.21.15.5:0/926441638 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe54c010910 con 0x7fe55c103ff0 2024-09-10T09:45:12.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:12.886+0000 7fe559600640 1 -- 172.21.15.5:0/926441638 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe54c0188f0 con 0x7fe55c103ff0 2024-09-10T09:45:12.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:12.886+0000 7fe562e00640 1 -- 172.21.15.5:0/926441638 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe55c1a71d0 con 0x7fe55c103ff0 2024-09-10T09:45:12.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:12.889+0000 7fe565889740 1 Processor -- start 2024-09-10T09:45:12.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:12.890+0000 7fe559600640 1 -- 172.21.15.5:0/926441638 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7fe54c010ac0 con 0x7fe55c103ff0 2024-09-10T09:45:12.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:12.890+0000 7fe565889740 1 -- start start 2024-09-10T09:45:12.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:12.890+0000 7fe565889740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563d51b96c70 0x563d51b97050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:45:12.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:12.890+0000 7fe565889740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563d51b97590 0x563d51ba1910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:45:12.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:12.890+0000 7fe565889740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563d51ba1e50 0x563d51ba4240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:45:12.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:12.890+0000 7fe565889740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x563d51afd3e0 con 0x563d51b97590 2024-09-10T09:45:12.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:12.890+0000 7fe565889740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x563d51aa4d80 con 0x563d51ba1e50 2024-09-10T09:45:12.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:12.890+0000 7fe565889740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x563d51af93a0 con 0x563d51b96c70 2024-09-10T09:45:12.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:12.890+0000 7fe55be00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563d51b97590 0x563d51ba1910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:45:12.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:12.890+0000 7fe560e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563d51b96c70 0x563d51b97050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:45:12.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:12.890+0000 7fe55be00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563d51b97590 0x563d51ba1910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34532/0 (socket says 172.21.15.5:34532) 2024-09-10T09:45:12.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:12.890+0000 7fe55be00640 1 -- 172.21.15.5:0/1802563641 learned_addr learned my addr 172.21.15.5:0/1802563641 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:45:12.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:12.890+0000 7fe553e00640 1 --2- 172.21.15.5:0/1802563641 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563d51ba1e50 0x563d51ba4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:45:12.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:12.891+0000 7fe560e00640 1 -- 172.21.15.5:0/1802563641 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563d51ba1e50 msgr2=0x563d51ba4240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:45:12.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:12.891+0000 7fe560e00640 1 --2- 172.21.15.5:0/1802563641 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563d51ba1e50 0x563d51ba4240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:12.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:12.891+0000 7fe560e00640 1 -- 172.21.15.5:0/1802563641 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563d51b97590 msgr2=0x563d51ba1910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:45:12.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:12.891+0000 7fe560e00640 1 --2- 172.21.15.5:0/1802563641 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563d51b97590 0x563d51ba1910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:12.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:12.891+0000 7fe560e00640 1 -- 172.21.15.5:0/1802563641 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563d51ac3ae0 con 0x563d51b96c70 2024-09-10T09:45:12.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:12.891+0000 7fe553e00640 1 --2- 172.21.15.5:0/1802563641 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563d51ba1e50 0x563d51ba4240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:45:12.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:12.891+0000 7fe559600640 1 --2- 172.21.15.5:0/926441638 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fe5400a2b80 0x7fe5400a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:45:12.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:12.892+0000 7fe560e00640 1 --2- 172.21.15.5:0/1802563641 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563d51b96c70 0x563d51b97050 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fe554009f00 tx=0x7fe554011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4a0494fabd90926e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:45:12.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:12.892+0000 7fe559600640 1 -- 172.21.15.5:0/926441638 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fe54c0cf7a0 con 0x7fe55c103ff0 2024-09-10T09:45:12.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:12.892+0000 7fe551600640 1 -- 172.21.15.5:0/1802563641 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe554004d20 con 0x563d51b96c70 2024-09-10T09:45:12.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:12.892+0000 7fe55be00640 1 --2- 172.21.15.5:0/926441638 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fe5400a2b80 0x7fe5400a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:45:12.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:12.892+0000 7fe551600640 1 -- 172.21.15.5:0/1802563641 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe554004050 con 0x563d51b96c70 2024-09-10T09:45:12.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:12.892+0000 7fe551600640 1 -- 172.21.15.5:0/1802563641 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe554005280 con 0x563d51b96c70 2024-09-10T09:45:12.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:12.893+0000 7fe565889740 1 -- 172.21.15.5:0/1802563641 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563d51b96c70 msgr2=0x563d51b97050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:45:12.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:12.893+0000 7fe565889740 1 --2- 172.21.15.5:0/1802563641 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563d51b96c70 0x563d51b97050 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fe554009f00 tx=0x7fe554011810 comp rx=0 tx=0).stop 2024-09-10T09:45:12.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:12.893+0000 7fe55be00640 1 --2- 172.21.15.5:0/926441638 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fe5400a2b80 0x7fe5400a5040 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fe54c09b9c0 tx=0x7fe54c0d5550 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:45:12.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:12.893+0000 7fe565889740 1 -- 172.21.15.5:0/1802563641 shutdown_connections 2024-09-10T09:45:12.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:12.893+0000 7fe565889740 1 --2- 172.21.15.5:0/1802563641 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563d51ba1e50 0x563d51ba4240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:12.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:12.893+0000 7fe565889740 1 --2- 172.21.15.5:0/1802563641 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563d51b97590 0x563d51ba1910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:12.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:12.893+0000 7fe565889740 1 --2- 172.21.15.5:0/1802563641 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563d51b96c70 0x563d51b97050 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:12.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:12.893+0000 7fe565889740 1 -- 172.21.15.5:0/1802563641 >> 172.21.15.5:0/1802563641 conn(0x563d51b93200 msgr2=0x563d51b935e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:45:12.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:12.893+0000 7fe565889740 1 -- 172.21.15.5:0/1802563641 shutdown_connections 2024-09-10T09:45:12.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:12.893+0000 7fe565889740 1 -- 172.21.15.5:0/1802563641 wait complete. 2024-09-10T09:45:12.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:12.894+0000 7fe565889740 1 -- start start 2024-09-10T09:45:12.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:12.894+0000 7fe565889740 1 Processor -- start 2024-09-10T09:45:12.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:12.895+0000 7fe565889740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563d51b96c70 0x563d51b9ed50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:45:12.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:12.895+0000 7fe565889740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563d51b97590 0x563d51b9f290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:45:12.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:12.895+0000 7fe565889740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563d51ba1e50 0x563d51b9f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:45:12.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:12.895+0000 7fe565889740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x563d51ba4a20 con 0x563d51b96c70 2024-09-10T09:45:12.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:12.895+0000 7fe565889740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x563d51ab80f0 con 0x563d51b97590 2024-09-10T09:45:12.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:12.895+0000 7fe565889740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x563d51ba4ba0 con 0x563d51ba1e50 2024-09-10T09:45:12.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:12.895+0000 7fe565889740 10 client.0 _ll_register_callbacks cb 0x563d51a9fbd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:45:12.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:12.895+0000 7fe553e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563d51b97590 0x563d51b9f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:45:12.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:12.895+0000 7fe553e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563d51b97590 0x563d51b9f290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:33950/0 (socket says 172.21.15.5:33950) 2024-09-10T09:45:12.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:12.895+0000 7fe553e00640 1 -- 172.21.15.5:0/908812207 learned_addr learned my addr 172.21.15.5:0/908812207 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:45:12.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:12.895+0000 7fe560e00640 1 --2- 172.21.15.5:0/908812207 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563d51b96c70 0x563d51b9ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:45:12.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:12.895+0000 7fe560e00640 1 --2- 172.21.15.5:0/908812207 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563d51ba1e50 0x563d51b9f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:45:12.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:12.896+0000 7fe553e00640 1 -- 172.21.15.5:0/908812207 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563d51ba1e50 msgr2=0x563d51b9f7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:45:12.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:12.896+0000 7fe553e00640 1 --2- 172.21.15.5:0/908812207 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563d51ba1e50 0x563d51b9f7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:12.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:12.896+0000 7fe553e00640 1 -- 172.21.15.5:0/908812207 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563d51b96c70 msgr2=0x563d51b9ed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:45:12.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:12.896+0000 7fe553e00640 1 --2- 172.21.15.5:0/908812207 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563d51b96c70 0x563d51b9ed50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:12.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:12.896+0000 7fe553e00640 1 -- 172.21.15.5:0/908812207 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563d5186e3c0 con 0x563d51b97590 2024-09-10T09:45:12.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:12.896+0000 7fe553e00640 1 --2- 172.21.15.5:0/908812207 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563d51b97590 0x563d51b9f290 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fe544033000 tx=0x7fe544005270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7ef85d12cb6d6afa server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:45:12.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:12.896+0000 7fe54a000640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-09-10T09:45:12.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:12.896+0000 7fe54a000640 1 -- 172.21.15.5:0/908812207 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe544035040 con 0x563d51b97590 2024-09-10T09:45:12.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:12.896+0000 7fe565889740 1 -- 172.21.15.5:0/908812207 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe554004050 con 0x563d51b97590 2024-09-10T09:45:12.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:12.896+0000 7fe54a000640 1 -- 172.21.15.5:0/908812207 <== mon.2 v2:172.21.15.102:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fe544004960 con 0x563d51b97590 2024-09-10T09:45:12.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:12.897+0000 7fe54a000640 1 -- 172.21.15.5:0/908812207 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fe54404e050 con 0x563d51b97590 2024-09-10T09:45:12.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:12.897+0000 7fe565889740 10 client.26768 fetch_fsmap learned FSMap version 34 2024-09-10T09:45:12.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:12.897+0000 7fe565889740 1 -- 172.21.15.5:0/908812207 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x563d51bb92d0 con 0x563d51b97590 2024-09-10T09:45:12.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:12.897+0000 7fe54a000640 1 -- 172.21.15.5:0/908812207 <== mon.2 v2:172.21.15.102:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe554004050 con 0x563d51b97590 2024-09-10T09:45:12.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:12.897+0000 7fe54a000640 1 -- 172.21.15.5:0/908812207 <== mon.2 v2:172.21.15.102:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7fe544031c50 con 0x563d51b97590 2024-09-10T09:45:12.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:12.897+0000 7fe565889740 10 client.26768 fetch_fsmap finished waiting for FSMap version 34 2024-09-10T09:45:12.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:12.897+0000 7fe565889740 10 client.26768 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-10T09:45:12.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:12.897+0000 7fe565889740 20 client.26768 populate_metadata read hostname 'smithi005' 2024-09-10T09:45:12.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:12.897+0000 7fe565889740 1 --2- 172.21.15.5:0/908812207 >> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] conn(0x563d51bba6e0 0x563d51bbcb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:45:12.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:12.897+0000 7fe565889740 4 client.26768 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:45:12.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:12.897+0000 7fe565889740 1 -- 172.21.15.5:0/908812207 --> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563d51b0a950 con 0x563d51bba6e0 2024-09-10T09:45:12.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:12.898+0000 7fe560e00640 1 --2- 172.21.15.5:0/908812207 >> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] conn(0x563d51bba6e0 0x563d51bbcb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:45:12.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:12.898+0000 7fe560e00640 1 --2- 172.21.15.5:0/908812207 >> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] conn(0x563d51bba6e0 0x563d51bbcb40 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7a69164328c6bfd2 server_cookie=40d344d3d94b68ba in_seq=0 out_seq=0 2024-09-10T09:45:12.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:12.898+0000 7fe54a000640 10 client.26768 ms_handle_connect on v2:172.21.15.159:6838/2073739599 2024-09-10T09:45:12.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:12.899+0000 7fe54a000640 1 -- 172.21.15.5:0/908812207 <== mds.1 v2:172.21.15.159:6838/2073739599 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563d51b0a950 con 0x563d51bba6e0 2024-09-10T09:45:12.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:12.899+0000 7fe54a000640 10 client.26768 handle_command_reply: tid=0 2024-09-10T09:45:12.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:12.899+0000 7fe565889740 1 client.26768 shutdown 2024-09-10T09:45:12.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:12.899+0000 7fe565889740 1 -- 172.21.15.5:0/908812207 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563d51b97590 msgr2=0x563d51b9f290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:45:12.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:12.900+0000 7fe565889740 1 --2- 172.21.15.5:0/908812207 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563d51b97590 0x563d51b9f290 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fe544033000 tx=0x7fe544005270 comp rx=0 tx=0).stop 2024-09-10T09:45:12.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:12.900+0000 7fe565889740 1 -- 172.21.15.5:0/908812207 shutdown_connections 2024-09-10T09:45:12.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:12.900+0000 7fe565889740 1 --2- 172.21.15.5:0/908812207 >> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] conn(0x563d51bba6e0 0x563d51bbcb40 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:12.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:12.900+0000 7fe565889740 1 --2- 172.21.15.5:0/908812207 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563d51ba1e50 0x563d51b9f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:12.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:12.900+0000 7fe565889740 1 --2- 172.21.15.5:0/908812207 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563d51b97590 0x563d51b9f290 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:12.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:12.900+0000 7fe565889740 1 --2- 172.21.15.5:0/908812207 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563d51b96c70 0x563d51b9ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:12.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:12.900+0000 7fe565889740 1 -- 172.21.15.5:0/908812207 >> 172.21.15.5:0/908812207 conn(0x563d51b93200 msgr2=0x563d51b92380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:45:12.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:12.900+0000 7fe54a000640 0 client.26768 ms_handle_reset on v2:172.21.15.159:6838/2073739599 2024-09-10T09:45:12.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:12.900+0000 7fe565889740 1 -- 172.21.15.5:0/908812207 shutdown_connections 2024-09-10T09:45:12.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:12.900+0000 7fe565889740 1 -- 172.21.15.5:0/908812207 wait complete. 2024-09-10T09:45:12.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:12.900+0000 7fe565889740 20 client.26768 trim_cache size 0 max 16384 2024-09-10T09:45:12.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:12.915+0000 7fe565889740 1 Processor -- start 2024-09-10T09:45:12.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:12.915+0000 7fe565889740 1 -- start start 2024-09-10T09:45:12.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:12.915+0000 7fe565889740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563d51b96c70 0x563d51b996d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:45:12.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:12.915+0000 7fe565889740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563d51b97590 0x563d51bc2410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:45:12.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:12.915+0000 7fe565889740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563d51ba1e50 0x563d51bc2950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:45:12.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:12.915+0000 7fe565889740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe55c1acbe0 con 0x563d51b97590 2024-09-10T09:45:12.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:12.915+0000 7fe565889740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fe55c1ae720 con 0x563d51ba1e50 2024-09-10T09:45:12.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:12.915+0000 7fe565889740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fe55c1ad2b0 con 0x563d51b96c70 2024-09-10T09:45:12.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:12.915+0000 7fe553e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563d51b97590 0x563d51bc2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:45:12.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:12.915+0000 7fe553e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563d51b97590 0x563d51bc2410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34550/0 (socket says 172.21.15.5:34550) 2024-09-10T09:45:12.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:12.915+0000 7fe553e00640 1 -- 172.21.15.5:0/1566393715 learned_addr learned my addr 172.21.15.5:0/1566393715 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:45:12.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:12.915+0000 7fe560e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563d51b96c70 0x563d51b996d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:45:12.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:12.916+0000 7fe560e00640 1 --2- 172.21.15.5:0/1566393715 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563d51ba1e50 0x563d51bc2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:45:12.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:12.916+0000 7fe553e00640 1 -- 172.21.15.5:0/1566393715 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563d51b96c70 msgr2=0x563d51b996d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:45:12.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:12.916+0000 7fe553e00640 1 --2- 172.21.15.5:0/1566393715 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563d51b96c70 0x563d51b996d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:12.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:12.916+0000 7fe553e00640 1 -- 172.21.15.5:0/1566393715 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563d51ba1e50 msgr2=0x563d51bc2950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:45:12.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:12.916+0000 7fe553e00640 1 --2- 172.21.15.5:0/1566393715 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563d51ba1e50 0x563d51bc2950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:12.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:12.916+0000 7fe553e00640 1 -- 172.21.15.5:0/1566393715 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563d51b8e770 con 0x563d51b97590 2024-09-10T09:45:12.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:12.916+0000 7fe560e00640 1 --2- 172.21.15.5:0/1566393715 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563d51ba1e50 0x563d51bc2950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:45:12.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:12.916+0000 7fe553e00640 1 --2- 172.21.15.5:0/1566393715 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563d51b97590 0x563d51bc2410 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fe544003730 tx=0x7fe54402a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=559b28d024bd9a88 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:45:12.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:12.917+0000 7fe550c00640 1 -- 172.21.15.5:0/1566393715 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe544035070 con 0x563d51b97590 2024-09-10T09:45:12.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:12.917+0000 7fe550c00640 1 -- 172.21.15.5:0/1566393715 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe544005850 con 0x563d51b97590 2024-09-10T09:45:12.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:12.917+0000 7fe550c00640 1 -- 172.21.15.5:0/1566393715 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe544003e70 con 0x563d51b97590 2024-09-10T09:45:12.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:12.918+0000 7fe565889740 1 -- 172.21.15.5:0/1566393715 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563d51b97590 msgr2=0x563d51bc2410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:45:12.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:12.918+0000 7fe565889740 1 --2- 172.21.15.5:0/1566393715 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563d51b97590 0x563d51bc2410 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fe544003730 tx=0x7fe54402a410 comp rx=0 tx=0).stop 2024-09-10T09:45:12.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:12.918+0000 7fe565889740 1 -- 172.21.15.5:0/1566393715 shutdown_connections 2024-09-10T09:45:12.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:12.918+0000 7fe565889740 1 --2- 172.21.15.5:0/1566393715 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563d51ba1e50 0x563d51bc2950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:12.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:12.918+0000 7fe565889740 1 --2- 172.21.15.5:0/1566393715 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563d51b97590 0x563d51bc2410 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:12.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:12.918+0000 7fe565889740 1 --2- 172.21.15.5:0/1566393715 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563d51b96c70 0x563d51b996d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:12.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:12.918+0000 7fe565889740 1 -- 172.21.15.5:0/1566393715 >> 172.21.15.5:0/1566393715 conn(0x563d51b93200 msgr2=0x563d51ba0dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:45:12.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:12.918+0000 7fe565889740 1 -- 172.21.15.5:0/1566393715 shutdown_connections 2024-09-10T09:45:12.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:12.918+0000 7fe565889740 1 -- 172.21.15.5:0/1566393715 wait complete. 2024-09-10T09:45:12.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:12.919+0000 7fe565889740 1 -- start start 2024-09-10T09:45:12.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:12.919+0000 7fe565889740 1 Processor -- start 2024-09-10T09:45:12.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:12.920+0000 7fe565889740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563d51b96c70 0x563d51bae600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:45:12.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:12.920+0000 7fe565889740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563d51b97590 0x563d51baeb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:45:12.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:12.920+0000 7fe565889740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563d51ba1e50 0x563d51baf080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:45:12.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:12.920+0000 7fe565889740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe55c1ae050 con 0x563d51b97590 2024-09-10T09:45:12.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:12.920+0000 7fe565889740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fe55c1ad980 con 0x563d51b96c70 2024-09-10T09:45:12.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:12.920+0000 7fe565889740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fe55c1aabe0 con 0x563d51ba1e50 2024-09-10T09:45:12.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:12.920+0000 7fe565889740 10 client.0 _ll_register_callbacks cb 0x7fe544009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:45:12.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:12.920+0000 7fe560e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563d51b96c70 0x563d51bae600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:45:12.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:12.920+0000 7fe560e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563d51b96c70 0x563d51bae600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:33970/0 (socket says 172.21.15.5:33970) 2024-09-10T09:45:12.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:12.920+0000 7fe560e00640 1 -- 172.21.15.5:0/323833734 learned_addr learned my addr 172.21.15.5:0/323833734 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:45:12.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:12.920+0000 7fe553e00640 1 --2- 172.21.15.5:0/323833734 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563d51b97590 0x563d51baeb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:45:12.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:12.920+0000 7fe560e00640 1 --2- 172.21.15.5:0/323833734 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563d51ba1e50 0x563d51baf080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:45:12.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:12.920+0000 7fe560e00640 1 -- 172.21.15.5:0/323833734 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563d51ba1e50 msgr2=0x563d51baf080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:45:12.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:12.920+0000 7fe560e00640 1 --2- 172.21.15.5:0/323833734 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563d51ba1e50 0x563d51baf080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:12.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:12.920+0000 7fe560e00640 1 -- 172.21.15.5:0/323833734 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563d51b97590 msgr2=0x563d51baeb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:45:12.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:12.920+0000 7fe560e00640 1 --2- 172.21.15.5:0/323833734 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563d51b97590 0x563d51baeb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:12.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:12.920+0000 7fe560e00640 1 -- 172.21.15.5:0/323833734 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563d51b93e90 con 0x563d51b96c70 2024-09-10T09:45:12.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:12.921+0000 7fe560e00640 1 --2- 172.21.15.5:0/323833734 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563d51b96c70 0x563d51bae600 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fe554009ed0 tx=0x7fe544005e00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5b99cc37adf714bf server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:45:12.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:12.921+0000 7fe54a000640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-09-10T09:45:12.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:12.921+0000 7fe54a000640 1 -- 172.21.15.5:0/323833734 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe554039060 con 0x563d51b96c70 2024-09-10T09:45:12.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:12.921+0000 7fe54a000640 1 -- 172.21.15.5:0/323833734 <== mon.2 v2:172.21.15.102:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fe554004490 con 0x563d51b96c70 2024-09-10T09:45:12.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:12.921+0000 7fe565889740 1 -- 172.21.15.5:0/323833734 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe544005850 con 0x563d51b96c70 2024-09-10T09:45:12.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:12.921+0000 7fe54a000640 1 -- 172.21.15.5:0/323833734 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fe554026430 con 0x563d51b96c70 2024-09-10T09:45:12.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:12.921+0000 7fe565889740 10 client.26774 fetch_fsmap learned FSMap version 34 2024-09-10T09:45:12.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:12.921+0000 7fe565889740 1 -- 172.21.15.5:0/323833734 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x563d51c256b0 con 0x563d51b96c70 2024-09-10T09:45:12.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:12.922+0000 7fe54a000640 1 -- 172.21.15.5:0/323833734 <== mon.2 v2:172.21.15.102:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe544005850 con 0x563d51b96c70 2024-09-10T09:45:12.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:12.922+0000 7fe54a000640 1 -- 172.21.15.5:0/323833734 <== mon.2 v2:172.21.15.102:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7fe554019550 con 0x563d51b96c70 2024-09-10T09:45:12.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:12.922+0000 7fe565889740 10 client.26774 fetch_fsmap finished waiting for FSMap version 34 2024-09-10T09:45:12.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:12.922+0000 7fe565889740 10 client.26774 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-10T09:45:12.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:12.922+0000 7fe565889740 20 client.26774 populate_metadata read hostname 'smithi005' 2024-09-10T09:45:12.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:12.922+0000 7fe565889740 1 --2- 172.21.15.5:0/323833734 >> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] conn(0x563d51c25be0 0x563d51c27fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:45:12.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:12.922+0000 7fe565889740 4 client.26774 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T09:45:12.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:12.922+0000 7fe565889740 1 -- 172.21.15.5:0/323833734 --> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563d51b04e60 con 0x563d51c25be0 2024-09-10T09:45:12.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:12.922+0000 7fe553e00640 1 --2- 172.21.15.5:0/323833734 >> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] conn(0x563d51c25be0 0x563d51c27fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:45:12.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:12.923+0000 7fe553e00640 1 --2- 172.21.15.5:0/323833734 >> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] conn(0x563d51c25be0 0x563d51c27fd0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c1a53e8f88cadb1a server_cookie=5aeec42d77696186 in_seq=0 out_seq=0 2024-09-10T09:45:12.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:12.923+0000 7fe54a000640 10 client.26774 ms_handle_connect on v2:172.21.15.159:6838/2073739599 2024-09-10T09:45:12.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:12.923+0000 7fe54a000640 1 -- 172.21.15.5:0/323833734 <== mds.1 v2:172.21.15.159:6838/2073739599 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563d51b04e60 con 0x563d51c25be0 2024-09-10T09:45:12.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:12.923+0000 7fe54a000640 10 client.26774 handle_command_reply: tid=0 2024-09-10T09:45:12.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:12.923+0000 7fe565889740 1 client.26774 shutdown 2024-09-10T09:45:12.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:12.923+0000 7fe565889740 1 -- 172.21.15.5:0/323833734 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563d51b96c70 msgr2=0x563d51bae600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:45:12.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:12.923+0000 7fe565889740 1 --2- 172.21.15.5:0/323833734 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563d51b96c70 0x563d51bae600 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fe554009ed0 tx=0x7fe544005e00 comp rx=0 tx=0).stop 2024-09-10T09:45:12.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:12.923+0000 7fe565889740 1 -- 172.21.15.5:0/323833734 shutdown_connections 2024-09-10T09:45:12.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:12.923+0000 7fe565889740 1 --2- 172.21.15.5:0/323833734 >> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] conn(0x563d51c25be0 0x563d51c27fd0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:12.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:12.923+0000 7fe565889740 1 --2- 172.21.15.5:0/323833734 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563d51ba1e50 0x563d51baf080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:12.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:12.923+0000 7fe565889740 1 --2- 172.21.15.5:0/323833734 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563d51b97590 0x563d51baeb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:12.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:12.923+0000 7fe565889740 1 --2- 172.21.15.5:0/323833734 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563d51b96c70 0x563d51bae600 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:12.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:12.923+0000 7fe565889740 1 -- 172.21.15.5:0/323833734 >> 172.21.15.5:0/323833734 conn(0x563d51b93200 msgr2=0x563d51b91b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:45:12.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:12.923+0000 7fe54a000640 0 client.26774 ms_handle_reset on v2:172.21.15.159:6838/2073739599 2024-09-10T09:45:12.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:12.924+0000 7fe565889740 1 -- 172.21.15.5:0/323833734 shutdown_connections 2024-09-10T09:45:12.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:12.924+0000 7fe565889740 1 -- 172.21.15.5:0/323833734 wait complete. 2024-09-10T09:45:12.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:12.924+0000 7fe565889740 20 client.26774 trim_cache size 0 max 16384 2024-09-10T09:45:12.929 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-10T09:45:12.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:12.924+0000 7fe562e00640 1 -- 172.21.15.5:0/926441638 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fe5400a2b80 msgr2=0x7fe5400a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:45:12.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:12.924+0000 7fe562e00640 1 --2- 172.21.15.5:0/926441638 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fe5400a2b80 0x7fe5400a5040 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fe54c09b9c0 tx=0x7fe54c0d5550 comp rx=0 tx=0).stop 2024-09-10T09:45:12.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:12.924+0000 7fe562e00640 1 -- 172.21.15.5:0/926441638 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe55c103ff0 msgr2=0x7fe55c19a650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:45:12.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:12.924+0000 7fe562e00640 1 --2- 172.21.15.5:0/926441638 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe55c103ff0 0x7fe55c19a650 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fe54c00e9f0 tx=0x7fe54c00eeb0 comp rx=0 tx=0).stop 2024-09-10T09:45:12.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:12.924+0000 7fe562e00640 1 -- 172.21.15.5:0/926441638 shutdown_connections 2024-09-10T09:45:12.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:12.924+0000 7fe562e00640 1 --2- 172.21.15.5:0/926441638 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fe5400a2b80 0x7fe5400a5040 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:12.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:12.924+0000 7fe562e00640 1 --2- 172.21.15.5:0/926441638 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe55c109120 0x7fe55c19ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:12.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:12.924+0000 7fe562e00640 1 --2- 172.21.15.5:0/926441638 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe55c1049a0 0x7fe55c19ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:12.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:12.925+0000 7fe562e00640 1 --2- 172.21.15.5:0/926441638 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe55c103ff0 0x7fe55c19a650 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:12.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:12.925+0000 7fe562e00640 1 -- 172.21.15.5:0/926441638 >> 172.21.15.5:0/926441638 conn(0x7fe55c0ffa90 msgr2=0x7fe55c100da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:45:12.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:12.925+0000 7fe562e00640 1 -- 172.21.15.5:0/926441638 shutdown_connections 2024-09-10T09:45:12.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:12.925+0000 7fe562e00640 1 -- 172.21.15.5:0/926441638 wait complete. 2024-09-10T09:45:13.055 INFO:tasks.workunit.client.0.smithi005.stdout: 1 359745 5.19 MB/sec execute 174 sec latency 13414.512 ms 2024-09-10T09:45:13.306 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:45:12 smithi005 ceph-mon[30350]: pgmap v599: 129 pgs: 129 active+clean; 143 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 490 KiB/s rd, 1.2 MiB/s wr, 72 op/s 2024-09-10T09:45:13.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:45:12 smithi159 ceph-mon[35513]: pgmap v599: 129 pgs: 129 active+clean; 143 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 490 KiB/s rd, 1.2 MiB/s wr, 72 op/s 2024-09-10T09:45:13.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:45:12 smithi102 ceph-mon[35442]: pgmap v599: 129 pgs: 129 active+clean; 143 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 490 KiB/s rd, 1.2 MiB/s wr, 72 op/s 2024-09-10T09:45:13.694 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-10T09:45:14.055 INFO:tasks.workunit.client.0.smithi005.stdout: 1 359745 5.16 MB/sec execute 175 sec latency 14414.561 ms 2024-09-10T09:45:14.073 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:45:14.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:14.616+0000 7f39e3600640 1 -- 172.21.15.5:0/2671964738 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f39dc112a50 msgr2=0x7f39dc114e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:45:14.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:14.616+0000 7f39e3600640 1 --2- 172.21.15.5:0/2671964738 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f39dc112a50 0x7f39dc114e40 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f39d4009ca0 tx=0x7f39d402a8c0 comp rx=0 tx=0).stop 2024-09-10T09:45:14.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:14.616+0000 7f39e3600640 1 -- 172.21.15.5:0/2671964738 shutdown_connections 2024-09-10T09:45:14.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:14.616+0000 7f39e3600640 1 --2- 172.21.15.5:0/2671964738 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f39dc112a50 0x7f39dc114e40 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:14.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:14.616+0000 7f39e3600640 1 --2- 172.21.15.5:0/2671964738 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f39dc077040 0x7f39dc112510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:14.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:14.616+0000 7f39e3600640 1 --2- 172.21.15.5:0/2671964738 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f39dc076720 0x7f39dc076b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:14.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:14.616+0000 7f39e3600640 1 -- 172.21.15.5:0/2671964738 >> 172.21.15.5:0/2671964738 conn(0x7f39dc0ffa50 msgr2=0x7f39dc101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:45:14.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:14.616+0000 7f39e3600640 1 -- 172.21.15.5:0/2671964738 shutdown_connections 2024-09-10T09:45:14.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:14.616+0000 7f39e3600640 1 -- 172.21.15.5:0/2671964738 wait complete. 2024-09-10T09:45:14.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:14.617+0000 7f39e3600640 1 Processor -- start 2024-09-10T09:45:14.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:14.617+0000 7f39e3600640 1 -- start start 2024-09-10T09:45:14.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:14.618+0000 7f39e3600640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f39dc076720 0x7f39dc19eb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:45:14.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:14.618+0000 7f39e3600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f39dc077040 0x7f39dc19f090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:45:14.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:14.618+0000 7f39e3600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f39dc112a50 0x7f39dc1a3450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:45:14.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:14.618+0000 7f39e3600640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f39dc1185f0 con 0x7f39dc077040 2024-09-10T09:45:14.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:14.618+0000 7f39e3600640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f39dc118470 con 0x7f39dc112a50 2024-09-10T09:45:14.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:14.618+0000 7f39e3600640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f39dc118770 con 0x7f39dc076720 2024-09-10T09:45:14.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:14.618+0000 7f39e1600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f39dc112a50 0x7f39dc1a3450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:45:14.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:14.618+0000 7f39dbe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f39dc077040 0x7f39dc19f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:45:14.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:14.618+0000 7f39dbe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f39dc077040 0x7f39dc19f090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34590/0 (socket says 172.21.15.5:34590) 2024-09-10T09:45:14.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:14.618+0000 7f39dbe00640 1 -- 172.21.15.5:0/2516384903 learned_addr learned my addr 172.21.15.5:0/2516384903 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:45:14.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:14.618+0000 7f39e1600640 1 --2- 172.21.15.5:0/2516384903 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f39dc112a50 0x7f39dc1a3450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:33988/0 (socket says 172.21.15.5:33988) 2024-09-10T09:45:14.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:14.618+0000 7f39e0c00640 1 --2- 172.21.15.5:0/2516384903 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f39dc076720 0x7f39dc19eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:45:14.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:14.619+0000 7f39dbe00640 1 -- 172.21.15.5:0/2516384903 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f39dc076720 msgr2=0x7f39dc19eb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:45:14.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:14.619+0000 7f39dbe00640 1 --2- 172.21.15.5:0/2516384903 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f39dc076720 0x7f39dc19eb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:14.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:14.619+0000 7f39dbe00640 1 -- 172.21.15.5:0/2516384903 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f39dc112a50 msgr2=0x7f39dc1a3450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:45:14.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:14.619+0000 7f39dbe00640 1 --2- 172.21.15.5:0/2516384903 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f39dc112a50 0x7f39dc1a3450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:14.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:14.619+0000 7f39dbe00640 1 -- 172.21.15.5:0/2516384903 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39dc1a3b80 con 0x7f39dc077040 2024-09-10T09:45:14.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:14.619+0000 7f39dbe00640 1 --2- 172.21.15.5:0/2516384903 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f39dc077040 0x7f39dc19f090 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f39c800e9f0 tx=0x7f39c800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:45:14.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:14.619+0000 7f39d9600640 1 -- 172.21.15.5:0/2516384903 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39c800cd80 con 0x7f39dc077040 2024-09-10T09:45:14.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:14.619+0000 7f39d9600640 1 -- 172.21.15.5:0/2516384903 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f39c8010910 con 0x7f39dc077040 2024-09-10T09:45:14.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:14.619+0000 7f39d9600640 1 -- 172.21.15.5:0/2516384903 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39c8018990 con 0x7f39dc077040 2024-09-10T09:45:14.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:14.619+0000 7f39e3600640 1 -- 172.21.15.5:0/2516384903 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39dc1a3e70 con 0x7f39dc077040 2024-09-10T09:45:14.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:14.620+0000 7f39e3600640 1 -- 172.21.15.5:0/2516384903 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39dc1ab700 con 0x7f39dc077040 2024-09-10T09:45:14.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:14.623+0000 7f39e5fca740 1 Processor -- start 2024-09-10T09:45:14.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:14.623+0000 7f39e5fca740 1 -- start start 2024-09-10T09:45:14.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:14.623+0000 7f39e5fca740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fdeaccbc70 0x55fdeaccc050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:45:14.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:14.623+0000 7f39e5fca740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fdeaccc590 0x55fdeacd6910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:45:14.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:14.623+0000 7f39e5fca740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fdeacd6e50 0x55fdeacd9240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:45:14.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:14.623+0000 7f39e5fca740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55fdeac323e0 con 0x55fdeaccbc70 2024-09-10T09:45:14.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:14.623+0000 7f39e5fca740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55fdeabd9d80 con 0x55fdeacd6e50 2024-09-10T09:45:14.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:14.623+0000 7f39e5fca740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55fdeac2e3a0 con 0x55fdeaccc590 2024-09-10T09:45:14.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:14.623+0000 7f39d9600640 1 -- 172.21.15.5:0/2516384903 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f39c8010ac0 con 0x7f39dc077040 2024-09-10T09:45:14.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:14.624+0000 7f39e0c00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fdeaccc590 0x55fdeacd6910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:45:14.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:14.624+0000 7f39e0c00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fdeaccc590 0x55fdeacd6910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:49506/0 (socket says 172.21.15.5:49506) 2024-09-10T09:45:14.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:14.624+0000 7f39e1600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fdeaccbc70 0x55fdeaccc050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:45:14.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:14.624+0000 7f39e0c00640 1 -- 172.21.15.5:0/4009518871 learned_addr learned my addr 172.21.15.5:0/4009518871 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:45:14.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:14.624+0000 7f39dbe00640 1 --2- 172.21.15.5:0/4009518871 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fdeacd6e50 0x55fdeacd9240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:45:14.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:14.624+0000 7f39e0c00640 1 -- 172.21.15.5:0/4009518871 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fdeacd6e50 msgr2=0x55fdeacd9240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:45:14.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:14.624+0000 7f39e0c00640 1 --2- 172.21.15.5:0/4009518871 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fdeacd6e50 0x55fdeacd9240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:14.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:14.624+0000 7f39e0c00640 1 -- 172.21.15.5:0/4009518871 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fdeaccbc70 msgr2=0x55fdeaccc050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:45:14.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:14.624+0000 7f39e0c00640 1 --2- 172.21.15.5:0/4009518871 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fdeaccbc70 0x55fdeaccc050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:14.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:14.624+0000 7f39e0c00640 1 -- 172.21.15.5:0/4009518871 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fdeabf8ae0 con 0x55fdeaccc590 2024-09-10T09:45:14.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:14.624+0000 7f39dbe00640 1 --2- 172.21.15.5:0/4009518871 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fdeacd6e50 0x55fdeacd9240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:45:14.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:14.625+0000 7f39e0c00640 1 --2- 172.21.15.5:0/4009518871 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fdeaccc590 0x55fdeacd6910 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f39d0009d20 tx=0x7f39d0011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ce6efeb0786d54c2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:45:14.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:14.625+0000 7f39cdc00640 1 -- 172.21.15.5:0/4009518871 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39d001e070 con 0x55fdeaccc590 2024-09-10T09:45:14.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:14.625+0000 7f39cdc00640 1 -- 172.21.15.5:0/4009518871 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f39d0019b10 con 0x55fdeaccc590 2024-09-10T09:45:14.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:14.625+0000 7f39cdc00640 1 -- 172.21.15.5:0/4009518871 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39d000eb30 con 0x55fdeaccc590 2024-09-10T09:45:14.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:14.625+0000 7f39d9600640 1 --2- 172.21.15.5:0/2516384903 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f39c40a2c00 0x7f39c40a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:45:14.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:14.625+0000 7f39d9600640 1 -- 172.21.15.5:0/2516384903 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f39c80cf660 con 0x7f39dc077040 2024-09-10T09:45:14.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:14.625+0000 7f39e1600640 1 --2- 172.21.15.5:0/2516384903 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f39c40a2c00 0x7f39c40a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:45:14.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:14.626+0000 7f39e5fca740 1 -- 172.21.15.5:0/4009518871 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fdeaccc590 msgr2=0x55fdeacd6910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:45:14.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:14.626+0000 7f39e5fca740 1 --2- 172.21.15.5:0/4009518871 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fdeaccc590 0x55fdeacd6910 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f39d0009d20 tx=0x7f39d0011810 comp rx=0 tx=0).stop 2024-09-10T09:45:14.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:14.626+0000 7f39e5fca740 1 -- 172.21.15.5:0/4009518871 shutdown_connections 2024-09-10T09:45:14.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:14.626+0000 7f39e5fca740 1 --2- 172.21.15.5:0/4009518871 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fdeacd6e50 0x55fdeacd9240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:14.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:14.626+0000 7f39e5fca740 1 --2- 172.21.15.5:0/4009518871 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fdeaccc590 0x55fdeacd6910 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:14.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:14.626+0000 7f39e5fca740 1 --2- 172.21.15.5:0/4009518871 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fdeaccbc70 0x55fdeaccc050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:14.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:14.626+0000 7f39e5fca740 1 -- 172.21.15.5:0/4009518871 >> 172.21.15.5:0/4009518871 conn(0x55fdeacc8200 msgr2=0x55fdeacc85e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:45:14.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:14.626+0000 7f39e5fca740 1 -- 172.21.15.5:0/4009518871 shutdown_connections 2024-09-10T09:45:14.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:14.626+0000 7f39e5fca740 1 -- 172.21.15.5:0/4009518871 wait complete. 2024-09-10T09:45:14.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:14.626+0000 7f39e1600640 1 --2- 172.21.15.5:0/2516384903 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f39c40a2c00 0x7f39c40a50c0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f39d4005e30 tx=0x7f39d4005d80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:45:14.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:14.627+0000 7f39e5fca740 1 -- start start 2024-09-10T09:45:14.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:14.627+0000 7f39e5fca740 1 Processor -- start 2024-09-10T09:45:14.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:14.627+0000 7f39e5fca740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fdeaccbc70 0x55fdeacd3cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:45:14.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:14.627+0000 7f39e5fca740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fdeaccc590 0x55fdeacd4230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:45:14.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:14.627+0000 7f39e5fca740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fdeacd6e50 0x55fdeacd4770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:45:14.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:14.627+0000 7f39e5fca740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55fdeacd9a20 con 0x55fdeacd6e50 2024-09-10T09:45:14.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:14.627+0000 7f39e5fca740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55fdeabed0f0 con 0x55fdeaccc590 2024-09-10T09:45:14.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:14.627+0000 7f39e5fca740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55fdeacd9ba0 con 0x55fdeaccbc70 2024-09-10T09:45:14.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:14.627+0000 7f39e5fca740 10 client.0 _ll_register_callbacks cb 0x55fdeabd4bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:45:14.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:14.628+0000 7f39e1600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fdeacd6e50 0x55fdeacd4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:45:14.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:14.628+0000 7f39e1600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fdeacd6e50 0x55fdeacd4770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34608/0 (socket says 172.21.15.5:34608) 2024-09-10T09:45:14.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:14.628+0000 7f39e1600640 1 -- 172.21.15.5:0/219483366 learned_addr learned my addr 172.21.15.5:0/219483366 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:45:14.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:14.628+0000 7f39e0c00640 1 --2- 172.21.15.5:0/219483366 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fdeaccbc70 0x55fdeacd3cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:45:14.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:14.628+0000 7f39dbe00640 1 --2- 172.21.15.5:0/219483366 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fdeaccc590 0x55fdeacd4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:45:14.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:14.628+0000 7f39e1600640 1 -- 172.21.15.5:0/219483366 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fdeaccbc70 msgr2=0x55fdeacd3cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:45:14.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:14.628+0000 7f39e1600640 1 --2- 172.21.15.5:0/219483366 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fdeaccbc70 0x55fdeacd3cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:14.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:14.628+0000 7f39e1600640 1 -- 172.21.15.5:0/219483366 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fdeaccc590 msgr2=0x55fdeacd4230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:45:14.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:14.628+0000 7f39e1600640 1 --2- 172.21.15.5:0/219483366 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fdeaccc590 0x55fdeacd4230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:14.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:14.628+0000 7f39e1600640 1 -- 172.21.15.5:0/219483366 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fdea9a33c0 con 0x55fdeacd6e50 2024-09-10T09:45:14.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:14.628+0000 7f39dbe00640 1 --2- 172.21.15.5:0/219483366 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fdeaccc590 0x55fdeacd4230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:45:14.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:14.629+0000 7f39e1600640 1 --2- 172.21.15.5:0/219483366 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fdeacd6e50 0x55fdeacd4770 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f39d4035000 tx=0x7f39d402a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3da94b24f4646a3e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:45:14.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:14.629+0000 7f39c2000640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-10T09:45:14.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:14.629+0000 7f39c2000640 1 -- 172.21.15.5:0/219483366 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39d4031660 con 0x55fdeacd6e50 2024-09-10T09:45:14.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:14.629+0000 7f39c2000640 1 -- 172.21.15.5:0/219483366 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f39d4031800 con 0x55fdeacd6e50 2024-09-10T09:45:14.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:14.629+0000 7f39e5fca740 1 -- 172.21.15.5:0/219483366 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f39d0019b10 con 0x55fdeacd6e50 2024-09-10T09:45:14.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:14.629+0000 7f39c2000640 1 -- 172.21.15.5:0/219483366 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f39d405c050 con 0x55fdeacd6e50 2024-09-10T09:45:14.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:14.629+0000 7f39e5fca740 10 client.16770 fetch_fsmap learned FSMap version 34 2024-09-10T09:45:14.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:14.629+0000 7f39e5fca740 1 -- 172.21.15.5:0/219483366 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55fdeace5a10 con 0x55fdeacd6e50 2024-09-10T09:45:14.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:14.630+0000 7f39c2000640 1 -- 172.21.15.5:0/219483366 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f39d0019b10 con 0x55fdeacd6e50 2024-09-10T09:45:14.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:14.630+0000 7f39c2000640 1 -- 172.21.15.5:0/219483366 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f39d4051020 con 0x55fdeacd6e50 2024-09-10T09:45:14.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:14.630+0000 7f39e5fca740 10 client.16770 fetch_fsmap finished waiting for FSMap version 34 2024-09-10T09:45:14.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:14.630+0000 7f39e5fca740 10 client.16770 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T09:45:14.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:14.630+0000 7f39e5fca740 20 client.16770 populate_metadata read hostname 'smithi005' 2024-09-10T09:45:14.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:14.630+0000 7f39e5fca740 1 --2- 172.21.15.5:0/219483366 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55fdeace6e00 0x55fdeace9230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:45:14.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:14.630+0000 7f39e5fca740 4 client.16770 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:45:14.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:14.630+0000 7f39e5fca740 1 -- 172.21.15.5:0/219483366 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fdeac3f950 con 0x55fdeace6e00 2024-09-10T09:45:14.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:14.630+0000 7f39e0c00640 1 --2- 172.21.15.5:0/219483366 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55fdeace6e00 0x55fdeace9230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:45:14.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:14.631+0000 7f39e0c00640 1 --2- 172.21.15.5:0/219483366 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55fdeace6e00 0x55fdeace9230 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4b11e28cda1af75c server_cookie=80d14f2ad805f176 in_seq=0 out_seq=0 2024-09-10T09:45:14.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:14.631+0000 7f39c2000640 10 client.16770 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T09:45:14.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:14.632+0000 7f39c2000640 1 -- 172.21.15.5:0/219483366 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55fdeac3f950 con 0x55fdeace6e00 2024-09-10T09:45:14.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:14.632+0000 7f39c2000640 10 client.16770 handle_command_reply: tid=0 2024-09-10T09:45:14.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:14.632+0000 7f39e5fca740 1 client.16770 shutdown 2024-09-10T09:45:14.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:14.632+0000 7f39e5fca740 1 -- 172.21.15.5:0/219483366 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fdeacd6e50 msgr2=0x55fdeacd4770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:45:14.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:14.632+0000 7f39e5fca740 1 --2- 172.21.15.5:0/219483366 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fdeacd6e50 0x55fdeacd4770 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f39d4035000 tx=0x7f39d402a8c0 comp rx=0 tx=0).stop 2024-09-10T09:45:14.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:14.632+0000 7f39e5fca740 1 -- 172.21.15.5:0/219483366 shutdown_connections 2024-09-10T09:45:14.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:14.632+0000 7f39e5fca740 1 --2- 172.21.15.5:0/219483366 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55fdeace6e00 0x55fdeace9230 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:14.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:14.632+0000 7f39e5fca740 1 --2- 172.21.15.5:0/219483366 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fdeacd6e50 0x55fdeacd4770 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:14.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:14.632+0000 7f39e5fca740 1 --2- 172.21.15.5:0/219483366 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fdeaccc590 0x55fdeacd4230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:14.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:14.632+0000 7f39e5fca740 1 --2- 172.21.15.5:0/219483366 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fdeaccbc70 0x55fdeacd3cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:14.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:14.632+0000 7f39e5fca740 1 -- 172.21.15.5:0/219483366 >> 172.21.15.5:0/219483366 conn(0x55fdeacc8200 msgr2=0x55fdeaccdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:45:14.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:14.632+0000 7f39c2000640 0 client.16770 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T09:45:14.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:14.632+0000 7f39e5fca740 1 -- 172.21.15.5:0/219483366 shutdown_connections 2024-09-10T09:45:14.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:14.632+0000 7f39e5fca740 1 -- 172.21.15.5:0/219483366 wait complete. 2024-09-10T09:45:14.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:14.632+0000 7f39e5fca740 20 client.16770 trim_cache size 0 max 16384 2024-09-10T09:45:14.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:14.646+0000 7f39e5fca740 1 Processor -- start 2024-09-10T09:45:14.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:14.646+0000 7f39e5fca740 1 -- start start 2024-09-10T09:45:14.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:14.646+0000 7f39e5fca740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fdeaccbc70 0x55fdeacce6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:45:14.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:14.646+0000 7f39e5fca740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fdeaccc590 0x55fdeacd5c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:45:14.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:14.646+0000 7f39e5fca740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fdeacd6e50 0x55fdeaceeb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:45:14.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:14.646+0000 7f39e5fca740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f39dc1b5500 con 0x55fdeaccc590 2024-09-10T09:45:14.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:14.646+0000 7f39e5fca740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f39dc1b7040 con 0x55fdeacd6e50 2024-09-10T09:45:14.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:14.646+0000 7f39e5fca740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f39dc1b5bd0 con 0x55fdeaccbc70 2024-09-10T09:45:14.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:14.647+0000 7f39dbe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fdeaccc590 0x55fdeacd5c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:45:14.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:14.647+0000 7f39dbe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fdeaccc590 0x55fdeacd5c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34622/0 (socket says 172.21.15.5:34622) 2024-09-10T09:45:14.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:14.647+0000 7f39dbe00640 1 -- 172.21.15.5:0/4089680514 learned_addr learned my addr 172.21.15.5:0/4089680514 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:45:14.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:14.647+0000 7f39e1600640 1 --2- 172.21.15.5:0/4089680514 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fdeacd6e50 0x55fdeaceeb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:45:14.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:14.647+0000 7f39e0c00640 1 --2- 172.21.15.5:0/4089680514 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fdeaccbc70 0x55fdeacce6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:45:14.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:14.647+0000 7f39e1600640 1 -- 172.21.15.5:0/4089680514 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fdeaccbc70 msgr2=0x55fdeacce6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:45:14.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:14.647+0000 7f39e1600640 1 --2- 172.21.15.5:0/4089680514 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fdeaccbc70 0x55fdeacce6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:14.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:14.647+0000 7f39e1600640 1 -- 172.21.15.5:0/4089680514 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fdeaccc590 msgr2=0x55fdeacd5c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:45:14.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:14.647+0000 7f39e1600640 1 --2- 172.21.15.5:0/4089680514 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fdeaccc590 0x55fdeacd5c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:14.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:14.647+0000 7f39e1600640 1 -- 172.21.15.5:0/4089680514 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fdeacc8030 con 0x55fdeacd6e50 2024-09-10T09:45:14.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:14.647+0000 7f39e1600640 1 --2- 172.21.15.5:0/4089680514 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fdeacd6e50 0x55fdeaceeb00 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f39d4006010 tx=0x7f39d4009880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cea5676ddfc09441 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:45:14.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:14.648+0000 7f39cd200640 1 -- 172.21.15.5:0/4089680514 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39d40305c0 con 0x55fdeacd6e50 2024-09-10T09:45:14.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:14.648+0000 7f39cd200640 1 -- 172.21.15.5:0/4089680514 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f39d4036050 con 0x55fdeacd6e50 2024-09-10T09:45:14.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:14.648+0000 7f39cd200640 1 -- 172.21.15.5:0/4089680514 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39d402fda0 con 0x55fdeacd6e50 2024-09-10T09:45:14.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:14.649+0000 7f39e5fca740 1 -- 172.21.15.5:0/4089680514 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fdeacd6e50 msgr2=0x55fdeaceeb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:45:14.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:14.649+0000 7f39e5fca740 1 --2- 172.21.15.5:0/4089680514 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fdeacd6e50 0x55fdeaceeb00 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f39d4006010 tx=0x7f39d4009880 comp rx=0 tx=0).stop 2024-09-10T09:45:14.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:14.649+0000 7f39e5fca740 1 -- 172.21.15.5:0/4089680514 shutdown_connections 2024-09-10T09:45:14.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:14.649+0000 7f39e5fca740 1 --2- 172.21.15.5:0/4089680514 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fdeacd6e50 0x55fdeaceeb00 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:14.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:14.649+0000 7f39e5fca740 1 --2- 172.21.15.5:0/4089680514 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fdeaccc590 0x55fdeacd5c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:14.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:14.649+0000 7f39e5fca740 1 --2- 172.21.15.5:0/4089680514 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fdeaccbc70 0x55fdeacce6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:14.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:14.649+0000 7f39e5fca740 1 -- 172.21.15.5:0/4089680514 >> 172.21.15.5:0/4089680514 conn(0x55fdeacc8200 msgr2=0x55fdeacd5550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:45:14.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:14.649+0000 7f39e5fca740 1 -- 172.21.15.5:0/4089680514 shutdown_connections 2024-09-10T09:45:14.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:14.649+0000 7f39e5fca740 1 -- 172.21.15.5:0/4089680514 wait complete. 2024-09-10T09:45:14.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:14.649+0000 7f39e5fca740 1 -- start start 2024-09-10T09:45:14.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:14.649+0000 7f39e5fca740 1 Processor -- start 2024-09-10T09:45:14.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:14.650+0000 7f39e5fca740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fdeaccbc70 0x55fdeacdf5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:45:14.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:14.650+0000 7f39e5fca740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fdeaccc590 0x55fdeacda8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:45:14.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:14.650+0000 7f39e5fca740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fdeacd6e50 0x55fdeacdae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:45:14.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:14.650+0000 7f39e5fca740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f39dc1b6970 con 0x55fdeacd6e50 2024-09-10T09:45:14.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:14.650+0000 7f39e5fca740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f39dc1b62a0 con 0x55fdeaccc590 2024-09-10T09:45:14.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:14.650+0000 7f39e5fca740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f39dc1b34e0 con 0x55fdeaccbc70 2024-09-10T09:45:14.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:14.650+0000 7f39e5fca740 10 client.0 _ll_register_callbacks cb 0x7f39d4009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:45:14.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:14.650+0000 7f39e1600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fdeacd6e50 0x55fdeacdae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:45:14.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:14.650+0000 7f39dbe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fdeaccc590 0x55fdeacda8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:45:14.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:14.650+0000 7f39e0c00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fdeaccbc70 0x55fdeacdf5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:45:14.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:14.650+0000 7f39e1600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fdeacd6e50 0x55fdeacdae10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34628/0 (socket says 172.21.15.5:34628) 2024-09-10T09:45:14.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:14.650+0000 7f39e1600640 1 -- 172.21.15.5:0/2505000690 learned_addr learned my addr 172.21.15.5:0/2505000690 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:45:14.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:14.651+0000 7f39dbe00640 1 -- 172.21.15.5:0/2505000690 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fdeaccbc70 msgr2=0x55fdeacdf5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:45:14.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:14.651+0000 7f39dbe00640 1 --2- 172.21.15.5:0/2505000690 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fdeaccbc70 0x55fdeacdf5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:14.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:14.651+0000 7f39dbe00640 1 -- 172.21.15.5:0/2505000690 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fdeacd6e50 msgr2=0x55fdeacdae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:45:14.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:14.651+0000 7f39dbe00640 1 --2- 172.21.15.5:0/2505000690 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fdeacd6e50 0x55fdeacdae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:14.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:14.651+0000 7f39dbe00640 1 -- 172.21.15.5:0/2505000690 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fdeacc3770 con 0x55fdeaccc590 2024-09-10T09:45:14.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:14.651+0000 7f39e1600640 1 --2- 172.21.15.5:0/2505000690 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fdeacd6e50 0x55fdeacdae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:45:14.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:14.651+0000 7f39e0c00640 1 --2- 172.21.15.5:0/2505000690 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fdeaccbc70 0x55fdeacdf5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:45:14.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:14.651+0000 7f39dbe00640 1 --2- 172.21.15.5:0/2505000690 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fdeaccc590 0x55fdeacda8d0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f39c809b820 tx=0x7f39c8097e80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a16d35170bbed06c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:45:14.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:14.651+0000 7f39c2000640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-09-10T09:45:14.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:14.651+0000 7f39c2000640 1 -- 172.21.15.5:0/2505000690 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39c809b480 con 0x55fdeaccc590 2024-09-10T09:45:14.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:14.651+0000 7f39c2000640 1 -- 172.21.15.5:0/2505000690 <== mon.2 v2:172.21.15.102:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f39c80cf2b0 con 0x55fdeaccc590 2024-09-10T09:45:14.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:14.651+0000 7f39e5fca740 1 -- 172.21.15.5:0/2505000690 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f39d4036050 con 0x55fdeaccc590 2024-09-10T09:45:14.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:14.652+0000 7f39c2000640 1 -- 172.21.15.5:0/2505000690 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f39c809bb20 con 0x55fdeaccc590 2024-09-10T09:45:14.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:14.652+0000 7f39e5fca740 10 client.26792 fetch_fsmap learned FSMap version 34 2024-09-10T09:45:14.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:14.652+0000 7f39e5fca740 1 -- 172.21.15.5:0/2505000690 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55fdead519c0 con 0x55fdeaccc590 2024-09-10T09:45:14.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:14.652+0000 7f39c2000640 1 -- 172.21.15.5:0/2505000690 <== mon.2 v2:172.21.15.102:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f39d4036050 con 0x55fdeaccc590 2024-09-10T09:45:14.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:14.652+0000 7f39c2000640 1 -- 172.21.15.5:0/2505000690 <== mon.2 v2:172.21.15.102:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f39c8010430 con 0x55fdeaccc590 2024-09-10T09:45:14.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:14.652+0000 7f39e5fca740 10 client.26792 fetch_fsmap finished waiting for FSMap version 34 2024-09-10T09:45:14.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:14.652+0000 7f39e5fca740 10 client.26792 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T09:45:14.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:14.652+0000 7f39e5fca740 20 client.26792 populate_metadata read hostname 'smithi005' 2024-09-10T09:45:14.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:14.652+0000 7f39e5fca740 1 --2- 172.21.15.5:0/2505000690 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55fdead52190 0x55fdead545b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:45:14.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:14.652+0000 7f39e5fca740 4 client.26792 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T09:45:14.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:14.652+0000 7f39e5fca740 1 -- 172.21.15.5:0/2505000690 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55fdeac39e60 con 0x55fdead52190 2024-09-10T09:45:14.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:14.653+0000 7f39e0c00640 1 --2- 172.21.15.5:0/2505000690 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55fdead52190 0x55fdead545b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:45:14.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:14.653+0000 7f39e0c00640 1 --2- 172.21.15.5:0/2505000690 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55fdead52190 0x55fdead545b0 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=39023050195ba4e6 server_cookie=4dd6da9171f4f900 in_seq=0 out_seq=0 2024-09-10T09:45:14.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:14.653+0000 7f39c2000640 10 client.26792 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T09:45:14.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:14.653+0000 7f39c2000640 1 -- 172.21.15.5:0/2505000690 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55fdeac39e60 con 0x55fdead52190 2024-09-10T09:45:14.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:14.653+0000 7f39c2000640 10 client.26792 handle_command_reply: tid=0 2024-09-10T09:45:14.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:14.653+0000 7f39e5fca740 1 client.26792 shutdown 2024-09-10T09:45:14.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:14.654+0000 7f39e5fca740 1 -- 172.21.15.5:0/2505000690 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fdeaccc590 msgr2=0x55fdeacda8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:45:14.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:14.654+0000 7f39e5fca740 1 --2- 172.21.15.5:0/2505000690 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fdeaccc590 0x55fdeacda8d0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f39c809b820 tx=0x7f39c8097e80 comp rx=0 tx=0).stop 2024-09-10T09:45:14.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:14.654+0000 7f39e5fca740 1 -- 172.21.15.5:0/2505000690 shutdown_connections 2024-09-10T09:45:14.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:14.654+0000 7f39e5fca740 1 --2- 172.21.15.5:0/2505000690 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55fdead52190 0x55fdead545b0 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:14.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:14.654+0000 7f39e5fca740 1 --2- 172.21.15.5:0/2505000690 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fdeacd6e50 0x55fdeacdae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:14.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:14.654+0000 7f39e5fca740 1 --2- 172.21.15.5:0/2505000690 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fdeaccc590 0x55fdeacda8d0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:14.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:14.654+0000 7f39e5fca740 1 --2- 172.21.15.5:0/2505000690 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fdeaccbc70 0x55fdeacdf5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:14.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:14.654+0000 7f39c2000640 0 client.26792 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T09:45:14.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:14.654+0000 7f39e5fca740 1 -- 172.21.15.5:0/2505000690 >> 172.21.15.5:0/2505000690 conn(0x55fdeacc8200 msgr2=0x55fdeacd5d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:45:14.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:14.654+0000 7f39e5fca740 1 -- 172.21.15.5:0/2505000690 shutdown_connections 2024-09-10T09:45:14.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:14.654+0000 7f39e5fca740 1 -- 172.21.15.5:0/2505000690 wait complete. 2024-09-10T09:45:14.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:14.654+0000 7f39e5fca740 20 client.26792 trim_cache size 0 max 16384 2024-09-10T09:45:14.660 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-10T09:45:14.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:14.655+0000 7f39e3600640 1 -- 172.21.15.5:0/2516384903 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f39c40a2c00 msgr2=0x7f39c40a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:45:14.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:14.655+0000 7f39e3600640 1 --2- 172.21.15.5:0/2516384903 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f39c40a2c00 0x7f39c40a50c0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f39d4005e30 tx=0x7f39d4005d80 comp rx=0 tx=0).stop 2024-09-10T09:45:14.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:14.655+0000 7f39e3600640 1 -- 172.21.15.5:0/2516384903 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f39dc077040 msgr2=0x7f39dc19f090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:45:14.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:14.655+0000 7f39e3600640 1 --2- 172.21.15.5:0/2516384903 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f39dc077040 0x7f39dc19f090 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f39c800e9f0 tx=0x7f39c800eeb0 comp rx=0 tx=0).stop 2024-09-10T09:45:14.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:14.655+0000 7f39e3600640 1 -- 172.21.15.5:0/2516384903 shutdown_connections 2024-09-10T09:45:14.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:14.655+0000 7f39e3600640 1 --2- 172.21.15.5:0/2516384903 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f39c40a2c00 0x7f39c40a50c0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:14.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:14.655+0000 7f39e3600640 1 --2- 172.21.15.5:0/2516384903 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f39dc112a50 0x7f39dc1a3450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:14.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:14.655+0000 7f39e3600640 1 --2- 172.21.15.5:0/2516384903 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f39dc077040 0x7f39dc19f090 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:14.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:14.655+0000 7f39e3600640 1 --2- 172.21.15.5:0/2516384903 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f39dc076720 0x7f39dc19eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:14.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:14.655+0000 7f39e3600640 1 -- 172.21.15.5:0/2516384903 >> 172.21.15.5:0/2516384903 conn(0x7f39dc0ffa50 msgr2=0x7f39dc101e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:45:14.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:14.655+0000 7f39e3600640 1 -- 172.21.15.5:0/2516384903 shutdown_connections 2024-09-10T09:45:14.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:14.655+0000 7f39e3600640 1 -- 172.21.15.5:0/2516384903 wait complete. 2024-09-10T09:45:15.055 INFO:tasks.workunit.client.0.smithi005.stdout: 1 359745 5.13 MB/sec execute 176 sec latency 15414.632 ms 2024-09-10T09:45:15.306 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:45:14 smithi005 ceph-mon[30350]: pgmap v600: 129 pgs: 129 active+clean; 142 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 200 KiB/s rd, 1018 KiB/s wr, 55 op/s 2024-09-10T09:45:15.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:45:14 smithi159 ceph-mon[35513]: pgmap v600: 129 pgs: 129 active+clean; 142 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 200 KiB/s rd, 1018 KiB/s wr, 55 op/s 2024-09-10T09:45:15.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:45:14 smithi102 ceph-mon[35442]: pgmap v600: 129 pgs: 129 active+clean; 142 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 200 KiB/s rd, 1018 KiB/s wr, 55 op/s 2024-09-10T09:45:16.055 INFO:tasks.workunit.client.0.smithi005.stdout: 1 359745 5.11 MB/sec execute 177 sec latency 16414.816 ms 2024-09-10T09:45:16.439 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-10T09:45:16.440 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-10T09:45:16.499 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T09:45:16.842 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:45:16.895 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:45:17.055 INFO:tasks.workunit.client.0.smithi005.stdout: 1 359745 5.08 MB/sec execute 178 sec latency 17414.865 ms 2024-09-10T09:45:17.265 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:45:16 smithi005 ceph-mon[30350]: pgmap v601: 129 pgs: 129 active+clean; 142 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 837 KiB/s wr, 48 op/s 2024-09-10T09:45:17.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:45:16 smithi159 ceph-mon[35513]: pgmap v601: 129 pgs: 129 active+clean; 142 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 837 KiB/s wr, 48 op/s 2024-09-10T09:45:17.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:45:16 smithi102 ceph-mon[35442]: pgmap v601: 129 pgs: 129 active+clean; 142 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 837 KiB/s wr, 48 op/s 2024-09-10T09:45:17.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:17.748+0000 7fd3f6200640 1 -- 172.21.15.5:0/2388760417 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd3f0103e50 msgr2=0x7fd3f0104230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:45:17.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:17.748+0000 7fd3f6200640 1 --2- 172.21.15.5:0/2388760417 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd3f0103e50 0x7fd3f0104230 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fd3e4009900 tx=0x7fd3e402a3f0 comp rx=0 tx=0).stop 2024-09-10T09:45:17.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:17.748+0000 7fd3f6200640 1 -- 172.21.15.5:0/2388760417 shutdown_connections 2024-09-10T09:45:17.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:17.748+0000 7fd3f6200640 1 --2- 172.21.15.5:0/2388760417 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd3f0109090 0x7fd3f0110c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:17.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:17.748+0000 7fd3f6200640 1 --2- 172.21.15.5:0/2388760417 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd3f0104800 0x7fd3f0108960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:17.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:17.748+0000 7fd3f6200640 1 --2- 172.21.15.5:0/2388760417 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd3f0103e50 0x7fd3f0104230 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:17.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:17.748+0000 7fd3f6200640 1 -- 172.21.15.5:0/2388760417 >> 172.21.15.5:0/2388760417 conn(0x7fd3f00ff8b0 msgr2=0x7fd3f0101cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:45:17.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:17.748+0000 7fd3f6200640 1 -- 172.21.15.5:0/2388760417 shutdown_connections 2024-09-10T09:45:17.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:17.748+0000 7fd3f6200640 1 -- 172.21.15.5:0/2388760417 wait complete. 2024-09-10T09:45:17.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:17.749+0000 7fd3f6200640 1 Processor -- start 2024-09-10T09:45:17.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:17.749+0000 7fd3f6200640 1 -- start start 2024-09-10T09:45:17.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:17.749+0000 7fd3f6200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd3f0103e50 0x7fd3f019a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:45:17.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:17.749+0000 7fd3f6200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd3f0104800 0x7fd3f019ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:45:17.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:17.749+0000 7fd3f6200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd3f0109090 0x7fd3f019ee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:45:17.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:17.749+0000 7fd3f6200640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd3f0114100 con 0x7fd3f0109090 2024-09-10T09:45:17.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:17.749+0000 7fd3f6200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fd3f0113f80 con 0x7fd3f0103e50 2024-09-10T09:45:17.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:17.749+0000 7fd3f6200640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fd3f0114280 con 0x7fd3f0104800 2024-09-10T09:45:17.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:17.750+0000 7fd3f5800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd3f0109090 0x7fd3f019ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:45:17.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:17.750+0000 7fd3f4e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd3f0103e50 0x7fd3f019a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:45:17.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:17.750+0000 7fd3f4e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd3f0103e50 0x7fd3f019a5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:34052/0 (socket says 172.21.15.5:34052) 2024-09-10T09:45:17.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:17.750+0000 7fd3f4e00640 1 -- 172.21.15.5:0/2625016128 learned_addr learned my addr 172.21.15.5:0/2625016128 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:45:17.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:17.750+0000 7fd3f5800640 1 --2- 172.21.15.5:0/2625016128 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd3f0109090 0x7fd3f019ee90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34632/0 (socket says 172.21.15.5:34632) 2024-09-10T09:45:17.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:17.750+0000 7fd3efe00640 1 --2- 172.21.15.5:0/2625016128 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd3f0104800 0x7fd3f019ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:45:17.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:17.750+0000 7fd3f5800640 1 -- 172.21.15.5:0/2625016128 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd3f0104800 msgr2=0x7fd3f019ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:45:17.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:17.750+0000 7fd3f5800640 1 --2- 172.21.15.5:0/2625016128 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd3f0104800 0x7fd3f019ab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:17.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:17.750+0000 7fd3f5800640 1 -- 172.21.15.5:0/2625016128 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd3f0103e50 msgr2=0x7fd3f019a5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:45:17.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:17.750+0000 7fd3f5800640 1 --2- 172.21.15.5:0/2625016128 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd3f0103e50 0x7fd3f019a5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:17.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:17.750+0000 7fd3f5800640 1 -- 172.21.15.5:0/2625016128 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3f019f570 con 0x7fd3f0109090 2024-09-10T09:45:17.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:17.751+0000 7fd3f5800640 1 --2- 172.21.15.5:0/2625016128 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd3f0109090 0x7fd3f019ee90 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fd3e800e940 tx=0x7fd3e800ee00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:45:17.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:17.751+0000 7fd3ed600640 1 -- 172.21.15.5:0/2625016128 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3e8004d20 con 0x7fd3f0109090 2024-09-10T09:45:17.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:17.751+0000 7fd3ed600640 1 -- 172.21.15.5:0/2625016128 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd3e8005020 con 0x7fd3f0109090 2024-09-10T09:45:17.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:17.751+0000 7fd3f6200640 1 -- 172.21.15.5:0/2625016128 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3f019f860 con 0x7fd3f0109090 2024-09-10T09:45:17.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:17.751+0000 7fd3ed600640 1 -- 172.21.15.5:0/2625016128 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3e80116b0 con 0x7fd3f0109090 2024-09-10T09:45:17.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:17.751+0000 7fd3f6200640 1 -- 172.21.15.5:0/2625016128 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3f01a7140 con 0x7fd3f0109090 2024-09-10T09:45:17.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:17.754+0000 7fd3ed600640 1 -- 172.21.15.5:0/2625016128 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7fd3e80054d0 con 0x7fd3f0109090 2024-09-10T09:45:17.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:17.754+0000 7fd3fa1e7740 1 Processor -- start 2024-09-10T09:45:17.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:17.754+0000 7fd3fa1e7740 1 -- start start 2024-09-10T09:45:17.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:17.755+0000 7fd3fa1e7740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cbc1738cb0 0x55cbc1739090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:45:17.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:17.755+0000 7fd3fa1e7740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cbc17395d0 0x55cbc1743950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:45:17.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:17.755+0000 7fd3fa1e7740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cbc1743e90 0x55cbc1746280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:45:17.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:17.755+0000 7fd3fa1e7740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55cbc169f290 con 0x55cbc1738cb0 2024-09-10T09:45:17.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:17.755+0000 7fd3fa1e7740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55cbc1646b40 con 0x55cbc1743e90 2024-09-10T09:45:17.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:17.755+0000 7fd3fa1e7740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55cbc169b250 con 0x55cbc17395d0 2024-09-10T09:45:17.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:17.755+0000 7fd3efe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cbc1738cb0 0x55cbc1739090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:45:17.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:17.755+0000 7fd3f4e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cbc17395d0 0x55cbc1743950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:45:17.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:17.755+0000 7fd3efe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cbc1738cb0 0x55cbc1739090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34642/0 (socket says 172.21.15.5:34642) 2024-09-10T09:45:17.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:17.755+0000 7fd3f4e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cbc17395d0 0x55cbc1743950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:49556/0 (socket says 172.21.15.5:49556) 2024-09-10T09:45:17.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:17.755+0000 7fd3efe00640 1 -- 172.21.15.5:0/2971499424 learned_addr learned my addr 172.21.15.5:0/2971499424 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:45:17.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:17.755+0000 7fd3efe00640 1 --2- 172.21.15.5:0/2971499424 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cbc1743e90 0x55cbc1746280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:45:17.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:17.755+0000 7fd3f4e00640 1 -- 172.21.15.5:0/2971499424 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cbc1743e90 msgr2=0x55cbc1746280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:45:17.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:17.755+0000 7fd3f4e00640 1 --2- 172.21.15.5:0/2971499424 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cbc1743e90 0x55cbc1746280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:17.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:17.755+0000 7fd3f4e00640 1 -- 172.21.15.5:0/2971499424 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cbc1738cb0 msgr2=0x55cbc1739090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:45:17.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:17.755+0000 7fd3f4e00640 1 --2- 172.21.15.5:0/2971499424 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cbc1738cb0 0x55cbc1739090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:17.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:17.755+0000 7fd3f4e00640 1 -- 172.21.15.5:0/2971499424 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cbc14106a0 con 0x55cbc17395d0 2024-09-10T09:45:17.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:17.756+0000 7fd3ed600640 1 --2- 172.21.15.5:0/2625016128 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fd3d80a2c00 0x7fd3d80a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:45:17.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:17.756+0000 7fd3ed600640 1 -- 172.21.15.5:0/2625016128 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fd3e80cf440 con 0x7fd3f0109090 2024-09-10T09:45:17.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:17.756+0000 7fd3f4e00640 1 --2- 172.21.15.5:0/2971499424 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cbc17395d0 0x55cbc1743950 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fd3e4031550 tx=0x7fd3e4005c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=304ddd95cee68435 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:45:17.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:17.756+0000 7fd3efe00640 1 --2- 172.21.15.5:0/2625016128 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fd3d80a2c00 0x7fd3d80a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:45:17.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:17.756+0000 7fd3e2000640 1 -- 172.21.15.5:0/2971499424 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3e4040070 con 0x55cbc17395d0 2024-09-10T09:45:17.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:17.756+0000 7fd3e2000640 1 -- 172.21.15.5:0/2971499424 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd3e40043c0 con 0x55cbc17395d0 2024-09-10T09:45:17.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:17.756+0000 7fd3e2000640 1 -- 172.21.15.5:0/2971499424 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3e4035040 con 0x55cbc17395d0 2024-09-10T09:45:17.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:17.757+0000 7fd3fa1e7740 1 -- 172.21.15.5:0/2971499424 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cbc17395d0 msgr2=0x55cbc1743950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:45:17.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:17.757+0000 7fd3efe00640 1 --2- 172.21.15.5:0/2625016128 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fd3d80a2c00 0x7fd3d80a50c0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x55cbc17365c0 tx=0x7fd3dc009780 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:45:17.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:17.757+0000 7fd3fa1e7740 1 --2- 172.21.15.5:0/2971499424 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cbc17395d0 0x55cbc1743950 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fd3e4031550 tx=0x7fd3e4005c00 comp rx=0 tx=0).stop 2024-09-10T09:45:17.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:17.757+0000 7fd3fa1e7740 1 -- 172.21.15.5:0/2971499424 shutdown_connections 2024-09-10T09:45:17.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:17.757+0000 7fd3fa1e7740 1 --2- 172.21.15.5:0/2971499424 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cbc1743e90 0x55cbc1746280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:17.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:17.757+0000 7fd3fa1e7740 1 --2- 172.21.15.5:0/2971499424 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cbc17395d0 0x55cbc1743950 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:17.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:17.757+0000 7fd3fa1e7740 1 --2- 172.21.15.5:0/2971499424 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cbc1738cb0 0x55cbc1739090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:17.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:17.757+0000 7fd3fa1e7740 1 -- 172.21.15.5:0/2971499424 >> 172.21.15.5:0/2971499424 conn(0x55cbc1735240 msgr2=0x55cbc1735620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:45:17.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:17.757+0000 7fd3fa1e7740 1 -- 172.21.15.5:0/2971499424 shutdown_connections 2024-09-10T09:45:17.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:17.757+0000 7fd3fa1e7740 1 -- 172.21.15.5:0/2971499424 wait complete. 2024-09-10T09:45:17.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:17.758+0000 7fd3fa1e7740 1 -- start start 2024-09-10T09:45:17.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:17.758+0000 7fd3fa1e7740 1 Processor -- start 2024-09-10T09:45:17.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:17.758+0000 7fd3fa1e7740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cbc1738cb0 0x55cbc1743790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:45:17.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:17.758+0000 7fd3fa1e7740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cbc17395d0 0x55cbc1740a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:45:17.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:17.758+0000 7fd3fa1e7740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cbc1743e90 0x55cbc1740f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:45:17.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:17.759+0000 7fd3fa1e7740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55cbc1746b00 con 0x55cbc1738cb0 2024-09-10T09:45:17.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:17.759+0000 7fd3fa1e7740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55cbc1659fa0 con 0x55cbc17395d0 2024-09-10T09:45:17.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:17.759+0000 7fd3fa1e7740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55cbc1746c80 con 0x55cbc1743e90 2024-09-10T09:45:17.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:17.759+0000 7fd3fa1e7740 10 client.0 _ll_register_callbacks cb 0x55cbc13adba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:45:17.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:17.759+0000 7fd3f4e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cbc1738cb0 0x55cbc1743790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:45:17.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:17.759+0000 7fd3f4e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cbc1738cb0 0x55cbc1743790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34644/0 (socket says 172.21.15.5:34644) 2024-09-10T09:45:17.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:17.759+0000 7fd3f4e00640 1 -- 172.21.15.5:0/1350838242 learned_addr learned my addr 172.21.15.5:0/1350838242 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:45:17.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:17.759+0000 7fd3efe00640 1 --2- 172.21.15.5:0/1350838242 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cbc17395d0 0x55cbc1740a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:45:17.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:17.759+0000 7fd3f5800640 1 --2- 172.21.15.5:0/1350838242 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cbc1743e90 0x55cbc1740f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:45:17.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:17.759+0000 7fd3f4e00640 1 -- 172.21.15.5:0/1350838242 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cbc1743e90 msgr2=0x55cbc1740f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:45:17.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:17.759+0000 7fd3f4e00640 1 --2- 172.21.15.5:0/1350838242 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cbc1743e90 0x55cbc1740f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:17.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:17.759+0000 7fd3f4e00640 1 -- 172.21.15.5:0/1350838242 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cbc17395d0 msgr2=0x55cbc1740a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:45:17.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:17.759+0000 7fd3f4e00640 1 --2- 172.21.15.5:0/1350838242 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cbc17395d0 0x55cbc1740a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:17.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:17.759+0000 7fd3f4e00640 1 -- 172.21.15.5:0/1350838242 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cbc17417d0 con 0x55cbc1738cb0 2024-09-10T09:45:17.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:17.759+0000 7fd3f5800640 1 --2- 172.21.15.5:0/1350838242 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cbc1743e90 0x55cbc1740f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T09:45:17.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:17.760+0000 7fd3f4e00640 1 --2- 172.21.15.5:0/1350838242 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cbc1738cb0 0x55cbc1743790 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fd3e402abe0 tx=0x7fd3e4009520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ba552a4f715a6e3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:45:17.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:17.760+0000 7fd3d6a00640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-10T09:45:17.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:17.760+0000 7fd3d6a00640 1 -- 172.21.15.5:0/1350838242 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3e404c020 con 0x55cbc1738cb0 2024-09-10T09:45:17.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:17.760+0000 7fd3d6a00640 1 -- 172.21.15.5:0/1350838242 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fd3e4005000 con 0x55cbc1738cb0 2024-09-10T09:45:17.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:17.760+0000 7fd3fa1e7740 1 -- 172.21.15.5:0/1350838242 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd3e40043c0 con 0x55cbc1738cb0 2024-09-10T09:45:17.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:17.760+0000 7fd3d6a00640 1 -- 172.21.15.5:0/1350838242 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fd3e4057050 con 0x55cbc1738cb0 2024-09-10T09:45:17.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:17.760+0000 7fd3fa1e7740 10 client.16788 fetch_fsmap learned FSMap version 34 2024-09-10T09:45:17.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:17.760+0000 7fd3fa1e7740 1 -- 172.21.15.5:0/1350838242 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55cbc17529e0 con 0x55cbc1738cb0 2024-09-10T09:45:17.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:17.760+0000 7fd3d6a00640 1 -- 172.21.15.5:0/1350838242 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd3e40043c0 con 0x55cbc1738cb0 2024-09-10T09:45:17.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:17.761+0000 7fd3d6a00640 1 -- 172.21.15.5:0/1350838242 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7fd3e4031ca0 con 0x55cbc1738cb0 2024-09-10T09:45:17.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:17.761+0000 7fd3fa1e7740 10 client.16788 fetch_fsmap finished waiting for FSMap version 34 2024-09-10T09:45:17.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:17.761+0000 7fd3fa1e7740 10 client.16788 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T09:45:17.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:17.761+0000 7fd3fa1e7740 20 client.16788 populate_metadata read hostname 'smithi005' 2024-09-10T09:45:17.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:17.761+0000 7fd3fa1e7740 1 --2- 172.21.15.5:0/1350838242 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55cbc1753d30 0x55cbc1756120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:45:17.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:17.761+0000 7fd3fa1e7740 4 client.16788 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:45:17.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:17.761+0000 7fd3fa1e7740 1 -- 172.21.15.5:0/1350838242 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cbc16ac800 con 0x55cbc1753d30 2024-09-10T09:45:17.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:17.761+0000 7fd3efe00640 1 --2- 172.21.15.5:0/1350838242 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55cbc1753d30 0x55cbc1756120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:45:17.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:17.762+0000 7fd3efe00640 1 --2- 172.21.15.5:0/1350838242 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55cbc1753d30 0x55cbc1756120 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cc839427f2dcadd5 server_cookie=e03bb738e00897f9 in_seq=0 out_seq=0 2024-09-10T09:45:17.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:17.762+0000 7fd3d6a00640 10 client.16788 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T09:45:17.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:17.762+0000 7fd3d6a00640 1 -- 172.21.15.5:0/1350838242 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55cbc16ac800 con 0x55cbc1753d30 2024-09-10T09:45:17.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:17.762+0000 7fd3d6a00640 10 client.16788 handle_command_reply: tid=0 2024-09-10T09:45:17.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:17.763+0000 7fd3fa1e7740 1 client.16788 shutdown 2024-09-10T09:45:17.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:17.763+0000 7fd3fa1e7740 1 -- 172.21.15.5:0/1350838242 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cbc1738cb0 msgr2=0x55cbc1743790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:45:17.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:17.763+0000 7fd3fa1e7740 1 --2- 172.21.15.5:0/1350838242 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cbc1738cb0 0x55cbc1743790 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fd3e402abe0 tx=0x7fd3e4009520 comp rx=0 tx=0).stop 2024-09-10T09:45:17.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:17.763+0000 7fd3fa1e7740 1 -- 172.21.15.5:0/1350838242 shutdown_connections 2024-09-10T09:45:17.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:17.763+0000 7fd3fa1e7740 1 --2- 172.21.15.5:0/1350838242 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55cbc1753d30 0x55cbc1756120 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:17.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:17.763+0000 7fd3fa1e7740 1 --2- 172.21.15.5:0/1350838242 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cbc1743e90 0x55cbc1740f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:17.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:17.763+0000 7fd3fa1e7740 1 --2- 172.21.15.5:0/1350838242 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cbc17395d0 0x55cbc1740a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:17.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:17.763+0000 7fd3fa1e7740 1 --2- 172.21.15.5:0/1350838242 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cbc1738cb0 0x55cbc1743790 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:17.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:17.763+0000 7fd3fa1e7740 1 -- 172.21.15.5:0/1350838242 >> 172.21.15.5:0/1350838242 conn(0x55cbc1735240 msgr2=0x55cbc173af70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:45:17.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:17.763+0000 7fd3d6a00640 0 client.16788 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T09:45:17.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:17.763+0000 7fd3fa1e7740 1 -- 172.21.15.5:0/1350838242 shutdown_connections 2024-09-10T09:45:17.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:17.763+0000 7fd3fa1e7740 1 -- 172.21.15.5:0/1350838242 wait complete. 2024-09-10T09:45:17.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:17.763+0000 7fd3fa1e7740 20 client.16788 trim_cache size 0 max 16384 2024-09-10T09:45:17.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:17.782+0000 7fd3fa1e7740 1 Processor -- start 2024-09-10T09:45:17.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:17.782+0000 7fd3fa1e7740 1 -- start start 2024-09-10T09:45:17.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:17.782+0000 7fd3fa1e7740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cbc1738cb0 0x55cbc1740e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:45:17.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:17.782+0000 7fd3fa1e7740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cbc17395d0 0x55cbc1734180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:45:17.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:17.782+0000 7fd3fa1e7740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cbc1743e90 0x55cbc175b780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:45:17.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:17.782+0000 7fd3fa1e7740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd3f01b0e20 con 0x55cbc1738cb0 2024-09-10T09:45:17.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:17.782+0000 7fd3fa1e7740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fd3f01b2960 con 0x55cbc17395d0 2024-09-10T09:45:17.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:17.782+0000 7fd3fa1e7740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fd3f01b14f0 con 0x55cbc1743e90 2024-09-10T09:45:17.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:17.783+0000 7fd3f4e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cbc1738cb0 0x55cbc1740e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:45:17.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:17.783+0000 7fd3f4e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cbc1738cb0 0x55cbc1740e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34660/0 (socket says 172.21.15.5:34660) 2024-09-10T09:45:17.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:17.783+0000 7fd3f4e00640 1 -- 172.21.15.5:0/648015335 learned_addr learned my addr 172.21.15.5:0/648015335 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:45:17.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:17.783+0000 7fd3efe00640 1 --2- 172.21.15.5:0/648015335 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cbc17395d0 0x55cbc1734180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:45:17.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:17.783+0000 7fd3f5800640 1 --2- 172.21.15.5:0/648015335 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cbc1743e90 0x55cbc175b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:45:17.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:17.783+0000 7fd3f4e00640 1 -- 172.21.15.5:0/648015335 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cbc1743e90 msgr2=0x55cbc175b780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:45:17.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:17.783+0000 7fd3f4e00640 1 --2- 172.21.15.5:0/648015335 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cbc1743e90 0x55cbc175b780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:17.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:17.783+0000 7fd3f4e00640 1 -- 172.21.15.5:0/648015335 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cbc17395d0 msgr2=0x55cbc1734180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:45:17.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:17.783+0000 7fd3f4e00640 1 --2- 172.21.15.5:0/648015335 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cbc17395d0 0x55cbc1734180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:17.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:17.783+0000 7fd3f4e00640 1 -- 172.21.15.5:0/648015335 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cbc173ef90 con 0x55cbc1738cb0 2024-09-10T09:45:17.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:17.783+0000 7fd3efe00640 1 --2- 172.21.15.5:0/648015335 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cbc17395d0 0x55cbc1734180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:45:17.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:17.783+0000 7fd3f4e00640 1 --2- 172.21.15.5:0/648015335 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cbc1738cb0 0x55cbc1740e60 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fd3e402a9e0 tx=0x7fd3e4005c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5aad4fe565fcec73 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:45:17.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:17.784+0000 7fd3e1600640 1 -- 172.21.15.5:0/648015335 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3e404c030 con 0x55cbc1738cb0 2024-09-10T09:45:17.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:17.784+0000 7fd3e1600640 1 -- 172.21.15.5:0/648015335 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd3e4003860 con 0x55cbc1738cb0 2024-09-10T09:45:17.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:17.784+0000 7fd3e1600640 1 -- 172.21.15.5:0/648015335 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3e4040070 con 0x55cbc1738cb0 2024-09-10T09:45:17.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:17.784+0000 7fd3fa1e7740 1 -- 172.21.15.5:0/648015335 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cbc1738cb0 msgr2=0x55cbc1740e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:45:17.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:17.784+0000 7fd3fa1e7740 1 --2- 172.21.15.5:0/648015335 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cbc1738cb0 0x55cbc1740e60 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fd3e402a9e0 tx=0x7fd3e4005c00 comp rx=0 tx=0).stop 2024-09-10T09:45:17.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:17.785+0000 7fd3fa1e7740 1 -- 172.21.15.5:0/648015335 shutdown_connections 2024-09-10T09:45:17.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:17.785+0000 7fd3fa1e7740 1 --2- 172.21.15.5:0/648015335 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cbc1743e90 0x55cbc175b780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:17.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:17.785+0000 7fd3fa1e7740 1 --2- 172.21.15.5:0/648015335 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cbc17395d0 0x55cbc1734180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:17.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:17.785+0000 7fd3fa1e7740 1 --2- 172.21.15.5:0/648015335 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cbc1738cb0 0x55cbc1740e60 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:17.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:17.785+0000 7fd3fa1e7740 1 -- 172.21.15.5:0/648015335 >> 172.21.15.5:0/648015335 conn(0x55cbc1735240 msgr2=0x55cbc1733a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:45:17.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:17.785+0000 7fd3fa1e7740 1 -- 172.21.15.5:0/648015335 shutdown_connections 2024-09-10T09:45:17.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:17.785+0000 7fd3fa1e7740 1 -- 172.21.15.5:0/648015335 wait complete. 2024-09-10T09:45:17.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:17.785+0000 7fd3fa1e7740 1 -- start start 2024-09-10T09:45:17.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:17.785+0000 7fd3fa1e7740 1 Processor -- start 2024-09-10T09:45:17.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:17.786+0000 7fd3fa1e7740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cbc1738cb0 0x55cbc1749c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:45:17.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:17.786+0000 7fd3fa1e7740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cbc17395d0 0x55cbc174a150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:45:17.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:17.786+0000 7fd3fa1e7740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cbc1743e90 0x55cbc174a690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:45:17.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:17.786+0000 7fd3fa1e7740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd3f01b2290 con 0x55cbc17395d0 2024-09-10T09:45:17.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:17.786+0000 7fd3fa1e7740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fd3f01b1bc0 con 0x55cbc1743e90 2024-09-10T09:45:17.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:17.786+0000 7fd3fa1e7740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fd3f01aee00 con 0x55cbc1738cb0 2024-09-10T09:45:17.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:17.786+0000 7fd3fa1e7740 10 client.0 _ll_register_callbacks cb 0x55cbc17419a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:45:17.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:17.786+0000 7fd3efe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cbc17395d0 0x55cbc174a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:45:17.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:17.786+0000 7fd3f4e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cbc1738cb0 0x55cbc1749c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:45:17.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:17.786+0000 7fd3f4e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cbc1738cb0 0x55cbc1749c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:49574/0 (socket says 172.21.15.5:49574) 2024-09-10T09:45:17.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:17.786+0000 7fd3efe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cbc17395d0 0x55cbc174a150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34664/0 (socket says 172.21.15.5:34664) 2024-09-10T09:45:17.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:17.786+0000 7fd3f4e00640 1 -- 172.21.15.5:0/503353147 learned_addr learned my addr 172.21.15.5:0/503353147 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:45:17.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:17.786+0000 7fd3f5800640 1 --2- 172.21.15.5:0/503353147 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cbc1743e90 0x55cbc174a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:45:17.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:17.786+0000 7fd3efe00640 1 -- 172.21.15.5:0/503353147 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cbc1738cb0 msgr2=0x55cbc1749c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:45:17.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:17.786+0000 7fd3efe00640 1 --2- 172.21.15.5:0/503353147 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cbc1738cb0 0x55cbc1749c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:17.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:17.786+0000 7fd3efe00640 1 -- 172.21.15.5:0/503353147 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cbc1743e90 msgr2=0x55cbc174a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:45:17.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:17.787+0000 7fd3efe00640 1 --2- 172.21.15.5:0/503353147 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cbc1743e90 0x55cbc174a690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:17.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:17.787+0000 7fd3efe00640 1 -- 172.21.15.5:0/503353147 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cbc17bd760 con 0x55cbc17395d0 2024-09-10T09:45:17.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:17.787+0000 7fd3efe00640 1 --2- 172.21.15.5:0/503353147 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cbc17395d0 0x55cbc174a150 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x55cbc1742080 tx=0x7fd3dc004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5672bd6bb457f02 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:45:17.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:17.787+0000 7fd3d6a00640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-10T09:45:17.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:17.787+0000 7fd3d6a00640 1 -- 172.21.15.5:0/503353147 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3dc003e90 con 0x55cbc17395d0 2024-09-10T09:45:17.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:17.787+0000 7fd3d6a00640 1 -- 172.21.15.5:0/503353147 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fd3dc020a30 con 0x55cbc17395d0 2024-09-10T09:45:17.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:17.787+0000 7fd3fa1e7740 1 -- 172.21.15.5:0/503353147 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd3e4003860 con 0x55cbc17395d0 2024-09-10T09:45:17.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:17.787+0000 7fd3d6a00640 1 -- 172.21.15.5:0/503353147 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fd3dc03f050 con 0x55cbc17395d0 2024-09-10T09:45:17.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:17.788+0000 7fd3fa1e7740 10 client.16794 fetch_fsmap learned FSMap version 34 2024-09-10T09:45:17.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:17.788+0000 7fd3fa1e7740 1 -- 172.21.15.5:0/503353147 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55cbc17beab0 con 0x55cbc17395d0 2024-09-10T09:45:17.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:17.788+0000 7fd3d6a00640 1 -- 172.21.15.5:0/503353147 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd3e4003860 con 0x55cbc17395d0 2024-09-10T09:45:17.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:17.788+0000 7fd3d6a00640 1 -- 172.21.15.5:0/503353147 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7fd3dc0048c0 con 0x55cbc17395d0 2024-09-10T09:45:17.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:17.788+0000 7fd3fa1e7740 10 client.16794 fetch_fsmap finished waiting for FSMap version 34 2024-09-10T09:45:17.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:17.788+0000 7fd3fa1e7740 10 client.16794 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T09:45:17.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:17.788+0000 7fd3fa1e7740 20 client.16794 populate_metadata read hostname 'smithi005' 2024-09-10T09:45:17.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:17.788+0000 7fd3fa1e7740 1 --2- 172.21.15.5:0/503353147 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55cbc17bf210 0x55cbc17c1630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:45:17.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:17.788+0000 7fd3fa1e7740 4 client.16794 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-10T09:45:17.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:17.788+0000 7fd3fa1e7740 1 -- 172.21.15.5:0/503353147 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55cbc16a6d10 con 0x55cbc17bf210 2024-09-10T09:45:17.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:17.788+0000 7fd3f4e00640 1 --2- 172.21.15.5:0/503353147 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55cbc17bf210 0x55cbc17c1630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:45:17.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:17.789+0000 7fd3f4e00640 1 --2- 172.21.15.5:0/503353147 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55cbc17bf210 0x55cbc17c1630 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f2b9546bffc97654 server_cookie=187c89294e303d44 in_seq=0 out_seq=0 2024-09-10T09:45:17.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:17.789+0000 7fd3d6a00640 10 client.16794 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T09:45:17.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:17.790+0000 7fd3d6a00640 1 -- 172.21.15.5:0/503353147 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55cbc16a6d10 con 0x55cbc17bf210 2024-09-10T09:45:17.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:17.790+0000 7fd3d6a00640 10 client.16794 handle_command_reply: tid=0 2024-09-10T09:45:17.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:17.790+0000 7fd3fa1e7740 1 client.16794 shutdown 2024-09-10T09:45:17.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:17.790+0000 7fd3fa1e7740 1 -- 172.21.15.5:0/503353147 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cbc17395d0 msgr2=0x55cbc174a150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:45:17.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:17.790+0000 7fd3fa1e7740 1 --2- 172.21.15.5:0/503353147 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cbc17395d0 0x55cbc174a150 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x55cbc1742080 tx=0x7fd3dc004730 comp rx=0 tx=0).stop 2024-09-10T09:45:17.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:17.791+0000 7fd3fa1e7740 1 -- 172.21.15.5:0/503353147 shutdown_connections 2024-09-10T09:45:17.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:17.791+0000 7fd3fa1e7740 1 --2- 172.21.15.5:0/503353147 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55cbc17bf210 0x55cbc17c1630 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:17.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:17.791+0000 7fd3fa1e7740 1 --2- 172.21.15.5:0/503353147 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cbc1743e90 0x55cbc174a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:17.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:17.791+0000 7fd3d6a00640 0 client.16794 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T09:45:17.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:17.791+0000 7fd3fa1e7740 1 --2- 172.21.15.5:0/503353147 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cbc17395d0 0x55cbc174a150 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:17.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:17.791+0000 7fd3fa1e7740 1 --2- 172.21.15.5:0/503353147 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cbc1738cb0 0x55cbc1749c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:17.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:17.791+0000 7fd3fa1e7740 1 -- 172.21.15.5:0/503353147 >> 172.21.15.5:0/503353147 conn(0x55cbc1735240 msgr2=0x55cbc1734290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:45:17.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:17.791+0000 7fd3fa1e7740 1 -- 172.21.15.5:0/503353147 shutdown_connections 2024-09-10T09:45:17.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:17.791+0000 7fd3fa1e7740 1 -- 172.21.15.5:0/503353147 wait complete. 2024-09-10T09:45:17.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:17.791+0000 7fd3fa1e7740 20 client.16794 trim_cache size 0 max 16384 2024-09-10T09:45:17.801 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-10T09:45:17.801 INFO:teuthology.orchestra.run.smithi005.stdout: "return_code": 0, 2024-09-10T09:45:17.801 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_tag": "0758dbc4-f003-42a3-b9ec-fa47855caff3", 2024-09-10T09:45:17.801 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": "asynchronous" 2024-09-10T09:45:17.801 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-10T09:45:17.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:17.792+0000 7fd3f6200640 1 -- 172.21.15.5:0/2625016128 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fd3d80a2c00 msgr2=0x7fd3d80a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:45:17.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:17.792+0000 7fd3f6200640 1 --2- 172.21.15.5:0/2625016128 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fd3d80a2c00 0x7fd3d80a50c0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x55cbc17365c0 tx=0x7fd3dc009780 comp rx=0 tx=0).stop 2024-09-10T09:45:17.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:17.792+0000 7fd3f6200640 1 -- 172.21.15.5:0/2625016128 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd3f0109090 msgr2=0x7fd3f019ee90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:45:17.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:17.792+0000 7fd3f6200640 1 --2- 172.21.15.5:0/2625016128 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd3f0109090 0x7fd3f019ee90 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fd3e800e940 tx=0x7fd3e800ee00 comp rx=0 tx=0).stop 2024-09-10T09:45:17.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:17.792+0000 7fd3f6200640 1 -- 172.21.15.5:0/2625016128 shutdown_connections 2024-09-10T09:45:17.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:17.792+0000 7fd3f6200640 1 --2- 172.21.15.5:0/2625016128 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fd3d80a2c00 0x7fd3d80a50c0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:17.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:17.792+0000 7fd3f6200640 1 --2- 172.21.15.5:0/2625016128 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd3f0109090 0x7fd3f019ee90 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:17.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:17.792+0000 7fd3f6200640 1 --2- 172.21.15.5:0/2625016128 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd3f0104800 0x7fd3f019ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:17.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:17.792+0000 7fd3f6200640 1 --2- 172.21.15.5:0/2625016128 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd3f0103e50 0x7fd3f019a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:17.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:17.792+0000 7fd3f6200640 1 -- 172.21.15.5:0/2625016128 >> 172.21.15.5:0/2625016128 conn(0x7fd3f00ff8b0 msgr2=0x7fd3f0100bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:45:17.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:17.792+0000 7fd3f6200640 1 -- 172.21.15.5:0/2625016128 shutdown_connections 2024-09-10T09:45:17.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:17.792+0000 7fd3f6200640 1 -- 172.21.15.5:0/2625016128 wait complete. 2024-09-10T09:45:18.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:18.033+0000 7ff588e00640 1 -- 172.21.15.5:0/1808248208 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff584103e30 msgr2=0x7ff584104210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:45:18.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:18.033+0000 7ff588e00640 1 --2- 172.21.15.5:0/1808248208 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff584103e30 0x7ff584104210 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7ff574009900 tx=0x7ff57402a3f0 comp rx=0 tx=0).stop 2024-09-10T09:45:18.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:18.033+0000 7ff588e00640 1 -- 172.21.15.5:0/1808248208 shutdown_connections 2024-09-10T09:45:18.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:18.033+0000 7ff588e00640 1 --2- 172.21.15.5:0/1808248208 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff584108f60 0x7ff584110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:18.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:18.033+0000 7ff588e00640 1 --2- 172.21.15.5:0/1808248208 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff5841047e0 0x7ff584108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:18.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:18.033+0000 7ff588e00640 1 --2- 172.21.15.5:0/1808248208 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff584103e30 0x7ff584104210 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:18.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:18.033+0000 7ff588e00640 1 -- 172.21.15.5:0/1808248208 >> 172.21.15.5:0/1808248208 conn(0x7ff5840ff8d0 msgr2=0x7ff584101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:45:18.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:18.033+0000 7ff588e00640 1 -- 172.21.15.5:0/1808248208 shutdown_connections 2024-09-10T09:45:18.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:18.033+0000 7ff588e00640 1 -- 172.21.15.5:0/1808248208 wait complete. 2024-09-10T09:45:18.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:18.034+0000 7ff588e00640 1 Processor -- start 2024-09-10T09:45:18.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:18.034+0000 7ff588e00640 1 -- start start 2024-09-10T09:45:18.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:18.035+0000 7ff588e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff584103e30 0x7ff58419a450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:45:18.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:18.035+0000 7ff588e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff5841047e0 0x7ff58419a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:45:18.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:18.035+0000 7ff588e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff584108f60 0x7ff58419ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:45:18.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:18.035+0000 7ff588e00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff584114010 con 0x7ff584103e30 2024-09-10T09:45:18.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:18.035+0000 7ff588e00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7ff584113e90 con 0x7ff584108f60 2024-09-10T09:45:18.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:18.035+0000 7ff588e00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7ff584114190 con 0x7ff5841047e0 2024-09-10T09:45:18.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:18.035+0000 7ff583e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff584108f60 0x7ff58419ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:45:18.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:18.035+0000 7ff583400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff584103e30 0x7ff58419a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:45:18.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:18.035+0000 7ff583e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff584108f60 0x7ff58419ed20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:34114/0 (socket says 172.21.15.5:34114) 2024-09-10T09:45:18.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:18.035+0000 7ff583e00640 1 -- 172.21.15.5:0/1127845971 learned_addr learned my addr 172.21.15.5:0/1127845971 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:45:18.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:18.035+0000 7ff582a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff5841047e0 0x7ff58419a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:45:18.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:18.036+0000 7ff583e00640 1 -- 172.21.15.5:0/1127845971 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff5841047e0 msgr2=0x7ff58419a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:45:18.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:18.036+0000 7ff583e00640 1 --2- 172.21.15.5:0/1127845971 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff5841047e0 0x7ff58419a990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:18.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:18.036+0000 7ff583e00640 1 -- 172.21.15.5:0/1127845971 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff584103e30 msgr2=0x7ff58419a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:45:18.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:18.036+0000 7ff583e00640 1 --2- 172.21.15.5:0/1127845971 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff584103e30 0x7ff58419a450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:18.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:18.036+0000 7ff583e00640 1 -- 172.21.15.5:0/1127845971 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff58419f400 con 0x7ff584108f60 2024-09-10T09:45:18.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:18.036+0000 7ff583e00640 1 --2- 172.21.15.5:0/1127845971 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff584108f60 0x7ff58419ed20 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7ff57c00eb10 tx=0x7ff57c00c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:45:18.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:18.036+0000 7ff57be00640 1 -- 172.21.15.5:0/1127845971 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff57c004b20 con 0x7ff584108f60 2024-09-10T09:45:18.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:18.036+0000 7ff57be00640 1 -- 172.21.15.5:0/1127845971 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff57c004cc0 con 0x7ff584108f60 2024-09-10T09:45:18.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:18.036+0000 7ff588e00640 1 -- 172.21.15.5:0/1127845971 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff58419f6f0 con 0x7ff584108f60 2024-09-10T09:45:18.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:18.036+0000 7ff57be00640 1 -- 172.21.15.5:0/1127845971 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff57c010760 con 0x7ff584108f60 2024-09-10T09:45:18.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:18.037+0000 7ff588e00640 1 -- 172.21.15.5:0/1127845971 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5841a6fd0 con 0x7ff584108f60 2024-09-10T09:45:18.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:18.038+0000 7ff588e00640 1 -- 172.21.15.5:0/1127845971 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff584104210 con 0x7ff584108f60 2024-09-10T09:45:18.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:18.039+0000 7ff57be00640 1 -- 172.21.15.5:0/1127845971 <== mon.2 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7ff57c005020 con 0x7ff584108f60 2024-09-10T09:45:18.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:18.041+0000 7ff57be00640 1 --2- 172.21.15.5:0/1127845971 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7ff5680a2c00 0x7ff5680a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:45:18.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:18.041+0000 7ff57be00640 1 -- 172.21.15.5:0/1127845971 <== mon.2 v2:172.21.15.102:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7ff57c0cf920 con 0x7ff584108f60 2024-09-10T09:45:18.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:18.041+0000 7ff583400640 1 --2- 172.21.15.5:0/1127845971 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7ff5680a2c00 0x7ff5680a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:45:18.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:18.042+0000 7ff583400640 1 --2- 172.21.15.5:0/1127845971 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7ff5680a2c00 0x7ff5680a50c0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7ff574000c00 tx=0x7ff574005b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:45:18.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:18.045+0000 7ff57be00640 1 -- 172.21.15.5:0/1127845971 <== mon.2 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff57c09c430 con 0x7ff584108f60 2024-09-10T09:45:18.055 INFO:tasks.workunit.client.0.smithi005.stdout: 1 359745 5.05 MB/sec execute 179 sec latency 18414.906 ms 2024-09-10T09:45:18.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:18.269+0000 7ff588e00640 1 -- 172.21.15.5:0/1127845971 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff58402d7c0 con 0x7ff584108f60 2024-09-10T09:45:18.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:18.270+0000 7ff57be00640 1 -- 172.21.15.5:0/1127845971 <== mon.2 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+11969 (secure 0 0 0) 0x7ff57c09bd30 con 0x7ff584108f60 2024-09-10T09:45:18.272 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-10T09:45:18.272 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":34,"btime":"2024-09-10T09:42:57:737230+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14454,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/2970594311","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2970594311},{"type":"v1","addr":"172.21.15.5:6835","nonce":2970594311}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14514,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6839/3616502174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":3616502174},{"type":"v1","addr":"172.21.15.5:6839","nonce":3616502174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":34,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T09:35:29.626287+0000","modified":"2024-09-10T09:42:56.961831+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24542,"mds_1":24535,"mds_2":24511,"mds_3":24506,"mds_4":24443},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14520":{"gid":14520,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/2073599842","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":2073599842},{"type":"v1","addr":"172.21.15.5:6841","nonce":2073599842}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"k","rank":4,"incarnation":26,"state":"up:active","state_seq":24,"addr":"172.21.15.102:6833/2457007427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":2457007427},{"type":"v1","addr":"172.21.15.102:6833","nonce":2457007427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/356445159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":356445159},{"type":"v1","addr":"172.21.15.159:6833","nonce":356445159}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6837/2894314507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":2894314507},{"type":"v1","addr":"172.21.15.5:6837","nonce":2894314507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.102:6835/1647549111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":1647549111},{"type":"v1","addr":"172.21.15.102:6835","nonce":1647549111}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6835/952657242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":952657242},{"type":"v1","addr":"172.21.15.159:6835","nonce":952657242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":19,"addr":"172.21.15.102:6837/2689776764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":2689776764},{"type":"v1","addr":"172.21.15.102:6837","nonce":2689776764}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.159:6837/620627639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":620627639},{"type":"v1","addr":"172.21.15.159:6837","nonce":620627639}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24535":{"gid":24535,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":18,"addr":"172.21.15.159:6839/2073739599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":2073739599},{"type":"v1","addr":"172.21.15.159:6839","nonce":2073739599}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24542":{"gid":24542,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.102:6839/335409749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6838","nonce":335409749},{"type":"v1","addr":"172.21.15.102:6839","nonce":335409749}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24542,"qdb_cluster":[24542,24535,24511,24506,24443]},"id":1}]} 2024-09-10T09:45:18.272 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 34 2024-09-10T09:45:18.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:18.273+0000 7ff588e00640 1 -- 172.21.15.5:0/1127845971 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7ff5680a2c00 msgr2=0x7ff5680a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:45:18.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:18.273+0000 7ff588e00640 1 --2- 172.21.15.5:0/1127845971 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7ff5680a2c00 0x7ff5680a50c0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7ff574000c00 tx=0x7ff574005b50 comp rx=0 tx=0).stop 2024-09-10T09:45:18.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:18.273+0000 7ff588e00640 1 -- 172.21.15.5:0/1127845971 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff584108f60 msgr2=0x7ff58419ed20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:45:18.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:18.273+0000 7ff588e00640 1 --2- 172.21.15.5:0/1127845971 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff584108f60 0x7ff58419ed20 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7ff57c00eb10 tx=0x7ff57c00c2d0 comp rx=0 tx=0).stop 2024-09-10T09:45:18.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:18.273+0000 7ff588e00640 1 -- 172.21.15.5:0/1127845971 shutdown_connections 2024-09-10T09:45:18.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:18.273+0000 7ff588e00640 1 --2- 172.21.15.5:0/1127845971 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7ff5680a2c00 0x7ff5680a50c0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:18.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:18.273+0000 7ff588e00640 1 --2- 172.21.15.5:0/1127845971 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff584108f60 0x7ff58419ed20 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:18.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:18.273+0000 7ff588e00640 1 --2- 172.21.15.5:0/1127845971 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff5841047e0 0x7ff58419a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:18.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:18.273+0000 7ff588e00640 1 --2- 172.21.15.5:0/1127845971 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff584103e30 0x7ff58419a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:18.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:18.273+0000 7ff588e00640 1 -- 172.21.15.5:0/1127845971 >> 172.21.15.5:0/1127845971 conn(0x7ff5840ff8d0 msgr2=0x7ff584100b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:45:18.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:18.273+0000 7ff588e00640 1 -- 172.21.15.5:0/1127845971 shutdown_connections 2024-09-10T09:45:18.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:18.273+0000 7ff588e00640 1 -- 172.21.15.5:0/1127845971 wait complete. 2024-09-10T09:45:19.056 INFO:tasks.workunit.client.0.smithi005.stdout: 1 359745 5.02 MB/sec execute 180 sec latency 19414.949 ms 2024-09-10T09:45:19.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:45:18 smithi005 ceph-mon[30350]: pgmap v602: 129 pgs: 129 active+clean; 142 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 521 KiB/s wr, 34 op/s 2024-09-10T09:45:19.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:45:18 smithi005 ceph-mon[30350]: scrub queued for path: / 2024-09-10T09:45:19.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:45:18 smithi005 ceph-mon[30350]: scrub summary: idle+waiting paths [/] 2024-09-10T09:45:19.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:45:18 smithi005 ceph-mon[30350]: scrub summary: active paths [/] 2024-09-10T09:45:19.057 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:45:18 smithi005 ceph-mon[30350]: scrub summary: active paths [/] 2024-09-10T09:45:19.057 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:45:18 smithi005 ceph-mon[30350]: scrub summary: idle+waiting paths [/] 2024-09-10T09:45:19.057 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:45:18 smithi005 ceph-mon[30350]: scrub summary: idle+waiting paths [/] 2024-09-10T09:45:19.057 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:45:18 smithi005 ceph-mon[30350]: from='client.? 172.21.15.5:0/1127845971' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:45:19.231 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24542 quiesce db --set-id 36f005be --release --await' 2024-09-10T09:45:19.231 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24542 quiesce db --set-id 36f005be --release --await 2024-09-10T09:45:19.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:45:18 smithi159 ceph-mon[35513]: pgmap v602: 129 pgs: 129 active+clean; 142 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 521 KiB/s wr, 34 op/s 2024-09-10T09:45:19.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:45:18 smithi159 ceph-mon[35513]: scrub queued for path: / 2024-09-10T09:45:19.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:45:18 smithi159 ceph-mon[35513]: scrub summary: idle+waiting paths [/] 2024-09-10T09:45:19.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:45:18 smithi159 ceph-mon[35513]: scrub summary: active paths [/] 2024-09-10T09:45:19.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:45:18 smithi159 ceph-mon[35513]: scrub summary: active paths [/] 2024-09-10T09:45:19.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:45:18 smithi159 ceph-mon[35513]: scrub summary: idle+waiting paths [/] 2024-09-10T09:45:19.351 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:45:18 smithi159 ceph-mon[35513]: scrub summary: idle+waiting paths [/] 2024-09-10T09:45:19.351 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:45:18 smithi159 ceph-mon[35513]: from='client.? 172.21.15.5:0/1127845971' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:45:19.385 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:45:18 smithi102 ceph-mon[35442]: pgmap v602: 129 pgs: 129 active+clean; 142 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 521 KiB/s wr, 34 op/s 2024-09-10T09:45:19.385 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:45:18 smithi102 ceph-mon[35442]: scrub queued for path: / 2024-09-10T09:45:19.385 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:45:18 smithi102 ceph-mon[35442]: scrub summary: idle+waiting paths [/] 2024-09-10T09:45:19.385 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:45:18 smithi102 ceph-mon[35442]: scrub summary: active paths [/] 2024-09-10T09:45:19.385 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:45:18 smithi102 ceph-mon[35442]: scrub summary: active paths [/] 2024-09-10T09:45:19.385 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:45:18 smithi102 ceph-mon[35442]: scrub summary: idle+waiting paths [/] 2024-09-10T09:45:19.385 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:45:18 smithi102 ceph-mon[35442]: scrub summary: idle+waiting paths [/] 2024-09-10T09:45:19.385 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:45:18 smithi102 ceph-mon[35442]: from='client.? 172.21.15.5:0/1127845971' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:45:19.752 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T09:45:19.801 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:45:20.056 INFO:tasks.workunit.client.0.smithi005.stdout: 1 359745 4.99 MB/sec execute 181 sec latency 20415.072 ms 2024-09-10T09:45:20.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:20.388+0000 7f141e200640 1 -- 172.21.15.5:0/702687257 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1418074ee0 msgr2=0x7f141810dee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:45:20.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:20.388+0000 7f141e200640 1 --2- 172.21.15.5:0/702687257 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1418074ee0 0x7f141810dee0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f1404009900 tx=0x7f140402a480 comp rx=0 tx=0).stop 2024-09-10T09:45:20.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:20.389+0000 7f141e200640 1 -- 172.21.15.5:0/702687257 shutdown_connections 2024-09-10T09:45:20.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:20.389+0000 7f141e200640 1 --2- 172.21.15.5:0/702687257 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f141810e420 0x7f1418110810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:20.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:20.389+0000 7f141e200640 1 --2- 172.21.15.5:0/702687257 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1418074ee0 0x7f141810dee0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:20.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:20.389+0000 7f141e200640 1 --2- 172.21.15.5:0/702687257 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f14180745c0 0x7f14180749a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:20.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:20.389+0000 7f141e200640 1 -- 172.21.15.5:0/702687257 >> 172.21.15.5:0/702687257 conn(0x7f14180fd8c0 msgr2=0x7f14180ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:45:20.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:20.389+0000 7f141e200640 1 -- 172.21.15.5:0/702687257 shutdown_connections 2024-09-10T09:45:20.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:20.389+0000 7f141e200640 1 -- 172.21.15.5:0/702687257 wait complete. 2024-09-10T09:45:20.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:20.390+0000 7f141e200640 1 Processor -- start 2024-09-10T09:45:20.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:20.390+0000 7f141e200640 1 -- start start 2024-09-10T09:45:20.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:20.391+0000 7f141e200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f14180745c0 0x7f141819a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:45:20.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:20.391+0000 7f141e200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1418074ee0 0x7f141819ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:45:20.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:20.391+0000 7f141e200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f141810e420 0x7f141819ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:45:20.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:20.391+0000 7f141e200640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f14181141d0 con 0x7f141810e420 2024-09-10T09:45:20.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:20.391+0000 7f141e200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f1418114050 con 0x7f14180745c0 2024-09-10T09:45:20.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:20.391+0000 7f141e200640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f1418114350 con 0x7f1418074ee0 2024-09-10T09:45:20.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:20.391+0000 7f1417400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f14180745c0 0x7f141819a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:45:20.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:20.391+0000 7f1417400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f14180745c0 0x7f141819a6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:34134/0 (socket says 172.21.15.5:34134) 2024-09-10T09:45:20.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:20.391+0000 7f1417400640 1 -- 172.21.15.5:0/168197921 learned_addr learned my addr 172.21.15.5:0/168197921 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:45:20.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:20.391+0000 7f1416a00640 1 --2- 172.21.15.5:0/168197921 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1418074ee0 0x7f141819ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:45:20.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:20.391+0000 7f1417e00640 1 --2- 172.21.15.5:0/168197921 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f141810e420 0x7f141819ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:45:20.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:20.392+0000 7f1417400640 1 -- 172.21.15.5:0/168197921 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1418074ee0 msgr2=0x7f141819ac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:45:20.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:20.392+0000 7f1417400640 1 --2- 172.21.15.5:0/168197921 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1418074ee0 0x7f141819ac00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:20.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:20.392+0000 7f1417400640 1 -- 172.21.15.5:0/168197921 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f141810e420 msgr2=0x7f141819ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:45:20.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:20.392+0000 7f1417400640 1 --2- 172.21.15.5:0/168197921 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f141810e420 0x7f141819ef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:20.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:20.392+0000 7f1417400640 1 -- 172.21.15.5:0/168197921 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f141819f670 con 0x7f14180745c0 2024-09-10T09:45:20.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:20.392+0000 7f1417400640 1 --2- 172.21.15.5:0/168197921 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f14180745c0 0x7f141819a6c0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f140800e9f0 tx=0x7f140800edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:45:20.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:20.392+0000 7f140fe00640 1 -- 172.21.15.5:0/168197921 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f140800cd80 con 0x7f14180745c0 2024-09-10T09:45:20.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:20.392+0000 7f141e200640 1 -- 172.21.15.5:0/168197921 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f141819f960 con 0x7f14180745c0 2024-09-10T09:45:20.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:20.392+0000 7f140fe00640 1 -- 172.21.15.5:0/168197921 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1408005500 con 0x7f14180745c0 2024-09-10T09:45:20.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:20.392+0000 7f140fe00640 1 -- 172.21.15.5:0/168197921 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1408010690 con 0x7f14180745c0 2024-09-10T09:45:20.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:20.393+0000 7f141e200640 1 -- 172.21.15.5:0/168197921 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14181a7240 con 0x7f14180745c0 2024-09-10T09:45:20.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:20.396+0000 7f1420e01740 1 Processor -- start 2024-09-10T09:45:20.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:20.396+0000 7f1420e01740 1 -- start start 2024-09-10T09:45:20.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:20.396+0000 7f1420e01740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5627dae07310 0x5627dae076f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:45:20.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:20.396+0000 7f1420e01740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5627dae07c30 0x5627dae11fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:45:20.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:20.396+0000 7f1420e01740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5627dae124f0 0x5627dae148e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:45:20.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:20.396+0000 7f1420e01740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5627dad6d650 con 0x5627dae07310 2024-09-10T09:45:20.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:20.396+0000 7f1420e01740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x5627dad14f10 con 0x5627dae124f0 2024-09-10T09:45:20.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:20.396+0000 7f1420e01740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5627dad69610 con 0x5627dae07c30 2024-09-10T09:45:20.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:20.397+0000 7f1417e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5627dae07310 0x5627dae076f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:45:20.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:20.397+0000 7f1417e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5627dae07310 0x5627dae076f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34712/0 (socket says 172.21.15.5:34712) 2024-09-10T09:45:20.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:20.397+0000 7f1417e00640 1 -- 172.21.15.5:0/1269176887 learned_addr learned my addr 172.21.15.5:0/1269176887 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:45:20.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:20.397+0000 7f1416a00640 1 --2- 172.21.15.5:0/1269176887 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5627dae124f0 0x5627dae148e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:45:20.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:20.397+0000 7f1417e00640 1 -- 172.21.15.5:0/1269176887 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5627dae07c30 msgr2=0x5627dae11fb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T09:45:20.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:20.397+0000 7f1417e00640 1 --2- 172.21.15.5:0/1269176887 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5627dae07c30 0x5627dae11fb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:20.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:20.397+0000 7f1417e00640 1 -- 172.21.15.5:0/1269176887 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5627dae124f0 msgr2=0x5627dae148e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:45:20.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:20.397+0000 7f1417e00640 1 --2- 172.21.15.5:0/1269176887 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5627dae124f0 0x5627dae148e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:20.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:20.397+0000 7f1417e00640 1 -- 172.21.15.5:0/1269176887 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5627daa7c580 con 0x5627dae07310 2024-09-10T09:45:20.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:20.397+0000 7f1416a00640 1 --2- 172.21.15.5:0/1269176887 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5627dae124f0 0x5627dae148e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:45:20.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:20.397+0000 7f140fe00640 1 -- 172.21.15.5:0/168197921 <== mon.2 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f14080046c0 con 0x7f14180745c0 2024-09-10T09:45:20.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:20.398+0000 7f1417e00640 1 --2- 172.21.15.5:0/1269176887 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5627dae07310 0x5627dae076f0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f1410009ef0 tx=0x7f141000f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c9c7395b4603ee5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:45:20.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:20.398+0000 7f1403e00640 1 -- 172.21.15.5:0/1269176887 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1410015070 con 0x5627dae07310 2024-09-10T09:45:20.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:20.398+0000 7f1403e00640 1 -- 172.21.15.5:0/1269176887 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1410011920 con 0x5627dae07310 2024-09-10T09:45:20.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:20.398+0000 7f1403e00640 1 -- 172.21.15.5:0/1269176887 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1410010940 con 0x5627dae07310 2024-09-10T09:45:20.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:20.399+0000 7f1420e01740 1 -- 172.21.15.5:0/1269176887 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5627dae07310 msgr2=0x5627dae076f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:45:20.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:20.399+0000 7f1420e01740 1 --2- 172.21.15.5:0/1269176887 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5627dae07310 0x5627dae076f0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f1410009ef0 tx=0x7f141000f730 comp rx=0 tx=0).stop 2024-09-10T09:45:20.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:20.399+0000 7f1420e01740 1 -- 172.21.15.5:0/1269176887 shutdown_connections 2024-09-10T09:45:20.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:20.399+0000 7f1420e01740 1 --2- 172.21.15.5:0/1269176887 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5627dae124f0 0x5627dae148e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:20.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:20.399+0000 7f1420e01740 1 --2- 172.21.15.5:0/1269176887 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5627dae07c30 0x5627dae11fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:20.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:20.399+0000 7f1420e01740 1 --2- 172.21.15.5:0/1269176887 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5627dae07310 0x5627dae076f0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:20.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:20.399+0000 7f1420e01740 1 -- 172.21.15.5:0/1269176887 >> 172.21.15.5:0/1269176887 conn(0x5627dae038a0 msgr2=0x5627dae03c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:45:20.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:20.399+0000 7f140fe00640 1 --2- 172.21.15.5:0/168197921 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f13fc0a2b80 0x7f13fc0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:45:20.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:20.399+0000 7f1420e01740 1 -- 172.21.15.5:0/1269176887 shutdown_connections 2024-09-10T09:45:20.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:20.399+0000 7f140fe00640 1 -- 172.21.15.5:0/168197921 <== mon.2 v2:172.21.15.102:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f14080cf920 con 0x7f14180745c0 2024-09-10T09:45:20.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:20.399+0000 7f1420e01740 1 -- 172.21.15.5:0/1269176887 wait complete. 2024-09-10T09:45:20.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:20.400+0000 7f1416a00640 1 --2- 172.21.15.5:0/168197921 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f13fc0a2b80 0x7f13fc0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:45:20.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:20.400+0000 7f1420e01740 1 -- start start 2024-09-10T09:45:20.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:20.400+0000 7f1420e01740 1 Processor -- start 2024-09-10T09:45:20.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:20.400+0000 7f1416a00640 1 --2- 172.21.15.5:0/168197921 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f13fc0a2b80 0x7f13fc0a5040 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f1404000c00 tx=0x7f1404005b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:45:20.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:20.401+0000 7f1420e01740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5627dae07310 0x5627dae0f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:45:20.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:20.401+0000 7f1420e01740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5627dae07c30 0x5627dae0f930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:45:20.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:20.401+0000 7f1420e01740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5627dae124f0 0x5627dae0fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:45:20.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:20.401+0000 7f1420e01740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5627dae150c0 con 0x5627dae07c30 2024-09-10T09:45:20.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:20.401+0000 7f1420e01740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x5627dad28350 con 0x5627dae07310 2024-09-10T09:45:20.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:20.401+0000 7f1420e01740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5627dae15240 con 0x5627dae124f0 2024-09-10T09:45:20.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:20.401+0000 7f1420e01740 10 client.0 _ll_register_callbacks cb 0x5627dad4b290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:45:20.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:20.401+0000 7f1416a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5627dae07c30 0x5627dae0f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:45:20.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:20.401+0000 7f1416a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5627dae07c30 0x5627dae0f930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34724/0 (socket says 172.21.15.5:34724) 2024-09-10T09:45:20.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:20.401+0000 7f1416a00640 1 -- 172.21.15.5:0/2341814492 learned_addr learned my addr 172.21.15.5:0/2341814492 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:45:20.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:20.401+0000 7f1417400640 1 --2- 172.21.15.5:0/2341814492 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5627dae07310 0x5627dae0f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:45:20.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:20.402+0000 7f1417e00640 1 --2- 172.21.15.5:0/2341814492 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5627dae124f0 0x5627dae0fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:45:20.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:20.402+0000 7f1416a00640 1 -- 172.21.15.5:0/2341814492 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5627dae124f0 msgr2=0x5627dae0fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:45:20.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:20.402+0000 7f1416a00640 1 --2- 172.21.15.5:0/2341814492 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5627dae124f0 0x5627dae0fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:20.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:20.402+0000 7f1416a00640 1 -- 172.21.15.5:0/2341814492 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5627dae07310 msgr2=0x5627dae0f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:45:20.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:20.402+0000 7f1416a00640 1 --2- 172.21.15.5:0/2341814492 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5627dae07310 0x5627dae0f3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:20.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:20.402+0000 7f1416a00640 1 -- 172.21.15.5:0/2341814492 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5627dab46960 con 0x5627dae07c30 2024-09-10T09:45:20.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:20.402+0000 7f1417e00640 1 --2- 172.21.15.5:0/2341814492 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5627dae124f0 0x5627dae0fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:45:20.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:20.402+0000 7f1416a00640 1 --2- 172.21.15.5:0/2341814492 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5627dae07c30 0x5627dae0f930 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f1404009900 tx=0x7f1404009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=244679680c7ff892 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:45:20.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:20.402+0000 7f1400c00640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-10T09:45:20.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:20.402+0000 7f1400c00640 1 -- 172.21.15.5:0/2341814492 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f140404e070 con 0x5627dae07c30 2024-09-10T09:45:20.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:20.403+0000 7f1420e01740 1 -- 172.21.15.5:0/2341814492 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1410011920 con 0x5627dae07c30 2024-09-10T09:45:20.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:20.403+0000 7f1400c00640 1 -- 172.21.15.5:0/2341814492 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f1404004b90 con 0x5627dae07c30 2024-09-10T09:45:20.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:20.403+0000 7f1400c00640 1 -- 172.21.15.5:0/2341814492 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f140405a050 con 0x5627dae07c30 2024-09-10T09:45:20.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:20.403+0000 7f1420e01740 10 client.16812 fetch_fsmap learned FSMap version 34 2024-09-10T09:45:20.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:20.403+0000 7f1420e01740 1 -- 172.21.15.5:0/2341814492 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5627dae20d40 con 0x5627dae07c30 2024-09-10T09:45:20.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:20.403+0000 7f1400c00640 1 -- 172.21.15.5:0/2341814492 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1410011920 con 0x5627dae07c30 2024-09-10T09:45:20.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:20.403+0000 7f1400c00640 1 -- 172.21.15.5:0/2341814492 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f1404003780 con 0x5627dae07c30 2024-09-10T09:45:20.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:20.404+0000 7f1420e01740 10 client.16812 fetch_fsmap finished waiting for FSMap version 34 2024-09-10T09:45:20.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:20.404+0000 7f1420e01740 10 client.16812 resolve_mds: validated gid 24542 aka daemon mds.b 2024-09-10T09:45:20.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:20.404+0000 7f1420e01740 20 client.16812 populate_metadata read hostname 'smithi005' 2024-09-10T09:45:20.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:20.404+0000 7f1420e01740 1 --2- 172.21.15.5:0/2341814492 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x5627dae221a0 0x5627dae245c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:45:20.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:20.404+0000 7f1420e01740 4 client.16812 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:45:20.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:20.404+0000 7f1420e01740 1 -- 172.21.15.5:0/2341814492 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5627dad7abc0 con 0x5627dae221a0 2024-09-10T09:45:20.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:20.404+0000 7f1417400640 1 --2- 172.21.15.5:0/2341814492 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x5627dae221a0 0x5627dae245c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:45:20.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:20.405+0000 7f1417400640 1 --2- 172.21.15.5:0/2341814492 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x5627dae221a0 0x5627dae245c0 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=27fb82f242b65c43 server_cookie=4d3f06941496f89d in_seq=0 out_seq=0 2024-09-10T09:45:20.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:20.405+0000 7f1400c00640 10 client.16812 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T09:45:20.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:20.405+0000 7f1400c00640 1 -- 172.21.15.5:0/2341814492 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5627dad7abc0 con 0x5627dae221a0 2024-09-10T09:45:20.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:20.405+0000 7f1400c00640 10 client.16812 handle_command_reply: tid=0 2024-09-10T09:45:20.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:20.405+0000 7f1420e01740 1 client.16812 shutdown 2024-09-10T09:45:20.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:20.406+0000 7f1420e01740 1 -- 172.21.15.5:0/2341814492 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5627dae07c30 msgr2=0x5627dae0f930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:45:20.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:20.406+0000 7f1420e01740 1 --2- 172.21.15.5:0/2341814492 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5627dae07c30 0x5627dae0f930 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f1404009900 tx=0x7f1404009260 comp rx=0 tx=0).stop 2024-09-10T09:45:20.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:20.406+0000 7f1420e01740 1 -- 172.21.15.5:0/2341814492 shutdown_connections 2024-09-10T09:45:20.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:20.406+0000 7f1420e01740 1 --2- 172.21.15.5:0/2341814492 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x5627dae221a0 0x5627dae245c0 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:20.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:20.406+0000 7f1420e01740 1 --2- 172.21.15.5:0/2341814492 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5627dae124f0 0x5627dae0fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:20.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:20.406+0000 7f1420e01740 1 --2- 172.21.15.5:0/2341814492 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5627dae07c30 0x5627dae0f930 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:20.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:20.406+0000 7f1420e01740 1 --2- 172.21.15.5:0/2341814492 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5627dae07310 0x5627dae0f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:20.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:20.406+0000 7f1420e01740 1 -- 172.21.15.5:0/2341814492 >> 172.21.15.5:0/2341814492 conn(0x5627dae038a0 msgr2=0x5627dae02aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:45:20.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:20.406+0000 7f1400c00640 0 client.16812 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T09:45:20.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:20.406+0000 7f1420e01740 1 -- 172.21.15.5:0/2341814492 shutdown_connections 2024-09-10T09:45:20.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:20.406+0000 7f1420e01740 1 -- 172.21.15.5:0/2341814492 wait complete. 2024-09-10T09:45:20.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:20.406+0000 7f1420e01740 20 client.16812 trim_cache size 0 max 16384 2024-09-10T09:45:20.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:20.426+0000 7f1420e01740 1 Processor -- start 2024-09-10T09:45:20.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:20.426+0000 7f1420e01740 1 -- start start 2024-09-10T09:45:20.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:20.427+0000 7f1420e01740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5627dae07310 0x5627dae0fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:45:20.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:20.427+0000 7f1420e01740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5627dae124f0 0x5627dae02aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:45:20.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:20.427+0000 7f1420e01740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5627dae29e90 0x5627dae2a310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:45:20.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:20.427+0000 7f1420e01740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f14181b0fb0 con 0x5627dae124f0 2024-09-10T09:45:20.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:20.427+0000 7f1420e01740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f14181b2af0 con 0x5627dae29e90 2024-09-10T09:45:20.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:20.427+0000 7f1420e01740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f14181b1680 con 0x5627dae07310 2024-09-10T09:45:20.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:20.427+0000 7f1417400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5627dae07310 0x5627dae0fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:45:20.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:20.427+0000 7f1416a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5627dae124f0 0x5627dae02aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:45:20.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:20.427+0000 7f1417400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5627dae07310 0x5627dae0fcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:49636/0 (socket says 172.21.15.5:49636) 2024-09-10T09:45:20.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:20.427+0000 7f1416a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5627dae124f0 0x5627dae02aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34730/0 (socket says 172.21.15.5:34730) 2024-09-10T09:45:20.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:20.427+0000 7f1417400640 1 -- 172.21.15.5:0/365466398 learned_addr learned my addr 172.21.15.5:0/365466398 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:45:20.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:20.428+0000 7f1417e00640 1 --2- 172.21.15.5:0/365466398 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5627dae29e90 0x5627dae2a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:45:20.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:20.428+0000 7f1416a00640 1 -- 172.21.15.5:0/365466398 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5627dae07310 msgr2=0x5627dae0fcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:45:20.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:20.428+0000 7f1416a00640 1 --2- 172.21.15.5:0/365466398 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5627dae07310 0x5627dae0fcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:20.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:20.428+0000 7f1416a00640 1 -- 172.21.15.5:0/365466398 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5627dae29e90 msgr2=0x5627dae2a310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:45:20.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:20.428+0000 7f1416a00640 1 --2- 172.21.15.5:0/365466398 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5627dae29e90 0x5627dae2a310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:20.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:20.428+0000 7f1416a00640 1 -- 172.21.15.5:0/365466398 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5627dae07a60 con 0x5627dae124f0 2024-09-10T09:45:20.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:20.428+0000 7f1417400640 1 --2- 172.21.15.5:0/365466398 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5627dae07310 0x5627dae0fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:45:20.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:20.428+0000 7f1416a00640 1 --2- 172.21.15.5:0/365466398 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5627dae124f0 0x5627dae02aa0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x5627dae104e0 tx=0x7f1404005750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36d99fe3b7ba9f78 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:45:20.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:20.428+0000 7f1403400640 1 -- 172.21.15.5:0/365466398 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1404062070 con 0x5627dae124f0 2024-09-10T09:45:20.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:20.428+0000 7f1403400640 1 -- 172.21.15.5:0/365466398 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1404004040 con 0x5627dae124f0 2024-09-10T09:45:20.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:20.429+0000 7f1403400640 1 -- 172.21.15.5:0/365466398 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f140404e070 con 0x5627dae124f0 2024-09-10T09:45:20.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:20.429+0000 7f1420e01740 1 -- 172.21.15.5:0/365466398 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5627dae124f0 msgr2=0x5627dae02aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:45:20.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:20.429+0000 7f1420e01740 1 --2- 172.21.15.5:0/365466398 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5627dae124f0 0x5627dae02aa0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x5627dae104e0 tx=0x7f1404005750 comp rx=0 tx=0).stop 2024-09-10T09:45:20.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:20.429+0000 7f1420e01740 1 -- 172.21.15.5:0/365466398 shutdown_connections 2024-09-10T09:45:20.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:20.429+0000 7f1420e01740 1 --2- 172.21.15.5:0/365466398 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5627dae29e90 0x5627dae2a310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:20.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:20.429+0000 7f1420e01740 1 --2- 172.21.15.5:0/365466398 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5627dae124f0 0x5627dae02aa0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:20.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:20.429+0000 7f1420e01740 1 --2- 172.21.15.5:0/365466398 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5627dae07310 0x5627dae0fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:20.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:20.429+0000 7f1420e01740 1 -- 172.21.15.5:0/365466398 >> 172.21.15.5:0/365466398 conn(0x5627dae038a0 msgr2=0x5627dae113e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:45:20.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:20.430+0000 7f1420e01740 1 -- 172.21.15.5:0/365466398 shutdown_connections 2024-09-10T09:45:20.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:20.430+0000 7f1420e01740 1 -- 172.21.15.5:0/365466398 wait complete. 2024-09-10T09:45:20.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:20.430+0000 7f1420e01740 1 -- start start 2024-09-10T09:45:20.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:20.430+0000 7f1420e01740 1 Processor -- start 2024-09-10T09:45:20.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:20.431+0000 7f1420e01740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5627dae07310 0x5627dae1a560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:45:20.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:20.431+0000 7f1420e01740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5627dae124f0 0x5627dae1aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:45:20.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:20.431+0000 7f1420e01740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5627dae29e90 0x5627dae169f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:45:20.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:20.431+0000 7f1420e01740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f14181b2420 con 0x5627dae124f0 2024-09-10T09:45:20.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:20.431+0000 7f1420e01740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f14181b1d50 con 0x5627dae29e90 2024-09-10T09:45:20.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:20.431+0000 7f1420e01740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f14181aef90 con 0x5627dae07310 2024-09-10T09:45:20.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:20.431+0000 7f1417400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5627dae07310 0x5627dae1a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:45:20.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:20.431+0000 7f1416a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5627dae124f0 0x5627dae1aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:45:20.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:20.431+0000 7f1417400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5627dae07310 0x5627dae1a560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:49648/0 (socket says 172.21.15.5:49648) 2024-09-10T09:45:20.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:20.431+0000 7f1417400640 1 -- 172.21.15.5:0/3509464254 learned_addr learned my addr 172.21.15.5:0/3509464254 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:45:20.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:20.431+0000 7f1417e00640 1 --2- 172.21.15.5:0/3509464254 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5627dae29e90 0x5627dae169f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:45:20.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:20.432+0000 7f1420e01740 10 client.0 _ll_register_callbacks cb 0x7f1404005e10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:45:20.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:20.432+0000 7f1416a00640 1 -- 172.21.15.5:0/3509464254 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5627dae07310 msgr2=0x5627dae1a560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:45:20.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:20.432+0000 7f1416a00640 1 --2- 172.21.15.5:0/3509464254 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5627dae07310 0x5627dae1a560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:20.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:20.432+0000 7f1416a00640 1 -- 172.21.15.5:0/3509464254 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5627dae29e90 msgr2=0x5627dae169f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:45:20.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:20.432+0000 7f1416a00640 1 --2- 172.21.15.5:0/3509464254 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5627dae29e90 0x5627dae169f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:20.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:20.432+0000 7f1416a00640 1 -- 172.21.15.5:0/3509464254 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5627dae17220 con 0x5627dae124f0 2024-09-10T09:45:20.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:20.432+0000 7f1416a00640 1 --2- 172.21.15.5:0/3509464254 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5627dae124f0 0x5627dae1aaa0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f14040058c0 tx=0x7f14040045d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=faed472d7406af2d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:45:20.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:20.433+0000 7f1400c00640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-10T09:45:20.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:20.433+0000 7f1400c00640 1 -- 172.21.15.5:0/3509464254 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1404062070 con 0x5627dae124f0 2024-09-10T09:45:20.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:20.433+0000 7f1400c00640 1 -- 172.21.15.5:0/3509464254 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f14040037f0 con 0x5627dae124f0 2024-09-10T09:45:20.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:20.433+0000 7f1420e01740 1 -- 172.21.15.5:0/3509464254 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1404004040 con 0x5627dae124f0 2024-09-10T09:45:20.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:20.433+0000 7f1400c00640 1 -- 172.21.15.5:0/3509464254 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f140404e070 con 0x5627dae124f0 2024-09-10T09:45:20.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:20.433+0000 7f1420e01740 10 client.16818 fetch_fsmap learned FSMap version 34 2024-09-10T09:45:20.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:20.433+0000 7f1420e01740 1 -- 172.21.15.5:0/3509464254 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5627dae8d910 con 0x5627dae124f0 2024-09-10T09:45:20.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:20.434+0000 7f1400c00640 1 -- 172.21.15.5:0/3509464254 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1404004040 con 0x5627dae124f0 2024-09-10T09:45:20.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:20.434+0000 7f1400c00640 1 -- 172.21.15.5:0/3509464254 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f1404071750 con 0x5627dae124f0 2024-09-10T09:45:20.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:20.434+0000 7f1420e01740 10 client.16818 fetch_fsmap finished waiting for FSMap version 34 2024-09-10T09:45:20.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:20.434+0000 7f1420e01740 10 client.16818 resolve_mds: validated gid 24542 aka daemon mds.b 2024-09-10T09:45:20.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:20.434+0000 7f1420e01740 20 client.16818 populate_metadata read hostname 'smithi005' 2024-09-10T09:45:20.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:20.434+0000 7f1420e01740 1 --2- 172.21.15.5:0/3509464254 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x5627dae8df10 0x5627dae90300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:45:20.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:20.434+0000 7f1420e01740 4 client.16818 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "36f005be", "release": true, "await": true}] 2024-09-10T09:45:20.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:20.434+0000 7f1420e01740 1 -- 172.21.15.5:0/3509464254 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "quiesce db", "set_id": "36f005be", "release": true, "await": true}) -- 0x5627dae04550 con 0x5627dae8df10 2024-09-10T09:45:20.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:20.434+0000 7f1417400640 1 --2- 172.21.15.5:0/3509464254 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x5627dae8df10 0x5627dae90300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:45:20.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:20.435+0000 7f1417400640 1 --2- 172.21.15.5:0/3509464254 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x5627dae8df10 0x5627dae90300 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=adafd4cd2ebb299b server_cookie=c2717a7bc67df223 in_seq=0 out_seq=0 2024-09-10T09:45:20.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:20.435+0000 7f1400c00640 10 client.16818 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T09:45:20.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:20.443+0000 7f1400c00640 1 -- 172.21.15.5:0/3509464254 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5627dae04550 con 0x5627dae8df10 2024-09-10T09:45:20.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:20.443+0000 7f1400c00640 10 client.16818 handle_command_reply: tid=0 2024-09-10T09:45:20.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:20.443+0000 7f1420e01740 1 client.16818 shutdown 2024-09-10T09:45:20.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:20.443+0000 7f1420e01740 1 -- 172.21.15.5:0/3509464254 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5627dae124f0 msgr2=0x5627dae1aaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:45:20.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:20.443+0000 7f1420e01740 1 --2- 172.21.15.5:0/3509464254 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5627dae124f0 0x5627dae1aaa0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f14040058c0 tx=0x7f14040045d0 comp rx=0 tx=0).stop 2024-09-10T09:45:20.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:20.443+0000 7f1420e01740 1 -- 172.21.15.5:0/3509464254 shutdown_connections 2024-09-10T09:45:20.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:20.443+0000 7f1420e01740 1 --2- 172.21.15.5:0/3509464254 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x5627dae8df10 0x5627dae90300 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:20.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:20.443+0000 7f1420e01740 1 --2- 172.21.15.5:0/3509464254 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5627dae29e90 0x5627dae169f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:20.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:20.443+0000 7f1420e01740 1 --2- 172.21.15.5:0/3509464254 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5627dae124f0 0x5627dae1aaa0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:20.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:20.443+0000 7f1420e01740 1 --2- 172.21.15.5:0/3509464254 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5627dae07310 0x5627dae1a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:20.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:20.443+0000 7f1420e01740 1 -- 172.21.15.5:0/3509464254 >> 172.21.15.5:0/3509464254 conn(0x5627dae038a0 msgr2=0x5627dae02280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:45:20.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:20.443+0000 7f1400c00640 0 client.16818 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T09:45:20.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:20.443+0000 7f1420e01740 1 -- 172.21.15.5:0/3509464254 shutdown_connections 2024-09-10T09:45:20.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:20.443+0000 7f1420e01740 1 -- 172.21.15.5:0/3509464254 wait complete. 2024-09-10T09:45:20.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:20.444+0000 7f1420e01740 20 client.16818 trim_cache size 0 max 16384 2024-09-10T09:45:20.446 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-10T09:45:20.446 INFO:teuthology.orchestra.run.smithi005.stdout: "epoch": 34, 2024-09-10T09:45:20.446 INFO:teuthology.orchestra.run.smithi005.stdout: "leader": 24542, 2024-09-10T09:45:20.446 INFO:teuthology.orchestra.run.smithi005.stdout: "set_version": 24, 2024-09-10T09:45:20.446 INFO:teuthology.orchestra.run.smithi005.stdout: "sets": { 2024-09-10T09:45:20.446 INFO:teuthology.orchestra.run.smithi005.stdout: "36f005be": { 2024-09-10T09:45:20.446 INFO:teuthology.orchestra.run.smithi005.stdout: "version": 24, 2024-09-10T09:45:20.446 INFO:teuthology.orchestra.run.smithi005.stdout: "age_ref": 0.0, 2024-09-10T09:45:20.446 INFO:teuthology.orchestra.run.smithi005.stdout: "state": { 2024-09-10T09:45:20.446 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "RELEASED", 2024-09-10T09:45:20.446 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0 2024-09-10T09:45:20.447 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T09:45:20.447 INFO:teuthology.orchestra.run.smithi005.stdout: "timeout": 90.0, 2024-09-10T09:45:20.447 INFO:teuthology.orchestra.run.smithi005.stdout: "expiration": 140.1, 2024-09-10T09:45:20.447 INFO:teuthology.orchestra.run.smithi005.stdout: "members": { 2024-09-10T09:45:20.447 INFO:teuthology.orchestra.run.smithi005.stdout: "file:/": { 2024-09-10T09:45:20.447 INFO:teuthology.orchestra.run.smithi005.stdout: "excluded": false, 2024-09-10T09:45:20.447 INFO:teuthology.orchestra.run.smithi005.stdout: "state": { 2024-09-10T09:45:20.447 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "RELEASED", 2024-09-10T09:45:20.447 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0 2024-09-10T09:45:20.447 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T09:45:20.447 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T09:45:20.447 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T09:45:20.447 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T09:45:20.447 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T09:45:20.447 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-10T09:45:20.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:20.445+0000 7f141e200640 1 -- 172.21.15.5:0/168197921 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f13fc0a2b80 msgr2=0x7f13fc0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:45:20.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:20.445+0000 7f141e200640 1 --2- 172.21.15.5:0/168197921 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f13fc0a2b80 0x7f13fc0a5040 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f1404000c00 tx=0x7f1404005b50 comp rx=0 tx=0).stop 2024-09-10T09:45:20.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:20.445+0000 7f141e200640 1 -- 172.21.15.5:0/168197921 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f14180745c0 msgr2=0x7f141819a6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:45:20.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:20.445+0000 7f141e200640 1 --2- 172.21.15.5:0/168197921 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f14180745c0 0x7f141819a6c0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f140800e9f0 tx=0x7f140800edf0 comp rx=0 tx=0).stop 2024-09-10T09:45:20.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:20.445+0000 7f141e200640 1 -- 172.21.15.5:0/168197921 shutdown_connections 2024-09-10T09:45:20.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:20.445+0000 7f141e200640 1 --2- 172.21.15.5:0/168197921 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f13fc0a2b80 0x7f13fc0a5040 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:20.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:20.445+0000 7f141e200640 1 --2- 172.21.15.5:0/168197921 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f141810e420 0x7f141819ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:20.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:20.445+0000 7f141e200640 1 --2- 172.21.15.5:0/168197921 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1418074ee0 0x7f141819ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:20.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:20.445+0000 7f141e200640 1 --2- 172.21.15.5:0/168197921 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f14180745c0 0x7f141819a6c0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:20.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:20.445+0000 7f141e200640 1 -- 172.21.15.5:0/168197921 >> 172.21.15.5:0/168197921 conn(0x7f14180fd8c0 msgr2=0x7f14180ff050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:45:20.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:20.445+0000 7f141e200640 1 -- 172.21.15.5:0/168197921 shutdown_connections 2024-09-10T09:45:20.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:20.445+0000 7f141e200640 1 -- 172.21.15.5:0/168197921 wait complete. 2024-09-10T09:45:20.451 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:45:21.056 INFO:tasks.workunit.client.0.smithi005.stdout: 1 361610 5.02 MB/sec execute 182 sec latency 20807.962 ms 2024-09-10T09:45:21.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:21.464+0000 7fe4b2000640 1 -- 172.21.15.5:0/1077167720 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe4ac102790 msgr2=0x7fe4ac1067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:45:21.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:21.464+0000 7fe4b2000640 1 --2- 172.21.15.5:0/1077167720 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe4ac102790 0x7fe4ac1067e0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fe498009900 tx=0x7fe49802a3f0 comp rx=0 tx=0).stop 2024-09-10T09:45:21.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:21.464+0000 7fe4b2000640 1 -- 172.21.15.5:0/1077167720 shutdown_connections 2024-09-10T09:45:21.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:21.464+0000 7fe4b2000640 1 --2- 172.21.15.5:0/1077167720 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe4ac106f10 0x7fe4ac10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:21.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:21.464+0000 7fe4b2000640 1 --2- 172.21.15.5:0/1077167720 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe4ac102790 0x7fe4ac1067e0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:21.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:21.464+0000 7fe4b2000640 1 --2- 172.21.15.5:0/1077167720 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe4ac101de0 0x7fe4ac1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:21.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:21.464+0000 7fe4b2000640 1 -- 172.21.15.5:0/1077167720 >> 172.21.15.5:0/1077167720 conn(0x7fe4ac0fd8c0 msgr2=0x7fe4ac0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:45:21.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:21.464+0000 7fe4b2000640 1 -- 172.21.15.5:0/1077167720 shutdown_connections 2024-09-10T09:45:21.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:21.464+0000 7fe4b2000640 1 -- 172.21.15.5:0/1077167720 wait complete. 2024-09-10T09:45:21.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:21.465+0000 7fe4b2000640 1 Processor -- start 2024-09-10T09:45:21.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:21.466+0000 7fe4b2000640 1 -- start start 2024-09-10T09:45:21.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:21.466+0000 7fe4b2000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe4ac101de0 0x7fe4ac0744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:45:21.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:21.466+0000 7fe4b2000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe4ac102790 0x7fe4ac074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:45:21.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:21.466+0000 7fe4b2000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe4ac106f10 0x7fe4ac074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:45:21.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:21.466+0000 7fe4b2000640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe4ac111fc0 con 0x7fe4ac101de0 2024-09-10T09:45:21.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:21.466+0000 7fe4b2000640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fe4ac111e40 con 0x7fe4ac102790 2024-09-10T09:45:21.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:21.466+0000 7fe4b2000640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fe4ac112140 con 0x7fe4ac106f10 2024-09-10T09:45:21.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:21.466+0000 7fe4ab400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe4ac101de0 0x7fe4ac0744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:45:21.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:21.466+0000 7fe4aaa00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe4ac102790 0x7fe4ac074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:45:21.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:21.466+0000 7fe4ab400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe4ac101de0 0x7fe4ac0744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47858/0 (socket says 172.21.15.5:47858) 2024-09-10T09:45:21.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:21.466+0000 7fe4ab400640 1 -- 172.21.15.5:0/3473509048 learned_addr learned my addr 172.21.15.5:0/3473509048 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:45:21.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:21.466+0000 7fe4abe00640 1 --2- 172.21.15.5:0/3473509048 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe4ac106f10 0x7fe4ac074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:45:21.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:21.467+0000 7fe4aaa00640 1 -- 172.21.15.5:0/3473509048 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe4ac106f10 msgr2=0x7fe4ac074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:45:21.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:21.467+0000 7fe4aaa00640 1 --2- 172.21.15.5:0/3473509048 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe4ac106f10 0x7fe4ac074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:21.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:21.467+0000 7fe4aaa00640 1 -- 172.21.15.5:0/3473509048 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe4ac101de0 msgr2=0x7fe4ac0744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:45:21.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:21.467+0000 7fe4aaa00640 1 --2- 172.21.15.5:0/3473509048 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe4ac101de0 0x7fe4ac0744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:21.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:21.467+0000 7fe4aaa00640 1 -- 172.21.15.5:0/3473509048 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4ac1a6960 con 0x7fe4ac102790 2024-09-10T09:45:21.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:21.467+0000 7fe4aaa00640 1 --2- 172.21.15.5:0/3473509048 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe4ac102790 0x7fe4ac074a30 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fe49802a910 tx=0x7fe498003b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:45:21.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:21.467+0000 7fe4a3e00640 1 -- 172.21.15.5:0/3473509048 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe49803f070 con 0x7fe4ac102790 2024-09-10T09:45:21.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:21.467+0000 7fe4b2000640 1 -- 172.21.15.5:0/3473509048 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4ac1a6bf0 con 0x7fe4ac102790 2024-09-10T09:45:21.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:21.467+0000 7fe4a3e00640 1 -- 172.21.15.5:0/3473509048 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe498031c90 con 0x7fe4ac102790 2024-09-10T09:45:21.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:21.467+0000 7fe4a3e00640 1 -- 172.21.15.5:0/3473509048 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe498035030 con 0x7fe4ac102790 2024-09-10T09:45:21.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:21.468+0000 7fe4b2000640 1 -- 172.21.15.5:0/3473509048 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4ac1a7120 con 0x7fe4ac102790 2024-09-10T09:45:21.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:21.470+0000 7fe4b4c46740 1 Processor -- start 2024-09-10T09:45:21.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:21.471+0000 7fe4b4c46740 1 -- start start 2024-09-10T09:45:21.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:21.471+0000 7fe4a3e00640 1 -- 172.21.15.5:0/3473509048 <== mon.2 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7fe498042070 con 0x7fe4ac102790 2024-09-10T09:45:21.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:21.471+0000 7fe4b4c46740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5609d4f87b90 0x5609d4f87f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:45:21.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:21.471+0000 7fe4b4c46740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5609d4f884b0 0x5609d4f92830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:45:21.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:21.471+0000 7fe4b4c46740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5609d4f92d70 0x5609d4f95160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:45:21.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:21.471+0000 7fe4b4c46740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5609d4eee3e0 con 0x5609d4f87b90 2024-09-10T09:45:21.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:21.471+0000 7fe4b4c46740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x5609d4e95d80 con 0x5609d4f92d70 2024-09-10T09:45:21.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:21.471+0000 7fe4b4c46740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5609d4eea3a0 con 0x5609d4f884b0 2024-09-10T09:45:21.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:21.471+0000 7fe4abe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5609d4f87b90 0x5609d4f87f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:45:21.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:21.471+0000 7fe4ab400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5609d4f884b0 0x5609d4f92830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:45:21.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:21.471+0000 7fe4abe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5609d4f87b90 0x5609d4f87f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47868/0 (socket says 172.21.15.5:47868) 2024-09-10T09:45:21.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:21.471+0000 7fe4abe00640 1 -- 172.21.15.5:0/4090287492 learned_addr learned my addr 172.21.15.5:0/4090287492 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:45:21.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:21.472+0000 7fe4aaa00640 1 --2- 172.21.15.5:0/4090287492 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5609d4f92d70 0x5609d4f95160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:45:21.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:21.472+0000 7fe4ab400640 1 -- 172.21.15.5:0/4090287492 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5609d4f92d70 msgr2=0x5609d4f95160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:45:21.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:21.472+0000 7fe4ab400640 1 --2- 172.21.15.5:0/4090287492 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5609d4f92d70 0x5609d4f95160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:21.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:21.472+0000 7fe4ab400640 1 -- 172.21.15.5:0/4090287492 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5609d4f87b90 msgr2=0x5609d4f87f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:45:21.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:21.472+0000 7fe4ab400640 1 --2- 172.21.15.5:0/4090287492 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5609d4f87b90 0x5609d4f87f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:21.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:21.472+0000 7fe4ab400640 1 -- 172.21.15.5:0/4090287492 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5609d4eb4ae0 con 0x5609d4f884b0 2024-09-10T09:45:21.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:21.472+0000 7fe4ab400640 1 --2- 172.21.15.5:0/4090287492 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5609d4f884b0 0x5609d4f92830 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fe49c004590 tx=0x7fe49c00bbb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=efcfeb6c37961957 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:45:21.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:21.473+0000 7fe497e00640 1 -- 172.21.15.5:0/4090287492 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe49c003bd0 con 0x5609d4f884b0 2024-09-10T09:45:21.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:21.473+0000 7fe497e00640 1 -- 172.21.15.5:0/4090287492 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe49c015070 con 0x5609d4f884b0 2024-09-10T09:45:21.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:21.473+0000 7fe497e00640 1 -- 172.21.15.5:0/4090287492 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe49c004e00 con 0x5609d4f884b0 2024-09-10T09:45:21.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:21.473+0000 7fe4a3e00640 1 --2- 172.21.15.5:0/3473509048 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fe4900a2c00 0x7fe4900a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:45:21.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:21.473+0000 7fe4a3e00640 1 -- 172.21.15.5:0/3473509048 <== mon.2 v2:172.21.15.102:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fe4980ef6e0 con 0x7fe4ac102790 2024-09-10T09:45:21.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:21.474+0000 7fe4abe00640 1 --2- 172.21.15.5:0/3473509048 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fe4900a2c00 0x7fe4900a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:45:21.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:21.474+0000 7fe4b4c46740 1 -- 172.21.15.5:0/4090287492 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5609d4f884b0 msgr2=0x5609d4f92830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:45:21.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:21.474+0000 7fe4b4c46740 1 --2- 172.21.15.5:0/4090287492 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5609d4f884b0 0x5609d4f92830 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fe49c004590 tx=0x7fe49c00bbb0 comp rx=0 tx=0).stop 2024-09-10T09:45:21.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:21.474+0000 7fe4b4c46740 1 -- 172.21.15.5:0/4090287492 shutdown_connections 2024-09-10T09:45:21.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:21.474+0000 7fe4b4c46740 1 --2- 172.21.15.5:0/4090287492 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5609d4f92d70 0x5609d4f95160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:21.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:21.474+0000 7fe4b4c46740 1 --2- 172.21.15.5:0/4090287492 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5609d4f884b0 0x5609d4f92830 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:21.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:21.474+0000 7fe4b4c46740 1 --2- 172.21.15.5:0/4090287492 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5609d4f87b90 0x5609d4f87f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:21.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:21.474+0000 7fe4b4c46740 1 -- 172.21.15.5:0/4090287492 >> 172.21.15.5:0/4090287492 conn(0x5609d4f84120 msgr2=0x5609d4f84500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:45:21.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:21.474+0000 7fe4b4c46740 1 -- 172.21.15.5:0/4090287492 shutdown_connections 2024-09-10T09:45:21.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:21.474+0000 7fe4b4c46740 1 -- 172.21.15.5:0/4090287492 wait complete. 2024-09-10T09:45:21.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:21.474+0000 7fe4abe00640 1 --2- 172.21.15.5:0/3473509048 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fe4900a2c00 0x7fe4900a50c0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fe4a4011630 tx=0x7fe4a4009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:45:21.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:21.475+0000 7fe4b4c46740 1 -- start start 2024-09-10T09:45:21.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:21.475+0000 7fe4b4c46740 1 Processor -- start 2024-09-10T09:45:21.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:21.475+0000 7fe4b4c46740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5609d4f87b90 0x5609d4f8fc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:45:21.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:21.475+0000 7fe4b4c46740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5609d4f884b0 0x5609d4f90170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:45:21.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:21.475+0000 7fe4b4c46740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5609d4f92d70 0x5609d4f906b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:45:21.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:21.475+0000 7fe4b4c46740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5609d4f959e0 con 0x5609d4f87b90 2024-09-10T09:45:21.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:21.475+0000 7fe4b4c46740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x5609d4ea90f0 con 0x5609d4f884b0 2024-09-10T09:45:21.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:21.475+0000 7fe4b4c46740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5609d4f95b60 con 0x5609d4f92d70 2024-09-10T09:45:21.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:21.476+0000 7fe4b4c46740 10 client.0 _ll_register_callbacks cb 0x5609d4e9a9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:45:21.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:21.476+0000 7fe4abe00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5609d4f92d70 0x5609d4f906b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:45:21.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:21.476+0000 7fe4abe00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5609d4f92d70 0x5609d4f906b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:49130/0 (socket says 172.21.15.5:49130) 2024-09-10T09:45:21.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:21.476+0000 7fe4abe00640 1 -- 172.21.15.5:0/2519619456 learned_addr learned my addr 172.21.15.5:0/2519619456 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:45:21.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:21.476+0000 7fe4ab400640 1 --2- 172.21.15.5:0/2519619456 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5609d4f87b90 0x5609d4f8fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:45:21.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:21.476+0000 7fe4aaa00640 1 --2- 172.21.15.5:0/2519619456 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5609d4f884b0 0x5609d4f90170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:45:21.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:21.476+0000 7fe4abe00640 1 -- 172.21.15.5:0/2519619456 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5609d4f884b0 msgr2=0x5609d4f90170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:45:21.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:21.476+0000 7fe4abe00640 1 --2- 172.21.15.5:0/2519619456 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5609d4f884b0 0x5609d4f90170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:21.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:21.476+0000 7fe4abe00640 1 -- 172.21.15.5:0/2519619456 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5609d4f87b90 msgr2=0x5609d4f8fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:45:21.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:21.476+0000 7fe4abe00640 1 --2- 172.21.15.5:0/2519619456 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5609d4f87b90 0x5609d4f8fc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:21.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:21.476+0000 7fe4abe00640 1 -- 172.21.15.5:0/2519619456 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5609d4c5f3c0 con 0x5609d4f92d70 2024-09-10T09:45:21.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:21.477+0000 7fe4abe00640 1 --2- 172.21.15.5:0/2519619456 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5609d4f92d70 0x5609d4f906b0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fe4a4003f40 tx=0x7fe4a4004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=28a710983dc940ef server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:45:21.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:21.477+0000 7fe494c00640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-09-10T09:45:21.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:21.477+0000 7fe494c00640 1 -- 172.21.15.5:0/2519619456 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4a400f910 con 0x5609d4f92d70 2024-09-10T09:45:21.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:21.477+0000 7fe4b4c46740 1 -- 172.21.15.5:0/2519619456 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe49c015070 con 0x5609d4f92d70 2024-09-10T09:45:21.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:21.477+0000 7fe494c00640 1 -- 172.21.15.5:0/2519619456 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fe4a400fab0 con 0x5609d4f92d70 2024-09-10T09:45:21.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:21.477+0000 7fe494c00640 1 -- 172.21.15.5:0/2519619456 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fe4a400fcb0 con 0x5609d4f92d70 2024-09-10T09:45:21.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:21.477+0000 7fe4b4c46740 10 client.26866 fetch_fsmap learned FSMap version 34 2024-09-10T09:45:21.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:21.477+0000 7fe4b4c46740 1 -- 172.21.15.5:0/2519619456 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5609d4fa1740 con 0x5609d4f92d70 2024-09-10T09:45:21.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:21.478+0000 7fe494c00640 1 -- 172.21.15.5:0/2519619456 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe49c015070 con 0x5609d4f92d70 2024-09-10T09:45:21.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:21.478+0000 7fe494c00640 1 -- 172.21.15.5:0/2519619456 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7fe4a4026070 con 0x5609d4f92d70 2024-09-10T09:45:21.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:21.478+0000 7fe4b4c46740 10 client.26866 fetch_fsmap finished waiting for FSMap version 34 2024-09-10T09:45:21.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:21.478+0000 7fe4b4c46740 10 client.26866 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T09:45:21.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:21.478+0000 7fe4b4c46740 20 client.26866 populate_metadata read hostname 'smithi005' 2024-09-10T09:45:21.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:21.478+0000 7fe4b4c46740 1 --2- 172.21.15.5:0/2519619456 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x5609d4fa2b50 0x5609d4fa4ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:45:21.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:21.478+0000 7fe4b4c46740 4 client.26866 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:45:21.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:21.478+0000 7fe4b4c46740 1 -- 172.21.15.5:0/2519619456 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5609d4efb950 con 0x5609d4fa2b50 2024-09-10T09:45:21.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:21.478+0000 7fe4ab400640 1 --2- 172.21.15.5:0/2519619456 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x5609d4fa2b50 0x5609d4fa4ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:45:21.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:21.479+0000 7fe4ab400640 1 --2- 172.21.15.5:0/2519619456 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x5609d4fa2b50 0x5609d4fa4ff0 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8c19eaccc2a6dfdd server_cookie=2546cd9c01aeeca1 in_seq=0 out_seq=0 2024-09-10T09:45:21.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:21.479+0000 7fe494c00640 10 client.26866 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T09:45:21.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:21.479+0000 7fe494c00640 1 -- 172.21.15.5:0/2519619456 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5609d4efb950 con 0x5609d4fa2b50 2024-09-10T09:45:21.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:21.479+0000 7fe494c00640 10 client.26866 handle_command_reply: tid=0 2024-09-10T09:45:21.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:21.479+0000 7fe4b4c46740 1 client.26866 shutdown 2024-09-10T09:45:21.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:21.480+0000 7fe4b4c46740 1 -- 172.21.15.5:0/2519619456 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5609d4f92d70 msgr2=0x5609d4f906b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:45:21.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:21.480+0000 7fe4b4c46740 1 --2- 172.21.15.5:0/2519619456 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5609d4f92d70 0x5609d4f906b0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fe4a4003f40 tx=0x7fe4a4004a40 comp rx=0 tx=0).stop 2024-09-10T09:45:21.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:21.480+0000 7fe4b4c46740 1 -- 172.21.15.5:0/2519619456 shutdown_connections 2024-09-10T09:45:21.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:21.480+0000 7fe4b4c46740 1 --2- 172.21.15.5:0/2519619456 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x5609d4fa2b50 0x5609d4fa4ff0 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:21.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:21.480+0000 7fe4b4c46740 1 --2- 172.21.15.5:0/2519619456 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5609d4f92d70 0x5609d4f906b0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:21.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:21.480+0000 7fe4b4c46740 1 --2- 172.21.15.5:0/2519619456 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5609d4f884b0 0x5609d4f90170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:21.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:21.480+0000 7fe4b4c46740 1 --2- 172.21.15.5:0/2519619456 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5609d4f87b90 0x5609d4f8fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:21.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:21.480+0000 7fe494c00640 0 client.26866 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T09:45:21.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:21.480+0000 7fe4b4c46740 1 -- 172.21.15.5:0/2519619456 >> 172.21.15.5:0/2519619456 conn(0x5609d4f84120 msgr2=0x5609d4f89e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:45:21.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:21.480+0000 7fe4b4c46740 1 -- 172.21.15.5:0/2519619456 shutdown_connections 2024-09-10T09:45:21.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:21.480+0000 7fe4b4c46740 1 -- 172.21.15.5:0/2519619456 wait complete. 2024-09-10T09:45:21.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:21.480+0000 7fe4b4c46740 20 client.26866 trim_cache size 0 max 16384 2024-09-10T09:45:21.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:21.498+0000 7fe4b4c46740 1 Processor -- start 2024-09-10T09:45:21.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:21.498+0000 7fe4b4c46740 1 -- start start 2024-09-10T09:45:21.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:21.499+0000 7fe4b4c46740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5609d4f87b90 0x5609d4f84500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:45:21.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:21.499+0000 7fe4b4c46740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5609d4f884b0 0x5609d4f89d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:45:21.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:21.499+0000 7fe4b4c46740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5609d4f92d70 0x5609d4faa910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:45:21.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:21.499+0000 7fe4b4c46740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe4ac1b1020 con 0x5609d4f87b90 2024-09-10T09:45:21.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:21.499+0000 7fe4b4c46740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fe4ac1b2b60 con 0x5609d4f884b0 2024-09-10T09:45:21.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:21.499+0000 7fe4b4c46740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fe4ac1b16f0 con 0x5609d4f92d70 2024-09-10T09:45:21.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:21.499+0000 7fe4ab400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5609d4f87b90 0x5609d4f84500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:45:21.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:21.499+0000 7fe4abe00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5609d4f92d70 0x5609d4faa910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:45:21.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:21.499+0000 7fe4ab400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5609d4f87b90 0x5609d4f84500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47878/0 (socket says 172.21.15.5:47878) 2024-09-10T09:45:21.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:21.499+0000 7fe4ab400640 1 -- 172.21.15.5:0/1273325474 learned_addr learned my addr 172.21.15.5:0/1273325474 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:45:21.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:21.499+0000 7fe4aaa00640 1 --2- 172.21.15.5:0/1273325474 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5609d4f884b0 0x5609d4f89d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:45:21.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:21.499+0000 7fe4ab400640 1 -- 172.21.15.5:0/1273325474 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5609d4f92d70 msgr2=0x5609d4faa910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:45:21.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:21.500+0000 7fe4ab400640 1 --2- 172.21.15.5:0/1273325474 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5609d4f92d70 0x5609d4faa910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:21.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:21.500+0000 7fe4ab400640 1 -- 172.21.15.5:0/1273325474 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5609d4f884b0 msgr2=0x5609d4f89d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:45:21.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:21.500+0000 7fe4ab400640 1 --2- 172.21.15.5:0/1273325474 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5609d4f884b0 0x5609d4f89d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:21.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:21.500+0000 7fe4ab400640 1 -- 172.21.15.5:0/1273325474 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5609d4f839b0 con 0x5609d4f87b90 2024-09-10T09:45:21.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:21.500+0000 7fe4ab400640 1 --2- 172.21.15.5:0/1273325474 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5609d4f87b90 0x5609d4f84500 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fe49c004350 tx=0x7fe49c004500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5c1be848e829d9c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:45:21.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:21.500+0000 7fe497400640 1 -- 172.21.15.5:0/1273325474 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe49c003dd0 con 0x5609d4f87b90 2024-09-10T09:45:21.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:21.500+0000 7fe497400640 1 -- 172.21.15.5:0/1273325474 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe49c015070 con 0x5609d4f87b90 2024-09-10T09:45:21.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:21.500+0000 7fe497400640 1 -- 172.21.15.5:0/1273325474 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe49c019620 con 0x5609d4f87b90 2024-09-10T09:45:21.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:21.501+0000 7fe4b4c46740 1 -- 172.21.15.5:0/1273325474 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5609d4f87b90 msgr2=0x5609d4f84500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:45:21.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:21.501+0000 7fe4b4c46740 1 --2- 172.21.15.5:0/1273325474 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5609d4f87b90 0x5609d4f84500 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fe49c004350 tx=0x7fe49c004500 comp rx=0 tx=0).stop 2024-09-10T09:45:21.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:21.501+0000 7fe4b4c46740 1 -- 172.21.15.5:0/1273325474 shutdown_connections 2024-09-10T09:45:21.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:21.501+0000 7fe4b4c46740 1 --2- 172.21.15.5:0/1273325474 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5609d4f92d70 0x5609d4faa910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:21.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:21.501+0000 7fe4b4c46740 1 --2- 172.21.15.5:0/1273325474 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5609d4f884b0 0x5609d4f89d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:21.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:21.501+0000 7fe4b4c46740 1 --2- 172.21.15.5:0/1273325474 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5609d4f87b90 0x5609d4f84500 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:21.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:21.501+0000 7fe4b4c46740 1 -- 172.21.15.5:0/1273325474 >> 172.21.15.5:0/1273325474 conn(0x5609d4f84120 msgr2=0x5609d4f89630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:45:21.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:21.501+0000 7fe4b4c46740 1 -- 172.21.15.5:0/1273325474 shutdown_connections 2024-09-10T09:45:21.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:21.501+0000 7fe4b4c46740 1 -- 172.21.15.5:0/1273325474 wait complete. 2024-09-10T09:45:21.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:21.502+0000 7fe4b4c46740 1 -- start start 2024-09-10T09:45:21.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:21.502+0000 7fe4b4c46740 1 Processor -- start 2024-09-10T09:45:21.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:21.502+0000 7fe4b4c46740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5609d4f87b90 0x5609d4f9ade0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:45:21.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:21.502+0000 7fe4b4c46740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5609d4f884b0 0x5609d4f9b320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:45:21.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:21.502+0000 7fe4b4c46740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5609d4f92d70 0x5609d4f97db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:45:21.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:21.502+0000 7fe4b4c46740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe4ac1b2490 con 0x5609d4f92d70 2024-09-10T09:45:21.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:21.502+0000 7fe4b4c46740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fe4ac1b1dc0 con 0x5609d4f884b0 2024-09-10T09:45:21.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:21.502+0000 7fe4b4c46740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fe4ac1af000 con 0x5609d4f87b90 2024-09-10T09:45:21.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:21.502+0000 7fe4b4c46740 10 client.0 _ll_register_callbacks cb 0x7fe4a4003e00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:45:21.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:21.503+0000 7fe4ab400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5609d4f87b90 0x5609d4f9ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:45:21.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:21.503+0000 7fe4ab400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5609d4f87b90 0x5609d4f9ade0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:49150/0 (socket says 172.21.15.5:49150) 2024-09-10T09:45:21.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:21.503+0000 7fe4ab400640 1 -- 172.21.15.5:0/2760178091 learned_addr learned my addr 172.21.15.5:0/2760178091 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:45:21.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:21.503+0000 7fe4aaa00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5609d4f884b0 0x5609d4f9b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:45:21.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:21.503+0000 7fe4abe00640 1 --2- 172.21.15.5:0/2760178091 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5609d4f92d70 0x5609d4f97db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:45:21.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:21.503+0000 7fe4aaa00640 1 -- 172.21.15.5:0/2760178091 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5609d4f87b90 msgr2=0x5609d4f9ade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:45:21.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:21.503+0000 7fe4aaa00640 1 --2- 172.21.15.5:0/2760178091 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5609d4f87b90 0x5609d4f9ade0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:21.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:21.503+0000 7fe4aaa00640 1 -- 172.21.15.5:0/2760178091 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5609d4f92d70 msgr2=0x5609d4f97db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:45:21.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:21.503+0000 7fe4aaa00640 1 --2- 172.21.15.5:0/2760178091 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5609d4f92d70 0x5609d4f97db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:21.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:21.503+0000 7fe4aaa00640 1 -- 172.21.15.5:0/2760178091 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5609d4f982f0 con 0x5609d4f884b0 2024-09-10T09:45:21.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:21.503+0000 7fe4aaa00640 1 --2- 172.21.15.5:0/2760178091 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5609d4f884b0 0x5609d4f9b320 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x5609d4f854a0 tx=0x7fe498003f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fce2d6e3ad8c9f71 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:45:21.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:21.503+0000 7fe494c00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-09-10T09:45:21.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:21.504+0000 7fe494c00640 1 -- 172.21.15.5:0/2760178091 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe498049040 con 0x5609d4f884b0 2024-09-10T09:45:21.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:21.504+0000 7fe494c00640 1 -- 172.21.15.5:0/2760178091 <== mon.2 v2:172.21.15.102:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fe4980ef370 con 0x5609d4f884b0 2024-09-10T09:45:21.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:21.504+0000 7fe4b4c46740 1 -- 172.21.15.5:0/2760178091 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe49c015070 con 0x5609d4f884b0 2024-09-10T09:45:21.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:21.504+0000 7fe494c00640 1 -- 172.21.15.5:0/2760178091 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fe4980bfea0 con 0x5609d4f884b0 2024-09-10T09:45:21.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:21.504+0000 7fe4b4c46740 10 client.26852 fetch_fsmap learned FSMap version 34 2024-09-10T09:45:21.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:21.504+0000 7fe4b4c46740 1 -- 172.21.15.5:0/2760178091 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5609d500e120 con 0x5609d4f884b0 2024-09-10T09:45:21.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:21.504+0000 7fe494c00640 1 -- 172.21.15.5:0/2760178091 <== mon.2 v2:172.21.15.102:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe4980d4020 con 0x5609d4f884b0 2024-09-10T09:45:21.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:21.504+0000 7fe494c00640 1 -- 172.21.15.5:0/2760178091 <== mon.2 v2:172.21.15.102:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7fe4980ef050 con 0x5609d4f884b0 2024-09-10T09:45:21.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:21.504+0000 7fe4b4c46740 10 client.26852 fetch_fsmap finished waiting for FSMap version 34 2024-09-10T09:45:21.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:21.504+0000 7fe4b4c46740 10 client.26852 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T09:45:21.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:21.504+0000 7fe4b4c46740 20 client.26852 populate_metadata read hostname 'smithi005' 2024-09-10T09:45:21.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:21.504+0000 7fe4b4c46740 1 --2- 172.21.15.5:0/2760178091 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x5609d500e890 0x5609d5010c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:45:21.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:21.504+0000 7fe4b4c46740 4 client.26852 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T09:45:21.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:21.504+0000 7fe4b4c46740 1 -- 172.21.15.5:0/2760178091 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5609d4ef5e60 con 0x5609d500e890 2024-09-10T09:45:21.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:21.505+0000 7fe4ab400640 1 --2- 172.21.15.5:0/2760178091 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x5609d500e890 0x5609d5010c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:45:21.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:21.505+0000 7fe4ab400640 1 --2- 172.21.15.5:0/2760178091 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x5609d500e890 0x5609d5010c80 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6f59317cd41cff98 server_cookie=628f5dd85a7855c7 in_seq=0 out_seq=0 2024-09-10T09:45:21.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:21.505+0000 7fe494c00640 10 client.26852 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T09:45:21.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:21.505+0000 7fe494c00640 1 -- 172.21.15.5:0/2760178091 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5609d4ef5e60 con 0x5609d500e890 2024-09-10T09:45:21.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:21.505+0000 7fe494c00640 10 client.26852 handle_command_reply: tid=0 2024-09-10T09:45:21.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:21.505+0000 7fe4b4c46740 1 client.26852 shutdown 2024-09-10T09:45:21.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:21.506+0000 7fe4b4c46740 1 -- 172.21.15.5:0/2760178091 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5609d4f884b0 msgr2=0x5609d4f9b320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:45:21.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:21.506+0000 7fe4b4c46740 1 --2- 172.21.15.5:0/2760178091 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5609d4f884b0 0x5609d4f9b320 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x5609d4f854a0 tx=0x7fe498003f90 comp rx=0 tx=0).stop 2024-09-10T09:45:21.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:21.506+0000 7fe4b4c46740 1 -- 172.21.15.5:0/2760178091 shutdown_connections 2024-09-10T09:45:21.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:21.506+0000 7fe4b4c46740 1 --2- 172.21.15.5:0/2760178091 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x5609d500e890 0x5609d5010c80 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:21.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:21.506+0000 7fe4b4c46740 1 --2- 172.21.15.5:0/2760178091 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5609d4f92d70 0x5609d4f97db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:21.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:21.506+0000 7fe4b4c46740 1 --2- 172.21.15.5:0/2760178091 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5609d4f884b0 0x5609d4f9b320 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:21.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:21.506+0000 7fe494c00640 0 client.26852 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T09:45:21.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:21.506+0000 7fe4b4c46740 1 --2- 172.21.15.5:0/2760178091 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5609d4f87b90 0x5609d4f9ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:21.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:21.506+0000 7fe4b4c46740 1 -- 172.21.15.5:0/2760178091 >> 172.21.15.5:0/2760178091 conn(0x5609d4f84120 msgr2=0x5609d4f89e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:45:21.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:21.506+0000 7fe4b4c46740 1 -- 172.21.15.5:0/2760178091 shutdown_connections 2024-09-10T09:45:21.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:21.506+0000 7fe4b4c46740 1 -- 172.21.15.5:0/2760178091 wait complete. 2024-09-10T09:45:21.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:21.506+0000 7fe4b4c46740 20 client.26852 trim_cache size 0 max 16384 2024-09-10T09:45:21.511 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-10T09:45:21.511 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-10T09:45:21.512 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2024-09-10T09:45:21.512 INFO:teuthology.orchestra.run.smithi005.stdout: "0758dbc4-f003-42a3-b9ec-fa47855caff3": { 2024-09-10T09:45:21.512 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2024-09-10T09:45:21.512 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "0758dbc4-f003-42a3-b9ec-fa47855caff3", 2024-09-10T09:45:21.512 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2024-09-10T09:45:21.512 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T09:45:21.512 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T09:45:21.512 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-10T09:45:21.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:21.507+0000 7fe4b2000640 1 -- 172.21.15.5:0/3473509048 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fe4900a2c00 msgr2=0x7fe4900a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:45:21.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:21.507+0000 7fe4b2000640 1 --2- 172.21.15.5:0/3473509048 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fe4900a2c00 0x7fe4900a50c0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fe4a4011630 tx=0x7fe4a4009500 comp rx=0 tx=0).stop 2024-09-10T09:45:21.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:21.507+0000 7fe4b2000640 1 -- 172.21.15.5:0/3473509048 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe4ac102790 msgr2=0x7fe4ac074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:45:21.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:21.507+0000 7fe4b2000640 1 --2- 172.21.15.5:0/3473509048 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe4ac102790 0x7fe4ac074a30 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fe49802a910 tx=0x7fe498003b90 comp rx=0 tx=0).stop 2024-09-10T09:45:21.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:21.507+0000 7fe4b2000640 1 -- 172.21.15.5:0/3473509048 shutdown_connections 2024-09-10T09:45:21.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:21.507+0000 7fe4b2000640 1 --2- 172.21.15.5:0/3473509048 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fe4900a2c00 0x7fe4900a50c0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:21.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:21.507+0000 7fe4b2000640 1 --2- 172.21.15.5:0/3473509048 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe4ac106f10 0x7fe4ac074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:21.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:21.507+0000 7fe4b2000640 1 --2- 172.21.15.5:0/3473509048 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe4ac102790 0x7fe4ac074a30 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:21.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:21.507+0000 7fe4b2000640 1 --2- 172.21.15.5:0/3473509048 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe4ac101de0 0x7fe4ac0744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:21.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:21.507+0000 7fe4b2000640 1 -- 172.21.15.5:0/3473509048 >> 172.21.15.5:0/3473509048 conn(0x7fe4ac0fd8c0 msgr2=0x7fe4ac0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:45:21.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:21.507+0000 7fe4b2000640 1 -- 172.21.15.5:0/3473509048 shutdown_connections 2024-09-10T09:45:21.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:21.507+0000 7fe4b2000640 1 -- 172.21.15.5:0/3473509048 wait complete. 2024-09-10T09:45:21.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:45:21 smithi102 ceph-mon[35442]: pgmap v603: 129 pgs: 129 active+clean; 142 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 334 KiB/s wr, 30 op/s 2024-09-10T09:45:22.053 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:45:21 smithi159 ceph-mon[35513]: pgmap v603: 129 pgs: 129 active+clean; 142 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 334 KiB/s wr, 30 op/s 2024-09-10T09:45:22.056 INFO:tasks.workunit.client.0.smithi005.stdout: 1 363350 5.04 MB/sec execute 183 sec latency 23.725 ms 2024-09-10T09:45:22.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:45:21 smithi005 ceph-mon[30350]: pgmap v603: 129 pgs: 129 active+clean; 142 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 334 KiB/s wr, 30 op/s 2024-09-10T09:45:22.535 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '36f005be', total seconds elapsed: 26.2 2024-09-10T09:45:22.846 INFO:tasks.cephfs.filesystem:scrub status for tag:0758dbc4-f003-42a3-b9ec-fa47855caff3 - {'path': '/', 'tag': '0758dbc4-f003-42a3-b9ec-fa47855caff3', 'options': 'recursive,force'} 2024-09-10T09:45:23.056 INFO:tasks.workunit.client.0.smithi005.stdout: 1 365816 5.09 MB/sec execute 184 sec latency 10.734 ms 2024-09-10T09:45:23.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:45:23 smithi102 ceph-mon[35442]: pgmap v604: 129 pgs: 129 active+clean; 142 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 63 KiB/s wr, 21 op/s 2024-09-10T09:45:24.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:45:23 smithi005 ceph-mon[30350]: pgmap v604: 129 pgs: 129 active+clean; 142 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 63 KiB/s wr, 21 op/s 2024-09-10T09:45:24.056 INFO:tasks.workunit.client.0.smithi005.stdout: 1 368240 5.13 MB/sec execute 185 sec latency 22.021 ms 2024-09-10T09:45:24.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:45:23 smithi159 ceph-mon[35513]: pgmap v604: 129 pgs: 129 active+clean; 142 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 63 KiB/s wr, 21 op/s 2024-09-10T09:45:25.056 INFO:tasks.workunit.client.0.smithi005.stdout: 1 370134 5.15 MB/sec execute 186 sec latency 25.316 ms 2024-09-10T09:45:25.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:45:25 smithi102 ceph-mon[35442]: pgmap v605: 129 pgs: 129 active+clean; 143 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 266 KiB/s rd, 84 KiB/s wr, 17 op/s 2024-09-10T09:45:26.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:45:25 smithi005 ceph-mon[30350]: pgmap v605: 129 pgs: 129 active+clean; 143 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 266 KiB/s rd, 84 KiB/s wr, 17 op/s 2024-09-10T09:45:26.056 INFO:tasks.workunit.client.0.smithi005.stdout: 1 372736 5.21 MB/sec execute 187 sec latency 14.975 ms 2024-09-10T09:45:26.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:45:25 smithi159 ceph-mon[35513]: pgmap v605: 129 pgs: 129 active+clean; 143 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 266 KiB/s rd, 84 KiB/s wr, 17 op/s 2024-09-10T09:45:27.056 INFO:tasks.workunit.client.0.smithi005.stdout: 1 374572 5.22 MB/sec execute 188 sec latency 120.484 ms 2024-09-10T09:45:27.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:45:27 smithi102 ceph-mon[35442]: pgmap v606: 129 pgs: 129 active+clean; 146 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 546 KiB/s rd, 398 KiB/s wr, 69 op/s 2024-09-10T09:45:28.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:45:27 smithi005 ceph-mon[30350]: pgmap v606: 129 pgs: 129 active+clean; 146 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 546 KiB/s rd, 398 KiB/s wr, 69 op/s 2024-09-10T09:45:28.056 INFO:tasks.workunit.client.0.smithi005.stdout: 1 376685 5.26 MB/sec execute 189 sec latency 23.757 ms 2024-09-10T09:45:28.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:45:27 smithi159 ceph-mon[35513]: pgmap v606: 129 pgs: 129 active+clean; 146 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 546 KiB/s rd, 398 KiB/s wr, 69 op/s 2024-09-10T09:45:29.057 INFO:tasks.workunit.client.0.smithi005.stdout: 1 379220 5.30 MB/sec execute 190 sec latency 14.436 ms 2024-09-10T09:45:29.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:45:29 smithi102 ceph-mon[35442]: pgmap v607: 129 pgs: 129 active+clean; 147 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 847 KiB/s rd, 499 KiB/s wr, 80 op/s 2024-09-10T09:45:30.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:45:29 smithi005 ceph-mon[30350]: pgmap v607: 129 pgs: 129 active+clean; 147 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 847 KiB/s rd, 499 KiB/s wr, 80 op/s 2024-09-10T09:45:30.056 INFO:tasks.workunit.client.0.smithi005.stdout: 1 381034 5.33 MB/sec execute 191 sec latency 24.594 ms 2024-09-10T09:45:30.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:45:29 smithi159 ceph-mon[35513]: pgmap v607: 129 pgs: 129 active+clean; 147 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 847 KiB/s rd, 499 KiB/s wr, 80 op/s 2024-09-10T09:45:31.057 INFO:tasks.workunit.client.0.smithi005.stdout: 1 383503 5.38 MB/sec execute 192 sec latency 9.377 ms 2024-09-10T09:45:31.670 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:45:31 smithi005 ceph-mon[30350]: pgmap v608: 129 pgs: 129 active+clean; 150 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 968 KiB/s wr, 143 op/s 2024-09-10T09:45:31.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:45:31 smithi102 ceph-mon[35442]: pgmap v608: 129 pgs: 129 active+clean; 150 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 968 KiB/s wr, 143 op/s 2024-09-10T09:45:32.060 INFO:tasks.workunit.client.0.smithi005.stdout: 1 383976 5.35 MB/sec execute 193 sec latency 462.362 ms 2024-09-10T09:45:32.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:45:31 smithi159 ceph-mon[35513]: pgmap v608: 129 pgs: 129 active+clean; 150 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 968 KiB/s wr, 143 op/s 2024-09-10T09:45:32.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:45:32 smithi102 ceph-mon[35442]: scrub summary: idle 2024-09-10T09:45:32.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:45:32 smithi102 ceph-mon[35442]: scrub summary: idle 2024-09-10T09:45:33.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:45:32 smithi005 ceph-mon[30350]: scrub summary: idle 2024-09-10T09:45:33.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:45:32 smithi005 ceph-mon[30350]: scrub summary: idle 2024-09-10T09:45:33.060 INFO:tasks.workunit.client.0.smithi005.stdout: 1 385735 5.36 MB/sec execute 194 sec latency 725.363 ms 2024-09-10T09:45:33.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:45:32 smithi159 ceph-mon[35513]: scrub summary: idle 2024-09-10T09:45:33.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:45:32 smithi159 ceph-mon[35513]: scrub summary: idle 2024-09-10T09:45:33.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:45:33 smithi102 ceph-mon[35442]: pgmap v609: 129 pgs: 129 active+clean; 153 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.2 MiB/s wr, 161 op/s 2024-09-10T09:45:34.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:45:33 smithi005 ceph-mon[30350]: pgmap v609: 129 pgs: 129 active+clean; 153 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.2 MiB/s wr, 161 op/s 2024-09-10T09:45:34.060 INFO:tasks.workunit.client.0.smithi005.stdout: 1 387745 5.40 MB/sec execute 195 sec latency 24.405 ms 2024-09-10T09:45:34.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:45:33 smithi159 ceph-mon[35513]: pgmap v609: 129 pgs: 129 active+clean; 153 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.2 MiB/s wr, 161 op/s 2024-09-10T09:45:35.060 INFO:tasks.workunit.client.0.smithi005.stdout: 1 390379 5.46 MB/sec execute 196 sec latency 19.051 ms 2024-09-10T09:45:36.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:45:35 smithi005 ceph-mon[30350]: pgmap v610: 129 pgs: 129 active+clean; 153 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.3 MiB/s wr, 167 op/s 2024-09-10T09:45:36.060 INFO:tasks.workunit.client.0.smithi005.stdout: 1 391995 5.46 MB/sec execute 197 sec latency 46.422 ms 2024-09-10T09:45:36.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:45:35 smithi159 ceph-mon[35513]: pgmap v610: 129 pgs: 129 active+clean; 153 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.3 MiB/s wr, 167 op/s 2024-09-10T09:45:36.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:45:35 smithi102 ceph-mon[35442]: pgmap v610: 129 pgs: 129 active+clean; 153 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.3 MiB/s wr, 167 op/s 2024-09-10T09:45:37.061 INFO:tasks.workunit.client.0.smithi005.stdout: 1 394297 5.50 MB/sec execute 198 sec latency 16.230 ms 2024-09-10T09:45:38.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:45:37 smithi005 ceph-mon[30350]: pgmap v611: 129 pgs: 129 active+clean; 153 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.6 MiB/s wr, 191 op/s 2024-09-10T09:45:38.060 INFO:tasks.workunit.client.0.smithi005.stdout: 1 396338 5.52 MB/sec execute 199 sec latency 22.305 ms 2024-09-10T09:45:38.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:45:37 smithi159 ceph-mon[35513]: pgmap v611: 129 pgs: 129 active+clean; 153 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.6 MiB/s wr, 191 op/s 2024-09-10T09:45:38.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:45:37 smithi102 ceph-mon[35442]: pgmap v611: 129 pgs: 129 active+clean; 153 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.6 MiB/s wr, 191 op/s 2024-09-10T09:45:39.061 INFO:tasks.workunit.client.0.smithi005.stdout: 1 398212 5.55 MB/sec execute 200 sec latency 23.424 ms 2024-09-10T09:45:40.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:45:39 smithi005 ceph-mon[30350]: pgmap v612: 129 pgs: 129 active+clean; 156 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.7 MiB/s wr, 195 op/s 2024-09-10T09:45:40.060 INFO:tasks.workunit.client.0.smithi005.stdout: 1 400949 5.60 MB/sec execute 201 sec latency 20.573 ms 2024-09-10T09:45:40.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:45:39 smithi159 ceph-mon[35513]: pgmap v612: 129 pgs: 129 active+clean; 156 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.7 MiB/s wr, 195 op/s 2024-09-10T09:45:40.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:45:39 smithi102 ceph-mon[35442]: pgmap v612: 129 pgs: 129 active+clean; 156 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.7 MiB/s wr, 195 op/s 2024-09-10T09:45:41.064 INFO:tasks.workunit.client.0.smithi005.stdout: 1 402480 5.61 MB/sec execute 202 sec latency 91.101 ms 2024-09-10T09:45:42.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:45:41 smithi005 ceph-mon[30350]: pgmap v613: 129 pgs: 129 active+clean; 157 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 1.7 MiB/s wr, 200 op/s 2024-09-10T09:45:42.061 INFO:tasks.workunit.client.0.smithi005.stdout: 1 404881 5.65 MB/sec execute 203 sec latency 9.978 ms 2024-09-10T09:45:42.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:45:41 smithi159 ceph-mon[35513]: pgmap v613: 129 pgs: 129 active+clean; 157 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 1.7 MiB/s wr, 200 op/s 2024-09-10T09:45:42.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:45:41 smithi102 ceph-mon[35442]: pgmap v613: 129 pgs: 129 active+clean; 157 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 1.7 MiB/s wr, 200 op/s 2024-09-10T09:45:43.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:45:42 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:45:43.061 INFO:tasks.workunit.client.0.smithi005.stdout: 1 407009 5.66 MB/sec execute 204 sec latency 20.910 ms 2024-09-10T09:45:43.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:45:42 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:45:43.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:45:42 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:45:44.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:45:43 smithi005 ceph-mon[30350]: pgmap v614: 129 pgs: 129 active+clean; 159 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.5 MiB/s wr, 175 op/s 2024-09-10T09:45:44.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:45:43 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:45:44.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:45:43 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:45:44.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:45:43 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:45:44.061 INFO:tasks.workunit.client.0.smithi005.stdout: 1 409132 5.70 MB/sec execute 205 sec latency 22.844 ms 2024-09-10T09:45:44.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:45:43 smithi159 ceph-mon[35513]: pgmap v614: 129 pgs: 129 active+clean; 159 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.5 MiB/s wr, 175 op/s 2024-09-10T09:45:44.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:45:43 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:45:44.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:45:43 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:45:44.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:45:43 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:45:44.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:45:43 smithi102 ceph-mon[35442]: pgmap v614: 129 pgs: 129 active+clean; 159 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.5 MiB/s wr, 175 op/s 2024-09-10T09:45:44.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:45:43 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:45:44.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:45:43 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:45:44.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:45:43 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:45:45.061 INFO:tasks.workunit.client.0.smithi005.stdout: 1 411735 5.75 MB/sec execute 206 sec latency 12.143 ms 2024-09-10T09:45:45.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:45:45 smithi005 ceph-mon[30350]: pgmap v615: 129 pgs: 129 active+clean; 160 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.3 MiB/s wr, 159 op/s 2024-09-10T09:45:46.061 INFO:tasks.workunit.client.0.smithi005.stdout: 1 413531 5.75 MB/sec execute 207 sec latency 28.350 ms 2024-09-10T09:45:46.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:45:45 smithi159 ceph-mon[35513]: pgmap v615: 129 pgs: 129 active+clean; 160 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.3 MiB/s wr, 159 op/s 2024-09-10T09:45:46.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:45:45 smithi102 ceph-mon[35442]: pgmap v615: 129 pgs: 129 active+clean; 160 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.3 MiB/s wr, 159 op/s 2024-09-10T09:45:47.062 INFO:tasks.workunit.client.0.smithi005.stdout: 1 415738 5.79 MB/sec execute 208 sec latency 34.037 ms 2024-09-10T09:45:48.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:45:47 smithi005 ceph-mon[30350]: pgmap v616: 129 pgs: 129 active+clean; 163 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.5 MiB/s wr, 209 op/s 2024-09-10T09:45:48.061 INFO:tasks.workunit.client.0.smithi005.stdout: 1 418013 5.82 MB/sec execute 209 sec latency 133.123 ms 2024-09-10T09:45:48.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:45:47 smithi159 ceph-mon[35513]: pgmap v616: 129 pgs: 129 active+clean; 163 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.5 MiB/s wr, 209 op/s 2024-09-10T09:45:48.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:45:47 smithi102 ceph-mon[35442]: pgmap v616: 129 pgs: 129 active+clean; 163 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.5 MiB/s wr, 209 op/s 2024-09-10T09:45:49.061 INFO:tasks.workunit.client.0.smithi005.stdout: 1 419985 5.85 MB/sec execute 210 sec latency 24.342 ms 2024-09-10T09:45:50.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:45:49 smithi005 ceph-mon[30350]: pgmap v617: 129 pgs: 129 active+clean; 165 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.4 MiB/s wr, 209 op/s 2024-09-10T09:45:50.061 INFO:tasks.workunit.client.0.smithi005.stdout: 1 422455 5.89 MB/sec execute 211 sec latency 10.154 ms 2024-09-10T09:45:50.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:45:49 smithi159 ceph-mon[35513]: pgmap v617: 129 pgs: 129 active+clean; 165 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.4 MiB/s wr, 209 op/s 2024-09-10T09:45:50.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:45:49 smithi102 ceph-mon[35442]: pgmap v617: 129 pgs: 129 active+clean; 165 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.4 MiB/s wr, 209 op/s 2024-09-10T09:45:51.062 INFO:tasks.workunit.client.0.smithi005.stdout: 1 424562 5.90 MB/sec execute 212 sec latency 26.427 ms 2024-09-10T09:45:51.921 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:45:51 smithi005 ceph-mon[30350]: pgmap v618: 129 pgs: 129 active+clean; 166 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 1.2 MiB/s wr, 176 op/s 2024-09-10T09:45:52.062 INFO:tasks.workunit.client.0.smithi005.stdout: 1 426362 5.93 MB/sec execute 213 sec latency 130.029 ms 2024-09-10T09:45:52.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:45:51 smithi159 ceph-mon[35513]: pgmap v618: 129 pgs: 129 active+clean; 166 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 1.2 MiB/s wr, 176 op/s 2024-09-10T09:45:52.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:45:51 smithi102 ceph-mon[35442]: pgmap v618: 129 pgs: 129 active+clean; 166 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 1.2 MiB/s wr, 176 op/s 2024-09-10T09:45:52.848 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T09:45:53.061 INFO:tasks.workunit.client.0.smithi005.stdout: 1 427365 5.92 MB/sec execute 214 sec latency 466.471 ms 2024-09-10T09:45:53.265 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:45:53.963 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:45:53 smithi005 ceph-mon[30350]: pgmap v619: 129 pgs: 129 active+clean; 170 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.4 MiB/s wr, 221 op/s 2024-09-10T09:45:53.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:53.975+0000 7f5013200640 1 -- 172.21.15.5:0/206810655 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f50141079c0 msgr2=0x7f5014107da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:45:53.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:53.976+0000 7f5013200640 1 --2- 172.21.15.5:0/206810655 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f50141079c0 0x7f5014107da0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f5004009900 tx=0x7f500402a3f0 comp rx=0 tx=0).stop 2024-09-10T09:45:53.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:53.976+0000 7f5013200640 1 -- 172.21.15.5:0/206810655 shutdown_connections 2024-09-10T09:45:53.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:53.976+0000 7f5013200640 1 --2- 172.21.15.5:0/206810655 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5014102360 0x7f501410e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:53.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:53.976+0000 7f5013200640 1 --2- 172.21.15.5:0/206810655 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f50141019c0 0x7f5014101e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:53.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:53.976+0000 7f5013200640 1 --2- 172.21.15.5:0/206810655 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f50141079c0 0x7f5014107da0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:53.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:53.976+0000 7f5013200640 1 -- 172.21.15.5:0/206810655 >> 172.21.15.5:0/206810655 conn(0x7f50140fd6e0 msgr2=0x7f50140ffb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:45:53.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:53.976+0000 7f5013200640 1 -- 172.21.15.5:0/206810655 shutdown_connections 2024-09-10T09:45:53.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:53.976+0000 7f5013200640 1 -- 172.21.15.5:0/206810655 wait complete. 2024-09-10T09:45:53.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:53.977+0000 7f5013200640 1 Processor -- start 2024-09-10T09:45:53.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:53.977+0000 7f5013200640 1 -- start start 2024-09-10T09:45:53.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:53.977+0000 7f5013200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f50141019c0 0x7f501419a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:45:53.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:53.977+0000 7f5013200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5014102360 0x7f501419a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:45:53.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:53.977+0000 7f5013200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f50141079c0 0x7f501419ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:45:53.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:53.977+0000 7f5013200640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f5014111e10 con 0x7f5014102360 2024-09-10T09:45:53.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:53.977+0000 7f5013200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f5014111c90 con 0x7f50141019c0 2024-09-10T09:45:53.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:53.977+0000 7f5013200640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f5014111f90 con 0x7f50141079c0 2024-09-10T09:45:53.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:53.978+0000 7f5011e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f50141019c0 0x7f501419a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:45:53.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:53.978+0000 7f5011400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5014102360 0x7f501419a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:45:53.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:53.978+0000 7f5011e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f50141019c0 0x7f501419a480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:46094/0 (socket says 172.21.15.5:46094) 2024-09-10T09:45:53.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:53.978+0000 7f5012800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f50141079c0 0x7f501419ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:45:53.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:53.978+0000 7f5011e00640 1 -- 172.21.15.5:0/3922624244 learned_addr learned my addr 172.21.15.5:0/3922624244 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:45:53.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:53.978+0000 7f5011e00640 1 -- 172.21.15.5:0/3922624244 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f50141079c0 msgr2=0x7f501419ed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:45:53.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:53.978+0000 7f5011e00640 1 --2- 172.21.15.5:0/3922624244 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f50141079c0 0x7f501419ed50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:53.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:53.978+0000 7f5011e00640 1 -- 172.21.15.5:0/3922624244 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5014102360 msgr2=0x7f501419a9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:45:53.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:53.978+0000 7f5011e00640 1 --2- 172.21.15.5:0/3922624244 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5014102360 0x7f501419a9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:53.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:53.978+0000 7f5011e00640 1 -- 172.21.15.5:0/3922624244 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f501419f430 con 0x7f50141019c0 2024-09-10T09:45:53.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:53.978+0000 7f5011400640 1 --2- 172.21.15.5:0/3922624244 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5014102360 0x7f501419a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:45:53.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:53.978+0000 7f5011e00640 1 --2- 172.21.15.5:0/3922624244 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f50141019c0 0x7f501419a480 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f500402a9e0 tx=0x7f5004003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:45:53.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:53.979+0000 7f500aa00640 1 -- 172.21.15.5:0/3922624244 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f500403f070 con 0x7f50141019c0 2024-09-10T09:45:53.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:53.979+0000 7f5013200640 1 -- 172.21.15.5:0/3922624244 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f501419f6c0 con 0x7f50141019c0 2024-09-10T09:45:53.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:53.979+0000 7f500aa00640 1 -- 172.21.15.5:0/3922624244 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5004031c30 con 0x7f50141019c0 2024-09-10T09:45:53.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:53.979+0000 7f500aa00640 1 -- 172.21.15.5:0/3922624244 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5004035030 con 0x7f50141019c0 2024-09-10T09:45:53.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:53.979+0000 7f5013200640 1 -- 172.21.15.5:0/3922624244 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50141a7000 con 0x7f50141019c0 2024-09-10T09:45:53.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:53.982+0000 7f501bc2f740 1 Processor -- start 2024-09-10T09:45:53.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:53.982+0000 7f501bc2f740 1 -- start start 2024-09-10T09:45:53.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:53.982+0000 7f501bc2f740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5645ad411d50 0x5645ad412130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:45:53.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:53.982+0000 7f501bc2f740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5645ad412670 0x5645ad41c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:45:53.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:53.982+0000 7f501bc2f740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5645ad41cf30 0x5645ad41f320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:45:53.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:53.982+0000 7f501bc2f740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5645ad3783e0 con 0x5645ad412670 2024-09-10T09:45:53.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:53.982+0000 7f501bc2f740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x5645ad31fd80 con 0x5645ad41cf30 2024-09-10T09:45:53.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:53.982+0000 7f501bc2f740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5645ad3743a0 con 0x5645ad411d50 2024-09-10T09:45:53.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:53.983+0000 7f500aa00640 1 -- 172.21.15.5:0/3922624244 <== mon.2 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f5004042070 con 0x7f50141019c0 2024-09-10T09:45:53.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:53.983+0000 7f5012800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5645ad411d50 0x5645ad412130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:45:53.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:53.983+0000 7f5012800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5645ad411d50 0x5645ad412130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:33892/0 (socket says 172.21.15.5:33892) 2024-09-10T09:45:53.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:53.983+0000 7f5012800640 1 -- 172.21.15.5:0/650376364 learned_addr learned my addr 172.21.15.5:0/650376364 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:45:53.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:53.983+0000 7f5011400640 1 --2- 172.21.15.5:0/650376364 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5645ad41cf30 0x5645ad41f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:45:53.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:53.983+0000 7f5011e00640 1 --2- 172.21.15.5:0/650376364 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5645ad412670 0x5645ad41c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:45:53.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:53.983+0000 7f5012800640 1 -- 172.21.15.5:0/650376364 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5645ad41cf30 msgr2=0x5645ad41f320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:45:53.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:53.983+0000 7f5012800640 1 --2- 172.21.15.5:0/650376364 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5645ad41cf30 0x5645ad41f320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:53.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:53.983+0000 7f5012800640 1 -- 172.21.15.5:0/650376364 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5645ad412670 msgr2=0x5645ad41c9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:45:53.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:53.983+0000 7f5012800640 1 --2- 172.21.15.5:0/650376364 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5645ad412670 0x5645ad41c9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:53.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:53.983+0000 7f5012800640 1 -- 172.21.15.5:0/650376364 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5645ad33eae0 con 0x5645ad411d50 2024-09-10T09:45:53.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:53.984+0000 7f5012800640 1 --2- 172.21.15.5:0/650376364 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5645ad411d50 0x5645ad412130 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f500c009f50 tx=0x7f500c0118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c14a46abe1a586d4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:45:53.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:53.984+0000 7f4fff400640 1 -- 172.21.15.5:0/650376364 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f500c004d20 con 0x5645ad411d50 2024-09-10T09:45:53.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:53.984+0000 7f4fff400640 1 -- 172.21.15.5:0/650376364 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f500c004050 con 0x5645ad411d50 2024-09-10T09:45:53.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:53.984+0000 7f4fff400640 1 -- 172.21.15.5:0/650376364 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f500c005280 con 0x5645ad411d50 2024-09-10T09:45:53.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:53.984+0000 7f500aa00640 1 --2- 172.21.15.5:0/3922624244 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f4ff80a2c80 0x7f4ff80a5140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:45:53.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:53.985+0000 7f500aa00640 1 -- 172.21.15.5:0/3922624244 <== mon.2 v2:172.21.15.102:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f50040ef640 con 0x7f50141019c0 2024-09-10T09:45:53.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:53.985+0000 7f5011e00640 1 --2- 172.21.15.5:0/3922624244 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f4ff80a2c80 0x7f4ff80a5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:45:53.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:53.985+0000 7f501bc2f740 1 -- 172.21.15.5:0/650376364 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5645ad411d50 msgr2=0x5645ad412130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:45:53.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:53.985+0000 7f501bc2f740 1 --2- 172.21.15.5:0/650376364 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5645ad411d50 0x5645ad412130 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f500c009f50 tx=0x7f500c0118d0 comp rx=0 tx=0).stop 2024-09-10T09:45:53.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:53.985+0000 7f501bc2f740 1 -- 172.21.15.5:0/650376364 shutdown_connections 2024-09-10T09:45:53.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:53.985+0000 7f501bc2f740 1 --2- 172.21.15.5:0/650376364 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5645ad41cf30 0x5645ad41f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:53.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:53.985+0000 7f501bc2f740 1 --2- 172.21.15.5:0/650376364 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5645ad412670 0x5645ad41c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:53.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:53.985+0000 7f501bc2f740 1 --2- 172.21.15.5:0/650376364 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5645ad411d50 0x5645ad412130 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:53.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:53.985+0000 7f501bc2f740 1 -- 172.21.15.5:0/650376364 >> 172.21.15.5:0/650376364 conn(0x5645ad40e2e0 msgr2=0x5645ad40e6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:45:53.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:53.985+0000 7f501bc2f740 1 -- 172.21.15.5:0/650376364 shutdown_connections 2024-09-10T09:45:53.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:53.985+0000 7f501bc2f740 1 -- 172.21.15.5:0/650376364 wait complete. 2024-09-10T09:45:53.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:53.985+0000 7f5011e00640 1 --2- 172.21.15.5:0/3922624244 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f4ff80a2c80 0x7f4ff80a5140 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f50040c4190 tx=0x7f500403b750 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:45:53.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:53.986+0000 7f501bc2f740 1 -- start start 2024-09-10T09:45:53.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:53.986+0000 7f501bc2f740 1 Processor -- start 2024-09-10T09:45:53.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:53.986+0000 7f501bc2f740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5645ad411d50 0x5645ad419db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:45:53.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:53.986+0000 7f501bc2f740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5645ad412670 0x5645ad41a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:45:53.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:53.986+0000 7f501bc2f740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5645ad41cf30 0x5645ad41a830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:45:53.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:53.986+0000 7f501bc2f740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5645ad41fba0 con 0x5645ad412670 2024-09-10T09:45:53.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:53.986+0000 7f501bc2f740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x5645ad3330f0 con 0x5645ad41cf30 2024-09-10T09:45:53.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:53.987+0000 7f501bc2f740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5645ad41fd20 con 0x5645ad411d50 2024-09-10T09:45:53.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:53.987+0000 7f501bc2f740 10 client.0 _ll_register_callbacks cb 0x5645ad3249e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:45:53.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:53.987+0000 7f5012800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5645ad411d50 0x5645ad419db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:45:53.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:53.987+0000 7f5011400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5645ad412670 0x5645ad41a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:45:53.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:53.987+0000 7f5012800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5645ad411d50 0x5645ad419db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:33904/0 (socket says 172.21.15.5:33904) 2024-09-10T09:45:53.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:53.987+0000 7f5012800640 1 -- 172.21.15.5:0/2281948779 learned_addr learned my addr 172.21.15.5:0/2281948779 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:45:53.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:53.987+0000 7f5012800640 1 --2- 172.21.15.5:0/2281948779 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5645ad41cf30 0x5645ad41a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:45:53.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:53.987+0000 7f5011400640 1 -- 172.21.15.5:0/2281948779 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5645ad411d50 msgr2=0x5645ad419db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:45:53.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:53.987+0000 7f5011400640 1 --2- 172.21.15.5:0/2281948779 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5645ad411d50 0x5645ad419db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:53.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:53.987+0000 7f5011400640 1 -- 172.21.15.5:0/2281948779 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5645ad41cf30 msgr2=0x5645ad41a830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:45:53.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:53.987+0000 7f5011400640 1 --2- 172.21.15.5:0/2281948779 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5645ad41cf30 0x5645ad41a830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:53.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:53.987+0000 7f5011400640 1 -- 172.21.15.5:0/2281948779 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5645ad0e93c0 con 0x5645ad412670 2024-09-10T09:45:53.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:53.987+0000 7f5012800640 1 --2- 172.21.15.5:0/2281948779 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5645ad411d50 0x5645ad419db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:45:53.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:53.988+0000 7f5011400640 1 --2- 172.21.15.5:0/2281948779 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5645ad412670 0x5645ad41a2f0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f5000012cf0 tx=0x7f500000f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56cd6b90b56daa8d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:45:53.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:53.988+0000 7f4ff3e00640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-10T09:45:53.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:53.988+0000 7f4ff3e00640 1 -- 172.21.15.5:0/2281948779 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5000012110 con 0x5645ad412670 2024-09-10T09:45:53.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:53.988+0000 7f501bc2f740 1 -- 172.21.15.5:0/2281948779 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f500c004050 con 0x5645ad412670 2024-09-10T09:45:53.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:53.988+0000 7f4ff3e00640 1 -- 172.21.15.5:0/2281948779 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f5000004050 con 0x5645ad412670 2024-09-10T09:45:53.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:53.988+0000 7f4ff3e00640 1 -- 172.21.15.5:0/2281948779 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f5000025050 con 0x5645ad412670 2024-09-10T09:45:53.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:53.988+0000 7f501bc2f740 10 client.16848 fetch_fsmap learned FSMap version 34 2024-09-10T09:45:53.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:53.988+0000 7f501bc2f740 1 -- 172.21.15.5:0/2281948779 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5645ad42b8c0 con 0x5645ad412670 2024-09-10T09:45:53.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:53.989+0000 7f4ff3e00640 1 -- 172.21.15.5:0/2281948779 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f500c004050 con 0x5645ad412670 2024-09-10T09:45:53.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:53.989+0000 7f4ff3e00640 1 -- 172.21.15.5:0/2281948779 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f5000010430 con 0x5645ad412670 2024-09-10T09:45:53.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:53.989+0000 7f501bc2f740 10 client.16848 fetch_fsmap finished waiting for FSMap version 34 2024-09-10T09:45:53.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:53.989+0000 7f501bc2f740 10 client.16848 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T09:45:53.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:53.989+0000 7f501bc2f740 20 client.16848 populate_metadata read hostname 'smithi005' 2024-09-10T09:45:53.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:53.989+0000 7f501bc2f740 1 --2- 172.21.15.5:0/2281948779 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x5645ad42cdd0 0x5645ad42f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:45:53.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:53.989+0000 7f501bc2f740 4 client.16848 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:45:53.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:53.989+0000 7f501bc2f740 1 -- 172.21.15.5:0/2281948779 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5645ad385950 con 0x5645ad42cdd0 2024-09-10T09:45:53.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:53.989+0000 7f5012800640 1 --2- 172.21.15.5:0/2281948779 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x5645ad42cdd0 0x5645ad42f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:45:53.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:53.990+0000 7f5012800640 1 --2- 172.21.15.5:0/2281948779 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x5645ad42cdd0 0x5645ad42f1c0 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9ad13d5dd2e8970 server_cookie=f03fd0a39a39c255 in_seq=0 out_seq=0 2024-09-10T09:45:53.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:53.990+0000 7f4ff3e00640 10 client.16848 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T09:45:53.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:53.991+0000 7f4ff3e00640 1 -- 172.21.15.5:0/2281948779 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5645ad385950 con 0x5645ad42cdd0 2024-09-10T09:45:53.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:53.991+0000 7f4ff3e00640 10 client.16848 handle_command_reply: tid=0 2024-09-10T09:45:53.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:53.991+0000 7f501bc2f740 1 client.16848 shutdown 2024-09-10T09:45:53.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:53.991+0000 7f501bc2f740 1 -- 172.21.15.5:0/2281948779 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5645ad412670 msgr2=0x5645ad41a2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:45:53.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:53.991+0000 7f501bc2f740 1 --2- 172.21.15.5:0/2281948779 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5645ad412670 0x5645ad41a2f0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f5000012cf0 tx=0x7f500000f790 comp rx=0 tx=0).stop 2024-09-10T09:45:53.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:53.991+0000 7f501bc2f740 1 -- 172.21.15.5:0/2281948779 shutdown_connections 2024-09-10T09:45:53.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:53.991+0000 7f501bc2f740 1 --2- 172.21.15.5:0/2281948779 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x5645ad42cdd0 0x5645ad42f1c0 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:53.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:53.991+0000 7f501bc2f740 1 --2- 172.21.15.5:0/2281948779 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5645ad41cf30 0x5645ad41a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:53.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:53.991+0000 7f4ff3e00640 0 client.16848 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T09:45:53.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:53.991+0000 7f501bc2f740 1 --2- 172.21.15.5:0/2281948779 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5645ad412670 0x5645ad41a2f0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:53.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:53.991+0000 7f501bc2f740 1 --2- 172.21.15.5:0/2281948779 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5645ad411d50 0x5645ad419db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:53.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:53.991+0000 7f501bc2f740 1 -- 172.21.15.5:0/2281948779 >> 172.21.15.5:0/2281948779 conn(0x5645ad40e2e0 msgr2=0x5645ad414010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:45:53.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:53.991+0000 7f501bc2f740 1 -- 172.21.15.5:0/2281948779 shutdown_connections 2024-09-10T09:45:53.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:53.991+0000 7f501bc2f740 1 -- 172.21.15.5:0/2281948779 wait complete. 2024-09-10T09:45:53.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:53.992+0000 7f501bc2f740 20 client.16848 trim_cache size 0 max 16384 2024-09-10T09:45:54.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:54.010+0000 7f501bc2f740 1 Processor -- start 2024-09-10T09:45:54.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:54.010+0000 7f501bc2f740 1 -- start start 2024-09-10T09:45:54.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:54.010+0000 7f501bc2f740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5645ad411d50 0x5645ad434ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:45:54.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:54.010+0000 7f501bc2f740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5645ad412670 0x5645ad434ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:45:54.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:54.010+0000 7f501bc2f740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5645ad41cf30 0x5645ad417030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:45:54.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:54.010+0000 7f501bc2f740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f50141b0e60 con 0x5645ad41cf30 2024-09-10T09:45:54.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:54.010+0000 7f501bc2f740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f50141b29a0 con 0x5645ad412670 2024-09-10T09:45:54.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:54.010+0000 7f501bc2f740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f50141b1530 con 0x5645ad411d50 2024-09-10T09:45:54.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:54.011+0000 7f5012800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5645ad411d50 0x5645ad434ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:45:54.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:54.011+0000 7f5012800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5645ad411d50 0x5645ad434ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:33914/0 (socket says 172.21.15.5:33914) 2024-09-10T09:45:54.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:54.011+0000 7f5012800640 1 -- 172.21.15.5:0/3887768250 learned_addr learned my addr 172.21.15.5:0/3887768250 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:45:54.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:54.011+0000 7f5011400640 1 --2- 172.21.15.5:0/3887768250 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5645ad412670 0x5645ad434ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:45:54.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:54.011+0000 7f5012800640 1 --2- 172.21.15.5:0/3887768250 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5645ad41cf30 0x5645ad417030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:45:54.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:54.011+0000 7f5012800640 1 -- 172.21.15.5:0/3887768250 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5645ad412670 msgr2=0x5645ad434ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:45:54.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:54.011+0000 7f5012800640 1 --2- 172.21.15.5:0/3887768250 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5645ad412670 0x5645ad434ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:54.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:54.011+0000 7f5012800640 1 -- 172.21.15.5:0/3887768250 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5645ad41cf30 msgr2=0x5645ad417030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:45:54.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:54.011+0000 7f5012800640 1 --2- 172.21.15.5:0/3887768250 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5645ad41cf30 0x5645ad417030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:54.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:54.011+0000 7f5012800640 1 -- 172.21.15.5:0/3887768250 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5645ad42f530 con 0x5645ad411d50 2024-09-10T09:45:54.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:54.011+0000 7f5011400640 1 --2- 172.21.15.5:0/3887768250 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5645ad412670 0x5645ad434ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:45:54.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:54.011+0000 7f5012800640 1 --2- 172.21.15.5:0/3887768250 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5645ad411d50 0x5645ad434ab0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f500c004920 tx=0x7f500c0118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=391ebe1c3e2c47a1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:45:54.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:54.012+0000 7f4ffea00640 1 -- 172.21.15.5:0/3887768250 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f500c03a070 con 0x5645ad411d50 2024-09-10T09:45:54.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:54.012+0000 7f4ffea00640 1 -- 172.21.15.5:0/3887768250 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f500c005040 con 0x5645ad411d50 2024-09-10T09:45:54.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:54.012+0000 7f4ffea00640 1 -- 172.21.15.5:0/3887768250 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f500c00f6e0 con 0x5645ad411d50 2024-09-10T09:45:54.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:54.012+0000 7f501bc2f740 1 -- 172.21.15.5:0/3887768250 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5645ad411d50 msgr2=0x5645ad434ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:45:54.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:54.012+0000 7f501bc2f740 1 --2- 172.21.15.5:0/3887768250 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5645ad411d50 0x5645ad434ab0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f500c004920 tx=0x7f500c0118d0 comp rx=0 tx=0).stop 2024-09-10T09:45:54.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:54.012+0000 7f501bc2f740 1 -- 172.21.15.5:0/3887768250 shutdown_connections 2024-09-10T09:45:54.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:54.012+0000 7f501bc2f740 1 --2- 172.21.15.5:0/3887768250 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5645ad41cf30 0x5645ad417030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:54.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:54.012+0000 7f501bc2f740 1 --2- 172.21.15.5:0/3887768250 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5645ad412670 0x5645ad434ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:54.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:54.012+0000 7f501bc2f740 1 --2- 172.21.15.5:0/3887768250 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5645ad411d50 0x5645ad434ab0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:54.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:54.012+0000 7f501bc2f740 1 -- 172.21.15.5:0/3887768250 >> 172.21.15.5:0/3887768250 conn(0x5645ad40e2e0 msgr2=0x5645ad41be30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:45:54.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:54.013+0000 7f501bc2f740 1 -- 172.21.15.5:0/3887768250 shutdown_connections 2024-09-10T09:45:54.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:54.013+0000 7f501bc2f740 1 -- 172.21.15.5:0/3887768250 wait complete. 2024-09-10T09:45:54.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:54.013+0000 7f501bc2f740 1 -- start start 2024-09-10T09:45:54.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:54.013+0000 7f501bc2f740 1 Processor -- start 2024-09-10T09:45:54.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:54.014+0000 7f501bc2f740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5645ad411d50 0x5645ad416f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:45:54.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:54.014+0000 7f501bc2f740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5645ad412670 0x5645ad4174b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:45:54.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:54.014+0000 7f501bc2f740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5645ad41cf30 0x5645ad4179f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:45:54.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:54.014+0000 7f501bc2f740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f50141b22d0 con 0x5645ad411d50 2024-09-10T09:45:54.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:54.014+0000 7f501bc2f740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f50141b1c00 con 0x5645ad412670 2024-09-10T09:45:54.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:54.014+0000 7f501bc2f740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f50141aee40 con 0x5645ad41cf30 2024-09-10T09:45:54.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:54.014+0000 7f501bc2f740 10 client.0 _ll_register_callbacks cb 0x7f5000012930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:45:54.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:54.014+0000 7f5011400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5645ad412670 0x5645ad4174b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:45:54.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:54.014+0000 7f5011400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5645ad412670 0x5645ad4174b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:46138/0 (socket says 172.21.15.5:46138) 2024-09-10T09:45:54.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:54.014+0000 7f5011400640 1 -- 172.21.15.5:0/557776886 learned_addr learned my addr 172.21.15.5:0/557776886 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:45:54.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:54.014+0000 7f5012800640 1 --2- 172.21.15.5:0/557776886 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5645ad411d50 0x5645ad416f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:45:54.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:54.014+0000 7f5012800640 1 --2- 172.21.15.5:0/557776886 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5645ad41cf30 0x5645ad4179f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:45:54.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:54.014+0000 7f5011400640 1 -- 172.21.15.5:0/557776886 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5645ad41cf30 msgr2=0x5645ad4179f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:45:54.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:54.014+0000 7f5011400640 1 --2- 172.21.15.5:0/557776886 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5645ad41cf30 0x5645ad4179f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:54.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:54.014+0000 7f5011400640 1 -- 172.21.15.5:0/557776886 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5645ad411d50 msgr2=0x5645ad416f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:45:54.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:54.014+0000 7f5011400640 1 --2- 172.21.15.5:0/557776886 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5645ad411d50 0x5645ad416f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:54.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:54.014+0000 7f5011400640 1 -- 172.21.15.5:0/557776886 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5645ad496be0 con 0x5645ad412670 2024-09-10T09:45:54.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:54.015+0000 7f5011400640 1 --2- 172.21.15.5:0/557776886 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5645ad412670 0x5645ad4174b0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f5000000c00 tx=0x7f5000012340 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a46f6549090a57f6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:45:54.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:54.015+0000 7f4ff3e00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-09-10T09:45:54.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:54.015+0000 7f4ff3e00640 1 -- 172.21.15.5:0/557776886 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5000022040 con 0x5645ad412670 2024-09-10T09:45:54.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:54.015+0000 7f4ff3e00640 1 -- 172.21.15.5:0/557776886 <== mon.2 v2:172.21.15.102:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f5000010a70 con 0x5645ad412670 2024-09-10T09:45:54.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:54.015+0000 7f501bc2f740 1 -- 172.21.15.5:0/557776886 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f500c005040 con 0x5645ad412670 2024-09-10T09:45:54.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:54.015+0000 7f4ff3e00640 1 -- 172.21.15.5:0/557776886 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f5000039050 con 0x5645ad412670 2024-09-10T09:45:54.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:54.015+0000 7f501bc2f740 10 client.26870 fetch_fsmap learned FSMap version 34 2024-09-10T09:45:54.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:54.015+0000 7f501bc2f740 1 -- 172.21.15.5:0/557776886 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5645ad497f00 con 0x5645ad412670 2024-09-10T09:45:54.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:54.016+0000 7f4ff3e00640 1 -- 172.21.15.5:0/557776886 <== mon.2 v2:172.21.15.102:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f500c005040 con 0x5645ad412670 2024-09-10T09:45:54.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:54.016+0000 7f4ff3e00640 1 -- 172.21.15.5:0/557776886 <== mon.2 v2:172.21.15.102:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f5000003710 con 0x5645ad412670 2024-09-10T09:45:54.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:54.016+0000 7f501bc2f740 10 client.26870 fetch_fsmap finished waiting for FSMap version 34 2024-09-10T09:45:54.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:54.016+0000 7f501bc2f740 10 client.26870 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T09:45:54.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:54.016+0000 7f501bc2f740 20 client.26870 populate_metadata read hostname 'smithi005' 2024-09-10T09:45:54.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:54.016+0000 7f501bc2f740 1 --2- 172.21.15.5:0/557776886 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x5645ad42cdd0 0x5645ad49a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:45:54.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:54.016+0000 7f501bc2f740 4 client.26870 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T09:45:54.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:54.016+0000 7f501bc2f740 1 -- 172.21.15.5:0/557776886 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5645ad37fe60 con 0x5645ad42cdd0 2024-09-10T09:45:54.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:54.016+0000 7f5012800640 1 --2- 172.21.15.5:0/557776886 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x5645ad42cdd0 0x5645ad49a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:45:54.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:54.017+0000 7f5012800640 1 --2- 172.21.15.5:0/557776886 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x5645ad42cdd0 0x5645ad49a6f0 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=491c6eb2aedea8b0 server_cookie=f2e8fc8270939086 in_seq=0 out_seq=0 2024-09-10T09:45:54.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:54.017+0000 7f4ff3e00640 10 client.26870 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T09:45:54.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:54.017+0000 7f4ff3e00640 1 -- 172.21.15.5:0/557776886 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5645ad37fe60 con 0x5645ad42cdd0 2024-09-10T09:45:54.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:54.017+0000 7f4ff3e00640 10 client.26870 handle_command_reply: tid=0 2024-09-10T09:45:54.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:54.017+0000 7f501bc2f740 1 client.26870 shutdown 2024-09-10T09:45:54.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:54.017+0000 7f501bc2f740 1 -- 172.21.15.5:0/557776886 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5645ad412670 msgr2=0x5645ad4174b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:45:54.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:54.017+0000 7f501bc2f740 1 --2- 172.21.15.5:0/557776886 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5645ad412670 0x5645ad4174b0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f5000000c00 tx=0x7f5000012340 comp rx=0 tx=0).stop 2024-09-10T09:45:54.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:54.017+0000 7f501bc2f740 1 -- 172.21.15.5:0/557776886 shutdown_connections 2024-09-10T09:45:54.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:54.017+0000 7f501bc2f740 1 --2- 172.21.15.5:0/557776886 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x5645ad42cdd0 0x5645ad49a6f0 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:54.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:54.017+0000 7f501bc2f740 1 --2- 172.21.15.5:0/557776886 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5645ad41cf30 0x5645ad4179f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:54.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:54.017+0000 7f501bc2f740 1 --2- 172.21.15.5:0/557776886 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5645ad412670 0x5645ad4174b0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:54.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:54.017+0000 7f501bc2f740 1 --2- 172.21.15.5:0/557776886 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5645ad411d50 0x5645ad416f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:54.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:54.017+0000 7f501bc2f740 1 -- 172.21.15.5:0/557776886 >> 172.21.15.5:0/557776886 conn(0x5645ad40e2e0 msgr2=0x5645ad4137a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:45:54.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:54.017+0000 7f4ff3e00640 0 client.26870 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T09:45:54.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:54.017+0000 7f501bc2f740 1 -- 172.21.15.5:0/557776886 shutdown_connections 2024-09-10T09:45:54.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:54.017+0000 7f501bc2f740 1 -- 172.21.15.5:0/557776886 wait complete. 2024-09-10T09:45:54.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:54.018+0000 7f501bc2f740 20 client.26870 trim_cache size 0 max 16384 2024-09-10T09:45:54.023 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-10T09:45:54.023 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "no active scrubs running", 2024-09-10T09:45:54.023 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": {} 2024-09-10T09:45:54.023 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-10T09:45:54.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:54.018+0000 7f5013200640 1 -- 172.21.15.5:0/3922624244 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f4ff80a2c80 msgr2=0x7f4ff80a5140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:45:54.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:54.018+0000 7f5013200640 1 --2- 172.21.15.5:0/3922624244 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f4ff80a2c80 0x7f4ff80a5140 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f50040c4190 tx=0x7f500403b750 comp rx=0 tx=0).stop 2024-09-10T09:45:54.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:54.018+0000 7f5013200640 1 -- 172.21.15.5:0/3922624244 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f50141019c0 msgr2=0x7f501419a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:45:54.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:54.018+0000 7f5013200640 1 --2- 172.21.15.5:0/3922624244 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f50141019c0 0x7f501419a480 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f500402a9e0 tx=0x7f5004003c10 comp rx=0 tx=0).stop 2024-09-10T09:45:54.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:54.018+0000 7f5013200640 1 -- 172.21.15.5:0/3922624244 shutdown_connections 2024-09-10T09:45:54.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:54.018+0000 7f5013200640 1 --2- 172.21.15.5:0/3922624244 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f4ff80a2c80 0x7f4ff80a5140 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:54.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:54.018+0000 7f5013200640 1 --2- 172.21.15.5:0/3922624244 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f50141079c0 0x7f501419ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:54.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:54.018+0000 7f5013200640 1 --2- 172.21.15.5:0/3922624244 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5014102360 0x7f501419a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:54.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:54.018+0000 7f5013200640 1 --2- 172.21.15.5:0/3922624244 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f50141019c0 0x7f501419a480 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:54.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:54.018+0000 7f5013200640 1 -- 172.21.15.5:0/3922624244 >> 172.21.15.5:0/3922624244 conn(0x7f50140fd6e0 msgr2=0x7f50140fec80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:45:54.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:54.019+0000 7f5013200640 1 -- 172.21.15.5:0/3922624244 shutdown_connections 2024-09-10T09:45:54.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:54.019+0000 7f5013200640 1 -- 172.21.15.5:0/3922624244 wait complete. 2024-09-10T09:45:54.061 INFO:tasks.workunit.client.0.smithi005.stdout: 1 429792 5.96 MB/sec execute 215 sec latency 9.512 ms 2024-09-10T09:45:54.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:45:53 smithi159 ceph-mon[35513]: pgmap v619: 129 pgs: 129 active+clean; 170 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.4 MiB/s wr, 221 op/s 2024-09-10T09:45:54.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:45:53 smithi102 ceph-mon[35442]: pgmap v619: 129 pgs: 129 active+clean; 170 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.4 MiB/s wr, 221 op/s 2024-09-10T09:45:54.918 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:45:54 smithi005 ceph-mon[30350]: pgmap v620: 129 pgs: 129 active+clean; 171 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 1.3 MiB/s wr, 190 op/s 2024-09-10T09:45:54.919 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-10T09:45:54.919 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T09:45:55.062 INFO:tasks.workunit.client.0.smithi005.stdout: 1 432192 5.99 MB/sec execute 216 sec latency 23.645 ms 2024-09-10T09:45:55.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:45:54 smithi159 ceph-mon[35513]: pgmap v620: 129 pgs: 129 active+clean; 171 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 1.3 MiB/s wr, 190 op/s 2024-09-10T09:45:55.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:45:54 smithi102 ceph-mon[35442]: pgmap v620: 129 pgs: 129 active+clean; 171 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 1.3 MiB/s wr, 190 op/s 2024-09-10T09:45:55.313 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:45:55.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:55.938+0000 7f3f66400640 1 -- 172.21.15.5:0/2162108784 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3f60076720 msgr2=0x7f3f60076b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:45:55.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:55.938+0000 7f3f66400640 1 --2- 172.21.15.5:0/2162108784 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3f60076720 0x7f3f60076b00 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f3f50009900 tx=0x7f3f5002a3f0 comp rx=0 tx=0).stop 2024-09-10T09:45:55.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:55.938+0000 7f3f66400640 1 -- 172.21.15.5:0/2162108784 shutdown_connections 2024-09-10T09:45:55.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:55.938+0000 7f3f66400640 1 --2- 172.21.15.5:0/2162108784 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3f60112a30 0x7f3f60114e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:55.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:55.938+0000 7f3f66400640 1 --2- 172.21.15.5:0/2162108784 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3f60077040 0x7f3f601124f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:55.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:55.938+0000 7f3f66400640 1 --2- 172.21.15.5:0/2162108784 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3f60076720 0x7f3f60076b00 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:55.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:55.938+0000 7f3f66400640 1 -- 172.21.15.5:0/2162108784 >> 172.21.15.5:0/2162108784 conn(0x7f3f600ffa90 msgr2=0x7f3f60101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:45:55.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:55.938+0000 7f3f66400640 1 -- 172.21.15.5:0/2162108784 shutdown_connections 2024-09-10T09:45:55.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:55.938+0000 7f3f66400640 1 -- 172.21.15.5:0/2162108784 wait complete. 2024-09-10T09:45:55.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:55.940+0000 7f3f66400640 1 Processor -- start 2024-09-10T09:45:55.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:55.940+0000 7f3f66400640 1 -- start start 2024-09-10T09:45:55.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:55.941+0000 7f3f66400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3f60076720 0x7f3f601a4710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:45:55.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:55.941+0000 7f3f66400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3f60077040 0x7f3f601a4c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:45:55.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:55.941+0000 7f3f66400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3f60112a30 0x7f3f6019e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:45:55.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:55.941+0000 7f3f66400640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f3f601185c0 con 0x7f3f60076720 2024-09-10T09:45:55.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:55.941+0000 7f3f66400640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f3f60118440 con 0x7f3f60077040 2024-09-10T09:45:55.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:55.941+0000 7f3f66400640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f3f60118740 con 0x7f3f60112a30 2024-09-10T09:45:55.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:55.941+0000 7f3f5f400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3f60076720 0x7f3f601a4710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:45:55.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:55.941+0000 7f3f5ea00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3f60077040 0x7f3f601a4c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:45:55.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:55.941+0000 7f3f5f400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3f60076720 0x7f3f601a4710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36606/0 (socket says 172.21.15.5:36606) 2024-09-10T09:45:55.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:55.941+0000 7f3f5f400640 1 -- 172.21.15.5:0/2336925069 learned_addr learned my addr 172.21.15.5:0/2336925069 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:45:55.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:55.942+0000 7f3f5fe00640 1 --2- 172.21.15.5:0/2336925069 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3f60112a30 0x7f3f6019e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:45:55.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:55.942+0000 7f3f5ea00640 1 -- 172.21.15.5:0/2336925069 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3f60112a30 msgr2=0x7f3f6019e8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:45:55.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:55.942+0000 7f3f5ea00640 1 --2- 172.21.15.5:0/2336925069 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3f60112a30 0x7f3f6019e8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:55.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:55.942+0000 7f3f5ea00640 1 -- 172.21.15.5:0/2336925069 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3f60076720 msgr2=0x7f3f601a4710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:45:55.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:55.942+0000 7f3f5ea00640 1 --2- 172.21.15.5:0/2336925069 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3f60076720 0x7f3f601a4710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:55.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:55.942+0000 7f3f5ea00640 1 -- 172.21.15.5:0/2336925069 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f6019f000 con 0x7f3f60077040 2024-09-10T09:45:55.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:55.942+0000 7f3f5ea00640 1 --2- 172.21.15.5:0/2336925069 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3f60077040 0x7f3f601a4c50 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f3f4c00e9f0 tx=0x7f3f4c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:45:55.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:55.942+0000 7f3f57e00640 1 -- 172.21.15.5:0/2336925069 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f4c00ccc0 con 0x7f3f60077040 2024-09-10T09:45:55.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:55.943+0000 7f3f66400640 1 -- 172.21.15.5:0/2336925069 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f6019f2f0 con 0x7f3f60077040 2024-09-10T09:45:55.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:55.943+0000 7f3f57e00640 1 -- 172.21.15.5:0/2336925069 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3f4c005020 con 0x7f3f60077040 2024-09-10T09:45:55.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:55.943+0000 7f3f57e00640 1 -- 172.21.15.5:0/2336925069 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f4c0049e0 con 0x7f3f60077040 2024-09-10T09:45:55.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:55.943+0000 7f3f66400640 1 -- 172.21.15.5:0/2336925069 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f601ab5e0 con 0x7f3f60077040 2024-09-10T09:45:55.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:55.944+0000 7f3f66400640 1 -- 172.21.15.5:0/2336925069 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f60076b00 con 0x7f3f60077040 2024-09-10T09:45:55.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:55.946+0000 7f3f57e00640 1 -- 172.21.15.5:0/2336925069 <== mon.2 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f3f4c005560 con 0x7f3f60077040 2024-09-10T09:45:55.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:55.949+0000 7f3f57e00640 1 --2- 172.21.15.5:0/2336925069 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f3f440a2c00 0x7f3f440a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:45:55.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:55.949+0000 7f3f57e00640 1 -- 172.21.15.5:0/2336925069 <== mon.2 v2:172.21.15.102:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f3f4c0cf360 con 0x7f3f60077040 2024-09-10T09:45:55.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:55.949+0000 7f3f5f400640 1 --2- 172.21.15.5:0/2336925069 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f3f440a2c00 0x7f3f440a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:45:55.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:55.950+0000 7f3f5f400640 1 --2- 172.21.15.5:0/2336925069 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f3f440a2c00 0x7f3f440a50c0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f3f500096c0 tx=0x7f3f50005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:45:55.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:55.952+0000 7f3f57e00640 1 -- 172.21.15.5:0/2336925069 <== mon.2 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f3f4c0051d0 con 0x7f3f60077040 2024-09-10T09:45:56.062 INFO:tasks.workunit.client.0.smithi005.stdout: 1 434175 6.01 MB/sec execute 217 sec latency 25.191 ms 2024-09-10T09:45:56.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:56.186+0000 7f3f66400640 1 -- 172.21.15.5:0/2336925069 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3f6010dbc0 con 0x7f3f60077040 2024-09-10T09:45:56.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:56.187+0000 7f3f57e00640 1 -- 172.21.15.5:0/2336925069 <== mon.2 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+11969 (secure 0 0 0) 0x7f3f4c09be70 con 0x7f3f60077040 2024-09-10T09:45:56.188 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-10T09:45:56.188 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":34,"btime":"2024-09-10T09:42:57:737230+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14454,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/2970594311","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2970594311},{"type":"v1","addr":"172.21.15.5:6835","nonce":2970594311}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14514,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6839/3616502174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":3616502174},{"type":"v1","addr":"172.21.15.5:6839","nonce":3616502174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":34,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T09:35:29.626287+0000","modified":"2024-09-10T09:42:56.961831+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24542,"mds_1":24535,"mds_2":24511,"mds_3":24506,"mds_4":24443},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14520":{"gid":14520,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/2073599842","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":2073599842},{"type":"v1","addr":"172.21.15.5:6841","nonce":2073599842}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"k","rank":4,"incarnation":26,"state":"up:active","state_seq":24,"addr":"172.21.15.102:6833/2457007427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":2457007427},{"type":"v1","addr":"172.21.15.102:6833","nonce":2457007427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/356445159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":356445159},{"type":"v1","addr":"172.21.15.159:6833","nonce":356445159}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6837/2894314507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":2894314507},{"type":"v1","addr":"172.21.15.5:6837","nonce":2894314507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.102:6835/1647549111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":1647549111},{"type":"v1","addr":"172.21.15.102:6835","nonce":1647549111}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6835/952657242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":952657242},{"type":"v1","addr":"172.21.15.159:6835","nonce":952657242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":19,"addr":"172.21.15.102:6837/2689776764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":2689776764},{"type":"v1","addr":"172.21.15.102:6837","nonce":2689776764}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.159:6837/620627639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":620627639},{"type":"v1","addr":"172.21.15.159:6837","nonce":620627639}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24535":{"gid":24535,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":18,"addr":"172.21.15.159:6839/2073739599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":2073739599},{"type":"v1","addr":"172.21.15.159:6839","nonce":2073739599}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24542":{"gid":24542,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.102:6839/335409749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6838","nonce":335409749},{"type":"v1","addr":"172.21.15.102:6839","nonce":335409749}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24542,"qdb_cluster":[24542,24535,24511,24506,24443]},"id":1}]} 2024-09-10T09:45:56.189 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 34 2024-09-10T09:45:56.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:56.189+0000 7f3f66400640 1 -- 172.21.15.5:0/2336925069 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f3f440a2c00 msgr2=0x7f3f440a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:45:56.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:56.189+0000 7f3f66400640 1 --2- 172.21.15.5:0/2336925069 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f3f440a2c00 0x7f3f440a50c0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f3f500096c0 tx=0x7f3f50005a20 comp rx=0 tx=0).stop 2024-09-10T09:45:56.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:56.189+0000 7f3f66400640 1 -- 172.21.15.5:0/2336925069 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3f60077040 msgr2=0x7f3f601a4c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:45:56.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:56.189+0000 7f3f66400640 1 --2- 172.21.15.5:0/2336925069 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3f60077040 0x7f3f601a4c50 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f3f4c00e9f0 tx=0x7f3f4c00eeb0 comp rx=0 tx=0).stop 2024-09-10T09:45:56.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:56.189+0000 7f3f66400640 1 -- 172.21.15.5:0/2336925069 shutdown_connections 2024-09-10T09:45:56.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:56.189+0000 7f3f66400640 1 --2- 172.21.15.5:0/2336925069 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f3f440a2c00 0x7f3f440a50c0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:56.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:56.189+0000 7f3f66400640 1 --2- 172.21.15.5:0/2336925069 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3f60112a30 0x7f3f6019e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:56.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:56.190+0000 7f3f66400640 1 --2- 172.21.15.5:0/2336925069 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3f60077040 0x7f3f601a4c50 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:56.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:56.190+0000 7f3f66400640 1 --2- 172.21.15.5:0/2336925069 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3f60076720 0x7f3f601a4710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:56.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:56.190+0000 7f3f66400640 1 -- 172.21.15.5:0/2336925069 >> 172.21.15.5:0/2336925069 conn(0x7f3f600ffa90 msgr2=0x7f3f60101e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:45:56.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:56.190+0000 7f3f66400640 1 -- 172.21.15.5:0/2336925069 shutdown_connections 2024-09-10T09:45:56.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:56.190+0000 7f3f66400640 1 -- 172.21.15.5:0/2336925069 wait complete. 2024-09-10T09:45:56.906 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:45:56 smithi005 ceph-mon[30350]: pgmap v621: 129 pgs: 129 active+clean; 172 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.5 MiB/s wr, 214 op/s 2024-09-10T09:45:56.906 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:45:56 smithi005 ceph-mon[30350]: from='client.? 172.21.15.5:0/2336925069' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:45:56.943 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-10T09:45:57.062 INFO:tasks.workunit.client.0.smithi005.stdout: 1 436577 6.05 MB/sec execute 218 sec latency 44.116 ms 2024-09-10T09:45:57.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:45:56 smithi159 ceph-mon[35513]: pgmap v621: 129 pgs: 129 active+clean; 172 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.5 MiB/s wr, 214 op/s 2024-09-10T09:45:57.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:45:56 smithi159 ceph-mon[35513]: from='client.? 172.21.15.5:0/2336925069' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:45:57.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:45:56 smithi102 ceph-mon[35442]: pgmap v621: 129 pgs: 129 active+clean; 172 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.5 MiB/s wr, 214 op/s 2024-09-10T09:45:57.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:45:56 smithi102 ceph-mon[35442]: from='client.? 172.21.15.5:0/2336925069' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:45:57.378 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:45:58.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:58.010+0000 7f7f51400640 1 -- 172.21.15.5:0/3007724830 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7f4c109120 msgr2=0x7f7f4c110ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:45:58.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:58.010+0000 7f7f51400640 1 --2- 172.21.15.5:0/3007724830 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7f4c109120 0x7f7f4c110ca0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f7f44009ca0 tx=0x7f7f4402a8b0 comp rx=0 tx=0).stop 2024-09-10T09:45:58.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:58.010+0000 7f7f51400640 1 -- 172.21.15.5:0/3007724830 shutdown_connections 2024-09-10T09:45:58.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:58.010+0000 7f7f51400640 1 --2- 172.21.15.5:0/3007724830 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7f4c109120 0x7f7f4c110ca0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:58.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:58.010+0000 7f7f51400640 1 --2- 172.21.15.5:0/3007724830 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7f4c1049a0 0x7f7f4c1089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:58.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:58.010+0000 7f7f51400640 1 --2- 172.21.15.5:0/3007724830 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7f4c103ff0 0x7f7f4c1043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:58.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:58.010+0000 7f7f51400640 1 -- 172.21.15.5:0/3007724830 >> 172.21.15.5:0/3007724830 conn(0x7f7f4c0ffa90 msgr2=0x7f7f4c101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:45:58.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:58.010+0000 7f7f51400640 1 -- 172.21.15.5:0/3007724830 shutdown_connections 2024-09-10T09:45:58.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:58.010+0000 7f7f51400640 1 -- 172.21.15.5:0/3007724830 wait complete. 2024-09-10T09:45:58.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:58.010+0000 7f7f51400640 1 Processor -- start 2024-09-10T09:45:58.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:58.010+0000 7f7f51400640 1 -- start start 2024-09-10T09:45:58.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:58.010+0000 7f7f51400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7f4c103ff0 0x7f7f4c19a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:45:58.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:58.010+0000 7f7f51400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7f4c1049a0 0x7f7f4c19ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:45:58.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:58.010+0000 7f7f51400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7f4c109120 0x7f7f4c19ee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:45:58.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:58.010+0000 7f7f51400640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f7f4c114180 con 0x7f7f4c109120 2024-09-10T09:45:58.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:58.010+0000 7f7f51400640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f7f4c114000 con 0x7f7f4c1049a0 2024-09-10T09:45:58.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:58.010+0000 7f7f51400640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f7f4c114300 con 0x7f7f4c103ff0 2024-09-10T09:45:58.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:58.010+0000 7f7f4b200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7f4c109120 0x7f7f4c19ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:45:58.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:58.010+0000 7f7f4a800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7f4c103ff0 0x7f7f4c19a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:45:58.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:58.010+0000 7f7f49e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7f4c1049a0 0x7f7f4c19ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:45:58.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:58.010+0000 7f7f49e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7f4c1049a0 0x7f7f4c19ab00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:46180/0 (socket says 172.21.15.5:46180) 2024-09-10T09:45:58.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:58.010+0000 7f7f49e00640 1 -- 172.21.15.5:0/336009010 learned_addr learned my addr 172.21.15.5:0/336009010 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:45:58.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:58.010+0000 7f7f4b200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7f4c109120 0x7f7f4c19ee90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36618/0 (socket says 172.21.15.5:36618) 2024-09-10T09:45:58.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:58.010+0000 7f7f4a800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7f4c103ff0 0x7f7f4c19a5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:33946/0 (socket says 172.21.15.5:33946) 2024-09-10T09:45:58.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:58.010+0000 7f7f49e00640 1 -- 172.21.15.5:0/336009010 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7f4c103ff0 msgr2=0x7f7f4c19a5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:45:58.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:58.010+0000 7f7f49e00640 1 --2- 172.21.15.5:0/336009010 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7f4c103ff0 0x7f7f4c19a5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:58.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:58.010+0000 7f7f49e00640 1 -- 172.21.15.5:0/336009010 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7f4c109120 msgr2=0x7f7f4c19ee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:45:58.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:58.010+0000 7f7f49e00640 1 --2- 172.21.15.5:0/336009010 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7f4c109120 0x7f7f4c19ee90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:58.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:58.010+0000 7f7f49e00640 1 -- 172.21.15.5:0/336009010 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f4c19f570 con 0x7f7f4c1049a0 2024-09-10T09:45:58.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:58.010+0000 7f7f49e00640 1 --2- 172.21.15.5:0/336009010 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7f4c1049a0 0x7f7f4c19ab00 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f7f3800ea60 tx=0x7f7f3800ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:45:58.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:58.010+0000 7f7f43400640 1 -- 172.21.15.5:0/336009010 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f3800cde0 con 0x7f7f4c1049a0 2024-09-10T09:45:58.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:58.010+0000 7f7f43400640 1 -- 172.21.15.5:0/336009010 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7f380035f0 con 0x7f7f4c1049a0 2024-09-10T09:45:58.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:58.010+0000 7f7f43400640 1 -- 172.21.15.5:0/336009010 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f38010670 con 0x7f7f4c1049a0 2024-09-10T09:45:58.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:58.010+0000 7f7f51400640 1 -- 172.21.15.5:0/336009010 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f4c19f860 con 0x7f7f4c1049a0 2024-09-10T09:45:58.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:58.010+0000 7f7f51400640 1 -- 172.21.15.5:0/336009010 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f4c1a7140 con 0x7f7f4c1049a0 2024-09-10T09:45:58.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:58.017+0000 7f7f53e49740 1 Processor -- start 2024-09-10T09:45:58.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:58.017+0000 7f7f53e49740 1 -- start start 2024-09-10T09:45:58.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:58.017+0000 7f7f43400640 1 -- 172.21.15.5:0/336009010 <== mon.2 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f7f380037a0 con 0x7f7f4c1049a0 2024-09-10T09:45:58.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:58.017+0000 7f7f53e49740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5641b567fc70 0x5641b5680050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:45:58.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:58.017+0000 7f7f53e49740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5641b5680590 0x5641b568a910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:45:58.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:58.017+0000 7f7f53e49740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5641b568ae50 0x5641b568d240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:45:58.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:58.017+0000 7f7f53e49740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5641b55e63e0 con 0x5641b5680590 2024-09-10T09:45:58.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:58.017+0000 7f7f53e49740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x5641b558dd80 con 0x5641b567fc70 2024-09-10T09:45:58.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:58.017+0000 7f7f53e49740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5641b55e23a0 con 0x5641b568ae50 2024-09-10T09:45:58.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:58.017+0000 7f7f4b200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5641b567fc70 0x5641b5680050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:45:58.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:58.017+0000 7f7f4a800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5641b5680590 0x5641b568a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:45:58.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:58.017+0000 7f7f4b200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5641b567fc70 0x5641b5680050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:46196/0 (socket says 172.21.15.5:46196) 2024-09-10T09:45:58.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:58.017+0000 7f7f4b200640 1 -- 172.21.15.5:0/1561402171 learned_addr learned my addr 172.21.15.5:0/1561402171 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:45:58.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:58.017+0000 7f7f4b200640 1 -- 172.21.15.5:0/1561402171 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5641b568ae50 msgr2=0x5641b568d240 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T09:45:58.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:58.017+0000 7f7f4b200640 1 --2- 172.21.15.5:0/1561402171 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5641b568ae50 0x5641b568d240 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:58.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:58.017+0000 7f7f4b200640 1 -- 172.21.15.5:0/1561402171 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5641b5680590 msgr2=0x5641b568a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:45:58.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:58.017+0000 7f7f4b200640 1 --2- 172.21.15.5:0/1561402171 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5641b5680590 0x5641b568a910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:58.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:58.017+0000 7f7f4b200640 1 -- 172.21.15.5:0/1561402171 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5641b55acae0 con 0x5641b567fc70 2024-09-10T09:45:58.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:58.017+0000 7f7f4b200640 1 --2- 172.21.15.5:0/1561402171 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5641b567fc70 0x5641b5680050 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f7f44009ca0 tx=0x7f7f440093e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c00a5d7750d63edf server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:45:58.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:58.017+0000 7f7f37400640 1 -- 172.21.15.5:0/1561402171 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f44030760 con 0x5641b567fc70 2024-09-10T09:45:58.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:58.017+0000 7f7f37400640 1 -- 172.21.15.5:0/1561402171 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7f44036030 con 0x5641b567fc70 2024-09-10T09:45:58.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:58.017+0000 7f7f37400640 1 -- 172.21.15.5:0/1561402171 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f440043c0 con 0x5641b567fc70 2024-09-10T09:45:58.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:58.017+0000 7f7f43400640 1 --2- 172.21.15.5:0/336009010 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f7f300a2c00 0x7f7f300a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:45:58.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:58.017+0000 7f7f43400640 1 -- 172.21.15.5:0/336009010 <== mon.2 v2:172.21.15.102:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f7f380cf660 con 0x7f7f4c1049a0 2024-09-10T09:45:58.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:58.017+0000 7f7f4a800640 1 --2- 172.21.15.5:0/336009010 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f7f300a2c00 0x7f7f300a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:45:58.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:58.017+0000 7f7f53e49740 1 -- 172.21.15.5:0/1561402171 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5641b567fc70 msgr2=0x5641b5680050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:45:58.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:58.017+0000 7f7f53e49740 1 --2- 172.21.15.5:0/1561402171 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5641b567fc70 0x5641b5680050 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f7f44009ca0 tx=0x7f7f440093e0 comp rx=0 tx=0).stop 2024-09-10T09:45:58.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:58.017+0000 7f7f53e49740 1 -- 172.21.15.5:0/1561402171 shutdown_connections 2024-09-10T09:45:58.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:58.017+0000 7f7f53e49740 1 --2- 172.21.15.5:0/1561402171 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5641b568ae50 0x5641b568d240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:58.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:58.017+0000 7f7f53e49740 1 --2- 172.21.15.5:0/1561402171 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5641b5680590 0x5641b568a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:58.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:58.017+0000 7f7f53e49740 1 --2- 172.21.15.5:0/1561402171 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5641b567fc70 0x5641b5680050 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:58.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:58.017+0000 7f7f53e49740 1 -- 172.21.15.5:0/1561402171 >> 172.21.15.5:0/1561402171 conn(0x5641b567c200 msgr2=0x5641b567c5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:45:58.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:58.023+0000 7f7f53e49740 1 -- 172.21.15.5:0/1561402171 shutdown_connections 2024-09-10T09:45:58.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:58.023+0000 7f7f53e49740 1 -- 172.21.15.5:0/1561402171 wait complete. 2024-09-10T09:45:58.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:58.023+0000 7f7f4a800640 1 --2- 172.21.15.5:0/336009010 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f7f300a2c00 0x7f7f300a50c0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x5641b567d060 tx=0x7f7f3c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:45:58.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:58.023+0000 7f7f53e49740 1 -- start start 2024-09-10T09:45:58.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:58.023+0000 7f7f53e49740 1 Processor -- start 2024-09-10T09:45:58.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:58.023+0000 7f7f53e49740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5641b567fc70 0x5641b5687da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:45:58.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:58.023+0000 7f7f53e49740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5641b5680590 0x5641b56882e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:45:58.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:58.023+0000 7f7f53e49740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5641b568ae50 0x5641b5688820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:45:58.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:58.023+0000 7f7f53e49740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5641b568da20 con 0x5641b5680590 2024-09-10T09:45:58.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:58.023+0000 7f7f53e49740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x5641b55a10f0 con 0x5641b567fc70 2024-09-10T09:45:58.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:58.023+0000 7f7f53e49740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5641b568dba0 con 0x5641b568ae50 2024-09-10T09:45:58.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:58.023+0000 7f7f53e49740 10 client.0 _ll_register_callbacks cb 0x5641b5588bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:45:58.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:58.023+0000 7f7f4a800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5641b567fc70 0x5641b5687da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:45:58.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:58.023+0000 7f7f4a800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5641b567fc70 0x5641b5687da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:46210/0 (socket says 172.21.15.5:46210) 2024-09-10T09:45:58.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:58.023+0000 7f7f4a800640 1 -- 172.21.15.5:0/3734203589 learned_addr learned my addr 172.21.15.5:0/3734203589 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:45:58.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:58.023+0000 7f7f49e00640 1 --2- 172.21.15.5:0/3734203589 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5641b5680590 0x5641b56882e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:45:58.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:58.023+0000 7f7f4a800640 1 -- 172.21.15.5:0/3734203589 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5641b568ae50 msgr2=0x5641b5688820 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T09:45:58.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:58.023+0000 7f7f4a800640 1 --2- 172.21.15.5:0/3734203589 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5641b568ae50 0x5641b5688820 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:58.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:58.023+0000 7f7f4a800640 1 -- 172.21.15.5:0/3734203589 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5641b5680590 msgr2=0x5641b56882e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:45:58.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:58.023+0000 7f7f4a800640 1 --2- 172.21.15.5:0/3734203589 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5641b5680590 0x5641b56882e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:58.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:58.023+0000 7f7f4a800640 1 -- 172.21.15.5:0/3734203589 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5641b53573c0 con 0x5641b567fc70 2024-09-10T09:45:58.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:58.023+0000 7f7f4a800640 1 --2- 172.21.15.5:0/3734203589 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5641b567fc70 0x5641b5687da0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f7f3c004640 tx=0x7f7f3c004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a1ad5f55f3e9023e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:45:58.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:58.023+0000 7f7f2be00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-09-10T09:45:58.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:58.023+0000 7f7f2be00640 1 -- 172.21.15.5:0/3734203589 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f3c01f610 con 0x5641b567fc70 2024-09-10T09:45:58.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:58.023+0000 7f7f53e49740 1 -- 172.21.15.5:0/3734203589 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7f44036030 con 0x5641b567fc70 2024-09-10T09:45:58.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:58.023+0000 7f7f2be00640 1 -- 172.21.15.5:0/3734203589 <== mon.2 v2:172.21.15.102:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f7f3c01f7b0 con 0x5641b567fc70 2024-09-10T09:45:58.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:58.023+0000 7f7f2be00640 1 -- 172.21.15.5:0/3734203589 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f7f3c032050 con 0x5641b567fc70 2024-09-10T09:45:58.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:58.023+0000 7f7f53e49740 10 client.26888 fetch_fsmap learned FSMap version 34 2024-09-10T09:45:58.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:58.023+0000 7f7f53e49740 1 -- 172.21.15.5:0/3734203589 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5641b5699ba0 con 0x5641b567fc70 2024-09-10T09:45:58.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:58.023+0000 7f7f2be00640 1 -- 172.21.15.5:0/3734203589 <== mon.2 v2:172.21.15.102:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7f44036030 con 0x5641b567fc70 2024-09-10T09:45:58.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:58.023+0000 7f7f2be00640 1 -- 172.21.15.5:0/3734203589 <== mon.2 v2:172.21.15.102:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f7f3c027020 con 0x5641b567fc70 2024-09-10T09:45:58.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:58.023+0000 7f7f53e49740 10 client.26888 fetch_fsmap finished waiting for FSMap version 34 2024-09-10T09:45:58.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:58.023+0000 7f7f53e49740 10 client.26888 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-09-10T09:45:58.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:58.023+0000 7f7f53e49740 20 client.26888 populate_metadata read hostname 'smithi005' 2024-09-10T09:45:58.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:58.023+0000 7f7f53e49740 1 --2- 172.21.15.5:0/3734203589 >> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] conn(0x5641b569afb0 0x5641b569d420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:45:58.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:58.023+0000 7f7f53e49740 4 client.26888 mds_command: new command op to 24443 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:45:58.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:58.023+0000 7f7f53e49740 1 -- 172.21.15.5:0/3734203589 --> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5641b55f3950 con 0x5641b569afb0 2024-09-10T09:45:58.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:58.023+0000 7f7f49e00640 1 --2- 172.21.15.5:0/3734203589 >> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] conn(0x5641b569afb0 0x5641b569d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:45:58.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:58.023+0000 7f7f49e00640 1 --2- 172.21.15.5:0/3734203589 >> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] conn(0x5641b569afb0 0x5641b569d420 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=ef0574d862625765 server_cookie=15756d634aebe0ed in_seq=0 out_seq=0 2024-09-10T09:45:58.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:58.023+0000 7f7f2be00640 10 client.26888 ms_handle_connect on v2:172.21.15.102:6832/2457007427 2024-09-10T09:45:58.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:58.028+0000 7f7f2be00640 1 -- 172.21.15.5:0/3734203589 <== mds.4 v2:172.21.15.102:6832/2457007427 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5641b55f3950 con 0x5641b569afb0 2024-09-10T09:45:58.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:58.028+0000 7f7f2be00640 10 client.26888 handle_command_reply: tid=0 2024-09-10T09:45:58.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:58.028+0000 7f7f53e49740 1 client.26888 shutdown 2024-09-10T09:45:58.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:58.028+0000 7f7f53e49740 1 -- 172.21.15.5:0/3734203589 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5641b567fc70 msgr2=0x5641b5687da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:45:58.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:58.028+0000 7f7f53e49740 1 --2- 172.21.15.5:0/3734203589 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5641b567fc70 0x5641b5687da0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f7f3c004640 tx=0x7f7f3c004730 comp rx=0 tx=0).stop 2024-09-10T09:45:58.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:58.028+0000 7f7f53e49740 1 -- 172.21.15.5:0/3734203589 shutdown_connections 2024-09-10T09:45:58.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:58.028+0000 7f7f53e49740 1 --2- 172.21.15.5:0/3734203589 >> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] conn(0x5641b569afb0 0x5641b569d420 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:58.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:58.028+0000 7f7f53e49740 1 --2- 172.21.15.5:0/3734203589 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5641b568ae50 0x5641b5688820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:58.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:58.028+0000 7f7f53e49740 1 --2- 172.21.15.5:0/3734203589 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5641b5680590 0x5641b56882e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:58.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:58.028+0000 7f7f53e49740 1 --2- 172.21.15.5:0/3734203589 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5641b567fc70 0x5641b5687da0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:58.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:58.028+0000 7f7f53e49740 1 -- 172.21.15.5:0/3734203589 >> 172.21.15.5:0/3734203589 conn(0x5641b567c200 msgr2=0x5641b5681f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:45:58.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:58.028+0000 7f7f2be00640 0 client.26888 ms_handle_reset on v2:172.21.15.102:6832/2457007427 2024-09-10T09:45:58.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:58.028+0000 7f7f53e49740 1 -- 172.21.15.5:0/3734203589 shutdown_connections 2024-09-10T09:45:58.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:58.028+0000 7f7f53e49740 1 -- 172.21.15.5:0/3734203589 wait complete. 2024-09-10T09:45:58.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:58.028+0000 7f7f53e49740 20 client.26888 trim_cache size 0 max 16384 2024-09-10T09:45:58.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:58.041+0000 7f7f53e49740 1 Processor -- start 2024-09-10T09:45:58.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:58.041+0000 7f7f53e49740 1 -- start start 2024-09-10T09:45:58.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:58.041+0000 7f7f53e49740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5641b567fc70 0x5641b56a2cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:45:58.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:58.041+0000 7f7f53e49740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5641b5680590 0x5641b56a3230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:45:58.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:58.041+0000 7f7f53e49740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5641b568ae50 0x5641b567ab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:45:58.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:58.041+0000 7f7f53e49740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f7f4c1b0fb0 con 0x5641b568ae50 2024-09-10T09:45:58.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:58.041+0000 7f7f53e49740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f7f4c1b2af0 con 0x5641b567fc70 2024-09-10T09:45:58.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:58.041+0000 7f7f53e49740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f7f4c1b1680 con 0x5641b5680590 2024-09-10T09:45:58.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:58.041+0000 7f7f4b200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5641b568ae50 0x5641b567ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:45:58.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:58.041+0000 7f7f4b200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5641b568ae50 0x5641b567ab60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36642/0 (socket says 172.21.15.5:36642) 2024-09-10T09:45:58.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:58.041+0000 7f7f4b200640 1 -- 172.21.15.5:0/1447650099 learned_addr learned my addr 172.21.15.5:0/1447650099 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:45:58.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:58.041+0000 7f7f49e00640 1 --2- 172.21.15.5:0/1447650099 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5641b5680590 0x5641b56a3230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:45:58.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:58.041+0000 7f7f4b200640 1 -- 172.21.15.5:0/1447650099 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5641b5680590 msgr2=0x5641b56a3230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:45:58.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:58.041+0000 7f7f4b200640 1 --2- 172.21.15.5:0/1447650099 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5641b5680590 0x5641b56a3230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:58.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:58.041+0000 7f7f4b200640 1 -- 172.21.15.5:0/1447650099 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5641b567fc70 msgr2=0x5641b56a2cf0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T09:45:58.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:58.041+0000 7f7f4b200640 1 --2- 172.21.15.5:0/1447650099 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5641b567fc70 0x5641b56a2cf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:58.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:58.041+0000 7f7f4b200640 1 -- 172.21.15.5:0/1447650099 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5641b56803c0 con 0x5641b568ae50 2024-09-10T09:45:58.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:58.041+0000 7f7f49e00640 1 --2- 172.21.15.5:0/1447650099 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5641b5680590 0x5641b56a3230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:45:58.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:58.041+0000 7f7f4b200640 1 --2- 172.21.15.5:0/1447650099 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5641b568ae50 0x5641b567ab60 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f7f44009bf0 tx=0x7f7f44000f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6a92e06ff783f4a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:45:58.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:58.041+0000 7f7f36a00640 1 -- 172.21.15.5:0/1447650099 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f44030560 con 0x5641b568ae50 2024-09-10T09:45:58.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:58.041+0000 7f7f36a00640 1 -- 172.21.15.5:0/1447650099 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7f44036040 con 0x5641b568ae50 2024-09-10T09:45:58.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:58.041+0000 7f7f36a00640 1 -- 172.21.15.5:0/1447650099 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f44004040 con 0x5641b568ae50 2024-09-10T09:45:58.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:58.046+0000 7f7f53e49740 1 -- 172.21.15.5:0/1447650099 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5641b568ae50 msgr2=0x5641b567ab60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:45:58.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:58.046+0000 7f7f53e49740 1 --2- 172.21.15.5:0/1447650099 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5641b568ae50 0x5641b567ab60 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f7f44009bf0 tx=0x7f7f44000f40 comp rx=0 tx=0).stop 2024-09-10T09:45:58.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:58.046+0000 7f7f53e49740 1 -- 172.21.15.5:0/1447650099 shutdown_connections 2024-09-10T09:45:58.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:58.046+0000 7f7f53e49740 1 --2- 172.21.15.5:0/1447650099 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5641b568ae50 0x5641b567ab60 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:58.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:58.046+0000 7f7f53e49740 1 --2- 172.21.15.5:0/1447650099 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5641b5680590 0x5641b56a3230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:58.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:58.046+0000 7f7f53e49740 1 --2- 172.21.15.5:0/1447650099 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5641b567fc70 0x5641b56a2cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:58.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:58.046+0000 7f7f53e49740 1 -- 172.21.15.5:0/1447650099 >> 172.21.15.5:0/1447650099 conn(0x5641b567c200 msgr2=0x5641b5686e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:45:58.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:58.046+0000 7f7f53e49740 1 -- 172.21.15.5:0/1447650099 shutdown_connections 2024-09-10T09:45:58.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:58.046+0000 7f7f53e49740 1 -- 172.21.15.5:0/1447650099 wait complete. 2024-09-10T09:45:58.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:58.046+0000 7f7f53e49740 1 -- start start 2024-09-10T09:45:58.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:58.046+0000 7f7f53e49740 1 Processor -- start 2024-09-10T09:45:58.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:58.046+0000 7f7f53e49740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5641b567fc70 0x5641b5693220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:45:58.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:58.046+0000 7f7f53e49740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5641b5680590 0x5641b568e550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:45:58.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:58.046+0000 7f7f53e49740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5641b568ae50 0x5641b568ea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:45:58.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:58.046+0000 7f7f53e49740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f7f4c1b2420 con 0x5641b5680590 2024-09-10T09:45:58.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:58.046+0000 7f7f53e49740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f7f4c1b1d50 con 0x5641b568ae50 2024-09-10T09:45:58.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:58.046+0000 7f7f53e49740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f7f4c1aef90 con 0x5641b567fc70 2024-09-10T09:45:58.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:58.046+0000 7f7f53e49740 10 client.0 _ll_register_callbacks cb 0x7f7f3c009cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:45:58.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:58.046+0000 7f7f49e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5641b5680590 0x5641b568e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:45:58.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:58.046+0000 7f7f49e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5641b5680590 0x5641b568e550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36658/0 (socket says 172.21.15.5:36658) 2024-09-10T09:45:58.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:58.046+0000 7f7f49e00640 1 -- 172.21.15.5:0/2462061905 learned_addr learned my addr 172.21.15.5:0/2462061905 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:45:58.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:58.046+0000 7f7f4a800640 1 --2- 172.21.15.5:0/2462061905 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5641b567fc70 0x5641b5693220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:45:58.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:58.046+0000 7f7f49e00640 1 -- 172.21.15.5:0/2462061905 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5641b567fc70 msgr2=0x5641b5693220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:45:58.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:58.046+0000 7f7f49e00640 1 --2- 172.21.15.5:0/2462061905 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5641b567fc70 0x5641b5693220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:58.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:58.046+0000 7f7f49e00640 1 -- 172.21.15.5:0/2462061905 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5641b568ae50 msgr2=0x5641b568ea90 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T09:45:58.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:58.046+0000 7f7f49e00640 1 --2- 172.21.15.5:0/2462061905 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5641b568ae50 0x5641b568ea90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:58.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:58.046+0000 7f7f49e00640 1 -- 172.21.15.5:0/2462061905 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5641b568efd0 con 0x5641b5680590 2024-09-10T09:45:58.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:58.046+0000 7f7f49e00640 1 --2- 172.21.15.5:0/2462061905 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5641b5680590 0x5641b568e550 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f7f3800d010 tx=0x7f7f3809bb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73fa1f3a044973f9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:45:58.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:58.046+0000 7f7f2be00640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-10T09:45:58.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:58.046+0000 7f7f2be00640 1 -- 172.21.15.5:0/2462061905 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f38099d70 con 0x5641b5680590 2024-09-10T09:45:58.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:58.046+0000 7f7f2be00640 1 -- 172.21.15.5:0/2462061905 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f7f38003b30 con 0x5641b5680590 2024-09-10T09:45:58.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:58.046+0000 7f7f53e49740 1 -- 172.21.15.5:0/2462061905 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7f44036040 con 0x5641b5680590 2024-09-10T09:45:58.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:58.046+0000 7f7f2be00640 1 -- 172.21.15.5:0/2462061905 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f7f380980e0 con 0x5641b5680590 2024-09-10T09:45:58.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:58.046+0000 7f7f53e49740 10 client.16878 fetch_fsmap learned FSMap version 34 2024-09-10T09:45:58.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:58.046+0000 7f7f53e49740 1 -- 172.21.15.5:0/2462061905 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5641b570e1e0 con 0x5641b5680590 2024-09-10T09:45:58.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:58.046+0000 7f7f2be00640 1 -- 172.21.15.5:0/2462061905 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7f44036040 con 0x5641b5680590 2024-09-10T09:45:58.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:58.046+0000 7f7f2be00640 1 -- 172.21.15.5:0/2462061905 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f7f380977c0 con 0x5641b5680590 2024-09-10T09:45:58.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:58.046+0000 7f7f53e49740 10 client.16878 fetch_fsmap finished waiting for FSMap version 34 2024-09-10T09:45:58.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:58.046+0000 7f7f53e49740 10 client.16878 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-09-10T09:45:58.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:58.046+0000 7f7f53e49740 20 client.16878 populate_metadata read hostname 'smithi005' 2024-09-10T09:45:58.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:58.046+0000 7f7f53e49740 1 --2- 172.21.15.5:0/2462061905 >> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] conn(0x5641b570e910 0x5641b5710d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:45:58.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:58.046+0000 7f7f53e49740 4 client.16878 mds_command: new command op to 24443 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T09:45:58.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:58.046+0000 7f7f53e49740 1 -- 172.21.15.5:0/2462061905 --> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5641b55ede60 con 0x5641b570e910 2024-09-10T09:45:58.062 INFO:tasks.workunit.client.0.smithi005.stdout: 1 436580 6.02 MB/sec execute 219 sec latency 698.431 ms 2024-09-10T09:45:58.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:58.064+0000 7f7f4a800640 1 --2- 172.21.15.5:0/2462061905 >> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] conn(0x5641b570e910 0x5641b5710d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:45:58.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:58.064+0000 7f7f4a800640 1 --2- 172.21.15.5:0/2462061905 >> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] conn(0x5641b570e910 0x5641b5710d30 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=9c244a3b347c5368 server_cookie=18bbc6aabcf14974 in_seq=0 out_seq=0 2024-09-10T09:45:58.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:58.064+0000 7f7f2be00640 10 client.16878 ms_handle_connect on v2:172.21.15.102:6832/2457007427 2024-09-10T09:45:58.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:58.064+0000 7f7f2be00640 1 -- 172.21.15.5:0/2462061905 <== mds.4 v2:172.21.15.102:6832/2457007427 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5641b55ede60 con 0x5641b570e910 2024-09-10T09:45:58.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:58.064+0000 7f7f2be00640 10 client.16878 handle_command_reply: tid=0 2024-09-10T09:45:58.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:58.064+0000 7f7f53e49740 1 client.16878 shutdown 2024-09-10T09:45:58.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:58.064+0000 7f7f53e49740 1 -- 172.21.15.5:0/2462061905 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5641b5680590 msgr2=0x5641b568e550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:45:58.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:58.064+0000 7f7f53e49740 1 --2- 172.21.15.5:0/2462061905 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5641b5680590 0x5641b568e550 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f7f3800d010 tx=0x7f7f3809bb30 comp rx=0 tx=0).stop 2024-09-10T09:45:58.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:58.064+0000 7f7f53e49740 1 -- 172.21.15.5:0/2462061905 shutdown_connections 2024-09-10T09:45:58.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:58.064+0000 7f7f53e49740 1 --2- 172.21.15.5:0/2462061905 >> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] conn(0x5641b570e910 0x5641b5710d30 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:58.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:58.064+0000 7f7f53e49740 1 --2- 172.21.15.5:0/2462061905 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5641b568ae50 0x5641b568ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:58.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:58.064+0000 7f7f53e49740 1 --2- 172.21.15.5:0/2462061905 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5641b5680590 0x5641b568e550 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:58.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:58.064+0000 7f7f53e49740 1 --2- 172.21.15.5:0/2462061905 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5641b567fc70 0x5641b5693220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:58.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:58.064+0000 7f7f53e49740 1 -- 172.21.15.5:0/2462061905 >> 172.21.15.5:0/2462061905 conn(0x5641b567c200 msgr2=0x5641b567ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:45:58.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:58.064+0000 7f7f2be00640 0 client.16878 ms_handle_reset on v2:172.21.15.102:6832/2457007427 2024-09-10T09:45:58.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:58.064+0000 7f7f53e49740 1 -- 172.21.15.5:0/2462061905 shutdown_connections 2024-09-10T09:45:58.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:58.064+0000 7f7f53e49740 1 -- 172.21.15.5:0/2462061905 wait complete. 2024-09-10T09:45:58.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:58.064+0000 7f7f53e49740 20 client.16878 trim_cache size 0 max 16384 2024-09-10T09:45:58.070 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-10T09:45:58.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:58.064+0000 7f7f51400640 1 -- 172.21.15.5:0/336009010 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f7f300a2c00 msgr2=0x7f7f300a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:45:58.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:58.064+0000 7f7f51400640 1 --2- 172.21.15.5:0/336009010 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f7f300a2c00 0x7f7f300a50c0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x5641b567d060 tx=0x7f7f3c005f90 comp rx=0 tx=0).stop 2024-09-10T09:45:58.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:58.064+0000 7f7f51400640 1 -- 172.21.15.5:0/336009010 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7f4c1049a0 msgr2=0x7f7f4c19ab00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:45:58.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:58.064+0000 7f7f51400640 1 --2- 172.21.15.5:0/336009010 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7f4c1049a0 0x7f7f4c19ab00 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f7f3800ea60 tx=0x7f7f3800ef20 comp rx=0 tx=0).stop 2024-09-10T09:45:58.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:58.064+0000 7f7f51400640 1 -- 172.21.15.5:0/336009010 shutdown_connections 2024-09-10T09:45:58.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:58.064+0000 7f7f51400640 1 --2- 172.21.15.5:0/336009010 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f7f300a2c00 0x7f7f300a50c0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:58.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:58.064+0000 7f7f51400640 1 --2- 172.21.15.5:0/336009010 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7f4c109120 0x7f7f4c19ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:58.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:58.064+0000 7f7f51400640 1 --2- 172.21.15.5:0/336009010 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7f4c1049a0 0x7f7f4c19ab00 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:58.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:58.064+0000 7f7f51400640 1 --2- 172.21.15.5:0/336009010 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7f4c103ff0 0x7f7f4c19a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:45:58.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:58.064+0000 7f7f51400640 1 -- 172.21.15.5:0/336009010 >> 172.21.15.5:0/336009010 conn(0x7f7f4c0ffa90 msgr2=0x7f7f4c1005b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:45:58.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:58.064+0000 7f7f51400640 1 -- 172.21.15.5:0/336009010 shutdown_connections 2024-09-10T09:45:58.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:45:58.064+0000 7f7f51400640 1 -- 172.21.15.5:0/336009010 wait complete. 2024-09-10T09:45:58.821 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:45:58 smithi005 ceph-mon[30350]: pgmap v622: 129 pgs: 129 active+clean; 174 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.4 MiB/s wr, 183 op/s 2024-09-10T09:45:59.062 INFO:tasks.workunit.client.0.smithi005.stdout: 1 436651 6.00 MB/sec execute 220 sec latency 1613.849 ms 2024-09-10T09:45:59.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:45:58 smithi159 ceph-mon[35513]: pgmap v622: 129 pgs: 129 active+clean; 174 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.4 MiB/s wr, 183 op/s 2024-09-10T09:45:59.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:45:58 smithi102 ceph-mon[35442]: pgmap v622: 129 pgs: 129 active+clean; 174 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.4 MiB/s wr, 183 op/s 2024-09-10T09:45:59.545 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-10T09:45:59.977 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:46:00.062 INFO:tasks.workunit.client.0.smithi005.stdout: 1 438649 6.00 MB/sec execute 221 sec latency 24.402 ms 2024-09-10T09:46:00.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:00.547+0000 7f11afc00640 1 -- 172.21.15.5:0/688099205 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f11a81007f0 msgr2=0x7f11a8109bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:46:00.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:00.547+0000 7f11afc00640 1 --2- 172.21.15.5:0/688099205 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f11a81007f0 0x7f11a8109bc0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f1194009900 tx=0x7f119402a3f0 comp rx=0 tx=0).stop 2024-09-10T09:46:00.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:00.547+0000 7f11afc00640 1 -- 172.21.15.5:0/688099205 shutdown_connections 2024-09-10T09:46:00.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:00.547+0000 7f11afc00640 1 --2- 172.21.15.5:0/688099205 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f11a810a2f0 0x7f11a8110c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:00.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:00.547+0000 7f11afc00640 1 --2- 172.21.15.5:0/688099205 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f11a81007f0 0x7f11a8109bc0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:00.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:00.547+0000 7f11afc00640 1 --2- 172.21.15.5:0/688099205 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f11a80ffe40 0x7f11a8100220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:00.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:00.547+0000 7f11afc00640 1 -- 172.21.15.5:0/688099205 >> 172.21.15.5:0/688099205 conn(0x7f11a80fba90 msgr2=0x7f11a80fdeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:46:00.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:00.547+0000 7f11afc00640 1 -- 172.21.15.5:0/688099205 shutdown_connections 2024-09-10T09:46:00.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:00.547+0000 7f11afc00640 1 -- 172.21.15.5:0/688099205 wait complete. 2024-09-10T09:46:00.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:00.548+0000 7f11afc00640 1 Processor -- start 2024-09-10T09:46:00.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:00.548+0000 7f11afc00640 1 -- start start 2024-09-10T09:46:00.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:00.549+0000 7f11afc00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f11a80ffe40 0x7f11a819a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:46:00.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:00.549+0000 7f11afc00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f11a81007f0 0x7f11a819ac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:46:00.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:00.549+0000 7f11afc00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f11a810a2f0 0x7f11a819f000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:46:00.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:00.549+0000 7f11afc00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f11a81140f0 con 0x7f11a80ffe40 2024-09-10T09:46:00.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:00.549+0000 7f11afc00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f11a8113f70 con 0x7f11a81007f0 2024-09-10T09:46:00.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:00.549+0000 7f11afc00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f11a8114270 con 0x7f11a810a2f0 2024-09-10T09:46:00.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:00.549+0000 7f11a7e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f11a81007f0 0x7f11a819ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:46:00.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:00.549+0000 7f11a7e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f11a81007f0 0x7f11a819ac70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:46254/0 (socket says 172.21.15.5:46254) 2024-09-10T09:46:00.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:00.549+0000 7f11a7e00640 1 -- 172.21.15.5:0/961697775 learned_addr learned my addr 172.21.15.5:0/961697775 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:46:00.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:00.549+0000 7f11adc00640 1 --2- 172.21.15.5:0/961697775 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f11a810a2f0 0x7f11a819f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:46:00.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:00.549+0000 7f11ad200640 1 --2- 172.21.15.5:0/961697775 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f11a80ffe40 0x7f11a819a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:46:00.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:00.549+0000 7f11a7e00640 1 -- 172.21.15.5:0/961697775 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f11a810a2f0 msgr2=0x7f11a819f000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:46:00.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:00.549+0000 7f11a7e00640 1 --2- 172.21.15.5:0/961697775 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f11a810a2f0 0x7f11a819f000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:00.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:00.549+0000 7f11a7e00640 1 -- 172.21.15.5:0/961697775 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f11a80ffe40 msgr2=0x7f11a819a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:46:00.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:00.549+0000 7f11a7e00640 1 --2- 172.21.15.5:0/961697775 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f11a80ffe40 0x7f11a819a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:00.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:00.549+0000 7f11a7e00640 1 -- 172.21.15.5:0/961697775 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11a819f6e0 con 0x7f11a81007f0 2024-09-10T09:46:00.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:00.550+0000 7f11a7e00640 1 --2- 172.21.15.5:0/961697775 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f11a81007f0 0x7f11a819ac70 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f1194006010 tx=0x7f11940042f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:46:00.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:00.550+0000 7f11a5600640 1 -- 172.21.15.5:0/961697775 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f119403f030 con 0x7f11a81007f0 2024-09-10T09:46:00.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:00.550+0000 7f11a5600640 1 -- 172.21.15.5:0/961697775 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1194003ce0 con 0x7f11a81007f0 2024-09-10T09:46:00.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:00.550+0000 7f11a5600640 1 -- 172.21.15.5:0/961697775 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1194035070 con 0x7f11a81007f0 2024-09-10T09:46:00.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:00.550+0000 7f11afc00640 1 -- 172.21.15.5:0/961697775 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11a819f970 con 0x7f11a81007f0 2024-09-10T09:46:00.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:00.550+0000 7f11afc00640 1 -- 172.21.15.5:0/961697775 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11a819fe00 con 0x7f11a81007f0 2024-09-10T09:46:00.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:00.553+0000 7f11a5600640 1 -- 172.21.15.5:0/961697775 <== mon.2 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f1194042030 con 0x7f11a81007f0 2024-09-10T09:46:00.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:00.553+0000 7f11b2864740 1 Processor -- start 2024-09-10T09:46:00.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:00.554+0000 7f11b2864740 1 -- start start 2024-09-10T09:46:00.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:00.554+0000 7f11b2864740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561421bddc70 0x561421bde050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:46:00.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:00.554+0000 7f11b2864740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561421bde590 0x561421be8910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:46:00.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:00.554+0000 7f11b2864740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561421be8e50 0x561421beb240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:46:00.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:00.554+0000 7f11b2864740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x561421b443e0 con 0x561421bde590 2024-09-10T09:46:00.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:00.554+0000 7f11b2864740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x561421aebd80 con 0x561421bddc70 2024-09-10T09:46:00.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:00.554+0000 7f11b2864740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x561421b403a0 con 0x561421be8e50 2024-09-10T09:46:00.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:00.554+0000 7f11a7e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561421be8e50 0x561421beb240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:46:00.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:00.554+0000 7f11a7e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561421be8e50 0x561421beb240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:33998/0 (socket says 172.21.15.5:33998) 2024-09-10T09:46:00.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:00.554+0000 7f11a7e00640 1 -- 172.21.15.5:0/914338804 learned_addr learned my addr 172.21.15.5:0/914338804 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:46:00.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:00.554+0000 7f11ad200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561421bde590 0x561421be8910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:46:00.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:00.554+0000 7f11adc00640 1 --2- 172.21.15.5:0/914338804 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561421bddc70 0x561421bde050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:46:00.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:00.555+0000 7f11a7e00640 1 -- 172.21.15.5:0/914338804 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561421bddc70 msgr2=0x561421bde050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:46:00.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:00.555+0000 7f11a7e00640 1 --2- 172.21.15.5:0/914338804 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561421bddc70 0x561421bde050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:00.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:00.555+0000 7f11a7e00640 1 -- 172.21.15.5:0/914338804 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561421bde590 msgr2=0x561421be8910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:46:00.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:00.555+0000 7f11a7e00640 1 --2- 172.21.15.5:0/914338804 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561421bde590 0x561421be8910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:00.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:00.555+0000 7f11a7e00640 1 -- 172.21.15.5:0/914338804 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561421b0aae0 con 0x561421be8e50 2024-09-10T09:46:00.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:00.555+0000 7f11a5600640 1 --2- 172.21.15.5:0/961697775 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f11900a2b80 0x7f11900a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:46:00.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:00.555+0000 7f11a7e00640 1 --2- 172.21.15.5:0/914338804 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561421be8e50 0x561421beb240 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f11940f5c20 tx=0x7f11940f5c50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3cd19c7c57abb32c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:46:00.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:00.555+0000 7f11adc00640 1 --2- 172.21.15.5:0/961697775 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f11900a2b80 0x7f11900a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:46:00.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:00.555+0000 7f119a000640 1 -- 172.21.15.5:0/914338804 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11940b81e0 con 0x561421be8e50 2024-09-10T09:46:00.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:00.555+0000 7f11a5600640 1 -- 172.21.15.5:0/961697775 <== mon.2 v2:172.21.15.102:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f11940ef9c0 con 0x7f11a81007f0 2024-09-10T09:46:00.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:00.555+0000 7f119a000640 1 -- 172.21.15.5:0/914338804 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f11940bbf40 con 0x561421be8e50 2024-09-10T09:46:00.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:00.556+0000 7f119a000640 1 -- 172.21.15.5:0/914338804 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1194003800 con 0x561421be8e50 2024-09-10T09:46:00.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:00.556+0000 7f11b2864740 1 -- 172.21.15.5:0/914338804 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561421be8e50 msgr2=0x561421beb240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:46:00.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:00.556+0000 7f11b2864740 1 --2- 172.21.15.5:0/914338804 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561421be8e50 0x561421beb240 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f11940f5c20 tx=0x7f11940f5c50 comp rx=0 tx=0).stop 2024-09-10T09:46:00.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:00.556+0000 7f11b2864740 1 -- 172.21.15.5:0/914338804 shutdown_connections 2024-09-10T09:46:00.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:00.556+0000 7f11b2864740 1 --2- 172.21.15.5:0/914338804 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561421be8e50 0x561421beb240 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:00.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:00.556+0000 7f11b2864740 1 --2- 172.21.15.5:0/914338804 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561421bde590 0x561421be8910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:00.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:00.556+0000 7f11b2864740 1 --2- 172.21.15.5:0/914338804 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561421bddc70 0x561421bde050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:00.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:00.556+0000 7f11b2864740 1 -- 172.21.15.5:0/914338804 >> 172.21.15.5:0/914338804 conn(0x561421bda200 msgr2=0x561421bda5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:46:00.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:00.556+0000 7f11b2864740 1 -- 172.21.15.5:0/914338804 shutdown_connections 2024-09-10T09:46:00.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:00.557+0000 7f11b2864740 1 -- 172.21.15.5:0/914338804 wait complete. 2024-09-10T09:46:00.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:00.557+0000 7f11adc00640 1 --2- 172.21.15.5:0/961697775 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f11900a2b80 0x7f11900a5040 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f11a0005fd0 tx=0x7f11a0009600 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:46:00.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:00.557+0000 7f11b2864740 1 -- start start 2024-09-10T09:46:00.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:00.557+0000 7f11b2864740 1 Processor -- start 2024-09-10T09:46:00.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:00.558+0000 7f11b2864740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561421bddc70 0x561421be5c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:46:00.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:00.558+0000 7f11b2864740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561421bde590 0x561421be61b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:46:00.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:00.558+0000 7f11b2864740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561421be8e50 0x561421be66f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:46:00.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:00.558+0000 7f11b2864740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x561421beb7e0 con 0x561421bde590 2024-09-10T09:46:00.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:00.558+0000 7f11b2864740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x561421aff0f0 con 0x561421bddc70 2024-09-10T09:46:00.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:00.558+0000 7f11b2864740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x561421beb960 con 0x561421be8e50 2024-09-10T09:46:00.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:00.558+0000 7f11b2864740 10 client.0 _ll_register_callbacks cb 0x561421ae6bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:46:00.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:00.558+0000 7f11a7e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561421bde590 0x561421be61b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:46:00.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:00.558+0000 7f11adc00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561421be8e50 0x561421be66f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:46:00.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:00.558+0000 7f11ad200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561421bddc70 0x561421be5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:46:00.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:00.558+0000 7f11adc00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561421be8e50 0x561421be66f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:34008/0 (socket says 172.21.15.5:34008) 2024-09-10T09:46:00.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:00.558+0000 7f11adc00640 1 -- 172.21.15.5:0/1795803175 learned_addr learned my addr 172.21.15.5:0/1795803175 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:46:00.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:00.559+0000 7f11adc00640 1 -- 172.21.15.5:0/1795803175 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561421bddc70 msgr2=0x561421be5c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:46:00.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:00.559+0000 7f11adc00640 1 --2- 172.21.15.5:0/1795803175 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561421bddc70 0x561421be5c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:00.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:00.559+0000 7f11adc00640 1 -- 172.21.15.5:0/1795803175 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561421bde590 msgr2=0x561421be61b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:46:00.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:00.559+0000 7f11adc00640 1 --2- 172.21.15.5:0/1795803175 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561421bde590 0x561421be61b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:00.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:00.559+0000 7f11adc00640 1 -- 172.21.15.5:0/1795803175 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5614218b53c0 con 0x561421be8e50 2024-09-10T09:46:00.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:00.559+0000 7f11adc00640 1 --2- 172.21.15.5:0/1795803175 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561421be8e50 0x561421be66f0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f11a0009fd0 tx=0x7f11a0004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=77fb95f3ec76f7d9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:46:00.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:00.559+0000 7f118ea00640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-09-10T09:46:00.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:00.559+0000 7f118ea00640 1 -- 172.21.15.5:0/1795803175 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11a0004d20 con 0x561421be8e50 2024-09-10T09:46:00.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:00.559+0000 7f11b2864740 1 -- 172.21.15.5:0/1795803175 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f11940bbf40 con 0x561421be8e50 2024-09-10T09:46:00.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:00.559+0000 7f118ea00640 1 -- 172.21.15.5:0/1795803175 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f11a0005020 con 0x561421be8e50 2024-09-10T09:46:00.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:00.560+0000 7f118ea00640 1 -- 172.21.15.5:0/1795803175 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f11a0005260 con 0x561421be8e50 2024-09-10T09:46:00.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:00.560+0000 7f11b2864740 10 client.26917 fetch_fsmap learned FSMap version 34 2024-09-10T09:46:00.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:00.560+0000 7f11b2864740 1 -- 172.21.15.5:0/1795803175 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x561421bf77c0 con 0x561421be8e50 2024-09-10T09:46:00.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:00.560+0000 7f118ea00640 1 -- 172.21.15.5:0/1795803175 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f11940bbf40 con 0x561421be8e50 2024-09-10T09:46:00.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:00.560+0000 7f118ea00640 1 -- 172.21.15.5:0/1795803175 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f11a000f040 con 0x561421be8e50 2024-09-10T09:46:00.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:00.560+0000 7f11b2864740 10 client.26917 fetch_fsmap finished waiting for FSMap version 34 2024-09-10T09:46:00.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:00.560+0000 7f11b2864740 10 client.26917 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-09-10T09:46:00.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:00.560+0000 7f11b2864740 20 client.26917 populate_metadata read hostname 'smithi005' 2024-09-10T09:46:00.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:00.560+0000 7f11b2864740 1 --2- 172.21.15.5:0/1795803175 >> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] conn(0x561421bf8bd0 0x561421bfb030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:46:00.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:00.560+0000 7f11b2864740 4 client.26917 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:46:00.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:00.560+0000 7f11b2864740 1 -- 172.21.15.5:0/1795803175 --> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561421b51950 con 0x561421bf8bd0 2024-09-10T09:46:00.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:00.561+0000 7f11ad200640 1 --2- 172.21.15.5:0/1795803175 >> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] conn(0x561421bf8bd0 0x561421bfb030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:46:00.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:00.561+0000 7f11ad200640 1 --2- 172.21.15.5:0/1795803175 >> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] conn(0x561421bf8bd0 0x561421bfb030 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=a03f26b62313042c server_cookie=39cde87413794c1f in_seq=0 out_seq=0 2024-09-10T09:46:00.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:00.561+0000 7f118ea00640 10 client.26917 ms_handle_connect on v2:172.21.15.102:6836/2689776764 2024-09-10T09:46:00.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:00.562+0000 7f118ea00640 1 -- 172.21.15.5:0/1795803175 <== mds.3 v2:172.21.15.102:6836/2689776764 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561421b51950 con 0x561421bf8bd0 2024-09-10T09:46:00.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:00.562+0000 7f118ea00640 10 client.26917 handle_command_reply: tid=0 2024-09-10T09:46:00.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:00.562+0000 7f11b2864740 1 client.26917 shutdown 2024-09-10T09:46:00.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:00.562+0000 7f11b2864740 1 -- 172.21.15.5:0/1795803175 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561421be8e50 msgr2=0x561421be66f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:46:00.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:00.562+0000 7f11b2864740 1 --2- 172.21.15.5:0/1795803175 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561421be8e50 0x561421be66f0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f11a0009fd0 tx=0x7f11a0004a40 comp rx=0 tx=0).stop 2024-09-10T09:46:00.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:00.563+0000 7f11b2864740 1 -- 172.21.15.5:0/1795803175 shutdown_connections 2024-09-10T09:46:00.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:00.563+0000 7f11b2864740 1 --2- 172.21.15.5:0/1795803175 >> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] conn(0x561421bf8bd0 0x561421bfb030 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:00.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:00.563+0000 7f11b2864740 1 --2- 172.21.15.5:0/1795803175 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561421be8e50 0x561421be66f0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:00.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:00.563+0000 7f11b2864740 1 --2- 172.21.15.5:0/1795803175 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561421bde590 0x561421be61b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:00.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:00.563+0000 7f118ea00640 0 client.26917 ms_handle_reset on v2:172.21.15.102:6836/2689776764 2024-09-10T09:46:00.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:00.563+0000 7f11b2864740 1 --2- 172.21.15.5:0/1795803175 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561421bddc70 0x561421be5c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:00.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:00.563+0000 7f11b2864740 1 -- 172.21.15.5:0/1795803175 >> 172.21.15.5:0/1795803175 conn(0x561421bda200 msgr2=0x561421bdff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:46:00.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:00.563+0000 7f11b2864740 1 -- 172.21.15.5:0/1795803175 shutdown_connections 2024-09-10T09:46:00.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:00.563+0000 7f11b2864740 1 -- 172.21.15.5:0/1795803175 wait complete. 2024-09-10T09:46:00.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:00.563+0000 7f11b2864740 20 client.26917 trim_cache size 0 max 16384 2024-09-10T09:46:00.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:00.577+0000 7f11b2864740 1 Processor -- start 2024-09-10T09:46:00.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:00.578+0000 7f11b2864740 1 -- start start 2024-09-10T09:46:00.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:00.578+0000 7f11b2864740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561421bddc70 0x561421c00950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:46:00.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:00.578+0000 7f11b2864740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561421bde590 0x561421c00e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:46:00.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:00.578+0000 7f11b2864740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561421be8e50 0x561421be2ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:46:00.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:00.578+0000 7f11b2864740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f11a81b1090 con 0x561421bde590 2024-09-10T09:46:00.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:00.578+0000 7f11b2864740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f11a81b2bd0 con 0x561421be8e50 2024-09-10T09:46:00.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:00.578+0000 7f11b2864740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f11a81b1760 con 0x561421bddc70 2024-09-10T09:46:00.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:00.578+0000 7f11ad200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561421bddc70 0x561421c00950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:46:00.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:00.578+0000 7f11a7e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561421bde590 0x561421c00e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:46:00.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:00.578+0000 7f11adc00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561421be8e50 0x561421be2ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:46:00.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:00.578+0000 7f11ad200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561421bddc70 0x561421c00950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:34016/0 (socket says 172.21.15.5:34016) 2024-09-10T09:46:00.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:00.578+0000 7f11ad200640 1 -- 172.21.15.5:0/715284058 learned_addr learned my addr 172.21.15.5:0/715284058 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:46:00.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:00.579+0000 7f11ad200640 1 -- 172.21.15.5:0/715284058 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561421be8e50 msgr2=0x561421be2ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:46:00.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:00.579+0000 7f11ad200640 1 --2- 172.21.15.5:0/715284058 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561421be8e50 0x561421be2ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:00.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:00.579+0000 7f11ad200640 1 -- 172.21.15.5:0/715284058 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561421bde590 msgr2=0x561421c00e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:46:00.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:00.579+0000 7f11ad200640 1 --2- 172.21.15.5:0/715284058 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561421bde590 0x561421c00e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:00.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:00.579+0000 7f11ad200640 1 -- 172.21.15.5:0/715284058 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561421bda030 con 0x561421bddc70 2024-09-10T09:46:00.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:00.579+0000 7f11adc00640 1 --2- 172.21.15.5:0/715284058 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561421be8e50 0x561421be2ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:46:00.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:00.579+0000 7f11ad200640 1 --2- 172.21.15.5:0/715284058 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561421bddc70 0x561421c00950 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f119c009d10 tx=0x7f119c005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8430220b5757cb50 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:46:00.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:00.580+0000 7f1199600640 1 -- 172.21.15.5:0/715284058 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f119c022070 con 0x561421bddc70 2024-09-10T09:46:00.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:00.580+0000 7f1199600640 1 -- 172.21.15.5:0/715284058 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f119c00fd50 con 0x561421bddc70 2024-09-10T09:46:00.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:00.580+0000 7f1199600640 1 -- 172.21.15.5:0/715284058 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f119c01ed80 con 0x561421bddc70 2024-09-10T09:46:00.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:00.581+0000 7f11b2864740 1 -- 172.21.15.5:0/715284058 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561421bddc70 msgr2=0x561421c00950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:46:00.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:00.581+0000 7f11b2864740 1 --2- 172.21.15.5:0/715284058 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561421bddc70 0x561421c00950 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f119c009d10 tx=0x7f119c005f90 comp rx=0 tx=0).stop 2024-09-10T09:46:00.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:00.581+0000 7f11b2864740 1 -- 172.21.15.5:0/715284058 shutdown_connections 2024-09-10T09:46:00.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:00.581+0000 7f11b2864740 1 --2- 172.21.15.5:0/715284058 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561421be8e50 0x561421be2ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:00.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:00.581+0000 7f11b2864740 1 --2- 172.21.15.5:0/715284058 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561421bde590 0x561421c00e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:00.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:00.581+0000 7f11b2864740 1 --2- 172.21.15.5:0/715284058 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561421bddc70 0x561421c00950 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:00.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:00.581+0000 7f11b2864740 1 -- 172.21.15.5:0/715284058 >> 172.21.15.5:0/715284058 conn(0x561421bda200 msgr2=0x561421bd9380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:46:00.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:00.581+0000 7f11b2864740 1 -- 172.21.15.5:0/715284058 shutdown_connections 2024-09-10T09:46:00.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:00.581+0000 7f11b2864740 1 -- 172.21.15.5:0/715284058 wait complete. 2024-09-10T09:46:00.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:00.581+0000 7f11b2864740 1 -- start start 2024-09-10T09:46:00.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:00.582+0000 7f11b2864740 1 Processor -- start 2024-09-10T09:46:00.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:00.582+0000 7f11b2864740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561421bddc70 0x561421bf0da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:46:00.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:00.582+0000 7f11b2864740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561421bde590 0x561421bf12e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:46:00.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:00.582+0000 7f11b2864740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561421be8e50 0x561421bebb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:46:00.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:00.582+0000 7f11b2864740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f11a81b2500 con 0x561421bde590 2024-09-10T09:46:00.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:00.582+0000 7f11b2864740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f11a81b1e30 con 0x561421bddc70 2024-09-10T09:46:00.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:00.582+0000 7f11b2864740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f11a81af070 con 0x561421be8e50 2024-09-10T09:46:00.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:00.582+0000 7f11b2864740 10 client.0 _ll_register_callbacks cb 0x7f11a0009d70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:46:00.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:00.583+0000 7f11ad200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561421bddc70 0x561421bf0da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:46:00.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:00.583+0000 7f11ad200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561421bddc70 0x561421bf0da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:46298/0 (socket says 172.21.15.5:46298) 2024-09-10T09:46:00.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:00.583+0000 7f11ad200640 1 -- 172.21.15.5:0/3424914706 learned_addr learned my addr 172.21.15.5:0/3424914706 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:46:00.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:00.583+0000 7f11adc00640 1 --2- 172.21.15.5:0/3424914706 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561421be8e50 0x561421bebb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:46:00.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:00.583+0000 7f11a7e00640 1 --2- 172.21.15.5:0/3424914706 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561421bde590 0x561421bf12e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:46:00.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:00.583+0000 7f11ad200640 1 -- 172.21.15.5:0/3424914706 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561421be8e50 msgr2=0x561421bebb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:46:00.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:00.583+0000 7f11ad200640 1 --2- 172.21.15.5:0/3424914706 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561421be8e50 0x561421bebb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:00.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:00.583+0000 7f11ad200640 1 -- 172.21.15.5:0/3424914706 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561421bde590 msgr2=0x561421bf12e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:46:00.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:00.583+0000 7f11ad200640 1 --2- 172.21.15.5:0/3424914706 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561421bde590 0x561421bf12e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:00.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:00.583+0000 7f11ad200640 1 -- 172.21.15.5:0/3424914706 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561421bf4a40 con 0x561421bddc70 2024-09-10T09:46:00.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:00.584+0000 7f11a7e00640 1 --2- 172.21.15.5:0/3424914706 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561421bde590 0x561421bf12e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:46:00.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:00.584+0000 7f11ad200640 1 --2- 172.21.15.5:0/3424914706 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561421bddc70 0x561421bf0da0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f119c000c00 tx=0x7f119c00ffd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c1e38ff2fe744c9d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:46:00.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:00.584+0000 7f118ea00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-09-10T09:46:00.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:00.584+0000 7f118ea00640 1 -- 172.21.15.5:0/3424914706 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f119c022040 con 0x561421bddc70 2024-09-10T09:46:00.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:00.584+0000 7f118ea00640 1 -- 172.21.15.5:0/3424914706 <== mon.2 v2:172.21.15.102:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f119c01eb00 con 0x561421bddc70 2024-09-10T09:46:00.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:00.585+0000 7f11b2864740 1 -- 172.21.15.5:0/3424914706 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f119c00fd50 con 0x561421bddc70 2024-09-10T09:46:00.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:00.585+0000 7f118ea00640 1 -- 172.21.15.5:0/3424914706 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f119c01ed00 con 0x561421bddc70 2024-09-10T09:46:00.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:00.585+0000 7f11b2864740 10 client.26906 fetch_fsmap learned FSMap version 34 2024-09-10T09:46:00.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:00.585+0000 7f11b2864740 1 -- 172.21.15.5:0/3424914706 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x561421bdae90 con 0x561421bddc70 2024-09-10T09:46:00.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:00.586+0000 7f118ea00640 1 -- 172.21.15.5:0/3424914706 <== mon.2 v2:172.21.15.102:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f119c00fd50 con 0x561421bddc70 2024-09-10T09:46:00.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:00.586+0000 7f118ea00640 1 -- 172.21.15.5:0/3424914706 <== mon.2 v2:172.21.15.102:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f119c005300 con 0x561421bddc70 2024-09-10T09:46:00.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:00.586+0000 7f11b2864740 10 client.26906 fetch_fsmap finished waiting for FSMap version 34 2024-09-10T09:46:00.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:00.586+0000 7f11b2864740 10 client.26906 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-09-10T09:46:00.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:00.586+0000 7f11b2864740 20 client.26906 populate_metadata read hostname 'smithi005' 2024-09-10T09:46:00.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:00.586+0000 7f11b2864740 1 --2- 172.21.15.5:0/3424914706 >> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] conn(0x561421c645b0 0x561421c669a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:46:00.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:00.586+0000 7f11b2864740 4 client.26906 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T09:46:00.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:00.586+0000 7f11b2864740 1 -- 172.21.15.5:0/3424914706 --> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561421b4be60 con 0x561421c645b0 2024-09-10T09:46:00.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:00.586+0000 7f11a7e00640 1 --2- 172.21.15.5:0/3424914706 >> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] conn(0x561421c645b0 0x561421c669a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:46:00.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:00.587+0000 7f11a7e00640 1 --2- 172.21.15.5:0/3424914706 >> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] conn(0x561421c645b0 0x561421c669a0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=407e0e0b25a282e server_cookie=bcec8594a4bcab0d in_seq=0 out_seq=0 2024-09-10T09:46:00.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:00.587+0000 7f118ea00640 10 client.26906 ms_handle_connect on v2:172.21.15.102:6836/2689776764 2024-09-10T09:46:00.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:00.587+0000 7f118ea00640 1 -- 172.21.15.5:0/3424914706 <== mds.3 v2:172.21.15.102:6836/2689776764 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561421b4be60 con 0x561421c645b0 2024-09-10T09:46:00.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:00.587+0000 7f118ea00640 10 client.26906 handle_command_reply: tid=0 2024-09-10T09:46:00.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:00.587+0000 7f11b2864740 1 client.26906 shutdown 2024-09-10T09:46:00.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:00.587+0000 7f11b2864740 1 -- 172.21.15.5:0/3424914706 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561421bddc70 msgr2=0x561421bf0da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:46:00.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:00.587+0000 7f11b2864740 1 --2- 172.21.15.5:0/3424914706 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561421bddc70 0x561421bf0da0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f119c000c00 tx=0x7f119c00ffd0 comp rx=0 tx=0).stop 2024-09-10T09:46:00.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:00.587+0000 7f11b2864740 1 -- 172.21.15.5:0/3424914706 shutdown_connections 2024-09-10T09:46:00.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:00.587+0000 7f11b2864740 1 --2- 172.21.15.5:0/3424914706 >> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] conn(0x561421c645b0 0x561421c669a0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:00.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:00.587+0000 7f11b2864740 1 --2- 172.21.15.5:0/3424914706 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561421be8e50 0x561421bebb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:00.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:00.587+0000 7f11b2864740 1 --2- 172.21.15.5:0/3424914706 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561421bde590 0x561421bf12e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:00.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:00.587+0000 7f11b2864740 1 --2- 172.21.15.5:0/3424914706 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561421bddc70 0x561421bf0da0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:00.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:00.587+0000 7f118ea00640 0 client.26906 ms_handle_reset on v2:172.21.15.102:6836/2689776764 2024-09-10T09:46:00.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:00.587+0000 7f11b2864740 1 -- 172.21.15.5:0/3424914706 >> 172.21.15.5:0/3424914706 conn(0x561421bda200 msgr2=0x561421be5c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:46:00.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:00.587+0000 7f11b2864740 1 -- 172.21.15.5:0/3424914706 shutdown_connections 2024-09-10T09:46:00.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:00.587+0000 7f11b2864740 1 -- 172.21.15.5:0/3424914706 wait complete. 2024-09-10T09:46:00.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:00.587+0000 7f11b2864740 20 client.26906 trim_cache size 0 max 16384 2024-09-10T09:46:00.609 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-10T09:46:00.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:00.588+0000 7f11afc00640 1 -- 172.21.15.5:0/961697775 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f11900a2b80 msgr2=0x7f11900a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:46:00.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:00.588+0000 7f11afc00640 1 --2- 172.21.15.5:0/961697775 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f11900a2b80 0x7f11900a5040 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f11a0005fd0 tx=0x7f11a0009600 comp rx=0 tx=0).stop 2024-09-10T09:46:00.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:00.588+0000 7f11afc00640 1 -- 172.21.15.5:0/961697775 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f11a81007f0 msgr2=0x7f11a819ac70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:46:00.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:00.588+0000 7f11afc00640 1 --2- 172.21.15.5:0/961697775 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f11a81007f0 0x7f11a819ac70 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f1194006010 tx=0x7f11940042f0 comp rx=0 tx=0).stop 2024-09-10T09:46:00.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:00.588+0000 7f11afc00640 1 -- 172.21.15.5:0/961697775 shutdown_connections 2024-09-10T09:46:00.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:00.588+0000 7f11afc00640 1 --2- 172.21.15.5:0/961697775 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f11900a2b80 0x7f11900a5040 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:00.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:00.588+0000 7f11afc00640 1 --2- 172.21.15.5:0/961697775 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f11a810a2f0 0x7f11a819f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:00.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:00.588+0000 7f11afc00640 1 --2- 172.21.15.5:0/961697775 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f11a81007f0 0x7f11a819ac70 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:00.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:00.588+0000 7f11afc00640 1 --2- 172.21.15.5:0/961697775 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f11a80ffe40 0x7f11a819a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:00.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:00.588+0000 7f11afc00640 1 -- 172.21.15.5:0/961697775 >> 172.21.15.5:0/961697775 conn(0x7f11a80fba90 msgr2=0x7f11a8107f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:46:00.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:00.589+0000 7f11afc00640 1 -- 172.21.15.5:0/961697775 shutdown_connections 2024-09-10T09:46:00.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:00.589+0000 7f11afc00640 1 -- 172.21.15.5:0/961697775 wait complete. 2024-09-10T09:46:01.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:46:00 smithi005 ceph-mon[30350]: pgmap v623: 129 pgs: 129 active+clean; 179 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.0 MiB/s wr, 206 op/s 2024-09-10T09:46:01.062 INFO:tasks.workunit.client.0.smithi005.stdout: 1 440645 6.03 MB/sec execute 222 sec latency 34.511 ms 2024-09-10T09:46:01.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:46:00 smithi159 ceph-mon[35513]: pgmap v623: 129 pgs: 129 active+clean; 179 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.0 MiB/s wr, 206 op/s 2024-09-10T09:46:01.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:46:00 smithi102 ceph-mon[35442]: pgmap v623: 129 pgs: 129 active+clean; 179 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.0 MiB/s wr, 206 op/s 2024-09-10T09:46:01.506 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-10T09:46:01.919 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:46:02.062 INFO:tasks.workunit.client.0.smithi005.stdout: 1 443146 6.07 MB/sec execute 223 sec latency 28.688 ms 2024-09-10T09:46:02.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:02.610+0000 7eff9de00640 1 -- 172.21.15.5:0/326933380 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7eff981019d0 msgr2=0x7eff98101e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:46:02.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:02.610+0000 7eff9de00640 1 --2- 172.21.15.5:0/326933380 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7eff981019d0 0x7eff98101e30 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7eff84009930 tx=0x7eff8402a440 comp rx=0 tx=0).stop 2024-09-10T09:46:02.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:02.610+0000 7eff9de00640 1 -- 172.21.15.5:0/326933380 shutdown_connections 2024-09-10T09:46:02.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:02.610+0000 7eff9de00640 1 --2- 172.21.15.5:0/326933380 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7eff98102370 0x7eff9810e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:02.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:02.610+0000 7eff9de00640 1 --2- 172.21.15.5:0/326933380 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7eff981019d0 0x7eff98101e30 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:02.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:02.610+0000 7eff9de00640 1 --2- 172.21.15.5:0/326933380 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7eff981079d0 0x7eff98107db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:02.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:02.610+0000 7eff9de00640 1 -- 172.21.15.5:0/326933380 >> 172.21.15.5:0/326933380 conn(0x7eff980fd710 msgr2=0x7eff980ffb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:46:02.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:02.610+0000 7eff9de00640 1 -- 172.21.15.5:0/326933380 shutdown_connections 2024-09-10T09:46:02.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:02.610+0000 7eff9de00640 1 -- 172.21.15.5:0/326933380 wait complete. 2024-09-10T09:46:02.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:02.611+0000 7eff9de00640 1 Processor -- start 2024-09-10T09:46:02.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:02.611+0000 7eff9de00640 1 -- start start 2024-09-10T09:46:02.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:02.612+0000 7eff9de00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7eff981019d0 0x7eff980744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:46:02.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:02.612+0000 7eff9de00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7eff98102370 0x7eff98074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:46:02.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:02.612+0000 7eff9de00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7eff981079d0 0x7eff98074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:46:02.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:02.612+0000 7eff9de00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7eff98111e30 con 0x7eff981019d0 2024-09-10T09:46:02.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:02.612+0000 7eff9de00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7eff98111cb0 con 0x7eff981079d0 2024-09-10T09:46:02.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:02.612+0000 7eff9de00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7eff98111fb0 con 0x7eff98102370 2024-09-10T09:46:02.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:02.612+0000 7eff9ca00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7eff981019d0 0x7eff980744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:46:02.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:02.612+0000 7eff9ca00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7eff981019d0 0x7eff980744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38288/0 (socket says 172.21.15.5:38288) 2024-09-10T09:46:02.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:02.612+0000 7eff9ca00640 1 -- 172.21.15.5:0/805645697 learned_addr learned my addr 172.21.15.5:0/805645697 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:46:02.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:02.612+0000 7eff9d400640 1 --2- 172.21.15.5:0/805645697 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7eff981079d0 0x7eff98074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:46:02.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:02.613+0000 7eff9ca00640 1 -- 172.21.15.5:0/805645697 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7eff98102370 msgr2=0x7eff98074a30 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-10T09:46:02.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:02.613+0000 7eff9ca00640 1 --2- 172.21.15.5:0/805645697 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7eff98102370 0x7eff98074a30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:02.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:02.613+0000 7eff9ca00640 1 -- 172.21.15.5:0/805645697 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7eff981079d0 msgr2=0x7eff98074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:46:02.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:02.613+0000 7eff9ca00640 1 --2- 172.21.15.5:0/805645697 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7eff981079d0 0x7eff98074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:02.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:02.613+0000 7eff9ca00640 1 -- 172.21.15.5:0/805645697 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff981a6830 con 0x7eff981019d0 2024-09-10T09:46:02.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:02.613+0000 7eff9ca00640 1 --2- 172.21.15.5:0/805645697 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7eff981019d0 0x7eff980744f0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7eff8c00b450 tx=0x7eff8c00b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:46:02.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:02.613+0000 7eff95600640 1 -- 172.21.15.5:0/805645697 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff8c015020 con 0x7eff981019d0 2024-09-10T09:46:02.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:02.613+0000 7eff95600640 1 -- 172.21.15.5:0/805645697 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7eff8c011d70 con 0x7eff981019d0 2024-09-10T09:46:02.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:02.613+0000 7eff9de00640 1 -- 172.21.15.5:0/805645697 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff981a6b20 con 0x7eff981019d0 2024-09-10T09:46:02.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:02.614+0000 7eff95600640 1 -- 172.21.15.5:0/805645697 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff8c00f340 con 0x7eff981019d0 2024-09-10T09:46:02.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:02.614+0000 7eff9de00640 1 -- 172.21.15.5:0/805645697 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff981a70b0 con 0x7eff981019d0 2024-09-10T09:46:02.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:02.617+0000 7eff95600640 1 -- 172.21.15.5:0/805645697 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7eff8c018020 con 0x7eff981019d0 2024-09-10T09:46:02.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:02.617+0000 7effa2012740 1 Processor -- start 2024-09-10T09:46:02.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:02.617+0000 7effa2012740 1 -- start start 2024-09-10T09:46:02.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:02.617+0000 7effa2012740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a2fb2aee40 0x55a2fb2af220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:46:02.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:02.618+0000 7effa2012740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a2fb2af760 0x55a2fb2b9ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:46:02.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:02.618+0000 7effa2012740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a2fb2ba020 0x55a2fb2bc410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:46:02.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:02.618+0000 7effa2012740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55a2fb2153e0 con 0x55a2fb2aee40 2024-09-10T09:46:02.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:02.618+0000 7effa2012740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55a2fb1bcd80 con 0x55a2fb2ba020 2024-09-10T09:46:02.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:02.618+0000 7effa2012740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55a2fb2113a0 con 0x55a2fb2af760 2024-09-10T09:46:02.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:02.618+0000 7eff9d400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a2fb2aee40 0x55a2fb2af220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:46:02.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:02.618+0000 7eff9d400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a2fb2aee40 0x55a2fb2af220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38302/0 (socket says 172.21.15.5:38302) 2024-09-10T09:46:02.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:02.618+0000 7eff9d400640 1 -- 172.21.15.5:0/3412027156 learned_addr learned my addr 172.21.15.5:0/3412027156 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:46:02.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:02.618+0000 7eff9ca00640 1 --2- 172.21.15.5:0/3412027156 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a2fb2af760 0x55a2fb2b9ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:46:02.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:02.618+0000 7eff95600640 1 --2- 172.21.15.5:0/805645697 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7eff800a2c00 0x7eff800a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:46:02.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:02.619+0000 7eff9ca00640 1 -- 172.21.15.5:0/3412027156 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a2fb2ba020 msgr2=0x55a2fb2bc410 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T09:46:02.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:02.619+0000 7eff9ca00640 1 --2- 172.21.15.5:0/3412027156 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a2fb2ba020 0x55a2fb2bc410 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:02.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:02.619+0000 7eff9ca00640 1 -- 172.21.15.5:0/3412027156 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a2fb2aee40 msgr2=0x55a2fb2af220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:46:02.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:02.619+0000 7eff9ca00640 1 --2- 172.21.15.5:0/3412027156 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a2fb2aee40 0x55a2fb2af220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:02.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:02.619+0000 7eff95600640 1 -- 172.21.15.5:0/805645697 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7eff8c0ceed0 con 0x7eff981019d0 2024-09-10T09:46:02.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:02.619+0000 7eff9ca00640 1 -- 172.21.15.5:0/3412027156 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a2fb1dbae0 con 0x55a2fb2af760 2024-09-10T09:46:02.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:02.619+0000 7eff9ca00640 1 --2- 172.21.15.5:0/3412027156 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a2fb2af760 0x55a2fb2b9ae0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7eff8c0cc520 tx=0x7eff8c0107f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=abadbe1d088488ac server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:46:02.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:02.619+0000 7eff9d400640 1 --2- 172.21.15.5:0/805645697 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7eff800a2c00 0x7eff800a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:46:02.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:02.619+0000 7eff89c00640 1 -- 172.21.15.5:0/3412027156 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff8c09b0b0 con 0x55a2fb2af760 2024-09-10T09:46:02.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:02.620+0000 7eff89c00640 1 -- 172.21.15.5:0/3412027156 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7eff8c095e50 con 0x55a2fb2af760 2024-09-10T09:46:02.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:02.620+0000 7eff89c00640 1 -- 172.21.15.5:0/3412027156 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff8c09ac40 con 0x55a2fb2af760 2024-09-10T09:46:02.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:02.620+0000 7effa2012740 1 -- 172.21.15.5:0/3412027156 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a2fb2af760 msgr2=0x55a2fb2b9ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:46:02.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:02.620+0000 7eff9d400640 1 --2- 172.21.15.5:0/805645697 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7eff800a2c00 0x7eff800a50c0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7eff90009fd0 tx=0x7eff90004a40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:46:02.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:02.620+0000 7effa2012740 1 --2- 172.21.15.5:0/3412027156 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a2fb2af760 0x55a2fb2b9ae0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7eff8c0cc520 tx=0x7eff8c0107f0 comp rx=0 tx=0).stop 2024-09-10T09:46:02.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:02.621+0000 7effa2012740 1 -- 172.21.15.5:0/3412027156 shutdown_connections 2024-09-10T09:46:02.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:02.621+0000 7effa2012740 1 --2- 172.21.15.5:0/3412027156 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a2fb2ba020 0x55a2fb2bc410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:02.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:02.621+0000 7effa2012740 1 --2- 172.21.15.5:0/3412027156 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a2fb2af760 0x55a2fb2b9ae0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:02.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:02.621+0000 7effa2012740 1 --2- 172.21.15.5:0/3412027156 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a2fb2aee40 0x55a2fb2af220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:02.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:02.621+0000 7effa2012740 1 -- 172.21.15.5:0/3412027156 >> 172.21.15.5:0/3412027156 conn(0x55a2fb2ab3d0 msgr2=0x55a2fb2ab7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:46:02.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:02.621+0000 7effa2012740 1 -- 172.21.15.5:0/3412027156 shutdown_connections 2024-09-10T09:46:02.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:02.621+0000 7effa2012740 1 -- 172.21.15.5:0/3412027156 wait complete. 2024-09-10T09:46:02.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:02.621+0000 7effa2012740 1 -- start start 2024-09-10T09:46:02.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:02.621+0000 7effa2012740 1 Processor -- start 2024-09-10T09:46:02.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:02.622+0000 7effa2012740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a2fb2aee40 0x55a2fb2b6e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:46:02.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:02.622+0000 7effa2012740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a2fb2af760 0x55a2fb2b73a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:46:02.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:02.622+0000 7effa2012740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a2fb2ba020 0x55a2fb2b78e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:46:02.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:02.622+0000 7effa2012740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55a2fb2bcbf0 con 0x55a2fb2aee40 2024-09-10T09:46:02.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:02.622+0000 7effa2012740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55a2fb1d00f0 con 0x55a2fb2af760 2024-09-10T09:46:02.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:02.622+0000 7effa2012740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55a2fb2bcd70 con 0x55a2fb2ba020 2024-09-10T09:46:02.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:02.622+0000 7effa2012740 10 client.0 _ll_register_callbacks cb 0x55a2fb1b7bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:46:02.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:02.622+0000 7eff97e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a2fb2af760 0x55a2fb2b73a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:46:02.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:02.622+0000 7eff97e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a2fb2af760 0x55a2fb2b73a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:36750/0 (socket says 172.21.15.5:36750) 2024-09-10T09:46:02.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:02.622+0000 7eff97e00640 1 -- 172.21.15.5:0/2702276696 learned_addr learned my addr 172.21.15.5:0/2702276696 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:46:02.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:02.623+0000 7eff9ca00640 1 --2- 172.21.15.5:0/2702276696 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a2fb2aee40 0x55a2fb2b6e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:46:02.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:02.623+0000 7eff9d400640 1 --2- 172.21.15.5:0/2702276696 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a2fb2ba020 0x55a2fb2b78e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:46:02.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:02.623+0000 7eff9ca00640 1 -- 172.21.15.5:0/2702276696 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a2fb2ba020 msgr2=0x55a2fb2b78e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:46:02.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:02.623+0000 7eff9ca00640 1 --2- 172.21.15.5:0/2702276696 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a2fb2ba020 0x55a2fb2b78e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:02.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:02.623+0000 7eff9ca00640 1 -- 172.21.15.5:0/2702276696 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a2fb2af760 msgr2=0x55a2fb2b73a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:46:02.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:02.623+0000 7eff9ca00640 1 --2- 172.21.15.5:0/2702276696 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a2fb2af760 0x55a2fb2b73a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:02.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:02.623+0000 7eff9ca00640 1 -- 172.21.15.5:0/2702276696 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a2faf863c0 con 0x55a2fb2aee40 2024-09-10T09:46:02.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:02.624+0000 7eff9ca00640 1 --2- 172.21.15.5:0/2702276696 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a2fb2aee40 0x55a2fb2b6e60 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x55a2fb2ac230 tx=0x7eff8c0d2750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a01d6f055874a7a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:46:02.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:02.624+0000 7eff7e000640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-10T09:46:02.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:02.624+0000 7eff7e000640 1 -- 172.21.15.5:0/2702276696 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff8c09ae20 con 0x55a2fb2aee40 2024-09-10T09:46:02.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:02.624+0000 7eff7e000640 1 -- 172.21.15.5:0/2702276696 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7eff8c0bb110 con 0x55a2fb2aee40 2024-09-10T09:46:02.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:02.624+0000 7effa2012740 1 -- 172.21.15.5:0/2702276696 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7eff8c095e50 con 0x55a2fb2aee40 2024-09-10T09:46:02.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:02.624+0000 7eff7e000640 1 -- 172.21.15.5:0/2702276696 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7eff8c09b0b0 con 0x55a2fb2aee40 2024-09-10T09:46:02.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:02.624+0000 7effa2012740 10 client.16908 fetch_fsmap learned FSMap version 34 2024-09-10T09:46:02.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:02.624+0000 7effa2012740 1 -- 172.21.15.5:0/2702276696 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55a2fb2c8bb0 con 0x55a2fb2aee40 2024-09-10T09:46:02.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:02.624+0000 7eff7e000640 1 -- 172.21.15.5:0/2702276696 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7eff8c095e50 con 0x55a2fb2aee40 2024-09-10T09:46:02.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:02.625+0000 7eff7e000640 1 -- 172.21.15.5:0/2702276696 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7eff8c09f170 con 0x55a2fb2aee40 2024-09-10T09:46:02.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:02.625+0000 7effa2012740 10 client.16908 fetch_fsmap finished waiting for FSMap version 34 2024-09-10T09:46:02.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:02.625+0000 7effa2012740 10 client.16908 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-10T09:46:02.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:02.625+0000 7effa2012740 20 client.16908 populate_metadata read hostname 'smithi005' 2024-09-10T09:46:02.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:02.625+0000 7effa2012740 1 --2- 172.21.15.5:0/2702276696 >> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] conn(0x55a2fb2c9f00 0x55a2fb2cc2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:46:02.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:02.625+0000 7effa2012740 4 client.16908 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:46:02.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:02.625+0000 7effa2012740 1 -- 172.21.15.5:0/2702276696 --> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a2fb222950 con 0x55a2fb2c9f00 2024-09-10T09:46:02.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:02.626+0000 7eff97e00640 1 --2- 172.21.15.5:0/2702276696 >> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] conn(0x55a2fb2c9f00 0x55a2fb2cc2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:46:02.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:02.626+0000 7eff97e00640 1 --2- 172.21.15.5:0/2702276696 >> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] conn(0x55a2fb2c9f00 0x55a2fb2cc2f0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=19c83c4b6abc6c6c server_cookie=6621301a8137282a in_seq=0 out_seq=0 2024-09-10T09:46:02.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:02.626+0000 7eff7e000640 10 client.16908 ms_handle_connect on v2:172.21.15.159:6836/620627639 2024-09-10T09:46:02.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:02.627+0000 7eff7e000640 1 -- 172.21.15.5:0/2702276696 <== mds.2 v2:172.21.15.159:6836/620627639 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a2fb222950 con 0x55a2fb2c9f00 2024-09-10T09:46:02.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:02.627+0000 7eff7e000640 10 client.16908 handle_command_reply: tid=0 2024-09-10T09:46:02.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:02.627+0000 7effa2012740 1 client.16908 shutdown 2024-09-10T09:46:02.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:02.627+0000 7effa2012740 1 -- 172.21.15.5:0/2702276696 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a2fb2aee40 msgr2=0x55a2fb2b6e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:46:02.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:02.627+0000 7effa2012740 1 --2- 172.21.15.5:0/2702276696 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a2fb2aee40 0x55a2fb2b6e60 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x55a2fb2ac230 tx=0x7eff8c0d2750 comp rx=0 tx=0).stop 2024-09-10T09:46:02.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:02.627+0000 7effa2012740 1 -- 172.21.15.5:0/2702276696 shutdown_connections 2024-09-10T09:46:02.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:02.627+0000 7effa2012740 1 --2- 172.21.15.5:0/2702276696 >> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] conn(0x55a2fb2c9f00 0x55a2fb2cc2f0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:02.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:02.627+0000 7effa2012740 1 --2- 172.21.15.5:0/2702276696 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a2fb2ba020 0x55a2fb2b78e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:02.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:02.627+0000 7effa2012740 1 --2- 172.21.15.5:0/2702276696 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a2fb2af760 0x55a2fb2b73a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:02.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:02.627+0000 7effa2012740 1 --2- 172.21.15.5:0/2702276696 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a2fb2aee40 0x55a2fb2b6e60 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:02.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:02.627+0000 7effa2012740 1 -- 172.21.15.5:0/2702276696 >> 172.21.15.5:0/2702276696 conn(0x55a2fb2ab3d0 msgr2=0x55a2fb2b1100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:46:02.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:02.627+0000 7eff7e000640 0 client.16908 ms_handle_reset on v2:172.21.15.159:6836/620627639 2024-09-10T09:46:02.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:02.628+0000 7effa2012740 1 -- 172.21.15.5:0/2702276696 shutdown_connections 2024-09-10T09:46:02.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:02.628+0000 7effa2012740 1 -- 172.21.15.5:0/2702276696 wait complete. 2024-09-10T09:46:02.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:02.628+0000 7effa2012740 20 client.16908 trim_cache size 0 max 16384 2024-09-10T09:46:02.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:02.641+0000 7effa2012740 1 Processor -- start 2024-09-10T09:46:02.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:02.642+0000 7effa2012740 1 -- start start 2024-09-10T09:46:02.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:02.642+0000 7effa2012740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a2fb2aee40 0x55a2fb2b18a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:46:02.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:02.642+0000 7effa2012740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a2fb2af760 0x55a2fb2b40c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:46:02.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:02.642+0000 7effa2012740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a2fb2ba020 0x55a2fb2b4600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:46:02.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:02.642+0000 7effa2012740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7eff981b1080 con 0x55a2fb2af760 2024-09-10T09:46:02.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:02.642+0000 7effa2012740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7eff981b2bc0 con 0x55a2fb2aee40 2024-09-10T09:46:02.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:02.642+0000 7effa2012740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7eff981b1750 con 0x55a2fb2ba020 2024-09-10T09:46:02.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:02.642+0000 7eff97e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a2fb2af760 0x55a2fb2b40c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:46:02.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:02.642+0000 7eff9ca00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a2fb2aee40 0x55a2fb2b18a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:46:02.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:02.642+0000 7eff97e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a2fb2af760 0x55a2fb2b40c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38316/0 (socket says 172.21.15.5:38316) 2024-09-10T09:46:02.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:02.642+0000 7eff97e00640 1 -- 172.21.15.5:0/3120084407 learned_addr learned my addr 172.21.15.5:0/3120084407 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:46:02.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:02.642+0000 7eff9d400640 1 --2- 172.21.15.5:0/3120084407 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a2fb2ba020 0x55a2fb2b4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:46:02.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:02.643+0000 7eff9ca00640 1 -- 172.21.15.5:0/3120084407 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a2fb2ba020 msgr2=0x55a2fb2b4600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:46:02.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:02.643+0000 7eff9ca00640 1 --2- 172.21.15.5:0/3120084407 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a2fb2ba020 0x55a2fb2b4600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:02.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:02.643+0000 7eff9ca00640 1 -- 172.21.15.5:0/3120084407 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a2fb2af760 msgr2=0x55a2fb2b40c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:46:02.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:02.643+0000 7eff9ca00640 1 --2- 172.21.15.5:0/3120084407 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a2fb2af760 0x55a2fb2b40c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:02.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:02.643+0000 7eff9ca00640 1 -- 172.21.15.5:0/3120084407 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a2fb2ab200 con 0x55a2fb2aee40 2024-09-10T09:46:02.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:02.643+0000 7eff9ca00640 1 --2- 172.21.15.5:0/3120084407 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a2fb2aee40 0x55a2fb2b18a0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7eff8c097c60 tx=0x7eff8c0107f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2c681439186fb453 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:46:02.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:02.643+0000 7eff89200640 1 -- 172.21.15.5:0/3120084407 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff8c09ac40 con 0x55a2fb2aee40 2024-09-10T09:46:02.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:02.643+0000 7eff89200640 1 -- 172.21.15.5:0/3120084407 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7eff8c095e50 con 0x55a2fb2aee40 2024-09-10T09:46:02.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:02.643+0000 7eff89200640 1 -- 172.21.15.5:0/3120084407 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff8c09f170 con 0x55a2fb2aee40 2024-09-10T09:46:02.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:02.644+0000 7effa2012740 1 -- 172.21.15.5:0/3120084407 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a2fb2aee40 msgr2=0x55a2fb2b18a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:46:02.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:02.644+0000 7effa2012740 1 --2- 172.21.15.5:0/3120084407 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a2fb2aee40 0x55a2fb2b18a0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7eff8c097c60 tx=0x7eff8c0107f0 comp rx=0 tx=0).stop 2024-09-10T09:46:02.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:02.644+0000 7effa2012740 1 -- 172.21.15.5:0/3120084407 shutdown_connections 2024-09-10T09:46:02.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:02.644+0000 7effa2012740 1 --2- 172.21.15.5:0/3120084407 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a2fb2ba020 0x55a2fb2b4600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:02.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:02.644+0000 7effa2012740 1 --2- 172.21.15.5:0/3120084407 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a2fb2af760 0x55a2fb2b40c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:02.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:02.644+0000 7effa2012740 1 --2- 172.21.15.5:0/3120084407 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a2fb2aee40 0x55a2fb2b18a0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:02.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:02.644+0000 7effa2012740 1 -- 172.21.15.5:0/3120084407 >> 172.21.15.5:0/3120084407 conn(0x55a2fb2ab3d0 msgr2=0x55a2fb2b8ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:46:02.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:02.644+0000 7effa2012740 1 -- 172.21.15.5:0/3120084407 shutdown_connections 2024-09-10T09:46:02.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:02.644+0000 7effa2012740 1 -- 172.21.15.5:0/3120084407 wait complete. 2024-09-10T09:46:02.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:02.644+0000 7effa2012740 1 -- start start 2024-09-10T09:46:02.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:02.645+0000 7effa2012740 1 Processor -- start 2024-09-10T09:46:02.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:02.645+0000 7effa2012740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a2fb2aee40 0x55a2fb2cf810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:46:02.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:02.645+0000 7effa2012740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a2fb2af760 0x55a2fb2ccb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:46:02.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:02.645+0000 7effa2012740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a2fb2ba020 0x55a2fb2cd2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:46:02.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:02.645+0000 7effa2012740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7eff981b24f0 con 0x55a2fb2ba020 2024-09-10T09:46:02.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:02.645+0000 7effa2012740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7eff981b1e20 con 0x55a2fb2af760 2024-09-10T09:46:02.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:02.645+0000 7effa2012740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7eff981af060 con 0x55a2fb2aee40 2024-09-10T09:46:02.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:02.645+0000 7effa2012740 10 client.0 _ll_register_callbacks cb 0x7eff90009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:46:02.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:02.645+0000 7eff9ca00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a2fb2aee40 0x55a2fb2cf810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:46:02.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:02.646+0000 7eff97e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a2fb2af760 0x55a2fb2ccb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:46:02.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:02.646+0000 7eff9ca00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a2fb2aee40 0x55a2fb2cf810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:33946/0 (socket says 172.21.15.5:33946) 2024-09-10T09:46:02.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:02.646+0000 7eff9ca00640 1 -- 172.21.15.5:0/200365073 learned_addr learned my addr 172.21.15.5:0/200365073 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:46:02.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:02.646+0000 7eff9d400640 1 --2- 172.21.15.5:0/200365073 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a2fb2ba020 0x55a2fb2cd2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:46:02.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:02.646+0000 7eff9ca00640 1 -- 172.21.15.5:0/200365073 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a2fb2af760 msgr2=0x55a2fb2ccb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:46:02.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:02.646+0000 7eff9ca00640 1 --2- 172.21.15.5:0/200365073 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a2fb2af760 0x55a2fb2ccb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:02.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:02.646+0000 7eff9ca00640 1 -- 172.21.15.5:0/200365073 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a2fb2ba020 msgr2=0x55a2fb2cd2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:46:02.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:02.646+0000 7eff9ca00640 1 --2- 172.21.15.5:0/200365073 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a2fb2ba020 0x55a2fb2cd2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:02.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:02.646+0000 7eff9ca00640 1 -- 172.21.15.5:0/200365073 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a2fb2aac80 con 0x55a2fb2aee40 2024-09-10T09:46:02.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:02.646+0000 7eff9ca00640 1 --2- 172.21.15.5:0/200365073 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a2fb2aee40 0x55a2fb2cf810 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7eff8c010780 tx=0x7eff8c009310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6d9e5681360abcf9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:46:02.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:02.646+0000 7eff7e000640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-09-10T09:46:02.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:02.646+0000 7eff7e000640 1 -- 172.21.15.5:0/200365073 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff8c09ac40 con 0x55a2fb2aee40 2024-09-10T09:46:02.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:02.646+0000 7eff7e000640 1 -- 172.21.15.5:0/200365073 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7eff8c008ce0 con 0x55a2fb2aee40 2024-09-10T09:46:02.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:02.646+0000 7effa2012740 1 -- 172.21.15.5:0/200365073 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7eff8c095e50 con 0x55a2fb2aee40 2024-09-10T09:46:02.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:02.647+0000 7eff7e000640 1 -- 172.21.15.5:0/200365073 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7eff8c09b2e0 con 0x55a2fb2aee40 2024-09-10T09:46:02.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:02.647+0000 7effa2012740 10 client.26938 fetch_fsmap learned FSMap version 34 2024-09-10T09:46:02.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:02.647+0000 7effa2012740 1 -- 172.21.15.5:0/200365073 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55a2fb2aac80 con 0x55a2fb2aee40 2024-09-10T09:46:02.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:02.647+0000 7eff7e000640 1 -- 172.21.15.5:0/200365073 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7eff8c095e50 con 0x55a2fb2aee40 2024-09-10T09:46:02.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:02.647+0000 7eff7e000640 1 -- 172.21.15.5:0/200365073 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7eff8c09a7c0 con 0x55a2fb2aee40 2024-09-10T09:46:02.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:02.647+0000 7effa2012740 10 client.26938 fetch_fsmap finished waiting for FSMap version 34 2024-09-10T09:46:02.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:02.647+0000 7effa2012740 10 client.26938 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-10T09:46:02.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:02.647+0000 7effa2012740 20 client.26938 populate_metadata read hostname 'smithi005' 2024-09-10T09:46:02.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:02.647+0000 7effa2012740 1 --2- 172.21.15.5:0/200365073 >> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] conn(0x55a2fb334aa0 0x55a2fb336e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:46:02.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:02.647+0000 7effa2012740 4 client.26938 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T09:46:02.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:02.647+0000 7effa2012740 1 -- 172.21.15.5:0/200365073 --> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a2fb21ce60 con 0x55a2fb334aa0 2024-09-10T09:46:02.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:02.647+0000 7eff97e00640 1 --2- 172.21.15.5:0/200365073 >> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] conn(0x55a2fb334aa0 0x55a2fb336e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:46:02.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:02.648+0000 7eff97e00640 1 --2- 172.21.15.5:0/200365073 >> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] conn(0x55a2fb334aa0 0x55a2fb336e90 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d8214007fd64087f server_cookie=76357666070c2cfb in_seq=0 out_seq=0 2024-09-10T09:46:02.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:02.648+0000 7eff7e000640 10 client.26938 ms_handle_connect on v2:172.21.15.159:6836/620627639 2024-09-10T09:46:02.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:02.648+0000 7eff7e000640 1 -- 172.21.15.5:0/200365073 <== mds.2 v2:172.21.15.159:6836/620627639 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a2fb21ce60 con 0x55a2fb334aa0 2024-09-10T09:46:02.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:02.648+0000 7eff7e000640 10 client.26938 handle_command_reply: tid=0 2024-09-10T09:46:02.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:02.648+0000 7effa2012740 1 client.26938 shutdown 2024-09-10T09:46:02.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:02.648+0000 7effa2012740 1 -- 172.21.15.5:0/200365073 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a2fb2aee40 msgr2=0x55a2fb2cf810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:46:02.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:02.648+0000 7effa2012740 1 --2- 172.21.15.5:0/200365073 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a2fb2aee40 0x55a2fb2cf810 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7eff8c010780 tx=0x7eff8c009310 comp rx=0 tx=0).stop 2024-09-10T09:46:02.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:02.648+0000 7effa2012740 1 -- 172.21.15.5:0/200365073 shutdown_connections 2024-09-10T09:46:02.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:02.648+0000 7effa2012740 1 --2- 172.21.15.5:0/200365073 >> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] conn(0x55a2fb334aa0 0x55a2fb336e90 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:02.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:02.648+0000 7effa2012740 1 --2- 172.21.15.5:0/200365073 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a2fb2ba020 0x55a2fb2cd2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:02.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:02.648+0000 7effa2012740 1 --2- 172.21.15.5:0/200365073 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a2fb2af760 0x55a2fb2ccb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:02.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:02.648+0000 7effa2012740 1 --2- 172.21.15.5:0/200365073 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a2fb2aee40 0x55a2fb2cf810 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:02.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:02.648+0000 7eff7e000640 0 client.26938 ms_handle_reset on v2:172.21.15.159:6836/620627639 2024-09-10T09:46:02.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:02.648+0000 7effa2012740 1 -- 172.21.15.5:0/200365073 >> 172.21.15.5:0/200365073 conn(0x55a2fb2ab3d0 msgr2=0x55a2fb2c4ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:46:02.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:02.649+0000 7effa2012740 1 -- 172.21.15.5:0/200365073 shutdown_connections 2024-09-10T09:46:02.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:02.649+0000 7effa2012740 1 -- 172.21.15.5:0/200365073 wait complete. 2024-09-10T09:46:02.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:02.649+0000 7effa2012740 20 client.26938 trim_cache size 0 max 16384 2024-09-10T09:46:02.654 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-10T09:46:02.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:02.649+0000 7eff9de00640 1 -- 172.21.15.5:0/805645697 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7eff800a2c00 msgr2=0x7eff800a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:46:02.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:02.649+0000 7eff9de00640 1 --2- 172.21.15.5:0/805645697 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7eff800a2c00 0x7eff800a50c0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7eff90009fd0 tx=0x7eff90004a40 comp rx=0 tx=0).stop 2024-09-10T09:46:02.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:02.649+0000 7eff9de00640 1 -- 172.21.15.5:0/805645697 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7eff981019d0 msgr2=0x7eff980744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:46:02.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:02.649+0000 7eff9de00640 1 --2- 172.21.15.5:0/805645697 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7eff981019d0 0x7eff980744f0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7eff8c00b450 tx=0x7eff8c00b910 comp rx=0 tx=0).stop 2024-09-10T09:46:02.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:02.649+0000 7eff9de00640 1 -- 172.21.15.5:0/805645697 shutdown_connections 2024-09-10T09:46:02.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:02.649+0000 7eff9de00640 1 --2- 172.21.15.5:0/805645697 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7eff800a2c00 0x7eff800a50c0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:02.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:02.649+0000 7eff9de00640 1 --2- 172.21.15.5:0/805645697 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7eff981079d0 0x7eff98074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:02.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:02.649+0000 7eff9de00640 1 --2- 172.21.15.5:0/805645697 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7eff98102370 0x7eff98074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:02.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:02.649+0000 7eff9de00640 1 --2- 172.21.15.5:0/805645697 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7eff981019d0 0x7eff980744f0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:02.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:02.649+0000 7eff9de00640 1 -- 172.21.15.5:0/805645697 >> 172.21.15.5:0/805645697 conn(0x7eff980fd710 msgr2=0x7eff980fdf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:46:02.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:02.650+0000 7eff9de00640 1 -- 172.21.15.5:0/805645697 shutdown_connections 2024-09-10T09:46:02.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:02.650+0000 7eff9de00640 1 -- 172.21.15.5:0/805645697 wait complete. 2024-09-10T09:46:02.859 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:46:02 smithi005 ceph-mon[30350]: pgmap v624: 129 pgs: 129 active+clean; 179 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.1 MiB/s wr, 201 op/s 2024-09-10T09:46:03.062 INFO:tasks.workunit.client.0.smithi005.stdout: 1 443225 6.04 MB/sec execute 224 sec latency 935.548 ms 2024-09-10T09:46:03.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:46:02 smithi159 ceph-mon[35513]: pgmap v624: 129 pgs: 129 active+clean; 179 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.1 MiB/s wr, 201 op/s 2024-09-10T09:46:03.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:46:02 smithi102 ceph-mon[35442]: pgmap v624: 129 pgs: 129 active+clean; 179 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.1 MiB/s wr, 201 op/s 2024-09-10T09:46:03.820 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-10T09:46:04.062 INFO:tasks.workunit.client.0.smithi005.stdout: 1 443823 6.03 MB/sec execute 225 sec latency 1564.840 ms 2024-09-10T09:46:04.213 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:46:04.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:04.832+0000 7fb93e800640 1 -- 172.21.15.5:0/3032377902 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb938074ee0 msgr2=0x7fb93810dd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:46:04.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:04.832+0000 7fb93e800640 1 --2- 172.21.15.5:0/3032377902 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb938074ee0 0x7fb93810dd20 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fb924009900 tx=0x7fb92402a480 comp rx=0 tx=0).stop 2024-09-10T09:46:04.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:04.833+0000 7fb93e800640 1 -- 172.21.15.5:0/3032377902 shutdown_connections 2024-09-10T09:46:04.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:04.833+0000 7fb93e800640 1 --2- 172.21.15.5:0/3032377902 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb93810e260 0x7fb938110650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:04.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:04.833+0000 7fb93e800640 1 --2- 172.21.15.5:0/3032377902 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb938074ee0 0x7fb93810dd20 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:04.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:04.833+0000 7fb93e800640 1 --2- 172.21.15.5:0/3032377902 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb9380745c0 0x7fb9380749a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:04.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:04.833+0000 7fb93e800640 1 -- 172.21.15.5:0/3032377902 >> 172.21.15.5:0/3032377902 conn(0x7fb9380fd740 msgr2=0x7fb9380ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:46:04.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:04.833+0000 7fb93e800640 1 -- 172.21.15.5:0/3032377902 shutdown_connections 2024-09-10T09:46:04.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:04.833+0000 7fb93e800640 1 -- 172.21.15.5:0/3032377902 wait complete. 2024-09-10T09:46:04.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:04.834+0000 7fb93e800640 1 Processor -- start 2024-09-10T09:46:04.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:04.834+0000 7fb93e800640 1 -- start start 2024-09-10T09:46:04.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:04.834+0000 7fb93e800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb9380745c0 0x7fb93819a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:46:04.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:04.834+0000 7fb93e800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb938074ee0 0x7fb93819aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:46:04.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:04.834+0000 7fb93e800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb93810e260 0x7fb93819edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:46:04.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:04.834+0000 7fb93e800640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fb938114000 con 0x7fb93810e260 2024-09-10T09:46:04.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:04.834+0000 7fb93e800640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fb938113e80 con 0x7fb938074ee0 2024-09-10T09:46:04.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:04.834+0000 7fb93e800640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fb938114180 con 0x7fb9380745c0 2024-09-10T09:46:04.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:04.835+0000 7fb93d400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb9380745c0 0x7fb93819a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:46:04.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:04.835+0000 7fb93ca00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb938074ee0 0x7fb93819aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:46:04.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:04.835+0000 7fb93d400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb9380745c0 0x7fb93819a4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:33956/0 (socket says 172.21.15.5:33956) 2024-09-10T09:46:04.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:04.835+0000 7fb93d400640 1 -- 172.21.15.5:0/2366002289 learned_addr learned my addr 172.21.15.5:0/2366002289 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:46:04.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:04.835+0000 7fb93de00640 1 --2- 172.21.15.5:0/2366002289 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb93810e260 0x7fb93819edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:46:04.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:04.835+0000 7fb93ca00640 1 -- 172.21.15.5:0/2366002289 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb9380745c0 msgr2=0x7fb93819a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:46:04.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:04.835+0000 7fb93ca00640 1 --2- 172.21.15.5:0/2366002289 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb9380745c0 0x7fb93819a4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:04.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:04.835+0000 7fb93ca00640 1 -- 172.21.15.5:0/2366002289 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb93810e260 msgr2=0x7fb93819edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:46:04.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:04.835+0000 7fb93ca00640 1 --2- 172.21.15.5:0/2366002289 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb93810e260 0x7fb93819edb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:04.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:04.835+0000 7fb93ca00640 1 -- 172.21.15.5:0/2366002289 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb93819f490 con 0x7fb938074ee0 2024-09-10T09:46:04.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:04.836+0000 7fb93de00640 1 --2- 172.21.15.5:0/2366002289 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb93810e260 0x7fb93819edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:46:04.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:04.836+0000 7fb93d400640 1 --2- 172.21.15.5:0/2366002289 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb9380745c0 0x7fb93819a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:46:04.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:04.836+0000 7fb93ca00640 1 --2- 172.21.15.5:0/2366002289 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb938074ee0 0x7fb93819aa20 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fb92402a9a0 tx=0x7fb924003b20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:46:04.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:04.836+0000 7fb936000640 1 -- 172.21.15.5:0/2366002289 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb92403f070 con 0x7fb938074ee0 2024-09-10T09:46:04.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:04.836+0000 7fb93e800640 1 -- 172.21.15.5:0/2366002289 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb93819f720 con 0x7fb938074ee0 2024-09-10T09:46:04.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:04.836+0000 7fb936000640 1 -- 172.21.15.5:0/2366002289 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb924031c90 con 0x7fb938074ee0 2024-09-10T09:46:04.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:04.836+0000 7fb936000640 1 -- 172.21.15.5:0/2366002289 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb924035030 con 0x7fb938074ee0 2024-09-10T09:46:04.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:04.837+0000 7fb93e800640 1 -- 172.21.15.5:0/2366002289 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9381a7060 con 0x7fb938074ee0 2024-09-10T09:46:04.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:04.840+0000 7fb936000640 1 -- 172.21.15.5:0/2366002289 <== mon.2 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7fb924042070 con 0x7fb938074ee0 2024-09-10T09:46:04.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:04.840+0000 7fb9429dd740 1 Processor -- start 2024-09-10T09:46:04.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:04.840+0000 7fb9429dd740 1 -- start start 2024-09-10T09:46:04.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:04.840+0000 7fb9429dd740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562e5ee6de40 0x562e5ee6e220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:46:04.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:04.840+0000 7fb9429dd740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562e5ee6e760 0x562e5ee78ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:46:04.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:04.840+0000 7fb9429dd740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562e5ee79020 0x562e5ee7b410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:46:04.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:04.840+0000 7fb9429dd740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x562e5edd43e0 con 0x562e5ee6de40 2024-09-10T09:46:04.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:04.840+0000 7fb9429dd740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x562e5ed7bd80 con 0x562e5ee6e760 2024-09-10T09:46:04.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:04.840+0000 7fb9429dd740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x562e5edd03a0 con 0x562e5ee79020 2024-09-10T09:46:04.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:04.841+0000 7fb93de00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562e5ee6de40 0x562e5ee6e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:46:04.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:04.841+0000 7fb93ca00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562e5ee79020 0x562e5ee7b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:46:04.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:04.841+0000 7fb93de00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562e5ee6de40 0x562e5ee6e220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38350/0 (socket says 172.21.15.5:38350) 2024-09-10T09:46:04.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:04.841+0000 7fb93de00640 1 -- 172.21.15.5:0/2138932513 learned_addr learned my addr 172.21.15.5:0/2138932513 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:46:04.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:04.841+0000 7fb93d400640 1 --2- 172.21.15.5:0/2138932513 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562e5ee6e760 0x562e5ee78ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:46:04.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:04.841+0000 7fb93de00640 1 -- 172.21.15.5:0/2138932513 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562e5ee79020 msgr2=0x562e5ee7b410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:46:04.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:04.841+0000 7fb93de00640 1 --2- 172.21.15.5:0/2138932513 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562e5ee79020 0x562e5ee7b410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:04.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:04.841+0000 7fb93de00640 1 -- 172.21.15.5:0/2138932513 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562e5ee6e760 msgr2=0x562e5ee78ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:46:04.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:04.841+0000 7fb93de00640 1 --2- 172.21.15.5:0/2138932513 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562e5ee6e760 0x562e5ee78ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:04.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:04.841+0000 7fb93de00640 1 -- 172.21.15.5:0/2138932513 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562e5ed9aae0 con 0x562e5ee6de40 2024-09-10T09:46:04.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:04.841+0000 7fb93d400640 1 --2- 172.21.15.5:0/2138932513 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562e5ee6e760 0x562e5ee78ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:46:04.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:04.841+0000 7fb936000640 1 --2- 172.21.15.5:0/2366002289 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fb9200a2b80 0x7fb9200a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:46:04.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:04.842+0000 7fb93de00640 1 --2- 172.21.15.5:0/2138932513 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562e5ee6de40 0x562e5ee6e220 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fb930009fc0 tx=0x7fb930010860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14a3be9a41de7efc server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:46:04.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:04.842+0000 7fb936000640 1 -- 172.21.15.5:0/2366002289 <== mon.2 v2:172.21.15.102:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fb9240efa10 con 0x7fb938074ee0 2024-09-10T09:46:04.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:04.842+0000 7fb93d400640 1 --2- 172.21.15.5:0/2366002289 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fb9200a2b80 0x7fb9200a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:46:04.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:04.842+0000 7fb92a600640 1 -- 172.21.15.5:0/2138932513 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb930011700 con 0x562e5ee6de40 2024-09-10T09:46:04.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:04.842+0000 7fb92a600640 1 -- 172.21.15.5:0/2138932513 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb930011d80 con 0x562e5ee6de40 2024-09-10T09:46:04.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:04.842+0000 7fb92a600640 1 -- 172.21.15.5:0/2138932513 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb93001a380 con 0x562e5ee6de40 2024-09-10T09:46:04.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:04.843+0000 7fb9429dd740 1 -- 172.21.15.5:0/2138932513 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562e5ee6de40 msgr2=0x562e5ee6e220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:46:04.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:04.843+0000 7fb93d400640 1 --2- 172.21.15.5:0/2366002289 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fb9200a2b80 0x7fb9200a5040 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x562e5ee6b230 tx=0x7fb92c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:46:04.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:04.843+0000 7fb9429dd740 1 --2- 172.21.15.5:0/2138932513 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562e5ee6de40 0x562e5ee6e220 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fb930009fc0 tx=0x7fb930010860 comp rx=0 tx=0).stop 2024-09-10T09:46:04.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:04.843+0000 7fb9429dd740 1 -- 172.21.15.5:0/2138932513 shutdown_connections 2024-09-10T09:46:04.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:04.843+0000 7fb9429dd740 1 --2- 172.21.15.5:0/2138932513 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562e5ee79020 0x562e5ee7b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:04.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:04.843+0000 7fb9429dd740 1 --2- 172.21.15.5:0/2138932513 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562e5ee6e760 0x562e5ee78ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:04.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:04.843+0000 7fb9429dd740 1 --2- 172.21.15.5:0/2138932513 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562e5ee6de40 0x562e5ee6e220 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:04.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:04.843+0000 7fb9429dd740 1 -- 172.21.15.5:0/2138932513 >> 172.21.15.5:0/2138932513 conn(0x562e5ee6a3d0 msgr2=0x562e5ee6a7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:46:04.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:04.843+0000 7fb9429dd740 1 -- 172.21.15.5:0/2138932513 shutdown_connections 2024-09-10T09:46:04.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:04.843+0000 7fb9429dd740 1 -- 172.21.15.5:0/2138932513 wait complete. 2024-09-10T09:46:04.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:04.844+0000 7fb9429dd740 1 -- start start 2024-09-10T09:46:04.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:04.844+0000 7fb9429dd740 1 Processor -- start 2024-09-10T09:46:04.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:04.845+0000 7fb9429dd740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562e5ee6de40 0x562e5ee75ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:46:04.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:04.845+0000 7fb9429dd740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562e5ee6e760 0x562e5ee76430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:46:04.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:04.845+0000 7fb9429dd740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562e5ee79020 0x562e5ee76970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:46:04.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:04.845+0000 7fb9429dd740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x562e5ee7bbf0 con 0x562e5ee6e760 2024-09-10T09:46:04.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:04.845+0000 7fb9429dd740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x562e5ed8f0f0 con 0x562e5ee79020 2024-09-10T09:46:04.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:04.845+0000 7fb9429dd740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x562e5ee7bd70 con 0x562e5ee6de40 2024-09-10T09:46:04.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:04.845+0000 7fb9429dd740 10 client.0 _ll_register_callbacks cb 0x562e5ed76bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:46:04.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:04.845+0000 7fb93d400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562e5ee6de40 0x562e5ee75ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:46:04.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:04.845+0000 7fb93de00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562e5ee79020 0x562e5ee76970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:46:04.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:04.845+0000 7fb93d400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562e5ee6de40 0x562e5ee75ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:33972/0 (socket says 172.21.15.5:33972) 2024-09-10T09:46:04.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:04.845+0000 7fb93d400640 1 -- 172.21.15.5:0/2898157694 learned_addr learned my addr 172.21.15.5:0/2898157694 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:46:04.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:04.845+0000 7fb93ca00640 1 --2- 172.21.15.5:0/2898157694 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562e5ee6e760 0x562e5ee76430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:46:04.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:04.845+0000 7fb93d400640 1 -- 172.21.15.5:0/2898157694 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562e5ee79020 msgr2=0x562e5ee76970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:46:04.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:04.845+0000 7fb93d400640 1 --2- 172.21.15.5:0/2898157694 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562e5ee79020 0x562e5ee76970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:04.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:04.845+0000 7fb93d400640 1 -- 172.21.15.5:0/2898157694 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562e5ee6e760 msgr2=0x562e5ee76430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:46:04.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:04.845+0000 7fb93d400640 1 --2- 172.21.15.5:0/2898157694 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562e5ee6e760 0x562e5ee76430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:04.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:04.845+0000 7fb93d400640 1 -- 172.21.15.5:0/2898157694 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562e5eb453c0 con 0x562e5ee6de40 2024-09-10T09:46:04.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:04.846+0000 7fb93d400640 1 --2- 172.21.15.5:0/2898157694 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562e5ee6de40 0x562e5ee75ef0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fb92c00ff40 tx=0x7fb92c004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a992df12ac9dadfa server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:46:04.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:04.846+0000 7fb91ea00640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-09-10T09:46:04.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:04.846+0000 7fb91ea00640 1 -- 172.21.15.5:0/2898157694 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb92c02e060 con 0x562e5ee6de40 2024-09-10T09:46:04.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:04.846+0000 7fb9429dd740 1 -- 172.21.15.5:0/2898157694 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb930011d80 con 0x562e5ee6de40 2024-09-10T09:46:04.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:04.846+0000 7fb91ea00640 1 -- 172.21.15.5:0/2898157694 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fb92c018030 con 0x562e5ee6de40 2024-09-10T09:46:04.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:04.847+0000 7fb91ea00640 1 -- 172.21.15.5:0/2898157694 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fb92c035050 con 0x562e5ee6de40 2024-09-10T09:46:04.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:04.847+0000 7fb9429dd740 10 client.26950 fetch_fsmap learned FSMap version 34 2024-09-10T09:46:04.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:04.847+0000 7fb9429dd740 1 -- 172.21.15.5:0/2898157694 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x562e5ee87cc0 con 0x562e5ee6de40 2024-09-10T09:46:04.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:04.847+0000 7fb91ea00640 1 -- 172.21.15.5:0/2898157694 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb930011d80 con 0x562e5ee6de40 2024-09-10T09:46:04.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:04.847+0000 7fb91ea00640 1 -- 172.21.15.5:0/2898157694 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7fb92c01c980 con 0x562e5ee6de40 2024-09-10T09:46:04.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:04.847+0000 7fb9429dd740 10 client.26950 fetch_fsmap finished waiting for FSMap version 34 2024-09-10T09:46:04.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:04.847+0000 7fb9429dd740 10 client.26950 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-10T09:46:04.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:04.847+0000 7fb9429dd740 20 client.26950 populate_metadata read hostname 'smithi005' 2024-09-10T09:46:04.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:04.847+0000 7fb9429dd740 1 --2- 172.21.15.5:0/2898157694 >> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] conn(0x562e5ee890d0 0x562e5ee8b520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:46:04.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:04.847+0000 7fb9429dd740 4 client.26950 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:46:04.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:04.847+0000 7fb9429dd740 1 -- 172.21.15.5:0/2898157694 --> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562e5ede1950 con 0x562e5ee890d0 2024-09-10T09:46:04.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:04.848+0000 7fb93ca00640 1 --2- 172.21.15.5:0/2898157694 >> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] conn(0x562e5ee890d0 0x562e5ee8b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:46:04.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:04.848+0000 7fb93ca00640 1 --2- 172.21.15.5:0/2898157694 >> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] conn(0x562e5ee890d0 0x562e5ee8b520 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=64770356f11e2b72 server_cookie=4e5dbf98202580eb in_seq=0 out_seq=0 2024-09-10T09:46:04.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:04.848+0000 7fb91ea00640 10 client.26950 ms_handle_connect on v2:172.21.15.159:6838/2073739599 2024-09-10T09:46:04.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:04.849+0000 7fb91ea00640 1 -- 172.21.15.5:0/2898157694 <== mds.1 v2:172.21.15.159:6838/2073739599 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562e5ede1950 con 0x562e5ee890d0 2024-09-10T09:46:04.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:04.849+0000 7fb91ea00640 10 client.26950 handle_command_reply: tid=0 2024-09-10T09:46:04.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:04.849+0000 7fb9429dd740 1 client.26950 shutdown 2024-09-10T09:46:04.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:04.849+0000 7fb9429dd740 1 -- 172.21.15.5:0/2898157694 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562e5ee6de40 msgr2=0x562e5ee75ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:46:04.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:04.849+0000 7fb9429dd740 1 --2- 172.21.15.5:0/2898157694 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562e5ee6de40 0x562e5ee75ef0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fb92c00ff40 tx=0x7fb92c004730 comp rx=0 tx=0).stop 2024-09-10T09:46:04.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:04.849+0000 7fb9429dd740 1 -- 172.21.15.5:0/2898157694 shutdown_connections 2024-09-10T09:46:04.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:04.849+0000 7fb9429dd740 1 --2- 172.21.15.5:0/2898157694 >> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] conn(0x562e5ee890d0 0x562e5ee8b520 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:04.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:04.849+0000 7fb9429dd740 1 --2- 172.21.15.5:0/2898157694 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562e5ee79020 0x562e5ee76970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:04.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:04.849+0000 7fb9429dd740 1 --2- 172.21.15.5:0/2898157694 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562e5ee6e760 0x562e5ee76430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:04.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:04.849+0000 7fb9429dd740 1 --2- 172.21.15.5:0/2898157694 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562e5ee6de40 0x562e5ee75ef0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:04.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:04.849+0000 7fb9429dd740 1 -- 172.21.15.5:0/2898157694 >> 172.21.15.5:0/2898157694 conn(0x562e5ee6a3d0 msgr2=0x562e5ee70100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:46:04.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:04.849+0000 7fb91ea00640 0 client.26950 ms_handle_reset on v2:172.21.15.159:6838/2073739599 2024-09-10T09:46:04.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:04.849+0000 7fb9429dd740 1 -- 172.21.15.5:0/2898157694 shutdown_connections 2024-09-10T09:46:04.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:04.849+0000 7fb9429dd740 1 -- 172.21.15.5:0/2898157694 wait complete. 2024-09-10T09:46:04.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:04.850+0000 7fb9429dd740 20 client.26950 trim_cache size 0 max 16384 2024-09-10T09:46:04.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:04.864+0000 7fb9429dd740 1 Processor -- start 2024-09-10T09:46:04.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:04.864+0000 7fb9429dd740 1 -- start start 2024-09-10T09:46:04.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:04.864+0000 7fb9429dd740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562e5ee6de40 0x562e5ee77e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:46:04.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:04.864+0000 7fb9429dd740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562e5ee6e760 0x562e5ee90df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:46:04.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:04.864+0000 7fb9429dd740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562e5ee79020 0x562e5ee91330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:46:04.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:04.864+0000 7fb9429dd740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fb9381b0f10 con 0x562e5ee79020 2024-09-10T09:46:04.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:04.864+0000 7fb9429dd740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fb9381b2a50 con 0x562e5ee6e760 2024-09-10T09:46:04.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:04.864+0000 7fb9429dd740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fb9381b15e0 con 0x562e5ee6de40 2024-09-10T09:46:04.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:04.865+0000 7fb93ca00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562e5ee6e760 0x562e5ee90df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:46:04.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:04.865+0000 7fb93d400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562e5ee6de40 0x562e5ee77e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:46:04.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:04.865+0000 7fb93ca00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562e5ee6e760 0x562e5ee90df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:36814/0 (socket says 172.21.15.5:36814) 2024-09-10T09:46:04.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:04.865+0000 7fb93ca00640 1 -- 172.21.15.5:0/2414766407 learned_addr learned my addr 172.21.15.5:0/2414766407 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:46:04.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:04.865+0000 7fb93de00640 1 --2- 172.21.15.5:0/2414766407 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562e5ee79020 0x562e5ee91330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:46:04.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:04.865+0000 7fb93ca00640 1 -- 172.21.15.5:0/2414766407 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562e5ee6de40 msgr2=0x562e5ee77e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:46:04.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:04.865+0000 7fb93ca00640 1 --2- 172.21.15.5:0/2414766407 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562e5ee6de40 0x562e5ee77e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:04.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:04.865+0000 7fb93ca00640 1 -- 172.21.15.5:0/2414766407 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562e5ee79020 msgr2=0x562e5ee91330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:46:04.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:04.865+0000 7fb93ca00640 1 --2- 172.21.15.5:0/2414766407 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562e5ee79020 0x562e5ee91330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:04.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:04.865+0000 7fb93ca00640 1 -- 172.21.15.5:0/2414766407 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562e5ee69e70 con 0x562e5ee6e760 2024-09-10T09:46:04.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:04.865+0000 7fb93ca00640 1 --2- 172.21.15.5:0/2414766407 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562e5ee6e760 0x562e5ee90df0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fb924006010 tx=0x7fb924005d50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f196c8de7e63b5c3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:46:04.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:04.866+0000 7fb929c00640 1 -- 172.21.15.5:0/2414766407 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb924049040 con 0x562e5ee6e760 2024-09-10T09:46:04.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:04.866+0000 7fb929c00640 1 -- 172.21.15.5:0/2414766407 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb924031e40 con 0x562e5ee6e760 2024-09-10T09:46:04.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:04.866+0000 7fb929c00640 1 -- 172.21.15.5:0/2414766407 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9240ba120 con 0x562e5ee6e760 2024-09-10T09:46:04.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:04.866+0000 7fb9429dd740 1 -- 172.21.15.5:0/2414766407 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562e5ee6e760 msgr2=0x562e5ee90df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:46:04.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:04.866+0000 7fb9429dd740 1 --2- 172.21.15.5:0/2414766407 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562e5ee6e760 0x562e5ee90df0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fb924006010 tx=0x7fb924005d50 comp rx=0 tx=0).stop 2024-09-10T09:46:04.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:04.867+0000 7fb9429dd740 1 -- 172.21.15.5:0/2414766407 shutdown_connections 2024-09-10T09:46:04.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:04.867+0000 7fb9429dd740 1 --2- 172.21.15.5:0/2414766407 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562e5ee79020 0x562e5ee91330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:04.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:04.867+0000 7fb9429dd740 1 --2- 172.21.15.5:0/2414766407 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562e5ee6e760 0x562e5ee90df0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:04.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:04.867+0000 7fb9429dd740 1 --2- 172.21.15.5:0/2414766407 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562e5ee6de40 0x562e5ee77e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:04.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:04.867+0000 7fb9429dd740 1 -- 172.21.15.5:0/2414766407 >> 172.21.15.5:0/2414766407 conn(0x562e5ee6a3d0 msgr2=0x562e5ee77750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:46:04.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:04.867+0000 7fb9429dd740 1 -- 172.21.15.5:0/2414766407 shutdown_connections 2024-09-10T09:46:04.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:04.867+0000 7fb9429dd740 1 -- 172.21.15.5:0/2414766407 wait complete. 2024-09-10T09:46:04.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:04.867+0000 7fb9429dd740 1 -- start start 2024-09-10T09:46:04.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:04.867+0000 7fb9429dd740 1 Processor -- start 2024-09-10T09:46:04.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:04.868+0000 7fb9429dd740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562e5ee6de40 0x562e5ee81890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:46:04.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:04.868+0000 7fb9429dd740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562e5ee6e760 0x562e5ee7cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:46:04.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:04.868+0000 7fb9429dd740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562e5ee79020 0x562e5ee7d350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:46:04.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:04.868+0000 7fb9429dd740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fb9381b2380 con 0x562e5ee6e760 2024-09-10T09:46:04.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:04.868+0000 7fb9429dd740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fb9381b1cb0 con 0x562e5ee79020 2024-09-10T09:46:04.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:04.868+0000 7fb9429dd740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fb9381aeef0 con 0x562e5ee6de40 2024-09-10T09:46:04.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:04.868+0000 7fb9429dd740 10 client.0 _ll_register_callbacks cb 0x7fb92c012610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:46:04.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:04.868+0000 7fb93d400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562e5ee6de40 0x562e5ee81890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:46:04.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:04.868+0000 7fb93d400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562e5ee6de40 0x562e5ee81890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:33986/0 (socket says 172.21.15.5:33986) 2024-09-10T09:46:04.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:04.868+0000 7fb93d400640 1 -- 172.21.15.5:0/1420756476 learned_addr learned my addr 172.21.15.5:0/1420756476 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:46:04.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:04.868+0000 7fb93de00640 1 --2- 172.21.15.5:0/1420756476 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562e5ee79020 0x562e5ee7d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:46:04.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:04.868+0000 7fb93ca00640 1 --2- 172.21.15.5:0/1420756476 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562e5ee6e760 0x562e5ee7cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:46:04.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:04.869+0000 7fb93d400640 1 -- 172.21.15.5:0/1420756476 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562e5ee79020 msgr2=0x562e5ee7d350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:46:04.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:04.869+0000 7fb93d400640 1 --2- 172.21.15.5:0/1420756476 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562e5ee79020 0x562e5ee7d350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:04.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:04.869+0000 7fb93d400640 1 -- 172.21.15.5:0/1420756476 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562e5ee6e760 msgr2=0x562e5ee7cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:46:04.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:04.869+0000 7fb93d400640 1 --2- 172.21.15.5:0/1420756476 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562e5ee6e760 0x562e5ee7cbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:04.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:04.869+0000 7fb93d400640 1 -- 172.21.15.5:0/1420756476 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562e5eef2dc0 con 0x562e5ee6de40 2024-09-10T09:46:04.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:04.869+0000 7fb93d400640 1 --2- 172.21.15.5:0/1420756476 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562e5ee6de40 0x562e5ee81890 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fb92c01cd50 tx=0x7fb92c01c650 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1f20fa285f8b6aca server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:46:04.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:04.869+0000 7fb91ea00640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-09-10T09:46:04.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:04.869+0000 7fb91ea00640 1 -- 172.21.15.5:0/1420756476 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb92c00fd30 con 0x562e5ee6de40 2024-09-10T09:46:04.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:04.869+0000 7fb9429dd740 1 -- 172.21.15.5:0/1420756476 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x562e5eddbe60 con 0x562e5ee6de40 2024-09-10T09:46:04.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:04.869+0000 7fb91ea00640 1 -- 172.21.15.5:0/1420756476 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fb92c01ca50 con 0x562e5ee6de40 2024-09-10T09:46:04.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:04.869+0000 7fb91ea00640 1 -- 172.21.15.5:0/1420756476 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fb92c046050 con 0x562e5ee6de40 2024-09-10T09:46:04.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:04.869+0000 7fb9429dd740 10 client.26956 fetch_fsmap learned FSMap version 34 2024-09-10T09:46:04.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:04.869+0000 7fb9429dd740 1 -- 172.21.15.5:0/1420756476 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x562e5eef3f50 con 0x562e5ee6de40 2024-09-10T09:46:04.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:04.870+0000 7fb91ea00640 1 -- 172.21.15.5:0/1420756476 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x562e5eddbe60 con 0x562e5ee6de40 2024-09-10T09:46:04.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:04.870+0000 7fb91ea00640 1 -- 172.21.15.5:0/1420756476 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7fb92c041030 con 0x562e5ee6de40 2024-09-10T09:46:04.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:04.870+0000 7fb9429dd740 10 client.26956 fetch_fsmap finished waiting for FSMap version 34 2024-09-10T09:46:04.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:04.870+0000 7fb9429dd740 10 client.26956 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-10T09:46:04.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:04.870+0000 7fb9429dd740 20 client.26956 populate_metadata read hostname 'smithi005' 2024-09-10T09:46:04.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:04.870+0000 7fb9429dd740 1 --2- 172.21.15.5:0/1420756476 >> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] conn(0x562e5eef4690 0x562e5eef6ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:46:04.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:04.870+0000 7fb9429dd740 4 client.26956 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T09:46:04.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:04.870+0000 7fb9429dd740 1 -- 172.21.15.5:0/1420756476 --> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562e5ed58010 con 0x562e5eef4690 2024-09-10T09:46:04.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:04.870+0000 7fb93ca00640 1 --2- 172.21.15.5:0/1420756476 >> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] conn(0x562e5eef4690 0x562e5eef6ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:46:04.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:04.871+0000 7fb93ca00640 1 --2- 172.21.15.5:0/1420756476 >> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] conn(0x562e5eef4690 0x562e5eef6ab0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e9513bb8830e6e4b server_cookie=5c7fa5158e615fd7 in_seq=0 out_seq=0 2024-09-10T09:46:04.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:04.871+0000 7fb91ea00640 10 client.26956 ms_handle_connect on v2:172.21.15.159:6838/2073739599 2024-09-10T09:46:04.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:04.871+0000 7fb91ea00640 1 -- 172.21.15.5:0/1420756476 <== mds.1 v2:172.21.15.159:6838/2073739599 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562e5ed58010 con 0x562e5eef4690 2024-09-10T09:46:04.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:04.871+0000 7fb91ea00640 10 client.26956 handle_command_reply: tid=0 2024-09-10T09:46:04.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:04.871+0000 7fb9429dd740 1 client.26956 shutdown 2024-09-10T09:46:04.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:04.871+0000 7fb9429dd740 1 -- 172.21.15.5:0/1420756476 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562e5ee6de40 msgr2=0x562e5ee81890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:46:04.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:04.871+0000 7fb9429dd740 1 --2- 172.21.15.5:0/1420756476 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562e5ee6de40 0x562e5ee81890 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fb92c01cd50 tx=0x7fb92c01c650 comp rx=0 tx=0).stop 2024-09-10T09:46:04.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:04.871+0000 7fb9429dd740 1 -- 172.21.15.5:0/1420756476 shutdown_connections 2024-09-10T09:46:04.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:04.871+0000 7fb9429dd740 1 --2- 172.21.15.5:0/1420756476 >> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] conn(0x562e5eef4690 0x562e5eef6ab0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:04.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:04.871+0000 7fb9429dd740 1 --2- 172.21.15.5:0/1420756476 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562e5ee79020 0x562e5ee7d350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:04.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:04.871+0000 7fb9429dd740 1 --2- 172.21.15.5:0/1420756476 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562e5ee6e760 0x562e5ee7cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:04.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:04.871+0000 7fb9429dd740 1 --2- 172.21.15.5:0/1420756476 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562e5ee6de40 0x562e5ee81890 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:04.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:04.871+0000 7fb91ea00640 0 client.26956 ms_handle_reset on v2:172.21.15.159:6838/2073739599 2024-09-10T09:46:04.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:04.871+0000 7fb9429dd740 1 -- 172.21.15.5:0/1420756476 >> 172.21.15.5:0/1420756476 conn(0x562e5ee6a3d0 msgr2=0x562e5ee77f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:46:04.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:04.872+0000 7fb9429dd740 1 -- 172.21.15.5:0/1420756476 shutdown_connections 2024-09-10T09:46:04.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:04.872+0000 7fb9429dd740 1 -- 172.21.15.5:0/1420756476 wait complete. 2024-09-10T09:46:04.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:04.872+0000 7fb9429dd740 20 client.26956 trim_cache size 0 max 16384 2024-09-10T09:46:04.877 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-10T09:46:04.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:04.873+0000 7fb93e800640 1 -- 172.21.15.5:0/2366002289 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fb9200a2b80 msgr2=0x7fb9200a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:46:04.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:04.873+0000 7fb93e800640 1 --2- 172.21.15.5:0/2366002289 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fb9200a2b80 0x7fb9200a5040 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x562e5ee6b230 tx=0x7fb92c005f90 comp rx=0 tx=0).stop 2024-09-10T09:46:04.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:04.873+0000 7fb93e800640 1 -- 172.21.15.5:0/2366002289 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb938074ee0 msgr2=0x7fb93819aa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:46:04.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:04.873+0000 7fb93e800640 1 --2- 172.21.15.5:0/2366002289 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb938074ee0 0x7fb93819aa20 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fb92402a9a0 tx=0x7fb924003b20 comp rx=0 tx=0).stop 2024-09-10T09:46:04.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:04.873+0000 7fb93e800640 1 -- 172.21.15.5:0/2366002289 shutdown_connections 2024-09-10T09:46:04.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:04.873+0000 7fb93e800640 1 --2- 172.21.15.5:0/2366002289 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fb9200a2b80 0x7fb9200a5040 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:04.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:04.873+0000 7fb93e800640 1 --2- 172.21.15.5:0/2366002289 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb93810e260 0x7fb93819edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:04.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:04.873+0000 7fb93e800640 1 --2- 172.21.15.5:0/2366002289 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb938074ee0 0x7fb93819aa20 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:04.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:04.873+0000 7fb93e800640 1 --2- 172.21.15.5:0/2366002289 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb9380745c0 0x7fb93819a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:04.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:04.873+0000 7fb93e800640 1 -- 172.21.15.5:0/2366002289 >> 172.21.15.5:0/2366002289 conn(0x7fb9380fd740 msgr2=0x7fb9380fe3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:46:04.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:04.873+0000 7fb93e800640 1 -- 172.21.15.5:0/2366002289 shutdown_connections 2024-09-10T09:46:04.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:04.873+0000 7fb93e800640 1 -- 172.21.15.5:0/2366002289 wait complete. 2024-09-10T09:46:05.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:46:04 smithi005 ceph-mon[30350]: pgmap v625: 129 pgs: 129 active+clean; 178 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.0 MiB/s wr, 149 op/s 2024-09-10T09:46:05.062 INFO:tasks.workunit.client.0.smithi005.stdout: 1 445794 6.04 MB/sec execute 226 sec latency 72.293 ms 2024-09-10T09:46:05.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:46:04 smithi159 ceph-mon[35513]: pgmap v625: 129 pgs: 129 active+clean; 178 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.0 MiB/s wr, 149 op/s 2024-09-10T09:46:05.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:46:04 smithi102 ceph-mon[35442]: pgmap v625: 129 pgs: 129 active+clean; 178 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.0 MiB/s wr, 149 op/s 2024-09-10T09:46:05.646 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-10T09:46:06.041 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:46:06.062 INFO:tasks.workunit.client.0.smithi005.stdout: 1 447737 6.07 MB/sec execute 227 sec latency 36.826 ms 2024-09-10T09:46:06.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:06.604+0000 7fe9b6a00640 1 -- 172.21.15.5:0/788971662 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe9b0077040 msgr2=0x7fe9b0112510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:46:06.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:06.604+0000 7fe9b6a00640 1 --2- 172.21.15.5:0/788971662 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe9b0077040 0x7fe9b0112510 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fe998009900 tx=0x7fe99802a3f0 comp rx=0 tx=0).stop 2024-09-10T09:46:06.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:06.605+0000 7fe9b6a00640 1 -- 172.21.15.5:0/788971662 shutdown_connections 2024-09-10T09:46:06.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:06.605+0000 7fe9b6a00640 1 --2- 172.21.15.5:0/788971662 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe9b0112a50 0x7fe9b0114e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:06.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:06.605+0000 7fe9b6a00640 1 --2- 172.21.15.5:0/788971662 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe9b0077040 0x7fe9b0112510 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:06.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:06.605+0000 7fe9b6a00640 1 --2- 172.21.15.5:0/788971662 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe9b0076720 0x7fe9b0076b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:06.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:06.605+0000 7fe9b6a00640 1 -- 172.21.15.5:0/788971662 >> 172.21.15.5:0/788971662 conn(0x7fe9b00ffa50 msgr2=0x7fe9b0101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:46:06.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:06.605+0000 7fe9b6a00640 1 -- 172.21.15.5:0/788971662 shutdown_connections 2024-09-10T09:46:06.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:06.605+0000 7fe9b6a00640 1 -- 172.21.15.5:0/788971662 wait complete. 2024-09-10T09:46:06.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:06.606+0000 7fe9b6a00640 1 Processor -- start 2024-09-10T09:46:06.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:06.606+0000 7fe9b6a00640 1 -- start start 2024-09-10T09:46:06.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:06.606+0000 7fe9b6a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe9b0076720 0x7fe9b01a47b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:46:06.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:06.606+0000 7fe9b6a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe9b0077040 0x7fe9b01a4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:46:06.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:06.606+0000 7fe9b6a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe9b0112a50 0x7fe9b019e970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:46:06.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:06.607+0000 7fe9b6a00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe9b0118620 con 0x7fe9b0077040 2024-09-10T09:46:06.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:06.607+0000 7fe9b6a00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fe9b01184a0 con 0x7fe9b0112a50 2024-09-10T09:46:06.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:06.607+0000 7fe9b6a00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fe9b01187a0 con 0x7fe9b0076720 2024-09-10T09:46:06.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:06.607+0000 7fe9afe00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe9b0076720 0x7fe9b01a47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:46:06.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:06.607+0000 7fe9afe00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe9b0076720 0x7fe9b01a47b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:33994/0 (socket says 172.21.15.5:33994) 2024-09-10T09:46:06.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:06.607+0000 7fe9afe00640 1 -- 172.21.15.5:0/647437127 learned_addr learned my addr 172.21.15.5:0/647437127 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:46:06.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:06.607+0000 7fe9a7e00640 1 --2- 172.21.15.5:0/647437127 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe9b0077040 0x7fe9b01a4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:46:06.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:06.607+0000 7fe9b4a00640 1 --2- 172.21.15.5:0/647437127 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe9b0112a50 0x7fe9b019e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:46:06.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:06.607+0000 7fe9afe00640 1 -- 172.21.15.5:0/647437127 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe9b0112a50 msgr2=0x7fe9b019e970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:46:06.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:06.607+0000 7fe9afe00640 1 --2- 172.21.15.5:0/647437127 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe9b0112a50 0x7fe9b019e970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:06.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:06.607+0000 7fe9afe00640 1 -- 172.21.15.5:0/647437127 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe9b0077040 msgr2=0x7fe9b01a4cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:46:06.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:06.607+0000 7fe9afe00640 1 --2- 172.21.15.5:0/647437127 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe9b0077040 0x7fe9b01a4cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:06.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:06.607+0000 7fe9afe00640 1 -- 172.21.15.5:0/647437127 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9b019f0a0 con 0x7fe9b0076720 2024-09-10T09:46:06.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:06.607+0000 7fe9b4a00640 1 --2- 172.21.15.5:0/647437127 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe9b0112a50 0x7fe9b019e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:46:06.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:06.608+0000 7fe9afe00640 1 --2- 172.21.15.5:0/647437127 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe9b0076720 0x7fe9b01a47b0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fe9a000ea60 tx=0x7fe9a000ef20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:46:06.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:06.608+0000 7fe9ad600640 1 -- 172.21.15.5:0/647437127 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9a000cde0 con 0x7fe9b0076720 2024-09-10T09:46:06.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:06.608+0000 7fe9ad600640 1 -- 172.21.15.5:0/647437127 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe9a00035f0 con 0x7fe9b0076720 2024-09-10T09:46:06.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:06.608+0000 7fe9b6a00640 1 -- 172.21.15.5:0/647437127 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9b019f390 con 0x7fe9b0076720 2024-09-10T09:46:06.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:06.608+0000 7fe9ad600640 1 -- 172.21.15.5:0/647437127 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9a0010670 con 0x7fe9b0076720 2024-09-10T09:46:06.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:06.608+0000 7fe9b6a00640 1 -- 172.21.15.5:0/647437127 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9b01ab680 con 0x7fe9b0076720 2024-09-10T09:46:06.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:06.611+0000 7fe9ad600640 1 -- 172.21.15.5:0/647437127 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7fe9a00037a0 con 0x7fe9b0076720 2024-09-10T09:46:06.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:06.612+0000 7fe9b964f740 1 Processor -- start 2024-09-10T09:46:06.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:06.612+0000 7fe9b964f740 1 -- start start 2024-09-10T09:46:06.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:06.612+0000 7fe9b964f740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55757ff48c70 0x55757ff49050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:46:06.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:06.612+0000 7fe9b964f740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55757ff49590 0x55757ff53910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:46:06.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:06.612+0000 7fe9b964f740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55757ff53e50 0x55757ff56240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:46:06.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:06.612+0000 7fe9b964f740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55757feaf3e0 con 0x55757ff49590 2024-09-10T09:46:06.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:06.612+0000 7fe9b964f740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55757fe56d80 con 0x55757ff48c70 2024-09-10T09:46:06.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:06.612+0000 7fe9b964f740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55757feab3a0 con 0x55757ff53e50 2024-09-10T09:46:06.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:06.613+0000 7fe9b4a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55757ff48c70 0x55757ff49050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:46:06.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:06.613+0000 7fe9a7e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55757ff53e50 0x55757ff56240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:46:06.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:06.613+0000 7fe9b4a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55757ff48c70 0x55757ff49050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:36862/0 (socket says 172.21.15.5:36862) 2024-09-10T09:46:06.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:06.613+0000 7fe9ad600640 1 --2- 172.21.15.5:0/647437127 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fe9940a2b80 0x7fe9940a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:46:06.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:06.613+0000 7fe9b4a00640 1 -- 172.21.15.5:0/2744451414 learned_addr learned my addr 172.21.15.5:0/2744451414 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:46:06.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:06.613+0000 7fe9afe00640 1 --2- 172.21.15.5:0/2744451414 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55757ff49590 0x55757ff53910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:46:06.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:06.613+0000 7fe9ad600640 1 -- 172.21.15.5:0/647437127 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fe9a00cf710 con 0x7fe9b0076720 2024-09-10T09:46:06.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:06.614+0000 7fe9b4a00640 1 --2- 172.21.15.5:0/647437127 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fe9940a2b80 0x7fe9940a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:46:06.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:06.614+0000 7fe9a7e00640 1 -- 172.21.15.5:0/2744451414 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55757ff48c70 msgr2=0x55757ff49050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:46:06.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:06.614+0000 7fe9a7e00640 1 --2- 172.21.15.5:0/2744451414 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55757ff48c70 0x55757ff49050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:06.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:06.614+0000 7fe9a7e00640 1 -- 172.21.15.5:0/2744451414 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55757ff49590 msgr2=0x55757ff53910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:46:06.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:06.614+0000 7fe9a7e00640 1 --2- 172.21.15.5:0/2744451414 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55757ff49590 0x55757ff53910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:06.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:06.614+0000 7fe9a7e00640 1 -- 172.21.15.5:0/2744451414 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55757fe75ae0 con 0x55757ff53e50 2024-09-10T09:46:06.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:06.614+0000 7fe9b4a00640 1 --2- 172.21.15.5:0/2744451414 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55757ff48c70 0x55757ff49050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:46:06.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:06.614+0000 7fe9afe00640 1 --2- 172.21.15.5:0/2744451414 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55757ff49590 0x55757ff53910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:46:06.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:06.614+0000 7fe9a7e00640 1 --2- 172.21.15.5:0/2744451414 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55757ff53e50 0x55757ff56240 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fe99802a9e0 tx=0x7fe998005de0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a094a2f67a5daf03 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:46:06.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:06.615+0000 7fe9a5200640 1 -- 172.21.15.5:0/2744451414 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe998035040 con 0x55757ff53e50 2024-09-10T09:46:06.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:06.615+0000 7fe9a5200640 1 -- 172.21.15.5:0/2744451414 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe998004400 con 0x55757ff53e50 2024-09-10T09:46:06.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:06.615+0000 7fe9a5200640 1 -- 172.21.15.5:0/2744451414 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9980035f0 con 0x55757ff53e50 2024-09-10T09:46:06.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:06.615+0000 7fe9b964f740 1 -- 172.21.15.5:0/2744451414 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55757ff53e50 msgr2=0x55757ff56240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:46:06.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:06.615+0000 7fe9b964f740 1 --2- 172.21.15.5:0/2744451414 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55757ff53e50 0x55757ff56240 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fe99802a9e0 tx=0x7fe998005de0 comp rx=0 tx=0).stop 2024-09-10T09:46:06.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:06.615+0000 7fe9b4a00640 1 --2- 172.21.15.5:0/647437127 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fe9940a2b80 0x7fe9940a5040 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fe9a8013980 tx=0x7fe9a8015040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:46:06.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:06.615+0000 7fe9b964f740 1 -- 172.21.15.5:0/2744451414 shutdown_connections 2024-09-10T09:46:06.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:06.615+0000 7fe9b964f740 1 --2- 172.21.15.5:0/2744451414 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55757ff53e50 0x55757ff56240 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:06.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:06.615+0000 7fe9b964f740 1 --2- 172.21.15.5:0/2744451414 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55757ff49590 0x55757ff53910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:06.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:06.615+0000 7fe9b964f740 1 --2- 172.21.15.5:0/2744451414 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55757ff48c70 0x55757ff49050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:06.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:06.616+0000 7fe9b964f740 1 -- 172.21.15.5:0/2744451414 >> 172.21.15.5:0/2744451414 conn(0x55757ff45200 msgr2=0x55757ff455e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:46:06.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:06.616+0000 7fe9b964f740 1 -- 172.21.15.5:0/2744451414 shutdown_connections 2024-09-10T09:46:06.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:06.616+0000 7fe9b964f740 1 -- 172.21.15.5:0/2744451414 wait complete. 2024-09-10T09:46:06.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:06.616+0000 7fe9b964f740 1 -- start start 2024-09-10T09:46:06.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:06.616+0000 7fe9b964f740 1 Processor -- start 2024-09-10T09:46:06.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:06.617+0000 7fe9b964f740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55757ff48c70 0x55757ff50d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:46:06.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:06.617+0000 7fe9b964f740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55757ff49590 0x55757ff512d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:46:06.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:06.617+0000 7fe9b964f740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55757ff53e50 0x55757ff51810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:46:06.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:06.617+0000 7fe9b964f740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55757ff56a20 con 0x55757ff53e50 2024-09-10T09:46:06.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:06.617+0000 7fe9b964f740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55757fe6a0f0 con 0x55757ff48c70 2024-09-10T09:46:06.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:06.617+0000 7fe9b964f740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55757ff56ba0 con 0x55757ff49590 2024-09-10T09:46:06.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:06.617+0000 7fe9b964f740 10 client.0 _ll_register_callbacks cb 0x55757fe51bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:46:06.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:06.617+0000 7fe9a7e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55757ff49590 0x55757ff512d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:46:06.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:06.617+0000 7fe9b4a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55757ff53e50 0x55757ff51810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:46:06.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:06.617+0000 7fe9a7e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55757ff49590 0x55757ff512d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:34004/0 (socket says 172.21.15.5:34004) 2024-09-10T09:46:06.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:06.617+0000 7fe9a7e00640 1 -- 172.21.15.5:0/2937824853 learned_addr learned my addr 172.21.15.5:0/2937824853 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:46:06.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:06.617+0000 7fe9afe00640 1 --2- 172.21.15.5:0/2937824853 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55757ff48c70 0x55757ff50d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:46:06.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:06.618+0000 7fe9a7e00640 1 -- 172.21.15.5:0/2937824853 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55757ff48c70 msgr2=0x55757ff50d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:46:06.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:06.618+0000 7fe9a7e00640 1 --2- 172.21.15.5:0/2937824853 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55757ff48c70 0x55757ff50d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:06.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:06.618+0000 7fe9a7e00640 1 -- 172.21.15.5:0/2937824853 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55757ff53e50 msgr2=0x55757ff51810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:46:06.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:06.618+0000 7fe9a7e00640 1 --2- 172.21.15.5:0/2937824853 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55757ff53e50 0x55757ff51810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:06.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:06.618+0000 7fe9a7e00640 1 -- 172.21.15.5:0/2937824853 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55757fc203c0 con 0x55757ff49590 2024-09-10T09:46:06.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:06.618+0000 7fe9afe00640 1 --2- 172.21.15.5:0/2937824853 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55757ff48c70 0x55757ff50d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:46:06.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:06.618+0000 7fe9a7e00640 1 --2- 172.21.15.5:0/2937824853 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55757ff49590 0x55757ff512d0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fe998004920 tx=0x7fe998004950 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3fd9b1ffdb0438cb server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:46:06.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:06.618+0000 7fe99d600640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-09-10T09:46:06.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:06.618+0000 7fe99d600640 1 -- 172.21.15.5:0/2937824853 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe998035070 con 0x55757ff49590 2024-09-10T09:46:06.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:06.618+0000 7fe99d600640 1 -- 172.21.15.5:0/2937824853 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fe998004200 con 0x55757ff49590 2024-09-10T09:46:06.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:06.618+0000 7fe9b964f740 1 -- 172.21.15.5:0/2937824853 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe998004400 con 0x55757ff49590 2024-09-10T09:46:06.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:06.619+0000 7fe99d600640 1 -- 172.21.15.5:0/2937824853 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fe998009520 con 0x55757ff49590 2024-09-10T09:46:06.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:06.619+0000 7fe9b964f740 10 client.26968 fetch_fsmap learned FSMap version 34 2024-09-10T09:46:06.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:06.619+0000 7fe9b964f740 1 -- 172.21.15.5:0/2937824853 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55757ff56a20 con 0x55757ff49590 2024-09-10T09:46:06.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:06.619+0000 7fe99d600640 1 -- 172.21.15.5:0/2937824853 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe998004400 con 0x55757ff49590 2024-09-10T09:46:06.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:06.619+0000 7fe99d600640 1 -- 172.21.15.5:0/2937824853 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7fe998031c70 con 0x55757ff49590 2024-09-10T09:46:06.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:06.619+0000 7fe9b964f740 10 client.26968 fetch_fsmap finished waiting for FSMap version 34 2024-09-10T09:46:06.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:06.619+0000 7fe9b964f740 10 client.26968 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T09:46:06.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:06.619+0000 7fe9b964f740 20 client.26968 populate_metadata read hostname 'smithi005' 2024-09-10T09:46:06.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:06.619+0000 7fe9b964f740 1 --2- 172.21.15.5:0/2937824853 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55757ff63c30 0x55757ff66020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:46:06.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:06.619+0000 7fe9b964f740 4 client.26968 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:46:06.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:06.619+0000 7fe9b964f740 1 -- 172.21.15.5:0/2937824853 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55757febc950 con 0x55757ff63c30 2024-09-10T09:46:06.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:06.619+0000 7fe9afe00640 1 --2- 172.21.15.5:0/2937824853 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55757ff63c30 0x55757ff66020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:46:06.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:06.620+0000 7fe9afe00640 1 --2- 172.21.15.5:0/2937824853 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55757ff63c30 0x55757ff66020 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3ca5f1e772c224ed server_cookie=edecbdc5eb8ef36c in_seq=0 out_seq=0 2024-09-10T09:46:06.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:06.620+0000 7fe99d600640 10 client.26968 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T09:46:06.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:06.621+0000 7fe99d600640 1 -- 172.21.15.5:0/2937824853 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55757febc950 con 0x55757ff63c30 2024-09-10T09:46:06.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:06.621+0000 7fe99d600640 10 client.26968 handle_command_reply: tid=0 2024-09-10T09:46:06.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:06.621+0000 7fe9b964f740 1 client.26968 shutdown 2024-09-10T09:46:06.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:06.621+0000 7fe9b964f740 1 -- 172.21.15.5:0/2937824853 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55757ff49590 msgr2=0x55757ff512d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:46:06.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:06.621+0000 7fe9b964f740 1 --2- 172.21.15.5:0/2937824853 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55757ff49590 0x55757ff512d0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fe998004920 tx=0x7fe998004950 comp rx=0 tx=0).stop 2024-09-10T09:46:06.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:06.621+0000 7fe9b964f740 1 -- 172.21.15.5:0/2937824853 shutdown_connections 2024-09-10T09:46:06.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:06.621+0000 7fe9b964f740 1 --2- 172.21.15.5:0/2937824853 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55757ff63c30 0x55757ff66020 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:06.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:06.621+0000 7fe9b964f740 1 --2- 172.21.15.5:0/2937824853 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55757ff53e50 0x55757ff51810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:06.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:06.621+0000 7fe9b964f740 1 --2- 172.21.15.5:0/2937824853 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55757ff49590 0x55757ff512d0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:06.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:06.621+0000 7fe9b964f740 1 --2- 172.21.15.5:0/2937824853 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55757ff48c70 0x55757ff50d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:06.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:06.621+0000 7fe99d600640 0 client.26968 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T09:46:06.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:06.621+0000 7fe9b964f740 1 -- 172.21.15.5:0/2937824853 >> 172.21.15.5:0/2937824853 conn(0x55757ff45200 msgr2=0x55757ff4af30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:46:06.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:06.621+0000 7fe9b964f740 1 -- 172.21.15.5:0/2937824853 shutdown_connections 2024-09-10T09:46:06.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:06.621+0000 7fe9b964f740 1 -- 172.21.15.5:0/2937824853 wait complete. 2024-09-10T09:46:06.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:06.622+0000 7fe9b964f740 20 client.26968 trim_cache size 0 max 16384 2024-09-10T09:46:06.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:06.637+0000 7fe9b964f740 1 Processor -- start 2024-09-10T09:46:06.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:06.637+0000 7fe9b964f740 1 -- start start 2024-09-10T09:46:06.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:06.638+0000 7fe9b964f740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55757ff48c70 0x55757ff50650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:46:06.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:06.638+0000 7fe9b964f740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55757ff49590 0x55757ff52d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:46:06.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:06.638+0000 7fe9b964f740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55757ff53e50 0x55757ff6b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:46:06.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:06.638+0000 7fe9b964f740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe9b01b54e0 con 0x55757ff48c70 2024-09-10T09:46:06.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:06.638+0000 7fe9b964f740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fe9b01b7020 con 0x55757ff53e50 2024-09-10T09:46:06.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:06.638+0000 7fe9b964f740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fe9b01b5bb0 con 0x55757ff49590 2024-09-10T09:46:06.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:06.638+0000 7fe9a7e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55757ff49590 0x55757ff52d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:46:06.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:06.638+0000 7fe9b4a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55757ff53e50 0x55757ff6b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:46:06.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:06.638+0000 7fe9afe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55757ff48c70 0x55757ff50650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:46:06.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:06.638+0000 7fe9b4a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55757ff53e50 0x55757ff6b6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:36878/0 (socket says 172.21.15.5:36878) 2024-09-10T09:46:06.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:06.638+0000 7fe9b4a00640 1 -- 172.21.15.5:0/2867578791 learned_addr learned my addr 172.21.15.5:0/2867578791 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:46:06.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:06.638+0000 7fe9a7e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55757ff49590 0x55757ff52d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:34018/0 (socket says 172.21.15.5:34018) 2024-09-10T09:46:06.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:06.638+0000 7fe9b4a00640 1 -- 172.21.15.5:0/2867578791 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55757ff49590 msgr2=0x55757ff52d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:46:06.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:06.638+0000 7fe9b4a00640 1 --2- 172.21.15.5:0/2867578791 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55757ff49590 0x55757ff52d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:06.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:06.638+0000 7fe9b4a00640 1 -- 172.21.15.5:0/2867578791 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55757ff48c70 msgr2=0x55757ff50650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:46:06.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:06.638+0000 7fe9b4a00640 1 --2- 172.21.15.5:0/2867578791 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55757ff48c70 0x55757ff50650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:06.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:06.638+0000 7fe9b4a00640 1 -- 172.21.15.5:0/2867578791 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55757ff48aa0 con 0x55757ff53e50 2024-09-10T09:46:06.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:06.638+0000 7fe9afe00640 1 --2- 172.21.15.5:0/2867578791 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55757ff48c70 0x55757ff50650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:46:06.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:06.639+0000 7fe9b4a00640 1 --2- 172.21.15.5:0/2867578791 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55757ff53e50 0x55757ff6b6d0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fe9a80133a0 tx=0x7fe9a8017000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=258a5929a59eac4d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:46:06.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:06.639+0000 7fe99fe00640 1 -- 172.21.15.5:0/2867578791 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9a8010930 con 0x55757ff53e50 2024-09-10T09:46:06.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:06.639+0000 7fe99fe00640 1 -- 172.21.15.5:0/2867578791 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe9a8020400 con 0x55757ff53e50 2024-09-10T09:46:06.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:06.639+0000 7fe99fe00640 1 -- 172.21.15.5:0/2867578791 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9a8029620 con 0x55757ff53e50 2024-09-10T09:46:06.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:06.640+0000 7fe9b964f740 1 -- 172.21.15.5:0/2867578791 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55757ff53e50 msgr2=0x55757ff6b6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:46:06.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:06.640+0000 7fe9b964f740 1 --2- 172.21.15.5:0/2867578791 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55757ff53e50 0x55757ff6b6d0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fe9a80133a0 tx=0x7fe9a8017000 comp rx=0 tx=0).stop 2024-09-10T09:46:06.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:06.640+0000 7fe9b964f740 1 -- 172.21.15.5:0/2867578791 shutdown_connections 2024-09-10T09:46:06.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:06.640+0000 7fe9b964f740 1 --2- 172.21.15.5:0/2867578791 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55757ff53e50 0x55757ff6b6d0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:06.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:06.640+0000 7fe9b964f740 1 --2- 172.21.15.5:0/2867578791 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55757ff49590 0x55757ff52d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:06.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:06.640+0000 7fe9b964f740 1 --2- 172.21.15.5:0/2867578791 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55757ff48c70 0x55757ff50650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:06.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:06.640+0000 7fe9b964f740 1 -- 172.21.15.5:0/2867578791 >> 172.21.15.5:0/2867578791 conn(0x55757ff45200 msgr2=0x55757ff525f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:46:06.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:06.640+0000 7fe9b964f740 1 -- 172.21.15.5:0/2867578791 shutdown_connections 2024-09-10T09:46:06.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:06.640+0000 7fe9b964f740 1 -- 172.21.15.5:0/2867578791 wait complete. 2024-09-10T09:46:06.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:06.640+0000 7fe9b964f740 1 -- start start 2024-09-10T09:46:06.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:06.640+0000 7fe9b964f740 1 Processor -- start 2024-09-10T09:46:06.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:06.641+0000 7fe9b964f740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55757ff48c70 0x55757ff57ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:46:06.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:06.641+0000 7fe9b964f740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55757ff49590 0x55757ff580e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:46:06.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:06.641+0000 7fe9b964f740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55757ff53e50 0x55757ff58620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:46:06.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:06.641+0000 7fe9b964f740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe9b01b6950 con 0x55757ff49590 2024-09-10T09:46:06.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:06.641+0000 7fe9b964f740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fe9b01b6280 con 0x55757ff53e50 2024-09-10T09:46:06.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:06.641+0000 7fe9b964f740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fe9b01b34c0 con 0x55757ff48c70 2024-09-10T09:46:06.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:06.641+0000 7fe9b964f740 10 client.0 _ll_register_callbacks cb 0x7fe9a0003e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:46:06.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:06.641+0000 7fe9afe00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55757ff48c70 0x55757ff57ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:46:06.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:06.641+0000 7fe9a7e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55757ff49590 0x55757ff580e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:46:06.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:06.641+0000 7fe9a7e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55757ff49590 0x55757ff580e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38448/0 (socket says 172.21.15.5:38448) 2024-09-10T09:46:06.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:06.641+0000 7fe9a7e00640 1 -- 172.21.15.5:0/2155991117 learned_addr learned my addr 172.21.15.5:0/2155991117 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:46:06.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:06.641+0000 7fe9afe00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55757ff48c70 0x55757ff57ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:34024/0 (socket says 172.21.15.5:34024) 2024-09-10T09:46:06.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:06.641+0000 7fe9b4a00640 1 --2- 172.21.15.5:0/2155991117 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55757ff53e50 0x55757ff58620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:46:06.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:06.642+0000 7fe9a7e00640 1 -- 172.21.15.5:0/2155991117 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55757ff48c70 msgr2=0x55757ff57ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:46:06.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:06.642+0000 7fe9a7e00640 1 --2- 172.21.15.5:0/2155991117 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55757ff48c70 0x55757ff57ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:06.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:06.642+0000 7fe9a7e00640 1 -- 172.21.15.5:0/2155991117 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55757ff53e50 msgr2=0x55757ff58620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:46:06.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:06.642+0000 7fe9a7e00640 1 --2- 172.21.15.5:0/2155991117 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55757ff53e50 0x55757ff58620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:06.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:06.642+0000 7fe9a7e00640 1 -- 172.21.15.5:0/2155991117 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55757ffcd6f0 con 0x55757ff49590 2024-09-10T09:46:06.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:06.642+0000 7fe9afe00640 1 --2- 172.21.15.5:0/2155991117 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55757ff48c70 0x55757ff57ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:46:06.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:06.642+0000 7fe9b4a00640 1 --2- 172.21.15.5:0/2155991117 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55757ff53e50 0x55757ff58620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:46:06.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:06.642+0000 7fe9a7e00640 1 --2- 172.21.15.5:0/2155991117 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55757ff49590 0x55757ff580e0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fe998009900 tx=0x7fe998004430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4084e44327398d25 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:46:06.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:06.642+0000 7fe99d600640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-10T09:46:06.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:06.642+0000 7fe99d600640 1 -- 172.21.15.5:0/2155991117 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe998035040 con 0x55757ff49590 2024-09-10T09:46:06.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:06.642+0000 7fe9b964f740 1 -- 172.21.15.5:0/2155991117 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe9a8020400 con 0x55757ff49590 2024-09-10T09:46:06.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:06.642+0000 7fe99d600640 1 -- 172.21.15.5:0/2155991117 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fe9980054b0 con 0x55757ff49590 2024-09-10T09:46:06.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:06.642+0000 7fe99d600640 1 -- 172.21.15.5:0/2155991117 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fe998065050 con 0x55757ff49590 2024-09-10T09:46:06.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:06.643+0000 7fe9b964f740 10 client.16950 fetch_fsmap learned FSMap version 34 2024-09-10T09:46:06.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:06.643+0000 7fe9b964f740 1 -- 172.21.15.5:0/2155991117 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55757ffce9d0 con 0x55757ff49590 2024-09-10T09:46:06.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:06.643+0000 7fe99d600640 1 -- 172.21.15.5:0/2155991117 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe9a8020400 con 0x55757ff49590 2024-09-10T09:46:06.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:06.643+0000 7fe99d600640 1 -- 172.21.15.5:0/2155991117 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7fe99805b030 con 0x55757ff49590 2024-09-10T09:46:06.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:06.643+0000 7fe9b964f740 10 client.16950 fetch_fsmap finished waiting for FSMap version 34 2024-09-10T09:46:06.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:06.643+0000 7fe9b964f740 10 client.16950 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T09:46:06.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:06.643+0000 7fe9b964f740 20 client.16950 populate_metadata read hostname 'smithi005' 2024-09-10T09:46:06.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:06.643+0000 7fe9b964f740 1 --2- 172.21.15.5:0/2155991117 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55757ffcf120 0x55757ffd1540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:46:06.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:06.643+0000 7fe9b964f740 4 client.16950 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T09:46:06.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:06.643+0000 7fe9b964f740 1 -- 172.21.15.5:0/2155991117 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55757feb6e60 con 0x55757ffcf120 2024-09-10T09:46:06.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:06.644+0000 7fe9afe00640 1 --2- 172.21.15.5:0/2155991117 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55757ffcf120 0x55757ffd1540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:46:06.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:06.644+0000 7fe9afe00640 1 --2- 172.21.15.5:0/2155991117 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55757ffcf120 0x55757ffd1540 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=abe6e9745a38d6b2 server_cookie=ad711394ffb7a72b in_seq=0 out_seq=0 2024-09-10T09:46:06.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:06.644+0000 7fe99d600640 10 client.16950 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T09:46:06.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:06.645+0000 7fe99d600640 1 -- 172.21.15.5:0/2155991117 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55757feb6e60 con 0x55757ffcf120 2024-09-10T09:46:06.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:06.645+0000 7fe99d600640 10 client.16950 handle_command_reply: tid=0 2024-09-10T09:46:06.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:06.645+0000 7fe9b964f740 1 client.16950 shutdown 2024-09-10T09:46:06.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:06.646+0000 7fe9b964f740 1 -- 172.21.15.5:0/2155991117 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55757ff49590 msgr2=0x55757ff580e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:46:06.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:06.646+0000 7fe9b964f740 1 --2- 172.21.15.5:0/2155991117 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55757ff49590 0x55757ff580e0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fe998009900 tx=0x7fe998004430 comp rx=0 tx=0).stop 2024-09-10T09:46:06.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:06.646+0000 7fe9b964f740 1 -- 172.21.15.5:0/2155991117 shutdown_connections 2024-09-10T09:46:06.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:06.646+0000 7fe9b964f740 1 --2- 172.21.15.5:0/2155991117 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55757ffcf120 0x55757ffd1540 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:06.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:06.646+0000 7fe9b964f740 1 --2- 172.21.15.5:0/2155991117 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55757ff53e50 0x55757ff58620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:06.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:06.646+0000 7fe9b964f740 1 --2- 172.21.15.5:0/2155991117 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55757ff49590 0x55757ff580e0 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:06.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:06.646+0000 7fe9b964f740 1 --2- 172.21.15.5:0/2155991117 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55757ff48c70 0x55757ff57ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:06.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:06.646+0000 7fe9b964f740 1 -- 172.21.15.5:0/2155991117 >> 172.21.15.5:0/2155991117 conn(0x55757ff45200 msgr2=0x55757ff44380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:46:06.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:06.646+0000 7fe99d600640 0 client.16950 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T09:46:06.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:06.646+0000 7fe9b964f740 1 -- 172.21.15.5:0/2155991117 shutdown_connections 2024-09-10T09:46:06.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:06.646+0000 7fe9b964f740 1 -- 172.21.15.5:0/2155991117 wait complete. 2024-09-10T09:46:06.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:06.646+0000 7fe9b964f740 20 client.16950 trim_cache size 0 max 16384 2024-09-10T09:46:06.650 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-10T09:46:06.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:06.647+0000 7fe9b6a00640 1 -- 172.21.15.5:0/647437127 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fe9940a2b80 msgr2=0x7fe9940a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:46:06.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:06.647+0000 7fe9b6a00640 1 --2- 172.21.15.5:0/647437127 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fe9940a2b80 0x7fe9940a5040 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fe9a8013980 tx=0x7fe9a8015040 comp rx=0 tx=0).stop 2024-09-10T09:46:06.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:06.647+0000 7fe9b6a00640 1 -- 172.21.15.5:0/647437127 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe9b0076720 msgr2=0x7fe9b01a47b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:46:06.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:06.647+0000 7fe9b6a00640 1 --2- 172.21.15.5:0/647437127 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe9b0076720 0x7fe9b01a47b0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fe9a000ea60 tx=0x7fe9a000ef20 comp rx=0 tx=0).stop 2024-09-10T09:46:06.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:06.647+0000 7fe9b6a00640 1 -- 172.21.15.5:0/647437127 shutdown_connections 2024-09-10T09:46:06.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:06.647+0000 7fe9b6a00640 1 --2- 172.21.15.5:0/647437127 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fe9940a2b80 0x7fe9940a5040 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:06.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:06.647+0000 7fe9b6a00640 1 --2- 172.21.15.5:0/647437127 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe9b0112a50 0x7fe9b019e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:06.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:06.647+0000 7fe9b6a00640 1 --2- 172.21.15.5:0/647437127 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe9b0077040 0x7fe9b01a4cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:06.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:06.647+0000 7fe9b6a00640 1 --2- 172.21.15.5:0/647437127 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe9b0076720 0x7fe9b01a47b0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:06.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:06.647+0000 7fe9b6a00640 1 -- 172.21.15.5:0/647437127 >> 172.21.15.5:0/647437127 conn(0x7fe9b00ffa50 msgr2=0x7fe9b0101e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:46:06.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:06.647+0000 7fe9b6a00640 1 -- 172.21.15.5:0/647437127 shutdown_connections 2024-09-10T09:46:06.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:06.647+0000 7fe9b6a00640 1 -- 172.21.15.5:0/647437127 wait complete. 2024-09-10T09:46:07.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:46:06 smithi005 ceph-mon[30350]: pgmap v626: 129 pgs: 129 active+clean; 182 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.3 MiB/s wr, 193 op/s 2024-09-10T09:46:07.062 INFO:tasks.workunit.client.0.smithi005.stdout: 1 449968 6.09 MB/sec execute 228 sec latency 20.693 ms 2024-09-10T09:46:07.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:46:06 smithi159 ceph-mon[35513]: pgmap v626: 129 pgs: 129 active+clean; 182 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.3 MiB/s wr, 193 op/s 2024-09-10T09:46:07.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:46:06 smithi102 ceph-mon[35442]: pgmap v626: 129 pgs: 129 active+clean; 182 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.3 MiB/s wr, 193 op/s 2024-09-10T09:46:08.063 INFO:tasks.workunit.client.0.smithi005.stdout: 1 450554 6.09 MB/sec execute 229 sec latency 819.766 ms 2024-09-10T09:46:08.632 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-10T09:46:08.633 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-10T09:46:09.027 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:46:09.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:46:08 smithi005 ceph-mon[30350]: pgmap v627: 129 pgs: 129 active+clean; 182 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.5 MiB/s wr, 181 op/s 2024-09-10T09:46:09.062 INFO:tasks.workunit.client.0.smithi005.stdout: 1 450587 6.06 MB/sec execute 230 sec latency 1779.855 ms 2024-09-10T09:46:09.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:46:08 smithi159 ceph-mon[35513]: pgmap v627: 129 pgs: 129 active+clean; 182 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.5 MiB/s wr, 181 op/s 2024-09-10T09:46:09.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:46:08 smithi102 ceph-mon[35442]: pgmap v627: 129 pgs: 129 active+clean; 182 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.5 MiB/s wr, 181 op/s 2024-09-10T09:46:09.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:09.639+0000 7f1cbdc00640 1 -- 172.21.15.5:0/1502094466 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1cb8102370 msgr2=0x7f1cb810e8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:46:09.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:09.639+0000 7f1cbdc00640 1 --2- 172.21.15.5:0/1502094466 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1cb8102370 0x7f1cb810e8a0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f1cb0009cd0 tx=0x7f1cb002a8f0 comp rx=0 tx=0).stop 2024-09-10T09:46:09.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:09.639+0000 7f1cbdc00640 1 -- 172.21.15.5:0/1502094466 shutdown_connections 2024-09-10T09:46:09.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:09.639+0000 7f1cbdc00640 1 --2- 172.21.15.5:0/1502094466 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1cb8102370 0x7f1cb810e8a0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:09.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:09.639+0000 7f1cbdc00640 1 --2- 172.21.15.5:0/1502094466 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1cb81019d0 0x7f1cb8101e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:09.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:09.639+0000 7f1cbdc00640 1 --2- 172.21.15.5:0/1502094466 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1cb81079d0 0x7f1cb8107db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:09.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:09.639+0000 7f1cbdc00640 1 -- 172.21.15.5:0/1502094466 >> 172.21.15.5:0/1502094466 conn(0x7f1cb80fd710 msgr2=0x7f1cb80ffb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:46:09.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:09.639+0000 7f1cbdc00640 1 -- 172.21.15.5:0/1502094466 shutdown_connections 2024-09-10T09:46:09.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:09.639+0000 7f1cbdc00640 1 -- 172.21.15.5:0/1502094466 wait complete. 2024-09-10T09:46:09.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:09.640+0000 7f1cbdc00640 1 Processor -- start 2024-09-10T09:46:09.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:09.640+0000 7f1cbdc00640 1 -- start start 2024-09-10T09:46:09.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:09.641+0000 7f1cbdc00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1cb81019d0 0x7f1cb8196040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:46:09.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:09.641+0000 7f1cbdc00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1cb8102370 0x7f1cb8196580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:46:09.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:09.641+0000 7f1cbdc00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1cb81079d0 0x7f1cb819a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:46:09.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:09.641+0000 7f1cbdc00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1cb810fc10 con 0x7f1cb81079d0 2024-09-10T09:46:09.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:09.641+0000 7f1cbdc00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f1cb810fa90 con 0x7f1cb8102370 2024-09-10T09:46:09.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:09.641+0000 7f1cbdc00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f1cb810fd90 con 0x7f1cb81019d0 2024-09-10T09:46:09.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:09.641+0000 7f1cb7400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1cb8102370 0x7f1cb8196580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:46:09.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:09.641+0000 7f1cb7400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1cb8102370 0x7f1cb8196580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:36904/0 (socket says 172.21.15.5:36904) 2024-09-10T09:46:09.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:09.641+0000 7f1cb7400640 1 -- 172.21.15.5:0/1321188971 learned_addr learned my addr 172.21.15.5:0/1321188971 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:46:09.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:09.641+0000 7f1cb7e00640 1 --2- 172.21.15.5:0/1321188971 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1cb81019d0 0x7f1cb8196040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:46:09.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:09.641+0000 7f1cbd200640 1 --2- 172.21.15.5:0/1321188971 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1cb81079d0 0x7f1cb819a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:46:09.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:09.642+0000 7f1cb7400640 1 -- 172.21.15.5:0/1321188971 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1cb81019d0 msgr2=0x7f1cb8196040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:46:09.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:09.642+0000 7f1cb7400640 1 --2- 172.21.15.5:0/1321188971 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1cb81019d0 0x7f1cb8196040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:09.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:09.642+0000 7f1cb7400640 1 -- 172.21.15.5:0/1321188971 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1cb81079d0 msgr2=0x7f1cb819a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:46:09.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:09.642+0000 7f1cb7400640 1 --2- 172.21.15.5:0/1321188971 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1cb81079d0 0x7f1cb819a910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:09.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:09.642+0000 7f1cb7400640 1 -- 172.21.15.5:0/1321188971 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1cb819aff0 con 0x7f1cb8102370 2024-09-10T09:46:09.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:09.642+0000 7f1cb7400640 1 --2- 172.21.15.5:0/1321188971 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1cb8102370 0x7f1cb8196580 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f1ca4009be0 tx=0x7f1ca40071c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:46:09.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:09.642+0000 7f1cb4c00640 1 -- 172.21.15.5:0/1321188971 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ca4007e00 con 0x7f1cb8102370 2024-09-10T09:46:09.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:09.642+0000 7f1cb4c00640 1 -- 172.21.15.5:0/1321188971 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1ca401e070 con 0x7f1cb8102370 2024-09-10T09:46:09.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:09.642+0000 7f1cb4c00640 1 -- 172.21.15.5:0/1321188971 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ca4018890 con 0x7f1cb8102370 2024-09-10T09:46:09.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:09.643+0000 7f1cbdc00640 1 -- 172.21.15.5:0/1321188971 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1cb819b2e0 con 0x7f1cb8102370 2024-09-10T09:46:09.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:09.643+0000 7f1cbdc00640 1 -- 172.21.15.5:0/1321188971 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1cb81a2bc0 con 0x7f1cb8102370 2024-09-10T09:46:09.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:09.646+0000 7f1cc1e30740 1 Processor -- start 2024-09-10T09:46:09.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:09.647+0000 7f1cc1e30740 1 -- start start 2024-09-10T09:46:09.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:09.647+0000 7f1cc1e30740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c7332bbcb0 0x55c7332bc090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:46:09.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:09.647+0000 7f1cc1e30740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c7332bc5d0 0x55c7332c6950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:46:09.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:09.647+0000 7f1cc1e30740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c7332c6e90 0x55c7332c9280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:46:09.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:09.647+0000 7f1cc1e30740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55c733222290 con 0x55c7332bc5d0 2024-09-10T09:46:09.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:09.647+0000 7f1cc1e30740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55c7331c9b40 con 0x55c7332c6e90 2024-09-10T09:46:09.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:09.647+0000 7f1cc1e30740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55c73321e250 con 0x55c7332bbcb0 2024-09-10T09:46:09.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:09.647+0000 7f1cb4c00640 1 -- 172.21.15.5:0/1321188971 <== mon.2 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f1ca4019b60 con 0x7f1cb8102370 2024-09-10T09:46:09.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:09.647+0000 7f1cbd200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c7332bbcb0 0x55c7332bc090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:46:09.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:09.648+0000 7f1cbd200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c7332bbcb0 0x55c7332bc090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:34048/0 (socket says 172.21.15.5:34048) 2024-09-10T09:46:09.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:09.648+0000 7f1cbd200640 1 -- 172.21.15.5:0/1129423181 learned_addr learned my addr 172.21.15.5:0/1129423181 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:46:09.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:09.648+0000 7f1cb7e00640 1 --2- 172.21.15.5:0/1129423181 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c7332bc5d0 0x55c7332c6950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:46:09.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:09.648+0000 7f1cbd200640 1 -- 172.21.15.5:0/1129423181 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c7332c6e90 msgr2=0x55c7332c9280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:46:09.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:09.648+0000 7f1cb7400640 1 --2- 172.21.15.5:0/1129423181 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c7332c6e90 0x55c7332c9280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:46:09.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:09.648+0000 7f1cbd200640 1 --2- 172.21.15.5:0/1129423181 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c7332c6e90 0x55c7332c9280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:09.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:09.648+0000 7f1cbd200640 1 -- 172.21.15.5:0/1129423181 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c7332bc5d0 msgr2=0x55c7332c6950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:46:09.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:09.648+0000 7f1cbd200640 1 --2- 172.21.15.5:0/1129423181 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c7332bc5d0 0x55c7332c6950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:09.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:09.648+0000 7f1cbd200640 1 -- 172.21.15.5:0/1129423181 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c732f936a0 con 0x55c7332bbcb0 2024-09-10T09:46:09.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:09.648+0000 7f1cb7400640 1 --2- 172.21.15.5:0/1129423181 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c7332c6e90 0x55c7332c9280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T09:46:09.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:09.648+0000 7f1cb7e00640 1 --2- 172.21.15.5:0/1129423181 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c7332bc5d0 0x55c7332c6950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:46:09.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:09.649+0000 7f1cbd200640 1 --2- 172.21.15.5:0/1129423181 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c7332bbcb0 0x55c7332bc090 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f1cb0009cd0 tx=0x7f1cb0005f80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dd16214d3b73bc2c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:46:09.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:09.649+0000 7f1cad600640 1 -- 172.21.15.5:0/1129423181 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cb0030760 con 0x55c7332bbcb0 2024-09-10T09:46:09.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:09.649+0000 7f1cad600640 1 -- 172.21.15.5:0/1129423181 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1cb0036030 con 0x55c7332bbcb0 2024-09-10T09:46:09.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:09.649+0000 7f1cad600640 1 -- 172.21.15.5:0/1129423181 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cb00043a0 con 0x55c7332bbcb0 2024-09-10T09:46:09.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:09.650+0000 7f1cb4c00640 1 --2- 172.21.15.5:0/1321188971 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f1c9c0a2c80 0x7f1c9c0a5140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:46:09.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:09.650+0000 7f1cb4c00640 1 -- 172.21.15.5:0/1321188971 <== mon.2 v2:172.21.15.102:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f1ca40d28b0 con 0x7f1cb8102370 2024-09-10T09:46:09.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:09.650+0000 7f1cc1e30740 1 -- 172.21.15.5:0/1129423181 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c7332bbcb0 msgr2=0x55c7332bc090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:46:09.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:09.650+0000 7f1cc1e30740 1 --2- 172.21.15.5:0/1129423181 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c7332bbcb0 0x55c7332bc090 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f1cb0009cd0 tx=0x7f1cb0005f80 comp rx=0 tx=0).stop 2024-09-10T09:46:09.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:09.650+0000 7f1cc1e30740 1 -- 172.21.15.5:0/1129423181 shutdown_connections 2024-09-10T09:46:09.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:09.650+0000 7f1cc1e30740 1 --2- 172.21.15.5:0/1129423181 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c7332c6e90 0x55c7332c9280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:09.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:09.650+0000 7f1cc1e30740 1 --2- 172.21.15.5:0/1129423181 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c7332bc5d0 0x55c7332c6950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:09.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:09.650+0000 7f1cc1e30740 1 --2- 172.21.15.5:0/1129423181 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c7332bbcb0 0x55c7332bc090 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:09.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:09.650+0000 7f1cc1e30740 1 -- 172.21.15.5:0/1129423181 >> 172.21.15.5:0/1129423181 conn(0x55c7332b8240 msgr2=0x55c7332b8620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:46:09.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:09.651+0000 7f1cb7e00640 1 --2- 172.21.15.5:0/1321188971 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f1c9c0a2c80 0x7f1c9c0a5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:46:09.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:09.651+0000 7f1cc1e30740 1 -- 172.21.15.5:0/1129423181 shutdown_connections 2024-09-10T09:46:09.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:09.651+0000 7f1cc1e30740 1 -- 172.21.15.5:0/1129423181 wait complete. 2024-09-10T09:46:09.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:09.652+0000 7f1cc1e30740 1 -- start start 2024-09-10T09:46:09.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:09.652+0000 7f1cb7e00640 1 --2- 172.21.15.5:0/1321188971 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f1c9c0a2c80 0x7f1c9c0a5140 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x55c7332b90a0 tx=0x7f1ca8005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:46:09.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:09.652+0000 7f1cc1e30740 1 Processor -- start 2024-09-10T09:46:09.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:09.652+0000 7f1cc1e30740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c7332bbcb0 0x55c7332c6730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:46:09.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:09.652+0000 7f1cc1e30740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c7332bc5d0 0x55c7332c39e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:46:09.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:09.652+0000 7f1cc1e30740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c7332c6e90 0x55c7332c3f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:46:09.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:09.652+0000 7f1cc1e30740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55c7332c9b00 con 0x55c7332bbcb0 2024-09-10T09:46:09.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:09.652+0000 7f1cc1e30740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55c7331dcfa0 con 0x55c7332c6e90 2024-09-10T09:46:09.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:09.652+0000 7f1cc1e30740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55c7332c9c80 con 0x55c7332bc5d0 2024-09-10T09:46:09.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:09.652+0000 7f1cc1e30740 10 client.0 _ll_register_callbacks cb 0x55c732f30ba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:46:09.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:09.653+0000 7f1cbd200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c7332c6e90 0x55c7332c3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:46:09.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:09.653+0000 7f1cb7400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c7332bc5d0 0x55c7332c39e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:46:09.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:09.653+0000 7f1cbd200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c7332c6e90 0x55c7332c3f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:36920/0 (socket says 172.21.15.5:36920) 2024-09-10T09:46:09.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:09.653+0000 7f1cbd200640 1 -- 172.21.15.5:0/86784975 learned_addr learned my addr 172.21.15.5:0/86784975 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:46:09.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:09.653+0000 7f1cb7e00640 1 --2- 172.21.15.5:0/86784975 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c7332bbcb0 0x55c7332c6730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:46:09.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:09.653+0000 7f1cb7400640 1 -- 172.21.15.5:0/86784975 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c7332c6e90 msgr2=0x55c7332c3f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:46:09.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:09.653+0000 7f1cb7400640 1 --2- 172.21.15.5:0/86784975 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c7332c6e90 0x55c7332c3f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:09.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:09.653+0000 7f1cb7400640 1 -- 172.21.15.5:0/86784975 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c7332bbcb0 msgr2=0x55c7332c6730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:46:09.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:09.653+0000 7f1cb7400640 1 --2- 172.21.15.5:0/86784975 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c7332bbcb0 0x55c7332c6730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:09.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:09.653+0000 7f1cb7400640 1 -- 172.21.15.5:0/86784975 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c7332c4720 con 0x55c7332bc5d0 2024-09-10T09:46:09.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:09.653+0000 7f1cbd200640 1 --2- 172.21.15.5:0/86784975 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c7332c6e90 0x55c7332c3f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:46:09.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:09.653+0000 7f1ca2000640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-09-10T09:46:09.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:09.653+0000 7f1cb7400640 1 --2- 172.21.15.5:0/86784975 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c7332bc5d0 0x55c7332c39e0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x55c7332b95c0 tx=0x7f1ca409e710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=484a7bce02e3844c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:46:09.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:09.654+0000 7f1ca2000640 1 -- 172.21.15.5:0/86784975 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ca409e310 con 0x55c7332bc5d0 2024-09-10T09:46:09.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:09.654+0000 7f1cc1e30740 1 -- 172.21.15.5:0/86784975 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1cb0036030 con 0x55c7332bc5d0 2024-09-10T09:46:09.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:09.654+0000 7f1ca2000640 1 -- 172.21.15.5:0/86784975 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f1ca409aa10 con 0x55c7332bc5d0 2024-09-10T09:46:09.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:09.654+0000 7f1ca2000640 1 -- 172.21.15.5:0/86784975 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f1ca40a7930 con 0x55c7332bc5d0 2024-09-10T09:46:09.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:09.654+0000 7f1cc1e30740 10 client.26986 fetch_fsmap learned FSMap version 34 2024-09-10T09:46:09.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:09.654+0000 7f1cc1e30740 1 -- 172.21.15.5:0/86784975 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55c7332d5970 con 0x55c7332bc5d0 2024-09-10T09:46:09.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:09.655+0000 7f1ca2000640 1 -- 172.21.15.5:0/86784975 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1cb0036030 con 0x55c7332bc5d0 2024-09-10T09:46:09.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:09.655+0000 7f1ca2000640 1 -- 172.21.15.5:0/86784975 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f1ca40bb020 con 0x55c7332bc5d0 2024-09-10T09:46:09.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:09.655+0000 7f1cc1e30740 10 client.26986 fetch_fsmap finished waiting for FSMap version 34 2024-09-10T09:46:09.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:09.655+0000 7f1cc1e30740 10 client.26986 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T09:46:09.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:09.655+0000 7f1cc1e30740 20 client.26986 populate_metadata read hostname 'smithi005' 2024-09-10T09:46:09.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:09.655+0000 7f1cc1e30740 1 --2- 172.21.15.5:0/86784975 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55c7332d6e30 0x55c7332d92c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:46:09.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:09.655+0000 7f1cc1e30740 4 client.26986 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:46:09.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:09.655+0000 7f1cc1e30740 1 -- 172.21.15.5:0/86784975 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c73322f800 con 0x55c7332d6e30 2024-09-10T09:46:09.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:09.655+0000 7f1cb7e00640 1 --2- 172.21.15.5:0/86784975 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55c7332d6e30 0x55c7332d92c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:46:09.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:09.656+0000 7f1ca2000640 10 client.26986 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T09:46:09.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:09.656+0000 7f1cb7e00640 1 --2- 172.21.15.5:0/86784975 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55c7332d6e30 0x55c7332d92c0 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9ebd8ca8e1c95dd server_cookie=e371193d9e9f7fb6 in_seq=0 out_seq=0 2024-09-10T09:46:09.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:09.657+0000 7f1ca2000640 1 -- 172.21.15.5:0/86784975 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c73322f800 con 0x55c7332d6e30 2024-09-10T09:46:09.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:09.657+0000 7f1ca2000640 10 client.26986 handle_command_reply: tid=0 2024-09-10T09:46:09.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:09.657+0000 7f1cc1e30740 1 client.26986 shutdown 2024-09-10T09:46:09.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:09.657+0000 7f1cc1e30740 1 -- 172.21.15.5:0/86784975 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c7332bc5d0 msgr2=0x55c7332c39e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:46:09.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:09.657+0000 7f1cc1e30740 1 --2- 172.21.15.5:0/86784975 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c7332bc5d0 0x55c7332c39e0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x55c7332b95c0 tx=0x7f1ca409e710 comp rx=0 tx=0).stop 2024-09-10T09:46:09.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:09.658+0000 7f1cc1e30740 1 -- 172.21.15.5:0/86784975 shutdown_connections 2024-09-10T09:46:09.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:09.658+0000 7f1cc1e30740 1 --2- 172.21.15.5:0/86784975 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55c7332d6e30 0x55c7332d92c0 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:09.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:09.658+0000 7f1cc1e30740 1 --2- 172.21.15.5:0/86784975 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c7332c6e90 0x55c7332c3f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:09.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:09.658+0000 7f1cc1e30740 1 --2- 172.21.15.5:0/86784975 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c7332bc5d0 0x55c7332c39e0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:09.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:09.658+0000 7f1cc1e30740 1 --2- 172.21.15.5:0/86784975 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c7332bbcb0 0x55c7332c6730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:09.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:09.658+0000 7f1ca2000640 0 client.26986 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T09:46:09.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:09.658+0000 7f1cc1e30740 1 -- 172.21.15.5:0/86784975 >> 172.21.15.5:0/86784975 conn(0x55c7332b8240 msgr2=0x55c7332bdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:46:09.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:09.658+0000 7f1cc1e30740 1 -- 172.21.15.5:0/86784975 shutdown_connections 2024-09-10T09:46:09.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:09.658+0000 7f1cc1e30740 1 -- 172.21.15.5:0/86784975 wait complete. 2024-09-10T09:46:09.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:09.658+0000 7f1cc1e30740 20 client.26986 trim_cache size 0 max 16384 2024-09-10T09:46:09.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:09.678+0000 7f1cc1e30740 1 Processor -- start 2024-09-10T09:46:09.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:09.678+0000 7f1cc1e30740 1 -- start start 2024-09-10T09:46:09.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:09.678+0000 7f1cc1e30740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c7332bbcb0 0x55c7332b7180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:46:09.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:09.678+0000 7f1cc1e30740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c7332bc5d0 0x55c7332deb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:46:09.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:09.678+0000 7f1cc1e30740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c7332c6e90 0x55c7332df0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:46:09.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:09.679+0000 7f1cc1e30740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1cb81acaf0 con 0x55c7332c6e90 2024-09-10T09:46:09.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:09.679+0000 7f1cc1e30740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f1cb81ae630 con 0x55c7332bbcb0 2024-09-10T09:46:09.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:09.679+0000 7f1cc1e30740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f1cb81ad1c0 con 0x55c7332bc5d0 2024-09-10T09:46:09.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:09.679+0000 7f1cb7e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c7332bbcb0 0x55c7332b7180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:46:09.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:09.679+0000 7f1cb7400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c7332bc5d0 0x55c7332deb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:46:09.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:09.679+0000 7f1cb7e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c7332bbcb0 0x55c7332b7180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:36922/0 (socket says 172.21.15.5:36922) 2024-09-10T09:46:09.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:09.679+0000 7f1cb7e00640 1 -- 172.21.15.5:0/3045321641 learned_addr learned my addr 172.21.15.5:0/3045321641 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:46:09.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:09.679+0000 7f1cbd200640 1 --2- 172.21.15.5:0/3045321641 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c7332c6e90 0x55c7332df0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:46:09.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:09.679+0000 7f1cb7e00640 1 -- 172.21.15.5:0/3045321641 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c7332bc5d0 msgr2=0x55c7332deb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:46:09.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:09.679+0000 7f1cb7e00640 1 --2- 172.21.15.5:0/3045321641 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c7332bc5d0 0x55c7332deb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:09.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:09.679+0000 7f1cb7e00640 1 -- 172.21.15.5:0/3045321641 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c7332c6e90 msgr2=0x55c7332df0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:46:09.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:09.679+0000 7f1cb7e00640 1 --2- 172.21.15.5:0/3045321641 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c7332c6e90 0x55c7332df0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:09.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:09.679+0000 7f1cb7e00640 1 -- 172.21.15.5:0/3045321641 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c7332b92f0 con 0x55c7332bbcb0 2024-09-10T09:46:09.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:09.679+0000 7f1cb7e00640 1 --2- 172.21.15.5:0/3045321641 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c7332bbcb0 0x55c7332b7180 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f1ca8012ec0 tx=0x7f1ca8019000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=61cc5c04d82a935c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:46:09.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:09.680+0000 7f1cacc00640 1 -- 172.21.15.5:0/3045321641 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ca80036b0 con 0x55c7332bbcb0 2024-09-10T09:46:09.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:09.680+0000 7f1cacc00640 1 -- 172.21.15.5:0/3045321641 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1ca8003d30 con 0x55c7332bbcb0 2024-09-10T09:46:09.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:09.680+0000 7f1cacc00640 1 -- 172.21.15.5:0/3045321641 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ca802ac80 con 0x55c7332bbcb0 2024-09-10T09:46:09.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:09.681+0000 7f1cc1e30740 1 -- 172.21.15.5:0/3045321641 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c7332bbcb0 msgr2=0x55c7332b7180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:46:09.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:09.681+0000 7f1cc1e30740 1 --2- 172.21.15.5:0/3045321641 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c7332bbcb0 0x55c7332b7180 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f1ca8012ec0 tx=0x7f1ca8019000 comp rx=0 tx=0).stop 2024-09-10T09:46:09.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:09.681+0000 7f1cc1e30740 1 -- 172.21.15.5:0/3045321641 shutdown_connections 2024-09-10T09:46:09.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:09.681+0000 7f1cc1e30740 1 --2- 172.21.15.5:0/3045321641 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c7332c6e90 0x55c7332df0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:09.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:09.681+0000 7f1cc1e30740 1 --2- 172.21.15.5:0/3045321641 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c7332bc5d0 0x55c7332deb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:09.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:09.681+0000 7f1cc1e30740 1 --2- 172.21.15.5:0/3045321641 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c7332bbcb0 0x55c7332b7180 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:09.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:09.681+0000 7f1cc1e30740 1 -- 172.21.15.5:0/3045321641 >> 172.21.15.5:0/3045321641 conn(0x55c7332b8240 msgr2=0x55c7332b6a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:46:09.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:09.681+0000 7f1cc1e30740 1 -- 172.21.15.5:0/3045321641 shutdown_connections 2024-09-10T09:46:09.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:09.681+0000 7f1cc1e30740 1 -- 172.21.15.5:0/3045321641 wait complete. 2024-09-10T09:46:09.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:09.681+0000 7f1cc1e30740 1 -- start start 2024-09-10T09:46:09.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:09.682+0000 7f1cc1e30740 1 Processor -- start 2024-09-10T09:46:09.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:09.682+0000 7f1cc1e30740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c7332bbcb0 0x55c7332cf720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:46:09.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:09.682+0000 7f1cc1e30740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c7332bc5d0 0x55c7332ccac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:46:09.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:09.682+0000 7f1cc1e30740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c7332c6e90 0x55c7332cd060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:46:09.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:09.682+0000 7f1cc1e30740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1cb81adf60 con 0x55c7332c6e90 2024-09-10T09:46:09.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:09.682+0000 7f1cc1e30740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f1cb81ad890 con 0x55c7332bbcb0 2024-09-10T09:46:09.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:09.682+0000 7f1cc1e30740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f1cb81aaad0 con 0x55c7332bc5d0 2024-09-10T09:46:09.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:09.682+0000 7f1cc1e30740 10 client.0 _ll_register_callbacks cb 0x7f1ca409e890 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:46:09.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:09.682+0000 7f1cb7400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c7332bc5d0 0x55c7332ccac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:46:09.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:09.682+0000 7f1cb7400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c7332bc5d0 0x55c7332ccac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:34072/0 (socket says 172.21.15.5:34072) 2024-09-10T09:46:09.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:09.682+0000 7f1cb7400640 1 -- 172.21.15.5:0/3105422543 learned_addr learned my addr 172.21.15.5:0/3105422543 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:46:09.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:09.682+0000 7f1cbd200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c7332c6e90 0x55c7332cd060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:46:09.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:09.682+0000 7f1cb7e00640 1 --2- 172.21.15.5:0/3105422543 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c7332bbcb0 0x55c7332cf720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:46:09.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:09.683+0000 7f1cb7400640 1 -- 172.21.15.5:0/3105422543 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c7332bbcb0 msgr2=0x55c7332cf720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:46:09.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:09.683+0000 7f1cb7400640 1 --2- 172.21.15.5:0/3105422543 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c7332bbcb0 0x55c7332cf720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:09.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:09.683+0000 7f1cb7400640 1 -- 172.21.15.5:0/3105422543 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c7332c6e90 msgr2=0x55c7332cd060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:46:09.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:09.683+0000 7f1cb7400640 1 --2- 172.21.15.5:0/3105422543 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c7332c6e90 0x55c7332cd060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:09.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:09.683+0000 7f1cb7400640 1 -- 172.21.15.5:0/3105422543 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c7332cd890 con 0x55c7332bc5d0 2024-09-10T09:46:09.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:09.683+0000 7f1cbd200640 1 --2- 172.21.15.5:0/3105422543 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c7332c6e90 0x55c7332cd060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:46:09.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:09.683+0000 7f1cb7e00640 1 --2- 172.21.15.5:0/3105422543 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c7332bbcb0 0x55c7332cf720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:46:09.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:09.683+0000 7f1cb7400640 1 --2- 172.21.15.5:0/3105422543 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c7332bc5d0 0x55c7332ccac0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f1ca409b690 tx=0x7f1ca4000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f55649da12bfb4ce server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:46:09.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:09.683+0000 7f1ca2000640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-09-10T09:46:09.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:09.684+0000 7f1ca2000640 1 -- 172.21.15.5:0/3105422543 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ca40bb040 con 0x55c7332bc5d0 2024-09-10T09:46:09.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:09.684+0000 7f1ca2000640 1 -- 172.21.15.5:0/3105422543 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f1ca407db30 con 0x55c7332bc5d0 2024-09-10T09:46:09.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:09.684+0000 7f1cc1e30740 1 -- 172.21.15.5:0/3105422543 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1ca8003d30 con 0x55c7332bc5d0 2024-09-10T09:46:09.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:09.684+0000 7f1ca2000640 1 -- 172.21.15.5:0/3105422543 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f1ca409ecc0 con 0x55c7332bc5d0 2024-09-10T09:46:09.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:09.684+0000 7f1cc1e30740 10 client.26992 fetch_fsmap learned FSMap version 34 2024-09-10T09:46:09.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:09.684+0000 7f1cc1e30740 1 -- 172.21.15.5:0/3105422543 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55c733341c80 con 0x55c7332bc5d0 2024-09-10T09:46:09.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:09.684+0000 7f1ca2000640 1 -- 172.21.15.5:0/3105422543 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1ca8003d30 con 0x55c7332bc5d0 2024-09-10T09:46:09.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:09.684+0000 7f1ca2000640 1 -- 172.21.15.5:0/3105422543 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f1ca40d2ce0 con 0x55c7332bc5d0 2024-09-10T09:46:09.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:09.684+0000 7f1cc1e30740 10 client.26992 fetch_fsmap finished waiting for FSMap version 34 2024-09-10T09:46:09.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:09.684+0000 7f1cc1e30740 10 client.26992 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T09:46:09.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:09.684+0000 7f1cc1e30740 20 client.26992 populate_metadata read hostname 'smithi005' 2024-09-10T09:46:09.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:09.684+0000 7f1cc1e30740 1 --2- 172.21.15.5:0/3105422543 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55c733342400 0x55c733344820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:46:09.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:09.684+0000 7f1cc1e30740 4 client.26992 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-10T09:46:09.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:09.684+0000 7f1cc1e30740 1 -- 172.21.15.5:0/3105422543 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55c733229d10 con 0x55c733342400 2024-09-10T09:46:09.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:09.685+0000 7f1cb7e00640 1 --2- 172.21.15.5:0/3105422543 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55c733342400 0x55c733344820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:46:09.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:09.685+0000 7f1cb7e00640 1 --2- 172.21.15.5:0/3105422543 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55c733342400 0x55c733344820 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ef99cc4f76058563 server_cookie=e2979f8b167bf9be in_seq=0 out_seq=0 2024-09-10T09:46:09.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:09.685+0000 7f1ca2000640 10 client.26992 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T09:46:09.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:09.686+0000 7f1ca2000640 1 -- 172.21.15.5:0/3105422543 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55c733229d10 con 0x55c733342400 2024-09-10T09:46:09.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:09.686+0000 7f1ca2000640 10 client.26992 handle_command_reply: tid=0 2024-09-10T09:46:09.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:09.686+0000 7f1cc1e30740 1 client.26992 shutdown 2024-09-10T09:46:09.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:09.686+0000 7f1cc1e30740 1 -- 172.21.15.5:0/3105422543 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c7332bc5d0 msgr2=0x55c7332ccac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:46:09.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:09.686+0000 7f1cc1e30740 1 --2- 172.21.15.5:0/3105422543 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c7332bc5d0 0x55c7332ccac0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f1ca409b690 tx=0x7f1ca4000bc0 comp rx=0 tx=0).stop 2024-09-10T09:46:09.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:09.687+0000 7f1cc1e30740 1 -- 172.21.15.5:0/3105422543 shutdown_connections 2024-09-10T09:46:09.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:09.687+0000 7f1cc1e30740 1 --2- 172.21.15.5:0/3105422543 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55c733342400 0x55c733344820 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:09.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:09.687+0000 7f1cc1e30740 1 --2- 172.21.15.5:0/3105422543 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c7332c6e90 0x55c7332cd060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:09.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:09.687+0000 7f1cc1e30740 1 --2- 172.21.15.5:0/3105422543 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c7332bc5d0 0x55c7332ccac0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:09.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:09.687+0000 7f1cc1e30740 1 --2- 172.21.15.5:0/3105422543 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c7332bbcb0 0x55c7332cf720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:09.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:09.687+0000 7f1cc1e30740 1 -- 172.21.15.5:0/3105422543 >> 172.21.15.5:0/3105422543 conn(0x55c7332b8240 msgr2=0x55c7332b7270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:46:09.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:09.687+0000 7f1ca2000640 0 client.26992 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T09:46:09.691 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-10T09:46:09.691 INFO:teuthology.orchestra.run.smithi005.stdout: "return_code": 0, 2024-09-10T09:46:09.691 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_tag": "dd9582e0-e8cd-405c-8c0c-6d0709a1a30c", 2024-09-10T09:46:09.691 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": "asynchronous" 2024-09-10T09:46:09.691 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-10T09:46:09.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:09.688+0000 7f1cc1e30740 1 -- 172.21.15.5:0/3105422543 shutdown_connections 2024-09-10T09:46:09.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:09.688+0000 7f1cc1e30740 1 -- 172.21.15.5:0/3105422543 wait complete. 2024-09-10T09:46:09.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:09.688+0000 7f1cc1e30740 20 client.26992 trim_cache size 0 max 16384 2024-09-10T09:46:09.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:09.689+0000 7f1cbdc00640 1 -- 172.21.15.5:0/1321188971 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f1c9c0a2c80 msgr2=0x7f1c9c0a5140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:46:09.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:09.689+0000 7f1cbdc00640 1 --2- 172.21.15.5:0/1321188971 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f1c9c0a2c80 0x7f1c9c0a5140 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x55c7332b90a0 tx=0x7f1ca8005f90 comp rx=0 tx=0).stop 2024-09-10T09:46:09.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:09.689+0000 7f1cbdc00640 1 -- 172.21.15.5:0/1321188971 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1cb8102370 msgr2=0x7f1cb8196580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:46:09.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:09.689+0000 7f1cbdc00640 1 --2- 172.21.15.5:0/1321188971 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1cb8102370 0x7f1cb8196580 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f1ca4009be0 tx=0x7f1ca40071c0 comp rx=0 tx=0).stop 2024-09-10T09:46:09.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:09.689+0000 7f1cbdc00640 1 -- 172.21.15.5:0/1321188971 shutdown_connections 2024-09-10T09:46:09.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:09.689+0000 7f1cbdc00640 1 --2- 172.21.15.5:0/1321188971 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f1c9c0a2c80 0x7f1c9c0a5140 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:09.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:09.689+0000 7f1cbdc00640 1 --2- 172.21.15.5:0/1321188971 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1cb81079d0 0x7f1cb819a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:09.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:09.689+0000 7f1cbdc00640 1 --2- 172.21.15.5:0/1321188971 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1cb8102370 0x7f1cb8196580 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:09.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:09.689+0000 7f1cbdc00640 1 --2- 172.21.15.5:0/1321188971 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1cb81019d0 0x7f1cb8196040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:09.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:09.689+0000 7f1cbdc00640 1 -- 172.21.15.5:0/1321188971 >> 172.21.15.5:0/1321188971 conn(0x7f1cb80fd710 msgr2=0x7f1cb80fdd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:46:09.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:09.689+0000 7f1cbdc00640 1 -- 172.21.15.5:0/1321188971 shutdown_connections 2024-09-10T09:46:09.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:09.689+0000 7f1cbdc00640 1 -- 172.21.15.5:0/1321188971 wait complete. 2024-09-10T09:46:10.063 INFO:tasks.workunit.client.0.smithi005.stdout: 1 452181 6.06 MB/sec execute 231 sec latency 19.047 ms 2024-09-10T09:46:10.576 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T09:46:10.964 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:46:11.063 INFO:tasks.workunit.client.0.smithi005.stdout: 1 454620 6.10 MB/sec execute 232 sec latency 10.269 ms 2024-09-10T09:46:11.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:46:10 smithi159 ceph-mon[35513]: scrub queued for path: / 2024-09-10T09:46:11.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:46:10 smithi159 ceph-mon[35513]: scrub summary: idle+waiting paths [/] 2024-09-10T09:46:11.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:46:10 smithi159 ceph-mon[35513]: scrub summary: active paths [/] 2024-09-10T09:46:11.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:46:10 smithi159 ceph-mon[35513]: scrub summary: active paths [/] 2024-09-10T09:46:11.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:46:10 smithi159 ceph-mon[35513]: scrub summary: idle+waiting paths [/] 2024-09-10T09:46:11.101 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:46:10 smithi159 ceph-mon[35513]: scrub summary: idle+waiting paths [/] 2024-09-10T09:46:11.101 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:46:10 smithi159 ceph-mon[35513]: pgmap v628: 129 pgs: 129 active+clean; 182 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.7 MiB/s wr, 205 op/s 2024-09-10T09:46:11.103 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:46:10 smithi005 ceph-mon[30350]: scrub queued for path: / 2024-09-10T09:46:11.103 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:46:10 smithi005 ceph-mon[30350]: scrub summary: idle+waiting paths [/] 2024-09-10T09:46:11.103 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:46:10 smithi005 ceph-mon[30350]: scrub summary: active paths [/] 2024-09-10T09:46:11.103 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:46:10 smithi005 ceph-mon[30350]: scrub summary: active paths [/] 2024-09-10T09:46:11.103 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:46:10 smithi005 ceph-mon[30350]: scrub summary: idle+waiting paths [/] 2024-09-10T09:46:11.103 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:46:10 smithi005 ceph-mon[30350]: scrub summary: idle+waiting paths [/] 2024-09-10T09:46:11.103 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:46:10 smithi005 ceph-mon[30350]: pgmap v628: 129 pgs: 129 active+clean; 182 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.7 MiB/s wr, 205 op/s 2024-09-10T09:46:11.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:46:10 smithi102 ceph-mon[35442]: scrub queued for path: / 2024-09-10T09:46:11.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:46:10 smithi102 ceph-mon[35442]: scrub summary: idle+waiting paths [/] 2024-09-10T09:46:11.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:46:10 smithi102 ceph-mon[35442]: scrub summary: active paths [/] 2024-09-10T09:46:11.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:46:10 smithi102 ceph-mon[35442]: scrub summary: active paths [/] 2024-09-10T09:46:11.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:46:10 smithi102 ceph-mon[35442]: scrub summary: idle+waiting paths [/] 2024-09-10T09:46:11.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:46:10 smithi102 ceph-mon[35442]: scrub summary: idle+waiting paths [/] 2024-09-10T09:46:11.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:46:10 smithi102 ceph-mon[35442]: pgmap v628: 129 pgs: 129 active+clean; 182 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.7 MiB/s wr, 205 op/s 2024-09-10T09:46:11.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:11.560+0000 7f50e1600640 1 -- 172.21.15.5:0/1445796276 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f50dc102360 msgr2=0x7f50dc10e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:46:11.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:11.560+0000 7f50e1600640 1 --2- 172.21.15.5:0/1445796276 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f50dc102360 0x7f50dc10e890 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f50d4009ca0 tx=0x7f50d402a8c0 comp rx=0 tx=0).stop 2024-09-10T09:46:11.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:11.560+0000 7f50e1600640 1 -- 172.21.15.5:0/1445796276 shutdown_connections 2024-09-10T09:46:11.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:11.560+0000 7f50e1600640 1 --2- 172.21.15.5:0/1445796276 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f50dc102360 0x7f50dc10e890 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:11.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:11.560+0000 7f50e1600640 1 --2- 172.21.15.5:0/1445796276 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f50dc1019c0 0x7f50dc101e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:11.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:11.560+0000 7f50e1600640 1 --2- 172.21.15.5:0/1445796276 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f50dc1079c0 0x7f50dc107da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:11.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:11.560+0000 7f50e1600640 1 -- 172.21.15.5:0/1445796276 >> 172.21.15.5:0/1445796276 conn(0x7f50dc0fd740 msgr2=0x7f50dc0ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:46:11.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:11.560+0000 7f50e1600640 1 -- 172.21.15.5:0/1445796276 shutdown_connections 2024-09-10T09:46:11.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:11.560+0000 7f50e1600640 1 -- 172.21.15.5:0/1445796276 wait complete. 2024-09-10T09:46:11.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:11.561+0000 7f50e1600640 1 Processor -- start 2024-09-10T09:46:11.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:11.561+0000 7f50e1600640 1 -- start start 2024-09-10T09:46:11.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:11.562+0000 7f50e1600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f50dc1019c0 0x7f50dc19a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:46:11.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:11.562+0000 7f50e1600640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f50dc102360 0x7f50dc19a970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:46:11.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:11.562+0000 7f50e1600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f50dc1079c0 0x7f50dc19ed00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:46:11.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:11.562+0000 7f50e1600640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f50dc111dc0 con 0x7f50dc1079c0 2024-09-10T09:46:11.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:11.562+0000 7f50e1600640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f50dc111c40 con 0x7f50dc1019c0 2024-09-10T09:46:11.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:11.562+0000 7f50e1600640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f50dc111f40 con 0x7f50dc102360 2024-09-10T09:46:11.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:11.562+0000 7f50dbe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f50dc1019c0 0x7f50dc19a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:46:11.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:11.562+0000 7f50dbe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f50dc1019c0 0x7f50dc19a430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:54302/0 (socket says 172.21.15.5:54302) 2024-09-10T09:46:11.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:11.562+0000 7f50dbe00640 1 -- 172.21.15.5:0/809353678 learned_addr learned my addr 172.21.15.5:0/809353678 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:46:11.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:11.562+0000 7f50db400640 1 --2- 172.21.15.5:0/809353678 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f50dc102360 0x7f50dc19a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:46:11.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:11.562+0000 7f50e0c00640 1 --2- 172.21.15.5:0/809353678 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f50dc1079c0 0x7f50dc19ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:46:11.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:11.562+0000 7f50dbe00640 1 -- 172.21.15.5:0/809353678 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f50dc102360 msgr2=0x7f50dc19a970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:46:11.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:11.562+0000 7f50dbe00640 1 --2- 172.21.15.5:0/809353678 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f50dc102360 0x7f50dc19a970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:11.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:11.562+0000 7f50dbe00640 1 -- 172.21.15.5:0/809353678 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f50dc1079c0 msgr2=0x7f50dc19ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:46:11.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:11.562+0000 7f50dbe00640 1 --2- 172.21.15.5:0/809353678 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f50dc1079c0 0x7f50dc19ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:11.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:11.562+0000 7f50dbe00640 1 -- 172.21.15.5:0/809353678 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50dc19f3e0 con 0x7f50dc1019c0 2024-09-10T09:46:11.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:11.563+0000 7f50dbe00640 1 --2- 172.21.15.5:0/809353678 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f50dc1019c0 0x7f50dc19a430 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f50cc00e9f0 tx=0x7f50cc00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:46:11.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:11.563+0000 7f50d8c00640 1 -- 172.21.15.5:0/809353678 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50cc00cd80 con 0x7f50dc1019c0 2024-09-10T09:46:11.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:11.563+0000 7f50d8c00640 1 -- 172.21.15.5:0/809353678 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f50cc010910 con 0x7f50dc1019c0 2024-09-10T09:46:11.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:11.563+0000 7f50e1600640 1 -- 172.21.15.5:0/809353678 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50dc19f6d0 con 0x7f50dc1019c0 2024-09-10T09:46:11.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:11.563+0000 7f50d8c00640 1 -- 172.21.15.5:0/809353678 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50cc018990 con 0x7f50dc1019c0 2024-09-10T09:46:11.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:11.563+0000 7f50e1600640 1 -- 172.21.15.5:0/809353678 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50dc1a6fb0 con 0x7f50dc1019c0 2024-09-10T09:46:11.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:11.568+0000 7f50d8c00640 1 -- 172.21.15.5:0/809353678 <== mon.2 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f50cc010ac0 con 0x7f50dc1019c0 2024-09-10T09:46:11.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:11.568+0000 7f50e561b740 1 Processor -- start 2024-09-10T09:46:11.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:11.568+0000 7f50e561b740 1 -- start start 2024-09-10T09:46:11.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:11.569+0000 7f50e561b740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e4e17a2d50 0x55e4e17a3130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:46:11.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:11.569+0000 7f50e561b740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e4e17a3670 0x55e4e17ad9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:46:11.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:11.569+0000 7f50e561b740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e4e17adf30 0x55e4e17b0320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:46:11.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:11.569+0000 7f50e561b740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55e4e17093e0 con 0x55e4e17a2d50 2024-09-10T09:46:11.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:11.569+0000 7f50e561b740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55e4e16b0d80 con 0x55e4e17adf30 2024-09-10T09:46:11.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:11.569+0000 7f50e561b740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55e4e17053a0 con 0x55e4e17a3670 2024-09-10T09:46:11.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:11.569+0000 7f50e0c00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e4e17a2d50 0x55e4e17a3130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:46:11.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:11.569+0000 7f50db400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e4e17adf30 0x55e4e17b0320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:46:11.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:11.569+0000 7f50e0c00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e4e17a2d50 0x55e4e17a3130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56006/0 (socket says 172.21.15.5:56006) 2024-09-10T09:46:11.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:11.569+0000 7f50e0c00640 1 -- 172.21.15.5:0/1507224282 learned_addr learned my addr 172.21.15.5:0/1507224282 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:46:11.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:11.569+0000 7f50dbe00640 1 --2- 172.21.15.5:0/1507224282 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e4e17a3670 0x55e4e17ad9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:46:11.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:11.570+0000 7f50db400640 1 -- 172.21.15.5:0/1507224282 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e4e17a3670 msgr2=0x55e4e17ad9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:46:11.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:11.570+0000 7f50db400640 1 --2- 172.21.15.5:0/1507224282 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e4e17a3670 0x55e4e17ad9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:11.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:11.570+0000 7f50db400640 1 -- 172.21.15.5:0/1507224282 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e4e17a2d50 msgr2=0x55e4e17a3130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:46:11.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:11.570+0000 7f50db400640 1 --2- 172.21.15.5:0/1507224282 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e4e17a2d50 0x55e4e17a3130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:11.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:11.570+0000 7f50db400640 1 -- 172.21.15.5:0/1507224282 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e4e16cfae0 con 0x55e4e17adf30 2024-09-10T09:46:11.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:11.570+0000 7f50d8c00640 1 --2- 172.21.15.5:0/809353678 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f50c00a2c00 0x7f50c00a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:46:11.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:11.570+0000 7f50d8c00640 1 -- 172.21.15.5:0/809353678 <== mon.2 v2:172.21.15.102:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f50cc0cf6b0 con 0x7f50dc1019c0 2024-09-10T09:46:11.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:11.570+0000 7f50db400640 1 --2- 172.21.15.5:0/1507224282 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e4e17adf30 0x55e4e17b0320 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f50c8009bf0 tx=0x7f50c8011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8a81bc086cf68d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:46:11.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:11.570+0000 7f50e0c00640 1 --2- 172.21.15.5:0/809353678 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f50c00a2c00 0x7f50c00a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:46:11.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:11.570+0000 7f50d1200640 1 -- 172.21.15.5:0/1507224282 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50c8004050 con 0x55e4e17adf30 2024-09-10T09:46:11.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:11.570+0000 7f50d1200640 1 -- 172.21.15.5:0/1507224282 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f50c8005500 con 0x55e4e17adf30 2024-09-10T09:46:11.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:11.570+0000 7f50d1200640 1 -- 172.21.15.5:0/1507224282 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50c80035f0 con 0x55e4e17adf30 2024-09-10T09:46:11.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:11.571+0000 7f50e561b740 1 -- 172.21.15.5:0/1507224282 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e4e17adf30 msgr2=0x55e4e17b0320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:46:11.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:11.571+0000 7f50e0c00640 1 --2- 172.21.15.5:0/809353678 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f50c00a2c00 0x7f50c00a50c0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f50d4005e30 tx=0x7f50d4005c30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:46:11.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:11.571+0000 7f50e561b740 1 --2- 172.21.15.5:0/1507224282 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e4e17adf30 0x55e4e17b0320 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f50c8009bf0 tx=0x7f50c8011810 comp rx=0 tx=0).stop 2024-09-10T09:46:11.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:11.571+0000 7f50e561b740 1 -- 172.21.15.5:0/1507224282 shutdown_connections 2024-09-10T09:46:11.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:11.571+0000 7f50e561b740 1 --2- 172.21.15.5:0/1507224282 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e4e17adf30 0x55e4e17b0320 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:11.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:11.571+0000 7f50e561b740 1 --2- 172.21.15.5:0/1507224282 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e4e17a3670 0x55e4e17ad9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:11.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:11.571+0000 7f50e561b740 1 --2- 172.21.15.5:0/1507224282 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e4e17a2d50 0x55e4e17a3130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:11.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:11.571+0000 7f50e561b740 1 -- 172.21.15.5:0/1507224282 >> 172.21.15.5:0/1507224282 conn(0x55e4e179f2e0 msgr2=0x55e4e179f6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:46:11.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:11.571+0000 7f50e561b740 1 -- 172.21.15.5:0/1507224282 shutdown_connections 2024-09-10T09:46:11.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:11.571+0000 7f50e561b740 1 -- 172.21.15.5:0/1507224282 wait complete. 2024-09-10T09:46:11.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:11.572+0000 7f50e561b740 1 -- start start 2024-09-10T09:46:11.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:11.572+0000 7f50e561b740 1 Processor -- start 2024-09-10T09:46:11.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:11.573+0000 7f50e561b740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e4e17a2d50 0x55e4e17aae40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:46:11.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:11.573+0000 7f50e561b740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e4e17a3670 0x55e4e17ab380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:46:11.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:11.573+0000 7f50e561b740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e4e17adf30 0x55e4e17ab8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:46:11.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:11.573+0000 7f50e561b740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55e4e17b0ba0 con 0x55e4e17a2d50 2024-09-10T09:46:11.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:11.573+0000 7f50e561b740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55e4e16c40f0 con 0x55e4e17adf30 2024-09-10T09:46:11.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:11.573+0000 7f50e561b740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55e4e17b0d20 con 0x55e4e17a3670 2024-09-10T09:46:11.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:11.573+0000 7f50e561b740 10 client.0 _ll_register_callbacks cb 0x55e4e16b59e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:46:11.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:11.573+0000 7f50dbe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e4e17a2d50 0x55e4e17aae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:46:11.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:11.573+0000 7f50e0c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e4e17adf30 0x55e4e17ab8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:46:11.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:11.573+0000 7f50db400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e4e17a3670 0x55e4e17ab380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:46:11.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:11.573+0000 7f50dbe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e4e17a2d50 0x55e4e17aae40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56020/0 (socket says 172.21.15.5:56020) 2024-09-10T09:46:11.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:11.573+0000 7f50dbe00640 1 -- 172.21.15.5:0/3526840556 learned_addr learned my addr 172.21.15.5:0/3526840556 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:46:11.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:11.573+0000 7f50dbe00640 1 -- 172.21.15.5:0/3526840556 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e4e17a3670 msgr2=0x55e4e17ab380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:46:11.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:11.573+0000 7f50dbe00640 1 --2- 172.21.15.5:0/3526840556 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e4e17a3670 0x55e4e17ab380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:11.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:11.573+0000 7f50dbe00640 1 -- 172.21.15.5:0/3526840556 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e4e17adf30 msgr2=0x55e4e17ab8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:46:11.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:11.573+0000 7f50dbe00640 1 --2- 172.21.15.5:0/3526840556 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e4e17adf30 0x55e4e17ab8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:11.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:11.573+0000 7f50dbe00640 1 -- 172.21.15.5:0/3526840556 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e4e147a3c0 con 0x55e4e17a2d50 2024-09-10T09:46:11.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:11.574+0000 7f50e0c00640 1 --2- 172.21.15.5:0/3526840556 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e4e17adf30 0x55e4e17ab8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:46:11.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:11.574+0000 7f50dbe00640 1 --2- 172.21.15.5:0/3526840556 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e4e17a2d50 0x55e4e17aae40 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f50cc00e9c0 tx=0x7f50cc09b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19303c0f76ada678 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:46:11.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:11.574+0000 7f50c5600640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-10T09:46:11.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:11.574+0000 7f50c5600640 1 -- 172.21.15.5:0/3526840556 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50cc09c220 con 0x55e4e17a2d50 2024-09-10T09:46:11.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:11.574+0000 7f50c5600640 1 -- 172.21.15.5:0/3526840556 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f50cc0cfb60 con 0x55e4e17a2d50 2024-09-10T09:46:11.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:11.574+0000 7f50e561b740 1 -- 172.21.15.5:0/3526840556 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f50c8005500 con 0x55e4e17a2d50 2024-09-10T09:46:11.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:11.574+0000 7f50c5600640 1 -- 172.21.15.5:0/3526840556 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f50cc09bb20 con 0x55e4e17a2d50 2024-09-10T09:46:11.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:11.574+0000 7f50e561b740 10 client.16980 fetch_fsmap learned FSMap version 34 2024-09-10T09:46:11.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:11.574+0000 7f50e561b740 1 -- 172.21.15.5:0/3526840556 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55e4e17bc8b0 con 0x55e4e17a2d50 2024-09-10T09:46:11.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:11.575+0000 7f50c5600640 1 -- 172.21.15.5:0/3526840556 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f50c8005500 con 0x55e4e17a2d50 2024-09-10T09:46:11.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:11.575+0000 7f50c5600640 1 -- 172.21.15.5:0/3526840556 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f50cc018cc0 con 0x55e4e17a2d50 2024-09-10T09:46:11.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:11.575+0000 7f50e561b740 10 client.16980 fetch_fsmap finished waiting for FSMap version 34 2024-09-10T09:46:11.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:11.575+0000 7f50e561b740 10 client.16980 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T09:46:11.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:11.575+0000 7f50e561b740 20 client.16980 populate_metadata read hostname 'smithi005' 2024-09-10T09:46:11.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:11.575+0000 7f50e561b740 1 --2- 172.21.15.5:0/3526840556 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55e4e17bdd70 0x55e4e17c0180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:46:11.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:11.575+0000 7f50e561b740 4 client.16980 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:46:11.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:11.575+0000 7f50e561b740 1 -- 172.21.15.5:0/3526840556 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e4e1716950 con 0x55e4e17bdd70 2024-09-10T09:46:11.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:11.576+0000 7f50db400640 1 --2- 172.21.15.5:0/3526840556 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55e4e17bdd70 0x55e4e17c0180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:46:11.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:11.576+0000 7f50db400640 1 --2- 172.21.15.5:0/3526840556 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55e4e17bdd70 0x55e4e17c0180 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c7393409dc2d4416 server_cookie=95ec3b2cb8938756 in_seq=0 out_seq=0 2024-09-10T09:46:11.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:11.576+0000 7f50c5600640 10 client.16980 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T09:46:11.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:11.577+0000 7f50c5600640 1 -- 172.21.15.5:0/3526840556 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e4e1716950 con 0x55e4e17bdd70 2024-09-10T09:46:11.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:11.577+0000 7f50c5600640 10 client.16980 handle_command_reply: tid=0 2024-09-10T09:46:11.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:11.577+0000 7f50e561b740 1 client.16980 shutdown 2024-09-10T09:46:11.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:11.577+0000 7f50e561b740 1 -- 172.21.15.5:0/3526840556 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e4e17a2d50 msgr2=0x55e4e17aae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:46:11.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:11.577+0000 7f50e561b740 1 --2- 172.21.15.5:0/3526840556 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e4e17a2d50 0x55e4e17aae40 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f50cc00e9c0 tx=0x7f50cc09b8f0 comp rx=0 tx=0).stop 2024-09-10T09:46:11.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:11.577+0000 7f50e561b740 1 -- 172.21.15.5:0/3526840556 shutdown_connections 2024-09-10T09:46:11.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:11.577+0000 7f50e561b740 1 --2- 172.21.15.5:0/3526840556 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55e4e17bdd70 0x55e4e17c0180 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:11.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:11.578+0000 7f50e561b740 1 --2- 172.21.15.5:0/3526840556 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e4e17adf30 0x55e4e17ab8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:11.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:11.578+0000 7f50e561b740 1 --2- 172.21.15.5:0/3526840556 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e4e17a3670 0x55e4e17ab380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:11.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:11.578+0000 7f50e561b740 1 --2- 172.21.15.5:0/3526840556 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e4e17a2d50 0x55e4e17aae40 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:11.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:11.578+0000 7f50e561b740 1 -- 172.21.15.5:0/3526840556 >> 172.21.15.5:0/3526840556 conn(0x55e4e179f2e0 msgr2=0x55e4e17a5010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:46:11.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:11.578+0000 7f50c5600640 0 client.16980 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T09:46:11.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:11.578+0000 7f50e561b740 1 -- 172.21.15.5:0/3526840556 shutdown_connections 2024-09-10T09:46:11.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:11.578+0000 7f50e561b740 1 -- 172.21.15.5:0/3526840556 wait complete. 2024-09-10T09:46:11.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:11.578+0000 7f50e561b740 20 client.16980 trim_cache size 0 max 16384 2024-09-10T09:46:11.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:11.596+0000 7f50e561b740 1 Processor -- start 2024-09-10T09:46:11.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:11.596+0000 7f50e561b740 1 -- start start 2024-09-10T09:46:11.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:11.597+0000 7f50e561b740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e4e17a2d50 0x55e4e17bfd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:46:11.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:11.597+0000 7f50e561b740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e4e17a3670 0x55e4e17acdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:46:11.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:11.597+0000 7f50e561b740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e4e17adf30 0x55e4e17c5a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:46:11.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:11.597+0000 7f50e561b740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f50dc1b0f10 con 0x55e4e17a2d50 2024-09-10T09:46:11.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:11.597+0000 7f50e561b740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f50dc1b2a50 con 0x55e4e17a3670 2024-09-10T09:46:11.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:11.597+0000 7f50e561b740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f50dc1b15e0 con 0x55e4e17adf30 2024-09-10T09:46:11.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:11.597+0000 7f50dbe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e4e17a2d50 0x55e4e17bfd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:46:11.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:11.597+0000 7f50e0c00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e4e17adf30 0x55e4e17c5a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:46:11.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:11.597+0000 7f50db400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e4e17a3670 0x55e4e17acdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:46:11.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:11.597+0000 7f50dbe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e4e17a2d50 0x55e4e17bfd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56024/0 (socket says 172.21.15.5:56024) 2024-09-10T09:46:11.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:11.597+0000 7f50e0c00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e4e17adf30 0x55e4e17c5a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:53232/0 (socket says 172.21.15.5:53232) 2024-09-10T09:46:11.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:11.597+0000 7f50dbe00640 1 -- 172.21.15.5:0/719904356 learned_addr learned my addr 172.21.15.5:0/719904356 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:46:11.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:11.597+0000 7f50db400640 1 -- 172.21.15.5:0/719904356 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e4e17adf30 msgr2=0x55e4e17c5a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:46:11.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:11.597+0000 7f50db400640 1 --2- 172.21.15.5:0/719904356 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e4e17adf30 0x55e4e17c5a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:11.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:11.597+0000 7f50db400640 1 -- 172.21.15.5:0/719904356 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e4e17a2d50 msgr2=0x55e4e17bfd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:46:11.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:11.597+0000 7f50db400640 1 --2- 172.21.15.5:0/719904356 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e4e17a2d50 0x55e4e17bfd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:11.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:11.597+0000 7f50db400640 1 -- 172.21.15.5:0/719904356 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e4e17a9df0 con 0x55e4e17a3670 2024-09-10T09:46:11.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:11.597+0000 7f50dbe00640 1 --2- 172.21.15.5:0/719904356 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e4e17a2d50 0x55e4e17bfd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:46:11.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:11.598+0000 7f50db400640 1 --2- 172.21.15.5:0/719904356 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e4e17a3670 0x55e4e17acdb0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f50c8004a20 tx=0x7f50c8004b20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=14fb32ea1baaccd0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:46:11.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:11.598+0000 7f50c7e00640 1 -- 172.21.15.5:0/719904356 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50c8009da0 con 0x55e4e17a3670 2024-09-10T09:46:11.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:11.598+0000 7f50c7e00640 1 -- 172.21.15.5:0/719904356 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f50c80035f0 con 0x55e4e17a3670 2024-09-10T09:46:11.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:11.598+0000 7f50c7e00640 1 -- 172.21.15.5:0/719904356 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50c802d690 con 0x55e4e17a3670 2024-09-10T09:46:11.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:11.599+0000 7f50e561b740 1 -- 172.21.15.5:0/719904356 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e4e17a3670 msgr2=0x55e4e17acdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:46:11.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:11.599+0000 7f50e561b740 1 --2- 172.21.15.5:0/719904356 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e4e17a3670 0x55e4e17acdb0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f50c8004a20 tx=0x7f50c8004b20 comp rx=0 tx=0).stop 2024-09-10T09:46:11.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:11.599+0000 7f50e561b740 1 -- 172.21.15.5:0/719904356 shutdown_connections 2024-09-10T09:46:11.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:11.599+0000 7f50e561b740 1 --2- 172.21.15.5:0/719904356 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e4e17adf30 0x55e4e17c5a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:11.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:11.599+0000 7f50e561b740 1 --2- 172.21.15.5:0/719904356 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e4e17a3670 0x55e4e17acdb0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:11.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:11.599+0000 7f50e561b740 1 --2- 172.21.15.5:0/719904356 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e4e17a2d50 0x55e4e17bfd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:11.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:11.599+0000 7f50e561b740 1 -- 172.21.15.5:0/719904356 >> 172.21.15.5:0/719904356 conn(0x55e4e179f2e0 msgr2=0x55e4e17ac6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:46:11.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:11.599+0000 7f50e561b740 1 -- 172.21.15.5:0/719904356 shutdown_connections 2024-09-10T09:46:11.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:11.599+0000 7f50e561b740 1 -- 172.21.15.5:0/719904356 wait complete. 2024-09-10T09:46:11.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:11.599+0000 7f50e561b740 1 -- start start 2024-09-10T09:46:11.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:11.600+0000 7f50e561b740 1 Processor -- start 2024-09-10T09:46:11.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:11.600+0000 7f50e561b740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e4e17a2d50 0x55e4e17b16e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:46:11.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:11.600+0000 7f50e561b740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e4e17a3670 0x55e4e17b1c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:46:11.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:11.600+0000 7f50e561b740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e4e17adf30 0x55e4e1820a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:46:11.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:11.600+0000 7f50e561b740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f50dc1b2380 con 0x55e4e17a2d50 2024-09-10T09:46:11.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:11.600+0000 7f50e561b740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f50dc1b1cb0 con 0x55e4e17a3670 2024-09-10T09:46:11.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:11.600+0000 7f50e561b740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f50dc1aeef0 con 0x55e4e17adf30 2024-09-10T09:46:11.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:11.600+0000 7f50e561b740 10 client.0 _ll_register_callbacks cb 0x7f50cc009700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:46:11.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:11.600+0000 7f50db400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e4e17a3670 0x55e4e17b1c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:46:11.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:11.600+0000 7f50e0c00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e4e17adf30 0x55e4e1820a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:46:11.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:11.600+0000 7f50e0c00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e4e17adf30 0x55e4e1820a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:53238/0 (socket says 172.21.15.5:53238) 2024-09-10T09:46:11.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:11.600+0000 7f50db400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e4e17a3670 0x55e4e17b1c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:54340/0 (socket says 172.21.15.5:54340) 2024-09-10T09:46:11.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:11.601+0000 7f50e0c00640 1 -- 172.21.15.5:0/45382147 learned_addr learned my addr 172.21.15.5:0/45382147 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:46:11.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:11.601+0000 7f50dbe00640 1 --2- 172.21.15.5:0/45382147 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e4e17a2d50 0x55e4e17b16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:46:11.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:11.601+0000 7f50e0c00640 1 -- 172.21.15.5:0/45382147 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e4e17a3670 msgr2=0x55e4e17b1c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:46:11.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:11.601+0000 7f50e0c00640 1 --2- 172.21.15.5:0/45382147 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e4e17a3670 0x55e4e17b1c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:11.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:11.601+0000 7f50e0c00640 1 -- 172.21.15.5:0/45382147 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e4e17a2d50 msgr2=0x55e4e17b16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:46:11.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:11.601+0000 7f50e0c00640 1 --2- 172.21.15.5:0/45382147 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e4e17a2d50 0x55e4e17b16e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:11.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:11.601+0000 7f50e0c00640 1 -- 172.21.15.5:0/45382147 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e4e1821060 con 0x55e4e17adf30 2024-09-10T09:46:11.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:11.601+0000 7f50dbe00640 1 --2- 172.21.15.5:0/45382147 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e4e17a2d50 0x55e4e17b16e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:46:11.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:11.601+0000 7f50e0c00640 1 --2- 172.21.15.5:0/45382147 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e4e17adf30 0x55e4e1820a20 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f50d4003f20 tx=0x7f50d402a8c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=11da88a3fd17bf5f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:46:11.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:11.601+0000 7f50c5600640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-09-10T09:46:11.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:11.601+0000 7f50c5600640 1 -- 172.21.15.5:0/45382147 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50d40317e0 con 0x55e4e17adf30 2024-09-10T09:46:11.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:11.602+0000 7f50c5600640 1 -- 172.21.15.5:0/45382147 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f50d4042030 con 0x55e4e17adf30 2024-09-10T09:46:11.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:11.602+0000 7f50e561b740 1 -- 172.21.15.5:0/45382147 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f50c80035f0 con 0x55e4e17adf30 2024-09-10T09:46:11.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:11.602+0000 7f50c5600640 1 -- 172.21.15.5:0/45382147 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f50d4031980 con 0x55e4e17adf30 2024-09-10T09:46:11.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:11.602+0000 7f50e561b740 10 client.27010 fetch_fsmap learned FSMap version 34 2024-09-10T09:46:11.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:11.602+0000 7f50e561b740 1 -- 172.21.15.5:0/45382147 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55e4e1828400 con 0x55e4e17adf30 2024-09-10T09:46:11.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:11.603+0000 7f50c5600640 1 -- 172.21.15.5:0/45382147 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f50c80035f0 con 0x55e4e17adf30 2024-09-10T09:46:11.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:11.603+0000 7f50c5600640 1 -- 172.21.15.5:0/45382147 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f50d402fa50 con 0x55e4e17adf30 2024-09-10T09:46:11.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:11.603+0000 7f50e561b740 10 client.27010 fetch_fsmap finished waiting for FSMap version 34 2024-09-10T09:46:11.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:11.603+0000 7f50e561b740 10 client.27010 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T09:46:11.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:11.603+0000 7f50e561b740 20 client.27010 populate_metadata read hostname 'smithi005' 2024-09-10T09:46:11.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:11.603+0000 7f50e561b740 1 --2- 172.21.15.5:0/45382147 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55e4e1828b80 0x55e4e182afa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:46:11.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:11.603+0000 7f50e561b740 4 client.27010 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T09:46:11.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:11.603+0000 7f50e561b740 1 -- 172.21.15.5:0/45382147 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55e4e1710e60 con 0x55e4e1828b80 2024-09-10T09:46:11.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:11.603+0000 7f50dbe00640 1 --2- 172.21.15.5:0/45382147 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55e4e1828b80 0x55e4e182afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:46:11.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:11.604+0000 7f50dbe00640 1 --2- 172.21.15.5:0/45382147 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55e4e1828b80 0x55e4e182afa0 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6eb872f0642404d6 server_cookie=66df4f1a020aa8e in_seq=0 out_seq=0 2024-09-10T09:46:11.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:11.604+0000 7f50c5600640 10 client.27010 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T09:46:11.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:11.604+0000 7f50c5600640 1 -- 172.21.15.5:0/45382147 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55e4e1710e60 con 0x55e4e1828b80 2024-09-10T09:46:11.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:11.604+0000 7f50c5600640 10 client.27010 handle_command_reply: tid=0 2024-09-10T09:46:11.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:11.604+0000 7f50e561b740 1 client.27010 shutdown 2024-09-10T09:46:11.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:11.605+0000 7f50e561b740 1 -- 172.21.15.5:0/45382147 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e4e17adf30 msgr2=0x55e4e1820a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:46:11.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:11.605+0000 7f50e561b740 1 --2- 172.21.15.5:0/45382147 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e4e17adf30 0x55e4e1820a20 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f50d4003f20 tx=0x7f50d402a8c0 comp rx=0 tx=0).stop 2024-09-10T09:46:11.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:11.605+0000 7f50e561b740 1 -- 172.21.15.5:0/45382147 shutdown_connections 2024-09-10T09:46:11.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:11.605+0000 7f50e561b740 1 --2- 172.21.15.5:0/45382147 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55e4e1828b80 0x55e4e182afa0 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:11.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:11.605+0000 7f50e561b740 1 --2- 172.21.15.5:0/45382147 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e4e17adf30 0x55e4e1820a20 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:11.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:11.605+0000 7f50e561b740 1 --2- 172.21.15.5:0/45382147 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e4e17a3670 0x55e4e17b1c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:11.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:11.605+0000 7f50e561b740 1 --2- 172.21.15.5:0/45382147 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e4e17a2d50 0x55e4e17b16e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:11.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:11.605+0000 7f50e561b740 1 -- 172.21.15.5:0/45382147 >> 172.21.15.5:0/45382147 conn(0x55e4e179f2e0 msgr2=0x55e4e17a4fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:46:11.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:11.605+0000 7f50c5600640 0 client.27010 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T09:46:11.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:11.605+0000 7f50e561b740 1 -- 172.21.15.5:0/45382147 shutdown_connections 2024-09-10T09:46:11.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:11.605+0000 7f50e561b740 1 -- 172.21.15.5:0/45382147 wait complete. 2024-09-10T09:46:11.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:11.605+0000 7f50e561b740 20 client.27010 trim_cache size 0 max 16384 2024-09-10T09:46:11.610 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-10T09:46:11.610 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-10T09:46:11.610 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2024-09-10T09:46:11.610 INFO:teuthology.orchestra.run.smithi005.stdout: "dd9582e0-e8cd-405c-8c0c-6d0709a1a30c": { 2024-09-10T09:46:11.610 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2024-09-10T09:46:11.610 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "dd9582e0-e8cd-405c-8c0c-6d0709a1a30c", 2024-09-10T09:46:11.610 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2024-09-10T09:46:11.610 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T09:46:11.610 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T09:46:11.610 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-10T09:46:11.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:11.606+0000 7f50e1600640 1 -- 172.21.15.5:0/809353678 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f50c00a2c00 msgr2=0x7f50c00a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:46:11.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:11.606+0000 7f50e1600640 1 --2- 172.21.15.5:0/809353678 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f50c00a2c00 0x7f50c00a50c0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f50d4005e30 tx=0x7f50d4005c30 comp rx=0 tx=0).stop 2024-09-10T09:46:11.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:11.606+0000 7f50e1600640 1 -- 172.21.15.5:0/809353678 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f50dc1019c0 msgr2=0x7f50dc19a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:46:11.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:11.606+0000 7f50e1600640 1 --2- 172.21.15.5:0/809353678 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f50dc1019c0 0x7f50dc19a430 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f50cc00e9f0 tx=0x7f50cc00eeb0 comp rx=0 tx=0).stop 2024-09-10T09:46:11.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:11.606+0000 7f50e1600640 1 -- 172.21.15.5:0/809353678 shutdown_connections 2024-09-10T09:46:11.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:11.606+0000 7f50e1600640 1 --2- 172.21.15.5:0/809353678 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f50c00a2c00 0x7f50c00a50c0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:11.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:11.606+0000 7f50e1600640 1 --2- 172.21.15.5:0/809353678 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f50dc1079c0 0x7f50dc19ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:11.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:11.606+0000 7f50e1600640 1 --2- 172.21.15.5:0/809353678 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f50dc102360 0x7f50dc19a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:11.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:11.606+0000 7f50e1600640 1 --2- 172.21.15.5:0/809353678 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f50dc1019c0 0x7f50dc19a430 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:11.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:11.606+0000 7f50e1600640 1 -- 172.21.15.5:0/809353678 >> 172.21.15.5:0/809353678 conn(0x7f50dc0fd740 msgr2=0x7f50dc0fec80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:46:11.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:11.606+0000 7f50e1600640 1 -- 172.21.15.5:0/809353678 shutdown_connections 2024-09-10T09:46:11.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:11.606+0000 7f50e1600640 1 -- 172.21.15.5:0/809353678 wait complete. 2024-09-10T09:46:12.064 INFO:tasks.workunit.client.0.smithi005.stdout: 1 457232 6.13 MB/sec execute 233 sec latency 23.387 ms 2024-09-10T09:46:12.549 INFO:tasks.cephfs.filesystem:scrub status for tag:dd9582e0-e8cd-405c-8c0c-6d0709a1a30c - {'path': '/', 'tag': 'dd9582e0-e8cd-405c-8c0c-6d0709a1a30c', 'options': 'recursive,force'} 2024-09-10T09:46:12.899 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:46:12 smithi005 ceph-mon[30350]: pgmap v629: 129 pgs: 129 active+clean; 182 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.4 MiB/s wr, 175 op/s 2024-09-10T09:46:13.064 INFO:tasks.workunit.client.0.smithi005.stdout: 1 457721 6.12 MB/sec execute 234 sec latency 717.468 ms 2024-09-10T09:46:13.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:46:12 smithi159 ceph-mon[35513]: pgmap v629: 129 pgs: 129 active+clean; 182 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.4 MiB/s wr, 175 op/s 2024-09-10T09:46:13.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:46:12 smithi102 ceph-mon[35442]: pgmap v629: 129 pgs: 129 active+clean; 182 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.4 MiB/s wr, 175 op/s 2024-09-10T09:46:14.064 INFO:tasks.workunit.client.0.smithi005.stdout: 1 457721 6.10 MB/sec execute 235 sec latency 1717.531 ms 2024-09-10T09:46:15.065 INFO:tasks.workunit.client.0.smithi005.stdout: 1 458659 6.13 MB/sec execute 236 sec latency 1864.806 ms 2024-09-10T09:46:15.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:46:14 smithi159 ceph-mon[35513]: pgmap v630: 129 pgs: 129 active+clean; 184 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.4 MiB/s wr, 169 op/s 2024-09-10T09:46:15.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:46:14 smithi102 ceph-mon[35442]: pgmap v630: 129 pgs: 129 active+clean; 184 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.4 MiB/s wr, 169 op/s 2024-09-10T09:46:15.306 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:46:14 smithi005 ceph-mon[30350]: pgmap v630: 129 pgs: 129 active+clean; 184 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.4 MiB/s wr, 169 op/s 2024-09-10T09:46:16.064 INFO:tasks.workunit.client.0.smithi005.stdout: 1 461242 6.19 MB/sec execute 237 sec latency 11.976 ms 2024-09-10T09:46:17.065 INFO:tasks.workunit.client.0.smithi005.stdout: 1 463184 6.20 MB/sec execute 238 sec latency 28.947 ms 2024-09-10T09:46:17.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:46:16 smithi159 ceph-mon[35513]: pgmap v631: 129 pgs: 129 active+clean; 185 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.7 MiB/s wr, 208 op/s 2024-09-10T09:46:17.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:46:16 smithi102 ceph-mon[35442]: pgmap v631: 129 pgs: 129 active+clean; 185 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.7 MiB/s wr, 208 op/s 2024-09-10T09:46:17.306 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:46:16 smithi005 ceph-mon[30350]: pgmap v631: 129 pgs: 129 active+clean; 185 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.7 MiB/s wr, 208 op/s 2024-09-10T09:46:18.065 INFO:tasks.workunit.client.0.smithi005.stdout: 1 464484 6.20 MB/sec execute 239 sec latency 563.488 ms 2024-09-10T09:46:19.065 INFO:tasks.workunit.client.0.smithi005.stdout: 1 464484 6.18 MB/sec execute 240 sec latency 1563.536 ms 2024-09-10T09:46:19.074 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:46:18 smithi005 ceph-mon[30350]: pgmap v632: 129 pgs: 129 active+clean; 182 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.4 MiB/s wr, 169 op/s 2024-09-10T09:46:19.074 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:46:18 smithi005 ceph-mon[30350]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T09:46:19.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:46:18 smithi102 ceph-mon[35442]: pgmap v632: 129 pgs: 129 active+clean; 182 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.4 MiB/s wr, 169 op/s 2024-09-10T09:46:19.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:46:18 smithi102 ceph-mon[35442]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T09:46:19.352 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:46:18 smithi159 ceph-mon[35513]: pgmap v632: 129 pgs: 129 active+clean; 182 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.4 MiB/s wr, 169 op/s 2024-09-10T09:46:19.353 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:46:18 smithi159 ceph-mon[35513]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T09:46:20.065 INFO:tasks.workunit.client.0.smithi005.stdout: 1 466007 6.20 MB/sec execute 241 sec latency 1838.473 ms 2024-09-10T09:46:21.065 INFO:tasks.workunit.client.0.smithi005.stdout: 1 468794 6.23 MB/sec execute 242 sec latency 23.811 ms 2024-09-10T09:46:21.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:46:20 smithi102 ceph-mon[35442]: pgmap v633: 129 pgs: 129 active+clean; 174 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.7 MiB/s wr, 215 op/s 2024-09-10T09:46:21.306 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:46:20 smithi005 ceph-mon[30350]: pgmap v633: 129 pgs: 129 active+clean; 174 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.7 MiB/s wr, 215 op/s 2024-09-10T09:46:21.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:46:20 smithi159 ceph-mon[35513]: pgmap v633: 129 pgs: 129 active+clean; 174 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.7 MiB/s wr, 215 op/s 2024-09-10T09:46:22.065 INFO:tasks.workunit.client.0.smithi005.stdout: 1 470652 6.25 MB/sec execute 243 sec latency 37.323 ms 2024-09-10T09:46:23.071 INFO:tasks.workunit.client.0.smithi005.stdout: 1 472371 6.26 MB/sec execute 244 sec latency 421.383 ms 2024-09-10T09:46:23.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:46:22 smithi102 ceph-mon[35442]: pgmap v634: 129 pgs: 129 active+clean; 171 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.5 MiB/s wr, 180 op/s 2024-09-10T09:46:23.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:46:22 smithi102 ceph-mon[35442]: scrub summary: idle 2024-09-10T09:46:23.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:46:22 smithi102 ceph-mon[35442]: scrub summary: idle 2024-09-10T09:46:23.152 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:46:22 smithi159 ceph-mon[35513]: pgmap v634: 129 pgs: 129 active+clean; 171 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.5 MiB/s wr, 180 op/s 2024-09-10T09:46:23.152 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:46:22 smithi159 ceph-mon[35513]: scrub summary: idle 2024-09-10T09:46:23.152 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:46:22 smithi159 ceph-mon[35513]: scrub summary: idle 2024-09-10T09:46:23.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:46:22 smithi005 ceph-mon[30350]: pgmap v634: 129 pgs: 129 active+clean; 171 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.5 MiB/s wr, 180 op/s 2024-09-10T09:46:23.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:46:22 smithi005 ceph-mon[30350]: scrub summary: idle 2024-09-10T09:46:23.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:46:22 smithi005 ceph-mon[30350]: scrub summary: idle 2024-09-10T09:46:24.071 INFO:tasks.workunit.client.0.smithi005.stdout: 1 472371 6.23 MB/sec execute 245 sec latency 1421.430 ms 2024-09-10T09:46:25.072 INFO:tasks.workunit.client.0.smithi005.stdout: 1 473703 6.23 MB/sec execute 246 sec latency 1622.479 ms 2024-09-10T09:46:25.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:46:24 smithi102 ceph-mon[35442]: pgmap v635: 129 pgs: 129 active+clean; 171 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 860 KiB/s rd, 2.3 MiB/s wr, 169 op/s 2024-09-10T09:46:25.306 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:46:24 smithi005 ceph-mon[30350]: pgmap v635: 129 pgs: 129 active+clean; 171 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 860 KiB/s rd, 2.3 MiB/s wr, 169 op/s 2024-09-10T09:46:25.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:46:24 smithi159 ceph-mon[35513]: pgmap v635: 129 pgs: 129 active+clean; 171 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 860 KiB/s rd, 2.3 MiB/s wr, 169 op/s 2024-09-10T09:46:26.072 INFO:tasks.workunit.client.0.smithi005.stdout: 1 476459 6.28 MB/sec execute 247 sec latency 11.230 ms 2024-09-10T09:46:27.072 INFO:tasks.workunit.client.0.smithi005.stdout: 1 478729 6.29 MB/sec execute 248 sec latency 23.436 ms 2024-09-10T09:46:27.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:46:26 smithi102 ceph-mon[35442]: pgmap v636: 129 pgs: 129 active+clean; 176 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 854 KiB/s rd, 2.6 MiB/s wr, 222 op/s 2024-09-10T09:46:27.306 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:46:26 smithi005 ceph-mon[30350]: pgmap v636: 129 pgs: 129 active+clean; 176 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 854 KiB/s rd, 2.6 MiB/s wr, 222 op/s 2024-09-10T09:46:27.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:46:26 smithi159 ceph-mon[35513]: pgmap v636: 129 pgs: 129 active+clean; 176 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 854 KiB/s rd, 2.6 MiB/s wr, 222 op/s 2024-09-10T09:46:28.072 INFO:tasks.workunit.client.0.smithi005.stdout: 1 480258 6.30 MB/sec execute 249 sec latency 318.292 ms 2024-09-10T09:46:29.072 INFO:tasks.workunit.client.0.smithi005.stdout: 1 480261 6.28 MB/sec execute 250 sec latency 667.260 ms 2024-09-10T09:46:29.129 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:46:28 smithi005 ceph-mon[30350]: pgmap v637: 129 pgs: 129 active+clean; 176 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 513 KiB/s rd, 2.4 MiB/s wr, 182 op/s 2024-09-10T09:46:29.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:46:28 smithi102 ceph-mon[35442]: pgmap v637: 129 pgs: 129 active+clean; 176 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 513 KiB/s rd, 2.4 MiB/s wr, 182 op/s 2024-09-10T09:46:29.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:46:28 smithi159 ceph-mon[35513]: pgmap v637: 129 pgs: 129 active+clean; 176 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 513 KiB/s rd, 2.4 MiB/s wr, 182 op/s 2024-09-10T09:46:30.072 INFO:tasks.workunit.client.0.smithi005.stdout: 1 481053 6.27 MB/sec execute 251 sec latency 1258.621 ms 2024-09-10T09:46:31.072 INFO:tasks.workunit.client.0.smithi005.stdout: 1 483611 6.31 MB/sec execute 252 sec latency 18.817 ms 2024-09-10T09:46:31.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:46:30 smithi102 ceph-mon[35442]: pgmap v638: 129 pgs: 129 active+clean; 176 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 646 KiB/s rd, 2.7 MiB/s wr, 228 op/s 2024-09-10T09:46:31.306 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:46:30 smithi005 ceph-mon[30350]: pgmap v638: 129 pgs: 129 active+clean; 176 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 646 KiB/s rd, 2.7 MiB/s wr, 228 op/s 2024-09-10T09:46:31.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:46:30 smithi159 ceph-mon[35513]: pgmap v638: 129 pgs: 129 active+clean; 176 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 646 KiB/s rd, 2.7 MiB/s wr, 228 op/s 2024-09-10T09:46:32.072 INFO:tasks.workunit.client.0.smithi005.stdout: 1 485977 6.32 MB/sec execute 253 sec latency 18.269 ms 2024-09-10T09:46:33.075 INFO:tasks.workunit.client.0.smithi005.stdout: 1 487471 6.33 MB/sec execute 254 sec latency 163.076 ms 2024-09-10T09:46:33.122 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:46:32 smithi005 ceph-mon[30350]: pgmap v639: 129 pgs: 129 active+clean; 176 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 446 KiB/s rd, 2.5 MiB/s wr, 179 op/s 2024-09-10T09:46:33.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:46:32 smithi159 ceph-mon[35513]: pgmap v639: 129 pgs: 129 active+clean; 176 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 446 KiB/s rd, 2.5 MiB/s wr, 179 op/s 2024-09-10T09:46:33.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:46:32 smithi102 ceph-mon[35442]: pgmap v639: 129 pgs: 129 active+clean; 176 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 446 KiB/s rd, 2.5 MiB/s wr, 179 op/s 2024-09-10T09:46:34.075 INFO:tasks.workunit.client.0.smithi005.stdout: 1 487561 6.31 MB/sec execute 255 sec latency 1087.974 ms 2024-09-10T09:46:35.075 INFO:tasks.workunit.client.0.smithi005.stdout: 1 490414 6.35 MB/sec execute 256 sec latency 13.178 ms 2024-09-10T09:46:35.306 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:46:34 smithi005 ceph-mon[30350]: pgmap v640: 129 pgs: 129 active+clean; 176 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 446 KiB/s rd, 2.3 MiB/s wr, 170 op/s 2024-09-10T09:46:35.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:46:34 smithi159 ceph-mon[35513]: pgmap v640: 129 pgs: 129 active+clean; 176 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 446 KiB/s rd, 2.3 MiB/s wr, 170 op/s 2024-09-10T09:46:35.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:46:34 smithi102 ceph-mon[35442]: pgmap v640: 129 pgs: 129 active+clean; 176 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 446 KiB/s rd, 2.3 MiB/s wr, 170 op/s 2024-09-10T09:46:36.075 INFO:tasks.workunit.client.0.smithi005.stdout: 1 492223 6.35 MB/sec execute 257 sec latency 24.145 ms 2024-09-10T09:46:37.075 INFO:tasks.workunit.client.0.smithi005.stdout: 1 494537 6.38 MB/sec execute 258 sec latency 13.953 ms 2024-09-10T09:46:37.193 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 13.4 2024-09-10T09:46:37.194 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T09:46:37.244 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:46:36 smithi005 ceph-mon[30350]: pgmap v641: 129 pgs: 129 active+clean; 172 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 710 KiB/s rd, 2.8 MiB/s wr, 230 op/s 2024-09-10T09:46:37.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:46:36 smithi159 ceph-mon[35513]: pgmap v641: 129 pgs: 129 active+clean; 172 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 710 KiB/s rd, 2.8 MiB/s wr, 230 op/s 2024-09-10T09:46:37.385 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:46:36 smithi102 ceph-mon[35442]: pgmap v641: 129 pgs: 129 active+clean; 172 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 710 KiB/s rd, 2.8 MiB/s wr, 230 op/s 2024-09-10T09:46:37.649 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:46:38.075 INFO:tasks.workunit.client.0.smithi005.stdout: 1 496422 6.39 MB/sec execute 259 sec latency 93.285 ms 2024-09-10T09:46:38.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:38.572+0000 7f73ffe00640 1 -- 172.21.15.5:0/2651771987 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7400076720 msgr2=0x7f7400076b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:46:38.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:38.572+0000 7f73ffe00640 1 --2- 172.21.15.5:0/2651771987 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7400076720 0x7f7400076b00 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f73f0009ca0 tx=0x7f73f002a8c0 comp rx=0 tx=0).stop 2024-09-10T09:46:38.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:38.572+0000 7f73ffe00640 1 -- 172.21.15.5:0/2651771987 shutdown_connections 2024-09-10T09:46:38.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:38.572+0000 7f73ffe00640 1 --2- 172.21.15.5:0/2651771987 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7400112870 0x7f7400114c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:38.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:38.572+0000 7f73ffe00640 1 --2- 172.21.15.5:0/2651771987 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7400077040 0x7f7400112330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:38.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:38.572+0000 7f73ffe00640 1 --2- 172.21.15.5:0/2651771987 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7400076720 0x7f7400076b00 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:38.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:38.572+0000 7f73ffe00640 1 -- 172.21.15.5:0/2651771987 >> 172.21.15.5:0/2651771987 conn(0x7f74000ff8d0 msgr2=0x7f7400101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:46:38.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:38.572+0000 7f73ffe00640 1 -- 172.21.15.5:0/2651771987 shutdown_connections 2024-09-10T09:46:38.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:38.572+0000 7f73ffe00640 1 -- 172.21.15.5:0/2651771987 wait complete. 2024-09-10T09:46:38.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:38.573+0000 7f73ffe00640 1 Processor -- start 2024-09-10T09:46:38.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:38.574+0000 7f73ffe00640 1 -- start start 2024-09-10T09:46:38.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:38.574+0000 7f73ffe00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7400076720 0x7f74001a4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:46:38.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:38.574+0000 7f73ffe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7400077040 0x7f74001a4b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:46:38.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:38.574+0000 7f73ffe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7400112870 0x7f740019e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:46:38.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:38.574+0000 7f73ffe00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f74001183e0 con 0x7f7400077040 2024-09-10T09:46:38.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:38.574+0000 7f73ffe00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f7400118260 con 0x7f7400112870 2024-09-10T09:46:38.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:38.574+0000 7f73ffe00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f7400118560 con 0x7f7400076720 2024-09-10T09:46:38.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:38.574+0000 7f73f6000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7400077040 0x7f74001a4b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:46:38.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:38.574+0000 7f73f6000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7400077040 0x7f74001a4b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42100/0 (socket says 172.21.15.5:42100) 2024-09-10T09:46:38.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:38.574+0000 7f73f6000640 1 -- 172.21.15.5:0/2868310610 learned_addr learned my addr 172.21.15.5:0/2868310610 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:46:38.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:38.574+0000 7f73fea00640 1 --2- 172.21.15.5:0/2868310610 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7400076720 0x7f74001a4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:46:38.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:38.575+0000 7f73ff400640 1 --2- 172.21.15.5:0/2868310610 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7400112870 0x7f740019e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:46:38.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:38.575+0000 7f73f6000640 1 -- 172.21.15.5:0/2868310610 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7400076720 msgr2=0x7f74001a4600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:46:38.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:38.575+0000 7f73f6000640 1 --2- 172.21.15.5:0/2868310610 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7400076720 0x7f74001a4600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:38.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:38.575+0000 7f73f6000640 1 -- 172.21.15.5:0/2868310610 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7400112870 msgr2=0x7f740019e6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:46:38.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:38.575+0000 7f73f6000640 1 --2- 172.21.15.5:0/2868310610 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7400112870 0x7f740019e6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:38.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:38.575+0000 7f73f6000640 1 -- 172.21.15.5:0/2868310610 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f740019ef40 con 0x7f7400077040 2024-09-10T09:46:38.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:38.575+0000 7f73fea00640 1 --2- 172.21.15.5:0/2868310610 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7400076720 0x7f74001a4600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:46:38.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:38.575+0000 7f73f6000640 1 --2- 172.21.15.5:0/2868310610 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7400077040 0x7f74001a4b40 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f73e800e9f0 tx=0x7f73e800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:46:38.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:38.575+0000 7f73f7e00640 1 -- 172.21.15.5:0/2868310610 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73e800cd80 con 0x7f7400077040 2024-09-10T09:46:38.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:38.576+0000 7f73ffe00640 1 -- 172.21.15.5:0/2868310610 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f740019f230 con 0x7f7400077040 2024-09-10T09:46:38.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:38.576+0000 7f73ffe00640 1 -- 172.21.15.5:0/2868310610 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74001ab580 con 0x7f7400077040 2024-09-10T09:46:38.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:38.576+0000 7f73f7e00640 1 -- 172.21.15.5:0/2868310610 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f73e8010910 con 0x7f7400077040 2024-09-10T09:46:38.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:38.576+0000 7f73f7e00640 1 -- 172.21.15.5:0/2868310610 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73e8018930 con 0x7f7400077040 2024-09-10T09:46:38.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:38.577+0000 7f73ffe00640 1 -- 172.21.15.5:0/2868310610 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f73dc005290 con 0x7f7400077040 2024-09-10T09:46:38.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:38.579+0000 7f73f7e00640 1 -- 172.21.15.5:0/2868310610 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f73e8010430 con 0x7f7400077040 2024-09-10T09:46:38.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:38.581+0000 7f73f7e00640 1 --2- 172.21.15.5:0/2868310610 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f73e40a2c00 0x7f73e40a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:46:38.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:38.581+0000 7f73f7e00640 1 -- 172.21.15.5:0/2868310610 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f73e80cf710 con 0x7f7400077040 2024-09-10T09:46:38.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:38.582+0000 7f73fea00640 1 --2- 172.21.15.5:0/2868310610 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f73e40a2c00 0x7f73e40a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:46:38.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:38.583+0000 7f73fea00640 1 --2- 172.21.15.5:0/2868310610 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f73e40a2c00 0x7f73e40a50c0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f73f0006010 tx=0x7f73f00093f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:46:38.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:38.586+0000 7f73f7e00640 1 -- 172.21.15.5:0/2868310610 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f73e809c220 con 0x7f7400077040 2024-09-10T09:46:38.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:38.825+0000 7f73ffe00640 1 -- 172.21.15.5:0/2868310610 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f73dc005580 con 0x7f7400077040 2024-09-10T09:46:38.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:38.826+0000 7f73f7e00640 1 -- 172.21.15.5:0/2868310610 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) ==== 94+0+11970 (secure 0 0 0) 0x7f73e809bb20 con 0x7f7400077040 2024-09-10T09:46:38.827 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-10T09:46:38.827 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":35,"btime":"2024-09-10T09:46:17:834001+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14454,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/2970594311","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2970594311},{"type":"v1","addr":"172.21.15.5:6835","nonce":2970594311}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14514,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6839/3616502174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":3616502174},{"type":"v1","addr":"172.21.15.5:6839","nonce":3616502174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":35,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T09:35:29.626287+0000","modified":"2024-09-10T09:46:16.966068+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24542,"mds_1":24535,"mds_2":24511,"mds_3":24506,"mds_4":24443},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14520":{"gid":14520,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/2073599842","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":2073599842},{"type":"v1","addr":"172.21.15.5:6841","nonce":2073599842}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"k","rank":4,"incarnation":26,"state":"up:active","state_seq":24,"addr":"172.21.15.102:6833/2457007427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":2457007427},{"type":"v1","addr":"172.21.15.102:6833","nonce":2457007427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/356445159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":356445159},{"type":"v1","addr":"172.21.15.159:6833","nonce":356445159}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6837/2894314507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":2894314507},{"type":"v1","addr":"172.21.15.5:6837","nonce":2894314507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.102:6835/1647549111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":1647549111},{"type":"v1","addr":"172.21.15.102:6835","nonce":1647549111}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6835/952657242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":952657242},{"type":"v1","addr":"172.21.15.159:6835","nonce":952657242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":19,"addr":"172.21.15.102:6837/2689776764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":2689776764},{"type":"v1","addr":"172.21.15.102:6837","nonce":2689776764}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.159:6837/620627639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":620627639},{"type":"v1","addr":"172.21.15.159:6837","nonce":620627639}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24535":{"gid":24535,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":18,"addr":"172.21.15.159:6839/2073739599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":2073739599},{"type":"v1","addr":"172.21.15.159:6839","nonce":2073739599}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24542":{"gid":24542,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.102:6839/335409749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6838","nonce":335409749},{"type":"v1","addr":"172.21.15.102:6839","nonce":335409749}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24542,"qdb_cluster":[24443,24506,24511,24535,24542]},"id":1}]} 2024-09-10T09:46:38.828 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 35 2024-09-10T09:46:38.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:38.828+0000 7f73ffe00640 1 -- 172.21.15.5:0/2868310610 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f73e40a2c00 msgr2=0x7f73e40a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:46:38.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:38.828+0000 7f73ffe00640 1 --2- 172.21.15.5:0/2868310610 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f73e40a2c00 0x7f73e40a50c0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f73f0006010 tx=0x7f73f00093f0 comp rx=0 tx=0).stop 2024-09-10T09:46:38.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:38.828+0000 7f73ffe00640 1 -- 172.21.15.5:0/2868310610 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7400077040 msgr2=0x7f74001a4b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:46:38.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:38.828+0000 7f73ffe00640 1 --2- 172.21.15.5:0/2868310610 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7400077040 0x7f74001a4b40 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f73e800e9f0 tx=0x7f73e800eeb0 comp rx=0 tx=0).stop 2024-09-10T09:46:38.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:38.829+0000 7f73ffe00640 1 -- 172.21.15.5:0/2868310610 shutdown_connections 2024-09-10T09:46:38.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:38.829+0000 7f73ffe00640 1 --2- 172.21.15.5:0/2868310610 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f73e40a2c00 0x7f73e40a50c0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:38.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:38.829+0000 7f73ffe00640 1 --2- 172.21.15.5:0/2868310610 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7400112870 0x7f740019e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:38.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:38.829+0000 7f73ffe00640 1 --2- 172.21.15.5:0/2868310610 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7400077040 0x7f74001a4b40 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:38.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:38.829+0000 7f73ffe00640 1 --2- 172.21.15.5:0/2868310610 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7400076720 0x7f74001a4600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:38.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:38.829+0000 7f73ffe00640 1 -- 172.21.15.5:0/2868310610 >> 172.21.15.5:0/2868310610 conn(0x7f74000ff8d0 msgr2=0x7f7400101cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:46:38.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:38.829+0000 7f73ffe00640 1 -- 172.21.15.5:0/2868310610 shutdown_connections 2024-09-10T09:46:38.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:38.829+0000 7f73ffe00640 1 -- 172.21.15.5:0/2868310610 wait complete. 2024-09-10T09:46:39.075 INFO:tasks.workunit.client.0.smithi005.stdout: 1 496422 6.36 MB/sec execute 260 sec latency 1093.329 ms 2024-09-10T09:46:39.085 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:46:38 smithi005 ceph-mon[30350]: pgmap v642: 129 pgs: 129 active+clean; 172 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 461 KiB/s rd, 2.3 MiB/s wr, 176 op/s 2024-09-10T09:46:39.085 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:46:38 smithi005 ceph-mon[30350]: from='client.? 172.21.15.5:0/2868310610' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:46:39.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:46:38 smithi159 ceph-mon[35513]: pgmap v642: 129 pgs: 129 active+clean; 172 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 461 KiB/s rd, 2.3 MiB/s wr, 176 op/s 2024-09-10T09:46:39.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:46:38 smithi159 ceph-mon[35513]: from='client.? 172.21.15.5:0/2868310610' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:46:39.386 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:46:38 smithi102 ceph-mon[35442]: pgmap v642: 129 pgs: 129 active+clean; 172 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 461 KiB/s rd, 2.3 MiB/s wr, 176 op/s 2024-09-10T09:46:39.386 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:46:38 smithi102 ceph-mon[35442]: from='client.? 172.21.15.5:0/2868310610' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:46:39.797 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24542 quiesce db / --timeout 90 --expiration 133.4 --await-for 45.0' 2024-09-10T09:46:39.798 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24542 quiesce db / --timeout 90 --expiration 133.4 --await-for 45.0 2024-09-10T09:46:40.075 INFO:tasks.workunit.client.0.smithi005.stdout: 1 498190 6.38 MB/sec execute 261 sec latency 1180.049 ms 2024-09-10T09:46:40.246 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:46:40.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:40.944+0000 7f289be00640 1 -- 172.21.15.5:0/864141066 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f289c074510 msgr2=0x7f289c0748f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:46:40.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:40.944+0000 7f289be00640 1 --2- 172.21.15.5:0/864141066 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f289c074510 0x7f289c0748f0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f288c009900 tx=0x7f288c02a480 comp rx=0 tx=0).stop 2024-09-10T09:46:40.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:40.944+0000 7f289be00640 1 -- 172.21.15.5:0/864141066 shutdown_connections 2024-09-10T09:46:40.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:40.944+0000 7f289be00640 1 --2- 172.21.15.5:0/864141066 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f289c10e280 0x7f289c110670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:40.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:40.944+0000 7f289be00640 1 --2- 172.21.15.5:0/864141066 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f289c074e30 0x7f289c10dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:40.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:40.944+0000 7f289be00640 1 --2- 172.21.15.5:0/864141066 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f289c074510 0x7f289c0748f0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:40.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:40.944+0000 7f289be00640 1 -- 172.21.15.5:0/864141066 >> 172.21.15.5:0/864141066 conn(0x7f289c0fd740 msgr2=0x7f289c0ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:46:40.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:40.944+0000 7f289be00640 1 -- 172.21.15.5:0/864141066 shutdown_connections 2024-09-10T09:46:40.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:40.944+0000 7f289be00640 1 -- 172.21.15.5:0/864141066 wait complete. 2024-09-10T09:46:40.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:40.945+0000 7f289be00640 1 Processor -- start 2024-09-10T09:46:40.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:40.945+0000 7f289be00640 1 -- start start 2024-09-10T09:46:40.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:40.945+0000 7f289be00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f289c074510 0x7f289c19a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:46:40.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:40.945+0000 7f289be00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f289c074e30 0x7f289c19aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:46:40.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:40.945+0000 7f289be00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f289c10e280 0x7f289c19ee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:46:40.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:40.945+0000 7f289be00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f289c113fc0 con 0x7f289c074e30 2024-09-10T09:46:40.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:40.945+0000 7f289be00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f289c113e40 con 0x7f289c10e280 2024-09-10T09:46:40.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:40.945+0000 7f289be00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f289c114140 con 0x7f289c074510 2024-09-10T09:46:40.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:40.946+0000 7f289aa00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f289c074510 0x7f289c19a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:46:40.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:40.946+0000 7f289aa00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f289c074510 0x7f289c19a540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:55042/0 (socket says 172.21.15.5:55042) 2024-09-10T09:46:40.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:40.946+0000 7f289aa00640 1 -- 172.21.15.5:0/2283328182 learned_addr learned my addr 172.21.15.5:0/2283328182 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:46:40.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:40.946+0000 7f289b400640 1 --2- 172.21.15.5:0/2283328182 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f289c10e280 0x7f289c19ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:46:40.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:40.946+0000 7f289a000640 1 --2- 172.21.15.5:0/2283328182 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f289c074e30 0x7f289c19aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:46:40.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:40.946+0000 7f289aa00640 1 -- 172.21.15.5:0/2283328182 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f289c10e280 msgr2=0x7f289c19ee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:46:40.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:40.946+0000 7f289aa00640 1 --2- 172.21.15.5:0/2283328182 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f289c10e280 0x7f289c19ee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:40.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:40.946+0000 7f289aa00640 1 -- 172.21.15.5:0/2283328182 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f289c074e30 msgr2=0x7f289c19aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:46:40.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:40.946+0000 7f289aa00640 1 --2- 172.21.15.5:0/2283328182 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f289c074e30 0x7f289c19aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:40.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:40.946+0000 7f289aa00640 1 -- 172.21.15.5:0/2283328182 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f289c19f4f0 con 0x7f289c074510 2024-09-10T09:46:40.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:40.946+0000 7f289a000640 1 --2- 172.21.15.5:0/2283328182 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f289c074e30 0x7f289c19aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:46:40.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:40.947+0000 7f289aa00640 1 --2- 172.21.15.5:0/2283328182 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f289c074510 0x7f289c19a540 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f288c004770 tx=0x7f288c003c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:46:40.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:40.947+0000 7f2893400640 1 -- 172.21.15.5:0/2283328182 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f288c03f070 con 0x7f289c074510 2024-09-10T09:46:40.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:40.947+0000 7f2893400640 1 -- 172.21.15.5:0/2283328182 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f288c031ca0 con 0x7f289c074510 2024-09-10T09:46:40.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:40.947+0000 7f2893400640 1 -- 172.21.15.5:0/2283328182 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f288c035030 con 0x7f289c074510 2024-09-10T09:46:40.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:40.947+0000 7f289be00640 1 -- 172.21.15.5:0/2283328182 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f289c19f780 con 0x7f289c074510 2024-09-10T09:46:40.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:40.947+0000 7f289be00640 1 -- 172.21.15.5:0/2283328182 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f289c1a70c0 con 0x7f289c074510 2024-09-10T09:46:40.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:40.951+0000 7f28a4256740 1 Processor -- start 2024-09-10T09:46:40.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:40.951+0000 7f2893400640 1 -- 172.21.15.5:0/2283328182 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f288c042070 con 0x7f289c074510 2024-09-10T09:46:40.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:40.951+0000 7f28a4256740 1 -- start start 2024-09-10T09:46:40.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:40.951+0000 7f28a4256740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d9968c0560 0x55d9968c0940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:46:40.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:40.951+0000 7f28a4256740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d9968c0e80 0x55d9968cb200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:46:40.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:40.951+0000 7f28a4256740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d9968cb740 0x55d9968cdb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:46:40.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:40.951+0000 7f28a4256740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55d996826530 con 0x55d9968cb740 2024-09-10T09:46:40.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:40.951+0000 7f28a4256740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55d9967cd860 con 0x55d9968c0e80 2024-09-10T09:46:40.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:40.951+0000 7f28a4256740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55d9968224f0 con 0x55d9968c0560 2024-09-10T09:46:40.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:40.952+0000 7f289b400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d9968c0560 0x55d9968c0940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:46:40.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:40.952+0000 7f289b400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d9968c0560 0x55d9968c0940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:55056/0 (socket says 172.21.15.5:55056) 2024-09-10T09:46:40.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:40.952+0000 7f289b400640 1 -- 172.21.15.5:0/3948270920 learned_addr learned my addr 172.21.15.5:0/3948270920 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:46:40.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:40.952+0000 7f289a000640 1 --2- 172.21.15.5:0/3948270920 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d9968cb740 0x55d9968cdb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:46:40.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:40.952+0000 7f289b400640 1 -- 172.21.15.5:0/3948270920 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d9968c0e80 msgr2=0x55d9968cb200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:46:40.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:40.952+0000 7f289b400640 1 --2- 172.21.15.5:0/3948270920 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d9968c0e80 0x55d9968cb200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:40.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:40.952+0000 7f289b400640 1 -- 172.21.15.5:0/3948270920 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d9968cb740 msgr2=0x55d9968cdb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:46:40.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:40.952+0000 7f289b400640 1 --2- 172.21.15.5:0/3948270920 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d9968cb740 0x55d9968cdb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:40.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:40.952+0000 7f289b400640 1 -- 172.21.15.5:0/3948270920 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d996597b90 con 0x55d9968c0560 2024-09-10T09:46:40.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:40.952+0000 7f289b400640 1 --2- 172.21.15.5:0/3948270920 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d9968c0560 0x55d9968c0940 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f2894009f30 tx=0x7f28940118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8c263a2e12d4b699 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:46:40.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:40.953+0000 7f2893400640 1 --2- 172.21.15.5:0/2283328182 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f28800a2c00 0x7f28800a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:46:40.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:40.953+0000 7f2887e00640 1 -- 172.21.15.5:0/3948270920 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2894004d20 con 0x55d9968c0560 2024-09-10T09:46:40.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:40.953+0000 7f2887e00640 1 -- 172.21.15.5:0/3948270920 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2894004050 con 0x55d9968c0560 2024-09-10T09:46:40.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:40.953+0000 7f2887e00640 1 -- 172.21.15.5:0/3948270920 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2894005260 con 0x55d9968c0560 2024-09-10T09:46:40.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:40.953+0000 7f2893400640 1 -- 172.21.15.5:0/2283328182 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f288c0efa50 con 0x7f289c074510 2024-09-10T09:46:40.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:40.953+0000 7f289aa00640 1 --2- 172.21.15.5:0/2283328182 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f28800a2c00 0x7f28800a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:46:40.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:40.953+0000 7f28a4256740 1 -- 172.21.15.5:0/3948270920 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d9968c0560 msgr2=0x55d9968c0940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:46:40.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:40.953+0000 7f28a4256740 1 --2- 172.21.15.5:0/3948270920 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d9968c0560 0x55d9968c0940 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f2894009f30 tx=0x7f28940118d0 comp rx=0 tx=0).stop 2024-09-10T09:46:40.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:40.953+0000 7f289aa00640 1 --2- 172.21.15.5:0/2283328182 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f28800a2c00 0x7f28800a50c0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f288c0c4660 tx=0x7f288c03b460 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:46:40.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:40.954+0000 7f28a4256740 1 -- 172.21.15.5:0/3948270920 shutdown_connections 2024-09-10T09:46:40.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:40.954+0000 7f28a4256740 1 --2- 172.21.15.5:0/3948270920 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d9968cb740 0x55d9968cdb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:40.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:40.954+0000 7f28a4256740 1 --2- 172.21.15.5:0/3948270920 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d9968c0e80 0x55d9968cb200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:40.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:40.954+0000 7f28a4256740 1 --2- 172.21.15.5:0/3948270920 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d9968c0560 0x55d9968c0940 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:40.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:40.954+0000 7f28a4256740 1 -- 172.21.15.5:0/3948270920 >> 172.21.15.5:0/3948270920 conn(0x55d9968bcaf0 msgr2=0x55d9968bced0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:46:40.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:40.954+0000 7f28a4256740 1 -- 172.21.15.5:0/3948270920 shutdown_connections 2024-09-10T09:46:40.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:40.954+0000 7f28a4256740 1 -- 172.21.15.5:0/3948270920 wait complete. 2024-09-10T09:46:40.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:40.954+0000 7f28a4256740 1 -- start start 2024-09-10T09:46:40.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:40.954+0000 7f28a4256740 1 Processor -- start 2024-09-10T09:46:40.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:40.955+0000 7f28a4256740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d9968c0560 0x55d9968c8620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:46:40.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:40.955+0000 7f28a4256740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d9968c0e80 0x55d9968c8b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:46:40.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:40.955+0000 7f28a4256740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d9968cb740 0x55d9968c90a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:46:40.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:40.955+0000 7f28a4256740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55d9968ce310 con 0x55d9968c0560 2024-09-10T09:46:40.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:40.955+0000 7f28a4256740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55d9967e0e60 con 0x55d9968cb740 2024-09-10T09:46:40.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:40.955+0000 7f28a4256740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55d9968ce490 con 0x55d9968c0e80 2024-09-10T09:46:40.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:40.955+0000 7f28a4256740 10 client.0 _ll_register_callbacks cb 0x55d9967e5cc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:46:40.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:40.955+0000 7f289a000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d9968c0e80 0x55d9968c8b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:46:40.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:40.955+0000 7f289b400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d9968c0560 0x55d9968c8620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:46:40.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:40.955+0000 7f289a000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d9968c0e80 0x55d9968c8b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:55066/0 (socket says 172.21.15.5:55066) 2024-09-10T09:46:40.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:40.955+0000 7f289a000640 1 -- 172.21.15.5:0/661901100 learned_addr learned my addr 172.21.15.5:0/661901100 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:46:40.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:40.956+0000 7f289b400640 1 --2- 172.21.15.5:0/661901100 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d9968cb740 0x55d9968c90a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:46:40.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:40.956+0000 7f289a000640 1 -- 172.21.15.5:0/661901100 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d9968cb740 msgr2=0x55d9968c90a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:46:40.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:40.956+0000 7f289a000640 1 --2- 172.21.15.5:0/661901100 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d9968cb740 0x55d9968c90a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:40.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:40.956+0000 7f289a000640 1 -- 172.21.15.5:0/661901100 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d9968c0560 msgr2=0x55d9968c8620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:46:40.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:40.956+0000 7f289a000640 1 --2- 172.21.15.5:0/661901100 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d9968c0560 0x55d9968c8620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:40.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:40.956+0000 7f289a000640 1 -- 172.21.15.5:0/661901100 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d9967ec990 con 0x55d9968c0e80 2024-09-10T09:46:40.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:40.956+0000 7f289a000640 1 --2- 172.21.15.5:0/661901100 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d9968c0e80 0x55d9968c8b60 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f2888013790 tx=0x7f2888013c50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2a92114d0d141639 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:46:40.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:40.956+0000 7f2884c00640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-09-10T09:46:40.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:40.956+0000 7f2884c00640 1 -- 172.21.15.5:0/661901100 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2888020060 con 0x55d9968c0e80 2024-09-10T09:46:40.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:40.957+0000 7f2884c00640 1 -- 172.21.15.5:0/661901100 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f2888011d50 con 0x55d9968c0e80 2024-09-10T09:46:40.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:40.957+0000 7f28a4256740 1 -- 172.21.15.5:0/661901100 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2894004050 con 0x55d9968c0e80 2024-09-10T09:46:40.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:40.957+0000 7f2884c00640 1 -- 172.21.15.5:0/661901100 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f2888027050 con 0x55d9968c0e80 2024-09-10T09:46:40.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:40.957+0000 7f28a4256740 10 client.27028 fetch_fsmap learned FSMap version 35 2024-09-10T09:46:40.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:40.957+0000 7f28a4256740 1 -- 172.21.15.5:0/661901100 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55d9968d9f70 con 0x55d9968c0e80 2024-09-10T09:46:40.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:40.958+0000 7f2884c00640 1 -- 172.21.15.5:0/661901100 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2894004050 con 0x55d9968c0e80 2024-09-10T09:46:40.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:40.958+0000 7f2884c00640 1 -- 172.21.15.5:0/661901100 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 35) ==== 8374+0+0 (secure 0 0 0) 0x7f288801a030 con 0x55d9968c0e80 2024-09-10T09:46:40.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:40.958+0000 7f28a4256740 10 client.27028 fetch_fsmap finished waiting for FSMap version 35 2024-09-10T09:46:40.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:40.958+0000 7f28a4256740 10 client.27028 resolve_mds: validated gid 24542 aka daemon mds.b 2024-09-10T09:46:40.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:40.958+0000 7f28a4256740 20 client.27028 populate_metadata read hostname 'smithi005' 2024-09-10T09:46:40.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:40.958+0000 7f28a4256740 1 --2- 172.21.15.5:0/661901100 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55d9968db430 0x55d9968dd840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:46:40.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:40.958+0000 7f28a4256740 4 client.27028 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:46:40.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:40.958+0000 7f28a4256740 1 -- 172.21.15.5:0/661901100 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d9967c9310 con 0x55d9968db430 2024-09-10T09:46:40.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:40.958+0000 7f289b400640 1 --2- 172.21.15.5:0/661901100 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55d9968db430 0x55d9968dd840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:46:40.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:40.959+0000 7f289b400640 1 --2- 172.21.15.5:0/661901100 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55d9968db430 0x55d9968dd840 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=58c0514e91340dd3 server_cookie=b1a0eab530a13ab6 in_seq=0 out_seq=0 2024-09-10T09:46:40.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:40.959+0000 7f2884c00640 10 client.27028 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T09:46:40.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:40.959+0000 7f2884c00640 1 -- 172.21.15.5:0/661901100 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d9967c9310 con 0x55d9968db430 2024-09-10T09:46:40.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:40.959+0000 7f2884c00640 10 client.27028 handle_command_reply: tid=0 2024-09-10T09:46:40.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:40.960+0000 7f28a4256740 1 client.27028 shutdown 2024-09-10T09:46:40.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:40.960+0000 7f28a4256740 1 -- 172.21.15.5:0/661901100 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d9968c0e80 msgr2=0x55d9968c8b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:46:40.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:40.960+0000 7f28a4256740 1 --2- 172.21.15.5:0/661901100 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d9968c0e80 0x55d9968c8b60 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f2888013790 tx=0x7f2888013c50 comp rx=0 tx=0).stop 2024-09-10T09:46:40.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:40.960+0000 7f28a4256740 1 -- 172.21.15.5:0/661901100 shutdown_connections 2024-09-10T09:46:40.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:40.960+0000 7f28a4256740 1 --2- 172.21.15.5:0/661901100 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55d9968db430 0x55d9968dd840 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:40.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:40.960+0000 7f28a4256740 1 --2- 172.21.15.5:0/661901100 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d9968cb740 0x55d9968c90a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:40.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:40.960+0000 7f28a4256740 1 --2- 172.21.15.5:0/661901100 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d9968c0e80 0x55d9968c8b60 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:40.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:40.960+0000 7f28a4256740 1 --2- 172.21.15.5:0/661901100 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d9968c0560 0x55d9968c8620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:40.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:40.960+0000 7f28a4256740 1 -- 172.21.15.5:0/661901100 >> 172.21.15.5:0/661901100 conn(0x55d9968bcaf0 msgr2=0x55d9968c27d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:46:40.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:40.960+0000 7f2884c00640 0 client.27028 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T09:46:40.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:40.960+0000 7f28a4256740 1 -- 172.21.15.5:0/661901100 shutdown_connections 2024-09-10T09:46:40.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:40.960+0000 7f28a4256740 1 -- 172.21.15.5:0/661901100 wait complete. 2024-09-10T09:46:40.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:40.960+0000 7f28a4256740 20 client.27028 trim_cache size 0 max 16384 2024-09-10T09:46:40.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:40.983+0000 7f28a4256740 1 Processor -- start 2024-09-10T09:46:40.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:40.983+0000 7f28a4256740 1 -- start start 2024-09-10T09:46:40.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:40.983+0000 7f28a4256740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d9968c0560 0x55d9968c7560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:46:40.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:40.983+0000 7f28a4256740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d9968c0e80 0x55d9968c2fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:46:40.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:40.983+0000 7f28a4256740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d9968cb740 0x55d9968e3150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:46:40.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:40.983+0000 7f28a4256740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f289c1b0f10 con 0x55d9968c0560 2024-09-10T09:46:40.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:40.983+0000 7f28a4256740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f289c1b2a80 con 0x55d9968c0e80 2024-09-10T09:46:40.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:40.983+0000 7f28a4256740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f289c1b15e0 con 0x55d9968cb740 2024-09-10T09:46:40.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:40.983+0000 7f289b400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d9968c0560 0x55d9968c7560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:46:40.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:40.983+0000 7f289b400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d9968c0560 0x55d9968c7560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46260/0 (socket says 172.21.15.5:46260) 2024-09-10T09:46:40.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:40.983+0000 7f289a000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d9968c0e80 0x55d9968c2fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:46:40.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:40.983+0000 7f289b400640 1 -- 172.21.15.5:0/3516525214 learned_addr learned my addr 172.21.15.5:0/3516525214 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:46:40.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:40.984+0000 7f289b400640 1 --2- 172.21.15.5:0/3516525214 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d9968cb740 0x55d9968e3150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:46:40.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:40.984+0000 7f289a000640 1 -- 172.21.15.5:0/3516525214 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d9968cb740 msgr2=0x55d9968e3150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:46:40.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:40.984+0000 7f289a000640 1 --2- 172.21.15.5:0/3516525214 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d9968cb740 0x55d9968e3150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:40.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:40.984+0000 7f289a000640 1 -- 172.21.15.5:0/3516525214 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d9968c0560 msgr2=0x55d9968c7560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:46:40.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:40.984+0000 7f289a000640 1 --2- 172.21.15.5:0/3516525214 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d9968c0560 0x55d9968c7560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:40.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:40.984+0000 7f289a000640 1 -- 172.21.15.5:0/3516525214 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d9968bc920 con 0x55d9968c0e80 2024-09-10T09:46:40.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:40.984+0000 7f289b400640 1 --2- 172.21.15.5:0/3516525214 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d9968c0560 0x55d9968c7560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:46:40.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:40.984+0000 7f289a000640 1 --2- 172.21.15.5:0/3516525214 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d9968c0e80 0x55d9968c2fc0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f288800e660 tx=0x7f2888009530 comp rx=0 tx=0).ready entity=mon.2 client_cookie=de002c4e42bc6e66 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:46:40.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:40.984+0000 7f2887400640 1 -- 172.21.15.5:0/3516525214 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2888030070 con 0x55d9968c0e80 2024-09-10T09:46:40.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:40.984+0000 7f2887400640 1 -- 172.21.15.5:0/3516525214 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f288800ea50 con 0x55d9968c0e80 2024-09-10T09:46:40.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:40.984+0000 7f2887400640 1 -- 172.21.15.5:0/3516525214 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28880035f0 con 0x55d9968c0e80 2024-09-10T09:46:40.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:40.985+0000 7f28a4256740 1 -- 172.21.15.5:0/3516525214 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d9968c0e80 msgr2=0x55d9968c2fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:46:40.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:40.985+0000 7f28a4256740 1 --2- 172.21.15.5:0/3516525214 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d9968c0e80 0x55d9968c2fc0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f288800e660 tx=0x7f2888009530 comp rx=0 tx=0).stop 2024-09-10T09:46:40.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:40.985+0000 7f28a4256740 1 -- 172.21.15.5:0/3516525214 shutdown_connections 2024-09-10T09:46:40.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:40.985+0000 7f28a4256740 1 --2- 172.21.15.5:0/3516525214 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d9968cb740 0x55d9968e3150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:40.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:40.985+0000 7f28a4256740 1 --2- 172.21.15.5:0/3516525214 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d9968c0e80 0x55d9968c2fc0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:40.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:40.985+0000 7f28a4256740 1 --2- 172.21.15.5:0/3516525214 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d9968c0560 0x55d9968c7560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:40.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:40.985+0000 7f28a4256740 1 -- 172.21.15.5:0/3516525214 >> 172.21.15.5:0/3516525214 conn(0x55d9968bcaf0 msgr2=0x55d9968ca590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:46:40.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:40.985+0000 7f28a4256740 1 -- 172.21.15.5:0/3516525214 shutdown_connections 2024-09-10T09:46:40.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:40.985+0000 7f28a4256740 1 -- 172.21.15.5:0/3516525214 wait complete. 2024-09-10T09:46:40.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:40.986+0000 7f28a4256740 1 -- start start 2024-09-10T09:46:40.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:40.986+0000 7f28a4256740 1 Processor -- start 2024-09-10T09:46:40.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:40.986+0000 7f28a4256740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d9968c0560 0x55d9968d36e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:46:40.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:40.986+0000 7f28a4256740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d9968c0e80 0x55d9968d3c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:46:40.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:40.986+0000 7f28a4256740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d9968cb740 0x55d9968d06b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:46:40.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:40.986+0000 7f28a4256740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f289c1b2380 con 0x55d9968c0560 2024-09-10T09:46:40.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:40.986+0000 7f28a4256740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f289c1b1cb0 con 0x55d9968cb740 2024-09-10T09:46:40.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:40.986+0000 7f28a4256740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f289c1aeef0 con 0x55d9968c0e80 2024-09-10T09:46:40.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:40.986+0000 7f28a4256740 10 client.0 _ll_register_callbacks cb 0x7f2888009650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:46:40.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:40.987+0000 7f289a000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d9968c0e80 0x55d9968d3c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:46:40.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:40.987+0000 7f289b400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d9968c0560 0x55d9968d36e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:46:40.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:40.987+0000 7f289a000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d9968c0e80 0x55d9968d3c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:55082/0 (socket says 172.21.15.5:55082) 2024-09-10T09:46:40.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:40.987+0000 7f289a000640 1 -- 172.21.15.5:0/3764439322 learned_addr learned my addr 172.21.15.5:0/3764439322 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:46:40.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:40.987+0000 7f289a000640 1 -- 172.21.15.5:0/3764439322 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d9968cb740 msgr2=0x55d9968d06b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:46:40.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:40.987+0000 7f289a000640 1 --2- 172.21.15.5:0/3764439322 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d9968cb740 0x55d9968d06b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:40.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:40.987+0000 7f289a000640 1 -- 172.21.15.5:0/3764439322 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d9968c0560 msgr2=0x55d9968d36e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:46:40.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:40.987+0000 7f289a000640 1 --2- 172.21.15.5:0/3764439322 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d9968c0560 0x55d9968d36e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:40.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:40.987+0000 7f289a000640 1 -- 172.21.15.5:0/3764439322 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d9968b8310 con 0x55d9968c0e80 2024-09-10T09:46:40.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:40.987+0000 7f289a000640 1 --2- 172.21.15.5:0/3764439322 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d9968c0e80 0x55d9968d3c20 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f2888013c90 tx=0x7f2888005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1021c5c571f78dca server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:46:40.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:40.987+0000 7f2884c00640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-09-10T09:46:40.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:40.988+0000 7f2884c00640 1 -- 172.21.15.5:0/3764439322 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2888030070 con 0x55d9968c0e80 2024-09-10T09:46:40.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:40.988+0000 7f2884c00640 1 -- 172.21.15.5:0/3764439322 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f2888005550 con 0x55d9968c0e80 2024-09-10T09:46:40.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:40.988+0000 7f28a4256740 1 -- 172.21.15.5:0/3764439322 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f288800ea50 con 0x55d9968c0e80 2024-09-10T09:46:40.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:40.988+0000 7f2884c00640 1 -- 172.21.15.5:0/3764439322 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f288800ee90 con 0x55d9968c0e80 2024-09-10T09:46:40.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:40.988+0000 7f28a4256740 10 client.27034 fetch_fsmap learned FSMap version 35 2024-09-10T09:46:40.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:40.988+0000 7f28a4256740 1 -- 172.21.15.5:0/3764439322 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55d9969465c0 con 0x55d9968c0e80 2024-09-10T09:46:40.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:40.988+0000 7f2884c00640 1 -- 172.21.15.5:0/3764439322 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f288800ea50 con 0x55d9968c0e80 2024-09-10T09:46:40.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:40.988+0000 7f2884c00640 1 -- 172.21.15.5:0/3764439322 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 35) ==== 8374+0+0 (secure 0 0 0) 0x7f288801a040 con 0x55d9968c0e80 2024-09-10T09:46:40.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:40.988+0000 7f28a4256740 10 client.27034 fetch_fsmap finished waiting for FSMap version 35 2024-09-10T09:46:40.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:40.988+0000 7f28a4256740 10 client.27034 resolve_mds: validated gid 24542 aka daemon mds.b 2024-09-10T09:46:40.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:40.988+0000 7f28a4256740 20 client.27034 populate_metadata read hostname 'smithi005' 2024-09-10T09:46:40.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:40.988+0000 7f28a4256740 1 --2- 172.21.15.5:0/3764439322 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55d996946cc0 0x55d9969490e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:46:40.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:40.989+0000 7f28a4256740 4 client.27034 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 133.4, "await_for": 45.0}] 2024-09-10T09:46:40.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:40.989+0000 7f28a4256740 1 -- 172.21.15.5:0/3764439322 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 133.4, "await_for": 45.0}) -- 0x55d996833c60 con 0x55d996946cc0 2024-09-10T09:46:40.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:40.989+0000 7f289b400640 1 --2- 172.21.15.5:0/3764439322 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55d996946cc0 0x55d9969490e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:46:40.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:40.989+0000 7f289b400640 1 --2- 172.21.15.5:0/3764439322 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55d996946cc0 0x55d9969490e0 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dd012bc57bc02ac6 server_cookie=1f91d6c8c1115323 in_seq=0 out_seq=0 2024-09-10T09:46:40.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:40.989+0000 7f2884c00640 10 client.27034 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T09:46:41.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:46:40 smithi005 ceph-mon[30350]: pgmap v643: 129 pgs: 129 active+clean; 172 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 500 KiB/s rd, 2.3 MiB/s wr, 188 op/s 2024-09-10T09:46:41.075 INFO:tasks.workunit.client.0.smithi005.stdout: 1 500929 6.42 MB/sec execute 262 sec latency 77.373 ms 2024-09-10T09:46:41.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:41.090+0000 7f2884c00640 1 -- 172.21.15.5:0/3764439322 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55d996833c60 con 0x55d996946cc0 2024-09-10T09:46:41.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:41.090+0000 7f2884c00640 10 client.27034 handle_command_reply: tid=0 2024-09-10T09:46:41.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:41.090+0000 7f28a4256740 1 client.27034 shutdown 2024-09-10T09:46:41.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:41.090+0000 7f28a4256740 1 -- 172.21.15.5:0/3764439322 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d9968c0e80 msgr2=0x55d9968d3c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:46:41.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:41.090+0000 7f28a4256740 1 --2- 172.21.15.5:0/3764439322 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d9968c0e80 0x55d9968d3c20 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f2888013c90 tx=0x7f2888005f90 comp rx=0 tx=0).stop 2024-09-10T09:46:41.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:41.090+0000 7f28a4256740 1 -- 172.21.15.5:0/3764439322 shutdown_connections 2024-09-10T09:46:41.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:41.090+0000 7f28a4256740 1 --2- 172.21.15.5:0/3764439322 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55d996946cc0 0x55d9969490e0 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:41.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:41.090+0000 7f28a4256740 1 --2- 172.21.15.5:0/3764439322 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d9968cb740 0x55d9968d06b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:41.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:41.090+0000 7f28a4256740 1 --2- 172.21.15.5:0/3764439322 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d9968c0e80 0x55d9968d3c20 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:41.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:41.090+0000 7f28a4256740 1 --2- 172.21.15.5:0/3764439322 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d9968c0560 0x55d9968d36e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:41.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:41.090+0000 7f2884c00640 0 client.27034 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T09:46:41.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:41.090+0000 7f28a4256740 1 -- 172.21.15.5:0/3764439322 >> 172.21.15.5:0/3764439322 conn(0x55d9968bcaf0 msgr2=0x55d9968bb450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:46:41.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:41.090+0000 7f28a4256740 1 -- 172.21.15.5:0/3764439322 shutdown_connections 2024-09-10T09:46:41.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:41.090+0000 7f28a4256740 1 -- 172.21.15.5:0/3764439322 wait complete. 2024-09-10T09:46:41.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:41.091+0000 7f28a4256740 20 client.27034 trim_cache size 0 max 16384 2024-09-10T09:46:41.093 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-10T09:46:41.093 INFO:teuthology.orchestra.run.smithi005.stdout: "epoch": 35, 2024-09-10T09:46:41.093 INFO:teuthology.orchestra.run.smithi005.stdout: "leader": 24542, 2024-09-10T09:46:41.093 INFO:teuthology.orchestra.run.smithi005.stdout: "set_version": 26, 2024-09-10T09:46:41.093 INFO:teuthology.orchestra.run.smithi005.stdout: "sets": { 2024-09-10T09:46:41.093 INFO:teuthology.orchestra.run.smithi005.stdout: "fcdeb12e": { 2024-09-10T09:46:41.093 INFO:teuthology.orchestra.run.smithi005.stdout: "version": 26, 2024-09-10T09:46:41.094 INFO:teuthology.orchestra.run.smithi005.stdout: "age_ref": 0.0, 2024-09-10T09:46:41.094 INFO:teuthology.orchestra.run.smithi005.stdout: "state": { 2024-09-10T09:46:41.094 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "QUIESCED", 2024-09-10T09:46:41.094 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0 2024-09-10T09:46:41.094 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T09:46:41.094 INFO:teuthology.orchestra.run.smithi005.stdout: "timeout": 90.0, 2024-09-10T09:46:41.094 INFO:teuthology.orchestra.run.smithi005.stdout: "expiration": 133.4, 2024-09-10T09:46:41.094 INFO:teuthology.orchestra.run.smithi005.stdout: "members": { 2024-09-10T09:46:41.094 INFO:teuthology.orchestra.run.smithi005.stdout: "file:/": { 2024-09-10T09:46:41.094 INFO:teuthology.orchestra.run.smithi005.stdout: "excluded": false, 2024-09-10T09:46:41.094 INFO:teuthology.orchestra.run.smithi005.stdout: "state": { 2024-09-10T09:46:41.094 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "QUIESCED", 2024-09-10T09:46:41.094 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0 2024-09-10T09:46:41.095 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T09:46:41.095 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T09:46:41.095 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T09:46:41.095 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T09:46:41.095 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T09:46:41.095 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-10T09:46:41.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:41.091+0000 7f289be00640 1 -- 172.21.15.5:0/2283328182 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f28800a2c00 msgr2=0x7f28800a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:46:41.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:41.091+0000 7f289be00640 1 --2- 172.21.15.5:0/2283328182 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f28800a2c00 0x7f28800a50c0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f288c0c4660 tx=0x7f288c03b460 comp rx=0 tx=0).stop 2024-09-10T09:46:41.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:41.091+0000 7f289be00640 1 -- 172.21.15.5:0/2283328182 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f289c074510 msgr2=0x7f289c19a540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:46:41.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:41.091+0000 7f289be00640 1 --2- 172.21.15.5:0/2283328182 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f289c074510 0x7f289c19a540 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f288c004770 tx=0x7f288c003c10 comp rx=0 tx=0).stop 2024-09-10T09:46:41.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:41.091+0000 7f289be00640 1 -- 172.21.15.5:0/2283328182 shutdown_connections 2024-09-10T09:46:41.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:41.091+0000 7f289be00640 1 --2- 172.21.15.5:0/2283328182 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f28800a2c00 0x7f28800a50c0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:41.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:41.091+0000 7f289be00640 1 --2- 172.21.15.5:0/2283328182 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f289c10e280 0x7f289c19ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:41.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:41.091+0000 7f289be00640 1 --2- 172.21.15.5:0/2283328182 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f289c074e30 0x7f289c19aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:41.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:41.091+0000 7f289be00640 1 --2- 172.21.15.5:0/2283328182 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f289c074510 0x7f289c19a540 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:41.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:41.091+0000 7f289be00640 1 -- 172.21.15.5:0/2283328182 >> 172.21.15.5:0/2283328182 conn(0x7f289c0fd740 msgr2=0x7f289c0fe440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:46:41.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:41.092+0000 7f289be00640 1 -- 172.21.15.5:0/2283328182 shutdown_connections 2024-09-10T09:46:41.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:41.092+0000 7f289be00640 1 -- 172.21.15.5:0/2283328182 wait complete. 2024-09-10T09:46:41.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:46:40 smithi159 ceph-mon[35513]: pgmap v643: 129 pgs: 129 active+clean; 172 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 500 KiB/s rd, 2.3 MiB/s wr, 188 op/s 2024-09-10T09:46:41.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:46:40 smithi102 ceph-mon[35442]: pgmap v643: 129 pgs: 129 active+clean; 172 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 500 KiB/s rd, 2.3 MiB/s wr, 188 op/s 2024-09-10T09:46:41.887 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'fcdeb12e', quiesce took 4.7 seconds. Will release after: 8.7 2024-09-10T09:46:42.075 INFO:tasks.workunit.client.0.smithi005.stdout: 1 500933 6.40 MB/sec execute 263 sec latency 984.679 ms 2024-09-10T09:46:42.550 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T09:46:42.945 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:46:43.077 INFO:tasks.workunit.client.0.smithi005.stdout: 1 500933 6.37 MB/sec execute 264 sec latency 1984.731 ms 2024-09-10T09:46:43.170 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:46:42 smithi005 ceph-mon[30350]: pgmap v644: 129 pgs: 129 active+clean; 177 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 452 KiB/s rd, 2.5 MiB/s wr, 180 op/s 2024-09-10T09:46:43.170 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:46:42 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:46:43.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:46:42 smithi159 ceph-mon[35513]: pgmap v644: 129 pgs: 129 active+clean; 177 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 452 KiB/s rd, 2.5 MiB/s wr, 180 op/s 2024-09-10T09:46:43.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:46:42 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:46:43.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:46:42 smithi102 ceph-mon[35442]: pgmap v644: 129 pgs: 129 active+clean; 177 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 452 KiB/s rd, 2.5 MiB/s wr, 180 op/s 2024-09-10T09:46:43.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:46:42 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:46:44.075 INFO:tasks.workunit.client.0.smithi005.stdout: 1 500933 6.35 MB/sec execute 265 sec latency 2984.806 ms 2024-09-10T09:46:44.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:44.095+0000 7f4e89c00640 1 -- 172.21.15.5:0/4091854316 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4e841034b0 msgr2=0x7f4e8410b9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:46:44.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:44.095+0000 7f4e89c00640 1 --2- 172.21.15.5:0/4091854316 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4e841034b0 0x7f4e8410b9a0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f4e70009900 tx=0x7f4e7002a480 comp rx=0 tx=0).stop 2024-09-10T09:46:44.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:44.100+0000 7f4e89c00640 1 -- 172.21.15.5:0/4091854316 shutdown_connections 2024-09-10T09:46:44.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:44.100+0000 7f4e89c00640 1 --2- 172.21.15.5:0/4091854316 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4e8410bfc0 0x7f4e8410e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:44.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:44.100+0000 7f4e89c00640 1 --2- 172.21.15.5:0/4091854316 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4e841034b0 0x7f4e8410b9a0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:44.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:44.100+0000 7f4e89c00640 1 --2- 172.21.15.5:0/4091854316 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4e84102b90 0x7f4e84102f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:44.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:44.100+0000 7f4e89c00640 1 -- 172.21.15.5:0/4091854316 >> 172.21.15.5:0/4091854316 conn(0x7f4e840fc610 msgr2=0x7f4e840fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:46:44.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:44.100+0000 7f4e89c00640 1 -- 172.21.15.5:0/4091854316 shutdown_connections 2024-09-10T09:46:44.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:44.100+0000 7f4e89c00640 1 -- 172.21.15.5:0/4091854316 wait complete. 2024-09-10T09:46:44.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:44.101+0000 7f4e89c00640 1 Processor -- start 2024-09-10T09:46:44.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:44.101+0000 7f4e89c00640 1 -- start start 2024-09-10T09:46:44.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:44.101+0000 7f4e89c00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4e84102b90 0x7f4e841007f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:46:44.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:44.101+0000 7f4e89c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4e841034b0 0x7f4e84100d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:46:44.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:44.101+0000 7f4e89c00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4e8410bfc0 0x7f4e84101270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:46:44.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:44.101+0000 7f4e89c00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f4e84111dc0 con 0x7f4e84102b90 2024-09-10T09:46:44.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:44.101+0000 7f4e89c00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f4e84111c40 con 0x7f4e841034b0 2024-09-10T09:46:44.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:44.101+0000 7f4e89c00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f4e84111f40 con 0x7f4e8410bfc0 2024-09-10T09:46:44.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:44.101+0000 7f4e83400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4e841034b0 0x7f4e84100d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:46:44.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:44.101+0000 7f4e83e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4e84102b90 0x7f4e841007f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:46:44.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:44.101+0000 7f4e83400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4e841034b0 0x7f4e84100d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:39764/0 (socket says 172.21.15.5:39764) 2024-09-10T09:46:44.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:44.101+0000 7f4e83400640 1 -- 172.21.15.5:0/3082013664 learned_addr learned my addr 172.21.15.5:0/3082013664 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:46:44.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:44.101+0000 7f4e89200640 1 --2- 172.21.15.5:0/3082013664 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4e8410bfc0 0x7f4e84101270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:46:44.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:44.101+0000 7f4e83400640 1 -- 172.21.15.5:0/3082013664 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4e8410bfc0 msgr2=0x7f4e84101270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:46:44.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:44.101+0000 7f4e83400640 1 --2- 172.21.15.5:0/3082013664 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4e8410bfc0 0x7f4e84101270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:44.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:44.101+0000 7f4e83400640 1 -- 172.21.15.5:0/3082013664 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4e84102b90 msgr2=0x7f4e841007f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:46:44.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:44.101+0000 7f4e83400640 1 --2- 172.21.15.5:0/3082013664 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4e84102b90 0x7f4e841007f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:44.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:44.101+0000 7f4e83400640 1 -- 172.21.15.5:0/3082013664 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e841aacd0 con 0x7f4e841034b0 2024-09-10T09:46:44.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:44.101+0000 7f4e83400640 1 --2- 172.21.15.5:0/3082013664 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4e841034b0 0x7f4e84100d30 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f4e70006010 tx=0x7f4e70004040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:46:44.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:44.101+0000 7f4e80c00640 1 -- 172.21.15.5:0/3082013664 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e7003f030 con 0x7f4e841034b0 2024-09-10T09:46:44.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:44.101+0000 7f4e80c00640 1 -- 172.21.15.5:0/3082013664 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4e70004440 con 0x7f4e841034b0 2024-09-10T09:46:44.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:44.101+0000 7f4e80c00640 1 -- 172.21.15.5:0/3082013664 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e70035070 con 0x7f4e841034b0 2024-09-10T09:46:44.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:44.101+0000 7f4e89c00640 1 -- 172.21.15.5:0/3082013664 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e841aaea0 con 0x7f4e841034b0 2024-09-10T09:46:44.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:44.101+0000 7f4e89c00640 1 -- 172.21.15.5:0/3082013664 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e841ab240 con 0x7f4e841034b0 2024-09-10T09:46:44.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:44.107+0000 7f4e8de19740 1 Processor -- start 2024-09-10T09:46:44.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:44.107+0000 7f4e8de19740 1 -- start start 2024-09-10T09:46:44.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:44.107+0000 7f4e8de19740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56190aca1d50 0x56190aca2130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:46:44.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:44.107+0000 7f4e8de19740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56190aca2670 0x56190acac9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:46:44.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:44.107+0000 7f4e8de19740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56190acacf30 0x56190acaf320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:46:44.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:44.107+0000 7f4e8de19740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x56190ac083e0 con 0x56190acacf30 2024-09-10T09:46:44.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:44.107+0000 7f4e8de19740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x56190abafd80 con 0x56190aca1d50 2024-09-10T09:46:44.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:44.107+0000 7f4e8de19740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x56190ac043a0 con 0x56190aca2670 2024-09-10T09:46:44.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:44.107+0000 7f4e83e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56190aca2670 0x56190acac9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:46:44.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:44.107+0000 7f4e89200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56190aca1d50 0x56190aca2130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:46:44.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:44.107+0000 7f4e83e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56190aca2670 0x56190acac9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:55102/0 (socket says 172.21.15.5:55102) 2024-09-10T09:46:44.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:44.107+0000 7f4e83e00640 1 -- 172.21.15.5:0/430766107 learned_addr learned my addr 172.21.15.5:0/430766107 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:46:44.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:44.107+0000 7f4e80c00640 1 -- 172.21.15.5:0/3082013664 <== mon.2 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f4e70042030 con 0x7f4e841034b0 2024-09-10T09:46:44.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:44.107+0000 7f4e89200640 1 -- 172.21.15.5:0/430766107 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56190aca2670 msgr2=0x56190acac9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:46:44.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:44.107+0000 7f4e89200640 1 --2- 172.21.15.5:0/430766107 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56190aca2670 0x56190acac9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:44.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:44.107+0000 7f4e89200640 1 -- 172.21.15.5:0/430766107 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56190acacf30 msgr2=0x56190acaf320 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T09:46:44.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:44.107+0000 7f4e83400640 1 --2- 172.21.15.5:0/430766107 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56190acacf30 0x56190acaf320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:46:44.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:44.107+0000 7f4e89200640 1 --2- 172.21.15.5:0/430766107 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56190acacf30 0x56190acaf320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:44.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:44.107+0000 7f4e89200640 1 -- 172.21.15.5:0/430766107 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56190abceae0 con 0x56190aca1d50 2024-09-10T09:46:44.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:44.107+0000 7f4e83e00640 1 --2- 172.21.15.5:0/430766107 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56190aca2670 0x56190acac9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:46:44.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:44.107+0000 7f4e83400640 1 --2- 172.21.15.5:0/430766107 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56190acacf30 0x56190acaf320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T09:46:44.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:44.107+0000 7f4e89200640 1 --2- 172.21.15.5:0/430766107 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56190aca1d50 0x56190aca2130 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f4e7c009f00 tx=0x7f4e7c011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8a4d7fe622dc03e4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:46:44.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:44.107+0000 7f4e79600640 1 -- 172.21.15.5:0/430766107 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e7c004d20 con 0x56190aca1d50 2024-09-10T09:46:44.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:44.107+0000 7f4e79600640 1 -- 172.21.15.5:0/430766107 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4e7c004050 con 0x56190aca1d50 2024-09-10T09:46:44.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:44.107+0000 7f4e79600640 1 -- 172.21.15.5:0/430766107 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e7c005280 con 0x56190aca1d50 2024-09-10T09:46:44.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:44.107+0000 7f4e8de19740 1 -- 172.21.15.5:0/430766107 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56190aca1d50 msgr2=0x56190aca2130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:46:44.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:44.107+0000 7f4e8de19740 1 --2- 172.21.15.5:0/430766107 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56190aca1d50 0x56190aca2130 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f4e7c009f00 tx=0x7f4e7c011810 comp rx=0 tx=0).stop 2024-09-10T09:46:44.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:44.107+0000 7f4e80c00640 1 --2- 172.21.15.5:0/3082013664 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f4e680a2b80 0x7f4e680a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:46:44.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:44.107+0000 7f4e8de19740 1 -- 172.21.15.5:0/430766107 shutdown_connections 2024-09-10T09:46:44.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:44.107+0000 7f4e8de19740 1 --2- 172.21.15.5:0/430766107 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56190acacf30 0x56190acaf320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:44.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:44.107+0000 7f4e8de19740 1 --2- 172.21.15.5:0/430766107 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56190aca2670 0x56190acac9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:44.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:44.107+0000 7f4e8de19740 1 --2- 172.21.15.5:0/430766107 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56190aca1d50 0x56190aca2130 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:44.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:44.107+0000 7f4e8de19740 1 -- 172.21.15.5:0/430766107 >> 172.21.15.5:0/430766107 conn(0x56190ac9e2e0 msgr2=0x56190ac9e6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:46:44.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:44.107+0000 7f4e80c00640 1 -- 172.21.15.5:0/3082013664 <== mon.2 v2:172.21.15.102:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f4e700ef780 con 0x7f4e841034b0 2024-09-10T09:46:44.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:44.112+0000 7f4e8de19740 1 -- 172.21.15.5:0/430766107 shutdown_connections 2024-09-10T09:46:44.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:44.112+0000 7f4e89200640 1 --2- 172.21.15.5:0/3082013664 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f4e680a2b80 0x7f4e680a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:46:44.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:44.112+0000 7f4e8de19740 1 -- 172.21.15.5:0/430766107 wait complete. 2024-09-10T09:46:44.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:44.112+0000 7f4e8de19740 1 -- start start 2024-09-10T09:46:44.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:44.113+0000 7f4e8de19740 1 Processor -- start 2024-09-10T09:46:44.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:44.113+0000 7f4e89200640 1 --2- 172.21.15.5:0/3082013664 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f4e680a2b80 0x7f4e680a5040 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f4e7c01b040 tx=0x7f4e7c004ba0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:46:44.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:44.113+0000 7f4e8de19740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56190aca1d50 0x56190aca9df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:46:44.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:44.113+0000 7f4e8de19740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56190aca2670 0x56190acaa330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:46:44.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:44.113+0000 7f4e8de19740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56190acacf30 0x56190acaa870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:46:44.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:44.113+0000 7f4e8de19740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x56190acafba0 con 0x56190aca1d50 2024-09-10T09:46:44.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:44.113+0000 7f4e8de19740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x56190abc30f0 con 0x56190aca2670 2024-09-10T09:46:44.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:44.113+0000 7f4e8de19740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x56190acafd20 con 0x56190acacf30 2024-09-10T09:46:44.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:44.113+0000 7f4e8de19740 10 client.0 _ll_register_callbacks cb 0x56190abb49e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:46:44.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:44.113+0000 7f4e83e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56190aca1d50 0x56190aca9df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:46:44.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:44.113+0000 7f4e83e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56190aca1d50 0x56190aca9df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46316/0 (socket says 172.21.15.5:46316) 2024-09-10T09:46:44.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:44.113+0000 7f4e83e00640 1 -- 172.21.15.5:0/1152776148 learned_addr learned my addr 172.21.15.5:0/1152776148 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:46:44.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:44.113+0000 7f4e89200640 1 --2- 172.21.15.5:0/1152776148 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56190acacf30 0x56190acaa870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:46:44.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:44.113+0000 7f4e83e00640 1 -- 172.21.15.5:0/1152776148 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56190acacf30 msgr2=0x56190acaa870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:46:44.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:44.113+0000 7f4e83e00640 1 --2- 172.21.15.5:0/1152776148 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56190acacf30 0x56190acaa870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:44.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:44.113+0000 7f4e83e00640 1 -- 172.21.15.5:0/1152776148 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56190aca2670 msgr2=0x56190acaa330 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T09:46:44.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:44.113+0000 7f4e83e00640 1 --2- 172.21.15.5:0/1152776148 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56190aca2670 0x56190acaa330 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:44.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:44.113+0000 7f4e83e00640 1 -- 172.21.15.5:0/1152776148 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56190a9793c0 con 0x56190aca1d50 2024-09-10T09:46:44.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:44.113+0000 7f4e83e00640 1 --2- 172.21.15.5:0/1152776148 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56190aca1d50 0x56190aca9df0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f4e7400ba60 tx=0x7f4e7400bf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80c9f98b151692d4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:46:44.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:44.113+0000 7f4e6e000640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-10T09:46:44.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:44.113+0000 7f4e6e000640 1 -- 172.21.15.5:0/1152776148 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e74011440 con 0x56190aca1d50 2024-09-10T09:46:44.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:44.113+0000 7f4e6e000640 1 -- 172.21.15.5:0/1152776148 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f4e740115e0 con 0x56190aca1d50 2024-09-10T09:46:44.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:44.113+0000 7f4e8de19740 1 -- 172.21.15.5:0/1152776148 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4e7c004050 con 0x56190aca1d50 2024-09-10T09:46:44.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:44.113+0000 7f4e6e000640 1 -- 172.21.15.5:0/1152776148 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f4e74005020 con 0x56190aca1d50 2024-09-10T09:46:44.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:44.113+0000 7f4e8de19740 10 client.17019 fetch_fsmap learned FSMap version 35 2024-09-10T09:46:44.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:44.113+0000 7f4e8de19740 1 -- 172.21.15.5:0/1152776148 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x56190acbb820 con 0x56190aca1d50 2024-09-10T09:46:44.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:44.113+0000 7f4e6e000640 1 -- 172.21.15.5:0/1152776148 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4e7c004050 con 0x56190aca1d50 2024-09-10T09:46:44.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:44.113+0000 7f4e6e000640 1 -- 172.21.15.5:0/1152776148 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 35) ==== 8374+0+0 (secure 0 0 0) 0x7f4e740047a0 con 0x56190aca1d50 2024-09-10T09:46:44.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:44.113+0000 7f4e8de19740 10 client.17019 fetch_fsmap finished waiting for FSMap version 35 2024-09-10T09:46:44.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:44.113+0000 7f4e8de19740 10 client.17019 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T09:46:44.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:44.113+0000 7f4e8de19740 20 client.17019 populate_metadata read hostname 'smithi005' 2024-09-10T09:46:44.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:44.113+0000 7f4e8de19740 1 --2- 172.21.15.5:0/1152776148 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x56190acbcd30 0x56190acbf1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:46:44.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:44.113+0000 7f4e8de19740 4 client.17019 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:46:44.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:44.113+0000 7f4e8de19740 1 -- 172.21.15.5:0/1152776148 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56190ac15950 con 0x56190acbcd30 2024-09-10T09:46:44.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:44.113+0000 7f4e83400640 1 --2- 172.21.15.5:0/1152776148 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x56190acbcd30 0x56190acbf1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:46:44.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:44.113+0000 7f4e83400640 1 --2- 172.21.15.5:0/1152776148 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x56190acbcd30 0x56190acbf1d0 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=26af311bd4c5ef6d server_cookie=f0d3a0ffe893532a in_seq=0 out_seq=0 2024-09-10T09:46:44.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:44.113+0000 7f4e6e000640 10 client.17019 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T09:46:44.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:44.113+0000 7f4e6e000640 1 -- 172.21.15.5:0/1152776148 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56190ac15950 con 0x56190acbcd30 2024-09-10T09:46:44.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:44.113+0000 7f4e6e000640 10 client.17019 handle_command_reply: tid=0 2024-09-10T09:46:44.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:44.113+0000 7f4e8de19740 1 client.17019 shutdown 2024-09-10T09:46:44.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:44.113+0000 7f4e8de19740 1 -- 172.21.15.5:0/1152776148 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56190aca1d50 msgr2=0x56190aca9df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:46:44.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:44.113+0000 7f4e8de19740 1 --2- 172.21.15.5:0/1152776148 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56190aca1d50 0x56190aca9df0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f4e7400ba60 tx=0x7f4e7400bf20 comp rx=0 tx=0).stop 2024-09-10T09:46:44.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:44.113+0000 7f4e8de19740 1 -- 172.21.15.5:0/1152776148 shutdown_connections 2024-09-10T09:46:44.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:44.113+0000 7f4e8de19740 1 --2- 172.21.15.5:0/1152776148 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x56190acbcd30 0x56190acbf1d0 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:44.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:44.113+0000 7f4e8de19740 1 --2- 172.21.15.5:0/1152776148 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56190acacf30 0x56190acaa870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:44.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:44.113+0000 7f4e8de19740 1 --2- 172.21.15.5:0/1152776148 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56190aca2670 0x56190acaa330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:44.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:44.113+0000 7f4e8de19740 1 --2- 172.21.15.5:0/1152776148 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56190aca1d50 0x56190aca9df0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:44.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:44.113+0000 7f4e8de19740 1 -- 172.21.15.5:0/1152776148 >> 172.21.15.5:0/1152776148 conn(0x56190ac9e2e0 msgr2=0x56190aca3fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:46:44.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:44.113+0000 7f4e6e000640 0 client.17019 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T09:46:44.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:44.119+0000 7f4e8de19740 1 -- 172.21.15.5:0/1152776148 shutdown_connections 2024-09-10T09:46:44.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:44.119+0000 7f4e8de19740 1 -- 172.21.15.5:0/1152776148 wait complete. 2024-09-10T09:46:44.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:44.119+0000 7f4e8de19740 20 client.17019 trim_cache size 0 max 16384 2024-09-10T09:46:44.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:44.142+0000 7f4e8de19740 1 Processor -- start 2024-09-10T09:46:44.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:44.142+0000 7f4e8de19740 1 -- start start 2024-09-10T09:46:44.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:44.142+0000 7f4e8de19740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56190aca1d50 0x56190acc4aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:46:44.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:44.142+0000 7f4e8de19740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56190aca2670 0x56190acc4fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:46:44.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:44.142+0000 7f4e8de19740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56190acacf30 0x56190aca70d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:46:44.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:44.142+0000 7f4e8de19740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f4e841b52f0 con 0x56190aca2670 2024-09-10T09:46:44.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:44.142+0000 7f4e8de19740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f4e841b6e60 con 0x56190acacf30 2024-09-10T09:46:44.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:44.142+0000 7f4e8de19740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f4e841b59c0 con 0x56190aca1d50 2024-09-10T09:46:44.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:44.142+0000 7f4e83e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56190aca1d50 0x56190acc4aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:46:44.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:44.142+0000 7f4e83e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56190aca1d50 0x56190acc4aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:55130/0 (socket says 172.21.15.5:55130) 2024-09-10T09:46:44.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:44.142+0000 7f4e83e00640 1 -- 172.21.15.5:0/4236306616 learned_addr learned my addr 172.21.15.5:0/4236306616 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:46:44.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:44.142+0000 7f4e83400640 1 --2- 172.21.15.5:0/4236306616 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56190aca2670 0x56190acc4fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:46:44.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:44.142+0000 7f4e89200640 1 --2- 172.21.15.5:0/4236306616 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56190acacf30 0x56190aca70d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:46:44.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:44.142+0000 7f4e83e00640 1 -- 172.21.15.5:0/4236306616 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56190acacf30 msgr2=0x56190aca70d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:46:44.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:44.142+0000 7f4e83e00640 1 --2- 172.21.15.5:0/4236306616 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56190acacf30 0x56190aca70d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:44.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:44.142+0000 7f4e83e00640 1 -- 172.21.15.5:0/4236306616 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56190aca2670 msgr2=0x56190acc4fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:46:44.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:44.142+0000 7f4e83e00640 1 --2- 172.21.15.5:0/4236306616 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56190aca2670 0x56190acc4fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:44.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:44.142+0000 7f4e83e00640 1 -- 172.21.15.5:0/4236306616 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56190acac760 con 0x56190aca1d50 2024-09-10T09:46:44.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:44.142+0000 7f4e83e00640 1 --2- 172.21.15.5:0/4236306616 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56190aca1d50 0x56190acc4aa0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f4e74005d80 tx=0x7f4e74003f60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e7d8647d3ffcc315 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:46:44.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:44.142+0000 7f4e78c00640 1 -- 172.21.15.5:0/4236306616 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e7402e070 con 0x56190aca1d50 2024-09-10T09:46:44.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:44.142+0000 7f4e78c00640 1 -- 172.21.15.5:0/4236306616 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4e7400b400 con 0x56190aca1d50 2024-09-10T09:46:44.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:44.142+0000 7f4e78c00640 1 -- 172.21.15.5:0/4236306616 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e74010810 con 0x56190aca1d50 2024-09-10T09:46:44.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:44.142+0000 7f4e8de19740 1 -- 172.21.15.5:0/4236306616 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56190aca1d50 msgr2=0x56190acc4aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:46:44.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:44.142+0000 7f4e8de19740 1 --2- 172.21.15.5:0/4236306616 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56190aca1d50 0x56190acc4aa0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f4e74005d80 tx=0x7f4e74003f60 comp rx=0 tx=0).stop 2024-09-10T09:46:44.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:44.142+0000 7f4e8de19740 1 -- 172.21.15.5:0/4236306616 shutdown_connections 2024-09-10T09:46:44.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:44.142+0000 7f4e8de19740 1 --2- 172.21.15.5:0/4236306616 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56190acacf30 0x56190aca70d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:44.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:44.142+0000 7f4e8de19740 1 --2- 172.21.15.5:0/4236306616 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56190aca2670 0x56190acc4fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:44.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:44.142+0000 7f4e8de19740 1 --2- 172.21.15.5:0/4236306616 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56190aca1d50 0x56190acc4aa0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:44.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:44.142+0000 7f4e8de19740 1 -- 172.21.15.5:0/4236306616 >> 172.21.15.5:0/4236306616 conn(0x56190ac9e2e0 msgr2=0x56190acabe70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:46:44.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:44.142+0000 7f4e8de19740 1 -- 172.21.15.5:0/4236306616 shutdown_connections 2024-09-10T09:46:44.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:44.142+0000 7f4e8de19740 1 -- 172.21.15.5:0/4236306616 wait complete. 2024-09-10T09:46:44.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:44.142+0000 7f4e8de19740 1 -- start start 2024-09-10T09:46:44.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:44.142+0000 7f4e8de19740 1 Processor -- start 2024-09-10T09:46:44.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:44.142+0000 7f4e8de19740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56190aca1d50 0x56190acb4f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:46:44.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:44.142+0000 7f4e8de19740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56190aca2670 0x56190acb54b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:46:44.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:44.142+0000 7f4e8de19740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56190acacf30 0x56190acb1f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:46:44.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:44.142+0000 7f4e8de19740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f4e841b6760 con 0x56190aca2670 2024-09-10T09:46:44.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:44.142+0000 7f4e8de19740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f4e841b6090 con 0x56190acacf30 2024-09-10T09:46:44.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:44.142+0000 7f4e8de19740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f4e841b32d0 con 0x56190aca1d50 2024-09-10T09:46:44.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:44.142+0000 7f4e8de19740 10 client.0 _ll_register_callbacks cb 0x7f4e7400b670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:46:44.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:44.142+0000 7f4e83e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56190aca1d50 0x56190acb4f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:46:44.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:44.142+0000 7f4e83e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56190aca1d50 0x56190acb4f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:55142/0 (socket says 172.21.15.5:55142) 2024-09-10T09:46:44.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:44.142+0000 7f4e83e00640 1 -- 172.21.15.5:0/2503001865 learned_addr learned my addr 172.21.15.5:0/2503001865 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:46:44.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:44.142+0000 7f4e83400640 1 --2- 172.21.15.5:0/2503001865 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56190aca2670 0x56190acb54b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:46:44.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:44.142+0000 7f4e89200640 1 --2- 172.21.15.5:0/2503001865 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56190acacf30 0x56190acb1f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:46:44.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:44.142+0000 7f4e83400640 1 -- 172.21.15.5:0/2503001865 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56190aca1d50 msgr2=0x56190acb4f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:46:44.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:44.142+0000 7f4e83400640 1 --2- 172.21.15.5:0/2503001865 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56190aca1d50 0x56190acb4f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:44.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:44.142+0000 7f4e83400640 1 -- 172.21.15.5:0/2503001865 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56190acacf30 msgr2=0x56190acb1f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:46:44.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:44.142+0000 7f4e83400640 1 --2- 172.21.15.5:0/2503001865 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56190acacf30 0x56190acb1f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:44.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:44.142+0000 7f4e83400640 1 -- 172.21.15.5:0/2503001865 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56190aca7d20 con 0x56190aca2670 2024-09-10T09:46:44.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:44.147+0000 7f4e83400640 1 --2- 172.21.15.5:0/2503001865 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56190aca2670 0x56190acb54b0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x56190ac9f660 tx=0x7f4e700bb5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdc29220eb07f060 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:46:44.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:44.147+0000 7f4e6e000640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-10T09:46:44.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:44.147+0000 7f4e6e000640 1 -- 172.21.15.5:0/2503001865 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e700c0000 con 0x56190aca2670 2024-09-10T09:46:44.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:44.147+0000 7f4e6e000640 1 -- 172.21.15.5:0/2503001865 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f4e700ecf40 con 0x56190aca2670 2024-09-10T09:46:44.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:44.147+0000 7f4e8de19740 1 -- 172.21.15.5:0/2503001865 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4e7400b400 con 0x56190aca2670 2024-09-10T09:46:44.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:44.147+0000 7f4e6e000640 1 -- 172.21.15.5:0/2503001865 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f4e700c48c0 con 0x56190aca2670 2024-09-10T09:46:44.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:44.147+0000 7f4e8de19740 10 client.17025 fetch_fsmap learned FSMap version 35 2024-09-10T09:46:44.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:44.147+0000 7f4e8de19740 1 -- 172.21.15.5:0/2503001865 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x56190ad27f30 con 0x56190aca2670 2024-09-10T09:46:44.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:44.147+0000 7f4e6e000640 1 -- 172.21.15.5:0/2503001865 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4e7400b400 con 0x56190aca2670 2024-09-10T09:46:44.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:44.147+0000 7f4e6e000640 1 -- 172.21.15.5:0/2503001865 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 35) ==== 8374+0+0 (secure 0 0 0) 0x7f4e700ee070 con 0x56190aca2670 2024-09-10T09:46:44.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:44.149+0000 7f4e8de19740 10 client.17025 fetch_fsmap finished waiting for FSMap version 35 2024-09-10T09:46:44.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:44.149+0000 7f4e8de19740 10 client.17025 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T09:46:44.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:44.149+0000 7f4e8de19740 20 client.17025 populate_metadata read hostname 'smithi005' 2024-09-10T09:46:44.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:44.149+0000 7f4e8de19740 1 --2- 172.21.15.5:0/2503001865 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x56190ad286e0 0x56190ad2ab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:46:44.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:44.149+0000 7f4e8de19740 4 client.17025 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T09:46:44.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:44.149+0000 7f4e8de19740 1 -- 172.21.15.5:0/2503001865 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56190ac0fe60 con 0x56190ad286e0 2024-09-10T09:46:44.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:44.149+0000 7f4e83e00640 1 --2- 172.21.15.5:0/2503001865 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x56190ad286e0 0x56190ad2ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:46:44.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:44.149+0000 7f4e83e00640 1 --2- 172.21.15.5:0/2503001865 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x56190ad286e0 0x56190ad2ab00 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=626d5c7269ecbc28 server_cookie=e750ca2eb4d3fc59 in_seq=0 out_seq=0 2024-09-10T09:46:44.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:44.149+0000 7f4e6e000640 10 client.17025 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T09:46:44.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:44.149+0000 7f4e6e000640 1 -- 172.21.15.5:0/2503001865 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x56190ac0fe60 con 0x56190ad286e0 2024-09-10T09:46:44.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:44.149+0000 7f4e6e000640 10 client.17025 handle_command_reply: tid=0 2024-09-10T09:46:44.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:44.149+0000 7f4e8de19740 1 client.17025 shutdown 2024-09-10T09:46:44.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:44.149+0000 7f4e8de19740 1 -- 172.21.15.5:0/2503001865 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56190aca2670 msgr2=0x56190acb54b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:46:44.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:44.149+0000 7f4e8de19740 1 --2- 172.21.15.5:0/2503001865 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56190aca2670 0x56190acb54b0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x56190ac9f660 tx=0x7f4e700bb5b0 comp rx=0 tx=0).stop 2024-09-10T09:46:44.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:44.149+0000 7f4e8de19740 1 -- 172.21.15.5:0/2503001865 shutdown_connections 2024-09-10T09:46:44.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:44.149+0000 7f4e8de19740 1 --2- 172.21.15.5:0/2503001865 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x56190ad286e0 0x56190ad2ab00 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:44.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:44.149+0000 7f4e8de19740 1 --2- 172.21.15.5:0/2503001865 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56190acacf30 0x56190acb1f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:44.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:44.149+0000 7f4e8de19740 1 --2- 172.21.15.5:0/2503001865 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56190aca2670 0x56190acb54b0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:44.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:44.149+0000 7f4e8de19740 1 --2- 172.21.15.5:0/2503001865 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56190aca1d50 0x56190acb4f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:44.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:44.149+0000 7f4e8de19740 1 -- 172.21.15.5:0/2503001865 >> 172.21.15.5:0/2503001865 conn(0x56190ac9e2e0 msgr2=0x56190ac9cb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:46:44.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:44.149+0000 7f4e6e000640 0 client.17025 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T09:46:44.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:44.149+0000 7f4e8de19740 1 -- 172.21.15.5:0/2503001865 shutdown_connections 2024-09-10T09:46:44.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:44.149+0000 7f4e8de19740 1 -- 172.21.15.5:0/2503001865 wait complete. 2024-09-10T09:46:44.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:44.149+0000 7f4e8de19740 20 client.17025 trim_cache size 0 max 16384 2024-09-10T09:46:44.155 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-10T09:46:44.155 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "no active scrubs running", 2024-09-10T09:46:44.155 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": {} 2024-09-10T09:46:44.155 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-10T09:46:44.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:44.149+0000 7f4e89c00640 1 -- 172.21.15.5:0/3082013664 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f4e680a2b80 msgr2=0x7f4e680a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:46:44.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:44.149+0000 7f4e89c00640 1 --2- 172.21.15.5:0/3082013664 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f4e680a2b80 0x7f4e680a5040 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f4e7c01b040 tx=0x7f4e7c004ba0 comp rx=0 tx=0).stop 2024-09-10T09:46:44.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:44.149+0000 7f4e89c00640 1 -- 172.21.15.5:0/3082013664 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4e841034b0 msgr2=0x7f4e84100d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:46:44.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:44.149+0000 7f4e89c00640 1 --2- 172.21.15.5:0/3082013664 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4e841034b0 0x7f4e84100d30 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f4e70006010 tx=0x7f4e70004040 comp rx=0 tx=0).stop 2024-09-10T09:46:44.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:44.149+0000 7f4e89c00640 1 -- 172.21.15.5:0/3082013664 shutdown_connections 2024-09-10T09:46:44.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:44.149+0000 7f4e89c00640 1 --2- 172.21.15.5:0/3082013664 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f4e680a2b80 0x7f4e680a5040 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:44.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:44.149+0000 7f4e89c00640 1 --2- 172.21.15.5:0/3082013664 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4e8410bfc0 0x7f4e84101270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:44.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:44.149+0000 7f4e89c00640 1 --2- 172.21.15.5:0/3082013664 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4e841034b0 0x7f4e84100d30 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:44.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:44.149+0000 7f4e89c00640 1 --2- 172.21.15.5:0/3082013664 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4e84102b90 0x7f4e841007f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:44.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:44.149+0000 7f4e89c00640 1 -- 172.21.15.5:0/3082013664 >> 172.21.15.5:0/3082013664 conn(0x7f4e840fc610 msgr2=0x7f4e840fd310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:46:44.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:44.149+0000 7f4e89c00640 1 -- 172.21.15.5:0/3082013664 shutdown_connections 2024-09-10T09:46:44.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:44.149+0000 7f4e89c00640 1 -- 172.21.15.5:0/3082013664 wait complete. 2024-09-10T09:46:45.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:46:44 smithi005 ceph-mon[30350]: pgmap v645: 129 pgs: 129 active+clean; 180 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 451 KiB/s rd, 2.5 MiB/s wr, 172 op/s 2024-09-10T09:46:45.057 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:46:44 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:46:45.057 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:46:44 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:46:45.057 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:46:44 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:46:45.075 INFO:tasks.workunit.client.0.smithi005.stdout: 1 500933 6.33 MB/sec execute 266 sec latency 3984.932 ms 2024-09-10T09:46:45.226 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-10T09:46:45.226 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T09:46:45.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:46:44 smithi159 ceph-mon[35513]: pgmap v645: 129 pgs: 129 active+clean; 180 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 451 KiB/s rd, 2.5 MiB/s wr, 172 op/s 2024-09-10T09:46:45.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:46:44 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:46:45.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:46:44 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:46:45.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:46:44 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:46:45.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:46:44 smithi102 ceph-mon[35442]: pgmap v645: 129 pgs: 129 active+clean; 180 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 451 KiB/s rd, 2.5 MiB/s wr, 172 op/s 2024-09-10T09:46:45.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:46:44 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:46:45.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:46:44 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:46:45.385 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:46:44 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:46:45.609 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:46:46.076 INFO:tasks.workunit.client.0.smithi005.stdout: 1 500933 6.30 MB/sec execute 267 sec latency 4985.016 ms 2024-09-10T09:46:46.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:46.128+0000 7fb45f800640 1 -- 172.21.15.5:0/1459923873 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb458074590 msgr2=0x7fb458074970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:46:46.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:46.128+0000 7fb45f800640 1 --2- 172.21.15.5:0/1459923873 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb458074590 0x7fb458074970 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fb44c009900 tx=0x7fb44c02a480 comp rx=0 tx=0).stop 2024-09-10T09:46:46.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:46.128+0000 7fb45f800640 1 -- 172.21.15.5:0/1459923873 shutdown_connections 2024-09-10T09:46:46.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:46.128+0000 7fb45f800640 1 --2- 172.21.15.5:0/1459923873 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb458112b90 0x7fb458114f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:46.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:46.128+0000 7fb45f800640 1 --2- 172.21.15.5:0/1459923873 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb458074eb0 0x7fb458112650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:46.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:46.128+0000 7fb45f800640 1 --2- 172.21.15.5:0/1459923873 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb458074590 0x7fb458074970 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:46.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:46.128+0000 7fb45f800640 1 -- 172.21.15.5:0/1459923873 >> 172.21.15.5:0/1459923873 conn(0x7fb4580fd8c0 msgr2=0x7fb4580ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:46:46.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:46.128+0000 7fb45f800640 1 -- 172.21.15.5:0/1459923873 shutdown_connections 2024-09-10T09:46:46.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:46.128+0000 7fb45f800640 1 -- 172.21.15.5:0/1459923873 wait complete. 2024-09-10T09:46:46.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:46.129+0000 7fb45f800640 1 Processor -- start 2024-09-10T09:46:46.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:46.129+0000 7fb45f800640 1 -- start start 2024-09-10T09:46:46.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:46.129+0000 7fb45f800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb458074590 0x7fb45819eb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:46:46.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:46.129+0000 7fb45f800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb458074eb0 0x7fb45819f090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:46:46.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:46.129+0000 7fb45f800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb458112b90 0x7fb4581a3450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:46:46.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:46.129+0000 7fb45f800640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fb4581185c0 con 0x7fb458112b90 2024-09-10T09:46:46.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:46.129+0000 7fb45f800640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fb458118440 con 0x7fb458074590 2024-09-10T09:46:46.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:46.130+0000 7fb45f800640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fb458118740 con 0x7fb458074eb0 2024-09-10T09:46:46.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:46.130+0000 7fb45d800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb458112b90 0x7fb4581a3450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:46:46.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:46.130+0000 7fb45d800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb458112b90 0x7fb4581a3450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46368/0 (socket says 172.21.15.5:46368) 2024-09-10T09:46:46.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:46.130+0000 7fb457e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb458074eb0 0x7fb45819f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:46:46.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:46.130+0000 7fb45d800640 1 -- 172.21.15.5:0/4138577083 learned_addr learned my addr 172.21.15.5:0/4138577083 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:46:46.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:46.130+0000 7fb45ce00640 1 --2- 172.21.15.5:0/4138577083 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb458074590 0x7fb45819eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:46:46.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:46.130+0000 7fb45d800640 1 -- 172.21.15.5:0/4138577083 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb458074eb0 msgr2=0x7fb45819f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:46:46.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:46.130+0000 7fb45d800640 1 --2- 172.21.15.5:0/4138577083 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb458074eb0 0x7fb45819f090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:46.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:46.130+0000 7fb45d800640 1 -- 172.21.15.5:0/4138577083 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb458074590 msgr2=0x7fb45819eb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:46:46.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:46.130+0000 7fb45d800640 1 --2- 172.21.15.5:0/4138577083 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb458074590 0x7fb45819eb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:46.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:46.130+0000 7fb45d800640 1 -- 172.21.15.5:0/4138577083 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4581a3b80 con 0x7fb458112b90 2024-09-10T09:46:46.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:46.130+0000 7fb457e00640 1 --2- 172.21.15.5:0/4138577083 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb458074eb0 0x7fb45819f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:46:46.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:46.131+0000 7fb45d800640 1 --2- 172.21.15.5:0/4138577083 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb458112b90 0x7fb4581a3450 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fb45000eb40 tx=0x7fb45000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:46:46.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:46.131+0000 7fb455600640 1 -- 172.21.15.5:0/4138577083 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb450004b20 con 0x7fb458112b90 2024-09-10T09:46:46.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:46.131+0000 7fb45f800640 1 -- 172.21.15.5:0/4138577083 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4581a3e70 con 0x7fb458112b90 2024-09-10T09:46:46.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:46.131+0000 7fb455600640 1 -- 172.21.15.5:0/4138577083 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb450004cc0 con 0x7fb458112b90 2024-09-10T09:46:46.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:46.131+0000 7fb455600640 1 -- 172.21.15.5:0/4138577083 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb450010850 con 0x7fb458112b90 2024-09-10T09:46:46.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:46.131+0000 7fb45f800640 1 -- 172.21.15.5:0/4138577083 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4581ab700 con 0x7fb458112b90 2024-09-10T09:46:46.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:46.132+0000 7fb45f800640 1 -- 172.21.15.5:0/4138577083 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb458074970 con 0x7fb458112b90 2024-09-10T09:46:46.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:46.134+0000 7fb455600640 1 -- 172.21.15.5:0/4138577083 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7fb450005020 con 0x7fb458112b90 2024-09-10T09:46:46.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:46.135+0000 7fb455600640 1 --2- 172.21.15.5:0/4138577083 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fb4400a2c00 0x7fb4400a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:46:46.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:46.136+0000 7fb455600640 1 -- 172.21.15.5:0/4138577083 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fb4500cfb90 con 0x7fb458112b90 2024-09-10T09:46:46.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:46.136+0000 7fb45ce00640 1 --2- 172.21.15.5:0/4138577083 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fb4400a2c00 0x7fb4400a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:46:46.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:46.136+0000 7fb45ce00640 1 --2- 172.21.15.5:0/4138577083 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fb4400a2c00 0x7fb4400a50c0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fb44c000c00 tx=0x7fb44c005b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:46:46.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:46.140+0000 7fb455600640 1 -- 172.21.15.5:0/4138577083 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb45009c6a0 con 0x7fb458112b90 2024-09-10T09:46:46.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:46.363+0000 7fb45f800640 1 -- 172.21.15.5:0/4138577083 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb458104fd0 con 0x7fb458112b90 2024-09-10T09:46:46.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:46.363+0000 7fb455600640 1 -- 172.21.15.5:0/4138577083 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) ==== 94+0+11970 (secure 0 0 0) 0x7fb45009bfa0 con 0x7fb458112b90 2024-09-10T09:46:46.365 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-10T09:46:46.365 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":35,"btime":"2024-09-10T09:46:17:834001+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14454,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/2970594311","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2970594311},{"type":"v1","addr":"172.21.15.5:6835","nonce":2970594311}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14514,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6839/3616502174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":3616502174},{"type":"v1","addr":"172.21.15.5:6839","nonce":3616502174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":35,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T09:35:29.626287+0000","modified":"2024-09-10T09:46:16.966068+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24542,"mds_1":24535,"mds_2":24511,"mds_3":24506,"mds_4":24443},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14520":{"gid":14520,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/2073599842","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":2073599842},{"type":"v1","addr":"172.21.15.5:6841","nonce":2073599842}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"k","rank":4,"incarnation":26,"state":"up:active","state_seq":24,"addr":"172.21.15.102:6833/2457007427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":2457007427},{"type":"v1","addr":"172.21.15.102:6833","nonce":2457007427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/356445159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":356445159},{"type":"v1","addr":"172.21.15.159:6833","nonce":356445159}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6837/2894314507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":2894314507},{"type":"v1","addr":"172.21.15.5:6837","nonce":2894314507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.102:6835/1647549111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":1647549111},{"type":"v1","addr":"172.21.15.102:6835","nonce":1647549111}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6835/952657242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":952657242},{"type":"v1","addr":"172.21.15.159:6835","nonce":952657242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":19,"addr":"172.21.15.102:6837/2689776764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":2689776764},{"type":"v1","addr":"172.21.15.102:6837","nonce":2689776764}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.159:6837/620627639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":620627639},{"type":"v1","addr":"172.21.15.159:6837","nonce":620627639}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24535":{"gid":24535,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":18,"addr":"172.21.15.159:6839/2073739599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":2073739599},{"type":"v1","addr":"172.21.15.159:6839","nonce":2073739599}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24542":{"gid":24542,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.102:6839/335409749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6838","nonce":335409749},{"type":"v1","addr":"172.21.15.102:6839","nonce":335409749}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24542,"qdb_cluster":[24443,24506,24511,24535,24542]},"id":1}]} 2024-09-10T09:46:46.365 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 35 2024-09-10T09:46:46.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:46.366+0000 7fb45f800640 1 -- 172.21.15.5:0/4138577083 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fb4400a2c00 msgr2=0x7fb4400a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:46:46.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:46.366+0000 7fb45f800640 1 --2- 172.21.15.5:0/4138577083 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fb4400a2c00 0x7fb4400a50c0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fb44c000c00 tx=0x7fb44c005b50 comp rx=0 tx=0).stop 2024-09-10T09:46:46.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:46.366+0000 7fb45f800640 1 -- 172.21.15.5:0/4138577083 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb458112b90 msgr2=0x7fb4581a3450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:46:46.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:46.366+0000 7fb45f800640 1 --2- 172.21.15.5:0/4138577083 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb458112b90 0x7fb4581a3450 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fb45000eb40 tx=0x7fb45000c2d0 comp rx=0 tx=0).stop 2024-09-10T09:46:46.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:46.366+0000 7fb45f800640 1 -- 172.21.15.5:0/4138577083 shutdown_connections 2024-09-10T09:46:46.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:46.366+0000 7fb45f800640 1 --2- 172.21.15.5:0/4138577083 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fb4400a2c00 0x7fb4400a50c0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:46.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:46.366+0000 7fb45f800640 1 --2- 172.21.15.5:0/4138577083 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb458112b90 0x7fb4581a3450 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:46.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:46.366+0000 7fb45f800640 1 --2- 172.21.15.5:0/4138577083 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb458074eb0 0x7fb45819f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:46.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:46.366+0000 7fb45f800640 1 --2- 172.21.15.5:0/4138577083 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb458074590 0x7fb45819eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:46.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:46.366+0000 7fb45f800640 1 -- 172.21.15.5:0/4138577083 >> 172.21.15.5:0/4138577083 conn(0x7fb4580fd8c0 msgr2=0x7fb4580ff050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:46:46.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:46.366+0000 7fb45f800640 1 -- 172.21.15.5:0/4138577083 shutdown_connections 2024-09-10T09:46:46.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:46.366+0000 7fb45f800640 1 -- 172.21.15.5:0/4138577083 wait complete. 2024-09-10T09:46:47.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:46:46 smithi005 ceph-mon[30350]: pgmap v646: 129 pgs: 129 active+clean; 182 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 585 KiB/s rd, 2.6 MiB/s wr, 199 op/s 2024-09-10T09:46:47.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:46:46 smithi005 ceph-mon[30350]: from='client.? 172.21.15.5:0/4138577083' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:46:47.075 INFO:tasks.workunit.client.0.smithi005.stdout: 1 500933 6.28 MB/sec execute 268 sec latency 5985.130 ms 2024-09-10T09:46:47.246 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-10T09:46:47.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:46:46 smithi159 ceph-mon[35513]: pgmap v646: 129 pgs: 129 active+clean; 182 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 585 KiB/s rd, 2.6 MiB/s wr, 199 op/s 2024-09-10T09:46:47.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:46:46 smithi159 ceph-mon[35513]: from='client.? 172.21.15.5:0/4138577083' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:46:47.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:46:46 smithi102 ceph-mon[35442]: pgmap v646: 129 pgs: 129 active+clean; 182 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 585 KiB/s rd, 2.6 MiB/s wr, 199 op/s 2024-09-10T09:46:47.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:46:46 smithi102 ceph-mon[35442]: from='client.? 172.21.15.5:0/4138577083' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:46:47.625 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:46:48.076 INFO:tasks.workunit.client.0.smithi005.stdout: 1 500933 6.26 MB/sec execute 269 sec latency 6985.189 ms 2024-09-10T09:46:48.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:48.173+0000 7fa901400640 1 -- 172.21.15.5:0/3234619415 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa8fc077040 msgr2=0x7fa8fc112510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:46:48.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:48.173+0000 7fa901400640 1 --2- 172.21.15.5:0/3234619415 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa8fc077040 0x7fa8fc112510 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fa8e4009860 tx=0x7fa8e402a340 comp rx=0 tx=0).stop 2024-09-10T09:46:48.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:48.173+0000 7fa901400640 1 -- 172.21.15.5:0/3234619415 shutdown_connections 2024-09-10T09:46:48.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:48.173+0000 7fa901400640 1 --2- 172.21.15.5:0/3234619415 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa8fc112a50 0x7fa8fc114e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:48.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:48.173+0000 7fa901400640 1 --2- 172.21.15.5:0/3234619415 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa8fc077040 0x7fa8fc112510 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:48.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:48.173+0000 7fa901400640 1 --2- 172.21.15.5:0/3234619415 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa8fc076720 0x7fa8fc076b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:48.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:48.173+0000 7fa901400640 1 -- 172.21.15.5:0/3234619415 >> 172.21.15.5:0/3234619415 conn(0x7fa8fc0ffa50 msgr2=0x7fa8fc101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:46:48.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:48.173+0000 7fa901400640 1 -- 172.21.15.5:0/3234619415 shutdown_connections 2024-09-10T09:46:48.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:48.173+0000 7fa901400640 1 -- 172.21.15.5:0/3234619415 wait complete. 2024-09-10T09:46:48.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:48.174+0000 7fa901400640 1 Processor -- start 2024-09-10T09:46:48.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:48.174+0000 7fa901400640 1 -- start start 2024-09-10T09:46:48.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:48.174+0000 7fa901400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa8fc076720 0x7fa8fc19eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:46:48.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:48.174+0000 7fa901400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa8fc077040 0x7fa8fc19f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:46:48.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:48.174+0000 7fa901400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa8fc112a50 0x7fa8fc1a3460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:46:48.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:48.174+0000 7fa901400640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa8fc1185c0 con 0x7fa8fc077040 2024-09-10T09:46:48.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:48.174+0000 7fa901400640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fa8fc118440 con 0x7fa8fc112a50 2024-09-10T09:46:48.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:48.174+0000 7fa901400640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fa8fc118740 con 0x7fa8fc076720 2024-09-10T09:46:48.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:48.175+0000 7fa8f3e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa8fc077040 0x7fa8fc19f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:46:48.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:48.175+0000 7fa8fa800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa8fc076720 0x7fa8fc19eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:46:48.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:48.175+0000 7fa8fb200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa8fc112a50 0x7fa8fc1a3460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:46:48.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:48.175+0000 7fa8fa800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa8fc076720 0x7fa8fc19eb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:55182/0 (socket says 172.21.15.5:55182) 2024-09-10T09:46:48.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:48.175+0000 7fa8fa800640 1 -- 172.21.15.5:0/4161235655 learned_addr learned my addr 172.21.15.5:0/4161235655 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:46:48.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:48.175+0000 7fa8f3e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa8fc077040 0x7fa8fc19f0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46390/0 (socket says 172.21.15.5:46390) 2024-09-10T09:46:48.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:48.175+0000 7fa8fb200640 1 -- 172.21.15.5:0/4161235655 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa8fc076720 msgr2=0x7fa8fc19eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:46:48.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:48.175+0000 7fa8fb200640 1 --2- 172.21.15.5:0/4161235655 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa8fc076720 0x7fa8fc19eb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:48.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:48.175+0000 7fa8fb200640 1 -- 172.21.15.5:0/4161235655 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa8fc077040 msgr2=0x7fa8fc19f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:46:48.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:48.175+0000 7fa8fb200640 1 --2- 172.21.15.5:0/4161235655 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa8fc077040 0x7fa8fc19f0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:48.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:48.175+0000 7fa8fb200640 1 -- 172.21.15.5:0/4161235655 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8fc1a3b90 con 0x7fa8fc112a50 2024-09-10T09:46:48.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:48.175+0000 7fa8f3e00640 1 --2- 172.21.15.5:0/4161235655 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa8fc077040 0x7fa8fc19f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:46:48.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:48.175+0000 7fa8fb200640 1 --2- 172.21.15.5:0/4161235655 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa8fc112a50 0x7fa8fc1a3460 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fa8fc07aa10 tx=0x7fa8f400c800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:46:48.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:48.176+0000 7fa8f3400640 1 -- 172.21.15.5:0/4161235655 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8f4003970 con 0x7fa8fc112a50 2024-09-10T09:46:48.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:48.176+0000 7fa901400640 1 -- 172.21.15.5:0/4161235655 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8fc1a3e80 con 0x7fa8fc112a50 2024-09-10T09:46:48.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:48.176+0000 7fa8f3400640 1 -- 172.21.15.5:0/4161235655 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa8f4004020 con 0x7fa8fc112a50 2024-09-10T09:46:48.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:48.176+0000 7fa8f3400640 1 -- 172.21.15.5:0/4161235655 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8f400b9c0 con 0x7fa8fc112a50 2024-09-10T09:46:48.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:48.176+0000 7fa901400640 1 -- 172.21.15.5:0/4161235655 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8fc1ab710 con 0x7fa8fc112a50 2024-09-10T09:46:48.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:48.179+0000 7fa903e48740 1 Processor -- start 2024-09-10T09:46:48.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:48.179+0000 7fa8f3400640 1 -- 172.21.15.5:0/4161235655 <== mon.2 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7fa8f401d020 con 0x7fa8fc112a50 2024-09-10T09:46:48.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:48.179+0000 7fa903e48740 1 -- start start 2024-09-10T09:46:48.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:48.179+0000 7fa903e48740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fe600aec70 0x55fe600af050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:46:48.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:48.179+0000 7fa903e48740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fe600af590 0x55fe600b9910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:46:48.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:48.180+0000 7fa903e48740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fe600b9e50 0x55fe600bc240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:46:48.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:48.180+0000 7fa903e48740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55fe600153e0 con 0x55fe600aec70 2024-09-10T09:46:48.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:48.180+0000 7fa903e48740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55fe5ffbcd80 con 0x55fe600af590 2024-09-10T09:46:48.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:48.180+0000 7fa903e48740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55fe600113a0 con 0x55fe600b9e50 2024-09-10T09:46:48.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:48.180+0000 7fa8f3e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fe600aec70 0x55fe600af050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:46:48.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:48.180+0000 7fa8fa800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fe600af590 0x55fe600b9910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:46:48.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:48.180+0000 7fa8f3e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fe600aec70 0x55fe600af050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46396/0 (socket says 172.21.15.5:46396) 2024-09-10T09:46:48.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:48.180+0000 7fa8f3e00640 1 -- 172.21.15.5:0/325679221 learned_addr learned my addr 172.21.15.5:0/325679221 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:46:48.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:48.180+0000 7fa8f3e00640 1 --2- 172.21.15.5:0/325679221 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fe600b9e50 0x55fe600bc240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:46:48.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:48.180+0000 7fa8fa800640 1 -- 172.21.15.5:0/325679221 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fe600b9e50 msgr2=0x55fe600bc240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:46:48.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:48.180+0000 7fa8fa800640 1 --2- 172.21.15.5:0/325679221 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fe600b9e50 0x55fe600bc240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:48.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:48.180+0000 7fa8fa800640 1 -- 172.21.15.5:0/325679221 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fe600aec70 msgr2=0x55fe600af050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:46:48.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:48.180+0000 7fa8fa800640 1 --2- 172.21.15.5:0/325679221 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fe600aec70 0x55fe600af050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:48.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:48.180+0000 7fa8fa800640 1 -- 172.21.15.5:0/325679221 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fe5ffdbae0 con 0x55fe600af590 2024-09-10T09:46:48.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:48.181+0000 7fa8f3400640 1 --2- 172.21.15.5:0/4161235655 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fa8e00a2b80 0x7fa8e00a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:46:48.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:48.181+0000 7fa8fa800640 1 --2- 172.21.15.5:0/325679221 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fe600af590 0x55fe600b9910 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fa8ec004590 tx=0x7fa8ec00bbb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9a86270d50c59c73 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:46:48.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:48.181+0000 7fa8f3400640 1 -- 172.21.15.5:0/4161235655 <== mon.2 v2:172.21.15.102:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fa8f40cf290 con 0x7fa8fc112a50 2024-09-10T09:46:48.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:48.181+0000 7fa8f3e00640 1 --2- 172.21.15.5:0/4161235655 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fa8e00a2b80 0x7fa8e00a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:46:48.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:48.181+0000 7fa8eb400640 1 -- 172.21.15.5:0/325679221 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8ec003bd0 con 0x55fe600af590 2024-09-10T09:46:48.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:48.181+0000 7fa8eb400640 1 -- 172.21.15.5:0/325679221 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa8ec015070 con 0x55fe600af590 2024-09-10T09:46:48.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:48.181+0000 7fa8eb400640 1 -- 172.21.15.5:0/325679221 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8ec004de0 con 0x55fe600af590 2024-09-10T09:46:48.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:48.182+0000 7fa903e48740 1 -- 172.21.15.5:0/325679221 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fe600af590 msgr2=0x55fe600b9910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:46:48.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:48.182+0000 7fa8f3e00640 1 --2- 172.21.15.5:0/4161235655 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fa8e00a2b80 0x7fa8e00a5040 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fa8e40057c0 tx=0x7fa8e40056d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:46:48.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:48.182+0000 7fa903e48740 1 --2- 172.21.15.5:0/325679221 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fe600af590 0x55fe600b9910 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fa8ec004590 tx=0x7fa8ec00bbb0 comp rx=0 tx=0).stop 2024-09-10T09:46:48.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:48.182+0000 7fa903e48740 1 -- 172.21.15.5:0/325679221 shutdown_connections 2024-09-10T09:46:48.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:48.182+0000 7fa903e48740 1 --2- 172.21.15.5:0/325679221 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fe600b9e50 0x55fe600bc240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:48.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:48.182+0000 7fa903e48740 1 --2- 172.21.15.5:0/325679221 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fe600af590 0x55fe600b9910 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:48.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:48.182+0000 7fa903e48740 1 --2- 172.21.15.5:0/325679221 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fe600aec70 0x55fe600af050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:48.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:48.182+0000 7fa903e48740 1 -- 172.21.15.5:0/325679221 >> 172.21.15.5:0/325679221 conn(0x55fe600ab200 msgr2=0x55fe600ab5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:46:48.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:48.182+0000 7fa903e48740 1 -- 172.21.15.5:0/325679221 shutdown_connections 2024-09-10T09:46:48.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:48.182+0000 7fa903e48740 1 -- 172.21.15.5:0/325679221 wait complete. 2024-09-10T09:46:48.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:48.183+0000 7fa903e48740 1 -- start start 2024-09-10T09:46:48.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:48.183+0000 7fa903e48740 1 Processor -- start 2024-09-10T09:46:48.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:48.184+0000 7fa903e48740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fe600aec70 0x55fe600b6ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:46:48.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:48.184+0000 7fa903e48740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fe600af590 0x55fe600b7220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:46:48.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:48.184+0000 7fa903e48740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fe600b9e50 0x55fe600b7760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:46:48.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:48.184+0000 7fa903e48740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55fe600bca20 con 0x55fe600af590 2024-09-10T09:46:48.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:48.184+0000 7fa903e48740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55fe5ffd00f0 con 0x55fe600aec70 2024-09-10T09:46:48.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:48.184+0000 7fa903e48740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55fe600bcba0 con 0x55fe600b9e50 2024-09-10T09:46:48.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:48.184+0000 7fa903e48740 10 client.0 _ll_register_callbacks cb 0x55fe5ffb7bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:46:48.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:48.184+0000 7fa8fa800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fe600aec70 0x55fe600b6ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:46:48.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:48.184+0000 7fa8fb200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fe600b9e50 0x55fe600b7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:46:48.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:48.184+0000 7fa8fa800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fe600aec70 0x55fe600b6ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:39870/0 (socket says 172.21.15.5:39870) 2024-09-10T09:46:48.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:48.184+0000 7fa8fb200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fe600b9e50 0x55fe600b7760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:55208/0 (socket says 172.21.15.5:55208) 2024-09-10T09:46:48.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:48.184+0000 7fa8fa800640 1 -- 172.21.15.5:0/868171563 learned_addr learned my addr 172.21.15.5:0/868171563 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:46:48.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:48.184+0000 7fa8f3e00640 1 --2- 172.21.15.5:0/868171563 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fe600af590 0x55fe600b7220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:46:48.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:48.185+0000 7fa8fb200640 1 -- 172.21.15.5:0/868171563 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fe600aec70 msgr2=0x55fe600b6ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:46:48.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:48.185+0000 7fa8fb200640 1 --2- 172.21.15.5:0/868171563 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fe600aec70 0x55fe600b6ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:48.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:48.185+0000 7fa8fb200640 1 -- 172.21.15.5:0/868171563 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fe600af590 msgr2=0x55fe600b7220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:46:48.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:48.185+0000 7fa8fb200640 1 --2- 172.21.15.5:0/868171563 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fe600af590 0x55fe600b7220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:48.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:48.185+0000 7fa8fb200640 1 -- 172.21.15.5:0/868171563 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fe5fd863c0 con 0x55fe600b9e50 2024-09-10T09:46:48.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:48.185+0000 7fa8fa800640 1 --2- 172.21.15.5:0/868171563 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fe600aec70 0x55fe600b6ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:46:48.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:48.185+0000 7fa8fb200640 1 --2- 172.21.15.5:0/868171563 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fe600b9e50 0x55fe600b7760 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fa8f40cc940 tx=0x7fa8f409be60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=64b07548e707affb server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:46:48.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:48.185+0000 7fa8dfe00640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-09-10T09:46:48.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:48.185+0000 7fa8dfe00640 1 -- 172.21.15.5:0/868171563 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8f409fa50 con 0x55fe600b9e50 2024-09-10T09:46:48.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:48.186+0000 7fa8dfe00640 1 -- 172.21.15.5:0/868171563 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fa8f4004560 con 0x55fe600b9e50 2024-09-10T09:46:48.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:48.186+0000 7fa903e48740 1 -- 172.21.15.5:0/868171563 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa8ec015070 con 0x55fe600b9e50 2024-09-10T09:46:48.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:48.186+0000 7fa8dfe00640 1 -- 172.21.15.5:0/868171563 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fa8f40a4310 con 0x55fe600b9e50 2024-09-10T09:46:48.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:48.186+0000 7fa903e48740 10 client.27070 fetch_fsmap learned FSMap version 35 2024-09-10T09:46:48.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:48.186+0000 7fa903e48740 1 -- 172.21.15.5:0/868171563 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55fe600c8aa0 con 0x55fe600b9e50 2024-09-10T09:46:48.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:48.186+0000 7fa8dfe00640 1 -- 172.21.15.5:0/868171563 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa8ec015070 con 0x55fe600b9e50 2024-09-10T09:46:48.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:48.186+0000 7fa8dfe00640 1 -- 172.21.15.5:0/868171563 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 35) ==== 8374+0+0 (secure 0 0 0) 0x7fa8f400b040 con 0x55fe600b9e50 2024-09-10T09:46:48.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:48.187+0000 7fa903e48740 10 client.27070 fetch_fsmap finished waiting for FSMap version 35 2024-09-10T09:46:48.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:48.187+0000 7fa903e48740 10 client.27070 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-09-10T09:46:48.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:48.187+0000 7fa903e48740 20 client.27070 populate_metadata read hostname 'smithi005' 2024-09-10T09:46:48.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:48.187+0000 7fa903e48740 1 --2- 172.21.15.5:0/868171563 >> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] conn(0x55fe600c9f60 0x55fe600cc350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:46:48.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:48.187+0000 7fa903e48740 4 client.27070 mds_command: new command op to 24443 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:46:48.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:48.187+0000 7fa903e48740 1 -- 172.21.15.5:0/868171563 --> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fe60022950 con 0x55fe600c9f60 2024-09-10T09:46:48.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:48.187+0000 7fa8fa800640 1 --2- 172.21.15.5:0/868171563 >> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] conn(0x55fe600c9f60 0x55fe600cc350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:46:48.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:48.188+0000 7fa8fa800640 1 --2- 172.21.15.5:0/868171563 >> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] conn(0x55fe600c9f60 0x55fe600cc350 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=de4e9e34fdb24ed7 server_cookie=87ef743853b2972a in_seq=0 out_seq=0 2024-09-10T09:46:48.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:48.188+0000 7fa8dfe00640 10 client.27070 ms_handle_connect on v2:172.21.15.102:6832/2457007427 2024-09-10T09:46:48.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:48.189+0000 7fa8dfe00640 1 -- 172.21.15.5:0/868171563 <== mds.4 v2:172.21.15.102:6832/2457007427 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55fe60022950 con 0x55fe600c9f60 2024-09-10T09:46:48.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:48.189+0000 7fa8dfe00640 10 client.27070 handle_command_reply: tid=0 2024-09-10T09:46:48.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:48.189+0000 7fa903e48740 1 client.27070 shutdown 2024-09-10T09:46:48.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:48.189+0000 7fa903e48740 1 -- 172.21.15.5:0/868171563 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fe600b9e50 msgr2=0x55fe600b7760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:46:48.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:48.189+0000 7fa903e48740 1 --2- 172.21.15.5:0/868171563 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fe600b9e50 0x55fe600b7760 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fa8f40cc940 tx=0x7fa8f409be60 comp rx=0 tx=0).stop 2024-09-10T09:46:48.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:48.192+0000 7fa903e48740 1 -- 172.21.15.5:0/868171563 shutdown_connections 2024-09-10T09:46:48.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:48.192+0000 7fa903e48740 1 --2- 172.21.15.5:0/868171563 >> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] conn(0x55fe600c9f60 0x55fe600cc350 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:48.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:48.192+0000 7fa903e48740 1 --2- 172.21.15.5:0/868171563 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fe600b9e50 0x55fe600b7760 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:48.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:48.192+0000 7fa903e48740 1 --2- 172.21.15.5:0/868171563 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fe600af590 0x55fe600b7220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:48.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:48.192+0000 7fa903e48740 1 --2- 172.21.15.5:0/868171563 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fe600aec70 0x55fe600b6ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:48.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:48.192+0000 7fa903e48740 1 -- 172.21.15.5:0/868171563 >> 172.21.15.5:0/868171563 conn(0x55fe600ab200 msgr2=0x55fe600b0f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:46:48.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:48.192+0000 7fa8dfe00640 0 client.27070 ms_handle_reset on v2:172.21.15.102:6832/2457007427 2024-09-10T09:46:48.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:48.193+0000 7fa903e48740 1 -- 172.21.15.5:0/868171563 shutdown_connections 2024-09-10T09:46:48.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:48.193+0000 7fa903e48740 1 -- 172.21.15.5:0/868171563 wait complete. 2024-09-10T09:46:48.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:48.193+0000 7fa903e48740 20 client.27070 trim_cache size 0 max 16384 2024-09-10T09:46:48.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:48.222+0000 7fa903e48740 1 Processor -- start 2024-09-10T09:46:48.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:48.223+0000 7fa903e48740 1 -- start start 2024-09-10T09:46:48.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:48.223+0000 7fa903e48740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fe600aec70 0x55fe600b1720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:46:48.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:48.223+0000 7fa903e48740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fe600af590 0x55fe600b8c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:46:48.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:48.223+0000 7fa903e48740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fe600b9e50 0x55fe600d1bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:46:48.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:48.223+0000 7fa903e48740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa8fc1b5410 con 0x55fe600af590 2024-09-10T09:46:48.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:48.223+0000 7fa903e48740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fa8fc1b6f80 con 0x55fe600b9e50 2024-09-10T09:46:48.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:48.223+0000 7fa903e48740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fa8fc1b5ae0 con 0x55fe600aec70 2024-09-10T09:46:48.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:48.224+0000 7fa8f3e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fe600af590 0x55fe600b8c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:46:48.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:48.225+0000 7fa8fb200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fe600b9e50 0x55fe600d1bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:46:48.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:48.225+0000 7fa8fb200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fe600b9e50 0x55fe600d1bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:39872/0 (socket says 172.21.15.5:39872) 2024-09-10T09:46:48.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:48.225+0000 7fa8fb200640 1 -- 172.21.15.5:0/1772909396 learned_addr learned my addr 172.21.15.5:0/1772909396 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:46:48.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:48.225+0000 7fa8fa800640 1 --2- 172.21.15.5:0/1772909396 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fe600aec70 0x55fe600b1720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:46:48.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:48.225+0000 7fa8fb200640 1 -- 172.21.15.5:0/1772909396 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fe600aec70 msgr2=0x55fe600b1720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:46:48.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:48.225+0000 7fa8fb200640 1 --2- 172.21.15.5:0/1772909396 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fe600aec70 0x55fe600b1720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:48.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:48.225+0000 7fa8fb200640 1 -- 172.21.15.5:0/1772909396 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fe600af590 msgr2=0x55fe600b8c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:46:48.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:48.225+0000 7fa8fb200640 1 --2- 172.21.15.5:0/1772909396 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fe600af590 0x55fe600b8c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:48.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:48.225+0000 7fa8fb200640 1 -- 172.21.15.5:0/1772909396 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fe600b5cb0 con 0x55fe600b9e50 2024-09-10T09:46:48.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:48.226+0000 7fa8fb200640 1 --2- 172.21.15.5:0/1772909396 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fe600b9e50 0x55fe600d1bd0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fa8f409b630 tx=0x7fa8f4004860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b78b7e9a701e719f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:46:48.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:48.226+0000 7fa8eaa00640 1 -- 172.21.15.5:0/1772909396 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8f409b1e0 con 0x55fe600b9e50 2024-09-10T09:46:48.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:48.227+0000 7fa8eaa00640 1 -- 172.21.15.5:0/1772909396 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa8f409fa50 con 0x55fe600b9e50 2024-09-10T09:46:48.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:48.227+0000 7fa8eaa00640 1 -- 172.21.15.5:0/1772909396 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8f409b1e0 con 0x55fe600b9e50 2024-09-10T09:46:48.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:48.228+0000 7fa903e48740 1 -- 172.21.15.5:0/1772909396 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fe600b9e50 msgr2=0x55fe600d1bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:46:48.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:48.228+0000 7fa903e48740 1 --2- 172.21.15.5:0/1772909396 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fe600b9e50 0x55fe600d1bd0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fa8f409b630 tx=0x7fa8f4004860 comp rx=0 tx=0).stop 2024-09-10T09:46:48.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:48.230+0000 7fa903e48740 1 -- 172.21.15.5:0/1772909396 shutdown_connections 2024-09-10T09:46:48.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:48.230+0000 7fa903e48740 1 --2- 172.21.15.5:0/1772909396 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fe600b9e50 0x55fe600d1bd0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:48.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:48.230+0000 7fa903e48740 1 --2- 172.21.15.5:0/1772909396 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fe600af590 0x55fe600b8c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:48.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:48.230+0000 7fa903e48740 1 --2- 172.21.15.5:0/1772909396 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fe600aec70 0x55fe600b1720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:48.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:48.230+0000 7fa903e48740 1 -- 172.21.15.5:0/1772909396 >> 172.21.15.5:0/1772909396 conn(0x55fe600ab200 msgr2=0x55fe600b8540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:46:48.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:48.230+0000 7fa903e48740 1 -- 172.21.15.5:0/1772909396 shutdown_connections 2024-09-10T09:46:48.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:48.230+0000 7fa903e48740 1 -- 172.21.15.5:0/1772909396 wait complete. 2024-09-10T09:46:48.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:48.231+0000 7fa903e48740 1 -- start start 2024-09-10T09:46:48.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:48.231+0000 7fa903e48740 1 Processor -- start 2024-09-10T09:46:48.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:48.232+0000 7fa903e48740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fe600aec70 0x55fe601300c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:46:48.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:48.232+0000 7fa903e48740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fe600af590 0x55fe60130600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:46:48.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:48.232+0000 7fa903e48740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fe600b9e50 0x55fe600c2040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:46:48.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:48.233+0000 7fa8fa800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fe600aec70 0x55fe601300c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:46:48.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:48.233+0000 7fa8f3e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fe600af590 0x55fe60130600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:46:48.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:48.233+0000 7fa8f3e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fe600af590 0x55fe60130600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:55214/0 (socket says 172.21.15.5:55214) 2024-09-10T09:46:48.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:48.233+0000 7fa8f3e00640 1 -- 172.21.15.5:0/2290526002 learned_addr learned my addr 172.21.15.5:0/2290526002 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:46:48.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:48.233+0000 7fa903e48740 1 -- 172.21.15.5:0/2290526002 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa8fc1b6880 con 0x55fe600b9e50 2024-09-10T09:46:48.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:48.233+0000 7fa903e48740 1 -- 172.21.15.5:0/2290526002 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fa8fc1b61b0 con 0x55fe600aec70 2024-09-10T09:46:48.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:48.233+0000 7fa903e48740 1 -- 172.21.15.5:0/2290526002 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fa8fc1b33f0 con 0x55fe600af590 2024-09-10T09:46:48.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:48.234+0000 7fa903e48740 10 client.0 _ll_register_callbacks cb 0x7fa8f4009900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:46:48.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:48.235+0000 7fa8f3e00640 1 -- 172.21.15.5:0/2290526002 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fe600aec70 msgr2=0x55fe601300c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:46:48.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:48.235+0000 7fa8f3e00640 1 --2- 172.21.15.5:0/2290526002 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fe600aec70 0x55fe601300c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:48.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:48.235+0000 7fa8f3e00640 1 -- 172.21.15.5:0/2290526002 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fe600b9e50 msgr2=0x55fe600c2040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:46:48.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:48.235+0000 7fa8f3e00640 1 --2- 172.21.15.5:0/2290526002 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fe600b9e50 0x55fe600c2040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:48.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:48.235+0000 7fa8f3e00640 1 -- 172.21.15.5:0/2290526002 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fe600c2870 con 0x55fe600af590 2024-09-10T09:46:48.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:48.237+0000 7fa8f3e00640 1 --2- 172.21.15.5:0/2290526002 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fe600af590 0x55fe60130600 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fa8e4035040 tx=0x7fa8e402a340 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ba8f3614b6152da0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:46:48.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:48.237+0000 7fa8dfe00640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-09-10T09:46:48.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:48.238+0000 7fa8dfe00640 1 -- 172.21.15.5:0/2290526002 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8e4037040 con 0x55fe600af590 2024-09-10T09:46:48.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:48.238+0000 7fa8dfe00640 1 -- 172.21.15.5:0/2290526002 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fa8e4032640 con 0x55fe600af590 2024-09-10T09:46:48.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:48.240+0000 7fa903e48740 1 -- 172.21.15.5:0/2290526002 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa8f409fa50 con 0x55fe600af590 2024-09-10T09:46:48.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:48.240+0000 7fa8dfe00640 1 -- 172.21.15.5:0/2290526002 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fa8e4032840 con 0x55fe600af590 2024-09-10T09:46:48.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:48.242+0000 7fa903e48740 10 client.27076 fetch_fsmap learned FSMap version 35 2024-09-10T09:46:48.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:48.242+0000 7fa903e48740 1 -- 172.21.15.5:0/2290526002 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55fe6012f990 con 0x55fe600af590 2024-09-10T09:46:48.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:48.242+0000 7fa8dfe00640 1 -- 172.21.15.5:0/2290526002 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa8f409fa50 con 0x55fe600af590 2024-09-10T09:46:48.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:48.244+0000 7fa8dfe00640 1 -- 172.21.15.5:0/2290526002 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 35) ==== 8374+0+0 (secure 0 0 0) 0x7fa8e403d070 con 0x55fe600af590 2024-09-10T09:46:48.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:48.244+0000 7fa903e48740 10 client.27076 fetch_fsmap finished waiting for FSMap version 35 2024-09-10T09:46:48.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:48.244+0000 7fa903e48740 10 client.27076 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-09-10T09:46:48.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:48.244+0000 7fa903e48740 20 client.27076 populate_metadata read hostname 'smithi005' 2024-09-10T09:46:48.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:48.244+0000 7fa903e48740 1 --2- 172.21.15.5:0/2290526002 >> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] conn(0x55fe601460a0 0x55fe601484c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:46:48.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:48.244+0000 7fa903e48740 4 client.27076 mds_command: new command op to 24443 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T09:46:48.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:48.244+0000 7fa903e48740 1 -- 172.21.15.5:0/2290526002 --> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55fe6001ce60 con 0x55fe601460a0 2024-09-10T09:46:48.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:48.245+0000 7fa8fa800640 1 --2- 172.21.15.5:0/2290526002 >> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] conn(0x55fe601460a0 0x55fe601484c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:46:48.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:48.245+0000 7fa8fa800640 1 --2- 172.21.15.5:0/2290526002 >> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] conn(0x55fe601460a0 0x55fe601484c0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=f25da3d679ebea3f server_cookie=e950a77e3709a5e1 in_seq=0 out_seq=0 2024-09-10T09:46:48.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:48.245+0000 7fa8dfe00640 10 client.27076 ms_handle_connect on v2:172.21.15.102:6832/2457007427 2024-09-10T09:46:48.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:48.246+0000 7fa8dfe00640 1 -- 172.21.15.5:0/2290526002 <== mds.4 v2:172.21.15.102:6832/2457007427 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55fe6001ce60 con 0x55fe601460a0 2024-09-10T09:46:48.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:48.246+0000 7fa8dfe00640 10 client.27076 handle_command_reply: tid=0 2024-09-10T09:46:48.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:48.246+0000 7fa903e48740 1 client.27076 shutdown 2024-09-10T09:46:48.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:48.254+0000 7fa903e48740 1 -- 172.21.15.5:0/2290526002 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fe600af590 msgr2=0x55fe60130600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:46:48.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:48.254+0000 7fa903e48740 1 --2- 172.21.15.5:0/2290526002 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fe600af590 0x55fe60130600 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fa8e4035040 tx=0x7fa8e402a340 comp rx=0 tx=0).stop 2024-09-10T09:46:48.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:48.256+0000 7fa903e48740 1 -- 172.21.15.5:0/2290526002 shutdown_connections 2024-09-10T09:46:48.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:48.256+0000 7fa903e48740 1 --2- 172.21.15.5:0/2290526002 >> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] conn(0x55fe601460a0 0x55fe601484c0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:48.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:48.256+0000 7fa8dfe00640 0 client.27076 ms_handle_reset on v2:172.21.15.102:6832/2457007427 2024-09-10T09:46:48.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:48.256+0000 7fa903e48740 1 --2- 172.21.15.5:0/2290526002 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fe600b9e50 0x55fe600c2040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:48.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:48.256+0000 7fa903e48740 1 --2- 172.21.15.5:0/2290526002 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55fe600af590 0x55fe60130600 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:48.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:48.256+0000 7fa903e48740 1 --2- 172.21.15.5:0/2290526002 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55fe600aec70 0x55fe601300c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:48.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:48.256+0000 7fa903e48740 1 -- 172.21.15.5:0/2290526002 >> 172.21.15.5:0/2290526002 conn(0x55fe600ab200 msgr2=0x55fe600aa380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:46:48.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:48.257+0000 7fa903e48740 1 -- 172.21.15.5:0/2290526002 shutdown_connections 2024-09-10T09:46:48.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:48.257+0000 7fa903e48740 1 -- 172.21.15.5:0/2290526002 wait complete. 2024-09-10T09:46:48.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:48.257+0000 7fa903e48740 20 client.27076 trim_cache size 0 max 16384 2024-09-10T09:46:48.259 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-10T09:46:48.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:48.259+0000 7fa901400640 1 -- 172.21.15.5:0/4161235655 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fa8e00a2b80 msgr2=0x7fa8e00a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:46:48.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:48.259+0000 7fa901400640 1 --2- 172.21.15.5:0/4161235655 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fa8e00a2b80 0x7fa8e00a5040 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fa8e40057c0 tx=0x7fa8e40056d0 comp rx=0 tx=0).stop 2024-09-10T09:46:48.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:48.259+0000 7fa901400640 1 -- 172.21.15.5:0/4161235655 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa8fc112a50 msgr2=0x7fa8fc1a3460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:46:48.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:48.259+0000 7fa901400640 1 --2- 172.21.15.5:0/4161235655 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa8fc112a50 0x7fa8fc1a3460 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fa8fc07aa10 tx=0x7fa8f400c800 comp rx=0 tx=0).stop 2024-09-10T09:46:48.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:48.259+0000 7fa901400640 1 -- 172.21.15.5:0/4161235655 shutdown_connections 2024-09-10T09:46:48.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:48.259+0000 7fa901400640 1 --2- 172.21.15.5:0/4161235655 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fa8e00a2b80 0x7fa8e00a5040 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:48.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:48.259+0000 7fa901400640 1 --2- 172.21.15.5:0/4161235655 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa8fc112a50 0x7fa8fc1a3460 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:48.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:48.259+0000 7fa901400640 1 --2- 172.21.15.5:0/4161235655 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa8fc077040 0x7fa8fc19f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:48.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:48.259+0000 7fa901400640 1 --2- 172.21.15.5:0/4161235655 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa8fc076720 0x7fa8fc19eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:48.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:48.259+0000 7fa901400640 1 -- 172.21.15.5:0/4161235655 >> 172.21.15.5:0/4161235655 conn(0x7fa8fc0ffa50 msgr2=0x7fa8fc101e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:46:48.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:48.259+0000 7fa901400640 1 -- 172.21.15.5:0/4161235655 shutdown_connections 2024-09-10T09:46:48.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:48.259+0000 7fa901400640 1 -- 172.21.15.5:0/4161235655 wait complete. 2024-09-10T09:46:49.076 INFO:tasks.workunit.client.0.smithi005.stdout: 1 500933 6.23 MB/sec execute 270 sec latency 7985.243 ms 2024-09-10T09:46:49.189 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:46:48 smithi005 ceph-mon[30350]: pgmap v647: 129 pgs: 129 active+clean; 183 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 321 KiB/s rd, 2.0 MiB/s wr, 134 op/s 2024-09-10T09:46:49.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:46:48 smithi159 ceph-mon[35513]: pgmap v647: 129 pgs: 129 active+clean; 183 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 321 KiB/s rd, 2.0 MiB/s wr, 134 op/s 2024-09-10T09:46:49.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:46:48 smithi102 ceph-mon[35442]: pgmap v647: 129 pgs: 129 active+clean; 183 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 321 KiB/s rd, 2.0 MiB/s wr, 134 op/s 2024-09-10T09:46:49.764 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-10T09:46:50.076 INFO:tasks.workunit.client.0.smithi005.stdout: 1 500933 6.21 MB/sec execute 271 sec latency 8985.294 ms 2024-09-10T09:46:50.147 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:46:50.588 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T09:46:50.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:50.762+0000 7f503d800640 1 -- 172.21.15.5:0/3144087469 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f50380745c0 msgr2=0x7f50380749a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:46:50.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:50.762+0000 7f503d800640 1 --2- 172.21.15.5:0/3144087469 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f50380745c0 0x7f50380749a0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f5028009900 tx=0x7f502802a480 comp rx=0 tx=0).stop 2024-09-10T09:46:50.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:50.762+0000 7f503d800640 1 -- 172.21.15.5:0/3144087469 shutdown_connections 2024-09-10T09:46:50.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:50.762+0000 7f503d800640 1 --2- 172.21.15.5:0/3144087469 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f503810e260 0x7f5038110650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:50.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:50.762+0000 7f503d800640 1 --2- 172.21.15.5:0/3144087469 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5038074ee0 0x7f503810dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:50.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:50.762+0000 7f503d800640 1 --2- 172.21.15.5:0/3144087469 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f50380745c0 0x7f50380749a0 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:50.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:50.762+0000 7f503d800640 1 -- 172.21.15.5:0/3144087469 >> 172.21.15.5:0/3144087469 conn(0x7f50380fd740 msgr2=0x7f50380ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:46:50.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:50.762+0000 7f503d800640 1 -- 172.21.15.5:0/3144087469 shutdown_connections 2024-09-10T09:46:50.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:50.762+0000 7f503d800640 1 -- 172.21.15.5:0/3144087469 wait complete. 2024-09-10T09:46:50.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:50.763+0000 7f503d800640 1 Processor -- start 2024-09-10T09:46:50.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:50.763+0000 7f503d800640 1 -- start start 2024-09-10T09:46:50.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:50.764+0000 7f503d800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f50380745c0 0x7f503819a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:46:50.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:50.764+0000 7f503d800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5038074ee0 0x7f503819a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:46:50.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:50.764+0000 7f503d800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f503810e260 0x7f503819ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:46:50.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:50.764+0000 7f503d800640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f5038113fb0 con 0x7f503810e260 2024-09-10T09:46:50.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:50.764+0000 7f503d800640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f5038113e30 con 0x7f50380745c0 2024-09-10T09:46:50.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:50.764+0000 7f503d800640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f5038114130 con 0x7f5038074ee0 2024-09-10T09:46:50.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:50.764+0000 7f503ce00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f503810e260 0x7f503819ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:46:50.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:50.764+0000 7f503ce00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f503810e260 0x7f503819ed80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46454/0 (socket says 172.21.15.5:46454) 2024-09-10T09:46:50.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:50.764+0000 7f503ce00640 1 -- 172.21.15.5:0/1099298812 learned_addr learned my addr 172.21.15.5:0/1099298812 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:46:50.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:50.764+0000 7f5037e00640 1 --2- 172.21.15.5:0/1099298812 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f50380745c0 0x7f503819a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:46:50.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:50.764+0000 7f5037400640 1 --2- 172.21.15.5:0/1099298812 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5038074ee0 0x7f503819a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:46:50.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:50.764+0000 7f503ce00640 1 -- 172.21.15.5:0/1099298812 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5038074ee0 msgr2=0x7f503819a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:46:50.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:50.764+0000 7f503ce00640 1 --2- 172.21.15.5:0/1099298812 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5038074ee0 0x7f503819a9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:50.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:50.764+0000 7f503ce00640 1 -- 172.21.15.5:0/1099298812 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f50380745c0 msgr2=0x7f503819a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:46:50.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:50.764+0000 7f503ce00640 1 --2- 172.21.15.5:0/1099298812 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f50380745c0 0x7f503819a4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:50.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:50.764+0000 7f503ce00640 1 -- 172.21.15.5:0/1099298812 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f503819f460 con 0x7f503810e260 2024-09-10T09:46:50.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:50.765+0000 7f503ce00640 1 --2- 172.21.15.5:0/1099298812 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f503810e260 0x7f503819ed80 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f503000eb40 tx=0x7f503000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:46:50.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:50.765+0000 7f5034c00640 1 -- 172.21.15.5:0/1099298812 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5030004b20 con 0x7f503810e260 2024-09-10T09:46:50.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:50.765+0000 7f5034c00640 1 -- 172.21.15.5:0/1099298812 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5030004cc0 con 0x7f503810e260 2024-09-10T09:46:50.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:50.765+0000 7f503d800640 1 -- 172.21.15.5:0/1099298812 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f503819f750 con 0x7f503810e260 2024-09-10T09:46:50.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:50.765+0000 7f5034c00640 1 -- 172.21.15.5:0/1099298812 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5030010850 con 0x7f503810e260 2024-09-10T09:46:50.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:50.765+0000 7f503d800640 1 -- 172.21.15.5:0/1099298812 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50381a7030 con 0x7f503810e260 2024-09-10T09:46:50.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:50.769+0000 7f5041a44740 1 Processor -- start 2024-09-10T09:46:50.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:50.770+0000 7f5034c00640 1 -- 172.21.15.5:0/1099298812 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f5030005020 con 0x7f503810e260 2024-09-10T09:46:50.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:50.770+0000 7f5041a44740 1 -- start start 2024-09-10T09:46:50.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:50.770+0000 7f5041a44740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556ee8080e40 0x556ee8081220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:46:50.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:50.770+0000 7f5041a44740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556ee8081760 0x556ee808bae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:46:50.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:50.770+0000 7f5041a44740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556ee808c020 0x556ee808e410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:46:50.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:50.770+0000 7f5041a44740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x556ee7fe73e0 con 0x556ee8081760 2024-09-10T09:46:50.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:50.770+0000 7f5041a44740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x556ee7f8ed80 con 0x556ee808c020 2024-09-10T09:46:50.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:50.770+0000 7f5041a44740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x556ee7fe33a0 con 0x556ee8080e40 2024-09-10T09:46:50.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:50.771+0000 7f5037e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556ee8081760 0x556ee808bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:46:50.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:50.771+0000 7f5037400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556ee8080e40 0x556ee8081220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:46:50.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:50.771+0000 7f5037400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556ee8080e40 0x556ee8081220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:55246/0 (socket says 172.21.15.5:55246) 2024-09-10T09:46:50.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:50.771+0000 7f5037400640 1 -- 172.21.15.5:0/3948483557 learned_addr learned my addr 172.21.15.5:0/3948483557 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:46:50.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:50.771+0000 7f5037e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556ee8081760 0x556ee808bae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46466/0 (socket says 172.21.15.5:46466) 2024-09-10T09:46:50.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:50.771+0000 7f5037400640 1 --2- 172.21.15.5:0/3948483557 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556ee808c020 0x556ee808e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:46:50.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:50.771+0000 7f5037400640 1 -- 172.21.15.5:0/3948483557 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556ee808c020 msgr2=0x556ee808e410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:46:50.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:50.771+0000 7f5037400640 1 --2- 172.21.15.5:0/3948483557 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556ee808c020 0x556ee808e410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:50.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:50.771+0000 7f5037400640 1 -- 172.21.15.5:0/3948483557 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556ee8081760 msgr2=0x556ee808bae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:46:50.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:50.771+0000 7f5037400640 1 --2- 172.21.15.5:0/3948483557 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556ee8081760 0x556ee808bae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:50.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:50.771+0000 7f5037400640 1 -- 172.21.15.5:0/3948483557 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556ee7fadae0 con 0x556ee8080e40 2024-09-10T09:46:50.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:50.772+0000 7f5037400640 1 --2- 172.21.15.5:0/3948483557 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556ee8080e40 0x556ee8081220 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f5024009fd0 tx=0x7f5024005850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d0d1c013b1eee6d2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:46:50.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:50.772+0000 7f5034c00640 1 --2- 172.21.15.5:0/1099298812 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f501c0a2b80 0x7f501c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:46:50.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:50.772+0000 7f5034c00640 1 -- 172.21.15.5:0/1099298812 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f50300cfc30 con 0x7f503810e260 2024-09-10T09:46:50.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:50.772+0000 7f502d600640 1 -- 172.21.15.5:0/3948483557 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5024004030 con 0x556ee8080e40 2024-09-10T09:46:50.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:50.772+0000 7f502d600640 1 -- 172.21.15.5:0/3948483557 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5024005500 con 0x556ee8080e40 2024-09-10T09:46:50.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:50.772+0000 7f502d600640 1 -- 172.21.15.5:0/3948483557 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5024004d70 con 0x556ee8080e40 2024-09-10T09:46:50.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:50.772+0000 7f5037e00640 1 --2- 172.21.15.5:0/1099298812 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f501c0a2b80 0x7f501c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:46:50.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:50.773+0000 7f5041a44740 1 -- 172.21.15.5:0/3948483557 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556ee8080e40 msgr2=0x556ee8081220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:46:50.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:50.773+0000 7f5037e00640 1 --2- 172.21.15.5:0/1099298812 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f501c0a2b80 0x7f501c0a5040 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f5028009230 tx=0x7f50280058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:46:50.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:50.773+0000 7f5041a44740 1 --2- 172.21.15.5:0/3948483557 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556ee8080e40 0x556ee8081220 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f5024009fd0 tx=0x7f5024005850 comp rx=0 tx=0).stop 2024-09-10T09:46:50.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:50.773+0000 7f5041a44740 1 -- 172.21.15.5:0/3948483557 shutdown_connections 2024-09-10T09:46:50.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:50.773+0000 7f5041a44740 1 --2- 172.21.15.5:0/3948483557 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556ee808c020 0x556ee808e410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:50.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:50.773+0000 7f5041a44740 1 --2- 172.21.15.5:0/3948483557 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556ee8081760 0x556ee808bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:50.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:50.773+0000 7f5041a44740 1 --2- 172.21.15.5:0/3948483557 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556ee8080e40 0x556ee8081220 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:50.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:50.773+0000 7f5041a44740 1 -- 172.21.15.5:0/3948483557 >> 172.21.15.5:0/3948483557 conn(0x556ee807d3d0 msgr2=0x556ee807d7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:46:50.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:50.773+0000 7f5041a44740 1 -- 172.21.15.5:0/3948483557 shutdown_connections 2024-09-10T09:46:50.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:50.773+0000 7f5041a44740 1 -- 172.21.15.5:0/3948483557 wait complete. 2024-09-10T09:46:50.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:50.773+0000 7f5041a44740 1 -- start start 2024-09-10T09:46:50.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:50.774+0000 7f5041a44740 1 Processor -- start 2024-09-10T09:46:50.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:50.774+0000 7f5041a44740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556ee8080e40 0x556ee8088f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:46:50.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:50.774+0000 7f5041a44740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556ee8081760 0x556ee8089490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:46:50.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:50.774+0000 7f5041a44740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556ee808c020 0x556ee80899d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:46:50.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:50.774+0000 7f5041a44740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x556ee808ebf0 con 0x556ee8080e40 2024-09-10T09:46:50.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:50.774+0000 7f5041a44740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x556ee7fa20f0 con 0x556ee808c020 2024-09-10T09:46:50.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:50.774+0000 7f5041a44740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x556ee808ed70 con 0x556ee8081760 2024-09-10T09:46:50.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:50.774+0000 7f5041a44740 10 client.0 _ll_register_callbacks cb 0x556ee7f89bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:46:50.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:50.775+0000 7f5037e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556ee8080e40 0x556ee8088f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:46:50.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:50.775+0000 7f503ce00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556ee808c020 0x556ee80899d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:46:50.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:50.775+0000 7f503ce00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556ee808c020 0x556ee80899d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:39930/0 (socket says 172.21.15.5:39930) 2024-09-10T09:46:50.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:50.775+0000 7f503ce00640 1 -- 172.21.15.5:0/2055393766 learned_addr learned my addr 172.21.15.5:0/2055393766 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:46:50.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:50.775+0000 7f5037400640 1 --2- 172.21.15.5:0/2055393766 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556ee8081760 0x556ee8089490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:46:50.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:50.775+0000 7f503ce00640 1 -- 172.21.15.5:0/2055393766 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556ee8081760 msgr2=0x556ee8089490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:46:50.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:50.775+0000 7f503ce00640 1 --2- 172.21.15.5:0/2055393766 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556ee8081760 0x556ee8089490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:50.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:50.775+0000 7f503ce00640 1 -- 172.21.15.5:0/2055393766 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556ee8080e40 msgr2=0x556ee8088f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:46:50.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:50.775+0000 7f503ce00640 1 --2- 172.21.15.5:0/2055393766 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556ee8080e40 0x556ee8088f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:50.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:50.775+0000 7f503ce00640 1 -- 172.21.15.5:0/2055393766 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556ee7d583c0 con 0x556ee808c020 2024-09-10T09:46:50.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:50.775+0000 7f5037e00640 1 --2- 172.21.15.5:0/2055393766 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556ee8080e40 0x556ee8088f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:46:50.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:50.775+0000 7f503ce00640 1 --2- 172.21.15.5:0/2055393766 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556ee808c020 0x556ee80899d0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f50300a4780 tx=0x7f503009ba60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ae1f12413b554995 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:46:50.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:50.775+0000 7f5022000640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-09-10T09:46:50.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:50.776+0000 7f5022000640 1 -- 172.21.15.5:0/2055393766 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f503009c040 con 0x556ee808c020 2024-09-10T09:46:50.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:50.776+0000 7f5041a44740 1 -- 172.21.15.5:0/2055393766 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5024005500 con 0x556ee808c020 2024-09-10T09:46:50.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:50.776+0000 7f5022000640 1 -- 172.21.15.5:0/2055393766 <== mon.2 v2:172.21.15.102:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f50300a00d0 con 0x556ee808c020 2024-09-10T09:46:50.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:50.776+0000 7f5022000640 1 -- 172.21.15.5:0/2055393766 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f50300b5020 con 0x556ee808c020 2024-09-10T09:46:50.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:50.776+0000 7f5041a44740 10 client.27056 fetch_fsmap learned FSMap version 35 2024-09-10T09:46:50.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:50.776+0000 7f5041a44740 1 -- 172.21.15.5:0/2055393766 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x556ee809ad30 con 0x556ee808c020 2024-09-10T09:46:50.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:50.776+0000 7f5022000640 1 -- 172.21.15.5:0/2055393766 <== mon.2 v2:172.21.15.102:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5024005500 con 0x556ee808c020 2024-09-10T09:46:50.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:50.776+0000 7f5022000640 1 -- 172.21.15.5:0/2055393766 <== mon.2 v2:172.21.15.102:3300/0 5 ==== fsmap(e 35) ==== 8374+0+0 (secure 0 0 0) 0x7f5030096bb0 con 0x556ee808c020 2024-09-10T09:46:50.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:50.777+0000 7f5041a44740 10 client.27056 fetch_fsmap finished waiting for FSMap version 35 2024-09-10T09:46:50.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:50.777+0000 7f5041a44740 10 client.27056 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-09-10T09:46:50.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:50.777+0000 7f5041a44740 20 client.27056 populate_metadata read hostname 'smithi005' 2024-09-10T09:46:50.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:50.777+0000 7f5041a44740 1 --2- 172.21.15.5:0/2055393766 >> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] conn(0x556ee809c140 0x556ee809e550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:46:50.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:50.777+0000 7f5041a44740 4 client.27056 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:46:50.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:50.777+0000 7f5041a44740 1 -- 172.21.15.5:0/2055393766 --> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556ee7ff4950 con 0x556ee809c140 2024-09-10T09:46:50.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:50.777+0000 7f5037e00640 1 --2- 172.21.15.5:0/2055393766 >> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] conn(0x556ee809c140 0x556ee809e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:46:50.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:50.777+0000 7f5037e00640 1 --2- 172.21.15.5:0/2055393766 >> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] conn(0x556ee809c140 0x556ee809e550 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=a77b93a226474bd0 server_cookie=3d5f8828204ad776 in_seq=0 out_seq=0 2024-09-10T09:46:50.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:50.777+0000 7f5022000640 10 client.27056 ms_handle_connect on v2:172.21.15.102:6836/2689776764 2024-09-10T09:46:50.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:50.778+0000 7f5022000640 1 -- 172.21.15.5:0/2055393766 <== mds.3 v2:172.21.15.102:6836/2689776764 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556ee7ff4950 con 0x556ee809c140 2024-09-10T09:46:50.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:50.778+0000 7f5022000640 10 client.27056 handle_command_reply: tid=0 2024-09-10T09:46:50.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:50.778+0000 7f5041a44740 1 client.27056 shutdown 2024-09-10T09:46:50.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:50.779+0000 7f5041a44740 1 -- 172.21.15.5:0/2055393766 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556ee808c020 msgr2=0x556ee80899d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:46:50.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:50.779+0000 7f5041a44740 1 --2- 172.21.15.5:0/2055393766 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556ee808c020 0x556ee80899d0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f50300a4780 tx=0x7f503009ba60 comp rx=0 tx=0).stop 2024-09-10T09:46:50.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:50.779+0000 7f5041a44740 1 -- 172.21.15.5:0/2055393766 shutdown_connections 2024-09-10T09:46:50.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:50.779+0000 7f5041a44740 1 --2- 172.21.15.5:0/2055393766 >> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] conn(0x556ee809c140 0x556ee809e550 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:50.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:50.779+0000 7f5041a44740 1 --2- 172.21.15.5:0/2055393766 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556ee808c020 0x556ee80899d0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:50.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:50.779+0000 7f5041a44740 1 --2- 172.21.15.5:0/2055393766 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556ee8081760 0x556ee8089490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:50.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:50.779+0000 7f5022000640 0 client.27056 ms_handle_reset on v2:172.21.15.102:6836/2689776764 2024-09-10T09:46:50.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:50.779+0000 7f5041a44740 1 --2- 172.21.15.5:0/2055393766 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556ee8080e40 0x556ee8088f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:50.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:50.779+0000 7f5041a44740 1 -- 172.21.15.5:0/2055393766 >> 172.21.15.5:0/2055393766 conn(0x556ee807d3d0 msgr2=0x556ee8083100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:46:50.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:50.779+0000 7f5041a44740 1 -- 172.21.15.5:0/2055393766 shutdown_connections 2024-09-10T09:46:50.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:50.779+0000 7f5041a44740 1 -- 172.21.15.5:0/2055393766 wait complete. 2024-09-10T09:46:50.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:50.779+0000 7f5041a44740 20 client.27056 trim_cache size 0 max 16384 2024-09-10T09:46:50.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:50.808+0000 7f5041a44740 1 Processor -- start 2024-09-10T09:46:50.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:50.808+0000 7f5041a44740 1 -- start start 2024-09-10T09:46:50.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:50.808+0000 7f5041a44740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556ee8080e40 0x556ee80838f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:46:50.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:50.809+0000 7f5041a44740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556ee8081760 0x556ee808aec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:46:50.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:50.809+0000 7f5041a44740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556ee808c020 0x556ee80a3e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:46:50.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:50.809+0000 7f5041a44740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f50381b0ef0 con 0x556ee8080e40 2024-09-10T09:46:50.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:50.809+0000 7f5041a44740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f50381b2a60 con 0x556ee808c020 2024-09-10T09:46:50.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:50.809+0000 7f5041a44740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f50381b15c0 con 0x556ee8081760 2024-09-10T09:46:50.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:50.809+0000 7f5037e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556ee8080e40 0x556ee80838f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:46:50.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:50.809+0000 7f5037400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556ee8081760 0x556ee808aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:46:50.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:50.809+0000 7f5037400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556ee8081760 0x556ee808aec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:53172/0 (socket says 172.21.15.5:53172) 2024-09-10T09:46:50.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:50.809+0000 7f5037400640 1 -- 172.21.15.5:0/1104893294 learned_addr learned my addr 172.21.15.5:0/1104893294 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:46:50.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:50.809+0000 7f5037e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556ee8080e40 0x556ee80838f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41836/0 (socket says 172.21.15.5:41836) 2024-09-10T09:46:50.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:50.809+0000 7f503ce00640 1 --2- 172.21.15.5:0/1104893294 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556ee808c020 0x556ee80a3e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:46:50.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:50.809+0000 7f5037400640 1 -- 172.21.15.5:0/1104893294 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556ee808c020 msgr2=0x556ee80a3e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:46:50.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:50.809+0000 7f5037400640 1 --2- 172.21.15.5:0/1104893294 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556ee808c020 0x556ee80a3e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:50.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:50.809+0000 7f5037400640 1 -- 172.21.15.5:0/1104893294 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556ee8080e40 msgr2=0x556ee80838f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:46:50.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:50.809+0000 7f5037400640 1 --2- 172.21.15.5:0/1104893294 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556ee8080e40 0x556ee80838f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:50.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:50.809+0000 7f5037400640 1 -- 172.21.15.5:0/1104893294 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556ee8083e30 con 0x556ee8081760 2024-09-10T09:46:50.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:50.810+0000 7f5037400640 1 --2- 172.21.15.5:0/1104893294 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556ee8081760 0x556ee808aec0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f50240058c0 tx=0x7f5024004950 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c944ee4173dbc260 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:46:50.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:50.810+0000 7f502cc00640 1 -- 172.21.15.5:0/1104893294 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f502402e070 con 0x556ee8081760 2024-09-10T09:46:50.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:50.810+0000 7f502cc00640 1 -- 172.21.15.5:0/1104893294 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f502400f790 con 0x556ee8081760 2024-09-10T09:46:50.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:50.810+0000 7f502cc00640 1 -- 172.21.15.5:0/1104893294 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5024003850 con 0x556ee8081760 2024-09-10T09:46:50.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:50.811+0000 7f5041a44740 1 -- 172.21.15.5:0/1104893294 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556ee8081760 msgr2=0x556ee808aec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:46:50.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:50.811+0000 7f5041a44740 1 --2- 172.21.15.5:0/1104893294 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556ee8081760 0x556ee808aec0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f50240058c0 tx=0x7f5024004950 comp rx=0 tx=0).stop 2024-09-10T09:46:50.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:50.811+0000 7f5041a44740 1 -- 172.21.15.5:0/1104893294 shutdown_connections 2024-09-10T09:46:50.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:50.811+0000 7f5041a44740 1 --2- 172.21.15.5:0/1104893294 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556ee808c020 0x556ee80a3e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:50.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:50.811+0000 7f5041a44740 1 --2- 172.21.15.5:0/1104893294 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556ee8081760 0x556ee808aec0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:50.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:50.811+0000 7f5041a44740 1 --2- 172.21.15.5:0/1104893294 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556ee8080e40 0x556ee80838f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:50.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:50.811+0000 7f5041a44740 1 -- 172.21.15.5:0/1104893294 >> 172.21.15.5:0/1104893294 conn(0x556ee807d3d0 msgr2=0x556ee808a7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:46:50.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:50.811+0000 7f5041a44740 1 -- 172.21.15.5:0/1104893294 shutdown_connections 2024-09-10T09:46:50.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:50.812+0000 7f5041a44740 1 -- 172.21.15.5:0/1104893294 wait complete. 2024-09-10T09:46:50.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:50.812+0000 7f5041a44740 1 -- start start 2024-09-10T09:46:50.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:50.812+0000 7f5041a44740 1 Processor -- start 2024-09-10T09:46:50.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:50.813+0000 7f5041a44740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556ee8080e40 0x556ee80942e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:46:50.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:50.813+0000 7f5041a44740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556ee8081760 0x556ee8094820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:46:50.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:50.813+0000 7f5041a44740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556ee808c020 0x556ee8085a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:46:50.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:50.813+0000 7f5041a44740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f50381b2360 con 0x556ee8080e40 2024-09-10T09:46:50.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:50.813+0000 7f5041a44740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f50381b1c90 con 0x556ee808c020 2024-09-10T09:46:50.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:50.813+0000 7f5041a44740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f50381aeed0 con 0x556ee8081760 2024-09-10T09:46:50.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:50.813+0000 7f5041a44740 10 client.0 _ll_register_callbacks cb 0x7f5030004a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:46:50.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:50.813+0000 7f5037e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556ee8080e40 0x556ee80942e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:46:50.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:50.813+0000 7f5037e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556ee8080e40 0x556ee80942e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41846/0 (socket says 172.21.15.5:41846) 2024-09-10T09:46:50.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:50.813+0000 7f5037e00640 1 -- 172.21.15.5:0/1772805881 learned_addr learned my addr 172.21.15.5:0/1772805881 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:46:50.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:50.813+0000 7f5037400640 1 --2- 172.21.15.5:0/1772805881 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556ee8081760 0x556ee8094820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:46:50.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:50.814+0000 7f503ce00640 1 --2- 172.21.15.5:0/1772805881 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556ee808c020 0x556ee8085a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:46:50.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:50.814+0000 7f5037e00640 1 -- 172.21.15.5:0/1772805881 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556ee8081760 msgr2=0x556ee8094820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:46:50.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:50.814+0000 7f5037e00640 1 --2- 172.21.15.5:0/1772805881 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556ee8081760 0x556ee8094820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:50.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:50.814+0000 7f5037e00640 1 -- 172.21.15.5:0/1772805881 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556ee808c020 msgr2=0x556ee8085a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:46:50.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:50.814+0000 7f5037e00640 1 --2- 172.21.15.5:0/1772805881 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556ee808c020 0x556ee8085a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:50.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:50.814+0000 7f5037e00640 1 -- 172.21.15.5:0/1772805881 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556ee8086270 con 0x556ee8080e40 2024-09-10T09:46:50.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:50.814+0000 7f5037400640 1 --2- 172.21.15.5:0/1772805881 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556ee8081760 0x556ee8094820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:46:50.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:50.814+0000 7f503ce00640 1 --2- 172.21.15.5:0/1772805881 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556ee808c020 0x556ee8085a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:46:50.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:50.814+0000 7f5037e00640 1 --2- 172.21.15.5:0/1772805881 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556ee8080e40 0x556ee80942e0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f5028054fd0 tx=0x7f5028009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61ca5814cf919ffa server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:46:50.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:50.814+0000 7f5022000640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-10T09:46:50.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:50.815+0000 7f5022000640 1 -- 172.21.15.5:0/1772805881 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5028035040 con 0x556ee8080e40 2024-09-10T09:46:50.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:50.815+0000 7f5022000640 1 -- 172.21.15.5:0/1772805881 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f5028003dd0 con 0x556ee8080e40 2024-09-10T09:46:50.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:50.815+0000 7f5041a44740 1 -- 172.21.15.5:0/1772805881 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f502400f790 con 0x556ee8080e40 2024-09-10T09:46:50.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:50.815+0000 7f5022000640 1 -- 172.21.15.5:0/1772805881 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f5028066050 con 0x556ee8080e40 2024-09-10T09:46:50.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:50.815+0000 7f5041a44740 10 client.17061 fetch_fsmap learned FSMap version 35 2024-09-10T09:46:50.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:50.815+0000 7f5041a44740 1 -- 172.21.15.5:0/1772805881 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x556ee8106ab0 con 0x556ee8080e40 2024-09-10T09:46:50.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:50.815+0000 7f5022000640 1 -- 172.21.15.5:0/1772805881 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f502400f790 con 0x556ee8080e40 2024-09-10T09:46:50.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:50.816+0000 7f5022000640 1 -- 172.21.15.5:0/1772805881 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 35) ==== 8374+0+0 (secure 0 0 0) 0x7f502805d470 con 0x556ee8080e40 2024-09-10T09:46:50.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:50.816+0000 7f5041a44740 10 client.17061 fetch_fsmap finished waiting for FSMap version 35 2024-09-10T09:46:50.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:50.816+0000 7f5041a44740 10 client.17061 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-09-10T09:46:50.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:50.816+0000 7f5041a44740 20 client.17061 populate_metadata read hostname 'smithi005' 2024-09-10T09:46:50.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:50.816+0000 7f5041a44740 1 --2- 172.21.15.5:0/1772805881 >> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] conn(0x556ee81071b0 0x556ee81095d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:46:50.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:50.816+0000 7f5041a44740 4 client.17061 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T09:46:50.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:50.816+0000 7f5041a44740 1 -- 172.21.15.5:0/1772805881 --> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556ee7feee60 con 0x556ee81071b0 2024-09-10T09:46:50.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:50.816+0000 7f5037400640 1 --2- 172.21.15.5:0/1772805881 >> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] conn(0x556ee81071b0 0x556ee81095d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:46:50.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:50.817+0000 7f5022000640 10 client.17061 ms_handle_connect on v2:172.21.15.102:6836/2689776764 2024-09-10T09:46:50.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:50.817+0000 7f5037400640 1 --2- 172.21.15.5:0/1772805881 >> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] conn(0x556ee81071b0 0x556ee81095d0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=bb77d98e84fb9c2d server_cookie=898084bd8315afdd in_seq=0 out_seq=0 2024-09-10T09:46:50.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:50.817+0000 7f5022000640 1 -- 172.21.15.5:0/1772805881 <== mds.3 v2:172.21.15.102:6836/2689776764 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556ee7feee60 con 0x556ee81071b0 2024-09-10T09:46:50.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:50.817+0000 7f5022000640 10 client.17061 handle_command_reply: tid=0 2024-09-10T09:46:50.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:50.817+0000 7f5041a44740 1 client.17061 shutdown 2024-09-10T09:46:50.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:50.818+0000 7f5041a44740 1 -- 172.21.15.5:0/1772805881 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556ee8080e40 msgr2=0x556ee80942e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:46:50.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:50.818+0000 7f5041a44740 1 --2- 172.21.15.5:0/1772805881 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556ee8080e40 0x556ee80942e0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f5028054fd0 tx=0x7f5028009260 comp rx=0 tx=0).stop 2024-09-10T09:46:50.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:50.818+0000 7f5041a44740 1 -- 172.21.15.5:0/1772805881 shutdown_connections 2024-09-10T09:46:50.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:50.818+0000 7f5041a44740 1 --2- 172.21.15.5:0/1772805881 >> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] conn(0x556ee81071b0 0x556ee81095d0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:50.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:50.818+0000 7f5041a44740 1 --2- 172.21.15.5:0/1772805881 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556ee808c020 0x556ee8085a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:50.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:50.818+0000 7f5041a44740 1 --2- 172.21.15.5:0/1772805881 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556ee8081760 0x556ee8094820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:50.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:50.818+0000 7f5041a44740 1 --2- 172.21.15.5:0/1772805881 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556ee8080e40 0x556ee80942e0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:50.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:50.818+0000 7f5041a44740 1 -- 172.21.15.5:0/1772805881 >> 172.21.15.5:0/1772805881 conn(0x556ee807d3d0 msgr2=0x556ee808afd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:46:50.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:50.818+0000 7f5022000640 0 client.17061 ms_handle_reset on v2:172.21.15.102:6836/2689776764 2024-09-10T09:46:50.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:50.818+0000 7f5041a44740 1 -- 172.21.15.5:0/1772805881 shutdown_connections 2024-09-10T09:46:50.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:50.818+0000 7f5041a44740 1 -- 172.21.15.5:0/1772805881 wait complete. 2024-09-10T09:46:50.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:50.818+0000 7f5041a44740 20 client.17061 trim_cache size 0 max 16384 2024-09-10T09:46:50.822 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-10T09:46:50.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:50.819+0000 7f503d800640 1 -- 172.21.15.5:0/1099298812 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f501c0a2b80 msgr2=0x7f501c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:46:50.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:50.819+0000 7f503d800640 1 --2- 172.21.15.5:0/1099298812 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f501c0a2b80 0x7f501c0a5040 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f5028009230 tx=0x7f50280058d0 comp rx=0 tx=0).stop 2024-09-10T09:46:50.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:50.819+0000 7f503d800640 1 -- 172.21.15.5:0/1099298812 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f503810e260 msgr2=0x7f503819ed80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:46:50.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:50.819+0000 7f503d800640 1 --2- 172.21.15.5:0/1099298812 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f503810e260 0x7f503819ed80 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f503000eb40 tx=0x7f503000c2d0 comp rx=0 tx=0).stop 2024-09-10T09:46:50.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:50.819+0000 7f503d800640 1 -- 172.21.15.5:0/1099298812 shutdown_connections 2024-09-10T09:46:50.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:50.819+0000 7f503d800640 1 --2- 172.21.15.5:0/1099298812 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f501c0a2b80 0x7f501c0a5040 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:50.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:50.819+0000 7f503d800640 1 --2- 172.21.15.5:0/1099298812 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f503810e260 0x7f503819ed80 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:50.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:50.819+0000 7f503d800640 1 --2- 172.21.15.5:0/1099298812 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5038074ee0 0x7f503819a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:50.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:50.819+0000 7f503d800640 1 --2- 172.21.15.5:0/1099298812 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f50380745c0 0x7f503819a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:50.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:50.819+0000 7f503d800640 1 -- 172.21.15.5:0/1099298812 >> 172.21.15.5:0/1099298812 conn(0x7f50380fd740 msgr2=0x7f50380fee20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:46:50.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:50.819+0000 7f503d800640 1 -- 172.21.15.5:0/1099298812 shutdown_connections 2024-09-10T09:46:50.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:50.819+0000 7f503d800640 1 -- 172.21.15.5:0/1099298812 wait complete. 2024-09-10T09:46:51.076 INFO:tasks.workunit.client.0.smithi005.stdout: 1 500933 6.19 MB/sec execute 272 sec latency 9985.357 ms 2024-09-10T09:46:51.306 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:46:50 smithi005 ceph-mon[30350]: pgmap v648: 129 pgs: 129 active+clean; 188 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 352 KiB/s rd, 2.6 MiB/s wr, 138 op/s 2024-09-10T09:46:51.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:46:50 smithi159 ceph-mon[35513]: pgmap v648: 129 pgs: 129 active+clean; 188 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 352 KiB/s rd, 2.6 MiB/s wr, 138 op/s 2024-09-10T09:46:51.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:46:50 smithi102 ceph-mon[35442]: pgmap v648: 129 pgs: 129 active+clean; 188 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 352 KiB/s rd, 2.6 MiB/s wr, 138 op/s 2024-09-10T09:46:51.854 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-10T09:46:51.879 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:46:52.076 INFO:tasks.workunit.client.0.smithi005.stdout: 1 500933 6.16 MB/sec execute 273 sec latency 10985.444 ms 2024-09-10T09:46:52.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:52.447+0000 7f0576200640 1 -- 172.21.15.5:0/3760938142 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f05701049a0 msgr2=0x7f05701089f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:46:52.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:52.447+0000 7f0576200640 1 --2- 172.21.15.5:0/3760938142 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f05701049a0 0x7f05701089f0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f055c009900 tx=0x7f055c02a410 comp rx=0 tx=0).stop 2024-09-10T09:46:52.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:52.447+0000 7f0576200640 1 -- 172.21.15.5:0/3760938142 shutdown_connections 2024-09-10T09:46:52.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:52.447+0000 7f0576200640 1 --2- 172.21.15.5:0/3760938142 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0570109120 0x7f0570110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:52.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:52.447+0000 7f0576200640 1 --2- 172.21.15.5:0/3760938142 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f05701049a0 0x7f05701089f0 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:52.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:52.447+0000 7f0576200640 1 --2- 172.21.15.5:0/3760938142 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0570103ff0 0x7f05701043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:52.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:52.447+0000 7f0576200640 1 -- 172.21.15.5:0/3760938142 >> 172.21.15.5:0/3760938142 conn(0x7f05700ffa90 msgr2=0x7f0570101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:46:52.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:52.447+0000 7f0576200640 1 -- 172.21.15.5:0/3760938142 shutdown_connections 2024-09-10T09:46:52.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:52.447+0000 7f0576200640 1 -- 172.21.15.5:0/3760938142 wait complete. 2024-09-10T09:46:52.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:52.448+0000 7f0576200640 1 Processor -- start 2024-09-10T09:46:52.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:52.448+0000 7f0576200640 1 -- start start 2024-09-10T09:46:52.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:52.448+0000 7f0576200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0570103ff0 0x7f05700739e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:46:52.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:52.448+0000 7f0576200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f05701049a0 0x7f057006ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:46:52.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:52.449+0000 7f0576200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0570109120 0x7f057006cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:46:52.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:52.449+0000 7f0576200640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f0570114120 con 0x7f05701049a0 2024-09-10T09:46:52.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:52.449+0000 7f0576200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f0570113fa0 con 0x7f0570109120 2024-09-10T09:46:52.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:52.449+0000 7f0576200640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f05701142a0 con 0x7f0570103ff0 2024-09-10T09:46:52.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:52.449+0000 7f056ea00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f05701049a0 0x7f057006ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:46:52.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:52.449+0000 7f056ea00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f05701049a0 0x7f057006ca70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41868/0 (socket says 172.21.15.5:41868) 2024-09-10T09:46:52.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:52.449+0000 7f056ea00640 1 -- 172.21.15.5:0/2710180063 learned_addr learned my addr 172.21.15.5:0/2710180063 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:46:52.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:52.449+0000 7f056f400640 1 --2- 172.21.15.5:0/2710180063 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0570103ff0 0x7f05700739e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:46:52.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:52.449+0000 7f056fe00640 1 --2- 172.21.15.5:0/2710180063 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0570109120 0x7f057006cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:46:52.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:52.449+0000 7f056ea00640 1 -- 172.21.15.5:0/2710180063 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0570103ff0 msgr2=0x7f05700739e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:46:52.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:52.449+0000 7f056ea00640 1 --2- 172.21.15.5:0/2710180063 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0570103ff0 0x7f05700739e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:52.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:52.449+0000 7f056ea00640 1 -- 172.21.15.5:0/2710180063 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0570109120 msgr2=0x7f057006cfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:46:52.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:52.449+0000 7f056ea00640 1 --2- 172.21.15.5:0/2710180063 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0570109120 0x7f057006cfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:52.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:52.449+0000 7f056ea00640 1 -- 172.21.15.5:0/2710180063 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f057006d570 con 0x7f05701049a0 2024-09-10T09:46:52.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:52.450+0000 7f056ea00640 1 --2- 172.21.15.5:0/2710180063 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f05701049a0 0x7f057006ca70 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f055c006010 tx=0x7f055c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:46:52.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:52.450+0000 7f0567e00640 1 -- 172.21.15.5:0/2710180063 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f055c03f030 con 0x7f05701049a0 2024-09-10T09:46:52.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:52.450+0000 7f0567e00640 1 -- 172.21.15.5:0/2710180063 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f055c003ce0 con 0x7f05701049a0 2024-09-10T09:46:52.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:52.450+0000 7f0567e00640 1 -- 172.21.15.5:0/2710180063 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f055c035070 con 0x7f05701049a0 2024-09-10T09:46:52.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:52.450+0000 7f0576200640 1 -- 172.21.15.5:0/2710180063 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f057006d800 con 0x7f05701049a0 2024-09-10T09:46:52.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:52.450+0000 7f0576200640 1 -- 172.21.15.5:0/2710180063 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0570071790 con 0x7f05701049a0 2024-09-10T09:46:52.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:52.452+0000 7f0576200640 1 -- 172.21.15.5:0/2710180063 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f05701054d0 con 0x7f05701049a0 2024-09-10T09:46:52.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:52.455+0000 7f0567e00640 1 -- 172.21.15.5:0/2710180063 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f055c042030 con 0x7f05701049a0 2024-09-10T09:46:52.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:52.456+0000 7f0567e00640 1 --2- 172.21.15.5:0/2710180063 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f05540a2b80 0x7f05540a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:46:52.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:52.456+0000 7f0567e00640 1 -- 172.21.15.5:0/2710180063 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f055c0efec0 con 0x7f05701049a0 2024-09-10T09:46:52.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:52.456+0000 7f056f400640 1 --2- 172.21.15.5:0/2710180063 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f05540a2b80 0x7f05540a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:46:52.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:52.457+0000 7f056f400640 1 --2- 172.21.15.5:0/2710180063 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f05540a2b80 0x7f05540a5040 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f0560009bd0 tx=0x7f0560005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:46:52.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:52.458+0000 7f0567e00640 1 -- 172.21.15.5:0/2710180063 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f055c049030 con 0x7f05701049a0 2024-09-10T09:46:52.492 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:46:52.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:52.686+0000 7f0576200640 1 -- 172.21.15.5:0/2710180063 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f05701043d0 con 0x7f05701049a0 2024-09-10T09:46:52.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:52.687+0000 7f0567e00640 1 -- 172.21.15.5:0/2710180063 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) ==== 94+0+11970 (secure 0 0 0) 0x7f05701043d0 con 0x7f05701049a0 2024-09-10T09:46:52.688 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-10T09:46:52.688 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":35,"btime":"2024-09-10T09:46:17:834001+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14454,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/2970594311","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2970594311},{"type":"v1","addr":"172.21.15.5:6835","nonce":2970594311}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14514,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6839/3616502174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":3616502174},{"type":"v1","addr":"172.21.15.5:6839","nonce":3616502174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":35,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T09:35:29.626287+0000","modified":"2024-09-10T09:46:16.966068+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24542,"mds_1":24535,"mds_2":24511,"mds_3":24506,"mds_4":24443},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14520":{"gid":14520,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/2073599842","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":2073599842},{"type":"v1","addr":"172.21.15.5:6841","nonce":2073599842}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"k","rank":4,"incarnation":26,"state":"up:active","state_seq":24,"addr":"172.21.15.102:6833/2457007427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":2457007427},{"type":"v1","addr":"172.21.15.102:6833","nonce":2457007427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/356445159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":356445159},{"type":"v1","addr":"172.21.15.159:6833","nonce":356445159}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6837/2894314507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":2894314507},{"type":"v1","addr":"172.21.15.5:6837","nonce":2894314507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.102:6835/1647549111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":1647549111},{"type":"v1","addr":"172.21.15.102:6835","nonce":1647549111}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6835/952657242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":952657242},{"type":"v1","addr":"172.21.15.159:6835","nonce":952657242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":19,"addr":"172.21.15.102:6837/2689776764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":2689776764},{"type":"v1","addr":"172.21.15.102:6837","nonce":2689776764}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.159:6837/620627639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":620627639},{"type":"v1","addr":"172.21.15.159:6837","nonce":620627639}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24535":{"gid":24535,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":18,"addr":"172.21.15.159:6839/2073739599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":2073739599},{"type":"v1","addr":"172.21.15.159:6839","nonce":2073739599}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24542":{"gid":24542,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.102:6839/335409749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6838","nonce":335409749},{"type":"v1","addr":"172.21.15.102:6839","nonce":335409749}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24542,"qdb_cluster":[24443,24506,24511,24535,24542]},"id":1}]} 2024-09-10T09:46:52.689 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 35 2024-09-10T09:46:52.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:52.689+0000 7f0576200640 1 -- 172.21.15.5:0/2710180063 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f05540a2b80 msgr2=0x7f05540a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:46:52.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:52.689+0000 7f0576200640 1 --2- 172.21.15.5:0/2710180063 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f05540a2b80 0x7f05540a5040 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f0560009bd0 tx=0x7f0560005f90 comp rx=0 tx=0).stop 2024-09-10T09:46:52.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:52.689+0000 7f0576200640 1 -- 172.21.15.5:0/2710180063 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f05701049a0 msgr2=0x7f057006ca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:46:52.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:52.689+0000 7f0576200640 1 --2- 172.21.15.5:0/2710180063 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f05701049a0 0x7f057006ca70 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f055c006010 tx=0x7f055c0042f0 comp rx=0 tx=0).stop 2024-09-10T09:46:52.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:52.689+0000 7f0576200640 1 -- 172.21.15.5:0/2710180063 shutdown_connections 2024-09-10T09:46:52.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:52.689+0000 7f0576200640 1 --2- 172.21.15.5:0/2710180063 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f05540a2b80 0x7f05540a5040 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:52.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:52.689+0000 7f0576200640 1 --2- 172.21.15.5:0/2710180063 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0570109120 0x7f057006cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:52.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:52.689+0000 7f0576200640 1 --2- 172.21.15.5:0/2710180063 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f05701049a0 0x7f057006ca70 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:52.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:52.689+0000 7f0576200640 1 --2- 172.21.15.5:0/2710180063 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0570103ff0 0x7f05700739e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:52.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:52.689+0000 7f0576200640 1 -- 172.21.15.5:0/2710180063 >> 172.21.15.5:0/2710180063 conn(0x7f05700ffa90 msgr2=0x7f0570100d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:46:52.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:52.690+0000 7f0576200640 1 -- 172.21.15.5:0/2710180063 shutdown_connections 2024-09-10T09:46:52.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:52.690+0000 7f0576200640 1 -- 172.21.15.5:0/2710180063 wait complete. 2024-09-10T09:46:53.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:46:52 smithi005 ceph-mon[30350]: pgmap v649: 129 pgs: 129 active+clean; 194 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 314 KiB/s rd, 3.1 MiB/s wr, 124 op/s 2024-09-10T09:46:53.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:46:52 smithi005 ceph-mon[30350]: from='client.? 172.21.15.5:0/2710180063' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:46:53.076 INFO:tasks.workunit.client.0.smithi005.stdout: 1 500933 6.14 MB/sec execute 274 sec latency 11985.506 ms 2024-09-10T09:46:53.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:46:52 smithi159 ceph-mon[35513]: pgmap v649: 129 pgs: 129 active+clean; 194 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 314 KiB/s rd, 3.1 MiB/s wr, 124 op/s 2024-09-10T09:46:53.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:46:52 smithi159 ceph-mon[35513]: from='client.? 172.21.15.5:0/2710180063' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:46:53.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:46:52 smithi102 ceph-mon[35442]: pgmap v649: 129 pgs: 129 active+clean; 194 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 314 KiB/s rd, 3.1 MiB/s wr, 124 op/s 2024-09-10T09:46:53.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:46:52 smithi102 ceph-mon[35442]: from='client.? 172.21.15.5:0/2710180063' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:46:54.076 INFO:tasks.workunit.client.0.smithi005.stdout: 1 500933 6.12 MB/sec execute 275 sec latency 12985.548 ms 2024-09-10T09:46:54.125 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24542 quiesce db --set-id fcdeb12e --release --await' 2024-09-10T09:46:54.125 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24542 quiesce db --set-id fcdeb12e --release --await 2024-09-10T09:46:54.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:54.389+0000 7f31f6e00640 1 -- 172.21.15.5:0/4182599953 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f31f0077040 msgr2=0x7f31f0112330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:46:54.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:54.389+0000 7f31f6e00640 1 --2- 172.21.15.5:0/4182599953 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f31f0077040 0x7f31f0112330 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f31dc009900 tx=0x7f31dc02a400 comp rx=0 tx=0).stop 2024-09-10T09:46:54.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:54.389+0000 7f31f6e00640 1 -- 172.21.15.5:0/4182599953 shutdown_connections 2024-09-10T09:46:54.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:54.389+0000 7f31f6e00640 1 --2- 172.21.15.5:0/4182599953 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f31f0112870 0x7f31f0114c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:54.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:54.389+0000 7f31f6e00640 1 --2- 172.21.15.5:0/4182599953 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f31f0077040 0x7f31f0112330 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:54.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:54.389+0000 7f31f6e00640 1 --2- 172.21.15.5:0/4182599953 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f31f0076720 0x7f31f0076b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:54.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:54.389+0000 7f31f6e00640 1 -- 172.21.15.5:0/4182599953 >> 172.21.15.5:0/4182599953 conn(0x7f31f00ff8f0 msgr2=0x7f31f0101d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:46:54.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:54.389+0000 7f31f6e00640 1 -- 172.21.15.5:0/4182599953 shutdown_connections 2024-09-10T09:46:54.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:54.389+0000 7f31f6e00640 1 -- 172.21.15.5:0/4182599953 wait complete. 2024-09-10T09:46:54.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:54.390+0000 7f31f6e00640 1 Processor -- start 2024-09-10T09:46:54.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:54.390+0000 7f31f6e00640 1 -- start start 2024-09-10T09:46:54.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:54.391+0000 7f31f6e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f31f0076720 0x7f31f01a4690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:46:54.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:54.391+0000 7f31f6e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f31f0077040 0x7f31f01a4bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:46:54.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:54.391+0000 7f31f6e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f31f0112870 0x7f31f019e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:46:54.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:54.391+0000 7f31f6e00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f31f0118450 con 0x7f31f0112870 2024-09-10T09:46:54.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:54.391+0000 7f31f6e00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f31f01182d0 con 0x7f31f0077040 2024-09-10T09:46:54.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:54.391+0000 7f31f6e00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f31f01185d0 con 0x7f31f0076720 2024-09-10T09:46:54.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:54.391+0000 7f31f5a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f31f0076720 0x7f31f01a4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:46:54.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:54.391+0000 7f31f5a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f31f0076720 0x7f31f01a4690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:53212/0 (socket says 172.21.15.5:53212) 2024-09-10T09:46:54.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:54.391+0000 7f31f5a00640 1 -- 172.21.15.5:0/187676750 learned_addr learned my addr 172.21.15.5:0/187676750 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:46:54.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:54.391+0000 7f31f6400640 1 --2- 172.21.15.5:0/187676750 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f31f0112870 0x7f31f019e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:46:54.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:54.392+0000 7f31f5a00640 1 -- 172.21.15.5:0/187676750 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f31f0077040 msgr2=0x7f31f01a4bd0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-10T09:46:54.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:54.392+0000 7f31f5a00640 1 --2- 172.21.15.5:0/187676750 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f31f0077040 0x7f31f01a4bd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:54.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:54.392+0000 7f31f5a00640 1 -- 172.21.15.5:0/187676750 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f31f0112870 msgr2=0x7f31f019e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:46:54.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:54.392+0000 7f31f5a00640 1 --2- 172.21.15.5:0/187676750 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f31f0112870 0x7f31f019e760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:54.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:54.392+0000 7f31f5a00640 1 -- 172.21.15.5:0/187676750 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31f019efd0 con 0x7f31f0076720 2024-09-10T09:46:54.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:54.392+0000 7f31f5a00640 1 --2- 172.21.15.5:0/187676750 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f31f0076720 0x7f31f01a4690 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f31e400e9f0 tx=0x7f31e400edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:46:54.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:54.392+0000 7f31ee000640 1 -- 172.21.15.5:0/187676750 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31e400cd80 con 0x7f31f0076720 2024-09-10T09:46:54.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:54.392+0000 7f31ee000640 1 -- 172.21.15.5:0/187676750 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f31e4005500 con 0x7f31f0076720 2024-09-10T09:46:54.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:54.393+0000 7f31f6e00640 1 -- 172.21.15.5:0/187676750 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31f019f2c0 con 0x7f31f0076720 2024-09-10T09:46:54.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:54.393+0000 7f31ee000640 1 -- 172.21.15.5:0/187676750 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31e4010690 con 0x7f31f0076720 2024-09-10T09:46:54.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:54.393+0000 7f31f6e00640 1 -- 172.21.15.5:0/187676750 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31f01ab610 con 0x7f31f0076720 2024-09-10T09:46:54.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:54.396+0000 7f31ee000640 1 -- 172.21.15.5:0/187676750 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f31e40046c0 con 0x7f31f0076720 2024-09-10T09:46:54.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:54.397+0000 7f31fafe6740 1 Processor -- start 2024-09-10T09:46:54.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:54.397+0000 7f31ee000640 1 --2- 172.21.15.5:0/187676750 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f31d80a2c00 0x7f31d80a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:46:54.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:54.397+0000 7f31fafe6740 1 -- start start 2024-09-10T09:46:54.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:54.398+0000 7f31ee000640 1 -- 172.21.15.5:0/187676750 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f31e40cf8c0 con 0x7f31f0076720 2024-09-10T09:46:54.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:54.398+0000 7f31f6400640 1 --2- 172.21.15.5:0/187676750 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f31d80a2c00 0x7f31d80a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:46:54.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:54.398+0000 7f31fafe6740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e787f7de40 0x55e787f7e220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:46:54.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:54.398+0000 7f31fafe6740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e787f7e760 0x55e787f88ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:46:54.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:54.398+0000 7f31fafe6740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e787f89020 0x55e787f8b410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:46:54.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:54.398+0000 7f31fafe6740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55e787ee43e0 con 0x55e787f7e760 2024-09-10T09:46:54.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:54.398+0000 7f31fafe6740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55e787e8bd80 con 0x55e787f7de40 2024-09-10T09:46:54.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:54.398+0000 7f31fafe6740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55e787ee03a0 con 0x55e787f89020 2024-09-10T09:46:54.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:54.398+0000 7f31f5a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e787f7de40 0x55e787f7e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:46:54.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:54.398+0000 7f31f5a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e787f7de40 0x55e787f7e220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:41958/0 (socket says 172.21.15.5:41958) 2024-09-10T09:46:54.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:54.398+0000 7f31f5a00640 1 -- 172.21.15.5:0/2144089304 learned_addr learned my addr 172.21.15.5:0/2144089304 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:46:54.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:54.398+0000 7f31f5000640 1 --2- 172.21.15.5:0/2144089304 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e787f7e760 0x55e787f88ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:46:54.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:54.399+0000 7f31f5a00640 1 -- 172.21.15.5:0/2144089304 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e787f89020 msgr2=0x55e787f8b410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:46:54.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:54.399+0000 7f31f5a00640 1 --2- 172.21.15.5:0/2144089304 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e787f89020 0x55e787f8b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:54.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:54.399+0000 7f31f5a00640 1 -- 172.21.15.5:0/2144089304 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e787f7e760 msgr2=0x55e787f88ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:46:54.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:54.399+0000 7f31f5a00640 1 --2- 172.21.15.5:0/2144089304 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e787f7e760 0x55e787f88ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:54.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:54.399+0000 7f31f5a00640 1 -- 172.21.15.5:0/2144089304 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e787eaaae0 con 0x55e787f7de40 2024-09-10T09:46:54.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:54.399+0000 7f31f6400640 1 --2- 172.21.15.5:0/187676750 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f31d80a2c00 0x7f31d80a50c0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f31f006a760 tx=0x7f31e8009600 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:46:54.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:54.406+0000 7f31f5a00640 1 --2- 172.21.15.5:0/2144089304 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e787f7de40 0x55e787f7e220 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f31e409bc60 tx=0x7f31e409b630 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bd1d8c41c9532041 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:46:54.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:54.406+0000 7f31e2a00640 1 -- 172.21.15.5:0/2144089304 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31e40a0080 con 0x55e787f7de40 2024-09-10T09:46:54.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:54.406+0000 7f31e2a00640 1 -- 172.21.15.5:0/2144089304 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f31e40a4940 con 0x55e787f7de40 2024-09-10T09:46:54.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:54.406+0000 7f31e2a00640 1 -- 172.21.15.5:0/2144089304 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31e409b380 con 0x55e787f7de40 2024-09-10T09:46:54.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:54.407+0000 7f31fafe6740 1 -- 172.21.15.5:0/2144089304 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e787f7de40 msgr2=0x55e787f7e220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:46:54.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:54.407+0000 7f31fafe6740 1 --2- 172.21.15.5:0/2144089304 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e787f7de40 0x55e787f7e220 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f31e409bc60 tx=0x7f31e409b630 comp rx=0 tx=0).stop 2024-09-10T09:46:54.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:54.408+0000 7f31fafe6740 1 -- 172.21.15.5:0/2144089304 shutdown_connections 2024-09-10T09:46:54.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:54.408+0000 7f31fafe6740 1 --2- 172.21.15.5:0/2144089304 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e787f89020 0x55e787f8b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:54.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:54.408+0000 7f31fafe6740 1 --2- 172.21.15.5:0/2144089304 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e787f7e760 0x55e787f88ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:54.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:54.408+0000 7f31fafe6740 1 --2- 172.21.15.5:0/2144089304 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e787f7de40 0x55e787f7e220 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:54.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:54.408+0000 7f31fafe6740 1 -- 172.21.15.5:0/2144089304 >> 172.21.15.5:0/2144089304 conn(0x55e787f7a3d0 msgr2=0x55e787f7a7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:46:54.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:54.408+0000 7f31fafe6740 1 -- 172.21.15.5:0/2144089304 shutdown_connections 2024-09-10T09:46:54.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:54.408+0000 7f31fafe6740 1 -- 172.21.15.5:0/2144089304 wait complete. 2024-09-10T09:46:54.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:54.408+0000 7f31fafe6740 1 -- start start 2024-09-10T09:46:54.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:54.409+0000 7f31fafe6740 1 Processor -- start 2024-09-10T09:46:54.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:54.409+0000 7f31fafe6740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e787f7de40 0x55e787f85f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:46:54.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:54.409+0000 7f31fafe6740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e787f7e760 0x55e787f86460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:46:54.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:54.409+0000 7f31fafe6740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e787f89020 0x55e787f869a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:46:54.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:54.409+0000 7f31fafe6740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55e787f8bbf0 con 0x55e787f7e760 2024-09-10T09:46:54.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:54.409+0000 7f31fafe6740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55e787e9f0f0 con 0x55e787f89020 2024-09-10T09:46:54.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:54.409+0000 7f31fafe6740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55e787f8bd70 con 0x55e787f7de40 2024-09-10T09:46:54.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:54.409+0000 7f31fafe6740 10 client.0 _ll_register_callbacks cb 0x55e787e86bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:46:54.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:54.410+0000 7f31f5000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e787f7e760 0x55e787f86460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:46:54.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:54.410+0000 7f31f5000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e787f7e760 0x55e787f86460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41906/0 (socket says 172.21.15.5:41906) 2024-09-10T09:46:54.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:54.410+0000 7f31f5000640 1 -- 172.21.15.5:0/3882202305 learned_addr learned my addr 172.21.15.5:0/3882202305 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:46:54.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:54.410+0000 7f31f5a00640 1 --2- 172.21.15.5:0/3882202305 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e787f7de40 0x55e787f85f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:46:54.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:54.410+0000 7f31f6400640 1 --2- 172.21.15.5:0/3882202305 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e787f89020 0x55e787f869a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:46:54.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:54.410+0000 7f31f5000640 1 -- 172.21.15.5:0/3882202305 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e787f7de40 msgr2=0x55e787f85f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:46:54.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:54.410+0000 7f31f5000640 1 --2- 172.21.15.5:0/3882202305 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e787f7de40 0x55e787f85f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:54.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:54.410+0000 7f31f5000640 1 -- 172.21.15.5:0/3882202305 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e787f89020 msgr2=0x55e787f869a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:46:54.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:54.410+0000 7f31f5000640 1 --2- 172.21.15.5:0/3882202305 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e787f89020 0x55e787f869a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:54.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:54.410+0000 7f31f5000640 1 -- 172.21.15.5:0/3882202305 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e787c553c0 con 0x55e787f7e760 2024-09-10T09:46:54.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:54.411+0000 7f31f5000640 1 --2- 172.21.15.5:0/3882202305 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e787f7e760 0x55e787f86460 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f31f019ffb0 tx=0x7f31dc005270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bc06ae35c52a490 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:46:54.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:54.411+0000 7f31d7400640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-10T09:46:54.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:54.411+0000 7f31d7400640 1 -- 172.21.15.5:0/3882202305 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31dc040070 con 0x55e787f7e760 2024-09-10T09:46:54.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:54.411+0000 7f31fafe6740 1 -- 172.21.15.5:0/3882202305 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f31e40a4940 con 0x55e787f7e760 2024-09-10T09:46:54.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:54.411+0000 7f31d7400640 1 -- 172.21.15.5:0/3882202305 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f31dc0046c0 con 0x55e787f7e760 2024-09-10T09:46:54.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:54.411+0000 7f31d7400640 1 -- 172.21.15.5:0/3882202305 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f31dc04c050 con 0x55e787f7e760 2024-09-10T09:46:54.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:54.411+0000 7f31fafe6740 10 client.17079 fetch_fsmap learned FSMap version 35 2024-09-10T09:46:54.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:54.411+0000 7f31fafe6740 1 -- 172.21.15.5:0/3882202305 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55e787f97c40 con 0x55e787f7e760 2024-09-10T09:46:54.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:54.412+0000 7f31d7400640 1 -- 172.21.15.5:0/3882202305 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f31e40a4940 con 0x55e787f7e760 2024-09-10T09:46:54.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:54.412+0000 7f31d7400640 1 -- 172.21.15.5:0/3882202305 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 35) ==== 8374+0+0 (secure 0 0 0) 0x7f31dc03a040 con 0x55e787f7e760 2024-09-10T09:46:54.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:54.412+0000 7f31fafe6740 10 client.17079 fetch_fsmap finished waiting for FSMap version 35 2024-09-10T09:46:54.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:54.412+0000 7f31fafe6740 10 client.17079 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-10T09:46:54.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:54.412+0000 7f31fafe6740 20 client.17079 populate_metadata read hostname 'smithi005' 2024-09-10T09:46:54.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:54.412+0000 7f31fafe6740 1 --2- 172.21.15.5:0/3882202305 >> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] conn(0x55e787f990a0 0x55e787f9b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:46:54.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:54.412+0000 7f31fafe6740 4 client.17079 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:46:54.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:54.412+0000 7f31fafe6740 1 -- 172.21.15.5:0/3882202305 --> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e787ef1950 con 0x55e787f990a0 2024-09-10T09:46:54.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:54.413+0000 7f31f5a00640 1 --2- 172.21.15.5:0/3882202305 >> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] conn(0x55e787f990a0 0x55e787f9b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:46:54.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:54.413+0000 7f31f5a00640 1 --2- 172.21.15.5:0/3882202305 >> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] conn(0x55e787f990a0 0x55e787f9b4e0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ac3fe34efbcbdfb6 server_cookie=29c5a567b6c67490 in_seq=0 out_seq=0 2024-09-10T09:46:54.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:54.413+0000 7f31d7400640 10 client.17079 ms_handle_connect on v2:172.21.15.159:6836/620627639 2024-09-10T09:46:54.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:54.414+0000 7f31d7400640 1 -- 172.21.15.5:0/3882202305 <== mds.2 v2:172.21.15.159:6836/620627639 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e787ef1950 con 0x55e787f990a0 2024-09-10T09:46:54.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:54.414+0000 7f31d7400640 10 client.17079 handle_command_reply: tid=0 2024-09-10T09:46:54.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:54.414+0000 7f31fafe6740 1 client.17079 shutdown 2024-09-10T09:46:54.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:54.414+0000 7f31fafe6740 1 -- 172.21.15.5:0/3882202305 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e787f7e760 msgr2=0x55e787f86460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:46:54.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:54.414+0000 7f31fafe6740 1 --2- 172.21.15.5:0/3882202305 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e787f7e760 0x55e787f86460 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f31f019ffb0 tx=0x7f31dc005270 comp rx=0 tx=0).stop 2024-09-10T09:46:54.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:54.414+0000 7f31fafe6740 1 -- 172.21.15.5:0/3882202305 shutdown_connections 2024-09-10T09:46:54.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:54.414+0000 7f31fafe6740 1 --2- 172.21.15.5:0/3882202305 >> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] conn(0x55e787f990a0 0x55e787f9b4e0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:54.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:54.414+0000 7f31d7400640 0 client.17079 ms_handle_reset on v2:172.21.15.159:6836/620627639 2024-09-10T09:46:54.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:54.414+0000 7f31fafe6740 1 --2- 172.21.15.5:0/3882202305 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e787f89020 0x55e787f869a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:54.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:54.415+0000 7f31fafe6740 1 --2- 172.21.15.5:0/3882202305 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e787f7e760 0x55e787f86460 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:54.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:54.415+0000 7f31fafe6740 1 --2- 172.21.15.5:0/3882202305 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e787f7de40 0x55e787f85f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:54.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:54.415+0000 7f31fafe6740 1 -- 172.21.15.5:0/3882202305 >> 172.21.15.5:0/3882202305 conn(0x55e787f7a3d0 msgr2=0x55e787f80100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:46:54.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:54.415+0000 7f31fafe6740 1 -- 172.21.15.5:0/3882202305 shutdown_connections 2024-09-10T09:46:54.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:54.415+0000 7f31fafe6740 1 -- 172.21.15.5:0/3882202305 wait complete. 2024-09-10T09:46:54.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:54.415+0000 7f31fafe6740 20 client.17079 trim_cache size 0 max 16384 2024-09-10T09:46:54.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:54.429+0000 7f31fafe6740 1 Processor -- start 2024-09-10T09:46:54.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:54.429+0000 7f31fafe6740 1 -- start start 2024-09-10T09:46:54.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:54.429+0000 7f31fafe6740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e787f7de40 0x55e787f847f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:46:54.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:54.429+0000 7f31fafe6740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e787f7e760 0x55e787f867f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:46:54.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:54.429+0000 7f31fafe6740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e787f89020 0x55e787fa0db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:46:54.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:54.429+0000 7f31fafe6740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f31f01b5330 con 0x55e787f7e760 2024-09-10T09:46:54.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:54.429+0000 7f31fafe6740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f31f01b6ea0 con 0x55e787f89020 2024-09-10T09:46:54.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:54.429+0000 7f31fafe6740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f31f01b5a00 con 0x55e787f7de40 2024-09-10T09:46:54.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:54.430+0000 7f31f5000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e787f7e760 0x55e787f867f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:46:54.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:54.430+0000 7f31f5a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e787f7de40 0x55e787f847f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:46:54.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:54.430+0000 7f31f5a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e787f7de40 0x55e787f847f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:53240/0 (socket says 172.21.15.5:53240) 2024-09-10T09:46:54.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:54.430+0000 7f31f5a00640 1 -- 172.21.15.5:0/491794565 learned_addr learned my addr 172.21.15.5:0/491794565 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:46:54.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:54.430+0000 7f31f6400640 1 --2- 172.21.15.5:0/491794565 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e787f89020 0x55e787fa0db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:46:54.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:54.430+0000 7f31f5a00640 1 -- 172.21.15.5:0/491794565 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e787f89020 msgr2=0x55e787fa0db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:46:54.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:54.430+0000 7f31f5a00640 1 --2- 172.21.15.5:0/491794565 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e787f89020 0x55e787fa0db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:54.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:54.430+0000 7f31f5a00640 1 -- 172.21.15.5:0/491794565 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e787f7e760 msgr2=0x55e787f867f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:46:54.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:54.430+0000 7f31f5a00640 1 --2- 172.21.15.5:0/491794565 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e787f7e760 0x55e787f867f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:54.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:54.430+0000 7f31f5a00640 1 -- 172.21.15.5:0/491794565 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e787f7e590 con 0x55e787f7de40 2024-09-10T09:46:54.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:54.430+0000 7f31f6400640 1 --2- 172.21.15.5:0/491794565 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e787f89020 0x55e787fa0db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:46:54.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:54.431+0000 7f31f5a00640 1 --2- 172.21.15.5:0/491794565 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e787f7de40 0x55e787f847f0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f31e4098130 tx=0x7f31e40a4550 comp rx=0 tx=0).ready entity=mon.1 client_cookie=472779f8c4d4baea server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:46:54.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:54.431+0000 7f31e2000640 1 -- 172.21.15.5:0/491794565 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31e40ba040 con 0x55e787f7de40 2024-09-10T09:46:54.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:54.431+0000 7f31e2000640 1 -- 172.21.15.5:0/491794565 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f31e40a9070 con 0x55e787f7de40 2024-09-10T09:46:54.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:54.431+0000 7f31e2000640 1 -- 172.21.15.5:0/491794565 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31e40984e0 con 0x55e787f7de40 2024-09-10T09:46:54.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:54.432+0000 7f31fafe6740 1 -- 172.21.15.5:0/491794565 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e787f7de40 msgr2=0x55e787f847f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:46:54.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:54.432+0000 7f31fafe6740 1 --2- 172.21.15.5:0/491794565 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e787f7de40 0x55e787f847f0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f31e4098130 tx=0x7f31e40a4550 comp rx=0 tx=0).stop 2024-09-10T09:46:54.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:54.432+0000 7f31fafe6740 1 -- 172.21.15.5:0/491794565 shutdown_connections 2024-09-10T09:46:54.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:54.432+0000 7f31fafe6740 1 --2- 172.21.15.5:0/491794565 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e787f89020 0x55e787fa0db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:54.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:54.432+0000 7f31fafe6740 1 --2- 172.21.15.5:0/491794565 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e787f7e760 0x55e787f867f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:54.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:54.432+0000 7f31fafe6740 1 --2- 172.21.15.5:0/491794565 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e787f7de40 0x55e787f847f0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:54.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:54.432+0000 7f31fafe6740 1 -- 172.21.15.5:0/491794565 >> 172.21.15.5:0/491794565 conn(0x55e787f7a3d0 msgr2=0x55e787f87fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:46:54.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:54.432+0000 7f31fafe6740 1 -- 172.21.15.5:0/491794565 shutdown_connections 2024-09-10T09:46:54.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:54.432+0000 7f31fafe6740 1 -- 172.21.15.5:0/491794565 wait complete. 2024-09-10T09:46:54.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:54.433+0000 7f31fafe6740 1 -- start start 2024-09-10T09:46:54.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:54.433+0000 7f31fafe6740 1 Processor -- start 2024-09-10T09:46:54.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:54.434+0000 7f31fafe6740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e787f7de40 0x55e787f91850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:46:54.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:54.434+0000 7f31fafe6740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e787f7e760 0x55e787f8cb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:46:54.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:54.434+0000 7f31fafe6740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e787f89020 0x55e787f8d190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:46:54.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:54.434+0000 7f31fafe6740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f31f01b67a0 con 0x55e787f7e760 2024-09-10T09:46:54.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:54.434+0000 7f31fafe6740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f31f01b60d0 con 0x55e787f7de40 2024-09-10T09:46:54.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:54.434+0000 7f31fafe6740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f31f01b3310 con 0x55e787f89020 2024-09-10T09:46:54.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:54.434+0000 7f31fafe6740 10 client.0 _ll_register_callbacks cb 0x7f31e4005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:46:54.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:54.434+0000 7f31f5a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e787f7de40 0x55e787f91850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:46:54.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:54.434+0000 7f31f5a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e787f7de40 0x55e787f91850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:42002/0 (socket says 172.21.15.5:42002) 2024-09-10T09:46:54.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:54.434+0000 7f31f5a00640 1 -- 172.21.15.5:0/2591925315 learned_addr learned my addr 172.21.15.5:0/2591925315 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:46:54.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:54.434+0000 7f31f5000640 1 --2- 172.21.15.5:0/2591925315 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e787f7e760 0x55e787f8cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:46:54.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:54.434+0000 7f31f6400640 1 --2- 172.21.15.5:0/2591925315 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e787f89020 0x55e787f8d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:46:54.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:54.435+0000 7f31f5a00640 1 -- 172.21.15.5:0/2591925315 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e787f89020 msgr2=0x55e787f8d190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:46:54.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:54.435+0000 7f31f5a00640 1 --2- 172.21.15.5:0/2591925315 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e787f89020 0x55e787f8d190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:54.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:54.435+0000 7f31f5a00640 1 -- 172.21.15.5:0/2591925315 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e787f7e760 msgr2=0x55e787f8cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:46:54.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:54.435+0000 7f31f5a00640 1 --2- 172.21.15.5:0/2591925315 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e787f7e760 0x55e787f8cb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:54.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:54.435+0000 7f31f5a00640 1 -- 172.21.15.5:0/2591925315 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e787f8d9c0 con 0x55e787f7de40 2024-09-10T09:46:54.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:54.435+0000 7f31f5a00640 1 --2- 172.21.15.5:0/2591925315 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e787f7de40 0x55e787f91850 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f31e400c6a0 tx=0x7f31e409b760 comp rx=0 tx=0).ready entity=mon.2 client_cookie=862a0ed6cdc262aa server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:46:54.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:54.435+0000 7f31d7400640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-09-10T09:46:54.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:54.435+0000 7f31d7400640 1 -- 172.21.15.5:0/2591925315 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31e40984c0 con 0x55e787f7de40 2024-09-10T09:46:54.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:54.436+0000 7f31d7400640 1 -- 172.21.15.5:0/2591925315 <== mon.2 v2:172.21.15.102:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f31e40a4b90 con 0x55e787f7de40 2024-09-10T09:46:54.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:54.436+0000 7f31fafe6740 1 -- 172.21.15.5:0/2591925315 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f31e40a9070 con 0x55e787f7de40 2024-09-10T09:46:54.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:54.436+0000 7f31d7400640 1 -- 172.21.15.5:0/2591925315 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f31e4088080 con 0x55e787f7de40 2024-09-10T09:46:54.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:54.436+0000 7f31fafe6740 10 client.27083 fetch_fsmap learned FSMap version 35 2024-09-10T09:46:54.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:54.436+0000 7f31fafe6740 1 -- 172.21.15.5:0/2591925315 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55e788003eb0 con 0x55e787f7de40 2024-09-10T09:46:54.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:54.436+0000 7f31d7400640 1 -- 172.21.15.5:0/2591925315 <== mon.2 v2:172.21.15.102:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f31e40a9070 con 0x55e787f7de40 2024-09-10T09:46:54.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:54.436+0000 7f31d7400640 1 -- 172.21.15.5:0/2591925315 <== mon.2 v2:172.21.15.102:3300/0 5 ==== fsmap(e 35) ==== 8374+0+0 (secure 0 0 0) 0x7f31e40b3020 con 0x55e787f7de40 2024-09-10T09:46:54.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:54.436+0000 7f31fafe6740 10 client.27083 fetch_fsmap finished waiting for FSMap version 35 2024-09-10T09:46:54.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:54.436+0000 7f31fafe6740 10 client.27083 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-10T09:46:54.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:54.436+0000 7f31fafe6740 20 client.27083 populate_metadata read hostname 'smithi005' 2024-09-10T09:46:54.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:54.436+0000 7f31fafe6740 1 --2- 172.21.15.5:0/2591925315 >> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] conn(0x55e787f990a0 0x55e788006140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:46:54.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:54.437+0000 7f31fafe6740 4 client.27083 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T09:46:54.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:54.437+0000 7f31fafe6740 1 -- 172.21.15.5:0/2591925315 --> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e787eebe60 con 0x55e787f990a0 2024-09-10T09:46:54.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:54.437+0000 7f31f5000640 1 --2- 172.21.15.5:0/2591925315 >> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] conn(0x55e787f990a0 0x55e788006140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:46:54.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:54.438+0000 7f31f5000640 1 --2- 172.21.15.5:0/2591925315 >> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] conn(0x55e787f990a0 0x55e788006140 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6b49cfa348982469 server_cookie=c0ef18979eb5f8af in_seq=0 out_seq=0 2024-09-10T09:46:54.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:54.438+0000 7f31d7400640 10 client.27083 ms_handle_connect on v2:172.21.15.159:6836/620627639 2024-09-10T09:46:54.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:54.438+0000 7f31d7400640 1 -- 172.21.15.5:0/2591925315 <== mds.2 v2:172.21.15.159:6836/620627639 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e787eebe60 con 0x55e787f990a0 2024-09-10T09:46:54.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:54.438+0000 7f31d7400640 10 client.27083 handle_command_reply: tid=0 2024-09-10T09:46:54.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:54.438+0000 7f31fafe6740 1 client.27083 shutdown 2024-09-10T09:46:54.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:54.438+0000 7f31fafe6740 1 -- 172.21.15.5:0/2591925315 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e787f7de40 msgr2=0x55e787f91850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:46:54.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:54.438+0000 7f31fafe6740 1 --2- 172.21.15.5:0/2591925315 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e787f7de40 0x55e787f91850 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f31e400c6a0 tx=0x7f31e409b760 comp rx=0 tx=0).stop 2024-09-10T09:46:54.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:54.438+0000 7f31fafe6740 1 -- 172.21.15.5:0/2591925315 shutdown_connections 2024-09-10T09:46:54.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:54.438+0000 7f31fafe6740 1 --2- 172.21.15.5:0/2591925315 >> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] conn(0x55e787f990a0 0x55e788006140 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:54.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:54.438+0000 7f31fafe6740 1 --2- 172.21.15.5:0/2591925315 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e787f89020 0x55e787f8d190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:54.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:54.438+0000 7f31fafe6740 1 --2- 172.21.15.5:0/2591925315 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e787f7e760 0x55e787f8cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:54.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:54.438+0000 7f31fafe6740 1 --2- 172.21.15.5:0/2591925315 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e787f7de40 0x55e787f91850 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:54.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:54.438+0000 7f31fafe6740 1 -- 172.21.15.5:0/2591925315 >> 172.21.15.5:0/2591925315 conn(0x55e787f7a3d0 msgr2=0x55e787f7f8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:46:54.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:54.438+0000 7f31d7400640 0 client.27083 ms_handle_reset on v2:172.21.15.159:6836/620627639 2024-09-10T09:46:54.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:54.439+0000 7f31fafe6740 1 -- 172.21.15.5:0/2591925315 shutdown_connections 2024-09-10T09:46:54.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:54.439+0000 7f31fafe6740 1 -- 172.21.15.5:0/2591925315 wait complete. 2024-09-10T09:46:54.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:54.439+0000 7f31fafe6740 20 client.27083 trim_cache size 0 max 16384 2024-09-10T09:46:54.443 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-10T09:46:54.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:54.439+0000 7f31f6e00640 1 -- 172.21.15.5:0/187676750 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f31d80a2c00 msgr2=0x7f31d80a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:46:54.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:54.439+0000 7f31f6e00640 1 --2- 172.21.15.5:0/187676750 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f31d80a2c00 0x7f31d80a50c0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f31f006a760 tx=0x7f31e8009600 comp rx=0 tx=0).stop 2024-09-10T09:46:54.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:54.439+0000 7f31f6e00640 1 -- 172.21.15.5:0/187676750 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f31f0076720 msgr2=0x7f31f01a4690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:46:54.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:54.439+0000 7f31f6e00640 1 --2- 172.21.15.5:0/187676750 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f31f0076720 0x7f31f01a4690 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f31e400e9f0 tx=0x7f31e400edf0 comp rx=0 tx=0).stop 2024-09-10T09:46:54.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:54.440+0000 7f31f6e00640 1 -- 172.21.15.5:0/187676750 shutdown_connections 2024-09-10T09:46:54.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:54.440+0000 7f31f6e00640 1 --2- 172.21.15.5:0/187676750 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f31d80a2c00 0x7f31d80a50c0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:54.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:54.440+0000 7f31f6e00640 1 --2- 172.21.15.5:0/187676750 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f31f0112870 0x7f31f019e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:54.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:54.440+0000 7f31f6e00640 1 --2- 172.21.15.5:0/187676750 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f31f0077040 0x7f31f01a4bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:54.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:54.440+0000 7f31f6e00640 1 --2- 172.21.15.5:0/187676750 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f31f0076720 0x7f31f01a4690 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:54.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:54.440+0000 7f31f6e00640 1 -- 172.21.15.5:0/187676750 >> 172.21.15.5:0/187676750 conn(0x7f31f00ff8f0 msgr2=0x7f31f0101ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:46:54.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:54.440+0000 7f31f6e00640 1 -- 172.21.15.5:0/187676750 shutdown_connections 2024-09-10T09:46:54.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:54.440+0000 7f31f6e00640 1 -- 172.21.15.5:0/187676750 wait complete. 2024-09-10T09:46:55.076 INFO:tasks.workunit.client.0.smithi005.stdout: 1 500933 6.10 MB/sec execute 276 sec latency 13985.647 ms 2024-09-10T09:46:55.306 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:46:54 smithi005 ceph-mon[30350]: pgmap v650: 129 pgs: 129 active+clean; 197 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 165 KiB/s rd, 2.7 MiB/s wr, 78 op/s 2024-09-10T09:46:55.326 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:46:55.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:46:54 smithi159 ceph-mon[35513]: pgmap v650: 129 pgs: 129 active+clean; 197 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 165 KiB/s rd, 2.7 MiB/s wr, 78 op/s 2024-09-10T09:46:55.360 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-10T09:46:55.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:46:54 smithi102 ceph-mon[35442]: pgmap v650: 129 pgs: 129 active+clean; 197 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 165 KiB/s rd, 2.7 MiB/s wr, 78 op/s 2024-09-10T09:46:55.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:55.867+0000 7f20aa600640 1 -- 172.21.15.5:0/4016559818 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f20a41049a0 msgr2=0x7f20a41089f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:46:55.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:55.867+0000 7f20aa600640 1 --2- 172.21.15.5:0/4016559818 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f20a41049a0 0x7f20a41089f0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f2090009900 tx=0x7f209002a3f0 comp rx=0 tx=0).stop 2024-09-10T09:46:55.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:55.868+0000 7f20aa600640 1 -- 172.21.15.5:0/4016559818 shutdown_connections 2024-09-10T09:46:55.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:55.868+0000 7f20aa600640 1 --2- 172.21.15.5:0/4016559818 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f20a4109120 0x7f20a4110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:55.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:55.868+0000 7f20aa600640 1 --2- 172.21.15.5:0/4016559818 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f20a41049a0 0x7f20a41089f0 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:55.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:55.868+0000 7f20aa600640 1 --2- 172.21.15.5:0/4016559818 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f20a4103ff0 0x7f20a41043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:55.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:55.868+0000 7f20aa600640 1 -- 172.21.15.5:0/4016559818 >> 172.21.15.5:0/4016559818 conn(0x7f20a40ffa90 msgr2=0x7f20a4101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:46:55.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:55.868+0000 7f20aa600640 1 -- 172.21.15.5:0/4016559818 shutdown_connections 2024-09-10T09:46:55.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:55.868+0000 7f20aa600640 1 -- 172.21.15.5:0/4016559818 wait complete. 2024-09-10T09:46:55.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:55.869+0000 7f20aa600640 1 Processor -- start 2024-09-10T09:46:55.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:55.869+0000 7f20aa600640 1 -- start start 2024-09-10T09:46:55.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:55.869+0000 7f20aa600640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f20a4103ff0 0x7f20a419a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:46:55.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:55.869+0000 7f20aa600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f20a41049a0 0x7f20a419ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:46:55.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:55.869+0000 7f20aa600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f20a4109120 0x7f20a419eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:46:55.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:55.869+0000 7f20aa600640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f20a4114190 con 0x7f20a4109120 2024-09-10T09:46:55.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:55.869+0000 7f20aa600640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f20a4114010 con 0x7f20a41049a0 2024-09-10T09:46:55.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:55.869+0000 7f20aa600640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f20a4114310 con 0x7f20a4103ff0 2024-09-10T09:46:55.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:55.870+0000 7f20a3e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f20a4109120 0x7f20a419eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:46:55.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:55.870+0000 7f20a3400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f20a4103ff0 0x7f20a419a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:46:55.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:55.870+0000 7f20a2a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f20a41049a0 0x7f20a419ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:46:55.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:55.870+0000 7f20a3e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f20a4109120 0x7f20a419eed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41954/0 (socket says 172.21.15.5:41954) 2024-09-10T09:46:55.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:55.870+0000 7f20a3400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f20a4103ff0 0x7f20a419a600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:53260/0 (socket says 172.21.15.5:53260) 2024-09-10T09:46:55.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:55.870+0000 7f20a3e00640 1 -- 172.21.15.5:0/2145770809 learned_addr learned my addr 172.21.15.5:0/2145770809 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:46:55.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:55.870+0000 7f20a3400640 1 -- 172.21.15.5:0/2145770809 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f20a41049a0 msgr2=0x7f20a419ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:46:55.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:55.870+0000 7f20a3400640 1 --2- 172.21.15.5:0/2145770809 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f20a41049a0 0x7f20a419ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:55.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:55.870+0000 7f20a3400640 1 -- 172.21.15.5:0/2145770809 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f20a4109120 msgr2=0x7f20a419eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:46:55.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:55.870+0000 7f20a3400640 1 --2- 172.21.15.5:0/2145770809 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f20a4109120 0x7f20a419eed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:55.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:55.870+0000 7f20a3400640 1 -- 172.21.15.5:0/2145770809 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20a419f5b0 con 0x7f20a4103ff0 2024-09-10T09:46:55.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:55.870+0000 7f20a3e00640 1 --2- 172.21.15.5:0/2145770809 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f20a4109120 0x7f20a419eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:46:55.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:55.871+0000 7f20a3400640 1 --2- 172.21.15.5:0/2145770809 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f20a4103ff0 0x7f20a419a600 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f209400e9f0 tx=0x7f209400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:46:55.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:55.871+0000 7f209be00640 1 -- 172.21.15.5:0/2145770809 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f209400cd80 con 0x7f20a4103ff0 2024-09-10T09:46:55.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:55.871+0000 7f209be00640 1 -- 172.21.15.5:0/2145770809 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f20940035f0 con 0x7f20a4103ff0 2024-09-10T09:46:55.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:55.871+0000 7f209be00640 1 -- 172.21.15.5:0/2145770809 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2094010690 con 0x7f20a4103ff0 2024-09-10T09:46:55.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:55.871+0000 7f20aa600640 1 -- 172.21.15.5:0/2145770809 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20a419f8a0 con 0x7f20a4103ff0 2024-09-10T09:46:55.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:55.871+0000 7f20aa600640 1 -- 172.21.15.5:0/2145770809 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20a41a7180 con 0x7f20a4103ff0 2024-09-10T09:46:55.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:55.874+0000 7f20ad260740 1 Processor -- start 2024-09-10T09:46:55.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:55.875+0000 7f209be00640 1 -- 172.21.15.5:0/2145770809 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f2094003b30 con 0x7f20a4103ff0 2024-09-10T09:46:55.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:55.875+0000 7f20ad260740 1 -- start start 2024-09-10T09:46:55.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:55.875+0000 7f20ad260740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5636dcdce310 0x5636dcdce6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:46:55.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:55.875+0000 7f20ad260740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5636dcdcec30 0x5636dcdd8fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:46:55.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:55.875+0000 7f20ad260740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5636dcdd94f0 0x5636dcddb8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:46:55.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:55.875+0000 7f20ad260740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5636dcd34650 con 0x5636dcdce310 2024-09-10T09:46:55.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:55.875+0000 7f20ad260740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x5636dccdbf10 con 0x5636dcdcec30 2024-09-10T09:46:55.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:55.875+0000 7f20ad260740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5636dcd30610 con 0x5636dcdd94f0 2024-09-10T09:46:55.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:55.875+0000 7f20a3e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5636dcdce310 0x5636dcdce6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:46:55.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:55.875+0000 7f20a3e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5636dcdce310 0x5636dcdce6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41962/0 (socket says 172.21.15.5:41962) 2024-09-10T09:46:55.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:55.875+0000 7f20a3e00640 1 -- 172.21.15.5:0/2854755851 learned_addr learned my addr 172.21.15.5:0/2854755851 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:46:55.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:55.875+0000 7f20a3400640 1 --2- 172.21.15.5:0/2854755851 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5636dcdcec30 0x5636dcdd8fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:46:55.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:55.876+0000 7f20a2a00640 1 --2- 172.21.15.5:0/2854755851 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5636dcdd94f0 0x5636dcddb8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:46:55.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:55.876+0000 7f20a3e00640 1 -- 172.21.15.5:0/2854755851 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5636dcdd94f0 msgr2=0x5636dcddb8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:46:55.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:55.876+0000 7f20a3e00640 1 --2- 172.21.15.5:0/2854755851 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5636dcdd94f0 0x5636dcddb8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:55.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:55.876+0000 7f20a3e00640 1 -- 172.21.15.5:0/2854755851 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5636dcdcec30 msgr2=0x5636dcdd8fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:46:55.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:55.876+0000 7f20a3e00640 1 --2- 172.21.15.5:0/2854755851 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5636dcdcec30 0x5636dcdd8fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:55.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:55.876+0000 7f20a3e00640 1 -- 172.21.15.5:0/2854755851 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5636dca43580 con 0x5636dcdce310 2024-09-10T09:46:55.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:55.876+0000 7f20a3e00640 1 --2- 172.21.15.5:0/2854755851 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5636dcdce310 0x5636dcdce6f0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f209c011850 tx=0x7f209c011d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4050151411f09c59 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:46:55.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:55.877+0000 7f209be00640 1 --2- 172.21.15.5:0/2145770809 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f20880a2c00 0x7f20880a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:46:55.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:55.877+0000 7f2098c00640 1 -- 172.21.15.5:0/2854755851 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f209c011330 con 0x5636dcdce310 2024-09-10T09:46:55.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:55.877+0000 7f2098c00640 1 -- 172.21.15.5:0/2854755851 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f209c00fe80 con 0x5636dcdce310 2024-09-10T09:46:55.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:55.877+0000 7f2098c00640 1 -- 172.21.15.5:0/2854755851 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f209c0193e0 con 0x5636dcdce310 2024-09-10T09:46:55.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:55.877+0000 7f209be00640 1 -- 172.21.15.5:0/2145770809 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f20940cf720 con 0x7f20a4103ff0 2024-09-10T09:46:55.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:55.877+0000 7f20a3400640 1 --2- 172.21.15.5:0/2145770809 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f20880a2c00 0x7f20880a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:46:55.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:55.878+0000 7f20ad260740 1 -- 172.21.15.5:0/2854755851 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5636dcdce310 msgr2=0x5636dcdce6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:46:55.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:55.878+0000 7f20ad260740 1 --2- 172.21.15.5:0/2854755851 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5636dcdce310 0x5636dcdce6f0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f209c011850 tx=0x7f209c011d10 comp rx=0 tx=0).stop 2024-09-10T09:46:55.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:55.878+0000 7f20ad260740 1 -- 172.21.15.5:0/2854755851 shutdown_connections 2024-09-10T09:46:55.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:55.878+0000 7f20ad260740 1 --2- 172.21.15.5:0/2854755851 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5636dcdd94f0 0x5636dcddb8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:55.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:55.878+0000 7f20ad260740 1 --2- 172.21.15.5:0/2854755851 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5636dcdcec30 0x5636dcdd8fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:55.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:55.878+0000 7f20ad260740 1 --2- 172.21.15.5:0/2854755851 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5636dcdce310 0x5636dcdce6f0 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:55.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:55.878+0000 7f20ad260740 1 -- 172.21.15.5:0/2854755851 >> 172.21.15.5:0/2854755851 conn(0x5636dcdca8a0 msgr2=0x5636dcdcac80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:46:55.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:55.878+0000 7f20a3400640 1 --2- 172.21.15.5:0/2145770809 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f20880a2c00 0x7f20880a50c0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f20940ccdd0 tx=0x7f20940d5530 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:46:55.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:55.878+0000 7f20ad260740 1 -- 172.21.15.5:0/2854755851 shutdown_connections 2024-09-10T09:46:55.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:55.878+0000 7f20ad260740 1 -- 172.21.15.5:0/2854755851 wait complete. 2024-09-10T09:46:55.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:55.879+0000 7f20ad260740 1 -- start start 2024-09-10T09:46:55.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:55.879+0000 7f20ad260740 1 Processor -- start 2024-09-10T09:46:55.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:55.879+0000 7f20ad260740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5636dcdce310 0x5636dcdd6320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:46:55.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:55.879+0000 7f20ad260740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5636dcdcec30 0x5636dcdd6860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:46:55.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:55.879+0000 7f20ad260740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5636dcdd94f0 0x5636dcdd6da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:46:55.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:55.879+0000 7f20ad260740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5636dcddc0c0 con 0x5636dcdcec30 2024-09-10T09:46:55.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:55.879+0000 7f20ad260740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x5636dccef350 con 0x5636dcdce310 2024-09-10T09:46:55.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:55.879+0000 7f20ad260740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5636dcddc240 con 0x5636dcdd94f0 2024-09-10T09:46:55.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:55.879+0000 7f20ad260740 10 client.0 _ll_register_callbacks cb 0x5636dcd12290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:46:55.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:55.880+0000 7f20a2a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5636dcdcec30 0x5636dcdd6860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:46:55.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:55.880+0000 7f20a3e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5636dcdce310 0x5636dcdd6320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:46:55.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:55.880+0000 7f20a3e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5636dcdce310 0x5636dcdd6320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:42036/0 (socket says 172.21.15.5:42036) 2024-09-10T09:46:55.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:55.880+0000 7f20a3e00640 1 -- 172.21.15.5:0/2682059845 learned_addr learned my addr 172.21.15.5:0/2682059845 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:46:55.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:55.880+0000 7f20a3e00640 1 --2- 172.21.15.5:0/2682059845 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5636dcdd94f0 0x5636dcdd6da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:46:55.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:55.880+0000 7f20a3e00640 1 -- 172.21.15.5:0/2682059845 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5636dcdd94f0 msgr2=0x5636dcdd6da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:46:55.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:55.880+0000 7f20a3e00640 1 --2- 172.21.15.5:0/2682059845 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5636dcdd94f0 0x5636dcdd6da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:55.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:55.880+0000 7f20a3e00640 1 -- 172.21.15.5:0/2682059845 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5636dcdcec30 msgr2=0x5636dcdd6860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:46:55.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:55.880+0000 7f20a3e00640 1 --2- 172.21.15.5:0/2682059845 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5636dcdcec30 0x5636dcdd6860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:55.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:55.880+0000 7f20a3e00640 1 -- 172.21.15.5:0/2682059845 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5636dcb0d960 con 0x5636dcdce310 2024-09-10T09:46:55.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:55.881+0000 7f20a3e00640 1 --2- 172.21.15.5:0/2682059845 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5636dcdce310 0x5636dcdd6320 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f209c00fe40 tx=0x7f209c010b30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5a9f978edcb15fe4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:46:55.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:55.881+0000 7f208d600640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-09-10T09:46:55.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:55.881+0000 7f208d600640 1 -- 172.21.15.5:0/2682059845 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f209c010640 con 0x5636dcdce310 2024-09-10T09:46:55.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:55.881+0000 7f20ad260740 1 -- 172.21.15.5:0/2682059845 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5636dcd41bc0 con 0x5636dcdce310 2024-09-10T09:46:55.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:55.881+0000 7f208d600640 1 -- 172.21.15.5:0/2682059845 <== mon.2 v2:172.21.15.102:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f209c01f070 con 0x5636dcdce310 2024-09-10T09:46:55.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:55.882+0000 7f208d600640 1 -- 172.21.15.5:0/2682059845 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f209c0107e0 con 0x5636dcdce310 2024-09-10T09:46:55.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:55.882+0000 7f20ad260740 10 client.27095 fetch_fsmap learned FSMap version 35 2024-09-10T09:46:55.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:55.882+0000 7f20ad260740 1 -- 172.21.15.5:0/2682059845 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5636dcdf0420 con 0x5636dcdce310 2024-09-10T09:46:55.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:55.882+0000 7f208d600640 1 -- 172.21.15.5:0/2682059845 <== mon.2 v2:172.21.15.102:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x5636dcd41bc0 con 0x5636dcdce310 2024-09-10T09:46:55.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:55.882+0000 7f208d600640 1 -- 172.21.15.5:0/2682059845 <== mon.2 v2:172.21.15.102:3300/0 5 ==== fsmap(e 35) ==== 8374+0+0 (secure 0 0 0) 0x7f209c018070 con 0x5636dcdce310 2024-09-10T09:46:55.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:55.882+0000 7f20ad260740 10 client.27095 fetch_fsmap finished waiting for FSMap version 35 2024-09-10T09:46:55.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:55.882+0000 7f20ad260740 10 client.27095 resolve_mds: validated gid 24542 aka daemon mds.b 2024-09-10T09:46:55.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:55.882+0000 7f20ad260740 20 client.27095 populate_metadata read hostname 'smithi005' 2024-09-10T09:46:55.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:55.882+0000 7f20ad260740 1 --2- 172.21.15.5:0/2682059845 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x5636dcdf1830 0x5636dcdf3c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:46:55.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:55.882+0000 7f20ad260740 4 client.27095 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:46:55.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:55.882+0000 7f20ad260740 1 -- 172.21.15.5:0/2682059845 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5636dcd3c0d0 con 0x5636dcdf1830 2024-09-10T09:46:55.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:55.883+0000 7f20a2a00640 1 --2- 172.21.15.5:0/2682059845 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x5636dcdf1830 0x5636dcdf3c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:46:55.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:55.883+0000 7f20a2a00640 1 --2- 172.21.15.5:0/2682059845 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x5636dcdf1830 0x5636dcdf3c20 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=557f1993c0d57f14 server_cookie=b6c29e5ceb4e268a in_seq=0 out_seq=0 2024-09-10T09:46:55.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:55.883+0000 7f208d600640 10 client.27095 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T09:46:55.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:55.884+0000 7f208d600640 1 -- 172.21.15.5:0/2682059845 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5636dcd3c0d0 con 0x5636dcdf1830 2024-09-10T09:46:55.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:55.884+0000 7f208d600640 10 client.27095 handle_command_reply: tid=0 2024-09-10T09:46:55.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:55.884+0000 7f20ad260740 1 client.27095 shutdown 2024-09-10T09:46:55.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:55.884+0000 7f20ad260740 1 -- 172.21.15.5:0/2682059845 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5636dcdce310 msgr2=0x5636dcdd6320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:46:55.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:55.884+0000 7f20ad260740 1 --2- 172.21.15.5:0/2682059845 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5636dcdce310 0x5636dcdd6320 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f209c00fe40 tx=0x7f209c010b30 comp rx=0 tx=0).stop 2024-09-10T09:46:55.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:55.884+0000 7f20ad260740 1 -- 172.21.15.5:0/2682059845 shutdown_connections 2024-09-10T09:46:55.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:55.884+0000 7f20ad260740 1 --2- 172.21.15.5:0/2682059845 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x5636dcdf1830 0x5636dcdf3c20 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:55.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:55.884+0000 7f20ad260740 1 --2- 172.21.15.5:0/2682059845 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5636dcdd94f0 0x5636dcdd6da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:55.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:55.884+0000 7f20ad260740 1 --2- 172.21.15.5:0/2682059845 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5636dcdcec30 0x5636dcdd6860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:55.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:55.884+0000 7f20ad260740 1 --2- 172.21.15.5:0/2682059845 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5636dcdce310 0x5636dcdd6320 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:55.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:55.884+0000 7f20ad260740 1 -- 172.21.15.5:0/2682059845 >> 172.21.15.5:0/2682059845 conn(0x5636dcdca8a0 msgr2=0x5636dcdc9aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:46:55.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:55.884+0000 7f208d600640 0 client.27095 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T09:46:55.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:55.885+0000 7f20ad260740 1 -- 172.21.15.5:0/2682059845 shutdown_connections 2024-09-10T09:46:55.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:55.885+0000 7f20ad260740 1 -- 172.21.15.5:0/2682059845 wait complete. 2024-09-10T09:46:55.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:55.885+0000 7f20ad260740 20 client.27095 trim_cache size 0 max 16384 2024-09-10T09:46:55.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:55.904+0000 7f20ad260740 1 Processor -- start 2024-09-10T09:46:55.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:55.904+0000 7f20ad260740 1 -- start start 2024-09-10T09:46:55.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:55.904+0000 7f20ad260740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5636dcdca8a0 0x5636dcdd6bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:46:55.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:55.904+0000 7f20ad260740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5636dcdce310 0x5636dcdf9390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:46:55.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:55.904+0000 7f20ad260740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5636dcdcec30 0x5636dcdf98d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:46:55.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:55.904+0000 7f20ad260740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f20a41accd0 con 0x5636dcdca8a0 2024-09-10T09:46:55.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:55.904+0000 7f20ad260740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f20a41ae840 con 0x5636dcdce310 2024-09-10T09:46:55.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:55.904+0000 7f20ad260740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f20a41ad3a0 con 0x5636dcdcec30 2024-09-10T09:46:55.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:55.905+0000 7f20a3e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5636dcdca8a0 0x5636dcdd6bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:46:55.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:55.905+0000 7f20a3e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5636dcdca8a0 0x5636dcdd6bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41974/0 (socket says 172.21.15.5:41974) 2024-09-10T09:46:55.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:55.905+0000 7f20a3e00640 1 -- 172.21.15.5:0/34203696 learned_addr learned my addr 172.21.15.5:0/34203696 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:46:55.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:55.905+0000 7f20a2a00640 1 --2- 172.21.15.5:0/34203696 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5636dcdce310 0x5636dcdf9390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:46:55.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:55.905+0000 7f20a3e00640 1 --2- 172.21.15.5:0/34203696 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5636dcdcec30 0x5636dcdf98d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:46:55.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:55.905+0000 7f20a3e00640 1 -- 172.21.15.5:0/34203696 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5636dcdcec30 msgr2=0x5636dcdf98d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:46:55.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:55.905+0000 7f20a3e00640 1 --2- 172.21.15.5:0/34203696 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5636dcdcec30 0x5636dcdf98d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:55.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:55.905+0000 7f20a3e00640 1 -- 172.21.15.5:0/34203696 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5636dcdce310 msgr2=0x5636dcdf9390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:46:55.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:55.905+0000 7f20a3e00640 1 --2- 172.21.15.5:0/34203696 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5636dcdce310 0x5636dcdf9390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:55.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:55.905+0000 7f20a3e00640 1 -- 172.21.15.5:0/34203696 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5636dcdcb530 con 0x5636dcdca8a0 2024-09-10T09:46:55.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:55.905+0000 7f20a2a00640 1 --2- 172.21.15.5:0/34203696 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5636dcdce310 0x5636dcdf9390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:46:55.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:55.906+0000 7f20a3e00640 1 --2- 172.21.15.5:0/34203696 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5636dcdca8a0 0x5636dcdd6bf0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f209c005a80 tx=0x7f209c014040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56cab035cec3f13c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:46:55.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:55.906+0000 7f208fe00640 1 -- 172.21.15.5:0/34203696 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f209c010640 con 0x5636dcdca8a0 2024-09-10T09:46:55.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:55.906+0000 7f208fe00640 1 -- 172.21.15.5:0/34203696 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f209c00f790 con 0x5636dcdca8a0 2024-09-10T09:46:55.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:55.906+0000 7f208fe00640 1 -- 172.21.15.5:0/34203696 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f209c01a440 con 0x5636dcdca8a0 2024-09-10T09:46:55.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:55.907+0000 7f20ad260740 1 -- 172.21.15.5:0/34203696 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5636dcdca8a0 msgr2=0x5636dcdd6bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:46:55.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:55.907+0000 7f20ad260740 1 --2- 172.21.15.5:0/34203696 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5636dcdca8a0 0x5636dcdd6bf0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f209c005a80 tx=0x7f209c014040 comp rx=0 tx=0).stop 2024-09-10T09:46:55.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:55.907+0000 7f20ad260740 1 -- 172.21.15.5:0/34203696 shutdown_connections 2024-09-10T09:46:55.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:55.907+0000 7f20ad260740 1 --2- 172.21.15.5:0/34203696 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5636dcdcec30 0x5636dcdf98d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:55.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:55.907+0000 7f20ad260740 1 --2- 172.21.15.5:0/34203696 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5636dcdce310 0x5636dcdf9390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:55.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:55.907+0000 7f20ad260740 1 --2- 172.21.15.5:0/34203696 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5636dcdca8a0 0x5636dcdd6bf0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:55.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:55.907+0000 7f20ad260740 1 -- 172.21.15.5:0/34203696 >> 172.21.15.5:0/34203696 conn(0x5636dcdd8380 msgr2=0x5636dcdc9280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:46:55.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:55.907+0000 7f20ad260740 1 -- 172.21.15.5:0/34203696 shutdown_connections 2024-09-10T09:46:55.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:55.907+0000 7f20ad260740 1 -- 172.21.15.5:0/34203696 wait complete. 2024-09-10T09:46:55.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:55.907+0000 7f20ad260740 1 -- start start 2024-09-10T09:46:55.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:55.908+0000 7f20ad260740 1 Processor -- start 2024-09-10T09:46:55.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:55.908+0000 7f20ad260740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5636dcdca8a0 0x5636dcde9d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:46:55.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:55.908+0000 7f20ad260740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5636dcdce310 0x5636dcdea270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:46:55.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:55.908+0000 7f20ad260740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5636dcdcec30 0x5636dcde4ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:46:55.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:55.908+0000 7f20ad260740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f20a41ae140 con 0x5636dcdce310 2024-09-10T09:46:55.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:55.908+0000 7f20ad260740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f20a41ada70 con 0x5636dcdca8a0 2024-09-10T09:46:55.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:55.908+0000 7f20ad260740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f20a41aacb0 con 0x5636dcdcec30 2024-09-10T09:46:55.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:55.908+0000 7f20ad260740 10 client.0 _ll_register_callbacks cb 0x7f209c0111c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:46:55.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:55.908+0000 7f20a2a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5636dcdce310 0x5636dcdea270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:46:55.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:55.908+0000 7f20a2a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5636dcdce310 0x5636dcdea270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41990/0 (socket says 172.21.15.5:41990) 2024-09-10T09:46:55.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:55.908+0000 7f20a2a00640 1 -- 172.21.15.5:0/1787351772 learned_addr learned my addr 172.21.15.5:0/1787351772 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:46:55.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:55.909+0000 7f20a3e00640 1 --2- 172.21.15.5:0/1787351772 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5636dcdca8a0 0x5636dcde9d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:46:55.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:55.909+0000 7f20a3e00640 1 --2- 172.21.15.5:0/1787351772 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5636dcdcec30 0x5636dcde4ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:46:55.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:55.909+0000 7f20a2a00640 1 -- 172.21.15.5:0/1787351772 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5636dcdcec30 msgr2=0x5636dcde4ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:46:55.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:55.909+0000 7f20a2a00640 1 --2- 172.21.15.5:0/1787351772 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5636dcdcec30 0x5636dcde4ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:55.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:55.909+0000 7f20a2a00640 1 -- 172.21.15.5:0/1787351772 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5636dcdca8a0 msgr2=0x5636dcde9d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:46:55.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:55.909+0000 7f20a2a00640 1 --2- 172.21.15.5:0/1787351772 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5636dcdca8a0 0x5636dcde9d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:55.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:55.909+0000 7f20a2a00640 1 -- 172.21.15.5:0/1787351772 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5636dcde52a0 con 0x5636dcdce310 2024-09-10T09:46:55.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:55.909+0000 7f20a3e00640 1 --2- 172.21.15.5:0/1787351772 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5636dcdcec30 0x5636dcde4ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T09:46:55.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:55.909+0000 7f20a2a00640 1 --2- 172.21.15.5:0/1787351772 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5636dcdce310 0x5636dcdea270 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f2090000c00 tx=0x7f20900045a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d6c104c4d95420a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:46:55.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:55.909+0000 7f208d600640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-10T09:46:55.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:55.910+0000 7f208d600640 1 -- 172.21.15.5:0/1787351772 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2090049070 con 0x5636dcdce310 2024-09-10T09:46:55.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:55.910+0000 7f208d600640 1 -- 172.21.15.5:0/1787351772 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f2090005590 con 0x5636dcdce310 2024-09-10T09:46:55.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:55.910+0000 7f20ad260740 1 -- 172.21.15.5:0/1787351772 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f209c00f790 con 0x5636dcdce310 2024-09-10T09:46:55.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:55.910+0000 7f208d600640 1 -- 172.21.15.5:0/1787351772 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f2090055050 con 0x5636dcdce310 2024-09-10T09:46:55.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:55.910+0000 7f20ad260740 10 client.17103 fetch_fsmap learned FSMap version 35 2024-09-10T09:46:55.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:55.910+0000 7f20ad260740 1 -- 172.21.15.5:0/1787351772 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5636dce5cc90 con 0x5636dcdce310 2024-09-10T09:46:55.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:55.911+0000 7f208d600640 1 -- 172.21.15.5:0/1787351772 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f209c00f790 con 0x5636dcdce310 2024-09-10T09:46:55.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:55.911+0000 7f208d600640 1 -- 172.21.15.5:0/1787351772 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 35) ==== 8374+0+0 (secure 0 0 0) 0x7f2090031ad0 con 0x5636dcdce310 2024-09-10T09:46:55.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:55.911+0000 7f20ad260740 10 client.17103 fetch_fsmap finished waiting for FSMap version 35 2024-09-10T09:46:55.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:55.911+0000 7f20ad260740 10 client.17103 resolve_mds: validated gid 24542 aka daemon mds.b 2024-09-10T09:46:55.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:55.911+0000 7f20ad260740 20 client.17103 populate_metadata read hostname 'smithi005' 2024-09-10T09:46:55.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:55.911+0000 7f20ad260740 1 --2- 172.21.15.5:0/1787351772 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x5636dcdd94f0 0x5636dce5f430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:46:55.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:55.911+0000 7f20ad260740 4 client.17103 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "fcdeb12e", "release": true, "await": true}] 2024-09-10T09:46:55.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:55.911+0000 7f20ad260740 1 -- 172.21.15.5:0/1787351772 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "quiesce db", "set_id": "fcdeb12e", "release": true, "await": true}) -- 0x5636dccb8430 con 0x5636dcdd94f0 2024-09-10T09:46:55.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:55.911+0000 7f20a3e00640 1 --2- 172.21.15.5:0/1787351772 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x5636dcdd94f0 0x5636dce5f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:46:55.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:55.912+0000 7f20a3e00640 1 --2- 172.21.15.5:0/1787351772 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x5636dcdd94f0 0x5636dce5f430 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ea504b54d1c5c668 server_cookie=ced0a4060c41e05d in_seq=0 out_seq=0 2024-09-10T09:46:55.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:55.912+0000 7f208d600640 10 client.17103 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T09:46:55.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:55.925+0000 7f208d600640 1 -- 172.21.15.5:0/1787351772 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5636dccb8430 con 0x5636dcdd94f0 2024-09-10T09:46:55.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:55.925+0000 7f208d600640 10 client.17103 handle_command_reply: tid=0 2024-09-10T09:46:55.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:55.925+0000 7f20ad260740 1 client.17103 shutdown 2024-09-10T09:46:55.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:55.925+0000 7f20ad260740 1 -- 172.21.15.5:0/1787351772 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5636dcdce310 msgr2=0x5636dcdea270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:46:55.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:55.925+0000 7f20ad260740 1 --2- 172.21.15.5:0/1787351772 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5636dcdce310 0x5636dcdea270 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f2090000c00 tx=0x7f20900045a0 comp rx=0 tx=0).stop 2024-09-10T09:46:55.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:55.925+0000 7f20ad260740 1 -- 172.21.15.5:0/1787351772 shutdown_connections 2024-09-10T09:46:55.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:55.925+0000 7f20ad260740 1 --2- 172.21.15.5:0/1787351772 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x5636dcdd94f0 0x5636dce5f430 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:55.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:55.925+0000 7f20ad260740 1 --2- 172.21.15.5:0/1787351772 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5636dcdcec30 0x5636dcde4ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:55.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:55.925+0000 7f20ad260740 1 --2- 172.21.15.5:0/1787351772 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5636dcdce310 0x5636dcdea270 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:55.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:55.925+0000 7f20ad260740 1 --2- 172.21.15.5:0/1787351772 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5636dcdca8a0 0x5636dcde9d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:55.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:55.925+0000 7f208d600640 0 client.17103 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T09:46:55.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:55.925+0000 7f20ad260740 1 -- 172.21.15.5:0/1787351772 >> 172.21.15.5:0/1787351772 conn(0x5636dcdd8380 msgr2=0x5636dcdecba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:46:55.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:55.926+0000 7f20ad260740 1 -- 172.21.15.5:0/1787351772 shutdown_connections 2024-09-10T09:46:55.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:55.926+0000 7f20ad260740 1 -- 172.21.15.5:0/1787351772 wait complete. 2024-09-10T09:46:55.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:55.926+0000 7f20ad260740 20 client.17103 trim_cache size 0 max 16384 2024-09-10T09:46:55.928 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-10T09:46:55.928 INFO:teuthology.orchestra.run.smithi005.stdout: "epoch": 35, 2024-09-10T09:46:55.928 INFO:teuthology.orchestra.run.smithi005.stdout: "leader": 24542, 2024-09-10T09:46:55.928 INFO:teuthology.orchestra.run.smithi005.stdout: "set_version": 28, 2024-09-10T09:46:55.928 INFO:teuthology.orchestra.run.smithi005.stdout: "sets": { 2024-09-10T09:46:55.928 INFO:teuthology.orchestra.run.smithi005.stdout: "fcdeb12e": { 2024-09-10T09:46:55.928 INFO:teuthology.orchestra.run.smithi005.stdout: "version": 28, 2024-09-10T09:46:55.928 INFO:teuthology.orchestra.run.smithi005.stdout: "age_ref": 0.0, 2024-09-10T09:46:55.929 INFO:teuthology.orchestra.run.smithi005.stdout: "state": { 2024-09-10T09:46:55.929 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "RELEASED", 2024-09-10T09:46:55.929 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0 2024-09-10T09:46:55.929 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T09:46:55.929 INFO:teuthology.orchestra.run.smithi005.stdout: "timeout": 90.0, 2024-09-10T09:46:55.929 INFO:teuthology.orchestra.run.smithi005.stdout: "expiration": 133.4, 2024-09-10T09:46:55.929 INFO:teuthology.orchestra.run.smithi005.stdout: "members": { 2024-09-10T09:46:55.929 INFO:teuthology.orchestra.run.smithi005.stdout: "file:/": { 2024-09-10T09:46:55.929 INFO:teuthology.orchestra.run.smithi005.stdout: "excluded": false, 2024-09-10T09:46:55.929 INFO:teuthology.orchestra.run.smithi005.stdout: "state": { 2024-09-10T09:46:55.929 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "RELEASED", 2024-09-10T09:46:55.929 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0 2024-09-10T09:46:55.929 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T09:46:55.929 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T09:46:55.930 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T09:46:55.930 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T09:46:55.930 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T09:46:55.930 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-10T09:46:55.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:55.927+0000 7f20aa600640 1 -- 172.21.15.5:0/2145770809 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f20880a2c00 msgr2=0x7f20880a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:46:55.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:55.927+0000 7f20aa600640 1 --2- 172.21.15.5:0/2145770809 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f20880a2c00 0x7f20880a50c0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f20940ccdd0 tx=0x7f20940d5530 comp rx=0 tx=0).stop 2024-09-10T09:46:55.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:55.927+0000 7f20aa600640 1 -- 172.21.15.5:0/2145770809 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f20a4103ff0 msgr2=0x7f20a419a600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:46:55.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:55.927+0000 7f20aa600640 1 --2- 172.21.15.5:0/2145770809 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f20a4103ff0 0x7f20a419a600 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f209400e9f0 tx=0x7f209400eeb0 comp rx=0 tx=0).stop 2024-09-10T09:46:55.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:55.927+0000 7f20aa600640 1 -- 172.21.15.5:0/2145770809 shutdown_connections 2024-09-10T09:46:55.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:55.927+0000 7f20aa600640 1 --2- 172.21.15.5:0/2145770809 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f20880a2c00 0x7f20880a50c0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:55.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:55.927+0000 7f20aa600640 1 --2- 172.21.15.5:0/2145770809 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f20a4109120 0x7f20a419eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:55.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:55.927+0000 7f20aa600640 1 --2- 172.21.15.5:0/2145770809 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f20a41049a0 0x7f20a419ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:55.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:55.927+0000 7f20aa600640 1 --2- 172.21.15.5:0/2145770809 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f20a4103ff0 0x7f20a419a600 secure :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f209400e9f0 tx=0x7f209400eeb0 comp rx=0 tx=0).stop 2024-09-10T09:46:55.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:55.927+0000 7f20aa600640 1 -- 172.21.15.5:0/2145770809 >> 172.21.15.5:0/2145770809 conn(0x7f20a40ffa90 msgr2=0x7f20a4100d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:46:55.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:55.927+0000 7f20aa600640 1 -- 172.21.15.5:0/2145770809 shutdown_connections 2024-09-10T09:46:55.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:55.927+0000 7f20aa600640 1 -- 172.21.15.5:0/2145770809 wait complete. 2024-09-10T09:46:55.937 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:46:56.076 INFO:tasks.workunit.client.0.smithi005.stdout: 1 501154 6.08 MB/sec execute 277 sec latency 14836.688 ms 2024-09-10T09:46:56.670 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'fcdeb12e', total seconds elapsed: 19.5 2024-09-10T09:46:57.076 INFO:tasks.workunit.client.0.smithi005.stdout: 1 502833 6.08 MB/sec execute 278 sec latency 16.367 ms 2024-09-10T09:46:57.170 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:46:56 smithi005 ceph-mon[30350]: pgmap v651: 129 pgs: 129 active+clean; 198 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 191 KiB/s rd, 2.5 MiB/s wr, 83 op/s 2024-09-10T09:46:57.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:57.190+0000 7f3d36c00640 1 -- 172.21.15.5:0/4026477850 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3d30106f70 msgr2=0x7f3d3010eab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:46:57.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:57.190+0000 7f3d36c00640 1 --2- 172.21.15.5:0/4026477850 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3d30106f70 0x7f3d3010eab0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f3d28009ca0 tx=0x7f3d2802a8c0 comp rx=0 tx=0).stop 2024-09-10T09:46:57.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:57.190+0000 7f3d36c00640 1 -- 172.21.15.5:0/4026477850 shutdown_connections 2024-09-10T09:46:57.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:57.190+0000 7f3d36c00640 1 --2- 172.21.15.5:0/4026477850 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3d30106f70 0x7f3d3010eab0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:57.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:57.190+0000 7f3d36c00640 1 --2- 172.21.15.5:0/4026477850 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3d30100eb0 0x7f3d30106840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:57.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:57.190+0000 7f3d36c00640 1 --2- 172.21.15.5:0/4026477850 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3d30100500 0x7f3d301008e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:57.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:57.190+0000 7f3d36c00640 1 -- 172.21.15.5:0/4026477850 >> 172.21.15.5:0/4026477850 conn(0x7f3d300fc5c0 msgr2=0x7f3d300fe9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:46:57.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:57.191+0000 7f3d36c00640 1 -- 172.21.15.5:0/4026477850 shutdown_connections 2024-09-10T09:46:57.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:57.191+0000 7f3d36c00640 1 -- 172.21.15.5:0/4026477850 wait complete. 2024-09-10T09:46:57.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:57.192+0000 7f3d36c00640 1 Processor -- start 2024-09-10T09:46:57.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:57.192+0000 7f3d36c00640 1 -- start start 2024-09-10T09:46:57.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:57.192+0000 7f3d36c00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3d30100500 0x7f3d30198480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:46:57.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:57.192+0000 7f3d36c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3d30100eb0 0x7f3d301989c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:46:57.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:57.192+0000 7f3d36c00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3d30106f70 0x7f3d3019cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:46:57.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:57.192+0000 7f3d36c00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f3d30111fe0 con 0x7f3d30106f70 2024-09-10T09:46:57.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:57.192+0000 7f3d36c00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f3d30111e60 con 0x7f3d30100eb0 2024-09-10T09:46:57.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:57.192+0000 7f3d36c00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f3d30112160 con 0x7f3d30100500 2024-09-10T09:46:57.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:57.192+0000 7f3d2fe00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3d30100500 0x7f3d30198480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:46:57.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:57.192+0000 7f3d2fe00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3d30100500 0x7f3d30198480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:53328/0 (socket says 172.21.15.5:53328) 2024-09-10T09:46:57.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:57.193+0000 7f3d2fe00640 1 -- 172.21.15.5:0/1774540555 learned_addr learned my addr 172.21.15.5:0/1774540555 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:46:57.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:57.193+0000 7f3d34c00640 1 --2- 172.21.15.5:0/1774540555 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3d30106f70 0x7f3d3019cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:46:57.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:57.193+0000 7f3d2f400640 1 --2- 172.21.15.5:0/1774540555 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3d30100eb0 0x7f3d301989c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:46:57.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:57.193+0000 7f3d2fe00640 1 -- 172.21.15.5:0/1774540555 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3d30100eb0 msgr2=0x7f3d301989c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:46:57.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:57.193+0000 7f3d2fe00640 1 --2- 172.21.15.5:0/1774540555 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3d30100eb0 0x7f3d301989c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:57.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:57.193+0000 7f3d2fe00640 1 -- 172.21.15.5:0/1774540555 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3d30106f70 msgr2=0x7f3d3019cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:46:57.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:57.193+0000 7f3d2fe00640 1 --2- 172.21.15.5:0/1774540555 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3d30106f70 0x7f3d3019cd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:57.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:57.193+0000 7f3d2fe00640 1 -- 172.21.15.5:0/1774540555 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d3019d430 con 0x7f3d30100500 2024-09-10T09:46:57.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:57.193+0000 7f3d34c00640 1 --2- 172.21.15.5:0/1774540555 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3d30106f70 0x7f3d3019cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:46:57.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:57.193+0000 7f3d2fe00640 1 --2- 172.21.15.5:0/1774540555 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3d30100500 0x7f3d30198480 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f3d2000e9c0 tx=0x7f3d2000ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:46:57.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:57.194+0000 7f3d2cc00640 1 -- 172.21.15.5:0/1774540555 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d2000cd80 con 0x7f3d30100500 2024-09-10T09:46:57.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:57.194+0000 7f3d2cc00640 1 -- 172.21.15.5:0/1774540555 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3d200035f0 con 0x7f3d30100500 2024-09-10T09:46:57.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:57.194+0000 7f3d36c00640 1 -- 172.21.15.5:0/1774540555 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d3019d720 con 0x7f3d30100500 2024-09-10T09:46:57.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:57.194+0000 7f3d2cc00640 1 -- 172.21.15.5:0/1774540555 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d20010690 con 0x7f3d30100500 2024-09-10T09:46:57.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:57.194+0000 7f3d36c00640 1 -- 172.21.15.5:0/1774540555 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d301a5000 con 0x7f3d30100500 2024-09-10T09:46:57.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:57.197+0000 7f3d397e5740 1 Processor -- start 2024-09-10T09:46:57.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:57.197+0000 7f3d2cc00640 1 -- 172.21.15.5:0/1774540555 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f3d200037a0 con 0x7f3d30100500 2024-09-10T09:46:57.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:57.197+0000 7f3d397e5740 1 -- start start 2024-09-10T09:46:57.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:57.198+0000 7f3d397e5740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562eefe0f1f0 0x562eefe0db10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:46:57.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:57.198+0000 7f3d397e5740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562eefe0e050 0x562eefe0e490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:46:57.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:57.198+0000 7f3d397e5740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562eefe214f0 0x562eefe238e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:46:57.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:57.198+0000 7f3d397e5740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x562eefd743e0 con 0x562eefe0f1f0 2024-09-10T09:46:57.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:57.198+0000 7f3d397e5740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x562eefd1bd80 con 0x562eefe0e050 2024-09-10T09:46:57.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:57.198+0000 7f3d397e5740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x562eefd703a0 con 0x562eefe214f0 2024-09-10T09:46:57.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:57.199+0000 7f3d34c00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562eefe0f1f0 0x562eefe0db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:46:57.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:57.199+0000 7f3d2f400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562eefe214f0 0x562eefe238e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:46:57.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:57.199+0000 7f3d2f400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562eefe214f0 0x562eefe238e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:53332/0 (socket says 172.21.15.5:53332) 2024-09-10T09:46:57.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:57.199+0000 7f3d2f400640 1 -- 172.21.15.5:0/8751224 learned_addr learned my addr 172.21.15.5:0/8751224 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:46:57.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:57.199+0000 7f3d2fe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562eefe0e050 0x562eefe0e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:46:57.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:57.199+0000 7f3d2f400640 1 -- 172.21.15.5:0/8751224 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562eefe0e050 msgr2=0x562eefe0e490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:46:57.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:57.199+0000 7f3d2f400640 1 --2- 172.21.15.5:0/8751224 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562eefe0e050 0x562eefe0e490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:57.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:57.199+0000 7f3d2f400640 1 -- 172.21.15.5:0/8751224 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562eefe0f1f0 msgr2=0x562eefe0db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:46:57.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:57.199+0000 7f3d2f400640 1 --2- 172.21.15.5:0/8751224 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562eefe0f1f0 0x562eefe0db10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:57.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:57.199+0000 7f3d2f400640 1 -- 172.21.15.5:0/8751224 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562eefd3aae0 con 0x562eefe214f0 2024-09-10T09:46:57.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:57.199+0000 7f3d2fe00640 1 --2- 172.21.15.5:0/8751224 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562eefe0e050 0x562eefe0e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:46:57.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:57.200+0000 7f3d34c00640 1 --2- 172.21.15.5:0/8751224 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562eefe0f1f0 0x562eefe0db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:46:57.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:57.200+0000 7f3d2cc00640 1 --2- 172.21.15.5:0/1774540555 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f3d140a2c80 0x7f3d140a5140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:46:57.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:57.200+0000 7f3d2cc00640 1 -- 172.21.15.5:0/1774540555 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f3d200cf7f0 con 0x7f3d30100500 2024-09-10T09:46:57.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:57.200+0000 7f3d2f400640 1 --2- 172.21.15.5:0/8751224 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562eefe214f0 0x562eefe238e0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f3d1c009bd0 tx=0x7f3d1c011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e4e4933ea16f487d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:46:57.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:57.200+0000 7f3d34c00640 1 --2- 172.21.15.5:0/1774540555 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f3d140a2c80 0x7f3d140a5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:46:57.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:57.200+0000 7f3d25200640 1 -- 172.21.15.5:0/8751224 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d1c004230 con 0x562eefe214f0 2024-09-10T09:46:57.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:57.200+0000 7f3d25200640 1 -- 172.21.15.5:0/8751224 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3d1c009e40 con 0x562eefe214f0 2024-09-10T09:46:57.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:57.200+0000 7f3d25200640 1 -- 172.21.15.5:0/8751224 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d1c004e40 con 0x562eefe214f0 2024-09-10T09:46:57.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:57.201+0000 7f3d397e5740 1 -- 172.21.15.5:0/8751224 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562eefe214f0 msgr2=0x562eefe238e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:46:57.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:57.201+0000 7f3d34c00640 1 --2- 172.21.15.5:0/1774540555 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f3d140a2c80 0x7f3d140a5140 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f3d28006010 tx=0x7f3d28005d80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:46:57.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:57.201+0000 7f3d397e5740 1 --2- 172.21.15.5:0/8751224 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562eefe214f0 0x562eefe238e0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f3d1c009bd0 tx=0x7f3d1c011810 comp rx=0 tx=0).stop 2024-09-10T09:46:57.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:57.201+0000 7f3d397e5740 1 -- 172.21.15.5:0/8751224 shutdown_connections 2024-09-10T09:46:57.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:57.201+0000 7f3d397e5740 1 --2- 172.21.15.5:0/8751224 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562eefe214f0 0x562eefe238e0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:57.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:57.201+0000 7f3d397e5740 1 --2- 172.21.15.5:0/8751224 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562eefe0e050 0x562eefe0e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:57.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:57.201+0000 7f3d397e5740 1 --2- 172.21.15.5:0/8751224 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562eefe0f1f0 0x562eefe0db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:57.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:57.201+0000 7f3d397e5740 1 -- 172.21.15.5:0/8751224 >> 172.21.15.5:0/8751224 conn(0x562eefe0a200 msgr2=0x562eefe0a5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:46:57.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:57.202+0000 7f3d397e5740 1 -- 172.21.15.5:0/8751224 shutdown_connections 2024-09-10T09:46:57.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:57.202+0000 7f3d397e5740 1 -- 172.21.15.5:0/8751224 wait complete. 2024-09-10T09:46:57.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:57.202+0000 7f3d397e5740 1 -- start start 2024-09-10T09:46:57.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:57.202+0000 7f3d397e5740 1 Processor -- start 2024-09-10T09:46:57.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:57.203+0000 7f3d397e5740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562eefe0e050 0x562eefe11950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:46:57.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:57.203+0000 7f3d397e5740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562eefe0f1f0 0x562eefe11e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:46:57.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:57.203+0000 7f3d397e5740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562eefe214f0 0x562eefe123d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:46:57.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:57.203+0000 7f3d397e5740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x562eefe24220 con 0x562eefe0f1f0 2024-09-10T09:46:57.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:57.203+0000 7f3d397e5740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x562eefd2f0f0 con 0x562eefe214f0 2024-09-10T09:46:57.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:57.203+0000 7f3d397e5740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x562eefe243a0 con 0x562eefe0e050 2024-09-10T09:46:57.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:57.203+0000 7f3d397e5740 10 client.0 _ll_register_callbacks cb 0x562eefd16bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:46:57.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:57.203+0000 7f3d34c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562eefe214f0 0x562eefe123d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:46:57.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:57.203+0000 7f3d2fe00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562eefe0e050 0x562eefe11950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:46:57.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:57.203+0000 7f3d2fe00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562eefe0e050 0x562eefe11950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:53344/0 (socket says 172.21.15.5:53344) 2024-09-10T09:46:57.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:57.203+0000 7f3d2fe00640 1 -- 172.21.15.5:0/3595994852 learned_addr learned my addr 172.21.15.5:0/3595994852 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:46:57.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:57.203+0000 7f3d2f400640 1 --2- 172.21.15.5:0/3595994852 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562eefe0f1f0 0x562eefe11e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:46:57.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:57.203+0000 7f3d2fe00640 1 -- 172.21.15.5:0/3595994852 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562eefe214f0 msgr2=0x562eefe123d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:46:57.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:57.204+0000 7f3d2fe00640 1 --2- 172.21.15.5:0/3595994852 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562eefe214f0 0x562eefe123d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:57.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:57.204+0000 7f3d2fe00640 1 -- 172.21.15.5:0/3595994852 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562eefe0f1f0 msgr2=0x562eefe11e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:46:57.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:57.204+0000 7f3d2fe00640 1 --2- 172.21.15.5:0/3595994852 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562eefe0f1f0 0x562eefe11e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:57.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:57.204+0000 7f3d2fe00640 1 -- 172.21.15.5:0/3595994852 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562eefae53c0 con 0x562eefe0e050 2024-09-10T09:46:57.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:57.204+0000 7f3d2fe00640 1 --2- 172.21.15.5:0/3595994852 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562eefe0e050 0x562eefe11950 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f3d2009bb90 tx=0x7f3d2009ba90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fcb9b7f496d56131 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:46:57.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:57.204+0000 7f3d19600640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-09-10T09:46:57.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:57.204+0000 7f3d19600640 1 -- 172.21.15.5:0/3595994852 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d2009bc00 con 0x562eefe0e050 2024-09-10T09:46:57.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:57.204+0000 7f3d19600640 1 -- 172.21.15.5:0/3595994852 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f3d200cfba0 con 0x562eefe0e050 2024-09-10T09:46:57.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:57.204+0000 7f3d397e5740 1 -- 172.21.15.5:0/3595994852 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x562eefd81950 con 0x562eefe0e050 2024-09-10T09:46:57.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:57.204+0000 7f3d19600640 1 -- 172.21.15.5:0/3595994852 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f3d2009ffb0 con 0x562eefe0e050 2024-09-10T09:46:57.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:57.204+0000 7f3d397e5740 10 client.27142 fetch_fsmap learned FSMap version 35 2024-09-10T09:46:57.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:57.204+0000 7f3d397e5740 1 -- 172.21.15.5:0/3595994852 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x562eefe303a0 con 0x562eefe0e050 2024-09-10T09:46:57.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:57.205+0000 7f3d19600640 1 -- 172.21.15.5:0/3595994852 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x562eefd81950 con 0x562eefe0e050 2024-09-10T09:46:57.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:57.205+0000 7f3d19600640 1 -- 172.21.15.5:0/3595994852 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 35) ==== 8374+0+0 (secure 0 0 0) 0x7f3d200d2ce0 con 0x562eefe0e050 2024-09-10T09:46:57.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:57.205+0000 7f3d397e5740 10 client.27142 fetch_fsmap finished waiting for FSMap version 35 2024-09-10T09:46:57.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:57.205+0000 7f3d397e5740 10 client.27142 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-10T09:46:57.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:57.205+0000 7f3d397e5740 20 client.27142 populate_metadata read hostname 'smithi005' 2024-09-10T09:46:57.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:57.205+0000 7f3d397e5740 1 --2- 172.21.15.5:0/3595994852 >> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] conn(0x562eefe317b0 0x562eefe33c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:46:57.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:57.205+0000 7f3d397e5740 4 client.27142 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:46:57.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:57.205+0000 7f3d397e5740 1 -- 172.21.15.5:0/3595994852 --> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562eefd7be60 con 0x562eefe317b0 2024-09-10T09:46:57.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:57.205+0000 7f3d2f400640 1 --2- 172.21.15.5:0/3595994852 >> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] conn(0x562eefe317b0 0x562eefe33c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:46:57.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:57.206+0000 7f3d2f400640 1 --2- 172.21.15.5:0/3595994852 >> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] conn(0x562eefe317b0 0x562eefe33c20 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b48f28e6f352ce2c server_cookie=752b888da577a1d5 in_seq=0 out_seq=0 2024-09-10T09:46:57.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:57.206+0000 7f3d19600640 10 client.27142 ms_handle_connect on v2:172.21.15.159:6838/2073739599 2024-09-10T09:46:57.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:57.207+0000 7f3d19600640 1 -- 172.21.15.5:0/3595994852 <== mds.1 v2:172.21.15.159:6838/2073739599 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562eefd7be60 con 0x562eefe317b0 2024-09-10T09:46:57.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:57.207+0000 7f3d19600640 10 client.27142 handle_command_reply: tid=0 2024-09-10T09:46:57.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:57.207+0000 7f3d397e5740 1 client.27142 shutdown 2024-09-10T09:46:57.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:57.207+0000 7f3d397e5740 1 -- 172.21.15.5:0/3595994852 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562eefe0e050 msgr2=0x562eefe11950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:46:57.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:57.207+0000 7f3d397e5740 1 --2- 172.21.15.5:0/3595994852 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562eefe0e050 0x562eefe11950 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f3d2009bb90 tx=0x7f3d2009ba90 comp rx=0 tx=0).stop 2024-09-10T09:46:57.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:57.207+0000 7f3d397e5740 1 -- 172.21.15.5:0/3595994852 shutdown_connections 2024-09-10T09:46:57.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:57.207+0000 7f3d397e5740 1 --2- 172.21.15.5:0/3595994852 >> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] conn(0x562eefe317b0 0x562eefe33c20 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:57.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:57.207+0000 7f3d397e5740 1 --2- 172.21.15.5:0/3595994852 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562eefe214f0 0x562eefe123d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:57.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:57.207+0000 7f3d397e5740 1 --2- 172.21.15.5:0/3595994852 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562eefe0f1f0 0x562eefe11e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:57.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:57.207+0000 7f3d397e5740 1 --2- 172.21.15.5:0/3595994852 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562eefe0e050 0x562eefe11950 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:57.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:57.207+0000 7f3d19600640 0 client.27142 ms_handle_reset on v2:172.21.15.159:6838/2073739599 2024-09-10T09:46:57.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:57.207+0000 7f3d397e5740 1 -- 172.21.15.5:0/3595994852 >> 172.21.15.5:0/3595994852 conn(0x562eefe0a200 msgr2=0x562eefe0ff60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:46:57.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:57.207+0000 7f3d397e5740 1 -- 172.21.15.5:0/3595994852 shutdown_connections 2024-09-10T09:46:57.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:57.208+0000 7f3d397e5740 1 -- 172.21.15.5:0/3595994852 wait complete. 2024-09-10T09:46:57.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:57.208+0000 7f3d397e5740 20 client.27142 trim_cache size 0 max 16384 2024-09-10T09:46:57.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:57.223+0000 7f3d397e5740 1 Processor -- start 2024-09-10T09:46:57.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:57.224+0000 7f3d397e5740 1 -- start start 2024-09-10T09:46:57.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:57.224+0000 7f3d397e5740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562eefe0e050 0x562eefe1d890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:46:57.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:57.224+0000 7f3d397e5740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562eefe0f1f0 0x562eefe1ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:46:57.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:57.224+0000 7f3d397e5740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562eefe214f0 0x562eefe39510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:46:57.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:57.224+0000 7f3d397e5740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f3d301acbd0 con 0x562eefe0e050 2024-09-10T09:46:57.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:57.224+0000 7f3d397e5740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f3d301ae740 con 0x562eefe0f1f0 2024-09-10T09:46:57.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:57.224+0000 7f3d397e5740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f3d301ad2a0 con 0x562eefe214f0 2024-09-10T09:46:57.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:57.224+0000 7f3d34c00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562eefe214f0 0x562eefe39510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:46:57.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:57.224+0000 7f3d2fe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562eefe0e050 0x562eefe1d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:46:57.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:57.224+0000 7f3d34c00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562eefe214f0 0x562eefe39510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:53350/0 (socket says 172.21.15.5:53350) 2024-09-10T09:46:57.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:57.224+0000 7f3d34c00640 1 -- 172.21.15.5:0/1433350147 learned_addr learned my addr 172.21.15.5:0/1433350147 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:46:57.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:57.224+0000 7f3d2f400640 1 --2- 172.21.15.5:0/1433350147 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562eefe0f1f0 0x562eefe1ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:46:57.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:57.225+0000 7f3d34c00640 1 -- 172.21.15.5:0/1433350147 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562eefe0f1f0 msgr2=0x562eefe1ddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:46:57.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:57.225+0000 7f3d34c00640 1 --2- 172.21.15.5:0/1433350147 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562eefe0f1f0 0x562eefe1ddd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:57.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:57.225+0000 7f3d34c00640 1 -- 172.21.15.5:0/1433350147 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562eefe0e050 msgr2=0x562eefe1d890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:46:57.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:57.225+0000 7f3d34c00640 1 --2- 172.21.15.5:0/1433350147 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562eefe0e050 0x562eefe1d890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:57.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:57.225+0000 7f3d34c00640 1 -- 172.21.15.5:0/1433350147 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562eefe14980 con 0x562eefe214f0 2024-09-10T09:46:57.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:57.225+0000 7f3d34c00640 1 --2- 172.21.15.5:0/1433350147 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562eefe214f0 0x562eefe39510 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f3d28009ca0 tx=0x7f3d28042000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1245c99cfe57f2c7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:46:57.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:57.225+0000 7f3d1be00640 1 -- 172.21.15.5:0/1433350147 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d28003d20 con 0x562eefe214f0 2024-09-10T09:46:57.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:57.226+0000 7f3d1be00640 1 -- 172.21.15.5:0/1433350147 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3d28036040 con 0x562eefe214f0 2024-09-10T09:46:57.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:57.226+0000 7f3d1be00640 1 -- 172.21.15.5:0/1433350147 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d2802f8c0 con 0x562eefe214f0 2024-09-10T09:46:57.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:57.226+0000 7f3d397e5740 1 -- 172.21.15.5:0/1433350147 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562eefe214f0 msgr2=0x562eefe39510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:46:57.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:57.226+0000 7f3d397e5740 1 --2- 172.21.15.5:0/1433350147 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562eefe214f0 0x562eefe39510 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f3d28009ca0 tx=0x7f3d28042000 comp rx=0 tx=0).stop 2024-09-10T09:46:57.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:57.227+0000 7f3d397e5740 1 -- 172.21.15.5:0/1433350147 shutdown_connections 2024-09-10T09:46:57.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:57.227+0000 7f3d397e5740 1 --2- 172.21.15.5:0/1433350147 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562eefe214f0 0x562eefe39510 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:57.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:57.227+0000 7f3d397e5740 1 --2- 172.21.15.5:0/1433350147 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562eefe0f1f0 0x562eefe1ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:57.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:57.227+0000 7f3d397e5740 1 --2- 172.21.15.5:0/1433350147 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562eefe0e050 0x562eefe1d890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:57.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:57.227+0000 7f3d397e5740 1 -- 172.21.15.5:0/1433350147 >> 172.21.15.5:0/1433350147 conn(0x562eefe0a200 msgr2=0x562eefe111b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:46:57.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:57.227+0000 7f3d397e5740 1 -- 172.21.15.5:0/1433350147 shutdown_connections 2024-09-10T09:46:57.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:57.227+0000 7f3d397e5740 1 -- 172.21.15.5:0/1433350147 wait complete. 2024-09-10T09:46:57.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:57.227+0000 7f3d397e5740 1 -- start start 2024-09-10T09:46:57.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:57.228+0000 7f3d397e5740 1 Processor -- start 2024-09-10T09:46:57.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:57.228+0000 7f3d397e5740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562eefe0e050 0x562eefe29c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:46:57.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:57.228+0000 7f3d397e5740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562eefe0f1f0 0x562eefe38c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:46:57.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:57.228+0000 7f3d397e5740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562eefe214f0 0x562eefe391c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:46:57.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:57.228+0000 7f3d397e5740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f3d301ae040 con 0x562eefe0f1f0 2024-09-10T09:46:57.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:57.228+0000 7f3d397e5740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f3d301ad970 con 0x562eefe0e050 2024-09-10T09:46:57.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:57.228+0000 7f3d397e5740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f3d301aabb0 con 0x562eefe214f0 2024-09-10T09:46:57.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:57.228+0000 7f3d397e5740 10 client.0 _ll_register_callbacks cb 0x7f3d20097ea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:46:57.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:57.228+0000 7f3d2fe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562eefe0e050 0x562eefe29c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:46:57.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:57.228+0000 7f3d34c00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562eefe214f0 0x562eefe391c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:46:57.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:57.228+0000 7f3d2f400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562eefe0f1f0 0x562eefe38c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:46:57.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:57.228+0000 7f3d2fe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562eefe0e050 0x562eefe29c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:42098/0 (socket says 172.21.15.5:42098) 2024-09-10T09:46:57.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:57.228+0000 7f3d34c00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562eefe214f0 0x562eefe391c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:53354/0 (socket says 172.21.15.5:53354) 2024-09-10T09:46:57.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:57.228+0000 7f3d2fe00640 1 -- 172.21.15.5:0/2857387406 learned_addr learned my addr 172.21.15.5:0/2857387406 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:46:57.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:57.229+0000 7f3d34c00640 1 -- 172.21.15.5:0/2857387406 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562eefe0e050 msgr2=0x562eefe29c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:46:57.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:57.229+0000 7f3d34c00640 1 --2- 172.21.15.5:0/2857387406 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562eefe0e050 0x562eefe29c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:57.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:57.229+0000 7f3d34c00640 1 -- 172.21.15.5:0/2857387406 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562eefe0f1f0 msgr2=0x562eefe38c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:46:57.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:57.229+0000 7f3d34c00640 1 --2- 172.21.15.5:0/2857387406 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562eefe0f1f0 0x562eefe38c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:57.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:57.229+0000 7f3d34c00640 1 -- 172.21.15.5:0/2857387406 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562eefe39830 con 0x562eefe214f0 2024-09-10T09:46:57.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:57.229+0000 7f3d2fe00640 1 --2- 172.21.15.5:0/2857387406 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562eefe0e050 0x562eefe29c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:46:57.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:57.229+0000 7f3d34c00640 1 --2- 172.21.15.5:0/2857387406 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562eefe214f0 0x562eefe391c0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f3d28004af0 tx=0x7f3d28004530 comp rx=0 tx=0).ready entity=mon.1 client_cookie=abe6e0a1d5b29259 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:46:57.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:57.229+0000 7f3d19600640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-09-10T09:46:57.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:57.230+0000 7f3d19600640 1 -- 172.21.15.5:0/2857387406 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d2802f720 con 0x562eefe214f0 2024-09-10T09:46:57.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:57.230+0000 7f3d19600640 1 -- 172.21.15.5:0/2857387406 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f3d2802f8c0 con 0x562eefe214f0 2024-09-10T09:46:57.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:57.230+0000 7f3d397e5740 1 -- 172.21.15.5:0/2857387406 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3d28036040 con 0x562eefe214f0 2024-09-10T09:46:57.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:57.230+0000 7f3d19600640 1 -- 172.21.15.5:0/2857387406 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f3d28004b40 con 0x562eefe214f0 2024-09-10T09:46:57.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:57.230+0000 7f3d397e5740 10 client.27148 fetch_fsmap learned FSMap version 35 2024-09-10T09:46:57.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:57.230+0000 7f3d397e5740 1 -- 172.21.15.5:0/2857387406 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x562eefe9c4e0 con 0x562eefe214f0 2024-09-10T09:46:57.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:57.230+0000 7f3d19600640 1 -- 172.21.15.5:0/2857387406 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3d28036040 con 0x562eefe214f0 2024-09-10T09:46:57.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:57.230+0000 7f3d19600640 1 -- 172.21.15.5:0/2857387406 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 35) ==== 8374+0+0 (secure 0 0 0) 0x7f3d280305c0 con 0x562eefe214f0 2024-09-10T09:46:57.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:57.230+0000 7f3d397e5740 10 client.27148 fetch_fsmap finished waiting for FSMap version 35 2024-09-10T09:46:57.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:57.230+0000 7f3d397e5740 10 client.27148 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-10T09:46:57.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:57.231+0000 7f3d397e5740 20 client.27148 populate_metadata read hostname 'smithi005' 2024-09-10T09:46:57.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:57.231+0000 7f3d397e5740 1 --2- 172.21.15.5:0/2857387406 >> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] conn(0x562eefe9ca40 0x562eefe9ee30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:46:57.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:57.231+0000 7f3d397e5740 4 client.27148 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T09:46:57.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:57.231+0000 7f3d397e5740 1 -- 172.21.15.5:0/2857387406 --> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562eefcf8010 con 0x562eefe9ca40 2024-09-10T09:46:57.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:57.231+0000 7f3d2fe00640 1 --2- 172.21.15.5:0/2857387406 >> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] conn(0x562eefe9ca40 0x562eefe9ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:46:57.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:57.231+0000 7f3d2fe00640 1 --2- 172.21.15.5:0/2857387406 >> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] conn(0x562eefe9ca40 0x562eefe9ee30 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f2d691897e833979 server_cookie=89204aeea9b00ca4 in_seq=0 out_seq=0 2024-09-10T09:46:57.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:57.231+0000 7f3d19600640 10 client.27148 ms_handle_connect on v2:172.21.15.159:6838/2073739599 2024-09-10T09:46:57.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:57.232+0000 7f3d19600640 1 -- 172.21.15.5:0/2857387406 <== mds.1 v2:172.21.15.159:6838/2073739599 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562eefcf8010 con 0x562eefe9ca40 2024-09-10T09:46:57.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:57.232+0000 7f3d19600640 10 client.27148 handle_command_reply: tid=0 2024-09-10T09:46:57.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:57.232+0000 7f3d397e5740 1 client.27148 shutdown 2024-09-10T09:46:57.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:57.232+0000 7f3d397e5740 1 -- 172.21.15.5:0/2857387406 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562eefe214f0 msgr2=0x562eefe391c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:46:57.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:57.232+0000 7f3d397e5740 1 --2- 172.21.15.5:0/2857387406 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562eefe214f0 0x562eefe391c0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f3d28004af0 tx=0x7f3d28004530 comp rx=0 tx=0).stop 2024-09-10T09:46:57.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:57.232+0000 7f3d397e5740 1 -- 172.21.15.5:0/2857387406 shutdown_connections 2024-09-10T09:46:57.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:57.232+0000 7f3d397e5740 1 --2- 172.21.15.5:0/2857387406 >> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] conn(0x562eefe9ca40 0x562eefe9ee30 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:57.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:57.232+0000 7f3d397e5740 1 --2- 172.21.15.5:0/2857387406 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562eefe214f0 0x562eefe391c0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:57.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:57.232+0000 7f3d397e5740 1 --2- 172.21.15.5:0/2857387406 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562eefe0f1f0 0x562eefe38c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:57.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:57.232+0000 7f3d397e5740 1 --2- 172.21.15.5:0/2857387406 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562eefe0e050 0x562eefe29c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:57.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:57.232+0000 7f3d19600640 0 client.27148 ms_handle_reset on v2:172.21.15.159:6838/2073739599 2024-09-10T09:46:57.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:57.232+0000 7f3d397e5740 1 -- 172.21.15.5:0/2857387406 >> 172.21.15.5:0/2857387406 conn(0x562eefe0a200 msgr2=0x562eefe09380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:46:57.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:57.232+0000 7f3d397e5740 1 -- 172.21.15.5:0/2857387406 shutdown_connections 2024-09-10T09:46:57.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:57.232+0000 7f3d397e5740 1 -- 172.21.15.5:0/2857387406 wait complete. 2024-09-10T09:46:57.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:57.233+0000 7f3d397e5740 20 client.27148 trim_cache size 0 max 16384 2024-09-10T09:46:57.236 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-10T09:46:57.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:57.234+0000 7f3d36c00640 1 -- 172.21.15.5:0/1774540555 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f3d140a2c80 msgr2=0x7f3d140a5140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:46:57.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:57.234+0000 7f3d36c00640 1 --2- 172.21.15.5:0/1774540555 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f3d140a2c80 0x7f3d140a5140 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f3d28006010 tx=0x7f3d28005d80 comp rx=0 tx=0).stop 2024-09-10T09:46:57.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:57.234+0000 7f3d36c00640 1 -- 172.21.15.5:0/1774540555 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3d30100500 msgr2=0x7f3d30198480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:46:57.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:57.234+0000 7f3d36c00640 1 --2- 172.21.15.5:0/1774540555 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3d30100500 0x7f3d30198480 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f3d2000e9c0 tx=0x7f3d2000ee80 comp rx=0 tx=0).stop 2024-09-10T09:46:57.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:57.234+0000 7f3d36c00640 1 -- 172.21.15.5:0/1774540555 shutdown_connections 2024-09-10T09:46:57.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:57.234+0000 7f3d36c00640 1 --2- 172.21.15.5:0/1774540555 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f3d140a2c80 0x7f3d140a5140 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:57.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:57.234+0000 7f3d36c00640 1 --2- 172.21.15.5:0/1774540555 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3d30106f70 0x7f3d3019cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:57.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:57.234+0000 7f3d36c00640 1 --2- 172.21.15.5:0/1774540555 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3d30100eb0 0x7f3d301989c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:57.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:57.234+0000 7f3d36c00640 1 --2- 172.21.15.5:0/1774540555 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3d30100500 0x7f3d30198480 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:57.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:57.234+0000 7f3d36c00640 1 -- 172.21.15.5:0/1774540555 >> 172.21.15.5:0/1774540555 conn(0x7f3d300fc5c0 msgr2=0x7f3d301057f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:46:57.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:57.234+0000 7f3d36c00640 1 -- 172.21.15.5:0/1774540555 shutdown_connections 2024-09-10T09:46:57.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:57.234+0000 7f3d36c00640 1 -- 172.21.15.5:0/1774540555 wait complete. 2024-09-10T09:46:57.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:46:56 smithi159 ceph-mon[35513]: pgmap v651: 129 pgs: 129 active+clean; 198 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 191 KiB/s rd, 2.5 MiB/s wr, 83 op/s 2024-09-10T09:46:57.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:46:56 smithi102 ceph-mon[35442]: pgmap v651: 129 pgs: 129 active+clean; 198 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 191 KiB/s rd, 2.5 MiB/s wr, 83 op/s 2024-09-10T09:46:57.995 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-10T09:46:58.076 INFO:tasks.workunit.client.0.smithi005.stdout: 1 505061 6.11 MB/sec execute 279 sec latency 31.999 ms 2024-09-10T09:46:58.414 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:46:59.009 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:46:58 smithi005 ceph-mon[30350]: pgmap v652: 129 pgs: 129 active+clean; 198 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 2.4 MiB/s wr, 54 op/s 2024-09-10T09:46:59.076 INFO:tasks.workunit.client.0.smithi005.stdout: 1 505615 6.09 MB/sec execute 280 sec latency 563.477 ms 2024-09-10T09:46:59.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:46:58 smithi159 ceph-mon[35513]: pgmap v652: 129 pgs: 129 active+clean; 198 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 2.4 MiB/s wr, 54 op/s 2024-09-10T09:46:59.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:46:58 smithi102 ceph-mon[35442]: pgmap v652: 129 pgs: 129 active+clean; 198 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 2.4 MiB/s wr, 54 op/s 2024-09-10T09:46:59.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:59.495+0000 7f4254400640 1 -- 172.21.15.5:0/1428231055 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f424c103ff0 msgr2=0x7f424c1043d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:46:59.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:59.495+0000 7f4254400640 1 --2- 172.21.15.5:0/1428231055 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f424c103ff0 0x7f424c1043d0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f4240009900 tx=0x7f424002a400 comp rx=0 tx=0).stop 2024-09-10T09:46:59.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:59.495+0000 7f4254400640 1 -- 172.21.15.5:0/1428231055 shutdown_connections 2024-09-10T09:46:59.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:59.495+0000 7f4254400640 1 --2- 172.21.15.5:0/1428231055 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f424c109120 0x7f424c110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:59.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:59.495+0000 7f4254400640 1 --2- 172.21.15.5:0/1428231055 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f424c1049a0 0x7f424c1089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:59.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:59.495+0000 7f4254400640 1 --2- 172.21.15.5:0/1428231055 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f424c103ff0 0x7f424c1043d0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:59.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:59.495+0000 7f4254400640 1 -- 172.21.15.5:0/1428231055 >> 172.21.15.5:0/1428231055 conn(0x7f424c0ffa90 msgr2=0x7f424c101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:46:59.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:59.495+0000 7f4254400640 1 -- 172.21.15.5:0/1428231055 shutdown_connections 2024-09-10T09:46:59.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:59.495+0000 7f4254400640 1 -- 172.21.15.5:0/1428231055 wait complete. 2024-09-10T09:46:59.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:59.496+0000 7f4254400640 1 Processor -- start 2024-09-10T09:46:59.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:59.497+0000 7f4254400640 1 -- start start 2024-09-10T09:46:59.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:59.497+0000 7f4254400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f424c103ff0 0x7f424c19a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:46:59.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:59.497+0000 7f4254400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f424c1049a0 0x7f424c19abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:46:59.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:59.497+0000 7f4254400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f424c109120 0x7f424c19ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:46:59.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:59.497+0000 7f4254400640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f424c114170 con 0x7f424c1049a0 2024-09-10T09:46:59.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:59.497+0000 7f4254400640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f424c113ff0 con 0x7f424c103ff0 2024-09-10T09:46:59.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:59.497+0000 7f4254400640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f424c1142f0 con 0x7f424c109120 2024-09-10T09:46:59.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:59.497+0000 7f4251000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f424c1049a0 0x7f424c19abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:46:59.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:59.498+0000 7f4251000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f424c1049a0 0x7f424c19abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42064/0 (socket says 172.21.15.5:42064) 2024-09-10T09:46:59.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:59.498+0000 7f4251000640 1 -- 172.21.15.5:0/1780794928 learned_addr learned my addr 172.21.15.5:0/1780794928 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:46:59.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:59.498+0000 7f4251a00640 1 --2- 172.21.15.5:0/1780794928 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f424c103ff0 0x7f424c19a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:46:59.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:59.498+0000 7f4251000640 1 -- 172.21.15.5:0/1780794928 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f424c109120 msgr2=0x7f424c19ef60 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-10T09:46:59.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:59.498+0000 7f4251000640 1 --2- 172.21.15.5:0/1780794928 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f424c109120 0x7f424c19ef60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:59.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:59.498+0000 7f4251000640 1 -- 172.21.15.5:0/1780794928 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f424c103ff0 msgr2=0x7f424c19a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:46:59.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:59.498+0000 7f4251000640 1 --2- 172.21.15.5:0/1780794928 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f424c103ff0 0x7f424c19a690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:59.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:59.498+0000 7f4251000640 1 -- 172.21.15.5:0/1780794928 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f424c19f640 con 0x7f424c1049a0 2024-09-10T09:46:59.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:59.498+0000 7f4251a00640 1 --2- 172.21.15.5:0/1780794928 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f424c103ff0 0x7f424c19a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:46:59.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:59.498+0000 7f4251000640 1 --2- 172.21.15.5:0/1780794928 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f424c1049a0 0x7f424c19abd0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f423800ea60 tx=0x7f423800ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:46:59.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:59.499+0000 7f424a000640 1 -- 172.21.15.5:0/1780794928 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f423800cde0 con 0x7f424c1049a0 2024-09-10T09:46:59.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:59.499+0000 7f424a000640 1 -- 172.21.15.5:0/1780794928 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f42380035f0 con 0x7f424c1049a0 2024-09-10T09:46:59.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:59.499+0000 7f424a000640 1 -- 172.21.15.5:0/1780794928 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4238010670 con 0x7f424c1049a0 2024-09-10T09:46:59.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:59.499+0000 7f4254400640 1 -- 172.21.15.5:0/1780794928 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f424c19f930 con 0x7f424c1049a0 2024-09-10T09:46:59.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:59.499+0000 7f4254400640 1 -- 172.21.15.5:0/1780794928 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f424c1a7210 con 0x7f424c1049a0 2024-09-10T09:46:59.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:59.502+0000 7f4256e1f740 1 Processor -- start 2024-09-10T09:46:59.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:59.502+0000 7f4256e1f740 1 -- start start 2024-09-10T09:46:59.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:59.502+0000 7f4256e1f740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559283c74c70 0x559283c75050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:46:59.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:59.502+0000 7f4256e1f740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559283c75590 0x559283c7f910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:46:59.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:59.502+0000 7f4256e1f740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559283c7fe50 0x559283c82240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:46:59.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:59.502+0000 7f4256e1f740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x559283bdb3e0 con 0x559283c7fe50 2024-09-10T09:46:59.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:59.502+0000 7f4256e1f740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x559283b82d80 con 0x559283c74c70 2024-09-10T09:46:59.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:59.502+0000 7f4256e1f740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x559283bd73a0 con 0x559283c75590 2024-09-10T09:46:59.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:59.503+0000 7f4251a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559283c75590 0x559283c7f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:46:59.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:59.503+0000 7f4251a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559283c75590 0x559283c7f910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:53372/0 (socket says 172.21.15.5:53372) 2024-09-10T09:46:59.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:59.503+0000 7f4251a00640 1 -- 172.21.15.5:0/947876729 learned_addr learned my addr 172.21.15.5:0/947876729 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:46:59.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:59.503+0000 7f424a000640 1 -- 172.21.15.5:0/1780794928 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f42380037a0 con 0x7f424c1049a0 2024-09-10T09:46:59.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:59.503+0000 7f4251a00640 1 -- 172.21.15.5:0/947876729 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559283c74c70 msgr2=0x559283c75050 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T09:46:59.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:59.503+0000 7f4251a00640 1 --2- 172.21.15.5:0/947876729 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559283c74c70 0x559283c75050 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:59.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:59.503+0000 7f4251a00640 1 -- 172.21.15.5:0/947876729 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559283c7fe50 msgr2=0x559283c82240 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T09:46:59.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:59.503+0000 7f4251a00640 1 --2- 172.21.15.5:0/947876729 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559283c7fe50 0x559283c82240 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:59.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:59.503+0000 7f4251a00640 1 -- 172.21.15.5:0/947876729 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559283ba1ae0 con 0x559283c75590 2024-09-10T09:46:59.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:59.504+0000 7f4251a00640 1 --2- 172.21.15.5:0/947876729 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559283c75590 0x559283c7f910 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f4240009900 tx=0x7f42400043f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b7e4a7d729fe949d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:46:59.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:59.504+0000 7f423ea00640 1 -- 172.21.15.5:0/947876729 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4240035040 con 0x559283c75590 2024-09-10T09:46:59.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:59.504+0000 7f423ea00640 1 -- 172.21.15.5:0/947876729 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4240004760 con 0x559283c75590 2024-09-10T09:46:59.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:59.504+0000 7f423ea00640 1 -- 172.21.15.5:0/947876729 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42400039e0 con 0x559283c75590 2024-09-10T09:46:59.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:59.505+0000 7f4256e1f740 1 -- 172.21.15.5:0/947876729 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559283c75590 msgr2=0x559283c7f910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:46:59.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:59.505+0000 7f4256e1f740 1 --2- 172.21.15.5:0/947876729 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559283c75590 0x559283c7f910 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f4240009900 tx=0x7f42400043f0 comp rx=0 tx=0).stop 2024-09-10T09:46:59.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:59.505+0000 7f4256e1f740 1 -- 172.21.15.5:0/947876729 shutdown_connections 2024-09-10T09:46:59.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:59.505+0000 7f4256e1f740 1 --2- 172.21.15.5:0/947876729 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559283c7fe50 0x559283c82240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:59.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:59.505+0000 7f4256e1f740 1 --2- 172.21.15.5:0/947876729 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559283c75590 0x559283c7f910 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:59.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:59.505+0000 7f4256e1f740 1 --2- 172.21.15.5:0/947876729 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559283c74c70 0x559283c75050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:59.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:59.505+0000 7f4256e1f740 1 -- 172.21.15.5:0/947876729 >> 172.21.15.5:0/947876729 conn(0x559283c71200 msgr2=0x559283c715e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:46:59.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:59.505+0000 7f4256e1f740 1 -- 172.21.15.5:0/947876729 shutdown_connections 2024-09-10T09:46:59.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:59.505+0000 7f4256e1f740 1 -- 172.21.15.5:0/947876729 wait complete. 2024-09-10T09:46:59.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:59.505+0000 7f424a000640 1 --2- 172.21.15.5:0/1780794928 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f42340a2b10 0x7f42340a4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:46:59.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:59.505+0000 7f4256e1f740 1 -- start start 2024-09-10T09:46:59.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:59.506+0000 7f424a000640 1 -- 172.21.15.5:0/1780794928 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f42380cf790 con 0x7f424c1049a0 2024-09-10T09:46:59.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:59.506+0000 7f4252400640 1 --2- 172.21.15.5:0/1780794928 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f42340a2b10 0x7f42340a4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:46:59.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:59.506+0000 7f4256e1f740 1 Processor -- start 2024-09-10T09:46:59.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:59.506+0000 7f4256e1f740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559283c74c70 0x559283c7cd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:46:59.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:59.506+0000 7f4256e1f740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559283c75590 0x559283c7d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:46:59.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:59.506+0000 7f4256e1f740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559283c7fe50 0x559283c7d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:46:59.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:59.506+0000 7f4256e1f740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x559283c82a20 con 0x559283c75590 2024-09-10T09:46:59.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:59.506+0000 7f4256e1f740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x559283b960f0 con 0x559283c7fe50 2024-09-10T09:46:59.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:59.506+0000 7f4256e1f740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x559283c82ba0 con 0x559283c74c70 2024-09-10T09:46:59.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:59.507+0000 7f4256e1f740 10 client.0 _ll_register_callbacks cb 0x559283b7dbd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:46:59.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:59.507+0000 7f4252400640 1 --2- 172.21.15.5:0/1780794928 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f42340a2b10 0x7f42340a4fd0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f4244009370 tx=0x7f4244009240 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:46:59.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:59.507+0000 7f4252400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559283c7fe50 0x559283c7d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:46:59.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:59.507+0000 7f4251a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559283c74c70 0x559283c7cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:46:59.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:59.507+0000 7f4251000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559283c75590 0x559283c7d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:46:59.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:59.507+0000 7f4251a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559283c74c70 0x559283c7cd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:53384/0 (socket says 172.21.15.5:53384) 2024-09-10T09:46:59.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:59.507+0000 7f4252400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559283c7fe50 0x559283c7d7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:42142/0 (socket says 172.21.15.5:42142) 2024-09-10T09:46:59.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:59.507+0000 7f4251a00640 1 -- 172.21.15.5:0/1896527354 learned_addr learned my addr 172.21.15.5:0/1896527354 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:46:59.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:59.507+0000 7f4251a00640 1 -- 172.21.15.5:0/1896527354 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559283c7fe50 msgr2=0x559283c7d7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:46:59.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:59.507+0000 7f4251a00640 1 --2- 172.21.15.5:0/1896527354 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559283c7fe50 0x559283c7d7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:59.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:59.507+0000 7f4251a00640 1 -- 172.21.15.5:0/1896527354 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559283c75590 msgr2=0x559283c7d2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:46:59.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:59.507+0000 7f4251a00640 1 --2- 172.21.15.5:0/1896527354 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559283c75590 0x559283c7d2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:59.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:59.507+0000 7f4251a00640 1 -- 172.21.15.5:0/1896527354 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55928394c3c0 con 0x559283c74c70 2024-09-10T09:46:59.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:59.507+0000 7f4252400640 1 --2- 172.21.15.5:0/1896527354 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559283c7fe50 0x559283c7d7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T09:46:59.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:59.508+0000 7f4251a00640 1 --2- 172.21.15.5:0/1896527354 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559283c74c70 0x559283c7cd70 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f4240004b70 tx=0x7f4240000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fafcec3579ff7b8d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:46:59.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:59.508+0000 7f4233400640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-09-10T09:46:59.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:59.508+0000 7f4233400640 1 -- 172.21.15.5:0/1896527354 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4240052060 con 0x559283c74c70 2024-09-10T09:46:59.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:59.508+0000 7f4256e1f740 1 -- 172.21.15.5:0/1896527354 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4240004760 con 0x559283c74c70 2024-09-10T09:46:59.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:59.508+0000 7f4233400640 1 -- 172.21.15.5:0/1896527354 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f4240005130 con 0x559283c74c70 2024-09-10T09:46:59.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:59.508+0000 7f4233400640 1 -- 172.21.15.5:0/1896527354 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f4240035070 con 0x559283c74c70 2024-09-10T09:46:59.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:59.508+0000 7f4256e1f740 10 client.27157 fetch_fsmap learned FSMap version 35 2024-09-10T09:46:59.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:59.508+0000 7f4256e1f740 1 -- 172.21.15.5:0/1896527354 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x559283c82a20 con 0x559283c74c70 2024-09-10T09:46:59.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:59.508+0000 7f4233400640 1 -- 172.21.15.5:0/1896527354 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4240004760 con 0x559283c74c70 2024-09-10T09:46:59.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:59.509+0000 7f4233400640 1 -- 172.21.15.5:0/1896527354 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 35) ==== 8374+0+0 (secure 0 0 0) 0x7f424003c4d0 con 0x559283c74c70 2024-09-10T09:46:59.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:59.509+0000 7f4256e1f740 10 client.27157 fetch_fsmap finished waiting for FSMap version 35 2024-09-10T09:46:59.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:59.509+0000 7f4256e1f740 10 client.27157 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T09:46:59.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:59.509+0000 7f4256e1f740 20 client.27157 populate_metadata read hostname 'smithi005' 2024-09-10T09:46:59.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:59.509+0000 7f4256e1f740 1 --2- 172.21.15.5:0/1896527354 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x559283c8fc10 0x559283c92000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:46:59.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:59.509+0000 7f4256e1f740 4 client.27157 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:46:59.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:59.509+0000 7f4256e1f740 1 -- 172.21.15.5:0/1896527354 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559283be8950 con 0x559283c8fc10 2024-09-10T09:46:59.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:59.509+0000 7f4251000640 1 --2- 172.21.15.5:0/1896527354 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x559283c8fc10 0x559283c92000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:46:59.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:59.510+0000 7f4251000640 1 --2- 172.21.15.5:0/1896527354 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x559283c8fc10 0x559283c92000 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=48f9bc66639a3a95 server_cookie=bd34b1e8284835fa in_seq=0 out_seq=0 2024-09-10T09:46:59.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:59.510+0000 7f4233400640 10 client.27157 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T09:46:59.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:59.510+0000 7f4233400640 1 -- 172.21.15.5:0/1896527354 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559283be8950 con 0x559283c8fc10 2024-09-10T09:46:59.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:59.510+0000 7f4233400640 10 client.27157 handle_command_reply: tid=0 2024-09-10T09:46:59.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:59.510+0000 7f4256e1f740 1 client.27157 shutdown 2024-09-10T09:46:59.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:59.511+0000 7f4256e1f740 1 -- 172.21.15.5:0/1896527354 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559283c74c70 msgr2=0x559283c7cd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:46:59.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:59.511+0000 7f4256e1f740 1 --2- 172.21.15.5:0/1896527354 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559283c74c70 0x559283c7cd70 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f4240004b70 tx=0x7f4240000bc0 comp rx=0 tx=0).stop 2024-09-10T09:46:59.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:59.511+0000 7f4256e1f740 1 -- 172.21.15.5:0/1896527354 shutdown_connections 2024-09-10T09:46:59.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:59.511+0000 7f4256e1f740 1 --2- 172.21.15.5:0/1896527354 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x559283c8fc10 0x559283c92000 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:59.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:59.511+0000 7f4256e1f740 1 --2- 172.21.15.5:0/1896527354 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559283c7fe50 0x559283c7d7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:59.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:59.511+0000 7f4256e1f740 1 --2- 172.21.15.5:0/1896527354 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559283c75590 0x559283c7d2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:59.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:59.511+0000 7f4256e1f740 1 --2- 172.21.15.5:0/1896527354 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559283c74c70 0x559283c7cd70 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:59.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:59.511+0000 7f4233400640 0 client.27157 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T09:46:59.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:59.511+0000 7f4256e1f740 1 -- 172.21.15.5:0/1896527354 >> 172.21.15.5:0/1896527354 conn(0x559283c71200 msgr2=0x559283c76ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:46:59.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:59.511+0000 7f4256e1f740 1 -- 172.21.15.5:0/1896527354 shutdown_connections 2024-09-10T09:46:59.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:59.511+0000 7f4256e1f740 1 -- 172.21.15.5:0/1896527354 wait complete. 2024-09-10T09:46:59.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:59.511+0000 7f4256e1f740 20 client.27157 trim_cache size 0 max 16384 2024-09-10T09:46:59.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:59.525+0000 7f4256e1f740 1 Processor -- start 2024-09-10T09:46:59.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:59.525+0000 7f4256e1f740 1 -- start start 2024-09-10T09:46:59.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:59.525+0000 7f4256e1f740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559283c74c70 0x559283c7e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:46:59.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:59.525+0000 7f4256e1f740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559283c75590 0x559283c7c540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:46:59.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:59.525+0000 7f4256e1f740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559283c7fe50 0x559283c97630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:46:59.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:59.525+0000 7f4256e1f740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f424c1b10c0 con 0x559283c75590 2024-09-10T09:46:59.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:59.525+0000 7f4256e1f740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f424c1b2c30 con 0x559283c74c70 2024-09-10T09:46:59.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:59.525+0000 7f4256e1f740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f424c1b1790 con 0x559283c7fe50 2024-09-10T09:46:59.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:59.526+0000 7f4251000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559283c75590 0x559283c7c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:46:59.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:59.526+0000 7f4251a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559283c74c70 0x559283c7e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:46:59.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:59.526+0000 7f4251a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559283c74c70 0x559283c7e5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:42146/0 (socket says 172.21.15.5:42146) 2024-09-10T09:46:59.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:59.526+0000 7f4251a00640 1 -- 172.21.15.5:0/78993059 learned_addr learned my addr 172.21.15.5:0/78993059 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:46:59.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:59.526+0000 7f4252400640 1 --2- 172.21.15.5:0/78993059 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559283c7fe50 0x559283c97630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:46:59.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:59.526+0000 7f4251a00640 1 -- 172.21.15.5:0/78993059 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559283c7fe50 msgr2=0x559283c97630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:46:59.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:59.526+0000 7f4251a00640 1 --2- 172.21.15.5:0/78993059 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559283c7fe50 0x559283c97630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:59.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:59.526+0000 7f4251a00640 1 -- 172.21.15.5:0/78993059 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559283c75590 msgr2=0x559283c7c540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:46:59.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:59.526+0000 7f4251a00640 1 --2- 172.21.15.5:0/78993059 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559283c75590 0x559283c7c540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:59.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:59.526+0000 7f4251a00640 1 -- 172.21.15.5:0/78993059 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559283c6c770 con 0x559283c74c70 2024-09-10T09:46:59.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:59.530+0000 7f4251a00640 1 --2- 172.21.15.5:0/78993059 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559283c74c70 0x559283c7e5d0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f424003c870 tx=0x7f424002a400 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eb1990980a65573a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:46:59.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:59.530+0000 7f423e000640 1 -- 172.21.15.5:0/78993059 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4240052070 con 0x559283c74c70 2024-09-10T09:46:59.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:59.531+0000 7f423e000640 1 -- 172.21.15.5:0/78993059 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4240004c60 con 0x559283c74c70 2024-09-10T09:46:59.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:59.531+0000 7f423e000640 1 -- 172.21.15.5:0/78993059 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42400315b0 con 0x559283c74c70 2024-09-10T09:46:59.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:59.531+0000 7f4256e1f740 1 -- 172.21.15.5:0/78993059 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559283c74c70 msgr2=0x559283c7e5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:46:59.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:59.531+0000 7f4256e1f740 1 --2- 172.21.15.5:0/78993059 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559283c74c70 0x559283c7e5d0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f424003c870 tx=0x7f424002a400 comp rx=0 tx=0).stop 2024-09-10T09:46:59.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:59.531+0000 7f4256e1f740 1 -- 172.21.15.5:0/78993059 shutdown_connections 2024-09-10T09:46:59.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:59.531+0000 7f4256e1f740 1 --2- 172.21.15.5:0/78993059 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559283c7fe50 0x559283c97630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:59.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:59.531+0000 7f4256e1f740 1 --2- 172.21.15.5:0/78993059 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559283c75590 0x559283c7c540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:59.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:59.531+0000 7f4256e1f740 1 --2- 172.21.15.5:0/78993059 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559283c74c70 0x559283c7e5d0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:59.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:59.531+0000 7f4256e1f740 1 -- 172.21.15.5:0/78993059 >> 172.21.15.5:0/78993059 conn(0x559283c71200 msgr2=0x559283c70380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:46:59.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:59.531+0000 7f4256e1f740 1 -- 172.21.15.5:0/78993059 shutdown_connections 2024-09-10T09:46:59.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:59.531+0000 7f4256e1f740 1 -- 172.21.15.5:0/78993059 wait complete. 2024-09-10T09:46:59.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:59.532+0000 7f4256e1f740 1 -- start start 2024-09-10T09:46:59.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:59.532+0000 7f4256e1f740 1 Processor -- start 2024-09-10T09:46:59.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:59.532+0000 7f4256e1f740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559283c74c70 0x559283c886b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:46:59.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:59.532+0000 7f4256e1f740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559283c75590 0x559283c83a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:46:59.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:59.532+0000 7f4256e1f740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559283c7fe50 0x559283c841c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:46:59.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:59.532+0000 7f4256e1f740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f424c1b2530 con 0x559283c75590 2024-09-10T09:46:59.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:59.532+0000 7f4256e1f740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f424c1b1e60 con 0x559283c74c70 2024-09-10T09:46:59.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:59.532+0000 7f4256e1f740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f424c1af0a0 con 0x559283c7fe50 2024-09-10T09:46:59.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:59.532+0000 7f4256e1f740 10 client.0 _ll_register_callbacks cb 0x7f4240005de0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:46:59.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:59.533+0000 7f4251000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559283c75590 0x559283c83a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:46:59.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:59.533+0000 7f4251000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559283c75590 0x559283c83a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42082/0 (socket says 172.21.15.5:42082) 2024-09-10T09:46:59.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:59.533+0000 7f4251000640 1 -- 172.21.15.5:0/4287275804 learned_addr learned my addr 172.21.15.5:0/4287275804 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:46:59.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:59.533+0000 7f4251a00640 1 --2- 172.21.15.5:0/4287275804 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559283c74c70 0x559283c886b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:46:59.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:59.533+0000 7f4251000640 1 -- 172.21.15.5:0/4287275804 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559283c7fe50 msgr2=0x559283c841c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T09:46:59.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:59.533+0000 7f4251000640 1 --2- 172.21.15.5:0/4287275804 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559283c7fe50 0x559283c841c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:59.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:59.533+0000 7f4251000640 1 -- 172.21.15.5:0/4287275804 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559283c74c70 msgr2=0x559283c886b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:46:59.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:59.533+0000 7f4251000640 1 --2- 172.21.15.5:0/4287275804 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559283c74c70 0x559283c886b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:59.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:59.533+0000 7f4251000640 1 -- 172.21.15.5:0/4287275804 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559283cf9600 con 0x559283c75590 2024-09-10T09:46:59.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:59.533+0000 7f4251000640 1 --2- 172.21.15.5:0/4287275804 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559283c75590 0x559283c83a30 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f42380cce40 tx=0x7f4238097f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16b76e7cee904016 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:46:59.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:59.533+0000 7f4233400640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-10T09:46:59.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:59.534+0000 7f4233400640 1 -- 172.21.15.5:0/4287275804 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4238003b30 con 0x559283c75590 2024-09-10T09:46:59.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:59.534+0000 7f4233400640 1 -- 172.21.15.5:0/4287275804 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f4238098210 con 0x559283c75590 2024-09-10T09:46:59.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:59.534+0000 7f4256e1f740 1 -- 172.21.15.5:0/4287275804 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4240004c60 con 0x559283c75590 2024-09-10T09:46:59.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:59.534+0000 7f4233400640 1 -- 172.21.15.5:0/4287275804 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f42380cf3e0 con 0x559283c75590 2024-09-10T09:46:59.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:59.534+0000 7f4256e1f740 10 client.17136 fetch_fsmap learned FSMap version 35 2024-09-10T09:46:59.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:59.534+0000 7f4256e1f740 1 -- 172.21.15.5:0/4287275804 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x559283cfa780 con 0x559283c75590 2024-09-10T09:46:59.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:59.534+0000 7f4233400640 1 -- 172.21.15.5:0/4287275804 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4240004c60 con 0x559283c75590 2024-09-10T09:46:59.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:59.534+0000 7f4233400640 1 -- 172.21.15.5:0/4287275804 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 35) ==== 8374+0+0 (secure 0 0 0) 0x7f42380978f0 con 0x559283c75590 2024-09-10T09:46:59.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:59.535+0000 7f4256e1f740 10 client.17136 fetch_fsmap finished waiting for FSMap version 35 2024-09-10T09:46:59.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:59.535+0000 7f4256e1f740 10 client.17136 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T09:46:59.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:59.535+0000 7f4256e1f740 20 client.17136 populate_metadata read hostname 'smithi005' 2024-09-10T09:46:59.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:59.535+0000 7f4256e1f740 1 --2- 172.21.15.5:0/4287275804 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x559283cfaee0 0x559283cfd300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:46:59.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:59.535+0000 7f4256e1f740 4 client.17136 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T09:46:59.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:59.535+0000 7f4256e1f740 1 -- 172.21.15.5:0/4287275804 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559283be2e60 con 0x559283cfaee0 2024-09-10T09:46:59.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:59.535+0000 7f4251a00640 1 --2- 172.21.15.5:0/4287275804 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x559283cfaee0 0x559283cfd300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:46:59.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:59.535+0000 7f4251a00640 1 --2- 172.21.15.5:0/4287275804 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x559283cfaee0 0x559283cfd300 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8ef061025322b063 server_cookie=f83c3db9950394e2 in_seq=0 out_seq=0 2024-09-10T09:46:59.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:59.535+0000 7f4233400640 10 client.17136 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T09:46:59.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:59.535+0000 7f4233400640 1 -- 172.21.15.5:0/4287275804 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559283be2e60 con 0x559283cfaee0 2024-09-10T09:46:59.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:59.536+0000 7f4233400640 10 client.17136 handle_command_reply: tid=0 2024-09-10T09:46:59.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:59.536+0000 7f4256e1f740 1 client.17136 shutdown 2024-09-10T09:46:59.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:59.536+0000 7f4256e1f740 1 -- 172.21.15.5:0/4287275804 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559283c75590 msgr2=0x559283c83a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:46:59.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:59.536+0000 7f4256e1f740 1 --2- 172.21.15.5:0/4287275804 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559283c75590 0x559283c83a30 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f42380cce40 tx=0x7f4238097f60 comp rx=0 tx=0).stop 2024-09-10T09:46:59.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:59.536+0000 7f4256e1f740 1 -- 172.21.15.5:0/4287275804 shutdown_connections 2024-09-10T09:46:59.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:59.536+0000 7f4256e1f740 1 --2- 172.21.15.5:0/4287275804 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x559283cfaee0 0x559283cfd300 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:59.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:59.536+0000 7f4256e1f740 1 --2- 172.21.15.5:0/4287275804 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559283c7fe50 0x559283c841c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:59.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:59.536+0000 7f4256e1f740 1 --2- 172.21.15.5:0/4287275804 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559283c75590 0x559283c83a30 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:59.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:59.536+0000 7f4256e1f740 1 --2- 172.21.15.5:0/4287275804 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559283c74c70 0x559283c886b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:59.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:59.536+0000 7f4233400640 0 client.17136 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T09:46:59.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:59.536+0000 7f4256e1f740 1 -- 172.21.15.5:0/4287275804 >> 172.21.15.5:0/4287275804 conn(0x559283c71200 msgr2=0x559283c766c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:46:59.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:59.536+0000 7f4256e1f740 1 -- 172.21.15.5:0/4287275804 shutdown_connections 2024-09-10T09:46:59.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:59.536+0000 7f4256e1f740 1 -- 172.21.15.5:0/4287275804 wait complete. 2024-09-10T09:46:59.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:59.536+0000 7f4256e1f740 20 client.17136 trim_cache size 0 max 16384 2024-09-10T09:46:59.540 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-10T09:46:59.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:59.537+0000 7f4254400640 1 -- 172.21.15.5:0/1780794928 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f42340a2b10 msgr2=0x7f42340a4fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:46:59.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:59.537+0000 7f4254400640 1 --2- 172.21.15.5:0/1780794928 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f42340a2b10 0x7f42340a4fd0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f4244009370 tx=0x7f4244009240 comp rx=0 tx=0).stop 2024-09-10T09:46:59.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:59.537+0000 7f4254400640 1 -- 172.21.15.5:0/1780794928 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f424c1049a0 msgr2=0x7f424c19abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:46:59.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:59.537+0000 7f4254400640 1 --2- 172.21.15.5:0/1780794928 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f424c1049a0 0x7f424c19abd0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f423800ea60 tx=0x7f423800ef20 comp rx=0 tx=0).stop 2024-09-10T09:46:59.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:59.537+0000 7f4254400640 1 -- 172.21.15.5:0/1780794928 shutdown_connections 2024-09-10T09:46:59.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:59.537+0000 7f4254400640 1 --2- 172.21.15.5:0/1780794928 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f42340a2b10 0x7f42340a4fd0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:59.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:59.537+0000 7f4254400640 1 --2- 172.21.15.5:0/1780794928 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f424c109120 0x7f424c19ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:59.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:59.537+0000 7f4254400640 1 --2- 172.21.15.5:0/1780794928 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f424c1049a0 0x7f424c19abd0 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:59.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:59.537+0000 7f4254400640 1 --2- 172.21.15.5:0/1780794928 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f424c103ff0 0x7f424c19a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:46:59.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:59.537+0000 7f4254400640 1 -- 172.21.15.5:0/1780794928 >> 172.21.15.5:0/1780794928 conn(0x7f424c0ffa90 msgr2=0x7f424c100d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:46:59.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:59.537+0000 7f4254400640 1 -- 172.21.15.5:0/1780794928 shutdown_connections 2024-09-10T09:46:59.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:46:59.537+0000 7f4254400640 1 -- 172.21.15.5:0/1780794928 wait complete. 2024-09-10T09:47:00.076 INFO:tasks.workunit.client.0.smithi005.stdout: 1 506751 6.08 MB/sec execute 281 sec latency 1184.314 ms 2024-09-10T09:47:01.077 INFO:tasks.workunit.client.0.smithi005.stdout: 1 508787 6.11 MB/sec execute 282 sec latency 24.125 ms 2024-09-10T09:47:01.306 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:47:00 smithi005 ceph-mon[30350]: pgmap v653: 129 pgs: 129 active+clean; 198 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 260 KiB/s rd, 2.7 MiB/s wr, 98 op/s 2024-09-10T09:47:01.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:47:00 smithi159 ceph-mon[35513]: pgmap v653: 129 pgs: 129 active+clean; 198 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 260 KiB/s rd, 2.7 MiB/s wr, 98 op/s 2024-09-10T09:47:01.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:47:00 smithi102 ceph-mon[35442]: pgmap v653: 129 pgs: 129 active+clean; 198 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 260 KiB/s rd, 2.7 MiB/s wr, 98 op/s 2024-09-10T09:47:01.488 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-10T09:47:01.489 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-10T09:47:01.916 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:47:02.077 INFO:tasks.workunit.client.0.smithi005.stdout: 1 511657 6.15 MB/sec execute 283 sec latency 12.835 ms 2024-09-10T09:47:02.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:02.486+0000 7fa488a00640 1 -- 172.21.15.5:0/2726769474 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa4841049a0 msgr2=0x7fa4841089f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:47:02.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:02.486+0000 7fa488a00640 1 --2- 172.21.15.5:0/2726769474 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa4841049a0 0x7fa4841089f0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fa470009900 tx=0x7fa47002a3f0 comp rx=0 tx=0).stop 2024-09-10T09:47:02.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:02.486+0000 7fa488a00640 1 -- 172.21.15.5:0/2726769474 shutdown_connections 2024-09-10T09:47:02.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:02.486+0000 7fa488a00640 1 --2- 172.21.15.5:0/2726769474 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa484109120 0x7fa484110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:02.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:02.486+0000 7fa488a00640 1 --2- 172.21.15.5:0/2726769474 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa4841049a0 0x7fa4841089f0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:02.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:02.486+0000 7fa488a00640 1 --2- 172.21.15.5:0/2726769474 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa484103ff0 0x7fa4841043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:02.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:02.486+0000 7fa488a00640 1 -- 172.21.15.5:0/2726769474 >> 172.21.15.5:0/2726769474 conn(0x7fa4840ffa90 msgr2=0x7fa484101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:47:02.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:02.486+0000 7fa488a00640 1 -- 172.21.15.5:0/2726769474 shutdown_connections 2024-09-10T09:47:02.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:02.486+0000 7fa488a00640 1 -- 172.21.15.5:0/2726769474 wait complete. 2024-09-10T09:47:02.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:02.487+0000 7fa488a00640 1 Processor -- start 2024-09-10T09:47:02.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:02.487+0000 7fa488a00640 1 -- start start 2024-09-10T09:47:02.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:02.488+0000 7fa488a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa484103ff0 0x7fa48419a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:47:02.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:02.488+0000 7fa488a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa4841049a0 0x7fa48419ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:47:02.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:02.488+0000 7fa488a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa484109120 0x7fa48419ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:47:02.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:02.488+0000 7fa488a00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa484114170 con 0x7fa484103ff0 2024-09-10T09:47:02.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:02.488+0000 7fa488a00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fa484113ff0 con 0x7fa4841049a0 2024-09-10T09:47:02.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:02.488+0000 7fa488a00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fa4841142f0 con 0x7fa484109120 2024-09-10T09:47:02.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:02.488+0000 7fa481400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa4841049a0 0x7fa48419ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:47:02.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:02.488+0000 7fa481e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa484103ff0 0x7fa48419a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:47:02.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:02.488+0000 7fa481400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa4841049a0 0x7fa48419ab90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:49208/0 (socket says 172.21.15.5:49208) 2024-09-10T09:47:02.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:02.488+0000 7fa482800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa484109120 0x7fa48419ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:47:02.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:02.488+0000 7fa481400640 1 -- 172.21.15.5:0/2952525391 learned_addr learned my addr 172.21.15.5:0/2952525391 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:47:02.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:02.489+0000 7fa481400640 1 -- 172.21.15.5:0/2952525391 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa484109120 msgr2=0x7fa48419ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:47:02.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:02.489+0000 7fa481400640 1 --2- 172.21.15.5:0/2952525391 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa484109120 0x7fa48419ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:02.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:02.489+0000 7fa481400640 1 -- 172.21.15.5:0/2952525391 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa484103ff0 msgr2=0x7fa48419a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:47:02.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:02.489+0000 7fa481400640 1 --2- 172.21.15.5:0/2952525391 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa484103ff0 0x7fa48419a650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:02.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:02.489+0000 7fa481400640 1 -- 172.21.15.5:0/2952525391 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa48419f600 con 0x7fa4841049a0 2024-09-10T09:47:02.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:02.489+0000 7fa482800640 1 --2- 172.21.15.5:0/2952525391 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa484109120 0x7fa48419ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:47:02.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:02.489+0000 7fa481e00640 1 --2- 172.21.15.5:0/2952525391 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa484103ff0 0x7fa48419a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:47:02.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:02.489+0000 7fa481400640 1 --2- 172.21.15.5:0/2952525391 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa4841049a0 0x7fa48419ab90 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fa470004770 tx=0x7fa470003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:47:02.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:02.489+0000 7fa47aa00640 1 -- 172.21.15.5:0/2952525391 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa47003f070 con 0x7fa4841049a0 2024-09-10T09:47:02.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:02.489+0000 7fa488a00640 1 -- 172.21.15.5:0/2952525391 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa48419f890 con 0x7fa4841049a0 2024-09-10T09:47:02.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:02.489+0000 7fa47aa00640 1 -- 172.21.15.5:0/2952525391 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa470031ca0 con 0x7fa4841049a0 2024-09-10T09:47:02.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:02.489+0000 7fa47aa00640 1 -- 172.21.15.5:0/2952525391 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa470035030 con 0x7fa4841049a0 2024-09-10T09:47:02.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:02.490+0000 7fa488a00640 1 -- 172.21.15.5:0/2952525391 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4841a71d0 con 0x7fa4841049a0 2024-09-10T09:47:02.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:02.492+0000 7fa48b62d740 1 Processor -- start 2024-09-10T09:47:02.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:02.493+0000 7fa48b62d740 1 -- start start 2024-09-10T09:47:02.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:02.493+0000 7fa47aa00640 1 -- 172.21.15.5:0/2952525391 <== mon.2 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7fa470042070 con 0x7fa4841049a0 2024-09-10T09:47:02.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:02.493+0000 7fa48b62d740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560525d22ac0 0x560525d22ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:47:02.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:02.493+0000 7fa48b62d740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560525d233e0 0x560525d2d760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:47:02.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:02.493+0000 7fa48b62d740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560525d2dca0 0x560525d30090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:47:02.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:02.493+0000 7fa48b62d740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x560525c89290 con 0x560525d22ac0 2024-09-10T09:47:02.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:02.493+0000 7fa48b62d740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x560525c30b40 con 0x560525d233e0 2024-09-10T09:47:02.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:02.493+0000 7fa48b62d740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x560525c85250 con 0x560525d2dca0 2024-09-10T09:47:02.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:02.493+0000 7fa481e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560525d233e0 0x560525d2d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:47:02.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:02.493+0000 7fa481e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560525d233e0 0x560525d2d760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:49210/0 (socket says 172.21.15.5:49210) 2024-09-10T09:47:02.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:02.494+0000 7fa481e00640 1 -- 172.21.15.5:0/3931158541 learned_addr learned my addr 172.21.15.5:0/3931158541 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:47:02.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:02.494+0000 7fa482800640 1 --2- 172.21.15.5:0/3931158541 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560525d22ac0 0x560525d22ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:47:02.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:02.494+0000 7fa481400640 1 --2- 172.21.15.5:0/3931158541 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560525d2dca0 0x560525d30090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:47:02.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:02.494+0000 7fa481e00640 1 -- 172.21.15.5:0/3931158541 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560525d2dca0 msgr2=0x560525d30090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:47:02.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:02.494+0000 7fa481e00640 1 --2- 172.21.15.5:0/3931158541 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560525d2dca0 0x560525d30090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:02.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:02.494+0000 7fa481e00640 1 -- 172.21.15.5:0/3931158541 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560525d22ac0 msgr2=0x560525d22ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:47:02.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:02.494+0000 7fa481e00640 1 --2- 172.21.15.5:0/3931158541 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560525d22ac0 0x560525d22ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:02.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:02.494+0000 7fa481e00640 1 -- 172.21.15.5:0/3931158541 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5605259fa6a0 con 0x560525d233e0 2024-09-10T09:47:02.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:02.494+0000 7fa482800640 1 --2- 172.21.15.5:0/3931158541 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560525d22ac0 0x560525d22ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:47:02.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:02.495+0000 7fa481e00640 1 --2- 172.21.15.5:0/3931158541 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560525d233e0 0x560525d2d760 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fa4740115f0 tx=0x7fa474011ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=efbc10c608f5f347 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:47:02.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:02.495+0000 7fa46f000640 1 -- 172.21.15.5:0/3931158541 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa47401e070 con 0x560525d233e0 2024-09-10T09:47:02.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:02.495+0000 7fa46f000640 1 -- 172.21.15.5:0/3931158541 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa47400fb20 con 0x560525d233e0 2024-09-10T09:47:02.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:02.495+0000 7fa46f000640 1 -- 172.21.15.5:0/3931158541 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa474019c40 con 0x560525d233e0 2024-09-10T09:47:02.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:02.495+0000 7fa47aa00640 1 --2- 172.21.15.5:0/2952525391 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fa4680a2c00 0x7fa4680a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:47:02.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:02.495+0000 7fa47aa00640 1 -- 172.21.15.5:0/2952525391 <== mon.2 v2:172.21.15.102:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fa4700ef730 con 0x7fa4841049a0 2024-09-10T09:47:02.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:02.495+0000 7fa482800640 1 --2- 172.21.15.5:0/2952525391 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fa4680a2c00 0x7fa4680a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:47:02.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:02.496+0000 7fa48b62d740 1 -- 172.21.15.5:0/3931158541 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560525d233e0 msgr2=0x560525d2d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:47:02.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:02.496+0000 7fa48b62d740 1 --2- 172.21.15.5:0/3931158541 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560525d233e0 0x560525d2d760 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fa4740115f0 tx=0x7fa474011ab0 comp rx=0 tx=0).stop 2024-09-10T09:47:02.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:02.496+0000 7fa48b62d740 1 -- 172.21.15.5:0/3931158541 shutdown_connections 2024-09-10T09:47:02.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:02.496+0000 7fa48b62d740 1 --2- 172.21.15.5:0/3931158541 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560525d2dca0 0x560525d30090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:02.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:02.496+0000 7fa48b62d740 1 --2- 172.21.15.5:0/3931158541 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560525d233e0 0x560525d2d760 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:02.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:02.496+0000 7fa48b62d740 1 --2- 172.21.15.5:0/3931158541 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560525d22ac0 0x560525d22ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:02.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:02.496+0000 7fa48b62d740 1 -- 172.21.15.5:0/3931158541 >> 172.21.15.5:0/3931158541 conn(0x560525d1f050 msgr2=0x560525d1f430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:47:02.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:02.496+0000 7fa48b62d740 1 -- 172.21.15.5:0/3931158541 shutdown_connections 2024-09-10T09:47:02.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:02.496+0000 7fa482800640 1 --2- 172.21.15.5:0/2952525391 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fa4680a2c00 0x7fa4680a50c0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fa47c005fd0 tx=0x7fa47c0096a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:47:02.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:02.496+0000 7fa48b62d740 1 -- 172.21.15.5:0/3931158541 wait complete. 2024-09-10T09:47:02.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:02.497+0000 7fa48b62d740 1 -- start start 2024-09-10T09:47:02.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:02.497+0000 7fa48b62d740 1 Processor -- start 2024-09-10T09:47:02.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:02.498+0000 7fa48b62d740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560525d22ac0 0x560525d2d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:47:02.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:02.498+0000 7fa48b62d740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560525d233e0 0x560525d2a870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:47:02.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:02.498+0000 7fa48b62d740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560525d2dca0 0x560525d2adb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:47:02.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:02.498+0000 7fa48b62d740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x560525d30910 con 0x560525d233e0 2024-09-10T09:47:02.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:02.498+0000 7fa48b62d740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x560525c43fa0 con 0x560525d22ac0 2024-09-10T09:47:02.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:02.498+0000 7fa48b62d740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x560525d30a90 con 0x560525d2dca0 2024-09-10T09:47:02.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:02.498+0000 7fa48b62d740 10 client.0 _ll_register_callbacks cb 0x560525997ba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:47:02.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:02.498+0000 7fa482800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560525d2dca0 0x560525d2adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:47:02.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:02.498+0000 7fa481e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560525d22ac0 0x560525d2d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:47:02.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:02.498+0000 7fa481400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560525d233e0 0x560525d2a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:47:02.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:02.498+0000 7fa482800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560525d2dca0 0x560525d2adb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:46510/0 (socket says 172.21.15.5:46510) 2024-09-10T09:47:02.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:02.498+0000 7fa482800640 1 -- 172.21.15.5:0/3546495718 learned_addr learned my addr 172.21.15.5:0/3546495718 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:47:02.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:02.498+0000 7fa481e00640 1 -- 172.21.15.5:0/3546495718 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560525d2dca0 msgr2=0x560525d2adb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:47:02.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:02.498+0000 7fa481e00640 1 --2- 172.21.15.5:0/3546495718 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560525d2dca0 0x560525d2adb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:02.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:02.498+0000 7fa481e00640 1 -- 172.21.15.5:0/3546495718 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560525d233e0 msgr2=0x560525d2a870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:47:02.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:02.498+0000 7fa481e00640 1 --2- 172.21.15.5:0/3546495718 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560525d233e0 0x560525d2a870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:02.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:02.498+0000 7fa481e00640 1 -- 172.21.15.5:0/3546495718 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560525d2b5b0 con 0x560525d22ac0 2024-09-10T09:47:02.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:02.498+0000 7fa482800640 1 --2- 172.21.15.5:0/3546495718 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560525d2dca0 0x560525d2adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:47:02.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:02.499+0000 7fa481e00640 1 --2- 172.21.15.5:0/3546495718 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560525d22ac0 0x560525d2d5c0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fa4740115c0 tx=0x7fa47400f850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cc7d64364689fbda server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:47:02.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:02.499+0000 7fa463400640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-09-10T09:47:02.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:02.499+0000 7fa463400640 1 -- 172.21.15.5:0/3546495718 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa474027060 con 0x560525d22ac0 2024-09-10T09:47:02.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:02.499+0000 7fa463400640 1 -- 172.21.15.5:0/3546495718 <== mon.2 v2:172.21.15.102:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fa4740199c0 con 0x560525d22ac0 2024-09-10T09:47:02.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:02.499+0000 7fa48b62d740 1 -- 172.21.15.5:0/3546495718 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa47400fb20 con 0x560525d22ac0 2024-09-10T09:47:02.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:02.499+0000 7fa463400640 1 -- 172.21.15.5:0/3546495718 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fa47401e040 con 0x560525d22ac0 2024-09-10T09:47:02.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:02.499+0000 7fa48b62d740 10 client.27143 fetch_fsmap learned FSMap version 35 2024-09-10T09:47:02.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:02.499+0000 7fa48b62d740 1 -- 172.21.15.5:0/3546495718 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x560525d3c820 con 0x560525d22ac0 2024-09-10T09:47:02.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:02.500+0000 7fa463400640 1 -- 172.21.15.5:0/3546495718 <== mon.2 v2:172.21.15.102:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa47400fb20 con 0x560525d22ac0 2024-09-10T09:47:02.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:02.500+0000 7fa463400640 1 -- 172.21.15.5:0/3546495718 <== mon.2 v2:172.21.15.102:3300/0 5 ==== fsmap(e 35) ==== 8374+0+0 (secure 0 0 0) 0x7fa474005070 con 0x560525d22ac0 2024-09-10T09:47:02.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:02.500+0000 7fa48b62d740 10 client.27143 fetch_fsmap finished waiting for FSMap version 35 2024-09-10T09:47:02.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:02.500+0000 7fa48b62d740 10 client.27143 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T09:47:02.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:02.500+0000 7fa48b62d740 20 client.27143 populate_metadata read hostname 'smithi005' 2024-09-10T09:47:02.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:02.500+0000 7fa48b62d740 1 --2- 172.21.15.5:0/3546495718 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x560525d3dce0 0x560525d400d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:47:02.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:02.500+0000 7fa48b62d740 4 client.27143 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:47:02.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:02.500+0000 7fa48b62d740 1 -- 172.21.15.5:0/3546495718 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560525c96800 con 0x560525d3dce0 2024-09-10T09:47:02.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:02.500+0000 7fa481400640 1 --2- 172.21.15.5:0/3546495718 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x560525d3dce0 0x560525d400d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:47:02.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:02.501+0000 7fa481400640 1 --2- 172.21.15.5:0/3546495718 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x560525d3dce0 0x560525d400d0 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f1a024edcb3440c8 server_cookie=23a70e8ccabcc1fb in_seq=0 out_seq=0 2024-09-10T09:47:02.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:02.501+0000 7fa463400640 10 client.27143 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T09:47:02.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:02.501+0000 7fa463400640 1 -- 172.21.15.5:0/3546495718 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560525c96800 con 0x560525d3dce0 2024-09-10T09:47:02.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:02.501+0000 7fa463400640 10 client.27143 handle_command_reply: tid=0 2024-09-10T09:47:02.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:02.502+0000 7fa48b62d740 1 client.27143 shutdown 2024-09-10T09:47:02.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:02.502+0000 7fa48b62d740 1 -- 172.21.15.5:0/3546495718 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560525d22ac0 msgr2=0x560525d2d5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:47:02.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:02.502+0000 7fa48b62d740 1 --2- 172.21.15.5:0/3546495718 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560525d22ac0 0x560525d2d5c0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fa4740115c0 tx=0x7fa47400f850 comp rx=0 tx=0).stop 2024-09-10T09:47:02.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:02.502+0000 7fa48b62d740 1 -- 172.21.15.5:0/3546495718 shutdown_connections 2024-09-10T09:47:02.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:02.502+0000 7fa48b62d740 1 --2- 172.21.15.5:0/3546495718 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x560525d3dce0 0x560525d400d0 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:02.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:02.502+0000 7fa48b62d740 1 --2- 172.21.15.5:0/3546495718 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560525d2dca0 0x560525d2adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:02.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:02.502+0000 7fa48b62d740 1 --2- 172.21.15.5:0/3546495718 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560525d233e0 0x560525d2a870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:02.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:02.502+0000 7fa48b62d740 1 --2- 172.21.15.5:0/3546495718 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560525d22ac0 0x560525d2d5c0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:02.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:02.502+0000 7fa48b62d740 1 -- 172.21.15.5:0/3546495718 >> 172.21.15.5:0/3546495718 conn(0x560525d1f050 msgr2=0x560525d1e1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:47:02.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:02.502+0000 7fa463400640 0 client.27143 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T09:47:02.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:02.502+0000 7fa48b62d740 1 -- 172.21.15.5:0/3546495718 shutdown_connections 2024-09-10T09:47:02.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:02.502+0000 7fa48b62d740 1 -- 172.21.15.5:0/3546495718 wait complete. 2024-09-10T09:47:02.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:02.503+0000 7fa48b62d740 20 client.27143 trim_cache size 0 max 16384 2024-09-10T09:47:02.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:02.524+0000 7fa48b62d740 1 Processor -- start 2024-09-10T09:47:02.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:02.524+0000 7fa48b62d740 1 -- start start 2024-09-10T09:47:02.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:02.525+0000 7fa48b62d740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560525d22ac0 0x560525d457c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:47:02.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:02.525+0000 7fa48b62d740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560525d233e0 0x560525d45d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:47:02.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:02.525+0000 7fa48b62d740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560525d2dca0 0x560525d24510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:47:02.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:02.525+0000 7fa48b62d740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa4841b1070 con 0x560525d22ac0 2024-09-10T09:47:02.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:02.525+0000 7fa48b62d740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fa4841b2be0 con 0x560525d2dca0 2024-09-10T09:47:02.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:02.525+0000 7fa48b62d740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fa4841b1740 con 0x560525d233e0 2024-09-10T09:47:02.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:02.525+0000 7fa481e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560525d22ac0 0x560525d457c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:47:02.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:02.525+0000 7fa481400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560525d233e0 0x560525d45d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:47:02.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:02.525+0000 7fa481400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560525d233e0 0x560525d45d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:46516/0 (socket says 172.21.15.5:46516) 2024-09-10T09:47:02.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:02.525+0000 7fa481400640 1 -- 172.21.15.5:0/2901330216 learned_addr learned my addr 172.21.15.5:0/2901330216 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:47:02.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:02.525+0000 7fa481e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560525d22ac0 0x560525d457c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48576/0 (socket says 172.21.15.5:48576) 2024-09-10T09:47:02.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:02.525+0000 7fa482800640 1 --2- 172.21.15.5:0/2901330216 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560525d2dca0 0x560525d24510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:47:02.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:02.525+0000 7fa481400640 1 -- 172.21.15.5:0/2901330216 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560525d2dca0 msgr2=0x560525d24510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:47:02.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:02.525+0000 7fa481400640 1 --2- 172.21.15.5:0/2901330216 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560525d2dca0 0x560525d24510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:02.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:02.525+0000 7fa481400640 1 -- 172.21.15.5:0/2901330216 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560525d22ac0 msgr2=0x560525d457c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:47:02.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:02.525+0000 7fa481400640 1 --2- 172.21.15.5:0/2901330216 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560525d22ac0 0x560525d457c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:02.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:02.526+0000 7fa481400640 1 -- 172.21.15.5:0/2901330216 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560525d1eaf0 con 0x560525d233e0 2024-09-10T09:47:02.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:02.526+0000 7fa482800640 1 --2- 172.21.15.5:0/2901330216 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560525d2dca0 0x560525d24510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:47:02.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:02.526+0000 7fa481e00640 1 --2- 172.21.15.5:0/2901330216 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560525d22ac0 0x560525d457c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:47:02.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:02.526+0000 7fa481400640 1 --2- 172.21.15.5:0/2901330216 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560525d233e0 0x560525d45d00 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fa470000c00 tx=0x7fa470005c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dd0aad45dbc45b75 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:47:02.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:02.526+0000 7fa46e600640 1 -- 172.21.15.5:0/2901330216 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa470049040 con 0x560525d233e0 2024-09-10T09:47:02.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:02.526+0000 7fa46e600640 1 -- 172.21.15.5:0/2901330216 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa4700a6040 con 0x560525d233e0 2024-09-10T09:47:02.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:02.526+0000 7fa46e600640 1 -- 172.21.15.5:0/2901330216 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4700ef3c0 con 0x560525d233e0 2024-09-10T09:47:02.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:02.527+0000 7fa48b62d740 1 -- 172.21.15.5:0/2901330216 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560525d233e0 msgr2=0x560525d45d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:47:02.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:02.527+0000 7fa48b62d740 1 --2- 172.21.15.5:0/2901330216 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560525d233e0 0x560525d45d00 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fa470000c00 tx=0x7fa470005c00 comp rx=0 tx=0).stop 2024-09-10T09:47:02.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:02.527+0000 7fa48b62d740 1 -- 172.21.15.5:0/2901330216 shutdown_connections 2024-09-10T09:47:02.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:02.527+0000 7fa48b62d740 1 --2- 172.21.15.5:0/2901330216 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560525d2dca0 0x560525d24510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:02.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:02.527+0000 7fa48b62d740 1 --2- 172.21.15.5:0/2901330216 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560525d233e0 0x560525d45d00 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:02.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:02.527+0000 7fa48b62d740 1 --2- 172.21.15.5:0/2901330216 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560525d22ac0 0x560525d457c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:02.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:02.527+0000 7fa48b62d740 1 -- 172.21.15.5:0/2901330216 >> 172.21.15.5:0/2901330216 conn(0x560525d1f050 msgr2=0x560525d29140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:47:02.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:02.527+0000 7fa48b62d740 1 -- 172.21.15.5:0/2901330216 shutdown_connections 2024-09-10T09:47:02.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:02.527+0000 7fa48b62d740 1 -- 172.21.15.5:0/2901330216 wait complete. 2024-09-10T09:47:02.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:02.528+0000 7fa48b62d740 1 -- start start 2024-09-10T09:47:02.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:02.528+0000 7fa48b62d740 1 Processor -- start 2024-09-10T09:47:02.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:02.528+0000 7fa48b62d740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560525d22ac0 0x560525d36090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:47:02.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:02.528+0000 7fa48b62d740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560525d233e0 0x560525d365d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:47:02.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:02.528+0000 7fa48b62d740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560525d2dca0 0x560525d31660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:47:02.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:02.528+0000 7fa48b62d740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa4841b24e0 con 0x560525d233e0 2024-09-10T09:47:02.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:02.528+0000 7fa48b62d740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fa4841b1e10 con 0x560525d22ac0 2024-09-10T09:47:02.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:02.528+0000 7fa48b62d740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fa4841af050 con 0x560525d2dca0 2024-09-10T09:47:02.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:02.528+0000 7fa48b62d740 10 client.0 _ll_register_callbacks cb 0x7fa47401c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:47:02.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:02.529+0000 7fa482800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560525d2dca0 0x560525d31660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:47:02.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:02.529+0000 7fa481e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560525d22ac0 0x560525d36090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:47:02.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:02.529+0000 7fa481e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560525d22ac0 0x560525d36090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:49238/0 (socket says 172.21.15.5:49238) 2024-09-10T09:47:02.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:02.529+0000 7fa481e00640 1 -- 172.21.15.5:0/2836365301 learned_addr learned my addr 172.21.15.5:0/2836365301 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:47:02.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:02.529+0000 7fa482800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560525d2dca0 0x560525d31660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:46518/0 (socket says 172.21.15.5:46518) 2024-09-10T09:47:02.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:02.529+0000 7fa481400640 1 --2- 172.21.15.5:0/2836365301 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560525d233e0 0x560525d365d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:47:02.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:02.529+0000 7fa482800640 1 -- 172.21.15.5:0/2836365301 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560525d22ac0 msgr2=0x560525d36090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:47:02.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:02.529+0000 7fa482800640 1 --2- 172.21.15.5:0/2836365301 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560525d22ac0 0x560525d36090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:02.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:02.529+0000 7fa482800640 1 -- 172.21.15.5:0/2836365301 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560525d233e0 msgr2=0x560525d365d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:47:02.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:02.529+0000 7fa482800640 1 --2- 172.21.15.5:0/2836365301 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560525d233e0 0x560525d365d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:02.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:02.529+0000 7fa482800640 1 -- 172.21.15.5:0/2836365301 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560525d31e90 con 0x560525d2dca0 2024-09-10T09:47:02.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:02.529+0000 7fa482800640 1 --2- 172.21.15.5:0/2836365301 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560525d2dca0 0x560525d31660 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fa47c010500 tx=0x7fa47c004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=29da0261a83d42e9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:47:02.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:02.529+0000 7fa463400640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-09-10T09:47:02.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:02.530+0000 7fa463400640 1 -- 172.21.15.5:0/2836365301 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa47c020db0 con 0x560525d2dca0 2024-09-10T09:47:02.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:02.530+0000 7fa463400640 1 -- 172.21.15.5:0/2836365301 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fa47c003970 con 0x560525d2dca0 2024-09-10T09:47:02.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:02.530+0000 7fa48b62d740 1 -- 172.21.15.5:0/2836365301 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa4700a6040 con 0x560525d2dca0 2024-09-10T09:47:02.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:02.530+0000 7fa463400640 1 -- 172.21.15.5:0/2836365301 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fa47c003b70 con 0x560525d2dca0 2024-09-10T09:47:02.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:02.530+0000 7fa48b62d740 10 client.27178 fetch_fsmap learned FSMap version 35 2024-09-10T09:47:02.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:02.530+0000 7fa48b62d740 1 -- 172.21.15.5:0/2836365301 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x560525da8490 con 0x560525d2dca0 2024-09-10T09:47:02.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:02.530+0000 7fa463400640 1 -- 172.21.15.5:0/2836365301 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa4700a6040 con 0x560525d2dca0 2024-09-10T09:47:02.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:02.531+0000 7fa463400640 1 -- 172.21.15.5:0/2836365301 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 35) ==== 8374+0+0 (secure 0 0 0) 0x7fa47c01d560 con 0x560525d2dca0 2024-09-10T09:47:02.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:02.531+0000 7fa48b62d740 10 client.27178 fetch_fsmap finished waiting for FSMap version 35 2024-09-10T09:47:02.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:02.531+0000 7fa48b62d740 10 client.27178 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T09:47:02.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:02.531+0000 7fa48b62d740 20 client.27178 populate_metadata read hostname 'smithi005' 2024-09-10T09:47:02.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:02.531+0000 7fa48b62d740 1 --2- 172.21.15.5:0/2836365301 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x560525da8c60 0x560525dab080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:47:02.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:02.531+0000 7fa48b62d740 4 client.27178 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-10T09:47:02.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:02.531+0000 7fa48b62d740 1 -- 172.21.15.5:0/2836365301 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x560525c90d10 con 0x560525da8c60 2024-09-10T09:47:02.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:02.531+0000 7fa481e00640 1 --2- 172.21.15.5:0/2836365301 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x560525da8c60 0x560525dab080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:47:02.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:02.532+0000 7fa481e00640 1 --2- 172.21.15.5:0/2836365301 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x560525da8c60 0x560525dab080 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2d29a05e5b196871 server_cookie=c8795bd10d13db8a in_seq=0 out_seq=0 2024-09-10T09:47:02.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:02.532+0000 7fa463400640 10 client.27178 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T09:47:02.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:02.533+0000 7fa463400640 1 -- 172.21.15.5:0/2836365301 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x560525c90d10 con 0x560525da8c60 2024-09-10T09:47:02.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:02.533+0000 7fa463400640 10 client.27178 handle_command_reply: tid=0 2024-09-10T09:47:02.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:02.533+0000 7fa48b62d740 1 client.27178 shutdown 2024-09-10T09:47:02.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:02.533+0000 7fa48b62d740 1 -- 172.21.15.5:0/2836365301 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560525d2dca0 msgr2=0x560525d31660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:47:02.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:02.533+0000 7fa48b62d740 1 --2- 172.21.15.5:0/2836365301 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560525d2dca0 0x560525d31660 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fa47c010500 tx=0x7fa47c004a40 comp rx=0 tx=0).stop 2024-09-10T09:47:02.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:02.533+0000 7fa48b62d740 1 -- 172.21.15.5:0/2836365301 shutdown_connections 2024-09-10T09:47:02.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:02.533+0000 7fa48b62d740 1 --2- 172.21.15.5:0/2836365301 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x560525da8c60 0x560525dab080 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:02.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:02.533+0000 7fa48b62d740 1 --2- 172.21.15.5:0/2836365301 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560525d2dca0 0x560525d31660 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:02.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:02.533+0000 7fa48b62d740 1 --2- 172.21.15.5:0/2836365301 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560525d233e0 0x560525d365d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:02.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:02.533+0000 7fa463400640 0 client.27178 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T09:47:02.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:02.533+0000 7fa48b62d740 1 --2- 172.21.15.5:0/2836365301 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560525d22ac0 0x560525d36090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:02.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:02.533+0000 7fa48b62d740 1 -- 172.21.15.5:0/2836365301 >> 172.21.15.5:0/2836365301 conn(0x560525d1f050 msgr2=0x560525d25db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:47:02.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:02.533+0000 7fa48b62d740 1 -- 172.21.15.5:0/2836365301 shutdown_connections 2024-09-10T09:47:02.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:02.533+0000 7fa48b62d740 1 -- 172.21.15.5:0/2836365301 wait complete. 2024-09-10T09:47:02.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:02.533+0000 7fa48b62d740 20 client.27178 trim_cache size 0 max 16384 2024-09-10T09:47:02.537 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-10T09:47:02.537 INFO:teuthology.orchestra.run.smithi005.stdout: "return_code": 0, 2024-09-10T09:47:02.537 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_tag": "8a32a92d-50b6-4ee6-b59e-d505b9c86d52", 2024-09-10T09:47:02.537 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": "asynchronous" 2024-09-10T09:47:02.537 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-10T09:47:02.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:02.534+0000 7fa488a00640 1 -- 172.21.15.5:0/2952525391 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fa4680a2c00 msgr2=0x7fa4680a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:47:02.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:02.534+0000 7fa488a00640 1 --2- 172.21.15.5:0/2952525391 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fa4680a2c00 0x7fa4680a50c0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fa47c005fd0 tx=0x7fa47c0096a0 comp rx=0 tx=0).stop 2024-09-10T09:47:02.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:02.534+0000 7fa488a00640 1 -- 172.21.15.5:0/2952525391 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa4841049a0 msgr2=0x7fa48419ab90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:47:02.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:02.534+0000 7fa488a00640 1 --2- 172.21.15.5:0/2952525391 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa4841049a0 0x7fa48419ab90 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fa470004770 tx=0x7fa470003c10 comp rx=0 tx=0).stop 2024-09-10T09:47:02.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:02.534+0000 7fa488a00640 1 -- 172.21.15.5:0/2952525391 shutdown_connections 2024-09-10T09:47:02.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:02.534+0000 7fa488a00640 1 --2- 172.21.15.5:0/2952525391 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fa4680a2c00 0x7fa4680a50c0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:02.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:02.534+0000 7fa488a00640 1 --2- 172.21.15.5:0/2952525391 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa484109120 0x7fa48419ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:02.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:02.534+0000 7fa488a00640 1 --2- 172.21.15.5:0/2952525391 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa4841049a0 0x7fa48419ab90 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:02.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:02.534+0000 7fa488a00640 1 --2- 172.21.15.5:0/2952525391 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa484103ff0 0x7fa48419a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:02.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:02.534+0000 7fa488a00640 1 -- 172.21.15.5:0/2952525391 >> 172.21.15.5:0/2952525391 conn(0x7fa4840ffa90 msgr2=0x7fa484100d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:47:02.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:02.534+0000 7fa488a00640 1 -- 172.21.15.5:0/2952525391 shutdown_connections 2024-09-10T09:47:02.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:02.534+0000 7fa488a00640 1 -- 172.21.15.5:0/2952525391 wait complete. 2024-09-10T09:47:03.077 INFO:tasks.workunit.client.0.smithi005.stdout: 1 513671 6.15 MB/sec execute 284 sec latency 20.186 ms 2024-09-10T09:47:03.306 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:47:02 smithi005 ceph-mon[30350]: pgmap v654: 129 pgs: 129 active+clean; 197 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 251 KiB/s rd, 2.3 MiB/s wr, 96 op/s 2024-09-10T09:47:03.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:47:02 smithi159 ceph-mon[35513]: pgmap v654: 129 pgs: 129 active+clean; 197 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 251 KiB/s rd, 2.3 MiB/s wr, 96 op/s 2024-09-10T09:47:03.381 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T09:47:03.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:47:02 smithi102 ceph-mon[35442]: pgmap v654: 129 pgs: 129 active+clean; 197 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 251 KiB/s rd, 2.3 MiB/s wr, 96 op/s 2024-09-10T09:47:03.805 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:47:04.077 INFO:tasks.workunit.client.0.smithi005.stdout: 1 515197 6.17 MB/sec execute 285 sec latency 271.832 ms 2024-09-10T09:47:04.174 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:47:03 smithi005 ceph-mon[30350]: scrub queued for path: / 2024-09-10T09:47:04.174 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:47:03 smithi005 ceph-mon[30350]: scrub summary: idle+waiting paths [/] 2024-09-10T09:47:04.174 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:47:03 smithi005 ceph-mon[30350]: scrub summary: active paths [/] 2024-09-10T09:47:04.174 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:47:03 smithi005 ceph-mon[30350]: scrub summary: active paths [/] 2024-09-10T09:47:04.174 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:47:03 smithi005 ceph-mon[30350]: scrub summary: idle+waiting paths [/] 2024-09-10T09:47:04.174 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:47:03 smithi005 ceph-mon[30350]: scrub summary: idle+waiting paths [/] 2024-09-10T09:47:04.355 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:47:03 smithi159 ceph-mon[35513]: scrub queued for path: / 2024-09-10T09:47:04.355 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:47:03 smithi159 ceph-mon[35513]: scrub summary: idle+waiting paths [/] 2024-09-10T09:47:04.355 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:47:03 smithi159 ceph-mon[35513]: scrub summary: active paths [/] 2024-09-10T09:47:04.356 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:47:03 smithi159 ceph-mon[35513]: scrub summary: active paths [/] 2024-09-10T09:47:04.356 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:47:03 smithi159 ceph-mon[35513]: scrub summary: idle+waiting paths [/] 2024-09-10T09:47:04.356 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:47:03 smithi159 ceph-mon[35513]: scrub summary: idle+waiting paths [/] 2024-09-10T09:47:04.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:47:03 smithi102 ceph-mon[35442]: scrub queued for path: / 2024-09-10T09:47:04.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:47:03 smithi102 ceph-mon[35442]: scrub summary: idle+waiting paths [/] 2024-09-10T09:47:04.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:47:03 smithi102 ceph-mon[35442]: scrub summary: active paths [/] 2024-09-10T09:47:04.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:47:03 smithi102 ceph-mon[35442]: scrub summary: active paths [/] 2024-09-10T09:47:04.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:47:03 smithi102 ceph-mon[35442]: scrub summary: idle+waiting paths [/] 2024-09-10T09:47:04.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:47:03 smithi102 ceph-mon[35442]: scrub summary: idle+waiting paths [/] 2024-09-10T09:47:04.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:04.398+0000 7fd9afc00640 1 -- 172.21.15.5:0/2645957143 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd9a8101df0 msgr2=0x7fd9a81021d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:47:04.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:04.398+0000 7fd9afc00640 1 --2- 172.21.15.5:0/2645957143 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd9a8101df0 0x7fd9a81021d0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fd99c009900 tx=0x7fd99c02a3f0 comp rx=0 tx=0).stop 2024-09-10T09:47:04.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:04.398+0000 7fd9afc00640 1 -- 172.21.15.5:0/2645957143 shutdown_connections 2024-09-10T09:47:04.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:04.398+0000 7fd9afc00640 1 --2- 172.21.15.5:0/2645957143 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd9a8106f20 0x7fd9a810eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:04.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:04.398+0000 7fd9afc00640 1 --2- 172.21.15.5:0/2645957143 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd9a81027a0 0x7fd9a81067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:04.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:04.398+0000 7fd9afc00640 1 --2- 172.21.15.5:0/2645957143 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd9a8101df0 0x7fd9a81021d0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:04.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:04.398+0000 7fd9afc00640 1 -- 172.21.15.5:0/2645957143 >> 172.21.15.5:0/2645957143 conn(0x7fd9a80fd8b0 msgr2=0x7fd9a80ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:47:04.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:04.398+0000 7fd9afc00640 1 -- 172.21.15.5:0/2645957143 shutdown_connections 2024-09-10T09:47:04.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:04.398+0000 7fd9afc00640 1 -- 172.21.15.5:0/2645957143 wait complete. 2024-09-10T09:47:04.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:04.398+0000 7fd9afc00640 1 Processor -- start 2024-09-10T09:47:04.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:04.403+0000 7fd9afc00640 1 -- start start 2024-09-10T09:47:04.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:04.403+0000 7fd9afc00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd9a8101df0 0x7fd9a81983e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:47:04.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:04.403+0000 7fd9afc00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd9a81027a0 0x7fd9a8198920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:47:04.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:04.403+0000 7fd9afc00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd9a8106f20 0x7fd9a819ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:47:04.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:04.403+0000 7fd9afc00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd9a8111f90 con 0x7fd9a81027a0 2024-09-10T09:47:04.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:04.403+0000 7fd9afc00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fd9a8111e10 con 0x7fd9a8106f20 2024-09-10T09:47:04.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:04.403+0000 7fd9afc00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fd9a8112110 con 0x7fd9a8101df0 2024-09-10T09:47:04.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:04.403+0000 7fd9ad200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd9a8101df0 0x7fd9a81983e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:47:04.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:04.403+0000 7fd9ad200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd9a8101df0 0x7fd9a81983e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:46530/0 (socket says 172.21.15.5:46530) 2024-09-10T09:47:04.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:04.403+0000 7fd9ad200640 1 -- 172.21.15.5:0/2709109367 learned_addr learned my addr 172.21.15.5:0/2709109367 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:47:04.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:04.403+0000 7fd9a7e00640 1 --2- 172.21.15.5:0/2709109367 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd9a81027a0 0x7fd9a8198920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:47:04.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:04.403+0000 7fd9adc00640 1 --2- 172.21.15.5:0/2709109367 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd9a8106f20 0x7fd9a819ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:47:04.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:04.403+0000 7fd9ad200640 1 -- 172.21.15.5:0/2709109367 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd9a8106f20 msgr2=0x7fd9a819ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:47:04.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:04.403+0000 7fd9ad200640 1 --2- 172.21.15.5:0/2709109367 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd9a8106f20 0x7fd9a819ccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:04.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:04.403+0000 7fd9ad200640 1 -- 172.21.15.5:0/2709109367 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd9a81027a0 msgr2=0x7fd9a8198920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:47:04.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:04.403+0000 7fd9ad200640 1 --2- 172.21.15.5:0/2709109367 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd9a81027a0 0x7fd9a8198920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:04.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:04.403+0000 7fd9ad200640 1 -- 172.21.15.5:0/2709109367 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9a819d390 con 0x7fd9a8101df0 2024-09-10T09:47:04.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:04.403+0000 7fd9adc00640 1 --2- 172.21.15.5:0/2709109367 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd9a8106f20 0x7fd9a819ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:47:04.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:04.403+0000 7fd9ad200640 1 --2- 172.21.15.5:0/2709109367 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd9a8101df0 0x7fd9a81983e0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fd99c004770 tx=0x7fd99c003c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:47:04.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:04.403+0000 7fd9a5600640 1 -- 172.21.15.5:0/2709109367 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd99c03f070 con 0x7fd9a8101df0 2024-09-10T09:47:04.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:04.403+0000 7fd9a5600640 1 -- 172.21.15.5:0/2709109367 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd99c031ca0 con 0x7fd9a8101df0 2024-09-10T09:47:04.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:04.403+0000 7fd9a5600640 1 -- 172.21.15.5:0/2709109367 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd99c035030 con 0x7fd9a8101df0 2024-09-10T09:47:04.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:04.403+0000 7fd9afc00640 1 -- 172.21.15.5:0/2709109367 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9a819d620 con 0x7fd9a8101df0 2024-09-10T09:47:04.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:04.403+0000 7fd9afc00640 1 -- 172.21.15.5:0/2709109367 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9a819dab0 con 0x7fd9a8101df0 2024-09-10T09:47:04.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:04.408+0000 7fd9a5600640 1 -- 172.21.15.5:0/2709109367 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7fd99c042070 con 0x7fd9a8101df0 2024-09-10T09:47:04.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:04.408+0000 7fd9b282d740 1 Processor -- start 2024-09-10T09:47:04.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:04.408+0000 7fd9b282d740 1 -- start start 2024-09-10T09:47:04.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:04.408+0000 7fd9b282d740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c97da52fa0 0x55c97da53380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:47:04.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:04.408+0000 7fd9b282d740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c97da538c0 0x55c97da5dc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:47:04.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:04.408+0000 7fd9b282d740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c97da5e180 0x55c97da60570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:47:04.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:04.408+0000 7fd9b282d740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55c97d9b53e0 con 0x55c97da52fa0 2024-09-10T09:47:04.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:04.408+0000 7fd9b282d740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55c97d95cd80 con 0x55c97da5e180 2024-09-10T09:47:04.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:04.408+0000 7fd9b282d740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55c97d9b13a0 con 0x55c97da538c0 2024-09-10T09:47:04.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:04.408+0000 7fd9adc00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c97da52fa0 0x55c97da53380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:47:04.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:04.408+0000 7fd9a7e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c97da5e180 0x55c97da60570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:47:04.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:04.408+0000 7fd9adc00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c97da52fa0 0x55c97da53380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48612/0 (socket says 172.21.15.5:48612) 2024-09-10T09:47:04.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:04.408+0000 7fd9ad200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c97da538c0 0x55c97da5dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:47:04.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:04.408+0000 7fd9adc00640 1 -- 172.21.15.5:0/2727327184 learned_addr learned my addr 172.21.15.5:0/2727327184 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:47:04.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:04.408+0000 7fd9a5600640 1 --2- 172.21.15.5:0/2709109367 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fd9900a2b80 0x7fd9900a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:47:04.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:04.408+0000 7fd9a7e00640 1 -- 172.21.15.5:0/2727327184 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c97da538c0 msgr2=0x55c97da5dc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:47:04.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:04.408+0000 7fd9a7e00640 1 --2- 172.21.15.5:0/2727327184 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c97da538c0 0x55c97da5dc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:04.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:04.410+0000 7fd9a7e00640 1 -- 172.21.15.5:0/2727327184 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c97da52fa0 msgr2=0x55c97da53380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:47:04.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:04.410+0000 7fd9a7e00640 1 --2- 172.21.15.5:0/2727327184 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c97da52fa0 0x55c97da53380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:04.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:04.410+0000 7fd9a7e00640 1 -- 172.21.15.5:0/2727327184 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c97d97bae0 con 0x55c97da5e180 2024-09-10T09:47:04.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:04.410+0000 7fd9a5600640 1 -- 172.21.15.5:0/2709109367 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fd99c0efba0 con 0x7fd9a8101df0 2024-09-10T09:47:04.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:04.410+0000 7fd9adc00640 1 --2- 172.21.15.5:0/2727327184 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c97da52fa0 0x55c97da53380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:47:04.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:04.410+0000 7fd9a7e00640 1 --2- 172.21.15.5:0/2727327184 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c97da5e180 0x55c97da60570 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fd994005d00 tx=0x7fd9940107a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9f006857ab85a014 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:47:04.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:04.410+0000 7fd9adc00640 1 --2- 172.21.15.5:0/2709109367 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fd9900a2b80 0x7fd9900a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:47:04.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:04.410+0000 7fd99a000640 1 -- 172.21.15.5:0/2727327184 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd99401c480 con 0x55c97da5e180 2024-09-10T09:47:04.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:04.410+0000 7fd99a000640 1 -- 172.21.15.5:0/2727327184 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd99401cb00 con 0x55c97da5e180 2024-09-10T09:47:04.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:04.410+0000 7fd99a000640 1 -- 172.21.15.5:0/2727327184 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd994011b80 con 0x55c97da5e180 2024-09-10T09:47:04.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:04.410+0000 7fd9b282d740 1 -- 172.21.15.5:0/2727327184 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c97da5e180 msgr2=0x55c97da60570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:47:04.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:04.410+0000 7fd9adc00640 1 --2- 172.21.15.5:0/2709109367 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fd9900a2b80 0x7fd9900a5040 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fd9a0005c40 tx=0x7fd9a0004a40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:47:04.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:04.410+0000 7fd9b282d740 1 --2- 172.21.15.5:0/2727327184 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c97da5e180 0x55c97da60570 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fd994005d00 tx=0x7fd9940107a0 comp rx=0 tx=0).stop 2024-09-10T09:47:04.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:04.410+0000 7fd9b282d740 1 -- 172.21.15.5:0/2727327184 shutdown_connections 2024-09-10T09:47:04.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:04.410+0000 7fd9b282d740 1 --2- 172.21.15.5:0/2727327184 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c97da5e180 0x55c97da60570 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:04.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:04.410+0000 7fd9b282d740 1 --2- 172.21.15.5:0/2727327184 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c97da538c0 0x55c97da5dc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:04.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:04.410+0000 7fd9b282d740 1 --2- 172.21.15.5:0/2727327184 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c97da52fa0 0x55c97da53380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:04.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:04.410+0000 7fd9b282d740 1 -- 172.21.15.5:0/2727327184 >> 172.21.15.5:0/2727327184 conn(0x55c97da4f530 msgr2=0x55c97da4f910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:47:04.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:04.410+0000 7fd9b282d740 1 -- 172.21.15.5:0/2727327184 shutdown_connections 2024-09-10T09:47:04.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:04.410+0000 7fd9b282d740 1 -- 172.21.15.5:0/2727327184 wait complete. 2024-09-10T09:47:04.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:04.410+0000 7fd9b282d740 1 -- start start 2024-09-10T09:47:04.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:04.410+0000 7fd9b282d740 1 Processor -- start 2024-09-10T09:47:04.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:04.410+0000 7fd9b282d740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c97da52fa0 0x55c97da57350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:47:04.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:04.410+0000 7fd9b282d740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c97da538c0 0x55c97da57890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:47:04.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:04.410+0000 7fd9b282d740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c97da5e180 0x55c97da5bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:47:04.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:04.410+0000 7fd9b282d740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55c97da4ad90 con 0x55c97da52fa0 2024-09-10T09:47:04.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:04.410+0000 7fd9b282d740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55c97d9700f0 con 0x55c97da538c0 2024-09-10T09:47:04.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:04.410+0000 7fd9b282d740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55c97da4af10 con 0x55c97da5e180 2024-09-10T09:47:04.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:04.410+0000 7fd9b282d740 10 client.0 _ll_register_callbacks cb 0x55c97d9619e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:47:04.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:04.410+0000 7fd9a7e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c97da538c0 0x55c97da57890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:47:04.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:04.410+0000 7fd9ad200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c97da52fa0 0x55c97da57350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:47:04.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:04.410+0000 7fd9a7e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c97da538c0 0x55c97da57890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:49280/0 (socket says 172.21.15.5:49280) 2024-09-10T09:47:04.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:04.410+0000 7fd9a7e00640 1 -- 172.21.15.5:0/3382654794 learned_addr learned my addr 172.21.15.5:0/3382654794 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:47:04.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:04.410+0000 7fd9adc00640 1 --2- 172.21.15.5:0/3382654794 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c97da5e180 0x55c97da5bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:47:04.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:04.410+0000 7fd9a7e00640 1 -- 172.21.15.5:0/3382654794 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c97da5e180 msgr2=0x55c97da5bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:47:04.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:04.410+0000 7fd9a7e00640 1 --2- 172.21.15.5:0/3382654794 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c97da5e180 0x55c97da5bbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:04.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:04.410+0000 7fd9a7e00640 1 -- 172.21.15.5:0/3382654794 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c97da52fa0 msgr2=0x55c97da57350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:47:04.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:04.410+0000 7fd9a7e00640 1 --2- 172.21.15.5:0/3382654794 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c97da52fa0 0x55c97da57350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:04.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:04.410+0000 7fd9a7e00640 1 -- 172.21.15.5:0/3382654794 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c97d7263c0 con 0x55c97da538c0 2024-09-10T09:47:04.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:04.410+0000 7fd9a7e00640 1 --2- 172.21.15.5:0/3382654794 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c97da538c0 0x55c97da57890 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fd99400bf80 tx=0x7fd994011660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a8250d15106a40bc server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:47:04.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:04.410+0000 7fd98ea00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-09-10T09:47:04.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:04.410+0000 7fd98ea00640 1 -- 172.21.15.5:0/3382654794 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd99401c690 con 0x55c97da538c0 2024-09-10T09:47:04.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:04.410+0000 7fd9b282d740 1 -- 172.21.15.5:0/3382654794 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd99401cb00 con 0x55c97da538c0 2024-09-10T09:47:04.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:04.410+0000 7fd98ea00640 1 -- 172.21.15.5:0/3382654794 <== mon.2 v2:172.21.15.102:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fd99401c830 con 0x55c97da538c0 2024-09-10T09:47:04.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:04.410+0000 7fd98ea00640 1 -- 172.21.15.5:0/3382654794 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fd994005030 con 0x55c97da538c0 2024-09-10T09:47:04.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:04.410+0000 7fd9b282d740 10 client.27161 fetch_fsmap learned FSMap version 35 2024-09-10T09:47:04.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:04.410+0000 7fd9b282d740 1 -- 172.21.15.5:0/3382654794 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55c97da68700 con 0x55c97da538c0 2024-09-10T09:47:04.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:04.415+0000 7fd98ea00640 1 -- 172.21.15.5:0/3382654794 <== mon.2 v2:172.21.15.102:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd99401cb00 con 0x55c97da538c0 2024-09-10T09:47:04.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:04.415+0000 7fd98ea00640 1 -- 172.21.15.5:0/3382654794 <== mon.2 v2:172.21.15.102:3300/0 5 ==== fsmap(e 35) ==== 8374+0+0 (secure 0 0 0) 0x7fd994004750 con 0x55c97da538c0 2024-09-10T09:47:04.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:04.415+0000 7fd9b282d740 10 client.27161 fetch_fsmap finished waiting for FSMap version 35 2024-09-10T09:47:04.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:04.415+0000 7fd9b282d740 10 client.27161 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T09:47:04.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:04.415+0000 7fd9b282d740 20 client.27161 populate_metadata read hostname 'smithi005' 2024-09-10T09:47:04.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:04.415+0000 7fd9b282d740 1 --2- 172.21.15.5:0/3382654794 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55c97da69b10 0x55c97da6bf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:47:04.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:04.415+0000 7fd9b282d740 4 client.27161 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:47:04.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:04.415+0000 7fd9b282d740 1 -- 172.21.15.5:0/3382654794 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c97d9c2950 con 0x55c97da69b10 2024-09-10T09:47:04.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:04.415+0000 7fd9ad200640 1 --2- 172.21.15.5:0/3382654794 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55c97da69b10 0x55c97da6bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:47:04.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:04.415+0000 7fd9ad200640 1 --2- 172.21.15.5:0/3382654794 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55c97da69b10 0x55c97da6bf00 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=93aa649e60c09454 server_cookie=a50af34fcfaeb9f6 in_seq=0 out_seq=0 2024-09-10T09:47:04.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:04.415+0000 7fd98ea00640 10 client.27161 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T09:47:04.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:04.415+0000 7fd98ea00640 1 -- 172.21.15.5:0/3382654794 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c97d9c2950 con 0x55c97da69b10 2024-09-10T09:47:04.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:04.415+0000 7fd98ea00640 10 client.27161 handle_command_reply: tid=0 2024-09-10T09:47:04.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:04.415+0000 7fd9b282d740 1 client.27161 shutdown 2024-09-10T09:47:04.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:04.415+0000 7fd9b282d740 1 -- 172.21.15.5:0/3382654794 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c97da538c0 msgr2=0x55c97da57890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:47:04.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:04.415+0000 7fd9b282d740 1 --2- 172.21.15.5:0/3382654794 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c97da538c0 0x55c97da57890 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fd99400bf80 tx=0x7fd994011660 comp rx=0 tx=0).stop 2024-09-10T09:47:04.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:04.415+0000 7fd9b282d740 1 -- 172.21.15.5:0/3382654794 shutdown_connections 2024-09-10T09:47:04.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:04.415+0000 7fd9b282d740 1 --2- 172.21.15.5:0/3382654794 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55c97da69b10 0x55c97da6bf00 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:04.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:04.415+0000 7fd9b282d740 1 --2- 172.21.15.5:0/3382654794 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c97da5e180 0x55c97da5bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:04.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:04.415+0000 7fd9b282d740 1 --2- 172.21.15.5:0/3382654794 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c97da538c0 0x55c97da57890 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:04.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:04.415+0000 7fd9b282d740 1 --2- 172.21.15.5:0/3382654794 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c97da52fa0 0x55c97da57350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:04.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:04.415+0000 7fd9b282d740 1 -- 172.21.15.5:0/3382654794 >> 172.21.15.5:0/3382654794 conn(0x55c97da4f530 msgr2=0x55c97da55260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:47:04.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:04.415+0000 7fd98ea00640 0 client.27161 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T09:47:04.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:04.415+0000 7fd9b282d740 1 -- 172.21.15.5:0/3382654794 shutdown_connections 2024-09-10T09:47:04.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:04.415+0000 7fd9b282d740 1 -- 172.21.15.5:0/3382654794 wait complete. 2024-09-10T09:47:04.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:04.415+0000 7fd9b282d740 20 client.27161 trim_cache size 0 max 16384 2024-09-10T09:47:04.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:04.434+0000 7fd9b282d740 1 Processor -- start 2024-09-10T09:47:04.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:04.434+0000 7fd9b282d740 1 -- start start 2024-09-10T09:47:04.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:04.434+0000 7fd9b282d740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c97da52fa0 0x55c97da55100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:47:04.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:04.434+0000 7fd9b282d740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c97da538c0 0x55c97da5bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:47:04.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:04.434+0000 7fd9b282d740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c97da5e180 0x55c97da71690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:47:04.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:04.434+0000 7fd9b282d740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd9a81acc90 con 0x55c97da538c0 2024-09-10T09:47:04.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:04.434+0000 7fd9b282d740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fd9a81ae800 con 0x55c97da52fa0 2024-09-10T09:47:04.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:04.434+0000 7fd9b282d740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fd9a81ad360 con 0x55c97da5e180 2024-09-10T09:47:04.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:04.434+0000 7fd9a7e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c97da538c0 0x55c97da5bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:47:04.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:04.434+0000 7fd9a7e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c97da538c0 0x55c97da5bbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48636/0 (socket says 172.21.15.5:48636) 2024-09-10T09:47:04.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:04.434+0000 7fd9a7e00640 1 -- 172.21.15.5:0/393601994 learned_addr learned my addr 172.21.15.5:0/393601994 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:47:04.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:04.434+0000 7fd9adc00640 1 --2- 172.21.15.5:0/393601994 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c97da5e180 0x55c97da71690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:47:04.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:04.434+0000 7fd9ad200640 1 --2- 172.21.15.5:0/393601994 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c97da52fa0 0x55c97da55100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:47:04.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:04.434+0000 7fd9adc00640 1 -- 172.21.15.5:0/393601994 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c97da52fa0 msgr2=0x55c97da55100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:47:04.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:04.434+0000 7fd9adc00640 1 --2- 172.21.15.5:0/393601994 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c97da52fa0 0x55c97da55100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:04.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:04.434+0000 7fd9adc00640 1 -- 172.21.15.5:0/393601994 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c97da538c0 msgr2=0x55c97da5bbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:47:04.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:04.434+0000 7fd9adc00640 1 --2- 172.21.15.5:0/393601994 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c97da538c0 0x55c97da5bbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:04.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:04.434+0000 7fd9adc00640 1 -- 172.21.15.5:0/393601994 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c97da4c310 con 0x55c97da5e180 2024-09-10T09:47:04.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:04.434+0000 7fd9ad200640 1 --2- 172.21.15.5:0/393601994 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c97da52fa0 0x55c97da55100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:47:04.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:04.434+0000 7fd9a7e00640 1 --2- 172.21.15.5:0/393601994 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c97da538c0 0x55c97da5bbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:47:04.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:04.434+0000 7fd9adc00640 1 --2- 172.21.15.5:0/393601994 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c97da5e180 0x55c97da71690 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fd9a0005fd0 tx=0x7fd9a0016000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fd254bd242987826 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:47:04.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:04.434+0000 7fd999600640 1 -- 172.21.15.5:0/393601994 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9a00130d0 con 0x55c97da5e180 2024-09-10T09:47:04.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:04.434+0000 7fd999600640 1 -- 172.21.15.5:0/393601994 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd9a0013270 con 0x55c97da5e180 2024-09-10T09:47:04.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:04.434+0000 7fd999600640 1 -- 172.21.15.5:0/393601994 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9a00053a0 con 0x55c97da5e180 2024-09-10T09:47:04.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:04.434+0000 7fd9b282d740 1 -- 172.21.15.5:0/393601994 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c97da5e180 msgr2=0x55c97da71690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:47:04.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:04.434+0000 7fd9b282d740 1 --2- 172.21.15.5:0/393601994 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c97da5e180 0x55c97da71690 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fd9a0005fd0 tx=0x7fd9a0016000 comp rx=0 tx=0).stop 2024-09-10T09:47:04.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:04.434+0000 7fd9b282d740 1 -- 172.21.15.5:0/393601994 shutdown_connections 2024-09-10T09:47:04.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:04.434+0000 7fd9b282d740 1 --2- 172.21.15.5:0/393601994 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c97da5e180 0x55c97da71690 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:04.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:04.434+0000 7fd9b282d740 1 --2- 172.21.15.5:0/393601994 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c97da538c0 0x55c97da5bbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:04.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:04.434+0000 7fd9b282d740 1 --2- 172.21.15.5:0/393601994 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c97da52fa0 0x55c97da55100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:04.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:04.434+0000 7fd9b282d740 1 -- 172.21.15.5:0/393601994 >> 172.21.15.5:0/393601994 conn(0x55c97da4f530 msgr2=0x55c97da549f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:47:04.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:04.434+0000 7fd9b282d740 1 -- 172.21.15.5:0/393601994 shutdown_connections 2024-09-10T09:47:04.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:04.434+0000 7fd9b282d740 1 -- 172.21.15.5:0/393601994 wait complete. 2024-09-10T09:47:04.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:04.439+0000 7fd9b282d740 1 -- start start 2024-09-10T09:47:04.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:04.439+0000 7fd9b282d740 1 Processor -- start 2024-09-10T09:47:04.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:04.439+0000 7fd9b282d740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c97da52fa0 0x55c97da65d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:47:04.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:04.439+0000 7fd9b282d740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c97da538c0 0x55c97da66270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:47:04.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:04.439+0000 7fd9b282d740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c97da5e180 0x55c97da4cf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:47:04.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:04.439+0000 7fd9b282d740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd9a81ae100 con 0x55c97da538c0 2024-09-10T09:47:04.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:04.439+0000 7fd9b282d740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fd9a81ada30 con 0x55c97da52fa0 2024-09-10T09:47:04.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:04.439+0000 7fd9b282d740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fd9a81aac70 con 0x55c97da5e180 2024-09-10T09:47:04.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:04.439+0000 7fd9b282d740 10 client.0 _ll_register_callbacks cb 0x7fd99400bae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:47:04.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:04.439+0000 7fd9ad200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c97da52fa0 0x55c97da65d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:47:04.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:04.439+0000 7fd9ad200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c97da52fa0 0x55c97da65d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:49294/0 (socket says 172.21.15.5:49294) 2024-09-10T09:47:04.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:04.439+0000 7fd9ad200640 1 -- 172.21.15.5:0/4111363949 learned_addr learned my addr 172.21.15.5:0/4111363949 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:47:04.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:04.439+0000 7fd9a7e00640 1 --2- 172.21.15.5:0/4111363949 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c97da538c0 0x55c97da66270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:47:04.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:04.439+0000 7fd9adc00640 1 --2- 172.21.15.5:0/4111363949 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c97da5e180 0x55c97da4cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:47:04.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:04.439+0000 7fd9ad200640 1 -- 172.21.15.5:0/4111363949 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c97da5e180 msgr2=0x55c97da4cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:47:04.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:04.439+0000 7fd9ad200640 1 --2- 172.21.15.5:0/4111363949 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c97da5e180 0x55c97da4cf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:04.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:04.439+0000 7fd9ad200640 1 -- 172.21.15.5:0/4111363949 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c97da538c0 msgr2=0x55c97da66270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:47:04.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:04.439+0000 7fd9ad200640 1 --2- 172.21.15.5:0/4111363949 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c97da538c0 0x55c97da66270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:04.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:04.439+0000 7fd9ad200640 1 -- 172.21.15.5:0/4111363949 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c97da4d540 con 0x55c97da52fa0 2024-09-10T09:47:04.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:04.439+0000 7fd9a7e00640 1 --2- 172.21.15.5:0/4111363949 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c97da538c0 0x55c97da66270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:47:04.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:04.439+0000 7fd9ad200640 1 --2- 172.21.15.5:0/4111363949 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c97da52fa0 0x55c97da65d30 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fd99c0bbf40 tx=0x7fd99c0ed120 comp rx=0 tx=0).ready entity=mon.2 client_cookie=44271c0236a46b15 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:47:04.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:04.439+0000 7fd98ea00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-09-10T09:47:04.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:04.439+0000 7fd98ea00640 1 -- 172.21.15.5:0/4111363949 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd99c049040 con 0x55c97da52fa0 2024-09-10T09:47:04.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:04.439+0000 7fd98ea00640 1 -- 172.21.15.5:0/4111363949 <== mon.2 v2:172.21.15.102:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fd99c0ed360 con 0x55c97da52fa0 2024-09-10T09:47:04.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:04.439+0000 7fd9b282d740 1 -- 172.21.15.5:0/4111363949 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd9a0013270 con 0x55c97da52fa0 2024-09-10T09:47:04.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:04.439+0000 7fd98ea00640 1 -- 172.21.15.5:0/4111363949 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fd99c0bb910 con 0x55c97da52fa0 2024-09-10T09:47:04.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:04.439+0000 7fd9b282d740 10 client.27167 fetch_fsmap learned FSMap version 35 2024-09-10T09:47:04.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:04.439+0000 7fd9b282d740 1 -- 172.21.15.5:0/4111363949 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55c97dad4010 con 0x55c97da52fa0 2024-09-10T09:47:04.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:04.439+0000 7fd98ea00640 1 -- 172.21.15.5:0/4111363949 <== mon.2 v2:172.21.15.102:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd9a0013270 con 0x55c97da52fa0 2024-09-10T09:47:04.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:04.439+0000 7fd98ea00640 1 -- 172.21.15.5:0/4111363949 <== mon.2 v2:172.21.15.102:3300/0 5 ==== fsmap(e 35) ==== 8374+0+0 (secure 0 0 0) 0x7fd99c0b8620 con 0x55c97da52fa0 2024-09-10T09:47:04.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:04.439+0000 7fd9b282d740 10 client.27167 fetch_fsmap finished waiting for FSMap version 35 2024-09-10T09:47:04.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:04.439+0000 7fd9b282d740 10 client.27167 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T09:47:04.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:04.439+0000 7fd9b282d740 20 client.27167 populate_metadata read hostname 'smithi005' 2024-09-10T09:47:04.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:04.439+0000 7fd9b282d740 1 --2- 172.21.15.5:0/4111363949 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55c97dad4760 0x55c97dad6b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:47:04.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:04.439+0000 7fd9b282d740 4 client.27167 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T09:47:04.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:04.439+0000 7fd9b282d740 1 -- 172.21.15.5:0/4111363949 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55c97d9bce60 con 0x55c97dad4760 2024-09-10T09:47:04.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:04.439+0000 7fd9a7e00640 1 --2- 172.21.15.5:0/4111363949 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55c97dad4760 0x55c97dad6b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:47:04.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:04.439+0000 7fd9a7e00640 1 --2- 172.21.15.5:0/4111363949 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55c97dad4760 0x55c97dad6b80 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d60d5dbe36d9009f server_cookie=7f29e6c9b337df13 in_seq=0 out_seq=0 2024-09-10T09:47:04.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:04.439+0000 7fd98ea00640 10 client.27167 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T09:47:04.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:04.439+0000 7fd98ea00640 1 -- 172.21.15.5:0/4111363949 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55c97d9bce60 con 0x55c97dad4760 2024-09-10T09:47:04.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:04.439+0000 7fd98ea00640 10 client.27167 handle_command_reply: tid=0 2024-09-10T09:47:04.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:04.439+0000 7fd9b282d740 1 client.27167 shutdown 2024-09-10T09:47:04.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:04.439+0000 7fd9b282d740 1 -- 172.21.15.5:0/4111363949 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c97da52fa0 msgr2=0x55c97da65d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:47:04.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:04.439+0000 7fd9b282d740 1 --2- 172.21.15.5:0/4111363949 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c97da52fa0 0x55c97da65d30 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fd99c0bbf40 tx=0x7fd99c0ed120 comp rx=0 tx=0).stop 2024-09-10T09:47:04.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:04.439+0000 7fd9b282d740 1 -- 172.21.15.5:0/4111363949 shutdown_connections 2024-09-10T09:47:04.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:04.439+0000 7fd9b282d740 1 --2- 172.21.15.5:0/4111363949 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55c97dad4760 0x55c97dad6b80 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:04.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:04.439+0000 7fd9b282d740 1 --2- 172.21.15.5:0/4111363949 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c97da5e180 0x55c97da4cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:04.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:04.439+0000 7fd9b282d740 1 --2- 172.21.15.5:0/4111363949 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c97da538c0 0x55c97da66270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:04.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:04.439+0000 7fd9b282d740 1 --2- 172.21.15.5:0/4111363949 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c97da52fa0 0x55c97da65d30 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:04.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:04.439+0000 7fd9b282d740 1 -- 172.21.15.5:0/4111363949 >> 172.21.15.5:0/4111363949 conn(0x55c97da4f530 msgr2=0x55c97da55210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:47:04.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:04.439+0000 7fd98ea00640 0 client.27167 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T09:47:04.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:04.439+0000 7fd9b282d740 1 -- 172.21.15.5:0/4111363949 shutdown_connections 2024-09-10T09:47:04.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:04.444+0000 7fd9b282d740 1 -- 172.21.15.5:0/4111363949 wait complete. 2024-09-10T09:47:04.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:04.444+0000 7fd9b282d740 20 client.27167 trim_cache size 0 max 16384 2024-09-10T09:47:04.449 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-10T09:47:04.449 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-10T09:47:04.449 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2024-09-10T09:47:04.449 INFO:teuthology.orchestra.run.smithi005.stdout: "8a32a92d-50b6-4ee6-b59e-d505b9c86d52": { 2024-09-10T09:47:04.449 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2024-09-10T09:47:04.449 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "8a32a92d-50b6-4ee6-b59e-d505b9c86d52", 2024-09-10T09:47:04.449 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2024-09-10T09:47:04.449 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T09:47:04.449 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T09:47:04.449 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-10T09:47:04.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:04.444+0000 7fd9afc00640 1 -- 172.21.15.5:0/2709109367 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fd9900a2b80 msgr2=0x7fd9900a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:47:04.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:04.444+0000 7fd9afc00640 1 --2- 172.21.15.5:0/2709109367 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fd9900a2b80 0x7fd9900a5040 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fd9a0005c40 tx=0x7fd9a0004a40 comp rx=0 tx=0).stop 2024-09-10T09:47:04.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:04.444+0000 7fd9afc00640 1 -- 172.21.15.5:0/2709109367 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd9a8101df0 msgr2=0x7fd9a81983e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:47:04.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:04.444+0000 7fd9afc00640 1 --2- 172.21.15.5:0/2709109367 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd9a8101df0 0x7fd9a81983e0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fd99c004770 tx=0x7fd99c003c10 comp rx=0 tx=0).stop 2024-09-10T09:47:04.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:04.444+0000 7fd9afc00640 1 -- 172.21.15.5:0/2709109367 shutdown_connections 2024-09-10T09:47:04.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:04.444+0000 7fd9afc00640 1 --2- 172.21.15.5:0/2709109367 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fd9900a2b80 0x7fd9900a5040 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:04.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:04.444+0000 7fd9afc00640 1 --2- 172.21.15.5:0/2709109367 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd9a8106f20 0x7fd9a819ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:04.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:04.444+0000 7fd9afc00640 1 --2- 172.21.15.5:0/2709109367 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd9a81027a0 0x7fd9a8198920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:04.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:04.444+0000 7fd9afc00640 1 --2- 172.21.15.5:0/2709109367 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd9a8101df0 0x7fd9a81983e0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:04.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:04.444+0000 7fd9afc00640 1 -- 172.21.15.5:0/2709109367 >> 172.21.15.5:0/2709109367 conn(0x7fd9a80fd8b0 msgr2=0x7fd9a80feb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:47:04.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:04.444+0000 7fd9afc00640 1 -- 172.21.15.5:0/2709109367 shutdown_connections 2024-09-10T09:47:04.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:04.444+0000 7fd9afc00640 1 -- 172.21.15.5:0/2709109367 wait complete. 2024-09-10T09:47:05.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:47:04 smithi005 ceph-mon[30350]: pgmap v655: 129 pgs: 129 active+clean; 196 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 335 KiB/s rd, 1.7 MiB/s wr, 94 op/s 2024-09-10T09:47:05.077 INFO:tasks.workunit.client.0.smithi005.stdout: 1 515563 6.16 MB/sec execute 286 sec latency 989.901 ms 2024-09-10T09:47:05.342 INFO:tasks.cephfs.filesystem:scrub status for tag:8a32a92d-50b6-4ee6-b59e-d505b9c86d52 - {'path': '/', 'tag': '8a32a92d-50b6-4ee6-b59e-d505b9c86d52', 'options': 'recursive,force'} 2024-09-10T09:47:05.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:47:04 smithi159 ceph-mon[35513]: pgmap v655: 129 pgs: 129 active+clean; 196 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 335 KiB/s rd, 1.7 MiB/s wr, 94 op/s 2024-09-10T09:47:05.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:47:04 smithi102 ceph-mon[35442]: pgmap v655: 129 pgs: 129 active+clean; 196 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 335 KiB/s rd, 1.7 MiB/s wr, 94 op/s 2024-09-10T09:47:06.077 INFO:tasks.workunit.client.0.smithi005.stdout: 1 517918 6.17 MB/sec execute 287 sec latency 26.168 ms 2024-09-10T09:47:07.077 INFO:tasks.workunit.client.0.smithi005.stdout: 1 520098 6.20 MB/sec execute 288 sec latency 24.842 ms 2024-09-10T09:47:07.306 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:47:06 smithi005 ceph-mon[30350]: pgmap v656: 129 pgs: 129 active+clean; 199 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 586 KiB/s rd, 1.9 MiB/s wr, 140 op/s 2024-09-10T09:47:07.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:47:06 smithi159 ceph-mon[35513]: pgmap v656: 129 pgs: 129 active+clean; 199 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 586 KiB/s rd, 1.9 MiB/s wr, 140 op/s 2024-09-10T09:47:07.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:47:06 smithi102 ceph-mon[35442]: pgmap v656: 129 pgs: 129 active+clean; 199 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 586 KiB/s rd, 1.9 MiB/s wr, 140 op/s 2024-09-10T09:47:08.077 INFO:tasks.workunit.client.0.smithi005.stdout: 1 522635 6.23 MB/sec execute 289 sec latency 10.634 ms 2024-09-10T09:47:09.077 INFO:tasks.workunit.client.0.smithi005.stdout: 1 523873 6.22 MB/sec execute 290 sec latency 413.281 ms 2024-09-10T09:47:09.209 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:47:08 smithi005 ceph-mon[30350]: pgmap v657: 129 pgs: 129 active+clean; 201 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 954 KiB/s rd, 2.0 MiB/s wr, 174 op/s 2024-09-10T09:47:09.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:47:08 smithi159 ceph-mon[35513]: pgmap v657: 129 pgs: 129 active+clean; 201 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 954 KiB/s rd, 2.0 MiB/s wr, 174 op/s 2024-09-10T09:47:09.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:47:08 smithi102 ceph-mon[35442]: pgmap v657: 129 pgs: 129 active+clean; 201 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 954 KiB/s rd, 2.0 MiB/s wr, 174 op/s 2024-09-10T09:47:10.077 INFO:tasks.workunit.client.0.smithi005.stdout: 1 523882 6.20 MB/sec execute 291 sec latency 839.052 ms 2024-09-10T09:47:11.077 INFO:tasks.workunit.client.0.smithi005.stdout: 1 526010 6.23 MB/sec execute 292 sec latency 798.658 ms 2024-09-10T09:47:11.306 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:47:10 smithi005 ceph-mon[30350]: pgmap v658: 129 pgs: 129 active+clean; 200 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 947 KiB/s rd, 2.0 MiB/s wr, 193 op/s 2024-09-10T09:47:11.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:47:10 smithi159 ceph-mon[35513]: pgmap v658: 129 pgs: 129 active+clean; 200 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 947 KiB/s rd, 2.0 MiB/s wr, 193 op/s 2024-09-10T09:47:11.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:47:10 smithi102 ceph-mon[35442]: pgmap v658: 129 pgs: 129 active+clean; 200 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 947 KiB/s rd, 2.0 MiB/s wr, 193 op/s 2024-09-10T09:47:12.077 INFO:tasks.workunit.client.0.smithi005.stdout: 1 528205 6.23 MB/sec execute 293 sec latency 23.401 ms 2024-09-10T09:47:13.077 INFO:tasks.workunit.client.0.smithi005.stdout: 1 530133 6.25 MB/sec execute 294 sec latency 25.307 ms 2024-09-10T09:47:13.306 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:47:12 smithi005 ceph-mon[30350]: pgmap v659: 129 pgs: 129 active+clean; 201 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.1 MiB/s wr, 190 op/s 2024-09-10T09:47:13.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:47:12 smithi159 ceph-mon[35513]: pgmap v659: 129 pgs: 129 active+clean; 201 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.1 MiB/s wr, 190 op/s 2024-09-10T09:47:13.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:47:12 smithi102 ceph-mon[35442]: pgmap v659: 129 pgs: 129 active+clean; 201 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.1 MiB/s wr, 190 op/s 2024-09-10T09:47:14.077 INFO:tasks.workunit.client.0.smithi005.stdout: 1 532254 6.27 MB/sec execute 295 sec latency 251.784 ms 2024-09-10T09:47:15.077 INFO:tasks.workunit.client.0.smithi005.stdout: 1 533598 6.27 MB/sec execute 296 sec latency 464.262 ms 2024-09-10T09:47:16.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:47:15 smithi005 ceph-mon[30350]: pgmap v660: 129 pgs: 129 active+clean; 201 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.0 MiB/s wr, 189 op/s 2024-09-10T09:47:16.077 INFO:tasks.workunit.client.0.smithi005.stdout: 1 536226 6.30 MB/sec execute 297 sec latency 14.065 ms 2024-09-10T09:47:16.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:47:15 smithi159 ceph-mon[35513]: pgmap v660: 129 pgs: 129 active+clean; 201 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.0 MiB/s wr, 189 op/s 2024-09-10T09:47:16.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:47:15 smithi102 ceph-mon[35442]: pgmap v660: 129 pgs: 129 active+clean; 201 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.0 MiB/s wr, 189 op/s 2024-09-10T09:47:17.078 INFO:tasks.workunit.client.0.smithi005.stdout: 1 537926 6.31 MB/sec execute 298 sec latency 27.770 ms 2024-09-10T09:47:18.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:47:17 smithi005 ceph-mon[30350]: pgmap v661: 129 pgs: 129 active+clean; 203 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.3 MiB/s wr, 237 op/s 2024-09-10T09:47:18.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:47:17 smithi005 ceph-mon[30350]: scrub summary: idle 2024-09-10T09:47:18.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:47:17 smithi005 ceph-mon[30350]: scrub summary: idle 2024-09-10T09:47:18.078 INFO:tasks.workunit.client.0.smithi005.stdout: 1 540350 6.34 MB/sec execute 299 sec latency 51.555 ms 2024-09-10T09:47:18.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:47:17 smithi159 ceph-mon[35513]: pgmap v661: 129 pgs: 129 active+clean; 203 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.3 MiB/s wr, 237 op/s 2024-09-10T09:47:18.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:47:17 smithi159 ceph-mon[35513]: scrub summary: idle 2024-09-10T09:47:18.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:47:17 smithi159 ceph-mon[35513]: scrub summary: idle 2024-09-10T09:47:18.136 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:47:17 smithi102 ceph-mon[35442]: pgmap v661: 129 pgs: 129 active+clean; 203 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.3 MiB/s wr, 237 op/s 2024-09-10T09:47:18.136 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:47:17 smithi102 ceph-mon[35442]: scrub summary: idle 2024-09-10T09:47:18.136 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:47:17 smithi102 ceph-mon[35442]: scrub summary: idle 2024-09-10T09:47:19.078 INFO:tasks.workunit.client.0.smithi005.stdout: 1 542745 6.35 MB/sec execute 300 sec latency 23.895 ms 2024-09-10T09:47:20.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:47:19 smithi005 ceph-mon[30350]: pgmap v662: 129 pgs: 129 active+clean; 203 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.9 MiB/s wr, 192 op/s 2024-09-10T09:47:20.078 INFO:tasks.workunit.client.0.smithi005.stdout: 1 544675 6.37 MB/sec execute 301 sec latency 23.942 ms 2024-09-10T09:47:20.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:47:19 smithi159 ceph-mon[35513]: pgmap v662: 129 pgs: 129 active+clean; 203 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.9 MiB/s wr, 192 op/s 2024-09-10T09:47:20.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:47:19 smithi102 ceph-mon[35442]: pgmap v662: 129 pgs: 129 active+clean; 203 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.9 MiB/s wr, 192 op/s 2024-09-10T09:47:21.078 INFO:tasks.workunit.client.0.smithi005.stdout: 1 547390 6.40 MB/sec execute 302 sec latency 11.029 ms 2024-09-10T09:47:21.557 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 24.6 2024-09-10T09:47:21.557 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T09:47:22.000 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:47:22.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:47:21 smithi005 ceph-mon[30350]: pgmap v663: 129 pgs: 129 active+clean; 206 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.1 MiB/s wr, 220 op/s 2024-09-10T09:47:22.078 INFO:tasks.workunit.client.0.smithi005.stdout: 1 549479 6.40 MB/sec execute 303 sec latency 22.553 ms 2024-09-10T09:47:22.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:47:21 smithi159 ceph-mon[35513]: pgmap v663: 129 pgs: 129 active+clean; 206 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.1 MiB/s wr, 220 op/s 2024-09-10T09:47:22.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:47:21 smithi102 ceph-mon[35442]: pgmap v663: 129 pgs: 129 active+clean; 206 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.1 MiB/s wr, 220 op/s 2024-09-10T09:47:22.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:22.561+0000 7ff26f200640 1 -- 172.21.15.5:0/3718536359 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff26810e850 msgr2=0x7ff268110c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:47:22.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:22.561+0000 7ff26f200640 1 --2- 172.21.15.5:0/3718536359 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff26810e850 0x7ff268110c40 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7ff260009ca0 tx=0x7ff26002a8b0 comp rx=0 tx=0).stop 2024-09-10T09:47:22.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:22.561+0000 7ff26f200640 1 -- 172.21.15.5:0/3718536359 shutdown_connections 2024-09-10T09:47:22.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:22.561+0000 7ff26f200640 1 --2- 172.21.15.5:0/3718536359 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff26810e850 0x7ff268110c40 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:22.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:22.561+0000 7ff26f200640 1 --2- 172.21.15.5:0/3718536359 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff268100610 0x7ff268100a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:22.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:22.561+0000 7ff26f200640 1 --2- 172.21.15.5:0/3718536359 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff268102cc0 0x7ff2681000d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:22.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:22.561+0000 7ff26f200640 1 -- 172.21.15.5:0/3718536359 >> 172.21.15.5:0/3718536359 conn(0x7ff268077210 msgr2=0x7ff2680fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:47:22.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:22.561+0000 7ff26f200640 1 -- 172.21.15.5:0/3718536359 shutdown_connections 2024-09-10T09:47:22.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:22.561+0000 7ff26f200640 1 -- 172.21.15.5:0/3718536359 wait complete. 2024-09-10T09:47:22.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:22.562+0000 7ff26f200640 1 Processor -- start 2024-09-10T09:47:22.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:22.562+0000 7ff26f200640 1 -- start start 2024-09-10T09:47:22.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:22.563+0000 7ff26f200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff268100610 0x7ff2681034c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:47:22.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:22.563+0000 7ff26f200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff268102cc0 0x7ff268103a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:47:22.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:22.563+0000 7ff26f200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff26810e850 0x7ff268103f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:47:22.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:22.563+0000 7ff26f200640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff2681143a0 con 0x7ff268100610 2024-09-10T09:47:22.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:22.563+0000 7ff26f200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7ff268114220 con 0x7ff268102cc0 2024-09-10T09:47:22.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:22.563+0000 7ff26f200640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7ff268114520 con 0x7ff26810e850 2024-09-10T09:47:22.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:22.563+0000 7ff26d200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff26810e850 0x7ff268103f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:47:22.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:22.563+0000 7ff267e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff268100610 0x7ff2681034c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:47:22.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:22.563+0000 7ff26d200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff26810e850 0x7ff268103f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:53612/0 (socket says 172.21.15.5:53612) 2024-09-10T09:47:22.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:22.563+0000 7ff26d200640 1 -- 172.21.15.5:0/2796154517 learned_addr learned my addr 172.21.15.5:0/2796154517 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:47:22.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:22.563+0000 7ff267400640 1 --2- 172.21.15.5:0/2796154517 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff268102cc0 0x7ff268103a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:47:22.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:22.563+0000 7ff26d200640 1 -- 172.21.15.5:0/2796154517 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff268102cc0 msgr2=0x7ff268103a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:47:22.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:22.563+0000 7ff26d200640 1 --2- 172.21.15.5:0/2796154517 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff268102cc0 0x7ff268103a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:22.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:22.563+0000 7ff26d200640 1 -- 172.21.15.5:0/2796154517 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff268100610 msgr2=0x7ff2681034c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:47:22.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:22.563+0000 7ff26d200640 1 --2- 172.21.15.5:0/2796154517 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff268100610 0x7ff2681034c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:22.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:22.563+0000 7ff26d200640 1 -- 172.21.15.5:0/2796154517 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2681aaee0 con 0x7ff26810e850 2024-09-10T09:47:22.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:22.563+0000 7ff267e00640 1 --2- 172.21.15.5:0/2796154517 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff268100610 0x7ff2681034c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:47:22.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:22.564+0000 7ff26d200640 1 --2- 172.21.15.5:0/2796154517 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff26810e850 0x7ff268103f40 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7ff260004a40 tx=0x7ff260004a70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:47:22.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:22.564+0000 7ff264c00640 1 -- 172.21.15.5:0/2796154517 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff260003e40 con 0x7ff26810e850 2024-09-10T09:47:22.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:22.564+0000 7ff264c00640 1 -- 172.21.15.5:0/2796154517 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff260040070 con 0x7ff26810e850 2024-09-10T09:47:22.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:22.564+0000 7ff264c00640 1 -- 172.21.15.5:0/2796154517 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff260031b40 con 0x7ff26810e850 2024-09-10T09:47:22.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:22.564+0000 7ff26f200640 1 -- 172.21.15.5:0/2796154517 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2681ab170 con 0x7ff26810e850 2024-09-10T09:47:22.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:22.564+0000 7ff26f200640 1 -- 172.21.15.5:0/2796154517 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2681ab700 con 0x7ff26810e850 2024-09-10T09:47:22.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:22.565+0000 7ff26f200640 1 -- 172.21.15.5:0/2796154517 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff268101380 con 0x7ff26810e850 2024-09-10T09:47:22.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:22.567+0000 7ff264c00640 1 -- 172.21.15.5:0/2796154517 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7ff260031ce0 con 0x7ff26810e850 2024-09-10T09:47:22.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:22.569+0000 7ff264c00640 1 --2- 172.21.15.5:0/2796154517 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7ff24c0a2c00 0x7ff24c0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:47:22.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:22.569+0000 7ff264c00640 1 -- 172.21.15.5:0/2796154517 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7ff2600f05c0 con 0x7ff26810e850 2024-09-10T09:47:22.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:22.569+0000 7ff267e00640 1 --2- 172.21.15.5:0/2796154517 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7ff24c0a2c00 0x7ff24c0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:47:22.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:22.570+0000 7ff267e00640 1 --2- 172.21.15.5:0/2796154517 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7ff24c0a2c00 0x7ff24c0a50c0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7ff258009f40 tx=0x7ff258005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:47:22.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:22.571+0000 7ff264c00640 1 -- 172.21.15.5:0/2796154517 <== mon.1 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff2600bd0d0 con 0x7ff26810e850 2024-09-10T09:47:22.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:22.805+0000 7ff26f200640 1 -- 172.21.15.5:0/2796154517 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff2681049f0 con 0x7ff26810e850 2024-09-10T09:47:22.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:22.805+0000 7ff264c00640 1 -- 172.21.15.5:0/2796154517 <== mon.1 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) ==== 94+0+11970 (secure 0 0 0) 0x7ff2600bc9d0 con 0x7ff26810e850 2024-09-10T09:47:22.807 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-10T09:47:22.807 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":35,"btime":"2024-09-10T09:46:17:834001+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14454,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/2970594311","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2970594311},{"type":"v1","addr":"172.21.15.5:6835","nonce":2970594311}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14514,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6839/3616502174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":3616502174},{"type":"v1","addr":"172.21.15.5:6839","nonce":3616502174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":35,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T09:35:29.626287+0000","modified":"2024-09-10T09:46:16.966068+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24542,"mds_1":24535,"mds_2":24511,"mds_3":24506,"mds_4":24443},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14520":{"gid":14520,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/2073599842","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":2073599842},{"type":"v1","addr":"172.21.15.5:6841","nonce":2073599842}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"k","rank":4,"incarnation":26,"state":"up:active","state_seq":24,"addr":"172.21.15.102:6833/2457007427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":2457007427},{"type":"v1","addr":"172.21.15.102:6833","nonce":2457007427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/356445159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":356445159},{"type":"v1","addr":"172.21.15.159:6833","nonce":356445159}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6837/2894314507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":2894314507},{"type":"v1","addr":"172.21.15.5:6837","nonce":2894314507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.102:6835/1647549111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":1647549111},{"type":"v1","addr":"172.21.15.102:6835","nonce":1647549111}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6835/952657242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":952657242},{"type":"v1","addr":"172.21.15.159:6835","nonce":952657242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":19,"addr":"172.21.15.102:6837/2689776764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":2689776764},{"type":"v1","addr":"172.21.15.102:6837","nonce":2689776764}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.159:6837/620627639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":620627639},{"type":"v1","addr":"172.21.15.159:6837","nonce":620627639}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24535":{"gid":24535,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":18,"addr":"172.21.15.159:6839/2073739599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":2073739599},{"type":"v1","addr":"172.21.15.159:6839","nonce":2073739599}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24542":{"gid":24542,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.102:6839/335409749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6838","nonce":335409749},{"type":"v1","addr":"172.21.15.102:6839","nonce":335409749}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24542,"qdb_cluster":[24443,24506,24511,24535,24542]},"id":1}]} 2024-09-10T09:47:22.807 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 35 2024-09-10T09:47:22.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:22.808+0000 7ff26f200640 1 -- 172.21.15.5:0/2796154517 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7ff24c0a2c00 msgr2=0x7ff24c0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:47:22.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:22.808+0000 7ff26f200640 1 --2- 172.21.15.5:0/2796154517 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7ff24c0a2c00 0x7ff24c0a50c0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7ff258009f40 tx=0x7ff258005f90 comp rx=0 tx=0).stop 2024-09-10T09:47:22.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:22.808+0000 7ff26f200640 1 -- 172.21.15.5:0/2796154517 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff26810e850 msgr2=0x7ff268103f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:47:22.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:22.808+0000 7ff26f200640 1 --2- 172.21.15.5:0/2796154517 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff26810e850 0x7ff268103f40 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7ff260004a40 tx=0x7ff260004a70 comp rx=0 tx=0).stop 2024-09-10T09:47:22.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:22.808+0000 7ff26f200640 1 -- 172.21.15.5:0/2796154517 shutdown_connections 2024-09-10T09:47:22.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:22.808+0000 7ff26f200640 1 --2- 172.21.15.5:0/2796154517 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7ff24c0a2c00 0x7ff24c0a50c0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:22.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:22.808+0000 7ff26f200640 1 --2- 172.21.15.5:0/2796154517 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff26810e850 0x7ff268103f40 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:22.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:22.808+0000 7ff26f200640 1 --2- 172.21.15.5:0/2796154517 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff268102cc0 0x7ff268103a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:22.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:22.808+0000 7ff26f200640 1 --2- 172.21.15.5:0/2796154517 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff268100610 0x7ff2681034c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:22.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:22.808+0000 7ff26f200640 1 -- 172.21.15.5:0/2796154517 >> 172.21.15.5:0/2796154517 conn(0x7ff268077210 msgr2=0x7ff26810fe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:47:22.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:22.808+0000 7ff26f200640 1 -- 172.21.15.5:0/2796154517 shutdown_connections 2024-09-10T09:47:22.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:22.808+0000 7ff26f200640 1 -- 172.21.15.5:0/2796154517 wait complete. 2024-09-10T09:47:23.078 INFO:tasks.workunit.client.0.smithi005.stdout: 1 551654 6.43 MB/sec execute 304 sec latency 26.171 ms 2024-09-10T09:47:23.709 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24542 quiesce db / --timeout 90 --expiration 144.6 --await-for 45.0' 2024-09-10T09:47:23.710 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24542 quiesce db / --timeout 90 --expiration 144.6 --await-for 45.0 2024-09-10T09:47:24.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:47:23 smithi005 ceph-mon[30350]: pgmap v664: 129 pgs: 129 active+clean; 206 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.9 MiB/s wr, 203 op/s 2024-09-10T09:47:24.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:47:23 smithi005 ceph-mon[30350]: from='client.? 172.21.15.5:0/2796154517' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:47:24.078 INFO:tasks.workunit.client.0.smithi005.stdout: 1 554442 6.46 MB/sec execute 305 sec latency 10.704 ms 2024-09-10T09:47:24.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:47:23 smithi159 ceph-mon[35513]: pgmap v664: 129 pgs: 129 active+clean; 206 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.9 MiB/s wr, 203 op/s 2024-09-10T09:47:24.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:47:23 smithi159 ceph-mon[35513]: from='client.? 172.21.15.5:0/2796154517' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:47:24.115 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:47:24.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:47:23 smithi102 ceph-mon[35442]: pgmap v664: 129 pgs: 129 active+clean; 206 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.9 MiB/s wr, 203 op/s 2024-09-10T09:47:24.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:47:23 smithi102 ceph-mon[35442]: from='client.? 172.21.15.5:0/2796154517' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:47:24.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:24.757+0000 7f1d09400640 1 -- 172.21.15.5:0/316252402 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1d041047e0 msgr2=0x7f1d04108830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:47:24.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:24.757+0000 7f1d09400640 1 --2- 172.21.15.5:0/316252402 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1d041047e0 0x7f1d04108830 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f1cec009900 tx=0x7f1cec02a410 comp rx=0 tx=0).stop 2024-09-10T09:47:24.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:24.757+0000 7f1d09400640 1 -- 172.21.15.5:0/316252402 shutdown_connections 2024-09-10T09:47:24.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:24.757+0000 7f1d09400640 1 --2- 172.21.15.5:0/316252402 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1d04108f60 0x7f1d04110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:24.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:24.757+0000 7f1d09400640 1 --2- 172.21.15.5:0/316252402 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1d041047e0 0x7f1d04108830 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:24.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:24.757+0000 7f1d09400640 1 --2- 172.21.15.5:0/316252402 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1d04103e30 0x7f1d04104210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:24.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:24.757+0000 7f1d09400640 1 -- 172.21.15.5:0/316252402 >> 172.21.15.5:0/316252402 conn(0x7f1d040ff8d0 msgr2=0x7f1d04101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:47:24.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:24.758+0000 7f1d09400640 1 -- 172.21.15.5:0/316252402 shutdown_connections 2024-09-10T09:47:24.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:24.758+0000 7f1d09400640 1 -- 172.21.15.5:0/316252402 wait complete. 2024-09-10T09:47:24.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:24.758+0000 7f1d09400640 1 Processor -- start 2024-09-10T09:47:24.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:24.759+0000 7f1d09400640 1 -- start start 2024-09-10T09:47:24.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:24.759+0000 7f1d09400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1d04103e30 0x7f1d0419a450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:47:24.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:24.759+0000 7f1d09400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1d041047e0 0x7f1d0419a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:47:24.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:24.759+0000 7f1d09400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1d04108f60 0x7f1d0419ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:47:24.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:24.759+0000 7f1d09400640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1d04114010 con 0x7f1d04108f60 2024-09-10T09:47:24.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:24.759+0000 7f1d09400640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f1d04113e90 con 0x7f1d041047e0 2024-09-10T09:47:24.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:24.759+0000 7f1d09400640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f1d04114190 con 0x7f1d04103e30 2024-09-10T09:47:24.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:24.759+0000 7f1d03e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1d04103e30 0x7f1d0419a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:47:24.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:24.759+0000 7f1cfbe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1d041047e0 0x7f1d0419a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:47:24.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:24.759+0000 7f1d03e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1d04103e30 0x7f1d0419a450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:53640/0 (socket says 172.21.15.5:53640) 2024-09-10T09:47:24.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:24.759+0000 7f1d03e00640 1 -- 172.21.15.5:0/2808297169 learned_addr learned my addr 172.21.15.5:0/2808297169 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:47:24.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:24.759+0000 7f1d08a00640 1 --2- 172.21.15.5:0/2808297169 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1d04108f60 0x7f1d0419ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:47:24.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:24.760+0000 7f1d03e00640 1 -- 172.21.15.5:0/2808297169 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1d041047e0 msgr2=0x7f1d0419a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:47:24.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:24.760+0000 7f1d03e00640 1 --2- 172.21.15.5:0/2808297169 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1d041047e0 0x7f1d0419a990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:24.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:24.760+0000 7f1d03e00640 1 -- 172.21.15.5:0/2808297169 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1d04108f60 msgr2=0x7f1d0419ed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:47:24.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:24.760+0000 7f1d03e00640 1 --2- 172.21.15.5:0/2808297169 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1d04108f60 0x7f1d0419ed20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:24.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:24.760+0000 7f1d03e00640 1 -- 172.21.15.5:0/2808297169 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d0419f400 con 0x7f1d04103e30 2024-09-10T09:47:24.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:24.760+0000 7f1d03e00640 1 --2- 172.21.15.5:0/2808297169 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1d04103e30 0x7f1d0419a450 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f1cf400e9f0 tx=0x7f1cf400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:47:24.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:24.760+0000 7f1d01600640 1 -- 172.21.15.5:0/2808297169 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cf400cd80 con 0x7f1d04103e30 2024-09-10T09:47:24.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:24.760+0000 7f1d01600640 1 -- 172.21.15.5:0/2808297169 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1cf4010910 con 0x7f1d04103e30 2024-09-10T09:47:24.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:24.760+0000 7f1d01600640 1 -- 172.21.15.5:0/2808297169 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cf4018990 con 0x7f1d04103e30 2024-09-10T09:47:24.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:24.760+0000 7f1d09400640 1 -- 172.21.15.5:0/2808297169 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d0419f6f0 con 0x7f1d04103e30 2024-09-10T09:47:24.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:24.761+0000 7f1d09400640 1 -- 172.21.15.5:0/2808297169 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d041a6fd0 con 0x7f1d04103e30 2024-09-10T09:47:24.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:24.764+0000 7f1d01600640 1 -- 172.21.15.5:0/2808297169 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f1cf4010ac0 con 0x7f1d04103e30 2024-09-10T09:47:24.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:24.764+0000 7f1d0d4d3740 1 Processor -- start 2024-09-10T09:47:24.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:24.764+0000 7f1d0d4d3740 1 -- start start 2024-09-10T09:47:24.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:24.765+0000 7f1d0d4d3740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55edfd643560 0x55edfd643940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:47:24.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:24.765+0000 7f1d0d4d3740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55edfd643e80 0x55edfd64e200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:47:24.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:24.765+0000 7f1d0d4d3740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55edfd64e740 0x55edfd650b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:47:24.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:24.765+0000 7f1d0d4d3740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55edfd5a9530 con 0x55edfd643e80 2024-09-10T09:47:24.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:24.765+0000 7f1d0d4d3740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55edfd550860 con 0x55edfd64e740 2024-09-10T09:47:24.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:24.765+0000 7f1d0d4d3740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55edfd5a54f0 con 0x55edfd643560 2024-09-10T09:47:24.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:24.765+0000 7f1cfbe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55edfd64e740 0x55edfd650b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:47:24.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:24.765+0000 7f1cfbe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55edfd64e740 0x55edfd650b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:40540/0 (socket says 172.21.15.5:40540) 2024-09-10T09:47:24.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:24.765+0000 7f1cfbe00640 1 -- 172.21.15.5:0/956702126 learned_addr learned my addr 172.21.15.5:0/956702126 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:47:24.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:24.765+0000 7f1d08a00640 1 --2- 172.21.15.5:0/956702126 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55edfd643560 0x55edfd643940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:47:24.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:24.765+0000 7f1d03e00640 1 --2- 172.21.15.5:0/956702126 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55edfd643e80 0x55edfd64e200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:47:24.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:24.766+0000 7f1d01600640 1 --2- 172.21.15.5:0/2808297169 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f1ce80a2c00 0x7f1ce80a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:47:24.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:24.766+0000 7f1d08a00640 1 -- 172.21.15.5:0/956702126 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55edfd64e740 msgr2=0x55edfd650b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:47:24.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:24.766+0000 7f1d08a00640 1 --2- 172.21.15.5:0/956702126 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55edfd64e740 0x55edfd650b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:24.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:24.766+0000 7f1d08a00640 1 -- 172.21.15.5:0/956702126 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55edfd643e80 msgr2=0x55edfd64e200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:47:24.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:24.766+0000 7f1d08a00640 1 --2- 172.21.15.5:0/956702126 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55edfd643e80 0x55edfd64e200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:24.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:24.766+0000 7f1d08a00640 1 -- 172.21.15.5:0/956702126 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55edfd31ab90 con 0x55edfd643560 2024-09-10T09:47:24.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:24.766+0000 7f1d01600640 1 -- 172.21.15.5:0/2808297169 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f1cf40cf700 con 0x7f1d04103e30 2024-09-10T09:47:24.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:24.766+0000 7f1d08a00640 1 --2- 172.21.15.5:0/2808297169 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f1ce80a2c00 0x7f1ce80a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:47:24.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:24.766+0000 7f1d08a00640 1 --2- 172.21.15.5:0/956702126 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55edfd643560 0x55edfd643940 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f1cfc009f00 tx=0x7f1cfc011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5ff1d9747e30e230 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:47:24.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:24.767+0000 7f1cf9200640 1 -- 172.21.15.5:0/956702126 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cfc004250 con 0x55edfd643560 2024-09-10T09:47:24.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:24.767+0000 7f1cf9200640 1 -- 172.21.15.5:0/956702126 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1cfc01be30 con 0x55edfd643560 2024-09-10T09:47:24.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:24.767+0000 7f1cf9200640 1 -- 172.21.15.5:0/956702126 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cfc019b90 con 0x55edfd643560 2024-09-10T09:47:24.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:24.768+0000 7f1d0d4d3740 1 -- 172.21.15.5:0/956702126 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55edfd643560 msgr2=0x55edfd643940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:47:24.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:24.768+0000 7f1d0d4d3740 1 --2- 172.21.15.5:0/956702126 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55edfd643560 0x55edfd643940 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f1cfc009f00 tx=0x7f1cfc011810 comp rx=0 tx=0).stop 2024-09-10T09:47:24.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:24.768+0000 7f1d08a00640 1 --2- 172.21.15.5:0/2808297169 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f1ce80a2c00 0x7f1ce80a50c0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f1cfc020000 tx=0x7f1cfc021000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:47:24.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:24.768+0000 7f1d0d4d3740 1 -- 172.21.15.5:0/956702126 shutdown_connections 2024-09-10T09:47:24.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:24.768+0000 7f1d0d4d3740 1 --2- 172.21.15.5:0/956702126 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55edfd64e740 0x55edfd650b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:24.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:24.768+0000 7f1d0d4d3740 1 --2- 172.21.15.5:0/956702126 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55edfd643e80 0x55edfd64e200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:24.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:24.768+0000 7f1d0d4d3740 1 --2- 172.21.15.5:0/956702126 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55edfd643560 0x55edfd643940 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:24.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:24.768+0000 7f1d0d4d3740 1 -- 172.21.15.5:0/956702126 >> 172.21.15.5:0/956702126 conn(0x55edfd63faf0 msgr2=0x55edfd63fed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:47:24.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:24.768+0000 7f1d0d4d3740 1 -- 172.21.15.5:0/956702126 shutdown_connections 2024-09-10T09:47:24.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:24.768+0000 7f1d0d4d3740 1 -- 172.21.15.5:0/956702126 wait complete. 2024-09-10T09:47:24.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:24.769+0000 7f1d0d4d3740 1 -- start start 2024-09-10T09:47:24.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:24.769+0000 7f1d0d4d3740 1 Processor -- start 2024-09-10T09:47:24.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:24.769+0000 7f1d0d4d3740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55edfd643560 0x55edfd64b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:47:24.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:24.769+0000 7f1d0d4d3740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55edfd643e80 0x55edfd64bb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:47:24.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:24.769+0000 7f1d0d4d3740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55edfd64e740 0x55edfd64c070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:47:24.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:24.769+0000 7f1d0d4d3740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55edfd651310 con 0x55edfd643e80 2024-09-10T09:47:24.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:24.769+0000 7f1d0d4d3740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55edfd563e60 con 0x55edfd643560 2024-09-10T09:47:24.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:24.769+0000 7f1d0d4d3740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55edfd651490 con 0x55edfd64e740 2024-09-10T09:47:24.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:24.770+0000 7f1d03e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55edfd643560 0x55edfd64b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:47:24.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:24.770+0000 7f1cfbe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55edfd643e80 0x55edfd64bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:47:24.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:24.770+0000 7f1d0d4d3740 10 client.0 _ll_register_callbacks cb 0x55edfd568cc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:47:24.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:24.770+0000 7f1cfbe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55edfd643e80 0x55edfd64bb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47662/0 (socket says 172.21.15.5:47662) 2024-09-10T09:47:24.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:24.770+0000 7f1d08a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55edfd64e740 0x55edfd64c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:47:24.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:24.770+0000 7f1d08a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55edfd64e740 0x55edfd64c070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:53658/0 (socket says 172.21.15.5:53658) 2024-09-10T09:47:24.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:24.770+0000 7f1d03e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55edfd643560 0x55edfd64b5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:40546/0 (socket says 172.21.15.5:40546) 2024-09-10T09:47:24.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:24.770+0000 7f1d08a00640 1 -- 172.21.15.5:0/1298905600 learned_addr learned my addr 172.21.15.5:0/1298905600 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:47:24.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:24.770+0000 7f1d03e00640 1 -- 172.21.15.5:0/1298905600 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55edfd64e740 msgr2=0x55edfd64c070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:47:24.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:24.770+0000 7f1d03e00640 1 --2- 172.21.15.5:0/1298905600 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55edfd64e740 0x55edfd64c070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:24.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:24.770+0000 7f1d03e00640 1 -- 172.21.15.5:0/1298905600 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55edfd643e80 msgr2=0x55edfd64bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:47:24.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:24.770+0000 7f1d03e00640 1 --2- 172.21.15.5:0/1298905600 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55edfd643e80 0x55edfd64bb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:24.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:24.770+0000 7f1d03e00640 1 -- 172.21.15.5:0/1298905600 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55edfd56f990 con 0x55edfd643560 2024-09-10T09:47:24.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:24.771+0000 7f1d03e00640 1 --2- 172.21.15.5:0/1298905600 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55edfd643560 0x55edfd64b5f0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f1cf40a4250 tx=0x7f1cf409b520 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c469d8d3f090833e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:47:24.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:24.771+0000 7f1cf1600640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-09-10T09:47:24.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:24.771+0000 7f1cf1600640 1 -- 172.21.15.5:0/1298905600 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cf409bbc0 con 0x55edfd643560 2024-09-10T09:47:24.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:24.771+0000 7f1cf1600640 1 -- 172.21.15.5:0/1298905600 <== mon.2 v2:172.21.15.102:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f1cf40a1650 con 0x55edfd643560 2024-09-10T09:47:24.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:24.771+0000 7f1d0d4d3740 1 -- 172.21.15.5:0/1298905600 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1cfc01be30 con 0x55edfd643560 2024-09-10T09:47:24.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:24.771+0000 7f1cf1600640 1 -- 172.21.15.5:0/1298905600 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f1cf409fec0 con 0x55edfd643560 2024-09-10T09:47:24.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:24.771+0000 7f1d0d4d3740 10 client.27185 fetch_fsmap learned FSMap version 35 2024-09-10T09:47:24.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:24.771+0000 7f1d0d4d3740 1 -- 172.21.15.5:0/1298905600 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55edfd65cf60 con 0x55edfd643560 2024-09-10T09:47:24.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:24.771+0000 7f1cf1600640 1 -- 172.21.15.5:0/1298905600 <== mon.2 v2:172.21.15.102:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1cfc01be30 con 0x55edfd643560 2024-09-10T09:47:24.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:24.772+0000 7f1cf1600640 1 -- 172.21.15.5:0/1298905600 <== mon.2 v2:172.21.15.102:3300/0 5 ==== fsmap(e 35) ==== 8374+0+0 (secure 0 0 0) 0x7f1cf409b010 con 0x55edfd643560 2024-09-10T09:47:24.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:24.772+0000 7f1d0d4d3740 10 client.27185 fetch_fsmap finished waiting for FSMap version 35 2024-09-10T09:47:24.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:24.772+0000 7f1d0d4d3740 10 client.27185 resolve_mds: validated gid 24542 aka daemon mds.b 2024-09-10T09:47:24.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:24.772+0000 7f1d0d4d3740 20 client.27185 populate_metadata read hostname 'smithi005' 2024-09-10T09:47:24.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:24.772+0000 7f1d0d4d3740 1 --2- 172.21.15.5:0/1298905600 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55edfd65e420 0x55edfd6608b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:47:24.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:24.772+0000 7f1d0d4d3740 4 client.27185 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:47:24.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:24.772+0000 7f1d0d4d3740 1 -- 172.21.15.5:0/1298905600 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55edfd54c310 con 0x55edfd65e420 2024-09-10T09:47:24.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:24.772+0000 7f1cfbe00640 1 --2- 172.21.15.5:0/1298905600 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55edfd65e420 0x55edfd6608b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:47:24.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:24.773+0000 7f1cfbe00640 1 --2- 172.21.15.5:0/1298905600 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55edfd65e420 0x55edfd6608b0 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d0f1e799af73e5af server_cookie=4a278a4d2b34b3eb in_seq=0 out_seq=0 2024-09-10T09:47:24.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:24.773+0000 7f1cf1600640 10 client.27185 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T09:47:24.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:24.773+0000 7f1cf1600640 1 -- 172.21.15.5:0/1298905600 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55edfd54c310 con 0x55edfd65e420 2024-09-10T09:47:24.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:24.773+0000 7f1cf1600640 10 client.27185 handle_command_reply: tid=0 2024-09-10T09:47:24.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:24.773+0000 7f1d0d4d3740 1 client.27185 shutdown 2024-09-10T09:47:24.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:24.774+0000 7f1d0d4d3740 1 -- 172.21.15.5:0/1298905600 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55edfd643560 msgr2=0x55edfd64b5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:47:24.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:24.774+0000 7f1d0d4d3740 1 --2- 172.21.15.5:0/1298905600 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55edfd643560 0x55edfd64b5f0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f1cf40a4250 tx=0x7f1cf409b520 comp rx=0 tx=0).stop 2024-09-10T09:47:24.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:24.774+0000 7f1d0d4d3740 1 -- 172.21.15.5:0/1298905600 shutdown_connections 2024-09-10T09:47:24.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:24.774+0000 7f1d0d4d3740 1 --2- 172.21.15.5:0/1298905600 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55edfd65e420 0x55edfd6608b0 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:24.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:24.774+0000 7f1d0d4d3740 1 --2- 172.21.15.5:0/1298905600 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55edfd64e740 0x55edfd64c070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:24.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:24.774+0000 7f1cf1600640 0 client.27185 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T09:47:24.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:24.774+0000 7f1d0d4d3740 1 --2- 172.21.15.5:0/1298905600 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55edfd643e80 0x55edfd64bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:24.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:24.774+0000 7f1d0d4d3740 1 --2- 172.21.15.5:0/1298905600 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55edfd643560 0x55edfd64b5f0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:24.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:24.774+0000 7f1d0d4d3740 1 -- 172.21.15.5:0/1298905600 >> 172.21.15.5:0/1298905600 conn(0x55edfd63faf0 msgr2=0x55edfd6457d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:47:24.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:24.774+0000 7f1d0d4d3740 1 -- 172.21.15.5:0/1298905600 shutdown_connections 2024-09-10T09:47:24.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:24.774+0000 7f1d0d4d3740 1 -- 172.21.15.5:0/1298905600 wait complete. 2024-09-10T09:47:24.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:24.774+0000 7f1d0d4d3740 20 client.27185 trim_cache size 0 max 16384 2024-09-10T09:47:24.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:24.794+0000 7f1d0d4d3740 1 Processor -- start 2024-09-10T09:47:24.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:24.795+0000 7f1d0d4d3740 1 -- start start 2024-09-10T09:47:24.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:24.795+0000 7f1d0d4d3740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55edfd63faf0 0x55edfd63eb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:47:24.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:24.795+0000 7f1d0d4d3740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55edfd643560 0x55edfd6661d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:47:24.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:24.795+0000 7f1d0d4d3740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55edfd643e80 0x55edfd666710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:47:24.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:24.795+0000 7f1d0d4d3740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1d041b0ed0 con 0x55edfd63faf0 2024-09-10T09:47:24.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:24.795+0000 7f1d0d4d3740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f1d041b2a40 con 0x55edfd643560 2024-09-10T09:47:24.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:24.795+0000 7f1d0d4d3740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f1d041b15a0 con 0x55edfd643e80 2024-09-10T09:47:24.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:24.795+0000 7f1d03e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55edfd63faf0 0x55edfd63eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:47:24.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:24.795+0000 7f1cfbe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55edfd643560 0x55edfd6661d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:47:24.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:24.795+0000 7f1d08a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55edfd643e80 0x55edfd666710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:47:24.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:24.795+0000 7f1cfbe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55edfd643560 0x55edfd6661d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:40548/0 (socket says 172.21.15.5:40548) 2024-09-10T09:47:24.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:24.795+0000 7f1d03e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55edfd63faf0 0x55edfd63eb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47676/0 (socket says 172.21.15.5:47676) 2024-09-10T09:47:24.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:24.795+0000 7f1cfbe00640 1 -- 172.21.15.5:0/1884894307 learned_addr learned my addr 172.21.15.5:0/1884894307 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:47:24.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:24.796+0000 7f1d08a00640 1 -- 172.21.15.5:0/1884894307 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55edfd643560 msgr2=0x55edfd6661d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:47:24.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:24.796+0000 7f1d08a00640 1 --2- 172.21.15.5:0/1884894307 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55edfd643560 0x55edfd6661d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:24.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:24.796+0000 7f1d08a00640 1 -- 172.21.15.5:0/1884894307 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55edfd63faf0 msgr2=0x55edfd63eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:47:24.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:24.796+0000 7f1d08a00640 1 --2- 172.21.15.5:0/1884894307 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55edfd63faf0 0x55edfd63eb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:24.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:24.796+0000 7f1d08a00640 1 -- 172.21.15.5:0/1884894307 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55edfd63f920 con 0x55edfd643e80 2024-09-10T09:47:24.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:24.796+0000 7f1d03e00640 1 --2- 172.21.15.5:0/1884894307 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55edfd63faf0 0x55edfd63eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:47:24.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:24.796+0000 7f1cfbe00640 1 --2- 172.21.15.5:0/1884894307 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55edfd643560 0x55edfd6661d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:47:24.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:24.796+0000 7f1d08a00640 1 --2- 172.21.15.5:0/1884894307 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55edfd643e80 0x55edfd666710 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f1cfc0198d0 tx=0x7f1cfc027000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3ee0f22dc8f44f18 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:47:24.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:24.796+0000 7f1cf3e00640 1 -- 172.21.15.5:0/1884894307 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cfc036070 con 0x55edfd643e80 2024-09-10T09:47:24.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:24.796+0000 7f1cf3e00640 1 -- 172.21.15.5:0/1884894307 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1cfc01be30 con 0x55edfd643e80 2024-09-10T09:47:24.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:24.796+0000 7f1cf3e00640 1 -- 172.21.15.5:0/1884894307 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cfc029040 con 0x55edfd643e80 2024-09-10T09:47:24.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:24.797+0000 7f1d0d4d3740 1 -- 172.21.15.5:0/1884894307 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55edfd643e80 msgr2=0x55edfd666710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:47:24.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:24.797+0000 7f1d0d4d3740 1 --2- 172.21.15.5:0/1884894307 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55edfd643e80 0x55edfd666710 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f1cfc0198d0 tx=0x7f1cfc027000 comp rx=0 tx=0).stop 2024-09-10T09:47:24.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:24.797+0000 7f1d0d4d3740 1 -- 172.21.15.5:0/1884894307 shutdown_connections 2024-09-10T09:47:24.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:24.797+0000 7f1d0d4d3740 1 --2- 172.21.15.5:0/1884894307 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55edfd643e80 0x55edfd666710 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:24.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:24.797+0000 7f1d0d4d3740 1 --2- 172.21.15.5:0/1884894307 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55edfd643560 0x55edfd6661d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:24.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:24.797+0000 7f1d0d4d3740 1 --2- 172.21.15.5:0/1884894307 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55edfd63faf0 0x55edfd63eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:24.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:24.797+0000 7f1d0d4d3740 1 -- 172.21.15.5:0/1884894307 >> 172.21.15.5:0/1884894307 conn(0x55edfd640570 msgr2=0x55edfd63e450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:47:24.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:24.797+0000 7f1d0d4d3740 1 -- 172.21.15.5:0/1884894307 shutdown_connections 2024-09-10T09:47:24.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:24.798+0000 7f1d0d4d3740 1 -- 172.21.15.5:0/1884894307 wait complete. 2024-09-10T09:47:24.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:24.798+0000 7f1d0d4d3740 1 -- start start 2024-09-10T09:47:24.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:24.798+0000 7f1d0d4d3740 1 Processor -- start 2024-09-10T09:47:24.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:24.799+0000 7f1d0d4d3740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55edfd63faf0 0x55edfd656980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:47:24.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:24.799+0000 7f1d0d4d3740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55edfd643560 0x55edfd656ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:47:24.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:24.799+0000 7f1d0d4d3740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55edfd643e80 0x55edfd653760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:47:24.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:24.799+0000 7f1d0d4d3740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1d041b2340 con 0x55edfd63faf0 2024-09-10T09:47:24.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:24.799+0000 7f1d0d4d3740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f1d041b1c70 con 0x55edfd643e80 2024-09-10T09:47:24.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:24.799+0000 7f1d0d4d3740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f1d041aeeb0 con 0x55edfd643560 2024-09-10T09:47:24.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:24.799+0000 7f1d0d4d3740 10 client.0 _ll_register_callbacks cb 0x7f1cf400cf20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:47:24.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:24.799+0000 7f1cfbe00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55edfd643560 0x55edfd656ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:47:24.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:24.799+0000 7f1d03e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55edfd63faf0 0x55edfd656980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:47:24.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:24.799+0000 7f1cfbe00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55edfd643560 0x55edfd656ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:53678/0 (socket says 172.21.15.5:53678) 2024-09-10T09:47:24.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:24.799+0000 7f1d08a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55edfd643e80 0x55edfd653760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:47:24.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:24.799+0000 7f1cfbe00640 1 -- 172.21.15.5:0/3983757442 learned_addr learned my addr 172.21.15.5:0/3983757442 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:47:24.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:24.799+0000 7f1d08a00640 1 -- 172.21.15.5:0/3983757442 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55edfd643560 msgr2=0x55edfd656ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:47:24.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:24.799+0000 7f1d08a00640 1 --2- 172.21.15.5:0/3983757442 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55edfd643560 0x55edfd656ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:24.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:24.799+0000 7f1d08a00640 1 -- 172.21.15.5:0/3983757442 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55edfd63faf0 msgr2=0x55edfd656980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:47:24.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:24.799+0000 7f1d08a00640 1 --2- 172.21.15.5:0/3983757442 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55edfd63faf0 0x55edfd656980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:24.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:24.799+0000 7f1d08a00640 1 -- 172.21.15.5:0/3983757442 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55edfd653f90 con 0x55edfd643e80 2024-09-10T09:47:24.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:24.799+0000 7f1cfbe00640 1 --2- 172.21.15.5:0/3983757442 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55edfd643560 0x55edfd656ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:47:24.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:24.799+0000 7f1d03e00640 1 --2- 172.21.15.5:0/3983757442 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55edfd63faf0 0x55edfd656980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:47:24.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:24.800+0000 7f1d08a00640 1 --2- 172.21.15.5:0/3983757442 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55edfd643e80 0x55edfd653760 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f1cfc0198d0 tx=0x7f1cfc0097d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e46e97ac9f98d9d0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:47:24.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:24.800+0000 7f1cf1600640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-09-10T09:47:24.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:24.800+0000 7f1cf1600640 1 -- 172.21.15.5:0/3983757442 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cfc03f030 con 0x55edfd643e80 2024-09-10T09:47:24.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:24.800+0000 7f1cf1600640 1 -- 172.21.15.5:0/3983757442 <== mon.2 v2:172.21.15.102:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f1cfc019460 con 0x55edfd643e80 2024-09-10T09:47:24.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:24.800+0000 7f1d0d4d3740 1 -- 172.21.15.5:0/3983757442 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1cfc01be30 con 0x55edfd643e80 2024-09-10T09:47:24.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:24.800+0000 7f1cf1600640 1 -- 172.21.15.5:0/3983757442 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f1cfc04c050 con 0x55edfd643e80 2024-09-10T09:47:24.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:24.801+0000 7f1d0d4d3740 10 client.27191 fetch_fsmap learned FSMap version 35 2024-09-10T09:47:24.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:24.801+0000 7f1d0d4d3740 1 -- 172.21.15.5:0/3983757442 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55edfd6c9d30 con 0x55edfd643e80 2024-09-10T09:47:24.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:24.801+0000 7f1cf1600640 1 -- 172.21.15.5:0/3983757442 <== mon.2 v2:172.21.15.102:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1cfc01be30 con 0x55edfd643e80 2024-09-10T09:47:24.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:24.801+0000 7f1cf1600640 1 -- 172.21.15.5:0/3983757442 <== mon.2 v2:172.21.15.102:3300/0 5 ==== fsmap(e 35) ==== 8374+0+0 (secure 0 0 0) 0x7f1cfc0464e0 con 0x55edfd643e80 2024-09-10T09:47:24.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:24.801+0000 7f1d0d4d3740 10 client.27191 fetch_fsmap finished waiting for FSMap version 35 2024-09-10T09:47:24.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:24.801+0000 7f1d0d4d3740 10 client.27191 resolve_mds: validated gid 24542 aka daemon mds.b 2024-09-10T09:47:24.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:24.801+0000 7f1d0d4d3740 20 client.27191 populate_metadata read hostname 'smithi005' 2024-09-10T09:47:24.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:24.801+0000 7f1d0d4d3740 1 --2- 172.21.15.5:0/3983757442 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55edfd64e740 0x55edfd6cc300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:47:24.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:24.801+0000 7f1d0d4d3740 4 client.27191 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 144.6, "await_for": 45.0}] 2024-09-10T09:47:24.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:24.801+0000 7f1d0d4d3740 1 -- 172.21.15.5:0/3983757442 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 144.6, "await_for": 45.0}) -- 0x55edfd5b6c60 con 0x55edfd64e740 2024-09-10T09:47:24.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:24.801+0000 7f1d03e00640 1 --2- 172.21.15.5:0/3983757442 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55edfd64e740 0x55edfd6cc300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:47:24.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:24.802+0000 7f1d03e00640 1 --2- 172.21.15.5:0/3983757442 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55edfd64e740 0x55edfd6cc300 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=df2d519733dd91b5 server_cookie=a2f2615821e2dfc5 in_seq=0 out_seq=0 2024-09-10T09:47:24.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:24.802+0000 7f1cf1600640 10 client.27191 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T09:47:24.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:24.905+0000 7f1cf1600640 1 -- 172.21.15.5:0/3983757442 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55edfd5b6c60 con 0x55edfd64e740 2024-09-10T09:47:24.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:24.905+0000 7f1cf1600640 10 client.27191 handle_command_reply: tid=0 2024-09-10T09:47:24.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:24.905+0000 7f1d0d4d3740 1 client.27191 shutdown 2024-09-10T09:47:24.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:24.905+0000 7f1d0d4d3740 1 -- 172.21.15.5:0/3983757442 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55edfd643e80 msgr2=0x55edfd653760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:47:24.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:24.905+0000 7f1d0d4d3740 1 --2- 172.21.15.5:0/3983757442 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55edfd643e80 0x55edfd653760 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f1cfc0198d0 tx=0x7f1cfc0097d0 comp rx=0 tx=0).stop 2024-09-10T09:47:24.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:24.905+0000 7f1d0d4d3740 1 -- 172.21.15.5:0/3983757442 shutdown_connections 2024-09-10T09:47:24.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:24.905+0000 7f1d0d4d3740 1 --2- 172.21.15.5:0/3983757442 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55edfd64e740 0x55edfd6cc300 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:24.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:24.906+0000 7f1d0d4d3740 1 --2- 172.21.15.5:0/3983757442 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55edfd643e80 0x55edfd653760 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:24.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:24.906+0000 7f1d0d4d3740 1 --2- 172.21.15.5:0/3983757442 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55edfd643560 0x55edfd656ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:24.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:24.906+0000 7f1cf1600640 0 client.27191 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T09:47:24.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:24.906+0000 7f1d0d4d3740 1 --2- 172.21.15.5:0/3983757442 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55edfd63faf0 0x55edfd656980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:24.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:24.906+0000 7f1d0d4d3740 1 -- 172.21.15.5:0/3983757442 >> 172.21.15.5:0/3983757442 conn(0x55edfd640570 msgr2=0x55edfd64b5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:47:24.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:24.906+0000 7f1d0d4d3740 1 -- 172.21.15.5:0/3983757442 shutdown_connections 2024-09-10T09:47:24.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:24.906+0000 7f1d0d4d3740 1 -- 172.21.15.5:0/3983757442 wait complete. 2024-09-10T09:47:24.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:24.906+0000 7f1d0d4d3740 20 client.27191 trim_cache size 0 max 16384 2024-09-10T09:47:24.909 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-10T09:47:24.909 INFO:teuthology.orchestra.run.smithi005.stdout: "epoch": 35, 2024-09-10T09:47:24.909 INFO:teuthology.orchestra.run.smithi005.stdout: "leader": 24542, 2024-09-10T09:47:24.909 INFO:teuthology.orchestra.run.smithi005.stdout: "set_version": 30, 2024-09-10T09:47:24.909 INFO:teuthology.orchestra.run.smithi005.stdout: "sets": { 2024-09-10T09:47:24.909 INFO:teuthology.orchestra.run.smithi005.stdout: "82aad4ea": { 2024-09-10T09:47:24.909 INFO:teuthology.orchestra.run.smithi005.stdout: "version": 30, 2024-09-10T09:47:24.909 INFO:teuthology.orchestra.run.smithi005.stdout: "age_ref": 0.0, 2024-09-10T09:47:24.909 INFO:teuthology.orchestra.run.smithi005.stdout: "state": { 2024-09-10T09:47:24.910 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "QUIESCED", 2024-09-10T09:47:24.910 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0 2024-09-10T09:47:24.910 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T09:47:24.910 INFO:teuthology.orchestra.run.smithi005.stdout: "timeout": 90.0, 2024-09-10T09:47:24.910 INFO:teuthology.orchestra.run.smithi005.stdout: "expiration": 144.6, 2024-09-10T09:47:24.910 INFO:teuthology.orchestra.run.smithi005.stdout: "members": { 2024-09-10T09:47:24.910 INFO:teuthology.orchestra.run.smithi005.stdout: "file:/": { 2024-09-10T09:47:24.910 INFO:teuthology.orchestra.run.smithi005.stdout: "excluded": false, 2024-09-10T09:47:24.910 INFO:teuthology.orchestra.run.smithi005.stdout: "state": { 2024-09-10T09:47:24.910 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "QUIESCED", 2024-09-10T09:47:24.910 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0 2024-09-10T09:47:24.910 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T09:47:24.910 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T09:47:24.911 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T09:47:24.911 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T09:47:24.911 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T09:47:24.911 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-10T09:47:24.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:24.907+0000 7f1d09400640 1 -- 172.21.15.5:0/2808297169 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f1ce80a2c00 msgr2=0x7f1ce80a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:47:24.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:24.907+0000 7f1d09400640 1 --2- 172.21.15.5:0/2808297169 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f1ce80a2c00 0x7f1ce80a50c0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f1cfc020000 tx=0x7f1cfc021000 comp rx=0 tx=0).stop 2024-09-10T09:47:24.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:24.907+0000 7f1d09400640 1 -- 172.21.15.5:0/2808297169 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1d04103e30 msgr2=0x7f1d0419a450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:47:24.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:24.907+0000 7f1d09400640 1 --2- 172.21.15.5:0/2808297169 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1d04103e30 0x7f1d0419a450 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f1cf400e9f0 tx=0x7f1cf400eeb0 comp rx=0 tx=0).stop 2024-09-10T09:47:24.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:24.907+0000 7f1d09400640 1 -- 172.21.15.5:0/2808297169 shutdown_connections 2024-09-10T09:47:24.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:24.907+0000 7f1d09400640 1 --2- 172.21.15.5:0/2808297169 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f1ce80a2c00 0x7f1ce80a50c0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:24.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:24.907+0000 7f1d09400640 1 --2- 172.21.15.5:0/2808297169 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1d04108f60 0x7f1d0419ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:24.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:24.907+0000 7f1d09400640 1 --2- 172.21.15.5:0/2808297169 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1d041047e0 0x7f1d0419a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:24.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:24.907+0000 7f1d09400640 1 --2- 172.21.15.5:0/2808297169 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1d04103e30 0x7f1d0419a450 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:24.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:24.907+0000 7f1d09400640 1 -- 172.21.15.5:0/2808297169 >> 172.21.15.5:0/2808297169 conn(0x7f1d040ff8d0 msgr2=0x7f1d04100b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:47:24.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:24.907+0000 7f1d09400640 1 -- 172.21.15.5:0/2808297169 shutdown_connections 2024-09-10T09:47:24.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:24.907+0000 7f1d09400640 1 -- 172.21.15.5:0/2808297169 wait complete. 2024-09-10T09:47:25.078 INFO:tasks.workunit.client.0.smithi005.stdout: 1 555754 6.46 MB/sec execute 306 sec latency 169.540 ms 2024-09-10T09:47:25.774 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '82aad4ea', quiesce took 4.2 seconds. Will release after: 20.400000000000002 2024-09-10T09:47:26.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:47:25 smithi005 ceph-mon[30350]: pgmap v665: 129 pgs: 129 active+clean; 207 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.5 MiB/s wr, 163 op/s 2024-09-10T09:47:26.078 INFO:tasks.workunit.client.0.smithi005.stdout: 1 555754 6.44 MB/sec execute 307 sec latency 1169.674 ms 2024-09-10T09:47:26.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:47:25 smithi159 ceph-mon[35513]: pgmap v665: 129 pgs: 129 active+clean; 207 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.5 MiB/s wr, 163 op/s 2024-09-10T09:47:26.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:47:25 smithi102 ceph-mon[35442]: pgmap v665: 129 pgs: 129 active+clean; 207 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.5 MiB/s wr, 163 op/s 2024-09-10T09:47:27.079 INFO:tasks.workunit.client.0.smithi005.stdout: 1 555754 6.42 MB/sec execute 308 sec latency 2169.760 ms 2024-09-10T09:47:28.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:47:27 smithi005 ceph-mon[30350]: pgmap v666: 129 pgs: 129 active+clean; 212 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.7 MiB/s wr, 221 op/s 2024-09-10T09:47:28.078 INFO:tasks.workunit.client.0.smithi005.stdout: 1 555754 6.40 MB/sec execute 309 sec latency 3169.868 ms 2024-09-10T09:47:28.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:47:27 smithi159 ceph-mon[35513]: pgmap v666: 129 pgs: 129 active+clean; 212 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.7 MiB/s wr, 221 op/s 2024-09-10T09:47:28.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:47:27 smithi102 ceph-mon[35442]: pgmap v666: 129 pgs: 129 active+clean; 212 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.7 MiB/s wr, 221 op/s 2024-09-10T09:47:29.079 INFO:tasks.workunit.client.0.smithi005.stdout: 1 555754 6.38 MB/sec execute 310 sec latency 4169.989 ms 2024-09-10T09:47:29.718 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:47:29 smithi005 ceph-mon[30350]: pgmap v667: 129 pgs: 129 active+clean; 212 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.4 MiB/s wr, 172 op/s 2024-09-10T09:47:30.079 INFO:tasks.workunit.client.0.smithi005.stdout: 1 555754 6.36 MB/sec execute 311 sec latency 5170.111 ms 2024-09-10T09:47:30.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:47:29 smithi159 ceph-mon[35513]: pgmap v667: 129 pgs: 129 active+clean; 212 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.4 MiB/s wr, 172 op/s 2024-09-10T09:47:30.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:47:29 smithi102 ceph-mon[35442]: pgmap v667: 129 pgs: 129 active+clean; 212 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.4 MiB/s wr, 172 op/s 2024-09-10T09:47:31.079 INFO:tasks.workunit.client.0.smithi005.stdout: 1 555754 6.34 MB/sec execute 312 sec latency 6170.186 ms 2024-09-10T09:47:31.936 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:47:31 smithi102 ceph-mon[35442]: pgmap v668: 129 pgs: 129 active+clean; 213 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.3 MiB/s wr, 176 op/s 2024-09-10T09:47:32.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:47:31 smithi005 ceph-mon[30350]: pgmap v668: 129 pgs: 129 active+clean; 213 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.3 MiB/s wr, 176 op/s 2024-09-10T09:47:32.079 INFO:tasks.workunit.client.0.smithi005.stdout: 1 555754 6.31 MB/sec execute 313 sec latency 7170.303 ms 2024-09-10T09:47:32.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:47:31 smithi159 ceph-mon[35513]: pgmap v668: 129 pgs: 129 active+clean; 213 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.3 MiB/s wr, 176 op/s 2024-09-10T09:47:33.080 INFO:tasks.workunit.client.0.smithi005.stdout: 1 555754 6.29 MB/sec execute 314 sec latency 8170.475 ms 2024-09-10T09:47:34.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:47:33 smithi005 ceph-mon[30350]: pgmap v669: 129 pgs: 129 active+clean; 214 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 784 KiB/s rd, 958 KiB/s wr, 107 op/s 2024-09-10T09:47:34.079 INFO:tasks.workunit.client.0.smithi005.stdout: 1 555754 6.27 MB/sec execute 315 sec latency 9170.615 ms 2024-09-10T09:47:34.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:47:33 smithi159 ceph-mon[35513]: pgmap v669: 129 pgs: 129 active+clean; 214 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 784 KiB/s rd, 958 KiB/s wr, 107 op/s 2024-09-10T09:47:34.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:47:33 smithi102 ceph-mon[35442]: pgmap v669: 129 pgs: 129 active+clean; 214 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 784 KiB/s rd, 958 KiB/s wr, 107 op/s 2024-09-10T09:47:35.086 INFO:tasks.workunit.client.0.smithi005.stdout: 1 555754 6.25 MB/sec execute 316 sec latency 10177.448 ms 2024-09-10T09:47:35.343 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T09:47:35.722 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:47:36.028 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:47:35 smithi005 ceph-mon[30350]: pgmap v670: 129 pgs: 129 active+clean; 213 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 656 KiB/s rd, 979 KiB/s wr, 102 op/s 2024-09-10T09:47:36.086 INFO:tasks.workunit.client.0.smithi005.stdout: 1 555754 6.24 MB/sec execute 317 sec latency 11177.513 ms 2024-09-10T09:47:36.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:47:35 smithi159 ceph-mon[35513]: pgmap v670: 129 pgs: 129 active+clean; 213 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 656 KiB/s rd, 979 KiB/s wr, 102 op/s 2024-09-10T09:47:36.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:47:35 smithi102 ceph-mon[35442]: pgmap v670: 129 pgs: 129 active+clean; 213 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 656 KiB/s rd, 979 KiB/s wr, 102 op/s 2024-09-10T09:47:36.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:36.288+0000 7f3921c00640 1 -- 172.21.15.5:0/4212828093 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f391c10e280 msgr2=0x7f391c110670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:47:36.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:36.288+0000 7f3921c00640 1 --2- 172.21.15.5:0/4212828093 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f391c10e280 0x7f391c110670 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f3914009ca0 tx=0x7f391402a950 comp rx=0 tx=0).stop 2024-09-10T09:47:36.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:36.288+0000 7f3921c00640 1 -- 172.21.15.5:0/4212828093 shutdown_connections 2024-09-10T09:47:36.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:36.288+0000 7f3921c00640 1 --2- 172.21.15.5:0/4212828093 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f391c10e280 0x7f391c110670 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:36.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:36.288+0000 7f3921c00640 1 --2- 172.21.15.5:0/4212828093 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f391c074e30 0x7f391c10dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:36.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:36.288+0000 7f3921c00640 1 --2- 172.21.15.5:0/4212828093 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f391c074510 0x7f391c0748f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:36.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:36.288+0000 7f3921c00640 1 -- 172.21.15.5:0/4212828093 >> 172.21.15.5:0/4212828093 conn(0x7f391c0fd740 msgr2=0x7f391c0ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:47:36.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:36.288+0000 7f3921c00640 1 -- 172.21.15.5:0/4212828093 shutdown_connections 2024-09-10T09:47:36.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:36.288+0000 7f3921c00640 1 -- 172.21.15.5:0/4212828093 wait complete. 2024-09-10T09:47:36.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:36.289+0000 7f3921c00640 1 Processor -- start 2024-09-10T09:47:36.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:36.289+0000 7f3921c00640 1 -- start start 2024-09-10T09:47:36.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:36.290+0000 7f3921c00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f391c074510 0x7f391c19a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:47:36.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:36.290+0000 7f3921c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f391c074e30 0x7f391c19a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:47:36.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:36.290+0000 7f3921c00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f391c10e280 0x7f391c19ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:47:36.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:36.290+0000 7f3921c00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f391c113fb0 con 0x7f391c074510 2024-09-10T09:47:36.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:36.290+0000 7f3921c00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f391c113e30 con 0x7f391c074e30 2024-09-10T09:47:36.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:36.290+0000 7f3921c00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f391c114130 con 0x7f391c10e280 2024-09-10T09:47:36.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:36.290+0000 7f391b400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f391c074e30 0x7f391c19a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:47:36.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:36.290+0000 7f391be00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f391c074510 0x7f391c19a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:47:36.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:36.290+0000 7f391b400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f391c074e30 0x7f391c19a9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:43090/0 (socket says 172.21.15.5:43090) 2024-09-10T09:47:36.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:36.290+0000 7f391b400640 1 -- 172.21.15.5:0/3261404528 learned_addr learned my addr 172.21.15.5:0/3261404528 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:47:36.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:36.290+0000 7f3921200640 1 --2- 172.21.15.5:0/3261404528 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f391c10e280 0x7f391c19ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:47:36.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:36.291+0000 7f391b400640 1 -- 172.21.15.5:0/3261404528 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f391c10e280 msgr2=0x7f391c19ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:47:36.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:36.291+0000 7f391b400640 1 --2- 172.21.15.5:0/3261404528 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f391c10e280 0x7f391c19ed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:36.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:36.291+0000 7f391b400640 1 -- 172.21.15.5:0/3261404528 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f391c074510 msgr2=0x7f391c19a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:47:36.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:36.291+0000 7f391b400640 1 --2- 172.21.15.5:0/3261404528 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f391c074510 0x7f391c19a4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:36.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:36.291+0000 7f391b400640 1 -- 172.21.15.5:0/3261404528 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f391c19f460 con 0x7f391c074e30 2024-09-10T09:47:36.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:36.291+0000 7f391b400640 1 --2- 172.21.15.5:0/3261404528 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f391c074e30 0x7f391c19a9f0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f390800d950 tx=0x7f390800de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:47:36.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:36.291+0000 7f3918c00640 1 -- 172.21.15.5:0/3261404528 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3908017070 con 0x7f391c074e30 2024-09-10T09:47:36.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:36.291+0000 7f3921c00640 1 -- 172.21.15.5:0/3261404528 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f391c19f750 con 0x7f391c074e30 2024-09-10T09:47:36.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:36.291+0000 7f3918c00640 1 -- 172.21.15.5:0/3261404528 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3908009c30 con 0x7f391c074e30 2024-09-10T09:47:36.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:36.291+0000 7f3918c00640 1 -- 172.21.15.5:0/3261404528 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f390800f6b0 con 0x7f391c074e30 2024-09-10T09:47:36.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:36.291+0000 7f3921c00640 1 -- 172.21.15.5:0/3261404528 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f391c1a7030 con 0x7f391c074e30 2024-09-10T09:47:36.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:36.294+0000 7f3925c5f740 1 Processor -- start 2024-09-10T09:47:36.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:36.295+0000 7f3925c5f740 1 -- start start 2024-09-10T09:47:36.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:36.295+0000 7f3925c5f740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558119625d50 0x558119626130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:47:36.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:36.295+0000 7f3925c5f740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558119626670 0x5581196309f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:47:36.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:36.295+0000 7f3925c5f740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558119630f30 0x558119633320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:47:36.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:36.295+0000 7f3925c5f740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55811958c3e0 con 0x558119626670 2024-09-10T09:47:36.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:36.295+0000 7f3925c5f740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x558119533d80 con 0x558119630f30 2024-09-10T09:47:36.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:36.295+0000 7f3925c5f740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5581195883a0 con 0x558119625d50 2024-09-10T09:47:36.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:36.295+0000 7f3918c00640 1 -- 172.21.15.5:0/3261404528 <== mon.2 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f39080109d0 con 0x7f391c074e30 2024-09-10T09:47:36.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:36.295+0000 7f391be00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558119626670 0x5581196309f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:47:36.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:36.295+0000 7f391be00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558119626670 0x5581196309f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53024/0 (socket says 172.21.15.5:53024) 2024-09-10T09:47:36.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:36.295+0000 7f391be00640 1 -- 172.21.15.5:0/2440866303 learned_addr learned my addr 172.21.15.5:0/2440866303 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:47:36.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:36.296+0000 7f3921200640 1 --2- 172.21.15.5:0/2440866303 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558119625d50 0x558119626130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:47:36.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:36.296+0000 7f391b400640 1 --2- 172.21.15.5:0/2440866303 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558119630f30 0x558119633320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:47:36.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:36.296+0000 7f391be00640 1 -- 172.21.15.5:0/2440866303 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558119625d50 msgr2=0x558119626130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:47:36.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:36.296+0000 7f391be00640 1 --2- 172.21.15.5:0/2440866303 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558119625d50 0x558119626130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:36.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:36.296+0000 7f391be00640 1 -- 172.21.15.5:0/2440866303 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558119630f30 msgr2=0x558119633320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:47:36.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:36.296+0000 7f391be00640 1 --2- 172.21.15.5:0/2440866303 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558119630f30 0x558119633320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:36.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:36.296+0000 7f391be00640 1 -- 172.21.15.5:0/2440866303 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558119552ae0 con 0x558119626670 2024-09-10T09:47:36.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:36.296+0000 7f391b400640 1 --2- 172.21.15.5:0/2440866303 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558119630f30 0x558119633320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T09:47:36.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:36.296+0000 7f391be00640 1 --2- 172.21.15.5:0/2440866303 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558119626670 0x5581196309f0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f390c009cf0 tx=0x7f390c011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b2edd7c96e5057 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:47:36.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:36.296+0000 7f3911600640 1 -- 172.21.15.5:0/2440866303 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f390c004230 con 0x558119626670 2024-09-10T09:47:36.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:36.297+0000 7f3911600640 1 -- 172.21.15.5:0/2440866303 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f390c0043d0 con 0x558119626670 2024-09-10T09:47:36.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:36.297+0000 7f3911600640 1 -- 172.21.15.5:0/2440866303 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f390c0198c0 con 0x558119626670 2024-09-10T09:47:36.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:36.297+0000 7f3918c00640 1 --2- 172.21.15.5:0/3261404528 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f39000a2b80 0x7f39000a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:47:36.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:36.297+0000 7f3918c00640 1 -- 172.21.15.5:0/3261404528 <== mon.2 v2:172.21.15.102:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f39080cef80 con 0x7f391c074e30 2024-09-10T09:47:36.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:36.297+0000 7f3921200640 1 --2- 172.21.15.5:0/3261404528 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f39000a2b80 0x7f39000a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:47:36.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:36.297+0000 7f3925c5f740 1 -- 172.21.15.5:0/2440866303 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558119626670 msgr2=0x5581196309f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:47:36.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:36.297+0000 7f3925c5f740 1 --2- 172.21.15.5:0/2440866303 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558119626670 0x5581196309f0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f390c009cf0 tx=0x7f390c011810 comp rx=0 tx=0).stop 2024-09-10T09:47:36.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:36.297+0000 7f3925c5f740 1 -- 172.21.15.5:0/2440866303 shutdown_connections 2024-09-10T09:47:36.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:36.297+0000 7f3925c5f740 1 --2- 172.21.15.5:0/2440866303 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558119630f30 0x558119633320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:36.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:36.297+0000 7f3925c5f740 1 --2- 172.21.15.5:0/2440866303 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558119626670 0x5581196309f0 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:36.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:36.297+0000 7f3925c5f740 1 --2- 172.21.15.5:0/2440866303 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558119625d50 0x558119626130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:36.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:36.297+0000 7f3925c5f740 1 -- 172.21.15.5:0/2440866303 >> 172.21.15.5:0/2440866303 conn(0x5581196222e0 msgr2=0x5581196226c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:47:36.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:36.298+0000 7f3925c5f740 1 -- 172.21.15.5:0/2440866303 shutdown_connections 2024-09-10T09:47:36.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:36.298+0000 7f3925c5f740 1 -- 172.21.15.5:0/2440866303 wait complete. 2024-09-10T09:47:36.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:36.298+0000 7f3925c5f740 1 -- start start 2024-09-10T09:47:36.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:36.298+0000 7f3921200640 1 --2- 172.21.15.5:0/3261404528 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f39000a2b80 0x7f39000a5040 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f3914005e30 tx=0x7f3914005c50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:47:36.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:36.298+0000 7f3925c5f740 1 Processor -- start 2024-09-10T09:47:36.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:36.299+0000 7f3925c5f740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558119625d50 0x55811962dd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:47:36.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:36.299+0000 7f3925c5f740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558119626670 0x55811962e280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:47:36.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:36.299+0000 7f3925c5f740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558119630f30 0x55811962e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:47:36.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:36.299+0000 7f3925c5f740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x558119633ba0 con 0x558119630f30 2024-09-10T09:47:36.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:36.299+0000 7f3925c5f740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x5581195470f0 con 0x558119625d50 2024-09-10T09:47:36.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:36.299+0000 7f3925c5f740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x558119633d20 con 0x558119626670 2024-09-10T09:47:36.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:36.299+0000 7f3925c5f740 10 client.0 _ll_register_callbacks cb 0x5581195389e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:47:36.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:36.299+0000 7f3921200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558119630f30 0x55811962e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:47:36.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:36.299+0000 7f391be00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558119625d50 0x55811962dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:47:36.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:36.299+0000 7f3921200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558119630f30 0x55811962e7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53040/0 (socket says 172.21.15.5:53040) 2024-09-10T09:47:36.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:36.299+0000 7f3921200640 1 -- 172.21.15.5:0/3725540730 learned_addr learned my addr 172.21.15.5:0/3725540730 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:47:36.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:36.299+0000 7f391b400640 1 --2- 172.21.15.5:0/3725540730 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558119626670 0x55811962e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:47:36.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:36.300+0000 7f391be00640 1 -- 172.21.15.5:0/3725540730 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558119626670 msgr2=0x55811962e280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:47:36.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:36.300+0000 7f391be00640 1 --2- 172.21.15.5:0/3725540730 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558119626670 0x55811962e280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:36.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:36.300+0000 7f391be00640 1 -- 172.21.15.5:0/3725540730 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558119630f30 msgr2=0x55811962e7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:47:36.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:36.300+0000 7f391be00640 1 --2- 172.21.15.5:0/3725540730 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558119630f30 0x55811962e7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:36.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:36.300+0000 7f391be00640 1 -- 172.21.15.5:0/3725540730 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5581192fd3c0 con 0x558119625d50 2024-09-10T09:47:36.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:36.300+0000 7f391be00640 1 --2- 172.21.15.5:0/3725540730 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558119625d50 0x55811962dd40 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f390c019460 tx=0x7f390c0038a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a6c7544db5c4bf2a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:47:36.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:36.300+0000 7f3906000640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-09-10T09:47:36.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:36.300+0000 7f3906000640 1 -- 172.21.15.5:0/3725540730 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f390c027070 con 0x558119625d50 2024-09-10T09:47:36.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:36.300+0000 7f3925c5f740 1 -- 172.21.15.5:0/3725540730 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f390c0043d0 con 0x558119625d50 2024-09-10T09:47:36.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:36.300+0000 7f3906000640 1 -- 172.21.15.5:0/3725540730 <== mon.2 v2:172.21.15.102:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f390c005250 con 0x558119625d50 2024-09-10T09:47:36.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:36.301+0000 7f3906000640 1 -- 172.21.15.5:0/3725540730 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f390c033050 con 0x558119625d50 2024-09-10T09:47:36.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:36.301+0000 7f3925c5f740 10 client.27200 fetch_fsmap learned FSMap version 35 2024-09-10T09:47:36.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:36.301+0000 7f3925c5f740 1 -- 172.21.15.5:0/3725540730 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55811963f710 con 0x558119625d50 2024-09-10T09:47:36.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:36.301+0000 7f3906000640 1 -- 172.21.15.5:0/3725540730 <== mon.2 v2:172.21.15.102:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f390c0043d0 con 0x558119625d50 2024-09-10T09:47:36.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:36.301+0000 7f3906000640 1 -- 172.21.15.5:0/3725540730 <== mon.2 v2:172.21.15.102:3300/0 5 ==== fsmap(e 35) ==== 8374+0+0 (secure 0 0 0) 0x7f390c0195c0 con 0x558119625d50 2024-09-10T09:47:36.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:36.301+0000 7f3925c5f740 10 client.27200 fetch_fsmap finished waiting for FSMap version 35 2024-09-10T09:47:36.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:36.301+0000 7f3925c5f740 10 client.27200 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T09:47:36.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:36.301+0000 7f3925c5f740 20 client.27200 populate_metadata read hostname 'smithi005' 2024-09-10T09:47:36.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:36.301+0000 7f3925c5f740 1 --2- 172.21.15.5:0/3725540730 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x558119640ac0 0x558119642eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:47:36.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:36.301+0000 7f3925c5f740 4 client.27200 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:47:36.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:36.301+0000 7f3925c5f740 1 -- 172.21.15.5:0/3725540730 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558119599950 con 0x558119640ac0 2024-09-10T09:47:36.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:36.302+0000 7f391b400640 1 --2- 172.21.15.5:0/3725540730 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x558119640ac0 0x558119642eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:47:36.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:36.302+0000 7f391b400640 1 --2- 172.21.15.5:0/3725540730 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x558119640ac0 0x558119642eb0 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c2969b26d0f0e4b7 server_cookie=cc1f2926f316496 in_seq=0 out_seq=0 2024-09-10T09:47:36.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:36.302+0000 7f3906000640 10 client.27200 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T09:47:36.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:36.303+0000 7f3906000640 1 -- 172.21.15.5:0/3725540730 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558119599950 con 0x558119640ac0 2024-09-10T09:47:36.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:36.303+0000 7f3906000640 10 client.27200 handle_command_reply: tid=0 2024-09-10T09:47:36.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:36.303+0000 7f3925c5f740 1 client.27200 shutdown 2024-09-10T09:47:36.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:36.303+0000 7f3925c5f740 1 -- 172.21.15.5:0/3725540730 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558119625d50 msgr2=0x55811962dd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:47:36.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:36.303+0000 7f3925c5f740 1 --2- 172.21.15.5:0/3725540730 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558119625d50 0x55811962dd40 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f390c019460 tx=0x7f390c0038a0 comp rx=0 tx=0).stop 2024-09-10T09:47:36.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:36.303+0000 7f3925c5f740 1 -- 172.21.15.5:0/3725540730 shutdown_connections 2024-09-10T09:47:36.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:36.303+0000 7f3925c5f740 1 --2- 172.21.15.5:0/3725540730 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x558119640ac0 0x558119642eb0 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:36.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:36.303+0000 7f3925c5f740 1 --2- 172.21.15.5:0/3725540730 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558119630f30 0x55811962e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:36.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:36.303+0000 7f3925c5f740 1 --2- 172.21.15.5:0/3725540730 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558119626670 0x55811962e280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:36.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:36.303+0000 7f3925c5f740 1 --2- 172.21.15.5:0/3725540730 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558119625d50 0x55811962dd40 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:36.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:36.303+0000 7f3906000640 0 client.27200 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T09:47:36.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:36.303+0000 7f3925c5f740 1 -- 172.21.15.5:0/3725540730 >> 172.21.15.5:0/3725540730 conn(0x5581196222e0 msgr2=0x558119627fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:47:36.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:36.303+0000 7f3925c5f740 1 -- 172.21.15.5:0/3725540730 shutdown_connections 2024-09-10T09:47:36.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:36.303+0000 7f3925c5f740 1 -- 172.21.15.5:0/3725540730 wait complete. 2024-09-10T09:47:36.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:36.303+0000 7f3925c5f740 20 client.27200 trim_cache size 0 max 16384 2024-09-10T09:47:36.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:36.321+0000 7f3925c5f740 1 Processor -- start 2024-09-10T09:47:36.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:36.321+0000 7f3925c5f740 1 -- start start 2024-09-10T09:47:36.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:36.322+0000 7f3925c5f740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558119625d50 0x55811962d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:47:36.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:36.322+0000 7f3925c5f740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558119626670 0x558119648620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:47:36.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:36.322+0000 7f3925c5f740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558119630f30 0x558119648b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:47:36.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:36.322+0000 7f3925c5f740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f391c1b0f80 con 0x558119625d50 2024-09-10T09:47:36.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:36.322+0000 7f3925c5f740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f391c1b2af0 con 0x558119626670 2024-09-10T09:47:36.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:36.322+0000 7f3925c5f740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f391c1b1650 con 0x558119630f30 2024-09-10T09:47:36.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:36.322+0000 7f3921200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558119630f30 0x558119648b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:47:36.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:36.322+0000 7f391be00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558119625d50 0x55811962d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:47:36.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:36.322+0000 7f3921200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558119630f30 0x558119648b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:38744/0 (socket says 172.21.15.5:38744) 2024-09-10T09:47:36.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:36.322+0000 7f391b400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558119626670 0x558119648620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:47:36.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:36.322+0000 7f3921200640 1 -- 172.21.15.5:0/3382513155 learned_addr learned my addr 172.21.15.5:0/3382513155 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:47:36.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:36.323+0000 7f391b400640 1 -- 172.21.15.5:0/3382513155 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558119630f30 msgr2=0x558119648b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:47:36.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:36.323+0000 7f391b400640 1 --2- 172.21.15.5:0/3382513155 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558119630f30 0x558119648b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:36.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:36.323+0000 7f391b400640 1 -- 172.21.15.5:0/3382513155 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558119625d50 msgr2=0x55811962d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:47:36.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:36.323+0000 7f391b400640 1 --2- 172.21.15.5:0/3382513155 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558119625d50 0x55811962d4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:36.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:36.323+0000 7f391b400640 1 -- 172.21.15.5:0/3382513155 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5581196272d0 con 0x558119626670 2024-09-10T09:47:36.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:36.323+0000 7f3921200640 1 --2- 172.21.15.5:0/3382513155 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558119630f30 0x558119648b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:47:36.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:36.323+0000 7f391b400640 1 --2- 172.21.15.5:0/3382513155 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558119626670 0x558119648620 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f39080cc6e0 tx=0x7f39080cc560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=956ca290c89995bb server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:47:36.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:36.323+0000 7f3910c00640 1 -- 172.21.15.5:0/3382513155 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3908020040 con 0x558119626670 2024-09-10T09:47:36.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:36.323+0000 7f3910c00640 1 -- 172.21.15.5:0/3382513155 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f390809f7a0 con 0x558119626670 2024-09-10T09:47:36.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:36.323+0000 7f3910c00640 1 -- 172.21.15.5:0/3382513155 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39080d10b0 con 0x558119626670 2024-09-10T09:47:36.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:36.324+0000 7f3925c5f740 1 -- 172.21.15.5:0/3382513155 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558119626670 msgr2=0x558119648620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:47:36.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:36.324+0000 7f3925c5f740 1 --2- 172.21.15.5:0/3382513155 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558119626670 0x558119648620 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f39080cc6e0 tx=0x7f39080cc560 comp rx=0 tx=0).stop 2024-09-10T09:47:36.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:36.324+0000 7f3925c5f740 1 -- 172.21.15.5:0/3382513155 shutdown_connections 2024-09-10T09:47:36.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:36.324+0000 7f3925c5f740 1 --2- 172.21.15.5:0/3382513155 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558119630f30 0x558119648b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:36.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:36.324+0000 7f3925c5f740 1 --2- 172.21.15.5:0/3382513155 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558119626670 0x558119648620 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:36.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:36.324+0000 7f3925c5f740 1 --2- 172.21.15.5:0/3382513155 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558119625d50 0x55811962d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:36.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:36.324+0000 7f3925c5f740 1 -- 172.21.15.5:0/3382513155 >> 172.21.15.5:0/3382513155 conn(0x5581196222e0 msgr2=0x558119627fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:47:36.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:36.324+0000 7f3925c5f740 1 -- 172.21.15.5:0/3382513155 shutdown_connections 2024-09-10T09:47:36.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:36.324+0000 7f3925c5f740 1 -- 172.21.15.5:0/3382513155 wait complete. 2024-09-10T09:47:36.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:36.324+0000 7f3925c5f740 1 -- start start 2024-09-10T09:47:36.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:36.324+0000 7f3925c5f740 1 Processor -- start 2024-09-10T09:47:36.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:36.325+0000 7f3925c5f740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558119625d50 0x558119639320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:47:36.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:36.325+0000 7f3925c5f740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558119626670 0x558119639860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:47:36.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:36.325+0000 7f3925c5f740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558119630f30 0x55811962d620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:47:36.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:36.325+0000 7f3925c5f740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f391c1b23f0 con 0x558119625d50 2024-09-10T09:47:36.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:36.325+0000 7f3925c5f740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f391c1b1d20 con 0x558119630f30 2024-09-10T09:47:36.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:36.325+0000 7f3925c5f740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f391c1aef60 con 0x558119626670 2024-09-10T09:47:36.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:36.325+0000 7f3925c5f740 10 client.0 _ll_register_callbacks cb 0x7f390c003cc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:47:36.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:36.325+0000 7f391be00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558119625d50 0x558119639320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:47:36.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:36.325+0000 7f391be00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558119625d50 0x558119639320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53058/0 (socket says 172.21.15.5:53058) 2024-09-10T09:47:36.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:36.325+0000 7f391be00640 1 -- 172.21.15.5:0/3664602886 learned_addr learned my addr 172.21.15.5:0/3664602886 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:47:36.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:36.325+0000 7f3921200640 1 --2- 172.21.15.5:0/3664602886 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558119630f30 0x55811962d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:47:36.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:36.325+0000 7f391b400640 1 --2- 172.21.15.5:0/3664602886 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558119626670 0x558119639860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:47:36.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:36.326+0000 7f391be00640 1 -- 172.21.15.5:0/3664602886 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558119626670 msgr2=0x558119639860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:47:36.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:36.326+0000 7f391be00640 1 --2- 172.21.15.5:0/3664602886 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558119626670 0x558119639860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:36.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:36.326+0000 7f391be00640 1 -- 172.21.15.5:0/3664602886 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558119630f30 msgr2=0x55811962d620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:47:36.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:36.326+0000 7f391be00640 1 --2- 172.21.15.5:0/3664602886 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558119630f30 0x55811962d620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:36.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:36.326+0000 7f391be00640 1 -- 172.21.15.5:0/3664602886 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55811962de20 con 0x558119625d50 2024-09-10T09:47:36.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:36.326+0000 7f391be00640 1 --2- 172.21.15.5:0/3664602886 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558119625d50 0x558119639320 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f390c011d10 tx=0x7f390c009f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38e133bf2e94b180 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:47:36.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:36.326+0000 7f3906000640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-10T09:47:36.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:36.326+0000 7f3906000640 1 -- 172.21.15.5:0/3664602886 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f390c03c060 con 0x558119625d50 2024-09-10T09:47:36.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:36.326+0000 7f3925c5f740 1 -- 172.21.15.5:0/3664602886 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f390809f7a0 con 0x558119625d50 2024-09-10T09:47:36.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:36.326+0000 7f3906000640 1 -- 172.21.15.5:0/3664602886 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f390c019730 con 0x558119625d50 2024-09-10T09:47:36.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:36.327+0000 7f3906000640 1 -- 172.21.15.5:0/3664602886 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f390c043050 con 0x558119625d50 2024-09-10T09:47:36.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:36.327+0000 7f3925c5f740 10 client.17214 fetch_fsmap learned FSMap version 35 2024-09-10T09:47:36.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:36.327+0000 7f3925c5f740 1 -- 172.21.15.5:0/3664602886 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x558119638cb0 con 0x558119625d50 2024-09-10T09:47:36.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:36.327+0000 7f3906000640 1 -- 172.21.15.5:0/3664602886 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f390809f7a0 con 0x558119625d50 2024-09-10T09:47:36.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:36.327+0000 7f3906000640 1 -- 172.21.15.5:0/3664602886 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 35) ==== 8374+0+0 (secure 0 0 0) 0x7f390c023460 con 0x558119625d50 2024-09-10T09:47:36.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:36.327+0000 7f3925c5f740 10 client.17214 fetch_fsmap finished waiting for FSMap version 35 2024-09-10T09:47:36.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:36.327+0000 7f3925c5f740 10 client.17214 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T09:47:36.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:36.327+0000 7f3925c5f740 20 client.17214 populate_metadata read hostname 'smithi005' 2024-09-10T09:47:36.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:36.327+0000 7f3925c5f740 1 --2- 172.21.15.5:0/3664602886 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x5581196ab940 0x5581196add60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:47:36.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:36.327+0000 7f3925c5f740 4 client.17214 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T09:47:36.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:36.327+0000 7f3925c5f740 1 -- 172.21.15.5:0/3664602886 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "scrub status"}) -- 0x558119593e60 con 0x5581196ab940 2024-09-10T09:47:36.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:36.328+0000 7f391b400640 1 --2- 172.21.15.5:0/3664602886 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x5581196ab940 0x5581196add60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:47:36.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:36.328+0000 7f391b400640 1 --2- 172.21.15.5:0/3664602886 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x5581196ab940 0x5581196add60 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ed843d0b537a4d98 server_cookie=f01a2704c98a9ede in_seq=0 out_seq=0 2024-09-10T09:47:36.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:36.328+0000 7f3906000640 10 client.17214 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T09:47:36.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:36.328+0000 7f3906000640 1 -- 172.21.15.5:0/3664602886 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x558119593e60 con 0x5581196ab940 2024-09-10T09:47:36.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:36.328+0000 7f3906000640 10 client.17214 handle_command_reply: tid=0 2024-09-10T09:47:36.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:36.328+0000 7f3925c5f740 1 client.17214 shutdown 2024-09-10T09:47:36.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:36.329+0000 7f3925c5f740 1 -- 172.21.15.5:0/3664602886 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558119625d50 msgr2=0x558119639320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:47:36.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:36.329+0000 7f3925c5f740 1 --2- 172.21.15.5:0/3664602886 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558119625d50 0x558119639320 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f390c011d10 tx=0x7f390c009f10 comp rx=0 tx=0).stop 2024-09-10T09:47:36.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:36.329+0000 7f3925c5f740 1 -- 172.21.15.5:0/3664602886 shutdown_connections 2024-09-10T09:47:36.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:36.329+0000 7f3925c5f740 1 --2- 172.21.15.5:0/3664602886 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x5581196ab940 0x5581196add60 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:36.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:36.329+0000 7f3925c5f740 1 --2- 172.21.15.5:0/3664602886 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558119630f30 0x55811962d620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:36.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:36.329+0000 7f3925c5f740 1 --2- 172.21.15.5:0/3664602886 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558119626670 0x558119639860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:36.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:36.329+0000 7f3925c5f740 1 --2- 172.21.15.5:0/3664602886 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558119625d50 0x558119639320 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:36.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:36.329+0000 7f3925c5f740 1 -- 172.21.15.5:0/3664602886 >> 172.21.15.5:0/3664602886 conn(0x5581196222e0 msgr2=0x558119620b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:47:36.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:36.329+0000 7f3906000640 0 client.17214 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T09:47:36.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:36.329+0000 7f3925c5f740 1 -- 172.21.15.5:0/3664602886 shutdown_connections 2024-09-10T09:47:36.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:36.329+0000 7f3925c5f740 1 -- 172.21.15.5:0/3664602886 wait complete. 2024-09-10T09:47:36.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:36.329+0000 7f3925c5f740 20 client.17214 trim_cache size 0 max 16384 2024-09-10T09:47:36.333 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-10T09:47:36.334 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "no active scrubs running", 2024-09-10T09:47:36.334 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": {} 2024-09-10T09:47:36.334 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-10T09:47:36.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:36.330+0000 7f3921c00640 1 -- 172.21.15.5:0/3261404528 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f39000a2b80 msgr2=0x7f39000a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:47:36.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:36.330+0000 7f3921c00640 1 --2- 172.21.15.5:0/3261404528 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f39000a2b80 0x7f39000a5040 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f3914005e30 tx=0x7f3914005c50 comp rx=0 tx=0).stop 2024-09-10T09:47:36.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:36.330+0000 7f3921c00640 1 -- 172.21.15.5:0/3261404528 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f391c074e30 msgr2=0x7f391c19a9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:47:36.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:36.330+0000 7f3921c00640 1 --2- 172.21.15.5:0/3261404528 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f391c074e30 0x7f391c19a9f0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f390800d950 tx=0x7f390800de10 comp rx=0 tx=0).stop 2024-09-10T09:47:36.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:36.330+0000 7f3921c00640 1 -- 172.21.15.5:0/3261404528 shutdown_connections 2024-09-10T09:47:36.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:36.330+0000 7f3921c00640 1 --2- 172.21.15.5:0/3261404528 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f39000a2b80 0x7f39000a5040 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:36.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:36.330+0000 7f3921c00640 1 --2- 172.21.15.5:0/3261404528 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f391c10e280 0x7f391c19ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:36.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:36.330+0000 7f3921c00640 1 --2- 172.21.15.5:0/3261404528 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f391c074e30 0x7f391c19a9f0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:36.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:36.330+0000 7f3921c00640 1 --2- 172.21.15.5:0/3261404528 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f391c074510 0x7f391c19a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:36.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:36.330+0000 7f3921c00640 1 -- 172.21.15.5:0/3261404528 >> 172.21.15.5:0/3261404528 conn(0x7f391c0fd740 msgr2=0x7f391c0fe3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:47:36.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:36.330+0000 7f3921c00640 1 -- 172.21.15.5:0/3261404528 shutdown_connections 2024-09-10T09:47:36.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:36.330+0000 7f3921c00640 1 -- 172.21.15.5:0/3261404528 wait complete. 2024-09-10T09:47:37.086 INFO:tasks.workunit.client.0.smithi005.stdout: 1 555754 6.22 MB/sec execute 318 sec latency 12177.802 ms 2024-09-10T09:47:37.453 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-10T09:47:37.453 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T09:47:37.831 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:47:37.970 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:47:37 smithi005 ceph-mon[30350]: pgmap v671: 129 pgs: 129 active+clean; 216 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 548 KiB/s rd, 1.2 MiB/s wr, 109 op/s 2024-09-10T09:47:38.086 INFO:tasks.workunit.client.0.smithi005.stdout: 1 555754 6.20 MB/sec execute 319 sec latency 13177.844 ms 2024-09-10T09:47:38.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:47:37 smithi159 ceph-mon[35513]: pgmap v671: 129 pgs: 129 active+clean; 216 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 548 KiB/s rd, 1.2 MiB/s wr, 109 op/s 2024-09-10T09:47:38.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:47:37 smithi102 ceph-mon[35442]: pgmap v671: 129 pgs: 129 active+clean; 216 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 548 KiB/s rd, 1.2 MiB/s wr, 109 op/s 2024-09-10T09:47:38.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:38.423+0000 7fed92800640 1 -- 172.21.15.5:0/3463442434 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fed8c102790 msgr2=0x7fed8c1067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:47:38.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:38.423+0000 7fed92800640 1 --2- 172.21.15.5:0/3463442434 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fed8c102790 0x7fed8c1067e0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fed74009900 tx=0x7fed7402a410 comp rx=0 tx=0).stop 2024-09-10T09:47:38.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:38.423+0000 7fed92800640 1 -- 172.21.15.5:0/3463442434 shutdown_connections 2024-09-10T09:47:38.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:38.423+0000 7fed92800640 1 --2- 172.21.15.5:0/3463442434 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fed8c106f10 0x7fed8c10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:38.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:38.423+0000 7fed92800640 1 --2- 172.21.15.5:0/3463442434 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fed8c102790 0x7fed8c1067e0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:38.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:38.423+0000 7fed92800640 1 --2- 172.21.15.5:0/3463442434 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fed8c101de0 0x7fed8c1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:38.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:38.423+0000 7fed92800640 1 -- 172.21.15.5:0/3463442434 >> 172.21.15.5:0/3463442434 conn(0x7fed8c0fd8c0 msgr2=0x7fed8c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:47:38.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:38.424+0000 7fed92800640 1 -- 172.21.15.5:0/3463442434 shutdown_connections 2024-09-10T09:47:38.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:38.424+0000 7fed92800640 1 -- 172.21.15.5:0/3463442434 wait complete. 2024-09-10T09:47:38.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:38.424+0000 7fed92800640 1 Processor -- start 2024-09-10T09:47:38.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:38.425+0000 7fed92800640 1 -- start start 2024-09-10T09:47:38.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:38.425+0000 7fed92800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fed8c101de0 0x7fed8c0744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:47:38.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:38.425+0000 7fed92800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fed8c102790 0x7fed8c074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:47:38.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:38.425+0000 7fed92800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fed8c106f10 0x7fed8c074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:47:38.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:38.425+0000 7fed92800640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fed8c111f80 con 0x7fed8c101de0 2024-09-10T09:47:38.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:38.425+0000 7fed92800640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fed8c111e00 con 0x7fed8c102790 2024-09-10T09:47:38.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:38.425+0000 7fed92800640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fed8c112100 con 0x7fed8c106f10 2024-09-10T09:47:38.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:38.425+0000 7fed82a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fed8c102790 0x7fed8c074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:47:38.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:38.425+0000 7fed82a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fed8c102790 0x7fed8c074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:43138/0 (socket says 172.21.15.5:43138) 2024-09-10T09:47:38.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:38.425+0000 7fed82a00640 1 -- 172.21.15.5:0/3678399349 learned_addr learned my addr 172.21.15.5:0/3678399349 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:47:38.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:38.425+0000 7fed8be00640 1 --2- 172.21.15.5:0/3678399349 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fed8c106f10 0x7fed8c074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:47:38.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:38.425+0000 7fed8b400640 1 --2- 172.21.15.5:0/3678399349 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fed8c101de0 0x7fed8c0744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:47:38.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:38.426+0000 7fed82a00640 1 -- 172.21.15.5:0/3678399349 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fed8c106f10 msgr2=0x7fed8c074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:47:38.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:38.426+0000 7fed82a00640 1 --2- 172.21.15.5:0/3678399349 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fed8c106f10 0x7fed8c074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:38.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:38.426+0000 7fed82a00640 1 -- 172.21.15.5:0/3678399349 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fed8c101de0 msgr2=0x7fed8c0744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:47:38.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:38.426+0000 7fed82a00640 1 --2- 172.21.15.5:0/3678399349 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fed8c101de0 0x7fed8c0744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:38.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:38.426+0000 7fed82a00640 1 -- 172.21.15.5:0/3678399349 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed8c1a68e0 con 0x7fed8c102790 2024-09-10T09:47:38.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:38.426+0000 7fed82a00640 1 --2- 172.21.15.5:0/3678399349 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fed8c102790 0x7fed8c074a30 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fed74004770 tx=0x7fed74003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:47:38.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:38.426+0000 7fed88c00640 1 -- 172.21.15.5:0/3678399349 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed7403f070 con 0x7fed8c102790 2024-09-10T09:47:38.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:38.426+0000 7fed92800640 1 -- 172.21.15.5:0/3678399349 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed8c1a6b70 con 0x7fed8c102790 2024-09-10T09:47:38.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:38.426+0000 7fed88c00640 1 -- 172.21.15.5:0/3678399349 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fed74031ca0 con 0x7fed8c102790 2024-09-10T09:47:38.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:38.426+0000 7fed88c00640 1 -- 172.21.15.5:0/3678399349 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed74035030 con 0x7fed8c102790 2024-09-10T09:47:38.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:38.426+0000 7fed92800640 1 -- 172.21.15.5:0/3678399349 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed8c1a70a0 con 0x7fed8c102790 2024-09-10T09:47:38.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:38.427+0000 7fed92800640 1 -- 172.21.15.5:0/3678399349 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed8c1021c0 con 0x7fed8c102790 2024-09-10T09:47:38.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:38.430+0000 7fed88c00640 1 -- 172.21.15.5:0/3678399349 <== mon.2 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7fed74042070 con 0x7fed8c102790 2024-09-10T09:47:38.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:38.431+0000 7fed88c00640 1 --2- 172.21.15.5:0/3678399349 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fed700a2b80 0x7fed700a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:47:38.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:38.431+0000 7fed88c00640 1 -- 172.21.15.5:0/3678399349 <== mon.2 v2:172.21.15.102:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fed740efff0 con 0x7fed8c102790 2024-09-10T09:47:38.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:38.431+0000 7fed8b400640 1 --2- 172.21.15.5:0/3678399349 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fed700a2b80 0x7fed700a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:47:38.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:38.432+0000 7fed8b400640 1 --2- 172.21.15.5:0/3678399349 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fed700a2b80 0x7fed700a5040 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fed7c00bbe0 tx=0x7fed7c00b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:47:38.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:38.433+0000 7fed88c00640 1 -- 172.21.15.5:0/3678399349 <== mon.2 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fed740bc4f0 con 0x7fed8c102790 2024-09-10T09:47:38.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:38.654+0000 7fed92800640 1 -- 172.21.15.5:0/3678399349 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fed8c075a20 con 0x7fed8c102790 2024-09-10T09:47:38.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:38.656+0000 7fed88c00640 1 -- 172.21.15.5:0/3678399349 <== mon.2 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) ==== 94+0+11970 (secure 0 0 0) 0x7fed8c075a20 con 0x7fed8c102790 2024-09-10T09:47:38.657 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-10T09:47:38.657 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":35,"btime":"2024-09-10T09:46:17:834001+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14454,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/2970594311","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2970594311},{"type":"v1","addr":"172.21.15.5:6835","nonce":2970594311}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14514,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6839/3616502174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":3616502174},{"type":"v1","addr":"172.21.15.5:6839","nonce":3616502174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":35,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T09:35:29.626287+0000","modified":"2024-09-10T09:46:16.966068+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24542,"mds_1":24535,"mds_2":24511,"mds_3":24506,"mds_4":24443},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14520":{"gid":14520,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/2073599842","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":2073599842},{"type":"v1","addr":"172.21.15.5:6841","nonce":2073599842}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"k","rank":4,"incarnation":26,"state":"up:active","state_seq":24,"addr":"172.21.15.102:6833/2457007427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":2457007427},{"type":"v1","addr":"172.21.15.102:6833","nonce":2457007427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/356445159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":356445159},{"type":"v1","addr":"172.21.15.159:6833","nonce":356445159}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6837/2894314507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":2894314507},{"type":"v1","addr":"172.21.15.5:6837","nonce":2894314507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.102:6835/1647549111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":1647549111},{"type":"v1","addr":"172.21.15.102:6835","nonce":1647549111}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6835/952657242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":952657242},{"type":"v1","addr":"172.21.15.159:6835","nonce":952657242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":19,"addr":"172.21.15.102:6837/2689776764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":2689776764},{"type":"v1","addr":"172.21.15.102:6837","nonce":2689776764}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.159:6837/620627639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":620627639},{"type":"v1","addr":"172.21.15.159:6837","nonce":620627639}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24535":{"gid":24535,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":18,"addr":"172.21.15.159:6839/2073739599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":2073739599},{"type":"v1","addr":"172.21.15.159:6839","nonce":2073739599}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24542":{"gid":24542,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.102:6839/335409749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6838","nonce":335409749},{"type":"v1","addr":"172.21.15.102:6839","nonce":335409749}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24542,"qdb_cluster":[24443,24506,24511,24535,24542]},"id":1}]} 2024-09-10T09:47:38.658 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 35 2024-09-10T09:47:38.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:38.658+0000 7fed92800640 1 -- 172.21.15.5:0/3678399349 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fed700a2b80 msgr2=0x7fed700a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:47:38.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:38.658+0000 7fed92800640 1 --2- 172.21.15.5:0/3678399349 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fed700a2b80 0x7fed700a5040 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fed7c00bbe0 tx=0x7fed7c00b390 comp rx=0 tx=0).stop 2024-09-10T09:47:38.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:38.658+0000 7fed92800640 1 -- 172.21.15.5:0/3678399349 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fed8c102790 msgr2=0x7fed8c074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:47:38.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:38.658+0000 7fed92800640 1 --2- 172.21.15.5:0/3678399349 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fed8c102790 0x7fed8c074a30 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fed74004770 tx=0x7fed74003c10 comp rx=0 tx=0).stop 2024-09-10T09:47:38.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:38.658+0000 7fed92800640 1 -- 172.21.15.5:0/3678399349 shutdown_connections 2024-09-10T09:47:38.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:38.658+0000 7fed92800640 1 --2- 172.21.15.5:0/3678399349 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fed700a2b80 0x7fed700a5040 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:38.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:38.658+0000 7fed92800640 1 --2- 172.21.15.5:0/3678399349 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fed8c106f10 0x7fed8c074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:38.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:38.658+0000 7fed92800640 1 --2- 172.21.15.5:0/3678399349 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fed8c102790 0x7fed8c074a30 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:38.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:38.658+0000 7fed92800640 1 --2- 172.21.15.5:0/3678399349 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fed8c101de0 0x7fed8c0744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:38.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:38.658+0000 7fed92800640 1 -- 172.21.15.5:0/3678399349 >> 172.21.15.5:0/3678399349 conn(0x7fed8c0fd8c0 msgr2=0x7fed8c0fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:47:38.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:38.659+0000 7fed92800640 1 -- 172.21.15.5:0/3678399349 shutdown_connections 2024-09-10T09:47:38.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:38.659+0000 7fed92800640 1 -- 172.21.15.5:0/3678399349 wait complete. 2024-09-10T09:47:39.022 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:47:38 smithi005 ceph-mon[30350]: pgmap v672: 129 pgs: 129 active+clean; 217 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 247 KiB/s rd, 1.0 MiB/s wr, 44 op/s 2024-09-10T09:47:39.022 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:47:38 smithi005 ceph-mon[30350]: from='client.? 172.21.15.5:0/3678399349' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:47:39.087 INFO:tasks.workunit.client.0.smithi005.stdout: 1 555754 6.18 MB/sec execute 320 sec latency 14177.885 ms 2024-09-10T09:47:39.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:47:38 smithi159 ceph-mon[35513]: pgmap v672: 129 pgs: 129 active+clean; 217 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 247 KiB/s rd, 1.0 MiB/s wr, 44 op/s 2024-09-10T09:47:39.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:47:38 smithi159 ceph-mon[35513]: from='client.? 172.21.15.5:0/3678399349' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:47:39.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:47:38 smithi102 ceph-mon[35442]: pgmap v672: 129 pgs: 129 active+clean; 217 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 247 KiB/s rd, 1.0 MiB/s wr, 44 op/s 2024-09-10T09:47:39.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:47:38 smithi102 ceph-mon[35442]: from='client.? 172.21.15.5:0/3678399349' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:47:39.583 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-10T09:47:39.974 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:47:40.087 INFO:tasks.workunit.client.0.smithi005.stdout: 1 555754 6.16 MB/sec execute 321 sec latency 15177.945 ms 2024-09-10T09:47:40.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:40.686+0000 7f37ef200640 1 -- 172.21.15.5:0/2768474235 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f37f0108f60 msgr2=0x7f37f0110ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:47:40.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:40.686+0000 7f37ef200640 1 --2- 172.21.15.5:0/2768474235 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f37f0108f60 0x7f37f0110ae0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f37e8009ca0 tx=0x7f37e802a8c0 comp rx=0 tx=0).stop 2024-09-10T09:47:40.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:40.686+0000 7f37ef200640 1 -- 172.21.15.5:0/2768474235 shutdown_connections 2024-09-10T09:47:40.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:40.686+0000 7f37ef200640 1 --2- 172.21.15.5:0/2768474235 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f37f0108f60 0x7f37f0110ae0 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:40.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:40.686+0000 7f37ef200640 1 --2- 172.21.15.5:0/2768474235 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f37f01047e0 0x7f37f0108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:40.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:40.686+0000 7f37ef200640 1 --2- 172.21.15.5:0/2768474235 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f37f0103e30 0x7f37f0104210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:40.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:40.686+0000 7f37ef200640 1 -- 172.21.15.5:0/2768474235 >> 172.21.15.5:0/2768474235 conn(0x7f37f00ff8d0 msgr2=0x7f37f0101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:47:40.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:40.686+0000 7f37ef200640 1 -- 172.21.15.5:0/2768474235 shutdown_connections 2024-09-10T09:47:40.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:40.686+0000 7f37ef200640 1 -- 172.21.15.5:0/2768474235 wait complete. 2024-09-10T09:47:40.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:40.692+0000 7f37ef200640 1 Processor -- start 2024-09-10T09:47:40.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:40.692+0000 7f37ef200640 1 -- start start 2024-09-10T09:47:40.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:40.692+0000 7f37ef200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f37f0103e30 0x7f37f019a400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:47:40.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:40.692+0000 7f37ef200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f37f01047e0 0x7f37f019a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:47:40.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:40.692+0000 7f37ef200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f37f0108f60 0x7f37f019ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:47:40.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:40.692+0000 7f37ef200640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f37f0113fc0 con 0x7f37f0103e30 2024-09-10T09:47:40.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:40.692+0000 7f37ef200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f37f0113e40 con 0x7f37f01047e0 2024-09-10T09:47:40.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:40.692+0000 7f37ef200640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f37f0114140 con 0x7f37f0108f60 2024-09-10T09:47:40.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:40.692+0000 7f37ede00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f37f0103e30 0x7f37f019a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:47:40.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:40.692+0000 7f37ede00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f37f0103e30 0x7f37f019a400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53102/0 (socket says 172.21.15.5:53102) 2024-09-10T09:47:40.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:40.692+0000 7f37ede00640 1 -- 172.21.15.5:0/1749367003 learned_addr learned my addr 172.21.15.5:0/1749367003 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:47:40.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:40.692+0000 7f37ee800640 1 --2- 172.21.15.5:0/1749367003 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f37f0108f60 0x7f37f019ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:47:40.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:40.692+0000 7f37e7e00640 1 --2- 172.21.15.5:0/1749367003 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f37f01047e0 0x7f37f019a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:47:40.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:40.692+0000 7f37ede00640 1 -- 172.21.15.5:0/1749367003 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f37f0108f60 msgr2=0x7f37f019ecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:47:40.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:40.692+0000 7f37ede00640 1 --2- 172.21.15.5:0/1749367003 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f37f0108f60 0x7f37f019ecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:40.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:40.692+0000 7f37ede00640 1 -- 172.21.15.5:0/1749367003 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f37f01047e0 msgr2=0x7f37f019a940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:47:40.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:40.692+0000 7f37ede00640 1 --2- 172.21.15.5:0/1749367003 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f37f01047e0 0x7f37f019a940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:40.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:40.692+0000 7f37ede00640 1 -- 172.21.15.5:0/1749367003 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37f019f3b0 con 0x7f37f0103e30 2024-09-10T09:47:40.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:40.692+0000 7f37ede00640 1 --2- 172.21.15.5:0/1749367003 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f37f0103e30 0x7f37f019a400 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f37e000e9f0 tx=0x7f37e000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:47:40.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:40.692+0000 7f37e6a00640 1 -- 172.21.15.5:0/1749367003 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37e000cd80 con 0x7f37f0103e30 2024-09-10T09:47:40.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:40.692+0000 7f37ef200640 1 -- 172.21.15.5:0/1749367003 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37f019f6a0 con 0x7f37f0103e30 2024-09-10T09:47:40.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:40.692+0000 7f37e6a00640 1 -- 172.21.15.5:0/1749367003 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f37e0005500 con 0x7f37f0103e30 2024-09-10T09:47:40.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:40.692+0000 7f37e6a00640 1 -- 172.21.15.5:0/1749367003 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37e0010690 con 0x7f37f0103e30 2024-09-10T09:47:40.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:40.692+0000 7f37ef200640 1 -- 172.21.15.5:0/1749367003 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37f01a6f80 con 0x7f37f0103e30 2024-09-10T09:47:40.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:40.697+0000 7f37f7e00740 1 Processor -- start 2024-09-10T09:47:40.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:40.697+0000 7f37e6a00640 1 -- 172.21.15.5:0/1749367003 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f37e00046c0 con 0x7f37f0103e30 2024-09-10T09:47:40.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:40.697+0000 7f37f7e00740 1 -- start start 2024-09-10T09:47:40.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:40.697+0000 7f37f7e00740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557b3c400e40 0x557b3c401220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:47:40.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:40.697+0000 7f37f7e00740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557b3c401760 0x557b3c40bae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:47:40.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:40.697+0000 7f37f7e00740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557b3c40c020 0x557b3c40e410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:47:40.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:40.697+0000 7f37f7e00740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x557b3c3673e0 con 0x557b3c40c020 2024-09-10T09:47:40.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:40.697+0000 7f37f7e00740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x557b3c30ed80 con 0x557b3c400e40 2024-09-10T09:47:40.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:40.697+0000 7f37f7e00740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x557b3c3633a0 con 0x557b3c401760 2024-09-10T09:47:40.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:40.697+0000 7f37e7e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557b3c40c020 0x557b3c40e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:47:40.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:40.697+0000 7f37ee800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557b3c400e40 0x557b3c401220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:47:40.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:40.697+0000 7f37ee800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557b3c400e40 0x557b3c401220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:43156/0 (socket says 172.21.15.5:43156) 2024-09-10T09:47:40.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:40.697+0000 7f37ee800640 1 -- 172.21.15.5:0/3786757771 learned_addr learned my addr 172.21.15.5:0/3786757771 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:47:40.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:40.697+0000 7f37ede00640 1 --2- 172.21.15.5:0/3786757771 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557b3c401760 0x557b3c40bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:47:40.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:40.697+0000 7f37ee800640 1 -- 172.21.15.5:0/3786757771 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557b3c401760 msgr2=0x557b3c40bae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:47:40.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:40.697+0000 7f37ee800640 1 --2- 172.21.15.5:0/3786757771 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557b3c401760 0x557b3c40bae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:40.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:40.697+0000 7f37ee800640 1 -- 172.21.15.5:0/3786757771 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557b3c40c020 msgr2=0x557b3c40e410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:47:40.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:40.697+0000 7f37ee800640 1 --2- 172.21.15.5:0/3786757771 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557b3c40c020 0x557b3c40e410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:40.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:40.697+0000 7f37ee800640 1 -- 172.21.15.5:0/3786757771 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557b3c32dae0 con 0x557b3c400e40 2024-09-10T09:47:40.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:40.697+0000 7f37e7e00640 1 --2- 172.21.15.5:0/3786757771 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557b3c40c020 0x557b3c40e410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:47:40.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:40.697+0000 7f37ee800640 1 --2- 172.21.15.5:0/3786757771 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557b3c400e40 0x557b3c401220 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f37e8009ca0 tx=0x7f37e8005f10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=564eabb3349bdb84 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:47:40.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:40.697+0000 7f37e6a00640 1 --2- 172.21.15.5:0/1749367003 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f37d40a2c00 0x7f37d40a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:47:40.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:40.697+0000 7f37e6a00640 1 -- 172.21.15.5:0/1749367003 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f37e00cf810 con 0x7f37f0103e30 2024-09-10T09:47:40.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:40.697+0000 7f37df400640 1 -- 172.21.15.5:0/3786757771 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37e8031db0 con 0x557b3c400e40 2024-09-10T09:47:40.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:40.697+0000 7f37df400640 1 -- 172.21.15.5:0/3786757771 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f37e8036040 con 0x557b3c400e40 2024-09-10T09:47:40.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:40.697+0000 7f37df400640 1 -- 172.21.15.5:0/3786757771 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37e80042a0 con 0x557b3c400e40 2024-09-10T09:47:40.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:40.697+0000 7f37ede00640 1 --2- 172.21.15.5:0/1749367003 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f37d40a2c00 0x7f37d40a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:47:40.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:40.697+0000 7f37ede00640 1 --2- 172.21.15.5:0/1749367003 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f37d40a2c00 0x7f37d40a50c0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f37e0098080 tx=0x7f37e00d5530 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:47:40.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:40.697+0000 7f37f7e00740 1 -- 172.21.15.5:0/3786757771 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557b3c400e40 msgr2=0x557b3c401220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:47:40.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:40.697+0000 7f37f7e00740 1 --2- 172.21.15.5:0/3786757771 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557b3c400e40 0x557b3c401220 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f37e8009ca0 tx=0x7f37e8005f10 comp rx=0 tx=0).stop 2024-09-10T09:47:40.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:40.697+0000 7f37f7e00740 1 -- 172.21.15.5:0/3786757771 shutdown_connections 2024-09-10T09:47:40.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:40.697+0000 7f37f7e00740 1 --2- 172.21.15.5:0/3786757771 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557b3c40c020 0x557b3c40e410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:40.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:40.697+0000 7f37f7e00740 1 --2- 172.21.15.5:0/3786757771 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557b3c401760 0x557b3c40bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:40.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:40.697+0000 7f37f7e00740 1 --2- 172.21.15.5:0/3786757771 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557b3c400e40 0x557b3c401220 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:40.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:40.697+0000 7f37f7e00740 1 -- 172.21.15.5:0/3786757771 >> 172.21.15.5:0/3786757771 conn(0x557b3c3fd3d0 msgr2=0x557b3c3fd7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:47:40.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:40.697+0000 7f37f7e00740 1 -- 172.21.15.5:0/3786757771 shutdown_connections 2024-09-10T09:47:40.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:40.697+0000 7f37f7e00740 1 -- 172.21.15.5:0/3786757771 wait complete. 2024-09-10T09:47:40.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:40.697+0000 7f37f7e00740 1 -- start start 2024-09-10T09:47:40.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:40.702+0000 7f37f7e00740 1 Processor -- start 2024-09-10T09:47:40.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:40.702+0000 7f37f7e00740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557b3c400e40 0x557b3c408ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:47:40.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:40.702+0000 7f37f7e00740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557b3c401760 0x557b3c4093e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:47:40.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:40.702+0000 7f37f7e00740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557b3c40c020 0x557b3c409920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:47:40.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:40.702+0000 7f37f7e00740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x557b3c40ea70 con 0x557b3c400e40 2024-09-10T09:47:40.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:40.702+0000 7f37f7e00740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x557b3c3220f0 con 0x557b3c40c020 2024-09-10T09:47:40.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:40.702+0000 7f37f7e00740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x557b3c40ebf0 con 0x557b3c401760 2024-09-10T09:47:40.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:40.702+0000 7f37f7e00740 10 client.0 _ll_register_callbacks cb 0x557b3c309bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:47:40.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:40.702+0000 7f37ee800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557b3c400e40 0x557b3c408ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:47:40.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:40.702+0000 7f37e7e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557b3c401760 0x557b3c4093e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:47:40.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:40.702+0000 7f37ee800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557b3c400e40 0x557b3c408ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53118/0 (socket says 172.21.15.5:53118) 2024-09-10T09:47:40.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:40.702+0000 7f37e7e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557b3c401760 0x557b3c4093e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:38780/0 (socket says 172.21.15.5:38780) 2024-09-10T09:47:40.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:40.702+0000 7f37ee800640 1 -- 172.21.15.5:0/1162466606 learned_addr learned my addr 172.21.15.5:0/1162466606 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:47:40.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:40.702+0000 7f37ee800640 1 --2- 172.21.15.5:0/1162466606 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557b3c40c020 0x557b3c409920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:47:40.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:40.702+0000 7f37ee800640 1 -- 172.21.15.5:0/1162466606 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557b3c401760 msgr2=0x557b3c4093e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:47:40.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:40.702+0000 7f37ee800640 1 --2- 172.21.15.5:0/1162466606 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557b3c401760 0x557b3c4093e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:40.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:40.702+0000 7f37ee800640 1 -- 172.21.15.5:0/1162466606 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557b3c40c020 msgr2=0x557b3c409920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:47:40.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:40.702+0000 7f37ee800640 1 --2- 172.21.15.5:0/1162466606 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557b3c40c020 0x557b3c409920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:40.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:40.702+0000 7f37ee800640 1 -- 172.21.15.5:0/1162466606 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557b3c0d83c0 con 0x557b3c400e40 2024-09-10T09:47:40.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:40.702+0000 7f37e7e00640 1 --2- 172.21.15.5:0/1162466606 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557b3c401760 0x557b3c4093e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:47:40.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:40.702+0000 7f37ee800640 1 --2- 172.21.15.5:0/1162466606 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557b3c400e40 0x557b3c408ea0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f37e802f470 tx=0x7f37e8031a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f906978f5b7a0f8e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:47:40.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:40.702+0000 7f37d3e00640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-10T09:47:40.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:40.702+0000 7f37d3e00640 1 -- 172.21.15.5:0/1162466606 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37e8003b80 con 0x557b3c400e40 2024-09-10T09:47:40.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:40.702+0000 7f37d3e00640 1 -- 172.21.15.5:0/1162466606 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f37e8003d20 con 0x557b3c400e40 2024-09-10T09:47:40.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:40.702+0000 7f37f7e00740 1 -- 172.21.15.5:0/1162466606 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f37e8036040 con 0x557b3c400e40 2024-09-10T09:47:40.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:40.702+0000 7f37d3e00640 1 -- 172.21.15.5:0/1162466606 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f37e8004b40 con 0x557b3c400e40 2024-09-10T09:47:40.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:40.702+0000 7f37f7e00740 10 client.17232 fetch_fsmap learned FSMap version 35 2024-09-10T09:47:40.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:40.702+0000 7f37f7e00740 1 -- 172.21.15.5:0/1162466606 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x557b3c40ea70 con 0x557b3c400e40 2024-09-10T09:47:40.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:40.702+0000 7f37d3e00640 1 -- 172.21.15.5:0/1162466606 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f37e8036040 con 0x557b3c400e40 2024-09-10T09:47:40.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:40.702+0000 7f37d3e00640 1 -- 172.21.15.5:0/1162466606 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 35) ==== 8374+0+0 (secure 0 0 0) 0x7f37e8050040 con 0x557b3c400e40 2024-09-10T09:47:40.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:40.702+0000 7f37f7e00740 10 client.17232 fetch_fsmap finished waiting for FSMap version 35 2024-09-10T09:47:40.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:40.702+0000 7f37f7e00740 10 client.17232 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-09-10T09:47:40.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:40.702+0000 7f37f7e00740 20 client.17232 populate_metadata read hostname 'smithi005' 2024-09-10T09:47:40.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:40.702+0000 7f37f7e00740 1 --2- 172.21.15.5:0/1162466606 >> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] conn(0x557b3c41bbc0 0x557b3c41dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:47:40.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:40.702+0000 7f37f7e00740 4 client.17232 mds_command: new command op to 24443 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:47:40.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:40.702+0000 7f37f7e00740 1 -- 172.21.15.5:0/1162466606 --> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557b3c374950 con 0x557b3c41bbc0 2024-09-10T09:47:40.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:40.702+0000 7f37e7e00640 1 --2- 172.21.15.5:0/1162466606 >> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] conn(0x557b3c41bbc0 0x557b3c41dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:47:40.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:40.702+0000 7f37e7e00640 1 --2- 172.21.15.5:0/1162466606 >> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] conn(0x557b3c41bbc0 0x557b3c41dfb0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=15893125edcefeba server_cookie=35362279954add80 in_seq=0 out_seq=0 2024-09-10T09:47:40.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:40.702+0000 7f37d3e00640 10 client.17232 ms_handle_connect on v2:172.21.15.102:6832/2457007427 2024-09-10T09:47:40.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:40.702+0000 7f37d3e00640 1 -- 172.21.15.5:0/1162466606 <== mds.4 v2:172.21.15.102:6832/2457007427 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557b3c374950 con 0x557b3c41bbc0 2024-09-10T09:47:40.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:40.707+0000 7f37d3e00640 10 client.17232 handle_command_reply: tid=0 2024-09-10T09:47:40.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:40.707+0000 7f37f7e00740 1 client.17232 shutdown 2024-09-10T09:47:40.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:40.707+0000 7f37f7e00740 1 -- 172.21.15.5:0/1162466606 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557b3c400e40 msgr2=0x557b3c408ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:47:40.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:40.707+0000 7f37f7e00740 1 --2- 172.21.15.5:0/1162466606 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557b3c400e40 0x557b3c408ea0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f37e802f470 tx=0x7f37e8031a10 comp rx=0 tx=0).stop 2024-09-10T09:47:40.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:40.707+0000 7f37f7e00740 1 -- 172.21.15.5:0/1162466606 shutdown_connections 2024-09-10T09:47:40.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:40.707+0000 7f37f7e00740 1 --2- 172.21.15.5:0/1162466606 >> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] conn(0x557b3c41bbc0 0x557b3c41dfb0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:40.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:40.707+0000 7f37f7e00740 1 --2- 172.21.15.5:0/1162466606 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557b3c40c020 0x557b3c409920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:40.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:40.707+0000 7f37d3e00640 0 client.17232 ms_handle_reset on v2:172.21.15.102:6832/2457007427 2024-09-10T09:47:40.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:40.707+0000 7f37f7e00740 1 --2- 172.21.15.5:0/1162466606 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557b3c401760 0x557b3c4093e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:40.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:40.707+0000 7f37f7e00740 1 --2- 172.21.15.5:0/1162466606 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557b3c400e40 0x557b3c408ea0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:40.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:40.707+0000 7f37f7e00740 1 -- 172.21.15.5:0/1162466606 >> 172.21.15.5:0/1162466606 conn(0x557b3c3fd3d0 msgr2=0x557b3c4030b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:47:40.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:40.707+0000 7f37f7e00740 1 -- 172.21.15.5:0/1162466606 shutdown_connections 2024-09-10T09:47:40.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:40.707+0000 7f37f7e00740 1 -- 172.21.15.5:0/1162466606 wait complete. 2024-09-10T09:47:40.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:40.707+0000 7f37f7e00740 20 client.17232 trim_cache size 0 max 16384 2024-09-10T09:47:40.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:40.722+0000 7f37f7e00740 1 Processor -- start 2024-09-10T09:47:40.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:40.722+0000 7f37f7e00740 1 -- start start 2024-09-10T09:47:40.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:40.722+0000 7f37f7e00740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557b3c400e40 0x557b3c423830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:47:40.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:40.722+0000 7f37f7e00740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557b3c401760 0x557b3c423d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:47:40.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:40.722+0000 7f37f7e00740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557b3c40c020 0x557b3c406120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:47:40.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:40.722+0000 7f37f7e00740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f37f01b0db0 con 0x557b3c40c020 2024-09-10T09:47:40.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:40.722+0000 7f37f7e00740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f37f01b2920 con 0x557b3c400e40 2024-09-10T09:47:40.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:40.722+0000 7f37f7e00740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f37f01b1480 con 0x557b3c401760 2024-09-10T09:47:40.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:40.722+0000 7f37ee800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557b3c400e40 0x557b3c423830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:47:40.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:40.722+0000 7f37ee800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557b3c400e40 0x557b3c423830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:43170/0 (socket says 172.21.15.5:43170) 2024-09-10T09:47:40.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:40.722+0000 7f37ee800640 1 -- 172.21.15.5:0/3986558810 learned_addr learned my addr 172.21.15.5:0/3986558810 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:47:40.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:40.722+0000 7f37e7e00640 1 --2- 172.21.15.5:0/3986558810 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557b3c401760 0x557b3c423d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:47:40.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:40.722+0000 7f37ee800640 1 --2- 172.21.15.5:0/3986558810 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557b3c40c020 0x557b3c406120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:47:40.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:40.722+0000 7f37e7e00640 1 -- 172.21.15.5:0/3986558810 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557b3c400e40 msgr2=0x557b3c423830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:47:40.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:40.722+0000 7f37e7e00640 1 --2- 172.21.15.5:0/3986558810 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557b3c400e40 0x557b3c423830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:40.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:40.722+0000 7f37e7e00640 1 -- 172.21.15.5:0/3986558810 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557b3c40c020 msgr2=0x557b3c406120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:47:40.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:40.722+0000 7f37e7e00640 1 --2- 172.21.15.5:0/3986558810 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557b3c40c020 0x557b3c406120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:40.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:40.722+0000 7f37e7e00640 1 -- 172.21.15.5:0/3986558810 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557b3c3fd200 con 0x557b3c401760 2024-09-10T09:47:40.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:40.722+0000 7f37e7e00640 1 --2- 172.21.15.5:0/3986558810 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557b3c401760 0x557b3c423d70 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x557b3c40a450 tx=0x7f37d8005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c8a7590ba2f3fe22 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:47:40.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:40.722+0000 7f37dea00640 1 -- 172.21.15.5:0/3986558810 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37d8005020 con 0x557b3c401760 2024-09-10T09:47:40.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:40.722+0000 7f37dea00640 1 -- 172.21.15.5:0/3986558810 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f37d800f7f0 con 0x557b3c401760 2024-09-10T09:47:40.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:40.722+0000 7f37dea00640 1 -- 172.21.15.5:0/3986558810 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37d80049e0 con 0x557b3c401760 2024-09-10T09:47:40.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:40.722+0000 7f37f7e00740 1 -- 172.21.15.5:0/3986558810 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557b3c401760 msgr2=0x557b3c423d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:47:40.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:40.722+0000 7f37f7e00740 1 --2- 172.21.15.5:0/3986558810 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557b3c401760 0x557b3c423d70 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x557b3c40a450 tx=0x7f37d8005f90 comp rx=0 tx=0).stop 2024-09-10T09:47:40.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:40.722+0000 7f37f7e00740 1 -- 172.21.15.5:0/3986558810 shutdown_connections 2024-09-10T09:47:40.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:40.722+0000 7f37f7e00740 1 --2- 172.21.15.5:0/3986558810 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557b3c40c020 0x557b3c406120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:40.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:40.722+0000 7f37f7e00740 1 --2- 172.21.15.5:0/3986558810 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557b3c401760 0x557b3c423d70 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:40.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:40.722+0000 7f37f7e00740 1 --2- 172.21.15.5:0/3986558810 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557b3c400e40 0x557b3c423830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:40.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:40.722+0000 7f37f7e00740 1 -- 172.21.15.5:0/3986558810 >> 172.21.15.5:0/3986558810 conn(0x557b3c3fd3d0 msgr2=0x557b3c40af20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:47:40.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:40.722+0000 7f37f7e00740 1 -- 172.21.15.5:0/3986558810 shutdown_connections 2024-09-10T09:47:40.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:40.722+0000 7f37f7e00740 1 -- 172.21.15.5:0/3986558810 wait complete. 2024-09-10T09:47:40.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:40.722+0000 7f37f7e00740 1 -- start start 2024-09-10T09:47:40.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:40.722+0000 7f37f7e00740 1 Processor -- start 2024-09-10T09:47:40.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:40.722+0000 7f37f7e00740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557b3c400e40 0x557b3c406040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:47:40.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:40.722+0000 7f37f7e00740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557b3c401760 0x557b3c406580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:47:40.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:40.722+0000 7f37f7e00740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557b3c40c020 0x557b3c406ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:47:40.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:40.722+0000 7f37f7e00740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f37f01b2220 con 0x557b3c400e40 2024-09-10T09:47:40.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:40.722+0000 7f37f7e00740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f37f01b1b50 con 0x557b3c40c020 2024-09-10T09:47:40.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:40.722+0000 7f37f7e00740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f37f01aed90 con 0x557b3c401760 2024-09-10T09:47:40.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:40.722+0000 7f37f7e00740 10 client.0 _ll_register_callbacks cb 0x7f37d80097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:47:40.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:40.722+0000 7f37ee800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557b3c400e40 0x557b3c406040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:47:40.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:40.722+0000 7f37ee800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557b3c400e40 0x557b3c406040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53142/0 (socket says 172.21.15.5:53142) 2024-09-10T09:47:40.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:40.722+0000 7f37ee800640 1 -- 172.21.15.5:0/1600773871 learned_addr learned my addr 172.21.15.5:0/1600773871 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:47:40.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:40.722+0000 7f37e7e00640 1 --2- 172.21.15.5:0/1600773871 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557b3c401760 0x557b3c406580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:47:40.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:40.722+0000 7f37ee800640 1 --2- 172.21.15.5:0/1600773871 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557b3c40c020 0x557b3c406ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:47:40.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:40.722+0000 7f37ee800640 1 -- 172.21.15.5:0/1600773871 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557b3c401760 msgr2=0x557b3c406580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:47:40.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:40.722+0000 7f37ee800640 1 --2- 172.21.15.5:0/1600773871 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557b3c401760 0x557b3c406580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:40.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:40.722+0000 7f37ee800640 1 -- 172.21.15.5:0/1600773871 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557b3c40c020 msgr2=0x557b3c406ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:47:40.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:40.722+0000 7f37ee800640 1 --2- 172.21.15.5:0/1600773871 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557b3c40c020 0x557b3c406ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:40.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:40.722+0000 7f37ee800640 1 -- 172.21.15.5:0/1600773871 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557b3c3fcc80 con 0x557b3c400e40 2024-09-10T09:47:40.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:40.727+0000 7f37ee800640 1 --2- 172.21.15.5:0/1600773871 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557b3c400e40 0x557b3c406040 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f37e8006010 tx=0x7f37e8031970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d23c5a5fa7b4663e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:47:40.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:40.727+0000 7f37d3e00640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-10T09:47:40.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:40.727+0000 7f37d3e00640 1 -- 172.21.15.5:0/1600773871 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37e8004800 con 0x557b3c400e40 2024-09-10T09:47:40.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:40.727+0000 7f37d3e00640 1 -- 172.21.15.5:0/1600773871 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f37e804a040 con 0x557b3c400e40 2024-09-10T09:47:40.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:40.727+0000 7f37f7e00740 1 -- 172.21.15.5:0/1600773871 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f37d800f7f0 con 0x557b3c400e40 2024-09-10T09:47:40.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:40.727+0000 7f37d3e00640 1 -- 172.21.15.5:0/1600773871 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f37e806a050 con 0x557b3c400e40 2024-09-10T09:47:40.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:40.727+0000 7f37f7e00740 10 client.17238 fetch_fsmap learned FSMap version 35 2024-09-10T09:47:40.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:40.727+0000 7f37f7e00740 1 -- 172.21.15.5:0/1600773871 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x557b3c403cb0 con 0x557b3c400e40 2024-09-10T09:47:40.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:40.727+0000 7f37d3e00640 1 -- 172.21.15.5:0/1600773871 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f37d800f7f0 con 0x557b3c400e40 2024-09-10T09:47:40.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:40.727+0000 7f37d3e00640 1 -- 172.21.15.5:0/1600773871 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 35) ==== 8374+0+0 (secure 0 0 0) 0x7f37e8043070 con 0x557b3c400e40 2024-09-10T09:47:40.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:40.727+0000 7f37f7e00740 10 client.17238 fetch_fsmap finished waiting for FSMap version 35 2024-09-10T09:47:40.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:40.727+0000 7f37f7e00740 10 client.17238 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-09-10T09:47:40.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:40.727+0000 7f37f7e00740 20 client.17238 populate_metadata read hostname 'smithi005' 2024-09-10T09:47:40.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:40.727+0000 7f37f7e00740 1 --2- 172.21.15.5:0/1600773871 >> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] conn(0x557b3c486ea0 0x557b3c4892c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:47:40.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:40.727+0000 7f37f7e00740 4 client.17238 mds_command: new command op to 24443 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T09:47:40.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:40.727+0000 7f37f7e00740 1 -- 172.21.15.5:0/1600773871 --> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557b3c36ee60 con 0x557b3c486ea0 2024-09-10T09:47:40.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:40.727+0000 7f37e7e00640 1 --2- 172.21.15.5:0/1600773871 >> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] conn(0x557b3c486ea0 0x557b3c4892c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:47:40.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:40.727+0000 7f37e7e00640 1 --2- 172.21.15.5:0/1600773871 >> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] conn(0x557b3c486ea0 0x557b3c4892c0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=94e7fc29c11ac66d server_cookie=b7d3fced9db76ee7 in_seq=0 out_seq=0 2024-09-10T09:47:40.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:40.727+0000 7f37d3e00640 10 client.17238 ms_handle_connect on v2:172.21.15.102:6832/2457007427 2024-09-10T09:47:40.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:40.727+0000 7f37d3e00640 1 -- 172.21.15.5:0/1600773871 <== mds.4 v2:172.21.15.102:6832/2457007427 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557b3c36ee60 con 0x557b3c486ea0 2024-09-10T09:47:40.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:40.727+0000 7f37d3e00640 10 client.17238 handle_command_reply: tid=0 2024-09-10T09:47:40.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:40.727+0000 7f37f7e00740 1 client.17238 shutdown 2024-09-10T09:47:40.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:40.727+0000 7f37f7e00740 1 -- 172.21.15.5:0/1600773871 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557b3c400e40 msgr2=0x557b3c406040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:47:40.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:40.727+0000 7f37f7e00740 1 --2- 172.21.15.5:0/1600773871 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557b3c400e40 0x557b3c406040 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f37e8006010 tx=0x7f37e8031970 comp rx=0 tx=0).stop 2024-09-10T09:47:40.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:40.727+0000 7f37f7e00740 1 -- 172.21.15.5:0/1600773871 shutdown_connections 2024-09-10T09:47:40.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:40.727+0000 7f37f7e00740 1 --2- 172.21.15.5:0/1600773871 >> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] conn(0x557b3c486ea0 0x557b3c4892c0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:40.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:40.727+0000 7f37f7e00740 1 --2- 172.21.15.5:0/1600773871 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557b3c40c020 0x557b3c406ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:40.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:40.727+0000 7f37f7e00740 1 --2- 172.21.15.5:0/1600773871 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557b3c401760 0x557b3c406580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:40.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:40.727+0000 7f37f7e00740 1 --2- 172.21.15.5:0/1600773871 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557b3c400e40 0x557b3c406040 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:40.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:40.727+0000 7f37f7e00740 1 -- 172.21.15.5:0/1600773871 >> 172.21.15.5:0/1600773871 conn(0x557b3c3fd3d0 msgr2=0x557b3c3fbc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:47:40.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:40.727+0000 7f37d3e00640 0 client.17238 ms_handle_reset on v2:172.21.15.102:6832/2457007427 2024-09-10T09:47:40.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:40.727+0000 7f37f7e00740 1 -- 172.21.15.5:0/1600773871 shutdown_connections 2024-09-10T09:47:40.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:40.727+0000 7f37f7e00740 1 -- 172.21.15.5:0/1600773871 wait complete. 2024-09-10T09:47:40.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:40.727+0000 7f37f7e00740 20 client.17238 trim_cache size 0 max 16384 2024-09-10T09:47:40.735 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-10T09:47:40.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:40.727+0000 7f37ef200640 1 -- 172.21.15.5:0/1749367003 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f37d40a2c00 msgr2=0x7f37d40a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:47:40.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:40.727+0000 7f37ef200640 1 --2- 172.21.15.5:0/1749367003 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f37d40a2c00 0x7f37d40a50c0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f37e0098080 tx=0x7f37e00d5530 comp rx=0 tx=0).stop 2024-09-10T09:47:40.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:40.727+0000 7f37ef200640 1 -- 172.21.15.5:0/1749367003 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f37f0103e30 msgr2=0x7f37f019a400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:47:40.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:40.727+0000 7f37ef200640 1 --2- 172.21.15.5:0/1749367003 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f37f0103e30 0x7f37f019a400 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f37e000e9f0 tx=0x7f37e000edf0 comp rx=0 tx=0).stop 2024-09-10T09:47:40.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:40.727+0000 7f37ef200640 1 -- 172.21.15.5:0/1749367003 shutdown_connections 2024-09-10T09:47:40.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:40.727+0000 7f37ef200640 1 --2- 172.21.15.5:0/1749367003 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f37d40a2c00 0x7f37d40a50c0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:40.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:40.727+0000 7f37ef200640 1 --2- 172.21.15.5:0/1749367003 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f37f0108f60 0x7f37f019ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:40.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:40.727+0000 7f37ef200640 1 --2- 172.21.15.5:0/1749367003 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f37f01047e0 0x7f37f019a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:40.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:40.727+0000 7f37ef200640 1 --2- 172.21.15.5:0/1749367003 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f37f0103e30 0x7f37f019a400 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:40.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:40.727+0000 7f37ef200640 1 -- 172.21.15.5:0/1749367003 >> 172.21.15.5:0/1749367003 conn(0x7f37f00ff8d0 msgr2=0x7f37f0100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:47:40.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:40.727+0000 7f37ef200640 1 -- 172.21.15.5:0/1749367003 shutdown_connections 2024-09-10T09:47:40.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:40.727+0000 7f37ef200640 1 -- 172.21.15.5:0/1749367003 wait complete. 2024-09-10T09:47:41.087 INFO:tasks.workunit.client.0.smithi005.stdout: 1 555754 6.14 MB/sec execute 322 sec latency 16178.048 ms 2024-09-10T09:47:41.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:47:40 smithi159 ceph-mon[35513]: pgmap v673: 129 pgs: 129 active+clean; 214 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 1.1 MiB/s wr, 55 op/s 2024-09-10T09:47:41.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:47:40 smithi102 ceph-mon[35442]: pgmap v673: 129 pgs: 129 active+clean; 214 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 1.1 MiB/s wr, 55 op/s 2024-09-10T09:47:41.235 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:47:40 smithi005 ceph-mon[30350]: pgmap v673: 129 pgs: 129 active+clean; 214 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 1.1 MiB/s wr, 55 op/s 2024-09-10T09:47:41.899 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-10T09:47:42.087 INFO:tasks.workunit.client.0.smithi005.stdout: 1 555754 6.12 MB/sec execute 323 sec latency 17178.129 ms 2024-09-10T09:47:42.296 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:47:42.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:42.838+0000 7faaca800640 1 -- 172.21.15.5:0/2520405460 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faac4103ff0 msgr2=0x7faac41043d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:47:42.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:42.838+0000 7faaca800640 1 --2- 172.21.15.5:0/2520405460 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faac4103ff0 0x7faac41043d0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7faab4009900 tx=0x7faab402a3f0 comp rx=0 tx=0).stop 2024-09-10T09:47:42.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:42.839+0000 7faaca800640 1 -- 172.21.15.5:0/2520405460 shutdown_connections 2024-09-10T09:47:42.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:42.839+0000 7faaca800640 1 --2- 172.21.15.5:0/2520405460 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7faac4109120 0x7faac4110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:42.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:42.839+0000 7faaca800640 1 --2- 172.21.15.5:0/2520405460 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7faac41049a0 0x7faac41089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:42.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:42.839+0000 7faaca800640 1 --2- 172.21.15.5:0/2520405460 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faac4103ff0 0x7faac41043d0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:42.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:42.839+0000 7faaca800640 1 -- 172.21.15.5:0/2520405460 >> 172.21.15.5:0/2520405460 conn(0x7faac40ffa90 msgr2=0x7faac4101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:47:42.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:42.839+0000 7faaca800640 1 -- 172.21.15.5:0/2520405460 shutdown_connections 2024-09-10T09:47:42.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:42.839+0000 7faaca800640 1 -- 172.21.15.5:0/2520405460 wait complete. 2024-09-10T09:47:42.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:42.840+0000 7faaca800640 1 Processor -- start 2024-09-10T09:47:42.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:42.840+0000 7faaca800640 1 -- start start 2024-09-10T09:47:42.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:42.840+0000 7faaca800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7faac4103ff0 0x7faac419a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:47:42.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:42.840+0000 7faaca800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faac41049a0 0x7faac419abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:47:42.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:42.840+0000 7faaca800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7faac4109120 0x7faac419ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:47:42.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:42.840+0000 7faaca800640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7faac4114170 con 0x7faac41049a0 2024-09-10T09:47:42.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:42.840+0000 7faaca800640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7faac4113ff0 con 0x7faac4109120 2024-09-10T09:47:42.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:42.840+0000 7faaca800640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7faac41142f0 con 0x7faac4103ff0 2024-09-10T09:47:42.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:42.840+0000 7faac2a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faac41049a0 0x7faac419abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:47:42.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:42.840+0000 7faac2a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faac41049a0 0x7faac419abb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57126/0 (socket says 172.21.15.5:57126) 2024-09-10T09:47:42.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:42.840+0000 7faac2a00640 1 -- 172.21.15.5:0/2478647722 learned_addr learned my addr 172.21.15.5:0/2478647722 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:47:42.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:42.841+0000 7faac3e00640 1 --2- 172.21.15.5:0/2478647722 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7faac4109120 0x7faac419ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:47:42.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:42.841+0000 7faac3400640 1 --2- 172.21.15.5:0/2478647722 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7faac4103ff0 0x7faac419a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:47:42.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:42.841+0000 7faac2a00640 1 -- 172.21.15.5:0/2478647722 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7faac4103ff0 msgr2=0x7faac419a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:47:42.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:42.841+0000 7faac2a00640 1 --2- 172.21.15.5:0/2478647722 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7faac4103ff0 0x7faac419a670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:42.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:42.841+0000 7faac2a00640 1 -- 172.21.15.5:0/2478647722 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7faac4109120 msgr2=0x7faac419ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:47:42.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:42.841+0000 7faac2a00640 1 --2- 172.21.15.5:0/2478647722 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7faac4109120 0x7faac419ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:42.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:42.841+0000 7faac2a00640 1 -- 172.21.15.5:0/2478647722 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faac419f620 con 0x7faac41049a0 2024-09-10T09:47:42.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:42.841+0000 7faac2a00640 1 --2- 172.21.15.5:0/2478647722 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faac41049a0 0x7faac419abb0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7faab000e9f0 tx=0x7faab000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:47:42.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:42.841+0000 7faabbe00640 1 -- 172.21.15.5:0/2478647722 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faab000cd80 con 0x7faac41049a0 2024-09-10T09:47:42.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:42.841+0000 7faabbe00640 1 -- 172.21.15.5:0/2478647722 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faab0010910 con 0x7faac41049a0 2024-09-10T09:47:42.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:42.841+0000 7faaca800640 1 -- 172.21.15.5:0/2478647722 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faac419f910 con 0x7faac41049a0 2024-09-10T09:47:42.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:42.842+0000 7faabbe00640 1 -- 172.21.15.5:0/2478647722 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faab0018990 con 0x7faac41049a0 2024-09-10T09:47:42.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:42.842+0000 7faaca800640 1 -- 172.21.15.5:0/2478647722 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faac41a71f0 con 0x7faac41049a0 2024-09-10T09:47:42.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:42.845+0000 7faacd2a2740 1 Processor -- start 2024-09-10T09:47:42.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:42.846+0000 7faacd2a2740 1 -- start start 2024-09-10T09:47:42.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:42.846+0000 7faacd2a2740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a299407c70 0x55a299408050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:47:42.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:42.846+0000 7faacd2a2740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a299408590 0x55a299412910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:47:42.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:42.846+0000 7faacd2a2740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a299412e50 0x55a299415240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:47:42.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:42.846+0000 7faacd2a2740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55a29936e3e0 con 0x55a299412e50 2024-09-10T09:47:42.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:42.846+0000 7faacd2a2740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55a299315d80 con 0x55a299408590 2024-09-10T09:47:42.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:42.846+0000 7faacd2a2740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55a29936a3a0 con 0x55a299407c70 2024-09-10T09:47:42.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:42.846+0000 7faabbe00640 1 -- 172.21.15.5:0/2478647722 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7faab0010ac0 con 0x7faac41049a0 2024-09-10T09:47:42.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:42.847+0000 7faac3e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a299407c70 0x55a299408050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:47:42.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:42.847+0000 7faac3400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a299408590 0x55a299412910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:47:42.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:42.847+0000 7faac3e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a299407c70 0x55a299408050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:60174/0 (socket says 172.21.15.5:60174) 2024-09-10T09:47:42.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:42.847+0000 7faac3e00640 1 -- 172.21.15.5:0/1024422907 learned_addr learned my addr 172.21.15.5:0/1024422907 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:47:42.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:42.847+0000 7faac3400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a299408590 0x55a299412910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:38570/0 (socket says 172.21.15.5:38570) 2024-09-10T09:47:42.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:42.847+0000 7faac2a00640 1 --2- 172.21.15.5:0/1024422907 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a299412e50 0x55a299415240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:47:42.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:42.847+0000 7faac3400640 1 -- 172.21.15.5:0/1024422907 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a299407c70 msgr2=0x55a299408050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:47:42.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:42.847+0000 7faac3400640 1 --2- 172.21.15.5:0/1024422907 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a299407c70 0x55a299408050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:42.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:42.847+0000 7faac3400640 1 -- 172.21.15.5:0/1024422907 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a299412e50 msgr2=0x55a299415240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:47:42.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:42.847+0000 7faac3400640 1 --2- 172.21.15.5:0/1024422907 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a299412e50 0x55a299415240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:42.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:42.847+0000 7faac3400640 1 -- 172.21.15.5:0/1024422907 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a299334ae0 con 0x55a299408590 2024-09-10T09:47:42.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:42.847+0000 7faac3e00640 1 --2- 172.21.15.5:0/1024422907 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a299407c70 0x55a299408050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:47:42.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:42.848+0000 7faac3400640 1 --2- 172.21.15.5:0/1024422907 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a299408590 0x55a299412910 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7faab402a9e0 tx=0x7faab4005de0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8f356f91bd8d9cf server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:47:42.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:42.848+0000 7faab8c00640 1 -- 172.21.15.5:0/1024422907 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faab4035040 con 0x55a299408590 2024-09-10T09:47:42.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:42.848+0000 7faab8c00640 1 -- 172.21.15.5:0/1024422907 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faab4004400 con 0x55a299408590 2024-09-10T09:47:42.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:42.848+0000 7faab8c00640 1 -- 172.21.15.5:0/1024422907 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faab40035f0 con 0x55a299408590 2024-09-10T09:47:42.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:42.848+0000 7faabbe00640 1 --2- 172.21.15.5:0/2478647722 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7faaa80a2b80 0x7faaa80a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:47:42.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:42.848+0000 7faabbe00640 1 -- 172.21.15.5:0/2478647722 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7faab00cf740 con 0x7faac41049a0 2024-09-10T09:47:42.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:42.848+0000 7faac3e00640 1 --2- 172.21.15.5:0/2478647722 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7faaa80a2b80 0x7faaa80a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:47:42.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:42.849+0000 7faacd2a2740 1 -- 172.21.15.5:0/1024422907 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a299408590 msgr2=0x55a299412910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:47:42.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:42.849+0000 7faacd2a2740 1 --2- 172.21.15.5:0/1024422907 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a299408590 0x55a299412910 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7faab402a9e0 tx=0x7faab4005de0 comp rx=0 tx=0).stop 2024-09-10T09:47:42.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:42.849+0000 7faacd2a2740 1 -- 172.21.15.5:0/1024422907 shutdown_connections 2024-09-10T09:47:42.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:42.849+0000 7faacd2a2740 1 --2- 172.21.15.5:0/1024422907 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a299412e50 0x55a299415240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:42.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:42.849+0000 7faacd2a2740 1 --2- 172.21.15.5:0/1024422907 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a299408590 0x55a299412910 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:42.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:42.849+0000 7faacd2a2740 1 --2- 172.21.15.5:0/1024422907 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a299407c70 0x55a299408050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:42.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:42.849+0000 7faacd2a2740 1 -- 172.21.15.5:0/1024422907 >> 172.21.15.5:0/1024422907 conn(0x55a299404200 msgr2=0x55a2994045e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:47:42.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:42.849+0000 7faac3e00640 1 --2- 172.21.15.5:0/2478647722 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7faaa80a2b80 0x7faaa80a5040 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7faabc009fd0 tx=0x7faabc009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:47:42.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:42.849+0000 7faacd2a2740 1 -- 172.21.15.5:0/1024422907 shutdown_connections 2024-09-10T09:47:42.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:42.849+0000 7faacd2a2740 1 -- 172.21.15.5:0/1024422907 wait complete. 2024-09-10T09:47:42.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:42.850+0000 7faacd2a2740 1 -- start start 2024-09-10T09:47:42.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:42.850+0000 7faacd2a2740 1 Processor -- start 2024-09-10T09:47:42.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:42.850+0000 7faacd2a2740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a299407c70 0x55a29940fd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:47:42.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:42.851+0000 7faacd2a2740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a299408590 0x55a2994102a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:47:42.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:42.851+0000 7faacd2a2740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a299412e50 0x55a2994107e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:47:42.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:42.851+0000 7faacd2a2740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55a299415a20 con 0x55a299412e50 2024-09-10T09:47:42.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:42.851+0000 7faacd2a2740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55a2993290f0 con 0x55a299407c70 2024-09-10T09:47:42.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:42.851+0000 7faacd2a2740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55a299415ba0 con 0x55a299408590 2024-09-10T09:47:42.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:42.851+0000 7faacd2a2740 10 client.0 _ll_register_callbacks cb 0x55a299310bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:47:42.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:42.851+0000 7faac2a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a299408590 0x55a2994102a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:47:42.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:42.851+0000 7faac2a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a299408590 0x55a2994102a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:60182/0 (socket says 172.21.15.5:60182) 2024-09-10T09:47:42.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:42.851+0000 7faac2a00640 1 -- 172.21.15.5:0/2949843086 learned_addr learned my addr 172.21.15.5:0/2949843086 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:47:42.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:42.851+0000 7faac3400640 1 --2- 172.21.15.5:0/2949843086 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a299407c70 0x55a29940fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:47:42.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:42.851+0000 7faac3e00640 1 --2- 172.21.15.5:0/2949843086 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a299412e50 0x55a2994107e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:47:42.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:42.851+0000 7faac2a00640 1 -- 172.21.15.5:0/2949843086 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a299407c70 msgr2=0x55a29940fd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:47:42.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:42.851+0000 7faac2a00640 1 --2- 172.21.15.5:0/2949843086 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a299407c70 0x55a29940fd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:42.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:42.851+0000 7faac2a00640 1 -- 172.21.15.5:0/2949843086 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a299412e50 msgr2=0x55a2994107e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:47:42.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:42.851+0000 7faac2a00640 1 --2- 172.21.15.5:0/2949843086 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a299412e50 0x55a2994107e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:42.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:42.851+0000 7faac2a00640 1 -- 172.21.15.5:0/2949843086 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a2990df3c0 con 0x55a299408590 2024-09-10T09:47:42.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:42.851+0000 7faac3400640 1 --2- 172.21.15.5:0/2949843086 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a299407c70 0x55a29940fd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:47:42.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:42.852+0000 7faac2a00640 1 --2- 172.21.15.5:0/2949843086 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a299408590 0x55a2994102a0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7faab00cce50 tx=0x7faab009ba30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8be0e24e82d1f137 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:47:42.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:42.852+0000 7faaad600640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-09-10T09:47:42.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:42.852+0000 7faaad600640 1 -- 172.21.15.5:0/2949843086 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faab009bc60 con 0x55a299408590 2024-09-10T09:47:42.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:42.852+0000 7faaad600640 1 -- 172.21.15.5:0/2949843086 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7faab009b0b0 con 0x55a299408590 2024-09-10T09:47:42.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:42.852+0000 7faacd2a2740 1 -- 172.21.15.5:0/2949843086 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faab4004400 con 0x55a299408590 2024-09-10T09:47:42.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:42.852+0000 7faaad600640 1 -- 172.21.15.5:0/2949843086 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7faab009ff60 con 0x55a299408590 2024-09-10T09:47:42.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:42.852+0000 7faacd2a2740 10 client.27274 fetch_fsmap learned FSMap version 35 2024-09-10T09:47:42.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:42.852+0000 7faacd2a2740 1 -- 172.21.15.5:0/2949843086 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55a299421af0 con 0x55a299408590 2024-09-10T09:47:42.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:42.852+0000 7faaad600640 1 -- 172.21.15.5:0/2949843086 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faab4004400 con 0x55a299408590 2024-09-10T09:47:42.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:42.852+0000 7faaad600640 1 -- 172.21.15.5:0/2949843086 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 35) ==== 8374+0+0 (secure 0 0 0) 0x7faab00979b0 con 0x55a299408590 2024-09-10T09:47:42.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:42.853+0000 7faacd2a2740 10 client.27274 fetch_fsmap finished waiting for FSMap version 35 2024-09-10T09:47:42.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:42.853+0000 7faacd2a2740 10 client.27274 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-09-10T09:47:42.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:42.853+0000 7faacd2a2740 20 client.27274 populate_metadata read hostname 'smithi005' 2024-09-10T09:47:42.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:42.853+0000 7faacd2a2740 1 --2- 172.21.15.5:0/2949843086 >> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] conn(0x55a299422f50 0x55a299425360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:47:42.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:42.853+0000 7faacd2a2740 4 client.27274 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:47:42.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:42.853+0000 7faacd2a2740 1 -- 172.21.15.5:0/2949843086 --> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a29937b950 con 0x55a299422f50 2024-09-10T09:47:42.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:42.853+0000 7faac3400640 1 --2- 172.21.15.5:0/2949843086 >> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] conn(0x55a299422f50 0x55a299425360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:47:42.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:42.854+0000 7faac3400640 1 --2- 172.21.15.5:0/2949843086 >> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] conn(0x55a299422f50 0x55a299425360 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=cbfb1fa077c9a79e server_cookie=d4fee68f81ad7ef3 in_seq=0 out_seq=0 2024-09-10T09:47:42.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:42.854+0000 7faaad600640 10 client.27274 ms_handle_connect on v2:172.21.15.102:6836/2689776764 2024-09-10T09:47:42.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:42.855+0000 7faaad600640 1 -- 172.21.15.5:0/2949843086 <== mds.3 v2:172.21.15.102:6836/2689776764 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a29937b950 con 0x55a299422f50 2024-09-10T09:47:42.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:42.855+0000 7faaad600640 10 client.27274 handle_command_reply: tid=0 2024-09-10T09:47:42.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:42.855+0000 7faacd2a2740 1 client.27274 shutdown 2024-09-10T09:47:42.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:42.855+0000 7faacd2a2740 1 -- 172.21.15.5:0/2949843086 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a299408590 msgr2=0x55a2994102a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:47:42.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:42.855+0000 7faacd2a2740 1 --2- 172.21.15.5:0/2949843086 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a299408590 0x55a2994102a0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7faab00cce50 tx=0x7faab009ba30 comp rx=0 tx=0).stop 2024-09-10T09:47:42.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:42.855+0000 7faacd2a2740 1 -- 172.21.15.5:0/2949843086 shutdown_connections 2024-09-10T09:47:42.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:42.855+0000 7faacd2a2740 1 --2- 172.21.15.5:0/2949843086 >> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] conn(0x55a299422f50 0x55a299425360 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:42.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:42.855+0000 7faacd2a2740 1 --2- 172.21.15.5:0/2949843086 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a299412e50 0x55a2994107e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:42.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:42.855+0000 7faacd2a2740 1 --2- 172.21.15.5:0/2949843086 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a299408590 0x55a2994102a0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:42.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:42.855+0000 7faacd2a2740 1 --2- 172.21.15.5:0/2949843086 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a299407c70 0x55a29940fd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:42.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:42.855+0000 7faaad600640 0 client.27274 ms_handle_reset on v2:172.21.15.102:6836/2689776764 2024-09-10T09:47:42.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:42.855+0000 7faacd2a2740 1 -- 172.21.15.5:0/2949843086 >> 172.21.15.5:0/2949843086 conn(0x55a299404200 msgr2=0x55a299403380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:47:42.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:42.855+0000 7faacd2a2740 1 -- 172.21.15.5:0/2949843086 shutdown_connections 2024-09-10T09:47:42.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:42.855+0000 7faacd2a2740 1 -- 172.21.15.5:0/2949843086 wait complete. 2024-09-10T09:47:42.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:42.855+0000 7faacd2a2740 20 client.27274 trim_cache size 0 max 16384 2024-09-10T09:47:42.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:42.869+0000 7faacd2a2740 1 Processor -- start 2024-09-10T09:47:42.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:42.869+0000 7faacd2a2740 1 -- start start 2024-09-10T09:47:42.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:42.869+0000 7faacd2a2740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a299407c70 0x55a299411cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:47:42.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:42.869+0000 7faacd2a2740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a299408590 0x55a29942ac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:47:42.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:42.869+0000 7faacd2a2740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a299412e50 0x55a29942b170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:47:42.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:42.869+0000 7faacd2a2740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7faac41b1010 con 0x55a299407c70 2024-09-10T09:47:42.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:42.869+0000 7faacd2a2740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7faac41b2b80 con 0x55a299412e50 2024-09-10T09:47:42.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:42.869+0000 7faacd2a2740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7faac41b16e0 con 0x55a299408590 2024-09-10T09:47:42.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:42.870+0000 7faac3400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a299407c70 0x55a299411cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:47:42.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:42.870+0000 7faac2a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a299408590 0x55a29942ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:47:42.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:42.870+0000 7faac3400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a299407c70 0x55a299411cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57164/0 (socket says 172.21.15.5:57164) 2024-09-10T09:47:42.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:42.870+0000 7faac2a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a299408590 0x55a29942ac30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:60198/0 (socket says 172.21.15.5:60198) 2024-09-10T09:47:42.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:42.870+0000 7faac3e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a299412e50 0x55a29942b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:47:42.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:42.870+0000 7faac2a00640 1 -- 172.21.15.5:0/4054855385 learned_addr learned my addr 172.21.15.5:0/4054855385 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:47:42.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:42.870+0000 7faac2a00640 1 -- 172.21.15.5:0/4054855385 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a299412e50 msgr2=0x55a29942b170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:47:42.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:42.870+0000 7faac2a00640 1 --2- 172.21.15.5:0/4054855385 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a299412e50 0x55a29942b170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:42.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:42.870+0000 7faac2a00640 1 -- 172.21.15.5:0/4054855385 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a299407c70 msgr2=0x55a299411cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:47:42.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:42.870+0000 7faac2a00640 1 --2- 172.21.15.5:0/4054855385 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a299407c70 0x55a299411cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:42.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:42.870+0000 7faac2a00640 1 -- 172.21.15.5:0/4054855385 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a2993ff770 con 0x55a299408590 2024-09-10T09:47:42.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:42.870+0000 7faac3400640 1 --2- 172.21.15.5:0/4054855385 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a299407c70 0x55a299411cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:47:42.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:42.870+0000 7faac2a00640 1 --2- 172.21.15.5:0/4054855385 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a299408590 0x55a29942ac30 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7faab0098630 tx=0x7faab0000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ecf7532b0f10d0a2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:47:42.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:42.871+0000 7faaafe00640 1 -- 172.21.15.5:0/4054855385 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faab009bc60 con 0x55a299408590 2024-09-10T09:47:42.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:42.871+0000 7faaafe00640 1 -- 172.21.15.5:0/4054855385 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faab00a4820 con 0x55a299408590 2024-09-10T09:47:42.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:42.871+0000 7faaafe00640 1 -- 172.21.15.5:0/4054855385 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faab009b7a0 con 0x55a299408590 2024-09-10T09:47:42.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:42.871+0000 7faacd2a2740 1 -- 172.21.15.5:0/4054855385 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a299408590 msgr2=0x55a29942ac30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:47:42.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:42.871+0000 7faacd2a2740 1 --2- 172.21.15.5:0/4054855385 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a299408590 0x55a29942ac30 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7faab0098630 tx=0x7faab0000bc0 comp rx=0 tx=0).stop 2024-09-10T09:47:42.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:42.871+0000 7faacd2a2740 1 -- 172.21.15.5:0/4054855385 shutdown_connections 2024-09-10T09:47:42.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:42.871+0000 7faacd2a2740 1 --2- 172.21.15.5:0/4054855385 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a299412e50 0x55a29942b170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:42.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:42.871+0000 7faacd2a2740 1 --2- 172.21.15.5:0/4054855385 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a299408590 0x55a29942ac30 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:42.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:42.871+0000 7faacd2a2740 1 --2- 172.21.15.5:0/4054855385 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a299407c70 0x55a299411cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:42.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:42.871+0000 7faacd2a2740 1 -- 172.21.15.5:0/4054855385 >> 172.21.15.5:0/4054855385 conn(0x55a299404200 msgr2=0x55a2994115c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:47:42.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:42.871+0000 7faacd2a2740 1 -- 172.21.15.5:0/4054855385 shutdown_connections 2024-09-10T09:47:42.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:42.871+0000 7faacd2a2740 1 -- 172.21.15.5:0/4054855385 wait complete. 2024-09-10T09:47:42.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:42.872+0000 7faacd2a2740 1 -- start start 2024-09-10T09:47:42.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:42.872+0000 7faacd2a2740 1 Processor -- start 2024-09-10T09:47:42.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:42.872+0000 7faacd2a2740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a299407c70 0x55a29941b520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:47:42.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:42.872+0000 7faacd2a2740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a299408590 0x55a299416850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:47:42.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:42.872+0000 7faacd2a2740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a299412e50 0x55a299416d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:47:42.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:42.872+0000 7faacd2a2740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7faac41b2480 con 0x55a299407c70 2024-09-10T09:47:42.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:42.872+0000 7faacd2a2740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7faac41b1db0 con 0x55a299412e50 2024-09-10T09:47:42.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:42.872+0000 7faacd2a2740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7faac41aeff0 con 0x55a299408590 2024-09-10T09:47:42.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:42.872+0000 7faacd2a2740 10 client.0 _ll_register_callbacks cb 0x7faab00d5240 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:47:42.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:42.873+0000 7faac3400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a299407c70 0x55a29941b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:47:42.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:42.873+0000 7faac3400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a299407c70 0x55a29941b520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57172/0 (socket says 172.21.15.5:57172) 2024-09-10T09:47:42.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:42.873+0000 7faac3400640 1 -- 172.21.15.5:0/1272057482 learned_addr learned my addr 172.21.15.5:0/1272057482 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:47:42.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:42.873+0000 7faac2a00640 1 --2- 172.21.15.5:0/1272057482 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a299408590 0x55a299416850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:47:42.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:42.873+0000 7faac3e00640 1 --2- 172.21.15.5:0/1272057482 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a299412e50 0x55a299416d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:47:42.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:42.873+0000 7faac3400640 1 -- 172.21.15.5:0/1272057482 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a299408590 msgr2=0x55a299416850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:47:42.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:42.873+0000 7faac3400640 1 --2- 172.21.15.5:0/1272057482 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a299408590 0x55a299416850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:42.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:42.873+0000 7faac3400640 1 -- 172.21.15.5:0/1272057482 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a299412e50 msgr2=0x55a299416d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:47:42.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:42.873+0000 7faac3400640 1 --2- 172.21.15.5:0/1272057482 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a299412e50 0x55a299416d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:42.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:42.873+0000 7faac3400640 1 -- 172.21.15.5:0/1272057482 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a2994175c0 con 0x55a299407c70 2024-09-10T09:47:42.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:42.873+0000 7faac3400640 1 --2- 172.21.15.5:0/1272057482 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a299407c70 0x55a29941b520 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7faab402a9e0 tx=0x7faab402a3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb316e1cc7a62d1f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:47:42.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:42.873+0000 7faaad600640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-10T09:47:42.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:42.874+0000 7faaad600640 1 -- 172.21.15.5:0/1272057482 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faab4035040 con 0x55a299407c70 2024-09-10T09:47:42.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:42.874+0000 7faacd2a2740 1 -- 172.21.15.5:0/1272057482 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faab00a4820 con 0x55a299407c70 2024-09-10T09:47:42.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:42.874+0000 7faaad600640 1 -- 172.21.15.5:0/1272057482 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7faab40048c0 con 0x55a299407c70 2024-09-10T09:47:42.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:42.874+0000 7faaad600640 1 -- 172.21.15.5:0/1272057482 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7faab405f050 con 0x55a299407c70 2024-09-10T09:47:42.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:42.874+0000 7faacd2a2740 10 client.17256 fetch_fsmap learned FSMap version 35 2024-09-10T09:47:42.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:42.874+0000 7faacd2a2740 1 -- 172.21.15.5:0/1272057482 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55a29948dbf0 con 0x55a299407c70 2024-09-10T09:47:42.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:42.874+0000 7faaad600640 1 -- 172.21.15.5:0/1272057482 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faab00a4820 con 0x55a299407c70 2024-09-10T09:47:42.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:42.875+0000 7faaad600640 1 -- 172.21.15.5:0/1272057482 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 35) ==== 8374+0+0 (secure 0 0 0) 0x7faab4003830 con 0x55a299407c70 2024-09-10T09:47:42.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:42.875+0000 7faacd2a2740 10 client.17256 fetch_fsmap finished waiting for FSMap version 35 2024-09-10T09:47:42.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:42.875+0000 7faacd2a2740 10 client.17256 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-09-10T09:47:42.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:42.875+0000 7faacd2a2740 20 client.17256 populate_metadata read hostname 'smithi005' 2024-09-10T09:47:42.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:42.875+0000 7faacd2a2740 1 --2- 172.21.15.5:0/1272057482 >> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] conn(0x55a29948dee0 0x55a2994902d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:47:42.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:42.875+0000 7faacd2a2740 4 client.17256 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T09:47:42.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:42.875+0000 7faacd2a2740 1 -- 172.21.15.5:0/1272057482 --> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a299375e60 con 0x55a29948dee0 2024-09-10T09:47:42.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:42.875+0000 7faac2a00640 1 --2- 172.21.15.5:0/1272057482 >> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] conn(0x55a29948dee0 0x55a2994902d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:47:42.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:42.876+0000 7faac2a00640 1 --2- 172.21.15.5:0/1272057482 >> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] conn(0x55a29948dee0 0x55a2994902d0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=f31615af117e9d11 server_cookie=f4158ad71449f3a7 in_seq=0 out_seq=0 2024-09-10T09:47:42.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:42.876+0000 7faaad600640 10 client.17256 ms_handle_connect on v2:172.21.15.102:6836/2689776764 2024-09-10T09:47:42.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:42.876+0000 7faaad600640 1 -- 172.21.15.5:0/1272057482 <== mds.3 v2:172.21.15.102:6836/2689776764 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a299375e60 con 0x55a29948dee0 2024-09-10T09:47:42.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:42.876+0000 7faaad600640 10 client.17256 handle_command_reply: tid=0 2024-09-10T09:47:42.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:42.876+0000 7faacd2a2740 1 client.17256 shutdown 2024-09-10T09:47:42.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:42.876+0000 7faacd2a2740 1 -- 172.21.15.5:0/1272057482 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a299407c70 msgr2=0x55a29941b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:47:42.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:42.876+0000 7faacd2a2740 1 --2- 172.21.15.5:0/1272057482 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a299407c70 0x55a29941b520 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7faab402a9e0 tx=0x7faab402a3f0 comp rx=0 tx=0).stop 2024-09-10T09:47:42.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:42.876+0000 7faacd2a2740 1 -- 172.21.15.5:0/1272057482 shutdown_connections 2024-09-10T09:47:42.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:42.876+0000 7faacd2a2740 1 --2- 172.21.15.5:0/1272057482 >> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] conn(0x55a29948dee0 0x55a2994902d0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:42.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:42.876+0000 7faacd2a2740 1 --2- 172.21.15.5:0/1272057482 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a299412e50 0x55a299416d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:42.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:42.876+0000 7faacd2a2740 1 --2- 172.21.15.5:0/1272057482 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a299408590 0x55a299416850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:42.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:42.876+0000 7faacd2a2740 1 --2- 172.21.15.5:0/1272057482 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a299407c70 0x55a29941b520 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:42.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:42.876+0000 7faacd2a2740 1 -- 172.21.15.5:0/1272057482 >> 172.21.15.5:0/1272057482 conn(0x55a299404200 msgr2=0x55a299403380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:47:42.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:42.876+0000 7faaad600640 0 client.17256 ms_handle_reset on v2:172.21.15.102:6836/2689776764 2024-09-10T09:47:42.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:42.876+0000 7faacd2a2740 1 -- 172.21.15.5:0/1272057482 shutdown_connections 2024-09-10T09:47:42.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:42.876+0000 7faacd2a2740 1 -- 172.21.15.5:0/1272057482 wait complete. 2024-09-10T09:47:42.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:42.876+0000 7faacd2a2740 20 client.17256 trim_cache size 0 max 16384 2024-09-10T09:47:42.882 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-10T09:47:42.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:42.877+0000 7faaca800640 1 -- 172.21.15.5:0/2478647722 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7faaa80a2b80 msgr2=0x7faaa80a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:47:42.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:42.877+0000 7faaca800640 1 --2- 172.21.15.5:0/2478647722 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7faaa80a2b80 0x7faaa80a5040 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7faabc009fd0 tx=0x7faabc009500 comp rx=0 tx=0).stop 2024-09-10T09:47:42.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:42.877+0000 7faaca800640 1 -- 172.21.15.5:0/2478647722 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faac41049a0 msgr2=0x7faac419abb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:47:42.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:42.877+0000 7faaca800640 1 --2- 172.21.15.5:0/2478647722 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faac41049a0 0x7faac419abb0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7faab000e9f0 tx=0x7faab000eeb0 comp rx=0 tx=0).stop 2024-09-10T09:47:42.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:42.877+0000 7faaca800640 1 -- 172.21.15.5:0/2478647722 shutdown_connections 2024-09-10T09:47:42.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:42.877+0000 7faaca800640 1 --2- 172.21.15.5:0/2478647722 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7faaa80a2b80 0x7faaa80a5040 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:42.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:42.877+0000 7faaca800640 1 --2- 172.21.15.5:0/2478647722 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7faac4109120 0x7faac419ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:42.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:42.877+0000 7faaca800640 1 --2- 172.21.15.5:0/2478647722 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faac41049a0 0x7faac419abb0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:42.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:42.877+0000 7faaca800640 1 --2- 172.21.15.5:0/2478647722 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7faac4103ff0 0x7faac419a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:42.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:42.877+0000 7faaca800640 1 -- 172.21.15.5:0/2478647722 >> 172.21.15.5:0/2478647722 conn(0x7faac40ffa90 msgr2=0x7faac4100d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:47:42.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:42.877+0000 7faaca800640 1 -- 172.21.15.5:0/2478647722 shutdown_connections 2024-09-10T09:47:42.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:42.878+0000 7faaca800640 1 -- 172.21.15.5:0/2478647722 wait complete. 2024-09-10T09:47:43.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:47:42 smithi005 ceph-mon[30350]: pgmap v674: 129 pgs: 129 active+clean; 214 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 1.2 MiB/s wr, 53 op/s 2024-09-10T09:47:43.087 INFO:tasks.workunit.client.0.smithi005.stdout: 1 555754 6.10 MB/sec execute 324 sec latency 18178.239 ms 2024-09-10T09:47:43.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:47:42 smithi159 ceph-mon[35513]: pgmap v674: 129 pgs: 129 active+clean; 214 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 1.2 MiB/s wr, 53 op/s 2024-09-10T09:47:43.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:47:42 smithi102 ceph-mon[35442]: pgmap v674: 129 pgs: 129 active+clean; 214 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 1.2 MiB/s wr, 53 op/s 2024-09-10T09:47:43.778 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-10T09:47:44.087 INFO:tasks.workunit.client.0.smithi005.stdout: 1 555754 6.08 MB/sec execute 325 sec latency 19178.295 ms 2024-09-10T09:47:44.156 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:47:44.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:44.862+0000 7ff769200640 1 -- 172.21.15.5:0/3869446200 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff7641034b0 msgr2=0x7ff76410b9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:47:44.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:44.862+0000 7ff769200640 1 --2- 172.21.15.5:0/3869446200 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff7641034b0 0x7ff76410b9a0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7ff750009900 tx=0x7ff75002a490 comp rx=0 tx=0).stop 2024-09-10T09:47:44.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:44.862+0000 7ff769200640 1 -- 172.21.15.5:0/3869446200 shutdown_connections 2024-09-10T09:47:44.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:44.862+0000 7ff769200640 1 --2- 172.21.15.5:0/3869446200 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff76410bfc0 0x7ff76410e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:44.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:44.862+0000 7ff769200640 1 --2- 172.21.15.5:0/3869446200 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff7641034b0 0x7ff76410b9a0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:44.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:44.862+0000 7ff769200640 1 --2- 172.21.15.5:0/3869446200 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff764102b90 0x7ff764102f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:44.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:44.862+0000 7ff769200640 1 -- 172.21.15.5:0/3869446200 >> 172.21.15.5:0/3869446200 conn(0x7ff7640fc610 msgr2=0x7ff7640fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:47:44.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:44.862+0000 7ff769200640 1 -- 172.21.15.5:0/3869446200 shutdown_connections 2024-09-10T09:47:44.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:44.862+0000 7ff769200640 1 -- 172.21.15.5:0/3869446200 wait complete. 2024-09-10T09:47:44.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:44.862+0000 7ff769200640 1 Processor -- start 2024-09-10T09:47:44.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:44.862+0000 7ff769200640 1 -- start start 2024-09-10T09:47:44.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:44.862+0000 7ff769200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff764102b90 0x7ff7641982c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:47:44.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:44.862+0000 7ff769200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff7641034b0 0x7ff764198800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:47:44.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:44.862+0000 7ff769200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff76410bfc0 0x7ff76419cb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:47:44.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:44.862+0000 7ff769200640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff764111dc0 con 0x7ff7641034b0 2024-09-10T09:47:44.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:44.862+0000 7ff769200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7ff764111c40 con 0x7ff76410bfc0 2024-09-10T09:47:44.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:44.862+0000 7ff769200640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7ff764111f40 con 0x7ff764102b90 2024-09-10T09:47:44.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:44.862+0000 7ff763400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff764102b90 0x7ff7641982c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:47:44.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:44.862+0000 7ff763400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff764102b90 0x7ff7641982c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:60224/0 (socket says 172.21.15.5:60224) 2024-09-10T09:47:44.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:44.862+0000 7ff763400640 1 -- 172.21.15.5:0/2911318761 learned_addr learned my addr 172.21.15.5:0/2911318761 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:47:44.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:44.862+0000 7ff763e00640 1 --2- 172.21.15.5:0/2911318761 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff76410bfc0 0x7ff76419cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:47:44.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:44.862+0000 7ff762a00640 1 --2- 172.21.15.5:0/2911318761 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff7641034b0 0x7ff764198800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:47:44.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:44.862+0000 7ff763400640 1 -- 172.21.15.5:0/2911318761 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff76410bfc0 msgr2=0x7ff76419cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:47:44.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:44.862+0000 7ff763400640 1 --2- 172.21.15.5:0/2911318761 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff76410bfc0 0x7ff76419cb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:44.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:44.862+0000 7ff763400640 1 -- 172.21.15.5:0/2911318761 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff7641034b0 msgr2=0x7ff764198800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:47:44.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:44.862+0000 7ff763400640 1 --2- 172.21.15.5:0/2911318761 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff7641034b0 0x7ff764198800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:44.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:44.862+0000 7ff763400640 1 -- 172.21.15.5:0/2911318761 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff76419d270 con 0x7ff764102b90 2024-09-10T09:47:44.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:44.862+0000 7ff763e00640 1 --2- 172.21.15.5:0/2911318761 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff76410bfc0 0x7ff76419cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:47:44.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:44.862+0000 7ff762a00640 1 --2- 172.21.15.5:0/2911318761 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff7641034b0 0x7ff764198800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:47:44.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:44.868+0000 7ff763400640 1 --2- 172.21.15.5:0/2911318761 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff764102b90 0x7ff7641982c0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7ff75400e9f0 tx=0x7ff75400edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:47:44.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:44.868+0000 7ff75be00640 1 -- 172.21.15.5:0/2911318761 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff75400cd80 con 0x7ff764102b90 2024-09-10T09:47:44.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:44.868+0000 7ff769200640 1 -- 172.21.15.5:0/2911318761 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff76419d560 con 0x7ff764102b90 2024-09-10T09:47:44.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:44.868+0000 7ff75be00640 1 -- 172.21.15.5:0/2911318761 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff754005500 con 0x7ff764102b90 2024-09-10T09:47:44.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:44.868+0000 7ff75be00640 1 -- 172.21.15.5:0/2911318761 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff754010690 con 0x7ff764102b90 2024-09-10T09:47:44.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:44.868+0000 7ff769200640 1 -- 172.21.15.5:0/2911318761 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7641a4e40 con 0x7ff764102b90 2024-09-10T09:47:44.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:44.871+0000 7ff75be00640 1 -- 172.21.15.5:0/2911318761 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7ff7540046c0 con 0x7ff764102b90 2024-09-10T09:47:44.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:44.871+0000 7ff76d3c6740 1 Processor -- start 2024-09-10T09:47:44.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:44.871+0000 7ff76d3c6740 1 -- start start 2024-09-10T09:47:44.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:44.871+0000 7ff76d3c6740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bd5846a080 0x55bd5846a460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:47:44.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:44.871+0000 7ff76d3c6740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bd5846a9a0 0x55bd584649d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:47:44.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:44.871+0000 7ff76d3c6740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bd58465040 0x55bd584654c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:47:44.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:44.871+0000 7ff76d3c6740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55bd583c13e0 con 0x55bd5846a9a0 2024-09-10T09:47:44.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:44.871+0000 7ff76d3c6740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55bd58368d80 con 0x55bd5846a080 2024-09-10T09:47:44.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:44.871+0000 7ff76d3c6740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55bd583bd3a0 con 0x55bd58465040 2024-09-10T09:47:44.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:44.871+0000 7ff763400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bd5846a9a0 0x55bd584649d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:47:44.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:44.871+0000 7ff762a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bd58465040 0x55bd584654c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:47:44.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:44.871+0000 7ff763e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bd5846a080 0x55bd5846a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:47:44.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:44.871+0000 7ff762a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bd58465040 0x55bd584654c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:60240/0 (socket says 172.21.15.5:60240) 2024-09-10T09:47:44.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:44.871+0000 7ff762a00640 1 -- 172.21.15.5:0/4192842433 learned_addr learned my addr 172.21.15.5:0/4192842433 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:47:44.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:44.871+0000 7ff75be00640 1 --2- 172.21.15.5:0/2911318761 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7ff7480a2b80 0x7ff7480a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:47:44.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:44.871+0000 7ff762a00640 1 -- 172.21.15.5:0/4192842433 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bd5846a080 msgr2=0x55bd5846a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:47:44.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:44.871+0000 7ff762a00640 1 --2- 172.21.15.5:0/4192842433 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bd5846a080 0x55bd5846a460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:44.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:44.871+0000 7ff762a00640 1 -- 172.21.15.5:0/4192842433 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bd5846a9a0 msgr2=0x55bd584649d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:47:44.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:44.871+0000 7ff762a00640 1 --2- 172.21.15.5:0/4192842433 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bd5846a9a0 0x55bd584649d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:44.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:44.871+0000 7ff762a00640 1 -- 172.21.15.5:0/4192842433 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bd58387ae0 con 0x55bd58465040 2024-09-10T09:47:44.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:44.871+0000 7ff75be00640 1 -- 172.21.15.5:0/2911318761 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7ff7540cf920 con 0x7ff764102b90 2024-09-10T09:47:44.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:44.871+0000 7ff763e00640 1 --2- 172.21.15.5:0/4192842433 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bd5846a080 0x55bd5846a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:47:44.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:44.871+0000 7ff762a00640 1 --2- 172.21.15.5:0/4192842433 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bd58465040 0x55bd584654c0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7ff750009900 tx=0x7ff750005e60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4e3f87bda0d48c23 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:47:44.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:44.871+0000 7ff763e00640 1 --2- 172.21.15.5:0/2911318761 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7ff7480a2b80 0x7ff7480a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:47:44.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:44.871+0000 7ff758c00640 1 -- 172.21.15.5:0/4192842433 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff75003f070 con 0x55bd58465040 2024-09-10T09:47:44.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:44.871+0000 7ff758c00640 1 -- 172.21.15.5:0/4192842433 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff750004030 con 0x55bd58465040 2024-09-10T09:47:44.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:44.871+0000 7ff758c00640 1 -- 172.21.15.5:0/4192842433 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff750035070 con 0x55bd58465040 2024-09-10T09:47:44.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:44.871+0000 7ff76d3c6740 1 -- 172.21.15.5:0/4192842433 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bd58465040 msgr2=0x55bd584654c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:47:44.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:44.871+0000 7ff76d3c6740 1 --2- 172.21.15.5:0/4192842433 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bd58465040 0x55bd584654c0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7ff750009900 tx=0x7ff750005e60 comp rx=0 tx=0).stop 2024-09-10T09:47:44.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:44.871+0000 7ff763e00640 1 --2- 172.21.15.5:0/2911318761 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7ff7480a2b80 0x7ff7480a5040 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7ff75c005fd0 tx=0x7ff75c014040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:47:44.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:44.871+0000 7ff76d3c6740 1 -- 172.21.15.5:0/4192842433 shutdown_connections 2024-09-10T09:47:44.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:44.871+0000 7ff76d3c6740 1 --2- 172.21.15.5:0/4192842433 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bd58465040 0x55bd584654c0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:44.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:44.871+0000 7ff76d3c6740 1 --2- 172.21.15.5:0/4192842433 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bd5846a9a0 0x55bd584649d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:44.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:44.871+0000 7ff76d3c6740 1 --2- 172.21.15.5:0/4192842433 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bd5846a080 0x55bd5846a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:44.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:44.871+0000 7ff76d3c6740 1 -- 172.21.15.5:0/4192842433 >> 172.21.15.5:0/4192842433 conn(0x55bd58456e80 msgr2=0x55bd58457260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:47:44.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:44.876+0000 7ff76d3c6740 1 -- 172.21.15.5:0/4192842433 shutdown_connections 2024-09-10T09:47:44.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:44.876+0000 7ff76d3c6740 1 -- 172.21.15.5:0/4192842433 wait complete. 2024-09-10T09:47:44.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:44.876+0000 7ff76d3c6740 1 -- start start 2024-09-10T09:47:44.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:44.876+0000 7ff76d3c6740 1 Processor -- start 2024-09-10T09:47:44.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:44.876+0000 7ff76d3c6740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bd58465040 0x55bd5846dec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:47:44.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:44.876+0000 7ff76d3c6740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bd5846a080 0x55bd5846e400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:47:44.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:44.876+0000 7ff76d3c6740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bd5846a9a0 0x55bd5846e940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:47:44.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:44.876+0000 7ff76d3c6740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55bd58471360 con 0x55bd58465040 2024-09-10T09:47:44.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:44.876+0000 7ff76d3c6740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55bd5837c0f0 con 0x55bd5846a080 2024-09-10T09:47:44.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:44.876+0000 7ff76d3c6740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55bd584714e0 con 0x55bd5846a9a0 2024-09-10T09:47:44.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:44.876+0000 7ff76d3c6740 10 client.0 _ll_register_callbacks cb 0x55bd58363bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:47:44.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:44.876+0000 7ff762a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bd5846a080 0x55bd5846e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:47:44.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:44.876+0000 7ff763400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bd58465040 0x55bd5846dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:47:44.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:44.876+0000 7ff762a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bd5846a080 0x55bd5846e400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:38644/0 (socket says 172.21.15.5:38644) 2024-09-10T09:47:44.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:44.876+0000 7ff762a00640 1 -- 172.21.15.5:0/404400109 learned_addr learned my addr 172.21.15.5:0/404400109 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:47:44.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:44.876+0000 7ff763e00640 1 --2- 172.21.15.5:0/404400109 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bd5846a9a0 0x55bd5846e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:47:44.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:44.876+0000 7ff762a00640 1 -- 172.21.15.5:0/404400109 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bd5846a9a0 msgr2=0x55bd5846e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:47:44.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:44.876+0000 7ff762a00640 1 --2- 172.21.15.5:0/404400109 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bd5846a9a0 0x55bd5846e940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:44.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:44.876+0000 7ff762a00640 1 -- 172.21.15.5:0/404400109 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bd58465040 msgr2=0x55bd5846dec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:47:44.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:44.876+0000 7ff762a00640 1 --2- 172.21.15.5:0/404400109 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bd58465040 0x55bd5846dec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:44.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:44.876+0000 7ff762a00640 1 -- 172.21.15.5:0/404400109 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bd581323c0 con 0x55bd5846a080 2024-09-10T09:47:44.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:44.876+0000 7ff762a00640 1 --2- 172.21.15.5:0/404400109 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bd5846a080 0x55bd5846e400 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7ff750032fe0 tx=0x7ff750031400 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ecda36ffa13c6971 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:47:44.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:44.876+0000 7ff74d600640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-09-10T09:47:44.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:44.876+0000 7ff74d600640 1 -- 172.21.15.5:0/404400109 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff750035070 con 0x55bd5846a080 2024-09-10T09:47:44.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:44.876+0000 7ff76d3c6740 1 -- 172.21.15.5:0/404400109 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff750004030 con 0x55bd5846a080 2024-09-10T09:47:44.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:44.876+0000 7ff74d600640 1 -- 172.21.15.5:0/404400109 <== mon.2 v2:172.21.15.102:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7ff7500315b0 con 0x55bd5846a080 2024-09-10T09:47:44.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:44.876+0000 7ff74d600640 1 -- 172.21.15.5:0/404400109 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7ff750055050 con 0x55bd5846a080 2024-09-10T09:47:44.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:44.876+0000 7ff76d3c6740 10 client.27260 fetch_fsmap learned FSMap version 35 2024-09-10T09:47:44.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:44.876+0000 7ff76d3c6740 1 -- 172.21.15.5:0/404400109 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55bd5847d4b0 con 0x55bd5846a080 2024-09-10T09:47:44.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:44.876+0000 7ff74d600640 1 -- 172.21.15.5:0/404400109 <== mon.2 v2:172.21.15.102:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff750004030 con 0x55bd5846a080 2024-09-10T09:47:44.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:44.876+0000 7ff74d600640 1 -- 172.21.15.5:0/404400109 <== mon.2 v2:172.21.15.102:3300/0 5 ==== fsmap(e 35) ==== 8374+0+0 (secure 0 0 0) 0x7ff7500046c0 con 0x55bd5846a080 2024-09-10T09:47:44.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:44.876+0000 7ff76d3c6740 10 client.27260 fetch_fsmap finished waiting for FSMap version 35 2024-09-10T09:47:44.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:44.876+0000 7ff76d3c6740 10 client.27260 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-10T09:47:44.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:44.876+0000 7ff76d3c6740 20 client.27260 populate_metadata read hostname 'smithi005' 2024-09-10T09:47:44.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:44.876+0000 7ff76d3c6740 1 --2- 172.21.15.5:0/404400109 >> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] conn(0x55bd5847e800 0x55bd58480bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:47:44.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:44.876+0000 7ff76d3c6740 4 client.27260 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:47:44.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:44.876+0000 7ff76d3c6740 1 -- 172.21.15.5:0/404400109 --> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bd583ce950 con 0x55bd5847e800 2024-09-10T09:47:44.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:44.876+0000 7ff763400640 1 --2- 172.21.15.5:0/404400109 >> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] conn(0x55bd5847e800 0x55bd58480bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:47:44.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:44.881+0000 7ff763400640 1 --2- 172.21.15.5:0/404400109 >> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] conn(0x55bd5847e800 0x55bd58480bf0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c12bc8b2be5eb1bb server_cookie=8dcaf7bfd977896 in_seq=0 out_seq=0 2024-09-10T09:47:44.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:44.881+0000 7ff74d600640 10 client.27260 ms_handle_connect on v2:172.21.15.159:6836/620627639 2024-09-10T09:47:44.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:44.881+0000 7ff74d600640 1 -- 172.21.15.5:0/404400109 <== mds.2 v2:172.21.15.159:6836/620627639 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55bd583ce950 con 0x55bd5847e800 2024-09-10T09:47:44.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:44.881+0000 7ff74d600640 10 client.27260 handle_command_reply: tid=0 2024-09-10T09:47:44.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:44.881+0000 7ff76d3c6740 1 client.27260 shutdown 2024-09-10T09:47:44.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:44.881+0000 7ff76d3c6740 1 -- 172.21.15.5:0/404400109 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bd5846a080 msgr2=0x55bd5846e400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:47:44.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:44.881+0000 7ff76d3c6740 1 --2- 172.21.15.5:0/404400109 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bd5846a080 0x55bd5846e400 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7ff750032fe0 tx=0x7ff750031400 comp rx=0 tx=0).stop 2024-09-10T09:47:44.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:44.881+0000 7ff76d3c6740 1 -- 172.21.15.5:0/404400109 shutdown_connections 2024-09-10T09:47:44.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:44.881+0000 7ff76d3c6740 1 --2- 172.21.15.5:0/404400109 >> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] conn(0x55bd5847e800 0x55bd58480bf0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:44.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:44.881+0000 7ff76d3c6740 1 --2- 172.21.15.5:0/404400109 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bd5846a9a0 0x55bd5846e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:44.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:44.881+0000 7ff76d3c6740 1 --2- 172.21.15.5:0/404400109 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bd5846a080 0x55bd5846e400 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:44.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:44.881+0000 7ff76d3c6740 1 --2- 172.21.15.5:0/404400109 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bd58465040 0x55bd5846dec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:44.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:44.881+0000 7ff76d3c6740 1 -- 172.21.15.5:0/404400109 >> 172.21.15.5:0/404400109 conn(0x55bd58456e80 msgr2=0x55bd58456490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:47:44.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:44.881+0000 7ff74d600640 0 client.27260 ms_handle_reset on v2:172.21.15.159:6836/620627639 2024-09-10T09:47:44.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:44.881+0000 7ff76d3c6740 1 -- 172.21.15.5:0/404400109 shutdown_connections 2024-09-10T09:47:44.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:44.881+0000 7ff76d3c6740 1 -- 172.21.15.5:0/404400109 wait complete. 2024-09-10T09:47:44.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:44.883+0000 7ff76d3c6740 20 client.27260 trim_cache size 0 max 16384 2024-09-10T09:47:44.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:44.895+0000 7ff76d3c6740 1 Processor -- start 2024-09-10T09:47:44.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:44.895+0000 7ff76d3c6740 1 -- start start 2024-09-10T09:47:44.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:44.895+0000 7ff76d3c6740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bd58465040 0x55bd584649d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:47:44.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:44.895+0000 7ff76d3c6740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bd5846a080 0x55bd58455c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:47:44.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:44.900+0000 7ff76d3c6740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bd5846a9a0 0x55bd584561b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:47:44.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:44.900+0000 7ff76d3c6740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff7641acab0 con 0x55bd5846a9a0 2024-09-10T09:47:44.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:44.900+0000 7ff76d3c6740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7ff7641ae620 con 0x55bd5846a080 2024-09-10T09:47:44.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:44.900+0000 7ff76d3c6740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7ff7641ad180 con 0x55bd58465040 2024-09-10T09:47:44.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:44.900+0000 7ff763400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bd58465040 0x55bd584649d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:47:44.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:44.900+0000 7ff763400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bd58465040 0x55bd584649d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:60256/0 (socket says 172.21.15.5:60256) 2024-09-10T09:47:44.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:44.900+0000 7ff763400640 1 -- 172.21.15.5:0/3617450566 learned_addr learned my addr 172.21.15.5:0/3617450566 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:47:44.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:44.900+0000 7ff762a00640 1 --2- 172.21.15.5:0/3617450566 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bd5846a080 0x55bd58455c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:47:44.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:44.900+0000 7ff763e00640 1 --2- 172.21.15.5:0/3617450566 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bd5846a9a0 0x55bd584561b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:47:44.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:44.900+0000 7ff763400640 1 -- 172.21.15.5:0/3617450566 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bd5846a080 msgr2=0x55bd58455c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:47:44.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:44.900+0000 7ff763400640 1 --2- 172.21.15.5:0/3617450566 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bd5846a080 0x55bd58455c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:44.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:44.900+0000 7ff763400640 1 -- 172.21.15.5:0/3617450566 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bd5846a9a0 msgr2=0x55bd584561b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:47:44.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:44.900+0000 7ff763400640 1 --2- 172.21.15.5:0/3617450566 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bd5846a9a0 0x55bd584561b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:44.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:44.900+0000 7ff763400640 1 -- 172.21.15.5:0/3617450566 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bd58469cc0 con 0x55bd58465040 2024-09-10T09:47:44.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:44.900+0000 7ff762a00640 1 --2- 172.21.15.5:0/3617450566 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bd5846a080 0x55bd58455c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:47:44.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:44.900+0000 7ff763400640 1 --2- 172.21.15.5:0/3617450566 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bd58465040 0x55bd584649d0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7ff75409ba30 tx=0x7ff7540d5530 comp rx=0 tx=0).ready entity=mon.1 client_cookie=27c82bd56e4b9852 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:47:44.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:44.900+0000 7ff74fe00640 1 -- 172.21.15.5:0/3617450566 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7540d5020 con 0x55bd58465040 2024-09-10T09:47:44.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:44.900+0000 7ff74fe00640 1 -- 172.21.15.5:0/3617450566 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff75409c430 con 0x55bd58465040 2024-09-10T09:47:44.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:44.900+0000 7ff74fe00640 1 -- 172.21.15.5:0/3617450566 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7540cd0e0 con 0x55bd58465040 2024-09-10T09:47:44.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:44.900+0000 7ff76d3c6740 1 -- 172.21.15.5:0/3617450566 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bd58465040 msgr2=0x55bd584649d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:47:44.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:44.900+0000 7ff76d3c6740 1 --2- 172.21.15.5:0/3617450566 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bd58465040 0x55bd584649d0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7ff75409ba30 tx=0x7ff7540d5530 comp rx=0 tx=0).stop 2024-09-10T09:47:44.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:44.900+0000 7ff76d3c6740 1 -- 172.21.15.5:0/3617450566 shutdown_connections 2024-09-10T09:47:44.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:44.900+0000 7ff76d3c6740 1 --2- 172.21.15.5:0/3617450566 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bd5846a9a0 0x55bd584561b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:44.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:44.900+0000 7ff76d3c6740 1 --2- 172.21.15.5:0/3617450566 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bd5846a080 0x55bd58455c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:44.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:44.900+0000 7ff76d3c6740 1 --2- 172.21.15.5:0/3617450566 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bd58465040 0x55bd584649d0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:44.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:44.900+0000 7ff76d3c6740 1 -- 172.21.15.5:0/3617450566 >> 172.21.15.5:0/3617450566 conn(0x55bd58456e80 msgr2=0x55bd5846cfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:47:44.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:44.900+0000 7ff76d3c6740 1 -- 172.21.15.5:0/3617450566 shutdown_connections 2024-09-10T09:47:44.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:44.900+0000 7ff76d3c6740 1 -- 172.21.15.5:0/3617450566 wait complete. 2024-09-10T09:47:44.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:44.900+0000 7ff76d3c6740 1 -- start start 2024-09-10T09:47:44.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:44.900+0000 7ff76d3c6740 1 Processor -- start 2024-09-10T09:47:44.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:44.900+0000 7ff76d3c6740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bd58465040 0x55bd58477060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:47:44.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:44.900+0000 7ff76d3c6740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bd5846a080 0x55bd58485d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:47:44.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:44.900+0000 7ff76d3c6740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bd5846a9a0 0x55bd58486490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:47:44.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:44.900+0000 7ff76d3c6740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff7641adf20 con 0x55bd58465040 2024-09-10T09:47:44.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:44.900+0000 7ff76d3c6740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7ff7641ad850 con 0x55bd5846a080 2024-09-10T09:47:44.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:44.900+0000 7ff76d3c6740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7ff7641aaa90 con 0x55bd5846a9a0 2024-09-10T09:47:44.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:44.900+0000 7ff76d3c6740 10 client.0 _ll_register_callbacks cb 0x7ff7540d5400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:47:44.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:44.900+0000 7ff763400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bd58465040 0x55bd58477060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:47:44.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:44.900+0000 7ff762a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bd5846a080 0x55bd58485d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:47:44.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:44.900+0000 7ff762a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bd5846a080 0x55bd58485d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:38666/0 (socket says 172.21.15.5:38666) 2024-09-10T09:47:44.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:44.900+0000 7ff762a00640 1 -- 172.21.15.5:0/2988951439 learned_addr learned my addr 172.21.15.5:0/2988951439 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:47:44.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:44.900+0000 7ff763e00640 1 --2- 172.21.15.5:0/2988951439 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bd5846a9a0 0x55bd58486490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:47:44.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:44.900+0000 7ff762a00640 1 -- 172.21.15.5:0/2988951439 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bd5846a9a0 msgr2=0x55bd58486490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:47:44.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:44.900+0000 7ff762a00640 1 --2- 172.21.15.5:0/2988951439 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bd5846a9a0 0x55bd58486490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:44.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:44.900+0000 7ff762a00640 1 -- 172.21.15.5:0/2988951439 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bd58465040 msgr2=0x55bd58477060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:47:44.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:44.900+0000 7ff762a00640 1 --2- 172.21.15.5:0/2988951439 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bd58465040 0x55bd58477060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:44.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:44.900+0000 7ff762a00640 1 -- 172.21.15.5:0/2988951439 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bd584e8160 con 0x55bd5846a080 2024-09-10T09:47:44.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:44.900+0000 7ff762a00640 1 --2- 172.21.15.5:0/2988951439 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bd5846a080 0x55bd58485d00 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7ff75002a9b0 tx=0x7ff750009520 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a74fa8fca082c74c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:47:44.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:44.900+0000 7ff74d600640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-09-10T09:47:44.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:44.905+0000 7ff74d600640 1 -- 172.21.15.5:0/2988951439 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff750035050 con 0x55bd5846a080 2024-09-10T09:47:44.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:44.905+0000 7ff76d3c6740 1 -- 172.21.15.5:0/2988951439 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff75409c430 con 0x55bd5846a080 2024-09-10T09:47:44.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:44.905+0000 7ff74d600640 1 -- 172.21.15.5:0/2988951439 <== mon.2 v2:172.21.15.102:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7ff7500315b0 con 0x55bd5846a080 2024-09-10T09:47:44.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:44.905+0000 7ff74d600640 1 -- 172.21.15.5:0/2988951439 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7ff75003f040 con 0x55bd5846a080 2024-09-10T09:47:44.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:44.905+0000 7ff76d3c6740 10 client.27266 fetch_fsmap learned FSMap version 35 2024-09-10T09:47:44.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:44.905+0000 7ff76d3c6740 1 -- 172.21.15.5:0/2988951439 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55bd584e9320 con 0x55bd5846a080 2024-09-10T09:47:44.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:44.905+0000 7ff74d600640 1 -- 172.21.15.5:0/2988951439 <== mon.2 v2:172.21.15.102:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff75409c430 con 0x55bd5846a080 2024-09-10T09:47:44.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:44.905+0000 7ff74d600640 1 -- 172.21.15.5:0/2988951439 <== mon.2 v2:172.21.15.102:3300/0 5 ==== fsmap(e 35) ==== 8374+0+0 (secure 0 0 0) 0x7ff750059430 con 0x55bd5846a080 2024-09-10T09:47:44.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:44.905+0000 7ff76d3c6740 10 client.27266 fetch_fsmap finished waiting for FSMap version 35 2024-09-10T09:47:44.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:44.905+0000 7ff76d3c6740 10 client.27266 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-10T09:47:44.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:44.905+0000 7ff76d3c6740 20 client.27266 populate_metadata read hostname 'smithi005' 2024-09-10T09:47:44.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:44.905+0000 7ff76d3c6740 1 --2- 172.21.15.5:0/2988951439 >> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] conn(0x55bd584e9b20 0x55bd584ebf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:47:44.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:44.905+0000 7ff76d3c6740 4 client.27266 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T09:47:44.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:44.905+0000 7ff76d3c6740 1 -- 172.21.15.5:0/2988951439 --> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55bd583c8e60 con 0x55bd584e9b20 2024-09-10T09:47:44.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:44.906+0000 7ff763400640 1 --2- 172.21.15.5:0/2988951439 >> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] conn(0x55bd584e9b20 0x55bd584ebf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:47:44.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:44.906+0000 7ff763400640 1 --2- 172.21.15.5:0/2988951439 >> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] conn(0x55bd584e9b20 0x55bd584ebf40 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c7fd9c0e162da3ad server_cookie=71aa219cea62cb38 in_seq=0 out_seq=0 2024-09-10T09:47:44.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:44.906+0000 7ff74d600640 10 client.27266 ms_handle_connect on v2:172.21.15.159:6836/620627639 2024-09-10T09:47:44.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:44.906+0000 7ff74d600640 1 -- 172.21.15.5:0/2988951439 <== mds.2 v2:172.21.15.159:6836/620627639 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55bd583c8e60 con 0x55bd584e9b20 2024-09-10T09:47:44.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:44.906+0000 7ff74d600640 10 client.27266 handle_command_reply: tid=0 2024-09-10T09:47:44.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:44.906+0000 7ff76d3c6740 1 client.27266 shutdown 2024-09-10T09:47:44.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:44.906+0000 7ff76d3c6740 1 -- 172.21.15.5:0/2988951439 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bd5846a080 msgr2=0x55bd58485d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:47:44.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:44.906+0000 7ff76d3c6740 1 --2- 172.21.15.5:0/2988951439 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bd5846a080 0x55bd58485d00 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7ff75002a9b0 tx=0x7ff750009520 comp rx=0 tx=0).stop 2024-09-10T09:47:44.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:44.906+0000 7ff76d3c6740 1 -- 172.21.15.5:0/2988951439 shutdown_connections 2024-09-10T09:47:44.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:44.906+0000 7ff76d3c6740 1 --2- 172.21.15.5:0/2988951439 >> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] conn(0x55bd584e9b20 0x55bd584ebf40 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:44.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:44.906+0000 7ff76d3c6740 1 --2- 172.21.15.5:0/2988951439 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bd5846a9a0 0x55bd58486490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:44.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:44.906+0000 7ff76d3c6740 1 --2- 172.21.15.5:0/2988951439 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bd5846a080 0x55bd58485d00 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:44.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:44.906+0000 7ff76d3c6740 1 --2- 172.21.15.5:0/2988951439 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bd58465040 0x55bd58477060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:44.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:44.906+0000 7ff74d600640 0 client.27266 ms_handle_reset on v2:172.21.15.159:6836/620627639 2024-09-10T09:47:44.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:44.906+0000 7ff76d3c6740 1 -- 172.21.15.5:0/2988951439 >> 172.21.15.5:0/2988951439 conn(0x55bd58456e80 msgr2=0x55bd5846f720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:47:44.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:44.906+0000 7ff76d3c6740 1 -- 172.21.15.5:0/2988951439 shutdown_connections 2024-09-10T09:47:44.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:44.906+0000 7ff76d3c6740 1 -- 172.21.15.5:0/2988951439 wait complete. 2024-09-10T09:47:44.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:44.906+0000 7ff76d3c6740 20 client.27266 trim_cache size 0 max 16384 2024-09-10T09:47:44.912 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-10T09:47:44.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:44.906+0000 7ff769200640 1 -- 172.21.15.5:0/2911318761 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7ff7480a2b80 msgr2=0x7ff7480a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:47:44.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:44.906+0000 7ff769200640 1 --2- 172.21.15.5:0/2911318761 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7ff7480a2b80 0x7ff7480a5040 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7ff75c005fd0 tx=0x7ff75c014040 comp rx=0 tx=0).stop 2024-09-10T09:47:44.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:44.906+0000 7ff769200640 1 -- 172.21.15.5:0/2911318761 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff764102b90 msgr2=0x7ff7641982c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:47:44.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:44.906+0000 7ff769200640 1 --2- 172.21.15.5:0/2911318761 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff764102b90 0x7ff7641982c0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7ff75400e9f0 tx=0x7ff75400edf0 comp rx=0 tx=0).stop 2024-09-10T09:47:44.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:44.906+0000 7ff769200640 1 -- 172.21.15.5:0/2911318761 shutdown_connections 2024-09-10T09:47:44.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:44.906+0000 7ff769200640 1 --2- 172.21.15.5:0/2911318761 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7ff7480a2b80 0x7ff7480a5040 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:44.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:44.906+0000 7ff769200640 1 --2- 172.21.15.5:0/2911318761 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff76410bfc0 0x7ff76419cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:44.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:44.906+0000 7ff769200640 1 --2- 172.21.15.5:0/2911318761 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff7641034b0 0x7ff764198800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:44.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:44.906+0000 7ff769200640 1 --2- 172.21.15.5:0/2911318761 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff764102b90 0x7ff7641982c0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:44.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:44.906+0000 7ff769200640 1 -- 172.21.15.5:0/2911318761 >> 172.21.15.5:0/2911318761 conn(0x7ff7640fc610 msgr2=0x7ff7640fd2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:47:44.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:44.906+0000 7ff769200640 1 -- 172.21.15.5:0/2911318761 shutdown_connections 2024-09-10T09:47:44.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:44.906+0000 7ff769200640 1 -- 172.21.15.5:0/2911318761 wait complete. 2024-09-10T09:47:45.087 INFO:tasks.workunit.client.0.smithi005.stdout: 1 555754 6.06 MB/sec execute 326 sec latency 20178.358 ms 2024-09-10T09:47:45.096 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:47:44 smithi005 ceph-mon[30350]: pgmap v675: 129 pgs: 129 active+clean; 213 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 1.2 MiB/s wr, 55 op/s 2024-09-10T09:47:45.096 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:47:44 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:47:45.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:47:44 smithi159 ceph-mon[35513]: pgmap v675: 129 pgs: 129 active+clean; 213 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 1.2 MiB/s wr, 55 op/s 2024-09-10T09:47:45.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:47:44 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:47:45.122 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:47:44 smithi102 ceph-mon[35442]: pgmap v675: 129 pgs: 129 active+clean; 213 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 1.2 MiB/s wr, 55 op/s 2024-09-10T09:47:45.122 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:47:44 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:47:45.885 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-10T09:47:46.087 INFO:tasks.workunit.client.0.smithi005.stdout: 1 555754 6.04 MB/sec execute 327 sec latency 21178.400 ms 2024-09-10T09:47:46.175 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T09:47:46.827 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:47:46.835 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:47:47.087 INFO:tasks.workunit.client.0.smithi005.stdout: 1 555754 6.03 MB/sec execute 328 sec latency 22178.482 ms 2024-09-10T09:47:47.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:47:46 smithi102 ceph-mon[35442]: pgmap v676: 129 pgs: 129 active+clean; 214 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 1.3 MiB/s wr, 64 op/s 2024-09-10T09:47:47.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:47:46 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:47:47.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:47:46 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:47:47.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:47:46 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:47:47.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:47:46 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:47:47.146 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:47:46 smithi005 ceph-mon[30350]: pgmap v676: 129 pgs: 129 active+clean; 214 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 1.3 MiB/s wr, 64 op/s 2024-09-10T09:47:47.146 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:47:46 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:47:47.146 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:47:46 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:47:47.146 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:47:46 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:47:47.146 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:47:46 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:47:47.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:47:46 smithi159 ceph-mon[35513]: pgmap v676: 129 pgs: 129 active+clean; 214 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 1.3 MiB/s wr, 64 op/s 2024-09-10T09:47:47.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:47:46 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:47:47.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:47:46 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:47:47.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:47:46 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:47:47.351 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:47:46 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:47:47.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:47.606+0000 7fee12400640 1 -- 172.21.15.5:0/4084504239 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fee0c1049c0 msgr2=0x7fee0c108a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:47:47.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:47.606+0000 7fee12400640 1 --2- 172.21.15.5:0/4084504239 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fee0c1049c0 0x7fee0c108a10 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fedf4009900 tx=0x7fedf402a410 comp rx=0 tx=0).stop 2024-09-10T09:47:47.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:47.607+0000 7fee12400640 1 -- 172.21.15.5:0/4084504239 shutdown_connections 2024-09-10T09:47:47.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:47.607+0000 7fee12400640 1 --2- 172.21.15.5:0/4084504239 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fee0c109140 0x7fee0c110cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:47.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:47.607+0000 7fee12400640 1 --2- 172.21.15.5:0/4084504239 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fee0c1049c0 0x7fee0c108a10 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:47.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:47.607+0000 7fee12400640 1 --2- 172.21.15.5:0/4084504239 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fee0c104010 0x7fee0c1043f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:47.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:47.607+0000 7fee12400640 1 -- 172.21.15.5:0/4084504239 >> 172.21.15.5:0/4084504239 conn(0x7fee0c0ffa50 msgr2=0x7fee0c101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:47:47.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:47.607+0000 7fee12400640 1 -- 172.21.15.5:0/4084504239 shutdown_connections 2024-09-10T09:47:47.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:47.607+0000 7fee12400640 1 -- 172.21.15.5:0/4084504239 wait complete. 2024-09-10T09:47:47.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:47.608+0000 7fee12400640 1 Processor -- start 2024-09-10T09:47:47.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:47.608+0000 7fee12400640 1 -- start start 2024-09-10T09:47:47.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:47.608+0000 7fee12400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fee0c104010 0x7fee0c19a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:47:47.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:47.608+0000 7fee12400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fee0c1049c0 0x7fee0c19ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:47:47.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:47.608+0000 7fee12400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fee0c109140 0x7fee0c19eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:47:47.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:47.608+0000 7fee12400640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fee0c114160 con 0x7fee0c109140 2024-09-10T09:47:47.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:47.608+0000 7fee12400640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fee0c113fe0 con 0x7fee0c104010 2024-09-10T09:47:47.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:47.608+0000 7fee12400640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fee0c1142e0 con 0x7fee0c1049c0 2024-09-10T09:47:47.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:47.608+0000 7fee0be00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fee0c109140 0x7fee0c19eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:47:47.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:47.608+0000 7fee0be00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fee0c109140 0x7fee0c19eff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57242/0 (socket says 172.21.15.5:57242) 2024-09-10T09:47:47.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:47.609+0000 7fee0be00640 1 -- 172.21.15.5:0/3176677273 learned_addr learned my addr 172.21.15.5:0/3176677273 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:47:47.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:47.609+0000 7fee02a00640 1 --2- 172.21.15.5:0/3176677273 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fee0c1049c0 0x7fee0c19ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:47:47.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:47.609+0000 7fee0b400640 1 --2- 172.21.15.5:0/3176677273 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fee0c104010 0x7fee0c19a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:47:47.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:47.609+0000 7fee0be00640 1 -- 172.21.15.5:0/3176677273 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fee0c1049c0 msgr2=0x7fee0c19ac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:47:47.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:47.609+0000 7fee0be00640 1 --2- 172.21.15.5:0/3176677273 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fee0c1049c0 0x7fee0c19ac60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:47.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:47.609+0000 7fee0be00640 1 -- 172.21.15.5:0/3176677273 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fee0c104010 msgr2=0x7fee0c19a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:47:47.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:47.609+0000 7fee0be00640 1 --2- 172.21.15.5:0/3176677273 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fee0c104010 0x7fee0c19a720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:47.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:47.609+0000 7fee0be00640 1 -- 172.21.15.5:0/3176677273 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee0c19f6d0 con 0x7fee0c109140 2024-09-10T09:47:47.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:47.609+0000 7fee0be00640 1 --2- 172.21.15.5:0/3176677273 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fee0c109140 0x7fee0c19eff0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fee04009fd0 tx=0x7fee0400b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:47:47.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:47.610+0000 7fee08c00640 1 -- 172.21.15.5:0/3176677273 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee04017070 con 0x7fee0c109140 2024-09-10T09:47:47.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:47.610+0000 7fee08c00640 1 -- 172.21.15.5:0/3176677273 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fee04003e50 con 0x7fee0c109140 2024-09-10T09:47:47.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:47.610+0000 7fee08c00640 1 -- 172.21.15.5:0/3176677273 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee040106c0 con 0x7fee0c109140 2024-09-10T09:47:47.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:47.610+0000 7fee12400640 1 -- 172.21.15.5:0/3176677273 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee0c19f9c0 con 0x7fee0c109140 2024-09-10T09:47:47.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:47.610+0000 7fee12400640 1 -- 172.21.15.5:0/3176677273 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee0c1a72a0 con 0x7fee0c109140 2024-09-10T09:47:47.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:47.611+0000 7fee12400640 1 -- 172.21.15.5:0/3176677273 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fee0c1043f0 con 0x7fee0c109140 2024-09-10T09:47:47.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:47.613+0000 7fee08c00640 1 -- 172.21.15.5:0/3176677273 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7fee04003970 con 0x7fee0c109140 2024-09-10T09:47:47.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:47.614+0000 7fee08c00640 1 --2- 172.21.15.5:0/3176677273 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fedf00a2c00 0x7fedf00a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:47:47.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:47.614+0000 7fee08c00640 1 -- 172.21.15.5:0/3176677273 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fee040cf080 con 0x7fee0c109140 2024-09-10T09:47:47.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:47.614+0000 7fee0b400640 1 --2- 172.21.15.5:0/3176677273 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fedf00a2c00 0x7fedf00a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:47:47.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:47.615+0000 7fee0b400640 1 --2- 172.21.15.5:0/3176677273 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fedf00a2c00 0x7fedf00a50c0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fedfc009bd0 tx=0x7fedfc005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:47:47.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:47.618+0000 7fee08c00640 1 -- 172.21.15.5:0/3176677273 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fee0409b540 con 0x7fee0c109140 2024-09-10T09:47:47.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:47.818+0000 7fbdaf200640 1 -- 172.21.15.5:0/476806327 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbdb01019c0 msgr2=0x7fbdb0101e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:47:47.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:47.818+0000 7fbdaf200640 1 --2- 172.21.15.5:0/476806327 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbdb01019c0 0x7fbdb0101e20 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fbd9c009900 tx=0x7fbd9c02a410 comp rx=0 tx=0).stop 2024-09-10T09:47:47.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:47.818+0000 7fbdaf200640 1 -- 172.21.15.5:0/476806327 shutdown_connections 2024-09-10T09:47:47.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:47.818+0000 7fbdaf200640 1 --2- 172.21.15.5:0/476806327 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbdb0102360 0x7fbdb010e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:47.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:47.818+0000 7fbdaf200640 1 --2- 172.21.15.5:0/476806327 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbdb01019c0 0x7fbdb0101e20 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:47.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:47.818+0000 7fbdaf200640 1 --2- 172.21.15.5:0/476806327 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbdb01079c0 0x7fbdb0107da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:47.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:47.818+0000 7fbdaf200640 1 -- 172.21.15.5:0/476806327 >> 172.21.15.5:0/476806327 conn(0x7fbdb00fd740 msgr2=0x7fbdb00ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:47:47.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:47.819+0000 7fbdaf200640 1 -- 172.21.15.5:0/476806327 shutdown_connections 2024-09-10T09:47:47.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:47.819+0000 7fbdaf200640 1 -- 172.21.15.5:0/476806327 wait complete. 2024-09-10T09:47:47.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:47.819+0000 7fbdaf200640 1 Processor -- start 2024-09-10T09:47:47.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:47.820+0000 7fbdaf200640 1 -- start start 2024-09-10T09:47:47.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:47.820+0000 7fbdaf200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbdb01019c0 0x7fbdb00744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:47:47.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:47.820+0000 7fbdaf200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbdb0102360 0x7fbdb0074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:47:47.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:47.820+0000 7fbdaf200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbdb01079c0 0x7fbdb0074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:47:47.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:47.820+0000 7fbdaf200640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fbdb0111dc0 con 0x7fbdb01019c0 2024-09-10T09:47:47.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:47.820+0000 7fbdaf200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fbdb0111c40 con 0x7fbdb0102360 2024-09-10T09:47:47.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:47.820+0000 7fbdaf200640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fbdb0111f40 con 0x7fbdb01079c0 2024-09-10T09:47:47.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:47.820+0000 7fbdade00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbdb01019c0 0x7fbdb00744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:47:47.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:47.820+0000 7fbdad400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbdb0102360 0x7fbdb0074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:47:47.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:47.820+0000 7fbdad400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbdb0102360 0x7fbdb0074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:38702/0 (socket says 172.21.15.5:38702) 2024-09-10T09:47:47.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:47.820+0000 7fbdad400640 1 -- 172.21.15.5:0/3397018592 learned_addr learned my addr 172.21.15.5:0/3397018592 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:47:47.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:47.820+0000 7fbdae800640 1 --2- 172.21.15.5:0/3397018592 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbdb01079c0 0x7fbdb0074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:47:47.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:47.821+0000 7fbdade00640 1 -- 172.21.15.5:0/3397018592 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbdb01079c0 msgr2=0x7fbdb0074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:47:47.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:47.821+0000 7fbdade00640 1 --2- 172.21.15.5:0/3397018592 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbdb01079c0 0x7fbdb0074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:47.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:47.821+0000 7fbdade00640 1 -- 172.21.15.5:0/3397018592 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbdb0102360 msgr2=0x7fbdb0074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:47:47.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:47.821+0000 7fbdade00640 1 --2- 172.21.15.5:0/3397018592 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbdb0102360 0x7fbdb0074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:47.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:47.821+0000 7fbdade00640 1 -- 172.21.15.5:0/3397018592 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbdb01a6780 con 0x7fbdb01019c0 2024-09-10T09:47:47.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:47.821+0000 7fbdade00640 1 --2- 172.21.15.5:0/3397018592 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbdb01019c0 0x7fbdb00744f0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fbda000e9f0 tx=0x7fbda000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:47:47.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:47.821+0000 7fbda6a00640 1 -- 172.21.15.5:0/3397018592 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbda000ccc0 con 0x7fbdb01019c0 2024-09-10T09:47:47.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:47.822+0000 7fbda6a00640 1 -- 172.21.15.5:0/3397018592 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbda0005020 con 0x7fbdb01019c0 2024-09-10T09:47:47.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:47.822+0000 7fbdaf200640 1 -- 172.21.15.5:0/3397018592 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbdb01a6a70 con 0x7fbdb01019c0 2024-09-10T09:47:47.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:47.822+0000 7fbda6a00640 1 -- 172.21.15.5:0/3397018592 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbda00049e0 con 0x7fbdb01019c0 2024-09-10T09:47:47.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:47.822+0000 7fbdaf200640 1 -- 172.21.15.5:0/3397018592 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbdb01a6fb0 con 0x7fbdb01019c0 2024-09-10T09:47:47.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:47.825+0000 7fbdb7608740 1 Processor -- start 2024-09-10T09:47:47.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:47.825+0000 7fbda6a00640 1 -- 172.21.15.5:0/3397018592 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7fbda0005560 con 0x7fbdb01019c0 2024-09-10T09:47:47.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:47.825+0000 7fbdb7608740 1 -- start start 2024-09-10T09:47:47.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:47.826+0000 7fbdb7608740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a42337de40 0x55a42337e220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:47:47.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:47.826+0000 7fbdb7608740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a42337e760 0x55a423388ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:47:47.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:47.826+0000 7fbdb7608740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a423389020 0x55a42338b410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:47:47.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:47.826+0000 7fbdb7608740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55a4232e43e0 con 0x55a423389020 2024-09-10T09:47:47.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:47.826+0000 7fbdb7608740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55a42328bd80 con 0x55a42337de40 2024-09-10T09:47:47.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:47.826+0000 7fbdb7608740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55a4232e03a0 con 0x55a42337e760 2024-09-10T09:47:47.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:47.826+0000 7fbdae800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a42337de40 0x55a42337e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:47:47.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:47.826+0000 7fbdae800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a42337de40 0x55a42337e220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:38712/0 (socket says 172.21.15.5:38712) 2024-09-10T09:47:47.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:47.826+0000 7fbdae800640 1 -- 172.21.15.5:0/676411782 learned_addr learned my addr 172.21.15.5:0/676411782 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:47:47.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:47.826+0000 7fbdade00640 1 --2- 172.21.15.5:0/676411782 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a42337e760 0x55a423388ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:47:47.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:47.826+0000 7fbdad400640 1 --2- 172.21.15.5:0/676411782 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a423389020 0x55a42338b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:47:47.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:47.827+0000 7fbdade00640 1 -- 172.21.15.5:0/676411782 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a42337de40 msgr2=0x55a42337e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:47:47.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:47.827+0000 7fbdade00640 1 --2- 172.21.15.5:0/676411782 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a42337de40 0x55a42337e220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:47.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:47.827+0000 7fbdade00640 1 -- 172.21.15.5:0/676411782 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a423389020 msgr2=0x55a42338b410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:47:47.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:47.827+0000 7fbdade00640 1 --2- 172.21.15.5:0/676411782 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a423389020 0x55a42338b410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:47.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:47.827+0000 7fbdade00640 1 -- 172.21.15.5:0/676411782 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a4232aaae0 con 0x55a42337e760 2024-09-10T09:47:47.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:47.827+0000 7fbdae800640 1 --2- 172.21.15.5:0/676411782 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a42337de40 0x55a42337e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:47:47.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:47.827+0000 7fbda6a00640 1 --2- 172.21.15.5:0/3397018592 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fbd940a2c00 0x7fbd940a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:47:47.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:47.827+0000 7fbda6a00640 1 -- 172.21.15.5:0/3397018592 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fbda00cf310 con 0x7fbdb01019c0 2024-09-10T09:47:47.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:47.827+0000 7fbdade00640 1 --2- 172.21.15.5:0/676411782 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a42337e760 0x55a423388ae0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fbda00cc960 tx=0x7fbda00d7540 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7400e5727dada9d5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:47:47.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:47.827+0000 7fbd9b000640 1 -- 172.21.15.5:0/676411782 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbda001c070 con 0x55a42337e760 2024-09-10T09:47:47.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:47.827+0000 7fbdae800640 1 --2- 172.21.15.5:0/3397018592 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fbd940a2c00 0x7fbd940a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:47:47.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:47.827+0000 7fbd9b000640 1 -- 172.21.15.5:0/676411782 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbda00ab050 con 0x55a42337e760 2024-09-10T09:47:47.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:47.827+0000 7fbd9b000640 1 -- 172.21.15.5:0/676411782 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbda009b4f0 con 0x55a42337e760 2024-09-10T09:47:47.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:47.828+0000 7fbdb7608740 1 -- 172.21.15.5:0/676411782 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a42337e760 msgr2=0x55a423388ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:47:47.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:47.828+0000 7fbdae800640 1 --2- 172.21.15.5:0/3397018592 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fbd940a2c00 0x7fbd940a50c0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fbda8011890 tx=0x7fbda8009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:47:47.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:47.828+0000 7fbdb7608740 1 --2- 172.21.15.5:0/676411782 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a42337e760 0x55a423388ae0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fbda00cc960 tx=0x7fbda00d7540 comp rx=0 tx=0).stop 2024-09-10T09:47:47.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:47.828+0000 7fbdb7608740 1 -- 172.21.15.5:0/676411782 shutdown_connections 2024-09-10T09:47:47.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:47.828+0000 7fbdb7608740 1 --2- 172.21.15.5:0/676411782 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a423389020 0x55a42338b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:47.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:47.828+0000 7fbdb7608740 1 --2- 172.21.15.5:0/676411782 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a42337e760 0x55a423388ae0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:47.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:47.828+0000 7fbdb7608740 1 --2- 172.21.15.5:0/676411782 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a42337de40 0x55a42337e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:47.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:47.828+0000 7fbdb7608740 1 -- 172.21.15.5:0/676411782 >> 172.21.15.5:0/676411782 conn(0x55a42337a3d0 msgr2=0x55a42337a7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:47:47.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:47.828+0000 7fbdb7608740 1 -- 172.21.15.5:0/676411782 shutdown_connections 2024-09-10T09:47:47.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:47.828+0000 7fbdb7608740 1 -- 172.21.15.5:0/676411782 wait complete. 2024-09-10T09:47:47.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:47.829+0000 7fbdb7608740 1 -- start start 2024-09-10T09:47:47.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:47.829+0000 7fbdb7608740 1 Processor -- start 2024-09-10T09:47:47.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:47.830+0000 7fbdb7608740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a42337de40 0x55a423385e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:47:47.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:47.830+0000 7fbdb7608740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a42337e760 0x55a4233863b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:47:47.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:47.830+0000 7fbdb7608740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a423389020 0x55a4233868f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:47:47.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:47.830+0000 7fbdb7608740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55a42338bbf0 con 0x55a423389020 2024-09-10T09:47:47.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:47.830+0000 7fbdb7608740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55a42329f0f0 con 0x55a42337de40 2024-09-10T09:47:47.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:47.830+0000 7fbdb7608740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55a42338bd70 con 0x55a42337e760 2024-09-10T09:47:47.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:47.830+0000 7fbdb7608740 10 client.0 _ll_register_callbacks cb 0x55a423286bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:47:47.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:47.830+0000 7fbdade00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a42337de40 0x55a423385e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:47:47.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:47.830+0000 7fbdade00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a42337de40 0x55a423385e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:38718/0 (socket says 172.21.15.5:38718) 2024-09-10T09:47:47.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:47.830+0000 7fbdade00640 1 -- 172.21.15.5:0/4168254263 learned_addr learned my addr 172.21.15.5:0/4168254263 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:47:47.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:47.830+0000 7fbdae800640 1 --2- 172.21.15.5:0/4168254263 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a423389020 0x55a4233868f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:47:47.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:47.830+0000 7fbdad400640 1 --2- 172.21.15.5:0/4168254263 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a42337e760 0x55a4233863b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:47:47.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:47.830+0000 7fbdade00640 1 -- 172.21.15.5:0/4168254263 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a42337e760 msgr2=0x55a4233863b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:47:47.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:47.830+0000 7fbdade00640 1 --2- 172.21.15.5:0/4168254263 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a42337e760 0x55a4233863b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:47.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:47.830+0000 7fbdade00640 1 -- 172.21.15.5:0/4168254263 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a423389020 msgr2=0x55a4233868f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:47:47.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:47.830+0000 7fbdade00640 1 --2- 172.21.15.5:0/4168254263 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a423389020 0x55a4233868f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:47.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:47.830+0000 7fbdade00640 1 -- 172.21.15.5:0/4168254263 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a4230553c0 con 0x55a42337de40 2024-09-10T09:47:47.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:47.831+0000 7fbdade00640 1 --2- 172.21.15.5:0/4168254263 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a42337de40 0x55a423385e70 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fbda001a040 tx=0x7fbda000c7d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2a3d4dea1497f244 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:47:47.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:47.831+0000 7fbd8f400640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-09-10T09:47:47.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:47.831+0000 7fbd8f400640 1 -- 172.21.15.5:0/4168254263 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbda001c050 con 0x55a42337de40 2024-09-10T09:47:47.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:47.831+0000 7fbd8f400640 1 -- 172.21.15.5:0/4168254263 <== mon.2 v2:172.21.15.102:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fbda00d7d40 con 0x55a42337de40 2024-09-10T09:47:47.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:47.831+0000 7fbdb7608740 1 -- 172.21.15.5:0/4168254263 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbda00ab050 con 0x55a42337de40 2024-09-10T09:47:47.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:47.831+0000 7fbd8f400640 1 -- 172.21.15.5:0/4168254263 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fbda00b4050 con 0x55a42337de40 2024-09-10T09:47:47.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:47.831+0000 7fbdb7608740 10 client.27284 fetch_fsmap learned FSMap version 35 2024-09-10T09:47:47.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:47.831+0000 7fbdb7608740 1 -- 172.21.15.5:0/4168254263 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55a42338bbf0 con 0x55a42337de40 2024-09-10T09:47:47.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:47.831+0000 7fbd8f400640 1 -- 172.21.15.5:0/4168254263 <== mon.2 v2:172.21.15.102:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbda00ab050 con 0x55a42337de40 2024-09-10T09:47:47.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:47.832+0000 7fbd8f400640 1 -- 172.21.15.5:0/4168254263 <== mon.2 v2:172.21.15.102:3300/0 5 ==== fsmap(e 35) ==== 8374+0+0 (secure 0 0 0) 0x7fbda0004ca0 con 0x55a42337de40 2024-09-10T09:47:47.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:47.832+0000 7fbdb7608740 10 client.27284 fetch_fsmap finished waiting for FSMap version 35 2024-09-10T09:47:47.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:47.832+0000 7fbdb7608740 10 client.27284 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-10T09:47:47.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:47.832+0000 7fbdb7608740 20 client.27284 populate_metadata read hostname 'smithi005' 2024-09-10T09:47:47.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:47.832+0000 7fbdb7608740 1 --2- 172.21.15.5:0/4168254263 >> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] conn(0x55a423398d10 0x55a42339b100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:47:47.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:47.832+0000 7fbdb7608740 4 client.27284 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:47:47.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:47.832+0000 7fbdb7608740 1 -- 172.21.15.5:0/4168254263 --> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a4232f1950 con 0x55a423398d10 2024-09-10T09:47:47.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:47.832+0000 7fbdad400640 1 --2- 172.21.15.5:0/4168254263 >> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] conn(0x55a423398d10 0x55a42339b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:47:47.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:47.833+0000 7fbdad400640 1 --2- 172.21.15.5:0/4168254263 >> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] conn(0x55a423398d10 0x55a42339b100 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5fa9af10fba3b6a3 server_cookie=29551cae004801fe in_seq=0 out_seq=0 2024-09-10T09:47:47.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:47.833+0000 7fbd8f400640 10 client.27284 ms_handle_connect on v2:172.21.15.159:6838/2073739599 2024-09-10T09:47:47.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:47.834+0000 7fbd8f400640 1 -- 172.21.15.5:0/4168254263 <== mds.1 v2:172.21.15.159:6838/2073739599 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a4232f1950 con 0x55a423398d10 2024-09-10T09:47:47.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:47.834+0000 7fbd8f400640 10 client.27284 handle_command_reply: tid=0 2024-09-10T09:47:47.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:47.834+0000 7fbdb7608740 1 client.27284 shutdown 2024-09-10T09:47:47.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:47.834+0000 7fbdb7608740 1 -- 172.21.15.5:0/4168254263 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a42337de40 msgr2=0x55a423385e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:47:47.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:47.834+0000 7fbdb7608740 1 --2- 172.21.15.5:0/4168254263 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a42337de40 0x55a423385e70 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fbda001a040 tx=0x7fbda000c7d0 comp rx=0 tx=0).stop 2024-09-10T09:47:47.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:47.835+0000 7fbdb7608740 1 -- 172.21.15.5:0/4168254263 shutdown_connections 2024-09-10T09:47:47.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:47.835+0000 7fbdb7608740 1 --2- 172.21.15.5:0/4168254263 >> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] conn(0x55a423398d10 0x55a42339b100 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:47.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:47.835+0000 7fbdb7608740 1 --2- 172.21.15.5:0/4168254263 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a423389020 0x55a4233868f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:47.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:47.835+0000 7fbdb7608740 1 --2- 172.21.15.5:0/4168254263 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a42337e760 0x55a4233863b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:47.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:47.835+0000 7fbdb7608740 1 --2- 172.21.15.5:0/4168254263 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a42337de40 0x55a423385e70 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:47.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:47.835+0000 7fbd8f400640 0 client.27284 ms_handle_reset on v2:172.21.15.159:6838/2073739599 2024-09-10T09:47:47.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:47.835+0000 7fbdb7608740 1 -- 172.21.15.5:0/4168254263 >> 172.21.15.5:0/4168254263 conn(0x55a42337a3d0 msgr2=0x55a4233800b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:47:47.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:47.835+0000 7fbdb7608740 1 -- 172.21.15.5:0/4168254263 shutdown_connections 2024-09-10T09:47:47.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:47.835+0000 7fbdb7608740 1 -- 172.21.15.5:0/4168254263 wait complete. 2024-09-10T09:47:47.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:47.836+0000 7fbdb7608740 20 client.27284 trim_cache size 0 max 16384 2024-09-10T09:47:47.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:47.850+0000 7fbdb7608740 1 Processor -- start 2024-09-10T09:47:47.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:47.850+0000 7fbdb7608740 1 -- start start 2024-09-10T09:47:47.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:47.850+0000 7fbdb7608740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a42337de40 0x55a423386740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:47:47.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:47.850+0000 7fbdb7608740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a42337e760 0x55a4233a07c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:47:47.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:47.850+0000 7fbdb7608740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a423389020 0x55a4233a0d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:47:47.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:47.850+0000 7fbdb7608740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fbdb01b0f00 con 0x55a42337e760 2024-09-10T09:47:47.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:47.850+0000 7fbdb7608740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fbdb01b2a70 con 0x55a423389020 2024-09-10T09:47:47.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:47.850+0000 7fbdb7608740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fbdb01b15d0 con 0x55a42337de40 2024-09-10T09:47:47.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:47.851+0000 7fbdad400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a42337e760 0x55a4233a07c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:47:47.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:47.851+0000 7fbdad400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a42337e760 0x55a4233a07c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57292/0 (socket says 172.21.15.5:57292) 2024-09-10T09:47:47.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:47.851+0000 7fbdad400640 1 -- 172.21.15.5:0/2352994638 learned_addr learned my addr 172.21.15.5:0/2352994638 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:47:47.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:47.851+0000 7fbdae800640 1 --2- 172.21.15.5:0/2352994638 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a423389020 0x55a4233a0d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:47:47.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:47.851+0000 7fbdade00640 1 --2- 172.21.15.5:0/2352994638 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a42337de40 0x55a423386740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:47:47.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:47.851+0000 7fbdade00640 1 -- 172.21.15.5:0/2352994638 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a423389020 msgr2=0x55a4233a0d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:47:47.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:47.851+0000 7fbdade00640 1 --2- 172.21.15.5:0/2352994638 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a423389020 0x55a4233a0d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:47.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:47.851+0000 7fbdade00640 1 -- 172.21.15.5:0/2352994638 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a42337e760 msgr2=0x55a4233a07c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:47:47.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:47.851+0000 7fbdade00640 1 --2- 172.21.15.5:0/2352994638 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a42337e760 0x55a4233a07c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:47.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:47.851+0000 7fbdade00640 1 -- 172.21.15.5:0/2352994638 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a42337a200 con 0x55a42337de40 2024-09-10T09:47:47.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:47.851+0000 7fbdad400640 1 --2- 172.21.15.5:0/2352994638 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a42337e760 0x55a4233a07c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:47:47.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:47.851+0000 7fbdae800640 1 --2- 172.21.15.5:0/2352994638 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a423389020 0x55a4233a0d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:47:47.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:47.852+0000 7fbdade00640 1 --2- 172.21.15.5:0/2352994638 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a42337de40 0x55a423386740 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fbda009b6b0 tx=0x7fbda00d7200 comp rx=0 tx=0).ready entity=mon.1 client_cookie=78af3a138d60882c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:47:47.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:47.852+0000 7fbd9a600640 1 -- 172.21.15.5:0/2352994638 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbda001c070 con 0x55a42337de40 2024-09-10T09:47:47.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:47.852+0000 7fbd9a600640 1 -- 172.21.15.5:0/2352994638 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbda00d7a40 con 0x55a42337de40 2024-09-10T09:47:47.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:47.852+0000 7fbd9a600640 1 -- 172.21.15.5:0/2352994638 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbda009b080 con 0x55a42337de40 2024-09-10T09:47:47.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:47.853+0000 7fbdb7608740 1 -- 172.21.15.5:0/2352994638 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a42337de40 msgr2=0x55a423386740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:47:47.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:47.853+0000 7fbdb7608740 1 --2- 172.21.15.5:0/2352994638 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a42337de40 0x55a423386740 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fbda009b6b0 tx=0x7fbda00d7200 comp rx=0 tx=0).stop 2024-09-10T09:47:47.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:47.853+0000 7fee12400640 1 -- 172.21.15.5:0/3176677273 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fee0c10c380 con 0x7fee0c109140 2024-09-10T09:47:47.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:47.853+0000 7fbdb7608740 1 -- 172.21.15.5:0/2352994638 shutdown_connections 2024-09-10T09:47:47.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:47.853+0000 7fbdb7608740 1 --2- 172.21.15.5:0/2352994638 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a423389020 0x55a4233a0d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:47.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:47.853+0000 7fbdb7608740 1 --2- 172.21.15.5:0/2352994638 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a42337e760 0x55a4233a07c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:47.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:47.853+0000 7fbdb7608740 1 --2- 172.21.15.5:0/2352994638 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a42337de40 0x55a423386740 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:47.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:47.853+0000 7fbdb7608740 1 -- 172.21.15.5:0/2352994638 >> 172.21.15.5:0/2352994638 conn(0x55a42337a3d0 msgr2=0x55a423387ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:47:47.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:47.853+0000 7fbdb7608740 1 -- 172.21.15.5:0/2352994638 shutdown_connections 2024-09-10T09:47:47.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:47.853+0000 7fbdb7608740 1 -- 172.21.15.5:0/2352994638 wait complete. 2024-09-10T09:47:47.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:47.853+0000 7fbdb7608740 1 -- start start 2024-09-10T09:47:47.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:47.854+0000 7fbdb7608740 1 Processor -- start 2024-09-10T09:47:47.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:47.854+0000 7fee08c00640 1 -- 172.21.15.5:0/3176677273 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) ==== 94+0+11970 (secure 0 0 0) 0x7fee0409f840 con 0x7fee0c109140 2024-09-10T09:47:47.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:47.854+0000 7fbdb7608740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a42337de40 0x55a423390fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:47:47.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:47.854+0000 7fbdb7608740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a42337e760 0x55a423391510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:47:47.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:47.854+0000 7fbdb7608740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a423389020 0x55a42338bf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:47:47.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:47.854+0000 7fbdb7608740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fbdb01b2370 con 0x55a42337de40 2024-09-10T09:47:47.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:47.854+0000 7fbdb7608740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fbdb01b1ca0 con 0x55a42337e760 2024-09-10T09:47:47.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:47.854+0000 7fbdb7608740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fbdb01aeee0 con 0x55a423389020 2024-09-10T09:47:47.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:47.854+0000 7fbdb7608740 10 client.0 _ll_register_callbacks cb 0x7fbda8009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:47:47.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:47.854+0000 7fbdade00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a42337de40 0x55a423390fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:47:47.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:47.855+0000 7fbdade00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a42337de40 0x55a423390fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57302/0 (socket says 172.21.15.5:57302) 2024-09-10T09:47:47.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:47.855+0000 7fbdade00640 1 -- 172.21.15.5:0/649312313 learned_addr learned my addr 172.21.15.5:0/649312313 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:47:47.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:47.855+0000 7fbdae800640 1 --2- 172.21.15.5:0/649312313 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a423389020 0x55a42338bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:47:47.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:47.855+0000 7fbdad400640 1 --2- 172.21.15.5:0/649312313 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a42337e760 0x55a423391510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:47:47.858 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-10T09:47:47.858 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":35,"btime":"2024-09-10T09:46:17:834001+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14454,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/2970594311","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2970594311},{"type":"v1","addr":"172.21.15.5:6835","nonce":2970594311}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14514,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6839/3616502174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":3616502174},{"type":"v1","addr":"172.21.15.5:6839","nonce":3616502174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":35,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T09:35:29.626287+0000","modified":"2024-09-10T09:46:16.966068+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24542,"mds_1":24535,"mds_2":24511,"mds_3":24506,"mds_4":24443},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14520":{"gid":14520,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/2073599842","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":2073599842},{"type":"v1","addr":"172.21.15.5:6841","nonce":2073599842}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"k","rank":4,"incarnation":26,"state":"up:active","state_seq":24,"addr":"172.21.15.102:6833/2457007427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":2457007427},{"type":"v1","addr":"172.21.15.102:6833","nonce":2457007427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/356445159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":356445159},{"type":"v1","addr":"172.21.15.159:6833","nonce":356445159}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6837/2894314507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":2894314507},{"type":"v1","addr":"172.21.15.5:6837","nonce":2894314507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.102:6835/1647549111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":1647549111},{"type":"v1","addr":"172.21.15.102:6835","nonce":1647549111}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6835/952657242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":952657242},{"type":"v1","addr":"172.21.15.159:6835","nonce":952657242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":19,"addr":"172.21.15.102:6837/2689776764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":2689776764},{"type":"v1","addr":"172.21.15.102:6837","nonce":2689776764}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.159:6837/620627639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":620627639},{"type":"v1","addr":"172.21.15.159:6837","nonce":620627639}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24535":{"gid":24535,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":18,"addr":"172.21.15.159:6839/2073739599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":2073739599},{"type":"v1","addr":"172.21.15.159:6839","nonce":2073739599}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24542":{"gid":24542,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.102:6839/335409749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6838","nonce":335409749},{"type":"v1","addr":"172.21.15.102:6839","nonce":335409749}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24542,"qdb_cluster":[24443,24506,24511,24535,24542]},"id":1}]} 2024-09-10T09:47:47.859 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 35 2024-09-10T09:47:47.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:47.855+0000 7fbdae800640 1 -- 172.21.15.5:0/649312313 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a42337e760 msgr2=0x55a423391510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:47:47.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:47.855+0000 7fbdae800640 1 --2- 172.21.15.5:0/649312313 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a42337e760 0x55a423391510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:47.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:47.855+0000 7fbdae800640 1 -- 172.21.15.5:0/649312313 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a42337de40 msgr2=0x55a423390fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:47:47.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:47.855+0000 7fbdae800640 1 --2- 172.21.15.5:0/649312313 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a42337de40 0x55a423390fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:47.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:47.855+0000 7fbdae800640 1 -- 172.21.15.5:0/649312313 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a42338c4c0 con 0x55a423389020 2024-09-10T09:47:47.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:47.855+0000 7fbdad400640 1 --2- 172.21.15.5:0/649312313 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a42337e760 0x55a423391510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:47:47.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:47.855+0000 7fbdade00640 1 --2- 172.21.15.5:0/649312313 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a42337de40 0x55a423390fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:47:47.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:47.856+0000 7fbdae800640 1 --2- 172.21.15.5:0/649312313 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a423389020 0x55a42338bf80 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fbda8005fd0 tx=0x7fbda8004ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a0b2a3955d5b3a18 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:47:47.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:47.856+0000 7fbd8f400640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-09-10T09:47:47.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:47.856+0000 7fbd8f400640 1 -- 172.21.15.5:0/649312313 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbda8004040 con 0x55a423389020 2024-09-10T09:47:47.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:47.856+0000 7fbd8f400640 1 -- 172.21.15.5:0/649312313 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fbda80041e0 con 0x55a423389020 2024-09-10T09:47:47.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:47.856+0000 7fbdb7608740 1 -- 172.21.15.5:0/649312313 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbda00d7a40 con 0x55a423389020 2024-09-10T09:47:47.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:47.856+0000 7fbd8f400640 1 -- 172.21.15.5:0/649312313 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fbda8004b20 con 0x55a423389020 2024-09-10T09:47:47.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:47.856+0000 7fbdb7608740 10 client.27322 fetch_fsmap learned FSMap version 35 2024-09-10T09:47:47.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:47.856+0000 7fbdb7608740 1 -- 172.21.15.5:0/649312313 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55a42340c7c0 con 0x55a423389020 2024-09-10T09:47:47.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:47.857+0000 7fbd8f400640 1 -- 172.21.15.5:0/649312313 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbda00d7a40 con 0x55a423389020 2024-09-10T09:47:47.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:47.857+0000 7fbd8f400640 1 -- 172.21.15.5:0/649312313 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 35) ==== 8374+0+0 (secure 0 0 0) 0x7fbda80055b0 con 0x55a423389020 2024-09-10T09:47:47.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:47.857+0000 7fbdb7608740 10 client.27322 fetch_fsmap finished waiting for FSMap version 35 2024-09-10T09:47:47.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:47.857+0000 7fbdb7608740 10 client.27322 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-10T09:47:47.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:47.857+0000 7fbdb7608740 20 client.27322 populate_metadata read hostname 'smithi005' 2024-09-10T09:47:47.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:47.857+0000 7fbdb7608740 1 --2- 172.21.15.5:0/649312313 >> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] conn(0x55a42340cf90 0x55a42340f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:47:47.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:47.857+0000 7fbdb7608740 4 client.27322 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T09:47:47.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:47.857+0000 7fbdb7608740 1 -- 172.21.15.5:0/649312313 --> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a4232ebe60 con 0x55a42340cf90 2024-09-10T09:47:47.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:47.857+0000 7fee12400640 1 -- 172.21.15.5:0/3176677273 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fedf00a2c00 msgr2=0x7fedf00a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:47:47.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:47.857+0000 7fee12400640 1 --2- 172.21.15.5:0/3176677273 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fedf00a2c00 0x7fedf00a50c0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fedfc009bd0 tx=0x7fedfc005f90 comp rx=0 tx=0).stop 2024-09-10T09:47:47.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:47.857+0000 7fee12400640 1 -- 172.21.15.5:0/3176677273 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fee0c109140 msgr2=0x7fee0c19eff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:47:47.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:47.857+0000 7fee12400640 1 --2- 172.21.15.5:0/3176677273 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fee0c109140 0x7fee0c19eff0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fee04009fd0 tx=0x7fee0400b260 comp rx=0 tx=0).stop 2024-09-10T09:47:47.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:47.857+0000 7fee12400640 1 -- 172.21.15.5:0/3176677273 shutdown_connections 2024-09-10T09:47:47.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:47.857+0000 7fee12400640 1 --2- 172.21.15.5:0/3176677273 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fedf00a2c00 0x7fedf00a50c0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:47.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:47.857+0000 7fee12400640 1 --2- 172.21.15.5:0/3176677273 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fee0c109140 0x7fee0c19eff0 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:47.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:47.857+0000 7fee12400640 1 --2- 172.21.15.5:0/3176677273 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fee0c1049c0 0x7fee0c19ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:47.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:47.857+0000 7fee12400640 1 --2- 172.21.15.5:0/3176677273 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fee0c104010 0x7fee0c19a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:47.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:47.857+0000 7fee12400640 1 -- 172.21.15.5:0/3176677273 >> 172.21.15.5:0/3176677273 conn(0x7fee0c0ffa50 msgr2=0x7fee0c100df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:47:47.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:47.858+0000 7fbdade00640 1 --2- 172.21.15.5:0/649312313 >> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] conn(0x55a42340cf90 0x55a42340f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:47:47.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:47.858+0000 7fee12400640 1 -- 172.21.15.5:0/3176677273 shutdown_connections 2024-09-10T09:47:47.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:47.858+0000 7fee12400640 1 -- 172.21.15.5:0/3176677273 wait complete. 2024-09-10T09:47:47.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:47.858+0000 7fbdade00640 1 --2- 172.21.15.5:0/649312313 >> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] conn(0x55a42340cf90 0x55a42340f3b0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1f4ce861e725b416 server_cookie=b8bf09542901ba9f in_seq=0 out_seq=0 2024-09-10T09:47:47.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:47.858+0000 7fbd8f400640 10 client.27322 ms_handle_connect on v2:172.21.15.159:6838/2073739599 2024-09-10T09:47:47.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:47.859+0000 7fbd8f400640 1 -- 172.21.15.5:0/649312313 <== mds.1 v2:172.21.15.159:6838/2073739599 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a4232ebe60 con 0x55a42340cf90 2024-09-10T09:47:47.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:47.859+0000 7fbd8f400640 10 client.27322 handle_command_reply: tid=0 2024-09-10T09:47:47.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:47.859+0000 7fbdb7608740 1 client.27322 shutdown 2024-09-10T09:47:47.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:47.859+0000 7fbdb7608740 1 -- 172.21.15.5:0/649312313 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a423389020 msgr2=0x55a42338bf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:47:47.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:47.859+0000 7fbdb7608740 1 --2- 172.21.15.5:0/649312313 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a423389020 0x55a42338bf80 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fbda8005fd0 tx=0x7fbda8004ab0 comp rx=0 tx=0).stop 2024-09-10T09:47:47.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:47.859+0000 7fbdb7608740 1 -- 172.21.15.5:0/649312313 shutdown_connections 2024-09-10T09:47:47.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:47.859+0000 7fbdb7608740 1 --2- 172.21.15.5:0/649312313 >> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] conn(0x55a42340cf90 0x55a42340f3b0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:47.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:47.859+0000 7fbdb7608740 1 --2- 172.21.15.5:0/649312313 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a423389020 0x55a42338bf80 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:47.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:47.859+0000 7fbdb7608740 1 --2- 172.21.15.5:0/649312313 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a42337e760 0x55a423391510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:47.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:47.859+0000 7fbdb7608740 1 --2- 172.21.15.5:0/649312313 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a42337de40 0x55a423390fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:47.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:47.859+0000 7fbdb7608740 1 -- 172.21.15.5:0/649312313 >> 172.21.15.5:0/649312313 conn(0x55a42337a3d0 msgr2=0x55a4233876d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:47:47.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:47.859+0000 7fbd8f400640 0 client.27322 ms_handle_reset on v2:172.21.15.159:6838/2073739599 2024-09-10T09:47:47.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:47.860+0000 7fbdb7608740 1 -- 172.21.15.5:0/649312313 shutdown_connections 2024-09-10T09:47:47.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:47.860+0000 7fbdb7608740 1 -- 172.21.15.5:0/649312313 wait complete. 2024-09-10T09:47:47.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:47.860+0000 7fbdb7608740 20 client.27322 trim_cache size 0 max 16384 2024-09-10T09:47:47.866 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-10T09:47:47.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:47.861+0000 7fbdaf200640 1 -- 172.21.15.5:0/3397018592 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fbd940a2c00 msgr2=0x7fbd940a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:47:47.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:47.861+0000 7fbdaf200640 1 --2- 172.21.15.5:0/3397018592 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fbd940a2c00 0x7fbd940a50c0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fbda8011890 tx=0x7fbda8009500 comp rx=0 tx=0).stop 2024-09-10T09:47:47.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:47.861+0000 7fbdaf200640 1 -- 172.21.15.5:0/3397018592 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbdb01019c0 msgr2=0x7fbdb00744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:47:47.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:47.861+0000 7fbdaf200640 1 --2- 172.21.15.5:0/3397018592 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbdb01019c0 0x7fbdb00744f0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fbda000e9f0 tx=0x7fbda000eeb0 comp rx=0 tx=0).stop 2024-09-10T09:47:47.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:47.861+0000 7fbdaf200640 1 -- 172.21.15.5:0/3397018592 shutdown_connections 2024-09-10T09:47:47.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:47.861+0000 7fbdaf200640 1 --2- 172.21.15.5:0/3397018592 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fbd940a2c00 0x7fbd940a50c0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:47.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:47.861+0000 7fbdaf200640 1 --2- 172.21.15.5:0/3397018592 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbdb01079c0 0x7fbdb0074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:47.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:47.861+0000 7fbdaf200640 1 --2- 172.21.15.5:0/3397018592 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbdb0102360 0x7fbdb0074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:47.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:47.861+0000 7fbdaf200640 1 --2- 172.21.15.5:0/3397018592 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbdb01019c0 0x7fbdb00744f0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:47.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:47.861+0000 7fbdaf200640 1 -- 172.21.15.5:0/3397018592 >> 172.21.15.5:0/3397018592 conn(0x7fbdb00fd740 msgr2=0x7fbdb00fdf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:47:47.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:47.862+0000 7fbdaf200640 1 -- 172.21.15.5:0/3397018592 shutdown_connections 2024-09-10T09:47:47.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:47.862+0000 7fbdaf200640 1 -- 172.21.15.5:0/3397018592 wait complete. 2024-09-10T09:47:48.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:47:47 smithi005 ceph-mon[30350]: from='client.? 172.21.15.5:0/3176677273' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:47:48.087 INFO:tasks.workunit.client.0.smithi005.stdout: 1 555754 6.01 MB/sec execute 329 sec latency 23178.522 ms 2024-09-10T09:47:48.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:47:47 smithi102 ceph-mon[35442]: from='client.? 172.21.15.5:0/3176677273' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:47:48.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:47:47 smithi159 ceph-mon[35513]: from='client.? 172.21.15.5:0/3176677273' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:47:48.732 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24542 quiesce db --set-id 82aad4ea --release --await' 2024-09-10T09:47:48.732 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24542 quiesce db --set-id 82aad4ea --release --await 2024-09-10T09:47:49.049 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:47:48 smithi005 ceph-mon[30350]: pgmap v677: 129 pgs: 129 active+clean; 212 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 1.2 MiB/s wr, 61 op/s 2024-09-10T09:47:49.087 INFO:tasks.workunit.client.0.smithi005.stdout: 1 555754 5.99 MB/sec execute 330 sec latency 24178.575 ms 2024-09-10T09:47:49.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:47:48 smithi102 ceph-mon[35442]: pgmap v677: 129 pgs: 129 active+clean; 212 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 1.2 MiB/s wr, 61 op/s 2024-09-10T09:47:49.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:47:48 smithi159 ceph-mon[35513]: pgmap v677: 129 pgs: 129 active+clean; 212 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 1.2 MiB/s wr, 61 op/s 2024-09-10T09:47:49.453 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-10T09:47:49.547 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:47:50.098 INFO:tasks.workunit.client.0.smithi005.stdout: 1 555754 5.97 MB/sec execute 331 sec latency 25189.246 ms 2024-09-10T09:47:51.098 INFO:tasks.workunit.client.0.smithi005.stdout: 1 555754 5.95 MB/sec execute 332 sec latency 26189.302 ms 2024-09-10T09:47:51.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:51.007+0000 7f1a94600640 1 -- 172.21.15.5:0/1566048327 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1a8c106f20 msgr2=0x7f1a8c10eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:47:51.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:51.007+0000 7f1a94600640 1 --2- 172.21.15.5:0/1566048327 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1a8c106f20 0x7f1a8c10eaa0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f1a84009ca0 tx=0x7f1a8402a8c0 comp rx=0 tx=0).stop 2024-09-10T09:47:51.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:51.007+0000 7f1a94600640 1 -- 172.21.15.5:0/1566048327 shutdown_connections 2024-09-10T09:47:51.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:51.007+0000 7f1a94600640 1 --2- 172.21.15.5:0/1566048327 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1a8c106f20 0x7f1a8c10eaa0 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:51.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:51.007+0000 7f1a94600640 1 --2- 172.21.15.5:0/1566048327 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1a8c1027a0 0x7f1a8c1067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:51.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:51.007+0000 7f1a94600640 1 --2- 172.21.15.5:0/1566048327 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1a8c101df0 0x7f1a8c1021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:51.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:51.007+0000 7f1a94600640 1 -- 172.21.15.5:0/1566048327 >> 172.21.15.5:0/1566048327 conn(0x7f1a8c0fd8b0 msgr2=0x7f1a8c0ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:47:51.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:51.007+0000 7f1a94600640 1 -- 172.21.15.5:0/1566048327 shutdown_connections 2024-09-10T09:47:51.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:51.008+0000 7f1a94600640 1 -- 172.21.15.5:0/1566048327 wait complete. 2024-09-10T09:47:51.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:51.009+0000 7f1a94600640 1 Processor -- start 2024-09-10T09:47:51.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:51.009+0000 7f1a94600640 1 -- start start 2024-09-10T09:47:51.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:51.010+0000 7f1a94600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1a8c101df0 0x7f1a8c1983b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:47:51.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:51.010+0000 7f1a94600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1a8c1027a0 0x7f1a8c1988f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:47:51.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:51.010+0000 7f1a94600640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1a8c106f20 0x7f1a8c19cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:47:51.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:51.010+0000 7f1a94600640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1a8c111fd0 con 0x7f1a8c101df0 2024-09-10T09:47:51.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:51.010+0000 7f1a94600640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f1a8c111e50 con 0x7f1a8c1027a0 2024-09-10T09:47:51.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:51.010+0000 7f1a94600640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f1a8c112150 con 0x7f1a8c106f20 2024-09-10T09:47:51.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:51.010+0000 7f1a91c00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1a8c101df0 0x7f1a8c1983b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:47:51.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:51.010+0000 7f1a92600640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1a8c106f20 0x7f1a8c19cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:47:51.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:51.010+0000 7f1a91c00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1a8c101df0 0x7f1a8c1983b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49092/0 (socket says 172.21.15.5:49092) 2024-09-10T09:47:51.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:51.010+0000 7f1a91c00640 1 -- 172.21.15.5:0/2972480757 learned_addr learned my addr 172.21.15.5:0/2972480757 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:47:51.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:51.010+0000 7f1a91200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1a8c1027a0 0x7f1a8c1988f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:47:51.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:51.011+0000 7f1a91200640 1 -- 172.21.15.5:0/2972480757 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1a8c106f20 msgr2=0x7f1a8c19cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:47:51.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:51.011+0000 7f1a91200640 1 --2- 172.21.15.5:0/2972480757 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1a8c106f20 0x7f1a8c19cc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:51.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:51.011+0000 7f1a91200640 1 -- 172.21.15.5:0/2972480757 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1a8c101df0 msgr2=0x7f1a8c1983b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:47:51.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:51.011+0000 7f1a91200640 1 --2- 172.21.15.5:0/2972480757 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1a8c101df0 0x7f1a8c1983b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:51.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:51.011+0000 7f1a91200640 1 -- 172.21.15.5:0/2972480757 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a8c19d360 con 0x7f1a8c1027a0 2024-09-10T09:47:51.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:51.011+0000 7f1a91200640 1 --2- 172.21.15.5:0/2972480757 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1a8c1027a0 0x7f1a8c1988f0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f1a7800e9f0 tx=0x7f1a7800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:47:51.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:51.011+0000 7f1a8a000640 1 -- 172.21.15.5:0/2972480757 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a7800cd80 con 0x7f1a8c1027a0 2024-09-10T09:47:51.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:51.011+0000 7f1a8a000640 1 -- 172.21.15.5:0/2972480757 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1a780035f0 con 0x7f1a8c1027a0 2024-09-10T09:47:51.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:51.011+0000 7f1a94600640 1 -- 172.21.15.5:0/2972480757 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a8c19d650 con 0x7f1a8c1027a0 2024-09-10T09:47:51.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:51.011+0000 7f1a8a000640 1 -- 172.21.15.5:0/2972480757 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a78010690 con 0x7f1a8c1027a0 2024-09-10T09:47:51.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:51.011+0000 7f1a94600640 1 -- 172.21.15.5:0/2972480757 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a8c1a4f30 con 0x7f1a8c1027a0 2024-09-10T09:47:51.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:51.014+0000 7f1a96fdc740 1 Processor -- start 2024-09-10T09:47:51.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:51.015+0000 7f1a96fdc740 1 -- start start 2024-09-10T09:47:51.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:51.015+0000 7f1a96fdc740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56141f96b310 0x56141f96b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:47:51.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:51.015+0000 7f1a96fdc740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56141f96bc30 0x56141f975fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:47:51.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:51.015+0000 7f1a96fdc740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56141f9764f0 0x56141f9788e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:47:51.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:51.015+0000 7f1a96fdc740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x56141f8d1650 con 0x56141f9764f0 2024-09-10T09:47:51.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:51.015+0000 7f1a96fdc740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x56141f878f10 con 0x56141f96b310 2024-09-10T09:47:51.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:51.015+0000 7f1a96fdc740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x56141f8cd610 con 0x56141f96bc30 2024-09-10T09:47:51.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:51.015+0000 7f1a8a000640 1 -- 172.21.15.5:0/2972480757 <== mon.2 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f1a78003b30 con 0x7f1a8c1027a0 2024-09-10T09:47:51.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:51.015+0000 7f1a92600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56141f96b310 0x56141f96b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:47:51.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:51.015+0000 7f1a92600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56141f96b310 0x56141f96b6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:41522/0 (socket says 172.21.15.5:41522) 2024-09-10T09:47:51.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:51.015+0000 7f1a92600640 1 -- 172.21.15.5:0/549439445 learned_addr learned my addr 172.21.15.5:0/549439445 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:47:51.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:51.016+0000 7f1a91c00640 1 --2- 172.21.15.5:0/549439445 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56141f96bc30 0x56141f975fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:47:51.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:51.016+0000 7f1a91200640 1 --2- 172.21.15.5:0/549439445 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56141f9764f0 0x56141f9788e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:47:51.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:51.016+0000 7f1a92600640 1 -- 172.21.15.5:0/549439445 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56141f96bc30 msgr2=0x56141f975fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:47:51.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:51.016+0000 7f1a92600640 1 --2- 172.21.15.5:0/549439445 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56141f96bc30 0x56141f975fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:51.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:51.016+0000 7f1a92600640 1 -- 172.21.15.5:0/549439445 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56141f9764f0 msgr2=0x56141f9788e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:47:51.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:51.016+0000 7f1a92600640 1 --2- 172.21.15.5:0/549439445 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56141f9764f0 0x56141f9788e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:51.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:51.016+0000 7f1a92600640 1 -- 172.21.15.5:0/549439445 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56141f5e0580 con 0x56141f96b310 2024-09-10T09:47:51.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:51.016+0000 7f1a91200640 1 --2- 172.21.15.5:0/549439445 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56141f9764f0 0x56141f9788e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T09:47:51.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:51.016+0000 7f1a92600640 1 --2- 172.21.15.5:0/549439445 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56141f96b310 0x56141f96b6f0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f1a84009ca0 tx=0x7f1a840093e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7395d2cf9430dd49 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:47:51.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:51.017+0000 7f1a7ea00640 1 -- 172.21.15.5:0/549439445 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a84031db0 con 0x56141f96b310 2024-09-10T09:47:51.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:51.017+0000 7f1a7ea00640 1 -- 172.21.15.5:0/549439445 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1a84036030 con 0x56141f96b310 2024-09-10T09:47:51.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:51.017+0000 7f1a7ea00640 1 -- 172.21.15.5:0/549439445 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a84004330 con 0x56141f96b310 2024-09-10T09:47:51.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:51.017+0000 7f1a96fdc740 1 -- 172.21.15.5:0/549439445 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56141f96b310 msgr2=0x56141f96b6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:47:51.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:51.017+0000 7f1a96fdc740 1 --2- 172.21.15.5:0/549439445 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56141f96b310 0x56141f96b6f0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f1a84009ca0 tx=0x7f1a840093e0 comp rx=0 tx=0).stop 2024-09-10T09:47:51.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:51.017+0000 7f1a96fdc740 1 -- 172.21.15.5:0/549439445 shutdown_connections 2024-09-10T09:47:51.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:51.017+0000 7f1a96fdc740 1 --2- 172.21.15.5:0/549439445 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56141f9764f0 0x56141f9788e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:51.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:51.017+0000 7f1a96fdc740 1 --2- 172.21.15.5:0/549439445 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56141f96bc30 0x56141f975fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:51.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:51.017+0000 7f1a96fdc740 1 --2- 172.21.15.5:0/549439445 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56141f96b310 0x56141f96b6f0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:51.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:51.017+0000 7f1a96fdc740 1 -- 172.21.15.5:0/549439445 >> 172.21.15.5:0/549439445 conn(0x56141f9678a0 msgr2=0x56141f967c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:47:51.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:51.018+0000 7f1a8a000640 1 --2- 172.21.15.5:0/2972480757 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f1a740a2b80 0x7f1a740a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:47:51.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:51.018+0000 7f1a96fdc740 1 -- 172.21.15.5:0/549439445 shutdown_connections 2024-09-10T09:47:51.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:51.018+0000 7f1a8a000640 1 -- 172.21.15.5:0/2972480757 <== mon.2 v2:172.21.15.102:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f1a780cf710 con 0x7f1a8c1027a0 2024-09-10T09:47:51.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:51.018+0000 7f1a96fdc740 1 -- 172.21.15.5:0/549439445 wait complete. 2024-09-10T09:47:51.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:51.018+0000 7f1a91c00640 1 --2- 172.21.15.5:0/2972480757 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f1a740a2b80 0x7f1a740a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:47:51.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:51.018+0000 7f1a96fdc740 1 -- start start 2024-09-10T09:47:51.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:51.018+0000 7f1a96fdc740 1 Processor -- start 2024-09-10T09:47:51.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:51.019+0000 7f1a91c00640 1 --2- 172.21.15.5:0/2972480757 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f1a740a2b80 0x7f1a740a5040 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x56141f968700 tx=0x7f1a80005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:47:51.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:51.019+0000 7f1a96fdc740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56141f96b310 0x56141f9733c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:47:51.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:51.019+0000 7f1a96fdc740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56141f96bc30 0x56141f973900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:47:51.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:51.019+0000 7f1a96fdc740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56141f9764f0 0x56141f973e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:47:51.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:51.019+0000 7f1a96fdc740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x56141f978f40 con 0x56141f96bc30 2024-09-10T09:47:51.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:51.019+0000 7f1a96fdc740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x56141f88c350 con 0x56141f9764f0 2024-09-10T09:47:51.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:51.019+0000 7f1a96fdc740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x56141f9790c0 con 0x56141f96b310 2024-09-10T09:47:51.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:51.019+0000 7f1a96fdc740 10 client.0 _ll_register_callbacks cb 0x56141f8af290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:47:51.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:51.020+0000 7f1a91c00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56141f96b310 0x56141f9733c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:47:51.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:51.020+0000 7f1a91200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56141f96bc30 0x56141f973900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:47:51.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:51.020+0000 7f1a92600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56141f9764f0 0x56141f973e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:47:51.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:51.020+0000 7f1a91c00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56141f96b310 0x56141f9733c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:57694/0 (socket says 172.21.15.5:57694) 2024-09-10T09:47:51.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:51.020+0000 7f1a91c00640 1 -- 172.21.15.5:0/88738480 learned_addr learned my addr 172.21.15.5:0/88738480 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:47:51.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:51.020+0000 7f1a91c00640 1 -- 172.21.15.5:0/88738480 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56141f9764f0 msgr2=0x56141f973e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:47:51.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:51.020+0000 7f1a91c00640 1 --2- 172.21.15.5:0/88738480 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56141f9764f0 0x56141f973e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:51.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:51.020+0000 7f1a91c00640 1 -- 172.21.15.5:0/88738480 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56141f96bc30 msgr2=0x56141f973900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:47:51.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:51.020+0000 7f1a91c00640 1 --2- 172.21.15.5:0/88738480 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56141f96bc30 0x56141f973900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:51.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:51.020+0000 7f1a91c00640 1 -- 172.21.15.5:0/88738480 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56141f6aa960 con 0x56141f96b310 2024-09-10T09:47:51.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:51.020+0000 7f1a92600640 1 --2- 172.21.15.5:0/88738480 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56141f9764f0 0x56141f973e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:47:51.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:51.020+0000 7f1a91c00640 1 --2- 172.21.15.5:0/88738480 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56141f96b310 0x56141f9733c0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f1a80004620 tx=0x7f1a80004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1886ee0f22d319cb server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:47:51.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:51.020+0000 7f1a73400640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-09-10T09:47:51.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:51.021+0000 7f1a73400640 1 -- 172.21.15.5:0/88738480 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a80029440 con 0x56141f96b310 2024-09-10T09:47:51.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:51.021+0000 7f1a96fdc740 1 -- 172.21.15.5:0/88738480 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1a84036030 con 0x56141f96b310 2024-09-10T09:47:51.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:51.021+0000 7f1a73400640 1 -- 172.21.15.5:0/88738480 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f1a8001f420 con 0x56141f96b310 2024-09-10T09:47:51.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:51.021+0000 7f1a73400640 1 -- 172.21.15.5:0/88738480 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f1a8001f620 con 0x56141f96b310 2024-09-10T09:47:51.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:51.021+0000 7f1a96fdc740 10 client.27334 fetch_fsmap learned FSMap version 35 2024-09-10T09:47:51.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:51.021+0000 7f1a96fdc740 1 -- 172.21.15.5:0/88738480 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x56141f97baa0 con 0x56141f96b310 2024-09-10T09:47:51.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:51.022+0000 7f1a73400640 1 -- 172.21.15.5:0/88738480 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1a84036030 con 0x56141f96b310 2024-09-10T09:47:51.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:51.022+0000 7f1a73400640 1 -- 172.21.15.5:0/88738480 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 35) ==== 8374+0+0 (secure 0 0 0) 0x7f1a80027070 con 0x56141f96b310 2024-09-10T09:47:51.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:51.022+0000 7f1a96fdc740 10 client.27334 fetch_fsmap finished waiting for FSMap version 35 2024-09-10T09:47:51.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:51.022+0000 7f1a96fdc740 10 client.27334 resolve_mds: validated gid 24542 aka daemon mds.b 2024-09-10T09:47:51.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:51.022+0000 7f1a96fdc740 20 client.27334 populate_metadata read hostname 'smithi005' 2024-09-10T09:47:51.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:51.022+0000 7f1a96fdc740 1 --2- 172.21.15.5:0/88738480 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x56141f97cfb0 0x56141f97f440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:47:51.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:51.022+0000 7f1a96fdc740 4 client.27334 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:47:51.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:51.022+0000 7f1a96fdc740 1 -- 172.21.15.5:0/88738480 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56141f8debc0 con 0x56141f97cfb0 2024-09-10T09:47:51.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:51.023+0000 7f1a91200640 1 --2- 172.21.15.5:0/88738480 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x56141f97cfb0 0x56141f97f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:47:51.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:51.023+0000 7f1a91200640 1 --2- 172.21.15.5:0/88738480 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x56141f97cfb0 0x56141f97f440 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7a57a7c338314ec0 server_cookie=a653ccc876068342 in_seq=0 out_seq=0 2024-09-10T09:47:51.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:51.023+0000 7f1a73400640 10 client.27334 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T09:47:51.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:51.024+0000 7f1a73400640 1 -- 172.21.15.5:0/88738480 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56141f8debc0 con 0x56141f97cfb0 2024-09-10T09:47:51.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:51.024+0000 7f1a73400640 10 client.27334 handle_command_reply: tid=0 2024-09-10T09:47:51.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:51.024+0000 7f1a96fdc740 1 client.27334 shutdown 2024-09-10T09:47:51.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:51.024+0000 7f1a96fdc740 1 -- 172.21.15.5:0/88738480 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56141f96b310 msgr2=0x56141f9733c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:47:51.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:51.024+0000 7f1a96fdc740 1 --2- 172.21.15.5:0/88738480 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56141f96b310 0x56141f9733c0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f1a80004620 tx=0x7f1a80004730 comp rx=0 tx=0).stop 2024-09-10T09:47:51.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:51.024+0000 7f1a96fdc740 1 -- 172.21.15.5:0/88738480 shutdown_connections 2024-09-10T09:47:51.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:51.024+0000 7f1a96fdc740 1 --2- 172.21.15.5:0/88738480 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x56141f97cfb0 0x56141f97f440 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:51.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:51.024+0000 7f1a96fdc740 1 --2- 172.21.15.5:0/88738480 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56141f9764f0 0x56141f973e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:51.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:51.024+0000 7f1a96fdc740 1 --2- 172.21.15.5:0/88738480 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56141f96bc30 0x56141f973900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:51.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:51.024+0000 7f1a96fdc740 1 --2- 172.21.15.5:0/88738480 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56141f96b310 0x56141f9733c0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:51.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:51.024+0000 7f1a73400640 0 client.27334 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T09:47:51.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:51.024+0000 7f1a96fdc740 1 -- 172.21.15.5:0/88738480 >> 172.21.15.5:0/88738480 conn(0x56141f9678a0 msgr2=0x56141f966aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:47:51.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:51.024+0000 7f1a96fdc740 1 -- 172.21.15.5:0/88738480 shutdown_connections 2024-09-10T09:47:51.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:51.024+0000 7f1a96fdc740 1 -- 172.21.15.5:0/88738480 wait complete. 2024-09-10T09:47:51.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:51.025+0000 7f1a96fdc740 20 client.27334 trim_cache size 0 max 16384 2024-09-10T09:47:51.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:51.054+0000 7f1a96fdc740 1 Processor -- start 2024-09-10T09:47:51.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:51.054+0000 7f1a96fdc740 1 -- start start 2024-09-10T09:47:51.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:51.055+0000 7f1a96fdc740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56141f96b310 0x56141f96dd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:47:51.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:51.055+0000 7f1a96fdc740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56141f96bc30 0x56141f98a220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:47:51.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:51.055+0000 7f1a96fdc740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56141f9764f0 0x56141f9a0c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:47:51.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:51.055+0000 7f1a96fdc740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1a8c1acbd0 con 0x56141f96bc30 2024-09-10T09:47:51.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:51.055+0000 7f1a96fdc740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f1a8c1ae740 con 0x56141f96b310 2024-09-10T09:47:51.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:51.055+0000 7f1a96fdc740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f1a8c1ad2a0 con 0x56141f9764f0 2024-09-10T09:47:51.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:51.055+0000 7f1a91c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56141f96b310 0x56141f96dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:47:51.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:51.055+0000 7f1a91c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56141f96b310 0x56141f96dd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:41542/0 (socket says 172.21.15.5:41542) 2024-09-10T09:47:51.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:51.055+0000 7f1a91200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56141f96bc30 0x56141f98a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:47:51.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:51.055+0000 7f1a91c00640 1 -- 172.21.15.5:0/414513749 learned_addr learned my addr 172.21.15.5:0/414513749 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:47:51.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:51.055+0000 7f1a92600640 1 --2- 172.21.15.5:0/414513749 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56141f9764f0 0x56141f9a0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:47:51.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:51.056+0000 7f1a91200640 1 -- 172.21.15.5:0/414513749 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56141f9764f0 msgr2=0x56141f9a0c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:47:51.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:51.056+0000 7f1a91200640 1 --2- 172.21.15.5:0/414513749 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56141f9764f0 0x56141f9a0c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:51.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:51.056+0000 7f1a91200640 1 -- 172.21.15.5:0/414513749 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56141f96b310 msgr2=0x56141f96dd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:47:51.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:51.056+0000 7f1a91200640 1 --2- 172.21.15.5:0/414513749 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56141f96b310 0x56141f96dd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:51.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:51.056+0000 7f1a91200640 1 -- 172.21.15.5:0/414513749 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56141f963260 con 0x56141f96bc30 2024-09-10T09:47:51.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:51.056+0000 7f1a91200640 1 --2- 172.21.15.5:0/414513749 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56141f96bc30 0x56141f98a220 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x56141f968c20 tx=0x7f1a78097720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=230a3b8bdd6e4010 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:47:51.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:51.056+0000 7f1a7e000640 1 -- 172.21.15.5:0/414513749 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a780b8020 con 0x56141f96bc30 2024-09-10T09:47:51.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:51.056+0000 7f1a7e000640 1 -- 172.21.15.5:0/414513749 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1a7809ce50 con 0x56141f96bc30 2024-09-10T09:47:51.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:51.056+0000 7f1a7e000640 1 -- 172.21.15.5:0/414513749 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a780037a0 con 0x56141f96bc30 2024-09-10T09:47:51.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:51.057+0000 7f1a96fdc740 1 -- 172.21.15.5:0/414513749 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56141f96bc30 msgr2=0x56141f98a220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:47:51.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:51.057+0000 7f1a96fdc740 1 --2- 172.21.15.5:0/414513749 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56141f96bc30 0x56141f98a220 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x56141f968c20 tx=0x7f1a78097720 comp rx=0 tx=0).stop 2024-09-10T09:47:51.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:51.057+0000 7f1a96fdc740 1 -- 172.21.15.5:0/414513749 shutdown_connections 2024-09-10T09:47:51.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:51.057+0000 7f1a96fdc740 1 --2- 172.21.15.5:0/414513749 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56141f9764f0 0x56141f9a0c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:51.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:51.057+0000 7f1a96fdc740 1 --2- 172.21.15.5:0/414513749 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56141f96bc30 0x56141f98a220 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:51.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:51.057+0000 7f1a96fdc740 1 --2- 172.21.15.5:0/414513749 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56141f96b310 0x56141f96dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:51.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:51.057+0000 7f1a96fdc740 1 -- 172.21.15.5:0/414513749 >> 172.21.15.5:0/414513749 conn(0x56141f9678a0 msgr2=0x56141f975330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:47:51.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:51.057+0000 7f1a96fdc740 1 -- 172.21.15.5:0/414513749 shutdown_connections 2024-09-10T09:47:51.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:51.057+0000 7f1a96fdc740 1 -- 172.21.15.5:0/414513749 wait complete. 2024-09-10T09:47:51.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:51.057+0000 7f1a96fdc740 1 -- start start 2024-09-10T09:47:51.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:51.058+0000 7f1a96fdc740 1 Processor -- start 2024-09-10T09:47:51.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:51.058+0000 7f1a96fdc740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56141f96b310 0x56141f984f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:47:51.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:51.058+0000 7f1a96fdc740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56141f96bc30 0x56141f9854a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:47:51.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:51.058+0000 7f1a96fdc740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56141f9764f0 0x56141f981d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:47:51.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:51.058+0000 7f1a96fdc740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1a8c1ae040 con 0x56141f96bc30 2024-09-10T09:47:51.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:51.058+0000 7f1a96fdc740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f1a8c1ad970 con 0x56141f9764f0 2024-09-10T09:47:51.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:51.058+0000 7f1a96fdc740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f1a8c1aabb0 con 0x56141f96b310 2024-09-10T09:47:51.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:51.058+0000 7f1a96fdc740 10 client.0 _ll_register_callbacks cb 0x7f1a80005a70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:47:51.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:51.058+0000 7f1a91c00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56141f96b310 0x56141f984f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:47:51.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:51.058+0000 7f1a91c00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56141f96b310 0x56141f984f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:57714/0 (socket says 172.21.15.5:57714) 2024-09-10T09:47:51.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:51.058+0000 7f1a91c00640 1 -- 172.21.15.5:0/1280799951 learned_addr learned my addr 172.21.15.5:0/1280799951 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:47:51.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:51.059+0000 7f1a91200640 1 --2- 172.21.15.5:0/1280799951 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56141f96bc30 0x56141f9854a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:47:51.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:51.059+0000 7f1a92600640 1 --2- 172.21.15.5:0/1280799951 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56141f9764f0 0x56141f981d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:47:51.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:51.059+0000 7f1a91c00640 1 -- 172.21.15.5:0/1280799951 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56141f9764f0 msgr2=0x56141f981d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:47:51.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:51.059+0000 7f1a91c00640 1 --2- 172.21.15.5:0/1280799951 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56141f9764f0 0x56141f981d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:51.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:51.059+0000 7f1a91c00640 1 -- 172.21.15.5:0/1280799951 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56141f96bc30 msgr2=0x56141f9854a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:47:51.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:51.059+0000 7f1a91c00640 1 --2- 172.21.15.5:0/1280799951 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56141f96bc30 0x56141f9854a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:51.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:51.059+0000 7f1a91c00640 1 -- 172.21.15.5:0/1280799951 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56141f982550 con 0x56141f96b310 2024-09-10T09:47:51.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:51.059+0000 7f1a91200640 1 --2- 172.21.15.5:0/1280799951 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56141f96bc30 0x56141f9854a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:47:51.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:51.059+0000 7f1a92600640 1 --2- 172.21.15.5:0/1280799951 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56141f9764f0 0x56141f981d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:47:51.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:51.059+0000 7f1a91c00640 1 --2- 172.21.15.5:0/1280799951 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56141f96b310 0x56141f984f60 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f1a80019040 tx=0x7f1a80009700 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3c3ed50e9131046f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:47:51.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:51.059+0000 7f1a73400640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-09-10T09:47:51.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:51.059+0000 7f1a73400640 1 -- 172.21.15.5:0/1280799951 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a8001f420 con 0x56141f96b310 2024-09-10T09:47:51.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:51.060+0000 7f1a96fdc740 1 -- 172.21.15.5:0/1280799951 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1a7809ce50 con 0x56141f96b310 2024-09-10T09:47:51.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:51.060+0000 7f1a73400640 1 -- 172.21.15.5:0/1280799951 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f1a80003e50 con 0x56141f96b310 2024-09-10T09:47:51.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:51.060+0000 7f1a73400640 1 -- 172.21.15.5:0/1280799951 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f1a80044050 con 0x56141f96b310 2024-09-10T09:47:51.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:51.060+0000 7f1a96fdc740 10 client.27340 fetch_fsmap learned FSMap version 35 2024-09-10T09:47:51.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:51.060+0000 7f1a96fdc740 1 -- 172.21.15.5:0/1280799951 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x56141fa02790 con 0x56141f96b310 2024-09-10T09:47:51.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:51.060+0000 7f1a73400640 1 -- 172.21.15.5:0/1280799951 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1a7809ce50 con 0x56141f96b310 2024-09-10T09:47:51.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:51.060+0000 7f1a73400640 1 -- 172.21.15.5:0/1280799951 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 35) ==== 8374+0+0 (secure 0 0 0) 0x7f1a80027040 con 0x56141f96b310 2024-09-10T09:47:51.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:51.060+0000 7f1a96fdc740 10 client.27340 fetch_fsmap finished waiting for FSMap version 35 2024-09-10T09:47:51.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:51.060+0000 7f1a96fdc740 10 client.27340 resolve_mds: validated gid 24542 aka daemon mds.b 2024-09-10T09:47:51.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:51.060+0000 7f1a96fdc740 20 client.27340 populate_metadata read hostname 'smithi005' 2024-09-10T09:47:51.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:51.060+0000 7f1a96fdc740 1 --2- 172.21.15.5:0/1280799951 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x56141fa02f60 0x56141fa05380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:47:51.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:51.060+0000 7f1a96fdc740 4 client.27340 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "82aad4ea", "release": true, "await": true}] 2024-09-10T09:47:51.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:51.060+0000 7f1a96fdc740 1 -- 172.21.15.5:0/1280799951 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "quiesce db", "set_id": "82aad4ea", "release": true, "await": true}) -- 0x56141f8d90d0 con 0x56141fa02f60 2024-09-10T09:47:51.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:51.061+0000 7f1a91200640 1 --2- 172.21.15.5:0/1280799951 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x56141fa02f60 0x56141fa05380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:47:51.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:51.061+0000 7f1a73400640 10 client.27340 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T09:47:51.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:51.061+0000 7f1a91200640 1 --2- 172.21.15.5:0/1280799951 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x56141fa02f60 0x56141fa05380 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=af9854af3560c8e4 server_cookie=9e7f05575075bfea in_seq=0 out_seq=0 2024-09-10T09:47:51.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:51.069+0000 7f1a73400640 1 -- 172.21.15.5:0/1280799951 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x56141f8d90d0 con 0x56141fa02f60 2024-09-10T09:47:51.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:51.069+0000 7f1a73400640 10 client.27340 handle_command_reply: tid=0 2024-09-10T09:47:51.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:51.069+0000 7f1a96fdc740 1 client.27340 shutdown 2024-09-10T09:47:51.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:51.069+0000 7f1a96fdc740 1 -- 172.21.15.5:0/1280799951 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56141f96b310 msgr2=0x56141f984f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:47:51.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:51.069+0000 7f1a96fdc740 1 --2- 172.21.15.5:0/1280799951 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56141f96b310 0x56141f984f60 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f1a80019040 tx=0x7f1a80009700 comp rx=0 tx=0).stop 2024-09-10T09:47:51.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:51.069+0000 7f1a96fdc740 1 -- 172.21.15.5:0/1280799951 shutdown_connections 2024-09-10T09:47:51.155 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-10T09:47:51.155 INFO:teuthology.orchestra.run.smithi005.stdout: "epoch": 35, 2024-09-10T09:47:51.156 INFO:teuthology.orchestra.run.smithi005.stdout: "leader": 24542, 2024-09-10T09:47:51.156 INFO:teuthology.orchestra.run.smithi005.stdout: "set_version": 32, 2024-09-10T09:47:51.156 INFO:teuthology.orchestra.run.smithi005.stdout: "sets": { 2024-09-10T09:47:51.156 INFO:teuthology.orchestra.run.smithi005.stdout: "82aad4ea": { 2024-09-10T09:47:51.156 INFO:teuthology.orchestra.run.smithi005.stdout: "version": 32, 2024-09-10T09:47:51.156 INFO:teuthology.orchestra.run.smithi005.stdout: "age_ref": 0.0, 2024-09-10T09:47:51.156 INFO:teuthology.orchestra.run.smithi005.stdout: "state": { 2024-09-10T09:47:51.156 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "RELEASED", 2024-09-10T09:47:51.156 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0 2024-09-10T09:47:51.156 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T09:47:51.156 INFO:teuthology.orchestra.run.smithi005.stdout: "timeout": 90.0, 2024-09-10T09:47:51.156 INFO:teuthology.orchestra.run.smithi005.stdout: "expiration": 144.6, 2024-09-10T09:47:51.156 INFO:teuthology.orchestra.run.smithi005.stdout: "members": { 2024-09-10T09:47:51.157 INFO:teuthology.orchestra.run.smithi005.stdout: "file:/": { 2024-09-10T09:47:51.157 INFO:teuthology.orchestra.run.smithi005.stdout: "excluded": false, 2024-09-10T09:47:51.157 INFO:teuthology.orchestra.run.smithi005.stdout: "state": { 2024-09-10T09:47:51.157 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "RELEASED", 2024-09-10T09:47:51.157 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0 2024-09-10T09:47:51.157 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T09:47:51.157 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T09:47:51.157 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T09:47:51.157 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T09:47:51.157 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T09:47:51.157 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-10T09:47:51.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:51.069+0000 7f1a96fdc740 1 --2- 172.21.15.5:0/1280799951 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x56141fa02f60 0x56141fa05380 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:51.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:51.069+0000 7f1a96fdc740 1 --2- 172.21.15.5:0/1280799951 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56141f9764f0 0x56141f981d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:51.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:51.069+0000 7f1a96fdc740 1 --2- 172.21.15.5:0/1280799951 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56141f96bc30 0x56141f9854a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:51.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:51.069+0000 7f1a73400640 0 client.27340 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T09:47:51.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:51.069+0000 7f1a96fdc740 1 --2- 172.21.15.5:0/1280799951 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56141f96b310 0x56141f984f60 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:51.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:51.069+0000 7f1a96fdc740 1 -- 172.21.15.5:0/1280799951 >> 172.21.15.5:0/1280799951 conn(0x56141f9678a0 msgr2=0x56141f974740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:47:51.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:51.069+0000 7f1a96fdc740 1 -- 172.21.15.5:0/1280799951 shutdown_connections 2024-09-10T09:47:51.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:51.069+0000 7f1a96fdc740 1 -- 172.21.15.5:0/1280799951 wait complete. 2024-09-10T09:47:51.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:51.069+0000 7f1a96fdc740 20 client.27340 trim_cache size 0 max 16384 2024-09-10T09:47:51.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:51.070+0000 7f1a94600640 1 -- 172.21.15.5:0/2972480757 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f1a740a2b80 msgr2=0x7f1a740a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:47:51.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:51.070+0000 7f1a94600640 1 --2- 172.21.15.5:0/2972480757 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f1a740a2b80 0x7f1a740a5040 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x56141f968700 tx=0x7f1a80005f90 comp rx=0 tx=0).stop 2024-09-10T09:47:51.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:51.070+0000 7f1a94600640 1 -- 172.21.15.5:0/2972480757 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1a8c1027a0 msgr2=0x7f1a8c1988f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:47:51.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:51.070+0000 7f1a94600640 1 --2- 172.21.15.5:0/2972480757 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1a8c1027a0 0x7f1a8c1988f0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f1a7800e9f0 tx=0x7f1a7800eeb0 comp rx=0 tx=0).stop 2024-09-10T09:47:51.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:51.070+0000 7f1a94600640 1 -- 172.21.15.5:0/2972480757 shutdown_connections 2024-09-10T09:47:51.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:51.070+0000 7f1a94600640 1 --2- 172.21.15.5:0/2972480757 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f1a740a2b80 0x7f1a740a5040 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:51.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:51.070+0000 7f1a94600640 1 --2- 172.21.15.5:0/2972480757 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1a8c106f20 0x7f1a8c19cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:51.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:51.070+0000 7f1a94600640 1 --2- 172.21.15.5:0/2972480757 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1a8c1027a0 0x7f1a8c1988f0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:51.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:51.070+0000 7f1a94600640 1 --2- 172.21.15.5:0/2972480757 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1a8c101df0 0x7f1a8c1983b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:51.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:51.070+0000 7f1a94600640 1 -- 172.21.15.5:0/2972480757 >> 172.21.15.5:0/2972480757 conn(0x7f1a8c0fd8b0 msgr2=0x7f1a8c0fe3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:47:51.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:51.071+0000 7f1a94600640 1 -- 172.21.15.5:0/2972480757 shutdown_connections 2024-09-10T09:47:51.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:51.071+0000 7f1a94600640 1 -- 172.21.15.5:0/2972480757 wait complete. 2024-09-10T09:47:51.350 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:47:51 smithi005 ceph-mon[30350]: pgmap v678: 129 pgs: 129 active+clean; 211 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 1.3 MiB/s wr, 69 op/s 2024-09-10T09:47:51.350 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:47:51 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:47:51.350 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:47:51 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:47:51.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:47:51 smithi159 ceph-mon[35513]: pgmap v678: 129 pgs: 129 active+clean; 211 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 1.3 MiB/s wr, 69 op/s 2024-09-10T09:47:51.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:47:51 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:47:51.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:47:51 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:47:51.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:47:51 smithi102 ceph-mon[35442]: pgmap v678: 129 pgs: 129 active+clean; 211 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 1.3 MiB/s wr, 69 op/s 2024-09-10T09:47:51.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:47:51 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:47:51.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:47:51 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:47:51.670 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:47:52.098 INFO:tasks.workunit.client.0.smithi005.stdout: 1 557368 5.96 MB/sec execute 333 sec latency 26575.339 ms 2024-09-10T09:47:52.199 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '82aad4ea', total seconds elapsed: 30.6 2024-09-10T09:47:52.503 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:47:52 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:47:52.503 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:47:52 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:47:52.503 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:47:52 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:47:52.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:47:52 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:47:52.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:47:52 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:47:52.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:47:52 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:47:52.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:47:52 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:47:52.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:47:52 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:47:52.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:47:52 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:47:52.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:52.805+0000 7f0c74000640 1 -- 172.21.15.5:0/3015257630 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0c6c104010 msgr2=0x7f0c6c1043f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:47:52.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:52.805+0000 7f0c74000640 1 --2- 172.21.15.5:0/3015257630 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0c6c104010 0x7f0c6c1043f0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f0c60009900 tx=0x7f0c6002a3f0 comp rx=0 tx=0).stop 2024-09-10T09:47:52.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:52.805+0000 7f0c74000640 1 -- 172.21.15.5:0/3015257630 shutdown_connections 2024-09-10T09:47:52.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:52.805+0000 7f0c74000640 1 --2- 172.21.15.5:0/3015257630 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0c6c109140 0x7f0c6c110cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:52.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:52.805+0000 7f0c74000640 1 --2- 172.21.15.5:0/3015257630 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0c6c1049c0 0x7f0c6c108a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:52.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:52.805+0000 7f0c74000640 1 --2- 172.21.15.5:0/3015257630 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0c6c104010 0x7f0c6c1043f0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:52.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:52.805+0000 7f0c74000640 1 -- 172.21.15.5:0/3015257630 >> 172.21.15.5:0/3015257630 conn(0x7f0c6c0ffa50 msgr2=0x7f0c6c101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:47:52.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:52.805+0000 7f0c74000640 1 -- 172.21.15.5:0/3015257630 shutdown_connections 2024-09-10T09:47:52.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:52.806+0000 7f0c74000640 1 -- 172.21.15.5:0/3015257630 wait complete. 2024-09-10T09:47:52.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:52.806+0000 7f0c74000640 1 Processor -- start 2024-09-10T09:47:52.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:52.807+0000 7f0c74000640 1 -- start start 2024-09-10T09:47:52.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:52.807+0000 7f0c74000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0c6c104010 0x7f0c6c19a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:47:52.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:52.807+0000 7f0c74000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0c6c1049c0 0x7f0c6c19ac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:47:52.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:52.807+0000 7f0c74000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0c6c109140 0x7f0c6c19efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:47:52.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:52.807+0000 7f0c74000640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f0c6c1141c0 con 0x7f0c6c109140 2024-09-10T09:47:52.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:52.807+0000 7f0c74000640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f0c6c114040 con 0x7f0c6c1049c0 2024-09-10T09:47:52.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:52.807+0000 7f0c74000640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f0c6c114340 con 0x7f0c6c104010 2024-09-10T09:47:52.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:52.807+0000 7f0c71600640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0c6c104010 0x7f0c6c19a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:47:52.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:52.807+0000 7f0c70c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0c6c1049c0 0x7f0c6c19ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:47:52.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:52.807+0000 7f0c71600640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0c6c104010 0x7f0c6c19a6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:57732/0 (socket says 172.21.15.5:57732) 2024-09-10T09:47:52.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:52.807+0000 7f0c72000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0c6c109140 0x7f0c6c19efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:47:52.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:52.807+0000 7f0c71600640 1 -- 172.21.15.5:0/4062382010 learned_addr learned my addr 172.21.15.5:0/4062382010 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:47:52.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:52.808+0000 7f0c70c00640 1 -- 172.21.15.5:0/4062382010 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0c6c104010 msgr2=0x7f0c6c19a6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:47:52.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:52.808+0000 7f0c70c00640 1 --2- 172.21.15.5:0/4062382010 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0c6c104010 0x7f0c6c19a6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:52.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:52.808+0000 7f0c70c00640 1 -- 172.21.15.5:0/4062382010 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0c6c109140 msgr2=0x7f0c6c19efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:47:52.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:52.808+0000 7f0c70c00640 1 --2- 172.21.15.5:0/4062382010 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0c6c109140 0x7f0c6c19efb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:52.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:52.808+0000 7f0c70c00640 1 -- 172.21.15.5:0/4062382010 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c6c19f690 con 0x7f0c6c1049c0 2024-09-10T09:47:52.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:52.808+0000 7f0c70c00640 1 --2- 172.21.15.5:0/4062382010 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0c6c1049c0 0x7f0c6c19ac20 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f0c5800e9f0 tx=0x7f0c5800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:47:52.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:52.808+0000 7f0c6a000640 1 -- 172.21.15.5:0/4062382010 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c5800cd80 con 0x7f0c6c1049c0 2024-09-10T09:47:52.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:52.808+0000 7f0c74000640 1 -- 172.21.15.5:0/4062382010 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c6c19f980 con 0x7f0c6c1049c0 2024-09-10T09:47:52.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:52.808+0000 7f0c6a000640 1 -- 172.21.15.5:0/4062382010 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0c580035f0 con 0x7f0c6c1049c0 2024-09-10T09:47:52.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:52.808+0000 7f0c6a000640 1 -- 172.21.15.5:0/4062382010 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c58010670 con 0x7f0c6c1049c0 2024-09-10T09:47:52.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:52.809+0000 7f0c74000640 1 -- 172.21.15.5:0/4062382010 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c6c1a7260 con 0x7f0c6c1049c0 2024-09-10T09:47:52.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:52.812+0000 7f0c76a3d740 1 Processor -- start 2024-09-10T09:47:52.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:52.812+0000 7f0c6a000640 1 -- 172.21.15.5:0/4062382010 <== mon.2 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f0c580037a0 con 0x7f0c6c1049c0 2024-09-10T09:47:52.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:52.812+0000 7f0c76a3d740 1 -- start start 2024-09-10T09:47:52.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:52.812+0000 7f0c76a3d740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556fba99bc70 0x556fba99c050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:47:52.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:52.812+0000 7f0c76a3d740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556fba99c590 0x556fba9a6910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:47:52.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:52.812+0000 7f0c76a3d740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556fba9a6e50 0x556fba9a9240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:47:52.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:52.812+0000 7f0c76a3d740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x556fba9023e0 con 0x556fba99bc70 2024-09-10T09:47:52.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:52.812+0000 7f0c76a3d740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x556fba8a9d80 con 0x556fba99c590 2024-09-10T09:47:52.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:52.812+0000 7f0c76a3d740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x556fba8fe3a0 con 0x556fba9a6e50 2024-09-10T09:47:52.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:52.812+0000 7f0c71600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556fba99c590 0x556fba9a6910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:47:52.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:52.812+0000 7f0c71600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556fba99c590 0x556fba9a6910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:41562/0 (socket says 172.21.15.5:41562) 2024-09-10T09:47:52.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:52.812+0000 7f0c71600640 1 -- 172.21.15.5:0/3416449230 learned_addr learned my addr 172.21.15.5:0/3416449230 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:47:52.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:52.812+0000 7f0c70c00640 1 --2- 172.21.15.5:0/3416449230 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556fba9a6e50 0x556fba9a9240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:47:52.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:52.812+0000 7f0c72000640 1 --2- 172.21.15.5:0/3416449230 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556fba99bc70 0x556fba99c050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:47:52.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:52.813+0000 7f0c71600640 1 -- 172.21.15.5:0/3416449230 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556fba9a6e50 msgr2=0x556fba9a9240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:47:52.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:52.813+0000 7f0c71600640 1 --2- 172.21.15.5:0/3416449230 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556fba9a6e50 0x556fba9a9240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:52.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:52.813+0000 7f0c71600640 1 -- 172.21.15.5:0/3416449230 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556fba99bc70 msgr2=0x556fba99c050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:47:52.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:52.813+0000 7f0c71600640 1 --2- 172.21.15.5:0/3416449230 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556fba99bc70 0x556fba99c050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:52.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:52.813+0000 7f0c71600640 1 -- 172.21.15.5:0/3416449230 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556fba8c8ae0 con 0x556fba99c590 2024-09-10T09:47:52.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:52.813+0000 7f0c71600640 1 --2- 172.21.15.5:0/3416449230 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556fba99c590 0x556fba9a6910 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f0c60005dd0 tx=0x7f0c60005c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2cb150bc27923b7c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:47:52.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:52.813+0000 7f0c6a000640 1 --2- 172.21.15.5:0/4062382010 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f0c540a2c00 0x7f0c540a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:47:52.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:52.814+0000 7f0c5e600640 1 -- 172.21.15.5:0/3416449230 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c60040070 con 0x556fba99c590 2024-09-10T09:47:52.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:52.814+0000 7f0c5e600640 1 -- 172.21.15.5:0/3416449230 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0c60004400 con 0x556fba99c590 2024-09-10T09:47:52.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:52.814+0000 7f0c5e600640 1 -- 172.21.15.5:0/3416449230 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c60035040 con 0x556fba99c590 2024-09-10T09:47:52.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:52.814+0000 7f0c6a000640 1 -- 172.21.15.5:0/4062382010 <== mon.2 v2:172.21.15.102:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f0c580cf6b0 con 0x7f0c6c1049c0 2024-09-10T09:47:52.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:52.814+0000 7f0c72000640 1 --2- 172.21.15.5:0/4062382010 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f0c540a2c00 0x7f0c540a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:47:52.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:52.814+0000 7f0c76a3d740 1 -- 172.21.15.5:0/3416449230 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556fba99c590 msgr2=0x556fba9a6910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:47:52.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:52.814+0000 7f0c76a3d740 1 --2- 172.21.15.5:0/3416449230 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556fba99c590 0x556fba9a6910 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f0c60005dd0 tx=0x7f0c60005c00 comp rx=0 tx=0).stop 2024-09-10T09:47:52.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:52.814+0000 7f0c76a3d740 1 -- 172.21.15.5:0/3416449230 shutdown_connections 2024-09-10T09:47:52.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:52.815+0000 7f0c76a3d740 1 --2- 172.21.15.5:0/3416449230 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556fba9a6e50 0x556fba9a9240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:52.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:52.815+0000 7f0c76a3d740 1 --2- 172.21.15.5:0/3416449230 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556fba99c590 0x556fba9a6910 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:52.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:52.815+0000 7f0c76a3d740 1 --2- 172.21.15.5:0/3416449230 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556fba99bc70 0x556fba99c050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:52.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:52.815+0000 7f0c76a3d740 1 -- 172.21.15.5:0/3416449230 >> 172.21.15.5:0/3416449230 conn(0x556fba998200 msgr2=0x556fba9985e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:47:52.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:52.815+0000 7f0c76a3d740 1 -- 172.21.15.5:0/3416449230 shutdown_connections 2024-09-10T09:47:52.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:52.815+0000 7f0c76a3d740 1 -- 172.21.15.5:0/3416449230 wait complete. 2024-09-10T09:47:52.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:52.815+0000 7f0c72000640 1 --2- 172.21.15.5:0/4062382010 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f0c540a2c00 0x7f0c540a50c0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f0c64005fd0 tx=0x7f0c64009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:47:52.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:52.815+0000 7f0c76a3d740 1 -- start start 2024-09-10T09:47:52.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:52.816+0000 7f0c76a3d740 1 Processor -- start 2024-09-10T09:47:52.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:52.816+0000 7f0c76a3d740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556fba99bc70 0x556fba9a3d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:47:52.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:52.816+0000 7f0c76a3d740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556fba99c590 0x556fba9a4260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:47:52.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:52.816+0000 7f0c76a3d740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556fba9a6e50 0x556fba9a47a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:47:52.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:52.816+0000 7f0c76a3d740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x556fba9a9a20 con 0x556fba99bc70 2024-09-10T09:47:52.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:52.816+0000 7f0c76a3d740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x556fba8bd0f0 con 0x556fba99c590 2024-09-10T09:47:52.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:52.816+0000 7f0c76a3d740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x556fba9a9ba0 con 0x556fba9a6e50 2024-09-10T09:47:52.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:52.816+0000 7f0c76a3d740 10 client.0 _ll_register_callbacks cb 0x556fba8a4bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:47:52.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:52.816+0000 7f0c70c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556fba99c590 0x556fba9a4260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:47:52.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:52.816+0000 7f0c72000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556fba9a6e50 0x556fba9a47a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:47:52.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:52.816+0000 7f0c70c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556fba99c590 0x556fba9a4260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:41572/0 (socket says 172.21.15.5:41572) 2024-09-10T09:47:52.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:52.816+0000 7f0c71600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556fba99bc70 0x556fba9a3d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:47:52.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:52.816+0000 7f0c70c00640 1 -- 172.21.15.5:0/4049004474 learned_addr learned my addr 172.21.15.5:0/4049004474 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:47:52.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:52.817+0000 7f0c70c00640 1 -- 172.21.15.5:0/4049004474 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556fba9a6e50 msgr2=0x556fba9a47a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:47:52.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:52.817+0000 7f0c70c00640 1 --2- 172.21.15.5:0/4049004474 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556fba9a6e50 0x556fba9a47a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:52.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:52.817+0000 7f0c70c00640 1 -- 172.21.15.5:0/4049004474 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556fba99bc70 msgr2=0x556fba9a3d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:47:52.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:52.817+0000 7f0c70c00640 1 --2- 172.21.15.5:0/4049004474 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556fba99bc70 0x556fba9a3d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:52.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:52.817+0000 7f0c70c00640 1 -- 172.21.15.5:0/4049004474 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556fba6733c0 con 0x556fba99c590 2024-09-10T09:47:52.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:52.817+0000 7f0c72000640 1 --2- 172.21.15.5:0/4049004474 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556fba9a6e50 0x556fba9a47a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:47:52.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:52.817+0000 7f0c70c00640 1 --2- 172.21.15.5:0/4049004474 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556fba99c590 0x556fba9a4260 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f0c5809ba50 tx=0x7f0c58097e90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6c6de11591f2c82e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:47:52.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:52.817+0000 7f0c52a00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-09-10T09:47:52.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:52.817+0000 7f0c52a00640 1 -- 172.21.15.5:0/4049004474 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c5809c1c0 con 0x556fba99c590 2024-09-10T09:47:52.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:52.817+0000 7f0c52a00640 1 -- 172.21.15.5:0/4049004474 <== mon.2 v2:172.21.15.102:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f0c58003b30 con 0x556fba99c590 2024-09-10T09:47:52.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:52.817+0000 7f0c76a3d740 1 -- 172.21.15.5:0/4049004474 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0c60004400 con 0x556fba99c590 2024-09-10T09:47:52.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:52.818+0000 7f0c52a00640 1 -- 172.21.15.5:0/4049004474 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f0c5809bac0 con 0x556fba99c590 2024-09-10T09:47:52.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:52.818+0000 7f0c76a3d740 10 client.27320 fetch_fsmap learned FSMap version 35 2024-09-10T09:47:52.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:52.818+0000 7f0c76a3d740 1 -- 172.21.15.5:0/4049004474 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x556fba9b5b50 con 0x556fba99c590 2024-09-10T09:47:52.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:52.818+0000 7f0c52a00640 1 -- 172.21.15.5:0/4049004474 <== mon.2 v2:172.21.15.102:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0c60004400 con 0x556fba99c590 2024-09-10T09:47:52.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:52.818+0000 7f0c52a00640 1 -- 172.21.15.5:0/4049004474 <== mon.2 v2:172.21.15.102:3300/0 5 ==== fsmap(e 35) ==== 8374+0+0 (secure 0 0 0) 0x7f0c580d1b70 con 0x556fba99c590 2024-09-10T09:47:52.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:52.818+0000 7f0c76a3d740 10 client.27320 fetch_fsmap finished waiting for FSMap version 35 2024-09-10T09:47:52.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:52.818+0000 7f0c76a3d740 10 client.27320 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T09:47:52.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:52.818+0000 7f0c76a3d740 20 client.27320 populate_metadata read hostname 'smithi005' 2024-09-10T09:47:52.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:52.818+0000 7f0c76a3d740 1 --2- 172.21.15.5:0/4049004474 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x556fba9b7010 0x556fba9b9400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:47:52.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:52.818+0000 7f0c76a3d740 4 client.27320 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:47:52.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:52.818+0000 7f0c76a3d740 1 -- 172.21.15.5:0/4049004474 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556fba90f950 con 0x556fba9b7010 2024-09-10T09:47:52.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:52.819+0000 7f0c71600640 1 --2- 172.21.15.5:0/4049004474 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x556fba9b7010 0x556fba9b9400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:47:52.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:52.819+0000 7f0c52a00640 10 client.27320 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T09:47:52.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:52.819+0000 7f0c71600640 1 --2- 172.21.15.5:0/4049004474 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x556fba9b7010 0x556fba9b9400 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7d6c8a9acc4f2ab4 server_cookie=9030cfe52880a6a4 in_seq=0 out_seq=0 2024-09-10T09:47:52.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:52.820+0000 7f0c52a00640 1 -- 172.21.15.5:0/4049004474 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556fba90f950 con 0x556fba9b7010 2024-09-10T09:47:52.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:52.820+0000 7f0c52a00640 10 client.27320 handle_command_reply: tid=0 2024-09-10T09:47:52.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:52.820+0000 7f0c76a3d740 1 client.27320 shutdown 2024-09-10T09:47:52.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:52.821+0000 7f0c76a3d740 1 -- 172.21.15.5:0/4049004474 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556fba99c590 msgr2=0x556fba9a4260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:47:52.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:52.821+0000 7f0c76a3d740 1 --2- 172.21.15.5:0/4049004474 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556fba99c590 0x556fba9a4260 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f0c5809ba50 tx=0x7f0c58097e90 comp rx=0 tx=0).stop 2024-09-10T09:47:52.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:52.821+0000 7f0c76a3d740 1 -- 172.21.15.5:0/4049004474 shutdown_connections 2024-09-10T09:47:52.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:52.821+0000 7f0c76a3d740 1 --2- 172.21.15.5:0/4049004474 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x556fba9b7010 0x556fba9b9400 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:52.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:52.821+0000 7f0c76a3d740 1 --2- 172.21.15.5:0/4049004474 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556fba9a6e50 0x556fba9a47a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:52.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:52.821+0000 7f0c76a3d740 1 --2- 172.21.15.5:0/4049004474 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556fba99c590 0x556fba9a4260 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:52.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:52.821+0000 7f0c52a00640 0 client.27320 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T09:47:52.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:52.821+0000 7f0c76a3d740 1 --2- 172.21.15.5:0/4049004474 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556fba99bc70 0x556fba9a3d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:52.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:52.821+0000 7f0c76a3d740 1 -- 172.21.15.5:0/4049004474 >> 172.21.15.5:0/4049004474 conn(0x556fba998200 msgr2=0x556fba99df30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:47:52.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:52.821+0000 7f0c76a3d740 1 -- 172.21.15.5:0/4049004474 shutdown_connections 2024-09-10T09:47:52.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:52.821+0000 7f0c76a3d740 1 -- 172.21.15.5:0/4049004474 wait complete. 2024-09-10T09:47:52.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:52.821+0000 7f0c76a3d740 20 client.27320 trim_cache size 0 max 16384 2024-09-10T09:47:52.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:52.836+0000 7f0c76a3d740 1 Processor -- start 2024-09-10T09:47:52.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:52.836+0000 7f0c76a3d740 1 -- start start 2024-09-10T09:47:52.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:52.836+0000 7f0c76a3d740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556fba99bc70 0x556fba9bec80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:47:52.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:52.836+0000 7f0c76a3d740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556fba99c590 0x556fba9bf1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:47:52.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:52.836+0000 7f0c76a3d740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556fba9a6e50 0x556fba9a0f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:47:52.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:52.836+0000 7f0c76a3d740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f0c6c1b10b0 con 0x556fba9a6e50 2024-09-10T09:47:52.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:52.836+0000 7f0c76a3d740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f0c6c1b2c20 con 0x556fba99c590 2024-09-10T09:47:52.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:52.836+0000 7f0c76a3d740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f0c6c1b1780 con 0x556fba99bc70 2024-09-10T09:47:52.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:52.836+0000 7f0c71600640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556fba99bc70 0x556fba9bec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:47:52.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:52.836+0000 7f0c71600640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556fba99bc70 0x556fba9bec80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:57758/0 (socket says 172.21.15.5:57758) 2024-09-10T09:47:52.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:52.836+0000 7f0c71600640 1 -- 172.21.15.5:0/265170750 learned_addr learned my addr 172.21.15.5:0/265170750 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:47:52.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:52.836+0000 7f0c70c00640 1 --2- 172.21.15.5:0/265170750 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556fba99c590 0x556fba9bf1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:47:52.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:52.836+0000 7f0c72000640 1 --2- 172.21.15.5:0/265170750 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556fba9a6e50 0x556fba9a0f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:47:52.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:52.837+0000 7f0c71600640 1 -- 172.21.15.5:0/265170750 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556fba99c590 msgr2=0x556fba9bf1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:47:52.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:52.837+0000 7f0c71600640 1 --2- 172.21.15.5:0/265170750 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556fba99c590 0x556fba9bf1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:52.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:52.837+0000 7f0c71600640 1 -- 172.21.15.5:0/265170750 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556fba9a6e50 msgr2=0x556fba9a0f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:47:52.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:52.837+0000 7f0c71600640 1 --2- 172.21.15.5:0/265170750 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556fba9a6e50 0x556fba9a0f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:52.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:52.837+0000 7f0c71600640 1 -- 172.21.15.5:0/265170750 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556fba993770 con 0x556fba99bc70 2024-09-10T09:47:52.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:52.837+0000 7f0c72000640 1 --2- 172.21.15.5:0/265170750 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556fba9a6e50 0x556fba9a0f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:47:52.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:52.837+0000 7f0c71600640 1 --2- 172.21.15.5:0/265170750 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556fba99bc70 0x556fba9bec80 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f0c60031550 tx=0x7f0c60004c40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fffddfce68ed35e9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:47:52.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:52.837+0000 7f0c5dc00640 1 -- 172.21.15.5:0/265170750 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c60040040 con 0x556fba99bc70 2024-09-10T09:47:52.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:52.837+0000 7f0c5dc00640 1 -- 172.21.15.5:0/265170750 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0c600046c0 con 0x556fba99bc70 2024-09-10T09:47:52.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:52.837+0000 7f0c5dc00640 1 -- 172.21.15.5:0/265170750 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c60035040 con 0x556fba99bc70 2024-09-10T09:47:52.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:52.838+0000 7f0c76a3d740 1 -- 172.21.15.5:0/265170750 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556fba99bc70 msgr2=0x556fba9bec80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:47:52.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:52.838+0000 7f0c76a3d740 1 --2- 172.21.15.5:0/265170750 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556fba99bc70 0x556fba9bec80 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f0c60031550 tx=0x7f0c60004c40 comp rx=0 tx=0).stop 2024-09-10T09:47:52.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:52.838+0000 7f0c76a3d740 1 -- 172.21.15.5:0/265170750 shutdown_connections 2024-09-10T09:47:52.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:52.838+0000 7f0c76a3d740 1 --2- 172.21.15.5:0/265170750 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556fba9a6e50 0x556fba9a0f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:52.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:52.838+0000 7f0c76a3d740 1 --2- 172.21.15.5:0/265170750 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556fba99c590 0x556fba9bf1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:52.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:52.838+0000 7f0c76a3d740 1 --2- 172.21.15.5:0/265170750 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556fba99bc70 0x556fba9bec80 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:52.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:52.838+0000 7f0c76a3d740 1 -- 172.21.15.5:0/265170750 >> 172.21.15.5:0/265170750 conn(0x556fba998200 msgr2=0x556fba9a5da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:47:52.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:52.838+0000 7f0c76a3d740 1 -- 172.21.15.5:0/265170750 shutdown_connections 2024-09-10T09:47:52.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:52.838+0000 7f0c76a3d740 1 -- 172.21.15.5:0/265170750 wait complete. 2024-09-10T09:47:52.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:52.839+0000 7f0c76a3d740 1 -- start start 2024-09-10T09:47:52.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:52.839+0000 7f0c76a3d740 1 Processor -- start 2024-09-10T09:47:52.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:52.839+0000 7f0c76a3d740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556fba99bc70 0x556fba9af650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:47:52.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:52.839+0000 7f0c76a3d740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556fba99c590 0x556fba9aa980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:47:52.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:52.839+0000 7f0c76a3d740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556fba9a6e50 0x556fba9aaf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:47:52.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:52.839+0000 7f0c76a3d740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f0c6c1b2520 con 0x556fba9a6e50 2024-09-10T09:47:52.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:52.839+0000 7f0c76a3d740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f0c6c1b1e50 con 0x556fba99bc70 2024-09-10T09:47:52.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:52.839+0000 7f0c76a3d740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f0c6c1af090 con 0x556fba99c590 2024-09-10T09:47:52.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:52.839+0000 7f0c76a3d740 10 client.0 _ll_register_callbacks cb 0x7f0c580d5220 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:47:52.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:52.840+0000 7f0c71600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556fba99bc70 0x556fba9af650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:47:52.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:52.840+0000 7f0c70c00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556fba99c590 0x556fba9aa980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:47:52.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:52.840+0000 7f0c71600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556fba99bc70 0x556fba9af650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:41592/0 (socket says 172.21.15.5:41592) 2024-09-10T09:47:52.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:52.840+0000 7f0c70c00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556fba99c590 0x556fba9aa980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:57772/0 (socket says 172.21.15.5:57772) 2024-09-10T09:47:52.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:52.840+0000 7f0c71600640 1 -- 172.21.15.5:0/408023422 learned_addr learned my addr 172.21.15.5:0/408023422 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:47:52.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:52.840+0000 7f0c72000640 1 --2- 172.21.15.5:0/408023422 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556fba9a6e50 0x556fba9aaf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:47:52.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:52.840+0000 7f0c70c00640 1 -- 172.21.15.5:0/408023422 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556fba99bc70 msgr2=0x556fba9af650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:47:52.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:52.840+0000 7f0c70c00640 1 --2- 172.21.15.5:0/408023422 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556fba99bc70 0x556fba9af650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:52.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:52.840+0000 7f0c70c00640 1 -- 172.21.15.5:0/408023422 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556fba9a6e50 msgr2=0x556fba9aaf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:47:52.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:52.840+0000 7f0c70c00640 1 --2- 172.21.15.5:0/408023422 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556fba9a6e50 0x556fba9aaf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:52.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:52.840+0000 7f0c70c00640 1 -- 172.21.15.5:0/408023422 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556fba99eae0 con 0x556fba99c590 2024-09-10T09:47:52.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:52.840+0000 7f0c70c00640 1 --2- 172.21.15.5:0/408023422 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556fba99c590 0x556fba9aa980 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f0c58097f20 tx=0x7f0c580d5360 comp rx=0 tx=0).ready entity=mon.1 client_cookie=90852522687fc342 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:47:52.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:52.840+0000 7f0c52a00640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-09-10T09:47:52.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:52.840+0000 7f0c52a00640 1 -- 172.21.15.5:0/408023422 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c5809c1c0 con 0x556fba99c590 2024-09-10T09:47:52.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:52.840+0000 7f0c52a00640 1 -- 172.21.15.5:0/408023422 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f0c5809b050 con 0x556fba99c590 2024-09-10T09:47:52.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:52.841+0000 7f0c76a3d740 1 -- 172.21.15.5:0/408023422 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0c600046c0 con 0x556fba99c590 2024-09-10T09:47:52.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:52.841+0000 7f0c52a00640 1 -- 172.21.15.5:0/408023422 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f0c5809fe70 con 0x556fba99c590 2024-09-10T09:47:52.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:52.841+0000 7f0c76a3d740 10 client.27358 fetch_fsmap learned FSMap version 35 2024-09-10T09:47:52.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:52.841+0000 7f0c76a3d740 1 -- 172.21.15.5:0/408023422 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x556fbaa21b20 con 0x556fba99c590 2024-09-10T09:47:52.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:52.841+0000 7f0c52a00640 1 -- 172.21.15.5:0/408023422 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0c600046c0 con 0x556fba99c590 2024-09-10T09:47:52.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:52.841+0000 7f0c52a00640 1 -- 172.21.15.5:0/408023422 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 35) ==== 8374+0+0 (secure 0 0 0) 0x7f0c5809a980 con 0x556fba99c590 2024-09-10T09:47:52.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:52.841+0000 7f0c76a3d740 10 client.27358 fetch_fsmap finished waiting for FSMap version 35 2024-09-10T09:47:52.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:52.841+0000 7f0c76a3d740 10 client.27358 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T09:47:52.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:52.841+0000 7f0c76a3d740 20 client.27358 populate_metadata read hostname 'smithi005' 2024-09-10T09:47:52.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:52.841+0000 7f0c76a3d740 1 --2- 172.21.15.5:0/408023422 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x556fbaa22140 0x556fbaa24560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:47:52.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:52.841+0000 7f0c76a3d740 4 client.27358 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T09:47:52.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:52.841+0000 7f0c76a3d740 1 -- 172.21.15.5:0/408023422 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556fba909e60 con 0x556fbaa22140 2024-09-10T09:47:52.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:52.842+0000 7f0c71600640 1 --2- 172.21.15.5:0/408023422 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x556fbaa22140 0x556fbaa24560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:47:52.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:52.842+0000 7f0c71600640 1 --2- 172.21.15.5:0/408023422 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x556fbaa22140 0x556fbaa24560 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d7685f612cfde421 server_cookie=b7bb3767697ac412 in_seq=0 out_seq=0 2024-09-10T09:47:52.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:52.842+0000 7f0c52a00640 10 client.27358 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T09:47:52.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:52.842+0000 7f0c52a00640 1 -- 172.21.15.5:0/408023422 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556fba909e60 con 0x556fbaa22140 2024-09-10T09:47:52.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:52.842+0000 7f0c52a00640 10 client.27358 handle_command_reply: tid=0 2024-09-10T09:47:52.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:52.842+0000 7f0c76a3d740 1 client.27358 shutdown 2024-09-10T09:47:52.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:52.842+0000 7f0c76a3d740 1 -- 172.21.15.5:0/408023422 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556fba99c590 msgr2=0x556fba9aa980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:47:52.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:52.842+0000 7f0c76a3d740 1 --2- 172.21.15.5:0/408023422 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556fba99c590 0x556fba9aa980 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f0c58097f20 tx=0x7f0c580d5360 comp rx=0 tx=0).stop 2024-09-10T09:47:52.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:52.842+0000 7f0c76a3d740 1 -- 172.21.15.5:0/408023422 shutdown_connections 2024-09-10T09:47:52.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:52.842+0000 7f0c76a3d740 1 --2- 172.21.15.5:0/408023422 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x556fbaa22140 0x556fbaa24560 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:52.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:52.842+0000 7f0c76a3d740 1 --2- 172.21.15.5:0/408023422 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556fba9a6e50 0x556fba9aaf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:52.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:52.843+0000 7f0c76a3d740 1 --2- 172.21.15.5:0/408023422 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556fba99c590 0x556fba9aa980 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:52.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:52.843+0000 7f0c52a00640 0 client.27358 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T09:47:52.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:52.843+0000 7f0c76a3d740 1 --2- 172.21.15.5:0/408023422 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556fba99bc70 0x556fba9af650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:52.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:52.843+0000 7f0c76a3d740 1 -- 172.21.15.5:0/408023422 >> 172.21.15.5:0/408023422 conn(0x556fba998200 msgr2=0x556fba996b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:47:52.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:52.843+0000 7f0c76a3d740 1 -- 172.21.15.5:0/408023422 shutdown_connections 2024-09-10T09:47:52.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:52.843+0000 7f0c76a3d740 1 -- 172.21.15.5:0/408023422 wait complete. 2024-09-10T09:47:52.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:52.843+0000 7f0c76a3d740 20 client.27358 trim_cache size 0 max 16384 2024-09-10T09:47:52.848 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-10T09:47:52.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:52.843+0000 7f0c74000640 1 -- 172.21.15.5:0/4062382010 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f0c540a2c00 msgr2=0x7f0c540a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:47:52.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:52.843+0000 7f0c74000640 1 --2- 172.21.15.5:0/4062382010 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f0c540a2c00 0x7f0c540a50c0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f0c64005fd0 tx=0x7f0c64009500 comp rx=0 tx=0).stop 2024-09-10T09:47:52.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:52.843+0000 7f0c74000640 1 -- 172.21.15.5:0/4062382010 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0c6c1049c0 msgr2=0x7f0c6c19ac20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:47:52.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:52.843+0000 7f0c74000640 1 --2- 172.21.15.5:0/4062382010 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0c6c1049c0 0x7f0c6c19ac20 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f0c5800e9f0 tx=0x7f0c5800eeb0 comp rx=0 tx=0).stop 2024-09-10T09:47:52.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:52.844+0000 7f0c74000640 1 -- 172.21.15.5:0/4062382010 shutdown_connections 2024-09-10T09:47:52.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:52.844+0000 7f0c74000640 1 --2- 172.21.15.5:0/4062382010 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f0c540a2c00 0x7f0c540a50c0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:52.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:52.844+0000 7f0c74000640 1 --2- 172.21.15.5:0/4062382010 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0c6c109140 0x7f0c6c19efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:52.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:52.844+0000 7f0c74000640 1 --2- 172.21.15.5:0/4062382010 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0c6c1049c0 0x7f0c6c19ac20 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:52.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:52.844+0000 7f0c74000640 1 --2- 172.21.15.5:0/4062382010 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0c6c104010 0x7f0c6c19a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:52.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:52.844+0000 7f0c74000640 1 -- 172.21.15.5:0/4062382010 >> 172.21.15.5:0/4062382010 conn(0x7f0c6c0ffa50 msgr2=0x7f0c6c100de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:47:52.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:52.844+0000 7f0c74000640 1 -- 172.21.15.5:0/4062382010 shutdown_connections 2024-09-10T09:47:52.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:52.844+0000 7f0c74000640 1 -- 172.21.15.5:0/4062382010 wait complete. 2024-09-10T09:47:53.098 INFO:tasks.workunit.client.0.smithi005.stdout: 1 558994 5.97 MB/sec execute 334 sec latency 31.594 ms 2024-09-10T09:47:53.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:47:53 smithi005 ceph-mon[30350]: pgmap v679: 129 pgs: 129 active+clean; 212 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 1.6 MiB/s wr, 64 op/s 2024-09-10T09:47:53.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:47:53 smithi159 ceph-mon[35513]: pgmap v679: 129 pgs: 129 active+clean; 212 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 1.6 MiB/s wr, 64 op/s 2024-09-10T09:47:53.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:47:53 smithi102 ceph-mon[35442]: pgmap v679: 129 pgs: 129 active+clean; 212 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 1.6 MiB/s wr, 64 op/s 2024-09-10T09:47:54.098 INFO:tasks.workunit.client.0.smithi005.stdout: 1 561528 6.00 MB/sec execute 335 sec latency 17.107 ms 2024-09-10T09:47:54.643 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-10T09:47:54.644 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-10T09:47:55.098 INFO:tasks.workunit.client.0.smithi005.stdout: 1 562475 5.99 MB/sec execute 336 sec latency 257.055 ms 2024-09-10T09:47:55.135 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:47:55.537 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:47:55 smithi005 ceph-mon[30350]: pgmap v680: 129 pgs: 129 active+clean; 212 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 1.5 MiB/s wr, 62 op/s 2024-09-10T09:47:55.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:47:55 smithi159 ceph-mon[35513]: pgmap v680: 129 pgs: 129 active+clean; 212 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 1.5 MiB/s wr, 62 op/s 2024-09-10T09:47:55.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:47:55 smithi102 ceph-mon[35442]: pgmap v680: 129 pgs: 129 active+clean; 212 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 1.5 MiB/s wr, 62 op/s 2024-09-10T09:47:56.098 INFO:tasks.workunit.client.0.smithi005.stdout: 1 562475 5.97 MB/sec execute 337 sec latency 1257.114 ms 2024-09-10T09:47:57.098 INFO:tasks.workunit.client.0.smithi005.stdout: 1 563601 5.97 MB/sec execute 338 sec latency 1805.863 ms 2024-09-10T09:47:57.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:56.932+0000 7f6836400640 1 -- 172.21.15.5:0/4221828279 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f683010e060 msgr2=0x7f683010e440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:47:57.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:56.932+0000 7f6836400640 1 --2- 172.21.15.5:0/4221828279 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f683010e060 0x7f683010e440 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f6824009930 tx=0x7f682402a440 comp rx=0 tx=0).stop 2024-09-10T09:47:57.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:56.933+0000 7f6836400640 1 -- 172.21.15.5:0/4221828279 shutdown_connections 2024-09-10T09:47:57.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:56.933+0000 7f6836400640 1 --2- 172.21.15.5:0/4221828279 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6830108980 0x7f6830114ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:57.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:56.933+0000 7f6836400640 1 --2- 172.21.15.5:0/4221828279 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6830108020 0x7f6830108440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:57.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:56.933+0000 7f6836400640 1 --2- 172.21.15.5:0/4221828279 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f683010e060 0x7f683010e440 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:57.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:56.933+0000 7f6836400640 1 -- 172.21.15.5:0/4221828279 >> 172.21.15.5:0/4221828279 conn(0x7f6830077980 msgr2=0x7f6830079da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:47:57.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:56.933+0000 7f6836400640 1 -- 172.21.15.5:0/4221828279 shutdown_connections 2024-09-10T09:47:57.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:56.933+0000 7f6836400640 1 -- 172.21.15.5:0/4221828279 wait complete. 2024-09-10T09:47:57.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:56.934+0000 7f6836400640 1 Processor -- start 2024-09-10T09:47:57.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:56.934+0000 7f6836400640 1 -- start start 2024-09-10T09:47:57.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:56.934+0000 7f6836400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6830108020 0x7f68301a4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:47:57.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:56.934+0000 7f6836400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6830108980 0x7f68301a4b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:47:57.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:56.934+0000 7f6836400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f683010e060 0x7f683019e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:47:57.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:56.934+0000 7f6836400640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f6830118360 con 0x7f6830108980 2024-09-10T09:47:57.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:56.934+0000 7f6836400640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f68301181e0 con 0x7f683010e060 2024-09-10T09:47:57.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:56.934+0000 7f6836400640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f68301184e0 con 0x7f6830108020 2024-09-10T09:47:57.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:56.935+0000 7f6835a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f683010e060 0x7f683019e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:47:57.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:56.935+0000 7f6835000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6830108020 0x7f68301a4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:47:57.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:56.935+0000 7f6835000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6830108020 0x7f68301a4600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:57790/0 (socket says 172.21.15.5:57790) 2024-09-10T09:47:57.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:56.935+0000 7f6835000640 1 -- 172.21.15.5:0/4115034965 learned_addr learned my addr 172.21.15.5:0/4115034965 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:47:57.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:56.935+0000 7f6835a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f683010e060 0x7f683019e6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:41620/0 (socket says 172.21.15.5:41620) 2024-09-10T09:47:57.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:56.935+0000 7f682fe00640 1 --2- 172.21.15.5:0/4115034965 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6830108980 0x7f68301a4b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:47:57.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:56.935+0000 7f6835a00640 1 -- 172.21.15.5:0/4115034965 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6830108020 msgr2=0x7f68301a4600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:47:57.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:56.935+0000 7f6835a00640 1 --2- 172.21.15.5:0/4115034965 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6830108020 0x7f68301a4600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:57.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:56.935+0000 7f6835a00640 1 -- 172.21.15.5:0/4115034965 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6830108980 msgr2=0x7f68301a4b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:47:57.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:56.935+0000 7f6835a00640 1 --2- 172.21.15.5:0/4115034965 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6830108980 0x7f68301a4b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:57.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:56.935+0000 7f6835a00640 1 -- 172.21.15.5:0/4115034965 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f683019ef40 con 0x7f683010e060 2024-09-10T09:47:57.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:56.935+0000 7f682fe00640 1 --2- 172.21.15.5:0/4115034965 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6830108980 0x7f68301a4b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:47:57.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:56.935+0000 7f6835a00640 1 --2- 172.21.15.5:0/4115034965 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f683010e060 0x7f683019e6d0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f6828005fd0 tx=0x7f682800ee60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:47:57.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:56.936+0000 7f682d600640 1 -- 172.21.15.5:0/4115034965 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f682800cd80 con 0x7f683010e060 2024-09-10T09:47:57.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:56.936+0000 7f6836400640 1 -- 172.21.15.5:0/4115034965 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f683019f230 con 0x7f683010e060 2024-09-10T09:47:57.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:56.936+0000 7f682d600640 1 -- 172.21.15.5:0/4115034965 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6828005020 con 0x7f683010e060 2024-09-10T09:47:57.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:56.936+0000 7f682d600640 1 -- 172.21.15.5:0/4115034965 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6828010720 con 0x7f683010e060 2024-09-10T09:47:57.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:56.936+0000 7f6836400640 1 -- 172.21.15.5:0/4115034965 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68301ab3b0 con 0x7f683010e060 2024-09-10T09:47:57.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:56.939+0000 7f683a5e0740 1 Processor -- start 2024-09-10T09:47:57.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:56.940+0000 7f682d600640 1 -- 172.21.15.5:0/4115034965 <== mon.2 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f68280051d0 con 0x7f683010e060 2024-09-10T09:47:57.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:56.940+0000 7f683a5e0740 1 -- start start 2024-09-10T09:47:57.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:56.940+0000 7f683a5e0740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564ce3d0bcb0 0x564ce3d0c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:47:57.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:56.940+0000 7f683a5e0740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564ce3d0c5d0 0x564ce3d16950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:47:57.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:56.940+0000 7f683a5e0740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564ce3d16e90 0x564ce3d19280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:47:57.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:56.940+0000 7f683a5e0740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x564ce3c72290 con 0x564ce3d16e90 2024-09-10T09:47:57.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:56.940+0000 7f683a5e0740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x564ce3c19b40 con 0x564ce3d0bcb0 2024-09-10T09:47:57.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:56.940+0000 7f683a5e0740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x564ce3c6e250 con 0x564ce3d0c5d0 2024-09-10T09:47:57.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:56.941+0000 7f6835000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564ce3d0c5d0 0x564ce3d16950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:47:57.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:56.941+0000 7f6835000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564ce3d0c5d0 0x564ce3d16950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:57804/0 (socket says 172.21.15.5:57804) 2024-09-10T09:47:57.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:56.941+0000 7f6835000640 1 -- 172.21.15.5:0/2006303427 learned_addr learned my addr 172.21.15.5:0/2006303427 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:47:57.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:56.941+0000 7f682fe00640 1 --2- 172.21.15.5:0/2006303427 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564ce3d0bcb0 0x564ce3d0c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:47:57.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:56.941+0000 7f682fe00640 1 --2- 172.21.15.5:0/2006303427 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564ce3d16e90 0x564ce3d19280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:47:57.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:56.941+0000 7f6835000640 1 -- 172.21.15.5:0/2006303427 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564ce3d0bcb0 msgr2=0x564ce3d0c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:47:57.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:56.941+0000 7f6835000640 1 --2- 172.21.15.5:0/2006303427 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564ce3d0bcb0 0x564ce3d0c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:57.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:56.941+0000 7f6835000640 1 -- 172.21.15.5:0/2006303427 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564ce3d16e90 msgr2=0x564ce3d19280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:47:57.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:56.941+0000 7f6835000640 1 --2- 172.21.15.5:0/2006303427 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564ce3d16e90 0x564ce3d19280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:57.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:56.941+0000 7f6835000640 1 -- 172.21.15.5:0/2006303427 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564ce39e36a0 con 0x564ce3d0c5d0 2024-09-10T09:47:57.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:56.941+0000 7f682fe00640 1 --2- 172.21.15.5:0/2006303427 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564ce3d16e90 0x564ce3d19280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T09:47:57.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:56.942+0000 7f6835000640 1 --2- 172.21.15.5:0/2006303427 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564ce3d0c5d0 0x564ce3d16950 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f6824006010 tx=0x7f6824005dc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=19cc8ac2a022417c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:47:57.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:56.942+0000 7f682d600640 1 --2- 172.21.15.5:0/4115034965 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f68180a2b80 0x7f68180a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:47:57.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:56.942+0000 7f682d600640 1 -- 172.21.15.5:0/4115034965 <== mon.2 v2:172.21.15.102:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f68280cfab0 con 0x7f683010e060 2024-09-10T09:47:57.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:56.942+0000 7f6822000640 1 -- 172.21.15.5:0/2006303427 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6824040070 con 0x564ce3d0c5d0 2024-09-10T09:47:57.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:56.942+0000 7f6822000640 1 -- 172.21.15.5:0/2006303427 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f68240318f0 con 0x564ce3d0c5d0 2024-09-10T09:47:57.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:56.942+0000 7f6822000640 1 -- 172.21.15.5:0/2006303427 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6824035040 con 0x564ce3d0c5d0 2024-09-10T09:47:57.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:56.942+0000 7f682fe00640 1 --2- 172.21.15.5:0/4115034965 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f68180a2b80 0x7f68180a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:47:57.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:56.943+0000 7f682fe00640 1 --2- 172.21.15.5:0/4115034965 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f68180a2b80 0x7f68180a5040 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x564ce3d095c0 tx=0x7f681c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:47:57.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:56.943+0000 7f683a5e0740 1 -- 172.21.15.5:0/2006303427 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564ce3d0c5d0 msgr2=0x564ce3d16950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:47:57.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:56.943+0000 7f683a5e0740 1 --2- 172.21.15.5:0/2006303427 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564ce3d0c5d0 0x564ce3d16950 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f6824006010 tx=0x7f6824005dc0 comp rx=0 tx=0).stop 2024-09-10T09:47:57.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:56.943+0000 7f683a5e0740 1 -- 172.21.15.5:0/2006303427 shutdown_connections 2024-09-10T09:47:57.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:56.943+0000 7f683a5e0740 1 --2- 172.21.15.5:0/2006303427 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564ce3d16e90 0x564ce3d19280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:57.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:56.943+0000 7f683a5e0740 1 --2- 172.21.15.5:0/2006303427 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564ce3d0c5d0 0x564ce3d16950 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:57.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:56.943+0000 7f683a5e0740 1 --2- 172.21.15.5:0/2006303427 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564ce3d0bcb0 0x564ce3d0c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:57.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:56.943+0000 7f683a5e0740 1 -- 172.21.15.5:0/2006303427 >> 172.21.15.5:0/2006303427 conn(0x564ce3d08240 msgr2=0x564ce3d08620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:47:57.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:56.944+0000 7f683a5e0740 1 -- 172.21.15.5:0/2006303427 shutdown_connections 2024-09-10T09:47:57.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:56.944+0000 7f683a5e0740 1 -- 172.21.15.5:0/2006303427 wait complete. 2024-09-10T09:47:57.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:56.944+0000 7f683a5e0740 1 -- start start 2024-09-10T09:47:57.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:56.944+0000 7f683a5e0740 1 Processor -- start 2024-09-10T09:47:57.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:56.945+0000 7f683a5e0740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564ce3d0bcb0 0x564ce3d16690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:47:57.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:56.945+0000 7f683a5e0740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564ce3d0c5d0 0x564ce3d13940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:47:57.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:56.945+0000 7f683a5e0740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564ce3d16e90 0x564ce3d13e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:47:57.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:56.945+0000 7f683a5e0740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x564ce3d19b00 con 0x564ce3d16e90 2024-09-10T09:47:57.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:56.945+0000 7f683a5e0740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x564ce3c2cfa0 con 0x564ce3d0bcb0 2024-09-10T09:47:57.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:56.945+0000 7f683a5e0740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x564ce3d19c80 con 0x564ce3d0c5d0 2024-09-10T09:47:57.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:56.945+0000 7f683a5e0740 10 client.0 _ll_register_callbacks cb 0x564ce3980ba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:47:57.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:56.945+0000 7f6835000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564ce3d0bcb0 0x564ce3d16690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:47:57.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:56.945+0000 7f6835000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564ce3d0bcb0 0x564ce3d16690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:41644/0 (socket says 172.21.15.5:41644) 2024-09-10T09:47:57.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:56.945+0000 7f6835000640 1 -- 172.21.15.5:0/241631338 learned_addr learned my addr 172.21.15.5:0/241631338 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:47:57.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:56.945+0000 7f6835a00640 1 --2- 172.21.15.5:0/241631338 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564ce3d16e90 0x564ce3d13e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:47:57.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:56.945+0000 7f6835000640 1 -- 172.21.15.5:0/241631338 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564ce3d0c5d0 msgr2=0x564ce3d13940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:47:57.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:56.945+0000 7f6835000640 1 --2- 172.21.15.5:0/241631338 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564ce3d0c5d0 0x564ce3d13940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:57.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:56.946+0000 7f6835000640 1 -- 172.21.15.5:0/241631338 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564ce3d16e90 msgr2=0x564ce3d13e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:47:57.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:56.946+0000 7f6835000640 1 --2- 172.21.15.5:0/241631338 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564ce3d16e90 0x564ce3d13e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:57.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:56.946+0000 7f6835000640 1 -- 172.21.15.5:0/241631338 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564ce3d146b0 con 0x564ce3d0bcb0 2024-09-10T09:47:57.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:56.946+0000 7f6835a00640 1 --2- 172.21.15.5:0/241631338 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564ce3d16e90 0x564ce3d13e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:47:57.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:56.946+0000 7f6835000640 1 --2- 172.21.15.5:0/241631338 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564ce3d0bcb0 0x564ce3d16690 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f6824009a50 tx=0x7f68240314e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=be3203c01263676e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:47:57.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:56.946+0000 7f6816a00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-09-10T09:47:57.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:56.946+0000 7f6816a00640 1 -- 172.21.15.5:0/241631338 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f682404c020 con 0x564ce3d0bcb0 2024-09-10T09:47:57.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:56.946+0000 7f683a5e0740 1 -- 172.21.15.5:0/241631338 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f68240318f0 con 0x564ce3d0bcb0 2024-09-10T09:47:57.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:56.946+0000 7f6816a00640 1 -- 172.21.15.5:0/241631338 <== mon.2 v2:172.21.15.102:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f6824004980 con 0x564ce3d0bcb0 2024-09-10T09:47:57.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:56.947+0000 7f6816a00640 1 -- 172.21.15.5:0/241631338 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f6824057050 con 0x564ce3d0bcb0 2024-09-10T09:47:57.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:56.947+0000 7f683a5e0740 10 client.27338 fetch_fsmap learned FSMap version 35 2024-09-10T09:47:57.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:56.947+0000 7f683a5e0740 1 -- 172.21.15.5:0/241631338 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x564ce3d25880 con 0x564ce3d0bcb0 2024-09-10T09:47:57.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:56.947+0000 7f6816a00640 1 -- 172.21.15.5:0/241631338 <== mon.2 v2:172.21.15.102:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f68240318f0 con 0x564ce3d0bcb0 2024-09-10T09:47:57.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:56.947+0000 7f6816a00640 1 -- 172.21.15.5:0/241631338 <== mon.2 v2:172.21.15.102:3300/0 5 ==== fsmap(e 35) ==== 8374+0+0 (secure 0 0 0) 0x7f682403cb60 con 0x564ce3d0bcb0 2024-09-10T09:47:57.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:56.947+0000 7f683a5e0740 10 client.27338 fetch_fsmap finished waiting for FSMap version 35 2024-09-10T09:47:57.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:56.947+0000 7f683a5e0740 10 client.27338 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T09:47:57.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:56.947+0000 7f683a5e0740 20 client.27338 populate_metadata read hostname 'smithi005' 2024-09-10T09:47:57.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:56.947+0000 7f683a5e0740 1 --2- 172.21.15.5:0/241631338 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x564ce3d26bd0 0x564ce3d28fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:47:57.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:56.947+0000 7f683a5e0740 4 client.27338 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:47:57.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:56.947+0000 7f683a5e0740 1 -- 172.21.15.5:0/241631338 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564ce3c7f800 con 0x564ce3d26bd0 2024-09-10T09:47:57.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:56.948+0000 7f682fe00640 1 --2- 172.21.15.5:0/241631338 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x564ce3d26bd0 0x564ce3d28fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:47:57.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:56.948+0000 7f6816a00640 10 client.27338 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T09:47:57.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:56.948+0000 7f682fe00640 1 --2- 172.21.15.5:0/241631338 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x564ce3d26bd0 0x564ce3d28fc0 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4455f7e3beb5a74 server_cookie=3f9ac37394c1a013 in_seq=0 out_seq=0 2024-09-10T09:47:57.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:56.949+0000 7f6816a00640 1 -- 172.21.15.5:0/241631338 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564ce3c7f800 con 0x564ce3d26bd0 2024-09-10T09:47:57.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:56.949+0000 7f6816a00640 10 client.27338 handle_command_reply: tid=0 2024-09-10T09:47:57.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:56.949+0000 7f683a5e0740 1 client.27338 shutdown 2024-09-10T09:47:57.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:56.949+0000 7f683a5e0740 1 -- 172.21.15.5:0/241631338 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564ce3d0bcb0 msgr2=0x564ce3d16690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:47:57.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:56.949+0000 7f683a5e0740 1 --2- 172.21.15.5:0/241631338 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564ce3d0bcb0 0x564ce3d16690 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f6824009a50 tx=0x7f68240314e0 comp rx=0 tx=0).stop 2024-09-10T09:47:57.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:56.949+0000 7f683a5e0740 1 -- 172.21.15.5:0/241631338 shutdown_connections 2024-09-10T09:47:57.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:56.949+0000 7f683a5e0740 1 --2- 172.21.15.5:0/241631338 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x564ce3d26bd0 0x564ce3d28fc0 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:57.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:56.950+0000 7f683a5e0740 1 --2- 172.21.15.5:0/241631338 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564ce3d16e90 0x564ce3d13e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:57.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:56.950+0000 7f683a5e0740 1 --2- 172.21.15.5:0/241631338 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564ce3d0c5d0 0x564ce3d13940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:57.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:56.950+0000 7f683a5e0740 1 --2- 172.21.15.5:0/241631338 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564ce3d0bcb0 0x564ce3d16690 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:57.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:56.950+0000 7f6816a00640 0 client.27338 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T09:47:57.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:56.950+0000 7f683a5e0740 1 -- 172.21.15.5:0/241631338 >> 172.21.15.5:0/241631338 conn(0x564ce3d08240 msgr2=0x564ce3d0df20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:47:57.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:56.950+0000 7f683a5e0740 1 -- 172.21.15.5:0/241631338 shutdown_connections 2024-09-10T09:47:57.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:56.950+0000 7f683a5e0740 1 -- 172.21.15.5:0/241631338 wait complete. 2024-09-10T09:47:57.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:56.950+0000 7f683a5e0740 20 client.27338 trim_cache size 0 max 16384 2024-09-10T09:47:57.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:56.970+0000 7f683a5e0740 1 Processor -- start 2024-09-10T09:47:57.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:56.970+0000 7f683a5e0740 1 -- start start 2024-09-10T09:47:57.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:56.970+0000 7f683a5e0740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564ce3d0bcb0 0x564ce3d07160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:47:57.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:56.970+0000 7f683a5e0740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564ce3d0c5d0 0x564ce3d161b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:47:57.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:56.970+0000 7f683a5e0740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564ce3d16e90 0x564ce3d2e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:47:57.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:56.970+0000 7f683a5e0740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f68301b52e0 con 0x564ce3d16e90 2024-09-10T09:47:57.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:56.970+0000 7f683a5e0740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f68301b6e50 con 0x564ce3d0c5d0 2024-09-10T09:47:57.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:56.970+0000 7f683a5e0740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f68301b59b0 con 0x564ce3d0bcb0 2024-09-10T09:47:57.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:56.971+0000 7f6835000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564ce3d0bcb0 0x564ce3d07160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:47:57.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:56.971+0000 7f682fe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564ce3d0c5d0 0x564ce3d161b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:47:57.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:56.971+0000 7f6835000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564ce3d0bcb0 0x564ce3d07160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:57816/0 (socket says 172.21.15.5:57816) 2024-09-10T09:47:57.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:56.971+0000 7f6835000640 1 -- 172.21.15.5:0/2676412431 learned_addr learned my addr 172.21.15.5:0/2676412431 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:47:57.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:56.971+0000 7f6835a00640 1 --2- 172.21.15.5:0/2676412431 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564ce3d16e90 0x564ce3d2e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:47:57.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:56.971+0000 7f6835000640 1 -- 172.21.15.5:0/2676412431 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564ce3d0c5d0 msgr2=0x564ce3d161b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:47:57.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:56.971+0000 7f6835000640 1 --2- 172.21.15.5:0/2676412431 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564ce3d0c5d0 0x564ce3d161b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:57.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:56.971+0000 7f6835000640 1 -- 172.21.15.5:0/2676412431 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564ce3d16e90 msgr2=0x564ce3d2e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:47:57.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:56.971+0000 7f6835000640 1 --2- 172.21.15.5:0/2676412431 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564ce3d16e90 0x564ce3d2e5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:57.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:56.971+0000 7f6835000640 1 -- 172.21.15.5:0/2676412431 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564ce3d128f0 con 0x564ce3d0bcb0 2024-09-10T09:47:57.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:56.971+0000 7f6835000640 1 --2- 172.21.15.5:0/2676412431 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564ce3d0bcb0 0x564ce3d07160 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f682403f040 tx=0x7f6824001e50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ac8866c0ac04f34f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:47:57.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:56.972+0000 7f6821600640 1 -- 172.21.15.5:0/2676412431 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f682404c040 con 0x564ce3d0bcb0 2024-09-10T09:47:57.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:56.972+0000 7f6821600640 1 -- 172.21.15.5:0/2676412431 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6824044e50 con 0x564ce3d0bcb0 2024-09-10T09:47:57.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:56.972+0000 7f6821600640 1 -- 172.21.15.5:0/2676412431 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6824040070 con 0x564ce3d0bcb0 2024-09-10T09:47:57.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:56.973+0000 7f683a5e0740 1 -- 172.21.15.5:0/2676412431 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564ce3d0bcb0 msgr2=0x564ce3d07160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:47:57.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:56.973+0000 7f683a5e0740 1 --2- 172.21.15.5:0/2676412431 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564ce3d0bcb0 0x564ce3d07160 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f682403f040 tx=0x7f6824001e50 comp rx=0 tx=0).stop 2024-09-10T09:47:57.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:56.973+0000 7f683a5e0740 1 -- 172.21.15.5:0/2676412431 shutdown_connections 2024-09-10T09:47:57.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:56.973+0000 7f683a5e0740 1 --2- 172.21.15.5:0/2676412431 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564ce3d16e90 0x564ce3d2e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:57.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:56.973+0000 7f683a5e0740 1 --2- 172.21.15.5:0/2676412431 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564ce3d0c5d0 0x564ce3d161b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:57.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:56.973+0000 7f683a5e0740 1 --2- 172.21.15.5:0/2676412431 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564ce3d0bcb0 0x564ce3d07160 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:57.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:56.973+0000 7f683a5e0740 1 -- 172.21.15.5:0/2676412431 >> 172.21.15.5:0/2676412431 conn(0x564ce3d08240 msgr2=0x564ce3d06a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:47:57.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:56.973+0000 7f683a5e0740 1 -- 172.21.15.5:0/2676412431 shutdown_connections 2024-09-10T09:47:57.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:56.973+0000 7f683a5e0740 1 -- 172.21.15.5:0/2676412431 wait complete. 2024-09-10T09:47:57.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:56.973+0000 7f683a5e0740 1 -- start start 2024-09-10T09:47:57.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:56.973+0000 7f683a5e0740 1 Processor -- start 2024-09-10T09:47:57.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:56.974+0000 7f683a5e0740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564ce3d0bcb0 0x564ce3d1f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:47:57.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:56.974+0000 7f683a5e0740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564ce3d0c5d0 0x564ce3d1f630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:47:57.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:56.974+0000 7f683a5e0740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564ce3d16e90 0x564ce3d19ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:47:57.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:56.974+0000 7f683a5e0740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f68301b6750 con 0x564ce3d0c5d0 2024-09-10T09:47:57.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:56.974+0000 7f683a5e0740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f68301b6080 con 0x564ce3d16e90 2024-09-10T09:47:57.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:56.974+0000 7f683a5e0740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f68301b32c0 con 0x564ce3d0bcb0 2024-09-10T09:47:57.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:56.974+0000 7f683a5e0740 10 client.0 _ll_register_callbacks cb 0x7f6824009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:47:57.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:56.974+0000 7f6835a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564ce3d16e90 0x564ce3d19ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:47:57.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:56.974+0000 7f6835000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564ce3d0bcb0 0x564ce3d1f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:47:57.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:56.974+0000 7f682fe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564ce3d0c5d0 0x564ce3d1f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:47:57.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:56.974+0000 7f6835a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564ce3d16e90 0x564ce3d19ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:41670/0 (socket says 172.21.15.5:41670) 2024-09-10T09:47:57.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:56.974+0000 7f6835000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564ce3d0bcb0 0x564ce3d1f0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:57824/0 (socket says 172.21.15.5:57824) 2024-09-10T09:47:57.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:56.974+0000 7f6835a00640 1 -- 172.21.15.5:0/4168026523 learned_addr learned my addr 172.21.15.5:0/4168026523 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:47:57.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:56.975+0000 7f6835a00640 1 -- 172.21.15.5:0/4168026523 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564ce3d0bcb0 msgr2=0x564ce3d1f0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:47:57.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:56.975+0000 7f6835a00640 1 --2- 172.21.15.5:0/4168026523 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564ce3d0bcb0 0x564ce3d1f0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:57.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:56.975+0000 7f6835a00640 1 -- 172.21.15.5:0/4168026523 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564ce3d0c5d0 msgr2=0x564ce3d1f630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:47:57.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:56.975+0000 7f6835a00640 1 --2- 172.21.15.5:0/4168026523 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564ce3d0c5d0 0x564ce3d1f630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:57.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:56.975+0000 7f6835a00640 1 -- 172.21.15.5:0/4168026523 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564ce3d1a580 con 0x564ce3d16e90 2024-09-10T09:47:57.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:56.975+0000 7f6835a00640 1 --2- 172.21.15.5:0/4168026523 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564ce3d16e90 0x564ce3d19ec0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f6828098320 tx=0x7f682809c680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=256ec5fe0156df1e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:47:57.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:56.975+0000 7f6816a00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-09-10T09:47:57.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:56.975+0000 7f6816a00640 1 -- 172.21.15.5:0/4168026523 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f682809c120 con 0x564ce3d16e90 2024-09-10T09:47:57.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:56.975+0000 7f6816a00640 1 -- 172.21.15.5:0/4168026523 <== mon.2 v2:172.21.15.102:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f682809c2c0 con 0x564ce3d16e90 2024-09-10T09:47:57.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:56.975+0000 7f683a5e0740 1 -- 172.21.15.5:0/4168026523 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6824044e50 con 0x564ce3d16e90 2024-09-10T09:47:57.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:56.975+0000 7f6816a00640 1 -- 172.21.15.5:0/4168026523 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f68280a46a0 con 0x564ce3d16e90 2024-09-10T09:47:57.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:56.976+0000 7f683a5e0740 10 client.27344 fetch_fsmap learned FSMap version 35 2024-09-10T09:47:57.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:56.976+0000 7f683a5e0740 1 -- 172.21.15.5:0/4168026523 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x564ce3d92000 con 0x564ce3d16e90 2024-09-10T09:47:57.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:56.976+0000 7f6816a00640 1 -- 172.21.15.5:0/4168026523 <== mon.2 v2:172.21.15.102:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6824044e50 con 0x564ce3d16e90 2024-09-10T09:47:57.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:56.976+0000 7f6816a00640 1 -- 172.21.15.5:0/4168026523 <== mon.2 v2:172.21.15.102:3300/0 5 ==== fsmap(e 35) ==== 8374+0+0 (secure 0 0 0) 0x7f6828097c10 con 0x564ce3d16e90 2024-09-10T09:47:57.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:56.976+0000 7f683a5e0740 10 client.27344 fetch_fsmap finished waiting for FSMap version 35 2024-09-10T09:47:57.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:56.976+0000 7f683a5e0740 10 client.27344 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T09:47:57.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:56.976+0000 7f683a5e0740 20 client.27344 populate_metadata read hostname 'smithi005' 2024-09-10T09:47:57.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:56.976+0000 7f683a5e0740 1 --2- 172.21.15.5:0/4168026523 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x564ce3d92750 0x564ce3d94b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:47:57.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:56.976+0000 7f683a5e0740 4 client.27344 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-10T09:47:57.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:56.976+0000 7f683a5e0740 1 -- 172.21.15.5:0/4168026523 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x564ce3c79d10 con 0x564ce3d92750 2024-09-10T09:47:57.232 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-10T09:47:57.232 INFO:teuthology.orchestra.run.smithi005.stdout: "return_code": 0, 2024-09-10T09:47:57.233 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_tag": "b9c7cea4-50b1-4f3b-a7fa-88cc09260e91", 2024-09-10T09:47:57.233 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": "asynchronous" 2024-09-10T09:47:57.233 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-10T09:47:57.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:56.976+0000 7f6835000640 1 --2- 172.21.15.5:0/4168026523 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x564ce3d92750 0x564ce3d94b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:47:57.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:56.977+0000 7f6835000640 1 --2- 172.21.15.5:0/4168026523 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x564ce3d92750 0x564ce3d94b70 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7e75b246484165d7 server_cookie=ee34a1e4ca12c1ac in_seq=0 out_seq=0 2024-09-10T09:47:57.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:56.977+0000 7f6816a00640 10 client.27344 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T09:47:57.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:56.978+0000 7f6816a00640 1 -- 172.21.15.5:0/4168026523 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x564ce3c79d10 con 0x564ce3d92750 2024-09-10T09:47:57.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:56.978+0000 7f6816a00640 10 client.27344 handle_command_reply: tid=0 2024-09-10T09:47:57.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:56.978+0000 7f683a5e0740 1 client.27344 shutdown 2024-09-10T09:47:57.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:56.978+0000 7f683a5e0740 1 -- 172.21.15.5:0/4168026523 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564ce3d16e90 msgr2=0x564ce3d19ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:47:57.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:56.978+0000 7f683a5e0740 1 --2- 172.21.15.5:0/4168026523 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564ce3d16e90 0x564ce3d19ec0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f6828098320 tx=0x7f682809c680 comp rx=0 tx=0).stop 2024-09-10T09:47:57.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:56.978+0000 7f683a5e0740 1 -- 172.21.15.5:0/4168026523 shutdown_connections 2024-09-10T09:47:57.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:56.978+0000 7f683a5e0740 1 --2- 172.21.15.5:0/4168026523 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x564ce3d92750 0x564ce3d94b70 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:57.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:56.978+0000 7f683a5e0740 1 --2- 172.21.15.5:0/4168026523 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564ce3d16e90 0x564ce3d19ec0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:57.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:56.978+0000 7f683a5e0740 1 --2- 172.21.15.5:0/4168026523 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564ce3d0c5d0 0x564ce3d1f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:57.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:56.978+0000 7f683a5e0740 1 --2- 172.21.15.5:0/4168026523 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564ce3d0bcb0 0x564ce3d1f0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:57.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:56.978+0000 7f6816a00640 0 client.27344 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T09:47:57.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:56.978+0000 7f683a5e0740 1 -- 172.21.15.5:0/4168026523 >> 172.21.15.5:0/4168026523 conn(0x564ce3d08240 msgr2=0x564ce3d21d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:47:57.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:56.979+0000 7f683a5e0740 1 -- 172.21.15.5:0/4168026523 shutdown_connections 2024-09-10T09:47:57.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:56.979+0000 7f683a5e0740 1 -- 172.21.15.5:0/4168026523 wait complete. 2024-09-10T09:47:57.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:56.979+0000 7f683a5e0740 20 client.27344 trim_cache size 0 max 16384 2024-09-10T09:47:57.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:56.980+0000 7f6836400640 1 -- 172.21.15.5:0/4115034965 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f68180a2b80 msgr2=0x7f68180a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:47:57.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:56.980+0000 7f6836400640 1 --2- 172.21.15.5:0/4115034965 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f68180a2b80 0x7f68180a5040 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x564ce3d095c0 tx=0x7f681c005f90 comp rx=0 tx=0).stop 2024-09-10T09:47:57.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:56.980+0000 7f6836400640 1 -- 172.21.15.5:0/4115034965 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f683010e060 msgr2=0x7f683019e6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:47:57.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:56.980+0000 7f6836400640 1 --2- 172.21.15.5:0/4115034965 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f683010e060 0x7f683019e6d0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f6828005fd0 tx=0x7f682800ee60 comp rx=0 tx=0).stop 2024-09-10T09:47:57.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:56.980+0000 7f6836400640 1 -- 172.21.15.5:0/4115034965 shutdown_connections 2024-09-10T09:47:57.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:56.980+0000 7f6836400640 1 --2- 172.21.15.5:0/4115034965 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f68180a2b80 0x7f68180a5040 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:57.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:56.980+0000 7f6836400640 1 --2- 172.21.15.5:0/4115034965 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f683010e060 0x7f683019e6d0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:57.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:56.980+0000 7f6836400640 1 --2- 172.21.15.5:0/4115034965 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6830108980 0x7f68301a4b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:57.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:56.980+0000 7f6836400640 1 --2- 172.21.15.5:0/4115034965 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6830108020 0x7f68301a4600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:57.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:56.980+0000 7f6836400640 1 -- 172.21.15.5:0/4115034965 >> 172.21.15.5:0/4115034965 conn(0x7f6830077980 msgr2=0x7f6830079270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:47:57.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:56.980+0000 7f6836400640 1 -- 172.21.15.5:0/4115034965 shutdown_connections 2024-09-10T09:47:57.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:56.981+0000 7f6836400640 1 -- 172.21.15.5:0/4115034965 wait complete. 2024-09-10T09:47:57.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:47:57 smithi005 ceph-mon[30350]: pgmap v681: 129 pgs: 129 active+clean; 214 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 650 KiB/s rd, 1.9 MiB/s wr, 113 op/s 2024-09-10T09:47:57.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:47:57 smithi005 ceph-mon[30350]: scrub queued for path: / 2024-09-10T09:47:57.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:47:57 smithi005 ceph-mon[30350]: scrub summary: idle+waiting paths [/] 2024-09-10T09:47:57.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:47:57 smithi005 ceph-mon[30350]: scrub summary: active paths [/] 2024-09-10T09:47:57.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:47:57 smithi005 ceph-mon[30350]: scrub summary: idle+waiting paths [/] 2024-09-10T09:47:57.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:47:57 smithi159 ceph-mon[35513]: pgmap v681: 129 pgs: 129 active+clean; 214 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 650 KiB/s rd, 1.9 MiB/s wr, 113 op/s 2024-09-10T09:47:57.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:47:57 smithi159 ceph-mon[35513]: scrub queued for path: / 2024-09-10T09:47:57.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:47:57 smithi159 ceph-mon[35513]: scrub summary: idle+waiting paths [/] 2024-09-10T09:47:57.601 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:47:57 smithi159 ceph-mon[35513]: scrub summary: active paths [/] 2024-09-10T09:47:57.601 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:47:57 smithi159 ceph-mon[35513]: scrub summary: idle+waiting paths [/] 2024-09-10T09:47:57.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:47:57 smithi102 ceph-mon[35442]: pgmap v681: 129 pgs: 129 active+clean; 214 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 650 KiB/s rd, 1.9 MiB/s wr, 113 op/s 2024-09-10T09:47:57.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:47:57 smithi102 ceph-mon[35442]: scrub queued for path: / 2024-09-10T09:47:57.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:47:57 smithi102 ceph-mon[35442]: scrub summary: idle+waiting paths [/] 2024-09-10T09:47:57.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:47:57 smithi102 ceph-mon[35442]: scrub summary: active paths [/] 2024-09-10T09:47:57.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:47:57 smithi102 ceph-mon[35442]: scrub summary: idle+waiting paths [/] 2024-09-10T09:47:58.098 INFO:tasks.workunit.client.0.smithi005.stdout: 1 565585 5.99 MB/sec execute 339 sec latency 25.924 ms 2024-09-10T09:47:58.100 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T09:47:58.503 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:47:58.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:47:58 smithi159 ceph-mon[35513]: scrub summary: active paths [/] 2024-09-10T09:47:58.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:47:58 smithi159 ceph-mon[35513]: scrub summary: idle+waiting paths [/] 2024-09-10T09:47:58.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:47:58 smithi102 ceph-mon[35442]: scrub summary: active paths [/] 2024-09-10T09:47:58.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:47:58 smithi102 ceph-mon[35442]: scrub summary: idle+waiting paths [/] 2024-09-10T09:47:58.646 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:47:58 smithi005 ceph-mon[30350]: scrub summary: active paths [/] 2024-09-10T09:47:58.646 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:47:58 smithi005 ceph-mon[30350]: scrub summary: idle+waiting paths [/] 2024-09-10T09:47:59.098 INFO:tasks.workunit.client.0.smithi005.stdout: 1 568294 6.02 MB/sec execute 340 sec latency 19.452 ms 2024-09-10T09:47:59.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:59.099+0000 7f364e600640 1 -- 172.21.15.5:0/857778095 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f36481027a0 msgr2=0x7f36481067f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:47:59.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:59.099+0000 7f364e600640 1 --2- 172.21.15.5:0/857778095 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f36481027a0 0x7f36481067f0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f3634009900 tx=0x7f363402a400 comp rx=0 tx=0).stop 2024-09-10T09:47:59.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:59.099+0000 7f364e600640 1 -- 172.21.15.5:0/857778095 shutdown_connections 2024-09-10T09:47:59.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:59.099+0000 7f364e600640 1 --2- 172.21.15.5:0/857778095 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3648106f20 0x7f364810eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:59.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:59.099+0000 7f364e600640 1 --2- 172.21.15.5:0/857778095 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f36481027a0 0x7f36481067f0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:59.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:59.099+0000 7f364e600640 1 --2- 172.21.15.5:0/857778095 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3648101df0 0x7f36481021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:59.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:59.099+0000 7f364e600640 1 -- 172.21.15.5:0/857778095 >> 172.21.15.5:0/857778095 conn(0x7f36480fd8b0 msgr2=0x7f36480ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:47:59.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:59.099+0000 7f364e600640 1 -- 172.21.15.5:0/857778095 shutdown_connections 2024-09-10T09:47:59.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:59.099+0000 7f364e600640 1 -- 172.21.15.5:0/857778095 wait complete. 2024-09-10T09:47:59.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:59.100+0000 7f364e600640 1 Processor -- start 2024-09-10T09:47:59.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:59.100+0000 7f364e600640 1 -- start start 2024-09-10T09:47:59.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:59.100+0000 7f364e600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3648101df0 0x7f36480744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:47:59.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:59.100+0000 7f364e600640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f36481027a0 0x7f3648074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:47:59.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:59.100+0000 7f364e600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3648106f20 0x7f3648074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:47:59.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:59.100+0000 7f364e600640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f3648111fd0 con 0x7f3648106f20 2024-09-10T09:47:59.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:59.100+0000 7f364e600640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f3648111e50 con 0x7f3648101df0 2024-09-10T09:47:59.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:59.100+0000 7f364e600640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f3648112150 con 0x7f36481027a0 2024-09-10T09:47:59.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:59.101+0000 7f3646a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f36481027a0 0x7f3648074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:47:59.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:59.101+0000 7f3647400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3648101df0 0x7f36480744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:47:59.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:59.101+0000 7f3646a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f36481027a0 0x7f3648074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:57844/0 (socket says 172.21.15.5:57844) 2024-09-10T09:47:59.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:59.101+0000 7f3646a00640 1 -- 172.21.15.5:0/409993515 learned_addr learned my addr 172.21.15.5:0/409993515 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:47:59.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:59.101+0000 7f3647e00640 1 --2- 172.21.15.5:0/409993515 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3648106f20 0x7f3648074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:47:59.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:59.101+0000 7f3646a00640 1 -- 172.21.15.5:0/409993515 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3648101df0 msgr2=0x7f36480744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:47:59.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:59.101+0000 7f3646a00640 1 --2- 172.21.15.5:0/409993515 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3648101df0 0x7f36480744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:59.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:59.101+0000 7f3646a00640 1 -- 172.21.15.5:0/409993515 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3648106f20 msgr2=0x7f3648074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:47:59.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:59.101+0000 7f3646a00640 1 --2- 172.21.15.5:0/409993515 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3648106f20 0x7f3648074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:59.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:59.101+0000 7f3646a00640 1 -- 172.21.15.5:0/409993515 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36481a6910 con 0x7f36481027a0 2024-09-10T09:47:59.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:59.101+0000 7f3647e00640 1 --2- 172.21.15.5:0/409993515 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3648106f20 0x7f3648074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:47:59.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:59.101+0000 7f3646a00640 1 --2- 172.21.15.5:0/409993515 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f36481027a0 0x7f3648074a30 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f3634004770 tx=0x7f3634003c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:47:59.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:59.102+0000 7f363fe00640 1 -- 172.21.15.5:0/409993515 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f363403f070 con 0x7f36481027a0 2024-09-10T09:47:59.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:59.102+0000 7f363fe00640 1 -- 172.21.15.5:0/409993515 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3634031ca0 con 0x7f36481027a0 2024-09-10T09:47:59.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:59.102+0000 7f363fe00640 1 -- 172.21.15.5:0/409993515 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3634035030 con 0x7f36481027a0 2024-09-10T09:47:59.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:59.102+0000 7f364e600640 1 -- 172.21.15.5:0/409993515 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36481a6ba0 con 0x7f36481027a0 2024-09-10T09:47:59.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:59.102+0000 7f364e600640 1 -- 172.21.15.5:0/409993515 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36481a7030 con 0x7f36481027a0 2024-09-10T09:47:59.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:59.105+0000 7f363fe00640 1 -- 172.21.15.5:0/409993515 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f3634042070 con 0x7f36481027a0 2024-09-10T09:47:59.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:59.105+0000 7f3650fe0740 1 Processor -- start 2024-09-10T09:47:59.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:59.106+0000 7f3650fe0740 1 -- start start 2024-09-10T09:47:59.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:59.106+0000 7f3650fe0740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55eee1f0eb90 0x55eee1f0ef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:47:59.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:59.106+0000 7f3650fe0740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eee1f0f4b0 0x55eee1f19830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:47:59.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:59.106+0000 7f3650fe0740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55eee1f19d70 0x55eee1f1c160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:47:59.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:59.106+0000 7f3650fe0740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55eee1e753e0 con 0x55eee1f0f4b0 2024-09-10T09:47:59.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:59.106+0000 7f3650fe0740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55eee1e1cd80 con 0x55eee1f19d70 2024-09-10T09:47:59.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:59.106+0000 7f3650fe0740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55eee1e713a0 con 0x55eee1f0eb90 2024-09-10T09:47:59.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:59.106+0000 7f3647e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55eee1f0eb90 0x55eee1f0ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:47:59.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:59.106+0000 7f3647e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55eee1f0eb90 0x55eee1f0ef70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:57852/0 (socket says 172.21.15.5:57852) 2024-09-10T09:47:59.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:59.106+0000 7f3647e00640 1 -- 172.21.15.5:0/2828189846 learned_addr learned my addr 172.21.15.5:0/2828189846 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:47:59.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:59.107+0000 7f3646a00640 1 --2- 172.21.15.5:0/2828189846 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55eee1f19d70 0x55eee1f1c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:47:59.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:59.107+0000 7f3647400640 1 --2- 172.21.15.5:0/2828189846 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eee1f0f4b0 0x55eee1f19830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:47:59.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:59.107+0000 7f3647e00640 1 -- 172.21.15.5:0/2828189846 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55eee1f19d70 msgr2=0x55eee1f1c160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:47:59.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:59.107+0000 7f3647e00640 1 --2- 172.21.15.5:0/2828189846 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55eee1f19d70 0x55eee1f1c160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:59.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:59.107+0000 7f3647e00640 1 -- 172.21.15.5:0/2828189846 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eee1f0f4b0 msgr2=0x55eee1f19830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:47:59.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:59.107+0000 7f3647e00640 1 --2- 172.21.15.5:0/2828189846 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eee1f0f4b0 0x55eee1f19830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:59.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:59.107+0000 7f3647e00640 1 -- 172.21.15.5:0/2828189846 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eee1e3bae0 con 0x55eee1f0eb90 2024-09-10T09:47:59.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:59.107+0000 7f3646a00640 1 --2- 172.21.15.5:0/2828189846 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55eee1f19d70 0x55eee1f1c160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:47:59.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:59.107+0000 7f3647e00640 1 --2- 172.21.15.5:0/2828189846 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55eee1f0eb90 0x55eee1f0ef70 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f3640009fd0 tx=0x7f3640010860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9c6bdf20ea5ece88 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:47:59.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:59.107+0000 7f363fe00640 1 --2- 172.21.15.5:0/409993515 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f362c0a2c00 0x7f362c0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:47:59.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:59.107+0000 7f363cc00640 1 -- 172.21.15.5:0/2828189846 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3640011700 con 0x55eee1f0eb90 2024-09-10T09:47:59.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:59.107+0000 7f363cc00640 1 -- 172.21.15.5:0/2828189846 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3640011d80 con 0x55eee1f0eb90 2024-09-10T09:47:59.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:59.108+0000 7f363cc00640 1 -- 172.21.15.5:0/2828189846 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f364000e3c0 con 0x55eee1f0eb90 2024-09-10T09:47:59.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:59.108+0000 7f363fe00640 1 -- 172.21.15.5:0/409993515 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f36340efa50 con 0x7f36481027a0 2024-09-10T09:47:59.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:59.108+0000 7f3647400640 1 --2- 172.21.15.5:0/409993515 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f362c0a2c00 0x7f362c0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:47:59.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:59.108+0000 7f3650fe0740 1 -- 172.21.15.5:0/2828189846 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55eee1f0eb90 msgr2=0x55eee1f0ef70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:47:59.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:59.108+0000 7f3650fe0740 1 --2- 172.21.15.5:0/2828189846 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55eee1f0eb90 0x55eee1f0ef70 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f3640009fd0 tx=0x7f3640010860 comp rx=0 tx=0).stop 2024-09-10T09:47:59.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:59.108+0000 7f3650fe0740 1 -- 172.21.15.5:0/2828189846 shutdown_connections 2024-09-10T09:47:59.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:59.108+0000 7f3650fe0740 1 --2- 172.21.15.5:0/2828189846 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55eee1f19d70 0x55eee1f1c160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:59.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:59.108+0000 7f3650fe0740 1 --2- 172.21.15.5:0/2828189846 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eee1f0f4b0 0x55eee1f19830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:59.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:59.108+0000 7f3650fe0740 1 --2- 172.21.15.5:0/2828189846 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55eee1f0eb90 0x55eee1f0ef70 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:59.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:59.108+0000 7f3650fe0740 1 -- 172.21.15.5:0/2828189846 >> 172.21.15.5:0/2828189846 conn(0x55eee1f0b120 msgr2=0x55eee1f0b500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:47:59.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:59.109+0000 7f3647400640 1 --2- 172.21.15.5:0/409993515 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f362c0a2c00 0x7f362c0a50c0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x55eee1f0bf80 tx=0x7f3638005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:47:59.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:59.109+0000 7f3650fe0740 1 -- 172.21.15.5:0/2828189846 shutdown_connections 2024-09-10T09:47:59.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:59.109+0000 7f3650fe0740 1 -- 172.21.15.5:0/2828189846 wait complete. 2024-09-10T09:47:59.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:59.109+0000 7f3650fe0740 1 -- start start 2024-09-10T09:47:59.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:59.109+0000 7f3650fe0740 1 Processor -- start 2024-09-10T09:47:59.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:59.110+0000 7f3650fe0740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55eee1f0eb90 0x55eee1f16d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:47:59.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:59.110+0000 7f3650fe0740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55eee1f0f4b0 0x55eee1f17280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:47:59.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:59.110+0000 7f3650fe0740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eee1f19d70 0x55eee1f177c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:47:59.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:59.110+0000 7f3650fe0740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55eee1f1c9e0 con 0x55eee1f19d70 2024-09-10T09:47:59.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:59.110+0000 7f3650fe0740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55eee1e300f0 con 0x55eee1f0f4b0 2024-09-10T09:47:59.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:59.110+0000 7f3650fe0740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55eee1f1cb60 con 0x55eee1f0eb90 2024-09-10T09:47:59.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:59.110+0000 7f3650fe0740 10 client.0 _ll_register_callbacks cb 0x55eee1e219e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:47:59.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:59.110+0000 7f3647400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55eee1f0eb90 0x55eee1f16d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:47:59.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:59.110+0000 7f3647400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55eee1f0eb90 0x55eee1f16d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:57864/0 (socket says 172.21.15.5:57864) 2024-09-10T09:47:59.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:59.110+0000 7f3647400640 1 -- 172.21.15.5:0/605417461 learned_addr learned my addr 172.21.15.5:0/605417461 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:47:59.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:59.110+0000 7f3647e00640 1 --2- 172.21.15.5:0/605417461 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eee1f19d70 0x55eee1f177c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:47:59.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:59.110+0000 7f3646a00640 1 --2- 172.21.15.5:0/605417461 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55eee1f0f4b0 0x55eee1f17280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:47:59.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:59.111+0000 7f3647400640 1 -- 172.21.15.5:0/605417461 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55eee1f0f4b0 msgr2=0x55eee1f17280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:47:59.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:59.111+0000 7f3647400640 1 --2- 172.21.15.5:0/605417461 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55eee1f0f4b0 0x55eee1f17280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:59.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:59.111+0000 7f3647400640 1 -- 172.21.15.5:0/605417461 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eee1f19d70 msgr2=0x55eee1f177c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:47:59.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:59.111+0000 7f3647400640 1 --2- 172.21.15.5:0/605417461 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eee1f19d70 0x55eee1f177c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:59.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:59.111+0000 7f3647400640 1 -- 172.21.15.5:0/605417461 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eee1be63c0 con 0x55eee1f0eb90 2024-09-10T09:47:59.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:59.111+0000 7f3646a00640 1 --2- 172.21.15.5:0/605417461 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55eee1f0f4b0 0x55eee1f17280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:47:59.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:59.111+0000 7f3647e00640 1 --2- 172.21.15.5:0/605417461 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eee1f19d70 0x55eee1f177c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:47:59.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:59.111+0000 7f3647400640 1 --2- 172.21.15.5:0/605417461 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55eee1f0eb90 0x55eee1f16d40 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f363801e9b0 tx=0x7f3638004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f5ee0405e2e00a22 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:47:59.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:59.111+0000 7f3631600640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-09-10T09:47:59.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:59.112+0000 7f3631600640 1 -- 172.21.15.5:0/605417461 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f363802e060 con 0x55eee1f0eb90 2024-09-10T09:47:59.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:59.112+0000 7f3631600640 1 -- 172.21.15.5:0/605417461 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f3638018030 con 0x55eee1f0eb90 2024-09-10T09:47:59.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:59.112+0000 7f3650fe0740 1 -- 172.21.15.5:0/605417461 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3640011d80 con 0x55eee1f0eb90 2024-09-10T09:47:59.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:59.112+0000 7f3631600640 1 -- 172.21.15.5:0/605417461 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f3638035050 con 0x55eee1f0eb90 2024-09-10T09:47:59.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:59.112+0000 7f3650fe0740 10 client.27385 fetch_fsmap learned FSMap version 35 2024-09-10T09:47:59.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:59.112+0000 7f3650fe0740 1 -- 172.21.15.5:0/605417461 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55eee1f287c0 con 0x55eee1f0eb90 2024-09-10T09:47:59.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:59.113+0000 7f3631600640 1 -- 172.21.15.5:0/605417461 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3640011d80 con 0x55eee1f0eb90 2024-09-10T09:47:59.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:59.113+0000 7f3631600640 1 -- 172.21.15.5:0/605417461 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 35) ==== 8374+0+0 (secure 0 0 0) 0x7f363801c4c0 con 0x55eee1f0eb90 2024-09-10T09:47:59.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:59.113+0000 7f3650fe0740 10 client.27385 fetch_fsmap finished waiting for FSMap version 35 2024-09-10T09:47:59.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:59.113+0000 7f3650fe0740 10 client.27385 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T09:47:59.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:59.113+0000 7f3650fe0740 20 client.27385 populate_metadata read hostname 'smithi005' 2024-09-10T09:47:59.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:59.113+0000 7f3650fe0740 1 --2- 172.21.15.5:0/605417461 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55eee1f2a050 0x55eee1f2c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:47:59.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:59.113+0000 7f3650fe0740 4 client.27385 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:47:59.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:59.113+0000 7f3650fe0740 1 -- 172.21.15.5:0/605417461 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55eee1e82950 con 0x55eee1f2a050 2024-09-10T09:47:59.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:59.114+0000 7f3646a00640 1 --2- 172.21.15.5:0/605417461 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55eee1f2a050 0x55eee1f2c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:47:59.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:59.114+0000 7f3646a00640 1 --2- 172.21.15.5:0/605417461 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55eee1f2a050 0x55eee1f2c4a0 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=358928cf2988730c server_cookie=6a05981e82774950 in_seq=0 out_seq=0 2024-09-10T09:47:59.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:59.114+0000 7f3631600640 10 client.27385 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T09:47:59.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:59.115+0000 7f3631600640 1 -- 172.21.15.5:0/605417461 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55eee1e82950 con 0x55eee1f2a050 2024-09-10T09:47:59.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:59.115+0000 7f3631600640 10 client.27385 handle_command_reply: tid=0 2024-09-10T09:47:59.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:59.115+0000 7f3650fe0740 1 client.27385 shutdown 2024-09-10T09:47:59.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:59.115+0000 7f3650fe0740 1 -- 172.21.15.5:0/605417461 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55eee1f0eb90 msgr2=0x55eee1f16d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:47:59.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:59.115+0000 7f3650fe0740 1 --2- 172.21.15.5:0/605417461 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55eee1f0eb90 0x55eee1f16d40 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f363801e9b0 tx=0x7f3638004730 comp rx=0 tx=0).stop 2024-09-10T09:47:59.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:59.116+0000 7f3650fe0740 1 -- 172.21.15.5:0/605417461 shutdown_connections 2024-09-10T09:47:59.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:59.116+0000 7f3650fe0740 1 --2- 172.21.15.5:0/605417461 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55eee1f2a050 0x55eee1f2c4a0 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:59.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:59.116+0000 7f3631600640 0 client.27385 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T09:47:59.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:59.116+0000 7f3650fe0740 1 --2- 172.21.15.5:0/605417461 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eee1f19d70 0x55eee1f177c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:59.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:59.116+0000 7f3650fe0740 1 --2- 172.21.15.5:0/605417461 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55eee1f0f4b0 0x55eee1f17280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:59.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:59.116+0000 7f3650fe0740 1 --2- 172.21.15.5:0/605417461 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55eee1f0eb90 0x55eee1f16d40 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:59.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:59.116+0000 7f3650fe0740 1 -- 172.21.15.5:0/605417461 >> 172.21.15.5:0/605417461 conn(0x55eee1f0b120 msgr2=0x55eee1f10e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:47:59.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:59.116+0000 7f3650fe0740 1 -- 172.21.15.5:0/605417461 shutdown_connections 2024-09-10T09:47:59.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:59.116+0000 7f3650fe0740 1 -- 172.21.15.5:0/605417461 wait complete. 2024-09-10T09:47:59.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:59.117+0000 7f3650fe0740 20 client.27385 trim_cache size 0 max 16384 2024-09-10T09:47:59.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:59.137+0000 7f3650fe0740 1 Processor -- start 2024-09-10T09:47:59.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:59.137+0000 7f3650fe0740 1 -- start start 2024-09-10T09:47:59.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:59.138+0000 7f3650fe0740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55eee1f0eb90 0x55eee1f17410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:47:59.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:59.138+0000 7f3650fe0740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eee1f0f4b0 0x55eee1f31db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:47:59.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:59.138+0000 7f3650fe0740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55eee1f19d70 0x55eee1f322f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:47:59.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:59.138+0000 7f3650fe0740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f36481b1030 con 0x55eee1f0f4b0 2024-09-10T09:47:59.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:59.138+0000 7f3650fe0740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f36481b2ba0 con 0x55eee1f0eb90 2024-09-10T09:47:59.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:59.138+0000 7f3650fe0740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f36481b1700 con 0x55eee1f19d70 2024-09-10T09:47:59.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:59.138+0000 7f3647400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55eee1f0eb90 0x55eee1f17410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:47:59.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:59.138+0000 7f3646a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eee1f0f4b0 0x55eee1f31db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:47:59.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:59.138+0000 7f3647400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55eee1f0eb90 0x55eee1f17410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:41700/0 (socket says 172.21.15.5:41700) 2024-09-10T09:47:59.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:59.138+0000 7f3647400640 1 -- 172.21.15.5:0/2107661308 learned_addr learned my addr 172.21.15.5:0/2107661308 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:47:59.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:59.138+0000 7f3647e00640 1 --2- 172.21.15.5:0/2107661308 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55eee1f19d70 0x55eee1f322f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:47:59.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:59.139+0000 7f3647400640 1 -- 172.21.15.5:0/2107661308 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55eee1f19d70 msgr2=0x55eee1f322f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:47:59.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:59.139+0000 7f3647400640 1 --2- 172.21.15.5:0/2107661308 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55eee1f19d70 0x55eee1f322f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:59.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:59.139+0000 7f3647400640 1 -- 172.21.15.5:0/2107661308 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eee1f0f4b0 msgr2=0x55eee1f31db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:47:59.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:59.139+0000 7f3647400640 1 --2- 172.21.15.5:0/2107661308 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eee1f0f4b0 0x55eee1f31db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:59.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:59.139+0000 7f3647400640 1 -- 172.21.15.5:0/2107661308 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eee1f0f2e0 con 0x55eee1f0eb90 2024-09-10T09:47:59.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:59.139+0000 7f3647400640 1 --2- 172.21.15.5:0/2107661308 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55eee1f0eb90 0x55eee1f17410 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f3638012d50 tx=0x7f3638005d70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a183305bd1ad7fcf server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:47:59.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:59.139+0000 7f3633e00640 1 -- 172.21.15.5:0/2107661308 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f363801cb00 con 0x55eee1f0eb90 2024-09-10T09:47:59.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:59.139+0000 7f3633e00640 1 -- 172.21.15.5:0/2107661308 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f363801cca0 con 0x55eee1f0eb90 2024-09-10T09:47:59.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:59.139+0000 7f3633e00640 1 -- 172.21.15.5:0/2107661308 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3638004b20 con 0x55eee1f0eb90 2024-09-10T09:47:59.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:59.140+0000 7f3650fe0740 1 -- 172.21.15.5:0/2107661308 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55eee1f0eb90 msgr2=0x55eee1f17410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:47:59.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:59.140+0000 7f3650fe0740 1 --2- 172.21.15.5:0/2107661308 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55eee1f0eb90 0x55eee1f17410 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f3638012d50 tx=0x7f3638005d70 comp rx=0 tx=0).stop 2024-09-10T09:47:59.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:59.140+0000 7f3650fe0740 1 -- 172.21.15.5:0/2107661308 shutdown_connections 2024-09-10T09:47:59.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:59.140+0000 7f3650fe0740 1 --2- 172.21.15.5:0/2107661308 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55eee1f19d70 0x55eee1f322f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:59.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:59.140+0000 7f3650fe0740 1 --2- 172.21.15.5:0/2107661308 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eee1f0f4b0 0x55eee1f31db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:59.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:59.140+0000 7f3650fe0740 1 --2- 172.21.15.5:0/2107661308 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55eee1f0eb90 0x55eee1f17410 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:59.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:59.140+0000 7f3650fe0740 1 -- 172.21.15.5:0/2107661308 >> 172.21.15.5:0/2107661308 conn(0x55eee1f0b120 msgr2=0x55eee1f16d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:47:59.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:59.140+0000 7f3650fe0740 1 -- 172.21.15.5:0/2107661308 shutdown_connections 2024-09-10T09:47:59.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:59.140+0000 7f3650fe0740 1 -- 172.21.15.5:0/2107661308 wait complete. 2024-09-10T09:47:59.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:59.141+0000 7f3650fe0740 1 -- start start 2024-09-10T09:47:59.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:59.141+0000 7f3650fe0740 1 Processor -- start 2024-09-10T09:47:59.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:59.141+0000 7f3650fe0740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55eee1f0eb90 0x55eee1f223d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:47:59.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:59.141+0000 7f3650fe0740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55eee1f0f4b0 0x55eee1f1d700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:47:59.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:59.141+0000 7f3650fe0740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eee1f19d70 0x55eee1f1dc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:47:59.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:59.141+0000 7f3650fe0740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f36481b24a0 con 0x55eee1f19d70 2024-09-10T09:47:59.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:59.141+0000 7f3650fe0740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f36481b1dd0 con 0x55eee1f0f4b0 2024-09-10T09:47:59.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:59.141+0000 7f3650fe0740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f36481af010 con 0x55eee1f0eb90 2024-09-10T09:47:59.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:59.142+0000 7f3650fe0740 10 client.0 _ll_register_callbacks cb 0x7f363801e870 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:47:59.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:59.142+0000 7f3646a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55eee1f0f4b0 0x55eee1f1d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:47:59.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:59.142+0000 7f3647400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55eee1f0eb90 0x55eee1f223d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:47:59.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:59.142+0000 7f3646a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55eee1f0f4b0 0x55eee1f1d700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:41712/0 (socket says 172.21.15.5:41712) 2024-09-10T09:47:59.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:59.142+0000 7f3646a00640 1 -- 172.21.15.5:0/377341300 learned_addr learned my addr 172.21.15.5:0/377341300 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:47:59.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:59.142+0000 7f3647e00640 1 --2- 172.21.15.5:0/377341300 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eee1f19d70 0x55eee1f1dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:47:59.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:59.142+0000 7f3646a00640 1 -- 172.21.15.5:0/377341300 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55eee1f0eb90 msgr2=0x55eee1f223d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:47:59.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:59.142+0000 7f3646a00640 1 --2- 172.21.15.5:0/377341300 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55eee1f0eb90 0x55eee1f223d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:59.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:59.142+0000 7f3646a00640 1 -- 172.21.15.5:0/377341300 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eee1f19d70 msgr2=0x55eee1f1dc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:47:59.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:59.142+0000 7f3646a00640 1 --2- 172.21.15.5:0/377341300 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eee1f19d70 0x55eee1f1dc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:59.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:59.142+0000 7f3646a00640 1 -- 172.21.15.5:0/377341300 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eee1f1e330 con 0x55eee1f0f4b0 2024-09-10T09:47:59.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:59.142+0000 7f3646a00640 1 --2- 172.21.15.5:0/377341300 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55eee1f0f4b0 0x55eee1f1d700 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f3634031550 tx=0x7f363402a400 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2013b0845905365b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:47:59.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:59.142+0000 7f3631600640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-09-10T09:47:59.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:59.143+0000 7f3631600640 1 -- 172.21.15.5:0/377341300 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3634049040 con 0x55eee1f0f4b0 2024-09-10T09:47:59.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:59.143+0000 7f3631600640 1 -- 172.21.15.5:0/377341300 <== mon.2 v2:172.21.15.102:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f36340ed210 con 0x55eee1f0f4b0 2024-09-10T09:47:59.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:59.143+0000 7f3650fe0740 1 -- 172.21.15.5:0/377341300 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f363801cca0 con 0x55eee1f0f4b0 2024-09-10T09:47:59.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:59.143+0000 7f3631600640 1 -- 172.21.15.5:0/377341300 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f36340c02d0 con 0x55eee1f0f4b0 2024-09-10T09:47:59.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:59.143+0000 7f3650fe0740 10 client.27362 fetch_fsmap learned FSMap version 35 2024-09-10T09:47:59.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:59.143+0000 7f3650fe0740 1 -- 172.21.15.5:0/377341300 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55eee1f94cb0 con 0x55eee1f0f4b0 2024-09-10T09:47:59.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:59.143+0000 7f3631600640 1 -- 172.21.15.5:0/377341300 <== mon.2 v2:172.21.15.102:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f363801cca0 con 0x55eee1f0f4b0 2024-09-10T09:47:59.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:59.143+0000 7f3631600640 1 -- 172.21.15.5:0/377341300 <== mon.2 v2:172.21.15.102:3300/0 5 ==== fsmap(e 35) ==== 8374+0+0 (secure 0 0 0) 0x7f36340bf020 con 0x55eee1f0f4b0 2024-09-10T09:47:59.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:59.143+0000 7f3650fe0740 10 client.27362 fetch_fsmap finished waiting for FSMap version 35 2024-09-10T09:47:59.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:59.143+0000 7f3650fe0740 10 client.27362 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T09:47:59.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:59.143+0000 7f3650fe0740 20 client.27362 populate_metadata read hostname 'smithi005' 2024-09-10T09:47:59.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:59.143+0000 7f3650fe0740 1 --2- 172.21.15.5:0/377341300 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55eee1f2a050 0x55eee1f97480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:47:59.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:59.143+0000 7f3650fe0740 4 client.27362 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T09:47:59.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:59.143+0000 7f3650fe0740 1 -- 172.21.15.5:0/377341300 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55eee1e7ce60 con 0x55eee1f2a050 2024-09-10T09:47:59.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:59.144+0000 7f3647400640 1 --2- 172.21.15.5:0/377341300 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55eee1f2a050 0x55eee1f97480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:47:59.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:59.144+0000 7f3647400640 1 --2- 172.21.15.5:0/377341300 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55eee1f2a050 0x55eee1f97480 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1d4976660ed30cd3 server_cookie=90a7753881662bc1 in_seq=0 out_seq=0 2024-09-10T09:47:59.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:59.144+0000 7f3631600640 10 client.27362 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T09:47:59.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:59.144+0000 7f3631600640 1 -- 172.21.15.5:0/377341300 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55eee1e7ce60 con 0x55eee1f2a050 2024-09-10T09:47:59.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:59.144+0000 7f3631600640 10 client.27362 handle_command_reply: tid=0 2024-09-10T09:47:59.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:59.144+0000 7f3650fe0740 1 client.27362 shutdown 2024-09-10T09:47:59.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:59.145+0000 7f3650fe0740 1 -- 172.21.15.5:0/377341300 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55eee1f0f4b0 msgr2=0x55eee1f1d700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:47:59.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:59.145+0000 7f3650fe0740 1 --2- 172.21.15.5:0/377341300 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55eee1f0f4b0 0x55eee1f1d700 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f3634031550 tx=0x7f363402a400 comp rx=0 tx=0).stop 2024-09-10T09:47:59.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:59.145+0000 7f3650fe0740 1 -- 172.21.15.5:0/377341300 shutdown_connections 2024-09-10T09:47:59.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:59.145+0000 7f3650fe0740 1 --2- 172.21.15.5:0/377341300 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55eee1f2a050 0x55eee1f97480 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:59.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:59.145+0000 7f3650fe0740 1 --2- 172.21.15.5:0/377341300 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eee1f19d70 0x55eee1f1dc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:59.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:59.145+0000 7f3650fe0740 1 --2- 172.21.15.5:0/377341300 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55eee1f0f4b0 0x55eee1f1d700 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:59.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:59.145+0000 7f3650fe0740 1 --2- 172.21.15.5:0/377341300 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55eee1f0eb90 0x55eee1f223d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:59.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:59.145+0000 7f3631600640 0 client.27362 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T09:47:59.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:59.145+0000 7f3650fe0740 1 -- 172.21.15.5:0/377341300 >> 172.21.15.5:0/377341300 conn(0x55eee1f0b120 msgr2=0x55eee1f17500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:47:59.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:59.145+0000 7f3650fe0740 1 -- 172.21.15.5:0/377341300 shutdown_connections 2024-09-10T09:47:59.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:59.145+0000 7f3650fe0740 1 -- 172.21.15.5:0/377341300 wait complete. 2024-09-10T09:47:59.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:59.145+0000 7f3650fe0740 20 client.27362 trim_cache size 0 max 16384 2024-09-10T09:47:59.151 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-10T09:47:59.151 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-10T09:47:59.151 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2024-09-10T09:47:59.151 INFO:teuthology.orchestra.run.smithi005.stdout: "b9c7cea4-50b1-4f3b-a7fa-88cc09260e91": { 2024-09-10T09:47:59.151 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2024-09-10T09:47:59.151 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "b9c7cea4-50b1-4f3b-a7fa-88cc09260e91", 2024-09-10T09:47:59.151 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2024-09-10T09:47:59.151 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T09:47:59.152 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T09:47:59.152 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-10T09:47:59.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:59.146+0000 7f364e600640 1 -- 172.21.15.5:0/409993515 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f362c0a2c00 msgr2=0x7f362c0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:47:59.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:59.146+0000 7f364e600640 1 --2- 172.21.15.5:0/409993515 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f362c0a2c00 0x7f362c0a50c0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x55eee1f0bf80 tx=0x7f3638005f90 comp rx=0 tx=0).stop 2024-09-10T09:47:59.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:59.146+0000 7f364e600640 1 -- 172.21.15.5:0/409993515 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f36481027a0 msgr2=0x7f3648074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:47:59.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:59.146+0000 7f364e600640 1 --2- 172.21.15.5:0/409993515 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f36481027a0 0x7f3648074a30 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f3634004770 tx=0x7f3634003c10 comp rx=0 tx=0).stop 2024-09-10T09:47:59.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:59.146+0000 7f364e600640 1 -- 172.21.15.5:0/409993515 shutdown_connections 2024-09-10T09:47:59.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:59.146+0000 7f364e600640 1 --2- 172.21.15.5:0/409993515 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f362c0a2c00 0x7f362c0a50c0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:59.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:59.146+0000 7f364e600640 1 --2- 172.21.15.5:0/409993515 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3648106f20 0x7f3648074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:59.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:59.146+0000 7f364e600640 1 --2- 172.21.15.5:0/409993515 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f36481027a0 0x7f3648074a30 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:59.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:59.146+0000 7f364e600640 1 --2- 172.21.15.5:0/409993515 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3648101df0 0x7f36480744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:47:59.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:59.146+0000 7f364e600640 1 -- 172.21.15.5:0/409993515 >> 172.21.15.5:0/409993515 conn(0x7f36480fd8b0 msgr2=0x7f36480febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:47:59.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:59.146+0000 7f364e600640 1 -- 172.21.15.5:0/409993515 shutdown_connections 2024-09-10T09:47:59.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:47:59.146+0000 7f364e600640 1 -- 172.21.15.5:0/409993515 wait complete. 2024-09-10T09:47:59.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:47:59 smithi159 ceph-mon[35513]: pgmap v682: 129 pgs: 129 active+clean; 212 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 999 KiB/s rd, 1.8 MiB/s wr, 115 op/s 2024-09-10T09:47:59.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:47:59 smithi102 ceph-mon[35442]: pgmap v682: 129 pgs: 129 active+clean; 212 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 999 KiB/s rd, 1.8 MiB/s wr, 115 op/s 2024-09-10T09:47:59.698 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:47:59 smithi005 ceph-mon[30350]: pgmap v682: 129 pgs: 129 active+clean; 212 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 999 KiB/s rd, 1.8 MiB/s wr, 115 op/s 2024-09-10T09:47:59.963 INFO:tasks.cephfs.filesystem:scrub status for tag:b9c7cea4-50b1-4f3b-a7fa-88cc09260e91 - {'path': '/', 'tag': 'b9c7cea4-50b1-4f3b-a7fa-88cc09260e91', 'options': 'recursive,force'} 2024-09-10T09:48:00.100 INFO:tasks.workunit.client.0.smithi005.stdout: 1 569966 6.02 MB/sec execute 341 sec latency 24.100 ms 2024-09-10T09:48:01.099 INFO:tasks.workunit.client.0.smithi005.stdout: 1 572432 6.04 MB/sec execute 342 sec latency 38.109 ms 2024-09-10T09:48:01.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:48:01 smithi159 ceph-mon[35513]: pgmap v683: 129 pgs: 129 active+clean; 214 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.4 MiB/s wr, 173 op/s 2024-09-10T09:48:01.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:48:01 smithi102 ceph-mon[35442]: pgmap v683: 129 pgs: 129 active+clean; 214 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.4 MiB/s wr, 173 op/s 2024-09-10T09:48:01.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:48:01 smithi005 ceph-mon[30350]: pgmap v683: 129 pgs: 129 active+clean; 214 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.4 MiB/s wr, 173 op/s 2024-09-10T09:48:02.099 INFO:tasks.workunit.client.0.smithi005.stdout: 1 574827 6.06 MB/sec execute 343 sec latency 25.342 ms 2024-09-10T09:48:03.099 INFO:tasks.workunit.client.0.smithi005.stdout: 1 576818 6.08 MB/sec execute 344 sec latency 21.928 ms 2024-09-10T09:48:03.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:48:03 smithi159 ceph-mon[35513]: pgmap v684: 129 pgs: 129 active+clean; 211 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.1 MiB/s wr, 169 op/s 2024-09-10T09:48:03.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:48:03 smithi102 ceph-mon[35442]: pgmap v684: 129 pgs: 129 active+clean; 211 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.1 MiB/s wr, 169 op/s 2024-09-10T09:48:03.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:48:03 smithi005 ceph-mon[30350]: pgmap v684: 129 pgs: 129 active+clean; 211 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.1 MiB/s wr, 169 op/s 2024-09-10T09:48:04.099 INFO:tasks.workunit.client.0.smithi005.stdout: 1 579305 6.10 MB/sec execute 345 sec latency 12.574 ms 2024-09-10T09:48:05.099 INFO:tasks.workunit.client.0.smithi005.stdout: 1 581444 6.11 MB/sec execute 346 sec latency 22.214 ms 2024-09-10T09:48:05.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:48:05 smithi159 ceph-mon[35513]: pgmap v685: 129 pgs: 129 active+clean; 212 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 1.8 MiB/s wr, 164 op/s 2024-09-10T09:48:05.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:48:05 smithi102 ceph-mon[35442]: pgmap v685: 129 pgs: 129 active+clean; 212 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 1.8 MiB/s wr, 164 op/s 2024-09-10T09:48:05.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:48:05 smithi005 ceph-mon[30350]: pgmap v685: 129 pgs: 129 active+clean; 212 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 1.8 MiB/s wr, 164 op/s 2024-09-10T09:48:06.099 INFO:tasks.workunit.client.0.smithi005.stdout: 1 583505 6.12 MB/sec execute 347 sec latency 25.054 ms 2024-09-10T09:48:07.099 INFO:tasks.workunit.client.0.smithi005.stdout: 1 586295 6.16 MB/sec execute 348 sec latency 16.399 ms 2024-09-10T09:48:07.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:48:07 smithi102 ceph-mon[35442]: pgmap v686: 129 pgs: 129 active+clean; 216 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.1 MiB/s wr, 235 op/s 2024-09-10T09:48:07.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:48:07 smithi005 ceph-mon[30350]: pgmap v686: 129 pgs: 129 active+clean; 216 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.1 MiB/s wr, 235 op/s 2024-09-10T09:48:07.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:48:07 smithi159 ceph-mon[35513]: pgmap v686: 129 pgs: 129 active+clean; 216 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.1 MiB/s wr, 235 op/s 2024-09-10T09:48:08.099 INFO:tasks.workunit.client.0.smithi005.stdout: 1 588255 6.16 MB/sec execute 349 sec latency 20.792 ms 2024-09-10T09:48:08.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:48:08 smithi102 ceph-mon[35442]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T09:48:08.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:48:08 smithi005 ceph-mon[30350]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T09:48:08.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:48:08 smithi159 ceph-mon[35513]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T09:48:09.099 INFO:tasks.workunit.client.0.smithi005.stdout: 1 590365 6.18 MB/sec execute 350 sec latency 23.295 ms 2024-09-10T09:48:09.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:48:09 smithi102 ceph-mon[35442]: pgmap v687: 129 pgs: 129 active+clean; 217 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 1.6 MiB/s wr, 185 op/s 2024-09-10T09:48:09.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:48:09 smithi005 ceph-mon[30350]: pgmap v687: 129 pgs: 129 active+clean; 217 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 1.6 MiB/s wr, 185 op/s 2024-09-10T09:48:09.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:48:09 smithi159 ceph-mon[35513]: pgmap v687: 129 pgs: 129 active+clean; 217 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 1.6 MiB/s wr, 185 op/s 2024-09-10T09:48:10.100 INFO:tasks.workunit.client.0.smithi005.stdout: 1 593164 6.21 MB/sec execute 351 sec latency 13.547 ms 2024-09-10T09:48:11.099 INFO:tasks.workunit.client.0.smithi005.stdout: 1 594753 6.21 MB/sec execute 352 sec latency 35.225 ms 2024-09-10T09:48:11.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:48:11 smithi102 ceph-mon[35442]: pgmap v688: 129 pgs: 129 active+clean; 221 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 1.9 MiB/s wr, 248 op/s 2024-09-10T09:48:11.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:48:11 smithi005 ceph-mon[30350]: pgmap v688: 129 pgs: 129 active+clean; 221 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 1.9 MiB/s wr, 248 op/s 2024-09-10T09:48:11.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:48:11 smithi159 ceph-mon[35513]: pgmap v688: 129 pgs: 129 active+clean; 221 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 1.9 MiB/s wr, 248 op/s 2024-09-10T09:48:12.100 INFO:tasks.workunit.client.0.smithi005.stdout: 1 597214 6.23 MB/sec execute 353 sec latency 8.829 ms 2024-09-10T09:48:12.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:48:12 smithi102 ceph-mon[35442]: scrub summary: idle 2024-09-10T09:48:12.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:48:12 smithi005 ceph-mon[30350]: scrub summary: idle 2024-09-10T09:48:12.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:48:12 smithi159 ceph-mon[35513]: scrub summary: idle 2024-09-10T09:48:13.100 INFO:tasks.workunit.client.0.smithi005.stdout: 1 599688 6.25 MB/sec execute 354 sec latency 21.815 ms 2024-09-10T09:48:13.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:48:13 smithi102 ceph-mon[35442]: pgmap v689: 129 pgs: 129 active+clean; 222 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 1.2 MiB/s wr, 191 op/s 2024-09-10T09:48:13.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:48:13 smithi102 ceph-mon[35442]: scrub summary: idle 2024-09-10T09:48:13.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:48:13 smithi005 ceph-mon[30350]: pgmap v689: 129 pgs: 129 active+clean; 222 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 1.2 MiB/s wr, 191 op/s 2024-09-10T09:48:13.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:48:13 smithi005 ceph-mon[30350]: scrub summary: idle 2024-09-10T09:48:13.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:48:13 smithi159 ceph-mon[35513]: pgmap v689: 129 pgs: 129 active+clean; 222 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 1.2 MiB/s wr, 191 op/s 2024-09-10T09:48:13.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:48:13 smithi159 ceph-mon[35513]: scrub summary: idle 2024-09-10T09:48:14.100 INFO:tasks.workunit.client.0.smithi005.stdout: 1 601425 6.26 MB/sec execute 355 sec latency 34.543 ms 2024-09-10T09:48:15.100 INFO:tasks.workunit.client.0.smithi005.stdout: 1 604055 6.29 MB/sec execute 356 sec latency 12.353 ms 2024-09-10T09:48:15.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:48:15 smithi102 ceph-mon[35442]: pgmap v690: 129 pgs: 129 active+clean; 222 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 1.2 MiB/s wr, 184 op/s 2024-09-10T09:48:15.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:48:15 smithi005 ceph-mon[30350]: pgmap v690: 129 pgs: 129 active+clean; 222 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 1.2 MiB/s wr, 184 op/s 2024-09-10T09:48:15.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:48:15 smithi159 ceph-mon[35513]: pgmap v690: 129 pgs: 129 active+clean; 222 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 1.2 MiB/s wr, 184 op/s 2024-09-10T09:48:16.100 INFO:tasks.workunit.client.0.smithi005.stdout: 1 605989 6.29 MB/sec execute 357 sec latency 26.589 ms 2024-09-10T09:48:17.100 INFO:tasks.workunit.client.0.smithi005.stdout: 1 608066 6.31 MB/sec execute 358 sec latency 19.381 ms 2024-09-10T09:48:17.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:48:17 smithi102 ceph-mon[35442]: pgmap v691: 129 pgs: 129 active+clean; 225 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 1.4 MiB/s wr, 235 op/s 2024-09-10T09:48:17.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:48:17 smithi005 ceph-mon[30350]: pgmap v691: 129 pgs: 129 active+clean; 225 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 1.4 MiB/s wr, 235 op/s 2024-09-10T09:48:17.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:48:17 smithi159 ceph-mon[35513]: pgmap v691: 129 pgs: 129 active+clean; 225 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 1.4 MiB/s wr, 235 op/s 2024-09-10T09:48:18.100 INFO:tasks.workunit.client.0.smithi005.stdout: 1 610837 6.34 MB/sec execute 359 sec latency 12.996 ms 2024-09-10T09:48:19.100 INFO:tasks.workunit.client.0.smithi005.stdout: 1 612400 6.34 MB/sec execute 360 sec latency 24.923 ms 2024-09-10T09:48:19.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:48:19 smithi102 ceph-mon[35442]: pgmap v692: 129 pgs: 129 active+clean; 228 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 1.3 MiB/s wr, 198 op/s 2024-09-10T09:48:19.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:48:19 smithi005 ceph-mon[30350]: pgmap v692: 129 pgs: 129 active+clean; 228 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 1.3 MiB/s wr, 198 op/s 2024-09-10T09:48:19.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:48:19 smithi159 ceph-mon[35513]: pgmap v692: 129 pgs: 129 active+clean; 228 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 1.3 MiB/s wr, 198 op/s 2024-09-10T09:48:20.100 INFO:tasks.workunit.client.0.smithi005.stdout: 1 614829 6.36 MB/sec execute 361 sec latency 10.828 ms 2024-09-10T09:48:21.100 INFO:tasks.workunit.client.0.smithi005.stdout: 1 616935 6.36 MB/sec execute 362 sec latency 26.089 ms 2024-09-10T09:48:21.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:48:21 smithi102 ceph-mon[35442]: pgmap v693: 129 pgs: 129 active+clean; 229 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 1.3 MiB/s wr, 219 op/s 2024-09-10T09:48:21.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:48:21 smithi005 ceph-mon[30350]: pgmap v693: 129 pgs: 129 active+clean; 229 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 1.3 MiB/s wr, 219 op/s 2024-09-10T09:48:21.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:48:21 smithi159 ceph-mon[35513]: pgmap v693: 129 pgs: 129 active+clean; 229 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 1.3 MiB/s wr, 219 op/s 2024-09-10T09:48:22.100 INFO:tasks.workunit.client.0.smithi005.stdout: 1 618963 6.38 MB/sec execute 363 sec latency 24.498 ms 2024-09-10T09:48:23.100 INFO:tasks.workunit.client.0.smithi005.stdout: 1 621667 6.41 MB/sec execute 364 sec latency 28.400 ms 2024-09-10T09:48:23.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:48:23 smithi102 ceph-mon[35442]: pgmap v694: 129 pgs: 129 active+clean; 232 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 1.2 MiB/s wr, 188 op/s 2024-09-10T09:48:23.651 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:48:23 smithi159 ceph-mon[35513]: pgmap v694: 129 pgs: 129 active+clean; 232 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 1.2 MiB/s wr, 188 op/s 2024-09-10T09:48:23.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:48:23 smithi005 ceph-mon[30350]: pgmap v694: 129 pgs: 129 active+clean; 232 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 1.2 MiB/s wr, 188 op/s 2024-09-10T09:48:24.100 INFO:tasks.workunit.client.0.smithi005.stdout: 1 623505 6.41 MB/sec execute 365 sec latency 23.395 ms 2024-09-10T09:48:25.101 INFO:tasks.workunit.client.0.smithi005.stdout: 1 625762 6.43 MB/sec execute 366 sec latency 23.785 ms 2024-09-10T09:48:25.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:48:25 smithi102 ceph-mon[35442]: pgmap v695: 129 pgs: 129 active+clean; 232 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 1.2 MiB/s wr, 184 op/s 2024-09-10T09:48:25.649 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:48:25 smithi005 ceph-mon[30350]: pgmap v695: 129 pgs: 129 active+clean; 232 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 1.2 MiB/s wr, 184 op/s 2024-09-10T09:48:25.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:48:25 smithi159 ceph-mon[35513]: pgmap v695: 129 pgs: 129 active+clean; 232 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 1.2 MiB/s wr, 184 op/s 2024-09-10T09:48:26.100 INFO:tasks.workunit.client.0.smithi005.stdout: 1 626564 6.43 MB/sec execute 367 sec latency 485.724 ms 2024-09-10T09:48:27.100 INFO:tasks.workunit.client.0.smithi005.stdout: 1 627835 6.43 MB/sec execute 368 sec latency 409.928 ms 2024-09-10T09:48:27.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:48:27 smithi005 ceph-mon[30350]: pgmap v696: 129 pgs: 129 active+clean; 237 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 1.6 MiB/s wr, 260 op/s 2024-09-10T09:48:27.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:48:27 smithi159 ceph-mon[35513]: pgmap v696: 129 pgs: 129 active+clean; 237 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 1.6 MiB/s wr, 260 op/s 2024-09-10T09:48:27.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:48:27 smithi102 ceph-mon[35442]: pgmap v696: 129 pgs: 129 active+clean; 237 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 1.6 MiB/s wr, 260 op/s 2024-09-10T09:48:28.100 INFO:tasks.workunit.client.0.smithi005.stdout: 1 629560 6.43 MB/sec execute 369 sec latency 20.637 ms 2024-09-10T09:48:29.100 INFO:tasks.workunit.client.0.smithi005.stdout: 1 632387 6.46 MB/sec execute 370 sec latency 13.198 ms 2024-09-10T09:48:29.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:48:29 smithi005 ceph-mon[30350]: pgmap v697: 129 pgs: 129 active+clean; 240 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 1.6 MiB/s wr, 213 op/s 2024-09-10T09:48:29.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:48:29 smithi159 ceph-mon[35513]: pgmap v697: 129 pgs: 129 active+clean; 240 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 1.6 MiB/s wr, 213 op/s 2024-09-10T09:48:29.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:48:29 smithi102 ceph-mon[35442]: pgmap v697: 129 pgs: 129 active+clean; 240 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 1.6 MiB/s wr, 213 op/s 2024-09-10T09:48:29.964 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T09:48:30.101 INFO:tasks.workunit.client.0.smithi005.stdout: 1 634206 6.46 MB/sec execute 371 sec latency 22.641 ms 2024-09-10T09:48:30.362 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:48:31.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:31.091+0000 7fdcfd800640 1 -- 172.21.15.5:0/463397484 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdcf81049c0 msgr2=0x7fdcf8108a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:48:31.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:31.091+0000 7fdcfd800640 1 --2- 172.21.15.5:0/463397484 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdcf81049c0 0x7fdcf8108a10 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fdce4009900 tx=0x7fdce402a410 comp rx=0 tx=0).stop 2024-09-10T09:48:31.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:31.091+0000 7fdcfd800640 1 -- 172.21.15.5:0/463397484 shutdown_connections 2024-09-10T09:48:31.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:31.091+0000 7fdcfd800640 1 --2- 172.21.15.5:0/463397484 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdcf8109140 0x7fdcf8110cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:31.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:31.091+0000 7fdcfd800640 1 --2- 172.21.15.5:0/463397484 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdcf81049c0 0x7fdcf8108a10 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:31.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:31.091+0000 7fdcfd800640 1 --2- 172.21.15.5:0/463397484 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdcf8104010 0x7fdcf81043f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:31.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:31.091+0000 7fdcfd800640 1 -- 172.21.15.5:0/463397484 >> 172.21.15.5:0/463397484 conn(0x7fdcf80ffa50 msgr2=0x7fdcf8101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:48:31.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:31.091+0000 7fdcfd800640 1 -- 172.21.15.5:0/463397484 shutdown_connections 2024-09-10T09:48:31.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:31.091+0000 7fdcfd800640 1 -- 172.21.15.5:0/463397484 wait complete. 2024-09-10T09:48:31.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:31.092+0000 7fdcfd800640 1 Processor -- start 2024-09-10T09:48:31.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:31.093+0000 7fdcfd800640 1 -- start start 2024-09-10T09:48:31.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:31.093+0000 7fdcfd800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdcf8104010 0x7fdcf81a0570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:48:31.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:31.093+0000 7fdcfd800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdcf81049c0 0x7fdcf81a0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:48:31.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:31.093+0000 7fdcfd800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdcf8109140 0x7fdcf819a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:48:31.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:31.093+0000 7fdcfd800640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fdcf8114230 con 0x7fdcf8104010 2024-09-10T09:48:31.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:31.093+0000 7fdcfd800640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fdcf81140b0 con 0x7fdcf81049c0 2024-09-10T09:48:31.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:31.093+0000 7fdcfd800640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fdcf81143b0 con 0x7fdcf8109140 2024-09-10T09:48:31.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:31.093+0000 7fdcf6a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdcf8104010 0x7fdcf81a0570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:48:31.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:31.094+0000 7fdcf7400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdcf8109140 0x7fdcf819a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:48:31.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:31.094+0000 7fdcf6a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdcf8104010 0x7fdcf81a0570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50328/0 (socket says 172.21.15.5:50328) 2024-09-10T09:48:31.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:31.094+0000 7fdcf6a00640 1 -- 172.21.15.5:0/1483862051 learned_addr learned my addr 172.21.15.5:0/1483862051 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:48:31.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:31.094+0000 7fdcf6000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdcf81049c0 0x7fdcf81a0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:48:31.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:31.094+0000 7fdcf6a00640 1 -- 172.21.15.5:0/1483862051 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdcf8109140 msgr2=0x7fdcf819a640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:48:31.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:31.094+0000 7fdcf6a00640 1 --2- 172.21.15.5:0/1483862051 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdcf8109140 0x7fdcf819a640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:31.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:31.094+0000 7fdcf6a00640 1 -- 172.21.15.5:0/1483862051 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdcf81049c0 msgr2=0x7fdcf81a0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:48:31.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:31.094+0000 7fdcf6a00640 1 --2- 172.21.15.5:0/1483862051 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdcf81049c0 0x7fdcf81a0ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:31.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:31.094+0000 7fdcf6a00640 1 -- 172.21.15.5:0/1483862051 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdcf819aeb0 con 0x7fdcf8104010 2024-09-10T09:48:31.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:31.095+0000 7fdcf6a00640 1 --2- 172.21.15.5:0/1483862051 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdcf8104010 0x7fdcf81a0570 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fdce800b450 tx=0x7fdce800b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:48:31.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:31.095+0000 7fdcef400640 1 -- 172.21.15.5:0/1483862051 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdce8015020 con 0x7fdcf8104010 2024-09-10T09:48:31.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:31.095+0000 7fdcef400640 1 -- 172.21.15.5:0/1483862051 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdce8011d70 con 0x7fdcf8104010 2024-09-10T09:48:31.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:31.095+0000 7fdcfd800640 1 -- 172.21.15.5:0/1483862051 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdcf819b1a0 con 0x7fdcf8104010 2024-09-10T09:48:31.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:31.095+0000 7fdcef400640 1 -- 172.21.15.5:0/1483862051 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdce800f340 con 0x7fdcf8104010 2024-09-10T09:48:31.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:31.095+0000 7fdcfd800640 1 -- 172.21.15.5:0/1483862051 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdcf81a74f0 con 0x7fdcf8104010 2024-09-10T09:48:31.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:31.099+0000 7fdcef400640 1 -- 172.21.15.5:0/1483862051 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7fdce8018020 con 0x7fdcf8104010 2024-09-10T09:48:31.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:31.099+0000 7fdd002c0740 1 Processor -- start 2024-09-10T09:48:31.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:31.099+0000 7fdd002c0740 1 -- start start 2024-09-10T09:48:31.101 INFO:tasks.workunit.client.0.smithi005.stdout: 1 635707 6.48 MB/sec execute 372 sec latency 454.901 ms 2024-09-10T09:48:31.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:31.100+0000 7fdd002c0740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562f97124ab0 0x562f97124e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:48:31.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:31.100+0000 7fdd002c0740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562f971253d0 0x562f97138040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:48:31.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:31.100+0000 7fdd002c0740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562f97138580 0x562f9713a970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:48:31.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:31.100+0000 7fdd002c0740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x562f9708b3e0 con 0x562f971253d0 2024-09-10T09:48:31.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:31.100+0000 7fdd002c0740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x562f97032d80 con 0x562f97124ab0 2024-09-10T09:48:31.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:31.100+0000 7fdd002c0740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x562f970873a0 con 0x562f97138580 2024-09-10T09:48:31.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:31.100+0000 7fdcf7400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562f97124ab0 0x562f97124e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:48:31.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:31.100+0000 7fdcf6a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562f971253d0 0x562f97138040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:48:31.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:31.101+0000 7fdcf6000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562f97138580 0x562f9713a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:48:31.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:31.101+0000 7fdcf6000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562f97138580 0x562f9713a970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:45782/0 (socket says 172.21.15.5:45782) 2024-09-10T09:48:31.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:31.101+0000 7fdcf6000640 1 -- 172.21.15.5:0/1497409070 learned_addr learned my addr 172.21.15.5:0/1497409070 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:48:31.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:31.101+0000 7fdcf7400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562f97124ab0 0x562f97124e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:33918/0 (socket says 172.21.15.5:33918) 2024-09-10T09:48:31.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:31.101+0000 7fdcef400640 1 --2- 172.21.15.5:0/1483862051 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fdcdc0a2c00 0x7fdcdc0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:48:31.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:31.101+0000 7fdcf6000640 1 -- 172.21.15.5:0/1497409070 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562f97124ab0 msgr2=0x562f97124e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:48:31.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:31.101+0000 7fdcf6000640 1 --2- 172.21.15.5:0/1497409070 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562f97124ab0 0x562f97124e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:31.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:31.101+0000 7fdcf6000640 1 -- 172.21.15.5:0/1497409070 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562f971253d0 msgr2=0x562f97138040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:48:31.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:31.102+0000 7fdcef400640 1 -- 172.21.15.5:0/1483862051 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fdce80ceec0 con 0x7fdcf8104010 2024-09-10T09:48:31.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:31.102+0000 7fdcf6000640 1 --2- 172.21.15.5:0/1497409070 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562f971253d0 0x562f97138040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:31.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:31.102+0000 7fdcf6000640 1 -- 172.21.15.5:0/1497409070 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562f97051ae0 con 0x562f97138580 2024-09-10T09:48:31.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:31.102+0000 7fdcf6a00640 1 --2- 172.21.15.5:0/1497409070 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562f971253d0 0x562f97138040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:48:31.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:31.102+0000 7fdcf7400640 1 --2- 172.21.15.5:0/1483862051 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fdcdc0a2c00 0x7fdcdc0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:48:31.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:31.102+0000 7fdcf6000640 1 --2- 172.21.15.5:0/1497409070 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562f97138580 0x562f9713a970 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fdce4009230 tx=0x7fdce4004180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eef552f21c5f9733 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:48:31.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:31.103+0000 7fdce3e00640 1 -- 172.21.15.5:0/1497409070 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdce4035040 con 0x562f97138580 2024-09-10T09:48:31.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:31.103+0000 7fdcf7400640 1 --2- 172.21.15.5:0/1483862051 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fdcdc0a2c00 0x7fdcdc0a50c0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fdcf0005fd0 tx=0x7fdcf0004ab0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:48:31.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:31.103+0000 7fdce3e00640 1 -- 172.21.15.5:0/1497409070 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdce4005520 con 0x562f97138580 2024-09-10T09:48:31.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:31.103+0000 7fdce3e00640 1 -- 172.21.15.5:0/1497409070 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdce4003850 con 0x562f97138580 2024-09-10T09:48:31.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:31.103+0000 7fdd002c0740 1 -- 172.21.15.5:0/1497409070 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562f97138580 msgr2=0x562f9713a970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:48:31.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:31.103+0000 7fdd002c0740 1 --2- 172.21.15.5:0/1497409070 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562f97138580 0x562f9713a970 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fdce4009230 tx=0x7fdce4004180 comp rx=0 tx=0).stop 2024-09-10T09:48:31.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:31.104+0000 7fdd002c0740 1 -- 172.21.15.5:0/1497409070 shutdown_connections 2024-09-10T09:48:31.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:31.104+0000 7fdd002c0740 1 --2- 172.21.15.5:0/1497409070 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562f97138580 0x562f9713a970 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:31.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:31.104+0000 7fdd002c0740 1 --2- 172.21.15.5:0/1497409070 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562f971253d0 0x562f97138040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:31.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:31.104+0000 7fdd002c0740 1 --2- 172.21.15.5:0/1497409070 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562f97124ab0 0x562f97124e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:31.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:31.104+0000 7fdd002c0740 1 -- 172.21.15.5:0/1497409070 >> 172.21.15.5:0/1497409070 conn(0x562f97121120 msgr2=0x562f97121500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:48:31.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:31.104+0000 7fdd002c0740 1 -- 172.21.15.5:0/1497409070 shutdown_connections 2024-09-10T09:48:31.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:31.104+0000 7fdd002c0740 1 -- 172.21.15.5:0/1497409070 wait complete. 2024-09-10T09:48:31.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:31.104+0000 7fdd002c0740 1 -- start start 2024-09-10T09:48:31.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:31.105+0000 7fdd002c0740 1 Processor -- start 2024-09-10T09:48:31.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:31.105+0000 7fdd002c0740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562f97124ab0 0x562f971354b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:48:31.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:31.105+0000 7fdd002c0740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562f971253d0 0x562f971359f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:48:31.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:31.105+0000 7fdd002c0740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562f97138580 0x562f97135f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:48:31.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:31.105+0000 7fdd002c0740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x562f9713b130 con 0x562f971253d0 2024-09-10T09:48:31.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:31.105+0000 7fdd002c0740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x562f970460f0 con 0x562f97124ab0 2024-09-10T09:48:31.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:31.105+0000 7fdd002c0740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x562f9713b2b0 con 0x562f97138580 2024-09-10T09:48:31.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:31.105+0000 7fdd002c0740 10 client.0 _ll_register_callbacks cb 0x562f970379e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:48:31.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:31.105+0000 7fdcf6a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562f97124ab0 0x562f971354b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:48:31.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:31.106+0000 7fdcf6a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562f97124ab0 0x562f971354b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:33930/0 (socket says 172.21.15.5:33930) 2024-09-10T09:48:31.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:31.106+0000 7fdcf6a00640 1 -- 172.21.15.5:0/1086364321 learned_addr learned my addr 172.21.15.5:0/1086364321 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:48:31.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:31.106+0000 7fdcf6000640 1 --2- 172.21.15.5:0/1086364321 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562f971253d0 0x562f971359f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:48:31.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:31.106+0000 7fdcf6a00640 1 -- 172.21.15.5:0/1086364321 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562f97138580 msgr2=0x562f97135f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:48:31.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:31.106+0000 7fdcf7400640 1 --2- 172.21.15.5:0/1086364321 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562f97138580 0x562f97135f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:48:31.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:31.106+0000 7fdcf6a00640 1 --2- 172.21.15.5:0/1086364321 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562f97138580 0x562f97135f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:31.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:31.106+0000 7fdcf6a00640 1 -- 172.21.15.5:0/1086364321 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562f971253d0 msgr2=0x562f971359f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:48:31.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:31.106+0000 7fdcf6a00640 1 --2- 172.21.15.5:0/1086364321 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562f971253d0 0x562f971359f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:31.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:31.106+0000 7fdcf6a00640 1 -- 172.21.15.5:0/1086364321 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562f96dfc3c0 con 0x562f97124ab0 2024-09-10T09:48:31.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:31.106+0000 7fdcf7400640 1 --2- 172.21.15.5:0/1086364321 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562f97138580 0x562f97135f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T09:48:31.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:31.107+0000 7fdcf6a00640 1 --2- 172.21.15.5:0/1086364321 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562f97124ab0 0x562f971354b0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fdce8000c00 tx=0x7fdce8009370 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eb6bfc99cf7d547d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:48:31.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:31.107+0000 7fdce0c00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-09-10T09:48:31.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:31.107+0000 7fdce0c00640 1 -- 172.21.15.5:0/1086364321 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdce809b9d0 con 0x562f97124ab0 2024-09-10T09:48:31.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:31.107+0000 7fdd002c0740 1 -- 172.21.15.5:0/1086364321 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdce4005520 con 0x562f97124ab0 2024-09-10T09:48:31.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:31.107+0000 7fdce0c00640 1 -- 172.21.15.5:0/1086364321 <== mon.2 v2:172.21.15.102:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fdce80bb0e0 con 0x562f97124ab0 2024-09-10T09:48:31.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:31.107+0000 7fdce0c00640 1 -- 172.21.15.5:0/1086364321 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fdce809b2d0 con 0x562f97124ab0 2024-09-10T09:48:31.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:31.107+0000 7fdd002c0740 10 client.27371 fetch_fsmap learned FSMap version 36 2024-09-10T09:48:31.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:31.107+0000 7fdd002c0740 1 -- 172.21.15.5:0/1086364321 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x562f97146f50 con 0x562f97124ab0 2024-09-10T09:48:31.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:31.107+0000 7fdce0c00640 1 -- 172.21.15.5:0/1086364321 <== mon.2 v2:172.21.15.102:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdce4005520 con 0x562f97124ab0 2024-09-10T09:48:31.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:31.108+0000 7fdce0c00640 1 -- 172.21.15.5:0/1086364321 <== mon.2 v2:172.21.15.102:3300/0 5 ==== fsmap(e 36) ==== 8370+0+0 (secure 0 0 0) 0x7fdce80b4140 con 0x562f97124ab0 2024-09-10T09:48:31.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:31.108+0000 7fdd002c0740 10 client.27371 fetch_fsmap finished waiting for FSMap version 36 2024-09-10T09:48:31.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:31.108+0000 7fdd002c0740 10 client.27371 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T09:48:31.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:31.108+0000 7fdd002c0740 20 client.27371 populate_metadata read hostname 'smithi005' 2024-09-10T09:48:31.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:31.108+0000 7fdd002c0740 1 --2- 172.21.15.5:0/1086364321 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x562f97148360 0x562f9714a750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:48:31.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:31.108+0000 7fdd002c0740 4 client.27371 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:48:31.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:31.108+0000 7fdd002c0740 1 -- 172.21.15.5:0/1086364321 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562f97098950 con 0x562f97148360 2024-09-10T09:48:31.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:31.131+0000 7fdcf6000640 1 --2- 172.21.15.5:0/1086364321 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x562f97148360 0x562f9714a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:48:31.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:31.190+0000 7fdcf6000640 1 --2- 172.21.15.5:0/1086364321 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x562f97148360 0x562f9714a750 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f5f0ee56e19f388f server_cookie=dbe7da82b5d9a4ed in_seq=0 out_seq=0 2024-09-10T09:48:31.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:31.190+0000 7fdce0c00640 10 client.27371 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T09:48:31.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:31.214+0000 7fdce0c00640 1 -- 172.21.15.5:0/1086364321 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562f97098950 con 0x562f97148360 2024-09-10T09:48:31.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:31.214+0000 7fdce0c00640 10 client.27371 handle_command_reply: tid=0 2024-09-10T09:48:31.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:31.214+0000 7fdd002c0740 1 client.27371 shutdown 2024-09-10T09:48:31.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:31.214+0000 7fdd002c0740 1 -- 172.21.15.5:0/1086364321 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562f97124ab0 msgr2=0x562f971354b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:48:31.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:31.214+0000 7fdd002c0740 1 --2- 172.21.15.5:0/1086364321 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562f97124ab0 0x562f971354b0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fdce8000c00 tx=0x7fdce8009370 comp rx=0 tx=0).stop 2024-09-10T09:48:31.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:31.215+0000 7fdd002c0740 1 -- 172.21.15.5:0/1086364321 shutdown_connections 2024-09-10T09:48:31.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:31.215+0000 7fdd002c0740 1 --2- 172.21.15.5:0/1086364321 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x562f97148360 0x562f9714a750 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:31.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:31.215+0000 7fdd002c0740 1 --2- 172.21.15.5:0/1086364321 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562f97138580 0x562f97135f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:31.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:31.215+0000 7fdd002c0740 1 --2- 172.21.15.5:0/1086364321 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562f971253d0 0x562f971359f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:31.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:31.215+0000 7fdd002c0740 1 --2- 172.21.15.5:0/1086364321 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562f97124ab0 0x562f971354b0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:31.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:31.215+0000 7fdd002c0740 1 -- 172.21.15.5:0/1086364321 >> 172.21.15.5:0/1086364321 conn(0x562f97121120 msgr2=0x562f97126db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:48:31.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:31.215+0000 7fdce0c00640 0 client.27371 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T09:48:31.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:31.215+0000 7fdd002c0740 1 -- 172.21.15.5:0/1086364321 shutdown_connections 2024-09-10T09:48:31.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:31.215+0000 7fdd002c0740 1 -- 172.21.15.5:0/1086364321 wait complete. 2024-09-10T09:48:31.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:31.215+0000 7fdd002c0740 20 client.27371 trim_cache size 0 max 16384 2024-09-10T09:48:31.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:31.233+0000 7fdd002c0740 1 Processor -- start 2024-09-10T09:48:31.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:31.233+0000 7fdd002c0740 1 -- start start 2024-09-10T09:48:31.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:31.234+0000 7fdd002c0740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562f97124ab0 0x562f97135b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:48:31.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:31.234+0000 7fdd002c0740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562f971253d0 0x562f97150000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:48:31.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:31.234+0000 7fdd002c0740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562f97138580 0x562f97150540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:48:31.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:31.234+0000 7fdd002c0740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fdcf81acd20 con 0x562f97124ab0 2024-09-10T09:48:31.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:31.234+0000 7fdd002c0740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fdcf81ae860 con 0x562f971253d0 2024-09-10T09:48:31.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:31.234+0000 7fdd002c0740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fdcf81ad3f0 con 0x562f97138580 2024-09-10T09:48:31.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:31.234+0000 7fdcf6a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562f97124ab0 0x562f97135b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:48:31.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:31.234+0000 7fdcf6a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562f97124ab0 0x562f97135b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50356/0 (socket says 172.21.15.5:50356) 2024-09-10T09:48:31.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:31.234+0000 7fdcf6a00640 1 -- 172.21.15.5:0/74707059 learned_addr learned my addr 172.21.15.5:0/74707059 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:48:31.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:31.234+0000 7fdcf6000640 1 --2- 172.21.15.5:0/74707059 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562f971253d0 0x562f97150000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:48:31.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:31.235+0000 7fdcf6a00640 1 -- 172.21.15.5:0/74707059 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562f97138580 msgr2=0x562f97150540 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T09:48:31.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:31.235+0000 7fdcf6a00640 1 --2- 172.21.15.5:0/74707059 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562f97138580 0x562f97150540 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:31.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:31.235+0000 7fdcf6a00640 1 -- 172.21.15.5:0/74707059 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562f971253d0 msgr2=0x562f97150000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:48:31.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:31.235+0000 7fdcf6a00640 1 --2- 172.21.15.5:0/74707059 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562f971253d0 0x562f97150000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:31.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:31.235+0000 7fdcf6a00640 1 -- 172.21.15.5:0/74707059 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562f97125f80 con 0x562f97124ab0 2024-09-10T09:48:31.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:31.235+0000 7fdcf6000640 1 --2- 172.21.15.5:0/74707059 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562f971253d0 0x562f97150000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:48:31.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:31.235+0000 7fdcf6a00640 1 --2- 172.21.15.5:0/74707059 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562f97124ab0 0x562f97135b80 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fdce800be10 tx=0x7fdce80d23d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64c2fcebe7cb4e2c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:48:31.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:31.235+0000 7fdce3400640 1 -- 172.21.15.5:0/74707059 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdce809aa10 con 0x562f97124ab0 2024-09-10T09:48:31.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:31.235+0000 7fdce3400640 1 -- 172.21.15.5:0/74707059 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdce809f680 con 0x562f97124ab0 2024-09-10T09:48:31.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:31.235+0000 7fdce3400640 1 -- 172.21.15.5:0/74707059 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdce80bb310 con 0x562f97124ab0 2024-09-10T09:48:31.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:31.236+0000 7fdd002c0740 1 -- 172.21.15.5:0/74707059 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562f97124ab0 msgr2=0x562f97135b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:48:31.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:31.236+0000 7fdd002c0740 1 --2- 172.21.15.5:0/74707059 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562f97124ab0 0x562f97135b80 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fdce800be10 tx=0x7fdce80d23d0 comp rx=0 tx=0).stop 2024-09-10T09:48:31.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:31.236+0000 7fdd002c0740 1 -- 172.21.15.5:0/74707059 shutdown_connections 2024-09-10T09:48:31.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:31.236+0000 7fdd002c0740 1 --2- 172.21.15.5:0/74707059 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562f97138580 0x562f97150540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:31.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:31.236+0000 7fdd002c0740 1 --2- 172.21.15.5:0/74707059 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562f971253d0 0x562f97150000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:31.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:31.236+0000 7fdd002c0740 1 --2- 172.21.15.5:0/74707059 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562f97124ab0 0x562f97135b80 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:31.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:31.236+0000 7fdd002c0740 1 -- 172.21.15.5:0/74707059 >> 172.21.15.5:0/74707059 conn(0x562f97121120 msgr2=0x562f97135470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:48:31.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:31.236+0000 7fdd002c0740 1 -- 172.21.15.5:0/74707059 shutdown_connections 2024-09-10T09:48:31.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:31.237+0000 7fdd002c0740 1 -- 172.21.15.5:0/74707059 wait complete. 2024-09-10T09:48:31.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:31.237+0000 7fdd002c0740 1 -- start start 2024-09-10T09:48:31.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:31.237+0000 7fdd002c0740 1 Processor -- start 2024-09-10T09:48:31.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:31.238+0000 7fdd002c0740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562f97124ab0 0x562f97132480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:48:31.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:31.238+0000 7fdd002c0740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562f971253d0 0x562f971329c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:48:31.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:31.238+0000 7fdd002c0740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562f97138580 0x562f97132f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:48:31.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:31.238+0000 7fdd002c0740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fdcf81ae190 con 0x562f971253d0 2024-09-10T09:48:31.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:31.238+0000 7fdd002c0740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fdcf81adac0 con 0x562f97138580 2024-09-10T09:48:31.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:31.238+0000 7fdd002c0740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fdcf81aad00 con 0x562f97124ab0 2024-09-10T09:48:31.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:31.238+0000 7fdd002c0740 10 client.0 _ll_register_callbacks cb 0x7fdce80092a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:48:31.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:31.238+0000 7fdcf6a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562f97124ab0 0x562f97132480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:48:31.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:31.238+0000 7fdcf6a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562f97124ab0 0x562f97132480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:45808/0 (socket says 172.21.15.5:45808) 2024-09-10T09:48:31.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:31.238+0000 7fdcf6a00640 1 -- 172.21.15.5:0/660540126 learned_addr learned my addr 172.21.15.5:0/660540126 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:48:31.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:31.238+0000 7fdcf6000640 1 --2- 172.21.15.5:0/660540126 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562f971253d0 0x562f971329c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:48:31.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:31.238+0000 7fdcf7400640 1 --2- 172.21.15.5:0/660540126 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562f97138580 0x562f97132f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:48:31.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:31.239+0000 7fdcf7400640 1 -- 172.21.15.5:0/660540126 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562f97124ab0 msgr2=0x562f97132480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:48:31.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:31.239+0000 7fdcf7400640 1 --2- 172.21.15.5:0/660540126 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562f97124ab0 0x562f97132480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:31.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:31.239+0000 7fdcf7400640 1 -- 172.21.15.5:0/660540126 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562f971253d0 msgr2=0x562f971329c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:48:31.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:31.239+0000 7fdcf7400640 1 --2- 172.21.15.5:0/660540126 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562f971253d0 0x562f971329c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:31.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:31.239+0000 7fdcf7400640 1 -- 172.21.15.5:0/660540126 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562f971b1fd0 con 0x562f97138580 2024-09-10T09:48:31.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:31.239+0000 7fdcf7400640 1 --2- 172.21.15.5:0/660540126 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562f97138580 0x562f97132f00 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fdcf0013390 tx=0x7fdcf0004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ac148d49c2403172 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:48:31.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:31.239+0000 7fdce0c00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-09-10T09:48:31.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:31.240+0000 7fdce0c00640 1 -- 172.21.15.5:0/660540126 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcf0026020 con 0x562f97138580 2024-09-10T09:48:31.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:31.240+0000 7fdce0c00640 1 -- 172.21.15.5:0/660540126 <== mon.2 v2:172.21.15.102:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fdcf0011cc0 con 0x562f97138580 2024-09-10T09:48:31.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:31.240+0000 7fdd002c0740 1 -- 172.21.15.5:0/660540126 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdce809f680 con 0x562f97138580 2024-09-10T09:48:31.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:31.240+0000 7fdce0c00640 1 -- 172.21.15.5:0/660540126 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fdcf0019070 con 0x562f97138580 2024-09-10T09:48:31.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:31.240+0000 7fdd002c0740 10 client.27377 fetch_fsmap learned FSMap version 36 2024-09-10T09:48:31.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:31.240+0000 7fdd002c0740 1 -- 172.21.15.5:0/660540126 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x562f971b32e0 con 0x562f97138580 2024-09-10T09:48:31.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:31.240+0000 7fdce0c00640 1 -- 172.21.15.5:0/660540126 <== mon.2 v2:172.21.15.102:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdce809f680 con 0x562f97138580 2024-09-10T09:48:31.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:31.241+0000 7fdce0c00640 1 -- 172.21.15.5:0/660540126 <== mon.2 v2:172.21.15.102:3300/0 5 ==== fsmap(e 36) ==== 8370+0+0 (secure 0 0 0) 0x7fdcf001f050 con 0x562f97138580 2024-09-10T09:48:31.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:31.241+0000 7fdd002c0740 10 client.27377 fetch_fsmap finished waiting for FSMap version 36 2024-09-10T09:48:31.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:31.241+0000 7fdd002c0740 10 client.27377 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T09:48:31.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:31.241+0000 7fdd002c0740 20 client.27377 populate_metadata read hostname 'smithi005' 2024-09-10T09:48:31.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:31.241+0000 7fdd002c0740 1 --2- 172.21.15.5:0/660540126 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x562f971b3a90 0x562f971b5eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:48:31.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:31.241+0000 7fdd002c0740 4 client.27377 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T09:48:31.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:31.241+0000 7fdd002c0740 1 -- 172.21.15.5:0/660540126 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "scrub status"}) -- 0x562f97092e60 con 0x562f971b3a90 2024-09-10T09:48:31.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:31.241+0000 7fdcf6a00640 1 --2- 172.21.15.5:0/660540126 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x562f971b3a90 0x562f971b5eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:48:31.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:31.241+0000 7fdcf6a00640 1 --2- 172.21.15.5:0/660540126 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x562f971b3a90 0x562f971b5eb0 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=857d22431eb684d6 server_cookie=4bbd0f9ba3a469bc in_seq=0 out_seq=0 2024-09-10T09:48:31.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:31.241+0000 7fdce0c00640 10 client.27377 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T09:48:31.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:31.242+0000 7fdce0c00640 1 -- 172.21.15.5:0/660540126 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x562f97092e60 con 0x562f971b3a90 2024-09-10T09:48:31.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:31.242+0000 7fdce0c00640 10 client.27377 handle_command_reply: tid=0 2024-09-10T09:48:31.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:31.242+0000 7fdd002c0740 1 client.27377 shutdown 2024-09-10T09:48:31.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:31.242+0000 7fdd002c0740 1 -- 172.21.15.5:0/660540126 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562f97138580 msgr2=0x562f97132f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:48:31.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:31.242+0000 7fdd002c0740 1 --2- 172.21.15.5:0/660540126 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562f97138580 0x562f97132f00 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fdcf0013390 tx=0x7fdcf0004a40 comp rx=0 tx=0).stop 2024-09-10T09:48:31.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:31.242+0000 7fdd002c0740 1 -- 172.21.15.5:0/660540126 shutdown_connections 2024-09-10T09:48:31.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:31.242+0000 7fdd002c0740 1 --2- 172.21.15.5:0/660540126 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x562f971b3a90 0x562f971b5eb0 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:31.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:31.242+0000 7fdd002c0740 1 --2- 172.21.15.5:0/660540126 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562f97138580 0x562f97132f00 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:31.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:31.242+0000 7fdd002c0740 1 --2- 172.21.15.5:0/660540126 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562f971253d0 0x562f971329c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:31.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:31.242+0000 7fdd002c0740 1 --2- 172.21.15.5:0/660540126 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562f97124ab0 0x562f97132480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:31.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:31.242+0000 7fdce0c00640 0 client.27377 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T09:48:31.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:31.242+0000 7fdd002c0740 1 -- 172.21.15.5:0/660540126 >> 172.21.15.5:0/660540126 conn(0x562f97121120 msgr2=0x562f97135c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:48:31.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:31.242+0000 7fdd002c0740 1 -- 172.21.15.5:0/660540126 shutdown_connections 2024-09-10T09:48:31.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:31.242+0000 7fdd002c0740 1 -- 172.21.15.5:0/660540126 wait complete. 2024-09-10T09:48:31.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:31.243+0000 7fdd002c0740 20 client.27377 trim_cache size 0 max 16384 2024-09-10T09:48:31.246 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-10T09:48:31.246 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "no active scrubs running", 2024-09-10T09:48:31.246 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": {} 2024-09-10T09:48:31.246 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-10T09:48:31.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:31.243+0000 7fdcfd800640 1 -- 172.21.15.5:0/1483862051 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fdcdc0a2c00 msgr2=0x7fdcdc0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:48:31.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:31.243+0000 7fdcfd800640 1 --2- 172.21.15.5:0/1483862051 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fdcdc0a2c00 0x7fdcdc0a50c0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fdcf0005fd0 tx=0x7fdcf0004ab0 comp rx=0 tx=0).stop 2024-09-10T09:48:31.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:31.243+0000 7fdcfd800640 1 -- 172.21.15.5:0/1483862051 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdcf8104010 msgr2=0x7fdcf81a0570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:48:31.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:31.243+0000 7fdcfd800640 1 --2- 172.21.15.5:0/1483862051 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdcf8104010 0x7fdcf81a0570 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fdce800b450 tx=0x7fdce800b910 comp rx=0 tx=0).stop 2024-09-10T09:48:31.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:31.243+0000 7fdcfd800640 1 -- 172.21.15.5:0/1483862051 shutdown_connections 2024-09-10T09:48:31.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:31.243+0000 7fdcfd800640 1 --2- 172.21.15.5:0/1483862051 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fdcdc0a2c00 0x7fdcdc0a50c0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:31.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:31.243+0000 7fdcfd800640 1 --2- 172.21.15.5:0/1483862051 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdcf8109140 0x7fdcf819a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:31.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:31.243+0000 7fdcfd800640 1 --2- 172.21.15.5:0/1483862051 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdcf81049c0 0x7fdcf81a0ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:31.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:31.243+0000 7fdcfd800640 1 --2- 172.21.15.5:0/1483862051 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdcf8104010 0x7fdcf81a0570 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:31.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:31.243+0000 7fdcfd800640 1 -- 172.21.15.5:0/1483862051 >> 172.21.15.5:0/1483862051 conn(0x7fdcf80ffa50 msgr2=0x7fdcf8100e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:48:31.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:31.244+0000 7fdcfd800640 1 -- 172.21.15.5:0/1483862051 shutdown_connections 2024-09-10T09:48:31.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:31.244+0000 7fdcfd800640 1 -- 172.21.15.5:0/1483862051 wait complete. 2024-09-10T09:48:31.685 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:48:31 smithi102 ceph-mon[35442]: pgmap v698: 129 pgs: 129 active+clean; 242 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 1.7 MiB/s wr, 219 op/s 2024-09-10T09:48:31.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:48:31 smithi005 ceph-mon[30350]: pgmap v698: 129 pgs: 129 active+clean; 242 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 1.7 MiB/s wr, 219 op/s 2024-09-10T09:48:31.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:48:31 smithi159 ceph-mon[35513]: pgmap v698: 129 pgs: 129 active+clean; 242 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 1.7 MiB/s wr, 219 op/s 2024-09-10T09:48:32.101 INFO:tasks.workunit.client.0.smithi005.stdout: 1 635707 6.46 MB/sec execute 373 sec latency 1454.958 ms 2024-09-10T09:48:32.771 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-10T09:48:32.771 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T09:48:33.101 INFO:tasks.workunit.client.0.smithi005.stdout: 1 637298 6.46 MB/sec execute 374 sec latency 1527.756 ms 2024-09-10T09:48:33.177 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:48:33.640 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:48:33 smithi005 ceph-mon[30350]: pgmap v699: 129 pgs: 129 active+clean; 242 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 2.0 MiB/s wr, 223 op/s 2024-09-10T09:48:33.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:33.779+0000 7f32cb400640 1 -- 172.21.15.5:0/3399104997 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32c40319b0 con 0x7f32cc109120 2024-09-10T09:48:33.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:33.780+0000 7f32d2e00640 1 -- 172.21.15.5:0/3399104997 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f32cc109120 msgr2=0x7f32cc110ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:48:33.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:33.780+0000 7f32d2e00640 1 --2- 172.21.15.5:0/3399104997 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f32cc109120 0x7f32cc110ca0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f32c4009900 tx=0x7f32c402a3f0 comp rx=0 tx=0).stop 2024-09-10T09:48:33.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:33.780+0000 7f32d2e00640 1 -- 172.21.15.5:0/3399104997 shutdown_connections 2024-09-10T09:48:33.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:33.780+0000 7f32d2e00640 1 --2- 172.21.15.5:0/3399104997 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f32cc109120 0x7f32cc110ca0 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:33.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:33.780+0000 7f32d2e00640 1 --2- 172.21.15.5:0/3399104997 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f32cc1049a0 0x7f32cc1089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:33.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:33.780+0000 7f32d2e00640 1 --2- 172.21.15.5:0/3399104997 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f32cc103ff0 0x7f32cc1043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:33.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:33.780+0000 7f32d2e00640 1 -- 172.21.15.5:0/3399104997 >> 172.21.15.5:0/3399104997 conn(0x7f32cc0ffa90 msgr2=0x7f32cc101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:48:33.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:33.780+0000 7f32d2e00640 1 -- 172.21.15.5:0/3399104997 shutdown_connections 2024-09-10T09:48:33.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:33.780+0000 7f32d2e00640 1 -- 172.21.15.5:0/3399104997 wait complete. 2024-09-10T09:48:33.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:33.781+0000 7f32d2e00640 1 Processor -- start 2024-09-10T09:48:33.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:33.781+0000 7f32d2e00640 1 -- start start 2024-09-10T09:48:33.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:33.781+0000 7f32d2e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f32cc103ff0 0x7f32cc19a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:48:33.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:33.782+0000 7f32d2e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f32cc1049a0 0x7f32cc19aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:48:33.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:33.782+0000 7f32d2e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f32cc109120 0x7f32cc19ee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:48:33.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:33.782+0000 7f32d2e00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f32cc114180 con 0x7f32cc1049a0 2024-09-10T09:48:33.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:33.782+0000 7f32d2e00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f32cc114000 con 0x7f32cc109120 2024-09-10T09:48:33.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:33.782+0000 7f32d2e00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f32cc114300 con 0x7f32cc103ff0 2024-09-10T09:48:33.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:33.782+0000 7f32cbe00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f32cc103ff0 0x7f32cc19a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:48:33.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:33.782+0000 7f32c3400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f32cc1049a0 0x7f32cc19aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:48:33.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:33.782+0000 7f32cbe00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f32cc103ff0 0x7f32cc19a590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:45838/0 (socket says 172.21.15.5:45838) 2024-09-10T09:48:33.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:33.782+0000 7f32cbe00640 1 -- 172.21.15.5:0/2407917382 learned_addr learned my addr 172.21.15.5:0/2407917382 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:48:33.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:33.782+0000 7f32d0e00640 1 --2- 172.21.15.5:0/2407917382 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f32cc109120 0x7f32cc19ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:48:33.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:33.782+0000 7f32cbe00640 1 -- 172.21.15.5:0/2407917382 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f32cc109120 msgr2=0x7f32cc19ee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:48:33.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:33.782+0000 7f32cbe00640 1 --2- 172.21.15.5:0/2407917382 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f32cc109120 0x7f32cc19ee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:33.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:33.782+0000 7f32cbe00640 1 -- 172.21.15.5:0/2407917382 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f32cc1049a0 msgr2=0x7f32cc19aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:48:33.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:33.782+0000 7f32cbe00640 1 --2- 172.21.15.5:0/2407917382 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f32cc1049a0 0x7f32cc19aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:33.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:33.782+0000 7f32cbe00640 1 -- 172.21.15.5:0/2407917382 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32cc19f540 con 0x7f32cc103ff0 2024-09-10T09:48:33.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:33.783+0000 7f32cbe00640 1 --2- 172.21.15.5:0/2407917382 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f32cc103ff0 0x7f32cc19a590 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f32bc00ebc0 tx=0x7f32bc00c6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:48:33.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:33.783+0000 7f32c9600640 1 -- 172.21.15.5:0/2407917382 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32bc010630 con 0x7f32cc103ff0 2024-09-10T09:48:33.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:33.783+0000 7f32c9600640 1 -- 172.21.15.5:0/2407917382 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f32bc010cb0 con 0x7f32cc103ff0 2024-09-10T09:48:33.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:33.783+0000 7f32d2e00640 1 -- 172.21.15.5:0/2407917382 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32cc19f830 con 0x7f32cc103ff0 2024-09-10T09:48:33.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:33.783+0000 7f32c9600640 1 -- 172.21.15.5:0/2407917382 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32bc018bd0 con 0x7f32cc103ff0 2024-09-10T09:48:33.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:33.783+0000 7f32d2e00640 1 -- 172.21.15.5:0/2407917382 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32cc1a7110 con 0x7f32cc103ff0 2024-09-10T09:48:33.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:33.784+0000 7f32d2e00640 1 -- 172.21.15.5:0/2407917382 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f32cc1043d0 con 0x7f32cc103ff0 2024-09-10T09:48:33.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:33.786+0000 7f32c9600640 1 -- 172.21.15.5:0/2407917382 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f32bc0107d0 con 0x7f32cc103ff0 2024-09-10T09:48:33.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:33.787+0000 7f32c9600640 1 --2- 172.21.15.5:0/2407917382 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f32b00a2c00 0x7f32b00a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:48:33.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:33.788+0000 7f32c9600640 1 -- 172.21.15.5:0/2407917382 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f32bc0cfae0 con 0x7f32cc103ff0 2024-09-10T09:48:33.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:33.788+0000 7f32c3400640 1 --2- 172.21.15.5:0/2407917382 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f32b00a2c00 0x7f32b00a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:48:33.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:33.789+0000 7f32c3400640 1 --2- 172.21.15.5:0/2407917382 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f32b00a2c00 0x7f32b00a50c0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f32b40095f0 tx=0x7f32b4005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:48:33.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:33.792+0000 7f32c9600640 1 -- 172.21.15.5:0/2407917382 <== mon.1 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f32bc09c5f0 con 0x7f32cc103ff0 2024-09-10T09:48:33.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:48:33 smithi159 ceph-mon[35513]: pgmap v699: 129 pgs: 129 active+clean; 242 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 2.0 MiB/s wr, 223 op/s 2024-09-10T09:48:33.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:48:33 smithi102 ceph-mon[35442]: pgmap v699: 129 pgs: 129 active+clean; 242 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 2.0 MiB/s wr, 223 op/s 2024-09-10T09:48:34.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:34.047+0000 7f32d2e00640 1 -- 172.21.15.5:0/2407917382 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f32cc1a7440 con 0x7f32cc103ff0 2024-09-10T09:48:34.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:34.048+0000 7f32c9600640 1 -- 172.21.15.5:0/2407917382 <== mon.1 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) ==== 94+0+11969 (secure 0 0 0) 0x7f32bc09bef0 con 0x7f32cc103ff0 2024-09-10T09:48:34.050 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-10T09:48:34.050 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":36,"btime":"2024-09-10T09:48:07:349075+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14454,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/2970594311","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2970594311},{"type":"v1","addr":"172.21.15.5:6835","nonce":2970594311}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14514,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6839/3616502174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":3616502174},{"type":"v1","addr":"172.21.15.5:6839","nonce":3616502174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":36,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T09:35:29.626287+0000","modified":"2024-09-10T09:48:06.970127+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24542,"mds_1":24535,"mds_2":24511,"mds_3":24506,"mds_4":24443},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14520":{"gid":14520,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/2073599842","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":2073599842},{"type":"v1","addr":"172.21.15.5:6841","nonce":2073599842}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"k","rank":4,"incarnation":26,"state":"up:active","state_seq":24,"addr":"172.21.15.102:6833/2457007427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":2457007427},{"type":"v1","addr":"172.21.15.102:6833","nonce":2457007427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/356445159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":356445159},{"type":"v1","addr":"172.21.15.159:6833","nonce":356445159}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6837/2894314507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":2894314507},{"type":"v1","addr":"172.21.15.5:6837","nonce":2894314507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.102:6835/1647549111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":1647549111},{"type":"v1","addr":"172.21.15.102:6835","nonce":1647549111}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6835/952657242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":952657242},{"type":"v1","addr":"172.21.15.159:6835","nonce":952657242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":19,"addr":"172.21.15.102:6837/2689776764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":2689776764},{"type":"v1","addr":"172.21.15.102:6837","nonce":2689776764}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.159:6837/620627639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":620627639},{"type":"v1","addr":"172.21.15.159:6837","nonce":620627639}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24535":{"gid":24535,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":18,"addr":"172.21.15.159:6839/2073739599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":2073739599},{"type":"v1","addr":"172.21.15.159:6839","nonce":2073739599}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24542":{"gid":24542,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.102:6839/335409749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6838","nonce":335409749},{"type":"v1","addr":"172.21.15.102:6839","nonce":335409749}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24542,"qdb_cluster":[24542,24535,24511,24506,24443]},"id":1}]} 2024-09-10T09:48:34.050 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 36 2024-09-10T09:48:34.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:34.052+0000 7f32d2e00640 1 -- 172.21.15.5:0/2407917382 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f32b00a2c00 msgr2=0x7f32b00a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:48:34.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:34.052+0000 7f32d2e00640 1 --2- 172.21.15.5:0/2407917382 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f32b00a2c00 0x7f32b00a50c0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f32b40095f0 tx=0x7f32b4005f90 comp rx=0 tx=0).stop 2024-09-10T09:48:34.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:34.052+0000 7f32d2e00640 1 -- 172.21.15.5:0/2407917382 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f32cc103ff0 msgr2=0x7f32cc19a590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:48:34.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:34.052+0000 7f32d2e00640 1 --2- 172.21.15.5:0/2407917382 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f32cc103ff0 0x7f32cc19a590 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f32bc00ebc0 tx=0x7f32bc00c6a0 comp rx=0 tx=0).stop 2024-09-10T09:48:34.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:34.052+0000 7f32d2e00640 1 -- 172.21.15.5:0/2407917382 shutdown_connections 2024-09-10T09:48:34.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:34.052+0000 7f32d2e00640 1 --2- 172.21.15.5:0/2407917382 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f32b00a2c00 0x7f32b00a50c0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:34.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:34.052+0000 7f32d2e00640 1 --2- 172.21.15.5:0/2407917382 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f32cc109120 0x7f32cc19ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:34.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:34.052+0000 7f32d2e00640 1 --2- 172.21.15.5:0/2407917382 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f32cc1049a0 0x7f32cc19aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:34.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:34.052+0000 7f32d2e00640 1 --2- 172.21.15.5:0/2407917382 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f32cc103ff0 0x7f32cc19a590 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:34.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:34.052+0000 7f32d2e00640 1 -- 172.21.15.5:0/2407917382 >> 172.21.15.5:0/2407917382 conn(0x7f32cc0ffa90 msgr2=0x7f32cc101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:48:34.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:34.052+0000 7f32d2e00640 1 -- 172.21.15.5:0/2407917382 shutdown_connections 2024-09-10T09:48:34.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:34.052+0000 7f32d2e00640 1 -- 172.21.15.5:0/2407917382 wait complete. 2024-09-10T09:48:34.101 INFO:tasks.workunit.client.0.smithi005.stdout: 1 639721 6.48 MB/sec execute 375 sec latency 11.497 ms 2024-09-10T09:48:34.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:48:34 smithi005 ceph-mon[30350]: from='client.? 172.21.15.5:0/2407917382' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:48:34.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:48:34 smithi159 ceph-mon[35513]: from='client.? 172.21.15.5:0/2407917382' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:48:34.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:48:34 smithi102 ceph-mon[35442]: from='client.? 172.21.15.5:0/2407917382' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:48:35.101 INFO:tasks.workunit.client.0.smithi005.stdout: 1 642031 6.49 MB/sec execute 376 sec latency 25.726 ms 2024-09-10T09:48:35.187 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-10T09:48:35.584 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:48:35.728 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:48:35 smithi005 ceph-mon[30350]: pgmap v700: 129 pgs: 129 active+clean; 241 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 1.9 MiB/s wr, 186 op/s 2024-09-10T09:48:35.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:48:35 smithi159 ceph-mon[35513]: pgmap v700: 129 pgs: 129 active+clean; 241 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 1.9 MiB/s wr, 186 op/s 2024-09-10T09:48:35.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:48:35 smithi102 ceph-mon[35442]: pgmap v700: 129 pgs: 129 active+clean; 241 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 1.9 MiB/s wr, 186 op/s 2024-09-10T09:48:36.101 INFO:tasks.workunit.client.0.smithi005.stdout: 1 643266 6.50 MB/sec execute 377 sec latency 241.163 ms 2024-09-10T09:48:37.101 INFO:tasks.workunit.client.0.smithi005.stdout: 1 643266 6.48 MB/sec execute 378 sec latency 1241.216 ms 2024-09-10T09:48:37.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:37.234+0000 7fb6ad000640 1 -- 172.21.15.5:0/1787499672 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb6a8074e30 msgr2=0x7fb6a810df00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:48:37.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:37.234+0000 7fb6ad000640 1 --2- 172.21.15.5:0/1787499672 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb6a8074e30 0x7fb6a810df00 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fb694009900 tx=0x7fb69402a480 comp rx=0 tx=0).stop 2024-09-10T09:48:37.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:37.234+0000 7fb6ad000640 1 -- 172.21.15.5:0/1787499672 shutdown_connections 2024-09-10T09:48:37.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:37.234+0000 7fb6ad000640 1 --2- 172.21.15.5:0/1787499672 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb6a810e440 0x7fb6a8110830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:37.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:37.234+0000 7fb6ad000640 1 --2- 172.21.15.5:0/1787499672 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb6a8074e30 0x7fb6a810df00 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:37.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:37.234+0000 7fb6ad000640 1 --2- 172.21.15.5:0/1787499672 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb6a8074510 0x7fb6a80748f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:37.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:37.234+0000 7fb6ad000640 1 -- 172.21.15.5:0/1787499672 >> 172.21.15.5:0/1787499672 conn(0x7fb6a80fd8c0 msgr2=0x7fb6a80ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:48:37.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:37.234+0000 7fb6ad000640 1 -- 172.21.15.5:0/1787499672 shutdown_connections 2024-09-10T09:48:37.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:37.234+0000 7fb6ad000640 1 -- 172.21.15.5:0/1787499672 wait complete. 2024-09-10T09:48:37.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:37.234+0000 7fb6ad000640 1 Processor -- start 2024-09-10T09:48:37.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:37.234+0000 7fb6ad000640 1 -- start start 2024-09-10T09:48:37.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:37.234+0000 7fb6ad000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb6a8074510 0x7fb6a819a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:48:37.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:37.234+0000 7fb6ad000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb6a8074e30 0x7fb6a819ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:48:37.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:37.234+0000 7fb6ad000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb6a810e440 0x7fb6a819efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:48:37.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:37.234+0000 7fb6ad000640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fb6a81141b0 con 0x7fb6a8074510 2024-09-10T09:48:37.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:37.234+0000 7fb6ad000640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fb6a8114030 con 0x7fb6a8074e30 2024-09-10T09:48:37.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:37.234+0000 7fb6ad000640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fb6a8114330 con 0x7fb6a810e440 2024-09-10T09:48:37.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:37.239+0000 7fb6a5e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb6a8074510 0x7fb6a819a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:48:37.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:37.239+0000 7fb6a5400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb6a8074e30 0x7fb6a819ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:48:37.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:37.239+0000 7fb6a5400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb6a8074e30 0x7fb6a819ac30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:33984/0 (socket says 172.21.15.5:33984) 2024-09-10T09:48:37.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:37.239+0000 7fb6a5400640 1 -- 172.21.15.5:0/2734643625 learned_addr learned my addr 172.21.15.5:0/2734643625 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:48:37.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:37.239+0000 7fb6a6800640 1 --2- 172.21.15.5:0/2734643625 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb6a810e440 0x7fb6a819efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:48:37.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:37.239+0000 7fb6a5400640 1 -- 172.21.15.5:0/2734643625 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb6a810e440 msgr2=0x7fb6a819efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:48:37.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:37.239+0000 7fb6a5400640 1 --2- 172.21.15.5:0/2734643625 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb6a810e440 0x7fb6a819efc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:37.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:37.239+0000 7fb6a5400640 1 -- 172.21.15.5:0/2734643625 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb6a8074510 msgr2=0x7fb6a819a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:48:37.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:37.239+0000 7fb6a5400640 1 --2- 172.21.15.5:0/2734643625 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb6a8074510 0x7fb6a819a6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:37.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:37.239+0000 7fb6a5400640 1 -- 172.21.15.5:0/2734643625 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6a819f6a0 con 0x7fb6a8074e30 2024-09-10T09:48:37.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:37.239+0000 7fb6a5e00640 1 --2- 172.21.15.5:0/2734643625 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb6a8074510 0x7fb6a819a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:48:37.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:37.239+0000 7fb6a5400640 1 --2- 172.21.15.5:0/2734643625 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb6a8074e30 0x7fb6a819ac30 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fb694004770 tx=0x7fb694003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:48:37.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:37.239+0000 7fb69ea00640 1 -- 172.21.15.5:0/2734643625 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb69403f070 con 0x7fb6a8074e30 2024-09-10T09:48:37.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:37.239+0000 7fb69ea00640 1 -- 172.21.15.5:0/2734643625 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb694031ca0 con 0x7fb6a8074e30 2024-09-10T09:48:37.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:37.239+0000 7fb6ad000640 1 -- 172.21.15.5:0/2734643625 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6a819f930 con 0x7fb6a8074e30 2024-09-10T09:48:37.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:37.239+0000 7fb69ea00640 1 -- 172.21.15.5:0/2734643625 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb694035030 con 0x7fb6a8074e30 2024-09-10T09:48:37.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:37.239+0000 7fb6ad000640 1 -- 172.21.15.5:0/2734643625 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6a81a7270 con 0x7fb6a8074e30 2024-09-10T09:48:37.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:37.242+0000 7fb6afbba740 1 Processor -- start 2024-09-10T09:48:37.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:37.242+0000 7fb6afbba740 1 -- start start 2024-09-10T09:48:37.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:37.242+0000 7fb69ea00640 1 -- 172.21.15.5:0/2734643625 <== mon.2 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7fb694042070 con 0x7fb6a8074e30 2024-09-10T09:48:37.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:37.242+0000 7fb6afbba740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5633f183ec70 0x5633f183f050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:48:37.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:37.242+0000 7fb6afbba740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5633f183f590 0x5633f1849910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:48:37.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:37.242+0000 7fb6afbba740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5633f1849e50 0x5633f184c240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:48:37.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:37.242+0000 7fb6afbba740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5633f17a53e0 con 0x5633f1849e50 2024-09-10T09:48:37.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:37.242+0000 7fb6afbba740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x5633f174cd80 con 0x5633f183ec70 2024-09-10T09:48:37.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:37.242+0000 7fb6afbba740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5633f17a13a0 con 0x5633f183f590 2024-09-10T09:48:37.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:37.242+0000 7fb6a6800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5633f183ec70 0x5633f183f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:48:37.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:37.242+0000 7fb6a6800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5633f183ec70 0x5633f183f050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:33994/0 (socket says 172.21.15.5:33994) 2024-09-10T09:48:37.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:37.242+0000 7fb6a6800640 1 -- 172.21.15.5:0/1612193417 learned_addr learned my addr 172.21.15.5:0/1612193417 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:48:37.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:37.242+0000 7fb6a5400640 1 --2- 172.21.15.5:0/1612193417 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5633f1849e50 0x5633f184c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:48:37.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:37.242+0000 7fb6a5e00640 1 --2- 172.21.15.5:0/1612193417 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5633f183f590 0x5633f1849910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:48:37.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:37.242+0000 7fb6a6800640 1 -- 172.21.15.5:0/1612193417 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5633f183f590 msgr2=0x5633f1849910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:48:37.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:37.242+0000 7fb6a6800640 1 --2- 172.21.15.5:0/1612193417 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5633f183f590 0x5633f1849910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:37.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:37.242+0000 7fb6a6800640 1 -- 172.21.15.5:0/1612193417 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5633f1849e50 msgr2=0x5633f184c240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:48:37.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:37.242+0000 7fb6a6800640 1 --2- 172.21.15.5:0/1612193417 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5633f1849e50 0x5633f184c240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:37.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:37.242+0000 7fb6a6800640 1 -- 172.21.15.5:0/1612193417 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5633f176bae0 con 0x5633f183ec70 2024-09-10T09:48:37.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:37.242+0000 7fb6a5e00640 1 --2- 172.21.15.5:0/1612193417 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5633f183f590 0x5633f1849910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:48:37.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:37.242+0000 7fb6a6800640 1 --2- 172.21.15.5:0/1612193417 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5633f183ec70 0x5633f183f050 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fb6a0009f00 tx=0x7fb6a00107a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a864da4dc8df7cd2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:48:37.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:37.242+0000 7fb693400640 1 -- 172.21.15.5:0/1612193417 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6a0003b70 con 0x5633f183ec70 2024-09-10T09:48:37.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:37.242+0000 7fb693400640 1 -- 172.21.15.5:0/1612193417 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb6a0003d10 con 0x5633f183ec70 2024-09-10T09:48:37.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:37.242+0000 7fb693400640 1 -- 172.21.15.5:0/1612193417 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6a0011ab0 con 0x5633f183ec70 2024-09-10T09:48:37.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:37.247+0000 7fb69ea00640 1 --2- 172.21.15.5:0/2734643625 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fb68c0a2c00 0x7fb68c0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:48:37.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:37.247+0000 7fb69ea00640 1 -- 172.21.15.5:0/2734643625 <== mon.2 v2:172.21.15.102:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fb6940efaa0 con 0x7fb6a8074e30 2024-09-10T09:48:37.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:37.247+0000 7fb6afbba740 1 -- 172.21.15.5:0/1612193417 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5633f183ec70 msgr2=0x5633f183f050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:48:37.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:37.247+0000 7fb6afbba740 1 --2- 172.21.15.5:0/1612193417 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5633f183ec70 0x5633f183f050 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fb6a0009f00 tx=0x7fb6a00107a0 comp rx=0 tx=0).stop 2024-09-10T09:48:37.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:37.247+0000 7fb6a5e00640 1 --2- 172.21.15.5:0/2734643625 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fb68c0a2c00 0x7fb68c0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:48:37.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:37.247+0000 7fb6afbba740 1 -- 172.21.15.5:0/1612193417 shutdown_connections 2024-09-10T09:48:37.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:37.247+0000 7fb6afbba740 1 --2- 172.21.15.5:0/1612193417 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5633f1849e50 0x5633f184c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:37.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:37.247+0000 7fb6afbba740 1 --2- 172.21.15.5:0/1612193417 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5633f183f590 0x5633f1849910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:37.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:37.247+0000 7fb6afbba740 1 --2- 172.21.15.5:0/1612193417 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5633f183ec70 0x5633f183f050 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:37.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:37.247+0000 7fb6afbba740 1 -- 172.21.15.5:0/1612193417 >> 172.21.15.5:0/1612193417 conn(0x5633f183b200 msgr2=0x5633f183b5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:48:37.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:37.247+0000 7fb6afbba740 1 -- 172.21.15.5:0/1612193417 shutdown_connections 2024-09-10T09:48:37.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:37.247+0000 7fb6afbba740 1 -- 172.21.15.5:0/1612193417 wait complete. 2024-09-10T09:48:37.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:37.247+0000 7fb6a5e00640 1 --2- 172.21.15.5:0/2734643625 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fb68c0a2c00 0x7fb68c0a50c0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x5633f183c060 tx=0x7fb698005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:48:37.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:37.247+0000 7fb6afbba740 1 -- start start 2024-09-10T09:48:37.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:37.247+0000 7fb6afbba740 1 Processor -- start 2024-09-10T09:48:37.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:37.247+0000 7fb6afbba740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5633f183ec70 0x5633f1846ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:48:37.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:37.247+0000 7fb6afbba740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5633f183f590 0x5633f18471e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:48:37.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:37.247+0000 7fb6afbba740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5633f1849e50 0x5633f1847720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:48:37.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:37.247+0000 7fb6afbba740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5633f184ca20 con 0x5633f183ec70 2024-09-10T09:48:37.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:37.247+0000 7fb6afbba740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x5633f17600f0 con 0x5633f1849e50 2024-09-10T09:48:37.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:37.247+0000 7fb6afbba740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5633f184cba0 con 0x5633f183f590 2024-09-10T09:48:37.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:37.247+0000 7fb6afbba740 10 client.0 _ll_register_callbacks cb 0x5633f1747bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:48:37.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:37.247+0000 7fb6a6800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5633f1849e50 0x5633f1847720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:48:37.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:37.247+0000 7fb6a5e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5633f183ec70 0x5633f1846ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:48:37.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:37.247+0000 7fb6a5400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5633f183f590 0x5633f18471e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:48:37.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:37.247+0000 7fb6a5e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5633f183ec70 0x5633f1846ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50422/0 (socket says 172.21.15.5:50422) 2024-09-10T09:48:37.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:37.247+0000 7fb6a5e00640 1 -- 172.21.15.5:0/2059258003 learned_addr learned my addr 172.21.15.5:0/2059258003 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:48:37.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:37.247+0000 7fb6a5e00640 1 -- 172.21.15.5:0/2059258003 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5633f183f590 msgr2=0x5633f18471e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:48:37.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:37.247+0000 7fb6a5e00640 1 --2- 172.21.15.5:0/2059258003 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5633f183f590 0x5633f18471e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:37.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:37.247+0000 7fb6a5e00640 1 -- 172.21.15.5:0/2059258003 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5633f1849e50 msgr2=0x5633f1847720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:48:37.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:37.247+0000 7fb6a5e00640 1 --2- 172.21.15.5:0/2059258003 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5633f1849e50 0x5633f1847720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:37.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:37.247+0000 7fb6a5e00640 1 -- 172.21.15.5:0/2059258003 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5633f15163c0 con 0x5633f183ec70 2024-09-10T09:48:37.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:37.247+0000 7fb6a6800640 1 --2- 172.21.15.5:0/2059258003 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5633f1849e50 0x5633f1847720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:48:37.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:37.247+0000 7fb6a5e00640 1 --2- 172.21.15.5:0/2059258003 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5633f183ec70 0x5633f1846ca0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fb698011c70 tx=0x7fb6980046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94811c3f7ec90cba server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:48:37.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:37.247+0000 7fb687e00640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-10T09:48:37.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:37.247+0000 7fb687e00640 1 -- 172.21.15.5:0/2059258003 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb698011ea0 con 0x5633f183ec70 2024-09-10T09:48:37.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:37.247+0000 7fb687e00640 1 -- 172.21.15.5:0/2059258003 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fb698004020 con 0x5633f183ec70 2024-09-10T09:48:37.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:37.247+0000 7fb6afbba740 1 -- 172.21.15.5:0/2059258003 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb6a0003d10 con 0x5633f183ec70 2024-09-10T09:48:37.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:37.247+0000 7fb687e00640 1 -- 172.21.15.5:0/2059258003 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fb698030050 con 0x5633f183ec70 2024-09-10T09:48:37.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:37.247+0000 7fb6afbba740 10 client.17397 fetch_fsmap learned FSMap version 36 2024-09-10T09:48:37.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:37.247+0000 7fb6afbba740 1 -- 172.21.15.5:0/2059258003 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x5633f1858a80 con 0x5633f183ec70 2024-09-10T09:48:37.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:37.247+0000 7fb687e00640 1 -- 172.21.15.5:0/2059258003 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb6a0003d10 con 0x5633f183ec70 2024-09-10T09:48:37.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:37.247+0000 7fb687e00640 1 -- 172.21.15.5:0/2059258003 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 36) ==== 8370+0+0 (secure 0 0 0) 0x7fb698028490 con 0x5633f183ec70 2024-09-10T09:48:37.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:37.247+0000 7fb6afbba740 10 client.17397 fetch_fsmap finished waiting for FSMap version 36 2024-09-10T09:48:37.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:37.247+0000 7fb6afbba740 10 client.17397 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-09-10T09:48:37.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:37.247+0000 7fb6afbba740 20 client.17397 populate_metadata read hostname 'smithi005' 2024-09-10T09:48:37.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:37.247+0000 7fb6afbba740 1 --2- 172.21.15.5:0/2059258003 >> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] conn(0x5633f1859f40 0x5633f185c350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:48:37.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:37.247+0000 7fb6afbba740 4 client.17397 mds_command: new command op to 24443 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:48:37.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:37.247+0000 7fb6afbba740 1 -- 172.21.15.5:0/2059258003 --> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5633f17b2950 con 0x5633f1859f40 2024-09-10T09:48:37.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:37.252+0000 7fb6a5400640 1 --2- 172.21.15.5:0/2059258003 >> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] conn(0x5633f1859f40 0x5633f185c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:48:37.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:37.252+0000 7fb6a5400640 1 --2- 172.21.15.5:0/2059258003 >> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] conn(0x5633f1859f40 0x5633f185c350 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=e2e48bcc899dc97c server_cookie=9388cc0825f57530 in_seq=0 out_seq=0 2024-09-10T09:48:37.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:37.252+0000 7fb687e00640 10 client.17397 ms_handle_connect on v2:172.21.15.102:6832/2457007427 2024-09-10T09:48:37.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:37.252+0000 7fb687e00640 1 -- 172.21.15.5:0/2059258003 <== mds.4 v2:172.21.15.102:6832/2457007427 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5633f17b2950 con 0x5633f1859f40 2024-09-10T09:48:37.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:37.252+0000 7fb687e00640 10 client.17397 handle_command_reply: tid=0 2024-09-10T09:48:37.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:37.252+0000 7fb6afbba740 1 client.17397 shutdown 2024-09-10T09:48:37.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:37.254+0000 7fb6afbba740 1 -- 172.21.15.5:0/2059258003 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5633f183ec70 msgr2=0x5633f1846ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:48:37.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:37.254+0000 7fb6afbba740 1 --2- 172.21.15.5:0/2059258003 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5633f183ec70 0x5633f1846ca0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fb698011c70 tx=0x7fb6980046c0 comp rx=0 tx=0).stop 2024-09-10T09:48:37.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:37.254+0000 7fb6afbba740 1 -- 172.21.15.5:0/2059258003 shutdown_connections 2024-09-10T09:48:37.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:37.254+0000 7fb6afbba740 1 --2- 172.21.15.5:0/2059258003 >> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] conn(0x5633f1859f40 0x5633f185c350 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:37.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:37.254+0000 7fb6afbba740 1 --2- 172.21.15.5:0/2059258003 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5633f1849e50 0x5633f1847720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:37.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:37.254+0000 7fb6afbba740 1 --2- 172.21.15.5:0/2059258003 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5633f183f590 0x5633f18471e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:37.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:37.254+0000 7fb6afbba740 1 --2- 172.21.15.5:0/2059258003 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5633f183ec70 0x5633f1846ca0 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:37.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:37.254+0000 7fb6afbba740 1 -- 172.21.15.5:0/2059258003 >> 172.21.15.5:0/2059258003 conn(0x5633f183b200 msgr2=0x5633f1840ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:48:37.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:37.254+0000 7fb687e00640 0 client.17397 ms_handle_reset on v2:172.21.15.102:6832/2457007427 2024-09-10T09:48:37.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:37.254+0000 7fb6afbba740 1 -- 172.21.15.5:0/2059258003 shutdown_connections 2024-09-10T09:48:37.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:37.254+0000 7fb6afbba740 1 -- 172.21.15.5:0/2059258003 wait complete. 2024-09-10T09:48:37.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:37.254+0000 7fb6afbba740 20 client.17397 trim_cache size 0 max 16384 2024-09-10T09:48:37.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:37.265+0000 7fb6afbba740 1 Processor -- start 2024-09-10T09:48:37.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:37.265+0000 7fb6afbba740 1 -- start start 2024-09-10T09:48:37.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:37.265+0000 7fb6afbba740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5633f183ec70 0x5633f1840dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:48:37.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:37.265+0000 7fb6afbba740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5633f183f590 0x5633f1861bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:48:37.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:37.265+0000 7fb6afbba740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5633f1849e50 0x5633f1862130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:48:37.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:37.265+0000 7fb6afbba740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fb6a81b1030 con 0x5633f1849e50 2024-09-10T09:48:37.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:37.265+0000 7fb6afbba740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fb6a81b2b70 con 0x5633f183f590 2024-09-10T09:48:37.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:37.265+0000 7fb6afbba740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fb6a81b1700 con 0x5633f183ec70 2024-09-10T09:48:37.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:37.265+0000 7fb6a5400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5633f183f590 0x5633f1861bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:48:37.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:37.265+0000 7fb6a6800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5633f1849e50 0x5633f1862130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:48:37.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:37.265+0000 7fb6a5400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5633f183f590 0x5633f1861bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:34024/0 (socket says 172.21.15.5:34024) 2024-09-10T09:48:37.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:37.265+0000 7fb6a5400640 1 -- 172.21.15.5:0/3232898916 learned_addr learned my addr 172.21.15.5:0/3232898916 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:48:37.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:37.265+0000 7fb6a5e00640 1 --2- 172.21.15.5:0/3232898916 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5633f183ec70 0x5633f1840dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:48:37.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:37.265+0000 7fb6a5400640 1 -- 172.21.15.5:0/3232898916 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5633f183ec70 msgr2=0x5633f1840dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:48:37.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:37.265+0000 7fb6a5400640 1 --2- 172.21.15.5:0/3232898916 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5633f183ec70 0x5633f1840dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:37.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:37.265+0000 7fb6a5400640 1 -- 172.21.15.5:0/3232898916 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5633f1849e50 msgr2=0x5633f1862130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:48:37.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:37.265+0000 7fb6a5400640 1 --2- 172.21.15.5:0/3232898916 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5633f1849e50 0x5633f1862130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:37.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:37.265+0000 7fb6a5400640 1 -- 172.21.15.5:0/3232898916 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5633f183c370 con 0x5633f183f590 2024-09-10T09:48:37.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:37.265+0000 7fb6a5e00640 1 --2- 172.21.15.5:0/3232898916 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5633f183ec70 0x5633f1840dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:48:37.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:37.265+0000 7fb6a6800640 1 --2- 172.21.15.5:0/3232898916 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5633f1849e50 0x5633f1862130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:48:37.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:37.265+0000 7fb6a5400640 1 --2- 172.21.15.5:0/3232898916 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5633f183f590 0x5633f1861bf0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fb6940ed1b0 tx=0x7fb69403b890 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9238352807753717 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:48:37.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:37.265+0000 7fb692a00640 1 -- 172.21.15.5:0/3232898916 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb694049040 con 0x5633f183f590 2024-09-10T09:48:37.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:37.265+0000 7fb692a00640 1 -- 172.21.15.5:0/3232898916 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb6940bc6d0 con 0x5633f183f590 2024-09-10T09:48:37.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:37.265+0000 7fb692a00640 1 -- 172.21.15.5:0/3232898916 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6940ba360 con 0x5633f183f590 2024-09-10T09:48:37.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:37.265+0000 7fb6afbba740 1 -- 172.21.15.5:0/3232898916 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5633f183f590 msgr2=0x5633f1861bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:48:37.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:37.265+0000 7fb6afbba740 1 --2- 172.21.15.5:0/3232898916 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5633f183f590 0x5633f1861bf0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fb6940ed1b0 tx=0x7fb69403b890 comp rx=0 tx=0).stop 2024-09-10T09:48:37.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:37.265+0000 7fb6afbba740 1 -- 172.21.15.5:0/3232898916 shutdown_connections 2024-09-10T09:48:37.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:37.265+0000 7fb6afbba740 1 --2- 172.21.15.5:0/3232898916 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5633f1849e50 0x5633f1862130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:37.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:37.265+0000 7fb6afbba740 1 --2- 172.21.15.5:0/3232898916 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5633f183f590 0x5633f1861bf0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:37.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:37.265+0000 7fb6afbba740 1 --2- 172.21.15.5:0/3232898916 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5633f183ec70 0x5633f1840dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:37.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:37.265+0000 7fb6afbba740 1 -- 172.21.15.5:0/3232898916 >> 172.21.15.5:0/3232898916 conn(0x5633f183b200 msgr2=0x5633f18406c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:48:37.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:37.265+0000 7fb6afbba740 1 -- 172.21.15.5:0/3232898916 shutdown_connections 2024-09-10T09:48:37.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:37.265+0000 7fb6afbba740 1 -- 172.21.15.5:0/3232898916 wait complete. 2024-09-10T09:48:37.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:37.265+0000 7fb6afbba740 1 -- start start 2024-09-10T09:48:37.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:37.271+0000 7fb6afbba740 1 Processor -- start 2024-09-10T09:48:37.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:37.271+0000 7fb6afbba740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5633f183ec70 0x5633f184ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:48:37.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:37.271+0000 7fb6afbba740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5633f183f590 0x5633f184e310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:48:37.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:37.271+0000 7fb6afbba740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5633f1849e50 0x5633f184e850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:48:37.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:37.271+0000 7fb6afbba740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fb6a81b24a0 con 0x5633f183ec70 2024-09-10T09:48:37.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:37.271+0000 7fb6afbba740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fb6a81b1dd0 con 0x5633f1849e50 2024-09-10T09:48:37.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:37.271+0000 7fb6afbba740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fb6a81af010 con 0x5633f183f590 2024-09-10T09:48:37.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:37.271+0000 7fb6afbba740 10 client.0 _ll_register_callbacks cb 0x7fb6980096b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:48:37.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:37.271+0000 7fb6a5e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5633f183ec70 0x5633f184ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:48:37.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:37.271+0000 7fb6a5e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5633f183ec70 0x5633f184ddd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50448/0 (socket says 172.21.15.5:50448) 2024-09-10T09:48:37.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:37.271+0000 7fb6a5e00640 1 -- 172.21.15.5:0/3204261295 learned_addr learned my addr 172.21.15.5:0/3204261295 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:48:37.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:37.271+0000 7fb6a6800640 1 --2- 172.21.15.5:0/3204261295 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5633f1849e50 0x5633f184e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:48:37.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:37.271+0000 7fb6a5400640 1 --2- 172.21.15.5:0/3204261295 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5633f183f590 0x5633f184e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:48:37.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:37.271+0000 7fb6a5400640 1 -- 172.21.15.5:0/3204261295 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5633f1849e50 msgr2=0x5633f184e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:48:37.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:37.271+0000 7fb6a5400640 1 --2- 172.21.15.5:0/3204261295 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5633f1849e50 0x5633f184e850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:37.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:37.271+0000 7fb6a5400640 1 -- 172.21.15.5:0/3204261295 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5633f183ec70 msgr2=0x5633f184ddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:48:37.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:37.271+0000 7fb6a5400640 1 --2- 172.21.15.5:0/3204261295 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5633f183ec70 0x5633f184ddd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:37.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:37.271+0000 7fb6a5400640 1 -- 172.21.15.5:0/3204261295 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5633f18c3e10 con 0x5633f183f590 2024-09-10T09:48:37.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:37.271+0000 7fb6a5400640 1 --2- 172.21.15.5:0/3204261295 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5633f183f590 0x5633f184e310 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fb6940ed1b0 tx=0x7fb69402a480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9dd0305726ed2d37 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:48:37.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:37.271+0000 7fb687e00640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-09-10T09:48:37.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:37.271+0000 7fb687e00640 1 -- 172.21.15.5:0/3204261295 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb694049070 con 0x5633f183f590 2024-09-10T09:48:37.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:37.271+0000 7fb6afbba740 1 -- 172.21.15.5:0/3204261295 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb6940bc6d0 con 0x5633f183f590 2024-09-10T09:48:37.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:37.271+0000 7fb687e00640 1 -- 172.21.15.5:0/3204261295 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fb6940ed200 con 0x5633f183f590 2024-09-10T09:48:37.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:37.271+0000 7fb687e00640 1 -- 172.21.15.5:0/3204261295 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fb6940c4c40 con 0x5633f183f590 2024-09-10T09:48:37.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:37.271+0000 7fb6afbba740 10 client.27427 fetch_fsmap learned FSMap version 36 2024-09-10T09:48:37.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:37.271+0000 7fb6afbba740 1 -- 172.21.15.5:0/3204261295 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x5633f18c50d0 con 0x5633f183f590 2024-09-10T09:48:37.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:37.271+0000 7fb687e00640 1 -- 172.21.15.5:0/3204261295 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb6940bc6d0 con 0x5633f183f590 2024-09-10T09:48:37.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:37.271+0000 7fb687e00640 1 -- 172.21.15.5:0/3204261295 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 36) ==== 8370+0+0 (secure 0 0 0) 0x7fb6940da070 con 0x5633f183f590 2024-09-10T09:48:37.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:37.271+0000 7fb6afbba740 10 client.27427 fetch_fsmap finished waiting for FSMap version 36 2024-09-10T09:48:37.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:37.271+0000 7fb6afbba740 10 client.27427 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-09-10T09:48:37.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:37.271+0000 7fb6afbba740 20 client.27427 populate_metadata read hostname 'smithi005' 2024-09-10T09:48:37.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:37.271+0000 7fb6afbba740 1 --2- 172.21.15.5:0/3204261295 >> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] conn(0x5633f18c55c0 0x5633f18c79b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:48:37.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:37.271+0000 7fb6afbba740 4 client.27427 mds_command: new command op to 24443 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T09:48:37.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:37.271+0000 7fb6afbba740 1 -- 172.21.15.5:0/3204261295 --> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5633f17ace60 con 0x5633f18c55c0 2024-09-10T09:48:37.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:37.271+0000 7fb6a5e00640 1 --2- 172.21.15.5:0/3204261295 >> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] conn(0x5633f18c55c0 0x5633f18c79b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:48:37.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:37.271+0000 7fb6a5e00640 1 --2- 172.21.15.5:0/3204261295 >> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] conn(0x5633f18c55c0 0x5633f18c79b0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=50d7df095589aec5 server_cookie=2021509e55e2cbed in_seq=0 out_seq=0 2024-09-10T09:48:37.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:37.271+0000 7fb687e00640 10 client.27427 ms_handle_connect on v2:172.21.15.102:6832/2457007427 2024-09-10T09:48:37.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:37.271+0000 7fb687e00640 1 -- 172.21.15.5:0/3204261295 <== mds.4 v2:172.21.15.102:6832/2457007427 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5633f17ace60 con 0x5633f18c55c0 2024-09-10T09:48:37.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:37.271+0000 7fb687e00640 10 client.27427 handle_command_reply: tid=0 2024-09-10T09:48:37.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:37.271+0000 7fb6afbba740 1 client.27427 shutdown 2024-09-10T09:48:37.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:37.271+0000 7fb6afbba740 1 -- 172.21.15.5:0/3204261295 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5633f183f590 msgr2=0x5633f184e310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:48:37.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:37.271+0000 7fb6afbba740 1 --2- 172.21.15.5:0/3204261295 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5633f183f590 0x5633f184e310 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fb6940ed1b0 tx=0x7fb69402a480 comp rx=0 tx=0).stop 2024-09-10T09:48:37.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:37.271+0000 7fb6afbba740 1 -- 172.21.15.5:0/3204261295 shutdown_connections 2024-09-10T09:48:37.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:37.271+0000 7fb6afbba740 1 --2- 172.21.15.5:0/3204261295 >> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] conn(0x5633f18c55c0 0x5633f18c79b0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:37.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:37.271+0000 7fb6afbba740 1 --2- 172.21.15.5:0/3204261295 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5633f1849e50 0x5633f184e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:37.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:37.271+0000 7fb6afbba740 1 --2- 172.21.15.5:0/3204261295 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5633f183f590 0x5633f184e310 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:37.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:37.271+0000 7fb6afbba740 1 --2- 172.21.15.5:0/3204261295 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5633f183ec70 0x5633f184ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:37.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:37.271+0000 7fb6afbba740 1 -- 172.21.15.5:0/3204261295 >> 172.21.15.5:0/3204261295 conn(0x5633f183b200 msgr2=0x5633f1840ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:48:37.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:37.271+0000 7fb687e00640 0 client.27427 ms_handle_reset on v2:172.21.15.102:6832/2457007427 2024-09-10T09:48:37.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:37.271+0000 7fb6afbba740 1 -- 172.21.15.5:0/3204261295 shutdown_connections 2024-09-10T09:48:37.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:37.271+0000 7fb6afbba740 1 -- 172.21.15.5:0/3204261295 wait complete. 2024-09-10T09:48:37.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:37.271+0000 7fb6afbba740 20 client.27427 trim_cache size 0 max 16384 2024-09-10T09:48:37.282 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-10T09:48:37.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:37.276+0000 7fb6ad000640 1 -- 172.21.15.5:0/2734643625 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fb68c0a2c00 msgr2=0x7fb68c0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:48:37.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:37.276+0000 7fb6ad000640 1 --2- 172.21.15.5:0/2734643625 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fb68c0a2c00 0x7fb68c0a50c0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x5633f183c060 tx=0x7fb698005f90 comp rx=0 tx=0).stop 2024-09-10T09:48:37.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:37.276+0000 7fb6ad000640 1 -- 172.21.15.5:0/2734643625 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb6a8074e30 msgr2=0x7fb6a819ac30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:48:37.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:37.276+0000 7fb6ad000640 1 --2- 172.21.15.5:0/2734643625 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb6a8074e30 0x7fb6a819ac30 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fb694004770 tx=0x7fb694003c10 comp rx=0 tx=0).stop 2024-09-10T09:48:37.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:37.276+0000 7fb6ad000640 1 -- 172.21.15.5:0/2734643625 shutdown_connections 2024-09-10T09:48:37.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:37.276+0000 7fb6ad000640 1 --2- 172.21.15.5:0/2734643625 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fb68c0a2c00 0x7fb68c0a50c0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:37.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:37.276+0000 7fb6ad000640 1 --2- 172.21.15.5:0/2734643625 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb6a810e440 0x7fb6a819efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:37.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:37.276+0000 7fb6ad000640 1 --2- 172.21.15.5:0/2734643625 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb6a8074e30 0x7fb6a819ac30 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:37.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:37.276+0000 7fb6ad000640 1 --2- 172.21.15.5:0/2734643625 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb6a8074510 0x7fb6a819a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:37.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:37.276+0000 7fb6ad000640 1 -- 172.21.15.5:0/2734643625 >> 172.21.15.5:0/2734643625 conn(0x7fb6a80fd8c0 msgr2=0x7fb6a80fe5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:48:37.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:37.276+0000 7fb6ad000640 1 -- 172.21.15.5:0/2734643625 shutdown_connections 2024-09-10T09:48:37.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:37.276+0000 7fb6ad000640 1 -- 172.21.15.5:0/2734643625 wait complete. 2024-09-10T09:48:37.479 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:48:37 smithi005 ceph-mon[30350]: pgmap v701: 129 pgs: 129 active+clean; 244 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 2.5 MiB/s wr, 247 op/s 2024-09-10T09:48:37.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:48:37 smithi159 ceph-mon[35513]: pgmap v701: 129 pgs: 129 active+clean; 244 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 2.5 MiB/s wr, 247 op/s 2024-09-10T09:48:37.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:48:37 smithi102 ceph-mon[35442]: pgmap v701: 129 pgs: 129 active+clean; 244 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 2.5 MiB/s wr, 247 op/s 2024-09-10T09:48:37.886 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 10 2024-09-10T09:48:37.886 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T09:48:38.101 INFO:tasks.workunit.client.0.smithi005.stdout: 1 643789 6.46 MB/sec execute 379 sec latency 1761.638 ms 2024-09-10T09:48:38.667 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-10T09:48:38.832 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:48:39.101 INFO:tasks.workunit.client.0.smithi005.stdout: 1 645994 6.48 MB/sec execute 380 sec latency 219.394 ms 2024-09-10T09:48:39.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:39.444+0000 7f1155000640 1 -- 172.21.15.5:0/958325770 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f11501079c0 msgr2=0x7f1150107da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:48:39.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:39.444+0000 7f1155000640 1 --2- 172.21.15.5:0/958325770 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f11501079c0 0x7f1150107da0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f1140009930 tx=0x7f114002a420 comp rx=0 tx=0).stop 2024-09-10T09:48:39.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:39.444+0000 7f1155000640 1 -- 172.21.15.5:0/958325770 shutdown_connections 2024-09-10T09:48:39.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:39.445+0000 7f1155000640 1 --2- 172.21.15.5:0/958325770 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1150102360 0x7f115010e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:39.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:39.445+0000 7f1155000640 1 --2- 172.21.15.5:0/958325770 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f11501019c0 0x7f1150101e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:39.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:39.445+0000 7f1155000640 1 --2- 172.21.15.5:0/958325770 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f11501079c0 0x7f1150107da0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:39.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:39.445+0000 7f1155000640 1 -- 172.21.15.5:0/958325770 >> 172.21.15.5:0/958325770 conn(0x7f11500fd740 msgr2=0x7f11500ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:48:39.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:39.445+0000 7f1155000640 1 -- 172.21.15.5:0/958325770 shutdown_connections 2024-09-10T09:48:39.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:39.445+0000 7f1155000640 1 -- 172.21.15.5:0/958325770 wait complete. 2024-09-10T09:48:39.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:39.446+0000 7f1155000640 1 Processor -- start 2024-09-10T09:48:39.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:39.446+0000 7f1155000640 1 -- start start 2024-09-10T09:48:39.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:39.446+0000 7f1155000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f11501019c0 0x7f115019a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:48:39.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:39.446+0000 7f1155000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1150102360 0x7f115019aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:48:39.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:39.446+0000 7f1155000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f11501079c0 0x7f115019edd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:48:39.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:39.446+0000 7f1155000640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1150111e10 con 0x7f11501079c0 2024-09-10T09:48:39.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:39.446+0000 7f1155000640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f1150111c90 con 0x7f1150102360 2024-09-10T09:48:39.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:39.446+0000 7f1155000640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f1150111f90 con 0x7f11501019c0 2024-09-10T09:48:39.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:39.446+0000 7f114ea00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1150102360 0x7f115019aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:48:39.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:39.447+0000 7f114ea00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1150102360 0x7f115019aa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:34044/0 (socket says 172.21.15.5:34044) 2024-09-10T09:48:39.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:39.447+0000 7f114ea00640 1 -- 172.21.15.5:0/3548931663 learned_addr learned my addr 172.21.15.5:0/3548931663 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:48:39.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:39.447+0000 7f114f400640 1 --2- 172.21.15.5:0/3548931663 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f11501019c0 0x7f115019a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:48:39.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:39.447+0000 7f114fe00640 1 --2- 172.21.15.5:0/3548931663 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f11501079c0 0x7f115019edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:48:39.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:39.447+0000 7f114f400640 1 -- 172.21.15.5:0/3548931663 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1150102360 msgr2=0x7f115019aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:48:39.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:39.447+0000 7f114f400640 1 --2- 172.21.15.5:0/3548931663 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1150102360 0x7f115019aa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:39.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:39.447+0000 7f114f400640 1 -- 172.21.15.5:0/3548931663 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f11501079c0 msgr2=0x7f115019edd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:48:39.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:39.447+0000 7f114f400640 1 --2- 172.21.15.5:0/3548931663 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f11501079c0 0x7f115019edd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:39.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:39.447+0000 7f114f400640 1 -- 172.21.15.5:0/3548931663 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f115019f4b0 con 0x7f11501019c0 2024-09-10T09:48:39.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:39.447+0000 7f114ea00640 1 --2- 172.21.15.5:0/3548931663 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1150102360 0x7f115019aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:48:39.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:39.447+0000 7f114f400640 1 --2- 172.21.15.5:0/3548931663 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f11501019c0 0x7f115019a500 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f114002a940 tx=0x7f1140003b20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:48:39.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:39.448+0000 7f1147e00640 1 -- 172.21.15.5:0/3548931663 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f114003f070 con 0x7f11501019c0 2024-09-10T09:48:39.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:39.448+0000 7f1147e00640 1 -- 172.21.15.5:0/3548931663 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1140031c90 con 0x7f11501019c0 2024-09-10T09:48:39.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:39.448+0000 7f1155000640 1 -- 172.21.15.5:0/3548931663 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f115019f740 con 0x7f11501019c0 2024-09-10T09:48:39.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:39.448+0000 7f1147e00640 1 -- 172.21.15.5:0/3548931663 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1140035030 con 0x7f11501019c0 2024-09-10T09:48:39.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:39.448+0000 7f1155000640 1 -- 172.21.15.5:0/3548931663 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11501a7080 con 0x7f11501019c0 2024-09-10T09:48:39.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:39.449+0000 7f1155000640 1 -- 172.21.15.5:0/3548931663 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1150101e20 con 0x7f11501019c0 2024-09-10T09:48:39.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:39.451+0000 7f1147e00640 1 -- 172.21.15.5:0/3548931663 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f1140042070 con 0x7f11501019c0 2024-09-10T09:48:39.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:39.454+0000 7f1147e00640 1 --2- 172.21.15.5:0/3548931663 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f11340a2c00 0x7f11340a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:48:39.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:39.454+0000 7f1147e00640 1 -- 172.21.15.5:0/3548931663 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f11400f02c0 con 0x7f11501019c0 2024-09-10T09:48:39.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:39.454+0000 7f114ea00640 1 --2- 172.21.15.5:0/3548931663 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f11340a2c00 0x7f11340a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:48:39.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:39.455+0000 7f114ea00640 1 --2- 172.21.15.5:0/3548931663 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f11340a2c00 0x7f11340a50c0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f115010e9e0 tx=0x7f113c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:48:39.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:39.455+0000 7f1147e00640 1 -- 172.21.15.5:0/3548931663 <== mon.1 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f11400bc780 con 0x7f11501019c0 2024-09-10T09:48:39.512 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:48:39.658 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:48:39 smithi005 ceph-mon[30350]: pgmap v702: 129 pgs: 129 active+clean; 245 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.3 MiB/s wr, 180 op/s 2024-09-10T09:48:39.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:39.716+0000 7f1155000640 1 -- 172.21.15.5:0/3548931663 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f115019b860 con 0x7f11501019c0 2024-09-10T09:48:39.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:39.717+0000 7f1147e00640 1 -- 172.21.15.5:0/3548931663 <== mon.1 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) ==== 94+0+11969 (secure 0 0 0) 0x7f11400c0a80 con 0x7f11501019c0 2024-09-10T09:48:39.718 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-10T09:48:39.718 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":36,"btime":"2024-09-10T09:48:07:349075+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14454,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/2970594311","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2970594311},{"type":"v1","addr":"172.21.15.5:6835","nonce":2970594311}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14514,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6839/3616502174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":3616502174},{"type":"v1","addr":"172.21.15.5:6839","nonce":3616502174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":36,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T09:35:29.626287+0000","modified":"2024-09-10T09:48:06.970127+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24542,"mds_1":24535,"mds_2":24511,"mds_3":24506,"mds_4":24443},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14520":{"gid":14520,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/2073599842","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":2073599842},{"type":"v1","addr":"172.21.15.5:6841","nonce":2073599842}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"k","rank":4,"incarnation":26,"state":"up:active","state_seq":24,"addr":"172.21.15.102:6833/2457007427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":2457007427},{"type":"v1","addr":"172.21.15.102:6833","nonce":2457007427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/356445159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":356445159},{"type":"v1","addr":"172.21.15.159:6833","nonce":356445159}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6837/2894314507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":2894314507},{"type":"v1","addr":"172.21.15.5:6837","nonce":2894314507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.102:6835/1647549111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":1647549111},{"type":"v1","addr":"172.21.15.102:6835","nonce":1647549111}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6835/952657242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":952657242},{"type":"v1","addr":"172.21.15.159:6835","nonce":952657242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":19,"addr":"172.21.15.102:6837/2689776764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":2689776764},{"type":"v1","addr":"172.21.15.102:6837","nonce":2689776764}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.159:6837/620627639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":620627639},{"type":"v1","addr":"172.21.15.159:6837","nonce":620627639}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24535":{"gid":24535,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":18,"addr":"172.21.15.159:6839/2073739599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":2073739599},{"type":"v1","addr":"172.21.15.159:6839","nonce":2073739599}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24542":{"gid":24542,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.102:6839/335409749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6838","nonce":335409749},{"type":"v1","addr":"172.21.15.102:6839","nonce":335409749}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24542,"qdb_cluster":[24542,24535,24511,24506,24443]},"id":1}]} 2024-09-10T09:48:39.718 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 36 2024-09-10T09:48:39.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:39.719+0000 7f1155000640 1 -- 172.21.15.5:0/3548931663 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f11340a2c00 msgr2=0x7f11340a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:48:39.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:39.719+0000 7f1155000640 1 --2- 172.21.15.5:0/3548931663 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f11340a2c00 0x7f11340a50c0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f115010e9e0 tx=0x7f113c005f90 comp rx=0 tx=0).stop 2024-09-10T09:48:39.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:39.719+0000 7f1155000640 1 -- 172.21.15.5:0/3548931663 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f11501019c0 msgr2=0x7f115019a500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:48:39.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:39.719+0000 7f1155000640 1 --2- 172.21.15.5:0/3548931663 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f11501019c0 0x7f115019a500 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f114002a940 tx=0x7f1140003b20 comp rx=0 tx=0).stop 2024-09-10T09:48:39.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:39.719+0000 7f1155000640 1 -- 172.21.15.5:0/3548931663 shutdown_connections 2024-09-10T09:48:39.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:39.719+0000 7f1155000640 1 --2- 172.21.15.5:0/3548931663 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f11340a2c00 0x7f11340a50c0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:39.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:39.719+0000 7f1155000640 1 --2- 172.21.15.5:0/3548931663 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f11501079c0 0x7f115019edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:39.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:39.719+0000 7f1155000640 1 --2- 172.21.15.5:0/3548931663 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1150102360 0x7f115019aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:39.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:39.719+0000 7f1155000640 1 --2- 172.21.15.5:0/3548931663 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f11501019c0 0x7f115019a500 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:39.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:39.719+0000 7f1155000640 1 -- 172.21.15.5:0/3548931663 >> 172.21.15.5:0/3548931663 conn(0x7f11500fd740 msgr2=0x7f115010cf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:48:39.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:39.719+0000 7f1155000640 1 -- 172.21.15.5:0/3548931663 shutdown_connections 2024-09-10T09:48:39.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:39.719+0000 7f1155000640 1 -- 172.21.15.5:0/3548931663 wait complete. 2024-09-10T09:48:39.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:48:39 smithi159 ceph-mon[35513]: pgmap v702: 129 pgs: 129 active+clean; 245 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.3 MiB/s wr, 180 op/s 2024-09-10T09:48:39.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:48:39 smithi102 ceph-mon[35442]: pgmap v702: 129 pgs: 129 active+clean; 245 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.3 MiB/s wr, 180 op/s 2024-09-10T09:48:40.101 INFO:tasks.workunit.client.0.smithi005.stdout: 1 647601 6.48 MB/sec execute 381 sec latency 25.842 ms 2024-09-10T09:48:40.432 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:48:40 smithi005 ceph-mon[30350]: from='client.? 172.21.15.5:0/3548931663' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:48:40.691 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24542 quiesce db / --timeout 90 --expiration 130 --await-for 45.0' 2024-09-10T09:48:40.691 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24542 quiesce db / --timeout 90 --expiration 130 --await-for 45.0 2024-09-10T09:48:40.852 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:48:40 smithi159 ceph-mon[35513]: from='client.? 172.21.15.5:0/3548931663' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:48:40.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:48:40 smithi102 ceph-mon[35442]: from='client.? 172.21.15.5:0/3548931663' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:48:41.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:40.430+0000 7fc610800640 1 -- 172.21.15.5:0/3001909266 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc608074ee0 msgr2=0x7fc60810def0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:48:41.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:40.430+0000 7fc610800640 1 --2- 172.21.15.5:0/3001909266 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc608074ee0 0x7fc60810def0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fc5f8009860 tx=0x7fc5f802a3d0 comp rx=0 tx=0).stop 2024-09-10T09:48:41.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:40.431+0000 7fc610800640 1 -- 172.21.15.5:0/3001909266 shutdown_connections 2024-09-10T09:48:41.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:40.431+0000 7fc610800640 1 --2- 172.21.15.5:0/3001909266 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc60810e430 0x7fc608110820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:41.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:40.431+0000 7fc610800640 1 --2- 172.21.15.5:0/3001909266 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc608074ee0 0x7fc60810def0 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:41.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:40.431+0000 7fc610800640 1 --2- 172.21.15.5:0/3001909266 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc6080745c0 0x7fc6080749a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:41.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:40.431+0000 7fc610800640 1 -- 172.21.15.5:0/3001909266 >> 172.21.15.5:0/3001909266 conn(0x7fc6080fd8b0 msgr2=0x7fc6080ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:48:41.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:40.431+0000 7fc610800640 1 -- 172.21.15.5:0/3001909266 shutdown_connections 2024-09-10T09:48:41.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:40.431+0000 7fc610800640 1 -- 172.21.15.5:0/3001909266 wait complete. 2024-09-10T09:48:41.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:40.432+0000 7fc610800640 1 Processor -- start 2024-09-10T09:48:41.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:40.432+0000 7fc610800640 1 -- start start 2024-09-10T09:48:41.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:40.432+0000 7fc610800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc6080745c0 0x7fc60819a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:48:41.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:40.432+0000 7fc610800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc608074ee0 0x7fc60819abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:48:41.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:40.432+0000 7fc610800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc60810e430 0x7fc60819ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:48:41.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:40.432+0000 7fc610800640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc6081141e0 con 0x7fc608074ee0 2024-09-10T09:48:41.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:40.432+0000 7fc610800640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fc608114060 con 0x7fc6080745c0 2024-09-10T09:48:41.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:40.432+0000 7fc610800640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fc608114360 con 0x7fc60810e430 2024-09-10T09:48:41.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:40.433+0000 7fc60de00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc6080745c0 0x7fc60819a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:48:41.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:40.433+0000 7fc60d400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc608074ee0 0x7fc60819abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:48:41.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:40.433+0000 7fc60de00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc6080745c0 0x7fc60819a6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:34070/0 (socket says 172.21.15.5:34070) 2024-09-10T09:48:41.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:40.433+0000 7fc60de00640 1 -- 172.21.15.5:0/2247342829 learned_addr learned my addr 172.21.15.5:0/2247342829 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:48:41.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:40.433+0000 7fc60e800640 1 --2- 172.21.15.5:0/2247342829 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc60810e430 0x7fc60819ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:48:41.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:40.433+0000 7fc60de00640 1 -- 172.21.15.5:0/2247342829 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc60810e430 msgr2=0x7fc60819ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:48:41.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:40.433+0000 7fc60de00640 1 --2- 172.21.15.5:0/2247342829 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc60810e430 0x7fc60819ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:41.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:40.433+0000 7fc60de00640 1 -- 172.21.15.5:0/2247342829 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc608074ee0 msgr2=0x7fc60819abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:48:41.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:40.433+0000 7fc60de00640 1 --2- 172.21.15.5:0/2247342829 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc608074ee0 0x7fc60819abe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:41.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:40.433+0000 7fc60de00640 1 -- 172.21.15.5:0/2247342829 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc60819f650 con 0x7fc6080745c0 2024-09-10T09:48:41.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:40.433+0000 7fc60de00640 1 --2- 172.21.15.5:0/2247342829 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc6080745c0 0x7fc60819a6a0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fc5fc00e9f0 tx=0x7fc5fc00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:48:41.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:40.433+0000 7fc602a00640 1 -- 172.21.15.5:0/2247342829 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5fc00ccc0 con 0x7fc6080745c0 2024-09-10T09:48:41.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:40.434+0000 7fc602a00640 1 -- 172.21.15.5:0/2247342829 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc5fc005500 con 0x7fc6080745c0 2024-09-10T09:48:41.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:40.434+0000 7fc610800640 1 -- 172.21.15.5:0/2247342829 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc60819f940 con 0x7fc6080745c0 2024-09-10T09:48:41.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:40.434+0000 7fc602a00640 1 -- 172.21.15.5:0/2247342829 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5fc010690 con 0x7fc6080745c0 2024-09-10T09:48:41.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:40.434+0000 7fc610800640 1 -- 172.21.15.5:0/2247342829 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6081a7220 con 0x7fc6080745c0 2024-09-10T09:48:41.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:40.437+0000 7fc6133fd740 1 Processor -- start 2024-09-10T09:48:41.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:40.437+0000 7fc602a00640 1 -- 172.21.15.5:0/2247342829 <== mon.2 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7fc5fc0046c0 con 0x7fc6080745c0 2024-09-10T09:48:41.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:40.437+0000 7fc6133fd740 1 -- start start 2024-09-10T09:48:41.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:40.437+0000 7fc6133fd740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56251f5a9c70 0x56251f5aa050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:48:41.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:40.437+0000 7fc6133fd740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56251f5aa590 0x56251f5b4910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:48:41.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:40.437+0000 7fc6133fd740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56251f5b4e50 0x56251f5b7240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:48:41.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:40.437+0000 7fc6133fd740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x56251f5103e0 con 0x56251f5a9c70 2024-09-10T09:48:41.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:40.437+0000 7fc6133fd740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x56251f4b7d80 con 0x56251f5aa590 2024-09-10T09:48:41.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:40.437+0000 7fc6133fd740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x56251f50c3a0 con 0x56251f5b4e50 2024-09-10T09:48:41.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:40.438+0000 7fc60de00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56251f5aa590 0x56251f5b4910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:48:41.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:40.438+0000 7fc60d400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56251f5b4e50 0x56251f5b7240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:48:41.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:40.438+0000 7fc60e800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56251f5a9c70 0x56251f5aa050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:48:41.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:40.438+0000 7fc60d400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56251f5b4e50 0x56251f5b7240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:45942/0 (socket says 172.21.15.5:45942) 2024-09-10T09:48:41.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:40.438+0000 7fc60d400640 1 -- 172.21.15.5:0/1301228316 learned_addr learned my addr 172.21.15.5:0/1301228316 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:48:41.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:40.438+0000 7fc60d400640 1 -- 172.21.15.5:0/1301228316 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56251f5aa590 msgr2=0x56251f5b4910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:48:41.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:40.438+0000 7fc60d400640 1 --2- 172.21.15.5:0/1301228316 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56251f5aa590 0x56251f5b4910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:41.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:40.438+0000 7fc60d400640 1 -- 172.21.15.5:0/1301228316 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56251f5a9c70 msgr2=0x56251f5aa050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:48:41.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:40.438+0000 7fc60d400640 1 --2- 172.21.15.5:0/1301228316 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56251f5a9c70 0x56251f5aa050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:41.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:40.438+0000 7fc60d400640 1 -- 172.21.15.5:0/1301228316 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56251f4d6ae0 con 0x56251f5b4e50 2024-09-10T09:48:41.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:40.438+0000 7fc60d400640 1 --2- 172.21.15.5:0/1301228316 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56251f5b4e50 0x56251f5b7240 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fc5f8009860 tx=0x7fc5f8030ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=64a4ff068633badc server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:48:41.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:40.439+0000 7fc602a00640 1 --2- 172.21.15.5:0/2247342829 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fc5f00a2c00 0x7fc5f00a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:48:41.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:40.439+0000 7fc5f7000640 1 -- 172.21.15.5:0/1301228316 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5f8004030 con 0x56251f5b4e50 2024-09-10T09:48:41.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:40.439+0000 7fc5f7000640 1 -- 172.21.15.5:0/1301228316 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc5f8041030 con 0x56251f5b4e50 2024-09-10T09:48:41.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:40.439+0000 7fc5f7000640 1 -- 172.21.15.5:0/1301228316 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5f8005460 con 0x56251f5b4e50 2024-09-10T09:48:41.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:40.439+0000 7fc602a00640 1 -- 172.21.15.5:0/2247342829 <== mon.2 v2:172.21.15.102:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fc5fc0cf780 con 0x7fc6080745c0 2024-09-10T09:48:41.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:40.439+0000 7fc60e800640 1 --2- 172.21.15.5:0/2247342829 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fc5f00a2c00 0x7fc5f00a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:48:41.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:40.440+0000 7fc6133fd740 1 -- 172.21.15.5:0/1301228316 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56251f5b4e50 msgr2=0x56251f5b7240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:48:41.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:40.440+0000 7fc6133fd740 1 --2- 172.21.15.5:0/1301228316 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56251f5b4e50 0x56251f5b7240 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fc5f8009860 tx=0x7fc5f8030ab0 comp rx=0 tx=0).stop 2024-09-10T09:48:41.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:40.440+0000 7fc60e800640 1 --2- 172.21.15.5:0/2247342829 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fc5f00a2c00 0x7fc5f00a50c0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fc60806b460 tx=0x7fc604009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:48:41.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:40.440+0000 7fc6133fd740 1 -- 172.21.15.5:0/1301228316 shutdown_connections 2024-09-10T09:48:41.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:40.440+0000 7fc6133fd740 1 --2- 172.21.15.5:0/1301228316 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56251f5b4e50 0x56251f5b7240 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:41.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:40.440+0000 7fc6133fd740 1 --2- 172.21.15.5:0/1301228316 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56251f5aa590 0x56251f5b4910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:41.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:40.440+0000 7fc6133fd740 1 --2- 172.21.15.5:0/1301228316 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56251f5a9c70 0x56251f5aa050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:41.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:40.440+0000 7fc6133fd740 1 -- 172.21.15.5:0/1301228316 >> 172.21.15.5:0/1301228316 conn(0x56251f5a6200 msgr2=0x56251f5a65e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:48:41.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:40.440+0000 7fc6133fd740 1 -- 172.21.15.5:0/1301228316 shutdown_connections 2024-09-10T09:48:41.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:40.440+0000 7fc6133fd740 1 -- 172.21.15.5:0/1301228316 wait complete. 2024-09-10T09:48:41.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:40.440+0000 7fc6133fd740 1 -- start start 2024-09-10T09:48:41.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:40.441+0000 7fc6133fd740 1 Processor -- start 2024-09-10T09:48:41.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:40.441+0000 7fc6133fd740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56251f5a9c70 0x56251f5b1ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:48:41.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:40.441+0000 7fc6133fd740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56251f5aa590 0x56251f5b21e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:48:41.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:40.441+0000 7fc6133fd740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56251f5b4e50 0x56251f5b2720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:48:41.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:40.441+0000 7fc6133fd740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x56251f5b7a20 con 0x56251f5b4e50 2024-09-10T09:48:41.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:40.441+0000 7fc6133fd740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x56251f4cb0f0 con 0x56251f5a9c70 2024-09-10T09:48:41.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:40.441+0000 7fc6133fd740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x56251f5b7ba0 con 0x56251f5aa590 2024-09-10T09:48:41.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:40.441+0000 7fc6133fd740 10 client.0 _ll_register_callbacks cb 0x56251f4b2bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:48:41.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:40.441+0000 7fc60d400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56251f5aa590 0x56251f5b21e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:48:41.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:40.441+0000 7fc60d400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56251f5aa590 0x56251f5b21e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:45956/0 (socket says 172.21.15.5:45956) 2024-09-10T09:48:41.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:40.441+0000 7fc60d400640 1 -- 172.21.15.5:0/1489633274 learned_addr learned my addr 172.21.15.5:0/1489633274 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:48:41.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:40.442+0000 7fc60de00640 1 --2- 172.21.15.5:0/1489633274 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56251f5a9c70 0x56251f5b1ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:48:41.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:40.442+0000 7fc60d400640 1 -- 172.21.15.5:0/1489633274 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56251f5a9c70 msgr2=0x56251f5b1ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:48:41.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:40.442+0000 7fc60d400640 1 --2- 172.21.15.5:0/1489633274 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56251f5a9c70 0x56251f5b1ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:41.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:40.442+0000 7fc60d400640 1 -- 172.21.15.5:0/1489633274 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56251f5b4e50 msgr2=0x56251f5b2720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:48:41.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:40.442+0000 7fc60e800640 1 --2- 172.21.15.5:0/1489633274 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56251f5b4e50 0x56251f5b2720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:48:41.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:40.442+0000 7fc60d400640 1 --2- 172.21.15.5:0/1489633274 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56251f5b4e50 0x56251f5b2720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:41.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:40.442+0000 7fc60d400640 1 -- 172.21.15.5:0/1489633274 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56251f2813c0 con 0x56251f5aa590 2024-09-10T09:48:41.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:40.442+0000 7fc60e800640 1 --2- 172.21.15.5:0/1489633274 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56251f5b4e50 0x56251f5b2720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T09:48:41.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:40.442+0000 7fc60d400640 1 --2- 172.21.15.5:0/1489633274 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56251f5aa590 0x56251f5b21e0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fc5f8005140 tx=0x7fc5f80044c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d60c4e4d2ba3b6f4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:48:41.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:40.442+0000 7fc5eb400640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-09-10T09:48:41.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:40.442+0000 7fc5eb400640 1 -- 172.21.15.5:0/1489633274 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5f8055060 con 0x56251f5aa590 2024-09-10T09:48:41.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:40.443+0000 7fc6133fd740 1 -- 172.21.15.5:0/1489633274 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc5f8041030 con 0x56251f5aa590 2024-09-10T09:48:41.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:40.443+0000 7fc5eb400640 1 -- 172.21.15.5:0/1489633274 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fc5f80055f0 con 0x56251f5aa590 2024-09-10T09:48:41.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:40.443+0000 7fc5eb400640 1 -- 172.21.15.5:0/1489633274 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fc5f805c050 con 0x56251f5aa590 2024-09-10T09:48:41.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:40.443+0000 7fc6133fd740 10 client.27445 fetch_fsmap learned FSMap version 36 2024-09-10T09:48:41.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:40.443+0000 7fc6133fd740 1 -- 172.21.15.5:0/1489633274 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x56251f5b7a20 con 0x56251f5aa590 2024-09-10T09:48:41.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:40.443+0000 7fc5eb400640 1 -- 172.21.15.5:0/1489633274 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc5f8041030 con 0x56251f5aa590 2024-09-10T09:48:41.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:40.443+0000 7fc5eb400640 1 -- 172.21.15.5:0/1489633274 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 36) ==== 8370+0+0 (secure 0 0 0) 0x7fc5f804f030 con 0x56251f5aa590 2024-09-10T09:48:41.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:40.443+0000 7fc6133fd740 10 client.27445 fetch_fsmap finished waiting for FSMap version 36 2024-09-10T09:48:41.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:40.443+0000 7fc6133fd740 10 client.27445 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-09-10T09:48:41.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:40.443+0000 7fc6133fd740 20 client.27445 populate_metadata read hostname 'smithi005' 2024-09-10T09:48:41.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:40.443+0000 7fc6133fd740 1 --2- 172.21.15.5:0/1489633274 >> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] conn(0x56251f5c4b70 0x56251f5c6f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:48:41.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:40.443+0000 7fc6133fd740 4 client.27445 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:48:41.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:40.443+0000 7fc6133fd740 1 -- 172.21.15.5:0/1489633274 --> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56251f51d950 con 0x56251f5c4b70 2024-09-10T09:48:41.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:40.444+0000 7fc60de00640 1 --2- 172.21.15.5:0/1489633274 >> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] conn(0x56251f5c4b70 0x56251f5c6f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:48:41.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:40.444+0000 7fc5eb400640 10 client.27445 ms_handle_connect on v2:172.21.15.102:6836/2689776764 2024-09-10T09:48:41.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:40.444+0000 7fc60de00640 1 --2- 172.21.15.5:0/1489633274 >> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] conn(0x56251f5c4b70 0x56251f5c6f60 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=c4b66222b0f35fbd server_cookie=8b9cc37213562fb4 in_seq=0 out_seq=0 2024-09-10T09:48:41.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:40.445+0000 7fc5eb400640 1 -- 172.21.15.5:0/1489633274 <== mds.3 v2:172.21.15.102:6836/2689776764 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56251f51d950 con 0x56251f5c4b70 2024-09-10T09:48:41.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:40.445+0000 7fc5eb400640 10 client.27445 handle_command_reply: tid=0 2024-09-10T09:48:41.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:40.445+0000 7fc6133fd740 1 client.27445 shutdown 2024-09-10T09:48:41.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:40.445+0000 7fc6133fd740 1 -- 172.21.15.5:0/1489633274 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56251f5aa590 msgr2=0x56251f5b21e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:48:41.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:40.445+0000 7fc6133fd740 1 --2- 172.21.15.5:0/1489633274 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56251f5aa590 0x56251f5b21e0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fc5f8005140 tx=0x7fc5f80044c0 comp rx=0 tx=0).stop 2024-09-10T09:48:41.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:40.445+0000 7fc6133fd740 1 -- 172.21.15.5:0/1489633274 shutdown_connections 2024-09-10T09:48:41.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:40.445+0000 7fc6133fd740 1 --2- 172.21.15.5:0/1489633274 >> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] conn(0x56251f5c4b70 0x56251f5c6f60 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:41.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:40.445+0000 7fc6133fd740 1 --2- 172.21.15.5:0/1489633274 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56251f5b4e50 0x56251f5b2720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:41.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:40.445+0000 7fc6133fd740 1 --2- 172.21.15.5:0/1489633274 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56251f5aa590 0x56251f5b21e0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:41.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:40.445+0000 7fc6133fd740 1 --2- 172.21.15.5:0/1489633274 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56251f5a9c70 0x56251f5b1ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:41.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:40.445+0000 7fc5eb400640 0 client.27445 ms_handle_reset on v2:172.21.15.102:6836/2689776764 2024-09-10T09:48:41.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:40.445+0000 7fc6133fd740 1 -- 172.21.15.5:0/1489633274 >> 172.21.15.5:0/1489633274 conn(0x56251f5a6200 msgr2=0x56251f5abf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:48:41.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:40.446+0000 7fc6133fd740 1 -- 172.21.15.5:0/1489633274 shutdown_connections 2024-09-10T09:48:41.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:40.446+0000 7fc6133fd740 1 -- 172.21.15.5:0/1489633274 wait complete. 2024-09-10T09:48:41.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:40.446+0000 7fc6133fd740 20 client.27445 trim_cache size 0 max 16384 2024-09-10T09:48:41.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:40.461+0000 7fc6133fd740 1 Processor -- start 2024-09-10T09:48:41.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:40.461+0000 7fc6133fd740 1 -- start start 2024-09-10T09:48:41.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:40.461+0000 7fc6133fd740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56251f5a9c70 0x56251f5ac6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:48:41.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:40.461+0000 7fc6133fd740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56251f5aa590 0x56251f5cc6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:48:41.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:40.462+0000 7fc6133fd740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56251f5b4e50 0x56251f5ccbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:48:41.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:40.462+0000 7fc6133fd740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc6081b1030 con 0x56251f5b4e50 2024-09-10T09:48:41.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:40.462+0000 7fc6133fd740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fc6081b2b70 con 0x56251f5a9c70 2024-09-10T09:48:41.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:40.462+0000 7fc6133fd740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fc6081b1700 con 0x56251f5aa590 2024-09-10T09:48:41.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:40.462+0000 7fc60de00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56251f5a9c70 0x56251f5ac6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:48:41.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:40.462+0000 7fc60de00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56251f5a9c70 0x56251f5ac6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:34084/0 (socket says 172.21.15.5:34084) 2024-09-10T09:48:41.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:40.462+0000 7fc60de00640 1 -- 172.21.15.5:0/3525209675 learned_addr learned my addr 172.21.15.5:0/3525209675 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:48:41.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:40.462+0000 7fc60e800640 1 --2- 172.21.15.5:0/3525209675 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56251f5b4e50 0x56251f5ccbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:48:41.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:40.462+0000 7fc60d400640 1 --2- 172.21.15.5:0/3525209675 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56251f5aa590 0x56251f5cc6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:48:41.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:40.462+0000 7fc60de00640 1 -- 172.21.15.5:0/3525209675 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56251f5aa590 msgr2=0x56251f5cc6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:48:41.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:40.462+0000 7fc60de00640 1 --2- 172.21.15.5:0/3525209675 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56251f5aa590 0x56251f5cc6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:41.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:40.462+0000 7fc60de00640 1 -- 172.21.15.5:0/3525209675 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56251f5b4e50 msgr2=0x56251f5ccbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:48:41.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:40.462+0000 7fc60de00640 1 --2- 172.21.15.5:0/3525209675 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56251f5b4e50 0x56251f5ccbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:41.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:40.462+0000 7fc60de00640 1 -- 172.21.15.5:0/3525209675 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56251f5a6030 con 0x56251f5a9c70 2024-09-10T09:48:41.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:40.463+0000 7fc60de00640 1 --2- 172.21.15.5:0/3525209675 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56251f5a9c70 0x56251f5ac6d0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fc5fc097ff0 tx=0x7fc5fc0d5530 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2e48ce4a229c322a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:48:41.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:40.463+0000 7fc5f6600640 1 -- 172.21.15.5:0/3525209675 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5fc09b4f0 con 0x56251f5a9c70 2024-09-10T09:48:41.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:40.463+0000 7fc5f6600640 1 -- 172.21.15.5:0/3525209675 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc5fc09c290 con 0x56251f5a9c70 2024-09-10T09:48:41.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:40.463+0000 7fc5f6600640 1 -- 172.21.15.5:0/3525209675 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5fc0056b0 con 0x56251f5a9c70 2024-09-10T09:48:41.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:40.464+0000 7fc6133fd740 1 -- 172.21.15.5:0/3525209675 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56251f5a9c70 msgr2=0x56251f5ac6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:48:41.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:40.464+0000 7fc6133fd740 1 --2- 172.21.15.5:0/3525209675 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56251f5a9c70 0x56251f5ac6d0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fc5fc097ff0 tx=0x7fc5fc0d5530 comp rx=0 tx=0).stop 2024-09-10T09:48:41.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:40.464+0000 7fc6133fd740 1 -- 172.21.15.5:0/3525209675 shutdown_connections 2024-09-10T09:48:41.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:40.464+0000 7fc6133fd740 1 --2- 172.21.15.5:0/3525209675 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56251f5b4e50 0x56251f5ccbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:41.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:40.464+0000 7fc6133fd740 1 --2- 172.21.15.5:0/3525209675 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56251f5aa590 0x56251f5cc6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:41.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:40.464+0000 7fc6133fd740 1 --2- 172.21.15.5:0/3525209675 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56251f5a9c70 0x56251f5ac6d0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:41.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:40.464+0000 7fc6133fd740 1 -- 172.21.15.5:0/3525209675 >> 172.21.15.5:0/3525209675 conn(0x56251f5a6200 msgr2=0x56251f5b0d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:48:41.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:40.464+0000 7fc6133fd740 1 -- 172.21.15.5:0/3525209675 shutdown_connections 2024-09-10T09:48:41.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:40.464+0000 7fc6133fd740 1 -- 172.21.15.5:0/3525209675 wait complete. 2024-09-10T09:48:41.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:40.464+0000 7fc6133fd740 1 -- start start 2024-09-10T09:48:41.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:40.465+0000 7fc6133fd740 1 Processor -- start 2024-09-10T09:48:41.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:40.465+0000 7fc6133fd740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56251f5a9c70 0x56251f5aec00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:48:41.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:40.465+0000 7fc6133fd740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56251f5aa590 0x56251f5af140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:48:41.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:40.465+0000 7fc6133fd740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56251f5b4e50 0x56251f5af680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:48:41.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:40.465+0000 7fc6133fd740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc6081b24a0 con 0x56251f5aa590 2024-09-10T09:48:41.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:40.465+0000 7fc6133fd740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fc6081b1dd0 con 0x56251f5a9c70 2024-09-10T09:48:41.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:40.465+0000 7fc6133fd740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fc6081af010 con 0x56251f5b4e50 2024-09-10T09:48:41.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:40.465+0000 7fc6133fd740 10 client.0 _ll_register_callbacks cb 0x7fc604009920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:48:41.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:40.465+0000 7fc60d400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56251f5aa590 0x56251f5af140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:48:41.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:40.465+0000 7fc60e800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56251f5b4e50 0x56251f5af680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:48:41.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:40.465+0000 7fc60de00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56251f5a9c70 0x56251f5aec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:48:41.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:40.465+0000 7fc60de00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56251f5a9c70 0x56251f5aec00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:34098/0 (socket says 172.21.15.5:34098) 2024-09-10T09:48:41.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:40.465+0000 7fc60e800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56251f5b4e50 0x56251f5af680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:45974/0 (socket says 172.21.15.5:45974) 2024-09-10T09:48:41.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:40.465+0000 7fc60d400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56251f5aa590 0x56251f5af140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50528/0 (socket says 172.21.15.5:50528) 2024-09-10T09:48:41.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:40.465+0000 7fc60de00640 1 -- 172.21.15.5:0/2563533309 learned_addr learned my addr 172.21.15.5:0/2563533309 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:48:41.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:40.466+0000 7fc60de00640 1 -- 172.21.15.5:0/2563533309 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56251f5b4e50 msgr2=0x56251f5af680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:48:41.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:40.466+0000 7fc60de00640 1 --2- 172.21.15.5:0/2563533309 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56251f5b4e50 0x56251f5af680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:41.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:40.466+0000 7fc60de00640 1 -- 172.21.15.5:0/2563533309 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56251f5aa590 msgr2=0x56251f5af140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:48:41.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:40.466+0000 7fc60de00640 1 --2- 172.21.15.5:0/2563533309 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56251f5aa590 0x56251f5af140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:41.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:40.466+0000 7fc60de00640 1 -- 172.21.15.5:0/2563533309 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56251f5a6e90 con 0x56251f5a9c70 2024-09-10T09:48:41.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:40.466+0000 7fc60de00640 1 --2- 172.21.15.5:0/2563533309 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56251f5a9c70 0x56251f5aec00 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fc5fc0a42d0 tx=0x7fc5fc004ba0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bf6e3729bc1941b1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:48:41.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:40.466+0000 7fc5eb400640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-09-10T09:48:41.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:40.466+0000 7fc5eb400640 1 -- 172.21.15.5:0/2563533309 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5fc0056b0 con 0x56251f5a9c70 2024-09-10T09:48:41.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:40.466+0000 7fc6133fd740 1 -- 172.21.15.5:0/2563533309 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc5fc09c290 con 0x56251f5a9c70 2024-09-10T09:48:41.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:40.466+0000 7fc5eb400640 1 -- 172.21.15.5:0/2563533309 <== mon.2 v2:172.21.15.102:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fc5fc098200 con 0x56251f5a9c70 2024-09-10T09:48:41.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:40.466+0000 7fc5eb400640 1 -- 172.21.15.5:0/2563533309 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fc5fc0a9050 con 0x56251f5a9c70 2024-09-10T09:48:41.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:40.466+0000 7fc6133fd740 10 client.27425 fetch_fsmap learned FSMap version 36 2024-09-10T09:48:41.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:40.466+0000 7fc6133fd740 1 -- 172.21.15.5:0/2563533309 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x56251f5a1770 con 0x56251f5a9c70 2024-09-10T09:48:41.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:40.467+0000 7fc5eb400640 1 -- 172.21.15.5:0/2563533309 <== mon.2 v2:172.21.15.102:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc5fc09c290 con 0x56251f5a9c70 2024-09-10T09:48:41.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:40.467+0000 7fc5eb400640 1 -- 172.21.15.5:0/2563533309 <== mon.2 v2:172.21.15.102:3300/0 5 ==== fsmap(e 36) ==== 8370+0+0 (secure 0 0 0) 0x7fc5fc0ce030 con 0x56251f5a9c70 2024-09-10T09:48:41.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:40.467+0000 7fc6133fd740 10 client.27425 fetch_fsmap finished waiting for FSMap version 36 2024-09-10T09:48:41.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:40.467+0000 7fc6133fd740 10 client.27425 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-09-10T09:48:41.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:40.467+0000 7fc6133fd740 20 client.27425 populate_metadata read hostname 'smithi005' 2024-09-10T09:48:41.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:40.467+0000 7fc6133fd740 1 --2- 172.21.15.5:0/2563533309 >> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] conn(0x56251f62fbb0 0x56251f631fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:48:41.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:40.467+0000 7fc6133fd740 4 client.27425 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T09:48:41.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:40.467+0000 7fc6133fd740 1 -- 172.21.15.5:0/2563533309 --> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56251f517e60 con 0x56251f62fbb0 2024-09-10T09:48:41.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:40.467+0000 7fc60d400640 1 --2- 172.21.15.5:0/2563533309 >> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] conn(0x56251f62fbb0 0x56251f631fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:48:41.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:40.468+0000 7fc60d400640 1 --2- 172.21.15.5:0/2563533309 >> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] conn(0x56251f62fbb0 0x56251f631fd0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=e85014162170d0ca server_cookie=1db109d82d370231 in_seq=0 out_seq=0 2024-09-10T09:48:41.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:40.468+0000 7fc5eb400640 10 client.27425 ms_handle_connect on v2:172.21.15.102:6836/2689776764 2024-09-10T09:48:41.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:40.468+0000 7fc5eb400640 1 -- 172.21.15.5:0/2563533309 <== mds.3 v2:172.21.15.102:6836/2689776764 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56251f517e60 con 0x56251f62fbb0 2024-09-10T09:48:41.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:40.468+0000 7fc5eb400640 10 client.27425 handle_command_reply: tid=0 2024-09-10T09:48:41.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:40.468+0000 7fc6133fd740 1 client.27425 shutdown 2024-09-10T09:48:41.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:40.468+0000 7fc6133fd740 1 -- 172.21.15.5:0/2563533309 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56251f5a9c70 msgr2=0x56251f5aec00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:48:41.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:40.468+0000 7fc6133fd740 1 --2- 172.21.15.5:0/2563533309 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56251f5a9c70 0x56251f5aec00 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fc5fc0a42d0 tx=0x7fc5fc004ba0 comp rx=0 tx=0).stop 2024-09-10T09:48:41.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:40.468+0000 7fc6133fd740 1 -- 172.21.15.5:0/2563533309 shutdown_connections 2024-09-10T09:48:41.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:40.468+0000 7fc6133fd740 1 --2- 172.21.15.5:0/2563533309 >> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] conn(0x56251f62fbb0 0x56251f631fd0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:41.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:40.468+0000 7fc6133fd740 1 --2- 172.21.15.5:0/2563533309 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56251f5b4e50 0x56251f5af680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:41.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:40.468+0000 7fc6133fd740 1 --2- 172.21.15.5:0/2563533309 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56251f5aa590 0x56251f5af140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:41.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:40.468+0000 7fc6133fd740 1 --2- 172.21.15.5:0/2563533309 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56251f5a9c70 0x56251f5aec00 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:41.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:40.468+0000 7fc6133fd740 1 -- 172.21.15.5:0/2563533309 >> 172.21.15.5:0/2563533309 conn(0x56251f5a6200 msgr2=0x56251f5a4b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:48:41.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:40.468+0000 7fc5eb400640 0 client.27425 ms_handle_reset on v2:172.21.15.102:6836/2689776764 2024-09-10T09:48:41.079 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-10T09:48:41.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:40.469+0000 7fc6133fd740 1 -- 172.21.15.5:0/2563533309 shutdown_connections 2024-09-10T09:48:41.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:40.469+0000 7fc6133fd740 1 -- 172.21.15.5:0/2563533309 wait complete. 2024-09-10T09:48:41.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:40.469+0000 7fc6133fd740 20 client.27425 trim_cache size 0 max 16384 2024-09-10T09:48:41.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:40.470+0000 7fc610800640 1 -- 172.21.15.5:0/2247342829 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fc5f00a2c00 msgr2=0x7fc5f00a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:48:41.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:40.470+0000 7fc610800640 1 --2- 172.21.15.5:0/2247342829 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fc5f00a2c00 0x7fc5f00a50c0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fc60806b460 tx=0x7fc604009500 comp rx=0 tx=0).stop 2024-09-10T09:48:41.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:40.470+0000 7fc610800640 1 -- 172.21.15.5:0/2247342829 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc6080745c0 msgr2=0x7fc60819a6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:48:41.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:40.470+0000 7fc610800640 1 --2- 172.21.15.5:0/2247342829 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc6080745c0 0x7fc60819a6a0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fc5fc00e9f0 tx=0x7fc5fc00eeb0 comp rx=0 tx=0).stop 2024-09-10T09:48:41.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:40.470+0000 7fc610800640 1 -- 172.21.15.5:0/2247342829 shutdown_connections 2024-09-10T09:48:41.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:40.470+0000 7fc610800640 1 --2- 172.21.15.5:0/2247342829 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fc5f00a2c00 0x7fc5f00a50c0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:41.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:40.470+0000 7fc610800640 1 --2- 172.21.15.5:0/2247342829 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc60810e430 0x7fc60819ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:41.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:40.470+0000 7fc610800640 1 --2- 172.21.15.5:0/2247342829 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc608074ee0 0x7fc60819abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:41.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:40.470+0000 7fc610800640 1 --2- 172.21.15.5:0/2247342829 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc6080745c0 0x7fc60819a6a0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:41.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:40.470+0000 7fc610800640 1 -- 172.21.15.5:0/2247342829 >> 172.21.15.5:0/2247342829 conn(0x7fc6080fd8b0 msgr2=0x7fc6080fe510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:48:41.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:40.470+0000 7fc610800640 1 -- 172.21.15.5:0/2247342829 shutdown_connections 2024-09-10T09:48:41.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:40.470+0000 7fc610800640 1 -- 172.21.15.5:0/2247342829 wait complete. 2024-09-10T09:48:41.101 INFO:tasks.workunit.client.0.smithi005.stdout: 1 649523 6.50 MB/sec execute 382 sec latency 269.282 ms 2024-09-10T09:48:41.554 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:48:41 smithi005 ceph-mon[30350]: pgmap v703: 129 pgs: 129 active+clean; 246 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.7 MiB/s wr, 228 op/s 2024-09-10T09:48:41.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:48:41 smithi159 ceph-mon[35513]: pgmap v703: 129 pgs: 129 active+clean; 246 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.7 MiB/s wr, 228 op/s 2024-09-10T09:48:41.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:48:41 smithi102 ceph-mon[35442]: pgmap v703: 129 pgs: 129 active+clean; 246 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.7 MiB/s wr, 228 op/s 2024-09-10T09:48:41.937 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-10T09:48:41.993 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:48:42.101 INFO:tasks.workunit.client.0.smithi005.stdout: 1 649524 6.48 MB/sec execute 383 sec latency 761.859 ms 2024-09-10T09:48:42.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:42.946+0000 7f6a84600640 1 -- 172.21.15.5:0/3025306215 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6a7c074590 msgr2=0x7f6a7c074970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:48:42.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:42.946+0000 7f6a84600640 1 --2- 172.21.15.5:0/3025306215 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6a7c074590 0x7f6a7c074970 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f6a70009900 tx=0x7f6a7002a480 comp rx=0 tx=0).stop 2024-09-10T09:48:42.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:42.946+0000 7f6a84600640 1 -- 172.21.15.5:0/3025306215 shutdown_connections 2024-09-10T09:48:42.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:42.946+0000 7f6a84600640 1 --2- 172.21.15.5:0/3025306215 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6a7c112b90 0x7f6a7c114f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:42.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:42.946+0000 7f6a84600640 1 --2- 172.21.15.5:0/3025306215 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6a7c074eb0 0x7f6a7c112650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:42.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:42.946+0000 7f6a84600640 1 --2- 172.21.15.5:0/3025306215 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6a7c074590 0x7f6a7c074970 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:42.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:42.946+0000 7f6a84600640 1 -- 172.21.15.5:0/3025306215 >> 172.21.15.5:0/3025306215 conn(0x7f6a7c0fd8c0 msgr2=0x7f6a7c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:48:42.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:42.946+0000 7f6a84600640 1 -- 172.21.15.5:0/3025306215 shutdown_connections 2024-09-10T09:48:42.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:42.946+0000 7f6a84600640 1 -- 172.21.15.5:0/3025306215 wait complete. 2024-09-10T09:48:42.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:42.947+0000 7f6a84600640 1 Processor -- start 2024-09-10T09:48:42.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:42.947+0000 7f6a84600640 1 -- start start 2024-09-10T09:48:42.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:42.948+0000 7f6a84600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6a7c074590 0x7f6a7c1a47b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:48:42.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:42.948+0000 7f6a84600640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6a7c074eb0 0x7f6a7c1a4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:48:42.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:42.948+0000 7f6a84600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6a7c112b90 0x7f6a7c19e970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:48:42.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:42.948+0000 7f6a84600640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f6a7c1185a0 con 0x7f6a7c074590 2024-09-10T09:48:42.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:42.948+0000 7f6a84600640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f6a7c118420 con 0x7f6a7c112b90 2024-09-10T09:48:42.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:42.948+0000 7f6a84600640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f6a7c118720 con 0x7f6a7c074eb0 2024-09-10T09:48:42.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:42.948+0000 7f6a81200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6a7c074eb0 0x7f6a7c1a4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:48:42.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:42.948+0000 7f6a81200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6a7c074eb0 0x7f6a7c1a4cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:56264/0 (socket says 172.21.15.5:56264) 2024-09-10T09:48:42.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:42.948+0000 7f6a81200640 1 -- 172.21.15.5:0/4272701400 learned_addr learned my addr 172.21.15.5:0/4272701400 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:48:42.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:42.948+0000 7f6a81c00640 1 --2- 172.21.15.5:0/4272701400 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6a7c074590 0x7f6a7c1a47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:48:42.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:42.948+0000 7f6a82600640 1 --2- 172.21.15.5:0/4272701400 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6a7c112b90 0x7f6a7c19e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:48:42.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:42.948+0000 7f6a81200640 1 -- 172.21.15.5:0/4272701400 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6a7c112b90 msgr2=0x7f6a7c19e970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:48:42.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:42.948+0000 7f6a81200640 1 --2- 172.21.15.5:0/4272701400 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6a7c112b90 0x7f6a7c19e970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:42.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:42.948+0000 7f6a81200640 1 -- 172.21.15.5:0/4272701400 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6a7c074590 msgr2=0x7f6a7c1a47b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:48:42.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:42.948+0000 7f6a81200640 1 --2- 172.21.15.5:0/4272701400 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6a7c074590 0x7f6a7c1a47b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:42.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:42.948+0000 7f6a81200640 1 -- 172.21.15.5:0/4272701400 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a7c19f0a0 con 0x7f6a7c074eb0 2024-09-10T09:48:42.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:42.949+0000 7f6a81200640 1 --2- 172.21.15.5:0/4272701400 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6a7c074eb0 0x7f6a7c1a4cf0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f6a6800ea60 tx=0x7f6a6800ef20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:48:42.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:42.949+0000 7f6a7a000640 1 -- 172.21.15.5:0/4272701400 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a6800cde0 con 0x7f6a7c074eb0 2024-09-10T09:48:42.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:42.949+0000 7f6a7a000640 1 -- 172.21.15.5:0/4272701400 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6a680035f0 con 0x7f6a7c074eb0 2024-09-10T09:48:42.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:42.949+0000 7f6a84600640 1 -- 172.21.15.5:0/4272701400 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a7c19f390 con 0x7f6a7c074eb0 2024-09-10T09:48:42.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:42.949+0000 7f6a7a000640 1 -- 172.21.15.5:0/4272701400 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a68010670 con 0x7f6a7c074eb0 2024-09-10T09:48:42.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:42.949+0000 7f6a84600640 1 -- 172.21.15.5:0/4272701400 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a7c1ab680 con 0x7f6a7c074eb0 2024-09-10T09:48:42.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:42.953+0000 7f6a7a000640 1 -- 172.21.15.5:0/4272701400 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f6a680037a0 con 0x7f6a7c074eb0 2024-09-10T09:48:42.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:42.954+0000 7f6a871dd740 1 Processor -- start 2024-09-10T09:48:42.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:42.954+0000 7f6a871dd740 1 -- start start 2024-09-10T09:48:42.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:42.954+0000 7f6a871dd740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563ed98763b0 0x563ed9876790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:48:42.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:42.954+0000 7f6a871dd740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563ed9876cd0 0x563ed9881050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:48:42.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:42.954+0000 7f6a871dd740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563ed9881590 0x563ed9883980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:48:42.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:42.954+0000 7f6a871dd740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x563ed97dc530 con 0x563ed9881590 2024-09-10T09:48:42.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:42.954+0000 7f6a871dd740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x563ed9783860 con 0x563ed98763b0 2024-09-10T09:48:42.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:42.954+0000 7f6a871dd740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x563ed97d84f0 con 0x563ed9876cd0 2024-09-10T09:48:42.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:42.954+0000 7f6a82600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563ed98763b0 0x563ed9876790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:48:42.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:42.954+0000 7f6a82600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563ed98763b0 0x563ed9876790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:41378/0 (socket says 172.21.15.5:41378) 2024-09-10T09:48:42.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:42.954+0000 7f6a81c00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563ed9876cd0 0x563ed9881050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:48:42.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:42.954+0000 7f6a82600640 1 -- 172.21.15.5:0/3190983612 learned_addr learned my addr 172.21.15.5:0/3190983612 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:48:42.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:42.954+0000 7f6a81200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563ed9881590 0x563ed9883980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:48:42.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:42.955+0000 7f6a82600640 1 -- 172.21.15.5:0/3190983612 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563ed9876cd0 msgr2=0x563ed9881050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:48:42.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:42.955+0000 7f6a82600640 1 --2- 172.21.15.5:0/3190983612 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563ed9876cd0 0x563ed9881050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:42.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:42.955+0000 7f6a82600640 1 -- 172.21.15.5:0/3190983612 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563ed9881590 msgr2=0x563ed9883980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:48:42.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:42.955+0000 7f6a82600640 1 --2- 172.21.15.5:0/3190983612 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563ed9881590 0x563ed9883980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:42.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:42.955+0000 7f6a82600640 1 -- 172.21.15.5:0/3190983612 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563ed954db90 con 0x563ed98763b0 2024-09-10T09:48:42.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:42.956+0000 7f6a82600640 1 --2- 172.21.15.5:0/3190983612 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563ed98763b0 0x563ed9876790 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f6a74009f00 tx=0x7f6a740107a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d3b4a7bb241096b3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:48:42.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:42.956+0000 7f6a7a000640 1 --2- 172.21.15.5:0/4272701400 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f6a640a2b80 0x7f6a640a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:48:42.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:42.956+0000 7f6a7a000640 1 -- 172.21.15.5:0/4272701400 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f6a680cf7c0 con 0x7f6a7c074eb0 2024-09-10T09:48:42.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:42.956+0000 7f6a6ea00640 1 -- 172.21.15.5:0/3190983612 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a74003b70 con 0x563ed98763b0 2024-09-10T09:48:42.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:42.956+0000 7f6a6ea00640 1 -- 172.21.15.5:0/3190983612 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6a74003d10 con 0x563ed98763b0 2024-09-10T09:48:42.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:42.956+0000 7f6a6ea00640 1 -- 172.21.15.5:0/3190983612 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a74011a40 con 0x563ed98763b0 2024-09-10T09:48:42.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:42.956+0000 7f6a81c00640 1 --2- 172.21.15.5:0/4272701400 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f6a640a2b80 0x7f6a640a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:48:42.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:42.957+0000 7f6a871dd740 1 -- 172.21.15.5:0/3190983612 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563ed98763b0 msgr2=0x563ed9876790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:48:42.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:42.957+0000 7f6a871dd740 1 --2- 172.21.15.5:0/3190983612 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563ed98763b0 0x563ed9876790 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f6a74009f00 tx=0x7f6a740107a0 comp rx=0 tx=0).stop 2024-09-10T09:48:42.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:42.957+0000 7f6a81c00640 1 --2- 172.21.15.5:0/4272701400 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f6a640a2b80 0x7f6a640a5040 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f6a70006010 tx=0x7f6a70000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:48:42.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:42.957+0000 7f6a871dd740 1 -- 172.21.15.5:0/3190983612 shutdown_connections 2024-09-10T09:48:42.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:42.957+0000 7f6a871dd740 1 --2- 172.21.15.5:0/3190983612 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563ed9881590 0x563ed9883980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:42.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:42.957+0000 7f6a871dd740 1 --2- 172.21.15.5:0/3190983612 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563ed9876cd0 0x563ed9881050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:42.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:42.957+0000 7f6a871dd740 1 --2- 172.21.15.5:0/3190983612 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563ed98763b0 0x563ed9876790 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:42.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:42.957+0000 7f6a871dd740 1 -- 172.21.15.5:0/3190983612 >> 172.21.15.5:0/3190983612 conn(0x563ed9872940 msgr2=0x563ed9872d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:48:42.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:42.957+0000 7f6a871dd740 1 -- 172.21.15.5:0/3190983612 shutdown_connections 2024-09-10T09:48:42.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:42.957+0000 7f6a871dd740 1 -- 172.21.15.5:0/3190983612 wait complete. 2024-09-10T09:48:42.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:42.958+0000 7f6a871dd740 1 -- start start 2024-09-10T09:48:42.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:42.958+0000 7f6a871dd740 1 Processor -- start 2024-09-10T09:48:42.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:42.959+0000 7f6a871dd740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563ed98763b0 0x563ed987e630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:48:42.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:42.959+0000 7f6a871dd740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563ed9876cd0 0x563ed987eb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:48:42.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:42.959+0000 7f6a871dd740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563ed9881590 0x563ed987f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:48:42.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:42.959+0000 7f6a871dd740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x563ed9883fb0 con 0x563ed9876cd0 2024-09-10T09:48:42.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:42.959+0000 7f6a871dd740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x563ed9796e60 con 0x563ed98763b0 2024-09-10T09:48:42.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:42.959+0000 7f6a871dd740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x563ed9884130 con 0x563ed9881590 2024-09-10T09:48:42.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:42.959+0000 7f6a871dd740 10 client.0 _ll_register_callbacks cb 0x563ed979bcc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:48:42.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:42.959+0000 7f6a81200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563ed9876cd0 0x563ed987eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:48:42.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:42.959+0000 7f6a81200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563ed9876cd0 0x563ed987eb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47698/0 (socket says 172.21.15.5:47698) 2024-09-10T09:48:42.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:42.959+0000 7f6a81200640 1 -- 172.21.15.5:0/2142147008 learned_addr learned my addr 172.21.15.5:0/2142147008 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:48:42.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:42.959+0000 7f6a81c00640 1 --2- 172.21.15.5:0/2142147008 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563ed98763b0 0x563ed987e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:48:42.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:42.959+0000 7f6a82600640 1 --2- 172.21.15.5:0/2142147008 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563ed9881590 0x563ed987f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:48:42.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:42.959+0000 7f6a81200640 1 -- 172.21.15.5:0/2142147008 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563ed9881590 msgr2=0x563ed987f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:48:42.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:42.959+0000 7f6a81200640 1 --2- 172.21.15.5:0/2142147008 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563ed9881590 0x563ed987f0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:42.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:42.959+0000 7f6a81200640 1 -- 172.21.15.5:0/2142147008 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563ed98763b0 msgr2=0x563ed987e630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:48:42.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:42.959+0000 7f6a81200640 1 --2- 172.21.15.5:0/2142147008 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563ed98763b0 0x563ed987e630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:42.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:42.960+0000 7f6a81200640 1 -- 172.21.15.5:0/2142147008 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563ed97a2990 con 0x563ed9876cd0 2024-09-10T09:48:42.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:42.960+0000 7f6a81200640 1 --2- 172.21.15.5:0/2142147008 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563ed9876cd0 0x563ed987eb70 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f6a680a4310 tx=0x7f6a68097810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d69c919927c70f80 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:48:42.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:42.960+0000 7f6a63400640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-10T09:48:42.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:42.961+0000 7f6a63400640 1 -- 172.21.15.5:0/2142147008 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a6809b530 con 0x563ed9876cd0 2024-09-10T09:48:42.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:42.961+0000 7f6a63400640 1 -- 172.21.15.5:0/2142147008 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f6a680cf430 con 0x563ed9876cd0 2024-09-10T09:48:42.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:42.961+0000 7f6a871dd740 1 -- 172.21.15.5:0/2142147008 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6a74003d10 con 0x563ed9876cd0 2024-09-10T09:48:42.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:42.961+0000 7f6a63400640 1 -- 172.21.15.5:0/2142147008 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f6a6809c2d0 con 0x563ed9876cd0 2024-09-10T09:48:42.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:42.961+0000 7f6a871dd740 10 client.17436 fetch_fsmap learned FSMap version 36 2024-09-10T09:48:42.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:42.961+0000 7f6a871dd740 1 -- 172.21.15.5:0/2142147008 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x563ed988fd00 con 0x563ed9876cd0 2024-09-10T09:48:42.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:42.961+0000 7f6a63400640 1 -- 172.21.15.5:0/2142147008 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6a74003d10 con 0x563ed9876cd0 2024-09-10T09:48:42.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:42.961+0000 7f6a63400640 1 -- 172.21.15.5:0/2142147008 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 36) ==== 8370+0+0 (secure 0 0 0) 0x7f6a68098240 con 0x563ed9876cd0 2024-09-10T09:48:42.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:42.962+0000 7f6a871dd740 10 client.17436 fetch_fsmap finished waiting for FSMap version 36 2024-09-10T09:48:42.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:42.962+0000 7f6a871dd740 10 client.17436 resolve_mds: validated gid 24542 aka daemon mds.b 2024-09-10T09:48:42.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:42.962+0000 7f6a871dd740 20 client.17436 populate_metadata read hostname 'smithi005' 2024-09-10T09:48:42.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:42.962+0000 7f6a871dd740 1 --2- 172.21.15.5:0/2142147008 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x563ed98911c0 0x563ed98935f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:48:42.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:42.962+0000 7f6a871dd740 4 client.17436 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:48:42.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:42.962+0000 7f6a871dd740 1 -- 172.21.15.5:0/2142147008 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563ed977f310 con 0x563ed98911c0 2024-09-10T09:48:42.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:42.962+0000 7f6a81c00640 1 --2- 172.21.15.5:0/2142147008 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x563ed98911c0 0x563ed98935f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:48:42.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:42.963+0000 7f6a81c00640 1 --2- 172.21.15.5:0/2142147008 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x563ed98911c0 0x563ed98935f0 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ad6dc96c4fd17df3 server_cookie=6c9108c7996c57c6 in_seq=0 out_seq=0 2024-09-10T09:48:42.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:42.963+0000 7f6a63400640 10 client.17436 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T09:48:42.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:42.963+0000 7f6a63400640 1 -- 172.21.15.5:0/2142147008 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563ed977f310 con 0x563ed98911c0 2024-09-10T09:48:42.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:42.963+0000 7f6a63400640 10 client.17436 handle_command_reply: tid=0 2024-09-10T09:48:42.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:42.963+0000 7f6a871dd740 1 client.17436 shutdown 2024-09-10T09:48:42.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:42.964+0000 7f6a871dd740 1 -- 172.21.15.5:0/2142147008 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563ed9876cd0 msgr2=0x563ed987eb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:48:42.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:42.964+0000 7f6a871dd740 1 --2- 172.21.15.5:0/2142147008 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563ed9876cd0 0x563ed987eb70 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f6a680a4310 tx=0x7f6a68097810 comp rx=0 tx=0).stop 2024-09-10T09:48:42.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:42.964+0000 7f6a871dd740 1 -- 172.21.15.5:0/2142147008 shutdown_connections 2024-09-10T09:48:42.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:42.964+0000 7f6a871dd740 1 --2- 172.21.15.5:0/2142147008 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x563ed98911c0 0x563ed98935f0 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:42.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:42.964+0000 7f6a871dd740 1 --2- 172.21.15.5:0/2142147008 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563ed9881590 0x563ed987f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:42.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:42.964+0000 7f6a871dd740 1 --2- 172.21.15.5:0/2142147008 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563ed9876cd0 0x563ed987eb70 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:42.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:42.964+0000 7f6a871dd740 1 --2- 172.21.15.5:0/2142147008 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563ed98763b0 0x563ed987e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:42.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:42.964+0000 7f6a871dd740 1 -- 172.21.15.5:0/2142147008 >> 172.21.15.5:0/2142147008 conn(0x563ed9872940 msgr2=0x563ed9871c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:48:42.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:42.964+0000 7f6a63400640 0 client.17436 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T09:48:42.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:42.965+0000 7f6a871dd740 1 -- 172.21.15.5:0/2142147008 shutdown_connections 2024-09-10T09:48:42.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:42.965+0000 7f6a871dd740 1 -- 172.21.15.5:0/2142147008 wait complete. 2024-09-10T09:48:42.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:42.965+0000 7f6a871dd740 20 client.17436 trim_cache size 0 max 16384 2024-09-10T09:48:42.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:42.994+0000 7f6a871dd740 1 Processor -- start 2024-09-10T09:48:42.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:42.994+0000 7f6a871dd740 1 -- start start 2024-09-10T09:48:42.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:42.995+0000 7f6a871dd740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563ed98763b0 0x563ed9871b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:48:42.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:42.995+0000 7f6a871dd740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563ed9881590 0x563ed9898ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:48:42.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:42.995+0000 7f6a871dd740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563ed9899400 0x563ed987b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:48:42.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:42.995+0000 7f6a871dd740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f6a7c1b5440 con 0x563ed9899400 2024-09-10T09:48:42.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:42.995+0000 7f6a871dd740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f6a7c1b6f80 con 0x563ed98763b0 2024-09-10T09:48:42.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:42.995+0000 7f6a871dd740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f6a7c1b5b10 con 0x563ed9881590 2024-09-10T09:48:42.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:42.995+0000 7f6a81200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563ed9881590 0x563ed9898ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:48:42.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:42.995+0000 7f6a81200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563ed9881590 0x563ed9898ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:56302/0 (socket says 172.21.15.5:56302) 2024-09-10T09:48:42.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:42.995+0000 7f6a81200640 1 -- 172.21.15.5:0/3463704725 learned_addr learned my addr 172.21.15.5:0/3463704725 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:48:42.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:42.995+0000 7f6a82600640 1 --2- 172.21.15.5:0/3463704725 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563ed9899400 0x563ed987b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:48:42.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:42.996+0000 7f6a81200640 1 -- 172.21.15.5:0/3463704725 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563ed98763b0 msgr2=0x563ed9871b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:48:42.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:42.996+0000 7f6a81c00640 1 --2- 172.21.15.5:0/3463704725 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563ed98763b0 0x563ed9871b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:48:42.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:42.996+0000 7f6a81200640 1 --2- 172.21.15.5:0/3463704725 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563ed98763b0 0x563ed9871b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:42.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:42.996+0000 7f6a81200640 1 -- 172.21.15.5:0/3463704725 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563ed9899400 msgr2=0x563ed987b8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:48:42.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:42.996+0000 7f6a81200640 1 --2- 172.21.15.5:0/3463704725 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563ed9899400 0x563ed987b8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:42.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:42.996+0000 7f6a81200640 1 -- 172.21.15.5:0/3463704725 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563ed987d5e0 con 0x563ed9881590 2024-09-10T09:48:42.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:42.996+0000 7f6a82600640 1 --2- 172.21.15.5:0/3463704725 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563ed9899400 0x563ed987b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:48:42.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:42.996+0000 7f6a81200640 1 --2- 172.21.15.5:0/3463704725 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563ed9881590 0x563ed9898ec0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f6a68099e80 tx=0x7f6a68003f30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6d077ad45805f702 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:48:42.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:42.996+0000 7f6a6e000640 1 -- 172.21.15.5:0/3463704725 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a6809b530 con 0x563ed9881590 2024-09-10T09:48:42.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:42.996+0000 7f6a6e000640 1 -- 172.21.15.5:0/3463704725 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6a6809b020 con 0x563ed9881590 2024-09-10T09:48:43.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:42.997+0000 7f6a6e000640 1 -- 172.21.15.5:0/3463704725 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a680cf430 con 0x563ed9881590 2024-09-10T09:48:43.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:42.998+0000 7f6a871dd740 1 -- 172.21.15.5:0/3463704725 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563ed9881590 msgr2=0x563ed9898ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:48:43.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:42.998+0000 7f6a871dd740 1 --2- 172.21.15.5:0/3463704725 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563ed9881590 0x563ed9898ec0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f6a68099e80 tx=0x7f6a68003f30 comp rx=0 tx=0).stop 2024-09-10T09:48:43.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:42.998+0000 7f6a871dd740 1 -- 172.21.15.5:0/3463704725 shutdown_connections 2024-09-10T09:48:43.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:42.998+0000 7f6a871dd740 1 --2- 172.21.15.5:0/3463704725 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563ed9899400 0x563ed987b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:43.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:42.998+0000 7f6a871dd740 1 --2- 172.21.15.5:0/3463704725 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563ed9881590 0x563ed9898ec0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:43.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:42.998+0000 7f6a871dd740 1 --2- 172.21.15.5:0/3463704725 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563ed98763b0 0x563ed9871b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:43.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:42.998+0000 7f6a871dd740 1 -- 172.21.15.5:0/3463704725 >> 172.21.15.5:0/3463704725 conn(0x563ed9872940 msgr2=0x563ed9871420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:48:43.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:42.998+0000 7f6a871dd740 1 -- 172.21.15.5:0/3463704725 shutdown_connections 2024-09-10T09:48:43.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:42.998+0000 7f6a871dd740 1 -- 172.21.15.5:0/3463704725 wait complete. 2024-09-10T09:48:43.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:42.999+0000 7f6a871dd740 1 -- start start 2024-09-10T09:48:43.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:43.000+0000 7f6a871dd740 1 Processor -- start 2024-09-10T09:48:43.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:43.000+0000 7f6a871dd740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563ed98763b0 0x563ed9889380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:48:43.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:43.000+0000 7f6a871dd740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563ed9881590 0x563ed98898c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:48:43.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:43.000+0000 7f6a871dd740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563ed9899400 0x563ed9886350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:48:43.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:43.000+0000 7f6a871dd740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f6a7c1b68b0 con 0x563ed98763b0 2024-09-10T09:48:43.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:43.000+0000 7f6a871dd740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f6a7c1b61e0 con 0x563ed9881590 2024-09-10T09:48:43.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:43.000+0000 7f6a871dd740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f6a7c1b3420 con 0x563ed9899400 2024-09-10T09:48:43.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:43.000+0000 7f6a871dd740 10 client.0 _ll_register_callbacks cb 0x7f6a6800ef90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:48:43.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:43.001+0000 7f6a81c00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563ed98763b0 0x563ed9889380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:48:43.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:43.001+0000 7f6a81c00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563ed98763b0 0x563ed9889380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47714/0 (socket says 172.21.15.5:47714) 2024-09-10T09:48:43.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:43.001+0000 7f6a81c00640 1 -- 172.21.15.5:0/1454364804 learned_addr learned my addr 172.21.15.5:0/1454364804 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:48:43.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:43.001+0000 7f6a81200640 1 --2- 172.21.15.5:0/1454364804 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563ed9881590 0x563ed98898c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:48:43.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:43.001+0000 7f6a82600640 1 --2- 172.21.15.5:0/1454364804 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563ed9899400 0x563ed9886350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:48:43.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:43.001+0000 7f6a82600640 1 -- 172.21.15.5:0/1454364804 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563ed9881590 msgr2=0x563ed98898c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:48:43.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:43.001+0000 7f6a82600640 1 --2- 172.21.15.5:0/1454364804 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563ed9881590 0x563ed98898c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:43.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:43.001+0000 7f6a82600640 1 -- 172.21.15.5:0/1454364804 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563ed98763b0 msgr2=0x563ed9889380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:48:43.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:43.002+0000 7f6a82600640 1 --2- 172.21.15.5:0/1454364804 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563ed98763b0 0x563ed9889380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:43.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:43.002+0000 7f6a82600640 1 -- 172.21.15.5:0/1454364804 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563ed9886990 con 0x563ed9899400 2024-09-10T09:48:43.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:43.002+0000 7f6a81200640 1 --2- 172.21.15.5:0/1454364804 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563ed9881590 0x563ed98898c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:48:43.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:43.002+0000 7f6a81c00640 1 --2- 172.21.15.5:0/1454364804 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563ed98763b0 0x563ed9889380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:48:43.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:43.003+0000 7f6a82600640 1 --2- 172.21.15.5:0/1454364804 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563ed9899400 0x563ed9886350 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f6a74000f80 tx=0x7f6a740114d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1e23bf17011e6237 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:48:43.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:43.003+0000 7f6a63400640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-09-10T09:48:43.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:43.003+0000 7f6a63400640 1 -- 172.21.15.5:0/1454364804 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a7402c060 con 0x563ed9899400 2024-09-10T09:48:43.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:43.003+0000 7f6a63400640 1 -- 172.21.15.5:0/1454364804 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f6a74011c20 con 0x563ed9899400 2024-09-10T09:48:43.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:43.003+0000 7f6a871dd740 1 -- 172.21.15.5:0/1454364804 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6a6809b020 con 0x563ed9899400 2024-09-10T09:48:43.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:43.004+0000 7f6a63400640 1 -- 172.21.15.5:0/1454364804 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f6a74011e40 con 0x563ed9899400 2024-09-10T09:48:43.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:43.004+0000 7f6a871dd740 10 client.27469 fetch_fsmap learned FSMap version 36 2024-09-10T09:48:43.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:43.004+0000 7f6a871dd740 1 -- 172.21.15.5:0/1454364804 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x563ed990d830 con 0x563ed9899400 2024-09-10T09:48:43.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:43.004+0000 7f6a63400640 1 -- 172.21.15.5:0/1454364804 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6a6809b020 con 0x563ed9899400 2024-09-10T09:48:43.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:43.005+0000 7f6a63400640 1 -- 172.21.15.5:0/1454364804 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 36) ==== 8370+0+0 (secure 0 0 0) 0x7f6a74027490 con 0x563ed9899400 2024-09-10T09:48:43.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:43.005+0000 7f6a871dd740 10 client.27469 fetch_fsmap finished waiting for FSMap version 36 2024-09-10T09:48:43.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:43.005+0000 7f6a871dd740 10 client.27469 resolve_mds: validated gid 24542 aka daemon mds.b 2024-09-10T09:48:43.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:43.005+0000 7f6a871dd740 20 client.27469 populate_metadata read hostname 'smithi005' 2024-09-10T09:48:43.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:43.005+0000 7f6a871dd740 1 --2- 172.21.15.5:0/1454364804 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x563ed990dc00 0x563ed990fff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:48:43.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:43.005+0000 7f6a871dd740 4 client.27469 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 130.0, "await_for": 45.0}] 2024-09-10T09:48:43.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:43.005+0000 7f6a871dd740 1 -- 172.21.15.5:0/1454364804 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 130.0, "await_for": 45.0}) -- 0x563ed97e9c60 con 0x563ed990dc00 2024-09-10T09:48:43.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:43.005+0000 7f6a81c00640 1 --2- 172.21.15.5:0/1454364804 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x563ed990dc00 0x563ed990fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:48:43.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:43.006+0000 7f6a81c00640 1 --2- 172.21.15.5:0/1454364804 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x563ed990dc00 0x563ed990fff0 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a7eb7a8309c52c4d server_cookie=5346bbc4c48971f7 in_seq=0 out_seq=0 2024-09-10T09:48:43.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:43.006+0000 7f6a63400640 10 client.27469 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T09:48:43.010 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:48:43.101 INFO:tasks.workunit.client.0.smithi005.stdout: 1 650014 6.47 MB/sec execute 384 sec latency 1413.339 ms 2024-09-10T09:48:43.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:43.172+0000 7f6a63400640 1 -- 172.21.15.5:0/1454364804 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x563ed97e9c60 con 0x563ed990dc00 2024-09-10T09:48:43.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:43.172+0000 7f6a63400640 10 client.27469 handle_command_reply: tid=0 2024-09-10T09:48:43.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:43.172+0000 7f6a871dd740 1 client.27469 shutdown 2024-09-10T09:48:43.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:43.173+0000 7f6a871dd740 1 -- 172.21.15.5:0/1454364804 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563ed9899400 msgr2=0x563ed9886350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:48:43.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:43.173+0000 7f6a871dd740 1 --2- 172.21.15.5:0/1454364804 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563ed9899400 0x563ed9886350 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f6a74000f80 tx=0x7f6a740114d0 comp rx=0 tx=0).stop 2024-09-10T09:48:43.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:43.173+0000 7f6a871dd740 1 -- 172.21.15.5:0/1454364804 shutdown_connections 2024-09-10T09:48:43.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:43.173+0000 7f6a871dd740 1 --2- 172.21.15.5:0/1454364804 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x563ed990dc00 0x563ed990fff0 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:43.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:43.173+0000 7f6a871dd740 1 --2- 172.21.15.5:0/1454364804 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563ed9899400 0x563ed9886350 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:43.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:43.173+0000 7f6a871dd740 1 --2- 172.21.15.5:0/1454364804 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563ed9881590 0x563ed98898c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:43.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:43.173+0000 7f6a871dd740 1 --2- 172.21.15.5:0/1454364804 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563ed98763b0 0x563ed9889380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:43.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:43.173+0000 7f6a63400640 0 client.27469 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T09:48:43.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:43.173+0000 7f6a871dd740 1 -- 172.21.15.5:0/1454364804 >> 172.21.15.5:0/1454364804 conn(0x563ed9872940 msgr2=0x563ed9871c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:48:43.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:43.173+0000 7f6a871dd740 1 -- 172.21.15.5:0/1454364804 shutdown_connections 2024-09-10T09:48:43.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:43.173+0000 7f6a871dd740 1 -- 172.21.15.5:0/1454364804 wait complete. 2024-09-10T09:48:43.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:43.173+0000 7f6a871dd740 20 client.27469 trim_cache size 0 max 16384 2024-09-10T09:48:43.177 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-10T09:48:43.177 INFO:teuthology.orchestra.run.smithi005.stdout: "epoch": 36, 2024-09-10T09:48:43.177 INFO:teuthology.orchestra.run.smithi005.stdout: "leader": 24542, 2024-09-10T09:48:43.177 INFO:teuthology.orchestra.run.smithi005.stdout: "set_version": 34, 2024-09-10T09:48:43.177 INFO:teuthology.orchestra.run.smithi005.stdout: "sets": { 2024-09-10T09:48:43.177 INFO:teuthology.orchestra.run.smithi005.stdout: "81e886f6": { 2024-09-10T09:48:43.177 INFO:teuthology.orchestra.run.smithi005.stdout: "version": 34, 2024-09-10T09:48:43.177 INFO:teuthology.orchestra.run.smithi005.stdout: "age_ref": 0.0, 2024-09-10T09:48:43.177 INFO:teuthology.orchestra.run.smithi005.stdout: "state": { 2024-09-10T09:48:43.178 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "QUIESCED", 2024-09-10T09:48:43.178 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0 2024-09-10T09:48:43.178 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T09:48:43.178 INFO:teuthology.orchestra.run.smithi005.stdout: "timeout": 90.0, 2024-09-10T09:48:43.178 INFO:teuthology.orchestra.run.smithi005.stdout: "expiration": 130.0, 2024-09-10T09:48:43.178 INFO:teuthology.orchestra.run.smithi005.stdout: "members": { 2024-09-10T09:48:43.178 INFO:teuthology.orchestra.run.smithi005.stdout: "file:/": { 2024-09-10T09:48:43.178 INFO:teuthology.orchestra.run.smithi005.stdout: "excluded": false, 2024-09-10T09:48:43.178 INFO:teuthology.orchestra.run.smithi005.stdout: "state": { 2024-09-10T09:48:43.178 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "QUIESCED", 2024-09-10T09:48:43.178 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0 2024-09-10T09:48:43.178 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T09:48:43.178 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T09:48:43.178 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T09:48:43.179 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T09:48:43.179 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T09:48:43.179 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-10T09:48:43.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:43.174+0000 7f6a84600640 1 -- 172.21.15.5:0/4272701400 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f6a640a2b80 msgr2=0x7f6a640a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:48:43.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:43.174+0000 7f6a84600640 1 --2- 172.21.15.5:0/4272701400 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f6a640a2b80 0x7f6a640a5040 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f6a70006010 tx=0x7f6a70000bc0 comp rx=0 tx=0).stop 2024-09-10T09:48:43.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:43.174+0000 7f6a84600640 1 -- 172.21.15.5:0/4272701400 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6a7c074eb0 msgr2=0x7f6a7c1a4cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:48:43.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:43.174+0000 7f6a84600640 1 --2- 172.21.15.5:0/4272701400 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6a7c074eb0 0x7f6a7c1a4cf0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f6a6800ea60 tx=0x7f6a6800ef20 comp rx=0 tx=0).stop 2024-09-10T09:48:43.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:43.174+0000 7f6a84600640 1 -- 172.21.15.5:0/4272701400 shutdown_connections 2024-09-10T09:48:43.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:43.174+0000 7f6a84600640 1 --2- 172.21.15.5:0/4272701400 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f6a640a2b80 0x7f6a640a5040 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:43.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:43.174+0000 7f6a84600640 1 --2- 172.21.15.5:0/4272701400 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6a7c112b90 0x7f6a7c19e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:43.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:43.174+0000 7f6a84600640 1 --2- 172.21.15.5:0/4272701400 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6a7c074eb0 0x7f6a7c1a4cf0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:43.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:43.174+0000 7f6a84600640 1 --2- 172.21.15.5:0/4272701400 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6a7c074590 0x7f6a7c1a47b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:43.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:43.174+0000 7f6a84600640 1 -- 172.21.15.5:0/4272701400 >> 172.21.15.5:0/4272701400 conn(0x7f6a7c0fd8c0 msgr2=0x7f6a7c0ff270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:48:43.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:43.174+0000 7f6a84600640 1 -- 172.21.15.5:0/4272701400 shutdown_connections 2024-09-10T09:48:43.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:43.174+0000 7f6a84600640 1 -- 172.21.15.5:0/4272701400 wait complete. 2024-09-10T09:48:43.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:48:43 smithi005 ceph-mon[30350]: pgmap v704: 129 pgs: 129 active+clean; 246 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.5 MiB/s wr, 189 op/s 2024-09-10T09:48:43.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:48:43 smithi159 ceph-mon[35513]: pgmap v704: 129 pgs: 129 active+clean; 246 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.5 MiB/s wr, 189 op/s 2024-09-10T09:48:43.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:48:43 smithi102 ceph-mon[35442]: pgmap v704: 129 pgs: 129 active+clean; 246 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.5 MiB/s wr, 189 op/s 2024-09-10T09:48:44.101 INFO:tasks.workunit.client.0.smithi005.stdout: 1 650014 6.46 MB/sec execute 385 sec latency 1082.117 ms 2024-09-10T09:48:44.197 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '81e886f6', quiesce took 6.3 seconds. Will release after: 3.7 2024-09-10T09:48:44.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:43.897+0000 7f61ea400640 1 -- 172.21.15.5:0/2399605309 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f61e4102790 msgr2=0x7f61e41067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:48:44.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:43.897+0000 7f61ea400640 1 --2- 172.21.15.5:0/2399605309 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f61e4102790 0x7f61e41067e0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f61d0009900 tx=0x7f61d002a400 comp rx=0 tx=0).stop 2024-09-10T09:48:44.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:43.897+0000 7f61ea400640 1 -- 172.21.15.5:0/2399605309 shutdown_connections 2024-09-10T09:48:44.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:43.897+0000 7f61ea400640 1 --2- 172.21.15.5:0/2399605309 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f61e4106f10 0x7f61e410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:44.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:43.897+0000 7f61ea400640 1 --2- 172.21.15.5:0/2399605309 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f61e4102790 0x7f61e41067e0 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:44.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:43.897+0000 7f61ea400640 1 --2- 172.21.15.5:0/2399605309 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f61e4101de0 0x7f61e41021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:44.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:43.897+0000 7f61ea400640 1 -- 172.21.15.5:0/2399605309 >> 172.21.15.5:0/2399605309 conn(0x7f61e40fd8c0 msgr2=0x7f61e40ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:48:44.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:43.897+0000 7f61ea400640 1 -- 172.21.15.5:0/2399605309 shutdown_connections 2024-09-10T09:48:44.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:43.897+0000 7f61ea400640 1 -- 172.21.15.5:0/2399605309 wait complete. 2024-09-10T09:48:44.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:43.898+0000 7f61ea400640 1 Processor -- start 2024-09-10T09:48:44.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:43.898+0000 7f61ea400640 1 -- start start 2024-09-10T09:48:44.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:43.899+0000 7f61ea400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f61e4101de0 0x7f61e419a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:48:44.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:43.899+0000 7f61ea400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f61e4102790 0x7f61e419ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:48:44.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:43.899+0000 7f61ea400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f61e4106f10 0x7f61e419ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:48:44.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:43.899+0000 7f61ea400640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f61e4111f70 con 0x7f61e4102790 2024-09-10T09:48:44.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:43.899+0000 7f61ea400640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f61e4111df0 con 0x7f61e4106f10 2024-09-10T09:48:44.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:43.899+0000 7f61ea400640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f61e41120f0 con 0x7f61e4101de0 2024-09-10T09:48:44.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:43.899+0000 7f61e2a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f61e4102790 0x7f61e419ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:48:44.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:43.899+0000 7f61e2a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f61e4102790 0x7f61e419ab70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47728/0 (socket says 172.21.15.5:47728) 2024-09-10T09:48:44.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:43.899+0000 7f61e2a00640 1 -- 172.21.15.5:0/2917614266 learned_addr learned my addr 172.21.15.5:0/2917614266 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:48:44.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:43.899+0000 7f61e3400640 1 --2- 172.21.15.5:0/2917614266 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f61e4101de0 0x7f61e419a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:48:44.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:43.899+0000 7f61e3e00640 1 --2- 172.21.15.5:0/2917614266 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f61e4106f10 0x7f61e419ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:48:44.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:43.899+0000 7f61e2a00640 1 -- 172.21.15.5:0/2917614266 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f61e4101de0 msgr2=0x7f61e419a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:48:44.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:43.899+0000 7f61e2a00640 1 --2- 172.21.15.5:0/2917614266 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f61e4101de0 0x7f61e419a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:44.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:43.899+0000 7f61e2a00640 1 -- 172.21.15.5:0/2917614266 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f61e4106f10 msgr2=0x7f61e419ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:48:44.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:43.899+0000 7f61e2a00640 1 --2- 172.21.15.5:0/2917614266 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f61e4106f10 0x7f61e419ef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:44.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:43.899+0000 7f61e2a00640 1 -- 172.21.15.5:0/2917614266 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61e419f5e0 con 0x7f61e4102790 2024-09-10T09:48:44.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:43.900+0000 7f61e2a00640 1 --2- 172.21.15.5:0/2917614266 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f61e4102790 0x7f61e419ab70 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f61d0006010 tx=0x7f61d00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:48:44.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:43.900+0000 7f61dbe00640 1 -- 172.21.15.5:0/2917614266 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61d003f030 con 0x7f61e4102790 2024-09-10T09:48:44.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:43.900+0000 7f61dbe00640 1 -- 172.21.15.5:0/2917614266 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f61d0003ce0 con 0x7f61e4102790 2024-09-10T09:48:44.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:43.900+0000 7f61dbe00640 1 -- 172.21.15.5:0/2917614266 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61d0035070 con 0x7f61e4102790 2024-09-10T09:48:44.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:43.900+0000 7f61ea400640 1 -- 172.21.15.5:0/2917614266 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61e419f870 con 0x7f61e4102790 2024-09-10T09:48:44.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:43.900+0000 7f61ea400640 1 -- 172.21.15.5:0/2917614266 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61e419fd00 con 0x7f61e4102790 2024-09-10T09:48:44.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:43.903+0000 7f61ed01b740 1 Processor -- start 2024-09-10T09:48:44.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:43.903+0000 7f61dbe00640 1 -- 172.21.15.5:0/2917614266 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f61d0042030 con 0x7f61e4102790 2024-09-10T09:48:44.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:43.904+0000 7f61ed01b740 1 -- start start 2024-09-10T09:48:44.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:43.904+0000 7f61ed01b740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556b8c877c70 0x556b8c878050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:48:44.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:43.904+0000 7f61ed01b740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556b8c878590 0x556b8c882910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:48:44.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:43.904+0000 7f61ed01b740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556b8c882e50 0x556b8c885240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:48:44.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:43.904+0000 7f61ed01b740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x556b8c7de3e0 con 0x556b8c878590 2024-09-10T09:48:44.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:43.904+0000 7f61ed01b740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x556b8c785d80 con 0x556b8c882e50 2024-09-10T09:48:44.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:43.904+0000 7f61ed01b740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x556b8c7da3a0 con 0x556b8c877c70 2024-09-10T09:48:44.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:43.904+0000 7f61e3400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556b8c878590 0x556b8c882910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:48:44.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:43.904+0000 7f61e3400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556b8c878590 0x556b8c882910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47730/0 (socket says 172.21.15.5:47730) 2024-09-10T09:48:44.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:43.904+0000 7f61e3400640 1 -- 172.21.15.5:0/3681920016 learned_addr learned my addr 172.21.15.5:0/3681920016 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:48:44.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:43.904+0000 7f61e3e00640 1 --2- 172.21.15.5:0/3681920016 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556b8c877c70 0x556b8c878050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:48:44.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:43.904+0000 7f61e2a00640 1 --2- 172.21.15.5:0/3681920016 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556b8c882e50 0x556b8c885240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:48:44.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:43.905+0000 7f61e3e00640 1 -- 172.21.15.5:0/3681920016 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556b8c882e50 msgr2=0x556b8c885240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:48:44.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:43.905+0000 7f61e3e00640 1 --2- 172.21.15.5:0/3681920016 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556b8c882e50 0x556b8c885240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:44.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:43.905+0000 7f61e3e00640 1 -- 172.21.15.5:0/3681920016 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556b8c878590 msgr2=0x556b8c882910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:48:44.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:43.905+0000 7f61e3e00640 1 --2- 172.21.15.5:0/3681920016 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556b8c878590 0x556b8c882910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:44.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:43.905+0000 7f61e3e00640 1 -- 172.21.15.5:0/3681920016 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556b8c7a4ae0 con 0x556b8c877c70 2024-09-10T09:48:44.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:43.905+0000 7f61e3400640 1 --2- 172.21.15.5:0/3681920016 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556b8c878590 0x556b8c882910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:48:44.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:43.905+0000 7f61dbe00640 1 --2- 172.21.15.5:0/2917614266 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f61c80a2c00 0x7f61c80a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:48:44.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:43.905+0000 7f61e3e00640 1 --2- 172.21.15.5:0/3681920016 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556b8c877c70 0x556b8c878050 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f61dc009eb0 tx=0x7f61dc010740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=95bb359a148ef9f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:48:44.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:43.905+0000 7f61dbe00640 1 -- 172.21.15.5:0/2917614266 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f61d00ef920 con 0x7f61e4102790 2024-09-10T09:48:44.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:43.905+0000 7f61e3400640 1 --2- 172.21.15.5:0/2917614266 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f61c80a2c00 0x7f61c80a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:48:44.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:43.905+0000 7f61d8c00640 1 -- 172.21.15.5:0/3681920016 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61dc016070 con 0x556b8c877c70 2024-09-10T09:48:44.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:43.905+0000 7f61d8c00640 1 -- 172.21.15.5:0/3681920016 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f61dc004d20 con 0x556b8c877c70 2024-09-10T09:48:44.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:43.906+0000 7f61d8c00640 1 -- 172.21.15.5:0/3681920016 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61dc0127b0 con 0x556b8c877c70 2024-09-10T09:48:44.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:43.906+0000 7f61ed01b740 1 -- 172.21.15.5:0/3681920016 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556b8c877c70 msgr2=0x556b8c878050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:48:44.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:43.906+0000 7f61e3400640 1 --2- 172.21.15.5:0/2917614266 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f61c80a2c00 0x7f61c80a50c0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x556b8c875060 tx=0x7f61d4005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:48:44.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:43.906+0000 7f61ed01b740 1 --2- 172.21.15.5:0/3681920016 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556b8c877c70 0x556b8c878050 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f61dc009eb0 tx=0x7f61dc010740 comp rx=0 tx=0).stop 2024-09-10T09:48:44.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:43.906+0000 7f61ed01b740 1 -- 172.21.15.5:0/3681920016 shutdown_connections 2024-09-10T09:48:44.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:43.906+0000 7f61ed01b740 1 --2- 172.21.15.5:0/3681920016 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556b8c882e50 0x556b8c885240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:44.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:43.906+0000 7f61ed01b740 1 --2- 172.21.15.5:0/3681920016 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556b8c878590 0x556b8c882910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:44.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:43.906+0000 7f61ed01b740 1 --2- 172.21.15.5:0/3681920016 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556b8c877c70 0x556b8c878050 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:44.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:43.906+0000 7f61ed01b740 1 -- 172.21.15.5:0/3681920016 >> 172.21.15.5:0/3681920016 conn(0x556b8c874200 msgr2=0x556b8c8745e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:48:44.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:43.906+0000 7f61ed01b740 1 -- 172.21.15.5:0/3681920016 shutdown_connections 2024-09-10T09:48:44.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:43.906+0000 7f61ed01b740 1 -- 172.21.15.5:0/3681920016 wait complete. 2024-09-10T09:48:44.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:43.907+0000 7f61ed01b740 1 -- start start 2024-09-10T09:48:44.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:43.907+0000 7f61ed01b740 1 Processor -- start 2024-09-10T09:48:44.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:43.907+0000 7f61ed01b740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556b8c877c70 0x556b8c87fd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:48:44.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:43.908+0000 7f61ed01b740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556b8c878590 0x556b8c880290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:48:44.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:43.908+0000 7f61ed01b740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556b8c882e50 0x556b8c8807d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:48:44.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:43.908+0000 7f61ed01b740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x556b8c885a20 con 0x556b8c878590 2024-09-10T09:48:44.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:43.908+0000 7f61ed01b740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x556b8c7990f0 con 0x556b8c877c70 2024-09-10T09:48:44.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:43.908+0000 7f61ed01b740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x556b8c885ba0 con 0x556b8c882e50 2024-09-10T09:48:44.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:43.908+0000 7f61ed01b740 10 client.0 _ll_register_callbacks cb 0x556b8c780bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:48:44.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:43.908+0000 7f61e2a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556b8c878590 0x556b8c880290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:48:44.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:43.908+0000 7f61e3400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556b8c877c70 0x556b8c87fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:48:44.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:43.908+0000 7f61e2a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556b8c878590 0x556b8c880290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47742/0 (socket says 172.21.15.5:47742) 2024-09-10T09:48:44.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:43.908+0000 7f61e2a00640 1 -- 172.21.15.5:0/1973017852 learned_addr learned my addr 172.21.15.5:0/1973017852 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:48:44.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:43.908+0000 7f61e3e00640 1 --2- 172.21.15.5:0/1973017852 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556b8c882e50 0x556b8c8807d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:48:44.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:43.908+0000 7f61e2a00640 1 -- 172.21.15.5:0/1973017852 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556b8c882e50 msgr2=0x556b8c8807d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:48:44.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:43.908+0000 7f61e2a00640 1 --2- 172.21.15.5:0/1973017852 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556b8c882e50 0x556b8c8807d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:44.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:43.908+0000 7f61e2a00640 1 -- 172.21.15.5:0/1973017852 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556b8c877c70 msgr2=0x556b8c87fd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:48:44.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:43.908+0000 7f61e2a00640 1 --2- 172.21.15.5:0/1973017852 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556b8c877c70 0x556b8c87fd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:44.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:43.908+0000 7f61e2a00640 1 -- 172.21.15.5:0/1973017852 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556b8c54f3c0 con 0x556b8c878590 2024-09-10T09:48:44.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:43.908+0000 7f61e3400640 1 --2- 172.21.15.5:0/1973017852 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556b8c877c70 0x556b8c87fd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:48:44.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:43.909+0000 7f61e2a00640 1 --2- 172.21.15.5:0/1973017852 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556b8c878590 0x556b8c880290 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f61d00b8300 tx=0x7f61d00092c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c389f2d11d247c7a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:48:44.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:43.909+0000 7f61cd600640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-10T09:48:44.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:43.909+0000 7f61cd600640 1 -- 172.21.15.5:0/1973017852 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61d0049070 con 0x556b8c878590 2024-09-10T09:48:44.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:43.909+0000 7f61cd600640 1 -- 172.21.15.5:0/1973017852 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f61d00c45d0 con 0x556b8c878590 2024-09-10T09:48:44.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:43.909+0000 7f61ed01b740 1 -- 172.21.15.5:0/1973017852 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f61dc004d20 con 0x556b8c878590 2024-09-10T09:48:44.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:43.909+0000 7f61cd600640 1 -- 172.21.15.5:0/1973017852 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f61d00b8140 con 0x556b8c878590 2024-09-10T09:48:44.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:43.909+0000 7f61ed01b740 10 client.17454 fetch_fsmap learned FSMap version 36 2024-09-10T09:48:44.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:43.909+0000 7f61ed01b740 1 -- 172.21.15.5:0/1973017852 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x556b8c891a70 con 0x556b8c878590 2024-09-10T09:48:44.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:43.909+0000 7f61cd600640 1 -- 172.21.15.5:0/1973017852 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f61dc004d20 con 0x556b8c878590 2024-09-10T09:48:44.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:43.909+0000 7f61cd600640 1 -- 172.21.15.5:0/1973017852 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 36) ==== 8370+0+0 (secure 0 0 0) 0x7f61d00a6bb0 con 0x556b8c878590 2024-09-10T09:48:44.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:43.910+0000 7f61ed01b740 10 client.17454 fetch_fsmap finished waiting for FSMap version 36 2024-09-10T09:48:44.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:43.910+0000 7f61ed01b740 10 client.17454 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-10T09:48:44.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:43.910+0000 7f61ed01b740 20 client.17454 populate_metadata read hostname 'smithi005' 2024-09-10T09:48:44.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:43.910+0000 7f61ed01b740 1 --2- 172.21.15.5:0/1973017852 >> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] conn(0x556b8c892dc0 0x556b8c8951b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:48:44.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:43.910+0000 7f61ed01b740 4 client.17454 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:48:44.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:43.910+0000 7f61ed01b740 1 -- 172.21.15.5:0/1973017852 --> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556b8c7eb950 con 0x556b8c892dc0 2024-09-10T09:48:44.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:43.910+0000 7f61e3400640 1 --2- 172.21.15.5:0/1973017852 >> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] conn(0x556b8c892dc0 0x556b8c8951b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:48:44.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:43.910+0000 7f61e3400640 1 --2- 172.21.15.5:0/1973017852 >> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] conn(0x556b8c892dc0 0x556b8c8951b0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=bfcc86f71a327bf9 server_cookie=d6b4de1e9df53d0a in_seq=0 out_seq=0 2024-09-10T09:48:44.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:43.910+0000 7f61cd600640 10 client.17454 ms_handle_connect on v2:172.21.15.159:6836/620627639 2024-09-10T09:48:44.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:43.911+0000 7f61cd600640 1 -- 172.21.15.5:0/1973017852 <== mds.2 v2:172.21.15.159:6836/620627639 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556b8c7eb950 con 0x556b8c892dc0 2024-09-10T09:48:44.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:43.912+0000 7f61cd600640 10 client.17454 handle_command_reply: tid=0 2024-09-10T09:48:44.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:43.912+0000 7f61ed01b740 1 client.17454 shutdown 2024-09-10T09:48:44.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:43.912+0000 7f61ed01b740 1 -- 172.21.15.5:0/1973017852 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556b8c878590 msgr2=0x556b8c880290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:48:44.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:43.912+0000 7f61ed01b740 1 --2- 172.21.15.5:0/1973017852 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556b8c878590 0x556b8c880290 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f61d00b8300 tx=0x7f61d00092c0 comp rx=0 tx=0).stop 2024-09-10T09:48:44.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:43.912+0000 7f61ed01b740 1 -- 172.21.15.5:0/1973017852 shutdown_connections 2024-09-10T09:48:44.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:43.912+0000 7f61ed01b740 1 --2- 172.21.15.5:0/1973017852 >> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] conn(0x556b8c892dc0 0x556b8c8951b0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:44.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:43.912+0000 7f61ed01b740 1 --2- 172.21.15.5:0/1973017852 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556b8c882e50 0x556b8c8807d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:44.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:43.912+0000 7f61ed01b740 1 --2- 172.21.15.5:0/1973017852 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556b8c878590 0x556b8c880290 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:44.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:43.912+0000 7f61ed01b740 1 --2- 172.21.15.5:0/1973017852 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556b8c877c70 0x556b8c87fd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:44.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:43.912+0000 7f61ed01b740 1 -- 172.21.15.5:0/1973017852 >> 172.21.15.5:0/1973017852 conn(0x556b8c874200 msgr2=0x556b8c879f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:48:44.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:43.912+0000 7f61cd600640 0 client.17454 ms_handle_reset on v2:172.21.15.159:6836/620627639 2024-09-10T09:48:44.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:43.912+0000 7f61ed01b740 1 -- 172.21.15.5:0/1973017852 shutdown_connections 2024-09-10T09:48:44.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:43.912+0000 7f61ed01b740 1 -- 172.21.15.5:0/1973017852 wait complete. 2024-09-10T09:48:44.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:43.912+0000 7f61ed01b740 20 client.17454 trim_cache size 0 max 16384 2024-09-10T09:48:44.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:43.926+0000 7f61ed01b740 1 Processor -- start 2024-09-10T09:48:44.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:43.926+0000 7f61ed01b740 1 -- start start 2024-09-10T09:48:44.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:43.926+0000 7f61ed01b740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556b8c877c70 0x556b8c87a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:48:44.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:43.926+0000 7f61ed01b740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556b8c878590 0x556b8c87e620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:48:44.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:43.926+0000 7f61ed01b740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556b8c882e50 0x556b8c89aa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:48:44.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:43.926+0000 7f61ed01b740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f61e41b1050 con 0x556b8c882e50 2024-09-10T09:48:44.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:43.926+0000 7f61ed01b740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f61e41b2b90 con 0x556b8c878590 2024-09-10T09:48:44.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:43.926+0000 7f61ed01b740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f61e41b1720 con 0x556b8c877c70 2024-09-10T09:48:44.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:43.927+0000 7f61e3e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556b8c882e50 0x556b8c89aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:48:44.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:43.927+0000 7f61e2a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556b8c878590 0x556b8c87e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:48:44.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:43.927+0000 7f61e3400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556b8c877c70 0x556b8c87a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:48:44.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:43.927+0000 7f61e2a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556b8c878590 0x556b8c87e620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:41428/0 (socket says 172.21.15.5:41428) 2024-09-10T09:48:44.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:43.927+0000 7f61e2a00640 1 -- 172.21.15.5:0/101489615 learned_addr learned my addr 172.21.15.5:0/101489615 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:48:44.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:43.927+0000 7f61e3e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556b8c882e50 0x556b8c89aa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47748/0 (socket says 172.21.15.5:47748) 2024-09-10T09:48:44.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:43.927+0000 7f61e2a00640 1 -- 172.21.15.5:0/101489615 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556b8c877c70 msgr2=0x556b8c87a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:48:44.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:43.927+0000 7f61e2a00640 1 --2- 172.21.15.5:0/101489615 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556b8c877c70 0x556b8c87a6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:44.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:43.927+0000 7f61e2a00640 1 -- 172.21.15.5:0/101489615 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556b8c882e50 msgr2=0x556b8c89aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:48:44.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:43.927+0000 7f61e2a00640 1 --2- 172.21.15.5:0/101489615 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556b8c882e50 0x556b8c89aa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:44.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:43.927+0000 7f61e2a00640 1 -- 172.21.15.5:0/101489615 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556b8c87ed00 con 0x556b8c878590 2024-09-10T09:48:44.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:43.927+0000 7f61e2a00640 1 --2- 172.21.15.5:0/101489615 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556b8c878590 0x556b8c87e620 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f61d003d040 tx=0x7f61d0031a60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=806498fa975eb727 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:48:44.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:43.927+0000 7f61cfe00640 1 -- 172.21.15.5:0/101489615 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61d0049070 con 0x556b8c878590 2024-09-10T09:48:44.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:43.928+0000 7f61cfe00640 1 -- 172.21.15.5:0/101489615 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f61d00bbea0 con 0x556b8c878590 2024-09-10T09:48:44.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:43.928+0000 7f61cfe00640 1 -- 172.21.15.5:0/101489615 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61d00044e0 con 0x556b8c878590 2024-09-10T09:48:44.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:43.928+0000 7f61ed01b740 1 -- 172.21.15.5:0/101489615 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556b8c878590 msgr2=0x556b8c87e620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:48:44.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:43.928+0000 7f61ed01b740 1 --2- 172.21.15.5:0/101489615 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556b8c878590 0x556b8c87e620 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f61d003d040 tx=0x7f61d0031a60 comp rx=0 tx=0).stop 2024-09-10T09:48:44.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:43.928+0000 7f61ed01b740 1 -- 172.21.15.5:0/101489615 shutdown_connections 2024-09-10T09:48:44.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:43.928+0000 7f61ed01b740 1 --2- 172.21.15.5:0/101489615 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556b8c882e50 0x556b8c89aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:44.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:43.928+0000 7f61ed01b740 1 --2- 172.21.15.5:0/101489615 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556b8c878590 0x556b8c87e620 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:44.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:43.928+0000 7f61ed01b740 1 --2- 172.21.15.5:0/101489615 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556b8c877c70 0x556b8c87a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:44.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:43.928+0000 7f61ed01b740 1 -- 172.21.15.5:0/101489615 >> 172.21.15.5:0/101489615 conn(0x556b8c874200 msgr2=0x556b8c881dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:48:44.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:43.928+0000 7f61ed01b740 1 -- 172.21.15.5:0/101489615 shutdown_connections 2024-09-10T09:48:44.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:43.928+0000 7f61ed01b740 1 -- 172.21.15.5:0/101489615 wait complete. 2024-09-10T09:48:44.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:43.929+0000 7f61ed01b740 1 -- start start 2024-09-10T09:48:44.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:43.929+0000 7f61ed01b740 1 Processor -- start 2024-09-10T09:48:44.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:43.929+0000 7f61ed01b740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556b8c877c70 0x556b8c886ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:48:44.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:43.929+0000 7f61ed01b740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556b8c878590 0x556b8c887020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:48:44.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:43.929+0000 7f61ed01b740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556b8c882e50 0x556b8c887560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:48:44.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:43.929+0000 7f61ed01b740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f61e41b24c0 con 0x556b8c878590 2024-09-10T09:48:44.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:43.929+0000 7f61ed01b740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f61e41b1df0 con 0x556b8c882e50 2024-09-10T09:48:44.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:43.929+0000 7f61ed01b740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f61e41af030 con 0x556b8c877c70 2024-09-10T09:48:44.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:43.929+0000 7f61ed01b740 10 client.0 _ll_register_callbacks cb 0x7f61d0005890 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:48:44.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:43.930+0000 7f61e2a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556b8c878590 0x556b8c887020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:48:44.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:43.930+0000 7f61e2a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556b8c878590 0x556b8c887020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47764/0 (socket says 172.21.15.5:47764) 2024-09-10T09:48:44.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:43.930+0000 7f61e2a00640 1 -- 172.21.15.5:0/3619522820 learned_addr learned my addr 172.21.15.5:0/3619522820 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:48:44.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:43.930+0000 7f61e3e00640 1 --2- 172.21.15.5:0/3619522820 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556b8c882e50 0x556b8c887560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:48:44.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:43.930+0000 7f61e3400640 1 --2- 172.21.15.5:0/3619522820 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556b8c877c70 0x556b8c886ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:48:44.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:43.930+0000 7f61e3e00640 1 -- 172.21.15.5:0/3619522820 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556b8c877c70 msgr2=0x556b8c886ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:48:44.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:43.930+0000 7f61e3e00640 1 --2- 172.21.15.5:0/3619522820 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556b8c877c70 0x556b8c886ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:44.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:43.930+0000 7f61e3e00640 1 -- 172.21.15.5:0/3619522820 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556b8c878590 msgr2=0x556b8c887020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:48:44.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:43.930+0000 7f61e3e00640 1 --2- 172.21.15.5:0/3619522820 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556b8c878590 0x556b8c887020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:44.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:43.930+0000 7f61e3e00640 1 -- 172.21.15.5:0/3619522820 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556b8c905240 con 0x556b8c882e50 2024-09-10T09:48:44.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:43.930+0000 7f61e2a00640 1 --2- 172.21.15.5:0/3619522820 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556b8c878590 0x556b8c887020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:48:44.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:43.930+0000 7f61e3e00640 1 --2- 172.21.15.5:0/3619522820 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556b8c882e50 0x556b8c887560 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f61dc005ae0 tx=0x7f61dc004c20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7ad268152c16dd0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:48:44.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:43.930+0000 7f61cd600640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-09-10T09:48:44.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:43.931+0000 7f61cd600640 1 -- 172.21.15.5:0/3619522820 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61dc016070 con 0x556b8c882e50 2024-09-10T09:48:44.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:43.931+0000 7f61ed01b740 1 -- 172.21.15.5:0/3619522820 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f61d00bbea0 con 0x556b8c882e50 2024-09-10T09:48:44.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:43.931+0000 7f61cd600640 1 -- 172.21.15.5:0/3619522820 <== mon.2 v2:172.21.15.102:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f61dc0051a0 con 0x556b8c882e50 2024-09-10T09:48:44.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:43.931+0000 7f61cd600640 1 -- 172.21.15.5:0/3619522820 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f61dc033050 con 0x556b8c882e50 2024-09-10T09:48:44.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:43.931+0000 7f61ed01b740 10 client.27458 fetch_fsmap learned FSMap version 36 2024-09-10T09:48:44.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:43.931+0000 7f61ed01b740 1 -- 172.21.15.5:0/3619522820 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x556b8c906510 con 0x556b8c882e50 2024-09-10T09:48:44.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:43.931+0000 7f61cd600640 1 -- 172.21.15.5:0/3619522820 <== mon.2 v2:172.21.15.102:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f61d00bbea0 con 0x556b8c882e50 2024-09-10T09:48:44.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:43.931+0000 7f61cd600640 1 -- 172.21.15.5:0/3619522820 <== mon.2 v2:172.21.15.102:3300/0 5 ==== fsmap(e 36) ==== 8370+0+0 (secure 0 0 0) 0x7f61dc00f040 con 0x556b8c882e50 2024-09-10T09:48:44.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:43.931+0000 7f61ed01b740 10 client.27458 fetch_fsmap finished waiting for FSMap version 36 2024-09-10T09:48:44.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:43.931+0000 7f61ed01b740 10 client.27458 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-10T09:48:44.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:43.931+0000 7f61ed01b740 20 client.27458 populate_metadata read hostname 'smithi005' 2024-09-10T09:48:44.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:43.932+0000 7f61ed01b740 1 --2- 172.21.15.5:0/3619522820 >> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] conn(0x556b8c906960 0x556b8c908d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:48:44.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:43.932+0000 7f61ed01b740 4 client.27458 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T09:48:44.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:43.932+0000 7f61ed01b740 1 -- 172.21.15.5:0/3619522820 --> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556b8c7e5e60 con 0x556b8c906960 2024-09-10T09:48:44.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:43.932+0000 7f61e3400640 1 --2- 172.21.15.5:0/3619522820 >> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] conn(0x556b8c906960 0x556b8c908d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:48:44.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:43.932+0000 7f61e3400640 1 --2- 172.21.15.5:0/3619522820 >> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] conn(0x556b8c906960 0x556b8c908d80 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c8442669c35e74e8 server_cookie=822c6533d8a0ec74 in_seq=0 out_seq=0 2024-09-10T09:48:44.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:43.932+0000 7f61cd600640 10 client.27458 ms_handle_connect on v2:172.21.15.159:6836/620627639 2024-09-10T09:48:44.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:43.933+0000 7f61cd600640 1 -- 172.21.15.5:0/3619522820 <== mds.2 v2:172.21.15.159:6836/620627639 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556b8c7e5e60 con 0x556b8c906960 2024-09-10T09:48:44.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:43.933+0000 7f61cd600640 10 client.27458 handle_command_reply: tid=0 2024-09-10T09:48:44.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:43.933+0000 7f61ed01b740 1 client.27458 shutdown 2024-09-10T09:48:44.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:43.933+0000 7f61ed01b740 1 -- 172.21.15.5:0/3619522820 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556b8c882e50 msgr2=0x556b8c887560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:48:44.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:43.933+0000 7f61ed01b740 1 --2- 172.21.15.5:0/3619522820 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556b8c882e50 0x556b8c887560 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f61dc005ae0 tx=0x7f61dc004c20 comp rx=0 tx=0).stop 2024-09-10T09:48:44.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:43.933+0000 7f61ed01b740 1 -- 172.21.15.5:0/3619522820 shutdown_connections 2024-09-10T09:48:44.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:43.933+0000 7f61ed01b740 1 --2- 172.21.15.5:0/3619522820 >> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] conn(0x556b8c906960 0x556b8c908d80 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:44.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:43.933+0000 7f61ed01b740 1 --2- 172.21.15.5:0/3619522820 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556b8c882e50 0x556b8c887560 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:44.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:43.933+0000 7f61ed01b740 1 --2- 172.21.15.5:0/3619522820 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556b8c878590 0x556b8c887020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:44.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:43.933+0000 7f61ed01b740 1 --2- 172.21.15.5:0/3619522820 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556b8c877c70 0x556b8c886ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:44.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:43.933+0000 7f61cd600640 0 client.27458 ms_handle_reset on v2:172.21.15.159:6836/620627639 2024-09-10T09:48:44.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:43.933+0000 7f61ed01b740 1 -- 172.21.15.5:0/3619522820 >> 172.21.15.5:0/3619522820 conn(0x556b8c874200 msgr2=0x556b8c873380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:48:44.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:43.933+0000 7f61ed01b740 1 -- 172.21.15.5:0/3619522820 shutdown_connections 2024-09-10T09:48:44.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:43.933+0000 7f61ed01b740 1 -- 172.21.15.5:0/3619522820 wait complete. 2024-09-10T09:48:44.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:43.933+0000 7f61ed01b740 20 client.27458 trim_cache size 0 max 16384 2024-09-10T09:48:44.347 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-10T09:48:44.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:43.934+0000 7f61ea400640 1 -- 172.21.15.5:0/2917614266 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f61c80a2c00 msgr2=0x7f61c80a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:48:44.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:43.934+0000 7f61ea400640 1 --2- 172.21.15.5:0/2917614266 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f61c80a2c00 0x7f61c80a50c0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x556b8c875060 tx=0x7f61d4005f90 comp rx=0 tx=0).stop 2024-09-10T09:48:44.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:43.934+0000 7f61ea400640 1 -- 172.21.15.5:0/2917614266 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f61e4102790 msgr2=0x7f61e419ab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:48:44.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:43.934+0000 7f61ea400640 1 --2- 172.21.15.5:0/2917614266 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f61e4102790 0x7f61e419ab70 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f61d0006010 tx=0x7f61d00042f0 comp rx=0 tx=0).stop 2024-09-10T09:48:44.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:43.934+0000 7f61ea400640 1 -- 172.21.15.5:0/2917614266 shutdown_connections 2024-09-10T09:48:44.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:43.934+0000 7f61ea400640 1 --2- 172.21.15.5:0/2917614266 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f61c80a2c00 0x7f61c80a50c0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:44.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:43.934+0000 7f61ea400640 1 --2- 172.21.15.5:0/2917614266 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f61e4106f10 0x7f61e419ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:44.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:43.934+0000 7f61ea400640 1 --2- 172.21.15.5:0/2917614266 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f61e4102790 0x7f61e419ab70 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:44.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:43.934+0000 7f61ea400640 1 --2- 172.21.15.5:0/2917614266 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f61e4101de0 0x7f61e419a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:44.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:43.934+0000 7f61ea400640 1 -- 172.21.15.5:0/2917614266 >> 172.21.15.5:0/2917614266 conn(0x7f61e40fd8c0 msgr2=0x7f61e40feb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:48:44.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:43.934+0000 7f61ea400640 1 -- 172.21.15.5:0/2917614266 shutdown_connections 2024-09-10T09:48:44.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:43.934+0000 7f61ea400640 1 -- 172.21.15.5:0/2917614266 wait complete. 2024-09-10T09:48:45.055 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-10T09:48:45.101 INFO:tasks.workunit.client.0.smithi005.stdout: 1 650014 6.44 MB/sec execute 386 sec latency 2082.179 ms 2024-09-10T09:48:45.433 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:48:45.761 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:48:45 smithi005 ceph-mon[30350]: pgmap v705: 129 pgs: 129 active+clean; 245 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.5 MiB/s wr, 169 op/s 2024-09-10T09:48:45.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:48:45 smithi159 ceph-mon[35513]: pgmap v705: 129 pgs: 129 active+clean; 245 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.5 MiB/s wr, 169 op/s 2024-09-10T09:48:45.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:48:45 smithi102 ceph-mon[35442]: pgmap v705: 129 pgs: 129 active+clean; 245 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.5 MiB/s wr, 169 op/s 2024-09-10T09:48:46.114 INFO:tasks.workunit.client.0.smithi005.stdout: 1 650014 6.42 MB/sec execute 387 sec latency 3094.624 ms 2024-09-10T09:48:46.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:46.101+0000 7f8587e00640 1 -- 172.21.15.5:0/178853677 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f85801049a0 msgr2=0x7f85801089f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:48:46.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:46.101+0000 7f8587e00640 1 --2- 172.21.15.5:0/178853677 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f85801049a0 0x7f85801089f0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f8574009900 tx=0x7f857402a3f0 comp rx=0 tx=0).stop 2024-09-10T09:48:46.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:46.102+0000 7f8587e00640 1 -- 172.21.15.5:0/178853677 shutdown_connections 2024-09-10T09:48:46.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:46.102+0000 7f8587e00640 1 --2- 172.21.15.5:0/178853677 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8580109120 0x7f8580110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:46.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:46.102+0000 7f8587e00640 1 --2- 172.21.15.5:0/178853677 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f85801049a0 0x7f85801089f0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:46.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:46.102+0000 7f8587e00640 1 --2- 172.21.15.5:0/178853677 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8580103ff0 0x7f85801043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:46.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:46.102+0000 7f8587e00640 1 -- 172.21.15.5:0/178853677 >> 172.21.15.5:0/178853677 conn(0x7f85800ffa90 msgr2=0x7f8580101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:48:46.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:46.102+0000 7f8587e00640 1 -- 172.21.15.5:0/178853677 shutdown_connections 2024-09-10T09:48:46.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:46.102+0000 7f8587e00640 1 -- 172.21.15.5:0/178853677 wait complete. 2024-09-10T09:48:46.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:46.103+0000 7f8587e00640 1 Processor -- start 2024-09-10T09:48:46.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:46.103+0000 7f8587e00640 1 -- start start 2024-09-10T09:48:46.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:46.104+0000 7f8587e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8580103ff0 0x7f8580110b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:48:46.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:46.104+0000 7f8587e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f85801049a0 0x7f858010dc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:48:46.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:46.104+0000 7f8587e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8580109120 0x7f858010e1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:48:46.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:46.104+0000 7f8587e00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f8580114210 con 0x7f8580103ff0 2024-09-10T09:48:46.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:46.104+0000 7f8587e00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f8580114090 con 0x7f85801049a0 2024-09-10T09:48:46.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:46.104+0000 7f8587e00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f8580114390 con 0x7f8580109120 2024-09-10T09:48:46.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:46.105+0000 7f8585400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8580103ff0 0x7f8580110b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:48:46.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:46.105+0000 7f8585400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8580103ff0 0x7f8580110b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47778/0 (socket says 172.21.15.5:47778) 2024-09-10T09:48:46.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:46.105+0000 7f8585400640 1 -- 172.21.15.5:0/710310197 learned_addr learned my addr 172.21.15.5:0/710310197 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:48:46.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:46.105+0000 7f8584a00640 1 --2- 172.21.15.5:0/710310197 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f85801049a0 0x7f858010dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:48:46.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:46.106+0000 7f8585400640 1 -- 172.21.15.5:0/710310197 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8580109120 msgr2=0x7f858010e1d0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-10T09:48:46.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:46.106+0000 7f8585400640 1 --2- 172.21.15.5:0/710310197 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8580109120 0x7f858010e1d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:46.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:46.106+0000 7f8585400640 1 -- 172.21.15.5:0/710310197 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f85801049a0 msgr2=0x7f858010dc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:48:46.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:46.106+0000 7f8585400640 1 --2- 172.21.15.5:0/710310197 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f85801049a0 0x7f858010dc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:46.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:46.106+0000 7f8585400640 1 -- 172.21.15.5:0/710310197 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f858010ea90 con 0x7f8580103ff0 2024-09-10T09:48:46.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:46.106+0000 7f8584a00640 1 --2- 172.21.15.5:0/710310197 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f85801049a0 0x7f858010dc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:48:46.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:46.106+0000 7f8585400640 1 --2- 172.21.15.5:0/710310197 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8580103ff0 0x7f8580110b50 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f857000e9f0 tx=0x7f857000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:48:46.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:46.107+0000 7f857e000640 1 -- 172.21.15.5:0/710310197 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f857000cd80 con 0x7f8580103ff0 2024-09-10T09:48:46.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:46.107+0000 7f857e000640 1 -- 172.21.15.5:0/710310197 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8570005500 con 0x7f8580103ff0 2024-09-10T09:48:46.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:46.107+0000 7f8587e00640 1 -- 172.21.15.5:0/710310197 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85801a0850 con 0x7f8580103ff0 2024-09-10T09:48:46.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:46.107+0000 7f857e000640 1 -- 172.21.15.5:0/710310197 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8570010690 con 0x7f8580103ff0 2024-09-10T09:48:46.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:46.107+0000 7f8587e00640 1 -- 172.21.15.5:0/710310197 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85801a0d60 con 0x7f8580103ff0 2024-09-10T09:48:46.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:46.112+0000 7f858a811740 1 Processor -- start 2024-09-10T09:48:46.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:46.112+0000 7f857e000640 1 -- 172.21.15.5:0/710310197 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f85700046c0 con 0x7f8580103ff0 2024-09-10T09:48:46.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:46.112+0000 7f858a811740 1 -- start start 2024-09-10T09:48:46.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:46.112+0000 7f858a811740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5615d3b19c70 0x5615d3b1a050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:48:46.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:46.112+0000 7f858a811740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5615d3b1a590 0x5615d3b24910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:48:46.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:46.113+0000 7f858a811740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5615d3b24e50 0x5615d3b27240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:48:46.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:46.113+0000 7f858a811740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5615d3a803e0 con 0x5615d3b19c70 2024-09-10T09:48:46.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:46.113+0000 7f858a811740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x5615d3a27d80 con 0x5615d3b1a590 2024-09-10T09:48:46.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:46.113+0000 7f858a811740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5615d3a7c3a0 con 0x5615d3b24e50 2024-09-10T09:48:46.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:46.113+0000 7f8585e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5615d3b19c70 0x5615d3b1a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:48:46.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:46.113+0000 7f8584a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5615d3b24e50 0x5615d3b27240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:48:46.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:46.113+0000 7f8584a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5615d3b24e50 0x5615d3b27240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:56410/0 (socket says 172.21.15.5:56410) 2024-09-10T09:48:46.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:46.113+0000 7f8584a00640 1 -- 172.21.15.5:0/805870756 learned_addr learned my addr 172.21.15.5:0/805870756 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:48:46.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:46.114+0000 7f857e000640 1 --2- 172.21.15.5:0/710310197 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f85680a2c00 0x7f85680a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:48:46.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:46.114+0000 7f857e000640 1 -- 172.21.15.5:0/710310197 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f85700cf920 con 0x7f8580103ff0 2024-09-10T09:48:46.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:46.114+0000 7f8585400640 1 --2- 172.21.15.5:0/805870756 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5615d3b1a590 0x5615d3b24910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:48:46.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:46.114+0000 7f8585e00640 1 -- 172.21.15.5:0/805870756 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5615d3b24e50 msgr2=0x5615d3b27240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:48:46.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:46.114+0000 7f8585e00640 1 --2- 172.21.15.5:0/805870756 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5615d3b24e50 0x5615d3b27240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:46.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:46.114+0000 7f8585e00640 1 -- 172.21.15.5:0/805870756 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5615d3b1a590 msgr2=0x5615d3b24910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:48:46.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:46.114+0000 7f8585e00640 1 --2- 172.21.15.5:0/805870756 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5615d3b1a590 0x5615d3b24910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:46.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:46.114+0000 7f8585e00640 1 -- 172.21.15.5:0/805870756 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5615d3a46ae0 con 0x5615d3b19c70 2024-09-10T09:48:46.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:46.115+0000 7f8585e00640 1 --2- 172.21.15.5:0/805870756 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5615d3b19c70 0x5615d3b1a050 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f8578009f20 tx=0x7f857800b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5895c6ccc007bfb9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:48:46.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:46.115+0000 7f8585e00640 1 --2- 172.21.15.5:0/710310197 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f85680a2c00 0x7f85680a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:48:46.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:46.116+0000 7f856e600640 1 -- 172.21.15.5:0/805870756 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85780135d0 con 0x5615d3b19c70 2024-09-10T09:48:46.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:46.116+0000 7f856e600640 1 -- 172.21.15.5:0/805870756 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8578013c90 con 0x5615d3b19c70 2024-09-10T09:48:46.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:46.116+0000 7f856e600640 1 -- 172.21.15.5:0/805870756 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f857801b330 con 0x5615d3b19c70 2024-09-10T09:48:46.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:46.116+0000 7f858a811740 1 -- 172.21.15.5:0/805870756 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5615d3b19c70 msgr2=0x5615d3b1a050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:48:46.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:46.116+0000 7f858a811740 1 --2- 172.21.15.5:0/805870756 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5615d3b19c70 0x5615d3b1a050 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f8578009f20 tx=0x7f857800b8a0 comp rx=0 tx=0).stop 2024-09-10T09:48:46.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:46.116+0000 7f8585e00640 1 --2- 172.21.15.5:0/710310197 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f85680a2c00 0x7f85680a50c0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f8578009fd0 tx=0x7f857801b4d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:48:46.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:46.117+0000 7f858a811740 1 -- 172.21.15.5:0/805870756 shutdown_connections 2024-09-10T09:48:46.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:46.117+0000 7f858a811740 1 --2- 172.21.15.5:0/805870756 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5615d3b24e50 0x5615d3b27240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:46.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:46.117+0000 7f858a811740 1 --2- 172.21.15.5:0/805870756 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5615d3b1a590 0x5615d3b24910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:46.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:46.117+0000 7f858a811740 1 --2- 172.21.15.5:0/805870756 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5615d3b19c70 0x5615d3b1a050 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:46.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:46.117+0000 7f858a811740 1 -- 172.21.15.5:0/805870756 >> 172.21.15.5:0/805870756 conn(0x5615d3b16200 msgr2=0x5615d3b165e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:48:46.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:46.117+0000 7f858a811740 1 -- 172.21.15.5:0/805870756 shutdown_connections 2024-09-10T09:48:46.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:46.117+0000 7f858a811740 1 -- 172.21.15.5:0/805870756 wait complete. 2024-09-10T09:48:46.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:46.117+0000 7f858a811740 1 -- start start 2024-09-10T09:48:46.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:46.118+0000 7f858a811740 1 Processor -- start 2024-09-10T09:48:46.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:46.118+0000 7f858a811740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5615d3b19c70 0x5615d3b21d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:48:46.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:46.118+0000 7f858a811740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5615d3b1a590 0x5615d3b22260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:48:46.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:46.118+0000 7f858a811740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5615d3b24e50 0x5615d3b227a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:48:46.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:46.118+0000 7f858a811740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5615d3b27a20 con 0x5615d3b1a590 2024-09-10T09:48:46.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:46.118+0000 7f858a811740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x5615d3a3b0f0 con 0x5615d3b24e50 2024-09-10T09:48:46.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:46.118+0000 7f858a811740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5615d3b27ba0 con 0x5615d3b19c70 2024-09-10T09:48:46.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:46.118+0000 7f858a811740 10 client.0 _ll_register_callbacks cb 0x5615d3a22bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:48:46.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:46.118+0000 7f8585e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5615d3b24e50 0x5615d3b227a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:48:46.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:46.118+0000 7f8585e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5615d3b24e50 0x5615d3b227a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:41478/0 (socket says 172.21.15.5:41478) 2024-09-10T09:48:46.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:46.118+0000 7f8585e00640 1 -- 172.21.15.5:0/1506036890 learned_addr learned my addr 172.21.15.5:0/1506036890 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:48:46.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:46.119+0000 7f8585400640 1 --2- 172.21.15.5:0/1506036890 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5615d3b19c70 0x5615d3b21d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:48:46.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:46.119+0000 7f8585e00640 1 -- 172.21.15.5:0/1506036890 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5615d3b19c70 msgr2=0x5615d3b21d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:48:46.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:46.119+0000 7f8585e00640 1 --2- 172.21.15.5:0/1506036890 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5615d3b19c70 0x5615d3b21d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:46.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:46.119+0000 7f8585e00640 1 -- 172.21.15.5:0/1506036890 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5615d3b1a590 msgr2=0x5615d3b22260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:48:46.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:46.119+0000 7f8585e00640 1 --2- 172.21.15.5:0/1506036890 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5615d3b1a590 0x5615d3b22260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:46.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:46.119+0000 7f8585e00640 1 -- 172.21.15.5:0/1506036890 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5615d37f13c0 con 0x5615d3b24e50 2024-09-10T09:48:46.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:46.120+0000 7f8585e00640 1 --2- 172.21.15.5:0/1506036890 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5615d3b24e50 0x5615d3b227a0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f8578012ae0 tx=0x7f8578014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=208f19a3237c76d0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:48:46.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:46.120+0000 7f8562a00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-09-10T09:48:46.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:46.120+0000 7f8562a00640 1 -- 172.21.15.5:0/1506036890 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f857802e070 con 0x5615d3b24e50 2024-09-10T09:48:46.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:46.120+0000 7f858a811740 1 -- 172.21.15.5:0/1506036890 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8578013c90 con 0x5615d3b24e50 2024-09-10T09:48:46.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:46.120+0000 7f8562a00640 1 -- 172.21.15.5:0/1506036890 <== mon.2 v2:172.21.15.102:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f857801b6e0 con 0x5615d3b24e50 2024-09-10T09:48:46.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:46.120+0000 7f8562a00640 1 -- 172.21.15.5:0/1506036890 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f857803a050 con 0x5615d3b24e50 2024-09-10T09:48:46.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:46.121+0000 7f858a811740 10 client.27470 fetch_fsmap learned FSMap version 36 2024-09-10T09:48:46.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:46.121+0000 7f858a811740 1 -- 172.21.15.5:0/1506036890 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x5615d3b33a90 con 0x5615d3b24e50 2024-09-10T09:48:46.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:46.121+0000 7f8562a00640 1 -- 172.21.15.5:0/1506036890 <== mon.2 v2:172.21.15.102:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8578013c90 con 0x5615d3b24e50 2024-09-10T09:48:46.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:46.121+0000 7f8562a00640 1 -- 172.21.15.5:0/1506036890 <== mon.2 v2:172.21.15.102:3300/0 5 ==== fsmap(e 36) ==== 8370+0+0 (secure 0 0 0) 0x7f8578028430 con 0x5615d3b24e50 2024-09-10T09:48:46.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:46.121+0000 7f858a811740 10 client.27470 fetch_fsmap finished waiting for FSMap version 36 2024-09-10T09:48:46.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:46.121+0000 7f858a811740 10 client.27470 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-10T09:48:46.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:46.121+0000 7f858a811740 20 client.27470 populate_metadata read hostname 'smithi005' 2024-09-10T09:48:46.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:46.121+0000 7f858a811740 1 --2- 172.21.15.5:0/1506036890 >> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] conn(0x5615d3b34e40 0x5615d3b37230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:48:46.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:46.121+0000 7f858a811740 4 client.27470 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:48:46.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:46.121+0000 7f858a811740 1 -- 172.21.15.5:0/1506036890 --> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5615d3a8d950 con 0x5615d3b34e40 2024-09-10T09:48:46.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:46.122+0000 7f8585400640 1 --2- 172.21.15.5:0/1506036890 >> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] conn(0x5615d3b34e40 0x5615d3b37230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:48:46.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:46.122+0000 7f8585400640 1 --2- 172.21.15.5:0/1506036890 >> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] conn(0x5615d3b34e40 0x5615d3b37230 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=96b75bb2e4ef80e2 server_cookie=9e12439c09e30e6d in_seq=0 out_seq=0 2024-09-10T09:48:46.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:46.122+0000 7f8562a00640 10 client.27470 ms_handle_connect on v2:172.21.15.159:6838/2073739599 2024-09-10T09:48:46.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:46.123+0000 7f8562a00640 1 -- 172.21.15.5:0/1506036890 <== mds.1 v2:172.21.15.159:6838/2073739599 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5615d3a8d950 con 0x5615d3b34e40 2024-09-10T09:48:46.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:46.123+0000 7f8562a00640 10 client.27470 handle_command_reply: tid=0 2024-09-10T09:48:46.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:46.123+0000 7f858a811740 1 client.27470 shutdown 2024-09-10T09:48:46.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:46.123+0000 7f858a811740 1 -- 172.21.15.5:0/1506036890 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5615d3b24e50 msgr2=0x5615d3b227a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:48:46.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:46.123+0000 7f858a811740 1 --2- 172.21.15.5:0/1506036890 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5615d3b24e50 0x5615d3b227a0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f8578012ae0 tx=0x7f8578014000 comp rx=0 tx=0).stop 2024-09-10T09:48:46.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:46.123+0000 7f858a811740 1 -- 172.21.15.5:0/1506036890 shutdown_connections 2024-09-10T09:48:46.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:46.123+0000 7f858a811740 1 --2- 172.21.15.5:0/1506036890 >> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] conn(0x5615d3b34e40 0x5615d3b37230 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:46.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:46.123+0000 7f858a811740 1 --2- 172.21.15.5:0/1506036890 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5615d3b24e50 0x5615d3b227a0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:46.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:46.123+0000 7f858a811740 1 --2- 172.21.15.5:0/1506036890 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5615d3b1a590 0x5615d3b22260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:46.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:46.123+0000 7f8562a00640 0 client.27470 ms_handle_reset on v2:172.21.15.159:6838/2073739599 2024-09-10T09:48:46.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:46.123+0000 7f858a811740 1 --2- 172.21.15.5:0/1506036890 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5615d3b19c70 0x5615d3b21d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:46.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:46.123+0000 7f858a811740 1 -- 172.21.15.5:0/1506036890 >> 172.21.15.5:0/1506036890 conn(0x5615d3b16200 msgr2=0x5615d3b15380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:48:46.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:46.123+0000 7f858a811740 1 -- 172.21.15.5:0/1506036890 shutdown_connections 2024-09-10T09:48:46.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:46.124+0000 7f858a811740 1 -- 172.21.15.5:0/1506036890 wait complete. 2024-09-10T09:48:46.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:46.124+0000 7f858a811740 20 client.27470 trim_cache size 0 max 16384 2024-09-10T09:48:46.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:46.146+0000 7f858a811740 1 Processor -- start 2024-09-10T09:48:46.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:46.146+0000 7f858a811740 1 -- start start 2024-09-10T09:48:46.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:46.147+0000 7f858a811740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5615d3b19c70 0x5615d3b1c720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:48:46.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:46.147+0000 7f858a811740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5615d3b1a590 0x5615d3b3ca80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:48:46.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:46.147+0000 7f858a811740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5615d3b24e50 0x5615d3b3cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:48:46.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:46.147+0000 7f858a811740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f85801bdd20 con 0x5615d3b1a590 2024-09-10T09:48:46.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:46.147+0000 7f858a811740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f85801bf860 con 0x5615d3b19c70 2024-09-10T09:48:46.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:46.147+0000 7f858a811740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f85801be3f0 con 0x5615d3b24e50 2024-09-10T09:48:46.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:46.147+0000 7f8584a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5615d3b1a590 0x5615d3b3ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:48:46.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:46.147+0000 7f8585e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5615d3b24e50 0x5615d3b3cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:48:46.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:46.147+0000 7f8585400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5615d3b19c70 0x5615d3b1c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:48:46.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:46.147+0000 7f8584a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5615d3b1a590 0x5615d3b3ca80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47804/0 (socket says 172.21.15.5:47804) 2024-09-10T09:48:46.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:46.147+0000 7f8585e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5615d3b24e50 0x5615d3b3cfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:56428/0 (socket says 172.21.15.5:56428) 2024-09-10T09:48:46.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:46.147+0000 7f8584a00640 1 -- 172.21.15.5:0/1515281473 learned_addr learned my addr 172.21.15.5:0/1515281473 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:48:46.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:46.148+0000 7f8585e00640 1 -- 172.21.15.5:0/1515281473 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5615d3b19c70 msgr2=0x5615d3b1c720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:48:46.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:46.148+0000 7f8585e00640 1 --2- 172.21.15.5:0/1515281473 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5615d3b19c70 0x5615d3b1c720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:46.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:46.148+0000 7f8585e00640 1 -- 172.21.15.5:0/1515281473 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5615d3b1a590 msgr2=0x5615d3b3ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:48:46.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:46.148+0000 7f8585e00640 1 --2- 172.21.15.5:0/1515281473 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5615d3b1a590 0x5615d3b3ca80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:46.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:46.148+0000 7f8585e00640 1 -- 172.21.15.5:0/1515281473 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5615d3b17390 con 0x5615d3b24e50 2024-09-10T09:48:46.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:46.148+0000 7f8585400640 1 --2- 172.21.15.5:0/1515281473 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5615d3b19c70 0x5615d3b1c720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:48:46.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:46.148+0000 7f8585e00640 1 --2- 172.21.15.5:0/1515281473 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5615d3b24e50 0x5615d3b3cfc0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f8578009610 tx=0x7f85780094b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9c9abc12f2fa82ac server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:48:46.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:46.148+0000 7f856dc00640 1 -- 172.21.15.5:0/1515281473 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f857800e070 con 0x5615d3b24e50 2024-09-10T09:48:46.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:46.148+0000 7f856dc00640 1 -- 172.21.15.5:0/1515281473 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f857801be90 con 0x5615d3b24e50 2024-09-10T09:48:46.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:46.148+0000 7f856dc00640 1 -- 172.21.15.5:0/1515281473 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f857802e040 con 0x5615d3b24e50 2024-09-10T09:48:46.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:46.149+0000 7f858a811740 1 -- 172.21.15.5:0/1515281473 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5615d3b24e50 msgr2=0x5615d3b3cfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:48:46.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:46.149+0000 7f858a811740 1 --2- 172.21.15.5:0/1515281473 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5615d3b24e50 0x5615d3b3cfc0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f8578009610 tx=0x7f85780094b0 comp rx=0 tx=0).stop 2024-09-10T09:48:46.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:46.149+0000 7f858a811740 1 -- 172.21.15.5:0/1515281473 shutdown_connections 2024-09-10T09:48:46.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:46.149+0000 7f858a811740 1 --2- 172.21.15.5:0/1515281473 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5615d3b24e50 0x5615d3b3cfc0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:46.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:46.149+0000 7f858a811740 1 --2- 172.21.15.5:0/1515281473 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5615d3b1a590 0x5615d3b3ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:46.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:46.149+0000 7f858a811740 1 --2- 172.21.15.5:0/1515281473 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5615d3b19c70 0x5615d3b1c720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:46.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:46.149+0000 7f858a811740 1 -- 172.21.15.5:0/1515281473 >> 172.21.15.5:0/1515281473 conn(0x5615d3b16200 msgr2=0x5615d3b23da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:48:46.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:46.149+0000 7f858a811740 1 -- 172.21.15.5:0/1515281473 shutdown_connections 2024-09-10T09:48:46.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:46.149+0000 7f858a811740 1 -- 172.21.15.5:0/1515281473 wait complete. 2024-09-10T09:48:46.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:46.150+0000 7f858a811740 1 -- start start 2024-09-10T09:48:46.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:46.150+0000 7f858a811740 1 Processor -- start 2024-09-10T09:48:46.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:46.150+0000 7f858a811740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5615d3b19c70 0x5615d3b2d290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:48:46.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:46.150+0000 7f858a811740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5615d3b1a590 0x5615d3b285c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:48:46.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:46.150+0000 7f858a811740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5615d3b24e50 0x5615d3b28b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:48:46.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:46.150+0000 7f858a811740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f85801bf190 con 0x5615d3b24e50 2024-09-10T09:48:46.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:46.150+0000 7f858a811740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f85801beac0 con 0x5615d3b19c70 2024-09-10T09:48:46.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:46.150+0000 7f858a811740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f85801bbd00 con 0x5615d3b1a590 2024-09-10T09:48:46.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:46.150+0000 7f858a811740 10 client.0 _ll_register_callbacks cb 0x7f8574009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:48:46.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:46.151+0000 7f8585e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5615d3b24e50 0x5615d3b28b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:48:46.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:46.151+0000 7f8585400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5615d3b19c70 0x5615d3b2d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:48:46.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:46.151+0000 7f8585400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5615d3b19c70 0x5615d3b2d290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:41498/0 (socket says 172.21.15.5:41498) 2024-09-10T09:48:46.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:46.151+0000 7f8585400640 1 -- 172.21.15.5:0/1746572425 learned_addr learned my addr 172.21.15.5:0/1746572425 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:48:46.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:46.151+0000 7f8584a00640 1 --2- 172.21.15.5:0/1746572425 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5615d3b1a590 0x5615d3b285c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:48:46.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:46.151+0000 7f8585400640 1 -- 172.21.15.5:0/1746572425 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5615d3b1a590 msgr2=0x5615d3b285c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:48:46.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:46.151+0000 7f8585400640 1 --2- 172.21.15.5:0/1746572425 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5615d3b1a590 0x5615d3b285c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:46.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:46.151+0000 7f8585400640 1 -- 172.21.15.5:0/1746572425 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5615d3b24e50 msgr2=0x5615d3b28b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:48:46.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:46.151+0000 7f8585400640 1 --2- 172.21.15.5:0/1746572425 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5615d3b24e50 0x5615d3b28b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:46.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:46.151+0000 7f8585400640 1 -- 172.21.15.5:0/1746572425 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5615d3b29040 con 0x5615d3b19c70 2024-09-10T09:48:46.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:46.151+0000 7f8585400640 1 --2- 172.21.15.5:0/1746572425 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5615d3b19c70 0x5615d3b2d290 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f8570098700 tx=0x7f857009bbd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3042ebd032044eed server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:48:46.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:46.151+0000 7f8562a00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-09-10T09:48:46.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:46.152+0000 7f8562a00640 1 -- 172.21.15.5:0/1746572425 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85700a00e0 con 0x5615d3b19c70 2024-09-10T09:48:46.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:46.152+0000 7f8562a00640 1 -- 172.21.15.5:0/1746572425 <== mon.2 v2:172.21.15.102:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f85700983a0 con 0x5615d3b19c70 2024-09-10T09:48:46.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:46.152+0000 7f858a811740 1 -- 172.21.15.5:0/1746572425 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f857801be90 con 0x5615d3b19c70 2024-09-10T09:48:46.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:46.152+0000 7f8562a00640 1 -- 172.21.15.5:0/1746572425 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f85700a49a0 con 0x5615d3b19c70 2024-09-10T09:48:46.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:46.152+0000 7f858a811740 10 client.27476 fetch_fsmap learned FSMap version 36 2024-09-10T09:48:46.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:46.152+0000 7f858a811740 1 -- 172.21.15.5:0/1746572425 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x5615d3b9f7a0 con 0x5615d3b19c70 2024-09-10T09:48:46.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:46.152+0000 7f8562a00640 1 -- 172.21.15.5:0/1746572425 <== mon.2 v2:172.21.15.102:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f857801be90 con 0x5615d3b19c70 2024-09-10T09:48:46.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:46.152+0000 7f8562a00640 1 -- 172.21.15.5:0/1746572425 <== mon.2 v2:172.21.15.102:3300/0 5 ==== fsmap(e 36) ==== 8370+0+0 (secure 0 0 0) 0x7f857009ad10 con 0x5615d3b19c70 2024-09-10T09:48:46.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:46.153+0000 7f858a811740 10 client.27476 fetch_fsmap finished waiting for FSMap version 36 2024-09-10T09:48:46.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:46.153+0000 7f858a811740 10 client.27476 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-10T09:48:46.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:46.153+0000 7f858a811740 20 client.27476 populate_metadata read hostname 'smithi005' 2024-09-10T09:48:46.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:46.153+0000 7f858a811740 1 --2- 172.21.15.5:0/1746572425 >> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] conn(0x5615d3b9fec0 0x5615d3ba22e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:48:46.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:46.153+0000 7f858a811740 4 client.27476 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T09:48:46.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:46.153+0000 7f858a811740 1 -- 172.21.15.5:0/1746572425 --> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5615d3a87e60 con 0x5615d3b9fec0 2024-09-10T09:48:46.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:46.153+0000 7f8584a00640 1 --2- 172.21.15.5:0/1746572425 >> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] conn(0x5615d3b9fec0 0x5615d3ba22e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:48:46.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:46.154+0000 7f8584a00640 1 --2- 172.21.15.5:0/1746572425 >> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] conn(0x5615d3b9fec0 0x5615d3ba22e0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=14c507b2d3e529b3 server_cookie=51bb58e77ea783da in_seq=0 out_seq=0 2024-09-10T09:48:46.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:46.154+0000 7f8562a00640 10 client.27476 ms_handle_connect on v2:172.21.15.159:6838/2073739599 2024-09-10T09:48:46.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:46.154+0000 7f8562a00640 1 -- 172.21.15.5:0/1746572425 <== mds.1 v2:172.21.15.159:6838/2073739599 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5615d3a87e60 con 0x5615d3b9fec0 2024-09-10T09:48:46.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:46.154+0000 7f8562a00640 10 client.27476 handle_command_reply: tid=0 2024-09-10T09:48:46.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:46.154+0000 7f858a811740 1 client.27476 shutdown 2024-09-10T09:48:46.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:46.155+0000 7f858a811740 1 -- 172.21.15.5:0/1746572425 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5615d3b19c70 msgr2=0x5615d3b2d290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:48:46.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:46.155+0000 7f858a811740 1 --2- 172.21.15.5:0/1746572425 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5615d3b19c70 0x5615d3b2d290 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f8570098700 tx=0x7f857009bbd0 comp rx=0 tx=0).stop 2024-09-10T09:48:46.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:46.155+0000 7f858a811740 1 -- 172.21.15.5:0/1746572425 shutdown_connections 2024-09-10T09:48:46.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:46.155+0000 7f858a811740 1 --2- 172.21.15.5:0/1746572425 >> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] conn(0x5615d3b9fec0 0x5615d3ba22e0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:46.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:46.155+0000 7f858a811740 1 --2- 172.21.15.5:0/1746572425 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5615d3b24e50 0x5615d3b28b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:46.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:46.155+0000 7f858a811740 1 --2- 172.21.15.5:0/1746572425 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5615d3b1a590 0x5615d3b285c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:46.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:46.155+0000 7f858a811740 1 --2- 172.21.15.5:0/1746572425 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5615d3b19c70 0x5615d3b2d290 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:46.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:46.155+0000 7f858a811740 1 -- 172.21.15.5:0/1746572425 >> 172.21.15.5:0/1746572425 conn(0x5615d3b16200 msgr2=0x5615d3b23da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:48:46.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:46.155+0000 7f8562a00640 0 client.27476 ms_handle_reset on v2:172.21.15.159:6838/2073739599 2024-09-10T09:48:46.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:46.155+0000 7f858a811740 1 -- 172.21.15.5:0/1746572425 shutdown_connections 2024-09-10T09:48:46.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:46.155+0000 7f858a811740 1 -- 172.21.15.5:0/1746572425 wait complete. 2024-09-10T09:48:46.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:46.155+0000 7f858a811740 20 client.27476 trim_cache size 0 max 16384 2024-09-10T09:48:46.178 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-10T09:48:46.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:46.156+0000 7f8587e00640 1 -- 172.21.15.5:0/710310197 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f85680a2c00 msgr2=0x7f85680a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:48:46.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:46.156+0000 7f8587e00640 1 --2- 172.21.15.5:0/710310197 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f85680a2c00 0x7f85680a50c0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f8578009fd0 tx=0x7f857801b4d0 comp rx=0 tx=0).stop 2024-09-10T09:48:46.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:46.156+0000 7f8587e00640 1 -- 172.21.15.5:0/710310197 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8580103ff0 msgr2=0x7f8580110b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:48:46.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:46.156+0000 7f8587e00640 1 --2- 172.21.15.5:0/710310197 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8580103ff0 0x7f8580110b50 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f857000e9f0 tx=0x7f857000edf0 comp rx=0 tx=0).stop 2024-09-10T09:48:46.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:46.156+0000 7f8587e00640 1 -- 172.21.15.5:0/710310197 shutdown_connections 2024-09-10T09:48:46.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:46.156+0000 7f8587e00640 1 --2- 172.21.15.5:0/710310197 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f85680a2c00 0x7f85680a50c0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:46.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:46.156+0000 7f8587e00640 1 --2- 172.21.15.5:0/710310197 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8580109120 0x7f858010e1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:46.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:46.156+0000 7f8587e00640 1 --2- 172.21.15.5:0/710310197 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f85801049a0 0x7f858010dc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:46.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:46.156+0000 7f8587e00640 1 --2- 172.21.15.5:0/710310197 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8580103ff0 0x7f8580110b50 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:46.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:46.156+0000 7f8587e00640 1 -- 172.21.15.5:0/710310197 >> 172.21.15.5:0/710310197 conn(0x7f85800ffa90 msgr2=0x7f8580100630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:48:46.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:46.157+0000 7f8587e00640 1 -- 172.21.15.5:0/710310197 shutdown_connections 2024-09-10T09:48:46.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:46.157+0000 7f8587e00640 1 -- 172.21.15.5:0/710310197 wait complete. 2024-09-10T09:48:47.114 INFO:tasks.workunit.client.0.smithi005.stdout: 1 650014 6.41 MB/sec execute 388 sec latency 4094.671 ms 2024-09-10T09:48:47.207 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-10T09:48:47.671 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:48:47.682 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:48:47 smithi005 ceph-mon[30350]: pgmap v706: 129 pgs: 129 active+clean; 246 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.6 MiB/s wr, 210 op/s 2024-09-10T09:48:47.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:48:47 smithi159 ceph-mon[35513]: pgmap v706: 129 pgs: 129 active+clean; 246 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.6 MiB/s wr, 210 op/s 2024-09-10T09:48:47.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:48:47 smithi102 ceph-mon[35442]: pgmap v706: 129 pgs: 129 active+clean; 246 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.6 MiB/s wr, 210 op/s 2024-09-10T09:48:47.898 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T09:48:48.114 INFO:tasks.workunit.client.0.smithi005.stdout: 1 650014 6.39 MB/sec execute 389 sec latency 5094.733 ms 2024-09-10T09:48:48.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:48.422+0000 7efd29c00640 1 -- 172.21.15.5:0/1087775962 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efd24076720 msgr2=0x7efd24076b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:48:48.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:48.422+0000 7efd29c00640 1 --2- 172.21.15.5:0/1087775962 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efd24076720 0x7efd24076b00 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7efd14009900 tx=0x7efd1402a3f0 comp rx=0 tx=0).stop 2024-09-10T09:48:48.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:48.422+0000 7efd29c00640 1 -- 172.21.15.5:0/1087775962 shutdown_connections 2024-09-10T09:48:48.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:48.422+0000 7efd29c00640 1 --2- 172.21.15.5:0/1087775962 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efd24112a30 0x7efd24114e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:48.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:48.422+0000 7efd29c00640 1 --2- 172.21.15.5:0/1087775962 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efd24077040 0x7efd241124f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:48.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:48.422+0000 7efd29c00640 1 --2- 172.21.15.5:0/1087775962 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efd24076720 0x7efd24076b00 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:48.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:48.422+0000 7efd29c00640 1 -- 172.21.15.5:0/1087775962 >> 172.21.15.5:0/1087775962 conn(0x7efd240ffa90 msgr2=0x7efd24101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:48:48.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:48.422+0000 7efd29c00640 1 -- 172.21.15.5:0/1087775962 shutdown_connections 2024-09-10T09:48:48.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:48.422+0000 7efd29c00640 1 -- 172.21.15.5:0/1087775962 wait complete. 2024-09-10T09:48:48.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:48.423+0000 7efd29c00640 1 Processor -- start 2024-09-10T09:48:48.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:48.423+0000 7efd29c00640 1 -- start start 2024-09-10T09:48:48.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:48.424+0000 7efd29c00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efd24076720 0x7efd241a46e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:48:48.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:48.424+0000 7efd29c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efd24077040 0x7efd241a4c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:48:48.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:48.424+0000 7efd29c00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efd24112a30 0x7efd2419e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:48:48.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:48.424+0000 7efd29c00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7efd241185f0 con 0x7efd24112a30 2024-09-10T09:48:48.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:48.424+0000 7efd29c00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7efd24118470 con 0x7efd24077040 2024-09-10T09:48:48.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:48.424+0000 7efd29c00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7efd24118770 con 0x7efd24076720 2024-09-10T09:48:48.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:48.424+0000 7efd22a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efd24076720 0x7efd241a46e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:48:48.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:48.424+0000 7efd1be00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efd24077040 0x7efd241a4c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:48:48.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:48.424+0000 7efd23400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efd24112a30 0x7efd2419e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:48:48.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:48.424+0000 7efd22a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efd24076720 0x7efd241a46e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:56470/0 (socket says 172.21.15.5:56470) 2024-09-10T09:48:48.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:48.424+0000 7efd1be00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efd24077040 0x7efd241a4c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:41516/0 (socket says 172.21.15.5:41516) 2024-09-10T09:48:48.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:48.424+0000 7efd22a00640 1 -- 172.21.15.5:0/4243870624 learned_addr learned my addr 172.21.15.5:0/4243870624 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:48:48.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:48.425+0000 7efd22a00640 1 -- 172.21.15.5:0/4243870624 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efd24077040 msgr2=0x7efd241a4c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:48:48.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:48.425+0000 7efd22a00640 1 --2- 172.21.15.5:0/4243870624 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efd24077040 0x7efd241a4c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:48.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:48.425+0000 7efd22a00640 1 -- 172.21.15.5:0/4243870624 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efd24112a30 msgr2=0x7efd2419e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:48:48.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:48.425+0000 7efd22a00640 1 --2- 172.21.15.5:0/4243870624 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efd24112a30 0x7efd2419e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:48.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:48.425+0000 7efd22a00640 1 -- 172.21.15.5:0/4243870624 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd2419efd0 con 0x7efd24076720 2024-09-10T09:48:48.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:48.425+0000 7efd22a00640 1 --2- 172.21.15.5:0/4243870624 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efd24076720 0x7efd241a46e0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7efd14004770 tx=0x7efd14003c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:48:48.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:48.425+0000 7efd1b400640 1 -- 172.21.15.5:0/4243870624 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd1403f070 con 0x7efd24076720 2024-09-10T09:48:48.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:48.425+0000 7efd1b400640 1 -- 172.21.15.5:0/4243870624 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efd14031ca0 con 0x7efd24076720 2024-09-10T09:48:48.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:48.425+0000 7efd1b400640 1 -- 172.21.15.5:0/4243870624 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd14035030 con 0x7efd24076720 2024-09-10T09:48:48.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:48.425+0000 7efd29c00640 1 -- 172.21.15.5:0/4243870624 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd2419f260 con 0x7efd24076720 2024-09-10T09:48:48.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:48.425+0000 7efd29c00640 1 -- 172.21.15.5:0/4243870624 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd241ab5b0 con 0x7efd24076720 2024-09-10T09:48:48.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:48.428+0000 7efd1b400640 1 -- 172.21.15.5:0/4243870624 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7efd14042070 con 0x7efd24076720 2024-09-10T09:48:48.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:48.428+0000 7efd2c60d740 1 Processor -- start 2024-09-10T09:48:48.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:48.429+0000 7efd2c60d740 1 -- start start 2024-09-10T09:48:48.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:48.429+0000 7efd2c60d740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56515dc16c70 0x56515dc17050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:48:48.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:48.429+0000 7efd2c60d740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56515dc17590 0x56515dc21910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:48:48.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:48.429+0000 7efd2c60d740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56515dc21e50 0x56515dc24240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:48:48.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:48.429+0000 7efd2c60d740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x56515db7d3e0 con 0x56515dc16c70 2024-09-10T09:48:48.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:48.429+0000 7efd2c60d740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x56515db24d80 con 0x56515dc21e50 2024-09-10T09:48:48.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:48.429+0000 7efd2c60d740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x56515db793a0 con 0x56515dc17590 2024-09-10T09:48:48.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:48.429+0000 7efd23400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56515dc16c70 0x56515dc17050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:48:48.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:48.429+0000 7efd22a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56515dc17590 0x56515dc21910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:48:48.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:48.429+0000 7efd23400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56515dc16c70 0x56515dc17050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47854/0 (socket says 172.21.15.5:47854) 2024-09-10T09:48:48.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:48.429+0000 7efd22a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56515dc17590 0x56515dc21910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:56482/0 (socket says 172.21.15.5:56482) 2024-09-10T09:48:48.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:48.429+0000 7efd23400640 1 -- 172.21.15.5:0/3391032282 learned_addr learned my addr 172.21.15.5:0/3391032282 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:48:48.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:48.430+0000 7efd1be00640 1 --2- 172.21.15.5:0/3391032282 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56515dc21e50 0x56515dc24240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:48:48.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:48.430+0000 7efd1be00640 1 -- 172.21.15.5:0/3391032282 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56515dc17590 msgr2=0x56515dc21910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:48:48.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:48.430+0000 7efd1be00640 1 --2- 172.21.15.5:0/3391032282 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56515dc17590 0x56515dc21910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:48.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:48.430+0000 7efd1be00640 1 -- 172.21.15.5:0/3391032282 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56515dc16c70 msgr2=0x56515dc17050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:48:48.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:48.430+0000 7efd1be00640 1 --2- 172.21.15.5:0/3391032282 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56515dc16c70 0x56515dc17050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:48.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:48.430+0000 7efd1be00640 1 -- 172.21.15.5:0/3391032282 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56515db43ae0 con 0x56515dc21e50 2024-09-10T09:48:48.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:48.430+0000 7efd1b400640 1 --2- 172.21.15.5:0/4243870624 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7efd080a2c00 0x7efd080a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:48:48.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:48.430+0000 7efd23400640 1 --2- 172.21.15.5:0/3391032282 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56515dc16c70 0x56515dc17050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:48:48.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:48.430+0000 7efd1b400640 1 -- 172.21.15.5:0/4243870624 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7efd140efa60 con 0x7efd24076720 2024-09-10T09:48:48.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:48.430+0000 7efd1be00640 1 --2- 172.21.15.5:0/3391032282 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56515dc21e50 0x56515dc24240 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7efd241a0090 tx=0x7efd0c0107a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6702c81fb337a983 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:48:48.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:48.431+0000 7efd22a00640 1 --2- 172.21.15.5:0/4243870624 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7efd080a2c00 0x7efd080a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:48:48.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:48.431+0000 7efd13e00640 1 -- 172.21.15.5:0/3391032282 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd0c01c480 con 0x56515dc21e50 2024-09-10T09:48:48.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:48.431+0000 7efd13e00640 1 -- 172.21.15.5:0/3391032282 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efd0c01cb00 con 0x56515dc21e50 2024-09-10T09:48:48.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:48.431+0000 7efd13e00640 1 -- 172.21.15.5:0/3391032282 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd0c011c10 con 0x56515dc21e50 2024-09-10T09:48:48.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:48.431+0000 7efd2c60d740 1 -- 172.21.15.5:0/3391032282 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56515dc21e50 msgr2=0x56515dc24240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:48:48.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:48.431+0000 7efd22a00640 1 --2- 172.21.15.5:0/4243870624 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7efd080a2c00 0x7efd080a50c0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7efd140b8380 tx=0x7efd140f5530 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:48:48.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:48.431+0000 7efd2c60d740 1 --2- 172.21.15.5:0/3391032282 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56515dc21e50 0x56515dc24240 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7efd241a0090 tx=0x7efd0c0107a0 comp rx=0 tx=0).stop 2024-09-10T09:48:48.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:48.432+0000 7efd2c60d740 1 -- 172.21.15.5:0/3391032282 shutdown_connections 2024-09-10T09:48:48.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:48.432+0000 7efd2c60d740 1 --2- 172.21.15.5:0/3391032282 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56515dc21e50 0x56515dc24240 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:48.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:48.432+0000 7efd2c60d740 1 --2- 172.21.15.5:0/3391032282 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56515dc17590 0x56515dc21910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:48.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:48.432+0000 7efd2c60d740 1 --2- 172.21.15.5:0/3391032282 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56515dc16c70 0x56515dc17050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:48.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:48.432+0000 7efd2c60d740 1 -- 172.21.15.5:0/3391032282 >> 172.21.15.5:0/3391032282 conn(0x56515dc13200 msgr2=0x56515dc135e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:48:48.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:48.432+0000 7efd2c60d740 1 -- 172.21.15.5:0/3391032282 shutdown_connections 2024-09-10T09:48:48.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:48.432+0000 7efd2c60d740 1 -- 172.21.15.5:0/3391032282 wait complete. 2024-09-10T09:48:48.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:48.433+0000 7efd2c60d740 1 -- start start 2024-09-10T09:48:48.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:48.433+0000 7efd2c60d740 1 Processor -- start 2024-09-10T09:48:48.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:48.433+0000 7efd2c60d740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56515dc16c70 0x56515dc1ed50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:48:48.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:48.433+0000 7efd2c60d740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56515dc17590 0x56515dc1f290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:48:48.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:48.433+0000 7efd2c60d740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56515dc21e50 0x56515dc1f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:48:48.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:48.433+0000 7efd2c60d740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x56515dc24a20 con 0x56515dc16c70 2024-09-10T09:48:48.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:48.433+0000 7efd2c60d740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x56515db380f0 con 0x56515dc17590 2024-09-10T09:48:48.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:48.433+0000 7efd2c60d740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x56515dc24ba0 con 0x56515dc21e50 2024-09-10T09:48:48.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:48.433+0000 7efd2c60d740 10 client.0 _ll_register_callbacks cb 0x56515db1fbd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:48:48.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:48.434+0000 7efd23400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56515dc16c70 0x56515dc1ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:48:48.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:48.434+0000 7efd23400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56515dc16c70 0x56515dc1ed50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47868/0 (socket says 172.21.15.5:47868) 2024-09-10T09:48:48.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:48.434+0000 7efd23400640 1 -- 172.21.15.5:0/4277674665 learned_addr learned my addr 172.21.15.5:0/4277674665 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:48:48.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:48.434+0000 7efd1be00640 1 --2- 172.21.15.5:0/4277674665 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56515dc17590 0x56515dc1f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:48:48.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:48.434+0000 7efd23400640 1 --2- 172.21.15.5:0/4277674665 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56515dc21e50 0x56515dc1f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:48:48.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:48.434+0000 7efd23400640 1 -- 172.21.15.5:0/4277674665 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56515dc21e50 msgr2=0x56515dc1f7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:48:48.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:48.434+0000 7efd23400640 1 --2- 172.21.15.5:0/4277674665 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56515dc21e50 0x56515dc1f7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:48.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:48.434+0000 7efd23400640 1 -- 172.21.15.5:0/4277674665 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56515dc17590 msgr2=0x56515dc1f290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:48:48.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:48.434+0000 7efd23400640 1 --2- 172.21.15.5:0/4277674665 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56515dc17590 0x56515dc1f290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:48.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:48.434+0000 7efd23400640 1 -- 172.21.15.5:0/4277674665 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56515d8ee3c0 con 0x56515dc16c70 2024-09-10T09:48:48.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:48.434+0000 7efd1be00640 1 --2- 172.21.15.5:0/4277674665 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56515dc17590 0x56515dc1f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:48:48.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:48.435+0000 7efd23400640 1 --2- 172.21.15.5:0/4277674665 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56515dc16c70 0x56515dc1ed50 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7efd1c00f410 tx=0x7efd1c009d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bb3f9729aac5250 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:48:48.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:48.435+0000 7efd10c00640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-10T09:48:48.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:48.435+0000 7efd10c00640 1 -- 172.21.15.5:0/4277674665 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd1c003d80 con 0x56515dc16c70 2024-09-10T09:48:48.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:48.435+0000 7efd2c60d740 1 -- 172.21.15.5:0/4277674665 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efd0c01cb00 con 0x56515dc16c70 2024-09-10T09:48:48.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:48.435+0000 7efd10c00640 1 -- 172.21.15.5:0/4277674665 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7efd1c004b20 con 0x56515dc16c70 2024-09-10T09:48:48.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:48.435+0000 7efd10c00640 1 -- 172.21.15.5:0/4277674665 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7efd1c027050 con 0x56515dc16c70 2024-09-10T09:48:48.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:48.435+0000 7efd2c60d740 10 client.17484 fetch_fsmap learned FSMap version 36 2024-09-10T09:48:48.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:48.435+0000 7efd2c60d740 1 -- 172.21.15.5:0/4277674665 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x56515dc30ad0 con 0x56515dc16c70 2024-09-10T09:48:48.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:48.436+0000 7efd10c00640 1 -- 172.21.15.5:0/4277674665 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efd0c01cb00 con 0x56515dc16c70 2024-09-10T09:48:48.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:48.436+0000 7efd10c00640 1 -- 172.21.15.5:0/4277674665 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 36) ==== 8370+0+0 (secure 0 0 0) 0x7efd1c01e070 con 0x56515dc16c70 2024-09-10T09:48:48.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:48.436+0000 7efd2c60d740 10 client.17484 fetch_fsmap finished waiting for FSMap version 36 2024-09-10T09:48:48.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:48.436+0000 7efd2c60d740 10 client.17484 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T09:48:48.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:48.436+0000 7efd2c60d740 20 client.17484 populate_metadata read hostname 'smithi005' 2024-09-10T09:48:48.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:48.436+0000 7efd2c60d740 1 --2- 172.21.15.5:0/4277674665 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x56515dc31ee0 0x56515dc34340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:48:48.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:48.436+0000 7efd2c60d740 4 client.17484 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:48:48.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:48.436+0000 7efd2c60d740 1 -- 172.21.15.5:0/4277674665 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56515db8a950 con 0x56515dc31ee0 2024-09-10T09:48:48.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:48.437+0000 7efd1be00640 1 --2- 172.21.15.5:0/4277674665 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x56515dc31ee0 0x56515dc34340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:48:48.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:48.437+0000 7efd1be00640 1 --2- 172.21.15.5:0/4277674665 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x56515dc31ee0 0x56515dc34340 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=97f36e85fdeeec24 server_cookie=b74c6c14176842fd in_seq=0 out_seq=0 2024-09-10T09:48:48.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:48.437+0000 7efd10c00640 10 client.17484 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T09:48:48.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:48.438+0000 7efd10c00640 1 -- 172.21.15.5:0/4277674665 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56515db8a950 con 0x56515dc31ee0 2024-09-10T09:48:48.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:48.438+0000 7efd10c00640 10 client.17484 handle_command_reply: tid=0 2024-09-10T09:48:48.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:48.438+0000 7efd2c60d740 1 client.17484 shutdown 2024-09-10T09:48:48.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:48.438+0000 7efd2c60d740 1 -- 172.21.15.5:0/4277674665 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56515dc16c70 msgr2=0x56515dc1ed50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:48:48.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:48.438+0000 7efd2c60d740 1 --2- 172.21.15.5:0/4277674665 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56515dc16c70 0x56515dc1ed50 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7efd1c00f410 tx=0x7efd1c009d70 comp rx=0 tx=0).stop 2024-09-10T09:48:48.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:48.439+0000 7efd2c60d740 1 -- 172.21.15.5:0/4277674665 shutdown_connections 2024-09-10T09:48:48.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:48.439+0000 7efd2c60d740 1 --2- 172.21.15.5:0/4277674665 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x56515dc31ee0 0x56515dc34340 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:48.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:48.439+0000 7efd2c60d740 1 --2- 172.21.15.5:0/4277674665 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56515dc21e50 0x56515dc1f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:48.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:48.439+0000 7efd2c60d740 1 --2- 172.21.15.5:0/4277674665 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56515dc17590 0x56515dc1f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:48.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:48.439+0000 7efd2c60d740 1 --2- 172.21.15.5:0/4277674665 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56515dc16c70 0x56515dc1ed50 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:48.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:48.439+0000 7efd2c60d740 1 -- 172.21.15.5:0/4277674665 >> 172.21.15.5:0/4277674665 conn(0x56515dc13200 msgr2=0x56515dc18f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:48:48.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:48.439+0000 7efd10c00640 0 client.17484 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T09:48:48.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:48.439+0000 7efd2c60d740 1 -- 172.21.15.5:0/4277674665 shutdown_connections 2024-09-10T09:48:48.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:48.439+0000 7efd2c60d740 1 -- 172.21.15.5:0/4277674665 wait complete. 2024-09-10T09:48:48.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:48.439+0000 7efd2c60d740 20 client.17484 trim_cache size 0 max 16384 2024-09-10T09:48:48.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:48.456+0000 7efd2c60d740 1 Processor -- start 2024-09-10T09:48:48.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:48.456+0000 7efd2c60d740 1 -- start start 2024-09-10T09:48:48.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:48.456+0000 7efd2c60d740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56515dc16c70 0x56515dc39c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:48:48.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:48.456+0000 7efd2c60d740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56515dc17590 0x56515dc3a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:48:48.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:48.456+0000 7efd2c60d740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56515dc21e50 0x56515dc1bff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:48:48.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:48.456+0000 7efd2c60d740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7efd241b5460 con 0x56515dc16c70 2024-09-10T09:48:48.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:48.456+0000 7efd2c60d740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7efd241b6fa0 con 0x56515dc17590 2024-09-10T09:48:48.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:48.456+0000 7efd2c60d740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7efd241b5b30 con 0x56515dc21e50 2024-09-10T09:48:48.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:48.456+0000 7efd23400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56515dc16c70 0x56515dc39c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:48:48.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:48.457+0000 7efd1be00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56515dc17590 0x56515dc3a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:48:48.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:48.457+0000 7efd23400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56515dc16c70 0x56515dc39c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47876/0 (socket says 172.21.15.5:47876) 2024-09-10T09:48:48.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:48.457+0000 7efd23400640 1 -- 172.21.15.5:0/3013286078 learned_addr learned my addr 172.21.15.5:0/3013286078 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:48:48.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:48.457+0000 7efd1be00640 1 -- 172.21.15.5:0/3013286078 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56515dc21e50 msgr2=0x56515dc1bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:48:48.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:48.457+0000 7efd23400640 1 --2- 172.21.15.5:0/3013286078 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56515dc21e50 0x56515dc1bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:48:48.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:48.457+0000 7efd1be00640 1 --2- 172.21.15.5:0/3013286078 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56515dc21e50 0x56515dc1bff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:48.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:48.457+0000 7efd1be00640 1 -- 172.21.15.5:0/3013286078 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56515dc16c70 msgr2=0x56515dc39c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:48:48.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:48.457+0000 7efd1be00640 1 --2- 172.21.15.5:0/3013286078 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56515dc16c70 0x56515dc39c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:48.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:48.457+0000 7efd1be00640 1 -- 172.21.15.5:0/3013286078 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56515dc13030 con 0x56515dc17590 2024-09-10T09:48:48.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:48.457+0000 7efd1be00640 1 --2- 172.21.15.5:0/3013286078 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56515dc17590 0x56515dc3a1a0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7efd0c005d80 tx=0x7efd0c011770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c4cdd5b537debfae server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:48:48.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:48.458+0000 7efd13400640 1 -- 172.21.15.5:0/3013286078 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd0c02c030 con 0x56515dc17590 2024-09-10T09:48:48.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:48.458+0000 7efd13400640 1 -- 172.21.15.5:0/3013286078 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efd0c00bd20 con 0x56515dc17590 2024-09-10T09:48:48.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:48.458+0000 7efd13400640 1 -- 172.21.15.5:0/3013286078 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd0c01c480 con 0x56515dc17590 2024-09-10T09:48:48.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:48.458+0000 7efd2c60d740 1 -- 172.21.15.5:0/3013286078 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56515dc17590 msgr2=0x56515dc3a1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:48:48.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:48.458+0000 7efd2c60d740 1 --2- 172.21.15.5:0/3013286078 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56515dc17590 0x56515dc3a1a0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7efd0c005d80 tx=0x7efd0c011770 comp rx=0 tx=0).stop 2024-09-10T09:48:48.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:48.459+0000 7efd2c60d740 1 -- 172.21.15.5:0/3013286078 shutdown_connections 2024-09-10T09:48:48.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:48.459+0000 7efd2c60d740 1 --2- 172.21.15.5:0/3013286078 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56515dc21e50 0x56515dc1bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:48.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:48.459+0000 7efd2c60d740 1 --2- 172.21.15.5:0/3013286078 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56515dc17590 0x56515dc3a1a0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:48.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:48.459+0000 7efd2c60d740 1 --2- 172.21.15.5:0/3013286078 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56515dc16c70 0x56515dc39c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:48.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:48.459+0000 7efd2c60d740 1 -- 172.21.15.5:0/3013286078 >> 172.21.15.5:0/3013286078 conn(0x56515dc13200 msgr2=0x56515dc20dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:48:48.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:48.459+0000 7efd2c60d740 1 -- 172.21.15.5:0/3013286078 shutdown_connections 2024-09-10T09:48:48.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:48.459+0000 7efd2c60d740 1 -- 172.21.15.5:0/3013286078 wait complete. 2024-09-10T09:48:48.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:48.459+0000 7efd2c60d740 1 -- start start 2024-09-10T09:48:48.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:48.459+0000 7efd2c60d740 1 Processor -- start 2024-09-10T09:48:48.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:48.460+0000 7efd2c60d740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56515dc16c70 0x56515dc29df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:48:48.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:48.460+0000 7efd2c60d740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56515dc17590 0x56515dc2a330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:48:48.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:48.460+0000 7efd2c60d740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56515dc21e50 0x56515dc26dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:48:48.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:48.460+0000 7efd2c60d740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7efd241b68d0 con 0x56515dc21e50 2024-09-10T09:48:48.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:48.460+0000 7efd2c60d740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7efd241b6200 con 0x56515dc17590 2024-09-10T09:48:48.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:48.460+0000 7efd2c60d740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7efd241b3440 con 0x56515dc16c70 2024-09-10T09:48:48.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:48.460+0000 7efd2c60d740 10 client.0 _ll_register_callbacks cb 0x7efd1c009cc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:48:48.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:48.460+0000 7efd1be00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56515dc17590 0x56515dc2a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:48:48.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:48.460+0000 7efd23400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56515dc16c70 0x56515dc29df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:48:48.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:48.460+0000 7efd1be00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56515dc17590 0x56515dc2a330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:41560/0 (socket says 172.21.15.5:41560) 2024-09-10T09:48:48.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:48.460+0000 7efd1be00640 1 -- 172.21.15.5:0/1119568022 learned_addr learned my addr 172.21.15.5:0/1119568022 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:48:48.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:48.460+0000 7efd23400640 1 --2- 172.21.15.5:0/1119568022 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56515dc21e50 0x56515dc26dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:48:48.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:48.461+0000 7efd1be00640 1 -- 172.21.15.5:0/1119568022 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56515dc16c70 msgr2=0x56515dc29df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:48:48.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:48.461+0000 7efd1be00640 1 --2- 172.21.15.5:0/1119568022 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56515dc16c70 0x56515dc29df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:48.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:48.461+0000 7efd1be00640 1 -- 172.21.15.5:0/1119568022 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56515dc21e50 msgr2=0x56515dc26dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:48:48.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:48.461+0000 7efd1be00640 1 --2- 172.21.15.5:0/1119568022 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56515dc21e50 0x56515dc26dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:48.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:48.461+0000 7efd1be00640 1 -- 172.21.15.5:0/1119568022 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56515dc13e90 con 0x56515dc17590 2024-09-10T09:48:48.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:48.461+0000 7efd1be00640 1 --2- 172.21.15.5:0/1119568022 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56515dc17590 0x56515dc2a330 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7efd0c000c00 tx=0x7efd0c0114d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4e42ee165f2face1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:48:48.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:48.461+0000 7efd10c00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-09-10T09:48:48.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:48.461+0000 7efd10c00640 1 -- 172.21.15.5:0/1119568022 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd0c036070 con 0x56515dc17590 2024-09-10T09:48:48.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:48.461+0000 7efd2c60d740 1 -- 172.21.15.5:0/1119568022 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efd0c00bd20 con 0x56515dc17590 2024-09-10T09:48:48.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:48.462+0000 7efd10c00640 1 -- 172.21.15.5:0/1119568022 <== mon.2 v2:172.21.15.102:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7efd0c004220 con 0x56515dc17590 2024-09-10T09:48:48.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:48.462+0000 7efd10c00640 1 -- 172.21.15.5:0/1119568022 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7efd0c03f050 con 0x56515dc17590 2024-09-10T09:48:48.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:48.462+0000 7efd2c60d740 10 client.27494 fetch_fsmap learned FSMap version 36 2024-09-10T09:48:48.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:48.462+0000 7efd2c60d740 1 -- 172.21.15.5:0/1119568022 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x56515dc0e770 con 0x56515dc17590 2024-09-10T09:48:48.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:48.462+0000 7efd10c00640 1 -- 172.21.15.5:0/1119568022 <== mon.2 v2:172.21.15.102:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efd0c00bd20 con 0x56515dc17590 2024-09-10T09:48:48.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:48.462+0000 7efd10c00640 1 -- 172.21.15.5:0/1119568022 <== mon.2 v2:172.21.15.102:3300/0 5 ==== fsmap(e 36) ==== 8370+0+0 (secure 0 0 0) 0x7efd0c039020 con 0x56515dc17590 2024-09-10T09:48:48.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:48.462+0000 7efd2c60d740 10 client.27494 fetch_fsmap finished waiting for FSMap version 36 2024-09-10T09:48:48.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:48.462+0000 7efd2c60d740 10 client.27494 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T09:48:48.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:48.462+0000 7efd2c60d740 20 client.27494 populate_metadata read hostname 'smithi005' 2024-09-10T09:48:48.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:48.462+0000 7efd2c60d740 1 --2- 172.21.15.5:0/1119568022 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x56515dc9d3c0 0x56515dc9f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:48:48.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:48.463+0000 7efd2c60d740 4 client.27494 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T09:48:48.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:48.463+0000 7efd2c60d740 1 -- 172.21.15.5:0/1119568022 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56515db84e60 con 0x56515dc9d3c0 2024-09-10T09:48:48.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:48.463+0000 7efd23400640 1 --2- 172.21.15.5:0/1119568022 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x56515dc9d3c0 0x56515dc9f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:48:48.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:48.464+0000 7efd10c00640 10 client.27494 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T09:48:48.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:48.464+0000 7efd23400640 1 --2- 172.21.15.5:0/1119568022 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x56515dc9d3c0 0x56515dc9f7e0 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a42c1e4a084354fb server_cookie=4810183091092009 in_seq=0 out_seq=0 2024-09-10T09:48:48.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:48.464+0000 7efd10c00640 1 -- 172.21.15.5:0/1119568022 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56515db84e60 con 0x56515dc9d3c0 2024-09-10T09:48:48.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:48.464+0000 7efd10c00640 10 client.27494 handle_command_reply: tid=0 2024-09-10T09:48:48.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:48.464+0000 7efd2c60d740 1 client.27494 shutdown 2024-09-10T09:48:48.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:48.464+0000 7efd2c60d740 1 -- 172.21.15.5:0/1119568022 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56515dc17590 msgr2=0x56515dc2a330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:48:48.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:48.464+0000 7efd2c60d740 1 --2- 172.21.15.5:0/1119568022 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56515dc17590 0x56515dc2a330 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7efd0c000c00 tx=0x7efd0c0114d0 comp rx=0 tx=0).stop 2024-09-10T09:48:48.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:48.464+0000 7efd2c60d740 1 -- 172.21.15.5:0/1119568022 shutdown_connections 2024-09-10T09:48:48.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:48.464+0000 7efd2c60d740 1 --2- 172.21.15.5:0/1119568022 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x56515dc9d3c0 0x56515dc9f7e0 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:48.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:48.464+0000 7efd2c60d740 1 --2- 172.21.15.5:0/1119568022 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56515dc21e50 0x56515dc26dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:48.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:48.464+0000 7efd2c60d740 1 --2- 172.21.15.5:0/1119568022 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56515dc17590 0x56515dc2a330 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:48.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:48.464+0000 7efd2c60d740 1 --2- 172.21.15.5:0/1119568022 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56515dc16c70 0x56515dc29df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:48.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:48.464+0000 7efd2c60d740 1 -- 172.21.15.5:0/1119568022 >> 172.21.15.5:0/1119568022 conn(0x56515dc13200 msgr2=0x56515dc186c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:48:48.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:48.464+0000 7efd10c00640 0 client.27494 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T09:48:48.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:48.464+0000 7efd2c60d740 1 -- 172.21.15.5:0/1119568022 shutdown_connections 2024-09-10T09:48:48.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:48.464+0000 7efd2c60d740 1 -- 172.21.15.5:0/1119568022 wait complete. 2024-09-10T09:48:48.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:48.465+0000 7efd2c60d740 20 client.27494 trim_cache size 0 max 16384 2024-09-10T09:48:48.469 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-10T09:48:48.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:48.465+0000 7efd29c00640 1 -- 172.21.15.5:0/4243870624 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7efd080a2c00 msgr2=0x7efd080a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:48:48.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:48.465+0000 7efd29c00640 1 --2- 172.21.15.5:0/4243870624 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7efd080a2c00 0x7efd080a50c0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7efd140b8380 tx=0x7efd140f5530 comp rx=0 tx=0).stop 2024-09-10T09:48:48.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:48.465+0000 7efd29c00640 1 -- 172.21.15.5:0/4243870624 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efd24076720 msgr2=0x7efd241a46e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:48:48.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:48.465+0000 7efd29c00640 1 --2- 172.21.15.5:0/4243870624 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efd24076720 0x7efd241a46e0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7efd14004770 tx=0x7efd14003c10 comp rx=0 tx=0).stop 2024-09-10T09:48:48.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:48.465+0000 7efd29c00640 1 -- 172.21.15.5:0/4243870624 shutdown_connections 2024-09-10T09:48:48.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:48.465+0000 7efd29c00640 1 --2- 172.21.15.5:0/4243870624 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7efd080a2c00 0x7efd080a50c0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:48.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:48.465+0000 7efd29c00640 1 --2- 172.21.15.5:0/4243870624 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efd24112a30 0x7efd2419e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:48.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:48.465+0000 7efd29c00640 1 --2- 172.21.15.5:0/4243870624 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efd24077040 0x7efd241a4c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:48.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:48.465+0000 7efd29c00640 1 --2- 172.21.15.5:0/4243870624 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efd24076720 0x7efd241a46e0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:48.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:48.465+0000 7efd29c00640 1 -- 172.21.15.5:0/4243870624 >> 172.21.15.5:0/4243870624 conn(0x7efd240ffa90 msgr2=0x7efd24101e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:48:48.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:48.466+0000 7efd29c00640 1 -- 172.21.15.5:0/4243870624 shutdown_connections 2024-09-10T09:48:48.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:48.466+0000 7efd29c00640 1 -- 172.21.15.5:0/4243870624 wait complete. 2024-09-10T09:48:49.042 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:48:49.114 INFO:tasks.workunit.client.0.smithi005.stdout: 1 650014 6.37 MB/sec execute 390 sec latency 6094.816 ms 2024-09-10T09:48:49.805 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:48:49 smithi005 ceph-mon[30350]: pgmap v707: 129 pgs: 129 active+clean; 246 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.3 MiB/s wr, 152 op/s 2024-09-10T09:48:49.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:48:49 smithi159 ceph-mon[35513]: pgmap v707: 129 pgs: 129 active+clean; 246 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.3 MiB/s wr, 152 op/s 2024-09-10T09:48:49.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:48:49 smithi102 ceph-mon[35442]: pgmap v707: 129 pgs: 129 active+clean; 246 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.3 MiB/s wr, 152 op/s 2024-09-10T09:48:50.114 INFO:tasks.workunit.client.0.smithi005.stdout: 1 650014 6.36 MB/sec execute 391 sec latency 7094.898 ms 2024-09-10T09:48:50.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:50.202+0000 7f5dab200640 1 -- 172.21.15.5:0/3716022393 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5da4101de0 msgr2=0x7f5da41021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:48:50.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:50.202+0000 7f5dab200640 1 --2- 172.21.15.5:0/3716022393 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5da4101de0 0x7f5da41021c0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f5d94009900 tx=0x7f5d9402a3f0 comp rx=0 tx=0).stop 2024-09-10T09:48:50.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:50.202+0000 7f5dab200640 1 -- 172.21.15.5:0/3716022393 shutdown_connections 2024-09-10T09:48:50.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:50.202+0000 7f5dab200640 1 --2- 172.21.15.5:0/3716022393 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5da4106f10 0x7f5da410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:50.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:50.202+0000 7f5dab200640 1 --2- 172.21.15.5:0/3716022393 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5da4102790 0x7f5da41067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:50.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:50.202+0000 7f5dab200640 1 --2- 172.21.15.5:0/3716022393 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5da4101de0 0x7f5da41021c0 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:50.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:50.202+0000 7f5dab200640 1 -- 172.21.15.5:0/3716022393 >> 172.21.15.5:0/3716022393 conn(0x7f5da40fd8c0 msgr2=0x7f5da40ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:48:50.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:50.202+0000 7f5dab200640 1 -- 172.21.15.5:0/3716022393 shutdown_connections 2024-09-10T09:48:50.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:50.202+0000 7f5dab200640 1 -- 172.21.15.5:0/3716022393 wait complete. 2024-09-10T09:48:50.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:50.204+0000 7f5dab200640 1 Processor -- start 2024-09-10T09:48:50.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:50.204+0000 7f5dab200640 1 -- start start 2024-09-10T09:48:50.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:50.204+0000 7f5dab200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5da4101de0 0x7f5da4198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:48:50.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:50.204+0000 7f5dab200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5da4102790 0x7f5da4198950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:48:50.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:50.204+0000 7f5dab200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5da4106f10 0x7f5da419cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:48:50.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:50.204+0000 7f5dab200640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f5da4111f70 con 0x7f5da4101de0 2024-09-10T09:48:50.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:50.204+0000 7f5dab200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f5da4111df0 con 0x7f5da4106f10 2024-09-10T09:48:50.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:50.204+0000 7f5dab200640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f5da41120f0 con 0x7f5da4102790 2024-09-10T09:48:50.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:50.204+0000 7f5da3e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5da4101de0 0x7f5da4198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:48:50.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:50.204+0000 7f5da3e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5da4101de0 0x7f5da4198410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47898/0 (socket says 172.21.15.5:47898) 2024-09-10T09:48:50.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:50.204+0000 7f5da3e00640 1 -- 172.21.15.5:0/890848642 learned_addr learned my addr 172.21.15.5:0/890848642 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:48:50.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:50.205+0000 7f5da3400640 1 --2- 172.21.15.5:0/890848642 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5da4102790 0x7f5da4198950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:48:50.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:50.205+0000 7f5da9200640 1 --2- 172.21.15.5:0/890848642 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5da4106f10 0x7f5da419cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:48:50.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:50.205+0000 7f5da3e00640 1 -- 172.21.15.5:0/890848642 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5da4102790 msgr2=0x7f5da4198950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:48:50.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:50.205+0000 7f5da3e00640 1 --2- 172.21.15.5:0/890848642 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5da4102790 0x7f5da4198950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:50.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:50.205+0000 7f5da3e00640 1 -- 172.21.15.5:0/890848642 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5da4106f10 msgr2=0x7f5da419cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:48:50.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:50.205+0000 7f5da3e00640 1 --2- 172.21.15.5:0/890848642 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5da4106f10 0x7f5da419cce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:50.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:50.205+0000 7f5da3e00640 1 -- 172.21.15.5:0/890848642 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5da419d3c0 con 0x7f5da4101de0 2024-09-10T09:48:50.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:50.205+0000 7f5da3e00640 1 --2- 172.21.15.5:0/890848642 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5da4101de0 0x7f5da4198410 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f5d94006010 tx=0x7f5d94004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:48:50.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:50.206+0000 7f5da0c00640 1 -- 172.21.15.5:0/890848642 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d9403f030 con 0x7f5da4101de0 2024-09-10T09:48:50.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:50.206+0000 7f5da0c00640 1 -- 172.21.15.5:0/890848642 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5d94004440 con 0x7f5da4101de0 2024-09-10T09:48:50.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:50.206+0000 7f5da0c00640 1 -- 172.21.15.5:0/890848642 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d94035070 con 0x7f5da4101de0 2024-09-10T09:48:50.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:50.206+0000 7f5dab200640 1 -- 172.21.15.5:0/890848642 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5da419d650 con 0x7f5da4101de0 2024-09-10T09:48:50.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:50.206+0000 7f5dab200640 1 -- 172.21.15.5:0/890848642 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5da419dae0 con 0x7f5da4101de0 2024-09-10T09:48:50.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:50.207+0000 7f5dab200640 1 -- 172.21.15.5:0/890848642 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5da41021c0 con 0x7f5da4101de0 2024-09-10T09:48:50.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:50.209+0000 7f5da0c00640 1 -- 172.21.15.5:0/890848642 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f5d94042030 con 0x7f5da4101de0 2024-09-10T09:48:50.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:50.210+0000 7f5da0c00640 1 --2- 172.21.15.5:0/890848642 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f5d880a2c00 0x7f5d880a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:48:50.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:50.210+0000 7f5da0c00640 1 -- 172.21.15.5:0/890848642 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f5d940ef800 con 0x7f5da4101de0 2024-09-10T09:48:50.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:50.211+0000 7f5da3400640 1 --2- 172.21.15.5:0/890848642 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f5d880a2c00 0x7f5d880a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:48:50.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:50.211+0000 7f5da3400640 1 --2- 172.21.15.5:0/890848642 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f5d880a2c00 0x7f5d880a50c0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f5da40fdce0 tx=0x7f5d90005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:48:50.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:50.215+0000 7f5da0c00640 1 -- 172.21.15.5:0/890848642 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f5d940bc310 con 0x7f5da4101de0 2024-09-10T09:48:50.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:50.439+0000 7f5dab200640 1 -- 172.21.15.5:0/890848642 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5da402d7c0 con 0x7f5da4101de0 2024-09-10T09:48:50.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:50.440+0000 7f5da0c00640 1 -- 172.21.15.5:0/890848642 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) ==== 94+0+11969 (secure 0 0 0) 0x7f5d940bbc10 con 0x7f5da4101de0 2024-09-10T09:48:50.442 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-10T09:48:50.443 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":36,"btime":"2024-09-10T09:48:07:349075+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14454,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/2970594311","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2970594311},{"type":"v1","addr":"172.21.15.5:6835","nonce":2970594311}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14514,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6839/3616502174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":3616502174},{"type":"v1","addr":"172.21.15.5:6839","nonce":3616502174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":36,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T09:35:29.626287+0000","modified":"2024-09-10T09:48:06.970127+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24542,"mds_1":24535,"mds_2":24511,"mds_3":24506,"mds_4":24443},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14520":{"gid":14520,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/2073599842","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":2073599842},{"type":"v1","addr":"172.21.15.5:6841","nonce":2073599842}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"k","rank":4,"incarnation":26,"state":"up:active","state_seq":24,"addr":"172.21.15.102:6833/2457007427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":2457007427},{"type":"v1","addr":"172.21.15.102:6833","nonce":2457007427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/356445159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":356445159},{"type":"v1","addr":"172.21.15.159:6833","nonce":356445159}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6837/2894314507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":2894314507},{"type":"v1","addr":"172.21.15.5:6837","nonce":2894314507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.102:6835/1647549111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":1647549111},{"type":"v1","addr":"172.21.15.102:6835","nonce":1647549111}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6835/952657242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":952657242},{"type":"v1","addr":"172.21.15.159:6835","nonce":952657242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":19,"addr":"172.21.15.102:6837/2689776764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":2689776764},{"type":"v1","addr":"172.21.15.102:6837","nonce":2689776764}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.159:6837/620627639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":620627639},{"type":"v1","addr":"172.21.15.159:6837","nonce":620627639}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24535":{"gid":24535,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":18,"addr":"172.21.15.159:6839/2073739599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":2073739599},{"type":"v1","addr":"172.21.15.159:6839","nonce":2073739599}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24542":{"gid":24542,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.102:6839/335409749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6838","nonce":335409749},{"type":"v1","addr":"172.21.15.102:6839","nonce":335409749}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24542,"qdb_cluster":[24542,24535,24511,24506,24443]},"id":1}]} 2024-09-10T09:48:50.443 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 36 2024-09-10T09:48:50.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:50.443+0000 7f5dab200640 1 -- 172.21.15.5:0/890848642 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f5d880a2c00 msgr2=0x7f5d880a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:48:50.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:50.443+0000 7f5dab200640 1 --2- 172.21.15.5:0/890848642 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f5d880a2c00 0x7f5d880a50c0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f5da40fdce0 tx=0x7f5d90005f90 comp rx=0 tx=0).stop 2024-09-10T09:48:50.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:50.443+0000 7f5dab200640 1 -- 172.21.15.5:0/890848642 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5da4101de0 msgr2=0x7f5da4198410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:48:50.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:50.443+0000 7f5dab200640 1 --2- 172.21.15.5:0/890848642 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5da4101de0 0x7f5da4198410 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f5d94006010 tx=0x7f5d94004040 comp rx=0 tx=0).stop 2024-09-10T09:48:50.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:50.443+0000 7f5dab200640 1 -- 172.21.15.5:0/890848642 shutdown_connections 2024-09-10T09:48:50.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:50.443+0000 7f5dab200640 1 --2- 172.21.15.5:0/890848642 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f5d880a2c00 0x7f5d880a50c0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:50.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:50.443+0000 7f5dab200640 1 --2- 172.21.15.5:0/890848642 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5da4106f10 0x7f5da419cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:50.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:50.443+0000 7f5dab200640 1 --2- 172.21.15.5:0/890848642 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5da4102790 0x7f5da4198950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:50.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:50.443+0000 7f5dab200640 1 --2- 172.21.15.5:0/890848642 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5da4101de0 0x7f5da4198410 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:50.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:50.443+0000 7f5dab200640 1 -- 172.21.15.5:0/890848642 >> 172.21.15.5:0/890848642 conn(0x7f5da40fd8c0 msgr2=0x7f5da40febd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:48:50.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:50.443+0000 7f5dab200640 1 -- 172.21.15.5:0/890848642 shutdown_connections 2024-09-10T09:48:50.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:50.443+0000 7f5dab200640 1 -- 172.21.15.5:0/890848642 wait complete. 2024-09-10T09:48:50.550 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-10T09:48:50.550 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-10T09:48:51.120 INFO:tasks.workunit.client.0.smithi005.stdout: 1 650014 6.34 MB/sec execute 392 sec latency 8100.897 ms 2024-09-10T09:48:51.209 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24542 quiesce db --set-id 81e886f6 --release --await' 2024-09-10T09:48:51.209 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24542 quiesce db --set-id 81e886f6 --release --await 2024-09-10T09:48:51.306 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:48:51.675 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:48:51 smithi005 ceph-mon[30350]: pgmap v708: 129 pgs: 129 active+clean; 249 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.4 MiB/s wr, 161 op/s 2024-09-10T09:48:51.675 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:48:51 smithi005 ceph-mon[30350]: from='client.? 172.21.15.5:0/890848642' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:48:51.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:48:51 smithi159 ceph-mon[35513]: pgmap v708: 129 pgs: 129 active+clean; 249 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.4 MiB/s wr, 161 op/s 2024-09-10T09:48:51.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:48:51 smithi159 ceph-mon[35513]: from='client.? 172.21.15.5:0/890848642' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:48:51.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:48:51 smithi102 ceph-mon[35442]: pgmap v708: 129 pgs: 129 active+clean; 249 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.4 MiB/s wr, 161 op/s 2024-09-10T09:48:51.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:48:51 smithi102 ceph-mon[35442]: from='client.? 172.21.15.5:0/890848642' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:48:52.120 INFO:tasks.workunit.client.0.smithi005.stdout: 1 650014 6.33 MB/sec execute 393 sec latency 9100.961 ms 2024-09-10T09:48:52.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:52.502+0000 7f9785e00640 1 -- 172.21.15.5:0/1929399990 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9780074620 msgr2=0x7f9780074a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:48:52.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:52.502+0000 7f9785e00640 1 --2- 172.21.15.5:0/1929399990 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9780074620 0x7f9780074a00 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f9778009ca0 tx=0x7f977802a950 comp rx=0 tx=0).stop 2024-09-10T09:48:52.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:52.502+0000 7f9785e00640 1 -- 172.21.15.5:0/1929399990 shutdown_connections 2024-09-10T09:48:52.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:52.502+0000 7f9785e00640 1 --2- 172.21.15.5:0/1929399990 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f978010e210 0x7f9780110660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:52.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:52.502+0000 7f9785e00640 1 --2- 172.21.15.5:0/1929399990 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9780074f40 0x7f978010dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:52.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:52.502+0000 7f9785e00640 1 --2- 172.21.15.5:0/1929399990 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9780074620 0x7f9780074a00 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:52.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:52.502+0000 7f9785e00640 1 -- 172.21.15.5:0/1929399990 >> 172.21.15.5:0/1929399990 conn(0x7f97800fd710 msgr2=0x7f97800ffb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:48:52.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:52.502+0000 7f9785e00640 1 -- 172.21.15.5:0/1929399990 shutdown_connections 2024-09-10T09:48:52.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:52.502+0000 7f9785e00640 1 -- 172.21.15.5:0/1929399990 wait complete. 2024-09-10T09:48:52.506 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:48:52 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:48:52.506 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:48:52 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:48:52.506 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:48:52 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:48:52.506 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:48:52 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:48:52.506 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:48:52 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:48:52.506 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:48:52 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:48:52.506 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:48:52 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:48:52.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:52.503+0000 7f9785e00640 1 Processor -- start 2024-09-10T09:48:52.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:52.504+0000 7f9785e00640 1 -- start start 2024-09-10T09:48:52.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:52.504+0000 7f9785e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9780074620 0x7f978019a450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:48:52.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:52.504+0000 7f9785e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9780074f40 0x7f978019a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:48:52.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:52.504+0000 7f9785e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f978010e210 0x7f978019ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:48:52.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:52.504+0000 7f9785e00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f9780113fd0 con 0x7f9780074f40 2024-09-10T09:48:52.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:52.504+0000 7f9785e00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f9780113e50 con 0x7f9780074620 2024-09-10T09:48:52.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:52.504+0000 7f9785e00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f9780114150 con 0x7f978010e210 2024-09-10T09:48:52.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:52.505+0000 7f977fe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9780074f40 0x7f978019a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:48:52.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:52.505+0000 7f977fe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9780074f40 0x7f978019a990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42030/0 (socket says 172.21.15.5:42030) 2024-09-10T09:48:52.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:52.505+0000 7f977fe00640 1 -- 172.21.15.5:0/3860760344 learned_addr learned my addr 172.21.15.5:0/3860760344 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:48:52.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:52.505+0000 7f977fe00640 1 -- 172.21.15.5:0/3860760344 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f978010e210 msgr2=0x7f978019ed20 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-10T09:48:52.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:52.505+0000 7f977fe00640 1 --2- 172.21.15.5:0/3860760344 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f978010e210 0x7f978019ed20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:52.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:52.506+0000 7f977fe00640 1 -- 172.21.15.5:0/3860760344 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9780074620 msgr2=0x7f978019a450 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-10T09:48:52.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:52.506+0000 7f977fe00640 1 --2- 172.21.15.5:0/3860760344 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9780074620 0x7f978019a450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:52.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:52.506+0000 7f977fe00640 1 -- 172.21.15.5:0/3860760344 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f978019f400 con 0x7f9780074f40 2024-09-10T09:48:52.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:52.506+0000 7f977fe00640 1 --2- 172.21.15.5:0/3860760344 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9780074f40 0x7f978019a990 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f976c00c920 tx=0x7f976c00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:48:52.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:52.507+0000 7f977d600640 1 -- 172.21.15.5:0/3860760344 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f976c007c70 con 0x7f9780074f40 2024-09-10T09:48:52.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:52.507+0000 7f9785e00640 1 -- 172.21.15.5:0/3860760344 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f978019f6f0 con 0x7f9780074f40 2024-09-10T09:48:52.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:52.507+0000 7f977d600640 1 -- 172.21.15.5:0/3860760344 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f976c007e10 con 0x7f9780074f40 2024-09-10T09:48:52.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:52.507+0000 7f977d600640 1 -- 172.21.15.5:0/3860760344 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f976c018890 con 0x7f9780074f40 2024-09-10T09:48:52.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:52.508+0000 7f9785e00640 1 -- 172.21.15.5:0/3860760344 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97801a6fd0 con 0x7f9780074f40 2024-09-10T09:48:52.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:52.512+0000 7f977d600640 1 -- 172.21.15.5:0/3860760344 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f976c018b30 con 0x7f9780074f40 2024-09-10T09:48:52.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:52.512+0000 7f978a078740 1 Processor -- start 2024-09-10T09:48:52.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:52.513+0000 7f978a078740 1 -- start start 2024-09-10T09:48:52.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:52.513+0000 7f978a078740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a7737decb0 0x55a7737df090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:48:52.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:52.513+0000 7f978a078740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a7737df5d0 0x55a7737e9950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:48:52.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:52.513+0000 7f978a078740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a7737e9e90 0x55a7737ec280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:48:52.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:52.513+0000 7f978a078740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55a773745290 con 0x55a7737e9e90 2024-09-10T09:48:52.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:52.513+0000 7f978a078740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55a7736ecb40 con 0x55a7737decb0 2024-09-10T09:48:52.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:52.513+0000 7f978a078740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55a773741250 con 0x55a7737df5d0 2024-09-10T09:48:52.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:52.514+0000 7f9785400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a7737decb0 0x55a7737df090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:48:52.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:52.514+0000 7f9785400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a7737decb0 0x55a7737df090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:54052/0 (socket says 172.21.15.5:54052) 2024-09-10T09:48:52.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:52.514+0000 7f9785400640 1 -- 172.21.15.5:0/2595364741 learned_addr learned my addr 172.21.15.5:0/2595364741 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:48:52.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:52.514+0000 7f9784a00640 1 --2- 172.21.15.5:0/2595364741 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a7737df5d0 0x55a7737e9950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:48:52.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:52.514+0000 7f977d600640 1 --2- 172.21.15.5:0/3860760344 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f97640a2c00 0x7f97640a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:48:52.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:52.514+0000 7f977d600640 1 -- 172.21.15.5:0/3860760344 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f976c0cf5d0 con 0x7f9780074f40 2024-09-10T09:48:52.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:52.515+0000 7f9785400640 1 --2- 172.21.15.5:0/3860760344 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f97640a2c00 0x7f97640a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:48:52.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:52.515+0000 7f977fe00640 1 --2- 172.21.15.5:0/2595364741 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a7737e9e90 0x55a7737ec280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:48:52.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:52.515+0000 7f9785400640 1 -- 172.21.15.5:0/2595364741 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a7737df5d0 msgr2=0x55a7737e9950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:48:52.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:52.515+0000 7f9785400640 1 --2- 172.21.15.5:0/2595364741 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a7737df5d0 0x55a7737e9950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:52.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:52.515+0000 7f9785400640 1 -- 172.21.15.5:0/2595364741 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a7737e9e90 msgr2=0x55a7737ec280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:48:52.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:52.515+0000 7f9785400640 1 --2- 172.21.15.5:0/2595364741 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a7737e9e90 0x55a7737ec280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:52.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:52.515+0000 7f9785400640 1 -- 172.21.15.5:0/2595364741 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a7734b66a0 con 0x55a7737decb0 2024-09-10T09:48:52.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:52.515+0000 7f9784a00640 1 --2- 172.21.15.5:0/2595364741 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a7737df5d0 0x55a7737e9950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:48:52.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:52.516+0000 7f9785400640 1 --2- 172.21.15.5:0/2595364741 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a7737decb0 0x55a7737df090 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f9770013430 tx=0x7f97700138f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ef93bd43da92498a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:48:52.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:52.516+0000 7f9785400640 1 --2- 172.21.15.5:0/3860760344 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f97640a2c00 0x7f97640a50c0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f9770018000 tx=0x7f977001a000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:48:52.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:52.516+0000 7f9775c00640 1 -- 172.21.15.5:0/2595364741 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f977001b810 con 0x55a7737decb0 2024-09-10T09:48:52.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:52.516+0000 7f9775c00640 1 -- 172.21.15.5:0/2595364741 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f977001a390 con 0x55a7737decb0 2024-09-10T09:48:52.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:52.516+0000 7f9775c00640 1 -- 172.21.15.5:0/2595364741 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f977000d5a0 con 0x55a7737decb0 2024-09-10T09:48:52.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:52.517+0000 7f978a078740 1 -- 172.21.15.5:0/2595364741 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a7737decb0 msgr2=0x55a7737df090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:48:52.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:52.517+0000 7f978a078740 1 --2- 172.21.15.5:0/2595364741 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a7737decb0 0x55a7737df090 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f9770013430 tx=0x7f97700138f0 comp rx=0 tx=0).stop 2024-09-10T09:48:52.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:52.517+0000 7f978a078740 1 -- 172.21.15.5:0/2595364741 shutdown_connections 2024-09-10T09:48:52.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:52.517+0000 7f978a078740 1 --2- 172.21.15.5:0/2595364741 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a7737e9e90 0x55a7737ec280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:52.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:52.517+0000 7f978a078740 1 --2- 172.21.15.5:0/2595364741 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a7737df5d0 0x55a7737e9950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:52.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:52.517+0000 7f978a078740 1 --2- 172.21.15.5:0/2595364741 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a7737decb0 0x55a7737df090 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:52.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:52.517+0000 7f978a078740 1 -- 172.21.15.5:0/2595364741 >> 172.21.15.5:0/2595364741 conn(0x55a7737db240 msgr2=0x55a7737db620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:48:52.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:52.518+0000 7f978a078740 1 -- 172.21.15.5:0/2595364741 shutdown_connections 2024-09-10T09:48:52.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:52.518+0000 7f978a078740 1 -- 172.21.15.5:0/2595364741 wait complete. 2024-09-10T09:48:52.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:52.518+0000 7f978a078740 1 -- start start 2024-09-10T09:48:52.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:52.519+0000 7f978a078740 1 Processor -- start 2024-09-10T09:48:52.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:52.519+0000 7f978a078740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a7737decb0 0x55a7737e97e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:48:52.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:52.519+0000 7f978a078740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a7737df5d0 0x55a7737e6a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:48:52.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:52.519+0000 7f978a078740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a7737e9e90 0x55a7737e6ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:48:52.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:52.519+0000 7f978a078740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55a7737ecb00 con 0x55a7737decb0 2024-09-10T09:48:52.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:52.519+0000 7f978a078740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55a7736fffa0 con 0x55a7737e9e90 2024-09-10T09:48:52.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:52.519+0000 7f978a078740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55a7737ecc80 con 0x55a7737df5d0 2024-09-10T09:48:52.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:52.519+0000 7f978a078740 10 client.0 _ll_register_callbacks cb 0x55a773453ba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:48:52.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:52.520+0000 7f9784a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a7737decb0 0x55a7737e97e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:48:52.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:52.520+0000 7f9784a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a7737decb0 0x55a7737e97e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42036/0 (socket says 172.21.15.5:42036) 2024-09-10T09:48:52.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:52.520+0000 7f9784a00640 1 -- 172.21.15.5:0/627261488 learned_addr learned my addr 172.21.15.5:0/627261488 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:48:52.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:52.520+0000 7f977fe00640 1 --2- 172.21.15.5:0/627261488 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a7737df5d0 0x55a7737e6a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:48:52.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:52.520+0000 7f9785400640 1 --2- 172.21.15.5:0/627261488 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a7737e9e90 0x55a7737e6ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:48:52.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:52.521+0000 7f9785400640 1 -- 172.21.15.5:0/627261488 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a7737df5d0 msgr2=0x55a7737e6a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:48:52.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:52.521+0000 7f9785400640 1 --2- 172.21.15.5:0/627261488 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a7737df5d0 0x55a7737e6a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:52.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:52.521+0000 7f9785400640 1 -- 172.21.15.5:0/627261488 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a7737decb0 msgr2=0x55a7737e97e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:48:52.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:52.521+0000 7f9785400640 1 --2- 172.21.15.5:0/627261488 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a7737decb0 0x55a7737e97e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:52.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:52.521+0000 7f9785400640 1 -- 172.21.15.5:0/627261488 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a7737e7820 con 0x55a7737e9e90 2024-09-10T09:48:52.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:52.521+0000 7f9784a00640 1 --2- 172.21.15.5:0/627261488 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a7737decb0 0x55a7737e97e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:48:52.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:52.521+0000 7f977fe00640 1 --2- 172.21.15.5:0/627261488 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a7737df5d0 0x55a7737e6a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:48:52.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:52.521+0000 7f9785400640 1 --2- 172.21.15.5:0/627261488 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a7737e9e90 0x55a7737e6ff0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f977000a530 tx=0x7f9770005aa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4ca0d89d75588b3a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:48:52.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:52.521+0000 7f976a000640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-09-10T09:48:52.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:52.522+0000 7f976a000640 1 -- 172.21.15.5:0/627261488 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f977001ba20 con 0x55a7737e9e90 2024-09-10T09:48:52.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:52.522+0000 7f976a000640 1 -- 172.21.15.5:0/627261488 <== mon.2 v2:172.21.15.102:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f9770024430 con 0x55a7737e9e90 2024-09-10T09:48:52.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:52.522+0000 7f978a078740 1 -- 172.21.15.5:0/627261488 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f977001a390 con 0x55a7737e9e90 2024-09-10T09:48:52.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:52.522+0000 7f976a000640 1 -- 172.21.15.5:0/627261488 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f977001a960 con 0x55a7737e9e90 2024-09-10T09:48:52.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:52.523+0000 7f978a078740 10 client.27509 fetch_fsmap learned FSMap version 36 2024-09-10T09:48:52.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:52.523+0000 7f978a078740 1 -- 172.21.15.5:0/627261488 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55a7737f8a40 con 0x55a7737e9e90 2024-09-10T09:48:52.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:52.523+0000 7f976a000640 1 -- 172.21.15.5:0/627261488 <== mon.2 v2:172.21.15.102:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9770031070 con 0x55a7737e9e90 2024-09-10T09:48:52.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:52.523+0000 7f976a000640 1 -- 172.21.15.5:0/627261488 <== mon.2 v2:172.21.15.102:3300/0 5 ==== fsmap(e 36) ==== 8370+0+0 (secure 0 0 0) 0x7f9770023070 con 0x55a7737e9e90 2024-09-10T09:48:52.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:52.523+0000 7f978a078740 10 client.27509 fetch_fsmap finished waiting for FSMap version 36 2024-09-10T09:48:52.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:52.523+0000 7f978a078740 10 client.27509 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T09:48:52.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:52.523+0000 7f978a078740 20 client.27509 populate_metadata read hostname 'smithi005' 2024-09-10T09:48:52.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:52.523+0000 7f978a078740 1 --2- 172.21.15.5:0/627261488 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55a7737f9ea0 0x55a7737fc290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:48:52.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:52.523+0000 7f978a078740 4 client.27509 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:48:52.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:52.523+0000 7f978a078740 1 -- 172.21.15.5:0/627261488 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f977001a390 con 0x55a7737f9ea0 2024-09-10T09:48:52.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:52.524+0000 7f9784a00640 1 --2- 172.21.15.5:0/627261488 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55a7737f9ea0 0x55a7737fc290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:48:52.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:52.525+0000 7f9784a00640 1 --2- 172.21.15.5:0/627261488 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55a7737f9ea0 0x55a7737fc290 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5c811049c8a5f8e0 server_cookie=d8736b40214270a9 in_seq=0 out_seq=0 2024-09-10T09:48:52.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:52.525+0000 7f976a000640 10 client.27509 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T09:48:52.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:52.526+0000 7f976a000640 1 -- 172.21.15.5:0/627261488 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f977001a390 con 0x55a7737f9ea0 2024-09-10T09:48:52.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:52.526+0000 7f976a000640 10 client.27509 handle_command_reply: tid=0 2024-09-10T09:48:52.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:52.526+0000 7f978a078740 1 client.27509 shutdown 2024-09-10T09:48:52.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:52.526+0000 7f978a078740 1 -- 172.21.15.5:0/627261488 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a7737e9e90 msgr2=0x55a7737e6ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:48:52.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:52.526+0000 7f978a078740 1 --2- 172.21.15.5:0/627261488 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a7737e9e90 0x55a7737e6ff0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f977000a530 tx=0x7f9770005aa0 comp rx=0 tx=0).stop 2024-09-10T09:48:52.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:52.526+0000 7f978a078740 1 -- 172.21.15.5:0/627261488 shutdown_connections 2024-09-10T09:48:52.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:52.526+0000 7f978a078740 1 --2- 172.21.15.5:0/627261488 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55a7737f9ea0 0x55a7737fc290 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:52.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:52.526+0000 7f978a078740 1 --2- 172.21.15.5:0/627261488 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a7737e9e90 0x55a7737e6ff0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:52.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:52.526+0000 7f978a078740 1 --2- 172.21.15.5:0/627261488 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a7737df5d0 0x55a7737e6a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:52.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:52.526+0000 7f978a078740 1 --2- 172.21.15.5:0/627261488 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a7737decb0 0x55a7737e97e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:52.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:52.526+0000 7f978a078740 1 -- 172.21.15.5:0/627261488 >> 172.21.15.5:0/627261488 conn(0x55a7737db240 msgr2=0x55a7737e0f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:48:52.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:52.526+0000 7f976a000640 0 client.27509 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T09:48:52.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:52.527+0000 7f978a078740 1 -- 172.21.15.5:0/627261488 shutdown_connections 2024-09-10T09:48:52.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:52.527+0000 7f978a078740 1 -- 172.21.15.5:0/627261488 wait complete. 2024-09-10T09:48:52.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:52.527+0000 7f978a078740 20 client.27509 trim_cache size 0 max 16384 2024-09-10T09:48:52.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:52.550+0000 7f978a078740 1 Processor -- start 2024-09-10T09:48:52.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:52.550+0000 7f978a078740 1 -- start start 2024-09-10T09:48:52.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:52.551+0000 7f978a078740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a7737decb0 0x55a7737da160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:48:52.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:52.551+0000 7f978a078740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a7737df5d0 0x55a773801a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:48:52.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:52.551+0000 7f978a078740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a7737e9e90 0x55a773801f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:48:52.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:52.551+0000 7f978a078740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f97801b0f30 con 0x55a7737df5d0 2024-09-10T09:48:52.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:52.551+0000 7f978a078740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f97801b2a70 con 0x55a7737decb0 2024-09-10T09:48:52.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:52.551+0000 7f978a078740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f97801b1600 con 0x55a7737e9e90 2024-09-10T09:48:52.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:52.551+0000 7f977fe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a7737df5d0 0x55a773801a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:48:52.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:52.551+0000 7f977fe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a7737df5d0 0x55a773801a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42050/0 (socket says 172.21.15.5:42050) 2024-09-10T09:48:52.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:52.551+0000 7f977fe00640 1 -- 172.21.15.5:0/348539740 learned_addr learned my addr 172.21.15.5:0/348539740 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:48:52.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:52.551+0000 7f9784a00640 1 --2- 172.21.15.5:0/348539740 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a7737decb0 0x55a7737da160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:48:52.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:52.552+0000 7f9784a00640 1 -- 172.21.15.5:0/348539740 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a7737e9e90 msgr2=0x55a773801f60 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T09:48:52.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:52.552+0000 7f9784a00640 1 --2- 172.21.15.5:0/348539740 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a7737e9e90 0x55a773801f60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:52.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:52.552+0000 7f9784a00640 1 -- 172.21.15.5:0/348539740 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a7737df5d0 msgr2=0x55a773801a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:48:52.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:52.552+0000 7f9784a00640 1 --2- 172.21.15.5:0/348539740 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a7737df5d0 0x55a773801a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:52.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:52.552+0000 7f9784a00640 1 -- 172.21.15.5:0/348539740 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a7737e7360 con 0x55a7737decb0 2024-09-10T09:48:52.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:52.552+0000 7f977fe00640 1 --2- 172.21.15.5:0/348539740 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a7737df5d0 0x55a773801a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:48:52.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:52.552+0000 7f9784a00640 1 --2- 172.21.15.5:0/348539740 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a7737decb0 0x55a7737da160 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x55a7737fc960 tx=0x7f977802a950 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2c69147a1bdb0a1e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:48:52.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:52.552+0000 7f9775200640 1 -- 172.21.15.5:0/348539740 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f977804a070 con 0x55a7737decb0 2024-09-10T09:48:52.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:52.552+0000 7f9775200640 1 -- 172.21.15.5:0/348539740 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f977802f730 con 0x55a7737decb0 2024-09-10T09:48:52.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:52.552+0000 7f9775200640 1 -- 172.21.15.5:0/348539740 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9778034040 con 0x55a7737decb0 2024-09-10T09:48:52.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:52.553+0000 7f978a078740 1 -- 172.21.15.5:0/348539740 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a7737decb0 msgr2=0x55a7737da160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:48:52.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:52.553+0000 7f978a078740 1 --2- 172.21.15.5:0/348539740 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a7737decb0 0x55a7737da160 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x55a7737fc960 tx=0x7f977802a950 comp rx=0 tx=0).stop 2024-09-10T09:48:52.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:52.553+0000 7f978a078740 1 -- 172.21.15.5:0/348539740 shutdown_connections 2024-09-10T09:48:52.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:52.553+0000 7f978a078740 1 --2- 172.21.15.5:0/348539740 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a7737e9e90 0x55a773801f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:52.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:52.553+0000 7f978a078740 1 --2- 172.21.15.5:0/348539740 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a7737df5d0 0x55a773801a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:52.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:52.553+0000 7f978a078740 1 --2- 172.21.15.5:0/348539740 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a7737decb0 0x55a7737da160 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:52.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:52.553+0000 7f978a078740 1 -- 172.21.15.5:0/348539740 >> 172.21.15.5:0/348539740 conn(0x55a7737db240 msgr2=0x55a7737d9a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:48:52.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:52.553+0000 7f978a078740 1 -- 172.21.15.5:0/348539740 shutdown_connections 2024-09-10T09:48:52.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:52.553+0000 7f978a078740 1 -- 172.21.15.5:0/348539740 wait complete. 2024-09-10T09:48:52.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:52.554+0000 7f978a078740 1 -- start start 2024-09-10T09:48:52.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:52.554+0000 7f978a078740 1 Processor -- start 2024-09-10T09:48:52.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:52.555+0000 7f978a078740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a7737decb0 0x55a7737efe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:48:52.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:52.555+0000 7f978a078740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a7737df5d0 0x55a7737f03a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:48:52.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:52.555+0000 7f978a078740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a7737e9e90 0x55a7737f08e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:48:52.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:52.555+0000 7f978a078740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f97801b23a0 con 0x55a7737df5d0 2024-09-10T09:48:52.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:52.555+0000 7f978a078740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f97801b1cd0 con 0x55a7737e9e90 2024-09-10T09:48:52.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:52.555+0000 7f978a078740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f97801aef10 con 0x55a7737decb0 2024-09-10T09:48:52.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:52.555+0000 7f978a078740 10 client.0 _ll_register_callbacks cb 0x55a7737e79f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:48:52.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:52.555+0000 7f977fe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a7737df5d0 0x55a7737f03a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:48:52.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:52.555+0000 7f977fe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a7737df5d0 0x55a7737f03a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42062/0 (socket says 172.21.15.5:42062) 2024-09-10T09:48:52.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:52.555+0000 7f977fe00640 1 -- 172.21.15.5:0/4186335581 learned_addr learned my addr 172.21.15.5:0/4186335581 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:48:52.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:52.555+0000 7f9785400640 1 --2- 172.21.15.5:0/4186335581 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a7737e9e90 0x55a7737f08e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:48:52.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:52.555+0000 7f9784a00640 1 --2- 172.21.15.5:0/4186335581 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a7737decb0 0x55a7737efe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:48:52.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:52.556+0000 7f9785400640 1 -- 172.21.15.5:0/4186335581 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a7737decb0 msgr2=0x55a7737efe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:48:52.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:52.556+0000 7f9785400640 1 --2- 172.21.15.5:0/4186335581 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a7737decb0 0x55a7737efe60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:52.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:52.556+0000 7f9785400640 1 -- 172.21.15.5:0/4186335581 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a7737df5d0 msgr2=0x55a7737f03a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:48:52.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:52.556+0000 7f9785400640 1 --2- 172.21.15.5:0/4186335581 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a7737df5d0 0x55a7737f03a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:52.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:52.556+0000 7f9785400640 1 -- 172.21.15.5:0/4186335581 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a77386c410 con 0x55a7737e9e90 2024-09-10T09:48:52.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:52.556+0000 7f977fe00640 1 --2- 172.21.15.5:0/4186335581 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a7737df5d0 0x55a7737f03a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:48:52.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:52.556+0000 7f9785400640 1 --2- 172.21.15.5:0/4186335581 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a7737e9e90 0x55a7737f08e0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f977002c730 tx=0x7f9770013f30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aca5597d8499d50f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:48:52.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:52.556+0000 7f976a000640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-09-10T09:48:52.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:52.556+0000 7f976a000640 1 -- 172.21.15.5:0/4186335581 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f977001a390 con 0x55a7737e9e90 2024-09-10T09:48:52.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:52.557+0000 7f976a000640 1 -- 172.21.15.5:0/4186335581 <== mon.2 v2:172.21.15.102:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f977000ade0 con 0x55a7737e9e90 2024-09-10T09:48:52.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:52.557+0000 7f978a078740 1 -- 172.21.15.5:0/4186335581 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f977802f730 con 0x55a7737e9e90 2024-09-10T09:48:52.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:52.557+0000 7f976a000640 1 -- 172.21.15.5:0/4186335581 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f9770044050 con 0x55a7737e9e90 2024-09-10T09:48:52.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:52.558+0000 7f978a078740 10 client.27515 fetch_fsmap learned FSMap version 36 2024-09-10T09:48:52.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:52.558+0000 7f978a078740 1 -- 172.21.15.5:0/4186335581 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55a77386d730 con 0x55a7737e9e90 2024-09-10T09:48:52.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:52.558+0000 7f976a000640 1 -- 172.21.15.5:0/4186335581 <== mon.2 v2:172.21.15.102:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f977802f730 con 0x55a7737e9e90 2024-09-10T09:48:52.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:52.559+0000 7f976a000640 1 -- 172.21.15.5:0/4186335581 <== mon.2 v2:172.21.15.102:3300/0 5 ==== fsmap(e 36) ==== 8370+0+0 (secure 0 0 0) 0x7f9770047020 con 0x55a7737e9e90 2024-09-10T09:48:52.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:52.559+0000 7f978a078740 10 client.27515 fetch_fsmap finished waiting for FSMap version 36 2024-09-10T09:48:52.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:52.559+0000 7f978a078740 10 client.27515 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T09:48:52.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:52.559+0000 7f978a078740 20 client.27515 populate_metadata read hostname 'smithi005' 2024-09-10T09:48:52.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:52.559+0000 7f978a078740 1 --2- 172.21.15.5:0/4186335581 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55a77386dd80 0x55a773870170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:48:52.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:52.559+0000 7f978a078740 4 client.27515 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-10T09:48:52.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:52.559+0000 7f978a078740 1 -- 172.21.15.5:0/4186335581 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55a773752800 con 0x55a77386dd80 2024-09-10T09:48:52.568 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:48:52.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:52.571+0000 7f9784a00640 1 --2- 172.21.15.5:0/4186335581 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55a77386dd80 0x55a773870170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:48:52.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:52.619+0000 7f9784a00640 1 --2- 172.21.15.5:0/4186335581 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55a77386dd80 0x55a773870170 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e45089cc35cef54c server_cookie=282449ab2c3078e7 in_seq=0 out_seq=0 2024-09-10T09:48:52.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:52.619+0000 7f976a000640 10 client.27515 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T09:48:52.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:52.631+0000 7f976a000640 1 -- 172.21.15.5:0/4186335581 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55a773752800 con 0x55a77386dd80 2024-09-10T09:48:52.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:52.631+0000 7f976a000640 10 client.27515 handle_command_reply: tid=0 2024-09-10T09:48:52.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:52.631+0000 7f978a078740 1 client.27515 shutdown 2024-09-10T09:48:52.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:52.631+0000 7f978a078740 1 -- 172.21.15.5:0/4186335581 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a7737e9e90 msgr2=0x55a7737f08e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:48:52.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:52.631+0000 7f978a078740 1 --2- 172.21.15.5:0/4186335581 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a7737e9e90 0x55a7737f08e0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f977002c730 tx=0x7f9770013f30 comp rx=0 tx=0).stop 2024-09-10T09:48:52.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:52.631+0000 7f978a078740 1 -- 172.21.15.5:0/4186335581 shutdown_connections 2024-09-10T09:48:52.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:52.631+0000 7f978a078740 1 --2- 172.21.15.5:0/4186335581 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55a77386dd80 0x55a773870170 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:52.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:52.631+0000 7f978a078740 1 --2- 172.21.15.5:0/4186335581 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a7737e9e90 0x55a7737f08e0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:52.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:52.631+0000 7f978a078740 1 --2- 172.21.15.5:0/4186335581 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a7737df5d0 0x55a7737f03a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:52.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:52.631+0000 7f976a000640 0 client.27515 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T09:48:52.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:52.631+0000 7f978a078740 1 --2- 172.21.15.5:0/4186335581 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a7737decb0 0x55a7737efe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:52.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:52.631+0000 7f978a078740 1 -- 172.21.15.5:0/4186335581 >> 172.21.15.5:0/4186335581 conn(0x55a7737db240 msgr2=0x55a7737e6b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:48:52.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:52.632+0000 7f978a078740 1 -- 172.21.15.5:0/4186335581 shutdown_connections 2024-09-10T09:48:52.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:52.632+0000 7f978a078740 1 -- 172.21.15.5:0/4186335581 wait complete. 2024-09-10T09:48:52.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:52.632+0000 7f978a078740 20 client.27515 trim_cache size 0 max 16384 2024-09-10T09:48:52.635 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-10T09:48:52.635 INFO:teuthology.orchestra.run.smithi005.stdout: "return_code": 0, 2024-09-10T09:48:52.635 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_tag": "821a9e03-24c6-4da3-9bb8-fd95bee630de", 2024-09-10T09:48:52.635 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": "asynchronous" 2024-09-10T09:48:52.636 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-10T09:48:52.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:52.633+0000 7f9785e00640 1 -- 172.21.15.5:0/3860760344 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f97640a2c00 msgr2=0x7f97640a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:48:52.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:52.633+0000 7f9785e00640 1 --2- 172.21.15.5:0/3860760344 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f97640a2c00 0x7f97640a50c0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f9770018000 tx=0x7f977001a000 comp rx=0 tx=0).stop 2024-09-10T09:48:52.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:52.633+0000 7f9785e00640 1 -- 172.21.15.5:0/3860760344 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9780074f40 msgr2=0x7f978019a990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:48:52.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:52.633+0000 7f9785e00640 1 --2- 172.21.15.5:0/3860760344 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9780074f40 0x7f978019a990 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f976c00c920 tx=0x7f976c00cde0 comp rx=0 tx=0).stop 2024-09-10T09:48:52.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:52.633+0000 7f9785e00640 1 -- 172.21.15.5:0/3860760344 shutdown_connections 2024-09-10T09:48:52.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:52.633+0000 7f9785e00640 1 --2- 172.21.15.5:0/3860760344 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f97640a2c00 0x7f97640a50c0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:52.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:52.633+0000 7f9785e00640 1 --2- 172.21.15.5:0/3860760344 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f978010e210 0x7f978019ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:52.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:52.633+0000 7f9785e00640 1 --2- 172.21.15.5:0/3860760344 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9780074f40 0x7f978019a990 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:52.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:52.633+0000 7f9785e00640 1 --2- 172.21.15.5:0/3860760344 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9780074620 0x7f978019a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:52.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:52.633+0000 7f9785e00640 1 -- 172.21.15.5:0/3860760344 >> 172.21.15.5:0/3860760344 conn(0x7f97800fd710 msgr2=0x7f97800fe370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:48:52.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:52.634+0000 7f9785e00640 1 -- 172.21.15.5:0/3860760344 shutdown_connections 2024-09-10T09:48:52.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:52.634+0000 7f9785e00640 1 -- 172.21.15.5:0/3860760344 wait complete. 2024-09-10T09:48:52.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:48:52 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:48:52.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:48:52 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:48:52.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:48:52 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:48:52.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:48:52 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:48:52.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:48:52 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:48:52.851 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:48:52 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:48:52.851 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:48:52 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:48:52.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:48:52 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:48:52.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:48:52 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:48:52.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:48:52 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:48:52.885 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:48:52 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:48:52.885 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:48:52 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:48:52.885 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:48:52 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:48:52.885 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:48:52 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:48:53.120 INFO:tasks.workunit.client.0.smithi005.stdout: 1 650014 6.31 MB/sec execute 394 sec latency 10101.020 ms 2024-09-10T09:48:53.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:53.419+0000 7fa4e3e00640 1 -- 172.21.15.5:0/3068884622 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa4dc109120 msgr2=0x7fa4dc110ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:48:53.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:53.419+0000 7fa4e3e00640 1 --2- 172.21.15.5:0/3068884622 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa4dc109120 0x7fa4dc110ca0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fa4d4009ca0 tx=0x7fa4d402a8c0 comp rx=0 tx=0).stop 2024-09-10T09:48:53.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:53.419+0000 7fa4e3e00640 1 -- 172.21.15.5:0/3068884622 shutdown_connections 2024-09-10T09:48:53.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:53.419+0000 7fa4e3e00640 1 --2- 172.21.15.5:0/3068884622 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa4dc109120 0x7fa4dc110ca0 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:53.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:53.419+0000 7fa4e3e00640 1 --2- 172.21.15.5:0/3068884622 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa4dc1049a0 0x7fa4dc1089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:53.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:53.419+0000 7fa4e3e00640 1 --2- 172.21.15.5:0/3068884622 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa4dc103ff0 0x7fa4dc1043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:53.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:53.419+0000 7fa4e3e00640 1 -- 172.21.15.5:0/3068884622 >> 172.21.15.5:0/3068884622 conn(0x7fa4dc0ffa90 msgr2=0x7fa4dc101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:48:53.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:53.419+0000 7fa4e3e00640 1 -- 172.21.15.5:0/3068884622 shutdown_connections 2024-09-10T09:48:53.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:53.419+0000 7fa4e3e00640 1 -- 172.21.15.5:0/3068884622 wait complete. 2024-09-10T09:48:53.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:53.420+0000 7fa4e3e00640 1 Processor -- start 2024-09-10T09:48:53.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:53.420+0000 7fa4e3e00640 1 -- start start 2024-09-10T09:48:53.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:53.420+0000 7fa4e3e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa4dc103ff0 0x7fa4dc19a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:48:53.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:53.420+0000 7fa4e3e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa4dc1049a0 0x7fa4dc19ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:48:53.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:53.420+0000 7fa4e3e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa4dc109120 0x7fa4dc19eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:48:53.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:53.420+0000 7fa4e3e00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa4dc1141a0 con 0x7fa4dc103ff0 2024-09-10T09:48:53.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:53.420+0000 7fa4e3e00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fa4dc114020 con 0x7fa4dc1049a0 2024-09-10T09:48:53.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:53.420+0000 7fa4e3e00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fa4dc114320 con 0x7fa4dc109120 2024-09-10T09:48:53.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:53.421+0000 7fa4e1400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa4dc103ff0 0x7fa4dc19a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:48:53.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:53.421+0000 7fa4e1400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa4dc103ff0 0x7fa4dc19a5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42080/0 (socket says 172.21.15.5:42080) 2024-09-10T09:48:53.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:53.421+0000 7fa4e1400640 1 -- 172.21.15.5:0/2784116989 learned_addr learned my addr 172.21.15.5:0/2784116989 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:48:53.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:53.421+0000 7fa4e0a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa4dc1049a0 0x7fa4dc19ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:48:53.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:53.421+0000 7fa4e0a00640 1 -- 172.21.15.5:0/2784116989 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa4dc109120 msgr2=0x7fa4dc19eeb0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-10T09:48:53.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:53.421+0000 7fa4e0a00640 1 --2- 172.21.15.5:0/2784116989 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa4dc109120 0x7fa4dc19eeb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:53.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:53.421+0000 7fa4e0a00640 1 -- 172.21.15.5:0/2784116989 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa4dc103ff0 msgr2=0x7fa4dc19a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:48:53.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:53.421+0000 7fa4e0a00640 1 --2- 172.21.15.5:0/2784116989 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa4dc103ff0 0x7fa4dc19a5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:53.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:53.421+0000 7fa4e0a00640 1 -- 172.21.15.5:0/2784116989 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4dc19f590 con 0x7fa4dc1049a0 2024-09-10T09:48:53.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:53.421+0000 7fa4e1400640 1 --2- 172.21.15.5:0/2784116989 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa4dc103ff0 0x7fa4dc19a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:48:53.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:53.422+0000 7fa4e0a00640 1 --2- 172.21.15.5:0/2784116989 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa4dc1049a0 0x7fa4dc19ab20 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fa4c800e9f0 tx=0x7fa4c800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:48:53.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:53.422+0000 7fa4da000640 1 -- 172.21.15.5:0/2784116989 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4c800ccc0 con 0x7fa4dc1049a0 2024-09-10T09:48:53.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:53.422+0000 7fa4e3e00640 1 -- 172.21.15.5:0/2784116989 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4dc19f880 con 0x7fa4dc1049a0 2024-09-10T09:48:53.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:53.422+0000 7fa4da000640 1 -- 172.21.15.5:0/2784116989 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa4c8010910 con 0x7fa4dc1049a0 2024-09-10T09:48:53.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:53.422+0000 7fa4da000640 1 -- 172.21.15.5:0/2784116989 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4c8018960 con 0x7fa4dc1049a0 2024-09-10T09:48:53.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:53.422+0000 7fa4e3e00640 1 -- 172.21.15.5:0/2784116989 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4dc1a7160 con 0x7fa4dc1049a0 2024-09-10T09:48:53.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:53.426+0000 7fa4e6a5c740 1 Processor -- start 2024-09-10T09:48:53.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:53.426+0000 7fa4e6a5c740 1 -- start start 2024-09-10T09:48:53.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:53.426+0000 7fa4e6a5c740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b413545310 0x55b4135456f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:48:53.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:53.426+0000 7fa4e6a5c740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b413545c30 0x55b41354ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:48:53.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:53.426+0000 7fa4e6a5c740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b4135504f0 0x55b4135528e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:48:53.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:53.426+0000 7fa4e6a5c740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55b4134ab650 con 0x55b413545c30 2024-09-10T09:48:53.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:53.427+0000 7fa4e6a5c740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55b413452f10 con 0x55b413545310 2024-09-10T09:48:53.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:53.427+0000 7fa4e6a5c740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55b4134a7610 con 0x55b4135504f0 2024-09-10T09:48:53.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:53.427+0000 7fa4da000640 1 -- 172.21.15.5:0/2784116989 <== mon.2 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7fa4c8010ac0 con 0x7fa4dc1049a0 2024-09-10T09:48:53.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:53.427+0000 7fa4e1e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b413545310 0x55b4135456f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:48:53.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:53.427+0000 7fa4e1e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b413545310 0x55b4135456f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:54118/0 (socket says 172.21.15.5:54118) 2024-09-10T09:48:53.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:53.427+0000 7fa4e1e00640 1 -- 172.21.15.5:0/3811500842 learned_addr learned my addr 172.21.15.5:0/3811500842 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:48:53.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:53.427+0000 7fa4e1400640 1 --2- 172.21.15.5:0/3811500842 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b413545c30 0x55b41354ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:48:53.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:53.427+0000 7fa4e0a00640 1 --2- 172.21.15.5:0/3811500842 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b4135504f0 0x55b4135528e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:48:53.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:53.428+0000 7fa4e1e00640 1 -- 172.21.15.5:0/3811500842 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b4135504f0 msgr2=0x55b4135528e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:48:53.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:53.428+0000 7fa4e1e00640 1 --2- 172.21.15.5:0/3811500842 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b4135504f0 0x55b4135528e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:53.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:53.428+0000 7fa4e1e00640 1 -- 172.21.15.5:0/3811500842 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b413545c30 msgr2=0x55b41354ffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:48:53.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:53.428+0000 7fa4e1e00640 1 --2- 172.21.15.5:0/3811500842 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b413545c30 0x55b41354ffb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:53.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:53.428+0000 7fa4e1e00640 1 -- 172.21.15.5:0/3811500842 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b4131ba580 con 0x55b413545310 2024-09-10T09:48:53.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:53.428+0000 7fa4e1e00640 1 --2- 172.21.15.5:0/3811500842 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b413545310 0x55b4135456f0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fa4d4005e30 tx=0x7fa4d4005ef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=39e854bf20873c8e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:48:53.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:53.429+0000 7fa4ce600640 1 -- 172.21.15.5:0/3811500842 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4d4031bb0 con 0x55b413545310 2024-09-10T09:48:53.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:53.429+0000 7fa4ce600640 1 -- 172.21.15.5:0/3811500842 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa4d403f070 con 0x55b413545310 2024-09-10T09:48:53.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:53.429+0000 7fa4ce600640 1 -- 172.21.15.5:0/3811500842 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4d40042a0 con 0x55b413545310 2024-09-10T09:48:53.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:53.429+0000 7fa4da000640 1 --2- 172.21.15.5:0/2784116989 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fa4c40a2b80 0x7fa4c40a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:48:53.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:53.429+0000 7fa4da000640 1 -- 172.21.15.5:0/2784116989 <== mon.2 v2:172.21.15.102:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fa4c80cf860 con 0x7fa4dc1049a0 2024-09-10T09:48:53.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:53.429+0000 7fa4e1400640 1 --2- 172.21.15.5:0/2784116989 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fa4c40a2b80 0x7fa4c40a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:48:53.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:53.429+0000 7fa4e6a5c740 1 -- 172.21.15.5:0/3811500842 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b413545310 msgr2=0x55b4135456f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:48:53.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:53.429+0000 7fa4e6a5c740 1 --2- 172.21.15.5:0/3811500842 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b413545310 0x55b4135456f0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fa4d4005e30 tx=0x7fa4d4005ef0 comp rx=0 tx=0).stop 2024-09-10T09:48:53.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:53.430+0000 7fa4e6a5c740 1 -- 172.21.15.5:0/3811500842 shutdown_connections 2024-09-10T09:48:53.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:53.430+0000 7fa4e6a5c740 1 --2- 172.21.15.5:0/3811500842 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b4135504f0 0x55b4135528e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:53.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:53.430+0000 7fa4e6a5c740 1 --2- 172.21.15.5:0/3811500842 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b413545c30 0x55b41354ffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:53.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:53.430+0000 7fa4e6a5c740 1 --2- 172.21.15.5:0/3811500842 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b413545310 0x55b4135456f0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:53.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:53.430+0000 7fa4e6a5c740 1 -- 172.21.15.5:0/3811500842 >> 172.21.15.5:0/3811500842 conn(0x55b4135418a0 msgr2=0x55b413541c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:48:53.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:53.430+0000 7fa4e1400640 1 --2- 172.21.15.5:0/2784116989 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fa4c40a2b80 0x7fa4c40a5040 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fa4d0009fd0 tx=0x7fa4d0005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:48:53.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:53.430+0000 7fa4e6a5c740 1 -- 172.21.15.5:0/3811500842 shutdown_connections 2024-09-10T09:48:53.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:53.431+0000 7fa4e6a5c740 1 -- 172.21.15.5:0/3811500842 wait complete. 2024-09-10T09:48:53.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:53.431+0000 7fa4e6a5c740 1 -- start start 2024-09-10T09:48:53.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:53.431+0000 7fa4e6a5c740 1 Processor -- start 2024-09-10T09:48:53.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:53.432+0000 7fa4e6a5c740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b413545310 0x55b41354d430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:48:53.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:53.432+0000 7fa4e6a5c740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b413545c30 0x55b41354d970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:48:53.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:53.432+0000 7fa4e6a5c740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b4135504f0 0x55b41354deb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:48:53.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:53.432+0000 7fa4e6a5c740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55b4135530c0 con 0x55b413545c30 2024-09-10T09:48:53.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:53.432+0000 7fa4e6a5c740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55b413466350 con 0x55b4135504f0 2024-09-10T09:48:53.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:53.432+0000 7fa4e6a5c740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55b413553240 con 0x55b413545310 2024-09-10T09:48:53.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:53.432+0000 7fa4e6a5c740 10 client.0 _ll_register_callbacks cb 0x55b413489290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:48:53.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:53.432+0000 7fa4e1400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b413545310 0x55b41354d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:48:53.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:53.433+0000 7fa4e1400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b413545310 0x55b41354d430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:35204/0 (socket says 172.21.15.5:35204) 2024-09-10T09:48:53.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:53.433+0000 7fa4e1400640 1 -- 172.21.15.5:0/1452483377 learned_addr learned my addr 172.21.15.5:0/1452483377 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:48:53.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:53.433+0000 7fa4e1e00640 1 --2- 172.21.15.5:0/1452483377 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b4135504f0 0x55b41354deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:48:53.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:53.433+0000 7fa4e1400640 1 -- 172.21.15.5:0/1452483377 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b4135504f0 msgr2=0x55b41354deb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:48:53.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:53.433+0000 7fa4e1400640 1 --2- 172.21.15.5:0/1452483377 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b4135504f0 0x55b41354deb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:53.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:53.433+0000 7fa4e1400640 1 -- 172.21.15.5:0/1452483377 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b413545c30 msgr2=0x55b41354d970 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T09:48:53.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:53.434+0000 7fa4e1400640 1 --2- 172.21.15.5:0/1452483377 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b413545c30 0x55b41354d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:53.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:53.434+0000 7fa4e1400640 1 -- 172.21.15.5:0/1452483377 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b413284960 con 0x55b413545310 2024-09-10T09:48:53.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:53.434+0000 7fa4e1400640 1 --2- 172.21.15.5:0/1452483377 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b413545310 0x55b41354d430 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fa4d0012f90 tx=0x7fa4d0004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fa817f93adbb6fe0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:48:53.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:53.434+0000 7fa4c2a00640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-09-10T09:48:53.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:53.434+0000 7fa4c2a00640 1 -- 172.21.15.5:0/1452483377 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4d000fe60 con 0x55b413545310 2024-09-10T09:48:53.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:53.434+0000 7fa4c2a00640 1 -- 172.21.15.5:0/1452483377 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fa4d0011b50 con 0x55b413545310 2024-09-10T09:48:53.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:53.434+0000 7fa4e6a5c740 1 -- 172.21.15.5:0/1452483377 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa4d403f070 con 0x55b413545310 2024-09-10T09:48:53.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:53.435+0000 7fa4c2a00640 1 -- 172.21.15.5:0/1452483377 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fa4d0011d50 con 0x55b413545310 2024-09-10T09:48:53.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:53.435+0000 7fa4e6a5c740 10 client.27538 fetch_fsmap learned FSMap version 36 2024-09-10T09:48:53.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:53.435+0000 7fa4e6a5c740 1 -- 172.21.15.5:0/1452483377 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55b41355ed80 con 0x55b413545310 2024-09-10T09:48:53.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:53.435+0000 7fa4c2a00640 1 -- 172.21.15.5:0/1452483377 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa4d403f070 con 0x55b413545310 2024-09-10T09:48:53.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:53.435+0000 7fa4c2a00640 1 -- 172.21.15.5:0/1452483377 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 36) ==== 8370+0+0 (secure 0 0 0) 0x7fa4d0027070 con 0x55b413545310 2024-09-10T09:48:53.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:53.436+0000 7fa4e6a5c740 10 client.27538 fetch_fsmap finished waiting for FSMap version 36 2024-09-10T09:48:53.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:53.436+0000 7fa4e6a5c740 10 client.27538 resolve_mds: validated gid 24542 aka daemon mds.b 2024-09-10T09:48:53.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:53.436+0000 7fa4e6a5c740 20 client.27538 populate_metadata read hostname 'smithi005' 2024-09-10T09:48:53.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:53.436+0000 7fa4e6a5c740 1 --2- 172.21.15.5:0/1452483377 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55b413560290 0x55b413562750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:48:53.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:53.436+0000 7fa4e6a5c740 4 client.27538 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:48:53.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:53.436+0000 7fa4e6a5c740 1 -- 172.21.15.5:0/1452483377 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b4134b8bc0 con 0x55b413560290 2024-09-10T09:48:53.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:53.436+0000 7fa4e0a00640 1 --2- 172.21.15.5:0/1452483377 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55b413560290 0x55b413562750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:48:53.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:53.436+0000 7fa4e0a00640 1 --2- 172.21.15.5:0/1452483377 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55b413560290 0x55b413562750 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7df0e220ab5a09f8 server_cookie=b08ad41e12af6cf3 in_seq=0 out_seq=0 2024-09-10T09:48:53.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:53.436+0000 7fa4c2a00640 10 client.27538 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T09:48:53.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:53.437+0000 7fa4c2a00640 1 -- 172.21.15.5:0/1452483377 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b4134b8bc0 con 0x55b413560290 2024-09-10T09:48:53.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:53.437+0000 7fa4c2a00640 10 client.27538 handle_command_reply: tid=0 2024-09-10T09:48:53.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:53.437+0000 7fa4e6a5c740 1 client.27538 shutdown 2024-09-10T09:48:53.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:53.438+0000 7fa4e6a5c740 1 -- 172.21.15.5:0/1452483377 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b413545310 msgr2=0x55b41354d430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:48:53.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:53.438+0000 7fa4e6a5c740 1 --2- 172.21.15.5:0/1452483377 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b413545310 0x55b41354d430 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fa4d0012f90 tx=0x7fa4d0004730 comp rx=0 tx=0).stop 2024-09-10T09:48:53.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:53.438+0000 7fa4e6a5c740 1 -- 172.21.15.5:0/1452483377 shutdown_connections 2024-09-10T09:48:53.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:53.438+0000 7fa4e6a5c740 1 --2- 172.21.15.5:0/1452483377 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55b413560290 0x55b413562750 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:53.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:53.438+0000 7fa4e6a5c740 1 --2- 172.21.15.5:0/1452483377 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b4135504f0 0x55b41354deb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:53.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:53.438+0000 7fa4c2a00640 0 client.27538 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T09:48:53.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:53.438+0000 7fa4e6a5c740 1 --2- 172.21.15.5:0/1452483377 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b413545c30 0x55b41354d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:53.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:53.438+0000 7fa4e6a5c740 1 --2- 172.21.15.5:0/1452483377 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b413545310 0x55b41354d430 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:53.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:53.438+0000 7fa4e6a5c740 1 -- 172.21.15.5:0/1452483377 >> 172.21.15.5:0/1452483377 conn(0x55b4135418a0 msgr2=0x55b413540aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:48:53.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:53.438+0000 7fa4e6a5c740 1 -- 172.21.15.5:0/1452483377 shutdown_connections 2024-09-10T09:48:53.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:53.438+0000 7fa4e6a5c740 1 -- 172.21.15.5:0/1452483377 wait complete. 2024-09-10T09:48:53.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:53.438+0000 7fa4e6a5c740 20 client.27538 trim_cache size 0 max 16384 2024-09-10T09:48:53.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:53.459+0000 7fa4e6a5c740 1 Processor -- start 2024-09-10T09:48:53.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:53.460+0000 7fa4e6a5c740 1 -- start start 2024-09-10T09:48:53.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:53.460+0000 7fa4e6a5c740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b413545310 0x55b413547dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:48:53.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:53.460+0000 7fa4e6a5c740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b413545c30 0x55b41355ba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:48:53.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:53.460+0000 7fa4e6a5c740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b4135504f0 0x55b41355bf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:48:53.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:53.460+0000 7fa4e6a5c740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa4dc1b1040 con 0x55b413545c30 2024-09-10T09:48:53.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:53.460+0000 7fa4e6a5c740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fa4dc1b2b80 con 0x55b413545310 2024-09-10T09:48:53.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:53.460+0000 7fa4e6a5c740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fa4dc1b1710 con 0x55b4135504f0 2024-09-10T09:48:53.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:53.460+0000 7fa4e0a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b413545c30 0x55b41355ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:48:53.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:53.460+0000 7fa4e1400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b413545310 0x55b413547dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:48:53.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:53.460+0000 7fa4e0a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b413545c30 0x55b41355ba20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42106/0 (socket says 172.21.15.5:42106) 2024-09-10T09:48:53.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:53.460+0000 7fa4e0a00640 1 -- 172.21.15.5:0/3328179329 learned_addr learned my addr 172.21.15.5:0/3328179329 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:48:53.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:53.460+0000 7fa4e1400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b413545310 0x55b413547dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:54132/0 (socket says 172.21.15.5:54132) 2024-09-10T09:48:53.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:53.461+0000 7fa4e1e00640 1 --2- 172.21.15.5:0/3328179329 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b4135504f0 0x55b41355bf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:48:53.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:53.461+0000 7fa4e0a00640 1 -- 172.21.15.5:0/3328179329 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b4135504f0 msgr2=0x55b41355bf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:48:53.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:53.461+0000 7fa4e0a00640 1 --2- 172.21.15.5:0/3328179329 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b4135504f0 0x55b41355bf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:53.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:53.461+0000 7fa4e0a00640 1 -- 172.21.15.5:0/3328179329 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b413545310 msgr2=0x55b413547dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:48:53.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:53.461+0000 7fa4e0a00640 1 --2- 172.21.15.5:0/3328179329 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b413545310 0x55b413547dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:53.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:53.461+0000 7fa4e0a00640 1 -- 172.21.15.5:0/3328179329 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b413542950 con 0x55b413545c30 2024-09-10T09:48:53.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:53.461+0000 7fa4e0a00640 1 --2- 172.21.15.5:0/3328179329 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b413545c30 0x55b41355ba20 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fa4c809bc00 tx=0x7fa4c80d1040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b934516d4ecd5d51 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:48:53.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:53.461+0000 7fa4cdc00640 1 -- 172.21.15.5:0/3328179329 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4c809bc70 con 0x55b413545c30 2024-09-10T09:48:53.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:53.461+0000 7fa4cdc00640 1 -- 172.21.15.5:0/3328179329 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa4c80a0020 con 0x55b413545c30 2024-09-10T09:48:53.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:53.462+0000 7fa4cdc00640 1 -- 172.21.15.5:0/3328179329 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4c809b320 con 0x55b413545c30 2024-09-10T09:48:53.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:53.462+0000 7fa4e6a5c740 1 -- 172.21.15.5:0/3328179329 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b413545c30 msgr2=0x55b41355ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:48:53.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:53.462+0000 7fa4e6a5c740 1 --2- 172.21.15.5:0/3328179329 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b413545c30 0x55b41355ba20 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fa4c809bc00 tx=0x7fa4c80d1040 comp rx=0 tx=0).stop 2024-09-10T09:48:53.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:53.462+0000 7fa4e6a5c740 1 -- 172.21.15.5:0/3328179329 shutdown_connections 2024-09-10T09:48:53.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:53.462+0000 7fa4e6a5c740 1 --2- 172.21.15.5:0/3328179329 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b4135504f0 0x55b41355bf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:53.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:53.462+0000 7fa4e6a5c740 1 --2- 172.21.15.5:0/3328179329 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b413545c30 0x55b41355ba20 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:53.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:53.462+0000 7fa4e6a5c740 1 --2- 172.21.15.5:0/3328179329 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b413545310 0x55b413547dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:53.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:53.462+0000 7fa4e6a5c740 1 -- 172.21.15.5:0/3328179329 >> 172.21.15.5:0/3328179329 conn(0x55b4135418a0 msgr2=0x55b41355b310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:48:53.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:53.463+0000 7fa4e6a5c740 1 -- 172.21.15.5:0/3328179329 shutdown_connections 2024-09-10T09:48:53.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:53.463+0000 7fa4e6a5c740 1 -- 172.21.15.5:0/3328179329 wait complete. 2024-09-10T09:48:53.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:53.463+0000 7fa4e6a5c740 1 -- start start 2024-09-10T09:48:53.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:53.463+0000 7fa4e6a5c740 1 Processor -- start 2024-09-10T09:48:53.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:53.464+0000 7fa4e6a5c740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b413545310 0x55b413554270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:48:53.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:53.464+0000 7fa4e6a5c740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b413545c30 0x55b4135547b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:48:53.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:53.464+0000 7fa4e6a5c740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b4135504f0 0x55b413554cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:48:53.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:53.464+0000 7fa4e6a5c740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa4dc1b24b0 con 0x55b4135504f0 2024-09-10T09:48:53.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:53.464+0000 7fa4e6a5c740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fa4dc1b1de0 con 0x55b413545c30 2024-09-10T09:48:53.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:53.464+0000 7fa4e6a5c740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fa4dc1af020 con 0x55b413545310 2024-09-10T09:48:53.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:53.464+0000 7fa4e6a5c740 10 client.0 _ll_register_callbacks cb 0x7fa4d0012e50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:48:53.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:53.464+0000 7fa4e1e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b4135504f0 0x55b413554cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:48:53.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:53.464+0000 7fa4e1e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b4135504f0 0x55b413554cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42122/0 (socket says 172.21.15.5:42122) 2024-09-10T09:48:53.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:53.464+0000 7fa4e1e00640 1 -- 172.21.15.5:0/2029540691 learned_addr learned my addr 172.21.15.5:0/2029540691 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:48:53.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:53.464+0000 7fa4e0a00640 1 --2- 172.21.15.5:0/2029540691 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b413545c30 0x55b4135547b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:48:53.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:53.465+0000 7fa4e1e00640 1 -- 172.21.15.5:0/2029540691 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b413545310 msgr2=0x55b413554270 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T09:48:53.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:53.465+0000 7fa4e1e00640 1 --2- 172.21.15.5:0/2029540691 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b413545310 0x55b413554270 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:53.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:53.465+0000 7fa4e1e00640 1 -- 172.21.15.5:0/2029540691 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b413545c30 msgr2=0x55b4135547b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:48:53.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:53.465+0000 7fa4e1e00640 1 --2- 172.21.15.5:0/2029540691 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b413545c30 0x55b4135547b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:53.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:53.465+0000 7fa4e1e00640 1 -- 172.21.15.5:0/2029540691 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b4135ca070 con 0x55b4135504f0 2024-09-10T09:48:53.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:53.465+0000 7fa4e1e00640 1 --2- 172.21.15.5:0/2029540691 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b4135504f0 0x55b413554cf0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fa4d4006010 tx=0x7fa4d40317f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4d201911b8563c3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:48:53.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:53.465+0000 7fa4c2a00640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-10T09:48:53.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:53.465+0000 7fa4c2a00640 1 -- 172.21.15.5:0/2029540691 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4d4003970 con 0x55b4135504f0 2024-09-10T09:48:53.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:53.465+0000 7fa4c2a00640 1 -- 172.21.15.5:0/2029540691 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fa4d4003b10 con 0x55b4135504f0 2024-09-10T09:48:53.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:53.465+0000 7fa4e6a5c740 1 -- 172.21.15.5:0/2029540691 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa4c80a0020 con 0x55b4135504f0 2024-09-10T09:48:53.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:53.466+0000 7fa4c2a00640 1 -- 172.21.15.5:0/2029540691 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fa4d4003d10 con 0x55b4135504f0 2024-09-10T09:48:53.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:53.466+0000 7fa4e6a5c740 10 client.17526 fetch_fsmap learned FSMap version 36 2024-09-10T09:48:53.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:53.466+0000 7fa4e6a5c740 1 -- 172.21.15.5:0/2029540691 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55b4135cb390 con 0x55b4135504f0 2024-09-10T09:48:53.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:53.466+0000 7fa4c2a00640 1 -- 172.21.15.5:0/2029540691 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa4c80a0020 con 0x55b4135504f0 2024-09-10T09:48:53.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:53.466+0000 7fa4c2a00640 1 -- 172.21.15.5:0/2029540691 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 36) ==== 8370+0+0 (secure 0 0 0) 0x7fa4d402f730 con 0x55b4135504f0 2024-09-10T09:48:53.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:53.466+0000 7fa4e6a5c740 10 client.17526 fetch_fsmap finished waiting for FSMap version 36 2024-09-10T09:48:53.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:53.466+0000 7fa4e6a5c740 10 client.17526 resolve_mds: validated gid 24542 aka daemon mds.b 2024-09-10T09:48:53.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:53.466+0000 7fa4e6a5c740 20 client.17526 populate_metadata read hostname 'smithi005' 2024-09-10T09:48:53.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:53.466+0000 7fa4e6a5c740 1 --2- 172.21.15.5:0/2029540691 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55b4135cb620 0x55b4135cda10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:48:53.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:53.466+0000 7fa4e6a5c740 4 client.17526 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "81e886f6", "release": true, "await": true}] 2024-09-10T09:48:53.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:53.466+0000 7fa4e6a5c740 1 -- 172.21.15.5:0/2029540691 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "quiesce db", "set_id": "81e886f6", "release": true, "await": true}) -- 0x55b4134b30d0 con 0x55b4135cb620 2024-09-10T09:48:53.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:53.467+0000 7fa4e1400640 1 --2- 172.21.15.5:0/2029540691 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55b4135cb620 0x55b4135cda10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:48:53.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:53.467+0000 7fa4e1400640 1 --2- 172.21.15.5:0/2029540691 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55b4135cb620 0x55b4135cda10 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7957c0c27b03046f server_cookie=d5e81cc71bd5926d in_seq=0 out_seq=0 2024-09-10T09:48:53.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:53.467+0000 7fa4c2a00640 10 client.17526 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T09:48:53.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:53.477+0000 7fa4c2a00640 1 -- 172.21.15.5:0/2029540691 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55b4134b30d0 con 0x55b4135cb620 2024-09-10T09:48:53.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:53.477+0000 7fa4c2a00640 10 client.17526 handle_command_reply: tid=0 2024-09-10T09:48:53.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:53.477+0000 7fa4e6a5c740 1 client.17526 shutdown 2024-09-10T09:48:53.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:53.477+0000 7fa4e6a5c740 1 -- 172.21.15.5:0/2029540691 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b4135504f0 msgr2=0x55b413554cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:48:53.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:53.477+0000 7fa4e6a5c740 1 --2- 172.21.15.5:0/2029540691 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b4135504f0 0x55b413554cf0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fa4d4006010 tx=0x7fa4d40317f0 comp rx=0 tx=0).stop 2024-09-10T09:48:53.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:53.477+0000 7fa4e6a5c740 1 -- 172.21.15.5:0/2029540691 shutdown_connections 2024-09-10T09:48:53.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:53.477+0000 7fa4e6a5c740 1 --2- 172.21.15.5:0/2029540691 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55b4135cb620 0x55b4135cda10 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:53.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:53.477+0000 7fa4e6a5c740 1 --2- 172.21.15.5:0/2029540691 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b4135504f0 0x55b413554cf0 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:53.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:53.477+0000 7fa4e6a5c740 1 --2- 172.21.15.5:0/2029540691 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b413545c30 0x55b4135547b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:53.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:53.477+0000 7fa4c2a00640 0 client.17526 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T09:48:53.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:53.477+0000 7fa4e6a5c740 1 --2- 172.21.15.5:0/2029540691 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b413545310 0x55b413554270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:53.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:53.477+0000 7fa4e6a5c740 1 -- 172.21.15.5:0/2029540691 >> 172.21.15.5:0/2029540691 conn(0x55b4135418a0 msgr2=0x55b41355bb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:48:53.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:53.478+0000 7fa4e6a5c740 1 -- 172.21.15.5:0/2029540691 shutdown_connections 2024-09-10T09:48:53.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:53.478+0000 7fa4e6a5c740 1 -- 172.21.15.5:0/2029540691 wait complete. 2024-09-10T09:48:53.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:53.478+0000 7fa4e6a5c740 20 client.17526 trim_cache size 0 max 16384 2024-09-10T09:48:53.482 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-10T09:48:53.482 INFO:teuthology.orchestra.run.smithi005.stdout: "epoch": 36, 2024-09-10T09:48:53.482 INFO:teuthology.orchestra.run.smithi005.stdout: "leader": 24542, 2024-09-10T09:48:53.482 INFO:teuthology.orchestra.run.smithi005.stdout: "set_version": 36, 2024-09-10T09:48:53.482 INFO:teuthology.orchestra.run.smithi005.stdout: "sets": { 2024-09-10T09:48:53.482 INFO:teuthology.orchestra.run.smithi005.stdout: "81e886f6": { 2024-09-10T09:48:53.482 INFO:teuthology.orchestra.run.smithi005.stdout: "version": 36, 2024-09-10T09:48:53.483 INFO:teuthology.orchestra.run.smithi005.stdout: "age_ref": 0.0, 2024-09-10T09:48:53.483 INFO:teuthology.orchestra.run.smithi005.stdout: "state": { 2024-09-10T09:48:53.483 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "RELEASED", 2024-09-10T09:48:53.483 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0 2024-09-10T09:48:53.483 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T09:48:53.483 INFO:teuthology.orchestra.run.smithi005.stdout: "timeout": 90.0, 2024-09-10T09:48:53.483 INFO:teuthology.orchestra.run.smithi005.stdout: "expiration": 130.0, 2024-09-10T09:48:53.483 INFO:teuthology.orchestra.run.smithi005.stdout: "members": { 2024-09-10T09:48:53.483 INFO:teuthology.orchestra.run.smithi005.stdout: "file:/": { 2024-09-10T09:48:53.483 INFO:teuthology.orchestra.run.smithi005.stdout: "excluded": false, 2024-09-10T09:48:53.483 INFO:teuthology.orchestra.run.smithi005.stdout: "state": { 2024-09-10T09:48:53.483 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "RELEASED", 2024-09-10T09:48:53.483 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0 2024-09-10T09:48:53.484 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T09:48:53.484 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T09:48:53.484 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T09:48:53.484 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T09:48:53.484 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T09:48:53.484 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-10T09:48:53.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:53.478+0000 7fa4e3e00640 1 -- 172.21.15.5:0/2784116989 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fa4c40a2b80 msgr2=0x7fa4c40a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:48:53.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:53.478+0000 7fa4e3e00640 1 --2- 172.21.15.5:0/2784116989 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fa4c40a2b80 0x7fa4c40a5040 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fa4d0009fd0 tx=0x7fa4d0005f90 comp rx=0 tx=0).stop 2024-09-10T09:48:53.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:53.479+0000 7fa4e3e00640 1 -- 172.21.15.5:0/2784116989 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa4dc1049a0 msgr2=0x7fa4dc19ab20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:48:53.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:53.479+0000 7fa4e3e00640 1 --2- 172.21.15.5:0/2784116989 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa4dc1049a0 0x7fa4dc19ab20 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fa4c800e9f0 tx=0x7fa4c800eeb0 comp rx=0 tx=0).stop 2024-09-10T09:48:53.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:53.479+0000 7fa4e3e00640 1 -- 172.21.15.5:0/2784116989 shutdown_connections 2024-09-10T09:48:53.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:53.479+0000 7fa4e3e00640 1 --2- 172.21.15.5:0/2784116989 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fa4c40a2b80 0x7fa4c40a5040 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:53.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:53.479+0000 7fa4e3e00640 1 --2- 172.21.15.5:0/2784116989 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa4dc109120 0x7fa4dc19eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:53.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:53.479+0000 7fa4e3e00640 1 --2- 172.21.15.5:0/2784116989 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa4dc1049a0 0x7fa4dc19ab20 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:53.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:53.479+0000 7fa4e3e00640 1 --2- 172.21.15.5:0/2784116989 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa4dc103ff0 0x7fa4dc19a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:53.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:53.479+0000 7fa4e3e00640 1 -- 172.21.15.5:0/2784116989 >> 172.21.15.5:0/2784116989 conn(0x7fa4dc0ffa90 msgr2=0x7fa4dc1005d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:48:53.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:53.479+0000 7fa4e3e00640 1 -- 172.21.15.5:0/2784116989 shutdown_connections 2024-09-10T09:48:53.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:53.479+0000 7fa4e3e00640 1 -- 172.21.15.5:0/2784116989 wait complete. 2024-09-10T09:48:53.617 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:48:53 smithi005 ceph-mon[30350]: pgmap v709: 129 pgs: 129 active+clean; 247 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 861 KiB/s rd, 1.8 MiB/s wr, 108 op/s 2024-09-10T09:48:53.617 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:48:53 smithi005 ceph-mon[30350]: scrub queued for path: / 2024-09-10T09:48:53.617 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:48:53 smithi005 ceph-mon[30350]: scrub summary: idle+waiting paths [/] 2024-09-10T09:48:53.617 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:48:53 smithi005 ceph-mon[30350]: scrub summary: active paths [/] 2024-09-10T09:48:53.617 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:48:53 smithi005 ceph-mon[30350]: scrub summary: idle+waiting paths [/] 2024-09-10T09:48:53.617 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:48:53 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:48:53.617 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:48:53 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:48:53.618 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:48:53 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:48:53.751 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:48:53 smithi159 ceph-mon[35513]: pgmap v709: 129 pgs: 129 active+clean; 247 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 861 KiB/s rd, 1.8 MiB/s wr, 108 op/s 2024-09-10T09:48:53.751 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:48:53 smithi159 ceph-mon[35513]: scrub queued for path: / 2024-09-10T09:48:53.751 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:48:53 smithi159 ceph-mon[35513]: scrub summary: idle+waiting paths [/] 2024-09-10T09:48:53.752 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:48:53 smithi159 ceph-mon[35513]: scrub summary: active paths [/] 2024-09-10T09:48:53.752 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:48:53 smithi159 ceph-mon[35513]: scrub summary: idle+waiting paths [/] 2024-09-10T09:48:53.752 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:48:53 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:48:53.752 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:48:53 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:48:53.752 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:48:53 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:48:53.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:48:53 smithi102 ceph-mon[35442]: pgmap v709: 129 pgs: 129 active+clean; 247 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 861 KiB/s rd, 1.8 MiB/s wr, 108 op/s 2024-09-10T09:48:53.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:48:53 smithi102 ceph-mon[35442]: scrub queued for path: / 2024-09-10T09:48:53.885 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:48:53 smithi102 ceph-mon[35442]: scrub summary: idle+waiting paths [/] 2024-09-10T09:48:53.885 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:48:53 smithi102 ceph-mon[35442]: scrub summary: active paths [/] 2024-09-10T09:48:53.885 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:48:53 smithi102 ceph-mon[35442]: scrub summary: idle+waiting paths [/] 2024-09-10T09:48:53.885 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:48:53 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:48:53.885 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:48:53 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:48:53.885 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:48:53 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:48:54.120 INFO:tasks.workunit.client.0.smithi005.stdout: 1 650446 6.30 MB/sec execute 395 sec latency 10649.501 ms 2024-09-10T09:48:54.433 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T09:48:54.738 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:48:54 smithi005 ceph-mon[30350]: scrub summary: active paths [/] 2024-09-10T09:48:54.739 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:48:54 smithi005 ceph-mon[30350]: scrub summary: idle+waiting paths [/] 2024-09-10T09:48:54.761 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:48:54 smithi159 ceph-mon[35513]: scrub summary: active paths [/] 2024-09-10T09:48:54.761 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:48:54 smithi159 ceph-mon[35513]: scrub summary: idle+waiting paths [/] 2024-09-10T09:48:54.796 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '81e886f6', total seconds elapsed: 16.9 2024-09-10T09:48:54.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:48:54 smithi102 ceph-mon[35442]: scrub summary: active paths [/] 2024-09-10T09:48:54.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:48:54 smithi102 ceph-mon[35442]: scrub summary: idle+waiting paths [/] 2024-09-10T09:48:54.908 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:48:55.120 INFO:tasks.workunit.client.0.smithi005.stdout: 1 652658 6.31 MB/sec execute 396 sec latency 22.823 ms 2024-09-10T09:48:55.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:55.513+0000 7f4552600640 1 -- 172.21.15.5:0/3032577933 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f454c1047e0 msgr2=0x7f454c108830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:48:55.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:55.513+0000 7f4552600640 1 --2- 172.21.15.5:0/3032577933 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f454c1047e0 0x7f454c108830 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f4538009900 tx=0x7f453802a410 comp rx=0 tx=0).stop 2024-09-10T09:48:55.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:55.513+0000 7f4552600640 1 -- 172.21.15.5:0/3032577933 shutdown_connections 2024-09-10T09:48:55.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:55.513+0000 7f4552600640 1 --2- 172.21.15.5:0/3032577933 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f454c108f60 0x7f454c110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:55.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:55.513+0000 7f4552600640 1 --2- 172.21.15.5:0/3032577933 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f454c1047e0 0x7f454c108830 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:55.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:55.513+0000 7f4552600640 1 --2- 172.21.15.5:0/3032577933 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f454c103e30 0x7f454c104210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:55.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:55.513+0000 7f4552600640 1 -- 172.21.15.5:0/3032577933 >> 172.21.15.5:0/3032577933 conn(0x7f454c0ff910 msgr2=0x7f454c101d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:48:55.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:55.513+0000 7f4552600640 1 -- 172.21.15.5:0/3032577933 shutdown_connections 2024-09-10T09:48:55.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:55.513+0000 7f4552600640 1 -- 172.21.15.5:0/3032577933 wait complete. 2024-09-10T09:48:55.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:55.514+0000 7f4552600640 1 Processor -- start 2024-09-10T09:48:55.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:55.515+0000 7f4552600640 1 -- start start 2024-09-10T09:48:55.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:55.515+0000 7f4552600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f454c103e30 0x7f454c11a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:48:55.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:55.515+0000 7f4552600640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f454c1047e0 0x7f454c11a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:48:55.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:55.515+0000 7f4552600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f454c108f60 0x7f454c10ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:48:55.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:55.515+0000 7f4552600640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f454c0781d0 con 0x7f454c103e30 2024-09-10T09:48:55.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:55.515+0000 7f4552600640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f454c078050 con 0x7f454c108f60 2024-09-10T09:48:55.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:55.515+0000 7f4552600640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f454c078350 con 0x7f454c1047e0 2024-09-10T09:48:55.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:55.515+0000 7f454be00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f454c1047e0 0x7f454c11a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:48:55.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:55.515+0000 7f454be00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f454c1047e0 0x7f454c11a730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:35248/0 (socket says 172.21.15.5:35248) 2024-09-10T09:48:55.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:55.515+0000 7f454be00640 1 -- 172.21.15.5:0/1218779393 learned_addr learned my addr 172.21.15.5:0/1218779393 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:48:55.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:55.515+0000 7f4551200640 1 --2- 172.21.15.5:0/1218779393 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f454c103e30 0x7f454c11a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:48:55.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:55.515+0000 7f4551c00640 1 --2- 172.21.15.5:0/1218779393 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f454c108f60 0x7f454c10ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:48:55.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:55.516+0000 7f4551c00640 1 -- 172.21.15.5:0/1218779393 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f454c1047e0 msgr2=0x7f454c11a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:48:55.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:55.516+0000 7f4551c00640 1 --2- 172.21.15.5:0/1218779393 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f454c1047e0 0x7f454c11a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:55.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:55.516+0000 7f4551c00640 1 -- 172.21.15.5:0/1218779393 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f454c103e30 msgr2=0x7f454c11a1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:48:55.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:55.516+0000 7f4551c00640 1 --2- 172.21.15.5:0/1218779393 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f454c103e30 0x7f454c11a1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:55.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:55.516+0000 7f4551c00640 1 -- 172.21.15.5:0/1218779393 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f454c110870 con 0x7f454c108f60 2024-09-10T09:48:55.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:55.516+0000 7f454be00640 1 --2- 172.21.15.5:0/1218779393 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f454c1047e0 0x7f454c11a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:48:55.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:55.516+0000 7f4551c00640 1 --2- 172.21.15.5:0/1218779393 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f454c108f60 0x7f454c10ffb0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f453c00e970 tx=0x7f453c00ee30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:48:55.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:55.516+0000 7f4549600640 1 -- 172.21.15.5:0/1218779393 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f453c00ccc0 con 0x7f454c108f60 2024-09-10T09:48:55.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:55.516+0000 7f4549600640 1 -- 172.21.15.5:0/1218779393 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f453c005020 con 0x7f454c108f60 2024-09-10T09:48:55.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:55.516+0000 7f4549600640 1 -- 172.21.15.5:0/1218779393 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f453c004a00 con 0x7f454c108f60 2024-09-10T09:48:55.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:55.517+0000 7f4552600640 1 -- 172.21.15.5:0/1218779393 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f454c110b60 con 0x7f454c108f60 2024-09-10T09:48:55.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:55.517+0000 7f4552600640 1 -- 172.21.15.5:0/1218779393 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f454c115e40 con 0x7f454c108f60 2024-09-10T09:48:55.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:55.521+0000 7f45566d2740 1 Processor -- start 2024-09-10T09:48:55.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:55.521+0000 7f4549600640 1 -- 172.21.15.5:0/1218779393 <== mon.2 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f453c0051d0 con 0x7f454c108f60 2024-09-10T09:48:55.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:55.522+0000 7f45566d2740 1 -- start start 2024-09-10T09:48:55.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:55.522+0000 7f45566d2740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560bdda43d50 0x560bdda44130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:48:55.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:55.522+0000 7f45566d2740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560bdda44670 0x560bdda4e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:48:55.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:55.522+0000 7f45566d2740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560bdda4ef30 0x560bdda51320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:48:55.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:55.522+0000 7f45566d2740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x560bdd9aa3e0 con 0x560bdda43d50 2024-09-10T09:48:55.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:55.522+0000 7f45566d2740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x560bdd951d80 con 0x560bdda4ef30 2024-09-10T09:48:55.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:55.522+0000 7f45566d2740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x560bdd9a63a0 con 0x560bdda44670 2024-09-10T09:48:55.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:55.522+0000 7f4551200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560bdda44670 0x560bdda4e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:48:55.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:55.522+0000 7f454be00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560bdda43d50 0x560bdda44130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:48:55.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:55.522+0000 7f4551200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560bdda44670 0x560bdda4e9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:35258/0 (socket says 172.21.15.5:35258) 2024-09-10T09:48:55.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:55.522+0000 7f4551200640 1 -- 172.21.15.5:0/1573883104 learned_addr learned my addr 172.21.15.5:0/1573883104 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:48:55.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:55.523+0000 7f454be00640 1 --2- 172.21.15.5:0/1573883104 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560bdda4ef30 0x560bdda51320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:48:55.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:55.523+0000 7f4551200640 1 -- 172.21.15.5:0/1573883104 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560bdda4ef30 msgr2=0x560bdda51320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:48:55.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:55.523+0000 7f4551200640 1 --2- 172.21.15.5:0/1573883104 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560bdda4ef30 0x560bdda51320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:55.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:55.523+0000 7f4551200640 1 -- 172.21.15.5:0/1573883104 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560bdda43d50 msgr2=0x560bdda44130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:48:55.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:55.523+0000 7f4551200640 1 --2- 172.21.15.5:0/1573883104 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560bdda43d50 0x560bdda44130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:55.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:55.523+0000 7f4551200640 1 -- 172.21.15.5:0/1573883104 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560bdd970ae0 con 0x560bdda44670 2024-09-10T09:48:55.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:55.523+0000 7f454be00640 1 --2- 172.21.15.5:0/1573883104 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560bdda43d50 0x560bdda44130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:48:55.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:55.523+0000 7f4551200640 1 --2- 172.21.15.5:0/1573883104 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560bdda44670 0x560bdda4e9f0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f45440114f0 tx=0x7f45440119b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=39796389bb072225 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:48:55.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:55.523+0000 7f4542000640 1 -- 172.21.15.5:0/1573883104 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f454401e070 con 0x560bdda44670 2024-09-10T09:48:55.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:55.524+0000 7f4542000640 1 -- 172.21.15.5:0/1573883104 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f454400fb00 con 0x560bdda44670 2024-09-10T09:48:55.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:55.524+0000 7f4542000640 1 -- 172.21.15.5:0/1573883104 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4544019bf0 con 0x560bdda44670 2024-09-10T09:48:55.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:55.524+0000 7f4549600640 1 --2- 172.21.15.5:0/1218779393 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f45300a2c00 0x7f45300a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:48:55.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:55.524+0000 7f4549600640 1 -- 172.21.15.5:0/1218779393 <== mon.2 v2:172.21.15.102:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f453c0cf330 con 0x7f454c108f60 2024-09-10T09:48:55.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:55.524+0000 7f454be00640 1 --2- 172.21.15.5:0/1218779393 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f45300a2c00 0x7f45300a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:48:55.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:55.524+0000 7f45566d2740 1 -- 172.21.15.5:0/1573883104 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560bdda44670 msgr2=0x560bdda4e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:48:55.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:55.524+0000 7f45566d2740 1 --2- 172.21.15.5:0/1573883104 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560bdda44670 0x560bdda4e9f0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f45440114f0 tx=0x7f45440119b0 comp rx=0 tx=0).stop 2024-09-10T09:48:55.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:55.525+0000 7f45566d2740 1 -- 172.21.15.5:0/1573883104 shutdown_connections 2024-09-10T09:48:55.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:55.525+0000 7f45566d2740 1 --2- 172.21.15.5:0/1573883104 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560bdda4ef30 0x560bdda51320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:55.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:55.525+0000 7f45566d2740 1 --2- 172.21.15.5:0/1573883104 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560bdda44670 0x560bdda4e9f0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:55.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:55.525+0000 7f45566d2740 1 --2- 172.21.15.5:0/1573883104 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560bdda43d50 0x560bdda44130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:55.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:55.525+0000 7f45566d2740 1 -- 172.21.15.5:0/1573883104 >> 172.21.15.5:0/1573883104 conn(0x560bdda402e0 msgr2=0x560bdda406c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:48:55.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:55.525+0000 7f45566d2740 1 -- 172.21.15.5:0/1573883104 shutdown_connections 2024-09-10T09:48:55.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:55.525+0000 7f45566d2740 1 -- 172.21.15.5:0/1573883104 wait complete. 2024-09-10T09:48:55.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:55.525+0000 7f454be00640 1 --2- 172.21.15.5:0/1218779393 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f45300a2c00 0x7f45300a50c0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f4538031520 tx=0x7f45380314b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:48:55.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:55.525+0000 7f45566d2740 1 -- start start 2024-09-10T09:48:55.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:55.526+0000 7f45566d2740 1 Processor -- start 2024-09-10T09:48:55.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:55.526+0000 7f45566d2740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560bdda43d50 0x560bdda4be20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:48:55.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:55.526+0000 7f45566d2740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560bdda44670 0x560bdda4c360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:48:55.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:55.526+0000 7f45566d2740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560bdda4ef30 0x560bdda4c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:48:55.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:55.526+0000 7f45566d2740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x560bdda51ba0 con 0x560bdda44670 2024-09-10T09:48:55.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:55.526+0000 7f45566d2740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x560bdd9650f0 con 0x560bdda43d50 2024-09-10T09:48:55.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:55.526+0000 7f45566d2740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x560bdda51d20 con 0x560bdda4ef30 2024-09-10T09:48:55.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:55.526+0000 7f45566d2740 10 client.0 _ll_register_callbacks cb 0x560bdd9569e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:48:55.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:55.526+0000 7f454be00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560bdda44670 0x560bdda4c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:48:55.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:55.526+0000 7f4551c00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560bdda4ef30 0x560bdda4c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:48:55.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:55.526+0000 7f454be00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560bdda44670 0x560bdda4c360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42162/0 (socket says 172.21.15.5:42162) 2024-09-10T09:48:55.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:55.526+0000 7f454be00640 1 -- 172.21.15.5:0/3448862468 learned_addr learned my addr 172.21.15.5:0/3448862468 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:48:55.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:55.526+0000 7f4551200640 1 --2- 172.21.15.5:0/3448862468 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560bdda43d50 0x560bdda4be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:48:55.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:55.527+0000 7f4551c00640 1 -- 172.21.15.5:0/3448862468 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560bdda43d50 msgr2=0x560bdda4be20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:48:55.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:55.527+0000 7f4551c00640 1 --2- 172.21.15.5:0/3448862468 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560bdda43d50 0x560bdda4be20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:55.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:55.527+0000 7f4551c00640 1 -- 172.21.15.5:0/3448862468 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560bdda44670 msgr2=0x560bdda4c360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:48:55.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:55.527+0000 7f4551c00640 1 --2- 172.21.15.5:0/3448862468 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560bdda44670 0x560bdda4c360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:55.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:55.527+0000 7f4551c00640 1 -- 172.21.15.5:0/3448862468 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560bdd71b3c0 con 0x560bdda4ef30 2024-09-10T09:48:55.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:55.527+0000 7f454be00640 1 --2- 172.21.15.5:0/3448862468 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560bdda44670 0x560bdda4c360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:48:55.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:55.527+0000 7f4551200640 1 --2- 172.21.15.5:0/3448862468 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560bdda43d50 0x560bdda4be20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:48:55.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:55.527+0000 7f4551c00640 1 --2- 172.21.15.5:0/3448862468 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560bdda4ef30 0x560bdda4c8a0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f453c09b440 tx=0x7f453c005620 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c2740f03993e7976 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:48:55.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:55.527+0000 7f4536a00640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-09-10T09:48:55.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:55.528+0000 7f4536a00640 1 -- 172.21.15.5:0/3448862468 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f453c01c070 con 0x560bdda4ef30 2024-09-10T09:48:55.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:55.528+0000 7f4536a00640 1 -- 172.21.15.5:0/3448862468 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f453c0a3f20 con 0x560bdda4ef30 2024-09-10T09:48:55.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:55.528+0000 7f45566d2740 1 -- 172.21.15.5:0/3448862468 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f454400fb00 con 0x560bdda4ef30 2024-09-10T09:48:55.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:55.528+0000 7f4536a00640 1 -- 172.21.15.5:0/3448862468 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f453c09faf0 con 0x560bdda4ef30 2024-09-10T09:48:55.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:55.528+0000 7f45566d2740 10 client.27553 fetch_fsmap learned FSMap version 36 2024-09-10T09:48:55.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:55.528+0000 7f45566d2740 1 -- 172.21.15.5:0/3448862468 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x560bdda5d850 con 0x560bdda4ef30 2024-09-10T09:48:55.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:55.529+0000 7f4536a00640 1 -- 172.21.15.5:0/3448862468 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f454400fb00 con 0x560bdda4ef30 2024-09-10T09:48:55.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:55.529+0000 7f4536a00640 1 -- 172.21.15.5:0/3448862468 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 36) ==== 8370+0+0 (secure 0 0 0) 0x7f453c09ab90 con 0x560bdda4ef30 2024-09-10T09:48:55.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:55.529+0000 7f45566d2740 10 client.27553 fetch_fsmap finished waiting for FSMap version 36 2024-09-10T09:48:55.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:55.529+0000 7f45566d2740 10 client.27553 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T09:48:55.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:55.529+0000 7f45566d2740 20 client.27553 populate_metadata read hostname 'smithi005' 2024-09-10T09:48:55.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:55.529+0000 7f45566d2740 1 --2- 172.21.15.5:0/3448862468 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x560bdda5ed10 0x560bdda61160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:48:55.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:55.529+0000 7f45566d2740 4 client.27553 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:48:55.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:55.529+0000 7f45566d2740 1 -- 172.21.15.5:0/3448862468 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560bdd9b7950 con 0x560bdda5ed10 2024-09-10T09:48:55.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:55.529+0000 7f4551200640 1 --2- 172.21.15.5:0/3448862468 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x560bdda5ed10 0x560bdda61160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:48:55.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:55.530+0000 7f4551200640 1 --2- 172.21.15.5:0/3448862468 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x560bdda5ed10 0x560bdda61160 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5b71c448a6557c80 server_cookie=bc8e81346d520547 in_seq=0 out_seq=0 2024-09-10T09:48:55.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:55.530+0000 7f4536a00640 10 client.27553 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T09:48:55.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:55.531+0000 7f4536a00640 1 -- 172.21.15.5:0/3448862468 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560bdd9b7950 con 0x560bdda5ed10 2024-09-10T09:48:55.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:55.531+0000 7f4536a00640 10 client.27553 handle_command_reply: tid=0 2024-09-10T09:48:55.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:55.531+0000 7f45566d2740 1 client.27553 shutdown 2024-09-10T09:48:55.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:55.531+0000 7f45566d2740 1 -- 172.21.15.5:0/3448862468 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560bdda4ef30 msgr2=0x560bdda4c8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:48:55.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:55.531+0000 7f45566d2740 1 --2- 172.21.15.5:0/3448862468 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560bdda4ef30 0x560bdda4c8a0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f453c09b440 tx=0x7f453c005620 comp rx=0 tx=0).stop 2024-09-10T09:48:55.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:55.531+0000 7f45566d2740 1 -- 172.21.15.5:0/3448862468 shutdown_connections 2024-09-10T09:48:55.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:55.531+0000 7f45566d2740 1 --2- 172.21.15.5:0/3448862468 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x560bdda5ed10 0x560bdda61160 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:55.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:55.531+0000 7f45566d2740 1 --2- 172.21.15.5:0/3448862468 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560bdda4ef30 0x560bdda4c8a0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:55.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:55.531+0000 7f45566d2740 1 --2- 172.21.15.5:0/3448862468 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560bdda44670 0x560bdda4c360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:55.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:55.531+0000 7f45566d2740 1 --2- 172.21.15.5:0/3448862468 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560bdda43d50 0x560bdda4be20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:55.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:55.531+0000 7f45566d2740 1 -- 172.21.15.5:0/3448862468 >> 172.21.15.5:0/3448862468 conn(0x560bdda402e0 msgr2=0x560bdda46010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:48:55.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:55.531+0000 7f4536a00640 0 client.27553 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T09:48:55.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:55.531+0000 7f45566d2740 1 -- 172.21.15.5:0/3448862468 shutdown_connections 2024-09-10T09:48:55.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:55.532+0000 7f45566d2740 1 -- 172.21.15.5:0/3448862468 wait complete. 2024-09-10T09:48:55.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:55.532+0000 7f45566d2740 20 client.27553 trim_cache size 0 max 16384 2024-09-10T09:48:55.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:55.552+0000 7f45566d2740 1 Processor -- start 2024-09-10T09:48:55.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:55.552+0000 7f45566d2740 1 -- start start 2024-09-10T09:48:55.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:55.552+0000 7f45566d2740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560bdda43d50 0x560bdda406c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:48:55.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:55.552+0000 7f45566d2740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560bdda44670 0x560bdda4c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:48:55.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:55.552+0000 7f45566d2740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560bdda4ef30 0x560bdda45eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:48:55.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:55.552+0000 7f45566d2740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f454c1bdad0 con 0x560bdda43d50 2024-09-10T09:48:55.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:55.552+0000 7f45566d2740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f454c1bf610 con 0x560bdda44670 2024-09-10T09:48:55.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:55.552+0000 7f45566d2740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f454c1be1a0 con 0x560bdda4ef30 2024-09-10T09:48:55.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:55.552+0000 7f4551200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560bdda43d50 0x560bdda406c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:48:55.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:55.552+0000 7f4551c00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560bdda4ef30 0x560bdda45eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:48:55.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:55.552+0000 7f454be00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560bdda44670 0x560bdda4c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:48:55.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:55.552+0000 7f4551c00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560bdda4ef30 0x560bdda45eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:35288/0 (socket says 172.21.15.5:35288) 2024-09-10T09:48:55.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:55.552+0000 7f454be00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560bdda44670 0x560bdda4c6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:54184/0 (socket says 172.21.15.5:54184) 2024-09-10T09:48:55.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:55.552+0000 7f4551c00640 1 -- 172.21.15.5:0/61714327 learned_addr learned my addr 172.21.15.5:0/61714327 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:48:55.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:55.553+0000 7f4551c00640 1 -- 172.21.15.5:0/61714327 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560bdda44670 msgr2=0x560bdda4c6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:48:55.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:55.553+0000 7f4551c00640 1 --2- 172.21.15.5:0/61714327 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560bdda44670 0x560bdda4c6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:55.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:55.553+0000 7f4551c00640 1 -- 172.21.15.5:0/61714327 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560bdda43d50 msgr2=0x560bdda406c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:48:55.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:55.553+0000 7f4551c00640 1 --2- 172.21.15.5:0/61714327 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560bdda43d50 0x560bdda406c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:55.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:55.553+0000 7f4551c00640 1 -- 172.21.15.5:0/61714327 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560bdda4add0 con 0x560bdda4ef30 2024-09-10T09:48:55.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:55.553+0000 7f4551c00640 1 --2- 172.21.15.5:0/61714327 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560bdda4ef30 0x560bdda45eb0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f453c097ba0 tx=0x7f453c086050 comp rx=0 tx=0).ready entity=mon.1 client_cookie=57d7b8c2c7e89212 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:48:55.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:55.553+0000 7f4541600640 1 -- 172.21.15.5:0/61714327 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f453c07f070 con 0x560bdda4ef30 2024-09-10T09:48:55.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:55.553+0000 7f4541600640 1 -- 172.21.15.5:0/61714327 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f453c0a43b0 con 0x560bdda4ef30 2024-09-10T09:48:55.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:55.553+0000 7f4541600640 1 -- 172.21.15.5:0/61714327 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f453c01c050 con 0x560bdda4ef30 2024-09-10T09:48:55.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:55.554+0000 7f45566d2740 1 -- 172.21.15.5:0/61714327 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560bdda4ef30 msgr2=0x560bdda45eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:48:55.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:55.554+0000 7f45566d2740 1 --2- 172.21.15.5:0/61714327 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560bdda4ef30 0x560bdda45eb0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f453c097ba0 tx=0x7f453c086050 comp rx=0 tx=0).stop 2024-09-10T09:48:55.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:55.554+0000 7f45566d2740 1 -- 172.21.15.5:0/61714327 shutdown_connections 2024-09-10T09:48:55.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:55.554+0000 7f45566d2740 1 --2- 172.21.15.5:0/61714327 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560bdda4ef30 0x560bdda45eb0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:55.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:55.554+0000 7f45566d2740 1 --2- 172.21.15.5:0/61714327 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560bdda44670 0x560bdda4c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:55.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:55.554+0000 7f45566d2740 1 --2- 172.21.15.5:0/61714327 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560bdda43d50 0x560bdda406c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:55.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:55.554+0000 7f45566d2740 1 -- 172.21.15.5:0/61714327 >> 172.21.15.5:0/61714327 conn(0x560bdda402e0 msgr2=0x560bdda457a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:48:55.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:55.554+0000 7f45566d2740 1 -- 172.21.15.5:0/61714327 shutdown_connections 2024-09-10T09:48:55.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:55.554+0000 7f45566d2740 1 -- 172.21.15.5:0/61714327 wait complete. 2024-09-10T09:48:55.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:55.555+0000 7f45566d2740 1 -- start start 2024-09-10T09:48:55.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:55.555+0000 7f45566d2740 1 Processor -- start 2024-09-10T09:48:55.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:55.555+0000 7f45566d2740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560bdda43d50 0x560bdda57140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:48:55.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:55.555+0000 7f45566d2740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560bdda44670 0x560bdda57680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:48:55.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:55.555+0000 7f45566d2740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560bdda4ef30 0x560bdda52530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:48:55.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:55.555+0000 7f45566d2740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f454c1bef40 con 0x560bdda4ef30 2024-09-10T09:48:55.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:55.555+0000 7f45566d2740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f454c1be870 con 0x560bdda44670 2024-09-10T09:48:55.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:55.555+0000 7f45566d2740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f454c1bbab0 con 0x560bdda43d50 2024-09-10T09:48:55.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:55.555+0000 7f45566d2740 10 client.0 _ll_register_callbacks cb 0x7f453c09b280 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:48:55.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:55.556+0000 7f454be00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560bdda44670 0x560bdda57680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:48:55.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:55.556+0000 7f454be00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560bdda44670 0x560bdda57680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:54200/0 (socket says 172.21.15.5:54200) 2024-09-10T09:48:55.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:55.556+0000 7f454be00640 1 -- 172.21.15.5:0/914127942 learned_addr learned my addr 172.21.15.5:0/914127942 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:48:55.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:55.556+0000 7f4551c00640 1 --2- 172.21.15.5:0/914127942 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560bdda4ef30 0x560bdda52530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:48:55.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:55.556+0000 7f4551200640 1 --2- 172.21.15.5:0/914127942 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560bdda43d50 0x560bdda57140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:48:55.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:55.556+0000 7f454be00640 1 -- 172.21.15.5:0/914127942 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560bdda43d50 msgr2=0x560bdda57140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:48:55.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:55.556+0000 7f454be00640 1 --2- 172.21.15.5:0/914127942 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560bdda43d50 0x560bdda57140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:55.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:55.556+0000 7f454be00640 1 -- 172.21.15.5:0/914127942 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560bdda4ef30 msgr2=0x560bdda52530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:48:55.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:55.556+0000 7f454be00640 1 --2- 172.21.15.5:0/914127942 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560bdda4ef30 0x560bdda52530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:55.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:55.556+0000 7f454be00640 1 -- 172.21.15.5:0/914127942 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560bdda444a0 con 0x560bdda44670 2024-09-10T09:48:55.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:55.557+0000 7f454be00640 1 --2- 172.21.15.5:0/914127942 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560bdda44670 0x560bdda57680 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x560bdda4b6b0 tx=0x7f453802a410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=744f769748a98dd8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:48:55.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:55.557+0000 7f4536a00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-09-10T09:48:55.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:55.557+0000 7f4536a00640 1 -- 172.21.15.5:0/914127942 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4538035040 con 0x560bdda44670 2024-09-10T09:48:55.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:55.557+0000 7f4536a00640 1 -- 172.21.15.5:0/914127942 <== mon.2 v2:172.21.15.102:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f4538003e40 con 0x560bdda44670 2024-09-10T09:48:55.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:55.557+0000 7f45566d2740 1 -- 172.21.15.5:0/914127942 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f453c0a43b0 con 0x560bdda44670 2024-09-10T09:48:55.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:55.557+0000 7f4536a00640 1 -- 172.21.15.5:0/914127942 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f453803d450 con 0x560bdda44670 2024-09-10T09:48:55.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:55.557+0000 7f45566d2740 10 client.27551 fetch_fsmap learned FSMap version 36 2024-09-10T09:48:55.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:55.557+0000 7f45566d2740 1 -- 172.21.15.5:0/914127942 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x560bddac9370 con 0x560bdda44670 2024-09-10T09:48:55.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:55.558+0000 7f4536a00640 1 -- 172.21.15.5:0/914127942 <== mon.2 v2:172.21.15.102:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f453c0a43b0 con 0x560bdda44670 2024-09-10T09:48:55.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:55.558+0000 7f4536a00640 1 -- 172.21.15.5:0/914127942 <== mon.2 v2:172.21.15.102:3300/0 5 ==== fsmap(e 36) ==== 8370+0+0 (secure 0 0 0) 0x7f453803c680 con 0x560bdda44670 2024-09-10T09:48:55.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:55.558+0000 7f45566d2740 10 client.27551 fetch_fsmap finished waiting for FSMap version 36 2024-09-10T09:48:55.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:55.558+0000 7f45566d2740 10 client.27551 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T09:48:55.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:55.558+0000 7f45566d2740 20 client.27551 populate_metadata read hostname 'smithi005' 2024-09-10T09:48:55.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:55.558+0000 7f45566d2740 1 --2- 172.21.15.5:0/914127942 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x560bddac9b20 0x560bddacbf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:48:55.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:55.558+0000 7f45566d2740 4 client.27551 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T09:48:55.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:55.558+0000 7f45566d2740 1 -- 172.21.15.5:0/914127942 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "scrub status"}) -- 0x560bdd9b1e60 con 0x560bddac9b20 2024-09-10T09:48:55.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:55.559+0000 7f4551200640 1 --2- 172.21.15.5:0/914127942 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x560bddac9b20 0x560bddacbf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:48:55.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:55.559+0000 7f4551200640 1 --2- 172.21.15.5:0/914127942 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x560bddac9b20 0x560bddacbf40 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=151f5041a11f0ec0 server_cookie=f17a722475d175e2 in_seq=0 out_seq=0 2024-09-10T09:48:55.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:55.559+0000 7f4536a00640 10 client.27551 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T09:48:55.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:55.560+0000 7f4536a00640 1 -- 172.21.15.5:0/914127942 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x560bdd9b1e60 con 0x560bddac9b20 2024-09-10T09:48:55.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:55.560+0000 7f4536a00640 10 client.27551 handle_command_reply: tid=0 2024-09-10T09:48:55.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:55.560+0000 7f45566d2740 1 client.27551 shutdown 2024-09-10T09:48:55.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:55.560+0000 7f45566d2740 1 -- 172.21.15.5:0/914127942 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560bdda44670 msgr2=0x560bdda57680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:48:55.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:55.560+0000 7f45566d2740 1 --2- 172.21.15.5:0/914127942 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560bdda44670 0x560bdda57680 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x560bdda4b6b0 tx=0x7f453802a410 comp rx=0 tx=0).stop 2024-09-10T09:48:55.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:55.560+0000 7f45566d2740 1 -- 172.21.15.5:0/914127942 shutdown_connections 2024-09-10T09:48:55.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:55.560+0000 7f45566d2740 1 --2- 172.21.15.5:0/914127942 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x560bddac9b20 0x560bddacbf40 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:55.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:55.560+0000 7f45566d2740 1 --2- 172.21.15.5:0/914127942 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560bdda4ef30 0x560bdda52530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:55.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:55.560+0000 7f45566d2740 1 --2- 172.21.15.5:0/914127942 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560bdda44670 0x560bdda57680 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:55.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:55.560+0000 7f45566d2740 1 --2- 172.21.15.5:0/914127942 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560bdda43d50 0x560bdda57140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:55.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:55.560+0000 7f4536a00640 0 client.27551 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T09:48:55.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:55.560+0000 7f45566d2740 1 -- 172.21.15.5:0/914127942 >> 172.21.15.5:0/914127942 conn(0x560bdda402e0 msgr2=0x560bdda45fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:48:55.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:55.560+0000 7f45566d2740 1 -- 172.21.15.5:0/914127942 shutdown_connections 2024-09-10T09:48:55.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:55.560+0000 7f45566d2740 1 -- 172.21.15.5:0/914127942 wait complete. 2024-09-10T09:48:55.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:55.560+0000 7f45566d2740 20 client.27551 trim_cache size 0 max 16384 2024-09-10T09:48:55.565 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-10T09:48:55.565 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-10T09:48:55.565 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2024-09-10T09:48:55.566 INFO:teuthology.orchestra.run.smithi005.stdout: "821a9e03-24c6-4da3-9bb8-fd95bee630de": { 2024-09-10T09:48:55.566 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2024-09-10T09:48:55.566 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "821a9e03-24c6-4da3-9bb8-fd95bee630de", 2024-09-10T09:48:55.566 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2024-09-10T09:48:55.566 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T09:48:55.566 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T09:48:55.566 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-10T09:48:55.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:55.561+0000 7f4552600640 1 -- 172.21.15.5:0/1218779393 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f45300a2c00 msgr2=0x7f45300a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:48:55.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:55.561+0000 7f4552600640 1 --2- 172.21.15.5:0/1218779393 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f45300a2c00 0x7f45300a50c0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f4538031520 tx=0x7f45380314b0 comp rx=0 tx=0).stop 2024-09-10T09:48:55.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:55.561+0000 7f4552600640 1 -- 172.21.15.5:0/1218779393 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f454c108f60 msgr2=0x7f454c10ffb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:48:55.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:55.561+0000 7f4552600640 1 --2- 172.21.15.5:0/1218779393 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f454c108f60 0x7f454c10ffb0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f453c00e970 tx=0x7f453c00ee30 comp rx=0 tx=0).stop 2024-09-10T09:48:55.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:55.561+0000 7f4552600640 1 -- 172.21.15.5:0/1218779393 shutdown_connections 2024-09-10T09:48:55.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:55.561+0000 7f4552600640 1 --2- 172.21.15.5:0/1218779393 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f45300a2c00 0x7f45300a50c0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:55.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:55.561+0000 7f4552600640 1 --2- 172.21.15.5:0/1218779393 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f454c108f60 0x7f454c10ffb0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:55.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:55.561+0000 7f4552600640 1 --2- 172.21.15.5:0/1218779393 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f454c1047e0 0x7f454c11a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:55.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:55.561+0000 7f4552600640 1 --2- 172.21.15.5:0/1218779393 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f454c103e30 0x7f454c11a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:48:55.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:55.561+0000 7f4552600640 1 -- 172.21.15.5:0/1218779393 >> 172.21.15.5:0/1218779393 conn(0x7f454c0ff910 msgr2=0x7f454c101d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:48:55.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:55.561+0000 7f4552600640 1 -- 172.21.15.5:0/1218779393 shutdown_connections 2024-09-10T09:48:55.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:48:55.561+0000 7f4552600640 1 -- 172.21.15.5:0/1218779393 wait complete. 2024-09-10T09:48:55.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:48:55 smithi005 ceph-mon[30350]: pgmap v710: 129 pgs: 129 active+clean; 249 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 671 KiB/s rd, 1.9 MiB/s wr, 107 op/s 2024-09-10T09:48:55.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:48:55 smithi159 ceph-mon[35513]: pgmap v710: 129 pgs: 129 active+clean; 249 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 671 KiB/s rd, 1.9 MiB/s wr, 107 op/s 2024-09-10T09:48:55.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:48:55 smithi102 ceph-mon[35442]: pgmap v710: 129 pgs: 129 active+clean; 249 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 671 KiB/s rd, 1.9 MiB/s wr, 107 op/s 2024-09-10T09:48:56.120 INFO:tasks.workunit.client.0.smithi005.stdout: 1 654315 6.32 MB/sec execute 397 sec latency 23.747 ms 2024-09-10T09:48:56.593 INFO:tasks.cephfs.filesystem:scrub status for tag:821a9e03-24c6-4da3-9bb8-fd95bee630de - {'path': '/', 'tag': '821a9e03-24c6-4da3-9bb8-fd95bee630de', 'options': 'recursive,force'} 2024-09-10T09:48:57.120 INFO:tasks.workunit.client.0.smithi005.stdout: 1 654430 6.30 MB/sec execute 398 sec latency 861.245 ms 2024-09-10T09:48:57.714 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:48:57 smithi005 ceph-mon[30350]: pgmap v711: 129 pgs: 129 active+clean; 247 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 579 KiB/s rd, 1.8 MiB/s wr, 117 op/s 2024-09-10T09:48:57.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:48:57 smithi159 ceph-mon[35513]: pgmap v711: 129 pgs: 129 active+clean; 247 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 579 KiB/s rd, 1.8 MiB/s wr, 117 op/s 2024-09-10T09:48:57.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:48:57 smithi102 ceph-mon[35442]: pgmap v711: 129 pgs: 129 active+clean; 247 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 579 KiB/s rd, 1.8 MiB/s wr, 117 op/s 2024-09-10T09:48:58.120 INFO:tasks.workunit.client.0.smithi005.stdout: 1 655355 6.30 MB/sec execute 399 sec latency 1263.884 ms 2024-09-10T09:48:59.121 INFO:tasks.workunit.client.0.smithi005.stdout: 1 657647 6.31 MB/sec execute 400 sec latency 35.951 ms 2024-09-10T09:48:59.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:48:59 smithi005 ceph-mon[30350]: pgmap v712: 129 pgs: 129 active+clean; 245 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 385 KiB/s rd, 1.7 MiB/s wr, 79 op/s 2024-09-10T09:48:59.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:48:59 smithi159 ceph-mon[35513]: pgmap v712: 129 pgs: 129 active+clean; 245 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 385 KiB/s rd, 1.7 MiB/s wr, 79 op/s 2024-09-10T09:48:59.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:48:59 smithi102 ceph-mon[35442]: pgmap v712: 129 pgs: 129 active+clean; 245 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 385 KiB/s rd, 1.7 MiB/s wr, 79 op/s 2024-09-10T09:49:00.121 INFO:tasks.workunit.client.0.smithi005.stdout: 1 660192 6.33 MB/sec execute 401 sec latency 18.823 ms 2024-09-10T09:49:01.121 INFO:tasks.workunit.client.0.smithi005.stdout: 1 662023 6.34 MB/sec execute 402 sec latency 24.193 ms 2024-09-10T09:49:01.610 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:49:01 smithi005 ceph-mon[30350]: pgmap v713: 129 pgs: 129 active+clean; 244 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.1 MiB/s wr, 132 op/s 2024-09-10T09:49:01.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:49:01 smithi159 ceph-mon[35513]: pgmap v713: 129 pgs: 129 active+clean; 244 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.1 MiB/s wr, 132 op/s 2024-09-10T09:49:01.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:49:01 smithi102 ceph-mon[35442]: pgmap v713: 129 pgs: 129 active+clean; 244 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.1 MiB/s wr, 132 op/s 2024-09-10T09:49:02.121 INFO:tasks.workunit.client.0.smithi005.stdout: 1 662509 6.33 MB/sec execute 403 sec latency 812.745 ms 2024-09-10T09:49:03.121 INFO:tasks.workunit.client.0.smithi005.stdout: 1 662621 6.31 MB/sec execute 404 sec latency 1120.469 ms 2024-09-10T09:49:03.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:49:03 smithi005 ceph-mon[30350]: pgmap v714: 129 pgs: 129 active+clean; 247 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.2 MiB/s wr, 154 op/s 2024-09-10T09:49:03.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:49:03 smithi159 ceph-mon[35513]: pgmap v714: 129 pgs: 129 active+clean; 247 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.2 MiB/s wr, 154 op/s 2024-09-10T09:49:03.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:49:03 smithi102 ceph-mon[35442]: pgmap v714: 129 pgs: 129 active+clean; 247 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.2 MiB/s wr, 154 op/s 2024-09-10T09:49:04.121 INFO:tasks.workunit.client.0.smithi005.stdout: 1 664666 6.33 MB/sec execute 405 sec latency 21.859 ms 2024-09-10T09:49:05.121 INFO:tasks.workunit.client.0.smithi005.stdout: 1 667069 6.35 MB/sec execute 406 sec latency 21.452 ms 2024-09-10T09:49:05.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:49:05 smithi005 ceph-mon[30350]: pgmap v715: 129 pgs: 129 active+clean; 252 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.5 MiB/s wr, 157 op/s 2024-09-10T09:49:05.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:49:05 smithi159 ceph-mon[35513]: pgmap v715: 129 pgs: 129 active+clean; 252 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.5 MiB/s wr, 157 op/s 2024-09-10T09:49:05.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:49:05 smithi102 ceph-mon[35442]: pgmap v715: 129 pgs: 129 active+clean; 252 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.5 MiB/s wr, 157 op/s 2024-09-10T09:49:06.121 INFO:tasks.workunit.client.0.smithi005.stdout: 1 668874 6.35 MB/sec execute 407 sec latency 24.635 ms 2024-09-10T09:49:07.121 INFO:tasks.workunit.client.0.smithi005.stdout: 1 669804 6.35 MB/sec execute 408 sec latency 330.144 ms 2024-09-10T09:49:07.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:49:07 smithi005 ceph-mon[30350]: pgmap v716: 129 pgs: 129 active+clean; 252 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.4 MiB/s wr, 178 op/s 2024-09-10T09:49:07.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:49:07 smithi005 ceph-mon[30350]: scrub summary: idle 2024-09-10T09:49:07.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:49:07 smithi005 ceph-mon[30350]: scrub summary: idle 2024-09-10T09:49:07.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:49:07 smithi159 ceph-mon[35513]: pgmap v716: 129 pgs: 129 active+clean; 252 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.4 MiB/s wr, 178 op/s 2024-09-10T09:49:07.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:49:07 smithi159 ceph-mon[35513]: scrub summary: idle 2024-09-10T09:49:07.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:49:07 smithi159 ceph-mon[35513]: scrub summary: idle 2024-09-10T09:49:07.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:49:07 smithi102 ceph-mon[35442]: pgmap v716: 129 pgs: 129 active+clean; 252 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.4 MiB/s wr, 178 op/s 2024-09-10T09:49:07.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:49:07 smithi102 ceph-mon[35442]: scrub summary: idle 2024-09-10T09:49:07.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:49:07 smithi102 ceph-mon[35442]: scrub summary: idle 2024-09-10T09:49:08.121 INFO:tasks.workunit.client.0.smithi005.stdout: 1 670977 6.35 MB/sec execute 409 sec latency 779.499 ms 2024-09-10T09:49:09.121 INFO:tasks.workunit.client.0.smithi005.stdout: 1 672673 6.36 MB/sec execute 410 sec latency 23.800 ms 2024-09-10T09:49:09.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:49:09 smithi005 ceph-mon[30350]: pgmap v717: 129 pgs: 129 active+clean; 252 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.5 MiB/s wr, 204 op/s 2024-09-10T09:49:09.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:49:09 smithi159 ceph-mon[35513]: pgmap v717: 129 pgs: 129 active+clean; 252 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.5 MiB/s wr, 204 op/s 2024-09-10T09:49:09.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:49:09 smithi102 ceph-mon[35442]: pgmap v717: 129 pgs: 129 active+clean; 252 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.5 MiB/s wr, 204 op/s 2024-09-10T09:49:10.121 INFO:tasks.workunit.client.0.smithi005.stdout: 1 675174 6.38 MB/sec execute 411 sec latency 10.540 ms 2024-09-10T09:49:11.121 INFO:tasks.workunit.client.0.smithi005.stdout: 1 677322 6.38 MB/sec execute 412 sec latency 22.524 ms 2024-09-10T09:49:11.731 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:49:11 smithi005 ceph-mon[30350]: pgmap v718: 129 pgs: 129 active+clean; 249 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.8 MiB/s wr, 230 op/s 2024-09-10T09:49:11.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:49:11 smithi159 ceph-mon[35513]: pgmap v718: 129 pgs: 129 active+clean; 249 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.8 MiB/s wr, 230 op/s 2024-09-10T09:49:11.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:49:11 smithi102 ceph-mon[35442]: pgmap v718: 129 pgs: 129 active+clean; 249 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.8 MiB/s wr, 230 op/s 2024-09-10T09:49:12.121 INFO:tasks.workunit.client.0.smithi005.stdout: 1 678537 6.39 MB/sec execute 413 sec latency 544.171 ms 2024-09-10T09:49:13.121 INFO:tasks.workunit.client.0.smithi005.stdout: 1 679694 6.39 MB/sec execute 414 sec latency 876.642 ms 2024-09-10T09:49:13.369 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 25.2 2024-09-10T09:49:13.369 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T09:49:13.776 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:49:13.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:49:13 smithi005 ceph-mon[30350]: pgmap v719: 129 pgs: 129 active+clean; 251 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.3 MiB/s wr, 215 op/s 2024-09-10T09:49:13.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:49:13 smithi159 ceph-mon[35513]: pgmap v719: 129 pgs: 129 active+clean; 251 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.3 MiB/s wr, 215 op/s 2024-09-10T09:49:13.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:49:13 smithi102 ceph-mon[35442]: pgmap v719: 129 pgs: 129 active+clean; 251 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.3 MiB/s wr, 215 op/s 2024-09-10T09:49:14.121 INFO:tasks.workunit.client.0.smithi005.stdout: 1 682237 6.41 MB/sec execute 415 sec latency 13.729 ms 2024-09-10T09:49:14.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:14.335+0000 7f09bfe00640 1 -- 172.21.15.5:0/2439808092 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f09c01079c0 msgr2=0x7f09c0107da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:49:14.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:14.335+0000 7f09bfe00640 1 --2- 172.21.15.5:0/2439808092 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f09c01079c0 0x7f09c0107da0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f09b0009930 tx=0x7f09b002a420 comp rx=0 tx=0).stop 2024-09-10T09:49:14.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:14.335+0000 7f09bfe00640 1 -- 172.21.15.5:0/2439808092 shutdown_connections 2024-09-10T09:49:14.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:14.335+0000 7f09bfe00640 1 --2- 172.21.15.5:0/2439808092 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f09c0102360 0x7f09c010e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:14.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:14.335+0000 7f09bfe00640 1 --2- 172.21.15.5:0/2439808092 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f09c01019c0 0x7f09c0101e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:14.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:14.335+0000 7f09bfe00640 1 --2- 172.21.15.5:0/2439808092 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f09c01079c0 0x7f09c0107da0 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:14.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:14.335+0000 7f09bfe00640 1 -- 172.21.15.5:0/2439808092 >> 172.21.15.5:0/2439808092 conn(0x7f09c00fd740 msgr2=0x7f09c00ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:49:14.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:14.335+0000 7f09bfe00640 1 -- 172.21.15.5:0/2439808092 shutdown_connections 2024-09-10T09:49:14.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:14.335+0000 7f09bfe00640 1 -- 172.21.15.5:0/2439808092 wait complete. 2024-09-10T09:49:14.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:14.336+0000 7f09bfe00640 1 Processor -- start 2024-09-10T09:49:14.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:14.337+0000 7f09bfe00640 1 -- start start 2024-09-10T09:49:14.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:14.337+0000 7f09bfe00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f09c01019c0 0x7f09c019a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:49:14.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:14.337+0000 7f09bfe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f09c0102360 0x7f09c019ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:49:14.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:14.337+0000 7f09bfe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f09c01079c0 0x7f09c019ee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:49:14.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:14.337+0000 7f09bfe00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f09c0111dc0 con 0x7f09c01079c0 2024-09-10T09:49:14.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:14.337+0000 7f09bfe00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f09c0111c40 con 0x7f09c0102360 2024-09-10T09:49:14.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:14.337+0000 7f09bfe00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f09c0111f40 con 0x7f09c01019c0 2024-09-10T09:49:14.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:14.337+0000 7f09bea00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f09c01019c0 0x7f09c019a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:49:14.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:14.337+0000 7f09bea00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f09c01019c0 0x7f09c019a5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:54426/0 (socket says 172.21.15.5:54426) 2024-09-10T09:49:14.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:14.337+0000 7f09bea00640 1 -- 172.21.15.5:0/2427206275 learned_addr learned my addr 172.21.15.5:0/2427206275 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:49:14.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:14.337+0000 7f09be000640 1 --2- 172.21.15.5:0/2427206275 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f09c0102360 0x7f09c019ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:49:14.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:14.337+0000 7f09bf400640 1 --2- 172.21.15.5:0/2427206275 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f09c01079c0 0x7f09c019ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:49:14.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:14.338+0000 7f09bea00640 1 -- 172.21.15.5:0/2427206275 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f09c0102360 msgr2=0x7f09c019ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:49:14.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:14.338+0000 7f09bea00640 1 --2- 172.21.15.5:0/2427206275 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f09c0102360 0x7f09c019ab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:14.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:14.338+0000 7f09bea00640 1 -- 172.21.15.5:0/2427206275 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f09c01079c0 msgr2=0x7f09c019ee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:49:14.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:14.338+0000 7f09bea00640 1 --2- 172.21.15.5:0/2427206275 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f09c01079c0 0x7f09c019ee90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:14.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:14.338+0000 7f09bea00640 1 -- 172.21.15.5:0/2427206275 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09c019f570 con 0x7f09c01019c0 2024-09-10T09:49:14.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:14.338+0000 7f09bf400640 1 --2- 172.21.15.5:0/2427206275 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f09c01079c0 0x7f09c019ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:49:14.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:14.338+0000 7f09be000640 1 --2- 172.21.15.5:0/2427206275 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f09c0102360 0x7f09c019ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:49:14.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:14.338+0000 7f09bea00640 1 --2- 172.21.15.5:0/2427206275 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f09c01019c0 0x7f09c019a5c0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f09b0009230 tx=0x7f09b0004040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:49:14.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:14.338+0000 7f09b7400640 1 -- 172.21.15.5:0/2427206275 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09b003f030 con 0x7f09c01019c0 2024-09-10T09:49:14.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:14.339+0000 7f09b7400640 1 -- 172.21.15.5:0/2427206275 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f09b0004440 con 0x7f09c01019c0 2024-09-10T09:49:14.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:14.339+0000 7f09b7400640 1 -- 172.21.15.5:0/2427206275 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09b0035070 con 0x7f09c01019c0 2024-09-10T09:49:14.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:14.339+0000 7f09bfe00640 1 -- 172.21.15.5:0/2427206275 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09c019f800 con 0x7f09c01019c0 2024-09-10T09:49:14.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:14.339+0000 7f09bfe00640 1 -- 172.21.15.5:0/2427206275 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09c01a7140 con 0x7f09c01019c0 2024-09-10T09:49:14.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:14.340+0000 7f09bfe00640 1 -- 172.21.15.5:0/2427206275 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f09c0101e20 con 0x7f09c01019c0 2024-09-10T09:49:14.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:14.342+0000 7f09b7400640 1 -- 172.21.15.5:0/2427206275 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f09b0042030 con 0x7f09c01019c0 2024-09-10T09:49:14.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:14.344+0000 7f09b7400640 1 --2- 172.21.15.5:0/2427206275 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f09a40a2c00 0x7f09a40a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:49:14.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:14.344+0000 7f09b7400640 1 -- 172.21.15.5:0/2427206275 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f09b00ef7b0 con 0x7f09c01019c0 2024-09-10T09:49:14.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:14.344+0000 7f09be000640 1 --2- 172.21.15.5:0/2427206275 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f09a40a2c00 0x7f09a40a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:49:14.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:14.345+0000 7f09be000640 1 --2- 172.21.15.5:0/2427206275 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f09a40a2c00 0x7f09a40a50c0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f09c00764a0 tx=0x7f09ac005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:49:14.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:14.349+0000 7f09b7400640 1 -- 172.21.15.5:0/2427206275 <== mon.1 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f09b0049030 con 0x7f09c01019c0 2024-09-10T09:49:14.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:14.609+0000 7f09bfe00640 1 -- 172.21.15.5:0/2427206275 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f09c01a7310 con 0x7f09c01019c0 2024-09-10T09:49:14.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:14.609+0000 7f09b7400640 1 -- 172.21.15.5:0/2427206275 <== mon.1 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) ==== 94+0+11969 (secure 0 0 0) 0x7f09b00bc2c0 con 0x7f09c01019c0 2024-09-10T09:49:14.611 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-10T09:49:14.611 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":36,"btime":"2024-09-10T09:48:07:349075+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14454,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/2970594311","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2970594311},{"type":"v1","addr":"172.21.15.5:6835","nonce":2970594311}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14514,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6839/3616502174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":3616502174},{"type":"v1","addr":"172.21.15.5:6839","nonce":3616502174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":36,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T09:35:29.626287+0000","modified":"2024-09-10T09:48:06.970127+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24542,"mds_1":24535,"mds_2":24511,"mds_3":24506,"mds_4":24443},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14520":{"gid":14520,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/2073599842","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":2073599842},{"type":"v1","addr":"172.21.15.5:6841","nonce":2073599842}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"k","rank":4,"incarnation":26,"state":"up:active","state_seq":24,"addr":"172.21.15.102:6833/2457007427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":2457007427},{"type":"v1","addr":"172.21.15.102:6833","nonce":2457007427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/356445159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":356445159},{"type":"v1","addr":"172.21.15.159:6833","nonce":356445159}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6837/2894314507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":2894314507},{"type":"v1","addr":"172.21.15.5:6837","nonce":2894314507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.102:6835/1647549111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":1647549111},{"type":"v1","addr":"172.21.15.102:6835","nonce":1647549111}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6835/952657242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":952657242},{"type":"v1","addr":"172.21.15.159:6835","nonce":952657242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":19,"addr":"172.21.15.102:6837/2689776764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":2689776764},{"type":"v1","addr":"172.21.15.102:6837","nonce":2689776764}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.159:6837/620627639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":620627639},{"type":"v1","addr":"172.21.15.159:6837","nonce":620627639}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24535":{"gid":24535,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":18,"addr":"172.21.15.159:6839/2073739599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":2073739599},{"type":"v1","addr":"172.21.15.159:6839","nonce":2073739599}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24542":{"gid":24542,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.102:6839/335409749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6838","nonce":335409749},{"type":"v1","addr":"172.21.15.102:6839","nonce":335409749}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24542,"qdb_cluster":[24542,24535,24511,24506,24443]},"id":1}]} 2024-09-10T09:49:14.611 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 36 2024-09-10T09:49:14.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:14.612+0000 7f09bfe00640 1 -- 172.21.15.5:0/2427206275 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f09a40a2c00 msgr2=0x7f09a40a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:49:14.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:14.612+0000 7f09bfe00640 1 --2- 172.21.15.5:0/2427206275 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f09a40a2c00 0x7f09a40a50c0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f09c00764a0 tx=0x7f09ac005f90 comp rx=0 tx=0).stop 2024-09-10T09:49:14.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:14.612+0000 7f09bfe00640 1 -- 172.21.15.5:0/2427206275 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f09c01019c0 msgr2=0x7f09c019a5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:49:14.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:14.612+0000 7f09bfe00640 1 --2- 172.21.15.5:0/2427206275 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f09c01019c0 0x7f09c019a5c0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f09b0009230 tx=0x7f09b0004040 comp rx=0 tx=0).stop 2024-09-10T09:49:14.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:14.612+0000 7f09bfe00640 1 -- 172.21.15.5:0/2427206275 shutdown_connections 2024-09-10T09:49:14.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:14.612+0000 7f09bfe00640 1 --2- 172.21.15.5:0/2427206275 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f09a40a2c00 0x7f09a40a50c0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:14.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:14.612+0000 7f09bfe00640 1 --2- 172.21.15.5:0/2427206275 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f09c01079c0 0x7f09c019ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:14.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:14.612+0000 7f09bfe00640 1 --2- 172.21.15.5:0/2427206275 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f09c0102360 0x7f09c019ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:14.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:14.612+0000 7f09bfe00640 1 --2- 172.21.15.5:0/2427206275 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f09c01019c0 0x7f09c019a5c0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:14.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:14.612+0000 7f09bfe00640 1 -- 172.21.15.5:0/2427206275 >> 172.21.15.5:0/2427206275 conn(0x7f09c00fd740 msgr2=0x7f09c00fed30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:49:14.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:14.612+0000 7f09bfe00640 1 -- 172.21.15.5:0/2427206275 shutdown_connections 2024-09-10T09:49:14.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:14.612+0000 7f09bfe00640 1 -- 172.21.15.5:0/2427206275 wait complete. 2024-09-10T09:49:15.121 INFO:tasks.workunit.client.0.smithi005.stdout: 1 684275 6.41 MB/sec execute 416 sec latency 24.993 ms 2024-09-10T09:49:15.486 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24542 quiesce db / --timeout 90 --expiration 145.2 --await-for 45.0' 2024-09-10T09:49:15.486 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24542 quiesce db / --timeout 90 --expiration 145.2 --await-for 45.0 2024-09-10T09:49:15.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:49:15 smithi005 ceph-mon[30350]: pgmap v720: 129 pgs: 129 active+clean; 251 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.0 MiB/s wr, 180 op/s 2024-09-10T09:49:15.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:49:15 smithi005 ceph-mon[30350]: from='client.? 172.21.15.5:0/2427206275' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:49:15.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:49:15 smithi159 ceph-mon[35513]: pgmap v720: 129 pgs: 129 active+clean; 251 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.0 MiB/s wr, 180 op/s 2024-09-10T09:49:15.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:49:15 smithi159 ceph-mon[35513]: from='client.? 172.21.15.5:0/2427206275' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:49:15.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:49:15 smithi102 ceph-mon[35442]: pgmap v720: 129 pgs: 129 active+clean; 251 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.0 MiB/s wr, 180 op/s 2024-09-10T09:49:15.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:49:15 smithi102 ceph-mon[35442]: from='client.? 172.21.15.5:0/2427206275' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:49:15.910 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:49:16.121 INFO:tasks.workunit.client.0.smithi005.stdout: 1 686144 6.42 MB/sec execute 417 sec latency 21.813 ms 2024-09-10T09:49:16.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:16.536+0000 7ff70e200640 1 -- 172.21.15.5:0/3734662735 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff708076720 msgr2=0x7ff708076b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:49:16.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:16.536+0000 7ff70e200640 1 --2- 172.21.15.5:0/3734662735 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff708076720 0x7ff708076b00 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7ff6fc009900 tx=0x7ff6fc02a3f0 comp rx=0 tx=0).stop 2024-09-10T09:49:16.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:16.536+0000 7ff70e200640 1 -- 172.21.15.5:0/3734662735 shutdown_connections 2024-09-10T09:49:16.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:16.536+0000 7ff70e200640 1 --2- 172.21.15.5:0/3734662735 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff708112870 0x7ff708114c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:16.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:16.536+0000 7ff70e200640 1 --2- 172.21.15.5:0/3734662735 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff708077040 0x7ff708112330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:16.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:16.536+0000 7ff70e200640 1 --2- 172.21.15.5:0/3734662735 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff708076720 0x7ff708076b00 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:16.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:16.536+0000 7ff70e200640 1 -- 172.21.15.5:0/3734662735 >> 172.21.15.5:0/3734662735 conn(0x7ff7080ff8d0 msgr2=0x7ff708101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:49:16.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:16.536+0000 7ff70e200640 1 -- 172.21.15.5:0/3734662735 shutdown_connections 2024-09-10T09:49:16.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:16.536+0000 7ff70e200640 1 -- 172.21.15.5:0/3734662735 wait complete. 2024-09-10T09:49:16.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:16.537+0000 7ff70e200640 1 Processor -- start 2024-09-10T09:49:16.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:16.537+0000 7ff70e200640 1 -- start start 2024-09-10T09:49:16.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:16.538+0000 7ff70e200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff708076720 0x7ff7081a4640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:49:16.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:16.538+0000 7ff70e200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff708077040 0x7ff7081a4b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:49:16.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:16.538+0000 7ff70e200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff708112870 0x7ff70819e710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:49:16.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:16.538+0000 7ff70e200640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff708118450 con 0x7ff708076720 2024-09-10T09:49:16.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:16.538+0000 7ff70e200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7ff7081182d0 con 0x7ff708077040 2024-09-10T09:49:16.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:16.538+0000 7ff70e200640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7ff7081185d0 con 0x7ff708112870 2024-09-10T09:49:16.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:16.538+0000 7ff70ce00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff708076720 0x7ff7081a4640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:49:16.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:16.538+0000 7ff70ce00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff708076720 0x7ff7081a4640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47838/0 (socket says 172.21.15.5:47838) 2024-09-10T09:49:16.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:16.538+0000 7ff70ce00640 1 -- 172.21.15.5:0/3941446753 learned_addr learned my addr 172.21.15.5:0/3941446753 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:49:16.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:16.538+0000 7ff707e00640 1 --2- 172.21.15.5:0/3941446753 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff708077040 0x7ff7081a4b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:49:16.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:16.538+0000 7ff70d800640 1 --2- 172.21.15.5:0/3941446753 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff708112870 0x7ff70819e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:49:16.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:16.539+0000 7ff707e00640 1 -- 172.21.15.5:0/3941446753 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff708112870 msgr2=0x7ff70819e710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:49:16.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:16.539+0000 7ff707e00640 1 --2- 172.21.15.5:0/3941446753 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff708112870 0x7ff70819e710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:16.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:16.539+0000 7ff707e00640 1 -- 172.21.15.5:0/3941446753 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff708076720 msgr2=0x7ff7081a4640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:49:16.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:16.539+0000 7ff707e00640 1 --2- 172.21.15.5:0/3941446753 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff708076720 0x7ff7081a4640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:16.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:16.539+0000 7ff707e00640 1 -- 172.21.15.5:0/3941446753 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff70819ef80 con 0x7ff708077040 2024-09-10T09:49:16.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:16.539+0000 7ff70d800640 1 --2- 172.21.15.5:0/3941446753 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff708112870 0x7ff70819e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:49:16.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:16.539+0000 7ff707e00640 1 --2- 172.21.15.5:0/3941446753 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff708077040 0x7ff7081a4b80 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7ff6f400e9f0 tx=0x7ff6f400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:49:16.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:16.539+0000 7ff705600640 1 -- 172.21.15.5:0/3941446753 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6f400ccc0 con 0x7ff708077040 2024-09-10T09:49:16.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:16.539+0000 7ff70e200640 1 -- 172.21.15.5:0/3941446753 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff70819f270 con 0x7ff708077040 2024-09-10T09:49:16.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:16.539+0000 7ff705600640 1 -- 172.21.15.5:0/3941446753 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff6f4005020 con 0x7ff708077040 2024-09-10T09:49:16.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:16.540+0000 7ff705600640 1 -- 172.21.15.5:0/3941446753 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6f40049e0 con 0x7ff708077040 2024-09-10T09:49:16.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:16.540+0000 7ff70e200640 1 -- 172.21.15.5:0/3941446753 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7081ab5c0 con 0x7ff708077040 2024-09-10T09:49:16.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:16.543+0000 7ff705600640 1 -- 172.21.15.5:0/3941446753 <== mon.2 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7ff6f4005560 con 0x7ff708077040 2024-09-10T09:49:16.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:16.543+0000 7ff71245a740 1 Processor -- start 2024-09-10T09:49:16.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:16.543+0000 7ff71245a740 1 -- start start 2024-09-10T09:49:16.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:16.543+0000 7ff71245a740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e7ca35b560 0x55e7ca35b940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:49:16.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:16.543+0000 7ff71245a740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e7ca35be80 0x55e7ca366200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:49:16.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:16.543+0000 7ff71245a740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e7ca366740 0x55e7ca368b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:49:16.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:16.543+0000 7ff71245a740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55e7ca2c1530 con 0x55e7ca35b560 2024-09-10T09:49:16.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:16.543+0000 7ff71245a740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55e7ca268860 con 0x55e7ca35be80 2024-09-10T09:49:16.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:16.543+0000 7ff71245a740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55e7ca2bd4f0 con 0x55e7ca366740 2024-09-10T09:49:16.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:16.544+0000 7ff70ce00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e7ca35be80 0x55e7ca366200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:49:16.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:16.544+0000 7ff707e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e7ca366740 0x55e7ca368b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:49:16.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:16.544+0000 7ff70ce00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e7ca35be80 0x55e7ca366200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:34074/0 (socket says 172.21.15.5:34074) 2024-09-10T09:49:16.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:16.544+0000 7ff70ce00640 1 -- 172.21.15.5:0/975263007 learned_addr learned my addr 172.21.15.5:0/975263007 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:49:16.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:16.544+0000 7ff70d800640 1 --2- 172.21.15.5:0/975263007 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e7ca35b560 0x55e7ca35b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:49:16.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:16.544+0000 7ff70ce00640 1 -- 172.21.15.5:0/975263007 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e7ca366740 msgr2=0x55e7ca368b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:49:16.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:16.544+0000 7ff70ce00640 1 --2- 172.21.15.5:0/975263007 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e7ca366740 0x55e7ca368b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:16.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:16.544+0000 7ff70ce00640 1 -- 172.21.15.5:0/975263007 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e7ca35b560 msgr2=0x55e7ca35b940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:49:16.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:16.544+0000 7ff70ce00640 1 --2- 172.21.15.5:0/975263007 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e7ca35b560 0x55e7ca35b940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:16.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:16.544+0000 7ff70ce00640 1 -- 172.21.15.5:0/975263007 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e7ca032b90 con 0x55e7ca35be80 2024-09-10T09:49:16.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:16.544+0000 7ff705600640 1 --2- 172.21.15.5:0/3941446753 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7ff6f00a2b80 0x7ff6f00a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:49:16.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:16.544+0000 7ff70ce00640 1 --2- 172.21.15.5:0/975263007 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e7ca35be80 0x55e7ca366200 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7ff6fc005dd0 tx=0x7ff6fc005c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e9ce45b923519470 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:49:16.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:16.545+0000 7ff705600640 1 -- 172.21.15.5:0/3941446753 <== mon.2 v2:172.21.15.102:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7ff6f40cf400 con 0x7ff708077040 2024-09-10T09:49:16.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:16.545+0000 7ff70d800640 1 --2- 172.21.15.5:0/3941446753 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7ff6f00a2b80 0x7ff6f00a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:49:16.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:16.545+0000 7ff6fa000640 1 -- 172.21.15.5:0/975263007 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6fc040070 con 0x55e7ca35be80 2024-09-10T09:49:16.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:16.545+0000 7ff6fa000640 1 -- 172.21.15.5:0/975263007 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff6fc004400 con 0x55e7ca35be80 2024-09-10T09:49:16.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:16.545+0000 7ff6fa000640 1 -- 172.21.15.5:0/975263007 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6fc035040 con 0x55e7ca35be80 2024-09-10T09:49:16.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:16.545+0000 7ff71245a740 1 -- 172.21.15.5:0/975263007 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e7ca35be80 msgr2=0x55e7ca366200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:49:16.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:16.545+0000 7ff71245a740 1 --2- 172.21.15.5:0/975263007 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e7ca35be80 0x55e7ca366200 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7ff6fc005dd0 tx=0x7ff6fc005c00 comp rx=0 tx=0).stop 2024-09-10T09:49:16.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:16.546+0000 7ff70d800640 1 --2- 172.21.15.5:0/3941446753 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7ff6f00a2b80 0x7ff6f00a5040 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7ff70806b3d0 tx=0x7ff700009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:49:16.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:16.546+0000 7ff71245a740 1 -- 172.21.15.5:0/975263007 shutdown_connections 2024-09-10T09:49:16.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:16.546+0000 7ff71245a740 1 --2- 172.21.15.5:0/975263007 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e7ca366740 0x55e7ca368b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:16.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:16.546+0000 7ff71245a740 1 --2- 172.21.15.5:0/975263007 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e7ca35be80 0x55e7ca366200 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:16.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:16.546+0000 7ff71245a740 1 --2- 172.21.15.5:0/975263007 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e7ca35b560 0x55e7ca35b940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:16.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:16.546+0000 7ff71245a740 1 -- 172.21.15.5:0/975263007 >> 172.21.15.5:0/975263007 conn(0x55e7ca357af0 msgr2=0x55e7ca357ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:49:16.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:16.546+0000 7ff71245a740 1 -- 172.21.15.5:0/975263007 shutdown_connections 2024-09-10T09:49:16.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:16.546+0000 7ff71245a740 1 -- 172.21.15.5:0/975263007 wait complete. 2024-09-10T09:49:16.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:16.546+0000 7ff71245a740 1 -- start start 2024-09-10T09:49:16.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:16.547+0000 7ff71245a740 1 Processor -- start 2024-09-10T09:49:16.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:16.547+0000 7ff71245a740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e7ca35b560 0x55e7ca366200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:49:16.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:16.547+0000 7ff71245a740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e7ca35be80 0x55e7ca363550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:49:16.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:16.547+0000 7ff71245a740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e7ca366740 0x55e7ca363ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:49:16.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:16.547+0000 7ff71245a740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55e7ca369310 con 0x55e7ca35b560 2024-09-10T09:49:16.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:16.547+0000 7ff71245a740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55e7ca27be60 con 0x55e7ca366740 2024-09-10T09:49:16.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:16.547+0000 7ff71245a740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55e7ca369490 con 0x55e7ca35be80 2024-09-10T09:49:16.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:16.547+0000 7ff71245a740 10 client.0 _ll_register_callbacks cb 0x55e7ca280cc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:49:16.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:16.547+0000 7ff70ce00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e7ca35b560 0x55e7ca366200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:49:16.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:16.547+0000 7ff707e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e7ca35be80 0x55e7ca363550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:49:16.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:16.547+0000 7ff70d800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e7ca366740 0x55e7ca363ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:49:16.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:16.547+0000 7ff70ce00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e7ca35b560 0x55e7ca366200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47856/0 (socket says 172.21.15.5:47856) 2024-09-10T09:49:16.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:16.547+0000 7ff707e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e7ca35be80 0x55e7ca363550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:54454/0 (socket says 172.21.15.5:54454) 2024-09-10T09:49:16.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:16.547+0000 7ff70ce00640 1 -- 172.21.15.5:0/633619592 learned_addr learned my addr 172.21.15.5:0/633619592 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:49:16.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:16.548+0000 7ff70d800640 1 -- 172.21.15.5:0/633619592 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e7ca35be80 msgr2=0x55e7ca363550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:49:16.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:16.548+0000 7ff70d800640 1 --2- 172.21.15.5:0/633619592 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e7ca35be80 0x55e7ca363550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:16.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:16.548+0000 7ff70d800640 1 -- 172.21.15.5:0/633619592 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e7ca35b560 msgr2=0x55e7ca366200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:49:16.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:16.548+0000 7ff70d800640 1 --2- 172.21.15.5:0/633619592 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e7ca35b560 0x55e7ca366200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:16.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:16.548+0000 7ff70d800640 1 -- 172.21.15.5:0/633619592 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e7ca287990 con 0x55e7ca366740 2024-09-10T09:49:16.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:16.548+0000 7ff70d800640 1 --2- 172.21.15.5:0/633619592 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e7ca366740 0x55e7ca363ce0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7ff700005fd0 tx=0x7ff700004ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=949131593e3cfe4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:49:16.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:16.548+0000 7ff6eea00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-09-10T09:49:16.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:16.548+0000 7ff6eea00640 1 -- 172.21.15.5:0/633619592 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7000045c0 con 0x55e7ca366740 2024-09-10T09:49:16.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:16.548+0000 7ff71245a740 1 -- 172.21.15.5:0/633619592 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff6fc004400 con 0x55e7ca366740 2024-09-10T09:49:16.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:16.548+0000 7ff6eea00640 1 -- 172.21.15.5:0/633619592 <== mon.2 v2:172.21.15.102:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7ff700004760 con 0x55e7ca366740 2024-09-10T09:49:16.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:16.549+0000 7ff6eea00640 1 -- 172.21.15.5:0/633619592 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7ff70002e580 con 0x55e7ca366740 2024-09-10T09:49:16.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:16.549+0000 7ff71245a740 10 client.27569 fetch_fsmap learned FSMap version 36 2024-09-10T09:49:16.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:16.549+0000 7ff71245a740 1 -- 172.21.15.5:0/633619592 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55e7ca374f20 con 0x55e7ca366740 2024-09-10T09:49:16.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:16.549+0000 7ff6eea00640 1 -- 172.21.15.5:0/633619592 <== mon.2 v2:172.21.15.102:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff6fc004400 con 0x55e7ca366740 2024-09-10T09:49:16.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:16.549+0000 7ff6eea00640 1 -- 172.21.15.5:0/633619592 <== mon.2 v2:172.21.15.102:3300/0 5 ==== fsmap(e 36) ==== 8370+0+0 (secure 0 0 0) 0x7ff700027070 con 0x55e7ca366740 2024-09-10T09:49:16.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:16.550+0000 7ff71245a740 10 client.27569 fetch_fsmap finished waiting for FSMap version 36 2024-09-10T09:49:16.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:16.550+0000 7ff71245a740 10 client.27569 resolve_mds: validated gid 24542 aka daemon mds.b 2024-09-10T09:49:16.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:16.550+0000 7ff71245a740 20 client.27569 populate_metadata read hostname 'smithi005' 2024-09-10T09:49:16.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:16.550+0000 7ff71245a740 1 --2- 172.21.15.5:0/633619592 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55e7ca376330 0x55e7ca378740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:49:16.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:16.550+0000 7ff71245a740 4 client.27569 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:49:16.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:16.550+0000 7ff71245a740 1 -- 172.21.15.5:0/633619592 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e7ca264310 con 0x55e7ca376330 2024-09-10T09:49:16.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:16.550+0000 7ff70ce00640 1 --2- 172.21.15.5:0/633619592 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55e7ca376330 0x55e7ca378740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:49:16.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:16.550+0000 7ff70ce00640 1 --2- 172.21.15.5:0/633619592 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55e7ca376330 0x55e7ca378740 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bc335a5c2e18427a server_cookie=7fb565f6bda1ff2 in_seq=0 out_seq=0 2024-09-10T09:49:16.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:16.550+0000 7ff6eea00640 10 client.27569 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T09:49:16.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:16.551+0000 7ff6eea00640 1 -- 172.21.15.5:0/633619592 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e7ca264310 con 0x55e7ca376330 2024-09-10T09:49:16.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:16.551+0000 7ff6eea00640 10 client.27569 handle_command_reply: tid=0 2024-09-10T09:49:16.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:16.551+0000 7ff71245a740 1 client.27569 shutdown 2024-09-10T09:49:16.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:16.552+0000 7ff71245a740 1 -- 172.21.15.5:0/633619592 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e7ca366740 msgr2=0x55e7ca363ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:49:16.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:16.552+0000 7ff71245a740 1 --2- 172.21.15.5:0/633619592 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e7ca366740 0x55e7ca363ce0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7ff700005fd0 tx=0x7ff700004ab0 comp rx=0 tx=0).stop 2024-09-10T09:49:16.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:16.552+0000 7ff71245a740 1 -- 172.21.15.5:0/633619592 shutdown_connections 2024-09-10T09:49:16.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:16.552+0000 7ff71245a740 1 --2- 172.21.15.5:0/633619592 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55e7ca376330 0x55e7ca378740 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:16.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:16.552+0000 7ff71245a740 1 --2- 172.21.15.5:0/633619592 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e7ca366740 0x55e7ca363ce0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:16.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:16.552+0000 7ff71245a740 1 --2- 172.21.15.5:0/633619592 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e7ca35be80 0x55e7ca363550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:16.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:16.552+0000 7ff71245a740 1 --2- 172.21.15.5:0/633619592 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e7ca35b560 0x55e7ca366200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:16.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:16.552+0000 7ff6eea00640 0 client.27569 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T09:49:16.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:16.552+0000 7ff71245a740 1 -- 172.21.15.5:0/633619592 >> 172.21.15.5:0/633619592 conn(0x55e7ca357af0 msgr2=0x55e7ca35d820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:49:16.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:16.552+0000 7ff71245a740 1 -- 172.21.15.5:0/633619592 shutdown_connections 2024-09-10T09:49:16.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:16.552+0000 7ff71245a740 1 -- 172.21.15.5:0/633619592 wait complete. 2024-09-10T09:49:16.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:16.552+0000 7ff71245a740 20 client.27569 trim_cache size 0 max 16384 2024-09-10T09:49:16.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:16.571+0000 7ff71245a740 1 Processor -- start 2024-09-10T09:49:16.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:16.571+0000 7ff71245a740 1 -- start start 2024-09-10T09:49:16.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:16.571+0000 7ff71245a740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e7ca35b560 0x55e7ca352d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:49:16.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:16.571+0000 7ff71245a740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e7ca35be80 0x55e7ca3653b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:49:16.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:16.571+0000 7ff71245a740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e7ca366740 0x55e7ca37e010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:49:16.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:16.571+0000 7ff71245a740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff7081b5300 con 0x55e7ca35be80 2024-09-10T09:49:16.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:16.571+0000 7ff71245a740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7ff7081b6e40 con 0x55e7ca366740 2024-09-10T09:49:16.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:16.571+0000 7ff71245a740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7ff7081b59d0 con 0x55e7ca35b560 2024-09-10T09:49:16.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:16.572+0000 7ff707e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e7ca35be80 0x55e7ca3653b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:49:16.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:16.572+0000 7ff70ce00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e7ca35b560 0x55e7ca352d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:49:16.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:16.572+0000 7ff70d800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e7ca366740 0x55e7ca37e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:49:16.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:16.572+0000 7ff707e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e7ca35be80 0x55e7ca3653b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47870/0 (socket says 172.21.15.5:47870) 2024-09-10T09:49:16.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:16.572+0000 7ff707e00640 1 -- 172.21.15.5:0/395607056 learned_addr learned my addr 172.21.15.5:0/395607056 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:49:16.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:16.572+0000 7ff70d800640 1 -- 172.21.15.5:0/395607056 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e7ca35b560 msgr2=0x55e7ca352d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:49:16.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:16.572+0000 7ff70d800640 1 --2- 172.21.15.5:0/395607056 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e7ca35b560 0x55e7ca352d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:16.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:16.572+0000 7ff70d800640 1 -- 172.21.15.5:0/395607056 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e7ca35be80 msgr2=0x55e7ca3653b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:49:16.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:16.572+0000 7ff70d800640 1 --2- 172.21.15.5:0/395607056 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e7ca35be80 0x55e7ca3653b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:16.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:16.572+0000 7ff70d800640 1 -- 172.21.15.5:0/395607056 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e7ca353310 con 0x55e7ca366740 2024-09-10T09:49:16.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:16.572+0000 7ff707e00640 1 --2- 172.21.15.5:0/395607056 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e7ca35be80 0x55e7ca3653b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:49:16.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:16.572+0000 7ff70d800640 1 --2- 172.21.15.5:0/395607056 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e7ca366740 0x55e7ca37e010 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7ff700030040 tx=0x7ff700012fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5286c1c2d01db09 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:49:16.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:16.573+0000 7ff6f9600640 1 -- 172.21.15.5:0/395607056 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff70001f480 con 0x55e7ca366740 2024-09-10T09:49:16.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:16.573+0000 7ff6f9600640 1 -- 172.21.15.5:0/395607056 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff70002e720 con 0x55e7ca366740 2024-09-10T09:49:16.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:16.573+0000 7ff6f9600640 1 -- 172.21.15.5:0/395607056 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7000296a0 con 0x55e7ca366740 2024-09-10T09:49:16.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:16.573+0000 7ff71245a740 1 -- 172.21.15.5:0/395607056 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e7ca366740 msgr2=0x55e7ca37e010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:49:16.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:16.573+0000 7ff71245a740 1 --2- 172.21.15.5:0/395607056 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e7ca366740 0x55e7ca37e010 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7ff700030040 tx=0x7ff700012fd0 comp rx=0 tx=0).stop 2024-09-10T09:49:16.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:16.574+0000 7ff71245a740 1 -- 172.21.15.5:0/395607056 shutdown_connections 2024-09-10T09:49:16.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:16.574+0000 7ff71245a740 1 --2- 172.21.15.5:0/395607056 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e7ca366740 0x55e7ca37e010 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:16.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:16.574+0000 7ff71245a740 1 --2- 172.21.15.5:0/395607056 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e7ca35be80 0x55e7ca3653b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:16.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:16.574+0000 7ff71245a740 1 --2- 172.21.15.5:0/395607056 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e7ca35b560 0x55e7ca352d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:16.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:16.574+0000 7ff71245a740 1 -- 172.21.15.5:0/395607056 >> 172.21.15.5:0/395607056 conn(0x55e7ca357af0 msgr2=0x55e7ca364ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:49:16.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:16.574+0000 7ff71245a740 1 -- 172.21.15.5:0/395607056 shutdown_connections 2024-09-10T09:49:16.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:16.574+0000 7ff71245a740 1 -- 172.21.15.5:0/395607056 wait complete. 2024-09-10T09:49:16.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:16.574+0000 7ff71245a740 1 -- start start 2024-09-10T09:49:16.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:16.574+0000 7ff71245a740 1 Processor -- start 2024-09-10T09:49:16.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:16.575+0000 7ff71245a740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e7ca35b560 0x55e7ca36aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:49:16.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:16.575+0000 7ff71245a740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e7ca35be80 0x55e7ca36afe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:49:16.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:16.575+0000 7ff71245a740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e7ca366740 0x55e7ca3dffe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:49:16.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:16.575+0000 7ff71245a740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff7081b6770 con 0x55e7ca35be80 2024-09-10T09:49:16.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:16.575+0000 7ff71245a740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7ff7081b60a0 con 0x55e7ca35b560 2024-09-10T09:49:16.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:16.575+0000 7ff71245a740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7ff7081b32e0 con 0x55e7ca366740 2024-09-10T09:49:16.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:16.575+0000 7ff71245a740 10 client.0 _ll_register_callbacks cb 0x7ff700012570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:49:16.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:16.575+0000 7ff70ce00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e7ca35b560 0x55e7ca36aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:49:16.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:16.575+0000 7ff70ce00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e7ca35b560 0x55e7ca36aaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:34110/0 (socket says 172.21.15.5:34110) 2024-09-10T09:49:16.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:16.575+0000 7ff70ce00640 1 -- 172.21.15.5:0/2600732962 learned_addr learned my addr 172.21.15.5:0/2600732962 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:49:16.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:16.575+0000 7ff70d800640 1 --2- 172.21.15.5:0/2600732962 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e7ca366740 0x55e7ca3dffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:49:16.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:16.575+0000 7ff707e00640 1 --2- 172.21.15.5:0/2600732962 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e7ca35be80 0x55e7ca36afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:49:16.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:16.576+0000 7ff70ce00640 1 -- 172.21.15.5:0/2600732962 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e7ca366740 msgr2=0x55e7ca3dffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:49:16.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:16.576+0000 7ff70ce00640 1 --2- 172.21.15.5:0/2600732962 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e7ca366740 0x55e7ca3dffe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:16.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:16.576+0000 7ff70ce00640 1 -- 172.21.15.5:0/2600732962 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e7ca35be80 msgr2=0x55e7ca36afe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:49:16.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:16.576+0000 7ff70ce00640 1 --2- 172.21.15.5:0/2600732962 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e7ca35be80 0x55e7ca36afe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:16.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:16.576+0000 7ff70ce00640 1 -- 172.21.15.5:0/2600732962 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e7ca366570 con 0x55e7ca35b560 2024-09-10T09:49:16.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:16.576+0000 7ff70d800640 1 --2- 172.21.15.5:0/2600732962 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e7ca366740 0x55e7ca3dffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:49:16.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:16.576+0000 7ff707e00640 1 --2- 172.21.15.5:0/2600732962 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e7ca35be80 0x55e7ca36afe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:49:16.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:16.576+0000 7ff70ce00640 1 --2- 172.21.15.5:0/2600732962 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e7ca35b560 0x55e7ca36aaa0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7ff6fc0096c0 tx=0x7ff6fc004d50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=811ed7111afa2de7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:49:16.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:16.576+0000 7ff6eea00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-09-10T09:49:16.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:16.576+0000 7ff6eea00640 1 -- 172.21.15.5:0/2600732962 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6fc040040 con 0x55e7ca35b560 2024-09-10T09:49:16.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:16.576+0000 7ff6eea00640 1 -- 172.21.15.5:0/2600732962 <== mon.2 v2:172.21.15.102:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7ff6fc0048c0 con 0x55e7ca35b560 2024-09-10T09:49:16.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:16.576+0000 7ff71245a740 1 -- 172.21.15.5:0/2600732962 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff70002e720 con 0x55e7ca35b560 2024-09-10T09:49:16.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:16.577+0000 7ff6eea00640 1 -- 172.21.15.5:0/2600732962 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7ff6fc060050 con 0x55e7ca35b560 2024-09-10T09:49:16.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:16.577+0000 7ff71245a740 10 client.27575 fetch_fsmap learned FSMap version 36 2024-09-10T09:49:16.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:16.577+0000 7ff71245a740 1 -- 172.21.15.5:0/2600732962 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55e7ca3e1530 con 0x55e7ca35b560 2024-09-10T09:49:16.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:16.577+0000 7ff6eea00640 1 -- 172.21.15.5:0/2600732962 <== mon.2 v2:172.21.15.102:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff70002e720 con 0x55e7ca35b560 2024-09-10T09:49:16.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:16.577+0000 7ff6eea00640 1 -- 172.21.15.5:0/2600732962 <== mon.2 v2:172.21.15.102:3300/0 5 ==== fsmap(e 36) ==== 8370+0+0 (secure 0 0 0) 0x7ff6fc04e070 con 0x55e7ca35b560 2024-09-10T09:49:16.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:16.577+0000 7ff71245a740 10 client.27575 fetch_fsmap finished waiting for FSMap version 36 2024-09-10T09:49:16.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:16.577+0000 7ff71245a740 10 client.27575 resolve_mds: validated gid 24542 aka daemon mds.b 2024-09-10T09:49:16.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:16.577+0000 7ff71245a740 20 client.27575 populate_metadata read hostname 'smithi005' 2024-09-10T09:49:16.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:16.577+0000 7ff71245a740 1 --2- 172.21.15.5:0/2600732962 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55e7ca3e1870 0x55e7ca3e3c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:49:16.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:16.577+0000 7ff71245a740 4 client.27575 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 145.2, "await_for": 45.0}] 2024-09-10T09:49:16.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:16.577+0000 7ff71245a740 1 -- 172.21.15.5:0/2600732962 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 145.2, "await_for": 45.0}) -- 0x55e7ca2cec60 con 0x55e7ca3e1870 2024-09-10T09:49:16.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:16.577+0000 7ff707e00640 1 --2- 172.21.15.5:0/2600732962 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55e7ca3e1870 0x55e7ca3e3c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:49:16.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:16.578+0000 7ff707e00640 1 --2- 172.21.15.5:0/2600732962 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55e7ca3e1870 0x55e7ca3e3c60 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f3c7e9f2c484c570 server_cookie=e6bcac3dbeb10117 in_seq=0 out_seq=0 2024-09-10T09:49:16.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:16.578+0000 7ff6eea00640 10 client.27575 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T09:49:16.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:16.642+0000 7ff6eea00640 1 -- 172.21.15.5:0/2600732962 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55e7ca2cec60 con 0x55e7ca3e1870 2024-09-10T09:49:16.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:16.642+0000 7ff6eea00640 10 client.27575 handle_command_reply: tid=0 2024-09-10T09:49:16.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:16.642+0000 7ff71245a740 1 client.27575 shutdown 2024-09-10T09:49:16.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:16.642+0000 7ff71245a740 1 -- 172.21.15.5:0/2600732962 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e7ca35b560 msgr2=0x55e7ca36aaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:49:16.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:16.642+0000 7ff71245a740 1 --2- 172.21.15.5:0/2600732962 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e7ca35b560 0x55e7ca36aaa0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7ff6fc0096c0 tx=0x7ff6fc004d50 comp rx=0 tx=0).stop 2024-09-10T09:49:16.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:16.642+0000 7ff71245a740 1 -- 172.21.15.5:0/2600732962 shutdown_connections 2024-09-10T09:49:16.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:16.642+0000 7ff71245a740 1 --2- 172.21.15.5:0/2600732962 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55e7ca3e1870 0x55e7ca3e3c60 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:16.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:16.642+0000 7ff71245a740 1 --2- 172.21.15.5:0/2600732962 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e7ca366740 0x55e7ca3dffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:16.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:16.642+0000 7ff71245a740 1 --2- 172.21.15.5:0/2600732962 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e7ca35be80 0x55e7ca36afe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:16.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:16.642+0000 7ff6eea00640 0 client.27575 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T09:49:16.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:16.642+0000 7ff71245a740 1 --2- 172.21.15.5:0/2600732962 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e7ca35b560 0x55e7ca36aaa0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:16.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:16.642+0000 7ff71245a740 1 -- 172.21.15.5:0/2600732962 >> 172.21.15.5:0/2600732962 conn(0x55e7ca357af0 msgr2=0x55e7ca356450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:49:16.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:16.643+0000 7ff71245a740 1 -- 172.21.15.5:0/2600732962 shutdown_connections 2024-09-10T09:49:16.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:16.643+0000 7ff71245a740 1 -- 172.21.15.5:0/2600732962 wait complete. 2024-09-10T09:49:16.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:16.643+0000 7ff71245a740 20 client.27575 trim_cache size 0 max 16384 2024-09-10T09:49:16.645 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-10T09:49:16.646 INFO:teuthology.orchestra.run.smithi005.stdout: "epoch": 36, 2024-09-10T09:49:16.646 INFO:teuthology.orchestra.run.smithi005.stdout: "leader": 24542, 2024-09-10T09:49:16.646 INFO:teuthology.orchestra.run.smithi005.stdout: "set_version": 38, 2024-09-10T09:49:16.646 INFO:teuthology.orchestra.run.smithi005.stdout: "sets": { 2024-09-10T09:49:16.646 INFO:teuthology.orchestra.run.smithi005.stdout: "a33fa5fe": { 2024-09-10T09:49:16.646 INFO:teuthology.orchestra.run.smithi005.stdout: "version": 38, 2024-09-10T09:49:16.646 INFO:teuthology.orchestra.run.smithi005.stdout: "age_ref": 0.0, 2024-09-10T09:49:16.646 INFO:teuthology.orchestra.run.smithi005.stdout: "state": { 2024-09-10T09:49:16.646 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "QUIESCED", 2024-09-10T09:49:16.646 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0 2024-09-10T09:49:16.646 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T09:49:16.646 INFO:teuthology.orchestra.run.smithi005.stdout: "timeout": 90.0, 2024-09-10T09:49:16.646 INFO:teuthology.orchestra.run.smithi005.stdout: "expiration": 145.2, 2024-09-10T09:49:16.646 INFO:teuthology.orchestra.run.smithi005.stdout: "members": { 2024-09-10T09:49:16.647 INFO:teuthology.orchestra.run.smithi005.stdout: "file:/": { 2024-09-10T09:49:16.647 INFO:teuthology.orchestra.run.smithi005.stdout: "excluded": false, 2024-09-10T09:49:16.647 INFO:teuthology.orchestra.run.smithi005.stdout: "state": { 2024-09-10T09:49:16.647 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "QUIESCED", 2024-09-10T09:49:16.647 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0 2024-09-10T09:49:16.647 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T09:49:16.647 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T09:49:16.647 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T09:49:16.647 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T09:49:16.647 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T09:49:16.647 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-10T09:49:16.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:16.643+0000 7ff70e200640 1 -- 172.21.15.5:0/3941446753 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7ff6f00a2b80 msgr2=0x7ff6f00a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:49:16.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:16.643+0000 7ff70e200640 1 --2- 172.21.15.5:0/3941446753 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7ff6f00a2b80 0x7ff6f00a5040 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7ff70806b3d0 tx=0x7ff700009500 comp rx=0 tx=0).stop 2024-09-10T09:49:16.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:16.643+0000 7ff70e200640 1 -- 172.21.15.5:0/3941446753 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff708077040 msgr2=0x7ff7081a4b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:49:16.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:16.643+0000 7ff70e200640 1 --2- 172.21.15.5:0/3941446753 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff708077040 0x7ff7081a4b80 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7ff6f400e9f0 tx=0x7ff6f400eeb0 comp rx=0 tx=0).stop 2024-09-10T09:49:16.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:16.643+0000 7ff70e200640 1 -- 172.21.15.5:0/3941446753 shutdown_connections 2024-09-10T09:49:16.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:16.643+0000 7ff70e200640 1 --2- 172.21.15.5:0/3941446753 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7ff6f00a2b80 0x7ff6f00a5040 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:16.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:16.643+0000 7ff70e200640 1 --2- 172.21.15.5:0/3941446753 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff708112870 0x7ff70819e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:16.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:16.643+0000 7ff70e200640 1 --2- 172.21.15.5:0/3941446753 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff708077040 0x7ff7081a4b80 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:16.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:16.643+0000 7ff70e200640 1 --2- 172.21.15.5:0/3941446753 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff708076720 0x7ff7081a4640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:16.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:16.644+0000 7ff70e200640 1 -- 172.21.15.5:0/3941446753 >> 172.21.15.5:0/3941446753 conn(0x7ff7080ff8d0 msgr2=0x7ff708101cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:49:16.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:16.644+0000 7ff70e200640 1 -- 172.21.15.5:0/3941446753 shutdown_connections 2024-09-10T09:49:16.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:16.644+0000 7ff70e200640 1 -- 172.21.15.5:0/3941446753 wait complete. 2024-09-10T09:49:17.124 INFO:tasks.workunit.client.0.smithi005.stdout: 1 687246 6.42 MB/sec execute 418 sec latency 484.981 ms 2024-09-10T09:49:17.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:49:17 smithi005 ceph-mon[30350]: pgmap v721: 129 pgs: 129 active+clean; 251 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 1.8 MiB/s wr, 197 op/s 2024-09-10T09:49:17.738 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'a33fa5fe', quiesce took 4.4 seconds. Will release after: 20.799999999999997 2024-09-10T09:49:17.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:49:17 smithi159 ceph-mon[35513]: pgmap v721: 129 pgs: 129 active+clean; 251 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 1.8 MiB/s wr, 197 op/s 2024-09-10T09:49:17.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:49:17 smithi102 ceph-mon[35442]: pgmap v721: 129 pgs: 129 active+clean; 251 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 1.8 MiB/s wr, 197 op/s 2024-09-10T09:49:18.122 INFO:tasks.workunit.client.0.smithi005.stdout: 1 687246 6.41 MB/sec execute 419 sec latency 1485.095 ms 2024-09-10T09:49:19.122 INFO:tasks.workunit.client.0.smithi005.stdout: 1 687246 6.39 MB/sec execute 420 sec latency 2485.216 ms 2024-09-10T09:49:19.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:49:19 smithi005 ceph-mon[30350]: pgmap v722: 129 pgs: 129 active+clean; 254 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.1 MiB/s wr, 218 op/s 2024-09-10T09:49:19.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:49:19 smithi159 ceph-mon[35513]: pgmap v722: 129 pgs: 129 active+clean; 254 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.1 MiB/s wr, 218 op/s 2024-09-10T09:49:19.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:49:19 smithi102 ceph-mon[35442]: pgmap v722: 129 pgs: 129 active+clean; 254 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.1 MiB/s wr, 218 op/s 2024-09-10T09:49:20.122 INFO:tasks.workunit.client.0.smithi005.stdout: 1 687246 6.38 MB/sec execute 421 sec latency 3485.374 ms 2024-09-10T09:49:21.122 INFO:tasks.workunit.client.0.smithi005.stdout: 1 687246 6.36 MB/sec execute 422 sec latency 4485.529 ms 2024-09-10T09:49:21.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:49:21 smithi005 ceph-mon[30350]: pgmap v723: 129 pgs: 129 active+clean; 254 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 1.8 MiB/s wr, 187 op/s 2024-09-10T09:49:21.851 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:49:21 smithi159 ceph-mon[35513]: pgmap v723: 129 pgs: 129 active+clean; 254 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 1.8 MiB/s wr, 187 op/s 2024-09-10T09:49:21.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:49:21 smithi102 ceph-mon[35442]: pgmap v723: 129 pgs: 129 active+clean; 254 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 1.8 MiB/s wr, 187 op/s 2024-09-10T09:49:22.122 INFO:tasks.workunit.client.0.smithi005.stdout: 1 687246 6.35 MB/sec execute 423 sec latency 5485.687 ms 2024-09-10T09:49:23.122 INFO:tasks.workunit.client.0.smithi005.stdout: 1 687246 6.33 MB/sec execute 424 sec latency 6485.818 ms 2024-09-10T09:49:23.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:49:23 smithi005 ceph-mon[30350]: pgmap v724: 129 pgs: 129 active+clean; 254 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.3 MiB/s wr, 152 op/s 2024-09-10T09:49:23.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:49:23 smithi159 ceph-mon[35513]: pgmap v724: 129 pgs: 129 active+clean; 254 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.3 MiB/s wr, 152 op/s 2024-09-10T09:49:23.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:49:23 smithi102 ceph-mon[35442]: pgmap v724: 129 pgs: 129 active+clean; 254 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.3 MiB/s wr, 152 op/s 2024-09-10T09:49:24.123 INFO:tasks.workunit.client.0.smithi005.stdout: 1 687246 6.32 MB/sec execute 425 sec latency 7485.941 ms 2024-09-10T09:49:25.123 INFO:tasks.workunit.client.0.smithi005.stdout: 1 687246 6.30 MB/sec execute 426 sec latency 8486.109 ms 2024-09-10T09:49:25.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:49:25 smithi005 ceph-mon[30350]: pgmap v725: 129 pgs: 129 active+clean; 254 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.0 MiB/s wr, 106 op/s 2024-09-10T09:49:25.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:49:25 smithi159 ceph-mon[35513]: pgmap v725: 129 pgs: 129 active+clean; 254 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.0 MiB/s wr, 106 op/s 2024-09-10T09:49:25.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:49:25 smithi102 ceph-mon[35442]: pgmap v725: 129 pgs: 129 active+clean; 254 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.0 MiB/s wr, 106 op/s 2024-09-10T09:49:26.123 INFO:tasks.workunit.client.0.smithi005.stdout: 1 687246 6.29 MB/sec execute 427 sec latency 9486.272 ms 2024-09-10T09:49:26.594 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T09:49:27.026 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:49:27.124 INFO:tasks.workunit.client.0.smithi005.stdout: 1 687246 6.27 MB/sec execute 428 sec latency 10487.113 ms 2024-09-10T09:49:27.758 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:49:27 smithi005 ceph-mon[30350]: pgmap v726: 129 pgs: 129 active+clean; 254 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 934 KiB/s wr, 100 op/s 2024-09-10T09:49:27.851 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:49:27 smithi159 ceph-mon[35513]: pgmap v726: 129 pgs: 129 active+clean; 254 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 934 KiB/s wr, 100 op/s 2024-09-10T09:49:27.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:27.872+0000 7f4aa6000640 1 -- 172.21.15.5:0/770155816 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4aa0074590 msgr2=0x7f4aa0074970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:49:27.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:27.872+0000 7f4aa6000640 1 --2- 172.21.15.5:0/770155816 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4aa0074590 0x7f4aa0074970 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f4a90009900 tx=0x7f4a9002a480 comp rx=0 tx=0).stop 2024-09-10T09:49:27.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:27.872+0000 7f4aa6000640 1 -- 172.21.15.5:0/770155816 shutdown_connections 2024-09-10T09:49:27.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:27.872+0000 7f4aa6000640 1 --2- 172.21.15.5:0/770155816 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4aa0112b90 0x7f4aa0114f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:27.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:27.872+0000 7f4aa6000640 1 --2- 172.21.15.5:0/770155816 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4aa0074eb0 0x7f4aa0112650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:27.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:27.872+0000 7f4aa6000640 1 --2- 172.21.15.5:0/770155816 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4aa0074590 0x7f4aa0074970 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:27.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:27.872+0000 7f4aa6000640 1 -- 172.21.15.5:0/770155816 >> 172.21.15.5:0/770155816 conn(0x7f4aa00fd8c0 msgr2=0x7f4aa00ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:49:27.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:27.873+0000 7f4aa6000640 1 -- 172.21.15.5:0/770155816 shutdown_connections 2024-09-10T09:49:27.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:27.873+0000 7f4aa6000640 1 -- 172.21.15.5:0/770155816 wait complete. 2024-09-10T09:49:27.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:27.873+0000 7f4aa6000640 1 Processor -- start 2024-09-10T09:49:27.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:27.874+0000 7f4aa6000640 1 -- start start 2024-09-10T09:49:27.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:27.874+0000 7f4aa6000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4aa0074590 0x7f4aa01a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:49:27.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:27.874+0000 7f4aa6000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4aa0074eb0 0x7f4aa01a4c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:49:27.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:27.874+0000 7f4aa6000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4aa0112b90 0x7f4aa019e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:49:27.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:27.874+0000 7f4aa6000640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f4aa01185c0 con 0x7f4aa0074eb0 2024-09-10T09:49:27.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:27.874+0000 7f4aa6000640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f4aa0118440 con 0x7f4aa0074590 2024-09-10T09:49:27.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:27.874+0000 7f4aa6000640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f4aa0118740 con 0x7f4aa0112b90 2024-09-10T09:49:27.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:27.874+0000 7f4a97e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4aa0074eb0 0x7f4aa01a4c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:49:27.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:27.874+0000 7f4a97e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4aa0074eb0 0x7f4aa01a4c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40440/0 (socket says 172.21.15.5:40440) 2024-09-10T09:49:27.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:27.874+0000 7f4a97e00640 1 -- 172.21.15.5:0/332814649 learned_addr learned my addr 172.21.15.5:0/332814649 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:49:27.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:27.874+0000 7f4a9f400640 1 --2- 172.21.15.5:0/332814649 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4aa0074590 0x7f4aa01a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:49:27.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:27.874+0000 7f4a9fe00640 1 --2- 172.21.15.5:0/332814649 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4aa0112b90 0x7f4aa019e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:49:27.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:27.875+0000 7f4a97e00640 1 -- 172.21.15.5:0/332814649 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4aa0112b90 msgr2=0x7f4aa019e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:49:27.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:27.875+0000 7f4a97e00640 1 --2- 172.21.15.5:0/332814649 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4aa0112b90 0x7f4aa019e8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:27.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:27.875+0000 7f4a97e00640 1 -- 172.21.15.5:0/332814649 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4aa0074590 msgr2=0x7f4aa01a46f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:49:27.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:27.875+0000 7f4a97e00640 1 --2- 172.21.15.5:0/332814649 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4aa0074590 0x7f4aa01a46f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:27.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:27.875+0000 7f4a97e00640 1 -- 172.21.15.5:0/332814649 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4aa019efe0 con 0x7f4aa0074eb0 2024-09-10T09:49:27.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:27.875+0000 7f4a97e00640 1 --2- 172.21.15.5:0/332814649 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4aa0074eb0 0x7f4aa01a4c30 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f4a8800e9f0 tx=0x7f4a8800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:49:27.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:27.875+0000 7f4a9cc00640 1 -- 172.21.15.5:0/332814649 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a8800cd80 con 0x7f4aa0074eb0 2024-09-10T09:49:27.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:27.875+0000 7f4aa6000640 1 -- 172.21.15.5:0/332814649 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4aa019f2d0 con 0x7f4aa0074eb0 2024-09-10T09:49:27.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:27.875+0000 7f4a9cc00640 1 -- 172.21.15.5:0/332814649 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4a88010910 con 0x7f4aa0074eb0 2024-09-10T09:49:27.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:27.876+0000 7f4aa6000640 1 -- 172.21.15.5:0/332814649 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4aa01ab5c0 con 0x7f4aa0074eb0 2024-09-10T09:49:27.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:27.876+0000 7f4a9cc00640 1 -- 172.21.15.5:0/332814649 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a88018910 con 0x7f4aa0074eb0 2024-09-10T09:49:27.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:27.879+0000 7f4aa89fe740 1 Processor -- start 2024-09-10T09:49:27.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:27.879+0000 7f4aa89fe740 1 -- start start 2024-09-10T09:49:27.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:27.879+0000 7f4a9cc00640 1 -- 172.21.15.5:0/332814649 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f4a88010ac0 con 0x7f4aa0074eb0 2024-09-10T09:49:27.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:27.879+0000 7f4aa89fe740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559375b18b90 0x559375b18f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:49:27.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:27.879+0000 7f4aa89fe740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559375b194b0 0x559375b23830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:49:27.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:27.879+0000 7f4aa89fe740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559375b23d70 0x559375b26160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:49:27.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:27.879+0000 7f4aa89fe740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x559375a7f3e0 con 0x559375b194b0 2024-09-10T09:49:27.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:27.879+0000 7f4aa89fe740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x559375a26d80 con 0x559375b23d70 2024-09-10T09:49:27.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:27.879+0000 7f4aa89fe740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x559375a7b3a0 con 0x559375b18b90 2024-09-10T09:49:27.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:27.879+0000 7f4a9fe00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559375b18b90 0x559375b18f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:49:27.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:27.879+0000 7f4a9f400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559375b194b0 0x559375b23830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:49:27.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:27.879+0000 7f4a9fe00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559375b18b90 0x559375b18f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:40136/0 (socket says 172.21.15.5:40136) 2024-09-10T09:49:27.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:27.880+0000 7f4a9fe00640 1 -- 172.21.15.5:0/2328132102 learned_addr learned my addr 172.21.15.5:0/2328132102 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:49:27.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:27.880+0000 7f4a97e00640 1 --2- 172.21.15.5:0/2328132102 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559375b23d70 0x559375b26160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:49:27.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:27.880+0000 7f4a9fe00640 1 -- 172.21.15.5:0/2328132102 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559375b23d70 msgr2=0x559375b26160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:49:27.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:27.880+0000 7f4a9fe00640 1 --2- 172.21.15.5:0/2328132102 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559375b23d70 0x559375b26160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:27.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:27.880+0000 7f4a9fe00640 1 -- 172.21.15.5:0/2328132102 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559375b194b0 msgr2=0x559375b23830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:49:27.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:27.880+0000 7f4a9fe00640 1 --2- 172.21.15.5:0/2328132102 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559375b194b0 0x559375b23830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:27.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:27.880+0000 7f4a9fe00640 1 -- 172.21.15.5:0/2328132102 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559375a45ae0 con 0x559375b18b90 2024-09-10T09:49:27.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:27.880+0000 7f4a9fe00640 1 --2- 172.21.15.5:0/2328132102 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559375b18b90 0x559375b18f70 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f4a98009f00 tx=0x7f4a980107a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d2fa5fcb9b683c39 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:49:27.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:27.881+0000 7f4a8fe00640 1 -- 172.21.15.5:0/2328132102 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a98003b70 con 0x559375b18b90 2024-09-10T09:49:27.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:27.881+0000 7f4a8fe00640 1 -- 172.21.15.5:0/2328132102 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4a98003d10 con 0x559375b18b90 2024-09-10T09:49:27.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:27.881+0000 7f4a8fe00640 1 -- 172.21.15.5:0/2328132102 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a98011a40 con 0x559375b18b90 2024-09-10T09:49:27.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:27.881+0000 7f4a9cc00640 1 --2- 172.21.15.5:0/332814649 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f4a840a2b80 0x7f4a840a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:49:27.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:27.881+0000 7f4a9cc00640 1 -- 172.21.15.5:0/332814649 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f4a880cf880 con 0x7f4aa0074eb0 2024-09-10T09:49:27.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:27.881+0000 7f4a9f400640 1 --2- 172.21.15.5:0/332814649 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f4a840a2b80 0x7f4a840a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:49:27.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:27.881+0000 7f4aa89fe740 1 -- 172.21.15.5:0/2328132102 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559375b18b90 msgr2=0x559375b18f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:49:27.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:27.881+0000 7f4aa89fe740 1 --2- 172.21.15.5:0/2328132102 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559375b18b90 0x559375b18f70 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f4a98009f00 tx=0x7f4a980107a0 comp rx=0 tx=0).stop 2024-09-10T09:49:27.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:27.882+0000 7f4aa89fe740 1 -- 172.21.15.5:0/2328132102 shutdown_connections 2024-09-10T09:49:27.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:27.882+0000 7f4aa89fe740 1 --2- 172.21.15.5:0/2328132102 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559375b23d70 0x559375b26160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:27.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:27.882+0000 7f4aa89fe740 1 --2- 172.21.15.5:0/2328132102 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559375b194b0 0x559375b23830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:27.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:27.882+0000 7f4aa89fe740 1 --2- 172.21.15.5:0/2328132102 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559375b18b90 0x559375b18f70 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:27.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:27.882+0000 7f4aa89fe740 1 -- 172.21.15.5:0/2328132102 >> 172.21.15.5:0/2328132102 conn(0x559375b15120 msgr2=0x559375b15500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:49:27.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:27.882+0000 7f4aa89fe740 1 -- 172.21.15.5:0/2328132102 shutdown_connections 2024-09-10T09:49:27.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:27.882+0000 7f4aa89fe740 1 -- 172.21.15.5:0/2328132102 wait complete. 2024-09-10T09:49:27.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:27.882+0000 7f4a9f400640 1 --2- 172.21.15.5:0/332814649 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f4a840a2b80 0x7f4a840a5040 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f4a90006010 tx=0x7f4a90000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:49:27.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:27.882+0000 7f4aa89fe740 1 -- start start 2024-09-10T09:49:27.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:27.882+0000 7f4aa89fe740 1 Processor -- start 2024-09-10T09:49:27.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:27.883+0000 7f4aa89fe740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559375b18b90 0x559375b20ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:49:27.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:27.883+0000 7f4aa89fe740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559375b194b0 0x559375b211e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:49:27.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:27.883+0000 7f4aa89fe740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559375b23d70 0x559375b21720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:49:27.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:27.883+0000 7f4aa89fe740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x559375b269e0 con 0x559375b18b90 2024-09-10T09:49:27.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:27.883+0000 7f4aa89fe740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x559375a3a0f0 con 0x559375b194b0 2024-09-10T09:49:27.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:27.883+0000 7f4aa89fe740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x559375b26b60 con 0x559375b23d70 2024-09-10T09:49:27.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:27.883+0000 7f4aa89fe740 10 client.0 _ll_register_callbacks cb 0x559375a2b9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:49:27.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:27.883+0000 7f4a9f400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559375b18b90 0x559375b20ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:49:27.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:27.883+0000 7f4a9f400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559375b18b90 0x559375b20ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40450/0 (socket says 172.21.15.5:40450) 2024-09-10T09:49:27.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:27.883+0000 7f4a9f400640 1 -- 172.21.15.5:0/3720840588 learned_addr learned my addr 172.21.15.5:0/3720840588 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:49:27.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:27.883+0000 7f4a9fe00640 1 --2- 172.21.15.5:0/3720840588 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559375b23d70 0x559375b21720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:49:27.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:27.883+0000 7f4a97e00640 1 --2- 172.21.15.5:0/3720840588 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559375b194b0 0x559375b211e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:49:27.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:27.884+0000 7f4a9f400640 1 -- 172.21.15.5:0/3720840588 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559375b23d70 msgr2=0x559375b21720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:49:27.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:27.884+0000 7f4a9f400640 1 --2- 172.21.15.5:0/3720840588 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559375b23d70 0x559375b21720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:27.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:27.884+0000 7f4a9f400640 1 -- 172.21.15.5:0/3720840588 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559375b194b0 msgr2=0x559375b211e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:49:27.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:27.884+0000 7f4a9f400640 1 --2- 172.21.15.5:0/3720840588 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559375b194b0 0x559375b211e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:27.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:27.884+0000 7f4a9f400640 1 -- 172.21.15.5:0/3720840588 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5593757f03c0 con 0x559375b18b90 2024-09-10T09:49:27.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:27.884+0000 7f4a9fe00640 1 --2- 172.21.15.5:0/3720840588 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559375b23d70 0x559375b21720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:49:27.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:27.884+0000 7f4a9f400640 1 --2- 172.21.15.5:0/3720840588 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559375b18b90 0x559375b20ca0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f4a90033000 tx=0x7f4a90009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=532c241f213fa9df server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:49:27.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:27.884+0000 7f4a8cc00640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-10T09:49:27.888 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:49:27 smithi102 ceph-mon[35442]: pgmap v726: 129 pgs: 129 active+clean; 254 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 934 KiB/s wr, 100 op/s 2024-09-10T09:49:27.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:27.885+0000 7f4a8cc00640 1 -- 172.21.15.5:0/3720840588 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a9004e070 con 0x559375b18b90 2024-09-10T09:49:27.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:27.885+0000 7f4aa89fe740 1 -- 172.21.15.5:0/3720840588 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4a98003d10 con 0x559375b18b90 2024-09-10T09:49:27.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:27.885+0000 7f4a8cc00640 1 -- 172.21.15.5:0/3720840588 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f4a90004b90 con 0x559375b18b90 2024-09-10T09:49:27.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:27.885+0000 7f4a8cc00640 1 -- 172.21.15.5:0/3720840588 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f4a9005a050 con 0x559375b18b90 2024-09-10T09:49:27.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:27.885+0000 7f4aa89fe740 10 client.17580 fetch_fsmap learned FSMap version 36 2024-09-10T09:49:27.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:27.885+0000 7f4aa89fe740 1 -- 172.21.15.5:0/3720840588 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x559375b32730 con 0x559375b18b90 2024-09-10T09:49:27.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:27.885+0000 7f4a8cc00640 1 -- 172.21.15.5:0/3720840588 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4a98003d10 con 0x559375b18b90 2024-09-10T09:49:27.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:27.885+0000 7f4a8cc00640 1 -- 172.21.15.5:0/3720840588 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 36) ==== 8370+0+0 (secure 0 0 0) 0x7f4a900036c0 con 0x559375b18b90 2024-09-10T09:49:27.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:27.885+0000 7f4aa89fe740 10 client.17580 fetch_fsmap finished waiting for FSMap version 36 2024-09-10T09:49:27.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:27.886+0000 7f4aa89fe740 10 client.17580 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T09:49:27.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:27.886+0000 7f4aa89fe740 20 client.17580 populate_metadata read hostname 'smithi005' 2024-09-10T09:49:27.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:27.886+0000 7f4aa89fe740 1 --2- 172.21.15.5:0/3720840588 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x559375b33b40 0x559375b35f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:49:27.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:27.886+0000 7f4aa89fe740 4 client.17580 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:49:27.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:27.886+0000 7f4aa89fe740 1 -- 172.21.15.5:0/3720840588 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559375a8c950 con 0x559375b33b40 2024-09-10T09:49:27.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:27.886+0000 7f4a97e00640 1 --2- 172.21.15.5:0/3720840588 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x559375b33b40 0x559375b35f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:49:27.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:27.886+0000 7f4a97e00640 1 --2- 172.21.15.5:0/3720840588 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x559375b33b40 0x559375b35f90 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8815010f959f941c server_cookie=be8a350fcd1f648 in_seq=0 out_seq=0 2024-09-10T09:49:27.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:27.886+0000 7f4a8cc00640 10 client.17580 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T09:49:27.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:27.887+0000 7f4a8cc00640 1 -- 172.21.15.5:0/3720840588 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559375a8c950 con 0x559375b33b40 2024-09-10T09:49:27.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:27.887+0000 7f4a8cc00640 10 client.17580 handle_command_reply: tid=0 2024-09-10T09:49:27.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:27.887+0000 7f4aa89fe740 1 client.17580 shutdown 2024-09-10T09:49:27.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:27.887+0000 7f4aa89fe740 1 -- 172.21.15.5:0/3720840588 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559375b18b90 msgr2=0x559375b20ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:49:27.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:27.887+0000 7f4aa89fe740 1 --2- 172.21.15.5:0/3720840588 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559375b18b90 0x559375b20ca0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f4a90033000 tx=0x7f4a90009260 comp rx=0 tx=0).stop 2024-09-10T09:49:27.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:27.887+0000 7f4aa89fe740 1 -- 172.21.15.5:0/3720840588 shutdown_connections 2024-09-10T09:49:27.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:27.887+0000 7f4aa89fe740 1 --2- 172.21.15.5:0/3720840588 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x559375b33b40 0x559375b35f90 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:27.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:27.888+0000 7f4aa89fe740 1 --2- 172.21.15.5:0/3720840588 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559375b23d70 0x559375b21720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:27.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:27.888+0000 7f4aa89fe740 1 --2- 172.21.15.5:0/3720840588 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559375b194b0 0x559375b211e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:27.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:27.888+0000 7f4aa89fe740 1 --2- 172.21.15.5:0/3720840588 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559375b18b90 0x559375b20ca0 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:27.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:27.888+0000 7f4aa89fe740 1 -- 172.21.15.5:0/3720840588 >> 172.21.15.5:0/3720840588 conn(0x559375b15120 msgr2=0x559375b1ae50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:49:27.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:27.888+0000 7f4a8cc00640 0 client.17580 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T09:49:27.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:27.888+0000 7f4aa89fe740 1 -- 172.21.15.5:0/3720840588 shutdown_connections 2024-09-10T09:49:27.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:27.888+0000 7f4aa89fe740 1 -- 172.21.15.5:0/3720840588 wait complete. 2024-09-10T09:49:27.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:27.888+0000 7f4aa89fe740 20 client.17580 trim_cache size 0 max 16384 2024-09-10T09:49:27.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:27.906+0000 7f4aa89fe740 1 Processor -- start 2024-09-10T09:49:27.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:27.906+0000 7f4aa89fe740 1 -- start start 2024-09-10T09:49:27.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:27.906+0000 7f4aa89fe740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559375b18b90 0x559375b1bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:49:27.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:27.906+0000 7f4aa89fe740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559375b194b0 0x559375b1acf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:49:27.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:27.906+0000 7f4aa89fe740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559375b23d70 0x559375b3b860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:49:27.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:27.906+0000 7f4aa89fe740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f4aa01b5490 con 0x559375b194b0 2024-09-10T09:49:27.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:27.906+0000 7f4aa89fe740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f4aa01b6fd0 con 0x559375b23d70 2024-09-10T09:49:27.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:27.906+0000 7f4aa89fe740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f4aa01b5b60 con 0x559375b18b90 2024-09-10T09:49:27.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:27.907+0000 7f4a97e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559375b194b0 0x559375b1acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:49:27.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:27.907+0000 7f4a97e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559375b194b0 0x559375b1acf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40454/0 (socket says 172.21.15.5:40454) 2024-09-10T09:49:27.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:27.907+0000 7f4a97e00640 1 -- 172.21.15.5:0/1693854308 learned_addr learned my addr 172.21.15.5:0/1693854308 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:49:27.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:27.907+0000 7f4a9fe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559375b23d70 0x559375b3b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:49:27.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:27.907+0000 7f4a9f400640 1 --2- 172.21.15.5:0/1693854308 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559375b18b90 0x559375b1bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:49:27.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:27.907+0000 7f4a97e00640 1 -- 172.21.15.5:0/1693854308 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559375b18b90 msgr2=0x559375b1bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:49:27.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:27.907+0000 7f4a97e00640 1 --2- 172.21.15.5:0/1693854308 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559375b18b90 0x559375b1bfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:27.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:27.907+0000 7f4a97e00640 1 -- 172.21.15.5:0/1693854308 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559375b23d70 msgr2=0x559375b3b860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:49:27.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:27.907+0000 7f4a97e00640 1 --2- 172.21.15.5:0/1693854308 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559375b23d70 0x559375b3b860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:27.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:27.907+0000 7f4a97e00640 1 -- 172.21.15.5:0/1693854308 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559375b1fc50 con 0x559375b194b0 2024-09-10T09:49:27.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:27.907+0000 7f4a9fe00640 1 --2- 172.21.15.5:0/1693854308 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559375b23d70 0x559375b3b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:49:27.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:27.907+0000 7f4a97e00640 1 --2- 172.21.15.5:0/1693854308 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559375b194b0 0x559375b1acf0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f4a880ccf30 tx=0x7f4a880d1cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb90e6a1bdeb12f3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:49:27.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:27.908+0000 7f4a8f400640 1 -- 172.21.15.5:0/1693854308 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a880a4900 con 0x559375b194b0 2024-09-10T09:49:27.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:27.908+0000 7f4a8f400640 1 -- 172.21.15.5:0/1693854308 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4a88099f90 con 0x559375b194b0 2024-09-10T09:49:27.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:27.908+0000 7f4a8f400640 1 -- 172.21.15.5:0/1693854308 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a880979e0 con 0x559375b194b0 2024-09-10T09:49:27.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:27.908+0000 7f4aa89fe740 1 -- 172.21.15.5:0/1693854308 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559375b194b0 msgr2=0x559375b1acf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:49:27.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:27.908+0000 7f4aa89fe740 1 --2- 172.21.15.5:0/1693854308 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559375b194b0 0x559375b1acf0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f4a880ccf30 tx=0x7f4a880d1cd0 comp rx=0 tx=0).stop 2024-09-10T09:49:27.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:27.908+0000 7f4aa89fe740 1 -- 172.21.15.5:0/1693854308 shutdown_connections 2024-09-10T09:49:27.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:27.908+0000 7f4aa89fe740 1 --2- 172.21.15.5:0/1693854308 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559375b23d70 0x559375b3b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:27.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:27.908+0000 7f4aa89fe740 1 --2- 172.21.15.5:0/1693854308 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559375b194b0 0x559375b1acf0 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:27.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:27.908+0000 7f4aa89fe740 1 --2- 172.21.15.5:0/1693854308 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559375b18b90 0x559375b1bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:27.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:27.908+0000 7f4aa89fe740 1 -- 172.21.15.5:0/1693854308 >> 172.21.15.5:0/1693854308 conn(0x559375b15120 msgr2=0x559375b1a5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:49:27.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:27.909+0000 7f4aa89fe740 1 -- 172.21.15.5:0/1693854308 shutdown_connections 2024-09-10T09:49:27.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:27.909+0000 7f4aa89fe740 1 -- 172.21.15.5:0/1693854308 wait complete. 2024-09-10T09:49:27.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:27.909+0000 7f4aa89fe740 1 -- start start 2024-09-10T09:49:27.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:27.909+0000 7f4aa89fe740 1 Processor -- start 2024-09-10T09:49:27.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:27.910+0000 7f4aa89fe740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559375b18b90 0x559375b27a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:49:27.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:27.910+0000 7f4aa89fe740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559375b194b0 0x559375b27fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:49:27.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:27.910+0000 7f4aa89fe740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559375b23d70 0x559375b28510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:49:27.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:27.910+0000 7f4aa89fe740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f4aa01b6900 con 0x559375b194b0 2024-09-10T09:49:27.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:27.910+0000 7f4aa89fe740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f4aa01b6230 con 0x559375b23d70 2024-09-10T09:49:27.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:27.910+0000 7f4aa89fe740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f4aa01b3470 con 0x559375b18b90 2024-09-10T09:49:27.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:27.910+0000 7f4aa89fe740 10 client.0 _ll_register_callbacks cb 0x7f4a90004410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:49:27.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:27.910+0000 7f4a97e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559375b194b0 0x559375b27fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:49:27.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:27.910+0000 7f4a9f400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559375b18b90 0x559375b27a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:49:27.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:27.910+0000 7f4a9f400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559375b18b90 0x559375b27a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:40170/0 (socket says 172.21.15.5:40170) 2024-09-10T09:49:27.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:27.910+0000 7f4a9f400640 1 -- 172.21.15.5:0/396187871 learned_addr learned my addr 172.21.15.5:0/396187871 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:49:27.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:27.910+0000 7f4a9fe00640 1 --2- 172.21.15.5:0/396187871 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559375b23d70 0x559375b28510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:49:27.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:27.910+0000 7f4a9f400640 1 -- 172.21.15.5:0/396187871 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559375b23d70 msgr2=0x559375b28510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:49:27.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:27.910+0000 7f4a9f400640 1 --2- 172.21.15.5:0/396187871 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559375b23d70 0x559375b28510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:27.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:27.910+0000 7f4a9f400640 1 -- 172.21.15.5:0/396187871 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559375b194b0 msgr2=0x559375b27fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:49:27.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:27.910+0000 7f4a9f400640 1 --2- 172.21.15.5:0/396187871 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559375b194b0 0x559375b27fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:27.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:27.910+0000 7f4a9f400640 1 -- 172.21.15.5:0/396187871 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559375b9d830 con 0x559375b18b90 2024-09-10T09:49:27.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:27.911+0000 7f4a9f400640 1 --2- 172.21.15.5:0/396187871 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559375b18b90 0x559375b27a90 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f4a900043e0 tx=0x7f4a90003820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f902306499693c1b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:49:27.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:27.911+0000 7f4a8cc00640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-09-10T09:49:27.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:27.911+0000 7f4a8cc00640 1 -- 172.21.15.5:0/396187871 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a90062020 con 0x559375b18b90 2024-09-10T09:49:27.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:27.911+0000 7f4a8cc00640 1 -- 172.21.15.5:0/396187871 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f4a90004990 con 0x559375b18b90 2024-09-10T09:49:27.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:27.911+0000 7f4aa89fe740 1 -- 172.21.15.5:0/396187871 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4a88099f90 con 0x559375b18b90 2024-09-10T09:49:27.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:27.911+0000 7f4a8cc00640 1 -- 172.21.15.5:0/396187871 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f4a9006d050 con 0x559375b18b90 2024-09-10T09:49:27.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:27.911+0000 7f4aa89fe740 10 client.27601 fetch_fsmap learned FSMap version 36 2024-09-10T09:49:27.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:27.911+0000 7f4aa89fe740 1 -- 172.21.15.5:0/396187871 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x559375b9eaa0 con 0x559375b18b90 2024-09-10T09:49:27.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:27.911+0000 7f4a8cc00640 1 -- 172.21.15.5:0/396187871 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4a88099f90 con 0x559375b18b90 2024-09-10T09:49:27.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:27.912+0000 7f4a8cc00640 1 -- 172.21.15.5:0/396187871 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 36) ==== 8370+0+0 (secure 0 0 0) 0x7f4a90032b30 con 0x559375b18b90 2024-09-10T09:49:27.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:27.912+0000 7f4aa89fe740 10 client.27601 fetch_fsmap finished waiting for FSMap version 36 2024-09-10T09:49:27.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:27.912+0000 7f4aa89fe740 10 client.27601 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T09:49:27.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:27.912+0000 7f4aa89fe740 20 client.27601 populate_metadata read hostname 'smithi005' 2024-09-10T09:49:27.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:27.912+0000 7f4aa89fe740 1 --2- 172.21.15.5:0/396187871 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x559375b9f1b0 0x559375ba15d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:49:27.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:27.912+0000 7f4aa89fe740 4 client.27601 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T09:49:27.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:27.912+0000 7f4aa89fe740 1 -- 172.21.15.5:0/396187871 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "scrub status"}) -- 0x559375a86e60 con 0x559375b9f1b0 2024-09-10T09:49:27.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:27.912+0000 7f4a97e00640 1 --2- 172.21.15.5:0/396187871 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x559375b9f1b0 0x559375ba15d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:49:27.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:27.913+0000 7f4a97e00640 1 --2- 172.21.15.5:0/396187871 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x559375b9f1b0 0x559375ba15d0 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5f2e9748dae37e55 server_cookie=3e2f433f2913b1f1 in_seq=0 out_seq=0 2024-09-10T09:49:27.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:27.913+0000 7f4a8cc00640 10 client.27601 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T09:49:27.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:27.913+0000 7f4a8cc00640 1 -- 172.21.15.5:0/396187871 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x559375a86e60 con 0x559375b9f1b0 2024-09-10T09:49:27.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:27.913+0000 7f4a8cc00640 10 client.27601 handle_command_reply: tid=0 2024-09-10T09:49:27.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:27.913+0000 7f4aa89fe740 1 client.27601 shutdown 2024-09-10T09:49:27.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:27.913+0000 7f4aa89fe740 1 -- 172.21.15.5:0/396187871 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559375b18b90 msgr2=0x559375b27a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:49:27.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:27.913+0000 7f4aa89fe740 1 --2- 172.21.15.5:0/396187871 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559375b18b90 0x559375b27a90 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f4a900043e0 tx=0x7f4a90003820 comp rx=0 tx=0).stop 2024-09-10T09:49:27.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:27.913+0000 7f4aa89fe740 1 -- 172.21.15.5:0/396187871 shutdown_connections 2024-09-10T09:49:27.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:27.913+0000 7f4aa89fe740 1 --2- 172.21.15.5:0/396187871 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x559375b9f1b0 0x559375ba15d0 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:27.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:27.913+0000 7f4aa89fe740 1 --2- 172.21.15.5:0/396187871 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559375b23d70 0x559375b28510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:27.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:27.913+0000 7f4aa89fe740 1 --2- 172.21.15.5:0/396187871 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559375b194b0 0x559375b27fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:27.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:27.913+0000 7f4aa89fe740 1 --2- 172.21.15.5:0/396187871 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559375b18b90 0x559375b27a90 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:27.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:27.913+0000 7f4a8cc00640 0 client.27601 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T09:49:27.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:27.913+0000 7f4aa89fe740 1 -- 172.21.15.5:0/396187871 >> 172.21.15.5:0/396187871 conn(0x559375b15120 msgr2=0x559375b1ae00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:49:27.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:27.913+0000 7f4aa89fe740 1 -- 172.21.15.5:0/396187871 shutdown_connections 2024-09-10T09:49:27.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:27.913+0000 7f4aa89fe740 1 -- 172.21.15.5:0/396187871 wait complete. 2024-09-10T09:49:27.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:27.913+0000 7f4aa89fe740 20 client.27601 trim_cache size 0 max 16384 2024-09-10T09:49:27.918 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-10T09:49:27.918 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "no active scrubs running", 2024-09-10T09:49:27.919 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": {} 2024-09-10T09:49:27.919 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-10T09:49:27.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:27.914+0000 7f4aa6000640 1 -- 172.21.15.5:0/332814649 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f4a840a2b80 msgr2=0x7f4a840a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:49:27.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:27.914+0000 7f4aa6000640 1 --2- 172.21.15.5:0/332814649 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f4a840a2b80 0x7f4a840a5040 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f4a90006010 tx=0x7f4a90000bc0 comp rx=0 tx=0).stop 2024-09-10T09:49:27.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:27.914+0000 7f4aa6000640 1 -- 172.21.15.5:0/332814649 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4aa0074eb0 msgr2=0x7f4aa01a4c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:49:27.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:27.914+0000 7f4aa6000640 1 --2- 172.21.15.5:0/332814649 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4aa0074eb0 0x7f4aa01a4c30 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f4a8800e9f0 tx=0x7f4a8800eeb0 comp rx=0 tx=0).stop 2024-09-10T09:49:27.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:27.914+0000 7f4aa6000640 1 -- 172.21.15.5:0/332814649 shutdown_connections 2024-09-10T09:49:27.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:27.914+0000 7f4aa6000640 1 --2- 172.21.15.5:0/332814649 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f4a840a2b80 0x7f4a840a5040 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:27.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:27.914+0000 7f4aa6000640 1 --2- 172.21.15.5:0/332814649 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4aa0112b90 0x7f4aa019e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:27.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:27.914+0000 7f4aa6000640 1 --2- 172.21.15.5:0/332814649 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4aa0074eb0 0x7f4aa01a4c30 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:27.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:27.914+0000 7f4aa6000640 1 --2- 172.21.15.5:0/332814649 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4aa0074590 0x7f4aa01a46f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:27.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:27.914+0000 7f4aa6000640 1 -- 172.21.15.5:0/332814649 >> 172.21.15.5:0/332814649 conn(0x7f4aa00fd8c0 msgr2=0x7f4aa00fef70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:49:27.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:27.914+0000 7f4aa6000640 1 -- 172.21.15.5:0/332814649 shutdown_connections 2024-09-10T09:49:27.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:27.914+0000 7f4aa6000640 1 -- 172.21.15.5:0/332814649 wait complete. 2024-09-10T09:49:28.124 INFO:tasks.workunit.client.0.smithi005.stdout: 1 687246 6.26 MB/sec execute 429 sec latency 11487.176 ms 2024-09-10T09:49:29.114 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-10T09:49:29.115 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T09:49:29.124 INFO:tasks.workunit.client.0.smithi005.stdout: 1 687246 6.24 MB/sec execute 430 sec latency 12487.216 ms 2024-09-10T09:49:29.497 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:49:29.801 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:49:29 smithi005 ceph-mon[30350]: pgmap v727: 129 pgs: 129 active+clean; 255 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 724 KiB/s rd, 671 KiB/s wr, 77 op/s 2024-09-10T09:49:29.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:49:29 smithi159 ceph-mon[35513]: pgmap v727: 129 pgs: 129 active+clean; 255 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 724 KiB/s rd, 671 KiB/s wr, 77 op/s 2024-09-10T09:49:29.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:49:29 smithi102 ceph-mon[35442]: pgmap v727: 129 pgs: 129 active+clean; 255 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 724 KiB/s rd, 671 KiB/s wr, 77 op/s 2024-09-10T09:49:30.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:30.055+0000 7f18db200640 1 -- 172.21.15.5:0/656182833 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f18d4112870 msgr2=0x7f18d4114c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:49:30.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:30.055+0000 7f18db200640 1 --2- 172.21.15.5:0/656182833 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f18d4112870 0x7f18d4114c60 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f18d0009ca0 tx=0x7f18d002a8c0 comp rx=0 tx=0).stop 2024-09-10T09:49:30.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:30.055+0000 7f18db200640 1 -- 172.21.15.5:0/656182833 shutdown_connections 2024-09-10T09:49:30.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:30.055+0000 7f18db200640 1 --2- 172.21.15.5:0/656182833 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f18d4112870 0x7f18d4114c60 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:30.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:30.055+0000 7f18db200640 1 --2- 172.21.15.5:0/656182833 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f18d4077040 0x7f18d4112330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:30.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:30.055+0000 7f18db200640 1 --2- 172.21.15.5:0/656182833 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f18d4076720 0x7f18d4076b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:30.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:30.055+0000 7f18db200640 1 -- 172.21.15.5:0/656182833 >> 172.21.15.5:0/656182833 conn(0x7f18d40ff8d0 msgr2=0x7f18d4101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:49:30.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:30.056+0000 7f18db200640 1 -- 172.21.15.5:0/656182833 shutdown_connections 2024-09-10T09:49:30.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:30.056+0000 7f18db200640 1 -- 172.21.15.5:0/656182833 wait complete. 2024-09-10T09:49:30.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:30.056+0000 7f18db200640 1 Processor -- start 2024-09-10T09:49:30.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:30.057+0000 7f18db200640 1 -- start start 2024-09-10T09:49:30.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:30.057+0000 7f18db200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f18d4076720 0x7f18d41a4650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:49:30.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:30.057+0000 7f18db200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f18d4077040 0x7f18d41a4b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:49:30.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:30.057+0000 7f18db200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f18d4112870 0x7f18d419e720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:49:30.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:30.057+0000 7f18db200640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f18d41183b0 con 0x7f18d4077040 2024-09-10T09:49:30.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:30.057+0000 7f18db200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f18d4118230 con 0x7f18d4076720 2024-09-10T09:49:30.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:30.057+0000 7f18db200640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f18d4118530 con 0x7f18d4112870 2024-09-10T09:49:30.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:30.057+0000 7f18d9400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f18d4077040 0x7f18d41a4b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:49:30.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:30.057+0000 7f18d9400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f18d4077040 0x7f18d41a4b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40486/0 (socket says 172.21.15.5:40486) 2024-09-10T09:49:30.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:30.057+0000 7f18d9400640 1 -- 172.21.15.5:0/3036934337 learned_addr learned my addr 172.21.15.5:0/3036934337 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:49:30.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:30.057+0000 7f18d9e00640 1 --2- 172.21.15.5:0/3036934337 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f18d4076720 0x7f18d41a4650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:49:30.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:30.057+0000 7f18da800640 1 --2- 172.21.15.5:0/3036934337 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f18d4112870 0x7f18d419e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:49:30.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:30.058+0000 7f18d9400640 1 -- 172.21.15.5:0/3036934337 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f18d4112870 msgr2=0x7f18d419e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:49:30.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:30.058+0000 7f18d9400640 1 --2- 172.21.15.5:0/3036934337 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f18d4112870 0x7f18d419e720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:30.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:30.058+0000 7f18d9400640 1 -- 172.21.15.5:0/3036934337 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f18d4076720 msgr2=0x7f18d41a4650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:49:30.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:30.058+0000 7f18d9400640 1 --2- 172.21.15.5:0/3036934337 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f18d4076720 0x7f18d41a4650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:30.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:30.058+0000 7f18d9400640 1 -- 172.21.15.5:0/3036934337 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18d419ef90 con 0x7f18d4077040 2024-09-10T09:49:30.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:30.058+0000 7f18d9400640 1 --2- 172.21.15.5:0/3036934337 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f18d4077040 0x7f18d41a4b90 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f18c400e9f0 tx=0x7f18c400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:49:30.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:30.058+0000 7f18cea00640 1 -- 172.21.15.5:0/3036934337 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18c400cd80 con 0x7f18d4077040 2024-09-10T09:49:30.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:30.058+0000 7f18cea00640 1 -- 172.21.15.5:0/3036934337 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f18c4010910 con 0x7f18d4077040 2024-09-10T09:49:30.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:30.058+0000 7f18db200640 1 -- 172.21.15.5:0/3036934337 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18d419f280 con 0x7f18d4077040 2024-09-10T09:49:30.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:30.058+0000 7f18cea00640 1 -- 172.21.15.5:0/3036934337 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18c4018990 con 0x7f18d4077040 2024-09-10T09:49:30.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:30.058+0000 7f18db200640 1 -- 172.21.15.5:0/3036934337 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18d41ab5d0 con 0x7f18d4077040 2024-09-10T09:49:30.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:30.059+0000 7f18db200640 1 -- 172.21.15.5:0/3036934337 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f18d4077e60 con 0x7f18d4077040 2024-09-10T09:49:30.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:30.062+0000 7f18cea00640 1 -- 172.21.15.5:0/3036934337 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f18c4010ac0 con 0x7f18d4077040 2024-09-10T09:49:30.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:30.063+0000 7f18cea00640 1 --2- 172.21.15.5:0/3036934337 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f18bc0a2c00 0x7f18bc0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:49:30.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:30.063+0000 7f18cea00640 1 -- 172.21.15.5:0/3036934337 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f18c40cfad0 con 0x7f18d4077040 2024-09-10T09:49:30.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:30.063+0000 7f18d9e00640 1 --2- 172.21.15.5:0/3036934337 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f18bc0a2c00 0x7f18bc0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:49:30.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:30.064+0000 7f18d9e00640 1 --2- 172.21.15.5:0/3036934337 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f18bc0a2c00 0x7f18bc0a50c0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f18c8005d00 tx=0x7f18c800b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:49:30.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:30.065+0000 7f18cea00640 1 -- 172.21.15.5:0/3036934337 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f18c409bf90 con 0x7f18d4077040 2024-09-10T09:49:30.124 INFO:tasks.workunit.client.0.smithi005.stdout: 1 687246 6.23 MB/sec execute 431 sec latency 13487.279 ms 2024-09-10T09:49:30.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:30.288+0000 7f18db200640 1 -- 172.21.15.5:0/3036934337 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f18d410da00 con 0x7f18d4077040 2024-09-10T09:49:30.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:30.289+0000 7f18cea00640 1 -- 172.21.15.5:0/3036934337 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) ==== 94+0+11969 (secure 0 0 0) 0x7f18c40a0290 con 0x7f18d4077040 2024-09-10T09:49:30.290 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-10T09:49:30.290 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":36,"btime":"2024-09-10T09:48:07:349075+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14454,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/2970594311","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2970594311},{"type":"v1","addr":"172.21.15.5:6835","nonce":2970594311}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14514,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6839/3616502174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":3616502174},{"type":"v1","addr":"172.21.15.5:6839","nonce":3616502174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":36,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T09:35:29.626287+0000","modified":"2024-09-10T09:48:06.970127+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24542,"mds_1":24535,"mds_2":24511,"mds_3":24506,"mds_4":24443},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14520":{"gid":14520,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/2073599842","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":2073599842},{"type":"v1","addr":"172.21.15.5:6841","nonce":2073599842}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"k","rank":4,"incarnation":26,"state":"up:active","state_seq":24,"addr":"172.21.15.102:6833/2457007427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":2457007427},{"type":"v1","addr":"172.21.15.102:6833","nonce":2457007427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/356445159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":356445159},{"type":"v1","addr":"172.21.15.159:6833","nonce":356445159}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6837/2894314507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":2894314507},{"type":"v1","addr":"172.21.15.5:6837","nonce":2894314507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.102:6835/1647549111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":1647549111},{"type":"v1","addr":"172.21.15.102:6835","nonce":1647549111}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6835/952657242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":952657242},{"type":"v1","addr":"172.21.15.159:6835","nonce":952657242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":19,"addr":"172.21.15.102:6837/2689776764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":2689776764},{"type":"v1","addr":"172.21.15.102:6837","nonce":2689776764}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.159:6837/620627639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":620627639},{"type":"v1","addr":"172.21.15.159:6837","nonce":620627639}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24535":{"gid":24535,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":18,"addr":"172.21.15.159:6839/2073739599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":2073739599},{"type":"v1","addr":"172.21.15.159:6839","nonce":2073739599}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24542":{"gid":24542,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.102:6839/335409749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6838","nonce":335409749},{"type":"v1","addr":"172.21.15.102:6839","nonce":335409749}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24542,"qdb_cluster":[24542,24535,24511,24506,24443]},"id":1}]} 2024-09-10T09:49:30.290 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 36 2024-09-10T09:49:30.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:30.291+0000 7f18db200640 1 -- 172.21.15.5:0/3036934337 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f18bc0a2c00 msgr2=0x7f18bc0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:49:30.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:30.291+0000 7f18db200640 1 --2- 172.21.15.5:0/3036934337 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f18bc0a2c00 0x7f18bc0a50c0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f18c8005d00 tx=0x7f18c800b410 comp rx=0 tx=0).stop 2024-09-10T09:49:30.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:30.291+0000 7f18db200640 1 -- 172.21.15.5:0/3036934337 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f18d4077040 msgr2=0x7f18d41a4b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:49:30.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:30.291+0000 7f18db200640 1 --2- 172.21.15.5:0/3036934337 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f18d4077040 0x7f18d41a4b90 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f18c400e9f0 tx=0x7f18c400eeb0 comp rx=0 tx=0).stop 2024-09-10T09:49:30.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:30.291+0000 7f18db200640 1 -- 172.21.15.5:0/3036934337 shutdown_connections 2024-09-10T09:49:30.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:30.291+0000 7f18db200640 1 --2- 172.21.15.5:0/3036934337 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f18bc0a2c00 0x7f18bc0a50c0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:30.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:30.291+0000 7f18db200640 1 --2- 172.21.15.5:0/3036934337 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f18d4112870 0x7f18d419e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:30.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:30.291+0000 7f18db200640 1 --2- 172.21.15.5:0/3036934337 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f18d4077040 0x7f18d41a4b90 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:30.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:30.291+0000 7f18db200640 1 --2- 172.21.15.5:0/3036934337 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f18d4076720 0x7f18d41a4650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:30.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:30.291+0000 7f18db200640 1 -- 172.21.15.5:0/3036934337 >> 172.21.15.5:0/3036934337 conn(0x7f18d40ff8d0 msgr2=0x7f18d4101cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:49:30.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:30.291+0000 7f18db200640 1 -- 172.21.15.5:0/3036934337 shutdown_connections 2024-09-10T09:49:30.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:30.291+0000 7f18db200640 1 -- 172.21.15.5:0/3036934337 wait complete. 2024-09-10T09:49:30.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:49:30 smithi005 ceph-mon[30350]: from='client.? 172.21.15.5:0/3036934337' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:49:30.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:49:30 smithi159 ceph-mon[35513]: from='client.? 172.21.15.5:0/3036934337' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:49:30.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:49:30 smithi102 ceph-mon[35442]: from='client.? 172.21.15.5:0/3036934337' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:49:31.090 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-10T09:49:31.124 INFO:tasks.workunit.client.0.smithi005.stdout: 1 687246 6.21 MB/sec execute 432 sec latency 14487.394 ms 2024-09-10T09:49:31.502 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:49:31.777 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:49:31 smithi005 ceph-mon[30350]: pgmap v728: 129 pgs: 129 active+clean; 256 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 219 KiB/s rd, 333 KiB/s wr, 34 op/s 2024-09-10T09:49:31.785 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:49:31 smithi102 ceph-mon[35442]: pgmap v728: 129 pgs: 129 active+clean; 256 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 219 KiB/s rd, 333 KiB/s wr, 34 op/s 2024-09-10T09:49:31.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:49:31 smithi159 ceph-mon[35513]: pgmap v728: 129 pgs: 129 active+clean; 256 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 219 KiB/s rd, 333 KiB/s wr, 34 op/s 2024-09-10T09:49:32.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:32.096+0000 7feb5e800640 1 -- 172.21.15.5:0/88471984 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7feb58109450 msgr2=0x7feb58110250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:49:32.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:32.096+0000 7feb5e800640 1 --2- 172.21.15.5:0/88471984 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7feb58109450 0x7feb58110250 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7feb5001acb0 tx=0x7feb5003b900 comp rx=0 tx=0).stop 2024-09-10T09:49:32.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:32.100+0000 7feb5e800640 1 -- 172.21.15.5:0/88471984 shutdown_connections 2024-09-10T09:49:32.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:32.100+0000 7feb5e800640 1 --2- 172.21.15.5:0/88471984 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7feb5811b140 0x7feb5811d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:32.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:32.100+0000 7feb5e800640 1 --2- 172.21.15.5:0/88471984 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feb58109830 0x7feb58109c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:32.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:32.100+0000 7feb5e800640 1 --2- 172.21.15.5:0/88471984 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7feb58109450 0x7feb58110250 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:32.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:32.100+0000 7feb5e800640 1 -- 172.21.15.5:0/88471984 >> 172.21.15.5:0/88471984 conn(0x7feb5806e140 msgr2=0x7feb5806e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:49:32.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:32.100+0000 7feb5e800640 1 -- 172.21.15.5:0/88471984 shutdown_connections 2024-09-10T09:49:32.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:32.100+0000 7feb5e800640 1 -- 172.21.15.5:0/88471984 wait complete. 2024-09-10T09:49:32.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:32.104+0000 7feb5e800640 1 Processor -- start 2024-09-10T09:49:32.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:32.104+0000 7feb5e800640 1 -- start start 2024-09-10T09:49:32.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:32.105+0000 7feb5e800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7feb58109450 0x7feb5811a860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:49:32.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:32.105+0000 7feb5e800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feb58109830 0x7feb58116950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:49:32.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:32.105+0000 7feb5e800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7feb5811b140 0x7feb58116e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:49:32.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:32.105+0000 7feb5e800640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7feb58113340 con 0x7feb58109830 2024-09-10T09:49:32.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:32.105+0000 7feb5e800640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7feb581131c0 con 0x7feb5811b140 2024-09-10T09:49:32.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:32.105+0000 7feb5e800640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7feb581134c0 con 0x7feb58109450 2024-09-10T09:49:32.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:32.105+0000 7feb57400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7feb58109450 0x7feb5811a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:49:32.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:32.105+0000 7feb57400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7feb58109450 0x7feb5811a860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:58382/0 (socket says 172.21.15.5:58382) 2024-09-10T09:49:32.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:32.105+0000 7feb57400640 1 -- 172.21.15.5:0/949020050 learned_addr learned my addr 172.21.15.5:0/949020050 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:49:32.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:32.105+0000 7feb57e00640 1 --2- 172.21.15.5:0/949020050 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7feb5811b140 0x7feb58116e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:49:32.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:32.106+0000 7feb56a00640 1 --2- 172.21.15.5:0/949020050 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feb58109830 0x7feb58116950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:49:32.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:32.106+0000 7feb57e00640 1 -- 172.21.15.5:0/949020050 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7feb58109450 msgr2=0x7feb5811a860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:49:32.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:32.106+0000 7feb57e00640 1 --2- 172.21.15.5:0/949020050 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7feb58109450 0x7feb5811a860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:32.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:32.106+0000 7feb57e00640 1 -- 172.21.15.5:0/949020050 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feb58109830 msgr2=0x7feb58116950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:49:32.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:32.106+0000 7feb57e00640 1 --2- 172.21.15.5:0/949020050 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feb58109830 0x7feb58116950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:32.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:32.106+0000 7feb57e00640 1 -- 172.21.15.5:0/949020050 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb58117590 con 0x7feb5811b140 2024-09-10T09:49:32.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:32.106+0000 7feb57e00640 1 --2- 172.21.15.5:0/949020050 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7feb5811b140 0x7feb58116e90 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7feb4400b410 tx=0x7feb4400b8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:49:32.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:32.107+0000 7feb4fe00640 1 -- 172.21.15.5:0/949020050 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb44015020 con 0x7feb5811b140 2024-09-10T09:49:32.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:32.107+0000 7feb4fe00640 1 -- 172.21.15.5:0/949020050 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feb44011d10 con 0x7feb5811b140 2024-09-10T09:49:32.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:32.107+0000 7feb4fe00640 1 -- 172.21.15.5:0/949020050 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb44010e30 con 0x7feb5811b140 2024-09-10T09:49:32.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:32.108+0000 7feb5e800640 1 -- 172.21.15.5:0/949020050 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb581ab170 con 0x7feb5811b140 2024-09-10T09:49:32.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:32.108+0000 7feb5e800640 1 -- 172.21.15.5:0/949020050 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb581ab570 con 0x7feb5811b140 2024-09-10T09:49:32.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:32.114+0000 7feb61457740 1 Processor -- start 2024-09-10T09:49:32.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:32.114+0000 7feb4fe00640 1 -- 172.21.15.5:0/949020050 <== mon.2 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7feb44018020 con 0x7feb5811b140 2024-09-10T09:49:32.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:32.116+0000 7feb61457740 1 -- start start 2024-09-10T09:49:32.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:32.116+0000 7feb61457740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555e64fb7c70 0x555e64fb8050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:49:32.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:32.116+0000 7feb61457740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555e64fb8590 0x555e64fc2910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:49:32.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:32.116+0000 7feb61457740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555e64fc2e50 0x555e64fc5240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:49:32.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:32.116+0000 7feb61457740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x555e64f1e3e0 con 0x555e64fb8590 2024-09-10T09:49:32.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:32.116+0000 7feb61457740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x555e64ec5d80 con 0x555e64fc2e50 2024-09-10T09:49:32.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:32.116+0000 7feb61457740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x555e64f1a3a0 con 0x555e64fb7c70 2024-09-10T09:49:32.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:32.117+0000 7feb56a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555e64fb7c70 0x555e64fb8050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:49:32.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:32.118+0000 7feb56a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555e64fb7c70 0x555e64fb8050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:58394/0 (socket says 172.21.15.5:58394) 2024-09-10T09:49:32.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:32.118+0000 7feb56a00640 1 -- 172.21.15.5:0/2823129230 learned_addr learned my addr 172.21.15.5:0/2823129230 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:49:32.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:32.118+0000 7feb56a00640 1 --2- 172.21.15.5:0/2823129230 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555e64fc2e50 0x555e64fc5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:49:32.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:32.117+0000 7feb57400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555e64fb8590 0x555e64fc2910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:49:32.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:32.120+0000 7feb56a00640 1 -- 172.21.15.5:0/2823129230 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555e64fc2e50 msgr2=0x555e64fc5240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:49:32.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:32.120+0000 7feb56a00640 1 --2- 172.21.15.5:0/2823129230 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555e64fc2e50 0x555e64fc5240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:32.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:32.120+0000 7feb56a00640 1 -- 172.21.15.5:0/2823129230 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555e64fb8590 msgr2=0x555e64fc2910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:49:32.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:32.120+0000 7feb56a00640 1 --2- 172.21.15.5:0/2823129230 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555e64fb8590 0x555e64fc2910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:32.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:32.121+0000 7feb56a00640 1 -- 172.21.15.5:0/2823129230 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555e64ee4ae0 con 0x555e64fb7c70 2024-09-10T09:49:32.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:32.121+0000 7feb57400640 1 --2- 172.21.15.5:0/2823129230 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555e64fb8590 0x555e64fc2910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T09:49:32.124 INFO:tasks.workunit.client.0.smithi005.stdout: 1 687246 6.20 MB/sec execute 433 sec latency 15487.462 ms 2024-09-10T09:49:32.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:32.122+0000 7feb4fe00640 1 --2- 172.21.15.5:0/949020050 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7feb3c0a2c00 0x7feb3c0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:49:32.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:32.123+0000 7feb4fe00640 1 -- 172.21.15.5:0/949020050 <== mon.2 v2:172.21.15.102:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7feb440cec80 con 0x7feb5811b140 2024-09-10T09:49:32.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:32.124+0000 7feb56a00640 1 --2- 172.21.15.5:0/2823129230 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555e64fb7c70 0x555e64fb8050 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7feb48009fd0 tx=0x7feb48009610 comp rx=0 tx=0).ready entity=mon.1 client_cookie=95ea430d3609afb9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:49:32.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:32.127+0000 7feb4cc00640 1 -- 172.21.15.5:0/2823129230 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb48014110 con 0x555e64fb7c70 2024-09-10T09:49:32.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:32.127+0000 7feb4cc00640 1 -- 172.21.15.5:0/2823129230 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feb480142b0 con 0x555e64fb7c70 2024-09-10T09:49:32.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:32.127+0000 7feb4cc00640 1 -- 172.21.15.5:0/2823129230 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb48003850 con 0x555e64fb7c70 2024-09-10T09:49:32.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:32.128+0000 7feb61457740 1 -- 172.21.15.5:0/2823129230 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555e64fb7c70 msgr2=0x555e64fb8050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:49:32.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:32.128+0000 7feb61457740 1 --2- 172.21.15.5:0/2823129230 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555e64fb7c70 0x555e64fb8050 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7feb48009fd0 tx=0x7feb48009610 comp rx=0 tx=0).stop 2024-09-10T09:49:32.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:32.128+0000 7feb61457740 1 -- 172.21.15.5:0/2823129230 shutdown_connections 2024-09-10T09:49:32.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:32.128+0000 7feb61457740 1 --2- 172.21.15.5:0/2823129230 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555e64fc2e50 0x555e64fc5240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:32.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:32.128+0000 7feb61457740 1 --2- 172.21.15.5:0/2823129230 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555e64fb8590 0x555e64fc2910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:32.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:32.128+0000 7feb61457740 1 --2- 172.21.15.5:0/2823129230 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555e64fb7c70 0x555e64fb8050 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:32.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:32.128+0000 7feb61457740 1 -- 172.21.15.5:0/2823129230 >> 172.21.15.5:0/2823129230 conn(0x555e64fb4200 msgr2=0x555e64fb45e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:49:32.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:32.128+0000 7feb61457740 1 -- 172.21.15.5:0/2823129230 shutdown_connections 2024-09-10T09:49:32.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:32.129+0000 7feb61457740 1 -- 172.21.15.5:0/2823129230 wait complete. 2024-09-10T09:49:32.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:32.129+0000 7feb61457740 1 -- start start 2024-09-10T09:49:32.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:32.130+0000 7feb57400640 1 --2- 172.21.15.5:0/949020050 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7feb3c0a2c00 0x7feb3c0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:49:32.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:32.130+0000 7feb61457740 1 Processor -- start 2024-09-10T09:49:32.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:32.132+0000 7feb61457740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555e64fb7c70 0x555e64fbfcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:49:32.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:32.132+0000 7feb61457740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555e64fb8590 0x555e64fc0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:49:32.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:32.133+0000 7feb61457740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555e64fc2e50 0x555e64fc0740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:49:32.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:32.133+0000 7feb56a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555e64fb8590 0x555e64fc0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:49:32.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:32.133+0000 7feb56a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555e64fb8590 0x555e64fc0200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:54704/0 (socket says 172.21.15.5:54704) 2024-09-10T09:49:32.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:32.133+0000 7feb57e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555e64fc2e50 0x555e64fc0740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:49:32.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:32.133+0000 7feb57e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555e64fc2e50 0x555e64fc0740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:58410/0 (socket says 172.21.15.5:58410) 2024-09-10T09:49:32.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:32.134+0000 7feb56a00640 1 -- 172.21.15.5:0/4213011492 learned_addr learned my addr 172.21.15.5:0/4213011492 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:49:32.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:32.134+0000 7feb61457740 1 -- 172.21.15.5:0/4213011492 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x555e64fc5a20 con 0x555e64fb7c70 2024-09-10T09:49:32.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:32.134+0000 7feb61457740 1 -- 172.21.15.5:0/4213011492 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x555e64ed90f0 con 0x555e64fb8590 2024-09-10T09:49:32.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:32.134+0000 7feb61457740 1 -- 172.21.15.5:0/4213011492 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x555e64fc5ba0 con 0x555e64fc2e50 2024-09-10T09:49:32.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:32.135+0000 7feb57400640 1 --2- 172.21.15.5:0/4213011492 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555e64fb7c70 0x555e64fbfcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:49:32.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:32.135+0000 7feb61457740 10 client.0 _ll_register_callbacks cb 0x555e64ec0bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:49:32.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:32.135+0000 7feb57400640 1 --2- 172.21.15.5:0/949020050 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7feb3c0a2c00 0x7feb3c0a50c0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7feb50005e30 tx=0x7feb50040510 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:49:32.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:32.136+0000 7feb57e00640 1 -- 172.21.15.5:0/4213011492 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555e64fb8590 msgr2=0x555e64fc0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:49:32.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:32.136+0000 7feb57e00640 1 --2- 172.21.15.5:0/4213011492 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555e64fb8590 0x555e64fc0200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:32.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:32.136+0000 7feb57e00640 1 -- 172.21.15.5:0/4213011492 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555e64fb7c70 msgr2=0x555e64fbfcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:49:32.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:32.136+0000 7feb57e00640 1 --2- 172.21.15.5:0/4213011492 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555e64fb7c70 0x555e64fbfcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:32.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:32.136+0000 7feb57e00640 1 -- 172.21.15.5:0/4213011492 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555e64c8f3c0 con 0x555e64fc2e50 2024-09-10T09:49:32.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:32.137+0000 7feb57e00640 1 --2- 172.21.15.5:0/4213011492 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555e64fc2e50 0x555e64fc0740 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7feb440cc390 tx=0x7feb4409ab10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e4cc0cd10d8e643d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:49:32.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:32.137+0000 7feb41600640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-09-10T09:49:32.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:32.137+0000 7feb41600640 1 -- 172.21.15.5:0/4213011492 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb440a3d60 con 0x555e64fc2e50 2024-09-10T09:49:32.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:32.137+0000 7feb61457740 1 -- 172.21.15.5:0/4213011492 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feb480142b0 con 0x555e64fc2e50 2024-09-10T09:49:32.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:32.137+0000 7feb41600640 1 -- 172.21.15.5:0/4213011492 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7feb440993f0 con 0x555e64fc2e50 2024-09-10T09:49:32.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:32.138+0000 7feb41600640 1 -- 172.21.15.5:0/4213011492 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7feb440a3d60 con 0x555e64fc2e50 2024-09-10T09:49:32.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:32.138+0000 7feb61457740 10 client.27619 fetch_fsmap learned FSMap version 36 2024-09-10T09:49:32.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:32.138+0000 7feb61457740 1 -- 172.21.15.5:0/4213011492 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x555e64fd19d0 con 0x555e64fc2e50 2024-09-10T09:49:32.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:32.138+0000 7feb41600640 1 -- 172.21.15.5:0/4213011492 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feb480142b0 con 0x555e64fc2e50 2024-09-10T09:49:32.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:32.139+0000 7feb41600640 1 -- 172.21.15.5:0/4213011492 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 36) ==== 8370+0+0 (secure 0 0 0) 0x7feb440b5030 con 0x555e64fc2e50 2024-09-10T09:49:32.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:32.140+0000 7feb61457740 10 client.27619 fetch_fsmap finished waiting for FSMap version 36 2024-09-10T09:49:32.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:32.140+0000 7feb61457740 10 client.27619 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-09-10T09:49:32.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:32.140+0000 7feb61457740 20 client.27619 populate_metadata read hostname 'smithi005' 2024-09-10T09:49:32.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:32.140+0000 7feb61457740 1 --2- 172.21.15.5:0/4213011492 >> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] conn(0x555e64fd2de0 0x555e64fd5230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:49:32.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:32.140+0000 7feb61457740 4 client.27619 mds_command: new command op to 24443 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:49:32.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:32.140+0000 7feb61457740 1 -- 172.21.15.5:0/4213011492 --> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555e64f2b950 con 0x555e64fd2de0 2024-09-10T09:49:32.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:32.142+0000 7feb57400640 1 --2- 172.21.15.5:0/4213011492 >> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] conn(0x555e64fd2de0 0x555e64fd5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:49:32.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:32.142+0000 7feb57400640 1 --2- 172.21.15.5:0/4213011492 >> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] conn(0x555e64fd2de0 0x555e64fd5230 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=c099c351db68801b server_cookie=b858423d736b688f in_seq=0 out_seq=0 2024-09-10T09:49:32.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:32.142+0000 7feb41600640 10 client.27619 ms_handle_connect on v2:172.21.15.102:6832/2457007427 2024-09-10T09:49:32.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:32.143+0000 7feb41600640 1 -- 172.21.15.5:0/4213011492 <== mds.4 v2:172.21.15.102:6832/2457007427 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555e64f2b950 con 0x555e64fd2de0 2024-09-10T09:49:32.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:32.144+0000 7feb41600640 10 client.27619 handle_command_reply: tid=0 2024-09-10T09:49:32.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:32.144+0000 7feb61457740 1 client.27619 shutdown 2024-09-10T09:49:32.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:32.144+0000 7feb61457740 1 -- 172.21.15.5:0/4213011492 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555e64fc2e50 msgr2=0x555e64fc0740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:49:32.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:32.144+0000 7feb61457740 1 --2- 172.21.15.5:0/4213011492 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555e64fc2e50 0x555e64fc0740 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7feb440cc390 tx=0x7feb4409ab10 comp rx=0 tx=0).stop 2024-09-10T09:49:32.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:32.144+0000 7feb61457740 1 -- 172.21.15.5:0/4213011492 shutdown_connections 2024-09-10T09:49:32.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:32.144+0000 7feb61457740 1 --2- 172.21.15.5:0/4213011492 >> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] conn(0x555e64fd2de0 0x555e64fd5230 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:32.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:32.144+0000 7feb61457740 1 --2- 172.21.15.5:0/4213011492 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555e64fc2e50 0x555e64fc0740 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:32.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:32.144+0000 7feb61457740 1 --2- 172.21.15.5:0/4213011492 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555e64fb8590 0x555e64fc0200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:32.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:32.144+0000 7feb61457740 1 --2- 172.21.15.5:0/4213011492 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555e64fb7c70 0x555e64fbfcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:32.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:32.144+0000 7feb61457740 1 -- 172.21.15.5:0/4213011492 >> 172.21.15.5:0/4213011492 conn(0x555e64fb4200 msgr2=0x555e64fb9f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:49:32.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:32.144+0000 7feb41600640 0 client.27619 ms_handle_reset on v2:172.21.15.102:6832/2457007427 2024-09-10T09:49:32.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:32.145+0000 7feb61457740 1 -- 172.21.15.5:0/4213011492 shutdown_connections 2024-09-10T09:49:32.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:32.145+0000 7feb61457740 1 -- 172.21.15.5:0/4213011492 wait complete. 2024-09-10T09:49:32.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:32.145+0000 7feb61457740 20 client.27619 trim_cache size 0 max 16384 2024-09-10T09:49:32.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:32.164+0000 7feb61457740 1 Processor -- start 2024-09-10T09:49:32.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:32.164+0000 7feb61457740 1 -- start start 2024-09-10T09:49:32.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:32.164+0000 7feb61457740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555e64fb7c70 0x555e64fba770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:49:32.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:32.164+0000 7feb61457740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555e64fb8590 0x555e64fdab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:49:32.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:32.164+0000 7feb61457740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555e64fc2e50 0x555e64fdb040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:49:32.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:32.164+0000 7feb61457740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7feb581b5450 con 0x555e64fb7c70 2024-09-10T09:49:32.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:32.165+0000 7feb61457740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7feb581b6f90 con 0x555e64fc2e50 2024-09-10T09:49:32.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:32.165+0000 7feb61457740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7feb581b5b20 con 0x555e64fb8590 2024-09-10T09:49:32.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:32.165+0000 7feb57e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555e64fc2e50 0x555e64fdb040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:49:32.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:32.165+0000 7feb57e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555e64fc2e50 0x555e64fdb040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:54720/0 (socket says 172.21.15.5:54720) 2024-09-10T09:49:32.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:32.165+0000 7feb57e00640 1 -- 172.21.15.5:0/2693557542 learned_addr learned my addr 172.21.15.5:0/2693557542 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:49:32.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:32.165+0000 7feb57400640 1 --2- 172.21.15.5:0/2693557542 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555e64fb7c70 0x555e64fba770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:49:32.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:32.165+0000 7feb56a00640 1 --2- 172.21.15.5:0/2693557542 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555e64fb8590 0x555e64fdab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:49:32.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:32.166+0000 7feb57e00640 1 -- 172.21.15.5:0/2693557542 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555e64fb8590 msgr2=0x555e64fdab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:49:32.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:32.166+0000 7feb57e00640 1 --2- 172.21.15.5:0/2693557542 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555e64fb8590 0x555e64fdab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:32.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:32.166+0000 7feb57e00640 1 -- 172.21.15.5:0/2693557542 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555e64fb7c70 msgr2=0x555e64fba770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:49:32.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:32.166+0000 7feb57e00640 1 --2- 172.21.15.5:0/2693557542 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555e64fb7c70 0x555e64fba770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:32.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:32.166+0000 7feb57e00640 1 -- 172.21.15.5:0/2693557542 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555e64fb4030 con 0x555e64fc2e50 2024-09-10T09:49:32.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:32.166+0000 7feb56a00640 1 --2- 172.21.15.5:0/2693557542 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555e64fb8590 0x555e64fdab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:49:32.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:32.166+0000 7feb57400640 1 --2- 172.21.15.5:0/2693557542 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555e64fb7c70 0x555e64fba770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:49:32.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:32.166+0000 7feb57e00640 1 --2- 172.21.15.5:0/2693557542 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555e64fc2e50 0x555e64fdb040 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7feb440978d0 tx=0x7feb4409ab80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=551b46dd0f26f0c7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:49:32.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:32.166+0000 7feb43e00640 1 -- 172.21.15.5:0/2693557542 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb440993f0 con 0x555e64fc2e50 2024-09-10T09:49:32.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:32.166+0000 7feb43e00640 1 -- 172.21.15.5:0/2693557542 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feb4409f160 con 0x555e64fc2e50 2024-09-10T09:49:32.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:32.166+0000 7feb43e00640 1 -- 172.21.15.5:0/2693557542 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb4409a7a0 con 0x555e64fc2e50 2024-09-10T09:49:32.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:32.167+0000 7feb61457740 1 -- 172.21.15.5:0/2693557542 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555e64fc2e50 msgr2=0x555e64fdb040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:49:32.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:32.167+0000 7feb61457740 1 --2- 172.21.15.5:0/2693557542 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555e64fc2e50 0x555e64fdb040 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7feb440978d0 tx=0x7feb4409ab80 comp rx=0 tx=0).stop 2024-09-10T09:49:32.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:32.167+0000 7feb61457740 1 -- 172.21.15.5:0/2693557542 shutdown_connections 2024-09-10T09:49:32.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:32.167+0000 7feb61457740 1 --2- 172.21.15.5:0/2693557542 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555e64fc2e50 0x555e64fdb040 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:32.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:32.167+0000 7feb61457740 1 --2- 172.21.15.5:0/2693557542 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555e64fb8590 0x555e64fdab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:32.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:32.167+0000 7feb61457740 1 --2- 172.21.15.5:0/2693557542 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555e64fb7c70 0x555e64fba770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:32.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:32.167+0000 7feb61457740 1 -- 172.21.15.5:0/2693557542 >> 172.21.15.5:0/2693557542 conn(0x555e64fb4200 msgr2=0x555e64fc1d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:49:32.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:32.168+0000 7feb61457740 1 -- 172.21.15.5:0/2693557542 shutdown_connections 2024-09-10T09:49:32.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:32.168+0000 7feb61457740 1 -- 172.21.15.5:0/2693557542 wait complete. 2024-09-10T09:49:32.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:32.168+0000 7feb61457740 1 -- start start 2024-09-10T09:49:32.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:32.168+0000 7feb61457740 1 Processor -- start 2024-09-10T09:49:32.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:32.169+0000 7feb61457740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555e64fb7c70 0x555e64fcae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:49:32.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:32.169+0000 7feb61457740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555e64fb8590 0x555e64fcb360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:49:32.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:32.169+0000 7feb61457740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555e64fc2e50 0x555e64fc7df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:49:32.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:32.169+0000 7feb61457740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7feb581b68c0 con 0x555e64fc2e50 2024-09-10T09:49:32.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:32.169+0000 7feb61457740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7feb581b61f0 con 0x555e64fb8590 2024-09-10T09:49:32.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:32.169+0000 7feb61457740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7feb581b3430 con 0x555e64fb7c70 2024-09-10T09:49:32.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:32.169+0000 7feb61457740 10 client.0 _ll_register_callbacks cb 0x7feb44011ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:49:32.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:32.169+0000 7feb57400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555e64fb7c70 0x555e64fcae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:49:32.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:32.169+0000 7feb57400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555e64fb7c70 0x555e64fcae20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:58422/0 (socket says 172.21.15.5:58422) 2024-09-10T09:49:32.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:32.169+0000 7feb57400640 1 -- 172.21.15.5:0/288580139 learned_addr learned my addr 172.21.15.5:0/288580139 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:49:32.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:32.169+0000 7feb56a00640 1 --2- 172.21.15.5:0/288580139 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555e64fb8590 0x555e64fcb360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:49:32.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:32.170+0000 7feb57e00640 1 --2- 172.21.15.5:0/288580139 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555e64fc2e50 0x555e64fc7df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:49:32.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:32.170+0000 7feb57400640 1 -- 172.21.15.5:0/288580139 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555e64fb8590 msgr2=0x555e64fcb360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:49:32.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:32.170+0000 7feb57400640 1 --2- 172.21.15.5:0/288580139 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555e64fb8590 0x555e64fcb360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:32.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:32.170+0000 7feb57400640 1 -- 172.21.15.5:0/288580139 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555e64fc2e50 msgr2=0x555e64fc7df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:49:32.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:32.170+0000 7feb57400640 1 --2- 172.21.15.5:0/288580139 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555e64fc2e50 0x555e64fc7df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:32.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:32.170+0000 7feb57400640 1 -- 172.21.15.5:0/288580139 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555e64fc8330 con 0x555e64fb7c70 2024-09-10T09:49:32.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:32.170+0000 7feb57e00640 1 --2- 172.21.15.5:0/288580139 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555e64fc2e50 0x555e64fc7df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:49:32.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:32.170+0000 7feb57400640 1 --2- 172.21.15.5:0/288580139 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555e64fb7c70 0x555e64fcae20 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7feb5003bef0 tx=0x7feb5001a5d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=89133427f33fe3b6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:49:32.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:32.170+0000 7feb41600640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-09-10T09:49:32.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:32.171+0000 7feb41600640 1 -- 172.21.15.5:0/288580139 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb50045040 con 0x555e64fb7c70 2024-09-10T09:49:32.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:32.171+0000 7feb61457740 1 -- 172.21.15.5:0/288580139 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feb4409f160 con 0x555e64fb7c70 2024-09-10T09:49:32.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:32.171+0000 7feb41600640 1 -- 172.21.15.5:0/288580139 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7feb50019040 con 0x555e64fb7c70 2024-09-10T09:49:32.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:32.171+0000 7feb41600640 1 -- 172.21.15.5:0/288580139 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7feb50041ad0 con 0x555e64fb7c70 2024-09-10T09:49:32.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:32.171+0000 7feb61457740 10 client.27625 fetch_fsmap learned FSMap version 36 2024-09-10T09:49:32.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:32.171+0000 7feb61457740 1 -- 172.21.15.5:0/288580139 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x555e6503e1a0 con 0x555e64fb7c70 2024-09-10T09:49:32.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:32.172+0000 7feb41600640 1 -- 172.21.15.5:0/288580139 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feb4409f160 con 0x555e64fb7c70 2024-09-10T09:49:32.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:32.172+0000 7feb41600640 1 -- 172.21.15.5:0/288580139 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 36) ==== 8370+0+0 (secure 0 0 0) 0x7feb50054040 con 0x555e64fb7c70 2024-09-10T09:49:32.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:32.172+0000 7feb61457740 10 client.27625 fetch_fsmap finished waiting for FSMap version 36 2024-09-10T09:49:32.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:32.172+0000 7feb61457740 10 client.27625 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-09-10T09:49:32.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:32.172+0000 7feb61457740 20 client.27625 populate_metadata read hostname 'smithi005' 2024-09-10T09:49:32.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:32.172+0000 7feb61457740 1 --2- 172.21.15.5:0/288580139 >> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] conn(0x555e6503e8a0 0x555e65040cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:49:32.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:32.172+0000 7feb61457740 4 client.27625 mds_command: new command op to 24443 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T09:49:32.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:32.172+0000 7feb61457740 1 -- 172.21.15.5:0/288580139 --> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555e64f25e60 con 0x555e6503e8a0 2024-09-10T09:49:32.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:32.172+0000 7feb56a00640 1 --2- 172.21.15.5:0/288580139 >> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] conn(0x555e6503e8a0 0x555e65040cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:49:32.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:32.173+0000 7feb56a00640 1 --2- 172.21.15.5:0/288580139 >> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] conn(0x555e6503e8a0 0x555e65040cc0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=4398e396c6a918f0 server_cookie=877164512d1ec8e3 in_seq=0 out_seq=0 2024-09-10T09:49:32.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:32.173+0000 7feb41600640 10 client.27625 ms_handle_connect on v2:172.21.15.102:6832/2457007427 2024-09-10T09:49:32.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:32.173+0000 7feb41600640 1 -- 172.21.15.5:0/288580139 <== mds.4 v2:172.21.15.102:6832/2457007427 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555e64f25e60 con 0x555e6503e8a0 2024-09-10T09:49:32.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:32.173+0000 7feb41600640 10 client.27625 handle_command_reply: tid=0 2024-09-10T09:49:32.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:32.173+0000 7feb61457740 1 client.27625 shutdown 2024-09-10T09:49:32.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:32.173+0000 7feb61457740 1 -- 172.21.15.5:0/288580139 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555e64fb7c70 msgr2=0x555e64fcae20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:49:32.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:32.173+0000 7feb61457740 1 --2- 172.21.15.5:0/288580139 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555e64fb7c70 0x555e64fcae20 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7feb5003bef0 tx=0x7feb5001a5d0 comp rx=0 tx=0).stop 2024-09-10T09:49:32.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:32.173+0000 7feb61457740 1 -- 172.21.15.5:0/288580139 shutdown_connections 2024-09-10T09:49:32.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:32.173+0000 7feb61457740 1 --2- 172.21.15.5:0/288580139 >> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] conn(0x555e6503e8a0 0x555e65040cc0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:32.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:32.173+0000 7feb61457740 1 --2- 172.21.15.5:0/288580139 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555e64fc2e50 0x555e64fc7df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:32.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:32.173+0000 7feb61457740 1 --2- 172.21.15.5:0/288580139 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555e64fb8590 0x555e64fcb360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:32.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:32.174+0000 7feb41600640 0 client.27625 ms_handle_reset on v2:172.21.15.102:6832/2457007427 2024-09-10T09:49:32.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:32.174+0000 7feb61457740 1 --2- 172.21.15.5:0/288580139 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555e64fb7c70 0x555e64fcae20 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:32.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:32.174+0000 7feb61457740 1 -- 172.21.15.5:0/288580139 >> 172.21.15.5:0/288580139 conn(0x555e64fb4200 msgr2=0x555e64fbfc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:49:32.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:32.174+0000 7feb61457740 1 -- 172.21.15.5:0/288580139 shutdown_connections 2024-09-10T09:49:32.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:32.174+0000 7feb61457740 1 -- 172.21.15.5:0/288580139 wait complete. 2024-09-10T09:49:32.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:32.174+0000 7feb61457740 20 client.27625 trim_cache size 0 max 16384 2024-09-10T09:49:32.179 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-10T09:49:32.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:32.175+0000 7feb5e800640 1 -- 172.21.15.5:0/949020050 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7feb3c0a2c00 msgr2=0x7feb3c0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:49:32.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:32.175+0000 7feb5e800640 1 --2- 172.21.15.5:0/949020050 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7feb3c0a2c00 0x7feb3c0a50c0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7feb50005e30 tx=0x7feb50040510 comp rx=0 tx=0).stop 2024-09-10T09:49:32.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:32.175+0000 7feb5e800640 1 -- 172.21.15.5:0/949020050 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7feb5811b140 msgr2=0x7feb58116e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:49:32.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:32.175+0000 7feb5e800640 1 --2- 172.21.15.5:0/949020050 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7feb5811b140 0x7feb58116e90 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7feb4400b410 tx=0x7feb4400b8d0 comp rx=0 tx=0).stop 2024-09-10T09:49:32.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:32.175+0000 7feb5e800640 1 -- 172.21.15.5:0/949020050 shutdown_connections 2024-09-10T09:49:32.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:32.175+0000 7feb5e800640 1 --2- 172.21.15.5:0/949020050 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7feb3c0a2c00 0x7feb3c0a50c0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:32.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:32.175+0000 7feb5e800640 1 --2- 172.21.15.5:0/949020050 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7feb5811b140 0x7feb58116e90 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:32.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:32.175+0000 7feb5e800640 1 --2- 172.21.15.5:0/949020050 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feb58109830 0x7feb58116950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:32.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:32.175+0000 7feb5e800640 1 --2- 172.21.15.5:0/949020050 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7feb58109450 0x7feb5811a860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:32.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:32.175+0000 7feb5e800640 1 -- 172.21.15.5:0/949020050 >> 172.21.15.5:0/949020050 conn(0x7feb5806e140 msgr2=0x7feb58072300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:49:32.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:32.175+0000 7feb5e800640 1 -- 172.21.15.5:0/949020050 shutdown_connections 2024-09-10T09:49:32.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:32.175+0000 7feb5e800640 1 -- 172.21.15.5:0/949020050 wait complete. 2024-09-10T09:49:33.124 INFO:tasks.workunit.client.0.smithi005.stdout: 1 687246 6.18 MB/sec execute 434 sec latency 16487.533 ms 2024-09-10T09:49:33.135 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-10T09:49:33.514 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:49:33.654 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:49:33 smithi005 ceph-mon[30350]: pgmap v729: 129 pgs: 129 active+clean; 257 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 302 KiB/s wr, 21 op/s 2024-09-10T09:49:33.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:49:33 smithi159 ceph-mon[35513]: pgmap v729: 129 pgs: 129 active+clean; 257 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 302 KiB/s wr, 21 op/s 2024-09-10T09:49:33.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:49:33 smithi102 ceph-mon[35442]: pgmap v729: 129 pgs: 129 active+clean; 257 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 302 KiB/s wr, 21 op/s 2024-09-10T09:49:34.125 INFO:tasks.workunit.client.0.smithi005.stdout: 1 687246 6.17 MB/sec execute 435 sec latency 17487.857 ms 2024-09-10T09:49:34.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:34.180+0000 7f4508800640 1 -- 172.21.15.5:0/344134406 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f45001049a0 msgr2=0x7f45001089f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:49:34.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:34.180+0000 7f4508800640 1 --2- 172.21.15.5:0/344134406 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f45001049a0 0x7f45001089f0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f44f0009900 tx=0x7f44f002a3f0 comp rx=0 tx=0).stop 2024-09-10T09:49:34.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:34.180+0000 7f4508800640 1 -- 172.21.15.5:0/344134406 shutdown_connections 2024-09-10T09:49:34.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:34.180+0000 7f4508800640 1 --2- 172.21.15.5:0/344134406 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4500109120 0x7f4500110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:34.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:34.180+0000 7f4508800640 1 --2- 172.21.15.5:0/344134406 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f45001049a0 0x7f45001089f0 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:34.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:34.180+0000 7f4508800640 1 --2- 172.21.15.5:0/344134406 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4500103ff0 0x7f45001043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:34.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:34.180+0000 7f4508800640 1 -- 172.21.15.5:0/344134406 >> 172.21.15.5:0/344134406 conn(0x7f45000ffa90 msgr2=0x7f4500101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:49:34.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:34.180+0000 7f4508800640 1 -- 172.21.15.5:0/344134406 shutdown_connections 2024-09-10T09:49:34.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:34.180+0000 7f4508800640 1 -- 172.21.15.5:0/344134406 wait complete. 2024-09-10T09:49:34.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:34.182+0000 7f4508800640 1 Processor -- start 2024-09-10T09:49:34.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:34.182+0000 7f4508800640 1 -- start start 2024-09-10T09:49:34.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:34.182+0000 7f4508800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4500103ff0 0x7f450019a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:49:34.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:34.182+0000 7f4508800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f45001049a0 0x7f450019ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:49:34.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:34.182+0000 7f4508800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4500109120 0x7f450019eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:49:34.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:34.182+0000 7f4508800640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f4500114190 con 0x7f45001049a0 2024-09-10T09:49:34.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:34.182+0000 7f4508800640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f4500114010 con 0x7f4500109120 2024-09-10T09:49:34.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:34.182+0000 7f4508800640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f4500114310 con 0x7f4500103ff0 2024-09-10T09:49:34.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:34.183+0000 7f4505400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f45001049a0 0x7f450019ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:49:34.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:34.183+0000 7f4505e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4500103ff0 0x7f450019a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:49:34.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:34.183+0000 7f4505400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f45001049a0 0x7f450019ab50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39888/0 (socket says 172.21.15.5:39888) 2024-09-10T09:49:34.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:34.183+0000 7f4505400640 1 -- 172.21.15.5:0/2508203255 learned_addr learned my addr 172.21.15.5:0/2508203255 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:49:34.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:34.183+0000 7f4506800640 1 --2- 172.21.15.5:0/2508203255 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4500109120 0x7f450019eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:49:34.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:34.183+0000 7f4505e00640 1 -- 172.21.15.5:0/2508203255 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4500109120 msgr2=0x7f450019eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:49:34.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:34.183+0000 7f4505e00640 1 --2- 172.21.15.5:0/2508203255 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4500109120 0x7f450019eee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:34.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:34.183+0000 7f4505e00640 1 -- 172.21.15.5:0/2508203255 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f45001049a0 msgr2=0x7f450019ab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:49:34.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:34.183+0000 7f4505e00640 1 --2- 172.21.15.5:0/2508203255 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f45001049a0 0x7f450019ab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:34.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:34.183+0000 7f4505e00640 1 -- 172.21.15.5:0/2508203255 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f450019f5c0 con 0x7f4500103ff0 2024-09-10T09:49:34.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:34.183+0000 7f4505400640 1 --2- 172.21.15.5:0/2508203255 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f45001049a0 0x7f450019ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:49:34.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:34.183+0000 7f4505e00640 1 --2- 172.21.15.5:0/2508203255 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4500103ff0 0x7f450019a610 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f44f400e9f0 tx=0x7f44f400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:49:34.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:34.184+0000 7f44faa00640 1 -- 172.21.15.5:0/2508203255 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44f400ccc0 con 0x7f4500103ff0 2024-09-10T09:49:34.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:34.184+0000 7f4508800640 1 -- 172.21.15.5:0/2508203255 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f450019f8b0 con 0x7f4500103ff0 2024-09-10T09:49:34.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:34.184+0000 7f44faa00640 1 -- 172.21.15.5:0/2508203255 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f44f4005020 con 0x7f4500103ff0 2024-09-10T09:49:34.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:34.184+0000 7f44faa00640 1 -- 172.21.15.5:0/2508203255 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44f40049e0 con 0x7f4500103ff0 2024-09-10T09:49:34.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:34.184+0000 7f4508800640 1 -- 172.21.15.5:0/2508203255 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45001a7190 con 0x7f4500103ff0 2024-09-10T09:49:34.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:34.187+0000 7f450b1d4740 1 Processor -- start 2024-09-10T09:49:34.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:34.187+0000 7f450b1d4740 1 -- start start 2024-09-10T09:49:34.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:34.187+0000 7f450b1d4740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558621721c70 0x558621722050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:49:34.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:34.187+0000 7f450b1d4740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558621722590 0x55862172c910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:49:34.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:34.187+0000 7f450b1d4740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55862172ce50 0x55862172f240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:49:34.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:34.187+0000 7f450b1d4740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5586216883e0 con 0x55862172ce50 2024-09-10T09:49:34.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:34.187+0000 7f450b1d4740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55862162fd80 con 0x558621721c70 2024-09-10T09:49:34.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:34.187+0000 7f450b1d4740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5586216843a0 con 0x558621722590 2024-09-10T09:49:34.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:34.187+0000 7f44faa00640 1 -- 172.21.15.5:0/2508203255 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f44f4005560 con 0x7f4500103ff0 2024-09-10T09:49:34.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:34.188+0000 7f4505400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55862172ce50 0x55862172f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:49:34.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:34.188+0000 7f4506800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558621721c70 0x558621722050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:49:34.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:34.188+0000 7f4505400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55862172ce50 0x55862172f240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39894/0 (socket says 172.21.15.5:39894) 2024-09-10T09:49:34.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:34.188+0000 7f4505400640 1 -- 172.21.15.5:0/1150401978 learned_addr learned my addr 172.21.15.5:0/1150401978 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:49:34.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:34.188+0000 7f4506800640 1 --2- 172.21.15.5:0/1150401978 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558621721c70 0x558621722050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:54754/0 (socket says 172.21.15.5:54754) 2024-09-10T09:49:34.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:34.188+0000 7f4505e00640 1 --2- 172.21.15.5:0/1150401978 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558621722590 0x55862172c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:49:34.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:34.188+0000 7f4506800640 1 -- 172.21.15.5:0/1150401978 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558621722590 msgr2=0x55862172c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:49:34.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:34.188+0000 7f4506800640 1 --2- 172.21.15.5:0/1150401978 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558621722590 0x55862172c910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:34.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:34.188+0000 7f4506800640 1 -- 172.21.15.5:0/1150401978 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55862172ce50 msgr2=0x55862172f240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:49:34.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:34.188+0000 7f4506800640 1 --2- 172.21.15.5:0/1150401978 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55862172ce50 0x55862172f240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:34.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:34.188+0000 7f4506800640 1 -- 172.21.15.5:0/1150401978 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55862164eae0 con 0x558621721c70 2024-09-10T09:49:34.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:34.188+0000 7f4505400640 1 --2- 172.21.15.5:0/1150401978 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55862172ce50 0x55862172f240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:49:34.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:34.189+0000 7f4506800640 1 --2- 172.21.15.5:0/1150401978 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558621721c70 0x558621722050 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f44fc009f00 tx=0x7f44fc011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bfcfdfa6518bab92 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:49:34.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:34.189+0000 7f44ef000640 1 -- 172.21.15.5:0/1150401978 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44fc004d20 con 0x558621721c70 2024-09-10T09:49:34.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:34.189+0000 7f44ef000640 1 -- 172.21.15.5:0/1150401978 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f44fc004050 con 0x558621721c70 2024-09-10T09:49:34.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:34.189+0000 7f44ef000640 1 -- 172.21.15.5:0/1150401978 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44fc005280 con 0x558621721c70 2024-09-10T09:49:34.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:34.189+0000 7f44faa00640 1 --2- 172.21.15.5:0/2508203255 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f44e80a2c00 0x7f44e80a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:49:34.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:34.189+0000 7f44faa00640 1 -- 172.21.15.5:0/2508203255 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f44f40cf3b0 con 0x7f4500103ff0 2024-09-10T09:49:34.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:34.189+0000 7f4505e00640 1 --2- 172.21.15.5:0/2508203255 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f44e80a2c00 0x7f44e80a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:49:34.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:34.190+0000 7f450b1d4740 1 -- 172.21.15.5:0/1150401978 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558621721c70 msgr2=0x558621722050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:49:34.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:34.190+0000 7f450b1d4740 1 --2- 172.21.15.5:0/1150401978 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558621721c70 0x558621722050 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f44fc009f00 tx=0x7f44fc011810 comp rx=0 tx=0).stop 2024-09-10T09:49:34.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:34.190+0000 7f450b1d4740 1 -- 172.21.15.5:0/1150401978 shutdown_connections 2024-09-10T09:49:34.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:34.190+0000 7f450b1d4740 1 --2- 172.21.15.5:0/1150401978 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55862172ce50 0x55862172f240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:34.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:34.190+0000 7f450b1d4740 1 --2- 172.21.15.5:0/1150401978 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558621722590 0x55862172c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:34.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:34.190+0000 7f450b1d4740 1 --2- 172.21.15.5:0/1150401978 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558621721c70 0x558621722050 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:34.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:34.190+0000 7f450b1d4740 1 -- 172.21.15.5:0/1150401978 >> 172.21.15.5:0/1150401978 conn(0x55862171e200 msgr2=0x55862171e5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:49:34.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:34.190+0000 7f450b1d4740 1 -- 172.21.15.5:0/1150401978 shutdown_connections 2024-09-10T09:49:34.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:34.190+0000 7f450b1d4740 1 -- 172.21.15.5:0/1150401978 wait complete. 2024-09-10T09:49:34.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:34.190+0000 7f4505e00640 1 --2- 172.21.15.5:0/2508203255 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f44e80a2c00 0x7f44e80a50c0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f44f40cca60 tx=0x7f44f40054d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:49:34.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:34.190+0000 7f450b1d4740 1 -- start start 2024-09-10T09:49:34.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:34.191+0000 7f450b1d4740 1 Processor -- start 2024-09-10T09:49:34.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:34.191+0000 7f450b1d4740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558621721c70 0x558621729d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:49:34.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:34.191+0000 7f450b1d4740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558621722590 0x55862172a260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:49:34.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:34.191+0000 7f450b1d4740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55862172ce50 0x55862172a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:49:34.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:34.191+0000 7f450b1d4740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55862172fa20 con 0x55862172ce50 2024-09-10T09:49:34.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:34.191+0000 7f450b1d4740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x5586216430f0 con 0x558621721c70 2024-09-10T09:49:34.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:34.191+0000 7f450b1d4740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55862172fba0 con 0x558621722590 2024-09-10T09:49:34.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:34.191+0000 7f450b1d4740 10 client.0 _ll_register_callbacks cb 0x55862162abd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:49:34.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:34.192+0000 7f4506800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558621721c70 0x558621729d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:49:34.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:34.192+0000 7f4506800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558621721c70 0x558621729d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:54758/0 (socket says 172.21.15.5:54758) 2024-09-10T09:49:34.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:34.192+0000 7f4506800640 1 -- 172.21.15.5:0/2042807289 learned_addr learned my addr 172.21.15.5:0/2042807289 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:49:34.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:34.192+0000 7f4505400640 1 --2- 172.21.15.5:0/2042807289 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558621722590 0x55862172a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:49:34.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:34.192+0000 7f4506800640 1 --2- 172.21.15.5:0/2042807289 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55862172ce50 0x55862172a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:49:34.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:34.192+0000 7f4506800640 1 -- 172.21.15.5:0/2042807289 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558621722590 msgr2=0x55862172a260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:49:34.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:34.192+0000 7f4506800640 1 --2- 172.21.15.5:0/2042807289 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558621722590 0x55862172a260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:34.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:34.192+0000 7f4506800640 1 -- 172.21.15.5:0/2042807289 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55862172ce50 msgr2=0x55862172a7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:49:34.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:34.192+0000 7f4506800640 1 --2- 172.21.15.5:0/2042807289 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55862172ce50 0x55862172a7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:34.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:34.192+0000 7f4506800640 1 -- 172.21.15.5:0/2042807289 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5586213f93c0 con 0x558621721c70 2024-09-10T09:49:34.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:34.192+0000 7f4506800640 1 --2- 172.21.15.5:0/2042807289 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558621721c70 0x558621729d20 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f44fc004780 tx=0x7f44fc0046f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f4f079395b8f4481 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:49:34.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:34.192+0000 7f44e3400640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-09-10T09:49:34.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:34.193+0000 7f44e3400640 1 -- 172.21.15.5:0/2042807289 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44fc02c020 con 0x558621721c70 2024-09-10T09:49:34.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:34.193+0000 7f44e3400640 1 -- 172.21.15.5:0/2042807289 <== mon.2 v2:172.21.15.102:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f44fc004a40 con 0x558621721c70 2024-09-10T09:49:34.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:34.193+0000 7f450b1d4740 1 -- 172.21.15.5:0/2042807289 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f44fc004050 con 0x558621721c70 2024-09-10T09:49:34.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:34.193+0000 7f44e3400640 1 -- 172.21.15.5:0/2042807289 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f44fc037050 con 0x558621721c70 2024-09-10T09:49:34.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:34.193+0000 7f450b1d4740 10 client.27626 fetch_fsmap learned FSMap version 36 2024-09-10T09:49:34.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:34.193+0000 7f450b1d4740 1 -- 172.21.15.5:0/2042807289 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55862173ba40 con 0x558621721c70 2024-09-10T09:49:34.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:34.193+0000 7f44e3400640 1 -- 172.21.15.5:0/2042807289 <== mon.2 v2:172.21.15.102:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f44fc004050 con 0x558621721c70 2024-09-10T09:49:34.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:34.194+0000 7f44e3400640 1 -- 172.21.15.5:0/2042807289 <== mon.2 v2:172.21.15.102:3300/0 5 ==== fsmap(e 36) ==== 8370+0+0 (secure 0 0 0) 0x7f44fc028930 con 0x558621721c70 2024-09-10T09:49:34.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:34.194+0000 7f450b1d4740 10 client.27626 fetch_fsmap finished waiting for FSMap version 36 2024-09-10T09:49:34.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:34.194+0000 7f450b1d4740 10 client.27626 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-09-10T09:49:34.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:34.194+0000 7f450b1d4740 20 client.27626 populate_metadata read hostname 'smithi005' 2024-09-10T09:49:34.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:34.194+0000 7f450b1d4740 1 --2- 172.21.15.5:0/2042807289 >> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] conn(0x55862173cea0 0x55862173f290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:49:34.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:34.194+0000 7f450b1d4740 4 client.27626 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:49:34.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:34.194+0000 7f450b1d4740 1 -- 172.21.15.5:0/2042807289 --> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558621695950 con 0x55862173cea0 2024-09-10T09:49:34.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:34.194+0000 7f4505400640 1 --2- 172.21.15.5:0/2042807289 >> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] conn(0x55862173cea0 0x55862173f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:49:34.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:34.195+0000 7f4505400640 1 --2- 172.21.15.5:0/2042807289 >> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] conn(0x55862173cea0 0x55862173f290 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=144e8b721012cda server_cookie=9e3d87833a23e45f in_seq=0 out_seq=0 2024-09-10T09:49:34.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:34.195+0000 7f44e3400640 10 client.27626 ms_handle_connect on v2:172.21.15.102:6836/2689776764 2024-09-10T09:49:34.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:34.195+0000 7f44e3400640 1 -- 172.21.15.5:0/2042807289 <== mds.3 v2:172.21.15.102:6836/2689776764 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558621695950 con 0x55862173cea0 2024-09-10T09:49:34.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:34.195+0000 7f44e3400640 10 client.27626 handle_command_reply: tid=0 2024-09-10T09:49:34.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:34.195+0000 7f450b1d4740 1 client.27626 shutdown 2024-09-10T09:49:34.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:34.196+0000 7f450b1d4740 1 -- 172.21.15.5:0/2042807289 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558621721c70 msgr2=0x558621729d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:49:34.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:34.196+0000 7f450b1d4740 1 --2- 172.21.15.5:0/2042807289 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558621721c70 0x558621729d20 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f44fc004780 tx=0x7f44fc0046f0 comp rx=0 tx=0).stop 2024-09-10T09:49:34.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:34.196+0000 7f450b1d4740 1 -- 172.21.15.5:0/2042807289 shutdown_connections 2024-09-10T09:49:34.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:34.196+0000 7f450b1d4740 1 --2- 172.21.15.5:0/2042807289 >> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] conn(0x55862173cea0 0x55862173f290 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:34.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:34.196+0000 7f450b1d4740 1 --2- 172.21.15.5:0/2042807289 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55862172ce50 0x55862172a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:34.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:34.196+0000 7f450b1d4740 1 --2- 172.21.15.5:0/2042807289 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558621722590 0x55862172a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:34.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:34.196+0000 7f450b1d4740 1 --2- 172.21.15.5:0/2042807289 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558621721c70 0x558621729d20 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:34.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:34.196+0000 7f450b1d4740 1 -- 172.21.15.5:0/2042807289 >> 172.21.15.5:0/2042807289 conn(0x55862171e200 msgr2=0x55862171d380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:49:34.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:34.196+0000 7f44e3400640 0 client.27626 ms_handle_reset on v2:172.21.15.102:6836/2689776764 2024-09-10T09:49:34.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:34.196+0000 7f450b1d4740 1 -- 172.21.15.5:0/2042807289 shutdown_connections 2024-09-10T09:49:34.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:34.196+0000 7f450b1d4740 1 -- 172.21.15.5:0/2042807289 wait complete. 2024-09-10T09:49:34.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:34.196+0000 7f450b1d4740 20 client.27626 trim_cache size 0 max 16384 2024-09-10T09:49:34.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:34.210+0000 7f450b1d4740 1 Processor -- start 2024-09-10T09:49:34.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:34.210+0000 7f450b1d4740 1 -- start start 2024-09-10T09:49:34.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:34.210+0000 7f450b1d4740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558621721c70 0x55862171e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:49:34.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:34.210+0000 7f450b1d4740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558621722590 0x55862172a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:49:34.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:34.210+0000 7f450b1d4740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55862172ce50 0x558621744a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:49:34.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:34.210+0000 7f450b1d4740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f45001b1020 con 0x558621722590 2024-09-10T09:49:34.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:34.210+0000 7f450b1d4740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f45001b2b60 con 0x558621721c70 2024-09-10T09:49:34.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:34.210+0000 7f450b1d4740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f45001b16f0 con 0x55862172ce50 2024-09-10T09:49:34.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:34.210+0000 7f4505400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558621722590 0x55862172a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:49:34.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:34.210+0000 7f4505400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558621722590 0x55862172a5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39904/0 (socket says 172.21.15.5:39904) 2024-09-10T09:49:34.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:34.210+0000 7f4505400640 1 -- 172.21.15.5:0/3265015658 learned_addr learned my addr 172.21.15.5:0/3265015658 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:49:34.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:34.210+0000 7f4506800640 1 --2- 172.21.15.5:0/3265015658 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558621721c70 0x55862171e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:49:34.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:34.211+0000 7f4506800640 1 --2- 172.21.15.5:0/3265015658 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55862172ce50 0x558621744a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:49:34.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:34.211+0000 7f4505400640 1 -- 172.21.15.5:0/3265015658 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55862172ce50 msgr2=0x558621744a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:49:34.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:34.211+0000 7f4505400640 1 --2- 172.21.15.5:0/3265015658 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55862172ce50 0x558621744a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:34.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:34.211+0000 7f4505400640 1 -- 172.21.15.5:0/3265015658 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558621721c70 msgr2=0x55862171e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:49:34.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:34.211+0000 7f4505400640 1 --2- 172.21.15.5:0/3265015658 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558621721c70 0x55862171e5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:34.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:34.211+0000 7f4505400640 1 -- 172.21.15.5:0/3265015658 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558621728cd0 con 0x558621722590 2024-09-10T09:49:34.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:34.211+0000 7f4506800640 1 --2- 172.21.15.5:0/3265015658 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558621721c70 0x55862171e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:49:34.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:34.211+0000 7f4505400640 1 --2- 172.21.15.5:0/3265015658 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558621722590 0x55862172a5f0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x55862171f580 tx=0x7f44f00050f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f29636a2fd64ff16 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:49:34.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:34.211+0000 7f44ee600640 1 -- 172.21.15.5:0/3265015658 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44f0035040 con 0x558621722590 2024-09-10T09:49:34.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:34.212+0000 7f44ee600640 1 -- 172.21.15.5:0/3265015658 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f44f0004c50 con 0x558621722590 2024-09-10T09:49:34.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:34.212+0000 7f44ee600640 1 -- 172.21.15.5:0/3265015658 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44f0031a50 con 0x558621722590 2024-09-10T09:49:34.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:34.212+0000 7f450b1d4740 1 -- 172.21.15.5:0/3265015658 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558621722590 msgr2=0x55862172a5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:49:34.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:34.212+0000 7f450b1d4740 1 --2- 172.21.15.5:0/3265015658 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558621722590 0x55862172a5f0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x55862171f580 tx=0x7f44f00050f0 comp rx=0 tx=0).stop 2024-09-10T09:49:34.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:34.212+0000 7f450b1d4740 1 -- 172.21.15.5:0/3265015658 shutdown_connections 2024-09-10T09:49:34.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:34.212+0000 7f450b1d4740 1 --2- 172.21.15.5:0/3265015658 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55862172ce50 0x558621744a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:34.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:34.212+0000 7f450b1d4740 1 --2- 172.21.15.5:0/3265015658 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558621722590 0x55862172a5f0 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:34.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:34.212+0000 7f450b1d4740 1 --2- 172.21.15.5:0/3265015658 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558621721c70 0x55862171e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:34.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:34.212+0000 7f450b1d4740 1 -- 172.21.15.5:0/3265015658 >> 172.21.15.5:0/3265015658 conn(0x55862171e200 msgr2=0x55862172bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:49:34.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:34.212+0000 7f450b1d4740 1 -- 172.21.15.5:0/3265015658 shutdown_connections 2024-09-10T09:49:34.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:34.212+0000 7f450b1d4740 1 -- 172.21.15.5:0/3265015658 wait complete. 2024-09-10T09:49:34.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:34.213+0000 7f450b1d4740 1 -- start start 2024-09-10T09:49:34.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:34.213+0000 7f450b1d4740 1 Processor -- start 2024-09-10T09:49:34.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:34.214+0000 7f450b1d4740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558621721c70 0x5586217355e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:49:34.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:34.214+0000 7f450b1d4740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558621722590 0x558621730910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:49:34.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:34.214+0000 7f450b1d4740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55862172ce50 0x558621730e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:49:34.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:34.214+0000 7f450b1d4740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f45001b2490 con 0x558621722590 2024-09-10T09:49:34.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:34.214+0000 7f450b1d4740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f45001b1dc0 con 0x55862172ce50 2024-09-10T09:49:34.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:34.214+0000 7f450b1d4740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f45001af000 con 0x558621721c70 2024-09-10T09:49:34.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:34.214+0000 7f450b1d4740 10 client.0 _ll_register_callbacks cb 0x7f44f4005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:49:34.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:34.214+0000 7f4505400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558621722590 0x558621730910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:49:34.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:34.214+0000 7f4505400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558621722590 0x558621730910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39916/0 (socket says 172.21.15.5:39916) 2024-09-10T09:49:34.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:34.214+0000 7f4505400640 1 -- 172.21.15.5:0/360035407 learned_addr learned my addr 172.21.15.5:0/360035407 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:49:34.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:34.214+0000 7f4506800640 1 --2- 172.21.15.5:0/360035407 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558621721c70 0x5586217355e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:49:34.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:34.214+0000 7f4506800640 1 --2- 172.21.15.5:0/360035407 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55862172ce50 0x558621730e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:49:34.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:34.214+0000 7f4505400640 1 -- 172.21.15.5:0/360035407 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558621721c70 msgr2=0x5586217355e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:49:34.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:34.214+0000 7f4505400640 1 --2- 172.21.15.5:0/360035407 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558621721c70 0x5586217355e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:34.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:34.214+0000 7f4505400640 1 -- 172.21.15.5:0/360035407 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55862172ce50 msgr2=0x558621730e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:49:34.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:34.214+0000 7f4505400640 1 --2- 172.21.15.5:0/360035407 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55862172ce50 0x558621730e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:34.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:34.214+0000 7f4505400640 1 -- 172.21.15.5:0/360035407 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5586217223c0 con 0x558621722590 2024-09-10T09:49:34.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:34.215+0000 7f4505400640 1 --2- 172.21.15.5:0/360035407 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558621722590 0x558621730910 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f44f002ab70 tx=0x7f44f002aba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1af70a45b5ab278c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:49:34.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:34.215+0000 7f44e3400640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-10T09:49:34.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:34.215+0000 7f44e3400640 1 -- 172.21.15.5:0/360035407 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44f0053030 con 0x558621722590 2024-09-10T09:49:34.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:34.215+0000 7f450b1d4740 1 -- 172.21.15.5:0/360035407 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f44f0004c50 con 0x558621722590 2024-09-10T09:49:34.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:34.215+0000 7f44e3400640 1 -- 172.21.15.5:0/360035407 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f44f0004770 con 0x558621722590 2024-09-10T09:49:34.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:34.215+0000 7f44e3400640 1 -- 172.21.15.5:0/360035407 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f44f0060050 con 0x558621722590 2024-09-10T09:49:34.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:34.215+0000 7f450b1d4740 10 client.17619 fetch_fsmap learned FSMap version 36 2024-09-10T09:49:34.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:34.215+0000 7f450b1d4740 1 -- 172.21.15.5:0/360035407 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x5586217a78c0 con 0x558621722590 2024-09-10T09:49:34.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:34.216+0000 7f44e3400640 1 -- 172.21.15.5:0/360035407 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f44f0004c50 con 0x558621722590 2024-09-10T09:49:34.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:34.216+0000 7f44e3400640 1 -- 172.21.15.5:0/360035407 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 36) ==== 8370+0+0 (secure 0 0 0) 0x7f44f00035f0 con 0x558621722590 2024-09-10T09:49:34.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:34.216+0000 7f450b1d4740 10 client.17619 fetch_fsmap finished waiting for FSMap version 36 2024-09-10T09:49:34.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:34.216+0000 7f450b1d4740 10 client.17619 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-09-10T09:49:34.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:34.216+0000 7f450b1d4740 20 client.17619 populate_metadata read hostname 'smithi005' 2024-09-10T09:49:34.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:34.216+0000 7f450b1d4740 1 --2- 172.21.15.5:0/360035407 >> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] conn(0x5586217a7ed0 0x5586217aa2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:49:34.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:34.216+0000 7f450b1d4740 4 client.17619 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T09:49:34.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:34.216+0000 7f450b1d4740 1 -- 172.21.15.5:0/360035407 --> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55862168fe60 con 0x5586217a7ed0 2024-09-10T09:49:34.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:34.216+0000 7f4506800640 1 --2- 172.21.15.5:0/360035407 >> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] conn(0x5586217a7ed0 0x5586217aa2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:49:34.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:34.217+0000 7f4506800640 1 --2- 172.21.15.5:0/360035407 >> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] conn(0x5586217a7ed0 0x5586217aa2f0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=fd15fd4168f78f2c server_cookie=384579f7517fa510 in_seq=0 out_seq=0 2024-09-10T09:49:34.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:34.217+0000 7f44e3400640 10 client.17619 ms_handle_connect on v2:172.21.15.102:6836/2689776764 2024-09-10T09:49:34.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:34.217+0000 7f44e3400640 1 -- 172.21.15.5:0/360035407 <== mds.3 v2:172.21.15.102:6836/2689776764 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55862168fe60 con 0x5586217a7ed0 2024-09-10T09:49:34.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:34.217+0000 7f44e3400640 10 client.17619 handle_command_reply: tid=0 2024-09-10T09:49:34.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:34.217+0000 7f450b1d4740 1 client.17619 shutdown 2024-09-10T09:49:34.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:34.217+0000 7f450b1d4740 1 -- 172.21.15.5:0/360035407 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558621722590 msgr2=0x558621730910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:49:34.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:34.217+0000 7f450b1d4740 1 --2- 172.21.15.5:0/360035407 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558621722590 0x558621730910 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f44f002ab70 tx=0x7f44f002aba0 comp rx=0 tx=0).stop 2024-09-10T09:49:34.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:34.217+0000 7f450b1d4740 1 -- 172.21.15.5:0/360035407 shutdown_connections 2024-09-10T09:49:34.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:34.217+0000 7f450b1d4740 1 --2- 172.21.15.5:0/360035407 >> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] conn(0x5586217a7ed0 0x5586217aa2f0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:34.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:34.217+0000 7f450b1d4740 1 --2- 172.21.15.5:0/360035407 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55862172ce50 0x558621730e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:34.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:34.217+0000 7f450b1d4740 1 --2- 172.21.15.5:0/360035407 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558621722590 0x558621730910 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:34.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:34.217+0000 7f450b1d4740 1 --2- 172.21.15.5:0/360035407 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558621721c70 0x5586217355e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:34.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:34.217+0000 7f44e3400640 0 client.17619 ms_handle_reset on v2:172.21.15.102:6836/2689776764 2024-09-10T09:49:34.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:34.217+0000 7f450b1d4740 1 -- 172.21.15.5:0/360035407 >> 172.21.15.5:0/360035407 conn(0x55862171e200 msgr2=0x55862171cb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:49:34.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:34.217+0000 7f450b1d4740 1 -- 172.21.15.5:0/360035407 shutdown_connections 2024-09-10T09:49:34.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:34.217+0000 7f450b1d4740 1 -- 172.21.15.5:0/360035407 wait complete. 2024-09-10T09:49:34.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:34.218+0000 7f450b1d4740 20 client.17619 trim_cache size 0 max 16384 2024-09-10T09:49:34.223 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-10T09:49:34.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:34.218+0000 7f4508800640 1 -- 172.21.15.5:0/2508203255 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f44e80a2c00 msgr2=0x7f44e80a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:49:34.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:34.218+0000 7f4508800640 1 --2- 172.21.15.5:0/2508203255 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f44e80a2c00 0x7f44e80a50c0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f44f40cca60 tx=0x7f44f40054d0 comp rx=0 tx=0).stop 2024-09-10T09:49:34.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:34.218+0000 7f4508800640 1 -- 172.21.15.5:0/2508203255 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4500103ff0 msgr2=0x7f450019a610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:49:34.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:34.218+0000 7f4508800640 1 --2- 172.21.15.5:0/2508203255 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4500103ff0 0x7f450019a610 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f44f400e9f0 tx=0x7f44f400eeb0 comp rx=0 tx=0).stop 2024-09-10T09:49:34.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:34.218+0000 7f4508800640 1 -- 172.21.15.5:0/2508203255 shutdown_connections 2024-09-10T09:49:34.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:34.218+0000 7f4508800640 1 --2- 172.21.15.5:0/2508203255 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f44e80a2c00 0x7f44e80a50c0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:34.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:34.218+0000 7f4508800640 1 --2- 172.21.15.5:0/2508203255 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4500109120 0x7f450019eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:34.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:34.218+0000 7f4508800640 1 --2- 172.21.15.5:0/2508203255 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f45001049a0 0x7f450019ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:34.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:34.218+0000 7f4508800640 1 --2- 172.21.15.5:0/2508203255 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4500103ff0 0x7f450019a610 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:34.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:34.218+0000 7f4508800640 1 -- 172.21.15.5:0/2508203255 >> 172.21.15.5:0/2508203255 conn(0x7f45000ffa90 msgr2=0x7f4500100d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:49:34.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:34.218+0000 7f4508800640 1 -- 172.21.15.5:0/2508203255 shutdown_connections 2024-09-10T09:49:34.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:34.218+0000 7f4508800640 1 -- 172.21.15.5:0/2508203255 wait complete. 2024-09-10T09:49:35.085 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-10T09:49:35.124 INFO:tasks.workunit.client.0.smithi005.stdout: 1 687246 6.16 MB/sec execute 436 sec latency 18487.917 ms 2024-09-10T09:49:35.464 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:49:35.778 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:49:35 smithi005 ceph-mon[30350]: pgmap v730: 129 pgs: 129 active+clean; 256 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 290 KiB/s wr, 20 op/s 2024-09-10T09:49:35.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:49:35 smithi159 ceph-mon[35513]: pgmap v730: 129 pgs: 129 active+clean; 256 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 290 KiB/s wr, 20 op/s 2024-09-10T09:49:35.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:49:35 smithi102 ceph-mon[35442]: pgmap v730: 129 pgs: 129 active+clean; 256 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 290 KiB/s wr, 20 op/s 2024-09-10T09:49:36.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:36.011+0000 7f766fa00640 1 -- 172.21.15.5:0/463223095 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7668103ff0 msgr2=0x7f76681043d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:49:36.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:36.011+0000 7f766fa00640 1 --2- 172.21.15.5:0/463223095 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7668103ff0 0x7f76681043d0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f765c009900 tx=0x7f765c02a3f0 comp rx=0 tx=0).stop 2024-09-10T09:49:36.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:36.011+0000 7f766fa00640 1 -- 172.21.15.5:0/463223095 shutdown_connections 2024-09-10T09:49:36.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:36.011+0000 7f766fa00640 1 --2- 172.21.15.5:0/463223095 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7668109120 0x7f7668110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:36.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:36.011+0000 7f766fa00640 1 --2- 172.21.15.5:0/463223095 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f76681049a0 0x7f76681089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:36.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:36.011+0000 7f766fa00640 1 --2- 172.21.15.5:0/463223095 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7668103ff0 0x7f76681043d0 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:36.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:36.011+0000 7f766fa00640 1 -- 172.21.15.5:0/463223095 >> 172.21.15.5:0/463223095 conn(0x7f76680ffa90 msgr2=0x7f7668101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:49:36.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:36.012+0000 7f766fa00640 1 -- 172.21.15.5:0/463223095 shutdown_connections 2024-09-10T09:49:36.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:36.012+0000 7f766fa00640 1 -- 172.21.15.5:0/463223095 wait complete. 2024-09-10T09:49:36.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:36.012+0000 7f766fa00640 1 Processor -- start 2024-09-10T09:49:36.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:36.013+0000 7f766fa00640 1 -- start start 2024-09-10T09:49:36.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:36.013+0000 7f766fa00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7668103ff0 0x7f766819a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:49:36.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:36.013+0000 7f766fa00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f76681049a0 0x7f766819ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:49:36.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:36.013+0000 7f766fa00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7668109120 0x7f766819eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:49:36.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:36.013+0000 7f766fa00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f7668114150 con 0x7f7668109120 2024-09-10T09:49:36.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:36.013+0000 7f766fa00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f7668113fd0 con 0x7f7668103ff0 2024-09-10T09:49:36.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:36.013+0000 7f766fa00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f76681142d0 con 0x7f76681049a0 2024-09-10T09:49:36.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:36.013+0000 7f766da00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7668109120 0x7f766819eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:49:36.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:36.013+0000 7f766da00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7668109120 0x7f766819eec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39924/0 (socket says 172.21.15.5:39924) 2024-09-10T09:49:36.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:36.013+0000 7f766da00640 1 -- 172.21.15.5:0/408129624 learned_addr learned my addr 172.21.15.5:0/408129624 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:49:36.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:36.013+0000 7f7667e00640 1 --2- 172.21.15.5:0/408129624 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f76681049a0 0x7f766819ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:49:36.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:36.013+0000 7f766d000640 1 --2- 172.21.15.5:0/408129624 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7668103ff0 0x7f766819a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:49:36.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:36.014+0000 7f766da00640 1 -- 172.21.15.5:0/408129624 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f76681049a0 msgr2=0x7f766819ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:49:36.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:36.014+0000 7f766da00640 1 --2- 172.21.15.5:0/408129624 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f76681049a0 0x7f766819ab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:36.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:36.014+0000 7f766da00640 1 -- 172.21.15.5:0/408129624 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7668103ff0 msgr2=0x7f766819a5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:49:36.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:36.014+0000 7f766da00640 1 --2- 172.21.15.5:0/408129624 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7668103ff0 0x7f766819a5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:36.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:36.014+0000 7f766da00640 1 -- 172.21.15.5:0/408129624 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f766819f5a0 con 0x7f7668109120 2024-09-10T09:49:36.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:36.014+0000 7f766da00640 1 --2- 172.21.15.5:0/408129624 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7668109120 0x7f766819eec0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f766000eb10 tx=0x7f766000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:49:36.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:36.014+0000 7f7665600640 1 -- 172.21.15.5:0/408129624 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7660004b20 con 0x7f7668109120 2024-09-10T09:49:36.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:36.014+0000 7f7665600640 1 -- 172.21.15.5:0/408129624 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7660004cc0 con 0x7f7668109120 2024-09-10T09:49:36.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:36.014+0000 7f7665600640 1 -- 172.21.15.5:0/408129624 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7660010760 con 0x7f7668109120 2024-09-10T09:49:36.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:36.014+0000 7f766fa00640 1 -- 172.21.15.5:0/408129624 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f766819f890 con 0x7f7668109120 2024-09-10T09:49:36.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:36.014+0000 7f766fa00640 1 -- 172.21.15.5:0/408129624 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76681a7170 con 0x7f7668109120 2024-09-10T09:49:36.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:36.017+0000 7f767249c740 1 Processor -- start 2024-09-10T09:49:36.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:36.017+0000 7f7665600640 1 -- 172.21.15.5:0/408129624 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f7660005020 con 0x7f7668109120 2024-09-10T09:49:36.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:36.018+0000 7f767249c740 1 -- start start 2024-09-10T09:49:36.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:36.018+0000 7f767249c740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e9224f3c70 0x55e9224f4050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:49:36.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:36.018+0000 7f767249c740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e9224f4590 0x55e9224fe910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:49:36.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:36.018+0000 7f767249c740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e9224fee50 0x55e922501240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:49:36.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:36.018+0000 7f767249c740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55e92245a3e0 con 0x55e9224f4590 2024-09-10T09:49:36.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:36.018+0000 7f767249c740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55e922401d80 con 0x55e9224f3c70 2024-09-10T09:49:36.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:36.018+0000 7f767249c740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55e9224563a0 con 0x55e9224fee50 2024-09-10T09:49:36.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:36.018+0000 7f766d000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e9224f4590 0x55e9224fe910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:49:36.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:36.018+0000 7f7667e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e9224f3c70 0x55e9224f4050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:49:36.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:36.018+0000 7f766d000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e9224f4590 0x55e9224fe910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39928/0 (socket says 172.21.15.5:39928) 2024-09-10T09:49:36.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:36.018+0000 7f766d000640 1 -- 172.21.15.5:0/1298521662 learned_addr learned my addr 172.21.15.5:0/1298521662 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:49:36.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:36.019+0000 7f7667e00640 1 --2- 172.21.15.5:0/1298521662 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e9224fee50 0x55e922501240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:49:36.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:36.019+0000 7f766d000640 1 -- 172.21.15.5:0/1298521662 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e9224fee50 msgr2=0x55e922501240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:49:36.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:36.019+0000 7f766d000640 1 --2- 172.21.15.5:0/1298521662 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e9224fee50 0x55e922501240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:36.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:36.019+0000 7f766d000640 1 -- 172.21.15.5:0/1298521662 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e9224f3c70 msgr2=0x55e9224f4050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:49:36.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:36.019+0000 7f766d000640 1 --2- 172.21.15.5:0/1298521662 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e9224f3c70 0x55e9224f4050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:36.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:36.019+0000 7f766d000640 1 -- 172.21.15.5:0/1298521662 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e922420ae0 con 0x55e9224f4590 2024-09-10T09:49:36.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:36.019+0000 7f7665600640 1 --2- 172.21.15.5:0/408129624 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f76500a2c00 0x7f76500a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:49:36.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:36.019+0000 7f766d000640 1 --2- 172.21.15.5:0/1298521662 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e9224f4590 0x55e9224fe910 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f765c009900 tx=0x7f765c005be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a90880443620c1d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:49:36.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:36.019+0000 7f7665600640 1 -- 172.21.15.5:0/408129624 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f76600cf9e0 con 0x7f7668109120 2024-09-10T09:49:36.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:36.019+0000 7f7667e00640 1 --2- 172.21.15.5:0/408129624 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f76500a2c00 0x7f76500a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:49:36.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:36.020+0000 7f765a000640 1 -- 172.21.15.5:0/1298521662 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f765c040070 con 0x55e9224f4590 2024-09-10T09:49:36.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:36.020+0000 7f765a000640 1 -- 172.21.15.5:0/1298521662 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f765c031870 con 0x55e9224f4590 2024-09-10T09:49:36.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:36.020+0000 7f765a000640 1 -- 172.21.15.5:0/1298521662 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f765c035040 con 0x55e9224f4590 2024-09-10T09:49:36.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:36.020+0000 7f767249c740 1 -- 172.21.15.5:0/1298521662 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e9224f4590 msgr2=0x55e9224fe910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:49:36.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:36.020+0000 7f767249c740 1 --2- 172.21.15.5:0/1298521662 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e9224f4590 0x55e9224fe910 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f765c009900 tx=0x7f765c005be0 comp rx=0 tx=0).stop 2024-09-10T09:49:36.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:36.020+0000 7f7667e00640 1 --2- 172.21.15.5:0/408129624 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f76500a2c00 0x7f76500a50c0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x55e9224f1580 tx=0x7f76540096c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:49:36.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:36.020+0000 7f767249c740 1 -- 172.21.15.5:0/1298521662 shutdown_connections 2024-09-10T09:49:36.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:36.020+0000 7f767249c740 1 --2- 172.21.15.5:0/1298521662 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e9224fee50 0x55e922501240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:36.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:36.020+0000 7f767249c740 1 --2- 172.21.15.5:0/1298521662 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e9224f4590 0x55e9224fe910 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:36.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:36.020+0000 7f767249c740 1 --2- 172.21.15.5:0/1298521662 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e9224f3c70 0x55e9224f4050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:36.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:36.020+0000 7f767249c740 1 -- 172.21.15.5:0/1298521662 >> 172.21.15.5:0/1298521662 conn(0x55e9224f0200 msgr2=0x55e9224f05e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:49:36.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:36.021+0000 7f767249c740 1 -- 172.21.15.5:0/1298521662 shutdown_connections 2024-09-10T09:49:36.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:36.021+0000 7f767249c740 1 -- 172.21.15.5:0/1298521662 wait complete. 2024-09-10T09:49:36.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:36.021+0000 7f767249c740 1 -- start start 2024-09-10T09:49:36.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:36.021+0000 7f767249c740 1 Processor -- start 2024-09-10T09:49:36.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:36.022+0000 7f767249c740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e9224f3c70 0x55e9224fbd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:49:36.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:36.022+0000 7f767249c740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e9224f4590 0x55e9224fc250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:49:36.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:36.022+0000 7f767249c740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e9224fee50 0x55e9224fc790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:49:36.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:36.022+0000 7f767249c740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55e922501a70 con 0x55e9224fee50 2024-09-10T09:49:36.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:36.022+0000 7f767249c740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55e9224150f0 con 0x55e9224f3c70 2024-09-10T09:49:36.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:36.022+0000 7f767249c740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55e922501bf0 con 0x55e9224f4590 2024-09-10T09:49:36.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:36.022+0000 7f767249c740 10 client.0 _ll_register_callbacks cb 0x55e9223fcbd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:49:36.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:36.022+0000 7f766da00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e9224fee50 0x55e9224fc790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:49:36.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:36.022+0000 7f7667e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e9224f4590 0x55e9224fc250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:49:36.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:36.022+0000 7f7667e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e9224f4590 0x55e9224fc250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:58520/0 (socket says 172.21.15.5:58520) 2024-09-10T09:49:36.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:36.022+0000 7f7667e00640 1 -- 172.21.15.5:0/2703914072 learned_addr learned my addr 172.21.15.5:0/2703914072 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:49:36.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:36.022+0000 7f766da00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e9224fee50 0x55e9224fc790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39938/0 (socket says 172.21.15.5:39938) 2024-09-10T09:49:36.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:36.022+0000 7f766d000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e9224f3c70 0x55e9224fbd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:49:36.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:36.022+0000 7f766da00640 1 -- 172.21.15.5:0/2703914072 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e9224f4590 msgr2=0x55e9224fc250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:49:36.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:36.022+0000 7f766da00640 1 --2- 172.21.15.5:0/2703914072 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e9224f4590 0x55e9224fc250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:36.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:36.022+0000 7f766da00640 1 -- 172.21.15.5:0/2703914072 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e9224f3c70 msgr2=0x55e9224fbd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:49:36.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:36.022+0000 7f766da00640 1 --2- 172.21.15.5:0/2703914072 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e9224f3c70 0x55e9224fbd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:36.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:36.022+0000 7f766da00640 1 -- 172.21.15.5:0/2703914072 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e9221cb3c0 con 0x55e9224fee50 2024-09-10T09:49:36.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:36.022+0000 7f766d000640 1 --2- 172.21.15.5:0/2703914072 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e9224f3c70 0x55e9224fbd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:49:36.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:36.023+0000 7f766da00640 1 --2- 172.21.15.5:0/2703914072 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e9224fee50 0x55e9224fc790 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f76600cd0f0 tx=0x7f7660004f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f12de9a8b5da8dbb server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:49:36.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:36.023+0000 7f764ea00640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-10T09:49:36.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:36.023+0000 7f764ea00640 1 -- 172.21.15.5:0/2703914072 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7660005510 con 0x55e9224fee50 2024-09-10T09:49:36.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:36.023+0000 7f764ea00640 1 -- 172.21.15.5:0/2703914072 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f766009b2a0 con 0x55e9224fee50 2024-09-10T09:49:36.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:36.023+0000 7f767249c740 1 -- 172.21.15.5:0/2703914072 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f765c031870 con 0x55e9224fee50 2024-09-10T09:49:36.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:36.023+0000 7f764ea00640 1 -- 172.21.15.5:0/2703914072 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f766009be50 con 0x55e9224fee50 2024-09-10T09:49:36.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:36.023+0000 7f767249c740 10 client.17631 fetch_fsmap learned FSMap version 36 2024-09-10T09:49:36.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:36.023+0000 7f767249c740 1 -- 172.21.15.5:0/2703914072 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55e92250dac0 con 0x55e9224fee50 2024-09-10T09:49:36.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:36.023+0000 7f764ea00640 1 -- 172.21.15.5:0/2703914072 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f765c031870 con 0x55e9224fee50 2024-09-10T09:49:36.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:36.024+0000 7f764ea00640 1 -- 172.21.15.5:0/2703914072 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 36) ==== 8370+0+0 (secure 0 0 0) 0x7f766009ae30 con 0x55e9224fee50 2024-09-10T09:49:36.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:36.024+0000 7f767249c740 10 client.17631 fetch_fsmap finished waiting for FSMap version 36 2024-09-10T09:49:36.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:36.024+0000 7f767249c740 10 client.17631 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-10T09:49:36.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:36.024+0000 7f767249c740 20 client.17631 populate_metadata read hostname 'smithi005' 2024-09-10T09:49:36.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:36.024+0000 7f767249c740 1 --2- 172.21.15.5:0/2703914072 >> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] conn(0x55e92250ef80 0x55e9225113e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:49:36.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:36.024+0000 7f767249c740 4 client.17631 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:49:36.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:36.024+0000 7f767249c740 1 -- 172.21.15.5:0/2703914072 --> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e922467950 con 0x55e92250ef80 2024-09-10T09:49:36.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:36.024+0000 7f766d000640 1 --2- 172.21.15.5:0/2703914072 >> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] conn(0x55e92250ef80 0x55e9225113e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:49:36.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:36.025+0000 7f766d000640 1 --2- 172.21.15.5:0/2703914072 >> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] conn(0x55e92250ef80 0x55e9225113e0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9a2e6b2b5244a6c5 server_cookie=d7c6dc4b2239be26 in_seq=0 out_seq=0 2024-09-10T09:49:36.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:36.025+0000 7f764ea00640 10 client.17631 ms_handle_connect on v2:172.21.15.159:6836/620627639 2024-09-10T09:49:36.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:36.026+0000 7f764ea00640 1 -- 172.21.15.5:0/2703914072 <== mds.2 v2:172.21.15.159:6836/620627639 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e922467950 con 0x55e92250ef80 2024-09-10T09:49:36.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:36.026+0000 7f764ea00640 10 client.17631 handle_command_reply: tid=0 2024-09-10T09:49:36.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:36.026+0000 7f767249c740 1 client.17631 shutdown 2024-09-10T09:49:36.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:36.026+0000 7f767249c740 1 -- 172.21.15.5:0/2703914072 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e9224fee50 msgr2=0x55e9224fc790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:49:36.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:36.026+0000 7f767249c740 1 --2- 172.21.15.5:0/2703914072 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e9224fee50 0x55e9224fc790 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f76600cd0f0 tx=0x7f7660004f20 comp rx=0 tx=0).stop 2024-09-10T09:49:36.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:36.026+0000 7f767249c740 1 -- 172.21.15.5:0/2703914072 shutdown_connections 2024-09-10T09:49:36.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:36.026+0000 7f767249c740 1 --2- 172.21.15.5:0/2703914072 >> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] conn(0x55e92250ef80 0x55e9225113e0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:36.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:36.027+0000 7f767249c740 1 --2- 172.21.15.5:0/2703914072 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e9224fee50 0x55e9224fc790 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:36.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:36.027+0000 7f767249c740 1 --2- 172.21.15.5:0/2703914072 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e9224f4590 0x55e9224fc250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:36.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:36.027+0000 7f767249c740 1 --2- 172.21.15.5:0/2703914072 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e9224f3c70 0x55e9224fbd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:36.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:36.027+0000 7f767249c740 1 -- 172.21.15.5:0/2703914072 >> 172.21.15.5:0/2703914072 conn(0x55e9224f0200 msgr2=0x55e9224ef380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:49:36.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:36.027+0000 7f764ea00640 0 client.17631 ms_handle_reset on v2:172.21.15.159:6836/620627639 2024-09-10T09:49:36.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:36.027+0000 7f767249c740 1 -- 172.21.15.5:0/2703914072 shutdown_connections 2024-09-10T09:49:36.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:36.027+0000 7f767249c740 1 -- 172.21.15.5:0/2703914072 wait complete. 2024-09-10T09:49:36.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:36.027+0000 7f767249c740 20 client.17631 trim_cache size 0 max 16384 2024-09-10T09:49:36.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:36.042+0000 7f767249c740 1 Processor -- start 2024-09-10T09:49:36.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:36.042+0000 7f767249c740 1 -- start start 2024-09-10T09:49:36.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:36.042+0000 7f767249c740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e9224f3c70 0x55e922516cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:49:36.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:36.042+0000 7f767249c740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e9224f4590 0x55e9225171f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:49:36.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:36.042+0000 7f767249c740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e9224fee50 0x55e9224f8f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:49:36.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:36.042+0000 7f767249c740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f76681b0ff0 con 0x55e9224f4590 2024-09-10T09:49:36.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:36.042+0000 7f767249c740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f76681b2b30 con 0x55e9224f3c70 2024-09-10T09:49:36.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:36.042+0000 7f767249c740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f76681b16c0 con 0x55e9224fee50 2024-09-10T09:49:36.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:36.042+0000 7f7667e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e9224f4590 0x55e9225171f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:49:36.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:36.042+0000 7f7667e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e9224f4590 0x55e9225171f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39946/0 (socket says 172.21.15.5:39946) 2024-09-10T09:49:36.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:36.042+0000 7f7667e00640 1 -- 172.21.15.5:0/2635315580 learned_addr learned my addr 172.21.15.5:0/2635315580 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:49:36.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:36.042+0000 7f766d000640 1 --2- 172.21.15.5:0/2635315580 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e9224f3c70 0x55e922516cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:49:36.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:36.042+0000 7f766da00640 1 --2- 172.21.15.5:0/2635315580 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e9224fee50 0x55e9224f8f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:49:36.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:36.043+0000 7f766d000640 1 -- 172.21.15.5:0/2635315580 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e9224fee50 msgr2=0x55e9224f8f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:49:36.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:36.043+0000 7f766d000640 1 --2- 172.21.15.5:0/2635315580 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e9224fee50 0x55e9224f8f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:36.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:36.043+0000 7f766d000640 1 -- 172.21.15.5:0/2635315580 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e9224f4590 msgr2=0x55e9225171f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:49:36.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:36.043+0000 7f766d000640 1 --2- 172.21.15.5:0/2635315580 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e9224f4590 0x55e9225171f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:36.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:36.043+0000 7f766d000640 1 -- 172.21.15.5:0/2635315580 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e9224f43c0 con 0x55e9224f3c70 2024-09-10T09:49:36.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:36.043+0000 7f766d000640 1 --2- 172.21.15.5:0/2635315580 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e9224f3c70 0x55e922516cb0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f765c006010 tx=0x7f765c047040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4662bdf0dfc11a0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:49:36.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:36.043+0000 7f7659600640 1 -- 172.21.15.5:0/2635315580 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f765c040040 con 0x55e9224f3c70 2024-09-10T09:49:36.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:36.043+0000 7f7659600640 1 -- 172.21.15.5:0/2635315580 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f765c031870 con 0x55e9224f3c70 2024-09-10T09:49:36.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:36.043+0000 7f7659600640 1 -- 172.21.15.5:0/2635315580 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f765c035040 con 0x55e9224f3c70 2024-09-10T09:49:36.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:36.044+0000 7f767249c740 1 -- 172.21.15.5:0/2635315580 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e9224f3c70 msgr2=0x55e922516cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:49:36.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:36.044+0000 7f767249c740 1 --2- 172.21.15.5:0/2635315580 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e9224f3c70 0x55e922516cb0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f765c006010 tx=0x7f765c047040 comp rx=0 tx=0).stop 2024-09-10T09:49:36.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:36.044+0000 7f767249c740 1 -- 172.21.15.5:0/2635315580 shutdown_connections 2024-09-10T09:49:36.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:36.044+0000 7f767249c740 1 --2- 172.21.15.5:0/2635315580 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e9224fee50 0x55e9224f8f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:36.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:36.044+0000 7f767249c740 1 --2- 172.21.15.5:0/2635315580 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e9224f4590 0x55e9225171f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:36.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:36.044+0000 7f767249c740 1 --2- 172.21.15.5:0/2635315580 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e9224f3c70 0x55e922516cb0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:36.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:36.044+0000 7f767249c740 1 -- 172.21.15.5:0/2635315580 >> 172.21.15.5:0/2635315580 conn(0x55e9224f0200 msgr2=0x55e9224fdd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:49:36.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:36.044+0000 7f767249c740 1 -- 172.21.15.5:0/2635315580 shutdown_connections 2024-09-10T09:49:36.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:36.045+0000 7f767249c740 1 -- 172.21.15.5:0/2635315580 wait complete. 2024-09-10T09:49:36.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:36.045+0000 7f767249c740 1 -- start start 2024-09-10T09:49:36.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:36.045+0000 7f767249c740 1 Processor -- start 2024-09-10T09:49:36.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:36.046+0000 7f767249c740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e9224f3c70 0x55e9225075a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:49:36.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:36.046+0000 7f767249c740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e9224f4590 0x55e922502920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:49:36.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:36.046+0000 7f767249c740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e9224fee50 0x55e922502e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:49:36.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:36.046+0000 7f767249c740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f76681b2460 con 0x55e9224f3c70 2024-09-10T09:49:36.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:36.046+0000 7f767249c740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f76681b1d90 con 0x55e9224f4590 2024-09-10T09:49:36.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:36.046+0000 7f767249c740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f76681aefd0 con 0x55e9224fee50 2024-09-10T09:49:36.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:36.046+0000 7f767249c740 10 client.0 _ll_register_callbacks cb 0x7f766009bc20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:49:36.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:36.046+0000 7f7667e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e9224f4590 0x55e922502920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:49:36.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:36.046+0000 7f7667e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e9224f4590 0x55e922502920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:54818/0 (socket says 172.21.15.5:54818) 2024-09-10T09:49:36.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:36.046+0000 7f7667e00640 1 -- 172.21.15.5:0/1764814319 learned_addr learned my addr 172.21.15.5:0/1764814319 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:49:36.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:36.046+0000 7f766d000640 1 --2- 172.21.15.5:0/1764814319 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e9224f3c70 0x55e9225075a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:49:36.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:36.046+0000 7f766da00640 1 --2- 172.21.15.5:0/1764814319 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e9224fee50 0x55e922502e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:49:36.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:36.047+0000 7f7667e00640 1 -- 172.21.15.5:0/1764814319 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e9224fee50 msgr2=0x55e922502e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:49:36.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:36.047+0000 7f7667e00640 1 --2- 172.21.15.5:0/1764814319 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e9224fee50 0x55e922502e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:36.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:36.047+0000 7f7667e00640 1 -- 172.21.15.5:0/1764814319 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e9224f3c70 msgr2=0x55e9225075a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:49:36.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:36.047+0000 7f7667e00640 1 --2- 172.21.15.5:0/1764814319 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e9224f3c70 0x55e9225075a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:36.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:36.047+0000 7f7667e00640 1 -- 172.21.15.5:0/1764814319 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e922503640 con 0x55e9224f4590 2024-09-10T09:49:36.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:36.047+0000 7f7667e00640 1 --2- 172.21.15.5:0/1764814319 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e9224f4590 0x55e922502920 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f7654009730 tx=0x7f76540046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=85da8a1f05dc1ece server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:49:36.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:36.047+0000 7f764ea00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-09-10T09:49:36.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:36.047+0000 7f764ea00640 1 -- 172.21.15.5:0/1764814319 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7654032060 con 0x55e9224f4590 2024-09-10T09:49:36.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:36.047+0000 7f767249c740 1 -- 172.21.15.5:0/1764814319 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f765c031870 con 0x55e9224f4590 2024-09-10T09:49:36.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:36.047+0000 7f764ea00640 1 -- 172.21.15.5:0/1764814319 <== mon.2 v2:172.21.15.102:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f7654022d20 con 0x55e9224f4590 2024-09-10T09:49:36.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:36.048+0000 7f764ea00640 1 -- 172.21.15.5:0/1764814319 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f7654039050 con 0x55e9224f4590 2024-09-10T09:49:36.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:36.048+0000 7f767249c740 10 client.27650 fetch_fsmap learned FSMap version 36 2024-09-10T09:49:36.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:36.048+0000 7f767249c740 1 -- 172.21.15.5:0/1764814319 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55e922579d40 con 0x55e9224f4590 2024-09-10T09:49:36.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:36.048+0000 7f764ea00640 1 -- 172.21.15.5:0/1764814319 <== mon.2 v2:172.21.15.102:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f765c031870 con 0x55e9224f4590 2024-09-10T09:49:36.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:36.048+0000 7f764ea00640 1 -- 172.21.15.5:0/1764814319 <== mon.2 v2:172.21.15.102:3300/0 5 ==== fsmap(e 36) ==== 8370+0+0 (secure 0 0 0) 0x7f7654020400 con 0x55e9224f4590 2024-09-10T09:49:36.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:36.048+0000 7f767249c740 10 client.27650 fetch_fsmap finished waiting for FSMap version 36 2024-09-10T09:49:36.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:36.048+0000 7f767249c740 10 client.27650 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-10T09:49:36.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:36.048+0000 7f767249c740 20 client.27650 populate_metadata read hostname 'smithi005' 2024-09-10T09:49:36.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:36.048+0000 7f767249c740 1 --2- 172.21.15.5:0/1764814319 >> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] conn(0x55e92257a4a0 0x55e92257c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:49:36.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:36.048+0000 7f767249c740 4 client.27650 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T09:49:36.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:36.048+0000 7f767249c740 1 -- 172.21.15.5:0/1764814319 --> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e922461e60 con 0x55e92257a4a0 2024-09-10T09:49:36.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:36.049+0000 7f766d000640 1 --2- 172.21.15.5:0/1764814319 >> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] conn(0x55e92257a4a0 0x55e92257c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:49:36.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:36.049+0000 7f766d000640 1 --2- 172.21.15.5:0/1764814319 >> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] conn(0x55e92257a4a0 0x55e92257c8c0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2b622f82c8b23305 server_cookie=bad760e0b8d7f0e2 in_seq=0 out_seq=0 2024-09-10T09:49:36.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:36.049+0000 7f764ea00640 10 client.27650 ms_handle_connect on v2:172.21.15.159:6836/620627639 2024-09-10T09:49:36.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:36.050+0000 7f764ea00640 1 -- 172.21.15.5:0/1764814319 <== mds.2 v2:172.21.15.159:6836/620627639 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e922461e60 con 0x55e92257a4a0 2024-09-10T09:49:36.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:36.050+0000 7f764ea00640 10 client.27650 handle_command_reply: tid=0 2024-09-10T09:49:36.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:36.050+0000 7f767249c740 1 client.27650 shutdown 2024-09-10T09:49:36.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:36.050+0000 7f767249c740 1 -- 172.21.15.5:0/1764814319 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e9224f4590 msgr2=0x55e922502920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:49:36.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:36.050+0000 7f767249c740 1 --2- 172.21.15.5:0/1764814319 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e9224f4590 0x55e922502920 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f7654009730 tx=0x7f76540046c0 comp rx=0 tx=0).stop 2024-09-10T09:49:36.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:36.050+0000 7f767249c740 1 -- 172.21.15.5:0/1764814319 shutdown_connections 2024-09-10T09:49:36.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:36.050+0000 7f767249c740 1 --2- 172.21.15.5:0/1764814319 >> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] conn(0x55e92257a4a0 0x55e92257c8c0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:36.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:36.050+0000 7f767249c740 1 --2- 172.21.15.5:0/1764814319 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e9224fee50 0x55e922502e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:36.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:36.050+0000 7f767249c740 1 --2- 172.21.15.5:0/1764814319 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e9224f4590 0x55e922502920 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:36.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:36.050+0000 7f767249c740 1 --2- 172.21.15.5:0/1764814319 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e9224f3c70 0x55e9225075a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:36.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:36.050+0000 7f764ea00640 0 client.27650 ms_handle_reset on v2:172.21.15.159:6836/620627639 2024-09-10T09:49:36.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:36.050+0000 7f767249c740 1 -- 172.21.15.5:0/1764814319 >> 172.21.15.5:0/1764814319 conn(0x55e9224f0200 msgr2=0x55e9224f56c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:49:36.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:36.050+0000 7f767249c740 1 -- 172.21.15.5:0/1764814319 shutdown_connections 2024-09-10T09:49:36.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:36.050+0000 7f767249c740 1 -- 172.21.15.5:0/1764814319 wait complete. 2024-09-10T09:49:36.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:36.050+0000 7f767249c740 20 client.27650 trim_cache size 0 max 16384 2024-09-10T09:49:36.056 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-10T09:49:36.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:36.051+0000 7f766fa00640 1 -- 172.21.15.5:0/408129624 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f76500a2c00 msgr2=0x7f76500a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:49:36.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:36.051+0000 7f766fa00640 1 --2- 172.21.15.5:0/408129624 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f76500a2c00 0x7f76500a50c0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x55e9224f1580 tx=0x7f76540096c0 comp rx=0 tx=0).stop 2024-09-10T09:49:36.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:36.051+0000 7f766fa00640 1 -- 172.21.15.5:0/408129624 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7668109120 msgr2=0x7f766819eec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:49:36.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:36.051+0000 7f766fa00640 1 --2- 172.21.15.5:0/408129624 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7668109120 0x7f766819eec0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f766000eb10 tx=0x7f766000c2d0 comp rx=0 tx=0).stop 2024-09-10T09:49:36.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:36.051+0000 7f766fa00640 1 -- 172.21.15.5:0/408129624 shutdown_connections 2024-09-10T09:49:36.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:36.051+0000 7f766fa00640 1 --2- 172.21.15.5:0/408129624 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f76500a2c00 0x7f76500a50c0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:36.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:36.051+0000 7f766fa00640 1 --2- 172.21.15.5:0/408129624 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7668109120 0x7f766819eec0 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:36.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:36.051+0000 7f766fa00640 1 --2- 172.21.15.5:0/408129624 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f76681049a0 0x7f766819ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:36.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:36.051+0000 7f766fa00640 1 --2- 172.21.15.5:0/408129624 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7668103ff0 0x7f766819a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:36.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:36.051+0000 7f766fa00640 1 -- 172.21.15.5:0/408129624 >> 172.21.15.5:0/408129624 conn(0x7f76680ffa90 msgr2=0x7f7668100e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:49:36.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:36.052+0000 7f766fa00640 1 -- 172.21.15.5:0/408129624 shutdown_connections 2024-09-10T09:49:36.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:36.052+0000 7f766fa00640 1 -- 172.21.15.5:0/408129624 wait complete. 2024-09-10T09:49:36.125 INFO:tasks.workunit.client.0.smithi005.stdout: 1 687246 6.14 MB/sec execute 437 sec latency 19487.984 ms 2024-09-10T09:49:36.913 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-10T09:49:37.125 INFO:tasks.workunit.client.0.smithi005.stdout: 1 687246 6.13 MB/sec execute 438 sec latency 20488.047 ms 2024-09-10T09:49:37.480 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:49:37.555 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:49:37 smithi005 ceph-mon[30350]: pgmap v731: 129 pgs: 129 active+clean; 258 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 568 KiB/s wr, 31 op/s 2024-09-10T09:49:37.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:49:37 smithi159 ceph-mon[35513]: pgmap v731: 129 pgs: 129 active+clean; 258 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 568 KiB/s wr, 31 op/s 2024-09-10T09:49:37.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:49:37 smithi102 ceph-mon[35442]: pgmap v731: 129 pgs: 129 active+clean; 258 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 568 KiB/s wr, 31 op/s 2024-09-10T09:49:38.125 INFO:tasks.workunit.client.0.smithi005.stdout: 1 687246 6.11 MB/sec execute 439 sec latency 21488.103 ms 2024-09-10T09:49:38.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:38.326+0000 7f120a400640 1 -- 172.21.15.5:0/3437004296 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1204112870 msgr2=0x7f1204114c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:49:38.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:38.326+0000 7f120a400640 1 --2- 172.21.15.5:0/3437004296 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1204112870 0x7f1204114c60 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f11fc009ca0 tx=0x7f11fc02a8c0 comp rx=0 tx=0).stop 2024-09-10T09:49:38.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:38.326+0000 7f120a400640 1 -- 172.21.15.5:0/3437004296 shutdown_connections 2024-09-10T09:49:38.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:38.326+0000 7f120a400640 1 --2- 172.21.15.5:0/3437004296 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1204112870 0x7f1204114c60 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:38.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:38.326+0000 7f120a400640 1 --2- 172.21.15.5:0/3437004296 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1204077040 0x7f1204112330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:38.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:38.326+0000 7f120a400640 1 --2- 172.21.15.5:0/3437004296 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1204076720 0x7f1204076b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:38.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:38.326+0000 7f120a400640 1 -- 172.21.15.5:0/3437004296 >> 172.21.15.5:0/3437004296 conn(0x7f12040ff8d0 msgr2=0x7f1204101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:49:38.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:38.327+0000 7f120a400640 1 -- 172.21.15.5:0/3437004296 shutdown_connections 2024-09-10T09:49:38.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:38.327+0000 7f120a400640 1 -- 172.21.15.5:0/3437004296 wait complete. 2024-09-10T09:49:38.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:38.328+0000 7f120a400640 1 Processor -- start 2024-09-10T09:49:38.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:38.328+0000 7f120a400640 1 -- start start 2024-09-10T09:49:38.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:38.328+0000 7f120a400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1204076720 0x7f12041a4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:49:38.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:38.328+0000 7f120a400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1204077040 0x7f12041a4b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:49:38.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:38.328+0000 7f120a400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1204112870 0x7f120419e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:49:38.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:38.328+0000 7f120a400640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1204118420 con 0x7f1204076720 2024-09-10T09:49:38.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:38.328+0000 7f120a400640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f12041182a0 con 0x7f1204077040 2024-09-10T09:49:38.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:38.328+0000 7f120a400640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f12041185a0 con 0x7f1204112870 2024-09-10T09:49:38.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:38.328+0000 7f1209000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1204076720 0x7f12041a4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:49:38.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:38.328+0000 7f1209000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1204076720 0x7f12041a4600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39986/0 (socket says 172.21.15.5:39986) 2024-09-10T09:49:38.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:38.328+0000 7f1209000640 1 -- 172.21.15.5:0/328854497 learned_addr learned my addr 172.21.15.5:0/328854497 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:49:38.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:38.329+0000 7f1203e00640 1 --2- 172.21.15.5:0/328854497 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1204077040 0x7f12041a4b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:49:38.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:38.329+0000 7f1209a00640 1 --2- 172.21.15.5:0/328854497 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1204112870 0x7f120419e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:49:38.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:38.329+0000 7f1209000640 1 -- 172.21.15.5:0/328854497 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1204112870 msgr2=0x7f120419e6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:49:38.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:38.329+0000 7f1209000640 1 --2- 172.21.15.5:0/328854497 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1204112870 0x7f120419e6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:38.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:38.329+0000 7f1209000640 1 -- 172.21.15.5:0/328854497 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1204077040 msgr2=0x7f12041a4b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:49:38.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:38.329+0000 7f1209000640 1 --2- 172.21.15.5:0/328854497 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1204077040 0x7f12041a4b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:38.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:38.329+0000 7f1209000640 1 -- 172.21.15.5:0/328854497 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f120419ef40 con 0x7f1204076720 2024-09-10T09:49:38.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:38.329+0000 7f1209a00640 1 --2- 172.21.15.5:0/328854497 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1204112870 0x7f120419e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:49:38.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:38.329+0000 7f1209000640 1 --2- 172.21.15.5:0/328854497 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1204076720 0x7f12041a4600 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f11f800e9f0 tx=0x7f11f800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:49:38.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:38.330+0000 7f1201600640 1 -- 172.21.15.5:0/328854497 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11f800cd80 con 0x7f1204076720 2024-09-10T09:49:38.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:38.330+0000 7f1201600640 1 -- 172.21.15.5:0/328854497 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f11f8010910 con 0x7f1204076720 2024-09-10T09:49:38.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:38.330+0000 7f1201600640 1 -- 172.21.15.5:0/328854497 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11f8018990 con 0x7f1204076720 2024-09-10T09:49:38.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:38.330+0000 7f120a400640 1 -- 172.21.15.5:0/328854497 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f120419f230 con 0x7f1204076720 2024-09-10T09:49:38.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:38.330+0000 7f120a400640 1 -- 172.21.15.5:0/328854497 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12041ab580 con 0x7f1204076720 2024-09-10T09:49:38.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:38.333+0000 7f120e65f740 1 Processor -- start 2024-09-10T09:49:38.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:38.333+0000 7f120e65f740 1 -- start start 2024-09-10T09:49:38.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:38.333+0000 7f120e65f740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56380fd17e40 0x56380fd18220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:49:38.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:38.333+0000 7f120e65f740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56380fd18760 0x56380fd22ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:49:38.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:38.334+0000 7f120e65f740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56380fd23020 0x56380fd25410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:49:38.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:38.334+0000 7f120e65f740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x56380fc7e3e0 con 0x56380fd18760 2024-09-10T09:49:38.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:38.334+0000 7f120e65f740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x56380fc25d80 con 0x56380fd17e40 2024-09-10T09:49:38.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:38.334+0000 7f120e65f740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x56380fc7a3a0 con 0x56380fd23020 2024-09-10T09:49:38.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:38.334+0000 7f1209a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56380fd17e40 0x56380fd18220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:49:38.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:38.334+0000 7f1209a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56380fd17e40 0x56380fd18220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:54850/0 (socket says 172.21.15.5:54850) 2024-09-10T09:49:38.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:38.334+0000 7f1209a00640 1 -- 172.21.15.5:0/4130172942 learned_addr learned my addr 172.21.15.5:0/4130172942 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:49:38.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:38.334+0000 7f1201600640 1 -- 172.21.15.5:0/328854497 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f11f8010ac0 con 0x7f1204076720 2024-09-10T09:49:38.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:38.334+0000 7f1203e00640 1 --2- 172.21.15.5:0/4130172942 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56380fd23020 0x56380fd25410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:49:38.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:38.334+0000 7f1209a00640 1 -- 172.21.15.5:0/4130172942 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56380fd23020 msgr2=0x56380fd25410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:49:38.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:38.334+0000 7f1209a00640 1 --2- 172.21.15.5:0/4130172942 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56380fd23020 0x56380fd25410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:38.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:38.334+0000 7f1209a00640 1 -- 172.21.15.5:0/4130172942 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56380fd18760 msgr2=0x56380fd22ae0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T09:49:38.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:38.334+0000 7f1209a00640 1 --2- 172.21.15.5:0/4130172942 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56380fd18760 0x56380fd22ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:38.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:38.334+0000 7f1209a00640 1 -- 172.21.15.5:0/4130172942 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56380fc44ae0 con 0x56380fd17e40 2024-09-10T09:49:38.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:38.335+0000 7f1209a00640 1 --2- 172.21.15.5:0/4130172942 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56380fd17e40 0x56380fd18220 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f120407aa10 tx=0x7f11fc009800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6392fcb47fcf180f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:49:38.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:38.335+0000 7f11f6000640 1 -- 172.21.15.5:0/4130172942 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11fc031db0 con 0x56380fd17e40 2024-09-10T09:49:38.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:38.335+0000 7f11f6000640 1 -- 172.21.15.5:0/4130172942 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f11fc036040 con 0x56380fd17e40 2024-09-10T09:49:38.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:38.335+0000 7f11f6000640 1 -- 172.21.15.5:0/4130172942 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11fc004330 con 0x56380fd17e40 2024-09-10T09:49:38.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:38.336+0000 7f1201600640 1 --2- 172.21.15.5:0/328854497 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f11ec0a2c00 0x7f11ec0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:49:38.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:38.336+0000 7f1201600640 1 -- 172.21.15.5:0/328854497 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f11f80cf7a0 con 0x7f1204076720 2024-09-10T09:49:38.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:38.336+0000 7f1209000640 1 --2- 172.21.15.5:0/328854497 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f11ec0a2c00 0x7f11ec0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:49:38.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:38.336+0000 7f120e65f740 1 -- 172.21.15.5:0/4130172942 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56380fd17e40 msgr2=0x56380fd18220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:49:38.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:38.336+0000 7f120e65f740 1 --2- 172.21.15.5:0/4130172942 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56380fd17e40 0x56380fd18220 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f120407aa10 tx=0x7f11fc009800 comp rx=0 tx=0).stop 2024-09-10T09:49:38.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:38.336+0000 7f120e65f740 1 -- 172.21.15.5:0/4130172942 shutdown_connections 2024-09-10T09:49:38.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:38.336+0000 7f120e65f740 1 --2- 172.21.15.5:0/4130172942 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56380fd23020 0x56380fd25410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:38.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:38.336+0000 7f120e65f740 1 --2- 172.21.15.5:0/4130172942 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56380fd18760 0x56380fd22ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:38.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:38.336+0000 7f120e65f740 1 --2- 172.21.15.5:0/4130172942 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56380fd17e40 0x56380fd18220 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:38.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:38.336+0000 7f120e65f740 1 -- 172.21.15.5:0/4130172942 >> 172.21.15.5:0/4130172942 conn(0x56380fd143d0 msgr2=0x56380fd147b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:49:38.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:38.336+0000 7f120e65f740 1 -- 172.21.15.5:0/4130172942 shutdown_connections 2024-09-10T09:49:38.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:38.337+0000 7f120e65f740 1 -- 172.21.15.5:0/4130172942 wait complete. 2024-09-10T09:49:38.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:38.337+0000 7f1209000640 1 --2- 172.21.15.5:0/328854497 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f11ec0a2c00 0x7f11ec0a50c0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f11f800cfd0 tx=0x7f11f800cf20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:49:38.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:38.337+0000 7f120e65f740 1 -- start start 2024-09-10T09:49:38.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:38.337+0000 7f120e65f740 1 Processor -- start 2024-09-10T09:49:38.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:38.338+0000 7f120e65f740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56380fd17e40 0x56380fd1fef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:49:38.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:38.338+0000 7f120e65f740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56380fd18760 0x56380fd20430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:49:38.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:38.338+0000 7f120e65f740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56380fd23020 0x56380fd20970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:49:38.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:38.338+0000 7f120e65f740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x56380fd25bf0 con 0x56380fd17e40 2024-09-10T09:49:38.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:38.338+0000 7f120e65f740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x56380fc390f0 con 0x56380fd18760 2024-09-10T09:49:38.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:38.338+0000 7f120e65f740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x56380fd25d70 con 0x56380fd23020 2024-09-10T09:49:38.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:38.338+0000 7f120e65f740 10 client.0 _ll_register_callbacks cb 0x56380fc20bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:49:38.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:38.338+0000 7f1209a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56380fd17e40 0x56380fd1fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:49:38.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:38.338+0000 7f1203e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56380fd18760 0x56380fd20430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:49:38.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:38.338+0000 7f1203e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56380fd18760 0x56380fd20430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:54852/0 (socket says 172.21.15.5:54852) 2024-09-10T09:49:38.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:38.338+0000 7f1203e00640 1 -- 172.21.15.5:0/2731343721 learned_addr learned my addr 172.21.15.5:0/2731343721 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:49:38.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:38.338+0000 7f1209a00640 1 --2- 172.21.15.5:0/2731343721 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56380fd23020 0x56380fd20970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:49:38.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:38.338+0000 7f1203e00640 1 -- 172.21.15.5:0/2731343721 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56380fd23020 msgr2=0x56380fd20970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:49:38.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:38.338+0000 7f1203e00640 1 --2- 172.21.15.5:0/2731343721 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56380fd23020 0x56380fd20970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:38.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:38.338+0000 7f1203e00640 1 -- 172.21.15.5:0/2731343721 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56380fd17e40 msgr2=0x56380fd1fef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:49:38.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:38.338+0000 7f1203e00640 1 --2- 172.21.15.5:0/2731343721 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56380fd17e40 0x56380fd1fef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:38.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:38.338+0000 7f1203e00640 1 -- 172.21.15.5:0/2731343721 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56380f9ef3c0 con 0x56380fd18760 2024-09-10T09:49:38.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:38.339+0000 7f1209a00640 1 --2- 172.21.15.5:0/2731343721 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56380fd23020 0x56380fd20970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T09:49:38.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:38.339+0000 7f1203e00640 1 --2- 172.21.15.5:0/2731343721 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56380fd18760 0x56380fd20430 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f11f0012700 tx=0x7f11f0012bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cc895ee3eb477c33 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:49:38.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:38.339+0000 7f11eaa00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-09-10T09:49:38.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:38.339+0000 7f11eaa00640 1 -- 172.21.15.5:0/2731343721 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11f0004050 con 0x56380fd18760 2024-09-10T09:49:38.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:38.339+0000 7f11eaa00640 1 -- 172.21.15.5:0/2731343721 <== mon.2 v2:172.21.15.102:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f11f00041f0 con 0x56380fd18760 2024-09-10T09:49:38.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:38.339+0000 7f120e65f740 1 -- 172.21.15.5:0/2731343721 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f11fc036040 con 0x56380fd18760 2024-09-10T09:49:38.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:38.339+0000 7f11eaa00640 1 -- 172.21.15.5:0/2731343721 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f11f00043f0 con 0x56380fd18760 2024-09-10T09:49:38.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:38.339+0000 7f120e65f740 10 client.27662 fetch_fsmap learned FSMap version 36 2024-09-10T09:49:38.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:38.339+0000 7f120e65f740 1 -- 172.21.15.5:0/2731343721 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x56380fd31ca0 con 0x56380fd18760 2024-09-10T09:49:38.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:38.340+0000 7f11eaa00640 1 -- 172.21.15.5:0/2731343721 <== mon.2 v2:172.21.15.102:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f11fc036040 con 0x56380fd18760 2024-09-10T09:49:38.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:38.340+0000 7f11eaa00640 1 -- 172.21.15.5:0/2731343721 <== mon.2 v2:172.21.15.102:3300/0 5 ==== fsmap(e 36) ==== 8370+0+0 (secure 0 0 0) 0x7f11f0003bf0 con 0x56380fd18760 2024-09-10T09:49:38.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:38.340+0000 7f120e65f740 10 client.27662 fetch_fsmap finished waiting for FSMap version 36 2024-09-10T09:49:38.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:38.340+0000 7f120e65f740 10 client.27662 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-10T09:49:38.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:38.340+0000 7f120e65f740 20 client.27662 populate_metadata read hostname 'smithi005' 2024-09-10T09:49:38.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:38.340+0000 7f120e65f740 1 --2- 172.21.15.5:0/2731343721 >> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] conn(0x56380fd33160 0x56380fd35590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:49:38.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:38.340+0000 7f120e65f740 4 client.27662 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:49:38.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:38.340+0000 7f120e65f740 1 -- 172.21.15.5:0/2731343721 --> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56380fc8b950 con 0x56380fd33160 2024-09-10T09:49:38.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:38.340+0000 7f1209a00640 1 --2- 172.21.15.5:0/2731343721 >> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] conn(0x56380fd33160 0x56380fd35590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:49:38.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:38.341+0000 7f1209a00640 1 --2- 172.21.15.5:0/2731343721 >> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] conn(0x56380fd33160 0x56380fd35590 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7c20c1522466e0eb server_cookie=44378ad22418e346 in_seq=0 out_seq=0 2024-09-10T09:49:38.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:38.341+0000 7f11eaa00640 10 client.27662 ms_handle_connect on v2:172.21.15.159:6838/2073739599 2024-09-10T09:49:38.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:38.342+0000 7f11eaa00640 1 -- 172.21.15.5:0/2731343721 <== mds.1 v2:172.21.15.159:6838/2073739599 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56380fc8b950 con 0x56380fd33160 2024-09-10T09:49:38.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:38.342+0000 7f11eaa00640 10 client.27662 handle_command_reply: tid=0 2024-09-10T09:49:38.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:38.342+0000 7f120e65f740 1 client.27662 shutdown 2024-09-10T09:49:38.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:38.342+0000 7f120e65f740 1 -- 172.21.15.5:0/2731343721 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56380fd18760 msgr2=0x56380fd20430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:49:38.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:38.342+0000 7f120e65f740 1 --2- 172.21.15.5:0/2731343721 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56380fd18760 0x56380fd20430 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f11f0012700 tx=0x7f11f0012bc0 comp rx=0 tx=0).stop 2024-09-10T09:49:38.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:38.342+0000 7f120e65f740 1 -- 172.21.15.5:0/2731343721 shutdown_connections 2024-09-10T09:49:38.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:38.342+0000 7f120e65f740 1 --2- 172.21.15.5:0/2731343721 >> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] conn(0x56380fd33160 0x56380fd35590 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:38.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:38.342+0000 7f120e65f740 1 --2- 172.21.15.5:0/2731343721 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56380fd23020 0x56380fd20970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:38.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:38.342+0000 7f120e65f740 1 --2- 172.21.15.5:0/2731343721 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56380fd18760 0x56380fd20430 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:38.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:38.342+0000 7f120e65f740 1 --2- 172.21.15.5:0/2731343721 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56380fd17e40 0x56380fd1fef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:38.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:38.342+0000 7f120e65f740 1 -- 172.21.15.5:0/2731343721 >> 172.21.15.5:0/2731343721 conn(0x56380fd143d0 msgr2=0x56380fd1a100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:49:38.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:38.342+0000 7f11eaa00640 0 client.27662 ms_handle_reset on v2:172.21.15.159:6838/2073739599 2024-09-10T09:49:38.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:38.342+0000 7f120e65f740 1 -- 172.21.15.5:0/2731343721 shutdown_connections 2024-09-10T09:49:38.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:38.343+0000 7f120e65f740 1 -- 172.21.15.5:0/2731343721 wait complete. 2024-09-10T09:49:38.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:38.343+0000 7f120e65f740 20 client.27662 trim_cache size 0 max 16384 2024-09-10T09:49:38.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:38.357+0000 7f120e65f740 1 Processor -- start 2024-09-10T09:49:38.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:38.357+0000 7f120e65f740 1 -- start start 2024-09-10T09:49:38.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:38.357+0000 7f120e65f740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56380fd17e40 0x56380fd1a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:49:38.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:38.357+0000 7f120e65f740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56380fd18760 0x56380fd207c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:49:38.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:38.357+0000 7f120e65f740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56380fd23020 0x56380fd21e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:49:38.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:38.357+0000 7f120e65f740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f12041b5390 con 0x56380fd17e40 2024-09-10T09:49:38.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:38.357+0000 7f120e65f740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f12041b6ed0 con 0x56380fd23020 2024-09-10T09:49:38.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:38.357+0000 7f120e65f740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f12041b5a60 con 0x56380fd18760 2024-09-10T09:49:38.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:38.357+0000 7f1209a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56380fd17e40 0x56380fd1a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:49:38.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:38.357+0000 7f1209a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56380fd17e40 0x56380fd1a8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40014/0 (socket says 172.21.15.5:40014) 2024-09-10T09:49:38.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:38.357+0000 7f1203e00640 1 --2- 172.21.15.5:0/2666961643 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56380fd18760 0x56380fd207c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:49:38.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:38.358+0000 7f1209a00640 1 -- 172.21.15.5:0/2666961643 learned_addr learned my addr 172.21.15.5:0/2666961643 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:49:38.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:38.358+0000 7f1209a00640 1 --2- 172.21.15.5:0/2666961643 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56380fd23020 0x56380fd21e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:49:38.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:38.358+0000 7f1203e00640 1 -- 172.21.15.5:0/2666961643 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56380fd23020 msgr2=0x56380fd21e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:49:38.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:38.358+0000 7f1203e00640 1 --2- 172.21.15.5:0/2666961643 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56380fd23020 0x56380fd21e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:38.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:38.358+0000 7f1203e00640 1 -- 172.21.15.5:0/2666961643 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56380fd17e40 msgr2=0x56380fd1a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:49:38.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:38.358+0000 7f1203e00640 1 --2- 172.21.15.5:0/2666961643 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56380fd17e40 0x56380fd1a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:38.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:38.358+0000 7f1203e00640 1 -- 172.21.15.5:0/2666961643 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56380fd14200 con 0x56380fd18760 2024-09-10T09:49:38.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:38.358+0000 7f1203e00640 1 --2- 172.21.15.5:0/2666961643 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56380fd18760 0x56380fd207c0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f11f0010480 tx=0x7f11f00109b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=286390d41fe50602 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:49:38.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:38.358+0000 7f11f5600640 1 -- 172.21.15.5:0/2666961643 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11f0003f60 con 0x56380fd18760 2024-09-10T09:49:38.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:38.359+0000 7f11f5600640 1 -- 172.21.15.5:0/2666961643 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f11f0004100 con 0x56380fd18760 2024-09-10T09:49:38.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:38.359+0000 7f11f5600640 1 -- 172.21.15.5:0/2666961643 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11f000fda0 con 0x56380fd18760 2024-09-10T09:49:38.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:38.359+0000 7f120e65f740 1 -- 172.21.15.5:0/2666961643 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56380fd18760 msgr2=0x56380fd207c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:49:38.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:38.359+0000 7f120e65f740 1 --2- 172.21.15.5:0/2666961643 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56380fd18760 0x56380fd207c0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f11f0010480 tx=0x7f11f00109b0 comp rx=0 tx=0).stop 2024-09-10T09:49:38.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:38.359+0000 7f120e65f740 1 -- 172.21.15.5:0/2666961643 shutdown_connections 2024-09-10T09:49:38.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:38.359+0000 7f120e65f740 1 --2- 172.21.15.5:0/2666961643 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56380fd23020 0x56380fd21e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:38.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:38.359+0000 7f120e65f740 1 --2- 172.21.15.5:0/2666961643 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56380fd18760 0x56380fd207c0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:38.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:38.359+0000 7f120e65f740 1 --2- 172.21.15.5:0/2666961643 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56380fd17e40 0x56380fd1a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:38.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:38.359+0000 7f120e65f740 1 -- 172.21.15.5:0/2666961643 >> 172.21.15.5:0/2666961643 conn(0x56380fd143d0 msgr2=0x56380fd21750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:49:38.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:38.359+0000 7f120e65f740 1 -- 172.21.15.5:0/2666961643 shutdown_connections 2024-09-10T09:49:38.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:38.359+0000 7f120e65f740 1 -- 172.21.15.5:0/2666961643 wait complete. 2024-09-10T09:49:38.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:38.360+0000 7f120e65f740 1 -- start start 2024-09-10T09:49:38.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:38.360+0000 7f120e65f740 1 Processor -- start 2024-09-10T09:49:38.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:38.360+0000 7f120e65f740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56380fd17e40 0x56380fd2b560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:49:38.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:38.360+0000 7f120e65f740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56380fd18760 0x56380fd268e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:49:38.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:38.360+0000 7f120e65f740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56380fd23020 0x56380fd26e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:49:38.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:38.360+0000 7f120e65f740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f12041b6800 con 0x56380fd23020 2024-09-10T09:49:38.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:38.360+0000 7f120e65f740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f12041b6130 con 0x56380fd18760 2024-09-10T09:49:38.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:38.360+0000 7f120e65f740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f12041b3370 con 0x56380fd17e40 2024-09-10T09:49:38.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:38.360+0000 7f120e65f740 10 client.0 _ll_register_callbacks cb 0x7f11f0009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:49:38.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:38.361+0000 7f1203e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56380fd18760 0x56380fd268e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:49:38.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:38.361+0000 7f1203e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56380fd18760 0x56380fd268e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:54878/0 (socket says 172.21.15.5:54878) 2024-09-10T09:49:38.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:38.361+0000 7f1203e00640 1 -- 172.21.15.5:0/492099397 learned_addr learned my addr 172.21.15.5:0/492099397 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:49:38.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:38.361+0000 7f1209a00640 1 --2- 172.21.15.5:0/492099397 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56380fd23020 0x56380fd26e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:49:38.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:38.361+0000 7f1209a00640 1 --2- 172.21.15.5:0/492099397 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56380fd17e40 0x56380fd2b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:49:38.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:38.361+0000 7f1203e00640 1 -- 172.21.15.5:0/492099397 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56380fd17e40 msgr2=0x56380fd2b560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:49:38.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:38.361+0000 7f1203e00640 1 --2- 172.21.15.5:0/492099397 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56380fd17e40 0x56380fd2b560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:38.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:38.361+0000 7f1203e00640 1 -- 172.21.15.5:0/492099397 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56380fd23020 msgr2=0x56380fd26e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:49:38.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:38.361+0000 7f1203e00640 1 --2- 172.21.15.5:0/492099397 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56380fd23020 0x56380fd26e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:38.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:38.361+0000 7f1203e00640 1 -- 172.21.15.5:0/492099397 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56380fd15480 con 0x56380fd18760 2024-09-10T09:49:38.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:38.361+0000 7f1209a00640 1 --2- 172.21.15.5:0/492099397 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56380fd23020 0x56380fd26e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:49:38.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:38.362+0000 7f1203e00640 1 --2- 172.21.15.5:0/492099397 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56380fd18760 0x56380fd268e0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f11f002b040 tx=0x7f11f000fef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8ad55ae94f33dc4d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:49:38.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:38.362+0000 7f11eaa00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-09-10T09:49:38.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:38.362+0000 7f11eaa00640 1 -- 172.21.15.5:0/492099397 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11f0036020 con 0x56380fd18760 2024-09-10T09:49:38.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:38.362+0000 7f11eaa00640 1 -- 172.21.15.5:0/492099397 <== mon.2 v2:172.21.15.102:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f11f0010c30 con 0x56380fd18760 2024-09-10T09:49:38.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:38.362+0000 7f120e65f740 1 -- 172.21.15.5:0/492099397 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f11f0004100 con 0x56380fd18760 2024-09-10T09:49:38.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:38.362+0000 7f11eaa00640 1 -- 172.21.15.5:0/492099397 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f11f0003f60 con 0x56380fd18760 2024-09-10T09:49:38.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:38.362+0000 7f120e65f740 10 client.27668 fetch_fsmap learned FSMap version 36 2024-09-10T09:49:38.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:38.362+0000 7f120e65f740 1 -- 172.21.15.5:0/492099397 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x56380fd9dab0 con 0x56380fd18760 2024-09-10T09:49:38.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:38.363+0000 7f11eaa00640 1 -- 172.21.15.5:0/492099397 <== mon.2 v2:172.21.15.102:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f11f0004100 con 0x56380fd18760 2024-09-10T09:49:38.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:38.363+0000 7f11eaa00640 1 -- 172.21.15.5:0/492099397 <== mon.2 v2:172.21.15.102:3300/0 5 ==== fsmap(e 36) ==== 8370+0+0 (secure 0 0 0) 0x7f11f0003770 con 0x56380fd18760 2024-09-10T09:49:38.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:38.363+0000 7f120e65f740 10 client.27668 fetch_fsmap finished waiting for FSMap version 36 2024-09-10T09:49:38.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:38.363+0000 7f120e65f740 10 client.27668 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-10T09:49:38.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:38.363+0000 7f120e65f740 20 client.27668 populate_metadata read hostname 'smithi005' 2024-09-10T09:49:38.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:38.363+0000 7f120e65f740 1 --2- 172.21.15.5:0/492099397 >> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] conn(0x56380fd9e1b0 0x56380fda05a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:49:38.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:38.363+0000 7f120e65f740 4 client.27668 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T09:49:38.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:38.363+0000 7f120e65f740 1 -- 172.21.15.5:0/492099397 --> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56380fc85e60 con 0x56380fd9e1b0 2024-09-10T09:49:38.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:38.364+0000 7f1209a00640 1 --2- 172.21.15.5:0/492099397 >> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] conn(0x56380fd9e1b0 0x56380fda05a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:49:38.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:38.364+0000 7f1209a00640 1 --2- 172.21.15.5:0/492099397 >> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] conn(0x56380fd9e1b0 0x56380fda05a0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=274faf2923ae5c5 server_cookie=9572a9555fce180e in_seq=0 out_seq=0 2024-09-10T09:49:38.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:38.364+0000 7f11eaa00640 10 client.27668 ms_handle_connect on v2:172.21.15.159:6838/2073739599 2024-09-10T09:49:38.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:38.364+0000 7f11eaa00640 1 -- 172.21.15.5:0/492099397 <== mds.1 v2:172.21.15.159:6838/2073739599 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56380fc85e60 con 0x56380fd9e1b0 2024-09-10T09:49:38.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:38.364+0000 7f11eaa00640 10 client.27668 handle_command_reply: tid=0 2024-09-10T09:49:38.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:38.364+0000 7f120e65f740 1 client.27668 shutdown 2024-09-10T09:49:38.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:38.365+0000 7f120e65f740 1 -- 172.21.15.5:0/492099397 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56380fd18760 msgr2=0x56380fd268e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:49:38.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:38.365+0000 7f120e65f740 1 --2- 172.21.15.5:0/492099397 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56380fd18760 0x56380fd268e0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f11f002b040 tx=0x7f11f000fef0 comp rx=0 tx=0).stop 2024-09-10T09:49:38.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:38.365+0000 7f120e65f740 1 -- 172.21.15.5:0/492099397 shutdown_connections 2024-09-10T09:49:38.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:38.365+0000 7f120e65f740 1 --2- 172.21.15.5:0/492099397 >> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] conn(0x56380fd9e1b0 0x56380fda05a0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:38.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:38.365+0000 7f120e65f740 1 --2- 172.21.15.5:0/492099397 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56380fd23020 0x56380fd26e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:38.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:38.365+0000 7f120e65f740 1 --2- 172.21.15.5:0/492099397 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56380fd18760 0x56380fd268e0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:38.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:38.365+0000 7f120e65f740 1 --2- 172.21.15.5:0/492099397 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56380fd17e40 0x56380fd2b560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:38.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:38.365+0000 7f11eaa00640 0 client.27668 ms_handle_reset on v2:172.21.15.159:6838/2073739599 2024-09-10T09:49:38.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:38.365+0000 7f120e65f740 1 -- 172.21.15.5:0/492099397 >> 172.21.15.5:0/492099397 conn(0x56380fd143d0 msgr2=0x56380fd21f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:49:38.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:38.365+0000 7f120e65f740 1 -- 172.21.15.5:0/492099397 shutdown_connections 2024-09-10T09:49:38.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:38.365+0000 7f120e65f740 1 -- 172.21.15.5:0/492099397 wait complete. 2024-09-10T09:49:38.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:38.365+0000 7f120e65f740 20 client.27668 trim_cache size 0 max 16384 2024-09-10T09:49:38.371 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-10T09:49:38.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:38.366+0000 7f120a400640 1 -- 172.21.15.5:0/328854497 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f11ec0a2c00 msgr2=0x7f11ec0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:49:38.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:38.366+0000 7f120a400640 1 --2- 172.21.15.5:0/328854497 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f11ec0a2c00 0x7f11ec0a50c0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f11f800cfd0 tx=0x7f11f800cf20 comp rx=0 tx=0).stop 2024-09-10T09:49:38.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:38.366+0000 7f120a400640 1 -- 172.21.15.5:0/328854497 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1204076720 msgr2=0x7f12041a4600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:49:38.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:38.366+0000 7f120a400640 1 --2- 172.21.15.5:0/328854497 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1204076720 0x7f12041a4600 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f11f800e9f0 tx=0x7f11f800eeb0 comp rx=0 tx=0).stop 2024-09-10T09:49:38.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:38.366+0000 7f120a400640 1 -- 172.21.15.5:0/328854497 shutdown_connections 2024-09-10T09:49:38.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:38.366+0000 7f120a400640 1 --2- 172.21.15.5:0/328854497 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f11ec0a2c00 0x7f11ec0a50c0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:38.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:38.366+0000 7f120a400640 1 --2- 172.21.15.5:0/328854497 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1204112870 0x7f120419e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:38.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:38.366+0000 7f120a400640 1 --2- 172.21.15.5:0/328854497 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1204077040 0x7f12041a4b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:38.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:38.366+0000 7f120a400640 1 --2- 172.21.15.5:0/328854497 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1204076720 0x7f12041a4600 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:38.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:38.366+0000 7f120a400640 1 -- 172.21.15.5:0/328854497 >> 172.21.15.5:0/328854497 conn(0x7f12040ff8d0 msgr2=0x7f1204101cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:49:38.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:38.366+0000 7f120a400640 1 -- 172.21.15.5:0/328854497 shutdown_connections 2024-09-10T09:49:38.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:38.366+0000 7f120a400640 1 -- 172.21.15.5:0/328854497 wait complete. 2024-09-10T09:49:38.538 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T09:49:39.125 INFO:tasks.workunit.client.0.smithi005.stdout: 1 687246 6.10 MB/sec execute 440 sec latency 22488.400 ms 2024-09-10T09:49:39.236 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-10T09:49:39.346 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:49:39.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:49:39 smithi005 ceph-mon[30350]: pgmap v732: 129 pgs: 129 active+clean; 259 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 773 KiB/s wr, 34 op/s 2024-09-10T09:49:39.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:49:39 smithi159 ceph-mon[35513]: pgmap v732: 129 pgs: 129 active+clean; 259 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 773 KiB/s wr, 34 op/s 2024-09-10T09:49:39.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:49:39 smithi102 ceph-mon[35442]: pgmap v732: 129 pgs: 129 active+clean; 259 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 773 KiB/s wr, 34 op/s 2024-09-10T09:49:39.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:39.956+0000 7ff558e00640 1 -- 172.21.15.5:0/1518126336 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff5541049a0 msgr2=0x7ff5541089f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:49:39.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:39.956+0000 7ff558e00640 1 --2- 172.21.15.5:0/1518126336 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff5541049a0 0x7ff5541089f0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7ff540009860 tx=0x7ff54002a340 comp rx=0 tx=0).stop 2024-09-10T09:49:39.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:39.956+0000 7ff558e00640 1 -- 172.21.15.5:0/1518126336 shutdown_connections 2024-09-10T09:49:39.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:39.956+0000 7ff558e00640 1 --2- 172.21.15.5:0/1518126336 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff554109120 0x7ff554110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:39.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:39.956+0000 7ff558e00640 1 --2- 172.21.15.5:0/1518126336 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff5541049a0 0x7ff5541089f0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:39.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:39.956+0000 7ff558e00640 1 --2- 172.21.15.5:0/1518126336 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff554103ff0 0x7ff5541043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:39.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:39.956+0000 7ff558e00640 1 -- 172.21.15.5:0/1518126336 >> 172.21.15.5:0/1518126336 conn(0x7ff5540ffa90 msgr2=0x7ff554101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:49:39.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:39.956+0000 7ff558e00640 1 -- 172.21.15.5:0/1518126336 shutdown_connections 2024-09-10T09:49:39.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:39.956+0000 7ff558e00640 1 -- 172.21.15.5:0/1518126336 wait complete. 2024-09-10T09:49:39.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:39.957+0000 7ff558e00640 1 Processor -- start 2024-09-10T09:49:39.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:39.958+0000 7ff558e00640 1 -- start start 2024-09-10T09:49:39.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:39.958+0000 7ff558e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff554103ff0 0x7ff55419a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:49:39.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:39.958+0000 7ff558e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff5541049a0 0x7ff55419abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:49:39.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:39.958+0000 7ff558e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff554109120 0x7ff55419ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:49:39.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:39.958+0000 7ff558e00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff554114140 con 0x7ff554103ff0 2024-09-10T09:49:39.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:39.958+0000 7ff558e00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7ff554113fc0 con 0x7ff5541049a0 2024-09-10T09:49:39.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:39.958+0000 7ff558e00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7ff5541142c0 con 0x7ff554109120 2024-09-10T09:49:39.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:39.958+0000 7ff551e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff554103ff0 0x7ff55419a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:49:39.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:39.958+0000 7ff551e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff554103ff0 0x7ff55419a6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40038/0 (socket says 172.21.15.5:40038) 2024-09-10T09:49:39.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:39.958+0000 7ff551e00640 1 -- 172.21.15.5:0/4105914527 learned_addr learned my addr 172.21.15.5:0/4105914527 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:49:39.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:39.959+0000 7ff551400640 1 --2- 172.21.15.5:0/4105914527 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff5541049a0 0x7ff55419abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:49:39.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:39.959+0000 7ff552800640 1 --2- 172.21.15.5:0/4105914527 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff554109120 0x7ff55419ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:49:39.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:39.959+0000 7ff551e00640 1 -- 172.21.15.5:0/4105914527 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff554109120 msgr2=0x7ff55419ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:49:39.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:39.959+0000 7ff551e00640 1 --2- 172.21.15.5:0/4105914527 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff554109120 0x7ff55419ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:39.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:39.959+0000 7ff551e00640 1 -- 172.21.15.5:0/4105914527 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff5541049a0 msgr2=0x7ff55419abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:49:39.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:39.959+0000 7ff551e00640 1 --2- 172.21.15.5:0/4105914527 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff5541049a0 0x7ff55419abe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:39.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:39.959+0000 7ff551e00640 1 -- 172.21.15.5:0/4105914527 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff55419f650 con 0x7ff554103ff0 2024-09-10T09:49:39.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:39.959+0000 7ff551e00640 1 --2- 172.21.15.5:0/4105914527 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff554103ff0 0x7ff55419a6a0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7ff54400e9c0 tx=0x7ff54400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:49:39.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:39.960+0000 7ff54aa00640 1 -- 172.21.15.5:0/4105914527 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff54400cde0 con 0x7ff554103ff0 2024-09-10T09:49:39.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:39.960+0000 7ff558e00640 1 -- 172.21.15.5:0/4105914527 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff55419f940 con 0x7ff554103ff0 2024-09-10T09:49:39.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:39.960+0000 7ff54aa00640 1 -- 172.21.15.5:0/4105914527 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff5440035f0 con 0x7ff554103ff0 2024-09-10T09:49:39.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:39.960+0000 7ff558e00640 1 -- 172.21.15.5:0/4105914527 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5541a7220 con 0x7ff554103ff0 2024-09-10T09:49:39.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:39.960+0000 7ff54aa00640 1 -- 172.21.15.5:0/4105914527 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff544018630 con 0x7ff554103ff0 2024-09-10T09:49:39.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:39.961+0000 7ff558e00640 1 -- 172.21.15.5:0/4105914527 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff5541043d0 con 0x7ff554103ff0 2024-09-10T09:49:39.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:39.964+0000 7ff54aa00640 1 -- 172.21.15.5:0/4105914527 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7ff5440037a0 con 0x7ff554103ff0 2024-09-10T09:49:39.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:39.966+0000 7ff54aa00640 1 --2- 172.21.15.5:0/4105914527 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7ff5380a2b80 0x7ff5380a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:49:39.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:39.966+0000 7ff54aa00640 1 -- 172.21.15.5:0/4105914527 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7ff5440cf710 con 0x7ff554103ff0 2024-09-10T09:49:39.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:39.966+0000 7ff551400640 1 --2- 172.21.15.5:0/4105914527 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7ff5380a2b80 0x7ff5380a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:49:39.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:39.967+0000 7ff551400640 1 --2- 172.21.15.5:0/4105914527 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7ff5380a2b80 0x7ff5380a5040 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7ff54002a930 tx=0x7ff5400056d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:49:39.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:39.970+0000 7ff54aa00640 1 -- 172.21.15.5:0/4105914527 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff54409c220 con 0x7ff554103ff0 2024-09-10T09:49:40.030 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:49:40.125 INFO:tasks.workunit.client.0.smithi005.stdout: 1 687246 6.09 MB/sec execute 441 sec latency 23488.462 ms 2024-09-10T09:49:40.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:40.197+0000 7ff558e00640 1 -- 172.21.15.5:0/4105914527 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff55410c390 con 0x7ff554103ff0 2024-09-10T09:49:40.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:40.198+0000 7ff54aa00640 1 -- 172.21.15.5:0/4105914527 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) ==== 94+0+11969 (secure 0 0 0) 0x7ff54409bb20 con 0x7ff554103ff0 2024-09-10T09:49:40.200 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-10T09:49:40.200 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":36,"btime":"2024-09-10T09:48:07:349075+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14454,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/2970594311","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2970594311},{"type":"v1","addr":"172.21.15.5:6835","nonce":2970594311}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14514,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6839/3616502174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":3616502174},{"type":"v1","addr":"172.21.15.5:6839","nonce":3616502174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":36,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T09:35:29.626287+0000","modified":"2024-09-10T09:48:06.970127+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24542,"mds_1":24535,"mds_2":24511,"mds_3":24506,"mds_4":24443},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14520":{"gid":14520,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/2073599842","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":2073599842},{"type":"v1","addr":"172.21.15.5:6841","nonce":2073599842}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"k","rank":4,"incarnation":26,"state":"up:active","state_seq":24,"addr":"172.21.15.102:6833/2457007427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":2457007427},{"type":"v1","addr":"172.21.15.102:6833","nonce":2457007427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/356445159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":356445159},{"type":"v1","addr":"172.21.15.159:6833","nonce":356445159}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6837/2894314507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":2894314507},{"type":"v1","addr":"172.21.15.5:6837","nonce":2894314507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.102:6835/1647549111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":1647549111},{"type":"v1","addr":"172.21.15.102:6835","nonce":1647549111}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6835/952657242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":952657242},{"type":"v1","addr":"172.21.15.159:6835","nonce":952657242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":19,"addr":"172.21.15.102:6837/2689776764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":2689776764},{"type":"v1","addr":"172.21.15.102:6837","nonce":2689776764}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.159:6837/620627639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":620627639},{"type":"v1","addr":"172.21.15.159:6837","nonce":620627639}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24535":{"gid":24535,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":18,"addr":"172.21.15.159:6839/2073739599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":2073739599},{"type":"v1","addr":"172.21.15.159:6839","nonce":2073739599}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24542":{"gid":24542,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.102:6839/335409749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6838","nonce":335409749},{"type":"v1","addr":"172.21.15.102:6839","nonce":335409749}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24542,"qdb_cluster":[24542,24535,24511,24506,24443]},"id":1}]} 2024-09-10T09:49:40.201 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 36 2024-09-10T09:49:40.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:40.200+0000 7ff558e00640 1 -- 172.21.15.5:0/4105914527 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7ff5380a2b80 msgr2=0x7ff5380a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:49:40.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:40.200+0000 7ff558e00640 1 --2- 172.21.15.5:0/4105914527 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7ff5380a2b80 0x7ff5380a5040 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7ff54002a930 tx=0x7ff5400056d0 comp rx=0 tx=0).stop 2024-09-10T09:49:40.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:40.200+0000 7ff558e00640 1 -- 172.21.15.5:0/4105914527 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff554103ff0 msgr2=0x7ff55419a6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:49:40.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:40.200+0000 7ff558e00640 1 --2- 172.21.15.5:0/4105914527 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff554103ff0 0x7ff55419a6a0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7ff54400e9c0 tx=0x7ff54400ee80 comp rx=0 tx=0).stop 2024-09-10T09:49:40.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:40.200+0000 7ff558e00640 1 -- 172.21.15.5:0/4105914527 shutdown_connections 2024-09-10T09:49:40.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:40.200+0000 7ff558e00640 1 --2- 172.21.15.5:0/4105914527 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7ff5380a2b80 0x7ff5380a5040 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:40.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:40.200+0000 7ff558e00640 1 --2- 172.21.15.5:0/4105914527 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff554109120 0x7ff55419ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:40.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:40.200+0000 7ff558e00640 1 --2- 172.21.15.5:0/4105914527 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff5541049a0 0x7ff55419abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:40.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:40.200+0000 7ff558e00640 1 --2- 172.21.15.5:0/4105914527 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff554103ff0 0x7ff55419a6a0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:40.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:40.200+0000 7ff558e00640 1 -- 172.21.15.5:0/4105914527 >> 172.21.15.5:0/4105914527 conn(0x7ff5540ffa90 msgr2=0x7ff554100d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:49:40.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:40.201+0000 7ff558e00640 1 -- 172.21.15.5:0/4105914527 shutdown_connections 2024-09-10T09:49:40.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:40.201+0000 7ff558e00640 1 -- 172.21.15.5:0/4105914527 wait complete. 2024-09-10T09:49:40.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:49:40 smithi005 ceph-mon[30350]: from='client.? 172.21.15.5:0/4105914527' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:49:40.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:49:40 smithi159 ceph-mon[35513]: from='client.? 172.21.15.5:0/4105914527' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:49:40.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:49:40 smithi102 ceph-mon[35442]: from='client.? 172.21.15.5:0/4105914527' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:49:41.115 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24542 quiesce db --set-id a33fa5fe --release --await' 2024-09-10T09:49:41.115 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24542 quiesce db --set-id a33fa5fe --release --await 2024-09-10T09:49:41.125 INFO:tasks.workunit.client.0.smithi005.stdout: 1 687246 6.07 MB/sec execute 442 sec latency 24488.520 ms 2024-09-10T09:49:41.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:41.395+0000 7f45df400640 1 -- 172.21.15.5:0/1312433489 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f45d8112780 msgr2=0x7f45d8114bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:49:41.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:41.395+0000 7f45df400640 1 --2- 172.21.15.5:0/1312433489 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f45d8112780 0x7f45d8114bf0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f45d0009ca0 tx=0x7f45d002a950 comp rx=0 tx=0).stop 2024-09-10T09:49:41.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:41.395+0000 7f45df400640 1 -- 172.21.15.5:0/1312433489 shutdown_connections 2024-09-10T09:49:41.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:41.395+0000 7f45df400640 1 --2- 172.21.15.5:0/1312433489 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f45d8112780 0x7f45d8114bf0 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:41.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:41.395+0000 7f45df400640 1 --2- 172.21.15.5:0/1312433489 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f45d8100e20 0x7f45d81121f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:41.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:41.395+0000 7f45df400640 1 --2- 172.21.15.5:0/1312433489 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f45d8100500 0x7f45d81008e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:41.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:41.395+0000 7f45df400640 1 -- 172.21.15.5:0/1312433489 >> 172.21.15.5:0/1312433489 conn(0x7f45d80fc5c0 msgr2=0x7f45d80fe9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:49:41.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:41.395+0000 7f45df400640 1 -- 172.21.15.5:0/1312433489 shutdown_connections 2024-09-10T09:49:41.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:41.395+0000 7f45df400640 1 -- 172.21.15.5:0/1312433489 wait complete. 2024-09-10T09:49:41.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:41.396+0000 7f45df400640 1 Processor -- start 2024-09-10T09:49:41.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:41.396+0000 7f45df400640 1 -- start start 2024-09-10T09:49:41.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:41.397+0000 7f45df400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f45d8100500 0x7f45d81a4690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:49:41.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:41.397+0000 7f45df400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f45d8100e20 0x7f45d81a4bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:49:41.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:41.397+0000 7f45df400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f45d8112780 0x7f45d819e810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:49:41.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:41.397+0000 7f45df400640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f45d8118530 con 0x7f45d8100500 2024-09-10T09:49:41.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:41.397+0000 7f45df400640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f45d81183b0 con 0x7f45d8100e20 2024-09-10T09:49:41.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:41.397+0000 7f45df400640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f45d81186b0 con 0x7f45d8112780 2024-09-10T09:49:41.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:41.397+0000 7f45dca00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f45d8100500 0x7f45d81a4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:49:41.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:41.397+0000 7f45dd400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f45d8112780 0x7f45d819e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:49:41.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:41.397+0000 7f45dd400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f45d8112780 0x7f45d819e810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:56648/0 (socket says 172.21.15.5:56648) 2024-09-10T09:49:41.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:41.397+0000 7f45dd400640 1 -- 172.21.15.5:0/2107361718 learned_addr learned my addr 172.21.15.5:0/2107361718 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:49:41.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:41.397+0000 7f45cfe00640 1 --2- 172.21.15.5:0/2107361718 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f45d8100e20 0x7f45d81a4bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:49:41.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:41.398+0000 7f45dd400640 1 -- 172.21.15.5:0/2107361718 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f45d8100e20 msgr2=0x7f45d81a4bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:49:41.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:41.398+0000 7f45dd400640 1 --2- 172.21.15.5:0/2107361718 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f45d8100e20 0x7f45d81a4bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:41.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:41.398+0000 7f45dd400640 1 -- 172.21.15.5:0/2107361718 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f45d8100500 msgr2=0x7f45d81a4690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:49:41.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:41.398+0000 7f45dd400640 1 --2- 172.21.15.5:0/2107361718 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f45d8100500 0x7f45d81a4690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:41.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:41.398+0000 7f45dd400640 1 -- 172.21.15.5:0/2107361718 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45d819efb0 con 0x7f45d8112780 2024-09-10T09:49:41.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:41.398+0000 7f45dd400640 1 --2- 172.21.15.5:0/2107361718 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f45d8112780 0x7f45d819e810 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f45d0004af0 tx=0x7f45d002f3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:49:41.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:41.399+0000 7f45d6000640 1 -- 172.21.15.5:0/2107361718 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45d0034040 con 0x7f45d8112780 2024-09-10T09:49:41.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:41.399+0000 7f45df400640 1 -- 172.21.15.5:0/2107361718 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45d819f240 con 0x7f45d8112780 2024-09-10T09:49:41.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:41.399+0000 7f45d6000640 1 -- 172.21.15.5:0/2107361718 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f45d0030920 con 0x7f45d8112780 2024-09-10T09:49:41.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:41.399+0000 7f45d6000640 1 -- 172.21.15.5:0/2107361718 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45d0004670 con 0x7f45d8112780 2024-09-10T09:49:41.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:41.399+0000 7f45df400640 1 -- 172.21.15.5:0/2107361718 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45d81ab4f0 con 0x7f45d8112780 2024-09-10T09:49:41.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:41.402+0000 7f45d6000640 1 -- 172.21.15.5:0/2107361718 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f45d0037020 con 0x7f45d8112780 2024-09-10T09:49:41.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:41.403+0000 7f45e19fd740 1 Processor -- start 2024-09-10T09:49:41.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:41.403+0000 7f45e19fd740 1 -- start start 2024-09-10T09:49:41.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:41.404+0000 7f45e19fd740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5589cf19cc70 0x5589cf19d050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:49:41.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:41.404+0000 7f45e19fd740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5589cf19d590 0x5589cf1a7910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:49:41.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:41.404+0000 7f45e19fd740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5589cf1a7e50 0x5589cf1aa240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:49:41.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:41.404+0000 7f45e19fd740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5589cf1033e0 con 0x5589cf1a7e50 2024-09-10T09:49:41.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:41.404+0000 7f45e19fd740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x5589cf0aad80 con 0x5589cf19d590 2024-09-10T09:49:41.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:41.404+0000 7f45e19fd740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5589cf0ff3a0 con 0x5589cf19cc70 2024-09-10T09:49:41.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:41.404+0000 7f45dca00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5589cf19d590 0x5589cf1a7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:49:41.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:41.404+0000 7f45d6000640 1 --2- 172.21.15.5:0/2107361718 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f45bc0a2b80 0x7f45bc0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:49:41.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:41.404+0000 7f45dca00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5589cf19d590 0x5589cf1a7910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:42384/0 (socket says 172.21.15.5:42384) 2024-09-10T09:49:41.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:41.404+0000 7f45dca00640 1 -- 172.21.15.5:0/2971184751 learned_addr learned my addr 172.21.15.5:0/2971184751 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:49:41.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:41.404+0000 7f45cfe00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5589cf19cc70 0x5589cf19d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:49:41.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:41.404+0000 7f45d6000640 1 -- 172.21.15.5:0/2107361718 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f45d00f0160 con 0x7f45d8112780 2024-09-10T09:49:41.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:41.404+0000 7f45cfe00640 1 --2- 172.21.15.5:0/2971184751 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5589cf1a7e50 0x5589cf1aa240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:49:41.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:41.405+0000 7f45dca00640 1 -- 172.21.15.5:0/2971184751 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5589cf19cc70 msgr2=0x5589cf19d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:49:41.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:41.405+0000 7f45dca00640 1 --2- 172.21.15.5:0/2971184751 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5589cf19cc70 0x5589cf19d050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:41.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:41.405+0000 7f45dca00640 1 -- 172.21.15.5:0/2971184751 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5589cf1a7e50 msgr2=0x5589cf1aa240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:49:41.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:41.405+0000 7f45dca00640 1 --2- 172.21.15.5:0/2971184751 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5589cf1a7e50 0x5589cf1aa240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:41.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:41.405+0000 7f45dca00640 1 -- 172.21.15.5:0/2971184751 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5589cf0c9ae0 con 0x5589cf19d590 2024-09-10T09:49:41.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:41.405+0000 7f45dd400640 1 --2- 172.21.15.5:0/2107361718 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f45bc0a2b80 0x7f45bc0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:49:41.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:41.405+0000 7f45dca00640 1 --2- 172.21.15.5:0/2971184751 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5589cf19d590 0x5589cf1a7910 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f45c8009d20 tx=0x7f45c8011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1b01d4e03e76c7c0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:49:41.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:41.405+0000 7f45cdc00640 1 -- 172.21.15.5:0/2971184751 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45c801e070 con 0x5589cf19d590 2024-09-10T09:49:41.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:41.405+0000 7f45cdc00640 1 -- 172.21.15.5:0/2971184751 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f45c8019b10 con 0x5589cf19d590 2024-09-10T09:49:41.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:41.405+0000 7f45cdc00640 1 -- 172.21.15.5:0/2971184751 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45c800eb80 con 0x5589cf19d590 2024-09-10T09:49:41.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:41.406+0000 7f45e19fd740 1 -- 172.21.15.5:0/2971184751 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5589cf19d590 msgr2=0x5589cf1a7910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:49:41.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:41.406+0000 7f45dd400640 1 --2- 172.21.15.5:0/2107361718 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f45bc0a2b80 0x7f45bc0a5040 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f45d002fde0 tx=0x7f45d002a950 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:49:41.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:41.406+0000 7f45e19fd740 1 --2- 172.21.15.5:0/2971184751 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5589cf19d590 0x5589cf1a7910 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f45c8009d20 tx=0x7f45c8011810 comp rx=0 tx=0).stop 2024-09-10T09:49:41.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:41.406+0000 7f45e19fd740 1 -- 172.21.15.5:0/2971184751 shutdown_connections 2024-09-10T09:49:41.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:41.406+0000 7f45e19fd740 1 --2- 172.21.15.5:0/2971184751 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5589cf1a7e50 0x5589cf1aa240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:41.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:41.406+0000 7f45e19fd740 1 --2- 172.21.15.5:0/2971184751 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5589cf19d590 0x5589cf1a7910 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:41.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:41.406+0000 7f45e19fd740 1 --2- 172.21.15.5:0/2971184751 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5589cf19cc70 0x5589cf19d050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:41.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:41.406+0000 7f45e19fd740 1 -- 172.21.15.5:0/2971184751 >> 172.21.15.5:0/2971184751 conn(0x5589cf199200 msgr2=0x5589cf1995e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:49:41.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:41.406+0000 7f45e19fd740 1 -- 172.21.15.5:0/2971184751 shutdown_connections 2024-09-10T09:49:41.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:41.406+0000 7f45e19fd740 1 -- 172.21.15.5:0/2971184751 wait complete. 2024-09-10T09:49:41.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:41.407+0000 7f45e19fd740 1 -- start start 2024-09-10T09:49:41.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:41.407+0000 7f45e19fd740 1 Processor -- start 2024-09-10T09:49:41.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:41.408+0000 7f45e19fd740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5589cf19cc70 0x5589cf1a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:49:41.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:41.408+0000 7f45e19fd740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5589cf19d590 0x5589cf1a5310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:49:41.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:41.408+0000 7f45e19fd740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5589cf1a7e50 0x5589cf1a5850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:49:41.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:41.408+0000 7f45e19fd740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5589cf1aaa20 con 0x5589cf1a7e50 2024-09-10T09:49:41.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:41.408+0000 7f45e19fd740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x5589cf0be0f0 con 0x5589cf19cc70 2024-09-10T09:49:41.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:41.408+0000 7f45e19fd740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5589cf1aaba0 con 0x5589cf19d590 2024-09-10T09:49:41.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:41.408+0000 7f45e19fd740 10 client.0 _ll_register_callbacks cb 0x5589cf0a5bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:49:41.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:41.408+0000 7f45dca00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5589cf19d590 0x5589cf1a5310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:49:41.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:41.408+0000 7f45dca00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5589cf19d590 0x5589cf1a5310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:56668/0 (socket says 172.21.15.5:56668) 2024-09-10T09:49:41.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:41.408+0000 7f45dca00640 1 -- 172.21.15.5:0/1222162861 learned_addr learned my addr 172.21.15.5:0/1222162861 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:49:41.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:41.408+0000 7f45cfe00640 1 --2- 172.21.15.5:0/1222162861 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5589cf1a7e50 0x5589cf1a5850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:49:41.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:41.408+0000 7f45cfe00640 1 --2- 172.21.15.5:0/1222162861 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5589cf19cc70 0x5589cf1a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:49:41.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:41.409+0000 7f45dca00640 1 -- 172.21.15.5:0/1222162861 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5589cf19cc70 msgr2=0x5589cf1a4dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:49:41.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:41.409+0000 7f45dca00640 1 --2- 172.21.15.5:0/1222162861 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5589cf19cc70 0x5589cf1a4dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:41.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:41.409+0000 7f45dca00640 1 -- 172.21.15.5:0/1222162861 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5589cf1a7e50 msgr2=0x5589cf1a5850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:49:41.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:41.409+0000 7f45dca00640 1 --2- 172.21.15.5:0/1222162861 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5589cf1a7e50 0x5589cf1a5850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:41.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:41.409+0000 7f45dca00640 1 -- 172.21.15.5:0/1222162861 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5589cee743c0 con 0x5589cf19d590 2024-09-10T09:49:41.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:41.409+0000 7f45dca00640 1 --2- 172.21.15.5:0/1222162861 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5589cf19d590 0x5589cf1a5310 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f45c800e720 tx=0x7f45c8019660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=47e5c2965afe51b4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:49:41.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:41.409+0000 7f45c6000640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-09-10T09:49:41.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:41.409+0000 7f45c6000640 1 -- 172.21.15.5:0/1222162861 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45c801e070 con 0x5589cf19d590 2024-09-10T09:49:41.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:41.409+0000 7f45e19fd740 1 -- 172.21.15.5:0/1222162861 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f45c8019b10 con 0x5589cf19d590 2024-09-10T09:49:41.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:41.409+0000 7f45c6000640 1 -- 172.21.15.5:0/1222162861 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f45c800ea80 con 0x5589cf19d590 2024-09-10T09:49:41.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:41.410+0000 7f45c6000640 1 -- 172.21.15.5:0/1222162861 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f45c8031050 con 0x5589cf19d590 2024-09-10T09:49:41.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:41.410+0000 7f45e19fd740 10 client.27694 fetch_fsmap learned FSMap version 36 2024-09-10T09:49:41.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:41.410+0000 7f45e19fd740 1 -- 172.21.15.5:0/1222162861 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x5589cf1b6ae0 con 0x5589cf19d590 2024-09-10T09:49:41.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:41.410+0000 7f45c6000640 1 -- 172.21.15.5:0/1222162861 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f45c8019b10 con 0x5589cf19d590 2024-09-10T09:49:41.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:41.410+0000 7f45c6000640 1 -- 172.21.15.5:0/1222162861 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 36) ==== 8370+0+0 (secure 0 0 0) 0x7f45c80046c0 con 0x5589cf19d590 2024-09-10T09:49:41.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:41.410+0000 7f45e19fd740 10 client.27694 fetch_fsmap finished waiting for FSMap version 36 2024-09-10T09:49:41.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:41.410+0000 7f45e19fd740 10 client.27694 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T09:49:41.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:41.410+0000 7f45e19fd740 20 client.27694 populate_metadata read hostname 'smithi005' 2024-09-10T09:49:41.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:41.410+0000 7f45e19fd740 1 --2- 172.21.15.5:0/1222162861 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x5589cf1b7ef0 0x5589cf1ba2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:49:41.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:41.411+0000 7f45e19fd740 4 client.27694 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:49:41.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:41.411+0000 7f45e19fd740 1 -- 172.21.15.5:0/1222162861 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5589cf110950 con 0x5589cf1b7ef0 2024-09-10T09:49:41.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:41.411+0000 7f45cfe00640 1 --2- 172.21.15.5:0/1222162861 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x5589cf1b7ef0 0x5589cf1ba2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:49:41.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:41.411+0000 7f45cfe00640 1 --2- 172.21.15.5:0/1222162861 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x5589cf1b7ef0 0x5589cf1ba2e0 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b0c0bf3eb997060f server_cookie=295785a73cb1c4fa in_seq=0 out_seq=0 2024-09-10T09:49:41.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:41.411+0000 7f45c6000640 10 client.27694 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T09:49:41.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:41.412+0000 7f45c6000640 1 -- 172.21.15.5:0/1222162861 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5589cf110950 con 0x5589cf1b7ef0 2024-09-10T09:49:41.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:41.412+0000 7f45c6000640 10 client.27694 handle_command_reply: tid=0 2024-09-10T09:49:41.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:41.412+0000 7f45e19fd740 1 client.27694 shutdown 2024-09-10T09:49:41.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:41.412+0000 7f45e19fd740 1 -- 172.21.15.5:0/1222162861 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5589cf19d590 msgr2=0x5589cf1a5310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:49:41.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:41.412+0000 7f45e19fd740 1 --2- 172.21.15.5:0/1222162861 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5589cf19d590 0x5589cf1a5310 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f45c800e720 tx=0x7f45c8019660 comp rx=0 tx=0).stop 2024-09-10T09:49:41.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:41.413+0000 7f45e19fd740 1 -- 172.21.15.5:0/1222162861 shutdown_connections 2024-09-10T09:49:41.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:41.413+0000 7f45e19fd740 1 --2- 172.21.15.5:0/1222162861 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x5589cf1b7ef0 0x5589cf1ba2e0 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:41.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:41.413+0000 7f45e19fd740 1 --2- 172.21.15.5:0/1222162861 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5589cf1a7e50 0x5589cf1a5850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:41.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:41.413+0000 7f45e19fd740 1 --2- 172.21.15.5:0/1222162861 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5589cf19d590 0x5589cf1a5310 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:41.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:41.413+0000 7f45e19fd740 1 --2- 172.21.15.5:0/1222162861 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5589cf19cc70 0x5589cf1a4dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:41.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:41.413+0000 7f45e19fd740 1 -- 172.21.15.5:0/1222162861 >> 172.21.15.5:0/1222162861 conn(0x5589cf199200 msgr2=0x5589cf19eee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:49:41.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:41.413+0000 7f45c6000640 0 client.27694 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T09:49:41.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:41.413+0000 7f45e19fd740 1 -- 172.21.15.5:0/1222162861 shutdown_connections 2024-09-10T09:49:41.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:41.413+0000 7f45e19fd740 1 -- 172.21.15.5:0/1222162861 wait complete. 2024-09-10T09:49:41.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:41.413+0000 7f45e19fd740 20 client.27694 trim_cache size 0 max 16384 2024-09-10T09:49:41.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:41.427+0000 7f45e19fd740 1 Processor -- start 2024-09-10T09:49:41.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:41.427+0000 7f45e19fd740 1 -- start start 2024-09-10T09:49:41.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:41.427+0000 7f45e19fd740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5589cf19cc70 0x5589cf1a6630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:49:41.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:41.427+0000 7f45e19fd740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5589cf19d590 0x5589cf1bfac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:49:41.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:41.427+0000 7f45e19fd740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5589cf1a7e50 0x5589cf1c0000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:49:41.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:41.427+0000 7f45e19fd740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f45d81b53c0 con 0x5589cf1a7e50 2024-09-10T09:49:41.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:41.427+0000 7f45e19fd740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f45d81b6f00 con 0x5589cf19cc70 2024-09-10T09:49:41.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:41.427+0000 7f45e19fd740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f45d81b5a90 con 0x5589cf19d590 2024-09-10T09:49:41.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:41.428+0000 7f45dca00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5589cf19d590 0x5589cf1bfac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:49:41.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:41.428+0000 7f45dca00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5589cf19d590 0x5589cf1bfac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:56674/0 (socket says 172.21.15.5:56674) 2024-09-10T09:49:41.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:41.428+0000 7f45dca00640 1 -- 172.21.15.5:0/153483568 learned_addr learned my addr 172.21.15.5:0/153483568 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:49:41.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:41.428+0000 7f45cfe00640 1 --2- 172.21.15.5:0/153483568 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5589cf19cc70 0x5589cf1a6630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:49:41.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:41.428+0000 7f45cfe00640 1 --2- 172.21.15.5:0/153483568 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5589cf1a7e50 0x5589cf1c0000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:49:41.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:41.428+0000 7f45dca00640 1 -- 172.21.15.5:0/153483568 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5589cf19cc70 msgr2=0x5589cf1a6630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:49:41.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:41.428+0000 7f45dca00640 1 --2- 172.21.15.5:0/153483568 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5589cf19cc70 0x5589cf1a6630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:41.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:41.428+0000 7f45dca00640 1 -- 172.21.15.5:0/153483568 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5589cf1a7e50 msgr2=0x5589cf1c0000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:49:41.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:41.428+0000 7f45dca00640 1 --2- 172.21.15.5:0/153483568 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5589cf1a7e50 0x5589cf1c0000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:41.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:41.428+0000 7f45dca00640 1 -- 172.21.15.5:0/153483568 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5589cf19fc20 con 0x5589cf19d590 2024-09-10T09:49:41.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:41.428+0000 7f45cfe00640 1 --2- 172.21.15.5:0/153483568 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5589cf1a7e50 0x5589cf1c0000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:49:41.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:41.428+0000 7f45dca00640 1 --2- 172.21.15.5:0/153483568 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5589cf19d590 0x5589cf1bfac0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f45c8011850 tx=0x7f45c8019710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=128392f9ae3e7d43 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:49:41.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:41.429+0000 7f45cd200640 1 -- 172.21.15.5:0/153483568 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45c803a070 con 0x5589cf19d590 2024-09-10T09:49:41.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:41.429+0000 7f45cd200640 1 -- 172.21.15.5:0/153483568 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f45c8015030 con 0x5589cf19d590 2024-09-10T09:49:41.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:41.429+0000 7f45cd200640 1 -- 172.21.15.5:0/153483568 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45c801e040 con 0x5589cf19d590 2024-09-10T09:49:41.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:41.429+0000 7f45e19fd740 1 -- 172.21.15.5:0/153483568 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5589cf19d590 msgr2=0x5589cf1bfac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:49:41.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:41.429+0000 7f45e19fd740 1 --2- 172.21.15.5:0/153483568 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5589cf19d590 0x5589cf1bfac0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f45c8011850 tx=0x7f45c8019710 comp rx=0 tx=0).stop 2024-09-10T09:49:41.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:41.429+0000 7f45e19fd740 1 -- 172.21.15.5:0/153483568 shutdown_connections 2024-09-10T09:49:41.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:41.430+0000 7f45e19fd740 1 --2- 172.21.15.5:0/153483568 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5589cf1a7e50 0x5589cf1c0000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:41.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:41.430+0000 7f45e19fd740 1 --2- 172.21.15.5:0/153483568 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5589cf19d590 0x5589cf1bfac0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:41.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:41.430+0000 7f45e19fd740 1 --2- 172.21.15.5:0/153483568 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5589cf19cc70 0x5589cf1a6630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:41.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:41.430+0000 7f45e19fd740 1 -- 172.21.15.5:0/153483568 >> 172.21.15.5:0/153483568 conn(0x5589cf199200 msgr2=0x5589cf19eee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:49:41.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:41.430+0000 7f45e19fd740 1 -- 172.21.15.5:0/153483568 shutdown_connections 2024-09-10T09:49:41.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:41.430+0000 7f45e19fd740 1 -- 172.21.15.5:0/153483568 wait complete. 2024-09-10T09:49:41.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:41.430+0000 7f45e19fd740 1 -- start start 2024-09-10T09:49:41.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:41.430+0000 7f45e19fd740 1 Processor -- start 2024-09-10T09:49:41.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:41.431+0000 7f45e19fd740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5589cf19cc70 0x5589cf1b02a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:49:41.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:41.431+0000 7f45e19fd740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5589cf19d590 0x5589cf1ab5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:49:41.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:41.431+0000 7f45e19fd740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5589cf1a7e50 0x5589cf1abb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:49:41.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:41.431+0000 7f45e19fd740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f45d81b6830 con 0x5589cf19d590 2024-09-10T09:49:41.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:41.431+0000 7f45e19fd740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f45d81b6160 con 0x5589cf19cc70 2024-09-10T09:49:41.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:41.431+0000 7f45e19fd740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f45d81b33a0 con 0x5589cf1a7e50 2024-09-10T09:49:41.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:41.431+0000 7f45cfe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5589cf19cc70 0x5589cf1b02a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:49:41.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:41.431+0000 7f45cfe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5589cf19cc70 0x5589cf1b02a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:42410/0 (socket says 172.21.15.5:42410) 2024-09-10T09:49:41.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:41.431+0000 7f45cfe00640 1 -- 172.21.15.5:0/2302653693 learned_addr learned my addr 172.21.15.5:0/2302653693 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:49:41.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:41.431+0000 7f45e19fd740 10 client.0 _ll_register_callbacks cb 0x7f45c801c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:49:41.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:41.431+0000 7f45cfe00640 1 --2- 172.21.15.5:0/2302653693 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5589cf1a7e50 0x5589cf1abb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:49:41.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:41.431+0000 7f45dca00640 1 --2- 172.21.15.5:0/2302653693 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5589cf19d590 0x5589cf1ab5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:49:41.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:41.432+0000 7f45cfe00640 1 -- 172.21.15.5:0/2302653693 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5589cf1a7e50 msgr2=0x5589cf1abb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:49:41.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:41.432+0000 7f45cfe00640 1 --2- 172.21.15.5:0/2302653693 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5589cf1a7e50 0x5589cf1abb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:41.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:41.432+0000 7f45cfe00640 1 -- 172.21.15.5:0/2302653693 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5589cf19d590 msgr2=0x5589cf1ab5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:49:41.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:41.432+0000 7f45cfe00640 1 --2- 172.21.15.5:0/2302653693 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5589cf19d590 0x5589cf1ab5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:41.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:41.432+0000 7f45cfe00640 1 -- 172.21.15.5:0/2302653693 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5589cf1ac080 con 0x5589cf19cc70 2024-09-10T09:49:41.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:41.432+0000 7f45dca00640 1 --2- 172.21.15.5:0/2302653693 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5589cf19d590 0x5589cf1ab5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T09:49:41.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:41.432+0000 7f45cfe00640 1 --2- 172.21.15.5:0/2302653693 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5589cf19cc70 0x5589cf1b02a0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f45d8114bc0 tx=0x7f45c00046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c01397018c3112 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:49:41.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:41.432+0000 7f45c6000640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-09-10T09:49:41.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:41.432+0000 7f45c6000640 1 -- 172.21.15.5:0/2302653693 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45c0004030 con 0x5589cf19cc70 2024-09-10T09:49:41.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:41.432+0000 7f45c6000640 1 -- 172.21.15.5:0/2302653693 <== mon.2 v2:172.21.15.102:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f45c0009040 con 0x5589cf19cc70 2024-09-10T09:49:41.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:41.432+0000 7f45e19fd740 1 -- 172.21.15.5:0/2302653693 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f45c8015030 con 0x5589cf19cc70 2024-09-10T09:49:41.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:41.433+0000 7f45c6000640 1 -- 172.21.15.5:0/2302653693 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f45c00041d0 con 0x5589cf19cc70 2024-09-10T09:49:41.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:41.433+0000 7f45e19fd740 10 client.27692 fetch_fsmap learned FSMap version 36 2024-09-10T09:49:41.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:41.433+0000 7f45e19fd740 1 -- 172.21.15.5:0/2302653693 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x5589cf222820 con 0x5589cf19cc70 2024-09-10T09:49:41.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:41.433+0000 7f45c6000640 1 -- 172.21.15.5:0/2302653693 <== mon.2 v2:172.21.15.102:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f45c8015030 con 0x5589cf19cc70 2024-09-10T09:49:41.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:41.433+0000 7f45c6000640 1 -- 172.21.15.5:0/2302653693 <== mon.2 v2:172.21.15.102:3300/0 5 ==== fsmap(e 36) ==== 8370+0+0 (secure 0 0 0) 0x7f45c0020040 con 0x5589cf19cc70 2024-09-10T09:49:41.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:41.433+0000 7f45e19fd740 10 client.27692 fetch_fsmap finished waiting for FSMap version 36 2024-09-10T09:49:41.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:41.433+0000 7f45e19fd740 10 client.27692 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T09:49:41.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:41.433+0000 7f45e19fd740 20 client.27692 populate_metadata read hostname 'smithi005' 2024-09-10T09:49:41.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:41.433+0000 7f45e19fd740 1 --2- 172.21.15.5:0/2302653693 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x5589cf223000 0x5589cf225420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:49:41.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:41.433+0000 7f45e19fd740 4 client.27692 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T09:49:41.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:41.433+0000 7f45e19fd740 1 -- 172.21.15.5:0/2302653693 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5589cf10ae60 con 0x5589cf223000 2024-09-10T09:49:41.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:41.434+0000 7f45dca00640 1 --2- 172.21.15.5:0/2302653693 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x5589cf223000 0x5589cf225420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:49:41.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:41.434+0000 7f45dca00640 1 --2- 172.21.15.5:0/2302653693 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x5589cf223000 0x5589cf225420 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8986ff17d593ac7 server_cookie=1a2b8516d5af9a25 in_seq=0 out_seq=0 2024-09-10T09:49:41.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:41.434+0000 7f45c6000640 10 client.27692 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T09:49:41.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:41.435+0000 7f45c6000640 1 -- 172.21.15.5:0/2302653693 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5589cf10ae60 con 0x5589cf223000 2024-09-10T09:49:41.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:41.435+0000 7f45c6000640 10 client.27692 handle_command_reply: tid=0 2024-09-10T09:49:41.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:41.435+0000 7f45e19fd740 1 client.27692 shutdown 2024-09-10T09:49:41.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:41.435+0000 7f45e19fd740 1 -- 172.21.15.5:0/2302653693 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5589cf19cc70 msgr2=0x5589cf1b02a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:49:41.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:41.435+0000 7f45e19fd740 1 --2- 172.21.15.5:0/2302653693 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5589cf19cc70 0x5589cf1b02a0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f45d8114bc0 tx=0x7f45c00046c0 comp rx=0 tx=0).stop 2024-09-10T09:49:41.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:41.435+0000 7f45e19fd740 1 -- 172.21.15.5:0/2302653693 shutdown_connections 2024-09-10T09:49:41.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:41.435+0000 7f45e19fd740 1 --2- 172.21.15.5:0/2302653693 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x5589cf223000 0x5589cf225420 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:41.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:41.435+0000 7f45e19fd740 1 --2- 172.21.15.5:0/2302653693 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5589cf1a7e50 0x5589cf1abb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:41.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:41.435+0000 7f45e19fd740 1 --2- 172.21.15.5:0/2302653693 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5589cf19d590 0x5589cf1ab5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:41.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:41.435+0000 7f45e19fd740 1 --2- 172.21.15.5:0/2302653693 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5589cf19cc70 0x5589cf1b02a0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:41.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:41.435+0000 7f45e19fd740 1 -- 172.21.15.5:0/2302653693 >> 172.21.15.5:0/2302653693 conn(0x5589cf199200 msgr2=0x5589cf19e6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:49:41.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:41.435+0000 7f45c6000640 0 client.27692 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T09:49:41.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:41.435+0000 7f45e19fd740 1 -- 172.21.15.5:0/2302653693 shutdown_connections 2024-09-10T09:49:41.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:41.435+0000 7f45e19fd740 1 -- 172.21.15.5:0/2302653693 wait complete. 2024-09-10T09:49:41.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:41.436+0000 7f45e19fd740 20 client.27692 trim_cache size 0 max 16384 2024-09-10T09:49:41.441 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-10T09:49:41.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:41.436+0000 7f45df400640 1 -- 172.21.15.5:0/2107361718 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f45bc0a2b80 msgr2=0x7f45bc0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:49:41.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:41.436+0000 7f45df400640 1 --2- 172.21.15.5:0/2107361718 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f45bc0a2b80 0x7f45bc0a5040 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f45d002fde0 tx=0x7f45d002a950 comp rx=0 tx=0).stop 2024-09-10T09:49:41.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:41.436+0000 7f45df400640 1 -- 172.21.15.5:0/2107361718 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f45d8112780 msgr2=0x7f45d819e810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:49:41.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:41.436+0000 7f45df400640 1 --2- 172.21.15.5:0/2107361718 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f45d8112780 0x7f45d819e810 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f45d0004af0 tx=0x7f45d002f3a0 comp rx=0 tx=0).stop 2024-09-10T09:49:41.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:41.436+0000 7f45df400640 1 -- 172.21.15.5:0/2107361718 shutdown_connections 2024-09-10T09:49:41.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:41.436+0000 7f45df400640 1 --2- 172.21.15.5:0/2107361718 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f45bc0a2b80 0x7f45bc0a5040 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:41.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:41.436+0000 7f45df400640 1 --2- 172.21.15.5:0/2107361718 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f45d8112780 0x7f45d819e810 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:41.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:41.436+0000 7f45df400640 1 --2- 172.21.15.5:0/2107361718 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f45d8100e20 0x7f45d81a4bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:41.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:41.436+0000 7f45df400640 1 --2- 172.21.15.5:0/2107361718 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f45d8100500 0x7f45d81a4690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:41.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:41.436+0000 7f45df400640 1 -- 172.21.15.5:0/2107361718 >> 172.21.15.5:0/2107361718 conn(0x7f45d80fc5c0 msgr2=0x7f45d8111160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:49:41.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:41.437+0000 7f45df400640 1 -- 172.21.15.5:0/2107361718 shutdown_connections 2024-09-10T09:49:41.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:41.437+0000 7f45df400640 1 -- 172.21.15.5:0/2107361718 wait complete. 2024-09-10T09:49:41.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:49:41 smithi005 ceph-mon[30350]: pgmap v733: 129 pgs: 129 active+clean; 258 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 1.0 MiB/s wr, 48 op/s 2024-09-10T09:49:41.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:49:41 smithi159 ceph-mon[35513]: pgmap v733: 129 pgs: 129 active+clean; 258 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 1.0 MiB/s wr, 48 op/s 2024-09-10T09:49:41.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:49:41 smithi102 ceph-mon[35442]: pgmap v733: 129 pgs: 129 active+clean; 258 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 1.0 MiB/s wr, 48 op/s 2024-09-10T09:49:42.125 INFO:tasks.workunit.client.0.smithi005.stdout: 1 687246 6.06 MB/sec execute 443 sec latency 25488.611 ms 2024-09-10T09:49:42.406 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:49:43.129 INFO:tasks.workunit.client.0.smithi005.stdout: 1 687246 6.05 MB/sec execute 444 sec latency 26488.690 ms 2024-09-10T09:49:43.250 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-10T09:49:43.250 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-10T09:49:43.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:49:43 smithi005 ceph-mon[30350]: pgmap v734: 129 pgs: 129 active+clean; 258 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 1.1 MiB/s wr, 48 op/s 2024-09-10T09:49:43.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:49:43 smithi159 ceph-mon[35513]: pgmap v734: 129 pgs: 129 active+clean; 258 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 1.1 MiB/s wr, 48 op/s 2024-09-10T09:49:43.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:49:43 smithi102 ceph-mon[35442]: pgmap v734: 129 pgs: 129 active+clean; 258 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 1.1 MiB/s wr, 48 op/s 2024-09-10T09:49:43.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:43.939+0000 7fcbdec00640 1 -- 172.21.15.5:0/813991959 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcbd8074510 msgr2=0x7fcbd80748f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:49:43.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:43.939+0000 7fcbdec00640 1 --2- 172.21.15.5:0/813991959 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcbd8074510 0x7fcbd80748f0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fcbcc009900 tx=0x7fcbcc02a480 comp rx=0 tx=0).stop 2024-09-10T09:49:43.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:43.939+0000 7fcbdec00640 1 -- 172.21.15.5:0/813991959 shutdown_connections 2024-09-10T09:49:43.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:43.939+0000 7fcbdec00640 1 --2- 172.21.15.5:0/813991959 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcbd810e280 0x7fcbd8110670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:43.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:43.939+0000 7fcbdec00640 1 --2- 172.21.15.5:0/813991959 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcbd8074e30 0x7fcbd810dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:43.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:43.939+0000 7fcbdec00640 1 --2- 172.21.15.5:0/813991959 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcbd8074510 0x7fcbd80748f0 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:43.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:43.939+0000 7fcbdec00640 1 -- 172.21.15.5:0/813991959 >> 172.21.15.5:0/813991959 conn(0x7fcbd80fd740 msgr2=0x7fcbd80ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:49:43.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:43.939+0000 7fcbdec00640 1 -- 172.21.15.5:0/813991959 shutdown_connections 2024-09-10T09:49:43.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:43.939+0000 7fcbdec00640 1 -- 172.21.15.5:0/813991959 wait complete. 2024-09-10T09:49:43.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:43.940+0000 7fcbdec00640 1 Processor -- start 2024-09-10T09:49:43.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:43.940+0000 7fcbdec00640 1 -- start start 2024-09-10T09:49:43.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:43.941+0000 7fcbdec00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcbd8074510 0x7fcbd819a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:49:43.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:43.941+0000 7fcbdec00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcbd8074e30 0x7fcbd819a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:49:43.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:43.941+0000 7fcbdec00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcbd810e280 0x7fcbd819ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:49:43.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:43.941+0000 7fcbdec00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fcbd8114000 con 0x7fcbd8074e30 2024-09-10T09:49:43.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:43.941+0000 7fcbdec00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fcbd8113e80 con 0x7fcbd810e280 2024-09-10T09:49:43.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:43.941+0000 7fcbdec00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fcbd8114180 con 0x7fcbd8074510 2024-09-10T09:49:43.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:43.941+0000 7fcbdd800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcbd8074510 0x7fcbd819a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:49:43.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:43.941+0000 7fcbdd800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcbd8074510 0x7fcbd819a4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:56706/0 (socket says 172.21.15.5:56706) 2024-09-10T09:49:43.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:43.941+0000 7fcbdd800640 1 -- 172.21.15.5:0/3530277105 learned_addr learned my addr 172.21.15.5:0/3530277105 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:49:43.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:43.941+0000 7fcbdce00640 1 --2- 172.21.15.5:0/3530277105 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcbd8074e30 0x7fcbd819a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:49:43.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:43.942+0000 7fcbde200640 1 --2- 172.21.15.5:0/3530277105 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcbd810e280 0x7fcbd819ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:49:43.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:43.942+0000 7fcbdd800640 1 -- 172.21.15.5:0/3530277105 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcbd810e280 msgr2=0x7fcbd819ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:49:43.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:43.942+0000 7fcbdd800640 1 --2- 172.21.15.5:0/3530277105 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcbd810e280 0x7fcbd819ed70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:43.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:43.942+0000 7fcbdd800640 1 -- 172.21.15.5:0/3530277105 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcbd8074e30 msgr2=0x7fcbd819a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:49:43.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:43.942+0000 7fcbdd800640 1 --2- 172.21.15.5:0/3530277105 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcbd8074e30 0x7fcbd819a9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:43.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:43.942+0000 7fcbdd800640 1 -- 172.21.15.5:0/3530277105 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcbd819f450 con 0x7fcbd8074510 2024-09-10T09:49:43.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:43.942+0000 7fcbdd800640 1 --2- 172.21.15.5:0/3530277105 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcbd8074510 0x7fcbd819a4a0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fcbcc004770 tx=0x7fcbcc003c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:49:43.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:43.942+0000 7fcbd6000640 1 -- 172.21.15.5:0/3530277105 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbcc03f070 con 0x7fcbd8074510 2024-09-10T09:49:43.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:43.943+0000 7fcbd6000640 1 -- 172.21.15.5:0/3530277105 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcbcc031ca0 con 0x7fcbd8074510 2024-09-10T09:49:43.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:43.943+0000 7fcbdec00640 1 -- 172.21.15.5:0/3530277105 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcbd819f6e0 con 0x7fcbd8074510 2024-09-10T09:49:43.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:43.943+0000 7fcbdec00640 1 -- 172.21.15.5:0/3530277105 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcbd81a7020 con 0x7fcbd8074510 2024-09-10T09:49:43.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:43.943+0000 7fcbd6000640 1 -- 172.21.15.5:0/3530277105 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbcc035030 con 0x7fcbd8074510 2024-09-10T09:49:43.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:43.946+0000 7fcbe2e7d740 1 Processor -- start 2024-09-10T09:49:43.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:43.946+0000 7fcbe2e7d740 1 -- start start 2024-09-10T09:49:43.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:43.946+0000 7fcbd6000640 1 -- 172.21.15.5:0/3530277105 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7fcbcc042070 con 0x7fcbd8074510 2024-09-10T09:49:43.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:43.946+0000 7fcbe2e7d740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56337c8c0d30 0x56337c8c1110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:49:43.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:43.946+0000 7fcbe2e7d740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56337c8c1650 0x56337c8cb9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:49:43.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:43.946+0000 7fcbe2e7d740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56337c8cbf10 0x56337c8ce300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:49:43.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:43.946+0000 7fcbe2e7d740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x56337c81e650 con 0x56337c8c1650 2024-09-10T09:49:43.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:43.946+0000 7fcbe2e7d740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x56337c7c5f10 con 0x56337c8cbf10 2024-09-10T09:49:43.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:43.946+0000 7fcbe2e7d740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x56337c81a610 con 0x56337c8c0d30 2024-09-10T09:49:43.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:43.947+0000 7fcbdce00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56337c8cbf10 0x56337c8ce300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:49:43.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:43.947+0000 7fcbdce00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56337c8cbf10 0x56337c8ce300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:42436/0 (socket says 172.21.15.5:42436) 2024-09-10T09:49:43.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:43.947+0000 7fcbdce00640 1 -- 172.21.15.5:0/550480860 learned_addr learned my addr 172.21.15.5:0/550480860 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:49:43.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:43.947+0000 7fcbdd800640 1 --2- 172.21.15.5:0/550480860 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56337c8c1650 0x56337c8cb9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:49:43.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:43.947+0000 7fcbde200640 1 --2- 172.21.15.5:0/550480860 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56337c8c0d30 0x56337c8c1110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:49:43.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:43.947+0000 7fcbdce00640 1 -- 172.21.15.5:0/550480860 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56337c8c0d30 msgr2=0x56337c8c1110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:49:43.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:43.947+0000 7fcbdce00640 1 --2- 172.21.15.5:0/550480860 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56337c8c0d30 0x56337c8c1110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:43.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:43.947+0000 7fcbdce00640 1 -- 172.21.15.5:0/550480860 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56337c8c1650 msgr2=0x56337c8cb9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:49:43.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:43.947+0000 7fcbdce00640 1 --2- 172.21.15.5:0/550480860 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56337c8c1650 0x56337c8cb9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:43.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:43.947+0000 7fcbdce00640 1 -- 172.21.15.5:0/550480860 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56337c52d580 con 0x56337c8cbf10 2024-09-10T09:49:43.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:43.948+0000 7fcbdce00640 1 --2- 172.21.15.5:0/550480860 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56337c8cbf10 0x56337c8ce300 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fcbc4009c90 tx=0x7fcbc4011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8de318da395f1a19 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:49:43.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:43.948+0000 7fcbcaa00640 1 -- 172.21.15.5:0/550480860 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbc401e070 con 0x56337c8cbf10 2024-09-10T09:49:43.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:43.948+0000 7fcbcaa00640 1 -- 172.21.15.5:0/550480860 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcbc4019b10 con 0x56337c8cbf10 2024-09-10T09:49:43.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:43.948+0000 7fcbd6000640 1 --2- 172.21.15.5:0/3530277105 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fcbc00a2b80 0x7fcbc00a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:49:43.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:43.948+0000 7fcbd6000640 1 -- 172.21.15.5:0/3530277105 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fcbcc0efb30 con 0x7fcbd8074510 2024-09-10T09:49:43.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:43.948+0000 7fcbcaa00640 1 -- 172.21.15.5:0/550480860 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbc400eac0 con 0x56337c8cbf10 2024-09-10T09:49:43.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:43.948+0000 7fcbde200640 1 --2- 172.21.15.5:0/3530277105 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fcbc00a2b80 0x7fcbc00a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:49:43.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:43.949+0000 7fcbe2e7d740 1 -- 172.21.15.5:0/550480860 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56337c8cbf10 msgr2=0x56337c8ce300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:49:43.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:43.949+0000 7fcbe2e7d740 1 --2- 172.21.15.5:0/550480860 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56337c8cbf10 0x56337c8ce300 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fcbc4009c90 tx=0x7fcbc4011810 comp rx=0 tx=0).stop 2024-09-10T09:49:43.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:43.949+0000 7fcbde200640 1 --2- 172.21.15.5:0/3530277105 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fcbc00a2b80 0x7fcbc00a5040 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fcbd80754f0 tx=0x7fcbd0009600 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:49:43.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:43.949+0000 7fcbe2e7d740 1 -- 172.21.15.5:0/550480860 shutdown_connections 2024-09-10T09:49:43.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:43.949+0000 7fcbe2e7d740 1 --2- 172.21.15.5:0/550480860 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56337c8cbf10 0x56337c8ce300 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:43.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:43.949+0000 7fcbe2e7d740 1 --2- 172.21.15.5:0/550480860 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56337c8c1650 0x56337c8cb9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:43.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:43.949+0000 7fcbe2e7d740 1 --2- 172.21.15.5:0/550480860 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56337c8c0d30 0x56337c8c1110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:43.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:43.949+0000 7fcbe2e7d740 1 -- 172.21.15.5:0/550480860 >> 172.21.15.5:0/550480860 conn(0x56337c8bd2c0 msgr2=0x56337c8bd6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:49:43.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:43.949+0000 7fcbe2e7d740 1 -- 172.21.15.5:0/550480860 shutdown_connections 2024-09-10T09:49:43.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:43.950+0000 7fcbe2e7d740 1 -- 172.21.15.5:0/550480860 wait complete. 2024-09-10T09:49:43.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:43.950+0000 7fcbe2e7d740 1 -- start start 2024-09-10T09:49:43.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:43.950+0000 7fcbe2e7d740 1 Processor -- start 2024-09-10T09:49:43.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:43.951+0000 7fcbe2e7d740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56337c8c0d30 0x56337c8c8e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:49:43.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:43.951+0000 7fcbe2e7d740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56337c8c1650 0x56337c8c93d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:49:43.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:43.951+0000 7fcbe2e7d740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56337c8cbf10 0x56337c8c9910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:49:43.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:43.951+0000 7fcbe2e7d740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x56337c8ceae0 con 0x56337c8c0d30 2024-09-10T09:49:43.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:43.951+0000 7fcbe2e7d740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x56337c7d9350 con 0x56337c8c1650 2024-09-10T09:49:43.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:43.951+0000 7fcbe2e7d740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x56337c8cec60 con 0x56337c8cbf10 2024-09-10T09:49:43.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:43.951+0000 7fcbe2e7d740 10 client.0 _ll_register_callbacks cb 0x56337c7fc290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:49:43.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:43.951+0000 7fcbdd800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56337c8c0d30 0x56337c8c8e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:49:43.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:43.951+0000 7fcbdd800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56337c8c0d30 0x56337c8c8e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41124/0 (socket says 172.21.15.5:41124) 2024-09-10T09:49:43.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:43.951+0000 7fcbdd800640 1 -- 172.21.15.5:0/3778776210 learned_addr learned my addr 172.21.15.5:0/3778776210 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:49:43.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:43.951+0000 7fcbdce00640 1 --2- 172.21.15.5:0/3778776210 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56337c8c1650 0x56337c8c93d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:49:43.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:43.952+0000 7fcbde200640 1 --2- 172.21.15.5:0/3778776210 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56337c8cbf10 0x56337c8c9910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:49:43.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:43.952+0000 7fcbdd800640 1 -- 172.21.15.5:0/3778776210 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56337c8cbf10 msgr2=0x56337c8c9910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:49:43.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:43.952+0000 7fcbdd800640 1 --2- 172.21.15.5:0/3778776210 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56337c8cbf10 0x56337c8c9910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:43.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:43.952+0000 7fcbdd800640 1 -- 172.21.15.5:0/3778776210 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56337c8c1650 msgr2=0x56337c8c93d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:49:43.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:43.952+0000 7fcbdd800640 1 --2- 172.21.15.5:0/3778776210 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56337c8c1650 0x56337c8c93d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:43.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:43.952+0000 7fcbdd800640 1 -- 172.21.15.5:0/3778776210 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56337c5f7960 con 0x56337c8c0d30 2024-09-10T09:49:43.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:43.952+0000 7fcbdd800640 1 --2- 172.21.15.5:0/3778776210 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56337c8c0d30 0x56337c8c8e90 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fcbcc0c4680 tx=0x7fcbcc009620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8d16ad8963876d9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:49:43.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:43.952+0000 7fcbbf400640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-10T09:49:43.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:43.953+0000 7fcbbf400640 1 -- 172.21.15.5:0/3778776210 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbcc049070 con 0x56337c8c0d30 2024-09-10T09:49:43.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:43.953+0000 7fcbbf400640 1 -- 172.21.15.5:0/3778776210 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fcbcc0b8660 con 0x56337c8c0d30 2024-09-10T09:49:43.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:43.953+0000 7fcbe2e7d740 1 -- 172.21.15.5:0/3778776210 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcbc4019b10 con 0x56337c8c0d30 2024-09-10T09:49:43.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:43.953+0000 7fcbbf400640 1 -- 172.21.15.5:0/3778776210 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fcbcc0bbb30 con 0x56337c8c0d30 2024-09-10T09:49:43.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:43.953+0000 7fcbe2e7d740 10 client.17685 fetch_fsmap learned FSMap version 36 2024-09-10T09:49:43.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:43.953+0000 7fcbe2e7d740 1 -- 172.21.15.5:0/3778776210 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x56337c8da790 con 0x56337c8c0d30 2024-09-10T09:49:43.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:43.953+0000 7fcbbf400640 1 -- 172.21.15.5:0/3778776210 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcbc4019b10 con 0x56337c8c0d30 2024-09-10T09:49:43.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:43.953+0000 7fcbbf400640 1 -- 172.21.15.5:0/3778776210 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 36) ==== 8370+0+0 (secure 0 0 0) 0x7fcbcc03b4f0 con 0x56337c8c0d30 2024-09-10T09:49:43.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:43.954+0000 7fcbe2e7d740 10 client.17685 fetch_fsmap finished waiting for FSMap version 36 2024-09-10T09:49:43.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:43.954+0000 7fcbe2e7d740 10 client.17685 resolve_mds: validated gid 24542 aka daemon mds.b 2024-09-10T09:49:43.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:43.954+0000 7fcbe2e7d740 20 client.17685 populate_metadata read hostname 'smithi005' 2024-09-10T09:49:43.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:43.954+0000 7fcbe2e7d740 1 --2- 172.21.15.5:0/3778776210 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x56337c8dba70 0x56337c8ddf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:49:43.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:43.954+0000 7fcbe2e7d740 4 client.17685 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:49:43.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:43.954+0000 7fcbe2e7d740 1 -- 172.21.15.5:0/3778776210 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56337c82bbc0 con 0x56337c8dba70 2024-09-10T09:49:43.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:43.954+0000 7fcbdce00640 1 --2- 172.21.15.5:0/3778776210 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x56337c8dba70 0x56337c8ddf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:49:43.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:43.955+0000 7fcbdce00640 1 --2- 172.21.15.5:0/3778776210 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x56337c8dba70 0x56337c8ddf10 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8aa86d5444a3f00d server_cookie=6bae2b4e4083253e in_seq=0 out_seq=0 2024-09-10T09:49:43.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:43.955+0000 7fcbbf400640 10 client.17685 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T09:49:43.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:43.955+0000 7fcbbf400640 1 -- 172.21.15.5:0/3778776210 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56337c82bbc0 con 0x56337c8dba70 2024-09-10T09:49:43.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:43.955+0000 7fcbbf400640 10 client.17685 handle_command_reply: tid=0 2024-09-10T09:49:43.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:43.955+0000 7fcbe2e7d740 1 client.17685 shutdown 2024-09-10T09:49:43.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:43.956+0000 7fcbe2e7d740 1 -- 172.21.15.5:0/3778776210 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56337c8c0d30 msgr2=0x56337c8c8e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:49:43.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:43.956+0000 7fcbe2e7d740 1 --2- 172.21.15.5:0/3778776210 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56337c8c0d30 0x56337c8c8e90 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fcbcc0c4680 tx=0x7fcbcc009620 comp rx=0 tx=0).stop 2024-09-10T09:49:43.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:43.956+0000 7fcbe2e7d740 1 -- 172.21.15.5:0/3778776210 shutdown_connections 2024-09-10T09:49:43.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:43.956+0000 7fcbe2e7d740 1 --2- 172.21.15.5:0/3778776210 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x56337c8dba70 0x56337c8ddf10 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:43.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:43.956+0000 7fcbe2e7d740 1 --2- 172.21.15.5:0/3778776210 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56337c8cbf10 0x56337c8c9910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:43.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:43.956+0000 7fcbe2e7d740 1 --2- 172.21.15.5:0/3778776210 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56337c8c1650 0x56337c8c93d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:43.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:43.956+0000 7fcbe2e7d740 1 --2- 172.21.15.5:0/3778776210 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56337c8c0d30 0x56337c8c8e90 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:43.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:43.956+0000 7fcbe2e7d740 1 -- 172.21.15.5:0/3778776210 >> 172.21.15.5:0/3778776210 conn(0x56337c8bd2c0 msgr2=0x56337c8b3cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:49:43.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:43.956+0000 7fcbbf400640 0 client.17685 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T09:49:43.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:43.956+0000 7fcbe2e7d740 1 -- 172.21.15.5:0/3778776210 shutdown_connections 2024-09-10T09:49:43.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:43.956+0000 7fcbe2e7d740 1 -- 172.21.15.5:0/3778776210 wait complete. 2024-09-10T09:49:43.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:43.956+0000 7fcbe2e7d740 20 client.17685 trim_cache size 0 max 16384 2024-09-10T09:49:43.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:43.991+0000 7fcbe2e7d740 1 Processor -- start 2024-09-10T09:49:43.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:43.991+0000 7fcbe2e7d740 1 -- start start 2024-09-10T09:49:43.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:43.991+0000 7fcbe2e7d740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56337c8c0d30 0x56337c8e3820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:49:43.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:43.991+0000 7fcbe2e7d740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56337c8c1650 0x56337c8e3d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:49:43.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:43.991+0000 7fcbe2e7d740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56337c8cbf10 0x56337c8c60f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:49:43.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:43.991+0000 7fcbe2e7d740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fcbd81acb00 con 0x56337c8cbf10 2024-09-10T09:49:43.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:43.991+0000 7fcbe2e7d740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fcbd81ae640 con 0x56337c8c0d30 2024-09-10T09:49:43.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:43.991+0000 7fcbe2e7d740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fcbd81ad1d0 con 0x56337c8c1650 2024-09-10T09:49:43.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:43.992+0000 7fcbdd800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56337c8c0d30 0x56337c8e3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:49:43.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:43.992+0000 7fcbdce00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56337c8c1650 0x56337c8e3d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:49:43.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:43.992+0000 7fcbdd800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56337c8c0d30 0x56337c8e3820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:42456/0 (socket says 172.21.15.5:42456) 2024-09-10T09:49:43.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:43.992+0000 7fcbdd800640 1 -- 172.21.15.5:0/2895490857 learned_addr learned my addr 172.21.15.5:0/2895490857 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:49:43.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:43.992+0000 7fcbdce00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56337c8c1650 0x56337c8e3d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:56730/0 (socket says 172.21.15.5:56730) 2024-09-10T09:49:43.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:43.992+0000 7fcbde200640 1 --2- 172.21.15.5:0/2895490857 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56337c8cbf10 0x56337c8c60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:49:43.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:43.992+0000 7fcbdd800640 1 -- 172.21.15.5:0/2895490857 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56337c8c1650 msgr2=0x56337c8e3d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:49:43.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:43.992+0000 7fcbdd800640 1 --2- 172.21.15.5:0/2895490857 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56337c8c1650 0x56337c8e3d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:43.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:43.992+0000 7fcbdd800640 1 -- 172.21.15.5:0/2895490857 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56337c8cbf10 msgr2=0x56337c8c60f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:49:43.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:43.992+0000 7fcbdd800640 1 --2- 172.21.15.5:0/2895490857 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56337c8cbf10 0x56337c8c60f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:43.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:43.992+0000 7fcbdd800640 1 -- 172.21.15.5:0/2895490857 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56337c8be450 con 0x56337c8c0d30 2024-09-10T09:49:43.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:43.993+0000 7fcbdd800640 1 --2- 172.21.15.5:0/2895490857 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56337c8c0d30 0x56337c8e3820 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fcbcc03fff0 tx=0x7fcbcc009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a43fb025918ed102 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:49:43.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:43.993+0000 7fcbca000640 1 -- 172.21.15.5:0/2895490857 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbcc049070 con 0x56337c8c0d30 2024-09-10T09:49:43.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:43.993+0000 7fcbca000640 1 -- 172.21.15.5:0/2895490857 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcbcc0c4bb0 con 0x56337c8c0d30 2024-09-10T09:49:43.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:43.993+0000 7fcbca000640 1 -- 172.21.15.5:0/2895490857 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbcc005d50 con 0x56337c8c0d30 2024-09-10T09:49:43.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:43.994+0000 7fcbe2e7d740 1 -- 172.21.15.5:0/2895490857 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56337c8c0d30 msgr2=0x56337c8e3820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:49:43.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:43.994+0000 7fcbe2e7d740 1 --2- 172.21.15.5:0/2895490857 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56337c8c0d30 0x56337c8e3820 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fcbcc03fff0 tx=0x7fcbcc009260 comp rx=0 tx=0).stop 2024-09-10T09:49:43.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:43.994+0000 7fcbe2e7d740 1 -- 172.21.15.5:0/2895490857 shutdown_connections 2024-09-10T09:49:43.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:43.994+0000 7fcbe2e7d740 1 --2- 172.21.15.5:0/2895490857 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56337c8cbf10 0x56337c8c60f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:43.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:43.994+0000 7fcbe2e7d740 1 --2- 172.21.15.5:0/2895490857 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56337c8c1650 0x56337c8e3d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:43.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:43.994+0000 7fcbe2e7d740 1 --2- 172.21.15.5:0/2895490857 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56337c8c0d30 0x56337c8e3820 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:43.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:43.994+0000 7fcbe2e7d740 1 -- 172.21.15.5:0/2895490857 >> 172.21.15.5:0/2895490857 conn(0x56337c8bd2c0 msgr2=0x56337c8b3cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:49:43.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:43.995+0000 7fcbe2e7d740 1 -- 172.21.15.5:0/2895490857 shutdown_connections 2024-09-10T09:49:43.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:43.995+0000 7fcbe2e7d740 1 -- 172.21.15.5:0/2895490857 wait complete. 2024-09-10T09:49:43.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:43.995+0000 7fcbe2e7d740 1 -- start start 2024-09-10T09:49:43.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:43.995+0000 7fcbe2e7d740 1 Processor -- start 2024-09-10T09:49:43.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:43.996+0000 7fcbe2e7d740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56337c8c0d30 0x56337c8cfee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:49:43.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:43.996+0000 7fcbe2e7d740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56337c8c1650 0x56337c8d0420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:49:43.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:43.996+0000 7fcbe2e7d740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56337c8cbf10 0x56337c8d0980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:49:43.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:43.996+0000 7fcbe2e7d740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fcbd81adf70 con 0x56337c8c0d30 2024-09-10T09:49:43.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:43.996+0000 7fcbe2e7d740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fcbd81ad8a0 con 0x56337c8c1650 2024-09-10T09:49:43.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:43.996+0000 7fcbe2e7d740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fcbd81aaae0 con 0x56337c8cbf10 2024-09-10T09:49:43.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:43.996+0000 7fcbe2e7d740 10 client.0 _ll_register_callbacks cb 0x7fcbcc0f3260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:49:43.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:43.997+0000 7fcbdce00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56337c8c1650 0x56337c8d0420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:49:43.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:43.997+0000 7fcbdce00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56337c8c1650 0x56337c8d0420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:42472/0 (socket says 172.21.15.5:42472) 2024-09-10T09:49:44.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:43.997+0000 7fcbdce00640 1 -- 172.21.15.5:0/3970684419 learned_addr learned my addr 172.21.15.5:0/3970684419 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:49:44.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:43.997+0000 7fcbdd800640 1 --2- 172.21.15.5:0/3970684419 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56337c8c0d30 0x56337c8cfee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:49:44.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:43.997+0000 7fcbde200640 1 --2- 172.21.15.5:0/3970684419 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56337c8cbf10 0x56337c8d0980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:49:44.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:43.997+0000 7fcbdce00640 1 -- 172.21.15.5:0/3970684419 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56337c8cbf10 msgr2=0x56337c8d0980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:49:44.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:43.997+0000 7fcbdce00640 1 --2- 172.21.15.5:0/3970684419 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56337c8cbf10 0x56337c8d0980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:44.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:43.997+0000 7fcbdce00640 1 -- 172.21.15.5:0/3970684419 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56337c8c0d30 msgr2=0x56337c8cfee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:49:44.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:43.997+0000 7fcbdce00640 1 --2- 172.21.15.5:0/3970684419 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56337c8c0d30 0x56337c8cfee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:44.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:43.997+0000 7fcbdce00640 1 -- 172.21.15.5:0/3970684419 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56337c945ac0 con 0x56337c8c1650 2024-09-10T09:49:44.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:43.997+0000 7fcbdd800640 1 --2- 172.21.15.5:0/3970684419 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56337c8c0d30 0x56337c8cfee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:49:44.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:43.998+0000 7fcbdce00640 1 --2- 172.21.15.5:0/3970684419 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56337c8c1650 0x56337c8d0420 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fcbc4005fd0 tx=0x7fcbc400e5a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4366c6cd9371eba3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:49:44.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:43.998+0000 7fcbbf400640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-09-10T09:49:44.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:43.998+0000 7fcbbf400640 1 -- 172.21.15.5:0/3970684419 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbc401e040 con 0x56337c8c1650 2024-09-10T09:49:44.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:43.998+0000 7fcbbf400640 1 -- 172.21.15.5:0/3970684419 <== mon.2 v2:172.21.15.102:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fcbc4005070 con 0x56337c8c1650 2024-09-10T09:49:44.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:43.998+0000 7fcbe2e7d740 1 -- 172.21.15.5:0/3970684419 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcbcc0c4bb0 con 0x56337c8c1650 2024-09-10T09:49:44.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:43.998+0000 7fcbbf400640 1 -- 172.21.15.5:0/3970684419 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fcbc4039050 con 0x56337c8c1650 2024-09-10T09:49:44.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:43.998+0000 7fcbe2e7d740 10 client.27710 fetch_fsmap learned FSMap version 36 2024-09-10T09:49:44.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:43.998+0000 7fcbe2e7d740 1 -- 172.21.15.5:0/3970684419 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x56337c946e00 con 0x56337c8c1650 2024-09-10T09:49:44.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:43.999+0000 7fcbbf400640 1 -- 172.21.15.5:0/3970684419 <== mon.2 v2:172.21.15.102:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcbcc0c4bb0 con 0x56337c8c1650 2024-09-10T09:49:44.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:43.999+0000 7fcbbf400640 1 -- 172.21.15.5:0/3970684419 <== mon.2 v2:172.21.15.102:3300/0 5 ==== fsmap(e 36) ==== 8370+0+0 (secure 0 0 0) 0x7fcbc4019990 con 0x56337c8c1650 2024-09-10T09:49:44.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:43.999+0000 7fcbe2e7d740 10 client.27710 fetch_fsmap finished waiting for FSMap version 36 2024-09-10T09:49:44.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:43.999+0000 7fcbe2e7d740 10 client.27710 resolve_mds: validated gid 24542 aka daemon mds.b 2024-09-10T09:49:44.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:43.999+0000 7fcbe2e7d740 20 client.27710 populate_metadata read hostname 'smithi005' 2024-09-10T09:49:44.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:43.999+0000 7fcbe2e7d740 1 --2- 172.21.15.5:0/3970684419 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x56337c9473f0 0x56337c949810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:49:44.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:43.999+0000 7fcbe2e7d740 4 client.27710 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "a33fa5fe", "release": true, "await": true}] 2024-09-10T09:49:44.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:43.999+0000 7fcbe2e7d740 1 -- 172.21.15.5:0/3970684419 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "quiesce db", "set_id": "a33fa5fe", "release": true, "await": true}) -- 0x56337c8260d0 con 0x56337c9473f0 2024-09-10T09:49:44.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:43.999+0000 7fcbdd800640 1 --2- 172.21.15.5:0/3970684419 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x56337c9473f0 0x56337c949810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:49:44.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:44.000+0000 7fcbdd800640 1 --2- 172.21.15.5:0/3970684419 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x56337c9473f0 0x56337c949810 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c4b840fea6885f57 server_cookie=83a5721fb73b6e2f in_seq=0 out_seq=0 2024-09-10T09:49:44.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:44.000+0000 7fcbbf400640 10 client.27710 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T09:49:44.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:44.010+0000 7fcbbf400640 1 -- 172.21.15.5:0/3970684419 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x56337c8260d0 con 0x56337c9473f0 2024-09-10T09:49:44.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:44.010+0000 7fcbbf400640 10 client.27710 handle_command_reply: tid=0 2024-09-10T09:49:44.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:44.010+0000 7fcbe2e7d740 1 client.27710 shutdown 2024-09-10T09:49:44.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:44.010+0000 7fcbe2e7d740 1 -- 172.21.15.5:0/3970684419 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56337c8c1650 msgr2=0x56337c8d0420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:49:44.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:44.010+0000 7fcbe2e7d740 1 --2- 172.21.15.5:0/3970684419 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56337c8c1650 0x56337c8d0420 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fcbc4005fd0 tx=0x7fcbc400e5a0 comp rx=0 tx=0).stop 2024-09-10T09:49:44.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:44.010+0000 7fcbe2e7d740 1 -- 172.21.15.5:0/3970684419 shutdown_connections 2024-09-10T09:49:44.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:44.010+0000 7fcbe2e7d740 1 --2- 172.21.15.5:0/3970684419 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x56337c9473f0 0x56337c949810 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:44.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:44.010+0000 7fcbe2e7d740 1 --2- 172.21.15.5:0/3970684419 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56337c8cbf10 0x56337c8d0980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:44.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:44.010+0000 7fcbbf400640 0 client.27710 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T09:49:44.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:44.010+0000 7fcbe2e7d740 1 --2- 172.21.15.5:0/3970684419 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56337c8c1650 0x56337c8d0420 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:44.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:44.010+0000 7fcbe2e7d740 1 --2- 172.21.15.5:0/3970684419 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56337c8c0d30 0x56337c8cfee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:44.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:44.010+0000 7fcbe2e7d740 1 -- 172.21.15.5:0/3970684419 >> 172.21.15.5:0/3970684419 conn(0x56337c8bd2c0 msgr2=0x56337c8b3cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:49:44.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:44.010+0000 7fcbe2e7d740 1 -- 172.21.15.5:0/3970684419 shutdown_connections 2024-09-10T09:49:44.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:44.010+0000 7fcbe2e7d740 1 -- 172.21.15.5:0/3970684419 wait complete. 2024-09-10T09:49:44.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:44.010+0000 7fcbe2e7d740 20 client.27710 trim_cache size 0 max 16384 2024-09-10T09:49:44.014 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-10T09:49:44.014 INFO:teuthology.orchestra.run.smithi005.stdout: "epoch": 36, 2024-09-10T09:49:44.014 INFO:teuthology.orchestra.run.smithi005.stdout: "leader": 24542, 2024-09-10T09:49:44.014 INFO:teuthology.orchestra.run.smithi005.stdout: "set_version": 40, 2024-09-10T09:49:44.014 INFO:teuthology.orchestra.run.smithi005.stdout: "sets": { 2024-09-10T09:49:44.014 INFO:teuthology.orchestra.run.smithi005.stdout: "a33fa5fe": { 2024-09-10T09:49:44.014 INFO:teuthology.orchestra.run.smithi005.stdout: "version": 40, 2024-09-10T09:49:44.014 INFO:teuthology.orchestra.run.smithi005.stdout: "age_ref": 0.0, 2024-09-10T09:49:44.014 INFO:teuthology.orchestra.run.smithi005.stdout: "state": { 2024-09-10T09:49:44.015 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "RELEASED", 2024-09-10T09:49:44.015 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0 2024-09-10T09:49:44.015 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T09:49:44.015 INFO:teuthology.orchestra.run.smithi005.stdout: "timeout": 90.0, 2024-09-10T09:49:44.015 INFO:teuthology.orchestra.run.smithi005.stdout: "expiration": 145.2, 2024-09-10T09:49:44.015 INFO:teuthology.orchestra.run.smithi005.stdout: "members": { 2024-09-10T09:49:44.015 INFO:teuthology.orchestra.run.smithi005.stdout: "file:/": { 2024-09-10T09:49:44.015 INFO:teuthology.orchestra.run.smithi005.stdout: "excluded": false, 2024-09-10T09:49:44.015 INFO:teuthology.orchestra.run.smithi005.stdout: "state": { 2024-09-10T09:49:44.015 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "RELEASED", 2024-09-10T09:49:44.015 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0 2024-09-10T09:49:44.015 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T09:49:44.015 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T09:49:44.015 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T09:49:44.016 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T09:49:44.016 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T09:49:44.016 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-10T09:49:44.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:44.011+0000 7fcbdec00640 1 -- 172.21.15.5:0/3530277105 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fcbc00a2b80 msgr2=0x7fcbc00a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:49:44.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:44.011+0000 7fcbdec00640 1 --2- 172.21.15.5:0/3530277105 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fcbc00a2b80 0x7fcbc00a5040 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fcbd80754f0 tx=0x7fcbd0009600 comp rx=0 tx=0).stop 2024-09-10T09:49:44.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:44.011+0000 7fcbdec00640 1 -- 172.21.15.5:0/3530277105 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcbd8074510 msgr2=0x7fcbd819a4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:49:44.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:44.011+0000 7fcbdec00640 1 --2- 172.21.15.5:0/3530277105 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcbd8074510 0x7fcbd819a4a0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fcbcc004770 tx=0x7fcbcc003c10 comp rx=0 tx=0).stop 2024-09-10T09:49:44.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:44.011+0000 7fcbdec00640 1 -- 172.21.15.5:0/3530277105 shutdown_connections 2024-09-10T09:49:44.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:44.011+0000 7fcbdec00640 1 --2- 172.21.15.5:0/3530277105 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fcbc00a2b80 0x7fcbc00a5040 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:44.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:44.011+0000 7fcbdec00640 1 --2- 172.21.15.5:0/3530277105 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcbd810e280 0x7fcbd819ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:44.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:44.011+0000 7fcbdec00640 1 --2- 172.21.15.5:0/3530277105 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcbd8074e30 0x7fcbd819a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:44.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:44.011+0000 7fcbdec00640 1 --2- 172.21.15.5:0/3530277105 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcbd8074510 0x7fcbd819a4a0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:44.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:44.011+0000 7fcbdec00640 1 -- 172.21.15.5:0/3530277105 >> 172.21.15.5:0/3530277105 conn(0x7fcbd80fd740 msgr2=0x7fcbd80fee50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:49:44.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:44.011+0000 7fcbdec00640 1 -- 172.21.15.5:0/3530277105 shutdown_connections 2024-09-10T09:49:44.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:44.012+0000 7fcbdec00640 1 -- 172.21.15.5:0/3530277105 wait complete. 2024-09-10T09:49:44.017 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:49:44.125 INFO:tasks.workunit.client.0.smithi005.stdout: 1 687464 6.03 MB/sec execute 445 sec latency 27387.181 ms 2024-09-10T09:49:45.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:45.025+0000 7fe8f0600640 1 -- 172.21.15.5:0/1726034726 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe8e8100500 msgr2=0x7fe8e810ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:49:45.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:45.025+0000 7fe8f0600640 1 --2- 172.21.15.5:0/1726034726 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe8e8100500 0x7fe8e810ea20 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fe8e0009cd0 tx=0x7fe8e002a8e0 comp rx=0 tx=0).stop 2024-09-10T09:49:45.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:45.025+0000 7fe8f0600640 1 -- 172.21.15.5:0/1726034726 shutdown_connections 2024-09-10T09:49:45.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:45.025+0000 7fe8f0600640 1 --2- 172.21.15.5:0/1726034726 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe8e8100500 0x7fe8e810ea20 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:45.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:45.025+0000 7fe8f0600640 1 --2- 172.21.15.5:0/1726034726 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe8e80ffb60 0x7fe8e80fffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:45.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:45.025+0000 7fe8f0600640 1 --2- 172.21.15.5:0/1726034726 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe8e8106b80 0x7fe8e8106f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:45.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:45.025+0000 7fe8f0600640 1 -- 172.21.15.5:0/1726034726 >> 172.21.15.5:0/1726034726 conn(0x7fe8e80fba90 msgr2=0x7fe8e80fdeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:49:45.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:45.026+0000 7fe8f0600640 1 -- 172.21.15.5:0/1726034726 shutdown_connections 2024-09-10T09:49:45.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:45.026+0000 7fe8f0600640 1 -- 172.21.15.5:0/1726034726 wait complete. 2024-09-10T09:49:45.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:45.026+0000 7fe8f0600640 1 Processor -- start 2024-09-10T09:49:45.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:45.027+0000 7fe8f0600640 1 -- start start 2024-09-10T09:49:45.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:45.027+0000 7fe8f0600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe8e80ffb60 0x7fe8e81983e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:49:45.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:45.027+0000 7fe8f0600640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe8e8100500 0x7fe8e8198920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:49:45.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:45.027+0000 7fe8f0600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe8e8106b80 0x7fe8e819ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:49:45.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:45.027+0000 7fe8f0600640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe8e8111fa0 con 0x7fe8e80ffb60 2024-09-10T09:49:45.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:45.027+0000 7fe8f0600640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fe8e8111e20 con 0x7fe8e8106b80 2024-09-10T09:49:45.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:45.027+0000 7fe8f0600640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fe8e8112120 con 0x7fe8e8100500 2024-09-10T09:49:45.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:45.027+0000 7fe8ed200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe8e8100500 0x7fe8e8198920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:49:45.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:45.027+0000 7fe8ed200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe8e8100500 0x7fe8e8198920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:56758/0 (socket says 172.21.15.5:56758) 2024-09-10T09:49:45.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:45.027+0000 7fe8ed200640 1 -- 172.21.15.5:0/1461515933 learned_addr learned my addr 172.21.15.5:0/1461515933 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:49:45.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:45.027+0000 7fe8edc00640 1 --2- 172.21.15.5:0/1461515933 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe8e80ffb60 0x7fe8e81983e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:49:45.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:45.027+0000 7fe8ee600640 1 --2- 172.21.15.5:0/1461515933 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe8e8106b80 0x7fe8e819ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:49:45.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:45.028+0000 7fe8ed200640 1 -- 172.21.15.5:0/1461515933 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe8e8106b80 msgr2=0x7fe8e819ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:49:45.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:45.028+0000 7fe8ed200640 1 --2- 172.21.15.5:0/1461515933 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe8e8106b80 0x7fe8e819ccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:45.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:45.028+0000 7fe8ed200640 1 -- 172.21.15.5:0/1461515933 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe8e80ffb60 msgr2=0x7fe8e81983e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:49:45.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:45.028+0000 7fe8ed200640 1 --2- 172.21.15.5:0/1461515933 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe8e80ffb60 0x7fe8e81983e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:45.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:45.028+0000 7fe8ed200640 1 -- 172.21.15.5:0/1461515933 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8e819d390 con 0x7fe8e8100500 2024-09-10T09:49:45.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:45.028+0000 7fe8ed200640 1 --2- 172.21.15.5:0/1461515933 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe8e8100500 0x7fe8e8198920 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fe8d400e9f0 tx=0x7fe8d400edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:49:45.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:45.028+0000 7fe8e6000640 1 -- 172.21.15.5:0/1461515933 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8d400cd80 con 0x7fe8e8100500 2024-09-10T09:49:45.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:45.028+0000 7fe8e6000640 1 -- 172.21.15.5:0/1461515933 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe8d4005500 con 0x7fe8e8100500 2024-09-10T09:49:45.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:45.028+0000 7fe8e6000640 1 -- 172.21.15.5:0/1461515933 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8d4010690 con 0x7fe8e8100500 2024-09-10T09:49:45.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:45.028+0000 7fe8f0600640 1 -- 172.21.15.5:0/1461515933 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8e819d680 con 0x7fe8e8100500 2024-09-10T09:49:45.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:45.029+0000 7fe8f0600640 1 -- 172.21.15.5:0/1461515933 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8e81a4f60 con 0x7fe8e8100500 2024-09-10T09:49:45.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:45.032+0000 7fe8f3089740 1 Processor -- start 2024-09-10T09:49:45.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:45.033+0000 7fe8f3089740 1 -- start start 2024-09-10T09:49:45.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:45.033+0000 7fe8e6000640 1 -- 172.21.15.5:0/1461515933 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7fe8d40046c0 con 0x7fe8e8100500 2024-09-10T09:49:45.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:45.033+0000 7fe8f3089740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559c6e927ac0 0x559c6e927ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:49:45.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:45.033+0000 7fe8f3089740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559c6e9283e0 0x559c6e932760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:49:45.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:45.033+0000 7fe8f3089740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559c6e932ca0 0x559c6e935090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:49:45.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:45.033+0000 7fe8f3089740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x559c6e88e290 con 0x559c6e932ca0 2024-09-10T09:49:45.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:45.033+0000 7fe8f3089740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x559c6e835b40 con 0x559c6e927ac0 2024-09-10T09:49:45.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:45.033+0000 7fe8f3089740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x559c6e88a250 con 0x559c6e9283e0 2024-09-10T09:49:45.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:45.034+0000 7fe8edc00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559c6e9283e0 0x559c6e932760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:49:45.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:45.034+0000 7fe8edc00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559c6e9283e0 0x559c6e932760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:56774/0 (socket says 172.21.15.5:56774) 2024-09-10T09:49:45.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:45.034+0000 7fe8edc00640 1 -- 172.21.15.5:0/3895522975 learned_addr learned my addr 172.21.15.5:0/3895522975 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:49:45.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:45.034+0000 7fe8ee600640 1 --2- 172.21.15.5:0/3895522975 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559c6e927ac0 0x559c6e927ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:49:45.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:45.034+0000 7fe8ed200640 1 --2- 172.21.15.5:0/3895522975 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559c6e932ca0 0x559c6e935090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:49:45.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:45.034+0000 7fe8ee600640 1 -- 172.21.15.5:0/3895522975 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559c6e9283e0 msgr2=0x559c6e932760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:49:45.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:45.034+0000 7fe8ee600640 1 --2- 172.21.15.5:0/3895522975 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559c6e9283e0 0x559c6e932760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:45.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:45.034+0000 7fe8ee600640 1 -- 172.21.15.5:0/3895522975 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559c6e932ca0 msgr2=0x559c6e935090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:49:45.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:45.034+0000 7fe8ee600640 1 --2- 172.21.15.5:0/3895522975 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559c6e932ca0 0x559c6e935090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:45.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:45.034+0000 7fe8ee600640 1 -- 172.21.15.5:0/3895522975 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559c6e5ff6a0 con 0x559c6e927ac0 2024-09-10T09:49:45.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:45.034+0000 7fe8edc00640 1 --2- 172.21.15.5:0/3895522975 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559c6e9283e0 0x559c6e932760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:49:45.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:45.034+0000 7fe8ee600640 1 --2- 172.21.15.5:0/3895522975 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559c6e927ac0 0x559c6e927ea0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fe8e0005e30 tx=0x7fe8e0009a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bc7de821ea314e5d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:49:45.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:45.035+0000 7fe8daa00640 1 -- 172.21.15.5:0/3895522975 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8e0048070 con 0x559c6e927ac0 2024-09-10T09:49:45.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:45.035+0000 7fe8daa00640 1 -- 172.21.15.5:0/3895522975 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe8e0036070 con 0x559c6e927ac0 2024-09-10T09:49:45.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:45.035+0000 7fe8daa00640 1 -- 172.21.15.5:0/3895522975 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8e0004280 con 0x559c6e927ac0 2024-09-10T09:49:45.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:45.035+0000 7fe8e6000640 1 --2- 172.21.15.5:0/1461515933 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fe8d00a2b80 0x7fe8d00a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:49:45.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:45.035+0000 7fe8e6000640 1 -- 172.21.15.5:0/1461515933 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fe8d40cf810 con 0x7fe8e8100500 2024-09-10T09:49:45.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:45.035+0000 7fe8edc00640 1 --2- 172.21.15.5:0/1461515933 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fe8d00a2b80 0x7fe8d00a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:49:45.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:45.035+0000 7fe8f3089740 1 -- 172.21.15.5:0/3895522975 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559c6e927ac0 msgr2=0x559c6e927ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:49:45.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:45.035+0000 7fe8f3089740 1 --2- 172.21.15.5:0/3895522975 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559c6e927ac0 0x559c6e927ea0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fe8e0005e30 tx=0x7fe8e0009a20 comp rx=0 tx=0).stop 2024-09-10T09:49:45.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:45.036+0000 7fe8f3089740 1 -- 172.21.15.5:0/3895522975 shutdown_connections 2024-09-10T09:49:45.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:45.036+0000 7fe8f3089740 1 --2- 172.21.15.5:0/3895522975 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559c6e932ca0 0x559c6e935090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:45.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:45.036+0000 7fe8f3089740 1 --2- 172.21.15.5:0/3895522975 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559c6e9283e0 0x559c6e932760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:45.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:45.036+0000 7fe8f3089740 1 --2- 172.21.15.5:0/3895522975 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559c6e927ac0 0x559c6e927ea0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:45.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:45.036+0000 7fe8f3089740 1 -- 172.21.15.5:0/3895522975 >> 172.21.15.5:0/3895522975 conn(0x559c6e924050 msgr2=0x559c6e924430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:49:45.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:45.036+0000 7fe8f3089740 1 -- 172.21.15.5:0/3895522975 shutdown_connections 2024-09-10T09:49:45.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:45.036+0000 7fe8f3089740 1 -- 172.21.15.5:0/3895522975 wait complete. 2024-09-10T09:49:45.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:45.036+0000 7fe8edc00640 1 --2- 172.21.15.5:0/1461515933 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fe8d00a2b80 0x7fe8d00a5040 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x559c6e924eb0 tx=0x7fe8dc005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:49:45.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:45.036+0000 7fe8f3089740 1 -- start start 2024-09-10T09:49:45.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:45.036+0000 7fe8f3089740 1 Processor -- start 2024-09-10T09:49:45.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:45.037+0000 7fe8f3089740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559c6e927ac0 0x559c6e932680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:49:45.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:45.037+0000 7fe8f3089740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559c6e9283e0 0x559c6e92f930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:49:45.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:45.037+0000 7fe8f3089740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559c6e932ca0 0x559c6e92ff00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:49:45.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:45.037+0000 7fe8f3089740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x559c6e935910 con 0x559c6e9283e0 2024-09-10T09:49:45.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:45.037+0000 7fe8f3089740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x559c6e848fa0 con 0x559c6e932ca0 2024-09-10T09:49:45.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:45.037+0000 7fe8f3089740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x559c6e935a90 con 0x559c6e927ac0 2024-09-10T09:49:45.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:45.037+0000 7fe8f3089740 10 client.0 _ll_register_callbacks cb 0x559c6e59cba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:49:45.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:45.037+0000 7fe8edc00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559c6e927ac0 0x559c6e932680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:49:45.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:45.037+0000 7fe8ee600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559c6e932ca0 0x559c6e92ff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:49:45.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:45.037+0000 7fe8ed200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559c6e9283e0 0x559c6e92f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:49:45.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:45.037+0000 7fe8ee600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559c6e932ca0 0x559c6e92ff00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:42504/0 (socket says 172.21.15.5:42504) 2024-09-10T09:49:45.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:45.037+0000 7fe8ee600640 1 -- 172.21.15.5:0/759762789 learned_addr learned my addr 172.21.15.5:0/759762789 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:49:45.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:45.038+0000 7fe8ee600640 1 -- 172.21.15.5:0/759762789 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559c6e927ac0 msgr2=0x559c6e932680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:49:45.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:45.038+0000 7fe8ee600640 1 --2- 172.21.15.5:0/759762789 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559c6e927ac0 0x559c6e932680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:45.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:45.038+0000 7fe8ee600640 1 -- 172.21.15.5:0/759762789 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559c6e9283e0 msgr2=0x559c6e92f930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:49:45.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:45.038+0000 7fe8ee600640 1 --2- 172.21.15.5:0/759762789 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559c6e9283e0 0x559c6e92f930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:45.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:45.038+0000 7fe8ee600640 1 -- 172.21.15.5:0/759762789 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559c6e930730 con 0x559c6e932ca0 2024-09-10T09:49:45.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:45.038+0000 7fe8ee600640 1 --2- 172.21.15.5:0/759762789 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559c6e932ca0 0x559c6e92ff00 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fe8e002ae00 tx=0x7fe8e0003c20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bc375c86bc5ca247 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:49:45.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:45.038+0000 7fe8cf400640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-09-10T09:49:45.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:45.038+0000 7fe8cf400640 1 -- 172.21.15.5:0/759762789 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8e0048040 con 0x559c6e932ca0 2024-09-10T09:49:45.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:45.038+0000 7fe8f3089740 1 -- 172.21.15.5:0/759762789 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe8e0036070 con 0x559c6e932ca0 2024-09-10T09:49:45.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:45.038+0000 7fe8cf400640 1 -- 172.21.15.5:0/759762789 <== mon.2 v2:172.21.15.102:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fe8e0031ac0 con 0x559c6e932ca0 2024-09-10T09:49:45.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:45.038+0000 7fe8cf400640 1 -- 172.21.15.5:0/759762789 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fe8e005c050 con 0x559c6e932ca0 2024-09-10T09:49:45.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:45.038+0000 7fe8f3089740 10 client.27722 fetch_fsmap learned FSMap version 36 2024-09-10T09:49:45.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:45.038+0000 7fe8f3089740 1 -- 172.21.15.5:0/759762789 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x559c6e935910 con 0x559c6e932ca0 2024-09-10T09:49:45.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:45.039+0000 7fe8cf400640 1 -- 172.21.15.5:0/759762789 <== mon.2 v2:172.21.15.102:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe8e0036070 con 0x559c6e932ca0 2024-09-10T09:49:45.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:45.039+0000 7fe8cf400640 1 -- 172.21.15.5:0/759762789 <== mon.2 v2:172.21.15.102:3300/0 5 ==== fsmap(e 36) ==== 8370+0+0 (secure 0 0 0) 0x7fe8e004b030 con 0x559c6e932ca0 2024-09-10T09:49:45.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:45.039+0000 7fe8f3089740 10 client.27722 fetch_fsmap finished waiting for FSMap version 36 2024-09-10T09:49:45.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:45.039+0000 7fe8f3089740 10 client.27722 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T09:49:45.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:45.039+0000 7fe8f3089740 20 client.27722 populate_metadata read hostname 'smithi005' 2024-09-10T09:49:45.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:45.039+0000 7fe8f3089740 1 --2- 172.21.15.5:0/759762789 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x559c6e946e50 0x559c6e949240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:49:45.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:45.039+0000 7fe8f3089740 4 client.27722 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:49:45.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:45.039+0000 7fe8f3089740 1 -- 172.21.15.5:0/759762789 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559c6e89b800 con 0x559c6e946e50 2024-09-10T09:49:45.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:45.040+0000 7fe8edc00640 1 --2- 172.21.15.5:0/759762789 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x559c6e946e50 0x559c6e949240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:49:45.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:45.040+0000 7fe8edc00640 1 --2- 172.21.15.5:0/759762789 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x559c6e946e50 0x559c6e949240 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ef00247ba8b9857d server_cookie=5a6b419205900d8c in_seq=0 out_seq=0 2024-09-10T09:49:45.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:45.040+0000 7fe8cf400640 10 client.27722 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T09:49:45.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:45.041+0000 7fe8cf400640 1 -- 172.21.15.5:0/759762789 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559c6e89b800 con 0x559c6e946e50 2024-09-10T09:49:45.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:45.041+0000 7fe8cf400640 10 client.27722 handle_command_reply: tid=0 2024-09-10T09:49:45.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:45.041+0000 7fe8f3089740 1 client.27722 shutdown 2024-09-10T09:49:45.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:45.041+0000 7fe8f3089740 1 -- 172.21.15.5:0/759762789 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559c6e932ca0 msgr2=0x559c6e92ff00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:49:45.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:45.041+0000 7fe8f3089740 1 --2- 172.21.15.5:0/759762789 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559c6e932ca0 0x559c6e92ff00 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fe8e002ae00 tx=0x7fe8e0003c20 comp rx=0 tx=0).stop 2024-09-10T09:49:45.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:45.041+0000 7fe8f3089740 1 -- 172.21.15.5:0/759762789 shutdown_connections 2024-09-10T09:49:45.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:45.041+0000 7fe8f3089740 1 --2- 172.21.15.5:0/759762789 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x559c6e946e50 0x559c6e949240 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:45.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:45.041+0000 7fe8f3089740 1 --2- 172.21.15.5:0/759762789 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559c6e932ca0 0x559c6e92ff00 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:45.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:45.041+0000 7fe8f3089740 1 --2- 172.21.15.5:0/759762789 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559c6e9283e0 0x559c6e92f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:45.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:45.041+0000 7fe8f3089740 1 --2- 172.21.15.5:0/759762789 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559c6e927ac0 0x559c6e932680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:45.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:45.041+0000 7fe8cf400640 0 client.27722 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T09:49:45.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:45.042+0000 7fe8f3089740 1 -- 172.21.15.5:0/759762789 >> 172.21.15.5:0/759762789 conn(0x559c6e924050 msgr2=0x559c6e9231c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:49:45.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:45.042+0000 7fe8f3089740 1 -- 172.21.15.5:0/759762789 shutdown_connections 2024-09-10T09:49:45.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:45.042+0000 7fe8f3089740 1 -- 172.21.15.5:0/759762789 wait complete. 2024-09-10T09:49:45.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:45.042+0000 7fe8f3089740 20 client.27722 trim_cache size 0 max 16384 2024-09-10T09:49:45.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:45.061+0000 7fe8f3089740 1 Processor -- start 2024-09-10T09:49:45.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:45.061+0000 7fe8f3089740 1 -- start start 2024-09-10T09:49:45.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:45.061+0000 7fe8f3089740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559c6e927ac0 0x559c6e92ada0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:49:45.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:45.061+0000 7fe8f3089740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559c6e9283e0 0x559c6e94e930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:49:45.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:45.062+0000 7fe8f3089740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559c6e932ca0 0x559c6e94ee70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:49:45.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:45.062+0000 7fe8f3089740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe8e81acab0 con 0x559c6e927ac0 2024-09-10T09:49:45.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:45.062+0000 7fe8f3089740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fe8e81ae5f0 con 0x559c6e932ca0 2024-09-10T09:49:45.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:45.062+0000 7fe8f3089740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fe8e81ad180 con 0x559c6e9283e0 2024-09-10T09:49:45.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:45.062+0000 7fe8ed200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559c6e9283e0 0x559c6e94e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:49:45.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:45.062+0000 7fe8ed200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559c6e9283e0 0x559c6e94e930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:56778/0 (socket says 172.21.15.5:56778) 2024-09-10T09:49:45.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:45.062+0000 7fe8ed200640 1 -- 172.21.15.5:0/849689572 learned_addr learned my addr 172.21.15.5:0/849689572 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:49:45.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:45.062+0000 7fe8ee600640 1 --2- 172.21.15.5:0/849689572 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559c6e932ca0 0x559c6e94ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:49:45.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:45.062+0000 7fe8edc00640 1 --2- 172.21.15.5:0/849689572 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559c6e927ac0 0x559c6e92ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:49:45.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:45.062+0000 7fe8ed200640 1 -- 172.21.15.5:0/849689572 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559c6e932ca0 msgr2=0x559c6e94ee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:49:45.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:45.062+0000 7fe8ed200640 1 --2- 172.21.15.5:0/849689572 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559c6e932ca0 0x559c6e94ee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:45.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:45.062+0000 7fe8ed200640 1 -- 172.21.15.5:0/849689572 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559c6e927ac0 msgr2=0x559c6e92ada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:49:45.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:45.062+0000 7fe8ed200640 1 --2- 172.21.15.5:0/849689572 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559c6e927ac0 0x559c6e92ada0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:45.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:45.062+0000 7fe8ed200640 1 -- 172.21.15.5:0/849689572 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559c6e935400 con 0x559c6e9283e0 2024-09-10T09:49:45.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:45.063+0000 7fe8ed200640 1 --2- 172.21.15.5:0/849689572 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559c6e9283e0 0x559c6e94e930 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fe8d409bc10 tx=0x7fe8d4004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f1dc07ee81ad20aa server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:49:45.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:45.063+0000 7fe8da000640 1 -- 172.21.15.5:0/849689572 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8d409b5e0 con 0x559c6e9283e0 2024-09-10T09:49:45.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:45.063+0000 7fe8da000640 1 -- 172.21.15.5:0/849689572 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe8d409c380 con 0x559c6e9283e0 2024-09-10T09:49:45.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:45.063+0000 7fe8da000640 1 -- 172.21.15.5:0/849689572 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8d409a010 con 0x559c6e9283e0 2024-09-10T09:49:45.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:45.064+0000 7fe8f3089740 1 -- 172.21.15.5:0/849689572 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559c6e9283e0 msgr2=0x559c6e94e930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:49:45.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:45.064+0000 7fe8f3089740 1 --2- 172.21.15.5:0/849689572 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559c6e9283e0 0x559c6e94e930 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fe8d409bc10 tx=0x7fe8d4004a40 comp rx=0 tx=0).stop 2024-09-10T09:49:45.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:45.064+0000 7fe8f3089740 1 -- 172.21.15.5:0/849689572 shutdown_connections 2024-09-10T09:49:45.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:45.064+0000 7fe8f3089740 1 --2- 172.21.15.5:0/849689572 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559c6e932ca0 0x559c6e94ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:45.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:45.064+0000 7fe8f3089740 1 --2- 172.21.15.5:0/849689572 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559c6e9283e0 0x559c6e94e930 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:45.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:45.064+0000 7fe8f3089740 1 --2- 172.21.15.5:0/849689572 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559c6e927ac0 0x559c6e92ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:45.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:45.064+0000 7fe8f3089740 1 -- 172.21.15.5:0/849689572 >> 172.21.15.5:0/849689572 conn(0x559c6e924050 msgr2=0x559c6e92a690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:49:45.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:45.064+0000 7fe8f3089740 1 -- 172.21.15.5:0/849689572 shutdown_connections 2024-09-10T09:49:45.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:45.064+0000 7fe8f3089740 1 -- 172.21.15.5:0/849689572 wait complete. 2024-09-10T09:49:45.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:45.064+0000 7fe8f3089740 1 -- start start 2024-09-10T09:49:45.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:45.065+0000 7fe8f3089740 1 Processor -- start 2024-09-10T09:49:45.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:45.065+0000 7fe8f3089740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559c6e927ac0 0x559c6e937cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:49:45.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:45.065+0000 7fe8f3089740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559c6e9283e0 0x559c6e9381f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:49:45.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:45.065+0000 7fe8f3089740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559c6e932ca0 0x559c6e93ed10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:49:45.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:45.065+0000 7fe8f3089740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe8e81adf20 con 0x559c6e9283e0 2024-09-10T09:49:45.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:45.065+0000 7fe8f3089740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fe8e81ad850 con 0x559c6e927ac0 2024-09-10T09:49:45.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:45.065+0000 7fe8f3089740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fe8e81aaa90 con 0x559c6e932ca0 2024-09-10T09:49:45.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:45.065+0000 7fe8f3089740 10 client.0 _ll_register_callbacks cb 0x7fe8e0046070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:49:45.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:45.065+0000 7fe8ee600640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559c6e932ca0 0x559c6e93ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:49:45.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:45.065+0000 7fe8edc00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559c6e927ac0 0x559c6e937cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:49:45.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:45.065+0000 7fe8ed200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559c6e9283e0 0x559c6e9381f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:49:45.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:45.065+0000 7fe8ee600640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559c6e932ca0 0x559c6e93ed10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:56784/0 (socket says 172.21.15.5:56784) 2024-09-10T09:49:45.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:45.065+0000 7fe8ee600640 1 -- 172.21.15.5:0/2442683885 learned_addr learned my addr 172.21.15.5:0/2442683885 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:49:45.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:45.066+0000 7fe8edc00640 1 -- 172.21.15.5:0/2442683885 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559c6e932ca0 msgr2=0x559c6e93ed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:49:45.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:45.066+0000 7fe8edc00640 1 --2- 172.21.15.5:0/2442683885 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559c6e932ca0 0x559c6e93ed10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:45.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:45.066+0000 7fe8edc00640 1 -- 172.21.15.5:0/2442683885 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559c6e9283e0 msgr2=0x559c6e9381f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:49:45.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:45.066+0000 7fe8edc00640 1 --2- 172.21.15.5:0/2442683885 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559c6e9283e0 0x559c6e9381f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:45.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:45.066+0000 7fe8edc00640 1 -- 172.21.15.5:0/2442683885 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559c6e93f250 con 0x559c6e927ac0 2024-09-10T09:49:45.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:45.066+0000 7fe8ee600640 1 --2- 172.21.15.5:0/2442683885 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559c6e932ca0 0x559c6e93ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:49:45.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:45.066+0000 7fe8edc00640 1 --2- 172.21.15.5:0/2442683885 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559c6e927ac0 0x559c6e937cb0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x559c6e930dc0 tx=0x7fe8dc0046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ad80837eea72bedf server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:49:45.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:45.066+0000 7fe8cf400640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-09-10T09:49:45.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:45.067+0000 7fe8cf400640 1 -- 172.21.15.5:0/2442683885 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8dc033430 con 0x559c6e927ac0 2024-09-10T09:49:45.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:45.067+0000 7fe8f3089740 1 -- 172.21.15.5:0/2442683885 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe8d409c380 con 0x559c6e927ac0 2024-09-10T09:49:45.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:45.067+0000 7fe8cf400640 1 -- 172.21.15.5:0/2442683885 <== mon.2 v2:172.21.15.102:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fe8dc02a420 con 0x559c6e927ac0 2024-09-10T09:49:45.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:45.067+0000 7fe8cf400640 1 -- 172.21.15.5:0/2442683885 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fe8dc02a620 con 0x559c6e927ac0 2024-09-10T09:49:45.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:45.067+0000 7fe8f3089740 10 client.27728 fetch_fsmap learned FSMap version 36 2024-09-10T09:49:45.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:45.067+0000 7fe8f3089740 1 -- 172.21.15.5:0/2442683885 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x559c6e9b6610 con 0x559c6e927ac0 2024-09-10T09:49:45.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:45.067+0000 7fe8cf400640 1 -- 172.21.15.5:0/2442683885 <== mon.2 v2:172.21.15.102:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe8d409c380 con 0x559c6e927ac0 2024-09-10T09:49:45.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:45.067+0000 7fe8cf400640 1 -- 172.21.15.5:0/2442683885 <== mon.2 v2:172.21.15.102:3300/0 5 ==== fsmap(e 36) ==== 8370+0+0 (secure 0 0 0) 0x7fe8dc033b30 con 0x559c6e927ac0 2024-09-10T09:49:45.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:45.067+0000 7fe8f3089740 10 client.27728 fetch_fsmap finished waiting for FSMap version 36 2024-09-10T09:49:45.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:45.067+0000 7fe8f3089740 10 client.27728 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T09:49:45.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:45.067+0000 7fe8f3089740 20 client.27728 populate_metadata read hostname 'smithi005' 2024-09-10T09:49:45.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:45.067+0000 7fe8f3089740 1 --2- 172.21.15.5:0/2442683885 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x559c6e9b6ce0 0x559c6e9b90d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:49:45.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:45.067+0000 7fe8f3089740 4 client.27728 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-10T09:49:45.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:45.067+0000 7fe8f3089740 1 -- 172.21.15.5:0/2442683885 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x559c6e895d10 con 0x559c6e9b6ce0 2024-09-10T09:49:45.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:45.068+0000 7fe8ed200640 1 --2- 172.21.15.5:0/2442683885 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x559c6e9b6ce0 0x559c6e9b90d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:49:45.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:45.068+0000 7fe8ed200640 1 --2- 172.21.15.5:0/2442683885 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x559c6e9b6ce0 0x559c6e9b90d0 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e759140b75056d44 server_cookie=1a183713ef2f8433 in_seq=0 out_seq=0 2024-09-10T09:49:45.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:45.068+0000 7fe8cf400640 10 client.27728 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T09:49:45.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:45.069+0000 7fe8cf400640 1 -- 172.21.15.5:0/2442683885 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x559c6e895d10 con 0x559c6e9b6ce0 2024-09-10T09:49:45.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:45.069+0000 7fe8cf400640 10 client.27728 handle_command_reply: tid=0 2024-09-10T09:49:45.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:45.069+0000 7fe8f3089740 1 client.27728 shutdown 2024-09-10T09:49:45.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:45.069+0000 7fe8f3089740 1 -- 172.21.15.5:0/2442683885 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559c6e927ac0 msgr2=0x559c6e937cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:49:45.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:45.069+0000 7fe8f3089740 1 --2- 172.21.15.5:0/2442683885 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559c6e927ac0 0x559c6e937cb0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x559c6e930dc0 tx=0x7fe8dc0046c0 comp rx=0 tx=0).stop 2024-09-10T09:49:45.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:45.069+0000 7fe8f3089740 1 -- 172.21.15.5:0/2442683885 shutdown_connections 2024-09-10T09:49:45.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:45.069+0000 7fe8f3089740 1 --2- 172.21.15.5:0/2442683885 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x559c6e9b6ce0 0x559c6e9b90d0 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:45.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:45.069+0000 7fe8f3089740 1 --2- 172.21.15.5:0/2442683885 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559c6e932ca0 0x559c6e93ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:45.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:45.069+0000 7fe8f3089740 1 --2- 172.21.15.5:0/2442683885 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559c6e9283e0 0x559c6e9381f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:45.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:45.069+0000 7fe8f3089740 1 --2- 172.21.15.5:0/2442683885 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559c6e927ac0 0x559c6e937cb0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:45.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:45.069+0000 7fe8f3089740 1 -- 172.21.15.5:0/2442683885 >> 172.21.15.5:0/2442683885 conn(0x559c6e924050 msgr2=0x559c6e92aeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:49:45.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:45.069+0000 7fe8cf400640 0 client.27728 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T09:49:45.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:45.070+0000 7fe8f3089740 1 -- 172.21.15.5:0/2442683885 shutdown_connections 2024-09-10T09:49:45.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:45.070+0000 7fe8f3089740 1 -- 172.21.15.5:0/2442683885 wait complete. 2024-09-10T09:49:45.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:45.070+0000 7fe8f3089740 20 client.27728 trim_cache size 0 max 16384 2024-09-10T09:49:45.075 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-10T09:49:45.075 INFO:teuthology.orchestra.run.smithi005.stdout: "return_code": 0, 2024-09-10T09:49:45.075 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_tag": "3047441d-c46e-455f-9b3a-fa2fbd1db02b", 2024-09-10T09:49:45.075 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": "asynchronous" 2024-09-10T09:49:45.075 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-10T09:49:45.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:45.071+0000 7fe8f0600640 1 -- 172.21.15.5:0/1461515933 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fe8d00a2b80 msgr2=0x7fe8d00a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:49:45.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:45.071+0000 7fe8f0600640 1 --2- 172.21.15.5:0/1461515933 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fe8d00a2b80 0x7fe8d00a5040 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x559c6e924eb0 tx=0x7fe8dc005f90 comp rx=0 tx=0).stop 2024-09-10T09:49:45.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:45.071+0000 7fe8f0600640 1 -- 172.21.15.5:0/1461515933 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe8e8100500 msgr2=0x7fe8e8198920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:49:45.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:45.071+0000 7fe8f0600640 1 --2- 172.21.15.5:0/1461515933 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe8e8100500 0x7fe8e8198920 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fe8d400e9f0 tx=0x7fe8d400edf0 comp rx=0 tx=0).stop 2024-09-10T09:49:45.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:45.071+0000 7fe8f0600640 1 -- 172.21.15.5:0/1461515933 shutdown_connections 2024-09-10T09:49:45.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:45.071+0000 7fe8f0600640 1 --2- 172.21.15.5:0/1461515933 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fe8d00a2b80 0x7fe8d00a5040 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:45.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:45.071+0000 7fe8f0600640 1 --2- 172.21.15.5:0/1461515933 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe8e8106b80 0x7fe8e819ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:45.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:45.071+0000 7fe8f0600640 1 --2- 172.21.15.5:0/1461515933 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe8e8100500 0x7fe8e8198920 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:45.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:45.071+0000 7fe8f0600640 1 --2- 172.21.15.5:0/1461515933 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe8e80ffb60 0x7fe8e81983e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:45.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:45.071+0000 7fe8f0600640 1 -- 172.21.15.5:0/1461515933 >> 172.21.15.5:0/1461515933 conn(0x7fe8e80fba90 msgr2=0x7fe8e80fc200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:49:45.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:45.071+0000 7fe8f0600640 1 -- 172.21.15.5:0/1461515933 shutdown_connections 2024-09-10T09:49:45.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:45.071+0000 7fe8f0600640 1 -- 172.21.15.5:0/1461515933 wait complete. 2024-09-10T09:49:45.125 INFO:tasks.workunit.client.0.smithi005.stdout: 1 689483 6.05 MB/sec execute 446 sec latency 17.156 ms 2024-09-10T09:49:45.771 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'a33fa5fe', total seconds elapsed: 32.4 2024-09-10T09:49:45.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:49:45 smithi159 ceph-mon[35513]: pgmap v735: 129 pgs: 129 active+clean; 259 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 1.2 MiB/s wr, 52 op/s 2024-09-10T09:49:45.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:49:45 smithi159 ceph-mon[35513]: scrub summary: active paths [/] 2024-09-10T09:49:45.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:49:45 smithi159 ceph-mon[35513]: scrub summary: idle+waiting paths [/] 2024-09-10T09:49:45.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:49:45 smithi102 ceph-mon[35442]: pgmap v735: 129 pgs: 129 active+clean; 259 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 1.2 MiB/s wr, 52 op/s 2024-09-10T09:49:45.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:49:45 smithi102 ceph-mon[35442]: scrub summary: active paths [/] 2024-09-10T09:49:45.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:49:45 smithi102 ceph-mon[35442]: scrub summary: idle+waiting paths [/] 2024-09-10T09:49:45.908 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:49:45 smithi005 ceph-mon[30350]: pgmap v735: 129 pgs: 129 active+clean; 259 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 1.2 MiB/s wr, 52 op/s 2024-09-10T09:49:45.908 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:49:45 smithi005 ceph-mon[30350]: scrub summary: active paths [/] 2024-09-10T09:49:45.908 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:49:45 smithi005 ceph-mon[30350]: scrub summary: idle+waiting paths [/] 2024-09-10T09:49:46.125 INFO:tasks.workunit.client.0.smithi005.stdout: 1 691848 6.06 MB/sec execute 447 sec latency 14.766 ms 2024-09-10T09:49:46.295 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T09:49:46.694 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:49:46.837 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:49:46 smithi005 ceph-mon[30350]: scrub queued for path: / 2024-09-10T09:49:46.838 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:49:46 smithi005 ceph-mon[30350]: scrub summary: idle+waiting paths [/] 2024-09-10T09:49:46.838 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:49:46 smithi005 ceph-mon[30350]: scrub summary: active paths [/] 2024-09-10T09:49:46.838 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:49:46 smithi005 ceph-mon[30350]: scrub summary: idle+waiting paths [/] 2024-09-10T09:49:46.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:49:46 smithi159 ceph-mon[35513]: scrub queued for path: / 2024-09-10T09:49:46.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:49:46 smithi159 ceph-mon[35513]: scrub summary: idle+waiting paths [/] 2024-09-10T09:49:46.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:49:46 smithi159 ceph-mon[35513]: scrub summary: active paths [/] 2024-09-10T09:49:46.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:49:46 smithi159 ceph-mon[35513]: scrub summary: idle+waiting paths [/] 2024-09-10T09:49:46.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:49:46 smithi102 ceph-mon[35442]: scrub queued for path: / 2024-09-10T09:49:46.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:49:46 smithi102 ceph-mon[35442]: scrub summary: idle+waiting paths [/] 2024-09-10T09:49:46.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:49:46 smithi102 ceph-mon[35442]: scrub summary: active paths [/] 2024-09-10T09:49:46.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:49:46 smithi102 ceph-mon[35442]: scrub summary: idle+waiting paths [/] 2024-09-10T09:49:47.125 INFO:tasks.workunit.client.0.smithi005.stdout: 1 693406 6.07 MB/sec execute 448 sec latency 45.433 ms 2024-09-10T09:49:47.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:47.269+0000 7fa955a00640 1 -- 172.21.15.5:0/255839855 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa950102da0 msgr2=0x7fa950103180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:49:47.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:47.269+0000 7fa955a00640 1 --2- 172.21.15.5:0/255839855 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa950102da0 0x7fa950103180 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fa940009ca0 tx=0x7fa94002a940 comp rx=0 tx=0).stop 2024-09-10T09:49:47.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:47.269+0000 7fa955a00640 1 -- 172.21.15.5:0/255839855 shutdown_connections 2024-09-10T09:49:47.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:47.269+0000 7fa955a00640 1 --2- 172.21.15.5:0/255839855 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa95010c180 0x7fa95010e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:47.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:47.269+0000 7fa955a00640 1 --2- 172.21.15.5:0/255839855 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa9501036c0 0x7fa95010bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:47.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:47.269+0000 7fa955a00640 1 --2- 172.21.15.5:0/255839855 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa950102da0 0x7fa950103180 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:47.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:47.269+0000 7fa955a00640 1 -- 172.21.15.5:0/255839855 >> 172.21.15.5:0/255839855 conn(0x7fa9500fc7a0 msgr2=0x7fa9500febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:49:47.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:47.269+0000 7fa955a00640 1 -- 172.21.15.5:0/255839855 shutdown_connections 2024-09-10T09:49:47.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:47.269+0000 7fa955a00640 1 -- 172.21.15.5:0/255839855 wait complete. 2024-09-10T09:49:47.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:47.270+0000 7fa955a00640 1 Processor -- start 2024-09-10T09:49:47.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:47.270+0000 7fa955a00640 1 -- start start 2024-09-10T09:49:47.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:47.271+0000 7fa955a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa950102da0 0x7fa95019a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:49:47.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:47.271+0000 7fa955a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa9501036c0 0x7fa95019ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:49:47.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:47.271+0000 7fa955a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa95010c180 0x7fa95019ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:49:47.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:47.271+0000 7fa955a00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa950111fb0 con 0x7fa950102da0 2024-09-10T09:49:47.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:47.271+0000 7fa955a00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fa950111e30 con 0x7fa9501036c0 2024-09-10T09:49:47.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:47.271+0000 7fa955a00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fa950112130 con 0x7fa95010c180 2024-09-10T09:49:47.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:47.271+0000 7fa94ea00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa950102da0 0x7fa95019a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:49:47.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:47.271+0000 7fa94f400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa95010c180 0x7fa95019ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:49:47.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:47.271+0000 7fa94f400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa95010c180 0x7fa95019ef20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:56810/0 (socket says 172.21.15.5:56810) 2024-09-10T09:49:47.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:47.271+0000 7fa94f400640 1 -- 172.21.15.5:0/1623383910 learned_addr learned my addr 172.21.15.5:0/1623383910 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:49:47.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:47.271+0000 7fa946000640 1 --2- 172.21.15.5:0/1623383910 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa9501036c0 0x7fa95019ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:49:47.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:47.272+0000 7fa94f400640 1 -- 172.21.15.5:0/1623383910 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa9501036c0 msgr2=0x7fa95019ac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:49:47.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:47.272+0000 7fa94f400640 1 --2- 172.21.15.5:0/1623383910 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa9501036c0 0x7fa95019ac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:47.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:47.272+0000 7fa94f400640 1 -- 172.21.15.5:0/1623383910 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa950102da0 msgr2=0x7fa95019a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:49:47.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:47.272+0000 7fa94f400640 1 --2- 172.21.15.5:0/1623383910 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa950102da0 0x7fa95019a6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:47.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:47.272+0000 7fa94f400640 1 -- 172.21.15.5:0/1623383910 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa95019f600 con 0x7fa95010c180 2024-09-10T09:49:47.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:47.272+0000 7fa946000640 1 --2- 172.21.15.5:0/1623383910 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa9501036c0 0x7fa95019ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:49:47.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:47.272+0000 7fa94ea00640 1 --2- 172.21.15.5:0/1623383910 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa950102da0 0x7fa95019a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:49:47.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:47.272+0000 7fa94f400640 1 --2- 172.21.15.5:0/1623383910 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa95010c180 0x7fa95019ef20 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fa94800e9b0 tx=0x7fa94800ee70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:49:47.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:47.272+0000 7fa947e00640 1 -- 172.21.15.5:0/1623383910 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa94800cd80 con 0x7fa95010c180 2024-09-10T09:49:47.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:47.272+0000 7fa947e00640 1 -- 172.21.15.5:0/1623383910 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa9480035f0 con 0x7fa95010c180 2024-09-10T09:49:47.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:47.272+0000 7fa955a00640 1 -- 172.21.15.5:0/1623383910 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa95019f8f0 con 0x7fa95010c180 2024-09-10T09:49:47.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:47.272+0000 7fa947e00640 1 -- 172.21.15.5:0/1623383910 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa948010670 con 0x7fa95010c180 2024-09-10T09:49:47.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:47.272+0000 7fa955a00640 1 -- 172.21.15.5:0/1623383910 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9501a71d0 con 0x7fa95010c180 2024-09-10T09:49:47.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:47.277+0000 7fa95841c740 1 Processor -- start 2024-09-10T09:49:47.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:47.277+0000 7fa947e00640 1 -- 172.21.15.5:0/1623383910 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7fa9480037a0 con 0x7fa95010c180 2024-09-10T09:49:47.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:47.277+0000 7fa95841c740 1 -- start start 2024-09-10T09:49:47.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:47.277+0000 7fa95841c740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561e8f720b90 0x561e8f720f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:49:47.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:47.277+0000 7fa95841c740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561e8f7214b0 0x561e8f72b830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:49:47.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:47.277+0000 7fa95841c740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561e8f72bd70 0x561e8f72e160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:49:47.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:47.277+0000 7fa95841c740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x561e8f6873e0 con 0x561e8f7214b0 2024-09-10T09:49:47.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:47.277+0000 7fa95841c740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x561e8f62ed80 con 0x561e8f72bd70 2024-09-10T09:49:47.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:47.277+0000 7fa95841c740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x561e8f6833a0 con 0x561e8f720b90 2024-09-10T09:49:47.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:47.278+0000 7fa946000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561e8f720b90 0x561e8f720f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:49:47.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:47.278+0000 7fa94ea00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561e8f7214b0 0x561e8f72b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:49:47.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:47.278+0000 7fa946000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561e8f720b90 0x561e8f720f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:56814/0 (socket says 172.21.15.5:56814) 2024-09-10T09:49:47.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:47.278+0000 7fa946000640 1 -- 172.21.15.5:0/3527510603 learned_addr learned my addr 172.21.15.5:0/3527510603 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:49:47.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:47.278+0000 7fa946000640 1 --2- 172.21.15.5:0/3527510603 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561e8f72bd70 0x561e8f72e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:49:47.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:47.278+0000 7fa94ea00640 1 -- 172.21.15.5:0/3527510603 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561e8f720b90 msgr2=0x561e8f720f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:49:47.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:47.278+0000 7fa94ea00640 1 --2- 172.21.15.5:0/3527510603 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561e8f720b90 0x561e8f720f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:47.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:47.278+0000 7fa94ea00640 1 -- 172.21.15.5:0/3527510603 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561e8f72bd70 msgr2=0x561e8f72e160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:49:47.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:47.278+0000 7fa94ea00640 1 --2- 172.21.15.5:0/3527510603 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561e8f72bd70 0x561e8f72e160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:47.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:47.278+0000 7fa94ea00640 1 -- 172.21.15.5:0/3527510603 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561e8f64dae0 con 0x561e8f7214b0 2024-09-10T09:49:47.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:47.279+0000 7fa94ea00640 1 --2- 172.21.15.5:0/3527510603 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561e8f7214b0 0x561e8f72b830 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fa940005e30 tx=0x7fa940005f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a10ddf402742fae4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:49:47.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:47.279+0000 7fa947e00640 1 --2- 172.21.15.5:0/1623383910 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fa9340a2b80 0x7fa9340a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:49:47.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:47.279+0000 7fa947e00640 1 -- 172.21.15.5:0/1623383910 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fa9480cf710 con 0x7fa95010c180 2024-09-10T09:49:47.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:47.279+0000 7fa93fe00640 1 -- 172.21.15.5:0/3527510603 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa940034040 con 0x561e8f7214b0 2024-09-10T09:49:47.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:47.279+0000 7fa93fe00640 1 -- 172.21.15.5:0/3527510603 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa9400309f0 con 0x561e8f7214b0 2024-09-10T09:49:47.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:47.279+0000 7fa93fe00640 1 -- 172.21.15.5:0/3527510603 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa94002f930 con 0x561e8f7214b0 2024-09-10T09:49:47.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:47.279+0000 7fa946000640 1 --2- 172.21.15.5:0/1623383910 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fa9340a2b80 0x7fa9340a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:49:47.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:47.280+0000 7fa946000640 1 --2- 172.21.15.5:0/1623383910 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fa9340a2b80 0x7fa9340a5040 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fa9380097e0 tx=0x7fa938012040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:49:47.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:47.280+0000 7fa95841c740 1 -- 172.21.15.5:0/3527510603 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561e8f7214b0 msgr2=0x561e8f72b830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:49:47.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:47.280+0000 7fa95841c740 1 --2- 172.21.15.5:0/3527510603 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561e8f7214b0 0x561e8f72b830 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fa940005e30 tx=0x7fa940005f80 comp rx=0 tx=0).stop 2024-09-10T09:49:47.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:47.280+0000 7fa95841c740 1 -- 172.21.15.5:0/3527510603 shutdown_connections 2024-09-10T09:49:47.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:47.280+0000 7fa95841c740 1 --2- 172.21.15.5:0/3527510603 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561e8f72bd70 0x561e8f72e160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:47.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:47.280+0000 7fa95841c740 1 --2- 172.21.15.5:0/3527510603 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561e8f7214b0 0x561e8f72b830 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:47.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:47.280+0000 7fa95841c740 1 --2- 172.21.15.5:0/3527510603 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561e8f720b90 0x561e8f720f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:47.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:47.280+0000 7fa95841c740 1 -- 172.21.15.5:0/3527510603 >> 172.21.15.5:0/3527510603 conn(0x561e8f71d120 msgr2=0x561e8f71d500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:49:47.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:47.280+0000 7fa95841c740 1 -- 172.21.15.5:0/3527510603 shutdown_connections 2024-09-10T09:49:47.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:47.280+0000 7fa95841c740 1 -- 172.21.15.5:0/3527510603 wait complete. 2024-09-10T09:49:47.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:47.281+0000 7fa95841c740 1 -- start start 2024-09-10T09:49:47.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:47.281+0000 7fa95841c740 1 Processor -- start 2024-09-10T09:49:47.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:47.281+0000 7fa95841c740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561e8f720b90 0x561e8f728c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:49:47.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:47.281+0000 7fa95841c740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561e8f7214b0 0x561e8f729150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:49:47.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:47.282+0000 7fa95841c740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561e8f72bd70 0x561e8f729690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:49:47.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:47.282+0000 7fa95841c740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x561e8f72e9e0 con 0x561e8f720b90 2024-09-10T09:49:47.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:47.282+0000 7fa95841c740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x561e8f6420f0 con 0x561e8f7214b0 2024-09-10T09:49:47.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:47.282+0000 7fa95841c740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x561e8f72eb60 con 0x561e8f72bd70 2024-09-10T09:49:47.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:47.282+0000 7fa95841c740 10 client.0 _ll_register_callbacks cb 0x561e8f6339e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:49:47.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:47.282+0000 7fa946000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561e8f7214b0 0x561e8f729150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:49:47.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:47.282+0000 7fa946000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561e8f7214b0 0x561e8f729150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:42562/0 (socket says 172.21.15.5:42562) 2024-09-10T09:49:47.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:47.282+0000 7fa94ea00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561e8f720b90 0x561e8f728c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:49:47.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:47.282+0000 7fa94f400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561e8f72bd70 0x561e8f729690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:49:47.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:47.282+0000 7fa946000640 1 -- 172.21.15.5:0/3562561733 learned_addr learned my addr 172.21.15.5:0/3562561733 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:49:47.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:47.282+0000 7fa94f400640 1 -- 172.21.15.5:0/3562561733 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561e8f7214b0 msgr2=0x561e8f729150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:49:47.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:47.282+0000 7fa94f400640 1 --2- 172.21.15.5:0/3562561733 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561e8f7214b0 0x561e8f729150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:47.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:47.282+0000 7fa94f400640 1 -- 172.21.15.5:0/3562561733 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561e8f720b90 msgr2=0x561e8f728c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:49:47.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:47.282+0000 7fa94f400640 1 --2- 172.21.15.5:0/3562561733 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561e8f720b90 0x561e8f728c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:47.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:47.282+0000 7fa94f400640 1 -- 172.21.15.5:0/3562561733 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561e8f3f83c0 con 0x561e8f72bd70 2024-09-10T09:49:47.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:47.283+0000 7fa94f400640 1 --2- 172.21.15.5:0/3562561733 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561e8f72bd70 0x561e8f729690 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fa9480ccdc0 tx=0x7fa94809c2a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=18b7fccdcb8cd88e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:49:47.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:47.283+0000 7fa93cc00640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-09-10T09:49:47.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:47.283+0000 7fa93cc00640 1 -- 172.21.15.5:0/3562561733 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9480a4790 con 0x561e8f72bd70 2024-09-10T09:49:47.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:47.283+0000 7fa93cc00640 1 -- 172.21.15.5:0/3562561733 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fa9480cf3a0 con 0x561e8f72bd70 2024-09-10T09:49:47.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:47.283+0000 7fa95841c740 1 -- 172.21.15.5:0/3562561733 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa9400309f0 con 0x561e8f72bd70 2024-09-10T09:49:47.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:47.283+0000 7fa93cc00640 1 -- 172.21.15.5:0/3562561733 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fa94809af70 con 0x561e8f72bd70 2024-09-10T09:49:47.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:47.283+0000 7fa95841c740 10 client.27748 fetch_fsmap learned FSMap version 36 2024-09-10T09:49:47.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:47.283+0000 7fa95841c740 1 -- 172.21.15.5:0/3562561733 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x561e8f73a640 con 0x561e8f72bd70 2024-09-10T09:49:47.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:47.283+0000 7fa93cc00640 1 -- 172.21.15.5:0/3562561733 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa9400309f0 con 0x561e8f72bd70 2024-09-10T09:49:47.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:47.284+0000 7fa93cc00640 1 -- 172.21.15.5:0/3562561733 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 36) ==== 8370+0+0 (secure 0 0 0) 0x7fa9480041c0 con 0x561e8f72bd70 2024-09-10T09:49:47.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:47.284+0000 7fa95841c740 10 client.27748 fetch_fsmap finished waiting for FSMap version 36 2024-09-10T09:49:47.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:47.284+0000 7fa95841c740 10 client.27748 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T09:49:47.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:47.284+0000 7fa95841c740 20 client.27748 populate_metadata read hostname 'smithi005' 2024-09-10T09:49:47.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:47.284+0000 7fa95841c740 1 --2- 172.21.15.5:0/3562561733 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x561e8f73baa0 0x561e8f73df30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:49:47.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:47.284+0000 7fa95841c740 4 client.27748 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:49:47.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:47.284+0000 7fa95841c740 1 -- 172.21.15.5:0/3562561733 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561e8f694950 con 0x561e8f73baa0 2024-09-10T09:49:47.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:47.284+0000 7fa94ea00640 1 --2- 172.21.15.5:0/3562561733 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x561e8f73baa0 0x561e8f73df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:49:47.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:47.285+0000 7fa94ea00640 1 --2- 172.21.15.5:0/3562561733 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x561e8f73baa0 0x561e8f73df30 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=45c66adc340ac5c3 server_cookie=86e79186fe6a7bf2 in_seq=0 out_seq=0 2024-09-10T09:49:47.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:47.285+0000 7fa93cc00640 10 client.27748 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T09:49:47.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:47.285+0000 7fa93cc00640 1 -- 172.21.15.5:0/3562561733 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561e8f694950 con 0x561e8f73baa0 2024-09-10T09:49:47.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:47.285+0000 7fa93cc00640 10 client.27748 handle_command_reply: tid=0 2024-09-10T09:49:47.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:47.285+0000 7fa95841c740 1 client.27748 shutdown 2024-09-10T09:49:47.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:47.286+0000 7fa95841c740 1 -- 172.21.15.5:0/3562561733 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561e8f72bd70 msgr2=0x561e8f729690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:49:47.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:47.286+0000 7fa95841c740 1 --2- 172.21.15.5:0/3562561733 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561e8f72bd70 0x561e8f729690 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fa9480ccdc0 tx=0x7fa94809c2a0 comp rx=0 tx=0).stop 2024-09-10T09:49:47.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:47.286+0000 7fa95841c740 1 -- 172.21.15.5:0/3562561733 shutdown_connections 2024-09-10T09:49:47.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:47.286+0000 7fa95841c740 1 --2- 172.21.15.5:0/3562561733 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x561e8f73baa0 0x561e8f73df30 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:47.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:47.286+0000 7fa95841c740 1 --2- 172.21.15.5:0/3562561733 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561e8f72bd70 0x561e8f729690 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:47.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:47.286+0000 7fa95841c740 1 --2- 172.21.15.5:0/3562561733 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561e8f7214b0 0x561e8f729150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:47.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:47.286+0000 7fa95841c740 1 --2- 172.21.15.5:0/3562561733 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561e8f720b90 0x561e8f728c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:47.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:47.286+0000 7fa93cc00640 0 client.27748 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T09:49:47.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:47.286+0000 7fa95841c740 1 -- 172.21.15.5:0/3562561733 >> 172.21.15.5:0/3562561733 conn(0x561e8f71d120 msgr2=0x561e8f722e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:49:47.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:47.286+0000 7fa95841c740 1 -- 172.21.15.5:0/3562561733 shutdown_connections 2024-09-10T09:49:47.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:47.286+0000 7fa95841c740 1 -- 172.21.15.5:0/3562561733 wait complete. 2024-09-10T09:49:47.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:47.286+0000 7fa95841c740 20 client.27748 trim_cache size 0 max 16384 2024-09-10T09:49:47.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:47.305+0000 7fa95841c740 1 Processor -- start 2024-09-10T09:49:47.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:47.306+0000 7fa95841c740 1 -- start start 2024-09-10T09:49:47.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:47.306+0000 7fa95841c740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561e8f720b90 0x561e8f743800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:49:47.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:47.306+0000 7fa95841c740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561e8f7214b0 0x561e8f743d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:49:47.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:47.306+0000 7fa95841c740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561e8f72bd70 0x561e8f725e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:49:47.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:47.306+0000 7fa95841c740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa9501b10b0 con 0x561e8f72bd70 2024-09-10T09:49:47.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:47.306+0000 7fa95841c740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fa9501b2bf0 con 0x561e8f7214b0 2024-09-10T09:49:47.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:47.306+0000 7fa95841c740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fa9501b1780 con 0x561e8f720b90 2024-09-10T09:49:47.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:47.306+0000 7fa946000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561e8f7214b0 0x561e8f743d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:49:47.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:47.306+0000 7fa94ea00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561e8f720b90 0x561e8f743800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:49:47.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:47.306+0000 7fa946000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561e8f7214b0 0x561e8f743d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:42568/0 (socket says 172.21.15.5:42568) 2024-09-10T09:49:47.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:47.306+0000 7fa946000640 1 -- 172.21.15.5:0/1651589107 learned_addr learned my addr 172.21.15.5:0/1651589107 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:49:47.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:47.306+0000 7fa94f400640 1 --2- 172.21.15.5:0/1651589107 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561e8f72bd70 0x561e8f725e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:49:47.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:47.307+0000 7fa946000640 1 -- 172.21.15.5:0/1651589107 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561e8f720b90 msgr2=0x561e8f743800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:49:47.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:47.307+0000 7fa946000640 1 --2- 172.21.15.5:0/1651589107 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561e8f720b90 0x561e8f743800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:47.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:47.307+0000 7fa946000640 1 -- 172.21.15.5:0/1651589107 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561e8f72bd70 msgr2=0x561e8f725e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:49:47.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:47.307+0000 7fa946000640 1 --2- 172.21.15.5:0/1651589107 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561e8f72bd70 0x561e8f725e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:47.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:47.307+0000 7fa946000640 1 -- 172.21.15.5:0/1651589107 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561e8f71ddb0 con 0x561e8f7214b0 2024-09-10T09:49:47.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:47.307+0000 7fa94ea00640 1 --2- 172.21.15.5:0/1651589107 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561e8f720b90 0x561e8f743800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:49:47.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:47.307+0000 7fa946000640 1 --2- 172.21.15.5:0/1651589107 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561e8f7214b0 0x561e8f743d40 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fa938010fd0 tx=0x7fa938009770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e0df3090b661b411 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:49:47.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:47.307+0000 7fa93f400640 1 -- 172.21.15.5:0/1651589107 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9380044d0 con 0x561e8f7214b0 2024-09-10T09:49:47.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:47.308+0000 7fa93f400640 1 -- 172.21.15.5:0/1651589107 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa938013e40 con 0x561e8f7214b0 2024-09-10T09:49:47.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:47.308+0000 7fa93f400640 1 -- 172.21.15.5:0/1651589107 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9380035f0 con 0x561e8f7214b0 2024-09-10T09:49:47.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:47.308+0000 7fa95841c740 1 -- 172.21.15.5:0/1651589107 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561e8f7214b0 msgr2=0x561e8f743d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:49:47.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:47.308+0000 7fa95841c740 1 --2- 172.21.15.5:0/1651589107 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561e8f7214b0 0x561e8f743d40 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fa938010fd0 tx=0x7fa938009770 comp rx=0 tx=0).stop 2024-09-10T09:49:47.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:47.308+0000 7fa95841c740 1 -- 172.21.15.5:0/1651589107 shutdown_connections 2024-09-10T09:49:47.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:47.308+0000 7fa95841c740 1 --2- 172.21.15.5:0/1651589107 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561e8f72bd70 0x561e8f725e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:47.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:47.308+0000 7fa95841c740 1 --2- 172.21.15.5:0/1651589107 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561e8f7214b0 0x561e8f743d40 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:47.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:47.308+0000 7fa95841c740 1 --2- 172.21.15.5:0/1651589107 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561e8f720b90 0x561e8f743800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:47.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:47.308+0000 7fa95841c740 1 -- 172.21.15.5:0/1651589107 >> 172.21.15.5:0/1651589107 conn(0x561e8f71d120 msgr2=0x561e8f72ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:49:47.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:47.309+0000 7fa95841c740 1 -- 172.21.15.5:0/1651589107 shutdown_connections 2024-09-10T09:49:47.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:47.309+0000 7fa95841c740 1 -- 172.21.15.5:0/1651589107 wait complete. 2024-09-10T09:49:47.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:47.309+0000 7fa95841c740 1 -- start start 2024-09-10T09:49:47.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:47.309+0000 7fa95841c740 1 Processor -- start 2024-09-10T09:49:47.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:47.310+0000 7fa95841c740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561e8f720b90 0x561e8f72fd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:49:47.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:47.310+0000 7fa95841c740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561e8f7214b0 0x561e8f730290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:49:47.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:47.310+0000 7fa95841c740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561e8f72bd70 0x561e8f7307d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:49:47.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:47.310+0000 7fa95841c740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa9501b2520 con 0x561e8f72bd70 2024-09-10T09:49:47.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:47.310+0000 7fa95841c740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fa9501b1e50 con 0x561e8f7214b0 2024-09-10T09:49:47.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:47.310+0000 7fa95841c740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fa9501af090 con 0x561e8f720b90 2024-09-10T09:49:47.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:47.310+0000 7fa95841c740 10 client.0 _ll_register_callbacks cb 0x7fa94809b480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:49:47.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:47.310+0000 7fa94ea00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561e8f720b90 0x561e8f72fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:49:47.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:47.310+0000 7fa94f400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561e8f72bd70 0x561e8f7307d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:49:47.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:47.310+0000 7fa94ea00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561e8f720b90 0x561e8f72fd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:56836/0 (socket says 172.21.15.5:56836) 2024-09-10T09:49:47.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:47.310+0000 7fa94ea00640 1 -- 172.21.15.5:0/36257396 learned_addr learned my addr 172.21.15.5:0/36257396 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:49:47.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:47.310+0000 7fa946000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561e8f7214b0 0x561e8f730290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:49:47.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:47.311+0000 7fa94ea00640 1 -- 172.21.15.5:0/36257396 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561e8f7214b0 msgr2=0x561e8f730290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:49:47.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:47.311+0000 7fa94ea00640 1 --2- 172.21.15.5:0/36257396 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561e8f7214b0 0x561e8f730290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:47.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:47.311+0000 7fa94ea00640 1 -- 172.21.15.5:0/36257396 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561e8f72bd70 msgr2=0x561e8f7307d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:49:47.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:47.311+0000 7fa94ea00640 1 --2- 172.21.15.5:0/36257396 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561e8f72bd70 0x561e8f7307d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:47.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:47.311+0000 7fa94ea00640 1 -- 172.21.15.5:0/36257396 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561e8f7a5a20 con 0x561e8f720b90 2024-09-10T09:49:47.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:47.311+0000 7fa94ea00640 1 --2- 172.21.15.5:0/36257396 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561e8f720b90 0x561e8f72fd50 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fa940006010 tx=0x7fa9400099d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=64ac9a56ece09003 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:49:47.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:47.311+0000 7fa93cc00640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-09-10T09:49:47.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:47.311+0000 7fa93cc00640 1 -- 172.21.15.5:0/36257396 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa940034040 con 0x561e8f720b90 2024-09-10T09:49:47.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:47.311+0000 7fa95841c740 1 -- 172.21.15.5:0/36257396 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x561e8f68ee60 con 0x561e8f720b90 2024-09-10T09:49:47.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:47.311+0000 7fa93cc00640 1 -- 172.21.15.5:0/36257396 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fa940040040 con 0x561e8f720b90 2024-09-10T09:49:47.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:47.312+0000 7fa93cc00640 1 -- 172.21.15.5:0/36257396 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fa94002f5a0 con 0x561e8f720b90 2024-09-10T09:49:47.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:47.312+0000 7fa95841c740 10 client.27754 fetch_fsmap learned FSMap version 36 2024-09-10T09:49:47.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:47.312+0000 7fa95841c740 1 -- 172.21.15.5:0/36257396 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x561e8f7a6d70 con 0x561e8f720b90 2024-09-10T09:49:47.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:47.312+0000 7fa93cc00640 1 -- 172.21.15.5:0/36257396 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x561e8f68ee60 con 0x561e8f720b90 2024-09-10T09:49:47.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:47.312+0000 7fa93cc00640 1 -- 172.21.15.5:0/36257396 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 36) ==== 8370+0+0 (secure 0 0 0) 0x7fa94002f070 con 0x561e8f720b90 2024-09-10T09:49:47.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:47.312+0000 7fa95841c740 10 client.27754 fetch_fsmap finished waiting for FSMap version 36 2024-09-10T09:49:47.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:47.312+0000 7fa95841c740 10 client.27754 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T09:49:47.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:47.312+0000 7fa95841c740 20 client.27754 populate_metadata read hostname 'smithi005' 2024-09-10T09:49:47.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:47.312+0000 7fa95841c740 1 --2- 172.21.15.5:0/36257396 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x561e8f7a70b0 0x561e8f7a94a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:49:47.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:47.312+0000 7fa95841c740 4 client.27754 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T09:49:47.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:47.312+0000 7fa95841c740 1 -- 172.21.15.5:0/36257396 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "scrub status"}) -- 0x561e8f60b0c0 con 0x561e8f7a70b0 2024-09-10T09:49:47.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:47.313+0000 7fa946000640 1 --2- 172.21.15.5:0/36257396 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x561e8f7a70b0 0x561e8f7a94a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:49:47.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:47.313+0000 7fa946000640 1 --2- 172.21.15.5:0/36257396 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x561e8f7a70b0 0x561e8f7a94a0 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7a7e8e9f2b929e80 server_cookie=90042ec845287eba in_seq=0 out_seq=0 2024-09-10T09:49:47.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:47.313+0000 7fa93cc00640 10 client.27754 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T09:49:47.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:47.314+0000 7fa93cc00640 1 -- 172.21.15.5:0/36257396 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x561e8f60b0c0 con 0x561e8f7a70b0 2024-09-10T09:49:47.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:47.314+0000 7fa93cc00640 10 client.27754 handle_command_reply: tid=0 2024-09-10T09:49:47.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:47.314+0000 7fa95841c740 1 client.27754 shutdown 2024-09-10T09:49:47.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:47.314+0000 7fa95841c740 1 -- 172.21.15.5:0/36257396 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561e8f720b90 msgr2=0x561e8f72fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:49:47.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:47.314+0000 7fa95841c740 1 --2- 172.21.15.5:0/36257396 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561e8f720b90 0x561e8f72fd50 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fa940006010 tx=0x7fa9400099d0 comp rx=0 tx=0).stop 2024-09-10T09:49:47.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:47.314+0000 7fa95841c740 1 -- 172.21.15.5:0/36257396 shutdown_connections 2024-09-10T09:49:47.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:47.314+0000 7fa95841c740 1 --2- 172.21.15.5:0/36257396 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x561e8f7a70b0 0x561e8f7a94a0 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:47.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:47.314+0000 7fa95841c740 1 --2- 172.21.15.5:0/36257396 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561e8f72bd70 0x561e8f7307d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:47.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:47.314+0000 7fa95841c740 1 --2- 172.21.15.5:0/36257396 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561e8f7214b0 0x561e8f730290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:47.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:47.314+0000 7fa95841c740 1 --2- 172.21.15.5:0/36257396 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561e8f720b90 0x561e8f72fd50 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:47.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:47.314+0000 7fa93cc00640 0 client.27754 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T09:49:47.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:47.314+0000 7fa95841c740 1 -- 172.21.15.5:0/36257396 >> 172.21.15.5:0/36257396 conn(0x561e8f71d120 msgr2=0x561e8f728bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:49:47.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:47.314+0000 7fa95841c740 1 -- 172.21.15.5:0/36257396 shutdown_connections 2024-09-10T09:49:47.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:47.314+0000 7fa95841c740 1 -- 172.21.15.5:0/36257396 wait complete. 2024-09-10T09:49:47.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:47.314+0000 7fa95841c740 20 client.27754 trim_cache size 0 max 16384 2024-09-10T09:49:47.319 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-10T09:49:47.319 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-10T09:49:47.319 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2024-09-10T09:49:47.319 INFO:teuthology.orchestra.run.smithi005.stdout: "3047441d-c46e-455f-9b3a-fa2fbd1db02b": { 2024-09-10T09:49:47.319 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2024-09-10T09:49:47.319 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "3047441d-c46e-455f-9b3a-fa2fbd1db02b", 2024-09-10T09:49:47.319 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2024-09-10T09:49:47.320 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T09:49:47.320 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T09:49:47.320 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-10T09:49:47.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:47.315+0000 7fa955a00640 1 -- 172.21.15.5:0/1623383910 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fa9340a2b80 msgr2=0x7fa9340a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:49:47.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:47.315+0000 7fa955a00640 1 --2- 172.21.15.5:0/1623383910 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fa9340a2b80 0x7fa9340a5040 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fa9380097e0 tx=0x7fa938012040 comp rx=0 tx=0).stop 2024-09-10T09:49:47.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:47.315+0000 7fa955a00640 1 -- 172.21.15.5:0/1623383910 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa95010c180 msgr2=0x7fa95019ef20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:49:47.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:47.315+0000 7fa955a00640 1 --2- 172.21.15.5:0/1623383910 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa95010c180 0x7fa95019ef20 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fa94800e9b0 tx=0x7fa94800ee70 comp rx=0 tx=0).stop 2024-09-10T09:49:47.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:47.315+0000 7fa955a00640 1 -- 172.21.15.5:0/1623383910 shutdown_connections 2024-09-10T09:49:47.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:47.315+0000 7fa955a00640 1 --2- 172.21.15.5:0/1623383910 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fa9340a2b80 0x7fa9340a5040 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:47.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:47.315+0000 7fa955a00640 1 --2- 172.21.15.5:0/1623383910 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa95010c180 0x7fa95019ef20 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:47.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:47.315+0000 7fa955a00640 1 --2- 172.21.15.5:0/1623383910 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa9501036c0 0x7fa95019ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:47.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:47.315+0000 7fa955a00640 1 --2- 172.21.15.5:0/1623383910 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa950102da0 0x7fa95019a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:49:47.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:47.315+0000 7fa955a00640 1 -- 172.21.15.5:0/1623383910 >> 172.21.15.5:0/1623383910 conn(0x7fa9500fc7a0 msgr2=0x7fa950101b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:49:47.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:47.315+0000 7fa955a00640 1 -- 172.21.15.5:0/1623383910 shutdown_connections 2024-09-10T09:49:47.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:49:47.315+0000 7fa955a00640 1 -- 172.21.15.5:0/1623383910 wait complete. 2024-09-10T09:49:47.706 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:49:47 smithi005 ceph-mon[30350]: pgmap v736: 129 pgs: 129 active+clean; 258 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 340 KiB/s rd, 1.4 MiB/s wr, 62 op/s 2024-09-10T09:49:47.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:49:47 smithi159 ceph-mon[35513]: pgmap v736: 129 pgs: 129 active+clean; 258 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 340 KiB/s rd, 1.4 MiB/s wr, 62 op/s 2024-09-10T09:49:47.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:49:47 smithi102 ceph-mon[35442]: pgmap v736: 129 pgs: 129 active+clean; 258 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 340 KiB/s rd, 1.4 MiB/s wr, 62 op/s 2024-09-10T09:49:48.125 INFO:tasks.workunit.client.0.smithi005.stdout: 1 694010 6.06 MB/sec execute 449 sec latency 593.280 ms 2024-09-10T09:49:48.931 INFO:tasks.cephfs.filesystem:scrub status for tag:3047441d-c46e-455f-9b3a-fa2fbd1db02b - {'path': '/', 'tag': '3047441d-c46e-455f-9b3a-fa2fbd1db02b', 'options': 'recursive,force'} 2024-09-10T09:49:49.126 INFO:tasks.workunit.client.0.smithi005.stdout: 1 694830 6.05 MB/sec execute 450 sec latency 846.221 ms 2024-09-10T09:49:49.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:49:49 smithi159 ceph-mon[35513]: pgmap v737: 129 pgs: 129 active+clean; 257 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 809 KiB/s rd, 1.6 MiB/s wr, 96 op/s 2024-09-10T09:49:49.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:49:49 smithi102 ceph-mon[35442]: pgmap v737: 129 pgs: 129 active+clean; 257 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 809 KiB/s rd, 1.6 MiB/s wr, 96 op/s 2024-09-10T09:49:50.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:49:49 smithi005 ceph-mon[30350]: pgmap v737: 129 pgs: 129 active+clean; 257 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 809 KiB/s rd, 1.6 MiB/s wr, 96 op/s 2024-09-10T09:49:50.126 INFO:tasks.workunit.client.0.smithi005.stdout: 1 696830 6.07 MB/sec execute 451 sec latency 23.129 ms 2024-09-10T09:49:51.126 INFO:tasks.workunit.client.0.smithi005.stdout: 1 699592 6.09 MB/sec execute 452 sec latency 11.604 ms 2024-09-10T09:49:51.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:49:51 smithi159 ceph-mon[35513]: pgmap v738: 129 pgs: 129 active+clean; 255 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.8 MiB/s wr, 128 op/s 2024-09-10T09:49:51.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:49:51 smithi102 ceph-mon[35442]: pgmap v738: 129 pgs: 129 active+clean; 255 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.8 MiB/s wr, 128 op/s 2024-09-10T09:49:52.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:49:51 smithi005 ceph-mon[30350]: pgmap v738: 129 pgs: 129 active+clean; 255 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.8 MiB/s wr, 128 op/s 2024-09-10T09:49:52.126 INFO:tasks.workunit.client.0.smithi005.stdout: 1 701174 6.09 MB/sec execute 453 sec latency 22.533 ms 2024-09-10T09:49:53.126 INFO:tasks.workunit.client.0.smithi005.stdout: 1 703638 6.11 MB/sec execute 454 sec latency 10.609 ms 2024-09-10T09:49:53.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:49:53 smithi159 ceph-mon[35513]: pgmap v739: 129 pgs: 129 active+clean; 256 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.8 MiB/s wr, 138 op/s 2024-09-10T09:49:53.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:49:53 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:49:53.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:49:53 smithi102 ceph-mon[35442]: pgmap v739: 129 pgs: 129 active+clean; 256 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.8 MiB/s wr, 138 op/s 2024-09-10T09:49:53.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:49:53 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:49:54.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:49:53 smithi005 ceph-mon[30350]: pgmap v739: 129 pgs: 129 active+clean; 256 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.8 MiB/s wr, 138 op/s 2024-09-10T09:49:54.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:49:53 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:49:54.126 INFO:tasks.workunit.client.0.smithi005.stdout: 1 705726 6.11 MB/sec execute 455 sec latency 36.867 ms 2024-09-10T09:49:54.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:49:54 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:49:54.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:49:54 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:49:54.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:49:54 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:49:54.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:49:54 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:49:54.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:49:54 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:49:54.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:49:54 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:49:55.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:49:54 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:49:55.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:49:54 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:49:55.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:49:54 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:49:55.126 INFO:tasks.workunit.client.0.smithi005.stdout: 1 707654 6.13 MB/sec execute 456 sec latency 29.223 ms 2024-09-10T09:49:55.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:49:55 smithi159 ceph-mon[35513]: pgmap v740: 129 pgs: 129 active+clean; 255 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.8 MiB/s wr, 139 op/s 2024-09-10T09:49:55.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:49:55 smithi102 ceph-mon[35442]: pgmap v740: 129 pgs: 129 active+clean; 255 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.8 MiB/s wr, 139 op/s 2024-09-10T09:49:56.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:49:55 smithi005 ceph-mon[30350]: pgmap v740: 129 pgs: 129 active+clean; 255 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.8 MiB/s wr, 139 op/s 2024-09-10T09:49:56.126 INFO:tasks.workunit.client.0.smithi005.stdout: 1 710378 6.15 MB/sec execute 457 sec latency 11.500 ms 2024-09-10T09:49:57.126 INFO:tasks.workunit.client.0.smithi005.stdout: 1 712139 6.15 MB/sec execute 458 sec latency 35.635 ms 2024-09-10T09:49:57.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:49:57 smithi159 ceph-mon[35513]: pgmap v741: 129 pgs: 129 active+clean; 258 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 1.9 MiB/s wr, 198 op/s 2024-09-10T09:49:57.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:49:57 smithi159 ceph-mon[35513]: scrub summary: idle 2024-09-10T09:49:57.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:49:57 smithi159 ceph-mon[35513]: scrub summary: idle 2024-09-10T09:49:57.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:49:57 smithi102 ceph-mon[35442]: pgmap v741: 129 pgs: 129 active+clean; 258 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 1.9 MiB/s wr, 198 op/s 2024-09-10T09:49:57.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:49:57 smithi102 ceph-mon[35442]: scrub summary: idle 2024-09-10T09:49:57.885 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:49:57 smithi102 ceph-mon[35442]: scrub summary: idle 2024-09-10T09:49:58.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:49:57 smithi005 ceph-mon[30350]: pgmap v741: 129 pgs: 129 active+clean; 258 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 1.9 MiB/s wr, 198 op/s 2024-09-10T09:49:58.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:49:57 smithi005 ceph-mon[30350]: scrub summary: idle 2024-09-10T09:49:58.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:49:57 smithi005 ceph-mon[30350]: scrub summary: idle 2024-09-10T09:49:58.126 INFO:tasks.workunit.client.0.smithi005.stdout: 1 714359 6.17 MB/sec execute 459 sec latency 24.614 ms 2024-09-10T09:49:59.126 INFO:tasks.workunit.client.0.smithi005.stdout: 1 717085 6.19 MB/sec execute 460 sec latency 11.343 ms 2024-09-10T09:49:59.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:49:59 smithi159 ceph-mon[35513]: pgmap v742: 129 pgs: 129 active+clean; 259 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 1.8 MiB/s wr, 202 op/s 2024-09-10T09:49:59.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:49:59 smithi102 ceph-mon[35442]: pgmap v742: 129 pgs: 129 active+clean; 259 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 1.8 MiB/s wr, 202 op/s 2024-09-10T09:50:00.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:49:59 smithi005 ceph-mon[30350]: pgmap v742: 129 pgs: 129 active+clean; 259 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 1.8 MiB/s wr, 202 op/s 2024-09-10T09:50:00.126 INFO:tasks.workunit.client.0.smithi005.stdout: 1 718871 6.19 MB/sec execute 461 sec latency 23.763 ms 2024-09-10T09:50:00.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:50:00 smithi159 ceph-mon[35513]: overall HEALTH_OK 2024-09-10T09:50:00.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:50:00 smithi102 ceph-mon[35442]: overall HEALTH_OK 2024-09-10T09:50:01.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:50:00 smithi005 ceph-mon[30350]: overall HEALTH_OK 2024-09-10T09:50:01.127 INFO:tasks.workunit.client.0.smithi005.stdout: 1 721290 6.21 MB/sec execute 462 sec latency 10.096 ms 2024-09-10T09:50:01.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:50:01 smithi102 ceph-mon[35442]: pgmap v743: 129 pgs: 129 active+clean; 263 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 1.7 MiB/s wr, 224 op/s 2024-09-10T09:50:02.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:50:01 smithi005 ceph-mon[30350]: pgmap v743: 129 pgs: 129 active+clean; 263 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 1.7 MiB/s wr, 224 op/s 2024-09-10T09:50:02.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:50:01 smithi159 ceph-mon[35513]: pgmap v743: 129 pgs: 129 active+clean; 263 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 1.7 MiB/s wr, 224 op/s 2024-09-10T09:50:02.127 INFO:tasks.workunit.client.0.smithi005.stdout: 1 723446 6.22 MB/sec execute 463 sec latency 24.706 ms 2024-09-10T09:50:03.127 INFO:tasks.workunit.client.0.smithi005.stdout: 1 725441 6.23 MB/sec execute 464 sec latency 39.948 ms 2024-09-10T09:50:03.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:50:03 smithi102 ceph-mon[35442]: pgmap v744: 129 pgs: 129 active+clean; 264 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 1.5 MiB/s wr, 202 op/s 2024-09-10T09:50:04.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:50:03 smithi005 ceph-mon[30350]: pgmap v744: 129 pgs: 129 active+clean; 264 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 1.5 MiB/s wr, 202 op/s 2024-09-10T09:50:04.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:50:03 smithi159 ceph-mon[35513]: pgmap v744: 129 pgs: 129 active+clean; 264 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 1.5 MiB/s wr, 202 op/s 2024-09-10T09:50:04.127 INFO:tasks.workunit.client.0.smithi005.stdout: 1 728191 6.25 MB/sec execute 465 sec latency 11.631 ms 2024-09-10T09:50:05.127 INFO:tasks.workunit.client.0.smithi005.stdout: 1 730283 6.26 MB/sec execute 466 sec latency 22.893 ms 2024-09-10T09:50:05.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:50:05 smithi102 ceph-mon[35442]: pgmap v745: 129 pgs: 129 active+clean; 265 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 1.2 MiB/s wr, 182 op/s 2024-09-10T09:50:06.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:50:05 smithi005 ceph-mon[30350]: pgmap v745: 129 pgs: 129 active+clean; 265 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 1.2 MiB/s wr, 182 op/s 2024-09-10T09:50:06.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:50:05 smithi159 ceph-mon[35513]: pgmap v745: 129 pgs: 129 active+clean; 265 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 1.2 MiB/s wr, 182 op/s 2024-09-10T09:50:06.127 INFO:tasks.workunit.client.0.smithi005.stdout: 1 732287 6.27 MB/sec execute 467 sec latency 24.806 ms 2024-09-10T09:50:07.127 INFO:tasks.workunit.client.0.smithi005.stdout: 1 735046 6.29 MB/sec execute 468 sec latency 11.821 ms 2024-09-10T09:50:07.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:50:07 smithi102 ceph-mon[35442]: pgmap v746: 129 pgs: 129 active+clean; 269 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 1.5 MiB/s wr, 242 op/s 2024-09-10T09:50:08.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:50:07 smithi005 ceph-mon[30350]: pgmap v746: 129 pgs: 129 active+clean; 269 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 1.5 MiB/s wr, 242 op/s 2024-09-10T09:50:08.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:50:07 smithi159 ceph-mon[35513]: pgmap v746: 129 pgs: 129 active+clean; 269 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 1.5 MiB/s wr, 242 op/s 2024-09-10T09:50:08.127 INFO:tasks.workunit.client.0.smithi005.stdout: 1 736801 6.29 MB/sec execute 469 sec latency 21.586 ms 2024-09-10T09:50:09.127 INFO:tasks.workunit.client.0.smithi005.stdout: 1 739178 6.31 MB/sec execute 470 sec latency 24.182 ms 2024-09-10T09:50:09.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:50:09 smithi102 ceph-mon[35442]: pgmap v747: 129 pgs: 129 active+clean; 270 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 1.3 MiB/s wr, 199 op/s 2024-09-10T09:50:10.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:50:09 smithi005 ceph-mon[30350]: pgmap v747: 129 pgs: 129 active+clean; 270 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 1.3 MiB/s wr, 199 op/s 2024-09-10T09:50:10.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:50:09 smithi159 ceph-mon[35513]: pgmap v747: 129 pgs: 129 active+clean; 270 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 1.3 MiB/s wr, 199 op/s 2024-09-10T09:50:10.127 INFO:tasks.workunit.client.0.smithi005.stdout: 1 741514 6.32 MB/sec execute 471 sec latency 46.658 ms 2024-09-10T09:50:11.127 INFO:tasks.workunit.client.0.smithi005.stdout: 1 743279 6.33 MB/sec execute 472 sec latency 25.379 ms 2024-09-10T09:50:11.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:50:11 smithi102 ceph-mon[35442]: pgmap v748: 129 pgs: 129 active+clean; 274 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 1.5 MiB/s wr, 249 op/s 2024-09-10T09:50:12.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:50:11 smithi005 ceph-mon[30350]: pgmap v748: 129 pgs: 129 active+clean; 274 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 1.5 MiB/s wr, 249 op/s 2024-09-10T09:50:12.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:50:11 smithi159 ceph-mon[35513]: pgmap v748: 129 pgs: 129 active+clean; 274 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 1.5 MiB/s wr, 249 op/s 2024-09-10T09:50:12.127 INFO:tasks.workunit.client.0.smithi005.stdout: 1 745917 6.35 MB/sec execute 473 sec latency 13.197 ms 2024-09-10T09:50:13.127 INFO:tasks.workunit.client.0.smithi005.stdout: 1 747655 6.35 MB/sec execute 474 sec latency 30.884 ms 2024-09-10T09:50:13.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:50:13 smithi102 ceph-mon[35442]: pgmap v749: 129 pgs: 129 active+clean; 274 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 1.3 MiB/s wr, 200 op/s 2024-09-10T09:50:14.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:50:13 smithi005 ceph-mon[30350]: pgmap v749: 129 pgs: 129 active+clean; 274 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 1.3 MiB/s wr, 200 op/s 2024-09-10T09:50:14.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:50:13 smithi159 ceph-mon[35513]: pgmap v749: 129 pgs: 129 active+clean; 274 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 1.3 MiB/s wr, 200 op/s 2024-09-10T09:50:14.127 INFO:tasks.workunit.client.0.smithi005.stdout: 1 749831 6.36 MB/sec execute 475 sec latency 23.097 ms 2024-09-10T09:50:15.127 INFO:tasks.workunit.client.0.smithi005.stdout: 1 752413 6.38 MB/sec execute 476 sec latency 12.378 ms 2024-09-10T09:50:15.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:50:15 smithi102 ceph-mon[35442]: pgmap v750: 129 pgs: 129 active+clean; 277 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 1.4 MiB/s wr, 226 op/s 2024-09-10T09:50:16.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:50:15 smithi005 ceph-mon[30350]: pgmap v750: 129 pgs: 129 active+clean; 277 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 1.4 MiB/s wr, 226 op/s 2024-09-10T09:50:16.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:50:15 smithi159 ceph-mon[35513]: pgmap v750: 129 pgs: 129 active+clean; 277 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 1.4 MiB/s wr, 226 op/s 2024-09-10T09:50:16.127 INFO:tasks.workunit.client.0.smithi005.stdout: 1 754171 6.39 MB/sec execute 477 sec latency 25.439 ms 2024-09-10T09:50:17.128 INFO:tasks.workunit.client.0.smithi005.stdout: 1 756687 6.40 MB/sec execute 478 sec latency 9.535 ms 2024-09-10T09:50:17.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:50:17 smithi102 ceph-mon[35442]: pgmap v751: 129 pgs: 129 active+clean; 278 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 1.4 MiB/s wr, 239 op/s 2024-09-10T09:50:18.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:50:17 smithi005 ceph-mon[30350]: pgmap v751: 129 pgs: 129 active+clean; 278 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 1.4 MiB/s wr, 239 op/s 2024-09-10T09:50:18.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:50:17 smithi159 ceph-mon[35513]: pgmap v751: 129 pgs: 129 active+clean; 278 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 1.4 MiB/s wr, 239 op/s 2024-09-10T09:50:18.128 INFO:tasks.workunit.client.0.smithi005.stdout: 1 758797 6.41 MB/sec execute 479 sec latency 22.060 ms 2024-09-10T09:50:18.933 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T09:50:19.134 INFO:tasks.workunit.client.0.smithi005.stdout: 1 758803 6.39 MB/sec execute 480 sec latency 850.035 ms 2024-09-10T09:50:19.338 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:50:19.766 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:50:19 smithi005 ceph-mon[30350]: pgmap v752: 129 pgs: 129 active+clean; 279 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 1.1 MiB/s wr, 187 op/s 2024-09-10T09:50:19.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:50:19 smithi102 ceph-mon[35442]: pgmap v752: 129 pgs: 129 active+clean; 279 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 1.1 MiB/s wr, 187 op/s 2024-09-10T09:50:20.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:20.049+0000 7f652f200640 1 -- 172.21.15.5:0/3544493956 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f653010e280 msgr2=0x7f6530110670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:50:20.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:20.049+0000 7f652f200640 1 --2- 172.21.15.5:0/3544493956 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f653010e280 0x7f6530110670 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f6528009ca0 tx=0x7f652802a940 comp rx=0 tx=0).stop 2024-09-10T09:50:20.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:20.049+0000 7f652f200640 1 -- 172.21.15.5:0/3544493956 shutdown_connections 2024-09-10T09:50:20.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:20.049+0000 7f652f200640 1 --2- 172.21.15.5:0/3544493956 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f653010e280 0x7f6530110670 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:20.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:20.049+0000 7f652f200640 1 --2- 172.21.15.5:0/3544493956 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6530074e30 0x7f653010dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:20.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:20.049+0000 7f652f200640 1 --2- 172.21.15.5:0/3544493956 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6530074510 0x7f65300748f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:20.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:20.049+0000 7f652f200640 1 -- 172.21.15.5:0/3544493956 >> 172.21.15.5:0/3544493956 conn(0x7f65300fd740 msgr2=0x7f65300ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:50:20.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:20.049+0000 7f652f200640 1 -- 172.21.15.5:0/3544493956 shutdown_connections 2024-09-10T09:50:20.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:20.049+0000 7f652f200640 1 -- 172.21.15.5:0/3544493956 wait complete. 2024-09-10T09:50:20.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:20.050+0000 7f652f200640 1 Processor -- start 2024-09-10T09:50:20.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:20.051+0000 7f652f200640 1 -- start start 2024-09-10T09:50:20.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:20.051+0000 7f652f200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6530074510 0x7f653019a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:50:20.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:20.051+0000 7f652f200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6530074e30 0x7f653019a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:50:20.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:20.051+0000 7f652f200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f653010e280 0x7f653019ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:50:20.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:20.051+0000 7f652f200640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f6530113fc0 con 0x7f653010e280 2024-09-10T09:50:20.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:20.051+0000 7f652f200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f6530113e40 con 0x7f6530074510 2024-09-10T09:50:20.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:20.051+0000 7f652f200640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f6530114140 con 0x7f6530074e30 2024-09-10T09:50:20.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:20.051+0000 7f652de00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6530074510 0x7f653019a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:50:20.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:20.051+0000 7f652d400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6530074e30 0x7f653019a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:50:20.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:20.051+0000 7f652de00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6530074510 0x7f653019a480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:55918/0 (socket says 172.21.15.5:55918) 2024-09-10T09:50:20.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:20.051+0000 7f652de00640 1 -- 172.21.15.5:0/1281801529 learned_addr learned my addr 172.21.15.5:0/1281801529 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:50:20.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:20.051+0000 7f652e800640 1 --2- 172.21.15.5:0/1281801529 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f653010e280 0x7f653019ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:50:20.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:20.052+0000 7f652de00640 1 -- 172.21.15.5:0/1281801529 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6530074e30 msgr2=0x7f653019a9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:50:20.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:20.052+0000 7f652de00640 1 --2- 172.21.15.5:0/1281801529 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6530074e30 0x7f653019a9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:20.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:20.052+0000 7f652de00640 1 -- 172.21.15.5:0/1281801529 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f653010e280 msgr2=0x7f653019ed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:50:20.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:20.052+0000 7f652de00640 1 --2- 172.21.15.5:0/1281801529 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f653010e280 0x7f653019ed50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:20.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:20.052+0000 7f652de00640 1 -- 172.21.15.5:0/1281801529 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f653019f430 con 0x7f6530074510 2024-09-10T09:50:20.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:20.052+0000 7f652e800640 1 --2- 172.21.15.5:0/1281801529 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f653010e280 0x7f653019ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:50:20.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:20.052+0000 7f652de00640 1 --2- 172.21.15.5:0/1281801529 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6530074510 0x7f653019a480 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f652000ea60 tx=0x7f652000ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:50:20.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:20.052+0000 7f6526a00640 1 -- 172.21.15.5:0/1281801529 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f652000cde0 con 0x7f6530074510 2024-09-10T09:50:20.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:20.052+0000 7f6526a00640 1 -- 172.21.15.5:0/1281801529 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f65200035f0 con 0x7f6530074510 2024-09-10T09:50:20.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:20.052+0000 7f652f200640 1 -- 172.21.15.5:0/1281801529 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f653019f720 con 0x7f6530074510 2024-09-10T09:50:20.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:20.052+0000 7f6526a00640 1 -- 172.21.15.5:0/1281801529 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6520010670 con 0x7f6530074510 2024-09-10T09:50:20.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:20.053+0000 7f652f200640 1 -- 172.21.15.5:0/1281801529 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65301a7000 con 0x7f6530074510 2024-09-10T09:50:20.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:20.056+0000 7f6526a00640 1 -- 172.21.15.5:0/1281801529 <== mon.2 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f65200037a0 con 0x7f6530074510 2024-09-10T09:50:20.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:20.056+0000 7f65375fc740 1 Processor -- start 2024-09-10T09:50:20.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:20.056+0000 7f65375fc740 1 -- start start 2024-09-10T09:50:20.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:20.056+0000 7f65375fc740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558ee71cfd50 0x558ee71d0130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:50:20.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:20.056+0000 7f65375fc740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558ee71d0670 0x558ee71da9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:50:20.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:20.056+0000 7f65375fc740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558ee71daf30 0x558ee71dd320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:50:20.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:20.056+0000 7f65375fc740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x558ee71363e0 con 0x558ee71d0670 2024-09-10T09:50:20.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:20.056+0000 7f65375fc740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x558ee70ddd80 con 0x558ee71cfd50 2024-09-10T09:50:20.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:20.056+0000 7f65375fc740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x558ee71323a0 con 0x558ee71daf30 2024-09-10T09:50:20.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:20.057+0000 7f652d400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558ee71daf30 0x558ee71dd320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:50:20.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:20.057+0000 7f652d400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558ee71daf30 0x558ee71dd320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:49394/0 (socket says 172.21.15.5:49394) 2024-09-10T09:50:20.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:20.057+0000 7f652d400640 1 -- 172.21.15.5:0/1378322830 learned_addr learned my addr 172.21.15.5:0/1378322830 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:50:20.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:20.057+0000 7f652de00640 1 --2- 172.21.15.5:0/1378322830 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558ee71d0670 0x558ee71da9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:50:20.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:20.057+0000 7f652e800640 1 --2- 172.21.15.5:0/1378322830 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558ee71cfd50 0x558ee71d0130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:50:20.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:20.057+0000 7f652d400640 1 -- 172.21.15.5:0/1378322830 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558ee71cfd50 msgr2=0x558ee71d0130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:50:20.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:20.057+0000 7f652d400640 1 --2- 172.21.15.5:0/1378322830 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558ee71cfd50 0x558ee71d0130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:20.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:20.057+0000 7f652d400640 1 -- 172.21.15.5:0/1378322830 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558ee71d0670 msgr2=0x558ee71da9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:50:20.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:20.057+0000 7f652d400640 1 --2- 172.21.15.5:0/1378322830 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558ee71d0670 0x558ee71da9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:20.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:20.057+0000 7f652d400640 1 -- 172.21.15.5:0/1378322830 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558ee70fcae0 con 0x558ee71daf30 2024-09-10T09:50:20.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:20.057+0000 7f652de00640 1 --2- 172.21.15.5:0/1378322830 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558ee71d0670 0x558ee71da9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:50:20.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:20.057+0000 7f652d400640 1 --2- 172.21.15.5:0/1378322830 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558ee71daf30 0x558ee71dd320 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f651c009c10 tx=0x7f651c011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9bfbcaa7ef7ef81b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:50:20.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:20.057+0000 7f6526a00640 1 --2- 172.21.15.5:0/1281801529 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f65140a2b80 0x7f65140a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:50:20.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:20.058+0000 7f651b000640 1 -- 172.21.15.5:0/1378322830 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f651c004230 con 0x558ee71daf30 2024-09-10T09:50:20.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:20.058+0000 7f6526a00640 1 -- 172.21.15.5:0/1281801529 <== mon.2 v2:172.21.15.102:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f65200cf7f0 con 0x7f6530074510 2024-09-10T09:50:20.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:20.058+0000 7f651b000640 1 -- 172.21.15.5:0/1378322830 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f651c0043d0 con 0x558ee71daf30 2024-09-10T09:50:20.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:20.058+0000 7f651b000640 1 -- 172.21.15.5:0/1378322830 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f651c0197e0 con 0x558ee71daf30 2024-09-10T09:50:20.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:20.058+0000 7f652e800640 1 --2- 172.21.15.5:0/1281801529 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f65140a2b80 0x7f65140a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:50:20.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:20.059+0000 7f65375fc740 1 -- 172.21.15.5:0/1378322830 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558ee71daf30 msgr2=0x558ee71dd320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:50:20.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:20.059+0000 7f65375fc740 1 --2- 172.21.15.5:0/1378322830 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558ee71daf30 0x558ee71dd320 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f651c009c10 tx=0x7f651c011810 comp rx=0 tx=0).stop 2024-09-10T09:50:20.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:20.059+0000 7f652e800640 1 --2- 172.21.15.5:0/1281801529 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f65140a2b80 0x7f65140a5040 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f6528005e30 tx=0x7f6528005dc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:50:20.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:20.059+0000 7f65375fc740 1 -- 172.21.15.5:0/1378322830 shutdown_connections 2024-09-10T09:50:20.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:20.059+0000 7f65375fc740 1 --2- 172.21.15.5:0/1378322830 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558ee71daf30 0x558ee71dd320 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:20.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:20.059+0000 7f65375fc740 1 --2- 172.21.15.5:0/1378322830 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558ee71d0670 0x558ee71da9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:20.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:20.059+0000 7f65375fc740 1 --2- 172.21.15.5:0/1378322830 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558ee71cfd50 0x558ee71d0130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:20.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:20.059+0000 7f65375fc740 1 -- 172.21.15.5:0/1378322830 >> 172.21.15.5:0/1378322830 conn(0x558ee71cc2e0 msgr2=0x558ee71cc6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:50:20.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:20.059+0000 7f65375fc740 1 -- 172.21.15.5:0/1378322830 shutdown_connections 2024-09-10T09:50:20.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:20.059+0000 7f65375fc740 1 -- 172.21.15.5:0/1378322830 wait complete. 2024-09-10T09:50:20.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:20.059+0000 7f65375fc740 1 -- start start 2024-09-10T09:50:20.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:20.060+0000 7f65375fc740 1 Processor -- start 2024-09-10T09:50:20.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:20.060+0000 7f65375fc740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558ee71cfd50 0x558ee71d7e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:50:20.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:20.060+0000 7f65375fc740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558ee71d0670 0x558ee71d83a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:50:20.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:20.060+0000 7f65375fc740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558ee71daf30 0x558ee71d88e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:50:20.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:20.060+0000 7f65375fc740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x558ee71ddba0 con 0x558ee71daf30 2024-09-10T09:50:20.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:20.060+0000 7f65375fc740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x558ee70f10f0 con 0x558ee71d0670 2024-09-10T09:50:20.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:20.060+0000 7f65375fc740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x558ee71ddd20 con 0x558ee71cfd50 2024-09-10T09:50:20.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:20.060+0000 7f65375fc740 10 client.0 _ll_register_callbacks cb 0x558ee70e29e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:50:20.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:20.060+0000 7f652de00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558ee71cfd50 0x558ee71d7e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:50:20.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:20.060+0000 7f652de00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558ee71cfd50 0x558ee71d7e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:49400/0 (socket says 172.21.15.5:49400) 2024-09-10T09:50:20.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:20.060+0000 7f652de00640 1 -- 172.21.15.5:0/1020589180 learned_addr learned my addr 172.21.15.5:0/1020589180 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:50:20.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:20.060+0000 7f652d400640 1 --2- 172.21.15.5:0/1020589180 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558ee71d0670 0x558ee71d83a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:50:20.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:20.061+0000 7f652e800640 1 --2- 172.21.15.5:0/1020589180 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558ee71daf30 0x558ee71d88e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:50:20.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:20.061+0000 7f652de00640 1 -- 172.21.15.5:0/1020589180 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558ee71d0670 msgr2=0x558ee71d83a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:50:20.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:20.061+0000 7f652de00640 1 --2- 172.21.15.5:0/1020589180 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558ee71d0670 0x558ee71d83a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:20.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:20.061+0000 7f652de00640 1 -- 172.21.15.5:0/1020589180 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558ee71daf30 msgr2=0x558ee71d88e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:50:20.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:20.061+0000 7f652de00640 1 --2- 172.21.15.5:0/1020589180 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558ee71daf30 0x558ee71d88e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:20.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:20.061+0000 7f652de00640 1 -- 172.21.15.5:0/1020589180 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558ee6ea73c0 con 0x558ee71cfd50 2024-09-10T09:50:20.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:20.061+0000 7f652de00640 1 --2- 172.21.15.5:0/1020589180 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558ee71cfd50 0x558ee71d7e60 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f652009bb90 tx=0x7f6520097fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9a48a3a0c4bd1957 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:50:20.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:20.061+0000 7f650f400640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-09-10T09:50:20.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:20.061+0000 7f650f400640 1 -- 172.21.15.5:0/1020589180 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65200b2060 con 0x558ee71cfd50 2024-09-10T09:50:20.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:20.061+0000 7f650f400640 1 -- 172.21.15.5:0/1020589180 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f6520098270 con 0x558ee71cfd50 2024-09-10T09:50:20.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:20.061+0000 7f65375fc740 1 -- 172.21.15.5:0/1020589180 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f651c0043d0 con 0x558ee71cfd50 2024-09-10T09:50:20.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:20.062+0000 7f650f400640 1 -- 172.21.15.5:0/1020589180 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f65200b9050 con 0x558ee71cfd50 2024-09-10T09:50:20.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:20.062+0000 7f65375fc740 10 client.27766 fetch_fsmap learned FSMap version 36 2024-09-10T09:50:20.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:20.062+0000 7f65375fc740 1 -- 172.21.15.5:0/1020589180 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x558ee71e9900 con 0x558ee71cfd50 2024-09-10T09:50:20.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:20.062+0000 7f650f400640 1 -- 172.21.15.5:0/1020589180 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f651c0043d0 con 0x558ee71cfd50 2024-09-10T09:50:20.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:20.062+0000 7f650f400640 1 -- 172.21.15.5:0/1020589180 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 36) ==== 8370+0+0 (secure 0 0 0) 0x7f65200cfbe0 con 0x558ee71cfd50 2024-09-10T09:50:20.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:20.062+0000 7f65375fc740 10 client.27766 fetch_fsmap finished waiting for FSMap version 36 2024-09-10T09:50:20.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:20.062+0000 7f65375fc740 10 client.27766 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T09:50:20.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:20.062+0000 7f65375fc740 20 client.27766 populate_metadata read hostname 'smithi005' 2024-09-10T09:50:20.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:20.062+0000 7f65375fc740 1 --2- 172.21.15.5:0/1020589180 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x558ee71eae10 0x558ee71ed260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:50:20.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:20.062+0000 7f65375fc740 4 client.27766 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:50:20.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:20.062+0000 7f65375fc740 1 -- 172.21.15.5:0/1020589180 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558ee7143950 con 0x558ee71eae10 2024-09-10T09:50:20.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:20.063+0000 7f652d400640 1 --2- 172.21.15.5:0/1020589180 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x558ee71eae10 0x558ee71ed260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:50:20.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:20.063+0000 7f652d400640 1 --2- 172.21.15.5:0/1020589180 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x558ee71eae10 0x558ee71ed260 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ab78cfb663392b55 server_cookie=9159c0827176cddc in_seq=0 out_seq=0 2024-09-10T09:50:20.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:20.063+0000 7f650f400640 10 client.27766 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T09:50:20.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:20.064+0000 7f650f400640 1 -- 172.21.15.5:0/1020589180 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558ee7143950 con 0x558ee71eae10 2024-09-10T09:50:20.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:20.064+0000 7f650f400640 10 client.27766 handle_command_reply: tid=0 2024-09-10T09:50:20.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:20.064+0000 7f65375fc740 1 client.27766 shutdown 2024-09-10T09:50:20.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:20.064+0000 7f65375fc740 1 -- 172.21.15.5:0/1020589180 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558ee71cfd50 msgr2=0x558ee71d7e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:50:20.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:20.064+0000 7f65375fc740 1 --2- 172.21.15.5:0/1020589180 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558ee71cfd50 0x558ee71d7e60 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f652009bb90 tx=0x7f6520097fd0 comp rx=0 tx=0).stop 2024-09-10T09:50:20.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:20.064+0000 7f65375fc740 1 -- 172.21.15.5:0/1020589180 shutdown_connections 2024-09-10T09:50:20.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:20.064+0000 7f65375fc740 1 --2- 172.21.15.5:0/1020589180 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x558ee71eae10 0x558ee71ed260 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:20.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:20.064+0000 7f65375fc740 1 --2- 172.21.15.5:0/1020589180 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558ee71daf30 0x558ee71d88e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:20.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:20.064+0000 7f65375fc740 1 --2- 172.21.15.5:0/1020589180 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558ee71d0670 0x558ee71d83a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:20.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:20.064+0000 7f65375fc740 1 --2- 172.21.15.5:0/1020589180 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558ee71cfd50 0x558ee71d7e60 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:20.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:20.064+0000 7f65375fc740 1 -- 172.21.15.5:0/1020589180 >> 172.21.15.5:0/1020589180 conn(0x558ee71cc2e0 msgr2=0x558ee71d2010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:50:20.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:20.064+0000 7f650f400640 0 client.27766 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T09:50:20.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:20.064+0000 7f65375fc740 1 -- 172.21.15.5:0/1020589180 shutdown_connections 2024-09-10T09:50:20.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:20.064+0000 7f65375fc740 1 -- 172.21.15.5:0/1020589180 wait complete. 2024-09-10T09:50:20.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:20.065+0000 7f65375fc740 20 client.27766 trim_cache size 0 max 16384 2024-09-10T09:50:20.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:20.084+0000 7f65375fc740 1 Processor -- start 2024-09-10T09:50:20.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:20.085+0000 7f65375fc740 1 -- start start 2024-09-10T09:50:20.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:20.085+0000 7f65375fc740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558ee71cfd50 0x558ee71d27b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:50:20.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:20.085+0000 7f65375fc740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558ee71d0670 0x558ee71d9dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:50:20.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:20.085+0000 7f65375fc740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558ee71daf30 0x558ee71f2b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:50:20.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:20.085+0000 7f65375fc740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f65301b0fd0 con 0x558ee71cfd50 2024-09-10T09:50:20.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:20.085+0000 7f65375fc740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f65301b2b10 con 0x558ee71d0670 2024-09-10T09:50:20.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:20.085+0000 7f65375fc740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f65301b16a0 con 0x558ee71daf30 2024-09-10T09:50:20.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:20.085+0000 7f652d400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558ee71d0670 0x558ee71d9dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:50:20.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:20.085+0000 7f652e800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558ee71daf30 0x558ee71f2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:50:20.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:20.085+0000 7f652d400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558ee71d0670 0x558ee71d9dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:55940/0 (socket says 172.21.15.5:55940) 2024-09-10T09:50:20.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:20.085+0000 7f652d400640 1 -- 172.21.15.5:0/779642649 learned_addr learned my addr 172.21.15.5:0/779642649 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:50:20.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:20.085+0000 7f652de00640 1 --2- 172.21.15.5:0/779642649 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558ee71cfd50 0x558ee71d27b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:50:20.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:20.086+0000 7f652d400640 1 -- 172.21.15.5:0/779642649 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558ee71daf30 msgr2=0x558ee71f2b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:50:20.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:20.086+0000 7f652d400640 1 --2- 172.21.15.5:0/779642649 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558ee71daf30 0x558ee71f2b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:20.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:20.086+0000 7f652d400640 1 -- 172.21.15.5:0/779642649 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558ee71cfd50 msgr2=0x558ee71d27b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:50:20.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:20.086+0000 7f652d400640 1 --2- 172.21.15.5:0/779642649 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558ee71cfd50 0x558ee71d27b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:20.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:20.086+0000 7f652d400640 1 -- 172.21.15.5:0/779642649 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558ee71cfb60 con 0x558ee71d0670 2024-09-10T09:50:20.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:20.086+0000 7f652d400640 1 --2- 172.21.15.5:0/779642649 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558ee71d0670 0x558ee71d9dd0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f651c009c10 tx=0x7f651c005850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4131b6e248f5cccc server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:50:20.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:20.086+0000 7f651a600640 1 -- 172.21.15.5:0/779642649 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f651c01f040 con 0x558ee71d0670 2024-09-10T09:50:20.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:20.086+0000 7f651a600640 1 -- 172.21.15.5:0/779642649 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f651c005070 con 0x558ee71d0670 2024-09-10T09:50:20.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:20.086+0000 7f651a600640 1 -- 172.21.15.5:0/779642649 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f651c01a7c0 con 0x558ee71d0670 2024-09-10T09:50:20.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:20.087+0000 7f65375fc740 1 -- 172.21.15.5:0/779642649 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558ee71d0670 msgr2=0x558ee71d9dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:50:20.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:20.087+0000 7f65375fc740 1 --2- 172.21.15.5:0/779642649 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558ee71d0670 0x558ee71d9dd0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f651c009c10 tx=0x7f651c005850 comp rx=0 tx=0).stop 2024-09-10T09:50:20.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:20.087+0000 7f65375fc740 1 -- 172.21.15.5:0/779642649 shutdown_connections 2024-09-10T09:50:20.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:20.087+0000 7f65375fc740 1 --2- 172.21.15.5:0/779642649 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558ee71daf30 0x558ee71f2b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:20.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:20.087+0000 7f65375fc740 1 --2- 172.21.15.5:0/779642649 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558ee71d0670 0x558ee71d9dd0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:20.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:20.087+0000 7f65375fc740 1 --2- 172.21.15.5:0/779642649 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558ee71cfd50 0x558ee71d27b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:20.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:20.087+0000 7f65375fc740 1 -- 172.21.15.5:0/779642649 >> 172.21.15.5:0/779642649 conn(0x558ee71cc2e0 msgr2=0x558ee71d96c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:50:20.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:20.087+0000 7f65375fc740 1 -- 172.21.15.5:0/779642649 shutdown_connections 2024-09-10T09:50:20.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:20.087+0000 7f65375fc740 1 -- 172.21.15.5:0/779642649 wait complete. 2024-09-10T09:50:20.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:20.088+0000 7f65375fc740 1 -- start start 2024-09-10T09:50:20.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:20.088+0000 7f65375fc740 1 Processor -- start 2024-09-10T09:50:20.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:20.088+0000 7f65375fc740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558ee71cfd50 0x558ee71e3370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:50:20.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:20.088+0000 7f65375fc740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558ee71d0670 0x558ee71d4850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:50:20.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:20.088+0000 7f65375fc740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558ee71daf30 0x558ee71d4d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:50:20.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:20.088+0000 7f65375fc740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f65301b2440 con 0x558ee71cfd50 2024-09-10T09:50:20.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:20.088+0000 7f65375fc740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f65301b1d70 con 0x558ee71daf30 2024-09-10T09:50:20.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:20.088+0000 7f65375fc740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f65301aefb0 con 0x558ee71d0670 2024-09-10T09:50:20.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:20.088+0000 7f65375fc740 10 client.0 _ll_register_callbacks cb 0x7f6520097780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:50:20.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:20.088+0000 7f652d400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558ee71d0670 0x558ee71d4850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:50:20.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:20.088+0000 7f652d400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558ee71d0670 0x558ee71d4850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:49426/0 (socket says 172.21.15.5:49426) 2024-09-10T09:50:20.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:20.088+0000 7f652d400640 1 -- 172.21.15.5:0/3040828063 learned_addr learned my addr 172.21.15.5:0/3040828063 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:50:20.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:20.089+0000 7f652de00640 1 --2- 172.21.15.5:0/3040828063 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558ee71cfd50 0x558ee71e3370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:50:20.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:20.089+0000 7f652e800640 1 --2- 172.21.15.5:0/3040828063 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558ee71daf30 0x558ee71d4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:50:20.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:20.089+0000 7f652d400640 1 -- 172.21.15.5:0/3040828063 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558ee71daf30 msgr2=0x558ee71d4d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:50:20.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:20.089+0000 7f652d400640 1 --2- 172.21.15.5:0/3040828063 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558ee71daf30 0x558ee71d4d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:20.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:20.089+0000 7f652d400640 1 -- 172.21.15.5:0/3040828063 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558ee71cfd50 msgr2=0x558ee71e3370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:50:20.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:20.089+0000 7f652d400640 1 --2- 172.21.15.5:0/3040828063 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558ee71cfd50 0x558ee71e3370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:20.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:20.089+0000 7f652d400640 1 -- 172.21.15.5:0/3040828063 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558ee71d52d0 con 0x558ee71d0670 2024-09-10T09:50:20.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:20.089+0000 7f652de00640 1 --2- 172.21.15.5:0/3040828063 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558ee71cfd50 0x558ee71e3370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:50:20.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:20.089+0000 7f652d400640 1 --2- 172.21.15.5:0/3040828063 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558ee71d0670 0x558ee71d4850 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f651c009be0 tx=0x7f651c005220 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4c3367e2550643bc server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:50:20.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:20.089+0000 7f650f400640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-09-10T09:50:20.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:20.089+0000 7f650f400640 1 -- 172.21.15.5:0/3040828063 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f651c0035f0 con 0x558ee71d0670 2024-09-10T09:50:20.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:20.089+0000 7f65375fc740 1 -- 172.21.15.5:0/3040828063 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f651c005070 con 0x558ee71d0670 2024-09-10T09:50:20.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:20.090+0000 7f650f400640 1 -- 172.21.15.5:0/3040828063 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f651c003790 con 0x558ee71d0670 2024-09-10T09:50:20.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:20.090+0000 7f650f400640 1 -- 172.21.15.5:0/3040828063 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f651c003990 con 0x558ee71d0670 2024-09-10T09:50:20.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:20.090+0000 7f65375fc740 10 client.27772 fetch_fsmap learned FSMap version 36 2024-09-10T09:50:20.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:20.090+0000 7f65375fc740 1 -- 172.21.15.5:0/3040828063 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x558ee7255770 con 0x558ee71d0670 2024-09-10T09:50:20.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:20.090+0000 7f650f400640 1 -- 172.21.15.5:0/3040828063 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f651c005070 con 0x558ee71d0670 2024-09-10T09:50:20.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:20.090+0000 7f650f400640 1 -- 172.21.15.5:0/3040828063 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 36) ==== 8370+0+0 (secure 0 0 0) 0x7f651c018070 con 0x558ee71d0670 2024-09-10T09:50:20.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:20.090+0000 7f65375fc740 10 client.27772 fetch_fsmap finished waiting for FSMap version 36 2024-09-10T09:50:20.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:20.090+0000 7f65375fc740 10 client.27772 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T09:50:20.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:20.090+0000 7f65375fc740 20 client.27772 populate_metadata read hostname 'smithi005' 2024-09-10T09:50:20.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:20.090+0000 7f65375fc740 1 --2- 172.21.15.5:0/3040828063 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x558ee7255cf0 0x558ee7258110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:50:20.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:20.090+0000 7f65375fc740 4 client.27772 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T09:50:20.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:20.090+0000 7f65375fc740 1 -- 172.21.15.5:0/3040828063 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "scrub status"}) -- 0x558ee713de60 con 0x558ee7255cf0 2024-09-10T09:50:20.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:20.091+0000 7f652de00640 1 --2- 172.21.15.5:0/3040828063 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x558ee7255cf0 0x558ee7258110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:50:20.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:20.091+0000 7f652de00640 1 --2- 172.21.15.5:0/3040828063 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x558ee7255cf0 0x558ee7258110 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b4b09b589a8df79d server_cookie=500c1c90d61fcd9f in_seq=0 out_seq=0 2024-09-10T09:50:20.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:20.091+0000 7f650f400640 10 client.27772 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T09:50:20.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:20.091+0000 7f650f400640 1 -- 172.21.15.5:0/3040828063 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x558ee713de60 con 0x558ee7255cf0 2024-09-10T09:50:20.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:20.091+0000 7f650f400640 10 client.27772 handle_command_reply: tid=0 2024-09-10T09:50:20.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:20.091+0000 7f65375fc740 1 client.27772 shutdown 2024-09-10T09:50:20.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:20.092+0000 7f65375fc740 1 -- 172.21.15.5:0/3040828063 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558ee71d0670 msgr2=0x558ee71d4850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:50:20.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:20.092+0000 7f65375fc740 1 --2- 172.21.15.5:0/3040828063 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558ee71d0670 0x558ee71d4850 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f651c009be0 tx=0x7f651c005220 comp rx=0 tx=0).stop 2024-09-10T09:50:20.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:20.092+0000 7f65375fc740 1 -- 172.21.15.5:0/3040828063 shutdown_connections 2024-09-10T09:50:20.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:20.092+0000 7f65375fc740 1 --2- 172.21.15.5:0/3040828063 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x558ee7255cf0 0x558ee7258110 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:20.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:20.092+0000 7f65375fc740 1 --2- 172.21.15.5:0/3040828063 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558ee71daf30 0x558ee71d4d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:20.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:20.092+0000 7f65375fc740 1 --2- 172.21.15.5:0/3040828063 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558ee71d0670 0x558ee71d4850 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:20.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:20.092+0000 7f65375fc740 1 --2- 172.21.15.5:0/3040828063 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558ee71cfd50 0x558ee71e3370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:20.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:20.092+0000 7f65375fc740 1 -- 172.21.15.5:0/3040828063 >> 172.21.15.5:0/3040828063 conn(0x558ee71cc2e0 msgr2=0x558ee71cb380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:50:20.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:20.092+0000 7f650f400640 0 client.27772 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T09:50:20.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:20.092+0000 7f65375fc740 1 -- 172.21.15.5:0/3040828063 shutdown_connections 2024-09-10T09:50:20.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:20.092+0000 7f65375fc740 1 -- 172.21.15.5:0/3040828063 wait complete. 2024-09-10T09:50:20.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:20.092+0000 7f65375fc740 20 client.27772 trim_cache size 0 max 16384 2024-09-10T09:50:20.098 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-10T09:50:20.099 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "no active scrubs running", 2024-09-10T09:50:20.099 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": {} 2024-09-10T09:50:20.099 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-10T09:50:20.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:20.093+0000 7f652f200640 1 -- 172.21.15.5:0/1281801529 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f65140a2b80 msgr2=0x7f65140a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:50:20.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:20.093+0000 7f652f200640 1 --2- 172.21.15.5:0/1281801529 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f65140a2b80 0x7f65140a5040 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f6528005e30 tx=0x7f6528005dc0 comp rx=0 tx=0).stop 2024-09-10T09:50:20.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:20.093+0000 7f652f200640 1 -- 172.21.15.5:0/1281801529 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6530074510 msgr2=0x7f653019a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:50:20.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:20.093+0000 7f652f200640 1 --2- 172.21.15.5:0/1281801529 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6530074510 0x7f653019a480 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f652000ea60 tx=0x7f652000ef20 comp rx=0 tx=0).stop 2024-09-10T09:50:20.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:20.093+0000 7f652f200640 1 -- 172.21.15.5:0/1281801529 shutdown_connections 2024-09-10T09:50:20.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:20.093+0000 7f652f200640 1 --2- 172.21.15.5:0/1281801529 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f65140a2b80 0x7f65140a5040 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:20.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:20.093+0000 7f652f200640 1 --2- 172.21.15.5:0/1281801529 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f653010e280 0x7f653019ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:20.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:20.093+0000 7f652f200640 1 --2- 172.21.15.5:0/1281801529 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6530074e30 0x7f653019a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:20.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:20.093+0000 7f652f200640 1 --2- 172.21.15.5:0/1281801529 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6530074510 0x7f653019a480 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:20.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:20.093+0000 7f652f200640 1 -- 172.21.15.5:0/1281801529 >> 172.21.15.5:0/1281801529 conn(0x7f65300fd740 msgr2=0x7f65300fe440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:50:20.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:20.093+0000 7f652f200640 1 -- 172.21.15.5:0/1281801529 shutdown_connections 2024-09-10T09:50:20.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:20.093+0000 7f652f200640 1 -- 172.21.15.5:0/1281801529 wait complete. 2024-09-10T09:50:20.101 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:50:19 smithi159 ceph-mon[35513]: pgmap v752: 129 pgs: 129 active+clean; 279 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 1.1 MiB/s wr, 187 op/s 2024-09-10T09:50:20.134 INFO:tasks.workunit.client.0.smithi005.stdout: 1 760397 6.40 MB/sec execute 481 sec latency 404.003 ms 2024-09-10T09:50:21.011 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-10T09:50:21.011 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T09:50:21.134 INFO:tasks.workunit.client.0.smithi005.stdout: 1 762838 6.42 MB/sec execute 482 sec latency 22.701 ms 2024-09-10T09:50:21.436 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:50:21.862 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:50:21 smithi005 ceph-mon[30350]: pgmap v753: 129 pgs: 129 active+clean; 284 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 1.6 MiB/s wr, 223 op/s 2024-09-10T09:50:21.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:50:21 smithi102 ceph-mon[35442]: pgmap v753: 129 pgs: 129 active+clean; 284 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 1.6 MiB/s wr, 223 op/s 2024-09-10T09:50:22.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:22.018+0000 7fddbee00640 1 -- 172.21.15.5:0/3102641908 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fddb8103e30 msgr2=0x7fddb8104210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:50:22.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:22.018+0000 7fddbee00640 1 --2- 172.21.15.5:0/3102641908 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fddb8103e30 0x7fddb8104210 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fddac009900 tx=0x7fddac02a3f0 comp rx=0 tx=0).stop 2024-09-10T09:50:22.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:22.018+0000 7fddbee00640 1 -- 172.21.15.5:0/3102641908 shutdown_connections 2024-09-10T09:50:22.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:22.018+0000 7fddbee00640 1 --2- 172.21.15.5:0/3102641908 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fddb8108f60 0x7fddb8110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:22.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:22.018+0000 7fddbee00640 1 --2- 172.21.15.5:0/3102641908 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fddb81047e0 0x7fddb8108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:22.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:22.018+0000 7fddbee00640 1 --2- 172.21.15.5:0/3102641908 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fddb8103e30 0x7fddb8104210 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:22.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:22.018+0000 7fddbee00640 1 -- 172.21.15.5:0/3102641908 >> 172.21.15.5:0/3102641908 conn(0x7fddb80ff8d0 msgr2=0x7fddb8101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:50:22.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:22.018+0000 7fddbee00640 1 -- 172.21.15.5:0/3102641908 shutdown_connections 2024-09-10T09:50:22.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:22.018+0000 7fddbee00640 1 -- 172.21.15.5:0/3102641908 wait complete. 2024-09-10T09:50:22.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:22.019+0000 7fddbee00640 1 Processor -- start 2024-09-10T09:50:22.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:22.020+0000 7fddbee00640 1 -- start start 2024-09-10T09:50:22.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:22.020+0000 7fddbee00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fddb8103e30 0x7fddb819a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:50:22.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:22.020+0000 7fddbee00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fddb81047e0 0x7fddb819a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:50:22.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:22.020+0000 7fddbee00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fddb8108f60 0x7fddb819ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:50:22.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:22.020+0000 7fddbee00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fddb8114010 con 0x7fddb8108f60 2024-09-10T09:50:22.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:22.020+0000 7fddbee00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fddb8113e90 con 0x7fddb81047e0 2024-09-10T09:50:22.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:22.020+0000 7fddbee00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fddb8114190 con 0x7fddb8103e30 2024-09-10T09:50:22.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:22.020+0000 7fddbda00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fddb8103e30 0x7fddb819a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:50:22.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:22.020+0000 7fddbda00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fddb8103e30 0x7fddb819a4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:48576/0 (socket says 172.21.15.5:48576) 2024-09-10T09:50:22.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:22.020+0000 7fddbda00640 1 -- 172.21.15.5:0/2182961598 learned_addr learned my addr 172.21.15.5:0/2182961598 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:50:22.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:22.020+0000 7fddbd000640 1 --2- 172.21.15.5:0/2182961598 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fddb81047e0 0x7fddb819a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:50:22.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:22.020+0000 7fddbe400640 1 --2- 172.21.15.5:0/2182961598 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fddb8108f60 0x7fddb819ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:50:22.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:22.021+0000 7fddbda00640 1 -- 172.21.15.5:0/2182961598 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fddb81047e0 msgr2=0x7fddb819a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:50:22.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:22.021+0000 7fddbda00640 1 --2- 172.21.15.5:0/2182961598 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fddb81047e0 0x7fddb819a9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:22.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:22.021+0000 7fddbda00640 1 -- 172.21.15.5:0/2182961598 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fddb8108f60 msgr2=0x7fddb819ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:50:22.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:22.021+0000 7fddbda00640 1 --2- 172.21.15.5:0/2182961598 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fddb8108f60 0x7fddb819ed70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:22.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:22.021+0000 7fddbda00640 1 -- 172.21.15.5:0/2182961598 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fddb819f450 con 0x7fddb8103e30 2024-09-10T09:50:22.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:22.021+0000 7fddbd000640 1 --2- 172.21.15.5:0/2182961598 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fddb81047e0 0x7fddb819a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:50:22.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:22.021+0000 7fddbda00640 1 --2- 172.21.15.5:0/2182961598 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fddb8103e30 0x7fddb819a4a0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fddac004770 tx=0x7fddac003c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:50:22.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:22.021+0000 7fddb6000640 1 -- 172.21.15.5:0/2182961598 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddac03f070 con 0x7fddb8103e30 2024-09-10T09:50:22.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:22.021+0000 7fddbee00640 1 -- 172.21.15.5:0/2182961598 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fddb819f6e0 con 0x7fddb8103e30 2024-09-10T09:50:22.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:22.022+0000 7fddbee00640 1 -- 172.21.15.5:0/2182961598 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fddb81a7020 con 0x7fddb8103e30 2024-09-10T09:50:22.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:22.022+0000 7fddb6000640 1 -- 172.21.15.5:0/2182961598 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fddac031ca0 con 0x7fddb8103e30 2024-09-10T09:50:22.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:22.022+0000 7fddb6000640 1 -- 172.21.15.5:0/2182961598 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddac035030 con 0x7fddb8103e30 2024-09-10T09:50:22.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:22.023+0000 7fddbee00640 1 -- 172.21.15.5:0/2182961598 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdd98005290 con 0x7fddb8103e30 2024-09-10T09:50:22.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:22.025+0000 7fddb6000640 1 -- 172.21.15.5:0/2182961598 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7fddac042070 con 0x7fddb8103e30 2024-09-10T09:50:22.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:22.026+0000 7fddb6000640 1 --2- 172.21.15.5:0/2182961598 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fdda00a2c00 0x7fdda00a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:50:22.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:22.026+0000 7fddb6000640 1 -- 172.21.15.5:0/2182961598 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fddac0ef960 con 0x7fddb8103e30 2024-09-10T09:50:22.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:22.026+0000 7fddbd000640 1 --2- 172.21.15.5:0/2182961598 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fdda00a2c00 0x7fdda00a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:50:22.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:22.027+0000 7fddbd000640 1 --2- 172.21.15.5:0/2182961598 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fdda00a2c00 0x7fdda00a50c0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fddb80ffd60 tx=0x7fdda400b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:50:22.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:22.030+0000 7fddb6000640 1 -- 172.21.15.5:0/2182961598 <== mon.1 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fddac0bc470 con 0x7fddb8103e30 2024-09-10T09:50:22.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:50:21 smithi159 ceph-mon[35513]: pgmap v753: 129 pgs: 129 active+clean; 284 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 1.6 MiB/s wr, 223 op/s 2024-09-10T09:50:22.134 INFO:tasks.workunit.client.0.smithi005.stdout: 1 764478 6.42 MB/sec execute 483 sec latency 23.362 ms 2024-09-10T09:50:22.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:22.266+0000 7fddbee00640 1 -- 172.21.15.5:0/2182961598 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdd98005580 con 0x7fddb8103e30 2024-09-10T09:50:22.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:22.266+0000 7fddb6000640 1 -- 172.21.15.5:0/2182961598 <== mon.1 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) ==== 94+0+11969 (secure 0 0 0) 0x7fddac0bbd70 con 0x7fddb8103e30 2024-09-10T09:50:22.269 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-10T09:50:22.269 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":36,"btime":"2024-09-10T09:48:07:349075+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14454,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/2970594311","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2970594311},{"type":"v1","addr":"172.21.15.5:6835","nonce":2970594311}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14514,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6839/3616502174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":3616502174},{"type":"v1","addr":"172.21.15.5:6839","nonce":3616502174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":36,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T09:35:29.626287+0000","modified":"2024-09-10T09:48:06.970127+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24542,"mds_1":24535,"mds_2":24511,"mds_3":24506,"mds_4":24443},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14520":{"gid":14520,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/2073599842","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":2073599842},{"type":"v1","addr":"172.21.15.5:6841","nonce":2073599842}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"k","rank":4,"incarnation":26,"state":"up:active","state_seq":24,"addr":"172.21.15.102:6833/2457007427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":2457007427},{"type":"v1","addr":"172.21.15.102:6833","nonce":2457007427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/356445159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":356445159},{"type":"v1","addr":"172.21.15.159:6833","nonce":356445159}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6837/2894314507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":2894314507},{"type":"v1","addr":"172.21.15.5:6837","nonce":2894314507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.102:6835/1647549111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":1647549111},{"type":"v1","addr":"172.21.15.102:6835","nonce":1647549111}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6835/952657242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":952657242},{"type":"v1","addr":"172.21.15.159:6835","nonce":952657242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":19,"addr":"172.21.15.102:6837/2689776764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":2689776764},{"type":"v1","addr":"172.21.15.102:6837","nonce":2689776764}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.159:6837/620627639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":620627639},{"type":"v1","addr":"172.21.15.159:6837","nonce":620627639}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24535":{"gid":24535,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":18,"addr":"172.21.15.159:6839/2073739599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":2073739599},{"type":"v1","addr":"172.21.15.159:6839","nonce":2073739599}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24542":{"gid":24542,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.102:6839/335409749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6838","nonce":335409749},{"type":"v1","addr":"172.21.15.102:6839","nonce":335409749}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24542,"qdb_cluster":[24542,24535,24511,24506,24443]},"id":1}]} 2024-09-10T09:50:22.269 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 36 2024-09-10T09:50:22.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:22.269+0000 7fddbee00640 1 -- 172.21.15.5:0/2182961598 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fdda00a2c00 msgr2=0x7fdda00a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:50:22.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:22.269+0000 7fddbee00640 1 --2- 172.21.15.5:0/2182961598 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fdda00a2c00 0x7fdda00a50c0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fddb80ffd60 tx=0x7fdda400b410 comp rx=0 tx=0).stop 2024-09-10T09:50:22.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:22.269+0000 7fddbee00640 1 -- 172.21.15.5:0/2182961598 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fddb8103e30 msgr2=0x7fddb819a4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:50:22.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:22.269+0000 7fddbee00640 1 --2- 172.21.15.5:0/2182961598 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fddb8103e30 0x7fddb819a4a0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fddac004770 tx=0x7fddac003c10 comp rx=0 tx=0).stop 2024-09-10T09:50:22.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:22.269+0000 7fddbee00640 1 -- 172.21.15.5:0/2182961598 shutdown_connections 2024-09-10T09:50:22.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:22.269+0000 7fddbee00640 1 --2- 172.21.15.5:0/2182961598 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fdda00a2c00 0x7fdda00a50c0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:22.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:22.269+0000 7fddbee00640 1 --2- 172.21.15.5:0/2182961598 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fddb8108f60 0x7fddb819ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:22.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:22.269+0000 7fddbee00640 1 --2- 172.21.15.5:0/2182961598 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fddb81047e0 0x7fddb819a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:22.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:22.269+0000 7fddbee00640 1 --2- 172.21.15.5:0/2182961598 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fddb8103e30 0x7fddb819a4a0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:22.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:22.269+0000 7fddbee00640 1 -- 172.21.15.5:0/2182961598 >> 172.21.15.5:0/2182961598 conn(0x7fddb80ff8d0 msgr2=0x7fddb8100b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:50:22.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:22.270+0000 7fddbee00640 1 -- 172.21.15.5:0/2182961598 shutdown_connections 2024-09-10T09:50:22.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:22.270+0000 7fddbee00640 1 -- 172.21.15.5:0/2182961598 wait complete. 2024-09-10T09:50:23.001 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:50:22 smithi159 ceph-mon[35513]: from='client.? 172.21.15.5:0/2182961598' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:50:23.049 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:50:22 smithi005 ceph-mon[30350]: from='client.? 172.21.15.5:0/2182961598' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:50:23.094 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-10T09:50:23.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:50:22 smithi102 ceph-mon[35442]: from='client.? 172.21.15.5:0/2182961598' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:50:23.135 INFO:tasks.workunit.client.0.smithi005.stdout: 1 767209 6.44 MB/sec execute 484 sec latency 14.341 ms 2024-09-10T09:50:23.548 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:50:23.880 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:50:23 smithi005 ceph-mon[30350]: pgmap v754: 129 pgs: 129 active+clean; 287 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 1.6 MiB/s wr, 203 op/s 2024-09-10T09:50:24.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:50:23 smithi159 ceph-mon[35513]: pgmap v754: 129 pgs: 129 active+clean; 287 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 1.6 MiB/s wr, 203 op/s 2024-09-10T09:50:24.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:50:23 smithi102 ceph-mon[35442]: pgmap v754: 129 pgs: 129 active+clean; 287 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 1.6 MiB/s wr, 203 op/s 2024-09-10T09:50:24.135 INFO:tasks.workunit.client.0.smithi005.stdout: 1 767341 6.43 MB/sec execute 485 sec latency 862.622 ms 2024-09-10T09:50:24.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:24.533+0000 7f7d76200640 1 -- 172.21.15.5:0/4019644652 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7d701049c0 msgr2=0x7f7d70108a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:50:24.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:24.533+0000 7f7d76200640 1 --2- 172.21.15.5:0/4019644652 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7d701049c0 0x7f7d70108a10 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f7d5c009900 tx=0x7f7d5c02a410 comp rx=0 tx=0).stop 2024-09-10T09:50:24.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:24.533+0000 7f7d76200640 1 -- 172.21.15.5:0/4019644652 shutdown_connections 2024-09-10T09:50:24.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:24.533+0000 7f7d76200640 1 --2- 172.21.15.5:0/4019644652 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7d70109140 0x7f7d70110cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:24.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:24.533+0000 7f7d76200640 1 --2- 172.21.15.5:0/4019644652 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7d701049c0 0x7f7d70108a10 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:24.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:24.533+0000 7f7d76200640 1 --2- 172.21.15.5:0/4019644652 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7d70104010 0x7f7d701043f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:24.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:24.533+0000 7f7d76200640 1 -- 172.21.15.5:0/4019644652 >> 172.21.15.5:0/4019644652 conn(0x7f7d700ffa50 msgr2=0x7f7d70101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:50:24.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:24.533+0000 7f7d76200640 1 -- 172.21.15.5:0/4019644652 shutdown_connections 2024-09-10T09:50:24.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:24.533+0000 7f7d76200640 1 -- 172.21.15.5:0/4019644652 wait complete. 2024-09-10T09:50:24.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:24.534+0000 7f7d76200640 1 Processor -- start 2024-09-10T09:50:24.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:24.535+0000 7f7d76200640 1 -- start start 2024-09-10T09:50:24.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:24.535+0000 7f7d76200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7d70104010 0x7f7d7019a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:50:24.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:24.535+0000 7f7d76200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7d701049c0 0x7f7d7019aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:50:24.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:24.535+0000 7f7d76200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7d70109140 0x7f7d7019ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:50:24.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:24.535+0000 7f7d76200640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f7d70114140 con 0x7f7d701049c0 2024-09-10T09:50:24.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:24.535+0000 7f7d76200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f7d70113fc0 con 0x7f7d70104010 2024-09-10T09:50:24.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:24.535+0000 7f7d76200640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f7d701142c0 con 0x7f7d70109140 2024-09-10T09:50:24.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:24.535+0000 7f7d6ea00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7d701049c0 0x7f7d7019aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:50:24.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:24.535+0000 7f7d6ea00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7d701049c0 0x7f7d7019aba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35570/0 (socket says 172.21.15.5:35570) 2024-09-10T09:50:24.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:24.535+0000 7f7d6ea00640 1 -- 172.21.15.5:0/2779524719 learned_addr learned my addr 172.21.15.5:0/2779524719 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:50:24.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:24.535+0000 7f7d6f400640 1 --2- 172.21.15.5:0/2779524719 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7d70104010 0x7f7d7019a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:50:24.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:24.536+0000 7f7d6fe00640 1 --2- 172.21.15.5:0/2779524719 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7d70109140 0x7f7d7019ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:50:24.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:24.536+0000 7f7d6ea00640 1 -- 172.21.15.5:0/2779524719 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7d70109140 msgr2=0x7f7d7019ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:50:24.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:24.536+0000 7f7d6ea00640 1 --2- 172.21.15.5:0/2779524719 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7d70109140 0x7f7d7019ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:24.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:24.536+0000 7f7d6ea00640 1 -- 172.21.15.5:0/2779524719 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7d70104010 msgr2=0x7f7d7019a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:50:24.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:24.536+0000 7f7d6ea00640 1 --2- 172.21.15.5:0/2779524719 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7d70104010 0x7f7d7019a660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:24.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:24.536+0000 7f7d6ea00640 1 -- 172.21.15.5:0/2779524719 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d7019f610 con 0x7f7d701049c0 2024-09-10T09:50:24.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:24.536+0000 7f7d6ea00640 1 --2- 172.21.15.5:0/2779524719 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7d701049c0 0x7f7d7019aba0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f7d5c006010 tx=0x7f7d5c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:50:24.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:24.537+0000 7f7d67e00640 1 -- 172.21.15.5:0/2779524719 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d5c03f030 con 0x7f7d701049c0 2024-09-10T09:50:24.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:24.537+0000 7f7d67e00640 1 -- 172.21.15.5:0/2779524719 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7d5c003ce0 con 0x7f7d701049c0 2024-09-10T09:50:24.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:24.537+0000 7f7d67e00640 1 -- 172.21.15.5:0/2779524719 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d5c035070 con 0x7f7d701049c0 2024-09-10T09:50:24.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:24.537+0000 7f7d76200640 1 -- 172.21.15.5:0/2779524719 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d7019f8a0 con 0x7f7d701049c0 2024-09-10T09:50:24.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:24.537+0000 7f7d76200640 1 -- 172.21.15.5:0/2779524719 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d701a71e0 con 0x7f7d701049c0 2024-09-10T09:50:24.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:24.540+0000 7f7d78e10740 1 Processor -- start 2024-09-10T09:50:24.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:24.540+0000 7f7d67e00640 1 -- 172.21.15.5:0/2779524719 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f7d5c042030 con 0x7f7d701049c0 2024-09-10T09:50:24.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:24.540+0000 7f7d78e10740 1 -- start start 2024-09-10T09:50:24.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:24.541+0000 7f7d78e10740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55610692dc70 0x55610692e050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:50:24.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:24.541+0000 7f7d78e10740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55610692e590 0x556106938910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:50:24.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:24.541+0000 7f7d78e10740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556106938e50 0x55610693b240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:50:24.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:24.541+0000 7f7d78e10740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5561068943e0 con 0x55610692dc70 2024-09-10T09:50:24.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:24.541+0000 7f7d78e10740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55610683bd80 con 0x55610692e590 2024-09-10T09:50:24.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:24.541+0000 7f7d78e10740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5561068903a0 con 0x556106938e50 2024-09-10T09:50:24.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:24.541+0000 7f7d6f400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55610692e590 0x556106938910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:50:24.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:24.541+0000 7f7d6f400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55610692e590 0x556106938910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:53986/0 (socket says 172.21.15.5:53986) 2024-09-10T09:50:24.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:24.541+0000 7f7d6f400640 1 -- 172.21.15.5:0/3390238261 learned_addr learned my addr 172.21.15.5:0/3390238261 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:50:24.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:24.541+0000 7f7d6fe00640 1 --2- 172.21.15.5:0/3390238261 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55610692dc70 0x55610692e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:50:24.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:24.541+0000 7f7d6ea00640 1 --2- 172.21.15.5:0/3390238261 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556106938e50 0x55610693b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:50:24.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:24.541+0000 7f7d6f400640 1 -- 172.21.15.5:0/3390238261 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556106938e50 msgr2=0x55610693b240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:50:24.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:24.541+0000 7f7d6f400640 1 --2- 172.21.15.5:0/3390238261 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556106938e50 0x55610693b240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:24.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:24.541+0000 7f7d6f400640 1 -- 172.21.15.5:0/3390238261 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55610692dc70 msgr2=0x55610692e050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:50:24.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:24.541+0000 7f7d6f400640 1 --2- 172.21.15.5:0/3390238261 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55610692dc70 0x55610692e050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:24.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:24.542+0000 7f7d6f400640 1 -- 172.21.15.5:0/3390238261 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55610685aae0 con 0x55610692e590 2024-09-10T09:50:24.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:24.542+0000 7f7d6f400640 1 --2- 172.21.15.5:0/3390238261 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55610692e590 0x556106938910 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f7d6000bd10 tx=0x7f7d600107a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a9000df7612c6a96 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:50:24.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:24.542+0000 7f7d67e00640 1 --2- 172.21.15.5:0/2779524719 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f7d540a2c00 0x7f7d540a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:50:24.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:24.542+0000 7f7d5be00640 1 -- 172.21.15.5:0/3390238261 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d60004030 con 0x55610692e590 2024-09-10T09:50:24.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:24.542+0000 7f7d5be00640 1 -- 172.21.15.5:0/3390238261 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7d600041d0 con 0x55610692e590 2024-09-10T09:50:24.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:24.542+0000 7f7d5be00640 1 -- 172.21.15.5:0/3390238261 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d60011820 con 0x55610692e590 2024-09-10T09:50:24.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:24.542+0000 7f7d67e00640 1 -- 172.21.15.5:0/2779524719 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f7d5c0ef970 con 0x7f7d701049c0 2024-09-10T09:50:24.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:24.543+0000 7f7d6fe00640 1 --2- 172.21.15.5:0/2779524719 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f7d540a2c00 0x7f7d540a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:50:24.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:24.543+0000 7f7d78e10740 1 -- 172.21.15.5:0/3390238261 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55610692e590 msgr2=0x556106938910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:50:24.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:24.543+0000 7f7d78e10740 1 --2- 172.21.15.5:0/3390238261 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55610692e590 0x556106938910 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f7d6000bd10 tx=0x7f7d600107a0 comp rx=0 tx=0).stop 2024-09-10T09:50:24.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:24.543+0000 7f7d78e10740 1 -- 172.21.15.5:0/3390238261 shutdown_connections 2024-09-10T09:50:24.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:24.543+0000 7f7d78e10740 1 --2- 172.21.15.5:0/3390238261 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556106938e50 0x55610693b240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:24.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:24.543+0000 7f7d78e10740 1 --2- 172.21.15.5:0/3390238261 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55610692e590 0x556106938910 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:24.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:24.543+0000 7f7d78e10740 1 --2- 172.21.15.5:0/3390238261 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55610692dc70 0x55610692e050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:24.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:24.543+0000 7f7d78e10740 1 -- 172.21.15.5:0/3390238261 >> 172.21.15.5:0/3390238261 conn(0x55610692a200 msgr2=0x55610692a5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:50:24.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:24.543+0000 7f7d6fe00640 1 --2- 172.21.15.5:0/2779524719 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f7d540a2c00 0x7f7d540a50c0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f7d68005fd0 tx=0x7f7d680096a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:50:24.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:24.543+0000 7f7d78e10740 1 -- 172.21.15.5:0/3390238261 shutdown_connections 2024-09-10T09:50:24.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:24.543+0000 7f7d78e10740 1 -- 172.21.15.5:0/3390238261 wait complete. 2024-09-10T09:50:24.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:24.544+0000 7f7d78e10740 1 -- start start 2024-09-10T09:50:24.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:24.544+0000 7f7d78e10740 1 Processor -- start 2024-09-10T09:50:24.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:24.544+0000 7f7d78e10740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55610692dc70 0x556106935cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:50:24.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:24.544+0000 7f7d78e10740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55610692e590 0x556106936230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:50:24.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:24.545+0000 7f7d78e10740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556106938e50 0x556106936770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:50:24.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:24.545+0000 7f7d78e10740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55610693ba20 con 0x55610692e590 2024-09-10T09:50:24.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:24.545+0000 7f7d78e10740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55610684f0f0 con 0x55610692dc70 2024-09-10T09:50:24.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:24.545+0000 7f7d78e10740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55610693bba0 con 0x556106938e50 2024-09-10T09:50:24.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:24.545+0000 7f7d78e10740 10 client.0 _ll_register_callbacks cb 0x556106836bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:50:24.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:24.545+0000 7f7d6ea00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55610692e590 0x556106936230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:50:24.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:24.545+0000 7f7d6ea00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55610692e590 0x556106936230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35590/0 (socket says 172.21.15.5:35590) 2024-09-10T09:50:24.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:24.545+0000 7f7d6ea00640 1 -- 172.21.15.5:0/2071851274 learned_addr learned my addr 172.21.15.5:0/2071851274 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:50:24.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:24.545+0000 7f7d6f400640 1 --2- 172.21.15.5:0/2071851274 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55610692dc70 0x556106935cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:50:24.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:24.545+0000 7f7d6fe00640 1 --2- 172.21.15.5:0/2071851274 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556106938e50 0x556106936770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:50:24.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:24.545+0000 7f7d6ea00640 1 -- 172.21.15.5:0/2071851274 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556106938e50 msgr2=0x556106936770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:50:24.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:24.545+0000 7f7d6ea00640 1 --2- 172.21.15.5:0/2071851274 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556106938e50 0x556106936770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:24.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:24.545+0000 7f7d6ea00640 1 -- 172.21.15.5:0/2071851274 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55610692dc70 msgr2=0x556106935cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:50:24.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:24.545+0000 7f7d6ea00640 1 --2- 172.21.15.5:0/2071851274 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55610692dc70 0x556106935cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:24.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:24.545+0000 7f7d6ea00640 1 -- 172.21.15.5:0/2071851274 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5561066053c0 con 0x55610692e590 2024-09-10T09:50:24.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:24.545+0000 7f7d6f400640 1 --2- 172.21.15.5:0/2071851274 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55610692dc70 0x556106935cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:50:24.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:24.546+0000 7f7d6ea00640 1 --2- 172.21.15.5:0/2071851274 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55610692e590 0x556106936230 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f7d5c0b8350 tx=0x7f7d5c0b7b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a9b83a0454f8ad7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:50:24.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:24.546+0000 7f7d58c00640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-10T09:50:24.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:24.546+0000 7f7d58c00640 1 -- 172.21.15.5:0/2071851274 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d5c0be060 con 0x55610692e590 2024-09-10T09:50:24.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:24.546+0000 7f7d58c00640 1 -- 172.21.15.5:0/2071851274 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f7d5c0bc150 con 0x55610692e590 2024-09-10T09:50:24.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:24.546+0000 7f7d78e10740 1 -- 172.21.15.5:0/2071851274 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7d600041d0 con 0x55610692e590 2024-09-10T09:50:24.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:24.546+0000 7f7d58c00640 1 -- 172.21.15.5:0/2071851274 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f7d5c049070 con 0x55610692e590 2024-09-10T09:50:24.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:24.546+0000 7f7d78e10740 10 client.17763 fetch_fsmap learned FSMap version 36 2024-09-10T09:50:24.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:24.546+0000 7f7d78e10740 1 -- 172.21.15.5:0/2071851274 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x556106947a10 con 0x55610692e590 2024-09-10T09:50:24.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:24.547+0000 7f7d58c00640 1 -- 172.21.15.5:0/2071851274 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7d600041d0 con 0x55610692e590 2024-09-10T09:50:24.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:24.547+0000 7f7d58c00640 1 -- 172.21.15.5:0/2071851274 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 36) ==== 8370+0+0 (secure 0 0 0) 0x7f7d5c0c4620 con 0x55610692e590 2024-09-10T09:50:24.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:24.547+0000 7f7d78e10740 10 client.17763 fetch_fsmap finished waiting for FSMap version 36 2024-09-10T09:50:24.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:24.547+0000 7f7d78e10740 10 client.17763 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-09-10T09:50:24.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:24.547+0000 7f7d78e10740 20 client.17763 populate_metadata read hostname 'smithi005' 2024-09-10T09:50:24.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:24.547+0000 7f7d78e10740 1 --2- 172.21.15.5:0/2071851274 >> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] conn(0x556106948e00 0x55610694b250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:50:24.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:24.547+0000 7f7d78e10740 4 client.17763 mds_command: new command op to 24443 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:50:24.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:24.547+0000 7f7d78e10740 1 -- 172.21.15.5:0/2071851274 --> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5561068a1950 con 0x556106948e00 2024-09-10T09:50:24.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:24.564+0000 7f7d6f400640 1 --2- 172.21.15.5:0/2071851274 >> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] conn(0x556106948e00 0x55610694b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:50:24.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:24.624+0000 7f7d6f400640 1 --2- 172.21.15.5:0/2071851274 >> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] conn(0x556106948e00 0x55610694b250 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=ced9276169efab59 server_cookie=cd45b7dd8340e193 in_seq=0 out_seq=0 2024-09-10T09:50:24.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:24.624+0000 7f7d58c00640 10 client.17763 ms_handle_connect on v2:172.21.15.102:6832/2457007427 2024-09-10T09:50:24.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:24.648+0000 7f7d58c00640 1 -- 172.21.15.5:0/2071851274 <== mds.4 v2:172.21.15.102:6832/2457007427 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5561068a1950 con 0x556106948e00 2024-09-10T09:50:24.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:24.648+0000 7f7d58c00640 10 client.17763 handle_command_reply: tid=0 2024-09-10T09:50:24.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:24.648+0000 7f7d78e10740 1 client.17763 shutdown 2024-09-10T09:50:24.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:24.648+0000 7f7d78e10740 1 -- 172.21.15.5:0/2071851274 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55610692e590 msgr2=0x556106936230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:50:24.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:24.648+0000 7f7d78e10740 1 --2- 172.21.15.5:0/2071851274 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55610692e590 0x556106936230 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f7d5c0b8350 tx=0x7f7d5c0b7b60 comp rx=0 tx=0).stop 2024-09-10T09:50:24.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:24.648+0000 7f7d78e10740 1 -- 172.21.15.5:0/2071851274 shutdown_connections 2024-09-10T09:50:24.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:24.648+0000 7f7d78e10740 1 --2- 172.21.15.5:0/2071851274 >> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] conn(0x556106948e00 0x55610694b250 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:24.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:24.648+0000 7f7d78e10740 1 --2- 172.21.15.5:0/2071851274 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556106938e50 0x556106936770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:24.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:24.648+0000 7f7d78e10740 1 --2- 172.21.15.5:0/2071851274 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55610692e590 0x556106936230 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:24.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:24.648+0000 7f7d78e10740 1 --2- 172.21.15.5:0/2071851274 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55610692dc70 0x556106935cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:24.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:24.648+0000 7f7d78e10740 1 -- 172.21.15.5:0/2071851274 >> 172.21.15.5:0/2071851274 conn(0x55610692a200 msgr2=0x55610692ff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:50:24.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:24.648+0000 7f7d58c00640 0 client.17763 ms_handle_reset on v2:172.21.15.102:6832/2457007427 2024-09-10T09:50:24.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:24.649+0000 7f7d78e10740 1 -- 172.21.15.5:0/2071851274 shutdown_connections 2024-09-10T09:50:24.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:24.649+0000 7f7d78e10740 1 -- 172.21.15.5:0/2071851274 wait complete. 2024-09-10T09:50:24.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:24.649+0000 7f7d78e10740 20 client.17763 trim_cache size 0 max 16384 2024-09-10T09:50:24.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:24.665+0000 7f7d78e10740 1 Processor -- start 2024-09-10T09:50:24.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:24.665+0000 7f7d78e10740 1 -- start start 2024-09-10T09:50:24.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:24.665+0000 7f7d78e10740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55610692dc70 0x556106950b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:50:24.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:24.665+0000 7f7d78e10740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55610692e590 0x556106951050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:50:24.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:24.665+0000 7f7d78e10740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556106938e50 0x556106928b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:50:24.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:24.665+0000 7f7d78e10740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f7d701b0fa0 con 0x55610692dc70 2024-09-10T09:50:24.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:24.665+0000 7f7d78e10740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f7d701b2ae0 con 0x55610692e590 2024-09-10T09:50:24.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:24.665+0000 7f7d78e10740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f7d701b1670 con 0x556106938e50 2024-09-10T09:50:24.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:24.665+0000 7f7d6f400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55610692dc70 0x556106950b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:50:24.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:24.665+0000 7f7d6ea00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55610692e590 0x556106951050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:50:24.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:24.666+0000 7f7d6f400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55610692dc70 0x556106950b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35592/0 (socket says 172.21.15.5:35592) 2024-09-10T09:50:24.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:24.666+0000 7f7d6f400640 1 -- 172.21.15.5:0/2966232918 learned_addr learned my addr 172.21.15.5:0/2966232918 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:50:24.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:24.666+0000 7f7d6fe00640 1 --2- 172.21.15.5:0/2966232918 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556106938e50 0x556106928b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:50:24.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:24.666+0000 7f7d6ea00640 1 -- 172.21.15.5:0/2966232918 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556106938e50 msgr2=0x556106928b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:50:24.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:24.666+0000 7f7d6ea00640 1 --2- 172.21.15.5:0/2966232918 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556106938e50 0x556106928b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:24.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:24.666+0000 7f7d6ea00640 1 -- 172.21.15.5:0/2966232918 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55610692dc70 msgr2=0x556106950b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:50:24.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:24.666+0000 7f7d6ea00640 1 --2- 172.21.15.5:0/2966232918 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55610692dc70 0x556106950b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:24.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:24.666+0000 7f7d6ea00640 1 -- 172.21.15.5:0/2966232918 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55610692a030 con 0x55610692e590 2024-09-10T09:50:24.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:24.672+0000 7f7d6ea00640 1 --2- 172.21.15.5:0/2966232918 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55610692e590 0x556106951050 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f7d5c0b7bd0 tx=0x7f7d5c0095d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ef2f615062821a40 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:50:24.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:24.672+0000 7f7d5b400640 1 -- 172.21.15.5:0/2966232918 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d5c0bb850 con 0x55610692e590 2024-09-10T09:50:24.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:24.672+0000 7f7d5b400640 1 -- 172.21.15.5:0/2966232918 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7d5c03b580 con 0x55610692e590 2024-09-10T09:50:24.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:24.672+0000 7f7d5b400640 1 -- 172.21.15.5:0/2966232918 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d5c0bba30 con 0x55610692e590 2024-09-10T09:50:24.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:24.673+0000 7f7d78e10740 1 -- 172.21.15.5:0/2966232918 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55610692e590 msgr2=0x556106951050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:50:24.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:24.673+0000 7f7d78e10740 1 --2- 172.21.15.5:0/2966232918 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55610692e590 0x556106951050 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f7d5c0b7bd0 tx=0x7f7d5c0095d0 comp rx=0 tx=0).stop 2024-09-10T09:50:24.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:24.673+0000 7f7d78e10740 1 -- 172.21.15.5:0/2966232918 shutdown_connections 2024-09-10T09:50:24.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:24.673+0000 7f7d78e10740 1 --2- 172.21.15.5:0/2966232918 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556106938e50 0x556106928b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:24.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:24.673+0000 7f7d78e10740 1 --2- 172.21.15.5:0/2966232918 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55610692e590 0x556106951050 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:24.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:24.673+0000 7f7d78e10740 1 --2- 172.21.15.5:0/2966232918 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55610692dc70 0x556106950b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:24.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:24.673+0000 7f7d78e10740 1 -- 172.21.15.5:0/2966232918 >> 172.21.15.5:0/2966232918 conn(0x55610692a200 msgr2=0x556106931090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:50:24.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:24.673+0000 7f7d78e10740 1 -- 172.21.15.5:0/2966232918 shutdown_connections 2024-09-10T09:50:24.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:24.673+0000 7f7d78e10740 1 -- 172.21.15.5:0/2966232918 wait complete. 2024-09-10T09:50:24.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:24.674+0000 7f7d78e10740 1 -- start start 2024-09-10T09:50:24.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:24.674+0000 7f7d78e10740 1 Processor -- start 2024-09-10T09:50:24.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:24.674+0000 7f7d78e10740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55610692dc70 0x55610693c570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:50:24.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:24.674+0000 7f7d78e10740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55610692e590 0x55610693cab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:50:24.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:24.674+0000 7f7d78e10740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556106938e50 0x5561069abaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:50:24.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:24.674+0000 7f7d78e10740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f7d701b2410 con 0x55610692e590 2024-09-10T09:50:24.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:24.675+0000 7f7d78e10740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f7d701b1d40 con 0x556106938e50 2024-09-10T09:50:24.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:24.675+0000 7f7d78e10740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f7d701aef80 con 0x55610692dc70 2024-09-10T09:50:24.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:24.675+0000 7f7d78e10740 10 client.0 _ll_register_callbacks cb 0x7f7d5c005890 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:50:24.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:24.675+0000 7f7d6ea00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55610692e590 0x55610693cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:50:24.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:24.675+0000 7f7d6ea00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55610692e590 0x55610693cab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35594/0 (socket says 172.21.15.5:35594) 2024-09-10T09:50:24.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:24.675+0000 7f7d6ea00640 1 -- 172.21.15.5:0/3101154525 learned_addr learned my addr 172.21.15.5:0/3101154525 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:50:24.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:24.675+0000 7f7d6fe00640 1 --2- 172.21.15.5:0/3101154525 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556106938e50 0x5561069abaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:50:24.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:24.675+0000 7f7d6ea00640 1 -- 172.21.15.5:0/3101154525 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55610692dc70 msgr2=0x55610693c570 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T09:50:24.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:24.675+0000 7f7d6ea00640 1 --2- 172.21.15.5:0/3101154525 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55610692dc70 0x55610693c570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:24.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:24.675+0000 7f7d6ea00640 1 -- 172.21.15.5:0/3101154525 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556106938e50 msgr2=0x5561069abaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:50:24.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:24.675+0000 7f7d6ea00640 1 --2- 172.21.15.5:0/3101154525 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556106938e50 0x5561069abaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:24.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:24.675+0000 7f7d6ea00640 1 -- 172.21.15.5:0/3101154525 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5561069ac110 con 0x55610692e590 2024-09-10T09:50:24.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:24.676+0000 7f7d6ea00640 1 --2- 172.21.15.5:0/3101154525 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55610692e590 0x55610693cab0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f7d5c03d040 tx=0x7f7d5c0c4ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a4a738332a311ba server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:50:24.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:24.676+0000 7f7d58c00640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-10T09:50:24.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:24.676+0000 7f7d58c00640 1 -- 172.21.15.5:0/3101154525 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d5c03bcb0 con 0x55610692e590 2024-09-10T09:50:24.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:24.676+0000 7f7d58c00640 1 -- 172.21.15.5:0/3101154525 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f7d5c0bc350 con 0x55610692e590 2024-09-10T09:50:24.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:24.676+0000 7f7d78e10740 1 -- 172.21.15.5:0/3101154525 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55610694b5e0 con 0x55610692e590 2024-09-10T09:50:24.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:24.676+0000 7f7d58c00640 1 -- 172.21.15.5:0/3101154525 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f7d5c0b0050 con 0x55610692e590 2024-09-10T09:50:24.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:24.676+0000 7f7d78e10740 10 client.17769 fetch_fsmap learned FSMap version 36 2024-09-10T09:50:24.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:24.676+0000 7f7d78e10740 1 -- 172.21.15.5:0/3101154525 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x5561069ac110 con 0x55610692e590 2024-09-10T09:50:24.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:24.676+0000 7f7d58c00640 1 -- 172.21.15.5:0/3101154525 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55610694b5e0 con 0x55610692e590 2024-09-10T09:50:24.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:24.677+0000 7f7d58c00640 1 -- 172.21.15.5:0/3101154525 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 36) ==== 8370+0+0 (secure 0 0 0) 0x7f7d5c0b7c40 con 0x55610692e590 2024-09-10T09:50:24.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:24.677+0000 7f7d78e10740 10 client.17769 fetch_fsmap finished waiting for FSMap version 36 2024-09-10T09:50:24.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:24.677+0000 7f7d78e10740 10 client.17769 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-09-10T09:50:24.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:24.677+0000 7f7d78e10740 20 client.17769 populate_metadata read hostname 'smithi005' 2024-09-10T09:50:24.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:24.677+0000 7f7d78e10740 1 --2- 172.21.15.5:0/3101154525 >> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] conn(0x5561069b37a0 0x5561069b5b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:50:24.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:24.677+0000 7f7d78e10740 4 client.17769 mds_command: new command op to 24443 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T09:50:24.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:24.677+0000 7f7d78e10740 1 -- 172.21.15.5:0/3101154525 --> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55610689be60 con 0x5561069b37a0 2024-09-10T09:50:24.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:24.695+0000 7f7d6f400640 1 --2- 172.21.15.5:0/3101154525 >> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] conn(0x5561069b37a0 0x5561069b5b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:50:24.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:24.755+0000 7f7d6f400640 1 --2- 172.21.15.5:0/3101154525 >> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] conn(0x5561069b37a0 0x5561069b5b90 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=d7f4394c65ee239e server_cookie=15a965c7720cda0a in_seq=0 out_seq=0 2024-09-10T09:50:24.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:24.755+0000 7f7d58c00640 10 client.17769 ms_handle_connect on v2:172.21.15.102:6832/2457007427 2024-09-10T09:50:24.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:24.767+0000 7f7d58c00640 1 -- 172.21.15.5:0/3101154525 <== mds.4 v2:172.21.15.102:6832/2457007427 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55610689be60 con 0x5561069b37a0 2024-09-10T09:50:24.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:24.767+0000 7f7d58c00640 10 client.17769 handle_command_reply: tid=0 2024-09-10T09:50:24.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:24.767+0000 7f7d78e10740 1 client.17769 shutdown 2024-09-10T09:50:24.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:24.767+0000 7f7d78e10740 1 -- 172.21.15.5:0/3101154525 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55610692e590 msgr2=0x55610693cab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:50:24.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:24.767+0000 7f7d78e10740 1 --2- 172.21.15.5:0/3101154525 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55610692e590 0x55610693cab0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f7d5c03d040 tx=0x7f7d5c0c4ab0 comp rx=0 tx=0).stop 2024-09-10T09:50:24.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:24.767+0000 7f7d78e10740 1 -- 172.21.15.5:0/3101154525 shutdown_connections 2024-09-10T09:50:24.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:24.767+0000 7f7d78e10740 1 --2- 172.21.15.5:0/3101154525 >> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] conn(0x5561069b37a0 0x5561069b5b90 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:24.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:24.767+0000 7f7d78e10740 1 --2- 172.21.15.5:0/3101154525 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556106938e50 0x5561069abaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:24.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:24.767+0000 7f7d78e10740 1 --2- 172.21.15.5:0/3101154525 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55610692e590 0x55610693cab0 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:24.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:24.767+0000 7f7d78e10740 1 --2- 172.21.15.5:0/3101154525 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55610692dc70 0x55610693c570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:24.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:24.767+0000 7f7d78e10740 1 -- 172.21.15.5:0/3101154525 >> 172.21.15.5:0/3101154525 conn(0x55610692a200 msgr2=0x556106935cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:50:24.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:24.767+0000 7f7d58c00640 0 client.17769 ms_handle_reset on v2:172.21.15.102:6832/2457007427 2024-09-10T09:50:24.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:24.767+0000 7f7d78e10740 1 -- 172.21.15.5:0/3101154525 shutdown_connections 2024-09-10T09:50:24.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:24.767+0000 7f7d78e10740 1 -- 172.21.15.5:0/3101154525 wait complete. 2024-09-10T09:50:24.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:24.767+0000 7f7d78e10740 20 client.17769 trim_cache size 0 max 16384 2024-09-10T09:50:24.771 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-10T09:50:24.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:24.768+0000 7f7d76200640 1 -- 172.21.15.5:0/2779524719 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f7d540a2c00 msgr2=0x7f7d540a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:50:24.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:24.768+0000 7f7d76200640 1 --2- 172.21.15.5:0/2779524719 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f7d540a2c00 0x7f7d540a50c0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f7d68005fd0 tx=0x7f7d680096a0 comp rx=0 tx=0).stop 2024-09-10T09:50:24.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:24.768+0000 7f7d76200640 1 -- 172.21.15.5:0/2779524719 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7d701049c0 msgr2=0x7f7d7019aba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:50:24.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:24.768+0000 7f7d76200640 1 --2- 172.21.15.5:0/2779524719 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7d701049c0 0x7f7d7019aba0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f7d5c006010 tx=0x7f7d5c0042f0 comp rx=0 tx=0).stop 2024-09-10T09:50:24.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:24.768+0000 7f7d76200640 1 -- 172.21.15.5:0/2779524719 shutdown_connections 2024-09-10T09:50:24.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:24.768+0000 7f7d76200640 1 --2- 172.21.15.5:0/2779524719 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f7d540a2c00 0x7f7d540a50c0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:24.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:24.768+0000 7f7d76200640 1 --2- 172.21.15.5:0/2779524719 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7d70109140 0x7f7d7019ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:24.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:24.768+0000 7f7d76200640 1 --2- 172.21.15.5:0/2779524719 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7d701049c0 0x7f7d7019aba0 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:24.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:24.768+0000 7f7d76200640 1 --2- 172.21.15.5:0/2779524719 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7d70104010 0x7f7d7019a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:24.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:24.768+0000 7f7d76200640 1 -- 172.21.15.5:0/2779524719 >> 172.21.15.5:0/2779524719 conn(0x7f7d700ffa50 msgr2=0x7f7d70100e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:50:24.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:24.768+0000 7f7d76200640 1 -- 172.21.15.5:0/2779524719 shutdown_connections 2024-09-10T09:50:24.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:24.768+0000 7f7d76200640 1 -- 172.21.15.5:0/2779524719 wait complete. 2024-09-10T09:50:25.134 INFO:tasks.workunit.client.0.smithi005.stdout: 1 767645 6.42 MB/sec execute 486 sec latency 1558.171 ms 2024-09-10T09:50:25.721 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-10T09:50:26.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:50:25 smithi005 ceph-mon[30350]: pgmap v755: 129 pgs: 129 active+clean; 288 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 2.0 MiB/s wr, 200 op/s 2024-09-10T09:50:26.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:50:25 smithi159 ceph-mon[35513]: pgmap v755: 129 pgs: 129 active+clean; 288 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 2.0 MiB/s wr, 200 op/s 2024-09-10T09:50:26.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:50:25 smithi102 ceph-mon[35442]: pgmap v755: 129 pgs: 129 active+clean; 288 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 2.0 MiB/s wr, 200 op/s 2024-09-10T09:50:26.134 INFO:tasks.workunit.client.0.smithi005.stdout: 1 770328 6.43 MB/sec execute 487 sec latency 19.723 ms 2024-09-10T09:50:26.178 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:50:26.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:26.749+0000 7fd236600640 1 -- 172.21.15.5:0/113493252 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd230104010 msgr2=0x7fd2301043f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:50:26.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:26.749+0000 7fd236600640 1 --2- 172.21.15.5:0/113493252 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd230104010 0x7fd2301043f0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fd220009900 tx=0x7fd22002a410 comp rx=0 tx=0).stop 2024-09-10T09:50:26.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:26.749+0000 7fd236600640 1 -- 172.21.15.5:0/113493252 shutdown_connections 2024-09-10T09:50:26.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:26.749+0000 7fd236600640 1 --2- 172.21.15.5:0/113493252 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd230109140 0x7fd230110cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:26.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:26.749+0000 7fd236600640 1 --2- 172.21.15.5:0/113493252 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd2301049c0 0x7fd230108a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:26.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:26.749+0000 7fd236600640 1 --2- 172.21.15.5:0/113493252 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd230104010 0x7fd2301043f0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:26.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:26.750+0000 7fd236600640 1 -- 172.21.15.5:0/113493252 >> 172.21.15.5:0/113493252 conn(0x7fd2300ffa50 msgr2=0x7fd230101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:50:26.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:26.750+0000 7fd236600640 1 -- 172.21.15.5:0/113493252 shutdown_connections 2024-09-10T09:50:26.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:26.750+0000 7fd236600640 1 -- 172.21.15.5:0/113493252 wait complete. 2024-09-10T09:50:26.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:26.751+0000 7fd236600640 1 Processor -- start 2024-09-10T09:50:26.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:26.751+0000 7fd236600640 1 -- start start 2024-09-10T09:50:26.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:26.751+0000 7fd236600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd230104010 0x7fd23019a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:50:26.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:26.751+0000 7fd236600640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd2301049c0 0x7fd23019abf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:50:26.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:26.751+0000 7fd236600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd230109140 0x7fd23019ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:50:26.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:26.751+0000 7fd236600640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd230114190 con 0x7fd230109140 2024-09-10T09:50:26.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:26.751+0000 7fd236600640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fd230114010 con 0x7fd230104010 2024-09-10T09:50:26.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:26.751+0000 7fd236600640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fd230114310 con 0x7fd2301049c0 2024-09-10T09:50:26.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:26.752+0000 7fd22ea00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd2301049c0 0x7fd23019abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:50:26.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:26.752+0000 7fd22ea00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd2301049c0 0x7fd23019abf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:48652/0 (socket says 172.21.15.5:48652) 2024-09-10T09:50:26.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:26.752+0000 7fd22ea00640 1 -- 172.21.15.5:0/3100910983 learned_addr learned my addr 172.21.15.5:0/3100910983 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:50:26.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:26.752+0000 7fd22f400640 1 --2- 172.21.15.5:0/3100910983 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd230104010 0x7fd23019a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:50:26.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:26.752+0000 7fd22fe00640 1 --2- 172.21.15.5:0/3100910983 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd230109140 0x7fd23019ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:50:26.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:26.752+0000 7fd22ea00640 1 -- 172.21.15.5:0/3100910983 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd230104010 msgr2=0x7fd23019a6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:50:26.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:26.752+0000 7fd22ea00640 1 --2- 172.21.15.5:0/3100910983 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd230104010 0x7fd23019a6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:26.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:26.752+0000 7fd22ea00640 1 -- 172.21.15.5:0/3100910983 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd230109140 msgr2=0x7fd23019ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:50:26.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:26.752+0000 7fd22ea00640 1 --2- 172.21.15.5:0/3100910983 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd230109140 0x7fd23019ef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:26.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:26.752+0000 7fd22ea00640 1 -- 172.21.15.5:0/3100910983 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd23019f660 con 0x7fd2301049c0 2024-09-10T09:50:26.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:26.752+0000 7fd22fe00640 1 --2- 172.21.15.5:0/3100910983 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd230109140 0x7fd23019ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:50:26.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:26.753+0000 7fd22ea00640 1 --2- 172.21.15.5:0/3100910983 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd2301049c0 0x7fd23019abf0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fd21c00e9f0 tx=0x7fd21c00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:50:26.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:26.753+0000 7fd227e00640 1 -- 172.21.15.5:0/3100910983 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd21c00cd80 con 0x7fd2301049c0 2024-09-10T09:50:26.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:26.753+0000 7fd227e00640 1 -- 172.21.15.5:0/3100910983 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd21c010910 con 0x7fd2301049c0 2024-09-10T09:50:26.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:26.753+0000 7fd236600640 1 -- 172.21.15.5:0/3100910983 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd23019f950 con 0x7fd2301049c0 2024-09-10T09:50:26.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:26.753+0000 7fd227e00640 1 -- 172.21.15.5:0/3100910983 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd21c018990 con 0x7fd2301049c0 2024-09-10T09:50:26.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:26.753+0000 7fd236600640 1 -- 172.21.15.5:0/3100910983 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2301a7230 con 0x7fd2301049c0 2024-09-10T09:50:26.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:26.757+0000 7fd239239740 1 Processor -- start 2024-09-10T09:50:26.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:26.757+0000 7fd239239740 1 -- start start 2024-09-10T09:50:26.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:26.758+0000 7fd239239740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564ab6e9ac70 0x564ab6e9b050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:50:26.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:26.758+0000 7fd239239740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564ab6e9b590 0x564ab6ea5910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:50:26.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:26.758+0000 7fd239239740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564ab6ea5e50 0x564ab6ea8240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:50:26.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:26.758+0000 7fd239239740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x564ab6e013e0 con 0x564ab6ea5e50 2024-09-10T09:50:26.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:26.758+0000 7fd239239740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x564ab6da8d80 con 0x564ab6e9b590 2024-09-10T09:50:26.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:26.758+0000 7fd239239740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x564ab6dfd3a0 con 0x564ab6e9ac70 2024-09-10T09:50:26.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:26.758+0000 7fd227e00640 1 -- 172.21.15.5:0/3100910983 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7fd21c010ac0 con 0x7fd2301049c0 2024-09-10T09:50:26.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:26.758+0000 7fd22fe00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564ab6e9ac70 0x564ab6e9b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:50:26.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:26.758+0000 7fd22fe00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564ab6e9ac70 0x564ab6e9b050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:48664/0 (socket says 172.21.15.5:48664) 2024-09-10T09:50:26.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:26.758+0000 7fd22fe00640 1 -- 172.21.15.5:0/1739169269 learned_addr learned my addr 172.21.15.5:0/1739169269 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:50:26.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:26.758+0000 7fd22f400640 1 --2- 172.21.15.5:0/1739169269 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564ab6e9b590 0x564ab6ea5910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:50:26.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:26.758+0000 7fd22ea00640 1 --2- 172.21.15.5:0/1739169269 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564ab6ea5e50 0x564ab6ea8240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:50:26.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:26.759+0000 7fd22fe00640 1 -- 172.21.15.5:0/1739169269 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564ab6e9b590 msgr2=0x564ab6ea5910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:50:26.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:26.759+0000 7fd22fe00640 1 --2- 172.21.15.5:0/1739169269 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564ab6e9b590 0x564ab6ea5910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:26.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:26.759+0000 7fd22fe00640 1 -- 172.21.15.5:0/1739169269 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564ab6ea5e50 msgr2=0x564ab6ea8240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:50:26.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:26.759+0000 7fd22fe00640 1 --2- 172.21.15.5:0/1739169269 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564ab6ea5e50 0x564ab6ea8240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:26.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:26.759+0000 7fd22fe00640 1 -- 172.21.15.5:0/1739169269 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564ab6dc7ae0 con 0x564ab6e9ac70 2024-09-10T09:50:26.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:26.759+0000 7fd22fe00640 1 --2- 172.21.15.5:0/1739169269 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564ab6e9ac70 0x564ab6e9b050 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fd228009fd0 tx=0x7fd228010860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b0c8fe48f43b9ea1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:50:26.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:26.759+0000 7fd224c00640 1 -- 172.21.15.5:0/1739169269 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd228011700 con 0x564ab6e9ac70 2024-09-10T09:50:26.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:26.759+0000 7fd224c00640 1 -- 172.21.15.5:0/1739169269 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd228011d80 con 0x564ab6e9ac70 2024-09-10T09:50:26.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:26.759+0000 7fd224c00640 1 -- 172.21.15.5:0/1739169269 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd22800e3c0 con 0x564ab6e9ac70 2024-09-10T09:50:26.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:26.760+0000 7fd227e00640 1 --2- 172.21.15.5:0/3100910983 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fd2140a2c80 0x7fd2140a5140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:50:26.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:26.760+0000 7fd227e00640 1 -- 172.21.15.5:0/3100910983 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fd21c0cf7b0 con 0x7fd2301049c0 2024-09-10T09:50:26.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:26.760+0000 7fd22f400640 1 --2- 172.21.15.5:0/3100910983 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fd2140a2c80 0x7fd2140a5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:50:26.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:26.760+0000 7fd239239740 1 -- 172.21.15.5:0/1739169269 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564ab6e9ac70 msgr2=0x564ab6e9b050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:50:26.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:26.760+0000 7fd239239740 1 --2- 172.21.15.5:0/1739169269 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564ab6e9ac70 0x564ab6e9b050 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fd228009fd0 tx=0x7fd228010860 comp rx=0 tx=0).stop 2024-09-10T09:50:26.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:26.761+0000 7fd239239740 1 -- 172.21.15.5:0/1739169269 shutdown_connections 2024-09-10T09:50:26.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:26.761+0000 7fd239239740 1 --2- 172.21.15.5:0/1739169269 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564ab6ea5e50 0x564ab6ea8240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:26.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:26.761+0000 7fd239239740 1 --2- 172.21.15.5:0/1739169269 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564ab6e9b590 0x564ab6ea5910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:26.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:26.761+0000 7fd239239740 1 --2- 172.21.15.5:0/1739169269 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564ab6e9ac70 0x564ab6e9b050 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:26.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:26.761+0000 7fd239239740 1 -- 172.21.15.5:0/1739169269 >> 172.21.15.5:0/1739169269 conn(0x564ab6e97200 msgr2=0x564ab6e975e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:50:26.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:26.761+0000 7fd239239740 1 -- 172.21.15.5:0/1739169269 shutdown_connections 2024-09-10T09:50:26.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:26.761+0000 7fd239239740 1 -- 172.21.15.5:0/1739169269 wait complete. 2024-09-10T09:50:26.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:26.762+0000 7fd239239740 1 -- start start 2024-09-10T09:50:26.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:26.762+0000 7fd22f400640 1 --2- 172.21.15.5:0/3100910983 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fd2140a2c80 0x7fd2140a5140 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fd220000c00 tx=0x7fd220005b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:50:26.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:26.762+0000 7fd239239740 1 Processor -- start 2024-09-10T09:50:26.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:26.762+0000 7fd239239740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564ab6e9ac70 0x564ab6ea2d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:50:26.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:26.763+0000 7fd239239740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564ab6e9b590 0x564ab6ea3270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:50:26.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:26.763+0000 7fd239239740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564ab6ea5e50 0x564ab6ea37b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:50:26.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:26.763+0000 7fd239239740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x564ab6ea8a20 con 0x564ab6e9b590 2024-09-10T09:50:26.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:26.763+0000 7fd239239740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x564ab6dbc0f0 con 0x564ab6ea5e50 2024-09-10T09:50:26.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:26.763+0000 7fd239239740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x564ab6ea8ba0 con 0x564ab6e9ac70 2024-09-10T09:50:26.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:26.763+0000 7fd239239740 10 client.0 _ll_register_callbacks cb 0x564ab6da3bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:50:26.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:26.763+0000 7fd22f400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564ab6e9ac70 0x564ab6ea2d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:50:26.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:26.763+0000 7fd22f400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564ab6e9ac70 0x564ab6ea2d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:48666/0 (socket says 172.21.15.5:48666) 2024-09-10T09:50:26.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:26.763+0000 7fd22f400640 1 -- 172.21.15.5:0/1213775282 learned_addr learned my addr 172.21.15.5:0/1213775282 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:50:26.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:26.763+0000 7fd22fe00640 1 --2- 172.21.15.5:0/1213775282 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564ab6ea5e50 0x564ab6ea37b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:50:26.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:26.763+0000 7fd22ea00640 1 --2- 172.21.15.5:0/1213775282 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564ab6e9b590 0x564ab6ea3270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:50:26.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:26.763+0000 7fd22f400640 1 -- 172.21.15.5:0/1213775282 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564ab6ea5e50 msgr2=0x564ab6ea37b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:50:26.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:26.763+0000 7fd22f400640 1 --2- 172.21.15.5:0/1213775282 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564ab6ea5e50 0x564ab6ea37b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:26.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:26.763+0000 7fd22f400640 1 -- 172.21.15.5:0/1213775282 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564ab6e9b590 msgr2=0x564ab6ea3270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:50:26.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:26.763+0000 7fd22f400640 1 --2- 172.21.15.5:0/1213775282 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564ab6e9b590 0x564ab6ea3270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:26.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:26.764+0000 7fd22f400640 1 -- 172.21.15.5:0/1213775282 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564ab6b723c0 con 0x564ab6e9ac70 2024-09-10T09:50:26.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:26.764+0000 7fd22f400640 1 --2- 172.21.15.5:0/1213775282 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564ab6e9ac70 0x564ab6ea2d30 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fd220033000 tx=0x7fd220009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c15f5163ac36c81a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:50:26.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:26.764+0000 7fd219600640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-09-10T09:50:26.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:26.764+0000 7fd219600640 1 -- 172.21.15.5:0/1213775282 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd22004d070 con 0x564ab6e9ac70 2024-09-10T09:50:26.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:26.764+0000 7fd239239740 1 -- 172.21.15.5:0/1213775282 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd228011d80 con 0x564ab6e9ac70 2024-09-10T09:50:26.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:26.764+0000 7fd219600640 1 -- 172.21.15.5:0/1213775282 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fd2200048c0 con 0x564ab6e9ac70 2024-09-10T09:50:26.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:26.765+0000 7fd219600640 1 -- 172.21.15.5:0/1213775282 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fd220059050 con 0x564ab6e9ac70 2024-09-10T09:50:26.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:26.765+0000 7fd239239740 10 client.27805 fetch_fsmap learned FSMap version 36 2024-09-10T09:50:26.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:26.765+0000 7fd239239740 1 -- 172.21.15.5:0/1213775282 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x564ab6ebd2c0 con 0x564ab6e9ac70 2024-09-10T09:50:26.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:26.765+0000 7fd219600640 1 -- 172.21.15.5:0/1213775282 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd228011d80 con 0x564ab6e9ac70 2024-09-10T09:50:26.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:26.766+0000 7fd219600640 1 -- 172.21.15.5:0/1213775282 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 36) ==== 8370+0+0 (secure 0 0 0) 0x7fd2200035f0 con 0x564ab6e9ac70 2024-09-10T09:50:26.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:26.766+0000 7fd239239740 10 client.27805 fetch_fsmap finished waiting for FSMap version 36 2024-09-10T09:50:26.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:26.766+0000 7fd239239740 10 client.27805 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-09-10T09:50:26.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:26.766+0000 7fd239239740 20 client.27805 populate_metadata read hostname 'smithi005' 2024-09-10T09:50:26.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:26.766+0000 7fd239239740 1 --2- 172.21.15.5:0/1213775282 >> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] conn(0x564ab6ebe6d0 0x564ab6ec0b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:50:26.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:26.766+0000 7fd239239740 4 client.27805 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:50:26.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:26.766+0000 7fd239239740 1 -- 172.21.15.5:0/1213775282 --> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564ab6e0e950 con 0x564ab6ebe6d0 2024-09-10T09:50:26.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:26.766+0000 7fd22ea00640 1 --2- 172.21.15.5:0/1213775282 >> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] conn(0x564ab6ebe6d0 0x564ab6ec0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:50:26.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:26.767+0000 7fd22ea00640 1 --2- 172.21.15.5:0/1213775282 >> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] conn(0x564ab6ebe6d0 0x564ab6ec0b60 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=7a00f06bbf4e889c server_cookie=b603d7bd1484db86 in_seq=0 out_seq=0 2024-09-10T09:50:26.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:26.767+0000 7fd219600640 10 client.27805 ms_handle_connect on v2:172.21.15.102:6836/2689776764 2024-09-10T09:50:26.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:26.768+0000 7fd219600640 1 -- 172.21.15.5:0/1213775282 <== mds.3 v2:172.21.15.102:6836/2689776764 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564ab6e0e950 con 0x564ab6ebe6d0 2024-09-10T09:50:26.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:26.768+0000 7fd219600640 10 client.27805 handle_command_reply: tid=0 2024-09-10T09:50:26.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:26.768+0000 7fd239239740 1 client.27805 shutdown 2024-09-10T09:50:26.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:26.768+0000 7fd239239740 1 -- 172.21.15.5:0/1213775282 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564ab6e9ac70 msgr2=0x564ab6ea2d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:50:26.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:26.768+0000 7fd239239740 1 --2- 172.21.15.5:0/1213775282 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564ab6e9ac70 0x564ab6ea2d30 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fd220033000 tx=0x7fd220009260 comp rx=0 tx=0).stop 2024-09-10T09:50:26.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:26.768+0000 7fd239239740 1 -- 172.21.15.5:0/1213775282 shutdown_connections 2024-09-10T09:50:26.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:26.768+0000 7fd239239740 1 --2- 172.21.15.5:0/1213775282 >> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] conn(0x564ab6ebe6d0 0x564ab6ec0b60 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:26.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:26.768+0000 7fd239239740 1 --2- 172.21.15.5:0/1213775282 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564ab6ea5e50 0x564ab6ea37b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:26.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:26.768+0000 7fd239239740 1 --2- 172.21.15.5:0/1213775282 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564ab6e9b590 0x564ab6ea3270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:26.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:26.768+0000 7fd239239740 1 --2- 172.21.15.5:0/1213775282 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564ab6e9ac70 0x564ab6ea2d30 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:26.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:26.768+0000 7fd219600640 0 client.27805 ms_handle_reset on v2:172.21.15.102:6836/2689776764 2024-09-10T09:50:26.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:26.768+0000 7fd239239740 1 -- 172.21.15.5:0/1213775282 >> 172.21.15.5:0/1213775282 conn(0x564ab6e97200 msgr2=0x564ab6e96380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:50:26.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:26.768+0000 7fd239239740 1 -- 172.21.15.5:0/1213775282 shutdown_connections 2024-09-10T09:50:26.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:26.769+0000 7fd239239740 1 -- 172.21.15.5:0/1213775282 wait complete. 2024-09-10T09:50:26.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:26.769+0000 7fd239239740 20 client.27805 trim_cache size 0 max 16384 2024-09-10T09:50:26.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:26.783+0000 7fd239239740 1 Processor -- start 2024-09-10T09:50:26.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:26.783+0000 7fd239239740 1 -- start start 2024-09-10T09:50:26.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:26.783+0000 7fd239239740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564ab6e9ac70 0x564ab6e9d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:50:26.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:26.783+0000 7fd239239740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564ab6e9b590 0x564ab6ea1600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:50:26.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:26.783+0000 7fd239239740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564ab6ea5e50 0x564ab6ec6430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:50:26.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:26.783+0000 7fd239239740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd2301acc90 con 0x564ab6ea5e50 2024-09-10T09:50:26.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:26.783+0000 7fd239239740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fd2301ae7d0 con 0x564ab6e9ac70 2024-09-10T09:50:26.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:26.783+0000 7fd239239740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fd2301ad360 con 0x564ab6e9b590 2024-09-10T09:50:26.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:26.784+0000 7fd22ea00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564ab6e9b590 0x564ab6ea1600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:50:26.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:26.784+0000 7fd22f400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564ab6e9ac70 0x564ab6e9d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:50:26.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:26.784+0000 7fd22ea00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564ab6e9b590 0x564ab6ea1600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:48682/0 (socket says 172.21.15.5:48682) 2024-09-10T09:50:26.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:26.784+0000 7fd22fe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564ab6ea5e50 0x564ab6ec6430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:50:26.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:26.784+0000 7fd22ea00640 1 -- 172.21.15.5:0/3209082279 learned_addr learned my addr 172.21.15.5:0/3209082279 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:50:26.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:26.784+0000 7fd22ea00640 1 -- 172.21.15.5:0/3209082279 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564ab6e9ac70 msgr2=0x564ab6e9d6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:50:26.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:26.784+0000 7fd22ea00640 1 --2- 172.21.15.5:0/3209082279 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564ab6e9ac70 0x564ab6e9d6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:26.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:26.784+0000 7fd22ea00640 1 -- 172.21.15.5:0/3209082279 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564ab6ea5e50 msgr2=0x564ab6ec6430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:50:26.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:26.784+0000 7fd22ea00640 1 --2- 172.21.15.5:0/3209082279 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564ab6ea5e50 0x564ab6ec6430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:26.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:26.784+0000 7fd22ea00640 1 -- 172.21.15.5:0/3209082279 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564ab6e9b3c0 con 0x564ab6e9b590 2024-09-10T09:50:26.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:26.784+0000 7fd22f400640 1 --2- 172.21.15.5:0/3209082279 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564ab6e9ac70 0x564ab6e9d6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:50:26.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:26.784+0000 7fd22ea00640 1 --2- 172.21.15.5:0/3209082279 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564ab6e9b590 0x564ab6ea1600 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fd21c098020 tx=0x7fd21c005f10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=30c7ac7e46fd4d67 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:50:26.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:26.785+0000 7fd21be00640 1 -- 172.21.15.5:0/3209082279 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd21c09ff70 con 0x564ab6e9b590 2024-09-10T09:50:26.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:26.785+0000 7fd21be00640 1 -- 172.21.15.5:0/3209082279 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd21c0a4830 con 0x564ab6e9b590 2024-09-10T09:50:26.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:26.785+0000 7fd21be00640 1 -- 172.21.15.5:0/3209082279 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd21c0cfbe0 con 0x564ab6e9b590 2024-09-10T09:50:26.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:26.785+0000 7fd239239740 1 -- 172.21.15.5:0/3209082279 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564ab6e9b590 msgr2=0x564ab6ea1600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:50:26.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:26.785+0000 7fd239239740 1 --2- 172.21.15.5:0/3209082279 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564ab6e9b590 0x564ab6ea1600 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fd21c098020 tx=0x7fd21c005f10 comp rx=0 tx=0).stop 2024-09-10T09:50:26.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:26.786+0000 7fd239239740 1 -- 172.21.15.5:0/3209082279 shutdown_connections 2024-09-10T09:50:26.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:26.786+0000 7fd239239740 1 --2- 172.21.15.5:0/3209082279 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564ab6ea5e50 0x564ab6ec6430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:26.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:26.786+0000 7fd239239740 1 --2- 172.21.15.5:0/3209082279 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564ab6e9b590 0x564ab6ea1600 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:26.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:26.786+0000 7fd239239740 1 --2- 172.21.15.5:0/3209082279 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564ab6e9ac70 0x564ab6e9d6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:26.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:26.786+0000 7fd239239740 1 -- 172.21.15.5:0/3209082279 >> 172.21.15.5:0/3209082279 conn(0x564ab6e97200 msgr2=0x564ab6ea4db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:50:26.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:26.786+0000 7fd239239740 1 -- 172.21.15.5:0/3209082279 shutdown_connections 2024-09-10T09:50:26.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:26.786+0000 7fd239239740 1 -- 172.21.15.5:0/3209082279 wait complete. 2024-09-10T09:50:26.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:26.786+0000 7fd239239740 1 -- start start 2024-09-10T09:50:26.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:26.786+0000 7fd239239740 1 Processor -- start 2024-09-10T09:50:26.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:26.787+0000 7fd239239740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564ab6e9ac70 0x564ab6eb6ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:50:26.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:26.787+0000 7fd239239740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564ab6e9b590 0x564ab6e9f720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:50:26.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:26.787+0000 7fd239239740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564ab6ea5e50 0x564ab6e9fc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:50:26.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:26.787+0000 7fd239239740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd2301ae100 con 0x564ab6e9b590 2024-09-10T09:50:26.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:26.787+0000 7fd239239740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fd2301ada30 con 0x564ab6ea5e50 2024-09-10T09:50:26.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:26.787+0000 7fd239239740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fd2301aac70 con 0x564ab6e9ac70 2024-09-10T09:50:26.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:26.787+0000 7fd239239740 10 client.0 _ll_register_callbacks cb 0x7fd2280093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:50:26.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:26.787+0000 7fd22f400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564ab6e9ac70 0x564ab6eb6ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:50:26.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:26.787+0000 7fd22ea00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564ab6e9b590 0x564ab6e9f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:50:26.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:26.787+0000 7fd22f400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564ab6e9ac70 0x564ab6eb6ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:48696/0 (socket says 172.21.15.5:48696) 2024-09-10T09:50:26.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:26.787+0000 7fd22f400640 1 -- 172.21.15.5:0/785335296 learned_addr learned my addr 172.21.15.5:0/785335296 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:50:26.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:26.787+0000 7fd22fe00640 1 --2- 172.21.15.5:0/785335296 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564ab6ea5e50 0x564ab6e9fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:50:26.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:26.788+0000 7fd22f400640 1 -- 172.21.15.5:0/785335296 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564ab6ea5e50 msgr2=0x564ab6e9fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:50:26.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:26.788+0000 7fd22f400640 1 --2- 172.21.15.5:0/785335296 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564ab6ea5e50 0x564ab6e9fc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:26.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:26.788+0000 7fd22f400640 1 -- 172.21.15.5:0/785335296 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564ab6e9b590 msgr2=0x564ab6e9f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:50:26.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:26.788+0000 7fd22f400640 1 --2- 172.21.15.5:0/785335296 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564ab6e9b590 0x564ab6e9f720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:26.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:26.788+0000 7fd22f400640 1 -- 172.21.15.5:0/785335296 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564ab6ea01d0 con 0x564ab6e9ac70 2024-09-10T09:50:26.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:26.788+0000 7fd22fe00640 1 --2- 172.21.15.5:0/785335296 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564ab6ea5e50 0x564ab6e9fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:50:26.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:26.788+0000 7fd22f400640 1 --2- 172.21.15.5:0/785335296 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564ab6e9ac70 0x564ab6eb6ae0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fd2200039c0 tx=0x7fd220004210 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c0716ae12a7d5ded server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:50:26.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:26.788+0000 7fd219600640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-09-10T09:50:26.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:26.788+0000 7fd219600640 1 -- 172.21.15.5:0/785335296 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd220061020 con 0x564ab6e9ac70 2024-09-10T09:50:26.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:26.788+0000 7fd239239740 1 -- 172.21.15.5:0/785335296 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd21c0a4830 con 0x564ab6e9ac70 2024-09-10T09:50:26.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:26.788+0000 7fd219600640 1 -- 172.21.15.5:0/785335296 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fd2200046c0 con 0x564ab6e9ac70 2024-09-10T09:50:26.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:26.789+0000 7fd219600640 1 -- 172.21.15.5:0/785335296 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fd22006c050 con 0x564ab6e9ac70 2024-09-10T09:50:26.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:26.789+0000 7fd239239740 10 client.27811 fetch_fsmap learned FSMap version 36 2024-09-10T09:50:26.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:26.789+0000 7fd239239740 1 -- 172.21.15.5:0/785335296 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x564ab6f29140 con 0x564ab6e9ac70 2024-09-10T09:50:26.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:26.789+0000 7fd219600640 1 -- 172.21.15.5:0/785335296 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd21c0a4830 con 0x564ab6e9ac70 2024-09-10T09:50:26.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:26.789+0000 7fd219600640 1 -- 172.21.15.5:0/785335296 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 36) ==== 8370+0+0 (secure 0 0 0) 0x7fd220032960 con 0x564ab6e9ac70 2024-09-10T09:50:26.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:26.789+0000 7fd239239740 10 client.27811 fetch_fsmap finished waiting for FSMap version 36 2024-09-10T09:50:26.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:26.789+0000 7fd239239740 10 client.27811 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-09-10T09:50:26.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:26.789+0000 7fd239239740 20 client.27811 populate_metadata read hostname 'smithi005' 2024-09-10T09:50:26.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:26.789+0000 7fd239239740 1 --2- 172.21.15.5:0/785335296 >> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] conn(0x564ab6f29850 0x564ab6f2bc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:50:26.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:26.789+0000 7fd239239740 4 client.27811 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T09:50:26.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:26.789+0000 7fd239239740 1 -- 172.21.15.5:0/785335296 --> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564ab6e08e60 con 0x564ab6f29850 2024-09-10T09:50:26.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:26.789+0000 7fd22ea00640 1 --2- 172.21.15.5:0/785335296 >> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] conn(0x564ab6f29850 0x564ab6f2bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:50:26.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:26.790+0000 7fd22ea00640 1 --2- 172.21.15.5:0/785335296 >> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] conn(0x564ab6f29850 0x564ab6f2bc70 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=2f5892ef02cb79f3 server_cookie=9e1844071dd5ee61 in_seq=0 out_seq=0 2024-09-10T09:50:26.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:26.790+0000 7fd219600640 10 client.27811 ms_handle_connect on v2:172.21.15.102:6836/2689776764 2024-09-10T09:50:26.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:26.790+0000 7fd219600640 1 -- 172.21.15.5:0/785335296 <== mds.3 v2:172.21.15.102:6836/2689776764 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564ab6e08e60 con 0x564ab6f29850 2024-09-10T09:50:26.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:26.790+0000 7fd219600640 10 client.27811 handle_command_reply: tid=0 2024-09-10T09:50:26.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:26.790+0000 7fd239239740 1 client.27811 shutdown 2024-09-10T09:50:26.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:26.790+0000 7fd239239740 1 -- 172.21.15.5:0/785335296 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564ab6e9ac70 msgr2=0x564ab6eb6ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:50:26.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:26.790+0000 7fd239239740 1 --2- 172.21.15.5:0/785335296 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564ab6e9ac70 0x564ab6eb6ae0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fd2200039c0 tx=0x7fd220004210 comp rx=0 tx=0).stop 2024-09-10T09:50:26.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:26.791+0000 7fd239239740 1 -- 172.21.15.5:0/785335296 shutdown_connections 2024-09-10T09:50:26.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:26.791+0000 7fd239239740 1 --2- 172.21.15.5:0/785335296 >> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] conn(0x564ab6f29850 0x564ab6f2bc70 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:26.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:26.791+0000 7fd239239740 1 --2- 172.21.15.5:0/785335296 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564ab6ea5e50 0x564ab6e9fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:26.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:26.791+0000 7fd239239740 1 --2- 172.21.15.5:0/785335296 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564ab6e9b590 0x564ab6e9f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:26.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:26.791+0000 7fd239239740 1 --2- 172.21.15.5:0/785335296 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564ab6e9ac70 0x564ab6eb6ae0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:26.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:26.791+0000 7fd219600640 0 client.27811 ms_handle_reset on v2:172.21.15.102:6836/2689776764 2024-09-10T09:50:26.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:26.791+0000 7fd239239740 1 -- 172.21.15.5:0/785335296 >> 172.21.15.5:0/785335296 conn(0x564ab6e97200 msgr2=0x564ab6e95b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:50:26.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:26.791+0000 7fd239239740 1 -- 172.21.15.5:0/785335296 shutdown_connections 2024-09-10T09:50:26.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:26.791+0000 7fd239239740 1 -- 172.21.15.5:0/785335296 wait complete. 2024-09-10T09:50:26.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:26.791+0000 7fd239239740 20 client.27811 trim_cache size 0 max 16384 2024-09-10T09:50:26.796 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-10T09:50:26.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:26.791+0000 7fd236600640 1 -- 172.21.15.5:0/3100910983 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fd2140a2c80 msgr2=0x7fd2140a5140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:50:26.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:26.791+0000 7fd236600640 1 --2- 172.21.15.5:0/3100910983 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fd2140a2c80 0x7fd2140a5140 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fd220000c00 tx=0x7fd220005b50 comp rx=0 tx=0).stop 2024-09-10T09:50:26.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:26.791+0000 7fd236600640 1 -- 172.21.15.5:0/3100910983 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd2301049c0 msgr2=0x7fd23019abf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:50:26.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:26.791+0000 7fd236600640 1 --2- 172.21.15.5:0/3100910983 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd2301049c0 0x7fd23019abf0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fd21c00e9f0 tx=0x7fd21c00eeb0 comp rx=0 tx=0).stop 2024-09-10T09:50:26.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:26.792+0000 7fd236600640 1 -- 172.21.15.5:0/3100910983 shutdown_connections 2024-09-10T09:50:26.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:26.792+0000 7fd236600640 1 --2- 172.21.15.5:0/3100910983 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fd2140a2c80 0x7fd2140a5140 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:26.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:26.792+0000 7fd236600640 1 --2- 172.21.15.5:0/3100910983 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd230109140 0x7fd23019ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:26.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:26.792+0000 7fd236600640 1 --2- 172.21.15.5:0/3100910983 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd2301049c0 0x7fd23019abf0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:26.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:26.792+0000 7fd236600640 1 --2- 172.21.15.5:0/3100910983 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd230104010 0x7fd23019a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:26.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:26.792+0000 7fd236600640 1 -- 172.21.15.5:0/3100910983 >> 172.21.15.5:0/3100910983 conn(0x7fd2300ffa50 msgr2=0x7fd230100de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:50:26.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:26.792+0000 7fd236600640 1 -- 172.21.15.5:0/3100910983 shutdown_connections 2024-09-10T09:50:26.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:26.792+0000 7fd236600640 1 -- 172.21.15.5:0/3100910983 wait complete. 2024-09-10T09:50:27.134 INFO:tasks.workunit.client.0.smithi005.stdout: 1 771987 6.44 MB/sec execute 488 sec latency 31.542 ms 2024-09-10T09:50:27.593 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-10T09:50:27.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:50:27 smithi005 ceph-mon[30350]: pgmap v756: 129 pgs: 129 active+clean; 290 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 1.9 MiB/s wr, 170 op/s 2024-09-10T09:50:27.996 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:50:28.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:50:27 smithi159 ceph-mon[35513]: pgmap v756: 129 pgs: 129 active+clean; 290 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 1.9 MiB/s wr, 170 op/s 2024-09-10T09:50:28.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:50:27 smithi102 ceph-mon[35442]: pgmap v756: 129 pgs: 129 active+clean; 290 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 1.9 MiB/s wr, 170 op/s 2024-09-10T09:50:28.134 INFO:tasks.workunit.client.0.smithi005.stdout: 1 774419 6.46 MB/sec execute 489 sec latency 18.297 ms 2024-09-10T09:50:28.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:28.844+0000 7f784de00640 1 -- 172.21.15.5:0/1546537300 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f78481049a0 msgr2=0x7f78481089f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:50:28.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:28.844+0000 7f784de00640 1 --2- 172.21.15.5:0/1546537300 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f78481049a0 0x7f78481089f0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f7834009900 tx=0x7f783402a410 comp rx=0 tx=0).stop 2024-09-10T09:50:28.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:28.844+0000 7f784de00640 1 -- 172.21.15.5:0/1546537300 shutdown_connections 2024-09-10T09:50:28.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:28.844+0000 7f784de00640 1 --2- 172.21.15.5:0/1546537300 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7848109120 0x7f7848110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:28.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:28.844+0000 7f784de00640 1 --2- 172.21.15.5:0/1546537300 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f78481049a0 0x7f78481089f0 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:28.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:28.844+0000 7f784de00640 1 --2- 172.21.15.5:0/1546537300 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7848103ff0 0x7f78481043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:28.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:28.844+0000 7f784de00640 1 -- 172.21.15.5:0/1546537300 >> 172.21.15.5:0/1546537300 conn(0x7f78480ffa90 msgr2=0x7f7848101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:50:28.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:28.844+0000 7f784de00640 1 -- 172.21.15.5:0/1546537300 shutdown_connections 2024-09-10T09:50:28.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:28.844+0000 7f784de00640 1 -- 172.21.15.5:0/1546537300 wait complete. 2024-09-10T09:50:28.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:28.845+0000 7f784de00640 1 Processor -- start 2024-09-10T09:50:28.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:28.845+0000 7f784de00640 1 -- start start 2024-09-10T09:50:28.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:28.846+0000 7f784de00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7848103ff0 0x7f784819a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:50:28.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:28.846+0000 7f784de00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f78481049a0 0x7f784819ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:50:28.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:28.846+0000 7f784de00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7848109120 0x7f784819ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:50:28.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:28.846+0000 7f784de00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f7848114170 con 0x7f7848103ff0 2024-09-10T09:50:28.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:28.846+0000 7f784de00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f7848113ff0 con 0x7f78481049a0 2024-09-10T09:50:28.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:28.846+0000 7f784de00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f78481142f0 con 0x7f7848109120 2024-09-10T09:50:28.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:28.846+0000 7f7846a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7848103ff0 0x7f784819a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:50:28.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:28.846+0000 7f7846000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f78481049a0 0x7f784819ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:50:28.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:28.846+0000 7f7846a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7848103ff0 0x7f784819a650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35648/0 (socket says 172.21.15.5:35648) 2024-09-10T09:50:28.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:28.846+0000 7f7846a00640 1 -- 172.21.15.5:0/2180631267 learned_addr learned my addr 172.21.15.5:0/2180631267 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:50:28.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:28.846+0000 7f7846000640 1 -- 172.21.15.5:0/2180631267 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7848109120 msgr2=0x7f784819ef20 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-10T09:50:28.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:28.846+0000 7f7846000640 1 --2- 172.21.15.5:0/2180631267 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7848109120 0x7f784819ef20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:28.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:28.847+0000 7f7846000640 1 -- 172.21.15.5:0/2180631267 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7848103ff0 msgr2=0x7f784819a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:50:28.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:28.847+0000 7f7846000640 1 --2- 172.21.15.5:0/2180631267 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7848103ff0 0x7f784819a650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:28.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:28.847+0000 7f7846000640 1 -- 172.21.15.5:0/2180631267 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f784819f600 con 0x7f78481049a0 2024-09-10T09:50:28.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:28.847+0000 7f7846000640 1 --2- 172.21.15.5:0/2180631267 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f78481049a0 0x7f784819ab90 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f783402a930 tx=0x7f7834003b20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:50:28.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:28.847+0000 7f784ca00640 1 -- 172.21.15.5:0/2180631267 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f783403f070 con 0x7f78481049a0 2024-09-10T09:50:28.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:28.847+0000 7f784de00640 1 -- 172.21.15.5:0/2180631267 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f784819f890 con 0x7f78481049a0 2024-09-10T09:50:28.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:28.847+0000 7f784ca00640 1 -- 172.21.15.5:0/2180631267 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7834031c90 con 0x7f78481049a0 2024-09-10T09:50:28.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:28.847+0000 7f784ca00640 1 -- 172.21.15.5:0/2180631267 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7834035030 con 0x7f78481049a0 2024-09-10T09:50:28.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:28.847+0000 7f784de00640 1 -- 172.21.15.5:0/2180631267 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78481a71d0 con 0x7f78481049a0 2024-09-10T09:50:28.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:28.851+0000 7f7850860740 1 Processor -- start 2024-09-10T09:50:28.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:28.851+0000 7f7850860740 1 -- start start 2024-09-10T09:50:28.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:28.851+0000 7f7850860740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5572c74a2c70 0x5572c74a3050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:50:28.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:28.851+0000 7f7850860740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5572c74a3590 0x5572c74ad910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:50:28.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:28.851+0000 7f7850860740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5572c74ade50 0x5572c74b0240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:50:28.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:28.851+0000 7f7850860740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5572c74093e0 con 0x5572c74ade50 2024-09-10T09:50:28.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:28.851+0000 7f7850860740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x5572c73b0d80 con 0x5572c74a2c70 2024-09-10T09:50:28.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:28.851+0000 7f7850860740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5572c74053a0 con 0x5572c74a3590 2024-09-10T09:50:28.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:28.851+0000 7f7846a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5572c74a3590 0x5572c74ad910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:50:28.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:28.851+0000 7f7846a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5572c74a3590 0x5572c74ad910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:48714/0 (socket says 172.21.15.5:48714) 2024-09-10T09:50:28.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:28.851+0000 7f7846a00640 1 -- 172.21.15.5:0/2037025531 learned_addr learned my addr 172.21.15.5:0/2037025531 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:50:28.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:28.852+0000 7f784ca00640 1 -- 172.21.15.5:0/2180631267 <== mon.2 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f7834042070 con 0x7f78481049a0 2024-09-10T09:50:28.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:28.852+0000 7f7846a00640 1 -- 172.21.15.5:0/2037025531 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5572c74a2c70 msgr2=0x5572c74a3050 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T09:50:28.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:28.852+0000 7f7846a00640 1 --2- 172.21.15.5:0/2037025531 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5572c74a2c70 0x5572c74a3050 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:28.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:28.852+0000 7f7846a00640 1 -- 172.21.15.5:0/2037025531 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5572c74ade50 msgr2=0x5572c74b0240 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T09:50:28.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:28.852+0000 7f7846a00640 1 --2- 172.21.15.5:0/2037025531 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5572c74ade50 0x5572c74b0240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:28.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:28.852+0000 7f7846a00640 1 -- 172.21.15.5:0/2037025531 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5572c73cfae0 con 0x5572c74a3590 2024-09-10T09:50:28.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:28.852+0000 7f7846a00640 1 --2- 172.21.15.5:0/2037025531 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5572c74a3590 0x5572c74ad910 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f7838009d40 tx=0x7f7838011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=11936b02045f4842 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:50:28.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:28.853+0000 7f7833e00640 1 -- 172.21.15.5:0/2037025531 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7838005220 con 0x5572c74a3590 2024-09-10T09:50:28.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:28.853+0000 7f7833e00640 1 -- 172.21.15.5:0/2037025531 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7838004c80 con 0x5572c74a3590 2024-09-10T09:50:28.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:28.853+0000 7f7833e00640 1 -- 172.21.15.5:0/2037025531 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7838019840 con 0x5572c74a3590 2024-09-10T09:50:28.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:28.853+0000 7f7850860740 1 -- 172.21.15.5:0/2037025531 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5572c74a3590 msgr2=0x5572c74ad910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:50:28.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:28.853+0000 7f7850860740 1 --2- 172.21.15.5:0/2037025531 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5572c74a3590 0x5572c74ad910 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f7838009d40 tx=0x7f7838011810 comp rx=0 tx=0).stop 2024-09-10T09:50:28.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:28.853+0000 7f784ca00640 1 --2- 172.21.15.5:0/2180631267 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f782c0a2b80 0x7f782c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:50:28.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:28.854+0000 7f7850860740 1 -- 172.21.15.5:0/2037025531 shutdown_connections 2024-09-10T09:50:28.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:28.854+0000 7f784ca00640 1 -- 172.21.15.5:0/2180631267 <== mon.2 v2:172.21.15.102:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f78340efa50 con 0x7f78481049a0 2024-09-10T09:50:28.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:28.854+0000 7f7850860740 1 --2- 172.21.15.5:0/2037025531 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5572c74ade50 0x5572c74b0240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:28.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:28.854+0000 7f7850860740 1 --2- 172.21.15.5:0/2037025531 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5572c74a3590 0x5572c74ad910 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:28.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:28.854+0000 7f7850860740 1 --2- 172.21.15.5:0/2037025531 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5572c74a2c70 0x5572c74a3050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:28.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:28.854+0000 7f7850860740 1 -- 172.21.15.5:0/2037025531 >> 172.21.15.5:0/2037025531 conn(0x5572c749f200 msgr2=0x5572c749f5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:50:28.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:28.854+0000 7f7850860740 1 -- 172.21.15.5:0/2037025531 shutdown_connections 2024-09-10T09:50:28.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:28.854+0000 7f7847400640 1 --2- 172.21.15.5:0/2180631267 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f782c0a2b80 0x7f782c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:50:28.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:28.854+0000 7f7850860740 1 -- 172.21.15.5:0/2037025531 wait complete. 2024-09-10T09:50:28.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:28.854+0000 7f7850860740 1 -- start start 2024-09-10T09:50:28.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:28.855+0000 7f7850860740 1 Processor -- start 2024-09-10T09:50:28.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:28.855+0000 7f7847400640 1 --2- 172.21.15.5:0/2180631267 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f782c0a2b80 0x7f782c0a5040 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f78400099a0 tx=0x7f7840009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:50:28.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:28.855+0000 7f7850860740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5572c74a2c70 0x5572c74aad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:50:28.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:28.855+0000 7f7850860740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5572c74a3590 0x5572c74ab250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:50:28.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:28.855+0000 7f7850860740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5572c74ade50 0x5572c74ab790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:50:28.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:28.855+0000 7f7850860740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5572c74b0a20 con 0x5572c74a3590 2024-09-10T09:50:28.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:28.855+0000 7f7850860740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x5572c73c40f0 con 0x5572c74a2c70 2024-09-10T09:50:28.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:28.855+0000 7f7850860740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5572c74b0ba0 con 0x5572c74ade50 2024-09-10T09:50:28.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:28.855+0000 7f7850860740 10 client.0 _ll_register_callbacks cb 0x5572c73abbd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:50:28.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:28.855+0000 7f7846000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5572c74a3590 0x5572c74ab250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:50:28.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:28.856+0000 7f7846000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5572c74a3590 0x5572c74ab250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35658/0 (socket says 172.21.15.5:35658) 2024-09-10T09:50:28.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:28.856+0000 7f7846000640 1 -- 172.21.15.5:0/294488110 learned_addr learned my addr 172.21.15.5:0/294488110 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:50:28.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:28.856+0000 7f7847400640 1 --2- 172.21.15.5:0/294488110 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5572c74ade50 0x5572c74ab790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:50:28.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:28.856+0000 7f7846a00640 1 --2- 172.21.15.5:0/294488110 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5572c74a2c70 0x5572c74aad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:50:28.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:28.856+0000 7f7846000640 1 -- 172.21.15.5:0/294488110 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5572c74ade50 msgr2=0x5572c74ab790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:50:28.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:28.856+0000 7f7846000640 1 --2- 172.21.15.5:0/294488110 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5572c74ade50 0x5572c74ab790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:28.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:28.856+0000 7f7846000640 1 -- 172.21.15.5:0/294488110 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5572c74a2c70 msgr2=0x5572c74aad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:50:28.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:28.856+0000 7f7846000640 1 --2- 172.21.15.5:0/294488110 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5572c74a2c70 0x5572c74aad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:28.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:28.856+0000 7f7846000640 1 -- 172.21.15.5:0/294488110 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5572c717a3c0 con 0x5572c74a3590 2024-09-10T09:50:28.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:28.856+0000 7f7847400640 1 --2- 172.21.15.5:0/294488110 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5572c74ade50 0x5572c74ab790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:50:28.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:28.856+0000 7f7846000640 1 --2- 172.21.15.5:0/294488110 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5572c74a3590 0x5572c74ab250 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f78340bbe50 tx=0x7f78340b7b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2931b589d84483cc server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:50:28.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:28.856+0000 7f7830c00640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-10T09:50:28.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:28.856+0000 7f7830c00640 1 -- 172.21.15.5:0/294488110 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7834049050 con 0x5572c74a3590 2024-09-10T09:50:28.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:28.857+0000 7f7830c00640 1 -- 172.21.15.5:0/294488110 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f78340b8530 con 0x5572c74a3590 2024-09-10T09:50:28.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:28.857+0000 7f7850860740 1 -- 172.21.15.5:0/294488110 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7838004c80 con 0x5572c74a3590 2024-09-10T09:50:28.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:28.857+0000 7f7830c00640 1 -- 172.21.15.5:0/294488110 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f78340bb820 con 0x5572c74a3590 2024-09-10T09:50:28.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:28.857+0000 7f7850860740 10 client.17796 fetch_fsmap learned FSMap version 36 2024-09-10T09:50:28.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:28.857+0000 7f7850860740 1 -- 172.21.15.5:0/294488110 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x5572c74bca30 con 0x5572c74a3590 2024-09-10T09:50:28.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:28.857+0000 7f7830c00640 1 -- 172.21.15.5:0/294488110 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7838004c80 con 0x5572c74a3590 2024-09-10T09:50:28.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:28.857+0000 7f7830c00640 1 -- 172.21.15.5:0/294488110 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 36) ==== 8370+0+0 (secure 0 0 0) 0x7f78340bc120 con 0x5572c74a3590 2024-09-10T09:50:28.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:28.857+0000 7f7850860740 10 client.17796 fetch_fsmap finished waiting for FSMap version 36 2024-09-10T09:50:28.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:28.857+0000 7f7850860740 10 client.17796 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-10T09:50:28.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:28.857+0000 7f7850860740 20 client.17796 populate_metadata read hostname 'smithi005' 2024-09-10T09:50:28.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:28.857+0000 7f7850860740 1 --2- 172.21.15.5:0/294488110 >> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] conn(0x5572c74bdec0 0x5572c74c02b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:50:28.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:28.857+0000 7f7850860740 4 client.17796 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:50:28.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:28.857+0000 7f7850860740 1 -- 172.21.15.5:0/294488110 --> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5572c7416950 con 0x5572c74bdec0 2024-09-10T09:50:28.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:28.858+0000 7f7846a00640 1 --2- 172.21.15.5:0/294488110 >> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] conn(0x5572c74bdec0 0x5572c74c02b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:50:28.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:28.859+0000 7f7846a00640 1 --2- 172.21.15.5:0/294488110 >> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] conn(0x5572c74bdec0 0x5572c74c02b0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6decabdbe25cfa98 server_cookie=468ba30258c87389 in_seq=0 out_seq=0 2024-09-10T09:50:28.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:28.859+0000 7f7830c00640 10 client.17796 ms_handle_connect on v2:172.21.15.159:6836/620627639 2024-09-10T09:50:28.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:28.865+0000 7f7830c00640 1 -- 172.21.15.5:0/294488110 <== mds.2 v2:172.21.15.159:6836/620627639 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5572c7416950 con 0x5572c74bdec0 2024-09-10T09:50:28.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:28.865+0000 7f7830c00640 10 client.17796 handle_command_reply: tid=0 2024-09-10T09:50:28.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:28.865+0000 7f7850860740 1 client.17796 shutdown 2024-09-10T09:50:28.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:28.866+0000 7f7850860740 1 -- 172.21.15.5:0/294488110 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5572c74a3590 msgr2=0x5572c74ab250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:50:28.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:28.866+0000 7f7850860740 1 --2- 172.21.15.5:0/294488110 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5572c74a3590 0x5572c74ab250 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f78340bbe50 tx=0x7f78340b7b00 comp rx=0 tx=0).stop 2024-09-10T09:50:28.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:28.866+0000 7f7850860740 1 -- 172.21.15.5:0/294488110 shutdown_connections 2024-09-10T09:50:28.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:28.866+0000 7f7850860740 1 --2- 172.21.15.5:0/294488110 >> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] conn(0x5572c74bdec0 0x5572c74c02b0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:28.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:28.866+0000 7f7850860740 1 --2- 172.21.15.5:0/294488110 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5572c74ade50 0x5572c74ab790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:28.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:28.866+0000 7f7850860740 1 --2- 172.21.15.5:0/294488110 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5572c74a3590 0x5572c74ab250 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:28.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:28.866+0000 7f7850860740 1 --2- 172.21.15.5:0/294488110 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5572c74a2c70 0x5572c74aad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:28.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:28.866+0000 7f7830c00640 0 client.17796 ms_handle_reset on v2:172.21.15.159:6836/620627639 2024-09-10T09:50:28.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:28.866+0000 7f7850860740 1 -- 172.21.15.5:0/294488110 >> 172.21.15.5:0/294488110 conn(0x5572c749f200 msgr2=0x5572c74a4ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:50:28.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:28.867+0000 7f7850860740 1 -- 172.21.15.5:0/294488110 shutdown_connections 2024-09-10T09:50:28.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:28.867+0000 7f7850860740 1 -- 172.21.15.5:0/294488110 wait complete. 2024-09-10T09:50:28.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:28.867+0000 7f7850860740 20 client.17796 trim_cache size 0 max 16384 2024-09-10T09:50:28.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:28.895+0000 7f7850860740 1 Processor -- start 2024-09-10T09:50:28.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:28.896+0000 7f7850860740 1 -- start start 2024-09-10T09:50:28.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:28.896+0000 7f7850860740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5572c74a2c70 0x5572c74c5b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:50:28.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:28.896+0000 7f7850860740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5572c74a3590 0x5572c74c60c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:50:28.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:28.896+0000 7f7850860740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5572c74ade50 0x5572c74a7f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:50:28.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:28.896+0000 7f7850860740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f78481b1090 con 0x5572c74a2c70 2024-09-10T09:50:28.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:28.896+0000 7f7850860740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f78481b2bd0 con 0x5572c74a3590 2024-09-10T09:50:28.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:28.896+0000 7f7850860740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f78481b1760 con 0x5572c74ade50 2024-09-10T09:50:28.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:28.897+0000 7f7846000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5572c74a3590 0x5572c74c60c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:50:28.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:28.897+0000 7f7846a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5572c74a2c70 0x5572c74c5b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:50:28.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:28.897+0000 7f7846000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5572c74a3590 0x5572c74c60c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:54102/0 (socket says 172.21.15.5:54102) 2024-09-10T09:50:28.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:28.897+0000 7f7846000640 1 -- 172.21.15.5:0/607015824 learned_addr learned my addr 172.21.15.5:0/607015824 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:50:28.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:28.897+0000 7f7847400640 1 --2- 172.21.15.5:0/607015824 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5572c74ade50 0x5572c74a7f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:50:28.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:28.898+0000 7f7846000640 1 -- 172.21.15.5:0/607015824 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5572c74ade50 msgr2=0x5572c74a7f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:50:28.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:28.898+0000 7f7846000640 1 --2- 172.21.15.5:0/607015824 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5572c74ade50 0x5572c74a7f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:28.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:28.898+0000 7f7846000640 1 -- 172.21.15.5:0/607015824 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5572c74a2c70 msgr2=0x5572c74c5b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:50:28.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:28.898+0000 7f7846000640 1 --2- 172.21.15.5:0/607015824 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5572c74a2c70 0x5572c74c5b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:28.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:28.898+0000 7f7846000640 1 -- 172.21.15.5:0/607015824 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5572c749a770 con 0x5572c74a3590 2024-09-10T09:50:28.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:28.898+0000 7f7846a00640 1 --2- 172.21.15.5:0/607015824 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5572c74a2c70 0x5572c74c5b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:50:28.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:28.899+0000 7f7846000640 1 --2- 172.21.15.5:0/607015824 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5572c74a3590 0x5572c74c60c0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f7834009900 tx=0x7f78340ba3d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a68f9a052155cc7c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:50:28.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:28.900+0000 7f7833400640 1 -- 172.21.15.5:0/607015824 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7834049070 con 0x5572c74a3590 2024-09-10T09:50:28.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:28.900+0000 7f7833400640 1 -- 172.21.15.5:0/607015824 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f78340bc5c0 con 0x5572c74a3590 2024-09-10T09:50:28.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:28.900+0000 7f7833400640 1 -- 172.21.15.5:0/607015824 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78340b8530 con 0x5572c74a3590 2024-09-10T09:50:28.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:28.901+0000 7f7850860740 1 -- 172.21.15.5:0/607015824 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5572c74a3590 msgr2=0x5572c74c60c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:50:28.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:28.901+0000 7f7850860740 1 --2- 172.21.15.5:0/607015824 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5572c74a3590 0x5572c74c60c0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f7834009900 tx=0x7f78340ba3d0 comp rx=0 tx=0).stop 2024-09-10T09:50:28.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:28.902+0000 7f7850860740 1 -- 172.21.15.5:0/607015824 shutdown_connections 2024-09-10T09:50:28.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:28.902+0000 7f7850860740 1 --2- 172.21.15.5:0/607015824 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5572c74ade50 0x5572c74a7f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:28.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:28.902+0000 7f7850860740 1 --2- 172.21.15.5:0/607015824 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5572c74a3590 0x5572c74c60c0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:28.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:28.902+0000 7f7850860740 1 --2- 172.21.15.5:0/607015824 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5572c74a2c70 0x5572c74c5b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:28.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:28.902+0000 7f7850860740 1 -- 172.21.15.5:0/607015824 >> 172.21.15.5:0/607015824 conn(0x5572c749f200 msgr2=0x5572c74acd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:50:28.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:28.902+0000 7f7850860740 1 -- 172.21.15.5:0/607015824 shutdown_connections 2024-09-10T09:50:28.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:28.902+0000 7f7850860740 1 -- 172.21.15.5:0/607015824 wait complete. 2024-09-10T09:50:28.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:28.903+0000 7f7850860740 1 -- start start 2024-09-10T09:50:28.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:28.903+0000 7f7850860740 1 Processor -- start 2024-09-10T09:50:28.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:28.904+0000 7f7850860740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5572c74a2c70 0x5572c74b6610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:50:28.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:28.904+0000 7f7850860740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5572c74a3590 0x5572c74a7b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:50:28.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:28.904+0000 7f7850860740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5572c74ade50 0x5572c74a8170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:50:28.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:28.904+0000 7f7850860740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f78481b2500 con 0x5572c74a3590 2024-09-10T09:50:28.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:28.904+0000 7f7850860740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f78481b1e30 con 0x5572c74ade50 2024-09-10T09:50:28.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:28.904+0000 7f7850860740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f78481af070 con 0x5572c74a2c70 2024-09-10T09:50:28.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:28.904+0000 7f7850860740 10 client.0 _ll_register_callbacks cb 0x7f7834005de0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:50:28.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:28.905+0000 7f7846a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5572c74a2c70 0x5572c74b6610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:50:28.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:28.905+0000 7f7846a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5572c74a2c70 0x5572c74b6610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:48750/0 (socket says 172.21.15.5:48750) 2024-09-10T09:50:28.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:28.905+0000 7f7846a00640 1 -- 172.21.15.5:0/3730977222 learned_addr learned my addr 172.21.15.5:0/3730977222 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:50:28.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:28.906+0000 7f7846a00640 1 -- 172.21.15.5:0/3730977222 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5572c74ade50 msgr2=0x5572c74a8170 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T09:50:28.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:28.906+0000 7f7846a00640 1 --2- 172.21.15.5:0/3730977222 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5572c74ade50 0x5572c74a8170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:28.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:28.906+0000 7f7846a00640 1 -- 172.21.15.5:0/3730977222 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5572c74a3590 msgr2=0x5572c74a7b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:50:28.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:28.906+0000 7f7846000640 1 --2- 172.21.15.5:0/3730977222 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5572c74a3590 0x5572c74a7b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:50:28.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:28.906+0000 7f7846a00640 1 --2- 172.21.15.5:0/3730977222 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5572c74a3590 0x5572c74a7b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:28.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:28.906+0000 7f7846a00640 1 -- 172.21.15.5:0/3730977222 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5572c74a8970 con 0x5572c74a2c70 2024-09-10T09:50:28.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:28.906+0000 7f7846000640 1 --2- 172.21.15.5:0/3730977222 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5572c74a3590 0x5572c74a7b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T09:50:28.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:28.912+0000 7f7846a00640 1 --2- 172.21.15.5:0/3730977222 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5572c74a2c70 0x5572c74b6610 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f7838009ce0 tx=0x7f78380054d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d0592999da133ab8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:50:28.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:28.912+0000 7f7830c00640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-09-10T09:50:28.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:28.912+0000 7f7830c00640 1 -- 172.21.15.5:0/3730977222 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7838005210 con 0x5572c74a2c70 2024-09-10T09:50:28.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:28.912+0000 7f7830c00640 1 -- 172.21.15.5:0/3730977222 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f78380035f0 con 0x5572c74a2c70 2024-09-10T09:50:28.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:28.913+0000 7f7850860740 1 -- 172.21.15.5:0/3730977222 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f78340bc5c0 con 0x5572c74a2c70 2024-09-10T09:50:28.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:28.913+0000 7f7830c00640 1 -- 172.21.15.5:0/3730977222 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f78380037f0 con 0x5572c74a2c70 2024-09-10T09:50:28.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:28.914+0000 7f7850860740 10 client.27826 fetch_fsmap learned FSMap version 36 2024-09-10T09:50:28.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:28.914+0000 7f7850860740 1 -- 172.21.15.5:0/3730977222 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x5572c7531460 con 0x5572c74a2c70 2024-09-10T09:50:28.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:28.914+0000 7f7830c00640 1 -- 172.21.15.5:0/3730977222 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f78340bc5c0 con 0x5572c74a2c70 2024-09-10T09:50:28.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:28.915+0000 7f7830c00640 1 -- 172.21.15.5:0/3730977222 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 36) ==== 8370+0+0 (secure 0 0 0) 0x7f7838019ba0 con 0x5572c74a2c70 2024-09-10T09:50:28.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:28.915+0000 7f7850860740 10 client.27826 fetch_fsmap finished waiting for FSMap version 36 2024-09-10T09:50:28.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:28.915+0000 7f7850860740 10 client.27826 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-10T09:50:28.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:28.915+0000 7f7850860740 20 client.27826 populate_metadata read hostname 'smithi005' 2024-09-10T09:50:28.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:28.915+0000 7f7850860740 1 --2- 172.21.15.5:0/3730977222 >> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] conn(0x5572c7531a40 0x5572c7533e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:50:28.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:28.915+0000 7f7850860740 4 client.27826 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T09:50:28.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:28.915+0000 7f7850860740 1 -- 172.21.15.5:0/3730977222 --> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5572c7410e60 con 0x5572c7531a40 2024-09-10T09:50:28.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:28.916+0000 7f7846000640 1 --2- 172.21.15.5:0/3730977222 >> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] conn(0x5572c7531a40 0x5572c7533e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:50:28.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:28.917+0000 7f7846000640 1 --2- 172.21.15.5:0/3730977222 >> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] conn(0x5572c7531a40 0x5572c7533e30 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=747ea3277a1a1fa8 server_cookie=9b0dd388e540ab31 in_seq=0 out_seq=0 2024-09-10T09:50:28.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:28.917+0000 7f7830c00640 10 client.27826 ms_handle_connect on v2:172.21.15.159:6836/620627639 2024-09-10T09:50:28.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:28.917+0000 7f7830c00640 1 -- 172.21.15.5:0/3730977222 <== mds.2 v2:172.21.15.159:6836/620627639 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5572c7410e60 con 0x5572c7531a40 2024-09-10T09:50:28.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:28.917+0000 7f7830c00640 10 client.27826 handle_command_reply: tid=0 2024-09-10T09:50:28.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:28.917+0000 7f7850860740 1 client.27826 shutdown 2024-09-10T09:50:28.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:28.917+0000 7f7850860740 1 -- 172.21.15.5:0/3730977222 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5572c74a2c70 msgr2=0x5572c74b6610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:50:28.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:28.917+0000 7f7850860740 1 --2- 172.21.15.5:0/3730977222 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5572c74a2c70 0x5572c74b6610 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f7838009ce0 tx=0x7f78380054d0 comp rx=0 tx=0).stop 2024-09-10T09:50:28.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:28.918+0000 7f7850860740 1 -- 172.21.15.5:0/3730977222 shutdown_connections 2024-09-10T09:50:28.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:28.918+0000 7f7850860740 1 --2- 172.21.15.5:0/3730977222 >> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] conn(0x5572c7531a40 0x5572c7533e30 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:28.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:28.918+0000 7f7850860740 1 --2- 172.21.15.5:0/3730977222 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5572c74ade50 0x5572c74a8170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:28.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:28.918+0000 7f7850860740 1 --2- 172.21.15.5:0/3730977222 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5572c74a3590 0x5572c74a7b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:28.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:28.918+0000 7f7850860740 1 --2- 172.21.15.5:0/3730977222 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5572c74a2c70 0x5572c74b6610 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:28.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:28.918+0000 7f7830c00640 0 client.27826 ms_handle_reset on v2:172.21.15.159:6836/620627639 2024-09-10T09:50:28.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:28.918+0000 7f7850860740 1 -- 172.21.15.5:0/3730977222 >> 172.21.15.5:0/3730977222 conn(0x5572c749f200 msgr2=0x5572c74acd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:50:28.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:28.918+0000 7f7850860740 1 -- 172.21.15.5:0/3730977222 shutdown_connections 2024-09-10T09:50:28.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:28.918+0000 7f7850860740 1 -- 172.21.15.5:0/3730977222 wait complete. 2024-09-10T09:50:28.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:28.918+0000 7f7850860740 20 client.27826 trim_cache size 0 max 16384 2024-09-10T09:50:28.922 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-10T09:50:28.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:28.921+0000 7f784de00640 1 -- 172.21.15.5:0/2180631267 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f782c0a2b80 msgr2=0x7f782c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:50:28.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:28.921+0000 7f784de00640 1 --2- 172.21.15.5:0/2180631267 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f782c0a2b80 0x7f782c0a5040 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f78400099a0 tx=0x7f7840009500 comp rx=0 tx=0).stop 2024-09-10T09:50:28.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:28.921+0000 7f784de00640 1 -- 172.21.15.5:0/2180631267 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f78481049a0 msgr2=0x7f784819ab90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:50:28.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:28.921+0000 7f784de00640 1 --2- 172.21.15.5:0/2180631267 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f78481049a0 0x7f784819ab90 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f783402a930 tx=0x7f7834003b20 comp rx=0 tx=0).stop 2024-09-10T09:50:28.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:28.922+0000 7f784de00640 1 -- 172.21.15.5:0/2180631267 shutdown_connections 2024-09-10T09:50:28.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:28.922+0000 7f784de00640 1 --2- 172.21.15.5:0/2180631267 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f782c0a2b80 0x7f782c0a5040 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:28.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:28.922+0000 7f784de00640 1 --2- 172.21.15.5:0/2180631267 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7848109120 0x7f784819ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:28.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:28.922+0000 7f784de00640 1 --2- 172.21.15.5:0/2180631267 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f78481049a0 0x7f784819ab90 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:28.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:28.922+0000 7f784de00640 1 --2- 172.21.15.5:0/2180631267 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7848103ff0 0x7f784819a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:28.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:28.922+0000 7f784de00640 1 -- 172.21.15.5:0/2180631267 >> 172.21.15.5:0/2180631267 conn(0x7f78480ffa90 msgr2=0x7f7848100d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:50:28.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:28.922+0000 7f784de00640 1 -- 172.21.15.5:0/2180631267 shutdown_connections 2024-09-10T09:50:28.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:28.922+0000 7f784de00640 1 -- 172.21.15.5:0/2180631267 wait complete. 2024-09-10T09:50:29.135 INFO:tasks.workunit.client.0.smithi005.stdout: 1 774651 6.44 MB/sec execute 490 sec latency 700.989 ms 2024-09-10T09:50:29.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:50:29 smithi005 ceph-mon[30350]: pgmap v757: 129 pgs: 129 active+clean; 293 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.3 MiB/s wr, 206 op/s 2024-09-10T09:50:30.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:50:29 smithi159 ceph-mon[35513]: pgmap v757: 129 pgs: 129 active+clean; 293 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.3 MiB/s wr, 206 op/s 2024-09-10T09:50:30.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:50:29 smithi102 ceph-mon[35442]: pgmap v757: 129 pgs: 129 active+clean; 293 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.3 MiB/s wr, 206 op/s 2024-09-10T09:50:30.135 INFO:tasks.workunit.client.0.smithi005.stdout: 1 774651 6.43 MB/sec execute 491 sec latency 1701.027 ms 2024-09-10T09:50:30.435 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-10T09:50:30.849 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:50:31.135 INFO:tasks.workunit.client.0.smithi005.stdout: 1 776484 6.43 MB/sec execute 492 sec latency 1816.773 ms 2024-09-10T09:50:31.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:31.645+0000 7fd8d8600640 1 -- 172.21.15.5:0/815648500 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd8d0077040 msgr2=0x7fd8d0112510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:50:31.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:31.645+0000 7fd8d8600640 1 --2- 172.21.15.5:0/815648500 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd8d0077040 0x7fd8d0112510 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fd8bc009900 tx=0x7fd8bc02a410 comp rx=0 tx=0).stop 2024-09-10T09:50:31.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:31.646+0000 7fd8d8600640 1 -- 172.21.15.5:0/815648500 shutdown_connections 2024-09-10T09:50:31.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:31.646+0000 7fd8d8600640 1 --2- 172.21.15.5:0/815648500 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd8d0112a50 0x7fd8d0114e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:31.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:31.646+0000 7fd8d8600640 1 --2- 172.21.15.5:0/815648500 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd8d0077040 0x7fd8d0112510 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:31.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:31.646+0000 7fd8d8600640 1 --2- 172.21.15.5:0/815648500 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd8d0076720 0x7fd8d0076b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:31.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:31.646+0000 7fd8d8600640 1 -- 172.21.15.5:0/815648500 >> 172.21.15.5:0/815648500 conn(0x7fd8d00ffa50 msgr2=0x7fd8d0101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:50:31.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:31.646+0000 7fd8d8600640 1 -- 172.21.15.5:0/815648500 shutdown_connections 2024-09-10T09:50:31.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:31.646+0000 7fd8d8600640 1 -- 172.21.15.5:0/815648500 wait complete. 2024-09-10T09:50:31.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:31.647+0000 7fd8d8600640 1 Processor -- start 2024-09-10T09:50:31.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:31.647+0000 7fd8d8600640 1 -- start start 2024-09-10T09:50:31.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:31.647+0000 7fd8d8600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd8d0076720 0x7fd8d01a4740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:50:31.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:31.647+0000 7fd8d8600640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd8d0077040 0x7fd8d01a4c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:50:31.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:31.647+0000 7fd8d8600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd8d0112a50 0x7fd8d019e900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:50:31.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:31.647+0000 7fd8d8600640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd8d0118610 con 0x7fd8d0112a50 2024-09-10T09:50:31.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:31.647+0000 7fd8d8600640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fd8d0118490 con 0x7fd8d0076720 2024-09-10T09:50:31.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:31.647+0000 7fd8d8600640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fd8d0118790 con 0x7fd8d0077040 2024-09-10T09:50:31.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:31.647+0000 7fd8d5200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd8d0077040 0x7fd8d01a4c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:50:31.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:31.647+0000 7fd8d5200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd8d0077040 0x7fd8d01a4c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:46456/0 (socket says 172.21.15.5:46456) 2024-09-10T09:50:31.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:31.647+0000 7fd8d5200640 1 -- 172.21.15.5:0/2683226208 learned_addr learned my addr 172.21.15.5:0/2683226208 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:50:31.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:31.648+0000 7fd8d6600640 1 --2- 172.21.15.5:0/2683226208 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd8d0112a50 0x7fd8d019e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:50:31.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:31.648+0000 7fd8d5c00640 1 --2- 172.21.15.5:0/2683226208 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd8d0076720 0x7fd8d01a4740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:50:31.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:31.648+0000 7fd8d5200640 1 -- 172.21.15.5:0/2683226208 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd8d0076720 msgr2=0x7fd8d01a4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:50:31.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:31.648+0000 7fd8d5200640 1 --2- 172.21.15.5:0/2683226208 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd8d0076720 0x7fd8d01a4740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:31.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:31.648+0000 7fd8d5200640 1 -- 172.21.15.5:0/2683226208 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd8d0112a50 msgr2=0x7fd8d019e900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:50:31.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:31.648+0000 7fd8d5200640 1 --2- 172.21.15.5:0/2683226208 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd8d0112a50 0x7fd8d019e900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:31.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:31.648+0000 7fd8d5200640 1 -- 172.21.15.5:0/2683226208 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8d019f030 con 0x7fd8d0077040 2024-09-10T09:50:31.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:31.648+0000 7fd8d5c00640 1 --2- 172.21.15.5:0/2683226208 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd8d0076720 0x7fd8d01a4740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:50:31.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:31.648+0000 7fd8d5200640 1 --2- 172.21.15.5:0/2683226208 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd8d0077040 0x7fd8d01a4c80 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fd8bc004770 tx=0x7fd8bc003c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:50:31.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:31.649+0000 7fd8ce000640 1 -- 172.21.15.5:0/2683226208 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8bc03f070 con 0x7fd8d0077040 2024-09-10T09:50:31.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:31.649+0000 7fd8ce000640 1 -- 172.21.15.5:0/2683226208 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd8bc031ca0 con 0x7fd8d0077040 2024-09-10T09:50:31.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:31.649+0000 7fd8d8600640 1 -- 172.21.15.5:0/2683226208 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8d019f2c0 con 0x7fd8d0077040 2024-09-10T09:50:31.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:31.649+0000 7fd8ce000640 1 -- 172.21.15.5:0/2683226208 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8bc035030 con 0x7fd8d0077040 2024-09-10T09:50:31.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:31.649+0000 7fd8d8600640 1 -- 172.21.15.5:0/2683226208 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8d01ab610 con 0x7fd8d0077040 2024-09-10T09:50:31.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:31.652+0000 7fd8dad22740 1 Processor -- start 2024-09-10T09:50:31.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:31.652+0000 7fd8ce000640 1 -- 172.21.15.5:0/2683226208 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7fd8bc042070 con 0x7fd8d0077040 2024-09-10T09:50:31.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:31.652+0000 7fd8dad22740 1 -- start start 2024-09-10T09:50:31.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:31.652+0000 7fd8dad22740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55811ab57c70 0x55811ab58050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:50:31.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:31.652+0000 7fd8dad22740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55811ab58590 0x55811ab62910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:50:31.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:31.652+0000 7fd8dad22740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55811ab62e50 0x55811ab65240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:50:31.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:31.652+0000 7fd8dad22740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55811aabe3e0 con 0x55811ab62e50 2024-09-10T09:50:31.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:31.652+0000 7fd8dad22740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55811aa65d80 con 0x55811ab58590 2024-09-10T09:50:31.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:31.652+0000 7fd8dad22740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55811aaba3a0 con 0x55811ab57c70 2024-09-10T09:50:31.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:31.653+0000 7fd8d6600640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55811ab57c70 0x55811ab58050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:50:31.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:31.653+0000 7fd8d6600640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55811ab57c70 0x55811ab58050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:46464/0 (socket says 172.21.15.5:46464) 2024-09-10T09:50:31.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:31.653+0000 7fd8d6600640 1 -- 172.21.15.5:0/4172409856 learned_addr learned my addr 172.21.15.5:0/4172409856 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:50:31.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:31.653+0000 7fd8d5c00640 1 --2- 172.21.15.5:0/4172409856 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55811ab58590 0x55811ab62910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:50:31.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:31.653+0000 7fd8d5200640 1 --2- 172.21.15.5:0/4172409856 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55811ab62e50 0x55811ab65240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:50:31.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:31.653+0000 7fd8d6600640 1 -- 172.21.15.5:0/4172409856 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55811ab58590 msgr2=0x55811ab62910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:50:31.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:31.653+0000 7fd8d6600640 1 --2- 172.21.15.5:0/4172409856 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55811ab58590 0x55811ab62910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:31.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:31.653+0000 7fd8d6600640 1 -- 172.21.15.5:0/4172409856 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55811ab62e50 msgr2=0x55811ab65240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:50:31.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:31.653+0000 7fd8d6600640 1 --2- 172.21.15.5:0/4172409856 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55811ab62e50 0x55811ab65240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:31.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:31.653+0000 7fd8d6600640 1 -- 172.21.15.5:0/4172409856 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55811aa84ae0 con 0x55811ab57c70 2024-09-10T09:50:31.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:31.653+0000 7fd8d5200640 1 --2- 172.21.15.5:0/4172409856 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55811ab62e50 0x55811ab65240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:50:31.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:31.653+0000 7fd8d6600640 1 --2- 172.21.15.5:0/4172409856 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55811ab57c70 0x55811ab58050 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fd8c8009f30 tx=0x7fd8c80118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d27aedf6bf5196ca server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:50:31.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:31.654+0000 7fd8c2a00640 1 -- 172.21.15.5:0/4172409856 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8c8004d20 con 0x55811ab57c70 2024-09-10T09:50:31.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:31.654+0000 7fd8c2a00640 1 -- 172.21.15.5:0/4172409856 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd8c8004050 con 0x55811ab57c70 2024-09-10T09:50:31.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:31.654+0000 7fd8c2a00640 1 -- 172.21.15.5:0/4172409856 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8c8005220 con 0x55811ab57c70 2024-09-10T09:50:31.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:31.654+0000 7fd8ce000640 1 --2- 172.21.15.5:0/2683226208 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fd8b80a2c00 0x7fd8b80a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:50:31.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:31.655+0000 7fd8dad22740 1 -- 172.21.15.5:0/4172409856 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55811ab57c70 msgr2=0x55811ab58050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:50:31.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:31.655+0000 7fd8dad22740 1 --2- 172.21.15.5:0/4172409856 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55811ab57c70 0x55811ab58050 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fd8c8009f30 tx=0x7fd8c80118d0 comp rx=0 tx=0).stop 2024-09-10T09:50:31.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:31.655+0000 7fd8ce000640 1 -- 172.21.15.5:0/2683226208 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fd8bc0efa50 con 0x7fd8d0077040 2024-09-10T09:50:31.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:31.655+0000 7fd8d5c00640 1 --2- 172.21.15.5:0/2683226208 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fd8b80a2c00 0x7fd8b80a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:50:31.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:31.655+0000 7fd8dad22740 1 -- 172.21.15.5:0/4172409856 shutdown_connections 2024-09-10T09:50:31.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:31.655+0000 7fd8dad22740 1 --2- 172.21.15.5:0/4172409856 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55811ab62e50 0x55811ab65240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:31.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:31.655+0000 7fd8dad22740 1 --2- 172.21.15.5:0/4172409856 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55811ab58590 0x55811ab62910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:31.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:31.655+0000 7fd8dad22740 1 --2- 172.21.15.5:0/4172409856 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55811ab57c70 0x55811ab58050 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:31.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:31.655+0000 7fd8dad22740 1 -- 172.21.15.5:0/4172409856 >> 172.21.15.5:0/4172409856 conn(0x55811ab54200 msgr2=0x55811ab545e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:50:31.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:31.655+0000 7fd8dad22740 1 -- 172.21.15.5:0/4172409856 shutdown_connections 2024-09-10T09:50:31.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:31.655+0000 7fd8dad22740 1 -- 172.21.15.5:0/4172409856 wait complete. 2024-09-10T09:50:31.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:31.656+0000 7fd8dad22740 1 -- start start 2024-09-10T09:50:31.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:31.656+0000 7fd8d5c00640 1 --2- 172.21.15.5:0/2683226208 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fd8b80a2c00 0x7fd8b80a50c0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x55811ab55060 tx=0x7fd8c4005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:50:31.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:31.656+0000 7fd8dad22740 1 Processor -- start 2024-09-10T09:50:31.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:31.656+0000 7fd8dad22740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55811ab57c70 0x55811ab5fd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:50:31.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:31.656+0000 7fd8dad22740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55811ab58590 0x55811ab602c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:50:31.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:31.656+0000 7fd8dad22740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55811ab62e50 0x55811ab60800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:50:31.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:31.656+0000 7fd8dad22740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55811ab65a20 con 0x55811ab62e50 2024-09-10T09:50:31.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:31.656+0000 7fd8dad22740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55811aa790f0 con 0x55811ab57c70 2024-09-10T09:50:31.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:31.656+0000 7fd8dad22740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55811ab65ba0 con 0x55811ab58590 2024-09-10T09:50:31.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:31.656+0000 7fd8dad22740 10 client.0 _ll_register_callbacks cb 0x55811aa60bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:50:31.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:31.657+0000 7fd8d5c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55811ab57c70 0x55811ab5fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:50:31.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:31.657+0000 7fd8d5c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55811ab57c70 0x55811ab5fd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:35732/0 (socket says 172.21.15.5:35732) 2024-09-10T09:50:31.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:31.657+0000 7fd8d5c00640 1 -- 172.21.15.5:0/512244341 learned_addr learned my addr 172.21.15.5:0/512244341 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:50:31.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:31.657+0000 7fd8d5200640 1 --2- 172.21.15.5:0/512244341 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55811ab58590 0x55811ab602c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:50:31.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:31.657+0000 7fd8d6600640 1 --2- 172.21.15.5:0/512244341 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55811ab62e50 0x55811ab60800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:50:31.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:31.657+0000 7fd8d5c00640 1 -- 172.21.15.5:0/512244341 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55811ab58590 msgr2=0x55811ab602c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:50:31.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:31.657+0000 7fd8d5c00640 1 --2- 172.21.15.5:0/512244341 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55811ab58590 0x55811ab602c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:31.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:31.657+0000 7fd8d5c00640 1 -- 172.21.15.5:0/512244341 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55811ab62e50 msgr2=0x55811ab60800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:50:31.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:31.657+0000 7fd8d5c00640 1 --2- 172.21.15.5:0/512244341 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55811ab62e50 0x55811ab60800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:31.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:31.657+0000 7fd8d5c00640 1 -- 172.21.15.5:0/512244341 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55811a82f3c0 con 0x55811ab57c70 2024-09-10T09:50:31.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:31.657+0000 7fd8d5200640 1 --2- 172.21.15.5:0/512244341 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55811ab58590 0x55811ab602c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:50:31.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:31.658+0000 7fd8d5c00640 1 --2- 172.21.15.5:0/512244341 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55811ab57c70 0x55811ab5fd80 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fd8c4011920 tx=0x7fd8c40046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6176dd73ffb00bc0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:50:31.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:31.658+0000 7fd8b7400640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-09-10T09:50:31.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:31.660+0000 7fd8b7400640 1 -- 172.21.15.5:0/512244341 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8c402b060 con 0x55811ab57c70 2024-09-10T09:50:31.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:31.660+0000 7fd8b7400640 1 -- 172.21.15.5:0/512244341 <== mon.2 v2:172.21.15.102:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fd8c401eb90 con 0x55811ab57c70 2024-09-10T09:50:31.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:31.660+0000 7fd8dad22740 1 -- 172.21.15.5:0/512244341 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd8c8004050 con 0x55811ab57c70 2024-09-10T09:50:31.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:31.661+0000 7fd8b7400640 1 -- 172.21.15.5:0/512244341 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fd8c4011ea0 con 0x55811ab57c70 2024-09-10T09:50:31.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:31.661+0000 7fd8dad22740 10 client.27830 fetch_fsmap learned FSMap version 36 2024-09-10T09:50:31.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:31.661+0000 7fd8dad22740 1 -- 172.21.15.5:0/512244341 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55811ab71b10 con 0x55811ab57c70 2024-09-10T09:50:31.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:31.662+0000 7fd8b7400640 1 -- 172.21.15.5:0/512244341 <== mon.2 v2:172.21.15.102:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd8c8004050 con 0x55811ab57c70 2024-09-10T09:50:31.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:31.662+0000 7fd8b7400640 1 -- 172.21.15.5:0/512244341 <== mon.2 v2:172.21.15.102:3300/0 5 ==== fsmap(e 36) ==== 8370+0+0 (secure 0 0 0) 0x7fd8c401a040 con 0x55811ab57c70 2024-09-10T09:50:31.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:31.662+0000 7fd8dad22740 10 client.27830 fetch_fsmap finished waiting for FSMap version 36 2024-09-10T09:50:31.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:31.662+0000 7fd8dad22740 10 client.27830 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-10T09:50:31.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:31.662+0000 7fd8dad22740 20 client.27830 populate_metadata read hostname 'smithi005' 2024-09-10T09:50:31.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:31.662+0000 7fd8dad22740 1 --2- 172.21.15.5:0/512244341 >> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] conn(0x55811ab72fd0 0x55811ab753e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:50:31.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:31.662+0000 7fd8dad22740 4 client.27830 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:50:31.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:31.662+0000 7fd8dad22740 1 -- 172.21.15.5:0/512244341 --> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55811aacb950 con 0x55811ab72fd0 2024-09-10T09:50:31.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:31.663+0000 7fd8d5200640 1 --2- 172.21.15.5:0/512244341 >> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] conn(0x55811ab72fd0 0x55811ab753e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:50:31.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:31.663+0000 7fd8d5200640 1 --2- 172.21.15.5:0/512244341 >> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] conn(0x55811ab72fd0 0x55811ab753e0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b73ab0c07f6c6248 server_cookie=9784cb2d405d6a9e in_seq=0 out_seq=0 2024-09-10T09:50:31.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:31.663+0000 7fd8b7400640 10 client.27830 ms_handle_connect on v2:172.21.15.159:6838/2073739599 2024-09-10T09:50:31.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:31.664+0000 7fd8b7400640 1 -- 172.21.15.5:0/512244341 <== mds.1 v2:172.21.15.159:6838/2073739599 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55811aacb950 con 0x55811ab72fd0 2024-09-10T09:50:31.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:31.664+0000 7fd8b7400640 10 client.27830 handle_command_reply: tid=0 2024-09-10T09:50:31.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:31.664+0000 7fd8dad22740 1 client.27830 shutdown 2024-09-10T09:50:31.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:31.664+0000 7fd8dad22740 1 -- 172.21.15.5:0/512244341 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55811ab57c70 msgr2=0x55811ab5fd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:50:31.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:31.664+0000 7fd8dad22740 1 --2- 172.21.15.5:0/512244341 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55811ab57c70 0x55811ab5fd80 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fd8c4011920 tx=0x7fd8c40046c0 comp rx=0 tx=0).stop 2024-09-10T09:50:31.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:31.664+0000 7fd8dad22740 1 -- 172.21.15.5:0/512244341 shutdown_connections 2024-09-10T09:50:31.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:31.664+0000 7fd8dad22740 1 --2- 172.21.15.5:0/512244341 >> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] conn(0x55811ab72fd0 0x55811ab753e0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:31.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:31.664+0000 7fd8dad22740 1 --2- 172.21.15.5:0/512244341 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55811ab62e50 0x55811ab60800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:31.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:31.664+0000 7fd8dad22740 1 --2- 172.21.15.5:0/512244341 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55811ab58590 0x55811ab602c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:31.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:31.664+0000 7fd8b7400640 0 client.27830 ms_handle_reset on v2:172.21.15.159:6838/2073739599 2024-09-10T09:50:31.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:31.664+0000 7fd8dad22740 1 --2- 172.21.15.5:0/512244341 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55811ab57c70 0x55811ab5fd80 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:31.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:31.664+0000 7fd8dad22740 1 -- 172.21.15.5:0/512244341 >> 172.21.15.5:0/512244341 conn(0x55811ab54200 msgr2=0x55811ab59f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:50:31.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:31.664+0000 7fd8dad22740 1 -- 172.21.15.5:0/512244341 shutdown_connections 2024-09-10T09:50:31.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:31.664+0000 7fd8dad22740 1 -- 172.21.15.5:0/512244341 wait complete. 2024-09-10T09:50:31.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:31.665+0000 7fd8dad22740 20 client.27830 trim_cache size 0 max 16384 2024-09-10T09:50:31.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:31.680+0000 7fd8dad22740 1 Processor -- start 2024-09-10T09:50:31.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:31.680+0000 7fd8dad22740 1 -- start start 2024-09-10T09:50:31.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:31.680+0000 7fd8dad22740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55811ab57c70 0x55811ab5a770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:50:31.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:31.680+0000 7fd8dad22740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55811ab58590 0x55811ab7acb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:50:31.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:31.680+0000 7fd8dad22740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55811ab62e50 0x55811ab7b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:50:31.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:31.680+0000 7fd8dad22740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd8d01b5500 con 0x55811ab57c70 2024-09-10T09:50:31.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:31.680+0000 7fd8dad22740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fd8d01b7040 con 0x55811ab62e50 2024-09-10T09:50:31.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:31.680+0000 7fd8dad22740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fd8d01b5bd0 con 0x55811ab58590 2024-09-10T09:50:31.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:31.680+0000 7fd8d5c00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55811ab57c70 0x55811ab5a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:50:31.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:31.680+0000 7fd8d5200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55811ab58590 0x55811ab7acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:50:31.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:31.680+0000 7fd8d6600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55811ab62e50 0x55811ab7b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:50:31.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:31.680+0000 7fd8d5200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55811ab58590 0x55811ab7acb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:46476/0 (socket says 172.21.15.5:46476) 2024-09-10T09:50:31.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:31.680+0000 7fd8d5c00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55811ab57c70 0x55811ab5a770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59410/0 (socket says 172.21.15.5:59410) 2024-09-10T09:50:31.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:31.680+0000 7fd8d5200640 1 -- 172.21.15.5:0/3892693849 learned_addr learned my addr 172.21.15.5:0/3892693849 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:50:31.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:31.681+0000 7fd8d5200640 1 -- 172.21.15.5:0/3892693849 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55811ab62e50 msgr2=0x55811ab7b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:50:31.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:31.681+0000 7fd8d5200640 1 --2- 172.21.15.5:0/3892693849 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55811ab62e50 0x55811ab7b1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:31.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:31.681+0000 7fd8d5200640 1 -- 172.21.15.5:0/3892693849 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55811ab57c70 msgr2=0x55811ab5a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:50:31.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:31.681+0000 7fd8d5200640 1 --2- 172.21.15.5:0/3892693849 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55811ab57c70 0x55811ab5a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:31.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:31.681+0000 7fd8d5200640 1 -- 172.21.15.5:0/3892693849 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55811ab4f770 con 0x55811ab58590 2024-09-10T09:50:31.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:31.681+0000 7fd8d5c00640 1 --2- 172.21.15.5:0/3892693849 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55811ab57c70 0x55811ab5a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:50:31.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:31.681+0000 7fd8d5200640 1 --2- 172.21.15.5:0/3892693849 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55811ab58590 0x55811ab7acb0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fd8bc0ed100 tx=0x7fd8bc009680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eb1b1516d1a25f4c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:50:31.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:31.681+0000 7fd8c2000640 1 -- 172.21.15.5:0/3892693849 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8bc049040 con 0x55811ab58590 2024-09-10T09:50:31.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:31.681+0000 7fd8c2000640 1 -- 172.21.15.5:0/3892693849 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd8bc0bbf20 con 0x55811ab58590 2024-09-10T09:50:31.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:31.681+0000 7fd8c2000640 1 -- 172.21.15.5:0/3892693849 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8bc03b4f0 con 0x55811ab58590 2024-09-10T09:50:31.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:31.682+0000 7fd8dad22740 1 -- 172.21.15.5:0/3892693849 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55811ab58590 msgr2=0x55811ab7acb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:50:31.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:31.682+0000 7fd8dad22740 1 --2- 172.21.15.5:0/3892693849 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55811ab58590 0x55811ab7acb0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fd8bc0ed100 tx=0x7fd8bc009680 comp rx=0 tx=0).stop 2024-09-10T09:50:31.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:31.682+0000 7fd8dad22740 1 -- 172.21.15.5:0/3892693849 shutdown_connections 2024-09-10T09:50:31.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:31.682+0000 7fd8dad22740 1 --2- 172.21.15.5:0/3892693849 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55811ab62e50 0x55811ab7b1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:31.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:31.682+0000 7fd8dad22740 1 --2- 172.21.15.5:0/3892693849 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55811ab58590 0x55811ab7acb0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:31.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:31.682+0000 7fd8dad22740 1 --2- 172.21.15.5:0/3892693849 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55811ab57c70 0x55811ab5a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:31.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:31.682+0000 7fd8dad22740 1 -- 172.21.15.5:0/3892693849 >> 172.21.15.5:0/3892693849 conn(0x55811ab54200 msgr2=0x55811ab61e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:50:31.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:31.682+0000 7fd8dad22740 1 -- 172.21.15.5:0/3892693849 shutdown_connections 2024-09-10T09:50:31.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:31.683+0000 7fd8dad22740 1 -- 172.21.15.5:0/3892693849 wait complete. 2024-09-10T09:50:31.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:31.683+0000 7fd8dad22740 1 -- start start 2024-09-10T09:50:31.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:31.683+0000 7fd8dad22740 1 Processor -- start 2024-09-10T09:50:31.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:31.684+0000 7fd8dad22740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55811ab57c70 0x55811ab6adf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:50:31.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:31.684+0000 7fd8dad22740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55811ab58590 0x55811ab6b330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:50:31.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:31.684+0000 7fd8dad22740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55811ab62e50 0x55811ab67dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:50:31.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:31.684+0000 7fd8dad22740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd8d01b6970 con 0x55811ab58590 2024-09-10T09:50:31.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:31.684+0000 7fd8dad22740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fd8d01b62a0 con 0x55811ab57c70 2024-09-10T09:50:31.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:31.684+0000 7fd8dad22740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fd8d01b34e0 con 0x55811ab62e50 2024-09-10T09:50:31.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:31.684+0000 7fd8dad22740 10 client.0 _ll_register_callbacks cb 0x7fd8c4009870 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:50:31.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:31.684+0000 7fd8d5c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55811ab57c70 0x55811ab6adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:50:31.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:31.684+0000 7fd8d6600640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55811ab62e50 0x55811ab67dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:50:31.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:31.684+0000 7fd8d5c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55811ab57c70 0x55811ab6adf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:35742/0 (socket says 172.21.15.5:35742) 2024-09-10T09:50:31.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:31.684+0000 7fd8d5c00640 1 -- 172.21.15.5:0/1435830074 learned_addr learned my addr 172.21.15.5:0/1435830074 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:50:31.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:31.684+0000 7fd8d5200640 1 --2- 172.21.15.5:0/1435830074 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55811ab58590 0x55811ab6b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:50:31.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:31.684+0000 7fd8d5c00640 1 -- 172.21.15.5:0/1435830074 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55811ab62e50 msgr2=0x55811ab67dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:50:31.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:31.684+0000 7fd8d5c00640 1 --2- 172.21.15.5:0/1435830074 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55811ab62e50 0x55811ab67dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:31.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:31.684+0000 7fd8d5c00640 1 -- 172.21.15.5:0/1435830074 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55811ab58590 msgr2=0x55811ab6b330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:50:31.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:31.684+0000 7fd8d5c00640 1 --2- 172.21.15.5:0/1435830074 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55811ab58590 0x55811ab6b330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:31.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:31.684+0000 7fd8d5c00640 1 -- 172.21.15.5:0/1435830074 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55811ab68300 con 0x55811ab57c70 2024-09-10T09:50:31.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:31.684+0000 7fd8d6600640 1 --2- 172.21.15.5:0/1435830074 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55811ab62e50 0x55811ab67dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:50:31.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:31.684+0000 7fd8d5200640 1 --2- 172.21.15.5:0/1435830074 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55811ab58590 0x55811ab6b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:50:31.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:31.685+0000 7fd8d5c00640 1 --2- 172.21.15.5:0/1435830074 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55811ab57c70 0x55811ab6adf0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fd8c4026510 tx=0x7fd8c401e910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5d753ff2039eb86a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:50:31.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:31.685+0000 7fd8b7400640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-09-10T09:50:31.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:31.685+0000 7fd8b7400640 1 -- 172.21.15.5:0/1435830074 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8c4005040 con 0x55811ab57c70 2024-09-10T09:50:31.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:31.685+0000 7fd8b7400640 1 -- 172.21.15.5:0/1435830074 <== mon.2 v2:172.21.15.102:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fd8c40051e0 con 0x55811ab57c70 2024-09-10T09:50:31.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:31.685+0000 7fd8dad22740 1 -- 172.21.15.5:0/1435830074 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd8bc0bbf20 con 0x55811ab57c70 2024-09-10T09:50:31.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:31.685+0000 7fd8b7400640 1 -- 172.21.15.5:0/1435830074 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fd8c40053e0 con 0x55811ab57c70 2024-09-10T09:50:31.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:31.685+0000 7fd8dad22740 10 client.27836 fetch_fsmap learned FSMap version 36 2024-09-10T09:50:31.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:31.685+0000 7fd8dad22740 1 -- 172.21.15.5:0/1435830074 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55811abde470 con 0x55811ab57c70 2024-09-10T09:50:31.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:31.686+0000 7fd8b7400640 1 -- 172.21.15.5:0/1435830074 <== mon.2 v2:172.21.15.102:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd8bc0bbf20 con 0x55811ab57c70 2024-09-10T09:50:31.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:31.686+0000 7fd8b7400640 1 -- 172.21.15.5:0/1435830074 <== mon.2 v2:172.21.15.102:3300/0 5 ==== fsmap(e 36) ==== 8370+0+0 (secure 0 0 0) 0x7fd8c401a040 con 0x55811ab57c70 2024-09-10T09:50:31.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:31.686+0000 7fd8dad22740 10 client.27836 fetch_fsmap finished waiting for FSMap version 36 2024-09-10T09:50:31.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:31.686+0000 7fd8dad22740 10 client.27836 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-10T09:50:31.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:31.686+0000 7fd8dad22740 20 client.27836 populate_metadata read hostname 'smithi005' 2024-09-10T09:50:31.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:31.686+0000 7fd8dad22740 1 --2- 172.21.15.5:0/1435830074 >> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] conn(0x55811abdec70 0x55811abe1090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:50:31.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:31.686+0000 7fd8dad22740 4 client.27836 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T09:50:31.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:31.686+0000 7fd8dad22740 1 -- 172.21.15.5:0/1435830074 --> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55811aac5e60 con 0x55811abdec70 2024-09-10T09:50:31.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:31.686+0000 7fd8d5200640 1 --2- 172.21.15.5:0/1435830074 >> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] conn(0x55811abdec70 0x55811abe1090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:50:31.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:31.687+0000 7fd8d5200640 1 --2- 172.21.15.5:0/1435830074 >> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] conn(0x55811abdec70 0x55811abe1090 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2243d0c51b6fce64 server_cookie=97c99bdf8b0cd458 in_seq=0 out_seq=0 2024-09-10T09:50:31.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:31.687+0000 7fd8b7400640 10 client.27836 ms_handle_connect on v2:172.21.15.159:6838/2073739599 2024-09-10T09:50:31.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:31.687+0000 7fd8b7400640 1 -- 172.21.15.5:0/1435830074 <== mds.1 v2:172.21.15.159:6838/2073739599 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55811aac5e60 con 0x55811abdec70 2024-09-10T09:50:31.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:31.687+0000 7fd8b7400640 10 client.27836 handle_command_reply: tid=0 2024-09-10T09:50:31.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:31.687+0000 7fd8dad22740 1 client.27836 shutdown 2024-09-10T09:50:31.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:31.687+0000 7fd8dad22740 1 -- 172.21.15.5:0/1435830074 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55811ab57c70 msgr2=0x55811ab6adf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:50:31.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:31.687+0000 7fd8dad22740 1 --2- 172.21.15.5:0/1435830074 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55811ab57c70 0x55811ab6adf0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fd8c4026510 tx=0x7fd8c401e910 comp rx=0 tx=0).stop 2024-09-10T09:50:31.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:31.687+0000 7fd8dad22740 1 -- 172.21.15.5:0/1435830074 shutdown_connections 2024-09-10T09:50:31.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:31.687+0000 7fd8dad22740 1 --2- 172.21.15.5:0/1435830074 >> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] conn(0x55811abdec70 0x55811abe1090 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:31.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:31.687+0000 7fd8dad22740 1 --2- 172.21.15.5:0/1435830074 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55811ab62e50 0x55811ab67dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:31.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:31.687+0000 7fd8dad22740 1 --2- 172.21.15.5:0/1435830074 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55811ab58590 0x55811ab6b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:31.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:31.687+0000 7fd8dad22740 1 --2- 172.21.15.5:0/1435830074 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55811ab57c70 0x55811ab6adf0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:31.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:31.687+0000 7fd8dad22740 1 -- 172.21.15.5:0/1435830074 >> 172.21.15.5:0/1435830074 conn(0x55811ab54200 msgr2=0x55811ab6dc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:50:31.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:31.687+0000 7fd8b7400640 0 client.27836 ms_handle_reset on v2:172.21.15.159:6838/2073739599 2024-09-10T09:50:31.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:31.687+0000 7fd8dad22740 1 -- 172.21.15.5:0/1435830074 shutdown_connections 2024-09-10T09:50:31.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:31.687+0000 7fd8dad22740 1 -- 172.21.15.5:0/1435830074 wait complete. 2024-09-10T09:50:31.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:31.687+0000 7fd8dad22740 20 client.27836 trim_cache size 0 max 16384 2024-09-10T09:50:31.696 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-10T09:50:31.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:31.688+0000 7fd8d8600640 1 -- 172.21.15.5:0/2683226208 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fd8b80a2c00 msgr2=0x7fd8b80a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:50:31.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:31.688+0000 7fd8d8600640 1 --2- 172.21.15.5:0/2683226208 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fd8b80a2c00 0x7fd8b80a50c0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x55811ab55060 tx=0x7fd8c4005f90 comp rx=0 tx=0).stop 2024-09-10T09:50:31.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:31.688+0000 7fd8d8600640 1 -- 172.21.15.5:0/2683226208 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd8d0077040 msgr2=0x7fd8d01a4c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:50:31.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:31.688+0000 7fd8d8600640 1 --2- 172.21.15.5:0/2683226208 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd8d0077040 0x7fd8d01a4c80 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fd8bc004770 tx=0x7fd8bc003c10 comp rx=0 tx=0).stop 2024-09-10T09:50:31.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:31.688+0000 7fd8d8600640 1 -- 172.21.15.5:0/2683226208 shutdown_connections 2024-09-10T09:50:31.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:31.688+0000 7fd8d8600640 1 --2- 172.21.15.5:0/2683226208 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fd8b80a2c00 0x7fd8b80a50c0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:31.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:31.688+0000 7fd8d8600640 1 --2- 172.21.15.5:0/2683226208 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd8d0112a50 0x7fd8d019e900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:31.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:31.688+0000 7fd8d8600640 1 --2- 172.21.15.5:0/2683226208 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd8d0077040 0x7fd8d01a4c80 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:31.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:31.688+0000 7fd8d8600640 1 --2- 172.21.15.5:0/2683226208 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd8d0076720 0x7fd8d01a4740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:31.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:31.688+0000 7fd8d8600640 1 -- 172.21.15.5:0/2683226208 >> 172.21.15.5:0/2683226208 conn(0x7fd8d00ffa50 msgr2=0x7fd8d0101e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:50:31.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:31.688+0000 7fd8d8600640 1 -- 172.21.15.5:0/2683226208 shutdown_connections 2024-09-10T09:50:31.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:31.688+0000 7fd8d8600640 1 -- 172.21.15.5:0/2683226208 wait complete. 2024-09-10T09:50:31.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:50:31 smithi005 ceph-mon[30350]: pgmap v758: 129 pgs: 129 active+clean; 293 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.9 MiB/s wr, 236 op/s 2024-09-10T09:50:31.937 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:50:31 smithi102 ceph-mon[35442]: pgmap v758: 129 pgs: 129 active+clean; 293 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.9 MiB/s wr, 236 op/s 2024-09-10T09:50:32.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:50:31 smithi159 ceph-mon[35513]: pgmap v758: 129 pgs: 129 active+clean; 293 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.9 MiB/s wr, 236 op/s 2024-09-10T09:50:32.135 INFO:tasks.workunit.client.0.smithi005.stdout: 1 778391 6.44 MB/sec execute 493 sec latency 21.420 ms 2024-09-10T09:50:32.572 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 18.1 2024-09-10T09:50:32.573 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T09:50:32.574 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-10T09:50:33.065 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:50:33.072 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:50:33.135 INFO:tasks.workunit.client.0.smithi005.stdout: 1 780927 6.46 MB/sec execute 494 sec latency 14.605 ms 2024-09-10T09:50:33.814 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:50:33 smithi005 ceph-mon[30350]: pgmap v759: 129 pgs: 129 active+clean; 293 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.5 MiB/s wr, 195 op/s 2024-09-10T09:50:34.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:50:33 smithi159 ceph-mon[35513]: pgmap v759: 129 pgs: 129 active+clean; 293 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.5 MiB/s wr, 195 op/s 2024-09-10T09:50:34.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:50:33 smithi102 ceph-mon[35442]: pgmap v759: 129 pgs: 129 active+clean; 293 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.5 MiB/s wr, 195 op/s 2024-09-10T09:50:34.135 INFO:tasks.workunit.client.0.smithi005.stdout: 1 781476 6.45 MB/sec execute 495 sec latency 652.992 ms 2024-09-10T09:50:34.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:34.340+0000 7f51d7000640 1 -- 172.21.15.5:0/2372364751 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f51d01049a0 msgr2=0x7f51d01089f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:50:34.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:34.340+0000 7f51d7000640 1 --2- 172.21.15.5:0/2372364751 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f51d01049a0 0x7f51d01089f0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f51bc009900 tx=0x7f51bc02a3f0 comp rx=0 tx=0).stop 2024-09-10T09:50:34.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:34.340+0000 7f51d7000640 1 -- 172.21.15.5:0/2372364751 shutdown_connections 2024-09-10T09:50:34.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:34.340+0000 7f51d7000640 1 --2- 172.21.15.5:0/2372364751 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f51d0109120 0x7f51d0110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:34.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:34.340+0000 7f51d7000640 1 --2- 172.21.15.5:0/2372364751 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f51d01049a0 0x7f51d01089f0 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:34.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:34.340+0000 7f51d7000640 1 --2- 172.21.15.5:0/2372364751 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f51d0103ff0 0x7f51d01043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:34.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:34.340+0000 7f51d7000640 1 -- 172.21.15.5:0/2372364751 >> 172.21.15.5:0/2372364751 conn(0x7f51d00ffa90 msgr2=0x7f51d0101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:50:34.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:34.340+0000 7f51d7000640 1 -- 172.21.15.5:0/2372364751 shutdown_connections 2024-09-10T09:50:34.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:34.340+0000 7f51d7000640 1 -- 172.21.15.5:0/2372364751 wait complete. 2024-09-10T09:50:34.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:34.340+0000 7f51d7000640 1 Processor -- start 2024-09-10T09:50:34.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:34.340+0000 7f51d7000640 1 -- start start 2024-09-10T09:50:34.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:34.340+0000 7f51d7000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f51d0103ff0 0x7f51d019a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:50:34.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:34.340+0000 7f51d7000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f51d01049a0 0x7f51d019ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:50:34.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:34.340+0000 7f51d7000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f51d0109120 0x7f51d019eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:50:34.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:34.340+0000 7f51d7000640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f51d0114190 con 0x7f51d0109120 2024-09-10T09:50:34.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:34.340+0000 7f51d7000640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f51d0114010 con 0x7f51d0103ff0 2024-09-10T09:50:34.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:34.340+0000 7f51d7000640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f51d0114310 con 0x7f51d01049a0 2024-09-10T09:50:34.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:34.340+0000 7f51cfe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f51d0103ff0 0x7f51d019a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:50:34.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:34.340+0000 7f51cfe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f51d0103ff0 0x7f51d019a5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:35768/0 (socket says 172.21.15.5:35768) 2024-09-10T09:50:34.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:34.340+0000 7f51cfe00640 1 -- 172.21.15.5:0/1988520491 learned_addr learned my addr 172.21.15.5:0/1988520491 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:50:34.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:34.340+0000 7f51d5000640 1 --2- 172.21.15.5:0/1988520491 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f51d0109120 0x7f51d019eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:50:34.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:34.340+0000 7f51cfe00640 1 -- 172.21.15.5:0/1988520491 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f51d01049a0 msgr2=0x7f51d019ab30 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-10T09:50:34.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:34.340+0000 7f51cfe00640 1 --2- 172.21.15.5:0/1988520491 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f51d01049a0 0x7f51d019ab30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:34.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:34.340+0000 7f51cfe00640 1 -- 172.21.15.5:0/1988520491 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f51d0109120 msgr2=0x7f51d019eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:50:34.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:34.340+0000 7f51cfe00640 1 --2- 172.21.15.5:0/1988520491 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f51d0109120 0x7f51d019eec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:34.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:34.340+0000 7f51cfe00640 1 -- 172.21.15.5:0/1988520491 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51d019f5a0 con 0x7f51d0103ff0 2024-09-10T09:50:34.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:34.340+0000 7f51d5000640 1 --2- 172.21.15.5:0/1988520491 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f51d0109120 0x7f51d019eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:50:34.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:34.340+0000 7f51cfe00640 1 --2- 172.21.15.5:0/1988520491 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f51d0103ff0 0x7f51d019a5f0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f51c000d950 tx=0x7f51c000de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:50:34.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:34.340+0000 7f51ccc00640 1 -- 172.21.15.5:0/1988520491 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51c0017030 con 0x7f51d0103ff0 2024-09-10T09:50:34.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:34.340+0000 7f51ccc00640 1 -- 172.21.15.5:0/1988520491 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f51c0004020 con 0x7f51d0103ff0 2024-09-10T09:50:34.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:34.340+0000 7f51ccc00640 1 -- 172.21.15.5:0/1988520491 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51c00047a0 con 0x7f51d0103ff0 2024-09-10T09:50:34.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:34.340+0000 7f51d7000640 1 -- 172.21.15.5:0/1988520491 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51d019f890 con 0x7f51d0103ff0 2024-09-10T09:50:34.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:34.340+0000 7f51d7000640 1 -- 172.21.15.5:0/1988520491 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51d01a7170 con 0x7f51d0103ff0 2024-09-10T09:50:34.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:34.340+0000 7f51d7000640 1 -- 172.21.15.5:0/1988520491 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f51d01043d0 con 0x7f51d0103ff0 2024-09-10T09:50:34.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:34.345+0000 7f51ccc00640 1 -- 172.21.15.5:0/1988520491 <== mon.2 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f51c00041d0 con 0x7f51d0103ff0 2024-09-10T09:50:34.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:34.348+0000 7f51ccc00640 1 --2- 172.21.15.5:0/1988520491 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f51b40a2c00 0x7f51b40a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:50:34.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:34.348+0000 7f51ccc00640 1 -- 172.21.15.5:0/1988520491 <== mon.2 v2:172.21.15.102:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f51c00ceeb0 con 0x7f51d0103ff0 2024-09-10T09:50:34.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:34.348+0000 7f51cf400640 1 --2- 172.21.15.5:0/1988520491 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f51b40a2c00 0x7f51b40a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:50:34.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:34.348+0000 7f51cf400640 1 --2- 172.21.15.5:0/1988520491 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f51b40a2c00 0x7f51b40a50c0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f51bc02a910 tx=0x7f51bc005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:50:34.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:34.348+0000 7f51ccc00640 1 -- 172.21.15.5:0/1988520491 <== mon.2 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f51c009b9c0 con 0x7f51d0103ff0 2024-09-10T09:50:34.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:34.577+0000 7f51d7000640 1 -- 172.21.15.5:0/1988520491 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f51d019b950 con 0x7f51d0103ff0 2024-09-10T09:50:34.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:34.579+0000 7f51ccc00640 1 -- 172.21.15.5:0/1988520491 <== mon.2 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) ==== 94+0+11969 (secure 0 0 0) 0x7f51d019b950 con 0x7f51d0103ff0 2024-09-10T09:50:34.580 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-10T09:50:34.580 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":36,"btime":"2024-09-10T09:48:07:349075+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14454,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/2970594311","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2970594311},{"type":"v1","addr":"172.21.15.5:6835","nonce":2970594311}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14514,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6839/3616502174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":3616502174},{"type":"v1","addr":"172.21.15.5:6839","nonce":3616502174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":36,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T09:35:29.626287+0000","modified":"2024-09-10T09:48:06.970127+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24542,"mds_1":24535,"mds_2":24511,"mds_3":24506,"mds_4":24443},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14520":{"gid":14520,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/2073599842","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":2073599842},{"type":"v1","addr":"172.21.15.5:6841","nonce":2073599842}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"k","rank":4,"incarnation":26,"state":"up:active","state_seq":24,"addr":"172.21.15.102:6833/2457007427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":2457007427},{"type":"v1","addr":"172.21.15.102:6833","nonce":2457007427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/356445159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":356445159},{"type":"v1","addr":"172.21.15.159:6833","nonce":356445159}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6837/2894314507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":2894314507},{"type":"v1","addr":"172.21.15.5:6837","nonce":2894314507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.102:6835/1647549111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":1647549111},{"type":"v1","addr":"172.21.15.102:6835","nonce":1647549111}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6835/952657242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":952657242},{"type":"v1","addr":"172.21.15.159:6835","nonce":952657242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":19,"addr":"172.21.15.102:6837/2689776764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":2689776764},{"type":"v1","addr":"172.21.15.102:6837","nonce":2689776764}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.159:6837/620627639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":620627639},{"type":"v1","addr":"172.21.15.159:6837","nonce":620627639}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24535":{"gid":24535,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":18,"addr":"172.21.15.159:6839/2073739599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":2073739599},{"type":"v1","addr":"172.21.15.159:6839","nonce":2073739599}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24542":{"gid":24542,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.102:6839/335409749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6838","nonce":335409749},{"type":"v1","addr":"172.21.15.102:6839","nonce":335409749}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24542,"qdb_cluster":[24542,24535,24511,24506,24443]},"id":1}]} 2024-09-10T09:50:34.580 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 36 2024-09-10T09:50:34.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:34.582+0000 7f51d7000640 1 -- 172.21.15.5:0/1988520491 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f51b40a2c00 msgr2=0x7f51b40a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:50:34.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:34.582+0000 7f51d7000640 1 --2- 172.21.15.5:0/1988520491 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f51b40a2c00 0x7f51b40a50c0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f51bc02a910 tx=0x7f51bc005fd0 comp rx=0 tx=0).stop 2024-09-10T09:50:34.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:34.582+0000 7f51d7000640 1 -- 172.21.15.5:0/1988520491 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f51d0103ff0 msgr2=0x7f51d019a5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:50:34.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:34.582+0000 7f51d7000640 1 --2- 172.21.15.5:0/1988520491 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f51d0103ff0 0x7f51d019a5f0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f51c000d950 tx=0x7f51c000de10 comp rx=0 tx=0).stop 2024-09-10T09:50:34.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:34.582+0000 7f51d7000640 1 -- 172.21.15.5:0/1988520491 shutdown_connections 2024-09-10T09:50:34.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:34.582+0000 7f51d7000640 1 --2- 172.21.15.5:0/1988520491 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f51b40a2c00 0x7f51b40a50c0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:34.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:34.582+0000 7f51d7000640 1 --2- 172.21.15.5:0/1988520491 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f51d0109120 0x7f51d019eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:34.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:34.582+0000 7f51d7000640 1 --2- 172.21.15.5:0/1988520491 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f51d01049a0 0x7f51d019ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:34.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:34.582+0000 7f51d7000640 1 --2- 172.21.15.5:0/1988520491 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f51d0103ff0 0x7f51d019a5f0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:34.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:34.582+0000 7f51d7000640 1 -- 172.21.15.5:0/1988520491 >> 172.21.15.5:0/1988520491 conn(0x7f51d00ffa90 msgr2=0x7f51d0101e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:50:34.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:34.582+0000 7f51d7000640 1 -- 172.21.15.5:0/1988520491 shutdown_connections 2024-09-10T09:50:34.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:34.582+0000 7f51d7000640 1 -- 172.21.15.5:0/1988520491 wait complete. 2024-09-10T09:50:34.906 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:50:34 smithi005 ceph-mon[30350]: from='client.? 172.21.15.5:0/1988520491' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:50:35.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:50:34 smithi159 ceph-mon[35513]: from='client.? 172.21.15.5:0/1988520491' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:50:35.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:50:34 smithi102 ceph-mon[35442]: from='client.? 172.21.15.5:0/1988520491' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:50:35.135 INFO:tasks.workunit.client.0.smithi005.stdout: 1 781476 6.44 MB/sec execute 496 sec latency 1653.040 ms 2024-09-10T09:50:35.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:34.944+0000 7fb985800640 1 -- 172.21.15.5:0/1909503483 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb9801019c0 msgr2=0x7fb980101e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:50:35.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:34.944+0000 7fb985800640 1 --2- 172.21.15.5:0/1909503483 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb9801019c0 0x7fb980101e20 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fb96c009930 tx=0x7fb96c02a420 comp rx=0 tx=0).stop 2024-09-10T09:50:35.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:34.945+0000 7fb985800640 1 -- 172.21.15.5:0/1909503483 shutdown_connections 2024-09-10T09:50:35.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:34.945+0000 7fb985800640 1 --2- 172.21.15.5:0/1909503483 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb980102360 0x7fb98010e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:35.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:34.945+0000 7fb985800640 1 --2- 172.21.15.5:0/1909503483 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb9801019c0 0x7fb980101e20 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:35.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:34.945+0000 7fb985800640 1 --2- 172.21.15.5:0/1909503483 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb9801079c0 0x7fb980107da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:35.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:34.945+0000 7fb985800640 1 -- 172.21.15.5:0/1909503483 >> 172.21.15.5:0/1909503483 conn(0x7fb9800fd740 msgr2=0x7fb9800ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:50:35.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:34.945+0000 7fb985800640 1 -- 172.21.15.5:0/1909503483 shutdown_connections 2024-09-10T09:50:35.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:34.945+0000 7fb985800640 1 -- 172.21.15.5:0/1909503483 wait complete. 2024-09-10T09:50:35.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:34.946+0000 7fb985800640 1 Processor -- start 2024-09-10T09:50:35.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:34.946+0000 7fb985800640 1 -- start start 2024-09-10T09:50:35.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:34.947+0000 7fb985800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb9801019c0 0x7fb9800744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:50:35.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:34.947+0000 7fb985800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb980102360 0x7fb980074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:50:35.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:34.947+0000 7fb985800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb9801079c0 0x7fb980074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:50:35.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:34.947+0000 7fb985800640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fb980111dc0 con 0x7fb9801079c0 2024-09-10T09:50:35.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:34.947+0000 7fb985800640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fb980111c40 con 0x7fb9801019c0 2024-09-10T09:50:35.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:34.947+0000 7fb985800640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fb980111f40 con 0x7fb980102360 2024-09-10T09:50:35.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:34.947+0000 7fb984e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb9801079c0 0x7fb980074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:50:35.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:34.947+0000 7fb984e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb9801079c0 0x7fb980074f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59458/0 (socket says 172.21.15.5:59458) 2024-09-10T09:50:35.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:34.947+0000 7fb984e00640 1 -- 172.21.15.5:0/4281099939 learned_addr learned my addr 172.21.15.5:0/4281099939 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:50:35.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:34.947+0000 7fb97fe00640 1 --2- 172.21.15.5:0/4281099939 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb9801019c0 0x7fb9800744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:50:35.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:34.947+0000 7fb97f400640 1 --2- 172.21.15.5:0/4281099939 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb980102360 0x7fb980074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:50:35.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:34.947+0000 7fb97fe00640 1 -- 172.21.15.5:0/4281099939 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb980102360 msgr2=0x7fb980074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:50:35.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:34.947+0000 7fb97fe00640 1 --2- 172.21.15.5:0/4281099939 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb980102360 0x7fb980074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:35.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:34.947+0000 7fb97fe00640 1 -- 172.21.15.5:0/4281099939 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb9801079c0 msgr2=0x7fb980074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:50:35.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:34.947+0000 7fb97fe00640 1 --2- 172.21.15.5:0/4281099939 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb9801079c0 0x7fb980074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:35.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:34.947+0000 7fb97fe00640 1 -- 172.21.15.5:0/4281099939 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9801a6760 con 0x7fb9801019c0 2024-09-10T09:50:35.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:34.947+0000 7fb97f400640 1 --2- 172.21.15.5:0/4281099939 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb980102360 0x7fb980074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:50:35.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:34.947+0000 7fb984e00640 1 --2- 172.21.15.5:0/4281099939 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb9801079c0 0x7fb980074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:50:35.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:34.947+0000 7fb97fe00640 1 --2- 172.21.15.5:0/4281099939 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb9801019c0 0x7fb9800744f0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fb97000e9f0 tx=0x7fb97000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:50:35.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:34.947+0000 7fb97cc00640 1 -- 172.21.15.5:0/4281099939 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb97000ccc0 con 0x7fb9801019c0 2024-09-10T09:50:35.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:34.947+0000 7fb97cc00640 1 -- 172.21.15.5:0/4281099939 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb970010910 con 0x7fb9801019c0 2024-09-10T09:50:35.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:34.947+0000 7fb985800640 1 -- 172.21.15.5:0/4281099939 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9801a6a50 con 0x7fb9801019c0 2024-09-10T09:50:35.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:34.947+0000 7fb97cc00640 1 -- 172.21.15.5:0/4281099939 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb970018960 con 0x7fb9801019c0 2024-09-10T09:50:35.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:34.947+0000 7fb985800640 1 -- 172.21.15.5:0/4281099939 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9801a6f60 con 0x7fb9801019c0 2024-09-10T09:50:35.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:34.951+0000 7fb97cc00640 1 -- 172.21.15.5:0/4281099939 <== mon.2 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7fb970010ac0 con 0x7fb9801019c0 2024-09-10T09:50:35.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:34.951+0000 7fb989a4f740 1 Processor -- start 2024-09-10T09:50:35.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:34.951+0000 7fb989a4f740 1 -- start start 2024-09-10T09:50:35.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:34.951+0000 7fb989a4f740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f372d83e40 0x55f372d84220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:50:35.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:34.951+0000 7fb989a4f740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f372d84760 0x55f372d8eae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:50:35.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:34.951+0000 7fb989a4f740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f372d8f020 0x55f372d91410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:50:35.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:34.951+0000 7fb989a4f740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55f372cea3e0 con 0x55f372d84760 2024-09-10T09:50:35.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:34.951+0000 7fb989a4f740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55f372c91d80 con 0x55f372d83e40 2024-09-10T09:50:35.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:34.951+0000 7fb989a4f740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55f372ce63a0 con 0x55f372d8f020 2024-09-10T09:50:35.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:34.951+0000 7fb97fe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f372d84760 0x55f372d8eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:50:35.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:34.951+0000 7fb97fe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f372d84760 0x55f372d8eae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59472/0 (socket says 172.21.15.5:59472) 2024-09-10T09:50:35.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:34.951+0000 7fb97fe00640 1 -- 172.21.15.5:0/698682462 learned_addr learned my addr 172.21.15.5:0/698682462 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:50:35.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:34.951+0000 7fb97f400640 1 --2- 172.21.15.5:0/698682462 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f372d8f020 0x55f372d91410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:50:35.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:34.951+0000 7fb984e00640 1 --2- 172.21.15.5:0/698682462 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f372d83e40 0x55f372d84220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:50:35.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:34.951+0000 7fb97fe00640 1 -- 172.21.15.5:0/698682462 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f372d8f020 msgr2=0x55f372d91410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:50:35.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:34.951+0000 7fb97fe00640 1 --2- 172.21.15.5:0/698682462 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f372d8f020 0x55f372d91410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:35.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:34.951+0000 7fb97fe00640 1 -- 172.21.15.5:0/698682462 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f372d83e40 msgr2=0x55f372d84220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:50:35.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:34.951+0000 7fb97fe00640 1 --2- 172.21.15.5:0/698682462 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f372d83e40 0x55f372d84220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:35.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:34.951+0000 7fb97fe00640 1 -- 172.21.15.5:0/698682462 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f372cb0ae0 con 0x55f372d84760 2024-09-10T09:50:35.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:34.951+0000 7fb97f400640 1 --2- 172.21.15.5:0/698682462 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f372d8f020 0x55f372d91410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:50:35.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:34.951+0000 7fb97cc00640 1 --2- 172.21.15.5:0/4281099939 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fb9640a2b80 0x7fb9640a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:50:35.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:34.951+0000 7fb97cc00640 1 -- 172.21.15.5:0/4281099939 <== mon.2 v2:172.21.15.102:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fb9700cf850 con 0x7fb9801019c0 2024-09-10T09:50:35.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:34.951+0000 7fb984e00640 1 --2- 172.21.15.5:0/4281099939 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fb9640a2b80 0x7fb9640a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:50:35.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:34.951+0000 7fb97fe00640 1 --2- 172.21.15.5:0/698682462 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f372d84760 0x55f372d8eae0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fb9700d5c20 tx=0x7fb9700d7670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71d0b87a34bd7e41 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:50:35.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:34.951+0000 7fb975600640 1 -- 172.21.15.5:0/698682462 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb970026030 con 0x55f372d84760 2024-09-10T09:50:35.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:34.951+0000 7fb975600640 1 -- 172.21.15.5:0/698682462 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb9700b1050 con 0x55f372d84760 2024-09-10T09:50:35.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:34.951+0000 7fb975600640 1 -- 172.21.15.5:0/698682462 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb97009c360 con 0x55f372d84760 2024-09-10T09:50:35.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:34.951+0000 7fb989a4f740 1 -- 172.21.15.5:0/698682462 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f372d84760 msgr2=0x55f372d8eae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:50:35.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:34.951+0000 7fb984e00640 1 --2- 172.21.15.5:0/4281099939 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fb9640a2b80 0x7fb9640a5040 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fb9780049d0 tx=0x7fb978009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:50:35.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:34.951+0000 7fb989a4f740 1 --2- 172.21.15.5:0/698682462 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f372d84760 0x55f372d8eae0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fb9700d5c20 tx=0x7fb9700d7670 comp rx=0 tx=0).stop 2024-09-10T09:50:35.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:34.951+0000 7fb989a4f740 1 -- 172.21.15.5:0/698682462 shutdown_connections 2024-09-10T09:50:35.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:34.951+0000 7fb989a4f740 1 --2- 172.21.15.5:0/698682462 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f372d8f020 0x55f372d91410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:35.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:34.951+0000 7fb989a4f740 1 --2- 172.21.15.5:0/698682462 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f372d84760 0x55f372d8eae0 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:35.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:34.951+0000 7fb989a4f740 1 --2- 172.21.15.5:0/698682462 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f372d83e40 0x55f372d84220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:35.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:34.951+0000 7fb989a4f740 1 -- 172.21.15.5:0/698682462 >> 172.21.15.5:0/698682462 conn(0x55f372d803d0 msgr2=0x55f372d807b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:50:35.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:34.951+0000 7fb989a4f740 1 -- 172.21.15.5:0/698682462 shutdown_connections 2024-09-10T09:50:35.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:34.951+0000 7fb989a4f740 1 -- 172.21.15.5:0/698682462 wait complete. 2024-09-10T09:50:35.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:34.951+0000 7fb989a4f740 1 -- start start 2024-09-10T09:50:35.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:34.951+0000 7fb989a4f740 1 Processor -- start 2024-09-10T09:50:35.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:34.956+0000 7fb989a4f740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f372d83e40 0x55f372d8bf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:50:35.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:34.956+0000 7fb989a4f740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f372d84760 0x55f372d8c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:50:35.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:34.956+0000 7fb989a4f740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f372d8f020 0x55f372d8c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:50:35.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:34.956+0000 7fb989a4f740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55f372d91bf0 con 0x55f372d83e40 2024-09-10T09:50:35.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:34.956+0000 7fb989a4f740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55f372ca50f0 con 0x55f372d84760 2024-09-10T09:50:35.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:34.956+0000 7fb989a4f740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55f372d91d70 con 0x55f372d8f020 2024-09-10T09:50:35.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:34.956+0000 7fb989a4f740 10 client.0 _ll_register_callbacks cb 0x55f372c8cbd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:50:35.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:34.956+0000 7fb97fe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f372d83e40 0x55f372d8bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:50:35.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:34.956+0000 7fb97f400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f372d84760 0x55f372d8c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:50:35.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:34.956+0000 7fb984e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f372d8f020 0x55f372d8c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:50:35.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:34.956+0000 7fb97f400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f372d84760 0x55f372d8c4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:35790/0 (socket says 172.21.15.5:35790) 2024-09-10T09:50:35.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:34.956+0000 7fb97f400640 1 -- 172.21.15.5:0/3269096761 learned_addr learned my addr 172.21.15.5:0/3269096761 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:50:35.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:34.956+0000 7fb97f400640 1 -- 172.21.15.5:0/3269096761 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f372d8f020 msgr2=0x55f372d8c9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:50:35.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:34.956+0000 7fb97f400640 1 --2- 172.21.15.5:0/3269096761 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f372d8f020 0x55f372d8c9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:35.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:34.956+0000 7fb97f400640 1 -- 172.21.15.5:0/3269096761 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f372d83e40 msgr2=0x55f372d8bf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:50:35.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:34.956+0000 7fb97f400640 1 --2- 172.21.15.5:0/3269096761 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f372d83e40 0x55f372d8bf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:35.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:34.956+0000 7fb97f400640 1 -- 172.21.15.5:0/3269096761 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f372a5b3c0 con 0x55f372d84760 2024-09-10T09:50:35.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:34.956+0000 7fb984e00640 1 --2- 172.21.15.5:0/3269096761 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f372d8f020 0x55f372d8c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:50:35.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:34.956+0000 7fb97f400640 1 --2- 172.21.15.5:0/3269096761 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f372d84760 0x55f372d8c4b0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fb96c033000 tx=0x7fb96c031860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b6a9d291b3da5cc3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:50:35.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:34.956+0000 7fb96a000640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-09-10T09:50:35.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:34.956+0000 7fb96a000640 1 -- 172.21.15.5:0/3269096761 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb96c035040 con 0x55f372d84760 2024-09-10T09:50:35.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:34.956+0000 7fb989a4f740 1 -- 172.21.15.5:0/3269096761 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb9700b1050 con 0x55f372d84760 2024-09-10T09:50:35.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:34.956+0000 7fb96a000640 1 -- 172.21.15.5:0/3269096761 <== mon.2 v2:172.21.15.102:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fb96c004460 con 0x55f372d84760 2024-09-10T09:50:35.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:34.956+0000 7fb96a000640 1 -- 172.21.15.5:0/3269096761 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fb96c005030 con 0x55f372d84760 2024-09-10T09:50:35.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:34.956+0000 7fb989a4f740 10 client.27854 fetch_fsmap learned FSMap version 36 2024-09-10T09:50:35.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:34.956+0000 7fb989a4f740 1 -- 172.21.15.5:0/3269096761 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55f372d9dce0 con 0x55f372d84760 2024-09-10T09:50:35.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:34.956+0000 7fb96a000640 1 -- 172.21.15.5:0/3269096761 <== mon.2 v2:172.21.15.102:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb9700b1050 con 0x55f372d84760 2024-09-10T09:50:35.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:34.956+0000 7fb96a000640 1 -- 172.21.15.5:0/3269096761 <== mon.2 v2:172.21.15.102:3300/0 5 ==== fsmap(e 36) ==== 8370+0+0 (secure 0 0 0) 0x7fb96c004c60 con 0x55f372d84760 2024-09-10T09:50:35.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:34.956+0000 7fb989a4f740 10 client.27854 fetch_fsmap finished waiting for FSMap version 36 2024-09-10T09:50:35.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:34.956+0000 7fb989a4f740 10 client.27854 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T09:50:35.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:34.956+0000 7fb989a4f740 20 client.27854 populate_metadata read hostname 'smithi005' 2024-09-10T09:50:35.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:34.956+0000 7fb989a4f740 1 --2- 172.21.15.5:0/3269096761 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55f372d9f0f0 0x55f372da14e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:50:35.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:34.956+0000 7fb989a4f740 4 client.27854 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:50:35.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:34.956+0000 7fb989a4f740 1 -- 172.21.15.5:0/3269096761 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f372cf7950 con 0x55f372d9f0f0 2024-09-10T09:50:35.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:34.956+0000 7fb97fe00640 1 --2- 172.21.15.5:0/3269096761 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55f372d9f0f0 0x55f372da14e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:50:35.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:34.956+0000 7fb97fe00640 1 --2- 172.21.15.5:0/3269096761 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55f372d9f0f0 0x55f372da14e0 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6b35d0448722f784 server_cookie=bd5c1e346018681a in_seq=0 out_seq=0 2024-09-10T09:50:35.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:34.956+0000 7fb96a000640 10 client.27854 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T09:50:35.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:34.956+0000 7fb96a000640 1 -- 172.21.15.5:0/3269096761 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f372cf7950 con 0x55f372d9f0f0 2024-09-10T09:50:35.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:34.956+0000 7fb96a000640 10 client.27854 handle_command_reply: tid=0 2024-09-10T09:50:35.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:34.956+0000 7fb989a4f740 1 client.27854 shutdown 2024-09-10T09:50:35.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:34.956+0000 7fb989a4f740 1 -- 172.21.15.5:0/3269096761 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f372d84760 msgr2=0x55f372d8c4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:50:35.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:34.956+0000 7fb989a4f740 1 --2- 172.21.15.5:0/3269096761 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f372d84760 0x55f372d8c4b0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fb96c033000 tx=0x7fb96c031860 comp rx=0 tx=0).stop 2024-09-10T09:50:35.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:34.956+0000 7fb989a4f740 1 -- 172.21.15.5:0/3269096761 shutdown_connections 2024-09-10T09:50:35.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:34.956+0000 7fb989a4f740 1 --2- 172.21.15.5:0/3269096761 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55f372d9f0f0 0x55f372da14e0 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:35.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:34.956+0000 7fb989a4f740 1 --2- 172.21.15.5:0/3269096761 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f372d8f020 0x55f372d8c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:35.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:34.956+0000 7fb989a4f740 1 --2- 172.21.15.5:0/3269096761 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f372d84760 0x55f372d8c4b0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:35.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:34.956+0000 7fb989a4f740 1 --2- 172.21.15.5:0/3269096761 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f372d83e40 0x55f372d8bf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:35.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:34.956+0000 7fb96a000640 0 client.27854 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T09:50:35.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:34.956+0000 7fb989a4f740 1 -- 172.21.15.5:0/3269096761 >> 172.21.15.5:0/3269096761 conn(0x55f372d803d0 msgr2=0x55f372d86100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:50:35.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:34.956+0000 7fb989a4f740 1 -- 172.21.15.5:0/3269096761 shutdown_connections 2024-09-10T09:50:35.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:34.961+0000 7fb989a4f740 1 -- 172.21.15.5:0/3269096761 wait complete. 2024-09-10T09:50:35.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:34.961+0000 7fb989a4f740 20 client.27854 trim_cache size 0 max 16384 2024-09-10T09:50:35.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:34.975+0000 7fb989a4f740 1 Processor -- start 2024-09-10T09:50:35.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:34.975+0000 7fb989a4f740 1 -- start start 2024-09-10T09:50:35.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:34.975+0000 7fb989a4f740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f372d83e40 0x55f372d868f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:50:35.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:34.975+0000 7fb989a4f740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f372d84760 0x55f372d7b300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:50:35.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:34.975+0000 7fb989a4f740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f372d8f020 0x55f372d8dee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:50:35.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:34.975+0000 7fb989a4f740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fb9801b0f00 con 0x55f372d83e40 2024-09-10T09:50:35.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:34.975+0000 7fb989a4f740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fb9801b2a40 con 0x55f372d8f020 2024-09-10T09:50:35.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:34.975+0000 7fb989a4f740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fb9801b15d0 con 0x55f372d84760 2024-09-10T09:50:35.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:34.975+0000 7fb97fe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f372d83e40 0x55f372d868f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:50:35.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:34.975+0000 7fb97fe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f372d83e40 0x55f372d868f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59480/0 (socket says 172.21.15.5:59480) 2024-09-10T09:50:35.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:34.975+0000 7fb97fe00640 1 -- 172.21.15.5:0/4283020843 learned_addr learned my addr 172.21.15.5:0/4283020843 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:50:35.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:34.975+0000 7fb984e00640 1 --2- 172.21.15.5:0/4283020843 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f372d8f020 0x55f372d8dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:50:35.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:34.975+0000 7fb97f400640 1 --2- 172.21.15.5:0/4283020843 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f372d84760 0x55f372d7b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:50:35.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:34.975+0000 7fb97fe00640 1 -- 172.21.15.5:0/4283020843 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f372d84760 msgr2=0x55f372d7b300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:50:35.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:34.975+0000 7fb97fe00640 1 --2- 172.21.15.5:0/4283020843 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f372d84760 0x55f372d7b300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:35.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:34.975+0000 7fb97fe00640 1 -- 172.21.15.5:0/4283020843 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f372d8f020 msgr2=0x55f372d8dee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:50:35.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:34.975+0000 7fb97fe00640 1 --2- 172.21.15.5:0/4283020843 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f372d8f020 0x55f372d8dee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:35.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:34.975+0000 7fb97fe00640 1 -- 172.21.15.5:0/4283020843 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f372d8e8e0 con 0x55f372d83e40 2024-09-10T09:50:35.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:34.975+0000 7fb97fe00640 1 --2- 172.21.15.5:0/4283020843 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f372d83e40 0x55f372d868f0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fb97000e9c0 tx=0x7fb970009700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76022403b50e3673 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:50:35.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:34.975+0000 7fb974c00640 1 -- 172.21.15.5:0/4283020843 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb97009c360 con 0x55f372d83e40 2024-09-10T09:50:35.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:34.975+0000 7fb974c00640 1 -- 172.21.15.5:0/4283020843 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb9700b1070 con 0x55f372d83e40 2024-09-10T09:50:35.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:34.975+0000 7fb974c00640 1 -- 172.21.15.5:0/4283020843 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9700d7de0 con 0x55f372d83e40 2024-09-10T09:50:35.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:34.975+0000 7fb989a4f740 1 -- 172.21.15.5:0/4283020843 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f372d83e40 msgr2=0x55f372d868f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:50:35.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:34.975+0000 7fb989a4f740 1 --2- 172.21.15.5:0/4283020843 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f372d83e40 0x55f372d868f0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fb97000e9c0 tx=0x7fb970009700 comp rx=0 tx=0).stop 2024-09-10T09:50:35.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:34.975+0000 7fb989a4f740 1 -- 172.21.15.5:0/4283020843 shutdown_connections 2024-09-10T09:50:35.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:34.975+0000 7fb989a4f740 1 --2- 172.21.15.5:0/4283020843 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f372d8f020 0x55f372d8dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:35.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:34.975+0000 7fb989a4f740 1 --2- 172.21.15.5:0/4283020843 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f372d84760 0x55f372d7b300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:35.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:34.975+0000 7fb989a4f740 1 --2- 172.21.15.5:0/4283020843 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f372d83e40 0x55f372d868f0 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:35.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:34.975+0000 7fb989a4f740 1 -- 172.21.15.5:0/4283020843 >> 172.21.15.5:0/4283020843 conn(0x55f372d803d0 msgr2=0x55f372d8d7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:50:35.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:34.975+0000 7fb989a4f740 1 -- 172.21.15.5:0/4283020843 shutdown_connections 2024-09-10T09:50:35.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:34.975+0000 7fb989a4f740 1 -- 172.21.15.5:0/4283020843 wait complete. 2024-09-10T09:50:35.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:34.975+0000 7fb989a4f740 1 -- start start 2024-09-10T09:50:35.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:34.975+0000 7fb989a4f740 1 Processor -- start 2024-09-10T09:50:35.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:34.975+0000 7fb989a4f740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f372d83e40 0x55f372d97700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:50:35.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:34.975+0000 7fb989a4f740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f372d84760 0x55f372d88c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:50:35.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:34.975+0000 7fb989a4f740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f372d8f020 0x55f372d89180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:50:35.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:34.975+0000 7fb989a4f740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fb9801b2370 con 0x55f372d84760 2024-09-10T09:50:35.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:34.975+0000 7fb989a4f740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fb9801b1ca0 con 0x55f372d8f020 2024-09-10T09:50:35.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:34.975+0000 7fb989a4f740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fb9801aeee0 con 0x55f372d83e40 2024-09-10T09:50:35.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:34.975+0000 7fb989a4f740 10 client.0 _ll_register_callbacks cb 0x7fb9780093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:50:35.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:34.975+0000 7fb97f400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f372d84760 0x55f372d88c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:50:35.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:34.975+0000 7fb97f400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f372d84760 0x55f372d88c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59494/0 (socket says 172.21.15.5:59494) 2024-09-10T09:50:35.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:34.975+0000 7fb97f400640 1 -- 172.21.15.5:0/89576398 learned_addr learned my addr 172.21.15.5:0/89576398 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:50:35.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:34.975+0000 7fb97fe00640 1 --2- 172.21.15.5:0/89576398 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f372d83e40 0x55f372d97700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:50:35.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:34.975+0000 7fb984e00640 1 --2- 172.21.15.5:0/89576398 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f372d8f020 0x55f372d89180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:50:35.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:34.975+0000 7fb97f400640 1 -- 172.21.15.5:0/89576398 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f372d83e40 msgr2=0x55f372d97700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:50:35.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:34.975+0000 7fb97f400640 1 --2- 172.21.15.5:0/89576398 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f372d83e40 0x55f372d97700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:35.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:34.975+0000 7fb97f400640 1 -- 172.21.15.5:0/89576398 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f372d8f020 msgr2=0x55f372d89180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:50:35.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:34.975+0000 7fb97f400640 1 --2- 172.21.15.5:0/89576398 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f372d8f020 0x55f372d89180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:35.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:34.975+0000 7fb97f400640 1 -- 172.21.15.5:0/89576398 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f372d899b0 con 0x55f372d84760 2024-09-10T09:50:35.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:34.975+0000 7fb984e00640 1 --2- 172.21.15.5:0/89576398 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f372d8f020 0x55f372d89180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:50:35.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:34.975+0000 7fb97f400640 1 --2- 172.21.15.5:0/89576398 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f372d84760 0x55f372d88c40 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fb96c02a940 tx=0x7fb96c004590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1832f673130d02b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:50:35.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:34.975+0000 7fb96a000640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-10T09:50:35.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:34.980+0000 7fb96a000640 1 -- 172.21.15.5:0/89576398 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb96c057060 con 0x55f372d84760 2024-09-10T09:50:35.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:34.980+0000 7fb96a000640 1 -- 172.21.15.5:0/89576398 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fb96c0051d0 con 0x55f372d84760 2024-09-10T09:50:35.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:34.980+0000 7fb989a4f740 1 -- 172.21.15.5:0/89576398 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb9700b1070 con 0x55f372d84760 2024-09-10T09:50:35.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:34.980+0000 7fb96a000640 1 -- 172.21.15.5:0/89576398 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fb96c035040 con 0x55f372d84760 2024-09-10T09:50:35.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:34.980+0000 7fb989a4f740 10 client.17841 fetch_fsmap learned FSMap version 36 2024-09-10T09:50:35.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:34.980+0000 7fb989a4f740 1 -- 172.21.15.5:0/89576398 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55f372e09d30 con 0x55f372d84760 2024-09-10T09:50:35.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:34.980+0000 7fb96a000640 1 -- 172.21.15.5:0/89576398 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb9700b1070 con 0x55f372d84760 2024-09-10T09:50:35.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:34.980+0000 7fb96a000640 1 -- 172.21.15.5:0/89576398 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 36) ==== 8370+0+0 (secure 0 0 0) 0x7fb96c0046c0 con 0x55f372d84760 2024-09-10T09:50:35.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:34.980+0000 7fb989a4f740 10 client.17841 fetch_fsmap finished waiting for FSMap version 36 2024-09-10T09:50:35.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:34.980+0000 7fb989a4f740 10 client.17841 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T09:50:35.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:34.980+0000 7fb989a4f740 20 client.17841 populate_metadata read hostname 'smithi005' 2024-09-10T09:50:35.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:34.980+0000 7fb989a4f740 1 --2- 172.21.15.5:0/89576398 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55f372e0a4b0 0x55f372e0c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:50:35.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:34.980+0000 7fb989a4f740 4 client.17841 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T09:50:35.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:34.980+0000 7fb989a4f740 1 -- 172.21.15.5:0/89576398 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f372cf1e60 con 0x55f372e0a4b0 2024-09-10T09:50:35.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:34.980+0000 7fb97fe00640 1 --2- 172.21.15.5:0/89576398 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55f372e0a4b0 0x55f372e0c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:50:35.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:34.980+0000 7fb97fe00640 1 --2- 172.21.15.5:0/89576398 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55f372e0a4b0 0x55f372e0c8d0 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6c5777c0139194e0 server_cookie=dbe3f7c537ce93 in_seq=0 out_seq=0 2024-09-10T09:50:35.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:34.980+0000 7fb96a000640 10 client.17841 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T09:50:35.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:34.980+0000 7fb96a000640 1 -- 172.21.15.5:0/89576398 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f372cf1e60 con 0x55f372e0a4b0 2024-09-10T09:50:35.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:34.980+0000 7fb96a000640 10 client.17841 handle_command_reply: tid=0 2024-09-10T09:50:35.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:34.980+0000 7fb989a4f740 1 client.17841 shutdown 2024-09-10T09:50:35.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:34.980+0000 7fb989a4f740 1 -- 172.21.15.5:0/89576398 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f372d84760 msgr2=0x55f372d88c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:50:35.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:34.980+0000 7fb989a4f740 1 --2- 172.21.15.5:0/89576398 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f372d84760 0x55f372d88c40 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fb96c02a940 tx=0x7fb96c004590 comp rx=0 tx=0).stop 2024-09-10T09:50:35.279 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-10T09:50:35.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:34.980+0000 7fb989a4f740 1 -- 172.21.15.5:0/89576398 shutdown_connections 2024-09-10T09:50:35.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:34.980+0000 7fb989a4f740 1 --2- 172.21.15.5:0/89576398 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55f372e0a4b0 0x55f372e0c8d0 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:35.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:34.980+0000 7fb989a4f740 1 --2- 172.21.15.5:0/89576398 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f372d8f020 0x55f372d89180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:35.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:34.980+0000 7fb989a4f740 1 --2- 172.21.15.5:0/89576398 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f372d84760 0x55f372d88c40 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:35.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:34.980+0000 7fb989a4f740 1 --2- 172.21.15.5:0/89576398 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f372d83e40 0x55f372d97700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:35.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:34.980+0000 7fb989a4f740 1 -- 172.21.15.5:0/89576398 >> 172.21.15.5:0/89576398 conn(0x55f372d803d0 msgr2=0x55f372d8dff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:50:35.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:34.980+0000 7fb96a000640 0 client.17841 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T09:50:35.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:34.980+0000 7fb989a4f740 1 -- 172.21.15.5:0/89576398 shutdown_connections 2024-09-10T09:50:35.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:34.980+0000 7fb989a4f740 1 -- 172.21.15.5:0/89576398 wait complete. 2024-09-10T09:50:35.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:34.980+0000 7fb989a4f740 20 client.17841 trim_cache size 0 max 16384 2024-09-10T09:50:35.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:34.980+0000 7fb985800640 1 -- 172.21.15.5:0/4281099939 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fb9640a2b80 msgr2=0x7fb9640a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:50:35.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:34.980+0000 7fb985800640 1 --2- 172.21.15.5:0/4281099939 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fb9640a2b80 0x7fb9640a5040 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fb9780049d0 tx=0x7fb978009500 comp rx=0 tx=0).stop 2024-09-10T09:50:35.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:34.980+0000 7fb985800640 1 -- 172.21.15.5:0/4281099939 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb9801019c0 msgr2=0x7fb9800744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:50:35.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:34.980+0000 7fb985800640 1 --2- 172.21.15.5:0/4281099939 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb9801019c0 0x7fb9800744f0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fb97000e9f0 tx=0x7fb97000eeb0 comp rx=0 tx=0).stop 2024-09-10T09:50:35.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:34.980+0000 7fb985800640 1 -- 172.21.15.5:0/4281099939 shutdown_connections 2024-09-10T09:50:35.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:34.980+0000 7fb985800640 1 --2- 172.21.15.5:0/4281099939 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fb9640a2b80 0x7fb9640a5040 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:35.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:34.980+0000 7fb985800640 1 --2- 172.21.15.5:0/4281099939 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb9801079c0 0x7fb980074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:35.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:34.980+0000 7fb985800640 1 --2- 172.21.15.5:0/4281099939 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb980102360 0x7fb980074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:35.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:34.980+0000 7fb985800640 1 --2- 172.21.15.5:0/4281099939 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb9801019c0 0x7fb9800744f0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:35.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:34.980+0000 7fb985800640 1 -- 172.21.15.5:0/4281099939 >> 172.21.15.5:0/4281099939 conn(0x7fb9800fd740 msgr2=0x7fb9800fe010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:50:35.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:34.980+0000 7fb985800640 1 -- 172.21.15.5:0/4281099939 shutdown_connections 2024-09-10T09:50:35.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:34.980+0000 7fb985800640 1 -- 172.21.15.5:0/4281099939 wait complete. 2024-09-10T09:50:36.019 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:50:35 smithi005 ceph-mon[30350]: pgmap v760: 129 pgs: 129 active+clean; 293 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.4 MiB/s wr, 163 op/s 2024-09-10T09:50:36.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:50:35 smithi159 ceph-mon[35513]: pgmap v760: 129 pgs: 129 active+clean; 293 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.4 MiB/s wr, 163 op/s 2024-09-10T09:50:36.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:50:35 smithi102 ceph-mon[35442]: pgmap v760: 129 pgs: 129 active+clean; 293 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.4 MiB/s wr, 163 op/s 2024-09-10T09:50:36.135 INFO:tasks.workunit.client.0.smithi005.stdout: 1 782211 6.43 MB/sec execute 497 sec latency 2037.083 ms 2024-09-10T09:50:36.681 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24542 quiesce db / --timeout 90 --expiration 138.1 --await-for 45.0' 2024-09-10T09:50:36.681 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24542 quiesce db / --timeout 90 --expiration 138.1 --await-for 45.0 2024-09-10T09:50:37.060 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-10T09:50:37.061 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-10T09:50:37.086 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:50:37.135 INFO:tasks.workunit.client.0.smithi005.stdout: 1 784897 6.45 MB/sec execute 498 sec latency 16.560 ms 2024-09-10T09:50:37.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:37.695+0000 7ff426a00640 1 -- 172.21.15.5:0/1062527434 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff4201047e0 msgr2=0x7ff420108830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:50:37.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:37.695+0000 7ff426a00640 1 --2- 172.21.15.5:0/1062527434 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff4201047e0 0x7ff420108830 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7ff40c009860 tx=0x7ff40c02a340 comp rx=0 tx=0).stop 2024-09-10T09:50:37.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:37.696+0000 7ff426a00640 1 -- 172.21.15.5:0/1062527434 shutdown_connections 2024-09-10T09:50:37.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:37.696+0000 7ff426a00640 1 --2- 172.21.15.5:0/1062527434 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff420108f60 0x7ff420110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:37.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:37.696+0000 7ff426a00640 1 --2- 172.21.15.5:0/1062527434 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff4201047e0 0x7ff420108830 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:37.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:37.696+0000 7ff426a00640 1 --2- 172.21.15.5:0/1062527434 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff420103e30 0x7ff420104210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:37.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:37.696+0000 7ff426a00640 1 -- 172.21.15.5:0/1062527434 >> 172.21.15.5:0/1062527434 conn(0x7ff4200ff910 msgr2=0x7ff420101d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:50:37.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:37.696+0000 7ff426a00640 1 -- 172.21.15.5:0/1062527434 shutdown_connections 2024-09-10T09:50:37.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:37.696+0000 7ff426a00640 1 -- 172.21.15.5:0/1062527434 wait complete. 2024-09-10T09:50:37.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:37.697+0000 7ff426a00640 1 Processor -- start 2024-09-10T09:50:37.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:37.697+0000 7ff426a00640 1 -- start start 2024-09-10T09:50:37.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:37.698+0000 7ff426a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff420103e30 0x7ff42019a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:50:37.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:37.698+0000 7ff426a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff4201047e0 0x7ff42019a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:50:37.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:37.698+0000 7ff426a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff420108f60 0x7ff42019ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:50:37.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:37.698+0000 7ff426a00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff420113fc0 con 0x7ff420103e30 2024-09-10T09:50:37.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:37.698+0000 7ff426a00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7ff420113e40 con 0x7ff4201047e0 2024-09-10T09:50:37.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:37.698+0000 7ff426a00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7ff420114140 con 0x7ff420108f60 2024-09-10T09:50:37.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:37.698+0000 7ff424c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff4201047e0 0x7ff42019a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:50:37.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:37.698+0000 7ff425600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff420103e30 0x7ff42019a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:50:37.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:37.698+0000 7ff424c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff4201047e0 0x7ff42019a9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:35830/0 (socket says 172.21.15.5:35830) 2024-09-10T09:50:37.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:37.698+0000 7ff424c00640 1 -- 172.21.15.5:0/3334137330 learned_addr learned my addr 172.21.15.5:0/3334137330 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:50:37.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:37.699+0000 7ff426000640 1 --2- 172.21.15.5:0/3334137330 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff420108f60 0x7ff42019ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:50:37.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:37.699+0000 7ff424c00640 1 -- 172.21.15.5:0/3334137330 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff420108f60 msgr2=0x7ff42019ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:50:37.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:37.699+0000 7ff424c00640 1 --2- 172.21.15.5:0/3334137330 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff420108f60 0x7ff42019ed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:37.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:37.699+0000 7ff424c00640 1 -- 172.21.15.5:0/3334137330 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff420103e30 msgr2=0x7ff42019a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:50:37.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:37.699+0000 7ff424c00640 1 --2- 172.21.15.5:0/3334137330 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff420103e30 0x7ff42019a4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:37.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:37.699+0000 7ff424c00640 1 -- 172.21.15.5:0/3334137330 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff42019f460 con 0x7ff4201047e0 2024-09-10T09:50:37.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:37.699+0000 7ff424c00640 1 --2- 172.21.15.5:0/3334137330 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff4201047e0 0x7ff42019a9f0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7ff40c035040 tx=0x7ff40c033b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:50:37.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:37.700+0000 7ff41e000640 1 -- 172.21.15.5:0/3334137330 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff40c037040 con 0x7ff4201047e0 2024-09-10T09:50:37.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:37.700+0000 7ff41e000640 1 -- 172.21.15.5:0/3334137330 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff40c040070 con 0x7ff4201047e0 2024-09-10T09:50:37.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:37.700+0000 7ff41e000640 1 -- 172.21.15.5:0/3334137330 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff40c0039b0 con 0x7ff4201047e0 2024-09-10T09:50:37.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:37.700+0000 7ff426a00640 1 -- 172.21.15.5:0/3334137330 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff42019f6f0 con 0x7ff4201047e0 2024-09-10T09:50:37.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:37.700+0000 7ff426a00640 1 -- 172.21.15.5:0/3334137330 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff42019fb80 con 0x7ff4201047e0 2024-09-10T09:50:37.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:37.704+0000 7ff41e000640 1 -- 172.21.15.5:0/3334137330 <== mon.2 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7ff40c0305e0 con 0x7ff4201047e0 2024-09-10T09:50:37.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:37.705+0000 7ff42abc6740 1 Processor -- start 2024-09-10T09:50:37.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:37.705+0000 7ff42abc6740 1 -- start start 2024-09-10T09:50:37.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:37.705+0000 7ff42abc6740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5567b80fb560 0x5567b80fb940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:50:37.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:37.705+0000 7ff42abc6740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5567b80fbe80 0x5567b8106200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:50:37.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:37.705+0000 7ff42abc6740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5567b8106740 0x5567b8108b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:50:37.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:37.705+0000 7ff41e000640 1 --2- 172.21.15.5:0/3334137330 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7ff4080a2c00 0x7ff4080a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:50:37.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:37.705+0000 7ff42abc6740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5567b8061530 con 0x5567b80fb560 2024-09-10T09:50:37.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:37.705+0000 7ff42abc6740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x5567b8008860 con 0x5567b80fbe80 2024-09-10T09:50:37.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:37.705+0000 7ff42abc6740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5567b805d4f0 con 0x5567b8106740 2024-09-10T09:50:37.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:37.706+0000 7ff41e000640 1 -- 172.21.15.5:0/3334137330 <== mon.2 v2:172.21.15.102:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7ff40c0ef380 con 0x7ff4201047e0 2024-09-10T09:50:37.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:37.706+0000 7ff425600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5567b80fbe80 0x5567b8106200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:50:37.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:37.706+0000 7ff425600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5567b80fbe80 0x5567b8106200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:35834/0 (socket says 172.21.15.5:35834) 2024-09-10T09:50:37.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:37.706+0000 7ff425600640 1 -- 172.21.15.5:0/2551508772 learned_addr learned my addr 172.21.15.5:0/2551508772 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:50:37.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:37.706+0000 7ff426000640 1 --2- 172.21.15.5:0/2551508772 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5567b80fb560 0x5567b80fb940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:50:37.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:37.706+0000 7ff426000640 1 --2- 172.21.15.5:0/2551508772 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5567b8106740 0x5567b8108b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:50:37.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:37.706+0000 7ff425600640 1 -- 172.21.15.5:0/2551508772 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5567b8106740 msgr2=0x5567b8108b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:50:37.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:37.706+0000 7ff425600640 1 --2- 172.21.15.5:0/2551508772 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5567b8106740 0x5567b8108b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:37.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:37.706+0000 7ff425600640 1 -- 172.21.15.5:0/2551508772 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5567b80fb560 msgr2=0x5567b80fb940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:50:37.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:37.706+0000 7ff425600640 1 --2- 172.21.15.5:0/2551508772 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5567b80fb560 0x5567b80fb940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:37.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:37.706+0000 7ff424c00640 1 --2- 172.21.15.5:0/3334137330 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7ff4080a2c00 0x7ff4080a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:50:37.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:37.706+0000 7ff425600640 1 -- 172.21.15.5:0/2551508772 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5567b7dd2b90 con 0x5567b80fbe80 2024-09-10T09:50:37.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:37.706+0000 7ff426000640 1 --2- 172.21.15.5:0/2551508772 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5567b80fb560 0x5567b80fb940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:50:37.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:37.707+0000 7ff425600640 1 --2- 172.21.15.5:0/2551508772 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5567b80fbe80 0x5567b8106200 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7ff414009cd0 tx=0x7ff414011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c95ecda1b8a4743a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:50:37.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:37.707+0000 7ff424c00640 1 --2- 172.21.15.5:0/3334137330 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7ff4080a2c00 0x7ff4080a50c0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7ff40c0b7bf0 tx=0x7ff40c033f50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:50:37.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:37.707+0000 7ff412600640 1 -- 172.21.15.5:0/2551508772 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff41401e070 con 0x5567b80fbe80 2024-09-10T09:50:37.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:37.707+0000 7ff412600640 1 -- 172.21.15.5:0/2551508772 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff414019b10 con 0x5567b80fbe80 2024-09-10T09:50:37.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:37.707+0000 7ff412600640 1 -- 172.21.15.5:0/2551508772 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff41400eb50 con 0x5567b80fbe80 2024-09-10T09:50:37.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:37.709+0000 7ff42abc6740 1 -- 172.21.15.5:0/2551508772 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5567b80fbe80 msgr2=0x5567b8106200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:50:37.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:37.709+0000 7ff42abc6740 1 --2- 172.21.15.5:0/2551508772 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5567b80fbe80 0x5567b8106200 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7ff414009cd0 tx=0x7ff414011810 comp rx=0 tx=0).stop 2024-09-10T09:50:37.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:37.709+0000 7ff42abc6740 1 -- 172.21.15.5:0/2551508772 shutdown_connections 2024-09-10T09:50:37.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:37.709+0000 7ff42abc6740 1 --2- 172.21.15.5:0/2551508772 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5567b8106740 0x5567b8108b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:37.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:37.709+0000 7ff42abc6740 1 --2- 172.21.15.5:0/2551508772 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5567b80fbe80 0x5567b8106200 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:37.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:37.709+0000 7ff42abc6740 1 --2- 172.21.15.5:0/2551508772 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5567b80fb560 0x5567b80fb940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:37.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:37.709+0000 7ff42abc6740 1 -- 172.21.15.5:0/2551508772 >> 172.21.15.5:0/2551508772 conn(0x5567b80f7af0 msgr2=0x5567b80f7ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:50:37.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:37.709+0000 7ff42abc6740 1 -- 172.21.15.5:0/2551508772 shutdown_connections 2024-09-10T09:50:37.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:37.709+0000 7ff42abc6740 1 -- 172.21.15.5:0/2551508772 wait complete. 2024-09-10T09:50:37.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:37.709+0000 7ff42abc6740 1 -- start start 2024-09-10T09:50:37.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:37.710+0000 7ff42abc6740 1 Processor -- start 2024-09-10T09:50:37.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:37.710+0000 7ff42abc6740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5567b80fb560 0x5567b8103610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:50:37.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:37.710+0000 7ff42abc6740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5567b80fbe80 0x5567b8103b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:50:37.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:37.710+0000 7ff42abc6740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5567b8106740 0x5567b8104090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:50:37.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:37.710+0000 7ff42abc6740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5567b8109310 con 0x5567b8106740 2024-09-10T09:50:37.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:37.710+0000 7ff42abc6740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x5567b801be60 con 0x5567b80fbe80 2024-09-10T09:50:37.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:37.710+0000 7ff42abc6740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5567b8109490 con 0x5567b80fb560 2024-09-10T09:50:37.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:37.710+0000 7ff42abc6740 10 client.0 _ll_register_callbacks cb 0x5567b8020cc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:50:37.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:37.711+0000 7ff425600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5567b80fbe80 0x5567b8103b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:50:37.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:37.711+0000 7ff425600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5567b80fbe80 0x5567b8103b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:35844/0 (socket says 172.21.15.5:35844) 2024-09-10T09:50:37.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:37.711+0000 7ff425600640 1 -- 172.21.15.5:0/3574201111 learned_addr learned my addr 172.21.15.5:0/3574201111 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:50:37.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:37.711+0000 7ff425600640 1 -- 172.21.15.5:0/3574201111 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5567b80fb560 msgr2=0x5567b8103610 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T09:50:37.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:37.711+0000 7ff425600640 1 --2- 172.21.15.5:0/3574201111 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5567b80fb560 0x5567b8103610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:37.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:37.711+0000 7ff425600640 1 -- 172.21.15.5:0/3574201111 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5567b8106740 msgr2=0x5567b8104090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:50:37.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:37.711+0000 7ff426000640 1 --2- 172.21.15.5:0/3574201111 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5567b8106740 0x5567b8104090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:50:37.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:37.711+0000 7ff425600640 1 --2- 172.21.15.5:0/3574201111 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5567b8106740 0x5567b8104090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:37.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:37.711+0000 7ff425600640 1 -- 172.21.15.5:0/3574201111 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5567b8027990 con 0x5567b80fbe80 2024-09-10T09:50:37.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:37.711+0000 7ff426000640 1 --2- 172.21.15.5:0/3574201111 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5567b8106740 0x5567b8104090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T09:50:37.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:37.712+0000 7ff425600640 1 --2- 172.21.15.5:0/3574201111 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5567b80fbe80 0x5567b8103b50 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7ff41401c040 tx=0x7ff4140198c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2d1ca081306919dc server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:50:37.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:37.712+0000 7ff406a00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-09-10T09:50:37.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:37.712+0000 7ff406a00640 1 -- 172.21.15.5:0/3574201111 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff414028070 con 0x5567b80fbe80 2024-09-10T09:50:37.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:37.712+0000 7ff42abc6740 1 -- 172.21.15.5:0/3574201111 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff414019b10 con 0x5567b80fbe80 2024-09-10T09:50:37.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:37.712+0000 7ff406a00640 1 -- 172.21.15.5:0/3574201111 <== mon.2 v2:172.21.15.102:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7ff414005040 con 0x5567b80fbe80 2024-09-10T09:50:37.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:37.712+0000 7ff406a00640 1 -- 172.21.15.5:0/3574201111 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7ff414034050 con 0x5567b80fbe80 2024-09-10T09:50:37.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:37.713+0000 7ff42abc6740 10 client.27872 fetch_fsmap learned FSMap version 36 2024-09-10T09:50:37.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:37.713+0000 7ff42abc6740 1 -- 172.21.15.5:0/3574201111 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x5567b8114fa0 con 0x5567b80fbe80 2024-09-10T09:50:37.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:37.713+0000 7ff406a00640 1 -- 172.21.15.5:0/3574201111 <== mon.2 v2:172.21.15.102:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff414019b10 con 0x5567b80fbe80 2024-09-10T09:50:37.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:37.714+0000 7ff406a00640 1 -- 172.21.15.5:0/3574201111 <== mon.2 v2:172.21.15.102:3300/0 5 ==== fsmap(e 36) ==== 8370+0+0 (secure 0 0 0) 0x7ff41400ec10 con 0x5567b80fbe80 2024-09-10T09:50:37.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:37.714+0000 7ff42abc6740 10 client.27872 fetch_fsmap finished waiting for FSMap version 36 2024-09-10T09:50:37.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:37.714+0000 7ff42abc6740 10 client.27872 resolve_mds: validated gid 24542 aka daemon mds.b 2024-09-10T09:50:37.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:37.714+0000 7ff42abc6740 20 client.27872 populate_metadata read hostname 'smithi005' 2024-09-10T09:50:37.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:37.714+0000 7ff42abc6740 1 --2- 172.21.15.5:0/3574201111 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x5567b81163b0 0x5567b81187a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:50:37.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:37.714+0000 7ff42abc6740 4 client.27872 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:50:37.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:37.714+0000 7ff42abc6740 1 -- 172.21.15.5:0/3574201111 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5567b8004310 con 0x5567b81163b0 2024-09-10T09:50:37.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:37.714+0000 7ff426000640 1 --2- 172.21.15.5:0/3574201111 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x5567b81163b0 0x5567b81187a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:50:37.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:37.715+0000 7ff426000640 1 --2- 172.21.15.5:0/3574201111 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x5567b81163b0 0x5567b81187a0 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d6aabc746bcd83dc server_cookie=ee7d03b3690bb327 in_seq=0 out_seq=0 2024-09-10T09:50:37.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:37.715+0000 7ff406a00640 10 client.27872 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T09:50:37.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:37.715+0000 7ff406a00640 1 -- 172.21.15.5:0/3574201111 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5567b8004310 con 0x5567b81163b0 2024-09-10T09:50:37.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:37.715+0000 7ff406a00640 10 client.27872 handle_command_reply: tid=0 2024-09-10T09:50:37.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:37.715+0000 7ff42abc6740 1 client.27872 shutdown 2024-09-10T09:50:37.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:37.716+0000 7ff42abc6740 1 -- 172.21.15.5:0/3574201111 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5567b80fbe80 msgr2=0x5567b8103b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:50:37.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:37.716+0000 7ff42abc6740 1 --2- 172.21.15.5:0/3574201111 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5567b80fbe80 0x5567b8103b50 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7ff41401c040 tx=0x7ff4140198c0 comp rx=0 tx=0).stop 2024-09-10T09:50:37.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:37.716+0000 7ff42abc6740 1 -- 172.21.15.5:0/3574201111 shutdown_connections 2024-09-10T09:50:37.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:37.716+0000 7ff42abc6740 1 --2- 172.21.15.5:0/3574201111 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x5567b81163b0 0x5567b81187a0 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:37.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:37.716+0000 7ff42abc6740 1 --2- 172.21.15.5:0/3574201111 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5567b8106740 0x5567b8104090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:37.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:37.716+0000 7ff42abc6740 1 --2- 172.21.15.5:0/3574201111 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5567b80fbe80 0x5567b8103b50 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:37.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:37.716+0000 7ff42abc6740 1 --2- 172.21.15.5:0/3574201111 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5567b80fb560 0x5567b8103610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:37.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:37.716+0000 7ff42abc6740 1 -- 172.21.15.5:0/3574201111 >> 172.21.15.5:0/3574201111 conn(0x5567b80f7af0 msgr2=0x5567b80fd820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:50:37.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:37.716+0000 7ff406a00640 0 client.27872 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T09:50:37.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:37.716+0000 7ff42abc6740 1 -- 172.21.15.5:0/3574201111 shutdown_connections 2024-09-10T09:50:37.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:37.716+0000 7ff42abc6740 1 -- 172.21.15.5:0/3574201111 wait complete. 2024-09-10T09:50:37.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:37.716+0000 7ff42abc6740 20 client.27872 trim_cache size 0 max 16384 2024-09-10T09:50:37.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:37.739+0000 7ff42abc6740 1 Processor -- start 2024-09-10T09:50:37.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:37.739+0000 7ff42abc6740 1 -- start start 2024-09-10T09:50:37.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:37.739+0000 7ff42abc6740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5567b80fb560 0x5567b8102550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:50:37.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:37.739+0000 7ff42abc6740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5567b80fbe80 0x5567b811df50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:50:37.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:37.739+0000 7ff42abc6740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5567b8106740 0x5567b811e490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:50:37.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:37.739+0000 7ff42abc6740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff4201b0f30 con 0x5567b80fb560 2024-09-10T09:50:37.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:37.739+0000 7ff42abc6740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7ff4201b2a70 con 0x5567b80fbe80 2024-09-10T09:50:37.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:37.739+0000 7ff42abc6740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7ff4201b1600 con 0x5567b8106740 2024-09-10T09:50:37.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:37.740+0000 7ff425600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5567b80fbe80 0x5567b811df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:50:37.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:37.740+0000 7ff425600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5567b80fbe80 0x5567b811df50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:35846/0 (socket says 172.21.15.5:35846) 2024-09-10T09:50:37.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:37.740+0000 7ff425600640 1 -- 172.21.15.5:0/3244735044 learned_addr learned my addr 172.21.15.5:0/3244735044 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:50:37.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:37.740+0000 7ff426000640 1 --2- 172.21.15.5:0/3244735044 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5567b80fb560 0x5567b8102550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:50:37.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:37.740+0000 7ff426000640 1 --2- 172.21.15.5:0/3244735044 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5567b8106740 0x5567b811e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:50:37.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:37.740+0000 7ff425600640 1 -- 172.21.15.5:0/3244735044 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5567b8106740 msgr2=0x5567b811e490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:50:37.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:37.740+0000 7ff425600640 1 --2- 172.21.15.5:0/3244735044 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5567b8106740 0x5567b811e490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:37.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:37.740+0000 7ff425600640 1 -- 172.21.15.5:0/3244735044 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5567b80fb560 msgr2=0x5567b8102550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:50:37.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:37.740+0000 7ff425600640 1 --2- 172.21.15.5:0/3244735044 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5567b80fb560 0x5567b8102550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:37.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:37.740+0000 7ff425600640 1 -- 172.21.15.5:0/3244735044 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5567b80f7920 con 0x5567b80fbe80 2024-09-10T09:50:37.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:37.740+0000 7ff425600640 1 --2- 172.21.15.5:0/3244735044 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5567b80fbe80 0x5567b811df50 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7ff414006010 tx=0x7ff414009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a2f5c78b602a8386 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:50:37.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:37.741+0000 7ff411c00640 1 -- 172.21.15.5:0/3244735044 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff414028070 con 0x5567b80fbe80 2024-09-10T09:50:37.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:37.741+0000 7ff411c00640 1 -- 172.21.15.5:0/3244735044 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff414019740 con 0x5567b80fbe80 2024-09-10T09:50:37.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:37.741+0000 7ff411c00640 1 -- 172.21.15.5:0/3244735044 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff41401e040 con 0x5567b80fbe80 2024-09-10T09:50:37.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:37.742+0000 7ff42abc6740 1 -- 172.21.15.5:0/3244735044 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5567b80fbe80 msgr2=0x5567b811df50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:50:37.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:37.742+0000 7ff42abc6740 1 --2- 172.21.15.5:0/3244735044 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5567b80fbe80 0x5567b811df50 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7ff414006010 tx=0x7ff414009260 comp rx=0 tx=0).stop 2024-09-10T09:50:37.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:37.742+0000 7ff42abc6740 1 -- 172.21.15.5:0/3244735044 shutdown_connections 2024-09-10T09:50:37.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:37.742+0000 7ff42abc6740 1 --2- 172.21.15.5:0/3244735044 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5567b8106740 0x5567b811e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:37.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:37.742+0000 7ff42abc6740 1 --2- 172.21.15.5:0/3244735044 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5567b80fbe80 0x5567b811df50 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:37.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:37.742+0000 7ff42abc6740 1 --2- 172.21.15.5:0/3244735044 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5567b80fb560 0x5567b8102550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:37.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:37.742+0000 7ff42abc6740 1 -- 172.21.15.5:0/3244735044 >> 172.21.15.5:0/3244735044 conn(0x5567b80f7af0 msgr2=0x5567b8105580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:50:37.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:37.742+0000 7ff42abc6740 1 -- 172.21.15.5:0/3244735044 shutdown_connections 2024-09-10T09:50:37.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:37.742+0000 7ff42abc6740 1 -- 172.21.15.5:0/3244735044 wait complete. 2024-09-10T09:50:37.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:37.743+0000 7ff42abc6740 1 -- start start 2024-09-10T09:50:37.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:37.743+0000 7ff42abc6740 1 Processor -- start 2024-09-10T09:50:37.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:37.744+0000 7ff42abc6740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5567b80fb560 0x5567b810ee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:50:37.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:37.744+0000 7ff42abc6740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5567b80fbe80 0x5567b810a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:50:37.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:37.744+0000 7ff42abc6740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5567b8106740 0x5567b810a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:50:37.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:37.744+0000 7ff42abc6740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff4201b23a0 con 0x5567b8106740 2024-09-10T09:50:37.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:37.744+0000 7ff42abc6740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7ff4201b1cd0 con 0x5567b80fb560 2024-09-10T09:50:37.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:37.744+0000 7ff42abc6740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7ff4201aef10 con 0x5567b80fbe80 2024-09-10T09:50:37.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:37.744+0000 7ff42abc6740 10 client.0 _ll_register_callbacks cb 0x7ff41401c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:50:37.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:37.744+0000 7ff426000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5567b8106740 0x5567b810a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:50:37.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:37.744+0000 7ff426000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5567b8106740 0x5567b810a6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59536/0 (socket says 172.21.15.5:59536) 2024-09-10T09:50:37.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:37.744+0000 7ff426000640 1 -- 172.21.15.5:0/3968388022 learned_addr learned my addr 172.21.15.5:0/3968388022 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:50:37.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:37.744+0000 7ff425600640 1 --2- 172.21.15.5:0/3968388022 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5567b80fbe80 0x5567b810a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:50:37.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:37.744+0000 7ff426000640 1 --2- 172.21.15.5:0/3968388022 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5567b80fb560 0x5567b810ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:50:37.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:37.745+0000 7ff425600640 1 -- 172.21.15.5:0/3968388022 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5567b80fb560 msgr2=0x5567b810ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:50:37.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:37.745+0000 7ff425600640 1 --2- 172.21.15.5:0/3968388022 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5567b80fb560 0x5567b810ee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:37.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:37.745+0000 7ff425600640 1 -- 172.21.15.5:0/3968388022 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5567b8106740 msgr2=0x5567b810a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:50:37.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:37.745+0000 7ff425600640 1 --2- 172.21.15.5:0/3968388022 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5567b8106740 0x5567b810a6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:37.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:37.745+0000 7ff425600640 1 -- 172.21.15.5:0/3968388022 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5567b810af40 con 0x5567b80fbe80 2024-09-10T09:50:37.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:37.745+0000 7ff425600640 1 --2- 172.21.15.5:0/3968388022 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5567b80fbe80 0x5567b810a1b0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x5567b80f74f0 tx=0x7ff414019a00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8f6bc162191d6afc server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:50:37.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:37.745+0000 7ff406a00640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-09-10T09:50:37.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:37.745+0000 7ff406a00640 1 -- 172.21.15.5:0/3968388022 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff414028040 con 0x5567b80fbe80 2024-09-10T09:50:37.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:37.746+0000 7ff406a00640 1 -- 172.21.15.5:0/3968388022 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7ff414019d10 con 0x5567b80fbe80 2024-09-10T09:50:37.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:37.746+0000 7ff42abc6740 1 -- 172.21.15.5:0/3968388022 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff414019740 con 0x5567b80fbe80 2024-09-10T09:50:37.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:37.746+0000 7ff406a00640 1 -- 172.21.15.5:0/3968388022 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7ff414049050 con 0x5567b80fbe80 2024-09-10T09:50:37.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:37.746+0000 7ff42abc6740 10 client.27880 fetch_fsmap learned FSMap version 36 2024-09-10T09:50:37.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:37.746+0000 7ff42abc6740 1 -- 172.21.15.5:0/3968388022 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x5567b8189740 con 0x5567b80fbe80 2024-09-10T09:50:37.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:37.746+0000 7ff406a00640 1 -- 172.21.15.5:0/3968388022 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff4140098e0 con 0x5567b80fbe80 2024-09-10T09:50:37.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:37.747+0000 7ff406a00640 1 -- 172.21.15.5:0/3968388022 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 36) ==== 8370+0+0 (secure 0 0 0) 0x7ff414023790 con 0x5567b80fbe80 2024-09-10T09:50:37.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:37.747+0000 7ff42abc6740 10 client.27880 fetch_fsmap finished waiting for FSMap version 36 2024-09-10T09:50:37.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:37.747+0000 7ff42abc6740 10 client.27880 resolve_mds: validated gid 24542 aka daemon mds.b 2024-09-10T09:50:37.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:37.747+0000 7ff42abc6740 20 client.27880 populate_metadata read hostname 'smithi005' 2024-09-10T09:50:37.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:37.747+0000 7ff42abc6740 1 --2- 172.21.15.5:0/3968388022 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x5567b8189dc0 0x5567b818c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:50:37.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:37.747+0000 7ff42abc6740 4 client.27880 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 138.1, "await_for": 45.0}] 2024-09-10T09:50:37.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:37.747+0000 7ff42abc6740 1 -- 172.21.15.5:0/3968388022 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 138.1, "await_for": 45.0}) -- 0x7ff414019740 con 0x5567b8189dc0 2024-09-10T09:50:37.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:37.747+0000 7ff426000640 1 --2- 172.21.15.5:0/3968388022 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x5567b8189dc0 0x5567b818c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:50:37.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:37.748+0000 7ff406a00640 10 client.27880 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T09:50:37.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:37.748+0000 7ff426000640 1 --2- 172.21.15.5:0/3968388022 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x5567b8189dc0 0x5567b818c1e0 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bdbc8e0c553444ed server_cookie=a3eb6f5fdc0ec56f in_seq=0 out_seq=0 2024-09-10T09:50:37.767 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:50:37.929 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:50:37 smithi005 ceph-mon[30350]: pgmap v761: 129 pgs: 129 active+clean; 291 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.4 MiB/s wr, 205 op/s 2024-09-10T09:50:37.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:37.940+0000 7ff406a00640 1 -- 172.21.15.5:0/3968388022 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x7ff414019740 con 0x5567b8189dc0 2024-09-10T09:50:37.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:37.940+0000 7ff406a00640 10 client.27880 handle_command_reply: tid=0 2024-09-10T09:50:37.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:37.940+0000 7ff42abc6740 1 client.27880 shutdown 2024-09-10T09:50:37.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:37.941+0000 7ff42abc6740 1 -- 172.21.15.5:0/3968388022 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5567b80fbe80 msgr2=0x5567b810a1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:50:37.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:37.941+0000 7ff42abc6740 1 --2- 172.21.15.5:0/3968388022 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5567b80fbe80 0x5567b810a1b0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x5567b80f74f0 tx=0x7ff414019a00 comp rx=0 tx=0).stop 2024-09-10T09:50:37.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:37.941+0000 7ff42abc6740 1 -- 172.21.15.5:0/3968388022 shutdown_connections 2024-09-10T09:50:37.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:37.941+0000 7ff42abc6740 1 --2- 172.21.15.5:0/3968388022 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x5567b8189dc0 0x5567b818c1e0 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:37.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:37.941+0000 7ff42abc6740 1 --2- 172.21.15.5:0/3968388022 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5567b8106740 0x5567b810a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:37.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:37.941+0000 7ff42abc6740 1 --2- 172.21.15.5:0/3968388022 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5567b80fbe80 0x5567b810a1b0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:37.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:37.941+0000 7ff42abc6740 1 --2- 172.21.15.5:0/3968388022 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5567b80fb560 0x5567b810ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:37.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:37.941+0000 7ff42abc6740 1 -- 172.21.15.5:0/3968388022 >> 172.21.15.5:0/3968388022 conn(0x5567b80f7af0 msgr2=0x5567b80f6450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:50:37.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:37.941+0000 7ff406a00640 0 client.27880 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T09:50:37.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:37.942+0000 7ff42abc6740 1 -- 172.21.15.5:0/3968388022 shutdown_connections 2024-09-10T09:50:37.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:37.942+0000 7ff42abc6740 1 -- 172.21.15.5:0/3968388022 wait complete. 2024-09-10T09:50:37.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:37.942+0000 7ff42abc6740 20 client.27880 trim_cache size 0 max 16384 2024-09-10T09:50:37.944 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-10T09:50:37.944 INFO:teuthology.orchestra.run.smithi005.stdout: "epoch": 36, 2024-09-10T09:50:37.944 INFO:teuthology.orchestra.run.smithi005.stdout: "leader": 24542, 2024-09-10T09:50:37.944 INFO:teuthology.orchestra.run.smithi005.stdout: "set_version": 42, 2024-09-10T09:50:37.944 INFO:teuthology.orchestra.run.smithi005.stdout: "sets": { 2024-09-10T09:50:37.945 INFO:teuthology.orchestra.run.smithi005.stdout: "6b75eb2": { 2024-09-10T09:50:37.945 INFO:teuthology.orchestra.run.smithi005.stdout: "version": 42, 2024-09-10T09:50:37.945 INFO:teuthology.orchestra.run.smithi005.stdout: "age_ref": 0.0, 2024-09-10T09:50:37.945 INFO:teuthology.orchestra.run.smithi005.stdout: "state": { 2024-09-10T09:50:37.945 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "QUIESCED", 2024-09-10T09:50:37.945 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0 2024-09-10T09:50:37.945 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T09:50:37.945 INFO:teuthology.orchestra.run.smithi005.stdout: "timeout": 90.0, 2024-09-10T09:50:37.945 INFO:teuthology.orchestra.run.smithi005.stdout: "expiration": 138.1, 2024-09-10T09:50:37.945 INFO:teuthology.orchestra.run.smithi005.stdout: "members": { 2024-09-10T09:50:37.945 INFO:teuthology.orchestra.run.smithi005.stdout: "file:/": { 2024-09-10T09:50:37.945 INFO:teuthology.orchestra.run.smithi005.stdout: "excluded": false, 2024-09-10T09:50:37.945 INFO:teuthology.orchestra.run.smithi005.stdout: "state": { 2024-09-10T09:50:37.945 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "QUIESCED", 2024-09-10T09:50:37.946 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0 2024-09-10T09:50:37.946 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T09:50:37.946 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T09:50:37.946 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T09:50:37.946 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T09:50:37.946 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T09:50:37.946 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-10T09:50:37.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:37.943+0000 7ff426a00640 1 -- 172.21.15.5:0/3334137330 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7ff4080a2c00 msgr2=0x7ff4080a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:50:37.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:37.943+0000 7ff426a00640 1 --2- 172.21.15.5:0/3334137330 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7ff4080a2c00 0x7ff4080a50c0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7ff40c0b7bf0 tx=0x7ff40c033f50 comp rx=0 tx=0).stop 2024-09-10T09:50:37.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:37.943+0000 7ff426a00640 1 -- 172.21.15.5:0/3334137330 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff4201047e0 msgr2=0x7ff42019a9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:50:37.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:37.943+0000 7ff426a00640 1 --2- 172.21.15.5:0/3334137330 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff4201047e0 0x7ff42019a9f0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7ff40c035040 tx=0x7ff40c033b60 comp rx=0 tx=0).stop 2024-09-10T09:50:37.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:37.944+0000 7ff426a00640 1 -- 172.21.15.5:0/3334137330 shutdown_connections 2024-09-10T09:50:37.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:37.944+0000 7ff426a00640 1 --2- 172.21.15.5:0/3334137330 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7ff4080a2c00 0x7ff4080a50c0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:37.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:37.944+0000 7ff426a00640 1 --2- 172.21.15.5:0/3334137330 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff420108f60 0x7ff42019ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:37.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:37.944+0000 7ff426a00640 1 --2- 172.21.15.5:0/3334137330 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff4201047e0 0x7ff42019a9f0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:37.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:37.944+0000 7ff426a00640 1 --2- 172.21.15.5:0/3334137330 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff420103e30 0x7ff42019a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:37.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:37.944+0000 7ff426a00640 1 -- 172.21.15.5:0/3334137330 >> 172.21.15.5:0/3334137330 conn(0x7ff4200ff910 msgr2=0x7ff420100ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:50:37.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:37.944+0000 7ff426a00640 1 -- 172.21.15.5:0/3334137330 shutdown_connections 2024-09-10T09:50:37.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:37.944+0000 7ff426a00640 1 -- 172.21.15.5:0/3334137330 wait complete. 2024-09-10T09:50:38.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:50:37 smithi159 ceph-mon[35513]: pgmap v761: 129 pgs: 129 active+clean; 291 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.4 MiB/s wr, 205 op/s 2024-09-10T09:50:38.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:50:37 smithi102 ceph-mon[35442]: pgmap v761: 129 pgs: 129 active+clean; 291 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.4 MiB/s wr, 205 op/s 2024-09-10T09:50:38.135 INFO:tasks.workunit.client.0.smithi005.stdout: 1 785836 6.44 MB/sec execute 499 sec latency 381.636 ms 2024-09-10T09:50:38.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:38.667+0000 7fcf6be00640 1 -- 172.21.15.5:0/2369088538 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcf6411b140 msgr2=0x7fcf6411d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:50:38.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:38.669+0000 7fcf6be00640 1 --2- 172.21.15.5:0/2369088538 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcf6411b140 0x7fcf6411d530 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fcf5c009ca0 tx=0x7fcf5c02a8d0 comp rx=0 tx=0).stop 2024-09-10T09:50:38.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:38.669+0000 7fcf6be00640 1 -- 172.21.15.5:0/2369088538 shutdown_connections 2024-09-10T09:50:38.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:38.669+0000 7fcf6be00640 1 --2- 172.21.15.5:0/2369088538 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcf6411b140 0x7fcf6411d530 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:38.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:38.669+0000 7fcf6be00640 1 --2- 172.21.15.5:0/2369088538 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcf64109830 0x7fcf64109c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:38.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:38.669+0000 7fcf6be00640 1 --2- 172.21.15.5:0/2369088538 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcf64109450 0x7fcf64110250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:38.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:38.669+0000 7fcf6be00640 1 -- 172.21.15.5:0/2369088538 >> 172.21.15.5:0/2369088538 conn(0x7fcf6406e140 msgr2=0x7fcf6406e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:50:38.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:38.669+0000 7fcf6be00640 1 -- 172.21.15.5:0/2369088538 shutdown_connections 2024-09-10T09:50:38.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:38.669+0000 7fcf6be00640 1 -- 172.21.15.5:0/2369088538 wait complete. 2024-09-10T09:50:38.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:38.670+0000 7fcf6be00640 1 Processor -- start 2024-09-10T09:50:38.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:38.670+0000 7fcf6be00640 1 -- start start 2024-09-10T09:50:38.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:38.670+0000 7fcf6be00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcf64109450 0x7fcf641a72c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:50:38.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:38.670+0000 7fcf6be00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcf64109830 0x7fcf641a7800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:50:38.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:38.670+0000 7fcf6be00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcf6411b140 0x7fcf641abb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:50:38.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:38.670+0000 7fcf6be00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fcf64120e10 con 0x7fcf64109830 2024-09-10T09:50:38.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:38.670+0000 7fcf6be00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fcf64120c90 con 0x7fcf6411b140 2024-09-10T09:50:38.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:38.670+0000 7fcf6be00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fcf64120f90 con 0x7fcf64109450 2024-09-10T09:50:38.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:38.670+0000 7fcf69e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcf6411b140 0x7fcf641abb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:50:38.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:38.670+0000 7fcf69e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcf6411b140 0x7fcf641abb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:35872/0 (socket says 172.21.15.5:35872) 2024-09-10T09:50:38.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:38.670+0000 7fcf69e00640 1 -- 172.21.15.5:0/3238398898 learned_addr learned my addr 172.21.15.5:0/3238398898 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:50:38.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:38.670+0000 7fcf69e00640 1 -- 172.21.15.5:0/3238398898 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcf64109450 msgr2=0x7fcf641a72c0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-10T09:50:38.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:38.670+0000 7fcf69e00640 1 --2- 172.21.15.5:0/3238398898 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcf64109450 0x7fcf641a72c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:38.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:38.670+0000 7fcf69e00640 1 -- 172.21.15.5:0/3238398898 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcf64109830 msgr2=0x7fcf641a7800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:50:38.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:38.670+0000 7fcf69e00640 1 --2- 172.21.15.5:0/3238398898 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcf64109830 0x7fcf641a7800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:38.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:38.670+0000 7fcf69e00640 1 -- 172.21.15.5:0/3238398898 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf641ac270 con 0x7fcf6411b140 2024-09-10T09:50:38.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:38.683+0000 7fcf69e00640 1 --2- 172.21.15.5:0/3238398898 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcf6411b140 0x7fcf641abb90 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fcf5c006010 tx=0x7fcf5c030f00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:50:38.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:38.683+0000 7fcf62000640 1 -- 172.21.15.5:0/3238398898 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf5c0318f0 con 0x7fcf6411b140 2024-09-10T09:50:38.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:38.683+0000 7fcf62000640 1 -- 172.21.15.5:0/3238398898 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcf5c046960 con 0x7fcf6411b140 2024-09-10T09:50:38.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:38.683+0000 7fcf62000640 1 -- 172.21.15.5:0/3238398898 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf5c03bb00 con 0x7fcf6411b140 2024-09-10T09:50:38.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:38.683+0000 7fcf6be00640 1 -- 172.21.15.5:0/3238398898 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf641ac500 con 0x7fcf6411b140 2024-09-10T09:50:38.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:38.683+0000 7fcf6be00640 1 -- 172.21.15.5:0/3238398898 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf641b3e40 con 0x7fcf6411b140 2024-09-10T09:50:38.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:38.696+0000 7fcf6ea75740 1 Processor -- start 2024-09-10T09:50:38.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:38.696+0000 7fcf62000640 1 -- 172.21.15.5:0/3238398898 <== mon.2 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7fcf5c03bca0 con 0x7fcf6411b140 2024-09-10T09:50:38.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:38.702+0000 7fcf62000640 1 --2- 172.21.15.5:0/3238398898 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fcf480a2c80 0x7fcf480a5140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:50:38.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:38.702+0000 7fcf62000640 1 -- 172.21.15.5:0/3238398898 <== mon.2 v2:172.21.15.102:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fcf5c0f09f0 con 0x7fcf6411b140 2024-09-10T09:50:38.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:38.702+0000 7fcf6ea75740 1 -- start start 2024-09-10T09:50:38.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:38.702+0000 7fcf6ea75740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556057898ac0 0x556057898ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:50:38.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:38.702+0000 7fcf6ea75740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5560578993e0 0x5560578a3760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:50:38.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:38.702+0000 7fcf6ea75740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5560578a3ca0 0x5560578a6090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:50:38.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:38.702+0000 7fcf6ea75740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5560577ff290 con 0x556057898ac0 2024-09-10T09:50:38.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:38.702+0000 7fcf6ea75740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x5560577a6b40 con 0x5560578993e0 2024-09-10T09:50:38.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:38.702+0000 7fcf6ea75740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5560577fb250 con 0x5560578a3ca0 2024-09-10T09:50:38.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:38.702+0000 7fcf68a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5560578993e0 0x5560578a3760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:50:38.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:38.702+0000 7fcf68a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5560578993e0 0x5560578a3760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:35880/0 (socket says 172.21.15.5:35880) 2024-09-10T09:50:38.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:38.702+0000 7fcf68a00640 1 -- 172.21.15.5:0/2862431327 learned_addr learned my addr 172.21.15.5:0/2862431327 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:50:38.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:38.702+0000 7fcf68a00640 1 --2- 172.21.15.5:0/3238398898 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fcf480a2c80 0x7fcf480a5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:50:38.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:38.709+0000 7fcf69e00640 1 --2- 172.21.15.5:0/2862431327 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5560578a3ca0 0x5560578a6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:50:38.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:38.709+0000 7fcf69400640 1 --2- 172.21.15.5:0/2862431327 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556057898ac0 0x556057898ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:50:38.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:38.710+0000 7fcf69e00640 1 -- 172.21.15.5:0/2862431327 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5560578993e0 msgr2=0x5560578a3760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:50:38.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:38.710+0000 7fcf69e00640 1 --2- 172.21.15.5:0/2862431327 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5560578993e0 0x5560578a3760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:38.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:38.710+0000 7fcf69e00640 1 -- 172.21.15.5:0/2862431327 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556057898ac0 msgr2=0x556057898ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:50:38.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:38.710+0000 7fcf69e00640 1 --2- 172.21.15.5:0/2862431327 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556057898ac0 0x556057898ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:38.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:38.710+0000 7fcf69e00640 1 -- 172.21.15.5:0/2862431327 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5560575706a0 con 0x5560578a3ca0 2024-09-10T09:50:38.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:38.710+0000 7fcf68a00640 1 --2- 172.21.15.5:0/3238398898 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fcf480a2c80 0x7fcf480a5140 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fcf5000cfd0 tx=0x7fcf5000c570 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:50:38.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:38.710+0000 7fcf69e00640 1 --2- 172.21.15.5:0/2862431327 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5560578a3ca0 0x5560578a6090 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fcf5c03b750 tx=0x7fcf5c03bf90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f2783e899ec63b18 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:50:38.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:38.710+0000 7fcf5a600640 1 -- 172.21.15.5:0/2862431327 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf5c0bce00 con 0x5560578a3ca0 2024-09-10T09:50:38.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:38.710+0000 7fcf5a600640 1 -- 172.21.15.5:0/2862431327 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcf5c036030 con 0x5560578a3ca0 2024-09-10T09:50:38.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:38.710+0000 7fcf5a600640 1 -- 172.21.15.5:0/2862431327 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf5c0b9470 con 0x5560578a3ca0 2024-09-10T09:50:38.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:38.710+0000 7fcf6ea75740 1 -- 172.21.15.5:0/2862431327 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5560578a3ca0 msgr2=0x5560578a6090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:50:38.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:38.710+0000 7fcf6ea75740 1 --2- 172.21.15.5:0/2862431327 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5560578a3ca0 0x5560578a6090 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fcf5c03b750 tx=0x7fcf5c03bf90 comp rx=0 tx=0).stop 2024-09-10T09:50:38.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:38.710+0000 7fcf6ea75740 1 -- 172.21.15.5:0/2862431327 shutdown_connections 2024-09-10T09:50:38.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:38.710+0000 7fcf6ea75740 1 --2- 172.21.15.5:0/2862431327 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5560578a3ca0 0x5560578a6090 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:38.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:38.710+0000 7fcf6ea75740 1 --2- 172.21.15.5:0/2862431327 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5560578993e0 0x5560578a3760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:38.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:38.710+0000 7fcf6ea75740 1 --2- 172.21.15.5:0/2862431327 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556057898ac0 0x556057898ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:38.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:38.710+0000 7fcf6ea75740 1 -- 172.21.15.5:0/2862431327 >> 172.21.15.5:0/2862431327 conn(0x556057895050 msgr2=0x556057895430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:50:38.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:38.716+0000 7fcf6ea75740 1 -- 172.21.15.5:0/2862431327 shutdown_connections 2024-09-10T09:50:38.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:38.716+0000 7fcf6ea75740 1 -- 172.21.15.5:0/2862431327 wait complete. 2024-09-10T09:50:38.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:38.716+0000 7fcf6ea75740 1 -- start start 2024-09-10T09:50:38.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:38.716+0000 7fcf6ea75740 1 Processor -- start 2024-09-10T09:50:38.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:38.716+0000 7fcf6ea75740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556057898ac0 0x5560578a36a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:50:38.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:38.716+0000 7fcf6ea75740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5560578993e0 0x5560578a0950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:50:38.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:38.716+0000 7fcf6ea75740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5560578a3ca0 0x5560578a0f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:50:38.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:38.716+0000 7fcf6ea75740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5560578a6630 con 0x556057898ac0 2024-09-10T09:50:38.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:38.716+0000 7fcf6ea75740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x5560577b9fa0 con 0x5560578a3ca0 2024-09-10T09:50:38.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:38.716+0000 7fcf6ea75740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5560578a67b0 con 0x5560578993e0 2024-09-10T09:50:38.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:38.716+0000 7fcf6ea75740 10 client.0 _ll_register_callbacks cb 0x55605750dba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:50:38.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:38.716+0000 7fcf68a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5560578993e0 0x5560578a0950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:50:38.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:38.716+0000 7fcf68a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5560578993e0 0x5560578a0950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:46626/0 (socket says 172.21.15.5:46626) 2024-09-10T09:50:38.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:38.716+0000 7fcf68a00640 1 -- 172.21.15.5:0/3886417685 learned_addr learned my addr 172.21.15.5:0/3886417685 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:50:38.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:38.716+0000 7fcf69e00640 1 --2- 172.21.15.5:0/3886417685 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5560578a3ca0 0x5560578a0f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:50:38.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:38.716+0000 7fcf69400640 1 --2- 172.21.15.5:0/3886417685 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556057898ac0 0x5560578a36a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:50:38.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:38.716+0000 7fcf68a00640 1 -- 172.21.15.5:0/3886417685 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5560578a3ca0 msgr2=0x5560578a0f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:50:38.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:38.716+0000 7fcf68a00640 1 --2- 172.21.15.5:0/3886417685 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5560578a3ca0 0x5560578a0f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:38.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:38.716+0000 7fcf68a00640 1 -- 172.21.15.5:0/3886417685 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556057898ac0 msgr2=0x5560578a36a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:50:38.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:38.721+0000 7fcf68a00640 1 --2- 172.21.15.5:0/3886417685 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556057898ac0 0x5560578a36a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:38.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:38.721+0000 7fcf68a00640 1 -- 172.21.15.5:0/3886417685 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5560578a1770 con 0x5560578993e0 2024-09-10T09:50:38.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:38.721+0000 7fcf69400640 1 --2- 172.21.15.5:0/3886417685 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556057898ac0 0x5560578a36a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T09:50:38.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:38.722+0000 7fcf68a00640 1 --2- 172.21.15.5:0/3886417685 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5560578993e0 0x5560578a0950 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fcf50004690 tx=0x7fcf5000c270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=28fc68e202a2a4ab server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:50:38.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:38.722+0000 7fcf4ea00640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-09-10T09:50:38.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:38.723+0000 7fcf4ea00640 1 -- 172.21.15.5:0/3886417685 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf50026070 con 0x5560578993e0 2024-09-10T09:50:38.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:38.723+0000 7fcf6ea75740 1 -- 172.21.15.5:0/3886417685 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcf5c036030 con 0x5560578993e0 2024-09-10T09:50:38.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:38.723+0000 7fcf4ea00640 1 -- 172.21.15.5:0/3886417685 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fcf50019070 con 0x5560578993e0 2024-09-10T09:50:38.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:38.723+0000 7fcf4ea00640 1 -- 172.21.15.5:0/3886417685 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fcf50026070 con 0x5560578993e0 2024-09-10T09:50:38.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:38.723+0000 7fcf6ea75740 10 client.27889 fetch_fsmap learned FSMap version 36 2024-09-10T09:50:38.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:38.723+0000 7fcf6ea75740 1 -- 172.21.15.5:0/3886417685 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x5560578c36b0 con 0x5560578993e0 2024-09-10T09:50:38.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:38.723+0000 7fcf4ea00640 1 -- 172.21.15.5:0/3886417685 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcf5c036030 con 0x5560578993e0 2024-09-10T09:50:38.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:38.723+0000 7fcf4ea00640 1 -- 172.21.15.5:0/3886417685 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 36) ==== 8370+0+0 (secure 0 0 0) 0x7fcf5001d040 con 0x5560578993e0 2024-09-10T09:50:38.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:38.723+0000 7fcf6ea75740 10 client.27889 fetch_fsmap finished waiting for FSMap version 36 2024-09-10T09:50:38.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:38.723+0000 7fcf6ea75740 10 client.27889 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T09:50:38.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:38.723+0000 7fcf6ea75740 20 client.27889 populate_metadata read hostname 'smithi005' 2024-09-10T09:50:38.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:38.723+0000 7fcf6ea75740 1 --2- 172.21.15.5:0/3886417685 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x5560578c4bb0 0x5560578c7030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:50:38.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:38.723+0000 7fcf6ea75740 4 client.27889 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:50:38.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:38.723+0000 7fcf6ea75740 1 -- 172.21.15.5:0/3886417685 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55605780c800 con 0x5560578c4bb0 2024-09-10T09:50:38.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:38.723+0000 7fcf69400640 1 --2- 172.21.15.5:0/3886417685 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x5560578c4bb0 0x5560578c7030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:50:38.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:38.723+0000 7fcf69400640 1 --2- 172.21.15.5:0/3886417685 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x5560578c4bb0 0x5560578c7030 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ea479a185c90d708 server_cookie=4d6490d3d52faf44 in_seq=0 out_seq=0 2024-09-10T09:50:38.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:38.723+0000 7fcf4ea00640 10 client.27889 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T09:50:38.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:38.729+0000 7fcf4ea00640 1 -- 172.21.15.5:0/3886417685 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55605780c800 con 0x5560578c4bb0 2024-09-10T09:50:38.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:38.729+0000 7fcf4ea00640 10 client.27889 handle_command_reply: tid=0 2024-09-10T09:50:38.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:38.729+0000 7fcf6ea75740 1 client.27889 shutdown 2024-09-10T09:50:38.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:38.729+0000 7fcf6ea75740 1 -- 172.21.15.5:0/3886417685 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5560578993e0 msgr2=0x5560578a0950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:50:38.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:38.729+0000 7fcf6ea75740 1 --2- 172.21.15.5:0/3886417685 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5560578993e0 0x5560578a0950 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fcf50004690 tx=0x7fcf5000c270 comp rx=0 tx=0).stop 2024-09-10T09:50:38.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:38.729+0000 7fcf6ea75740 1 -- 172.21.15.5:0/3886417685 shutdown_connections 2024-09-10T09:50:38.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:38.729+0000 7fcf6ea75740 1 --2- 172.21.15.5:0/3886417685 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x5560578c4bb0 0x5560578c7030 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:38.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:38.729+0000 7fcf6ea75740 1 --2- 172.21.15.5:0/3886417685 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5560578a3ca0 0x5560578a0f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:38.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:38.729+0000 7fcf6ea75740 1 --2- 172.21.15.5:0/3886417685 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5560578993e0 0x5560578a0950 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:38.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:38.729+0000 7fcf6ea75740 1 --2- 172.21.15.5:0/3886417685 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556057898ac0 0x5560578a36a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:38.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:38.729+0000 7fcf6ea75740 1 -- 172.21.15.5:0/3886417685 >> 172.21.15.5:0/3886417685 conn(0x556057895050 msgr2=0x55605789ad80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:50:38.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:38.729+0000 7fcf4ea00640 0 client.27889 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T09:50:38.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:38.729+0000 7fcf6ea75740 1 -- 172.21.15.5:0/3886417685 shutdown_connections 2024-09-10T09:50:38.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:38.729+0000 7fcf6ea75740 1 -- 172.21.15.5:0/3886417685 wait complete. 2024-09-10T09:50:38.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:38.729+0000 7fcf6ea75740 20 client.27889 trim_cache size 0 max 16384 2024-09-10T09:50:38.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:38.760+0000 7fcf6ea75740 1 Processor -- start 2024-09-10T09:50:38.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:38.760+0000 7fcf6ea75740 1 -- start start 2024-09-10T09:50:38.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:38.765+0000 7fcf6ea75740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556057898ac0 0x55605789ac70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:50:38.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:38.765+0000 7fcf6ea75740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5560578993e0 0x5560578cc900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:50:38.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:38.765+0000 7fcf6ea75740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5560578a3ca0 0x5560578cce40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:50:38.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:38.765+0000 7fcf6ea75740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fcf4400fdd0 con 0x556057898ac0 2024-09-10T09:50:38.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:38.765+0000 7fcf6ea75740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fcf44011910 con 0x5560578993e0 2024-09-10T09:50:38.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:38.765+0000 7fcf6ea75740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fcf440104a0 con 0x5560578a3ca0 2024-09-10T09:50:38.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:38.765+0000 7fcf69400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556057898ac0 0x55605789ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:50:38.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:38.765+0000 7fcf68a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5560578993e0 0x5560578cc900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:50:38.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:38.765+0000 7fcf68a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5560578993e0 0x5560578cc900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:35894/0 (socket says 172.21.15.5:35894) 2024-09-10T09:50:38.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:38.765+0000 7fcf68a00640 1 -- 172.21.15.5:0/3281540372 learned_addr learned my addr 172.21.15.5:0/3281540372 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:50:38.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:38.765+0000 7fcf68a00640 1 -- 172.21.15.5:0/3281540372 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5560578a3ca0 msgr2=0x5560578cce40 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T09:50:38.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:38.765+0000 7fcf68a00640 1 --2- 172.21.15.5:0/3281540372 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5560578a3ca0 0x5560578cce40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:38.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:38.765+0000 7fcf68a00640 1 -- 172.21.15.5:0/3281540372 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556057898ac0 msgr2=0x55605789ac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:50:38.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:38.765+0000 7fcf68a00640 1 --2- 172.21.15.5:0/3281540372 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556057898ac0 0x55605789ac70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:38.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:38.765+0000 7fcf68a00640 1 -- 172.21.15.5:0/3281540372 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55605789eea0 con 0x5560578993e0 2024-09-10T09:50:38.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:38.765+0000 7fcf68a00640 1 --2- 172.21.15.5:0/3281540372 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5560578993e0 0x5560578cc900 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fcf50004b30 tx=0x7fcf500046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a9bd88088d686742 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:50:38.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:38.765+0000 7fcf59c00640 1 -- 172.21.15.5:0/3281540372 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf500047e0 con 0x5560578993e0 2024-09-10T09:50:38.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:38.765+0000 7fcf59c00640 1 -- 172.21.15.5:0/3281540372 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcf50004d20 con 0x5560578993e0 2024-09-10T09:50:38.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:38.765+0000 7fcf59c00640 1 -- 172.21.15.5:0/3281540372 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf500051f0 con 0x5560578993e0 2024-09-10T09:50:38.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:38.765+0000 7fcf6ea75740 1 -- 172.21.15.5:0/3281540372 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5560578993e0 msgr2=0x5560578cc900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:50:38.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:38.765+0000 7fcf6ea75740 1 --2- 172.21.15.5:0/3281540372 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5560578993e0 0x5560578cc900 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fcf50004b30 tx=0x7fcf500046c0 comp rx=0 tx=0).stop 2024-09-10T09:50:38.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:38.765+0000 7fcf6ea75740 1 -- 172.21.15.5:0/3281540372 shutdown_connections 2024-09-10T09:50:38.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:38.765+0000 7fcf6ea75740 1 --2- 172.21.15.5:0/3281540372 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5560578a3ca0 0x5560578cce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:38.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:38.765+0000 7fcf6ea75740 1 --2- 172.21.15.5:0/3281540372 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5560578993e0 0x5560578cc900 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:38.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:38.765+0000 7fcf6ea75740 1 --2- 172.21.15.5:0/3281540372 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556057898ac0 0x55605789ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:38.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:38.765+0000 7fcf6ea75740 1 -- 172.21.15.5:0/3281540372 >> 172.21.15.5:0/3281540372 conn(0x556057895050 msgr2=0x55605789a560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:50:38.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:38.765+0000 7fcf6ea75740 1 -- 172.21.15.5:0/3281540372 shutdown_connections 2024-09-10T09:50:38.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:38.765+0000 7fcf6ea75740 1 -- 172.21.15.5:0/3281540372 wait complete. 2024-09-10T09:50:38.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:38.765+0000 7fcf6ea75740 1 -- start start 2024-09-10T09:50:38.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:38.765+0000 7fcf6ea75740 1 Processor -- start 2024-09-10T09:50:38.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:38.770+0000 7fcf6ea75740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556057898ac0 0x5560578abe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:50:38.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:38.770+0000 7fcf6ea75740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5560578993e0 0x5560578ac360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:50:38.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:38.770+0000 7fcf6ea75740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5560578a3ca0 0x5560578a83b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:50:38.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:38.770+0000 7fcf6ea75740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fcf44011240 con 0x5560578a3ca0 2024-09-10T09:50:38.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:38.770+0000 7fcf6ea75740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fcf44010b70 con 0x5560578993e0 2024-09-10T09:50:38.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:38.770+0000 7fcf6ea75740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fcf4400ddb0 con 0x556057898ac0 2024-09-10T09:50:38.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:38.770+0000 7fcf6ea75740 10 client.0 _ll_register_callbacks cb 0x7fcf5000c8e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:50:38.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:38.770+0000 7fcf68a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5560578993e0 0x5560578ac360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:50:38.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:38.770+0000 7fcf68a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5560578993e0 0x5560578ac360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:35906/0 (socket says 172.21.15.5:35906) 2024-09-10T09:50:38.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:38.770+0000 7fcf68a00640 1 -- 172.21.15.5:0/3371245573 learned_addr learned my addr 172.21.15.5:0/3371245573 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:50:38.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:38.770+0000 7fcf69e00640 1 --2- 172.21.15.5:0/3371245573 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5560578a3ca0 0x5560578a83b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:50:38.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:38.770+0000 7fcf68a00640 1 -- 172.21.15.5:0/3371245573 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556057898ac0 msgr2=0x5560578abe20 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T09:50:38.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:38.770+0000 7fcf68a00640 1 --2- 172.21.15.5:0/3371245573 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556057898ac0 0x5560578abe20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:38.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:38.770+0000 7fcf68a00640 1 -- 172.21.15.5:0/3371245573 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5560578a3ca0 msgr2=0x5560578a83b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:50:38.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:38.770+0000 7fcf68a00640 1 --2- 172.21.15.5:0/3371245573 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5560578a3ca0 0x5560578a83b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:38.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:38.770+0000 7fcf68a00640 1 -- 172.21.15.5:0/3371245573 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5560578a8be0 con 0x5560578993e0 2024-09-10T09:50:38.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:38.772+0000 7fcf68a00640 1 --2- 172.21.15.5:0/3371245573 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5560578993e0 0x5560578ac360 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fcf500136d0 tx=0x7fcf5000c990 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b5627b83d3af0767 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:50:38.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:38.772+0000 7fcf4ea00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-09-10T09:50:38.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:38.772+0000 7fcf4ea00640 1 -- 172.21.15.5:0/3371245573 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf500047e0 con 0x5560578993e0 2024-09-10T09:50:38.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:38.772+0000 7fcf4ea00640 1 -- 172.21.15.5:0/3371245573 <== mon.2 v2:172.21.15.102:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fcf500051f0 con 0x5560578993e0 2024-09-10T09:50:38.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:38.772+0000 7fcf6ea75740 1 -- 172.21.15.5:0/3371245573 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcf50004d20 con 0x5560578993e0 2024-09-10T09:50:38.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:38.772+0000 7fcf4ea00640 1 -- 172.21.15.5:0/3371245573 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fcf50047050 con 0x5560578993e0 2024-09-10T09:50:38.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:38.772+0000 7fcf6ea75740 10 client.27893 fetch_fsmap learned FSMap version 36 2024-09-10T09:50:38.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:38.772+0000 7fcf6ea75740 1 -- 172.21.15.5:0/3371245573 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x5560579305d0 con 0x5560578993e0 2024-09-10T09:50:38.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:38.772+0000 7fcf4ea00640 1 -- 172.21.15.5:0/3371245573 <== mon.2 v2:172.21.15.102:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcf50004d20 con 0x5560578993e0 2024-09-10T09:50:38.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:38.772+0000 7fcf4ea00640 1 -- 172.21.15.5:0/3371245573 <== mon.2 v2:172.21.15.102:3300/0 5 ==== fsmap(e 36) ==== 8370+0+0 (secure 0 0 0) 0x7fcf5001d040 con 0x5560578993e0 2024-09-10T09:50:38.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:38.772+0000 7fcf6ea75740 10 client.27893 fetch_fsmap finished waiting for FSMap version 36 2024-09-10T09:50:38.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:38.772+0000 7fcf6ea75740 10 client.27893 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T09:50:38.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:38.772+0000 7fcf6ea75740 20 client.27893 populate_metadata read hostname 'smithi005' 2024-09-10T09:50:38.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:38.772+0000 7fcf6ea75740 1 --2- 172.21.15.5:0/3371245573 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x556057930cc0 0x5560579330e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:50:38.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:38.772+0000 7fcf6ea75740 4 client.27893 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-10T09:50:38.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:38.772+0000 7fcf6ea75740 1 -- 172.21.15.5:0/3371245573 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x556057806d10 con 0x556057930cc0 2024-09-10T09:50:38.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:38.772+0000 7fcf69400640 1 --2- 172.21.15.5:0/3371245573 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x556057930cc0 0x5560579330e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:50:38.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:38.772+0000 7fcf69400640 1 --2- 172.21.15.5:0/3371245573 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x556057930cc0 0x5560579330e0 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8eb5adbd386e4d8f server_cookie=af03084b24218763 in_seq=0 out_seq=0 2024-09-10T09:50:38.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:38.772+0000 7fcf4ea00640 10 client.27893 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T09:50:38.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:38.772+0000 7fcf4ea00640 1 -- 172.21.15.5:0/3371245573 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x556057806d10 con 0x556057930cc0 2024-09-10T09:50:38.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:38.772+0000 7fcf4ea00640 10 client.27893 handle_command_reply: tid=0 2024-09-10T09:50:38.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:38.772+0000 7fcf6ea75740 1 client.27893 shutdown 2024-09-10T09:50:38.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:38.772+0000 7fcf6ea75740 1 -- 172.21.15.5:0/3371245573 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5560578993e0 msgr2=0x5560578ac360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:50:38.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:38.772+0000 7fcf6ea75740 1 --2- 172.21.15.5:0/3371245573 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5560578993e0 0x5560578ac360 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fcf500136d0 tx=0x7fcf5000c990 comp rx=0 tx=0).stop 2024-09-10T09:50:38.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:38.772+0000 7fcf6ea75740 1 -- 172.21.15.5:0/3371245573 shutdown_connections 2024-09-10T09:50:38.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:38.772+0000 7fcf6ea75740 1 --2- 172.21.15.5:0/3371245573 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x556057930cc0 0x5560579330e0 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:38.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:38.772+0000 7fcf6ea75740 1 --2- 172.21.15.5:0/3371245573 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5560578a3ca0 0x5560578a83b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:38.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:38.772+0000 7fcf6ea75740 1 --2- 172.21.15.5:0/3371245573 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5560578993e0 0x5560578ac360 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:38.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:38.772+0000 7fcf6ea75740 1 --2- 172.21.15.5:0/3371245573 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556057898ac0 0x5560578abe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:38.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:38.772+0000 7fcf6ea75740 1 -- 172.21.15.5:0/3371245573 >> 172.21.15.5:0/3371245573 conn(0x556057895050 msgr2=0x5560578941c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:50:38.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:38.772+0000 7fcf4ea00640 0 client.27893 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T09:50:38.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:38.772+0000 7fcf6ea75740 1 -- 172.21.15.5:0/3371245573 shutdown_connections 2024-09-10T09:50:38.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:38.772+0000 7fcf6ea75740 1 -- 172.21.15.5:0/3371245573 wait complete. 2024-09-10T09:50:38.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:38.772+0000 7fcf6ea75740 20 client.27893 trim_cache size 0 max 16384 2024-09-10T09:50:38.779 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-10T09:50:38.779 INFO:teuthology.orchestra.run.smithi005.stdout: "return_code": 0, 2024-09-10T09:50:38.779 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_tag": "a7c127b0-5849-4704-b626-c076d9f90aed", 2024-09-10T09:50:38.779 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": "asynchronous" 2024-09-10T09:50:38.779 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-10T09:50:38.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:38.778+0000 7fcf6be00640 1 -- 172.21.15.5:0/3238398898 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fcf480a2c80 msgr2=0x7fcf480a5140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:50:38.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:38.778+0000 7fcf6be00640 1 --2- 172.21.15.5:0/3238398898 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fcf480a2c80 0x7fcf480a5140 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fcf5000cfd0 tx=0x7fcf5000c570 comp rx=0 tx=0).stop 2024-09-10T09:50:38.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:38.778+0000 7fcf6be00640 1 -- 172.21.15.5:0/3238398898 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcf6411b140 msgr2=0x7fcf641abb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:50:38.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:38.778+0000 7fcf6be00640 1 --2- 172.21.15.5:0/3238398898 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcf6411b140 0x7fcf641abb90 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fcf5c006010 tx=0x7fcf5c030f00 comp rx=0 tx=0).stop 2024-09-10T09:50:38.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:38.778+0000 7fcf6be00640 1 -- 172.21.15.5:0/3238398898 shutdown_connections 2024-09-10T09:50:38.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:38.778+0000 7fcf6be00640 1 --2- 172.21.15.5:0/3238398898 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fcf480a2c80 0x7fcf480a5140 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:38.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:38.778+0000 7fcf6be00640 1 --2- 172.21.15.5:0/3238398898 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcf6411b140 0x7fcf641abb90 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:38.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:38.778+0000 7fcf6be00640 1 --2- 172.21.15.5:0/3238398898 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcf64109830 0x7fcf641a7800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:38.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:38.778+0000 7fcf6be00640 1 --2- 172.21.15.5:0/3238398898 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcf64109450 0x7fcf641a72c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:38.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:38.778+0000 7fcf6be00640 1 -- 172.21.15.5:0/3238398898 >> 172.21.15.5:0/3238398898 conn(0x7fcf6406e140 msgr2=0x7fcf6411bd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:50:38.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:38.778+0000 7fcf6be00640 1 -- 172.21.15.5:0/3238398898 shutdown_connections 2024-09-10T09:50:38.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:38.778+0000 7fcf6be00640 1 -- 172.21.15.5:0/3238398898 wait complete. 2024-09-10T09:50:39.135 INFO:tasks.workunit.client.0.smithi005.stdout: 1 785836 6.43 MB/sec execute 500 sec latency 1381.679 ms 2024-09-10T09:50:39.495 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '6b75eb2', quiesce took 6.9 seconds. Will release after: 11.200000000000001 2024-09-10T09:50:39.886 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:50:39 smithi005 ceph-mon[30350]: pgmap v762: 129 pgs: 129 active+clean; 291 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.5 MiB/s wr, 206 op/s 2024-09-10T09:50:39.886 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:50:39 smithi005 ceph-mon[30350]: scrub queued for path: / 2024-09-10T09:50:39.886 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:50:39 smithi005 ceph-mon[30350]: scrub summary: idle+waiting paths [/] 2024-09-10T09:50:39.886 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:50:39 smithi005 ceph-mon[30350]: scrub summary: active paths [/] 2024-09-10T09:50:39.887 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:50:39 smithi005 ceph-mon[30350]: scrub summary: active paths [/] 2024-09-10T09:50:39.887 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:50:39 smithi005 ceph-mon[30350]: scrub summary: idle+waiting paths [/] 2024-09-10T09:50:39.887 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:50:39 smithi005 ceph-mon[30350]: scrub summary: idle+waiting paths [/] 2024-09-10T09:50:40.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:50:39 smithi159 ceph-mon[35513]: pgmap v762: 129 pgs: 129 active+clean; 291 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.5 MiB/s wr, 206 op/s 2024-09-10T09:50:40.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:50:39 smithi159 ceph-mon[35513]: scrub queued for path: / 2024-09-10T09:50:40.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:50:39 smithi159 ceph-mon[35513]: scrub summary: idle+waiting paths [/] 2024-09-10T09:50:40.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:50:39 smithi159 ceph-mon[35513]: scrub summary: active paths [/] 2024-09-10T09:50:40.101 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:50:39 smithi159 ceph-mon[35513]: scrub summary: active paths [/] 2024-09-10T09:50:40.101 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:50:39 smithi159 ceph-mon[35513]: scrub summary: idle+waiting paths [/] 2024-09-10T09:50:40.101 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:50:39 smithi159 ceph-mon[35513]: scrub summary: idle+waiting paths [/] 2024-09-10T09:50:40.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:50:39 smithi102 ceph-mon[35442]: pgmap v762: 129 pgs: 129 active+clean; 291 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.5 MiB/s wr, 206 op/s 2024-09-10T09:50:40.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:50:39 smithi102 ceph-mon[35442]: scrub queued for path: / 2024-09-10T09:50:40.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:50:39 smithi102 ceph-mon[35442]: scrub summary: idle+waiting paths [/] 2024-09-10T09:50:40.135 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:50:39 smithi102 ceph-mon[35442]: scrub summary: active paths [/] 2024-09-10T09:50:40.135 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:50:39 smithi102 ceph-mon[35442]: scrub summary: active paths [/] 2024-09-10T09:50:40.135 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:50:39 smithi102 ceph-mon[35442]: scrub summary: idle+waiting paths [/] 2024-09-10T09:50:40.135 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:50:39 smithi102 ceph-mon[35442]: scrub summary: idle+waiting paths [/] 2024-09-10T09:50:40.135 INFO:tasks.workunit.client.0.smithi005.stdout: 1 785836 6.42 MB/sec execute 501 sec latency 2381.729 ms 2024-09-10T09:50:40.167 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T09:50:40.549 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:50:41.136 INFO:tasks.workunit.client.0.smithi005.stdout: 1 785836 6.41 MB/sec execute 502 sec latency 3381.802 ms 2024-09-10T09:50:41.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:41.165+0000 7fbb5ac00640 1 -- 172.21.15.5:0/4159049391 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbb54109120 msgr2=0x7fbb54110ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:50:41.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:41.165+0000 7fbb5ac00640 1 --2- 172.21.15.5:0/4159049391 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbb54109120 0x7fbb54110ca0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fbb4c009ca0 tx=0x7fbb4c02a8c0 comp rx=0 tx=0).stop 2024-09-10T09:50:41.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:41.165+0000 7fbb5ac00640 1 -- 172.21.15.5:0/4159049391 shutdown_connections 2024-09-10T09:50:41.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:41.165+0000 7fbb5ac00640 1 --2- 172.21.15.5:0/4159049391 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbb54109120 0x7fbb54110ca0 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:41.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:41.165+0000 7fbb5ac00640 1 --2- 172.21.15.5:0/4159049391 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbb541049a0 0x7fbb541089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:41.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:41.165+0000 7fbb5ac00640 1 --2- 172.21.15.5:0/4159049391 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbb54103ff0 0x7fbb541043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:41.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:41.165+0000 7fbb5ac00640 1 -- 172.21.15.5:0/4159049391 >> 172.21.15.5:0/4159049391 conn(0x7fbb540ffa90 msgr2=0x7fbb54101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:50:41.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:41.165+0000 7fbb5ac00640 1 -- 172.21.15.5:0/4159049391 shutdown_connections 2024-09-10T09:50:41.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:41.165+0000 7fbb5ac00640 1 -- 172.21.15.5:0/4159049391 wait complete. 2024-09-10T09:50:41.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:41.167+0000 7fbb5ac00640 1 Processor -- start 2024-09-10T09:50:41.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:41.167+0000 7fbb5ac00640 1 -- start start 2024-09-10T09:50:41.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:41.167+0000 7fbb5ac00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbb54103ff0 0x7fbb5419a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:50:41.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:41.167+0000 7fbb5ac00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbb541049a0 0x7fbb5419ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:50:41.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:41.167+0000 7fbb5ac00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbb54109120 0x7fbb5419ee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:50:41.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:41.167+0000 7fbb5ac00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fbb541141a0 con 0x7fbb54103ff0 2024-09-10T09:50:41.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:41.167+0000 7fbb5ac00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fbb54114020 con 0x7fbb54109120 2024-09-10T09:50:41.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:41.167+0000 7fbb5ac00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fbb54114320 con 0x7fbb541049a0 2024-09-10T09:50:41.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:41.167+0000 7fbb53e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbb54103ff0 0x7fbb5419a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:50:41.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:41.168+0000 7fbb53e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbb54103ff0 0x7fbb5419a5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52234/0 (socket says 172.21.15.5:52234) 2024-09-10T09:50:41.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:41.168+0000 7fbb53e00640 1 -- 172.21.15.5:0/95236319 learned_addr learned my addr 172.21.15.5:0/95236319 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:50:41.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:41.168+0000 7fbb4be00640 1 --2- 172.21.15.5:0/95236319 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbb541049a0 0x7fbb5419ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:50:41.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:41.168+0000 7fbb58c00640 1 --2- 172.21.15.5:0/95236319 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbb54109120 0x7fbb5419ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:50:41.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:41.168+0000 7fbb53e00640 1 -- 172.21.15.5:0/95236319 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbb541049a0 msgr2=0x7fbb5419ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:50:41.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:41.168+0000 7fbb53e00640 1 --2- 172.21.15.5:0/95236319 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbb541049a0 0x7fbb5419ab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:41.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:41.168+0000 7fbb53e00640 1 -- 172.21.15.5:0/95236319 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbb54109120 msgr2=0x7fbb5419ee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:50:41.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:41.168+0000 7fbb53e00640 1 --2- 172.21.15.5:0/95236319 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbb54109120 0x7fbb5419ee90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:41.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:41.168+0000 7fbb53e00640 1 -- 172.21.15.5:0/95236319 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb5419f570 con 0x7fbb54103ff0 2024-09-10T09:50:41.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:41.168+0000 7fbb53e00640 1 --2- 172.21.15.5:0/95236319 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbb54103ff0 0x7fbb5419a5c0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fbb4400e9f0 tx=0x7fbb4400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:50:41.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:41.168+0000 7fbb51600640 1 -- 172.21.15.5:0/95236319 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb4400cd80 con 0x7fbb54103ff0 2024-09-10T09:50:41.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:41.169+0000 7fbb51600640 1 -- 172.21.15.5:0/95236319 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbb44005500 con 0x7fbb54103ff0 2024-09-10T09:50:41.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:41.169+0000 7fbb5ac00640 1 -- 172.21.15.5:0/95236319 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb5419f860 con 0x7fbb54103ff0 2024-09-10T09:50:41.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:41.169+0000 7fbb51600640 1 -- 172.21.15.5:0/95236319 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb44010690 con 0x7fbb54103ff0 2024-09-10T09:50:41.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:41.169+0000 7fbb5ac00640 1 -- 172.21.15.5:0/95236319 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb541a7140 con 0x7fbb54103ff0 2024-09-10T09:50:41.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:41.172+0000 7fbb5d691740 1 Processor -- start 2024-09-10T09:50:41.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:41.172+0000 7fbb51600640 1 -- 172.21.15.5:0/95236319 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7fbb440046c0 con 0x7fbb54103ff0 2024-09-10T09:50:41.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:41.172+0000 7fbb5d691740 1 -- start start 2024-09-10T09:50:41.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:41.172+0000 7fbb5d691740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b02f9d6b90 0x55b02f9d6f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:50:41.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:41.172+0000 7fbb5d691740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b02f9d74b0 0x55b02f9e1830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:50:41.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:41.172+0000 7fbb5d691740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b02f9e1d70 0x55b02f9e4160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:50:41.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:41.172+0000 7fbb5d691740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55b02f93d3e0 con 0x55b02f9d74b0 2024-09-10T09:50:41.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:41.172+0000 7fbb5d691740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55b02f8e4d80 con 0x55b02f9d6b90 2024-09-10T09:50:41.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:41.172+0000 7fbb5d691740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55b02f9393a0 con 0x55b02f9e1d70 2024-09-10T09:50:41.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:41.172+0000 7fbb53e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b02f9d74b0 0x55b02f9e1830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:50:41.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:41.173+0000 7fbb58c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b02f9d6b90 0x55b02f9d6f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:50:41.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:41.173+0000 7fbb53e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b02f9d74b0 0x55b02f9e1830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52248/0 (socket says 172.21.15.5:52248) 2024-09-10T09:50:41.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:41.173+0000 7fbb53e00640 1 -- 172.21.15.5:0/3176807277 learned_addr learned my addr 172.21.15.5:0/3176807277 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:50:41.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:41.173+0000 7fbb4be00640 1 --2- 172.21.15.5:0/3176807277 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b02f9e1d70 0x55b02f9e4160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:50:41.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:41.173+0000 7fbb53e00640 1 -- 172.21.15.5:0/3176807277 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b02f9e1d70 msgr2=0x55b02f9e4160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:50:41.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:41.173+0000 7fbb53e00640 1 --2- 172.21.15.5:0/3176807277 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b02f9e1d70 0x55b02f9e4160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:41.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:41.173+0000 7fbb53e00640 1 -- 172.21.15.5:0/3176807277 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b02f9d6b90 msgr2=0x55b02f9d6f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:50:41.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:41.173+0000 7fbb53e00640 1 --2- 172.21.15.5:0/3176807277 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b02f9d6b90 0x55b02f9d6f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:41.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:41.173+0000 7fbb53e00640 1 -- 172.21.15.5:0/3176807277 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b02f903ae0 con 0x55b02f9d74b0 2024-09-10T09:50:41.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:41.173+0000 7fbb4be00640 1 --2- 172.21.15.5:0/3176807277 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b02f9e1d70 0x55b02f9e4160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:50:41.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:41.173+0000 7fbb53e00640 1 --2- 172.21.15.5:0/3176807277 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b02f9d74b0 0x55b02f9e1830 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fbb440d5a90 tx=0x7fbb440d5b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1266dd95f52023e3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:50:41.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:41.174+0000 7fbb51600640 1 --2- 172.21.15.5:0/95236319 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fbb380a2c00 0x7fbb380a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:50:41.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:41.174+0000 7fbb51600640 1 -- 172.21.15.5:0/95236319 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fbb440cf8c0 con 0x7fbb54103ff0 2024-09-10T09:50:41.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:41.174+0000 7fbb58c00640 1 --2- 172.21.15.5:0/95236319 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fbb380a2c00 0x7fbb380a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:50:41.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:41.174+0000 7fbb49200640 1 -- 172.21.15.5:0/3176807277 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb440de070 con 0x55b02f9d74b0 2024-09-10T09:50:41.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:41.174+0000 7fbb49200640 1 -- 172.21.15.5:0/3176807277 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbb4409c3d0 con 0x55b02f9d74b0 2024-09-10T09:50:41.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:41.174+0000 7fbb49200640 1 -- 172.21.15.5:0/3176807277 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb4409b810 con 0x55b02f9d74b0 2024-09-10T09:50:41.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:41.175+0000 7fbb5d691740 1 -- 172.21.15.5:0/3176807277 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b02f9d74b0 msgr2=0x55b02f9e1830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:50:41.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:41.175+0000 7fbb5d691740 1 --2- 172.21.15.5:0/3176807277 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b02f9d74b0 0x55b02f9e1830 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fbb440d5a90 tx=0x7fbb440d5b80 comp rx=0 tx=0).stop 2024-09-10T09:50:41.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:41.175+0000 7fbb58c00640 1 --2- 172.21.15.5:0/95236319 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fbb380a2c00 0x7fbb380a50c0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fbb4c02f540 tx=0x7fbb4c02f3b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:50:41.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:41.175+0000 7fbb5d691740 1 -- 172.21.15.5:0/3176807277 shutdown_connections 2024-09-10T09:50:41.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:41.175+0000 7fbb5d691740 1 --2- 172.21.15.5:0/3176807277 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b02f9e1d70 0x55b02f9e4160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:41.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:41.175+0000 7fbb5d691740 1 --2- 172.21.15.5:0/3176807277 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b02f9d74b0 0x55b02f9e1830 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:41.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:41.175+0000 7fbb5d691740 1 --2- 172.21.15.5:0/3176807277 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b02f9d6b90 0x55b02f9d6f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:41.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:41.175+0000 7fbb5d691740 1 -- 172.21.15.5:0/3176807277 >> 172.21.15.5:0/3176807277 conn(0x55b02f9d3120 msgr2=0x55b02f9d3500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:50:41.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:41.175+0000 7fbb5d691740 1 -- 172.21.15.5:0/3176807277 shutdown_connections 2024-09-10T09:50:41.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:41.175+0000 7fbb5d691740 1 -- 172.21.15.5:0/3176807277 wait complete. 2024-09-10T09:50:41.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:41.175+0000 7fbb5d691740 1 -- start start 2024-09-10T09:50:41.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:41.175+0000 7fbb5d691740 1 Processor -- start 2024-09-10T09:50:41.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:41.176+0000 7fbb5d691740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b02f9d6b90 0x55b02f9dec20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:50:41.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:41.176+0000 7fbb5d691740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b02f9d74b0 0x55b02f9df160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:50:41.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:41.176+0000 7fbb5d691740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b02f9e1d70 0x55b02f9df6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:50:41.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:41.176+0000 7fbb5d691740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55b02f9e49e0 con 0x55b02f9e1d70 2024-09-10T09:50:41.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:41.176+0000 7fbb5d691740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55b02f8f80f0 con 0x55b02f9d74b0 2024-09-10T09:50:41.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:41.176+0000 7fbb5d691740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55b02f9e4b60 con 0x55b02f9d6b90 2024-09-10T09:50:41.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:41.176+0000 7fbb5d691740 10 client.0 _ll_register_callbacks cb 0x55b02f8e99e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:50:41.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:41.176+0000 7fbb58c00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b02f9e1d70 0x55b02f9df6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:50:41.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:41.176+0000 7fbb4be00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b02f9d74b0 0x55b02f9df160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:50:41.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:41.176+0000 7fbb53e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b02f9d6b90 0x55b02f9dec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:50:41.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:41.176+0000 7fbb58c00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b02f9e1d70 0x55b02f9df6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52254/0 (socket says 172.21.15.5:52254) 2024-09-10T09:50:41.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:41.176+0000 7fbb58c00640 1 -- 172.21.15.5:0/2194400375 learned_addr learned my addr 172.21.15.5:0/2194400375 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:50:41.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:41.177+0000 7fbb4be00640 1 -- 172.21.15.5:0/2194400375 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b02f9d6b90 msgr2=0x55b02f9dec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:50:41.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:41.177+0000 7fbb4be00640 1 --2- 172.21.15.5:0/2194400375 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b02f9d6b90 0x55b02f9dec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:41.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:41.177+0000 7fbb4be00640 1 -- 172.21.15.5:0/2194400375 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b02f9e1d70 msgr2=0x55b02f9df6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:50:41.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:41.177+0000 7fbb4be00640 1 --2- 172.21.15.5:0/2194400375 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b02f9e1d70 0x55b02f9df6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:41.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:41.177+0000 7fbb4be00640 1 -- 172.21.15.5:0/2194400375 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b02f6ae3c0 con 0x55b02f9d74b0 2024-09-10T09:50:41.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:41.177+0000 7fbb53e00640 1 --2- 172.21.15.5:0/2194400375 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b02f9d6b90 0x55b02f9dec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:50:41.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:41.177+0000 7fbb4be00640 1 --2- 172.21.15.5:0/2194400375 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b02f9d74b0 0x55b02f9df160 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fbb3c005fd0 tx=0x7fbb3c00bf90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ca27cc14ccf2166 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:50:41.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:41.177+0000 7fbb41600640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-09-10T09:50:41.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:41.177+0000 7fbb41600640 1 -- 172.21.15.5:0/2194400375 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb3c004020 con 0x55b02f9d74b0 2024-09-10T09:50:41.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:41.177+0000 7fbb5d691740 1 -- 172.21.15.5:0/2194400375 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbb4409c3d0 con 0x55b02f9d74b0 2024-09-10T09:50:41.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:41.177+0000 7fbb41600640 1 -- 172.21.15.5:0/2194400375 <== mon.2 v2:172.21.15.102:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fbb3c0041c0 con 0x55b02f9d74b0 2024-09-10T09:50:41.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:41.178+0000 7fbb41600640 1 -- 172.21.15.5:0/2194400375 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fbb3c0043c0 con 0x55b02f9d74b0 2024-09-10T09:50:41.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:41.178+0000 7fbb5d691740 10 client.27905 fetch_fsmap learned FSMap version 36 2024-09-10T09:50:41.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:41.178+0000 7fbb5d691740 1 -- 172.21.15.5:0/2194400375 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55b02f9f06b0 con 0x55b02f9d74b0 2024-09-10T09:50:41.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:41.178+0000 7fbb41600640 1 -- 172.21.15.5:0/2194400375 <== mon.2 v2:172.21.15.102:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbb4409c3d0 con 0x55b02f9d74b0 2024-09-10T09:50:41.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:41.178+0000 7fbb41600640 1 -- 172.21.15.5:0/2194400375 <== mon.2 v2:172.21.15.102:3300/0 5 ==== fsmap(e 36) ==== 8370+0+0 (secure 0 0 0) 0x7fbb3c010700 con 0x55b02f9d74b0 2024-09-10T09:50:41.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:41.178+0000 7fbb5d691740 10 client.27905 fetch_fsmap finished waiting for FSMap version 36 2024-09-10T09:50:41.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:41.178+0000 7fbb5d691740 10 client.27905 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T09:50:41.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:41.178+0000 7fbb5d691740 20 client.27905 populate_metadata read hostname 'smithi005' 2024-09-10T09:50:41.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:41.178+0000 7fbb5d691740 1 --2- 172.21.15.5:0/2194400375 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55b02f9f1b60 0x55b02f9f3f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:50:41.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:41.178+0000 7fbb5d691740 4 client.27905 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:50:41.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:41.178+0000 7fbb5d691740 1 -- 172.21.15.5:0/2194400375 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b02f94a950 con 0x55b02f9f1b60 2024-09-10T09:50:41.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:41.179+0000 7fbb53e00640 1 --2- 172.21.15.5:0/2194400375 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55b02f9f1b60 0x55b02f9f3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:50:41.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:41.179+0000 7fbb53e00640 1 --2- 172.21.15.5:0/2194400375 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55b02f9f1b60 0x55b02f9f3f80 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=649dc350508e1493 server_cookie=f7dd87d6828fd7e3 in_seq=0 out_seq=0 2024-09-10T09:50:41.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:41.179+0000 7fbb41600640 10 client.27905 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T09:50:41.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:41.180+0000 7fbb41600640 1 -- 172.21.15.5:0/2194400375 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b02f94a950 con 0x55b02f9f1b60 2024-09-10T09:50:41.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:41.180+0000 7fbb41600640 10 client.27905 handle_command_reply: tid=0 2024-09-10T09:50:41.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:41.180+0000 7fbb5d691740 1 client.27905 shutdown 2024-09-10T09:50:41.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:41.180+0000 7fbb5d691740 1 -- 172.21.15.5:0/2194400375 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b02f9d74b0 msgr2=0x55b02f9df160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:50:41.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:41.180+0000 7fbb5d691740 1 --2- 172.21.15.5:0/2194400375 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b02f9d74b0 0x55b02f9df160 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fbb3c005fd0 tx=0x7fbb3c00bf90 comp rx=0 tx=0).stop 2024-09-10T09:50:41.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:41.180+0000 7fbb5d691740 1 -- 172.21.15.5:0/2194400375 shutdown_connections 2024-09-10T09:50:41.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:41.180+0000 7fbb5d691740 1 --2- 172.21.15.5:0/2194400375 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55b02f9f1b60 0x55b02f9f3f80 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:41.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:41.180+0000 7fbb5d691740 1 --2- 172.21.15.5:0/2194400375 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b02f9e1d70 0x55b02f9df6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:41.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:41.180+0000 7fbb5d691740 1 --2- 172.21.15.5:0/2194400375 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b02f9d74b0 0x55b02f9df160 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:41.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:41.180+0000 7fbb41600640 0 client.27905 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T09:50:41.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:41.180+0000 7fbb5d691740 1 --2- 172.21.15.5:0/2194400375 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b02f9d6b90 0x55b02f9dec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:41.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:41.180+0000 7fbb5d691740 1 -- 172.21.15.5:0/2194400375 >> 172.21.15.5:0/2194400375 conn(0x55b02f9d3120 msgr2=0x55b02f9d8e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:50:41.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:41.180+0000 7fbb5d691740 1 -- 172.21.15.5:0/2194400375 shutdown_connections 2024-09-10T09:50:41.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:41.180+0000 7fbb5d691740 1 -- 172.21.15.5:0/2194400375 wait complete. 2024-09-10T09:50:41.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:41.180+0000 7fbb5d691740 20 client.27905 trim_cache size 0 max 16384 2024-09-10T09:50:41.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:41.199+0000 7fbb5d691740 1 Processor -- start 2024-09-10T09:50:41.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:41.199+0000 7fbb5d691740 1 -- start start 2024-09-10T09:50:41.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:41.200+0000 7fbb5d691740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b02f9d6b90 0x55b02f9dd4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:50:41.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:41.200+0000 7fbb5d691740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b02f9d74b0 0x55b02f9df4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:50:41.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:41.200+0000 7fbb5d691740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b02f9e1d70 0x55b02f9f9850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:50:41.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:41.200+0000 7fbb5d691740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fbb541b10c0 con 0x55b02f9d74b0 2024-09-10T09:50:41.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:41.200+0000 7fbb5d691740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fbb541b2c00 con 0x55b02f9e1d70 2024-09-10T09:50:41.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:41.200+0000 7fbb5d691740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fbb541b1790 con 0x55b02f9d6b90 2024-09-10T09:50:41.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:41.200+0000 7fbb4be00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b02f9d74b0 0x55b02f9df4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:50:41.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:41.200+0000 7fbb58c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b02f9e1d70 0x55b02f9f9850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:50:41.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:41.200+0000 7fbb53e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b02f9d6b90 0x55b02f9dd4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:50:41.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:41.200+0000 7fbb58c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b02f9e1d70 0x55b02f9f9850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:49530/0 (socket says 172.21.15.5:49530) 2024-09-10T09:50:41.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:41.200+0000 7fbb58c00640 1 -- 172.21.15.5:0/4006272680 learned_addr learned my addr 172.21.15.5:0/4006272680 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:50:41.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:41.200+0000 7fbb58c00640 1 -- 172.21.15.5:0/4006272680 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b02f9d6b90 msgr2=0x55b02f9dd4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:50:41.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:41.200+0000 7fbb58c00640 1 --2- 172.21.15.5:0/4006272680 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b02f9d6b90 0x55b02f9dd4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:41.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:41.200+0000 7fbb58c00640 1 -- 172.21.15.5:0/4006272680 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b02f9d74b0 msgr2=0x55b02f9df4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:50:41.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:41.200+0000 7fbb58c00640 1 --2- 172.21.15.5:0/4006272680 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b02f9d74b0 0x55b02f9df4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:41.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:41.200+0000 7fbb58c00640 1 -- 172.21.15.5:0/4006272680 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b02f9d2f50 con 0x55b02f9e1d70 2024-09-10T09:50:41.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:41.200+0000 7fbb4be00640 1 --2- 172.21.15.5:0/4006272680 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b02f9d74b0 0x55b02f9df4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:50:41.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:41.201+0000 7fbb58c00640 1 --2- 172.21.15.5:0/4006272680 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b02f9e1d70 0x55b02f9f9850 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fbb4c031880 tx=0x7fbb4c044000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c07a9877df788ef7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:50:41.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:41.201+0000 7fbb43e00640 1 -- 172.21.15.5:0/4006272680 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb4c02fb90 con 0x55b02f9e1d70 2024-09-10T09:50:41.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:41.201+0000 7fbb43e00640 1 -- 172.21.15.5:0/4006272680 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbb4c036040 con 0x55b02f9e1d70 2024-09-10T09:50:41.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:41.201+0000 7fbb43e00640 1 -- 172.21.15.5:0/4006272680 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb4c004660 con 0x55b02f9e1d70 2024-09-10T09:50:41.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:41.202+0000 7fbb5d691740 1 -- 172.21.15.5:0/4006272680 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b02f9e1d70 msgr2=0x55b02f9f9850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:50:41.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:41.202+0000 7fbb5d691740 1 --2- 172.21.15.5:0/4006272680 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b02f9e1d70 0x55b02f9f9850 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fbb4c031880 tx=0x7fbb4c044000 comp rx=0 tx=0).stop 2024-09-10T09:50:41.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:41.202+0000 7fbb5d691740 1 -- 172.21.15.5:0/4006272680 shutdown_connections 2024-09-10T09:50:41.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:41.202+0000 7fbb5d691740 1 --2- 172.21.15.5:0/4006272680 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b02f9e1d70 0x55b02f9f9850 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:41.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:41.202+0000 7fbb5d691740 1 --2- 172.21.15.5:0/4006272680 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b02f9d74b0 0x55b02f9df4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:41.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:41.202+0000 7fbb5d691740 1 --2- 172.21.15.5:0/4006272680 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b02f9d6b90 0x55b02f9dd4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:41.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:41.202+0000 7fbb5d691740 1 -- 172.21.15.5:0/4006272680 >> 172.21.15.5:0/4006272680 conn(0x55b02f9d3120 msgr2=0x55b02f9e0ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:50:41.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:41.202+0000 7fbb5d691740 1 -- 172.21.15.5:0/4006272680 shutdown_connections 2024-09-10T09:50:41.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:41.202+0000 7fbb5d691740 1 -- 172.21.15.5:0/4006272680 wait complete. 2024-09-10T09:50:41.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:41.202+0000 7fbb5d691740 1 -- start start 2024-09-10T09:50:41.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:41.202+0000 7fbb5d691740 1 Processor -- start 2024-09-10T09:50:41.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:41.203+0000 7fbb5d691740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b02f9d6b90 0x55b02f9e5ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:50:41.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:41.203+0000 7fbb5d691740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b02f9d74b0 0x55b02f9e60e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:50:41.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:41.203+0000 7fbb5d691740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b02f9e1d70 0x55b02f9e6620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:50:41.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:41.203+0000 7fbb5d691740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fbb541b2530 con 0x55b02f9d6b90 2024-09-10T09:50:41.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:41.203+0000 7fbb5d691740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fbb541b1e60 con 0x55b02f9d74b0 2024-09-10T09:50:41.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:41.203+0000 7fbb5d691740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fbb541af0a0 con 0x55b02f9e1d70 2024-09-10T09:50:41.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:41.203+0000 7fbb5d691740 10 client.0 _ll_register_callbacks cb 0x7fbb3c00b9d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:50:41.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:41.203+0000 7fbb53e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b02f9d6b90 0x55b02f9e5ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:50:41.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:41.203+0000 7fbb53e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b02f9d6b90 0x55b02f9e5ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52272/0 (socket says 172.21.15.5:52272) 2024-09-10T09:50:41.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:41.203+0000 7fbb4be00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b02f9d74b0 0x55b02f9e60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:50:41.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:41.203+0000 7fbb53e00640 1 -- 172.21.15.5:0/4106801568 learned_addr learned my addr 172.21.15.5:0/4106801568 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:50:41.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:41.203+0000 7fbb58c00640 1 --2- 172.21.15.5:0/4106801568 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b02f9e1d70 0x55b02f9e6620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:50:41.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:41.204+0000 7fbb4be00640 1 -- 172.21.15.5:0/4106801568 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b02f9e1d70 msgr2=0x55b02f9e6620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:50:41.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:41.204+0000 7fbb4be00640 1 --2- 172.21.15.5:0/4106801568 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b02f9e1d70 0x55b02f9e6620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:41.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:41.204+0000 7fbb4be00640 1 -- 172.21.15.5:0/4106801568 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b02f9d6b90 msgr2=0x55b02f9e5ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:50:41.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:41.204+0000 7fbb4be00640 1 --2- 172.21.15.5:0/4106801568 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b02f9d6b90 0x55b02f9e5ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:41.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:41.204+0000 7fbb4be00640 1 -- 172.21.15.5:0/4106801568 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b02fa5b8d0 con 0x55b02f9d74b0 2024-09-10T09:50:41.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:41.204+0000 7fbb58c00640 1 --2- 172.21.15.5:0/4106801568 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b02f9e1d70 0x55b02f9e6620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:50:41.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:41.204+0000 7fbb53e00640 1 --2- 172.21.15.5:0/4106801568 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b02f9d6b90 0x55b02f9e5ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:50:41.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:41.204+0000 7fbb4be00640 1 --2- 172.21.15.5:0/4106801568 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b02f9d74b0 0x55b02f9e60e0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x55b02f9e01d0 tx=0x7fbb3c00b7e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=98afda149f70313c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:50:41.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:41.204+0000 7fbb41600640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-09-10T09:50:41.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:41.204+0000 7fbb41600640 1 -- 172.21.15.5:0/4106801568 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb3c005250 con 0x55b02f9d74b0 2024-09-10T09:50:41.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:41.204+0000 7fbb5d691740 1 -- 172.21.15.5:0/4106801568 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbb4c036040 con 0x55b02f9d74b0 2024-09-10T09:50:41.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:41.204+0000 7fbb41600640 1 -- 172.21.15.5:0/4106801568 <== mon.2 v2:172.21.15.102:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fbb3c017070 con 0x55b02f9d74b0 2024-09-10T09:50:41.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:41.204+0000 7fbb41600640 1 -- 172.21.15.5:0/4106801568 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fbb3c039050 con 0x55b02f9d74b0 2024-09-10T09:50:41.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:41.204+0000 7fbb5d691740 10 client.27911 fetch_fsmap learned FSMap version 36 2024-09-10T09:50:41.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:41.204+0000 7fbb5d691740 1 -- 172.21.15.5:0/4106801568 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55b02fa5cba0 con 0x55b02f9d74b0 2024-09-10T09:50:41.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:41.205+0000 7fbb41600640 1 -- 172.21.15.5:0/4106801568 <== mon.2 v2:172.21.15.102:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbb4c036040 con 0x55b02f9d74b0 2024-09-10T09:50:41.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:41.205+0000 7fbb41600640 1 -- 172.21.15.5:0/4106801568 <== mon.2 v2:172.21.15.102:3300/0 5 ==== fsmap(e 36) ==== 8370+0+0 (secure 0 0 0) 0x7fbb3c029430 con 0x55b02f9d74b0 2024-09-10T09:50:41.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:41.205+0000 7fbb5d691740 10 client.27911 fetch_fsmap finished waiting for FSMap version 36 2024-09-10T09:50:41.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:41.205+0000 7fbb5d691740 10 client.27911 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T09:50:41.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:41.205+0000 7fbb5d691740 20 client.27911 populate_metadata read hostname 'smithi005' 2024-09-10T09:50:41.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:41.205+0000 7fbb5d691740 1 --2- 172.21.15.5:0/4106801568 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55b02fa5d310 0x55b02fa5f730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:50:41.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:41.205+0000 7fbb5d691740 4 client.27911 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T09:50:41.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:41.205+0000 7fbb5d691740 1 -- 172.21.15.5:0/4106801568 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55b02f944e60 con 0x55b02fa5d310 2024-09-10T09:50:41.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:41.205+0000 7fbb53e00640 1 --2- 172.21.15.5:0/4106801568 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55b02fa5d310 0x55b02fa5f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:50:41.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:41.206+0000 7fbb53e00640 1 --2- 172.21.15.5:0/4106801568 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55b02fa5d310 0x55b02fa5f730 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6eceebf76a9195f3 server_cookie=afc73808c1458f19 in_seq=0 out_seq=0 2024-09-10T09:50:41.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:41.206+0000 7fbb41600640 10 client.27911 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T09:50:41.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:41.206+0000 7fbb41600640 1 -- 172.21.15.5:0/4106801568 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55b02f944e60 con 0x55b02fa5d310 2024-09-10T09:50:41.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:41.206+0000 7fbb41600640 10 client.27911 handle_command_reply: tid=0 2024-09-10T09:50:41.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:41.206+0000 7fbb5d691740 1 client.27911 shutdown 2024-09-10T09:50:41.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:41.206+0000 7fbb5d691740 1 -- 172.21.15.5:0/4106801568 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b02f9d74b0 msgr2=0x55b02f9e60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:50:41.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:41.206+0000 7fbb5d691740 1 --2- 172.21.15.5:0/4106801568 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b02f9d74b0 0x55b02f9e60e0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x55b02f9e01d0 tx=0x7fbb3c00b7e0 comp rx=0 tx=0).stop 2024-09-10T09:50:41.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:41.206+0000 7fbb5d691740 1 -- 172.21.15.5:0/4106801568 shutdown_connections 2024-09-10T09:50:41.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:41.206+0000 7fbb5d691740 1 --2- 172.21.15.5:0/4106801568 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55b02fa5d310 0x55b02fa5f730 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:41.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:41.206+0000 7fbb5d691740 1 --2- 172.21.15.5:0/4106801568 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b02f9e1d70 0x55b02f9e6620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:41.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:41.206+0000 7fbb5d691740 1 --2- 172.21.15.5:0/4106801568 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b02f9d74b0 0x55b02f9e60e0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:41.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:41.206+0000 7fbb5d691740 1 --2- 172.21.15.5:0/4106801568 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b02f9d6b90 0x55b02f9e5ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:41.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:41.206+0000 7fbb41600640 0 client.27911 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T09:50:41.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:41.206+0000 7fbb5d691740 1 -- 172.21.15.5:0/4106801568 >> 172.21.15.5:0/4106801568 conn(0x55b02f9d3120 msgr2=0x55b02f9d19a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:50:41.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:41.207+0000 7fbb5d691740 1 -- 172.21.15.5:0/4106801568 shutdown_connections 2024-09-10T09:50:41.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:41.207+0000 7fbb5d691740 1 -- 172.21.15.5:0/4106801568 wait complete. 2024-09-10T09:50:41.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:41.207+0000 7fbb5d691740 20 client.27911 trim_cache size 0 max 16384 2024-09-10T09:50:41.212 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-10T09:50:41.212 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-10T09:50:41.212 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2024-09-10T09:50:41.212 INFO:teuthology.orchestra.run.smithi005.stdout: "a7c127b0-5849-4704-b626-c076d9f90aed": { 2024-09-10T09:50:41.212 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2024-09-10T09:50:41.213 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "a7c127b0-5849-4704-b626-c076d9f90aed", 2024-09-10T09:50:41.213 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2024-09-10T09:50:41.213 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T09:50:41.213 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T09:50:41.213 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-10T09:50:41.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:41.207+0000 7fbb5ac00640 1 -- 172.21.15.5:0/95236319 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fbb380a2c00 msgr2=0x7fbb380a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:50:41.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:41.207+0000 7fbb5ac00640 1 --2- 172.21.15.5:0/95236319 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fbb380a2c00 0x7fbb380a50c0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fbb4c02f540 tx=0x7fbb4c02f3b0 comp rx=0 tx=0).stop 2024-09-10T09:50:41.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:41.207+0000 7fbb5ac00640 1 -- 172.21.15.5:0/95236319 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbb54103ff0 msgr2=0x7fbb5419a5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:50:41.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:41.207+0000 7fbb5ac00640 1 --2- 172.21.15.5:0/95236319 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbb54103ff0 0x7fbb5419a5c0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fbb4400e9f0 tx=0x7fbb4400edf0 comp rx=0 tx=0).stop 2024-09-10T09:50:41.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:41.208+0000 7fbb5ac00640 1 -- 172.21.15.5:0/95236319 shutdown_connections 2024-09-10T09:50:41.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:41.208+0000 7fbb5ac00640 1 --2- 172.21.15.5:0/95236319 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fbb380a2c00 0x7fbb380a50c0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:41.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:41.208+0000 7fbb5ac00640 1 --2- 172.21.15.5:0/95236319 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbb54109120 0x7fbb5419ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:41.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:41.208+0000 7fbb5ac00640 1 --2- 172.21.15.5:0/95236319 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbb541049a0 0x7fbb5419ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:41.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:41.208+0000 7fbb5ac00640 1 --2- 172.21.15.5:0/95236319 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbb54103ff0 0x7fbb5419a5c0 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:41.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:41.208+0000 7fbb5ac00640 1 -- 172.21.15.5:0/95236319 >> 172.21.15.5:0/95236319 conn(0x7fbb540ffa90 msgr2=0x7fbb541005b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:50:41.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:41.208+0000 7fbb5ac00640 1 -- 172.21.15.5:0/95236319 shutdown_connections 2024-09-10T09:50:41.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:41.208+0000 7fbb5ac00640 1 -- 172.21.15.5:0/95236319 wait complete. 2024-09-10T09:50:41.968 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:50:41 smithi005 ceph-mon[30350]: pgmap v763: 129 pgs: 129 active+clean; 295 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.8 MiB/s wr, 204 op/s 2024-09-10T09:50:42.011 INFO:tasks.cephfs.filesystem:scrub status for tag:a7c127b0-5849-4704-b626-c076d9f90aed - {'path': '/', 'tag': 'a7c127b0-5849-4704-b626-c076d9f90aed', 'options': 'recursive,force'} 2024-09-10T09:50:42.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:50:41 smithi159 ceph-mon[35513]: pgmap v763: 129 pgs: 129 active+clean; 295 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.8 MiB/s wr, 204 op/s 2024-09-10T09:50:42.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:50:41 smithi102 ceph-mon[35442]: pgmap v763: 129 pgs: 129 active+clean; 295 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.8 MiB/s wr, 204 op/s 2024-09-10T09:50:42.136 INFO:tasks.workunit.client.0.smithi005.stdout: 1 785836 6.39 MB/sec execute 503 sec latency 4381.903 ms 2024-09-10T09:50:43.136 INFO:tasks.workunit.client.0.smithi005.stdout: 1 785836 6.38 MB/sec execute 504 sec latency 5382.045 ms 2024-09-10T09:50:44.012 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:50:43 smithi005 ceph-mon[30350]: pgmap v764: 129 pgs: 129 active+clean; 296 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.3 MiB/s wr, 166 op/s 2024-09-10T09:50:44.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:50:43 smithi159 ceph-mon[35513]: pgmap v764: 129 pgs: 129 active+clean; 296 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.3 MiB/s wr, 166 op/s 2024-09-10T09:50:44.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:50:43 smithi102 ceph-mon[35442]: pgmap v764: 129 pgs: 129 active+clean; 296 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.3 MiB/s wr, 166 op/s 2024-09-10T09:50:44.136 INFO:tasks.workunit.client.0.smithi005.stdout: 1 785836 6.37 MB/sec execute 505 sec latency 6382.115 ms 2024-09-10T09:50:45.136 INFO:tasks.workunit.client.0.smithi005.stdout: 1 785836 6.36 MB/sec execute 506 sec latency 7382.177 ms 2024-09-10T09:50:45.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:50:45 smithi005 ceph-mon[30350]: pgmap v765: 129 pgs: 129 active+clean; 296 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.4 MiB/s wr, 161 op/s 2024-09-10T09:50:46.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:50:45 smithi159 ceph-mon[35513]: pgmap v765: 129 pgs: 129 active+clean; 296 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.4 MiB/s wr, 161 op/s 2024-09-10T09:50:46.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:50:45 smithi102 ceph-mon[35442]: pgmap v765: 129 pgs: 129 active+clean; 296 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.4 MiB/s wr, 161 op/s 2024-09-10T09:50:46.136 INFO:tasks.workunit.client.0.smithi005.stdout: 1 785836 6.34 MB/sec execute 507 sec latency 8382.288 ms 2024-09-10T09:50:47.136 INFO:tasks.workunit.client.0.smithi005.stdout: 1 785836 6.33 MB/sec execute 508 sec latency 9382.368 ms 2024-09-10T09:50:48.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:50:47 smithi005 ceph-mon[30350]: pgmap v766: 129 pgs: 129 active+clean; 293 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.2 MiB/s wr, 159 op/s 2024-09-10T09:50:48.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:50:47 smithi159 ceph-mon[35513]: pgmap v766: 129 pgs: 129 active+clean; 293 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.2 MiB/s wr, 159 op/s 2024-09-10T09:50:48.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:50:47 smithi102 ceph-mon[35442]: pgmap v766: 129 pgs: 129 active+clean; 293 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.2 MiB/s wr, 159 op/s 2024-09-10T09:50:48.136 INFO:tasks.workunit.client.0.smithi005.stdout: 1 785836 6.32 MB/sec execute 509 sec latency 10382.442 ms 2024-09-10T09:50:49.136 INFO:tasks.workunit.client.0.smithi005.stdout: 1 785836 6.31 MB/sec execute 510 sec latency 11382.495 ms 2024-09-10T09:50:49.853 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:50:49 smithi005 ceph-mon[30350]: pgmap v767: 129 pgs: 129 active+clean; 295 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 872 KiB/s rd, 2.0 MiB/s wr, 109 op/s 2024-09-10T09:50:50.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:50:49 smithi159 ceph-mon[35513]: pgmap v767: 129 pgs: 129 active+clean; 295 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 872 KiB/s rd, 2.0 MiB/s wr, 109 op/s 2024-09-10T09:50:50.136 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:50:49 smithi102 ceph-mon[35442]: pgmap v767: 129 pgs: 129 active+clean; 295 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 872 KiB/s rd, 2.0 MiB/s wr, 109 op/s 2024-09-10T09:50:50.136 INFO:tasks.workunit.client.0.smithi005.stdout: 1 785836 6.29 MB/sec execute 511 sec latency 12382.544 ms 2024-09-10T09:50:50.696 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T09:50:51.079 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:50:51.137 INFO:tasks.workunit.client.0.smithi005.stdout: 1 785836 6.28 MB/sec execute 512 sec latency 13382.620 ms 2024-09-10T09:50:51.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:51.646+0000 7f7d72e00640 1 -- 172.21.15.5:0/2638859836 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7d6c1049a0 msgr2=0x7f7d6c1089f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:50:51.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:51.646+0000 7f7d72e00640 1 --2- 172.21.15.5:0/2638859836 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7d6c1049a0 0x7f7d6c1089f0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f7d54009900 tx=0x7f7d5402a3f0 comp rx=0 tx=0).stop 2024-09-10T09:50:51.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:51.646+0000 7f7d72e00640 1 -- 172.21.15.5:0/2638859836 shutdown_connections 2024-09-10T09:50:51.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:51.646+0000 7f7d72e00640 1 --2- 172.21.15.5:0/2638859836 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7d6c109120 0x7f7d6c110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:51.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:51.646+0000 7f7d72e00640 1 --2- 172.21.15.5:0/2638859836 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7d6c1049a0 0x7f7d6c1089f0 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:51.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:51.646+0000 7f7d72e00640 1 --2- 172.21.15.5:0/2638859836 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7d6c103ff0 0x7f7d6c1043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:51.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:51.646+0000 7f7d72e00640 1 -- 172.21.15.5:0/2638859836 >> 172.21.15.5:0/2638859836 conn(0x7f7d6c0ffa90 msgr2=0x7f7d6c101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:50:51.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:51.646+0000 7f7d72e00640 1 -- 172.21.15.5:0/2638859836 shutdown_connections 2024-09-10T09:50:51.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:51.646+0000 7f7d72e00640 1 -- 172.21.15.5:0/2638859836 wait complete. 2024-09-10T09:50:51.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:51.647+0000 7f7d72e00640 1 Processor -- start 2024-09-10T09:50:51.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:51.647+0000 7f7d72e00640 1 -- start start 2024-09-10T09:50:51.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:51.648+0000 7f7d72e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7d6c103ff0 0x7f7d6c19a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:50:51.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:51.648+0000 7f7d72e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7d6c1049a0 0x7f7d6c19ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:50:51.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:51.648+0000 7f7d72e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7d6c109120 0x7f7d6c19eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:50:51.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:51.648+0000 7f7d72e00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f7d6c114190 con 0x7f7d6c1049a0 2024-09-10T09:50:51.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:51.648+0000 7f7d72e00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f7d6c114010 con 0x7f7d6c109120 2024-09-10T09:50:51.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:51.648+0000 7f7d72e00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f7d6c114310 con 0x7f7d6c103ff0 2024-09-10T09:50:51.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:51.648+0000 7f7d6be00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7d6c103ff0 0x7f7d6c19a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:50:51.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:51.648+0000 7f7d6be00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7d6c103ff0 0x7f7d6c19a5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:59436/0 (socket says 172.21.15.5:59436) 2024-09-10T09:50:51.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:51.648+0000 7f7d6be00640 1 -- 172.21.15.5:0/2784543259 learned_addr learned my addr 172.21.15.5:0/2784543259 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:50:51.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:51.648+0000 7f7d63400640 1 --2- 172.21.15.5:0/2784543259 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7d6c1049a0 0x7f7d6c19ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:50:51.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:51.648+0000 7f7d70e00640 1 --2- 172.21.15.5:0/2784543259 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7d6c109120 0x7f7d6c19eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:50:51.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:51.648+0000 7f7d6be00640 1 -- 172.21.15.5:0/2784543259 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7d6c109120 msgr2=0x7f7d6c19eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:50:51.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:51.648+0000 7f7d6be00640 1 --2- 172.21.15.5:0/2784543259 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7d6c109120 0x7f7d6c19eec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:51.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:51.648+0000 7f7d6be00640 1 -- 172.21.15.5:0/2784543259 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7d6c1049a0 msgr2=0x7f7d6c19ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:50:51.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:51.648+0000 7f7d6be00640 1 --2- 172.21.15.5:0/2784543259 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7d6c1049a0 0x7f7d6c19ab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:51.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:51.648+0000 7f7d6be00640 1 -- 172.21.15.5:0/2784543259 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d6c19f5a0 con 0x7f7d6c103ff0 2024-09-10T09:50:51.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:51.649+0000 7f7d6be00640 1 --2- 172.21.15.5:0/2784543259 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7d6c103ff0 0x7f7d6c19a5f0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f7d5c00e9f0 tx=0x7f7d5c00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:50:51.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:51.649+0000 7f7d69600640 1 -- 172.21.15.5:0/2784543259 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d5c00cd80 con 0x7f7d6c103ff0 2024-09-10T09:50:51.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:51.649+0000 7f7d72e00640 1 -- 172.21.15.5:0/2784543259 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d6c19f890 con 0x7f7d6c103ff0 2024-09-10T09:50:51.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:51.649+0000 7f7d69600640 1 -- 172.21.15.5:0/2784543259 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7d5c010910 con 0x7f7d6c103ff0 2024-09-10T09:50:51.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:51.649+0000 7f7d69600640 1 -- 172.21.15.5:0/2784543259 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d5c0188f0 con 0x7f7d6c103ff0 2024-09-10T09:50:51.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:51.649+0000 7f7d72e00640 1 -- 172.21.15.5:0/2784543259 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d6c1a7170 con 0x7f7d6c103ff0 2024-09-10T09:50:51.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:51.650+0000 7f7d72e00640 1 -- 172.21.15.5:0/2784543259 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d6c1043d0 con 0x7f7d6c103ff0 2024-09-10T09:50:51.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:51.652+0000 7f7d69600640 1 -- 172.21.15.5:0/2784543259 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f7d5c010ac0 con 0x7f7d6c103ff0 2024-09-10T09:50:51.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:51.653+0000 7f7d69600640 1 --2- 172.21.15.5:0/2784543259 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f7d500a2b80 0x7f7d500a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:50:51.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:51.654+0000 7f7d69600640 1 -- 172.21.15.5:0/2784543259 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f7d5c0cfc20 con 0x7f7d6c103ff0 2024-09-10T09:50:51.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:51.654+0000 7f7d63400640 1 --2- 172.21.15.5:0/2784543259 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f7d500a2b80 0x7f7d500a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:50:51.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:51.654+0000 7f7d63400640 1 --2- 172.21.15.5:0/2784543259 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f7d500a2b80 0x7f7d500a5040 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f7d54000c00 tx=0x7f7d54005b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:50:51.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:51.656+0000 7f7d69600640 1 -- 172.21.15.5:0/2784543259 <== mon.1 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f7d5c09c0e0 con 0x7f7d6c103ff0 2024-09-10T09:50:51.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:51.882+0000 7f7d72e00640 1 -- 172.21.15.5:0/2784543259 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7d6c1a7340 con 0x7f7d6c103ff0 2024-09-10T09:50:51.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:51.883+0000 7f7d69600640 1 -- 172.21.15.5:0/2784543259 <== mon.1 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) ==== 94+0+11969 (secure 0 0 0) 0x7f7d5c0a03e0 con 0x7f7d6c103ff0 2024-09-10T09:50:51.885 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-10T09:50:51.885 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":36,"btime":"2024-09-10T09:48:07:349075+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14454,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/2970594311","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2970594311},{"type":"v1","addr":"172.21.15.5:6835","nonce":2970594311}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14514,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6839/3616502174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":3616502174},{"type":"v1","addr":"172.21.15.5:6839","nonce":3616502174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":36,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T09:35:29.626287+0000","modified":"2024-09-10T09:48:06.970127+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24542,"mds_1":24535,"mds_2":24511,"mds_3":24506,"mds_4":24443},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14520":{"gid":14520,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/2073599842","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":2073599842},{"type":"v1","addr":"172.21.15.5:6841","nonce":2073599842}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"k","rank":4,"incarnation":26,"state":"up:active","state_seq":24,"addr":"172.21.15.102:6833/2457007427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":2457007427},{"type":"v1","addr":"172.21.15.102:6833","nonce":2457007427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/356445159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":356445159},{"type":"v1","addr":"172.21.15.159:6833","nonce":356445159}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6837/2894314507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":2894314507},{"type":"v1","addr":"172.21.15.5:6837","nonce":2894314507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.102:6835/1647549111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":1647549111},{"type":"v1","addr":"172.21.15.102:6835","nonce":1647549111}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6835/952657242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":952657242},{"type":"v1","addr":"172.21.15.159:6835","nonce":952657242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":19,"addr":"172.21.15.102:6837/2689776764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":2689776764},{"type":"v1","addr":"172.21.15.102:6837","nonce":2689776764}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.159:6837/620627639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":620627639},{"type":"v1","addr":"172.21.15.159:6837","nonce":620627639}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24535":{"gid":24535,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":18,"addr":"172.21.15.159:6839/2073739599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":2073739599},{"type":"v1","addr":"172.21.15.159:6839","nonce":2073739599}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24542":{"gid":24542,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.102:6839/335409749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6838","nonce":335409749},{"type":"v1","addr":"172.21.15.102:6839","nonce":335409749}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24542,"qdb_cluster":[24542,24535,24511,24506,24443]},"id":1}]} 2024-09-10T09:50:51.885 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 36 2024-09-10T09:50:51.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:51.886+0000 7f7d72e00640 1 -- 172.21.15.5:0/2784543259 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f7d500a2b80 msgr2=0x7f7d500a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:50:51.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:51.886+0000 7f7d72e00640 1 --2- 172.21.15.5:0/2784543259 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f7d500a2b80 0x7f7d500a5040 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f7d54000c00 tx=0x7f7d54005b50 comp rx=0 tx=0).stop 2024-09-10T09:50:51.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:51.886+0000 7f7d72e00640 1 -- 172.21.15.5:0/2784543259 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7d6c103ff0 msgr2=0x7f7d6c19a5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:50:51.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:51.886+0000 7f7d72e00640 1 --2- 172.21.15.5:0/2784543259 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7d6c103ff0 0x7f7d6c19a5f0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f7d5c00e9f0 tx=0x7f7d5c00eeb0 comp rx=0 tx=0).stop 2024-09-10T09:50:51.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:51.886+0000 7f7d72e00640 1 -- 172.21.15.5:0/2784543259 shutdown_connections 2024-09-10T09:50:51.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:51.886+0000 7f7d72e00640 1 --2- 172.21.15.5:0/2784543259 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f7d500a2b80 0x7f7d500a5040 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:51.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:51.886+0000 7f7d72e00640 1 --2- 172.21.15.5:0/2784543259 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7d6c109120 0x7f7d6c19eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:51.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:51.886+0000 7f7d72e00640 1 --2- 172.21.15.5:0/2784543259 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7d6c1049a0 0x7f7d6c19ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:51.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:51.886+0000 7f7d72e00640 1 --2- 172.21.15.5:0/2784543259 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7d6c103ff0 0x7f7d6c19a5f0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:51.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:51.886+0000 7f7d72e00640 1 -- 172.21.15.5:0/2784543259 >> 172.21.15.5:0/2784543259 conn(0x7f7d6c0ffa90 msgr2=0x7f7d6c101e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:50:51.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:51.886+0000 7f7d72e00640 1 -- 172.21.15.5:0/2784543259 shutdown_connections 2024-09-10T09:50:51.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:51.886+0000 7f7d72e00640 1 -- 172.21.15.5:0/2784543259 wait complete. 2024-09-10T09:50:51.901 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:50:51 smithi005 ceph-mon[30350]: pgmap v768: 129 pgs: 129 active+clean; 293 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 661 KiB/s rd, 2.0 MiB/s wr, 115 op/s 2024-09-10T09:50:52.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:50:51 smithi159 ceph-mon[35513]: pgmap v768: 129 pgs: 129 active+clean; 293 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 661 KiB/s rd, 2.0 MiB/s wr, 115 op/s 2024-09-10T09:50:52.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:50:51 smithi102 ceph-mon[35442]: pgmap v768: 129 pgs: 129 active+clean; 293 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 661 KiB/s rd, 2.0 MiB/s wr, 115 op/s 2024-09-10T09:50:52.137 INFO:tasks.workunit.client.0.smithi005.stdout: 1 785836 6.27 MB/sec execute 513 sec latency 14383.146 ms 2024-09-10T09:50:52.823 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24542 quiesce db --set-id 6b75eb2 --release --await' 2024-09-10T09:50:52.823 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24542 quiesce db --set-id 6b75eb2 --release --await 2024-09-10T09:50:53.001 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:50:52 smithi159 ceph-mon[35513]: from='client.? 172.21.15.5:0/2784543259' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:50:53.002 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:50:52 smithi159 ceph-mon[35513]: scrub summary: idle 2024-09-10T09:50:53.002 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:50:52 smithi159 ceph-mon[35513]: scrub summary: idle 2024-09-10T09:50:53.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:50:52 smithi005 ceph-mon[30350]: from='client.? 172.21.15.5:0/2784543259' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:50:53.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:50:52 smithi005 ceph-mon[30350]: scrub summary: idle 2024-09-10T09:50:53.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:50:52 smithi005 ceph-mon[30350]: scrub summary: idle 2024-09-10T09:50:53.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:50:52 smithi102 ceph-mon[35442]: from='client.? 172.21.15.5:0/2784543259' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:50:53.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:50:52 smithi102 ceph-mon[35442]: scrub summary: idle 2024-09-10T09:50:53.135 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:50:52 smithi102 ceph-mon[35442]: scrub summary: idle 2024-09-10T09:50:53.137 INFO:tasks.workunit.client.0.smithi005.stdout: 1 785836 6.26 MB/sec execute 514 sec latency 15383.217 ms 2024-09-10T09:50:53.200 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:50:53.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:53.751+0000 7f8da7400640 1 -- 172.21.15.5:0/846306550 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8da00745c0 msgr2=0x7f8da00749a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:50:53.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:53.751+0000 7f8da7400640 1 --2- 172.21.15.5:0/846306550 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8da00745c0 0x7f8da00749a0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f8d94009900 tx=0x7f8d9402a480 comp rx=0 tx=0).stop 2024-09-10T09:50:53.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:53.751+0000 7f8da7400640 1 -- 172.21.15.5:0/846306550 shutdown_connections 2024-09-10T09:50:53.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:53.751+0000 7f8da7400640 1 --2- 172.21.15.5:0/846306550 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8da010e430 0x7f8da0110820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:53.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:53.751+0000 7f8da7400640 1 --2- 172.21.15.5:0/846306550 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8da0074ee0 0x7f8da010def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:53.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:53.751+0000 7f8da7400640 1 --2- 172.21.15.5:0/846306550 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8da00745c0 0x7f8da00749a0 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:53.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:53.751+0000 7f8da7400640 1 -- 172.21.15.5:0/846306550 >> 172.21.15.5:0/846306550 conn(0x7f8da00fd8b0 msgr2=0x7f8da00ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:50:53.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:53.751+0000 7f8da7400640 1 -- 172.21.15.5:0/846306550 shutdown_connections 2024-09-10T09:50:53.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:53.751+0000 7f8da7400640 1 -- 172.21.15.5:0/846306550 wait complete. 2024-09-10T09:50:53.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:53.752+0000 7f8da7400640 1 Processor -- start 2024-09-10T09:50:53.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:53.752+0000 7f8da7400640 1 -- start start 2024-09-10T09:50:53.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:53.752+0000 7f8da7400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8da00745c0 0x7f8da019a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:50:53.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:53.752+0000 7f8da7400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8da0074ee0 0x7f8da019abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:50:53.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:53.752+0000 7f8da7400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8da010e430 0x7f8da019ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:50:53.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:53.752+0000 7f8da7400640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f8da01141f0 con 0x7f8da0074ee0 2024-09-10T09:50:53.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:53.752+0000 7f8da7400640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f8da0114070 con 0x7f8da00745c0 2024-09-10T09:50:53.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:53.752+0000 7f8da7400640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f8da0114370 con 0x7f8da010e430 2024-09-10T09:50:53.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:53.753+0000 7f8d9fe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8da0074ee0 0x7f8da019abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:50:53.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:53.753+0000 7f8d9fe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8da0074ee0 0x7f8da019abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39252/0 (socket says 172.21.15.5:39252) 2024-09-10T09:50:53.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:53.753+0000 7f8d9fe00640 1 -- 172.21.15.5:0/404977917 learned_addr learned my addr 172.21.15.5:0/404977917 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:50:53.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:53.753+0000 7f8da5400640 1 --2- 172.21.15.5:0/404977917 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8da010e430 0x7f8da019ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:50:53.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:53.753+0000 7f8da4a00640 1 --2- 172.21.15.5:0/404977917 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8da00745c0 0x7f8da019a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:50:53.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:53.753+0000 7f8d9fe00640 1 -- 172.21.15.5:0/404977917 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8da010e430 msgr2=0x7f8da019ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:50:53.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:53.753+0000 7f8d9fe00640 1 --2- 172.21.15.5:0/404977917 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8da010e430 0x7f8da019ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:53.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:53.753+0000 7f8d9fe00640 1 -- 172.21.15.5:0/404977917 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8da00745c0 msgr2=0x7f8da019a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:50:53.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:53.753+0000 7f8d9fe00640 1 --2- 172.21.15.5:0/404977917 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8da00745c0 0x7f8da019a690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:53.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:53.753+0000 7f8d9fe00640 1 -- 172.21.15.5:0/404977917 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8da019f640 con 0x7f8da0074ee0 2024-09-10T09:50:53.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:53.754+0000 7f8d9fe00640 1 --2- 172.21.15.5:0/404977917 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8da0074ee0 0x7f8da019abd0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f8d8c00e9f0 tx=0x7f8d8c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:50:53.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:53.754+0000 7f8d9d600640 1 -- 172.21.15.5:0/404977917 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d8c00ccc0 con 0x7f8da0074ee0 2024-09-10T09:50:53.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:53.754+0000 7f8da7400640 1 -- 172.21.15.5:0/404977917 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8da019f930 con 0x7f8da0074ee0 2024-09-10T09:50:53.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:53.754+0000 7f8d9d600640 1 -- 172.21.15.5:0/404977917 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8d8c00ce60 con 0x7f8da0074ee0 2024-09-10T09:50:53.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:53.754+0000 7f8d9d600640 1 -- 172.21.15.5:0/404977917 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d8c010690 con 0x7f8da0074ee0 2024-09-10T09:50:53.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:53.754+0000 7f8da7400640 1 -- 172.21.15.5:0/404977917 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8da01a7210 con 0x7f8da0074ee0 2024-09-10T09:50:53.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:53.757+0000 7f8da9e59740 1 Processor -- start 2024-09-10T09:50:53.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:53.757+0000 7f8da9e59740 1 -- start start 2024-09-10T09:50:53.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:53.757+0000 7f8d9d600640 1 -- 172.21.15.5:0/404977917 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f8d8c0046c0 con 0x7f8da0074ee0 2024-09-10T09:50:53.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:53.758+0000 7f8da9e59740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555e0bd4b310 0x555e0bd4b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:50:53.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:53.758+0000 7f8da9e59740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555e0bd4bc30 0x555e0bd55fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:50:53.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:53.758+0000 7f8da9e59740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555e0bd564f0 0x555e0bd588e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:50:53.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:53.758+0000 7f8da9e59740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x555e0bcb1650 con 0x555e0bd564f0 2024-09-10T09:50:53.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:53.758+0000 7f8da9e59740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x555e0bc58f10 con 0x555e0bd4b310 2024-09-10T09:50:53.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:53.758+0000 7f8da9e59740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x555e0bcad610 con 0x555e0bd4bc30 2024-09-10T09:50:53.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:53.758+0000 7f8da4a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555e0bd4bc30 0x555e0bd55fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:50:53.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:53.758+0000 7f8da5400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555e0bd4b310 0x555e0bd4b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:50:53.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:53.758+0000 7f8da5400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555e0bd4b310 0x555e0bd4b6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:56014/0 (socket says 172.21.15.5:56014) 2024-09-10T09:50:53.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:53.758+0000 7f8da4a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555e0bd4bc30 0x555e0bd55fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:59452/0 (socket says 172.21.15.5:59452) 2024-09-10T09:50:53.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:53.758+0000 7f8da5400640 1 -- 172.21.15.5:0/3109249522 learned_addr learned my addr 172.21.15.5:0/3109249522 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:50:53.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:53.758+0000 7f8da5400640 1 -- 172.21.15.5:0/3109249522 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555e0bd4bc30 msgr2=0x555e0bd55fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:50:53.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:53.758+0000 7f8da5400640 1 --2- 172.21.15.5:0/3109249522 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555e0bd4bc30 0x555e0bd55fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:53.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:53.758+0000 7f8da5400640 1 -- 172.21.15.5:0/3109249522 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555e0bd564f0 msgr2=0x555e0bd588e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:50:53.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:53.758+0000 7f8da5400640 1 --2- 172.21.15.5:0/3109249522 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555e0bd564f0 0x555e0bd588e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:53.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:53.758+0000 7f8da5400640 1 -- 172.21.15.5:0/3109249522 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555e0b9c0580 con 0x555e0bd4b310 2024-09-10T09:50:53.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:53.759+0000 7f8da4a00640 1 --2- 172.21.15.5:0/3109249522 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555e0bd4bc30 0x555e0bd55fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:50:53.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:53.759+0000 7f8da5400640 1 --2- 172.21.15.5:0/3109249522 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555e0bd4b310 0x555e0bd4b6f0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f8d98009f40 tx=0x7f8d980118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=126a88a1d6b98b9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:50:53.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:53.759+0000 7f8d91c00640 1 -- 172.21.15.5:0/3109249522 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d98004d20 con 0x555e0bd4b310 2024-09-10T09:50:53.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:53.759+0000 7f8d91c00640 1 -- 172.21.15.5:0/3109249522 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8d98004050 con 0x555e0bd4b310 2024-09-10T09:50:53.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:53.759+0000 7f8d91c00640 1 -- 172.21.15.5:0/3109249522 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d98005280 con 0x555e0bd4b310 2024-09-10T09:50:53.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:53.759+0000 7f8d9d600640 1 --2- 172.21.15.5:0/404977917 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f8d880a2c00 0x7f8d880a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:50:53.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:53.760+0000 7f8d9d600640 1 -- 172.21.15.5:0/404977917 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f8d8c0cf7f0 con 0x7f8da0074ee0 2024-09-10T09:50:53.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:53.760+0000 7f8da4a00640 1 --2- 172.21.15.5:0/404977917 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f8d880a2c00 0x7f8d880a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:50:53.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:53.760+0000 7f8da9e59740 1 -- 172.21.15.5:0/3109249522 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555e0bd4b310 msgr2=0x555e0bd4b6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:50:53.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:53.760+0000 7f8da9e59740 1 --2- 172.21.15.5:0/3109249522 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555e0bd4b310 0x555e0bd4b6f0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f8d98009f40 tx=0x7f8d980118d0 comp rx=0 tx=0).stop 2024-09-10T09:50:53.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:53.760+0000 7f8da9e59740 1 -- 172.21.15.5:0/3109249522 shutdown_connections 2024-09-10T09:50:53.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:53.760+0000 7f8da9e59740 1 --2- 172.21.15.5:0/3109249522 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555e0bd564f0 0x555e0bd588e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:53.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:53.760+0000 7f8da9e59740 1 --2- 172.21.15.5:0/3109249522 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555e0bd4bc30 0x555e0bd55fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:53.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:53.760+0000 7f8da9e59740 1 --2- 172.21.15.5:0/3109249522 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555e0bd4b310 0x555e0bd4b6f0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:53.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:53.760+0000 7f8da9e59740 1 -- 172.21.15.5:0/3109249522 >> 172.21.15.5:0/3109249522 conn(0x555e0bd478a0 msgr2=0x555e0bd47c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:50:53.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:53.760+0000 7f8da9e59740 1 -- 172.21.15.5:0/3109249522 shutdown_connections 2024-09-10T09:50:53.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:53.760+0000 7f8da9e59740 1 -- 172.21.15.5:0/3109249522 wait complete. 2024-09-10T09:50:53.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:53.761+0000 7f8da9e59740 1 -- start start 2024-09-10T09:50:53.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:53.761+0000 7f8da4a00640 1 --2- 172.21.15.5:0/404977917 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f8d880a2c00 0x7f8d880a50c0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f8d94000c00 tx=0x7f8d94005b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:50:53.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:53.761+0000 7f8da9e59740 1 Processor -- start 2024-09-10T09:50:53.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:53.761+0000 7f8da9e59740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555e0bd4b310 0x555e0bd534a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:50:53.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:53.761+0000 7f8da9e59740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555e0bd4bc30 0x555e0bd539e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:50:53.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:53.761+0000 7f8da9e59740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555e0bd564f0 0x555e0bd53f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:50:53.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:53.762+0000 7f8da9e59740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x555e0bd590c0 con 0x555e0bd564f0 2024-09-10T09:50:53.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:53.762+0000 7f8da9e59740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x555e0bc6c350 con 0x555e0bd4b310 2024-09-10T09:50:53.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:53.762+0000 7f8da9e59740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x555e0bd59240 con 0x555e0bd4bc30 2024-09-10T09:50:53.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:53.762+0000 7f8da9e59740 10 client.0 _ll_register_callbacks cb 0x555e0bc8f290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:50:53.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:53.762+0000 7f8da4a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555e0bd4b310 0x555e0bd534a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:50:53.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:53.762+0000 7f8da5400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555e0bd564f0 0x555e0bd53f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:50:53.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:53.762+0000 7f8da4a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555e0bd4b310 0x555e0bd534a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:56030/0 (socket says 172.21.15.5:56030) 2024-09-10T09:50:53.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:53.762+0000 7f8da4a00640 1 -- 172.21.15.5:0/1789158328 learned_addr learned my addr 172.21.15.5:0/1789158328 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:50:53.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:53.762+0000 7f8da5400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555e0bd564f0 0x555e0bd53f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39262/0 (socket says 172.21.15.5:39262) 2024-09-10T09:50:53.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:53.762+0000 7f8d9fe00640 1 --2- 172.21.15.5:0/1789158328 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555e0bd4bc30 0x555e0bd539e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:50:53.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:53.762+0000 7f8da4a00640 1 -- 172.21.15.5:0/1789158328 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555e0bd4bc30 msgr2=0x555e0bd539e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:50:53.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:53.762+0000 7f8da4a00640 1 --2- 172.21.15.5:0/1789158328 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555e0bd4bc30 0x555e0bd539e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:53.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:53.762+0000 7f8da4a00640 1 -- 172.21.15.5:0/1789158328 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555e0bd564f0 msgr2=0x555e0bd53f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:50:53.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:53.762+0000 7f8da4a00640 1 --2- 172.21.15.5:0/1789158328 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555e0bd564f0 0x555e0bd53f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:53.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:53.762+0000 7f8da4a00640 1 -- 172.21.15.5:0/1789158328 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555e0ba8a960 con 0x555e0bd4b310 2024-09-10T09:50:53.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:53.763+0000 7f8da4a00640 1 --2- 172.21.15.5:0/1789158328 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555e0bd4b310 0x555e0bd534a0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f8d94009900 tx=0x7f8d94009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cbebe47f28206b5f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:50:53.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:53.763+0000 7f8d86000640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-09-10T09:50:53.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:53.763+0000 7f8d86000640 1 -- 172.21.15.5:0/1789158328 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d9404e070 con 0x555e0bd4b310 2024-09-10T09:50:53.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:53.763+0000 7f8da9e59740 1 -- 172.21.15.5:0/1789158328 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8d98004050 con 0x555e0bd4b310 2024-09-10T09:50:53.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:53.763+0000 7f8d86000640 1 -- 172.21.15.5:0/1789158328 <== mon.2 v2:172.21.15.102:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f8d94004ad0 con 0x555e0bd4b310 2024-09-10T09:50:53.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:53.763+0000 7f8d86000640 1 -- 172.21.15.5:0/1789158328 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f8d9405a050 con 0x555e0bd4b310 2024-09-10T09:50:53.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:53.763+0000 7f8da9e59740 10 client.27929 fetch_fsmap learned FSMap version 36 2024-09-10T09:50:53.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:53.763+0000 7f8da9e59740 1 -- 172.21.15.5:0/1789158328 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x555e0bd64df0 con 0x555e0bd4b310 2024-09-10T09:50:53.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:53.764+0000 7f8d86000640 1 -- 172.21.15.5:0/1789158328 <== mon.2 v2:172.21.15.102:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8d98004050 con 0x555e0bd4b310 2024-09-10T09:50:53.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:53.764+0000 7f8d86000640 1 -- 172.21.15.5:0/1789158328 <== mon.2 v2:172.21.15.102:3300/0 5 ==== fsmap(e 36) ==== 8370+0+0 (secure 0 0 0) 0x7f8d94003660 con 0x555e0bd4b310 2024-09-10T09:50:53.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:53.764+0000 7f8da9e59740 10 client.27929 fetch_fsmap finished waiting for FSMap version 36 2024-09-10T09:50:53.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:53.764+0000 7f8da9e59740 10 client.27929 resolve_mds: validated gid 24542 aka daemon mds.b 2024-09-10T09:50:53.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:53.764+0000 7f8da9e59740 20 client.27929 populate_metadata read hostname 'smithi005' 2024-09-10T09:50:53.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:53.764+0000 7f8da9e59740 1 --2- 172.21.15.5:0/1789158328 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x555e0bd66200 0x555e0bd68610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:50:53.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:53.764+0000 7f8da9e59740 4 client.27929 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:50:53.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:53.764+0000 7f8da9e59740 1 -- 172.21.15.5:0/1789158328 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555e0bcbebc0 con 0x555e0bd66200 2024-09-10T09:50:53.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:53.764+0000 7f8d9fe00640 1 --2- 172.21.15.5:0/1789158328 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x555e0bd66200 0x555e0bd68610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:50:53.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:53.765+0000 7f8d9fe00640 1 --2- 172.21.15.5:0/1789158328 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x555e0bd66200 0x555e0bd68610 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9c5b6bc07a931312 server_cookie=8fa3e53ae3afd2f0 in_seq=0 out_seq=0 2024-09-10T09:50:53.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:53.765+0000 7f8d86000640 10 client.27929 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T09:50:53.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:53.766+0000 7f8d86000640 1 -- 172.21.15.5:0/1789158328 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555e0bcbebc0 con 0x555e0bd66200 2024-09-10T09:50:53.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:53.766+0000 7f8d86000640 10 client.27929 handle_command_reply: tid=0 2024-09-10T09:50:53.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:53.766+0000 7f8da9e59740 1 client.27929 shutdown 2024-09-10T09:50:53.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:53.766+0000 7f8da9e59740 1 -- 172.21.15.5:0/1789158328 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555e0bd4b310 msgr2=0x555e0bd534a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:50:53.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:53.766+0000 7f8da9e59740 1 --2- 172.21.15.5:0/1789158328 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555e0bd4b310 0x555e0bd534a0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f8d94009900 tx=0x7f8d94009260 comp rx=0 tx=0).stop 2024-09-10T09:50:53.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:53.766+0000 7f8da9e59740 1 -- 172.21.15.5:0/1789158328 shutdown_connections 2024-09-10T09:50:53.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:53.766+0000 7f8da9e59740 1 --2- 172.21.15.5:0/1789158328 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x555e0bd66200 0x555e0bd68610 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:53.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:53.766+0000 7f8da9e59740 1 --2- 172.21.15.5:0/1789158328 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555e0bd564f0 0x555e0bd53f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:53.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:53.766+0000 7f8da9e59740 1 --2- 172.21.15.5:0/1789158328 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555e0bd4bc30 0x555e0bd539e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:53.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:53.766+0000 7f8da9e59740 1 --2- 172.21.15.5:0/1789158328 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555e0bd4b310 0x555e0bd534a0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:53.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:53.766+0000 7f8d86000640 0 client.27929 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T09:50:53.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:53.766+0000 7f8da9e59740 1 -- 172.21.15.5:0/1789158328 >> 172.21.15.5:0/1789158328 conn(0x555e0bd478a0 msgr2=0x555e0bd46aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:50:53.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:53.766+0000 7f8da9e59740 1 -- 172.21.15.5:0/1789158328 shutdown_connections 2024-09-10T09:50:53.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:53.767+0000 7f8da9e59740 1 -- 172.21.15.5:0/1789158328 wait complete. 2024-09-10T09:50:53.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:53.767+0000 7f8da9e59740 20 client.27929 trim_cache size 0 max 16384 2024-09-10T09:50:53.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:53.784+0000 7f8da9e59740 1 Processor -- start 2024-09-10T09:50:53.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:53.784+0000 7f8da9e59740 1 -- start start 2024-09-10T09:50:53.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:53.785+0000 7f8da9e59740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555e0bd4b310 0x555e0bd55300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:50:53.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:53.785+0000 7f8da9e59740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555e0bd4bc30 0x555e0bd6dee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:50:53.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:53.785+0000 7f8da9e59740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555e0bd564f0 0x555e0bd6e420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:50:53.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:53.785+0000 7f8da9e59740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f8da01b1020 con 0x555e0bd564f0 2024-09-10T09:50:53.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:53.785+0000 7f8da9e59740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f8da01b2b60 con 0x555e0bd4bc30 2024-09-10T09:50:53.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:53.785+0000 7f8da9e59740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f8da01b16f0 con 0x555e0bd4b310 2024-09-10T09:50:53.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:53.785+0000 7f8da5400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555e0bd564f0 0x555e0bd6e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:50:53.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:53.785+0000 7f8d9fe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555e0bd4bc30 0x555e0bd6dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:50:53.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:53.785+0000 7f8da5400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555e0bd564f0 0x555e0bd6e420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39266/0 (socket says 172.21.15.5:39266) 2024-09-10T09:50:53.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:53.785+0000 7f8da5400640 1 -- 172.21.15.5:0/3333511183 learned_addr learned my addr 172.21.15.5:0/3333511183 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:50:53.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:53.785+0000 7f8da4a00640 1 --2- 172.21.15.5:0/3333511183 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555e0bd4b310 0x555e0bd55300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:50:53.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:53.785+0000 7f8d9fe00640 1 -- 172.21.15.5:0/3333511183 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555e0bd4b310 msgr2=0x555e0bd55300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:50:53.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:53.785+0000 7f8d9fe00640 1 --2- 172.21.15.5:0/3333511183 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555e0bd4b310 0x555e0bd55300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:53.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:53.785+0000 7f8d9fe00640 1 -- 172.21.15.5:0/3333511183 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555e0bd564f0 msgr2=0x555e0bd6e420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:50:53.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:53.785+0000 7f8d9fe00640 1 --2- 172.21.15.5:0/3333511183 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555e0bd564f0 0x555e0bd6e420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:53.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:53.785+0000 7f8d9fe00640 1 -- 172.21.15.5:0/3333511183 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555e0bd4dbf0 con 0x555e0bd4bc30 2024-09-10T09:50:53.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:53.785+0000 7f8da5400640 1 --2- 172.21.15.5:0/3333511183 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555e0bd564f0 0x555e0bd6e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:50:53.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:53.786+0000 7f8d9fe00640 1 --2- 172.21.15.5:0/3333511183 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555e0bd4bc30 0x555e0bd6dee0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x555e0bd48c20 tx=0x7f8d8c005720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1a05783b521b900b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:50:53.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:53.786+0000 7f8d91200640 1 -- 172.21.15.5:0/3333511183 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d8c0b9070 con 0x555e0bd4bc30 2024-09-10T09:50:53.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:53.786+0000 7f8d91200640 1 -- 172.21.15.5:0/3333511183 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8d8c0b2070 con 0x555e0bd4bc30 2024-09-10T09:50:53.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:53.786+0000 7f8d91200640 1 -- 172.21.15.5:0/3333511183 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d8c0cf440 con 0x555e0bd4bc30 2024-09-10T09:50:53.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:53.787+0000 7f8da9e59740 1 -- 172.21.15.5:0/3333511183 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555e0bd4bc30 msgr2=0x555e0bd6dee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:50:53.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:53.787+0000 7f8da9e59740 1 --2- 172.21.15.5:0/3333511183 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555e0bd4bc30 0x555e0bd6dee0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x555e0bd48c20 tx=0x7f8d8c005720 comp rx=0 tx=0).stop 2024-09-10T09:50:53.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:53.787+0000 7f8da9e59740 1 -- 172.21.15.5:0/3333511183 shutdown_connections 2024-09-10T09:50:53.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:53.787+0000 7f8da9e59740 1 --2- 172.21.15.5:0/3333511183 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555e0bd564f0 0x555e0bd6e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:53.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:53.787+0000 7f8da9e59740 1 --2- 172.21.15.5:0/3333511183 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555e0bd4bc30 0x555e0bd6dee0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:53.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:53.787+0000 7f8da9e59740 1 --2- 172.21.15.5:0/3333511183 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555e0bd4b310 0x555e0bd55300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:53.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:53.787+0000 7f8da9e59740 1 -- 172.21.15.5:0/3333511183 >> 172.21.15.5:0/3333511183 conn(0x555e0bd478a0 msgr2=0x555e0bd54bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:50:53.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:53.787+0000 7f8da9e59740 1 -- 172.21.15.5:0/3333511183 shutdown_connections 2024-09-10T09:50:53.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:53.787+0000 7f8da9e59740 1 -- 172.21.15.5:0/3333511183 wait complete. 2024-09-10T09:50:53.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:53.787+0000 7f8da9e59740 1 -- start start 2024-09-10T09:50:53.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:53.787+0000 7f8da9e59740 1 Processor -- start 2024-09-10T09:50:53.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:53.788+0000 7f8da9e59740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555e0bd4b310 0x555e0bd5eb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:50:53.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:53.788+0000 7f8da9e59740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555e0bd4bc30 0x555e0bd5b460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:50:53.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:53.788+0000 7f8da9e59740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555e0bd564f0 0x555e0bd5b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:50:53.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:53.788+0000 7f8da9e59740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f8da01b2490 con 0x555e0bd4b310 2024-09-10T09:50:53.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:53.788+0000 7f8da9e59740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f8da01b1dc0 con 0x555e0bd4bc30 2024-09-10T09:50:53.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:53.788+0000 7f8da9e59740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f8da01af000 con 0x555e0bd564f0 2024-09-10T09:50:53.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:53.788+0000 7f8da9e59740 10 client.0 _ll_register_callbacks cb 0x7f8d94009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:50:53.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:53.788+0000 7f8d9fe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555e0bd4bc30 0x555e0bd5b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:50:53.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:53.788+0000 7f8da4a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555e0bd4b310 0x555e0bd5eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:50:53.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:53.788+0000 7f8da5400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555e0bd564f0 0x555e0bd5b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:50:53.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:53.788+0000 7f8d9fe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555e0bd4bc30 0x555e0bd5b460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:56044/0 (socket says 172.21.15.5:56044) 2024-09-10T09:50:53.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:53.788+0000 7f8d9fe00640 1 -- 172.21.15.5:0/3063640543 learned_addr learned my addr 172.21.15.5:0/3063640543 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:50:53.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:53.789+0000 7f8d9fe00640 1 -- 172.21.15.5:0/3063640543 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555e0bd564f0 msgr2=0x555e0bd5b9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:50:53.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:53.789+0000 7f8d9fe00640 1 --2- 172.21.15.5:0/3063640543 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555e0bd564f0 0x555e0bd5b9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:53.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:53.789+0000 7f8d9fe00640 1 -- 172.21.15.5:0/3063640543 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555e0bd4b310 msgr2=0x555e0bd5eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:50:53.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:53.789+0000 7f8d9fe00640 1 --2- 172.21.15.5:0/3063640543 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555e0bd4b310 0x555e0bd5eb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:53.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:53.789+0000 7f8d9fe00640 1 -- 172.21.15.5:0/3063640543 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555e0bd5c1d0 con 0x555e0bd4bc30 2024-09-10T09:50:53.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:53.789+0000 7f8da5400640 1 --2- 172.21.15.5:0/3063640543 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555e0bd564f0 0x555e0bd5b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:50:53.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:53.789+0000 7f8d9fe00640 1 --2- 172.21.15.5:0/3063640543 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555e0bd4bc30 0x555e0bd5b460 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f8d8c0985d0 tx=0x7f8d8c09bd40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a3c39de3f059c36e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:50:53.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:53.789+0000 7f8d86000640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-09-10T09:50:53.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:53.789+0000 7f8d86000640 1 -- 172.21.15.5:0/3063640543 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d8c0b9070 con 0x555e0bd4bc30 2024-09-10T09:50:53.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:53.789+0000 7f8d86000640 1 -- 172.21.15.5:0/3063640543 <== mon.2 v2:172.21.15.102:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f8d8c098270 con 0x555e0bd4bc30 2024-09-10T09:50:53.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:53.789+0000 7f8da9e59740 1 -- 172.21.15.5:0/3063640543 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8d8c0b2070 con 0x555e0bd4bc30 2024-09-10T09:50:53.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:53.789+0000 7f8d86000640 1 -- 172.21.15.5:0/3063640543 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f8d8c0cf440 con 0x555e0bd4bc30 2024-09-10T09:50:53.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:53.789+0000 7f8da9e59740 10 client.27935 fetch_fsmap learned FSMap version 36 2024-09-10T09:50:53.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:53.789+0000 7f8da9e59740 1 -- 172.21.15.5:0/3063640543 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x555e0bdd1e70 con 0x555e0bd4bc30 2024-09-10T09:50:53.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:53.790+0000 7f8d86000640 1 -- 172.21.15.5:0/3063640543 <== mon.2 v2:172.21.15.102:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8d8c0b2070 con 0x555e0bd4bc30 2024-09-10T09:50:53.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:53.790+0000 7f8d86000640 1 -- 172.21.15.5:0/3063640543 <== mon.2 v2:172.21.15.102:3300/0 5 ==== fsmap(e 36) ==== 8370+0+0 (secure 0 0 0) 0x7f8d8c09b050 con 0x555e0bd4bc30 2024-09-10T09:50:53.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:53.790+0000 7f8da9e59740 10 client.27935 fetch_fsmap finished waiting for FSMap version 36 2024-09-10T09:50:53.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:53.790+0000 7f8da9e59740 10 client.27935 resolve_mds: validated gid 24542 aka daemon mds.b 2024-09-10T09:50:53.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:53.790+0000 7f8da9e59740 20 client.27935 populate_metadata read hostname 'smithi005' 2024-09-10T09:50:53.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:53.790+0000 7f8da9e59740 1 --2- 172.21.15.5:0/3063640543 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x555e0bdd2500 0x555e0bdd4920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:50:53.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:53.790+0000 7f8da9e59740 4 client.27935 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "6b75eb2", "release": true, "await": true}] 2024-09-10T09:50:53.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:53.790+0000 7f8da9e59740 1 -- 172.21.15.5:0/3063640543 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "quiesce db", "set_id": "6b75eb2", "release": true, "await": true}) -- 0x555e0bcb90d0 con 0x555e0bdd2500 2024-09-10T09:50:53.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:53.790+0000 7f8da4a00640 1 --2- 172.21.15.5:0/3063640543 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x555e0bdd2500 0x555e0bdd4920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:50:53.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:53.791+0000 7f8da4a00640 1 --2- 172.21.15.5:0/3063640543 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x555e0bdd2500 0x555e0bdd4920 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f982f52d2354f9ba server_cookie=936c8329a47e8942 in_seq=0 out_seq=0 2024-09-10T09:50:53.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:53.791+0000 7f8d86000640 10 client.27935 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T09:50:53.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:53.799+0000 7f8d86000640 1 -- 172.21.15.5:0/3063640543 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x555e0bcb90d0 con 0x555e0bdd2500 2024-09-10T09:50:53.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:53.799+0000 7f8d86000640 10 client.27935 handle_command_reply: tid=0 2024-09-10T09:50:53.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:53.799+0000 7f8da9e59740 1 client.27935 shutdown 2024-09-10T09:50:53.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:53.799+0000 7f8da9e59740 1 -- 172.21.15.5:0/3063640543 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555e0bd4bc30 msgr2=0x555e0bd5b460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:50:53.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:53.799+0000 7f8da9e59740 1 --2- 172.21.15.5:0/3063640543 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555e0bd4bc30 0x555e0bd5b460 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f8d8c0985d0 tx=0x7f8d8c09bd40 comp rx=0 tx=0).stop 2024-09-10T09:50:53.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:53.799+0000 7f8da9e59740 1 -- 172.21.15.5:0/3063640543 shutdown_connections 2024-09-10T09:50:53.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:53.799+0000 7f8da9e59740 1 --2- 172.21.15.5:0/3063640543 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x555e0bdd2500 0x555e0bdd4920 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:53.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:53.799+0000 7f8da9e59740 1 --2- 172.21.15.5:0/3063640543 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555e0bd564f0 0x555e0bd5b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:53.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:53.799+0000 7f8d86000640 0 client.27935 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T09:50:53.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:53.799+0000 7f8da9e59740 1 --2- 172.21.15.5:0/3063640543 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555e0bd4bc30 0x555e0bd5b460 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:53.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:53.799+0000 7f8da9e59740 1 --2- 172.21.15.5:0/3063640543 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555e0bd4b310 0x555e0bd5eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:53.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:53.799+0000 7f8da9e59740 1 -- 172.21.15.5:0/3063640543 >> 172.21.15.5:0/3063640543 conn(0x555e0bd478a0 msgr2=0x555e0bd46aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:50:53.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:53.799+0000 7f8da9e59740 1 -- 172.21.15.5:0/3063640543 shutdown_connections 2024-09-10T09:50:53.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:53.799+0000 7f8da9e59740 1 -- 172.21.15.5:0/3063640543 wait complete. 2024-09-10T09:50:53.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:53.799+0000 7f8da9e59740 20 client.27935 trim_cache size 0 max 16384 2024-09-10T09:50:53.802 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-10T09:50:53.802 INFO:teuthology.orchestra.run.smithi005.stdout: "epoch": 36, 2024-09-10T09:50:53.802 INFO:teuthology.orchestra.run.smithi005.stdout: "leader": 24542, 2024-09-10T09:50:53.802 INFO:teuthology.orchestra.run.smithi005.stdout: "set_version": 44, 2024-09-10T09:50:53.802 INFO:teuthology.orchestra.run.smithi005.stdout: "sets": { 2024-09-10T09:50:53.803 INFO:teuthology.orchestra.run.smithi005.stdout: "6b75eb2": { 2024-09-10T09:50:53.803 INFO:teuthology.orchestra.run.smithi005.stdout: "version": 44, 2024-09-10T09:50:53.803 INFO:teuthology.orchestra.run.smithi005.stdout: "age_ref": 0.0, 2024-09-10T09:50:53.803 INFO:teuthology.orchestra.run.smithi005.stdout: "state": { 2024-09-10T09:50:53.803 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "RELEASED", 2024-09-10T09:50:53.803 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0 2024-09-10T09:50:53.803 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T09:50:53.803 INFO:teuthology.orchestra.run.smithi005.stdout: "timeout": 90.0, 2024-09-10T09:50:53.803 INFO:teuthology.orchestra.run.smithi005.stdout: "expiration": 138.1, 2024-09-10T09:50:53.803 INFO:teuthology.orchestra.run.smithi005.stdout: "members": { 2024-09-10T09:50:53.803 INFO:teuthology.orchestra.run.smithi005.stdout: "file:/": { 2024-09-10T09:50:53.804 INFO:teuthology.orchestra.run.smithi005.stdout: "excluded": false, 2024-09-10T09:50:53.804 INFO:teuthology.orchestra.run.smithi005.stdout: "state": { 2024-09-10T09:50:53.804 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "RELEASED", 2024-09-10T09:50:53.804 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0 2024-09-10T09:50:53.804 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T09:50:53.804 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T09:50:53.804 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T09:50:53.804 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T09:50:53.804 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T09:50:53.804 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-10T09:50:53.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:53.800+0000 7f8da7400640 1 -- 172.21.15.5:0/404977917 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f8d880a2c00 msgr2=0x7f8d880a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:50:53.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:53.800+0000 7f8da7400640 1 --2- 172.21.15.5:0/404977917 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f8d880a2c00 0x7f8d880a50c0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f8d94000c00 tx=0x7f8d94005b50 comp rx=0 tx=0).stop 2024-09-10T09:50:53.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:53.800+0000 7f8da7400640 1 -- 172.21.15.5:0/404977917 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8da0074ee0 msgr2=0x7f8da019abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:50:53.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:53.800+0000 7f8da7400640 1 --2- 172.21.15.5:0/404977917 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8da0074ee0 0x7f8da019abd0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f8d8c00e9f0 tx=0x7f8d8c00edf0 comp rx=0 tx=0).stop 2024-09-10T09:50:53.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:53.800+0000 7f8da7400640 1 -- 172.21.15.5:0/404977917 shutdown_connections 2024-09-10T09:50:53.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:53.800+0000 7f8da7400640 1 --2- 172.21.15.5:0/404977917 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f8d880a2c00 0x7f8d880a50c0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:53.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:53.800+0000 7f8da7400640 1 --2- 172.21.15.5:0/404977917 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8da010e430 0x7f8da019ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:53.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:53.800+0000 7f8da7400640 1 --2- 172.21.15.5:0/404977917 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8da0074ee0 0x7f8da019abd0 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:53.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:53.800+0000 7f8da7400640 1 --2- 172.21.15.5:0/404977917 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8da00745c0 0x7f8da019a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:50:53.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:53.800+0000 7f8da7400640 1 -- 172.21.15.5:0/404977917 >> 172.21.15.5:0/404977917 conn(0x7f8da00fd8b0 msgr2=0x7f8da00fe510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:50:53.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:53.800+0000 7f8da7400640 1 -- 172.21.15.5:0/404977917 shutdown_connections 2024-09-10T09:50:53.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:50:53.800+0000 7f8da7400640 1 -- 172.21.15.5:0/404977917 wait complete. 2024-09-10T09:50:53.974 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:50:53 smithi005 ceph-mon[30350]: pgmap v769: 129 pgs: 129 active+clean; 292 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 197 KiB/s rd, 1.4 MiB/s wr, 70 op/s 2024-09-10T09:50:54.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:50:53 smithi159 ceph-mon[35513]: pgmap v769: 129 pgs: 129 active+clean; 292 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 197 KiB/s rd, 1.4 MiB/s wr, 70 op/s 2024-09-10T09:50:54.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:50:53 smithi102 ceph-mon[35442]: pgmap v769: 129 pgs: 129 active+clean; 292 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 197 KiB/s rd, 1.4 MiB/s wr, 70 op/s 2024-09-10T09:50:54.146 INFO:tasks.workunit.client.0.smithi005.stdout: 1 786177 6.25 MB/sec execute 515 sec latency 16047.433 ms 2024-09-10T09:50:54.725 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '6b75eb2', total seconds elapsed: 22.2 2024-09-10T09:50:54.936 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:50:54 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:50:55.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:50:54 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:50:55.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:50:54 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:50:55.146 INFO:tasks.workunit.client.0.smithi005.stdout: 1 786177 6.24 MB/sec execute 516 sec latency 1150.960 ms 2024-09-10T09:50:56.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:50:55 smithi005 ceph-mon[30350]: pgmap v770: 129 pgs: 129 active+clean; 291 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 1.5 MiB/s wr, 71 op/s 2024-09-10T09:50:56.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:50:55 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:50:56.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:50:55 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:50:56.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:50:55 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:50:56.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:50:55 smithi159 ceph-mon[35513]: pgmap v770: 129 pgs: 129 active+clean; 291 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 1.5 MiB/s wr, 71 op/s 2024-09-10T09:50:56.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:50:55 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:50:56.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:50:55 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:50:56.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:50:55 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:50:56.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:50:55 smithi102 ceph-mon[35442]: pgmap v770: 129 pgs: 129 active+clean; 291 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 1.5 MiB/s wr, 71 op/s 2024-09-10T09:50:56.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:50:55 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:50:56.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:50:55 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:50:56.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:50:55 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:50:56.146 INFO:tasks.workunit.client.0.smithi005.stdout: 1 788235 6.25 MB/sec execute 517 sec latency 1332.526 ms 2024-09-10T09:50:57.146 INFO:tasks.workunit.client.0.smithi005.stdout: 1 789788 6.25 MB/sec execute 518 sec latency 36.081 ms 2024-09-10T09:50:58.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:50:57 smithi005 ceph-mon[30350]: pgmap v771: 129 pgs: 129 active+clean; 292 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 1.5 MiB/s wr, 72 op/s 2024-09-10T09:50:58.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:50:57 smithi159 ceph-mon[35513]: pgmap v771: 129 pgs: 129 active+clean; 292 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 1.5 MiB/s wr, 72 op/s 2024-09-10T09:50:58.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:50:57 smithi102 ceph-mon[35442]: pgmap v771: 129 pgs: 129 active+clean; 292 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 1.5 MiB/s wr, 72 op/s 2024-09-10T09:50:58.146 INFO:tasks.workunit.client.0.smithi005.stdout: 1 792223 6.27 MB/sec execute 519 sec latency 10.507 ms 2024-09-10T09:50:59.146 INFO:tasks.workunit.client.0.smithi005.stdout: 1 794262 6.27 MB/sec execute 520 sec latency 77.752 ms 2024-09-10T09:50:59.961 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:50:59 smithi005 ceph-mon[30350]: pgmap v772: 129 pgs: 129 active+clean; 288 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 296 KiB/s rd, 1.6 MiB/s wr, 73 op/s 2024-09-10T09:51:00.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:50:59 smithi159 ceph-mon[35513]: pgmap v772: 129 pgs: 129 active+clean; 288 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 296 KiB/s rd, 1.6 MiB/s wr, 73 op/s 2024-09-10T09:51:00.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:50:59 smithi102 ceph-mon[35442]: pgmap v772: 129 pgs: 129 active+clean; 288 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 296 KiB/s rd, 1.6 MiB/s wr, 73 op/s 2024-09-10T09:51:00.146 INFO:tasks.workunit.client.0.smithi005.stdout: 1 794262 6.26 MB/sec execute 521 sec latency 1077.816 ms 2024-09-10T09:51:01.146 INFO:tasks.workunit.client.0.smithi005.stdout: 1 795916 6.27 MB/sec execute 522 sec latency 1212.436 ms 2024-09-10T09:51:02.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:51:01 smithi005 ceph-mon[30350]: pgmap v773: 129 pgs: 129 active+clean; 291 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.0 MiB/s wr, 144 op/s 2024-09-10T09:51:02.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:51:01 smithi159 ceph-mon[35513]: pgmap v773: 129 pgs: 129 active+clean; 291 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.0 MiB/s wr, 144 op/s 2024-09-10T09:51:02.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:51:01 smithi102 ceph-mon[35442]: pgmap v773: 129 pgs: 129 active+clean; 291 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.0 MiB/s wr, 144 op/s 2024-09-10T09:51:02.146 INFO:tasks.workunit.client.0.smithi005.stdout: 1 798293 6.28 MB/sec execute 523 sec latency 15.179 ms 2024-09-10T09:51:03.147 INFO:tasks.workunit.client.0.smithi005.stdout: 1 799855 6.28 MB/sec execute 524 sec latency 25.697 ms 2024-09-10T09:51:04.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:51:03 smithi005 ceph-mon[30350]: pgmap v774: 129 pgs: 129 active+clean; 291 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.9 MiB/s wr, 132 op/s 2024-09-10T09:51:04.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:51:03 smithi159 ceph-mon[35513]: pgmap v774: 129 pgs: 129 active+clean; 291 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.9 MiB/s wr, 132 op/s 2024-09-10T09:51:04.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:51:03 smithi102 ceph-mon[35442]: pgmap v774: 129 pgs: 129 active+clean; 291 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.9 MiB/s wr, 132 op/s 2024-09-10T09:51:04.146 INFO:tasks.workunit.client.0.smithi005.stdout: 1 802530 6.30 MB/sec execute 525 sec latency 13.951 ms 2024-09-10T09:51:05.147 INFO:tasks.workunit.client.0.smithi005.stdout: 1 802625 6.29 MB/sec execute 526 sec latency 656.567 ms 2024-09-10T09:51:06.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:51:05 smithi005 ceph-mon[30350]: pgmap v775: 129 pgs: 129 active+clean; 291 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.9 MiB/s wr, 137 op/s 2024-09-10T09:51:06.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:51:05 smithi159 ceph-mon[35513]: pgmap v775: 129 pgs: 129 active+clean; 291 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.9 MiB/s wr, 137 op/s 2024-09-10T09:51:06.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:51:05 smithi102 ceph-mon[35442]: pgmap v775: 129 pgs: 129 active+clean; 291 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.9 MiB/s wr, 137 op/s 2024-09-10T09:51:06.146 INFO:tasks.workunit.client.0.smithi005.stdout: 1 803515 6.28 MB/sec execute 527 sec latency 512.107 ms 2024-09-10T09:51:07.147 INFO:tasks.workunit.client.0.smithi005.stdout: 1 806087 6.30 MB/sec execute 528 sec latency 14.554 ms 2024-09-10T09:51:08.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:51:07 smithi005 ceph-mon[30350]: pgmap v776: 129 pgs: 129 active+clean; 293 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.1 MiB/s wr, 190 op/s 2024-09-10T09:51:08.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:51:07 smithi159 ceph-mon[35513]: pgmap v776: 129 pgs: 129 active+clean; 293 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.1 MiB/s wr, 190 op/s 2024-09-10T09:51:08.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:51:07 smithi102 ceph-mon[35442]: pgmap v776: 129 pgs: 129 active+clean; 293 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.1 MiB/s wr, 190 op/s 2024-09-10T09:51:08.147 INFO:tasks.workunit.client.0.smithi005.stdout: 1 807718 6.30 MB/sec execute 529 sec latency 24.353 ms 2024-09-10T09:51:09.147 INFO:tasks.workunit.client.0.smithi005.stdout: 1 809984 6.32 MB/sec execute 530 sec latency 15.581 ms 2024-09-10T09:51:09.771 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:51:09 smithi005 ceph-mon[30350]: pgmap v777: 129 pgs: 129 active+clean; 293 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.1 MiB/s wr, 188 op/s 2024-09-10T09:51:10.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:51:09 smithi159 ceph-mon[35513]: pgmap v777: 129 pgs: 129 active+clean; 293 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.1 MiB/s wr, 188 op/s 2024-09-10T09:51:10.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:51:09 smithi102 ceph-mon[35442]: pgmap v777: 129 pgs: 129 active+clean; 293 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.1 MiB/s wr, 188 op/s 2024-09-10T09:51:10.147 INFO:tasks.workunit.client.0.smithi005.stdout: 1 810481 6.31 MB/sec execute 531 sec latency 692.948 ms 2024-09-10T09:51:11.147 INFO:tasks.workunit.client.0.smithi005.stdout: 1 813308 6.33 MB/sec execute 532 sec latency 17.722 ms 2024-09-10T09:51:12.011 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T09:51:12.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:51:11 smithi005 ceph-mon[30350]: pgmap v778: 129 pgs: 129 active+clean; 296 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.5 MiB/s wr, 242 op/s 2024-09-10T09:51:12.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:51:11 smithi159 ceph-mon[35513]: pgmap v778: 129 pgs: 129 active+clean; 296 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.5 MiB/s wr, 242 op/s 2024-09-10T09:51:12.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:51:11 smithi102 ceph-mon[35442]: pgmap v778: 129 pgs: 129 active+clean; 296 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.5 MiB/s wr, 242 op/s 2024-09-10T09:51:12.147 INFO:tasks.workunit.client.0.smithi005.stdout: 1 815439 6.33 MB/sec execute 533 sec latency 20.989 ms 2024-09-10T09:51:12.423 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:51:13.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:13.058+0000 7f0d84000640 1 -- 172.21.15.5:0/1390400431 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0d7c1049c0 msgr2=0x7f0d7c108a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:51:13.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:13.058+0000 7f0d84000640 1 --2- 172.21.15.5:0/1390400431 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0d7c1049c0 0x7f0d7c108a10 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f0d68009900 tx=0x7f0d6802a3f0 comp rx=0 tx=0).stop 2024-09-10T09:51:13.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:13.059+0000 7f0d84000640 1 -- 172.21.15.5:0/1390400431 shutdown_connections 2024-09-10T09:51:13.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:13.059+0000 7f0d84000640 1 --2- 172.21.15.5:0/1390400431 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0d7c109140 0x7f0d7c110cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:13.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:13.059+0000 7f0d84000640 1 --2- 172.21.15.5:0/1390400431 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0d7c1049c0 0x7f0d7c108a10 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:13.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:13.059+0000 7f0d84000640 1 --2- 172.21.15.5:0/1390400431 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0d7c104010 0x7f0d7c1043f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:13.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:13.059+0000 7f0d84000640 1 -- 172.21.15.5:0/1390400431 >> 172.21.15.5:0/1390400431 conn(0x7f0d7c0ffa50 msgr2=0x7f0d7c101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:51:13.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:13.059+0000 7f0d84000640 1 -- 172.21.15.5:0/1390400431 shutdown_connections 2024-09-10T09:51:13.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:13.059+0000 7f0d84000640 1 -- 172.21.15.5:0/1390400431 wait complete. 2024-09-10T09:51:13.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:13.060+0000 7f0d84000640 1 Processor -- start 2024-09-10T09:51:13.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:13.060+0000 7f0d84000640 1 -- start start 2024-09-10T09:51:13.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:13.060+0000 7f0d84000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0d7c104010 0x7f0d7c073a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:51:13.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:13.060+0000 7f0d84000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0d7c1049c0 0x7f0d7c06cb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:51:13.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:13.061+0000 7f0d84000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0d7c109140 0x7f0d7c06d040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:51:13.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:13.061+0000 7f0d84000640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f0d7c114190 con 0x7f0d7c109140 2024-09-10T09:51:13.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:13.061+0000 7f0d84000640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f0d7c114010 con 0x7f0d7c104010 2024-09-10T09:51:13.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:13.061+0000 7f0d84000640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f0d7c114310 con 0x7f0d7c1049c0 2024-09-10T09:51:13.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:13.061+0000 7f0d81600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0d7c104010 0x7f0d7c073a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:51:13.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:13.061+0000 7f0d81600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0d7c104010 0x7f0d7c073a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:33494/0 (socket says 172.21.15.5:33494) 2024-09-10T09:51:13.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:13.061+0000 7f0d81600640 1 -- 172.21.15.5:0/241025268 learned_addr learned my addr 172.21.15.5:0/241025268 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:51:13.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:13.061+0000 7f0d80c00640 1 --2- 172.21.15.5:0/241025268 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0d7c1049c0 0x7f0d7c06cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:51:13.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:13.061+0000 7f0d82000640 1 --2- 172.21.15.5:0/241025268 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0d7c109140 0x7f0d7c06d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:51:13.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:13.061+0000 7f0d81600640 1 -- 172.21.15.5:0/241025268 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0d7c1049c0 msgr2=0x7f0d7c06cb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:51:13.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:13.061+0000 7f0d81600640 1 --2- 172.21.15.5:0/241025268 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0d7c1049c0 0x7f0d7c06cb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:13.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:13.061+0000 7f0d81600640 1 -- 172.21.15.5:0/241025268 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0d7c109140 msgr2=0x7f0d7c06d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:51:13.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:13.061+0000 7f0d81600640 1 --2- 172.21.15.5:0/241025268 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0d7c109140 0x7f0d7c06d040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:13.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:13.062+0000 7f0d81600640 1 -- 172.21.15.5:0/241025268 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d7c06d680 con 0x7f0d7c104010 2024-09-10T09:51:13.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:13.062+0000 7f0d81600640 1 --2- 172.21.15.5:0/241025268 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0d7c104010 0x7f0d7c073a70 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f0d7000d980 tx=0x7f0d7000de40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:51:13.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:13.062+0000 7f0d7a000640 1 -- 172.21.15.5:0/241025268 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d70017030 con 0x7f0d7c104010 2024-09-10T09:51:13.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:13.062+0000 7f0d7a000640 1 -- 172.21.15.5:0/241025268 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0d70004020 con 0x7f0d7c104010 2024-09-10T09:51:13.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:13.062+0000 7f0d84000640 1 -- 172.21.15.5:0/241025268 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d7c0715a0 con 0x7f0d7c104010 2024-09-10T09:51:13.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:13.062+0000 7f0d7a000640 1 -- 172.21.15.5:0/241025268 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d700047a0 con 0x7f0d7c104010 2024-09-10T09:51:13.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:13.062+0000 7f0d84000640 1 -- 172.21.15.5:0/241025268 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d7c071b00 con 0x7f0d7c104010 2024-09-10T09:51:13.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:13.066+0000 7f0d7a000640 1 -- 172.21.15.5:0/241025268 <== mon.2 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f0d70009a30 con 0x7f0d7c104010 2024-09-10T09:51:13.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:13.066+0000 7f0d86c54740 1 Processor -- start 2024-09-10T09:51:13.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:13.066+0000 7f0d86c54740 1 -- start start 2024-09-10T09:51:13.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:13.067+0000 7f0d86c54740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f90b27ab90 0x55f90b27af70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:51:13.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:13.067+0000 7f0d86c54740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f90b27b4b0 0x55f90b285830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:51:13.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:13.067+0000 7f0d86c54740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f90b285d70 0x55f90b288160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:51:13.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:13.067+0000 7f0d86c54740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55f90b1e13e0 con 0x55f90b285d70 2024-09-10T09:51:13.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:13.067+0000 7f0d86c54740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55f90b188d80 con 0x55f90b27ab90 2024-09-10T09:51:13.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:13.067+0000 7f0d86c54740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55f90b1dd3a0 con 0x55f90b27b4b0 2024-09-10T09:51:13.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:13.067+0000 7f0d82000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f90b27ab90 0x55f90b27af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:51:13.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:13.067+0000 7f0d82000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f90b27ab90 0x55f90b27af70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:33500/0 (socket says 172.21.15.5:33500) 2024-09-10T09:51:13.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:13.067+0000 7f0d80c00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f90b285d70 0x55f90b288160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:51:13.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:13.067+0000 7f0d82000640 1 -- 172.21.15.5:0/2021985257 learned_addr learned my addr 172.21.15.5:0/2021985257 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:51:13.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:13.067+0000 7f0d81600640 1 --2- 172.21.15.5:0/2021985257 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f90b27b4b0 0x55f90b285830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:51:13.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:13.068+0000 7f0d82000640 1 -- 172.21.15.5:0/2021985257 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f90b27b4b0 msgr2=0x55f90b285830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:51:13.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:13.068+0000 7f0d82000640 1 --2- 172.21.15.5:0/2021985257 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f90b27b4b0 0x55f90b285830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:13.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:13.068+0000 7f0d82000640 1 -- 172.21.15.5:0/2021985257 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f90b285d70 msgr2=0x55f90b288160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:51:13.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:13.068+0000 7f0d82000640 1 --2- 172.21.15.5:0/2021985257 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f90b285d70 0x55f90b288160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:13.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:13.068+0000 7f0d82000640 1 -- 172.21.15.5:0/2021985257 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f90b1a7ae0 con 0x55f90b27ab90 2024-09-10T09:51:13.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:13.068+0000 7f0d7a000640 1 --2- 172.21.15.5:0/241025268 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f0d640a2c00 0x7f0d640a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:51:13.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:13.068+0000 7f0d82000640 1 --2- 172.21.15.5:0/2021985257 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f90b27ab90 0x55f90b27af70 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f0d74009f20 tx=0x7f0d74010860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=376a95f68f8d38e5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:51:13.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:13.068+0000 7f0d7a000640 1 -- 172.21.15.5:0/241025268 <== mon.2 v2:172.21.15.102:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f0d700ceee0 con 0x7f0d7c104010 2024-09-10T09:51:13.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:13.068+0000 7f0d81600640 1 --2- 172.21.15.5:0/241025268 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f0d640a2c00 0x7f0d640a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:51:13.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:13.069+0000 7f0d6e600640 1 -- 172.21.15.5:0/2021985257 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d74011700 con 0x55f90b27ab90 2024-09-10T09:51:13.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:13.069+0000 7f0d6e600640 1 -- 172.21.15.5:0/2021985257 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0d74011d80 con 0x55f90b27ab90 2024-09-10T09:51:13.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:13.069+0000 7f0d6e600640 1 -- 172.21.15.5:0/2021985257 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d7401a380 con 0x55f90b27ab90 2024-09-10T09:51:13.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:13.070+0000 7f0d86c54740 1 -- 172.21.15.5:0/2021985257 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f90b27ab90 msgr2=0x55f90b27af70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:51:13.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:13.070+0000 7f0d81600640 1 --2- 172.21.15.5:0/241025268 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f0d640a2c00 0x7f0d640a50c0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f0d7009b280 tx=0x7f0d700d2920 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:51:13.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:13.070+0000 7f0d86c54740 1 --2- 172.21.15.5:0/2021985257 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f90b27ab90 0x55f90b27af70 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f0d74009f20 tx=0x7f0d74010860 comp rx=0 tx=0).stop 2024-09-10T09:51:13.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:13.070+0000 7f0d86c54740 1 -- 172.21.15.5:0/2021985257 shutdown_connections 2024-09-10T09:51:13.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:13.070+0000 7f0d86c54740 1 --2- 172.21.15.5:0/2021985257 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f90b285d70 0x55f90b288160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:13.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:13.070+0000 7f0d86c54740 1 --2- 172.21.15.5:0/2021985257 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f90b27b4b0 0x55f90b285830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:13.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:13.070+0000 7f0d86c54740 1 --2- 172.21.15.5:0/2021985257 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f90b27ab90 0x55f90b27af70 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:13.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:13.070+0000 7f0d86c54740 1 -- 172.21.15.5:0/2021985257 >> 172.21.15.5:0/2021985257 conn(0x55f90b277120 msgr2=0x55f90b277500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:51:13.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:13.070+0000 7f0d86c54740 1 -- 172.21.15.5:0/2021985257 shutdown_connections 2024-09-10T09:51:13.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:13.070+0000 7f0d86c54740 1 -- 172.21.15.5:0/2021985257 wait complete. 2024-09-10T09:51:13.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:13.071+0000 7f0d86c54740 1 -- start start 2024-09-10T09:51:13.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:13.071+0000 7f0d86c54740 1 Processor -- start 2024-09-10T09:51:13.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:13.072+0000 7f0d86c54740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f90b27ab90 0x55f90b282d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:51:13.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:13.072+0000 7f0d86c54740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f90b27b4b0 0x55f90b283270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:51:13.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:13.072+0000 7f0d86c54740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f90b285d70 0x55f90b2837b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:51:13.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:13.072+0000 7f0d86c54740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55f90b2889e0 con 0x55f90b27ab90 2024-09-10T09:51:13.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:13.072+0000 7f0d86c54740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55f90b19c0f0 con 0x55f90b27b4b0 2024-09-10T09:51:13.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:13.072+0000 7f0d86c54740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55f90b288b60 con 0x55f90b285d70 2024-09-10T09:51:13.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:13.072+0000 7f0d86c54740 10 client.0 _ll_register_callbacks cb 0x55f90b18d9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:51:13.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:13.072+0000 7f0d82000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f90b27ab90 0x55f90b282d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:51:13.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:13.072+0000 7f0d82000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f90b27ab90 0x55f90b282d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46546/0 (socket says 172.21.15.5:46546) 2024-09-10T09:51:13.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:13.072+0000 7f0d82000640 1 -- 172.21.15.5:0/2839566298 learned_addr learned my addr 172.21.15.5:0/2839566298 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:51:13.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:13.072+0000 7f0d82000640 1 --2- 172.21.15.5:0/2839566298 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f90b285d70 0x55f90b2837b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:51:13.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:13.072+0000 7f0d80c00640 1 --2- 172.21.15.5:0/2839566298 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f90b27b4b0 0x55f90b283270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:51:13.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:13.073+0000 7f0d80c00640 1 -- 172.21.15.5:0/2839566298 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f90b285d70 msgr2=0x55f90b2837b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:51:13.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:13.073+0000 7f0d80c00640 1 --2- 172.21.15.5:0/2839566298 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f90b285d70 0x55f90b2837b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:13.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:13.073+0000 7f0d80c00640 1 -- 172.21.15.5:0/2839566298 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f90b27ab90 msgr2=0x55f90b282d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:51:13.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:13.073+0000 7f0d80c00640 1 --2- 172.21.15.5:0/2839566298 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f90b27ab90 0x55f90b282d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:13.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:13.073+0000 7f0d80c00640 1 -- 172.21.15.5:0/2839566298 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f90af523c0 con 0x55f90b27b4b0 2024-09-10T09:51:13.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:13.073+0000 7f0d82000640 1 --2- 172.21.15.5:0/2839566298 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f90b27ab90 0x55f90b282d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:51:13.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:13.073+0000 7f0d80c00640 1 --2- 172.21.15.5:0/2839566298 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f90b27b4b0 0x55f90b283270 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f0d6802a9e0 tx=0x7f0d68004440 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dba42461fc141c1d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:51:13.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:13.073+0000 7f0d62a00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-09-10T09:51:13.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:13.074+0000 7f0d62a00640 1 -- 172.21.15.5:0/2839566298 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d68035040 con 0x55f90b27b4b0 2024-09-10T09:51:13.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:13.074+0000 7f0d62a00640 1 -- 172.21.15.5:0/2839566298 <== mon.2 v2:172.21.15.102:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f0d68005450 con 0x55f90b27b4b0 2024-09-10T09:51:13.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:13.074+0000 7f0d86c54740 1 -- 172.21.15.5:0/2839566298 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0d74011d80 con 0x55f90b27b4b0 2024-09-10T09:51:13.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:13.074+0000 7f0d62a00640 1 -- 172.21.15.5:0/2839566298 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f0d6804f050 con 0x55f90b27b4b0 2024-09-10T09:51:13.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:13.074+0000 7f0d86c54740 10 client.27947 fetch_fsmap learned FSMap version 36 2024-09-10T09:51:13.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:13.074+0000 7f0d86c54740 1 -- 172.21.15.5:0/2839566298 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55f90b29cf60 con 0x55f90b27b4b0 2024-09-10T09:51:13.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:13.074+0000 7f0d62a00640 1 -- 172.21.15.5:0/2839566298 <== mon.2 v2:172.21.15.102:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0d74011d80 con 0x55f90b27b4b0 2024-09-10T09:51:13.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:13.074+0000 7f0d62a00640 1 -- 172.21.15.5:0/2839566298 <== mon.2 v2:172.21.15.102:3300/0 5 ==== fsmap(e 36) ==== 8370+0+0 (secure 0 0 0) 0x7f0d68003a30 con 0x55f90b27b4b0 2024-09-10T09:51:13.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:13.075+0000 7f0d86c54740 10 client.27947 fetch_fsmap finished waiting for FSMap version 36 2024-09-10T09:51:13.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:13.075+0000 7f0d86c54740 10 client.27947 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T09:51:13.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:13.075+0000 7f0d86c54740 20 client.27947 populate_metadata read hostname 'smithi005' 2024-09-10T09:51:13.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:13.075+0000 7f0d86c54740 1 --2- 172.21.15.5:0/2839566298 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55f90b29e420 0x55f90b2a0810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:51:13.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:13.075+0000 7f0d86c54740 4 client.27947 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:51:13.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:13.075+0000 7f0d86c54740 1 -- 172.21.15.5:0/2839566298 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f90b1ee950 con 0x55f90b29e420 2024-09-10T09:51:13.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:13.075+0000 7f0d82000640 1 --2- 172.21.15.5:0/2839566298 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55f90b29e420 0x55f90b2a0810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:51:13.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:13.075+0000 7f0d82000640 1 --2- 172.21.15.5:0/2839566298 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55f90b29e420 0x55f90b2a0810 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f70115d0747dcf45 server_cookie=f3ad5bb7a60f35a1 in_seq=0 out_seq=0 2024-09-10T09:51:13.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:13.075+0000 7f0d62a00640 10 client.27947 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T09:51:13.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:13.076+0000 7f0d62a00640 1 -- 172.21.15.5:0/2839566298 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f90b1ee950 con 0x55f90b29e420 2024-09-10T09:51:13.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:13.076+0000 7f0d62a00640 10 client.27947 handle_command_reply: tid=0 2024-09-10T09:51:13.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:13.076+0000 7f0d86c54740 1 client.27947 shutdown 2024-09-10T09:51:13.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:13.077+0000 7f0d86c54740 1 -- 172.21.15.5:0/2839566298 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f90b27b4b0 msgr2=0x55f90b283270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:51:13.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:13.077+0000 7f0d86c54740 1 --2- 172.21.15.5:0/2839566298 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f90b27b4b0 0x55f90b283270 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f0d6802a9e0 tx=0x7f0d68004440 comp rx=0 tx=0).stop 2024-09-10T09:51:13.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:13.077+0000 7f0d86c54740 1 -- 172.21.15.5:0/2839566298 shutdown_connections 2024-09-10T09:51:13.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:13.077+0000 7f0d86c54740 1 --2- 172.21.15.5:0/2839566298 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55f90b29e420 0x55f90b2a0810 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:13.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:13.077+0000 7f0d86c54740 1 --2- 172.21.15.5:0/2839566298 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f90b285d70 0x55f90b2837b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:13.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:13.077+0000 7f0d62a00640 0 client.27947 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T09:51:13.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:13.077+0000 7f0d86c54740 1 --2- 172.21.15.5:0/2839566298 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f90b27b4b0 0x55f90b283270 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:13.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:13.077+0000 7f0d86c54740 1 --2- 172.21.15.5:0/2839566298 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f90b27ab90 0x55f90b282d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:13.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:13.077+0000 7f0d86c54740 1 -- 172.21.15.5:0/2839566298 >> 172.21.15.5:0/2839566298 conn(0x55f90b277120 msgr2=0x55f90b27ce00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:51:13.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:13.077+0000 7f0d86c54740 1 -- 172.21.15.5:0/2839566298 shutdown_connections 2024-09-10T09:51:13.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:13.077+0000 7f0d86c54740 1 -- 172.21.15.5:0/2839566298 wait complete. 2024-09-10T09:51:13.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:13.077+0000 7f0d86c54740 20 client.27947 trim_cache size 0 max 16384 2024-09-10T09:51:13.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:13.098+0000 7f0d86c54740 1 Processor -- start 2024-09-10T09:51:13.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:13.098+0000 7f0d86c54740 1 -- start start 2024-09-10T09:51:13.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:13.098+0000 7f0d86c54740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f90b27ab90 0x55f90b284ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:51:13.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:13.098+0000 7f0d86c54740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f90b285d70 0x55f90b2a60e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:51:13.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:13.098+0000 7f0d86c54740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f90b2851e0 0x55f90b2a6620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:51:13.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:13.098+0000 7f0d86c54740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f0d7c1b54b0 con 0x55f90b285d70 2024-09-10T09:51:13.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:13.098+0000 7f0d86c54740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f0d7c1b6ff0 con 0x55f90b27ab90 2024-09-10T09:51:13.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:13.098+0000 7f0d86c54740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f0d7c1b5b80 con 0x55f90b2851e0 2024-09-10T09:51:13.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:13.099+0000 7f0d82000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f90b27ab90 0x55f90b284ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:51:13.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:13.099+0000 7f0d80c00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f90b285d70 0x55f90b2a60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:51:13.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:13.099+0000 7f0d82000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f90b27ab90 0x55f90b284ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:33524/0 (socket says 172.21.15.5:33524) 2024-09-10T09:51:13.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:13.099+0000 7f0d82000640 1 -- 172.21.15.5:0/2111485579 learned_addr learned my addr 172.21.15.5:0/2111485579 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:51:13.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:13.099+0000 7f0d82000640 1 --2- 172.21.15.5:0/2111485579 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f90b2851e0 0x55f90b2a6620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:51:13.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:13.099+0000 7f0d80c00640 1 -- 172.21.15.5:0/2111485579 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f90b2851e0 msgr2=0x55f90b2a6620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:51:13.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:13.099+0000 7f0d80c00640 1 --2- 172.21.15.5:0/2111485579 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f90b2851e0 0x55f90b2a6620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:13.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:13.099+0000 7f0d80c00640 1 -- 172.21.15.5:0/2111485579 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f90b27ab90 msgr2=0x55f90b284ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:51:13.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:13.099+0000 7f0d80c00640 1 --2- 172.21.15.5:0/2111485579 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f90b27ab90 0x55f90b284ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:13.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:13.099+0000 7f0d80c00640 1 -- 172.21.15.5:0/2111485579 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f90b277db0 con 0x55f90b285d70 2024-09-10T09:51:13.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:13.099+0000 7f0d80c00640 1 --2- 172.21.15.5:0/2111485579 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f90b285d70 0x55f90b2a60e0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x55f90b2842e0 tx=0x7f0d6802a3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=edbdd183db18e228 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:51:13.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:13.100+0000 7f0d6dc00640 1 -- 172.21.15.5:0/2111485579 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d68059070 con 0x55f90b285d70 2024-09-10T09:51:13.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:13.100+0000 7f0d6dc00640 1 -- 172.21.15.5:0/2111485579 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0d68041070 con 0x55f90b285d70 2024-09-10T09:51:13.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:13.100+0000 7f0d6dc00640 1 -- 172.21.15.5:0/2111485579 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d68035030 con 0x55f90b285d70 2024-09-10T09:51:13.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:13.101+0000 7f0d86c54740 1 -- 172.21.15.5:0/2111485579 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f90b285d70 msgr2=0x55f90b2a60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:51:13.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:13.101+0000 7f0d86c54740 1 --2- 172.21.15.5:0/2111485579 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f90b285d70 0x55f90b2a60e0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x55f90b2842e0 tx=0x7f0d6802a3f0 comp rx=0 tx=0).stop 2024-09-10T09:51:13.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:13.101+0000 7f0d86c54740 1 -- 172.21.15.5:0/2111485579 shutdown_connections 2024-09-10T09:51:13.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:13.101+0000 7f0d86c54740 1 --2- 172.21.15.5:0/2111485579 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f90b2851e0 0x55f90b2a6620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:13.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:13.101+0000 7f0d86c54740 1 --2- 172.21.15.5:0/2111485579 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f90b285d70 0x55f90b2a60e0 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:13.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:13.101+0000 7f0d86c54740 1 --2- 172.21.15.5:0/2111485579 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f90b27ab90 0x55f90b284ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:13.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:13.101+0000 7f0d86c54740 1 -- 172.21.15.5:0/2111485579 >> 172.21.15.5:0/2111485579 conn(0x55f90b277120 msgr2=0x55f90b284590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:51:13.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:13.101+0000 7f0d86c54740 1 -- 172.21.15.5:0/2111485579 shutdown_connections 2024-09-10T09:51:13.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:13.101+0000 7f0d86c54740 1 -- 172.21.15.5:0/2111485579 wait complete. 2024-09-10T09:51:13.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:13.101+0000 7f0d86c54740 1 -- start start 2024-09-10T09:51:13.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:13.102+0000 7f0d86c54740 1 Processor -- start 2024-09-10T09:51:13.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:13.102+0000 7f0d86c54740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f90b27ab90 0x55f90b296640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:51:13.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:13.102+0000 7f0d86c54740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f90b285d70 0x55f90b296b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:51:13.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:13.102+0000 7f0d86c54740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f90b293610 0x55f90b293a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:51:13.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:13.102+0000 7f0d86c54740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f0d7c1b6920 con 0x55f90b293610 2024-09-10T09:51:13.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:13.102+0000 7f0d86c54740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f0d7c1b6250 con 0x55f90b285d70 2024-09-10T09:51:13.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:13.102+0000 7f0d86c54740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f0d7c1b3530 con 0x55f90b27ab90 2024-09-10T09:51:13.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:13.102+0000 7f0d86c54740 10 client.0 _ll_register_callbacks cb 0x7f0d680040d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:51:13.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:13.102+0000 7f0d82000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f90b27ab90 0x55f90b296640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:51:13.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:13.102+0000 7f0d82000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f90b293610 0x55f90b293a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:51:13.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:13.102+0000 7f0d80c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f90b285d70 0x55f90b296b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:51:13.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:13.103+0000 7f0d82000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f90b293610 0x55f90b293a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46560/0 (socket says 172.21.15.5:46560) 2024-09-10T09:51:13.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:13.103+0000 7f0d82000640 1 -- 172.21.15.5:0/1496429638 learned_addr learned my addr 172.21.15.5:0/1496429638 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:51:13.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:13.103+0000 7f0d82000640 1 -- 172.21.15.5:0/1496429638 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f90b27ab90 msgr2=0x55f90b296640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:51:13.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:13.103+0000 7f0d82000640 1 --2- 172.21.15.5:0/1496429638 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f90b27ab90 0x55f90b296640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:13.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:13.103+0000 7f0d82000640 1 -- 172.21.15.5:0/1496429638 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f90b285d70 msgr2=0x55f90b296b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:51:13.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:13.103+0000 7f0d82000640 1 --2- 172.21.15.5:0/1496429638 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f90b285d70 0x55f90b296b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:13.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:13.103+0000 7f0d82000640 1 -- 172.21.15.5:0/1496429638 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f90b293fd0 con 0x55f90b293610 2024-09-10T09:51:13.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:13.103+0000 7f0d80c00640 1 --2- 172.21.15.5:0/1496429638 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f90b285d70 0x55f90b296b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:51:13.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:13.103+0000 7f0d82000640 1 --2- 172.21.15.5:0/1496429638 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f90b293610 0x55f90b293a90 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f0d74005c40 tx=0x7f0d74004f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efdd86ffb2218c88 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:51:13.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:13.103+0000 7f0d62a00640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-10T09:51:13.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:13.104+0000 7f0d62a00640 1 -- 172.21.15.5:0/1496429638 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d74025040 con 0x55f90b293610 2024-09-10T09:51:13.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:13.104+0000 7f0d62a00640 1 -- 172.21.15.5:0/1496429638 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f0d7402e590 con 0x55f90b293610 2024-09-10T09:51:13.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:13.104+0000 7f0d86c54740 1 -- 172.21.15.5:0/1496429638 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0d68041070 con 0x55f90b293610 2024-09-10T09:51:13.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:13.104+0000 7f0d62a00640 1 -- 172.21.15.5:0/1496429638 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f0d7402e790 con 0x55f90b293610 2024-09-10T09:51:13.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:13.104+0000 7f0d86c54740 10 client.17922 fetch_fsmap learned FSMap version 36 2024-09-10T09:51:13.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:13.104+0000 7f0d86c54740 1 -- 172.21.15.5:0/1496429638 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55f90b309c90 con 0x55f90b293610 2024-09-10T09:51:13.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:13.105+0000 7f0d62a00640 1 -- 172.21.15.5:0/1496429638 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0d68041070 con 0x55f90b293610 2024-09-10T09:51:13.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:13.105+0000 7f0d62a00640 1 -- 172.21.15.5:0/1496429638 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 36) ==== 8370+0+0 (secure 0 0 0) 0x7f0d74011c10 con 0x55f90b293610 2024-09-10T09:51:13.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:13.105+0000 7f0d86c54740 10 client.17922 fetch_fsmap finished waiting for FSMap version 36 2024-09-10T09:51:13.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:13.105+0000 7f0d86c54740 10 client.17922 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T09:51:13.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:13.105+0000 7f0d86c54740 20 client.17922 populate_metadata read hostname 'smithi005' 2024-09-10T09:51:13.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:13.105+0000 7f0d86c54740 1 --2- 172.21.15.5:0/1496429638 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55f90b30a200 0x55f90b30c620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:51:13.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:13.105+0000 7f0d86c54740 4 client.17922 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T09:51:13.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:13.105+0000 7f0d86c54740 1 -- 172.21.15.5:0/1496429638 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55f90b1e8e60 con 0x55f90b30a200 2024-09-10T09:51:13.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:13.105+0000 7f0d80c00640 1 --2- 172.21.15.5:0/1496429638 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55f90b30a200 0x55f90b30c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:51:13.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:13.106+0000 7f0d80c00640 1 --2- 172.21.15.5:0/1496429638 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55f90b30a200 0x55f90b30c620 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8020b82dc7b6516 server_cookie=73d06ee82302861a in_seq=0 out_seq=0 2024-09-10T09:51:13.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:13.106+0000 7f0d62a00640 10 client.17922 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T09:51:13.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:13.106+0000 7f0d62a00640 1 -- 172.21.15.5:0/1496429638 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55f90b1e8e60 con 0x55f90b30a200 2024-09-10T09:51:13.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:13.106+0000 7f0d62a00640 10 client.17922 handle_command_reply: tid=0 2024-09-10T09:51:13.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:13.106+0000 7f0d86c54740 1 client.17922 shutdown 2024-09-10T09:51:13.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:13.106+0000 7f0d86c54740 1 -- 172.21.15.5:0/1496429638 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f90b293610 msgr2=0x55f90b293a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:51:13.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:13.106+0000 7f0d86c54740 1 --2- 172.21.15.5:0/1496429638 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f90b293610 0x55f90b293a90 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f0d74005c40 tx=0x7f0d74004f90 comp rx=0 tx=0).stop 2024-09-10T09:51:13.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:13.106+0000 7f0d86c54740 1 -- 172.21.15.5:0/1496429638 shutdown_connections 2024-09-10T09:51:13.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:13.106+0000 7f0d86c54740 1 --2- 172.21.15.5:0/1496429638 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55f90b30a200 0x55f90b30c620 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:13.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:13.106+0000 7f0d86c54740 1 --2- 172.21.15.5:0/1496429638 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f90b293610 0x55f90b293a90 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:13.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:13.106+0000 7f0d86c54740 1 --2- 172.21.15.5:0/1496429638 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f90b285d70 0x55f90b296b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:13.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:13.106+0000 7f0d86c54740 1 --2- 172.21.15.5:0/1496429638 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f90b27ab90 0x55f90b296640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:13.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:13.107+0000 7f0d86c54740 1 -- 172.21.15.5:0/1496429638 >> 172.21.15.5:0/1496429638 conn(0x55f90b277120 msgr2=0x55f90b282cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:51:13.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:13.107+0000 7f0d62a00640 0 client.17922 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T09:51:13.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:13.107+0000 7f0d86c54740 1 -- 172.21.15.5:0/1496429638 shutdown_connections 2024-09-10T09:51:13.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:13.107+0000 7f0d86c54740 1 -- 172.21.15.5:0/1496429638 wait complete. 2024-09-10T09:51:13.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:13.107+0000 7f0d86c54740 20 client.17922 trim_cache size 0 max 16384 2024-09-10T09:51:13.112 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-10T09:51:13.112 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "no active scrubs running", 2024-09-10T09:51:13.112 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": {} 2024-09-10T09:51:13.112 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-10T09:51:13.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:13.108+0000 7f0d84000640 1 -- 172.21.15.5:0/241025268 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f0d640a2c00 msgr2=0x7f0d640a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:51:13.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:13.108+0000 7f0d84000640 1 --2- 172.21.15.5:0/241025268 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f0d640a2c00 0x7f0d640a50c0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f0d7009b280 tx=0x7f0d700d2920 comp rx=0 tx=0).stop 2024-09-10T09:51:13.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:13.108+0000 7f0d84000640 1 -- 172.21.15.5:0/241025268 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0d7c104010 msgr2=0x7f0d7c073a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:51:13.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:13.108+0000 7f0d84000640 1 --2- 172.21.15.5:0/241025268 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0d7c104010 0x7f0d7c073a70 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f0d7000d980 tx=0x7f0d7000de40 comp rx=0 tx=0).stop 2024-09-10T09:51:13.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:13.108+0000 7f0d84000640 1 -- 172.21.15.5:0/241025268 shutdown_connections 2024-09-10T09:51:13.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:13.108+0000 7f0d84000640 1 --2- 172.21.15.5:0/241025268 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f0d640a2c00 0x7f0d640a50c0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:13.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:13.108+0000 7f0d84000640 1 --2- 172.21.15.5:0/241025268 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0d7c109140 0x7f0d7c06d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:13.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:13.108+0000 7f0d84000640 1 --2- 172.21.15.5:0/241025268 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0d7c1049c0 0x7f0d7c06cb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:13.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:13.108+0000 7f0d84000640 1 --2- 172.21.15.5:0/241025268 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0d7c104010 0x7f0d7c073a70 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:13.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:13.108+0000 7f0d84000640 1 -- 172.21.15.5:0/241025268 >> 172.21.15.5:0/241025268 conn(0x7f0d7c0ffa50 msgr2=0x7f0d7c100de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:51:13.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:13.108+0000 7f0d84000640 1 -- 172.21.15.5:0/241025268 shutdown_connections 2024-09-10T09:51:13.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:13.108+0000 7f0d84000640 1 -- 172.21.15.5:0/241025268 wait complete. 2024-09-10T09:51:13.147 INFO:tasks.workunit.client.0.smithi005.stdout: 1 817399 6.34 MB/sec execute 534 sec latency 34.599 ms 2024-09-10T09:51:13.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:51:13 smithi005 ceph-mon[30350]: pgmap v779: 129 pgs: 129 active+clean; 296 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.8 MiB/s wr, 171 op/s 2024-09-10T09:51:14.015 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-10T09:51:14.016 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T09:51:14.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:51:13 smithi159 ceph-mon[35513]: pgmap v779: 129 pgs: 129 active+clean; 296 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.8 MiB/s wr, 171 op/s 2024-09-10T09:51:14.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:51:13 smithi102 ceph-mon[35442]: pgmap v779: 129 pgs: 129 active+clean; 296 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.8 MiB/s wr, 171 op/s 2024-09-10T09:51:14.147 INFO:tasks.workunit.client.0.smithi005.stdout: 1 820255 6.36 MB/sec execute 535 sec latency 11.816 ms 2024-09-10T09:51:14.451 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:51:15.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:15.060+0000 7f1991e00640 1 -- 172.21.15.5:0/2260386528 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f198c103e50 msgr2=0x7f198c104230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:51:15.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:15.060+0000 7f1991e00640 1 --2- 172.21.15.5:0/2260386528 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f198c103e50 0x7f198c104230 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f1980009900 tx=0x7f198002a3f0 comp rx=0 tx=0).stop 2024-09-10T09:51:15.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:15.060+0000 7f1991e00640 1 -- 172.21.15.5:0/2260386528 shutdown_connections 2024-09-10T09:51:15.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:15.060+0000 7f1991e00640 1 --2- 172.21.15.5:0/2260386528 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f198c109090 0x7f198c110c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:15.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:15.060+0000 7f1991e00640 1 --2- 172.21.15.5:0/2260386528 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f198c104800 0x7f198c108960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:15.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:15.060+0000 7f1991e00640 1 --2- 172.21.15.5:0/2260386528 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f198c103e50 0x7f198c104230 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:15.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:15.060+0000 7f1991e00640 1 -- 172.21.15.5:0/2260386528 >> 172.21.15.5:0/2260386528 conn(0x7f198c0ff8b0 msgr2=0x7f198c101cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:51:15.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:15.060+0000 7f1991e00640 1 -- 172.21.15.5:0/2260386528 shutdown_connections 2024-09-10T09:51:15.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:15.060+0000 7f1991e00640 1 -- 172.21.15.5:0/2260386528 wait complete. 2024-09-10T09:51:15.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:15.061+0000 7f1991e00640 1 Processor -- start 2024-09-10T09:51:15.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:15.061+0000 7f1991e00640 1 -- start start 2024-09-10T09:51:15.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:15.061+0000 7f1991e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f198c103e50 0x7f198c19a570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:51:15.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:15.061+0000 7f1991e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f198c104800 0x7f198c19aab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:51:15.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:15.061+0000 7f1991e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f198c109090 0x7f198c19ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:51:15.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:15.061+0000 7f1991e00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f198c1140e0 con 0x7f198c103e50 2024-09-10T09:51:15.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:15.061+0000 7f1991e00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f198c113f60 con 0x7f198c104800 2024-09-10T09:51:15.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:15.061+0000 7f1991e00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f198c114260 con 0x7f198c109090 2024-09-10T09:51:15.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:15.062+0000 7f198be00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f198c104800 0x7f198c19aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:51:15.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:15.062+0000 7f198be00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f198c104800 0x7f198c19aab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:33552/0 (socket says 172.21.15.5:33552) 2024-09-10T09:51:15.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:15.062+0000 7f1990a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f198c103e50 0x7f198c19a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:51:15.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:15.062+0000 7f198be00640 1 -- 172.21.15.5:0/1042636047 learned_addr learned my addr 172.21.15.5:0/1042636047 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:51:15.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:15.062+0000 7f1991400640 1 --2- 172.21.15.5:0/1042636047 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f198c109090 0x7f198c19ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:51:15.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:15.062+0000 7f198be00640 1 -- 172.21.15.5:0/1042636047 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f198c109090 msgr2=0x7f198c19ee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:51:15.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:15.062+0000 7f198be00640 1 --2- 172.21.15.5:0/1042636047 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f198c109090 0x7f198c19ee40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:15.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:15.062+0000 7f198be00640 1 -- 172.21.15.5:0/1042636047 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f198c103e50 msgr2=0x7f198c19a570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:51:15.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:15.062+0000 7f198be00640 1 --2- 172.21.15.5:0/1042636047 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f198c103e50 0x7f198c19a570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:15.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:15.062+0000 7f198be00640 1 -- 172.21.15.5:0/1042636047 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f198c19f520 con 0x7f198c104800 2024-09-10T09:51:15.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:15.062+0000 7f1990a00640 1 --2- 172.21.15.5:0/1042636047 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f198c103e50 0x7f198c19a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:51:15.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:15.062+0000 7f198be00640 1 --2- 172.21.15.5:0/1042636047 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f198c104800 0x7f198c19aab0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f197800e9f0 tx=0x7f197800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:51:15.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:15.063+0000 7f1989600640 1 -- 172.21.15.5:0/1042636047 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f197800cd80 con 0x7f198c104800 2024-09-10T09:51:15.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:15.063+0000 7f1989600640 1 -- 172.21.15.5:0/1042636047 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1978010910 con 0x7f198c104800 2024-09-10T09:51:15.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:15.063+0000 7f1989600640 1 -- 172.21.15.5:0/1042636047 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1978018990 con 0x7f198c104800 2024-09-10T09:51:15.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:15.063+0000 7f1991e00640 1 -- 172.21.15.5:0/1042636047 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f198c19f810 con 0x7f198c104800 2024-09-10T09:51:15.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:15.063+0000 7f1991e00640 1 -- 172.21.15.5:0/1042636047 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f198c1a70f0 con 0x7f198c104800 2024-09-10T09:51:15.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:15.064+0000 7f1991e00640 1 -- 172.21.15.5:0/1042636047 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f198c105330 con 0x7f198c104800 2024-09-10T09:51:15.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:15.066+0000 7f1989600640 1 -- 172.21.15.5:0/1042636047 <== mon.2 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f1978010ac0 con 0x7f198c104800 2024-09-10T09:51:15.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:15.068+0000 7f1989600640 1 --2- 172.21.15.5:0/1042636047 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f19740a2c00 0x7f19740a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:51:15.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:15.068+0000 7f1989600640 1 -- 172.21.15.5:0/1042636047 <== mon.2 v2:172.21.15.102:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f19780cfd60 con 0x7f198c104800 2024-09-10T09:51:15.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:15.068+0000 7f1990a00640 1 --2- 172.21.15.5:0/1042636047 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f19740a2c00 0x7f19740a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:51:15.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:15.069+0000 7f1989600640 1 -- 172.21.15.5:0/1042636047 <== mon.2 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f197809c870 con 0x7f198c104800 2024-09-10T09:51:15.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:15.069+0000 7f1990a00640 1 --2- 172.21.15.5:0/1042636047 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f19740a2c00 0x7f19740a50c0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f1980000c00 tx=0x7f1980005a40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:51:15.147 INFO:tasks.workunit.client.0.smithi005.stdout: 1 821961 6.36 MB/sec execute 536 sec latency 27.705 ms 2024-09-10T09:51:15.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:15.309+0000 7f1991e00640 1 -- 172.21.15.5:0/1042636047 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f198c10c270 con 0x7f198c104800 2024-09-10T09:51:15.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:15.310+0000 7f1989600640 1 -- 172.21.15.5:0/1042636047 <== mon.2 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) ==== 94+0+11969 (secure 0 0 0) 0x7f197809c170 con 0x7f198c104800 2024-09-10T09:51:15.312 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-10T09:51:15.312 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 36 2024-09-10T09:51:15.313 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":36,"btime":"2024-09-10T09:48:07:349075+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14454,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/2970594311","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2970594311},{"type":"v1","addr":"172.21.15.5:6835","nonce":2970594311}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14514,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6839/3616502174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":3616502174},{"type":"v1","addr":"172.21.15.5:6839","nonce":3616502174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":36,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T09:35:29.626287+0000","modified":"2024-09-10T09:48:06.970127+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24542,"mds_1":24535,"mds_2":24511,"mds_3":24506,"mds_4":24443},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14520":{"gid":14520,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/2073599842","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":2073599842},{"type":"v1","addr":"172.21.15.5:6841","nonce":2073599842}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"k","rank":4,"incarnation":26,"state":"up:active","state_seq":24,"addr":"172.21.15.102:6833/2457007427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":2457007427},{"type":"v1","addr":"172.21.15.102:6833","nonce":2457007427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/356445159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":356445159},{"type":"v1","addr":"172.21.15.159:6833","nonce":356445159}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6837/2894314507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":2894314507},{"type":"v1","addr":"172.21.15.5:6837","nonce":2894314507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.102:6835/1647549111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":1647549111},{"type":"v1","addr":"172.21.15.102:6835","nonce":1647549111}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6835/952657242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":952657242},{"type":"v1","addr":"172.21.15.159:6835","nonce":952657242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":19,"addr":"172.21.15.102:6837/2689776764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":2689776764},{"type":"v1","addr":"172.21.15.102:6837","nonce":2689776764}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.159:6837/620627639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":620627639},{"type":"v1","addr":"172.21.15.159:6837","nonce":620627639}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24535":{"gid":24535,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":18,"addr":"172.21.15.159:6839/2073739599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":2073739599},{"type":"v1","addr":"172.21.15.159:6839","nonce":2073739599}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24542":{"gid":24542,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.102:6839/335409749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6838","nonce":335409749},{"type":"v1","addr":"172.21.15.102:6839","nonce":335409749}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24542,"qdb_cluster":[24542,24535,24511,24506,24443]},"id":1}]} 2024-09-10T09:51:15.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:15.312+0000 7f1991e00640 1 -- 172.21.15.5:0/1042636047 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f19740a2c00 msgr2=0x7f19740a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:51:15.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:15.312+0000 7f1991e00640 1 --2- 172.21.15.5:0/1042636047 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f19740a2c00 0x7f19740a50c0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f1980000c00 tx=0x7f1980005a40 comp rx=0 tx=0).stop 2024-09-10T09:51:15.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:15.312+0000 7f1991e00640 1 -- 172.21.15.5:0/1042636047 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f198c104800 msgr2=0x7f198c19aab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:51:15.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:15.312+0000 7f1991e00640 1 --2- 172.21.15.5:0/1042636047 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f198c104800 0x7f198c19aab0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f197800e9f0 tx=0x7f197800eeb0 comp rx=0 tx=0).stop 2024-09-10T09:51:15.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:15.312+0000 7f1991e00640 1 -- 172.21.15.5:0/1042636047 shutdown_connections 2024-09-10T09:51:15.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:15.312+0000 7f1991e00640 1 --2- 172.21.15.5:0/1042636047 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f19740a2c00 0x7f19740a50c0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:15.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:15.312+0000 7f1991e00640 1 --2- 172.21.15.5:0/1042636047 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f198c109090 0x7f198c19ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:15.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:15.312+0000 7f1991e00640 1 --2- 172.21.15.5:0/1042636047 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f198c104800 0x7f198c19aab0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:15.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:15.312+0000 7f1991e00640 1 --2- 172.21.15.5:0/1042636047 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f198c103e50 0x7f198c19a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:15.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:15.312+0000 7f1991e00640 1 -- 172.21.15.5:0/1042636047 >> 172.21.15.5:0/1042636047 conn(0x7f198c0ff8b0 msgr2=0x7f198c100b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:51:15.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:15.313+0000 7f1991e00640 1 -- 172.21.15.5:0/1042636047 shutdown_connections 2024-09-10T09:51:15.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:15.313+0000 7f1991e00640 1 -- 172.21.15.5:0/1042636047 wait complete. 2024-09-10T09:51:16.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:51:15 smithi005 ceph-mon[30350]: pgmap v780: 129 pgs: 129 active+clean; 296 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 1.7 MiB/s wr, 173 op/s 2024-09-10T09:51:16.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:51:15 smithi005 ceph-mon[30350]: from='client.? 172.21.15.5:0/1042636047' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:51:16.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:51:15 smithi159 ceph-mon[35513]: pgmap v780: 129 pgs: 129 active+clean; 296 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 1.7 MiB/s wr, 173 op/s 2024-09-10T09:51:16.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:51:15 smithi159 ceph-mon[35513]: from='client.? 172.21.15.5:0/1042636047' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:51:16.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:51:15 smithi102 ceph-mon[35442]: pgmap v780: 129 pgs: 129 active+clean; 296 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 1.7 MiB/s wr, 173 op/s 2024-09-10T09:51:16.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:51:15 smithi102 ceph-mon[35442]: from='client.? 172.21.15.5:0/1042636047' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:51:16.147 INFO:tasks.workunit.client.0.smithi005.stdout: 1 824301 6.38 MB/sec execute 537 sec latency 24.343 ms 2024-09-10T09:51:16.341 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-10T09:51:16.740 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:51:17.147 INFO:tasks.workunit.client.0.smithi005.stdout: 1 826906 6.39 MB/sec execute 538 sec latency 12.167 ms 2024-09-10T09:51:17.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:17.372+0000 7fa2b0a00640 1 -- 172.21.15.5:0/1369175251 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa2ac109120 msgr2=0x7fa2ac110ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:51:17.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:17.372+0000 7fa2b0a00640 1 --2- 172.21.15.5:0/1369175251 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa2ac109120 0x7fa2ac110ca0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fa2a4009ca0 tx=0x7fa2a402a8c0 comp rx=0 tx=0).stop 2024-09-10T09:51:17.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:17.372+0000 7fa2b0a00640 1 -- 172.21.15.5:0/1369175251 shutdown_connections 2024-09-10T09:51:17.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:17.372+0000 7fa2b0a00640 1 --2- 172.21.15.5:0/1369175251 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa2ac109120 0x7fa2ac110ca0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:17.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:17.372+0000 7fa2b0a00640 1 --2- 172.21.15.5:0/1369175251 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa2ac1049a0 0x7fa2ac1089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:17.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:17.372+0000 7fa2b0a00640 1 --2- 172.21.15.5:0/1369175251 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa2ac103ff0 0x7fa2ac1043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:17.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:17.372+0000 7fa2b0a00640 1 -- 172.21.15.5:0/1369175251 >> 172.21.15.5:0/1369175251 conn(0x7fa2ac0ffa90 msgr2=0x7fa2ac101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:51:17.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:17.372+0000 7fa2b0a00640 1 -- 172.21.15.5:0/1369175251 shutdown_connections 2024-09-10T09:51:17.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:17.372+0000 7fa2b0a00640 1 -- 172.21.15.5:0/1369175251 wait complete. 2024-09-10T09:51:17.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:17.373+0000 7fa2b0a00640 1 Processor -- start 2024-09-10T09:51:17.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:17.373+0000 7fa2b0a00640 1 -- start start 2024-09-10T09:51:17.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:17.374+0000 7fa2b0a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa2ac103ff0 0x7fa2ac19a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:51:17.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:17.374+0000 7fa2b0a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa2ac1049a0 0x7fa2ac19aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:51:17.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:17.374+0000 7fa2b0a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa2ac109120 0x7fa2ac19ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:51:17.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:17.374+0000 7fa2b0a00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa2ac114150 con 0x7fa2ac103ff0 2024-09-10T09:51:17.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:17.374+0000 7fa2b0a00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fa2ac113fd0 con 0x7fa2ac109120 2024-09-10T09:51:17.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:17.374+0000 7fa2b0a00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fa2ac1142d0 con 0x7fa2ac1049a0 2024-09-10T09:51:17.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:17.374+0000 7fa2a3e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa2ac1049a0 0x7fa2ac19aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:51:17.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:17.374+0000 7fa2a9e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa2ac103ff0 0x7fa2ac19a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:51:17.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:17.374+0000 7fa2a3e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa2ac1049a0 0x7fa2ac19aaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:51908/0 (socket says 172.21.15.5:51908) 2024-09-10T09:51:17.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:17.374+0000 7fa2a3e00640 1 -- 172.21.15.5:0/2924146775 learned_addr learned my addr 172.21.15.5:0/2924146775 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:51:17.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:17.374+0000 7fa2aa800640 1 --2- 172.21.15.5:0/2924146775 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa2ac109120 0x7fa2ac19ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:51:17.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:17.375+0000 7fa2aa800640 1 -- 172.21.15.5:0/2924146775 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa2ac1049a0 msgr2=0x7fa2ac19aaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:51:17.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:17.375+0000 7fa2aa800640 1 --2- 172.21.15.5:0/2924146775 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa2ac1049a0 0x7fa2ac19aaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:17.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:17.375+0000 7fa2aa800640 1 -- 172.21.15.5:0/2924146775 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa2ac103ff0 msgr2=0x7fa2ac19a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:51:17.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:17.375+0000 7fa2aa800640 1 --2- 172.21.15.5:0/2924146775 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa2ac103ff0 0x7fa2ac19a5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:17.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:17.375+0000 7fa2aa800640 1 -- 172.21.15.5:0/2924146775 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2ac19f560 con 0x7fa2ac109120 2024-09-10T09:51:17.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:17.375+0000 7fa2a3e00640 1 --2- 172.21.15.5:0/2924146775 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa2ac1049a0 0x7fa2ac19aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:51:17.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:17.375+0000 7fa2aa800640 1 --2- 172.21.15.5:0/2924146775 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa2ac109120 0x7fa2ac19ee80 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fa2a402ade0 tx=0x7fa2a40319f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:51:17.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:17.375+0000 7fa2a2a00640 1 -- 172.21.15.5:0/2924146775 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2a4003b70 con 0x7fa2ac109120 2024-09-10T09:51:17.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:17.375+0000 7fa2a2a00640 1 -- 172.21.15.5:0/2924146775 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa2a4003d10 con 0x7fa2ac109120 2024-09-10T09:51:17.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:17.375+0000 7fa2a2a00640 1 -- 172.21.15.5:0/2924146775 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2a402fc40 con 0x7fa2ac109120 2024-09-10T09:51:17.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:17.375+0000 7fa2b0a00640 1 -- 172.21.15.5:0/2924146775 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2ac19f7f0 con 0x7fa2ac109120 2024-09-10T09:51:17.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:17.376+0000 7fa2b0a00640 1 -- 172.21.15.5:0/2924146775 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2ac1a7130 con 0x7fa2ac109120 2024-09-10T09:51:17.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:17.378+0000 7fa2b348e740 1 Processor -- start 2024-09-10T09:51:17.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:17.378+0000 7fa2a2a00640 1 -- 172.21.15.5:0/2924146775 <== mon.2 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7fa2a4004050 con 0x7fa2ac109120 2024-09-10T09:51:17.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:17.379+0000 7fa2b348e740 1 -- start start 2024-09-10T09:51:17.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:17.379+0000 7fa2b348e740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5628988bac70 0x5628988bb050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:51:17.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:17.379+0000 7fa2b348e740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5628988bb590 0x5628988c5910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:51:17.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:17.379+0000 7fa2b348e740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5628988c5e50 0x5628988c8240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:51:17.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:17.379+0000 7fa2b348e740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5628988213e0 con 0x5628988bb590 2024-09-10T09:51:17.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:17.379+0000 7fa2b348e740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x5628987c8d80 con 0x5628988c5e50 2024-09-10T09:51:17.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:17.379+0000 7fa2b348e740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x56289881d3a0 con 0x5628988bac70 2024-09-10T09:51:17.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:17.379+0000 7fa2a9e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5628988bb590 0x5628988c5910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:51:17.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:17.379+0000 7fa2a3e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5628988bac70 0x5628988bb050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:51:17.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:17.379+0000 7fa2a9e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5628988bb590 0x5628988c5910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46614/0 (socket says 172.21.15.5:46614) 2024-09-10T09:51:17.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:17.380+0000 7fa2a9e00640 1 -- 172.21.15.5:0/1677143021 learned_addr learned my addr 172.21.15.5:0/1677143021 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:51:17.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:17.380+0000 7fa2a3e00640 1 --2- 172.21.15.5:0/1677143021 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5628988c5e50 0x5628988c8240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:51:17.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:17.380+0000 7fa2a3e00640 1 -- 172.21.15.5:0/1677143021 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5628988c5e50 msgr2=0x5628988c8240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:51:17.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:17.380+0000 7fa2a3e00640 1 --2- 172.21.15.5:0/1677143021 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5628988c5e50 0x5628988c8240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:17.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:17.380+0000 7fa2a3e00640 1 -- 172.21.15.5:0/1677143021 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5628988bb590 msgr2=0x5628988c5910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:51:17.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:17.380+0000 7fa2a3e00640 1 --2- 172.21.15.5:0/1677143021 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5628988bb590 0x5628988c5910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:17.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:17.380+0000 7fa2a3e00640 1 -- 172.21.15.5:0/1677143021 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5628987e7ae0 con 0x5628988bac70 2024-09-10T09:51:17.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:17.380+0000 7fa2a9e00640 1 --2- 172.21.15.5:0/1677143021 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5628988bb590 0x5628988c5910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:51:17.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:17.380+0000 7fa2a2a00640 1 --2- 172.21.15.5:0/2924146775 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fa2900a2c00 0x7fa2900a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:51:17.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:17.380+0000 7fa2a3e00640 1 --2- 172.21.15.5:0/1677143021 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5628988bac70 0x5628988bb050 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fa294009640 tx=0x7fa294009530 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e4c340570a60daf0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:51:17.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:17.381+0000 7fa2a2a00640 1 -- 172.21.15.5:0/2924146775 <== mon.2 v2:172.21.15.102:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fa2a40f0330 con 0x7fa2ac109120 2024-09-10T09:51:17.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:17.381+0000 7fa29b000640 1 -- 172.21.15.5:0/1677143021 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa29400fc20 con 0x5628988bac70 2024-09-10T09:51:17.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:17.381+0000 7fa29b000640 1 -- 172.21.15.5:0/1677143021 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa29400fdc0 con 0x5628988bac70 2024-09-10T09:51:17.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:17.381+0000 7fa2a9e00640 1 --2- 172.21.15.5:0/2924146775 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fa2900a2c00 0x7fa2900a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:51:17.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:17.381+0000 7fa29b000640 1 -- 172.21.15.5:0/1677143021 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa29401e3f0 con 0x5628988bac70 2024-09-10T09:51:17.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:17.382+0000 7fa2b348e740 1 -- 172.21.15.5:0/1677143021 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5628988bac70 msgr2=0x5628988bb050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:51:17.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:17.382+0000 7fa2a9e00640 1 --2- 172.21.15.5:0/2924146775 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fa2900a2c00 0x7fa2900a50c0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x5628988b8060 tx=0x7fa29c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:51:17.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:17.382+0000 7fa2b348e740 1 --2- 172.21.15.5:0/1677143021 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5628988bac70 0x5628988bb050 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fa294009640 tx=0x7fa294009530 comp rx=0 tx=0).stop 2024-09-10T09:51:17.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:17.382+0000 7fa2b348e740 1 -- 172.21.15.5:0/1677143021 shutdown_connections 2024-09-10T09:51:17.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:17.382+0000 7fa2b348e740 1 --2- 172.21.15.5:0/1677143021 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5628988c5e50 0x5628988c8240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:17.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:17.382+0000 7fa2b348e740 1 --2- 172.21.15.5:0/1677143021 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5628988bb590 0x5628988c5910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:17.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:17.382+0000 7fa2b348e740 1 --2- 172.21.15.5:0/1677143021 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5628988bac70 0x5628988bb050 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:17.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:17.382+0000 7fa2b348e740 1 -- 172.21.15.5:0/1677143021 >> 172.21.15.5:0/1677143021 conn(0x5628988b7200 msgr2=0x5628988b75e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:51:17.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:17.382+0000 7fa2b348e740 1 -- 172.21.15.5:0/1677143021 shutdown_connections 2024-09-10T09:51:17.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:17.382+0000 7fa2b348e740 1 -- 172.21.15.5:0/1677143021 wait complete. 2024-09-10T09:51:17.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:17.383+0000 7fa2b348e740 1 -- start start 2024-09-10T09:51:17.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:17.383+0000 7fa2b348e740 1 Processor -- start 2024-09-10T09:51:17.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:17.383+0000 7fa2b348e740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5628988bac70 0x5628988c2ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:51:17.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:17.383+0000 7fa2b348e740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5628988bb590 0x5628988c3410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:51:17.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:17.384+0000 7fa2b348e740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5628988c5e50 0x5628988c3950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:51:17.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:17.384+0000 7fa2b348e740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5628988c8a20 con 0x5628988bb590 2024-09-10T09:51:17.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:17.384+0000 7fa2b348e740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x5628987dc0f0 con 0x5628988bac70 2024-09-10T09:51:17.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:17.384+0000 7fa2b348e740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5628988c8ba0 con 0x5628988c5e50 2024-09-10T09:51:17.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:17.384+0000 7fa2b348e740 10 client.0 _ll_register_callbacks cb 0x5628987c3bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:51:17.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:17.384+0000 7fa2a9e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5628988bac70 0x5628988c2ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:51:17.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:17.384+0000 7fa2aa800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5628988c5e50 0x5628988c3950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:51:17.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:17.384+0000 7fa2a9e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5628988bac70 0x5628988c2ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:33596/0 (socket says 172.21.15.5:33596) 2024-09-10T09:51:17.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:17.384+0000 7fa2aa800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5628988c5e50 0x5628988c3950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:51926/0 (socket says 172.21.15.5:51926) 2024-09-10T09:51:17.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:17.384+0000 7fa2a9e00640 1 -- 172.21.15.5:0/989172038 learned_addr learned my addr 172.21.15.5:0/989172038 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:51:17.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:17.384+0000 7fa2a3e00640 1 --2- 172.21.15.5:0/989172038 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5628988bb590 0x5628988c3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:51:17.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:17.384+0000 7fa2a9e00640 1 -- 172.21.15.5:0/989172038 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5628988c5e50 msgr2=0x5628988c3950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:51:17.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:17.384+0000 7fa2a9e00640 1 --2- 172.21.15.5:0/989172038 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5628988c5e50 0x5628988c3950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:17.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:17.384+0000 7fa2a9e00640 1 -- 172.21.15.5:0/989172038 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5628988bb590 msgr2=0x5628988c3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:51:17.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:17.384+0000 7fa2a9e00640 1 --2- 172.21.15.5:0/989172038 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5628988bb590 0x5628988c3410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:17.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:17.384+0000 7fa2a9e00640 1 -- 172.21.15.5:0/989172038 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5628985923c0 con 0x5628988bac70 2024-09-10T09:51:17.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:17.384+0000 7fa2aa800640 1 --2- 172.21.15.5:0/989172038 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5628988c5e50 0x5628988c3950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:51:17.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:17.385+0000 7fa2a9e00640 1 --2- 172.21.15.5:0/989172038 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5628988bac70 0x5628988c2ed0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fa29c004620 tx=0x7fa29c004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c983081bcd09c9a3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:51:17.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:17.385+0000 7fa28f400640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-09-10T09:51:17.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:17.385+0000 7fa28f400640 1 -- 172.21.15.5:0/989172038 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa29c01f640 con 0x5628988bac70 2024-09-10T09:51:17.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:17.385+0000 7fa2b348e740 1 -- 172.21.15.5:0/989172038 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa29400fdc0 con 0x5628988bac70 2024-09-10T09:51:17.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:17.385+0000 7fa28f400640 1 -- 172.21.15.5:0/989172038 <== mon.2 v2:172.21.15.102:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fa29c01f7e0 con 0x5628988bac70 2024-09-10T09:51:17.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:17.385+0000 7fa28f400640 1 -- 172.21.15.5:0/989172038 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fa29c01fa20 con 0x5628988bac70 2024-09-10T09:51:17.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:17.385+0000 7fa2b348e740 10 client.27971 fetch_fsmap learned FSMap version 36 2024-09-10T09:51:17.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:17.385+0000 7fa2b348e740 1 -- 172.21.15.5:0/989172038 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x5628988d4c90 con 0x5628988bac70 2024-09-10T09:51:17.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:17.385+0000 7fa28f400640 1 -- 172.21.15.5:0/989172038 <== mon.2 v2:172.21.15.102:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa29400fdc0 con 0x5628988bac70 2024-09-10T09:51:17.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:17.386+0000 7fa28f400640 1 -- 172.21.15.5:0/989172038 <== mon.2 v2:172.21.15.102:3300/0 5 ==== fsmap(e 36) ==== 8370+0+0 (secure 0 0 0) 0x7fa29c027070 con 0x5628988bac70 2024-09-10T09:51:17.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:17.386+0000 7fa2b348e740 10 client.27971 fetch_fsmap finished waiting for FSMap version 36 2024-09-10T09:51:17.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:17.386+0000 7fa2b348e740 10 client.27971 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-09-10T09:51:17.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:17.386+0000 7fa2b348e740 20 client.27971 populate_metadata read hostname 'smithi005' 2024-09-10T09:51:17.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:17.386+0000 7fa2b348e740 1 --2- 172.21.15.5:0/989172038 >> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] conn(0x5628988d60a0 0x5628988d84f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:51:17.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:17.386+0000 7fa2b348e740 4 client.27971 mds_command: new command op to 24443 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:51:17.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:17.386+0000 7fa2b348e740 1 -- 172.21.15.5:0/989172038 --> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56289882e950 con 0x5628988d60a0 2024-09-10T09:51:17.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:17.386+0000 7fa2a3e00640 1 --2- 172.21.15.5:0/989172038 >> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] conn(0x5628988d60a0 0x5628988d84f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:51:17.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:17.387+0000 7fa2a3e00640 1 --2- 172.21.15.5:0/989172038 >> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] conn(0x5628988d60a0 0x5628988d84f0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=8f9b037cd59a57fb server_cookie=bf4fc848208fcfe3 in_seq=0 out_seq=0 2024-09-10T09:51:17.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:17.387+0000 7fa28f400640 10 client.27971 ms_handle_connect on v2:172.21.15.102:6832/2457007427 2024-09-10T09:51:17.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:17.387+0000 7fa28f400640 1 -- 172.21.15.5:0/989172038 <== mds.4 v2:172.21.15.102:6832/2457007427 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56289882e950 con 0x5628988d60a0 2024-09-10T09:51:17.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:17.387+0000 7fa28f400640 10 client.27971 handle_command_reply: tid=0 2024-09-10T09:51:17.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:17.387+0000 7fa2b348e740 1 client.27971 shutdown 2024-09-10T09:51:17.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:17.388+0000 7fa2b348e740 1 -- 172.21.15.5:0/989172038 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5628988bac70 msgr2=0x5628988c2ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:51:17.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:17.388+0000 7fa2b348e740 1 --2- 172.21.15.5:0/989172038 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5628988bac70 0x5628988c2ed0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fa29c004620 tx=0x7fa29c004730 comp rx=0 tx=0).stop 2024-09-10T09:51:17.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:17.388+0000 7fa2b348e740 1 -- 172.21.15.5:0/989172038 shutdown_connections 2024-09-10T09:51:17.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:17.388+0000 7fa2b348e740 1 --2- 172.21.15.5:0/989172038 >> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] conn(0x5628988d60a0 0x5628988d84f0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:17.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:17.388+0000 7fa2b348e740 1 --2- 172.21.15.5:0/989172038 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5628988c5e50 0x5628988c3950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:17.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:17.388+0000 7fa2b348e740 1 --2- 172.21.15.5:0/989172038 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5628988bb590 0x5628988c3410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:17.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:17.388+0000 7fa2b348e740 1 --2- 172.21.15.5:0/989172038 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5628988bac70 0x5628988c2ed0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:17.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:17.388+0000 7fa2b348e740 1 -- 172.21.15.5:0/989172038 >> 172.21.15.5:0/989172038 conn(0x5628988b7200 msgr2=0x5628988b6380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:51:17.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:17.388+0000 7fa28f400640 0 client.27971 ms_handle_reset on v2:172.21.15.102:6832/2457007427 2024-09-10T09:51:17.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:17.388+0000 7fa2b348e740 1 -- 172.21.15.5:0/989172038 shutdown_connections 2024-09-10T09:51:17.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:17.388+0000 7fa2b348e740 1 -- 172.21.15.5:0/989172038 wait complete. 2024-09-10T09:51:17.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:17.388+0000 7fa2b348e740 20 client.27971 trim_cache size 0 max 16384 2024-09-10T09:51:17.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:17.402+0000 7fa2b348e740 1 Processor -- start 2024-09-10T09:51:17.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:17.402+0000 7fa2b348e740 1 -- start start 2024-09-10T09:51:17.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:17.402+0000 7fa2b348e740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5628988bac70 0x5628988bd6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:51:17.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:17.402+0000 7fa2b348e740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5628988bb590 0x5628988dddc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:51:17.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:17.402+0000 7fa2b348e740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5628988c5e50 0x5628988de300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:51:17.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:17.402+0000 7fa2b348e740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa2ac1b0ff0 con 0x5628988bac70 2024-09-10T09:51:17.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:17.402+0000 7fa2b348e740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fa2ac1b2b30 con 0x5628988c5e50 2024-09-10T09:51:17.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:17.402+0000 7fa2b348e740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fa2ac1b16c0 con 0x5628988bb590 2024-09-10T09:51:17.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:17.402+0000 7fa2a9e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5628988bac70 0x5628988bd6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:51:17.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:17.402+0000 7fa2aa800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5628988c5e50 0x5628988de300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:51:17.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:17.402+0000 7fa2a9e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5628988bac70 0x5628988bd6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46636/0 (socket says 172.21.15.5:46636) 2024-09-10T09:51:17.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:17.403+0000 7fa2a9e00640 1 -- 172.21.15.5:0/1505989567 learned_addr learned my addr 172.21.15.5:0/1505989567 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:51:17.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:17.403+0000 7fa2a3e00640 1 --2- 172.21.15.5:0/1505989567 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5628988bb590 0x5628988dddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:51:17.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:17.403+0000 7fa2aa800640 1 -- 172.21.15.5:0/1505989567 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5628988bb590 msgr2=0x5628988dddc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:51:17.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:17.403+0000 7fa2aa800640 1 --2- 172.21.15.5:0/1505989567 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5628988bb590 0x5628988dddc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:17.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:17.403+0000 7fa2aa800640 1 -- 172.21.15.5:0/1505989567 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5628988bac70 msgr2=0x5628988bd6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:51:17.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:17.403+0000 7fa2aa800640 1 --2- 172.21.15.5:0/1505989567 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5628988bac70 0x5628988bd6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:17.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:17.403+0000 7fa2aa800640 1 -- 172.21.15.5:0/1505989567 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5628988baaa0 con 0x5628988c5e50 2024-09-10T09:51:17.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:17.403+0000 7fa2aa800640 1 --2- 172.21.15.5:0/1505989567 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5628988c5e50 0x5628988de300 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fa2a4031f80 tx=0x7fa2a403d040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=53e1e41b57075f17 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:51:17.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:17.403+0000 7fa29a600640 1 -- 172.21.15.5:0/1505989567 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2a40bcf60 con 0x5628988c5e50 2024-09-10T09:51:17.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:17.403+0000 7fa29a600640 1 -- 172.21.15.5:0/1505989567 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa2a4036040 con 0x5628988c5e50 2024-09-10T09:51:17.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:17.403+0000 7fa29a600640 1 -- 172.21.15.5:0/1505989567 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2a402fde0 con 0x5628988c5e50 2024-09-10T09:51:17.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:17.404+0000 7fa2b348e740 1 -- 172.21.15.5:0/1505989567 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5628988c5e50 msgr2=0x5628988de300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:51:17.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:17.404+0000 7fa2b348e740 1 --2- 172.21.15.5:0/1505989567 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5628988c5e50 0x5628988de300 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fa2a4031f80 tx=0x7fa2a403d040 comp rx=0 tx=0).stop 2024-09-10T09:51:17.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:17.404+0000 7fa2b348e740 1 -- 172.21.15.5:0/1505989567 shutdown_connections 2024-09-10T09:51:17.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:17.404+0000 7fa2b348e740 1 --2- 172.21.15.5:0/1505989567 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5628988c5e50 0x5628988de300 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:17.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:17.404+0000 7fa2b348e740 1 --2- 172.21.15.5:0/1505989567 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5628988bb590 0x5628988dddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:17.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:17.404+0000 7fa2b348e740 1 --2- 172.21.15.5:0/1505989567 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5628988bac70 0x5628988bd6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:17.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:17.404+0000 7fa2b348e740 1 -- 172.21.15.5:0/1505989567 >> 172.21.15.5:0/1505989567 conn(0x5628988b7200 msgr2=0x5628988c4f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:51:17.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:17.404+0000 7fa2b348e740 1 -- 172.21.15.5:0/1505989567 shutdown_connections 2024-09-10T09:51:17.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:17.404+0000 7fa2b348e740 1 -- 172.21.15.5:0/1505989567 wait complete. 2024-09-10T09:51:17.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:17.405+0000 7fa2b348e740 1 -- start start 2024-09-10T09:51:17.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:17.405+0000 7fa2b348e740 1 Processor -- start 2024-09-10T09:51:17.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:17.405+0000 7fa2b348e740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5628988bac70 0x5628988c9cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:51:17.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:17.405+0000 7fa2b348e740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5628988bb590 0x5628988ca1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:51:17.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:17.405+0000 7fa2b348e740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5628988c5e50 0x5628988ca730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:51:17.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:17.406+0000 7fa2b348e740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa2ac1b2460 con 0x5628988bac70 2024-09-10T09:51:17.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:17.406+0000 7fa2b348e740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fa2ac1b1d90 con 0x5628988c5e50 2024-09-10T09:51:17.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:17.406+0000 7fa2b348e740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fa2ac1aefd0 con 0x5628988bb590 2024-09-10T09:51:17.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:17.406+0000 7fa2b348e740 10 client.0 _ll_register_callbacks cb 0x7fa29c009f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:51:17.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:17.406+0000 7fa2a3e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5628988bb590 0x5628988ca1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:51:17.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:17.406+0000 7fa2a9e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5628988bac70 0x5628988c9cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:51:17.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:17.406+0000 7fa2a3e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5628988bb590 0x5628988ca1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:51936/0 (socket says 172.21.15.5:51936) 2024-09-10T09:51:17.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:17.406+0000 7fa2a3e00640 1 -- 172.21.15.5:0/3187258585 learned_addr learned my addr 172.21.15.5:0/3187258585 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:51:17.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:17.406+0000 7fa2aa800640 1 --2- 172.21.15.5:0/3187258585 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5628988c5e50 0x5628988ca730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:51:17.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:17.406+0000 7fa2a3e00640 1 -- 172.21.15.5:0/3187258585 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5628988c5e50 msgr2=0x5628988ca730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:51:17.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:17.406+0000 7fa2a3e00640 1 --2- 172.21.15.5:0/3187258585 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5628988c5e50 0x5628988ca730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:17.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:17.406+0000 7fa2a3e00640 1 -- 172.21.15.5:0/3187258585 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5628988bac70 msgr2=0x5628988c9cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:51:17.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:17.406+0000 7fa2a3e00640 1 --2- 172.21.15.5:0/3187258585 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5628988bac70 0x5628988c9cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:17.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:17.406+0000 7fa2a3e00640 1 -- 172.21.15.5:0/3187258585 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5628988b2770 con 0x5628988bb590 2024-09-10T09:51:17.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:17.407+0000 7fa2a3e00640 1 --2- 172.21.15.5:0/3187258585 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5628988bb590 0x5628988ca1f0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fa294004aa0 tx=0x7fa294004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aabcfdc61080698f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:51:17.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:17.407+0000 7fa28f400640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-09-10T09:51:17.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:17.407+0000 7fa28f400640 1 -- 172.21.15.5:0/3187258585 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2940114c0 con 0x5628988bb590 2024-09-10T09:51:17.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:17.407+0000 7fa2b348e740 1 -- 172.21.15.5:0/3187258585 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa2a4036040 con 0x5628988bb590 2024-09-10T09:51:17.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:17.407+0000 7fa28f400640 1 -- 172.21.15.5:0/3187258585 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fa29402a480 con 0x5628988bb590 2024-09-10T09:51:17.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:17.407+0000 7fa28f400640 1 -- 172.21.15.5:0/3187258585 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fa29402a680 con 0x5628988bb590 2024-09-10T09:51:17.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:17.407+0000 7fa2b348e740 10 client.27973 fetch_fsmap learned FSMap version 36 2024-09-10T09:51:17.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:17.407+0000 7fa2b348e740 1 -- 172.21.15.5:0/3187258585 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x562898940ff0 con 0x5628988bb590 2024-09-10T09:51:17.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:17.407+0000 7fa28f400640 1 -- 172.21.15.5:0/3187258585 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa2a4036040 con 0x5628988bb590 2024-09-10T09:51:17.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:17.408+0000 7fa28f400640 1 -- 172.21.15.5:0/3187258585 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 36) ==== 8370+0+0 (secure 0 0 0) 0x7fa294027050 con 0x5628988bb590 2024-09-10T09:51:17.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:17.408+0000 7fa2b348e740 10 client.27973 fetch_fsmap finished waiting for FSMap version 36 2024-09-10T09:51:17.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:17.408+0000 7fa2b348e740 10 client.27973 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-09-10T09:51:17.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:17.408+0000 7fa2b348e740 20 client.27973 populate_metadata read hostname 'smithi005' 2024-09-10T09:51:17.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:17.408+0000 7fa2b348e740 1 --2- 172.21.15.5:0/3187258585 >> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] conn(0x5628989413f0 0x562898943810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:51:17.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:17.408+0000 7fa2b348e740 4 client.27973 mds_command: new command op to 24443 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T09:51:17.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:17.408+0000 7fa2b348e740 1 -- 172.21.15.5:0/3187258585 --> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562898828e60 con 0x5628989413f0 2024-09-10T09:51:17.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:17.408+0000 7fa2a9e00640 1 --2- 172.21.15.5:0/3187258585 >> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] conn(0x5628989413f0 0x562898943810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:51:17.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:17.409+0000 7fa2a9e00640 1 --2- 172.21.15.5:0/3187258585 >> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] conn(0x5628989413f0 0x562898943810 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=36857aa010560d79 server_cookie=fd43978bcc1a4864 in_seq=0 out_seq=0 2024-09-10T09:51:17.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:17.409+0000 7fa28f400640 10 client.27973 ms_handle_connect on v2:172.21.15.102:6832/2457007427 2024-09-10T09:51:17.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:17.409+0000 7fa28f400640 1 -- 172.21.15.5:0/3187258585 <== mds.4 v2:172.21.15.102:6832/2457007427 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562898828e60 con 0x5628989413f0 2024-09-10T09:51:17.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:17.409+0000 7fa28f400640 10 client.27973 handle_command_reply: tid=0 2024-09-10T09:51:17.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:17.409+0000 7fa2b348e740 1 client.27973 shutdown 2024-09-10T09:51:17.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:17.409+0000 7fa2b348e740 1 -- 172.21.15.5:0/3187258585 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5628988bb590 msgr2=0x5628988ca1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:51:17.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:17.409+0000 7fa2b348e740 1 --2- 172.21.15.5:0/3187258585 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5628988bb590 0x5628988ca1f0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fa294004aa0 tx=0x7fa294004ad0 comp rx=0 tx=0).stop 2024-09-10T09:51:17.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:17.409+0000 7fa2b348e740 1 -- 172.21.15.5:0/3187258585 shutdown_connections 2024-09-10T09:51:17.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:17.409+0000 7fa2b348e740 1 --2- 172.21.15.5:0/3187258585 >> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] conn(0x5628989413f0 0x562898943810 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:17.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:17.409+0000 7fa2b348e740 1 --2- 172.21.15.5:0/3187258585 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5628988c5e50 0x5628988ca730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:17.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:17.409+0000 7fa2b348e740 1 --2- 172.21.15.5:0/3187258585 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5628988bb590 0x5628988ca1f0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:17.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:17.409+0000 7fa2b348e740 1 --2- 172.21.15.5:0/3187258585 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5628988bac70 0x5628988c9cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:17.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:17.409+0000 7fa28f400640 0 client.27973 ms_handle_reset on v2:172.21.15.102:6832/2457007427 2024-09-10T09:51:17.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:17.409+0000 7fa2b348e740 1 -- 172.21.15.5:0/3187258585 >> 172.21.15.5:0/3187258585 conn(0x5628988b7200 msgr2=0x5628988b5b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:51:17.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:17.409+0000 7fa2b348e740 1 -- 172.21.15.5:0/3187258585 shutdown_connections 2024-09-10T09:51:17.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:17.409+0000 7fa2b348e740 1 -- 172.21.15.5:0/3187258585 wait complete. 2024-09-10T09:51:17.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:17.409+0000 7fa2b348e740 20 client.27973 trim_cache size 0 max 16384 2024-09-10T09:51:17.424 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-10T09:51:17.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:17.410+0000 7fa2b0a00640 1 -- 172.21.15.5:0/2924146775 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fa2900a2c00 msgr2=0x7fa2900a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:51:17.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:17.410+0000 7fa2b0a00640 1 --2- 172.21.15.5:0/2924146775 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fa2900a2c00 0x7fa2900a50c0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x5628988b8060 tx=0x7fa29c005f90 comp rx=0 tx=0).stop 2024-09-10T09:51:17.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:17.410+0000 7fa2b0a00640 1 -- 172.21.15.5:0/2924146775 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa2ac109120 msgr2=0x7fa2ac19ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:51:17.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:17.410+0000 7fa2b0a00640 1 --2- 172.21.15.5:0/2924146775 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa2ac109120 0x7fa2ac19ee80 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fa2a402ade0 tx=0x7fa2a40319f0 comp rx=0 tx=0).stop 2024-09-10T09:51:17.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:17.410+0000 7fa2b0a00640 1 -- 172.21.15.5:0/2924146775 shutdown_connections 2024-09-10T09:51:17.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:17.410+0000 7fa2b0a00640 1 --2- 172.21.15.5:0/2924146775 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fa2900a2c00 0x7fa2900a50c0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:17.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:17.410+0000 7fa2b0a00640 1 --2- 172.21.15.5:0/2924146775 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa2ac109120 0x7fa2ac19ee80 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:17.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:17.410+0000 7fa2b0a00640 1 --2- 172.21.15.5:0/2924146775 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa2ac1049a0 0x7fa2ac19aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:17.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:17.410+0000 7fa2b0a00640 1 --2- 172.21.15.5:0/2924146775 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa2ac103ff0 0x7fa2ac19a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:17.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:17.410+0000 7fa2b0a00640 1 -- 172.21.15.5:0/2924146775 >> 172.21.15.5:0/2924146775 conn(0x7fa2ac0ffa90 msgr2=0x7fa2ac101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:51:17.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:17.410+0000 7fa2b0a00640 1 -- 172.21.15.5:0/2924146775 shutdown_connections 2024-09-10T09:51:17.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:17.410+0000 7fa2b0a00640 1 -- 172.21.15.5:0/2924146775 wait complete. 2024-09-10T09:51:18.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:51:17 smithi005 ceph-mon[30350]: pgmap v781: 129 pgs: 129 active+clean; 299 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 1.9 MiB/s wr, 234 op/s 2024-09-10T09:51:18.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:51:17 smithi159 ceph-mon[35513]: pgmap v781: 129 pgs: 129 active+clean; 299 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 1.9 MiB/s wr, 234 op/s 2024-09-10T09:51:18.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:51:17 smithi102 ceph-mon[35442]: pgmap v781: 129 pgs: 129 active+clean; 299 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 1.9 MiB/s wr, 234 op/s 2024-09-10T09:51:18.147 INFO:tasks.workunit.client.0.smithi005.stdout: 1 828652 6.40 MB/sec execute 539 sec latency 22.370 ms 2024-09-10T09:51:18.339 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-10T09:51:18.742 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:51:19.147 INFO:tasks.workunit.client.0.smithi005.stdout: 1 831099 6.41 MB/sec execute 540 sec latency 13.986 ms 2024-09-10T09:51:19.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:19.347+0000 7f956a600640 1 -- 172.21.15.5:0/4019695830 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9564104010 msgr2=0x7f95641043f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:51:19.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:19.347+0000 7f956a600640 1 --2- 172.21.15.5:0/4019695830 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9564104010 0x7f95641043f0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f9554009900 tx=0x7f955402a410 comp rx=0 tx=0).stop 2024-09-10T09:51:19.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:19.348+0000 7f956a600640 1 -- 172.21.15.5:0/4019695830 shutdown_connections 2024-09-10T09:51:19.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:19.348+0000 7f956a600640 1 --2- 172.21.15.5:0/4019695830 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9564109140 0x7f9564110cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:19.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:19.348+0000 7f956a600640 1 --2- 172.21.15.5:0/4019695830 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f95641049c0 0x7f9564108a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:19.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:19.348+0000 7f956a600640 1 --2- 172.21.15.5:0/4019695830 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9564104010 0x7f95641043f0 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:19.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:19.348+0000 7f956a600640 1 -- 172.21.15.5:0/4019695830 >> 172.21.15.5:0/4019695830 conn(0x7f95640ffa50 msgr2=0x7f9564101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:51:19.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:19.348+0000 7f956a600640 1 -- 172.21.15.5:0/4019695830 shutdown_connections 2024-09-10T09:51:19.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:19.348+0000 7f956a600640 1 -- 172.21.15.5:0/4019695830 wait complete. 2024-09-10T09:51:19.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:19.349+0000 7f956a600640 1 Processor -- start 2024-09-10T09:51:19.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:19.349+0000 7f956a600640 1 -- start start 2024-09-10T09:51:19.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:19.350+0000 7f956a600640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9564104010 0x7f956419a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:51:19.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:19.350+0000 7f956a600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f95641049c0 0x7f956419ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:51:19.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:19.350+0000 7f956a600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9564109140 0x7f956419efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:51:19.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:19.350+0000 7f956a600640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f9564114190 con 0x7f9564109140 2024-09-10T09:51:19.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:19.350+0000 7f956a600640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f9564114010 con 0x7f95641049c0 2024-09-10T09:51:19.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:19.350+0000 7f956a600640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f9564114310 con 0x7f9564104010 2024-09-10T09:51:19.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:19.350+0000 7f9563400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9564104010 0x7f956419a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:51:19.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:19.350+0000 7f9562a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f95641049c0 0x7f956419ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:51:19.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:19.350+0000 7f9562a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f95641049c0 0x7f956419ac10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:33644/0 (socket says 172.21.15.5:33644) 2024-09-10T09:51:19.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:19.350+0000 7f9563400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9564104010 0x7f956419a6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:51960/0 (socket says 172.21.15.5:51960) 2024-09-10T09:51:19.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:19.350+0000 7f9562a00640 1 -- 172.21.15.5:0/1031192489 learned_addr learned my addr 172.21.15.5:0/1031192489 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:51:19.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:19.350+0000 7f9563e00640 1 --2- 172.21.15.5:0/1031192489 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9564109140 0x7f956419efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:51:19.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:19.350+0000 7f9562a00640 1 -- 172.21.15.5:0/1031192489 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9564104010 msgr2=0x7f956419a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:51:19.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:19.350+0000 7f9562a00640 1 --2- 172.21.15.5:0/1031192489 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9564104010 0x7f956419a6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:19.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:19.350+0000 7f9562a00640 1 -- 172.21.15.5:0/1031192489 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9564109140 msgr2=0x7f956419efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:51:19.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:19.350+0000 7f9562a00640 1 --2- 172.21.15.5:0/1031192489 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9564109140 0x7f956419efa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:19.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:19.350+0000 7f9562a00640 1 -- 172.21.15.5:0/1031192489 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f956419f680 con 0x7f95641049c0 2024-09-10T09:51:19.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:19.351+0000 7f9562a00640 1 --2- 172.21.15.5:0/1031192489 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f95641049c0 0x7f956419ac10 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f955000e9f0 tx=0x7f955000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:51:19.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:19.351+0000 7f955be00640 1 -- 172.21.15.5:0/1031192489 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f955000cd80 con 0x7f95641049c0 2024-09-10T09:51:19.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:19.351+0000 7f955be00640 1 -- 172.21.15.5:0/1031192489 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9550010910 con 0x7f95641049c0 2024-09-10T09:51:19.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:19.351+0000 7f956a600640 1 -- 172.21.15.5:0/1031192489 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f956419f970 con 0x7f95641049c0 2024-09-10T09:51:19.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:19.351+0000 7f955be00640 1 -- 172.21.15.5:0/1031192489 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9550018990 con 0x7f95641049c0 2024-09-10T09:51:19.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:19.351+0000 7f956a600640 1 -- 172.21.15.5:0/1031192489 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95641a7250 con 0x7f95641049c0 2024-09-10T09:51:19.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:19.354+0000 7f956cfec740 1 Processor -- start 2024-09-10T09:51:19.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:19.354+0000 7f956cfec740 1 -- start start 2024-09-10T09:51:19.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:19.355+0000 7f955be00640 1 -- 172.21.15.5:0/1031192489 <== mon.2 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f9550010ac0 con 0x7f95641049c0 2024-09-10T09:51:19.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:19.355+0000 7f956cfec740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55636368ac70 0x55636368b050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:51:19.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:19.355+0000 7f956cfec740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55636368b590 0x556363695910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:51:19.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:19.355+0000 7f956cfec740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556363695e50 0x556363698240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:51:19.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:19.355+0000 7f956cfec740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5563635f13e0 con 0x55636368ac70 2024-09-10T09:51:19.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:19.355+0000 7f956cfec740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x556363598d80 con 0x556363695e50 2024-09-10T09:51:19.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:19.355+0000 7f956cfec740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5563635ed3a0 con 0x55636368b590 2024-09-10T09:51:19.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:19.355+0000 7f9563400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55636368b590 0x556363695910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:51:19.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:19.355+0000 7f9563e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55636368ac70 0x55636368b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:51:19.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:19.355+0000 7f9563400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55636368b590 0x556363695910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:51972/0 (socket says 172.21.15.5:51972) 2024-09-10T09:51:19.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:19.355+0000 7f9563400640 1 -- 172.21.15.5:0/763654104 learned_addr learned my addr 172.21.15.5:0/763654104 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:51:19.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:19.355+0000 7f9563e00640 1 --2- 172.21.15.5:0/763654104 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55636368ac70 0x55636368b050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46684/0 (socket says 172.21.15.5:46684) 2024-09-10T09:51:19.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:19.355+0000 7f9562a00640 1 --2- 172.21.15.5:0/763654104 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556363695e50 0x556363698240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:51:19.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:19.356+0000 7f9563400640 1 -- 172.21.15.5:0/763654104 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556363695e50 msgr2=0x556363698240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:51:19.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:19.356+0000 7f9563400640 1 --2- 172.21.15.5:0/763654104 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556363695e50 0x556363698240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:19.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:19.356+0000 7f9563400640 1 -- 172.21.15.5:0/763654104 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55636368ac70 msgr2=0x55636368b050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:51:19.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:19.356+0000 7f9563400640 1 --2- 172.21.15.5:0/763654104 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55636368ac70 0x55636368b050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:19.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:19.356+0000 7f9563400640 1 -- 172.21.15.5:0/763654104 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5563635b7ae0 con 0x55636368b590 2024-09-10T09:51:19.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:19.356+0000 7f9562a00640 1 --2- 172.21.15.5:0/763654104 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556363695e50 0x556363698240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T09:51:19.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:19.356+0000 7f9563400640 1 --2- 172.21.15.5:0/763654104 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55636368b590 0x556363695910 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f9554009900 tx=0x7f9554004040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=753874bcc610361e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:51:19.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:19.356+0000 7f9558c00640 1 -- 172.21.15.5:0/763654104 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9554035040 con 0x55636368b590 2024-09-10T09:51:19.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:19.356+0000 7f9558c00640 1 -- 172.21.15.5:0/763654104 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9554005520 con 0x55636368b590 2024-09-10T09:51:19.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:19.356+0000 7f9558c00640 1 -- 172.21.15.5:0/763654104 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95540035f0 con 0x55636368b590 2024-09-10T09:51:19.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:19.357+0000 7f955be00640 1 --2- 172.21.15.5:0/1031192489 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f95480a2b80 0x7f95480a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:51:19.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:19.357+0000 7f956cfec740 1 -- 172.21.15.5:0/763654104 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55636368b590 msgr2=0x556363695910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:51:19.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:19.357+0000 7f956cfec740 1 --2- 172.21.15.5:0/763654104 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55636368b590 0x556363695910 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f9554009900 tx=0x7f9554004040 comp rx=0 tx=0).stop 2024-09-10T09:51:19.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:19.357+0000 7f955be00640 1 -- 172.21.15.5:0/1031192489 <== mon.2 v2:172.21.15.102:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f95500cf850 con 0x7f95641049c0 2024-09-10T09:51:19.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:19.357+0000 7f9563e00640 1 --2- 172.21.15.5:0/1031192489 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f95480a2b80 0x7f95480a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:51:19.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:19.357+0000 7f956cfec740 1 -- 172.21.15.5:0/763654104 shutdown_connections 2024-09-10T09:51:19.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:19.357+0000 7f956cfec740 1 --2- 172.21.15.5:0/763654104 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556363695e50 0x556363698240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:19.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:19.357+0000 7f956cfec740 1 --2- 172.21.15.5:0/763654104 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55636368b590 0x556363695910 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:19.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:19.357+0000 7f956cfec740 1 --2- 172.21.15.5:0/763654104 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55636368ac70 0x55636368b050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:19.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:19.357+0000 7f956cfec740 1 -- 172.21.15.5:0/763654104 >> 172.21.15.5:0/763654104 conn(0x556363687200 msgr2=0x5563636875e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:51:19.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:19.357+0000 7f956cfec740 1 -- 172.21.15.5:0/763654104 shutdown_connections 2024-09-10T09:51:19.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:19.358+0000 7f956cfec740 1 -- 172.21.15.5:0/763654104 wait complete. 2024-09-10T09:51:19.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:19.358+0000 7f9563e00640 1 --2- 172.21.15.5:0/1031192489 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f95480a2b80 0x7f95480a5040 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f955c005fd0 tx=0x7f955c009760 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:51:19.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:19.358+0000 7f956cfec740 1 -- start start 2024-09-10T09:51:19.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:19.358+0000 7f956cfec740 1 Processor -- start 2024-09-10T09:51:19.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:19.359+0000 7f956cfec740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55636368ac70 0x556363692cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:51:19.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:19.359+0000 7f956cfec740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55636368b590 0x556363693230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:51:19.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:19.359+0000 7f956cfec740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556363695e50 0x556363693770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:51:19.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:19.359+0000 7f956cfec740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x556363698a20 con 0x55636368b590 2024-09-10T09:51:19.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:19.359+0000 7f956cfec740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x5563635ac0f0 con 0x55636368ac70 2024-09-10T09:51:19.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:19.359+0000 7f956cfec740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x556363698ba0 con 0x556363695e50 2024-09-10T09:51:19.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:19.359+0000 7f956cfec740 10 client.0 _ll_register_callbacks cb 0x556363593bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:51:19.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:19.359+0000 7f9563400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55636368ac70 0x556363692cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:51:19.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:19.359+0000 7f9563400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55636368ac70 0x556363692cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:33662/0 (socket says 172.21.15.5:33662) 2024-09-10T09:51:19.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:19.359+0000 7f9562a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55636368b590 0x556363693230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:51:19.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:19.359+0000 7f9563400640 1 -- 172.21.15.5:0/3265147542 learned_addr learned my addr 172.21.15.5:0/3265147542 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:51:19.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:19.360+0000 7f9563400640 1 -- 172.21.15.5:0/3265147542 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556363695e50 msgr2=0x556363693770 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T09:51:19.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:19.360+0000 7f9563400640 1 --2- 172.21.15.5:0/3265147542 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556363695e50 0x556363693770 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:19.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:19.360+0000 7f9563400640 1 -- 172.21.15.5:0/3265147542 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55636368b590 msgr2=0x556363693230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:51:19.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:19.360+0000 7f9563400640 1 --2- 172.21.15.5:0/3265147542 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55636368b590 0x556363693230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:19.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:19.360+0000 7f9563400640 1 -- 172.21.15.5:0/3265147542 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5563633623c0 con 0x55636368ac70 2024-09-10T09:51:19.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:19.360+0000 7f9562a00640 1 --2- 172.21.15.5:0/3265147542 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55636368b590 0x556363693230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:51:19.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:19.360+0000 7f9563400640 1 --2- 172.21.15.5:0/3265147542 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55636368ac70 0x556363692cf0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f9554004920 tx=0x7f9554004bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6fade92346944990 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:51:19.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:19.360+0000 7f954d600640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-09-10T09:51:19.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:19.360+0000 7f954d600640 1 -- 172.21.15.5:0/3265147542 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9554052060 con 0x55636368ac70 2024-09-10T09:51:19.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:19.360+0000 7f954d600640 1 -- 172.21.15.5:0/3265147542 <== mon.2 v2:172.21.15.102:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f9554004de0 con 0x55636368ac70 2024-09-10T09:51:19.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:19.360+0000 7f956cfec740 1 -- 172.21.15.5:0/3265147542 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9554005520 con 0x55636368ac70 2024-09-10T09:51:19.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:19.361+0000 7f954d600640 1 -- 172.21.15.5:0/3265147542 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f9554035070 con 0x55636368ac70 2024-09-10T09:51:19.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:19.361+0000 7f956cfec740 10 client.27986 fetch_fsmap learned FSMap version 36 2024-09-10T09:51:19.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:19.361+0000 7f956cfec740 1 -- 172.21.15.5:0/3265147542 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x5563636a4980 con 0x55636368ac70 2024-09-10T09:51:19.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:19.361+0000 7f954d600640 1 -- 172.21.15.5:0/3265147542 <== mon.2 v2:172.21.15.102:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9554005520 con 0x55636368ac70 2024-09-10T09:51:19.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:19.361+0000 7f954d600640 1 -- 172.21.15.5:0/3265147542 <== mon.2 v2:172.21.15.102:3300/0 5 ==== fsmap(e 36) ==== 8370+0+0 (secure 0 0 0) 0x7f9554003ab0 con 0x55636368ac70 2024-09-10T09:51:19.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:19.361+0000 7f956cfec740 10 client.27986 fetch_fsmap finished waiting for FSMap version 36 2024-09-10T09:51:19.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:19.361+0000 7f956cfec740 10 client.27986 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-09-10T09:51:19.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:19.361+0000 7f956cfec740 20 client.27986 populate_metadata read hostname 'smithi005' 2024-09-10T09:51:19.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:19.361+0000 7f956cfec740 1 --2- 172.21.15.5:0/3265147542 >> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] conn(0x5563636a5cd0 0x5563636a80c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:51:19.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:19.361+0000 7f956cfec740 4 client.27986 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:51:19.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:19.361+0000 7f956cfec740 1 -- 172.21.15.5:0/3265147542 --> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5563635fe950 con 0x5563636a5cd0 2024-09-10T09:51:19.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:19.362+0000 7f9562a00640 1 --2- 172.21.15.5:0/3265147542 >> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] conn(0x5563636a5cd0 0x5563636a80c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:51:19.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:19.362+0000 7f9562a00640 1 --2- 172.21.15.5:0/3265147542 >> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] conn(0x5563636a5cd0 0x5563636a80c0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=d89f07377eaceb3b server_cookie=b7de83d383ca1f6e in_seq=0 out_seq=0 2024-09-10T09:51:19.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:19.362+0000 7f954d600640 10 client.27986 ms_handle_connect on v2:172.21.15.102:6836/2689776764 2024-09-10T09:51:19.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:19.363+0000 7f954d600640 1 -- 172.21.15.5:0/3265147542 <== mds.3 v2:172.21.15.102:6836/2689776764 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5563635fe950 con 0x5563636a5cd0 2024-09-10T09:51:19.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:19.363+0000 7f954d600640 10 client.27986 handle_command_reply: tid=0 2024-09-10T09:51:19.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:19.363+0000 7f956cfec740 1 client.27986 shutdown 2024-09-10T09:51:19.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:19.363+0000 7f956cfec740 1 -- 172.21.15.5:0/3265147542 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55636368ac70 msgr2=0x556363692cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:51:19.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:19.363+0000 7f956cfec740 1 --2- 172.21.15.5:0/3265147542 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55636368ac70 0x556363692cf0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f9554004920 tx=0x7f9554004bc0 comp rx=0 tx=0).stop 2024-09-10T09:51:19.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:19.363+0000 7f956cfec740 1 -- 172.21.15.5:0/3265147542 shutdown_connections 2024-09-10T09:51:19.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:19.363+0000 7f956cfec740 1 --2- 172.21.15.5:0/3265147542 >> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] conn(0x5563636a5cd0 0x5563636a80c0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:19.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:19.363+0000 7f956cfec740 1 --2- 172.21.15.5:0/3265147542 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556363695e50 0x556363693770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:19.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:19.363+0000 7f956cfec740 1 --2- 172.21.15.5:0/3265147542 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55636368b590 0x556363693230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:19.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:19.363+0000 7f954d600640 0 client.27986 ms_handle_reset on v2:172.21.15.102:6836/2689776764 2024-09-10T09:51:19.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:19.363+0000 7f956cfec740 1 --2- 172.21.15.5:0/3265147542 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55636368ac70 0x556363692cf0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:19.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:19.363+0000 7f956cfec740 1 -- 172.21.15.5:0/3265147542 >> 172.21.15.5:0/3265147542 conn(0x556363687200 msgr2=0x55636368cf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:51:19.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:19.363+0000 7f956cfec740 1 -- 172.21.15.5:0/3265147542 shutdown_connections 2024-09-10T09:51:19.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:19.363+0000 7f956cfec740 1 -- 172.21.15.5:0/3265147542 wait complete. 2024-09-10T09:51:19.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:19.364+0000 7f956cfec740 20 client.27986 trim_cache size 0 max 16384 2024-09-10T09:51:19.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:19.377+0000 7f956cfec740 1 Processor -- start 2024-09-10T09:51:19.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:19.378+0000 7f956cfec740 1 -- start start 2024-09-10T09:51:19.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:19.378+0000 7f956cfec740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55636368ac70 0x556363686270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:51:19.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:19.378+0000 7f956cfec740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55636368b590 0x5563636ab0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:51:19.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:19.378+0000 7f956cfec740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556363695e50 0x5563636ab620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:51:19.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:19.378+0000 7f956cfec740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f95641b1010 con 0x55636368ac70 2024-09-10T09:51:19.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:19.378+0000 7f956cfec740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f95641b2b50 con 0x55636368b590 2024-09-10T09:51:19.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:19.378+0000 7f956cfec740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f95641b16e0 con 0x556363695e50 2024-09-10T09:51:19.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:19.378+0000 7f9563400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55636368ac70 0x556363686270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:51:19.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:19.378+0000 7f9563e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556363695e50 0x5563636ab620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:51:19.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:19.378+0000 7f9563400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55636368ac70 0x556363686270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46692/0 (socket says 172.21.15.5:46692) 2024-09-10T09:51:19.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:19.378+0000 7f9563e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556363695e50 0x5563636ab620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:51992/0 (socket says 172.21.15.5:51992) 2024-09-10T09:51:19.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:19.378+0000 7f9563400640 1 -- 172.21.15.5:0/328889170 learned_addr learned my addr 172.21.15.5:0/328889170 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:51:19.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:19.378+0000 7f9562a00640 1 --2- 172.21.15.5:0/328889170 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55636368b590 0x5563636ab0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:51:19.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:19.379+0000 7f9563e00640 1 -- 172.21.15.5:0/328889170 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55636368b590 msgr2=0x5563636ab0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:51:19.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:19.379+0000 7f9563e00640 1 --2- 172.21.15.5:0/328889170 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55636368b590 0x5563636ab0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:19.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:19.379+0000 7f9563e00640 1 -- 172.21.15.5:0/328889170 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55636368ac70 msgr2=0x556363686270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:51:19.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:19.379+0000 7f9563e00640 1 --2- 172.21.15.5:0/328889170 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55636368ac70 0x556363686270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:19.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:19.379+0000 7f9563e00640 1 -- 172.21.15.5:0/328889170 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55636368b3c0 con 0x556363695e50 2024-09-10T09:51:19.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:19.379+0000 7f9562a00640 1 --2- 172.21.15.5:0/328889170 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55636368b590 0x5563636ab0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:51:19.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:19.379+0000 7f9563400640 1 --2- 172.21.15.5:0/328889170 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55636368ac70 0x556363686270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:51:19.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:19.379+0000 7f9563e00640 1 --2- 172.21.15.5:0/328889170 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556363695e50 0x5563636ab620 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f955c012cb0 tx=0x7f955c015000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6f4f9300e53bbbb6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:51:19.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:19.379+0000 7f954fe00640 1 -- 172.21.15.5:0/328889170 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f955c003b70 con 0x556363695e50 2024-09-10T09:51:19.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:19.379+0000 7f954fe00640 1 -- 172.21.15.5:0/328889170 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f955c003d10 con 0x556363695e50 2024-09-10T09:51:19.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:19.379+0000 7f954fe00640 1 -- 172.21.15.5:0/328889170 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f955c01daf0 con 0x556363695e50 2024-09-10T09:51:19.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:19.380+0000 7f956cfec740 1 -- 172.21.15.5:0/328889170 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556363695e50 msgr2=0x5563636ab620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:51:19.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:19.380+0000 7f956cfec740 1 --2- 172.21.15.5:0/328889170 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556363695e50 0x5563636ab620 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f955c012cb0 tx=0x7f955c015000 comp rx=0 tx=0).stop 2024-09-10T09:51:19.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:19.380+0000 7f956cfec740 1 -- 172.21.15.5:0/328889170 shutdown_connections 2024-09-10T09:51:19.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:19.380+0000 7f956cfec740 1 --2- 172.21.15.5:0/328889170 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556363695e50 0x5563636ab620 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:19.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:19.380+0000 7f956cfec740 1 --2- 172.21.15.5:0/328889170 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55636368b590 0x5563636ab0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:19.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:19.380+0000 7f956cfec740 1 --2- 172.21.15.5:0/328889170 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55636368ac70 0x556363686270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:19.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:19.380+0000 7f956cfec740 1 -- 172.21.15.5:0/328889170 >> 172.21.15.5:0/328889170 conn(0x556363687200 msgr2=0x556363685b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:51:19.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:19.380+0000 7f956cfec740 1 -- 172.21.15.5:0/328889170 shutdown_connections 2024-09-10T09:51:19.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:19.380+0000 7f956cfec740 1 -- 172.21.15.5:0/328889170 wait complete. 2024-09-10T09:51:19.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:19.381+0000 7f956cfec740 1 -- start start 2024-09-10T09:51:19.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:19.381+0000 7f956cfec740 1 Processor -- start 2024-09-10T09:51:19.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:19.381+0000 7f956cfec740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55636368ac70 0x5563636ab4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:51:19.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:19.381+0000 7f956cfec740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55636368b590 0x5563636aba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:51:19.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:19.381+0000 7f956cfec740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556363695e50 0x5563636a8a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:51:19.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:19.381+0000 7f956cfec740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f95641b2480 con 0x55636368b590 2024-09-10T09:51:19.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:19.381+0000 7f956cfec740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f95641b1db0 con 0x55636368ac70 2024-09-10T09:51:19.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:19.381+0000 7f956cfec740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f95641aeff0 con 0x556363695e50 2024-09-10T09:51:19.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:19.381+0000 7f956cfec740 10 client.0 _ll_register_callbacks cb 0x7f955c009600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:51:19.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:19.382+0000 7f9563400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55636368ac70 0x5563636ab4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:51:19.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:19.382+0000 7f9563e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556363695e50 0x5563636a8a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:51:19.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:19.382+0000 7f9563400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55636368ac70 0x5563636ab4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:33682/0 (socket says 172.21.15.5:33682) 2024-09-10T09:51:19.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:19.382+0000 7f9563400640 1 -- 172.21.15.5:0/1617262242 learned_addr learned my addr 172.21.15.5:0/1617262242 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:51:19.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:19.382+0000 7f9562a00640 1 --2- 172.21.15.5:0/1617262242 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55636368b590 0x5563636aba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:51:19.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:19.382+0000 7f9563400640 1 -- 172.21.15.5:0/1617262242 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556363695e50 msgr2=0x5563636a8a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:51:19.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:19.382+0000 7f9563400640 1 --2- 172.21.15.5:0/1617262242 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556363695e50 0x5563636a8a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:19.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:19.382+0000 7f9563400640 1 -- 172.21.15.5:0/1617262242 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55636368b590 msgr2=0x5563636aba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:51:19.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:19.382+0000 7f9563400640 1 --2- 172.21.15.5:0/1617262242 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55636368b590 0x5563636aba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:19.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:19.382+0000 7f9563400640 1 -- 172.21.15.5:0/1617262242 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5563636a92b0 con 0x55636368ac70 2024-09-10T09:51:19.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:19.382+0000 7f9563400640 1 --2- 172.21.15.5:0/1617262242 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55636368ac70 0x5563636ab4e0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f9554003f00 tx=0x7f9554004b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=191b66654751e53f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:51:19.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:19.382+0000 7f954d600640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-09-10T09:51:19.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:19.383+0000 7f954d600640 1 -- 172.21.15.5:0/1617262242 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9554052070 con 0x55636368ac70 2024-09-10T09:51:19.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:19.383+0000 7f954d600640 1 -- 172.21.15.5:0/1617262242 <== mon.2 v2:172.21.15.102:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f955403cad0 con 0x55636368ac70 2024-09-10T09:51:19.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:19.383+0000 7f956cfec740 1 -- 172.21.15.5:0/1617262242 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f955c003d10 con 0x55636368ac70 2024-09-10T09:51:19.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:19.383+0000 7f954d600640 1 -- 172.21.15.5:0/1617262242 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f9554009520 con 0x55636368ac70 2024-09-10T09:51:19.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:19.383+0000 7f956cfec740 10 client.27992 fetch_fsmap learned FSMap version 36 2024-09-10T09:51:19.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:19.383+0000 7f956cfec740 1 -- 172.21.15.5:0/1617262242 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x556363710320 con 0x55636368ac70 2024-09-10T09:51:19.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:19.383+0000 7f954d600640 1 -- 172.21.15.5:0/1617262242 <== mon.2 v2:172.21.15.102:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f955c003d10 con 0x55636368ac70 2024-09-10T09:51:19.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:19.383+0000 7f954d600640 1 -- 172.21.15.5:0/1617262242 <== mon.2 v2:172.21.15.102:3300/0 5 ==== fsmap(e 36) ==== 8370+0+0 (secure 0 0 0) 0x7f95540315b0 con 0x55636368ac70 2024-09-10T09:51:19.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:19.383+0000 7f956cfec740 10 client.27992 fetch_fsmap finished waiting for FSMap version 36 2024-09-10T09:51:19.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:19.383+0000 7f956cfec740 10 client.27992 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-09-10T09:51:19.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:19.383+0000 7f956cfec740 20 client.27992 populate_metadata read hostname 'smithi005' 2024-09-10T09:51:19.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:19.383+0000 7f956cfec740 1 --2- 172.21.15.5:0/1617262242 >> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] conn(0x556363710900 0x556363712cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:51:19.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:19.383+0000 7f956cfec740 4 client.27992 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T09:51:19.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:19.383+0000 7f956cfec740 1 -- 172.21.15.5:0/1617262242 --> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5563635f8e60 con 0x556363710900 2024-09-10T09:51:19.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:19.384+0000 7f9562a00640 1 --2- 172.21.15.5:0/1617262242 >> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] conn(0x556363710900 0x556363712cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:51:19.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:19.384+0000 7f9562a00640 1 --2- 172.21.15.5:0/1617262242 >> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] conn(0x556363710900 0x556363712cf0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=a886e711bf3f799d server_cookie=362dc6f8160d268f in_seq=0 out_seq=0 2024-09-10T09:51:19.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:19.384+0000 7f954d600640 10 client.27992 ms_handle_connect on v2:172.21.15.102:6836/2689776764 2024-09-10T09:51:19.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:19.384+0000 7f954d600640 1 -- 172.21.15.5:0/1617262242 <== mds.3 v2:172.21.15.102:6836/2689776764 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5563635f8e60 con 0x556363710900 2024-09-10T09:51:19.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:19.384+0000 7f954d600640 10 client.27992 handle_command_reply: tid=0 2024-09-10T09:51:19.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:19.384+0000 7f956cfec740 1 client.27992 shutdown 2024-09-10T09:51:19.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:19.385+0000 7f956cfec740 1 -- 172.21.15.5:0/1617262242 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55636368ac70 msgr2=0x5563636ab4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:51:19.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:19.385+0000 7f956cfec740 1 --2- 172.21.15.5:0/1617262242 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55636368ac70 0x5563636ab4e0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f9554003f00 tx=0x7f9554004b50 comp rx=0 tx=0).stop 2024-09-10T09:51:19.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:19.385+0000 7f956cfec740 1 -- 172.21.15.5:0/1617262242 shutdown_connections 2024-09-10T09:51:19.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:19.385+0000 7f956cfec740 1 --2- 172.21.15.5:0/1617262242 >> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] conn(0x556363710900 0x556363712cf0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:19.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:19.385+0000 7f956cfec740 1 --2- 172.21.15.5:0/1617262242 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556363695e50 0x5563636a8a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:19.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:19.385+0000 7f956cfec740 1 --2- 172.21.15.5:0/1617262242 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55636368b590 0x5563636aba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:19.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:19.385+0000 7f956cfec740 1 --2- 172.21.15.5:0/1617262242 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55636368ac70 0x5563636ab4e0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:19.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:19.385+0000 7f954d600640 0 client.27992 ms_handle_reset on v2:172.21.15.102:6836/2689776764 2024-09-10T09:51:19.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:19.385+0000 7f956cfec740 1 -- 172.21.15.5:0/1617262242 >> 172.21.15.5:0/1617262242 conn(0x556363687200 msgr2=0x556363685b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:51:19.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:19.385+0000 7f956cfec740 1 -- 172.21.15.5:0/1617262242 shutdown_connections 2024-09-10T09:51:19.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:19.385+0000 7f956cfec740 1 -- 172.21.15.5:0/1617262242 wait complete. 2024-09-10T09:51:19.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:19.385+0000 7f956cfec740 20 client.27992 trim_cache size 0 max 16384 2024-09-10T09:51:19.391 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-10T09:51:19.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:19.386+0000 7f956a600640 1 -- 172.21.15.5:0/1031192489 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f95480a2b80 msgr2=0x7f95480a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:51:19.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:19.386+0000 7f956a600640 1 --2- 172.21.15.5:0/1031192489 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f95480a2b80 0x7f95480a5040 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f955c005fd0 tx=0x7f955c009760 comp rx=0 tx=0).stop 2024-09-10T09:51:19.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:19.386+0000 7f956a600640 1 -- 172.21.15.5:0/1031192489 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f95641049c0 msgr2=0x7f956419ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:51:19.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:19.386+0000 7f956a600640 1 --2- 172.21.15.5:0/1031192489 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f95641049c0 0x7f956419ac10 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f955000e9f0 tx=0x7f955000eeb0 comp rx=0 tx=0).stop 2024-09-10T09:51:19.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:19.386+0000 7f956a600640 1 -- 172.21.15.5:0/1031192489 shutdown_connections 2024-09-10T09:51:19.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:19.386+0000 7f956a600640 1 --2- 172.21.15.5:0/1031192489 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f95480a2b80 0x7f95480a5040 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:19.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:19.386+0000 7f956a600640 1 --2- 172.21.15.5:0/1031192489 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9564109140 0x7f956419efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:19.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:19.386+0000 7f956a600640 1 --2- 172.21.15.5:0/1031192489 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f95641049c0 0x7f956419ac10 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:19.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:19.386+0000 7f956a600640 1 --2- 172.21.15.5:0/1031192489 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9564104010 0x7f956419a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:19.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:19.386+0000 7f956a600640 1 -- 172.21.15.5:0/1031192489 >> 172.21.15.5:0/1031192489 conn(0x7f95640ffa50 msgr2=0x7f9564100e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:51:19.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:19.386+0000 7f956a600640 1 -- 172.21.15.5:0/1031192489 shutdown_connections 2024-09-10T09:51:19.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:19.386+0000 7f956a600640 1 -- 172.21.15.5:0/1031192489 wait complete. 2024-09-10T09:51:20.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:51:19 smithi005 ceph-mon[30350]: pgmap v782: 129 pgs: 129 active+clean; 300 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 1.6 MiB/s wr, 184 op/s 2024-09-10T09:51:20.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:51:19 smithi159 ceph-mon[35513]: pgmap v782: 129 pgs: 129 active+clean; 300 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 1.6 MiB/s wr, 184 op/s 2024-09-10T09:51:20.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:51:19 smithi102 ceph-mon[35442]: pgmap v782: 129 pgs: 129 active+clean; 300 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 1.6 MiB/s wr, 184 op/s 2024-09-10T09:51:20.147 INFO:tasks.workunit.client.0.smithi005.stdout: 1 832692 6.41 MB/sec execute 541 sec latency 60.794 ms 2024-09-10T09:51:20.297 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-10T09:51:20.714 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:51:21.148 INFO:tasks.workunit.client.0.smithi005.stdout: 1 834897 6.42 MB/sec execute 542 sec latency 32.815 ms 2024-09-10T09:51:21.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:21.314+0000 7f2896a00640 1 -- 172.21.15.5:0/1171414371 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2890103550 msgr2=0x7f289010ba40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:51:21.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:21.314+0000 7f2896a00640 1 --2- 172.21.15.5:0/1171414371 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2890103550 0x7f289010ba40 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f287c009860 tx=0x7f287c02a3d0 comp rx=0 tx=0).stop 2024-09-10T09:51:21.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:21.315+0000 7f2896a00640 1 -- 172.21.15.5:0/1171414371 shutdown_connections 2024-09-10T09:51:21.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:21.315+0000 7f2896a00640 1 --2- 172.21.15.5:0/1171414371 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f289010bfd0 0x7f289010e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:21.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:21.315+0000 7f2896a00640 1 --2- 172.21.15.5:0/1171414371 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2890103550 0x7f289010ba40 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:21.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:21.315+0000 7f2896a00640 1 --2- 172.21.15.5:0/1171414371 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2890102c30 0x7f2890103010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:21.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:21.315+0000 7f2896a00640 1 -- 172.21.15.5:0/1171414371 >> 172.21.15.5:0/1171414371 conn(0x7f28900fc610 msgr2=0x7f28900fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:51:21.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:21.315+0000 7f2896a00640 1 -- 172.21.15.5:0/1171414371 shutdown_connections 2024-09-10T09:51:21.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:21.315+0000 7f2896a00640 1 -- 172.21.15.5:0/1171414371 wait complete. 2024-09-10T09:51:21.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:21.316+0000 7f2896a00640 1 Processor -- start 2024-09-10T09:51:21.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:21.316+0000 7f2896a00640 1 -- start start 2024-09-10T09:51:21.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:21.316+0000 7f2896a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2890102c30 0x7f28901982d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:51:21.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:21.316+0000 7f2896a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2890103550 0x7f2890198810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:51:21.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:21.316+0000 7f2896a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f289010bfd0 0x7f289019cba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:51:21.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:21.316+0000 7f2896a00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f2890111dd0 con 0x7f289010bfd0 2024-09-10T09:51:21.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:21.316+0000 7f2896a00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f2890111c50 con 0x7f2890103550 2024-09-10T09:51:21.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:21.316+0000 7f2896a00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f2890111f50 con 0x7f2890102c30 2024-09-10T09:51:21.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:21.317+0000 7f2895600640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2890102c30 0x7f28901982d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:51:21.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:21.317+0000 7f2895600640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2890102c30 0x7f28901982d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:43718/0 (socket says 172.21.15.5:43718) 2024-09-10T09:51:21.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:21.317+0000 7f2894c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2890103550 0x7f2890198810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:51:21.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:21.317+0000 7f2895600640 1 -- 172.21.15.5:0/509059712 learned_addr learned my addr 172.21.15.5:0/509059712 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:51:21.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:21.317+0000 7f2896000640 1 --2- 172.21.15.5:0/509059712 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f289010bfd0 0x7f289019cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:51:21.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:21.317+0000 7f2895600640 1 -- 172.21.15.5:0/509059712 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2890103550 msgr2=0x7f2890198810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:51:21.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:21.317+0000 7f2895600640 1 --2- 172.21.15.5:0/509059712 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2890103550 0x7f2890198810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:21.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:21.317+0000 7f2895600640 1 -- 172.21.15.5:0/509059712 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f289010bfd0 msgr2=0x7f289019cba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:51:21.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:21.317+0000 7f2895600640 1 --2- 172.21.15.5:0/509059712 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f289010bfd0 0x7f289019cba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:21.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:21.317+0000 7f2895600640 1 -- 172.21.15.5:0/509059712 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f289019d280 con 0x7f2890102c30 2024-09-10T09:51:21.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:21.317+0000 7f2895600640 1 --2- 172.21.15.5:0/509059712 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2890102c30 0x7f28901982d0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f288400e9f0 tx=0x7f288400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:51:21.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:21.318+0000 7f288e000640 1 -- 172.21.15.5:0/509059712 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f288400cd80 con 0x7f2890102c30 2024-09-10T09:51:21.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:21.318+0000 7f288e000640 1 -- 172.21.15.5:0/509059712 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2884010910 con 0x7f2890102c30 2024-09-10T09:51:21.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:21.318+0000 7f2896a00640 1 -- 172.21.15.5:0/509059712 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f289019d570 con 0x7f2890102c30 2024-09-10T09:51:21.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:21.318+0000 7f288e000640 1 -- 172.21.15.5:0/509059712 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2884018980 con 0x7f2890102c30 2024-09-10T09:51:21.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:21.318+0000 7f2896a00640 1 -- 172.21.15.5:0/509059712 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28901a4e50 con 0x7f2890102c30 2024-09-10T09:51:21.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:21.322+0000 7f288e000640 1 -- 172.21.15.5:0/509059712 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f2884010ac0 con 0x7f2890102c30 2024-09-10T09:51:21.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:21.322+0000 7f289ac2d740 1 Processor -- start 2024-09-10T09:51:21.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:21.322+0000 7f289ac2d740 1 -- start start 2024-09-10T09:51:21.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:21.322+0000 7f289ac2d740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d23f80fe40 0x55d23f810220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:51:21.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:21.322+0000 7f289ac2d740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d23f810760 0x55d23f81aae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:51:21.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:21.322+0000 7f289ac2d740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d23f81b020 0x55d23f81d410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:51:21.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:21.322+0000 7f289ac2d740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55d23f7763e0 con 0x55d23f810760 2024-09-10T09:51:21.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:21.322+0000 7f289ac2d740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55d23f71dd80 con 0x55d23f81b020 2024-09-10T09:51:21.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:21.322+0000 7f289ac2d740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55d23f7723a0 con 0x55d23f80fe40 2024-09-10T09:51:21.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:21.323+0000 7f2896000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d23f80fe40 0x55d23f810220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:51:21.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:21.323+0000 7f2896000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d23f80fe40 0x55d23f810220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:43724/0 (socket says 172.21.15.5:43724) 2024-09-10T09:51:21.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:21.323+0000 7f2895600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d23f810760 0x55d23f81aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:51:21.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:21.323+0000 7f2896000640 1 -- 172.21.15.5:0/758897984 learned_addr learned my addr 172.21.15.5:0/758897984 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:51:21.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:21.323+0000 7f2894c00640 1 --2- 172.21.15.5:0/758897984 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d23f81b020 0x55d23f81d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:51:21.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:21.323+0000 7f2896000640 1 -- 172.21.15.5:0/758897984 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d23f81b020 msgr2=0x55d23f81d410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:51:21.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:21.323+0000 7f2896000640 1 --2- 172.21.15.5:0/758897984 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d23f81b020 0x55d23f81d410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:21.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:21.323+0000 7f2896000640 1 -- 172.21.15.5:0/758897984 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d23f810760 msgr2=0x55d23f81aae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:51:21.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:21.323+0000 7f2896000640 1 --2- 172.21.15.5:0/758897984 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d23f810760 0x55d23f81aae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:21.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:21.323+0000 7f2896000640 1 -- 172.21.15.5:0/758897984 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d23f73cae0 con 0x55d23f80fe40 2024-09-10T09:51:21.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:21.323+0000 7f2894c00640 1 --2- 172.21.15.5:0/758897984 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d23f81b020 0x55d23f81d410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:51:21.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:21.324+0000 7f288e000640 1 --2- 172.21.15.5:0/509059712 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f28780a2b80 0x7f28780a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:51:21.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:21.324+0000 7f2896000640 1 --2- 172.21.15.5:0/758897984 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d23f80fe40 0x55d23f810220 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f2888009ee0 tx=0x7f288800f7d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=688c6d8e14a8c6f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:51:21.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:21.324+0000 7f288e000640 1 -- 172.21.15.5:0/509059712 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f28840cf820 con 0x7f2890102c30 2024-09-10T09:51:21.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:21.324+0000 7f2895600640 1 --2- 172.21.15.5:0/509059712 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f28780a2b80 0x7f28780a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:51:21.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:21.324+0000 7f2882600640 1 -- 172.21.15.5:0/758897984 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f288801d070 con 0x55d23f80fe40 2024-09-10T09:51:21.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:21.324+0000 7f2882600640 1 -- 172.21.15.5:0/758897984 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f288800dad0 con 0x55d23f80fe40 2024-09-10T09:51:21.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:21.324+0000 7f2882600640 1 -- 172.21.15.5:0/758897984 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2888018cb0 con 0x55d23f80fe40 2024-09-10T09:51:21.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:21.325+0000 7f289ac2d740 1 -- 172.21.15.5:0/758897984 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d23f80fe40 msgr2=0x55d23f810220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:51:21.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:21.325+0000 7f2895600640 1 --2- 172.21.15.5:0/509059712 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f28780a2b80 0x7f28780a5040 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f2884098600 tx=0x7f28840d5530 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:51:21.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:21.325+0000 7f289ac2d740 1 --2- 172.21.15.5:0/758897984 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d23f80fe40 0x55d23f810220 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f2888009ee0 tx=0x7f288800f7d0 comp rx=0 tx=0).stop 2024-09-10T09:51:21.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:21.325+0000 7f289ac2d740 1 -- 172.21.15.5:0/758897984 shutdown_connections 2024-09-10T09:51:21.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:21.325+0000 7f289ac2d740 1 --2- 172.21.15.5:0/758897984 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d23f81b020 0x55d23f81d410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:21.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:21.325+0000 7f289ac2d740 1 --2- 172.21.15.5:0/758897984 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d23f810760 0x55d23f81aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:21.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:21.325+0000 7f289ac2d740 1 --2- 172.21.15.5:0/758897984 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d23f80fe40 0x55d23f810220 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:21.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:21.325+0000 7f289ac2d740 1 -- 172.21.15.5:0/758897984 >> 172.21.15.5:0/758897984 conn(0x55d23f80c3d0 msgr2=0x55d23f80c7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:51:21.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:21.325+0000 7f289ac2d740 1 -- 172.21.15.5:0/758897984 shutdown_connections 2024-09-10T09:51:21.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:21.325+0000 7f289ac2d740 1 -- 172.21.15.5:0/758897984 wait complete. 2024-09-10T09:51:21.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:21.326+0000 7f289ac2d740 1 -- start start 2024-09-10T09:51:21.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:21.326+0000 7f289ac2d740 1 Processor -- start 2024-09-10T09:51:21.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:21.326+0000 7f289ac2d740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d23f80fe40 0x55d23f81f590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:51:21.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:21.326+0000 7f289ac2d740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d23f810760 0x55d23f81fad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:51:21.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:21.326+0000 7f289ac2d740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d23f81b020 0x55d23f823c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:51:21.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:21.326+0000 7f289ac2d740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55d23f81dbf0 con 0x55d23f810760 2024-09-10T09:51:21.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:21.326+0000 7f289ac2d740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55d23f7310f0 con 0x55d23f81b020 2024-09-10T09:51:21.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:21.326+0000 7f289ac2d740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55d23f81dd70 con 0x55d23f80fe40 2024-09-10T09:51:21.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:21.327+0000 7f289ac2d740 10 client.0 _ll_register_callbacks cb 0x55d23f718bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:51:21.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:21.327+0000 7f2896000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d23f80fe40 0x55d23f81f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:51:21.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:21.327+0000 7f2896000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d23f80fe40 0x55d23f81f590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:43730/0 (socket says 172.21.15.5:43730) 2024-09-10T09:51:21.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:21.327+0000 7f2896000640 1 -- 172.21.15.5:0/4204506371 learned_addr learned my addr 172.21.15.5:0/4204506371 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:51:21.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:21.327+0000 7f2894c00640 1 --2- 172.21.15.5:0/4204506371 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d23f810760 0x55d23f81fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:51:21.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:21.327+0000 7f2896000640 1 --2- 172.21.15.5:0/4204506371 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d23f81b020 0x55d23f823c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:51:21.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:21.327+0000 7f2894c00640 1 -- 172.21.15.5:0/4204506371 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d23f80fe40 msgr2=0x55d23f81f590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:51:21.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:21.327+0000 7f2894c00640 1 --2- 172.21.15.5:0/4204506371 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d23f80fe40 0x55d23f81f590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:21.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:21.327+0000 7f2894c00640 1 -- 172.21.15.5:0/4204506371 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d23f81b020 msgr2=0x55d23f823c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:51:21.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:21.327+0000 7f2894c00640 1 --2- 172.21.15.5:0/4204506371 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d23f81b020 0x55d23f823c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:21.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:21.327+0000 7f2894c00640 1 -- 172.21.15.5:0/4204506371 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d23f4e73c0 con 0x55d23f810760 2024-09-10T09:51:21.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:21.327+0000 7f2896000640 1 --2- 172.21.15.5:0/4204506371 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d23f80fe40 0x55d23f81f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:51:21.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:21.328+0000 7f2894c00640 1 --2- 172.21.15.5:0/4204506371 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d23f810760 0x55d23f81fad0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f287c009830 tx=0x7f287c0305d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5320f689e5e4878 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:51:21.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:21.328+0000 7f2876a00640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-10T09:51:21.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:21.328+0000 7f2876a00640 1 -- 172.21.15.5:0/4204506371 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f287c004030 con 0x55d23f810760 2024-09-10T09:51:21.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:21.328+0000 7f289ac2d740 1 -- 172.21.15.5:0/4204506371 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f288800dad0 con 0x55d23f810760 2024-09-10T09:51:21.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:21.328+0000 7f2876a00640 1 -- 172.21.15.5:0/4204506371 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f287c0041d0 con 0x55d23f810760 2024-09-10T09:51:21.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:21.328+0000 7f2876a00640 1 -- 172.21.15.5:0/4204506371 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f287c04f050 con 0x55d23f810760 2024-09-10T09:51:21.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:21.328+0000 7f289ac2d740 10 client.17976 fetch_fsmap learned FSMap version 36 2024-09-10T09:51:21.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:21.328+0000 7f289ac2d740 1 -- 172.21.15.5:0/4204506371 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55d23f82e130 con 0x55d23f810760 2024-09-10T09:51:21.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:21.329+0000 7f2876a00640 1 -- 172.21.15.5:0/4204506371 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f288800dad0 con 0x55d23f810760 2024-09-10T09:51:21.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:21.329+0000 7f2876a00640 1 -- 172.21.15.5:0/4204506371 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 36) ==== 8370+0+0 (secure 0 0 0) 0x7f287c0049c0 con 0x55d23f810760 2024-09-10T09:51:21.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:21.329+0000 7f289ac2d740 10 client.17976 fetch_fsmap finished waiting for FSMap version 36 2024-09-10T09:51:21.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:21.329+0000 7f289ac2d740 10 client.17976 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-10T09:51:21.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:21.329+0000 7f289ac2d740 20 client.17976 populate_metadata read hostname 'smithi005' 2024-09-10T09:51:21.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:21.329+0000 7f289ac2d740 1 --2- 172.21.15.5:0/4204506371 >> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] conn(0x55d23f82f590 0x55d23f831980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:51:21.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:21.329+0000 7f289ac2d740 4 client.17976 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:51:21.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:21.329+0000 7f289ac2d740 1 -- 172.21.15.5:0/4204506371 --> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d23f783950 con 0x55d23f82f590 2024-09-10T09:51:21.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:21.329+0000 7f2896000640 1 --2- 172.21.15.5:0/4204506371 >> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] conn(0x55d23f82f590 0x55d23f831980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:51:21.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:21.330+0000 7f2896000640 1 --2- 172.21.15.5:0/4204506371 >> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] conn(0x55d23f82f590 0x55d23f831980 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7cb9317451d8b15b server_cookie=577315e5ab7a66c9 in_seq=0 out_seq=0 2024-09-10T09:51:21.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:21.330+0000 7f2876a00640 10 client.17976 ms_handle_connect on v2:172.21.15.159:6836/620627639 2024-09-10T09:51:21.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:21.331+0000 7f2876a00640 1 -- 172.21.15.5:0/4204506371 <== mds.2 v2:172.21.15.159:6836/620627639 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d23f783950 con 0x55d23f82f590 2024-09-10T09:51:21.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:21.331+0000 7f2876a00640 10 client.17976 handle_command_reply: tid=0 2024-09-10T09:51:21.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:21.331+0000 7f289ac2d740 1 client.17976 shutdown 2024-09-10T09:51:21.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:21.332+0000 7f289ac2d740 1 -- 172.21.15.5:0/4204506371 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d23f810760 msgr2=0x55d23f81fad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:51:21.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:21.332+0000 7f289ac2d740 1 --2- 172.21.15.5:0/4204506371 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d23f810760 0x55d23f81fad0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f287c009830 tx=0x7f287c0305d0 comp rx=0 tx=0).stop 2024-09-10T09:51:21.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:21.332+0000 7f289ac2d740 1 -- 172.21.15.5:0/4204506371 shutdown_connections 2024-09-10T09:51:21.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:21.332+0000 7f289ac2d740 1 --2- 172.21.15.5:0/4204506371 >> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] conn(0x55d23f82f590 0x55d23f831980 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:21.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:21.332+0000 7f289ac2d740 1 --2- 172.21.15.5:0/4204506371 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d23f81b020 0x55d23f823c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:21.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:21.332+0000 7f289ac2d740 1 --2- 172.21.15.5:0/4204506371 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d23f810760 0x55d23f81fad0 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:21.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:21.332+0000 7f289ac2d740 1 --2- 172.21.15.5:0/4204506371 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d23f80fe40 0x55d23f81f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:21.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:21.332+0000 7f2876a00640 0 client.17976 ms_handle_reset on v2:172.21.15.159:6836/620627639 2024-09-10T09:51:21.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:21.332+0000 7f289ac2d740 1 -- 172.21.15.5:0/4204506371 >> 172.21.15.5:0/4204506371 conn(0x55d23f80c3d0 msgr2=0x55d23f812100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:51:21.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:21.332+0000 7f289ac2d740 1 -- 172.21.15.5:0/4204506371 shutdown_connections 2024-09-10T09:51:21.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:21.332+0000 7f289ac2d740 1 -- 172.21.15.5:0/4204506371 wait complete. 2024-09-10T09:51:21.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:21.332+0000 7f289ac2d740 20 client.17976 trim_cache size 0 max 16384 2024-09-10T09:51:21.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:21.347+0000 7f289ac2d740 1 Processor -- start 2024-09-10T09:51:21.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:21.347+0000 7f289ac2d740 1 -- start start 2024-09-10T09:51:21.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:21.348+0000 7f289ac2d740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d23f80fe40 0x55d23f81f590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:51:21.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:21.348+0000 7f289ac2d740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d23f810760 0x55d23f837250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:51:21.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:21.348+0000 7f289ac2d740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d23f81b020 0x55d23f837790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:51:21.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:21.348+0000 7f289ac2d740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f28901acb50 con 0x55d23f80fe40 2024-09-10T09:51:21.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:21.348+0000 7f289ac2d740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f28901ae690 con 0x55d23f810760 2024-09-10T09:51:21.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:21.348+0000 7f289ac2d740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f28901ad220 con 0x55d23f81b020 2024-09-10T09:51:21.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:21.348+0000 7f2894c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d23f810760 0x55d23f837250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:51:21.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:21.348+0000 7f2894c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d23f810760 0x55d23f837250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:52370/0 (socket says 172.21.15.5:52370) 2024-09-10T09:51:21.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:21.348+0000 7f2894c00640 1 -- 172.21.15.5:0/4002284490 learned_addr learned my addr 172.21.15.5:0/4002284490 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:51:21.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:21.348+0000 7f2896000640 1 --2- 172.21.15.5:0/4002284490 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d23f80fe40 0x55d23f81f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:51:21.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:21.348+0000 7f2896000640 1 --2- 172.21.15.5:0/4002284490 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d23f81b020 0x55d23f837790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:51:21.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:21.348+0000 7f2894c00640 1 -- 172.21.15.5:0/4002284490 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d23f81b020 msgr2=0x55d23f837790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:51:21.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:21.348+0000 7f2894c00640 1 --2- 172.21.15.5:0/4002284490 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d23f81b020 0x55d23f837790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:21.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:21.348+0000 7f2894c00640 1 -- 172.21.15.5:0/4002284490 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d23f80fe40 msgr2=0x55d23f81f590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:51:21.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:21.348+0000 7f2894c00640 1 --2- 172.21.15.5:0/4002284490 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d23f80fe40 0x55d23f81f590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:21.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:21.348+0000 7f2894c00640 1 -- 172.21.15.5:0/4002284490 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d23f80fc70 con 0x55d23f810760 2024-09-10T09:51:21.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:21.348+0000 7f2896000640 1 --2- 172.21.15.5:0/4002284490 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d23f80fe40 0x55d23f81f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:51:21.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:21.349+0000 7f2894c00640 1 --2- 172.21.15.5:0/4002284490 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d23f810760 0x55d23f837250 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f287c009780 tx=0x7f287c02a3d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b1fdcf4cd4751706 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:51:21.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:21.349+0000 7f2881c00640 1 -- 172.21.15.5:0/4002284490 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f287c059070 con 0x55d23f810760 2024-09-10T09:51:21.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:21.349+0000 7f2881c00640 1 -- 172.21.15.5:0/4002284490 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f287c04c070 con 0x55d23f810760 2024-09-10T09:51:21.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:21.349+0000 7f2881c00640 1 -- 172.21.15.5:0/4002284490 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f287c0053c0 con 0x55d23f810760 2024-09-10T09:51:21.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:21.350+0000 7f289ac2d740 1 -- 172.21.15.5:0/4002284490 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d23f810760 msgr2=0x55d23f837250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:51:21.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:21.350+0000 7f289ac2d740 1 --2- 172.21.15.5:0/4002284490 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d23f810760 0x55d23f837250 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f287c009780 tx=0x7f287c02a3d0 comp rx=0 tx=0).stop 2024-09-10T09:51:21.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:21.350+0000 7f289ac2d740 1 -- 172.21.15.5:0/4002284490 shutdown_connections 2024-09-10T09:51:21.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:21.350+0000 7f289ac2d740 1 --2- 172.21.15.5:0/4002284490 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d23f81b020 0x55d23f837790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:21.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:21.350+0000 7f289ac2d740 1 --2- 172.21.15.5:0/4002284490 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d23f810760 0x55d23f837250 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:21.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:21.350+0000 7f289ac2d740 1 --2- 172.21.15.5:0/4002284490 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d23f80fe40 0x55d23f81f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:21.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:21.350+0000 7f289ac2d740 1 -- 172.21.15.5:0/4002284490 >> 172.21.15.5:0/4002284490 conn(0x55d23f80c3d0 msgr2=0x55d23f8120b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:51:21.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:21.350+0000 7f289ac2d740 1 -- 172.21.15.5:0/4002284490 shutdown_connections 2024-09-10T09:51:21.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:21.350+0000 7f289ac2d740 1 -- 172.21.15.5:0/4002284490 wait complete. 2024-09-10T09:51:21.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:21.351+0000 7f289ac2d740 1 -- start start 2024-09-10T09:51:21.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:21.351+0000 7f289ac2d740 1 Processor -- start 2024-09-10T09:51:21.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:21.351+0000 7f289ac2d740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d23f80fe40 0x55d23f82bb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:51:21.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:21.351+0000 7f289ac2d740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d23f810760 0x55d23f827e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:51:21.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:21.352+0000 7f289ac2d740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d23f81b020 0x55d23f828430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:51:21.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:21.352+0000 7f289ac2d740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f28901adfc0 con 0x55d23f80fe40 2024-09-10T09:51:21.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:21.352+0000 7f289ac2d740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f28901ad8f0 con 0x55d23f81b020 2024-09-10T09:51:21.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:21.352+0000 7f289ac2d740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f28901aab30 con 0x55d23f810760 2024-09-10T09:51:21.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:21.352+0000 7f289ac2d740 10 client.0 _ll_register_callbacks cb 0x7f287c0096b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:51:21.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:21.352+0000 7f2896000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d23f80fe40 0x55d23f82bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:51:21.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:21.352+0000 7f2894c00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d23f810760 0x55d23f827e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:51:21.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:21.352+0000 7f2896000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d23f81b020 0x55d23f828430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:51:21.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:21.352+0000 7f2896000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d23f81b020 0x55d23f828430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:52382/0 (socket says 172.21.15.5:52382) 2024-09-10T09:51:21.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:21.352+0000 7f2896000640 1 -- 172.21.15.5:0/231007167 learned_addr learned my addr 172.21.15.5:0/231007167 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:51:21.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:21.352+0000 7f2894c00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d23f810760 0x55d23f827e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:43740/0 (socket says 172.21.15.5:43740) 2024-09-10T09:51:21.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:21.353+0000 7f2896000640 1 -- 172.21.15.5:0/231007167 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d23f810760 msgr2=0x55d23f827e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:51:21.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:21.353+0000 7f2896000640 1 --2- 172.21.15.5:0/231007167 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d23f810760 0x55d23f827e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:21.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:21.353+0000 7f2896000640 1 -- 172.21.15.5:0/231007167 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d23f80fe40 msgr2=0x55d23f82bb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:51:21.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:21.353+0000 7f2896000640 1 --2- 172.21.15.5:0/231007167 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d23f80fe40 0x55d23f82bb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:21.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:21.353+0000 7f2896000640 1 -- 172.21.15.5:0/231007167 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d23f80bc80 con 0x55d23f81b020 2024-09-10T09:51:21.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:21.353+0000 7f2896000640 1 --2- 172.21.15.5:0/231007167 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d23f81b020 0x55d23f828430 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f2888009e80 tx=0x7f288800f7d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2e101ed79071e357 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:51:21.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:21.353+0000 7f2876a00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-09-10T09:51:21.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:21.353+0000 7f2876a00640 1 -- 172.21.15.5:0/231007167 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f288801d040 con 0x55d23f81b020 2024-09-10T09:51:21.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:21.353+0000 7f2876a00640 1 -- 172.21.15.5:0/231007167 <== mon.2 v2:172.21.15.102:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f2888015070 con 0x55d23f81b020 2024-09-10T09:51:21.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:21.354+0000 7f289ac2d740 1 -- 172.21.15.5:0/231007167 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f287c04c070 con 0x55d23f81b020 2024-09-10T09:51:21.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:21.355+0000 7f2876a00640 1 -- 172.21.15.5:0/231007167 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f288800d830 con 0x55d23f81b020 2024-09-10T09:51:21.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:21.355+0000 7f289ac2d740 10 client.28010 fetch_fsmap learned FSMap version 36 2024-09-10T09:51:21.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:21.355+0000 7f289ac2d740 1 -- 172.21.15.5:0/231007167 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55d23f89b120 con 0x55d23f81b020 2024-09-10T09:51:21.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:21.355+0000 7f2876a00640 1 -- 172.21.15.5:0/231007167 <== mon.2 v2:172.21.15.102:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f287c04c070 con 0x55d23f81b020 2024-09-10T09:51:21.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:21.355+0000 7f2876a00640 1 -- 172.21.15.5:0/231007167 <== mon.2 v2:172.21.15.102:3300/0 5 ==== fsmap(e 36) ==== 8370+0+0 (secure 0 0 0) 0x7f2888023040 con 0x55d23f81b020 2024-09-10T09:51:21.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:21.355+0000 7f289ac2d740 10 client.28010 fetch_fsmap finished waiting for FSMap version 36 2024-09-10T09:51:21.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:21.355+0000 7f289ac2d740 10 client.28010 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-10T09:51:21.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:21.355+0000 7f289ac2d740 20 client.28010 populate_metadata read hostname 'smithi005' 2024-09-10T09:51:21.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:21.355+0000 7f289ac2d740 1 --2- 172.21.15.5:0/231007167 >> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] conn(0x55d23f89b560 0x55d23f89d950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:51:21.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:21.355+0000 7f289ac2d740 4 client.28010 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T09:51:21.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:21.355+0000 7f289ac2d740 1 -- 172.21.15.5:0/231007167 --> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d23f77de60 con 0x55d23f89b560 2024-09-10T09:51:21.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:21.356+0000 7f2894c00640 1 --2- 172.21.15.5:0/231007167 >> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] conn(0x55d23f89b560 0x55d23f89d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:51:21.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:21.357+0000 7f2876a00640 10 client.28010 ms_handle_connect on v2:172.21.15.159:6836/620627639 2024-09-10T09:51:21.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:21.357+0000 7f2894c00640 1 --2- 172.21.15.5:0/231007167 >> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] conn(0x55d23f89b560 0x55d23f89d950 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8fb3febda564fa39 server_cookie=1e313eb5ed53a33b in_seq=0 out_seq=0 2024-09-10T09:51:21.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:21.357+0000 7f2876a00640 1 -- 172.21.15.5:0/231007167 <== mds.2 v2:172.21.15.159:6836/620627639 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d23f77de60 con 0x55d23f89b560 2024-09-10T09:51:21.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:21.357+0000 7f2876a00640 10 client.28010 handle_command_reply: tid=0 2024-09-10T09:51:21.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:21.357+0000 7f289ac2d740 1 client.28010 shutdown 2024-09-10T09:51:21.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:21.358+0000 7f289ac2d740 1 -- 172.21.15.5:0/231007167 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d23f81b020 msgr2=0x55d23f828430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:51:21.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:21.358+0000 7f289ac2d740 1 --2- 172.21.15.5:0/231007167 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d23f81b020 0x55d23f828430 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f2888009e80 tx=0x7f288800f7d0 comp rx=0 tx=0).stop 2024-09-10T09:51:21.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:21.358+0000 7f289ac2d740 1 -- 172.21.15.5:0/231007167 shutdown_connections 2024-09-10T09:51:21.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:21.358+0000 7f289ac2d740 1 --2- 172.21.15.5:0/231007167 >> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] conn(0x55d23f89b560 0x55d23f89d950 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:21.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:21.358+0000 7f289ac2d740 1 --2- 172.21.15.5:0/231007167 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d23f81b020 0x55d23f828430 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:21.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:21.358+0000 7f289ac2d740 1 --2- 172.21.15.5:0/231007167 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d23f810760 0x55d23f827e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:21.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:21.358+0000 7f289ac2d740 1 --2- 172.21.15.5:0/231007167 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d23f80fe40 0x55d23f82bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:21.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:21.358+0000 7f289ac2d740 1 -- 172.21.15.5:0/231007167 >> 172.21.15.5:0/231007167 conn(0x55d23f80c3d0 msgr2=0x55d23f80ac70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:51:21.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:21.358+0000 7f2876a00640 0 client.28010 ms_handle_reset on v2:172.21.15.159:6836/620627639 2024-09-10T09:51:21.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:21.358+0000 7f289ac2d740 1 -- 172.21.15.5:0/231007167 shutdown_connections 2024-09-10T09:51:21.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:21.358+0000 7f289ac2d740 1 -- 172.21.15.5:0/231007167 wait complete. 2024-09-10T09:51:21.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:21.358+0000 7f289ac2d740 20 client.28010 trim_cache size 0 max 16384 2024-09-10T09:51:21.362 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-10T09:51:21.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:21.359+0000 7f2896a00640 1 -- 172.21.15.5:0/509059712 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f28780a2b80 msgr2=0x7f28780a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:51:21.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:21.359+0000 7f2896a00640 1 --2- 172.21.15.5:0/509059712 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f28780a2b80 0x7f28780a5040 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f2884098600 tx=0x7f28840d5530 comp rx=0 tx=0).stop 2024-09-10T09:51:21.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:21.359+0000 7f2896a00640 1 -- 172.21.15.5:0/509059712 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2890102c30 msgr2=0x7f28901982d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:51:21.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:21.359+0000 7f2896a00640 1 --2- 172.21.15.5:0/509059712 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2890102c30 0x7f28901982d0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f288400e9f0 tx=0x7f288400eeb0 comp rx=0 tx=0).stop 2024-09-10T09:51:21.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:21.360+0000 7f2896a00640 1 -- 172.21.15.5:0/509059712 shutdown_connections 2024-09-10T09:51:21.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:21.360+0000 7f2896a00640 1 --2- 172.21.15.5:0/509059712 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f28780a2b80 0x7f28780a5040 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:21.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:21.360+0000 7f2896a00640 1 --2- 172.21.15.5:0/509059712 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f289010bfd0 0x7f289019cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:21.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:21.360+0000 7f2896a00640 1 --2- 172.21.15.5:0/509059712 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2890103550 0x7f2890198810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:21.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:21.360+0000 7f2896a00640 1 --2- 172.21.15.5:0/509059712 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2890102c30 0x7f28901982d0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:21.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:21.360+0000 7f2896a00640 1 -- 172.21.15.5:0/509059712 >> 172.21.15.5:0/509059712 conn(0x7f28900fc610 msgr2=0x7f28900fd350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:51:21.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:21.360+0000 7f2896a00640 1 -- 172.21.15.5:0/509059712 shutdown_connections 2024-09-10T09:51:21.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:21.360+0000 7f2896a00640 1 -- 172.21.15.5:0/509059712 wait complete. 2024-09-10T09:51:22.020 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:51:21 smithi005 ceph-mon[30350]: pgmap v783: 129 pgs: 129 active+clean; 302 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 1.6 MiB/s wr, 218 op/s 2024-09-10T09:51:22.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:51:21 smithi159 ceph-mon[35513]: pgmap v783: 129 pgs: 129 active+clean; 302 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 1.6 MiB/s wr, 218 op/s 2024-09-10T09:51:22.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:51:21 smithi102 ceph-mon[35442]: pgmap v783: 129 pgs: 129 active+clean; 302 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 1.6 MiB/s wr, 218 op/s 2024-09-10T09:51:22.148 INFO:tasks.workunit.client.0.smithi005.stdout: 1 837251 6.43 MB/sec execute 543 sec latency 16.606 ms 2024-09-10T09:51:22.560 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-10T09:51:22.961 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:51:23.148 INFO:tasks.workunit.client.0.smithi005.stdout: 1 838944 6.44 MB/sec execute 544 sec latency 21.811 ms 2024-09-10T09:51:23.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:23.551+0000 7f8380a00640 1 -- 172.21.15.5:0/4165049158 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f837c10bfc0 msgr2=0x7f837c10e460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:51:23.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:23.551+0000 7f8380a00640 1 --2- 172.21.15.5:0/4165049158 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f837c10bfc0 0x7f837c10e460 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f8374009be0 tx=0x7f837402a750 comp rx=0 tx=0).stop 2024-09-10T09:51:23.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:23.551+0000 7f8380a00640 1 -- 172.21.15.5:0/4165049158 shutdown_connections 2024-09-10T09:51:23.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:23.551+0000 7f8380a00640 1 --2- 172.21.15.5:0/4165049158 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f837c10bfc0 0x7f837c10e460 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:23.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:23.551+0000 7f8380a00640 1 --2- 172.21.15.5:0/4165049158 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f837c1034b0 0x7f837c10b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:23.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:23.551+0000 7f8380a00640 1 --2- 172.21.15.5:0/4165049158 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f837c102b90 0x7f837c102f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:23.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:23.551+0000 7f8380a00640 1 -- 172.21.15.5:0/4165049158 >> 172.21.15.5:0/4165049158 conn(0x7f837c0fc610 msgr2=0x7f837c0fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:51:23.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:23.551+0000 7f8380a00640 1 -- 172.21.15.5:0/4165049158 shutdown_connections 2024-09-10T09:51:23.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:23.551+0000 7f8380a00640 1 -- 172.21.15.5:0/4165049158 wait complete. 2024-09-10T09:51:23.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:23.552+0000 7f8380a00640 1 Processor -- start 2024-09-10T09:51:23.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:23.552+0000 7f8380a00640 1 -- start start 2024-09-10T09:51:23.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:23.553+0000 7f8380a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f837c102b90 0x7f837c198230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:51:23.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:23.553+0000 7f8380a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f837c1034b0 0x7f837c198770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:51:23.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:23.553+0000 7f8380a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f837c10bfc0 0x7f837c19cb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:51:23.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:23.553+0000 7f8380a00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f837c111d90 con 0x7f837c10bfc0 2024-09-10T09:51:23.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:23.553+0000 7f8380a00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f837c111c10 con 0x7f837c102b90 2024-09-10T09:51:23.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:23.553+0000 7f8380a00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f837c111f10 con 0x7f837c1034b0 2024-09-10T09:51:23.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:23.553+0000 7f8379e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f837c1034b0 0x7f837c198770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:51:23.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:23.553+0000 7f837a800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f837c102b90 0x7f837c198230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:51:23.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:23.553+0000 7f8379e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f837c1034b0 0x7f837c198770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:43758/0 (socket says 172.21.15.5:43758) 2024-09-10T09:51:23.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:23.553+0000 7f837a800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f837c102b90 0x7f837c198230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:52396/0 (socket says 172.21.15.5:52396) 2024-09-10T09:51:23.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:23.553+0000 7f8379e00640 1 -- 172.21.15.5:0/1550324041 learned_addr learned my addr 172.21.15.5:0/1550324041 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:51:23.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:23.553+0000 7f837b200640 1 --2- 172.21.15.5:0/1550324041 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f837c10bfc0 0x7f837c19cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:51:23.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:23.554+0000 7f8379e00640 1 -- 172.21.15.5:0/1550324041 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f837c102b90 msgr2=0x7f837c198230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:51:23.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:23.554+0000 7f8379e00640 1 --2- 172.21.15.5:0/1550324041 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f837c102b90 0x7f837c198230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:23.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:23.554+0000 7f8379e00640 1 -- 172.21.15.5:0/1550324041 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f837c10bfc0 msgr2=0x7f837c19cb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:51:23.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:23.554+0000 7f8379e00640 1 --2- 172.21.15.5:0/1550324041 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f837c10bfc0 0x7f837c19cb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:23.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:23.554+0000 7f8379e00640 1 -- 172.21.15.5:0/1550324041 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f837c19d1e0 con 0x7f837c1034b0 2024-09-10T09:51:23.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:23.554+0000 7f837a800640 1 --2- 172.21.15.5:0/1550324041 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f837c102b90 0x7f837c198230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:51:23.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:23.554+0000 7f8379e00640 1 --2- 172.21.15.5:0/1550324041 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f837c1034b0 0x7f837c198770 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f836800e9f0 tx=0x7f836800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:51:23.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:23.554+0000 7f8373400640 1 -- 172.21.15.5:0/1550324041 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f836800ccc0 con 0x7f837c1034b0 2024-09-10T09:51:23.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:23.554+0000 7f8373400640 1 -- 172.21.15.5:0/1550324041 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8368010910 con 0x7f837c1034b0 2024-09-10T09:51:23.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:23.554+0000 7f8373400640 1 -- 172.21.15.5:0/1550324041 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83680188e0 con 0x7f837c1034b0 2024-09-10T09:51:23.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:23.554+0000 7f8380a00640 1 -- 172.21.15.5:0/1550324041 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f837c19d4d0 con 0x7f837c1034b0 2024-09-10T09:51:23.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:23.554+0000 7f8380a00640 1 -- 172.21.15.5:0/1550324041 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f837c100720 con 0x7f837c1034b0 2024-09-10T09:51:23.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:23.557+0000 7f8383fce740 1 Processor -- start 2024-09-10T09:51:23.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:23.558+0000 7f8383fce740 1 -- start start 2024-09-10T09:51:23.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:23.558+0000 7f8373400640 1 -- 172.21.15.5:0/1550324041 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f8368010430 con 0x7f837c1034b0 2024-09-10T09:51:23.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:23.558+0000 7f8383fce740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d7519d4d60 0x55d7519d5140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:51:23.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:23.558+0000 7f8383fce740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d7519d5680 0x55d7519e82f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:51:23.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:23.558+0000 7f8383fce740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d7519e8830 0x55d7519eac20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:51:23.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:23.558+0000 7f8383fce740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55d75193b3e0 con 0x55d7519d4d60 2024-09-10T09:51:23.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:23.558+0000 7f8383fce740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55d7518e2d80 con 0x55d7519e8830 2024-09-10T09:51:23.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:23.558+0000 7f8383fce740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55d7519373a0 con 0x55d7519d5680 2024-09-10T09:51:23.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:23.558+0000 7f837b200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d7519d4d60 0x55d7519d5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:51:23.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:23.558+0000 7f837a800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d7519d5680 0x55d7519e82f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:51:23.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:23.559+0000 7f837b200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d7519d4d60 0x55d7519d5140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38868/0 (socket says 172.21.15.5:38868) 2024-09-10T09:51:23.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:23.559+0000 7f837b200640 1 -- 172.21.15.5:0/2406992596 learned_addr learned my addr 172.21.15.5:0/2406992596 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:51:23.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:23.559+0000 7f8379e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d7519e8830 0x55d7519eac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:51:23.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:23.559+0000 7f837a800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d7519d5680 0x55d7519e82f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:43762/0 (socket says 172.21.15.5:43762) 2024-09-10T09:51:23.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:23.559+0000 7f837b200640 1 -- 172.21.15.5:0/2406992596 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d7519d5680 msgr2=0x55d7519e82f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:51:23.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:23.559+0000 7f837b200640 1 --2- 172.21.15.5:0/2406992596 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d7519d5680 0x55d7519e82f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:23.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:23.559+0000 7f837b200640 1 -- 172.21.15.5:0/2406992596 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d7519e8830 msgr2=0x55d7519eac20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:51:23.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:23.559+0000 7f837b200640 1 --2- 172.21.15.5:0/2406992596 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d7519e8830 0x55d7519eac20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:23.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:23.559+0000 7f837b200640 1 -- 172.21.15.5:0/2406992596 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d751901ae0 con 0x55d7519d4d60 2024-09-10T09:51:23.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:23.559+0000 7f837b200640 1 --2- 172.21.15.5:0/2406992596 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d7519d4d60 0x55d7519d5140 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f8374009be0 tx=0x7f8374030aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afc1d3e3fd77ad61 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:51:23.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:23.559+0000 7f8373400640 1 --2- 172.21.15.5:0/1550324041 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f83600a2c00 0x7f83600a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:51:23.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:23.560+0000 7f8373400640 1 -- 172.21.15.5:0/1550324041 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f83680cf610 con 0x7f837c1034b0 2024-09-10T09:51:23.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:23.560+0000 7f837a800640 1 --2- 172.21.15.5:0/1550324041 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f83600a2c00 0x7f83600a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:51:23.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:23.560+0000 7f8367400640 1 -- 172.21.15.5:0/2406992596 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8374041030 con 0x55d7519d4d60 2024-09-10T09:51:23.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:23.560+0000 7f8367400640 1 -- 172.21.15.5:0/2406992596 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8374030c50 con 0x55d7519d4d60 2024-09-10T09:51:23.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:23.560+0000 7f8367400640 1 -- 172.21.15.5:0/2406992596 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8374036040 con 0x55d7519d4d60 2024-09-10T09:51:23.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:23.561+0000 7f8383fce740 1 -- 172.21.15.5:0/2406992596 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d7519d4d60 msgr2=0x55d7519d5140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:51:23.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:23.561+0000 7f837a800640 1 --2- 172.21.15.5:0/1550324041 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f83600a2c00 0x7f83600a50c0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f836c009530 tx=0x7f836c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:51:23.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:23.561+0000 7f8383fce740 1 --2- 172.21.15.5:0/2406992596 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d7519d4d60 0x55d7519d5140 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f8374009be0 tx=0x7f8374030aa0 comp rx=0 tx=0).stop 2024-09-10T09:51:23.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:23.561+0000 7f8383fce740 1 -- 172.21.15.5:0/2406992596 shutdown_connections 2024-09-10T09:51:23.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:23.561+0000 7f8383fce740 1 --2- 172.21.15.5:0/2406992596 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d7519e8830 0x55d7519eac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:23.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:23.561+0000 7f8383fce740 1 --2- 172.21.15.5:0/2406992596 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d7519d5680 0x55d7519e82f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:23.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:23.561+0000 7f8383fce740 1 --2- 172.21.15.5:0/2406992596 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d7519d4d60 0x55d7519d5140 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:23.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:23.561+0000 7f8383fce740 1 -- 172.21.15.5:0/2406992596 >> 172.21.15.5:0/2406992596 conn(0x55d7519d13d0 msgr2=0x55d7519d17b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:51:23.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:23.561+0000 7f8383fce740 1 -- 172.21.15.5:0/2406992596 shutdown_connections 2024-09-10T09:51:23.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:23.561+0000 7f8383fce740 1 -- 172.21.15.5:0/2406992596 wait complete. 2024-09-10T09:51:23.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:23.562+0000 7f8383fce740 1 -- start start 2024-09-10T09:51:23.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:23.562+0000 7f8383fce740 1 Processor -- start 2024-09-10T09:51:23.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:23.562+0000 7f8383fce740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d7519d4d60 0x55d7519e5830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:51:23.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:23.562+0000 7f8383fce740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d7519d5680 0x55d7519e5d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:51:23.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:23.562+0000 7f8383fce740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d7519e8830 0x55d7519e62b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:51:23.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:23.563+0000 7f8383fce740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55d7519eb340 con 0x55d7519d4d60 2024-09-10T09:51:23.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:23.563+0000 7f8383fce740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55d7518f60f0 con 0x55d7519e8830 2024-09-10T09:51:23.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:23.563+0000 7f8383fce740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55d7519eb4c0 con 0x55d7519d5680 2024-09-10T09:51:23.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:23.563+0000 7f8383fce740 10 client.0 _ll_register_callbacks cb 0x55d7518ddbd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:51:23.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:23.563+0000 7f837a800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d7519d4d60 0x55d7519e5830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:51:23.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:23.563+0000 7f837a800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d7519d4d60 0x55d7519e5830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38880/0 (socket says 172.21.15.5:38880) 2024-09-10T09:51:23.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:23.563+0000 7f837a800640 1 -- 172.21.15.5:0/1386978287 learned_addr learned my addr 172.21.15.5:0/1386978287 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:51:23.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:23.563+0000 7f8379e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d7519d5680 0x55d7519e5d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:51:23.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:23.563+0000 7f837b200640 1 --2- 172.21.15.5:0/1386978287 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d7519e8830 0x55d7519e62b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:51:23.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:23.563+0000 7f8379e00640 1 -- 172.21.15.5:0/1386978287 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d7519e8830 msgr2=0x55d7519e62b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:51:23.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:23.563+0000 7f8379e00640 1 --2- 172.21.15.5:0/1386978287 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d7519e8830 0x55d7519e62b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:23.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:23.563+0000 7f8379e00640 1 -- 172.21.15.5:0/1386978287 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d7519d4d60 msgr2=0x55d7519e5830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:51:23.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:23.563+0000 7f8379e00640 1 --2- 172.21.15.5:0/1386978287 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d7519d4d60 0x55d7519e5830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:23.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:23.563+0000 7f8379e00640 1 -- 172.21.15.5:0/1386978287 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d7516ac3c0 con 0x55d7519d5680 2024-09-10T09:51:23.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:23.564+0000 7f8379e00640 1 --2- 172.21.15.5:0/1386978287 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d7519d5680 0x55d7519e5d70 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f83680ccc90 tx=0x7f8368097e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e66cfbb8e053eb53 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:51:23.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:23.564+0000 7f835be00640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-09-10T09:51:23.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:23.564+0000 7f835be00640 1 -- 172.21.15.5:0/1386978287 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8368010dc0 con 0x55d7519d5680 2024-09-10T09:51:23.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:23.564+0000 7f835be00640 1 -- 172.21.15.5:0/1386978287 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f83680cf280 con 0x55d7519d5680 2024-09-10T09:51:23.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:23.564+0000 7f8383fce740 1 -- 172.21.15.5:0/1386978287 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8374030c50 con 0x55d7519d5680 2024-09-10T09:51:23.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:23.564+0000 7f835be00640 1 -- 172.21.15.5:0/1386978287 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f836809bad0 con 0x55d7519d5680 2024-09-10T09:51:23.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:23.564+0000 7f8383fce740 10 client.28018 fetch_fsmap learned FSMap version 36 2024-09-10T09:51:23.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:23.564+0000 7f8383fce740 1 -- 172.21.15.5:0/1386978287 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55d7519f7510 con 0x55d7519d5680 2024-09-10T09:51:23.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:23.565+0000 7f835be00640 1 -- 172.21.15.5:0/1386978287 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8374030c50 con 0x55d7519d5680 2024-09-10T09:51:23.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:23.565+0000 7f835be00640 1 -- 172.21.15.5:0/1386978287 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 36) ==== 8370+0+0 (secure 0 0 0) 0x7f836809af20 con 0x55d7519d5680 2024-09-10T09:51:23.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:23.565+0000 7f8383fce740 10 client.28018 fetch_fsmap finished waiting for FSMap version 36 2024-09-10T09:51:23.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:23.565+0000 7f8383fce740 10 client.28018 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-10T09:51:23.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:23.565+0000 7f8383fce740 20 client.28018 populate_metadata read hostname 'smithi005' 2024-09-10T09:51:23.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:23.565+0000 7f8383fce740 1 --2- 172.21.15.5:0/1386978287 >> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] conn(0x55d7519f8920 0x55d7519fad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:51:23.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:23.565+0000 7f8383fce740 4 client.28018 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:51:23.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:23.565+0000 7f8383fce740 1 -- 172.21.15.5:0/1386978287 --> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d751948950 con 0x55d7519f8920 2024-09-10T09:51:23.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:23.565+0000 7f837a800640 1 --2- 172.21.15.5:0/1386978287 >> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] conn(0x55d7519f8920 0x55d7519fad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:51:23.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:23.566+0000 7f837a800640 1 --2- 172.21.15.5:0/1386978287 >> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] conn(0x55d7519f8920 0x55d7519fad10 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f7b4cb709e61303d server_cookie=a097ab86089fd8a7 in_seq=0 out_seq=0 2024-09-10T09:51:23.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:23.566+0000 7f835be00640 10 client.28018 ms_handle_connect on v2:172.21.15.159:6838/2073739599 2024-09-10T09:51:23.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:23.566+0000 7f835be00640 1 -- 172.21.15.5:0/1386978287 <== mds.1 v2:172.21.15.159:6838/2073739599 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d751948950 con 0x55d7519f8920 2024-09-10T09:51:23.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:23.566+0000 7f835be00640 10 client.28018 handle_command_reply: tid=0 2024-09-10T09:51:23.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:23.567+0000 7f8383fce740 1 client.28018 shutdown 2024-09-10T09:51:23.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:23.567+0000 7f8383fce740 1 -- 172.21.15.5:0/1386978287 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d7519d5680 msgr2=0x55d7519e5d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:51:23.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:23.567+0000 7f8383fce740 1 --2- 172.21.15.5:0/1386978287 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d7519d5680 0x55d7519e5d70 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f83680ccc90 tx=0x7f8368097e30 comp rx=0 tx=0).stop 2024-09-10T09:51:23.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:23.567+0000 7f8383fce740 1 -- 172.21.15.5:0/1386978287 shutdown_connections 2024-09-10T09:51:23.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:23.567+0000 7f8383fce740 1 --2- 172.21.15.5:0/1386978287 >> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] conn(0x55d7519f8920 0x55d7519fad10 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:23.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:23.567+0000 7f8383fce740 1 --2- 172.21.15.5:0/1386978287 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d7519e8830 0x55d7519e62b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:23.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:23.567+0000 7f8383fce740 1 --2- 172.21.15.5:0/1386978287 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d7519d5680 0x55d7519e5d70 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:23.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:23.567+0000 7f8383fce740 1 --2- 172.21.15.5:0/1386978287 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d7519d4d60 0x55d7519e5830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:23.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:23.567+0000 7f8383fce740 1 -- 172.21.15.5:0/1386978287 >> 172.21.15.5:0/1386978287 conn(0x55d7519d13d0 msgr2=0x55d7519d7060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:51:23.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:23.567+0000 7f835be00640 0 client.28018 ms_handle_reset on v2:172.21.15.159:6838/2073739599 2024-09-10T09:51:23.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:23.567+0000 7f8383fce740 1 -- 172.21.15.5:0/1386978287 shutdown_connections 2024-09-10T09:51:23.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:23.568+0000 7f8383fce740 1 -- 172.21.15.5:0/1386978287 wait complete. 2024-09-10T09:51:23.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:23.568+0000 7f8383fce740 20 client.28018 trim_cache size 0 max 16384 2024-09-10T09:51:23.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:23.582+0000 7f8383fce740 1 Processor -- start 2024-09-10T09:51:23.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:23.583+0000 7f8383fce740 1 -- start start 2024-09-10T09:51:23.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:23.583+0000 7f8383fce740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d7519d4d60 0x55d7519e77a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:51:23.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:23.583+0000 7f8383fce740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d7519d5680 0x55d751a005e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:51:23.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:23.583+0000 7f8383fce740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d7519e8830 0x55d751a00b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:51:23.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:23.583+0000 7f8383fce740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f837c1aca30 con 0x55d7519d4d60 2024-09-10T09:51:23.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:23.583+0000 7f8383fce740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f837c1ae570 con 0x55d7519d5680 2024-09-10T09:51:23.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:23.583+0000 7f8383fce740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f837c1ad100 con 0x55d7519e8830 2024-09-10T09:51:23.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:23.583+0000 7f837a800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d7519d4d60 0x55d7519e77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:51:23.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:23.583+0000 7f837b200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d7519e8830 0x55d751a00b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:51:23.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:23.583+0000 7f8379e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d7519d5680 0x55d751a005e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:51:23.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:23.583+0000 7f837b200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d7519e8830 0x55d751a00b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:43768/0 (socket says 172.21.15.5:43768) 2024-09-10T09:51:23.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:23.583+0000 7f837b200640 1 -- 172.21.15.5:0/2393352750 learned_addr learned my addr 172.21.15.5:0/2393352750 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:51:23.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:23.584+0000 7f8379e00640 1 -- 172.21.15.5:0/2393352750 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d7519e8830 msgr2=0x55d751a00b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:51:23.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:23.584+0000 7f8379e00640 1 --2- 172.21.15.5:0/2393352750 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d7519e8830 0x55d751a00b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:23.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:23.584+0000 7f8379e00640 1 -- 172.21.15.5:0/2393352750 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d7519d4d60 msgr2=0x55d7519e77a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:51:23.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:23.584+0000 7f8379e00640 1 --2- 172.21.15.5:0/2393352750 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d7519d4d60 0x55d7519e77a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:23.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:23.584+0000 7f8379e00640 1 -- 172.21.15.5:0/2393352750 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d7519d2060 con 0x55d7519d5680 2024-09-10T09:51:23.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:23.584+0000 7f8379e00640 1 --2- 172.21.15.5:0/2393352750 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d7519d5680 0x55d751a005e0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x55d7519e6e70 tx=0x7f8368097ea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=746ca673c0973933 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:51:23.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:23.584+0000 7f8366a00640 1 -- 172.21.15.5:0/2393352750 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8368010dc0 con 0x55d7519d5680 2024-09-10T09:51:23.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:23.584+0000 7f8366a00640 1 -- 172.21.15.5:0/2393352750 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f83680ce070 con 0x55d7519d5680 2024-09-10T09:51:23.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:23.584+0000 7f8366a00640 1 -- 172.21.15.5:0/2393352750 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83680cf280 con 0x55d7519d5680 2024-09-10T09:51:23.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:23.585+0000 7f8383fce740 1 -- 172.21.15.5:0/2393352750 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d7519d5680 msgr2=0x55d751a005e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:51:23.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:23.585+0000 7f8383fce740 1 --2- 172.21.15.5:0/2393352750 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d7519d5680 0x55d751a005e0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x55d7519e6e70 tx=0x7f8368097ea0 comp rx=0 tx=0).stop 2024-09-10T09:51:23.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:23.585+0000 7f8383fce740 1 -- 172.21.15.5:0/2393352750 shutdown_connections 2024-09-10T09:51:23.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:23.585+0000 7f8383fce740 1 --2- 172.21.15.5:0/2393352750 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d7519e8830 0x55d751a00b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:23.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:23.585+0000 7f8383fce740 1 --2- 172.21.15.5:0/2393352750 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d7519d5680 0x55d751a005e0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:23.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:23.585+0000 7f8383fce740 1 --2- 172.21.15.5:0/2393352750 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d7519d4d60 0x55d7519e77a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:23.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:23.585+0000 7f8383fce740 1 -- 172.21.15.5:0/2393352750 >> 172.21.15.5:0/2393352750 conn(0x55d7519d13d0 msgr2=0x55d7519e7090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:51:23.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:23.585+0000 7f8383fce740 1 -- 172.21.15.5:0/2393352750 shutdown_connections 2024-09-10T09:51:23.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:23.585+0000 7f8383fce740 1 -- 172.21.15.5:0/2393352750 wait complete. 2024-09-10T09:51:23.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:23.586+0000 7f8383fce740 1 -- start start 2024-09-10T09:51:23.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:23.586+0000 7f8383fce740 1 Processor -- start 2024-09-10T09:51:23.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:23.586+0000 7f8383fce740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d7519d4d60 0x55d7519e29e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:51:23.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:23.586+0000 7f8383fce740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d7519d5680 0x55d7519e2f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:51:23.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:23.586+0000 7f8383fce740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d7519e8830 0x55d7519e3460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:51:23.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:23.586+0000 7f8383fce740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f837c1adea0 con 0x55d7519e8830 2024-09-10T09:51:23.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:23.586+0000 7f8383fce740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f837c1ad7d0 con 0x55d7519d4d60 2024-09-10T09:51:23.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:23.586+0000 7f8383fce740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f837c1aaa10 con 0x55d7519d5680 2024-09-10T09:51:23.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:23.586+0000 7f8383fce740 10 client.0 _ll_register_callbacks cb 0x7f8368009590 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:51:23.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:23.587+0000 7f8379e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d7519d5680 0x55d7519e2f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:51:23.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:23.587+0000 7f8379e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d7519d5680 0x55d7519e2f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:43780/0 (socket says 172.21.15.5:43780) 2024-09-10T09:51:23.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:23.587+0000 7f8379e00640 1 -- 172.21.15.5:0/1648617879 learned_addr learned my addr 172.21.15.5:0/1648617879 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:51:23.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:23.587+0000 7f837a800640 1 --2- 172.21.15.5:0/1648617879 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d7519d4d60 0x55d7519e29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:51:23.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:23.587+0000 7f837b200640 1 --2- 172.21.15.5:0/1648617879 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d7519e8830 0x55d7519e3460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:51:23.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:23.587+0000 7f837a800640 1 -- 172.21.15.5:0/1648617879 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d7519d5680 msgr2=0x55d7519e2f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:51:23.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:23.587+0000 7f837a800640 1 --2- 172.21.15.5:0/1648617879 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d7519d5680 0x55d7519e2f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:23.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:23.587+0000 7f837a800640 1 -- 172.21.15.5:0/1648617879 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d7519e8830 msgr2=0x55d7519e3460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:51:23.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:23.587+0000 7f837a800640 1 --2- 172.21.15.5:0/1648617879 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d7519e8830 0x55d7519e3460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:23.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:23.587+0000 7f837a800640 1 -- 172.21.15.5:0/1648617879 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d751a62710 con 0x55d7519d4d60 2024-09-10T09:51:23.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:23.587+0000 7f8379e00640 1 --2- 172.21.15.5:0/1648617879 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d7519d5680 0x55d7519e2f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:51:23.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:23.587+0000 7f837a800640 1 --2- 172.21.15.5:0/1648617879 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d7519d4d60 0x55d7519e29e0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f83680d2df0 tx=0x7f836c0046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dd49ce4cbbea7cbc server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:51:23.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:23.587+0000 7f835be00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-09-10T09:51:23.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:23.588+0000 7f835be00640 1 -- 172.21.15.5:0/1648617879 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f836c037060 con 0x55d7519d4d60 2024-09-10T09:51:23.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:23.588+0000 7f835be00640 1 -- 172.21.15.5:0/1648617879 <== mon.2 v2:172.21.15.102:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f836c02a580 con 0x55d7519d4d60 2024-09-10T09:51:23.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:23.588+0000 7f8383fce740 1 -- 172.21.15.5:0/1648617879 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f83680ce070 con 0x55d7519d4d60 2024-09-10T09:51:23.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:23.588+0000 7f835be00640 1 -- 172.21.15.5:0/1648617879 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f836c03e050 con 0x55d7519d4d60 2024-09-10T09:51:23.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:23.588+0000 7f8383fce740 10 client.28028 fetch_fsmap learned FSMap version 36 2024-09-10T09:51:23.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:23.588+0000 7f8383fce740 1 -- 172.21.15.5:0/1648617879 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55d751a63a30 con 0x55d7519d4d60 2024-09-10T09:51:23.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:23.588+0000 7f835be00640 1 -- 172.21.15.5:0/1648617879 <== mon.2 v2:172.21.15.102:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f83680ce070 con 0x55d7519d4d60 2024-09-10T09:51:23.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:23.589+0000 7f835be00640 1 -- 172.21.15.5:0/1648617879 <== mon.2 v2:172.21.15.102:3300/0 5 ==== fsmap(e 36) ==== 8370+0+0 (secure 0 0 0) 0x7f836c01eaf0 con 0x55d7519d4d60 2024-09-10T09:51:23.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:23.589+0000 7f8383fce740 10 client.28028 fetch_fsmap finished waiting for FSMap version 36 2024-09-10T09:51:23.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:23.589+0000 7f8383fce740 10 client.28028 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-10T09:51:23.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:23.589+0000 7f8383fce740 20 client.28028 populate_metadata read hostname 'smithi005' 2024-09-10T09:51:23.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:23.589+0000 7f8383fce740 1 --2- 172.21.15.5:0/1648617879 >> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] conn(0x55d751a64100 0x55d751a664f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:51:23.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:23.589+0000 7f8383fce740 4 client.28028 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T09:51:23.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:23.589+0000 7f8383fce740 1 -- 172.21.15.5:0/1648617879 --> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d751942e60 con 0x55d751a64100 2024-09-10T09:51:23.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:23.589+0000 7f8379e00640 1 --2- 172.21.15.5:0/1648617879 >> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] conn(0x55d751a64100 0x55d751a664f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:51:23.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:23.589+0000 7f8379e00640 1 --2- 172.21.15.5:0/1648617879 >> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] conn(0x55d751a64100 0x55d751a664f0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b0e7f60f55f9d1a2 server_cookie=a0cf5250930faee9 in_seq=0 out_seq=0 2024-09-10T09:51:23.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:23.589+0000 7f835be00640 10 client.28028 ms_handle_connect on v2:172.21.15.159:6838/2073739599 2024-09-10T09:51:23.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:23.590+0000 7f835be00640 1 -- 172.21.15.5:0/1648617879 <== mds.1 v2:172.21.15.159:6838/2073739599 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d751942e60 con 0x55d751a64100 2024-09-10T09:51:23.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:23.590+0000 7f835be00640 10 client.28028 handle_command_reply: tid=0 2024-09-10T09:51:23.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:23.590+0000 7f8383fce740 1 client.28028 shutdown 2024-09-10T09:51:23.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:23.590+0000 7f8383fce740 1 -- 172.21.15.5:0/1648617879 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d7519d4d60 msgr2=0x55d7519e29e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:51:23.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:23.590+0000 7f8383fce740 1 --2- 172.21.15.5:0/1648617879 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d7519d4d60 0x55d7519e29e0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f83680d2df0 tx=0x7f836c0046c0 comp rx=0 tx=0).stop 2024-09-10T09:51:23.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:23.590+0000 7f8383fce740 1 -- 172.21.15.5:0/1648617879 shutdown_connections 2024-09-10T09:51:23.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:23.590+0000 7f8383fce740 1 --2- 172.21.15.5:0/1648617879 >> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] conn(0x55d751a64100 0x55d751a664f0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:23.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:23.590+0000 7f8383fce740 1 --2- 172.21.15.5:0/1648617879 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d7519e8830 0x55d7519e3460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:23.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:23.590+0000 7f8383fce740 1 --2- 172.21.15.5:0/1648617879 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d7519d5680 0x55d7519e2f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:23.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:23.590+0000 7f835be00640 0 client.28028 ms_handle_reset on v2:172.21.15.159:6838/2073739599 2024-09-10T09:51:23.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:23.590+0000 7f8383fce740 1 --2- 172.21.15.5:0/1648617879 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d7519d4d60 0x55d7519e29e0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:23.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:23.590+0000 7f8383fce740 1 -- 172.21.15.5:0/1648617879 >> 172.21.15.5:0/1648617879 conn(0x55d7519d13d0 msgr2=0x55d7519e57f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:51:23.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:23.590+0000 7f8383fce740 1 -- 172.21.15.5:0/1648617879 shutdown_connections 2024-09-10T09:51:23.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:23.590+0000 7f8383fce740 1 -- 172.21.15.5:0/1648617879 wait complete. 2024-09-10T09:51:23.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:23.590+0000 7f8383fce740 20 client.28028 trim_cache size 0 max 16384 2024-09-10T09:51:23.596 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-10T09:51:23.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:23.591+0000 7f8380a00640 1 -- 172.21.15.5:0/1550324041 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f83600a2c00 msgr2=0x7f83600a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:51:23.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:23.591+0000 7f8380a00640 1 --2- 172.21.15.5:0/1550324041 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f83600a2c00 0x7f83600a50c0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f836c009530 tx=0x7f836c009260 comp rx=0 tx=0).stop 2024-09-10T09:51:23.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:23.591+0000 7f8380a00640 1 -- 172.21.15.5:0/1550324041 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f837c1034b0 msgr2=0x7f837c198770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:51:23.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:23.591+0000 7f8380a00640 1 --2- 172.21.15.5:0/1550324041 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f837c1034b0 0x7f837c198770 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f836800e9f0 tx=0x7f836800eeb0 comp rx=0 tx=0).stop 2024-09-10T09:51:23.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:23.591+0000 7f8380a00640 1 -- 172.21.15.5:0/1550324041 shutdown_connections 2024-09-10T09:51:23.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:23.591+0000 7f8380a00640 1 --2- 172.21.15.5:0/1550324041 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f83600a2c00 0x7f83600a50c0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:23.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:23.591+0000 7f8380a00640 1 --2- 172.21.15.5:0/1550324041 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f837c10bfc0 0x7f837c19cb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:23.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:23.591+0000 7f8380a00640 1 --2- 172.21.15.5:0/1550324041 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f837c1034b0 0x7f837c198770 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:23.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:23.591+0000 7f8380a00640 1 --2- 172.21.15.5:0/1550324041 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f837c102b90 0x7f837c198230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:23.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:23.591+0000 7f8380a00640 1 -- 172.21.15.5:0/1550324041 >> 172.21.15.5:0/1550324041 conn(0x7f837c0fc610 msgr2=0x7f837c0fd270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:51:23.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:23.592+0000 7f8380a00640 1 -- 172.21.15.5:0/1550324041 shutdown_connections 2024-09-10T09:51:23.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:23.592+0000 7f8380a00640 1 -- 172.21.15.5:0/1550324041 wait complete. 2024-09-10T09:51:24.034 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:51:23 smithi005 ceph-mon[30350]: pgmap v784: 129 pgs: 129 active+clean; 305 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 1.3 MiB/s wr, 201 op/s 2024-09-10T09:51:24.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:51:23 smithi159 ceph-mon[35513]: pgmap v784: 129 pgs: 129 active+clean; 305 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 1.3 MiB/s wr, 201 op/s 2024-09-10T09:51:24.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:51:23 smithi102 ceph-mon[35442]: pgmap v784: 129 pgs: 129 active+clean; 305 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 1.3 MiB/s wr, 201 op/s 2024-09-10T09:51:24.148 INFO:tasks.workunit.client.0.smithi005.stdout: 1 841590 6.46 MB/sec execute 545 sec latency 13.179 ms 2024-09-10T09:51:24.408 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-10T09:51:24.838 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:51:25.149 INFO:tasks.workunit.client.0.smithi005.stdout: 1 843057 6.46 MB/sec execute 546 sec latency 52.732 ms 2024-09-10T09:51:25.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:25.741+0000 7f1235e00640 1 -- 172.21.15.5:0/2204912569 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1230108f60 msgr2=0x7f1230110ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:51:25.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:25.741+0000 7f1235e00640 1 --2- 172.21.15.5:0/2204912569 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1230108f60 0x7f1230110ae0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f1228009ca0 tx=0x7f122802a8c0 comp rx=0 tx=0).stop 2024-09-10T09:51:25.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:25.741+0000 7f1235e00640 1 -- 172.21.15.5:0/2204912569 shutdown_connections 2024-09-10T09:51:25.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:25.741+0000 7f1235e00640 1 --2- 172.21.15.5:0/2204912569 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1230108f60 0x7f1230110ae0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:25.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:25.741+0000 7f1235e00640 1 --2- 172.21.15.5:0/2204912569 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f12301047e0 0x7f1230108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:25.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:25.741+0000 7f1235e00640 1 --2- 172.21.15.5:0/2204912569 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1230103e30 0x7f1230104210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:25.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:25.741+0000 7f1235e00640 1 -- 172.21.15.5:0/2204912569 >> 172.21.15.5:0/2204912569 conn(0x7f12300ff910 msgr2=0x7f1230101d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:51:25.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:25.741+0000 7f1235e00640 1 -- 172.21.15.5:0/2204912569 shutdown_connections 2024-09-10T09:51:25.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:25.741+0000 7f1235e00640 1 -- 172.21.15.5:0/2204912569 wait complete. 2024-09-10T09:51:25.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:25.743+0000 7f1235e00640 1 Processor -- start 2024-09-10T09:51:25.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:25.743+0000 7f1235e00640 1 -- start start 2024-09-10T09:51:25.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:25.743+0000 7f1235e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1230103e30 0x7f123019a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:51:25.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:25.743+0000 7f1235e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f12301047e0 0x7f123019a930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:51:25.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:25.743+0000 7f1235e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1230108f60 0x7f123019ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:51:25.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:25.743+0000 7f1235e00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1230114010 con 0x7f1230108f60 2024-09-10T09:51:25.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:25.743+0000 7f1235e00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f1230113e90 con 0x7f1230103e30 2024-09-10T09:51:25.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:25.743+0000 7f1235e00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f1230114190 con 0x7f12301047e0 2024-09-10T09:51:25.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:25.743+0000 7f122fe00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f12301047e0 0x7f123019a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:51:25.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:25.743+0000 7f122fe00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f12301047e0 0x7f123019a930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:43798/0 (socket says 172.21.15.5:43798) 2024-09-10T09:51:25.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:25.743+0000 7f1234a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1230103e30 0x7f123019a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:51:25.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:25.743+0000 7f122fe00640 1 -- 172.21.15.5:0/3339487848 learned_addr learned my addr 172.21.15.5:0/3339487848 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:51:25.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:25.743+0000 7f1235400640 1 --2- 172.21.15.5:0/3339487848 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1230108f60 0x7f123019ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:51:25.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:25.744+0000 7f1234a00640 1 -- 172.21.15.5:0/3339487848 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f12301047e0 msgr2=0x7f123019a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:51:25.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:25.744+0000 7f1234a00640 1 --2- 172.21.15.5:0/3339487848 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f12301047e0 0x7f123019a930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:25.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:25.744+0000 7f1234a00640 1 -- 172.21.15.5:0/3339487848 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1230108f60 msgr2=0x7f123019ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:51:25.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:25.744+0000 7f1234a00640 1 --2- 172.21.15.5:0/3339487848 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1230108f60 0x7f123019ecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:25.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:25.744+0000 7f1234a00640 1 -- 172.21.15.5:0/3339487848 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f123019f3a0 con 0x7f1230103e30 2024-09-10T09:51:25.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:25.744+0000 7f1234a00640 1 --2- 172.21.15.5:0/3339487848 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1230103e30 0x7f123019a3f0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f122400e9f0 tx=0x7f122400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:51:25.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:25.744+0000 7f122d600640 1 -- 172.21.15.5:0/3339487848 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f122400ccc0 con 0x7f1230103e30 2024-09-10T09:51:25.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:25.745+0000 7f122d600640 1 -- 172.21.15.5:0/3339487848 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1224010910 con 0x7f1230103e30 2024-09-10T09:51:25.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:25.745+0000 7f1235e00640 1 -- 172.21.15.5:0/3339487848 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f123019f690 con 0x7f1230103e30 2024-09-10T09:51:25.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:25.745+0000 7f122d600640 1 -- 172.21.15.5:0/3339487848 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1224018960 con 0x7f1230103e30 2024-09-10T09:51:25.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:25.745+0000 7f1235e00640 1 -- 172.21.15.5:0/3339487848 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12301a6f70 con 0x7f1230103e30 2024-09-10T09:51:25.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:25.748+0000 7f122d600640 1 -- 172.21.15.5:0/3339487848 <== mon.2 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f1224010ac0 con 0x7f1230103e30 2024-09-10T09:51:25.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:25.748+0000 7f1239fbd740 1 Processor -- start 2024-09-10T09:51:25.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:25.748+0000 7f1239fbd740 1 -- start start 2024-09-10T09:51:25.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:25.748+0000 7f1239fbd740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d12130de40 0x55d12130e220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:51:25.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:25.748+0000 7f1239fbd740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d12130e760 0x55d121318ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:51:25.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:25.749+0000 7f1239fbd740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d121319020 0x55d12131b410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:51:25.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:25.749+0000 7f1239fbd740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55d1212743e0 con 0x55d121319020 2024-09-10T09:51:25.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:25.749+0000 7f1239fbd740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55d12121bd80 con 0x55d12130de40 2024-09-10T09:51:25.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:25.749+0000 7f1239fbd740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55d1212703a0 con 0x55d12130e760 2024-09-10T09:51:25.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:25.749+0000 7f1234a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d12130e760 0x55d121318ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:51:25.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:25.749+0000 7f1234a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d12130e760 0x55d121318ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:43802/0 (socket says 172.21.15.5:43802) 2024-09-10T09:51:25.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:25.749+0000 7f1234a00640 1 -- 172.21.15.5:0/2580126295 learned_addr learned my addr 172.21.15.5:0/2580126295 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:51:25.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:25.749+0000 7f122fe00640 1 --2- 172.21.15.5:0/2580126295 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d121319020 0x55d12131b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:51:25.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:25.749+0000 7f1235400640 1 --2- 172.21.15.5:0/2580126295 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d12130de40 0x55d12130e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:51:25.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:25.749+0000 7f1234a00640 1 -- 172.21.15.5:0/2580126295 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d12130de40 msgr2=0x55d12130e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:51:25.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:25.749+0000 7f1234a00640 1 --2- 172.21.15.5:0/2580126295 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d12130de40 0x55d12130e220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:25.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:25.749+0000 7f1234a00640 1 -- 172.21.15.5:0/2580126295 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d121319020 msgr2=0x55d12131b410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:51:25.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:25.749+0000 7f1234a00640 1 --2- 172.21.15.5:0/2580126295 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d121319020 0x55d12131b410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:25.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:25.749+0000 7f1234a00640 1 -- 172.21.15.5:0/2580126295 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d12123aae0 con 0x55d12130e760 2024-09-10T09:51:25.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:25.750+0000 7f1234a00640 1 --2- 172.21.15.5:0/2580126295 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d12130e760 0x55d121318ae0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f12240d5c20 tx=0x7f12240d7670 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cf8001327ba0d72 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:51:25.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:25.750+0000 7f1221c00640 1 -- 172.21.15.5:0/2580126295 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12240de070 con 0x55d12130e760 2024-09-10T09:51:25.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:25.750+0000 7f1221c00640 1 -- 172.21.15.5:0/2580126295 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f12240cfcc0 con 0x55d12130e760 2024-09-10T09:51:25.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:25.750+0000 7f1221c00640 1 -- 172.21.15.5:0/2580126295 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12240da820 con 0x55d12130e760 2024-09-10T09:51:25.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:25.750+0000 7f122d600640 1 --2- 172.21.15.5:0/3339487848 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f12180a2c00 0x7f12180a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:51:25.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:25.750+0000 7f122d600640 1 -- 172.21.15.5:0/3339487848 <== mon.2 v2:172.21.15.102:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f12240cf6d0 con 0x7f1230103e30 2024-09-10T09:51:25.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:25.750+0000 7f1235400640 1 --2- 172.21.15.5:0/3339487848 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f12180a2c00 0x7f12180a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:51:25.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:25.751+0000 7f1239fbd740 1 -- 172.21.15.5:0/2580126295 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d12130e760 msgr2=0x55d121318ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:51:25.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:25.751+0000 7f1239fbd740 1 --2- 172.21.15.5:0/2580126295 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d12130e760 0x55d121318ae0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f12240d5c20 tx=0x7f12240d7670 comp rx=0 tx=0).stop 2024-09-10T09:51:25.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:25.751+0000 7f1239fbd740 1 -- 172.21.15.5:0/2580126295 shutdown_connections 2024-09-10T09:51:25.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:25.751+0000 7f1239fbd740 1 --2- 172.21.15.5:0/2580126295 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d121319020 0x55d12131b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:25.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:25.751+0000 7f1239fbd740 1 --2- 172.21.15.5:0/2580126295 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d12130e760 0x55d121318ae0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:25.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:25.751+0000 7f1239fbd740 1 --2- 172.21.15.5:0/2580126295 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d12130de40 0x55d12130e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:25.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:25.751+0000 7f1239fbd740 1 -- 172.21.15.5:0/2580126295 >> 172.21.15.5:0/2580126295 conn(0x55d12130a3d0 msgr2=0x55d12130a7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:51:25.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:25.751+0000 7f1239fbd740 1 -- 172.21.15.5:0/2580126295 shutdown_connections 2024-09-10T09:51:25.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:25.751+0000 7f1235400640 1 --2- 172.21.15.5:0/3339487848 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f12180a2c00 0x7f12180a50c0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f1228006010 tx=0x7f1228005d80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:51:25.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:25.751+0000 7f1239fbd740 1 -- 172.21.15.5:0/2580126295 wait complete. 2024-09-10T09:51:25.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:25.752+0000 7f1239fbd740 1 -- start start 2024-09-10T09:51:25.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:25.752+0000 7f1239fbd740 1 Processor -- start 2024-09-10T09:51:25.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:25.752+0000 7f1239fbd740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d12130de40 0x55d121315df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:51:25.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:25.752+0000 7f1239fbd740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d12130e760 0x55d121316330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:51:25.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:25.753+0000 7f1239fbd740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d121319020 0x55d121316870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:51:25.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:25.753+0000 7f1239fbd740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55d12131bbf0 con 0x55d12130e760 2024-09-10T09:51:25.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:25.753+0000 7f1239fbd740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55d12122f0f0 con 0x55d12130de40 2024-09-10T09:51:25.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:25.753+0000 7f1239fbd740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55d12131bd70 con 0x55d121319020 2024-09-10T09:51:25.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:25.753+0000 7f1239fbd740 10 client.0 _ll_register_callbacks cb 0x55d121216bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:51:25.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:25.753+0000 7f122fe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d12130e760 0x55d121316330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:51:25.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:25.753+0000 7f1234a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d12130de40 0x55d121315df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:51:25.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:25.753+0000 7f122fe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d12130e760 0x55d121316330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38938/0 (socket says 172.21.15.5:38938) 2024-09-10T09:51:25.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:25.753+0000 7f122fe00640 1 -- 172.21.15.5:0/4040059120 learned_addr learned my addr 172.21.15.5:0/4040059120 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:51:25.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:25.753+0000 7f1235400640 1 --2- 172.21.15.5:0/4040059120 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d121319020 0x55d121316870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:51:25.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:25.753+0000 7f1234a00640 1 -- 172.21.15.5:0/4040059120 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d121319020 msgr2=0x55d121316870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:51:25.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:25.753+0000 7f1234a00640 1 --2- 172.21.15.5:0/4040059120 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d121319020 0x55d121316870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:25.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:25.753+0000 7f1234a00640 1 -- 172.21.15.5:0/4040059120 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d12130e760 msgr2=0x55d121316330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:51:25.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:25.753+0000 7f1234a00640 1 --2- 172.21.15.5:0/4040059120 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d12130e760 0x55d121316330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:25.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:25.753+0000 7f1234a00640 1 -- 172.21.15.5:0/4040059120 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d120fe53c0 con 0x55d12130de40 2024-09-10T09:51:25.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:25.753+0000 7f1235400640 1 --2- 172.21.15.5:0/4040059120 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d121319020 0x55d121316870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:51:25.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:25.754+0000 7f1234a00640 1 --2- 172.21.15.5:0/4040059120 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d12130de40 0x55d121315df0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f1224018490 tx=0x7f1224009700 comp rx=0 tx=0).ready entity=mon.2 client_cookie=771c9c07921eb317 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:51:25.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:25.754+0000 7f1216000640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-09-10T09:51:25.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:25.754+0000 7f1216000640 1 -- 172.21.15.5:0/4040059120 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12240de030 con 0x55d12130de40 2024-09-10T09:51:25.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:25.754+0000 7f1216000640 1 -- 172.21.15.5:0/4040059120 <== mon.2 v2:172.21.15.102:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f12240cce90 con 0x55d12130de40 2024-09-10T09:51:25.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:25.754+0000 7f1239fbd740 1 -- 172.21.15.5:0/4040059120 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f12240cfcc0 con 0x55d12130de40 2024-09-10T09:51:25.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:25.754+0000 7f1216000640 1 -- 172.21.15.5:0/4040059120 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f1224097e30 con 0x55d12130de40 2024-09-10T09:51:25.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:25.754+0000 7f1239fbd740 10 client.28040 fetch_fsmap learned FSMap version 36 2024-09-10T09:51:25.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:25.754+0000 7f1239fbd740 1 -- 172.21.15.5:0/4040059120 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55d121327ba0 con 0x55d12130de40 2024-09-10T09:51:25.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:25.754+0000 7f1216000640 1 -- 172.21.15.5:0/4040059120 <== mon.2 v2:172.21.15.102:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f12240cfcc0 con 0x55d12130de40 2024-09-10T09:51:25.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:25.755+0000 7f1216000640 1 -- 172.21.15.5:0/4040059120 <== mon.2 v2:172.21.15.102:3300/0 5 ==== fsmap(e 36) ==== 8370+0+0 (secure 0 0 0) 0x7f122409bdf0 con 0x55d12130de40 2024-09-10T09:51:25.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:25.755+0000 7f1239fbd740 10 client.28040 fetch_fsmap finished waiting for FSMap version 36 2024-09-10T09:51:25.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:25.755+0000 7f1239fbd740 10 client.28040 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T09:51:25.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:25.755+0000 7f1239fbd740 20 client.28040 populate_metadata read hostname 'smithi005' 2024-09-10T09:51:25.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:25.755+0000 7f1239fbd740 1 --2- 172.21.15.5:0/4040059120 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55d121328ef0 0x55d12132b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:51:25.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:25.755+0000 7f1239fbd740 4 client.28040 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:51:25.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:25.755+0000 7f1239fbd740 1 -- 172.21.15.5:0/4040059120 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d121281950 con 0x55d121328ef0 2024-09-10T09:51:25.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:25.755+0000 7f122fe00640 1 --2- 172.21.15.5:0/4040059120 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55d121328ef0 0x55d12132b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:51:25.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:25.756+0000 7f122fe00640 1 --2- 172.21.15.5:0/4040059120 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55d121328ef0 0x55d12132b2e0 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d2df082515a6b27c server_cookie=c757965371418b9a in_seq=0 out_seq=0 2024-09-10T09:51:25.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:25.756+0000 7f1216000640 10 client.28040 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T09:51:25.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:25.756+0000 7f1216000640 1 -- 172.21.15.5:0/4040059120 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d121281950 con 0x55d121328ef0 2024-09-10T09:51:25.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:25.756+0000 7f1216000640 10 client.28040 handle_command_reply: tid=0 2024-09-10T09:51:25.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:25.756+0000 7f1239fbd740 1 client.28040 shutdown 2024-09-10T09:51:25.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:25.757+0000 7f1239fbd740 1 -- 172.21.15.5:0/4040059120 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d12130de40 msgr2=0x55d121315df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:51:25.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:25.757+0000 7f1239fbd740 1 --2- 172.21.15.5:0/4040059120 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d12130de40 0x55d121315df0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f1224018490 tx=0x7f1224009700 comp rx=0 tx=0).stop 2024-09-10T09:51:25.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:25.757+0000 7f1239fbd740 1 -- 172.21.15.5:0/4040059120 shutdown_connections 2024-09-10T09:51:25.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:25.757+0000 7f1239fbd740 1 --2- 172.21.15.5:0/4040059120 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55d121328ef0 0x55d12132b2e0 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:25.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:25.757+0000 7f1239fbd740 1 --2- 172.21.15.5:0/4040059120 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d121319020 0x55d121316870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:25.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:25.757+0000 7f1239fbd740 1 --2- 172.21.15.5:0/4040059120 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d12130e760 0x55d121316330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:25.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:25.757+0000 7f1239fbd740 1 --2- 172.21.15.5:0/4040059120 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d12130de40 0x55d121315df0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:25.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:25.757+0000 7f1239fbd740 1 -- 172.21.15.5:0/4040059120 >> 172.21.15.5:0/4040059120 conn(0x55d12130a3d0 msgr2=0x55d1213100b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:51:25.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:25.757+0000 7f1216000640 0 client.28040 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T09:51:25.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:25.757+0000 7f1239fbd740 1 -- 172.21.15.5:0/4040059120 shutdown_connections 2024-09-10T09:51:25.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:25.757+0000 7f1239fbd740 1 -- 172.21.15.5:0/4040059120 wait complete. 2024-09-10T09:51:25.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:25.757+0000 7f1239fbd740 20 client.28040 trim_cache size 0 max 16384 2024-09-10T09:51:25.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:25.774+0000 7f1239fbd740 1 Processor -- start 2024-09-10T09:51:25.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:25.774+0000 7f1239fbd740 1 -- start start 2024-09-10T09:51:25.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:25.774+0000 7f1239fbd740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d12130de40 0x55d121324760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:51:25.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:25.775+0000 7f1239fbd740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d12130e760 0x55d121313050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:51:25.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:25.775+0000 7f1239fbd740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d121319020 0x55d121313590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:51:25.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:25.775+0000 7f1239fbd740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f121000fcd0 con 0x55d12130e760 2024-09-10T09:51:25.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:25.775+0000 7f1239fbd740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f1210011810 con 0x55d121319020 2024-09-10T09:51:25.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:25.775+0000 7f1239fbd740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f12100103a0 con 0x55d12130de40 2024-09-10T09:51:25.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:25.775+0000 7f1234a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d12130de40 0x55d121324760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:51:25.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:25.775+0000 7f1234a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d12130de40 0x55d121324760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:43812/0 (socket says 172.21.15.5:43812) 2024-09-10T09:51:25.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:25.775+0000 7f1234a00640 1 -- 172.21.15.5:0/2120250430 learned_addr learned my addr 172.21.15.5:0/2120250430 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:51:25.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:25.775+0000 7f122fe00640 1 --2- 172.21.15.5:0/2120250430 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d12130e760 0x55d121313050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:51:25.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:25.775+0000 7f1235400640 1 --2- 172.21.15.5:0/2120250430 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d121319020 0x55d121313590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:51:25.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:25.775+0000 7f1234a00640 1 -- 172.21.15.5:0/2120250430 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d121319020 msgr2=0x55d121313590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:51:25.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:25.775+0000 7f1234a00640 1 --2- 172.21.15.5:0/2120250430 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d121319020 0x55d121313590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:25.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:25.775+0000 7f1234a00640 1 -- 172.21.15.5:0/2120250430 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d12130e760 msgr2=0x55d121313050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:51:25.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:25.775+0000 7f1234a00640 1 --2- 172.21.15.5:0/2120250430 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d12130e760 0x55d121313050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:25.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:25.775+0000 7f1234a00640 1 -- 172.21.15.5:0/2120250430 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d121309c80 con 0x55d12130de40 2024-09-10T09:51:25.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:25.776+0000 7f1234a00640 1 --2- 172.21.15.5:0/2120250430 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d12130de40 0x55d121324760 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f1224009770 tx=0x7f122409a0a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f1ad8737a85e4694 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:51:25.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:25.776+0000 7f1221200640 1 -- 172.21.15.5:0/2120250430 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12240de070 con 0x55d12130de40 2024-09-10T09:51:25.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:25.776+0000 7f1221200640 1 -- 172.21.15.5:0/2120250430 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f12240b3030 con 0x55d12130de40 2024-09-10T09:51:25.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:25.776+0000 7f1221200640 1 -- 172.21.15.5:0/2120250430 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f122400ce60 con 0x55d12130de40 2024-09-10T09:51:25.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:25.777+0000 7f1239fbd740 1 -- 172.21.15.5:0/2120250430 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d12130de40 msgr2=0x55d121324760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:51:25.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:25.777+0000 7f1239fbd740 1 --2- 172.21.15.5:0/2120250430 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d12130de40 0x55d121324760 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f1224009770 tx=0x7f122409a0a0 comp rx=0 tx=0).stop 2024-09-10T09:51:25.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:25.777+0000 7f1239fbd740 1 -- 172.21.15.5:0/2120250430 shutdown_connections 2024-09-10T09:51:25.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:25.777+0000 7f1239fbd740 1 --2- 172.21.15.5:0/2120250430 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d121319020 0x55d121313590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:25.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:25.777+0000 7f1239fbd740 1 --2- 172.21.15.5:0/2120250430 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d12130e760 0x55d121313050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:25.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:25.777+0000 7f1239fbd740 1 --2- 172.21.15.5:0/2120250430 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d12130de40 0x55d121324760 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:25.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:25.777+0000 7f1239fbd740 1 -- 172.21.15.5:0/2120250430 >> 172.21.15.5:0/2120250430 conn(0x55d12130a3d0 msgr2=0x55d121324050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:51:25.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:25.777+0000 7f1239fbd740 1 -- 172.21.15.5:0/2120250430 shutdown_connections 2024-09-10T09:51:25.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:25.777+0000 7f1239fbd740 1 -- 172.21.15.5:0/2120250430 wait complete. 2024-09-10T09:51:25.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:25.778+0000 7f1239fbd740 1 -- start start 2024-09-10T09:51:25.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:25.778+0000 7f1239fbd740 1 Processor -- start 2024-09-10T09:51:25.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:25.779+0000 7f1239fbd740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d12130de40 0x55d12132e670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:51:25.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:25.779+0000 7f1239fbd740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d12130e760 0x55d1213299f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:51:25.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:25.779+0000 7f1239fbd740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d121319020 0x55d121329f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:51:25.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:25.779+0000 7f1239fbd740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1210011140 con 0x55d12130e760 2024-09-10T09:51:25.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:25.779+0000 7f1239fbd740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f1210010a70 con 0x55d121319020 2024-09-10T09:51:25.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:25.779+0000 7f1239fbd740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f121000dcb0 con 0x55d12130de40 2024-09-10T09:51:25.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:25.779+0000 7f1239fbd740 10 client.0 _ll_register_callbacks cb 0x7f121c005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:51:25.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:25.779+0000 7f1234a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d12130de40 0x55d12132e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:51:25.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:25.779+0000 7f1234a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d12130de40 0x55d12132e670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:43816/0 (socket says 172.21.15.5:43816) 2024-09-10T09:51:25.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:25.779+0000 7f1234a00640 1 -- 172.21.15.5:0/1038723279 learned_addr learned my addr 172.21.15.5:0/1038723279 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:51:25.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:25.779+0000 7f122fe00640 1 --2- 172.21.15.5:0/1038723279 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d12130e760 0x55d1213299f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:51:25.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:25.779+0000 7f1235400640 1 --2- 172.21.15.5:0/1038723279 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d121319020 0x55d121329f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:51:25.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:25.780+0000 7f1234a00640 1 -- 172.21.15.5:0/1038723279 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d121319020 msgr2=0x55d121329f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:51:25.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:25.780+0000 7f1234a00640 1 --2- 172.21.15.5:0/1038723279 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d121319020 0x55d121329f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:25.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:25.780+0000 7f1234a00640 1 -- 172.21.15.5:0/1038723279 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d12130e760 msgr2=0x55d1213299f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:51:25.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:25.780+0000 7f1234a00640 1 --2- 172.21.15.5:0/1038723279 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d12130e760 0x55d1213299f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:25.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:25.780+0000 7f1234a00640 1 -- 172.21.15.5:0/1038723279 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d12132a760 con 0x55d12130de40 2024-09-10T09:51:25.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:25.780+0000 7f1234a00640 1 --2- 172.21.15.5:0/1038723279 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d12130de40 0x55d12132e670 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f122409b890 tx=0x7f12240d5be0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a11480700da922cb server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:51:25.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:25.780+0000 7f1216000640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-09-10T09:51:25.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:25.780+0000 7f1216000640 1 -- 172.21.15.5:0/1038723279 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12240de030 con 0x55d12130de40 2024-09-10T09:51:25.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:25.781+0000 7f1216000640 1 -- 172.21.15.5:0/1038723279 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f122409bdf0 con 0x55d12130de40 2024-09-10T09:51:25.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:25.781+0000 7f1239fbd740 1 -- 172.21.15.5:0/1038723279 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f12240b3030 con 0x55d12130de40 2024-09-10T09:51:25.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:25.781+0000 7f1216000640 1 -- 172.21.15.5:0/1038723279 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f12240b9070 con 0x55d12130de40 2024-09-10T09:51:25.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:25.781+0000 7f1239fbd740 10 client.28042 fetch_fsmap learned FSMap version 36 2024-09-10T09:51:25.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:25.781+0000 7f1239fbd740 1 -- 172.21.15.5:0/1038723279 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55d12132a760 con 0x55d12130de40 2024-09-10T09:51:25.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:25.781+0000 7f1216000640 1 -- 172.21.15.5:0/1038723279 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f12240b3030 con 0x55d12130de40 2024-09-10T09:51:25.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:25.781+0000 7f1216000640 1 -- 172.21.15.5:0/1038723279 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 36) ==== 8370+0+0 (secure 0 0 0) 0x7f12240b4ce0 con 0x55d12130de40 2024-09-10T09:51:25.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:25.781+0000 7f1239fbd740 10 client.28042 fetch_fsmap finished waiting for FSMap version 36 2024-09-10T09:51:25.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:25.781+0000 7f1239fbd740 10 client.28042 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T09:51:25.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:25.781+0000 7f1239fbd740 20 client.28042 populate_metadata read hostname 'smithi005' 2024-09-10T09:51:25.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:25.781+0000 7f1239fbd740 1 --2- 172.21.15.5:0/1038723279 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55d121393a60 0x55d121395e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:51:25.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:25.781+0000 7f1239fbd740 4 client.28042 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T09:51:25.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:25.781+0000 7f1239fbd740 1 -- 172.21.15.5:0/1038723279 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d12127be60 con 0x55d121393a60 2024-09-10T09:51:25.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:25.782+0000 7f122fe00640 1 --2- 172.21.15.5:0/1038723279 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55d121393a60 0x55d121395e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:51:25.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:25.782+0000 7f122fe00640 1 --2- 172.21.15.5:0/1038723279 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55d121393a60 0x55d121395e50 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dfb53c0a6820723 server_cookie=5225242dc0415a28 in_seq=0 out_seq=0 2024-09-10T09:51:25.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:25.782+0000 7f1216000640 10 client.28042 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T09:51:25.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:25.783+0000 7f1216000640 1 -- 172.21.15.5:0/1038723279 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d12127be60 con 0x55d121393a60 2024-09-10T09:51:25.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:25.783+0000 7f1216000640 10 client.28042 handle_command_reply: tid=0 2024-09-10T09:51:25.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:25.783+0000 7f1239fbd740 1 client.28042 shutdown 2024-09-10T09:51:25.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:25.783+0000 7f1239fbd740 1 -- 172.21.15.5:0/1038723279 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d12130de40 msgr2=0x55d12132e670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:51:25.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:25.783+0000 7f1239fbd740 1 --2- 172.21.15.5:0/1038723279 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d12130de40 0x55d12132e670 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f122409b890 tx=0x7f12240d5be0 comp rx=0 tx=0).stop 2024-09-10T09:51:25.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:25.783+0000 7f1239fbd740 1 -- 172.21.15.5:0/1038723279 shutdown_connections 2024-09-10T09:51:25.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:25.783+0000 7f1239fbd740 1 --2- 172.21.15.5:0/1038723279 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55d121393a60 0x55d121395e50 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:25.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:25.783+0000 7f1239fbd740 1 --2- 172.21.15.5:0/1038723279 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d121319020 0x55d121329f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:25.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:25.783+0000 7f1216000640 0 client.28042 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T09:51:25.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:25.783+0000 7f1239fbd740 1 --2- 172.21.15.5:0/1038723279 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d12130e760 0x55d1213299f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:25.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:25.783+0000 7f1239fbd740 1 --2- 172.21.15.5:0/1038723279 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d12130de40 0x55d12132e670 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:25.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:25.783+0000 7f1239fbd740 1 -- 172.21.15.5:0/1038723279 >> 172.21.15.5:0/1038723279 conn(0x55d12130a3d0 msgr2=0x55d121324870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:51:25.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:25.784+0000 7f1239fbd740 1 -- 172.21.15.5:0/1038723279 shutdown_connections 2024-09-10T09:51:25.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:25.784+0000 7f1239fbd740 1 -- 172.21.15.5:0/1038723279 wait complete. 2024-09-10T09:51:25.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:25.784+0000 7f1239fbd740 20 client.28042 trim_cache size 0 max 16384 2024-09-10T09:51:25.788 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-10T09:51:25.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:25.785+0000 7f1235e00640 1 -- 172.21.15.5:0/3339487848 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f12180a2c00 msgr2=0x7f12180a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:51:25.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:25.785+0000 7f1235e00640 1 --2- 172.21.15.5:0/3339487848 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f12180a2c00 0x7f12180a50c0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f1228006010 tx=0x7f1228005d80 comp rx=0 tx=0).stop 2024-09-10T09:51:25.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:25.785+0000 7f1235e00640 1 -- 172.21.15.5:0/3339487848 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1230103e30 msgr2=0x7f123019a3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:51:25.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:25.785+0000 7f1235e00640 1 --2- 172.21.15.5:0/3339487848 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1230103e30 0x7f123019a3f0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f122400e9f0 tx=0x7f122400eeb0 comp rx=0 tx=0).stop 2024-09-10T09:51:25.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:25.785+0000 7f1235e00640 1 -- 172.21.15.5:0/3339487848 shutdown_connections 2024-09-10T09:51:25.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:25.785+0000 7f1235e00640 1 --2- 172.21.15.5:0/3339487848 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f12180a2c00 0x7f12180a50c0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:25.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:25.785+0000 7f1235e00640 1 --2- 172.21.15.5:0/3339487848 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1230108f60 0x7f123019ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:25.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:25.785+0000 7f1235e00640 1 --2- 172.21.15.5:0/3339487848 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f12301047e0 0x7f123019a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:25.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:25.785+0000 7f1235e00640 1 --2- 172.21.15.5:0/3339487848 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1230103e30 0x7f123019a3f0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:25.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:25.785+0000 7f1235e00640 1 -- 172.21.15.5:0/3339487848 >> 172.21.15.5:0/3339487848 conn(0x7f12300ff910 msgr2=0x7f1230100bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:51:25.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:25.785+0000 7f1235e00640 1 -- 172.21.15.5:0/3339487848 shutdown_connections 2024-09-10T09:51:25.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:25.785+0000 7f1235e00640 1 -- 172.21.15.5:0/3339487848 wait complete. 2024-09-10T09:51:26.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:51:25 smithi005 ceph-mon[30350]: pgmap v785: 129 pgs: 129 active+clean; 306 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 1.3 MiB/s wr, 202 op/s 2024-09-10T09:51:26.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:51:25 smithi159 ceph-mon[35513]: pgmap v785: 129 pgs: 129 active+clean; 306 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 1.3 MiB/s wr, 202 op/s 2024-09-10T09:51:26.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:51:25 smithi102 ceph-mon[35442]: pgmap v785: 129 pgs: 129 active+clean; 306 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 1.3 MiB/s wr, 202 op/s 2024-09-10T09:51:26.148 INFO:tasks.workunit.client.0.smithi005.stdout: 1 845479 6.47 MB/sec execute 547 sec latency 9.592 ms 2024-09-10T09:51:27.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:51:26 smithi005 ceph-mon[30350]: pgmap v786: 129 pgs: 129 active+clean; 307 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 1.3 MiB/s wr, 215 op/s 2024-09-10T09:51:27.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:51:26 smithi159 ceph-mon[35513]: pgmap v786: 129 pgs: 129 active+clean; 307 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 1.3 MiB/s wr, 215 op/s 2024-09-10T09:51:27.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:51:26 smithi102 ceph-mon[35442]: pgmap v786: 129 pgs: 129 active+clean; 307 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 1.3 MiB/s wr, 215 op/s 2024-09-10T09:51:27.149 INFO:tasks.workunit.client.0.smithi005.stdout: 1 847427 6.47 MB/sec execute 548 sec latency 77.583 ms 2024-09-10T09:51:27.689 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-10T09:51:27.689 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-10T09:51:28.122 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:51:28.148 INFO:tasks.workunit.client.0.smithi005.stdout: 1 849355 6.48 MB/sec execute 549 sec latency 40.313 ms 2024-09-10T09:51:28.340 INFO:tasks.quiescer.fs.[cephfs]:Total duration (43.0) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-09-10T09:51:28.340 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 21.5 2024-09-10T09:51:28.340 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T09:51:28.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:28.709+0000 7fd715c00640 1 -- 172.21.15.5:0/1263331090 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd710101de0 msgr2=0x7fd7101021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:51:28.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:28.709+0000 7fd715c00640 1 --2- 172.21.15.5:0/1263331090 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd710101de0 0x7fd7101021c0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fd700009900 tx=0x7fd70002a3f0 comp rx=0 tx=0).stop 2024-09-10T09:51:28.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:28.709+0000 7fd715c00640 1 -- 172.21.15.5:0/1263331090 shutdown_connections 2024-09-10T09:51:28.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:28.709+0000 7fd715c00640 1 --2- 172.21.15.5:0/1263331090 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd710106f10 0x7fd71010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:28.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:28.709+0000 7fd715c00640 1 --2- 172.21.15.5:0/1263331090 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd710102790 0x7fd7101067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:28.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:28.709+0000 7fd715c00640 1 --2- 172.21.15.5:0/1263331090 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd710101de0 0x7fd7101021c0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:28.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:28.709+0000 7fd715c00640 1 -- 172.21.15.5:0/1263331090 >> 172.21.15.5:0/1263331090 conn(0x7fd7100fd8c0 msgr2=0x7fd7100ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:51:28.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:28.709+0000 7fd715c00640 1 -- 172.21.15.5:0/1263331090 shutdown_connections 2024-09-10T09:51:28.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:28.709+0000 7fd715c00640 1 -- 172.21.15.5:0/1263331090 wait complete. 2024-09-10T09:51:28.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:28.710+0000 7fd715c00640 1 Processor -- start 2024-09-10T09:51:28.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:28.710+0000 7fd715c00640 1 -- start start 2024-09-10T09:51:28.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:28.711+0000 7fd715c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd710101de0 0x7fd71019a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:51:28.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:28.711+0000 7fd715c00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd710102790 0x7fd71019ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:51:28.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:28.711+0000 7fd715c00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd710106f10 0x7fd71019eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:51:28.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:28.711+0000 7fd715c00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd710111f30 con 0x7fd710106f10 2024-09-10T09:51:28.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:28.711+0000 7fd715c00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fd710111db0 con 0x7fd710101de0 2024-09-10T09:51:28.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:28.711+0000 7fd715c00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fd7101120b0 con 0x7fd710102790 2024-09-10T09:51:28.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:28.711+0000 7fd70ea00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd710101de0 0x7fd71019a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:51:28.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:28.711+0000 7fd70ea00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd710101de0 0x7fd71019a610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:52522/0 (socket says 172.21.15.5:52522) 2024-09-10T09:51:28.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:28.711+0000 7fd70ea00640 1 -- 172.21.15.5:0/4266607576 learned_addr learned my addr 172.21.15.5:0/4266607576 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:51:28.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:28.711+0000 7fd70e000640 1 --2- 172.21.15.5:0/4266607576 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd710102790 0x7fd71019ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:51:28.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:28.711+0000 7fd70f400640 1 --2- 172.21.15.5:0/4266607576 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd710106f10 0x7fd71019eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:51:28.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:28.711+0000 7fd70ea00640 1 -- 172.21.15.5:0/4266607576 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd710102790 msgr2=0x7fd71019ab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:51:28.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:28.711+0000 7fd70ea00640 1 --2- 172.21.15.5:0/4266607576 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd710102790 0x7fd71019ab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:28.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:28.711+0000 7fd70ea00640 1 -- 172.21.15.5:0/4266607576 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd710106f10 msgr2=0x7fd71019eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:51:28.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:28.711+0000 7fd70ea00640 1 --2- 172.21.15.5:0/4266607576 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd710106f10 0x7fd71019eee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:28.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:28.711+0000 7fd70ea00640 1 -- 172.21.15.5:0/4266607576 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd71019f5c0 con 0x7fd710101de0 2024-09-10T09:51:28.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:28.712+0000 7fd70ea00640 1 --2- 172.21.15.5:0/4266607576 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd710101de0 0x7fd71019a610 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fd700006010 tx=0x7fd7000042f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:51:28.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:28.712+0000 7fd707400640 1 -- 172.21.15.5:0/4266607576 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd70003f030 con 0x7fd710101de0 2024-09-10T09:51:28.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:28.712+0000 7fd715c00640 1 -- 172.21.15.5:0/4266607576 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd71019f850 con 0x7fd710101de0 2024-09-10T09:51:28.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:28.712+0000 7fd707400640 1 -- 172.21.15.5:0/4266607576 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd700003ce0 con 0x7fd710101de0 2024-09-10T09:51:28.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:28.712+0000 7fd707400640 1 -- 172.21.15.5:0/4266607576 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd700035070 con 0x7fd710101de0 2024-09-10T09:51:28.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:28.712+0000 7fd715c00640 1 -- 172.21.15.5:0/4266607576 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7101a7190 con 0x7fd710101de0 2024-09-10T09:51:28.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:28.715+0000 7fd707400640 1 -- 172.21.15.5:0/4266607576 <== mon.2 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7fd700042030 con 0x7fd710101de0 2024-09-10T09:51:28.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:28.717+0000 7fd7187cd740 1 Processor -- start 2024-09-10T09:51:28.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:28.717+0000 7fd707400640 1 --2- 172.21.15.5:0/4266607576 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fd6f40a2b80 0x7fd6f40a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:51:28.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:28.717+0000 7fd7187cd740 1 -- start start 2024-09-10T09:51:28.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:28.717+0000 7fd707400640 1 -- 172.21.15.5:0/4266607576 <== mon.2 v2:172.21.15.102:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fd7000ef850 con 0x7fd710101de0 2024-09-10T09:51:28.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:28.717+0000 7fd7187cd740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558af019dac0 0x558af019dea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:51:28.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:28.717+0000 7fd7187cd740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558af019e3e0 0x558af01a8760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:51:28.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:28.717+0000 7fd7187cd740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558af01a8ca0 0x558af01ab090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:51:28.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:28.717+0000 7fd70f400640 1 --2- 172.21.15.5:0/4266607576 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fd6f40a2b80 0x7fd6f40a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:51:28.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:28.717+0000 7fd7187cd740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x558af0104290 con 0x558af01a8ca0 2024-09-10T09:51:28.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:28.717+0000 7fd7187cd740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x558af00abb40 con 0x558af019dac0 2024-09-10T09:51:28.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:28.717+0000 7fd7187cd740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x558af0100250 con 0x558af019e3e0 2024-09-10T09:51:28.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:28.718+0000 7fd70f400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558af01a8ca0 0x558af01ab090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:51:28.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:28.718+0000 7fd70ea00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558af019dac0 0x558af019dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:51:28.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:28.718+0000 7fd70f400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558af01a8ca0 0x558af01ab090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38976/0 (socket says 172.21.15.5:38976) 2024-09-10T09:51:28.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:28.718+0000 7fd70f400640 1 -- 172.21.15.5:0/1453332149 learned_addr learned my addr 172.21.15.5:0/1453332149 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:51:28.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:28.718+0000 7fd70e000640 1 --2- 172.21.15.5:0/1453332149 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558af019e3e0 0x558af01a8760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:51:28.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:28.719+0000 7fd70ea00640 1 -- 172.21.15.5:0/1453332149 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558af019e3e0 msgr2=0x558af01a8760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:51:28.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:28.719+0000 7fd70ea00640 1 --2- 172.21.15.5:0/1453332149 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558af019e3e0 0x558af01a8760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:28.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:28.719+0000 7fd70ea00640 1 -- 172.21.15.5:0/1453332149 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558af01a8ca0 msgr2=0x558af01ab090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:51:28.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:28.719+0000 7fd70ea00640 1 --2- 172.21.15.5:0/1453332149 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558af01a8ca0 0x558af01ab090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:28.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:28.719+0000 7fd70ea00640 1 -- 172.21.15.5:0/1453332149 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558aefe756a0 con 0x558af019dac0 2024-09-10T09:51:28.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:28.719+0000 7fd70e000640 1 --2- 172.21.15.5:0/1453332149 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558af019e3e0 0x558af01a8760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:51:28.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:28.719+0000 7fd70f400640 1 --2- 172.21.15.5:0/1453332149 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558af01a8ca0 0x558af01ab090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:51:28.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:28.719+0000 7fd70ea00640 1 --2- 172.21.15.5:0/1453332149 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558af019dac0 0x558af019dea0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fd7000b80c0 tx=0x7fd7000bbaf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b07025a463b0d139 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:51:28.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:28.719+0000 7fd6fbe00640 1 -- 172.21.15.5:0/1453332149 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd700049040 con 0x558af019dac0 2024-09-10T09:51:28.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:28.719+0000 7fd6fbe00640 1 -- 172.21.15.5:0/1453332149 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd7000d1070 con 0x558af019dac0 2024-09-10T09:51:28.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:28.719+0000 7fd6fbe00640 1 -- 172.21.15.5:0/1453332149 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7000b82d0 con 0x558af019dac0 2024-09-10T09:51:28.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:28.720+0000 7fd7187cd740 1 -- 172.21.15.5:0/1453332149 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558af019dac0 msgr2=0x558af019dea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:51:28.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:28.720+0000 7fd7187cd740 1 --2- 172.21.15.5:0/1453332149 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558af019dac0 0x558af019dea0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fd7000b80c0 tx=0x7fd7000bbaf0 comp rx=0 tx=0).stop 2024-09-10T09:51:28.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:28.720+0000 7fd70f400640 1 --2- 172.21.15.5:0/4266607576 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fd6f40a2b80 0x7fd6f40a5040 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fd708012bf0 tx=0x7fd7080126e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:51:28.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:28.720+0000 7fd7187cd740 1 -- 172.21.15.5:0/1453332149 shutdown_connections 2024-09-10T09:51:28.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:28.721+0000 7fd7187cd740 1 --2- 172.21.15.5:0/1453332149 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558af01a8ca0 0x558af01ab090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:28.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:28.721+0000 7fd7187cd740 1 --2- 172.21.15.5:0/1453332149 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558af019e3e0 0x558af01a8760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:28.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:28.721+0000 7fd7187cd740 1 --2- 172.21.15.5:0/1453332149 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558af019dac0 0x558af019dea0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:28.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:28.721+0000 7fd7187cd740 1 -- 172.21.15.5:0/1453332149 >> 172.21.15.5:0/1453332149 conn(0x558af019a050 msgr2=0x558af019a430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:51:28.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:28.721+0000 7fd7187cd740 1 -- 172.21.15.5:0/1453332149 shutdown_connections 2024-09-10T09:51:28.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:28.721+0000 7fd7187cd740 1 -- 172.21.15.5:0/1453332149 wait complete. 2024-09-10T09:51:28.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:28.721+0000 7fd7187cd740 1 -- start start 2024-09-10T09:51:28.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:28.721+0000 7fd7187cd740 1 Processor -- start 2024-09-10T09:51:28.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:28.722+0000 7fd7187cd740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558af019e3e0 0x558af01a5b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:51:28.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:28.722+0000 7fd7187cd740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558af01a8ca0 0x558af01a6080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:51:28.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:28.722+0000 7fd7187cd740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558af01a65c0 0x558af01b4aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:51:28.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:28.722+0000 7fd7187cd740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x558af01ab630 con 0x558af01a65c0 2024-09-10T09:51:28.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:28.722+0000 7fd7187cd740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x558af00befa0 con 0x558af01a8ca0 2024-09-10T09:51:28.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:28.722+0000 7fd7187cd740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x558af01ab7b0 con 0x558af019e3e0 2024-09-10T09:51:28.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:28.722+0000 7fd7187cd740 10 client.0 _ll_register_callbacks cb 0x558aefe12ba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:51:28.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:28.722+0000 7fd70ea00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558af019e3e0 0x558af01a5b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:51:28.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:28.722+0000 7fd70ea00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558af019e3e0 0x558af01a5b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:43872/0 (socket says 172.21.15.5:43872) 2024-09-10T09:51:28.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:28.722+0000 7fd70ea00640 1 -- 172.21.15.5:0/885424229 learned_addr learned my addr 172.21.15.5:0/885424229 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:51:28.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:28.722+0000 7fd70f400640 1 --2- 172.21.15.5:0/885424229 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558af01a65c0 0x558af01b4aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:51:28.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:28.723+0000 7fd70ea00640 1 -- 172.21.15.5:0/885424229 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558af01a8ca0 msgr2=0x558af01a6080 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T09:51:28.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:28.723+0000 7fd70ea00640 1 --2- 172.21.15.5:0/885424229 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558af01a8ca0 0x558af01a6080 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:28.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:28.723+0000 7fd70ea00640 1 -- 172.21.15.5:0/885424229 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558af01a65c0 msgr2=0x558af01b4aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:51:28.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:28.723+0000 7fd70ea00640 1 --2- 172.21.15.5:0/885424229 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558af01a65c0 0x558af01b4aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:28.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:28.723+0000 7fd70ea00640 1 -- 172.21.15.5:0/885424229 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558af01b4fe0 con 0x558af019e3e0 2024-09-10T09:51:28.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:28.723+0000 7fd70ea00640 1 --2- 172.21.15.5:0/885424229 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558af019e3e0 0x558af01a5b40 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fd70003d040 tx=0x7fd700009560 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4e43e3b2dd25c19f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:51:28.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:28.723+0000 7fd6f8c00640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-09-10T09:51:28.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:28.723+0000 7fd6f8c00640 1 -- 172.21.15.5:0/885424229 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7000d9070 con 0x558af019e3e0 2024-09-10T09:51:28.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:28.723+0000 7fd7187cd740 1 -- 172.21.15.5:0/885424229 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd7000d1070 con 0x558af019e3e0 2024-09-10T09:51:28.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:28.723+0000 7fd6f8c00640 1 -- 172.21.15.5:0/885424229 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fd7000c4440 con 0x558af019e3e0 2024-09-10T09:51:28.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:28.723+0000 7fd6f8c00640 1 -- 172.21.15.5:0/885424229 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fd7000d4e60 con 0x558af019e3e0 2024-09-10T09:51:28.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:28.723+0000 7fd7187cd740 10 client.28051 fetch_fsmap learned FSMap version 36 2024-09-10T09:51:28.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:28.723+0000 7fd7187cd740 1 -- 172.21.15.5:0/885424229 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x558af01ab630 con 0x558af019e3e0 2024-09-10T09:51:28.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:28.724+0000 7fd6f8c00640 1 -- 172.21.15.5:0/885424229 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd7000d1070 con 0x558af019e3e0 2024-09-10T09:51:28.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:28.724+0000 7fd6f8c00640 1 -- 172.21.15.5:0/885424229 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 36) ==== 8370+0+0 (secure 0 0 0) 0x7fd70003bb90 con 0x558af019e3e0 2024-09-10T09:51:28.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:28.724+0000 7fd7187cd740 10 client.28051 fetch_fsmap finished waiting for FSMap version 36 2024-09-10T09:51:28.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:28.724+0000 7fd7187cd740 10 client.28051 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T09:51:28.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:28.724+0000 7fd7187cd740 20 client.28051 populate_metadata read hostname 'smithi005' 2024-09-10T09:51:28.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:28.724+0000 7fd7187cd740 1 --2- 172.21.15.5:0/885424229 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x558af01b8de0 0x558af01bb210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:51:28.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:28.724+0000 7fd7187cd740 4 client.28051 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:51:28.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:28.724+0000 7fd7187cd740 1 -- 172.21.15.5:0/885424229 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558af0111800 con 0x558af01b8de0 2024-09-10T09:51:28.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:28.725+0000 7fd70e000640 1 --2- 172.21.15.5:0/885424229 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x558af01b8de0 0x558af01bb210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:51:28.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:28.725+0000 7fd70e000640 1 --2- 172.21.15.5:0/885424229 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x558af01b8de0 0x558af01bb210 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9c70011708f181a8 server_cookie=322da7d9de17c9c9 in_seq=0 out_seq=0 2024-09-10T09:51:28.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:28.725+0000 7fd6f8c00640 10 client.28051 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T09:51:28.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:28.726+0000 7fd6f8c00640 1 -- 172.21.15.5:0/885424229 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558af0111800 con 0x558af01b8de0 2024-09-10T09:51:28.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:28.726+0000 7fd6f8c00640 10 client.28051 handle_command_reply: tid=0 2024-09-10T09:51:28.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:28.726+0000 7fd7187cd740 1 client.28051 shutdown 2024-09-10T09:51:28.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:28.726+0000 7fd7187cd740 1 -- 172.21.15.5:0/885424229 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558af019e3e0 msgr2=0x558af01a5b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:51:28.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:28.726+0000 7fd7187cd740 1 --2- 172.21.15.5:0/885424229 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558af019e3e0 0x558af01a5b40 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fd70003d040 tx=0x7fd700009560 comp rx=0 tx=0).stop 2024-09-10T09:51:28.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:28.726+0000 7fd7187cd740 1 -- 172.21.15.5:0/885424229 shutdown_connections 2024-09-10T09:51:28.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:28.726+0000 7fd7187cd740 1 --2- 172.21.15.5:0/885424229 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x558af01b8de0 0x558af01bb210 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:28.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:28.726+0000 7fd7187cd740 1 --2- 172.21.15.5:0/885424229 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558af01a65c0 0x558af01b4aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:28.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:28.726+0000 7fd7187cd740 1 --2- 172.21.15.5:0/885424229 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558af01a8ca0 0x558af01a6080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:28.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:28.726+0000 7fd7187cd740 1 --2- 172.21.15.5:0/885424229 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558af019e3e0 0x558af01a5b40 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:28.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:28.726+0000 7fd7187cd740 1 -- 172.21.15.5:0/885424229 >> 172.21.15.5:0/885424229 conn(0x558af019a050 msgr2=0x558af019fd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:51:28.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:28.726+0000 7fd6f8c00640 0 client.28051 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T09:51:28.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:28.727+0000 7fd7187cd740 1 -- 172.21.15.5:0/885424229 shutdown_connections 2024-09-10T09:51:28.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:28.727+0000 7fd7187cd740 1 -- 172.21.15.5:0/885424229 wait complete. 2024-09-10T09:51:28.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:28.727+0000 7fd7187cd740 20 client.28051 trim_cache size 0 max 16384 2024-09-10T09:51:28.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:28.749+0000 7fd7187cd740 1 Processor -- start 2024-09-10T09:51:28.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:28.749+0000 7fd7187cd740 1 -- start start 2024-09-10T09:51:28.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:28.750+0000 7fd7187cd740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558af019e3e0 0x558af01b4890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:51:28.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:28.750+0000 7fd7187cd740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558af01a65c0 0x558af019fc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:51:28.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:28.750+0000 7fd7187cd740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558af01a8ca0 0x558af01c0ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:51:28.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:28.750+0000 7fd7187cd740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd7101b0f70 con 0x558af01a65c0 2024-09-10T09:51:28.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:28.750+0000 7fd7187cd740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fd7101b2ab0 con 0x558af01a8ca0 2024-09-10T09:51:28.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:28.750+0000 7fd7187cd740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fd7101b1640 con 0x558af019e3e0 2024-09-10T09:51:28.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:28.750+0000 7fd70ea00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558af019e3e0 0x558af01b4890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:51:28.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:28.750+0000 7fd70f400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558af01a8ca0 0x558af01c0ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:51:28.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:28.750+0000 7fd70ea00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558af019e3e0 0x558af01b4890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:43876/0 (socket says 172.21.15.5:43876) 2024-09-10T09:51:28.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:28.750+0000 7fd70ea00640 1 -- 172.21.15.5:0/2741865605 learned_addr learned my addr 172.21.15.5:0/2741865605 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:51:28.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:28.750+0000 7fd70e000640 1 --2- 172.21.15.5:0/2741865605 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558af01a65c0 0x558af019fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:51:28.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:28.750+0000 7fd70ea00640 1 -- 172.21.15.5:0/2741865605 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558af01a8ca0 msgr2=0x558af01c0ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:51:28.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:28.750+0000 7fd70ea00640 1 --2- 172.21.15.5:0/2741865605 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558af01a8ca0 0x558af01c0ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:28.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:28.750+0000 7fd70ea00640 1 -- 172.21.15.5:0/2741865605 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558af01a65c0 msgr2=0x558af019fc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:51:28.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:28.751+0000 7fd70ea00640 1 --2- 172.21.15.5:0/2741865605 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558af01a65c0 0x558af019fc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:28.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:28.751+0000 7fd70ea00640 1 -- 172.21.15.5:0/2741865605 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558af019ec60 con 0x558af019e3e0 2024-09-10T09:51:28.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:28.751+0000 7fd70ea00640 1 --2- 172.21.15.5:0/2741865605 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558af019e3e0 0x558af01b4890 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fd7000c43a0 tx=0x7fd70002a3f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=685312292a4fee1b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:51:28.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:28.751+0000 7fd6fb400640 1 -- 172.21.15.5:0/2741865605 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7000d9070 con 0x558af019e3e0 2024-09-10T09:51:28.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:28.751+0000 7fd6fb400640 1 -- 172.21.15.5:0/2741865605 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd7000d1070 con 0x558af019e3e0 2024-09-10T09:51:28.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:28.751+0000 7fd6fb400640 1 -- 172.21.15.5:0/2741865605 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd700049070 con 0x558af019e3e0 2024-09-10T09:51:28.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:28.752+0000 7fd7187cd740 1 -- 172.21.15.5:0/2741865605 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558af019e3e0 msgr2=0x558af01b4890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:51:28.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:28.752+0000 7fd7187cd740 1 --2- 172.21.15.5:0/2741865605 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558af019e3e0 0x558af01b4890 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fd7000c43a0 tx=0x7fd70002a3f0 comp rx=0 tx=0).stop 2024-09-10T09:51:28.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:28.752+0000 7fd7187cd740 1 -- 172.21.15.5:0/2741865605 shutdown_connections 2024-09-10T09:51:28.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:28.752+0000 7fd7187cd740 1 --2- 172.21.15.5:0/2741865605 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558af01a8ca0 0x558af01c0ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:28.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:28.752+0000 7fd7187cd740 1 --2- 172.21.15.5:0/2741865605 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558af01a65c0 0x558af019fc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:28.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:28.752+0000 7fd7187cd740 1 --2- 172.21.15.5:0/2741865605 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558af019e3e0 0x558af01b4890 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:28.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:28.752+0000 7fd7187cd740 1 -- 172.21.15.5:0/2741865605 >> 172.21.15.5:0/2741865605 conn(0x558af019a050 msgr2=0x558af019f510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:51:28.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:28.752+0000 7fd7187cd740 1 -- 172.21.15.5:0/2741865605 shutdown_connections 2024-09-10T09:51:28.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:28.752+0000 7fd7187cd740 1 -- 172.21.15.5:0/2741865605 wait complete. 2024-09-10T09:51:28.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:28.753+0000 7fd7187cd740 1 -- start start 2024-09-10T09:51:28.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:28.753+0000 7fd7187cd740 1 Processor -- start 2024-09-10T09:51:28.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:28.754+0000 7fd7187cd740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558af019e3e0 0x558af01ac8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:51:28.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:28.754+0000 7fd7187cd740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558af01a65c0 0x558af01ace10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:51:28.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:28.754+0000 7fd7187cd740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558af01a8ca0 0x558af01ad350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:51:28.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:28.755+0000 7fd7187cd740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd7101b23e0 con 0x558af019e3e0 2024-09-10T09:51:28.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:28.755+0000 7fd7187cd740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fd7101b1d10 con 0x558af01a8ca0 2024-09-10T09:51:28.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:28.755+0000 7fd7187cd740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fd7101aef50 con 0x558af01a65c0 2024-09-10T09:51:28.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:28.755+0000 7fd70e000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558af01a65c0 0x558af01ace10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:51:28.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:28.755+0000 7fd70e000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558af01a65c0 0x558af01ace10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:43882/0 (socket says 172.21.15.5:43882) 2024-09-10T09:51:28.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:28.755+0000 7fd70e000640 1 -- 172.21.15.5:0/1175585980 learned_addr learned my addr 172.21.15.5:0/1175585980 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:51:28.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:28.755+0000 7fd7187cd740 10 client.0 _ll_register_callbacks cb 0x7fd7080125b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:51:28.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:28.755+0000 7fd70ea00640 1 --2- 172.21.15.5:0/1175585980 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558af019e3e0 0x558af01ac8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:51:28.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:28.755+0000 7fd70f400640 1 --2- 172.21.15.5:0/1175585980 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558af01a8ca0 0x558af01ad350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:51:28.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:28.755+0000 7fd70e000640 1 -- 172.21.15.5:0/1175585980 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558af01a8ca0 msgr2=0x558af01ad350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:51:28.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:28.755+0000 7fd70e000640 1 --2- 172.21.15.5:0/1175585980 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558af01a8ca0 0x558af01ad350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:28.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:28.755+0000 7fd70e000640 1 -- 172.21.15.5:0/1175585980 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558af019e3e0 msgr2=0x558af01ac8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:51:28.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:28.755+0000 7fd70e000640 1 --2- 172.21.15.5:0/1175585980 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558af019e3e0 0x558af01ac8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:28.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:28.755+0000 7fd70e000640 1 -- 172.21.15.5:0/1175585980 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558af022b460 con 0x558af01a65c0 2024-09-10T09:51:28.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:28.755+0000 7fd70ea00640 1 --2- 172.21.15.5:0/1175585980 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558af019e3e0 0x558af01ac8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:51:28.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:28.755+0000 7fd70f400640 1 --2- 172.21.15.5:0/1175585980 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558af01a8ca0 0x558af01ad350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:51:28.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:28.756+0000 7fd70e000640 1 --2- 172.21.15.5:0/1175585980 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558af01a65c0 0x558af01ace10 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fd6fc00aac0 tx=0x7fd6fc0118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=67514e91f8b71e6d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:51:28.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:28.756+0000 7fd6f8c00640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-09-10T09:51:28.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:28.756+0000 7fd6f8c00640 1 -- 172.21.15.5:0/1175585980 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6fc005220 con 0x558af01a65c0 2024-09-10T09:51:28.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:28.756+0000 7fd7187cd740 1 -- 172.21.15.5:0/1175585980 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd7000d1070 con 0x558af01a65c0 2024-09-10T09:51:28.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:28.756+0000 7fd6f8c00640 1 -- 172.21.15.5:0/1175585980 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fd6fc0053c0 con 0x558af01a65c0 2024-09-10T09:51:28.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:28.756+0000 7fd6f8c00640 1 -- 172.21.15.5:0/1175585980 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fd6fc02b050 con 0x558af01a65c0 2024-09-10T09:51:28.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:28.756+0000 7fd7187cd740 10 client.28057 fetch_fsmap learned FSMap version 36 2024-09-10T09:51:28.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:28.756+0000 7fd7187cd740 1 -- 172.21.15.5:0/1175585980 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x558af022c7b0 con 0x558af01a65c0 2024-09-10T09:51:28.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:28.757+0000 7fd6f8c00640 1 -- 172.21.15.5:0/1175585980 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd7000d1070 con 0x558af01a65c0 2024-09-10T09:51:28.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:28.757+0000 7fd6f8c00640 1 -- 172.21.15.5:0/1175585980 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 36) ==== 8370+0+0 (secure 0 0 0) 0x7fd6fc01a930 con 0x558af01a65c0 2024-09-10T09:51:28.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:28.757+0000 7fd7187cd740 10 client.28057 fetch_fsmap finished waiting for FSMap version 36 2024-09-10T09:51:28.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:28.757+0000 7fd7187cd740 10 client.28057 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T09:51:28.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:28.757+0000 7fd7187cd740 20 client.28057 populate_metadata read hostname 'smithi005' 2024-09-10T09:51:28.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:28.757+0000 7fd7187cd740 1 --2- 172.21.15.5:0/1175585980 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x558af022cf10 0x558af022f330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:51:28.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:28.758+0000 7fd7187cd740 4 client.28057 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-10T09:51:28.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:28.758+0000 7fd7187cd740 1 -- 172.21.15.5:0/1175585980 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x558af01a6410 con 0x558af022cf10 2024-09-10T09:51:28.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:28.758+0000 7fd70ea00640 1 --2- 172.21.15.5:0/1175585980 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x558af022cf10 0x558af022f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:51:28.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:28.759+0000 7fd70ea00640 1 --2- 172.21.15.5:0/1175585980 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x558af022cf10 0x558af022f330 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9c147997182617f0 server_cookie=b8f1ab8a20a6406f in_seq=0 out_seq=0 2024-09-10T09:51:28.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:28.759+0000 7fd6f8c00640 10 client.28057 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T09:51:28.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:28.760+0000 7fd6f8c00640 1 -- 172.21.15.5:0/1175585980 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x558af01a6410 con 0x558af022cf10 2024-09-10T09:51:28.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:28.760+0000 7fd6f8c00640 10 client.28057 handle_command_reply: tid=0 2024-09-10T09:51:28.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:28.760+0000 7fd7187cd740 1 client.28057 shutdown 2024-09-10T09:51:28.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:28.760+0000 7fd7187cd740 1 -- 172.21.15.5:0/1175585980 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558af01a65c0 msgr2=0x558af01ace10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:51:28.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:28.760+0000 7fd7187cd740 1 --2- 172.21.15.5:0/1175585980 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558af01a65c0 0x558af01ace10 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fd6fc00aac0 tx=0x7fd6fc0118d0 comp rx=0 tx=0).stop 2024-09-10T09:51:28.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:28.760+0000 7fd7187cd740 1 -- 172.21.15.5:0/1175585980 shutdown_connections 2024-09-10T09:51:28.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:28.760+0000 7fd7187cd740 1 --2- 172.21.15.5:0/1175585980 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x558af022cf10 0x558af022f330 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:28.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:28.760+0000 7fd7187cd740 1 --2- 172.21.15.5:0/1175585980 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558af01a8ca0 0x558af01ad350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:28.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:28.760+0000 7fd7187cd740 1 --2- 172.21.15.5:0/1175585980 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558af01a65c0 0x558af01ace10 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:28.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:28.760+0000 7fd7187cd740 1 --2- 172.21.15.5:0/1175585980 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558af019e3e0 0x558af01ac8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:28.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:28.760+0000 7fd7187cd740 1 -- 172.21.15.5:0/1175585980 >> 172.21.15.5:0/1175585980 conn(0x558af019a050 msgr2=0x558af01991c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:51:28.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:28.760+0000 7fd6f8c00640 0 client.28057 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T09:51:28.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:28.760+0000 7fd7187cd740 1 -- 172.21.15.5:0/1175585980 shutdown_connections 2024-09-10T09:51:28.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:28.761+0000 7fd7187cd740 1 -- 172.21.15.5:0/1175585980 wait complete. 2024-09-10T09:51:28.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:28.761+0000 7fd7187cd740 20 client.28057 trim_cache size 0 max 16384 2024-09-10T09:51:28.764 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-10T09:51:28.764 INFO:teuthology.orchestra.run.smithi005.stdout: "return_code": 0, 2024-09-10T09:51:28.764 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_tag": "3e65a2aa-6d00-45aa-85bb-da0edd624968", 2024-09-10T09:51:28.764 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": "asynchronous" 2024-09-10T09:51:28.764 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-10T09:51:28.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:28.762+0000 7fd715c00640 1 -- 172.21.15.5:0/4266607576 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fd6f40a2b80 msgr2=0x7fd6f40a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:51:28.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:28.763+0000 7fd715c00640 1 --2- 172.21.15.5:0/4266607576 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fd6f40a2b80 0x7fd6f40a5040 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fd708012bf0 tx=0x7fd7080126e0 comp rx=0 tx=0).stop 2024-09-10T09:51:28.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:28.763+0000 7fd715c00640 1 -- 172.21.15.5:0/4266607576 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd710101de0 msgr2=0x7fd71019a610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:51:28.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:28.763+0000 7fd715c00640 1 --2- 172.21.15.5:0/4266607576 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd710101de0 0x7fd71019a610 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fd700006010 tx=0x7fd7000042f0 comp rx=0 tx=0).stop 2024-09-10T09:51:28.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:28.763+0000 7fd715c00640 1 -- 172.21.15.5:0/4266607576 shutdown_connections 2024-09-10T09:51:28.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:28.763+0000 7fd715c00640 1 --2- 172.21.15.5:0/4266607576 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fd6f40a2b80 0x7fd6f40a5040 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:28.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:28.763+0000 7fd715c00640 1 --2- 172.21.15.5:0/4266607576 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd710106f10 0x7fd71019eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:28.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:28.763+0000 7fd715c00640 1 --2- 172.21.15.5:0/4266607576 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd710102790 0x7fd71019ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:28.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:28.763+0000 7fd715c00640 1 --2- 172.21.15.5:0/4266607576 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd710101de0 0x7fd71019a610 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:28.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:28.763+0000 7fd715c00640 1 -- 172.21.15.5:0/4266607576 >> 172.21.15.5:0/4266607576 conn(0x7fd7100fd8c0 msgr2=0x7fd7100feb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:51:28.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:28.763+0000 7fd715c00640 1 -- 172.21.15.5:0/4266607576 shutdown_connections 2024-09-10T09:51:28.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:28.763+0000 7fd715c00640 1 -- 172.21.15.5:0/4266607576 wait complete. 2024-09-10T09:51:29.149 INFO:tasks.workunit.client.0.smithi005.stdout: 1 851913 6.50 MB/sec execute 550 sec latency 16.706 ms 2024-09-10T09:51:29.166 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:51:29.291 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:51:28 smithi005 ceph-mon[30350]: pgmap v787: 129 pgs: 129 active+clean; 309 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 1.1 MiB/s wr, 171 op/s 2024-09-10T09:51:29.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:51:28 smithi159 ceph-mon[35513]: pgmap v787: 129 pgs: 129 active+clean; 309 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 1.1 MiB/s wr, 171 op/s 2024-09-10T09:51:29.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:51:28 smithi102 ceph-mon[35442]: pgmap v787: 129 pgs: 129 active+clean; 309 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 1.1 MiB/s wr, 171 op/s 2024-09-10T09:51:29.659 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T09:51:30.149 INFO:tasks.workunit.client.0.smithi005.stdout: 1 852862 6.49 MB/sec execute 551 sec latency 220.275 ms 2024-09-10T09:51:30.170 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:51:29 smithi005 ceph-mon[30350]: scrub queued for path: / 2024-09-10T09:51:30.170 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:51:29 smithi005 ceph-mon[30350]: scrub summary: idle+waiting paths [/] 2024-09-10T09:51:30.170 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:51:29 smithi005 ceph-mon[30350]: scrub summary: active paths [/] 2024-09-10T09:51:30.170 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:51:29 smithi005 ceph-mon[30350]: scrub summary: active paths [/] 2024-09-10T09:51:30.170 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:51:29 smithi005 ceph-mon[30350]: scrub summary: idle+waiting paths [/] 2024-09-10T09:51:30.171 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:51:29 smithi005 ceph-mon[30350]: scrub summary: idle+waiting paths [/] 2024-09-10T09:51:30.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:51:29 smithi159 ceph-mon[35513]: scrub queued for path: / 2024-09-10T09:51:30.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:51:29 smithi159 ceph-mon[35513]: scrub summary: idle+waiting paths [/] 2024-09-10T09:51:30.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:51:29 smithi159 ceph-mon[35513]: scrub summary: active paths [/] 2024-09-10T09:51:30.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:51:29 smithi159 ceph-mon[35513]: scrub summary: active paths [/] 2024-09-10T09:51:30.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:51:29 smithi159 ceph-mon[35513]: scrub summary: idle+waiting paths [/] 2024-09-10T09:51:30.351 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:51:29 smithi159 ceph-mon[35513]: scrub summary: idle+waiting paths [/] 2024-09-10T09:51:30.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:51:29 smithi102 ceph-mon[35442]: scrub queued for path: / 2024-09-10T09:51:30.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:51:29 smithi102 ceph-mon[35442]: scrub summary: idle+waiting paths [/] 2024-09-10T09:51:30.385 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:51:29 smithi102 ceph-mon[35442]: scrub summary: active paths [/] 2024-09-10T09:51:30.385 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:51:29 smithi102 ceph-mon[35442]: scrub summary: active paths [/] 2024-09-10T09:51:30.385 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:51:29 smithi102 ceph-mon[35442]: scrub summary: idle+waiting paths [/] 2024-09-10T09:51:30.385 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:51:29 smithi102 ceph-mon[35442]: scrub summary: idle+waiting paths [/] 2024-09-10T09:51:31.149 INFO:tasks.workunit.client.0.smithi005.stdout: 1 852862 6.48 MB/sec execute 552 sec latency 1220.334 ms 2024-09-10T09:51:31.184 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:51:30 smithi005 ceph-mon[30350]: pgmap v788: 129 pgs: 129 active+clean; 313 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 1.4 MiB/s wr, 233 op/s 2024-09-10T09:51:31.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:31.147+0000 7f09e3e00640 1 -- 172.21.15.5:0/440117671 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f09e41079c0 msgr2=0x7f09e4107da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:51:31.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:31.147+0000 7f09e3e00640 1 --2- 172.21.15.5:0/440117671 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f09e41079c0 0x7f09e4107da0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f09d4009900 tx=0x7f09d402a3f0 comp rx=0 tx=0).stop 2024-09-10T09:51:31.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:31.148+0000 7f09e3e00640 1 -- 172.21.15.5:0/440117671 shutdown_connections 2024-09-10T09:51:31.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:31.148+0000 7f09e3e00640 1 --2- 172.21.15.5:0/440117671 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f09e4102360 0x7f09e410e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:31.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:31.148+0000 7f09e3e00640 1 --2- 172.21.15.5:0/440117671 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f09e41019c0 0x7f09e4101e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:31.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:31.148+0000 7f09e3e00640 1 --2- 172.21.15.5:0/440117671 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f09e41079c0 0x7f09e4107da0 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:31.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:31.148+0000 7f09e3e00640 1 -- 172.21.15.5:0/440117671 >> 172.21.15.5:0/440117671 conn(0x7f09e40fd740 msgr2=0x7f09e40ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:51:31.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:31.148+0000 7f09e3e00640 1 -- 172.21.15.5:0/440117671 shutdown_connections 2024-09-10T09:51:31.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:31.148+0000 7f09e3e00640 1 -- 172.21.15.5:0/440117671 wait complete. 2024-09-10T09:51:31.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:31.149+0000 7f09e3e00640 1 Processor -- start 2024-09-10T09:51:31.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:31.149+0000 7f09e3e00640 1 -- start start 2024-09-10T09:51:31.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:31.149+0000 7f09e3e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f09e41019c0 0x7f09e40744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:51:31.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:31.149+0000 7f09e3e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f09e4102360 0x7f09e4074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:51:31.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:31.149+0000 7f09e3e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f09e41079c0 0x7f09e4074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:51:31.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:31.149+0000 7f09e3e00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f09e4111dc0 con 0x7f09e41019c0 2024-09-10T09:51:31.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:31.149+0000 7f09e3e00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f09e4111c40 con 0x7f09e41079c0 2024-09-10T09:51:31.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:31.149+0000 7f09e3e00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f09e4111f40 con 0x7f09e4102360 2024-09-10T09:51:31.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:31.149+0000 7f09e2a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f09e41019c0 0x7f09e40744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:51:31.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:31.149+0000 7f09e2000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f09e4102360 0x7f09e4074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:51:31.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:31.149+0000 7f09e2a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f09e41019c0 0x7f09e40744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39012/0 (socket says 172.21.15.5:39012) 2024-09-10T09:51:31.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:31.149+0000 7f09e2a00640 1 -- 172.21.15.5:0/1668194176 learned_addr learned my addr 172.21.15.5:0/1668194176 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:51:31.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:31.150+0000 7f09e3400640 1 --2- 172.21.15.5:0/1668194176 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f09e41079c0 0x7f09e4074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:51:31.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:31.150+0000 7f09e2000640 1 -- 172.21.15.5:0/1668194176 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f09e41079c0 msgr2=0x7f09e4074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:51:31.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:31.150+0000 7f09e2000640 1 --2- 172.21.15.5:0/1668194176 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f09e41079c0 0x7f09e4074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:31.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:31.150+0000 7f09e2000640 1 -- 172.21.15.5:0/1668194176 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f09e41019c0 msgr2=0x7f09e40744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:51:31.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:31.150+0000 7f09e2000640 1 --2- 172.21.15.5:0/1668194176 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f09e41019c0 0x7f09e40744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:31.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:31.150+0000 7f09e2000640 1 -- 172.21.15.5:0/1668194176 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09e41a67a0 con 0x7f09e4102360 2024-09-10T09:51:31.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:31.150+0000 7f09e3400640 1 --2- 172.21.15.5:0/1668194176 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f09e41079c0 0x7f09e4074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:51:31.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:31.150+0000 7f09e2a00640 1 --2- 172.21.15.5:0/1668194176 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f09e41019c0 0x7f09e40744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:51:31.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:31.150+0000 7f09e2000640 1 --2- 172.21.15.5:0/1668194176 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f09e4102360 0x7f09e4074a30 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f09d000e9f0 tx=0x7f09d000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:51:31.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:31.151+0000 7f09db400640 1 -- 172.21.15.5:0/1668194176 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09d000ccc0 con 0x7f09e4102360 2024-09-10T09:51:31.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:31.151+0000 7f09e3e00640 1 -- 172.21.15.5:0/1668194176 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09e41a6a90 con 0x7f09e4102360 2024-09-10T09:51:31.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:31.151+0000 7f09db400640 1 -- 172.21.15.5:0/1668194176 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f09d0005020 con 0x7f09e4102360 2024-09-10T09:51:31.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:31.151+0000 7f09db400640 1 -- 172.21.15.5:0/1668194176 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09d00049e0 con 0x7f09e4102360 2024-09-10T09:51:31.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:31.151+0000 7f09e3e00640 1 -- 172.21.15.5:0/1668194176 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09e41a6fd0 con 0x7f09e4102360 2024-09-10T09:51:31.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:31.152+0000 7f09e3e00640 1 -- 172.21.15.5:0/1668194176 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f09e4101e20 con 0x7f09e4102360 2024-09-10T09:51:31.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:31.156+0000 7f09db400640 1 -- 172.21.15.5:0/1668194176 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f09d0005560 con 0x7f09e4102360 2024-09-10T09:51:31.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:31.157+0000 7f09db400640 1 --2- 172.21.15.5:0/1668194176 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f09c80a2c00 0x7f09c80a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:51:31.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:31.157+0000 7f09db400640 1 -- 172.21.15.5:0/1668194176 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f09d00cf380 con 0x7f09e4102360 2024-09-10T09:51:31.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:31.157+0000 7f09e2a00640 1 --2- 172.21.15.5:0/1668194176 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f09c80a2c00 0x7f09c80a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:51:31.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:31.157+0000 7f09db400640 1 -- 172.21.15.5:0/1668194176 <== mon.1 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f09d00051d0 con 0x7f09e4102360 2024-09-10T09:51:31.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:31.158+0000 7f09e2a00640 1 --2- 172.21.15.5:0/1668194176 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f09c80a2c00 0x7f09c80a50c0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f09d40096c0 tx=0x7f09d4005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:51:31.287 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:51:30 smithi102 ceph-mon[35442]: pgmap v788: 129 pgs: 129 active+clean; 313 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 1.4 MiB/s wr, 233 op/s 2024-09-10T09:51:31.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:51:30 smithi159 ceph-mon[35513]: pgmap v788: 129 pgs: 129 active+clean; 313 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 1.4 MiB/s wr, 233 op/s 2024-09-10T09:51:31.370 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:51:31.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:31.433+0000 7f09e3e00640 1 -- 172.21.15.5:0/1668194176 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f09e4075a20 con 0x7f09e4102360 2024-09-10T09:51:31.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:31.434+0000 7f09db400640 1 -- 172.21.15.5:0/1668194176 <== mon.1 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) ==== 94+0+11969 (secure 0 0 0) 0x7f09d009be90 con 0x7f09e4102360 2024-09-10T09:51:31.435 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-10T09:51:31.436 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":36,"btime":"2024-09-10T09:48:07:349075+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14454,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/2970594311","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2970594311},{"type":"v1","addr":"172.21.15.5:6835","nonce":2970594311}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14514,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6839/3616502174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":3616502174},{"type":"v1","addr":"172.21.15.5:6839","nonce":3616502174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":36,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T09:35:29.626287+0000","modified":"2024-09-10T09:48:06.970127+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24542,"mds_1":24535,"mds_2":24511,"mds_3":24506,"mds_4":24443},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14520":{"gid":14520,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/2073599842","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":2073599842},{"type":"v1","addr":"172.21.15.5:6841","nonce":2073599842}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"k","rank":4,"incarnation":26,"state":"up:active","state_seq":24,"addr":"172.21.15.102:6833/2457007427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":2457007427},{"type":"v1","addr":"172.21.15.102:6833","nonce":2457007427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/356445159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":356445159},{"type":"v1","addr":"172.21.15.159:6833","nonce":356445159}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6837/2894314507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":2894314507},{"type":"v1","addr":"172.21.15.5:6837","nonce":2894314507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.102:6835/1647549111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":1647549111},{"type":"v1","addr":"172.21.15.102:6835","nonce":1647549111}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6835/952657242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":952657242},{"type":"v1","addr":"172.21.15.159:6835","nonce":952657242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":19,"addr":"172.21.15.102:6837/2689776764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":2689776764},{"type":"v1","addr":"172.21.15.102:6837","nonce":2689776764}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.159:6837/620627639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":620627639},{"type":"v1","addr":"172.21.15.159:6837","nonce":620627639}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24535":{"gid":24535,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":18,"addr":"172.21.15.159:6839/2073739599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":2073739599},{"type":"v1","addr":"172.21.15.159:6839","nonce":2073739599}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24542":{"gid":24542,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.102:6839/335409749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6838","nonce":335409749},{"type":"v1","addr":"172.21.15.102:6839","nonce":335409749}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24542,"qdb_cluster":[24542,24535,24511,24506,24443]},"id":1}]} 2024-09-10T09:51:31.436 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 36 2024-09-10T09:51:31.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:31.437+0000 7f09e3e00640 1 -- 172.21.15.5:0/1668194176 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f09c80a2c00 msgr2=0x7f09c80a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:51:31.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:31.437+0000 7f09e3e00640 1 --2- 172.21.15.5:0/1668194176 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f09c80a2c00 0x7f09c80a50c0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f09d40096c0 tx=0x7f09d4005a20 comp rx=0 tx=0).stop 2024-09-10T09:51:31.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:31.437+0000 7f09e3e00640 1 -- 172.21.15.5:0/1668194176 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f09e4102360 msgr2=0x7f09e4074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:51:31.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:31.437+0000 7f09e3e00640 1 --2- 172.21.15.5:0/1668194176 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f09e4102360 0x7f09e4074a30 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f09d000e9f0 tx=0x7f09d000eeb0 comp rx=0 tx=0).stop 2024-09-10T09:51:31.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:31.437+0000 7f09e3e00640 1 -- 172.21.15.5:0/1668194176 shutdown_connections 2024-09-10T09:51:31.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:31.437+0000 7f09e3e00640 1 --2- 172.21.15.5:0/1668194176 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f09c80a2c00 0x7f09c80a50c0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:31.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:31.437+0000 7f09e3e00640 1 --2- 172.21.15.5:0/1668194176 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f09e41079c0 0x7f09e4074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:31.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:31.437+0000 7f09e3e00640 1 --2- 172.21.15.5:0/1668194176 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f09e4102360 0x7f09e4074a30 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:31.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:31.437+0000 7f09e3e00640 1 --2- 172.21.15.5:0/1668194176 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f09e41019c0 0x7f09e40744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:31.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:31.437+0000 7f09e3e00640 1 -- 172.21.15.5:0/1668194176 >> 172.21.15.5:0/1668194176 conn(0x7f09e40fd740 msgr2=0x7f09e40fe070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:51:31.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:31.437+0000 7f09e3e00640 1 -- 172.21.15.5:0/1668194176 shutdown_connections 2024-09-10T09:51:31.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:31.437+0000 7f09e3e00640 1 -- 172.21.15.5:0/1668194176 wait complete. 2024-09-10T09:51:32.149 INFO:tasks.workunit.client.0.smithi005.stdout: 1 855318 6.49 MB/sec execute 553 sec latency 1290.493 ms 2024-09-10T09:51:32.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:32.179+0000 7fc3a8c00640 1 -- 172.21.15.5:0/842534614 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc3a4076720 msgr2=0x7fc3a4076b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:51:32.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:32.179+0000 7fc3a8c00640 1 --2- 172.21.15.5:0/842534614 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc3a4076720 0x7fc3a4076b00 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fc394009900 tx=0x7fc39402a400 comp rx=0 tx=0).stop 2024-09-10T09:51:32.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:32.179+0000 7fc3a8c00640 1 -- 172.21.15.5:0/842534614 shutdown_connections 2024-09-10T09:51:32.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:32.179+0000 7fc3a8c00640 1 --2- 172.21.15.5:0/842534614 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc3a4112a30 0x7fc3a4114e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:32.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:32.179+0000 7fc3a8c00640 1 --2- 172.21.15.5:0/842534614 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc3a4077040 0x7fc3a41124f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:32.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:32.179+0000 7fc3a8c00640 1 --2- 172.21.15.5:0/842534614 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc3a4076720 0x7fc3a4076b00 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:32.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:32.179+0000 7fc3a8c00640 1 -- 172.21.15.5:0/842534614 >> 172.21.15.5:0/842534614 conn(0x7fc3a40ffa90 msgr2=0x7fc3a4101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:51:32.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:32.179+0000 7fc3a8c00640 1 -- 172.21.15.5:0/842534614 shutdown_connections 2024-09-10T09:51:32.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:32.179+0000 7fc3a8c00640 1 -- 172.21.15.5:0/842534614 wait complete. 2024-09-10T09:51:32.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:32.180+0000 7fc3a8c00640 1 Processor -- start 2024-09-10T09:51:32.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:32.181+0000 7fc3a8c00640 1 -- start start 2024-09-10T09:51:32.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:32.181+0000 7fc3a8c00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc3a4076720 0x7fc3a41a46b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:51:32.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:32.181+0000 7fc3a8c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc3a4077040 0x7fc3a41a4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:51:32.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:32.181+0000 7fc3a8c00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc3a4112a30 0x7fc3a419e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:51:32.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:32.181+0000 7fc3a8c00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc3a41185f0 con 0x7fc3a4076720 2024-09-10T09:51:32.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:32.181+0000 7fc3a8c00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fc3a4118470 con 0x7fc3a4077040 2024-09-10T09:51:32.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:32.181+0000 7fc3a8c00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fc3a4118770 con 0x7fc3a4112a30 2024-09-10T09:51:32.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:32.182+0000 7fc3a1400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc3a4077040 0x7fc3a41a4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:51:32.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:32.182+0000 7fc3a1400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc3a4077040 0x7fc3a41a4bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:32808/0 (socket says 172.21.15.5:32808) 2024-09-10T09:51:32.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:32.182+0000 7fc3a1400640 1 -- 172.21.15.5:0/4114331133 learned_addr learned my addr 172.21.15.5:0/4114331133 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:51:32.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:32.182+0000 7fc3a2800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc3a4112a30 0x7fc3a419e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:51:32.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:32.182+0000 7fc3a1e00640 1 --2- 172.21.15.5:0/4114331133 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc3a4076720 0x7fc3a41a46b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:51:32.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:32.182+0000 7fc3a2800640 1 -- 172.21.15.5:0/4114331133 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc3a4077040 msgr2=0x7fc3a41a4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:51:32.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:32.182+0000 7fc3a2800640 1 --2- 172.21.15.5:0/4114331133 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc3a4077040 0x7fc3a41a4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:32.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:32.182+0000 7fc3a2800640 1 -- 172.21.15.5:0/4114331133 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc3a4076720 msgr2=0x7fc3a41a46b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:51:32.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:32.182+0000 7fc3a2800640 1 --2- 172.21.15.5:0/4114331133 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc3a4076720 0x7fc3a41a46b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:32.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:32.182+0000 7fc3a2800640 1 -- 172.21.15.5:0/4114331133 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3a419efa0 con 0x7fc3a4112a30 2024-09-10T09:51:32.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:32.183+0000 7fc3a2800640 1 --2- 172.21.15.5:0/4114331133 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc3a4112a30 0x7fc3a419e870 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fc3a407aa10 tx=0x7fc39c00ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:51:32.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:32.183+0000 7fc39aa00640 1 -- 172.21.15.5:0/4114331133 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc39c00cdf0 con 0x7fc3a4112a30 2024-09-10T09:51:32.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:32.183+0000 7fc39aa00640 1 -- 172.21.15.5:0/4114331133 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc39c005020 con 0x7fc3a4112a30 2024-09-10T09:51:32.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:32.183+0000 7fc3a8c00640 1 -- 172.21.15.5:0/4114331133 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3a419f290 con 0x7fc3a4112a30 2024-09-10T09:51:32.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:32.183+0000 7fc39aa00640 1 -- 172.21.15.5:0/4114331133 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc39c0186d0 con 0x7fc3a4112a30 2024-09-10T09:51:32.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:32.183+0000 7fc3a8c00640 1 -- 172.21.15.5:0/4114331133 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3a41ab580 con 0x7fc3a4112a30 2024-09-10T09:51:32.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:32.187+0000 7fc39aa00640 1 -- 172.21.15.5:0/4114331133 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7fc39c004a40 con 0x7fc3a4112a30 2024-09-10T09:51:32.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:32.188+0000 7fc3ab814740 1 Processor -- start 2024-09-10T09:51:32.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:32.188+0000 7fc3ab814740 1 -- start start 2024-09-10T09:51:32.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:32.188+0000 7fc3ab814740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564fd820bb90 0x564fd820bf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:51:32.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:32.188+0000 7fc3ab814740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564fd820c4b0 0x564fd8216830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:51:32.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:32.188+0000 7fc3ab814740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564fd8216d70 0x564fd8219160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:51:32.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:32.188+0000 7fc3ab814740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x564fd81723e0 con 0x564fd820c4b0 2024-09-10T09:51:32.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:32.188+0000 7fc3ab814740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x564fd8119d80 con 0x564fd820bb90 2024-09-10T09:51:32.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:32.188+0000 7fc3ab814740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x564fd816e3a0 con 0x564fd8216d70 2024-09-10T09:51:32.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:32.189+0000 7fc3a1400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564fd820bb90 0x564fd820bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:51:32.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:32.189+0000 7fc3a1e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564fd820c4b0 0x564fd8216830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:51:32.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:32.189+0000 7fc3a1400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564fd820bb90 0x564fd820bf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:32822/0 (socket says 172.21.15.5:32822) 2024-09-10T09:51:32.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:32.189+0000 7fc3a1e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564fd820c4b0 0x564fd8216830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39044/0 (socket says 172.21.15.5:39044) 2024-09-10T09:51:32.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:32.189+0000 7fc3a1400640 1 -- 172.21.15.5:0/2606791838 learned_addr learned my addr 172.21.15.5:0/2606791838 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:51:32.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:32.189+0000 7fc3a1400640 1 --2- 172.21.15.5:0/2606791838 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564fd8216d70 0x564fd8219160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:51:32.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:32.189+0000 7fc39aa00640 1 --2- 172.21.15.5:0/4114331133 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fc3880a2c00 0x7fc3880a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:51:32.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:32.189+0000 7fc3a1e00640 1 -- 172.21.15.5:0/2606791838 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564fd8216d70 msgr2=0x564fd8219160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:51:32.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:32.189+0000 7fc3a1e00640 1 --2- 172.21.15.5:0/2606791838 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564fd8216d70 0x564fd8219160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:32.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:32.189+0000 7fc3a1e00640 1 -- 172.21.15.5:0/2606791838 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564fd820bb90 msgr2=0x564fd820bf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:51:32.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:32.189+0000 7fc3a1e00640 1 --2- 172.21.15.5:0/2606791838 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564fd820bb90 0x564fd820bf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:32.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:32.189+0000 7fc3a1e00640 1 -- 172.21.15.5:0/2606791838 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564fd8138ae0 con 0x564fd820c4b0 2024-09-10T09:51:32.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:32.190+0000 7fc39aa00640 1 -- 172.21.15.5:0/4114331133 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fc39c0cfab0 con 0x7fc3a4112a30 2024-09-10T09:51:32.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:32.190+0000 7fc3a2800640 1 --2- 172.21.15.5:0/4114331133 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fc3880a2c00 0x7fc3880a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:51:32.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:32.190+0000 7fc3a1e00640 1 --2- 172.21.15.5:0/2606791838 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564fd820c4b0 0x564fd8216830 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fc39402a9f0 tx=0x7fc394005e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bb5ee881ba72b09 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:51:32.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:32.190+0000 7fc38f000640 1 -- 172.21.15.5:0/2606791838 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc394035030 con 0x564fd820c4b0 2024-09-10T09:51:32.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:32.190+0000 7fc38f000640 1 -- 172.21.15.5:0/2606791838 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc3940043c0 con 0x564fd820c4b0 2024-09-10T09:51:32.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:32.190+0000 7fc38f000640 1 -- 172.21.15.5:0/2606791838 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc394004de0 con 0x564fd820c4b0 2024-09-10T09:51:32.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:32.190+0000 7fc3a2800640 1 --2- 172.21.15.5:0/4114331133 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fc3880a2c00 0x7fc3880a50c0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fc39c098890 tx=0x7fc39c005f70 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:51:32.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:32.191+0000 7fc3ab814740 1 -- 172.21.15.5:0/2606791838 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564fd820c4b0 msgr2=0x564fd8216830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:51:32.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:32.191+0000 7fc3ab814740 1 --2- 172.21.15.5:0/2606791838 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564fd820c4b0 0x564fd8216830 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fc39402a9f0 tx=0x7fc394005e60 comp rx=0 tx=0).stop 2024-09-10T09:51:32.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:32.191+0000 7fc3ab814740 1 -- 172.21.15.5:0/2606791838 shutdown_connections 2024-09-10T09:51:32.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:32.191+0000 7fc3ab814740 1 --2- 172.21.15.5:0/2606791838 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564fd8216d70 0x564fd8219160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:32.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:32.191+0000 7fc3ab814740 1 --2- 172.21.15.5:0/2606791838 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564fd820c4b0 0x564fd8216830 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:32.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:32.191+0000 7fc3ab814740 1 --2- 172.21.15.5:0/2606791838 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564fd820bb90 0x564fd820bf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:32.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:32.191+0000 7fc3ab814740 1 -- 172.21.15.5:0/2606791838 >> 172.21.15.5:0/2606791838 conn(0x564fd8208120 msgr2=0x564fd8208500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:51:32.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:32.191+0000 7fc3ab814740 1 -- 172.21.15.5:0/2606791838 shutdown_connections 2024-09-10T09:51:32.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:32.191+0000 7fc3ab814740 1 -- 172.21.15.5:0/2606791838 wait complete. 2024-09-10T09:51:32.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:32.192+0000 7fc3ab814740 1 -- start start 2024-09-10T09:51:32.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:32.192+0000 7fc3ab814740 1 Processor -- start 2024-09-10T09:51:32.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:32.193+0000 7fc3ab814740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564fd820bb90 0x564fd8213ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:51:32.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:32.193+0000 7fc3ab814740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564fd820c4b0 0x564fd8214220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:51:32.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:32.193+0000 7fc3ab814740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564fd8216d70 0x564fd8214760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:51:32.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:32.193+0000 7fc3ab814740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x564fd82199e0 con 0x564fd8216d70 2024-09-10T09:51:32.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:32.193+0000 7fc3ab814740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x564fd812d0f0 con 0x564fd820bb90 2024-09-10T09:51:32.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:32.193+0000 7fc3ab814740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x564fd8219b60 con 0x564fd820c4b0 2024-09-10T09:51:32.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:32.193+0000 7fc3ab814740 10 client.0 _ll_register_callbacks cb 0x564fd811e9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:51:32.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:32.193+0000 7fc3a1400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564fd820bb90 0x564fd8213ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:51:32.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:32.193+0000 7fc3a1400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564fd820bb90 0x564fd8213ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:32832/0 (socket says 172.21.15.5:32832) 2024-09-10T09:51:32.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:32.193+0000 7fc3a1400640 1 -- 172.21.15.5:0/244256795 learned_addr learned my addr 172.21.15.5:0/244256795 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:51:32.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:32.193+0000 7fc3a1400640 1 --2- 172.21.15.5:0/244256795 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564fd8216d70 0x564fd8214760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:51:32.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:32.193+0000 7fc3a1e00640 1 --2- 172.21.15.5:0/244256795 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564fd820c4b0 0x564fd8214220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:51:32.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:32.194+0000 7fc3a1400640 1 -- 172.21.15.5:0/244256795 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564fd820c4b0 msgr2=0x564fd8214220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:51:32.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:32.194+0000 7fc3a1400640 1 --2- 172.21.15.5:0/244256795 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564fd820c4b0 0x564fd8214220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:32.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:32.194+0000 7fc3a1400640 1 -- 172.21.15.5:0/244256795 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564fd8216d70 msgr2=0x564fd8214760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:51:32.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:32.194+0000 7fc3a1400640 1 --2- 172.21.15.5:0/244256795 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564fd8216d70 0x564fd8214760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:32.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:32.194+0000 7fc3a1400640 1 -- 172.21.15.5:0/244256795 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564fd7ee33c0 con 0x564fd820bb90 2024-09-10T09:51:32.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:32.194+0000 7fc3a1400640 1 --2- 172.21.15.5:0/244256795 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564fd820bb90 0x564fd8213ce0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fc390013960 tx=0x7fc390005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=46e6f7f22fa5d147 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:51:32.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:32.194+0000 7fc383400640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-09-10T09:51:32.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:32.194+0000 7fc383400640 1 -- 172.21.15.5:0/244256795 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc390004260 con 0x564fd820bb90 2024-09-10T09:51:32.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:32.195+0000 7fc383400640 1 -- 172.21.15.5:0/244256795 <== mon.2 v2:172.21.15.102:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fc390004400 con 0x564fd820bb90 2024-09-10T09:51:32.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:32.195+0000 7fc3ab814740 1 -- 172.21.15.5:0/244256795 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc3940043c0 con 0x564fd820bb90 2024-09-10T09:51:32.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:32.195+0000 7fc383400640 1 -- 172.21.15.5:0/244256795 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fc390028050 con 0x564fd820bb90 2024-09-10T09:51:32.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:32.195+0000 7fc3ab814740 10 client.28079 fetch_fsmap learned FSMap version 36 2024-09-10T09:51:32.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:32.195+0000 7fc3ab814740 1 -- 172.21.15.5:0/244256795 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x564fd8225760 con 0x564fd820bb90 2024-09-10T09:51:32.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:32.195+0000 7fc383400640 1 -- 172.21.15.5:0/244256795 <== mon.2 v2:172.21.15.102:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc3940043c0 con 0x564fd820bb90 2024-09-10T09:51:32.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:32.196+0000 7fc383400640 1 -- 172.21.15.5:0/244256795 <== mon.2 v2:172.21.15.102:3300/0 5 ==== fsmap(e 36) ==== 8370+0+0 (secure 0 0 0) 0x7fc390003920 con 0x564fd820bb90 2024-09-10T09:51:32.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:32.196+0000 7fc3ab814740 10 client.28079 fetch_fsmap finished waiting for FSMap version 36 2024-09-10T09:51:32.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:32.196+0000 7fc3ab814740 10 client.28079 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T09:51:32.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:32.196+0000 7fc3ab814740 20 client.28079 populate_metadata read hostname 'smithi005' 2024-09-10T09:51:32.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:32.196+0000 7fc3ab814740 1 --2- 172.21.15.5:0/244256795 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x564fd8226c20 0x564fd8229010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:51:32.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:32.196+0000 7fc3ab814740 4 client.28079 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:51:32.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:32.196+0000 7fc3ab814740 1 -- 172.21.15.5:0/244256795 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564fd817f950 con 0x564fd8226c20 2024-09-10T09:51:32.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:32.196+0000 7fc3a1e00640 1 --2- 172.21.15.5:0/244256795 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x564fd8226c20 0x564fd8229010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:51:32.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:32.197+0000 7fc3a1e00640 1 --2- 172.21.15.5:0/244256795 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x564fd8226c20 0x564fd8229010 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dd424ed1e383b226 server_cookie=2ba4db79f9fa1fe in_seq=0 out_seq=0 2024-09-10T09:51:32.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:32.197+0000 7fc383400640 10 client.28079 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T09:51:32.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:32.197+0000 7fc383400640 1 -- 172.21.15.5:0/244256795 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564fd817f950 con 0x564fd8226c20 2024-09-10T09:51:32.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:32.197+0000 7fc383400640 10 client.28079 handle_command_reply: tid=0 2024-09-10T09:51:32.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:32.197+0000 7fc3ab814740 1 client.28079 shutdown 2024-09-10T09:51:32.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:32.198+0000 7fc3ab814740 1 -- 172.21.15.5:0/244256795 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564fd820bb90 msgr2=0x564fd8213ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:51:32.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:32.198+0000 7fc3ab814740 1 --2- 172.21.15.5:0/244256795 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564fd820bb90 0x564fd8213ce0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fc390013960 tx=0x7fc390005f90 comp rx=0 tx=0).stop 2024-09-10T09:51:32.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:32.198+0000 7fc3ab814740 1 -- 172.21.15.5:0/244256795 shutdown_connections 2024-09-10T09:51:32.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:32.198+0000 7fc3ab814740 1 --2- 172.21.15.5:0/244256795 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x564fd8226c20 0x564fd8229010 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:32.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:32.198+0000 7fc3ab814740 1 --2- 172.21.15.5:0/244256795 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564fd8216d70 0x564fd8214760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:32.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:32.198+0000 7fc3ab814740 1 --2- 172.21.15.5:0/244256795 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564fd820c4b0 0x564fd8214220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:32.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:32.198+0000 7fc3ab814740 1 --2- 172.21.15.5:0/244256795 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564fd820bb90 0x564fd8213ce0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:32.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:32.198+0000 7fc3ab814740 1 -- 172.21.15.5:0/244256795 >> 172.21.15.5:0/244256795 conn(0x564fd8208120 msgr2=0x564fd820de50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:51:32.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:32.198+0000 7fc383400640 0 client.28079 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T09:51:32.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:32.198+0000 7fc3ab814740 1 -- 172.21.15.5:0/244256795 shutdown_connections 2024-09-10T09:51:32.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:32.198+0000 7fc3ab814740 1 -- 172.21.15.5:0/244256795 wait complete. 2024-09-10T09:51:32.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:32.198+0000 7fc3ab814740 20 client.28079 trim_cache size 0 max 16384 2024-09-10T09:51:32.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:32.218+0000 7fc3ab814740 1 Processor -- start 2024-09-10T09:51:32.220 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:51:31 smithi005 ceph-mon[30350]: from='client.? 172.21.15.5:0/1668194176' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:51:32.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:32.218+0000 7fc3ab814740 1 -- start start 2024-09-10T09:51:32.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:32.219+0000 7fc3ab814740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564fd820bb90 0x564fd822e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:51:32.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:32.219+0000 7fc3ab814740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564fd820c4b0 0x564fd822ee20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:51:32.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:32.219+0000 7fc3ab814740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564fd8216d70 0x564fd820d630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:51:32.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:32.219+0000 7fc3ab814740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc3a41b5440 con 0x564fd820bb90 2024-09-10T09:51:32.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:32.219+0000 7fc3ab814740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fc3a41b6f80 con 0x564fd8216d70 2024-09-10T09:51:32.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:32.219+0000 7fc3ab814740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fc3a41b5b10 con 0x564fd820c4b0 2024-09-10T09:51:32.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:32.219+0000 7fc3a1e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564fd820c4b0 0x564fd822ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:51:32.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:32.219+0000 7fc3a1e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564fd820c4b0 0x564fd822ee20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:55274/0 (socket says 172.21.15.5:55274) 2024-09-10T09:51:32.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:32.219+0000 7fc3a1400640 1 --2- 172.21.15.5:0/990917669 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564fd820bb90 0x564fd822e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:51:32.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:32.219+0000 7fc3a1e00640 1 -- 172.21.15.5:0/990917669 learned_addr learned my addr 172.21.15.5:0/990917669 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:51:32.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:32.219+0000 7fc3a1400640 1 --2- 172.21.15.5:0/990917669 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564fd8216d70 0x564fd820d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:51:32.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:32.220+0000 7fc3a1e00640 1 -- 172.21.15.5:0/990917669 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564fd8216d70 msgr2=0x564fd820d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:51:32.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:32.220+0000 7fc3a1e00640 1 --2- 172.21.15.5:0/990917669 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564fd8216d70 0x564fd820d630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:32.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:32.220+0000 7fc3a1e00640 1 -- 172.21.15.5:0/990917669 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564fd820bb90 msgr2=0x564fd822e8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:51:32.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:32.220+0000 7fc3a1e00640 1 --2- 172.21.15.5:0/990917669 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564fd820bb90 0x564fd822e8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:32.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:32.220+0000 7fc3a1e00640 1 -- 172.21.15.5:0/990917669 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564fd8207bc0 con 0x564fd820c4b0 2024-09-10T09:51:32.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:32.220+0000 7fc3a1400640 1 --2- 172.21.15.5:0/990917669 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564fd820bb90 0x564fd822e8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:51:32.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:32.220+0000 7fc3a1e00640 1 --2- 172.21.15.5:0/990917669 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564fd820c4b0 0x564fd822ee20 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fc39402a440 tx=0x7fc394005b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e06d087aaadedb40 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:51:32.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:32.221+0000 7fc38e600640 1 -- 172.21.15.5:0/990917669 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc394035040 con 0x564fd820c4b0 2024-09-10T09:51:32.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:32.221+0000 7fc38e600640 1 -- 172.21.15.5:0/990917669 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc3940042c0 con 0x564fd820c4b0 2024-09-10T09:51:32.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:32.221+0000 7fc38e600640 1 -- 172.21.15.5:0/990917669 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc394003850 con 0x564fd820c4b0 2024-09-10T09:51:32.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:32.222+0000 7fc3ab814740 1 -- 172.21.15.5:0/990917669 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564fd820c4b0 msgr2=0x564fd822ee20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:51:32.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:32.222+0000 7fc3ab814740 1 --2- 172.21.15.5:0/990917669 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564fd820c4b0 0x564fd822ee20 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fc39402a440 tx=0x7fc394005b50 comp rx=0 tx=0).stop 2024-09-10T09:51:32.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:32.222+0000 7fc3ab814740 1 -- 172.21.15.5:0/990917669 shutdown_connections 2024-09-10T09:51:32.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:32.222+0000 7fc3ab814740 1 --2- 172.21.15.5:0/990917669 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564fd8216d70 0x564fd820d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:32.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:32.222+0000 7fc3ab814740 1 --2- 172.21.15.5:0/990917669 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564fd820c4b0 0x564fd822ee20 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:32.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:32.222+0000 7fc3ab814740 1 --2- 172.21.15.5:0/990917669 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564fd820bb90 0x564fd822e8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:32.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:32.222+0000 7fc3ab814740 1 -- 172.21.15.5:0/990917669 >> 172.21.15.5:0/990917669 conn(0x564fd8208120 msgr2=0x564fd8212dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:51:32.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:32.222+0000 7fc3ab814740 1 -- 172.21.15.5:0/990917669 shutdown_connections 2024-09-10T09:51:32.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:32.222+0000 7fc3ab814740 1 -- 172.21.15.5:0/990917669 wait complete. 2024-09-10T09:51:32.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:32.223+0000 7fc3ab814740 1 -- start start 2024-09-10T09:51:32.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:32.223+0000 7fc3ab814740 1 Processor -- start 2024-09-10T09:51:32.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:32.224+0000 7fc3ab814740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564fd820bb90 0x564fd821f520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:51:32.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:32.224+0000 7fc3ab814740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564fd820c4b0 0x564fd821a850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:51:32.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:32.224+0000 7fc3ab814740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564fd8216d70 0x564fd821ad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:51:32.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:32.224+0000 7fc3ab814740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc3a41b68b0 con 0x564fd8216d70 2024-09-10T09:51:32.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:32.224+0000 7fc3ab814740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fc3a41b61e0 con 0x564fd820c4b0 2024-09-10T09:51:32.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:32.224+0000 7fc3ab814740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fc3a41b3420 con 0x564fd820bb90 2024-09-10T09:51:32.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:32.224+0000 7fc3ab814740 10 client.0 _ll_register_callbacks cb 0x7fc3900137b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:51:32.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:32.224+0000 7fc3a1e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564fd820c4b0 0x564fd821a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:51:32.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:32.224+0000 7fc3a1e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564fd820c4b0 0x564fd821a850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:32838/0 (socket says 172.21.15.5:32838) 2024-09-10T09:51:32.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:32.224+0000 7fc3a1400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564fd820bb90 0x564fd821f520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:51:32.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:32.224+0000 7fc3a1e00640 1 -- 172.21.15.5:0/2481625040 learned_addr learned my addr 172.21.15.5:0/2481625040 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:51:32.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:32.225+0000 7fc3a1e00640 1 -- 172.21.15.5:0/2481625040 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564fd820bb90 msgr2=0x564fd821f520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:51:32.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:32.225+0000 7fc3a1e00640 1 --2- 172.21.15.5:0/2481625040 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564fd820bb90 0x564fd821f520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:32.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:32.225+0000 7fc3a1e00640 1 -- 172.21.15.5:0/2481625040 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564fd8216d70 msgr2=0x564fd821ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:51:32.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:32.225+0000 7fc3a1e00640 1 --2- 172.21.15.5:0/2481625040 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564fd8216d70 0x564fd821ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:32.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:32.225+0000 7fc3a1e00640 1 -- 172.21.15.5:0/2481625040 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564fd821b5e0 con 0x564fd820c4b0 2024-09-10T09:51:32.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:32.225+0000 7fc3a1400640 1 --2- 172.21.15.5:0/2481625040 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564fd820bb90 0x564fd821f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:51:32.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:32.225+0000 7fc3a1e00640 1 --2- 172.21.15.5:0/2481625040 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564fd820c4b0 0x564fd821a850 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fc394005c40 tx=0x7fc3940098c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=128f82904d4c03f6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:51:32.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:32.225+0000 7fc383400640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-09-10T09:51:32.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:32.225+0000 7fc383400640 1 -- 172.21.15.5:0/2481625040 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3940315b0 con 0x564fd820c4b0 2024-09-10T09:51:32.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:32.225+0000 7fc3ab814740 1 -- 172.21.15.5:0/2481625040 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc3940042c0 con 0x564fd820c4b0 2024-09-10T09:51:32.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:32.225+0000 7fc383400640 1 -- 172.21.15.5:0/2481625040 <== mon.2 v2:172.21.15.102:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fc394031750 con 0x564fd820c4b0 2024-09-10T09:51:32.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:32.226+0000 7fc383400640 1 -- 172.21.15.5:0/2481625040 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fc394031950 con 0x564fd820c4b0 2024-09-10T09:51:32.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:32.226+0000 7fc3ab814740 10 client.28085 fetch_fsmap learned FSMap version 36 2024-09-10T09:51:32.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:32.226+0000 7fc3ab814740 1 -- 172.21.15.5:0/2481625040 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x564fd82918f0 con 0x564fd820c4b0 2024-09-10T09:51:32.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:32.226+0000 7fc383400640 1 -- 172.21.15.5:0/2481625040 <== mon.2 v2:172.21.15.102:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc3940042c0 con 0x564fd820c4b0 2024-09-10T09:51:32.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:32.226+0000 7fc383400640 1 -- 172.21.15.5:0/2481625040 <== mon.2 v2:172.21.15.102:3300/0 5 ==== fsmap(e 36) ==== 8370+0+0 (secure 0 0 0) 0x7fc39404e5c0 con 0x564fd820c4b0 2024-09-10T09:51:32.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:32.226+0000 7fc3ab814740 10 client.28085 fetch_fsmap finished waiting for FSMap version 36 2024-09-10T09:51:32.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:32.226+0000 7fc3ab814740 10 client.28085 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T09:51:32.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:32.226+0000 7fc3ab814740 20 client.28085 populate_metadata read hostname 'smithi005' 2024-09-10T09:51:32.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:32.226+0000 7fc3ab814740 1 --2- 172.21.15.5:0/2481625040 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x564fd8291b80 0x564fd8293f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:51:32.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:32.226+0000 7fc3ab814740 4 client.28085 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T09:51:32.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:32.226+0000 7fc3ab814740 1 -- 172.21.15.5:0/2481625040 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "scrub status"}) -- 0x564fd8179e60 con 0x564fd8291b80 2024-09-10T09:51:32.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:32.227+0000 7fc3a1400640 1 --2- 172.21.15.5:0/2481625040 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x564fd8291b80 0x564fd8293f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:51:32.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:32.227+0000 7fc3a1400640 1 --2- 172.21.15.5:0/2481625040 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x564fd8291b80 0x564fd8293f70 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fabe941580e1a20d server_cookie=51d607f22df07c7e in_seq=0 out_seq=0 2024-09-10T09:51:32.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:32.227+0000 7fc383400640 10 client.28085 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T09:51:32.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:32.228+0000 7fc383400640 1 -- 172.21.15.5:0/2481625040 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x564fd8179e60 con 0x564fd8291b80 2024-09-10T09:51:32.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:32.228+0000 7fc383400640 10 client.28085 handle_command_reply: tid=0 2024-09-10T09:51:32.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:32.228+0000 7fc3ab814740 1 client.28085 shutdown 2024-09-10T09:51:32.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:32.229+0000 7fc3ab814740 1 -- 172.21.15.5:0/2481625040 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564fd820c4b0 msgr2=0x564fd821a850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:51:32.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:32.229+0000 7fc3ab814740 1 --2- 172.21.15.5:0/2481625040 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564fd820c4b0 0x564fd821a850 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fc394005c40 tx=0x7fc3940098c0 comp rx=0 tx=0).stop 2024-09-10T09:51:32.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:32.229+0000 7fc3ab814740 1 -- 172.21.15.5:0/2481625040 shutdown_connections 2024-09-10T09:51:32.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:32.229+0000 7fc3ab814740 1 --2- 172.21.15.5:0/2481625040 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x564fd8291b80 0x564fd8293f70 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:32.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:32.229+0000 7fc3ab814740 1 --2- 172.21.15.5:0/2481625040 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564fd8216d70 0x564fd821ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:32.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:32.229+0000 7fc383400640 0 client.28085 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T09:51:32.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:32.229+0000 7fc3ab814740 1 --2- 172.21.15.5:0/2481625040 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564fd820c4b0 0x564fd821a850 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:32.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:32.229+0000 7fc3ab814740 1 --2- 172.21.15.5:0/2481625040 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564fd820bb90 0x564fd821f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:32.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:32.229+0000 7fc3ab814740 1 -- 172.21.15.5:0/2481625040 >> 172.21.15.5:0/2481625040 conn(0x564fd8208120 msgr2=0x564fd8215540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:51:32.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:32.229+0000 7fc3ab814740 1 -- 172.21.15.5:0/2481625040 shutdown_connections 2024-09-10T09:51:32.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:32.230+0000 7fc3ab814740 1 -- 172.21.15.5:0/2481625040 wait complete. 2024-09-10T09:51:32.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:32.231+0000 7fc3ab814740 20 client.28085 trim_cache size 0 max 16384 2024-09-10T09:51:32.232 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-10T09:51:32.232 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-10T09:51:32.232 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2024-09-10T09:51:32.232 INFO:teuthology.orchestra.run.smithi005.stdout: "3e65a2aa-6d00-45aa-85bb-da0edd624968": { 2024-09-10T09:51:32.232 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2024-09-10T09:51:32.232 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "3e65a2aa-6d00-45aa-85bb-da0edd624968", 2024-09-10T09:51:32.233 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2024-09-10T09:51:32.233 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T09:51:32.233 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T09:51:32.233 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-10T09:51:32.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:32.231+0000 7fc3a8c00640 1 -- 172.21.15.5:0/4114331133 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fc3880a2c00 msgr2=0x7fc3880a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:51:32.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:32.231+0000 7fc3a8c00640 1 --2- 172.21.15.5:0/4114331133 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fc3880a2c00 0x7fc3880a50c0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fc39c098890 tx=0x7fc39c005f70 comp rx=0 tx=0).stop 2024-09-10T09:51:32.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:32.231+0000 7fc3a8c00640 1 -- 172.21.15.5:0/4114331133 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc3a4112a30 msgr2=0x7fc3a419e870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:51:32.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:32.231+0000 7fc3a8c00640 1 --2- 172.21.15.5:0/4114331133 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc3a4112a30 0x7fc3a419e870 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fc3a407aa10 tx=0x7fc39c00ef10 comp rx=0 tx=0).stop 2024-09-10T09:51:32.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:32.232+0000 7fc3a8c00640 1 -- 172.21.15.5:0/4114331133 shutdown_connections 2024-09-10T09:51:32.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:32.232+0000 7fc3a8c00640 1 --2- 172.21.15.5:0/4114331133 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fc3880a2c00 0x7fc3880a50c0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:32.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:32.232+0000 7fc3a8c00640 1 --2- 172.21.15.5:0/4114331133 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc3a4112a30 0x7fc3a419e870 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:32.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:32.232+0000 7fc3a8c00640 1 --2- 172.21.15.5:0/4114331133 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc3a4077040 0x7fc3a41a4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:32.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:32.232+0000 7fc3a8c00640 1 --2- 172.21.15.5:0/4114331133 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc3a4076720 0x7fc3a41a46b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:32.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:32.232+0000 7fc3a8c00640 1 -- 172.21.15.5:0/4114331133 >> 172.21.15.5:0/4114331133 conn(0x7fc3a40ffa90 msgr2=0x7fc3a4101e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:51:32.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:32.232+0000 7fc3a8c00640 1 -- 172.21.15.5:0/4114331133 shutdown_connections 2024-09-10T09:51:32.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:32.232+0000 7fc3a8c00640 1 -- 172.21.15.5:0/4114331133 wait complete. 2024-09-10T09:51:32.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:51:31 smithi159 ceph-mon[35513]: from='client.? 172.21.15.5:0/1668194176' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:51:32.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:51:31 smithi102 ceph-mon[35442]: from='client.? 172.21.15.5:0/1668194176' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:51:33.110 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24542 quiesce db / --timeout 90 --expiration 141.5 --await-for 45.0' 2024-09-10T09:51:33.111 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24542 quiesce db / --timeout 90 --expiration 141.5 --await-for 45.0 2024-09-10T09:51:33.149 INFO:tasks.workunit.client.0.smithi005.stdout: 1 857079 6.50 MB/sec execute 554 sec latency 20.013 ms 2024-09-10T09:51:33.306 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:51:32 smithi005 ceph-mon[30350]: pgmap v789: 129 pgs: 129 active+clean; 313 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 1.4 MiB/s wr, 198 op/s 2024-09-10T09:51:33.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:51:32 smithi159 ceph-mon[35513]: pgmap v789: 129 pgs: 129 active+clean; 313 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 1.4 MiB/s wr, 198 op/s 2024-09-10T09:51:33.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:51:32 smithi102 ceph-mon[35442]: pgmap v789: 129 pgs: 129 active+clean; 313 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 1.4 MiB/s wr, 198 op/s 2024-09-10T09:51:33.508 INFO:tasks.cephfs.filesystem:scrub status for tag:3e65a2aa-6d00-45aa-85bb-da0edd624968 - {'path': '/', 'tag': '3e65a2aa-6d00-45aa-85bb-da0edd624968', 'options': 'recursive,force'} 2024-09-10T09:51:33.601 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:51:34.149 INFO:tasks.workunit.client.0.smithi005.stdout: 1 859594 6.51 MB/sec execute 555 sec latency 14.907 ms 2024-09-10T09:51:34.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:34.174+0000 7f35be200640 1 -- 172.21.15.5:0/2799121896 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f35b810ac90 msgr2=0x7f35b810d080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:51:34.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:34.174+0000 7f35be200640 1 --2- 172.21.15.5:0/2799121896 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f35b810ac90 0x7f35b810d080 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f35b0009be0 tx=0x7f35b002a6c0 comp rx=0 tx=0).stop 2024-09-10T09:51:34.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:34.174+0000 7f35be200640 1 -- 172.21.15.5:0/2799121896 shutdown_connections 2024-09-10T09:51:34.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:34.174+0000 7f35be200640 1 --2- 172.21.15.5:0/2799121896 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f35b810ac90 0x7f35b810d080 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:34.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:34.174+0000 7f35be200640 1 --2- 172.21.15.5:0/2799121896 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f35b8100910 0x7f35b8104f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:34.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:34.174+0000 7f35be200640 1 --2- 172.21.15.5:0/2799121896 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f35b8069290 0x7f35b81049f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:34.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:34.174+0000 7f35be200640 1 -- 172.21.15.5:0/2799121896 >> 172.21.15.5:0/2799121896 conn(0x7f35b80fba90 msgr2=0x7f35b80fdeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:51:34.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:34.174+0000 7f35be200640 1 -- 172.21.15.5:0/2799121896 shutdown_connections 2024-09-10T09:51:34.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:34.174+0000 7f35be200640 1 -- 172.21.15.5:0/2799121896 wait complete. 2024-09-10T09:51:34.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:34.176+0000 7f35be200640 1 Processor -- start 2024-09-10T09:51:34.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:34.176+0000 7f35be200640 1 -- start start 2024-09-10T09:51:34.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:34.176+0000 7f35be200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f35b8069290 0x7f35b8109ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:51:34.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:34.176+0000 7f35be200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f35b8100910 0x7f35b8109ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:51:34.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:34.176+0000 7f35be200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f35b810ac90 0x7f35b819eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:51:34.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:34.176+0000 7f35be200640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f35b8111f40 con 0x7f35b8069290 2024-09-10T09:51:34.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:34.176+0000 7f35be200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f35b8111dc0 con 0x7f35b810ac90 2024-09-10T09:51:34.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:34.176+0000 7f35be200640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f35b81120c0 con 0x7f35b8100910 2024-09-10T09:51:34.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:34.176+0000 7f35b6a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f35b8100910 0x7f35b8109ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:51:34.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:34.176+0000 7f35b7400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f35b8069290 0x7f35b8109ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:51:34.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:34.176+0000 7f35b6a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f35b8100910 0x7f35b8109ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:55290/0 (socket says 172.21.15.5:55290) 2024-09-10T09:51:34.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:34.176+0000 7f35b6a00640 1 -- 172.21.15.5:0/1672017349 learned_addr learned my addr 172.21.15.5:0/1672017349 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:51:34.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:34.176+0000 7f35b7e00640 1 --2- 172.21.15.5:0/1672017349 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f35b810ac90 0x7f35b819eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:51:34.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:34.177+0000 7f35b7e00640 1 -- 172.21.15.5:0/1672017349 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f35b8100910 msgr2=0x7f35b8109ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:51:34.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:34.177+0000 7f35b7e00640 1 --2- 172.21.15.5:0/1672017349 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f35b8100910 0x7f35b8109ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:34.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:34.177+0000 7f35b7e00640 1 -- 172.21.15.5:0/1672017349 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f35b8069290 msgr2=0x7f35b8109ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:51:34.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:34.177+0000 7f35b7e00640 1 --2- 172.21.15.5:0/1672017349 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f35b8069290 0x7f35b8109ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:34.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:34.177+0000 7f35b7e00640 1 -- 172.21.15.5:0/1672017349 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35b819f5b0 con 0x7f35b810ac90 2024-09-10T09:51:34.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:34.177+0000 7f35b7e00640 1 --2- 172.21.15.5:0/1672017349 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f35b810ac90 0x7f35b819eed0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f35b0034040 tx=0x7f35b00305d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:51:34.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:34.178+0000 7f35afe00640 1 -- 172.21.15.5:0/1672017349 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35b0041020 con 0x7f35b810ac90 2024-09-10T09:51:34.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:34.178+0000 7f35be200640 1 -- 172.21.15.5:0/1672017349 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35b819f840 con 0x7f35b810ac90 2024-09-10T09:51:34.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:34.178+0000 7f35be200640 1 -- 172.21.15.5:0/1672017349 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35b81a7180 con 0x7f35b810ac90 2024-09-10T09:51:34.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:34.178+0000 7f35afe00640 1 -- 172.21.15.5:0/1672017349 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f35b0003e50 con 0x7f35b810ac90 2024-09-10T09:51:34.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:34.178+0000 7f35afe00640 1 -- 172.21.15.5:0/1672017349 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35b0036040 con 0x7f35b810ac90 2024-09-10T09:51:34.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:34.181+0000 7f35c0922740 1 Processor -- start 2024-09-10T09:51:34.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:34.181+0000 7f35afe00640 1 -- 172.21.15.5:0/1672017349 <== mon.2 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f35b0044020 con 0x7f35b810ac90 2024-09-10T09:51:34.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:34.181+0000 7f35c0922740 1 -- start start 2024-09-10T09:51:34.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:34.182+0000 7f35c0922740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56252bf003b0 0x56252bf00790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:51:34.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:34.182+0000 7f35c0922740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56252bf00cd0 0x56252bf0b050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:51:34.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:34.182+0000 7f35c0922740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56252bf0b590 0x56252bf0d980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:51:34.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:34.182+0000 7f35c0922740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x56252be66530 con 0x56252bf00cd0 2024-09-10T09:51:34.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:34.182+0000 7f35c0922740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x56252be0d860 con 0x56252bf0b590 2024-09-10T09:51:34.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:34.182+0000 7f35c0922740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x56252be624f0 con 0x56252bf003b0 2024-09-10T09:51:34.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:34.182+0000 7f35b6a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56252bf003b0 0x56252bf00790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:51:34.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:34.182+0000 7f35b7400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56252bf00cd0 0x56252bf0b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:51:34.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:34.182+0000 7f35b7400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56252bf00cd0 0x56252bf0b050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39098/0 (socket says 172.21.15.5:39098) 2024-09-10T09:51:34.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:34.182+0000 7f35b7400640 1 -- 172.21.15.5:0/508508626 learned_addr learned my addr 172.21.15.5:0/508508626 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:51:34.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:34.182+0000 7f35b6a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56252bf003b0 0x56252bf00790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:55294/0 (socket says 172.21.15.5:55294) 2024-09-10T09:51:34.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:34.182+0000 7f35b6a00640 1 --2- 172.21.15.5:0/508508626 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56252bf0b590 0x56252bf0d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:51:34.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:34.183+0000 7f35b7400640 1 -- 172.21.15.5:0/508508626 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56252bf003b0 msgr2=0x56252bf00790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:51:34.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:34.183+0000 7f35b7400640 1 --2- 172.21.15.5:0/508508626 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56252bf003b0 0x56252bf00790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:34.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:34.183+0000 7f35b7400640 1 -- 172.21.15.5:0/508508626 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56252bf0b590 msgr2=0x56252bf0d980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:51:34.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:34.183+0000 7f35b7400640 1 --2- 172.21.15.5:0/508508626 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56252bf0b590 0x56252bf0d980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:34.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:34.183+0000 7f35b7400640 1 -- 172.21.15.5:0/508508626 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56252bbd7b90 con 0x56252bf00cd0 2024-09-10T09:51:34.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:34.183+0000 7f35afe00640 1 --2- 172.21.15.5:0/1672017349 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f359c0a2c00 0x7f359c0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:51:34.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:34.183+0000 7f35b7400640 1 --2- 172.21.15.5:0/508508626 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56252bf00cd0 0x56252bf0b050 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f35a8009cf0 tx=0x7f35a8011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93cd4ad346a43eca server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:51:34.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:34.183+0000 7f35afe00640 1 -- 172.21.15.5:0/1672017349 <== mon.2 v2:172.21.15.102:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f35b00efc20 con 0x7f35b810ac90 2024-09-10T09:51:34.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:34.183+0000 7f35b6a00640 1 --2- 172.21.15.5:0/1672017349 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f359c0a2c00 0x7f359c0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:51:34.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:34.184+0000 7f35a3e00640 1 -- 172.21.15.5:0/508508626 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35a8004230 con 0x56252bf00cd0 2024-09-10T09:51:34.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:34.184+0000 7f35a3e00640 1 -- 172.21.15.5:0/508508626 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f35a80043d0 con 0x56252bf00cd0 2024-09-10T09:51:34.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:34.184+0000 7f35a3e00640 1 -- 172.21.15.5:0/508508626 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35a8019840 con 0x56252bf00cd0 2024-09-10T09:51:34.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:34.184+0000 7f35c0922740 1 -- 172.21.15.5:0/508508626 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56252bf00cd0 msgr2=0x56252bf0b050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:51:34.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:34.184+0000 7f35b6a00640 1 --2- 172.21.15.5:0/1672017349 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f359c0a2c00 0x7f359c0a50c0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f35a4009fd0 tx=0x7f35a4012040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:51:34.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:34.184+0000 7f35c0922740 1 --2- 172.21.15.5:0/508508626 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56252bf00cd0 0x56252bf0b050 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f35a8009cf0 tx=0x7f35a8011810 comp rx=0 tx=0).stop 2024-09-10T09:51:34.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:34.185+0000 7f35c0922740 1 -- 172.21.15.5:0/508508626 shutdown_connections 2024-09-10T09:51:34.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:34.185+0000 7f35c0922740 1 --2- 172.21.15.5:0/508508626 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56252bf0b590 0x56252bf0d980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:34.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:34.185+0000 7f35c0922740 1 --2- 172.21.15.5:0/508508626 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56252bf00cd0 0x56252bf0b050 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:34.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:34.185+0000 7f35c0922740 1 --2- 172.21.15.5:0/508508626 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56252bf003b0 0x56252bf00790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:34.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:34.185+0000 7f35c0922740 1 -- 172.21.15.5:0/508508626 >> 172.21.15.5:0/508508626 conn(0x56252befc940 msgr2=0x56252befcd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:51:34.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:34.185+0000 7f35c0922740 1 -- 172.21.15.5:0/508508626 shutdown_connections 2024-09-10T09:51:34.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:34.185+0000 7f35c0922740 1 -- 172.21.15.5:0/508508626 wait complete. 2024-09-10T09:51:34.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:34.185+0000 7f35c0922740 1 -- start start 2024-09-10T09:51:34.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:34.186+0000 7f35c0922740 1 Processor -- start 2024-09-10T09:51:34.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:34.186+0000 7f35c0922740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56252bf003b0 0x56252bf086c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:51:34.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:34.186+0000 7f35c0922740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56252bf00cd0 0x56252bf08c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:51:34.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:34.186+0000 7f35c0922740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56252bf0b590 0x56252bf09140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:51:34.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:34.186+0000 7f35c0922740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x56252bf0dfb0 con 0x56252bf00cd0 2024-09-10T09:51:34.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:34.186+0000 7f35c0922740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x56252be20e60 con 0x56252bf003b0 2024-09-10T09:51:34.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:34.186+0000 7f35c0922740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x56252bf0e130 con 0x56252bf0b590 2024-09-10T09:51:34.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:34.186+0000 7f35c0922740 10 client.0 _ll_register_callbacks cb 0x56252be25cc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:51:34.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:34.186+0000 7f35b7400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56252bf003b0 0x56252bf086c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:51:34.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:34.186+0000 7f35b7400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56252bf003b0 0x56252bf086c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:32886/0 (socket says 172.21.15.5:32886) 2024-09-10T09:51:34.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:34.186+0000 7f35b7400640 1 -- 172.21.15.5:0/2979066091 learned_addr learned my addr 172.21.15.5:0/2979066091 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:51:34.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:34.186+0000 7f35b7e00640 1 --2- 172.21.15.5:0/2979066091 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56252bf0b590 0x56252bf09140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:51:34.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:34.186+0000 7f35b6a00640 1 --2- 172.21.15.5:0/2979066091 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56252bf00cd0 0x56252bf08c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:51:34.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:34.187+0000 7f35b7e00640 1 -- 172.21.15.5:0/2979066091 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56252bf003b0 msgr2=0x56252bf086c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:51:34.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:34.187+0000 7f35b7e00640 1 --2- 172.21.15.5:0/2979066091 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56252bf003b0 0x56252bf086c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:34.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:34.187+0000 7f35b7e00640 1 -- 172.21.15.5:0/2979066091 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56252bf00cd0 msgr2=0x56252bf08c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:51:34.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:34.187+0000 7f35b7e00640 1 --2- 172.21.15.5:0/2979066091 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56252bf00cd0 0x56252bf08c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:34.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:34.187+0000 7f35b7e00640 1 -- 172.21.15.5:0/2979066091 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56252be2c990 con 0x56252bf0b590 2024-09-10T09:51:34.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:34.187+0000 7f35b7400640 1 --2- 172.21.15.5:0/2979066091 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56252bf003b0 0x56252bf086c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:51:34.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:34.187+0000 7f35b7e00640 1 --2- 172.21.15.5:0/2979066091 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56252bf0b590 0x56252bf09140 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f35b00ed2d0 tx=0x7f35b00b7ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5cca7cae70567a2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:51:34.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:34.187+0000 7f35a0c00640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-09-10T09:51:34.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:34.187+0000 7f35a0c00640 1 -- 172.21.15.5:0/2979066091 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35b00bc030 con 0x56252bf0b590 2024-09-10T09:51:34.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:34.187+0000 7f35c0922740 1 -- 172.21.15.5:0/2979066091 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f35a80043d0 con 0x56252bf0b590 2024-09-10T09:51:34.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:34.187+0000 7f35a0c00640 1 -- 172.21.15.5:0/2979066091 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f35b00315d0 con 0x56252bf0b590 2024-09-10T09:51:34.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:34.188+0000 7f35a0c00640 1 -- 172.21.15.5:0/2979066091 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f35b00c03e0 con 0x56252bf0b590 2024-09-10T09:51:34.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:34.188+0000 7f35c0922740 10 client.28093 fetch_fsmap learned FSMap version 36 2024-09-10T09:51:34.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:34.188+0000 7f35c0922740 1 -- 172.21.15.5:0/2979066091 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x56252bf0e060 con 0x56252bf0b590 2024-09-10T09:51:34.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:34.188+0000 7f35a0c00640 1 -- 172.21.15.5:0/2979066091 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f35a80043d0 con 0x56252bf0b590 2024-09-10T09:51:34.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:34.188+0000 7f35a0c00640 1 -- 172.21.15.5:0/2979066091 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 36) ==== 8370+0+0 (secure 0 0 0) 0x7f35b0004c60 con 0x56252bf0b590 2024-09-10T09:51:34.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:34.188+0000 7f35c0922740 10 client.28093 fetch_fsmap finished waiting for FSMap version 36 2024-09-10T09:51:34.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:34.188+0000 7f35c0922740 10 client.28093 resolve_mds: validated gid 24542 aka daemon mds.b 2024-09-10T09:51:34.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:34.188+0000 7f35c0922740 20 client.28093 populate_metadata read hostname 'smithi005' 2024-09-10T09:51:34.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:34.188+0000 7f35c0922740 1 --2- 172.21.15.5:0/2979066091 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x56252bf1b210 0x56252bf1d680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:51:34.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:34.188+0000 7f35c0922740 4 client.28093 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:51:34.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:34.188+0000 7f35c0922740 1 -- 172.21.15.5:0/2979066091 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56252be09310 con 0x56252bf1b210 2024-09-10T09:51:34.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:34.189+0000 7f35b7400640 1 --2- 172.21.15.5:0/2979066091 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x56252bf1b210 0x56252bf1d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:51:34.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:34.189+0000 7f35b7400640 1 --2- 172.21.15.5:0/2979066091 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x56252bf1b210 0x56252bf1d680 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=adfb376757faba12 server_cookie=facf363888a96fe in_seq=0 out_seq=0 2024-09-10T09:51:34.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:34.189+0000 7f35a0c00640 10 client.28093 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T09:51:34.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:34.190+0000 7f35a0c00640 1 -- 172.21.15.5:0/2979066091 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56252be09310 con 0x56252bf1b210 2024-09-10T09:51:34.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:34.190+0000 7f35a0c00640 10 client.28093 handle_command_reply: tid=0 2024-09-10T09:51:34.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:34.190+0000 7f35c0922740 1 client.28093 shutdown 2024-09-10T09:51:34.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:34.190+0000 7f35c0922740 1 -- 172.21.15.5:0/2979066091 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56252bf0b590 msgr2=0x56252bf09140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:51:34.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:34.190+0000 7f35c0922740 1 --2- 172.21.15.5:0/2979066091 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56252bf0b590 0x56252bf09140 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f35b00ed2d0 tx=0x7f35b00b7ca0 comp rx=0 tx=0).stop 2024-09-10T09:51:34.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:34.190+0000 7f35c0922740 1 -- 172.21.15.5:0/2979066091 shutdown_connections 2024-09-10T09:51:34.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:34.190+0000 7f35c0922740 1 --2- 172.21.15.5:0/2979066091 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x56252bf1b210 0x56252bf1d680 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:34.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:34.190+0000 7f35c0922740 1 --2- 172.21.15.5:0/2979066091 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56252bf0b590 0x56252bf09140 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:34.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:34.190+0000 7f35c0922740 1 --2- 172.21.15.5:0/2979066091 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56252bf00cd0 0x56252bf08c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:34.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:34.190+0000 7f35c0922740 1 --2- 172.21.15.5:0/2979066091 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56252bf003b0 0x56252bf086c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:34.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:34.190+0000 7f35a0c00640 0 client.28093 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T09:51:34.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:34.190+0000 7f35c0922740 1 -- 172.21.15.5:0/2979066091 >> 172.21.15.5:0/2979066091 conn(0x56252befc940 msgr2=0x56252befbc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:51:34.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:34.190+0000 7f35c0922740 1 -- 172.21.15.5:0/2979066091 shutdown_connections 2024-09-10T09:51:34.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:34.190+0000 7f35c0922740 1 -- 172.21.15.5:0/2979066091 wait complete. 2024-09-10T09:51:34.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:34.191+0000 7f35c0922740 20 client.28093 trim_cache size 0 max 16384 2024-09-10T09:51:34.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:34.209+0000 7f35c0922740 1 Processor -- start 2024-09-10T09:51:34.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:34.209+0000 7f35c0922740 1 -- start start 2024-09-10T09:51:34.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:34.210+0000 7f35c0922740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56252bf003b0 0x56252bf0a520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:51:34.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:34.210+0000 7f35c0922740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56252bf00cd0 0x56252bf22f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:51:34.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:34.210+0000 7f35c0922740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56252bf0b590 0x56252bf234a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:51:34.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:34.210+0000 7f35c0922740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f35b81b0fc0 con 0x56252bf0b590 2024-09-10T09:51:34.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:34.210+0000 7f35c0922740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f35b81b2b00 con 0x56252bf00cd0 2024-09-10T09:51:34.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:34.210+0000 7f35c0922740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f35b81b1690 con 0x56252bf003b0 2024-09-10T09:51:34.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:34.210+0000 7f35b7400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56252bf003b0 0x56252bf0a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:51:34.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:34.210+0000 7f35b7400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56252bf003b0 0x56252bf0a520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:55316/0 (socket says 172.21.15.5:55316) 2024-09-10T09:51:34.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:34.210+0000 7f35b7400640 1 -- 172.21.15.5:0/41108274 learned_addr learned my addr 172.21.15.5:0/41108274 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:51:34.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:34.210+0000 7f35b6a00640 1 --2- 172.21.15.5:0/41108274 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56252bf00cd0 0x56252bf22f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:51:34.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:34.210+0000 7f35b7e00640 1 --2- 172.21.15.5:0/41108274 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56252bf0b590 0x56252bf234a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:51:34.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:34.210+0000 7f35b7400640 1 -- 172.21.15.5:0/41108274 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56252bf00cd0 msgr2=0x56252bf22f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:51:34.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:34.210+0000 7f35b7400640 1 --2- 172.21.15.5:0/41108274 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56252bf00cd0 0x56252bf22f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:34.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:34.210+0000 7f35b7400640 1 -- 172.21.15.5:0/41108274 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56252bf0b590 msgr2=0x56252bf234a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:51:34.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:34.210+0000 7f35b7400640 1 --2- 172.21.15.5:0/41108274 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56252bf0b590 0x56252bf234a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:34.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:34.210+0000 7f35b7400640 1 -- 172.21.15.5:0/41108274 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56252bf1dcd0 con 0x56252bf003b0 2024-09-10T09:51:34.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:34.211+0000 7f35b7400640 1 --2- 172.21.15.5:0/41108274 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56252bf003b0 0x56252bf0a520 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f35a8009c40 tx=0x7f35a8019420 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1a795e6b4266aa62 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:51:34.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:34.211+0000 7f35a3400640 1 -- 172.21.15.5:0/41108274 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35a8031070 con 0x56252bf003b0 2024-09-10T09:51:34.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:34.211+0000 7f35a3400640 1 -- 172.21.15.5:0/41108274 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f35a8005520 con 0x56252bf003b0 2024-09-10T09:51:34.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:34.211+0000 7f35a3400640 1 -- 172.21.15.5:0/41108274 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35a8019490 con 0x56252bf003b0 2024-09-10T09:51:34.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:34.212+0000 7f35c0922740 1 -- 172.21.15.5:0/41108274 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56252bf003b0 msgr2=0x56252bf0a520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:51:34.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:34.212+0000 7f35c0922740 1 --2- 172.21.15.5:0/41108274 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56252bf003b0 0x56252bf0a520 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f35a8009c40 tx=0x7f35a8019420 comp rx=0 tx=0).stop 2024-09-10T09:51:34.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:34.212+0000 7f35c0922740 1 -- 172.21.15.5:0/41108274 shutdown_connections 2024-09-10T09:51:34.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:34.212+0000 7f35c0922740 1 --2- 172.21.15.5:0/41108274 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56252bf0b590 0x56252bf234a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:34.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:34.212+0000 7f35c0922740 1 --2- 172.21.15.5:0/41108274 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56252bf00cd0 0x56252bf22f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:34.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:34.212+0000 7f35c0922740 1 --2- 172.21.15.5:0/41108274 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56252bf003b0 0x56252bf0a520 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:34.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:34.212+0000 7f35c0922740 1 -- 172.21.15.5:0/41108274 >> 172.21.15.5:0/41108274 conn(0x56252befc940 msgr2=0x56252bf09e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:51:34.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:34.212+0000 7f35c0922740 1 -- 172.21.15.5:0/41108274 shutdown_connections 2024-09-10T09:51:34.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:34.212+0000 7f35c0922740 1 -- 172.21.15.5:0/41108274 wait complete. 2024-09-10T09:51:34.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:34.212+0000 7f35c0922740 1 -- start start 2024-09-10T09:51:34.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:34.213+0000 7f35c0922740 1 Processor -- start 2024-09-10T09:51:34.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:34.213+0000 7f35c0922740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56252bf003b0 0x56252bf13ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:51:34.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:34.213+0000 7f35c0922740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56252bf00cd0 0x56252bf103a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:51:34.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:34.213+0000 7f35c0922740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56252bf0b590 0x56252bf108e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:51:34.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:34.213+0000 7f35c0922740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f35b81b2430 con 0x56252bf003b0 2024-09-10T09:51:34.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:34.213+0000 7f35c0922740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f35b81b1d60 con 0x56252bf0b590 2024-09-10T09:51:34.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:34.213+0000 7f35c0922740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f35b81aefa0 con 0x56252bf00cd0 2024-09-10T09:51:34.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:34.213+0000 7f35c0922740 10 client.0 _ll_register_callbacks cb 0x7f35b0005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:51:34.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:34.213+0000 7f35b6a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56252bf00cd0 0x56252bf103a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:51:34.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:34.213+0000 7f35b7e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56252bf0b590 0x56252bf108e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:51:34.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:34.213+0000 7f35b7e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56252bf0b590 0x56252bf108e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:32896/0 (socket says 172.21.15.5:32896) 2024-09-10T09:51:34.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:34.213+0000 7f35b6a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56252bf00cd0 0x56252bf103a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:55330/0 (socket says 172.21.15.5:55330) 2024-09-10T09:51:34.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:34.213+0000 7f35b7e00640 1 -- 172.21.15.5:0/4005156624 learned_addr learned my addr 172.21.15.5:0/4005156624 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:51:34.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:34.213+0000 7f35b7400640 1 --2- 172.21.15.5:0/4005156624 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56252bf003b0 0x56252bf13ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:51:34.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:34.214+0000 7f35b7e00640 1 -- 172.21.15.5:0/4005156624 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56252bf00cd0 msgr2=0x56252bf103a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:51:34.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:34.214+0000 7f35b7e00640 1 --2- 172.21.15.5:0/4005156624 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56252bf00cd0 0x56252bf103a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:34.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:34.214+0000 7f35b7e00640 1 -- 172.21.15.5:0/4005156624 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56252bf003b0 msgr2=0x56252bf13ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:51:34.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:34.214+0000 7f35b7e00640 1 --2- 172.21.15.5:0/4005156624 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56252bf003b0 0x56252bf13ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:34.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:34.214+0000 7f35b7e00640 1 -- 172.21.15.5:0/4005156624 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56252bf001e0 con 0x56252bf0b590 2024-09-10T09:51:34.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:34.214+0000 7f35b7400640 1 --2- 172.21.15.5:0/4005156624 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56252bf003b0 0x56252bf13ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:51:34.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:34.214+0000 7f35b7e00640 1 --2- 172.21.15.5:0/4005156624 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56252bf0b590 0x56252bf108e0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f35b0004af0 tx=0x7f35b00bb580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=32cd08cc9ac5e805 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:51:34.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:34.214+0000 7f35a0c00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-09-10T09:51:34.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:34.214+0000 7f35a0c00640 1 -- 172.21.15.5:0/4005156624 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35b00bc030 con 0x56252bf0b590 2024-09-10T09:51:34.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:34.214+0000 7f35a0c00640 1 -- 172.21.15.5:0/4005156624 <== mon.2 v2:172.21.15.102:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f35b0004d40 con 0x56252bf0b590 2024-09-10T09:51:34.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:34.214+0000 7f35c0922740 1 -- 172.21.15.5:0/4005156624 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f35a8005520 con 0x56252bf0b590 2024-09-10T09:51:34.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:34.215+0000 7f35a0c00640 1 -- 172.21.15.5:0/4005156624 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f35b00bb990 con 0x56252bf0b590 2024-09-10T09:51:34.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:34.215+0000 7f35c0922740 10 client.28103 fetch_fsmap learned FSMap version 36 2024-09-10T09:51:34.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:34.215+0000 7f35c0922740 1 -- 172.21.15.5:0/4005156624 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x56252bf001e0 con 0x56252bf0b590 2024-09-10T09:51:34.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:34.215+0000 7f35a0c00640 1 -- 172.21.15.5:0/4005156624 <== mon.2 v2:172.21.15.102:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f35a8005520 con 0x56252bf0b590 2024-09-10T09:51:34.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:34.215+0000 7f35a0c00640 1 -- 172.21.15.5:0/4005156624 <== mon.2 v2:172.21.15.102:3300/0 5 ==== fsmap(e 36) ==== 8370+0+0 (secure 0 0 0) 0x7f35b00a2060 con 0x56252bf0b590 2024-09-10T09:51:34.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:34.215+0000 7f35c0922740 10 client.28103 fetch_fsmap finished waiting for FSMap version 36 2024-09-10T09:51:34.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:34.215+0000 7f35c0922740 10 client.28103 resolve_mds: validated gid 24542 aka daemon mds.b 2024-09-10T09:51:34.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:34.215+0000 7f35c0922740 20 client.28103 populate_metadata read hostname 'smithi005' 2024-09-10T09:51:34.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:34.215+0000 7f35c0922740 1 --2- 172.21.15.5:0/4005156624 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x56252bf870e0 0x56252bf894d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:51:34.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:34.215+0000 7f35c0922740 4 client.28103 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 141.5, "await_for": 45.0}] 2024-09-10T09:51:34.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:34.215+0000 7f35c0922740 1 -- 172.21.15.5:0/4005156624 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 141.5, "await_for": 45.0}) -- 0x56252be73c60 con 0x56252bf870e0 2024-09-10T09:51:34.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:34.215+0000 7f35b7400640 1 --2- 172.21.15.5:0/4005156624 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x56252bf870e0 0x56252bf894d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:51:34.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:34.216+0000 7f35b7400640 1 --2- 172.21.15.5:0/4005156624 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x56252bf870e0 0x56252bf894d0 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8f869d9d5f9386cb server_cookie=c575b4b7cf837e93 in_seq=0 out_seq=0 2024-09-10T09:51:34.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:34.216+0000 7f35a0c00640 10 client.28103 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T09:51:34.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:34.413+0000 7f35a0c00640 1 -- 172.21.15.5:0/4005156624 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x56252be73c60 con 0x56252bf870e0 2024-09-10T09:51:34.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:34.413+0000 7f35a0c00640 10 client.28103 handle_command_reply: tid=0 2024-09-10T09:51:34.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:34.413+0000 7f35c0922740 1 client.28103 shutdown 2024-09-10T09:51:34.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:34.413+0000 7f35c0922740 1 -- 172.21.15.5:0/4005156624 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56252bf0b590 msgr2=0x56252bf108e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:51:34.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:34.413+0000 7f35c0922740 1 --2- 172.21.15.5:0/4005156624 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56252bf0b590 0x56252bf108e0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f35b0004af0 tx=0x7f35b00bb580 comp rx=0 tx=0).stop 2024-09-10T09:51:34.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:34.413+0000 7f35c0922740 1 -- 172.21.15.5:0/4005156624 shutdown_connections 2024-09-10T09:51:34.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:34.413+0000 7f35c0922740 1 --2- 172.21.15.5:0/4005156624 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x56252bf870e0 0x56252bf894d0 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:34.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:34.413+0000 7f35c0922740 1 --2- 172.21.15.5:0/4005156624 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56252bf0b590 0x56252bf108e0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:34.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:34.413+0000 7f35c0922740 1 --2- 172.21.15.5:0/4005156624 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56252bf00cd0 0x56252bf103a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:34.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:34.413+0000 7f35c0922740 1 --2- 172.21.15.5:0/4005156624 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56252bf003b0 0x56252bf13ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:34.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:34.413+0000 7f35c0922740 1 -- 172.21.15.5:0/4005156624 >> 172.21.15.5:0/4005156624 conn(0x56252befc940 msgr2=0x56252bf16a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:51:34.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:34.413+0000 7f35a0c00640 0 client.28103 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T09:51:34.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:34.413+0000 7f35c0922740 1 -- 172.21.15.5:0/4005156624 shutdown_connections 2024-09-10T09:51:34.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:34.413+0000 7f35c0922740 1 -- 172.21.15.5:0/4005156624 wait complete. 2024-09-10T09:51:34.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:34.413+0000 7f35c0922740 20 client.28103 trim_cache size 0 max 16384 2024-09-10T09:51:34.416 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-10T09:51:34.416 INFO:teuthology.orchestra.run.smithi005.stdout: "epoch": 36, 2024-09-10T09:51:34.416 INFO:teuthology.orchestra.run.smithi005.stdout: "leader": 24542, 2024-09-10T09:51:34.416 INFO:teuthology.orchestra.run.smithi005.stdout: "set_version": 46, 2024-09-10T09:51:34.416 INFO:teuthology.orchestra.run.smithi005.stdout: "sets": { 2024-09-10T09:51:34.416 INFO:teuthology.orchestra.run.smithi005.stdout: "9df515f": { 2024-09-10T09:51:34.416 INFO:teuthology.orchestra.run.smithi005.stdout: "version": 46, 2024-09-10T09:51:34.417 INFO:teuthology.orchestra.run.smithi005.stdout: "age_ref": 0.0, 2024-09-10T09:51:34.417 INFO:teuthology.orchestra.run.smithi005.stdout: "state": { 2024-09-10T09:51:34.417 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "QUIESCED", 2024-09-10T09:51:34.417 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0 2024-09-10T09:51:34.417 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T09:51:34.417 INFO:teuthology.orchestra.run.smithi005.stdout: "timeout": 90.0, 2024-09-10T09:51:34.417 INFO:teuthology.orchestra.run.smithi005.stdout: "expiration": 141.5, 2024-09-10T09:51:34.417 INFO:teuthology.orchestra.run.smithi005.stdout: "members": { 2024-09-10T09:51:34.417 INFO:teuthology.orchestra.run.smithi005.stdout: "file:/": { 2024-09-10T09:51:34.417 INFO:teuthology.orchestra.run.smithi005.stdout: "excluded": false, 2024-09-10T09:51:34.417 INFO:teuthology.orchestra.run.smithi005.stdout: "state": { 2024-09-10T09:51:34.417 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "QUIESCED", 2024-09-10T09:51:34.417 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0 2024-09-10T09:51:34.417 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T09:51:34.418 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T09:51:34.418 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T09:51:34.418 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T09:51:34.418 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T09:51:34.418 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-10T09:51:34.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:34.414+0000 7f35be200640 1 -- 172.21.15.5:0/1672017349 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f359c0a2c00 msgr2=0x7f359c0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:51:34.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:34.414+0000 7f35be200640 1 --2- 172.21.15.5:0/1672017349 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f359c0a2c00 0x7f359c0a50c0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f35a4009fd0 tx=0x7f35a4012040 comp rx=0 tx=0).stop 2024-09-10T09:51:34.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:34.414+0000 7f35be200640 1 -- 172.21.15.5:0/1672017349 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f35b810ac90 msgr2=0x7f35b819eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:51:34.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:34.414+0000 7f35be200640 1 --2- 172.21.15.5:0/1672017349 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f35b810ac90 0x7f35b819eed0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f35b0034040 tx=0x7f35b00305d0 comp rx=0 tx=0).stop 2024-09-10T09:51:34.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:34.414+0000 7f35be200640 1 -- 172.21.15.5:0/1672017349 shutdown_connections 2024-09-10T09:51:34.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:34.414+0000 7f35be200640 1 --2- 172.21.15.5:0/1672017349 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f359c0a2c00 0x7f359c0a50c0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:34.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:34.414+0000 7f35be200640 1 --2- 172.21.15.5:0/1672017349 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f35b810ac90 0x7f35b819eed0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:34.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:34.414+0000 7f35be200640 1 --2- 172.21.15.5:0/1672017349 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f35b8100910 0x7f35b8109ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:34.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:34.414+0000 7f35be200640 1 --2- 172.21.15.5:0/1672017349 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f35b8069290 0x7f35b8109ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:34.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:34.414+0000 7f35be200640 1 -- 172.21.15.5:0/1672017349 >> 172.21.15.5:0/1672017349 conn(0x7f35b80fba90 msgr2=0x7f35b810b300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:51:34.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:34.414+0000 7f35be200640 1 -- 172.21.15.5:0/1672017349 shutdown_connections 2024-09-10T09:51:34.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:34.415+0000 7f35be200640 1 -- 172.21.15.5:0/1672017349 wait complete. 2024-09-10T09:51:35.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:51:34 smithi005 ceph-mon[30350]: pgmap v790: 129 pgs: 129 active+clean; 315 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 1.6 MiB/s wr, 192 op/s 2024-09-10T09:51:35.149 INFO:tasks.workunit.client.0.smithi005.stdout: 1 859732 6.50 MB/sec execute 556 sec latency 726.868 ms 2024-09-10T09:51:35.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:51:34 smithi159 ceph-mon[35513]: pgmap v790: 129 pgs: 129 active+clean; 315 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 1.6 MiB/s wr, 192 op/s 2024-09-10T09:51:35.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:51:34 smithi102 ceph-mon[35442]: pgmap v790: 129 pgs: 129 active+clean; 315 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 1.6 MiB/s wr, 192 op/s 2024-09-10T09:51:35.390 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '9df515f', quiesce took 7.1 seconds. Will release after: 14.4 2024-09-10T09:51:36.149 INFO:tasks.workunit.client.0.smithi005.stdout: 1 859732 6.49 MB/sec execute 557 sec latency 1726.920 ms 2024-09-10T09:51:37.149 INFO:tasks.workunit.client.0.smithi005.stdout: 1 859732 6.48 MB/sec execute 558 sec latency 2726.986 ms 2024-09-10T09:51:37.306 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:51:36 smithi005 ceph-mon[30350]: pgmap v791: 129 pgs: 129 active+clean; 315 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 1.7 MiB/s wr, 211 op/s 2024-09-10T09:51:37.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:51:36 smithi159 ceph-mon[35513]: pgmap v791: 129 pgs: 129 active+clean; 315 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 1.7 MiB/s wr, 211 op/s 2024-09-10T09:51:37.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:51:36 smithi102 ceph-mon[35442]: pgmap v791: 129 pgs: 129 active+clean; 315 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 1.7 MiB/s wr, 211 op/s 2024-09-10T09:51:38.150 INFO:tasks.workunit.client.0.smithi005.stdout: 1 859732 6.47 MB/sec execute 559 sec latency 3727.027 ms 2024-09-10T09:51:39.150 INFO:tasks.workunit.client.0.smithi005.stdout: 1 859732 6.46 MB/sec execute 560 sec latency 4727.153 ms 2024-09-10T09:51:39.306 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:51:38 smithi005 ceph-mon[30350]: pgmap v792: 129 pgs: 129 active+clean; 318 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 1.9 MiB/s wr, 196 op/s 2024-09-10T09:51:39.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:51:38 smithi159 ceph-mon[35513]: pgmap v792: 129 pgs: 129 active+clean; 318 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 1.9 MiB/s wr, 196 op/s 2024-09-10T09:51:39.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:51:38 smithi102 ceph-mon[35442]: pgmap v792: 129 pgs: 129 active+clean; 318 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 1.9 MiB/s wr, 196 op/s 2024-09-10T09:51:40.150 INFO:tasks.workunit.client.0.smithi005.stdout: 1 859732 6.44 MB/sec execute 561 sec latency 5727.247 ms 2024-09-10T09:51:41.150 INFO:tasks.workunit.client.0.smithi005.stdout: 1 859732 6.43 MB/sec execute 562 sec latency 6727.305 ms 2024-09-10T09:51:41.207 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:51:40 smithi005 ceph-mon[30350]: pgmap v793: 129 pgs: 129 active+clean; 318 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.1 MiB/s wr, 198 op/s 2024-09-10T09:51:41.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:51:40 smithi159 ceph-mon[35513]: pgmap v793: 129 pgs: 129 active+clean; 318 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.1 MiB/s wr, 198 op/s 2024-09-10T09:51:41.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:51:40 smithi102 ceph-mon[35442]: pgmap v793: 129 pgs: 129 active+clean; 318 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.1 MiB/s wr, 198 op/s 2024-09-10T09:51:42.150 INFO:tasks.workunit.client.0.smithi005.stdout: 1 859732 6.42 MB/sec execute 563 sec latency 7727.346 ms 2024-09-10T09:51:43.150 INFO:tasks.workunit.client.0.smithi005.stdout: 1 859732 6.41 MB/sec execute 564 sec latency 8727.395 ms 2024-09-10T09:51:43.306 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:51:43 smithi005 ceph-mon[30350]: pgmap v794: 129 pgs: 129 active+clean; 318 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.8 MiB/s wr, 128 op/s 2024-09-10T09:51:43.307 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:51:43 smithi005 ceph-mon[30350]: scrub summary: idle 2024-09-10T09:51:43.307 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:51:43 smithi005 ceph-mon[30350]: scrub summary: idle 2024-09-10T09:51:43.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:51:43 smithi159 ceph-mon[35513]: pgmap v794: 129 pgs: 129 active+clean; 318 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.8 MiB/s wr, 128 op/s 2024-09-10T09:51:43.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:51:43 smithi159 ceph-mon[35513]: scrub summary: idle 2024-09-10T09:51:43.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:51:43 smithi159 ceph-mon[35513]: scrub summary: idle 2024-09-10T09:51:43.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:51:43 smithi102 ceph-mon[35442]: pgmap v794: 129 pgs: 129 active+clean; 318 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.8 MiB/s wr, 128 op/s 2024-09-10T09:51:43.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:51:43 smithi102 ceph-mon[35442]: scrub summary: idle 2024-09-10T09:51:43.385 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:51:43 smithi102 ceph-mon[35442]: scrub summary: idle 2024-09-10T09:51:44.150 INFO:tasks.workunit.client.0.smithi005.stdout: 1 859732 6.40 MB/sec execute 565 sec latency 9727.525 ms 2024-09-10T09:51:45.153 INFO:tasks.workunit.client.0.smithi005.stdout: 1 859732 6.39 MB/sec execute 566 sec latency 10727.620 ms 2024-09-10T09:51:45.261 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:51:45 smithi005 ceph-mon[30350]: pgmap v795: 129 pgs: 129 active+clean; 316 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.8 MiB/s wr, 127 op/s 2024-09-10T09:51:45.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:51:45 smithi159 ceph-mon[35513]: pgmap v795: 129 pgs: 129 active+clean; 316 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.8 MiB/s wr, 127 op/s 2024-09-10T09:51:45.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:51:45 smithi102 ceph-mon[35442]: pgmap v795: 129 pgs: 129 active+clean; 316 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.8 MiB/s wr, 127 op/s 2024-09-10T09:51:46.157 INFO:tasks.workunit.client.0.smithi005.stdout: 1 859732 6.38 MB/sec execute 567 sec latency 11734.871 ms 2024-09-10T09:51:47.078 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:51:47 smithi005 ceph-mon[30350]: pgmap v796: 129 pgs: 129 active+clean; 316 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 763 KiB/s rd, 1.5 MiB/s wr, 90 op/s 2024-09-10T09:51:47.157 INFO:tasks.workunit.client.0.smithi005.stdout: 1 859732 6.37 MB/sec execute 568 sec latency 12734.995 ms 2024-09-10T09:51:47.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:51:47 smithi159 ceph-mon[35513]: pgmap v796: 129 pgs: 129 active+clean; 316 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 763 KiB/s rd, 1.5 MiB/s wr, 90 op/s 2024-09-10T09:51:47.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:51:47 smithi102 ceph-mon[35442]: pgmap v796: 129 pgs: 129 active+clean; 316 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 763 KiB/s rd, 1.5 MiB/s wr, 90 op/s 2024-09-10T09:51:48.158 INFO:tasks.workunit.client.0.smithi005.stdout: 1 859732 6.35 MB/sec execute 569 sec latency 13735.109 ms 2024-09-10T09:51:49.158 INFO:tasks.workunit.client.0.smithi005.stdout: 1 859732 6.34 MB/sec execute 570 sec latency 14735.188 ms 2024-09-10T09:51:49.306 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:51:49 smithi005 ceph-mon[30350]: pgmap v797: 129 pgs: 129 active+clean; 312 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 448 KiB/s rd, 1.4 MiB/s wr, 72 op/s 2024-09-10T09:51:49.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:51:49 smithi159 ceph-mon[35513]: pgmap v797: 129 pgs: 129 active+clean; 312 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 448 KiB/s rd, 1.4 MiB/s wr, 72 op/s 2024-09-10T09:51:49.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:51:49 smithi102 ceph-mon[35442]: pgmap v797: 129 pgs: 129 active+clean; 312 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 448 KiB/s rd, 1.4 MiB/s wr, 72 op/s 2024-09-10T09:51:49.792 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T09:51:50.158 INFO:tasks.workunit.client.0.smithi005.stdout: 1 859732 6.33 MB/sec execute 571 sec latency 15735.247 ms 2024-09-10T09:51:50.186 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:51:51.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:51.101+0000 7f994ea00640 1 -- 172.21.15.5:0/2378027800 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f99481079c0 msgr2=0x7f9948107da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:51:51.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:51.101+0000 7f994ea00640 1 --2- 172.21.15.5:0/2378027800 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f99481079c0 0x7f9948107da0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f993c009930 tx=0x7f993c02a420 comp rx=0 tx=0).stop 2024-09-10T09:51:51.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:51.101+0000 7f994ea00640 1 -- 172.21.15.5:0/2378027800 shutdown_connections 2024-09-10T09:51:51.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:51.101+0000 7f994ea00640 1 --2- 172.21.15.5:0/2378027800 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9948102360 0x7f994810e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:51.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:51.101+0000 7f994ea00640 1 --2- 172.21.15.5:0/2378027800 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f99481019c0 0x7f9948101e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:51.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:51.101+0000 7f994ea00640 1 --2- 172.21.15.5:0/2378027800 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f99481079c0 0x7f9948107da0 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:51.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:51.101+0000 7f994ea00640 1 -- 172.21.15.5:0/2378027800 >> 172.21.15.5:0/2378027800 conn(0x7f99480fd740 msgr2=0x7f99480ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:51:51.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:51.101+0000 7f994ea00640 1 -- 172.21.15.5:0/2378027800 shutdown_connections 2024-09-10T09:51:51.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:51.101+0000 7f994ea00640 1 -- 172.21.15.5:0/2378027800 wait complete. 2024-09-10T09:51:51.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:51.103+0000 7f994ea00640 1 Processor -- start 2024-09-10T09:51:51.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:51.103+0000 7f994ea00640 1 -- start start 2024-09-10T09:51:51.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:51.103+0000 7f994ea00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f99481019c0 0x7f99481982c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:51:51.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:51.103+0000 7f994ea00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9948102360 0x7f9948198800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:51:51.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:51.103+0000 7f994ea00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f99481079c0 0x7f994819cb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:51:51.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:51.103+0000 7f994ea00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f9948111dc0 con 0x7f99481019c0 2024-09-10T09:51:51.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:51.103+0000 7f994ea00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f9948111c40 con 0x7f99481079c0 2024-09-10T09:51:51.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:51.103+0000 7f994ea00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f9948111f40 con 0x7f9948102360 2024-09-10T09:51:51.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:51.104+0000 7f994d600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f99481019c0 0x7f99481982c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:51:51.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:51.104+0000 7f994d600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f99481019c0 0x7f99481982c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60208/0 (socket says 172.21.15.5:60208) 2024-09-10T09:51:51.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:51.104+0000 7f994d600640 1 -- 172.21.15.5:0/1971520740 learned_addr learned my addr 172.21.15.5:0/1971520740 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:51:51.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:51.104+0000 7f994e000640 1 --2- 172.21.15.5:0/1971520740 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f99481079c0 0x7f994819cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:51:51.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:51.104+0000 7f994cc00640 1 --2- 172.21.15.5:0/1971520740 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9948102360 0x7f9948198800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:51:51.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:51.104+0000 7f994d600640 1 -- 172.21.15.5:0/1971520740 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9948102360 msgr2=0x7f9948198800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:51:51.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:51.104+0000 7f994d600640 1 --2- 172.21.15.5:0/1971520740 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9948102360 0x7f9948198800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:51.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:51.104+0000 7f994d600640 1 -- 172.21.15.5:0/1971520740 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f99481079c0 msgr2=0x7f994819cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:51:51.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:51.104+0000 7f994d600640 1 --2- 172.21.15.5:0/1971520740 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f99481079c0 0x7f994819cb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:51.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:51.104+0000 7f994d600640 1 -- 172.21.15.5:0/1971520740 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f994819d270 con 0x7f99481019c0 2024-09-10T09:51:51.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:51.105+0000 7f994d600640 1 --2- 172.21.15.5:0/1971520740 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f99481019c0 0x7f99481982c0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f993c009230 tx=0x7f993c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:51:51.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:51.105+0000 7f9946000640 1 -- 172.21.15.5:0/1971520740 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f993c03f030 con 0x7f99481019c0 2024-09-10T09:51:51.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:51.105+0000 7f9946000640 1 -- 172.21.15.5:0/1971520740 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f993c003ad0 con 0x7f99481019c0 2024-09-10T09:51:51.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:51.105+0000 7f9946000640 1 -- 172.21.15.5:0/1971520740 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f993c035070 con 0x7f99481019c0 2024-09-10T09:51:51.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:51.105+0000 7f994ea00640 1 -- 172.21.15.5:0/1971520740 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f994819d500 con 0x7f99481019c0 2024-09-10T09:51:51.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:51.105+0000 7f994ea00640 1 -- 172.21.15.5:0/1971520740 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f994819d990 con 0x7f99481019c0 2024-09-10T09:51:51.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:51.106+0000 7f994ea00640 1 -- 172.21.15.5:0/1971520740 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9948101e20 con 0x7f99481019c0 2024-09-10T09:51:51.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:51.108+0000 7f9946000640 1 -- 172.21.15.5:0/1971520740 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f993c042030 con 0x7f99481019c0 2024-09-10T09:51:51.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:51.110+0000 7f9946000640 1 --2- 172.21.15.5:0/1971520740 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f99300a2c00 0x7f99300a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:51:51.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:51.110+0000 7f9946000640 1 -- 172.21.15.5:0/1971520740 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f993c0efd90 con 0x7f99481019c0 2024-09-10T09:51:51.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:51.111+0000 7f994cc00640 1 --2- 172.21.15.5:0/1971520740 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f99300a2c00 0x7f99300a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:51:51.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:51.111+0000 7f994cc00640 1 --2- 172.21.15.5:0/1971520740 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f99300a2c00 0x7f99300a50c0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f99340095f0 tx=0x7f9934005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:51:51.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:51.114+0000 7f9946000640 1 -- 172.21.15.5:0/1971520740 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f993c0bc250 con 0x7f99481019c0 2024-09-10T09:51:51.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:51:51 smithi005 ceph-mon[30350]: pgmap v798: 129 pgs: 129 active+clean; 291 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 1.7 MiB/s wr, 87 op/s 2024-09-10T09:51:51.158 INFO:tasks.workunit.client.0.smithi005.stdout: 1 859732 6.32 MB/sec execute 572 sec latency 16735.306 ms 2024-09-10T09:51:51.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:51:51 smithi159 ceph-mon[35513]: pgmap v798: 129 pgs: 129 active+clean; 291 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 1.7 MiB/s wr, 87 op/s 2024-09-10T09:51:51.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:51.360+0000 7f994ea00640 1 -- 172.21.15.5:0/1971520740 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f994802d7c0 con 0x7f99481019c0 2024-09-10T09:51:51.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:51.361+0000 7f9946000640 1 -- 172.21.15.5:0/1971520740 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) ==== 94+0+11969 (secure 0 0 0) 0x7f993c0c0550 con 0x7f99481019c0 2024-09-10T09:51:51.363 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-10T09:51:51.363 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":36,"btime":"2024-09-10T09:48:07:349075+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14454,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/2970594311","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2970594311},{"type":"v1","addr":"172.21.15.5:6835","nonce":2970594311}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14514,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6839/3616502174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":3616502174},{"type":"v1","addr":"172.21.15.5:6839","nonce":3616502174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":36,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T09:35:29.626287+0000","modified":"2024-09-10T09:48:06.970127+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24542,"mds_1":24535,"mds_2":24511,"mds_3":24506,"mds_4":24443},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14520":{"gid":14520,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/2073599842","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":2073599842},{"type":"v1","addr":"172.21.15.5:6841","nonce":2073599842}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"k","rank":4,"incarnation":26,"state":"up:active","state_seq":24,"addr":"172.21.15.102:6833/2457007427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":2457007427},{"type":"v1","addr":"172.21.15.102:6833","nonce":2457007427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/356445159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":356445159},{"type":"v1","addr":"172.21.15.159:6833","nonce":356445159}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6837/2894314507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":2894314507},{"type":"v1","addr":"172.21.15.5:6837","nonce":2894314507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.102:6835/1647549111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":1647549111},{"type":"v1","addr":"172.21.15.102:6835","nonce":1647549111}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6835/952657242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":952657242},{"type":"v1","addr":"172.21.15.159:6835","nonce":952657242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":19,"addr":"172.21.15.102:6837/2689776764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":2689776764},{"type":"v1","addr":"172.21.15.102:6837","nonce":2689776764}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.159:6837/620627639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":620627639},{"type":"v1","addr":"172.21.15.159:6837","nonce":620627639}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24535":{"gid":24535,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":18,"addr":"172.21.15.159:6839/2073739599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":2073739599},{"type":"v1","addr":"172.21.15.159:6839","nonce":2073739599}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24542":{"gid":24542,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.102:6839/335409749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6838","nonce":335409749},{"type":"v1","addr":"172.21.15.102:6839","nonce":335409749}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24542,"qdb_cluster":[24542,24535,24511,24506,24443]},"id":1}]} 2024-09-10T09:51:51.363 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 36 2024-09-10T09:51:51.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:51.364+0000 7f994ea00640 1 -- 172.21.15.5:0/1971520740 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f99300a2c00 msgr2=0x7f99300a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:51:51.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:51.364+0000 7f994ea00640 1 --2- 172.21.15.5:0/1971520740 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f99300a2c00 0x7f99300a50c0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f99340095f0 tx=0x7f9934005f90 comp rx=0 tx=0).stop 2024-09-10T09:51:51.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:51.364+0000 7f994ea00640 1 -- 172.21.15.5:0/1971520740 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f99481019c0 msgr2=0x7f99481982c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:51:51.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:51.364+0000 7f994ea00640 1 --2- 172.21.15.5:0/1971520740 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f99481019c0 0x7f99481982c0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f993c009230 tx=0x7f993c004270 comp rx=0 tx=0).stop 2024-09-10T09:51:51.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:51.364+0000 7f994ea00640 1 -- 172.21.15.5:0/1971520740 shutdown_connections 2024-09-10T09:51:51.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:51.364+0000 7f994ea00640 1 --2- 172.21.15.5:0/1971520740 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f99300a2c00 0x7f99300a50c0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:51.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:51.364+0000 7f994ea00640 1 --2- 172.21.15.5:0/1971520740 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f99481079c0 0x7f994819cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:51.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:51.364+0000 7f994ea00640 1 --2- 172.21.15.5:0/1971520740 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9948102360 0x7f9948198800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:51.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:51.364+0000 7f994ea00640 1 --2- 172.21.15.5:0/1971520740 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f99481019c0 0x7f99481982c0 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:51.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:51.364+0000 7f994ea00640 1 -- 172.21.15.5:0/1971520740 >> 172.21.15.5:0/1971520740 conn(0x7f99480fd740 msgr2=0x7f99480fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:51:51.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:51.364+0000 7f994ea00640 1 -- 172.21.15.5:0/1971520740 shutdown_connections 2024-09-10T09:51:51.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:51.364+0000 7f994ea00640 1 -- 172.21.15.5:0/1971520740 wait complete. 2024-09-10T09:51:51.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:51:51 smithi102 ceph-mon[35442]: pgmap v798: 129 pgs: 129 active+clean; 291 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 1.7 MiB/s wr, 87 op/s 2024-09-10T09:51:52.124 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:51:52 smithi005 ceph-mon[30350]: from='client.? 172.21.15.5:0/1971520740' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:51:52.158 INFO:tasks.workunit.client.0.smithi005.stdout: 1 859732 6.31 MB/sec execute 573 sec latency 17735.358 ms 2024-09-10T09:51:52.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:51:52 smithi159 ceph-mon[35513]: from='client.? 172.21.15.5:0/1971520740' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:51:52.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:51:52 smithi102 ceph-mon[35442]: from='client.? 172.21.15.5:0/1971520740' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:51:52.549 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24542 quiesce db --set-id 9df515f --release --await' 2024-09-10T09:51:52.550 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24542 quiesce db --set-id 9df515f --release --await 2024-09-10T09:51:52.935 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:51:53.158 INFO:tasks.workunit.client.0.smithi005.stdout: 1 859732 6.30 MB/sec execute 574 sec latency 18735.495 ms 2024-09-10T09:51:53.306 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:51:53 smithi005 ceph-mon[30350]: pgmap v799: 129 pgs: 129 active+clean; 291 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 1.4 MiB/s wr, 66 op/s 2024-09-10T09:51:53.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:51:53 smithi159 ceph-mon[35513]: pgmap v799: 129 pgs: 129 active+clean; 291 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 1.4 MiB/s wr, 66 op/s 2024-09-10T09:51:53.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:51:53 smithi102 ceph-mon[35442]: pgmap v799: 129 pgs: 129 active+clean; 291 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 1.4 MiB/s wr, 66 op/s 2024-09-10T09:51:53.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:53.714+0000 7fcdc2600640 1 -- 172.21.15.5:0/305009234 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcdbc1027a0 msgr2=0x7fcdbc1067f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:51:53.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:53.714+0000 7fcdc2600640 1 --2- 172.21.15.5:0/305009234 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcdbc1027a0 0x7fcdbc1067f0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fcda4009900 tx=0x7fcda402a3f0 comp rx=0 tx=0).stop 2024-09-10T09:51:53.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:53.714+0000 7fcdc2600640 1 -- 172.21.15.5:0/305009234 shutdown_connections 2024-09-10T09:51:53.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:53.714+0000 7fcdc2600640 1 --2- 172.21.15.5:0/305009234 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcdbc106f20 0x7fcdbc10eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:53.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:53.714+0000 7fcdc2600640 1 --2- 172.21.15.5:0/305009234 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcdbc1027a0 0x7fcdbc1067f0 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:53.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:53.714+0000 7fcdc2600640 1 --2- 172.21.15.5:0/305009234 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcdbc101df0 0x7fcdbc1021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:53.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:53.714+0000 7fcdc2600640 1 -- 172.21.15.5:0/305009234 >> 172.21.15.5:0/305009234 conn(0x7fcdbc0fd8b0 msgr2=0x7fcdbc0ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:51:53.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:53.714+0000 7fcdc2600640 1 -- 172.21.15.5:0/305009234 shutdown_connections 2024-09-10T09:51:53.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:53.714+0000 7fcdc2600640 1 -- 172.21.15.5:0/305009234 wait complete. 2024-09-10T09:51:53.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:53.715+0000 7fcdc2600640 1 Processor -- start 2024-09-10T09:51:53.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:53.715+0000 7fcdc2600640 1 -- start start 2024-09-10T09:51:53.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:53.716+0000 7fcdc2600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcdbc101df0 0x7fcdbc198460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:51:53.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:53.716+0000 7fcdc2600640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcdbc1027a0 0x7fcdbc1989a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:51:53.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:53.716+0000 7fcdc2600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcdbc106f20 0x7fcdbc19cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:51:53.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:53.716+0000 7fcdc2600640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fcdbc111f80 con 0x7fcdbc106f20 2024-09-10T09:51:53.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:53.716+0000 7fcdc2600640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fcdbc111e00 con 0x7fcdbc101df0 2024-09-10T09:51:53.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:53.716+0000 7fcdc2600640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fcdbc112100 con 0x7fcdbc1027a0 2024-09-10T09:51:53.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:53.716+0000 7fcdb3e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcdbc1027a0 0x7fcdbc1989a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:51:53.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:53.716+0000 7fcdb3e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcdbc1027a0 0x7fcdbc1989a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:48488/0 (socket says 172.21.15.5:48488) 2024-09-10T09:51:53.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:53.716+0000 7fcdb3e00640 1 -- 172.21.15.5:0/1660917467 learned_addr learned my addr 172.21.15.5:0/1660917467 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:51:53.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:53.716+0000 7fcdbbe00640 1 --2- 172.21.15.5:0/1660917467 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcdbc106f20 0x7fcdbc19cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:51:53.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:53.716+0000 7fcdbb400640 1 --2- 172.21.15.5:0/1660917467 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcdbc101df0 0x7fcdbc198460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:51:53.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:53.717+0000 7fcdbb400640 1 -- 172.21.15.5:0/1660917467 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcdbc1027a0 msgr2=0x7fcdbc1989a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:51:53.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:53.717+0000 7fcdbb400640 1 --2- 172.21.15.5:0/1660917467 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcdbc1027a0 0x7fcdbc1989a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:53.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:53.717+0000 7fcdbb400640 1 -- 172.21.15.5:0/1660917467 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcdbc106f20 msgr2=0x7fcdbc19cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:51:53.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:53.717+0000 7fcdbb400640 1 --2- 172.21.15.5:0/1660917467 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcdbc106f20 0x7fcdbc19cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:53.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:53.717+0000 7fcdbb400640 1 -- 172.21.15.5:0/1660917467 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcdbc19d410 con 0x7fcdbc101df0 2024-09-10T09:51:53.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:53.717+0000 7fcdbbe00640 1 --2- 172.21.15.5:0/1660917467 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcdbc106f20 0x7fcdbc19cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:51:53.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:53.717+0000 7fcdbb400640 1 --2- 172.21.15.5:0/1660917467 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcdbc101df0 0x7fcdbc198460 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fcdac00e9f0 tx=0x7fcdac00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:51:53.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:53.717+0000 7fcdb8c00640 1 -- 172.21.15.5:0/1660917467 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdac00ccc0 con 0x7fcdbc101df0 2024-09-10T09:51:53.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:53.717+0000 7fcdb8c00640 1 -- 172.21.15.5:0/1660917467 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcdac010910 con 0x7fcdbc101df0 2024-09-10T09:51:53.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:53.717+0000 7fcdb8c00640 1 -- 172.21.15.5:0/1660917467 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdac018960 con 0x7fcdbc101df0 2024-09-10T09:51:53.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:53.717+0000 7fcdc2600640 1 -- 172.21.15.5:0/1660917467 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcdbc19d700 con 0x7fcdbc101df0 2024-09-10T09:51:53.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:53.717+0000 7fcdc2600640 1 -- 172.21.15.5:0/1660917467 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcdbc1a4fe0 con 0x7fcdbc101df0 2024-09-10T09:51:53.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:53.720+0000 7fcdc51bd740 1 Processor -- start 2024-09-10T09:51:53.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:53.721+0000 7fcdb8c00640 1 -- 172.21.15.5:0/1660917467 <== mon.2 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7fcdac010ac0 con 0x7fcdbc101df0 2024-09-10T09:51:53.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:53.721+0000 7fcdc51bd740 1 -- start start 2024-09-10T09:51:53.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:53.721+0000 7fcdc51bd740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562d6fbc7210 0x562d6fbc75f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:51:53.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:53.721+0000 7fcdc51bd740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562d6fbc7b30 0x562d6fbd63c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:51:53.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:53.721+0000 7fcdc51bd740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562d6fbd6900 0x562d6fbd8cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:51:53.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:53.721+0000 7fcdc51bd740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x562d6fb2d650 con 0x562d6fbc7b30 2024-09-10T09:51:53.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:53.721+0000 7fcdc51bd740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x562d6fad4f10 con 0x562d6fbc7210 2024-09-10T09:51:53.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:53.721+0000 7fcdc51bd740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x562d6fb29610 con 0x562d6fbd6900 2024-09-10T09:51:53.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:53.721+0000 7fcdbb400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562d6fbc7b30 0x562d6fbd63c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:51:53.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:53.721+0000 7fcdb3e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562d6fbd6900 0x562d6fbd8cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:51:53.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:53.722+0000 7fcdbb400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562d6fbc7b30 0x562d6fbd63c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60226/0 (socket says 172.21.15.5:60226) 2024-09-10T09:51:53.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:53.722+0000 7fcdbb400640 1 -- 172.21.15.5:0/2032609469 learned_addr learned my addr 172.21.15.5:0/2032609469 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:51:53.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:53.722+0000 7fcdbbe00640 1 --2- 172.21.15.5:0/2032609469 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562d6fbc7210 0x562d6fbc75f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:51:53.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:53.722+0000 7fcdb3e00640 1 -- 172.21.15.5:0/2032609469 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562d6fbc7210 msgr2=0x562d6fbc75f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:51:53.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:53.722+0000 7fcdb3e00640 1 --2- 172.21.15.5:0/2032609469 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562d6fbc7210 0x562d6fbc75f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:53.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:53.722+0000 7fcdb3e00640 1 -- 172.21.15.5:0/2032609469 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562d6fbc7b30 msgr2=0x562d6fbd63c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:51:53.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:53.722+0000 7fcdb3e00640 1 --2- 172.21.15.5:0/2032609469 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562d6fbc7b30 0x562d6fbd63c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:53.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:53.722+0000 7fcdb3e00640 1 -- 172.21.15.5:0/2032609469 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562d6f83c580 con 0x562d6fbd6900 2024-09-10T09:51:53.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:53.722+0000 7fcdbbe00640 1 --2- 172.21.15.5:0/2032609469 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562d6fbc7210 0x562d6fbc75f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:51:53.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:53.722+0000 7fcdb3e00640 1 --2- 172.21.15.5:0/2032609469 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562d6fbd6900 0x562d6fbd8cf0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fcda4009230 tx=0x7fcda4005dd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8dcbf61d5be89a05 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:51:53.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:53.722+0000 7fcdb8c00640 1 --2- 172.21.15.5:0/1660917467 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fcda00a2c00 0x7fcda00a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:51:53.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:53.723+0000 7fcdb8c00640 1 -- 172.21.15.5:0/1660917467 <== mon.2 v2:172.21.15.102:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fcdac0cf7a0 con 0x7fcdbc101df0 2024-09-10T09:51:53.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:53.723+0000 7fcdb0c00640 1 -- 172.21.15.5:0/2032609469 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcda4040070 con 0x562d6fbd6900 2024-09-10T09:51:53.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:53.723+0000 7fcdbbe00640 1 --2- 172.21.15.5:0/1660917467 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fcda00a2c00 0x7fcda00a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:51:53.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:53.723+0000 7fcdb0c00640 1 -- 172.21.15.5:0/2032609469 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcda4004030 con 0x562d6fbd6900 2024-09-10T09:51:53.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:53.723+0000 7fcdb0c00640 1 -- 172.21.15.5:0/2032609469 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcda4035040 con 0x562d6fbd6900 2024-09-10T09:51:53.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:53.724+0000 7fcdc51bd740 1 -- 172.21.15.5:0/2032609469 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562d6fbd6900 msgr2=0x562d6fbd8cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:51:53.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:53.724+0000 7fcdbbe00640 1 --2- 172.21.15.5:0/1660917467 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fcda00a2c00 0x7fcda00a50c0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fcdb4009fd0 tx=0x7fcdb4009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:51:53.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:53.724+0000 7fcdc51bd740 1 --2- 172.21.15.5:0/2032609469 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562d6fbd6900 0x562d6fbd8cf0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fcda4009230 tx=0x7fcda4005dd0 comp rx=0 tx=0).stop 2024-09-10T09:51:53.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:53.724+0000 7fcdc51bd740 1 -- 172.21.15.5:0/2032609469 shutdown_connections 2024-09-10T09:51:53.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:53.724+0000 7fcdc51bd740 1 --2- 172.21.15.5:0/2032609469 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562d6fbd6900 0x562d6fbd8cf0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:53.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:53.724+0000 7fcdc51bd740 1 --2- 172.21.15.5:0/2032609469 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562d6fbc7b30 0x562d6fbd63c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:53.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:53.724+0000 7fcdc51bd740 1 --2- 172.21.15.5:0/2032609469 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562d6fbc7210 0x562d6fbc75f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:53.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:53.724+0000 7fcdc51bd740 1 -- 172.21.15.5:0/2032609469 >> 172.21.15.5:0/2032609469 conn(0x562d6fbc38a0 msgr2=0x562d6fbc3c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:51:53.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:53.724+0000 7fcdc51bd740 1 -- 172.21.15.5:0/2032609469 shutdown_connections 2024-09-10T09:51:53.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:53.724+0000 7fcdc51bd740 1 -- 172.21.15.5:0/2032609469 wait complete. 2024-09-10T09:51:53.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:53.724+0000 7fcdc51bd740 1 -- start start 2024-09-10T09:51:53.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:53.725+0000 7fcdc51bd740 1 Processor -- start 2024-09-10T09:51:53.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:53.725+0000 7fcdc51bd740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562d6fbc7210 0x562d6fbd38a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:51:53.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:53.725+0000 7fcdc51bd740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562d6fbc7b30 0x562d6fbd3de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:51:53.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:53.725+0000 7fcdc51bd740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562d6fbd6900 0x562d6fbd4320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:51:53.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:53.725+0000 7fcdc51bd740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x562d6fbd94d0 con 0x562d6fbd6900 2024-09-10T09:51:53.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:53.725+0000 7fcdc51bd740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x562d6fae8350 con 0x562d6fbc7210 2024-09-10T09:51:53.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:53.725+0000 7fcdc51bd740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x562d6fbd9650 con 0x562d6fbc7b30 2024-09-10T09:51:53.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:53.725+0000 7fcdc51bd740 10 client.0 _ll_register_callbacks cb 0x562d6fb0b290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:51:53.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:53.725+0000 7fcdbbe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562d6fbd6900 0x562d6fbd4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:51:53.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:53.725+0000 7fcdb3e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562d6fbc7b30 0x562d6fbd3de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:51:53.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:53.725+0000 7fcdbb400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562d6fbc7210 0x562d6fbd38a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:51:53.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:53.725+0000 7fcdb3e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562d6fbc7b30 0x562d6fbd3de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:48504/0 (socket says 172.21.15.5:48504) 2024-09-10T09:51:53.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:53.725+0000 7fcdb3e00640 1 -- 172.21.15.5:0/877809408 learned_addr learned my addr 172.21.15.5:0/877809408 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:51:53.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:53.726+0000 7fcdbb400640 1 -- 172.21.15.5:0/877809408 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562d6fbc7b30 msgr2=0x562d6fbd3de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:51:53.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:53.726+0000 7fcdbb400640 1 --2- 172.21.15.5:0/877809408 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562d6fbc7b30 0x562d6fbd3de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:53.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:53.726+0000 7fcdbb400640 1 -- 172.21.15.5:0/877809408 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562d6fbd6900 msgr2=0x562d6fbd4320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:51:53.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:53.726+0000 7fcdbb400640 1 --2- 172.21.15.5:0/877809408 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562d6fbd6900 0x562d6fbd4320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:53.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:53.726+0000 7fcdbb400640 1 -- 172.21.15.5:0/877809408 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562d6f906960 con 0x562d6fbc7210 2024-09-10T09:51:53.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:53.726+0000 7fcdbb400640 1 --2- 172.21.15.5:0/877809408 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562d6fbc7210 0x562d6fbd38a0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fcdac098630 tx=0x7fcdac097fc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c493d0247640322e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:51:53.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:53.726+0000 7fcda9600640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-09-10T09:51:53.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:53.726+0000 7fcda9600640 1 -- 172.21.15.5:0/877809408 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdac09b5c0 con 0x562d6fbc7210 2024-09-10T09:51:53.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:53.726+0000 7fcdc51bd740 1 -- 172.21.15.5:0/877809408 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcda4004030 con 0x562d6fbc7210 2024-09-10T09:51:53.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:53.726+0000 7fcda9600640 1 -- 172.21.15.5:0/877809408 <== mon.2 v2:172.21.15.102:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fcdac0cfb90 con 0x562d6fbc7210 2024-09-10T09:51:53.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:53.726+0000 7fcda9600640 1 -- 172.21.15.5:0/877809408 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fcdac09bc60 con 0x562d6fbc7210 2024-09-10T09:51:53.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:53.726+0000 7fcdc51bd740 10 client.28121 fetch_fsmap learned FSMap version 36 2024-09-10T09:51:53.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:53.727+0000 7fcdc51bd740 1 -- 172.21.15.5:0/877809408 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x562d6fbe5210 con 0x562d6fbc7210 2024-09-10T09:51:53.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:53.727+0000 7fcda9600640 1 -- 172.21.15.5:0/877809408 <== mon.2 v2:172.21.15.102:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcdac0b3070 con 0x562d6fbc7210 2024-09-10T09:51:53.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:53.727+0000 7fcda9600640 1 -- 172.21.15.5:0/877809408 <== mon.2 v2:172.21.15.102:3300/0 5 ==== fsmap(e 36) ==== 8370+0+0 (secure 0 0 0) 0x7fcdac09b140 con 0x562d6fbc7210 2024-09-10T09:51:53.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:53.727+0000 7fcdc51bd740 10 client.28121 fetch_fsmap finished waiting for FSMap version 36 2024-09-10T09:51:53.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:53.727+0000 7fcdc51bd740 10 client.28121 resolve_mds: validated gid 24542 aka daemon mds.b 2024-09-10T09:51:53.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:53.727+0000 7fcdc51bd740 20 client.28121 populate_metadata read hostname 'smithi005' 2024-09-10T09:51:53.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:53.727+0000 7fcdc51bd740 1 --2- 172.21.15.5:0/877809408 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x562d6fbe6620 0x562d6fbe8a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:51:53.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:53.727+0000 7fcdc51bd740 4 client.28121 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:51:53.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:53.727+0000 7fcdc51bd740 1 -- 172.21.15.5:0/877809408 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562d6fb3abc0 con 0x562d6fbe6620 2024-09-10T09:51:53.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:53.727+0000 7fcdb3e00640 1 --2- 172.21.15.5:0/877809408 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x562d6fbe6620 0x562d6fbe8a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:51:53.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:53.728+0000 7fcdb3e00640 1 --2- 172.21.15.5:0/877809408 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x562d6fbe6620 0x562d6fbe8a50 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bdcd210f55394cdc server_cookie=2e27fa42cc9f5303 in_seq=0 out_seq=0 2024-09-10T09:51:53.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:53.728+0000 7fcda9600640 10 client.28121 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T09:51:53.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:53.728+0000 7fcda9600640 1 -- 172.21.15.5:0/877809408 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562d6fb3abc0 con 0x562d6fbe6620 2024-09-10T09:51:53.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:53.729+0000 7fcda9600640 10 client.28121 handle_command_reply: tid=0 2024-09-10T09:51:53.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:53.729+0000 7fcdc51bd740 1 client.28121 shutdown 2024-09-10T09:51:53.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:53.729+0000 7fcdc51bd740 1 -- 172.21.15.5:0/877809408 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562d6fbc7210 msgr2=0x562d6fbd38a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:51:53.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:53.729+0000 7fcdc51bd740 1 --2- 172.21.15.5:0/877809408 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562d6fbc7210 0x562d6fbd38a0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fcdac098630 tx=0x7fcdac097fc0 comp rx=0 tx=0).stop 2024-09-10T09:51:53.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:53.729+0000 7fcdc51bd740 1 -- 172.21.15.5:0/877809408 shutdown_connections 2024-09-10T09:51:53.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:53.729+0000 7fcdc51bd740 1 --2- 172.21.15.5:0/877809408 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x562d6fbe6620 0x562d6fbe8a50 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:53.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:53.729+0000 7fcdc51bd740 1 --2- 172.21.15.5:0/877809408 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562d6fbd6900 0x562d6fbd4320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:53.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:53.729+0000 7fcdc51bd740 1 --2- 172.21.15.5:0/877809408 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562d6fbc7b30 0x562d6fbd3de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:53.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:53.729+0000 7fcdc51bd740 1 --2- 172.21.15.5:0/877809408 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562d6fbc7210 0x562d6fbd38a0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:53.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:53.729+0000 7fcda9600640 0 client.28121 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T09:51:53.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:53.729+0000 7fcdc51bd740 1 -- 172.21.15.5:0/877809408 >> 172.21.15.5:0/877809408 conn(0x562d6fbc38a0 msgr2=0x562d6fbc9600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:51:53.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:53.729+0000 7fcdc51bd740 1 -- 172.21.15.5:0/877809408 shutdown_connections 2024-09-10T09:51:53.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:53.729+0000 7fcdc51bd740 1 -- 172.21.15.5:0/877809408 wait complete. 2024-09-10T09:51:53.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:53.729+0000 7fcdc51bd740 20 client.28121 trim_cache size 0 max 16384 2024-09-10T09:51:53.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:53.747+0000 7fcdc51bd740 1 Processor -- start 2024-09-10T09:51:53.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:53.747+0000 7fcdc51bd740 1 -- start start 2024-09-10T09:51:53.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:53.748+0000 7fcdc51bd740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562d6fbc7210 0x562d6fbee320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:51:53.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:53.748+0000 7fcdc51bd740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562d6fbc7b30 0x562d6fbee860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:51:53.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:53.748+0000 7fcdc51bd740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562d6fbd6900 0x562d6fbd0b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:51:53.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:53.748+0000 7fcdc51bd740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fcdbc1acc40 con 0x562d6fbd6900 2024-09-10T09:51:53.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:53.748+0000 7fcdc51bd740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fcdbc1ae780 con 0x562d6fbc7210 2024-09-10T09:51:53.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:53.748+0000 7fcdc51bd740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fcdbc1ad310 con 0x562d6fbc7b30 2024-09-10T09:51:53.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:53.748+0000 7fcdbbe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562d6fbd6900 0x562d6fbd0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:51:53.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:53.748+0000 7fcdbb400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562d6fbc7210 0x562d6fbee320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:51:53.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:53.748+0000 7fcdbbe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562d6fbd6900 0x562d6fbd0b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60238/0 (socket says 172.21.15.5:60238) 2024-09-10T09:51:53.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:53.748+0000 7fcdbbe00640 1 -- 172.21.15.5:0/2247257318 learned_addr learned my addr 172.21.15.5:0/2247257318 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:51:53.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:53.748+0000 7fcdbb400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562d6fbc7210 0x562d6fbee320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:39534/0 (socket says 172.21.15.5:39534) 2024-09-10T09:51:53.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:53.748+0000 7fcdb3e00640 1 --2- 172.21.15.5:0/2247257318 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562d6fbc7b30 0x562d6fbee860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:51:53.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:53.748+0000 7fcdbb400640 1 -- 172.21.15.5:0/2247257318 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562d6fbc7b30 msgr2=0x562d6fbee860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:51:53.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:53.748+0000 7fcdbb400640 1 --2- 172.21.15.5:0/2247257318 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562d6fbc7b30 0x562d6fbee860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:53.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:53.748+0000 7fcdbb400640 1 -- 172.21.15.5:0/2247257318 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562d6fbd6900 msgr2=0x562d6fbd0b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:51:53.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:53.748+0000 7fcdbb400640 1 --2- 172.21.15.5:0/2247257318 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562d6fbd6900 0x562d6fbd0b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:53.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:53.748+0000 7fcdbb400640 1 -- 172.21.15.5:0/2247257318 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562d6fbc3340 con 0x562d6fbc7210 2024-09-10T09:51:53.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:53.749+0000 7fcdbb400640 1 --2- 172.21.15.5:0/2247257318 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562d6fbc7210 0x562d6fbee320 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fcdac09b960 tx=0x7fcdac098030 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8c1d3e1b773ee51 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:51:53.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:53.749+0000 7fcdabe00640 1 -- 172.21.15.5:0/2247257318 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdac09ff60 con 0x562d6fbc7210 2024-09-10T09:51:53.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:53.749+0000 7fcdabe00640 1 -- 172.21.15.5:0/2247257318 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcdac0b3070 con 0x562d6fbc7210 2024-09-10T09:51:53.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:53.749+0000 7fcdabe00640 1 -- 172.21.15.5:0/2247257318 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdac09b330 con 0x562d6fbc7210 2024-09-10T09:51:53.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:53.749+0000 7fcdc51bd740 1 -- 172.21.15.5:0/2247257318 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562d6fbc7210 msgr2=0x562d6fbee320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:51:53.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:53.749+0000 7fcdc51bd740 1 --2- 172.21.15.5:0/2247257318 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562d6fbc7210 0x562d6fbee320 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fcdac09b960 tx=0x7fcdac098030 comp rx=0 tx=0).stop 2024-09-10T09:51:53.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:53.750+0000 7fcdc51bd740 1 -- 172.21.15.5:0/2247257318 shutdown_connections 2024-09-10T09:51:53.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:53.750+0000 7fcdc51bd740 1 --2- 172.21.15.5:0/2247257318 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562d6fbd6900 0x562d6fbd0b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:53.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:53.750+0000 7fcdc51bd740 1 --2- 172.21.15.5:0/2247257318 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562d6fbc7b30 0x562d6fbee860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:53.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:53.750+0000 7fcdc51bd740 1 --2- 172.21.15.5:0/2247257318 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562d6fbc7210 0x562d6fbee320 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:53.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:53.750+0000 7fcdc51bd740 1 -- 172.21.15.5:0/2247257318 >> 172.21.15.5:0/2247257318 conn(0x562d6fbc38a0 msgr2=0x562d6fbc95b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:51:53.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:53.750+0000 7fcdc51bd740 1 -- 172.21.15.5:0/2247257318 shutdown_connections 2024-09-10T09:51:53.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:53.750+0000 7fcdc51bd740 1 -- 172.21.15.5:0/2247257318 wait complete. 2024-09-10T09:51:53.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:53.750+0000 7fcdc51bd740 1 -- start start 2024-09-10T09:51:53.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:53.750+0000 7fcdc51bd740 1 Processor -- start 2024-09-10T09:51:53.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:53.751+0000 7fcdc51bd740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562d6fbc7210 0x562d6fbde8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:51:53.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:53.751+0000 7fcdc51bd740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562d6fbc7b30 0x562d6fbdee20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:51:53.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:53.751+0000 7fcdc51bd740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562d6fbd6900 0x562d6fbcb4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:51:53.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:53.751+0000 7fcdc51bd740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fcdbc1ae0b0 con 0x562d6fbc7b30 2024-09-10T09:51:53.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:53.751+0000 7fcdc51bd740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fcdbc1ad9e0 con 0x562d6fbc7210 2024-09-10T09:51:53.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:53.751+0000 7fcdc51bd740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fcdbc1aac20 con 0x562d6fbd6900 2024-09-10T09:51:53.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:53.751+0000 7fcdc51bd740 10 client.0 _ll_register_callbacks cb 0x7fcdac0d5220 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:51:53.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:53.751+0000 7fcdbbe00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562d6fbd6900 0x562d6fbcb4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:51:53.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:53.751+0000 7fcdbbe00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562d6fbd6900 0x562d6fbcb4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:48520/0 (socket says 172.21.15.5:48520) 2024-09-10T09:51:53.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:53.751+0000 7fcdb3e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562d6fbc7b30 0x562d6fbdee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:51:53.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:53.751+0000 7fcdbbe00640 1 -- 172.21.15.5:0/2767900936 learned_addr learned my addr 172.21.15.5:0/2767900936 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:51:53.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:53.751+0000 7fcdbb400640 1 --2- 172.21.15.5:0/2767900936 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562d6fbc7210 0x562d6fbde8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:51:53.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:53.751+0000 7fcdbbe00640 1 -- 172.21.15.5:0/2767900936 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562d6fbc7210 msgr2=0x562d6fbde8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:51:53.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:53.751+0000 7fcdbbe00640 1 --2- 172.21.15.5:0/2767900936 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562d6fbc7210 0x562d6fbde8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:53.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:53.751+0000 7fcdbbe00640 1 -- 172.21.15.5:0/2767900936 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562d6fbc7b30 msgr2=0x562d6fbdee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:51:53.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:53.751+0000 7fcdbbe00640 1 --2- 172.21.15.5:0/2767900936 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562d6fbc7b30 0x562d6fbdee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:53.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:53.752+0000 7fcdbbe00640 1 -- 172.21.15.5:0/2767900936 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562d6fbcba10 con 0x562d6fbd6900 2024-09-10T09:51:53.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:53.752+0000 7fcdbb400640 1 --2- 172.21.15.5:0/2767900936 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562d6fbc7210 0x562d6fbde8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:51:53.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:53.752+0000 7fcdbbe00640 1 --2- 172.21.15.5:0/2767900936 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562d6fbd6900 0x562d6fbcb4d0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fcdb4012270 tx=0x7fcdb4004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d54b186f1539485c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:51:53.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:53.752+0000 7fcda9600640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-09-10T09:51:53.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:53.752+0000 7fcda9600640 1 -- 172.21.15.5:0/2767900936 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdb402f060 con 0x562d6fbd6900 2024-09-10T09:51:53.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:53.752+0000 7fcda9600640 1 -- 172.21.15.5:0/2767900936 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fcdb4011b50 con 0x562d6fbd6900 2024-09-10T09:51:53.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:53.752+0000 7fcdc51bd740 1 -- 172.21.15.5:0/2767900936 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcdac0b3070 con 0x562d6fbd6900 2024-09-10T09:51:53.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:53.752+0000 7fcda9600640 1 -- 172.21.15.5:0/2767900936 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fcdb4036050 con 0x562d6fbd6900 2024-09-10T09:51:53.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:53.753+0000 7fcdc51bd740 10 client.28123 fetch_fsmap learned FSMap version 36 2024-09-10T09:51:53.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:53.753+0000 7fcdc51bd740 1 -- 172.21.15.5:0/2767900936 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x562d6fc56280 con 0x562d6fbd6900 2024-09-10T09:51:53.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:53.753+0000 7fcda9600640 1 -- 172.21.15.5:0/2767900936 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcdac0b3070 con 0x562d6fbd6900 2024-09-10T09:51:53.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:53.753+0000 7fcda9600640 1 -- 172.21.15.5:0/2767900936 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 36) ==== 8370+0+0 (secure 0 0 0) 0x7fcdb401e410 con 0x562d6fbd6900 2024-09-10T09:51:53.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:53.753+0000 7fcdc51bd740 10 client.28123 fetch_fsmap finished waiting for FSMap version 36 2024-09-10T09:51:53.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:53.753+0000 7fcdc51bd740 10 client.28123 resolve_mds: validated gid 24542 aka daemon mds.b 2024-09-10T09:51:53.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:53.753+0000 7fcdc51bd740 20 client.28123 populate_metadata read hostname 'smithi005' 2024-09-10T09:51:53.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:53.753+0000 7fcdc51bd740 1 --2- 172.21.15.5:0/2767900936 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x562d6fc56a00 0x562d6fc58e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:51:53.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:53.753+0000 7fcdc51bd740 4 client.28123 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "9df515f", "release": true, "await": true}] 2024-09-10T09:51:53.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:53.753+0000 7fcdc51bd740 1 -- 172.21.15.5:0/2767900936 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "quiesce db", "set_id": "9df515f", "release": true, "await": true}) -- 0x562d6fb350d0 con 0x562d6fc56a00 2024-09-10T09:51:53.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:53.754+0000 7fcdbb400640 1 --2- 172.21.15.5:0/2767900936 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x562d6fc56a00 0x562d6fc58e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:51:53.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:53.754+0000 7fcdbb400640 1 --2- 172.21.15.5:0/2767900936 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x562d6fc56a00 0x562d6fc58e20 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9712daf2517fc81b server_cookie=93cedcf194ddd577 in_seq=0 out_seq=0 2024-09-10T09:51:53.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:53.754+0000 7fcda9600640 10 client.28123 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T09:51:53.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:53.764+0000 7fcda9600640 1 -- 172.21.15.5:0/2767900936 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x562d6fb350d0 con 0x562d6fc56a00 2024-09-10T09:51:53.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:53.764+0000 7fcda9600640 10 client.28123 handle_command_reply: tid=0 2024-09-10T09:51:53.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:53.764+0000 7fcdc51bd740 1 client.28123 shutdown 2024-09-10T09:51:53.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:53.764+0000 7fcdc51bd740 1 -- 172.21.15.5:0/2767900936 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562d6fbd6900 msgr2=0x562d6fbcb4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:51:53.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:53.764+0000 7fcdc51bd740 1 --2- 172.21.15.5:0/2767900936 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562d6fbd6900 0x562d6fbcb4d0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fcdb4012270 tx=0x7fcdb4004a40 comp rx=0 tx=0).stop 2024-09-10T09:51:53.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:53.765+0000 7fcdc51bd740 1 -- 172.21.15.5:0/2767900936 shutdown_connections 2024-09-10T09:51:53.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:53.765+0000 7fcdc51bd740 1 --2- 172.21.15.5:0/2767900936 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x562d6fc56a00 0x562d6fc58e20 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:53.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:53.765+0000 7fcdc51bd740 1 --2- 172.21.15.5:0/2767900936 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562d6fbd6900 0x562d6fbcb4d0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:53.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:53.765+0000 7fcdc51bd740 1 --2- 172.21.15.5:0/2767900936 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562d6fbc7b30 0x562d6fbdee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:53.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:53.765+0000 7fcdc51bd740 1 --2- 172.21.15.5:0/2767900936 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562d6fbc7210 0x562d6fbde8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:53.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:53.765+0000 7fcda9600640 0 client.28123 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T09:51:53.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:53.765+0000 7fcdc51bd740 1 -- 172.21.15.5:0/2767900936 >> 172.21.15.5:0/2767900936 conn(0x562d6fbc38a0 msgr2=0x562d6fbd3860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:51:53.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:53.765+0000 7fcdc51bd740 1 -- 172.21.15.5:0/2767900936 shutdown_connections 2024-09-10T09:51:53.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:53.765+0000 7fcdc51bd740 1 -- 172.21.15.5:0/2767900936 wait complete. 2024-09-10T09:51:53.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:53.765+0000 7fcdc51bd740 20 client.28123 trim_cache size 0 max 16384 2024-09-10T09:51:53.767 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-10T09:51:53.767 INFO:teuthology.orchestra.run.smithi005.stdout: "epoch": 36, 2024-09-10T09:51:53.767 INFO:teuthology.orchestra.run.smithi005.stdout: "leader": 24542, 2024-09-10T09:51:53.767 INFO:teuthology.orchestra.run.smithi005.stdout: "set_version": 48, 2024-09-10T09:51:53.767 INFO:teuthology.orchestra.run.smithi005.stdout: "sets": { 2024-09-10T09:51:53.768 INFO:teuthology.orchestra.run.smithi005.stdout: "9df515f": { 2024-09-10T09:51:53.768 INFO:teuthology.orchestra.run.smithi005.stdout: "version": 48, 2024-09-10T09:51:53.768 INFO:teuthology.orchestra.run.smithi005.stdout: "age_ref": 0.0, 2024-09-10T09:51:53.768 INFO:teuthology.orchestra.run.smithi005.stdout: "state": { 2024-09-10T09:51:53.768 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "RELEASED", 2024-09-10T09:51:53.768 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0 2024-09-10T09:51:53.768 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T09:51:53.768 INFO:teuthology.orchestra.run.smithi005.stdout: "timeout": 90.0, 2024-09-10T09:51:53.768 INFO:teuthology.orchestra.run.smithi005.stdout: "expiration": 141.5, 2024-09-10T09:51:53.768 INFO:teuthology.orchestra.run.smithi005.stdout: "members": { 2024-09-10T09:51:53.768 INFO:teuthology.orchestra.run.smithi005.stdout: "file:/": { 2024-09-10T09:51:53.768 INFO:teuthology.orchestra.run.smithi005.stdout: "excluded": false, 2024-09-10T09:51:53.768 INFO:teuthology.orchestra.run.smithi005.stdout: "state": { 2024-09-10T09:51:53.769 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "RELEASED", 2024-09-10T09:51:53.769 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0 2024-09-10T09:51:53.769 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T09:51:53.769 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T09:51:53.769 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T09:51:53.769 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T09:51:53.769 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T09:51:53.769 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-10T09:51:53.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:53.766+0000 7fcdc2600640 1 -- 172.21.15.5:0/1660917467 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fcda00a2c00 msgr2=0x7fcda00a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:51:53.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:53.766+0000 7fcdc2600640 1 --2- 172.21.15.5:0/1660917467 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fcda00a2c00 0x7fcda00a50c0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fcdb4009fd0 tx=0x7fcdb4009500 comp rx=0 tx=0).stop 2024-09-10T09:51:53.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:53.766+0000 7fcdc2600640 1 -- 172.21.15.5:0/1660917467 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcdbc101df0 msgr2=0x7fcdbc198460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:51:53.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:53.766+0000 7fcdc2600640 1 --2- 172.21.15.5:0/1660917467 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcdbc101df0 0x7fcdbc198460 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fcdac00e9f0 tx=0x7fcdac00eeb0 comp rx=0 tx=0).stop 2024-09-10T09:51:53.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:53.766+0000 7fcdc2600640 1 -- 172.21.15.5:0/1660917467 shutdown_connections 2024-09-10T09:51:53.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:53.766+0000 7fcdc2600640 1 --2- 172.21.15.5:0/1660917467 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fcda00a2c00 0x7fcda00a50c0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:53.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:53.766+0000 7fcdc2600640 1 --2- 172.21.15.5:0/1660917467 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcdbc106f20 0x7fcdbc19cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:53.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:53.766+0000 7fcdc2600640 1 --2- 172.21.15.5:0/1660917467 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcdbc1027a0 0x7fcdbc1989a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:53.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:53.766+0000 7fcdc2600640 1 --2- 172.21.15.5:0/1660917467 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcdbc101df0 0x7fcdbc198460 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:53.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:53.766+0000 7fcdc2600640 1 -- 172.21.15.5:0/1660917467 >> 172.21.15.5:0/1660917467 conn(0x7fcdbc0fd8b0 msgr2=0x7fcdbc0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:51:53.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:53.766+0000 7fcdc2600640 1 -- 172.21.15.5:0/1660917467 shutdown_connections 2024-09-10T09:51:53.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:53.766+0000 7fcdc2600640 1 -- 172.21.15.5:0/1660917467 wait complete. 2024-09-10T09:51:54.158 INFO:tasks.workunit.client.0.smithi005.stdout: 1 860338 6.29 MB/sec execute 575 sec latency 19345.045 ms 2024-09-10T09:51:54.731 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '9df515f', total seconds elapsed: 26.4 2024-09-10T09:51:54.731 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 21.5 2024-09-10T09:51:54.731 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T09:51:55.137 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:51:55.158 INFO:tasks.workunit.client.0.smithi005.stdout: 1 863016 6.31 MB/sec execute 576 sec latency 21.666 ms 2024-09-10T09:51:55.306 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:51:55 smithi005 ceph-mon[30350]: pgmap v800: 129 pgs: 129 active+clean; 290 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 1.5 MiB/s wr, 74 op/s 2024-09-10T09:51:55.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:51:55 smithi159 ceph-mon[35513]: pgmap v800: 129 pgs: 129 active+clean; 290 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 1.5 MiB/s wr, 74 op/s 2024-09-10T09:51:55.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:51:55 smithi102 ceph-mon[35442]: pgmap v800: 129 pgs: 129 active+clean; 290 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 1.5 MiB/s wr, 74 op/s 2024-09-10T09:51:56.118 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:51:56 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:51:56.164 INFO:tasks.workunit.client.0.smithi005.stdout: 1 863320 6.30 MB/sec execute 577 sec latency 648.934 ms 2024-09-10T09:51:56.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:51:56 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:51:56.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:51:56 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:51:56.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:56.442+0000 7fb2e0a00640 1 -- 172.21.15.5:0/3967869304 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb2dc1047e0 msgr2=0x7fb2dc108830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:51:56.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:56.442+0000 7fb2e0a00640 1 --2- 172.21.15.5:0/3967869304 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb2dc1047e0 0x7fb2dc108830 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fb2c8009900 tx=0x7fb2c802a410 comp rx=0 tx=0).stop 2024-09-10T09:51:56.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:56.443+0000 7fb2e0a00640 1 -- 172.21.15.5:0/3967869304 shutdown_connections 2024-09-10T09:51:56.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:56.443+0000 7fb2e0a00640 1 --2- 172.21.15.5:0/3967869304 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb2dc108f60 0x7fb2dc110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:56.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:56.443+0000 7fb2e0a00640 1 --2- 172.21.15.5:0/3967869304 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb2dc1047e0 0x7fb2dc108830 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:56.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:56.443+0000 7fb2e0a00640 1 --2- 172.21.15.5:0/3967869304 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb2dc103e30 0x7fb2dc104210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:56.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:56.443+0000 7fb2e0a00640 1 -- 172.21.15.5:0/3967869304 >> 172.21.15.5:0/3967869304 conn(0x7fb2dc0ff8d0 msgr2=0x7fb2dc101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:51:56.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:56.443+0000 7fb2e0a00640 1 -- 172.21.15.5:0/3967869304 shutdown_connections 2024-09-10T09:51:56.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:56.443+0000 7fb2e0a00640 1 -- 172.21.15.5:0/3967869304 wait complete. 2024-09-10T09:51:56.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:56.444+0000 7fb2e0a00640 1 Processor -- start 2024-09-10T09:51:56.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:56.445+0000 7fb2e0a00640 1 -- start start 2024-09-10T09:51:56.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:56.445+0000 7fb2e0a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb2dc103e30 0x7fb2dc076700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:51:56.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:56.445+0000 7fb2e0a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb2dc1047e0 0x7fb2dc076c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:51:56.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:56.445+0000 7fb2e0a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb2dc108f60 0x7fb2dc077180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:51:56.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:56.445+0000 7fb2e0a00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fb2dc114070 con 0x7fb2dc108f60 2024-09-10T09:51:56.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:56.445+0000 7fb2e0a00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fb2dc113ef0 con 0x7fb2dc1047e0 2024-09-10T09:51:56.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:56.445+0000 7fb2e0a00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fb2dc1141f0 con 0x7fb2dc103e30 2024-09-10T09:51:56.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:56.445+0000 7fb2da800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb2dc103e30 0x7fb2dc076700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:51:56.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:56.445+0000 7fb2da800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb2dc103e30 0x7fb2dc076700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:48538/0 (socket says 172.21.15.5:48538) 2024-09-10T09:51:56.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:56.445+0000 7fb2da800640 1 -- 172.21.15.5:0/1165272754 learned_addr learned my addr 172.21.15.5:0/1165272754 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:51:56.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:56.446+0000 7fb2d9e00640 1 --2- 172.21.15.5:0/1165272754 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb2dc1047e0 0x7fb2dc076c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:51:56.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:56.446+0000 7fb2db200640 1 --2- 172.21.15.5:0/1165272754 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb2dc108f60 0x7fb2dc077180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:51:56.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:56.446+0000 7fb2da800640 1 -- 172.21.15.5:0/1165272754 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb2dc1047e0 msgr2=0x7fb2dc076c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:51:56.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:56.446+0000 7fb2da800640 1 --2- 172.21.15.5:0/1165272754 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb2dc1047e0 0x7fb2dc076c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:56.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:56.446+0000 7fb2da800640 1 -- 172.21.15.5:0/1165272754 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb2dc108f60 msgr2=0x7fb2dc077180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:51:56.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:56.446+0000 7fb2da800640 1 --2- 172.21.15.5:0/1165272754 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb2dc108f60 0x7fb2dc077180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:56.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:56.446+0000 7fb2da800640 1 -- 172.21.15.5:0/1165272754 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2dc1aabd0 con 0x7fb2dc103e30 2024-09-10T09:51:56.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:56.446+0000 7fb2db200640 1 --2- 172.21.15.5:0/1165272754 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb2dc108f60 0x7fb2dc077180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T09:51:56.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:56.446+0000 7fb2da800640 1 --2- 172.21.15.5:0/1165272754 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb2dc103e30 0x7fb2dc076700 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fb2cc00ea20 tx=0x7fb2cc00ee20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:51:56.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:56.447+0000 7fb2d3400640 1 -- 172.21.15.5:0/1165272754 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2cc00cd80 con 0x7fb2dc103e30 2024-09-10T09:51:56.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:56.447+0000 7fb2d3400640 1 -- 172.21.15.5:0/1165272754 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb2cc005500 con 0x7fb2dc103e30 2024-09-10T09:51:56.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:56.447+0000 7fb2d3400640 1 -- 172.21.15.5:0/1165272754 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2cc010690 con 0x7fb2dc103e30 2024-09-10T09:51:56.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:56.447+0000 7fb2e0a00640 1 -- 172.21.15.5:0/1165272754 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2dc1aaec0 con 0x7fb2dc103e30 2024-09-10T09:51:56.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:56.447+0000 7fb2e0a00640 1 -- 172.21.15.5:0/1165272754 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2dc1ab320 con 0x7fb2dc103e30 2024-09-10T09:51:56.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:56.448+0000 7fb2e0a00640 1 -- 172.21.15.5:0/1165272754 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb2dc105310 con 0x7fb2dc103e30 2024-09-10T09:51:56.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:56.450+0000 7fb2d3400640 1 -- 172.21.15.5:0/1165272754 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7fb2cc0046c0 con 0x7fb2dc103e30 2024-09-10T09:51:56.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:56.451+0000 7fb2d3400640 1 --2- 172.21.15.5:0/1165272754 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fb2c00a2c00 0x7fb2c00a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:51:56.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:56.452+0000 7fb2d3400640 1 -- 172.21.15.5:0/1165272754 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fb2cc0cf7c0 con 0x7fb2dc103e30 2024-09-10T09:51:56.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:56.452+0000 7fb2d9e00640 1 --2- 172.21.15.5:0/1165272754 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fb2c00a2c00 0x7fb2c00a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:51:56.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:56.453+0000 7fb2d9e00640 1 --2- 172.21.15.5:0/1165272754 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fb2c00a2c00 0x7fb2c00a50c0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fb2c8000c00 tx=0x7fb2c8005b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:51:56.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:56.456+0000 7fb2d3400640 1 -- 172.21.15.5:0/1165272754 <== mon.1 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb2cc09c2d0 con 0x7fb2dc103e30 2024-09-10T09:51:56.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:56.700+0000 7fb2e0a00640 1 -- 172.21.15.5:0/1165272754 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb2dc02d7c0 con 0x7fb2dc103e30 2024-09-10T09:51:56.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:56.701+0000 7fb2d3400640 1 -- 172.21.15.5:0/1165272754 <== mon.1 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) ==== 94+0+11969 (secure 0 0 0) 0x7fb2cc09bbd0 con 0x7fb2dc103e30 2024-09-10T09:51:56.702 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-10T09:51:56.703 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 36 2024-09-10T09:51:56.703 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":36,"btime":"2024-09-10T09:48:07:349075+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14454,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/2970594311","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2970594311},{"type":"v1","addr":"172.21.15.5:6835","nonce":2970594311}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14514,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6839/3616502174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":3616502174},{"type":"v1","addr":"172.21.15.5:6839","nonce":3616502174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":36,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T09:35:29.626287+0000","modified":"2024-09-10T09:48:06.970127+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24542,"mds_1":24535,"mds_2":24511,"mds_3":24506,"mds_4":24443},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14520":{"gid":14520,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/2073599842","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":2073599842},{"type":"v1","addr":"172.21.15.5:6841","nonce":2073599842}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"k","rank":4,"incarnation":26,"state":"up:active","state_seq":24,"addr":"172.21.15.102:6833/2457007427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":2457007427},{"type":"v1","addr":"172.21.15.102:6833","nonce":2457007427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/356445159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":356445159},{"type":"v1","addr":"172.21.15.159:6833","nonce":356445159}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6837/2894314507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":2894314507},{"type":"v1","addr":"172.21.15.5:6837","nonce":2894314507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.102:6835/1647549111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":1647549111},{"type":"v1","addr":"172.21.15.102:6835","nonce":1647549111}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6835/952657242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":952657242},{"type":"v1","addr":"172.21.15.159:6835","nonce":952657242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":19,"addr":"172.21.15.102:6837/2689776764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":2689776764},{"type":"v1","addr":"172.21.15.102:6837","nonce":2689776764}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.159:6837/620627639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":620627639},{"type":"v1","addr":"172.21.15.159:6837","nonce":620627639}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24535":{"gid":24535,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":18,"addr":"172.21.15.159:6839/2073739599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":2073739599},{"type":"v1","addr":"172.21.15.159:6839","nonce":2073739599}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24542":{"gid":24542,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.102:6839/335409749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6838","nonce":335409749},{"type":"v1","addr":"172.21.15.102:6839","nonce":335409749}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24542,"qdb_cluster":[24542,24535,24511,24506,24443]},"id":1}]} 2024-09-10T09:51:56.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:56.704+0000 7fb2e0a00640 1 -- 172.21.15.5:0/1165272754 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fb2c00a2c00 msgr2=0x7fb2c00a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:51:56.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:56.704+0000 7fb2e0a00640 1 --2- 172.21.15.5:0/1165272754 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fb2c00a2c00 0x7fb2c00a50c0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fb2c8000c00 tx=0x7fb2c8005b50 comp rx=0 tx=0).stop 2024-09-10T09:51:56.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:56.704+0000 7fb2e0a00640 1 -- 172.21.15.5:0/1165272754 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb2dc103e30 msgr2=0x7fb2dc076700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:51:56.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:56.704+0000 7fb2e0a00640 1 --2- 172.21.15.5:0/1165272754 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb2dc103e30 0x7fb2dc076700 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fb2cc00ea20 tx=0x7fb2cc00ee20 comp rx=0 tx=0).stop 2024-09-10T09:51:56.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:56.704+0000 7fb2e0a00640 1 -- 172.21.15.5:0/1165272754 shutdown_connections 2024-09-10T09:51:56.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:56.704+0000 7fb2e0a00640 1 --2- 172.21.15.5:0/1165272754 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fb2c00a2c00 0x7fb2c00a50c0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:56.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:56.704+0000 7fb2e0a00640 1 --2- 172.21.15.5:0/1165272754 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb2dc108f60 0x7fb2dc077180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:56.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:56.704+0000 7fb2e0a00640 1 --2- 172.21.15.5:0/1165272754 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb2dc1047e0 0x7fb2dc076c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:56.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:56.704+0000 7fb2e0a00640 1 --2- 172.21.15.5:0/1165272754 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb2dc103e30 0x7fb2dc076700 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:56.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:56.704+0000 7fb2e0a00640 1 -- 172.21.15.5:0/1165272754 >> 172.21.15.5:0/1165272754 conn(0x7fb2dc0ff8d0 msgr2=0x7fb2dc100bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:51:56.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:56.704+0000 7fb2e0a00640 1 -- 172.21.15.5:0/1165272754 shutdown_connections 2024-09-10T09:51:56.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:56.704+0000 7fb2e0a00640 1 -- 172.21.15.5:0/1165272754 wait complete. 2024-09-10T09:51:57.164 INFO:tasks.workunit.client.0.smithi005.stdout: 1 863321 6.29 MB/sec execute 578 sec latency 1595.444 ms 2024-09-10T09:51:57.231 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:51:57 smithi005 ceph-mon[30350]: pgmap v801: 129 pgs: 129 active+clean; 290 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 134 KiB/s rd, 1.6 MiB/s wr, 80 op/s 2024-09-10T09:51:57.231 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:51:57 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:51:57.231 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:51:57 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:51:57.231 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:51:57 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:51:57.231 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:51:57 smithi005 ceph-mon[30350]: from='client.? 172.21.15.5:0/1165272754' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:51:57.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:51:57 smithi159 ceph-mon[35513]: pgmap v801: 129 pgs: 129 active+clean; 290 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 134 KiB/s rd, 1.6 MiB/s wr, 80 op/s 2024-09-10T09:51:57.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:51:57 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:51:57.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:51:57 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:51:57.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:51:57 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:51:57.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:51:57 smithi159 ceph-mon[35513]: from='client.? 172.21.15.5:0/1165272754' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:51:57.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:51:57 smithi102 ceph-mon[35442]: pgmap v801: 129 pgs: 129 active+clean; 290 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 134 KiB/s rd, 1.6 MiB/s wr, 80 op/s 2024-09-10T09:51:57.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:51:57 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:51:57.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:51:57 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:51:57.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:51:57 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:51:57.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:51:57 smithi102 ceph-mon[35442]: from='client.? 172.21.15.5:0/1165272754' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:51:57.676 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24542 quiesce db / --timeout 90 --expiration 141.5 --await-for 45.0' 2024-09-10T09:51:57.676 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24542 quiesce db / --timeout 90 --expiration 141.5 --await-for 45.0 2024-09-10T09:51:58.111 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:51:58.164 INFO:tasks.workunit.client.0.smithi005.stdout: 1 864638 6.29 MB/sec execute 579 sec latency 298.490 ms 2024-09-10T09:51:58.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:58.720+0000 7fd690a00640 1 -- 172.21.15.5:0/2265633357 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd68c074e30 msgr2=0x7fd68c10dd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:51:58.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:58.720+0000 7fd690a00640 1 --2- 172.21.15.5:0/2265633357 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd68c074e30 0x7fd68c10dd40 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fd678009900 tx=0x7fd67802a480 comp rx=0 tx=0).stop 2024-09-10T09:51:58.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:58.720+0000 7fd690a00640 1 -- 172.21.15.5:0/2265633357 shutdown_connections 2024-09-10T09:51:58.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:58.720+0000 7fd690a00640 1 --2- 172.21.15.5:0/2265633357 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd68c10e280 0x7fd68c110670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:58.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:58.720+0000 7fd690a00640 1 --2- 172.21.15.5:0/2265633357 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd68c074e30 0x7fd68c10dd40 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:58.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:58.720+0000 7fd690a00640 1 --2- 172.21.15.5:0/2265633357 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd68c074510 0x7fd68c0748f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:58.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:58.720+0000 7fd690a00640 1 -- 172.21.15.5:0/2265633357 >> 172.21.15.5:0/2265633357 conn(0x7fd68c0fd740 msgr2=0x7fd68c0ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:51:58.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:58.720+0000 7fd690a00640 1 -- 172.21.15.5:0/2265633357 shutdown_connections 2024-09-10T09:51:58.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:58.721+0000 7fd690a00640 1 -- 172.21.15.5:0/2265633357 wait complete. 2024-09-10T09:51:58.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:58.721+0000 7fd690a00640 1 Processor -- start 2024-09-10T09:51:58.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:58.722+0000 7fd690a00640 1 -- start start 2024-09-10T09:51:58.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:58.722+0000 7fd690a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd68c074510 0x7fd68c10dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:51:58.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:58.722+0000 7fd690a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd68c074e30 0x7fd68c10ad60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:51:58.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:58.722+0000 7fd690a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd68c10e280 0x7fd68c10b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:51:58.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:58.722+0000 7fd690a00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd68c113fb0 con 0x7fd68c074e30 2024-09-10T09:51:58.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:58.722+0000 7fd690a00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fd68c113e30 con 0x7fd68c10e280 2024-09-10T09:51:58.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:58.722+0000 7fd690a00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fd68c114130 con 0x7fd68c074510 2024-09-10T09:51:58.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:58.722+0000 7fd68a800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd68c074510 0x7fd68c10dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:51:58.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:58.722+0000 7fd68a800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd68c074510 0x7fd68c10dcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:48560/0 (socket says 172.21.15.5:48560) 2024-09-10T09:51:58.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:58.722+0000 7fd68a800640 1 -- 172.21.15.5:0/3011980004 learned_addr learned my addr 172.21.15.5:0/3011980004 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:51:58.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:58.722+0000 7fd689e00640 1 --2- 172.21.15.5:0/3011980004 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd68c074e30 0x7fd68c10ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:51:58.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:58.722+0000 7fd68b200640 1 --2- 172.21.15.5:0/3011980004 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd68c10e280 0x7fd68c10b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:51:58.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:58.723+0000 7fd68a800640 1 -- 172.21.15.5:0/3011980004 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd68c10e280 msgr2=0x7fd68c10b4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:51:58.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:58.723+0000 7fd68a800640 1 --2- 172.21.15.5:0/3011980004 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd68c10e280 0x7fd68c10b4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:58.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:58.723+0000 7fd68a800640 1 -- 172.21.15.5:0/3011980004 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd68c074e30 msgr2=0x7fd68c10ad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:51:58.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:58.723+0000 7fd68a800640 1 --2- 172.21.15.5:0/3011980004 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd68c074e30 0x7fd68c10ad60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:58.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:58.723+0000 7fd68a800640 1 -- 172.21.15.5:0/3011980004 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd68c1aacd0 con 0x7fd68c074510 2024-09-10T09:51:58.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:58.723+0000 7fd68b200640 1 --2- 172.21.15.5:0/3011980004 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd68c10e280 0x7fd68c10b4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:51:58.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:58.723+0000 7fd68a800640 1 --2- 172.21.15.5:0/3011980004 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd68c074510 0x7fd68c10dcb0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fd67c00e9f0 tx=0x7fd67c00edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:51:58.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:58.723+0000 7fd683400640 1 -- 172.21.15.5:0/3011980004 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd67c00cd80 con 0x7fd68c074510 2024-09-10T09:51:58.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:58.724+0000 7fd683400640 1 -- 172.21.15.5:0/3011980004 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd67c005500 con 0x7fd68c074510 2024-09-10T09:51:58.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:58.724+0000 7fd683400640 1 -- 172.21.15.5:0/3011980004 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd67c010690 con 0x7fd68c074510 2024-09-10T09:51:58.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:58.724+0000 7fd690a00640 1 -- 172.21.15.5:0/3011980004 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd68c1aaea0 con 0x7fd68c074510 2024-09-10T09:51:58.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:58.724+0000 7fd690a00640 1 -- 172.21.15.5:0/3011980004 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd68c1ab240 con 0x7fd68c074510 2024-09-10T09:51:58.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:58.727+0000 7fd694056740 1 Processor -- start 2024-09-10T09:51:58.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:58.727+0000 7fd683400640 1 -- 172.21.15.5:0/3011980004 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7fd67c0046c0 con 0x7fd68c074510 2024-09-10T09:51:58.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:58.727+0000 7fd694056740 1 -- start start 2024-09-10T09:51:58.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:58.728+0000 7fd694056740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55efd7944560 0x55efd7944940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:51:58.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:58.728+0000 7fd694056740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55efd7944e80 0x55efd794f200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:51:58.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:58.728+0000 7fd694056740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55efd794f740 0x55efd7951b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:51:58.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:58.728+0000 7fd694056740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55efd78aa530 con 0x55efd7944560 2024-09-10T09:51:58.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:58.728+0000 7fd694056740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55efd7851860 con 0x55efd7944e80 2024-09-10T09:51:58.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:58.728+0000 7fd694056740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55efd78a64f0 con 0x55efd794f740 2024-09-10T09:51:58.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:58.728+0000 7fd68b200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55efd7944560 0x55efd7944940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:51:58.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:58.728+0000 7fd68a800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55efd7944e80 0x55efd794f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:51:58.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:58.728+0000 7fd68b200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55efd7944560 0x55efd7944940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60292/0 (socket says 172.21.15.5:60292) 2024-09-10T09:51:58.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:58.728+0000 7fd689e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55efd794f740 0x55efd7951b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:51:58.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:58.728+0000 7fd68b200640 1 -- 172.21.15.5:0/3652919863 learned_addr learned my addr 172.21.15.5:0/3652919863 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:51:58.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:58.728+0000 7fd68b200640 1 -- 172.21.15.5:0/3652919863 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55efd794f740 msgr2=0x55efd7951b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:51:58.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:58.729+0000 7fd68b200640 1 --2- 172.21.15.5:0/3652919863 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55efd794f740 0x55efd7951b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:58.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:58.729+0000 7fd68b200640 1 -- 172.21.15.5:0/3652919863 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55efd7944e80 msgr2=0x55efd794f200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:51:58.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:58.729+0000 7fd68b200640 1 --2- 172.21.15.5:0/3652919863 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55efd7944e80 0x55efd794f200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:58.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:58.729+0000 7fd68b200640 1 -- 172.21.15.5:0/3652919863 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55efd761bb90 con 0x55efd7944560 2024-09-10T09:51:58.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:58.729+0000 7fd689e00640 1 --2- 172.21.15.5:0/3652919863 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55efd794f740 0x55efd7951b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:51:58.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:58.729+0000 7fd68b200640 1 --2- 172.21.15.5:0/3652919863 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55efd7944560 0x55efd7944940 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fd684011760 tx=0x7fd684011c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8279d84710e72ae7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:51:58.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:58.729+0000 7fd683400640 1 --2- 172.21.15.5:0/3011980004 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fd6700a2b80 0x7fd6700a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:51:58.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:58.729+0000 7fd677400640 1 -- 172.21.15.5:0/3652919863 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd684004d20 con 0x55efd7944560 2024-09-10T09:51:58.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:58.729+0000 7fd677400640 1 -- 172.21.15.5:0/3652919863 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd68400f9a0 con 0x55efd7944560 2024-09-10T09:51:58.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:58.729+0000 7fd677400640 1 -- 172.21.15.5:0/3652919863 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd684004470 con 0x55efd7944560 2024-09-10T09:51:58.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:58.729+0000 7fd683400640 1 -- 172.21.15.5:0/3011980004 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fd67c0cf860 con 0x7fd68c074510 2024-09-10T09:51:58.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:58.730+0000 7fd68a800640 1 --2- 172.21.15.5:0/3011980004 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fd6700a2b80 0x7fd6700a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:51:58.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:58.730+0000 7fd694056740 1 -- 172.21.15.5:0/3652919863 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55efd7944560 msgr2=0x55efd7944940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:51:58.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:58.730+0000 7fd694056740 1 --2- 172.21.15.5:0/3652919863 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55efd7944560 0x55efd7944940 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fd684011760 tx=0x7fd684011c20 comp rx=0 tx=0).stop 2024-09-10T09:51:58.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:58.730+0000 7fd68a800640 1 --2- 172.21.15.5:0/3011980004 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fd6700a2b80 0x7fd6700a5040 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fd67c0980d0 tx=0x7fd67c0d5530 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:51:58.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:58.730+0000 7fd694056740 1 -- 172.21.15.5:0/3652919863 shutdown_connections 2024-09-10T09:51:58.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:58.730+0000 7fd694056740 1 --2- 172.21.15.5:0/3652919863 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55efd794f740 0x55efd7951b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:58.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:58.730+0000 7fd694056740 1 --2- 172.21.15.5:0/3652919863 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55efd7944e80 0x55efd794f200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:58.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:58.730+0000 7fd694056740 1 --2- 172.21.15.5:0/3652919863 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55efd7944560 0x55efd7944940 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:58.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:58.730+0000 7fd694056740 1 -- 172.21.15.5:0/3652919863 >> 172.21.15.5:0/3652919863 conn(0x55efd7940af0 msgr2=0x55efd7940ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:51:58.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:58.730+0000 7fd694056740 1 -- 172.21.15.5:0/3652919863 shutdown_connections 2024-09-10T09:51:58.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:58.730+0000 7fd694056740 1 -- 172.21.15.5:0/3652919863 wait complete. 2024-09-10T09:51:58.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:58.731+0000 7fd694056740 1 -- start start 2024-09-10T09:51:58.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:58.731+0000 7fd694056740 1 Processor -- start 2024-09-10T09:51:58.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:58.732+0000 7fd694056740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55efd7944560 0x55efd794c590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:51:58.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:58.732+0000 7fd694056740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55efd7944e80 0x55efd794cad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:51:58.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:58.732+0000 7fd694056740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55efd794f740 0x55efd794d010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:51:58.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:58.732+0000 7fd694056740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55efd7952310 con 0x55efd7944560 2024-09-10T09:51:58.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:58.732+0000 7fd694056740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55efd7864e60 con 0x55efd794f740 2024-09-10T09:51:58.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:58.732+0000 7fd694056740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55efd7952490 con 0x55efd7944e80 2024-09-10T09:51:58.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:58.732+0000 7fd694056740 10 client.0 _ll_register_callbacks cb 0x55efd7869cc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:51:58.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:58.732+0000 7fd689e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55efd7944e80 0x55efd794cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:51:58.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:58.732+0000 7fd689e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55efd7944e80 0x55efd794cad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:48574/0 (socket says 172.21.15.5:48574) 2024-09-10T09:51:58.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:58.732+0000 7fd689e00640 1 -- 172.21.15.5:0/4225147817 learned_addr learned my addr 172.21.15.5:0/4225147817 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:51:58.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:58.732+0000 7fd68b200640 1 --2- 172.21.15.5:0/4225147817 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55efd7944560 0x55efd794c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:51:58.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:58.732+0000 7fd68b200640 1 --2- 172.21.15.5:0/4225147817 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55efd794f740 0x55efd794d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:51:58.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:58.732+0000 7fd689e00640 1 -- 172.21.15.5:0/4225147817 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55efd794f740 msgr2=0x55efd794d010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:51:58.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:58.732+0000 7fd689e00640 1 --2- 172.21.15.5:0/4225147817 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55efd794f740 0x55efd794d010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:58.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:58.732+0000 7fd689e00640 1 -- 172.21.15.5:0/4225147817 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55efd7944560 msgr2=0x55efd794c590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:51:58.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:58.732+0000 7fd689e00640 1 --2- 172.21.15.5:0/4225147817 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55efd7944560 0x55efd794c590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:58.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:58.732+0000 7fd689e00640 1 -- 172.21.15.5:0/4225147817 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55efd7870990 con 0x55efd7944e80 2024-09-10T09:51:58.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:58.732+0000 7fd68b200640 1 --2- 172.21.15.5:0/4225147817 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55efd7944560 0x55efd794c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:51:58.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:58.733+0000 7fd689e00640 1 --2- 172.21.15.5:0/4225147817 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55efd7944e80 0x55efd794cad0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fd678009900 tx=0x7fd6780317e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=582933db11736f55 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:51:58.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:58.733+0000 7fd66be00640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-09-10T09:51:58.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:58.733+0000 7fd66be00640 1 -- 172.21.15.5:0/4225147817 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd678035040 con 0x55efd7944e80 2024-09-10T09:51:58.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:58.733+0000 7fd66be00640 1 -- 172.21.15.5:0/4225147817 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fd678004240 con 0x55efd7944e80 2024-09-10T09:51:58.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:58.733+0000 7fd694056740 1 -- 172.21.15.5:0/4225147817 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd68400f9a0 con 0x55efd7944e80 2024-09-10T09:51:58.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:58.733+0000 7fd66be00640 1 -- 172.21.15.5:0/4225147817 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fd678031a10 con 0x55efd7944e80 2024-09-10T09:51:58.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:58.733+0000 7fd694056740 10 client.28141 fetch_fsmap learned FSMap version 36 2024-09-10T09:51:58.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:58.734+0000 7fd694056740 1 -- 172.21.15.5:0/4225147817 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55efd795dea0 con 0x55efd7944e80 2024-09-10T09:51:58.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:58.734+0000 7fd66be00640 1 -- 172.21.15.5:0/4225147817 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd68400f9a0 con 0x55efd7944e80 2024-09-10T09:51:58.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:58.734+0000 7fd66be00640 1 -- 172.21.15.5:0/4225147817 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 36) ==== 8370+0+0 (secure 0 0 0) 0x7fd678046600 con 0x55efd7944e80 2024-09-10T09:51:58.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:58.734+0000 7fd694056740 10 client.28141 fetch_fsmap finished waiting for FSMap version 36 2024-09-10T09:51:58.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:58.734+0000 7fd694056740 10 client.28141 resolve_mds: validated gid 24542 aka daemon mds.b 2024-09-10T09:51:58.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:58.734+0000 7fd694056740 20 client.28141 populate_metadata read hostname 'smithi005' 2024-09-10T09:51:58.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:58.734+0000 7fd694056740 1 --2- 172.21.15.5:0/4225147817 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55efd795f360 0x55efd7961790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:51:58.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:58.734+0000 7fd694056740 4 client.28141 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:51:58.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:58.734+0000 7fd694056740 1 -- 172.21.15.5:0/4225147817 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55efd784d310 con 0x55efd795f360 2024-09-10T09:51:58.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:58.734+0000 7fd68b200640 1 --2- 172.21.15.5:0/4225147817 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55efd795f360 0x55efd7961790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:51:58.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:58.735+0000 7fd68b200640 1 --2- 172.21.15.5:0/4225147817 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55efd795f360 0x55efd7961790 crc :-1 s=READY pgs=207 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d7b70de38708dc05 server_cookie=89a026b9bc1e76d5 in_seq=0 out_seq=0 2024-09-10T09:51:58.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:58.735+0000 7fd66be00640 10 client.28141 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T09:51:58.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:58.736+0000 7fd66be00640 1 -- 172.21.15.5:0/4225147817 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55efd784d310 con 0x55efd795f360 2024-09-10T09:51:58.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:58.736+0000 7fd66be00640 10 client.28141 handle_command_reply: tid=0 2024-09-10T09:51:58.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:58.736+0000 7fd694056740 1 client.28141 shutdown 2024-09-10T09:51:58.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:58.736+0000 7fd694056740 1 -- 172.21.15.5:0/4225147817 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55efd7944e80 msgr2=0x55efd794cad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:51:58.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:58.736+0000 7fd694056740 1 --2- 172.21.15.5:0/4225147817 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55efd7944e80 0x55efd794cad0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fd678009900 tx=0x7fd6780317e0 comp rx=0 tx=0).stop 2024-09-10T09:51:58.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:58.736+0000 7fd694056740 1 -- 172.21.15.5:0/4225147817 shutdown_connections 2024-09-10T09:51:58.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:58.736+0000 7fd694056740 1 --2- 172.21.15.5:0/4225147817 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55efd795f360 0x55efd7961790 crc :-1 s=READY pgs=207 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:58.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:58.736+0000 7fd694056740 1 --2- 172.21.15.5:0/4225147817 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55efd794f740 0x55efd794d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:58.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:58.736+0000 7fd694056740 1 --2- 172.21.15.5:0/4225147817 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55efd7944e80 0x55efd794cad0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:58.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:58.736+0000 7fd694056740 1 --2- 172.21.15.5:0/4225147817 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55efd7944560 0x55efd794c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:58.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:58.736+0000 7fd66be00640 0 client.28141 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T09:51:58.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:58.736+0000 7fd694056740 1 -- 172.21.15.5:0/4225147817 >> 172.21.15.5:0/4225147817 conn(0x55efd7940af0 msgr2=0x55efd7946820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:51:58.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:58.736+0000 7fd694056740 1 -- 172.21.15.5:0/4225147817 shutdown_connections 2024-09-10T09:51:58.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:58.736+0000 7fd694056740 1 -- 172.21.15.5:0/4225147817 wait complete. 2024-09-10T09:51:58.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:58.736+0000 7fd694056740 20 client.28141 trim_cache size 0 max 16384 2024-09-10T09:51:58.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:58.756+0000 7fd694056740 1 Processor -- start 2024-09-10T09:51:58.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:58.756+0000 7fd694056740 1 -- start start 2024-09-10T09:51:58.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:58.756+0000 7fd694056740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55efd7944560 0x55efd794b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:51:58.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:58.756+0000 7fd694056740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55efd7944e80 0x55efd79670a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:51:58.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:58.756+0000 7fd694056740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55efd794f740 0x55efd79675e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:51:58.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:58.756+0000 7fd694056740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd68c1b52c0 con 0x55efd7944560 2024-09-10T09:51:58.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:58.756+0000 7fd694056740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fd68c1b6e00 con 0x55efd794f740 2024-09-10T09:51:58.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:58.756+0000 7fd694056740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fd68c1b5990 con 0x55efd7944e80 2024-09-10T09:51:58.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:58.756+0000 7fd689e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55efd7944e80 0x55efd79670a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:51:58.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:58.756+0000 7fd68b200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55efd7944560 0x55efd794b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:51:58.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:58.757+0000 7fd689e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55efd7944e80 0x55efd79670a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:48578/0 (socket says 172.21.15.5:48578) 2024-09-10T09:51:58.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:58.757+0000 7fd689e00640 1 -- 172.21.15.5:0/2784618509 learned_addr learned my addr 172.21.15.5:0/2784618509 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:51:58.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:58.757+0000 7fd68b200640 1 --2- 172.21.15.5:0/2784618509 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55efd794f740 0x55efd79675e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:51:58.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:58.757+0000 7fd689e00640 1 -- 172.21.15.5:0/2784618509 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55efd794f740 msgr2=0x55efd79675e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:51:58.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:58.757+0000 7fd689e00640 1 --2- 172.21.15.5:0/2784618509 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55efd794f740 0x55efd79675e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:58.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:58.757+0000 7fd689e00640 1 -- 172.21.15.5:0/2784618509 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55efd7944560 msgr2=0x55efd794b4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:51:58.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:58.757+0000 7fd689e00640 1 --2- 172.21.15.5:0/2784618509 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55efd7944560 0x55efd794b4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:58.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:58.757+0000 7fd689e00640 1 -- 172.21.15.5:0/2784618509 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55efd7940920 con 0x55efd7944e80 2024-09-10T09:51:58.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:58.757+0000 7fd68b200640 1 --2- 172.21.15.5:0/2784618509 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55efd794f740 0x55efd79675e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:51:58.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:58.757+0000 7fd689e00640 1 --2- 172.21.15.5:0/2784618509 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55efd7944e80 0x55efd79670a0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fd678031550 tx=0x7fd678005fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2cd6729db2a46b51 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:51:58.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:58.757+0000 7fd676a00640 1 -- 172.21.15.5:0/2784618509 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd678058070 con 0x55efd7944e80 2024-09-10T09:51:58.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:58.758+0000 7fd676a00640 1 -- 172.21.15.5:0/2784618509 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd678005520 con 0x55efd7944e80 2024-09-10T09:51:58.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:58.758+0000 7fd676a00640 1 -- 172.21.15.5:0/2784618509 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd678035030 con 0x55efd7944e80 2024-09-10T09:51:58.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:58.758+0000 7fd694056740 1 -- 172.21.15.5:0/2784618509 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55efd7944e80 msgr2=0x55efd79670a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:51:58.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:58.758+0000 7fd694056740 1 --2- 172.21.15.5:0/2784618509 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55efd7944e80 0x55efd79670a0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fd678031550 tx=0x7fd678005fd0 comp rx=0 tx=0).stop 2024-09-10T09:51:58.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:58.758+0000 7fd694056740 1 -- 172.21.15.5:0/2784618509 shutdown_connections 2024-09-10T09:51:58.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:58.758+0000 7fd694056740 1 --2- 172.21.15.5:0/2784618509 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55efd794f740 0x55efd79675e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:58.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:58.758+0000 7fd694056740 1 --2- 172.21.15.5:0/2784618509 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55efd7944e80 0x55efd79670a0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:58.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:58.758+0000 7fd694056740 1 --2- 172.21.15.5:0/2784618509 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55efd7944560 0x55efd794b4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:58.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:58.758+0000 7fd694056740 1 -- 172.21.15.5:0/2784618509 >> 172.21.15.5:0/2784618509 conn(0x55efd7940af0 msgr2=0x55efd79467d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:51:58.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:58.759+0000 7fd694056740 1 -- 172.21.15.5:0/2784618509 shutdown_connections 2024-09-10T09:51:58.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:58.759+0000 7fd694056740 1 -- 172.21.15.5:0/2784618509 wait complete. 2024-09-10T09:51:58.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:58.759+0000 7fd694056740 1 -- start start 2024-09-10T09:51:58.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:58.759+0000 7fd694056740 1 Processor -- start 2024-09-10T09:51:58.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:58.760+0000 7fd694056740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55efd7944560 0x55efd79578c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:51:58.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:58.760+0000 7fd694056740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55efd7944e80 0x55efd7957e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:51:58.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:58.760+0000 7fd694056740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55efd794f740 0x55efd79546b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:51:58.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:58.760+0000 7fd694056740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd68c1b6730 con 0x55efd7944560 2024-09-10T09:51:58.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:58.760+0000 7fd694056740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fd68c1b6060 con 0x55efd794f740 2024-09-10T09:51:58.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:58.760+0000 7fd694056740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fd68c1b32a0 con 0x55efd7944e80 2024-09-10T09:51:58.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:58.760+0000 7fd694056740 10 client.0 _ll_register_callbacks cb 0x7fd678009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:51:58.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:58.760+0000 7fd689e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55efd7944e80 0x55efd7957e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:51:58.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:58.760+0000 7fd689e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55efd7944e80 0x55efd7957e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:48580/0 (socket says 172.21.15.5:48580) 2024-09-10T09:51:58.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:58.760+0000 7fd689e00640 1 -- 172.21.15.5:0/1274559200 learned_addr learned my addr 172.21.15.5:0/1274559200 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:51:58.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:58.760+0000 7fd68b200640 1 --2- 172.21.15.5:0/1274559200 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55efd7944560 0x55efd79578c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:51:58.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:58.760+0000 7fd68b200640 1 --2- 172.21.15.5:0/1274559200 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55efd794f740 0x55efd79546b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:51:58.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:58.760+0000 7fd689e00640 1 -- 172.21.15.5:0/1274559200 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55efd794f740 msgr2=0x55efd79546b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:51:58.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:58.760+0000 7fd689e00640 1 --2- 172.21.15.5:0/1274559200 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55efd794f740 0x55efd79546b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:58.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:58.760+0000 7fd689e00640 1 -- 172.21.15.5:0/1274559200 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55efd7944560 msgr2=0x55efd79578c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:51:58.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:58.760+0000 7fd689e00640 1 --2- 172.21.15.5:0/1274559200 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55efd7944560 0x55efd79578c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:58.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:58.760+0000 7fd689e00640 1 -- 172.21.15.5:0/1274559200 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55efd7954eb0 con 0x55efd7944e80 2024-09-10T09:51:58.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:58.761+0000 7fd689e00640 1 --2- 172.21.15.5:0/1274559200 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55efd7944e80 0x55efd7957e00 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fd678046a50 tx=0x7fd678005a20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f4acd4902abeeebc server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:51:58.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:58.761+0000 7fd66be00640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-09-10T09:51:58.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:58.761+0000 7fd66be00640 1 -- 172.21.15.5:0/1274559200 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd678058040 con 0x55efd7944e80 2024-09-10T09:51:58.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:58.761+0000 7fd694056740 1 -- 172.21.15.5:0/1274559200 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd678005520 con 0x55efd7944e80 2024-09-10T09:51:58.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:58.761+0000 7fd66be00640 1 -- 172.21.15.5:0/1274559200 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fd6780046c0 con 0x55efd7944e80 2024-09-10T09:51:58.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:58.761+0000 7fd66be00640 1 -- 172.21.15.5:0/1274559200 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fd6780048c0 con 0x55efd7944e80 2024-09-10T09:51:58.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:58.761+0000 7fd694056740 10 client.28147 fetch_fsmap learned FSMap version 36 2024-09-10T09:51:58.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:58.761+0000 7fd694056740 1 -- 172.21.15.5:0/1274559200 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55efd79caae0 con 0x55efd7944e80 2024-09-10T09:51:58.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:58.762+0000 7fd66be00640 1 -- 172.21.15.5:0/1274559200 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd678005520 con 0x55efd7944e80 2024-09-10T09:51:58.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:58.762+0000 7fd66be00640 1 -- 172.21.15.5:0/1274559200 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 36) ==== 8370+0+0 (secure 0 0 0) 0x7fd67805d620 con 0x55efd7944e80 2024-09-10T09:51:58.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:58.762+0000 7fd694056740 10 client.28147 fetch_fsmap finished waiting for FSMap version 36 2024-09-10T09:51:58.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:58.762+0000 7fd694056740 10 client.28147 resolve_mds: validated gid 24542 aka daemon mds.b 2024-09-10T09:51:58.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:58.762+0000 7fd694056740 20 client.28147 populate_metadata read hostname 'smithi005' 2024-09-10T09:51:58.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:58.762+0000 7fd694056740 1 --2- 172.21.15.5:0/1274559200 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55efd79cafd0 0x55efd79cd3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:51:58.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:58.762+0000 7fd694056740 4 client.28147 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 141.5, "await_for": 45.0}] 2024-09-10T09:51:58.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:58.762+0000 7fd694056740 1 -- 172.21.15.5:0/1274559200 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 141.5, "await_for": 45.0}) -- 0x55efd78b7c60 con 0x55efd79cafd0 2024-09-10T09:51:58.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:58.762+0000 7fd68b200640 1 --2- 172.21.15.5:0/1274559200 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55efd79cafd0 0x55efd79cd3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:51:58.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:58.763+0000 7fd68b200640 1 --2- 172.21.15.5:0/1274559200 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55efd79cafd0 0x55efd79cd3c0 crc :-1 s=READY pgs=208 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=198972f7b7907f89 server_cookie=bbe06c0da62d33f4 in_seq=0 out_seq=0 2024-09-10T09:51:58.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:58.763+0000 7fd66be00640 10 client.28147 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T09:51:58.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:58.966+0000 7fd66be00640 1 -- 172.21.15.5:0/1274559200 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55efd78b7c60 con 0x55efd79cafd0 2024-09-10T09:51:58.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:58.966+0000 7fd66be00640 10 client.28147 handle_command_reply: tid=0 2024-09-10T09:51:58.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:58.966+0000 7fd694056740 1 client.28147 shutdown 2024-09-10T09:51:58.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:58.967+0000 7fd694056740 1 -- 172.21.15.5:0/1274559200 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55efd7944e80 msgr2=0x55efd7957e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:51:58.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:58.967+0000 7fd694056740 1 --2- 172.21.15.5:0/1274559200 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55efd7944e80 0x55efd7957e00 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fd678046a50 tx=0x7fd678005a20 comp rx=0 tx=0).stop 2024-09-10T09:51:58.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:58.967+0000 7fd694056740 1 -- 172.21.15.5:0/1274559200 shutdown_connections 2024-09-10T09:51:58.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:58.967+0000 7fd694056740 1 --2- 172.21.15.5:0/1274559200 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55efd79cafd0 0x55efd79cd3c0 crc :-1 s=READY pgs=208 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:58.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:58.967+0000 7fd694056740 1 --2- 172.21.15.5:0/1274559200 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55efd794f740 0x55efd79546b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:58.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:58.967+0000 7fd694056740 1 --2- 172.21.15.5:0/1274559200 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55efd7944e80 0x55efd7957e00 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:58.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:58.967+0000 7fd66be00640 0 client.28147 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T09:51:58.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:58.967+0000 7fd694056740 1 --2- 172.21.15.5:0/1274559200 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55efd7944560 0x55efd79578c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:58.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:58.967+0000 7fd694056740 1 -- 172.21.15.5:0/1274559200 >> 172.21.15.5:0/1274559200 conn(0x55efd7940af0 msgr2=0x55efd794c4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:51:58.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:58.967+0000 7fd694056740 1 -- 172.21.15.5:0/1274559200 shutdown_connections 2024-09-10T09:51:58.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:58.967+0000 7fd694056740 1 -- 172.21.15.5:0/1274559200 wait complete. 2024-09-10T09:51:58.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:58.967+0000 7fd694056740 20 client.28147 trim_cache size 0 max 16384 2024-09-10T09:51:58.971 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-10T09:51:58.971 INFO:teuthology.orchestra.run.smithi005.stdout: "epoch": 36, 2024-09-10T09:51:58.971 INFO:teuthology.orchestra.run.smithi005.stdout: "leader": 24542, 2024-09-10T09:51:58.971 INFO:teuthology.orchestra.run.smithi005.stdout: "set_version": 50, 2024-09-10T09:51:58.971 INFO:teuthology.orchestra.run.smithi005.stdout: "sets": { 2024-09-10T09:51:58.971 INFO:teuthology.orchestra.run.smithi005.stdout: "dbf77f9f": { 2024-09-10T09:51:58.971 INFO:teuthology.orchestra.run.smithi005.stdout: "version": 50, 2024-09-10T09:51:58.971 INFO:teuthology.orchestra.run.smithi005.stdout: "age_ref": 0.0, 2024-09-10T09:51:58.971 INFO:teuthology.orchestra.run.smithi005.stdout: "state": { 2024-09-10T09:51:58.971 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "QUIESCED", 2024-09-10T09:51:58.971 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0 2024-09-10T09:51:58.971 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T09:51:58.972 INFO:teuthology.orchestra.run.smithi005.stdout: "timeout": 90.0, 2024-09-10T09:51:58.972 INFO:teuthology.orchestra.run.smithi005.stdout: "expiration": 141.5, 2024-09-10T09:51:58.972 INFO:teuthology.orchestra.run.smithi005.stdout: "members": { 2024-09-10T09:51:58.972 INFO:teuthology.orchestra.run.smithi005.stdout: "file:/": { 2024-09-10T09:51:58.972 INFO:teuthology.orchestra.run.smithi005.stdout: "excluded": false, 2024-09-10T09:51:58.972 INFO:teuthology.orchestra.run.smithi005.stdout: "state": { 2024-09-10T09:51:58.972 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "QUIESCED", 2024-09-10T09:51:58.972 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0 2024-09-10T09:51:58.972 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T09:51:58.972 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T09:51:58.972 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T09:51:58.972 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T09:51:58.972 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T09:51:58.973 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-10T09:51:58.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:58.968+0000 7fd690a00640 1 -- 172.21.15.5:0/3011980004 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fd6700a2b80 msgr2=0x7fd6700a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:51:58.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:58.968+0000 7fd690a00640 1 --2- 172.21.15.5:0/3011980004 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fd6700a2b80 0x7fd6700a5040 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fd67c0980d0 tx=0x7fd67c0d5530 comp rx=0 tx=0).stop 2024-09-10T09:51:58.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:58.968+0000 7fd690a00640 1 -- 172.21.15.5:0/3011980004 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd68c074510 msgr2=0x7fd68c10dcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:51:58.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:58.968+0000 7fd690a00640 1 --2- 172.21.15.5:0/3011980004 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd68c074510 0x7fd68c10dcb0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fd67c00e9f0 tx=0x7fd67c00edf0 comp rx=0 tx=0).stop 2024-09-10T09:51:58.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:58.968+0000 7fd690a00640 1 -- 172.21.15.5:0/3011980004 shutdown_connections 2024-09-10T09:51:58.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:58.968+0000 7fd690a00640 1 --2- 172.21.15.5:0/3011980004 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fd6700a2b80 0x7fd6700a5040 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:58.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:58.968+0000 7fd690a00640 1 --2- 172.21.15.5:0/3011980004 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd68c10e280 0x7fd68c10b4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:58.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:58.968+0000 7fd690a00640 1 --2- 172.21.15.5:0/3011980004 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd68c074e30 0x7fd68c10ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:58.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:58.968+0000 7fd690a00640 1 --2- 172.21.15.5:0/3011980004 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd68c074510 0x7fd68c10dcb0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:51:58.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:58.968+0000 7fd690a00640 1 -- 172.21.15.5:0/3011980004 >> 172.21.15.5:0/3011980004 conn(0x7fd68c0fd740 msgr2=0x7fd68c0fef70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:51:58.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:58.968+0000 7fd690a00640 1 -- 172.21.15.5:0/3011980004 shutdown_connections 2024-09-10T09:51:58.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:51:58.968+0000 7fd690a00640 1 -- 172.21.15.5:0/3011980004 wait complete. 2024-09-10T09:51:59.165 INFO:tasks.workunit.client.0.smithi005.stdout: 1 866428 6.30 MB/sec execute 580 sec latency 304.948 ms 2024-09-10T09:51:59.306 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:51:59 smithi005 ceph-mon[30350]: pgmap v802: 129 pgs: 129 active+clean; 289 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 294 KiB/s rd, 1.7 MiB/s wr, 82 op/s 2024-09-10T09:51:59.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:51:59 smithi159 ceph-mon[35513]: pgmap v802: 129 pgs: 129 active+clean; 289 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 294 KiB/s rd, 1.7 MiB/s wr, 82 op/s 2024-09-10T09:51:59.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:51:59 smithi102 ceph-mon[35442]: pgmap v802: 129 pgs: 129 active+clean; 289 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 294 KiB/s rd, 1.7 MiB/s wr, 82 op/s 2024-09-10T09:51:59.878 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'dbf77f9f', quiesce took 5.1 seconds. Will release after: 16.4 2024-09-10T09:52:00.165 INFO:tasks.workunit.client.0.smithi005.stdout: 1 866428 6.29 MB/sec execute 581 sec latency 1305.084 ms 2024-09-10T09:52:01.165 INFO:tasks.workunit.client.0.smithi005.stdout: 1 866428 6.28 MB/sec execute 582 sec latency 2305.153 ms 2024-09-10T09:52:01.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:52:01 smithi159 ceph-mon[35513]: pgmap v803: 129 pgs: 129 active+clean; 291 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.3 MiB/s wr, 137 op/s 2024-09-10T09:52:01.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:52:01 smithi102 ceph-mon[35442]: pgmap v803: 129 pgs: 129 active+clean; 291 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.3 MiB/s wr, 137 op/s 2024-09-10T09:52:01.434 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:52:01 smithi005 ceph-mon[30350]: pgmap v803: 129 pgs: 129 active+clean; 291 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.3 MiB/s wr, 137 op/s 2024-09-10T09:52:02.165 INFO:tasks.workunit.client.0.smithi005.stdout: 1 866428 6.26 MB/sec execute 583 sec latency 3305.229 ms 2024-09-10T09:52:03.165 INFO:tasks.workunit.client.0.smithi005.stdout: 1 866428 6.25 MB/sec execute 584 sec latency 4305.376 ms 2024-09-10T09:52:03.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:52:03 smithi159 ceph-mon[35513]: pgmap v804: 129 pgs: 129 active+clean; 291 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.9 MiB/s wr, 122 op/s 2024-09-10T09:52:03.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:52:03 smithi102 ceph-mon[35442]: pgmap v804: 129 pgs: 129 active+clean; 291 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.9 MiB/s wr, 122 op/s 2024-09-10T09:52:03.510 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T09:52:03.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:52:03 smithi005 ceph-mon[30350]: pgmap v804: 129 pgs: 129 active+clean; 291 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.9 MiB/s wr, 122 op/s 2024-09-10T09:52:03.892 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:52:04.165 INFO:tasks.workunit.client.0.smithi005.stdout: 1 866428 6.24 MB/sec execute 585 sec latency 5305.425 ms 2024-09-10T09:52:04.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:04.446+0000 7fa6df600640 1 -- 172.21.15.5:0/1752123178 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa6d8109140 msgr2=0x7fa6d8110cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:52:04.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:04.446+0000 7fa6df600640 1 --2- 172.21.15.5:0/1752123178 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa6d8109140 0x7fa6d8110cc0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fa6d0009ca0 tx=0x7fa6d002a8c0 comp rx=0 tx=0).stop 2024-09-10T09:52:04.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:04.446+0000 7fa6df600640 1 -- 172.21.15.5:0/1752123178 shutdown_connections 2024-09-10T09:52:04.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:04.446+0000 7fa6df600640 1 --2- 172.21.15.5:0/1752123178 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa6d8109140 0x7fa6d8110cc0 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:04.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:04.446+0000 7fa6df600640 1 --2- 172.21.15.5:0/1752123178 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa6d81049c0 0x7fa6d8108a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:04.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:04.446+0000 7fa6df600640 1 --2- 172.21.15.5:0/1752123178 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa6d8104010 0x7fa6d81043f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:04.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:04.446+0000 7fa6df600640 1 -- 172.21.15.5:0/1752123178 >> 172.21.15.5:0/1752123178 conn(0x7fa6d80ffa50 msgr2=0x7fa6d8101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:52:04.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:04.446+0000 7fa6df600640 1 -- 172.21.15.5:0/1752123178 shutdown_connections 2024-09-10T09:52:04.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:04.446+0000 7fa6df600640 1 -- 172.21.15.5:0/1752123178 wait complete. 2024-09-10T09:52:04.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:04.447+0000 7fa6df600640 1 Processor -- start 2024-09-10T09:52:04.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:04.447+0000 7fa6df600640 1 -- start start 2024-09-10T09:52:04.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:04.447+0000 7fa6df600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa6d8104010 0x7fa6d819a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:52:04.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:04.447+0000 7fa6df600640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa6d81049c0 0x7fa6d819ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:52:04.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:04.448+0000 7fa6df600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa6d8109140 0x7fa6d819ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:52:04.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:04.448+0000 7fa6df600640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa6d8114170 con 0x7fa6d8109140 2024-09-10T09:52:04.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:04.448+0000 7fa6df600640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fa6d8113ff0 con 0x7fa6d8104010 2024-09-10T09:52:04.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:04.448+0000 7fa6df600640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fa6d81142f0 con 0x7fa6d81049c0 2024-09-10T09:52:04.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:04.448+0000 7fa6dd600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa6d8109140 0x7fa6d819ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:52:04.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:04.448+0000 7fa6dd600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa6d8109140 0x7fa6d819ef00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57144/0 (socket says 172.21.15.5:57144) 2024-09-10T09:52:04.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:04.448+0000 7fa6dd600640 1 -- 172.21.15.5:0/1321145620 learned_addr learned my addr 172.21.15.5:0/1321145620 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:52:04.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:04.448+0000 7fa6dcc00640 1 --2- 172.21.15.5:0/1321145620 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa6d8104010 0x7fa6d819a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:52:04.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:04.448+0000 7fa6d7e00640 1 --2- 172.21.15.5:0/1321145620 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa6d81049c0 0x7fa6d819ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:52:04.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:04.448+0000 7fa6dd600640 1 -- 172.21.15.5:0/1321145620 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa6d81049c0 msgr2=0x7fa6d819ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:52:04.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:04.448+0000 7fa6dd600640 1 --2- 172.21.15.5:0/1321145620 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa6d81049c0 0x7fa6d819ab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:04.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:04.448+0000 7fa6dd600640 1 -- 172.21.15.5:0/1321145620 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa6d8104010 msgr2=0x7fa6d819a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:52:04.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:04.448+0000 7fa6dd600640 1 --2- 172.21.15.5:0/1321145620 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa6d8104010 0x7fa6d819a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:04.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:04.448+0000 7fa6dd600640 1 -- 172.21.15.5:0/1321145620 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6d819f5e0 con 0x7fa6d8109140 2024-09-10T09:52:04.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:04.449+0000 7fa6dd600640 1 --2- 172.21.15.5:0/1321145620 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa6d8109140 0x7fa6d819ef00 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fa6d0031a20 tx=0x7fa6d0031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:52:04.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:04.449+0000 7fa6d5600640 1 -- 172.21.15.5:0/1321145620 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6d0003b70 con 0x7fa6d8109140 2024-09-10T09:52:04.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:04.449+0000 7fa6d5600640 1 -- 172.21.15.5:0/1321145620 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa6d0003d10 con 0x7fa6d8109140 2024-09-10T09:52:04.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:04.449+0000 7fa6df600640 1 -- 172.21.15.5:0/1321145620 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6d819f870 con 0x7fa6d8109140 2024-09-10T09:52:04.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:04.449+0000 7fa6d5600640 1 -- 172.21.15.5:0/1321145620 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6d002fc40 con 0x7fa6d8109140 2024-09-10T09:52:04.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:04.449+0000 7fa6df600640 1 -- 172.21.15.5:0/1321145620 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6d81a71b0 con 0x7fa6d8109140 2024-09-10T09:52:04.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:04.452+0000 7fa6d5600640 1 -- 172.21.15.5:0/1321145620 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7fa6d0004050 con 0x7fa6d8109140 2024-09-10T09:52:04.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:04.452+0000 7fa6e21d1740 1 Processor -- start 2024-09-10T09:52:04.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:04.452+0000 7fa6e21d1740 1 -- start start 2024-09-10T09:52:04.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:04.453+0000 7fa6e21d1740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d7c06b8b90 0x55d7c06b8f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:52:04.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:04.453+0000 7fa6e21d1740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d7c06b94b0 0x55d7c06c3830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:52:04.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:04.453+0000 7fa6e21d1740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d7c06c3d70 0x55d7c06c6160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:52:04.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:04.453+0000 7fa6e21d1740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55d7c061f3e0 con 0x55d7c06b8b90 2024-09-10T09:52:04.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:04.453+0000 7fa6e21d1740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55d7c05c6d80 con 0x55d7c06c3d70 2024-09-10T09:52:04.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:04.453+0000 7fa6e21d1740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55d7c061b3a0 con 0x55d7c06b94b0 2024-09-10T09:52:04.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:04.453+0000 7fa6d7e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d7c06b8b90 0x55d7c06b8f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:52:04.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:04.453+0000 7fa6dcc00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d7c06b94b0 0x55d7c06c3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:52:04.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:04.453+0000 7fa6d7e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d7c06b8b90 0x55d7c06b8f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57152/0 (socket says 172.21.15.5:57152) 2024-09-10T09:52:04.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:04.453+0000 7fa6dcc00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d7c06b94b0 0x55d7c06c3830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:45276/0 (socket says 172.21.15.5:45276) 2024-09-10T09:52:04.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:04.453+0000 7fa6d7e00640 1 -- 172.21.15.5:0/754231881 learned_addr learned my addr 172.21.15.5:0/754231881 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:52:04.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:04.453+0000 7fa6d7e00640 1 --2- 172.21.15.5:0/754231881 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d7c06c3d70 0x55d7c06c6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:52:04.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:04.453+0000 7fa6dcc00640 1 -- 172.21.15.5:0/754231881 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d7c06c3d70 msgr2=0x55d7c06c6160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:52:04.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:04.453+0000 7fa6dcc00640 1 --2- 172.21.15.5:0/754231881 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d7c06c3d70 0x55d7c06c6160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:04.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:04.453+0000 7fa6dcc00640 1 -- 172.21.15.5:0/754231881 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d7c06b8b90 msgr2=0x55d7c06b8f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:52:04.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:04.453+0000 7fa6dcc00640 1 --2- 172.21.15.5:0/754231881 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d7c06b8b90 0x55d7c06b8f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:04.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:04.453+0000 7fa6dcc00640 1 -- 172.21.15.5:0/754231881 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d7c05e5ae0 con 0x55d7c06b94b0 2024-09-10T09:52:04.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:04.454+0000 7fa6d5600640 1 --2- 172.21.15.5:0/1321145620 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fa6c00a2b80 0x7fa6c00a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:52:04.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:04.454+0000 7fa6dcc00640 1 --2- 172.21.15.5:0/754231881 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d7c06b94b0 0x55d7c06c3830 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fa6cc009c20 tx=0x7fa6cc011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d26999985c22d7e7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:52:04.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:04.454+0000 7fa6d5600640 1 -- 172.21.15.5:0/1321145620 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fa6d00f0580 con 0x7fa6d8109140 2024-09-10T09:52:04.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:04.454+0000 7fa6c9c00640 1 -- 172.21.15.5:0/754231881 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6cc01e070 con 0x55d7c06b94b0 2024-09-10T09:52:04.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:04.454+0000 7fa6c9c00640 1 -- 172.21.15.5:0/754231881 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa6cc009dd0 con 0x55d7c06b94b0 2024-09-10T09:52:04.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:04.454+0000 7fa6d7e00640 1 --2- 172.21.15.5:0/1321145620 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fa6c00a2b80 0x7fa6c00a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:52:04.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:04.454+0000 7fa6c9c00640 1 -- 172.21.15.5:0/754231881 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6cc018810 con 0x55d7c06b94b0 2024-09-10T09:52:04.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:04.455+0000 7fa6e21d1740 1 -- 172.21.15.5:0/754231881 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d7c06b94b0 msgr2=0x55d7c06c3830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:52:04.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:04.455+0000 7fa6d7e00640 1 --2- 172.21.15.5:0/1321145620 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fa6c00a2b80 0x7fa6c00a5040 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fa6c4005f60 tx=0x7fa6c4015040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:52:04.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:04.455+0000 7fa6e21d1740 1 --2- 172.21.15.5:0/754231881 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d7c06b94b0 0x55d7c06c3830 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fa6cc009c20 tx=0x7fa6cc011810 comp rx=0 tx=0).stop 2024-09-10T09:52:04.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:04.455+0000 7fa6e21d1740 1 -- 172.21.15.5:0/754231881 shutdown_connections 2024-09-10T09:52:04.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:04.455+0000 7fa6e21d1740 1 --2- 172.21.15.5:0/754231881 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d7c06c3d70 0x55d7c06c6160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:04.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:04.455+0000 7fa6e21d1740 1 --2- 172.21.15.5:0/754231881 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d7c06b94b0 0x55d7c06c3830 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:04.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:04.455+0000 7fa6e21d1740 1 --2- 172.21.15.5:0/754231881 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d7c06b8b90 0x55d7c06b8f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:04.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:04.455+0000 7fa6e21d1740 1 -- 172.21.15.5:0/754231881 >> 172.21.15.5:0/754231881 conn(0x55d7c06b5120 msgr2=0x55d7c06b5500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:52:04.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:04.455+0000 7fa6e21d1740 1 -- 172.21.15.5:0/754231881 shutdown_connections 2024-09-10T09:52:04.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:04.455+0000 7fa6e21d1740 1 -- 172.21.15.5:0/754231881 wait complete. 2024-09-10T09:52:04.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:04.456+0000 7fa6e21d1740 1 -- start start 2024-09-10T09:52:04.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:04.456+0000 7fa6e21d1740 1 Processor -- start 2024-09-10T09:52:04.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:04.456+0000 7fa6e21d1740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d7c06b8b90 0x55d7c06c0c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:52:04.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:04.456+0000 7fa6e21d1740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d7c06b94b0 0x55d7c06c1150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:52:04.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:04.456+0000 7fa6e21d1740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d7c06c3d70 0x55d7c06c1690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:52:04.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:04.456+0000 7fa6e21d1740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55d7c06c69e0 con 0x55d7c06b8b90 2024-09-10T09:52:04.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:04.457+0000 7fa6e21d1740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55d7c05da0f0 con 0x55d7c06c3d70 2024-09-10T09:52:04.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:04.457+0000 7fa6e21d1740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55d7c06c6b60 con 0x55d7c06b94b0 2024-09-10T09:52:04.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:04.457+0000 7fa6e21d1740 10 client.0 _ll_register_callbacks cb 0x55d7c05cb9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:52:04.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:04.457+0000 7fa6dcc00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d7c06b8b90 0x55d7c06c0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:52:04.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:04.457+0000 7fa6dcc00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d7c06b8b90 0x55d7c06c0c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57160/0 (socket says 172.21.15.5:57160) 2024-09-10T09:52:04.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:04.457+0000 7fa6dcc00640 1 -- 172.21.15.5:0/4073170328 learned_addr learned my addr 172.21.15.5:0/4073170328 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:52:04.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:04.457+0000 7fa6d7e00640 1 --2- 172.21.15.5:0/4073170328 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d7c06b94b0 0x55d7c06c1150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:52:04.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:04.457+0000 7fa6dd600640 1 --2- 172.21.15.5:0/4073170328 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d7c06c3d70 0x55d7c06c1690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:52:04.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:04.457+0000 7fa6dcc00640 1 -- 172.21.15.5:0/4073170328 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d7c06b94b0 msgr2=0x55d7c06c1150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:52:04.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:04.457+0000 7fa6dcc00640 1 --2- 172.21.15.5:0/4073170328 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d7c06b94b0 0x55d7c06c1150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:04.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:04.457+0000 7fa6dcc00640 1 -- 172.21.15.5:0/4073170328 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d7c06c3d70 msgr2=0x55d7c06c1690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:52:04.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:04.457+0000 7fa6dcc00640 1 --2- 172.21.15.5:0/4073170328 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d7c06c3d70 0x55d7c06c1690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:04.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:04.457+0000 7fa6dcc00640 1 -- 172.21.15.5:0/4073170328 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d7c03903c0 con 0x55d7c06b8b90 2024-09-10T09:52:04.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:04.458+0000 7fa6dcc00640 1 --2- 172.21.15.5:0/4073170328 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d7c06b8b90 0x55d7c06c0c10 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fa6cc01c040 tx=0x7fa6cc005730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1113803dea431f17 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:52:04.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:04.458+0000 7fa6be000640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-10T09:52:04.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:04.458+0000 7fa6be000640 1 -- 172.21.15.5:0/4073170328 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6cc01e040 con 0x55d7c06b8b90 2024-09-10T09:52:04.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:04.458+0000 7fa6be000640 1 -- 172.21.15.5:0/4073170328 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fa6cc0035f0 con 0x55d7c06b8b90 2024-09-10T09:52:04.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:04.458+0000 7fa6e21d1740 1 -- 172.21.15.5:0/4073170328 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa6cc009dd0 con 0x55d7c06b8b90 2024-09-10T09:52:04.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:04.458+0000 7fa6be000640 1 -- 172.21.15.5:0/4073170328 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fa6cc032050 con 0x55d7c06b8b90 2024-09-10T09:52:04.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:04.458+0000 7fa6e21d1740 10 client.18129 fetch_fsmap learned FSMap version 36 2024-09-10T09:52:04.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:04.458+0000 7fa6e21d1740 1 -- 172.21.15.5:0/4073170328 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55d7c06d2640 con 0x55d7c06b8b90 2024-09-10T09:52:04.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:04.458+0000 7fa6be000640 1 -- 172.21.15.5:0/4073170328 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa6cc009dd0 con 0x55d7c06b8b90 2024-09-10T09:52:04.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:04.459+0000 7fa6be000640 1 -- 172.21.15.5:0/4073170328 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 36) ==== 8370+0+0 (secure 0 0 0) 0x7fa6cc021030 con 0x55d7c06b8b90 2024-09-10T09:52:04.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:04.459+0000 7fa6e21d1740 10 client.18129 fetch_fsmap finished waiting for FSMap version 36 2024-09-10T09:52:04.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:04.459+0000 7fa6e21d1740 10 client.18129 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T09:52:04.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:04.459+0000 7fa6e21d1740 20 client.18129 populate_metadata read hostname 'smithi005' 2024-09-10T09:52:04.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:04.459+0000 7fa6e21d1740 1 --2- 172.21.15.5:0/4073170328 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55d7c06d39f0 0x55d7c06d5de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:52:04.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:04.459+0000 7fa6e21d1740 4 client.18129 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:52:04.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:04.459+0000 7fa6e21d1740 1 -- 172.21.15.5:0/4073170328 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d7c062c950 con 0x55d7c06d39f0 2024-09-10T09:52:04.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:04.459+0000 7fa6d7e00640 1 --2- 172.21.15.5:0/4073170328 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55d7c06d39f0 0x55d7c06d5de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:52:04.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:04.460+0000 7fa6d7e00640 1 --2- 172.21.15.5:0/4073170328 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55d7c06d39f0 0x55d7c06d5de0 crc :-1 s=READY pgs=209 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=50238740d87cb497 server_cookie=18ab9cde72e18ec6 in_seq=0 out_seq=0 2024-09-10T09:52:04.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:04.460+0000 7fa6be000640 10 client.18129 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T09:52:04.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:04.460+0000 7fa6be000640 1 -- 172.21.15.5:0/4073170328 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d7c062c950 con 0x55d7c06d39f0 2024-09-10T09:52:04.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:04.461+0000 7fa6be000640 10 client.18129 handle_command_reply: tid=0 2024-09-10T09:52:04.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:04.461+0000 7fa6e21d1740 1 client.18129 shutdown 2024-09-10T09:52:04.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:04.461+0000 7fa6e21d1740 1 -- 172.21.15.5:0/4073170328 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d7c06b8b90 msgr2=0x55d7c06c0c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:52:04.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:04.461+0000 7fa6e21d1740 1 --2- 172.21.15.5:0/4073170328 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d7c06b8b90 0x55d7c06c0c10 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fa6cc01c040 tx=0x7fa6cc005730 comp rx=0 tx=0).stop 2024-09-10T09:52:04.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:04.461+0000 7fa6e21d1740 1 -- 172.21.15.5:0/4073170328 shutdown_connections 2024-09-10T09:52:04.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:04.461+0000 7fa6e21d1740 1 --2- 172.21.15.5:0/4073170328 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55d7c06d39f0 0x55d7c06d5de0 crc :-1 s=READY pgs=209 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:04.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:04.461+0000 7fa6e21d1740 1 --2- 172.21.15.5:0/4073170328 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d7c06c3d70 0x55d7c06c1690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:04.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:04.461+0000 7fa6e21d1740 1 --2- 172.21.15.5:0/4073170328 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d7c06b94b0 0x55d7c06c1150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:04.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:04.461+0000 7fa6e21d1740 1 --2- 172.21.15.5:0/4073170328 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d7c06b8b90 0x55d7c06c0c10 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:04.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:04.461+0000 7fa6e21d1740 1 -- 172.21.15.5:0/4073170328 >> 172.21.15.5:0/4073170328 conn(0x55d7c06b5120 msgr2=0x55d7c06bae50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:52:04.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:04.461+0000 7fa6be000640 0 client.18129 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T09:52:04.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:04.461+0000 7fa6e21d1740 1 -- 172.21.15.5:0/4073170328 shutdown_connections 2024-09-10T09:52:04.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:04.461+0000 7fa6e21d1740 1 -- 172.21.15.5:0/4073170328 wait complete. 2024-09-10T09:52:04.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:04.461+0000 7fa6e21d1740 20 client.18129 trim_cache size 0 max 16384 2024-09-10T09:52:04.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:04.479+0000 7fa6e21d1740 1 Processor -- start 2024-09-10T09:52:04.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:04.480+0000 7fa6e21d1740 1 -- start start 2024-09-10T09:52:04.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:04.480+0000 7fa6e21d1740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d7c06b8b90 0x55d7c06bb5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:52:04.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:04.480+0000 7fa6e21d1740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d7c06b94b0 0x55d7c06db5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:52:04.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:04.480+0000 7fa6e21d1740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d7c06c3d70 0x55d7c06dbb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:52:04.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:04.480+0000 7fa6e21d1740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa6d81b1090 con 0x55d7c06b8b90 2024-09-10T09:52:04.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:04.480+0000 7fa6e21d1740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fa6d81b2bd0 con 0x55d7c06c3d70 2024-09-10T09:52:04.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:04.480+0000 7fa6e21d1740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fa6d81b1760 con 0x55d7c06b94b0 2024-09-10T09:52:04.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:04.480+0000 7fa6d7e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d7c06b94b0 0x55d7c06db5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:52:04.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:04.480+0000 7fa6d7e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d7c06b94b0 0x55d7c06db5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:45296/0 (socket says 172.21.15.5:45296) 2024-09-10T09:52:04.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:04.480+0000 7fa6d7e00640 1 -- 172.21.15.5:0/3815806155 learned_addr learned my addr 172.21.15.5:0/3815806155 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:52:04.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:04.480+0000 7fa6dcc00640 1 --2- 172.21.15.5:0/3815806155 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d7c06b8b90 0x55d7c06bb5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:52:04.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:04.480+0000 7fa6dd600640 1 --2- 172.21.15.5:0/3815806155 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d7c06c3d70 0x55d7c06dbb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:52:04.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:04.481+0000 7fa6d7e00640 1 -- 172.21.15.5:0/3815806155 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d7c06c3d70 msgr2=0x55d7c06dbb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:52:04.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:04.481+0000 7fa6d7e00640 1 --2- 172.21.15.5:0/3815806155 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d7c06c3d70 0x55d7c06dbb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:04.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:04.481+0000 7fa6d7e00640 1 -- 172.21.15.5:0/3815806155 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d7c06b8b90 msgr2=0x55d7c06bb5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:52:04.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:04.481+0000 7fa6d7e00640 1 --2- 172.21.15.5:0/3815806155 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d7c06b8b90 0x55d7c06bb5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:04.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:04.481+0000 7fa6d7e00640 1 -- 172.21.15.5:0/3815806155 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d7c06b4f50 con 0x55d7c06b94b0 2024-09-10T09:52:04.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:04.481+0000 7fa6d7e00640 1 --2- 172.21.15.5:0/3815806155 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d7c06b94b0 0x55d7c06db5d0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fa6c4009a30 tx=0x7fa6c4004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ee0eca8b61efb526 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:52:04.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:04.481+0000 7fa6c9200640 1 -- 172.21.15.5:0/3815806155 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6c402e6c0 con 0x55d7c06b94b0 2024-09-10T09:52:04.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:04.481+0000 7fa6c9200640 1 -- 172.21.15.5:0/3815806155 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa6c4020400 con 0x55d7c06b94b0 2024-09-10T09:52:04.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:04.481+0000 7fa6c9200640 1 -- 172.21.15.5:0/3815806155 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6c401f5f0 con 0x55d7c06b94b0 2024-09-10T09:52:04.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:04.482+0000 7fa6e21d1740 1 -- 172.21.15.5:0/3815806155 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d7c06b94b0 msgr2=0x55d7c06db5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:52:04.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:04.482+0000 7fa6e21d1740 1 --2- 172.21.15.5:0/3815806155 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d7c06b94b0 0x55d7c06db5d0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fa6c4009a30 tx=0x7fa6c4004730 comp rx=0 tx=0).stop 2024-09-10T09:52:04.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:04.482+0000 7fa6e21d1740 1 -- 172.21.15.5:0/3815806155 shutdown_connections 2024-09-10T09:52:04.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:04.482+0000 7fa6e21d1740 1 --2- 172.21.15.5:0/3815806155 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d7c06c3d70 0x55d7c06dbb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:04.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:04.482+0000 7fa6e21d1740 1 --2- 172.21.15.5:0/3815806155 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d7c06b94b0 0x55d7c06db5d0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:04.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:04.482+0000 7fa6e21d1740 1 --2- 172.21.15.5:0/3815806155 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d7c06b8b90 0x55d7c06bb5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:04.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:04.482+0000 7fa6e21d1740 1 -- 172.21.15.5:0/3815806155 >> 172.21.15.5:0/3815806155 conn(0x55d7c06b5120 msgr2=0x55d7c06bfd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:52:04.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:04.482+0000 7fa6e21d1740 1 -- 172.21.15.5:0/3815806155 shutdown_connections 2024-09-10T09:52:04.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:04.482+0000 7fa6e21d1740 1 -- 172.21.15.5:0/3815806155 wait complete. 2024-09-10T09:52:04.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:04.483+0000 7fa6e21d1740 1 -- start start 2024-09-10T09:52:04.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:04.483+0000 7fa6e21d1740 1 Processor -- start 2024-09-10T09:52:04.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:04.483+0000 7fa6e21d1740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d7c06b8b90 0x55d7c06c7330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:52:04.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:04.483+0000 7fa6e21d1740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d7c06b94b0 0x55d7c06c7870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:52:04.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:04.483+0000 7fa6e21d1740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d7c06c3d70 0x55d7c073c590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:52:04.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:04.483+0000 7fa6e21d1740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa6d81b2500 con 0x55d7c06b94b0 2024-09-10T09:52:04.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:04.483+0000 7fa6e21d1740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fa6d81b1e30 con 0x55d7c06b8b90 2024-09-10T09:52:04.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:04.483+0000 7fa6e21d1740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fa6d81af070 con 0x55d7c06c3d70 2024-09-10T09:52:04.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:04.483+0000 7fa6e21d1740 10 client.0 _ll_register_callbacks cb 0x7fa6cc009a70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:52:04.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:04.484+0000 7fa6dd600640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d7c06c3d70 0x55d7c073c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:52:04.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:04.484+0000 7fa6d7e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d7c06b94b0 0x55d7c06c7870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:52:04.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:04.484+0000 7fa6dd600640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d7c06c3d70 0x55d7c073c590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:45304/0 (socket says 172.21.15.5:45304) 2024-09-10T09:52:04.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:04.484+0000 7fa6dd600640 1 -- 172.21.15.5:0/2443242230 learned_addr learned my addr 172.21.15.5:0/2443242230 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:52:04.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:04.484+0000 7fa6dcc00640 1 --2- 172.21.15.5:0/2443242230 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d7c06b8b90 0x55d7c06c7330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:52:04.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:04.484+0000 7fa6dd600640 1 -- 172.21.15.5:0/2443242230 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d7c06b8b90 msgr2=0x55d7c06c7330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:52:04.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:04.484+0000 7fa6dd600640 1 --2- 172.21.15.5:0/2443242230 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d7c06b8b90 0x55d7c06c7330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:04.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:04.484+0000 7fa6dd600640 1 -- 172.21.15.5:0/2443242230 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d7c06b94b0 msgr2=0x55d7c06c7870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:52:04.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:04.484+0000 7fa6dd600640 1 --2- 172.21.15.5:0/2443242230 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d7c06b94b0 0x55d7c06c7870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:04.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:04.484+0000 7fa6dd600640 1 -- 172.21.15.5:0/2443242230 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d7c073cad0 con 0x55d7c06c3d70 2024-09-10T09:52:04.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:04.484+0000 7fa6d7e00640 1 --2- 172.21.15.5:0/2443242230 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d7c06b94b0 0x55d7c06c7870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:52:04.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:04.484+0000 7fa6dd600640 1 --2- 172.21.15.5:0/2443242230 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d7c06c3d70 0x55d7c073c590 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fa6d0006010 tx=0x7fa6d00bc380 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f597d3ce2f49b33f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:52:04.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:04.484+0000 7fa6be000640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-09-10T09:52:04.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:04.484+0000 7fa6be000640 1 -- 172.21.15.5:0/2443242230 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6d00c0dd0 con 0x55d7c06c3d70 2024-09-10T09:52:04.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:04.485+0000 7fa6e21d1740 1 -- 172.21.15.5:0/2443242230 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa6c4020400 con 0x55d7c06c3d70 2024-09-10T09:52:04.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:04.485+0000 7fa6be000640 1 -- 172.21.15.5:0/2443242230 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fa6d002fde0 con 0x55d7c06c3d70 2024-09-10T09:52:04.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:04.485+0000 7fa6be000640 1 -- 172.21.15.5:0/2443242230 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fa6d00c5600 con 0x55d7c06c3d70 2024-09-10T09:52:04.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:04.485+0000 7fa6e21d1740 10 client.28165 fetch_fsmap learned FSMap version 36 2024-09-10T09:52:04.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:04.485+0000 7fa6e21d1740 1 -- 172.21.15.5:0/2443242230 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55d7c073de00 con 0x55d7c06c3d70 2024-09-10T09:52:04.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:04.485+0000 7fa6be000640 1 -- 172.21.15.5:0/2443242230 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa6c4020400 con 0x55d7c06c3d70 2024-09-10T09:52:04.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:04.485+0000 7fa6be000640 1 -- 172.21.15.5:0/2443242230 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 36) ==== 8370+0+0 (secure 0 0 0) 0x7fa6d00c30c0 con 0x55d7c06c3d70 2024-09-10T09:52:04.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:04.485+0000 7fa6e21d1740 10 client.28165 fetch_fsmap finished waiting for FSMap version 36 2024-09-10T09:52:04.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:04.485+0000 7fa6e21d1740 10 client.28165 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T09:52:04.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:04.485+0000 7fa6e21d1740 20 client.28165 populate_metadata read hostname 'smithi005' 2024-09-10T09:52:04.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:04.485+0000 7fa6e21d1740 1 --2- 172.21.15.5:0/2443242230 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55d7c073e500 0x55d7c0740920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:52:04.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:04.485+0000 7fa6e21d1740 4 client.28165 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T09:52:04.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:04.485+0000 7fa6e21d1740 1 -- 172.21.15.5:0/2443242230 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55d7c0626e60 con 0x55d7c073e500 2024-09-10T09:52:04.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:04.486+0000 7fa6dcc00640 1 --2- 172.21.15.5:0/2443242230 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55d7c073e500 0x55d7c0740920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:52:04.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:04.486+0000 7fa6dcc00640 1 --2- 172.21.15.5:0/2443242230 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55d7c073e500 0x55d7c0740920 crc :-1 s=READY pgs=210 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=92931554a270c9d1 server_cookie=cceae95271500c08 in_seq=0 out_seq=0 2024-09-10T09:52:04.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:04.486+0000 7fa6be000640 10 client.28165 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T09:52:04.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:04.486+0000 7fa6be000640 1 -- 172.21.15.5:0/2443242230 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55d7c0626e60 con 0x55d7c073e500 2024-09-10T09:52:04.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:04.487+0000 7fa6be000640 10 client.28165 handle_command_reply: tid=0 2024-09-10T09:52:04.508 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-10T09:52:04.509 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "no active scrubs running", 2024-09-10T09:52:04.509 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": {} 2024-09-10T09:52:04.509 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-10T09:52:04.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:04.487+0000 7fa6e21d1740 1 client.28165 shutdown 2024-09-10T09:52:04.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:04.487+0000 7fa6e21d1740 1 -- 172.21.15.5:0/2443242230 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d7c06c3d70 msgr2=0x55d7c073c590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:52:04.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:04.487+0000 7fa6e21d1740 1 --2- 172.21.15.5:0/2443242230 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d7c06c3d70 0x55d7c073c590 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fa6d0006010 tx=0x7fa6d00bc380 comp rx=0 tx=0).stop 2024-09-10T09:52:04.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:04.487+0000 7fa6e21d1740 1 -- 172.21.15.5:0/2443242230 shutdown_connections 2024-09-10T09:52:04.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:04.487+0000 7fa6e21d1740 1 --2- 172.21.15.5:0/2443242230 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55d7c073e500 0x55d7c0740920 crc :-1 s=READY pgs=210 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:04.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:04.487+0000 7fa6e21d1740 1 --2- 172.21.15.5:0/2443242230 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d7c06c3d70 0x55d7c073c590 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:04.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:04.487+0000 7fa6e21d1740 1 --2- 172.21.15.5:0/2443242230 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d7c06b94b0 0x55d7c06c7870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:04.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:04.487+0000 7fa6be000640 0 client.28165 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T09:52:04.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:04.487+0000 7fa6e21d1740 1 --2- 172.21.15.5:0/2443242230 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d7c06b8b90 0x55d7c06c7330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:04.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:04.487+0000 7fa6e21d1740 1 -- 172.21.15.5:0/2443242230 >> 172.21.15.5:0/2443242230 conn(0x55d7c06b5120 msgr2=0x55d7c06b39a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:52:04.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:04.487+0000 7fa6e21d1740 1 -- 172.21.15.5:0/2443242230 shutdown_connections 2024-09-10T09:52:04.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:04.487+0000 7fa6e21d1740 1 -- 172.21.15.5:0/2443242230 wait complete. 2024-09-10T09:52:04.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:04.487+0000 7fa6e21d1740 20 client.28165 trim_cache size 0 max 16384 2024-09-10T09:52:04.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:04.488+0000 7fa6df600640 1 -- 172.21.15.5:0/1321145620 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fa6c00a2b80 msgr2=0x7fa6c00a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:52:04.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:04.488+0000 7fa6df600640 1 --2- 172.21.15.5:0/1321145620 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fa6c00a2b80 0x7fa6c00a5040 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fa6c4005f60 tx=0x7fa6c4015040 comp rx=0 tx=0).stop 2024-09-10T09:52:04.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:04.488+0000 7fa6df600640 1 -- 172.21.15.5:0/1321145620 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa6d8109140 msgr2=0x7fa6d819ef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:52:04.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:04.488+0000 7fa6df600640 1 --2- 172.21.15.5:0/1321145620 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa6d8109140 0x7fa6d819ef00 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fa6d0031a20 tx=0x7fa6d0031a50 comp rx=0 tx=0).stop 2024-09-10T09:52:04.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:04.488+0000 7fa6df600640 1 -- 172.21.15.5:0/1321145620 shutdown_connections 2024-09-10T09:52:04.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:04.488+0000 7fa6df600640 1 --2- 172.21.15.5:0/1321145620 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fa6c00a2b80 0x7fa6c00a5040 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:04.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:04.488+0000 7fa6df600640 1 --2- 172.21.15.5:0/1321145620 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa6d8109140 0x7fa6d819ef00 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:04.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:04.488+0000 7fa6df600640 1 --2- 172.21.15.5:0/1321145620 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa6d81049c0 0x7fa6d819ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:04.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:04.488+0000 7fa6df600640 1 --2- 172.21.15.5:0/1321145620 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa6d8104010 0x7fa6d819a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:04.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:04.488+0000 7fa6df600640 1 -- 172.21.15.5:0/1321145620 >> 172.21.15.5:0/1321145620 conn(0x7fa6d80ffa50 msgr2=0x7fa6d8100d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:52:04.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:04.488+0000 7fa6df600640 1 -- 172.21.15.5:0/1321145620 shutdown_connections 2024-09-10T09:52:04.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:04.488+0000 7fa6df600640 1 -- 172.21.15.5:0/1321145620 wait complete. 2024-09-10T09:52:05.165 INFO:tasks.workunit.client.0.smithi005.stdout: 1 866428 6.23 MB/sec execute 586 sec latency 6305.561 ms 2024-09-10T09:52:05.288 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:52:05 smithi005 ceph-mon[30350]: pgmap v805: 129 pgs: 129 active+clean; 292 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.1 MiB/s wr, 124 op/s 2024-09-10T09:52:05.325 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-10T09:52:05.326 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T09:52:05.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:52:05 smithi159 ceph-mon[35513]: pgmap v805: 129 pgs: 129 active+clean; 292 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.1 MiB/s wr, 124 op/s 2024-09-10T09:52:05.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:52:05 smithi102 ceph-mon[35442]: pgmap v805: 129 pgs: 129 active+clean; 292 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.1 MiB/s wr, 124 op/s 2024-09-10T09:52:05.762 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:52:06.170 INFO:tasks.workunit.client.0.smithi005.stdout: 1 866428 6.22 MB/sec execute 587 sec latency 7310.979 ms 2024-09-10T09:52:06.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:06.802+0000 7fc684000640 1 -- 172.21.15.5:0/487144371 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc67c102790 msgr2=0x7fc67c1067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:52:06.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:06.802+0000 7fc684000640 1 --2- 172.21.15.5:0/487144371 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc67c102790 0x7fc67c1067e0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fc668009900 tx=0x7fc66802a3f0 comp rx=0 tx=0).stop 2024-09-10T09:52:06.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:06.802+0000 7fc684000640 1 -- 172.21.15.5:0/487144371 shutdown_connections 2024-09-10T09:52:06.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:06.802+0000 7fc684000640 1 --2- 172.21.15.5:0/487144371 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc67c106f10 0x7fc67c10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:06.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:06.802+0000 7fc684000640 1 --2- 172.21.15.5:0/487144371 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc67c102790 0x7fc67c1067e0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:06.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:06.802+0000 7fc684000640 1 --2- 172.21.15.5:0/487144371 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc67c101de0 0x7fc67c1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:06.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:06.802+0000 7fc684000640 1 -- 172.21.15.5:0/487144371 >> 172.21.15.5:0/487144371 conn(0x7fc67c0fd8c0 msgr2=0x7fc67c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:52:06.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:06.802+0000 7fc684000640 1 -- 172.21.15.5:0/487144371 shutdown_connections 2024-09-10T09:52:06.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:06.802+0000 7fc684000640 1 -- 172.21.15.5:0/487144371 wait complete. 2024-09-10T09:52:06.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:06.804+0000 7fc684000640 1 Processor -- start 2024-09-10T09:52:06.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:06.804+0000 7fc684000640 1 -- start start 2024-09-10T09:52:06.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:06.804+0000 7fc684000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc67c101de0 0x7fc67c0744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:52:06.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:06.804+0000 7fc684000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc67c102790 0x7fc67c074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:52:06.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:06.804+0000 7fc684000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc67c106f10 0x7fc67c074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:52:06.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:06.804+0000 7fc684000640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc67c111f30 con 0x7fc67c106f10 2024-09-10T09:52:06.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:06.804+0000 7fc684000640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fc67c111db0 con 0x7fc67c101de0 2024-09-10T09:52:06.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:06.804+0000 7fc684000640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fc67c1120b0 con 0x7fc67c102790 2024-09-10T09:52:06.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:06.805+0000 7fc680c00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc67c102790 0x7fc67c074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:52:06.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:06.805+0000 7fc680c00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc67c102790 0x7fc67c074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:45326/0 (socket says 172.21.15.5:45326) 2024-09-10T09:52:06.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:06.805+0000 7fc680c00640 1 -- 172.21.15.5:0/152196406 learned_addr learned my addr 172.21.15.5:0/152196406 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:52:06.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:06.805+0000 7fc682000640 1 --2- 172.21.15.5:0/152196406 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc67c106f10 0x7fc67c074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:52:06.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:06.805+0000 7fc681600640 1 --2- 172.21.15.5:0/152196406 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc67c101de0 0x7fc67c0744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:52:06.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:06.805+0000 7fc682000640 1 -- 172.21.15.5:0/152196406 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc67c102790 msgr2=0x7fc67c074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:52:06.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:06.805+0000 7fc682000640 1 --2- 172.21.15.5:0/152196406 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc67c102790 0x7fc67c074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:06.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:06.805+0000 7fc682000640 1 -- 172.21.15.5:0/152196406 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc67c101de0 msgr2=0x7fc67c0744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:52:06.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:06.805+0000 7fc682000640 1 --2- 172.21.15.5:0/152196406 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc67c101de0 0x7fc67c0744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:06.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:06.805+0000 7fc682000640 1 -- 172.21.15.5:0/152196406 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc67c1a68d0 con 0x7fc67c106f10 2024-09-10T09:52:06.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:06.805+0000 7fc681600640 1 --2- 172.21.15.5:0/152196406 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc67c101de0 0x7fc67c0744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:52:06.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:06.806+0000 7fc682000640 1 --2- 172.21.15.5:0/152196406 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc67c106f10 0x7fc67c074f70 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fc674009fd0 tx=0x7fc67400ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:52:06.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:06.806+0000 7fc67a000640 1 -- 172.21.15.5:0/152196406 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc67400ce20 con 0x7fc67c106f10 2024-09-10T09:52:06.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:06.806+0000 7fc67a000640 1 -- 172.21.15.5:0/152196406 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc674005020 con 0x7fc67c106f10 2024-09-10T09:52:06.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:06.806+0000 7fc684000640 1 -- 172.21.15.5:0/152196406 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc67c1a6bc0 con 0x7fc67c106f10 2024-09-10T09:52:06.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:06.806+0000 7fc67a000640 1 -- 172.21.15.5:0/152196406 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc674010720 con 0x7fc67c106f10 2024-09-10T09:52:06.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:06.806+0000 7fc684000640 1 -- 172.21.15.5:0/152196406 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc67c1a7100 con 0x7fc67c106f10 2024-09-10T09:52:06.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:06.807+0000 7fc684000640 1 -- 172.21.15.5:0/152196406 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc67c1021c0 con 0x7fc67c106f10 2024-09-10T09:52:06.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:06.809+0000 7fc67a000640 1 -- 172.21.15.5:0/152196406 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7fc674005540 con 0x7fc67c106f10 2024-09-10T09:52:06.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:06.810+0000 7fc67a000640 1 --2- 172.21.15.5:0/152196406 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fc6640a2c00 0x7fc6640a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:52:06.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:06.811+0000 7fc67a000640 1 -- 172.21.15.5:0/152196406 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fc6740cfa50 con 0x7fc67c106f10 2024-09-10T09:52:06.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:06.811+0000 7fc681600640 1 --2- 172.21.15.5:0/152196406 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fc6640a2c00 0x7fc6640a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:52:06.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:06.811+0000 7fc681600640 1 --2- 172.21.15.5:0/152196406 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fc6640a2c00 0x7fc6640a50c0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fc670009fc0 tx=0x7fc670005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:52:06.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:06.815+0000 7fc67a000640 1 -- 172.21.15.5:0/152196406 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc67409c560 con 0x7fc67c106f10 2024-09-10T09:52:07.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:07.092+0000 7fc684000640 1 -- 172.21.15.5:0/152196406 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc67c02d7c0 con 0x7fc67c106f10 2024-09-10T09:52:07.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:07.096+0000 7fc67a000640 1 -- 172.21.15.5:0/152196406 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) ==== 94+0+11969 (secure 0 0 0) 0x7fc67409be60 con 0x7fc67c106f10 2024-09-10T09:52:07.099 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-10T09:52:07.099 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":36,"btime":"2024-09-10T09:48:07:349075+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14454,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/2970594311","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2970594311},{"type":"v1","addr":"172.21.15.5:6835","nonce":2970594311}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14514,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6839/3616502174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":3616502174},{"type":"v1","addr":"172.21.15.5:6839","nonce":3616502174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":36,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T09:35:29.626287+0000","modified":"2024-09-10T09:48:06.970127+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24542,"mds_1":24535,"mds_2":24511,"mds_3":24506,"mds_4":24443},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14520":{"gid":14520,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/2073599842","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":2073599842},{"type":"v1","addr":"172.21.15.5:6841","nonce":2073599842}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"k","rank":4,"incarnation":26,"state":"up:active","state_seq":24,"addr":"172.21.15.102:6833/2457007427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":2457007427},{"type":"v1","addr":"172.21.15.102:6833","nonce":2457007427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/356445159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":356445159},{"type":"v1","addr":"172.21.15.159:6833","nonce":356445159}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6837/2894314507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":2894314507},{"type":"v1","addr":"172.21.15.5:6837","nonce":2894314507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.102:6835/1647549111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":1647549111},{"type":"v1","addr":"172.21.15.102:6835","nonce":1647549111}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6835/952657242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":952657242},{"type":"v1","addr":"172.21.15.159:6835","nonce":952657242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":19,"addr":"172.21.15.102:6837/2689776764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":2689776764},{"type":"v1","addr":"172.21.15.102:6837","nonce":2689776764}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.159:6837/620627639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":620627639},{"type":"v1","addr":"172.21.15.159:6837","nonce":620627639}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24535":{"gid":24535,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":18,"addr":"172.21.15.159:6839/2073739599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":2073739599},{"type":"v1","addr":"172.21.15.159:6839","nonce":2073739599}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24542":{"gid":24542,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.102:6839/335409749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6838","nonce":335409749},{"type":"v1","addr":"172.21.15.102:6839","nonce":335409749}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24542,"qdb_cluster":[24542,24535,24511,24506,24443]},"id":1}]} 2024-09-10T09:52:07.100 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 36 2024-09-10T09:52:07.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:07.099+0000 7fc684000640 1 -- 172.21.15.5:0/152196406 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fc6640a2c00 msgr2=0x7fc6640a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:52:07.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:07.099+0000 7fc684000640 1 --2- 172.21.15.5:0/152196406 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fc6640a2c00 0x7fc6640a50c0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fc670009fc0 tx=0x7fc670005f90 comp rx=0 tx=0).stop 2024-09-10T09:52:07.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:07.099+0000 7fc684000640 1 -- 172.21.15.5:0/152196406 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc67c106f10 msgr2=0x7fc67c074f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:52:07.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:07.099+0000 7fc684000640 1 --2- 172.21.15.5:0/152196406 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc67c106f10 0x7fc67c074f70 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fc674009fd0 tx=0x7fc67400ef10 comp rx=0 tx=0).stop 2024-09-10T09:52:07.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:07.100+0000 7fc684000640 1 -- 172.21.15.5:0/152196406 shutdown_connections 2024-09-10T09:52:07.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:07.100+0000 7fc684000640 1 --2- 172.21.15.5:0/152196406 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fc6640a2c00 0x7fc6640a50c0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:07.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:07.100+0000 7fc684000640 1 --2- 172.21.15.5:0/152196406 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc67c106f10 0x7fc67c074f70 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:07.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:07.100+0000 7fc684000640 1 --2- 172.21.15.5:0/152196406 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc67c102790 0x7fc67c074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:07.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:07.100+0000 7fc684000640 1 --2- 172.21.15.5:0/152196406 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc67c101de0 0x7fc67c0744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:07.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:07.100+0000 7fc684000640 1 -- 172.21.15.5:0/152196406 >> 172.21.15.5:0/152196406 conn(0x7fc67c0fd8c0 msgr2=0x7fc67c0fec80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:52:07.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:07.100+0000 7fc684000640 1 -- 172.21.15.5:0/152196406 shutdown_connections 2024-09-10T09:52:07.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:07.100+0000 7fc684000640 1 -- 172.21.15.5:0/152196406 wait complete. 2024-09-10T09:52:07.170 INFO:tasks.workunit.client.0.smithi005.stdout: 1 866428 6.21 MB/sec execute 588 sec latency 8311.088 ms 2024-09-10T09:52:07.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:52:07 smithi159 ceph-mon[35513]: pgmap v806: 129 pgs: 129 active+clean; 292 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.0 MiB/s wr, 122 op/s 2024-09-10T09:52:07.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:52:07 smithi102 ceph-mon[35442]: pgmap v806: 129 pgs: 129 active+clean; 292 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.0 MiB/s wr, 122 op/s 2024-09-10T09:52:07.483 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:52:07 smithi005 ceph-mon[30350]: pgmap v806: 129 pgs: 129 active+clean; 292 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.0 MiB/s wr, 122 op/s 2024-09-10T09:52:07.913 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-10T09:52:08.171 INFO:tasks.workunit.client.0.smithi005.stdout: 1 866428 6.20 MB/sec execute 589 sec latency 9311.148 ms 2024-09-10T09:52:08.317 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:52:08.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:52:08 smithi102 ceph-mon[35442]: from='client.? 172.21.15.5:0/152196406' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:52:08.457 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:52:08 smithi005 ceph-mon[30350]: from='client.? 172.21.15.5:0/152196406' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:52:08.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:52:08 smithi159 ceph-mon[35513]: from='client.? 172.21.15.5:0/152196406' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:52:08.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:08.925+0000 7f3786400640 1 -- 172.21.15.5:0/4215769912 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3780109120 msgr2=0x7f3780110ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:52:08.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:08.925+0000 7f3786400640 1 --2- 172.21.15.5:0/4215769912 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3780109120 0x7f3780110ca0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f3778009ca0 tx=0x7f377802a8c0 comp rx=0 tx=0).stop 2024-09-10T09:52:08.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:08.925+0000 7f3786400640 1 -- 172.21.15.5:0/4215769912 shutdown_connections 2024-09-10T09:52:08.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:08.925+0000 7f3786400640 1 --2- 172.21.15.5:0/4215769912 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3780109120 0x7f3780110ca0 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:08.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:08.925+0000 7f3786400640 1 --2- 172.21.15.5:0/4215769912 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f37801049a0 0x7f37801089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:08.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:08.925+0000 7f3786400640 1 --2- 172.21.15.5:0/4215769912 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3780103ff0 0x7f37801043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:08.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:08.925+0000 7f3786400640 1 -- 172.21.15.5:0/4215769912 >> 172.21.15.5:0/4215769912 conn(0x7f37800ffa90 msgr2=0x7f3780101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:52:08.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:08.925+0000 7f3786400640 1 -- 172.21.15.5:0/4215769912 shutdown_connections 2024-09-10T09:52:08.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:08.925+0000 7f3786400640 1 -- 172.21.15.5:0/4215769912 wait complete. 2024-09-10T09:52:08.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:08.926+0000 7f3786400640 1 Processor -- start 2024-09-10T09:52:08.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:08.926+0000 7f3786400640 1 -- start start 2024-09-10T09:52:08.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:08.927+0000 7f3786400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3780103ff0 0x7f378019a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:52:08.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:08.927+0000 7f3786400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f37801049a0 0x7f378019aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:52:08.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:08.927+0000 7f3786400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3780109120 0x7f378019ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:52:08.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:08.927+0000 7f3786400640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f3780114150 con 0x7f37801049a0 2024-09-10T09:52:08.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:08.927+0000 7f3786400640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f3780113fd0 con 0x7f3780109120 2024-09-10T09:52:08.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:08.927+0000 7f3786400640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f37801142d0 con 0x7f3780103ff0 2024-09-10T09:52:08.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:08.927+0000 7f377ea00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f37801049a0 0x7f378019aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:52:08.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:08.927+0000 7f377ea00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f37801049a0 0x7f378019aaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57236/0 (socket says 172.21.15.5:57236) 2024-09-10T09:52:08.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:08.927+0000 7f377fe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3780109120 0x7f378019ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:52:08.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:08.927+0000 7f377ea00640 1 -- 172.21.15.5:0/1218056431 learned_addr learned my addr 172.21.15.5:0/1218056431 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:52:08.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:08.927+0000 7f377f400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3780103ff0 0x7f378019a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:52:08.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:08.927+0000 7f377fe00640 1 -- 172.21.15.5:0/1218056431 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3780103ff0 msgr2=0x7f378019a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:52:08.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:08.927+0000 7f377fe00640 1 --2- 172.21.15.5:0/1218056431 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3780103ff0 0x7f378019a5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:08.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:08.927+0000 7f377fe00640 1 -- 172.21.15.5:0/1218056431 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f37801049a0 msgr2=0x7f378019aaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:52:08.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:08.927+0000 7f377fe00640 1 --2- 172.21.15.5:0/1218056431 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f37801049a0 0x7f378019aaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:08.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:08.928+0000 7f377fe00640 1 -- 172.21.15.5:0/1218056431 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f378019f560 con 0x7f3780109120 2024-09-10T09:52:08.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:08.928+0000 7f377ea00640 1 --2- 172.21.15.5:0/1218056431 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f37801049a0 0x7f378019aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:52:08.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:08.928+0000 7f377fe00640 1 --2- 172.21.15.5:0/1218056431 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3780109120 0x7f378019ee80 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f377802ade0 tx=0x7f37780319f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:52:08.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:08.928+0000 7f3777e00640 1 -- 172.21.15.5:0/1218056431 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3778003b70 con 0x7f3780109120 2024-09-10T09:52:08.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:08.928+0000 7f3777e00640 1 -- 172.21.15.5:0/1218056431 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3778003d10 con 0x7f3780109120 2024-09-10T09:52:08.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:08.928+0000 7f3786400640 1 -- 172.21.15.5:0/1218056431 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f378019f7f0 con 0x7f3780109120 2024-09-10T09:52:08.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:08.928+0000 7f3777e00640 1 -- 172.21.15.5:0/1218056431 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f377802fc40 con 0x7f3780109120 2024-09-10T09:52:08.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:08.928+0000 7f3786400640 1 -- 172.21.15.5:0/1218056431 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37801a7130 con 0x7f3780109120 2024-09-10T09:52:08.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:08.931+0000 7f3789044740 1 Processor -- start 2024-09-10T09:52:08.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:08.932+0000 7f3777e00640 1 -- 172.21.15.5:0/1218056431 <== mon.2 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f3778004050 con 0x7f3780109120 2024-09-10T09:52:08.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:08.932+0000 7f3789044740 1 -- start start 2024-09-10T09:52:08.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:08.932+0000 7f3789044740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555b88f72c70 0x555b88f73050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:52:08.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:08.932+0000 7f3789044740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555b88f73590 0x555b88f7d910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:52:08.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:08.932+0000 7f3789044740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555b88f7de50 0x555b88f80240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:52:08.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:08.932+0000 7f3789044740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x555b88ed93e0 con 0x555b88f72c70 2024-09-10T09:52:08.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:08.932+0000 7f3789044740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x555b88e80d80 con 0x555b88f7de50 2024-09-10T09:52:08.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:08.932+0000 7f3789044740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x555b88ed53a0 con 0x555b88f73590 2024-09-10T09:52:08.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:08.932+0000 7f377ea00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555b88f72c70 0x555b88f73050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:52:08.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:08.932+0000 7f377f400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555b88f73590 0x555b88f7d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:52:08.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:08.933+0000 7f377f400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555b88f73590 0x555b88f7d910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:45360/0 (socket says 172.21.15.5:45360) 2024-09-10T09:52:08.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:08.933+0000 7f377f400640 1 -- 172.21.15.5:0/2322220260 learned_addr learned my addr 172.21.15.5:0/2322220260 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:52:08.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:08.933+0000 7f377ea00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555b88f72c70 0x555b88f73050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57242/0 (socket says 172.21.15.5:57242) 2024-09-10T09:52:08.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:08.933+0000 7f377ea00640 1 --2- 172.21.15.5:0/2322220260 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555b88f7de50 0x555b88f80240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:52:08.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:08.933+0000 7f377f400640 1 -- 172.21.15.5:0/2322220260 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555b88f7de50 msgr2=0x555b88f80240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:52:08.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:08.933+0000 7f377f400640 1 --2- 172.21.15.5:0/2322220260 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555b88f7de50 0x555b88f80240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:08.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:08.933+0000 7f377f400640 1 -- 172.21.15.5:0/2322220260 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555b88f72c70 msgr2=0x555b88f73050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:52:08.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:08.933+0000 7f377f400640 1 --2- 172.21.15.5:0/2322220260 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555b88f72c70 0x555b88f73050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:08.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:08.933+0000 7f377f400640 1 -- 172.21.15.5:0/2322220260 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555b88e9fae0 con 0x555b88f73590 2024-09-10T09:52:08.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:08.933+0000 7f377ea00640 1 --2- 172.21.15.5:0/2322220260 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555b88f72c70 0x555b88f73050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:52:08.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:08.933+0000 7f377f400640 1 --2- 172.21.15.5:0/2322220260 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555b88f73590 0x555b88f7d910 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f3770009cf0 tx=0x7f3770011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=10c4469948dd47b9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:52:08.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:08.933+0000 7f3777e00640 1 --2- 172.21.15.5:0/1218056431 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f37640a2b80 0x7f37640a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:52:08.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:08.934+0000 7f3774c00640 1 -- 172.21.15.5:0/2322220260 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3770004230 con 0x555b88f73590 2024-09-10T09:52:08.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:08.934+0000 7f3774c00640 1 -- 172.21.15.5:0/2322220260 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f37700043d0 con 0x555b88f73590 2024-09-10T09:52:08.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:08.934+0000 7f3774c00640 1 -- 172.21.15.5:0/2322220260 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3770019900 con 0x555b88f73590 2024-09-10T09:52:08.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:08.934+0000 7f3777e00640 1 -- 172.21.15.5:0/1218056431 <== mon.2 v2:172.21.15.102:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f37780f03e0 con 0x7f3780109120 2024-09-10T09:52:08.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:08.934+0000 7f377ea00640 1 --2- 172.21.15.5:0/1218056431 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f37640a2b80 0x7f37640a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:52:08.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:08.934+0000 7f3789044740 1 -- 172.21.15.5:0/2322220260 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555b88f73590 msgr2=0x555b88f7d910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:52:08.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:08.934+0000 7f3789044740 1 --2- 172.21.15.5:0/2322220260 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555b88f73590 0x555b88f7d910 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f3770009cf0 tx=0x7f3770011810 comp rx=0 tx=0).stop 2024-09-10T09:52:08.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:08.935+0000 7f3789044740 1 -- 172.21.15.5:0/2322220260 shutdown_connections 2024-09-10T09:52:08.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:08.935+0000 7f377ea00640 1 --2- 172.21.15.5:0/1218056431 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f37640a2b80 0x7f37640a5040 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x555b88f70580 tx=0x7f376c009600 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:52:08.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:08.935+0000 7f3789044740 1 --2- 172.21.15.5:0/2322220260 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555b88f7de50 0x555b88f80240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:08.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:08.935+0000 7f3789044740 1 --2- 172.21.15.5:0/2322220260 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555b88f73590 0x555b88f7d910 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:08.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:08.935+0000 7f3789044740 1 --2- 172.21.15.5:0/2322220260 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555b88f72c70 0x555b88f73050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:08.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:08.935+0000 7f3789044740 1 -- 172.21.15.5:0/2322220260 >> 172.21.15.5:0/2322220260 conn(0x555b88f6f200 msgr2=0x555b88f6f5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:52:08.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:08.935+0000 7f3789044740 1 -- 172.21.15.5:0/2322220260 shutdown_connections 2024-09-10T09:52:08.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:08.935+0000 7f3789044740 1 -- 172.21.15.5:0/2322220260 wait complete. 2024-09-10T09:52:08.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:08.935+0000 7f3789044740 1 -- start start 2024-09-10T09:52:08.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:08.935+0000 7f3789044740 1 Processor -- start 2024-09-10T09:52:08.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:08.936+0000 7f3789044740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555b88f72c70 0x555b88f7aca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:52:08.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:08.936+0000 7f3789044740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555b88f73590 0x555b88f7b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:52:08.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:08.936+0000 7f3789044740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555b88f7de50 0x555b88f7b720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:52:08.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:08.936+0000 7f3789044740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x555b88f80a20 con 0x555b88f7de50 2024-09-10T09:52:08.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:08.936+0000 7f3789044740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x555b88e940f0 con 0x555b88f72c70 2024-09-10T09:52:08.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:08.936+0000 7f3789044740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x555b88f80ba0 con 0x555b88f73590 2024-09-10T09:52:08.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:08.936+0000 7f3789044740 10 client.0 _ll_register_callbacks cb 0x555b88e7bbd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:52:08.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:08.936+0000 7f377ea00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555b88f73590 0x555b88f7b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:52:08.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:08.936+0000 7f377fe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555b88f7de50 0x555b88f7b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:52:08.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:08.936+0000 7f377ea00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555b88f73590 0x555b88f7b1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:45372/0 (socket says 172.21.15.5:45372) 2024-09-10T09:52:08.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:08.936+0000 7f377ea00640 1 -- 172.21.15.5:0/1070129480 learned_addr learned my addr 172.21.15.5:0/1070129480 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:52:08.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:08.936+0000 7f377f400640 1 --2- 172.21.15.5:0/1070129480 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555b88f72c70 0x555b88f7aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:52:08.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:08.937+0000 7f377ea00640 1 -- 172.21.15.5:0/1070129480 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555b88f72c70 msgr2=0x555b88f7aca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:52:08.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:08.937+0000 7f377ea00640 1 --2- 172.21.15.5:0/1070129480 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555b88f72c70 0x555b88f7aca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:08.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:08.937+0000 7f377ea00640 1 -- 172.21.15.5:0/1070129480 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555b88f7de50 msgr2=0x555b88f7b720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:52:08.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:08.937+0000 7f377ea00640 1 --2- 172.21.15.5:0/1070129480 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555b88f7de50 0x555b88f7b720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:08.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:08.937+0000 7f377ea00640 1 -- 172.21.15.5:0/1070129480 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555b88c4a3c0 con 0x555b88f73590 2024-09-10T09:52:08.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:08.937+0000 7f377fe00640 1 --2- 172.21.15.5:0/1070129480 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555b88f7de50 0x555b88f7b720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:52:08.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:08.937+0000 7f377ea00640 1 --2- 172.21.15.5:0/1070129480 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555b88f73590 0x555b88f7b1e0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f376c010ef0 tx=0x7f376c004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6ea0630c8d0dc0d5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:52:08.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:08.937+0000 7f3769600640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-09-10T09:52:08.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:08.937+0000 7f3769600640 1 -- 172.21.15.5:0/1070129480 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f376c021d30 con 0x555b88f73590 2024-09-10T09:52:08.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:08.937+0000 7f3789044740 1 -- 172.21.15.5:0/1070129480 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f37700043d0 con 0x555b88f73590 2024-09-10T09:52:08.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:08.938+0000 7f3769600640 1 -- 172.21.15.5:0/1070129480 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f376c005020 con 0x555b88f73590 2024-09-10T09:52:08.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:08.938+0000 7f3769600640 1 -- 172.21.15.5:0/1070129480 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f376c005220 con 0x555b88f73590 2024-09-10T09:52:08.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:08.938+0000 7f3789044740 10 client.28183 fetch_fsmap learned FSMap version 36 2024-09-10T09:52:08.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:08.938+0000 7f3789044740 1 -- 172.21.15.5:0/1070129480 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x555b88f8ca70 con 0x555b88f73590 2024-09-10T09:52:08.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:08.938+0000 7f3769600640 1 -- 172.21.15.5:0/1070129480 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f37700043d0 con 0x555b88f73590 2024-09-10T09:52:08.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:08.938+0000 7f3769600640 1 -- 172.21.15.5:0/1070129480 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 36) ==== 8370+0+0 (secure 0 0 0) 0x7f376c02c550 con 0x555b88f73590 2024-09-10T09:52:08.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:08.938+0000 7f3789044740 10 client.28183 fetch_fsmap finished waiting for FSMap version 36 2024-09-10T09:52:08.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:08.938+0000 7f3789044740 10 client.28183 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-09-10T09:52:08.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:08.938+0000 7f3789044740 20 client.28183 populate_metadata read hostname 'smithi005' 2024-09-10T09:52:08.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:08.939+0000 7f3789044740 1 --2- 172.21.15.5:0/1070129480 >> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] conn(0x555b88f8ded0 0x555b88f90310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:52:08.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:08.939+0000 7f3789044740 4 client.28183 mds_command: new command op to 24443 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:52:08.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:08.939+0000 7f3789044740 1 -- 172.21.15.5:0/1070129480 --> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555b88ee6950 con 0x555b88f8ded0 2024-09-10T09:52:08.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:08.939+0000 7f377f400640 1 --2- 172.21.15.5:0/1070129480 >> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] conn(0x555b88f8ded0 0x555b88f90310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:52:08.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:08.940+0000 7f377f400640 1 --2- 172.21.15.5:0/1070129480 >> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] conn(0x555b88f8ded0 0x555b88f90310 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=be219d022950cc server_cookie=c081daa0986ca9d9 in_seq=0 out_seq=0 2024-09-10T09:52:08.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:08.940+0000 7f3769600640 10 client.28183 ms_handle_connect on v2:172.21.15.102:6832/2457007427 2024-09-10T09:52:08.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:08.940+0000 7f3769600640 1 -- 172.21.15.5:0/1070129480 <== mds.4 v2:172.21.15.102:6832/2457007427 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555b88ee6950 con 0x555b88f8ded0 2024-09-10T09:52:08.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:08.940+0000 7f3769600640 10 client.28183 handle_command_reply: tid=0 2024-09-10T09:52:08.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:08.940+0000 7f3789044740 1 client.28183 shutdown 2024-09-10T09:52:08.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:08.941+0000 7f3789044740 1 -- 172.21.15.5:0/1070129480 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555b88f73590 msgr2=0x555b88f7b1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:52:08.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:08.941+0000 7f3789044740 1 --2- 172.21.15.5:0/1070129480 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555b88f73590 0x555b88f7b1e0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f376c010ef0 tx=0x7f376c004730 comp rx=0 tx=0).stop 2024-09-10T09:52:08.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:08.941+0000 7f3789044740 1 -- 172.21.15.5:0/1070129480 shutdown_connections 2024-09-10T09:52:08.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:08.941+0000 7f3789044740 1 --2- 172.21.15.5:0/1070129480 >> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] conn(0x555b88f8ded0 0x555b88f90310 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:08.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:08.941+0000 7f3789044740 1 --2- 172.21.15.5:0/1070129480 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555b88f7de50 0x555b88f7b720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:08.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:08.941+0000 7f3789044740 1 --2- 172.21.15.5:0/1070129480 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555b88f73590 0x555b88f7b1e0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:08.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:08.941+0000 7f3769600640 0 client.28183 ms_handle_reset on v2:172.21.15.102:6832/2457007427 2024-09-10T09:52:08.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:08.941+0000 7f3789044740 1 --2- 172.21.15.5:0/1070129480 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555b88f72c70 0x555b88f7aca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:08.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:08.941+0000 7f3789044740 1 -- 172.21.15.5:0/1070129480 >> 172.21.15.5:0/1070129480 conn(0x555b88f6f200 msgr2=0x555b88f74f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:52:08.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:08.941+0000 7f3789044740 1 -- 172.21.15.5:0/1070129480 shutdown_connections 2024-09-10T09:52:08.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:08.941+0000 7f3789044740 1 -- 172.21.15.5:0/1070129480 wait complete. 2024-09-10T09:52:08.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:08.941+0000 7f3789044740 20 client.28183 trim_cache size 0 max 16384 2024-09-10T09:52:08.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:08.955+0000 7f3789044740 1 Processor -- start 2024-09-10T09:52:08.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:08.955+0000 7f3789044740 1 -- start start 2024-09-10T09:52:08.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:08.955+0000 7f3789044740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555b88f72c70 0x555b88f756d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:52:08.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:08.955+0000 7f3789044740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555b88f73590 0x555b88f95be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:52:08.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:08.955+0000 7f3789044740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555b88f7de50 0x555b88f96120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:52:08.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:08.955+0000 7f3789044740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f37801b0fc0 con 0x555b88f72c70 2024-09-10T09:52:08.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:08.955+0000 7f3789044740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f37801b2b00 con 0x555b88f73590 2024-09-10T09:52:08.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:08.955+0000 7f3789044740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f37801b1690 con 0x555b88f7de50 2024-09-10T09:52:08.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:08.955+0000 7f377f400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555b88f72c70 0x555b88f756d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:52:08.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:08.956+0000 7f377f400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555b88f72c70 0x555b88f756d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57258/0 (socket says 172.21.15.5:57258) 2024-09-10T09:52:08.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:08.956+0000 7f377ea00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555b88f73590 0x555b88f95be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:52:08.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:08.956+0000 7f377f400640 1 -- 172.21.15.5:0/4181856875 learned_addr learned my addr 172.21.15.5:0/4181856875 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:52:08.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:08.956+0000 7f377fe00640 1 --2- 172.21.15.5:0/4181856875 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555b88f7de50 0x555b88f96120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:52:08.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:08.956+0000 7f377f400640 1 -- 172.21.15.5:0/4181856875 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555b88f7de50 msgr2=0x555b88f96120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:52:08.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:08.956+0000 7f377f400640 1 --2- 172.21.15.5:0/4181856875 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555b88f7de50 0x555b88f96120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:08.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:08.956+0000 7f377f400640 1 -- 172.21.15.5:0/4181856875 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555b88f73590 msgr2=0x555b88f95be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:52:08.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:08.956+0000 7f377f400640 1 --2- 172.21.15.5:0/4181856875 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555b88f73590 0x555b88f95be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:08.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:08.956+0000 7f377f400640 1 -- 172.21.15.5:0/4181856875 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555b88f6f030 con 0x555b88f72c70 2024-09-10T09:52:08.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:08.956+0000 7f377f400640 1 --2- 172.21.15.5:0/4181856875 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555b88f72c70 0x555b88f756d0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f3770005fd0 tx=0x7f3770011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1ce4939f51da2ef server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:52:08.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:08.956+0000 7f376be00640 1 -- 172.21.15.5:0/4181856875 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3770005250 con 0x555b88f72c70 2024-09-10T09:52:08.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:08.957+0000 7f376be00640 1 -- 172.21.15.5:0/4181856875 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3770003ad0 con 0x555b88f72c70 2024-09-10T09:52:08.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:08.957+0000 7f376be00640 1 -- 172.21.15.5:0/4181856875 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3770019830 con 0x555b88f72c70 2024-09-10T09:52:08.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:08.957+0000 7f3789044740 1 -- 172.21.15.5:0/4181856875 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555b88f72c70 msgr2=0x555b88f756d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:52:08.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:08.957+0000 7f3789044740 1 --2- 172.21.15.5:0/4181856875 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555b88f72c70 0x555b88f756d0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f3770005fd0 tx=0x7f3770011810 comp rx=0 tx=0).stop 2024-09-10T09:52:08.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:08.957+0000 7f3789044740 1 -- 172.21.15.5:0/4181856875 shutdown_connections 2024-09-10T09:52:08.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:08.957+0000 7f3789044740 1 --2- 172.21.15.5:0/4181856875 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555b88f7de50 0x555b88f96120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:08.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:08.957+0000 7f3789044740 1 --2- 172.21.15.5:0/4181856875 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555b88f73590 0x555b88f95be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:08.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:08.957+0000 7f3789044740 1 --2- 172.21.15.5:0/4181856875 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555b88f72c70 0x555b88f756d0 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:08.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:08.957+0000 7f3789044740 1 -- 172.21.15.5:0/4181856875 >> 172.21.15.5:0/4181856875 conn(0x555b88f6f200 msgr2=0x555b88f79d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:52:08.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:08.957+0000 7f3789044740 1 -- 172.21.15.5:0/4181856875 shutdown_connections 2024-09-10T09:52:08.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:08.957+0000 7f3789044740 1 -- 172.21.15.5:0/4181856875 wait complete. 2024-09-10T09:52:08.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:08.958+0000 7f3789044740 1 -- start start 2024-09-10T09:52:08.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:08.958+0000 7f3789044740 1 Processor -- start 2024-09-10T09:52:08.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:08.959+0000 7f3789044740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555b88f72c70 0x555b88f86080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:52:08.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:08.959+0000 7f3789044740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555b88f73590 0x555b88f865c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:52:08.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:08.959+0000 7f3789044740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555b88f7de50 0x555b88f775b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:52:08.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:08.959+0000 7f3789044740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f37801b2430 con 0x555b88f7de50 2024-09-10T09:52:08.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:08.959+0000 7f3789044740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f37801b1d60 con 0x555b88f73590 2024-09-10T09:52:08.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:08.959+0000 7f3789044740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f37801aefa0 con 0x555b88f72c70 2024-09-10T09:52:08.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:08.959+0000 7f3789044740 10 client.0 _ll_register_callbacks cb 0x7f376c015850 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:52:08.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:08.959+0000 7f377fe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555b88f7de50 0x555b88f775b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:52:08.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:08.959+0000 7f377f400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555b88f72c70 0x555b88f86080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:52:08.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:08.959+0000 7f377f400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555b88f72c70 0x555b88f86080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:45390/0 (socket says 172.21.15.5:45390) 2024-09-10T09:52:08.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:08.959+0000 7f377f400640 1 -- 172.21.15.5:0/2450437383 learned_addr learned my addr 172.21.15.5:0/2450437383 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:52:08.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:08.959+0000 7f377ea00640 1 --2- 172.21.15.5:0/2450437383 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555b88f73590 0x555b88f865c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:52:08.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:08.959+0000 7f377f400640 1 -- 172.21.15.5:0/2450437383 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555b88f73590 msgr2=0x555b88f865c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:52:08.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:08.959+0000 7f377f400640 1 --2- 172.21.15.5:0/2450437383 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555b88f73590 0x555b88f865c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:08.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:08.959+0000 7f377f400640 1 -- 172.21.15.5:0/2450437383 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555b88f7de50 msgr2=0x555b88f775b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:52:08.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:08.959+0000 7f377f400640 1 --2- 172.21.15.5:0/2450437383 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555b88f7de50 0x555b88f775b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:08.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:08.959+0000 7f377f400640 1 -- 172.21.15.5:0/2450437383 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555b88f6a770 con 0x555b88f72c70 2024-09-10T09:52:08.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:08.959+0000 7f377fe00640 1 --2- 172.21.15.5:0/2450437383 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555b88f7de50 0x555b88f775b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:52:08.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:08.960+0000 7f377f400640 1 --2- 172.21.15.5:0/2450437383 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555b88f72c70 0x555b88f86080 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f37700058c0 tx=0x7f37700096f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b1f85f958ab7e575 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:52:08.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:08.960+0000 7f3769600640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-09-10T09:52:08.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:08.960+0000 7f3769600640 1 -- 172.21.15.5:0/2450437383 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3770003c80 con 0x555b88f72c70 2024-09-10T09:52:08.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:08.960+0000 7f3789044740 1 -- 172.21.15.5:0/2450437383 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3770003ad0 con 0x555b88f72c70 2024-09-10T09:52:08.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:08.960+0000 7f3769600640 1 -- 172.21.15.5:0/2450437383 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f3770005250 con 0x555b88f72c70 2024-09-10T09:52:08.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:08.960+0000 7f3769600640 1 -- 172.21.15.5:0/2450437383 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f3770043050 con 0x555b88f72c70 2024-09-10T09:52:08.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:08.960+0000 7f3789044740 10 client.28189 fetch_fsmap learned FSMap version 36 2024-09-10T09:52:08.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:08.960+0000 7f3789044740 1 -- 172.21.15.5:0/2450437383 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x555b88ff8550 con 0x555b88f72c70 2024-09-10T09:52:08.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:08.961+0000 7f3769600640 1 -- 172.21.15.5:0/2450437383 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3770003ad0 con 0x555b88f72c70 2024-09-10T09:52:08.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:08.961+0000 7f3769600640 1 -- 172.21.15.5:0/2450437383 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 36) ==== 8370+0+0 (secure 0 0 0) 0x7f3770035440 con 0x555b88f72c70 2024-09-10T09:52:08.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:08.961+0000 7f3789044740 10 client.28189 fetch_fsmap finished waiting for FSMap version 36 2024-09-10T09:52:08.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:08.961+0000 7f3789044740 10 client.28189 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-09-10T09:52:08.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:08.961+0000 7f3789044740 20 client.28189 populate_metadata read hostname 'smithi005' 2024-09-10T09:52:08.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:08.961+0000 7f3789044740 1 --2- 172.21.15.5:0/2450437383 >> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] conn(0x555b88ff8a70 0x555b88ffae60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:52:08.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:08.961+0000 7f3789044740 4 client.28189 mds_command: new command op to 24443 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T09:52:08.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:08.961+0000 7f3789044740 1 -- 172.21.15.5:0/2450437383 --> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555b88ee0e60 con 0x555b88ff8a70 2024-09-10T09:52:08.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:08.961+0000 7f377ea00640 1 --2- 172.21.15.5:0/2450437383 >> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] conn(0x555b88ff8a70 0x555b88ffae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:52:08.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:08.962+0000 7f377ea00640 1 --2- 172.21.15.5:0/2450437383 >> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] conn(0x555b88ff8a70 0x555b88ffae60 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=428c9dce31675268 server_cookie=9abda9dd48c1dccc in_seq=0 out_seq=0 2024-09-10T09:52:08.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:08.962+0000 7f3769600640 10 client.28189 ms_handle_connect on v2:172.21.15.102:6832/2457007427 2024-09-10T09:52:08.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:08.962+0000 7f3769600640 1 -- 172.21.15.5:0/2450437383 <== mds.4 v2:172.21.15.102:6832/2457007427 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555b88ee0e60 con 0x555b88ff8a70 2024-09-10T09:52:08.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:08.962+0000 7f3769600640 10 client.28189 handle_command_reply: tid=0 2024-09-10T09:52:08.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:08.962+0000 7f3789044740 1 client.28189 shutdown 2024-09-10T09:52:08.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:08.962+0000 7f3789044740 1 -- 172.21.15.5:0/2450437383 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555b88f72c70 msgr2=0x555b88f86080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:52:08.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:08.962+0000 7f3789044740 1 --2- 172.21.15.5:0/2450437383 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555b88f72c70 0x555b88f86080 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f37700058c0 tx=0x7f37700096f0 comp rx=0 tx=0).stop 2024-09-10T09:52:08.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:08.962+0000 7f3789044740 1 -- 172.21.15.5:0/2450437383 shutdown_connections 2024-09-10T09:52:08.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:08.962+0000 7f3789044740 1 --2- 172.21.15.5:0/2450437383 >> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] conn(0x555b88ff8a70 0x555b88ffae60 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:08.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:08.962+0000 7f3789044740 1 --2- 172.21.15.5:0/2450437383 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555b88f7de50 0x555b88f775b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:08.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:08.963+0000 7f3789044740 1 --2- 172.21.15.5:0/2450437383 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555b88f73590 0x555b88f865c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:08.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:08.963+0000 7f3789044740 1 --2- 172.21.15.5:0/2450437383 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555b88f72c70 0x555b88f86080 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:08.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:08.963+0000 7f3769600640 0 client.28189 ms_handle_reset on v2:172.21.15.102:6832/2457007427 2024-09-10T09:52:08.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:08.963+0000 7f3789044740 1 -- 172.21.15.5:0/2450437383 >> 172.21.15.5:0/2450437383 conn(0x555b88f6f200 msgr2=0x555b88f6db60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:52:08.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:08.963+0000 7f3789044740 1 -- 172.21.15.5:0/2450437383 shutdown_connections 2024-09-10T09:52:08.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:08.963+0000 7f3789044740 1 -- 172.21.15.5:0/2450437383 wait complete. 2024-09-10T09:52:08.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:08.963+0000 7f3789044740 20 client.28189 trim_cache size 0 max 16384 2024-09-10T09:52:08.968 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-10T09:52:08.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:08.963+0000 7f3786400640 1 -- 172.21.15.5:0/1218056431 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f37640a2b80 msgr2=0x7f37640a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:52:08.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:08.963+0000 7f3786400640 1 --2- 172.21.15.5:0/1218056431 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f37640a2b80 0x7f37640a5040 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x555b88f70580 tx=0x7f376c009600 comp rx=0 tx=0).stop 2024-09-10T09:52:08.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:08.963+0000 7f3786400640 1 -- 172.21.15.5:0/1218056431 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3780109120 msgr2=0x7f378019ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:52:08.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:08.963+0000 7f3786400640 1 --2- 172.21.15.5:0/1218056431 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3780109120 0x7f378019ee80 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f377802ade0 tx=0x7f37780319f0 comp rx=0 tx=0).stop 2024-09-10T09:52:08.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:08.964+0000 7f3786400640 1 -- 172.21.15.5:0/1218056431 shutdown_connections 2024-09-10T09:52:08.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:08.964+0000 7f3786400640 1 --2- 172.21.15.5:0/1218056431 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f37640a2b80 0x7f37640a5040 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:08.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:08.964+0000 7f3786400640 1 --2- 172.21.15.5:0/1218056431 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3780109120 0x7f378019ee80 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:08.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:08.964+0000 7f3786400640 1 --2- 172.21.15.5:0/1218056431 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f37801049a0 0x7f378019aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:08.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:08.964+0000 7f3786400640 1 --2- 172.21.15.5:0/1218056431 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3780103ff0 0x7f378019a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:08.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:08.964+0000 7f3786400640 1 -- 172.21.15.5:0/1218056431 >> 172.21.15.5:0/1218056431 conn(0x7f37800ffa90 msgr2=0x7f3780101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:52:08.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:08.964+0000 7f3786400640 1 -- 172.21.15.5:0/1218056431 shutdown_connections 2024-09-10T09:52:08.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:08.964+0000 7f3786400640 1 -- 172.21.15.5:0/1218056431 wait complete. 2024-09-10T09:52:09.171 INFO:tasks.workunit.client.0.smithi005.stdout: 1 866428 6.19 MB/sec execute 590 sec latency 10311.260 ms 2024-09-10T09:52:09.306 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:52:09 smithi005 ceph-mon[30350]: pgmap v807: 129 pgs: 129 active+clean; 291 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.0 MiB/s wr, 115 op/s 2024-09-10T09:52:09.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:52:09 smithi102 ceph-mon[35442]: pgmap v807: 129 pgs: 129 active+clean; 291 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.0 MiB/s wr, 115 op/s 2024-09-10T09:52:09.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:52:09 smithi159 ceph-mon[35513]: pgmap v807: 129 pgs: 129 active+clean; 291 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.0 MiB/s wr, 115 op/s 2024-09-10T09:52:09.887 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-10T09:52:10.171 INFO:tasks.workunit.client.0.smithi005.stdout: 1 866428 6.18 MB/sec execute 591 sec latency 11311.349 ms 2024-09-10T09:52:10.267 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:52:10.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:10.866+0000 7ffb38800640 1 -- 172.21.15.5:0/1202688493 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffb301049a0 msgr2=0x7ffb301089f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:52:10.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:10.866+0000 7ffb38800640 1 --2- 172.21.15.5:0/1202688493 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffb301049a0 0x7ffb301089f0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7ffb20009900 tx=0x7ffb2002a3f0 comp rx=0 tx=0).stop 2024-09-10T09:52:10.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:10.866+0000 7ffb38800640 1 -- 172.21.15.5:0/1202688493 shutdown_connections 2024-09-10T09:52:10.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:10.866+0000 7ffb38800640 1 --2- 172.21.15.5:0/1202688493 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ffb30109120 0x7ffb30110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:10.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:10.866+0000 7ffb38800640 1 --2- 172.21.15.5:0/1202688493 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffb301049a0 0x7ffb301089f0 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:10.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:10.866+0000 7ffb38800640 1 --2- 172.21.15.5:0/1202688493 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ffb30103ff0 0x7ffb301043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:10.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:10.866+0000 7ffb38800640 1 -- 172.21.15.5:0/1202688493 >> 172.21.15.5:0/1202688493 conn(0x7ffb300ffa90 msgr2=0x7ffb30101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:52:10.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:10.866+0000 7ffb38800640 1 -- 172.21.15.5:0/1202688493 shutdown_connections 2024-09-10T09:52:10.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:10.866+0000 7ffb38800640 1 -- 172.21.15.5:0/1202688493 wait complete. 2024-09-10T09:52:10.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:10.867+0000 7ffb38800640 1 Processor -- start 2024-09-10T09:52:10.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:10.867+0000 7ffb38800640 1 -- start start 2024-09-10T09:52:10.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:10.868+0000 7ffb38800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffb30103ff0 0x7ffb3019a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:52:10.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:10.868+0000 7ffb38800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ffb301049a0 0x7ffb3019ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:52:10.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:10.868+0000 7ffb38800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ffb30109120 0x7ffb3019eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:52:10.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:10.868+0000 7ffb38800640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ffb30114190 con 0x7ffb30103ff0 2024-09-10T09:52:10.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:10.868+0000 7ffb38800640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7ffb30114010 con 0x7ffb301049a0 2024-09-10T09:52:10.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:10.868+0000 7ffb38800640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7ffb30114310 con 0x7ffb30109120 2024-09-10T09:52:10.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:10.868+0000 7ffb35e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffb30103ff0 0x7ffb3019a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:52:10.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:10.868+0000 7ffb35e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffb30103ff0 0x7ffb3019a5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47610/0 (socket says 172.21.15.5:47610) 2024-09-10T09:52:10.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:10.868+0000 7ffb35e00640 1 -- 172.21.15.5:0/3058604487 learned_addr learned my addr 172.21.15.5:0/3058604487 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:52:10.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:10.868+0000 7ffb35400640 1 --2- 172.21.15.5:0/3058604487 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ffb301049a0 0x7ffb3019ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:52:10.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:10.868+0000 7ffb36800640 1 --2- 172.21.15.5:0/3058604487 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ffb30109120 0x7ffb3019eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:52:10.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:10.869+0000 7ffb35e00640 1 -- 172.21.15.5:0/3058604487 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ffb30109120 msgr2=0x7ffb3019eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:52:10.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:10.869+0000 7ffb35e00640 1 --2- 172.21.15.5:0/3058604487 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ffb30109120 0x7ffb3019eec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:10.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:10.869+0000 7ffb35e00640 1 -- 172.21.15.5:0/3058604487 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ffb301049a0 msgr2=0x7ffb3019ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:52:10.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:10.869+0000 7ffb35e00640 1 --2- 172.21.15.5:0/3058604487 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ffb301049a0 0x7ffb3019ab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:10.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:10.869+0000 7ffb35e00640 1 -- 172.21.15.5:0/3058604487 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb3019f5a0 con 0x7ffb30103ff0 2024-09-10T09:52:10.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:10.869+0000 7ffb35e00640 1 --2- 172.21.15.5:0/3058604487 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffb30103ff0 0x7ffb3019a5f0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7ffb2400e9f0 tx=0x7ffb2400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:52:10.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:10.869+0000 7ffb2aa00640 1 -- 172.21.15.5:0/3058604487 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb2400cd80 con 0x7ffb30103ff0 2024-09-10T09:52:10.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:10.869+0000 7ffb2aa00640 1 -- 172.21.15.5:0/3058604487 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffb24010910 con 0x7ffb30103ff0 2024-09-10T09:52:10.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:10.869+0000 7ffb38800640 1 -- 172.21.15.5:0/3058604487 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb3019f890 con 0x7ffb30103ff0 2024-09-10T09:52:10.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:10.869+0000 7ffb2aa00640 1 -- 172.21.15.5:0/3058604487 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb24018990 con 0x7ffb30103ff0 2024-09-10T09:52:10.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:10.869+0000 7ffb38800640 1 -- 172.21.15.5:0/3058604487 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb301a7170 con 0x7ffb30103ff0 2024-09-10T09:52:10.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:10.873+0000 7ffb2aa00640 1 -- 172.21.15.5:0/3058604487 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7ffb24010ac0 con 0x7ffb30103ff0 2024-09-10T09:52:10.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:10.873+0000 7ffb3b27b740 1 Processor -- start 2024-09-10T09:52:10.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:10.873+0000 7ffb3b27b740 1 -- start start 2024-09-10T09:52:10.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:10.873+0000 7ffb3b27b740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564f1472cc70 0x564f1472d050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:52:10.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:10.873+0000 7ffb3b27b740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564f1472d590 0x564f14737910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:52:10.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:10.873+0000 7ffb3b27b740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564f14737e50 0x564f1473a240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:52:10.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:10.873+0000 7ffb3b27b740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x564f146933e0 con 0x564f1472cc70 2024-09-10T09:52:10.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:10.873+0000 7ffb3b27b740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x564f1463ad80 con 0x564f14737e50 2024-09-10T09:52:10.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:10.873+0000 7ffb3b27b740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x564f1468f3a0 con 0x564f1472d590 2024-09-10T09:52:10.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:10.874+0000 7ffb36800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564f1472cc70 0x564f1472d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:52:10.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:10.874+0000 7ffb36800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564f1472cc70 0x564f1472d050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47614/0 (socket says 172.21.15.5:47614) 2024-09-10T09:52:10.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:10.874+0000 7ffb36800640 1 -- 172.21.15.5:0/4088507381 learned_addr learned my addr 172.21.15.5:0/4088507381 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:52:10.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:10.874+0000 7ffb35400640 1 --2- 172.21.15.5:0/4088507381 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564f14737e50 0x564f1473a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:52:10.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:10.874+0000 7ffb35e00640 1 --2- 172.21.15.5:0/4088507381 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564f1472d590 0x564f14737910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:52:10.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:10.874+0000 7ffb35400640 1 -- 172.21.15.5:0/4088507381 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564f1472d590 msgr2=0x564f14737910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:52:10.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:10.874+0000 7ffb35400640 1 --2- 172.21.15.5:0/4088507381 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564f1472d590 0x564f14737910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:10.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:10.874+0000 7ffb35400640 1 -- 172.21.15.5:0/4088507381 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564f1472cc70 msgr2=0x564f1472d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:52:10.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:10.874+0000 7ffb35400640 1 --2- 172.21.15.5:0/4088507381 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564f1472cc70 0x564f1472d050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:10.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:10.874+0000 7ffb35400640 1 -- 172.21.15.5:0/4088507381 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564f14659ae0 con 0x564f14737e50 2024-09-10T09:52:10.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:10.874+0000 7ffb2aa00640 1 --2- 172.21.15.5:0/3058604487 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7ffb180a2b80 0x7ffb180a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:52:10.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:10.875+0000 7ffb2aa00640 1 -- 172.21.15.5:0/3058604487 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7ffb240cf7a0 con 0x7ffb30103ff0 2024-09-10T09:52:10.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:10.875+0000 7ffb35400640 1 --2- 172.21.15.5:0/4088507381 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564f14737e50 0x564f1473a240 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7ffb2002a9e0 tx=0x7ffb20004040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a06d48e64be7763b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:52:10.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:10.875+0000 7ffb36800640 1 --2- 172.21.15.5:0/3058604487 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7ffb180a2b80 0x7ffb180a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:52:10.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:10.875+0000 7ffb1f000640 1 -- 172.21.15.5:0/4088507381 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb20035040 con 0x564f14737e50 2024-09-10T09:52:10.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:10.875+0000 7ffb1f000640 1 -- 172.21.15.5:0/4088507381 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffb20005520 con 0x564f14737e50 2024-09-10T09:52:10.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:10.875+0000 7ffb1f000640 1 -- 172.21.15.5:0/4088507381 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb20003850 con 0x564f14737e50 2024-09-10T09:52:10.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:10.876+0000 7ffb3b27b740 1 -- 172.21.15.5:0/4088507381 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564f14737e50 msgr2=0x564f1473a240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:52:10.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:10.876+0000 7ffb36800640 1 --2- 172.21.15.5:0/3058604487 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7ffb180a2b80 0x7ffb180a5040 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7ffb2c005f40 tx=0x7ffb2c009600 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:52:10.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:10.876+0000 7ffb3b27b740 1 --2- 172.21.15.5:0/4088507381 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564f14737e50 0x564f1473a240 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7ffb2002a9e0 tx=0x7ffb20004040 comp rx=0 tx=0).stop 2024-09-10T09:52:10.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:10.876+0000 7ffb3b27b740 1 -- 172.21.15.5:0/4088507381 shutdown_connections 2024-09-10T09:52:10.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:10.876+0000 7ffb3b27b740 1 --2- 172.21.15.5:0/4088507381 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564f14737e50 0x564f1473a240 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:10.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:10.876+0000 7ffb3b27b740 1 --2- 172.21.15.5:0/4088507381 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564f1472d590 0x564f14737910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:10.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:10.876+0000 7ffb3b27b740 1 --2- 172.21.15.5:0/4088507381 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564f1472cc70 0x564f1472d050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:10.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:10.876+0000 7ffb3b27b740 1 -- 172.21.15.5:0/4088507381 >> 172.21.15.5:0/4088507381 conn(0x564f14729200 msgr2=0x564f147295e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:52:10.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:10.876+0000 7ffb3b27b740 1 -- 172.21.15.5:0/4088507381 shutdown_connections 2024-09-10T09:52:10.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:10.876+0000 7ffb3b27b740 1 -- 172.21.15.5:0/4088507381 wait complete. 2024-09-10T09:52:10.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:10.876+0000 7ffb3b27b740 1 -- start start 2024-09-10T09:52:10.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:10.877+0000 7ffb3b27b740 1 Processor -- start 2024-09-10T09:52:10.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:10.877+0000 7ffb3b27b740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564f1472cc70 0x564f14734d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:52:10.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:10.877+0000 7ffb3b27b740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564f1472d590 0x564f14735290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:52:10.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:10.877+0000 7ffb3b27b740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564f14737e50 0x564f147357d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:52:10.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:10.877+0000 7ffb3b27b740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x564f1473aa20 con 0x564f1472cc70 2024-09-10T09:52:10.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:10.877+0000 7ffb3b27b740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x564f1464e0f0 con 0x564f14737e50 2024-09-10T09:52:10.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:10.877+0000 7ffb3b27b740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x564f1473aba0 con 0x564f1472d590 2024-09-10T09:52:10.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:10.877+0000 7ffb3b27b740 10 client.0 _ll_register_callbacks cb 0x564f14635bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:52:10.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:10.877+0000 7ffb35e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564f1472cc70 0x564f14734d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:52:10.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:10.877+0000 7ffb36800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564f14737e50 0x564f147357d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:52:10.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:10.878+0000 7ffb35e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564f1472cc70 0x564f14734d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47630/0 (socket says 172.21.15.5:47630) 2024-09-10T09:52:10.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:10.878+0000 7ffb35e00640 1 -- 172.21.15.5:0/1777238828 learned_addr learned my addr 172.21.15.5:0/1777238828 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:52:10.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:10.878+0000 7ffb36800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564f14737e50 0x564f147357d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:37998/0 (socket says 172.21.15.5:37998) 2024-09-10T09:52:10.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:10.878+0000 7ffb35400640 1 --2- 172.21.15.5:0/1777238828 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564f1472d590 0x564f14735290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:52:10.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:10.878+0000 7ffb35e00640 1 -- 172.21.15.5:0/1777238828 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564f1472d590 msgr2=0x564f14735290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:52:10.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:10.878+0000 7ffb35e00640 1 --2- 172.21.15.5:0/1777238828 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564f1472d590 0x564f14735290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:10.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:10.878+0000 7ffb35e00640 1 -- 172.21.15.5:0/1777238828 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564f14737e50 msgr2=0x564f147357d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:52:10.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:10.878+0000 7ffb35e00640 1 --2- 172.21.15.5:0/1777238828 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564f14737e50 0x564f147357d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:10.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:10.878+0000 7ffb35e00640 1 -- 172.21.15.5:0/1777238828 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564f144043c0 con 0x564f1472cc70 2024-09-10T09:52:10.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:10.878+0000 7ffb36800640 1 --2- 172.21.15.5:0/1777238828 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564f14737e50 0x564f147357d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:52:10.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:10.878+0000 7ffb35e00640 1 --2- 172.21.15.5:0/1777238828 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564f1472cc70 0x564f14734d50 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7ffb2409b960 tx=0x7ffb2409bab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=966fe6d2bf7917d9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:52:10.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:10.878+0000 7ffb13400640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-10T09:52:10.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:10.879+0000 7ffb13400640 1 -- 172.21.15.5:0/1777238828 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb2409ff60 con 0x564f1472cc70 2024-09-10T09:52:10.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:10.879+0000 7ffb13400640 1 -- 172.21.15.5:0/1777238828 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7ffb240cfb90 con 0x564f1472cc70 2024-09-10T09:52:10.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:10.879+0000 7ffb3b27b740 1 -- 172.21.15.5:0/1777238828 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffb20005520 con 0x564f1472cc70 2024-09-10T09:52:10.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:10.879+0000 7ffb13400640 1 -- 172.21.15.5:0/1777238828 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7ffb240b2020 con 0x564f1472cc70 2024-09-10T09:52:10.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:10.879+0000 7ffb3b27b740 10 client.18171 fetch_fsmap learned FSMap version 36 2024-09-10T09:52:10.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:10.879+0000 7ffb3b27b740 1 -- 172.21.15.5:0/1777238828 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x564f14746a70 con 0x564f1472cc70 2024-09-10T09:52:10.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:10.879+0000 7ffb13400640 1 -- 172.21.15.5:0/1777238828 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffb20005520 con 0x564f1472cc70 2024-09-10T09:52:10.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:10.879+0000 7ffb13400640 1 -- 172.21.15.5:0/1777238828 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 36) ==== 8370+0+0 (secure 0 0 0) 0x7ffb2409b0b0 con 0x564f1472cc70 2024-09-10T09:52:10.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:10.879+0000 7ffb3b27b740 10 client.18171 fetch_fsmap finished waiting for FSMap version 36 2024-09-10T09:52:10.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:10.879+0000 7ffb3b27b740 10 client.18171 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-09-10T09:52:10.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:10.879+0000 7ffb3b27b740 20 client.18171 populate_metadata read hostname 'smithi005' 2024-09-10T09:52:10.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:10.879+0000 7ffb3b27b740 1 --2- 172.21.15.5:0/1777238828 >> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] conn(0x564f14747f30 0x564f1474a340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:52:10.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:10.879+0000 7ffb3b27b740 4 client.18171 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:52:10.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:10.879+0000 7ffb3b27b740 1 -- 172.21.15.5:0/1777238828 --> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564f146a0950 con 0x564f14747f30 2024-09-10T09:52:10.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:10.880+0000 7ffb35400640 1 --2- 172.21.15.5:0/1777238828 >> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] conn(0x564f14747f30 0x564f1474a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:52:10.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:10.880+0000 7ffb35400640 1 --2- 172.21.15.5:0/1777238828 >> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] conn(0x564f14747f30 0x564f1474a340 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=1614ba16fd37e7f3 server_cookie=336c08e6a5c93e2c in_seq=0 out_seq=0 2024-09-10T09:52:10.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:10.880+0000 7ffb13400640 10 client.18171 ms_handle_connect on v2:172.21.15.102:6836/2689776764 2024-09-10T09:52:10.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:10.881+0000 7ffb13400640 1 -- 172.21.15.5:0/1777238828 <== mds.3 v2:172.21.15.102:6836/2689776764 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564f146a0950 con 0x564f14747f30 2024-09-10T09:52:10.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:10.881+0000 7ffb13400640 10 client.18171 handle_command_reply: tid=0 2024-09-10T09:52:10.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:10.881+0000 7ffb3b27b740 1 client.18171 shutdown 2024-09-10T09:52:10.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:10.882+0000 7ffb3b27b740 1 -- 172.21.15.5:0/1777238828 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564f1472cc70 msgr2=0x564f14734d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:52:10.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:10.882+0000 7ffb3b27b740 1 --2- 172.21.15.5:0/1777238828 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564f1472cc70 0x564f14734d50 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7ffb2409b960 tx=0x7ffb2409bab0 comp rx=0 tx=0).stop 2024-09-10T09:52:10.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:10.882+0000 7ffb3b27b740 1 -- 172.21.15.5:0/1777238828 shutdown_connections 2024-09-10T09:52:10.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:10.882+0000 7ffb3b27b740 1 --2- 172.21.15.5:0/1777238828 >> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] conn(0x564f14747f30 0x564f1474a340 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:10.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:10.882+0000 7ffb3b27b740 1 --2- 172.21.15.5:0/1777238828 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564f14737e50 0x564f147357d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:10.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:10.882+0000 7ffb3b27b740 1 --2- 172.21.15.5:0/1777238828 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564f1472d590 0x564f14735290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:10.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:10.882+0000 7ffb13400640 0 client.18171 ms_handle_reset on v2:172.21.15.102:6836/2689776764 2024-09-10T09:52:10.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:10.882+0000 7ffb3b27b740 1 --2- 172.21.15.5:0/1777238828 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564f1472cc70 0x564f14734d50 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:10.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:10.882+0000 7ffb3b27b740 1 -- 172.21.15.5:0/1777238828 >> 172.21.15.5:0/1777238828 conn(0x564f14729200 msgr2=0x564f1472ef30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:52:10.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:10.882+0000 7ffb3b27b740 1 -- 172.21.15.5:0/1777238828 shutdown_connections 2024-09-10T09:52:10.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:10.882+0000 7ffb3b27b740 1 -- 172.21.15.5:0/1777238828 wait complete. 2024-09-10T09:52:10.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:10.882+0000 7ffb3b27b740 20 client.18171 trim_cache size 0 max 16384 2024-09-10T09:52:10.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:10.896+0000 7ffb3b27b740 1 Processor -- start 2024-09-10T09:52:10.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:10.896+0000 7ffb3b27b740 1 -- start start 2024-09-10T09:52:10.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:10.896+0000 7ffb3b27b740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564f1472cc70 0x564f1472f720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:52:10.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:10.896+0000 7ffb3b27b740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564f1472d590 0x564f14736cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:52:10.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:10.896+0000 7ffb3b27b740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564f14737e50 0x564f1474fc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:52:10.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:10.896+0000 7ffb3b27b740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ffb301b1010 con 0x564f14737e50 2024-09-10T09:52:10.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:10.896+0000 7ffb3b27b740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7ffb301b2b50 con 0x564f1472cc70 2024-09-10T09:52:10.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:10.896+0000 7ffb3b27b740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7ffb301b16e0 con 0x564f1472d590 2024-09-10T09:52:10.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:10.896+0000 7ffb36800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564f14737e50 0x564f1474fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:52:10.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:10.896+0000 7ffb35e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564f1472cc70 0x564f1472f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:52:10.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:10.896+0000 7ffb35e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564f1472cc70 0x564f1472f720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:38004/0 (socket says 172.21.15.5:38004) 2024-09-10T09:52:10.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:10.896+0000 7ffb35e00640 1 -- 172.21.15.5:0/4159256424 learned_addr learned my addr 172.21.15.5:0/4159256424 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:52:10.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:10.896+0000 7ffb35400640 1 --2- 172.21.15.5:0/4159256424 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564f1472d590 0x564f14736cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:52:10.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:10.897+0000 7ffb36800640 1 -- 172.21.15.5:0/4159256424 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564f1472d590 msgr2=0x564f14736cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:52:10.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:10.897+0000 7ffb36800640 1 --2- 172.21.15.5:0/4159256424 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564f1472d590 0x564f14736cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:10.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:10.897+0000 7ffb36800640 1 -- 172.21.15.5:0/4159256424 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564f1472cc70 msgr2=0x564f1472f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:52:10.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:10.897+0000 7ffb36800640 1 --2- 172.21.15.5:0/4159256424 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564f1472cc70 0x564f1472f720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:10.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:10.897+0000 7ffb36800640 1 -- 172.21.15.5:0/4159256424 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564f14729030 con 0x564f14737e50 2024-09-10T09:52:10.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:10.897+0000 7ffb36800640 1 --2- 172.21.15.5:0/4159256424 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564f14737e50 0x564f1474fc10 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7ffb2c005fd0 tx=0x7ffb2c016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=482e2235f8d8d63f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:52:10.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:10.897+0000 7ffb1e600640 1 -- 172.21.15.5:0/4159256424 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb2c003970 con 0x564f14737e50 2024-09-10T09:52:10.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:10.897+0000 7ffb1e600640 1 -- 172.21.15.5:0/4159256424 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffb2c004050 con 0x564f14737e50 2024-09-10T09:52:10.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:10.897+0000 7ffb1e600640 1 -- 172.21.15.5:0/4159256424 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb2c0208e0 con 0x564f14737e50 2024-09-10T09:52:10.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:10.898+0000 7ffb3b27b740 1 -- 172.21.15.5:0/4159256424 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564f14737e50 msgr2=0x564f1474fc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:52:10.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:10.898+0000 7ffb3b27b740 1 --2- 172.21.15.5:0/4159256424 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564f14737e50 0x564f1474fc10 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7ffb2c005fd0 tx=0x7ffb2c016000 comp rx=0 tx=0).stop 2024-09-10T09:52:10.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:10.898+0000 7ffb3b27b740 1 -- 172.21.15.5:0/4159256424 shutdown_connections 2024-09-10T09:52:10.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:10.898+0000 7ffb3b27b740 1 --2- 172.21.15.5:0/4159256424 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564f14737e50 0x564f1474fc10 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:10.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:10.898+0000 7ffb3b27b740 1 --2- 172.21.15.5:0/4159256424 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564f1472d590 0x564f14736cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:10.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:10.898+0000 7ffb3b27b740 1 --2- 172.21.15.5:0/4159256424 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564f1472cc70 0x564f1472f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:10.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:10.898+0000 7ffb3b27b740 1 -- 172.21.15.5:0/4159256424 >> 172.21.15.5:0/4159256424 conn(0x564f14729200 msgr2=0x564f147365b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:52:10.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:10.898+0000 7ffb3b27b740 1 -- 172.21.15.5:0/4159256424 shutdown_connections 2024-09-10T09:52:10.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:10.898+0000 7ffb3b27b740 1 -- 172.21.15.5:0/4159256424 wait complete. 2024-09-10T09:52:10.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:10.899+0000 7ffb3b27b740 1 -- start start 2024-09-10T09:52:10.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:10.899+0000 7ffb3b27b740 1 Processor -- start 2024-09-10T09:52:10.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:10.899+0000 7ffb3b27b740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564f1472cc70 0x564f147400f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:52:10.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:10.899+0000 7ffb3b27b740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564f1472d590 0x564f14740630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:52:10.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:10.899+0000 7ffb3b27b740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564f14737e50 0x564f14731750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:52:10.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:10.899+0000 7ffb3b27b740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ffb301b2480 con 0x564f14737e50 2024-09-10T09:52:10.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:10.899+0000 7ffb3b27b740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7ffb301b1db0 con 0x564f1472cc70 2024-09-10T09:52:10.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:10.899+0000 7ffb3b27b740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7ffb301aeff0 con 0x564f1472d590 2024-09-10T09:52:10.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:10.900+0000 7ffb3b27b740 10 client.0 _ll_register_callbacks cb 0x7ffb240d5220 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:52:10.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:10.900+0000 7ffb36800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564f14737e50 0x564f14731750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:52:10.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:10.900+0000 7ffb36800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564f14737e50 0x564f14731750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47654/0 (socket says 172.21.15.5:47654) 2024-09-10T09:52:10.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:10.900+0000 7ffb36800640 1 -- 172.21.15.5:0/603481280 learned_addr learned my addr 172.21.15.5:0/603481280 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:52:10.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:10.900+0000 7ffb35e00640 1 --2- 172.21.15.5:0/603481280 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564f1472cc70 0x564f147400f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:52:10.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:10.900+0000 7ffb35400640 1 --2- 172.21.15.5:0/603481280 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564f1472d590 0x564f14740630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:52:10.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:10.900+0000 7ffb36800640 1 -- 172.21.15.5:0/603481280 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564f1472d590 msgr2=0x564f14740630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:52:10.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:10.900+0000 7ffb36800640 1 --2- 172.21.15.5:0/603481280 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564f1472d590 0x564f14740630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:10.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:10.900+0000 7ffb36800640 1 -- 172.21.15.5:0/603481280 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564f1472cc70 msgr2=0x564f147400f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:52:10.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:10.900+0000 7ffb36800640 1 --2- 172.21.15.5:0/603481280 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564f1472cc70 0x564f147400f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:10.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:10.900+0000 7ffb36800640 1 -- 172.21.15.5:0/603481280 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564f14731f20 con 0x564f14737e50 2024-09-10T09:52:10.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:10.901+0000 7ffb36800640 1 --2- 172.21.15.5:0/603481280 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564f14737e50 0x564f14731750 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7ffb2c003ba0 tx=0x7ffb2c003bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65c08c2df165bfea server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:52:10.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:10.901+0000 7ffb13400640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-10T09:52:10.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:10.901+0000 7ffb13400640 1 -- 172.21.15.5:0/603481280 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb2c004a40 con 0x564f14737e50 2024-09-10T09:52:10.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:10.901+0000 7ffb3b27b740 1 -- 172.21.15.5:0/603481280 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffb2c004050 con 0x564f14737e50 2024-09-10T09:52:10.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:10.901+0000 7ffb13400640 1 -- 172.21.15.5:0/603481280 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7ffb2c026040 con 0x564f14737e50 2024-09-10T09:52:10.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:10.901+0000 7ffb13400640 1 -- 172.21.15.5:0/603481280 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7ffb2c03f050 con 0x564f14737e50 2024-09-10T09:52:10.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:10.901+0000 7ffb3b27b740 10 client.18177 fetch_fsmap learned FSMap version 36 2024-09-10T09:52:10.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:10.901+0000 7ffb3b27b740 1 -- 172.21.15.5:0/603481280 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x564f147b27a0 con 0x564f14737e50 2024-09-10T09:52:10.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:10.901+0000 7ffb13400640 1 -- 172.21.15.5:0/603481280 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffb2c004050 con 0x564f14737e50 2024-09-10T09:52:10.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:10.901+0000 7ffb13400640 1 -- 172.21.15.5:0/603481280 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 36) ==== 8370+0+0 (secure 0 0 0) 0x7ffb2c00f040 con 0x564f14737e50 2024-09-10T09:52:10.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:10.902+0000 7ffb3b27b740 10 client.18177 fetch_fsmap finished waiting for FSMap version 36 2024-09-10T09:52:10.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:10.902+0000 7ffb3b27b740 10 client.18177 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-09-10T09:52:10.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:10.902+0000 7ffb3b27b740 20 client.18177 populate_metadata read hostname 'smithi005' 2024-09-10T09:52:10.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:10.902+0000 7ffb3b27b740 1 --2- 172.21.15.5:0/603481280 >> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] conn(0x564f147b2e20 0x564f147b5240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:52:10.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:10.902+0000 7ffb3b27b740 4 client.18177 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T09:52:10.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:10.902+0000 7ffb3b27b740 1 -- 172.21.15.5:0/603481280 --> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564f1469ae60 con 0x564f147b2e20 2024-09-10T09:52:10.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:10.902+0000 7ffb35e00640 1 --2- 172.21.15.5:0/603481280 >> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] conn(0x564f147b2e20 0x564f147b5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:52:10.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:10.903+0000 7ffb35e00640 1 --2- 172.21.15.5:0/603481280 >> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] conn(0x564f147b2e20 0x564f147b5240 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=bd774ab2cfb80047 server_cookie=40e666cc29356423 in_seq=0 out_seq=0 2024-09-10T09:52:10.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:10.903+0000 7ffb13400640 10 client.18177 ms_handle_connect on v2:172.21.15.102:6836/2689776764 2024-09-10T09:52:10.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:10.903+0000 7ffb13400640 1 -- 172.21.15.5:0/603481280 <== mds.3 v2:172.21.15.102:6836/2689776764 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564f1469ae60 con 0x564f147b2e20 2024-09-10T09:52:10.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:10.903+0000 7ffb13400640 10 client.18177 handle_command_reply: tid=0 2024-09-10T09:52:10.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:10.903+0000 7ffb3b27b740 1 client.18177 shutdown 2024-09-10T09:52:10.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:10.903+0000 7ffb3b27b740 1 -- 172.21.15.5:0/603481280 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564f14737e50 msgr2=0x564f14731750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:52:10.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:10.903+0000 7ffb3b27b740 1 --2- 172.21.15.5:0/603481280 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564f14737e50 0x564f14731750 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7ffb2c003ba0 tx=0x7ffb2c003bd0 comp rx=0 tx=0).stop 2024-09-10T09:52:10.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:10.903+0000 7ffb3b27b740 1 -- 172.21.15.5:0/603481280 shutdown_connections 2024-09-10T09:52:10.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:10.903+0000 7ffb3b27b740 1 --2- 172.21.15.5:0/603481280 >> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] conn(0x564f147b2e20 0x564f147b5240 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:10.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:10.903+0000 7ffb3b27b740 1 --2- 172.21.15.5:0/603481280 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564f14737e50 0x564f14731750 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:10.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:10.903+0000 7ffb3b27b740 1 --2- 172.21.15.5:0/603481280 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564f1472d590 0x564f14740630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:10.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:10.903+0000 7ffb3b27b740 1 --2- 172.21.15.5:0/603481280 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564f1472cc70 0x564f147400f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:10.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:10.903+0000 7ffb3b27b740 1 -- 172.21.15.5:0/603481280 >> 172.21.15.5:0/603481280 conn(0x564f14729200 msgr2=0x564f14728380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:52:10.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:10.903+0000 7ffb13400640 0 client.18177 ms_handle_reset on v2:172.21.15.102:6836/2689776764 2024-09-10T09:52:10.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:10.903+0000 7ffb3b27b740 1 -- 172.21.15.5:0/603481280 shutdown_connections 2024-09-10T09:52:10.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:10.903+0000 7ffb3b27b740 1 -- 172.21.15.5:0/603481280 wait complete. 2024-09-10T09:52:10.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:10.904+0000 7ffb3b27b740 20 client.18177 trim_cache size 0 max 16384 2024-09-10T09:52:10.909 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-10T09:52:10.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:10.904+0000 7ffb38800640 1 -- 172.21.15.5:0/3058604487 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7ffb180a2b80 msgr2=0x7ffb180a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:52:10.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:10.904+0000 7ffb38800640 1 --2- 172.21.15.5:0/3058604487 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7ffb180a2b80 0x7ffb180a5040 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7ffb2c005f40 tx=0x7ffb2c009600 comp rx=0 tx=0).stop 2024-09-10T09:52:10.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:10.904+0000 7ffb38800640 1 -- 172.21.15.5:0/3058604487 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffb30103ff0 msgr2=0x7ffb3019a5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:52:10.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:10.904+0000 7ffb38800640 1 --2- 172.21.15.5:0/3058604487 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffb30103ff0 0x7ffb3019a5f0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7ffb2400e9f0 tx=0x7ffb2400eeb0 comp rx=0 tx=0).stop 2024-09-10T09:52:10.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:10.904+0000 7ffb38800640 1 -- 172.21.15.5:0/3058604487 shutdown_connections 2024-09-10T09:52:10.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:10.904+0000 7ffb38800640 1 --2- 172.21.15.5:0/3058604487 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7ffb180a2b80 0x7ffb180a5040 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:10.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:10.904+0000 7ffb38800640 1 --2- 172.21.15.5:0/3058604487 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ffb30109120 0x7ffb3019eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:10.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:10.904+0000 7ffb38800640 1 --2- 172.21.15.5:0/3058604487 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ffb301049a0 0x7ffb3019ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:10.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:10.904+0000 7ffb38800640 1 --2- 172.21.15.5:0/3058604487 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffb30103ff0 0x7ffb3019a5f0 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:10.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:10.904+0000 7ffb38800640 1 -- 172.21.15.5:0/3058604487 >> 172.21.15.5:0/3058604487 conn(0x7ffb300ffa90 msgr2=0x7ffb30101e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:52:10.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:10.905+0000 7ffb38800640 1 -- 172.21.15.5:0/3058604487 shutdown_connections 2024-09-10T09:52:10.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:10.905+0000 7ffb38800640 1 -- 172.21.15.5:0/3058604487 wait complete. 2024-09-10T09:52:11.171 INFO:tasks.workunit.client.0.smithi005.stdout: 1 866428 6.17 MB/sec execute 592 sec latency 12311.439 ms 2024-09-10T09:52:11.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:52:11 smithi102 ceph-mon[35442]: pgmap v808: 129 pgs: 129 active+clean; 290 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1000 KiB/s rd, 2.1 MiB/s wr, 121 op/s 2024-09-10T09:52:11.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:52:11 smithi005 ceph-mon[30350]: pgmap v808: 129 pgs: 129 active+clean; 290 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1000 KiB/s rd, 2.1 MiB/s wr, 121 op/s 2024-09-10T09:52:11.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:52:11 smithi159 ceph-mon[35513]: pgmap v808: 129 pgs: 129 active+clean; 290 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1000 KiB/s rd, 2.1 MiB/s wr, 121 op/s 2024-09-10T09:52:11.713 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-10T09:52:12.092 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:52:12.171 INFO:tasks.workunit.client.0.smithi005.stdout: 1 866428 6.16 MB/sec execute 593 sec latency 13311.529 ms 2024-09-10T09:52:12.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:12.718+0000 7f13dac00640 1 -- 172.21.15.5:0/1993867324 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f13d41049a0 msgr2=0x7f13d41089f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:52:12.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:12.718+0000 7f13dac00640 1 --2- 172.21.15.5:0/1993867324 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f13d41049a0 0x7f13d41089f0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f13bc009900 tx=0x7f13bc02a410 comp rx=0 tx=0).stop 2024-09-10T09:52:12.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:12.718+0000 7f13dac00640 1 -- 172.21.15.5:0/1993867324 shutdown_connections 2024-09-10T09:52:12.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:12.718+0000 7f13dac00640 1 --2- 172.21.15.5:0/1993867324 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f13d4109120 0x7f13d4110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:12.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:12.718+0000 7f13dac00640 1 --2- 172.21.15.5:0/1993867324 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f13d41049a0 0x7f13d41089f0 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:12.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:12.718+0000 7f13dac00640 1 --2- 172.21.15.5:0/1993867324 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f13d4103ff0 0x7f13d41043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:12.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:12.718+0000 7f13dac00640 1 -- 172.21.15.5:0/1993867324 >> 172.21.15.5:0/1993867324 conn(0x7f13d40ffa90 msgr2=0x7f13d4101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:52:12.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:12.718+0000 7f13dac00640 1 -- 172.21.15.5:0/1993867324 shutdown_connections 2024-09-10T09:52:12.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:12.718+0000 7f13dac00640 1 -- 172.21.15.5:0/1993867324 wait complete. 2024-09-10T09:52:12.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:12.719+0000 7f13dac00640 1 Processor -- start 2024-09-10T09:52:12.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:12.719+0000 7f13dac00640 1 -- start start 2024-09-10T09:52:12.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:12.719+0000 7f13dac00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f13d4103ff0 0x7f13d4076700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:52:12.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:12.720+0000 7f13dac00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f13d41049a0 0x7f13d4076c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:52:12.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:12.720+0000 7f13dac00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f13d4109120 0x7f13d4077180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:52:12.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:12.720+0000 7f13dac00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f13d4114190 con 0x7f13d41049a0 2024-09-10T09:52:12.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:12.720+0000 7f13dac00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f13d4114010 con 0x7f13d4109120 2024-09-10T09:52:12.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:12.720+0000 7f13dac00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f13d4114310 con 0x7f13d4103ff0 2024-09-10T09:52:12.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:12.720+0000 7f13d3e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f13d4103ff0 0x7f13d4076700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:52:12.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:12.720+0000 7f13d3e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f13d4103ff0 0x7f13d4076700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:41618/0 (socket says 172.21.15.5:41618) 2024-09-10T09:52:12.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:12.720+0000 7f13d3e00640 1 -- 172.21.15.5:0/2692854449 learned_addr learned my addr 172.21.15.5:0/2692854449 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:52:12.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:12.720+0000 7f13d8c00640 1 --2- 172.21.15.5:0/2692854449 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f13d4109120 0x7f13d4077180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:52:12.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:12.720+0000 7f13cbe00640 1 --2- 172.21.15.5:0/2692854449 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f13d41049a0 0x7f13d4076c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:52:12.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:12.720+0000 7f13d3e00640 1 -- 172.21.15.5:0/2692854449 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f13d4109120 msgr2=0x7f13d4077180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:52:12.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:12.720+0000 7f13d3e00640 1 --2- 172.21.15.5:0/2692854449 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f13d4109120 0x7f13d4077180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:12.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:12.720+0000 7f13d3e00640 1 -- 172.21.15.5:0/2692854449 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f13d41049a0 msgr2=0x7f13d4076c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:52:12.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:12.720+0000 7f13d3e00640 1 --2- 172.21.15.5:0/2692854449 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f13d41049a0 0x7f13d4076c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:12.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:12.720+0000 7f13d3e00640 1 -- 172.21.15.5:0/2692854449 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13d41aac90 con 0x7f13d4103ff0 2024-09-10T09:52:12.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:12.720+0000 7f13cbe00640 1 --2- 172.21.15.5:0/2692854449 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f13d41049a0 0x7f13d4076c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:52:12.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:12.721+0000 7f13d3e00640 1 --2- 172.21.15.5:0/2692854449 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f13d4103ff0 0x7f13d4076700 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f13c400ea20 tx=0x7f13c400eee0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:52:12.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:12.721+0000 7f13d1600640 1 -- 172.21.15.5:0/2692854449 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13c400cd80 con 0x7f13d4103ff0 2024-09-10T09:52:12.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:12.721+0000 7f13dac00640 1 -- 172.21.15.5:0/2692854449 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13d41aaf80 con 0x7f13d4103ff0 2024-09-10T09:52:12.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:12.721+0000 7f13d1600640 1 -- 172.21.15.5:0/2692854449 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f13c4010910 con 0x7f13d4103ff0 2024-09-10T09:52:12.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:12.721+0000 7f13d1600640 1 -- 172.21.15.5:0/2692854449 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13c4018920 con 0x7f13d4103ff0 2024-09-10T09:52:12.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:12.721+0000 7f13dac00640 1 -- 172.21.15.5:0/2692854449 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13d41ab3e0 con 0x7f13d4103ff0 2024-09-10T09:52:12.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:12.724+0000 7f13d1600640 1 -- 172.21.15.5:0/2692854449 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f13c4010ac0 con 0x7f13d4103ff0 2024-09-10T09:52:12.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:12.724+0000 7f13dd62d740 1 Processor -- start 2024-09-10T09:52:12.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:12.724+0000 7f13dd62d740 1 -- start start 2024-09-10T09:52:12.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:12.725+0000 7f13dd62d740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55883f03dc70 0x55883f03e050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:52:12.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:12.725+0000 7f13dd62d740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55883f03e590 0x55883f048910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:52:12.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:12.725+0000 7f13dd62d740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55883f048e50 0x55883f04b240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:52:12.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:12.725+0000 7f13dd62d740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55883efa43e0 con 0x55883f03e590 2024-09-10T09:52:12.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:12.725+0000 7f13dd62d740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55883ef4bd80 con 0x55883f048e50 2024-09-10T09:52:12.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:12.725+0000 7f13dd62d740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55883efa03a0 con 0x55883f03dc70 2024-09-10T09:52:12.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:12.725+0000 7f13d8c00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55883f03dc70 0x55883f03e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:52:12.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:12.725+0000 7f13d3e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55883f03e590 0x55883f048910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:52:12.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:12.725+0000 7f13d8c00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55883f03dc70 0x55883f03e050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:41634/0 (socket says 172.21.15.5:41634) 2024-09-10T09:52:12.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:12.725+0000 7f13d8c00640 1 -- 172.21.15.5:0/3526308464 learned_addr learned my addr 172.21.15.5:0/3526308464 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:52:12.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:12.725+0000 7f13cbe00640 1 --2- 172.21.15.5:0/3526308464 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55883f048e50 0x55883f04b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:52:12.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:12.726+0000 7f13d8c00640 1 -- 172.21.15.5:0/3526308464 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55883f048e50 msgr2=0x55883f04b240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:52:12.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:12.726+0000 7f13d8c00640 1 --2- 172.21.15.5:0/3526308464 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55883f048e50 0x55883f04b240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:12.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:12.726+0000 7f13d8c00640 1 -- 172.21.15.5:0/3526308464 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55883f03e590 msgr2=0x55883f048910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:52:12.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:12.726+0000 7f13d8c00640 1 --2- 172.21.15.5:0/3526308464 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55883f03e590 0x55883f048910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:12.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:12.726+0000 7f13d8c00640 1 -- 172.21.15.5:0/3526308464 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55883ef6aae0 con 0x55883f03dc70 2024-09-10T09:52:12.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:12.726+0000 7f13d8c00640 1 --2- 172.21.15.5:0/3526308464 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55883f03dc70 0x55883f03e050 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f13cc009f70 tx=0x7f13cc010800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fb54e061955ed645 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:52:12.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:12.726+0000 7f13d1600640 1 --2- 172.21.15.5:0/2692854449 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f13b80a2b80 0x7f13b80a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:52:12.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:12.726+0000 7f13c9200640 1 -- 172.21.15.5:0/3526308464 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13cc016070 con 0x55883f03dc70 2024-09-10T09:52:12.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:12.726+0000 7f13c9200640 1 -- 172.21.15.5:0/3526308464 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f13cc004d20 con 0x55883f03dc70 2024-09-10T09:52:12.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:12.726+0000 7f13c9200640 1 -- 172.21.15.5:0/3526308464 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13cc012910 con 0x55883f03dc70 2024-09-10T09:52:12.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:12.727+0000 7f13d1600640 1 -- 172.21.15.5:0/2692854449 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f13c40cf8d0 con 0x7f13d4103ff0 2024-09-10T09:52:12.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:12.727+0000 7f13d3e00640 1 --2- 172.21.15.5:0/2692854449 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f13b80a2b80 0x7f13b80a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:52:12.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:12.727+0000 7f13dd62d740 1 -- 172.21.15.5:0/3526308464 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55883f03dc70 msgr2=0x55883f03e050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:52:12.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:12.727+0000 7f13dd62d740 1 --2- 172.21.15.5:0/3526308464 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55883f03dc70 0x55883f03e050 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f13cc009f70 tx=0x7f13cc010800 comp rx=0 tx=0).stop 2024-09-10T09:52:12.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:12.727+0000 7f13dd62d740 1 -- 172.21.15.5:0/3526308464 shutdown_connections 2024-09-10T09:52:12.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:12.727+0000 7f13dd62d740 1 --2- 172.21.15.5:0/3526308464 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55883f048e50 0x55883f04b240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:12.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:12.727+0000 7f13dd62d740 1 --2- 172.21.15.5:0/3526308464 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55883f03e590 0x55883f048910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:12.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:12.727+0000 7f13dd62d740 1 --2- 172.21.15.5:0/3526308464 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55883f03dc70 0x55883f03e050 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:12.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:12.727+0000 7f13dd62d740 1 -- 172.21.15.5:0/3526308464 >> 172.21.15.5:0/3526308464 conn(0x55883f03a200 msgr2=0x55883f03a5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:52:12.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:12.727+0000 7f13d3e00640 1 --2- 172.21.15.5:0/2692854449 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f13b80a2b80 0x7f13b80a5040 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f13c40ccf80 tx=0x7f13c40d5530 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:52:12.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:12.727+0000 7f13dd62d740 1 -- 172.21.15.5:0/3526308464 shutdown_connections 2024-09-10T09:52:12.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:12.728+0000 7f13dd62d740 1 -- 172.21.15.5:0/3526308464 wait complete. 2024-09-10T09:52:12.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:12.728+0000 7f13dd62d740 1 -- start start 2024-09-10T09:52:12.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:12.728+0000 7f13dd62d740 1 Processor -- start 2024-09-10T09:52:12.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:12.729+0000 7f13dd62d740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55883f03dc70 0x55883f045ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:52:12.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:12.729+0000 7f13dd62d740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55883f03e590 0x55883f046220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:52:12.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:12.729+0000 7f13dd62d740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55883f048e50 0x55883f046760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:52:12.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:12.729+0000 7f13dd62d740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55883f04ba20 con 0x55883f048e50 2024-09-10T09:52:12.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:12.729+0000 7f13dd62d740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55883ef5f0f0 con 0x55883f03dc70 2024-09-10T09:52:12.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:12.729+0000 7f13dd62d740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55883f04bba0 con 0x55883f03e590 2024-09-10T09:52:12.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:12.729+0000 7f13dd62d740 10 client.0 _ll_register_callbacks cb 0x55883ef46bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:52:12.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:12.729+0000 7f13cbe00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55883f03e590 0x55883f046220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:52:12.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:12.729+0000 7f13cbe00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55883f03e590 0x55883f046220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:41642/0 (socket says 172.21.15.5:41642) 2024-09-10T09:52:12.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:12.729+0000 7f13cbe00640 1 -- 172.21.15.5:0/2313637557 learned_addr learned my addr 172.21.15.5:0/2313637557 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:52:12.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:12.729+0000 7f13d8c00640 1 --2- 172.21.15.5:0/2313637557 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55883f03dc70 0x55883f045ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:52:12.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:12.729+0000 7f13d8c00640 1 --2- 172.21.15.5:0/2313637557 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55883f048e50 0x55883f046760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:52:12.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:12.729+0000 7f13cbe00640 1 -- 172.21.15.5:0/2313637557 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55883f03dc70 msgr2=0x55883f045ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:52:12.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:12.730+0000 7f13cbe00640 1 --2- 172.21.15.5:0/2313637557 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55883f03dc70 0x55883f045ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:12.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:12.730+0000 7f13cbe00640 1 -- 172.21.15.5:0/2313637557 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55883f048e50 msgr2=0x55883f046760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:52:12.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:12.730+0000 7f13cbe00640 1 --2- 172.21.15.5:0/2313637557 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55883f048e50 0x55883f046760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:12.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:12.730+0000 7f13cbe00640 1 -- 172.21.15.5:0/2313637557 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55883ed153c0 con 0x55883f03e590 2024-09-10T09:52:12.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:12.730+0000 7f13d8c00640 1 --2- 172.21.15.5:0/2313637557 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55883f048e50 0x55883f046760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:52:12.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:12.730+0000 7f13cbe00640 1 --2- 172.21.15.5:0/2313637557 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55883f03e590 0x55883f046220 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f13bc009230 tx=0x7f13bc005250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f0094708a20baa4f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:52:12.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:12.730+0000 7f13c1600640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-09-10T09:52:12.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:12.730+0000 7f13c1600640 1 -- 172.21.15.5:0/2313637557 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13bc035040 con 0x55883f03e590 2024-09-10T09:52:12.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:12.730+0000 7f13dd62d740 1 -- 172.21.15.5:0/2313637557 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f13cc004d20 con 0x55883f03e590 2024-09-10T09:52:12.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:12.730+0000 7f13c1600640 1 -- 172.21.15.5:0/2313637557 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f13bc0048c0 con 0x55883f03e590 2024-09-10T09:52:12.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:12.730+0000 7f13c1600640 1 -- 172.21.15.5:0/2313637557 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f13bc04e050 con 0x55883f03e590 2024-09-10T09:52:12.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:12.730+0000 7f13dd62d740 10 client.28219 fetch_fsmap learned FSMap version 36 2024-09-10T09:52:12.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:12.731+0000 7f13dd62d740 1 -- 172.21.15.5:0/2313637557 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55883f057a40 con 0x55883f03e590 2024-09-10T09:52:12.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:12.731+0000 7f13c1600640 1 -- 172.21.15.5:0/2313637557 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f13cc004d20 con 0x55883f03e590 2024-09-10T09:52:12.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:12.731+0000 7f13c1600640 1 -- 172.21.15.5:0/2313637557 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 36) ==== 8370+0+0 (secure 0 0 0) 0x7f13bc003bc0 con 0x55883f03e590 2024-09-10T09:52:12.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:12.731+0000 7f13dd62d740 10 client.28219 fetch_fsmap finished waiting for FSMap version 36 2024-09-10T09:52:12.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:12.731+0000 7f13dd62d740 10 client.28219 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-10T09:52:12.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:12.731+0000 7f13dd62d740 20 client.28219 populate_metadata read hostname 'smithi005' 2024-09-10T09:52:12.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:12.731+0000 7f13dd62d740 1 --2- 172.21.15.5:0/2313637557 >> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] conn(0x55883f058e50 0x55883f05b270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:52:12.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:12.731+0000 7f13dd62d740 4 client.28219 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:52:12.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:12.731+0000 7f13dd62d740 1 -- 172.21.15.5:0/2313637557 --> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55883efb1950 con 0x55883f058e50 2024-09-10T09:52:12.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:12.732+0000 7f13d8c00640 1 --2- 172.21.15.5:0/2313637557 >> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] conn(0x55883f058e50 0x55883f05b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:52:12.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:12.732+0000 7f13d8c00640 1 --2- 172.21.15.5:0/2313637557 >> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] conn(0x55883f058e50 0x55883f05b270 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2cdc7a81ca6c5f3f server_cookie=c92688e1dd329626 in_seq=0 out_seq=0 2024-09-10T09:52:12.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:12.732+0000 7f13c1600640 10 client.28219 ms_handle_connect on v2:172.21.15.159:6836/620627639 2024-09-10T09:52:12.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:12.733+0000 7f13c1600640 1 -- 172.21.15.5:0/2313637557 <== mds.2 v2:172.21.15.159:6836/620627639 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55883efb1950 con 0x55883f058e50 2024-09-10T09:52:12.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:12.733+0000 7f13c1600640 10 client.28219 handle_command_reply: tid=0 2024-09-10T09:52:12.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:12.733+0000 7f13dd62d740 1 client.28219 shutdown 2024-09-10T09:52:12.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:12.733+0000 7f13dd62d740 1 -- 172.21.15.5:0/2313637557 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55883f03e590 msgr2=0x55883f046220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:52:12.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:12.733+0000 7f13dd62d740 1 --2- 172.21.15.5:0/2313637557 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55883f03e590 0x55883f046220 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f13bc009230 tx=0x7f13bc005250 comp rx=0 tx=0).stop 2024-09-10T09:52:12.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:12.733+0000 7f13dd62d740 1 -- 172.21.15.5:0/2313637557 shutdown_connections 2024-09-10T09:52:12.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:12.733+0000 7f13dd62d740 1 --2- 172.21.15.5:0/2313637557 >> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] conn(0x55883f058e50 0x55883f05b270 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:12.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:12.733+0000 7f13dd62d740 1 --2- 172.21.15.5:0/2313637557 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55883f048e50 0x55883f046760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:12.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:12.733+0000 7f13dd62d740 1 --2- 172.21.15.5:0/2313637557 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55883f03e590 0x55883f046220 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:12.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:12.733+0000 7f13dd62d740 1 --2- 172.21.15.5:0/2313637557 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55883f03dc70 0x55883f045ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:12.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:12.733+0000 7f13dd62d740 1 -- 172.21.15.5:0/2313637557 >> 172.21.15.5:0/2313637557 conn(0x55883f03a200 msgr2=0x55883f039380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:52:12.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:12.733+0000 7f13c1600640 0 client.28219 ms_handle_reset on v2:172.21.15.159:6836/620627639 2024-09-10T09:52:12.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:12.733+0000 7f13dd62d740 1 -- 172.21.15.5:0/2313637557 shutdown_connections 2024-09-10T09:52:12.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:12.733+0000 7f13dd62d740 1 -- 172.21.15.5:0/2313637557 wait complete. 2024-09-10T09:52:12.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:12.734+0000 7f13dd62d740 20 client.28219 trim_cache size 0 max 16384 2024-09-10T09:52:12.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:12.747+0000 7f13dd62d740 1 Processor -- start 2024-09-10T09:52:12.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:12.747+0000 7f13dd62d740 1 -- start start 2024-09-10T09:52:12.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:12.747+0000 7f13dd62d740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55883f03dc70 0x55883f0445b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:52:12.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:12.747+0000 7f13dd62d740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55883f03e590 0x55883f060b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:52:12.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:12.747+0000 7f13dd62d740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55883f048e50 0x55883f061080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:52:12.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:12.748+0000 7f13dd62d740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f13d41b54a0 con 0x55883f03dc70 2024-09-10T09:52:12.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:12.748+0000 7f13dd62d740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f13d41b6fe0 con 0x55883f03e590 2024-09-10T09:52:12.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:12.748+0000 7f13dd62d740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f13d41b5b70 con 0x55883f048e50 2024-09-10T09:52:12.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:12.748+0000 7f13d8c00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55883f03dc70 0x55883f0445b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:52:12.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:12.748+0000 7f13d8c00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55883f03dc70 0x55883f0445b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47690/0 (socket says 172.21.15.5:47690) 2024-09-10T09:52:12.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:12.748+0000 7f13d8c00640 1 -- 172.21.15.5:0/308080865 learned_addr learned my addr 172.21.15.5:0/308080865 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:52:12.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:12.748+0000 7f13cbe00640 1 --2- 172.21.15.5:0/308080865 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55883f03e590 0x55883f060b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:52:12.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:12.748+0000 7f13d8c00640 1 --2- 172.21.15.5:0/308080865 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55883f048e50 0x55883f061080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:52:12.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:12.748+0000 7f13d8c00640 1 -- 172.21.15.5:0/308080865 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55883f048e50 msgr2=0x55883f061080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:52:12.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:12.748+0000 7f13d8c00640 1 --2- 172.21.15.5:0/308080865 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55883f048e50 0x55883f061080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:12.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:12.748+0000 7f13d8c00640 1 -- 172.21.15.5:0/308080865 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55883f03e590 msgr2=0x55883f060b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:52:12.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:12.748+0000 7f13d8c00640 1 --2- 172.21.15.5:0/308080865 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55883f03e590 0x55883f060b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:12.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:12.748+0000 7f13d8c00640 1 -- 172.21.15.5:0/308080865 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55883f035770 con 0x55883f03dc70 2024-09-10T09:52:12.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:12.748+0000 7f13cbe00640 1 --2- 172.21.15.5:0/308080865 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55883f03e590 0x55883f060b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:52:12.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:12.749+0000 7f13d8c00640 1 --2- 172.21.15.5:0/308080865 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55883f03dc70 0x55883f0445b0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f13cc005940 tx=0x7f13cc010fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=778ae315205b25ef server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:52:12.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:12.749+0000 7f13c3e00640 1 -- 172.21.15.5:0/308080865 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13cc016040 con 0x55883f03dc70 2024-09-10T09:52:12.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:12.749+0000 7f13c3e00640 1 -- 172.21.15.5:0/308080865 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f13cc005430 con 0x55883f03dc70 2024-09-10T09:52:12.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:12.749+0000 7f13c3e00640 1 -- 172.21.15.5:0/308080865 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13cc023040 con 0x55883f03dc70 2024-09-10T09:52:12.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:12.749+0000 7f13dd62d740 1 -- 172.21.15.5:0/308080865 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55883f03dc70 msgr2=0x55883f0445b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:52:12.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:12.749+0000 7f13dd62d740 1 --2- 172.21.15.5:0/308080865 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55883f03dc70 0x55883f0445b0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f13cc005940 tx=0x7f13cc010fd0 comp rx=0 tx=0).stop 2024-09-10T09:52:12.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:12.750+0000 7f13dd62d740 1 -- 172.21.15.5:0/308080865 shutdown_connections 2024-09-10T09:52:12.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:12.750+0000 7f13dd62d740 1 --2- 172.21.15.5:0/308080865 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55883f048e50 0x55883f061080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:12.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:12.750+0000 7f13dd62d740 1 --2- 172.21.15.5:0/308080865 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55883f03e590 0x55883f060b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:12.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:12.750+0000 7f13dd62d740 1 --2- 172.21.15.5:0/308080865 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55883f03dc70 0x55883f0445b0 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:12.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:12.750+0000 7f13dd62d740 1 -- 172.21.15.5:0/308080865 >> 172.21.15.5:0/308080865 conn(0x55883f03a200 msgr2=0x55883f047d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:52:12.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:12.750+0000 7f13dd62d740 1 -- 172.21.15.5:0/308080865 shutdown_connections 2024-09-10T09:52:12.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:12.750+0000 7f13dd62d740 1 -- 172.21.15.5:0/308080865 wait complete. 2024-09-10T09:52:12.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:12.750+0000 7f13dd62d740 1 -- start start 2024-09-10T09:52:12.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:12.751+0000 7f13dd62d740 1 Processor -- start 2024-09-10T09:52:12.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:12.751+0000 7f13dd62d740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55883f03dc70 0x55883f04d230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:52:12.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:12.751+0000 7f13dd62d740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55883f03e590 0x55883f04d770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:52:12.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:12.751+0000 7f13dd62d740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55883f048e50 0x55883f0c2b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:52:12.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:12.751+0000 7f13dd62d740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f13d41b6910 con 0x55883f03e590 2024-09-10T09:52:12.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:12.751+0000 7f13dd62d740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f13d41b6240 con 0x55883f03dc70 2024-09-10T09:52:12.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:12.751+0000 7f13dd62d740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f13d41b3480 con 0x55883f048e50 2024-09-10T09:52:12.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:12.751+0000 7f13dd62d740 10 client.0 _ll_register_callbacks cb 0x7f13bc009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:52:12.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:12.751+0000 7f13cbe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55883f03e590 0x55883f04d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:52:12.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:12.751+0000 7f13cbe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55883f03e590 0x55883f04d770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47698/0 (socket says 172.21.15.5:47698) 2024-09-10T09:52:12.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:12.751+0000 7f13cbe00640 1 -- 172.21.15.5:0/4285021824 learned_addr learned my addr 172.21.15.5:0/4285021824 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:52:12.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:12.751+0000 7f13d8c00640 1 --2- 172.21.15.5:0/4285021824 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55883f03dc70 0x55883f04d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:52:12.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:12.752+0000 7f13d8c00640 1 --2- 172.21.15.5:0/4285021824 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55883f048e50 0x55883f0c2b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:52:12.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:12.752+0000 7f13cbe00640 1 -- 172.21.15.5:0/4285021824 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55883f048e50 msgr2=0x55883f0c2b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:52:12.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:12.752+0000 7f13cbe00640 1 --2- 172.21.15.5:0/4285021824 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55883f048e50 0x55883f0c2b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:12.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:12.752+0000 7f13cbe00640 1 -- 172.21.15.5:0/4285021824 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55883f03dc70 msgr2=0x55883f04d230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:52:12.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:12.752+0000 7f13cbe00640 1 --2- 172.21.15.5:0/4285021824 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55883f03dc70 0x55883f04d230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:12.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:12.752+0000 7f13cbe00640 1 -- 172.21.15.5:0/4285021824 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55883f0c3050 con 0x55883f03e590 2024-09-10T09:52:12.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:12.752+0000 7f13d8c00640 1 --2- 172.21.15.5:0/4285021824 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55883f048e50 0x55883f0c2b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:52:12.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:12.752+0000 7f13cbe00640 1 --2- 172.21.15.5:0/4285021824 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55883f03e590 0x55883f04d770 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f13bc009900 tx=0x7f13bc004610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f9e1001e5e40570 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:52:12.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:12.752+0000 7f13c1600640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-10T09:52:12.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:12.753+0000 7f13c1600640 1 -- 172.21.15.5:0/4285021824 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13bc035040 con 0x55883f03e590 2024-09-10T09:52:12.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:12.753+0000 7f13c1600640 1 -- 172.21.15.5:0/4285021824 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f13bc031980 con 0x55883f03e590 2024-09-10T09:52:12.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:12.753+0000 7f13dd62d740 1 -- 172.21.15.5:0/4285021824 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f13cc005430 con 0x55883f03e590 2024-09-10T09:52:12.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:12.753+0000 7f13c1600640 1 -- 172.21.15.5:0/4285021824 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f13bc05f050 con 0x55883f03e590 2024-09-10T09:52:12.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:12.753+0000 7f13dd62d740 10 client.18195 fetch_fsmap learned FSMap version 36 2024-09-10T09:52:12.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:12.753+0000 7f13dd62d740 1 -- 172.21.15.5:0/4285021824 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55883f0c42f0 con 0x55883f03e590 2024-09-10T09:52:12.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:12.753+0000 7f13c1600640 1 -- 172.21.15.5:0/4285021824 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f13cc005430 con 0x55883f03e590 2024-09-10T09:52:12.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:12.753+0000 7f13c1600640 1 -- 172.21.15.5:0/4285021824 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 36) ==== 8370+0+0 (secure 0 0 0) 0x7f13bc05a030 con 0x55883f03e590 2024-09-10T09:52:12.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:12.753+0000 7f13dd62d740 10 client.18195 fetch_fsmap finished waiting for FSMap version 36 2024-09-10T09:52:12.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:12.754+0000 7f13dd62d740 10 client.18195 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-10T09:52:12.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:12.754+0000 7f13dd62d740 20 client.18195 populate_metadata read hostname 'smithi005' 2024-09-10T09:52:12.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:12.754+0000 7f13dd62d740 1 --2- 172.21.15.5:0/4285021824 >> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] conn(0x55883f0c4ac0 0x55883f0c6ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:52:12.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:12.754+0000 7f13dd62d740 4 client.18195 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T09:52:12.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:12.754+0000 7f13dd62d740 1 -- 172.21.15.5:0/4285021824 --> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55883efabe60 con 0x55883f0c4ac0 2024-09-10T09:52:12.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:12.754+0000 7f13d8c00640 1 --2- 172.21.15.5:0/4285021824 >> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] conn(0x55883f0c4ac0 0x55883f0c6ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:52:12.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:12.754+0000 7f13d8c00640 1 --2- 172.21.15.5:0/4285021824 >> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] conn(0x55883f0c4ac0 0x55883f0c6ee0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e89d35d83002d92d server_cookie=7ac3cb322d955a45 in_seq=0 out_seq=0 2024-09-10T09:52:12.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:12.754+0000 7f13c1600640 10 client.18195 ms_handle_connect on v2:172.21.15.159:6836/620627639 2024-09-10T09:52:12.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:12.755+0000 7f13c1600640 1 -- 172.21.15.5:0/4285021824 <== mds.2 v2:172.21.15.159:6836/620627639 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55883efabe60 con 0x55883f0c4ac0 2024-09-10T09:52:12.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:12.755+0000 7f13c1600640 10 client.18195 handle_command_reply: tid=0 2024-09-10T09:52:12.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:12.755+0000 7f13dd62d740 1 client.18195 shutdown 2024-09-10T09:52:12.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:12.755+0000 7f13dd62d740 1 -- 172.21.15.5:0/4285021824 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55883f03e590 msgr2=0x55883f04d770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:52:12.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:12.755+0000 7f13dd62d740 1 --2- 172.21.15.5:0/4285021824 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55883f03e590 0x55883f04d770 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f13bc009900 tx=0x7f13bc004610 comp rx=0 tx=0).stop 2024-09-10T09:52:12.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:12.755+0000 7f13dd62d740 1 -- 172.21.15.5:0/4285021824 shutdown_connections 2024-09-10T09:52:12.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:12.755+0000 7f13dd62d740 1 --2- 172.21.15.5:0/4285021824 >> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] conn(0x55883f0c4ac0 0x55883f0c6ee0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:12.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:12.755+0000 7f13dd62d740 1 --2- 172.21.15.5:0/4285021824 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55883f048e50 0x55883f0c2b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:12.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:12.755+0000 7f13dd62d740 1 --2- 172.21.15.5:0/4285021824 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55883f03e590 0x55883f04d770 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:12.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:12.755+0000 7f13dd62d740 1 --2- 172.21.15.5:0/4285021824 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55883f03dc70 0x55883f04d230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:12.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:12.755+0000 7f13dd62d740 1 -- 172.21.15.5:0/4285021824 >> 172.21.15.5:0/4285021824 conn(0x55883f03a200 msgr2=0x55883f047540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:52:12.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:12.755+0000 7f13c1600640 0 client.18195 ms_handle_reset on v2:172.21.15.159:6836/620627639 2024-09-10T09:52:12.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:12.755+0000 7f13dd62d740 1 -- 172.21.15.5:0/4285021824 shutdown_connections 2024-09-10T09:52:12.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:12.755+0000 7f13dd62d740 1 -- 172.21.15.5:0/4285021824 wait complete. 2024-09-10T09:52:12.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:12.756+0000 7f13dd62d740 20 client.18195 trim_cache size 0 max 16384 2024-09-10T09:52:12.761 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-10T09:52:12.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:12.756+0000 7f13dac00640 1 -- 172.21.15.5:0/2692854449 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f13b80a2b80 msgr2=0x7f13b80a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:52:12.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:12.756+0000 7f13dac00640 1 --2- 172.21.15.5:0/2692854449 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f13b80a2b80 0x7f13b80a5040 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f13c40ccf80 tx=0x7f13c40d5530 comp rx=0 tx=0).stop 2024-09-10T09:52:12.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:12.756+0000 7f13dac00640 1 -- 172.21.15.5:0/2692854449 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f13d4103ff0 msgr2=0x7f13d4076700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:52:12.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:12.756+0000 7f13dac00640 1 --2- 172.21.15.5:0/2692854449 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f13d4103ff0 0x7f13d4076700 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f13c400ea20 tx=0x7f13c400eee0 comp rx=0 tx=0).stop 2024-09-10T09:52:12.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:12.756+0000 7f13dac00640 1 -- 172.21.15.5:0/2692854449 shutdown_connections 2024-09-10T09:52:12.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:12.756+0000 7f13dac00640 1 --2- 172.21.15.5:0/2692854449 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f13b80a2b80 0x7f13b80a5040 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:12.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:12.756+0000 7f13dac00640 1 --2- 172.21.15.5:0/2692854449 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f13d4109120 0x7f13d4077180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:12.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:12.756+0000 7f13dac00640 1 --2- 172.21.15.5:0/2692854449 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f13d41049a0 0x7f13d4076c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:12.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:12.756+0000 7f13dac00640 1 --2- 172.21.15.5:0/2692854449 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f13d4103ff0 0x7f13d4076700 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:12.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:12.756+0000 7f13dac00640 1 -- 172.21.15.5:0/2692854449 >> 172.21.15.5:0/2692854449 conn(0x7f13d40ffa90 msgr2=0x7f13d4101e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:52:12.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:12.757+0000 7f13dac00640 1 -- 172.21.15.5:0/2692854449 shutdown_connections 2024-09-10T09:52:12.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:12.757+0000 7f13dac00640 1 -- 172.21.15.5:0/2692854449 wait complete. 2024-09-10T09:52:13.171 INFO:tasks.workunit.client.0.smithi005.stdout: 1 866428 6.15 MB/sec execute 594 sec latency 14311.679 ms 2024-09-10T09:52:13.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:52:13 smithi102 ceph-mon[35442]: pgmap v809: 129 pgs: 129 active+clean; 288 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 214 KiB/s rd, 1.3 MiB/s wr, 61 op/s 2024-09-10T09:52:13.484 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:52:13 smithi005 ceph-mon[30350]: pgmap v809: 129 pgs: 129 active+clean; 288 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 214 KiB/s rd, 1.3 MiB/s wr, 61 op/s 2024-09-10T09:52:13.521 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-10T09:52:13.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:52:13 smithi159 ceph-mon[35513]: pgmap v809: 129 pgs: 129 active+clean; 288 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 214 KiB/s rd, 1.3 MiB/s wr, 61 op/s 2024-09-10T09:52:13.901 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:52:14.171 INFO:tasks.workunit.client.0.smithi005.stdout: 1 866428 6.14 MB/sec execute 595 sec latency 15311.828 ms 2024-09-10T09:52:14.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:14.652+0000 7fc56e200640 1 -- 172.21.15.5:0/1803635521 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc568101de0 msgr2=0x7fc5681021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:52:14.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:14.652+0000 7fc56e200640 1 --2- 172.21.15.5:0/1803635521 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc568101de0 0x7fc5681021c0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fc558009900 tx=0x7fc55802a3f0 comp rx=0 tx=0).stop 2024-09-10T09:52:14.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:14.652+0000 7fc56e200640 1 -- 172.21.15.5:0/1803635521 shutdown_connections 2024-09-10T09:52:14.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:14.652+0000 7fc56e200640 1 --2- 172.21.15.5:0/1803635521 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc568106f10 0x7fc56810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:14.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:14.652+0000 7fc56e200640 1 --2- 172.21.15.5:0/1803635521 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc568102790 0x7fc5681067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:14.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:14.652+0000 7fc56e200640 1 --2- 172.21.15.5:0/1803635521 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc568101de0 0x7fc5681021c0 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:14.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:14.652+0000 7fc56e200640 1 -- 172.21.15.5:0/1803635521 >> 172.21.15.5:0/1803635521 conn(0x7fc5680fd8c0 msgr2=0x7fc5680ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:52:14.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:14.653+0000 7fc56e200640 1 -- 172.21.15.5:0/1803635521 shutdown_connections 2024-09-10T09:52:14.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:14.653+0000 7fc56e200640 1 -- 172.21.15.5:0/1803635521 wait complete. 2024-09-10T09:52:14.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:14.654+0000 7fc56e200640 1 Processor -- start 2024-09-10T09:52:14.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:14.654+0000 7fc56e200640 1 -- start start 2024-09-10T09:52:14.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:14.654+0000 7fc56e200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc568101de0 0x7fc56819a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:52:14.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:14.654+0000 7fc56e200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc568102790 0x7fc56819ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:52:14.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:14.654+0000 7fc56e200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc568106f10 0x7fc56819efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:52:14.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:14.654+0000 7fc56e200640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc568111f70 con 0x7fc568102790 2024-09-10T09:52:14.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:14.654+0000 7fc56e200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fc568111df0 con 0x7fc568106f10 2024-09-10T09:52:14.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:14.654+0000 7fc56e200640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fc5681120f0 con 0x7fc568101de0 2024-09-10T09:52:14.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:14.654+0000 7fc55fe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc568102790 0x7fc56819ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:52:14.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:14.654+0000 7fc567400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc568101de0 0x7fc56819a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:52:14.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:14.654+0000 7fc55fe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc568102790 0x7fc56819ac30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47716/0 (socket says 172.21.15.5:47716) 2024-09-10T09:52:14.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:14.654+0000 7fc55fe00640 1 -- 172.21.15.5:0/3930152706 learned_addr learned my addr 172.21.15.5:0/3930152706 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:52:14.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:14.655+0000 7fc567e00640 1 --2- 172.21.15.5:0/3930152706 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc568106f10 0x7fc56819efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:52:14.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:14.655+0000 7fc55fe00640 1 -- 172.21.15.5:0/3930152706 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc568101de0 msgr2=0x7fc56819a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:52:14.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:14.655+0000 7fc55fe00640 1 --2- 172.21.15.5:0/3930152706 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc568101de0 0x7fc56819a6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:14.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:14.655+0000 7fc55fe00640 1 -- 172.21.15.5:0/3930152706 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc568106f10 msgr2=0x7fc56819efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:52:14.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:14.655+0000 7fc55fe00640 1 --2- 172.21.15.5:0/3930152706 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc568106f10 0x7fc56819efc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:14.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:14.655+0000 7fc55fe00640 1 -- 172.21.15.5:0/3930152706 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc56819f6a0 con 0x7fc568102790 2024-09-10T09:52:14.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:14.655+0000 7fc55fe00640 1 --2- 172.21.15.5:0/3930152706 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc568102790 0x7fc56819ac30 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fc55000e9f0 tx=0x7fc55000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:52:14.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:14.655+0000 7fc564c00640 1 -- 172.21.15.5:0/3930152706 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc55000cd80 con 0x7fc568102790 2024-09-10T09:52:14.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:14.655+0000 7fc564c00640 1 -- 172.21.15.5:0/3930152706 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc550005500 con 0x7fc568102790 2024-09-10T09:52:14.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:14.655+0000 7fc564c00640 1 -- 172.21.15.5:0/3930152706 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc550010690 con 0x7fc568102790 2024-09-10T09:52:14.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:14.656+0000 7fc56e200640 1 -- 172.21.15.5:0/3930152706 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc56819f990 con 0x7fc568102790 2024-09-10T09:52:14.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:14.656+0000 7fc56e200640 1 -- 172.21.15.5:0/3930152706 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5681a7270 con 0x7fc568102790 2024-09-10T09:52:14.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:14.659+0000 7fc564c00640 1 -- 172.21.15.5:0/3930152706 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7fc5500046c0 con 0x7fc568102790 2024-09-10T09:52:14.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:14.659+0000 7fc570e25740 1 Processor -- start 2024-09-10T09:52:14.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:14.659+0000 7fc570e25740 1 -- start start 2024-09-10T09:52:14.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:14.659+0000 7fc570e25740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564ae96d1c70 0x564ae96d2050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:52:14.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:14.659+0000 7fc570e25740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564ae96d2590 0x564ae96dc910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:52:14.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:14.659+0000 7fc570e25740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564ae96dce50 0x564ae96df240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:52:14.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:14.659+0000 7fc570e25740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x564ae96383e0 con 0x564ae96dce50 2024-09-10T09:52:14.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:14.659+0000 7fc570e25740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x564ae95dfd80 con 0x564ae96d1c70 2024-09-10T09:52:14.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:14.659+0000 7fc570e25740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x564ae96343a0 con 0x564ae96d2590 2024-09-10T09:52:14.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:14.660+0000 7fc567400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564ae96d2590 0x564ae96dc910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:52:14.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:14.660+0000 7fc567400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564ae96d2590 0x564ae96dc910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:41696/0 (socket says 172.21.15.5:41696) 2024-09-10T09:52:14.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:14.660+0000 7fc567400640 1 -- 172.21.15.5:0/3171307758 learned_addr learned my addr 172.21.15.5:0/3171307758 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:52:14.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:14.660+0000 7fc567e00640 1 --2- 172.21.15.5:0/3171307758 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564ae96d1c70 0x564ae96d2050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:52:14.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:14.660+0000 7fc55fe00640 1 --2- 172.21.15.5:0/3171307758 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564ae96dce50 0x564ae96df240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:52:14.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:14.660+0000 7fc567e00640 1 -- 172.21.15.5:0/3171307758 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564ae96d2590 msgr2=0x564ae96dc910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:52:14.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:14.660+0000 7fc567e00640 1 --2- 172.21.15.5:0/3171307758 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564ae96d2590 0x564ae96dc910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:14.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:14.660+0000 7fc567e00640 1 -- 172.21.15.5:0/3171307758 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564ae96dce50 msgr2=0x564ae96df240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:52:14.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:14.660+0000 7fc567e00640 1 --2- 172.21.15.5:0/3171307758 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564ae96dce50 0x564ae96df240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:14.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:14.660+0000 7fc567e00640 1 -- 172.21.15.5:0/3171307758 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564ae95feae0 con 0x564ae96d1c70 2024-09-10T09:52:14.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:14.660+0000 7fc567e00640 1 --2- 172.21.15.5:0/3171307758 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564ae96d1c70 0x564ae96d2050 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fc560009eb0 tx=0x7fc5600107a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d600e1604f98f366 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:52:14.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:14.661+0000 7fc564c00640 1 --2- 172.21.15.5:0/3930152706 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fc54c0a2c00 0x7fc54c0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:52:14.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:14.661+0000 7fc557e00640 1 -- 172.21.15.5:0/3171307758 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc560003b70 con 0x564ae96d1c70 2024-09-10T09:52:14.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:14.661+0000 7fc557e00640 1 -- 172.21.15.5:0/3171307758 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc560015070 con 0x564ae96d1c70 2024-09-10T09:52:14.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:14.661+0000 7fc557e00640 1 -- 172.21.15.5:0/3171307758 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc56001a8d0 con 0x564ae96d1c70 2024-09-10T09:52:14.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:14.661+0000 7fc564c00640 1 -- 172.21.15.5:0/3930152706 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fc5500cf810 con 0x7fc568102790 2024-09-10T09:52:14.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:14.661+0000 7fc567400640 1 --2- 172.21.15.5:0/3930152706 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fc54c0a2c00 0x7fc54c0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:52:14.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:14.662+0000 7fc570e25740 1 -- 172.21.15.5:0/3171307758 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564ae96d1c70 msgr2=0x564ae96d2050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:52:14.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:14.662+0000 7fc570e25740 1 --2- 172.21.15.5:0/3171307758 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564ae96d1c70 0x564ae96d2050 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fc560009eb0 tx=0x7fc5600107a0 comp rx=0 tx=0).stop 2024-09-10T09:52:14.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:14.662+0000 7fc570e25740 1 -- 172.21.15.5:0/3171307758 shutdown_connections 2024-09-10T09:52:14.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:14.662+0000 7fc570e25740 1 --2- 172.21.15.5:0/3171307758 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564ae96dce50 0x564ae96df240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:14.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:14.662+0000 7fc570e25740 1 --2- 172.21.15.5:0/3171307758 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564ae96d2590 0x564ae96dc910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:14.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:14.662+0000 7fc570e25740 1 --2- 172.21.15.5:0/3171307758 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564ae96d1c70 0x564ae96d2050 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:14.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:14.662+0000 7fc570e25740 1 -- 172.21.15.5:0/3171307758 >> 172.21.15.5:0/3171307758 conn(0x564ae96ce200 msgr2=0x564ae96ce5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:52:14.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:14.662+0000 7fc570e25740 1 -- 172.21.15.5:0/3171307758 shutdown_connections 2024-09-10T09:52:14.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:14.662+0000 7fc567400640 1 --2- 172.21.15.5:0/3930152706 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fc54c0a2c00 0x7fc54c0a50c0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fc5580096c0 tx=0x7fc5580058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:52:14.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:14.662+0000 7fc570e25740 1 -- 172.21.15.5:0/3171307758 wait complete. 2024-09-10T09:52:14.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:14.662+0000 7fc570e25740 1 -- start start 2024-09-10T09:52:14.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:14.663+0000 7fc570e25740 1 Processor -- start 2024-09-10T09:52:14.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:14.663+0000 7fc570e25740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564ae96d1c70 0x564ae96d9d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:52:14.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:14.663+0000 7fc570e25740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564ae96d2590 0x564ae96da2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:52:14.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:14.663+0000 7fc570e25740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564ae96dce50 0x564ae96da7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:52:14.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:14.663+0000 7fc570e25740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x564ae96dfa20 con 0x564ae96dce50 2024-09-10T09:52:14.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:14.663+0000 7fc570e25740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x564ae95f30f0 con 0x564ae96d2590 2024-09-10T09:52:14.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:14.663+0000 7fc570e25740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x564ae96dfba0 con 0x564ae96d1c70 2024-09-10T09:52:14.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:14.663+0000 7fc570e25740 10 client.0 _ll_register_callbacks cb 0x564ae95dabd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:52:14.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:14.663+0000 7fc567e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564ae96dce50 0x564ae96da7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:52:14.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:14.664+0000 7fc567e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564ae96dce50 0x564ae96da7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47734/0 (socket says 172.21.15.5:47734) 2024-09-10T09:52:14.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:14.664+0000 7fc567e00640 1 -- 172.21.15.5:0/2139715127 learned_addr learned my addr 172.21.15.5:0/2139715127 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:52:14.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:14.664+0000 7fc55fe00640 1 --2- 172.21.15.5:0/2139715127 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564ae96d2590 0x564ae96da2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:52:14.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:14.664+0000 7fc567400640 1 --2- 172.21.15.5:0/2139715127 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564ae96d1c70 0x564ae96d9d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:52:14.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:14.664+0000 7fc55fe00640 1 -- 172.21.15.5:0/2139715127 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564ae96d1c70 msgr2=0x564ae96d9d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:52:14.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:14.664+0000 7fc55fe00640 1 --2- 172.21.15.5:0/2139715127 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564ae96d1c70 0x564ae96d9d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:14.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:14.664+0000 7fc55fe00640 1 -- 172.21.15.5:0/2139715127 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564ae96dce50 msgr2=0x564ae96da7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:52:14.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:14.664+0000 7fc55fe00640 1 --2- 172.21.15.5:0/2139715127 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564ae96dce50 0x564ae96da7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:14.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:14.664+0000 7fc55fe00640 1 -- 172.21.15.5:0/2139715127 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564ae93a93c0 con 0x564ae96d2590 2024-09-10T09:52:14.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:14.664+0000 7fc55fe00640 1 --2- 172.21.15.5:0/2139715127 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564ae96d2590 0x564ae96da2a0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fc5500cce90 tx=0x7fc550097f80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=681e1ed8827f6720 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:52:14.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:14.664+0000 7fc554c00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-09-10T09:52:14.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:14.664+0000 7fc554c00640 1 -- 172.21.15.5:0/2139715127 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5500a4600 con 0x564ae96d2590 2024-09-10T09:52:14.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:14.665+0000 7fc554c00640 1 -- 172.21.15.5:0/2139715127 <== mon.2 v2:172.21.15.102:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fc55009b070 con 0x564ae96d2590 2024-09-10T09:52:14.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:14.665+0000 7fc570e25740 1 -- 172.21.15.5:0/2139715127 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc560015070 con 0x564ae96d2590 2024-09-10T09:52:14.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:14.665+0000 7fc554c00640 1 -- 172.21.15.5:0/2139715127 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fc55009c320 con 0x564ae96d2590 2024-09-10T09:52:14.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:14.665+0000 7fc570e25740 10 client.28238 fetch_fsmap learned FSMap version 36 2024-09-10T09:52:14.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:14.665+0000 7fc570e25740 1 -- 172.21.15.5:0/2139715127 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x564ae96ebb20 con 0x564ae96d2590 2024-09-10T09:52:14.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:14.665+0000 7fc554c00640 1 -- 172.21.15.5:0/2139715127 <== mon.2 v2:172.21.15.102:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc560015070 con 0x564ae96d2590 2024-09-10T09:52:14.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:14.665+0000 7fc554c00640 1 -- 172.21.15.5:0/2139715127 <== mon.2 v2:172.21.15.102:3300/0 5 ==== fsmap(e 36) ==== 8370+0+0 (secure 0 0 0) 0x7fc55009aae0 con 0x564ae96d2590 2024-09-10T09:52:14.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:14.665+0000 7fc570e25740 10 client.28238 fetch_fsmap finished waiting for FSMap version 36 2024-09-10T09:52:14.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:14.665+0000 7fc570e25740 10 client.28238 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-10T09:52:14.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:14.665+0000 7fc570e25740 20 client.28238 populate_metadata read hostname 'smithi005' 2024-09-10T09:52:14.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:14.665+0000 7fc570e25740 1 --2- 172.21.15.5:0/2139715127 >> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] conn(0x564ae96ecfe0 0x564ae96ef3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:52:14.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:14.665+0000 7fc570e25740 4 client.28238 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:52:14.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:14.665+0000 7fc570e25740 1 -- 172.21.15.5:0/2139715127 --> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564ae9645950 con 0x564ae96ecfe0 2024-09-10T09:52:14.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:14.666+0000 7fc567400640 1 --2- 172.21.15.5:0/2139715127 >> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] conn(0x564ae96ecfe0 0x564ae96ef3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:52:14.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:14.666+0000 7fc567400640 1 --2- 172.21.15.5:0/2139715127 >> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] conn(0x564ae96ecfe0 0x564ae96ef3d0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1126c7abcd64da9 server_cookie=1ef74eef00e17f80 in_seq=0 out_seq=0 2024-09-10T09:52:14.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:14.666+0000 7fc554c00640 10 client.28238 ms_handle_connect on v2:172.21.15.159:6838/2073739599 2024-09-10T09:52:14.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:14.667+0000 7fc554c00640 1 -- 172.21.15.5:0/2139715127 <== mds.1 v2:172.21.15.159:6838/2073739599 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564ae9645950 con 0x564ae96ecfe0 2024-09-10T09:52:14.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:14.667+0000 7fc554c00640 10 client.28238 handle_command_reply: tid=0 2024-09-10T09:52:14.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:14.667+0000 7fc570e25740 1 client.28238 shutdown 2024-09-10T09:52:14.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:14.668+0000 7fc570e25740 1 -- 172.21.15.5:0/2139715127 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564ae96d2590 msgr2=0x564ae96da2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:52:14.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:14.668+0000 7fc570e25740 1 --2- 172.21.15.5:0/2139715127 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564ae96d2590 0x564ae96da2a0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fc5500cce90 tx=0x7fc550097f80 comp rx=0 tx=0).stop 2024-09-10T09:52:14.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:14.668+0000 7fc570e25740 1 -- 172.21.15.5:0/2139715127 shutdown_connections 2024-09-10T09:52:14.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:14.668+0000 7fc570e25740 1 --2- 172.21.15.5:0/2139715127 >> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] conn(0x564ae96ecfe0 0x564ae96ef3d0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:14.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:14.668+0000 7fc570e25740 1 --2- 172.21.15.5:0/2139715127 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564ae96dce50 0x564ae96da7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:14.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:14.668+0000 7fc570e25740 1 --2- 172.21.15.5:0/2139715127 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564ae96d2590 0x564ae96da2a0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:14.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:14.668+0000 7fc554c00640 0 client.28238 ms_handle_reset on v2:172.21.15.159:6838/2073739599 2024-09-10T09:52:14.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:14.668+0000 7fc570e25740 1 --2- 172.21.15.5:0/2139715127 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564ae96d1c70 0x564ae96d9d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:14.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:14.668+0000 7fc570e25740 1 -- 172.21.15.5:0/2139715127 >> 172.21.15.5:0/2139715127 conn(0x564ae96ce200 msgr2=0x564ae96cd380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:52:14.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:14.668+0000 7fc570e25740 1 -- 172.21.15.5:0/2139715127 shutdown_connections 2024-09-10T09:52:14.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:14.668+0000 7fc570e25740 1 -- 172.21.15.5:0/2139715127 wait complete. 2024-09-10T09:52:14.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:14.668+0000 7fc570e25740 20 client.28238 trim_cache size 0 max 16384 2024-09-10T09:52:14.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:14.681+0000 7fc570e25740 1 Processor -- start 2024-09-10T09:52:14.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:14.682+0000 7fc570e25740 1 -- start start 2024-09-10T09:52:14.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:14.682+0000 7fc570e25740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564ae96d1c70 0x564ae96dbcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:52:14.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:14.682+0000 7fc570e25740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564ae96d2590 0x564ae96f4ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:52:14.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:14.682+0000 7fc570e25740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564ae96dce50 0x564ae96f51e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:52:14.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:14.682+0000 7fc570e25740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc5681b10a0 con 0x564ae96dce50 2024-09-10T09:52:14.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:14.682+0000 7fc570e25740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fc5681b2be0 con 0x564ae96d1c70 2024-09-10T09:52:14.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:14.682+0000 7fc570e25740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fc5681b1770 con 0x564ae96d2590 2024-09-10T09:52:14.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:14.682+0000 7fc567e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564ae96dce50 0x564ae96f51e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:52:14.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:14.682+0000 7fc567e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564ae96dce50 0x564ae96f51e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47746/0 (socket says 172.21.15.5:47746) 2024-09-10T09:52:14.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:14.682+0000 7fc567e00640 1 -- 172.21.15.5:0/1354553011 learned_addr learned my addr 172.21.15.5:0/1354553011 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:52:14.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:14.682+0000 7fc567400640 1 --2- 172.21.15.5:0/1354553011 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564ae96d1c70 0x564ae96dbcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:52:14.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:14.682+0000 7fc55fe00640 1 --2- 172.21.15.5:0/1354553011 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564ae96d2590 0x564ae96f4ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:52:14.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:14.683+0000 7fc567e00640 1 -- 172.21.15.5:0/1354553011 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564ae96d2590 msgr2=0x564ae96f4ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:52:14.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:14.683+0000 7fc567e00640 1 --2- 172.21.15.5:0/1354553011 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564ae96d2590 0x564ae96f4ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:14.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:14.683+0000 7fc567e00640 1 -- 172.21.15.5:0/1354553011 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564ae96d1c70 msgr2=0x564ae96dbcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:52:14.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:14.683+0000 7fc567e00640 1 --2- 172.21.15.5:0/1354553011 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564ae96d1c70 0x564ae96dbcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:14.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:14.683+0000 7fc567e00640 1 -- 172.21.15.5:0/1354553011 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564ae96c9770 con 0x564ae96dce50 2024-09-10T09:52:14.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:14.683+0000 7fc567e00640 1 --2- 172.21.15.5:0/1354553011 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564ae96dce50 0x564ae96f51e0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fc560006010 tx=0x7fc560005bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9544c2bd55ba613 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:52:14.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:14.683+0000 7fc557400640 1 -- 172.21.15.5:0/1354553011 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc56002b070 con 0x564ae96dce50 2024-09-10T09:52:14.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:14.683+0000 7fc557400640 1 -- 172.21.15.5:0/1354553011 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc560015040 con 0x564ae96dce50 2024-09-10T09:52:14.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:14.683+0000 7fc557400640 1 -- 172.21.15.5:0/1354553011 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc56001a800 con 0x564ae96dce50 2024-09-10T09:52:14.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:14.684+0000 7fc570e25740 1 -- 172.21.15.5:0/1354553011 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564ae96dce50 msgr2=0x564ae96f51e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:52:14.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:14.684+0000 7fc570e25740 1 --2- 172.21.15.5:0/1354553011 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564ae96dce50 0x564ae96f51e0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fc560006010 tx=0x7fc560005bd0 comp rx=0 tx=0).stop 2024-09-10T09:52:14.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:14.684+0000 7fc570e25740 1 -- 172.21.15.5:0/1354553011 shutdown_connections 2024-09-10T09:52:14.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:14.684+0000 7fc570e25740 1 --2- 172.21.15.5:0/1354553011 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564ae96dce50 0x564ae96f51e0 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:14.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:14.684+0000 7fc570e25740 1 --2- 172.21.15.5:0/1354553011 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564ae96d2590 0x564ae96f4ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:14.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:14.684+0000 7fc570e25740 1 --2- 172.21.15.5:0/1354553011 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564ae96d1c70 0x564ae96dbcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:14.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:14.684+0000 7fc570e25740 1 -- 172.21.15.5:0/1354553011 >> 172.21.15.5:0/1354553011 conn(0x564ae96ce200 msgr2=0x564ae96db5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:52:14.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:14.684+0000 7fc570e25740 1 -- 172.21.15.5:0/1354553011 shutdown_connections 2024-09-10T09:52:14.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:14.684+0000 7fc570e25740 1 -- 172.21.15.5:0/1354553011 wait complete. 2024-09-10T09:52:14.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:14.685+0000 7fc570e25740 1 -- start start 2024-09-10T09:52:14.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:14.685+0000 7fc570e25740 1 Processor -- start 2024-09-10T09:52:14.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:14.685+0000 7fc570e25740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564ae96d1c70 0x564ae96e0b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:52:14.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:14.685+0000 7fc570e25740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564ae96d2590 0x564ae96e1040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:52:14.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:14.685+0000 7fc570e25740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564ae96dce50 0x564ae96e1580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:52:14.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:14.685+0000 7fc570e25740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc5681b2510 con 0x564ae96d2590 2024-09-10T09:52:14.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:14.685+0000 7fc570e25740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fc5681b1e40 con 0x564ae96dce50 2024-09-10T09:52:14.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:14.685+0000 7fc570e25740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fc5681af080 con 0x564ae96d1c70 2024-09-10T09:52:14.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:14.685+0000 7fc570e25740 10 client.0 _ll_register_callbacks cb 0x7fc5500d5220 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:52:14.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:14.686+0000 7fc567e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564ae96dce50 0x564ae96e1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:52:14.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:14.686+0000 7fc55fe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564ae96d2590 0x564ae96e1040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:52:14.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:14.686+0000 7fc567e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564ae96dce50 0x564ae96e1580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:38110/0 (socket says 172.21.15.5:38110) 2024-09-10T09:52:14.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:14.686+0000 7fc567e00640 1 -- 172.21.15.5:0/1660596406 learned_addr learned my addr 172.21.15.5:0/1660596406 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:52:14.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:14.686+0000 7fc567400640 1 --2- 172.21.15.5:0/1660596406 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564ae96d1c70 0x564ae96e0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:52:14.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:14.686+0000 7fc567e00640 1 -- 172.21.15.5:0/1660596406 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564ae96d1c70 msgr2=0x564ae96e0b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:52:14.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:14.686+0000 7fc567e00640 1 --2- 172.21.15.5:0/1660596406 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564ae96d1c70 0x564ae96e0b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:14.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:14.686+0000 7fc567e00640 1 -- 172.21.15.5:0/1660596406 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564ae96d2590 msgr2=0x564ae96e1040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:52:14.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:14.686+0000 7fc567e00640 1 --2- 172.21.15.5:0/1660596406 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564ae96d2590 0x564ae96e1040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:14.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:14.686+0000 7fc567e00640 1 -- 172.21.15.5:0/1660596406 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564ae96cee90 con 0x564ae96dce50 2024-09-10T09:52:14.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:14.686+0000 7fc567e00640 1 --2- 172.21.15.5:0/1660596406 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564ae96dce50 0x564ae96e1580 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fc5600096e0 tx=0x7fc560004870 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3360b056d4303626 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:52:14.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:14.686+0000 7fc554c00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-09-10T09:52:14.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:14.687+0000 7fc554c00640 1 -- 172.21.15.5:0/1660596406 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc560035060 con 0x564ae96dce50 2024-09-10T09:52:14.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:14.687+0000 7fc570e25740 1 -- 172.21.15.5:0/1660596406 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc560015040 con 0x564ae96dce50 2024-09-10T09:52:14.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:14.687+0000 7fc554c00640 1 -- 172.21.15.5:0/1660596406 <== mon.2 v2:172.21.15.102:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fc560005040 con 0x564ae96dce50 2024-09-10T09:52:14.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:14.687+0000 7fc554c00640 1 -- 172.21.15.5:0/1660596406 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fc56003c050 con 0x564ae96dce50 2024-09-10T09:52:14.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:14.687+0000 7fc570e25740 10 client.28244 fetch_fsmap learned FSMap version 36 2024-09-10T09:52:14.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:14.687+0000 7fc570e25740 1 -- 172.21.15.5:0/1660596406 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x564ae9757cc0 con 0x564ae96dce50 2024-09-10T09:52:14.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:14.687+0000 7fc554c00640 1 -- 172.21.15.5:0/1660596406 <== mon.2 v2:172.21.15.102:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc560015040 con 0x564ae96dce50 2024-09-10T09:52:14.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:14.687+0000 7fc554c00640 1 -- 172.21.15.5:0/1660596406 <== mon.2 v2:172.21.15.102:3300/0 5 ==== fsmap(e 36) ==== 8370+0+0 (secure 0 0 0) 0x7fc560025030 con 0x564ae96dce50 2024-09-10T09:52:14.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:14.687+0000 7fc570e25740 10 client.28244 fetch_fsmap finished waiting for FSMap version 36 2024-09-10T09:52:14.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:14.687+0000 7fc570e25740 10 client.28244 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-10T09:52:14.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:14.688+0000 7fc570e25740 20 client.28244 populate_metadata read hostname 'smithi005' 2024-09-10T09:52:14.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:14.688+0000 7fc570e25740 1 --2- 172.21.15.5:0/1660596406 >> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] conn(0x564ae97580e0 0x564ae975a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:52:14.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:14.688+0000 7fc570e25740 4 client.28244 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T09:52:14.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:14.688+0000 7fc570e25740 1 -- 172.21.15.5:0/1660596406 --> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564ae963fe60 con 0x564ae97580e0 2024-09-10T09:52:14.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:14.688+0000 7fc567400640 1 --2- 172.21.15.5:0/1660596406 >> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] conn(0x564ae97580e0 0x564ae975a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:52:14.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:14.688+0000 7fc567400640 1 --2- 172.21.15.5:0/1660596406 >> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] conn(0x564ae97580e0 0x564ae975a4d0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7538636f3d308d37 server_cookie=fd453d93248a1d11 in_seq=0 out_seq=0 2024-09-10T09:52:14.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:14.688+0000 7fc554c00640 10 client.28244 ms_handle_connect on v2:172.21.15.159:6838/2073739599 2024-09-10T09:52:14.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:14.689+0000 7fc554c00640 1 -- 172.21.15.5:0/1660596406 <== mds.1 v2:172.21.15.159:6838/2073739599 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564ae963fe60 con 0x564ae97580e0 2024-09-10T09:52:14.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:14.689+0000 7fc554c00640 10 client.28244 handle_command_reply: tid=0 2024-09-10T09:52:14.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:14.689+0000 7fc570e25740 1 client.28244 shutdown 2024-09-10T09:52:14.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:14.689+0000 7fc570e25740 1 -- 172.21.15.5:0/1660596406 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564ae96dce50 msgr2=0x564ae96e1580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:52:14.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:14.689+0000 7fc570e25740 1 --2- 172.21.15.5:0/1660596406 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564ae96dce50 0x564ae96e1580 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fc5600096e0 tx=0x7fc560004870 comp rx=0 tx=0).stop 2024-09-10T09:52:14.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:14.689+0000 7fc570e25740 1 -- 172.21.15.5:0/1660596406 shutdown_connections 2024-09-10T09:52:14.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:14.689+0000 7fc570e25740 1 --2- 172.21.15.5:0/1660596406 >> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] conn(0x564ae97580e0 0x564ae975a4d0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:14.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:14.689+0000 7fc570e25740 1 --2- 172.21.15.5:0/1660596406 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564ae96dce50 0x564ae96e1580 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:14.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:14.689+0000 7fc570e25740 1 --2- 172.21.15.5:0/1660596406 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564ae96d2590 0x564ae96e1040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:14.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:14.689+0000 7fc570e25740 1 --2- 172.21.15.5:0/1660596406 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564ae96d1c70 0x564ae96e0b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:14.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:14.689+0000 7fc570e25740 1 -- 172.21.15.5:0/1660596406 >> 172.21.15.5:0/1660596406 conn(0x564ae96ce200 msgr2=0x564ae96d8230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:52:14.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:14.689+0000 7fc554c00640 0 client.28244 ms_handle_reset on v2:172.21.15.159:6838/2073739599 2024-09-10T09:52:14.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:14.689+0000 7fc570e25740 1 -- 172.21.15.5:0/1660596406 shutdown_connections 2024-09-10T09:52:14.780 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-10T09:52:14.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:14.689+0000 7fc570e25740 1 -- 172.21.15.5:0/1660596406 wait complete. 2024-09-10T09:52:14.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:14.689+0000 7fc570e25740 20 client.28244 trim_cache size 0 max 16384 2024-09-10T09:52:14.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:14.690+0000 7fc56e200640 1 -- 172.21.15.5:0/3930152706 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fc54c0a2c00 msgr2=0x7fc54c0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:52:14.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:14.690+0000 7fc56e200640 1 --2- 172.21.15.5:0/3930152706 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fc54c0a2c00 0x7fc54c0a50c0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fc5580096c0 tx=0x7fc5580058d0 comp rx=0 tx=0).stop 2024-09-10T09:52:14.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:14.690+0000 7fc56e200640 1 -- 172.21.15.5:0/3930152706 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc568102790 msgr2=0x7fc56819ac30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:52:14.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:14.690+0000 7fc56e200640 1 --2- 172.21.15.5:0/3930152706 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc568102790 0x7fc56819ac30 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fc55000e9f0 tx=0x7fc55000edf0 comp rx=0 tx=0).stop 2024-09-10T09:52:14.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:14.690+0000 7fc56e200640 1 -- 172.21.15.5:0/3930152706 shutdown_connections 2024-09-10T09:52:14.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:14.690+0000 7fc56e200640 1 --2- 172.21.15.5:0/3930152706 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fc54c0a2c00 0x7fc54c0a50c0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:14.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:14.690+0000 7fc56e200640 1 --2- 172.21.15.5:0/3930152706 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc568106f10 0x7fc56819efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:14.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:14.690+0000 7fc56e200640 1 --2- 172.21.15.5:0/3930152706 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc568102790 0x7fc56819ac30 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:14.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:14.690+0000 7fc56e200640 1 --2- 172.21.15.5:0/3930152706 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc568101de0 0x7fc56819a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:14.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:14.690+0000 7fc56e200640 1 -- 172.21.15.5:0/3930152706 >> 172.21.15.5:0/3930152706 conn(0x7fc5680fd8c0 msgr2=0x7fc5680ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:52:14.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:14.690+0000 7fc56e200640 1 -- 172.21.15.5:0/3930152706 shutdown_connections 2024-09-10T09:52:14.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:14.690+0000 7fc56e200640 1 -- 172.21.15.5:0/3930152706 wait complete. 2024-09-10T09:52:15.171 INFO:tasks.workunit.client.0.smithi005.stdout: 1 866428 6.13 MB/sec execute 596 sec latency 16311.994 ms 2024-09-10T09:52:15.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:52:15 smithi102 ceph-mon[35442]: pgmap v810: 129 pgs: 129 active+clean; 287 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 1.1 MiB/s wr, 52 op/s 2024-09-10T09:52:15.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:52:15 smithi005 ceph-mon[30350]: pgmap v810: 129 pgs: 129 active+clean; 287 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 1.1 MiB/s wr, 52 op/s 2024-09-10T09:52:15.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:52:15 smithi159 ceph-mon[35513]: pgmap v810: 129 pgs: 129 active+clean; 287 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 1.1 MiB/s wr, 52 op/s 2024-09-10T09:52:15.615 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-10T09:52:15.993 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:52:16.172 INFO:tasks.workunit.client.0.smithi005.stdout: 1 866428 6.12 MB/sec execute 597 sec latency 17312.135 ms 2024-09-10T09:52:16.279 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T09:52:16.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:16.750+0000 7f1353200640 1 -- 172.21.15.5:0/807917472 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1354102360 msgr2=0x7f135410e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:52:16.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:16.750+0000 7f1353200640 1 --2- 172.21.15.5:0/807917472 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1354102360 0x7f135410e890 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f134c009930 tx=0x7f134c02a420 comp rx=0 tx=0).stop 2024-09-10T09:52:16.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:16.750+0000 7f1353200640 1 -- 172.21.15.5:0/807917472 shutdown_connections 2024-09-10T09:52:16.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:16.750+0000 7f1353200640 1 --2- 172.21.15.5:0/807917472 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1354102360 0x7f135410e890 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:16.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:16.750+0000 7f1353200640 1 --2- 172.21.15.5:0/807917472 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f13541019c0 0x7f1354101e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:16.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:16.750+0000 7f1353200640 1 --2- 172.21.15.5:0/807917472 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f13541079c0 0x7f1354107da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:16.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:16.750+0000 7f1353200640 1 -- 172.21.15.5:0/807917472 >> 172.21.15.5:0/807917472 conn(0x7f13540fd6e0 msgr2=0x7f13540ffb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:52:16.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:16.750+0000 7f1353200640 1 -- 172.21.15.5:0/807917472 shutdown_connections 2024-09-10T09:52:16.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:16.750+0000 7f1353200640 1 -- 172.21.15.5:0/807917472 wait complete. 2024-09-10T09:52:16.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:16.751+0000 7f1353200640 1 Processor -- start 2024-09-10T09:52:16.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:16.751+0000 7f1353200640 1 -- start start 2024-09-10T09:52:16.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:16.752+0000 7f1353200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f13541019c0 0x7f1354198220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:52:16.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:16.752+0000 7f1353200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1354102360 0x7f1354198760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:52:16.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:16.752+0000 7f1353200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f13541079c0 0x7f135419caf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:52:16.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:16.752+0000 7f1353200640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1354111dc0 con 0x7f13541079c0 2024-09-10T09:52:16.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:16.752+0000 7f1353200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f1354111c40 con 0x7f1354102360 2024-09-10T09:52:16.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:16.752+0000 7f1353200640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f1354111f40 con 0x7f13541019c0 2024-09-10T09:52:16.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:16.752+0000 7f1351e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f13541019c0 0x7f1354198220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:52:16.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:16.752+0000 7f1351e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f13541019c0 0x7f1354198220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:41734/0 (socket says 172.21.15.5:41734) 2024-09-10T09:52:16.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:16.752+0000 7f1351e00640 1 -- 172.21.15.5:0/2585408837 learned_addr learned my addr 172.21.15.5:0/2585408837 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:52:16.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:16.752+0000 7f1349400640 1 --2- 172.21.15.5:0/2585408837 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1354102360 0x7f1354198760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:52:16.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:16.752+0000 7f1352800640 1 --2- 172.21.15.5:0/2585408837 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f13541079c0 0x7f135419caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:52:16.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:16.753+0000 7f1351e00640 1 -- 172.21.15.5:0/2585408837 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1354102360 msgr2=0x7f1354198760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:52:16.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:16.753+0000 7f1351e00640 1 --2- 172.21.15.5:0/2585408837 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1354102360 0x7f1354198760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:16.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:16.753+0000 7f1351e00640 1 -- 172.21.15.5:0/2585408837 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f13541079c0 msgr2=0x7f135419caf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:52:16.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:16.753+0000 7f1351e00640 1 --2- 172.21.15.5:0/2585408837 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f13541079c0 0x7f135419caf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:16.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:16.753+0000 7f1351e00640 1 -- 172.21.15.5:0/2585408837 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f135419d1d0 con 0x7f13541019c0 2024-09-10T09:52:16.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:16.753+0000 7f1349400640 1 --2- 172.21.15.5:0/2585408837 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1354102360 0x7f1354198760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:52:16.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:16.753+0000 7f1351e00640 1 --2- 172.21.15.5:0/2585408837 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f13541019c0 0x7f1354198220 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f134400ebc0 tx=0x7f134400c6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:52:16.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:16.753+0000 7f134b400640 1 -- 172.21.15.5:0/2585408837 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1344010630 con 0x7f13541019c0 2024-09-10T09:52:16.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:16.753+0000 7f134b400640 1 -- 172.21.15.5:0/2585408837 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1344010cb0 con 0x7f13541019c0 2024-09-10T09:52:16.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:16.753+0000 7f1353200640 1 -- 172.21.15.5:0/2585408837 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f135419d4c0 con 0x7f13541019c0 2024-09-10T09:52:16.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:16.753+0000 7f134b400640 1 -- 172.21.15.5:0/2585408837 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1344018bd0 con 0x7f13541019c0 2024-09-10T09:52:16.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:16.753+0000 7f1353200640 1 -- 172.21.15.5:0/2585408837 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1354074420 con 0x7f13541019c0 2024-09-10T09:52:16.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:16.756+0000 7f135bc33740 1 Processor -- start 2024-09-10T09:52:16.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:16.757+0000 7f135bc33740 1 -- start start 2024-09-10T09:52:16.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:16.757+0000 7f134b400640 1 -- 172.21.15.5:0/2585408837 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f13440107d0 con 0x7f13541019c0 2024-09-10T09:52:16.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:16.757+0000 7f135bc33740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55663f955e40 0x55663f956220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:52:16.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:16.757+0000 7f135bc33740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55663f956760 0x55663f960ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:52:16.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:16.757+0000 7f135bc33740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55663f961020 0x55663f963410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:52:16.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:16.757+0000 7f135bc33740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55663f8bc3e0 con 0x55663f956760 2024-09-10T09:52:16.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:16.757+0000 7f135bc33740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55663f863d80 con 0x55663f955e40 2024-09-10T09:52:16.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:16.757+0000 7f135bc33740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55663f8b83a0 con 0x55663f961020 2024-09-10T09:52:16.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:16.757+0000 7f1351e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55663f956760 0x55663f960ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:52:16.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:16.757+0000 7f1352800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55663f955e40 0x55663f956220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:52:16.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:16.757+0000 7f1352800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55663f955e40 0x55663f956220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:38136/0 (socket says 172.21.15.5:38136) 2024-09-10T09:52:16.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:16.757+0000 7f1351e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55663f956760 0x55663f960ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47786/0 (socket says 172.21.15.5:47786) 2024-09-10T09:52:16.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:16.757+0000 7f1352800640 1 -- 172.21.15.5:0/2182282393 learned_addr learned my addr 172.21.15.5:0/2182282393 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:52:16.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:16.757+0000 7f1349400640 1 --2- 172.21.15.5:0/2182282393 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55663f961020 0x55663f963410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:52:16.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:16.758+0000 7f1352800640 1 -- 172.21.15.5:0/2182282393 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55663f961020 msgr2=0x55663f963410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:52:16.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:16.758+0000 7f1352800640 1 --2- 172.21.15.5:0/2182282393 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55663f961020 0x55663f963410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:16.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:16.758+0000 7f1352800640 1 -- 172.21.15.5:0/2182282393 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55663f956760 msgr2=0x55663f960ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:52:16.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:16.758+0000 7f1352800640 1 --2- 172.21.15.5:0/2182282393 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55663f956760 0x55663f960ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:16.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:16.758+0000 7f1352800640 1 -- 172.21.15.5:0/2182282393 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55663f882ae0 con 0x55663f955e40 2024-09-10T09:52:16.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:16.758+0000 7f1349400640 1 --2- 172.21.15.5:0/2182282393 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55663f961020 0x55663f963410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:52:16.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:16.758+0000 7f1352800640 1 --2- 172.21.15.5:0/2182282393 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55663f955e40 0x55663f956220 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f134c009930 tx=0x7f134c005d50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c410f3785980224a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:52:16.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:16.758+0000 7f1343400640 1 -- 172.21.15.5:0/2182282393 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f134c040070 con 0x55663f955e40 2024-09-10T09:52:16.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:16.758+0000 7f1343400640 1 -- 172.21.15.5:0/2182282393 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f134c0317b0 con 0x55663f955e40 2024-09-10T09:52:16.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:16.758+0000 7f1343400640 1 -- 172.21.15.5:0/2182282393 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f134c035040 con 0x55663f955e40 2024-09-10T09:52:16.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:16.758+0000 7f134b400640 1 --2- 172.21.15.5:0/2585408837 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f13380a2b80 0x7f13380a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:52:16.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:16.759+0000 7f134b400640 1 -- 172.21.15.5:0/2585408837 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f13440cf9f0 con 0x7f13541019c0 2024-09-10T09:52:16.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:16.759+0000 7f1351e00640 1 --2- 172.21.15.5:0/2585408837 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f13380a2b80 0x7f13380a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:52:16.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:16.759+0000 7f135bc33740 1 -- 172.21.15.5:0/2182282393 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55663f955e40 msgr2=0x55663f956220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:52:16.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:16.759+0000 7f135bc33740 1 --2- 172.21.15.5:0/2182282393 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55663f955e40 0x55663f956220 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f134c009930 tx=0x7f134c005d50 comp rx=0 tx=0).stop 2024-09-10T09:52:16.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:16.759+0000 7f135bc33740 1 -- 172.21.15.5:0/2182282393 shutdown_connections 2024-09-10T09:52:16.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:16.759+0000 7f135bc33740 1 --2- 172.21.15.5:0/2182282393 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55663f961020 0x55663f963410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:16.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:16.759+0000 7f135bc33740 1 --2- 172.21.15.5:0/2182282393 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55663f956760 0x55663f960ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:16.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:16.759+0000 7f135bc33740 1 --2- 172.21.15.5:0/2182282393 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55663f955e40 0x55663f956220 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:16.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:16.759+0000 7f135bc33740 1 -- 172.21.15.5:0/2182282393 >> 172.21.15.5:0/2182282393 conn(0x55663f9523d0 msgr2=0x55663f9527b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:52:16.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:16.759+0000 7f1351e00640 1 --2- 172.21.15.5:0/2585408837 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f13380a2b80 0x7f13380a5040 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f1344098310 tx=0x7f13440d5530 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:52:16.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:16.760+0000 7f135bc33740 1 -- 172.21.15.5:0/2182282393 shutdown_connections 2024-09-10T09:52:16.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:16.760+0000 7f135bc33740 1 -- 172.21.15.5:0/2182282393 wait complete. 2024-09-10T09:52:16.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:16.760+0000 7f135bc33740 1 -- start start 2024-09-10T09:52:16.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:16.760+0000 7f135bc33740 1 Processor -- start 2024-09-10T09:52:16.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:16.761+0000 7f135bc33740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55663f955e40 0x55663f95de70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:52:16.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:16.761+0000 7f135bc33740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55663f956760 0x55663f95e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:52:16.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:16.761+0000 7f135bc33740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55663f961020 0x55663f95e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:52:16.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:16.761+0000 7f135bc33740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55663f963bf0 con 0x55663f955e40 2024-09-10T09:52:16.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:16.761+0000 7f135bc33740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55663f8770f0 con 0x55663f956760 2024-09-10T09:52:16.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:16.761+0000 7f135bc33740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55663f963d70 con 0x55663f961020 2024-09-10T09:52:16.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:16.761+0000 7f135bc33740 10 client.0 _ll_register_callbacks cb 0x55663f85ebd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:52:16.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:16.761+0000 7f1349400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55663f956760 0x55663f95e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:52:16.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:16.761+0000 7f1352800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55663f955e40 0x55663f95de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:52:16.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:16.761+0000 7f1349400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55663f956760 0x55663f95e3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:38140/0 (socket says 172.21.15.5:38140) 2024-09-10T09:52:16.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:16.761+0000 7f1349400640 1 -- 172.21.15.5:0/846271492 learned_addr learned my addr 172.21.15.5:0/846271492 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:52:16.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:16.761+0000 7f1352800640 1 --2- 172.21.15.5:0/846271492 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55663f961020 0x55663f95e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:52:16.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:16.761+0000 7f1349400640 1 -- 172.21.15.5:0/846271492 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55663f961020 msgr2=0x55663f95e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:52:16.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:16.761+0000 7f1349400640 1 --2- 172.21.15.5:0/846271492 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55663f961020 0x55663f95e8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:16.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:16.761+0000 7f1349400640 1 -- 172.21.15.5:0/846271492 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55663f955e40 msgr2=0x55663f95de70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:52:16.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:16.761+0000 7f1349400640 1 --2- 172.21.15.5:0/846271492 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55663f955e40 0x55663f95de70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:16.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:16.762+0000 7f1349400640 1 -- 172.21.15.5:0/846271492 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55663f62d3c0 con 0x55663f956760 2024-09-10T09:52:16.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:16.762+0000 7f1352800640 1 --2- 172.21.15.5:0/846271492 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55663f961020 0x55663f95e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T09:52:16.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:16.762+0000 7f1349400640 1 --2- 172.21.15.5:0/846271492 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55663f956760 0x55663f95e3b0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f133c00b340 tx=0x7f133c00b430 comp rx=0 tx=0).ready entity=mon.2 client_cookie=649c0a616e6c9421 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:52:16.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:16.762+0000 7f1337e00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-09-10T09:52:16.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:16.762+0000 7f1337e00640 1 -- 172.21.15.5:0/846271492 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f133c023060 con 0x55663f956760 2024-09-10T09:52:16.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:16.762+0000 7f135bc33740 1 -- 172.21.15.5:0/846271492 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f134c0317b0 con 0x55663f956760 2024-09-10T09:52:16.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:16.762+0000 7f1337e00640 1 -- 172.21.15.5:0/846271492 <== mon.2 v2:172.21.15.102:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f133c01e680 con 0x55663f956760 2024-09-10T09:52:16.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:16.763+0000 7f1337e00640 1 -- 172.21.15.5:0/846271492 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f133c02a050 con 0x55663f956760 2024-09-10T09:52:16.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:16.763+0000 7f135bc33740 10 client.28256 fetch_fsmap learned FSMap version 36 2024-09-10T09:52:16.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:16.763+0000 7f135bc33740 1 -- 172.21.15.5:0/846271492 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55663f96fc70 con 0x55663f956760 2024-09-10T09:52:16.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:16.763+0000 7f1337e00640 1 -- 172.21.15.5:0/846271492 <== mon.2 v2:172.21.15.102:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f134c0317b0 con 0x55663f956760 2024-09-10T09:52:16.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:16.763+0000 7f1337e00640 1 -- 172.21.15.5:0/846271492 <== mon.2 v2:172.21.15.102:3300/0 5 ==== fsmap(e 36) ==== 8370+0+0 (secure 0 0 0) 0x7f133c01d9a0 con 0x55663f956760 2024-09-10T09:52:16.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:16.763+0000 7f135bc33740 10 client.28256 fetch_fsmap finished waiting for FSMap version 36 2024-09-10T09:52:16.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:16.763+0000 7f135bc33740 10 client.28256 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T09:52:16.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:16.763+0000 7f135bc33740 20 client.28256 populate_metadata read hostname 'smithi005' 2024-09-10T09:52:16.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:16.763+0000 7f135bc33740 1 --2- 172.21.15.5:0/846271492 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55663f971080 0x55663f973470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:52:16.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:16.763+0000 7f135bc33740 4 client.28256 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:52:16.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:16.763+0000 7f135bc33740 1 -- 172.21.15.5:0/846271492 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55663f8c9950 con 0x55663f971080 2024-09-10T09:52:16.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:16.764+0000 7f1352800640 1 --2- 172.21.15.5:0/846271492 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55663f971080 0x55663f973470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:52:16.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:16.764+0000 7f1352800640 1 --2- 172.21.15.5:0/846271492 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55663f971080 0x55663f973470 crc :-1 s=READY pgs=211 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a872e0d0fa42a964 server_cookie=4950be99831483 in_seq=0 out_seq=0 2024-09-10T09:52:16.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:16.764+0000 7f1337e00640 10 client.28256 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T09:52:16.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:16.765+0000 7f1337e00640 1 -- 172.21.15.5:0/846271492 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55663f8c9950 con 0x55663f971080 2024-09-10T09:52:16.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:16.765+0000 7f1337e00640 10 client.28256 handle_command_reply: tid=0 2024-09-10T09:52:16.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:16.765+0000 7f135bc33740 1 client.28256 shutdown 2024-09-10T09:52:16.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:16.765+0000 7f135bc33740 1 -- 172.21.15.5:0/846271492 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55663f956760 msgr2=0x55663f95e3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:52:16.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:16.765+0000 7f135bc33740 1 --2- 172.21.15.5:0/846271492 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55663f956760 0x55663f95e3b0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f133c00b340 tx=0x7f133c00b430 comp rx=0 tx=0).stop 2024-09-10T09:52:16.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:16.765+0000 7f135bc33740 1 -- 172.21.15.5:0/846271492 shutdown_connections 2024-09-10T09:52:16.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:16.765+0000 7f135bc33740 1 --2- 172.21.15.5:0/846271492 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55663f971080 0x55663f973470 crc :-1 s=READY pgs=211 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:16.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:16.765+0000 7f135bc33740 1 --2- 172.21.15.5:0/846271492 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55663f961020 0x55663f95e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:16.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:16.765+0000 7f135bc33740 1 --2- 172.21.15.5:0/846271492 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55663f956760 0x55663f95e3b0 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:16.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:16.765+0000 7f135bc33740 1 --2- 172.21.15.5:0/846271492 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55663f955e40 0x55663f95de70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:16.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:16.765+0000 7f135bc33740 1 -- 172.21.15.5:0/846271492 >> 172.21.15.5:0/846271492 conn(0x55663f9523d0 msgr2=0x55663f958100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:52:16.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:16.766+0000 7f1337e00640 0 client.28256 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T09:52:16.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:16.766+0000 7f135bc33740 1 -- 172.21.15.5:0/846271492 shutdown_connections 2024-09-10T09:52:16.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:16.766+0000 7f135bc33740 1 -- 172.21.15.5:0/846271492 wait complete. 2024-09-10T09:52:16.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:16.766+0000 7f135bc33740 20 client.28256 trim_cache size 0 max 16384 2024-09-10T09:52:16.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:16.780+0000 7f135bc33740 1 Processor -- start 2024-09-10T09:52:16.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:16.780+0000 7f135bc33740 1 -- start start 2024-09-10T09:52:16.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:16.781+0000 7f135bc33740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55663f955e40 0x55663f95e740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:52:16.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:16.781+0000 7f135bc33740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55663f956760 0x55663f9513a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:52:16.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:16.781+0000 7f135bc33740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55663f961020 0x55663f978d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:52:16.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:16.781+0000 7f135bc33740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f13541acaf0 con 0x55663f955e40 2024-09-10T09:52:16.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:16.781+0000 7f135bc33740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f13541ae630 con 0x55663f956760 2024-09-10T09:52:16.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:16.781+0000 7f135bc33740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f13541ad1c0 con 0x55663f961020 2024-09-10T09:52:16.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:16.781+0000 7f1349400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55663f956760 0x55663f9513a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:52:16.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:16.781+0000 7f1352800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55663f955e40 0x55663f95e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:52:16.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:16.781+0000 7f1349400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55663f956760 0x55663f9513a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:38150/0 (socket says 172.21.15.5:38150) 2024-09-10T09:52:16.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:16.781+0000 7f1349400640 1 -- 172.21.15.5:0/3385902229 learned_addr learned my addr 172.21.15.5:0/3385902229 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:52:16.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:16.781+0000 7f1352800640 1 --2- 172.21.15.5:0/3385902229 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55663f961020 0x55663f978d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:52:16.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:16.782+0000 7f1349400640 1 -- 172.21.15.5:0/3385902229 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55663f961020 msgr2=0x55663f978d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:52:16.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:16.782+0000 7f1349400640 1 --2- 172.21.15.5:0/3385902229 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55663f961020 0x55663f978d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:16.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:16.782+0000 7f1349400640 1 -- 172.21.15.5:0/3385902229 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55663f955e40 msgr2=0x55663f95e740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:52:16.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:16.782+0000 7f1349400640 1 --2- 172.21.15.5:0/3385902229 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55663f955e40 0x55663f95e740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:16.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:16.782+0000 7f1349400640 1 -- 172.21.15.5:0/3385902229 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55663f952200 con 0x55663f956760 2024-09-10T09:52:16.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:16.782+0000 7f1349400640 1 --2- 172.21.15.5:0/3385902229 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55663f956760 0x55663f9513a0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f133c01df60 tx=0x7f133c005690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bf20cba787b4a7ce server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:52:16.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:16.782+0000 7f1342a00640 1 -- 172.21.15.5:0/3385902229 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f133c01d5e0 con 0x55663f956760 2024-09-10T09:52:16.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:16.782+0000 7f1342a00640 1 -- 172.21.15.5:0/3385902229 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f133c00b930 con 0x55663f956760 2024-09-10T09:52:16.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:16.782+0000 7f1342a00640 1 -- 172.21.15.5:0/3385902229 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f133c01ed10 con 0x55663f956760 2024-09-10T09:52:16.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:16.783+0000 7f135bc33740 1 -- 172.21.15.5:0/3385902229 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55663f956760 msgr2=0x55663f9513a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:52:16.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:16.783+0000 7f135bc33740 1 --2- 172.21.15.5:0/3385902229 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55663f956760 0x55663f9513a0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f133c01df60 tx=0x7f133c005690 comp rx=0 tx=0).stop 2024-09-10T09:52:16.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:16.783+0000 7f135bc33740 1 -- 172.21.15.5:0/3385902229 shutdown_connections 2024-09-10T09:52:16.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:16.783+0000 7f135bc33740 1 --2- 172.21.15.5:0/3385902229 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55663f961020 0x55663f978d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:16.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:16.783+0000 7f135bc33740 1 --2- 172.21.15.5:0/3385902229 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55663f956760 0x55663f9513a0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:16.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:16.783+0000 7f135bc33740 1 --2- 172.21.15.5:0/3385902229 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55663f955e40 0x55663f95e740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:16.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:16.783+0000 7f135bc33740 1 -- 172.21.15.5:0/3385902229 >> 172.21.15.5:0/3385902229 conn(0x55663f9523d0 msgr2=0x55663f950c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:52:16.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:16.783+0000 7f135bc33740 1 -- 172.21.15.5:0/3385902229 shutdown_connections 2024-09-10T09:52:16.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:16.783+0000 7f135bc33740 1 -- 172.21.15.5:0/3385902229 wait complete. 2024-09-10T09:52:16.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:16.784+0000 7f135bc33740 1 -- start start 2024-09-10T09:52:16.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:16.784+0000 7f135bc33740 1 Processor -- start 2024-09-10T09:52:16.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:16.785+0000 7f135bc33740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55663f955e40 0x55663f95aed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:52:16.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:16.785+0000 7f135bc33740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55663f956760 0x55663f95b410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:52:16.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:16.785+0000 7f135bc33740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55663f961020 0x55663f95b950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:52:16.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:16.785+0000 7f135bc33740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f13541adf60 con 0x55663f961020 2024-09-10T09:52:16.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:16.785+0000 7f135bc33740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f13541ad890 con 0x55663f955e40 2024-09-10T09:52:16.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:16.785+0000 7f135bc33740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f13541aaad0 con 0x55663f956760 2024-09-10T09:52:16.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:16.785+0000 7f135bc33740 10 client.0 _ll_register_callbacks cb 0x7f133c009760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:52:16.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:16.785+0000 7f1349400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55663f956760 0x55663f95b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:52:16.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:16.785+0000 7f1352800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55663f955e40 0x55663f95aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:52:16.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:16.785+0000 7f1349400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55663f956760 0x55663f95b410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:41766/0 (socket says 172.21.15.5:41766) 2024-09-10T09:52:16.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:16.785+0000 7f1349400640 1 -- 172.21.15.5:0/2856897033 learned_addr learned my addr 172.21.15.5:0/2856897033 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:52:16.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:16.785+0000 7f1352800640 1 --2- 172.21.15.5:0/2856897033 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55663f961020 0x55663f95b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:52:16.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:16.785+0000 7f1349400640 1 -- 172.21.15.5:0/2856897033 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55663f955e40 msgr2=0x55663f95aed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:52:16.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:16.785+0000 7f1349400640 1 --2- 172.21.15.5:0/2856897033 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55663f955e40 0x55663f95aed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:16.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:16.785+0000 7f1349400640 1 -- 172.21.15.5:0/2856897033 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55663f961020 msgr2=0x55663f95b950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:52:16.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:16.785+0000 7f1349400640 1 --2- 172.21.15.5:0/2856897033 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55663f961020 0x55663f95b950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:16.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:16.785+0000 7f1349400640 1 -- 172.21.15.5:0/2856897033 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55663f952f80 con 0x55663f956760 2024-09-10T09:52:16.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:16.786+0000 7f1349400640 1 --2- 172.21.15.5:0/2856897033 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55663f956760 0x55663f95b410 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f133c01ddf0 tx=0x7f133c01e850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c8365af6b045cc9c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:52:16.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:16.786+0000 7f1337e00640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-09-10T09:52:16.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:16.786+0000 7f1337e00640 1 -- 172.21.15.5:0/2856897033 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f133c023040 con 0x55663f956760 2024-09-10T09:52:16.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:16.786+0000 7f135bc33740 1 -- 172.21.15.5:0/2856897033 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f133c00b930 con 0x55663f956760 2024-09-10T09:52:16.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:16.786+0000 7f1337e00640 1 -- 172.21.15.5:0/2856897033 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f133c01eda0 con 0x55663f956760 2024-09-10T09:52:16.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:16.786+0000 7f1337e00640 1 -- 172.21.15.5:0/2856897033 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f133c046050 con 0x55663f956760 2024-09-10T09:52:16.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:16.786+0000 7f135bc33740 10 client.28258 fetch_fsmap learned FSMap version 36 2024-09-10T09:52:16.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:16.786+0000 7f135bc33740 1 -- 172.21.15.5:0/2856897033 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55663f953480 con 0x55663f956760 2024-09-10T09:52:16.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:16.787+0000 7f1337e00640 1 -- 172.21.15.5:0/2856897033 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f133c00b930 con 0x55663f956760 2024-09-10T09:52:16.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:16.787+0000 7f1337e00640 1 -- 172.21.15.5:0/2856897033 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 36) ==== 8370+0+0 (secure 0 0 0) 0x7f133c02d070 con 0x55663f956760 2024-09-10T09:52:16.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:16.787+0000 7f135bc33740 10 client.28258 fetch_fsmap finished waiting for FSMap version 36 2024-09-10T09:52:16.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:16.787+0000 7f135bc33740 10 client.28258 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T09:52:16.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:16.787+0000 7f135bc33740 20 client.28258 populate_metadata read hostname 'smithi005' 2024-09-10T09:52:16.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:16.787+0000 7f135bc33740 1 --2- 172.21.15.5:0/2856897033 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55663f9dc190 0x55663f9de580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:52:16.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:16.787+0000 7f135bc33740 4 client.28258 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T09:52:16.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:16.787+0000 7f135bc33740 1 -- 172.21.15.5:0/2856897033 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55663f8c3e60 con 0x55663f9dc190 2024-09-10T09:52:16.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:16.787+0000 7f1352800640 1 --2- 172.21.15.5:0/2856897033 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55663f9dc190 0x55663f9de580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:52:16.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:16.788+0000 7f1352800640 1 --2- 172.21.15.5:0/2856897033 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55663f9dc190 0x55663f9de580 crc :-1 s=READY pgs=212 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=af6f8c56cb11d74b server_cookie=59262ba7cd60f89b in_seq=0 out_seq=0 2024-09-10T09:52:16.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:16.788+0000 7f1337e00640 10 client.28258 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T09:52:16.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:16.788+0000 7f1337e00640 1 -- 172.21.15.5:0/2856897033 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55663f8c3e60 con 0x55663f9dc190 2024-09-10T09:52:16.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:16.788+0000 7f1337e00640 10 client.28258 handle_command_reply: tid=0 2024-09-10T09:52:16.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:16.788+0000 7f135bc33740 1 client.28258 shutdown 2024-09-10T09:52:16.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:16.788+0000 7f135bc33740 1 -- 172.21.15.5:0/2856897033 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55663f956760 msgr2=0x55663f95b410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:52:16.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:16.788+0000 7f135bc33740 1 --2- 172.21.15.5:0/2856897033 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55663f956760 0x55663f95b410 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f133c01ddf0 tx=0x7f133c01e850 comp rx=0 tx=0).stop 2024-09-10T09:52:16.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:16.789+0000 7f135bc33740 1 -- 172.21.15.5:0/2856897033 shutdown_connections 2024-09-10T09:52:16.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:16.789+0000 7f135bc33740 1 --2- 172.21.15.5:0/2856897033 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55663f9dc190 0x55663f9de580 crc :-1 s=READY pgs=212 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:16.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:16.789+0000 7f135bc33740 1 --2- 172.21.15.5:0/2856897033 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55663f961020 0x55663f95b950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:16.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:16.789+0000 7f135bc33740 1 --2- 172.21.15.5:0/2856897033 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55663f956760 0x55663f95b410 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:16.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:16.789+0000 7f135bc33740 1 --2- 172.21.15.5:0/2856897033 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55663f955e40 0x55663f95aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:16.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:16.789+0000 7f1337e00640 0 client.28258 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T09:52:16.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:16.789+0000 7f135bc33740 1 -- 172.21.15.5:0/2856897033 >> 172.21.15.5:0/2856897033 conn(0x55663f9523d0 msgr2=0x55663f95fef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:52:16.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:16.789+0000 7f135bc33740 1 -- 172.21.15.5:0/2856897033 shutdown_connections 2024-09-10T09:52:16.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:16.789+0000 7f135bc33740 1 -- 172.21.15.5:0/2856897033 wait complete. 2024-09-10T09:52:16.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:16.789+0000 7f135bc33740 20 client.28258 trim_cache size 0 max 16384 2024-09-10T09:52:16.793 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-10T09:52:16.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:16.790+0000 7f1353200640 1 -- 172.21.15.5:0/2585408837 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f13380a2b80 msgr2=0x7f13380a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:52:16.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:16.790+0000 7f1353200640 1 --2- 172.21.15.5:0/2585408837 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f13380a2b80 0x7f13380a5040 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f1344098310 tx=0x7f13440d5530 comp rx=0 tx=0).stop 2024-09-10T09:52:16.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:16.790+0000 7f1353200640 1 -- 172.21.15.5:0/2585408837 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f13541019c0 msgr2=0x7f1354198220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:52:16.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:16.790+0000 7f1353200640 1 --2- 172.21.15.5:0/2585408837 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f13541019c0 0x7f1354198220 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f134400ebc0 tx=0x7f134400c6a0 comp rx=0 tx=0).stop 2024-09-10T09:52:16.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:16.790+0000 7f1353200640 1 -- 172.21.15.5:0/2585408837 shutdown_connections 2024-09-10T09:52:16.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:16.790+0000 7f1353200640 1 --2- 172.21.15.5:0/2585408837 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f13380a2b80 0x7f13380a5040 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:16.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:16.790+0000 7f1353200640 1 --2- 172.21.15.5:0/2585408837 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f13541079c0 0x7f135419caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:16.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:16.790+0000 7f1353200640 1 --2- 172.21.15.5:0/2585408837 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1354102360 0x7f1354198760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:16.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:16.790+0000 7f1353200640 1 --2- 172.21.15.5:0/2585408837 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f13541019c0 0x7f1354198220 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:16.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:16.790+0000 7f1353200640 1 -- 172.21.15.5:0/2585408837 >> 172.21.15.5:0/2585408837 conn(0x7f13540fd6e0 msgr2=0x7f13540fde70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:52:16.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:16.790+0000 7f1353200640 1 -- 172.21.15.5:0/2585408837 shutdown_connections 2024-09-10T09:52:16.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:16.790+0000 7f1353200640 1 -- 172.21.15.5:0/2585408837 wait complete. 2024-09-10T09:52:17.172 INFO:tasks.workunit.client.0.smithi005.stdout: 1 866428 6.11 MB/sec execute 598 sec latency 18312.195 ms 2024-09-10T09:52:17.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:52:17 smithi102 ceph-mon[35442]: pgmap v811: 129 pgs: 129 active+clean; 287 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 837 KiB/s wr, 45 op/s 2024-09-10T09:52:17.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:52:17 smithi005 ceph-mon[30350]: pgmap v811: 129 pgs: 129 active+clean; 287 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 837 KiB/s wr, 45 op/s 2024-09-10T09:52:17.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:52:17 smithi159 ceph-mon[35513]: pgmap v811: 129 pgs: 129 active+clean; 287 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 837 KiB/s wr, 45 op/s 2024-09-10T09:52:17.733 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:52:18.172 INFO:tasks.workunit.client.0.smithi005.stdout: 1 866428 6.10 MB/sec execute 599 sec latency 19312.232 ms 2024-09-10T09:52:18.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:18.303+0000 7fc03fe00640 1 -- 172.21.15.5:0/272420887 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc040108f60 msgr2=0x7fc040110ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:52:18.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:18.303+0000 7fc03fe00640 1 --2- 172.21.15.5:0/272420887 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc040108f60 0x7fc040110ae0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fc038009ca0 tx=0x7fc03802a8c0 comp rx=0 tx=0).stop 2024-09-10T09:52:18.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:18.303+0000 7fc03fe00640 1 -- 172.21.15.5:0/272420887 shutdown_connections 2024-09-10T09:52:18.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:18.303+0000 7fc03fe00640 1 --2- 172.21.15.5:0/272420887 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc040108f60 0x7fc040110ae0 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:18.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:18.303+0000 7fc03fe00640 1 --2- 172.21.15.5:0/272420887 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc0401047e0 0x7fc040108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:18.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:18.303+0000 7fc03fe00640 1 --2- 172.21.15.5:0/272420887 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc040103e30 0x7fc040104210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:18.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:18.303+0000 7fc03fe00640 1 -- 172.21.15.5:0/272420887 >> 172.21.15.5:0/272420887 conn(0x7fc0400ff8d0 msgr2=0x7fc040101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:52:18.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:18.303+0000 7fc03fe00640 1 -- 172.21.15.5:0/272420887 shutdown_connections 2024-09-10T09:52:18.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:18.303+0000 7fc03fe00640 1 -- 172.21.15.5:0/272420887 wait complete. 2024-09-10T09:52:18.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:18.304+0000 7fc03fe00640 1 Processor -- start 2024-09-10T09:52:18.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:18.304+0000 7fc03fe00640 1 -- start start 2024-09-10T09:52:18.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:18.304+0000 7fc03fe00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc040103e30 0x7fc04019a400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:52:18.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:18.304+0000 7fc03fe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc0401047e0 0x7fc04019a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:52:18.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:18.304+0000 7fc03fe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc040108f60 0x7fc04019ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:52:18.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:18.304+0000 7fc03fe00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc040113fc0 con 0x7fc0401047e0 2024-09-10T09:52:18.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:18.304+0000 7fc03fe00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fc040113e40 con 0x7fc040108f60 2024-09-10T09:52:18.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:18.304+0000 7fc03fe00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fc040114140 con 0x7fc040103e30 2024-09-10T09:52:18.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:18.305+0000 7fc03ea00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc040103e30 0x7fc04019a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:52:18.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:18.305+0000 7fc036000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc0401047e0 0x7fc04019a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:52:18.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:18.305+0000 7fc03ea00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc040103e30 0x7fc04019a400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:41786/0 (socket says 172.21.15.5:41786) 2024-09-10T09:52:18.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:18.305+0000 7fc03ea00640 1 -- 172.21.15.5:0/631024852 learned_addr learned my addr 172.21.15.5:0/631024852 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:52:18.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:18.305+0000 7fc03f400640 1 --2- 172.21.15.5:0/631024852 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc040108f60 0x7fc04019ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:52:18.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:18.305+0000 7fc03ea00640 1 -- 172.21.15.5:0/631024852 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc040108f60 msgr2=0x7fc04019ecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:52:18.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:18.305+0000 7fc03ea00640 1 --2- 172.21.15.5:0/631024852 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc040108f60 0x7fc04019ecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:18.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:18.305+0000 7fc03ea00640 1 -- 172.21.15.5:0/631024852 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc0401047e0 msgr2=0x7fc04019a940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:52:18.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:18.305+0000 7fc03ea00640 1 --2- 172.21.15.5:0/631024852 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc0401047e0 0x7fc04019a940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:18.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:18.305+0000 7fc03ea00640 1 -- 172.21.15.5:0/631024852 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc04019f3b0 con 0x7fc040103e30 2024-09-10T09:52:18.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:18.305+0000 7fc03ea00640 1 --2- 172.21.15.5:0/631024852 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc040103e30 0x7fc04019a400 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fc03000e9f0 tx=0x7fc03000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:52:18.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:18.306+0000 7fc037e00640 1 -- 172.21.15.5:0/631024852 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc03000cd80 con 0x7fc040103e30 2024-09-10T09:52:18.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:18.306+0000 7fc037e00640 1 -- 172.21.15.5:0/631024852 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc030010910 con 0x7fc040103e30 2024-09-10T09:52:18.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:18.306+0000 7fc037e00640 1 -- 172.21.15.5:0/631024852 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc030018990 con 0x7fc040103e30 2024-09-10T09:52:18.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:18.306+0000 7fc03fe00640 1 -- 172.21.15.5:0/631024852 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc04019f6a0 con 0x7fc040103e30 2024-09-10T09:52:18.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:18.306+0000 7fc03fe00640 1 -- 172.21.15.5:0/631024852 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0401a6f80 con 0x7fc040103e30 2024-09-10T09:52:18.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:18.307+0000 7fc03fe00640 1 -- 172.21.15.5:0/631024852 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc040104210 con 0x7fc040103e30 2024-09-10T09:52:18.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:18.309+0000 7fc037e00640 1 -- 172.21.15.5:0/631024852 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7fc030010ac0 con 0x7fc040103e30 2024-09-10T09:52:18.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:18.310+0000 7fc037e00640 1 --2- 172.21.15.5:0/631024852 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fc0240a2b80 0x7fc0240a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:52:18.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:18.310+0000 7fc037e00640 1 -- 172.21.15.5:0/631024852 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fc0300cfcf0 con 0x7fc040103e30 2024-09-10T09:52:18.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:18.310+0000 7fc036000640 1 --2- 172.21.15.5:0/631024852 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fc0240a2b80 0x7fc0240a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:52:18.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:18.311+0000 7fc036000640 1 --2- 172.21.15.5:0/631024852 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fc0240a2b80 0x7fc0240a5040 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fc0400ffcf0 tx=0x7fc02800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:52:18.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:18.312+0000 7fc037e00640 1 -- 172.21.15.5:0/631024852 <== mon.1 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc03009c1b0 con 0x7fc040103e30 2024-09-10T09:52:18.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:18.535+0000 7fc03fe00640 1 -- 172.21.15.5:0/631024852 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc04019b760 con 0x7fc040103e30 2024-09-10T09:52:18.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:18.536+0000 7fc037e00640 1 -- 172.21.15.5:0/631024852 <== mon.1 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) ==== 94+0+11969 (secure 0 0 0) 0x7fc0300a04b0 con 0x7fc040103e30 2024-09-10T09:52:18.537 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-10T09:52:18.538 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":36,"btime":"2024-09-10T09:48:07:349075+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14454,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/2970594311","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2970594311},{"type":"v1","addr":"172.21.15.5:6835","nonce":2970594311}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14514,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6839/3616502174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":3616502174},{"type":"v1","addr":"172.21.15.5:6839","nonce":3616502174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":36,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T09:35:29.626287+0000","modified":"2024-09-10T09:48:06.970127+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24542,"mds_1":24535,"mds_2":24511,"mds_3":24506,"mds_4":24443},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14520":{"gid":14520,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/2073599842","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":2073599842},{"type":"v1","addr":"172.21.15.5:6841","nonce":2073599842}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"k","rank":4,"incarnation":26,"state":"up:active","state_seq":24,"addr":"172.21.15.102:6833/2457007427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":2457007427},{"type":"v1","addr":"172.21.15.102:6833","nonce":2457007427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/356445159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":356445159},{"type":"v1","addr":"172.21.15.159:6833","nonce":356445159}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6837/2894314507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":2894314507},{"type":"v1","addr":"172.21.15.5:6837","nonce":2894314507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.102:6835/1647549111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":1647549111},{"type":"v1","addr":"172.21.15.102:6835","nonce":1647549111}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6835/952657242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":952657242},{"type":"v1","addr":"172.21.15.159:6835","nonce":952657242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":19,"addr":"172.21.15.102:6837/2689776764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":2689776764},{"type":"v1","addr":"172.21.15.102:6837","nonce":2689776764}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.159:6837/620627639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":620627639},{"type":"v1","addr":"172.21.15.159:6837","nonce":620627639}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24535":{"gid":24535,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":18,"addr":"172.21.15.159:6839/2073739599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":2073739599},{"type":"v1","addr":"172.21.15.159:6839","nonce":2073739599}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24542":{"gid":24542,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.102:6839/335409749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6838","nonce":335409749},{"type":"v1","addr":"172.21.15.102:6839","nonce":335409749}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24542,"qdb_cluster":[24542,24535,24511,24506,24443]},"id":1}]} 2024-09-10T09:52:18.538 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 36 2024-09-10T09:52:18.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:18.538+0000 7fc03fe00640 1 -- 172.21.15.5:0/631024852 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fc0240a2b80 msgr2=0x7fc0240a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:52:18.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:18.539+0000 7fc03fe00640 1 --2- 172.21.15.5:0/631024852 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fc0240a2b80 0x7fc0240a5040 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fc0400ffcf0 tx=0x7fc02800b390 comp rx=0 tx=0).stop 2024-09-10T09:52:18.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:18.539+0000 7fc03fe00640 1 -- 172.21.15.5:0/631024852 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc040103e30 msgr2=0x7fc04019a400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:52:18.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:18.539+0000 7fc03fe00640 1 --2- 172.21.15.5:0/631024852 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc040103e30 0x7fc04019a400 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fc03000e9f0 tx=0x7fc03000eeb0 comp rx=0 tx=0).stop 2024-09-10T09:52:18.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:18.539+0000 7fc03fe00640 1 -- 172.21.15.5:0/631024852 shutdown_connections 2024-09-10T09:52:18.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:18.539+0000 7fc03fe00640 1 --2- 172.21.15.5:0/631024852 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fc0240a2b80 0x7fc0240a5040 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:18.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:18.539+0000 7fc03fe00640 1 --2- 172.21.15.5:0/631024852 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc040108f60 0x7fc04019ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:18.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:18.539+0000 7fc03fe00640 1 --2- 172.21.15.5:0/631024852 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc0401047e0 0x7fc04019a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:18.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:18.539+0000 7fc03fe00640 1 --2- 172.21.15.5:0/631024852 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc040103e30 0x7fc04019a400 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:18.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:18.539+0000 7fc03fe00640 1 -- 172.21.15.5:0/631024852 >> 172.21.15.5:0/631024852 conn(0x7fc0400ff8d0 msgr2=0x7fc040100b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:52:18.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:18.539+0000 7fc03fe00640 1 -- 172.21.15.5:0/631024852 shutdown_connections 2024-09-10T09:52:18.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:18.539+0000 7fc03fe00640 1 -- 172.21.15.5:0/631024852 wait complete. 2024-09-10T09:52:18.632 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-10T09:52:18.633 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-10T09:52:19.172 INFO:tasks.workunit.client.0.smithi005.stdout: 1 cleanup 600 sec 2024-09-10T09:52:19.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:52:19 smithi102 ceph-mon[35442]: pgmap v812: 129 pgs: 129 active+clean; 287 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 585 KiB/s wr, 35 op/s 2024-09-10T09:52:19.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:52:19 smithi102 ceph-mon[35442]: from='client.? 172.21.15.5:0/631024852' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:52:19.414 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:52:19 smithi005 ceph-mon[30350]: pgmap v812: 129 pgs: 129 active+clean; 287 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 585 KiB/s wr, 35 op/s 2024-09-10T09:52:19.414 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:52:19 smithi005 ceph-mon[30350]: from='client.? 172.21.15.5:0/631024852' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:52:19.458 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24542 quiesce db --set-id dbf77f9f --release --await' 2024-09-10T09:52:19.458 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24542 quiesce db --set-id dbf77f9f --release --await 2024-09-10T09:52:19.484 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:52:19.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:52:19 smithi159 ceph-mon[35513]: pgmap v812: 129 pgs: 129 active+clean; 287 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 585 KiB/s wr, 35 op/s 2024-09-10T09:52:19.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:52:19 smithi159 ceph-mon[35513]: from='client.? 172.21.15.5:0/631024852' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:52:20.172 INFO:tasks.workunit.client.0.smithi005.stdout: 1 cleanup 601 sec 2024-09-10T09:52:20.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:20.043+0000 7f98a3200640 1 -- 172.21.15.5:0/2602477155 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f98a410e280 msgr2=0x7f98a4110670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:52:20.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:20.043+0000 7f98a3200640 1 --2- 172.21.15.5:0/2602477155 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f98a410e280 0x7f98a4110670 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f989c009ca0 tx=0x7f989c02a950 comp rx=0 tx=0).stop 2024-09-10T09:52:20.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:20.043+0000 7f98a3200640 1 -- 172.21.15.5:0/2602477155 shutdown_connections 2024-09-10T09:52:20.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:20.043+0000 7f98a3200640 1 --2- 172.21.15.5:0/2602477155 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f98a410e280 0x7f98a4110670 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:20.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:20.043+0000 7f98a3200640 1 --2- 172.21.15.5:0/2602477155 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f98a4074e30 0x7f98a410dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:20.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:20.043+0000 7f98a3200640 1 --2- 172.21.15.5:0/2602477155 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f98a4074510 0x7f98a40748f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:20.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:20.043+0000 7f98a3200640 1 -- 172.21.15.5:0/2602477155 >> 172.21.15.5:0/2602477155 conn(0x7f98a40fd740 msgr2=0x7f98a40ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:52:20.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:20.043+0000 7f98a3200640 1 -- 172.21.15.5:0/2602477155 shutdown_connections 2024-09-10T09:52:20.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:20.043+0000 7f98a3200640 1 -- 172.21.15.5:0/2602477155 wait complete. 2024-09-10T09:52:20.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:20.044+0000 7f98a3200640 1 Processor -- start 2024-09-10T09:52:20.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:20.044+0000 7f98a3200640 1 -- start start 2024-09-10T09:52:20.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:20.045+0000 7f98a3200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f98a4074510 0x7f98a419a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:52:20.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:20.045+0000 7f98a3200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f98a4074e30 0x7f98a419a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:52:20.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:20.045+0000 7f98a3200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f98a410e280 0x7f98a419ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:52:20.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:20.045+0000 7f98a3200640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f98a4113fb0 con 0x7f98a4074510 2024-09-10T09:52:20.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:20.045+0000 7f98a3200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f98a4113e30 con 0x7f98a410e280 2024-09-10T09:52:20.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:20.045+0000 7f98a3200640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f98a4114130 con 0x7f98a4074e30 2024-09-10T09:52:20.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:20.045+0000 7f98a1e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f98a4074510 0x7f98a419a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:52:20.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:20.045+0000 7f9899400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f98a4074e30 0x7f98a419a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:52:20.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:20.045+0000 7f98a1e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f98a4074510 0x7f98a419a460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47838/0 (socket says 172.21.15.5:47838) 2024-09-10T09:52:20.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:20.045+0000 7f9899400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f98a4074e30 0x7f98a419a9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:41798/0 (socket says 172.21.15.5:41798) 2024-09-10T09:52:20.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:20.045+0000 7f98a1e00640 1 -- 172.21.15.5:0/1975150609 learned_addr learned my addr 172.21.15.5:0/1975150609 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:52:20.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:20.045+0000 7f98a2800640 1 --2- 172.21.15.5:0/1975150609 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f98a410e280 0x7f98a419ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:52:20.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:20.046+0000 7f98a1e00640 1 -- 172.21.15.5:0/1975150609 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f98a4074e30 msgr2=0x7f98a419a9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:52:20.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:20.046+0000 7f98a1e00640 1 --2- 172.21.15.5:0/1975150609 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f98a4074e30 0x7f98a419a9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:20.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:20.046+0000 7f98a1e00640 1 -- 172.21.15.5:0/1975150609 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f98a410e280 msgr2=0x7f98a419ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:52:20.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:20.046+0000 7f98a1e00640 1 --2- 172.21.15.5:0/1975150609 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f98a410e280 0x7f98a419ed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:20.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:20.046+0000 7f98a1e00640 1 -- 172.21.15.5:0/1975150609 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98a419f410 con 0x7f98a4074510 2024-09-10T09:52:20.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:20.046+0000 7f9899400640 1 --2- 172.21.15.5:0/1975150609 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f98a4074e30 0x7f98a419a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:52:20.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:20.046+0000 7f98a1e00640 1 --2- 172.21.15.5:0/1975150609 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f98a4074510 0x7f98a419a460 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f989400e9f0 tx=0x7f989400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:52:20.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:20.046+0000 7f989b400640 1 -- 172.21.15.5:0/1975150609 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f989400cd80 con 0x7f98a4074510 2024-09-10T09:52:20.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:20.046+0000 7f989b400640 1 -- 172.21.15.5:0/1975150609 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9894010910 con 0x7f98a4074510 2024-09-10T09:52:20.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:20.046+0000 7f98a3200640 1 -- 172.21.15.5:0/1975150609 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98a419f700 con 0x7f98a4074510 2024-09-10T09:52:20.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:20.046+0000 7f989b400640 1 -- 172.21.15.5:0/1975150609 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9894018990 con 0x7f98a4074510 2024-09-10T09:52:20.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:20.046+0000 7f98a3200640 1 -- 172.21.15.5:0/1975150609 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98a41a6fe0 con 0x7f98a4074510 2024-09-10T09:52:20.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:20.049+0000 7f989b400640 1 -- 172.21.15.5:0/1975150609 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f9894010ac0 con 0x7f98a4074510 2024-09-10T09:52:20.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:20.050+0000 7f98ab9f9740 1 Processor -- start 2024-09-10T09:52:20.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:20.050+0000 7f98ab9f9740 1 -- start start 2024-09-10T09:52:20.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:20.050+0000 7f98ab9f9740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5564175fccb0 0x5564175fd090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:52:20.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:20.050+0000 7f98ab9f9740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5564175fd5d0 0x556417607950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:52:20.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:20.050+0000 7f98ab9f9740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556417607e90 0x55641760a280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:52:20.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:20.050+0000 7f98ab9f9740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x556417563290 con 0x556417607e90 2024-09-10T09:52:20.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:20.050+0000 7f98ab9f9740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55641750ab40 con 0x5564175fd5d0 2024-09-10T09:52:20.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:20.050+0000 7f98ab9f9740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55641755f250 con 0x5564175fccb0 2024-09-10T09:52:20.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:20.050+0000 7f9899400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556417607e90 0x55641760a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:52:20.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:20.050+0000 7f98a1e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5564175fd5d0 0x556417607950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:52:20.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:20.050+0000 7f9899400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556417607e90 0x55641760a280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47840/0 (socket says 172.21.15.5:47840) 2024-09-10T09:52:20.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:20.050+0000 7f9899400640 1 -- 172.21.15.5:0/3737358619 learned_addr learned my addr 172.21.15.5:0/3737358619 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:52:20.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:20.051+0000 7f98a2800640 1 --2- 172.21.15.5:0/3737358619 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5564175fccb0 0x5564175fd090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:52:20.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:20.051+0000 7f9899400640 1 -- 172.21.15.5:0/3737358619 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5564175fccb0 msgr2=0x5564175fd090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:52:20.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:20.051+0000 7f9899400640 1 --2- 172.21.15.5:0/3737358619 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5564175fccb0 0x5564175fd090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:20.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:20.051+0000 7f9899400640 1 -- 172.21.15.5:0/3737358619 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5564175fd5d0 msgr2=0x556417607950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:52:20.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:20.051+0000 7f9899400640 1 --2- 172.21.15.5:0/3737358619 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5564175fd5d0 0x556417607950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:20.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:20.051+0000 7f9899400640 1 -- 172.21.15.5:0/3737358619 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5564172d46a0 con 0x556417607e90 2024-09-10T09:52:20.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:20.051+0000 7f989b400640 1 --2- 172.21.15.5:0/1975150609 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f98880a2b80 0x7f98880a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:52:20.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:20.051+0000 7f9899400640 1 --2- 172.21.15.5:0/3737358619 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556417607e90 0x55641760a280 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f988c011470 tx=0x7f988c011930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aff0cda014d711e9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:52:20.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:20.051+0000 7f989b400640 1 -- 172.21.15.5:0/1975150609 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f98940cf780 con 0x7f98a4074510 2024-09-10T09:52:20.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:20.051+0000 7f98a2800640 1 --2- 172.21.15.5:0/1975150609 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f98880a2b80 0x7f98880a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:52:20.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:20.052+0000 7f9893400640 1 -- 172.21.15.5:0/3737358619 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f988c01e070 con 0x556417607e90 2024-09-10T09:52:20.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:20.052+0000 7f9893400640 1 -- 172.21.15.5:0/3737358619 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f988c004310 con 0x556417607e90 2024-09-10T09:52:20.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:20.052+0000 7f9893400640 1 -- 172.21.15.5:0/3737358619 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f988c00faa0 con 0x556417607e90 2024-09-10T09:52:20.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:20.052+0000 7f98ab9f9740 1 -- 172.21.15.5:0/3737358619 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556417607e90 msgr2=0x55641760a280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:52:20.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:20.052+0000 7f98a2800640 1 --2- 172.21.15.5:0/1975150609 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f98880a2b80 0x7f98880a5040 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f989c005e30 tx=0x7f989c005c50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:52:20.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:20.052+0000 7f98ab9f9740 1 --2- 172.21.15.5:0/3737358619 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556417607e90 0x55641760a280 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f988c011470 tx=0x7f988c011930 comp rx=0 tx=0).stop 2024-09-10T09:52:20.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:20.052+0000 7f98ab9f9740 1 -- 172.21.15.5:0/3737358619 shutdown_connections 2024-09-10T09:52:20.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:20.052+0000 7f98ab9f9740 1 --2- 172.21.15.5:0/3737358619 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556417607e90 0x55641760a280 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:20.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:20.052+0000 7f98ab9f9740 1 --2- 172.21.15.5:0/3737358619 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5564175fd5d0 0x556417607950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:20.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:20.052+0000 7f98ab9f9740 1 --2- 172.21.15.5:0/3737358619 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5564175fccb0 0x5564175fd090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:20.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:20.052+0000 7f98ab9f9740 1 -- 172.21.15.5:0/3737358619 >> 172.21.15.5:0/3737358619 conn(0x5564175f9240 msgr2=0x5564175f9620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:52:20.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:20.053+0000 7f98ab9f9740 1 -- 172.21.15.5:0/3737358619 shutdown_connections 2024-09-10T09:52:20.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:20.053+0000 7f98ab9f9740 1 -- 172.21.15.5:0/3737358619 wait complete. 2024-09-10T09:52:20.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:20.053+0000 7f98ab9f9740 1 -- start start 2024-09-10T09:52:20.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:20.053+0000 7f98ab9f9740 1 Processor -- start 2024-09-10T09:52:20.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:20.054+0000 7f98ab9f9740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5564175fccb0 0x5564176077a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:52:20.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:20.054+0000 7f98ab9f9740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5564175fd5d0 0x556417604a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:52:20.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:20.054+0000 7f98ab9f9740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556417607e90 0x556417604f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:52:20.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:20.054+0000 7f98ab9f9740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55641760ab00 con 0x556417607e90 2024-09-10T09:52:20.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:20.054+0000 7f98ab9f9740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55641751dfa0 con 0x5564175fd5d0 2024-09-10T09:52:20.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:20.054+0000 7f98ab9f9740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55641760ac80 con 0x5564175fccb0 2024-09-10T09:52:20.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:20.054+0000 7f98ab9f9740 10 client.0 _ll_register_callbacks cb 0x556417271ba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:52:20.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:20.054+0000 7f98a2800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556417607e90 0x556417604f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:52:20.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:20.054+0000 7f98a2800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556417607e90 0x556417604f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47854/0 (socket says 172.21.15.5:47854) 2024-09-10T09:52:20.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:20.054+0000 7f98a2800640 1 -- 172.21.15.5:0/2959649672 learned_addr learned my addr 172.21.15.5:0/2959649672 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:52:20.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:20.054+0000 7f98a1e00640 1 --2- 172.21.15.5:0/2959649672 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5564175fccb0 0x5564176077a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:52:20.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:20.054+0000 7f9899400640 1 --2- 172.21.15.5:0/2959649672 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5564175fd5d0 0x556417604a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:52:20.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:20.054+0000 7f98a2800640 1 -- 172.21.15.5:0/2959649672 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5564175fccb0 msgr2=0x5564176077a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:52:20.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:20.054+0000 7f98a2800640 1 --2- 172.21.15.5:0/2959649672 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5564175fccb0 0x5564176077a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:20.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:20.054+0000 7f98a2800640 1 -- 172.21.15.5:0/2959649672 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5564175fd5d0 msgr2=0x556417604a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:52:20.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:20.054+0000 7f98a2800640 1 --2- 172.21.15.5:0/2959649672 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5564175fd5d0 0x556417604a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:20.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:20.054+0000 7f98a2800640 1 -- 172.21.15.5:0/2959649672 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556417605790 con 0x556417607e90 2024-09-10T09:52:20.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:20.055+0000 7f98a2800640 1 --2- 172.21.15.5:0/2959649672 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556417607e90 0x556417604f90 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f98a40754f0 tx=0x7f989c0095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=751d0931361abd2f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:52:20.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:20.055+0000 7f9887e00640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-10T09:52:20.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:20.055+0000 7f9887e00640 1 -- 172.21.15.5:0/2959649672 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f989c056060 con 0x556417607e90 2024-09-10T09:52:20.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:20.055+0000 7f9887e00640 1 -- 172.21.15.5:0/2959649672 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f989c003f00 con 0x556417607e90 2024-09-10T09:52:20.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:20.055+0000 7f98ab9f9740 1 -- 172.21.15.5:0/2959649672 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f988c004310 con 0x556417607e90 2024-09-10T09:52:20.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:20.055+0000 7f9887e00640 1 -- 172.21.15.5:0/2959649672 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f989c05d050 con 0x556417607e90 2024-09-10T09:52:20.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:20.055+0000 7f98ab9f9740 10 client.18249 fetch_fsmap learned FSMap version 36 2024-09-10T09:52:20.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:20.055+0000 7f98ab9f9740 1 -- 172.21.15.5:0/2959649672 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x556417616a30 con 0x556417607e90 2024-09-10T09:52:20.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:20.056+0000 7f9887e00640 1 -- 172.21.15.5:0/2959649672 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f988c004310 con 0x556417607e90 2024-09-10T09:52:20.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:20.056+0000 7f9887e00640 1 -- 172.21.15.5:0/2959649672 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 36) ==== 8370+0+0 (secure 0 0 0) 0x7f989c030870 con 0x556417607e90 2024-09-10T09:52:20.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:20.056+0000 7f98ab9f9740 10 client.18249 fetch_fsmap finished waiting for FSMap version 36 2024-09-10T09:52:20.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:20.056+0000 7f98ab9f9740 10 client.18249 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T09:52:20.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:20.056+0000 7f98ab9f9740 20 client.18249 populate_metadata read hostname 'smithi005' 2024-09-10T09:52:20.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:20.056+0000 7f98ab9f9740 1 --2- 172.21.15.5:0/2959649672 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x556417617ef0 0x55641761a340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:52:20.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:20.056+0000 7f98ab9f9740 4 client.18249 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:52:20.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:20.056+0000 7f98ab9f9740 1 -- 172.21.15.5:0/2959649672 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556417570800 con 0x556417617ef0 2024-09-10T09:52:20.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:20.056+0000 7f98a1e00640 1 --2- 172.21.15.5:0/2959649672 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x556417617ef0 0x55641761a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:52:20.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:20.057+0000 7f98a1e00640 1 --2- 172.21.15.5:0/2959649672 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x556417617ef0 0x55641761a340 crc :-1 s=READY pgs=213 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a8a52f2ec6e7291e server_cookie=7ef0cc72ea3bbcb2 in_seq=0 out_seq=0 2024-09-10T09:52:20.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:20.057+0000 7f9887e00640 10 client.18249 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T09:52:20.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:20.058+0000 7f9887e00640 1 -- 172.21.15.5:0/2959649672 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556417570800 con 0x556417617ef0 2024-09-10T09:52:20.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:20.058+0000 7f9887e00640 10 client.18249 handle_command_reply: tid=0 2024-09-10T09:52:20.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:20.058+0000 7f98ab9f9740 1 client.18249 shutdown 2024-09-10T09:52:20.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:20.058+0000 7f98ab9f9740 1 -- 172.21.15.5:0/2959649672 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556417607e90 msgr2=0x556417604f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:52:20.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:20.058+0000 7f98ab9f9740 1 --2- 172.21.15.5:0/2959649672 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556417607e90 0x556417604f90 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f98a40754f0 tx=0x7f989c0095c0 comp rx=0 tx=0).stop 2024-09-10T09:52:20.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:20.058+0000 7f98ab9f9740 1 -- 172.21.15.5:0/2959649672 shutdown_connections 2024-09-10T09:52:20.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:20.058+0000 7f98ab9f9740 1 --2- 172.21.15.5:0/2959649672 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x556417617ef0 0x55641761a340 crc :-1 s=READY pgs=213 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:20.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:20.058+0000 7f98ab9f9740 1 --2- 172.21.15.5:0/2959649672 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556417607e90 0x556417604f90 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:20.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:20.058+0000 7f98ab9f9740 1 --2- 172.21.15.5:0/2959649672 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5564175fd5d0 0x556417604a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:20.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:20.058+0000 7f98ab9f9740 1 --2- 172.21.15.5:0/2959649672 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5564175fccb0 0x5564176077a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:20.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:20.058+0000 7f98ab9f9740 1 -- 172.21.15.5:0/2959649672 >> 172.21.15.5:0/2959649672 conn(0x5564175f9240 msgr2=0x5564175fef20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:52:20.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:20.058+0000 7f9887e00640 0 client.18249 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T09:52:20.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:20.058+0000 7f98ab9f9740 1 -- 172.21.15.5:0/2959649672 shutdown_connections 2024-09-10T09:52:20.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:20.058+0000 7f98ab9f9740 1 -- 172.21.15.5:0/2959649672 wait complete. 2024-09-10T09:52:20.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:20.059+0000 7f98ab9f9740 20 client.18249 trim_cache size 0 max 16384 2024-09-10T09:52:20.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:20.077+0000 7f98ab9f9740 1 Processor -- start 2024-09-10T09:52:20.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:20.077+0000 7f98ab9f9740 1 -- start start 2024-09-10T09:52:20.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:20.077+0000 7f98ab9f9740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5564175f9240 0x5564175ffeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:52:20.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:20.077+0000 7f98ab9f9740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5564175fccb0 0x5564175f8160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:52:20.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:20.077+0000 7f98ab9f9740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5564175fd5d0 0x55641761fc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:52:20.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:20.077+0000 7f98ab9f9740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f98a41b0de0 con 0x5564175f9240 2024-09-10T09:52:20.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:20.077+0000 7f98ab9f9740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f98a41b2920 con 0x5564175fd5d0 2024-09-10T09:52:20.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:20.077+0000 7f98ab9f9740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f98a41b14b0 con 0x5564175fccb0 2024-09-10T09:52:20.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:20.077+0000 7f98a1e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5564175f9240 0x5564175ffeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:52:20.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:20.077+0000 7f98a1e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5564175f9240 0x5564175ffeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47864/0 (socket says 172.21.15.5:47864) 2024-09-10T09:52:20.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:20.077+0000 7f98a1e00640 1 -- 172.21.15.5:0/3713360972 learned_addr learned my addr 172.21.15.5:0/3713360972 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:52:20.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:20.077+0000 7f98a2800640 1 --2- 172.21.15.5:0/3713360972 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5564175fd5d0 0x55641761fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:52:20.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:20.078+0000 7f9899400640 1 --2- 172.21.15.5:0/3713360972 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5564175fccb0 0x5564175f8160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:52:20.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:20.078+0000 7f98a1e00640 1 -- 172.21.15.5:0/3713360972 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5564175fccb0 msgr2=0x5564175f8160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:52:20.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:20.078+0000 7f98a1e00640 1 --2- 172.21.15.5:0/3713360972 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5564175fccb0 0x5564175f8160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:20.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:20.078+0000 7f98a1e00640 1 -- 172.21.15.5:0/3713360972 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5564175fd5d0 msgr2=0x55641761fc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:52:20.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:20.078+0000 7f98a1e00640 1 --2- 172.21.15.5:0/3713360972 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5564175fd5d0 0x55641761fc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:20.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:20.078+0000 7f98a1e00640 1 -- 172.21.15.5:0/3713360972 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5564175fa3c0 con 0x5564175f9240 2024-09-10T09:52:20.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:20.078+0000 7f98a2800640 1 --2- 172.21.15.5:0/3713360972 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5564175fd5d0 0x55641761fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:52:20.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:20.078+0000 7f98a1e00640 1 --2- 172.21.15.5:0/3713360972 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5564175f9240 0x5564175ffeb0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f989409bbd0 tx=0x7f98940d5530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=315a0ac4d78aa1de server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:52:20.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:20.078+0000 7f9892a00640 1 -- 172.21.15.5:0/3713360972 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98940d5020 con 0x5564175f9240 2024-09-10T09:52:20.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:20.078+0000 7f9892a00640 1 -- 172.21.15.5:0/3713360972 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f989409ff40 con 0x5564175f9240 2024-09-10T09:52:20.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:20.078+0000 7f9892a00640 1 -- 172.21.15.5:0/3713360972 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98940cf3d0 con 0x5564175f9240 2024-09-10T09:52:20.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:20.079+0000 7f98ab9f9740 1 -- 172.21.15.5:0/3713360972 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5564175f9240 msgr2=0x5564175ffeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:52:20.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:20.079+0000 7f98ab9f9740 1 --2- 172.21.15.5:0/3713360972 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5564175f9240 0x5564175ffeb0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f989409bbd0 tx=0x7f98940d5530 comp rx=0 tx=0).stop 2024-09-10T09:52:20.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:20.079+0000 7f98ab9f9740 1 -- 172.21.15.5:0/3713360972 shutdown_connections 2024-09-10T09:52:20.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:20.079+0000 7f98ab9f9740 1 --2- 172.21.15.5:0/3713360972 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5564175fd5d0 0x55641761fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:20.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:20.079+0000 7f98ab9f9740 1 --2- 172.21.15.5:0/3713360972 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5564175fccb0 0x5564175f8160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:20.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:20.079+0000 7f98ab9f9740 1 --2- 172.21.15.5:0/3713360972 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5564175f9240 0x5564175ffeb0 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:20.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:20.079+0000 7f98ab9f9740 1 -- 172.21.15.5:0/3713360972 >> 172.21.15.5:0/3713360972 conn(0x5564176050f0 msgr2=0x5564175f7a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:52:20.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:20.079+0000 7f98ab9f9740 1 -- 172.21.15.5:0/3713360972 shutdown_connections 2024-09-10T09:52:20.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:20.079+0000 7f98ab9f9740 1 -- 172.21.15.5:0/3713360972 wait complete. 2024-09-10T09:52:20.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:20.079+0000 7f98ab9f9740 1 -- start start 2024-09-10T09:52:20.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:20.080+0000 7f98ab9f9740 1 Processor -- start 2024-09-10T09:52:20.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:20.080+0000 7f98ab9f9740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5564175f9240 0x55641760c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:52:20.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:20.080+0000 7f98ab9f9740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5564175fccb0 0x55641760c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:52:20.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:20.080+0000 7f98ab9f9740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5564175fd5d0 0x55641767bc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:52:20.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:20.080+0000 7f98ab9f9740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f98a41b2250 con 0x5564175f9240 2024-09-10T09:52:20.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:20.080+0000 7f98ab9f9740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f98a41b1b80 con 0x5564175fccb0 2024-09-10T09:52:20.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:20.080+0000 7f98ab9f9740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f98a41aedc0 con 0x5564175fd5d0 2024-09-10T09:52:20.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:20.080+0000 7f98ab9f9740 10 client.0 _ll_register_callbacks cb 0x7f989c009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:52:20.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:20.080+0000 7f98a1e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5564175f9240 0x55641760c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:52:20.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:20.080+0000 7f98a1e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5564175f9240 0x55641760c4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47878/0 (socket says 172.21.15.5:47878) 2024-09-10T09:52:20.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:20.080+0000 7f98a1e00640 1 -- 172.21.15.5:0/731093477 learned_addr learned my addr 172.21.15.5:0/731093477 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:52:20.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:20.081+0000 7f9899400640 1 --2- 172.21.15.5:0/731093477 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5564175fccb0 0x55641760c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:52:20.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:20.081+0000 7f98a2800640 1 --2- 172.21.15.5:0/731093477 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5564175fd5d0 0x55641767bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:52:20.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:20.081+0000 7f98a1e00640 1 -- 172.21.15.5:0/731093477 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5564175fd5d0 msgr2=0x55641767bc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:52:20.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:20.081+0000 7f98a1e00640 1 --2- 172.21.15.5:0/731093477 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5564175fd5d0 0x55641767bc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:20.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:20.081+0000 7f98a1e00640 1 -- 172.21.15.5:0/731093477 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5564175fccb0 msgr2=0x55641760c9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:52:20.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:20.081+0000 7f98a1e00640 1 --2- 172.21.15.5:0/731093477 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5564175fccb0 0x55641760c9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:20.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:20.081+0000 7f98a1e00640 1 -- 172.21.15.5:0/731093477 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55641767c260 con 0x5564175f9240 2024-09-10T09:52:20.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:20.081+0000 7f98a1e00640 1 --2- 172.21.15.5:0/731093477 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5564175f9240 0x55641760c4b0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f98940a42d0 tx=0x7f98940ccdf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eef75d97286919e7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:52:20.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:20.081+0000 7f9887e00640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-10T09:52:20.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:20.081+0000 7f9887e00640 1 -- 172.21.15.5:0/731093477 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98940d5020 con 0x5564175f9240 2024-09-10T09:52:20.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:20.081+0000 7f98ab9f9740 1 -- 172.21.15.5:0/731093477 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f989409ff40 con 0x5564175f9240 2024-09-10T09:52:20.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:20.081+0000 7f9887e00640 1 -- 172.21.15.5:0/731093477 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f98940982b0 con 0x5564175f9240 2024-09-10T09:52:20.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:20.082+0000 7f9887e00640 1 -- 172.21.15.5:0/731093477 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f98940a4800 con 0x5564175f9240 2024-09-10T09:52:20.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:20.082+0000 7f98ab9f9740 10 client.18255 fetch_fsmap learned FSMap version 36 2024-09-10T09:52:20.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:20.082+0000 7f98ab9f9740 1 -- 172.21.15.5:0/731093477 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x556417683610 con 0x5564175f9240 2024-09-10T09:52:20.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:20.082+0000 7f9887e00640 1 -- 172.21.15.5:0/731093477 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f989409ff40 con 0x5564175f9240 2024-09-10T09:52:20.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:20.082+0000 7f9887e00640 1 -- 172.21.15.5:0/731093477 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 36) ==== 8370+0+0 (secure 0 0 0) 0x7f98940ce040 con 0x5564175f9240 2024-09-10T09:52:20.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:20.082+0000 7f98ab9f9740 10 client.18255 fetch_fsmap finished waiting for FSMap version 36 2024-09-10T09:52:20.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:20.082+0000 7f98ab9f9740 10 client.18255 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T09:52:20.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:20.082+0000 7f98ab9f9740 20 client.18255 populate_metadata read hostname 'smithi005' 2024-09-10T09:52:20.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:20.082+0000 7f98ab9f9740 1 --2- 172.21.15.5:0/731093477 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x556417607e90 0x556417685c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:52:20.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:20.082+0000 7f98ab9f9740 4 client.18255 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-10T09:52:20.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:20.082+0000 7f98ab9f9740 1 -- 172.21.15.5:0/731093477 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55641756ad10 con 0x556417607e90 2024-09-10T09:52:20.202 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-10T09:52:20.202 INFO:teuthology.orchestra.run.smithi005.stdout: "return_code": 0, 2024-09-10T09:52:20.202 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_tag": "e709f78a-84db-4fa8-a1fc-ddc55265697b", 2024-09-10T09:52:20.202 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": "asynchronous" 2024-09-10T09:52:20.202 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-10T09:52:20.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:20.083+0000 7f9899400640 1 --2- 172.21.15.5:0/731093477 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x556417607e90 0x556417685c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:52:20.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:20.083+0000 7f9899400640 1 --2- 172.21.15.5:0/731093477 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x556417607e90 0x556417685c70 crc :-1 s=READY pgs=214 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=19f9a5b141f4161b server_cookie=bb65829b9d306282 in_seq=0 out_seq=0 2024-09-10T09:52:20.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:20.083+0000 7f9887e00640 10 client.18255 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T09:52:20.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:20.084+0000 7f9887e00640 1 -- 172.21.15.5:0/731093477 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55641756ad10 con 0x556417607e90 2024-09-10T09:52:20.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:20.084+0000 7f9887e00640 10 client.18255 handle_command_reply: tid=0 2024-09-10T09:52:20.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:20.084+0000 7f98ab9f9740 1 client.18255 shutdown 2024-09-10T09:52:20.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:20.084+0000 7f98ab9f9740 1 -- 172.21.15.5:0/731093477 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5564175f9240 msgr2=0x55641760c4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:52:20.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:20.084+0000 7f98ab9f9740 1 --2- 172.21.15.5:0/731093477 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5564175f9240 0x55641760c4b0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f98940a42d0 tx=0x7f98940ccdf0 comp rx=0 tx=0).stop 2024-09-10T09:52:20.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:20.085+0000 7f98ab9f9740 1 -- 172.21.15.5:0/731093477 shutdown_connections 2024-09-10T09:52:20.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:20.085+0000 7f98ab9f9740 1 --2- 172.21.15.5:0/731093477 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x556417607e90 0x556417685c70 crc :-1 s=READY pgs=214 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:20.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:20.085+0000 7f98ab9f9740 1 --2- 172.21.15.5:0/731093477 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5564175fd5d0 0x55641767bc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:20.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:20.085+0000 7f98ab9f9740 1 --2- 172.21.15.5:0/731093477 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5564175fccb0 0x55641760c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:20.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:20.085+0000 7f98ab9f9740 1 --2- 172.21.15.5:0/731093477 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5564175f9240 0x55641760c4b0 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:20.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:20.085+0000 7f98ab9f9740 1 -- 172.21.15.5:0/731093477 >> 172.21.15.5:0/731093477 conn(0x5564176050f0 msgr2=0x5564175f8270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:52:20.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:20.085+0000 7f9887e00640 0 client.18255 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T09:52:20.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:20.085+0000 7f98ab9f9740 1 -- 172.21.15.5:0/731093477 shutdown_connections 2024-09-10T09:52:20.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:20.085+0000 7f98ab9f9740 1 -- 172.21.15.5:0/731093477 wait complete. 2024-09-10T09:52:20.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:20.085+0000 7f98ab9f9740 20 client.18255 trim_cache size 0 max 16384 2024-09-10T09:52:20.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:20.085+0000 7f98a3200640 1 -- 172.21.15.5:0/1975150609 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f98880a2b80 msgr2=0x7f98880a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:52:20.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:20.085+0000 7f98a3200640 1 --2- 172.21.15.5:0/1975150609 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f98880a2b80 0x7f98880a5040 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f989c005e30 tx=0x7f989c005c50 comp rx=0 tx=0).stop 2024-09-10T09:52:20.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:20.085+0000 7f98a3200640 1 -- 172.21.15.5:0/1975150609 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f98a4074510 msgr2=0x7f98a419a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:52:20.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:20.085+0000 7f98a3200640 1 --2- 172.21.15.5:0/1975150609 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f98a4074510 0x7f98a419a460 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f989400e9f0 tx=0x7f989400eeb0 comp rx=0 tx=0).stop 2024-09-10T09:52:20.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:20.086+0000 7f98a3200640 1 -- 172.21.15.5:0/1975150609 shutdown_connections 2024-09-10T09:52:20.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:20.086+0000 7f98a3200640 1 --2- 172.21.15.5:0/1975150609 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f98880a2b80 0x7f98880a5040 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:20.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:20.086+0000 7f98a3200640 1 --2- 172.21.15.5:0/1975150609 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f98a410e280 0x7f98a419ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:20.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:20.086+0000 7f98a3200640 1 --2- 172.21.15.5:0/1975150609 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f98a4074e30 0x7f98a419a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:20.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:20.086+0000 7f98a3200640 1 --2- 172.21.15.5:0/1975150609 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f98a4074510 0x7f98a419a460 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:20.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:20.086+0000 7f98a3200640 1 -- 172.21.15.5:0/1975150609 >> 172.21.15.5:0/1975150609 conn(0x7f98a40fd740 msgr2=0x7f98a40fe3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:52:20.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:20.086+0000 7f98a3200640 1 -- 172.21.15.5:0/1975150609 shutdown_connections 2024-09-10T09:52:20.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:20.086+0000 7f98a3200640 1 -- 172.21.15.5:0/1975150609 wait complete. 2024-09-10T09:52:20.979 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:52:21.002 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T09:52:21.172 INFO:tasks.workunit.client.0.smithi005.stdout: 1 cleanup 602 sec 2024-09-10T09:52:21.292 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:52:21 smithi005 ceph-mon[30350]: pgmap v813: 129 pgs: 129 active+clean; 287 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 340 KiB/s wr, 30 op/s 2024-09-10T09:52:21.292 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:52:21 smithi005 ceph-mon[30350]: scrub queued for path: / 2024-09-10T09:52:21.292 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:52:21 smithi005 ceph-mon[30350]: scrub summary: idle+waiting paths [/] 2024-09-10T09:52:21.292 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:52:21 smithi005 ceph-mon[30350]: scrub summary: active paths [/] 2024-09-10T09:52:21.292 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:52:21 smithi005 ceph-mon[30350]: scrub summary: active paths [/] 2024-09-10T09:52:21.293 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:52:21 smithi005 ceph-mon[30350]: scrub summary: idle+waiting paths [/] 2024-09-10T09:52:21.293 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:52:21 smithi005 ceph-mon[30350]: scrub summary: idle+waiting paths [/] 2024-09-10T09:52:21.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:52:21 smithi102 ceph-mon[35442]: pgmap v813: 129 pgs: 129 active+clean; 287 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 340 KiB/s wr, 30 op/s 2024-09-10T09:52:21.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:52:21 smithi102 ceph-mon[35442]: scrub queued for path: / 2024-09-10T09:52:21.385 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:52:21 smithi102 ceph-mon[35442]: scrub summary: idle+waiting paths [/] 2024-09-10T09:52:21.385 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:52:21 smithi102 ceph-mon[35442]: scrub summary: active paths [/] 2024-09-10T09:52:21.385 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:52:21 smithi102 ceph-mon[35442]: scrub summary: active paths [/] 2024-09-10T09:52:21.385 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:52:21 smithi102 ceph-mon[35442]: scrub summary: idle+waiting paths [/] 2024-09-10T09:52:21.385 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:52:21 smithi102 ceph-mon[35442]: scrub summary: idle+waiting paths [/] 2024-09-10T09:52:21.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:52:21 smithi159 ceph-mon[35513]: pgmap v813: 129 pgs: 129 active+clean; 287 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 340 KiB/s wr, 30 op/s 2024-09-10T09:52:21.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:52:21 smithi159 ceph-mon[35513]: scrub queued for path: / 2024-09-10T09:52:21.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:52:21 smithi159 ceph-mon[35513]: scrub summary: idle+waiting paths [/] 2024-09-10T09:52:21.601 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:52:21 smithi159 ceph-mon[35513]: scrub summary: active paths [/] 2024-09-10T09:52:21.601 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:52:21 smithi159 ceph-mon[35513]: scrub summary: active paths [/] 2024-09-10T09:52:21.601 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:52:21 smithi159 ceph-mon[35513]: scrub summary: idle+waiting paths [/] 2024-09-10T09:52:21.601 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:52:21 smithi159 ceph-mon[35513]: scrub summary: idle+waiting paths [/] 2024-09-10T09:52:21.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:21.560+0000 7fc507e00640 1 -- 172.21.15.5:0/3918322970 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc5081019a0 msgr2=0x7fc508101e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:52:21.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:21.560+0000 7fc507e00640 1 --2- 172.21.15.5:0/3918322970 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc5081019a0 0x7fc508101e00 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fc4f0009930 tx=0x7fc4f002a420 comp rx=0 tx=0).stop 2024-09-10T09:52:21.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:21.560+0000 7fc507e00640 1 -- 172.21.15.5:0/3918322970 shutdown_connections 2024-09-10T09:52:21.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:21.560+0000 7fc507e00640 1 --2- 172.21.15.5:0/3918322970 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc508102340 0x7fc50810e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:21.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:21.560+0000 7fc507e00640 1 --2- 172.21.15.5:0/3918322970 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc5081019a0 0x7fc508101e00 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:21.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:21.560+0000 7fc507e00640 1 --2- 172.21.15.5:0/3918322970 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc5081079a0 0x7fc508107d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:21.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:21.560+0000 7fc507e00640 1 -- 172.21.15.5:0/3918322970 >> 172.21.15.5:0/3918322970 conn(0x7fc5080fd700 msgr2=0x7fc5080ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:52:21.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:21.560+0000 7fc507e00640 1 -- 172.21.15.5:0/3918322970 shutdown_connections 2024-09-10T09:52:21.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:21.560+0000 7fc507e00640 1 -- 172.21.15.5:0/3918322970 wait complete. 2024-09-10T09:52:21.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:21.561+0000 7fc507e00640 1 Processor -- start 2024-09-10T09:52:21.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:21.561+0000 7fc507e00640 1 -- start start 2024-09-10T09:52:21.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:21.561+0000 7fc507e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc5081019a0 0x7fc50819a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:52:21.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:21.561+0000 7fc507e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc508102340 0x7fc50819a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:52:21.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:21.561+0000 7fc507e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc5081079a0 0x7fc50819ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:52:21.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:21.561+0000 7fc507e00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc508111df0 con 0x7fc5081019a0 2024-09-10T09:52:21.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:21.561+0000 7fc507e00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fc508111c70 con 0x7fc5081079a0 2024-09-10T09:52:21.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:21.561+0000 7fc507e00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fc508111f70 con 0x7fc508102340 2024-09-10T09:52:21.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:21.561+0000 7fc506a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc5081019a0 0x7fc50819a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:52:21.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:21.562+0000 7fc506a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc5081019a0 0x7fc50819a4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43988/0 (socket says 172.21.15.5:43988) 2024-09-10T09:52:21.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:21.562+0000 7fc506a00640 1 -- 172.21.15.5:0/4030761711 learned_addr learned my addr 172.21.15.5:0/4030761711 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:52:21.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:21.562+0000 7fc4ffe00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc508102340 0x7fc50819a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:52:21.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:21.562+0000 7fc507400640 1 --2- 172.21.15.5:0/4030761711 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc5081079a0 0x7fc50819ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:52:21.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:21.562+0000 7fc4ffe00640 1 -- 172.21.15.5:0/4030761711 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc5081079a0 msgr2=0x7fc50819ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:52:21.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:21.562+0000 7fc4ffe00640 1 --2- 172.21.15.5:0/4030761711 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc5081079a0 0x7fc50819ed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:21.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:21.562+0000 7fc4ffe00640 1 -- 172.21.15.5:0/4030761711 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc5081019a0 msgr2=0x7fc50819a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:52:21.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:21.562+0000 7fc4ffe00640 1 --2- 172.21.15.5:0/4030761711 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc5081019a0 0x7fc50819a4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:21.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:21.562+0000 7fc4ffe00640 1 -- 172.21.15.5:0/4030761711 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc50819f460 con 0x7fc508102340 2024-09-10T09:52:21.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:21.563+0000 7fc4ffe00640 1 --2- 172.21.15.5:0/4030761711 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc508102340 0x7fc50819a9f0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fc4f002a940 tx=0x7fc4f0003b20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:52:21.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:21.563+0000 7fc4ff400640 1 -- 172.21.15.5:0/4030761711 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4f003f070 con 0x7fc508102340 2024-09-10T09:52:21.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:21.563+0000 7fc507e00640 1 -- 172.21.15.5:0/4030761711 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc50819f6f0 con 0x7fc508102340 2024-09-10T09:52:21.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:21.563+0000 7fc4ff400640 1 -- 172.21.15.5:0/4030761711 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc4f0031c90 con 0x7fc508102340 2024-09-10T09:52:21.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:21.563+0000 7fc4ff400640 1 -- 172.21.15.5:0/4030761711 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4f0035030 con 0x7fc508102340 2024-09-10T09:52:21.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:21.563+0000 7fc507e00640 1 -- 172.21.15.5:0/4030761711 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5081a7030 con 0x7fc508102340 2024-09-10T09:52:21.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:21.566+0000 7fc51049b740 1 Processor -- start 2024-09-10T09:52:21.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:21.566+0000 7fc51049b740 1 -- start start 2024-09-10T09:52:21.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:21.566+0000 7fc4ff400640 1 -- 172.21.15.5:0/4030761711 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7fc4f0042070 con 0x7fc508102340 2024-09-10T09:52:21.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:21.567+0000 7fc51049b740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563550f10520 0x563550f10900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:52:21.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:21.567+0000 7fc51049b740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563550f10e40 0x563550f1b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:52:21.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:21.567+0000 7fc51049b740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563550f1b700 0x563550f1daf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:52:21.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:21.567+0000 7fc51049b740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x563550e76650 con 0x563550f10520 2024-09-10T09:52:21.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:21.567+0000 7fc51049b740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x563550e1df10 con 0x563550f1b700 2024-09-10T09:52:21.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:21.567+0000 7fc51049b740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x563550e72610 con 0x563550f10e40 2024-09-10T09:52:21.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:21.567+0000 7fc507400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563550f10520 0x563550f10900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:52:21.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:21.567+0000 7fc506a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563550f10e40 0x563550f1b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:52:21.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:21.567+0000 7fc507400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563550f10520 0x563550f10900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44000/0 (socket says 172.21.15.5:44000) 2024-09-10T09:52:21.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:21.567+0000 7fc506a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563550f10e40 0x563550f1b1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:39212/0 (socket says 172.21.15.5:39212) 2024-09-10T09:52:21.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:21.567+0000 7fc4ffe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563550f1b700 0x563550f1daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:52:21.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:21.567+0000 7fc507400640 1 -- 172.21.15.5:0/1452361138 learned_addr learned my addr 172.21.15.5:0/1452361138 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:52:21.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:21.568+0000 7fc506a00640 1 -- 172.21.15.5:0/1452361138 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563550f1b700 msgr2=0x563550f1daf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:52:21.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:21.568+0000 7fc506a00640 1 --2- 172.21.15.5:0/1452361138 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563550f1b700 0x563550f1daf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:21.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:21.568+0000 7fc506a00640 1 -- 172.21.15.5:0/1452361138 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563550f10520 msgr2=0x563550f10900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:52:21.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:21.568+0000 7fc506a00640 1 --2- 172.21.15.5:0/1452361138 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563550f10520 0x563550f10900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:21.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:21.568+0000 7fc506a00640 1 -- 172.21.15.5:0/1452361138 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563550b85580 con 0x563550f10e40 2024-09-10T09:52:21.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:21.568+0000 7fc507400640 1 --2- 172.21.15.5:0/1452361138 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563550f10520 0x563550f10900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:52:21.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:21.568+0000 7fc506a00640 1 --2- 172.21.15.5:0/1452361138 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563550f10e40 0x563550f1b1c0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fc4f8009c70 tx=0x7fc4f8011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d4205e26618c54b5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:52:21.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:21.568+0000 7fc4f7e00640 1 -- 172.21.15.5:0/1452361138 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4f8005220 con 0x563550f10e40 2024-09-10T09:52:21.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:21.568+0000 7fc4f7e00640 1 -- 172.21.15.5:0/1452361138 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc4f8004c80 con 0x563550f10e40 2024-09-10T09:52:21.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:21.568+0000 7fc4f7e00640 1 -- 172.21.15.5:0/1452361138 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4f8019890 con 0x563550f10e40 2024-09-10T09:52:21.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:21.568+0000 7fc4ff400640 1 --2- 172.21.15.5:0/4030761711 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fc4ec0a2b80 0x7fc4ec0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:52:21.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:21.569+0000 7fc4ff400640 1 -- 172.21.15.5:0/4030761711 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fc4f00efd90 con 0x7fc508102340 2024-09-10T09:52:21.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:21.569+0000 7fc507400640 1 --2- 172.21.15.5:0/4030761711 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fc4ec0a2b80 0x7fc4ec0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:52:21.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:21.569+0000 7fc51049b740 1 -- 172.21.15.5:0/1452361138 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563550f10e40 msgr2=0x563550f1b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:52:21.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:21.569+0000 7fc51049b740 1 --2- 172.21.15.5:0/1452361138 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563550f10e40 0x563550f1b1c0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fc4f8009c70 tx=0x7fc4f8011810 comp rx=0 tx=0).stop 2024-09-10T09:52:21.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:21.569+0000 7fc51049b740 1 -- 172.21.15.5:0/1452361138 shutdown_connections 2024-09-10T09:52:21.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:21.569+0000 7fc51049b740 1 --2- 172.21.15.5:0/1452361138 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563550f1b700 0x563550f1daf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:21.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:21.569+0000 7fc51049b740 1 --2- 172.21.15.5:0/1452361138 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563550f10e40 0x563550f1b1c0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:21.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:21.569+0000 7fc51049b740 1 --2- 172.21.15.5:0/1452361138 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563550f10520 0x563550f10900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:21.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:21.569+0000 7fc51049b740 1 -- 172.21.15.5:0/1452361138 >> 172.21.15.5:0/1452361138 conn(0x563550f0cab0 msgr2=0x563550f0ce90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:52:21.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:21.569+0000 7fc51049b740 1 -- 172.21.15.5:0/1452361138 shutdown_connections 2024-09-10T09:52:21.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:21.569+0000 7fc51049b740 1 -- 172.21.15.5:0/1452361138 wait complete. 2024-09-10T09:52:21.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:21.570+0000 7fc507400640 1 --2- 172.21.15.5:0/4030761711 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fc4ec0a2b80 0x7fc4ec0a5040 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fc500011750 tx=0x7fc500009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:52:21.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:21.570+0000 7fc51049b740 1 -- start start 2024-09-10T09:52:21.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:21.570+0000 7fc51049b740 1 Processor -- start 2024-09-10T09:52:21.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:21.570+0000 7fc51049b740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563550f10520 0x563550f185a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:52:21.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:21.570+0000 7fc51049b740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563550f10e40 0x563550f18ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:52:21.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:21.570+0000 7fc51049b740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563550f1b700 0x563550f19020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:52:21.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:21.571+0000 7fc51049b740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x563550f1e2d0 con 0x563550f1b700 2024-09-10T09:52:21.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:21.571+0000 7fc51049b740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x563550e31350 con 0x563550f10e40 2024-09-10T09:52:21.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:21.571+0000 7fc51049b740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x563550f1e450 con 0x563550f10520 2024-09-10T09:52:21.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:21.571+0000 7fc51049b740 10 client.0 _ll_register_callbacks cb 0x563550e54290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:52:21.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:21.571+0000 7fc507400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563550f1b700 0x563550f19020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:52:21.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:21.571+0000 7fc507400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563550f1b700 0x563550f19020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44008/0 (socket says 172.21.15.5:44008) 2024-09-10T09:52:21.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:21.571+0000 7fc507400640 1 -- 172.21.15.5:0/714677966 learned_addr learned my addr 172.21.15.5:0/714677966 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:52:21.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:21.571+0000 7fc506a00640 1 --2- 172.21.15.5:0/714677966 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563550f10520 0x563550f185a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:52:21.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:21.571+0000 7fc4ffe00640 1 --2- 172.21.15.5:0/714677966 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563550f10e40 0x563550f18ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:52:21.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:21.571+0000 7fc507400640 1 -- 172.21.15.5:0/714677966 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563550f10520 msgr2=0x563550f185a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:52:21.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:21.571+0000 7fc507400640 1 --2- 172.21.15.5:0/714677966 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563550f10520 0x563550f185a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:21.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:21.571+0000 7fc507400640 1 -- 172.21.15.5:0/714677966 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563550f10e40 msgr2=0x563550f18ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:52:21.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:21.571+0000 7fc507400640 1 --2- 172.21.15.5:0/714677966 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563550f10e40 0x563550f18ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:21.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:21.571+0000 7fc507400640 1 -- 172.21.15.5:0/714677966 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563550c4f960 con 0x563550f1b700 2024-09-10T09:52:21.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:21.572+0000 7fc507400640 1 --2- 172.21.15.5:0/714677966 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563550f1b700 0x563550f19020 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fc500010a80 tx=0x7fc500004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c1e36b016b9859c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:52:21.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:21.572+0000 7fc4f4c00640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-10T09:52:21.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:21.572+0000 7fc4f4c00640 1 -- 172.21.15.5:0/714677966 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc50001fbc0 con 0x563550f1b700 2024-09-10T09:52:21.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:21.572+0000 7fc4f4c00640 1 -- 172.21.15.5:0/714677966 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fc50001fd60 con 0x563550f1b700 2024-09-10T09:52:21.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:21.572+0000 7fc51049b740 1 -- 172.21.15.5:0/714677966 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc4f8004c80 con 0x563550f1b700 2024-09-10T09:52:21.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:21.572+0000 7fc4f4c00640 1 -- 172.21.15.5:0/714677966 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fc500004b20 con 0x563550f1b700 2024-09-10T09:52:21.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:21.572+0000 7fc51049b740 10 client.18267 fetch_fsmap learned FSMap version 36 2024-09-10T09:52:21.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:21.572+0000 7fc51049b740 1 -- 172.21.15.5:0/714677966 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x563550f29f30 con 0x563550f1b700 2024-09-10T09:52:21.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:21.572+0000 7fc4f4c00640 1 -- 172.21.15.5:0/714677966 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc4f8004c80 con 0x563550f1b700 2024-09-10T09:52:21.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:21.573+0000 7fc4f4c00640 1 -- 172.21.15.5:0/714677966 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 36) ==== 8370+0+0 (secure 0 0 0) 0x7fc500026070 con 0x563550f1b700 2024-09-10T09:52:21.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:21.573+0000 7fc51049b740 10 client.18267 fetch_fsmap finished waiting for FSMap version 36 2024-09-10T09:52:21.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:21.573+0000 7fc51049b740 10 client.18267 resolve_mds: validated gid 24542 aka daemon mds.b 2024-09-10T09:52:21.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:21.573+0000 7fc51049b740 20 client.18267 populate_metadata read hostname 'smithi005' 2024-09-10T09:52:21.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:21.573+0000 7fc51049b740 1 --2- 172.21.15.5:0/714677966 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x563550f2b3f0 0x563550f2d890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:52:21.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:21.573+0000 7fc51049b740 4 client.18267 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:52:21.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:21.573+0000 7fc51049b740 1 -- 172.21.15.5:0/714677966 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563550e83bc0 con 0x563550f2b3f0 2024-09-10T09:52:21.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:21.573+0000 7fc506a00640 1 --2- 172.21.15.5:0/714677966 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x563550f2b3f0 0x563550f2d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:52:21.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:21.574+0000 7fc506a00640 1 --2- 172.21.15.5:0/714677966 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x563550f2b3f0 0x563550f2d890 crc :-1 s=READY pgs=215 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aef20da76ef7698f server_cookie=de636c24291764a8 in_seq=0 out_seq=0 2024-09-10T09:52:21.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:21.574+0000 7fc4f4c00640 10 client.18267 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T09:52:21.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:21.574+0000 7fc4f4c00640 1 -- 172.21.15.5:0/714677966 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563550e83bc0 con 0x563550f2b3f0 2024-09-10T09:52:21.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:21.574+0000 7fc4f4c00640 10 client.18267 handle_command_reply: tid=0 2024-09-10T09:52:21.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:21.575+0000 7fc51049b740 1 client.18267 shutdown 2024-09-10T09:52:21.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:21.575+0000 7fc51049b740 1 -- 172.21.15.5:0/714677966 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563550f1b700 msgr2=0x563550f19020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:52:21.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:21.575+0000 7fc51049b740 1 --2- 172.21.15.5:0/714677966 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563550f1b700 0x563550f19020 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fc500010a80 tx=0x7fc500004ab0 comp rx=0 tx=0).stop 2024-09-10T09:52:21.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:21.575+0000 7fc51049b740 1 -- 172.21.15.5:0/714677966 shutdown_connections 2024-09-10T09:52:21.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:21.575+0000 7fc51049b740 1 --2- 172.21.15.5:0/714677966 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x563550f2b3f0 0x563550f2d890 crc :-1 s=READY pgs=215 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:21.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:21.575+0000 7fc51049b740 1 --2- 172.21.15.5:0/714677966 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563550f1b700 0x563550f19020 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:21.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:21.575+0000 7fc51049b740 1 --2- 172.21.15.5:0/714677966 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563550f10e40 0x563550f18ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:21.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:21.575+0000 7fc4f4c00640 0 client.18267 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T09:52:21.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:21.575+0000 7fc51049b740 1 --2- 172.21.15.5:0/714677966 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563550f10520 0x563550f185a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:21.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:21.575+0000 7fc51049b740 1 -- 172.21.15.5:0/714677966 >> 172.21.15.5:0/714677966 conn(0x563550f0cab0 msgr2=0x563550f0bcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:52:21.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:21.575+0000 7fc51049b740 1 -- 172.21.15.5:0/714677966 shutdown_connections 2024-09-10T09:52:21.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:21.575+0000 7fc51049b740 1 -- 172.21.15.5:0/714677966 wait complete. 2024-09-10T09:52:21.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:21.575+0000 7fc51049b740 20 client.18267 trim_cache size 0 max 16384 2024-09-10T09:52:21.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:21.593+0000 7fc51049b740 1 Processor -- start 2024-09-10T09:52:21.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:21.594+0000 7fc51049b740 1 -- start start 2024-09-10T09:52:21.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:21.594+0000 7fc51049b740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563550f10520 0x563550f174e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:52:21.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:21.594+0000 7fc51049b740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563550f10e40 0x563550f331b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:52:21.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:21.594+0000 7fc51049b740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563550f1b700 0x563550f336f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:52:21.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:21.594+0000 7fc51049b740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc5081b0e40 con 0x563550f10520 2024-09-10T09:52:21.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:21.594+0000 7fc51049b740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fc5081b2980 con 0x563550f1b700 2024-09-10T09:52:21.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:21.594+0000 7fc51049b740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fc5081b1510 con 0x563550f10e40 2024-09-10T09:52:21.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:21.594+0000 7fc506a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563550f10520 0x563550f174e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:52:21.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:21.594+0000 7fc506a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563550f10520 0x563550f174e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44024/0 (socket says 172.21.15.5:44024) 2024-09-10T09:52:21.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:21.594+0000 7fc506a00640 1 -- 172.21.15.5:0/2119279655 learned_addr learned my addr 172.21.15.5:0/2119279655 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:52:21.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:21.594+0000 7fc4ffe00640 1 --2- 172.21.15.5:0/2119279655 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563550f10e40 0x563550f331b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:52:21.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:21.594+0000 7fc507400640 1 --2- 172.21.15.5:0/2119279655 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563550f1b700 0x563550f336f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:52:21.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:21.595+0000 7fc506a00640 1 -- 172.21.15.5:0/2119279655 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563550f10e40 msgr2=0x563550f331b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:52:21.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:21.595+0000 7fc506a00640 1 --2- 172.21.15.5:0/2119279655 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563550f10e40 0x563550f331b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:21.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:21.595+0000 7fc506a00640 1 -- 172.21.15.5:0/2119279655 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563550f1b700 msgr2=0x563550f336f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:52:21.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:21.595+0000 7fc506a00640 1 --2- 172.21.15.5:0/2119279655 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563550f1b700 0x563550f336f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:21.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:21.595+0000 7fc506a00640 1 -- 172.21.15.5:0/2119279655 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563550f08480 con 0x563550f10520 2024-09-10T09:52:21.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:21.595+0000 7fc506a00640 1 --2- 172.21.15.5:0/2119279655 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563550f10520 0x563550f174e0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x563550f0d910 tx=0x7fc4f80095e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7120a51c6820cbe server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:52:21.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:21.595+0000 7fc4f7400640 1 -- 172.21.15.5:0/2119279655 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4f8005490 con 0x563550f10520 2024-09-10T09:52:21.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:21.595+0000 7fc4f7400640 1 -- 172.21.15.5:0/2119279655 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc4f8004b60 con 0x563550f10520 2024-09-10T09:52:21.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:21.595+0000 7fc4f7400640 1 -- 172.21.15.5:0/2119279655 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4f80197d0 con 0x563550f10520 2024-09-10T09:52:21.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:21.596+0000 7fc51049b740 1 -- 172.21.15.5:0/2119279655 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563550f10520 msgr2=0x563550f174e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:52:21.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:21.596+0000 7fc51049b740 1 --2- 172.21.15.5:0/2119279655 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563550f10520 0x563550f174e0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x563550f0d910 tx=0x7fc4f80095e0 comp rx=0 tx=0).stop 2024-09-10T09:52:21.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:21.596+0000 7fc51049b740 1 -- 172.21.15.5:0/2119279655 shutdown_connections 2024-09-10T09:52:21.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:21.596+0000 7fc51049b740 1 --2- 172.21.15.5:0/2119279655 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563550f1b700 0x563550f336f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:21.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:21.596+0000 7fc51049b740 1 --2- 172.21.15.5:0/2119279655 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563550f10e40 0x563550f331b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:21.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:21.596+0000 7fc51049b740 1 --2- 172.21.15.5:0/2119279655 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563550f10520 0x563550f174e0 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:21.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:21.596+0000 7fc51049b740 1 -- 172.21.15.5:0/2119279655 >> 172.21.15.5:0/2119279655 conn(0x563550f0cab0 msgr2=0x563550f0bcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:52:21.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:21.596+0000 7fc51049b740 1 -- 172.21.15.5:0/2119279655 shutdown_connections 2024-09-10T09:52:21.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:21.596+0000 7fc51049b740 1 -- 172.21.15.5:0/2119279655 wait complete. 2024-09-10T09:52:21.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:21.597+0000 7fc51049b740 1 -- start start 2024-09-10T09:52:21.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:21.597+0000 7fc51049b740 1 Processor -- start 2024-09-10T09:52:21.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:21.597+0000 7fc51049b740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563550f10520 0x563550f1f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:52:21.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:21.597+0000 7fc51049b740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563550f10e40 0x563550f1fb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:52:21.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:21.597+0000 7fc51049b740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563550f1b700 0x563550f20050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:52:21.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:21.597+0000 7fc51049b740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc5081b22b0 con 0x563550f10e40 2024-09-10T09:52:21.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:21.597+0000 7fc51049b740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fc5081b1be0 con 0x563550f10520 2024-09-10T09:52:21.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:21.597+0000 7fc51049b740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fc5081aee20 con 0x563550f1b700 2024-09-10T09:52:21.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:21.597+0000 7fc51049b740 10 client.0 _ll_register_callbacks cb 0x7fc500009b70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:52:21.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:21.598+0000 7fc4ffe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563550f10e40 0x563550f1fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:52:21.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:21.598+0000 7fc506a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563550f10520 0x563550f1f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:52:21.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:21.598+0000 7fc506a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563550f10520 0x563550f1f5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:39256/0 (socket says 172.21.15.5:39256) 2024-09-10T09:52:21.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:21.598+0000 7fc506a00640 1 -- 172.21.15.5:0/8637029 learned_addr learned my addr 172.21.15.5:0/8637029 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:52:21.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:21.598+0000 7fc507400640 1 --2- 172.21.15.5:0/8637029 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563550f1b700 0x563550f20050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:52:21.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:21.598+0000 7fc4ffe00640 1 -- 172.21.15.5:0/8637029 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563550f1b700 msgr2=0x563550f20050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:52:21.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:21.598+0000 7fc4ffe00640 1 --2- 172.21.15.5:0/8637029 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563550f1b700 0x563550f20050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:21.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:21.598+0000 7fc4ffe00640 1 -- 172.21.15.5:0/8637029 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563550f10520 msgr2=0x563550f1f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:52:21.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:21.598+0000 7fc4ffe00640 1 --2- 172.21.15.5:0/8637029 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563550f10520 0x563550f1f5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:21.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:21.598+0000 7fc4ffe00640 1 -- 172.21.15.5:0/8637029 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563550f95320 con 0x563550f10e40 2024-09-10T09:52:21.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:21.598+0000 7fc506a00640 1 --2- 172.21.15.5:0/8637029 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563550f10520 0x563550f1f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:52:21.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:21.598+0000 7fc507400640 1 --2- 172.21.15.5:0/8637029 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563550f1b700 0x563550f20050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:52:21.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:21.598+0000 7fc4ffe00640 1 --2- 172.21.15.5:0/8637029 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563550f10e40 0x563550f1fb10 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x563550f0de30 tx=0x7fc4f003b720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ea89982e34c3308 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:52:21.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:21.598+0000 7fc4f4c00640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-10T09:52:21.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:21.598+0000 7fc4f4c00640 1 -- 172.21.15.5:0/8637029 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4f0049040 con 0x563550f10e40 2024-09-10T09:52:21.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:21.598+0000 7fc4f4c00640 1 -- 172.21.15.5:0/8637029 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fc4f00b8810 con 0x563550f10e40 2024-09-10T09:52:21.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:21.598+0000 7fc51049b740 1 -- 172.21.15.5:0/8637029 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc4f8004b60 con 0x563550f10e40 2024-09-10T09:52:21.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:21.599+0000 7fc4f4c00640 1 -- 172.21.15.5:0/8637029 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fc4f00bbce0 con 0x563550f10e40 2024-09-10T09:52:21.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:21.599+0000 7fc51049b740 10 client.18273 fetch_fsmap learned FSMap version 36 2024-09-10T09:52:21.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:21.599+0000 7fc51049b740 1 -- 172.21.15.5:0/8637029 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x563550f966a0 con 0x563550f10e40 2024-09-10T09:52:21.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:21.599+0000 7fc4f4c00640 1 -- 172.21.15.5:0/8637029 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc4f8004b60 con 0x563550f10e40 2024-09-10T09:52:21.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:21.599+0000 7fc4f4c00640 1 -- 172.21.15.5:0/8637029 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 36) ==== 8370+0+0 (secure 0 0 0) 0x7fc4f00dc0c0 con 0x563550f10e40 2024-09-10T09:52:21.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:21.599+0000 7fc51049b740 10 client.18273 fetch_fsmap finished waiting for FSMap version 36 2024-09-10T09:52:21.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:21.599+0000 7fc51049b740 10 client.18273 resolve_mds: validated gid 24542 aka daemon mds.b 2024-09-10T09:52:21.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:21.599+0000 7fc51049b740 20 client.18273 populate_metadata read hostname 'smithi005' 2024-09-10T09:52:21.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:21.599+0000 7fc51049b740 1 --2- 172.21.15.5:0/8637029 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x563550f96e50 0x563550f99270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:52:21.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:21.599+0000 7fc51049b740 4 client.18273 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "dbf77f9f", "release": true, "await": true}] 2024-09-10T09:52:21.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:21.599+0000 7fc51049b740 1 -- 172.21.15.5:0/8637029 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "quiesce db", "set_id": "dbf77f9f", "release": true, "await": true}) -- 0x563550e7e0d0 con 0x563550f96e50 2024-09-10T09:52:21.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:21.600+0000 7fc506a00640 1 --2- 172.21.15.5:0/8637029 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x563550f96e50 0x563550f99270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:52:21.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:21.600+0000 7fc506a00640 1 --2- 172.21.15.5:0/8637029 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x563550f96e50 0x563550f99270 crc :-1 s=READY pgs=216 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=828357ed6ff802ec server_cookie=18a6c0c074d69415 in_seq=0 out_seq=0 2024-09-10T09:52:21.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:21.600+0000 7fc4f4c00640 10 client.18273 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T09:52:21.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:21.610+0000 7fc4f4c00640 1 -- 172.21.15.5:0/8637029 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x563550e7e0d0 con 0x563550f96e50 2024-09-10T09:52:21.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:21.610+0000 7fc4f4c00640 10 client.18273 handle_command_reply: tid=0 2024-09-10T09:52:21.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:21.610+0000 7fc51049b740 1 client.18273 shutdown 2024-09-10T09:52:21.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:21.610+0000 7fc51049b740 1 -- 172.21.15.5:0/8637029 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563550f10e40 msgr2=0x563550f1fb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:52:21.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:21.610+0000 7fc51049b740 1 --2- 172.21.15.5:0/8637029 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563550f10e40 0x563550f1fb10 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x563550f0de30 tx=0x7fc4f003b720 comp rx=0 tx=0).stop 2024-09-10T09:52:21.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:21.610+0000 7fc51049b740 1 -- 172.21.15.5:0/8637029 shutdown_connections 2024-09-10T09:52:21.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:21.610+0000 7fc51049b740 1 --2- 172.21.15.5:0/8637029 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x563550f96e50 0x563550f99270 crc :-1 s=READY pgs=216 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:21.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:21.610+0000 7fc51049b740 1 --2- 172.21.15.5:0/8637029 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563550f1b700 0x563550f20050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:21.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:21.610+0000 7fc51049b740 1 --2- 172.21.15.5:0/8637029 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563550f10e40 0x563550f1fb10 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:21.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:21.610+0000 7fc4f4c00640 0 client.18273 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T09:52:21.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:21.610+0000 7fc51049b740 1 --2- 172.21.15.5:0/8637029 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563550f10520 0x563550f1f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:21.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:21.610+0000 7fc51049b740 1 -- 172.21.15.5:0/8637029 >> 172.21.15.5:0/8637029 conn(0x563550f0cab0 msgr2=0x563550f18560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:52:21.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:21.610+0000 7fc51049b740 1 -- 172.21.15.5:0/8637029 shutdown_connections 2024-09-10T09:52:21.681 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-10T09:52:21.681 INFO:teuthology.orchestra.run.smithi005.stdout: "epoch": 36, 2024-09-10T09:52:21.681 INFO:teuthology.orchestra.run.smithi005.stdout: "leader": 24542, 2024-09-10T09:52:21.681 INFO:teuthology.orchestra.run.smithi005.stdout: "set_version": 52, 2024-09-10T09:52:21.681 INFO:teuthology.orchestra.run.smithi005.stdout: "sets": { 2024-09-10T09:52:21.681 INFO:teuthology.orchestra.run.smithi005.stdout: "dbf77f9f": { 2024-09-10T09:52:21.681 INFO:teuthology.orchestra.run.smithi005.stdout: "version": 52, 2024-09-10T09:52:21.681 INFO:teuthology.orchestra.run.smithi005.stdout: "age_ref": 0.0, 2024-09-10T09:52:21.681 INFO:teuthology.orchestra.run.smithi005.stdout: "state": { 2024-09-10T09:52:21.681 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "RELEASED", 2024-09-10T09:52:21.681 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0 2024-09-10T09:52:21.681 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T09:52:21.681 INFO:teuthology.orchestra.run.smithi005.stdout: "timeout": 90.0, 2024-09-10T09:52:21.682 INFO:teuthology.orchestra.run.smithi005.stdout: "expiration": 141.5, 2024-09-10T09:52:21.682 INFO:teuthology.orchestra.run.smithi005.stdout: "members": { 2024-09-10T09:52:21.682 INFO:teuthology.orchestra.run.smithi005.stdout: "file:/": { 2024-09-10T09:52:21.682 INFO:teuthology.orchestra.run.smithi005.stdout: "excluded": false, 2024-09-10T09:52:21.682 INFO:teuthology.orchestra.run.smithi005.stdout: "state": { 2024-09-10T09:52:21.682 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "RELEASED", 2024-09-10T09:52:21.682 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0 2024-09-10T09:52:21.682 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T09:52:21.682 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T09:52:21.682 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T09:52:21.682 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T09:52:21.682 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T09:52:21.682 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-10T09:52:21.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:21.610+0000 7fc51049b740 1 -- 172.21.15.5:0/8637029 wait complete. 2024-09-10T09:52:21.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:21.610+0000 7fc51049b740 20 client.18273 trim_cache size 0 max 16384 2024-09-10T09:52:21.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:21.611+0000 7fc507e00640 1 -- 172.21.15.5:0/4030761711 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fc4ec0a2b80 msgr2=0x7fc4ec0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:52:21.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:21.611+0000 7fc507e00640 1 --2- 172.21.15.5:0/4030761711 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fc4ec0a2b80 0x7fc4ec0a5040 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fc500011750 tx=0x7fc500009500 comp rx=0 tx=0).stop 2024-09-10T09:52:21.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:21.611+0000 7fc507e00640 1 -- 172.21.15.5:0/4030761711 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc508102340 msgr2=0x7fc50819a9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:52:21.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:21.611+0000 7fc507e00640 1 --2- 172.21.15.5:0/4030761711 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc508102340 0x7fc50819a9f0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fc4f002a940 tx=0x7fc4f0003b20 comp rx=0 tx=0).stop 2024-09-10T09:52:21.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:21.611+0000 7fc507e00640 1 -- 172.21.15.5:0/4030761711 shutdown_connections 2024-09-10T09:52:21.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:21.611+0000 7fc507e00640 1 --2- 172.21.15.5:0/4030761711 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fc4ec0a2b80 0x7fc4ec0a5040 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:21.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:21.611+0000 7fc507e00640 1 --2- 172.21.15.5:0/4030761711 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc5081079a0 0x7fc50819ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:21.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:21.611+0000 7fc507e00640 1 --2- 172.21.15.5:0/4030761711 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc508102340 0x7fc50819a9f0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:21.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:21.611+0000 7fc507e00640 1 --2- 172.21.15.5:0/4030761711 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc5081019a0 0x7fc50819a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:21.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:21.611+0000 7fc507e00640 1 -- 172.21.15.5:0/4030761711 >> 172.21.15.5:0/4030761711 conn(0x7fc5080fd700 msgr2=0x7fc50810cc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:52:21.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:21.611+0000 7fc507e00640 1 -- 172.21.15.5:0/4030761711 shutdown_connections 2024-09-10T09:52:21.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:21.611+0000 7fc507e00640 1 -- 172.21.15.5:0/4030761711 wait complete. 2024-09-10T09:52:21.810 INFO:tasks.workunit.client.0.smithi005.stdout: 0 cleanup 603 sec 2024-09-10T09:52:21.810 INFO:tasks.workunit.client.0.smithi005.stdout: 2024-09-10T09:52:21.810 INFO:tasks.workunit.client.0.smithi005.stdout: Operation Count AvgLat MaxLat 2024-09-10T09:52:21.810 INFO:tasks.workunit.client.0.smithi005.stdout: ---------------------------------------- 2024-09-10T09:52:21.810 INFO:tasks.workunit.client.0.smithi005.stdout: NTCreateX 116583 2.200 27387.168 2024-09-10T09:52:21.810 INFO:tasks.workunit.client.0.smithi005.stdout: Close 85653 0.005 0.547 2024-09-10T09:52:21.810 INFO:tasks.workunit.client.0.smithi005.stdout: Rename 4940 13.474 22753.082 2024-09-10T09:52:21.810 INFO:tasks.workunit.client.0.smithi005.stdout: Unlink 23539 4.257 19345.011 2024-09-10T09:52:21.811 INFO:tasks.workunit.client.0.smithi005.stdout: Deltree 2 67.855 100.196 2024-09-10T09:52:21.811 INFO:tasks.workunit.client.0.smithi005.stdout: Mkdir 1 0.013 0.013 2024-09-10T09:52:21.811 INFO:tasks.workunit.client.0.smithi005.stdout: Qpathinfo 105709 0.929 2037.074 2024-09-10T09:52:21.811 INFO:tasks.workunit.client.0.smithi005.stdout: Qfileinfo 18500 0.002 1.977 2024-09-10T09:52:21.811 INFO:tasks.workunit.client.0.smithi005.stdout: Qfsinfo 19381 0.258 35.126 2024-09-10T09:52:21.811 INFO:tasks.workunit.client.0.smithi005.stdout: Sfileinfo 9504 0.048 12.304 2024-09-10T09:52:21.811 INFO:tasks.workunit.client.0.smithi005.stdout: Find 40874 0.291 1816.765 2024-09-10T09:52:21.811 INFO:tasks.workunit.client.0.smithi005.stdout: WriteX 57960 0.364 19346.637 2024-09-10T09:52:21.811 INFO:tasks.workunit.client.0.smithi005.stdout: ReadX 182908 0.038 37.419 2024-09-10T09:52:21.811 INFO:tasks.workunit.client.0.smithi005.stdout: LockX 380 0.851 51.550 2024-09-10T09:52:21.812 INFO:tasks.workunit.client.0.smithi005.stdout: UnlockX 380 0.650 2.531 2024-09-10T09:52:21.812 INFO:tasks.workunit.client.0.smithi005.stdout: Flush 8176 4.415 1263.875 2024-09-10T09:52:21.812 INFO:tasks.workunit.client.0.smithi005.stdout: 2024-09-10T09:52:21.812 INFO:tasks.workunit.client.0.smithi005.stdout:Throughput 6.09733 MB/sec 1 clients 1 procs max_latency=27387.181 ms 2024-09-10T09:52:21.812 INFO:tasks.workunit.client.0.smithi005.stderr:+ dbench 10 2024-09-10T09:52:21.813 INFO:tasks.workunit.client.0.smithi005.stdout:dbench version 4.00 - Copyright Andrew Tridgell 1999-2004 2024-09-10T09:52:21.813 INFO:tasks.workunit.client.0.smithi005.stdout: 2024-09-10T09:52:21.813 INFO:tasks.workunit.client.0.smithi005.stdout:Running for 600 seconds with load '/usr/share/dbench/client.txt' and minimum warmup 120 secs 2024-09-10T09:52:21.814 INFO:tasks.workunit.client.0.smithi005.stdout:8 of 10 processes prepared for launch 0 sec 2024-09-10T09:52:21.914 INFO:tasks.workunit.client.0.smithi005.stdout:10 of 10 processes prepared for launch 0 sec 2024-09-10T09:52:21.914 INFO:tasks.workunit.client.0.smithi005.stdout:releasing clients 2024-09-10T09:52:22.602 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:52:22.641 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'dbf77f9f', total seconds elapsed: 27.9 2024-09-10T09:52:22.915 INFO:tasks.workunit.client.0.smithi005.stdout: 10 432 152.30 MB/sec warmup 1 sec latency 43.504 ms 2024-09-10T09:52:23.225 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:52:23 smithi005 ceph-mon[30350]: pgmap v814: 129 pgs: 129 active+clean; 287 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.1 KiB/s wr, 13 op/s 2024-09-10T09:52:23.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:23.276+0000 7f839e600640 1 -- 172.21.15.5:0/3544883180 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8398103fd0 msgr2=0x7f83981043b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:52:23.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:23.276+0000 7f839e600640 1 --2- 172.21.15.5:0/3544883180 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8398103fd0 0x7f83981043b0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f8388009860 tx=0x7f838802a340 comp rx=0 tx=0).stop 2024-09-10T09:52:23.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:23.276+0000 7f839e600640 1 -- 172.21.15.5:0/3544883180 shutdown_connections 2024-09-10T09:52:23.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:23.276+0000 7f839e600640 1 --2- 172.21.15.5:0/3544883180 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8398109100 0x7f8398110c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:23.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:23.276+0000 7f839e600640 1 --2- 172.21.15.5:0/3544883180 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8398104980 0x7f83981089d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:23.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:23.276+0000 7f839e600640 1 --2- 172.21.15.5:0/3544883180 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8398103fd0 0x7f83981043b0 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:23.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:23.276+0000 7f839e600640 1 -- 172.21.15.5:0/3544883180 >> 172.21.15.5:0/3544883180 conn(0x7f83980ffa30 msgr2=0x7f8398101e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:52:23.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:23.277+0000 7f839e600640 1 -- 172.21.15.5:0/3544883180 shutdown_connections 2024-09-10T09:52:23.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:23.277+0000 7f839e600640 1 -- 172.21.15.5:0/3544883180 wait complete. 2024-09-10T09:52:23.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:23.279+0000 7f839e600640 1 Processor -- start 2024-09-10T09:52:23.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:23.280+0000 7f839e600640 1 -- start start 2024-09-10T09:52:23.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:23.280+0000 7f839e600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8398103fd0 0x7f83981a36e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:52:23.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:23.280+0000 7f839e600640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8398104980 0x7f83981a3c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:52:23.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:23.280+0000 7f839e600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8398109100 0x7f839810f680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:52:23.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:23.280+0000 7f839e600640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f8398077950 con 0x7f8398109100 2024-09-10T09:52:23.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:23.280+0000 7f839e600640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f83980777d0 con 0x7f8398103fd0 2024-09-10T09:52:23.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:23.280+0000 7f839e600640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f8398077ad0 con 0x7f8398104980 2024-09-10T09:52:23.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:23.282+0000 7f8397400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8398103fd0 0x7f83981a36e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:52:23.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:23.282+0000 7f8397400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8398103fd0 0x7f83981a36e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:39274/0 (socket says 172.21.15.5:39274) 2024-09-10T09:52:23.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:23.282+0000 7f8397400640 1 -- 172.21.15.5:0/3842233630 learned_addr learned my addr 172.21.15.5:0/3842233630 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:52:23.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:23.283+0000 7f8396a00640 1 --2- 172.21.15.5:0/3842233630 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8398104980 0x7f83981a3c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:52:23.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:23.283+0000 7f8397e00640 1 --2- 172.21.15.5:0/3842233630 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8398109100 0x7f839810f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:52:23.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:23.284+0000 7f8397400640 1 -- 172.21.15.5:0/3842233630 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8398104980 msgr2=0x7f83981a3c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:52:23.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:23.284+0000 7f8397400640 1 --2- 172.21.15.5:0/3842233630 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8398104980 0x7f83981a3c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:23.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:23.284+0000 7f8397400640 1 -- 172.21.15.5:0/3842233630 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8398109100 msgr2=0x7f839810f680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:52:23.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:23.284+0000 7f8397400640 1 --2- 172.21.15.5:0/3842233630 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8398109100 0x7f839810f680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:23.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:23.284+0000 7f8397400640 1 -- 172.21.15.5:0/3842233630 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f839810fef0 con 0x7f8398103fd0 2024-09-10T09:52:23.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:23.285+0000 7f8397400640 1 --2- 172.21.15.5:0/3842233630 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8398103fd0 0x7f83981a36e0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f838802a8b0 tx=0x7f8388033d10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:52:23.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:23.286+0000 7f838fe00640 1 -- 172.21.15.5:0/3842233630 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8388037040 con 0x7f8398103fd0 2024-09-10T09:52:23.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:23.286+0000 7f838fe00640 1 -- 172.21.15.5:0/3842233630 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8388030550 con 0x7f8398103fd0 2024-09-10T09:52:23.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:23.286+0000 7f838fe00640 1 -- 172.21.15.5:0/3842233630 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8388003970 con 0x7f8398103fd0 2024-09-10T09:52:23.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:23.286+0000 7f839e600640 1 -- 172.21.15.5:0/3842233630 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8398110180 con 0x7f8398103fd0 2024-09-10T09:52:23.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:23.287+0000 7f839e600640 1 -- 172.21.15.5:0/3842233630 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83981ab440 con 0x7f8398103fd0 2024-09-10T09:52:23.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:23.294+0000 7f838fe00640 1 -- 172.21.15.5:0/3842233630 <== mon.2 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f8388042030 con 0x7f8398103fd0 2024-09-10T09:52:23.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:23.295+0000 7f83a0fe5740 1 Processor -- start 2024-09-10T09:52:23.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:23.296+0000 7f83a0fe5740 1 -- start start 2024-09-10T09:52:23.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:23.296+0000 7f83a0fe5740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561583eddb90 0x561583eddf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:52:23.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:23.296+0000 7f83a0fe5740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561583ede4b0 0x561583ee8830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:52:23.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:23.296+0000 7f83a0fe5740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561583ee8d70 0x561583eeb160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:52:23.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:23.296+0000 7f83a0fe5740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x561583e443e0 con 0x561583eddb90 2024-09-10T09:52:23.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:23.296+0000 7f83a0fe5740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x561583debd80 con 0x561583ee8d70 2024-09-10T09:52:23.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:23.296+0000 7f83a0fe5740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x561583e403a0 con 0x561583ede4b0 2024-09-10T09:52:23.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:23.297+0000 7f8396a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561583ee8d70 0x561583eeb160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:52:23.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:23.298+0000 7f8396a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561583ee8d70 0x561583eeb160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:39286/0 (socket says 172.21.15.5:39286) 2024-09-10T09:52:23.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:23.298+0000 7f8396a00640 1 -- 172.21.15.5:0/3375279264 learned_addr learned my addr 172.21.15.5:0/3375279264 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:52:23.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:23.298+0000 7f8397e00640 1 --2- 172.21.15.5:0/3375279264 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561583eddb90 0x561583eddf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:52:23.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:23.299+0000 7f8397400640 1 --2- 172.21.15.5:0/3375279264 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561583ede4b0 0x561583ee8830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:52:23.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:23.299+0000 7f8396a00640 1 -- 172.21.15.5:0/3375279264 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561583ede4b0 msgr2=0x561583ee8830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:52:23.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:23.299+0000 7f8396a00640 1 --2- 172.21.15.5:0/3375279264 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561583ede4b0 0x561583ee8830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:23.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:23.299+0000 7f8396a00640 1 -- 172.21.15.5:0/3375279264 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561583eddb90 msgr2=0x561583eddf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:52:23.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:23.299+0000 7f8396a00640 1 --2- 172.21.15.5:0/3375279264 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561583eddb90 0x561583eddf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:23.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:23.299+0000 7f8396a00640 1 -- 172.21.15.5:0/3375279264 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561583e0aae0 con 0x561583ee8d70 2024-09-10T09:52:23.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:23.300+0000 7f838fe00640 1 --2- 172.21.15.5:0/3842233630 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f837c0a2c40 0x7f837c0a5100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:52:23.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:23.300+0000 7f8397400640 1 --2- 172.21.15.5:0/3375279264 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561583ede4b0 0x561583ee8830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T09:52:23.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:23.300+0000 7f838fe00640 1 -- 172.21.15.5:0/3842233630 <== mon.2 v2:172.21.15.102:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f83880ee710 con 0x7f8398103fd0 2024-09-10T09:52:23.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:23.301+0000 7f8396a00640 1 --2- 172.21.15.5:0/3375279264 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561583ee8d70 0x561583eeb160 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f8384009c60 tx=0x7f8384011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=700feb3f89fd3a0c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:52:23.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:23.302+0000 7f838cc00640 1 -- 172.21.15.5:0/3375279264 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8384004230 con 0x561583ee8d70 2024-09-10T09:52:23.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:23.302+0000 7f8397e00640 1 --2- 172.21.15.5:0/3842233630 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f837c0a2c40 0x7f837c0a5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:52:23.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:23.302+0000 7f838cc00640 1 -- 172.21.15.5:0/3375279264 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f83840043d0 con 0x561583ee8d70 2024-09-10T09:52:23.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:23.302+0000 7f838cc00640 1 -- 172.21.15.5:0/3375279264 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8384019880 con 0x561583ee8d70 2024-09-10T09:52:23.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:23.303+0000 7f83a0fe5740 1 -- 172.21.15.5:0/3375279264 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561583ee8d70 msgr2=0x561583eeb160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:52:23.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:23.303+0000 7f83a0fe5740 1 --2- 172.21.15.5:0/3375279264 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561583ee8d70 0x561583eeb160 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f8384009c60 tx=0x7f8384011810 comp rx=0 tx=0).stop 2024-09-10T09:52:23.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:23.304+0000 7f8397e00640 1 --2- 172.21.15.5:0/3842233630 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f837c0a2c40 0x7f837c0a5100 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f8390005cc0 tx=0x7f839000b580 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:52:23.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:23.304+0000 7f83a0fe5740 1 -- 172.21.15.5:0/3375279264 shutdown_connections 2024-09-10T09:52:23.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:23.304+0000 7f83a0fe5740 1 --2- 172.21.15.5:0/3375279264 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561583ee8d70 0x561583eeb160 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:23.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:23.304+0000 7f83a0fe5740 1 --2- 172.21.15.5:0/3375279264 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561583ede4b0 0x561583ee8830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:23.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:23.304+0000 7f83a0fe5740 1 --2- 172.21.15.5:0/3375279264 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561583eddb90 0x561583eddf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:23.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:23.304+0000 7f83a0fe5740 1 -- 172.21.15.5:0/3375279264 >> 172.21.15.5:0/3375279264 conn(0x561583eda120 msgr2=0x561583eda500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:52:23.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:23.306+0000 7f83a0fe5740 1 -- 172.21.15.5:0/3375279264 shutdown_connections 2024-09-10T09:52:23.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:23.306+0000 7f83a0fe5740 1 -- 172.21.15.5:0/3375279264 wait complete. 2024-09-10T09:52:23.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:23.307+0000 7f83a0fe5740 1 -- start start 2024-09-10T09:52:23.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:23.307+0000 7f83a0fe5740 1 Processor -- start 2024-09-10T09:52:23.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:23.308+0000 7f83a0fe5740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561583eddb90 0x561583ee5bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:52:23.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:23.308+0000 7f83a0fe5740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561583ede4b0 0x561583ee60f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:52:23.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:23.308+0000 7f83a0fe5740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561583ee8d70 0x561583ee6630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:52:23.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:23.308+0000 7f83a0fe5740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x561583eeb9e0 con 0x561583ede4b0 2024-09-10T09:52:23.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:23.308+0000 7f83a0fe5740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x561583dff0f0 con 0x561583eddb90 2024-09-10T09:52:23.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:23.308+0000 7f83a0fe5740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x561583eebb60 con 0x561583ee8d70 2024-09-10T09:52:23.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:23.308+0000 7f83a0fe5740 10 client.0 _ll_register_callbacks cb 0x561583df09e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:52:23.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:23.309+0000 7f8397400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561583eddb90 0x561583ee5bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:52:23.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:23.309+0000 7f8397400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561583eddb90 0x561583ee5bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:39300/0 (socket says 172.21.15.5:39300) 2024-09-10T09:52:23.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:23.309+0000 7f8397400640 1 -- 172.21.15.5:0/3234365185 learned_addr learned my addr 172.21.15.5:0/3234365185 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:52:23.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:23.309+0000 7f8396a00640 1 --2- 172.21.15.5:0/3234365185 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561583ede4b0 0x561583ee60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:52:23.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:23.309+0000 7f8397e00640 1 --2- 172.21.15.5:0/3234365185 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561583ee8d70 0x561583ee6630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:52:23.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:23.310+0000 7f8397400640 1 -- 172.21.15.5:0/3234365185 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561583ee8d70 msgr2=0x561583ee6630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:52:23.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:23.310+0000 7f8397400640 1 --2- 172.21.15.5:0/3234365185 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561583ee8d70 0x561583ee6630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:23.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:23.310+0000 7f8397400640 1 -- 172.21.15.5:0/3234365185 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561583ede4b0 msgr2=0x561583ee60f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:52:23.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:23.310+0000 7f8397400640 1 --2- 172.21.15.5:0/3234365185 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561583ede4b0 0x561583ee60f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:23.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:23.310+0000 7f8397400640 1 -- 172.21.15.5:0/3234365185 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561583bb53c0 con 0x561583eddb90 2024-09-10T09:52:23.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:23.311+0000 7f8397400640 1 --2- 172.21.15.5:0/3234365185 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561583eddb90 0x561583ee5bb0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f8388009980 tx=0x7f83880ba810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=127c7beb8b65a930 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:52:23.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:23.311+0000 7f8381600640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-09-10T09:52:23.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:23.312+0000 7f8381600640 1 -- 172.21.15.5:0/3234365185 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8388042320 con 0x561583eddb90 2024-09-10T09:52:23.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:23.312+0000 7f8381600640 1 -- 172.21.15.5:0/3234365185 <== mon.2 v2:172.21.15.102:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f83880ee3e0 con 0x561583eddb90 2024-09-10T09:52:23.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:23.312+0000 7f83a0fe5740 1 -- 172.21.15.5:0/3234365185 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f83840043d0 con 0x561583eddb90 2024-09-10T09:52:23.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:23.313+0000 7f8381600640 1 -- 172.21.15.5:0/3234365185 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f83880bb310 con 0x561583eddb90 2024-09-10T09:52:23.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:23.313+0000 7f83a0fe5740 10 client.28316 fetch_fsmap learned FSMap version 36 2024-09-10T09:52:23.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:23.313+0000 7f83a0fe5740 1 -- 172.21.15.5:0/3234365185 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x561583eeb9e0 con 0x561583eddb90 2024-09-10T09:52:23.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:23.313+0000 7f8381600640 1 -- 172.21.15.5:0/3234365185 <== mon.2 v2:172.21.15.102:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f83840043d0 con 0x561583eddb90 2024-09-10T09:52:23.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:23.313+0000 7f8381600640 1 -- 172.21.15.5:0/3234365185 <== mon.2 v2:172.21.15.102:3300/0 5 ==== fsmap(e 36) ==== 8370+0+0 (secure 0 0 0) 0x7f838803a070 con 0x561583eddb90 2024-09-10T09:52:23.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:23.313+0000 7f83a0fe5740 10 client.28316 fetch_fsmap finished waiting for FSMap version 36 2024-09-10T09:52:23.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:23.313+0000 7f83a0fe5740 10 client.28316 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T09:52:23.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:23.313+0000 7f83a0fe5740 20 client.28316 populate_metadata read hostname 'smithi005' 2024-09-10T09:52:23.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:23.313+0000 7f83a0fe5740 1 --2- 172.21.15.5:0/3234365185 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x561583ef89a0 0x561583efadd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:52:23.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:23.313+0000 7f83a0fe5740 4 client.28316 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:52:23.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:23.313+0000 7f83a0fe5740 1 -- 172.21.15.5:0/3234365185 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561583e51950 con 0x561583ef89a0 2024-09-10T09:52:23.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:23.314+0000 7f8396a00640 1 --2- 172.21.15.5:0/3234365185 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x561583ef89a0 0x561583efadd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:52:23.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:23.315+0000 7f8381600640 10 client.28316 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T09:52:23.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:23.316+0000 7f8396a00640 1 --2- 172.21.15.5:0/3234365185 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x561583ef89a0 0x561583efadd0 crc :-1 s=READY pgs=217 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6edb7b4c5b8b8392 server_cookie=eb727fd4268632ed in_seq=0 out_seq=0 2024-09-10T09:52:23.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:23.317+0000 7f8381600640 1 -- 172.21.15.5:0/3234365185 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561583e51950 con 0x561583ef89a0 2024-09-10T09:52:23.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:23.317+0000 7f8381600640 10 client.28316 handle_command_reply: tid=0 2024-09-10T09:52:23.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:23.317+0000 7f83a0fe5740 1 client.28316 shutdown 2024-09-10T09:52:23.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:23.317+0000 7f83a0fe5740 1 -- 172.21.15.5:0/3234365185 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561583eddb90 msgr2=0x561583ee5bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:52:23.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:23.317+0000 7f83a0fe5740 1 --2- 172.21.15.5:0/3234365185 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561583eddb90 0x561583ee5bb0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f8388009980 tx=0x7f83880ba810 comp rx=0 tx=0).stop 2024-09-10T09:52:23.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:23.317+0000 7f83a0fe5740 1 -- 172.21.15.5:0/3234365185 shutdown_connections 2024-09-10T09:52:23.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:23.317+0000 7f83a0fe5740 1 --2- 172.21.15.5:0/3234365185 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x561583ef89a0 0x561583efadd0 crc :-1 s=READY pgs=217 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:23.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:23.318+0000 7f83a0fe5740 1 --2- 172.21.15.5:0/3234365185 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561583ee8d70 0x561583ee6630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:23.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:23.318+0000 7f83a0fe5740 1 --2- 172.21.15.5:0/3234365185 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561583ede4b0 0x561583ee60f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:23.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:23.318+0000 7f83a0fe5740 1 --2- 172.21.15.5:0/3234365185 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561583eddb90 0x561583ee5bb0 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:23.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:23.318+0000 7f83a0fe5740 1 -- 172.21.15.5:0/3234365185 >> 172.21.15.5:0/3234365185 conn(0x561583eda120 msgr2=0x561583edfe50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:52:23.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:23.318+0000 7f8381600640 0 client.28316 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T09:52:23.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:23.319+0000 7f83a0fe5740 1 -- 172.21.15.5:0/3234365185 shutdown_connections 2024-09-10T09:52:23.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:23.319+0000 7f83a0fe5740 1 -- 172.21.15.5:0/3234365185 wait complete. 2024-09-10T09:52:23.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:23.320+0000 7f83a0fe5740 20 client.28316 trim_cache size 0 max 16384 2024-09-10T09:52:23.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:23.353+0000 7f83a0fe5740 1 Processor -- start 2024-09-10T09:52:23.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:23.353+0000 7f83a0fe5740 1 -- start start 2024-09-10T09:52:23.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:23.354+0000 7f83a0fe5740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561583eddb90 0x561583ee05f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:52:23.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:23.354+0000 7f83a0fe5740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561583ede4b0 0x561583ee6280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:52:23.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:23.354+0000 7f83a0fe5740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561583ee8d70 0x561583f00680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:52:23.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:23.354+0000 7f83a0fe5740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f83981b5490 con 0x561583ede4b0 2024-09-10T09:52:23.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:23.354+0000 7f83a0fe5740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f83981b6fd0 con 0x561583eddb90 2024-09-10T09:52:23.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:23.354+0000 7f83a0fe5740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f83981b5b60 con 0x561583ee8d70 2024-09-10T09:52:23.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:23.354+0000 7f8397e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561583ee8d70 0x561583f00680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:52:23.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:23.355+0000 7f8397e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561583ee8d70 0x561583f00680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:39286/0 (socket says 172.21.15.5:39286) 2024-09-10T09:52:23.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:23.355+0000 7f8397e00640 1 -- 172.21.15.5:0/1328515206 learned_addr learned my addr 172.21.15.5:0/1328515206 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:52:23.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:23.355+0000 7f8397400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561583eddb90 0x561583ee05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:52:23.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:23.355+0000 7f8396a00640 1 --2- 172.21.15.5:0/1328515206 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561583ede4b0 0x561583ee6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:52:23.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:23.356+0000 7f8397e00640 1 -- 172.21.15.5:0/1328515206 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561583eddb90 msgr2=0x561583ee05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:52:23.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:23.356+0000 7f8397e00640 1 --2- 172.21.15.5:0/1328515206 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561583eddb90 0x561583ee05f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:23.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:23.356+0000 7f8397e00640 1 -- 172.21.15.5:0/1328515206 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561583ede4b0 msgr2=0x561583ee6280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:52:23.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:23.356+0000 7f8397e00640 1 --2- 172.21.15.5:0/1328515206 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561583ede4b0 0x561583ee6280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:23.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:23.356+0000 7f8397e00640 1 -- 172.21.15.5:0/1328515206 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561583ed99b0 con 0x561583ee8d70 2024-09-10T09:52:23.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:23.356+0000 7f8396a00640 1 --2- 172.21.15.5:0/1328515206 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561583ede4b0 0x561583ee6280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T09:52:23.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:23.357+0000 7f8397e00640 1 --2- 172.21.15.5:0/1328515206 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561583ee8d70 0x561583f00680 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f839001e660 tx=0x7f8390013000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a0122c4e11ef69ee server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:52:23.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:23.357+0000 7f8383e00640 1 -- 172.21.15.5:0/1328515206 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f839001c570 con 0x561583ee8d70 2024-09-10T09:52:23.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:23.357+0000 7f8383e00640 1 -- 172.21.15.5:0/1328515206 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f839001cbf0 con 0x561583ee8d70 2024-09-10T09:52:23.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:23.357+0000 7f8383e00640 1 -- 172.21.15.5:0/1328515206 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8390027d20 con 0x561583ee8d70 2024-09-10T09:52:23.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:23.358+0000 7f83a0fe5740 1 -- 172.21.15.5:0/1328515206 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561583ee8d70 msgr2=0x561583f00680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:52:23.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:23.359+0000 7f83a0fe5740 1 --2- 172.21.15.5:0/1328515206 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561583ee8d70 0x561583f00680 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f839001e660 tx=0x7f8390013000 comp rx=0 tx=0).stop 2024-09-10T09:52:23.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:23.359+0000 7f83a0fe5740 1 -- 172.21.15.5:0/1328515206 shutdown_connections 2024-09-10T09:52:23.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:23.359+0000 7f83a0fe5740 1 --2- 172.21.15.5:0/1328515206 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561583ee8d70 0x561583f00680 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:23.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:23.359+0000 7f83a0fe5740 1 --2- 172.21.15.5:0/1328515206 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561583ede4b0 0x561583ee6280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:23.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:23.359+0000 7f83a0fe5740 1 --2- 172.21.15.5:0/1328515206 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561583eddb90 0x561583ee05f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:23.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:23.359+0000 7f83a0fe5740 1 -- 172.21.15.5:0/1328515206 >> 172.21.15.5:0/1328515206 conn(0x561583eda120 msgr2=0x561583ee5b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:52:23.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:23.359+0000 7f83a0fe5740 1 -- 172.21.15.5:0/1328515206 shutdown_connections 2024-09-10T09:52:23.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:23.359+0000 7f83a0fe5740 1 -- 172.21.15.5:0/1328515206 wait complete. 2024-09-10T09:52:23.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:23.360+0000 7f83a0fe5740 1 -- start start 2024-09-10T09:52:23.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:23.361+0000 7f83a0fe5740 1 Processor -- start 2024-09-10T09:52:23.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:23.362+0000 7f83a0fe5740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561583eddb90 0x561583ee2d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:52:23.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:23.362+0000 7f83a0fe5740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561583ede4b0 0x561583ee32c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:52:23.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:23.362+0000 7f83a0fe5740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561583ee8d70 0x561583ee3800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:52:23.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:23.362+0000 7f83a0fe5740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f83981b6900 con 0x561583ede4b0 2024-09-10T09:52:23.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:23.362+0000 7f83a0fe5740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f83981b6230 con 0x561583ee8d70 2024-09-10T09:52:23.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:23.362+0000 7f83a0fe5740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f83981b3470 con 0x561583eddb90 2024-09-10T09:52:23.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:23.362+0000 7f83a0fe5740 10 client.0 _ll_register_callbacks cb 0x7f83880b6f40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:52:23.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:23.362+0000 7f8397400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561583eddb90 0x561583ee2d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:52:23.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:23.362+0000 7f8397400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561583eddb90 0x561583ee2d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:39292/0 (socket says 172.21.15.5:39292) 2024-09-10T09:52:23.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:23.362+0000 7f8397400640 1 -- 172.21.15.5:0/1486394616 learned_addr learned my addr 172.21.15.5:0/1486394616 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:52:23.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:23.362+0000 7f8397e00640 1 --2- 172.21.15.5:0/1486394616 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561583ee8d70 0x561583ee3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:52:23.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:23.362+0000 7f8397400640 1 -- 172.21.15.5:0/1486394616 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561583ee8d70 msgr2=0x561583ee3800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:52:23.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:23.362+0000 7f8397400640 1 --2- 172.21.15.5:0/1486394616 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561583ee8d70 0x561583ee3800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:23.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:23.362+0000 7f8397400640 1 -- 172.21.15.5:0/1486394616 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561583ede4b0 msgr2=0x561583ee32c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:52:23.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:23.362+0000 7f8397400640 1 --2- 172.21.15.5:0/1486394616 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561583ede4b0 0x561583ee32c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:23.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:23.362+0000 7f8397400640 1 -- 172.21.15.5:0/1486394616 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561583f62750 con 0x561583eddb90 2024-09-10T09:52:23.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:23.363+0000 7f8397400640 1 --2- 172.21.15.5:0/1486394616 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561583eddb90 0x561583ee2d80 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f83880ec190 tx=0x7f838802a340 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1fdb17a2526d5ecd server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:52:23.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:23.363+0000 7f8381600640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-09-10T09:52:23.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:23.363+0000 7f8381600640 1 -- 172.21.15.5:0/1486394616 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8388042320 con 0x561583eddb90 2024-09-10T09:52:23.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:23.363+0000 7f8381600640 1 -- 172.21.15.5:0/1486394616 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f83880eebc0 con 0x561583eddb90 2024-09-10T09:52:23.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:23.364+0000 7f83a0fe5740 1 -- 172.21.15.5:0/1486394616 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f839001cbf0 con 0x561583eddb90 2024-09-10T09:52:23.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:23.364+0000 7f8381600640 1 -- 172.21.15.5:0/1486394616 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f83880bac10 con 0x561583eddb90 2024-09-10T09:52:23.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:23.364+0000 7f83a0fe5740 10 client.28315 fetch_fsmap learned FSMap version 36 2024-09-10T09:52:23.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:23.364+0000 7f83a0fe5740 1 -- 172.21.15.5:0/1486394616 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x561583f62750 con 0x561583eddb90 2024-09-10T09:52:23.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:23.364+0000 7f8381600640 1 -- 172.21.15.5:0/1486394616 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f839001cbf0 con 0x561583eddb90 2024-09-10T09:52:23.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:23.364+0000 7f8381600640 1 -- 172.21.15.5:0/1486394616 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 36) ==== 8370+0+0 (secure 0 0 0) 0x7f838803a070 con 0x561583eddb90 2024-09-10T09:52:23.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:23.364+0000 7f83a0fe5740 10 client.28315 fetch_fsmap finished waiting for FSMap version 36 2024-09-10T09:52:23.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:23.364+0000 7f83a0fe5740 10 client.28315 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T09:52:23.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:23.364+0000 7f83a0fe5740 20 client.28315 populate_metadata read hostname 'smithi005' 2024-09-10T09:52:23.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:23.364+0000 7f83a0fe5740 1 --2- 172.21.15.5:0/1486394616 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x561583f63d80 0x561583f661a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:52:23.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:23.364+0000 7f83a0fe5740 4 client.28315 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T09:52:23.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:23.364+0000 7f83a0fe5740 1 -- 172.21.15.5:0/1486394616 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "scrub status"}) -- 0x561583e4be60 con 0x561583f63d80 2024-09-10T09:52:23.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:23.365+0000 7f8396a00640 1 --2- 172.21.15.5:0/1486394616 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x561583f63d80 0x561583f661a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:52:23.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:23.366+0000 7f8381600640 10 client.28315 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T09:52:23.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:23.366+0000 7f8396a00640 1 --2- 172.21.15.5:0/1486394616 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x561583f63d80 0x561583f661a0 crc :-1 s=READY pgs=218 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c82149e9627537c5 server_cookie=e856bcde8bbc3446 in_seq=0 out_seq=0 2024-09-10T09:52:23.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:23.366+0000 7f8381600640 1 -- 172.21.15.5:0/1486394616 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x561583e4be60 con 0x561583f63d80 2024-09-10T09:52:23.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:23.366+0000 7f8381600640 10 client.28315 handle_command_reply: tid=0 2024-09-10T09:52:23.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:23.366+0000 7f83a0fe5740 1 client.28315 shutdown 2024-09-10T09:52:23.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:23.366+0000 7f83a0fe5740 1 -- 172.21.15.5:0/1486394616 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561583eddb90 msgr2=0x561583ee2d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:52:23.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:23.366+0000 7f83a0fe5740 1 --2- 172.21.15.5:0/1486394616 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561583eddb90 0x561583ee2d80 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f83880ec190 tx=0x7f838802a340 comp rx=0 tx=0).stop 2024-09-10T09:52:23.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:23.366+0000 7f83a0fe5740 1 -- 172.21.15.5:0/1486394616 shutdown_connections 2024-09-10T09:52:23.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:23.366+0000 7f83a0fe5740 1 --2- 172.21.15.5:0/1486394616 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x561583f63d80 0x561583f661a0 crc :-1 s=READY pgs=218 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:23.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:23.366+0000 7f83a0fe5740 1 --2- 172.21.15.5:0/1486394616 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561583ee8d70 0x561583ee3800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:23.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:23.366+0000 7f83a0fe5740 1 --2- 172.21.15.5:0/1486394616 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561583ede4b0 0x561583ee32c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:23.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:23.366+0000 7f83a0fe5740 1 --2- 172.21.15.5:0/1486394616 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561583eddb90 0x561583ee2d80 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:23.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:23.366+0000 7f83a0fe5740 1 -- 172.21.15.5:0/1486394616 >> 172.21.15.5:0/1486394616 conn(0x561583eda120 msgr2=0x561583ed91c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:52:23.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:23.367+0000 7f8381600640 0 client.28315 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T09:52:23.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:23.367+0000 7f83a0fe5740 1 -- 172.21.15.5:0/1486394616 shutdown_connections 2024-09-10T09:52:23.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:23.367+0000 7f83a0fe5740 1 -- 172.21.15.5:0/1486394616 wait complete. 2024-09-10T09:52:23.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:23.367+0000 7f83a0fe5740 20 client.28315 trim_cache size 0 max 16384 2024-09-10T09:52:23.370 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-10T09:52:23.370 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-10T09:52:23.370 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2024-09-10T09:52:23.370 INFO:teuthology.orchestra.run.smithi005.stdout: "e709f78a-84db-4fa8-a1fc-ddc55265697b": { 2024-09-10T09:52:23.370 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2024-09-10T09:52:23.371 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "e709f78a-84db-4fa8-a1fc-ddc55265697b", 2024-09-10T09:52:23.371 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2024-09-10T09:52:23.371 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T09:52:23.371 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T09:52:23.371 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-10T09:52:23.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:23.368+0000 7f839e600640 1 -- 172.21.15.5:0/3842233630 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f837c0a2c40 msgr2=0x7f837c0a5100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:52:23.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:23.368+0000 7f839e600640 1 --2- 172.21.15.5:0/3842233630 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f837c0a2c40 0x7f837c0a5100 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f8390005cc0 tx=0x7f839000b580 comp rx=0 tx=0).stop 2024-09-10T09:52:23.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:23.368+0000 7f839e600640 1 -- 172.21.15.5:0/3842233630 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8398103fd0 msgr2=0x7f83981a36e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:52:23.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:23.368+0000 7f839e600640 1 --2- 172.21.15.5:0/3842233630 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8398103fd0 0x7f83981a36e0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f838802a8b0 tx=0x7f8388033d10 comp rx=0 tx=0).stop 2024-09-10T09:52:23.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:23.368+0000 7f839e600640 1 -- 172.21.15.5:0/3842233630 shutdown_connections 2024-09-10T09:52:23.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:23.369+0000 7f839e600640 1 --2- 172.21.15.5:0/3842233630 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f837c0a2c40 0x7f837c0a5100 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:23.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:23.369+0000 7f839e600640 1 --2- 172.21.15.5:0/3842233630 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8398109100 0x7f839810f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:23.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:23.369+0000 7f839e600640 1 --2- 172.21.15.5:0/3842233630 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8398104980 0x7f83981a3c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:52:23.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:23.369+0000 7f839e600640 1 --2- 172.21.15.5:0/3842233630 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8398103fd0 0x7f83981a36e0 secure :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f838802a8b0 tx=0x7f8388033d10 comp rx=0 tx=0).stop 2024-09-10T09:52:23.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:23.369+0000 7f839e600640 1 -- 172.21.15.5:0/3842233630 >> 172.21.15.5:0/3842233630 conn(0x7f83980ffa30 msgr2=0x7f8398100d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:52:23.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:23.369+0000 7f839e600640 1 -- 172.21.15.5:0/3842233630 shutdown_connections 2024-09-10T09:52:23.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:52:23.369+0000 7f839e600640 1 -- 172.21.15.5:0/3842233630 wait complete. 2024-09-10T09:52:23.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:52:23 smithi102 ceph-mon[35442]: pgmap v814: 129 pgs: 129 active+clean; 287 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.1 KiB/s wr, 13 op/s 2024-09-10T09:52:23.402 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:52:23 smithi159 ceph-mon[35513]: pgmap v814: 129 pgs: 129 active+clean; 287 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.1 KiB/s wr, 13 op/s 2024-09-10T09:52:23.915 INFO:tasks.workunit.client.0.smithi005.stdout: 10 958 109.45 MB/sec warmup 2 sec latency 73.132 ms 2024-09-10T09:52:24.208 INFO:tasks.cephfs.filesystem:scrub status for tag:e709f78a-84db-4fa8-a1fc-ddc55265697b - {'path': '/', 'tag': 'e709f78a-84db-4fa8-a1fc-ddc55265697b', 'options': 'recursive,force'} 2024-09-10T09:52:24.915 INFO:tasks.workunit.client.0.smithi005.stdout: 10 1697 86.51 MB/sec warmup 3 sec latency 48.774 ms 2024-09-10T09:52:25.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:52:25 smithi102 ceph-mon[35442]: pgmap v815: 129 pgs: 129 active+clean; 288 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 77 KiB/s wr, 13 op/s 2024-09-10T09:52:25.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:52:25 smithi005 ceph-mon[30350]: pgmap v815: 129 pgs: 129 active+clean; 288 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 77 KiB/s wr, 13 op/s 2024-09-10T09:52:25.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:52:25 smithi159 ceph-mon[35513]: pgmap v815: 129 pgs: 129 active+clean; 288 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 77 KiB/s wr, 13 op/s 2024-09-10T09:52:25.915 INFO:tasks.workunit.client.0.smithi005.stdout: 10 2522 73.23 MB/sec warmup 4 sec latency 77.653 ms 2024-09-10T09:52:26.915 INFO:tasks.workunit.client.0.smithi005.stdout: 10 2754 63.62 MB/sec warmup 5 sec latency 790.736 ms 2024-09-10T09:52:27.248 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:52:27 smithi005 ceph-mon[30350]: pgmap v816: 129 pgs: 129 active+clean; 291 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 224 KiB/s rd, 548 KiB/s wr, 75 op/s 2024-09-10T09:52:27.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:52:27 smithi102 ceph-mon[35442]: pgmap v816: 129 pgs: 129 active+clean; 291 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 224 KiB/s rd, 548 KiB/s wr, 75 op/s 2024-09-10T09:52:27.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:52:27 smithi159 ceph-mon[35513]: pgmap v816: 129 pgs: 129 active+clean; 291 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 224 KiB/s rd, 548 KiB/s wr, 75 op/s 2024-09-10T09:52:27.915 INFO:tasks.workunit.client.0.smithi005.stdout: 10 3528 62.30 MB/sec warmup 6 sec latency 868.949 ms 2024-09-10T09:52:28.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:52:28 smithi005 ceph-mon[30350]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T09:52:28.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:52:28 smithi159 ceph-mon[35513]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T09:52:28.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:52:28 smithi102 ceph-mon[35442]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T09:52:28.916 INFO:tasks.workunit.client.0.smithi005.stdout: 10 4140 57.40 MB/sec warmup 7 sec latency 43.555 ms 2024-09-10T09:52:29.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:52:29 smithi005 ceph-mon[30350]: pgmap v817: 129 pgs: 129 active+clean; 285 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 215 KiB/s rd, 850 KiB/s wr, 88 op/s 2024-09-10T09:52:29.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:52:29 smithi159 ceph-mon[35513]: pgmap v817: 129 pgs: 129 active+clean; 285 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 215 KiB/s rd, 850 KiB/s wr, 88 op/s 2024-09-10T09:52:29.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:52:29 smithi102 ceph-mon[35442]: pgmap v817: 129 pgs: 129 active+clean; 285 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 215 KiB/s rd, 850 KiB/s wr, 88 op/s 2024-09-10T09:52:29.916 INFO:tasks.workunit.client.0.smithi005.stdout: 10 4701 53.32 MB/sec warmup 8 sec latency 98.713 ms 2024-09-10T09:52:30.916 INFO:tasks.workunit.client.0.smithi005.stdout: 10 5393 51.85 MB/sec warmup 9 sec latency 63.094 ms 2024-09-10T09:52:31.408 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:52:31 smithi005 ceph-mon[30350]: pgmap v818: 129 pgs: 129 active+clean; 288 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 928 KiB/s rd, 2.2 MiB/s wr, 222 op/s 2024-09-10T09:52:31.436 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:52:31 smithi102 ceph-mon[35442]: pgmap v818: 129 pgs: 129 active+clean; 288 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 928 KiB/s rd, 2.2 MiB/s wr, 222 op/s 2024-09-10T09:52:31.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:52:31 smithi159 ceph-mon[35513]: pgmap v818: 129 pgs: 129 active+clean; 288 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 928 KiB/s rd, 2.2 MiB/s wr, 222 op/s 2024-09-10T09:52:31.916 INFO:tasks.workunit.client.0.smithi005.stdout: 10 5988 49.27 MB/sec warmup 10 sec latency 328.054 ms 2024-09-10T09:52:32.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:52:32 smithi005 ceph-mon[30350]: scrub summary: idle 2024-09-10T09:52:32.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:52:32 smithi159 ceph-mon[35513]: scrub summary: idle 2024-09-10T09:52:32.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:52:32 smithi102 ceph-mon[35442]: scrub summary: idle 2024-09-10T09:52:32.916 INFO:tasks.workunit.client.0.smithi005.stdout: 10 6921 49.38 MB/sec warmup 11 sec latency 1146.285 ms 2024-09-10T09:52:33.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:52:33 smithi005 ceph-mon[30350]: pgmap v819: 129 pgs: 129 active+clean; 292 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1011 KiB/s rd, 2.7 MiB/s wr, 252 op/s 2024-09-10T09:52:33.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:52:33 smithi005 ceph-mon[30350]: scrub summary: idle 2024-09-10T09:52:33.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:52:33 smithi159 ceph-mon[35513]: pgmap v819: 129 pgs: 129 active+clean; 292 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1011 KiB/s rd, 2.7 MiB/s wr, 252 op/s 2024-09-10T09:52:33.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:52:33 smithi159 ceph-mon[35513]: scrub summary: idle 2024-09-10T09:52:33.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:52:33 smithi102 ceph-mon[35442]: pgmap v819: 129 pgs: 129 active+clean; 292 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1011 KiB/s rd, 2.7 MiB/s wr, 252 op/s 2024-09-10T09:52:33.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:52:33 smithi102 ceph-mon[35442]: scrub summary: idle 2024-09-10T09:52:33.916 INFO:tasks.workunit.client.0.smithi005.stdout: 10 7805 50.41 MB/sec warmup 12 sec latency 1612.146 ms 2024-09-10T09:52:34.916 INFO:tasks.workunit.client.0.smithi005.stdout: 10 8438 48.53 MB/sec warmup 13 sec latency 91.440 ms 2024-09-10T09:52:35.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:52:35 smithi005 ceph-mon[30350]: pgmap v820: 129 pgs: 129 active+clean; 295 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 3.1 MiB/s wr, 289 op/s 2024-09-10T09:52:35.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:52:35 smithi159 ceph-mon[35513]: pgmap v820: 129 pgs: 129 active+clean; 295 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 3.1 MiB/s wr, 289 op/s 2024-09-10T09:52:35.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:52:35 smithi102 ceph-mon[35442]: pgmap v820: 129 pgs: 129 active+clean; 295 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 3.1 MiB/s wr, 289 op/s 2024-09-10T09:52:35.916 INFO:tasks.workunit.client.0.smithi005.stdout: 10 9120 47.72 MB/sec warmup 14 sec latency 74.862 ms 2024-09-10T09:52:36.916 INFO:tasks.workunit.client.0.smithi005.stdout: 10 9640 46.68 MB/sec warmup 15 sec latency 183.298 ms 2024-09-10T09:52:37.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:52:37 smithi005 ceph-mon[30350]: pgmap v821: 129 pgs: 129 active+clean; 302 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.8 MiB/s wr, 392 op/s 2024-09-10T09:52:37.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:52:37 smithi159 ceph-mon[35513]: pgmap v821: 129 pgs: 129 active+clean; 302 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.8 MiB/s wr, 392 op/s 2024-09-10T09:52:37.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:52:37 smithi102 ceph-mon[35442]: pgmap v821: 129 pgs: 129 active+clean; 302 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.8 MiB/s wr, 392 op/s 2024-09-10T09:52:37.916 INFO:tasks.workunit.client.0.smithi005.stdout: 10 10421 45.44 MB/sec warmup 16 sec latency 53.878 ms 2024-09-10T09:52:38.916 INFO:tasks.workunit.client.0.smithi005.stdout: 10 11380 46.19 MB/sec warmup 17 sec latency 54.660 ms 2024-09-10T09:52:39.429 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:52:39 smithi005 ceph-mon[30350]: pgmap v822: 129 pgs: 129 active+clean; 310 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.3 MiB/s wr, 447 op/s 2024-09-10T09:52:39.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:52:39 smithi159 ceph-mon[35513]: pgmap v822: 129 pgs: 129 active+clean; 310 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.3 MiB/s wr, 447 op/s 2024-09-10T09:52:39.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:52:39 smithi102 ceph-mon[35442]: pgmap v822: 129 pgs: 129 active+clean; 310 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.3 MiB/s wr, 447 op/s 2024-09-10T09:52:39.916 INFO:tasks.workunit.client.0.smithi005.stdout: 10 12238 46.58 MB/sec warmup 18 sec latency 50.862 ms 2024-09-10T09:52:40.916 INFO:tasks.workunit.client.0.smithi005.stdout: 10 12848 45.63 MB/sec warmup 19 sec latency 53.486 ms 2024-09-10T09:52:41.559 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:52:41 smithi005 ceph-mon[30350]: pgmap v823: 129 pgs: 129 active+clean; 316 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.7 MiB/s wr, 515 op/s 2024-09-10T09:52:41.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:52:41 smithi159 ceph-mon[35513]: pgmap v823: 129 pgs: 129 active+clean; 316 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.7 MiB/s wr, 515 op/s 2024-09-10T09:52:41.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:52:41 smithi102 ceph-mon[35442]: pgmap v823: 129 pgs: 129 active+clean; 316 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.7 MiB/s wr, 515 op/s 2024-09-10T09:52:41.917 INFO:tasks.workunit.client.0.smithi005.stdout: 10 13411 44.74 MB/sec warmup 20 sec latency 238.596 ms 2024-09-10T09:52:42.917 INFO:tasks.workunit.client.0.smithi005.stdout: 10 14103 44.66 MB/sec warmup 21 sec latency 146.417 ms 2024-09-10T09:52:43.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:52:43 smithi005 ceph-mon[30350]: pgmap v824: 129 pgs: 129 active+clean; 326 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.3 MiB/s wr, 481 op/s 2024-09-10T09:52:43.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:52:43 smithi159 ceph-mon[35513]: pgmap v824: 129 pgs: 129 active+clean; 326 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.3 MiB/s wr, 481 op/s 2024-09-10T09:52:43.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:52:43 smithi102 ceph-mon[35442]: pgmap v824: 129 pgs: 129 active+clean; 326 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.3 MiB/s wr, 481 op/s 2024-09-10T09:52:43.917 INFO:tasks.workunit.client.0.smithi005.stdout: 10 14972 44.09 MB/sec warmup 22 sec latency 58.401 ms 2024-09-10T09:52:44.917 INFO:tasks.workunit.client.0.smithi005.stdout: 10 15864 44.64 MB/sec warmup 23 sec latency 54.480 ms 2024-09-10T09:52:45.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:52:45 smithi005 ceph-mon[30350]: pgmap v825: 129 pgs: 129 active+clean; 329 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 5.2 MiB/s wr, 504 op/s 2024-09-10T09:52:45.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:52:45 smithi159 ceph-mon[35513]: pgmap v825: 129 pgs: 129 active+clean; 329 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 5.2 MiB/s wr, 504 op/s 2024-09-10T09:52:45.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:52:45 smithi102 ceph-mon[35442]: pgmap v825: 129 pgs: 129 active+clean; 329 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 5.2 MiB/s wr, 504 op/s 2024-09-10T09:52:45.917 INFO:tasks.workunit.client.0.smithi005.stdout: 10 16677 45.13 MB/sec warmup 24 sec latency 48.063 ms 2024-09-10T09:52:46.918 INFO:tasks.workunit.client.0.smithi005.stdout: 10 17044 43.65 MB/sec warmup 25 sec latency 289.638 ms 2024-09-10T09:52:47.445 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:52:47 smithi005 ceph-mon[30350]: pgmap v826: 129 pgs: 129 active+clean; 339 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 6.8 MiB/s wr, 568 op/s 2024-09-10T09:52:47.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:52:47 smithi159 ceph-mon[35513]: pgmap v826: 129 pgs: 129 active+clean; 339 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 6.8 MiB/s wr, 568 op/s 2024-09-10T09:52:47.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:52:47 smithi102 ceph-mon[35442]: pgmap v826: 129 pgs: 129 active+clean; 339 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 6.8 MiB/s wr, 568 op/s 2024-09-10T09:52:47.917 INFO:tasks.workunit.client.0.smithi005.stdout: 10 17083 42.00 MB/sec warmup 26 sec latency 448.652 ms 2024-09-10T09:52:48.935 INFO:tasks.workunit.client.0.smithi005.stdout: 10 17094 40.43 MB/sec warmup 27 sec latency 1045.522 ms 2024-09-10T09:52:49.359 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:52:49 smithi005 ceph-mon[30350]: pgmap v827: 129 pgs: 129 active+clean; 367 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 8.5 MiB/s wr, 524 op/s 2024-09-10T09:52:49.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:52:49 smithi159 ceph-mon[35513]: pgmap v827: 129 pgs: 129 active+clean; 367 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 8.5 MiB/s wr, 524 op/s 2024-09-10T09:52:49.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:52:49 smithi102 ceph-mon[35442]: pgmap v827: 129 pgs: 129 active+clean; 367 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 8.5 MiB/s wr, 524 op/s 2024-09-10T09:52:49.941 INFO:tasks.workunit.client.0.smithi005.stdout: 10 17096 38.98 MB/sec warmup 28 sec latency 2052.375 ms 2024-09-10T09:52:50.942 INFO:tasks.workunit.client.0.smithi005.stdout: 10 17097 37.63 MB/sec warmup 29 sec latency 2812.239 ms 2024-09-10T09:52:51.336 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:52:51 smithi005 ceph-mon[30350]: pgmap v828: 129 pgs: 129 active+clean; 516 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 21 MiB/s wr, 643 op/s 2024-09-10T09:52:51.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:52:51 smithi159 ceph-mon[35513]: pgmap v828: 129 pgs: 129 active+clean; 516 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 21 MiB/s wr, 643 op/s 2024-09-10T09:52:51.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:52:51 smithi102 ceph-mon[35442]: pgmap v828: 129 pgs: 129 active+clean; 516 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 21 MiB/s wr, 643 op/s 2024-09-10T09:52:51.941 INFO:tasks.workunit.client.0.smithi005.stdout: 10 17098 36.38 MB/sec warmup 30 sec latency 3284.982 ms 2024-09-10T09:52:52.942 INFO:tasks.workunit.client.0.smithi005.stdout: 10 17098 35.21 MB/sec warmup 31 sec latency 4097.648 ms 2024-09-10T09:52:53.362 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:52:53 smithi005 ceph-mon[30350]: pgmap v829: 129 pgs: 129 active+clean; 562 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 26 MiB/s wr, 651 op/s 2024-09-10T09:52:53.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:52:53 smithi159 ceph-mon[35513]: pgmap v829: 129 pgs: 129 active+clean; 562 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 26 MiB/s wr, 651 op/s 2024-09-10T09:52:53.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:52:53 smithi102 ceph-mon[35442]: pgmap v829: 129 pgs: 129 active+clean; 562 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 26 MiB/s wr, 651 op/s 2024-09-10T09:52:53.942 INFO:tasks.workunit.client.0.smithi005.stdout: 10 17099 34.11 MB/sec warmup 32 sec latency 5097.701 ms 2024-09-10T09:52:54.209 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T09:52:54.630 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:52:54.942 INFO:tasks.workunit.client.0.smithi005.stdout: 10 17160 33.22 MB/sec warmup 33 sec latency 6097.742 ms 2024-09-10T09:52:55.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:52:55 smithi005 ceph-mon[30350]: pgmap v830: 129 pgs: 129 active+clean; 539 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 27 MiB/s wr, 620 op/s 2024-09-10T09:52:55.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:52:55 smithi159 ceph-mon[35513]: pgmap v830: 129 pgs: 129 active+clean; 539 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 27 MiB/s wr, 620 op/s 2024-09-10T09:52:55.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:52:55 smithi102 ceph-mon[35442]: pgmap v830: 129 pgs: 129 active+clean; 539 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 27 MiB/s wr, 620 op/s 2024-09-10T09:52:55.942 INFO:tasks.workunit.client.0.smithi005.stdout: 10 17160 32.24 MB/sec warmup 34 sec latency 7097.783 ms 2024-09-10T09:52:56.942 INFO:tasks.workunit.client.0.smithi005.stdout: 10 17160 31.32 MB/sec warmup 35 sec latency 8097.837 ms 2024-09-10T09:52:57.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:52:57 smithi005 ceph-mon[30350]: pgmap v831: 129 pgs: 129 active+clean; 517 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 26 MiB/s wr, 613 op/s 2024-09-10T09:52:57.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:52:57 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:52:57.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:52:57 smithi159 ceph-mon[35513]: pgmap v831: 129 pgs: 129 active+clean; 517 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 26 MiB/s wr, 613 op/s 2024-09-10T09:52:57.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:52:57 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:52:57.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:52:57 smithi102 ceph-mon[35442]: pgmap v831: 129 pgs: 129 active+clean; 517 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 26 MiB/s wr, 613 op/s 2024-09-10T09:52:57.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:52:57 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:52:57.942 INFO:tasks.workunit.client.0.smithi005.stdout: 10 17160 30.45 MB/sec warmup 36 sec latency 9098.119 ms 2024-09-10T09:52:58.942 INFO:tasks.workunit.client.0.smithi005.stdout: 10 17160 29.63 MB/sec warmup 37 sec latency 10098.161 ms 2024-09-10T09:52:59.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:52:59 smithi005 ceph-mon[30350]: pgmap v832: 129 pgs: 129 active+clean; 503 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 24 MiB/s wr, 531 op/s 2024-09-10T09:52:59.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:52:59 smithi159 ceph-mon[35513]: pgmap v832: 129 pgs: 129 active+clean; 503 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 24 MiB/s wr, 531 op/s 2024-09-10T09:52:59.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:52:59 smithi102 ceph-mon[35442]: pgmap v832: 129 pgs: 129 active+clean; 503 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 24 MiB/s wr, 531 op/s 2024-09-10T09:52:59.942 INFO:tasks.workunit.client.0.smithi005.stdout: 10 17160 28.85 MB/sec warmup 38 sec latency 11098.217 ms 2024-09-10T09:53:00.942 INFO:tasks.workunit.client.0.smithi005.stdout: 10 17160 28.11 MB/sec warmup 39 sec latency 12098.271 ms 2024-09-10T09:53:01.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:53:01 smithi005 ceph-mon[30350]: pgmap v833: 129 pgs: 129 active+clean; 503 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 21 MiB/s wr, 476 op/s 2024-09-10T09:53:01.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:53:01 smithi159 ceph-mon[35513]: pgmap v833: 129 pgs: 129 active+clean; 503 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 21 MiB/s wr, 476 op/s 2024-09-10T09:53:01.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:53:01 smithi102 ceph-mon[35442]: pgmap v833: 129 pgs: 129 active+clean; 503 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 21 MiB/s wr, 476 op/s 2024-09-10T09:53:01.942 INFO:tasks.workunit.client.0.smithi005.stdout: 10 17160 27.41 MB/sec warmup 40 sec latency 13098.326 ms 2024-09-10T09:53:02.942 INFO:tasks.workunit.client.0.smithi005.stdout: 10 17160 26.74 MB/sec warmup 41 sec latency 14098.383 ms 2024-09-10T09:53:03.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:53:03 smithi005 ceph-mon[30350]: pgmap v834: 129 pgs: 129 active+clean; 503 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 276 KiB/s rd, 7.8 MiB/s wr, 241 op/s 2024-09-10T09:53:03.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:53:03 smithi159 ceph-mon[35513]: pgmap v834: 129 pgs: 129 active+clean; 503 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 276 KiB/s rd, 7.8 MiB/s wr, 241 op/s 2024-09-10T09:53:03.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:53:03 smithi102 ceph-mon[35442]: pgmap v834: 129 pgs: 129 active+clean; 503 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 276 KiB/s rd, 7.8 MiB/s wr, 241 op/s 2024-09-10T09:53:03.942 INFO:tasks.workunit.client.0.smithi005.stdout: 10 17160 26.10 MB/sec warmup 42 sec latency 15098.437 ms 2024-09-10T09:53:04.942 INFO:tasks.workunit.client.0.smithi005.stdout: 10 17160 25.50 MB/sec warmup 43 sec latency 16098.501 ms 2024-09-10T09:53:05.448 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:53:05 smithi005 ceph-mon[30350]: pgmap v835: 129 pgs: 129 active+clean; 503 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 256 KiB/s rd, 1.9 MiB/s wr, 148 op/s 2024-09-10T09:53:05.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:53:05 smithi159 ceph-mon[35513]: pgmap v835: 129 pgs: 129 active+clean; 503 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 256 KiB/s rd, 1.9 MiB/s wr, 148 op/s 2024-09-10T09:53:05.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:53:05 smithi102 ceph-mon[35442]: pgmap v835: 129 pgs: 129 active+clean; 503 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 256 KiB/s rd, 1.9 MiB/s wr, 148 op/s 2024-09-10T09:53:05.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:05.705+0000 7f9437e00640 1 -- 172.21.15.5:0/1785203060 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9430101620 msgr2=0x7f9430101a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:53:05.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:05.705+0000 7f9437e00640 1 --2- 172.21.15.5:0/1785203060 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9430101620 0x7f9430101a80 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f941c009900 tx=0x7f941c02a410 comp rx=0 tx=0).stop 2024-09-10T09:53:05.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:05.705+0000 7f9437e00640 1 -- 172.21.15.5:0/1785203060 shutdown_connections 2024-09-10T09:53:05.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:05.705+0000 7f9437e00640 1 --2- 172.21.15.5:0/1785203060 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9430101fc0 0x7f943010d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:05.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:05.705+0000 7f9437e00640 1 --2- 172.21.15.5:0/1785203060 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9430101620 0x7f9430101a80 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:05.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:05.705+0000 7f9437e00640 1 --2- 172.21.15.5:0/1785203060 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9430105aa0 0x7f9430105e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:05.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:05.705+0000 7f9437e00640 1 -- 172.21.15.5:0/1785203060 >> 172.21.15.5:0/1785203060 conn(0x7f94300fba90 msgr2=0x7f94300fdeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:53:05.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:05.705+0000 7f9437e00640 1 -- 172.21.15.5:0/1785203060 shutdown_connections 2024-09-10T09:53:05.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:05.705+0000 7f9437e00640 1 -- 172.21.15.5:0/1785203060 wait complete. 2024-09-10T09:53:05.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:05.706+0000 7f9437e00640 1 Processor -- start 2024-09-10T09:53:05.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:05.706+0000 7f9437e00640 1 -- start start 2024-09-10T09:53:05.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:05.707+0000 7f9437e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9430101620 0x7f94301027a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:05.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:05.707+0000 7f9437e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9430101fc0 0x7f9430102ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:05.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:05.707+0000 7f9437e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9430105aa0 0x7f9430103220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:05.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:05.707+0000 7f9437e00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f9430110ec0 con 0x7f9430105aa0 2024-09-10T09:53:05.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:05.707+0000 7f9437e00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f9430110d40 con 0x7f9430101fc0 2024-09-10T09:53:05.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:05.707+0000 7f9437e00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f9430111040 con 0x7f9430101620 2024-09-10T09:53:05.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:05.707+0000 7f9435400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9430101620 0x7f94301027a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:05.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:05.707+0000 7f9434a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9430101fc0 0x7f9430102ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:05.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:05.707+0000 7f9435400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9430101620 0x7f94301027a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:49124/0 (socket says 172.21.15.5:49124) 2024-09-10T09:53:05.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:05.707+0000 7f9435400640 1 -- 172.21.15.5:0/3916255539 learned_addr learned my addr 172.21.15.5:0/3916255539 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:53:05.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:05.707+0000 7f9435e00640 1 --2- 172.21.15.5:0/3916255539 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9430105aa0 0x7f9430103220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:05.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:05.707+0000 7f9435400640 1 -- 172.21.15.5:0/3916255539 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9430101fc0 msgr2=0x7f9430102ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:53:05.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:05.707+0000 7f9435400640 1 --2- 172.21.15.5:0/3916255539 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9430101fc0 0x7f9430102ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:05.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:05.708+0000 7f9435400640 1 -- 172.21.15.5:0/3916255539 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9430105aa0 msgr2=0x7f9430103220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:53:05.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:05.708+0000 7f9435400640 1 --2- 172.21.15.5:0/3916255539 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9430105aa0 0x7f9430103220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:05.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:05.708+0000 7f9435400640 1 -- 172.21.15.5:0/3916255539 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94301aade0 con 0x7f9430101620 2024-09-10T09:53:05.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:05.708+0000 7f9434a00640 1 --2- 172.21.15.5:0/3916255539 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9430101fc0 0x7f9430102ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:53:05.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:05.708+0000 7f9435e00640 1 --2- 172.21.15.5:0/3916255539 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9430105aa0 0x7f9430103220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:53:05.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:05.708+0000 7f9435400640 1 --2- 172.21.15.5:0/3916255539 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9430101620 0x7f94301027a0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f942400e9f0 tx=0x7f942400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:53:05.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:05.708+0000 7f942e000640 1 -- 172.21.15.5:0/3916255539 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f942400cd80 con 0x7f9430101620 2024-09-10T09:53:05.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:05.708+0000 7f9437e00640 1 -- 172.21.15.5:0/3916255539 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94301aafb0 con 0x7f9430101620 2024-09-10T09:53:05.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:05.708+0000 7f942e000640 1 -- 172.21.15.5:0/3916255539 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9424010910 con 0x7f9430101620 2024-09-10T09:53:05.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:05.708+0000 7f942e000640 1 -- 172.21.15.5:0/3916255539 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9424018950 con 0x7f9430101620 2024-09-10T09:53:05.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:05.708+0000 7f9437e00640 1 -- 172.21.15.5:0/3916255539 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94301ab440 con 0x7f9430101620 2024-09-10T09:53:05.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:05.711+0000 7f943aa2b740 1 Processor -- start 2024-09-10T09:53:05.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:05.712+0000 7f943aa2b740 1 -- start start 2024-09-10T09:53:05.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:05.712+0000 7f942e000640 1 -- 172.21.15.5:0/3916255539 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f9424018bd0 con 0x7f9430101620 2024-09-10T09:53:05.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:05.712+0000 7f943aa2b740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x565406758b90 0x565406758f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:05.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:05.712+0000 7f943aa2b740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5654067594b0 0x565406763830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:05.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:05.712+0000 7f943aa2b740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x565406763d70 0x565406766160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:05.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:05.712+0000 7f943aa2b740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5654066bf3e0 con 0x565406758b90 2024-09-10T09:53:05.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:05.712+0000 7f943aa2b740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x565406666d80 con 0x565406763d70 2024-09-10T09:53:05.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:05.712+0000 7f943aa2b740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5654066bb3a0 con 0x5654067594b0 2024-09-10T09:53:05.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:05.712+0000 7f9435e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x565406758b90 0x565406758f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:05.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:05.712+0000 7f9435e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x565406758b90 0x565406758f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36748/0 (socket says 172.21.15.5:36748) 2024-09-10T09:53:05.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:05.712+0000 7f9435e00640 1 -- 172.21.15.5:0/408750292 learned_addr learned my addr 172.21.15.5:0/408750292 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:53:05.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:05.713+0000 7f9435400640 1 --2- 172.21.15.5:0/408750292 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5654067594b0 0x565406763830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:05.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:05.713+0000 7f9434a00640 1 --2- 172.21.15.5:0/408750292 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x565406763d70 0x565406766160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:05.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:05.713+0000 7f9435e00640 1 -- 172.21.15.5:0/408750292 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5654067594b0 msgr2=0x565406763830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:53:05.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:05.713+0000 7f9435e00640 1 --2- 172.21.15.5:0/408750292 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5654067594b0 0x565406763830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:05.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:05.713+0000 7f9435e00640 1 -- 172.21.15.5:0/408750292 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x565406763d70 msgr2=0x565406766160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:53:05.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:05.713+0000 7f9435e00640 1 --2- 172.21.15.5:0/408750292 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x565406763d70 0x565406766160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:05.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:05.713+0000 7f9435e00640 1 -- 172.21.15.5:0/408750292 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565406685ae0 con 0x565406758b90 2024-09-10T09:53:05.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:05.713+0000 7f9435e00640 1 --2- 172.21.15.5:0/408750292 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x565406758b90 0x565406758f70 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f9428009fd0 tx=0x7f94280108e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe97c2f3c0a50dd1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:53:05.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:05.713+0000 7f9422600640 1 -- 172.21.15.5:0/408750292 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9428011760 con 0x565406758b90 2024-09-10T09:53:05.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:05.713+0000 7f9422600640 1 -- 172.21.15.5:0/408750292 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9428011de0 con 0x565406758b90 2024-09-10T09:53:05.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:05.714+0000 7f9422600640 1 -- 172.21.15.5:0/408750292 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f942800e3e0 con 0x565406758b90 2024-09-10T09:53:05.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:05.714+0000 7f943aa2b740 1 -- 172.21.15.5:0/408750292 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x565406758b90 msgr2=0x565406758f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:53:05.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:05.714+0000 7f943aa2b740 1 --2- 172.21.15.5:0/408750292 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x565406758b90 0x565406758f70 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f9428009fd0 tx=0x7f94280108e0 comp rx=0 tx=0).stop 2024-09-10T09:53:05.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:05.714+0000 7f942e000640 1 --2- 172.21.15.5:0/3916255539 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f94180a2c00 0x7f94180a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:05.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:05.714+0000 7f943aa2b740 1 -- 172.21.15.5:0/408750292 shutdown_connections 2024-09-10T09:53:05.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:05.714+0000 7f943aa2b740 1 --2- 172.21.15.5:0/408750292 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x565406763d70 0x565406766160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:05.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:05.714+0000 7f943aa2b740 1 --2- 172.21.15.5:0/408750292 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5654067594b0 0x565406763830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:05.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:05.714+0000 7f943aa2b740 1 --2- 172.21.15.5:0/408750292 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x565406758b90 0x565406758f70 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:05.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:05.714+0000 7f943aa2b740 1 -- 172.21.15.5:0/408750292 >> 172.21.15.5:0/408750292 conn(0x565406755120 msgr2=0x565406755500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:53:05.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:05.714+0000 7f942e000640 1 -- 172.21.15.5:0/3916255539 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f94240cf7c0 con 0x7f9430101620 2024-09-10T09:53:05.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:05.715+0000 7f9435400640 1 --2- 172.21.15.5:0/3916255539 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f94180a2c00 0x7f94180a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:05.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:05.715+0000 7f9435400640 1 --2- 172.21.15.5:0/3916255539 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f94180a2c00 0x7f94180a50c0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x565406756080 tx=0x7f9424013000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:53:05.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:05.716+0000 7f943aa2b740 1 -- 172.21.15.5:0/408750292 shutdown_connections 2024-09-10T09:53:05.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:05.716+0000 7f943aa2b740 1 -- 172.21.15.5:0/408750292 wait complete. 2024-09-10T09:53:05.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:05.717+0000 7f943aa2b740 1 -- start start 2024-09-10T09:53:05.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:05.717+0000 7f943aa2b740 1 Processor -- start 2024-09-10T09:53:05.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:05.717+0000 7f943aa2b740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x565406758b90 0x565406760ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:05.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:05.717+0000 7f943aa2b740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x565406763d70 0x565406761400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:05.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:05.717+0000 7f943aa2b740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x565406761940 0x56540676fe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:05.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:05.717+0000 7f943aa2b740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5654067669e0 con 0x565406758b90 2024-09-10T09:53:05.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:05.717+0000 7f943aa2b740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x56540667a0f0 con 0x565406761940 2024-09-10T09:53:05.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:05.717+0000 7f943aa2b740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x565406766b60 con 0x565406763d70 2024-09-10T09:53:05.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:05.717+0000 7f943aa2b740 10 client.0 _ll_register_callbacks cb 0x56540666b9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:53:05.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:05.718+0000 7f9435e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x565406758b90 0x565406760ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:05.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:05.718+0000 7f9435e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x565406758b90 0x565406760ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36750/0 (socket says 172.21.15.5:36750) 2024-09-10T09:53:05.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:05.718+0000 7f9435e00640 1 -- 172.21.15.5:0/1133701056 learned_addr learned my addr 172.21.15.5:0/1133701056 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:53:05.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:05.718+0000 7f9434a00640 1 --2- 172.21.15.5:0/1133701056 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x565406763d70 0x565406761400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:05.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:05.718+0000 7f9435e00640 1 --2- 172.21.15.5:0/1133701056 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x565406761940 0x56540676fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:05.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:05.718+0000 7f9434a00640 1 -- 172.21.15.5:0/1133701056 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x565406761940 msgr2=0x56540676fe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:53:05.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:05.718+0000 7f9434a00640 1 --2- 172.21.15.5:0/1133701056 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x565406761940 0x56540676fe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:05.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:05.718+0000 7f9434a00640 1 -- 172.21.15.5:0/1133701056 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x565406758b90 msgr2=0x565406760ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:53:05.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:05.718+0000 7f9434a00640 1 --2- 172.21.15.5:0/1133701056 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x565406758b90 0x565406760ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:05.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:05.718+0000 7f9434a00640 1 -- 172.21.15.5:0/1133701056 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5654064303c0 con 0x565406763d70 2024-09-10T09:53:05.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:05.719+0000 7f9434a00640 1 --2- 172.21.15.5:0/1133701056 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x565406763d70 0x565406761400 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f941c005940 tx=0x7f941c0317e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=430b283a16d2a76 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:53:05.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:05.719+0000 7f9416a00640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-09-10T09:53:05.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:05.719+0000 7f9416a00640 1 -- 172.21.15.5:0/1133701056 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f941c035040 con 0x565406763d70 2024-09-10T09:53:05.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:05.719+0000 7f9416a00640 1 -- 172.21.15.5:0/1133701056 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f941c004240 con 0x565406763d70 2024-09-10T09:53:05.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:05.719+0000 7f943aa2b740 1 -- 172.21.15.5:0/1133701056 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9428011de0 con 0x565406763d70 2024-09-10T09:53:05.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:05.719+0000 7f9416a00640 1 -- 172.21.15.5:0/1133701056 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f941c004440 con 0x565406763d70 2024-09-10T09:53:05.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:05.719+0000 7f943aa2b740 10 client.28327 fetch_fsmap learned FSMap version 37 2024-09-10T09:53:05.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:05.719+0000 7f943aa2b740 1 -- 172.21.15.5:0/1133701056 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x565406772cd0 con 0x565406763d70 2024-09-10T09:53:05.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:05.720+0000 7f9416a00640 1 -- 172.21.15.5:0/1133701056 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9428011de0 con 0x565406763d70 2024-09-10T09:53:05.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:05.720+0000 7f9416a00640 1 -- 172.21.15.5:0/1133701056 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 37) ==== 8390+0+0 (secure 0 0 0) 0x7f941c004c20 con 0x565406763d70 2024-09-10T09:53:05.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:05.720+0000 7f943aa2b740 10 client.28327 fetch_fsmap finished waiting for FSMap version 37 2024-09-10T09:53:05.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:05.720+0000 7f943aa2b740 10 client.28327 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T09:53:05.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:05.720+0000 7f943aa2b740 20 client.28327 populate_metadata read hostname 'smithi005' 2024-09-10T09:53:05.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:05.720+0000 7f943aa2b740 1 --2- 172.21.15.5:0/1133701056 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x565406774190 0x5654067765a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:05.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:05.720+0000 7f943aa2b740 4 client.28327 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:53:05.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:05.720+0000 7f943aa2b740 1 -- 172.21.15.5:0/1133701056 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5654066cc950 con 0x565406774190 2024-09-10T09:53:05.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:05.721+0000 7f9435e00640 1 --2- 172.21.15.5:0/1133701056 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x565406774190 0x5654067765a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:05.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:05.721+0000 7f9435e00640 1 --2- 172.21.15.5:0/1133701056 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x565406774190 0x5654067765a0 crc :-1 s=READY pgs=219 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4f49fee7c8d5befe server_cookie=aef9c0ab61cc34bf in_seq=0 out_seq=0 2024-09-10T09:53:05.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:05.721+0000 7f9416a00640 10 client.28327 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T09:53:05.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:05.722+0000 7f9416a00640 1 -- 172.21.15.5:0/1133701056 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5654066cc950 con 0x565406774190 2024-09-10T09:53:05.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:05.722+0000 7f9416a00640 10 client.28327 handle_command_reply: tid=0 2024-09-10T09:53:05.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:05.722+0000 7f943aa2b740 1 client.28327 shutdown 2024-09-10T09:53:05.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:05.722+0000 7f943aa2b740 1 -- 172.21.15.5:0/1133701056 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x565406763d70 msgr2=0x565406761400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:53:05.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:05.722+0000 7f943aa2b740 1 --2- 172.21.15.5:0/1133701056 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x565406763d70 0x565406761400 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f941c005940 tx=0x7f941c0317e0 comp rx=0 tx=0).stop 2024-09-10T09:53:05.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:05.722+0000 7f943aa2b740 1 -- 172.21.15.5:0/1133701056 shutdown_connections 2024-09-10T09:53:05.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:05.722+0000 7f943aa2b740 1 --2- 172.21.15.5:0/1133701056 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x565406774190 0x5654067765a0 crc :-1 s=READY pgs=219 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:05.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:05.722+0000 7f943aa2b740 1 --2- 172.21.15.5:0/1133701056 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x565406761940 0x56540676fe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:05.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:05.722+0000 7f943aa2b740 1 --2- 172.21.15.5:0/1133701056 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x565406763d70 0x565406761400 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:05.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:05.722+0000 7f943aa2b740 1 --2- 172.21.15.5:0/1133701056 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x565406758b90 0x565406760ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:05.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:05.722+0000 7f9416a00640 0 client.28327 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T09:53:05.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:05.722+0000 7f943aa2b740 1 -- 172.21.15.5:0/1133701056 >> 172.21.15.5:0/1133701056 conn(0x565406755120 msgr2=0x56540675ae00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:53:05.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:05.723+0000 7f943aa2b740 1 -- 172.21.15.5:0/1133701056 shutdown_connections 2024-09-10T09:53:05.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:05.723+0000 7f943aa2b740 1 -- 172.21.15.5:0/1133701056 wait complete. 2024-09-10T09:53:05.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:05.723+0000 7f943aa2b740 20 client.28327 trim_cache size 0 max 16384 2024-09-10T09:53:05.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:05.755+0000 7f943aa2b740 1 Processor -- start 2024-09-10T09:53:05.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:05.755+0000 7f943aa2b740 1 -- start start 2024-09-10T09:53:05.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:05.756+0000 7f943aa2b740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x565406758b90 0x56540676fc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:05.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:05.756+0000 7f943aa2b740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x565406761940 0x56540677be70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:05.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:05.756+0000 7f943aa2b740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x565406763d70 0x56540677c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:05.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:05.756+0000 7f943aa2b740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f94301b5560 con 0x565406758b90 2024-09-10T09:53:05.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:05.756+0000 7f943aa2b740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f94301b7190 con 0x565406763d70 2024-09-10T09:53:05.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:05.756+0000 7f943aa2b740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f94301b5c30 con 0x565406761940 2024-09-10T09:53:05.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:05.756+0000 7f9435e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x565406758b90 0x56540676fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:05.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:05.756+0000 7f9435e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x565406758b90 0x56540676fc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36756/0 (socket says 172.21.15.5:36756) 2024-09-10T09:53:05.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:05.756+0000 7f9435e00640 1 -- 172.21.15.5:0/2486995403 learned_addr learned my addr 172.21.15.5:0/2486995403 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:53:05.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:05.756+0000 7f9434a00640 1 --2- 172.21.15.5:0/2486995403 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x565406761940 0x56540677be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:05.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:05.756+0000 7f9435e00640 1 --2- 172.21.15.5:0/2486995403 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x565406763d70 0x56540677c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:05.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:05.756+0000 7f9435e00640 1 -- 172.21.15.5:0/2486995403 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x565406761940 msgr2=0x56540677be70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:53:05.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:05.756+0000 7f9435e00640 1 --2- 172.21.15.5:0/2486995403 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x565406761940 0x56540677be70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:05.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:05.756+0000 7f9435e00640 1 -- 172.21.15.5:0/2486995403 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x565406763d70 msgr2=0x56540677c3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:53:05.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:05.756+0000 7f9435e00640 1 --2- 172.21.15.5:0/2486995403 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x565406763d70 0x56540677c3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:05.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:05.756+0000 7f9435e00640 1 -- 172.21.15.5:0/2486995403 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565406755db0 con 0x565406758b90 2024-09-10T09:53:05.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:05.757+0000 7f9435e00640 1 --2- 172.21.15.5:0/2486995403 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x565406758b90 0x56540676fc40 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f9428010fe0 tx=0x7f942801bb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17bad99079582f43 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:53:05.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:05.757+0000 7f9421c00640 1 -- 172.21.15.5:0/2486995403 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f942801b6b0 con 0x565406758b90 2024-09-10T09:53:05.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:05.757+0000 7f9421c00640 1 -- 172.21.15.5:0/2486995403 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9428011560 con 0x565406758b90 2024-09-10T09:53:05.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:05.757+0000 7f9421c00640 1 -- 172.21.15.5:0/2486995403 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f942801b850 con 0x565406758b90 2024-09-10T09:53:05.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:05.758+0000 7f943aa2b740 1 -- 172.21.15.5:0/2486995403 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x565406758b90 msgr2=0x56540676fc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:53:05.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:05.758+0000 7f943aa2b740 1 --2- 172.21.15.5:0/2486995403 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x565406758b90 0x56540676fc40 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f9428010fe0 tx=0x7f942801bb50 comp rx=0 tx=0).stop 2024-09-10T09:53:05.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:05.758+0000 7f943aa2b740 1 -- 172.21.15.5:0/2486995403 shutdown_connections 2024-09-10T09:53:05.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:05.758+0000 7f943aa2b740 1 --2- 172.21.15.5:0/2486995403 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x565406763d70 0x56540677c3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:05.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:05.758+0000 7f943aa2b740 1 --2- 172.21.15.5:0/2486995403 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x565406761940 0x56540677be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:05.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:05.758+0000 7f943aa2b740 1 --2- 172.21.15.5:0/2486995403 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x565406758b90 0x56540676fc40 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:05.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:05.758+0000 7f943aa2b740 1 -- 172.21.15.5:0/2486995403 >> 172.21.15.5:0/2486995403 conn(0x565406755120 msgr2=0x5654067541e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:53:05.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:05.758+0000 7f943aa2b740 1 -- 172.21.15.5:0/2486995403 shutdown_connections 2024-09-10T09:53:05.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:05.758+0000 7f943aa2b740 1 -- 172.21.15.5:0/2486995403 wait complete. 2024-09-10T09:53:05.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:05.758+0000 7f943aa2b740 1 -- start start 2024-09-10T09:53:05.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:05.759+0000 7f943aa2b740 1 Processor -- start 2024-09-10T09:53:05.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:05.759+0000 7f943aa2b740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x565406758b90 0x565406768430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:05.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:05.759+0000 7f943aa2b740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x565406761940 0x565406768970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:05.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:05.759+0000 7f943aa2b740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x565406763d70 0x5654067d7e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:05.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:05.759+0000 7f943aa2b740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f94301b6a90 con 0x565406763d70 2024-09-10T09:53:05.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:05.759+0000 7f943aa2b740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f94301b63c0 con 0x565406761940 2024-09-10T09:53:05.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:05.759+0000 7f943aa2b740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f94301b3540 con 0x565406758b90 2024-09-10T09:53:05.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:05.759+0000 7f943aa2b740 10 client.0 _ll_register_callbacks cb 0x7f941c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:53:05.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:05.759+0000 7f9434a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x565406761940 0x565406768970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:05.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:05.760+0000 7f9434a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x565406761940 0x565406768970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:59180/0 (socket says 172.21.15.5:59180) 2024-09-10T09:53:05.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:05.760+0000 7f9434a00640 1 -- 172.21.15.5:0/690557527 learned_addr learned my addr 172.21.15.5:0/690557527 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:53:05.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:05.760+0000 7f9435e00640 1 --2- 172.21.15.5:0/690557527 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x565406758b90 0x565406768430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:05.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:05.760+0000 7f9435e00640 1 --2- 172.21.15.5:0/690557527 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x565406763d70 0x5654067d7e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:05.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:05.760+0000 7f9434a00640 1 -- 172.21.15.5:0/690557527 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x565406758b90 msgr2=0x565406768430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:53:05.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:05.760+0000 7f9434a00640 1 --2- 172.21.15.5:0/690557527 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x565406758b90 0x565406768430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:05.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:05.760+0000 7f9434a00640 1 -- 172.21.15.5:0/690557527 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x565406763d70 msgr2=0x5654067d7e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:53:05.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:05.760+0000 7f9434a00640 1 --2- 172.21.15.5:0/690557527 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x565406763d70 0x5654067d7e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:05.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:05.760+0000 7f9434a00640 1 -- 172.21.15.5:0/690557527 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5654067d84c0 con 0x565406761940 2024-09-10T09:53:05.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:05.760+0000 7f9435e00640 1 --2- 172.21.15.5:0/690557527 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x565406758b90 0x565406768430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:53:05.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:05.760+0000 7f9434a00640 1 --2- 172.21.15.5:0/690557527 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x565406761940 0x565406768970 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f941c003760 tx=0x7f941c02a410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3dbc01390cfc5bf server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:53:05.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:05.760+0000 7f9416a00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-09-10T09:53:05.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:05.761+0000 7f9416a00640 1 -- 172.21.15.5:0/690557527 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f941c035040 con 0x565406761940 2024-09-10T09:53:05.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:05.761+0000 7f9416a00640 1 -- 172.21.15.5:0/690557527 <== mon.2 v2:172.21.15.102:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f941c004fc0 con 0x565406761940 2024-09-10T09:53:05.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:05.761+0000 7f943aa2b740 1 -- 172.21.15.5:0/690557527 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9428011560 con 0x565406761940 2024-09-10T09:53:05.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:05.761+0000 7f9416a00640 1 -- 172.21.15.5:0/690557527 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f941c000e60 con 0x565406761940 2024-09-10T09:53:05.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:05.761+0000 7f943aa2b740 10 client.28340 fetch_fsmap learned FSMap version 37 2024-09-10T09:53:05.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:05.761+0000 7f943aa2b740 1 -- 172.21.15.5:0/690557527 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x5654067df8b0 con 0x565406761940 2024-09-10T09:53:05.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:05.761+0000 7f9416a00640 1 -- 172.21.15.5:0/690557527 <== mon.2 v2:172.21.15.102:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9428011560 con 0x565406761940 2024-09-10T09:53:05.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:05.762+0000 7f9416a00640 1 -- 172.21.15.5:0/690557527 <== mon.2 v2:172.21.15.102:3300/0 5 ==== fsmap(e 37) ==== 8390+0+0 (secure 0 0 0) 0x7f941c004aa0 con 0x565406761940 2024-09-10T09:53:05.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:05.762+0000 7f943aa2b740 10 client.28340 fetch_fsmap finished waiting for FSMap version 37 2024-09-10T09:53:05.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:05.762+0000 7f943aa2b740 10 client.28340 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T09:53:05.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:05.762+0000 7f943aa2b740 20 client.28340 populate_metadata read hostname 'smithi005' 2024-09-10T09:53:05.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:05.762+0000 7f943aa2b740 1 --2- 172.21.15.5:0/690557527 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x5654067e0060 0x5654067e2480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:05.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:05.762+0000 7f943aa2b740 4 client.28340 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T09:53:05.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:05.762+0000 7f943aa2b740 1 -- 172.21.15.5:0/690557527 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5654066c6e60 con 0x5654067e0060 2024-09-10T09:53:05.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:05.762+0000 7f9435e00640 1 --2- 172.21.15.5:0/690557527 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x5654067e0060 0x5654067e2480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:05.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:05.762+0000 7f9435e00640 1 --2- 172.21.15.5:0/690557527 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x5654067e0060 0x5654067e2480 crc :-1 s=READY pgs=220 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=61d7982e1b3333e7 server_cookie=f0d5249aaf5b6d54 in_seq=0 out_seq=0 2024-09-10T09:53:05.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:05.762+0000 7f9416a00640 10 client.28340 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T09:53:05.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:05.763+0000 7f9416a00640 1 -- 172.21.15.5:0/690557527 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5654066c6e60 con 0x5654067e0060 2024-09-10T09:53:05.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:05.763+0000 7f9416a00640 10 client.28340 handle_command_reply: tid=0 2024-09-10T09:53:05.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:05.763+0000 7f943aa2b740 1 client.28340 shutdown 2024-09-10T09:53:05.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:05.763+0000 7f943aa2b740 1 -- 172.21.15.5:0/690557527 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x565406761940 msgr2=0x565406768970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:53:05.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:05.763+0000 7f943aa2b740 1 --2- 172.21.15.5:0/690557527 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x565406761940 0x565406768970 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f941c003760 tx=0x7f941c02a410 comp rx=0 tx=0).stop 2024-09-10T09:53:05.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:05.763+0000 7f943aa2b740 1 -- 172.21.15.5:0/690557527 shutdown_connections 2024-09-10T09:53:05.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:05.763+0000 7f943aa2b740 1 --2- 172.21.15.5:0/690557527 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x5654067e0060 0x5654067e2480 crc :-1 s=READY pgs=220 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:05.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:05.763+0000 7f943aa2b740 1 --2- 172.21.15.5:0/690557527 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x565406763d70 0x5654067d7e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:05.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:05.763+0000 7f943aa2b740 1 --2- 172.21.15.5:0/690557527 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x565406761940 0x565406768970 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:05.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:05.763+0000 7f9416a00640 0 client.28340 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T09:53:05.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:05.763+0000 7f943aa2b740 1 --2- 172.21.15.5:0/690557527 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x565406758b90 0x565406768430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:05.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:05.763+0000 7f943aa2b740 1 -- 172.21.15.5:0/690557527 >> 172.21.15.5:0/690557527 conn(0x565406755120 msgr2=0x565406762720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:53:05.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:05.763+0000 7f943aa2b740 1 -- 172.21.15.5:0/690557527 shutdown_connections 2024-09-10T09:53:05.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:05.763+0000 7f943aa2b740 1 -- 172.21.15.5:0/690557527 wait complete. 2024-09-10T09:53:05.872 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-10T09:53:05.872 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "no active scrubs running", 2024-09-10T09:53:05.872 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": {} 2024-09-10T09:53:05.872 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-10T09:53:05.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:05.763+0000 7f943aa2b740 20 client.28340 trim_cache size 0 max 16384 2024-09-10T09:53:05.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:05.764+0000 7f9437e00640 1 -- 172.21.15.5:0/3916255539 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f94180a2c00 msgr2=0x7f94180a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:53:05.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:05.764+0000 7f9437e00640 1 --2- 172.21.15.5:0/3916255539 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f94180a2c00 0x7f94180a50c0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x565406756080 tx=0x7f9424013000 comp rx=0 tx=0).stop 2024-09-10T09:53:05.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:05.764+0000 7f9437e00640 1 -- 172.21.15.5:0/3916255539 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9430101620 msgr2=0x7f94301027a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:53:05.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:05.764+0000 7f9437e00640 1 --2- 172.21.15.5:0/3916255539 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9430101620 0x7f94301027a0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f942400e9f0 tx=0x7f942400eeb0 comp rx=0 tx=0).stop 2024-09-10T09:53:05.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:05.764+0000 7f9437e00640 1 -- 172.21.15.5:0/3916255539 shutdown_connections 2024-09-10T09:53:05.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:05.764+0000 7f9437e00640 1 --2- 172.21.15.5:0/3916255539 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f94180a2c00 0x7f94180a50c0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:05.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:05.764+0000 7f9437e00640 1 --2- 172.21.15.5:0/3916255539 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9430105aa0 0x7f9430103220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:05.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:05.764+0000 7f9437e00640 1 --2- 172.21.15.5:0/3916255539 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9430101fc0 0x7f9430102ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:05.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:05.764+0000 7f9437e00640 1 --2- 172.21.15.5:0/3916255539 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9430101620 0x7f94301027a0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:05.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:05.764+0000 7f9437e00640 1 -- 172.21.15.5:0/3916255539 >> 172.21.15.5:0/3916255539 conn(0x7f94300fba90 msgr2=0x7f94300fd9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:53:05.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:05.764+0000 7f9437e00640 1 -- 172.21.15.5:0/3916255539 shutdown_connections 2024-09-10T09:53:05.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:05.764+0000 7f9437e00640 1 -- 172.21.15.5:0/3916255539 wait complete. 2024-09-10T09:53:05.943 INFO:tasks.workunit.client.0.smithi005.stdout: 10 17173 24.94 MB/sec warmup 44 sec latency 17098.542 ms 2024-09-10T09:53:06.264 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:53:06 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:53:06.264 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:53:06 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:53:06.264 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:53:06 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:53:06.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:53:06 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:53:06.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:53:06 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:53:06.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:53:06 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:53:06.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:53:06 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:53:06.635 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:53:06 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:53:06.635 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:53:06 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:53:06.942 INFO:tasks.workunit.client.0.smithi005.stdout: 10 17173 24.39 MB/sec warmup 45 sec latency 18098.584 ms 2024-09-10T09:53:07.704 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:53:07 smithi005 ceph-mon[30350]: pgmap v836: 129 pgs: 129 active+clean; 503 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 256 KiB/s rd, 256 KiB/s wr, 83 op/s 2024-09-10T09:53:07.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:53:07 smithi159 ceph-mon[35513]: pgmap v836: 129 pgs: 129 active+clean; 503 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 256 KiB/s rd, 256 KiB/s wr, 83 op/s 2024-09-10T09:53:07.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:53:07 smithi102 ceph-mon[35442]: pgmap v836: 129 pgs: 129 active+clean; 503 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 256 KiB/s rd, 256 KiB/s wr, 83 op/s 2024-09-10T09:53:07.943 INFO:tasks.workunit.client.0.smithi005.stdout: 10 17173 23.86 MB/sec warmup 46 sec latency 19098.631 ms 2024-09-10T09:53:08.279 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-10T09:53:08.279 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T09:53:08.731 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:53:08.942 INFO:tasks.workunit.client.0.smithi005.stdout: 10 17522 23.63 MB/sec warmup 47 sec latency 19579.454 ms 2024-09-10T09:53:09.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:09.423+0000 7f4bf9a00640 1 -- 172.21.15.5:0/409673787 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4bf4112a00 msgr2=0x7f4bf4114df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:53:09.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:09.423+0000 7f4bf9a00640 1 --2- 172.21.15.5:0/409673787 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4bf4112a00 0x7f4bf4114df0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f4bec009ca0 tx=0x7f4bec02a8c0 comp rx=0 tx=0).stop 2024-09-10T09:53:09.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:09.424+0000 7f4bf9a00640 1 -- 172.21.15.5:0/409673787 shutdown_connections 2024-09-10T09:53:09.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:09.424+0000 7f4bf9a00640 1 --2- 172.21.15.5:0/409673787 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4bf4112a00 0x7f4bf4114df0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:09.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:09.424+0000 7f4bf9a00640 1 --2- 172.21.15.5:0/409673787 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4bf4077040 0x7f4bf41124c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:09.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:09.424+0000 7f4bf9a00640 1 --2- 172.21.15.5:0/409673787 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4bf4076720 0x7f4bf4076b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:09.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:09.424+0000 7f4bf9a00640 1 -- 172.21.15.5:0/409673787 >> 172.21.15.5:0/409673787 conn(0x7f4bf40ffa40 msgr2=0x7f4bf4101e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:53:09.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:09.424+0000 7f4bf9a00640 1 -- 172.21.15.5:0/409673787 shutdown_connections 2024-09-10T09:53:09.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:09.424+0000 7f4bf9a00640 1 -- 172.21.15.5:0/409673787 wait complete. 2024-09-10T09:53:09.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:09.426+0000 7f4bf9a00640 1 Processor -- start 2024-09-10T09:53:09.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:09.426+0000 7f4bf9a00640 1 -- start start 2024-09-10T09:53:09.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:09.427+0000 7f4bf9a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4bf4076720 0x7f4bf4111f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:09.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:09.428+0000 7f4bf9a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4bf4077040 0x7f4bf4112490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:09.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:09.428+0000 7f4bf2a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4bf4076720 0x7f4bf4111f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:09.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:09.428+0000 7f4bf9a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4bf4112a00 0x7f4bf410b190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:09.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:09.429+0000 7f4bf9a00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f4bf4118540 con 0x7f4bf4112a00 2024-09-10T09:53:09.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:09.429+0000 7f4bf2000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4bf4077040 0x7f4bf4112490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:09.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:09.429+0000 7f4bf2000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4bf4077040 0x7f4bf4112490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:49162/0 (socket says 172.21.15.5:49162) 2024-09-10T09:53:09.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:09.429+0000 7f4bf2000640 1 -- 172.21.15.5:0/140422017 learned_addr learned my addr 172.21.15.5:0/140422017 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:53:09.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:09.429+0000 7f4bf2a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4bf4076720 0x7f4bf4111f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:59204/0 (socket says 172.21.15.5:59204) 2024-09-10T09:53:09.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:09.429+0000 7f4bf3400640 1 --2- 172.21.15.5:0/140422017 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4bf4112a00 0x7f4bf410b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:09.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:09.429+0000 7f4bf9a00640 1 -- 172.21.15.5:0/140422017 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f4bf41183c0 con 0x7f4bf4076720 2024-09-10T09:53:09.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:09.429+0000 7f4bf9a00640 1 -- 172.21.15.5:0/140422017 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f4bf41186c0 con 0x7f4bf4077040 2024-09-10T09:53:09.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:09.430+0000 7f4bf2000640 1 -- 172.21.15.5:0/140422017 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4bf4076720 msgr2=0x7f4bf4111f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:53:09.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:09.430+0000 7f4bf2000640 1 --2- 172.21.15.5:0/140422017 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4bf4076720 0x7f4bf4111f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:09.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:09.430+0000 7f4bf2000640 1 -- 172.21.15.5:0/140422017 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4bf4112a00 msgr2=0x7f4bf410b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:53:09.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:09.430+0000 7f4bf2000640 1 --2- 172.21.15.5:0/140422017 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4bf4112a00 0x7f4bf410b190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:09.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:09.430+0000 7f4bf2000640 1 -- 172.21.15.5:0/140422017 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4bf410ba50 con 0x7f4bf4077040 2024-09-10T09:53:09.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:09.431+0000 7f4bf2000640 1 --2- 172.21.15.5:0/140422017 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4bf4077040 0x7f4bf4112490 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f4be000c920 tx=0x7f4be000cde0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:53:09.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:09.431+0000 7f4beb400640 1 -- 172.21.15.5:0/140422017 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4be0007c70 con 0x7f4bf4077040 2024-09-10T09:53:09.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:09.431+0000 7f4bf9a00640 1 -- 172.21.15.5:0/140422017 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4bf410bd40 con 0x7f4bf4077040 2024-09-10T09:53:09.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:09.432+0000 7f4beb400640 1 -- 172.21.15.5:0/140422017 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4be0007e10 con 0x7f4bf4077040 2024-09-10T09:53:09.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:09.432+0000 7f4beb400640 1 -- 172.21.15.5:0/140422017 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4be0018890 con 0x7f4bf4077040 2024-09-10T09:53:09.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:09.432+0000 7f4bf9a00640 1 -- 172.21.15.5:0/140422017 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4bf410fd60 con 0x7f4bf4077040 2024-09-10T09:53:09.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:09.435+0000 7f4bf9a00640 1 -- 172.21.15.5:0/140422017 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4bf4077e60 con 0x7f4bf4077040 2024-09-10T09:53:09.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:09.436+0000 7f4beb400640 1 -- 172.21.15.5:0/140422017 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f4be0018a70 con 0x7f4bf4077040 2024-09-10T09:53:09.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:09.440+0000 7f4beb400640 1 --2- 172.21.15.5:0/140422017 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f4bd80a2b80 0x7f4bd80a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:09.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:09.440+0000 7f4beb400640 1 -- 172.21.15.5:0/140422017 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f4be00cf480 con 0x7f4bf4077040 2024-09-10T09:53:09.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:09.441+0000 7f4bf2a00640 1 --2- 172.21.15.5:0/140422017 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f4bd80a2b80 0x7f4bd80a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:09.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:09.442+0000 7f4bf2a00640 1 --2- 172.21.15.5:0/140422017 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f4bd80a2b80 0x7f4bd80a5040 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f4be400bbe0 tx=0x7f4be400b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:53:09.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:09.444+0000 7f4beb400640 1 -- 172.21.15.5:0/140422017 <== mon.1 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f4be0014030 con 0x7f4bf4077040 2024-09-10T09:53:09.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:53:09 smithi005 ceph-mon[30350]: pgmap v837: 129 pgs: 129 active+clean; 503 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 40 KiB/s wr, 35 op/s 2024-09-10T09:53:09.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:09.842+0000 7f4bf9a00640 1 -- 172.21.15.5:0/140422017 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4bf41107e0 con 0x7f4bf4077040 2024-09-10T09:53:09.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:09.844+0000 7f4beb400640 1 -- 172.21.15.5:0/140422017 <== mon.1 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) ==== 94+0+11977 (secure 0 0 0) 0x7f4be009bff0 con 0x7f4bf4077040 2024-09-10T09:53:09.849 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-10T09:53:09.849 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 37 2024-09-10T09:53:09.850 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":37,"btime":"2024-09-10T09:52:27:129371+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14454,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/2970594311","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2970594311},{"type":"v1","addr":"172.21.15.5:6835","nonce":2970594311}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14514,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6839/3616502174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":3616502174},{"type":"v1","addr":"172.21.15.5:6839","nonce":3616502174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":37,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T09:35:29.626287+0000","modified":"2024-09-10T09:52:26.980498+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24542,"mds_1":24535,"mds_2":24511,"mds_3":24506,"mds_4":24443},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14520":{"gid":14520,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/2073599842","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":2073599842},{"type":"v1","addr":"172.21.15.5:6841","nonce":2073599842}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"k","rank":4,"incarnation":26,"state":"up:active","state_seq":24,"addr":"172.21.15.102:6833/2457007427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":2457007427},{"type":"v1","addr":"172.21.15.102:6833","nonce":2457007427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/356445159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":356445159},{"type":"v1","addr":"172.21.15.159:6833","nonce":356445159}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6837/2894314507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":2894314507},{"type":"v1","addr":"172.21.15.5:6837","nonce":2894314507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.102:6835/1647549111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":1647549111},{"type":"v1","addr":"172.21.15.102:6835","nonce":1647549111}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6835/952657242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":952657242},{"type":"v1","addr":"172.21.15.159:6835","nonce":952657242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":19,"addr":"172.21.15.102:6837/2689776764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":2689776764},{"type":"v1","addr":"172.21.15.102:6837","nonce":2689776764}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.159:6837/620627639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":620627639},{"type":"v1","addr":"172.21.15.159:6837","nonce":620627639}]},"join_fscid":-1,"export_targets":[0,1,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24535":{"gid":24535,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":18,"addr":"172.21.15.159:6839/2073739599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":2073739599},{"type":"v1","addr":"172.21.15.159:6839","nonce":2073739599}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24542":{"gid":24542,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.102:6839/335409749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6838","nonce":335409749},{"type":"v1","addr":"172.21.15.102:6839","nonce":335409749}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24542,"qdb_cluster":[24443,24506,24511,24535,24542]},"id":1}]} 2024-09-10T09:53:09.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:09.849+0000 7f4bf9a00640 1 -- 172.21.15.5:0/140422017 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f4bd80a2b80 msgr2=0x7f4bd80a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:53:09.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:09.849+0000 7f4bf9a00640 1 --2- 172.21.15.5:0/140422017 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f4bd80a2b80 0x7f4bd80a5040 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f4be400bbe0 tx=0x7f4be400b390 comp rx=0 tx=0).stop 2024-09-10T09:53:09.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:09.849+0000 7f4bf9a00640 1 -- 172.21.15.5:0/140422017 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4bf4077040 msgr2=0x7f4bf4112490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:53:09.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:09.849+0000 7f4bf9a00640 1 --2- 172.21.15.5:0/140422017 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4bf4077040 0x7f4bf4112490 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f4be000c920 tx=0x7f4be000cde0 comp rx=0 tx=0).stop 2024-09-10T09:53:09.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:09.849+0000 7f4bf9a00640 1 -- 172.21.15.5:0/140422017 shutdown_connections 2024-09-10T09:53:09.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:09.849+0000 7f4bf9a00640 1 --2- 172.21.15.5:0/140422017 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f4bd80a2b80 0x7f4bd80a5040 secure :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f4be400bbe0 tx=0x7f4be400b390 comp rx=0 tx=0).stop 2024-09-10T09:53:09.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:09.849+0000 7f4bf9a00640 1 --2- 172.21.15.5:0/140422017 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4bf4112a00 0x7f4bf410b190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:09.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:09.849+0000 7f4bf9a00640 1 --2- 172.21.15.5:0/140422017 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4bf4077040 0x7f4bf4112490 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:09.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:09.849+0000 7f4bf9a00640 1 --2- 172.21.15.5:0/140422017 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4bf4076720 0x7f4bf4111f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:09.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:09.849+0000 7f4bf9a00640 1 -- 172.21.15.5:0/140422017 >> 172.21.15.5:0/140422017 conn(0x7f4bf40ffa40 msgr2=0x7f4bf4101e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:53:09.851 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:53:09 smithi159 ceph-mon[35513]: pgmap v837: 129 pgs: 129 active+clean; 503 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 40 KiB/s wr, 35 op/s 2024-09-10T09:53:09.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:09.850+0000 7f4bf9a00640 1 -- 172.21.15.5:0/140422017 shutdown_connections 2024-09-10T09:53:09.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:09.850+0000 7f4bf9a00640 1 -- 172.21.15.5:0/140422017 wait complete. 2024-09-10T09:53:09.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:53:09 smithi102 ceph-mon[35442]: pgmap v837: 129 pgs: 129 active+clean; 503 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 40 KiB/s wr, 35 op/s 2024-09-10T09:53:09.947 INFO:tasks.workunit.client.0.smithi005.stdout: 10 18245 24.07 MB/sec warmup 48 sec latency 67.583 ms 2024-09-10T09:53:10.841 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-10T09:53:10.947 INFO:tasks.workunit.client.0.smithi005.stdout: 10 19174 24.51 MB/sec warmup 49 sec latency 54.914 ms 2024-09-10T09:53:11.057 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:53:10 smithi005 ceph-mon[30350]: from='client.? 172.21.15.5:0/140422017' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:53:11.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:53:10 smithi159 ceph-mon[35513]: from='client.? 172.21.15.5:0/140422017' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:53:11.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:53:10 smithi102 ceph-mon[35442]: from='client.? 172.21.15.5:0/140422017' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:53:11.434 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:53:11.947 INFO:tasks.workunit.client.0.smithi005.stdout: 10 19898 24.77 MB/sec warmup 50 sec latency 71.153 ms 2024-09-10T09:53:11.948 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:53:11 smithi005 ceph-mon[30350]: pgmap v838: 129 pgs: 129 active+clean; 505 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 138 KiB/s rd, 262 KiB/s wr, 50 op/s 2024-09-10T09:53:12.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:53:11 smithi159 ceph-mon[35513]: pgmap v838: 129 pgs: 129 active+clean; 505 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 138 KiB/s rd, 262 KiB/s wr, 50 op/s 2024-09-10T09:53:12.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:12.118+0000 7f7544200640 1 -- 172.21.15.5:0/522374946 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f753c100b30 msgr2=0x7f753c100f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:53:12.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:12.118+0000 7f7544200640 1 --2- 172.21.15.5:0/522374946 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f753c100b30 0x7f753c100f10 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f7530009900 tx=0x7f753002a470 comp rx=0 tx=0).stop 2024-09-10T09:53:12.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:12.118+0000 7f7544200640 1 -- 172.21.15.5:0/522374946 shutdown_connections 2024-09-10T09:53:12.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:12.118+0000 7f7544200640 1 --2- 172.21.15.5:0/522374946 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f753c105f40 0x7f753c1063d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:12.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:12.118+0000 7f7544200640 1 --2- 172.21.15.5:0/522374946 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f753c101450 0x7f753c105920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:12.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:12.118+0000 7f7544200640 1 --2- 172.21.15.5:0/522374946 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f753c100b30 0x7f753c100f10 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:12.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:12.118+0000 7f7544200640 1 -- 172.21.15.5:0/522374946 >> 172.21.15.5:0/522374946 conn(0x7f753c0fc7a0 msgr2=0x7f753c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:53:12.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:12.118+0000 7f7544200640 1 -- 172.21.15.5:0/522374946 shutdown_connections 2024-09-10T09:53:12.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:12.118+0000 7f7544200640 1 -- 172.21.15.5:0/522374946 wait complete. 2024-09-10T09:53:12.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:12.120+0000 7f7544200640 1 Processor -- start 2024-09-10T09:53:12.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:12.120+0000 7f7544200640 1 -- start start 2024-09-10T09:53:12.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:12.120+0000 7f7544200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f753c100b30 0x7f753c1a4750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:12.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:12.120+0000 7f7544200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f753c101450 0x7f753c1a4c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:12.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:12.121+0000 7f7541800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f753c100b30 0x7f753c1a4750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:12.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:12.121+0000 7f7544200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f753c105f40 0x7f753c19e910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:12.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:12.121+0000 7f7544200640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f753c1185a0 con 0x7f753c105f40 2024-09-10T09:53:12.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:12.121+0000 7f7544200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f753c118420 con 0x7f753c101450 2024-09-10T09:53:12.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:12.121+0000 7f7544200640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f753c118720 con 0x7f753c100b30 2024-09-10T09:53:12.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:12.121+0000 7f7541800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f753c100b30 0x7f753c1a4750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:47738/0 (socket says 172.21.15.5:47738) 2024-09-10T09:53:12.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:12.121+0000 7f7541800640 1 -- 172.21.15.5:0/2669435794 learned_addr learned my addr 172.21.15.5:0/2669435794 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:53:12.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:12.121+0000 7f7542200640 1 --2- 172.21.15.5:0/2669435794 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f753c105f40 0x7f753c19e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:12.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:12.122+0000 7f7540e00640 1 --2- 172.21.15.5:0/2669435794 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f753c101450 0x7f753c1a4c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:12.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:12.122+0000 7f7541800640 1 -- 172.21.15.5:0/2669435794 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f753c101450 msgr2=0x7f753c1a4c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:53:12.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:12.122+0000 7f7541800640 1 --2- 172.21.15.5:0/2669435794 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f753c101450 0x7f753c1a4c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:12.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:12.122+0000 7f7541800640 1 -- 172.21.15.5:0/2669435794 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f753c105f40 msgr2=0x7f753c19e910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:53:12.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:12.122+0000 7f7541800640 1 --2- 172.21.15.5:0/2669435794 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f753c105f40 0x7f753c19e910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:12.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:12.122+0000 7f7541800640 1 -- 172.21.15.5:0/2669435794 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f753c19f040 con 0x7f753c100b30 2024-09-10T09:53:12.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:12.123+0000 7f7541800640 1 --2- 172.21.15.5:0/2669435794 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f753c100b30 0x7f753c1a4750 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f7530004770 tx=0x7f7530003c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:53:12.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:12.123+0000 7f753a000640 1 -- 172.21.15.5:0/2669435794 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f753003f070 con 0x7f753c100b30 2024-09-10T09:53:12.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:12.123+0000 7f753a000640 1 -- 172.21.15.5:0/2669435794 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7530031ca0 con 0x7f753c100b30 2024-09-10T09:53:12.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:12.123+0000 7f753a000640 1 -- 172.21.15.5:0/2669435794 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7530035030 con 0x7f753c100b30 2024-09-10T09:53:12.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:12.123+0000 7f7544200640 1 -- 172.21.15.5:0/2669435794 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f753c19f2d0 con 0x7f753c100b30 2024-09-10T09:53:12.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:12.124+0000 7f7544200640 1 -- 172.21.15.5:0/2669435794 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f753c1ab620 con 0x7f753c100b30 2024-09-10T09:53:12.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:12.127+0000 7f753a000640 1 -- 172.21.15.5:0/2669435794 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f7530042070 con 0x7f753c100b30 2024-09-10T09:53:12.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:12.129+0000 7f753a000640 1 --2- 172.21.15.5:0/2669435794 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f75240a2c00 0x7f75240a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:12.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:12.130+0000 7f753a000640 1 -- 172.21.15.5:0/2669435794 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f75300ef710 con 0x7f753c100b30 2024-09-10T09:53:12.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:12.130+0000 7f7540e00640 1 --2- 172.21.15.5:0/2669435794 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f75240a2c00 0x7f75240a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:12.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:12.130+0000 7f7546e66740 1 Processor -- start 2024-09-10T09:53:12.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:12.131+0000 7f7546e66740 1 -- start start 2024-09-10T09:53:12.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:12.131+0000 7f7546e66740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e5f2589c70 0x55e5f258a050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:12.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:12.131+0000 7f7546e66740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e5f258a590 0x55e5f2594910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:12.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:12.131+0000 7f7546e66740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e5f2594e50 0x55e5f2597240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:12.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:12.131+0000 7f7546e66740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55e5f24f03e0 con 0x55e5f2589c70 2024-09-10T09:53:12.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:12.131+0000 7f7546e66740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55e5f2497d80 con 0x55e5f2594e50 2024-09-10T09:53:12.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:12.131+0000 7f7546e66740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55e5f24ec3a0 con 0x55e5f258a590 2024-09-10T09:53:12.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:12.131+0000 7f7540e00640 1 --2- 172.21.15.5:0/2669435794 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f75240a2c00 0x7f75240a50c0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f753c0ffcb0 tx=0x7f7528005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:53:12.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:12.131+0000 7f7540e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e5f258a590 0x55e5f2594910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:12.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:12.132+0000 7f7542200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e5f2594e50 0x55e5f2597240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:12.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:12.132+0000 7f7542200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e5f2594e50 0x55e5f2597240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:41446/0 (socket says 172.21.15.5:41446) 2024-09-10T09:53:12.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:12.132+0000 7f7542200640 1 -- 172.21.15.5:0/1659250833 learned_addr learned my addr 172.21.15.5:0/1659250833 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:53:12.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:12.132+0000 7f7540e00640 1 -- 172.21.15.5:0/1659250833 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e5f2594e50 msgr2=0x55e5f2597240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:53:12.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:12.132+0000 7f7540e00640 1 --2- 172.21.15.5:0/1659250833 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e5f2594e50 0x55e5f2597240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:12.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:12.132+0000 7f7540e00640 1 -- 172.21.15.5:0/1659250833 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e5f2589c70 msgr2=0x55e5f258a050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:53:12.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:12.132+0000 7f7540e00640 1 --2- 172.21.15.5:0/1659250833 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e5f2589c70 0x55e5f258a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:12.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:12.132+0000 7f7540e00640 1 -- 172.21.15.5:0/1659250833 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e5f24b6ae0 con 0x55e5f258a590 2024-09-10T09:53:12.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:12.133+0000 7f7540e00640 1 --2- 172.21.15.5:0/1659250833 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e5f258a590 0x55e5f2594910 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f7528011f30 tx=0x7f7528014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4251e27b9291f4d3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:53:12.135 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:53:11 smithi102 ceph-mon[35442]: pgmap v838: 129 pgs: 129 active+clean; 505 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 138 KiB/s rd, 262 KiB/s wr, 50 op/s 2024-09-10T09:53:12.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:12.133+0000 7f752ea00640 1 -- 172.21.15.5:0/1659250833 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7528025070 con 0x55e5f258a590 2024-09-10T09:53:12.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:12.133+0000 7f752ea00640 1 -- 172.21.15.5:0/1659250833 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7528016070 con 0x55e5f258a590 2024-09-10T09:53:12.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:12.133+0000 7f752ea00640 1 -- 172.21.15.5:0/1659250833 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7528020690 con 0x55e5f258a590 2024-09-10T09:53:12.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:12.134+0000 7f7546e66740 1 -- 172.21.15.5:0/1659250833 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e5f258a590 msgr2=0x55e5f2594910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:53:12.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:12.134+0000 7f7546e66740 1 --2- 172.21.15.5:0/1659250833 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e5f258a590 0x55e5f2594910 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f7528011f30 tx=0x7f7528014000 comp rx=0 tx=0).stop 2024-09-10T09:53:12.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:12.134+0000 7f7546e66740 1 -- 172.21.15.5:0/1659250833 shutdown_connections 2024-09-10T09:53:12.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:12.134+0000 7f7546e66740 1 --2- 172.21.15.5:0/1659250833 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e5f2594e50 0x55e5f2597240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:12.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:12.134+0000 7f7546e66740 1 --2- 172.21.15.5:0/1659250833 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e5f258a590 0x55e5f2594910 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:12.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:12.134+0000 7f7546e66740 1 --2- 172.21.15.5:0/1659250833 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e5f2589c70 0x55e5f258a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:12.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:12.134+0000 7f7546e66740 1 -- 172.21.15.5:0/1659250833 >> 172.21.15.5:0/1659250833 conn(0x55e5f2586200 msgr2=0x55e5f25865e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:53:12.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:12.135+0000 7f7546e66740 1 -- 172.21.15.5:0/1659250833 shutdown_connections 2024-09-10T09:53:12.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:12.135+0000 7f7546e66740 1 -- 172.21.15.5:0/1659250833 wait complete. 2024-09-10T09:53:12.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:12.135+0000 7f7546e66740 1 -- start start 2024-09-10T09:53:12.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:12.135+0000 7f7546e66740 1 Processor -- start 2024-09-10T09:53:12.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:12.136+0000 7f7546e66740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e5f2589c70 0x55e5f2591ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:12.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:12.136+0000 7f7546e66740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e5f258a590 0x55e5f2592430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:12.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:12.136+0000 7f7546e66740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e5f2594e50 0x55e5f2592970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:12.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:12.136+0000 7f7541800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e5f2589c70 0x55e5f2591ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:12.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:12.136+0000 7f7541800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e5f2589c70 0x55e5f2591ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38978/0 (socket says 172.21.15.5:38978) 2024-09-10T09:53:12.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:12.137+0000 7f7541800640 1 -- 172.21.15.5:0/1487922190 learned_addr learned my addr 172.21.15.5:0/1487922190 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:53:12.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:12.137+0000 7f7546e66740 1 -- 172.21.15.5:0/1487922190 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55e5f2597a20 con 0x55e5f2589c70 2024-09-10T09:53:12.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:12.137+0000 7f7546e66740 1 -- 172.21.15.5:0/1487922190 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55e5f24ab0f0 con 0x55e5f2594e50 2024-09-10T09:53:12.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:12.137+0000 7f7546e66740 1 -- 172.21.15.5:0/1487922190 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55e5f2597ba0 con 0x55e5f258a590 2024-09-10T09:53:12.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:12.137+0000 7f7542200640 1 --2- 172.21.15.5:0/1487922190 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e5f2594e50 0x55e5f2592970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:12.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:12.137+0000 7f7540e00640 1 --2- 172.21.15.5:0/1487922190 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e5f258a590 0x55e5f2592430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:12.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:12.137+0000 7f7546e66740 10 client.0 _ll_register_callbacks cb 0x55e5f2492bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:53:12.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:12.137+0000 7f7542200640 1 -- 172.21.15.5:0/1487922190 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e5f258a590 msgr2=0x55e5f2592430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:53:12.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:12.137+0000 7f7542200640 1 --2- 172.21.15.5:0/1487922190 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e5f258a590 0x55e5f2592430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:12.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:12.137+0000 7f7542200640 1 -- 172.21.15.5:0/1487922190 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e5f2589c70 msgr2=0x55e5f2591ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:53:12.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:12.137+0000 7f7542200640 1 --2- 172.21.15.5:0/1487922190 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e5f2589c70 0x55e5f2591ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:12.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:12.137+0000 7f7542200640 1 -- 172.21.15.5:0/1487922190 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e5f22613c0 con 0x55e5f2594e50 2024-09-10T09:53:12.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:12.138+0000 7f7546e66740 1 -- 172.21.15.5:0/1487922190 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7528016070 con 0x55e5f2594e50 2024-09-10T09:53:12.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:12.138+0000 7f7542200640 1 --2- 172.21.15.5:0/1487922190 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e5f2594e50 0x55e5f2592970 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f7534011ce0 tx=0x7f753400f790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=79f89046f3afcee1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:53:12.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:12.138+0000 7f7523400640 10 client.28358 ms_handle_connect on v2:172.21.15.102:3300/0 2024-09-10T09:53:12.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:12.138+0000 7f7523400640 1 -- 172.21.15.5:0/1487922190 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7534009e30 con 0x55e5f2594e50 2024-09-10T09:53:12.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:12.138+0000 7f7523400640 1 -- 172.21.15.5:0/1487922190 <== mon.2 v2:172.21.15.102:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f7534004b20 con 0x55e5f2594e50 2024-09-10T09:53:12.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:12.138+0000 7f7523400640 1 -- 172.21.15.5:0/1487922190 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f7534004d60 con 0x55e5f2594e50 2024-09-10T09:53:12.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:12.138+0000 7f7546e66740 10 client.28358 fetch_fsmap learned FSMap version 37 2024-09-10T09:53:12.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:12.138+0000 7f7546e66740 1 -- 172.21.15.5:0/1487922190 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({fsmap=37}) -- 0x55e5f25a3b60 con 0x55e5f2594e50 2024-09-10T09:53:12.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:12.139+0000 7f7523400640 1 -- 172.21.15.5:0/1487922190 <== mon.2 v2:172.21.15.102:3300/0 4 ==== fsmap(e 37) ==== 8390+0+0 (secure 0 0 0) 0x7f7534019070 con 0x55e5f2594e50 2024-09-10T09:53:12.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:12.139+0000 7f7546e66740 10 client.28358 fetch_fsmap finished waiting for FSMap version 37 2024-09-10T09:53:12.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:12.139+0000 7f7546e66740 10 client.28358 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-09-10T09:53:12.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:12.139+0000 7f7546e66740 20 client.28358 populate_metadata read hostname 'smithi005' 2024-09-10T09:53:12.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:12.139+0000 7f7546e66740 1 --2- 172.21.15.5:0/1487922190 >> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] conn(0x55e5f25a4f10 0x55e5f25a7320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:12.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:12.139+0000 7f7546e66740 4 client.28358 mds_command: new command op to 24443 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:53:12.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:12.139+0000 7f7546e66740 1 -- 172.21.15.5:0/1487922190 --> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e5f24fd950 con 0x55e5f25a4f10 2024-09-10T09:53:12.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:12.140+0000 7f7541800640 1 --2- 172.21.15.5:0/1487922190 >> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] conn(0x55e5f25a4f10 0x55e5f25a7320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:12.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:12.140+0000 7f7541800640 1 --2- 172.21.15.5:0/1487922190 >> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] conn(0x55e5f25a4f10 0x55e5f25a7320 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=3644f225f7d82bf3 server_cookie=258daad3d95e4574 in_seq=0 out_seq=0 2024-09-10T09:53:12.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:12.140+0000 7f7523400640 10 client.28358 ms_handle_connect on v2:172.21.15.102:6832/2457007427 2024-09-10T09:53:12.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:12.141+0000 7f7523400640 1 -- 172.21.15.5:0/1487922190 <== mds.4 v2:172.21.15.102:6832/2457007427 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e5f24fd950 con 0x55e5f25a4f10 2024-09-10T09:53:12.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:12.141+0000 7f7523400640 10 client.28358 handle_command_reply: tid=0 2024-09-10T09:53:12.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:12.141+0000 7f7546e66740 1 client.28358 shutdown 2024-09-10T09:53:12.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:12.141+0000 7f7546e66740 1 -- 172.21.15.5:0/1487922190 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e5f2594e50 msgr2=0x55e5f2592970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:53:12.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:12.141+0000 7f7546e66740 1 --2- 172.21.15.5:0/1487922190 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e5f2594e50 0x55e5f2592970 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f7534011ce0 tx=0x7f753400f790 comp rx=0 tx=0).stop 2024-09-10T09:53:12.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:12.141+0000 7f7546e66740 1 -- 172.21.15.5:0/1487922190 shutdown_connections 2024-09-10T09:53:12.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:12.141+0000 7f7546e66740 1 --2- 172.21.15.5:0/1487922190 >> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] conn(0x55e5f25a4f10 0x55e5f25a7320 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:12.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:12.141+0000 7f7546e66740 1 --2- 172.21.15.5:0/1487922190 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e5f2594e50 0x55e5f2592970 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:12.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:12.141+0000 7f7546e66740 1 --2- 172.21.15.5:0/1487922190 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e5f258a590 0x55e5f2592430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:12.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:12.141+0000 7f7546e66740 1 --2- 172.21.15.5:0/1487922190 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e5f2589c70 0x55e5f2591ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:12.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:12.141+0000 7f7546e66740 1 -- 172.21.15.5:0/1487922190 >> 172.21.15.5:0/1487922190 conn(0x55e5f2586200 msgr2=0x55e5f258bf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:53:12.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:12.141+0000 7f7523400640 0 client.28358 ms_handle_reset on v2:172.21.15.102:6832/2457007427 2024-09-10T09:53:12.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:12.142+0000 7f7546e66740 1 -- 172.21.15.5:0/1487922190 shutdown_connections 2024-09-10T09:53:12.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:12.142+0000 7f7546e66740 1 -- 172.21.15.5:0/1487922190 wait complete. 2024-09-10T09:53:12.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:12.142+0000 7f7546e66740 20 client.28358 trim_cache size 0 max 16384 2024-09-10T09:53:12.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:12.158+0000 7f7546e66740 1 Processor -- start 2024-09-10T09:53:12.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:12.158+0000 7f7546e66740 1 -- start start 2024-09-10T09:53:12.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:12.159+0000 7f7546e66740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e5f2589c70 0x55e5f258c720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:12.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:12.159+0000 7f7546e66740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e5f258a590 0x55e5f25acbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:12.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:12.159+0000 7f7546e66740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e5f2594e50 0x55e5f25ad120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:12.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:12.159+0000 7f7546e66740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f753c1b5420 con 0x55e5f2589c70 2024-09-10T09:53:12.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:12.159+0000 7f7546e66740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f753c1b7050 con 0x55e5f2594e50 2024-09-10T09:53:12.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:12.159+0000 7f7546e66740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f753c1b5af0 con 0x55e5f258a590 2024-09-10T09:53:12.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:12.159+0000 7f7540e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e5f258a590 0x55e5f25acbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:12.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:12.159+0000 7f7540e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e5f258a590 0x55e5f25acbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:47768/0 (socket says 172.21.15.5:47768) 2024-09-10T09:53:12.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:12.159+0000 7f7540e00640 1 -- 172.21.15.5:0/2668157203 learned_addr learned my addr 172.21.15.5:0/2668157203 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:53:12.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:12.159+0000 7f7541800640 1 --2- 172.21.15.5:0/2668157203 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e5f2589c70 0x55e5f258c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:12.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:12.159+0000 7f7542200640 1 --2- 172.21.15.5:0/2668157203 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e5f2594e50 0x55e5f25ad120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:12.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:12.160+0000 7f7540e00640 1 -- 172.21.15.5:0/2668157203 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e5f2594e50 msgr2=0x55e5f25ad120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:53:12.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:12.160+0000 7f7540e00640 1 --2- 172.21.15.5:0/2668157203 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e5f2594e50 0x55e5f25ad120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:12.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:12.160+0000 7f7540e00640 1 -- 172.21.15.5:0/2668157203 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e5f2589c70 msgr2=0x55e5f258c720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:53:12.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:12.160+0000 7f7540e00640 1 --2- 172.21.15.5:0/2668157203 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e5f2589c70 0x55e5f258c720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:12.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:12.160+0000 7f7540e00640 1 -- 172.21.15.5:0/2668157203 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e5f258a3c0 con 0x55e5f258a590 2024-09-10T09:53:12.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:12.160+0000 7f7542200640 1 --2- 172.21.15.5:0/2668157203 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e5f2594e50 0x55e5f25ad120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:53:12.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:12.160+0000 7f7540e00640 1 --2- 172.21.15.5:0/2668157203 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e5f258a590 0x55e5f25acbe0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f75280039c0 tx=0x7f7528012cc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b730d57071590d1e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:53:12.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:12.160+0000 7f752e000640 1 -- 172.21.15.5:0/2668157203 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7528030030 con 0x55e5f258a590 2024-09-10T09:53:12.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:12.160+0000 7f752e000640 1 -- 172.21.15.5:0/2668157203 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7528003b70 con 0x55e5f258a590 2024-09-10T09:53:12.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:12.160+0000 7f752e000640 1 -- 172.21.15.5:0/2668157203 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7528025040 con 0x55e5f258a590 2024-09-10T09:53:12.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:12.161+0000 7f7546e66740 1 -- 172.21.15.5:0/2668157203 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e5f258a590 msgr2=0x55e5f25acbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:53:12.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:12.161+0000 7f7546e66740 1 --2- 172.21.15.5:0/2668157203 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e5f258a590 0x55e5f25acbe0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f75280039c0 tx=0x7f7528012cc0 comp rx=0 tx=0).stop 2024-09-10T09:53:12.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:12.161+0000 7f7546e66740 1 -- 172.21.15.5:0/2668157203 shutdown_connections 2024-09-10T09:53:12.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:12.161+0000 7f7546e66740 1 --2- 172.21.15.5:0/2668157203 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e5f2594e50 0x55e5f25ad120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:12.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:12.161+0000 7f7546e66740 1 --2- 172.21.15.5:0/2668157203 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e5f258a590 0x55e5f25acbe0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:12.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:12.161+0000 7f7546e66740 1 --2- 172.21.15.5:0/2668157203 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e5f2589c70 0x55e5f258c720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:12.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:12.161+0000 7f7546e66740 1 -- 172.21.15.5:0/2668157203 >> 172.21.15.5:0/2668157203 conn(0x55e5f2586200 msgr2=0x55e5f2593f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:53:12.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:12.161+0000 7f7546e66740 1 -- 172.21.15.5:0/2668157203 shutdown_connections 2024-09-10T09:53:12.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:12.161+0000 7f7546e66740 1 -- 172.21.15.5:0/2668157203 wait complete. 2024-09-10T09:53:12.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:12.162+0000 7f7546e66740 1 -- start start 2024-09-10T09:53:12.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:12.162+0000 7f7546e66740 1 Processor -- start 2024-09-10T09:53:12.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:12.163+0000 7f7546e66740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e5f2589c70 0x55e5f258eec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:12.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:12.163+0000 7f7546e66740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e5f258a590 0x55e5f258f400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:12.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:12.163+0000 7f7546e66740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e5f2594e50 0x55e5f258f940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:12.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:12.163+0000 7f7546e66740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f753c1b6950 con 0x55e5f2594e50 2024-09-10T09:53:12.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:12.163+0000 7f7546e66740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f753c1b6280 con 0x55e5f258a590 2024-09-10T09:53:12.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:12.163+0000 7f7546e66740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f753c1b3400 con 0x55e5f2589c70 2024-09-10T09:53:12.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:12.163+0000 7f7546e66740 10 client.0 _ll_register_callbacks cb 0x7f7534009cc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:53:12.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:12.163+0000 7f7540e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e5f258a590 0x55e5f258f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:12.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:12.163+0000 7f7540e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e5f258a590 0x55e5f258f400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:41470/0 (socket says 172.21.15.5:41470) 2024-09-10T09:53:12.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:12.163+0000 7f7540e00640 1 -- 172.21.15.5:0/350578377 learned_addr learned my addr 172.21.15.5:0/350578377 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:53:12.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:12.163+0000 7f7542200640 1 --2- 172.21.15.5:0/350578377 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e5f2594e50 0x55e5f258f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:12.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:12.163+0000 7f7541800640 1 --2- 172.21.15.5:0/350578377 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e5f2589c70 0x55e5f258eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:12.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:12.163+0000 7f7541800640 1 -- 172.21.15.5:0/350578377 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e5f258a590 msgr2=0x55e5f258f400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:53:12.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:12.163+0000 7f7541800640 1 --2- 172.21.15.5:0/350578377 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e5f258a590 0x55e5f258f400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:12.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:12.163+0000 7f7541800640 1 -- 172.21.15.5:0/350578377 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e5f2594e50 msgr2=0x55e5f258f940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:53:12.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:12.163+0000 7f7541800640 1 --2- 172.21.15.5:0/350578377 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e5f2594e50 0x55e5f258f940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:12.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:12.164+0000 7f7541800640 1 -- 172.21.15.5:0/350578377 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e5f260ebb0 con 0x55e5f2589c70 2024-09-10T09:53:12.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:12.164+0000 7f7540e00640 1 --2- 172.21.15.5:0/350578377 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e5f258a590 0x55e5f258f400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:53:12.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:12.164+0000 7f7541800640 1 --2- 172.21.15.5:0/350578377 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e5f2589c70 0x55e5f258eec0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f75300c4320 tx=0x7f753002a470 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b662f6cf5cea4ec1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:53:12.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:12.164+0000 7f7523400640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-09-10T09:53:12.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:12.164+0000 7f7523400640 1 -- 172.21.15.5:0/350578377 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75300aa070 con 0x55e5f2589c70 2024-09-10T09:53:12.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:12.164+0000 7f7523400640 1 -- 172.21.15.5:0/350578377 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f75300b9ee0 con 0x55e5f2589c70 2024-09-10T09:53:12.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:12.164+0000 7f7546e66740 1 -- 172.21.15.5:0/350578377 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7528003b70 con 0x55e5f2589c70 2024-09-10T09:53:12.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:12.164+0000 7f7523400640 1 -- 172.21.15.5:0/350578377 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f75300b6050 con 0x55e5f2589c70 2024-09-10T09:53:12.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:12.164+0000 7f7546e66740 10 client.28357 fetch_fsmap learned FSMap version 37 2024-09-10T09:53:12.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:12.164+0000 7f7546e66740 1 -- 172.21.15.5:0/350578377 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55e5f260fe70 con 0x55e5f2589c70 2024-09-10T09:53:12.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:12.165+0000 7f7523400640 1 -- 172.21.15.5:0/350578377 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7528003b70 con 0x55e5f2589c70 2024-09-10T09:53:12.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:12.165+0000 7f7523400640 1 -- 172.21.15.5:0/350578377 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 37) ==== 8390+0+0 (secure 0 0 0) 0x7f75300ef040 con 0x55e5f2589c70 2024-09-10T09:53:12.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:12.165+0000 7f7546e66740 10 client.28357 fetch_fsmap finished waiting for FSMap version 37 2024-09-10T09:53:12.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:12.165+0000 7f7546e66740 10 client.28357 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-09-10T09:53:12.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:12.165+0000 7f7546e66740 20 client.28357 populate_metadata read hostname 'smithi005' 2024-09-10T09:53:12.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:12.165+0000 7f7546e66740 1 --2- 172.21.15.5:0/350578377 >> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] conn(0x55e5f2610540 0x55e5f2612930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:12.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:12.165+0000 7f7546e66740 4 client.28357 mds_command: new command op to 24443 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T09:53:12.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:12.165+0000 7f7546e66740 1 -- 172.21.15.5:0/350578377 --> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e5f24f7e60 con 0x55e5f2610540 2024-09-10T09:53:12.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:12.166+0000 7f7540e00640 1 --2- 172.21.15.5:0/350578377 >> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] conn(0x55e5f2610540 0x55e5f2612930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:12.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:12.166+0000 7f7540e00640 1 --2- 172.21.15.5:0/350578377 >> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] conn(0x55e5f2610540 0x55e5f2612930 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=9ac3761841f116e0 server_cookie=6e24377cdedeba5b in_seq=0 out_seq=0 2024-09-10T09:53:12.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:12.166+0000 7f7523400640 10 client.28357 ms_handle_connect on v2:172.21.15.102:6832/2457007427 2024-09-10T09:53:12.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:12.167+0000 7f7523400640 1 -- 172.21.15.5:0/350578377 <== mds.4 v2:172.21.15.102:6832/2457007427 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e5f24f7e60 con 0x55e5f2610540 2024-09-10T09:53:12.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:12.167+0000 7f7523400640 10 client.28357 handle_command_reply: tid=0 2024-09-10T09:53:12.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:12.167+0000 7f7546e66740 1 client.28357 shutdown 2024-09-10T09:53:12.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:12.167+0000 7f7546e66740 1 -- 172.21.15.5:0/350578377 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e5f2589c70 msgr2=0x55e5f258eec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:53:12.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:12.167+0000 7f7546e66740 1 --2- 172.21.15.5:0/350578377 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e5f2589c70 0x55e5f258eec0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f75300c4320 tx=0x7f753002a470 comp rx=0 tx=0).stop 2024-09-10T09:53:12.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:12.167+0000 7f7546e66740 1 -- 172.21.15.5:0/350578377 shutdown_connections 2024-09-10T09:53:12.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:12.167+0000 7f7546e66740 1 --2- 172.21.15.5:0/350578377 >> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] conn(0x55e5f2610540 0x55e5f2612930 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:12.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:12.167+0000 7f7523400640 0 client.28357 ms_handle_reset on v2:172.21.15.102:6832/2457007427 2024-09-10T09:53:12.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:12.167+0000 7f7546e66740 1 --2- 172.21.15.5:0/350578377 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e5f2594e50 0x55e5f258f940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:12.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:12.167+0000 7f7546e66740 1 --2- 172.21.15.5:0/350578377 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e5f258a590 0x55e5f258f400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:12.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:12.167+0000 7f7546e66740 1 --2- 172.21.15.5:0/350578377 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e5f2589c70 0x55e5f258eec0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:12.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:12.167+0000 7f7546e66740 1 -- 172.21.15.5:0/350578377 >> 172.21.15.5:0/350578377 conn(0x55e5f2586200 msgr2=0x55e5f2584b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:53:12.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:12.167+0000 7f7546e66740 1 -- 172.21.15.5:0/350578377 shutdown_connections 2024-09-10T09:53:12.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:12.167+0000 7f7546e66740 1 -- 172.21.15.5:0/350578377 wait complete. 2024-09-10T09:53:12.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:12.167+0000 7f7546e66740 20 client.28357 trim_cache size 0 max 16384 2024-09-10T09:53:12.172 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-10T09:53:12.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:12.168+0000 7f7544200640 1 -- 172.21.15.5:0/2669435794 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f75240a2c00 msgr2=0x7f75240a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:53:12.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:12.168+0000 7f7544200640 1 --2- 172.21.15.5:0/2669435794 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f75240a2c00 0x7f75240a50c0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f753c0ffcb0 tx=0x7f7528005f90 comp rx=0 tx=0).stop 2024-09-10T09:53:12.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:12.168+0000 7f7544200640 1 -- 172.21.15.5:0/2669435794 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f753c100b30 msgr2=0x7f753c1a4750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:53:12.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:12.168+0000 7f7544200640 1 --2- 172.21.15.5:0/2669435794 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f753c100b30 0x7f753c1a4750 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f7530004770 tx=0x7f7530003c10 comp rx=0 tx=0).stop 2024-09-10T09:53:12.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:12.169+0000 7f7544200640 1 -- 172.21.15.5:0/2669435794 shutdown_connections 2024-09-10T09:53:12.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:12.169+0000 7f7544200640 1 --2- 172.21.15.5:0/2669435794 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f75240a2c00 0x7f75240a50c0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:12.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:12.169+0000 7f7544200640 1 --2- 172.21.15.5:0/2669435794 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f753c105f40 0x7f753c19e910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:12.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:12.169+0000 7f7544200640 1 --2- 172.21.15.5:0/2669435794 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f753c101450 0x7f753c1a4c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:12.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:12.169+0000 7f7544200640 1 --2- 172.21.15.5:0/2669435794 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f753c100b30 0x7f753c1a4750 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:12.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:12.169+0000 7f7544200640 1 -- 172.21.15.5:0/2669435794 >> 172.21.15.5:0/2669435794 conn(0x7f753c0fc7a0 msgr2=0x7f753c10f700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:53:12.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:12.169+0000 7f7544200640 1 -- 172.21.15.5:0/2669435794 shutdown_connections 2024-09-10T09:53:12.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:12.169+0000 7f7544200640 1 -- 172.21.15.5:0/2669435794 wait complete. 2024-09-10T09:53:12.947 INFO:tasks.workunit.client.0.smithi005.stdout: 10 20744 25.48 MB/sec warmup 51 sec latency 50.732 ms 2024-09-10T09:53:13.772 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-10T09:53:13.948 INFO:tasks.workunit.client.0.smithi005.stdout: 10 21258 25.54 MB/sec warmup 52 sec latency 209.881 ms 2024-09-10T09:53:13.960 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:53:13 smithi005 ceph-mon[30350]: pgmap v839: 129 pgs: 129 active+clean; 507 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 395 KiB/s wr, 50 op/s 2024-09-10T09:53:14.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:53:13 smithi159 ceph-mon[35513]: pgmap v839: 129 pgs: 129 active+clean; 507 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 395 KiB/s wr, 50 op/s 2024-09-10T09:53:14.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:53:13 smithi102 ceph-mon[35442]: pgmap v839: 129 pgs: 129 active+clean; 507 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 395 KiB/s wr, 50 op/s 2024-09-10T09:53:14.351 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:53:14.948 INFO:tasks.workunit.client.0.smithi005.stdout: 10 21324 25.14 MB/sec warmup 53 sec latency 1209.952 ms 2024-09-10T09:53:15.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:15.755+0000 7f0bb9800640 1 -- 172.21.15.5:0/2671261583 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0bb4109120 msgr2=0x7f0bb4110ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:53:15.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:15.756+0000 7f0bb9800640 1 --2- 172.21.15.5:0/2671261583 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0bb4109120 0x7f0bb4110ca0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f0bac009be0 tx=0x7f0bac02a6f0 comp rx=0 tx=0).stop 2024-09-10T09:53:15.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:15.756+0000 7f0bb9800640 1 -- 172.21.15.5:0/2671261583 shutdown_connections 2024-09-10T09:53:15.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:15.756+0000 7f0bb9800640 1 --2- 172.21.15.5:0/2671261583 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0bb4109120 0x7f0bb4110ca0 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:15.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:15.756+0000 7f0bb9800640 1 --2- 172.21.15.5:0/2671261583 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0bb41049a0 0x7f0bb41089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:15.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:15.756+0000 7f0bb9800640 1 --2- 172.21.15.5:0/2671261583 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0bb4103ff0 0x7f0bb41043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:15.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:15.756+0000 7f0bb9800640 1 -- 172.21.15.5:0/2671261583 >> 172.21.15.5:0/2671261583 conn(0x7f0bb40ffa90 msgr2=0x7f0bb4101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:53:15.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:15.756+0000 7f0bb9800640 1 -- 172.21.15.5:0/2671261583 shutdown_connections 2024-09-10T09:53:15.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:15.756+0000 7f0bb9800640 1 -- 172.21.15.5:0/2671261583 wait complete. 2024-09-10T09:53:15.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:15.757+0000 7f0bb9800640 1 Processor -- start 2024-09-10T09:53:15.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:15.757+0000 7f0bb9800640 1 -- start start 2024-09-10T09:53:15.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:15.758+0000 7f0bb9800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0bb4103ff0 0x7f0bb419a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:15.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:15.758+0000 7f0bb9800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0bb41049a0 0x7f0bb419ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:15.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:15.758+0000 7f0bb9800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0bb4109120 0x7f0bb419eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:15.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:15.758+0000 7f0bb9800640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f0bb41141c0 con 0x7f0bb41049a0 2024-09-10T09:53:15.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:15.758+0000 7f0bb9800640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f0bb4114040 con 0x7f0bb4103ff0 2024-09-10T09:53:15.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:15.758+0000 7f0bb9800640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f0bb4114340 con 0x7f0bb4109120 2024-09-10T09:53:15.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:15.758+0000 7f0bb3400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0bb4109120 0x7f0bb419eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:15.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:15.758+0000 7f0babe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0bb41049a0 0x7f0bb419ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:15.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:15.758+0000 7f0bb3400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0bb4109120 0x7f0bb419eef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:47808/0 (socket says 172.21.15.5:47808) 2024-09-10T09:53:15.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:15.758+0000 7f0bb3400640 1 -- 172.21.15.5:0/2244307076 learned_addr learned my addr 172.21.15.5:0/2244307076 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:53:15.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:15.758+0000 7f0bb2a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0bb4103ff0 0x7f0bb419a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:15.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:15.758+0000 7f0bb3400640 1 -- 172.21.15.5:0/2244307076 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0bb4103ff0 msgr2=0x7f0bb419a620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:53:15.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:15.758+0000 7f0bb3400640 1 --2- 172.21.15.5:0/2244307076 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0bb4103ff0 0x7f0bb419a620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:15.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:15.758+0000 7f0bb3400640 1 -- 172.21.15.5:0/2244307076 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0bb41049a0 msgr2=0x7f0bb419ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:53:15.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:15.758+0000 7f0bb3400640 1 --2- 172.21.15.5:0/2244307076 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0bb41049a0 0x7f0bb419ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:15.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:15.759+0000 7f0bb3400640 1 -- 172.21.15.5:0/2244307076 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0bb419f5d0 con 0x7f0bb4109120 2024-09-10T09:53:15.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:15.759+0000 7f0bb3400640 1 --2- 172.21.15.5:0/2244307076 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0bb4109120 0x7f0bb419eef0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f0bac009b30 tx=0x7f0bac0305b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:53:15.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:15.759+0000 7f0bab400640 1 -- 172.21.15.5:0/2244307076 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bac041020 con 0x7f0bb4109120 2024-09-10T09:53:15.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:15.759+0000 7f0bab400640 1 -- 172.21.15.5:0/2244307076 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0bac003e50 con 0x7f0bb4109120 2024-09-10T09:53:15.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:15.759+0000 7f0bb9800640 1 -- 172.21.15.5:0/2244307076 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0bb419f860 con 0x7f0bb4109120 2024-09-10T09:53:15.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:15.759+0000 7f0bab400640 1 -- 172.21.15.5:0/2244307076 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bac036040 con 0x7f0bb4109120 2024-09-10T09:53:15.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:15.759+0000 7f0bb9800640 1 -- 172.21.15.5:0/2244307076 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0bb41a71a0 con 0x7f0bb4109120 2024-09-10T09:53:15.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:15.763+0000 7f0bbc3d1740 1 Processor -- start 2024-09-10T09:53:15.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:15.764+0000 7f0bab400640 1 -- 172.21.15.5:0/2244307076 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f0bac044020 con 0x7f0bb4109120 2024-09-10T09:53:15.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:15.764+0000 7f0bbc3d1740 1 -- start start 2024-09-10T09:53:15.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:15.764+0000 7f0bbc3d1740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564c20d2dc70 0x564c20d2e050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:15.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:15.764+0000 7f0bbc3d1740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564c20d2e590 0x564c20d38910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:15.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:15.764+0000 7f0bbc3d1740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564c20d38e50 0x564c20d3b240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:15.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:15.764+0000 7f0bbc3d1740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x564c20c943e0 con 0x564c20d2e590 2024-09-10T09:53:15.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:15.764+0000 7f0bbc3d1740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x564c20c3bd80 con 0x564c20d2dc70 2024-09-10T09:53:15.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:15.764+0000 7f0bbc3d1740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x564c20c903a0 con 0x564c20d38e50 2024-09-10T09:53:15.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:15.765+0000 7f0babe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564c20d2dc70 0x564c20d2e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:15.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:15.765+0000 7f0bb2a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564c20d2e590 0x564c20d38910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:15.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:15.765+0000 7f0babe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564c20d2dc70 0x564c20d2e050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:41494/0 (socket says 172.21.15.5:41494) 2024-09-10T09:53:15.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:15.765+0000 7f0babe00640 1 -- 172.21.15.5:0/967380964 learned_addr learned my addr 172.21.15.5:0/967380964 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:53:15.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:15.765+0000 7f0babe00640 1 --2- 172.21.15.5:0/967380964 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564c20d38e50 0x564c20d3b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:15.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:15.765+0000 7f0babe00640 1 -- 172.21.15.5:0/967380964 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564c20d38e50 msgr2=0x564c20d3b240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:53:15.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:15.765+0000 7f0babe00640 1 --2- 172.21.15.5:0/967380964 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564c20d38e50 0x564c20d3b240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:15.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:15.765+0000 7f0babe00640 1 -- 172.21.15.5:0/967380964 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564c20d2e590 msgr2=0x564c20d38910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:53:15.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:15.765+0000 7f0babe00640 1 --2- 172.21.15.5:0/967380964 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564c20d2e590 0x564c20d38910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:15.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:15.765+0000 7f0babe00640 1 -- 172.21.15.5:0/967380964 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564c20c5aae0 con 0x564c20d2dc70 2024-09-10T09:53:15.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:15.765+0000 7f0bab400640 1 --2- 172.21.15.5:0/2244307076 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f0b980a2b80 0x7f0b980a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:15.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:15.766+0000 7f0babe00640 1 --2- 172.21.15.5:0/967380964 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564c20d2dc70 0x564c20d2e050 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f0b9c009fd0 tx=0x7f0b9c009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=701d79f8fdf16839 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:53:15.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:15.766+0000 7f0bab400640 1 -- 172.21.15.5:0/2244307076 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f0bac0f0220 con 0x7f0bb4109120 2024-09-10T09:53:15.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:15.766+0000 7f0ba3e00640 1 -- 172.21.15.5:0/967380964 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b9c004050 con 0x564c20d2dc70 2024-09-10T09:53:15.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:15.766+0000 7f0ba3e00640 1 -- 172.21.15.5:0/967380964 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0b9c0041f0 con 0x564c20d2dc70 2024-09-10T09:53:15.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:15.766+0000 7f0bb2a00640 1 --2- 172.21.15.5:0/2244307076 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f0b980a2b80 0x7f0b980a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:15.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:15.766+0000 7f0ba3e00640 1 -- 172.21.15.5:0/967380964 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b9c00f6e0 con 0x564c20d2dc70 2024-09-10T09:53:15.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:15.767+0000 7f0bbc3d1740 1 -- 172.21.15.5:0/967380964 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564c20d2dc70 msgr2=0x564c20d2e050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:53:15.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:15.767+0000 7f0bbc3d1740 1 --2- 172.21.15.5:0/967380964 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564c20d2dc70 0x564c20d2e050 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f0b9c009fd0 tx=0x7f0b9c009260 comp rx=0 tx=0).stop 2024-09-10T09:53:15.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:15.767+0000 7f0bb2a00640 1 --2- 172.21.15.5:0/2244307076 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f0b980a2b80 0x7f0b980a5040 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x564c20d2b060 tx=0x7f0ba40090e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:53:15.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:15.768+0000 7f0bbc3d1740 1 -- 172.21.15.5:0/967380964 shutdown_connections 2024-09-10T09:53:15.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:15.768+0000 7f0bbc3d1740 1 --2- 172.21.15.5:0/967380964 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564c20d38e50 0x564c20d3b240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:15.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:15.768+0000 7f0bbc3d1740 1 --2- 172.21.15.5:0/967380964 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564c20d2e590 0x564c20d38910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:15.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:15.768+0000 7f0bbc3d1740 1 --2- 172.21.15.5:0/967380964 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564c20d2dc70 0x564c20d2e050 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:15.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:15.768+0000 7f0bbc3d1740 1 -- 172.21.15.5:0/967380964 >> 172.21.15.5:0/967380964 conn(0x564c20d2a200 msgr2=0x564c20d2a5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:53:15.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:15.768+0000 7f0bbc3d1740 1 -- 172.21.15.5:0/967380964 shutdown_connections 2024-09-10T09:53:15.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:15.768+0000 7f0bbc3d1740 1 -- 172.21.15.5:0/967380964 wait complete. 2024-09-10T09:53:15.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:15.768+0000 7f0bbc3d1740 1 -- start start 2024-09-10T09:53:15.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:15.769+0000 7f0bbc3d1740 1 Processor -- start 2024-09-10T09:53:15.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:15.769+0000 7f0bbc3d1740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564c20d2dc70 0x564c20d35d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:15.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:15.769+0000 7f0bbc3d1740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564c20d2e590 0x564c20d362a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:15.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:15.769+0000 7f0bbc3d1740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564c20d38e50 0x564c20d367e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:15.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:15.769+0000 7f0bbc3d1740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x564c20d4ca30 con 0x564c20d38e50 2024-09-10T09:53:15.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:15.769+0000 7f0bbc3d1740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x564c20c4f0f0 con 0x564c20d2dc70 2024-09-10T09:53:15.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:15.769+0000 7f0bbc3d1740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x564c20d4cbb0 con 0x564c20d2e590 2024-09-10T09:53:15.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:15.769+0000 7f0bbc3d1740 10 client.0 _ll_register_callbacks cb 0x564c20c36bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:53:15.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:15.769+0000 7f0bb2a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564c20d2dc70 0x564c20d35d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:15.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:15.769+0000 7f0bb2a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564c20d2dc70 0x564c20d35d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:41502/0 (socket says 172.21.15.5:41502) 2024-09-10T09:53:15.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:15.770+0000 7f0bb2a00640 1 -- 172.21.15.5:0/3076288837 learned_addr learned my addr 172.21.15.5:0/3076288837 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:53:15.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:15.770+0000 7f0bb3400640 1 --2- 172.21.15.5:0/3076288837 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564c20d38e50 0x564c20d367e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:15.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:15.770+0000 7f0bb2a00640 1 -- 172.21.15.5:0/3076288837 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564c20d2e590 msgr2=0x564c20d362a0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T09:53:15.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:15.770+0000 7f0bb2a00640 1 --2- 172.21.15.5:0/3076288837 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564c20d2e590 0x564c20d362a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:15.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:15.770+0000 7f0bb2a00640 1 -- 172.21.15.5:0/3076288837 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564c20d38e50 msgr2=0x564c20d367e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:53:15.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:15.770+0000 7f0bb2a00640 1 --2- 172.21.15.5:0/3076288837 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564c20d38e50 0x564c20d367e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:15.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:15.770+0000 7f0bb2a00640 1 -- 172.21.15.5:0/3076288837 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564c20a053c0 con 0x564c20d2dc70 2024-09-10T09:53:15.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:15.770+0000 7f0bb2a00640 1 --2- 172.21.15.5:0/3076288837 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564c20d2dc70 0x564c20d35d60 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f0ba40149f0 tx=0x7f0ba40046e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5079342bc8bcf1dd server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:53:15.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:15.770+0000 7f0ba0c00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-09-10T09:53:15.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:15.771+0000 7f0ba0c00640 1 -- 172.21.15.5:0/3076288837 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ba4025070 con 0x564c20d2dc70 2024-09-10T09:53:15.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:15.771+0000 7f0bbc3d1740 1 -- 172.21.15.5:0/3076288837 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0b9c0041f0 con 0x564c20d2dc70 2024-09-10T09:53:15.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:15.771+0000 7f0ba0c00640 1 -- 172.21.15.5:0/3076288837 <== mon.2 v2:172.21.15.102:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f0ba4021600 con 0x564c20d2dc70 2024-09-10T09:53:15.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:15.771+0000 7f0ba0c00640 1 -- 172.21.15.5:0/3076288837 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f0ba4021800 con 0x564c20d2dc70 2024-09-10T09:53:15.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:15.771+0000 7f0bbc3d1740 10 client.28373 fetch_fsmap learned FSMap version 37 2024-09-10T09:53:15.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:15.771+0000 7f0bbc3d1740 1 -- 172.21.15.5:0/3076288837 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x564c20d58b00 con 0x564c20d2dc70 2024-09-10T09:53:15.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:15.771+0000 7f0ba0c00640 1 -- 172.21.15.5:0/3076288837 <== mon.2 v2:172.21.15.102:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0b9c0041f0 con 0x564c20d2dc70 2024-09-10T09:53:15.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:15.772+0000 7f0ba0c00640 1 -- 172.21.15.5:0/3076288837 <== mon.2 v2:172.21.15.102:3300/0 5 ==== fsmap(e 37) ==== 8390+0+0 (secure 0 0 0) 0x7f0ba4020070 con 0x564c20d2dc70 2024-09-10T09:53:15.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:15.772+0000 7f0bbc3d1740 10 client.28373 fetch_fsmap finished waiting for FSMap version 37 2024-09-10T09:53:15.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:15.772+0000 7f0bbc3d1740 10 client.28373 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-09-10T09:53:15.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:15.772+0000 7f0bbc3d1740 20 client.28373 populate_metadata read hostname 'smithi005' 2024-09-10T09:53:15.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:15.772+0000 7f0bbc3d1740 1 --2- 172.21.15.5:0/3076288837 >> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] conn(0x564c20d59f60 0x564c20d5c400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:15.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:15.772+0000 7f0bbc3d1740 4 client.28373 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:53:15.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:15.772+0000 7f0bbc3d1740 1 -- 172.21.15.5:0/3076288837 --> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564c20ca1950 con 0x564c20d59f60 2024-09-10T09:53:15.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:15.795+0000 7f0babe00640 1 --2- 172.21.15.5:0/3076288837 >> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] conn(0x564c20d59f60 0x564c20d5c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:15.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:15.854+0000 7f0babe00640 1 --2- 172.21.15.5:0/3076288837 >> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] conn(0x564c20d59f60 0x564c20d5c400 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=feb82d28e05ce044 server_cookie=e6959d20b6aacb8d in_seq=0 out_seq=0 2024-09-10T09:53:15.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:15.854+0000 7f0ba0c00640 10 client.28373 ms_handle_connect on v2:172.21.15.102:6836/2689776764 2024-09-10T09:53:15.867 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:53:15 smithi005 ceph-mon[30350]: pgmap v840: 129 pgs: 129 active+clean; 514 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 142 op/s 2024-09-10T09:53:15.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:15.866+0000 7f0ba0c00640 1 -- 172.21.15.5:0/3076288837 <== mds.3 v2:172.21.15.102:6836/2689776764 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564c20ca1950 con 0x564c20d59f60 2024-09-10T09:53:15.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:15.866+0000 7f0ba0c00640 10 client.28373 handle_command_reply: tid=0 2024-09-10T09:53:15.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:15.866+0000 7f0bbc3d1740 1 client.28373 shutdown 2024-09-10T09:53:15.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:15.867+0000 7f0bbc3d1740 1 -- 172.21.15.5:0/3076288837 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564c20d2dc70 msgr2=0x564c20d35d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:53:15.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:15.867+0000 7f0bbc3d1740 1 --2- 172.21.15.5:0/3076288837 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564c20d2dc70 0x564c20d35d60 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f0ba40149f0 tx=0x7f0ba40046e0 comp rx=0 tx=0).stop 2024-09-10T09:53:15.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:15.867+0000 7f0bbc3d1740 1 -- 172.21.15.5:0/3076288837 shutdown_connections 2024-09-10T09:53:15.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:15.867+0000 7f0bbc3d1740 1 --2- 172.21.15.5:0/3076288837 >> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] conn(0x564c20d59f60 0x564c20d5c400 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:15.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:15.867+0000 7f0bbc3d1740 1 --2- 172.21.15.5:0/3076288837 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564c20d38e50 0x564c20d367e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:15.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:15.867+0000 7f0bbc3d1740 1 --2- 172.21.15.5:0/3076288837 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564c20d2e590 0x564c20d362a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:15.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:15.867+0000 7f0bbc3d1740 1 --2- 172.21.15.5:0/3076288837 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564c20d2dc70 0x564c20d35d60 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:15.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:15.867+0000 7f0bbc3d1740 1 -- 172.21.15.5:0/3076288837 >> 172.21.15.5:0/3076288837 conn(0x564c20d2a200 msgr2=0x564c20d2ff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:53:15.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:15.867+0000 7f0ba0c00640 0 client.28373 ms_handle_reset on v2:172.21.15.102:6836/2689776764 2024-09-10T09:53:15.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:15.867+0000 7f0bbc3d1740 1 -- 172.21.15.5:0/3076288837 shutdown_connections 2024-09-10T09:53:15.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:15.868+0000 7f0bbc3d1740 1 -- 172.21.15.5:0/3076288837 wait complete. 2024-09-10T09:53:15.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:15.868+0000 7f0bbc3d1740 20 client.28373 trim_cache size 0 max 16384 2024-09-10T09:53:15.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:15.882+0000 7f0bbc3d1740 1 Processor -- start 2024-09-10T09:53:15.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:15.883+0000 7f0bbc3d1740 1 -- start start 2024-09-10T09:53:15.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:15.883+0000 7f0bbc3d1740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564c20d2dc70 0x564c20d306d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:15.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:15.883+0000 7f0bbc3d1740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564c20d2e590 0x564c20d36630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:15.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:15.883+0000 7f0bbc3d1740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564c20d38e50 0x564c20d61cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:15.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:15.883+0000 7f0bbc3d1740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f0bb4071490 con 0x564c20d2e590 2024-09-10T09:53:15.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:15.883+0000 7f0bbc3d1740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f0bb40730c0 con 0x564c20d38e50 2024-09-10T09:53:15.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:15.883+0000 7f0bbc3d1740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f0bb4071b60 con 0x564c20d2dc70 2024-09-10T09:53:15.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:15.883+0000 7f0babe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564c20d2e590 0x564c20d36630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:15.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:15.883+0000 7f0bb2a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564c20d2dc70 0x564c20d306d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:15.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:15.883+0000 7f0bb2a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564c20d2dc70 0x564c20d306d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:47832/0 (socket says 172.21.15.5:47832) 2024-09-10T09:53:15.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:15.883+0000 7f0bb2a00640 1 -- 172.21.15.5:0/1196431431 learned_addr learned my addr 172.21.15.5:0/1196431431 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:53:15.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:15.883+0000 7f0babe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564c20d2e590 0x564c20d36630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39070/0 (socket says 172.21.15.5:39070) 2024-09-10T09:53:15.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:15.884+0000 7f0bb2a00640 1 -- 172.21.15.5:0/1196431431 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564c20d38e50 msgr2=0x564c20d61cd0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T09:53:15.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:15.884+0000 7f0bb2a00640 1 --2- 172.21.15.5:0/1196431431 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564c20d38e50 0x564c20d61cd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:15.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:15.884+0000 7f0bb2a00640 1 -- 172.21.15.5:0/1196431431 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564c20d2e590 msgr2=0x564c20d36630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:53:15.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:15.884+0000 7f0bb2a00640 1 --2- 172.21.15.5:0/1196431431 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564c20d2e590 0x564c20d36630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:15.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:15.884+0000 7f0bb2a00640 1 -- 172.21.15.5:0/1196431431 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564c20d2daa0 con 0x564c20d2dc70 2024-09-10T09:53:15.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:15.884+0000 7f0bb2a00640 1 --2- 172.21.15.5:0/1196431431 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564c20d2dc70 0x564c20d306d0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f0ba4029d00 tx=0x7f0ba4021790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a7d2c28fdc7e6550 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:53:15.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:15.884+0000 7f0ba3400640 1 -- 172.21.15.5:0/1196431431 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ba403a070 con 0x564c20d2dc70 2024-09-10T09:53:15.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:15.885+0000 7f0ba3400640 1 -- 172.21.15.5:0/1196431431 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0ba401b070 con 0x564c20d2dc70 2024-09-10T09:53:15.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:15.885+0000 7f0ba3400640 1 -- 172.21.15.5:0/1196431431 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ba4025070 con 0x564c20d2dc70 2024-09-10T09:53:15.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:15.885+0000 7f0bbc3d1740 1 -- 172.21.15.5:0/1196431431 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564c20d2dc70 msgr2=0x564c20d306d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:53:15.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:15.885+0000 7f0bbc3d1740 1 --2- 172.21.15.5:0/1196431431 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564c20d2dc70 0x564c20d306d0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f0ba4029d00 tx=0x7f0ba4021790 comp rx=0 tx=0).stop 2024-09-10T09:53:15.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:15.885+0000 7f0bbc3d1740 1 -- 172.21.15.5:0/1196431431 shutdown_connections 2024-09-10T09:53:15.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:15.885+0000 7f0bbc3d1740 1 --2- 172.21.15.5:0/1196431431 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564c20d38e50 0x564c20d61cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:15.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:15.885+0000 7f0bbc3d1740 1 --2- 172.21.15.5:0/1196431431 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564c20d2e590 0x564c20d36630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:15.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:15.885+0000 7f0bbc3d1740 1 --2- 172.21.15.5:0/1196431431 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564c20d2dc70 0x564c20d306d0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:15.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:15.885+0000 7f0bbc3d1740 1 -- 172.21.15.5:0/1196431431 >> 172.21.15.5:0/1196431431 conn(0x564c20d2a200 msgr2=0x564c20d29380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:53:15.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:15.885+0000 7f0bbc3d1740 1 -- 172.21.15.5:0/1196431431 shutdown_connections 2024-09-10T09:53:15.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:15.886+0000 7f0bbc3d1740 1 -- 172.21.15.5:0/1196431431 wait complete. 2024-09-10T09:53:15.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:15.886+0000 7f0bbc3d1740 1 -- start start 2024-09-10T09:53:15.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:15.886+0000 7f0bbc3d1740 1 Processor -- start 2024-09-10T09:53:15.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:15.887+0000 7f0bbc3d1740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564c20d2dc70 0x564c20d524f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:15.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:15.887+0000 7f0bbc3d1740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564c20d2e590 0x564c20d4d820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:15.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:15.887+0000 7f0bbc3d1740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564c20d38e50 0x564c20d4dd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:15.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:15.887+0000 7f0bbc3d1740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f0bb40729c0 con 0x564c20d38e50 2024-09-10T09:53:15.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:15.887+0000 7f0bbc3d1740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f0bb40722f0 con 0x564c20d2e590 2024-09-10T09:53:15.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:15.887+0000 7f0bbc3d1740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f0bb406f4d0 con 0x564c20d2dc70 2024-09-10T09:53:15.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:15.887+0000 7f0bbc3d1740 10 client.0 _ll_register_callbacks cb 0x7f0ba40094a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:53:15.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:15.887+0000 7f0babe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564c20d2e590 0x564c20d4d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:15.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:15.887+0000 7f0babe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564c20d2e590 0x564c20d4d820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:41530/0 (socket says 172.21.15.5:41530) 2024-09-10T09:53:15.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:15.887+0000 7f0babe00640 1 -- 172.21.15.5:0/565046675 learned_addr learned my addr 172.21.15.5:0/565046675 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:53:15.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:15.887+0000 7f0bb3400640 1 --2- 172.21.15.5:0/565046675 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564c20d38e50 0x564c20d4dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:15.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:15.887+0000 7f0babe00640 1 -- 172.21.15.5:0/565046675 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564c20d2dc70 msgr2=0x564c20d524f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T09:53:15.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:15.887+0000 7f0babe00640 1 --2- 172.21.15.5:0/565046675 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564c20d2dc70 0x564c20d524f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:15.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:15.887+0000 7f0babe00640 1 -- 172.21.15.5:0/565046675 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564c20d38e50 msgr2=0x564c20d4dd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:53:15.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:15.887+0000 7f0babe00640 1 --2- 172.21.15.5:0/565046675 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564c20d38e50 0x564c20d4dd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:15.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:15.887+0000 7f0babe00640 1 -- 172.21.15.5:0/565046675 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564c20d25770 con 0x564c20d2e590 2024-09-10T09:53:15.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:15.888+0000 7f0babe00640 1 --2- 172.21.15.5:0/565046675 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564c20d2e590 0x564c20d4d820 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f0b9c0058c0 tx=0x7f0b9c003d60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e9aa8ab3d049b434 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:53:15.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:15.888+0000 7f0ba0c00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-09-10T09:53:15.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:15.888+0000 7f0ba0c00640 1 -- 172.21.15.5:0/565046675 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b9c02e060 con 0x564c20d2e590 2024-09-10T09:53:15.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:15.888+0000 7f0bbc3d1740 1 -- 172.21.15.5:0/565046675 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0ba401b070 con 0x564c20d2e590 2024-09-10T09:53:15.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:15.888+0000 7f0ba0c00640 1 -- 172.21.15.5:0/565046675 <== mon.2 v2:172.21.15.102:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f0b9c0037f0 con 0x564c20d2e590 2024-09-10T09:53:15.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:15.888+0000 7f0ba0c00640 1 -- 172.21.15.5:0/565046675 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f0b9c035050 con 0x564c20d2e590 2024-09-10T09:53:15.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:15.888+0000 7f0bbc3d1740 10 client.28376 fetch_fsmap learned FSMap version 37 2024-09-10T09:53:15.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:15.888+0000 7f0bbc3d1740 1 -- 172.21.15.5:0/565046675 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x564c20dc4a90 con 0x564c20d2e590 2024-09-10T09:53:15.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:15.889+0000 7f0ba0c00640 1 -- 172.21.15.5:0/565046675 <== mon.2 v2:172.21.15.102:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0ba401b070 con 0x564c20d2e590 2024-09-10T09:53:15.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:15.889+0000 7f0ba0c00640 1 -- 172.21.15.5:0/565046675 <== mon.2 v2:172.21.15.102:3300/0 5 ==== fsmap(e 37) ==== 8390+0+0 (secure 0 0 0) 0x7f0b9c029540 con 0x564c20d2e590 2024-09-10T09:53:15.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:15.889+0000 7f0bbc3d1740 10 client.28376 fetch_fsmap finished waiting for FSMap version 37 2024-09-10T09:53:15.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:15.889+0000 7f0bbc3d1740 10 client.28376 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-09-10T09:53:15.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:15.889+0000 7f0bbc3d1740 20 client.28376 populate_metadata read hostname 'smithi005' 2024-09-10T09:53:15.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:15.889+0000 7f0bbc3d1740 1 --2- 172.21.15.5:0/565046675 >> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] conn(0x564c20dc5050 0x564c20dc7470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:15.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:15.889+0000 7f0bbc3d1740 4 client.28376 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T09:53:15.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:15.889+0000 7f0bbc3d1740 1 -- 172.21.15.5:0/565046675 --> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564c20c9be60 con 0x564c20dc5050 2024-09-10T09:53:15.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:15.889+0000 7f0bb2a00640 1 --2- 172.21.15.5:0/565046675 >> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] conn(0x564c20dc5050 0x564c20dc7470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:15.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:15.890+0000 7f0bb2a00640 1 --2- 172.21.15.5:0/565046675 >> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] conn(0x564c20dc5050 0x564c20dc7470 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=c2a16ce2dd64b91a server_cookie=7eccd456f7dfb075 in_seq=0 out_seq=0 2024-09-10T09:53:15.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:15.890+0000 7f0ba0c00640 10 client.28376 ms_handle_connect on v2:172.21.15.102:6836/2689776764 2024-09-10T09:53:15.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:15.890+0000 7f0ba0c00640 1 -- 172.21.15.5:0/565046675 <== mds.3 v2:172.21.15.102:6836/2689776764 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564c20c9be60 con 0x564c20dc5050 2024-09-10T09:53:15.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:15.890+0000 7f0ba0c00640 10 client.28376 handle_command_reply: tid=0 2024-09-10T09:53:15.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:15.890+0000 7f0bbc3d1740 1 client.28376 shutdown 2024-09-10T09:53:15.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:15.890+0000 7f0bbc3d1740 1 -- 172.21.15.5:0/565046675 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564c20d2e590 msgr2=0x564c20d4d820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:53:15.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:15.890+0000 7f0bbc3d1740 1 --2- 172.21.15.5:0/565046675 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564c20d2e590 0x564c20d4d820 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f0b9c0058c0 tx=0x7f0b9c003d60 comp rx=0 tx=0).stop 2024-09-10T09:53:15.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:15.890+0000 7f0bbc3d1740 1 -- 172.21.15.5:0/565046675 shutdown_connections 2024-09-10T09:53:15.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:15.890+0000 7f0bbc3d1740 1 --2- 172.21.15.5:0/565046675 >> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] conn(0x564c20dc5050 0x564c20dc7470 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:15.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:15.890+0000 7f0bbc3d1740 1 --2- 172.21.15.5:0/565046675 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564c20d38e50 0x564c20d4dd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:15.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:15.890+0000 7f0bbc3d1740 1 --2- 172.21.15.5:0/565046675 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564c20d2e590 0x564c20d4d820 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:15.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:15.890+0000 7f0bbc3d1740 1 --2- 172.21.15.5:0/565046675 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564c20d2dc70 0x564c20d524f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:15.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:15.890+0000 7f0bbc3d1740 1 -- 172.21.15.5:0/565046675 >> 172.21.15.5:0/565046675 conn(0x564c20d2a200 msgr2=0x564c20d54c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:53:15.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:15.891+0000 7f0ba0c00640 0 client.28376 ms_handle_reset on v2:172.21.15.102:6836/2689776764 2024-09-10T09:53:15.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:15.891+0000 7f0bbc3d1740 1 -- 172.21.15.5:0/565046675 shutdown_connections 2024-09-10T09:53:15.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:15.891+0000 7f0bbc3d1740 1 -- 172.21.15.5:0/565046675 wait complete. 2024-09-10T09:53:15.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:15.891+0000 7f0bbc3d1740 20 client.28376 trim_cache size 0 max 16384 2024-09-10T09:53:15.895 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-10T09:53:15.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:15.892+0000 7f0bb9800640 1 -- 172.21.15.5:0/2244307076 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f0b980a2b80 msgr2=0x7f0b980a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:53:15.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:15.892+0000 7f0bb9800640 1 --2- 172.21.15.5:0/2244307076 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f0b980a2b80 0x7f0b980a5040 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x564c20d2b060 tx=0x7f0ba40090e0 comp rx=0 tx=0).stop 2024-09-10T09:53:15.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:15.892+0000 7f0bb9800640 1 -- 172.21.15.5:0/2244307076 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0bb4109120 msgr2=0x7f0bb419eef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:53:15.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:15.892+0000 7f0bb9800640 1 --2- 172.21.15.5:0/2244307076 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0bb4109120 0x7f0bb419eef0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f0bac009b30 tx=0x7f0bac0305b0 comp rx=0 tx=0).stop 2024-09-10T09:53:15.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:15.892+0000 7f0bb9800640 1 -- 172.21.15.5:0/2244307076 shutdown_connections 2024-09-10T09:53:15.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:15.892+0000 7f0bb9800640 1 --2- 172.21.15.5:0/2244307076 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f0b980a2b80 0x7f0b980a5040 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:15.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:15.892+0000 7f0bb9800640 1 --2- 172.21.15.5:0/2244307076 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0bb4109120 0x7f0bb419eef0 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:15.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:15.892+0000 7f0bb9800640 1 --2- 172.21.15.5:0/2244307076 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0bb41049a0 0x7f0bb419ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:15.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:15.892+0000 7f0bb9800640 1 --2- 172.21.15.5:0/2244307076 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0bb4103ff0 0x7f0bb419a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:15.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:15.892+0000 7f0bb9800640 1 -- 172.21.15.5:0/2244307076 >> 172.21.15.5:0/2244307076 conn(0x7f0bb40ffa90 msgr2=0x7f0bb4101ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:53:15.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:15.892+0000 7f0bb9800640 1 -- 172.21.15.5:0/2244307076 shutdown_connections 2024-09-10T09:53:15.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:15.892+0000 7f0bb9800640 1 -- 172.21.15.5:0/2244307076 wait complete. 2024-09-10T09:53:15.948 INFO:tasks.workunit.client.0.smithi005.stdout: 10 21360 24.69 MB/sec warmup 54 sec latency 2209.049 ms 2024-09-10T09:53:16.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:53:15 smithi159 ceph-mon[35513]: pgmap v840: 129 pgs: 129 active+clean; 514 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 142 op/s 2024-09-10T09:53:16.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:53:15 smithi102 ceph-mon[35442]: pgmap v840: 129 pgs: 129 active+clean; 514 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 142 op/s 2024-09-10T09:53:16.819 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-10T09:53:16.948 INFO:tasks.workunit.client.0.smithi005.stdout: 10 21748 24.50 MB/sec warmup 55 sec latency 2605.871 ms 2024-09-10T09:53:17.274 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:53:17.883 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:53:17 smithi005 ceph-mon[30350]: pgmap v841: 129 pgs: 129 active+clean; 522 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.1 MiB/s wr, 252 op/s 2024-09-10T09:53:17.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:17.905+0000 7ff828400640 1 -- 172.21.15.5:0/4255965759 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff820109450 msgr2=0x7ff820110250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:53:17.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:17.905+0000 7ff828400640 1 --2- 172.21.15.5:0/4255965759 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff820109450 0x7ff820110250 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7ff814009860 tx=0x7ff81402a340 comp rx=0 tx=0).stop 2024-09-10T09:53:17.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:17.905+0000 7ff828400640 1 -- 172.21.15.5:0/4255965759 shutdown_connections 2024-09-10T09:53:17.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:17.905+0000 7ff828400640 1 --2- 172.21.15.5:0/4255965759 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff82011b140 0x7ff82011d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:17.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:17.905+0000 7ff828400640 1 --2- 172.21.15.5:0/4255965759 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff820109830 0x7ff820109c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:17.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:17.905+0000 7ff828400640 1 --2- 172.21.15.5:0/4255965759 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff820109450 0x7ff820110250 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:17.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:17.905+0000 7ff828400640 1 -- 172.21.15.5:0/4255965759 >> 172.21.15.5:0/4255965759 conn(0x7ff82006e140 msgr2=0x7ff82006e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:53:17.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:17.906+0000 7ff828400640 1 -- 172.21.15.5:0/4255965759 shutdown_connections 2024-09-10T09:53:17.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:17.906+0000 7ff828400640 1 -- 172.21.15.5:0/4255965759 wait complete. 2024-09-10T09:53:17.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:17.907+0000 7ff828400640 1 Processor -- start 2024-09-10T09:53:17.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:17.908+0000 7ff828400640 1 -- start start 2024-09-10T09:53:17.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:17.909+0000 7ff828400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff820109450 0x7ff820118750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:17.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:17.909+0000 7ff828400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff820109830 0x7ff8201117e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:17.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:17.909+0000 7ff828400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff82011b140 0x7ff820111d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:17.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:17.909+0000 7ff828400640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff820120da0 con 0x7ff820109830 2024-09-10T09:53:17.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:17.909+0000 7ff828400640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7ff820120c20 con 0x7ff820109450 2024-09-10T09:53:17.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:17.909+0000 7ff828400640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7ff820120f20 con 0x7ff82011b140 2024-09-10T09:53:17.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:17.909+0000 7ff825a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff820109450 0x7ff820118750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:17.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:17.909+0000 7ff825a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff820109450 0x7ff820118750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:41540/0 (socket says 172.21.15.5:41540) 2024-09-10T09:53:17.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:17.909+0000 7ff825a00640 1 -- 172.21.15.5:0/740697748 learned_addr learned my addr 172.21.15.5:0/740697748 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:53:17.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:17.909+0000 7ff826400640 1 --2- 172.21.15.5:0/740697748 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff82011b140 0x7ff820111d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:17.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:17.910+0000 7ff825000640 1 --2- 172.21.15.5:0/740697748 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff820109830 0x7ff8201117e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:17.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:17.910+0000 7ff826400640 1 -- 172.21.15.5:0/740697748 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff820109450 msgr2=0x7ff820118750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:53:17.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:17.910+0000 7ff826400640 1 --2- 172.21.15.5:0/740697748 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff820109450 0x7ff820118750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:17.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:17.910+0000 7ff826400640 1 -- 172.21.15.5:0/740697748 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff820109830 msgr2=0x7ff8201117e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:53:17.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:17.910+0000 7ff826400640 1 --2- 172.21.15.5:0/740697748 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff820109830 0x7ff8201117e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:17.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:17.910+0000 7ff826400640 1 -- 172.21.15.5:0/740697748 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff820112360 con 0x7ff82011b140 2024-09-10T09:53:17.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:17.910+0000 7ff825a00640 1 --2- 172.21.15.5:0/740697748 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff820109450 0x7ff820118750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:53:17.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:17.911+0000 7ff826400640 1 --2- 172.21.15.5:0/740697748 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff82011b140 0x7ff820111d20 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7ff81800ec50 tx=0x7ff81800c6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:53:17.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:17.911+0000 7ff81e000640 1 -- 172.21.15.5:0/740697748 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff818003b70 con 0x7ff82011b140 2024-09-10T09:53:17.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:17.912+0000 7ff81e000640 1 -- 172.21.15.5:0/740697748 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff81800ce50 con 0x7ff82011b140 2024-09-10T09:53:17.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:17.912+0000 7ff81e000640 1 -- 172.21.15.5:0/740697748 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff818010930 con 0x7ff82011b140 2024-09-10T09:53:17.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:17.913+0000 7ff828400640 1 -- 172.21.15.5:0/740697748 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff820116280 con 0x7ff82011b140 2024-09-10T09:53:17.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:17.913+0000 7ff828400640 1 -- 172.21.15.5:0/740697748 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff820116740 con 0x7ff82011b140 2024-09-10T09:53:17.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:17.919+0000 7ff82aecd740 1 Processor -- start 2024-09-10T09:53:17.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:17.920+0000 7ff81e000640 1 -- 172.21.15.5:0/740697748 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7ff818010ad0 con 0x7ff82011b140 2024-09-10T09:53:17.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:17.920+0000 7ff82aecd740 1 -- start start 2024-09-10T09:53:17.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:17.921+0000 7ff82aecd740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5650806dfc70 0x5650806e0050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:17.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:17.921+0000 7ff82aecd740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5650806e0590 0x5650806ea910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:17.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:17.921+0000 7ff82aecd740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5650806eae50 0x5650806ed240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:17.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:17.921+0000 7ff82aecd740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5650806463e0 con 0x5650806eae50 2024-09-10T09:53:17.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:17.921+0000 7ff82aecd740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x5650805edd80 con 0x5650806e0590 2024-09-10T09:53:17.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:17.921+0000 7ff82aecd740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5650806423a0 con 0x5650806dfc70 2024-09-10T09:53:17.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:17.921+0000 7ff825a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5650806e0590 0x5650806ea910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:17.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:17.921+0000 7ff825a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5650806e0590 0x5650806ea910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:41542/0 (socket says 172.21.15.5:41542) 2024-09-10T09:53:17.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:17.922+0000 7ff825a00640 1 -- 172.21.15.5:0/2889707259 learned_addr learned my addr 172.21.15.5:0/2889707259 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:53:17.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:17.922+0000 7ff825a00640 1 -- 172.21.15.5:0/2889707259 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5650806dfc70 msgr2=0x5650806e0050 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T09:53:17.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:17.922+0000 7ff825a00640 1 --2- 172.21.15.5:0/2889707259 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5650806dfc70 0x5650806e0050 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:17.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:17.922+0000 7ff825a00640 1 -- 172.21.15.5:0/2889707259 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5650806eae50 msgr2=0x5650806ed240 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T09:53:17.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:17.922+0000 7ff825a00640 1 --2- 172.21.15.5:0/2889707259 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5650806eae50 0x5650806ed240 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:17.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:17.922+0000 7ff825a00640 1 -- 172.21.15.5:0/2889707259 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56508060cae0 con 0x5650806e0590 2024-09-10T09:53:17.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:17.923+0000 7ff825a00640 1 --2- 172.21.15.5:0/2889707259 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5650806e0590 0x5650806ea910 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7ff8140303f0 tx=0x7ff814030420 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5d470db1d28d71bb server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:53:17.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:17.923+0000 7ff812a00640 1 -- 172.21.15.5:0/2889707259 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff814037040 con 0x5650806e0590 2024-09-10T09:53:17.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:17.923+0000 7ff81e000640 1 --2- 172.21.15.5:0/740697748 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7ff8080a2b80 0x7ff8080a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:17.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:17.924+0000 7ff812a00640 1 -- 172.21.15.5:0/2889707259 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff814041070 con 0x5650806e0590 2024-09-10T09:53:17.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:17.924+0000 7ff812a00640 1 -- 172.21.15.5:0/2889707259 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff814037040 con 0x5650806e0590 2024-09-10T09:53:17.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:17.925+0000 7ff82aecd740 1 -- 172.21.15.5:0/2889707259 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5650806e0590 msgr2=0x5650806ea910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:53:17.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:17.925+0000 7ff82aecd740 1 --2- 172.21.15.5:0/2889707259 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5650806e0590 0x5650806ea910 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7ff8140303f0 tx=0x7ff814030420 comp rx=0 tx=0).stop 2024-09-10T09:53:17.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:17.925+0000 7ff82aecd740 1 -- 172.21.15.5:0/2889707259 shutdown_connections 2024-09-10T09:53:17.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:17.925+0000 7ff82aecd740 1 --2- 172.21.15.5:0/2889707259 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5650806eae50 0x5650806ed240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:17.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:17.925+0000 7ff82aecd740 1 --2- 172.21.15.5:0/2889707259 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5650806e0590 0x5650806ea910 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:17.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:17.925+0000 7ff82aecd740 1 --2- 172.21.15.5:0/2889707259 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5650806dfc70 0x5650806e0050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:17.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:17.925+0000 7ff82aecd740 1 -- 172.21.15.5:0/2889707259 >> 172.21.15.5:0/2889707259 conn(0x5650806dc200 msgr2=0x5650806dc5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:53:17.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:17.926+0000 7ff81e000640 1 -- 172.21.15.5:0/740697748 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7ff8180cfae0 con 0x7ff82011b140 2024-09-10T09:53:17.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:17.926+0000 7ff825000640 1 --2- 172.21.15.5:0/740697748 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7ff8080a2b80 0x7ff8080a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:17.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:17.928+0000 7ff82aecd740 1 -- 172.21.15.5:0/2889707259 shutdown_connections 2024-09-10T09:53:17.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:17.928+0000 7ff825000640 1 --2- 172.21.15.5:0/740697748 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7ff8080a2b80 0x7ff8080a5040 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x5650806dd580 tx=0x7ff80c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:53:17.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:17.928+0000 7ff82aecd740 1 -- 172.21.15.5:0/2889707259 wait complete. 2024-09-10T09:53:17.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:17.929+0000 7ff82aecd740 1 -- start start 2024-09-10T09:53:17.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:17.929+0000 7ff82aecd740 1 Processor -- start 2024-09-10T09:53:17.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:17.931+0000 7ff82aecd740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5650806dfc70 0x5650806e7cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:17.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:17.931+0000 7ff82aecd740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5650806e0590 0x5650806e8200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:17.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:17.931+0000 7ff82aecd740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5650806eae50 0x5650806e8740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:17.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:17.931+0000 7ff82aecd740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5650806eda20 con 0x5650806e0590 2024-09-10T09:53:17.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:17.931+0000 7ff82aecd740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x5650806010f0 con 0x5650806dfc70 2024-09-10T09:53:17.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:17.931+0000 7ff82aecd740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5650806edba0 con 0x5650806eae50 2024-09-10T09:53:17.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:17.931+0000 7ff82aecd740 10 client.0 _ll_register_callbacks cb 0x5650805e8bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:53:17.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:17.931+0000 7ff825a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5650806dfc70 0x5650806e7cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:17.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:17.931+0000 7ff825a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5650806dfc70 0x5650806e7cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:41548/0 (socket says 172.21.15.5:41548) 2024-09-10T09:53:17.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:17.931+0000 7ff825a00640 1 -- 172.21.15.5:0/422590592 learned_addr learned my addr 172.21.15.5:0/422590592 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:53:17.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:17.932+0000 7ff825000640 1 --2- 172.21.15.5:0/422590592 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5650806e0590 0x5650806e8200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:17.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:17.932+0000 7ff826400640 1 --2- 172.21.15.5:0/422590592 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5650806eae50 0x5650806e8740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:17.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:17.933+0000 7ff825a00640 1 -- 172.21.15.5:0/422590592 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5650806eae50 msgr2=0x5650806e8740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:53:17.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:17.933+0000 7ff825a00640 1 --2- 172.21.15.5:0/422590592 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5650806eae50 0x5650806e8740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:17.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:17.933+0000 7ff825a00640 1 -- 172.21.15.5:0/422590592 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5650806e0590 msgr2=0x5650806e8200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:53:17.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:17.933+0000 7ff825a00640 1 --2- 172.21.15.5:0/422590592 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5650806e0590 0x5650806e8200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:17.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:17.933+0000 7ff825a00640 1 -- 172.21.15.5:0/422590592 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5650803b73c0 con 0x5650806dfc70 2024-09-10T09:53:17.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:17.933+0000 7ff825000640 1 --2- 172.21.15.5:0/422590592 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5650806e0590 0x5650806e8200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:53:17.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:17.933+0000 7ff826400640 1 --2- 172.21.15.5:0/422590592 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5650806eae50 0x5650806e8740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:53:17.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:17.935+0000 7ff825a00640 1 --2- 172.21.15.5:0/422590592 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5650806dfc70 0x5650806e7cc0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7ff8140303c0 tx=0x7ff81402a9c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=37d286615d9f793e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:53:17.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:17.936+0000 7ff807400640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-09-10T09:53:17.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:17.936+0000 7ff807400640 1 -- 172.21.15.5:0/422590592 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff814049070 con 0x5650806dfc70 2024-09-10T09:53:17.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:17.936+0000 7ff82aecd740 1 -- 172.21.15.5:0/422590592 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff814041070 con 0x5650806dfc70 2024-09-10T09:53:17.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:17.937+0000 7ff807400640 1 -- 172.21.15.5:0/422590592 <== mon.2 v2:172.21.15.102:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7ff8140052e0 con 0x5650806dfc70 2024-09-10T09:53:17.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:17.937+0000 7ff807400640 1 -- 172.21.15.5:0/422590592 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7ff814055050 con 0x5650806dfc70 2024-09-10T09:53:17.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:17.937+0000 7ff82aecd740 10 client.28388 fetch_fsmap learned FSMap version 37 2024-09-10T09:53:17.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:17.937+0000 7ff82aecd740 1 -- 172.21.15.5:0/422590592 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x5650806f9a40 con 0x5650806dfc70 2024-09-10T09:53:17.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:17.938+0000 7ff807400640 1 -- 172.21.15.5:0/422590592 <== mon.2 v2:172.21.15.102:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff814041070 con 0x5650806dfc70 2024-09-10T09:53:17.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:17.938+0000 7ff807400640 1 -- 172.21.15.5:0/422590592 <== mon.2 v2:172.21.15.102:3300/0 5 ==== fsmap(e 37) ==== 8390+0+0 (secure 0 0 0) 0x7ff8140035f0 con 0x5650806dfc70 2024-09-10T09:53:17.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:17.939+0000 7ff82aecd740 10 client.28388 fetch_fsmap finished waiting for FSMap version 37 2024-09-10T09:53:17.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:17.939+0000 7ff82aecd740 10 client.28388 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-10T09:53:17.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:17.939+0000 7ff82aecd740 20 client.28388 populate_metadata read hostname 'smithi005' 2024-09-10T09:53:17.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:17.939+0000 7ff82aecd740 1 --2- 172.21.15.5:0/422590592 >> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] conn(0x5650806fad90 0x5650806fd180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:17.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:17.939+0000 7ff82aecd740 4 client.28388 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:53:17.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:17.939+0000 7ff82aecd740 1 -- 172.21.15.5:0/422590592 --> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x565080653950 con 0x5650806fad90 2024-09-10T09:53:17.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:17.940+0000 7ff825000640 1 --2- 172.21.15.5:0/422590592 >> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] conn(0x5650806fad90 0x5650806fd180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:17.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:17.941+0000 7ff807400640 10 client.28388 ms_handle_connect on v2:172.21.15.159:6836/620627639 2024-09-10T09:53:17.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:17.943+0000 7ff825000640 1 --2- 172.21.15.5:0/422590592 >> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] conn(0x5650806fad90 0x5650806fd180 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9694c28fa69253a1 server_cookie=dc24de028046e686 in_seq=0 out_seq=0 2024-09-10T09:53:17.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:17.946+0000 7ff807400640 1 -- 172.21.15.5:0/422590592 <== mds.2 v2:172.21.15.159:6836/620627639 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x565080653950 con 0x5650806fad90 2024-09-10T09:53:17.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:17.946+0000 7ff807400640 10 client.28388 handle_command_reply: tid=0 2024-09-10T09:53:17.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:17.946+0000 7ff82aecd740 1 client.28388 shutdown 2024-09-10T09:53:17.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:17.947+0000 7ff82aecd740 1 -- 172.21.15.5:0/422590592 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5650806dfc70 msgr2=0x5650806e7cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:53:17.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:17.947+0000 7ff82aecd740 1 --2- 172.21.15.5:0/422590592 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5650806dfc70 0x5650806e7cc0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7ff8140303c0 tx=0x7ff81402a9c0 comp rx=0 tx=0).stop 2024-09-10T09:53:17.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:17.947+0000 7ff82aecd740 1 -- 172.21.15.5:0/422590592 shutdown_connections 2024-09-10T09:53:17.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:17.947+0000 7ff82aecd740 1 --2- 172.21.15.5:0/422590592 >> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] conn(0x5650806fad90 0x5650806fd180 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:17.948 INFO:tasks.workunit.client.0.smithi005.stdout: 10 22404 24.61 MB/sec warmup 56 sec latency 95.721 ms 2024-09-10T09:53:17.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:17.947+0000 7ff82aecd740 1 --2- 172.21.15.5:0/422590592 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5650806eae50 0x5650806e8740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:17.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:17.947+0000 7ff82aecd740 1 --2- 172.21.15.5:0/422590592 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5650806e0590 0x5650806e8200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:17.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:17.947+0000 7ff82aecd740 1 --2- 172.21.15.5:0/422590592 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5650806dfc70 0x5650806e7cc0 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:17.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:17.947+0000 7ff82aecd740 1 -- 172.21.15.5:0/422590592 >> 172.21.15.5:0/422590592 conn(0x5650806dc200 msgr2=0x5650806db380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:53:17.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:17.947+0000 7ff807400640 0 client.28388 ms_handle_reset on v2:172.21.15.159:6836/620627639 2024-09-10T09:53:17.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:17.947+0000 7ff82aecd740 1 -- 172.21.15.5:0/422590592 shutdown_connections 2024-09-10T09:53:17.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:17.948+0000 7ff82aecd740 1 -- 172.21.15.5:0/422590592 wait complete. 2024-09-10T09:53:17.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:17.948+0000 7ff82aecd740 20 client.28388 trim_cache size 0 max 16384 2024-09-10T09:53:17.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:17.973+0000 7ff82aecd740 1 Processor -- start 2024-09-10T09:53:17.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:17.975+0000 7ff82aecd740 1 -- start start 2024-09-10T09:53:17.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:17.975+0000 7ff82aecd740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5650806dfc70 0x5650806e6590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:17.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:17.975+0000 7ff82aecd740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5650806e0590 0x5650806ffdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:17.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:17.975+0000 7ff82aecd740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5650806eae50 0x565080700300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:17.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:17.975+0000 7ff82aecd740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff8201c64c0 con 0x5650806e0590 2024-09-10T09:53:17.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:17.975+0000 7ff82aecd740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7ff8201c80f0 con 0x5650806eae50 2024-09-10T09:53:17.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:17.975+0000 7ff82aecd740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7ff8201c6b90 con 0x5650806dfc70 2024-09-10T09:53:17.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:17.976+0000 7ff826400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5650806eae50 0x565080700300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:17.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:17.976+0000 7ff825a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5650806dfc70 0x5650806e6590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:17.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:17.976+0000 7ff826400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5650806eae50 0x565080700300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:41552/0 (socket says 172.21.15.5:41552) 2024-09-10T09:53:17.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:17.976+0000 7ff826400640 1 -- 172.21.15.5:0/3138135049 learned_addr learned my addr 172.21.15.5:0/3138135049 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:53:17.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:17.977+0000 7ff825000640 1 --2- 172.21.15.5:0/3138135049 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5650806e0590 0x5650806ffdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:17.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:17.978+0000 7ff825a00640 1 -- 172.21.15.5:0/3138135049 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5650806eae50 msgr2=0x565080700300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:53:17.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:17.978+0000 7ff825a00640 1 --2- 172.21.15.5:0/3138135049 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5650806eae50 0x565080700300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:17.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:17.978+0000 7ff825a00640 1 -- 172.21.15.5:0/3138135049 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5650806e0590 msgr2=0x5650806ffdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:53:17.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:17.978+0000 7ff825a00640 1 --2- 172.21.15.5:0/3138135049 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5650806e0590 0x5650806ffdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:17.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:17.978+0000 7ff825a00640 1 -- 172.21.15.5:0/3138135049 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5650806e6c70 con 0x5650806dfc70 2024-09-10T09:53:17.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:17.978+0000 7ff826400640 1 --2- 172.21.15.5:0/3138135049 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5650806eae50 0x565080700300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:53:17.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:17.979+0000 7ff825a00640 1 --2- 172.21.15.5:0/3138135049 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5650806dfc70 0x5650806e6590 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7ff8140039f0 tx=0x7ff8140096f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5b70e89eb5adff5b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:53:17.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:17.979+0000 7ff812000640 1 -- 172.21.15.5:0/3138135049 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff814000e60 con 0x5650806dfc70 2024-09-10T09:53:17.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:17.979+0000 7ff812000640 1 -- 172.21.15.5:0/3138135049 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff814041040 con 0x5650806dfc70 2024-09-10T09:53:17.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:17.979+0000 7ff812000640 1 -- 172.21.15.5:0/3138135049 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff814049070 con 0x5650806dfc70 2024-09-10T09:53:17.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:17.980+0000 7ff82aecd740 1 -- 172.21.15.5:0/3138135049 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5650806dfc70 msgr2=0x5650806e6590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:53:17.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:17.980+0000 7ff82aecd740 1 --2- 172.21.15.5:0/3138135049 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5650806dfc70 0x5650806e6590 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7ff8140039f0 tx=0x7ff8140096f0 comp rx=0 tx=0).stop 2024-09-10T09:53:17.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:17.980+0000 7ff82aecd740 1 -- 172.21.15.5:0/3138135049 shutdown_connections 2024-09-10T09:53:17.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:17.980+0000 7ff82aecd740 1 --2- 172.21.15.5:0/3138135049 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5650806eae50 0x565080700300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:17.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:17.980+0000 7ff82aecd740 1 --2- 172.21.15.5:0/3138135049 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5650806e0590 0x5650806ffdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:17.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:17.980+0000 7ff82aecd740 1 --2- 172.21.15.5:0/3138135049 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5650806dfc70 0x5650806e6590 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:17.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:17.980+0000 7ff82aecd740 1 -- 172.21.15.5:0/3138135049 >> 172.21.15.5:0/3138135049 conn(0x5650806dc200 msgr2=0x5650806f6670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:53:17.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:17.981+0000 7ff82aecd740 1 -- 172.21.15.5:0/3138135049 shutdown_connections 2024-09-10T09:53:17.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:17.981+0000 7ff82aecd740 1 -- 172.21.15.5:0/3138135049 wait complete. 2024-09-10T09:53:17.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:17.981+0000 7ff82aecd740 1 -- start start 2024-09-10T09:53:17.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:17.981+0000 7ff82aecd740 1 Processor -- start 2024-09-10T09:53:17.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:17.982+0000 7ff82aecd740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5650806dfc70 0x565080700500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:17.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:17.982+0000 7ff82aecd740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5650806e0590 0x5650806fd850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:17.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:17.982+0000 7ff82aecd740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5650806eae50 0x5650806fdd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:17.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:17.982+0000 7ff82aecd740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff8201c79f0 con 0x5650806dfc70 2024-09-10T09:53:17.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:17.982+0000 7ff82aecd740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7ff8201c7320 con 0x5650806eae50 2024-09-10T09:53:17.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:17.982+0000 7ff82aecd740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7ff8201c44a0 con 0x5650806e0590 2024-09-10T09:53:17.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:17.982+0000 7ff82aecd740 10 client.0 _ll_register_callbacks cb 0x7ff8180093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:53:17.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:17.984+0000 7ff826400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5650806eae50 0x5650806fdd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:17.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:17.984+0000 7ff826400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5650806eae50 0x5650806fdd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:41566/0 (socket says 172.21.15.5:41566) 2024-09-10T09:53:17.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:17.984+0000 7ff826400640 1 -- 172.21.15.5:0/1029745985 learned_addr learned my addr 172.21.15.5:0/1029745985 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:53:17.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:17.984+0000 7ff825000640 1 --2- 172.21.15.5:0/1029745985 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5650806e0590 0x5650806fd850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:17.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:17.984+0000 7ff825a00640 1 --2- 172.21.15.5:0/1029745985 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5650806dfc70 0x565080700500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:17.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:17.985+0000 7ff826400640 1 -- 172.21.15.5:0/1029745985 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5650806e0590 msgr2=0x5650806fd850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:53:17.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:17.985+0000 7ff826400640 1 --2- 172.21.15.5:0/1029745985 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5650806e0590 0x5650806fd850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:17.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:17.985+0000 7ff826400640 1 -- 172.21.15.5:0/1029745985 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5650806dfc70 msgr2=0x565080700500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:53:17.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:17.985+0000 7ff826400640 1 --2- 172.21.15.5:0/1029745985 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5650806dfc70 0x565080700500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:17.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:17.985+0000 7ff826400640 1 -- 172.21.15.5:0/1029745985 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5650806fe5c0 con 0x5650806eae50 2024-09-10T09:53:17.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:17.985+0000 7ff825a00640 1 --2- 172.21.15.5:0/1029745985 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5650806dfc70 0x565080700500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T09:53:17.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:17.986+0000 7ff826400640 1 --2- 172.21.15.5:0/1029745985 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5650806eae50 0x5650806fdd90 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7ff8180a4690 tx=0x7ff81809b970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6b91beadacb59d3c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:53:17.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:17.986+0000 7ff807400640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-09-10T09:53:17.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:17.986+0000 7ff807400640 1 -- 172.21.15.5:0/1029745985 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8180a0300 con 0x5650806eae50 2024-09-10T09:53:17.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:17.986+0000 7ff807400640 1 -- 172.21.15.5:0/1029745985 <== mon.2 v2:172.21.15.102:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7ff8180a4730 con 0x5650806eae50 2024-09-10T09:53:17.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:17.987+0000 7ff82aecd740 1 -- 172.21.15.5:0/1029745985 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff814041040 con 0x5650806eae50 2024-09-10T09:53:17.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:17.987+0000 7ff807400640 1 -- 172.21.15.5:0/1029745985 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7ff8180a4bc0 con 0x5650806eae50 2024-09-10T09:53:17.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:17.987+0000 7ff82aecd740 10 client.28394 fetch_fsmap learned FSMap version 37 2024-09-10T09:53:17.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:17.987+0000 7ff82aecd740 1 -- 172.21.15.5:0/1029745985 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x565080765720 con 0x5650806eae50 2024-09-10T09:53:17.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:17.987+0000 7ff807400640 1 -- 172.21.15.5:0/1029745985 <== mon.2 v2:172.21.15.102:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff814041040 con 0x5650806eae50 2024-09-10T09:53:17.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:17.988+0000 7ff807400640 1 -- 172.21.15.5:0/1029745985 <== mon.2 v2:172.21.15.102:3300/0 5 ==== fsmap(e 37) ==== 8390+0+0 (secure 0 0 0) 0x7ff8180b9070 con 0x5650806eae50 2024-09-10T09:53:17.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:17.988+0000 7ff82aecd740 10 client.28394 fetch_fsmap finished waiting for FSMap version 37 2024-09-10T09:53:17.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:17.988+0000 7ff82aecd740 10 client.28394 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-10T09:53:17.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:17.988+0000 7ff82aecd740 20 client.28394 populate_metadata read hostname 'smithi005' 2024-09-10T09:53:17.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:17.988+0000 7ff82aecd740 1 --2- 172.21.15.5:0/1029745985 >> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] conn(0x565080765f50 0x565080768370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:17.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:17.988+0000 7ff82aecd740 4 client.28394 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T09:53:17.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:17.988+0000 7ff82aecd740 1 -- 172.21.15.5:0/1029745985 --> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56508064de60 con 0x565080765f50 2024-09-10T09:53:17.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:17.989+0000 7ff825a00640 1 --2- 172.21.15.5:0/1029745985 >> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] conn(0x565080765f50 0x565080768370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:17.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:17.989+0000 7ff825a00640 1 --2- 172.21.15.5:0/1029745985 >> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] conn(0x565080765f50 0x565080768370 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e129b8a26f8dfdec server_cookie=e214ade1eef8e51 in_seq=0 out_seq=0 2024-09-10T09:53:17.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:17.989+0000 7ff807400640 10 client.28394 ms_handle_connect on v2:172.21.15.159:6836/620627639 2024-09-10T09:53:17.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:17.990+0000 7ff807400640 1 -- 172.21.15.5:0/1029745985 <== mds.2 v2:172.21.15.159:6836/620627639 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56508064de60 con 0x565080765f50 2024-09-10T09:53:17.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:17.990+0000 7ff807400640 10 client.28394 handle_command_reply: tid=0 2024-09-10T09:53:17.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:17.990+0000 7ff82aecd740 1 client.28394 shutdown 2024-09-10T09:53:17.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:17.991+0000 7ff82aecd740 1 -- 172.21.15.5:0/1029745985 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5650806eae50 msgr2=0x5650806fdd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:53:17.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:17.991+0000 7ff82aecd740 1 --2- 172.21.15.5:0/1029745985 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5650806eae50 0x5650806fdd90 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7ff8180a4690 tx=0x7ff81809b970 comp rx=0 tx=0).stop 2024-09-10T09:53:17.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:17.993+0000 7ff82aecd740 1 -- 172.21.15.5:0/1029745985 shutdown_connections 2024-09-10T09:53:17.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:17.993+0000 7ff82aecd740 1 --2- 172.21.15.5:0/1029745985 >> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] conn(0x565080765f50 0x565080768370 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:17.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:17.993+0000 7ff82aecd740 1 --2- 172.21.15.5:0/1029745985 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5650806eae50 0x5650806fdd90 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:17.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:17.993+0000 7ff82aecd740 1 --2- 172.21.15.5:0/1029745985 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5650806e0590 0x5650806fd850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:17.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:17.993+0000 7ff807400640 0 client.28394 ms_handle_reset on v2:172.21.15.159:6836/620627639 2024-09-10T09:53:17.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:17.993+0000 7ff82aecd740 1 --2- 172.21.15.5:0/1029745985 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5650806dfc70 0x565080700500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:17.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:17.993+0000 7ff82aecd740 1 -- 172.21.15.5:0/1029745985 >> 172.21.15.5:0/1029745985 conn(0x5650806dc200 msgr2=0x5650806e1ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:53:17.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:17.995+0000 7ff82aecd740 1 -- 172.21.15.5:0/1029745985 shutdown_connections 2024-09-10T09:53:17.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:17.995+0000 7ff82aecd740 1 -- 172.21.15.5:0/1029745985 wait complete. 2024-09-10T09:53:17.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:17.995+0000 7ff82aecd740 20 client.28394 trim_cache size 0 max 16384 2024-09-10T09:53:17.997 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-10T09:53:17.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:17.997+0000 7ff828400640 1 -- 172.21.15.5:0/740697748 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7ff8080a2b80 msgr2=0x7ff8080a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:53:17.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:17.997+0000 7ff828400640 1 --2- 172.21.15.5:0/740697748 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7ff8080a2b80 0x7ff8080a5040 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x5650806dd580 tx=0x7ff80c009260 comp rx=0 tx=0).stop 2024-09-10T09:53:17.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:17.997+0000 7ff828400640 1 -- 172.21.15.5:0/740697748 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff82011b140 msgr2=0x7ff820111d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:53:17.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:17.997+0000 7ff828400640 1 --2- 172.21.15.5:0/740697748 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff82011b140 0x7ff820111d20 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7ff81800ec50 tx=0x7ff81800c6a0 comp rx=0 tx=0).stop 2024-09-10T09:53:17.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:17.997+0000 7ff828400640 1 -- 172.21.15.5:0/740697748 shutdown_connections 2024-09-10T09:53:17.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:17.997+0000 7ff828400640 1 --2- 172.21.15.5:0/740697748 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7ff8080a2b80 0x7ff8080a5040 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:17.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:17.997+0000 7ff828400640 1 --2- 172.21.15.5:0/740697748 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff82011b140 0x7ff820111d20 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:17.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:17.997+0000 7ff828400640 1 --2- 172.21.15.5:0/740697748 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff820109830 0x7ff8201117e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:17.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:17.997+0000 7ff828400640 1 --2- 172.21.15.5:0/740697748 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff820109450 0x7ff820118750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:17.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:17.997+0000 7ff828400640 1 -- 172.21.15.5:0/740697748 >> 172.21.15.5:0/740697748 conn(0x7ff82006e140 msgr2=0x7ff82011bea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:53:17.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:17.997+0000 7ff828400640 1 -- 172.21.15.5:0/740697748 shutdown_connections 2024-09-10T09:53:17.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:17.998+0000 7ff828400640 1 -- 172.21.15.5:0/740697748 wait complete. 2024-09-10T09:53:18.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:53:17 smithi159 ceph-mon[35513]: pgmap v841: 129 pgs: 129 active+clean; 522 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.1 MiB/s wr, 252 op/s 2024-09-10T09:53:18.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:53:17 smithi102 ceph-mon[35442]: pgmap v841: 129 pgs: 129 active+clean; 522 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.1 MiB/s wr, 252 op/s 2024-09-10T09:53:18.949 INFO:tasks.workunit.client.0.smithi005.stdout: 10 22962 24.69 MB/sec warmup 57 sec latency 174.267 ms 2024-09-10T09:53:19.949 INFO:tasks.workunit.client.0.smithi005.stdout: 10 22979 24.27 MB/sec warmup 58 sec latency 1174.318 ms 2024-09-10T09:53:20.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:53:19 smithi005 ceph-mon[30350]: pgmap v842: 129 pgs: 129 active+clean; 525 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.7 MiB/s wr, 296 op/s 2024-09-10T09:53:20.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:53:19 smithi159 ceph-mon[35513]: pgmap v842: 129 pgs: 129 active+clean; 525 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.7 MiB/s wr, 296 op/s 2024-09-10T09:53:20.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:53:19 smithi102 ceph-mon[35442]: pgmap v842: 129 pgs: 129 active+clean; 525 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.7 MiB/s wr, 296 op/s 2024-09-10T09:53:20.949 INFO:tasks.workunit.client.0.smithi005.stdout: 10 22979 23.86 MB/sec warmup 59 sec latency 2174.360 ms 2024-09-10T09:53:21.636 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-10T09:53:21.949 INFO:tasks.workunit.client.0.smithi005.stdout: 10 23371 23.84 MB/sec warmup 60 sec latency 2691.814 ms 2024-09-10T09:53:22.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:53:21 smithi005 ceph-mon[30350]: pgmap v843: 129 pgs: 129 active+clean; 533 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 4.1 MiB/s wr, 422 op/s 2024-09-10T09:53:22.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:53:21 smithi159 ceph-mon[35513]: pgmap v843: 129 pgs: 129 active+clean; 533 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 4.1 MiB/s wr, 422 op/s 2024-09-10T09:53:22.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:53:21 smithi102 ceph-mon[35442]: pgmap v843: 129 pgs: 129 active+clean; 533 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 4.1 MiB/s wr, 422 op/s 2024-09-10T09:53:22.148 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:53:22.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:22.812+0000 7fd9aac00640 1 -- 172.21.15.5:0/2175518141 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd9a41034b0 msgr2=0x7fd9a410b9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:53:22.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:22.812+0000 7fd9aac00640 1 --2- 172.21.15.5:0/2175518141 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd9a41034b0 0x7fd9a410b9a0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fd990009900 tx=0x7fd99002a490 comp rx=0 tx=0).stop 2024-09-10T09:53:22.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:22.812+0000 7fd9aac00640 1 -- 172.21.15.5:0/2175518141 shutdown_connections 2024-09-10T09:53:22.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:22.812+0000 7fd9aac00640 1 --2- 172.21.15.5:0/2175518141 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd9a410bfc0 0x7fd9a410e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:22.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:22.812+0000 7fd9aac00640 1 --2- 172.21.15.5:0/2175518141 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd9a41034b0 0x7fd9a410b9a0 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:22.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:22.812+0000 7fd9aac00640 1 --2- 172.21.15.5:0/2175518141 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd9a4102b90 0x7fd9a4102f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:22.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:22.812+0000 7fd9aac00640 1 -- 172.21.15.5:0/2175518141 >> 172.21.15.5:0/2175518141 conn(0x7fd9a40fc610 msgr2=0x7fd9a40fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:53:22.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:22.813+0000 7fd9aac00640 1 -- 172.21.15.5:0/2175518141 shutdown_connections 2024-09-10T09:53:22.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:22.813+0000 7fd9aac00640 1 -- 172.21.15.5:0/2175518141 wait complete. 2024-09-10T09:53:22.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:22.814+0000 7fd9aac00640 1 Processor -- start 2024-09-10T09:53:22.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:22.814+0000 7fd9aac00640 1 -- start start 2024-09-10T09:53:22.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:22.814+0000 7fd9aac00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd9a4102b90 0x7fd9a419a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:22.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:22.815+0000 7fd9aac00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd9a41034b0 0x7fd9a419a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:22.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:22.815+0000 7fd9aac00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd9a410bfc0 0x7fd9a419ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:22.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:22.815+0000 7fd9aac00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd9a4111e10 con 0x7fd9a41034b0 2024-09-10T09:53:22.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:22.815+0000 7fd9aac00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fd9a4111c90 con 0x7fd9a4102b90 2024-09-10T09:53:22.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:22.815+0000 7fd9aac00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fd9a4111f90 con 0x7fd9a410bfc0 2024-09-10T09:53:22.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:22.815+0000 7fd9a9800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd9a4102b90 0x7fd9a419a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:22.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:22.815+0000 7fd9a9800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd9a4102b90 0x7fd9a419a4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:49822/0 (socket says 172.21.15.5:49822) 2024-09-10T09:53:22.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:22.815+0000 7fd9a9800640 1 -- 172.21.15.5:0/3349615321 learned_addr learned my addr 172.21.15.5:0/3349615321 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:53:22.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:22.815+0000 7fd9aa200640 1 --2- 172.21.15.5:0/3349615321 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd9a410bfc0 0x7fd9a419ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:22.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:22.815+0000 7fd9a8e00640 1 --2- 172.21.15.5:0/3349615321 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd9a41034b0 0x7fd9a419a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:22.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:22.816+0000 7fd9a9800640 1 -- 172.21.15.5:0/3349615321 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd9a410bfc0 msgr2=0x7fd9a419ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:53:22.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:22.816+0000 7fd9a9800640 1 --2- 172.21.15.5:0/3349615321 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd9a410bfc0 0x7fd9a419ed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:22.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:22.816+0000 7fd9a9800640 1 -- 172.21.15.5:0/3349615321 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd9a41034b0 msgr2=0x7fd9a419a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:53:22.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:22.816+0000 7fd9a9800640 1 --2- 172.21.15.5:0/3349615321 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd9a41034b0 0x7fd9a419a9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:22.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:22.816+0000 7fd9a9800640 1 -- 172.21.15.5:0/3349615321 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9a419f460 con 0x7fd9a4102b90 2024-09-10T09:53:22.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:22.816+0000 7fd9a8e00640 1 --2- 172.21.15.5:0/3349615321 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd9a41034b0 0x7fd9a419a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:53:22.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:22.816+0000 7fd9aa200640 1 --2- 172.21.15.5:0/3349615321 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd9a410bfc0 0x7fd9a419ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:53:22.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:22.816+0000 7fd9a9800640 1 --2- 172.21.15.5:0/3349615321 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd9a4102b90 0x7fd9a419a4b0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fd99800d950 tx=0x7fd99800de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:53:22.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:22.816+0000 7fd9a2000640 1 -- 172.21.15.5:0/3349615321 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd998017070 con 0x7fd9a4102b90 2024-09-10T09:53:22.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:22.816+0000 7fd9a2000640 1 -- 172.21.15.5:0/3349615321 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd998009c30 con 0x7fd9a4102b90 2024-09-10T09:53:22.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:22.816+0000 7fd9a2000640 1 -- 172.21.15.5:0/3349615321 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd99800f6b0 con 0x7fd9a4102b90 2024-09-10T09:53:22.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:22.816+0000 7fd9aac00640 1 -- 172.21.15.5:0/3349615321 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9a419f750 con 0x7fd9a4102b90 2024-09-10T09:53:22.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:22.816+0000 7fd9aac00640 1 -- 172.21.15.5:0/3349615321 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9a41a7030 con 0x7fd9a4102b90 2024-09-10T09:53:22.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:22.819+0000 7fd9aedd8740 1 Processor -- start 2024-09-10T09:53:22.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:22.820+0000 7fd9aedd8740 1 -- start start 2024-09-10T09:53:22.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:22.820+0000 7fd9aedd8740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55635c4cde40 0x55635c4ce220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:22.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:22.820+0000 7fd9aedd8740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55635c4ce760 0x55635c4d8ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:22.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:22.820+0000 7fd9aedd8740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55635c4d9020 0x55635c4db410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:22.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:22.820+0000 7fd9aedd8740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55635c4343e0 con 0x55635c4d9020 2024-09-10T09:53:22.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:22.820+0000 7fd9aedd8740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55635c3dbd80 con 0x55635c4ce760 2024-09-10T09:53:22.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:22.820+0000 7fd9aedd8740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55635c4303a0 con 0x55635c4cde40 2024-09-10T09:53:22.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:22.820+0000 7fd9a2000640 1 -- 172.21.15.5:0/3349615321 <== mon.2 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7fd9980109d0 con 0x7fd9a4102b90 2024-09-10T09:53:22.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:22.820+0000 7fd9aa200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55635c4cde40 0x55635c4ce220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:22.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:22.820+0000 7fd9a8e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55635c4d9020 0x55635c4db410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:22.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:22.820+0000 7fd9aa200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55635c4cde40 0x55635c4ce220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:52066/0 (socket says 172.21.15.5:52066) 2024-09-10T09:53:22.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:22.820+0000 7fd9aa200640 1 -- 172.21.15.5:0/48428013 learned_addr learned my addr 172.21.15.5:0/48428013 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:53:22.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:22.821+0000 7fd9a9800640 1 --2- 172.21.15.5:0/48428013 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55635c4ce760 0x55635c4d8ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:22.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:22.821+0000 7fd9aa200640 1 -- 172.21.15.5:0/48428013 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55635c4ce760 msgr2=0x55635c4d8ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:53:22.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:22.821+0000 7fd9aa200640 1 --2- 172.21.15.5:0/48428013 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55635c4ce760 0x55635c4d8ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:22.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:22.821+0000 7fd9aa200640 1 -- 172.21.15.5:0/48428013 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55635c4d9020 msgr2=0x55635c4db410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:53:22.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:22.821+0000 7fd9aa200640 1 --2- 172.21.15.5:0/48428013 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55635c4d9020 0x55635c4db410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:22.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:22.821+0000 7fd9aa200640 1 -- 172.21.15.5:0/48428013 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55635c3faae0 con 0x55635c4cde40 2024-09-10T09:53:22.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:22.821+0000 7fd9aa200640 1 --2- 172.21.15.5:0/48428013 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55635c4cde40 0x55635c4ce220 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fd99c011790 tx=0x7fd99c011c50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5a9a5cdb2e248d7c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:53:22.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:22.822+0000 7fd996a00640 1 -- 172.21.15.5:0/48428013 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd99c003c30 con 0x55635c4cde40 2024-09-10T09:53:22.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:22.822+0000 7fd996a00640 1 -- 172.21.15.5:0/48428013 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd99c00fe80 con 0x55635c4cde40 2024-09-10T09:53:22.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:22.822+0000 7fd996a00640 1 -- 172.21.15.5:0/48428013 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd99c0193e0 con 0x55635c4cde40 2024-09-10T09:53:22.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:22.822+0000 7fd9a2000640 1 --2- 172.21.15.5:0/3349615321 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fd98c0a2b80 0x7fd98c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:22.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:22.822+0000 7fd9a2000640 1 -- 172.21.15.5:0/3349615321 <== mon.2 v2:172.21.15.102:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fd9980cf100 con 0x7fd9a4102b90 2024-09-10T09:53:22.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:22.822+0000 7fd9a9800640 1 --2- 172.21.15.5:0/3349615321 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fd98c0a2b80 0x7fd98c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:22.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:22.823+0000 7fd9aedd8740 1 -- 172.21.15.5:0/48428013 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55635c4cde40 msgr2=0x55635c4ce220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:53:22.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:22.823+0000 7fd9aedd8740 1 --2- 172.21.15.5:0/48428013 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55635c4cde40 0x55635c4ce220 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fd99c011790 tx=0x7fd99c011c50 comp rx=0 tx=0).stop 2024-09-10T09:53:22.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:22.823+0000 7fd9aedd8740 1 -- 172.21.15.5:0/48428013 shutdown_connections 2024-09-10T09:53:22.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:22.823+0000 7fd9aedd8740 1 --2- 172.21.15.5:0/48428013 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55635c4d9020 0x55635c4db410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:22.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:22.823+0000 7fd9aedd8740 1 --2- 172.21.15.5:0/48428013 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55635c4ce760 0x55635c4d8ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:22.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:22.823+0000 7fd9aedd8740 1 --2- 172.21.15.5:0/48428013 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55635c4cde40 0x55635c4ce220 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:22.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:22.823+0000 7fd9aedd8740 1 -- 172.21.15.5:0/48428013 >> 172.21.15.5:0/48428013 conn(0x55635c4ca3d0 msgr2=0x55635c4ca7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:53:22.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:22.823+0000 7fd9aedd8740 1 -- 172.21.15.5:0/48428013 shutdown_connections 2024-09-10T09:53:22.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:22.823+0000 7fd9a9800640 1 --2- 172.21.15.5:0/3349615321 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fd98c0a2b80 0x7fd98c0a5040 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fd998097ee0 tx=0x7fd998000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:53:22.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:22.823+0000 7fd9aedd8740 1 -- 172.21.15.5:0/48428013 wait complete. 2024-09-10T09:53:22.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:22.824+0000 7fd9aedd8740 1 -- start start 2024-09-10T09:53:22.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:22.824+0000 7fd9aedd8740 1 Processor -- start 2024-09-10T09:53:22.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:22.825+0000 7fd9aedd8740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55635c4cde40 0x55635c4d5f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:22.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:22.825+0000 7fd9aedd8740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55635c4ce760 0x55635c4d6440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:22.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:22.825+0000 7fd9aedd8740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55635c4d9020 0x55635c4d6980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:22.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:22.825+0000 7fd9aedd8740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55635c4dbbf0 con 0x55635c4cde40 2024-09-10T09:53:22.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:22.825+0000 7fd9aedd8740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55635c3ef0f0 con 0x55635c4d9020 2024-09-10T09:53:22.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:22.825+0000 7fd9aedd8740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55635c4dbd70 con 0x55635c4ce760 2024-09-10T09:53:22.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:22.825+0000 7fd9aedd8740 10 client.0 _ll_register_callbacks cb 0x55635c3d6bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:53:22.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:22.825+0000 7fd9a8e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55635c4ce760 0x55635c4d6440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:22.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:22.825+0000 7fd9a8e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55635c4ce760 0x55635c4d6440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:52074/0 (socket says 172.21.15.5:52074) 2024-09-10T09:53:22.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:22.825+0000 7fd9a8e00640 1 -- 172.21.15.5:0/3410426614 learned_addr learned my addr 172.21.15.5:0/3410426614 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:53:22.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:22.825+0000 7fd9aa200640 1 --2- 172.21.15.5:0/3410426614 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55635c4cde40 0x55635c4d5f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:22.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:22.825+0000 7fd9aa200640 1 --2- 172.21.15.5:0/3410426614 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55635c4d9020 0x55635c4d6980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:22.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:22.826+0000 7fd9a8e00640 1 -- 172.21.15.5:0/3410426614 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55635c4d9020 msgr2=0x55635c4d6980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:53:22.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:22.826+0000 7fd9a8e00640 1 --2- 172.21.15.5:0/3410426614 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55635c4d9020 0x55635c4d6980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:22.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:22.826+0000 7fd9a8e00640 1 -- 172.21.15.5:0/3410426614 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55635c4cde40 msgr2=0x55635c4d5f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:53:22.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:22.826+0000 7fd9a8e00640 1 --2- 172.21.15.5:0/3410426614 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55635c4cde40 0x55635c4d5f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:22.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:22.826+0000 7fd9a8e00640 1 -- 172.21.15.5:0/3410426614 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55635c1a53c0 con 0x55635c4ce760 2024-09-10T09:53:22.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:22.826+0000 7fd9aa200640 1 --2- 172.21.15.5:0/3410426614 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55635c4d9020 0x55635c4d6980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:53:22.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:22.826+0000 7fd9a8e00640 1 --2- 172.21.15.5:0/3410426614 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55635c4ce760 0x55635c4d6440 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fd990009900 tx=0x7fd990005270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a2f2fc2671470951 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:53:22.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:22.826+0000 7fd98b400640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-09-10T09:53:22.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:22.826+0000 7fd98b400640 1 -- 172.21.15.5:0/3410426614 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd990035040 con 0x55635c4ce760 2024-09-10T09:53:22.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:22.826+0000 7fd98b400640 1 -- 172.21.15.5:0/3410426614 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fd9900048c0 con 0x55635c4ce760 2024-09-10T09:53:22.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:22.826+0000 7fd9aedd8740 1 -- 172.21.15.5:0/3410426614 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55635c441950 con 0x55635c4ce760 2024-09-10T09:53:22.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:22.826+0000 7fd98b400640 1 -- 172.21.15.5:0/3410426614 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fd99004e050 con 0x55635c4ce760 2024-09-10T09:53:22.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:22.826+0000 7fd9aedd8740 10 client.28396 fetch_fsmap learned FSMap version 37 2024-09-10T09:53:22.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:22.826+0000 7fd9aedd8740 1 -- 172.21.15.5:0/3410426614 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55635c4f0440 con 0x55635c4ce760 2024-09-10T09:53:22.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:22.827+0000 7fd98b400640 1 -- 172.21.15.5:0/3410426614 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55635c441950 con 0x55635c4ce760 2024-09-10T09:53:22.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:22.827+0000 7fd98b400640 1 -- 172.21.15.5:0/3410426614 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 37) ==== 8390+0+0 (secure 0 0 0) 0x7fd990031ae0 con 0x55635c4ce760 2024-09-10T09:53:22.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:22.827+0000 7fd9aedd8740 10 client.28396 fetch_fsmap finished waiting for FSMap version 37 2024-09-10T09:53:22.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:22.827+0000 7fd9aedd8740 10 client.28396 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-10T09:53:22.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:22.827+0000 7fd9aedd8740 20 client.28396 populate_metadata read hostname 'smithi005' 2024-09-10T09:53:22.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:22.827+0000 7fd9aedd8740 1 --2- 172.21.15.5:0/3410426614 >> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] conn(0x55635c4f1900 0x55635c4f3d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:22.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:22.827+0000 7fd9aedd8740 4 client.28396 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:53:22.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:22.827+0000 7fd9aedd8740 1 -- 172.21.15.5:0/3410426614 --> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55635c43be60 con 0x55635c4f1900 2024-09-10T09:53:22.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:22.828+0000 7fd9aa200640 1 --2- 172.21.15.5:0/3410426614 >> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] conn(0x55635c4f1900 0x55635c4f3d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:22.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:22.828+0000 7fd9aa200640 1 --2- 172.21.15.5:0/3410426614 >> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] conn(0x55635c4f1900 0x55635c4f3d10 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3a09d2704d8724da server_cookie=32ccbc7b73a2659b in_seq=0 out_seq=0 2024-09-10T09:53:22.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:22.828+0000 7fd98b400640 10 client.28396 ms_handle_connect on v2:172.21.15.159:6838/2073739599 2024-09-10T09:53:22.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:22.829+0000 7fd98b400640 1 -- 172.21.15.5:0/3410426614 <== mds.1 v2:172.21.15.159:6838/2073739599 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55635c43be60 con 0x55635c4f1900 2024-09-10T09:53:22.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:22.829+0000 7fd98b400640 10 client.28396 handle_command_reply: tid=0 2024-09-10T09:53:22.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:22.829+0000 7fd9aedd8740 1 client.28396 shutdown 2024-09-10T09:53:22.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:22.829+0000 7fd9aedd8740 1 -- 172.21.15.5:0/3410426614 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55635c4ce760 msgr2=0x55635c4d6440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:53:22.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:22.830+0000 7fd9aedd8740 1 --2- 172.21.15.5:0/3410426614 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55635c4ce760 0x55635c4d6440 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fd990009900 tx=0x7fd990005270 comp rx=0 tx=0).stop 2024-09-10T09:53:22.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:22.830+0000 7fd9aedd8740 1 -- 172.21.15.5:0/3410426614 shutdown_connections 2024-09-10T09:53:22.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:22.830+0000 7fd9aedd8740 1 --2- 172.21.15.5:0/3410426614 >> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] conn(0x55635c4f1900 0x55635c4f3d10 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:22.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:22.830+0000 7fd9aedd8740 1 --2- 172.21.15.5:0/3410426614 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55635c4d9020 0x55635c4d6980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:22.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:22.830+0000 7fd9aedd8740 1 --2- 172.21.15.5:0/3410426614 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55635c4ce760 0x55635c4d6440 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:22.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:22.830+0000 7fd9aedd8740 1 --2- 172.21.15.5:0/3410426614 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55635c4cde40 0x55635c4d5f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:22.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:22.830+0000 7fd9aedd8740 1 -- 172.21.15.5:0/3410426614 >> 172.21.15.5:0/3410426614 conn(0x55635c4ca3d0 msgr2=0x55635c4d00b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:53:22.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:22.830+0000 7fd98b400640 0 client.28396 ms_handle_reset on v2:172.21.15.159:6838/2073739599 2024-09-10T09:53:22.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:22.830+0000 7fd9aedd8740 1 -- 172.21.15.5:0/3410426614 shutdown_connections 2024-09-10T09:53:22.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:22.830+0000 7fd9aedd8740 1 -- 172.21.15.5:0/3410426614 wait complete. 2024-09-10T09:53:22.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:22.831+0000 7fd9aedd8740 20 client.28396 trim_cache size 0 max 16384 2024-09-10T09:53:22.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:22.849+0000 7fd9aedd8740 1 Processor -- start 2024-09-10T09:53:22.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:22.849+0000 7fd9aedd8740 1 -- start start 2024-09-10T09:53:22.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:22.849+0000 7fd9aedd8740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55635c4cde40 0x55635c4d57c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:22.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:22.849+0000 7fd9aedd8740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55635c4ce760 0x55635c4d7e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:22.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:22.849+0000 7fd9aedd8740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55635c4d9020 0x55635c4f9620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:22.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:22.849+0000 7fd9aedd8740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd9a41acb90 con 0x55635c4ce760 2024-09-10T09:53:22.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:22.849+0000 7fd9aedd8740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fd9a41ae7c0 con 0x55635c4cde40 2024-09-10T09:53:22.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:22.849+0000 7fd9aedd8740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fd9a41ad260 con 0x55635c4d9020 2024-09-10T09:53:22.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:22.849+0000 7fd9aa200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55635c4cde40 0x55635c4d57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:22.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:22.849+0000 7fd9a8e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55635c4ce760 0x55635c4d7e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:22.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:22.849+0000 7fd9aa200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55635c4cde40 0x55635c4d57c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:49852/0 (socket says 172.21.15.5:49852) 2024-09-10T09:53:22.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:22.849+0000 7fd9aa200640 1 -- 172.21.15.5:0/2017381983 learned_addr learned my addr 172.21.15.5:0/2017381983 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:53:22.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:22.850+0000 7fd9aa200640 1 --2- 172.21.15.5:0/2017381983 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55635c4d9020 0x55635c4f9620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:22.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:22.850+0000 7fd9aa200640 1 -- 172.21.15.5:0/2017381983 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55635c4d9020 msgr2=0x55635c4f9620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:53:22.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:22.850+0000 7fd9aa200640 1 --2- 172.21.15.5:0/2017381983 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55635c4d9020 0x55635c4f9620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:22.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:22.850+0000 7fd9aa200640 1 -- 172.21.15.5:0/2017381983 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55635c4ce760 msgr2=0x55635c4d7e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:53:22.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:22.850+0000 7fd9aa200640 1 --2- 172.21.15.5:0/2017381983 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55635c4ce760 0x55635c4d7e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:22.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:22.850+0000 7fd9aa200640 1 -- 172.21.15.5:0/2017381983 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55635c4dbad0 con 0x55635c4cde40 2024-09-10T09:53:22.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:22.850+0000 7fd9a8e00640 1 --2- 172.21.15.5:0/2017381983 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55635c4ce760 0x55635c4d7e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:53:22.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:22.850+0000 7fd9aa200640 1 --2- 172.21.15.5:0/2017381983 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55635c4cde40 0x55635c4d57c0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fd99c004600 tx=0x7fd99c004490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b04b58cf043c5adc server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:53:22.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:22.853+0000 7fd996000640 1 -- 172.21.15.5:0/2017381983 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd99c0058b0 con 0x55635c4cde40 2024-09-10T09:53:22.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:22.853+0000 7fd996000640 1 -- 172.21.15.5:0/2017381983 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd99c011020 con 0x55635c4cde40 2024-09-10T09:53:22.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:22.853+0000 7fd996000640 1 -- 172.21.15.5:0/2017381983 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd99c010690 con 0x55635c4cde40 2024-09-10T09:53:22.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:22.854+0000 7fd9aedd8740 1 -- 172.21.15.5:0/2017381983 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55635c4cde40 msgr2=0x55635c4d57c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:53:22.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:22.854+0000 7fd9aedd8740 1 --2- 172.21.15.5:0/2017381983 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55635c4cde40 0x55635c4d57c0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fd99c004600 tx=0x7fd99c004490 comp rx=0 tx=0).stop 2024-09-10T09:53:22.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:22.854+0000 7fd9aedd8740 1 -- 172.21.15.5:0/2017381983 shutdown_connections 2024-09-10T09:53:22.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:22.854+0000 7fd9aedd8740 1 --2- 172.21.15.5:0/2017381983 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55635c4d9020 0x55635c4f9620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:22.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:22.854+0000 7fd9aedd8740 1 --2- 172.21.15.5:0/2017381983 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55635c4ce760 0x55635c4d7e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:22.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:22.854+0000 7fd9aedd8740 1 --2- 172.21.15.5:0/2017381983 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55635c4cde40 0x55635c4d57c0 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:22.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:22.854+0000 7fd9aedd8740 1 -- 172.21.15.5:0/2017381983 >> 172.21.15.5:0/2017381983 conn(0x55635c4ca3d0 msgr2=0x55635c4d7760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:53:22.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:22.855+0000 7fd9aedd8740 1 -- 172.21.15.5:0/2017381983 shutdown_connections 2024-09-10T09:53:22.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:22.855+0000 7fd9aedd8740 1 -- 172.21.15.5:0/2017381983 wait complete. 2024-09-10T09:53:22.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:22.855+0000 7fd9aedd8740 1 -- start start 2024-09-10T09:53:22.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:22.856+0000 7fd9aedd8740 1 Processor -- start 2024-09-10T09:53:22.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:22.856+0000 7fd9aedd8740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55635c4cde40 0x55635c4e54c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:22.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:22.856+0000 7fd9aedd8740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55635c4ce760 0x55635c4e5a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:22.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:22.856+0000 7fd9aedd8740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55635c4d9020 0x55635c4e5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:22.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:22.856+0000 7fd9aedd8740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd9a41ae0c0 con 0x55635c4ce760 2024-09-10T09:53:22.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:22.856+0000 7fd9aedd8740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fd9a41ad9f0 con 0x55635c4cde40 2024-09-10T09:53:22.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:22.856+0000 7fd9aedd8740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fd9a41aabd0 con 0x55635c4d9020 2024-09-10T09:53:22.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:22.856+0000 7fd9aedd8740 10 client.0 _ll_register_callbacks cb 0x7fd99c0093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:53:22.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:22.856+0000 7fd9aa200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55635c4cde40 0x55635c4e54c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:22.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:22.856+0000 7fd9a8e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55635c4ce760 0x55635c4e5a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:22.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:22.856+0000 7fd9aa200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55635c4cde40 0x55635c4e54c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:49866/0 (socket says 172.21.15.5:49866) 2024-09-10T09:53:22.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:22.856+0000 7fd9aa200640 1 -- 172.21.15.5:0/3600676250 learned_addr learned my addr 172.21.15.5:0/3600676250 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:53:22.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:22.856+0000 7fd9aa200640 1 --2- 172.21.15.5:0/3600676250 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55635c4d9020 0x55635c4e5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:22.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:22.857+0000 7fd9a8e00640 1 -- 172.21.15.5:0/3600676250 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55635c4d9020 msgr2=0x55635c4e5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:53:22.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:22.857+0000 7fd9a8e00640 1 --2- 172.21.15.5:0/3600676250 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55635c4d9020 0x55635c4e5f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:22.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:22.857+0000 7fd9a8e00640 1 -- 172.21.15.5:0/3600676250 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55635c4cde40 msgr2=0x55635c4e54c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:53:22.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:22.857+0000 7fd9a8e00640 1 --2- 172.21.15.5:0/3600676250 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55635c4cde40 0x55635c4e54c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:22.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:22.857+0000 7fd9a8e00640 1 -- 172.21.15.5:0/3600676250 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55635c55b5f0 con 0x55635c4ce760 2024-09-10T09:53:22.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:22.857+0000 7fd9aa200640 1 --2- 172.21.15.5:0/3600676250 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55635c4cde40 0x55635c4e54c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:53:22.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:22.857+0000 7fd9a8e00640 1 --2- 172.21.15.5:0/3600676250 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55635c4ce760 0x55635c4e5a00 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7fd990031ee0 tx=0x7fd990005c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9468fedaa0f3009 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:53:22.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:22.857+0000 7fd98b400640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-10T09:53:22.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:22.859+0000 7fd98b400640 1 -- 172.21.15.5:0/3600676250 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd990056020 con 0x55635c4ce760 2024-09-10T09:53:22.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:22.860+0000 7fd98b400640 1 -- 172.21.15.5:0/3600676250 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fd9900038d0 con 0x55635c4ce760 2024-09-10T09:53:22.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:22.860+0000 7fd9aedd8740 1 -- 172.21.15.5:0/3600676250 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd99c011020 con 0x55635c4ce760 2024-09-10T09:53:22.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:22.860+0000 7fd98b400640 1 -- 172.21.15.5:0/3600676250 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fd990035050 con 0x55635c4ce760 2024-09-10T09:53:22.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:22.861+0000 7fd9aedd8740 10 client.18360 fetch_fsmap learned FSMap version 37 2024-09-10T09:53:22.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:22.861+0000 7fd9aedd8740 1 -- 172.21.15.5:0/3600676250 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55635c55c7f0 con 0x55635c4ce760 2024-09-10T09:53:22.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:22.861+0000 7fd98b400640 1 -- 172.21.15.5:0/3600676250 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd99c011020 con 0x55635c4ce760 2024-09-10T09:53:22.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:22.861+0000 7fd98b400640 1 -- 172.21.15.5:0/3600676250 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 37) ==== 8390+0+0 (secure 0 0 0) 0x7fd9900046c0 con 0x55635c4ce760 2024-09-10T09:53:22.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:22.861+0000 7fd9aedd8740 10 client.18360 fetch_fsmap finished waiting for FSMap version 37 2024-09-10T09:53:22.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:22.861+0000 7fd9aedd8740 10 client.18360 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-10T09:53:22.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:22.861+0000 7fd9aedd8740 20 client.18360 populate_metadata read hostname 'smithi005' 2024-09-10T09:53:22.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:22.861+0000 7fd9aedd8740 1 --2- 172.21.15.5:0/3600676250 >> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] conn(0x55635c55cf60 0x55635c55f380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:22.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:22.861+0000 7fd9aedd8740 4 client.18360 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T09:53:22.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:22.861+0000 7fd9aedd8740 1 -- 172.21.15.5:0/3600676250 --> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55635c3b8010 con 0x55635c55cf60 2024-09-10T09:53:22.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:22.862+0000 7fd9aa200640 1 --2- 172.21.15.5:0/3600676250 >> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] conn(0x55635c55cf60 0x55635c55f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:22.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:22.862+0000 7fd9aa200640 1 --2- 172.21.15.5:0/3600676250 >> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] conn(0x55635c55cf60 0x55635c55f380 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=740cff1eb2505b3f server_cookie=71915e31b73a4b8b in_seq=0 out_seq=0 2024-09-10T09:53:22.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:22.862+0000 7fd98b400640 10 client.18360 ms_handle_connect on v2:172.21.15.159:6838/2073739599 2024-09-10T09:53:22.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:22.863+0000 7fd98b400640 1 -- 172.21.15.5:0/3600676250 <== mds.1 v2:172.21.15.159:6838/2073739599 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55635c3b8010 con 0x55635c55cf60 2024-09-10T09:53:22.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:22.863+0000 7fd98b400640 10 client.18360 handle_command_reply: tid=0 2024-09-10T09:53:22.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:22.863+0000 7fd9aedd8740 1 client.18360 shutdown 2024-09-10T09:53:22.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:22.863+0000 7fd9aedd8740 1 -- 172.21.15.5:0/3600676250 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55635c4ce760 msgr2=0x55635c4e5a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:53:22.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:22.863+0000 7fd9aedd8740 1 --2- 172.21.15.5:0/3600676250 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55635c4ce760 0x55635c4e5a00 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7fd990031ee0 tx=0x7fd990005c00 comp rx=0 tx=0).stop 2024-09-10T09:53:22.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:22.863+0000 7fd9aedd8740 1 -- 172.21.15.5:0/3600676250 shutdown_connections 2024-09-10T09:53:22.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:22.863+0000 7fd9aedd8740 1 --2- 172.21.15.5:0/3600676250 >> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] conn(0x55635c55cf60 0x55635c55f380 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:22.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:22.863+0000 7fd9aedd8740 1 --2- 172.21.15.5:0/3600676250 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55635c4d9020 0x55635c4e5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:22.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:22.863+0000 7fd9aedd8740 1 --2- 172.21.15.5:0/3600676250 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55635c4ce760 0x55635c4e5a00 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:22.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:22.863+0000 7fd9aedd8740 1 --2- 172.21.15.5:0/3600676250 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55635c4cde40 0x55635c4e54c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:22.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:22.863+0000 7fd9aedd8740 1 -- 172.21.15.5:0/3600676250 >> 172.21.15.5:0/3600676250 conn(0x55635c4ca3d0 msgr2=0x55635c4d7f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:53:22.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:22.863+0000 7fd98b400640 0 client.18360 ms_handle_reset on v2:172.21.15.159:6838/2073739599 2024-09-10T09:53:22.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:22.863+0000 7fd9aedd8740 1 -- 172.21.15.5:0/3600676250 shutdown_connections 2024-09-10T09:53:22.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:22.863+0000 7fd9aedd8740 1 -- 172.21.15.5:0/3600676250 wait complete. 2024-09-10T09:53:22.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:22.863+0000 7fd9aedd8740 20 client.18360 trim_cache size 0 max 16384 2024-09-10T09:53:22.866 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-10T09:53:22.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:22.864+0000 7fd9aac00640 1 -- 172.21.15.5:0/3349615321 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fd98c0a2b80 msgr2=0x7fd98c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:53:22.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:22.864+0000 7fd9aac00640 1 --2- 172.21.15.5:0/3349615321 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fd98c0a2b80 0x7fd98c0a5040 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fd998097ee0 tx=0x7fd998000bc0 comp rx=0 tx=0).stop 2024-09-10T09:53:22.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:22.864+0000 7fd9aac00640 1 -- 172.21.15.5:0/3349615321 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd9a4102b90 msgr2=0x7fd9a419a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:53:22.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:22.864+0000 7fd9aac00640 1 --2- 172.21.15.5:0/3349615321 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd9a4102b90 0x7fd9a419a4b0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fd99800d950 tx=0x7fd99800de10 comp rx=0 tx=0).stop 2024-09-10T09:53:22.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:22.864+0000 7fd9aac00640 1 -- 172.21.15.5:0/3349615321 shutdown_connections 2024-09-10T09:53:22.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:22.864+0000 7fd9aac00640 1 --2- 172.21.15.5:0/3349615321 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fd98c0a2b80 0x7fd98c0a5040 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:22.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:22.864+0000 7fd9aac00640 1 --2- 172.21.15.5:0/3349615321 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd9a410bfc0 0x7fd9a419ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:22.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:22.864+0000 7fd9aac00640 1 --2- 172.21.15.5:0/3349615321 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd9a41034b0 0x7fd9a419a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:22.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:22.865+0000 7fd9aac00640 1 --2- 172.21.15.5:0/3349615321 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd9a4102b90 0x7fd9a419a4b0 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:22.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:22.865+0000 7fd9aac00640 1 -- 172.21.15.5:0/3349615321 >> 172.21.15.5:0/3349615321 conn(0x7fd9a40fc610 msgr2=0x7fd9a40fd210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:53:22.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:22.865+0000 7fd9aac00640 1 -- 172.21.15.5:0/3349615321 shutdown_connections 2024-09-10T09:53:22.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:22.865+0000 7fd9aac00640 1 -- 172.21.15.5:0/3349615321 wait complete. 2024-09-10T09:53:22.949 INFO:tasks.workunit.client.0.smithi005.stdout: 10 24352 24.24 MB/sec warmup 61 sec latency 153.655 ms 2024-09-10T09:53:23.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:53:22 smithi005 ceph-mon[30350]: pgmap v844: 129 pgs: 129 active+clean; 534 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 4.5 MiB/s wr, 425 op/s 2024-09-10T09:53:23.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:53:22 smithi102 ceph-mon[35442]: pgmap v844: 129 pgs: 129 active+clean; 534 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 4.5 MiB/s wr, 425 op/s 2024-09-10T09:53:23.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:53:22 smithi159 ceph-mon[35513]: pgmap v844: 129 pgs: 129 active+clean; 534 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 4.5 MiB/s wr, 425 op/s 2024-09-10T09:53:23.701 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-10T09:53:23.949 INFO:tasks.workunit.client.0.smithi005.stdout: 10 25217 24.80 MB/sec warmup 62 sec latency 69.099 ms 2024-09-10T09:53:24.195 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:53:24.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:24.831+0000 7fc975400640 1 -- 172.21.15.5:0/1680205132 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc970112870 msgr2=0x7fc970114c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:53:24.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:24.831+0000 7fc975400640 1 --2- 172.21.15.5:0/1680205132 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc970112870 0x7fc970114c60 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fc968009ca0 tx=0x7fc96802a8b0 comp rx=0 tx=0).stop 2024-09-10T09:53:24.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:24.831+0000 7fc975400640 1 -- 172.21.15.5:0/1680205132 shutdown_connections 2024-09-10T09:53:24.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:24.831+0000 7fc975400640 1 --2- 172.21.15.5:0/1680205132 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc970112870 0x7fc970114c60 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:24.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:24.831+0000 7fc975400640 1 --2- 172.21.15.5:0/1680205132 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc970077040 0x7fc970112330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:24.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:24.831+0000 7fc975400640 1 --2- 172.21.15.5:0/1680205132 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc970076720 0x7fc970076b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:24.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:24.831+0000 7fc975400640 1 -- 172.21.15.5:0/1680205132 >> 172.21.15.5:0/1680205132 conn(0x7fc9700ff910 msgr2=0x7fc970101d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:53:24.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:24.831+0000 7fc975400640 1 -- 172.21.15.5:0/1680205132 shutdown_connections 2024-09-10T09:53:24.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:24.831+0000 7fc975400640 1 -- 172.21.15.5:0/1680205132 wait complete. 2024-09-10T09:53:24.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:24.832+0000 7fc975400640 1 Processor -- start 2024-09-10T09:53:24.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:24.833+0000 7fc975400640 1 -- start start 2024-09-10T09:53:24.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:24.833+0000 7fc975400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc970076720 0x7fc9701a45c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:24.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:24.833+0000 7fc975400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc970077040 0x7fc9701a4b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:24.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:24.833+0000 7fc975400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc970112870 0x7fc97019e690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:24.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:24.833+0000 7fc975400640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc970118400 con 0x7fc970077040 2024-09-10T09:53:24.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:24.833+0000 7fc975400640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fc970118280 con 0x7fc970076720 2024-09-10T09:53:24.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:24.833+0000 7fc975400640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fc970118580 con 0x7fc970112870 2024-09-10T09:53:24.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:24.833+0000 7fc96fe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc970076720 0x7fc9701a45c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:24.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:24.833+0000 7fc974a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc970112870 0x7fc97019e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:24.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:24.833+0000 7fc96fe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc970076720 0x7fc9701a45c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:49884/0 (socket says 172.21.15.5:49884) 2024-09-10T09:53:24.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:24.833+0000 7fc96fe00640 1 -- 172.21.15.5:0/1829210188 learned_addr learned my addr 172.21.15.5:0/1829210188 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:53:24.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:24.834+0000 7fc96f400640 1 --2- 172.21.15.5:0/1829210188 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc970077040 0x7fc9701a4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:24.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:24.834+0000 7fc974a00640 1 -- 172.21.15.5:0/1829210188 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc970076720 msgr2=0x7fc9701a45c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:53:24.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:24.835+0000 7fc974a00640 1 --2- 172.21.15.5:0/1829210188 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc970076720 0x7fc9701a45c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:24.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:24.835+0000 7fc974a00640 1 -- 172.21.15.5:0/1829210188 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc970077040 msgr2=0x7fc9701a4b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:53:24.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:24.835+0000 7fc974a00640 1 --2- 172.21.15.5:0/1829210188 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc970077040 0x7fc9701a4b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:24.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:24.835+0000 7fc974a00640 1 -- 172.21.15.5:0/1829210188 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc97019ef00 con 0x7fc970112870 2024-09-10T09:53:24.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:24.835+0000 7fc96f400640 1 --2- 172.21.15.5:0/1829210188 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc970077040 0x7fc9701a4b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:53:24.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:24.835+0000 7fc96fe00640 1 --2- 172.21.15.5:0/1829210188 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc970076720 0x7fc9701a45c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:53:24.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:24.835+0000 7fc974a00640 1 --2- 172.21.15.5:0/1829210188 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc970112870 0x7fc97019e690 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fc96802aea0 tx=0x7fc968003f10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:53:24.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:24.836+0000 7fc96cc00640 1 -- 172.21.15.5:0/1829210188 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc968030e40 con 0x7fc970112870 2024-09-10T09:53:24.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:24.836+0000 7fc975400640 1 -- 172.21.15.5:0/1829210188 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc97019f190 con 0x7fc970112870 2024-09-10T09:53:24.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:24.836+0000 7fc96cc00640 1 -- 172.21.15.5:0/1829210188 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc968040070 con 0x7fc970112870 2024-09-10T09:53:24.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:24.836+0000 7fc975400640 1 -- 172.21.15.5:0/1829210188 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9701ab370 con 0x7fc970112870 2024-09-10T09:53:24.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:24.836+0000 7fc96cc00640 1 -- 172.21.15.5:0/1829210188 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc968031ac0 con 0x7fc970112870 2024-09-10T09:53:24.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:24.839+0000 7fc979609740 1 Processor -- start 2024-09-10T09:53:24.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:24.840+0000 7fc96cc00640 1 -- 172.21.15.5:0/1829210188 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7fc968031c60 con 0x7fc970112870 2024-09-10T09:53:24.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:24.840+0000 7fc979609740 1 -- start start 2024-09-10T09:53:24.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:24.841+0000 7fc979609740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563fc997ce40 0x563fc997d220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:24.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:24.841+0000 7fc979609740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563fc997d760 0x563fc9987ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:24.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:24.841+0000 7fc979609740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563fc9988020 0x563fc998a410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:24.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:24.841+0000 7fc979609740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x563fc98e33e0 con 0x563fc997d760 2024-09-10T09:53:24.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:24.841+0000 7fc979609740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x563fc988ad80 con 0x563fc997ce40 2024-09-10T09:53:24.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:24.841+0000 7fc979609740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x563fc98df3a0 con 0x563fc9988020 2024-09-10T09:53:24.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:24.841+0000 7fc96f400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563fc997ce40 0x563fc997d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:24.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:24.841+0000 7fc96f400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563fc997ce40 0x563fc997d220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:49896/0 (socket says 172.21.15.5:49896) 2024-09-10T09:53:24.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:24.841+0000 7fc96f400640 1 -- 172.21.15.5:0/2763569728 learned_addr learned my addr 172.21.15.5:0/2763569728 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:53:24.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:24.841+0000 7fc96fe00640 1 --2- 172.21.15.5:0/2763569728 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563fc997d760 0x563fc9987ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:24.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:24.842+0000 7fc96f400640 1 --2- 172.21.15.5:0/2763569728 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563fc9988020 0x563fc998a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:24.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:24.842+0000 7fc96f400640 1 -- 172.21.15.5:0/2763569728 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563fc9988020 msgr2=0x563fc998a410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:53:24.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:24.842+0000 7fc96f400640 1 --2- 172.21.15.5:0/2763569728 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563fc9988020 0x563fc998a410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:24.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:24.842+0000 7fc96f400640 1 -- 172.21.15.5:0/2763569728 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563fc997d760 msgr2=0x563fc9987ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:53:24.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:24.842+0000 7fc96f400640 1 --2- 172.21.15.5:0/2763569728 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563fc997d760 0x563fc9987ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:24.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:24.842+0000 7fc96f400640 1 -- 172.21.15.5:0/2763569728 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563fc98a9ae0 con 0x563fc997ce40 2024-09-10T09:53:24.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:24.842+0000 7fc96fe00640 1 --2- 172.21.15.5:0/2763569728 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563fc997d760 0x563fc9987ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:53:24.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:24.842+0000 7fc96f400640 1 --2- 172.21.15.5:0/2763569728 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563fc997ce40 0x563fc997d220 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fc95c013970 tx=0x7fc95c009d80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a3368cd497d82788 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:53:24.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:24.842+0000 7fc96cc00640 1 --2- 172.21.15.5:0/1829210188 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fc9540a2b80 0x7fc9540a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:24.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:24.843+0000 7fc965200640 1 -- 172.21.15.5:0/2763569728 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc95c00f8c0 con 0x563fc997ce40 2024-09-10T09:53:24.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:24.843+0000 7fc96cc00640 1 -- 172.21.15.5:0/1829210188 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fc9680f0720 con 0x7fc970112870 2024-09-10T09:53:24.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:24.843+0000 7fc965200640 1 -- 172.21.15.5:0/2763569728 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc95c010430 con 0x563fc997ce40 2024-09-10T09:53:24.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:24.843+0000 7fc965200640 1 -- 172.21.15.5:0/2763569728 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc95c00e5f0 con 0x563fc997ce40 2024-09-10T09:53:24.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:24.843+0000 7fc96fe00640 1 --2- 172.21.15.5:0/1829210188 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fc9540a2b80 0x7fc9540a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:24.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:24.844+0000 7fc979609740 1 -- 172.21.15.5:0/2763569728 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563fc997ce40 msgr2=0x563fc997d220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:53:24.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:24.844+0000 7fc979609740 1 --2- 172.21.15.5:0/2763569728 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563fc997ce40 0x563fc997d220 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fc95c013970 tx=0x7fc95c009d80 comp rx=0 tx=0).stop 2024-09-10T09:53:24.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:24.844+0000 7fc96fe00640 1 --2- 172.21.15.5:0/1829210188 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fc9540a2b80 0x7fc9540a5040 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fc96000afd0 tx=0x7fc96000a380 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:53:24.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:24.845+0000 7fc979609740 1 -- 172.21.15.5:0/2763569728 shutdown_connections 2024-09-10T09:53:24.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:24.845+0000 7fc979609740 1 --2- 172.21.15.5:0/2763569728 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563fc9988020 0x563fc998a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:24.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:24.845+0000 7fc979609740 1 --2- 172.21.15.5:0/2763569728 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563fc997d760 0x563fc9987ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:24.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:24.845+0000 7fc979609740 1 --2- 172.21.15.5:0/2763569728 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563fc997ce40 0x563fc997d220 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:24.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:24.845+0000 7fc979609740 1 -- 172.21.15.5:0/2763569728 >> 172.21.15.5:0/2763569728 conn(0x563fc99793d0 msgr2=0x563fc99797b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:53:24.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:24.845+0000 7fc979609740 1 -- 172.21.15.5:0/2763569728 shutdown_connections 2024-09-10T09:53:24.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:24.845+0000 7fc979609740 1 -- 172.21.15.5:0/2763569728 wait complete. 2024-09-10T09:53:24.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:24.846+0000 7fc979609740 1 -- start start 2024-09-10T09:53:24.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:24.846+0000 7fc979609740 1 Processor -- start 2024-09-10T09:53:24.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:24.846+0000 7fc979609740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563fc997ce40 0x563fc9984f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:24.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:24.846+0000 7fc979609740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563fc997d760 0x563fc9985480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:24.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:24.846+0000 7fc979609740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563fc9988020 0x563fc99859c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:24.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:24.846+0000 7fc979609740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x563fc998abf0 con 0x563fc997d760 2024-09-10T09:53:24.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:24.846+0000 7fc979609740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x563fc989e0f0 con 0x563fc997ce40 2024-09-10T09:53:24.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:24.846+0000 7fc979609740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x563fc998ad70 con 0x563fc9988020 2024-09-10T09:53:24.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:24.846+0000 7fc979609740 10 client.0 _ll_register_callbacks cb 0x563fc9885bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:53:24.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:24.847+0000 7fc96fe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563fc997ce40 0x563fc9984f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:24.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:24.847+0000 7fc96fe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563fc997ce40 0x563fc9984f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:49904/0 (socket says 172.21.15.5:49904) 2024-09-10T09:53:24.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:24.847+0000 7fc96fe00640 1 -- 172.21.15.5:0/3102347400 learned_addr learned my addr 172.21.15.5:0/3102347400 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:53:24.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:24.847+0000 7fc974a00640 1 --2- 172.21.15.5:0/3102347400 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563fc9988020 0x563fc99859c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:24.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:24.847+0000 7fc96f400640 1 --2- 172.21.15.5:0/3102347400 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563fc997d760 0x563fc9985480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:24.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:24.848+0000 7fc974a00640 1 -- 172.21.15.5:0/3102347400 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563fc997ce40 msgr2=0x563fc9984f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:53:24.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:24.848+0000 7fc974a00640 1 --2- 172.21.15.5:0/3102347400 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563fc997ce40 0x563fc9984f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:24.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:24.848+0000 7fc974a00640 1 -- 172.21.15.5:0/3102347400 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563fc997d760 msgr2=0x563fc9985480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:53:24.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:24.848+0000 7fc974a00640 1 --2- 172.21.15.5:0/3102347400 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563fc997d760 0x563fc9985480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:24.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:24.848+0000 7fc974a00640 1 -- 172.21.15.5:0/3102347400 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563fc96543c0 con 0x563fc9988020 2024-09-10T09:53:24.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:24.848+0000 7fc96f400640 1 --2- 172.21.15.5:0/3102347400 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563fc997d760 0x563fc9985480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:53:24.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:24.848+0000 7fc974a00640 1 --2- 172.21.15.5:0/3102347400 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563fc9988020 0x563fc99859c0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fc9680edda0 tx=0x7fc9680bc490 comp rx=0 tx=0).ready entity=mon.1 client_cookie=89126e568dcc80a6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:53:24.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:24.848+0000 7fc959600640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-09-10T09:53:24.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:24.848+0000 7fc959600640 1 -- 172.21.15.5:0/3102347400 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9680bde60 con 0x563fc9988020 2024-09-10T09:53:24.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:24.848+0000 7fc959600640 1 -- 172.21.15.5:0/3102347400 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fc9680f03b0 con 0x563fc9988020 2024-09-10T09:53:24.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:24.848+0000 7fc979609740 1 -- 172.21.15.5:0/3102347400 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc95c010430 con 0x563fc9988020 2024-09-10T09:53:24.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:24.849+0000 7fc959600640 1 -- 172.21.15.5:0/3102347400 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fc9680bcb30 con 0x563fc9988020 2024-09-10T09:53:24.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:24.849+0000 7fc979609740 10 client.28414 fetch_fsmap learned FSMap version 37 2024-09-10T09:53:24.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:24.849+0000 7fc979609740 1 -- 172.21.15.5:0/3102347400 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x563fc998aca0 con 0x563fc9988020 2024-09-10T09:53:24.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:24.849+0000 7fc959600640 1 -- 172.21.15.5:0/3102347400 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc95c010430 con 0x563fc9988020 2024-09-10T09:53:24.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:24.849+0000 7fc959600640 1 -- 172.21.15.5:0/3102347400 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 37) ==== 8390+0+0 (secure 0 0 0) 0x7fc9680c5310 con 0x563fc9988020 2024-09-10T09:53:24.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:24.849+0000 7fc979609740 10 client.28414 fetch_fsmap finished waiting for FSMap version 37 2024-09-10T09:53:24.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:24.849+0000 7fc979609740 10 client.28414 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T09:53:24.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:24.849+0000 7fc979609740 20 client.28414 populate_metadata read hostname 'smithi005' 2024-09-10T09:53:24.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:24.850+0000 7fc979609740 1 --2- 172.21.15.5:0/3102347400 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x563fc99980f0 0x563fc999a520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:24.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:24.850+0000 7fc979609740 4 client.28414 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:53:24.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:24.850+0000 7fc979609740 1 -- 172.21.15.5:0/3102347400 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563fc98f0950 con 0x563fc99980f0 2024-09-10T09:53:24.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:24.850+0000 7fc96fe00640 1 --2- 172.21.15.5:0/3102347400 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x563fc99980f0 0x563fc999a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:24.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:24.851+0000 7fc959600640 10 client.28414 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T09:53:24.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:24.851+0000 7fc96fe00640 1 --2- 172.21.15.5:0/3102347400 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x563fc99980f0 0x563fc999a520 crc :-1 s=READY pgs=221 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c889330d2ccbc6f2 server_cookie=3da0fd08b205821e in_seq=0 out_seq=0 2024-09-10T09:53:24.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:24.852+0000 7fc959600640 1 -- 172.21.15.5:0/3102347400 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563fc98f0950 con 0x563fc99980f0 2024-09-10T09:53:24.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:24.852+0000 7fc959600640 10 client.28414 handle_command_reply: tid=0 2024-09-10T09:53:24.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:24.852+0000 7fc979609740 1 client.28414 shutdown 2024-09-10T09:53:24.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:24.853+0000 7fc979609740 1 -- 172.21.15.5:0/3102347400 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563fc9988020 msgr2=0x563fc99859c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:53:24.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:24.853+0000 7fc979609740 1 --2- 172.21.15.5:0/3102347400 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563fc9988020 0x563fc99859c0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fc9680edda0 tx=0x7fc9680bc490 comp rx=0 tx=0).stop 2024-09-10T09:53:24.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:24.853+0000 7fc979609740 1 -- 172.21.15.5:0/3102347400 shutdown_connections 2024-09-10T09:53:24.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:24.853+0000 7fc979609740 1 --2- 172.21.15.5:0/3102347400 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x563fc99980f0 0x563fc999a520 crc :-1 s=READY pgs=221 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:24.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:24.853+0000 7fc979609740 1 --2- 172.21.15.5:0/3102347400 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563fc9988020 0x563fc99859c0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:24.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:24.853+0000 7fc979609740 1 --2- 172.21.15.5:0/3102347400 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563fc997d760 0x563fc9985480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:24.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:24.853+0000 7fc959600640 0 client.28414 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T09:53:24.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:24.853+0000 7fc979609740 1 --2- 172.21.15.5:0/3102347400 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563fc997ce40 0x563fc9984f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:24.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:24.853+0000 7fc979609740 1 -- 172.21.15.5:0/3102347400 >> 172.21.15.5:0/3102347400 conn(0x563fc99793d0 msgr2=0x563fc997f100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:53:24.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:24.853+0000 7fc979609740 1 -- 172.21.15.5:0/3102347400 shutdown_connections 2024-09-10T09:53:24.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:24.853+0000 7fc979609740 1 -- 172.21.15.5:0/3102347400 wait complete. 2024-09-10T09:53:24.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:24.854+0000 7fc979609740 20 client.28414 trim_cache size 0 max 16384 2024-09-10T09:53:24.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:24.874+0000 7fc979609740 1 Processor -- start 2024-09-10T09:53:24.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:24.874+0000 7fc979609740 1 -- start start 2024-09-10T09:53:24.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:24.874+0000 7fc979609740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563fc997ce40 0x563fc997f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:24.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:24.874+0000 7fc979609740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563fc997d760 0x563fc99797b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:24.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:24.874+0000 7fc979609740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563fc9988020 0x563fc999fe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:24.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:24.874+0000 7fc979609740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc9701ac8d0 con 0x563fc9988020 2024-09-10T09:53:24.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:24.874+0000 7fc979609740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fc9701ae500 con 0x563fc997ce40 2024-09-10T09:53:24.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:24.874+0000 7fc979609740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fc9701acfa0 con 0x563fc997d760 2024-09-10T09:53:24.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:24.875+0000 7fc96f400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563fc997d760 0x563fc99797b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:24.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:24.875+0000 7fc96f400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563fc997d760 0x563fc99797b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:52134/0 (socket says 172.21.15.5:52134) 2024-09-10T09:53:24.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:24.875+0000 7fc96f400640 1 -- 172.21.15.5:0/1011695996 learned_addr learned my addr 172.21.15.5:0/1011695996 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:53:24.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:24.875+0000 7fc974a00640 1 --2- 172.21.15.5:0/1011695996 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563fc9988020 0x563fc999fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:24.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:24.875+0000 7fc96fe00640 1 --2- 172.21.15.5:0/1011695996 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563fc997ce40 0x563fc997f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:24.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:24.876+0000 7fc96f400640 1 -- 172.21.15.5:0/1011695996 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563fc997ce40 msgr2=0x563fc997f8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:53:24.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:24.876+0000 7fc96f400640 1 --2- 172.21.15.5:0/1011695996 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563fc997ce40 0x563fc997f8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:24.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:24.876+0000 7fc96f400640 1 -- 172.21.15.5:0/1011695996 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563fc9988020 msgr2=0x563fc999fe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:53:24.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:24.876+0000 7fc96f400640 1 --2- 172.21.15.5:0/1011695996 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563fc9988020 0x563fc999fe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:24.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:24.876+0000 7fc96f400640 1 -- 172.21.15.5:0/1011695996 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563fc9983ef0 con 0x563fc997d760 2024-09-10T09:53:24.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:24.876+0000 7fc96f400640 1 --2- 172.21.15.5:0/1011695996 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563fc997d760 0x563fc99797b0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fc95c0093e0 tx=0x7fc95c009cb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d55e127350a5db3d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:53:24.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:24.877+0000 7fc95be00640 1 -- 172.21.15.5:0/1011695996 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc95c005c50 con 0x563fc997d760 2024-09-10T09:53:24.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:24.877+0000 7fc95be00640 1 -- 172.21.15.5:0/1011695996 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc95c00fda0 con 0x563fc997d760 2024-09-10T09:53:24.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:24.877+0000 7fc95be00640 1 -- 172.21.15.5:0/1011695996 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc95c003850 con 0x563fc997d760 2024-09-10T09:53:24.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:24.878+0000 7fc979609740 1 -- 172.21.15.5:0/1011695996 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563fc997d760 msgr2=0x563fc99797b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:53:24.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:24.878+0000 7fc979609740 1 --2- 172.21.15.5:0/1011695996 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563fc997d760 0x563fc99797b0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fc95c0093e0 tx=0x7fc95c009cb0 comp rx=0 tx=0).stop 2024-09-10T09:53:24.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:24.878+0000 7fc979609740 1 -- 172.21.15.5:0/1011695996 shutdown_connections 2024-09-10T09:53:24.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:24.878+0000 7fc979609740 1 --2- 172.21.15.5:0/1011695996 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563fc9988020 0x563fc999fe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:24.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:24.878+0000 7fc979609740 1 --2- 172.21.15.5:0/1011695996 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563fc997d760 0x563fc99797b0 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:24.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:24.878+0000 7fc979609740 1 --2- 172.21.15.5:0/1011695996 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563fc997ce40 0x563fc997f8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:24.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:24.878+0000 7fc979609740 1 -- 172.21.15.5:0/1011695996 >> 172.21.15.5:0/1011695996 conn(0x563fc99793d0 msgr2=0x563fc9986fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:53:24.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:24.878+0000 7fc979609740 1 -- 172.21.15.5:0/1011695996 shutdown_connections 2024-09-10T09:53:24.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:24.878+0000 7fc979609740 1 -- 172.21.15.5:0/1011695996 wait complete. 2024-09-10T09:53:24.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:24.879+0000 7fc979609740 1 -- start start 2024-09-10T09:53:24.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:24.879+0000 7fc979609740 1 Processor -- start 2024-09-10T09:53:24.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:24.880+0000 7fc979609740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563fc997ce40 0x563fc998c060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:24.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:24.880+0000 7fc979609740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563fc997d760 0x563fc998c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:24.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:24.880+0000 7fc979609740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563fc9988020 0x563fc9a01e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:24.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:24.880+0000 7fc979609740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc9701ade00 con 0x563fc997ce40 2024-09-10T09:53:24.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:24.880+0000 7fc979609740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fc9701ad730 con 0x563fc997d760 2024-09-10T09:53:24.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:24.880+0000 7fc96fe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563fc997ce40 0x563fc998c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:24.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:24.880+0000 7fc96fe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563fc997ce40 0x563fc998c060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53592/0 (socket says 172.21.15.5:53592) 2024-09-10T09:53:24.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:24.880+0000 7fc96fe00640 1 -- 172.21.15.5:0/1619608719 learned_addr learned my addr 172.21.15.5:0/1619608719 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:53:24.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:24.881+0000 7fc96f400640 1 --2- 172.21.15.5:0/1619608719 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563fc997d760 0x563fc998c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:24.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:24.881+0000 7fc974a00640 1 --2- 172.21.15.5:0/1619608719 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563fc9988020 0x563fc9a01e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:24.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:24.881+0000 7fc979609740 1 -- 172.21.15.5:0/1619608719 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fc970077e60 con 0x563fc9988020 2024-09-10T09:53:24.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:24.881+0000 7fc979609740 10 client.0 _ll_register_callbacks cb 0x7fc9680bc900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:53:24.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:24.882+0000 7fc96fe00640 1 -- 172.21.15.5:0/1619608719 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563fc9988020 msgr2=0x563fc9a01e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:53:24.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:24.882+0000 7fc96fe00640 1 --2- 172.21.15.5:0/1619608719 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563fc9988020 0x563fc9a01e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:24.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:24.882+0000 7fc96fe00640 1 -- 172.21.15.5:0/1619608719 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563fc997d760 msgr2=0x563fc998c5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:53:24.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:24.882+0000 7fc96fe00640 1 --2- 172.21.15.5:0/1619608719 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563fc997d760 0x563fc998c5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:24.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:24.882+0000 7fc96fe00640 1 -- 172.21.15.5:0/1619608719 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563fc998ce80 con 0x563fc997ce40 2024-09-10T09:53:24.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:24.882+0000 7fc96fe00640 1 --2- 172.21.15.5:0/1619608719 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563fc997ce40 0x563fc998c060 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x563fc999add0 tx=0x7fc9600046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9701fa68a9b883cb server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:53:24.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:24.882+0000 7fc959600640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-10T09:53:24.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:24.883+0000 7fc959600640 1 -- 172.21.15.5:0/1619608719 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc96002e030 con 0x563fc997ce40 2024-09-10T09:53:24.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:24.883+0000 7fc979609740 1 -- 172.21.15.5:0/1619608719 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc95c00fda0 con 0x563fc997ce40 2024-09-10T09:53:24.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:24.883+0000 7fc959600640 1 -- 172.21.15.5:0/1619608719 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fc96002f070 con 0x563fc997ce40 2024-09-10T09:53:24.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:24.884+0000 7fc959600640 1 -- 172.21.15.5:0/1619608719 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fc96003b050 con 0x563fc997ce40 2024-09-10T09:53:24.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:24.884+0000 7fc979609740 10 client.18375 fetch_fsmap learned FSMap version 37 2024-09-10T09:53:24.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:24.884+0000 7fc979609740 1 -- 172.21.15.5:0/1619608719 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x563fc9a034d0 con 0x563fc997ce40 2024-09-10T09:53:24.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:24.885+0000 7fc959600640 1 -- 172.21.15.5:0/1619608719 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc95c00fda0 con 0x563fc997ce40 2024-09-10T09:53:24.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:24.885+0000 7fc959600640 1 -- 172.21.15.5:0/1619608719 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 37) ==== 8390+0+0 (secure 0 0 0) 0x7fc960033430 con 0x563fc997ce40 2024-09-10T09:53:24.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:24.886+0000 7fc979609740 10 client.18375 fetch_fsmap finished waiting for FSMap version 37 2024-09-10T09:53:24.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:24.886+0000 7fc979609740 10 client.18375 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T09:53:24.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:24.886+0000 7fc979609740 20 client.18375 populate_metadata read hostname 'smithi005' 2024-09-10T09:53:24.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:24.886+0000 7fc979609740 1 --2- 172.21.15.5:0/1619608719 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x563fc9a03c50 0x563fc9a06070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:24.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:24.886+0000 7fc979609740 4 client.18375 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T09:53:24.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:24.886+0000 7fc979609740 1 -- 172.21.15.5:0/1619608719 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563fc98eae60 con 0x563fc9a03c50 2024-09-10T09:53:24.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:24.886+0000 7fc96f400640 1 --2- 172.21.15.5:0/1619608719 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x563fc9a03c50 0x563fc9a06070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:24.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:24.887+0000 7fc96f400640 1 --2- 172.21.15.5:0/1619608719 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x563fc9a03c50 0x563fc9a06070 crc :-1 s=READY pgs=222 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=58c40c467af50187 server_cookie=f830629d0076a919 in_seq=0 out_seq=0 2024-09-10T09:53:24.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:24.887+0000 7fc959600640 10 client.18375 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T09:53:24.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:24.887+0000 7fc959600640 1 -- 172.21.15.5:0/1619608719 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563fc98eae60 con 0x563fc9a03c50 2024-09-10T09:53:24.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:24.887+0000 7fc959600640 10 client.18375 handle_command_reply: tid=0 2024-09-10T09:53:24.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:24.887+0000 7fc979609740 1 client.18375 shutdown 2024-09-10T09:53:24.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:24.888+0000 7fc979609740 1 -- 172.21.15.5:0/1619608719 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563fc997ce40 msgr2=0x563fc998c060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:53:24.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:24.888+0000 7fc979609740 1 --2- 172.21.15.5:0/1619608719 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563fc997ce40 0x563fc998c060 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x563fc999add0 tx=0x7fc9600046c0 comp rx=0 tx=0).stop 2024-09-10T09:53:24.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:24.888+0000 7fc979609740 1 -- 172.21.15.5:0/1619608719 shutdown_connections 2024-09-10T09:53:24.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:24.888+0000 7fc979609740 1 --2- 172.21.15.5:0/1619608719 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x563fc9a03c50 0x563fc9a06070 crc :-1 s=READY pgs=222 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:24.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:24.888+0000 7fc979609740 1 --2- 172.21.15.5:0/1619608719 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563fc9988020 0x563fc9a01e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:24.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:24.888+0000 7fc959600640 0 client.18375 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T09:53:24.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:24.888+0000 7fc979609740 1 --2- 172.21.15.5:0/1619608719 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563fc997d760 0x563fc998c5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:24.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:24.888+0000 7fc979609740 1 --2- 172.21.15.5:0/1619608719 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563fc997ce40 0x563fc998c060 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:24.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:24.888+0000 7fc979609740 1 -- 172.21.15.5:0/1619608719 >> 172.21.15.5:0/1619608719 conn(0x563fc99793d0 msgr2=0x563fc99867a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:53:24.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:24.889+0000 7fc979609740 1 -- 172.21.15.5:0/1619608719 shutdown_connections 2024-09-10T09:53:24.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:24.890+0000 7fc979609740 1 -- 172.21.15.5:0/1619608719 wait complete. 2024-09-10T09:53:24.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:24.890+0000 7fc979609740 20 client.18375 trim_cache size 0 max 16384 2024-09-10T09:53:24.892 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-10T09:53:24.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:24.891+0000 7fc975400640 1 -- 172.21.15.5:0/1829210188 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fc9540a2b80 msgr2=0x7fc9540a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:53:24.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:24.891+0000 7fc975400640 1 --2- 172.21.15.5:0/1829210188 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fc9540a2b80 0x7fc9540a5040 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fc96000afd0 tx=0x7fc96000a380 comp rx=0 tx=0).stop 2024-09-10T09:53:24.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:24.891+0000 7fc975400640 1 -- 172.21.15.5:0/1829210188 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc970112870 msgr2=0x7fc97019e690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:53:24.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:24.891+0000 7fc975400640 1 --2- 172.21.15.5:0/1829210188 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc970112870 0x7fc97019e690 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fc96802aea0 tx=0x7fc968003f10 comp rx=0 tx=0).stop 2024-09-10T09:53:24.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:24.891+0000 7fc975400640 1 -- 172.21.15.5:0/1829210188 shutdown_connections 2024-09-10T09:53:24.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:24.891+0000 7fc975400640 1 --2- 172.21.15.5:0/1829210188 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fc9540a2b80 0x7fc9540a5040 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:24.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:24.891+0000 7fc975400640 1 --2- 172.21.15.5:0/1829210188 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc970112870 0x7fc97019e690 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:24.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:24.891+0000 7fc975400640 1 --2- 172.21.15.5:0/1829210188 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc970077040 0x7fc9701a4b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:24.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:24.891+0000 7fc975400640 1 --2- 172.21.15.5:0/1829210188 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc970076720 0x7fc9701a45c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:24.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:24.891+0000 7fc975400640 1 -- 172.21.15.5:0/1829210188 >> 172.21.15.5:0/1829210188 conn(0x7fc9700ff910 msgr2=0x7fc970101d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:53:24.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:24.892+0000 7fc975400640 1 -- 172.21.15.5:0/1829210188 shutdown_connections 2024-09-10T09:53:24.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:24.892+0000 7fc975400640 1 -- 172.21.15.5:0/1829210188 wait complete. 2024-09-10T09:53:24.949 INFO:tasks.workunit.client.0.smithi005.stdout: 10 25762 24.85 MB/sec warmup 63 sec latency 246.233 ms 2024-09-10T09:53:25.306 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:53:24 smithi005 ceph-mon[30350]: pgmap v845: 129 pgs: 129 active+clean; 537 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 5.3 MiB/s wr, 471 op/s 2024-09-10T09:53:25.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:53:24 smithi159 ceph-mon[35513]: pgmap v845: 129 pgs: 129 active+clean; 537 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 5.3 MiB/s wr, 471 op/s 2024-09-10T09:53:25.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:53:24 smithi102 ceph-mon[35442]: pgmap v845: 129 pgs: 129 active+clean; 537 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 5.3 MiB/s wr, 471 op/s 2024-09-10T09:53:25.950 INFO:tasks.workunit.client.0.smithi005.stdout: 10 26323 24.87 MB/sec warmup 64 sec latency 96.295 ms 2024-09-10T09:53:26.843 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-10T09:53:26.843 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-10T09:53:26.950 INFO:tasks.workunit.client.0.smithi005.stdout: 10 27107 25.15 MB/sec warmup 65 sec latency 85.371 ms 2024-09-10T09:53:27.307 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:53:26 smithi005 ceph-mon[30350]: pgmap v846: 129 pgs: 129 active+clean; 532 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 5.0 MiB/s wr, 460 op/s 2024-09-10T09:53:27.339 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:53:27.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:53:26 smithi159 ceph-mon[35513]: pgmap v846: 129 pgs: 129 active+clean; 532 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 5.0 MiB/s wr, 460 op/s 2024-09-10T09:53:27.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:53:26 smithi102 ceph-mon[35442]: pgmap v846: 129 pgs: 129 active+clean; 532 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 5.0 MiB/s wr, 460 op/s 2024-09-10T09:53:27.950 INFO:tasks.workunit.client.0.smithi005.stdout: 10 27989 25.35 MB/sec warmup 66 sec latency 49.421 ms 2024-09-10T09:53:28.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:28.047+0000 7f5a3ec00640 1 -- 172.21.15.5:0/2161924156 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5a38101430 msgr2=0x7f5a38105900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:53:28.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:28.047+0000 7f5a3ec00640 1 --2- 172.21.15.5:0/2161924156 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5a38101430 0x7f5a38105900 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f5a28009900 tx=0x7f5a2802a470 comp rx=0 tx=0).stop 2024-09-10T09:53:28.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:28.048+0000 7f5a3ec00640 1 -- 172.21.15.5:0/2161924156 shutdown_connections 2024-09-10T09:53:28.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:28.048+0000 7f5a3ec00640 1 --2- 172.21.15.5:0/2161924156 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5a38105f20 0x7f5a381063b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:28.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:28.048+0000 7f5a3ec00640 1 --2- 172.21.15.5:0/2161924156 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5a38101430 0x7f5a38105900 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:28.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:28.048+0000 7f5a3ec00640 1 --2- 172.21.15.5:0/2161924156 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5a38100b10 0x7f5a38100ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:28.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:28.048+0000 7f5a3ec00640 1 -- 172.21.15.5:0/2161924156 >> 172.21.15.5:0/2161924156 conn(0x7f5a380fc780 msgr2=0x7f5a380feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:53:28.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:28.048+0000 7f5a3ec00640 1 -- 172.21.15.5:0/2161924156 shutdown_connections 2024-09-10T09:53:28.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:28.048+0000 7f5a3ec00640 1 -- 172.21.15.5:0/2161924156 wait complete. 2024-09-10T09:53:28.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:28.049+0000 7f5a3ec00640 1 Processor -- start 2024-09-10T09:53:28.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:28.049+0000 7f5a3ec00640 1 -- start start 2024-09-10T09:53:28.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:28.050+0000 7f5a3ec00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5a38100b10 0x7f5a3819ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:28.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:28.050+0000 7f5a3ec00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5a38101430 0x7f5a3819f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:28.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:28.050+0000 7f5a3ec00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5a38105f20 0x7f5a381a34d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:28.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:28.050+0000 7f5a3ec00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f5a38118570 con 0x7f5a38101430 2024-09-10T09:53:28.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:28.050+0000 7f5a3ec00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f5a381183f0 con 0x7f5a38105f20 2024-09-10T09:53:28.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:28.050+0000 7f5a3ec00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f5a381186f0 con 0x7f5a38100b10 2024-09-10T09:53:28.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:28.050+0000 7f5a3cc00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5a38105f20 0x7f5a381a34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:28.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:28.050+0000 7f5a37e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5a38100b10 0x7f5a3819ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:28.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:28.050+0000 7f5a3cc00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5a38105f20 0x7f5a381a34d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:49944/0 (socket says 172.21.15.5:49944) 2024-09-10T09:53:28.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:28.050+0000 7f5a3cc00640 1 -- 172.21.15.5:0/4079256983 learned_addr learned my addr 172.21.15.5:0/4079256983 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:53:28.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:28.050+0000 7f5a37e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5a38100b10 0x7f5a3819ebd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:52156/0 (socket says 172.21.15.5:52156) 2024-09-10T09:53:28.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:28.051+0000 7f5a37400640 1 --2- 172.21.15.5:0/4079256983 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5a38101430 0x7f5a3819f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:28.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:28.051+0000 7f5a37e00640 1 -- 172.21.15.5:0/4079256983 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5a38105f20 msgr2=0x7f5a381a34d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:53:28.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:28.051+0000 7f5a37e00640 1 --2- 172.21.15.5:0/4079256983 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5a38105f20 0x7f5a381a34d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:28.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:28.051+0000 7f5a37e00640 1 -- 172.21.15.5:0/4079256983 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5a38101430 msgr2=0x7f5a3819f110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:53:28.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:28.051+0000 7f5a37e00640 1 --2- 172.21.15.5:0/4079256983 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5a38101430 0x7f5a3819f110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:28.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:28.051+0000 7f5a37e00640 1 -- 172.21.15.5:0/4079256983 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a381a3c00 con 0x7f5a38100b10 2024-09-10T09:53:28.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:28.052+0000 7f5a37e00640 1 --2- 172.21.15.5:0/4079256983 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5a38100b10 0x7f5a3819ebd0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f5a3000e9f0 tx=0x7f5a3000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:53:28.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:28.053+0000 7f5a34c00640 1 -- 172.21.15.5:0/4079256983 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a3000cd80 con 0x7f5a38100b10 2024-09-10T09:53:28.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:28.053+0000 7f5a34c00640 1 -- 172.21.15.5:0/4079256983 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5a30011950 con 0x7f5a38100b10 2024-09-10T09:53:28.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:28.053+0000 7f5a34c00640 1 -- 172.21.15.5:0/4079256983 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a30010a40 con 0x7f5a38100b10 2024-09-10T09:53:28.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:28.054+0000 7f5a3ec00640 1 -- 172.21.15.5:0/4079256983 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a381a3ef0 con 0x7f5a38100b10 2024-09-10T09:53:28.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:28.054+0000 7f5a3ec00640 1 -- 172.21.15.5:0/4079256983 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a381ab780 con 0x7f5a38100b10 2024-09-10T09:53:28.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:28.059+0000 7f5a34c00640 1 -- 172.21.15.5:0/4079256983 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f5a30011b00 con 0x7f5a38100b10 2024-09-10T09:53:28.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:28.061+0000 7f5a34c00640 1 --2- 172.21.15.5:0/4079256983 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f5a1c0a2c40 0x7f5a1c0a5100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:28.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:28.061+0000 7f5a34c00640 1 -- 172.21.15.5:0/4079256983 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f5a300cf8c0 con 0x7f5a38100b10 2024-09-10T09:53:28.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:28.062+0000 7f5a417ec740 1 Processor -- start 2024-09-10T09:53:28.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:28.062+0000 7f5a37400640 1 --2- 172.21.15.5:0/4079256983 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f5a1c0a2c40 0x7f5a1c0a5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:28.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:28.063+0000 7f5a417ec740 1 -- start start 2024-09-10T09:53:28.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:28.063+0000 7f5a417ec740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d335253ac0 0x55d335253ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:28.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:28.063+0000 7f5a417ec740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d3352543e0 0x55d33525e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:28.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:28.063+0000 7f5a417ec740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d33525eca0 0x55d335261090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:28.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:28.063+0000 7f5a417ec740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55d3351ba290 con 0x55d335253ac0 2024-09-10T09:53:28.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:28.063+0000 7f5a417ec740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55d335161b40 con 0x55d3352543e0 2024-09-10T09:53:28.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:28.063+0000 7f5a417ec740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55d3351b6250 con 0x55d33525eca0 2024-09-10T09:53:28.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:28.063+0000 7f5a37e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d335253ac0 0x55d335253ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:28.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:28.064+0000 7f5a37e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d335253ac0 0x55d335253ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53618/0 (socket says 172.21.15.5:53618) 2024-09-10T09:53:28.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:28.064+0000 7f5a37e00640 1 -- 172.21.15.5:0/4196205233 learned_addr learned my addr 172.21.15.5:0/4196205233 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:53:28.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:28.064+0000 7f5a3cc00640 1 --2- 172.21.15.5:0/4196205233 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d33525eca0 0x55d335261090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:28.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:28.064+0000 7f5a3cc00640 1 -- 172.21.15.5:0/4196205233 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d3352543e0 msgr2=0x55d33525e760 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T09:53:28.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:28.064+0000 7f5a3cc00640 1 --2- 172.21.15.5:0/4196205233 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d3352543e0 0x55d33525e760 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:28.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:28.064+0000 7f5a3cc00640 1 -- 172.21.15.5:0/4196205233 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d335253ac0 msgr2=0x55d335253ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:53:28.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:28.064+0000 7f5a3cc00640 1 --2- 172.21.15.5:0/4196205233 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d335253ac0 0x55d335253ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:28.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:28.064+0000 7f5a3cc00640 1 -- 172.21.15.5:0/4196205233 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d334f2b6a0 con 0x55d33525eca0 2024-09-10T09:53:28.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:28.065+0000 7f5a37400640 1 --2- 172.21.15.5:0/4079256983 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f5a1c0a2c40 0x7f5a1c0a5100 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f5a28000c00 tx=0x7f5a28004110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:53:28.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:28.068+0000 7f5a3cc00640 1 --2- 172.21.15.5:0/4196205233 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d33525eca0 0x55d335261090 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f5a20005fd0 tx=0x7f5a200107a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b0d695dd6cba6d1c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:53:28.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:28.070+0000 7f5a2d200640 1 -- 172.21.15.5:0/4196205233 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a20005020 con 0x55d33525eca0 2024-09-10T09:53:28.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:28.070+0000 7f5a2d200640 1 -- 172.21.15.5:0/4196205233 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5a200047a0 con 0x55d33525eca0 2024-09-10T09:53:28.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:28.070+0000 7f5a2d200640 1 -- 172.21.15.5:0/4196205233 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a200116a0 con 0x55d33525eca0 2024-09-10T09:53:28.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:28.072+0000 7f5a417ec740 1 -- 172.21.15.5:0/4196205233 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d33525eca0 msgr2=0x55d335261090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:53:28.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:28.072+0000 7f5a417ec740 1 --2- 172.21.15.5:0/4196205233 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d33525eca0 0x55d335261090 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f5a20005fd0 tx=0x7f5a200107a0 comp rx=0 tx=0).stop 2024-09-10T09:53:28.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:28.073+0000 7f5a417ec740 1 -- 172.21.15.5:0/4196205233 shutdown_connections 2024-09-10T09:53:28.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:28.073+0000 7f5a417ec740 1 --2- 172.21.15.5:0/4196205233 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d33525eca0 0x55d335261090 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:28.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:28.073+0000 7f5a417ec740 1 --2- 172.21.15.5:0/4196205233 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d3352543e0 0x55d33525e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:28.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:28.073+0000 7f5a417ec740 1 --2- 172.21.15.5:0/4196205233 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d335253ac0 0x55d335253ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:28.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:28.073+0000 7f5a417ec740 1 -- 172.21.15.5:0/4196205233 >> 172.21.15.5:0/4196205233 conn(0x55d335250050 msgr2=0x55d335250430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:53:28.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:28.074+0000 7f5a417ec740 1 -- 172.21.15.5:0/4196205233 shutdown_connections 2024-09-10T09:53:28.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:28.075+0000 7f5a417ec740 1 -- 172.21.15.5:0/4196205233 wait complete. 2024-09-10T09:53:28.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:28.075+0000 7f5a417ec740 1 -- start start 2024-09-10T09:53:28.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:28.076+0000 7f5a417ec740 1 Processor -- start 2024-09-10T09:53:28.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:28.076+0000 7f5a417ec740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d335253ac0 0x55d33525e660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:28.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:28.076+0000 7f5a417ec740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d3352543e0 0x55d33525b910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:28.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:28.077+0000 7f5a417ec740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d33525eca0 0x55d33525bf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:28.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:28.077+0000 7f5a37e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d335253ac0 0x55d33525e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:28.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:28.077+0000 7f5a37e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d335253ac0 0x55d33525e660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:52180/0 (socket says 172.21.15.5:52180) 2024-09-10T09:53:28.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:28.078+0000 7f5a3cc00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d33525eca0 0x55d33525bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:28.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:28.079+0000 7f5a37400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d3352543e0 0x55d33525b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:28.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:28.079+0000 7f5a37400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d3352543e0 0x55d33525b910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:49964/0 (socket says 172.21.15.5:49964) 2024-09-10T09:53:28.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:28.079+0000 7f5a3cc00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d33525eca0 0x55d33525bf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53626/0 (socket says 172.21.15.5:53626) 2024-09-10T09:53:28.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:28.080+0000 7f5a417ec740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55d335272920 con 0x55d33525eca0 2024-09-10T09:53:28.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:28.080+0000 7f5a417ec740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55d335174fa0 con 0x55d3352543e0 2024-09-10T09:53:28.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:28.080+0000 7f5a417ec740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55d335272aa0 con 0x55d335253ac0 2024-09-10T09:53:28.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:28.080+0000 7f5a417ec740 10 client.0 _ll_register_callbacks cb 0x55d334ec8ba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:53:28.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:28.080+0000 7f5a37e00640 1 -- 172.21.15.5:0/3792007163 learned_addr learned my addr 172.21.15.5:0/3792007163 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:53:28.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:28.081+0000 7f5a37e00640 1 -- 172.21.15.5:0/3792007163 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d3352543e0 msgr2=0x55d33525b910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:53:28.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:28.081+0000 7f5a37e00640 1 --2- 172.21.15.5:0/3792007163 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d3352543e0 0x55d33525b910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:28.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:28.081+0000 7f5a37e00640 1 -- 172.21.15.5:0/3792007163 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d33525eca0 msgr2=0x55d33525bf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:53:28.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:28.082+0000 7f5a37e00640 1 --2- 172.21.15.5:0/3792007163 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d33525eca0 0x55d33525bf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:28.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:28.082+0000 7f5a37e00640 1 -- 172.21.15.5:0/3792007163 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d33525c730 con 0x55d335253ac0 2024-09-10T09:53:28.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:28.082+0000 7f5a3cc00640 1 --2- 172.21.15.5:0/3792007163 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d33525eca0 0x55d33525bf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:53:28.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:28.082+0000 7f5a37400640 1 --2- 172.21.15.5:0/3792007163 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d3352543e0 0x55d33525b910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T09:53:28.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:28.083+0000 7f5a37e00640 1 --2- 172.21.15.5:0/3792007163 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d335253ac0 0x55d33525e660 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f5a300cd340 tx=0x7f5a3009c4f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e6c6bba7677cb3c6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:53:28.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:28.083+0000 7f5a25600640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-09-10T09:53:28.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:28.083+0000 7f5a25600640 1 -- 172.21.15.5:0/3792007163 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a300a4710 con 0x55d335253ac0 2024-09-10T09:53:28.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:28.083+0000 7f5a25600640 1 -- 172.21.15.5:0/3792007163 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f5a300ba5e0 con 0x55d335253ac0 2024-09-10T09:53:28.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:28.084+0000 7f5a417ec740 1 -- 172.21.15.5:0/3792007163 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5a200047a0 con 0x55d335253ac0 2024-09-10T09:53:28.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:28.084+0000 7f5a25600640 1 -- 172.21.15.5:0/3792007163 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f5a300a49a0 con 0x55d335253ac0 2024-09-10T09:53:28.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:28.084+0000 7f5a417ec740 10 client.28432 fetch_fsmap learned FSMap version 37 2024-09-10T09:53:28.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:28.084+0000 7f5a417ec740 1 -- 172.21.15.5:0/3792007163 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55d33527e910 con 0x55d335253ac0 2024-09-10T09:53:28.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:28.085+0000 7f5a25600640 1 -- 172.21.15.5:0/3792007163 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5a200047a0 con 0x55d335253ac0 2024-09-10T09:53:28.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:28.085+0000 7f5a25600640 1 -- 172.21.15.5:0/3792007163 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 37) ==== 8390+0+0 (secure 0 0 0) 0x7f5a300b2070 con 0x55d335253ac0 2024-09-10T09:53:28.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:28.085+0000 7f5a417ec740 10 client.28432 fetch_fsmap finished waiting for FSMap version 37 2024-09-10T09:53:28.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:28.085+0000 7f5a417ec740 10 client.28432 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T09:53:28.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:28.086+0000 7f5a417ec740 20 client.28432 populate_metadata read hostname 'smithi005' 2024-09-10T09:53:28.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:28.086+0000 7f5a417ec740 1 --2- 172.21.15.5:0/3792007163 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55d33527fcb0 0x55d3352820c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:28.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:28.086+0000 7f5a417ec740 4 client.28432 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:53:28.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:28.086+0000 7f5a417ec740 1 -- 172.21.15.5:0/3792007163 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d3351c7800 con 0x55d33527fcb0 2024-09-10T09:53:28.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:28.086+0000 7f5a37400640 1 --2- 172.21.15.5:0/3792007163 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55d33527fcb0 0x55d3352820c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:28.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:28.087+0000 7f5a37400640 1 --2- 172.21.15.5:0/3792007163 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55d33527fcb0 0x55d3352820c0 crc :-1 s=READY pgs=223 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=79bf691fa4218861 server_cookie=50fab80a34a7aa8d in_seq=0 out_seq=0 2024-09-10T09:53:28.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:28.087+0000 7f5a25600640 10 client.28432 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T09:53:28.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:28.088+0000 7f5a25600640 1 -- 172.21.15.5:0/3792007163 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d3351c7800 con 0x55d33527fcb0 2024-09-10T09:53:28.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:28.088+0000 7f5a25600640 10 client.28432 handle_command_reply: tid=0 2024-09-10T09:53:28.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:28.088+0000 7f5a417ec740 1 client.28432 shutdown 2024-09-10T09:53:28.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:28.089+0000 7f5a417ec740 1 -- 172.21.15.5:0/3792007163 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d335253ac0 msgr2=0x55d33525e660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:53:28.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:28.089+0000 7f5a417ec740 1 --2- 172.21.15.5:0/3792007163 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d335253ac0 0x55d33525e660 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f5a300cd340 tx=0x7f5a3009c4f0 comp rx=0 tx=0).stop 2024-09-10T09:53:28.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:28.089+0000 7f5a417ec740 1 -- 172.21.15.5:0/3792007163 shutdown_connections 2024-09-10T09:53:28.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:28.089+0000 7f5a417ec740 1 --2- 172.21.15.5:0/3792007163 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55d33527fcb0 0x55d3352820c0 crc :-1 s=READY pgs=223 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:28.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:28.089+0000 7f5a417ec740 1 --2- 172.21.15.5:0/3792007163 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d33525eca0 0x55d33525bf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:28.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:28.089+0000 7f5a417ec740 1 --2- 172.21.15.5:0/3792007163 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d3352543e0 0x55d33525b910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:28.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:28.089+0000 7f5a25600640 0 client.28432 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T09:53:28.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:28.089+0000 7f5a417ec740 1 --2- 172.21.15.5:0/3792007163 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d335253ac0 0x55d33525e660 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:28.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:28.089+0000 7f5a417ec740 1 -- 172.21.15.5:0/3792007163 >> 172.21.15.5:0/3792007163 conn(0x55d335250050 msgr2=0x55d335255d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:53:28.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:28.090+0000 7f5a417ec740 1 -- 172.21.15.5:0/3792007163 shutdown_connections 2024-09-10T09:53:28.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:28.090+0000 7f5a417ec740 1 -- 172.21.15.5:0/3792007163 wait complete. 2024-09-10T09:53:28.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:28.090+0000 7f5a417ec740 20 client.28432 trim_cache size 0 max 16384 2024-09-10T09:53:28.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:28.126+0000 7f5a417ec740 1 Processor -- start 2024-09-10T09:53:28.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:28.126+0000 7f5a417ec740 1 -- start start 2024-09-10T09:53:28.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:28.127+0000 7f5a417ec740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d335253ac0 0x55d335255c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:28.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:28.127+0000 7f5a417ec740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d3352543e0 0x55d33525b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:28.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:28.127+0000 7f5a417ec740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d33525eca0 0x55d33525b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:28.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:28.127+0000 7f5a37400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d3352543e0 0x55d33525b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:28.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:28.127+0000 7f5a37400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d3352543e0 0x55d33525b1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:49970/0 (socket says 172.21.15.5:49970) 2024-09-10T09:53:28.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:28.128+0000 7f5a37e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d335253ac0 0x55d335255c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:28.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:28.128+0000 7f5a37e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d335253ac0 0x55d335255c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:52186/0 (socket says 172.21.15.5:52186) 2024-09-10T09:53:28.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:28.128+0000 7f5a3cc00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d33525eca0 0x55d33525b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:28.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:28.128+0000 7f5a37400640 1 -- 172.21.15.5:0/4197023731 learned_addr learned my addr 172.21.15.5:0/4197023731 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:53:28.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:28.128+0000 7f5a417ec740 1 -- 172.21.15.5:0/4197023731 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f5a381acd00 con 0x55d33525eca0 2024-09-10T09:53:28.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:28.128+0000 7f5a417ec740 1 -- 172.21.15.5:0/4197023731 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f5a381ae930 con 0x55d3352543e0 2024-09-10T09:53:28.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:28.128+0000 7f5a417ec740 1 -- 172.21.15.5:0/4197023731 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f5a381ad3d0 con 0x55d335253ac0 2024-09-10T09:53:28.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:28.129+0000 7f5a37400640 1 -- 172.21.15.5:0/4197023731 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d335253ac0 msgr2=0x55d335255c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:53:28.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:28.129+0000 7f5a37400640 1 --2- 172.21.15.5:0/4197023731 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d335253ac0 0x55d335255c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:28.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:28.129+0000 7f5a37400640 1 -- 172.21.15.5:0/4197023731 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d33525eca0 msgr2=0x55d33525b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:53:28.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:28.129+0000 7f5a37400640 1 --2- 172.21.15.5:0/4197023731 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d33525eca0 0x55d33525b730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:28.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:28.129+0000 7f5a37400640 1 -- 172.21.15.5:0/4197023731 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d335254210 con 0x55d3352543e0 2024-09-10T09:53:28.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:28.130+0000 7f5a37400640 1 --2- 172.21.15.5:0/4197023731 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d3352543e0 0x55d33525b1f0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f5a28009230 tx=0x7f5a2803e000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f488e7fd3e4d94bf server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:53:28.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:28.130+0000 7f5a27e00640 1 -- 172.21.15.5:0/4197023731 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a28035050 con 0x55d3352543e0 2024-09-10T09:53:28.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:28.130+0000 7f5a27e00640 1 -- 172.21.15.5:0/4197023731 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5a28055070 con 0x55d3352543e0 2024-09-10T09:53:28.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:28.130+0000 7f5a27e00640 1 -- 172.21.15.5:0/4197023731 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a28032c30 con 0x55d3352543e0 2024-09-10T09:53:28.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:28.131+0000 7f5a417ec740 1 -- 172.21.15.5:0/4197023731 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d3352543e0 msgr2=0x55d33525b1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:53:28.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:28.131+0000 7f5a417ec740 1 --2- 172.21.15.5:0/4197023731 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d3352543e0 0x55d33525b1f0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f5a28009230 tx=0x7f5a2803e000 comp rx=0 tx=0).stop 2024-09-10T09:53:28.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:28.132+0000 7f5a417ec740 1 -- 172.21.15.5:0/4197023731 shutdown_connections 2024-09-10T09:53:28.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:28.132+0000 7f5a417ec740 1 --2- 172.21.15.5:0/4197023731 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d33525eca0 0x55d33525b730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:28.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:28.132+0000 7f5a417ec740 1 --2- 172.21.15.5:0/4197023731 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d3352543e0 0x55d33525b1f0 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:28.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:28.132+0000 7f5a417ec740 1 --2- 172.21.15.5:0/4197023731 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d335253ac0 0x55d335255c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:28.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:28.132+0000 7f5a417ec740 1 -- 172.21.15.5:0/4197023731 >> 172.21.15.5:0/4197023731 conn(0x55d335250050 msgr2=0x55d335255560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:53:28.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:28.132+0000 7f5a417ec740 1 -- 172.21.15.5:0/4197023731 shutdown_connections 2024-09-10T09:53:28.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:28.133+0000 7f5a417ec740 1 -- 172.21.15.5:0/4197023731 wait complete. 2024-09-10T09:53:28.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:28.134+0000 7f5a417ec740 1 -- start start 2024-09-10T09:53:28.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:28.134+0000 7f5a417ec740 1 Processor -- start 2024-09-10T09:53:28.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:28.136+0000 7f5a417ec740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d335253ac0 0x55d335274310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:28.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:28.136+0000 7f5a417ec740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d3352543e0 0x55d335274850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:28.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:28.136+0000 7f5a37e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d335253ac0 0x55d335274310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:28.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:28.136+0000 7f5a417ec740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d33525eca0 0x55d3352e39a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:28.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:28.137+0000 7f5a37e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d335253ac0 0x55d335274310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:52196/0 (socket says 172.21.15.5:52196) 2024-09-10T09:53:28.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:28.137+0000 7f5a37400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d3352543e0 0x55d335274850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:28.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:28.137+0000 7f5a37400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d3352543e0 0x55d335274850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:49978/0 (socket says 172.21.15.5:49978) 2024-09-10T09:53:28.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:28.137+0000 7f5a3cc00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d33525eca0 0x55d3352e39a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:28.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:28.137+0000 7f5a417ec740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f5a381ae230 con 0x55d33525eca0 2024-09-10T09:53:28.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:28.137+0000 7f5a417ec740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f5a381adb60 con 0x55d3352543e0 2024-09-10T09:53:28.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:28.137+0000 7f5a417ec740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f5a38105900 con 0x55d335253ac0 2024-09-10T09:53:28.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:28.137+0000 7f5a417ec740 10 client.0 _ll_register_callbacks cb 0x7f5a3009b8e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:53:28.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:28.138+0000 7f5a37e00640 1 -- 172.21.15.5:0/2600166601 learned_addr learned my addr 172.21.15.5:0/2600166601 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:53:28.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:28.139+0000 7f5a37e00640 1 -- 172.21.15.5:0/2600166601 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d3352543e0 msgr2=0x55d335274850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:53:28.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:28.139+0000 7f5a37e00640 1 --2- 172.21.15.5:0/2600166601 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d3352543e0 0x55d335274850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:28.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:28.139+0000 7f5a37e00640 1 -- 172.21.15.5:0/2600166601 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d33525eca0 msgr2=0x55d3352e39a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:53:28.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:28.139+0000 7f5a37e00640 1 --2- 172.21.15.5:0/2600166601 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d33525eca0 0x55d3352e39a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:28.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:28.139+0000 7f5a37e00640 1 -- 172.21.15.5:0/2600166601 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d3352e3fe0 con 0x55d335253ac0 2024-09-10T09:53:28.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:28.139+0000 7f5a3cc00640 1 --2- 172.21.15.5:0/2600166601 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d33525eca0 0x55d3352e39a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T09:53:28.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:28.140+0000 7f5a37e00640 1 --2- 172.21.15.5:0/2600166601 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d335253ac0 0x55d335274310 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f5a30005c40 tx=0x7f5a300104d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4272d9dbfaafd85a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:53:28.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:28.140+0000 7f5a25600640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-09-10T09:53:28.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:28.140+0000 7f5a25600640 1 -- 172.21.15.5:0/2600166601 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a3007ada0 con 0x55d335253ac0 2024-09-10T09:53:28.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:28.140+0000 7f5a25600640 1 -- 172.21.15.5:0/2600166601 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f5a30011470 con 0x55d335253ac0 2024-09-10T09:53:28.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:28.141+0000 7f5a417ec740 1 -- 172.21.15.5:0/2600166601 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5a28055070 con 0x55d335253ac0 2024-09-10T09:53:28.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:28.141+0000 7f5a25600640 1 -- 172.21.15.5:0/2600166601 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f5a30011670 con 0x55d335253ac0 2024-09-10T09:53:28.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:28.144+0000 7f5a417ec740 10 client.28438 fetch_fsmap learned FSMap version 37 2024-09-10T09:53:28.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:28.144+0000 7f5a417ec740 1 -- 172.21.15.5:0/2600166601 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55d3352eb3d0 con 0x55d335253ac0 2024-09-10T09:53:28.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:28.144+0000 7f5a25600640 1 -- 172.21.15.5:0/2600166601 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5a28055070 con 0x55d335253ac0 2024-09-10T09:53:28.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:28.144+0000 7f5a25600640 1 -- 172.21.15.5:0/2600166601 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 37) ==== 8390+0+0 (secure 0 0 0) 0x7f5a300b2040 con 0x55d335253ac0 2024-09-10T09:53:28.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:28.145+0000 7f5a417ec740 10 client.28438 fetch_fsmap finished waiting for FSMap version 37 2024-09-10T09:53:28.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:28.145+0000 7f5a417ec740 10 client.28438 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T09:53:28.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:28.145+0000 7f5a417ec740 20 client.28438 populate_metadata read hostname 'smithi005' 2024-09-10T09:53:28.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:28.145+0000 7f5a417ec740 1 --2- 172.21.15.5:0/2600166601 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55d3352ebb20 0x55d3352edf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:28.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:28.145+0000 7f5a417ec740 4 client.28438 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-10T09:53:28.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:28.145+0000 7f5a417ec740 1 -- 172.21.15.5:0/2600166601 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55d3351c1d10 con 0x55d3352ebb20 2024-09-10T09:53:28.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:28.145+0000 7f5a37400640 1 --2- 172.21.15.5:0/2600166601 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55d3352ebb20 0x55d3352edf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:28.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:28.146+0000 7f5a37400640 1 --2- 172.21.15.5:0/2600166601 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55d3352ebb20 0x55d3352edf40 crc :-1 s=READY pgs=224 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e2aed9dc000a4840 server_cookie=e5f24ffb76fc8f34 in_seq=0 out_seq=0 2024-09-10T09:53:28.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:28.146+0000 7f5a25600640 10 client.28438 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T09:53:28.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:28.147+0000 7f5a25600640 1 -- 172.21.15.5:0/2600166601 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55d3351c1d10 con 0x55d3352ebb20 2024-09-10T09:53:28.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:28.147+0000 7f5a25600640 10 client.28438 handle_command_reply: tid=0 2024-09-10T09:53:28.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:28.148+0000 7f5a417ec740 1 client.28438 shutdown 2024-09-10T09:53:28.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:28.148+0000 7f5a417ec740 1 -- 172.21.15.5:0/2600166601 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d335253ac0 msgr2=0x55d335274310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:53:28.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:28.148+0000 7f5a417ec740 1 --2- 172.21.15.5:0/2600166601 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d335253ac0 0x55d335274310 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f5a30005c40 tx=0x7f5a300104d0 comp rx=0 tx=0).stop 2024-09-10T09:53:28.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:28.148+0000 7f5a417ec740 1 -- 172.21.15.5:0/2600166601 shutdown_connections 2024-09-10T09:53:28.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:28.148+0000 7f5a417ec740 1 --2- 172.21.15.5:0/2600166601 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55d3352ebb20 0x55d3352edf40 crc :-1 s=READY pgs=224 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:28.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:28.148+0000 7f5a417ec740 1 --2- 172.21.15.5:0/2600166601 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d33525eca0 0x55d3352e39a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:28.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:28.148+0000 7f5a417ec740 1 --2- 172.21.15.5:0/2600166601 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d3352543e0 0x55d335274850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:28.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:28.148+0000 7f5a417ec740 1 --2- 172.21.15.5:0/2600166601 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d335253ac0 0x55d335274310 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:28.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:28.148+0000 7f5a417ec740 1 -- 172.21.15.5:0/2600166601 >> 172.21.15.5:0/2600166601 conn(0x55d335250050 msgr2=0x55d335255d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:53:28.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:28.148+0000 7f5a25600640 0 client.28438 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T09:53:28.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:28.148+0000 7f5a417ec740 1 -- 172.21.15.5:0/2600166601 shutdown_connections 2024-09-10T09:53:28.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:28.148+0000 7f5a417ec740 1 -- 172.21.15.5:0/2600166601 wait complete. 2024-09-10T09:53:28.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:28.149+0000 7f5a417ec740 20 client.28438 trim_cache size 0 max 16384 2024-09-10T09:53:28.153 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-10T09:53:28.153 INFO:teuthology.orchestra.run.smithi005.stdout: "return_code": 0, 2024-09-10T09:53:28.153 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_tag": "f545d029-1355-466e-a909-f109a5713e5e", 2024-09-10T09:53:28.153 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": "asynchronous" 2024-09-10T09:53:28.153 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-10T09:53:28.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:28.150+0000 7f5a3ec00640 1 -- 172.21.15.5:0/4079256983 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f5a1c0a2c40 msgr2=0x7f5a1c0a5100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:53:28.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:28.150+0000 7f5a3ec00640 1 --2- 172.21.15.5:0/4079256983 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f5a1c0a2c40 0x7f5a1c0a5100 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f5a28000c00 tx=0x7f5a28004110 comp rx=0 tx=0).stop 2024-09-10T09:53:28.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:28.150+0000 7f5a3ec00640 1 -- 172.21.15.5:0/4079256983 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5a38100b10 msgr2=0x7f5a3819ebd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:53:28.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:28.150+0000 7f5a3ec00640 1 --2- 172.21.15.5:0/4079256983 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5a38100b10 0x7f5a3819ebd0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f5a3000e9f0 tx=0x7f5a3000eeb0 comp rx=0 tx=0).stop 2024-09-10T09:53:28.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:28.150+0000 7f5a3ec00640 1 -- 172.21.15.5:0/4079256983 shutdown_connections 2024-09-10T09:53:28.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:28.150+0000 7f5a3ec00640 1 --2- 172.21.15.5:0/4079256983 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f5a1c0a2c40 0x7f5a1c0a5100 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:28.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:28.150+0000 7f5a3ec00640 1 --2- 172.21.15.5:0/4079256983 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5a38105f20 0x7f5a381a34d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:28.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:28.150+0000 7f5a3ec00640 1 --2- 172.21.15.5:0/4079256983 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5a38101430 0x7f5a3819f110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:28.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:28.150+0000 7f5a3ec00640 1 --2- 172.21.15.5:0/4079256983 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5a38100b10 0x7f5a3819ebd0 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:28.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:28.150+0000 7f5a3ec00640 1 -- 172.21.15.5:0/4079256983 >> 172.21.15.5:0/4079256983 conn(0x7f5a380fc780 msgr2=0x7f5a3810f6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:53:28.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:28.151+0000 7f5a3ec00640 1 -- 172.21.15.5:0/4079256983 shutdown_connections 2024-09-10T09:53:28.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:28.151+0000 7f5a3ec00640 1 -- 172.21.15.5:0/4079256983 wait complete. 2024-09-10T09:53:28.950 INFO:tasks.workunit.client.0.smithi005.stdout: 10 28680 25.61 MB/sec warmup 67 sec latency 90.939 ms 2024-09-10T09:53:28.990 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T09:53:29.181 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:53:28 smithi005 ceph-mon[30350]: pgmap v847: 129 pgs: 129 active+clean; 532 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 4.5 MiB/s wr, 398 op/s 2024-09-10T09:53:29.182 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:53:28 smithi005 ceph-mon[30350]: scrub summary: active paths [/] 2024-09-10T09:53:29.182 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:53:28 smithi005 ceph-mon[30350]: scrub summary: active paths [/] 2024-09-10T09:53:29.182 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:53:28 smithi005 ceph-mon[30350]: scrub summary: idle+waiting paths [/] 2024-09-10T09:53:29.182 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:53:28 smithi005 ceph-mon[30350]: scrub summary: active paths [/] 2024-09-10T09:53:29.182 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:53:28 smithi005 ceph-mon[30350]: scrub summary: active paths [/] 2024-09-10T09:53:29.182 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:53:28 smithi005 ceph-mon[30350]: scrub summary: idle+waiting paths [/] 2024-09-10T09:53:29.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:53:28 smithi159 ceph-mon[35513]: pgmap v847: 129 pgs: 129 active+clean; 532 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 4.5 MiB/s wr, 398 op/s 2024-09-10T09:53:29.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:53:28 smithi159 ceph-mon[35513]: scrub summary: active paths [/] 2024-09-10T09:53:29.351 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:53:28 smithi159 ceph-mon[35513]: scrub summary: active paths [/] 2024-09-10T09:53:29.351 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:53:28 smithi159 ceph-mon[35513]: scrub summary: idle+waiting paths [/] 2024-09-10T09:53:29.351 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:53:28 smithi159 ceph-mon[35513]: scrub summary: active paths [/] 2024-09-10T09:53:29.351 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:53:28 smithi159 ceph-mon[35513]: scrub summary: active paths [/] 2024-09-10T09:53:29.351 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:53:28 smithi159 ceph-mon[35513]: scrub summary: idle+waiting paths [/] 2024-09-10T09:53:29.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:53:28 smithi102 ceph-mon[35442]: pgmap v847: 129 pgs: 129 active+clean; 532 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 4.5 MiB/s wr, 398 op/s 2024-09-10T09:53:29.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:53:28 smithi102 ceph-mon[35442]: scrub summary: active paths [/] 2024-09-10T09:53:29.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:53:28 smithi102 ceph-mon[35442]: scrub summary: active paths [/] 2024-09-10T09:53:29.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:53:28 smithi102 ceph-mon[35442]: scrub summary: idle+waiting paths [/] 2024-09-10T09:53:29.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:53:28 smithi102 ceph-mon[35442]: scrub summary: active paths [/] 2024-09-10T09:53:29.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:53:28 smithi102 ceph-mon[35442]: scrub summary: active paths [/] 2024-09-10T09:53:29.385 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:53:28 smithi102 ceph-mon[35442]: scrub summary: idle+waiting paths [/] 2024-09-10T09:53:29.537 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:53:29.950 INFO:tasks.workunit.client.0.smithi005.stdout: 10 29545 26.14 MB/sec warmup 68 sec latency 101.455 ms 2024-09-10T09:53:30.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:30.210+0000 7f2bd9800640 1 -- 172.21.15.5:0/3364372598 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2bd4102b90 msgr2=0x7f2bd4102f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:53:30.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:30.210+0000 7f2bd9800640 1 --2- 172.21.15.5:0/3364372598 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2bd4102b90 0x7f2bd4102f70 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f2bc4009860 tx=0x7f2bc402a3d0 comp rx=0 tx=0).stop 2024-09-10T09:53:30.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:30.210+0000 7f2bd9800640 1 -- 172.21.15.5:0/3364372598 shutdown_connections 2024-09-10T09:53:30.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:30.210+0000 7f2bd9800640 1 --2- 172.21.15.5:0/3364372598 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2bd410bfc0 0x7f2bd410e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:30.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:30.210+0000 7f2bd9800640 1 --2- 172.21.15.5:0/3364372598 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2bd41034b0 0x7f2bd410b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:30.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:30.210+0000 7f2bd9800640 1 --2- 172.21.15.5:0/3364372598 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2bd4102b90 0x7f2bd4102f70 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:30.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:30.210+0000 7f2bd9800640 1 -- 172.21.15.5:0/3364372598 >> 172.21.15.5:0/3364372598 conn(0x7f2bd40fc610 msgr2=0x7f2bd40fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:53:30.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:30.211+0000 7f2bd9800640 1 -- 172.21.15.5:0/3364372598 shutdown_connections 2024-09-10T09:53:30.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:30.211+0000 7f2bd9800640 1 -- 172.21.15.5:0/3364372598 wait complete. 2024-09-10T09:53:30.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:30.212+0000 7f2bd9800640 1 Processor -- start 2024-09-10T09:53:30.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:30.212+0000 7f2bd9800640 1 -- start start 2024-09-10T09:53:30.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:30.212+0000 7f2bd9800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2bd4102b90 0x7f2bd419a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:30.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:30.212+0000 7f2bd9800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2bd41034b0 0x7f2bd419aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:30.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:30.212+0000 7f2bd9800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2bd410bfc0 0x7f2bd419edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:30.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:30.212+0000 7f2bd9800640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f2bd4111e10 con 0x7f2bd41034b0 2024-09-10T09:53:30.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:30.212+0000 7f2bd9800640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f2bd4111c90 con 0x7f2bd4102b90 2024-09-10T09:53:30.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:30.212+0000 7f2bd9800640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f2bd4111f90 con 0x7f2bd410bfc0 2024-09-10T09:53:30.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:30.213+0000 7f2bd3400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2bd41034b0 0x7f2bd419aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:30.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:30.213+0000 7f2bd3e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2bd4102b90 0x7f2bd419a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:30.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:30.213+0000 7f2bd3e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2bd4102b90 0x7f2bd419a4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:50008/0 (socket says 172.21.15.5:50008) 2024-09-10T09:53:30.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:30.213+0000 7f2bd3400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2bd41034b0 0x7f2bd419aa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53656/0 (socket says 172.21.15.5:53656) 2024-09-10T09:53:30.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:30.213+0000 7f2bd3e00640 1 -- 172.21.15.5:0/3354049853 learned_addr learned my addr 172.21.15.5:0/3354049853 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:53:30.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:30.213+0000 7f2bd8e00640 1 --2- 172.21.15.5:0/3354049853 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2bd410bfc0 0x7f2bd419edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:30.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:30.213+0000 7f2bd3e00640 1 -- 172.21.15.5:0/3354049853 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2bd410bfc0 msgr2=0x7f2bd419edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:53:30.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:30.213+0000 7f2bd3e00640 1 --2- 172.21.15.5:0/3354049853 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2bd410bfc0 0x7f2bd419edb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:30.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:30.213+0000 7f2bd3e00640 1 -- 172.21.15.5:0/3354049853 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2bd41034b0 msgr2=0x7f2bd419aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:53:30.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:30.213+0000 7f2bd3e00640 1 --2- 172.21.15.5:0/3354049853 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2bd41034b0 0x7f2bd419aa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:30.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:30.213+0000 7f2bd3e00640 1 -- 172.21.15.5:0/3354049853 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2bd419f490 con 0x7f2bd4102b90 2024-09-10T09:53:30.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:30.214+0000 7f2bd3e00640 1 --2- 172.21.15.5:0/3354049853 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2bd4102b90 0x7f2bd419a4e0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f2bc4009980 tx=0x7f2bc40309a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:53:30.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:30.214+0000 7f2bd0c00640 1 -- 172.21.15.5:0/3354049853 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bc4040070 con 0x7f2bd4102b90 2024-09-10T09:53:30.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:30.214+0000 7f2bd0c00640 1 -- 172.21.15.5:0/3354049853 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2bc4005620 con 0x7f2bd4102b90 2024-09-10T09:53:30.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:30.214+0000 7f2bd0c00640 1 -- 172.21.15.5:0/3354049853 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bc40049b0 con 0x7f2bd4102b90 2024-09-10T09:53:30.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:30.214+0000 7f2bd9800640 1 -- 172.21.15.5:0/3354049853 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2bd419f720 con 0x7f2bd4102b90 2024-09-10T09:53:30.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:30.214+0000 7f2bd9800640 1 -- 172.21.15.5:0/3354049853 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2bd419fbb0 con 0x7f2bd4102b90 2024-09-10T09:53:30.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:30.218+0000 7f2bd0c00640 1 -- 172.21.15.5:0/3354049853 <== mon.2 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f2bc4043070 con 0x7f2bd4102b90 2024-09-10T09:53:30.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:30.218+0000 7f2bdd82e740 1 Processor -- start 2024-09-10T09:53:30.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:30.219+0000 7f2bdd82e740 1 -- start start 2024-09-10T09:53:30.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:30.219+0000 7f2bdd82e740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560fa2347d50 0x560fa2348130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:30.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:30.219+0000 7f2bdd82e740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560fa2348670 0x560fa23529f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:30.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:30.219+0000 7f2bdd82e740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560fa2352f30 0x560fa2355320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:30.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:30.219+0000 7f2bdd82e740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x560fa22ae3e0 con 0x560fa2347d50 2024-09-10T09:53:30.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:30.219+0000 7f2bdd82e740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x560fa2255d80 con 0x560fa2352f30 2024-09-10T09:53:30.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:30.219+0000 7f2bdd82e740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x560fa22aa3a0 con 0x560fa2348670 2024-09-10T09:53:30.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:30.219+0000 7f2bd3e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560fa2348670 0x560fa23529f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:30.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:30.219+0000 7f2bd3e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560fa2348670 0x560fa23529f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:52212/0 (socket says 172.21.15.5:52212) 2024-09-10T09:53:30.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:30.219+0000 7f2bd3e00640 1 -- 172.21.15.5:0/3917125741 learned_addr learned my addr 172.21.15.5:0/3917125741 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:53:30.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:30.219+0000 7f2bd3400640 1 --2- 172.21.15.5:0/3917125741 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560fa2352f30 0x560fa2355320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:30.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:30.219+0000 7f2bd0c00640 1 --2- 172.21.15.5:0/3354049853 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f2bb80a2c00 0x7f2bb80a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:30.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:30.220+0000 7f2bd3e00640 1 -- 172.21.15.5:0/3917125741 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560fa2352f30 msgr2=0x560fa2355320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:53:30.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:30.220+0000 7f2bd3e00640 1 --2- 172.21.15.5:0/3917125741 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560fa2352f30 0x560fa2355320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:30.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:30.220+0000 7f2bd3e00640 1 -- 172.21.15.5:0/3917125741 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560fa2347d50 msgr2=0x560fa2348130 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T09:53:30.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:30.220+0000 7f2bd0c00640 1 -- 172.21.15.5:0/3354049853 <== mon.2 v2:172.21.15.102:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f2bc40f00e0 con 0x7f2bd4102b90 2024-09-10T09:53:30.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:30.220+0000 7f2bd3e00640 1 --2- 172.21.15.5:0/3917125741 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560fa2347d50 0x560fa2348130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:30.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:30.220+0000 7f2bd3e00640 1 -- 172.21.15.5:0/3917125741 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560fa2274ae0 con 0x560fa2348670 2024-09-10T09:53:30.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:30.220+0000 7f2bd3400640 1 --2- 172.21.15.5:0/3917125741 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560fa2352f30 0x560fa2355320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:53:30.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:30.220+0000 7f2bd3e00640 1 --2- 172.21.15.5:0/3917125741 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560fa2348670 0x560fa23529f0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f2bc40b8a00 tx=0x7f2bc402a3d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=91dff299c3e34fb1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:53:30.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:30.221+0000 7f2bc9600640 1 -- 172.21.15.5:0/3917125741 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bc404a070 con 0x560fa2348670 2024-09-10T09:53:30.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:30.221+0000 7f2bc9600640 1 -- 172.21.15.5:0/3917125741 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2bc40bc5a0 con 0x560fa2348670 2024-09-10T09:53:30.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:30.221+0000 7f2bc9600640 1 -- 172.21.15.5:0/3917125741 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bc40ba640 con 0x560fa2348670 2024-09-10T09:53:30.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:30.221+0000 7f2bd8e00640 1 --2- 172.21.15.5:0/3354049853 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f2bb80a2c00 0x7f2bb80a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:30.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:30.221+0000 7f2bdd82e740 1 -- 172.21.15.5:0/3917125741 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560fa2348670 msgr2=0x560fa23529f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:53:30.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:30.221+0000 7f2bdd82e740 1 --2- 172.21.15.5:0/3917125741 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560fa2348670 0x560fa23529f0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f2bc40b8a00 tx=0x7f2bc402a3d0 comp rx=0 tx=0).stop 2024-09-10T09:53:30.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:30.222+0000 7f2bdd82e740 1 -- 172.21.15.5:0/3917125741 shutdown_connections 2024-09-10T09:53:30.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:30.222+0000 7f2bdd82e740 1 --2- 172.21.15.5:0/3917125741 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560fa2352f30 0x560fa2355320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:30.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:30.222+0000 7f2bdd82e740 1 --2- 172.21.15.5:0/3917125741 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560fa2348670 0x560fa23529f0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:30.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:30.222+0000 7f2bdd82e740 1 --2- 172.21.15.5:0/3917125741 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560fa2347d50 0x560fa2348130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:30.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:30.222+0000 7f2bdd82e740 1 -- 172.21.15.5:0/3917125741 >> 172.21.15.5:0/3917125741 conn(0x560fa23442e0 msgr2=0x560fa23446c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:53:30.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:30.222+0000 7f2bd8e00640 1 --2- 172.21.15.5:0/3354049853 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f2bb80a2c00 0x7f2bb80a50c0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f2bcc005fd0 tx=0x7f2bcc009b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:53:30.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:30.222+0000 7f2bdd82e740 1 -- 172.21.15.5:0/3917125741 shutdown_connections 2024-09-10T09:53:30.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:30.222+0000 7f2bdd82e740 1 -- 172.21.15.5:0/3917125741 wait complete. 2024-09-10T09:53:30.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:30.222+0000 7f2bdd82e740 1 -- start start 2024-09-10T09:53:30.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:30.222+0000 7f2bdd82e740 1 Processor -- start 2024-09-10T09:53:30.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:30.223+0000 7f2bdd82e740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560fa2347d50 0x560fa234fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:30.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:30.223+0000 7f2bdd82e740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560fa2348670 0x560fa2350310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:30.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:30.223+0000 7f2bdd82e740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560fa2352f30 0x560fa2350850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:30.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:30.223+0000 7f2bdd82e740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x560fa23558c0 con 0x560fa2348670 2024-09-10T09:53:30.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:30.223+0000 7f2bdd82e740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x560fa22690f0 con 0x560fa2352f30 2024-09-10T09:53:30.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:30.223+0000 7f2bdd82e740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x560fa2355a40 con 0x560fa2347d50 2024-09-10T09:53:30.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:30.223+0000 7f2bdd82e740 10 client.0 _ll_register_callbacks cb 0x560fa225a9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:53:30.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:30.223+0000 7f2bd3400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560fa2348670 0x560fa2350310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:30.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:30.223+0000 7f2bd3e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560fa2347d50 0x560fa234fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:30.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:30.223+0000 7f2bd8e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560fa2352f30 0x560fa2350850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:30.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:30.223+0000 7f2bd3e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560fa2347d50 0x560fa234fdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:52220/0 (socket says 172.21.15.5:52220) 2024-09-10T09:53:30.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:30.223+0000 7f2bd3e00640 1 -- 172.21.15.5:0/1649754223 learned_addr learned my addr 172.21.15.5:0/1649754223 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:53:30.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:30.223+0000 7f2bd3e00640 1 -- 172.21.15.5:0/1649754223 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560fa2352f30 msgr2=0x560fa2350850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:53:30.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:30.223+0000 7f2bd3e00640 1 --2- 172.21.15.5:0/1649754223 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560fa2352f30 0x560fa2350850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:30.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:30.223+0000 7f2bd3e00640 1 -- 172.21.15.5:0/1649754223 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560fa2348670 msgr2=0x560fa2350310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:53:30.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:30.223+0000 7f2bd3e00640 1 --2- 172.21.15.5:0/1649754223 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560fa2348670 0x560fa2350310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:30.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:30.223+0000 7f2bd3e00640 1 -- 172.21.15.5:0/1649754223 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560fa201f3c0 con 0x560fa2347d50 2024-09-10T09:53:30.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:30.224+0000 7f2bd3e00640 1 --2- 172.21.15.5:0/1649754223 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560fa2347d50 0x560fa234fdd0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f2bc40c4c30 tx=0x7f2bc40c08a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2ea53ed18aea1034 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:53:30.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:30.224+0000 7f2bbe000640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-09-10T09:53:30.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:30.224+0000 7f2bbe000640 1 -- 172.21.15.5:0/1649754223 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bc404a040 con 0x560fa2347d50 2024-09-10T09:53:30.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:30.224+0000 7f2bbe000640 1 -- 172.21.15.5:0/1649754223 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f2bc4031c30 con 0x560fa2347d50 2024-09-10T09:53:30.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:30.224+0000 7f2bdd82e740 1 -- 172.21.15.5:0/1649754223 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2bc40bc5a0 con 0x560fa2347d50 2024-09-10T09:53:30.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:30.224+0000 7f2bbe000640 1 -- 172.21.15.5:0/1649754223 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f2bc40bbf00 con 0x560fa2347d50 2024-09-10T09:53:30.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:30.224+0000 7f2bdd82e740 10 client.28450 fetch_fsmap learned FSMap version 37 2024-09-10T09:53:30.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:30.224+0000 7f2bdd82e740 1 -- 172.21.15.5:0/1649754223 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x560fa23558c0 con 0x560fa2347d50 2024-09-10T09:53:30.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:30.224+0000 7f2bbe000640 1 -- 172.21.15.5:0/1649754223 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2bc40f3650 con 0x560fa2347d50 2024-09-10T09:53:30.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:30.224+0000 7f2bbe000640 1 -- 172.21.15.5:0/1649754223 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 37) ==== 8390+0+0 (secure 0 0 0) 0x7f2bc4031620 con 0x560fa2347d50 2024-09-10T09:53:30.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:30.225+0000 7f2bdd82e740 10 client.28450 fetch_fsmap finished waiting for FSMap version 37 2024-09-10T09:53:30.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:30.225+0000 7f2bdd82e740 10 client.28450 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T09:53:30.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:30.225+0000 7f2bdd82e740 20 client.28450 populate_metadata read hostname 'smithi005' 2024-09-10T09:53:30.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:30.225+0000 7f2bdd82e740 1 --2- 172.21.15.5:0/1649754223 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x560fa23626a0 0x560fa2364a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:30.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:30.225+0000 7f2bdd82e740 4 client.28450 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:53:30.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:30.225+0000 7f2bdd82e740 1 -- 172.21.15.5:0/1649754223 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f2bc40bc5a0 con 0x560fa23626a0 2024-09-10T09:53:30.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:30.225+0000 7f2bd3400640 1 --2- 172.21.15.5:0/1649754223 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x560fa23626a0 0x560fa2364a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:30.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:30.225+0000 7f2bd3400640 1 --2- 172.21.15.5:0/1649754223 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x560fa23626a0 0x560fa2364a90 crc :-1 s=READY pgs=225 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a68c26955c441e18 server_cookie=f69d2fc538e5aeb1 in_seq=0 out_seq=0 2024-09-10T09:53:30.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:30.225+0000 7f2bbe000640 10 client.28450 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T09:53:30.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:30.228+0000 7f2bbe000640 1 -- 172.21.15.5:0/1649754223 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f2bc40bc5a0 con 0x560fa23626a0 2024-09-10T09:53:30.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:30.228+0000 7f2bbe000640 10 client.28450 handle_command_reply: tid=0 2024-09-10T09:53:30.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:30.228+0000 7f2bdd82e740 1 client.28450 shutdown 2024-09-10T09:53:30.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:30.228+0000 7f2bdd82e740 1 -- 172.21.15.5:0/1649754223 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560fa2347d50 msgr2=0x560fa234fdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:53:30.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:30.228+0000 7f2bdd82e740 1 --2- 172.21.15.5:0/1649754223 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560fa2347d50 0x560fa234fdd0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f2bc40c4c30 tx=0x7f2bc40c08a0 comp rx=0 tx=0).stop 2024-09-10T09:53:30.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:30.228+0000 7f2bdd82e740 1 -- 172.21.15.5:0/1649754223 shutdown_connections 2024-09-10T09:53:30.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:30.228+0000 7f2bdd82e740 1 --2- 172.21.15.5:0/1649754223 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x560fa23626a0 0x560fa2364a90 crc :-1 s=READY pgs=225 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:30.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:30.228+0000 7f2bdd82e740 1 --2- 172.21.15.5:0/1649754223 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560fa2352f30 0x560fa2350850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:30.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:30.228+0000 7f2bdd82e740 1 --2- 172.21.15.5:0/1649754223 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560fa2348670 0x560fa2350310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:30.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:30.228+0000 7f2bdd82e740 1 --2- 172.21.15.5:0/1649754223 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560fa2347d50 0x560fa234fdd0 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:30.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:30.228+0000 7f2bdd82e740 1 -- 172.21.15.5:0/1649754223 >> 172.21.15.5:0/1649754223 conn(0x560fa23442e0 msgr2=0x560fa234a010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:53:30.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:30.228+0000 7f2bbe000640 0 client.28450 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T09:53:30.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:30.228+0000 7f2bdd82e740 1 -- 172.21.15.5:0/1649754223 shutdown_connections 2024-09-10T09:53:30.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:30.228+0000 7f2bdd82e740 1 -- 172.21.15.5:0/1649754223 wait complete. 2024-09-10T09:53:30.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:30.229+0000 7f2bdd82e740 20 client.28450 trim_cache size 0 max 16384 2024-09-10T09:53:30.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:30.255+0000 7f2bdd82e740 1 Processor -- start 2024-09-10T09:53:30.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:30.255+0000 7f2bdd82e740 1 -- start start 2024-09-10T09:53:30.256 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:53:29 smithi005 ceph-mon[30350]: scrub queued for path: / 2024-09-10T09:53:30.256 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:53:29 smithi005 ceph-mon[30350]: scrub summary: idle+waiting paths [/] 2024-09-10T09:53:30.256 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:53:29 smithi005 ceph-mon[30350]: scrub summary: active paths [/] 2024-09-10T09:53:30.256 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:53:29 smithi005 ceph-mon[30350]: scrub summary: idle+waiting paths [/] 2024-09-10T09:53:30.256 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:53:29 smithi005 ceph-mon[30350]: scrub summary: active paths [/] 2024-09-10T09:53:30.257 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:53:29 smithi005 ceph-mon[30350]: scrub summary: idle+waiting paths [/] 2024-09-10T09:53:30.257 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:53:29 smithi005 ceph-mon[30350]: scrub summary: idle+waiting paths [/] 2024-09-10T09:53:30.257 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:53:29 smithi005 ceph-mon[30350]: scrub summary: active paths [/] 2024-09-10T09:53:30.257 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:53:29 smithi005 ceph-mon[30350]: scrub summary: idle+waiting paths [/] 2024-09-10T09:53:30.257 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:53:29 smithi005 ceph-mon[30350]: scrub summary: idle+waiting paths [/] 2024-09-10T09:53:30.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:30.255+0000 7f2bdd82e740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560fa2347d50 0x560fa233f060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:30.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:30.255+0000 7f2bdd82e740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560fa2348670 0x560fa23506a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:30.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:30.255+0000 7f2bdd82e740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560fa2352f30 0x560fa2343270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:30.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:30.255+0000 7f2bdd82e740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f2bd41b0f10 con 0x560fa2352f30 2024-09-10T09:53:30.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:30.255+0000 7f2bdd82e740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f2bd41b2b40 con 0x560fa2347d50 2024-09-10T09:53:30.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:30.255+0000 7f2bdd82e740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f2bd41b15e0 con 0x560fa2348670 2024-09-10T09:53:30.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:30.256+0000 7f2bd3e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560fa2347d50 0x560fa233f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:30.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:30.256+0000 7f2bd3e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560fa2347d50 0x560fa233f060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:50050/0 (socket says 172.21.15.5:50050) 2024-09-10T09:53:30.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:30.256+0000 7f2bd3e00640 1 -- 172.21.15.5:0/1361627452 learned_addr learned my addr 172.21.15.5:0/1361627452 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:53:30.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:30.256+0000 7f2bd3400640 1 --2- 172.21.15.5:0/1361627452 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560fa2348670 0x560fa23506a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:30.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:30.256+0000 7f2bd8e00640 1 --2- 172.21.15.5:0/1361627452 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560fa2352f30 0x560fa2343270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:30.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:30.256+0000 7f2bd3e00640 1 -- 172.21.15.5:0/1361627452 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560fa2348670 msgr2=0x560fa23506a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:53:30.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:30.256+0000 7f2bd3e00640 1 --2- 172.21.15.5:0/1361627452 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560fa2348670 0x560fa23506a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:30.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:30.256+0000 7f2bd3e00640 1 -- 172.21.15.5:0/1361627452 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560fa2352f30 msgr2=0x560fa2343270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:53:30.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:30.256+0000 7f2bd3e00640 1 --2- 172.21.15.5:0/1361627452 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560fa2352f30 0x560fa2343270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:30.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:30.256+0000 7f2bd3e00640 1 -- 172.21.15.5:0/1361627452 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560fa234edc0 con 0x560fa2347d50 2024-09-10T09:53:30.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:30.257+0000 7f2bd3e00640 1 --2- 172.21.15.5:0/1361627452 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560fa2347d50 0x560fa233f060 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f2bc40b8f20 tx=0x7f2bc40baab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=71bdfaa81830297a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:53:30.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:30.257+0000 7f2bc8c00640 1 -- 172.21.15.5:0/1361627452 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bc404a040 con 0x560fa2347d50 2024-09-10T09:53:30.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:30.257+0000 7f2bc8c00640 1 -- 172.21.15.5:0/1361627452 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2bc40bc5a0 con 0x560fa2347d50 2024-09-10T09:53:30.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:30.257+0000 7f2bc8c00640 1 -- 172.21.15.5:0/1361627452 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bc4004b50 con 0x560fa2347d50 2024-09-10T09:53:30.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:30.258+0000 7f2bdd82e740 1 -- 172.21.15.5:0/1361627452 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560fa2347d50 msgr2=0x560fa233f060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:53:30.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:30.258+0000 7f2bdd82e740 1 --2- 172.21.15.5:0/1361627452 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560fa2347d50 0x560fa233f060 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f2bc40b8f20 tx=0x7f2bc40baab0 comp rx=0 tx=0).stop 2024-09-10T09:53:30.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:30.259+0000 7f2bdd82e740 1 -- 172.21.15.5:0/1361627452 shutdown_connections 2024-09-10T09:53:30.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:30.259+0000 7f2bdd82e740 1 --2- 172.21.15.5:0/1361627452 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560fa2352f30 0x560fa2343270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:30.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:30.259+0000 7f2bdd82e740 1 --2- 172.21.15.5:0/1361627452 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560fa2348670 0x560fa23506a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:30.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:30.259+0000 7f2bdd82e740 1 --2- 172.21.15.5:0/1361627452 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560fa2347d50 0x560fa233f060 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:30.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:30.259+0000 7f2bdd82e740 1 -- 172.21.15.5:0/1361627452 >> 172.21.15.5:0/1361627452 conn(0x560fa23442e0 msgr2=0x560fa2342b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:53:30.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:30.260+0000 7f2bdd82e740 1 -- 172.21.15.5:0/1361627452 shutdown_connections 2024-09-10T09:53:30.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:30.260+0000 7f2bdd82e740 1 -- 172.21.15.5:0/1361627452 wait complete. 2024-09-10T09:53:30.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:30.260+0000 7f2bdd82e740 1 -- start start 2024-09-10T09:53:30.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:30.261+0000 7f2bdd82e740 1 Processor -- start 2024-09-10T09:53:30.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:30.261+0000 7f2bdd82e740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560fa2347d50 0x560fa235aef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:30.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:30.261+0000 7f2bdd82e740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560fa2348670 0x560fa235b430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:30.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:30.261+0000 7f2bdd82e740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560fa2352f30 0x560fa2357c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:30.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:30.261+0000 7f2bdd82e740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f2bd41b2440 con 0x560fa2347d50 2024-09-10T09:53:30.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:30.261+0000 7f2bdd82e740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f2bd41b1d70 con 0x560fa2348670 2024-09-10T09:53:30.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:30.261+0000 7f2bdd82e740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f2bd41aeef0 con 0x560fa2352f30 2024-09-10T09:53:30.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:30.262+0000 7f2bdd82e740 10 client.0 _ll_register_callbacks cb 0x7f2bc40bc370 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:53:30.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:30.262+0000 7f2bd8e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560fa2352f30 0x560fa2357c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:30.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:30.262+0000 7f2bd8e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560fa2352f30 0x560fa2357c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:52238/0 (socket says 172.21.15.5:52238) 2024-09-10T09:53:30.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:30.262+0000 7f2bd8e00640 1 -- 172.21.15.5:0/2671431677 learned_addr learned my addr 172.21.15.5:0/2671431677 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:53:30.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:30.262+0000 7f2bd3400640 1 --2- 172.21.15.5:0/2671431677 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560fa2348670 0x560fa235b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:30.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:30.263+0000 7f2bd3e00640 1 --2- 172.21.15.5:0/2671431677 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560fa2347d50 0x560fa235aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:30.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:30.263+0000 7f2bd8e00640 1 -- 172.21.15.5:0/2671431677 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560fa2348670 msgr2=0x560fa235b430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:53:30.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:30.263+0000 7f2bd8e00640 1 --2- 172.21.15.5:0/2671431677 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560fa2348670 0x560fa235b430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:30.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:30.263+0000 7f2bd8e00640 1 -- 172.21.15.5:0/2671431677 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560fa2347d50 msgr2=0x560fa235aef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:53:30.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:30.263+0000 7f2bd8e00640 1 --2- 172.21.15.5:0/2671431677 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560fa2347d50 0x560fa235aef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:30.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:30.263+0000 7f2bd8e00640 1 -- 172.21.15.5:0/2671431677 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560fa2358460 con 0x560fa2352f30 2024-09-10T09:53:30.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:30.263+0000 7f2bd3e00640 1 --2- 172.21.15.5:0/2671431677 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560fa2347d50 0x560fa235aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T09:53:30.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:30.264+0000 7f2bd8e00640 1 --2- 172.21.15.5:0/2671431677 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560fa2352f30 0x560fa2357c30 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f2bcc01dd60 tx=0x7f2bcc009500 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3d9a9cd1ecf18d31 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:53:30.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:30.264+0000 7f2bbe000640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-09-10T09:53:30.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:30.264+0000 7f2bbe000640 1 -- 172.21.15.5:0/2671431677 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bcc003b70 con 0x560fa2352f30 2024-09-10T09:53:30.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:30.264+0000 7f2bbe000640 1 -- 172.21.15.5:0/2671431677 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f2bcc003d10 con 0x560fa2352f30 2024-09-10T09:53:30.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:30.264+0000 7f2bdd82e740 1 -- 172.21.15.5:0/2671431677 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2bc40bc5a0 con 0x560fa2352f30 2024-09-10T09:53:30.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:30.265+0000 7f2bbe000640 1 -- 172.21.15.5:0/2671431677 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f2bcc004050 con 0x560fa2352f30 2024-09-10T09:53:30.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:30.265+0000 7f2bdd82e740 10 client.28456 fetch_fsmap learned FSMap version 37 2024-09-10T09:53:30.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:30.265+0000 7f2bdd82e740 1 -- 172.21.15.5:0/2671431677 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x560fa23d65c0 con 0x560fa2352f30 2024-09-10T09:53:30.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:30.266+0000 7f2bbe000640 1 -- 172.21.15.5:0/2671431677 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2bc40bc5a0 con 0x560fa2352f30 2024-09-10T09:53:30.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:30.266+0000 7f2bbe000640 1 -- 172.21.15.5:0/2671431677 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 37) ==== 8390+0+0 (secure 0 0 0) 0x7f2bcc0329e0 con 0x560fa2352f30 2024-09-10T09:53:30.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:30.266+0000 7f2bdd82e740 10 client.28456 fetch_fsmap finished waiting for FSMap version 37 2024-09-10T09:53:30.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:30.266+0000 7f2bdd82e740 10 client.28456 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T09:53:30.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:30.266+0000 7f2bdd82e740 20 client.28456 populate_metadata read hostname 'smithi005' 2024-09-10T09:53:30.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:30.266+0000 7f2bdd82e740 1 --2- 172.21.15.5:0/2671431677 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x560fa23d6d70 0x560fa23d9190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:30.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:30.266+0000 7f2bdd82e740 4 client.28456 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T09:53:30.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:30.266+0000 7f2bdd82e740 1 -- 172.21.15.5:0/2671431677 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "scrub status"}) -- 0x560fa22bb950 con 0x560fa23d6d70 2024-09-10T09:53:30.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:30.266+0000 7f2bd3e00640 1 --2- 172.21.15.5:0/2671431677 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x560fa23d6d70 0x560fa23d9190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:30.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:30.267+0000 7f2bd3e00640 1 --2- 172.21.15.5:0/2671431677 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x560fa23d6d70 0x560fa23d9190 crc :-1 s=READY pgs=226 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=84473bb730283b20 server_cookie=2bceacaaec1a1aaa in_seq=0 out_seq=0 2024-09-10T09:53:30.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:30.267+0000 7f2bbe000640 10 client.28456 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T09:53:30.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:30.268+0000 7f2bbe000640 1 -- 172.21.15.5:0/2671431677 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x560fa22bb950 con 0x560fa23d6d70 2024-09-10T09:53:30.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:30.268+0000 7f2bbe000640 10 client.28456 handle_command_reply: tid=0 2024-09-10T09:53:30.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:30.268+0000 7f2bdd82e740 1 client.28456 shutdown 2024-09-10T09:53:30.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:30.268+0000 7f2bdd82e740 1 -- 172.21.15.5:0/2671431677 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560fa2352f30 msgr2=0x560fa2357c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:53:30.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:30.268+0000 7f2bdd82e740 1 --2- 172.21.15.5:0/2671431677 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560fa2352f30 0x560fa2357c30 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f2bcc01dd60 tx=0x7f2bcc009500 comp rx=0 tx=0).stop 2024-09-10T09:53:30.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:30.269+0000 7f2bdd82e740 1 -- 172.21.15.5:0/2671431677 shutdown_connections 2024-09-10T09:53:30.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:30.269+0000 7f2bdd82e740 1 --2- 172.21.15.5:0/2671431677 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x560fa23d6d70 0x560fa23d9190 crc :-1 s=READY pgs=226 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:30.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:30.269+0000 7f2bdd82e740 1 --2- 172.21.15.5:0/2671431677 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560fa2352f30 0x560fa2357c30 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:30.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:30.269+0000 7f2bdd82e740 1 --2- 172.21.15.5:0/2671431677 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560fa2348670 0x560fa235b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:30.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:30.269+0000 7f2bdd82e740 1 --2- 172.21.15.5:0/2671431677 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560fa2347d50 0x560fa235aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:30.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:30.269+0000 7f2bdd82e740 1 -- 172.21.15.5:0/2671431677 >> 172.21.15.5:0/2671431677 conn(0x560fa23442e0 msgr2=0x560fa2343380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:53:30.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:30.269+0000 7f2bbe000640 0 client.28456 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T09:53:30.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:30.270+0000 7f2bdd82e740 1 -- 172.21.15.5:0/2671431677 shutdown_connections 2024-09-10T09:53:30.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:30.270+0000 7f2bdd82e740 1 -- 172.21.15.5:0/2671431677 wait complete. 2024-09-10T09:53:30.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:30.271+0000 7f2bdd82e740 20 client.28456 trim_cache size 0 max 16384 2024-09-10T09:53:30.273 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-10T09:53:30.273 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-10T09:53:30.273 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2024-09-10T09:53:30.273 INFO:teuthology.orchestra.run.smithi005.stdout: "f545d029-1355-466e-a909-f109a5713e5e": { 2024-09-10T09:53:30.273 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2024-09-10T09:53:30.273 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "f545d029-1355-466e-a909-f109a5713e5e", 2024-09-10T09:53:30.273 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2024-09-10T09:53:30.273 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T09:53:30.273 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T09:53:30.273 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-10T09:53:30.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:30.272+0000 7f2bd9800640 1 -- 172.21.15.5:0/3354049853 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f2bb80a2c00 msgr2=0x7f2bb80a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:53:30.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:30.272+0000 7f2bd9800640 1 --2- 172.21.15.5:0/3354049853 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f2bb80a2c00 0x7f2bb80a50c0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f2bcc005fd0 tx=0x7f2bcc009b50 comp rx=0 tx=0).stop 2024-09-10T09:53:30.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:30.272+0000 7f2bd9800640 1 -- 172.21.15.5:0/3354049853 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2bd4102b90 msgr2=0x7f2bd419a4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:53:30.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:30.272+0000 7f2bd9800640 1 --2- 172.21.15.5:0/3354049853 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2bd4102b90 0x7f2bd419a4e0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f2bc4009980 tx=0x7f2bc40309a0 comp rx=0 tx=0).stop 2024-09-10T09:53:30.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:30.273+0000 7f2bd9800640 1 -- 172.21.15.5:0/3354049853 shutdown_connections 2024-09-10T09:53:30.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:30.273+0000 7f2bd9800640 1 --2- 172.21.15.5:0/3354049853 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f2bb80a2c00 0x7f2bb80a50c0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:30.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:30.273+0000 7f2bd9800640 1 --2- 172.21.15.5:0/3354049853 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2bd410bfc0 0x7f2bd419edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:30.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:30.273+0000 7f2bd9800640 1 --2- 172.21.15.5:0/3354049853 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2bd41034b0 0x7f2bd419aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:30.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:30.273+0000 7f2bd9800640 1 --2- 172.21.15.5:0/3354049853 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2bd4102b90 0x7f2bd419a4e0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:30.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:30.273+0000 7f2bd9800640 1 -- 172.21.15.5:0/3354049853 >> 172.21.15.5:0/3354049853 conn(0x7f2bd40fc610 msgr2=0x7f2bd40fea00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:53:30.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:30.273+0000 7f2bd9800640 1 -- 172.21.15.5:0/3354049853 shutdown_connections 2024-09-10T09:53:30.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:30.273+0000 7f2bd9800640 1 -- 172.21.15.5:0/3354049853 wait complete. 2024-09-10T09:53:30.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:53:29 smithi159 ceph-mon[35513]: scrub queued for path: / 2024-09-10T09:53:30.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:53:29 smithi159 ceph-mon[35513]: scrub summary: idle+waiting paths [/] 2024-09-10T09:53:30.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:53:29 smithi159 ceph-mon[35513]: scrub summary: active paths [/] 2024-09-10T09:53:30.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:53:29 smithi159 ceph-mon[35513]: scrub summary: idle+waiting paths [/] 2024-09-10T09:53:30.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:53:29 smithi159 ceph-mon[35513]: scrub summary: active paths [/] 2024-09-10T09:53:30.351 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:53:29 smithi159 ceph-mon[35513]: scrub summary: idle+waiting paths [/] 2024-09-10T09:53:30.351 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:53:29 smithi159 ceph-mon[35513]: scrub summary: idle+waiting paths [/] 2024-09-10T09:53:30.351 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:53:29 smithi159 ceph-mon[35513]: scrub summary: active paths [/] 2024-09-10T09:53:30.351 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:53:29 smithi159 ceph-mon[35513]: scrub summary: idle+waiting paths [/] 2024-09-10T09:53:30.351 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:53:29 smithi159 ceph-mon[35513]: scrub summary: idle+waiting paths [/] 2024-09-10T09:53:30.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:53:29 smithi102 ceph-mon[35442]: scrub queued for path: / 2024-09-10T09:53:30.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:53:29 smithi102 ceph-mon[35442]: scrub summary: idle+waiting paths [/] 2024-09-10T09:53:30.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:53:29 smithi102 ceph-mon[35442]: scrub summary: active paths [/] 2024-09-10T09:53:30.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:53:29 smithi102 ceph-mon[35442]: scrub summary: idle+waiting paths [/] 2024-09-10T09:53:30.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:53:29 smithi102 ceph-mon[35442]: scrub summary: active paths [/] 2024-09-10T09:53:30.385 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:53:29 smithi102 ceph-mon[35442]: scrub summary: idle+waiting paths [/] 2024-09-10T09:53:30.385 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:53:29 smithi102 ceph-mon[35442]: scrub summary: idle+waiting paths [/] 2024-09-10T09:53:30.385 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:53:29 smithi102 ceph-mon[35442]: scrub summary: active paths [/] 2024-09-10T09:53:30.385 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:53:29 smithi102 ceph-mon[35442]: scrub summary: idle+waiting paths [/] 2024-09-10T09:53:30.385 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:53:29 smithi102 ceph-mon[35442]: scrub summary: idle+waiting paths [/] 2024-09-10T09:53:30.950 INFO:tasks.workunit.client.0.smithi005.stdout: 10 30144 26.10 MB/sec warmup 69 sec latency 72.922 ms 2024-09-10T09:53:31.197 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:53:30 smithi005 ceph-mon[30350]: pgmap v848: 129 pgs: 129 active+clean; 527 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 5.4 MiB/s wr, 565 op/s 2024-09-10T09:53:31.285 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:53:30 smithi102 ceph-mon[35442]: pgmap v848: 129 pgs: 129 active+clean; 527 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 5.4 MiB/s wr, 565 op/s 2024-09-10T09:53:31.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:53:30 smithi159 ceph-mon[35513]: pgmap v848: 129 pgs: 129 active+clean; 527 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 5.4 MiB/s wr, 565 op/s 2024-09-10T09:53:31.447 INFO:tasks.cephfs.filesystem:scrub status for tag:f545d029-1355-466e-a909-f109a5713e5e - {'path': '/', 'tag': 'f545d029-1355-466e-a909-f109a5713e5e', 'options': 'recursive,force'} 2024-09-10T09:53:31.951 INFO:tasks.workunit.client.0.smithi005.stdout: 10 30614 25.96 MB/sec warmup 70 sec latency 143.367 ms 2024-09-10T09:53:32.951 INFO:tasks.workunit.client.0.smithi005.stdout: 10 31446 26.30 MB/sec warmup 71 sec latency 73.942 ms 2024-09-10T09:53:33.306 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:53:33 smithi005 ceph-mon[30350]: pgmap v849: 129 pgs: 129 active+clean; 528 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 4.2 MiB/s wr, 459 op/s 2024-09-10T09:53:33.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:53:33 smithi159 ceph-mon[35513]: pgmap v849: 129 pgs: 129 active+clean; 528 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 4.2 MiB/s wr, 459 op/s 2024-09-10T09:53:33.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:53:33 smithi102 ceph-mon[35442]: pgmap v849: 129 pgs: 129 active+clean; 528 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 4.2 MiB/s wr, 459 op/s 2024-09-10T09:53:33.951 INFO:tasks.workunit.client.0.smithi005.stdout: 10 32497 26.53 MB/sec warmup 72 sec latency 58.573 ms 2024-09-10T09:53:34.951 INFO:tasks.workunit.client.0.smithi005.stdout: 10 33222 26.91 MB/sec warmup 73 sec latency 109.396 ms 2024-09-10T09:53:35.293 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:53:35 smithi005 ceph-mon[30350]: pgmap v850: 129 pgs: 129 active+clean; 531 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 4.0 MiB/s wr, 480 op/s 2024-09-10T09:53:35.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:53:35 smithi159 ceph-mon[35513]: pgmap v850: 129 pgs: 129 active+clean; 531 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 4.0 MiB/s wr, 480 op/s 2024-09-10T09:53:35.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:53:35 smithi102 ceph-mon[35442]: pgmap v850: 129 pgs: 129 active+clean; 531 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 4.0 MiB/s wr, 480 op/s 2024-09-10T09:53:35.952 INFO:tasks.workunit.client.0.smithi005.stdout: 10 33984 27.18 MB/sec warmup 74 sec latency 41.599 ms 2024-09-10T09:53:36.953 INFO:tasks.workunit.client.0.smithi005.stdout: 10 34594 27.15 MB/sec warmup 75 sec latency 81.052 ms 2024-09-10T09:53:37.306 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:53:37 smithi005 ceph-mon[30350]: pgmap v851: 129 pgs: 129 active+clean; 538 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 3.7 MiB/s wr, 527 op/s 2024-09-10T09:53:37.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:53:37 smithi159 ceph-mon[35513]: pgmap v851: 129 pgs: 129 active+clean; 538 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 3.7 MiB/s wr, 527 op/s 2024-09-10T09:53:37.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:53:37 smithi102 ceph-mon[35442]: pgmap v851: 129 pgs: 129 active+clean; 538 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 3.7 MiB/s wr, 527 op/s 2024-09-10T09:53:37.953 INFO:tasks.workunit.client.0.smithi005.stdout: 10 34980 27.04 MB/sec warmup 76 sec latency 137.637 ms 2024-09-10T09:53:38.955 INFO:tasks.workunit.client.0.smithi005.stdout: 10 35805 27.26 MB/sec warmup 77 sec latency 46.376 ms 2024-09-10T09:53:39.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:53:39 smithi005 ceph-mon[30350]: pgmap v852: 129 pgs: 129 active+clean; 545 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 4.0 MiB/s wr, 531 op/s 2024-09-10T09:53:39.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:53:39 smithi159 ceph-mon[35513]: pgmap v852: 129 pgs: 129 active+clean; 545 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 4.0 MiB/s wr, 531 op/s 2024-09-10T09:53:39.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:53:39 smithi102 ceph-mon[35442]: pgmap v852: 129 pgs: 129 active+clean; 545 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 4.0 MiB/s wr, 531 op/s 2024-09-10T09:53:39.953 INFO:tasks.workunit.client.0.smithi005.stdout: 10 36237 27.16 MB/sec warmup 78 sec latency 626.442 ms 2024-09-10T09:53:40.953 INFO:tasks.workunit.client.0.smithi005.stdout: 10 36664 27.02 MB/sec warmup 79 sec latency 1213.254 ms 2024-09-10T09:53:41.953 INFO:tasks.workunit.client.0.smithi005.stdout: 10 37534 27.40 MB/sec warmup 80 sec latency 113.505 ms 2024-09-10T09:53:42.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:53:41 smithi005 ceph-mon[30350]: pgmap v853: 129 pgs: 129 active+clean; 559 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 5.1 MiB/s wr, 680 op/s 2024-09-10T09:53:42.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:53:41 smithi159 ceph-mon[35513]: pgmap v853: 129 pgs: 129 active+clean; 559 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 5.1 MiB/s wr, 680 op/s 2024-09-10T09:53:42.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:53:41 smithi102 ceph-mon[35442]: pgmap v853: 129 pgs: 129 active+clean; 559 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 5.1 MiB/s wr, 680 op/s 2024-09-10T09:53:42.499 INFO:tasks.quiescer.fs.[cephfs]:Total duration (39.7) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-09-10T09:53:42.499 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 19.85 2024-09-10T09:53:42.499 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T09:53:42.953 INFO:tasks.workunit.client.0.smithi005.stdout: 10 38140 27.66 MB/sec warmup 81 sec latency 319.857 ms 2024-09-10T09:53:42.979 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:53:43.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:53:42 smithi005 ceph-mon[30350]: scrub summary: idle 2024-09-10T09:53:43.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:53:42 smithi005 ceph-mon[30350]: scrub summary: idle 2024-09-10T09:53:43.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:53:42 smithi005 ceph-mon[30350]: scrub summary: idle 2024-09-10T09:53:43.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:53:42 smithi005 ceph-mon[30350]: scrub summary: idle 2024-09-10T09:53:43.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:53:42 smithi005 ceph-mon[30350]: scrub summary: idle 2024-09-10T09:53:43.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:53:42 smithi159 ceph-mon[35513]: scrub summary: idle 2024-09-10T09:53:43.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:53:42 smithi159 ceph-mon[35513]: scrub summary: idle 2024-09-10T09:53:43.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:53:42 smithi159 ceph-mon[35513]: scrub summary: idle 2024-09-10T09:53:43.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:53:42 smithi159 ceph-mon[35513]: scrub summary: idle 2024-09-10T09:53:43.101 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:53:42 smithi159 ceph-mon[35513]: scrub summary: idle 2024-09-10T09:53:43.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:53:42 smithi102 ceph-mon[35442]: scrub summary: idle 2024-09-10T09:53:43.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:53:42 smithi102 ceph-mon[35442]: scrub summary: idle 2024-09-10T09:53:43.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:53:42 smithi102 ceph-mon[35442]: scrub summary: idle 2024-09-10T09:53:43.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:53:42 smithi102 ceph-mon[35442]: scrub summary: idle 2024-09-10T09:53:43.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:53:42 smithi102 ceph-mon[35442]: scrub summary: idle 2024-09-10T09:53:43.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:43.640+0000 7ff93de00640 1 -- 172.21.15.5:0/1061195380 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff938074e30 msgr2=0x7ff93810df00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:53:43.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:43.640+0000 7ff93de00640 1 --2- 172.21.15.5:0/1061195380 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff938074e30 0x7ff93810df00 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7ff920009860 tx=0x7ff92002a3d0 comp rx=0 tx=0).stop 2024-09-10T09:53:43.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:43.640+0000 7ff93de00640 1 -- 172.21.15.5:0/1061195380 shutdown_connections 2024-09-10T09:53:43.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:43.640+0000 7ff93de00640 1 --2- 172.21.15.5:0/1061195380 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff93810e440 0x7ff938110830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:43.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:43.640+0000 7ff93de00640 1 --2- 172.21.15.5:0/1061195380 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff938074e30 0x7ff93810df00 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:43.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:43.640+0000 7ff93de00640 1 --2- 172.21.15.5:0/1061195380 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff938074510 0x7ff9380748f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:43.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:43.640+0000 7ff93de00640 1 -- 172.21.15.5:0/1061195380 >> 172.21.15.5:0/1061195380 conn(0x7ff9380fd8c0 msgr2=0x7ff9380ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:53:43.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:43.640+0000 7ff93de00640 1 -- 172.21.15.5:0/1061195380 shutdown_connections 2024-09-10T09:53:43.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:43.640+0000 7ff93de00640 1 -- 172.21.15.5:0/1061195380 wait complete. 2024-09-10T09:53:43.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:43.641+0000 7ff93de00640 1 Processor -- start 2024-09-10T09:53:43.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:43.642+0000 7ff93de00640 1 -- start start 2024-09-10T09:53:43.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:43.642+0000 7ff93de00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff938074510 0x7ff93810dde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:43.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:43.642+0000 7ff93de00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff938074e30 0x7ff93810ae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:43.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:43.642+0000 7ff93de00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff93810e440 0x7ff93810b410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:43.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:43.642+0000 7ff93de00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff938114190 con 0x7ff938074510 2024-09-10T09:53:43.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:43.642+0000 7ff93de00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7ff938114010 con 0x7ff938074e30 2024-09-10T09:53:43.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:43.642+0000 7ff93de00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7ff938114310 con 0x7ff93810e440 2024-09-10T09:53:43.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:43.643+0000 7ff936a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff938074510 0x7ff93810dde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:43.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:43.643+0000 7ff92fe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff938074e30 0x7ff93810ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:43.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:43.643+0000 7ff936a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff938074510 0x7ff93810dde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38284/0 (socket says 172.21.15.5:38284) 2024-09-10T09:53:43.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:43.643+0000 7ff936a00640 1 -- 172.21.15.5:0/1037554123 learned_addr learned my addr 172.21.15.5:0/1037554123 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:53:43.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:43.643+0000 7ff937400640 1 --2- 172.21.15.5:0/1037554123 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff93810e440 0x7ff93810b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:43.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:43.643+0000 7ff936a00640 1 -- 172.21.15.5:0/1037554123 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff93810e440 msgr2=0x7ff93810b410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:53:43.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:43.643+0000 7ff936a00640 1 --2- 172.21.15.5:0/1037554123 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff93810e440 0x7ff93810b410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:43.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:43.643+0000 7ff936a00640 1 -- 172.21.15.5:0/1037554123 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff938074e30 msgr2=0x7ff93810ae90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:53:43.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:43.643+0000 7ff936a00640 1 --2- 172.21.15.5:0/1037554123 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff938074e30 0x7ff93810ae90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:43.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:43.643+0000 7ff936a00640 1 -- 172.21.15.5:0/1037554123 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff93810bcd0 con 0x7ff938074510 2024-09-10T09:53:43.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:43.644+0000 7ff936a00640 1 --2- 172.21.15.5:0/1037554123 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff938074510 0x7ff93810dde0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7ff92800e9c0 tx=0x7ff92800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:53:43.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:43.644+0000 7ff93ca00640 1 -- 172.21.15.5:0/1037554123 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff92800cde0 con 0x7ff938074510 2024-09-10T09:53:43.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:43.644+0000 7ff93de00640 1 -- 172.21.15.5:0/1037554123 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9381aaff0 con 0x7ff938074510 2024-09-10T09:53:43.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:43.644+0000 7ff93de00640 1 -- 172.21.15.5:0/1037554123 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9381ab500 con 0x7ff938074510 2024-09-10T09:53:43.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:43.645+0000 7ff93ca00640 1 -- 172.21.15.5:0/1037554123 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff9280035f0 con 0x7ff938074510 2024-09-10T09:53:43.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:43.645+0000 7ff93ca00640 1 -- 172.21.15.5:0/1037554123 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff928010690 con 0x7ff938074510 2024-09-10T09:53:43.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:43.646+0000 7ff93de00640 1 -- 172.21.15.5:0/1037554123 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff914005290 con 0x7ff938074510 2024-09-10T09:53:43.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:43.648+0000 7ff93ca00640 1 -- 172.21.15.5:0/1037554123 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7ff928003fe0 con 0x7ff938074510 2024-09-10T09:53:43.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:43.650+0000 7ff93ca00640 1 --2- 172.21.15.5:0/1037554123 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7ff91c0a2c00 0x7ff91c0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:43.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:43.650+0000 7ff93ca00640 1 -- 172.21.15.5:0/1037554123 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7ff9280cf5a0 con 0x7ff938074510 2024-09-10T09:53:43.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:43.650+0000 7ff92fe00640 1 --2- 172.21.15.5:0/1037554123 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7ff91c0a2c00 0x7ff91c0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:43.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:43.651+0000 7ff92fe00640 1 --2- 172.21.15.5:0/1037554123 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7ff91c0a2c00 0x7ff91c0a50c0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7ff93810c6e0 tx=0x7ff920005c90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:53:43.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:43.655+0000 7ff93ca00640 1 -- 172.21.15.5:0/1037554123 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff92809c0b0 con 0x7ff938074510 2024-09-10T09:53:43.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:43.922+0000 7ff93de00640 1 -- 172.21.15.5:0/1037554123 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff914005580 con 0x7ff938074510 2024-09-10T09:53:43.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:43.923+0000 7ff93ca00640 1 -- 172.21.15.5:0/1037554123 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) ==== 94+0+11977 (secure 0 0 0) 0x7ff92809b9b0 con 0x7ff938074510 2024-09-10T09:53:43.925 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-10T09:53:43.926 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":37,"btime":"2024-09-10T09:52:27:129371+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14454,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/2970594311","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2970594311},{"type":"v1","addr":"172.21.15.5:6835","nonce":2970594311}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14514,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6839/3616502174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":3616502174},{"type":"v1","addr":"172.21.15.5:6839","nonce":3616502174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":37,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T09:35:29.626287+0000","modified":"2024-09-10T09:52:26.980498+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24542,"mds_1":24535,"mds_2":24511,"mds_3":24506,"mds_4":24443},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14520":{"gid":14520,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/2073599842","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":2073599842},{"type":"v1","addr":"172.21.15.5:6841","nonce":2073599842}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"k","rank":4,"incarnation":26,"state":"up:active","state_seq":24,"addr":"172.21.15.102:6833/2457007427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":2457007427},{"type":"v1","addr":"172.21.15.102:6833","nonce":2457007427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/356445159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":356445159},{"type":"v1","addr":"172.21.15.159:6833","nonce":356445159}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6837/2894314507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":2894314507},{"type":"v1","addr":"172.21.15.5:6837","nonce":2894314507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.102:6835/1647549111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":1647549111},{"type":"v1","addr":"172.21.15.102:6835","nonce":1647549111}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6835/952657242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":952657242},{"type":"v1","addr":"172.21.15.159:6835","nonce":952657242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":19,"addr":"172.21.15.102:6837/2689776764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":2689776764},{"type":"v1","addr":"172.21.15.102:6837","nonce":2689776764}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.159:6837/620627639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":620627639},{"type":"v1","addr":"172.21.15.159:6837","nonce":620627639}]},"join_fscid":-1,"export_targets":[0,1,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24535":{"gid":24535,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":18,"addr":"172.21.15.159:6839/2073739599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":2073739599},{"type":"v1","addr":"172.21.15.159:6839","nonce":2073739599}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24542":{"gid":24542,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.102:6839/335409749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6838","nonce":335409749},{"type":"v1","addr":"172.21.15.102:6839","nonce":335409749}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24542,"qdb_cluster":[24443,24506,24511,24535,24542]},"id":1}]} 2024-09-10T09:53:43.926 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 37 2024-09-10T09:53:43.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:43.926+0000 7ff93de00640 1 -- 172.21.15.5:0/1037554123 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7ff91c0a2c00 msgr2=0x7ff91c0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:53:43.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:43.926+0000 7ff93de00640 1 --2- 172.21.15.5:0/1037554123 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7ff91c0a2c00 0x7ff91c0a50c0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7ff93810c6e0 tx=0x7ff920005c90 comp rx=0 tx=0).stop 2024-09-10T09:53:43.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:43.926+0000 7ff93de00640 1 -- 172.21.15.5:0/1037554123 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff938074510 msgr2=0x7ff93810dde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:53:43.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:43.926+0000 7ff93de00640 1 --2- 172.21.15.5:0/1037554123 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff938074510 0x7ff93810dde0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7ff92800e9c0 tx=0x7ff92800ee80 comp rx=0 tx=0).stop 2024-09-10T09:53:43.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:43.926+0000 7ff93de00640 1 -- 172.21.15.5:0/1037554123 shutdown_connections 2024-09-10T09:53:43.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:43.926+0000 7ff93de00640 1 --2- 172.21.15.5:0/1037554123 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7ff91c0a2c00 0x7ff91c0a50c0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:43.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:43.926+0000 7ff93de00640 1 --2- 172.21.15.5:0/1037554123 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff93810e440 0x7ff93810b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:43.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:43.926+0000 7ff93de00640 1 --2- 172.21.15.5:0/1037554123 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff938074e30 0x7ff93810ae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:43.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:43.926+0000 7ff93de00640 1 --2- 172.21.15.5:0/1037554123 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff938074510 0x7ff93810dde0 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:43.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:43.926+0000 7ff93de00640 1 -- 172.21.15.5:0/1037554123 >> 172.21.15.5:0/1037554123 conn(0x7ff9380fd8c0 msgr2=0x7ff9380fe560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:53:43.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:43.927+0000 7ff93de00640 1 -- 172.21.15.5:0/1037554123 shutdown_connections 2024-09-10T09:53:43.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:43.927+0000 7ff93de00640 1 -- 172.21.15.5:0/1037554123 wait complete. 2024-09-10T09:53:43.953 INFO:tasks.workunit.client.0.smithi005.stdout: 10 38708 27.62 MB/sec warmup 82 sec latency 61.374 ms 2024-09-10T09:53:44.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:53:43 smithi005 ceph-mon[30350]: pgmap v854: 129 pgs: 129 active+clean; 561 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 4.2 MiB/s wr, 488 op/s 2024-09-10T09:53:44.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:53:43 smithi159 ceph-mon[35513]: pgmap v854: 129 pgs: 129 active+clean; 561 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 4.2 MiB/s wr, 488 op/s 2024-09-10T09:53:44.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:53:43 smithi102 ceph-mon[35442]: pgmap v854: 129 pgs: 129 active+clean; 561 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 4.2 MiB/s wr, 488 op/s 2024-09-10T09:53:44.897 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:53:44 smithi005 ceph-mon[30350]: from='client.? 172.21.15.5:0/1037554123' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:53:44.953 INFO:tasks.workunit.client.0.smithi005.stdout: 10 39023 27.42 MB/sec warmup 83 sec latency 465.564 ms 2024-09-10T09:53:45.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:53:44 smithi159 ceph-mon[35513]: from='client.? 172.21.15.5:0/1037554123' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:53:45.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:53:44 smithi102 ceph-mon[35442]: from='client.? 172.21.15.5:0/1037554123' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:53:45.953 INFO:tasks.workunit.client.0.smithi005.stdout: 10 39028 27.09 MB/sec warmup 84 sec latency 1424.115 ms 2024-09-10T09:53:45.996 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:53:45 smithi005 ceph-mon[30350]: pgmap v855: 129 pgs: 129 active+clean; 566 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 4.7 MiB/s wr, 552 op/s 2024-09-10T09:53:46.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:53:45 smithi159 ceph-mon[35513]: pgmap v855: 129 pgs: 129 active+clean; 566 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 4.7 MiB/s wr, 552 op/s 2024-09-10T09:53:46.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:53:45 smithi102 ceph-mon[35442]: pgmap v855: 129 pgs: 129 active+clean; 566 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 4.7 MiB/s wr, 552 op/s 2024-09-10T09:53:46.953 INFO:tasks.workunit.client.0.smithi005.stdout: 10 39030 26.77 MB/sec warmup 85 sec latency 2424.185 ms 2024-09-10T09:53:47.580 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24542 quiesce db / --timeout 90 --expiration 139.85 --await-for 45.0' 2024-09-10T09:53:47.580 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24542 quiesce db / --timeout 90 --expiration 139.85 --await-for 45.0 2024-09-10T09:53:47.953 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:53:47 smithi005 ceph-mon[30350]: pgmap v856: 129 pgs: 129 active+clean; 575 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 5.6 MiB/s wr, 608 op/s 2024-09-10T09:53:47.953 INFO:tasks.workunit.client.0.smithi005.stdout: 10 39030 26.46 MB/sec warmup 86 sec latency 3424.234 ms 2024-09-10T09:53:47.992 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:53:48.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:53:47 smithi159 ceph-mon[35513]: pgmap v856: 129 pgs: 129 active+clean; 575 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 5.6 MiB/s wr, 608 op/s 2024-09-10T09:53:48.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:53:47 smithi102 ceph-mon[35442]: pgmap v856: 129 pgs: 129 active+clean; 575 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 5.6 MiB/s wr, 608 op/s 2024-09-10T09:53:48.953 INFO:tasks.workunit.client.0.smithi005.stdout: 10 39031 26.16 MB/sec warmup 87 sec latency 4424.286 ms 2024-09-10T09:53:49.084 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:53:48 smithi005 ceph-mon[30350]: pgmap v857: 129 pgs: 129 active+clean; 578 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 6.0 MiB/s wr, 548 op/s 2024-09-10T09:53:49.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:53:48 smithi102 ceph-mon[35442]: pgmap v857: 129 pgs: 129 active+clean; 578 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 6.0 MiB/s wr, 548 op/s 2024-09-10T09:53:49.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:53:48 smithi159 ceph-mon[35513]: pgmap v857: 129 pgs: 129 active+clean; 578 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 6.0 MiB/s wr, 548 op/s 2024-09-10T09:53:49.953 INFO:tasks.workunit.client.0.smithi005.stdout: 10 39031 25.86 MB/sec warmup 88 sec latency 4904.193 ms 2024-09-10T09:53:50.953 INFO:tasks.workunit.client.0.smithi005.stdout: 10 39038 25.58 MB/sec warmup 89 sec latency 5901.651 ms 2024-09-10T09:53:51.248 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:53:50 smithi005 ceph-mon[30350]: pgmap v858: 129 pgs: 129 active+clean; 573 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 6.7 MiB/s wr, 536 op/s 2024-09-10T09:53:51.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:53:51 smithi159 ceph-mon[35513]: pgmap v858: 129 pgs: 129 active+clean; 573 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 6.7 MiB/s wr, 536 op/s 2024-09-10T09:53:51.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:53:50 smithi102 ceph-mon[35442]: pgmap v858: 129 pgs: 129 active+clean; 573 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 6.7 MiB/s wr, 536 op/s 2024-09-10T09:53:51.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:51.390+0000 7f22c1e00640 1 -- 172.21.15.5:0/3984177766 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f22bc11b140 msgr2=0x7f22bc11d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:53:51.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:51.390+0000 7f22c1e00640 1 --2- 172.21.15.5:0/3984177766 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f22bc11b140 0x7f22bc11d530 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f22b4009ca0 tx=0x7f22b402a8c0 comp rx=0 tx=0).stop 2024-09-10T09:53:51.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:51.391+0000 7f22c1e00640 1 -- 172.21.15.5:0/3984177766 shutdown_connections 2024-09-10T09:53:51.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:51.391+0000 7f22c1e00640 1 --2- 172.21.15.5:0/3984177766 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f22bc11b140 0x7f22bc11d530 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:51.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:51.391+0000 7f22c1e00640 1 --2- 172.21.15.5:0/3984177766 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f22bc109830 0x7f22bc109c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:51.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:51.391+0000 7f22c1e00640 1 --2- 172.21.15.5:0/3984177766 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f22bc109450 0x7f22bc110250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:51.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:51.391+0000 7f22c1e00640 1 -- 172.21.15.5:0/3984177766 >> 172.21.15.5:0/3984177766 conn(0x7f22bc06e140 msgr2=0x7f22bc06e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:53:51.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:51.392+0000 7f22c1e00640 1 -- 172.21.15.5:0/3984177766 shutdown_connections 2024-09-10T09:53:51.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:51.392+0000 7f22c1e00640 1 -- 172.21.15.5:0/3984177766 wait complete. 2024-09-10T09:53:51.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:51.393+0000 7f22c1e00640 1 Processor -- start 2024-09-10T09:53:51.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:51.394+0000 7f22c1e00640 1 -- start start 2024-09-10T09:53:51.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:51.394+0000 7f22c1e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f22bc109450 0x7f22bc118680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:51.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:51.395+0000 7f22c1e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f22bc109830 0x7f22bc118bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:51.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:51.395+0000 7f22c1e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f22bc11b140 0x7f22bc1118c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:51.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:51.395+0000 7f22baa00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f22bc109450 0x7f22bc118680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:51.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:51.395+0000 7f22c1e00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f22bc120da0 con 0x7f22bc109450 2024-09-10T09:53:51.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:51.395+0000 7f22c1e00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f22bc120c20 con 0x7f22bc11b140 2024-09-10T09:53:51.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:51.395+0000 7f22c1e00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f22bc120f20 con 0x7f22bc109830 2024-09-10T09:53:51.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:51.395+0000 7f22bb400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f22bc11b140 0x7f22bc1118c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:51.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:51.395+0000 7f22bb400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f22bc11b140 0x7f22bc1118c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:42110/0 (socket says 172.21.15.5:42110) 2024-09-10T09:53:51.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:51.395+0000 7f22bb400640 1 -- 172.21.15.5:0/1011534700 learned_addr learned my addr 172.21.15.5:0/1011534700 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:53:51.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:51.395+0000 7f22ba000640 1 --2- 172.21.15.5:0/1011534700 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f22bc109830 0x7f22bc118bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:51.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:51.396+0000 7f22bb400640 1 -- 172.21.15.5:0/1011534700 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f22bc109830 msgr2=0x7f22bc118bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:53:51.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:51.396+0000 7f22bb400640 1 --2- 172.21.15.5:0/1011534700 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f22bc109830 0x7f22bc118bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:51.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:51.396+0000 7f22bb400640 1 -- 172.21.15.5:0/1011534700 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f22bc109450 msgr2=0x7f22bc118680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:53:51.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:51.396+0000 7f22bb400640 1 --2- 172.21.15.5:0/1011534700 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f22bc109450 0x7f22bc118680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:51.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:51.396+0000 7f22bb400640 1 -- 172.21.15.5:0/1011534700 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22bc112180 con 0x7f22bc11b140 2024-09-10T09:53:51.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:51.396+0000 7f22ba000640 1 --2- 172.21.15.5:0/1011534700 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f22bc109830 0x7f22bc118bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:53:51.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:51.397+0000 7f22bb400640 1 --2- 172.21.15.5:0/1011534700 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f22bc11b140 0x7f22bc1118c0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f22b40098d0 tx=0x7f22b4030c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:53:51.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:51.397+0000 7f22c0a00640 1 -- 172.21.15.5:0/1011534700 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22b402f380 con 0x7f22bc11b140 2024-09-10T09:53:51.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:51.397+0000 7f22c0a00640 1 -- 172.21.15.5:0/1011534700 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f22b4003e50 con 0x7f22bc11b140 2024-09-10T09:53:51.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:51.397+0000 7f22c0a00640 1 -- 172.21.15.5:0/1011534700 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22b40043e0 con 0x7f22bc11b140 2024-09-10T09:53:51.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:51.398+0000 7f22c1e00640 1 -- 172.21.15.5:0/1011534700 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22bc112410 con 0x7f22bc11b140 2024-09-10T09:53:51.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:51.398+0000 7f22c1e00640 1 -- 172.21.15.5:0/1011534700 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22bc1163e0 con 0x7f22bc11b140 2024-09-10T09:53:51.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:51.404+0000 7f22c48ad740 1 Processor -- start 2024-09-10T09:53:51.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:51.404+0000 7f22c0a00640 1 -- 172.21.15.5:0/1011534700 <== mon.2 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f22b4031450 con 0x7f22bc11b140 2024-09-10T09:53:51.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:51.404+0000 7f22c48ad740 1 -- start start 2024-09-10T09:53:51.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:51.404+0000 7f22c48ad740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d756cf24e0 0x55d756cf28c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:51.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:51.404+0000 7f22c48ad740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d756cf2e00 0x55d756cfd180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:51.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:51.404+0000 7f22c48ad740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d756cfd6c0 0x55d756cffab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:51.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:51.404+0000 7f22c48ad740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55d756c57260 con 0x55d756cf24e0 2024-09-10T09:53:51.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:51.404+0000 7f22c48ad740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55d756bfe500 con 0x55d756cf2e00 2024-09-10T09:53:51.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:51.404+0000 7f22c48ad740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55d756c53220 con 0x55d756cfd6c0 2024-09-10T09:53:51.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:51.405+0000 7f22baa00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d756cf2e00 0x55d756cfd180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:51.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:51.405+0000 7f22baa00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d756cf2e00 0x55d756cfd180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:42126/0 (socket says 172.21.15.5:42126) 2024-09-10T09:53:51.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:51.405+0000 7f22baa00640 1 -- 172.21.15.5:0/2413282899 learned_addr learned my addr 172.21.15.5:0/2413282899 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:53:51.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:51.406+0000 7f22ba000640 1 --2- 172.21.15.5:0/2413282899 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d756cf24e0 0x55d756cf28c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:51.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:51.408+0000 7f22ba000640 1 --2- 172.21.15.5:0/2413282899 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d756cfd6c0 0x55d756cffab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:51.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:51.408+0000 7f22ba000640 1 -- 172.21.15.5:0/2413282899 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d756cfd6c0 msgr2=0x55d756cffab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:53:51.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:51.408+0000 7f22ba000640 1 --2- 172.21.15.5:0/2413282899 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d756cfd6c0 0x55d756cffab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:51.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:51.408+0000 7f22ba000640 1 -- 172.21.15.5:0/2413282899 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d756cf2e00 msgr2=0x55d756cfd180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:53:51.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:51.408+0000 7f22ba000640 1 --2- 172.21.15.5:0/2413282899 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d756cf2e00 0x55d756cfd180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:51.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:51.408+0000 7f22ba000640 1 -- 172.21.15.5:0/2413282899 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d7569c8ba0 con 0x55d756cf24e0 2024-09-10T09:53:51.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:51.409+0000 7f22ba000640 1 --2- 172.21.15.5:0/2413282899 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d756cf24e0 0x55d756cf28c0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f22a800bfd0 tx=0x7f22a800b410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d882436ebe2cb9a9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:53:51.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:51.409+0000 7f22baa00640 1 --2- 172.21.15.5:0/2413282899 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d756cf2e00 0x55d756cfd180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:53:51.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:51.409+0000 7f22c0a00640 1 --2- 172.21.15.5:0/1011534700 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f22a00a2cc0 0x7f22a00a5180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:51.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:51.409+0000 7f22a7e00640 1 -- 172.21.15.5:0/2413282899 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22a8014e30 con 0x55d756cf24e0 2024-09-10T09:53:51.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:51.409+0000 7f22a7e00640 1 -- 172.21.15.5:0/2413282899 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f22a800d6b0 con 0x55d756cf24e0 2024-09-10T09:53:51.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:51.409+0000 7f22a7e00640 1 -- 172.21.15.5:0/2413282899 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22a8003850 con 0x55d756cf24e0 2024-09-10T09:53:51.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:51.411+0000 7f22c48ad740 1 -- 172.21.15.5:0/2413282899 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d756cf24e0 msgr2=0x55d756cf28c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:53:51.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:51.411+0000 7f22c48ad740 1 --2- 172.21.15.5:0/2413282899 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d756cf24e0 0x55d756cf28c0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f22a800bfd0 tx=0x7f22a800b410 comp rx=0 tx=0).stop 2024-09-10T09:53:51.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:51.411+0000 7f22c0a00640 1 -- 172.21.15.5:0/1011534700 <== mon.2 v2:172.21.15.102:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f22b40f0a70 con 0x7f22bc11b140 2024-09-10T09:53:51.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:51.412+0000 7f22baa00640 1 --2- 172.21.15.5:0/1011534700 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f22a00a2cc0 0x7f22a00a5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:51.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:51.412+0000 7f22c48ad740 1 -- 172.21.15.5:0/2413282899 shutdown_connections 2024-09-10T09:53:51.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:51.412+0000 7f22c48ad740 1 --2- 172.21.15.5:0/2413282899 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d756cfd6c0 0x55d756cffab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:51.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:51.412+0000 7f22c48ad740 1 --2- 172.21.15.5:0/2413282899 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d756cf2e00 0x55d756cfd180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:51.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:51.412+0000 7f22c48ad740 1 --2- 172.21.15.5:0/2413282899 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d756cf24e0 0x55d756cf28c0 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:51.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:51.412+0000 7f22c48ad740 1 -- 172.21.15.5:0/2413282899 >> 172.21.15.5:0/2413282899 conn(0x55d756ceea70 msgr2=0x55d756ceee50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:53:51.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:51.413+0000 7f22c48ad740 1 -- 172.21.15.5:0/2413282899 shutdown_connections 2024-09-10T09:53:51.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:51.414+0000 7f22c48ad740 1 -- 172.21.15.5:0/2413282899 wait complete. 2024-09-10T09:53:51.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:51.415+0000 7f22baa00640 1 --2- 172.21.15.5:0/1011534700 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f22a00a2cc0 0x7f22a00a5180 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x55d756cef8d0 tx=0x7f22ac005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:53:51.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:51.415+0000 7f22c48ad740 1 -- start start 2024-09-10T09:53:51.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:51.416+0000 7f22c48ad740 1 Processor -- start 2024-09-10T09:53:51.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:51.416+0000 7f22c48ad740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d756cf24e0 0x55d756cfa5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:51.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:51.416+0000 7f22c48ad740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d756cf2e00 0x55d756cfab30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:51.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:51.416+0000 7f22c48ad740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d756cfd6c0 0x55d756cfb070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:51.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:51.416+0000 7f22c48ad740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55d756d00290 con 0x55d756cfd6c0 2024-09-10T09:53:51.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:51.416+0000 7f22c48ad740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55d756c11bd0 con 0x55d756cf2e00 2024-09-10T09:53:51.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:51.416+0000 7f22c48ad740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55d756d00410 con 0x55d756cf24e0 2024-09-10T09:53:51.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:51.416+0000 7f22c48ad740 10 client.0 _ll_register_callbacks cb 0x55d7569660a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:53:51.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:51.417+0000 7f22baa00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d756cf24e0 0x55d756cfa5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:51.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:51.417+0000 7f22baa00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d756cf24e0 0x55d756cfa5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:49136/0 (socket says 172.21.15.5:49136) 2024-09-10T09:53:51.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:51.417+0000 7f22baa00640 1 -- 172.21.15.5:0/3427296467 learned_addr learned my addr 172.21.15.5:0/3427296467 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:53:51.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:51.417+0000 7f22ba000640 1 --2- 172.21.15.5:0/3427296467 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d756cf2e00 0x55d756cfab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:51.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:51.417+0000 7f22bb400640 1 --2- 172.21.15.5:0/3427296467 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d756cfd6c0 0x55d756cfb070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:51.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:51.417+0000 7f22ba000640 1 -- 172.21.15.5:0/3427296467 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d756cf24e0 msgr2=0x55d756cfa5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:53:51.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:51.417+0000 7f22ba000640 1 --2- 172.21.15.5:0/3427296467 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d756cf24e0 0x55d756cfa5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:51.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:51.417+0000 7f22ba000640 1 -- 172.21.15.5:0/3427296467 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d756cfd6c0 msgr2=0x55d756cfb070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:53:51.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:51.417+0000 7f22ba000640 1 --2- 172.21.15.5:0/3427296467 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d756cfd6c0 0x55d756cfb070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:51.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:51.418+0000 7f22ba000640 1 -- 172.21.15.5:0/3427296467 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d756d09760 con 0x55d756cf2e00 2024-09-10T09:53:51.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:51.418+0000 7f22ba000640 1 --2- 172.21.15.5:0/3427296467 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d756cf2e00 0x55d756cfab30 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f22a801b000 tx=0x7f22a8014e30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=75cf05e2fddde60 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:53:51.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:51.418+0000 7f22a4c00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-09-10T09:53:51.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:51.419+0000 7f22a4c00640 1 -- 172.21.15.5:0/3427296467 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22a8023070 con 0x55d756cf2e00 2024-09-10T09:53:51.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:51.419+0000 7f22c48ad740 1 -- 172.21.15.5:0/3427296467 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f22a800d6b0 con 0x55d756cf2e00 2024-09-10T09:53:51.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:51.419+0000 7f22a4c00640 1 -- 172.21.15.5:0/3427296467 <== mon.2 v2:172.21.15.102:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f22a800dc70 con 0x55d756cf2e00 2024-09-10T09:53:51.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:51.420+0000 7f22a4c00640 1 -- 172.21.15.5:0/3427296467 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f22a802c050 con 0x55d756cf2e00 2024-09-10T09:53:51.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:51.420+0000 7f22c48ad740 10 client.28478 fetch_fsmap learned FSMap version 37 2024-09-10T09:53:51.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:51.420+0000 7f22c48ad740 1 -- 172.21.15.5:0/3427296467 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55d756d0c0d0 con 0x55d756cf2e00 2024-09-10T09:53:51.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:51.420+0000 7f22a4c00640 1 -- 172.21.15.5:0/3427296467 <== mon.2 v2:172.21.15.102:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f22a800d6b0 con 0x55d756cf2e00 2024-09-10T09:53:51.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:51.421+0000 7f22a4c00640 1 -- 172.21.15.5:0/3427296467 <== mon.2 v2:172.21.15.102:3300/0 5 ==== fsmap(e 37) ==== 8390+0+0 (secure 0 0 0) 0x7f22a8026020 con 0x55d756cf2e00 2024-09-10T09:53:51.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:51.421+0000 7f22c48ad740 10 client.28478 fetch_fsmap finished waiting for FSMap version 37 2024-09-10T09:53:51.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:51.421+0000 7f22c48ad740 10 client.28478 resolve_mds: validated gid 24542 aka daemon mds.b 2024-09-10T09:53:51.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:51.421+0000 7f22c48ad740 20 client.28478 populate_metadata read hostname 'smithi005' 2024-09-10T09:53:51.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:51.421+0000 7f22c48ad740 1 --2- 172.21.15.5:0/3427296467 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55d756d0d480 0x55d756d0f870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:51.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:51.421+0000 7f22c48ad740 4 client.28478 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:53:51.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:51.421+0000 7f22c48ad740 1 -- 172.21.15.5:0/3427296467 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d756c64990 con 0x55d756d0d480 2024-09-10T09:53:51.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:51.422+0000 7f22baa00640 1 --2- 172.21.15.5:0/3427296467 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55d756d0d480 0x55d756d0f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:51.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:51.423+0000 7f22baa00640 1 --2- 172.21.15.5:0/3427296467 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55d756d0d480 0x55d756d0f870 crc :-1 s=READY pgs=227 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=39f5a60f74fcadbb server_cookie=ce41cbe94a60d20e in_seq=0 out_seq=0 2024-09-10T09:53:51.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:51.423+0000 7f22a4c00640 10 client.28478 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T09:53:51.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:51.424+0000 7f22a4c00640 1 -- 172.21.15.5:0/3427296467 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d756c64990 con 0x55d756d0d480 2024-09-10T09:53:51.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:51.424+0000 7f22a4c00640 10 client.28478 handle_command_reply: tid=0 2024-09-10T09:53:51.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:51.424+0000 7f22c48ad740 1 client.28478 shutdown 2024-09-10T09:53:51.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:51.426+0000 7f22c48ad740 1 -- 172.21.15.5:0/3427296467 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d756cf2e00 msgr2=0x55d756cfab30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:53:51.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:51.426+0000 7f22c48ad740 1 --2- 172.21.15.5:0/3427296467 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d756cf2e00 0x55d756cfab30 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f22a801b000 tx=0x7f22a8014e30 comp rx=0 tx=0).stop 2024-09-10T09:53:51.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:51.426+0000 7f22c48ad740 1 -- 172.21.15.5:0/3427296467 shutdown_connections 2024-09-10T09:53:51.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:51.426+0000 7f22c48ad740 1 --2- 172.21.15.5:0/3427296467 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55d756d0d480 0x55d756d0f870 crc :-1 s=READY pgs=227 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:51.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:51.426+0000 7f22a4c00640 0 client.28478 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T09:53:51.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:51.426+0000 7f22c48ad740 1 --2- 172.21.15.5:0/3427296467 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d756cfd6c0 0x55d756cfb070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:51.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:51.426+0000 7f22c48ad740 1 --2- 172.21.15.5:0/3427296467 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d756cf2e00 0x55d756cfab30 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:51.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:51.427+0000 7f22c48ad740 1 --2- 172.21.15.5:0/3427296467 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d756cf24e0 0x55d756cfa5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:51.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:51.427+0000 7f22c48ad740 1 -- 172.21.15.5:0/3427296467 >> 172.21.15.5:0/3427296467 conn(0x55d756ceea70 msgr2=0x55d756cf47a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:53:51.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:51.427+0000 7f22c48ad740 1 -- 172.21.15.5:0/3427296467 shutdown_connections 2024-09-10T09:53:51.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:51.427+0000 7f22c48ad740 1 -- 172.21.15.5:0/3427296467 wait complete. 2024-09-10T09:53:51.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:51.428+0000 7f22c48ad740 20 client.28478 trim_cache size 0 max 16384 2024-09-10T09:53:51.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:51.460+0000 7f22c48ad740 1 Processor -- start 2024-09-10T09:53:51.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:51.460+0000 7f22c48ad740 1 -- start start 2024-09-10T09:53:51.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:51.461+0000 7f22c48ad740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d756cf24e0 0x55d756cf4f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:51.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:51.461+0000 7f22c48ad740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d756cf2e00 0x55d756d14ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:51.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:51.461+0000 7f22c48ad740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d756cfd6c0 0x55d756d15410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:51.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:51.461+0000 7f22c48ad740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f22bc1c6420 con 0x55d756cf24e0 2024-09-10T09:53:51.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:51.461+0000 7f22c48ad740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f22bc1c8050 con 0x55d756cf2e00 2024-09-10T09:53:51.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:51.461+0000 7f22c48ad740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f22bc1c6af0 con 0x55d756cfd6c0 2024-09-10T09:53:51.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:51.462+0000 7f22baa00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d756cf24e0 0x55d756cf4f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:51.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:51.462+0000 7f22baa00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d756cf24e0 0x55d756cf4f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57064/0 (socket says 172.21.15.5:57064) 2024-09-10T09:53:51.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:51.462+0000 7f22baa00640 1 -- 172.21.15.5:0/4115001327 learned_addr learned my addr 172.21.15.5:0/4115001327 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:53:51.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:51.463+0000 7f22ba000640 1 --2- 172.21.15.5:0/4115001327 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d756cf2e00 0x55d756d14ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:51.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:51.463+0000 7f22bb400640 1 --2- 172.21.15.5:0/4115001327 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d756cfd6c0 0x55d756d15410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:51.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:51.463+0000 7f22baa00640 1 -- 172.21.15.5:0/4115001327 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d756cfd6c0 msgr2=0x55d756d15410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:53:51.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:51.464+0000 7f22baa00640 1 --2- 172.21.15.5:0/4115001327 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d756cfd6c0 0x55d756d15410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:51.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:51.464+0000 7f22baa00640 1 -- 172.21.15.5:0/4115001327 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d756cf2e00 msgr2=0x55d756d14ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:53:51.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:51.464+0000 7f22baa00640 1 --2- 172.21.15.5:0/4115001327 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d756cf2e00 0x55d756d14ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:51.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:51.464+0000 7f22baa00640 1 -- 172.21.15.5:0/4115001327 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d756ce8fe0 con 0x55d756cf24e0 2024-09-10T09:53:51.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:51.465+0000 7f22ba000640 1 --2- 172.21.15.5:0/4115001327 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d756cf2e00 0x55d756d14ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:53:51.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:51.465+0000 7f22baa00640 1 --2- 172.21.15.5:0/4115001327 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d756cf24e0 0x55d756cf4f90 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f22ac01f940 tx=0x7f22ac022040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7667a0c8c13d16cd server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:53:51.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:51.466+0000 7f22a7400640 1 -- 172.21.15.5:0/4115001327 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22ac031070 con 0x55d756cf24e0 2024-09-10T09:53:51.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:51.466+0000 7f22a7400640 1 -- 172.21.15.5:0/4115001327 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f22ac0035f0 con 0x55d756cf24e0 2024-09-10T09:53:51.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:51.466+0000 7f22a7400640 1 -- 172.21.15.5:0/4115001327 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22ac02b690 con 0x55d756cf24e0 2024-09-10T09:53:51.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:51.467+0000 7f22c48ad740 1 -- 172.21.15.5:0/4115001327 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d756cf24e0 msgr2=0x55d756cf4f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:53:51.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:51.467+0000 7f22c48ad740 1 --2- 172.21.15.5:0/4115001327 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d756cf24e0 0x55d756cf4f90 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f22ac01f940 tx=0x7f22ac022040 comp rx=0 tx=0).stop 2024-09-10T09:53:51.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:51.468+0000 7f22c48ad740 1 -- 172.21.15.5:0/4115001327 shutdown_connections 2024-09-10T09:53:51.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:51.468+0000 7f22c48ad740 1 --2- 172.21.15.5:0/4115001327 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d756cfd6c0 0x55d756d15410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:51.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:51.468+0000 7f22c48ad740 1 --2- 172.21.15.5:0/4115001327 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d756cf2e00 0x55d756d14ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:51.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:51.468+0000 7f22c48ad740 1 --2- 172.21.15.5:0/4115001327 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d756cf24e0 0x55d756cf4f90 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:51.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:51.468+0000 7f22c48ad740 1 -- 172.21.15.5:0/4115001327 >> 172.21.15.5:0/4115001327 conn(0x55d756ceea70 msgr2=0x55d756cfc560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:53:51.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:51.468+0000 7f22c48ad740 1 -- 172.21.15.5:0/4115001327 shutdown_connections 2024-09-10T09:53:51.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:51.468+0000 7f22c48ad740 1 -- 172.21.15.5:0/4115001327 wait complete. 2024-09-10T09:53:51.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:51.469+0000 7f22c48ad740 1 -- start start 2024-09-10T09:53:51.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:51.469+0000 7f22c48ad740 1 Processor -- start 2024-09-10T09:53:51.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:51.470+0000 7f22c48ad740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d756cf24e0 0x55d756cf7610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:51.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:51.470+0000 7f22c48ad740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d756cf2e00 0x55d756cf7b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:51.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:51.470+0000 7f22c48ad740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d756cfd6c0 0x55d756cf8090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:51.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:51.470+0000 7f22c48ad740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f22bc1c7950 con 0x55d756cf2e00 2024-09-10T09:53:51.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:51.470+0000 7f22c48ad740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f22bc1c7280 con 0x55d756cf24e0 2024-09-10T09:53:51.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:51.470+0000 7f22c48ad740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f22bc1c4400 con 0x55d756cfd6c0 2024-09-10T09:53:51.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:51.470+0000 7f22c48ad740 10 client.0 _ll_register_callbacks cb 0x7f22ac0096a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:53:51.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:51.471+0000 7f22baa00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d756cf24e0 0x55d756cf7610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:51.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:51.471+0000 7f22baa00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d756cf24e0 0x55d756cf7610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:42144/0 (socket says 172.21.15.5:42144) 2024-09-10T09:53:51.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:51.471+0000 7f22baa00640 1 -- 172.21.15.5:0/2548514313 learned_addr learned my addr 172.21.15.5:0/2548514313 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:53:51.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:51.471+0000 7f22bb400640 1 --2- 172.21.15.5:0/2548514313 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d756cfd6c0 0x55d756cf8090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:51.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:51.471+0000 7f22baa00640 1 -- 172.21.15.5:0/2548514313 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d756cfd6c0 msgr2=0x55d756cf8090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:53:51.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:51.471+0000 7f22baa00640 1 --2- 172.21.15.5:0/2548514313 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d756cfd6c0 0x55d756cf8090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:51.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:51.471+0000 7f22baa00640 1 -- 172.21.15.5:0/2548514313 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d756cf2e00 msgr2=0x55d756cf7b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:53:51.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:51.471+0000 7f22baa00640 1 --2- 172.21.15.5:0/2548514313 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d756cf2e00 0x55d756cf7b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:51.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:51.471+0000 7f22baa00640 1 -- 172.21.15.5:0/2548514313 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d756d76ea0 con 0x55d756cf24e0 2024-09-10T09:53:51.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:51.472+0000 7f22baa00640 1 --2- 172.21.15.5:0/2548514313 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d756cf24e0 0x55d756cf7610 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f22ac01fe00 tx=0x7f22ac01f570 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2094ec64a0046745 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:53:51.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:51.472+0000 7f22a4c00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-09-10T09:53:51.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:51.472+0000 7f22a4c00640 1 -- 172.21.15.5:0/2548514313 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22ac03a020 con 0x55d756cf24e0 2024-09-10T09:53:51.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:51.472+0000 7f22a4c00640 1 -- 172.21.15.5:0/2548514313 <== mon.2 v2:172.21.15.102:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f22ac004250 con 0x55d756cf24e0 2024-09-10T09:53:51.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:51.472+0000 7f22c48ad740 1 -- 172.21.15.5:0/2548514313 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f22ac0035f0 con 0x55d756cf24e0 2024-09-10T09:53:51.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:51.473+0000 7f22a4c00640 1 -- 172.21.15.5:0/2548514313 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f22ac045050 con 0x55d756cf24e0 2024-09-10T09:53:51.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:51.473+0000 7f22c48ad740 10 client.28484 fetch_fsmap learned FSMap version 37 2024-09-10T09:53:51.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:51.473+0000 7f22c48ad740 1 -- 172.21.15.5:0/2548514313 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55d756d78160 con 0x55d756cf24e0 2024-09-10T09:53:51.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:51.473+0000 7f22a4c00640 1 -- 172.21.15.5:0/2548514313 <== mon.2 v2:172.21.15.102:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f22ac0035f0 con 0x55d756cf24e0 2024-09-10T09:53:51.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:51.473+0000 7f22a4c00640 1 -- 172.21.15.5:0/2548514313 <== mon.2 v2:172.21.15.102:3300/0 5 ==== fsmap(e 37) ==== 8390+0+0 (secure 0 0 0) 0x7f22ac0046c0 con 0x55d756cf24e0 2024-09-10T09:53:51.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:51.473+0000 7f22c48ad740 10 client.28484 fetch_fsmap finished waiting for FSMap version 37 2024-09-10T09:53:51.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:51.473+0000 7f22c48ad740 10 client.28484 resolve_mds: validated gid 24542 aka daemon mds.b 2024-09-10T09:53:51.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:51.473+0000 7f22c48ad740 20 client.28484 populate_metadata read hostname 'smithi005' 2024-09-10T09:53:51.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:51.473+0000 7f22c48ad740 1 --2- 172.21.15.5:0/2548514313 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55d756d0d480 0x55d756d7a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:53:51.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:51.473+0000 7f22c48ad740 4 client.28484 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 139.85, "await_for": 45.0}] 2024-09-10T09:53:51.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:51.473+0000 7f22c48ad740 1 -- 172.21.15.5:0/2548514313 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 139.85, "await_for": 45.0}) -- 0x55d756c5ece0 con 0x55d756d0d480 2024-09-10T09:53:51.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:51.474+0000 7f22ba000640 1 --2- 172.21.15.5:0/2548514313 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55d756d0d480 0x55d756d7a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:53:51.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:51.474+0000 7f22ba000640 1 --2- 172.21.15.5:0/2548514313 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55d756d0d480 0x55d756d7a8c0 crc :-1 s=READY pgs=228 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5f1ffc56dd60de52 server_cookie=5e20ecb30ea5ad8a in_seq=0 out_seq=0 2024-09-10T09:53:51.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:51.474+0000 7f22a4c00640 10 client.28484 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T09:53:51.953 INFO:tasks.workunit.client.0.smithi005.stdout: 10 39042 25.29 MB/sec warmup 90 sec latency 6462.279 ms 2024-09-10T09:53:52.954 INFO:tasks.workunit.client.0.smithi005.stdout: 10 39042 25.01 MB/sec warmup 91 sec latency 4470.707 ms 2024-09-10T09:53:53.307 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:53:52 smithi005 ceph-mon[30350]: pgmap v859: 129 pgs: 129 active+clean; 572 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 5.5 MiB/s wr, 356 op/s 2024-09-10T09:53:53.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:53:52 smithi159 ceph-mon[35513]: pgmap v859: 129 pgs: 129 active+clean; 572 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 5.5 MiB/s wr, 356 op/s 2024-09-10T09:53:53.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:53:52 smithi102 ceph-mon[35442]: pgmap v859: 129 pgs: 129 active+clean; 572 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 5.5 MiB/s wr, 356 op/s 2024-09-10T09:53:53.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:53.660+0000 7f22a4c00640 1 -- 172.21.15.5:0/2548514313 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55d756c5ece0 con 0x55d756d0d480 2024-09-10T09:53:53.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:53.660+0000 7f22a4c00640 10 client.28484 handle_command_reply: tid=0 2024-09-10T09:53:53.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:53.660+0000 7f22c48ad740 1 client.28484 shutdown 2024-09-10T09:53:53.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:53.660+0000 7f22c48ad740 1 -- 172.21.15.5:0/2548514313 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d756cf24e0 msgr2=0x55d756cf7610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:53:53.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:53.660+0000 7f22c48ad740 1 --2- 172.21.15.5:0/2548514313 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d756cf24e0 0x55d756cf7610 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f22ac01fe00 tx=0x7f22ac01f570 comp rx=0 tx=0).stop 2024-09-10T09:53:53.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:53.661+0000 7f22c48ad740 1 -- 172.21.15.5:0/2548514313 shutdown_connections 2024-09-10T09:53:53.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:53.661+0000 7f22c48ad740 1 --2- 172.21.15.5:0/2548514313 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55d756d0d480 0x55d756d7a8c0 crc :-1 s=READY pgs=228 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:53.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:53.661+0000 7f22c48ad740 1 --2- 172.21.15.5:0/2548514313 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d756cfd6c0 0x55d756cf8090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:53.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:53.661+0000 7f22c48ad740 1 --2- 172.21.15.5:0/2548514313 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d756cf2e00 0x55d756cf7b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:53.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:53.661+0000 7f22c48ad740 1 --2- 172.21.15.5:0/2548514313 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d756cf24e0 0x55d756cf7610 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:53.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:53.661+0000 7f22a4c00640 0 client.28484 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T09:53:53.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:53.661+0000 7f22c48ad740 1 -- 172.21.15.5:0/2548514313 >> 172.21.15.5:0/2548514313 conn(0x55d756ceea70 msgr2=0x55d756ced3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:53:53.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:53.661+0000 7f22c48ad740 1 -- 172.21.15.5:0/2548514313 shutdown_connections 2024-09-10T09:53:53.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:53.661+0000 7f22c48ad740 1 -- 172.21.15.5:0/2548514313 wait complete. 2024-09-10T09:53:53.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:53.661+0000 7f22c48ad740 20 client.28484 trim_cache size 0 max 16384 2024-09-10T09:53:53.663 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-10T09:53:53.664 INFO:teuthology.orchestra.run.smithi005.stdout: "epoch": 37, 2024-09-10T09:53:53.664 INFO:teuthology.orchestra.run.smithi005.stdout: "leader": 24542, 2024-09-10T09:53:53.664 INFO:teuthology.orchestra.run.smithi005.stdout: "set_version": 54, 2024-09-10T09:53:53.664 INFO:teuthology.orchestra.run.smithi005.stdout: "sets": { 2024-09-10T09:53:53.664 INFO:teuthology.orchestra.run.smithi005.stdout: "6cbb0d55": { 2024-09-10T09:53:53.664 INFO:teuthology.orchestra.run.smithi005.stdout: "version": 54, 2024-09-10T09:53:53.664 INFO:teuthology.orchestra.run.smithi005.stdout: "age_ref": 0.0, 2024-09-10T09:53:53.664 INFO:teuthology.orchestra.run.smithi005.stdout: "state": { 2024-09-10T09:53:53.664 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "QUIESCED", 2024-09-10T09:53:53.664 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0 2024-09-10T09:53:53.664 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T09:53:53.664 INFO:teuthology.orchestra.run.smithi005.stdout: "timeout": 90.0, 2024-09-10T09:53:53.664 INFO:teuthology.orchestra.run.smithi005.stdout: "expiration": 139.8, 2024-09-10T09:53:53.664 INFO:teuthology.orchestra.run.smithi005.stdout: "members": { 2024-09-10T09:53:53.665 INFO:teuthology.orchestra.run.smithi005.stdout: "file:/": { 2024-09-10T09:53:53.665 INFO:teuthology.orchestra.run.smithi005.stdout: "excluded": false, 2024-09-10T09:53:53.665 INFO:teuthology.orchestra.run.smithi005.stdout: "state": { 2024-09-10T09:53:53.665 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "QUIESCED", 2024-09-10T09:53:53.665 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0 2024-09-10T09:53:53.665 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T09:53:53.665 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T09:53:53.665 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T09:53:53.665 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T09:53:53.665 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T09:53:53.665 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-10T09:53:53.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:53.662+0000 7f22c1e00640 1 -- 172.21.15.5:0/1011534700 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f22a00a2cc0 msgr2=0x7f22a00a5180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:53:53.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:53.662+0000 7f22c1e00640 1 --2- 172.21.15.5:0/1011534700 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f22a00a2cc0 0x7f22a00a5180 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x55d756cef8d0 tx=0x7f22ac005f90 comp rx=0 tx=0).stop 2024-09-10T09:53:53.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:53.662+0000 7f22c1e00640 1 -- 172.21.15.5:0/1011534700 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f22bc11b140 msgr2=0x7f22bc1118c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:53:53.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:53.662+0000 7f22c1e00640 1 --2- 172.21.15.5:0/1011534700 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f22bc11b140 0x7f22bc1118c0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f22b40098d0 tx=0x7f22b4030c30 comp rx=0 tx=0).stop 2024-09-10T09:53:53.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:53.662+0000 7f22c1e00640 1 -- 172.21.15.5:0/1011534700 shutdown_connections 2024-09-10T09:53:53.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:53.662+0000 7f22c1e00640 1 --2- 172.21.15.5:0/1011534700 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f22a00a2cc0 0x7f22a00a5180 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:53.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:53.662+0000 7f22c1e00640 1 --2- 172.21.15.5:0/1011534700 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f22bc11b140 0x7f22bc1118c0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:53.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:53.662+0000 7f22c1e00640 1 --2- 172.21.15.5:0/1011534700 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f22bc109830 0x7f22bc118bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:53.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:53.662+0000 7f22c1e00640 1 --2- 172.21.15.5:0/1011534700 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f22bc109450 0x7f22bc118680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:53:53.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:53.662+0000 7f22c1e00640 1 -- 172.21.15.5:0/1011534700 >> 172.21.15.5:0/1011534700 conn(0x7f22bc06e140 msgr2=0x7f22bc11c220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:53:53.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:53.663+0000 7f22c1e00640 1 -- 172.21.15.5:0/1011534700 shutdown_connections 2024-09-10T09:53:53.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:53:53.663+0000 7f22c1e00640 1 -- 172.21.15.5:0/1011534700 wait complete. 2024-09-10T09:53:53.953 INFO:tasks.workunit.client.0.smithi005.stdout: 10 39048 24.74 MB/sec warmup 92 sec latency 4496.393 ms 2024-09-10T09:53:54.554 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '6cbb0d55', quiesce took 12.1 seconds. Will release after: 7.750000000000002 2024-09-10T09:53:54.954 INFO:tasks.workunit.client.0.smithi005.stdout: 10 39048 24.48 MB/sec warmup 93 sec latency 4715.586 ms 2024-09-10T09:53:55.249 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:53:54 smithi005 ceph-mon[30350]: pgmap v860: 129 pgs: 129 active+clean; 571 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 5.3 MiB/s wr, 374 op/s 2024-09-10T09:53:55.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:53:54 smithi159 ceph-mon[35513]: pgmap v860: 129 pgs: 129 active+clean; 571 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 5.3 MiB/s wr, 374 op/s 2024-09-10T09:53:55.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:53:54 smithi102 ceph-mon[35442]: pgmap v860: 129 pgs: 129 active+clean; 571 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 5.3 MiB/s wr, 374 op/s 2024-09-10T09:53:55.954 INFO:tasks.workunit.client.0.smithi005.stdout: 10 39048 24.22 MB/sec warmup 94 sec latency 5715.632 ms 2024-09-10T09:53:56.954 INFO:tasks.workunit.client.0.smithi005.stdout: 10 39048 23.96 MB/sec warmup 95 sec latency 6715.692 ms 2024-09-10T09:53:57.276 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:53:56 smithi005 ceph-mon[30350]: pgmap v861: 129 pgs: 129 active+clean; 574 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 5.8 MiB/s wr, 339 op/s 2024-09-10T09:53:57.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:53:56 smithi159 ceph-mon[35513]: pgmap v861: 129 pgs: 129 active+clean; 574 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 5.8 MiB/s wr, 339 op/s 2024-09-10T09:53:57.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:53:56 smithi102 ceph-mon[35442]: pgmap v861: 129 pgs: 129 active+clean; 574 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 5.8 MiB/s wr, 339 op/s 2024-09-10T09:53:57.954 INFO:tasks.workunit.client.0.smithi005.stdout: 10 39048 23.71 MB/sec warmup 96 sec latency 7715.755 ms 2024-09-10T09:53:58.954 INFO:tasks.workunit.client.0.smithi005.stdout: 10 39048 23.47 MB/sec warmup 97 sec latency 8715.816 ms 2024-09-10T09:53:59.014 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:53:59 smithi005 ceph-mon[30350]: pgmap v862: 129 pgs: 129 active+clean; 582 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 927 KiB/s rd, 5.5 MiB/s wr, 251 op/s 2024-09-10T09:53:59.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:53:59 smithi159 ceph-mon[35513]: pgmap v862: 129 pgs: 129 active+clean; 582 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 927 KiB/s rd, 5.5 MiB/s wr, 251 op/s 2024-09-10T09:53:59.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:53:59 smithi102 ceph-mon[35442]: pgmap v862: 129 pgs: 129 active+clean; 582 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 927 KiB/s rd, 5.5 MiB/s wr, 251 op/s 2024-09-10T09:53:59.954 INFO:tasks.workunit.client.0.smithi005.stdout: 10 39048 23.23 MB/sec warmup 98 sec latency 9715.881 ms 2024-09-10T09:54:00.954 INFO:tasks.workunit.client.0.smithi005.stdout: 10 39048 22.99 MB/sec warmup 99 sec latency 10715.923 ms 2024-09-10T09:54:01.255 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:54:01 smithi005 ceph-mon[30350]: pgmap v863: 129 pgs: 129 active+clean; 592 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 684 KiB/s rd, 5.7 MiB/s wr, 246 op/s 2024-09-10T09:54:01.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:54:01 smithi159 ceph-mon[35513]: pgmap v863: 129 pgs: 129 active+clean; 592 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 684 KiB/s rd, 5.7 MiB/s wr, 246 op/s 2024-09-10T09:54:01.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:54:01 smithi102 ceph-mon[35442]: pgmap v863: 129 pgs: 129 active+clean; 592 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 684 KiB/s rd, 5.7 MiB/s wr, 246 op/s 2024-09-10T09:54:01.448 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T09:54:01.828 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:54:01.954 INFO:tasks.workunit.client.0.smithi005.stdout: 10 39048 22.76 MB/sec warmup 100 sec latency 11715.983 ms 2024-09-10T09:54:02.305 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T09:54:02.954 INFO:tasks.workunit.client.0.smithi005.stdout: 10 39048 22.54 MB/sec warmup 101 sec latency 12716.045 ms 2024-09-10T09:54:03.176 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:54:03 smithi005 ceph-mon[30350]: pgmap v864: 129 pgs: 129 active+clean; 593 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 293 KiB/s rd, 4.9 MiB/s wr, 195 op/s 2024-09-10T09:54:03.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:54:03 smithi159 ceph-mon[35513]: pgmap v864: 129 pgs: 129 active+clean; 593 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 293 KiB/s rd, 4.9 MiB/s wr, 195 op/s 2024-09-10T09:54:03.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:54:03 smithi102 ceph-mon[35442]: pgmap v864: 129 pgs: 129 active+clean; 593 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 293 KiB/s rd, 4.9 MiB/s wr, 195 op/s 2024-09-10T09:54:03.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:03.434+0000 7f2634c00640 1 -- 172.21.15.5:0/3661512781 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2630102360 msgr2=0x7f263010e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:03.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:03.434+0000 7f2634c00640 1 --2- 172.21.15.5:0/3661512781 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2630102360 0x7f263010e890 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f2628009cd0 tx=0x7f262802a8f0 comp rx=0 tx=0).stop 2024-09-10T09:54:03.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:03.434+0000 7f2634c00640 1 -- 172.21.15.5:0/3661512781 shutdown_connections 2024-09-10T09:54:03.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:03.434+0000 7f2634c00640 1 --2- 172.21.15.5:0/3661512781 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2630102360 0x7f263010e890 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:03.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:03.434+0000 7f2634c00640 1 --2- 172.21.15.5:0/3661512781 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f26301019c0 0x7f2630101e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:03.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:03.434+0000 7f2634c00640 1 --2- 172.21.15.5:0/3661512781 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f26301079c0 0x7f2630107da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:03.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:03.434+0000 7f2634c00640 1 -- 172.21.15.5:0/3661512781 >> 172.21.15.5:0/3661512781 conn(0x7f26300fd740 msgr2=0x7f26300ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:54:03.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:03.434+0000 7f2634c00640 1 -- 172.21.15.5:0/3661512781 shutdown_connections 2024-09-10T09:54:03.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:03.434+0000 7f2634c00640 1 -- 172.21.15.5:0/3661512781 wait complete. 2024-09-10T09:54:03.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:03.434+0000 7f2634c00640 1 Processor -- start 2024-09-10T09:54:03.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:03.434+0000 7f2634c00640 1 -- start start 2024-09-10T09:54:03.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:03.434+0000 7f2634c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f26301019c0 0x7f26300744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:03.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:03.434+0000 7f2634c00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2630102360 0x7f2630074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:03.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:03.434+0000 7f2634c00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f26301079c0 0x7f2630074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:03.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:03.434+0000 7f2634c00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f2630111dc0 con 0x7f2630102360 2024-09-10T09:54:03.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:03.434+0000 7f2634c00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f2630111c40 con 0x7f26301019c0 2024-09-10T09:54:03.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:03.434+0000 7f2634c00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f2630111f40 con 0x7f26301079c0 2024-09-10T09:54:03.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:03.434+0000 7f262ea00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2630102360 0x7f2630074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:03.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:03.434+0000 7f262ea00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2630102360 0x7f2630074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53356/0 (socket says 172.21.15.5:53356) 2024-09-10T09:54:03.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:03.434+0000 7f262ea00640 1 -- 172.21.15.5:0/2785455072 learned_addr learned my addr 172.21.15.5:0/2785455072 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:54:03.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:03.434+0000 7f262f400640 1 --2- 172.21.15.5:0/2785455072 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f26301019c0 0x7f26300744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:03.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:03.440+0000 7f262fe00640 1 --2- 172.21.15.5:0/2785455072 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f26301079c0 0x7f2630074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:03.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:03.440+0000 7f262ea00640 1 -- 172.21.15.5:0/2785455072 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f26301079c0 msgr2=0x7f2630074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:03.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:03.440+0000 7f262ea00640 1 --2- 172.21.15.5:0/2785455072 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f26301079c0 0x7f2630074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:03.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:03.440+0000 7f262ea00640 1 -- 172.21.15.5:0/2785455072 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f26301019c0 msgr2=0x7f26300744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:03.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:03.440+0000 7f262ea00640 1 --2- 172.21.15.5:0/2785455072 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f26301019c0 0x7f26300744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:03.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:03.440+0000 7f262ea00640 1 -- 172.21.15.5:0/2785455072 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26301a6720 con 0x7f2630102360 2024-09-10T09:54:03.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:03.440+0000 7f262f400640 1 --2- 172.21.15.5:0/2785455072 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f26301019c0 0x7f26300744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:54:03.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:03.440+0000 7f262ea00640 1 --2- 172.21.15.5:0/2785455072 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2630102360 0x7f2630074a30 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f261c00e9f0 tx=0x7f261c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:54:03.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:03.440+0000 7f2627e00640 1 -- 172.21.15.5:0/2785455072 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f261c00cd80 con 0x7f2630102360 2024-09-10T09:54:03.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:03.440+0000 7f2627e00640 1 -- 172.21.15.5:0/2785455072 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f261c005500 con 0x7f2630102360 2024-09-10T09:54:03.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:03.440+0000 7f2634c00640 1 -- 172.21.15.5:0/2785455072 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26301a6a10 con 0x7f2630102360 2024-09-10T09:54:03.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:03.440+0000 7f2627e00640 1 -- 172.21.15.5:0/2785455072 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f261c010690 con 0x7f2630102360 2024-09-10T09:54:03.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:03.440+0000 7f2634c00640 1 -- 172.21.15.5:0/2785455072 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26301a6f00 con 0x7f2630102360 2024-09-10T09:54:03.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:03.440+0000 7f2638ca7740 1 Processor -- start 2024-09-10T09:54:03.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:03.440+0000 7f2638ca7740 1 -- start start 2024-09-10T09:54:03.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:03.440+0000 7f2627e00640 1 -- 172.21.15.5:0/2785455072 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f261c0046c0 con 0x7f2630102360 2024-09-10T09:54:03.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:03.440+0000 7f2638ca7740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56295cb48d50 0x56295cb49130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:03.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:03.440+0000 7f2638ca7740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56295cb49670 0x56295cb539f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:03.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:03.440+0000 7f2638ca7740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56295cb53f30 0x56295cb56320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:03.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:03.445+0000 7f2638ca7740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x56295caaf3e0 con 0x56295cb53f30 2024-09-10T09:54:03.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:03.445+0000 7f2638ca7740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x56295ca56d80 con 0x56295cb48d50 2024-09-10T09:54:03.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:03.445+0000 7f2638ca7740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x56295caab3a0 con 0x56295cb49670 2024-09-10T09:54:03.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:03.445+0000 7f262f400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56295cb49670 0x56295cb539f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:03.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:03.445+0000 7f262f400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56295cb49670 0x56295cb539f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:54352/0 (socket says 172.21.15.5:54352) 2024-09-10T09:54:03.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:03.445+0000 7f262f400640 1 -- 172.21.15.5:0/1892112880 learned_addr learned my addr 172.21.15.5:0/1892112880 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:54:03.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:03.445+0000 7f262ea00640 1 --2- 172.21.15.5:0/1892112880 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56295cb53f30 0x56295cb56320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:03.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:03.445+0000 7f262fe00640 1 --2- 172.21.15.5:0/1892112880 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56295cb48d50 0x56295cb49130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:03.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:03.445+0000 7f262f400640 1 -- 172.21.15.5:0/1892112880 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56295cb48d50 msgr2=0x56295cb49130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:54:03.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:03.445+0000 7f262f400640 1 --2- 172.21.15.5:0/1892112880 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56295cb48d50 0x56295cb49130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:03.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:03.445+0000 7f262f400640 1 -- 172.21.15.5:0/1892112880 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56295cb53f30 msgr2=0x56295cb56320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:54:03.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:03.445+0000 7f262f400640 1 --2- 172.21.15.5:0/1892112880 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56295cb53f30 0x56295cb56320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:03.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:03.445+0000 7f262f400640 1 -- 172.21.15.5:0/1892112880 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56295ca75ae0 con 0x56295cb49670 2024-09-10T09:54:03.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:03.446+0000 7f262f400640 1 --2- 172.21.15.5:0/1892112880 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56295cb49670 0x56295cb539f0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f26200114f0 tx=0x7f26200119b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4cd18a386e4e333a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:54:03.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:03.446+0000 7f261be00640 1 -- 172.21.15.5:0/1892112880 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f262001e070 con 0x56295cb49670 2024-09-10T09:54:03.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:03.446+0000 7f261be00640 1 -- 172.21.15.5:0/1892112880 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f262000faa0 con 0x56295cb49670 2024-09-10T09:54:03.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:03.446+0000 7f261be00640 1 -- 172.21.15.5:0/1892112880 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2620019b50 con 0x56295cb49670 2024-09-10T09:54:03.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:03.446+0000 7f2638ca7740 1 -- 172.21.15.5:0/1892112880 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56295cb49670 msgr2=0x56295cb539f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:03.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:03.446+0000 7f2638ca7740 1 --2- 172.21.15.5:0/1892112880 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56295cb49670 0x56295cb539f0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f26200114f0 tx=0x7f26200119b0 comp rx=0 tx=0).stop 2024-09-10T09:54:03.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:03.446+0000 7f2638ca7740 1 -- 172.21.15.5:0/1892112880 shutdown_connections 2024-09-10T09:54:03.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:03.446+0000 7f2638ca7740 1 --2- 172.21.15.5:0/1892112880 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56295cb53f30 0x56295cb56320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:03.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:03.446+0000 7f2638ca7740 1 --2- 172.21.15.5:0/1892112880 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56295cb49670 0x56295cb539f0 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:03.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:03.446+0000 7f2638ca7740 1 --2- 172.21.15.5:0/1892112880 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56295cb48d50 0x56295cb49130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:03.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:03.446+0000 7f2638ca7740 1 -- 172.21.15.5:0/1892112880 >> 172.21.15.5:0/1892112880 conn(0x56295cb452e0 msgr2=0x56295cb456c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:54:03.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:03.446+0000 7f2638ca7740 1 -- 172.21.15.5:0/1892112880 shutdown_connections 2024-09-10T09:54:03.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:03.446+0000 7f2638ca7740 1 -- 172.21.15.5:0/1892112880 wait complete. 2024-09-10T09:54:03.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:03.446+0000 7f2627e00640 1 --2- 172.21.15.5:0/2785455072 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f26140a2a90 0x7f26140a4f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:03.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:03.446+0000 7f2627e00640 1 -- 172.21.15.5:0/2785455072 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f261c0cf870 con 0x7f2630102360 2024-09-10T09:54:03.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:03.446+0000 7f262f400640 1 --2- 172.21.15.5:0/2785455072 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f26140a2a90 0x7f26140a4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:03.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:03.446+0000 7f2638ca7740 1 -- start start 2024-09-10T09:54:03.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:03.446+0000 7f2638ca7740 1 Processor -- start 2024-09-10T09:54:03.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:03.446+0000 7f262f400640 1 --2- 172.21.15.5:0/2785455072 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f26140a2a90 0x7f26140a4f50 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f2620009e20 tx=0x7f26200096c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:54:03.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:03.446+0000 7f2638ca7740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56295cb48d50 0x56295cb50dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:03.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:03.446+0000 7f2638ca7740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56295cb49670 0x56295cb51300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:03.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:03.446+0000 7f2638ca7740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56295cb53f30 0x56295cb51840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:03.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:03.446+0000 7f2638ca7740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x56295cb56ba0 con 0x56295cb48d50 2024-09-10T09:54:03.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:03.446+0000 7f2638ca7740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x56295ca6a0f0 con 0x56295cb53f30 2024-09-10T09:54:03.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:03.446+0000 7f2638ca7740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x56295cb56d20 con 0x56295cb49670 2024-09-10T09:54:03.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:03.446+0000 7f2638ca7740 10 client.0 _ll_register_callbacks cb 0x56295ca5b9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:54:03.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:03.446+0000 7f262f400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56295cb48d50 0x56295cb50dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:03.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:03.446+0000 7f262f400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56295cb48d50 0x56295cb50dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53370/0 (socket says 172.21.15.5:53370) 2024-09-10T09:54:03.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:03.446+0000 7f262f400640 1 -- 172.21.15.5:0/3888151424 learned_addr learned my addr 172.21.15.5:0/3888151424 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:54:03.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:03.446+0000 7f262f400640 1 -- 172.21.15.5:0/3888151424 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56295cb49670 msgr2=0x56295cb51300 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T09:54:03.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:03.446+0000 7f262f400640 1 --2- 172.21.15.5:0/3888151424 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56295cb49670 0x56295cb51300 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:03.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:03.446+0000 7f262f400640 1 -- 172.21.15.5:0/3888151424 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56295cb53f30 msgr2=0x56295cb51840 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T09:54:03.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:03.446+0000 7f262f400640 1 --2- 172.21.15.5:0/3888151424 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56295cb53f30 0x56295cb51840 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:03.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:03.446+0000 7f262f400640 1 -- 172.21.15.5:0/3888151424 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56295c8203c0 con 0x56295cb48d50 2024-09-10T09:54:03.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:03.446+0000 7f262f400640 1 --2- 172.21.15.5:0/3888151424 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56295cb48d50 0x56295cb50dc0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f262001b000 tx=0x7f2620019670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=973df2466ae4828f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:54:03.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:03.446+0000 7f2618c00640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-10T09:54:03.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:03.446+0000 7f2618c00640 1 -- 172.21.15.5:0/3888151424 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2620033060 con 0x56295cb48d50 2024-09-10T09:54:03.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:03.446+0000 7f2618c00640 1 -- 172.21.15.5:0/3888151424 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f2620019890 con 0x56295cb48d50 2024-09-10T09:54:03.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:03.446+0000 7f2638ca7740 1 -- 172.21.15.5:0/3888151424 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f262000faa0 con 0x56295cb48d50 2024-09-10T09:54:03.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:03.451+0000 7f2618c00640 1 -- 172.21.15.5:0/3888151424 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f262001e040 con 0x56295cb48d50 2024-09-10T09:54:03.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:03.451+0000 7f2638ca7740 10 client.18435 fetch_fsmap learned FSMap version 37 2024-09-10T09:54:03.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:03.451+0000 7f2638ca7740 1 -- 172.21.15.5:0/3888151424 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x56295cb627e0 con 0x56295cb48d50 2024-09-10T09:54:03.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:03.451+0000 7f2618c00640 1 -- 172.21.15.5:0/3888151424 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f262000faa0 con 0x56295cb48d50 2024-09-10T09:54:03.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:03.451+0000 7f2618c00640 1 -- 172.21.15.5:0/3888151424 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 37) ==== 8390+0+0 (secure 0 0 0) 0x7f262002d030 con 0x56295cb48d50 2024-09-10T09:54:03.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:03.451+0000 7f2638ca7740 10 client.18435 fetch_fsmap finished waiting for FSMap version 37 2024-09-10T09:54:03.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:03.451+0000 7f2638ca7740 10 client.18435 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T09:54:03.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:03.451+0000 7f2638ca7740 20 client.18435 populate_metadata read hostname 'smithi005' 2024-09-10T09:54:03.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:03.451+0000 7f2638ca7740 1 --2- 172.21.15.5:0/3888151424 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x56295cb63b80 0x56295cb66010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:03.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:03.451+0000 7f2638ca7740 4 client.18435 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:54:03.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:03.451+0000 7f2638ca7740 1 -- 172.21.15.5:0/3888151424 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56295cabc950 con 0x56295cb63b80 2024-09-10T09:54:03.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:03.451+0000 7f262ea00640 1 --2- 172.21.15.5:0/3888151424 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x56295cb63b80 0x56295cb66010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:03.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:03.451+0000 7f262ea00640 1 --2- 172.21.15.5:0/3888151424 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x56295cb63b80 0x56295cb66010 crc :-1 s=READY pgs=229 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4112eafffdba3759 server_cookie=75d0db8ceeba5eda in_seq=0 out_seq=0 2024-09-10T09:54:03.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:03.451+0000 7f2618c00640 10 client.18435 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T09:54:03.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:03.451+0000 7f2618c00640 1 -- 172.21.15.5:0/3888151424 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56295cabc950 con 0x56295cb63b80 2024-09-10T09:54:03.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:03.451+0000 7f2618c00640 10 client.18435 handle_command_reply: tid=0 2024-09-10T09:54:03.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:03.451+0000 7f2638ca7740 1 client.18435 shutdown 2024-09-10T09:54:03.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:03.451+0000 7f2638ca7740 1 -- 172.21.15.5:0/3888151424 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56295cb48d50 msgr2=0x56295cb50dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:03.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:03.451+0000 7f2638ca7740 1 --2- 172.21.15.5:0/3888151424 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56295cb48d50 0x56295cb50dc0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f262001b000 tx=0x7f2620019670 comp rx=0 tx=0).stop 2024-09-10T09:54:03.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:03.451+0000 7f2638ca7740 1 -- 172.21.15.5:0/3888151424 shutdown_connections 2024-09-10T09:54:03.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:03.451+0000 7f2638ca7740 1 --2- 172.21.15.5:0/3888151424 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x56295cb63b80 0x56295cb66010 crc :-1 s=READY pgs=229 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:03.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:03.451+0000 7f2638ca7740 1 --2- 172.21.15.5:0/3888151424 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56295cb53f30 0x56295cb51840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:03.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:03.451+0000 7f2638ca7740 1 --2- 172.21.15.5:0/3888151424 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56295cb49670 0x56295cb51300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:03.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:03.451+0000 7f2638ca7740 1 --2- 172.21.15.5:0/3888151424 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56295cb48d50 0x56295cb50dc0 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:03.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:03.451+0000 7f2638ca7740 1 -- 172.21.15.5:0/3888151424 >> 172.21.15.5:0/3888151424 conn(0x56295cb452e0 msgr2=0x56295cb4afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:54:03.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:03.451+0000 7f2618c00640 0 client.18435 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T09:54:03.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:03.451+0000 7f2638ca7740 1 -- 172.21.15.5:0/3888151424 shutdown_connections 2024-09-10T09:54:03.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:03.451+0000 7f2638ca7740 1 -- 172.21.15.5:0/3888151424 wait complete. 2024-09-10T09:54:03.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:03.451+0000 7f2638ca7740 20 client.18435 trim_cache size 0 max 16384 2024-09-10T09:54:03.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:03.469+0000 7f2638ca7740 1 Processor -- start 2024-09-10T09:54:03.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:03.469+0000 7f2638ca7740 1 -- start start 2024-09-10T09:54:03.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:03.469+0000 7f2638ca7740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56295cb48d50 0x56295cb4f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:03.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:03.469+0000 7f2638ca7740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56295cb49670 0x56295cb44270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:03.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:03.469+0000 7f2638ca7740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56295cb53f30 0x56295cb6b920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:03.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:03.469+0000 7f2638ca7740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f26301b0ef0 con 0x56295cb53f30 2024-09-10T09:54:03.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:03.469+0000 7f2638ca7740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f26301b2b20 con 0x56295cb48d50 2024-09-10T09:54:03.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:03.469+0000 7f2638ca7740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f26301b15c0 con 0x56295cb49670 2024-09-10T09:54:03.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:03.469+0000 7f262f400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56295cb48d50 0x56295cb4f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:03.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:03.469+0000 7f262f400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56295cb48d50 0x56295cb4f6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:45428/0 (socket says 172.21.15.5:45428) 2024-09-10T09:54:03.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:03.469+0000 7f262f400640 1 -- 172.21.15.5:0/446206905 learned_addr learned my addr 172.21.15.5:0/446206905 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:54:03.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:03.469+0000 7f262ea00640 1 --2- 172.21.15.5:0/446206905 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56295cb49670 0x56295cb44270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:03.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:03.469+0000 7f262fe00640 1 --2- 172.21.15.5:0/446206905 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56295cb53f30 0x56295cb6b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:03.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:03.469+0000 7f262f400640 1 -- 172.21.15.5:0/446206905 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56295cb49670 msgr2=0x56295cb44270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:54:03.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:03.469+0000 7f262f400640 1 --2- 172.21.15.5:0/446206905 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56295cb49670 0x56295cb44270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:03.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:03.469+0000 7f262f400640 1 -- 172.21.15.5:0/446206905 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56295cb53f30 msgr2=0x56295cb6b920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:54:03.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:03.469+0000 7f262f400640 1 --2- 172.21.15.5:0/446206905 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56295cb53f30 0x56295cb6b920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:03.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:03.469+0000 7f262f400640 1 -- 172.21.15.5:0/446206905 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56295cb4fd90 con 0x56295cb48d50 2024-09-10T09:54:03.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:03.469+0000 7f262ea00640 1 --2- 172.21.15.5:0/446206905 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56295cb49670 0x56295cb44270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:54:03.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:03.469+0000 7f262f400640 1 --2- 172.21.15.5:0/446206905 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56295cb48d50 0x56295cb4f6b0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f2620009bd0 tx=0x7f2620009f60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cf91f433302a1525 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:54:03.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:03.469+0000 7f261b400640 1 -- 172.21.15.5:0/446206905 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2620011030 con 0x56295cb48d50 2024-09-10T09:54:03.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:03.469+0000 7f261b400640 1 -- 172.21.15.5:0/446206905 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2620019d00 con 0x56295cb48d50 2024-09-10T09:54:03.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:03.469+0000 7f261b400640 1 -- 172.21.15.5:0/446206905 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2620003a60 con 0x56295cb48d50 2024-09-10T09:54:03.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:03.469+0000 7f2638ca7740 1 -- 172.21.15.5:0/446206905 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56295cb48d50 msgr2=0x56295cb4f6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:03.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:03.469+0000 7f2638ca7740 1 --2- 172.21.15.5:0/446206905 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56295cb48d50 0x56295cb4f6b0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f2620009bd0 tx=0x7f2620009f60 comp rx=0 tx=0).stop 2024-09-10T09:54:03.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:03.469+0000 7f2638ca7740 1 -- 172.21.15.5:0/446206905 shutdown_connections 2024-09-10T09:54:03.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:03.469+0000 7f2638ca7740 1 --2- 172.21.15.5:0/446206905 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56295cb53f30 0x56295cb6b920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:03.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:03.469+0000 7f2638ca7740 1 --2- 172.21.15.5:0/446206905 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56295cb49670 0x56295cb44270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:03.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:03.469+0000 7f2638ca7740 1 --2- 172.21.15.5:0/446206905 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56295cb48d50 0x56295cb4f6b0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:03.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:03.469+0000 7f2638ca7740 1 -- 172.21.15.5:0/446206905 >> 172.21.15.5:0/446206905 conn(0x56295cb452e0 msgr2=0x56295cb43b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:54:03.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:03.469+0000 7f2638ca7740 1 -- 172.21.15.5:0/446206905 shutdown_connections 2024-09-10T09:54:03.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:03.469+0000 7f2638ca7740 1 -- 172.21.15.5:0/446206905 wait complete. 2024-09-10T09:54:03.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:03.475+0000 7f2638ca7740 1 -- start start 2024-09-10T09:54:03.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:03.475+0000 7f2638ca7740 1 Processor -- start 2024-09-10T09:54:03.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:03.475+0000 7f2638ca7740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56295cb48d50 0x56295cb5c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:03.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:03.475+0000 7f2638ca7740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56295cb49670 0x56295cb4d990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:03.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:03.475+0000 7f2638ca7740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56295cb53f30 0x56295cb4ded0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:03.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:03.475+0000 7f2638ca7740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f26301b2420 con 0x56295cb53f30 2024-09-10T09:54:03.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:03.475+0000 7f2638ca7740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f26301b1d50 con 0x56295cb48d50 2024-09-10T09:54:03.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:03.475+0000 7f2638ca7740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f26301aeed0 con 0x56295cb49670 2024-09-10T09:54:03.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:03.475+0000 7f2638ca7740 10 client.0 _ll_register_callbacks cb 0x7f2620019550 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:54:03.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:03.475+0000 7f262f400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56295cb48d50 0x56295cb5c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:03.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:03.475+0000 7f262f400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56295cb48d50 0x56295cb5c4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:45442/0 (socket says 172.21.15.5:45442) 2024-09-10T09:54:03.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:03.475+0000 7f262f400640 1 -- 172.21.15.5:0/3140841429 learned_addr learned my addr 172.21.15.5:0/3140841429 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:54:03.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:03.475+0000 7f262fe00640 1 --2- 172.21.15.5:0/3140841429 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56295cb53f30 0x56295cb4ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:03.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:03.475+0000 7f262ea00640 1 --2- 172.21.15.5:0/3140841429 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56295cb49670 0x56295cb4d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:03.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:03.475+0000 7f262f400640 1 -- 172.21.15.5:0/3140841429 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56295cb49670 msgr2=0x56295cb4d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:54:03.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:03.475+0000 7f262f400640 1 --2- 172.21.15.5:0/3140841429 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56295cb49670 0x56295cb4d990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:03.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:03.475+0000 7f262f400640 1 -- 172.21.15.5:0/3140841429 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56295cb53f30 msgr2=0x56295cb4ded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:54:03.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:03.475+0000 7f262f400640 1 --2- 172.21.15.5:0/3140841429 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56295cb53f30 0x56295cb4ded0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:03.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:03.475+0000 7f262f400640 1 -- 172.21.15.5:0/3140841429 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56295cb4e510 con 0x56295cb48d50 2024-09-10T09:54:03.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:03.475+0000 7f262ea00640 1 --2- 172.21.15.5:0/3140841429 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56295cb49670 0x56295cb4d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:54:03.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:03.475+0000 7f262f400640 1 --2- 172.21.15.5:0/3140841429 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56295cb48d50 0x56295cb5c4b0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f2620024490 tx=0x7f2620005f10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=749affdc85bd8b17 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:54:03.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:03.475+0000 7f2618c00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-09-10T09:54:03.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:03.475+0000 7f2618c00640 1 -- 172.21.15.5:0/3140841429 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26200037e0 con 0x56295cb48d50 2024-09-10T09:54:03.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:03.475+0000 7f2638ca7740 1 -- 172.21.15.5:0/3140841429 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2620019d00 con 0x56295cb48d50 2024-09-10T09:54:03.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:03.475+0000 7f2618c00640 1 -- 172.21.15.5:0/3140841429 <== mon.2 v2:172.21.15.102:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f2620003980 con 0x56295cb48d50 2024-09-10T09:54:03.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:03.475+0000 7f2618c00640 1 -- 172.21.15.5:0/3140841429 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f2620003b80 con 0x56295cb48d50 2024-09-10T09:54:03.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:03.475+0000 7f2638ca7740 10 client.28499 fetch_fsmap learned FSMap version 37 2024-09-10T09:54:03.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:03.475+0000 7f2638ca7740 1 -- 172.21.15.5:0/3140841429 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x56295cbce770 con 0x56295cb48d50 2024-09-10T09:54:03.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:03.475+0000 7f2618c00640 1 -- 172.21.15.5:0/3140841429 <== mon.2 v2:172.21.15.102:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2620019d00 con 0x56295cb48d50 2024-09-10T09:54:03.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:03.475+0000 7f2618c00640 1 -- 172.21.15.5:0/3140841429 <== mon.2 v2:172.21.15.102:3300/0 5 ==== fsmap(e 37) ==== 8390+0+0 (secure 0 0 0) 0x7f262002d070 con 0x56295cb48d50 2024-09-10T09:54:03.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:03.475+0000 7f2638ca7740 10 client.28499 fetch_fsmap finished waiting for FSMap version 37 2024-09-10T09:54:03.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:03.475+0000 7f2638ca7740 10 client.28499 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T09:54:03.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:03.475+0000 7f2638ca7740 20 client.28499 populate_metadata read hostname 'smithi005' 2024-09-10T09:54:03.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:03.475+0000 7f2638ca7740 1 --2- 172.21.15.5:0/3140841429 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x56295cbcec00 0x56295cbd1020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:03.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:03.475+0000 7f2638ca7740 4 client.28499 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T09:54:03.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:03.475+0000 7f2638ca7740 1 -- 172.21.15.5:0/3140841429 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56295cab6e60 con 0x56295cbcec00 2024-09-10T09:54:03.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:03.475+0000 7f262ea00640 1 --2- 172.21.15.5:0/3140841429 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x56295cbcec00 0x56295cbd1020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:03.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:03.475+0000 7f262ea00640 1 --2- 172.21.15.5:0/3140841429 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x56295cbcec00 0x56295cbd1020 crc :-1 s=READY pgs=230 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bff64ce2868db81 server_cookie=4e28baff6f782320 in_seq=0 out_seq=0 2024-09-10T09:54:03.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:03.475+0000 7f2618c00640 10 client.28499 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T09:54:03.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:03.475+0000 7f2618c00640 1 -- 172.21.15.5:0/3140841429 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x56295cab6e60 con 0x56295cbcec00 2024-09-10T09:54:03.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:03.475+0000 7f2618c00640 10 client.28499 handle_command_reply: tid=0 2024-09-10T09:54:03.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:03.475+0000 7f2638ca7740 1 client.28499 shutdown 2024-09-10T09:54:03.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:03.475+0000 7f2638ca7740 1 -- 172.21.15.5:0/3140841429 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56295cb48d50 msgr2=0x56295cb5c4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:03.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:03.475+0000 7f2638ca7740 1 --2- 172.21.15.5:0/3140841429 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56295cb48d50 0x56295cb5c4b0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f2620024490 tx=0x7f2620005f10 comp rx=0 tx=0).stop 2024-09-10T09:54:03.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:03.475+0000 7f2638ca7740 1 -- 172.21.15.5:0/3140841429 shutdown_connections 2024-09-10T09:54:03.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:03.475+0000 7f2638ca7740 1 --2- 172.21.15.5:0/3140841429 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x56295cbcec00 0x56295cbd1020 crc :-1 s=READY pgs=230 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:03.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:03.475+0000 7f2638ca7740 1 --2- 172.21.15.5:0/3140841429 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56295cb53f30 0x56295cb4ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:03.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:03.475+0000 7f2638ca7740 1 --2- 172.21.15.5:0/3140841429 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56295cb49670 0x56295cb4d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:03.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:03.475+0000 7f2638ca7740 1 --2- 172.21.15.5:0/3140841429 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56295cb48d50 0x56295cb5c4b0 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:03.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:03.475+0000 7f2638ca7740 1 -- 172.21.15.5:0/3140841429 >> 172.21.15.5:0/3140841429 conn(0x56295cb452e0 msgr2=0x56295cb43b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:54:03.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:03.480+0000 7f2618c00640 0 client.28499 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T09:54:03.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:03.480+0000 7f2638ca7740 1 -- 172.21.15.5:0/3140841429 shutdown_connections 2024-09-10T09:54:03.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:03.480+0000 7f2638ca7740 1 -- 172.21.15.5:0/3140841429 wait complete. 2024-09-10T09:54:03.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:03.480+0000 7f2638ca7740 20 client.28499 trim_cache size 0 max 16384 2024-09-10T09:54:03.611 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-10T09:54:03.612 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "no active scrubs running", 2024-09-10T09:54:03.612 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": {} 2024-09-10T09:54:03.612 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-10T09:54:03.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:03.480+0000 7f2634c00640 1 -- 172.21.15.5:0/2785455072 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f26140a2a90 msgr2=0x7f26140a4f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:03.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:03.480+0000 7f2634c00640 1 --2- 172.21.15.5:0/2785455072 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f26140a2a90 0x7f26140a4f50 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f2620009e20 tx=0x7f26200096c0 comp rx=0 tx=0).stop 2024-09-10T09:54:03.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:03.480+0000 7f2634c00640 1 -- 172.21.15.5:0/2785455072 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2630102360 msgr2=0x7f2630074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:03.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:03.480+0000 7f2634c00640 1 --2- 172.21.15.5:0/2785455072 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2630102360 0x7f2630074a30 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f261c00e9f0 tx=0x7f261c00edf0 comp rx=0 tx=0).stop 2024-09-10T09:54:03.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:03.480+0000 7f2634c00640 1 -- 172.21.15.5:0/2785455072 shutdown_connections 2024-09-10T09:54:03.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:03.480+0000 7f2634c00640 1 --2- 172.21.15.5:0/2785455072 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f26140a2a90 0x7f26140a4f50 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:03.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:03.480+0000 7f2634c00640 1 --2- 172.21.15.5:0/2785455072 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f26301079c0 0x7f2630074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:03.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:03.480+0000 7f2634c00640 1 --2- 172.21.15.5:0/2785455072 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2630102360 0x7f2630074a30 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:03.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:03.480+0000 7f2634c00640 1 --2- 172.21.15.5:0/2785455072 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f26301019c0 0x7f26300744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:03.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:03.480+0000 7f2634c00640 1 -- 172.21.15.5:0/2785455072 >> 172.21.15.5:0/2785455072 conn(0x7f26300fd740 msgr2=0x7f26300fdf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:54:03.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:03.480+0000 7f2634c00640 1 -- 172.21.15.5:0/2785455072 shutdown_connections 2024-09-10T09:54:03.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:03.480+0000 7f2634c00640 1 -- 172.21.15.5:0/2785455072 wait complete. 2024-09-10T09:54:03.660 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:54:03.954 INFO:tasks.workunit.client.0.smithi005.stdout: 10 39048 22.32 MB/sec warmup 102 sec latency 13716.313 ms 2024-09-10T09:54:04.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:04.411+0000 7f2f29200640 1 -- 172.21.15.5:0/1152747099 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2f241091f0 msgr2=0x7f2f241095d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:04.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:04.411+0000 7f2f29200640 1 --2- 172.21.15.5:0/1152747099 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2f241091f0 0x7f2f241095d0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f2f1c009ca0 tx=0x7f2f1c02a8c0 comp rx=0 tx=0).stop 2024-09-10T09:54:04.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:04.411+0000 7f2f29200640 1 -- 172.21.15.5:0/1152747099 shutdown_connections 2024-09-10T09:54:04.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:04.411+0000 7f2f29200640 1 --2- 172.21.15.5:0/1152747099 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2f24116ea0 0x7f2f24119290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:04.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:04.411+0000 7f2f29200640 1 --2- 172.21.15.5:0/1152747099 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2f24109b10 0x7f2f24116960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:04.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:04.411+0000 7f2f29200640 1 --2- 172.21.15.5:0/1152747099 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2f241091f0 0x7f2f241095d0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:04.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:04.412+0000 7f2f29200640 1 -- 172.21.15.5:0/1152747099 >> 172.21.15.5:0/1152747099 conn(0x7f2f2406e140 msgr2=0x7f2f2406e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:54:04.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:04.412+0000 7f2f29200640 1 -- 172.21.15.5:0/1152747099 shutdown_connections 2024-09-10T09:54:04.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:04.413+0000 7f2f29200640 1 -- 172.21.15.5:0/1152747099 wait complete. 2024-09-10T09:54:04.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:04.414+0000 7f2f29200640 1 Processor -- start 2024-09-10T09:54:04.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:04.414+0000 7f2f29200640 1 -- start start 2024-09-10T09:54:04.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:04.414+0000 7f2f29200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2f241091f0 0x7f2f241a2dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:04.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:04.414+0000 7f2f29200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2f24109b10 0x7f2f241a3310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:04.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:04.414+0000 7f2f29200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2f24116ea0 0x7f2f241a76a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:04.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:04.414+0000 7f2f29200640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f2f2411c9b0 con 0x7f2f241091f0 2024-09-10T09:54:04.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:04.414+0000 7f2f29200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f2f2411c830 con 0x7f2f24109b10 2024-09-10T09:54:04.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:04.414+0000 7f2f29200640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f2f2411cb30 con 0x7f2f24116ea0 2024-09-10T09:54:04.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:04.415+0000 7f2f21e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2f241091f0 0x7f2f241a2dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:04.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:04.415+0000 7f2f21e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2f241091f0 0x7f2f241a2dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53410/0 (socket says 172.21.15.5:53410) 2024-09-10T09:54:04.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:04.415+0000 7f2f21e00640 1 -- 172.21.15.5:0/1750639481 learned_addr learned my addr 172.21.15.5:0/1750639481 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:54:04.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:04.415+0000 7f2f22800640 1 --2- 172.21.15.5:0/1750639481 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2f24116ea0 0x7f2f241a76a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:04.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:04.415+0000 7f2f21400640 1 --2- 172.21.15.5:0/1750639481 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2f24109b10 0x7f2f241a3310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:04.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:04.415+0000 7f2f21e00640 1 -- 172.21.15.5:0/1750639481 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2f24116ea0 msgr2=0x7f2f241a76a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:04.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:04.415+0000 7f2f21e00640 1 --2- 172.21.15.5:0/1750639481 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2f24116ea0 0x7f2f241a76a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:04.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:04.415+0000 7f2f21e00640 1 -- 172.21.15.5:0/1750639481 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2f24109b10 msgr2=0x7f2f241a3310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:04.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:04.415+0000 7f2f21e00640 1 --2- 172.21.15.5:0/1750639481 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2f24109b10 0x7f2f241a3310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:04.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:04.416+0000 7f2f21e00640 1 -- 172.21.15.5:0/1750639481 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f241a7d80 con 0x7f2f241091f0 2024-09-10T09:54:04.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:04.417+0000 7f2f21e00640 1 --2- 172.21.15.5:0/1750639481 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2f241091f0 0x7f2f241a2dd0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f2f1c02ade0 tx=0x7f2f1c02f4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:54:04.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:04.418+0000 7f2f1aa00640 1 -- 172.21.15.5:0/1750639481 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f1c02faa0 con 0x7f2f241091f0 2024-09-10T09:54:04.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:04.418+0000 7f2f29200640 1 -- 172.21.15.5:0/1750639481 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f241a8010 con 0x7f2f241091f0 2024-09-10T09:54:04.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:04.418+0000 7f2f1aa00640 1 -- 172.21.15.5:0/1750639481 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2f1c02fc40 con 0x7f2f241091f0 2024-09-10T09:54:04.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:04.418+0000 7f2f1aa00640 1 -- 172.21.15.5:0/1750639481 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f1c057840 con 0x7f2f241091f0 2024-09-10T09:54:04.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:04.418+0000 7f2f29200640 1 -- 172.21.15.5:0/1750639481 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f2410c510 con 0x7f2f241091f0 2024-09-10T09:54:04.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:04.422+0000 7f2f29200640 1 -- 172.21.15.5:0/1750639481 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f241095d0 con 0x7f2f241091f0 2024-09-10T09:54:04.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:04.423+0000 7f2f1aa00640 1 -- 172.21.15.5:0/1750639481 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f2f1c0305c0 con 0x7f2f241091f0 2024-09-10T09:54:04.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:04.425+0000 7f2f1aa00640 1 --2- 172.21.15.5:0/1750639481 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f2f080a2c00 0x7f2f080a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:04.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:04.426+0000 7f2f1aa00640 1 -- 172.21.15.5:0/1750639481 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f2f1c101880 con 0x7f2f241091f0 2024-09-10T09:54:04.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:04.429+0000 7f2f21400640 1 --2- 172.21.15.5:0/1750639481 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f2f080a2c00 0x7f2f080a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:04.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:04.433+0000 7f2f21400640 1 --2- 172.21.15.5:0/1750639481 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f2f080a2c00 0x7f2f080a50c0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f2f241185c0 tx=0x7f2f0c013ba0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:54:04.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:04.435+0000 7f2f1aa00640 1 -- 172.21.15.5:0/1750639481 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f2f1c0ce420 con 0x7f2f241091f0 2024-09-10T09:54:04.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:04.683+0000 7f2f29200640 1 -- 172.21.15.5:0/1750639481 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2f2410d9b0 con 0x7f2f241091f0 2024-09-10T09:54:04.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:04.684+0000 7f2f1aa00640 1 -- 172.21.15.5:0/1750639481 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) ==== 94+0+11977 (secure 0 0 0) 0x7f2f1c0e5ee0 con 0x7f2f241091f0 2024-09-10T09:54:04.685 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-10T09:54:04.685 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":37,"btime":"2024-09-10T09:52:27:129371+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14454,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/2970594311","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2970594311},{"type":"v1","addr":"172.21.15.5:6835","nonce":2970594311}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14514,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6839/3616502174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":3616502174},{"type":"v1","addr":"172.21.15.5:6839","nonce":3616502174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":37,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T09:35:29.626287+0000","modified":"2024-09-10T09:52:26.980498+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24542,"mds_1":24535,"mds_2":24511,"mds_3":24506,"mds_4":24443},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14520":{"gid":14520,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/2073599842","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":2073599842},{"type":"v1","addr":"172.21.15.5:6841","nonce":2073599842}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"k","rank":4,"incarnation":26,"state":"up:active","state_seq":24,"addr":"172.21.15.102:6833/2457007427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":2457007427},{"type":"v1","addr":"172.21.15.102:6833","nonce":2457007427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/356445159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":356445159},{"type":"v1","addr":"172.21.15.159:6833","nonce":356445159}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6837/2894314507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":2894314507},{"type":"v1","addr":"172.21.15.5:6837","nonce":2894314507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.102:6835/1647549111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":1647549111},{"type":"v1","addr":"172.21.15.102:6835","nonce":1647549111}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6835/952657242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":952657242},{"type":"v1","addr":"172.21.15.159:6835","nonce":952657242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":19,"addr":"172.21.15.102:6837/2689776764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":2689776764},{"type":"v1","addr":"172.21.15.102:6837","nonce":2689776764}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.159:6837/620627639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":620627639},{"type":"v1","addr":"172.21.15.159:6837","nonce":620627639}]},"join_fscid":-1,"export_targets":[0,1,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24535":{"gid":24535,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":18,"addr":"172.21.15.159:6839/2073739599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":2073739599},{"type":"v1","addr":"172.21.15.159:6839","nonce":2073739599}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24542":{"gid":24542,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.102:6839/335409749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6838","nonce":335409749},{"type":"v1","addr":"172.21.15.102:6839","nonce":335409749}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24542,"qdb_cluster":[24443,24506,24511,24535,24542]},"id":1}]} 2024-09-10T09:54:04.686 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 37 2024-09-10T09:54:04.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:04.687+0000 7f2f29200640 1 -- 172.21.15.5:0/1750639481 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f2f080a2c00 msgr2=0x7f2f080a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:04.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:04.687+0000 7f2f29200640 1 --2- 172.21.15.5:0/1750639481 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f2f080a2c00 0x7f2f080a50c0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f2f241185c0 tx=0x7f2f0c013ba0 comp rx=0 tx=0).stop 2024-09-10T09:54:04.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:04.687+0000 7f2f29200640 1 -- 172.21.15.5:0/1750639481 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2f241091f0 msgr2=0x7f2f241a2dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:04.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:04.687+0000 7f2f29200640 1 --2- 172.21.15.5:0/1750639481 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2f241091f0 0x7f2f241a2dd0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f2f1c02ade0 tx=0x7f2f1c02f4e0 comp rx=0 tx=0).stop 2024-09-10T09:54:04.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:04.687+0000 7f2f29200640 1 -- 172.21.15.5:0/1750639481 shutdown_connections 2024-09-10T09:54:04.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:04.687+0000 7f2f29200640 1 --2- 172.21.15.5:0/1750639481 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f2f080a2c00 0x7f2f080a50c0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:04.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:04.687+0000 7f2f29200640 1 --2- 172.21.15.5:0/1750639481 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2f24116ea0 0x7f2f241a76a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:04.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:04.687+0000 7f2f29200640 1 --2- 172.21.15.5:0/1750639481 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2f24109b10 0x7f2f241a3310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:04.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:04.687+0000 7f2f29200640 1 --2- 172.21.15.5:0/1750639481 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2f241091f0 0x7f2f241a2dd0 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:04.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:04.687+0000 7f2f29200640 1 -- 172.21.15.5:0/1750639481 >> 172.21.15.5:0/1750639481 conn(0x7f2f2406e140 msgr2=0x7f2f24117380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:54:04.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:04.687+0000 7f2f29200640 1 -- 172.21.15.5:0/1750639481 shutdown_connections 2024-09-10T09:54:04.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:04.687+0000 7f2f29200640 1 -- 172.21.15.5:0/1750639481 wait complete. 2024-09-10T09:54:04.954 INFO:tasks.workunit.client.0.smithi005.stdout: 10 39048 22.10 MB/sec warmup 103 sec latency 14716.355 ms 2024-09-10T09:54:05.153 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-10T09:54:05.153 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T09:54:05.186 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:54:05 smithi005 ceph-mon[30350]: pgmap v865: 129 pgs: 129 active+clean; 595 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 271 KiB/s rd, 5.3 MiB/s wr, 201 op/s 2024-09-10T09:54:05.186 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:54:05 smithi005 ceph-mon[30350]: from='client.? 172.21.15.5:0/1750639481' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:54:05.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:54:05 smithi159 ceph-mon[35513]: pgmap v865: 129 pgs: 129 active+clean; 595 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 271 KiB/s rd, 5.3 MiB/s wr, 201 op/s 2024-09-10T09:54:05.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:54:05 smithi159 ceph-mon[35513]: from='client.? 172.21.15.5:0/1750639481' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:54:05.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:54:05 smithi102 ceph-mon[35442]: pgmap v865: 129 pgs: 129 active+clean; 595 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 271 KiB/s rd, 5.3 MiB/s wr, 201 op/s 2024-09-10T09:54:05.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:54:05 smithi102 ceph-mon[35442]: from='client.? 172.21.15.5:0/1750639481' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:54:05.964 INFO:tasks.workunit.client.0.smithi005.stdout: 10 39048 21.89 MB/sec warmup 104 sec latency 15726.044 ms 2024-09-10T09:54:06.035 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:54:06 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:54:06.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:54:06 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:54:06.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:54:06 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:54:06.932 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24542 quiesce db --set-id 6cbb0d55 --release --await' 2024-09-10T09:54:06.932 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24542 quiesce db --set-id 6cbb0d55 --release --await 2024-09-10T09:54:06.964 INFO:tasks.workunit.client.0.smithi005.stdout: 10 39048 21.68 MB/sec warmup 105 sec latency 16726.099 ms 2024-09-10T09:54:07.024 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:54:07.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:54:07 smithi005 ceph-mon[30350]: pgmap v866: 129 pgs: 129 active+clean; 598 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 262 KiB/s rd, 6.1 MiB/s wr, 198 op/s 2024-09-10T09:54:07.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:54:07 smithi159 ceph-mon[35513]: pgmap v866: 129 pgs: 129 active+clean; 598 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 262 KiB/s rd, 6.1 MiB/s wr, 198 op/s 2024-09-10T09:54:07.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:54:07 smithi102 ceph-mon[35442]: pgmap v866: 129 pgs: 129 active+clean; 598 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 262 KiB/s rd, 6.1 MiB/s wr, 198 op/s 2024-09-10T09:54:07.964 INFO:tasks.workunit.client.0.smithi005.stdout: 10 39048 21.47 MB/sec warmup 106 sec latency 17726.160 ms 2024-09-10T09:54:08.286 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:54:08 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:54:08.286 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:54:08 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:54:08.287 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:54:08 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:54:08.287 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:54:08 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:54:08.287 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:54:08 smithi005 ceph-mon[30350]: Adjusting osd_memory_target on smithi005 to 177.5M 2024-09-10T09:54:08.287 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:54:08 smithi005 ceph-mon[30350]: Unable to set osd_memory_target on smithi005 to 186147225: error parsing value: Value '186147225' is below minimum 939524096 2024-09-10T09:54:08.287 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:54:08 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:54:08.287 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:54:08 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:54:08.287 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:54:08 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:54:08.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:54:08 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:54:08.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:54:08 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:54:08.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:54:08 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:54:08.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:54:08 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:54:08.351 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:54:08 smithi159 ceph-mon[35513]: Adjusting osd_memory_target on smithi005 to 177.5M 2024-09-10T09:54:08.351 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:54:08 smithi159 ceph-mon[35513]: Unable to set osd_memory_target on smithi005 to 186147225: error parsing value: Value '186147225' is below minimum 939524096 2024-09-10T09:54:08.351 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:54:08 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:54:08.351 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:54:08 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:54:08.351 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:54:08 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:54:08.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:54:08 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:54:08.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:54:08 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:54:08.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:54:08 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:54:08.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:54:08 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:54:08.385 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:54:08 smithi102 ceph-mon[35442]: Adjusting osd_memory_target on smithi005 to 177.5M 2024-09-10T09:54:08.385 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:54:08 smithi102 ceph-mon[35442]: Unable to set osd_memory_target on smithi005 to 186147225: error parsing value: Value '186147225' is below minimum 939524096 2024-09-10T09:54:08.385 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:54:08 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:54:08.385 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:54:08 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:54:08.385 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:54:08 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:54:08.964 INFO:tasks.workunit.client.0.smithi005.stdout: 10 39048 21.27 MB/sec warmup 107 sec latency 18726.226 ms 2024-09-10T09:54:09.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:08.787+0000 7fa222800640 1 -- 172.21.15.5:0/4153460240 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa21c103e30 msgr2=0x7fa21c104210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:09.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:08.787+0000 7fa222800640 1 --2- 172.21.15.5:0/4153460240 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa21c103e30 0x7fa21c104210 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fa210009900 tx=0x7fa21002a410 comp rx=0 tx=0).stop 2024-09-10T09:54:09.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:08.787+0000 7fa222800640 1 -- 172.21.15.5:0/4153460240 shutdown_connections 2024-09-10T09:54:09.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:08.787+0000 7fa222800640 1 --2- 172.21.15.5:0/4153460240 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa21c108f60 0x7fa21c110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:09.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:08.787+0000 7fa222800640 1 --2- 172.21.15.5:0/4153460240 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa21c1047e0 0x7fa21c108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:09.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:08.787+0000 7fa222800640 1 --2- 172.21.15.5:0/4153460240 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa21c103e30 0x7fa21c104210 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:09.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:08.787+0000 7fa222800640 1 -- 172.21.15.5:0/4153460240 >> 172.21.15.5:0/4153460240 conn(0x7fa21c0ff910 msgr2=0x7fa21c101d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:54:09.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:08.787+0000 7fa222800640 1 -- 172.21.15.5:0/4153460240 shutdown_connections 2024-09-10T09:54:09.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:08.787+0000 7fa222800640 1 -- 172.21.15.5:0/4153460240 wait complete. 2024-09-10T09:54:09.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:08.788+0000 7fa222800640 1 Processor -- start 2024-09-10T09:54:09.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:08.789+0000 7fa222800640 1 -- start start 2024-09-10T09:54:09.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:08.789+0000 7fa222800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa21c103e30 0x7fa21c19a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:09.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:08.789+0000 7fa222800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa21c1047e0 0x7fa21c19a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:09.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:08.789+0000 7fa222800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa21c108f60 0x7fa21c19ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:09.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:08.789+0000 7fa222800640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa21c114010 con 0x7fa21c1047e0 2024-09-10T09:54:09.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:08.789+0000 7fa222800640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fa21c113e90 con 0x7fa21c108f60 2024-09-10T09:54:09.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:08.789+0000 7fa222800640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fa21c114190 con 0x7fa21c103e30 2024-09-10T09:54:09.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:08.789+0000 7fa221400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa21c103e30 0x7fa21c19a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:09.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:08.789+0000 7fa221400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa21c103e30 0x7fa21c19a4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:54422/0 (socket says 172.21.15.5:54422) 2024-09-10T09:54:09.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:08.789+0000 7fa221400640 1 -- 172.21.15.5:0/2916818365 learned_addr learned my addr 172.21.15.5:0/2916818365 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:54:09.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:08.789+0000 7fa220a00640 1 --2- 172.21.15.5:0/2916818365 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa21c1047e0 0x7fa21c19a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:09.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:08.790+0000 7fa221e00640 1 --2- 172.21.15.5:0/2916818365 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa21c108f60 0x7fa21c19ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:09.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:08.790+0000 7fa221400640 1 -- 172.21.15.5:0/2916818365 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa21c108f60 msgr2=0x7fa21c19ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:09.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:08.790+0000 7fa221400640 1 --2- 172.21.15.5:0/2916818365 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa21c108f60 0x7fa21c19ed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:09.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:08.790+0000 7fa221400640 1 -- 172.21.15.5:0/2916818365 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa21c1047e0 msgr2=0x7fa21c19a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:09.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:08.790+0000 7fa221400640 1 --2- 172.21.15.5:0/2916818365 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa21c1047e0 0x7fa21c19a9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:09.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:08.790+0000 7fa221400640 1 -- 172.21.15.5:0/2916818365 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa21c19f460 con 0x7fa21c103e30 2024-09-10T09:54:09.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:08.790+0000 7fa221e00640 1 --2- 172.21.15.5:0/2916818365 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa21c108f60 0x7fa21c19ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T09:54:09.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:08.790+0000 7fa220a00640 1 --2- 172.21.15.5:0/2916818365 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa21c1047e0 0x7fa21c19a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:54:09.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:08.790+0000 7fa221400640 1 --2- 172.21.15.5:0/2916818365 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa21c103e30 0x7fa21c19a4b0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fa210004770 tx=0x7fa210003c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:54:09.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:08.790+0000 7fa21a000640 1 -- 172.21.15.5:0/2916818365 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa21003f070 con 0x7fa21c103e30 2024-09-10T09:54:09.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:08.790+0000 7fa222800640 1 -- 172.21.15.5:0/2916818365 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa21c19f6f0 con 0x7fa21c103e30 2024-09-10T09:54:09.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:08.790+0000 7fa21a000640 1 -- 172.21.15.5:0/2916818365 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa210031ca0 con 0x7fa21c103e30 2024-09-10T09:54:09.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:08.790+0000 7fa21a000640 1 -- 172.21.15.5:0/2916818365 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa210035030 con 0x7fa21c103e30 2024-09-10T09:54:09.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:08.791+0000 7fa222800640 1 -- 172.21.15.5:0/2916818365 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa21c1a7030 con 0x7fa21c103e30 2024-09-10T09:54:09.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:08.792+0000 7fa222800640 1 -- 172.21.15.5:0/2916818365 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa21c104210 con 0x7fa21c103e30 2024-09-10T09:54:09.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:08.794+0000 7fa21a000640 1 -- 172.21.15.5:0/2916818365 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7fa210042070 con 0x7fa21c103e30 2024-09-10T09:54:09.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:08.795+0000 7fa21a000640 1 --2- 172.21.15.5:0/2916818365 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fa2040a2b80 0x7fa2040a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:09.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:08.795+0000 7fa21a000640 1 -- 172.21.15.5:0/2916818365 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fa2100efdd0 con 0x7fa21c103e30 2024-09-10T09:54:09.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:08.795+0000 7fa220a00640 1 --2- 172.21.15.5:0/2916818365 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fa2040a2b80 0x7fa2040a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:09.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:08.796+0000 7fa220a00640 1 --2- 172.21.15.5:0/2916818365 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fa2040a2b80 0x7fa2040a5040 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fa21c0ffd30 tx=0x7fa208005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:54:09.150 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-10T09:54:09.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:08.797+0000 7fa21a000640 1 -- 172.21.15.5:0/2916818365 <== mon.1 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa2100bc9a0 con 0x7fa21c103e30 2024-09-10T09:54:09.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:09.024+0000 7fa222800640 1 -- 172.21.15.5:0/2916818365 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa21c10c140 con 0x7fa21c103e30 2024-09-10T09:54:09.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:09.025+0000 7fa21a000640 1 -- 172.21.15.5:0/2916818365 <== mon.1 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) ==== 94+0+11977 (secure 0 0 0) 0x7fa2100dae20 con 0x7fa21c103e30 2024-09-10T09:54:09.151 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":37,"btime":"2024-09-10T09:52:27:129371+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14454,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/2970594311","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2970594311},{"type":"v1","addr":"172.21.15.5:6835","nonce":2970594311}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14514,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6839/3616502174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":3616502174},{"type":"v1","addr":"172.21.15.5:6839","nonce":3616502174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":37,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T09:35:29.626287+0000","modified":"2024-09-10T09:52:26.980498+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24542,"mds_1":24535,"mds_2":24511,"mds_3":24506,"mds_4":24443},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14520":{"gid":14520,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/2073599842","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":2073599842},{"type":"v1","addr":"172.21.15.5:6841","nonce":2073599842}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"k","rank":4,"incarnation":26,"state":"up:active","state_seq":24,"addr":"172.21.15.102:6833/2457007427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":2457007427},{"type":"v1","addr":"172.21.15.102:6833","nonce":2457007427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/356445159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":356445159},{"type":"v1","addr":"172.21.15.159:6833","nonce":356445159}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6837/2894314507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":2894314507},{"type":"v1","addr":"172.21.15.5:6837","nonce":2894314507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.102:6835/1647549111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":1647549111},{"type":"v1","addr":"172.21.15.102:6835","nonce":1647549111}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6835/952657242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":952657242},{"type":"v1","addr":"172.21.15.159:6835","nonce":952657242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":19,"addr":"172.21.15.102:6837/2689776764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":2689776764},{"type":"v1","addr":"172.21.15.102:6837","nonce":2689776764}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.159:6837/620627639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":620627639},{"type":"v1","addr":"172.21.15.159:6837","nonce":620627639}]},"join_fscid":-1,"export_targets":[0,1,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24535":{"gid":24535,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":18,"addr":"172.21.15.159:6839/2073739599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":2073739599},{"type":"v1","addr":"172.21.15.159:6839","nonce":2073739599}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24542":{"gid":24542,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.102:6839/335409749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6838","nonce":335409749},{"type":"v1","addr":"172.21.15.102:6839","nonce":335409749}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24542,"qdb_cluster":[24443,24506,24511,24535,24542]},"id":1}]} 2024-09-10T09:54:09.151 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 37 2024-09-10T09:54:09.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:09.027+0000 7fa222800640 1 -- 172.21.15.5:0/2916818365 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fa2040a2b80 msgr2=0x7fa2040a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:09.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:09.027+0000 7fa222800640 1 --2- 172.21.15.5:0/2916818365 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fa2040a2b80 0x7fa2040a5040 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fa21c0ffd30 tx=0x7fa208005f90 comp rx=0 tx=0).stop 2024-09-10T09:54:09.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:09.027+0000 7fa222800640 1 -- 172.21.15.5:0/2916818365 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa21c103e30 msgr2=0x7fa21c19a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:09.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:09.027+0000 7fa222800640 1 --2- 172.21.15.5:0/2916818365 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa21c103e30 0x7fa21c19a4b0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fa210004770 tx=0x7fa210003c10 comp rx=0 tx=0).stop 2024-09-10T09:54:09.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:09.027+0000 7fa222800640 1 -- 172.21.15.5:0/2916818365 shutdown_connections 2024-09-10T09:54:09.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:09.027+0000 7fa222800640 1 --2- 172.21.15.5:0/2916818365 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fa2040a2b80 0x7fa2040a5040 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:09.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:09.027+0000 7fa222800640 1 --2- 172.21.15.5:0/2916818365 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa21c108f60 0x7fa21c19ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:09.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:09.027+0000 7fa222800640 1 --2- 172.21.15.5:0/2916818365 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa21c1047e0 0x7fa21c19a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:09.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:09.027+0000 7fa222800640 1 --2- 172.21.15.5:0/2916818365 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa21c103e30 0x7fa21c19a4b0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:09.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:09.027+0000 7fa222800640 1 -- 172.21.15.5:0/2916818365 >> 172.21.15.5:0/2916818365 conn(0x7fa21c0ff910 msgr2=0x7fa21c100c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:54:09.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:09.028+0000 7fa222800640 1 -- 172.21.15.5:0/2916818365 shutdown_connections 2024-09-10T09:54:09.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:09.028+0000 7fa222800640 1 -- 172.21.15.5:0/2916818365 wait complete. 2024-09-10T09:54:09.677 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:54:09 smithi005 ceph-mon[30350]: pgmap v867: 129 pgs: 129 active+clean; 605 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 206 KiB/s rd, 6.0 MiB/s wr, 176 op/s 2024-09-10T09:54:09.677 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:54:09 smithi005 ceph-mon[30350]: from='client.? 172.21.15.5:0/2916818365' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:54:09.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:54:09 smithi159 ceph-mon[35513]: pgmap v867: 129 pgs: 129 active+clean; 605 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 206 KiB/s rd, 6.0 MiB/s wr, 176 op/s 2024-09-10T09:54:09.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:54:09 smithi159 ceph-mon[35513]: from='client.? 172.21.15.5:0/2916818365' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:54:09.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:54:09 smithi102 ceph-mon[35442]: pgmap v867: 129 pgs: 129 active+clean; 605 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 206 KiB/s rd, 6.0 MiB/s wr, 176 op/s 2024-09-10T09:54:09.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:54:09 smithi102 ceph-mon[35442]: from='client.? 172.21.15.5:0/2916818365' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:54:09.964 INFO:tasks.workunit.client.0.smithi005.stdout: 10 39048 21.08 MB/sec warmup 108 sec latency 19726.272 ms 2024-09-10T09:54:10.964 INFO:tasks.workunit.client.0.smithi005.stdout: 10 39048 20.88 MB/sec warmup 109 sec latency 20726.332 ms 2024-09-10T09:54:11.573 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:54:11 smithi005 ceph-mon[30350]: pgmap v868: 129 pgs: 129 active+clean; 587 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 255 KiB/s rd, 6.7 MiB/s wr, 219 op/s 2024-09-10T09:54:11.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:54:11 smithi159 ceph-mon[35513]: pgmap v868: 129 pgs: 129 active+clean; 587 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 255 KiB/s rd, 6.7 MiB/s wr, 219 op/s 2024-09-10T09:54:11.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:54:11 smithi102 ceph-mon[35442]: pgmap v868: 129 pgs: 129 active+clean; 587 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 255 KiB/s rd, 6.7 MiB/s wr, 219 op/s 2024-09-10T09:54:11.964 INFO:tasks.workunit.client.0.smithi005.stdout: 10 39048 20.69 MB/sec warmup 110 sec latency 21726.402 ms 2024-09-10T09:54:11.980 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:54:12.009 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-10T09:54:12.964 INFO:tasks.workunit.client.0.smithi005.stdout: 10 39048 20.51 MB/sec warmup 111 sec latency 22726.457 ms 2024-09-10T09:54:13.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:54:13 smithi005 ceph-mon[30350]: pgmap v869: 129 pgs: 129 active+clean; 573 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 232 KiB/s rd, 5.6 MiB/s wr, 205 op/s 2024-09-10T09:54:13.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:54:13 smithi159 ceph-mon[35513]: pgmap v869: 129 pgs: 129 active+clean; 573 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 232 KiB/s rd, 5.6 MiB/s wr, 205 op/s 2024-09-10T09:54:13.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:54:13 smithi102 ceph-mon[35442]: pgmap v869: 129 pgs: 129 active+clean; 573 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 232 KiB/s rd, 5.6 MiB/s wr, 205 op/s 2024-09-10T09:54:13.964 INFO:tasks.workunit.client.0.smithi005.stdout: 10 39050 20.32 MB/sec warmup 112 sec latency 23726.494 ms 2024-09-10T09:54:13.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:13.831+0000 7f4a68a00640 1 -- 172.21.15.5:0/3701818723 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4a64104010 msgr2=0x7f4a641043f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:13.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:13.831+0000 7f4a68a00640 1 --2- 172.21.15.5:0/3701818723 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4a64104010 0x7f4a641043f0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f4a54009900 tx=0x7f4a5402a3f0 comp rx=0 tx=0).stop 2024-09-10T09:54:13.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:13.831+0000 7f4a68a00640 1 -- 172.21.15.5:0/3701818723 shutdown_connections 2024-09-10T09:54:13.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:13.831+0000 7f4a68a00640 1 --2- 172.21.15.5:0/3701818723 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4a64109140 0x7f4a64110cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:13.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:13.831+0000 7f4a68a00640 1 --2- 172.21.15.5:0/3701818723 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4a641049c0 0x7f4a64108a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:13.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:13.831+0000 7f4a68a00640 1 --2- 172.21.15.5:0/3701818723 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4a64104010 0x7f4a641043f0 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:13.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:13.831+0000 7f4a68a00640 1 -- 172.21.15.5:0/3701818723 >> 172.21.15.5:0/3701818723 conn(0x7f4a640ffa50 msgr2=0x7f4a64101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:54:13.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:13.831+0000 7f4a68a00640 1 -- 172.21.15.5:0/3701818723 shutdown_connections 2024-09-10T09:54:13.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:13.831+0000 7f4a68a00640 1 -- 172.21.15.5:0/3701818723 wait complete. 2024-09-10T09:54:13.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:13.831+0000 7f4a68a00640 1 Processor -- start 2024-09-10T09:54:13.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:13.831+0000 7f4a68a00640 1 -- start start 2024-09-10T09:54:13.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:13.831+0000 7f4a68a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4a64104010 0x7f4a6419a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:13.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:13.831+0000 7f4a68a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4a641049c0 0x7f4a6419ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:13.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:13.831+0000 7f4a68a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4a64109140 0x7f4a6419eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:13.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:13.831+0000 7f4a68a00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f4a64114160 con 0x7f4a64104010 2024-09-10T09:54:13.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:13.831+0000 7f4a68a00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f4a64113fe0 con 0x7f4a64109140 2024-09-10T09:54:13.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:13.831+0000 7f4a68a00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f4a641142e0 con 0x7f4a641049c0 2024-09-10T09:54:13.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:13.831+0000 7f4a61e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4a64104010 0x7f4a6419a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:13.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:13.831+0000 7f4a61e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4a64104010 0x7f4a6419a620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54976/0 (socket says 172.21.15.5:54976) 2024-09-10T09:54:13.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:13.831+0000 7f4a61e00640 1 -- 172.21.15.5:0/1263499746 learned_addr learned my addr 172.21.15.5:0/1263499746 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:54:13.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:13.831+0000 7f4a61400640 1 --2- 172.21.15.5:0/1263499746 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4a641049c0 0x7f4a6419ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:13.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:13.836+0000 7f4a62800640 1 --2- 172.21.15.5:0/1263499746 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4a64109140 0x7f4a6419eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:13.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:13.836+0000 7f4a61400640 1 -- 172.21.15.5:0/1263499746 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4a64109140 msgr2=0x7f4a6419eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:13.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:13.836+0000 7f4a61400640 1 --2- 172.21.15.5:0/1263499746 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4a64109140 0x7f4a6419eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:13.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:13.836+0000 7f4a61400640 1 -- 172.21.15.5:0/1263499746 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4a64104010 msgr2=0x7f4a6419a620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:13.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:13.836+0000 7f4a61400640 1 --2- 172.21.15.5:0/1263499746 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4a64104010 0x7f4a6419a620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:13.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:13.836+0000 7f4a61400640 1 -- 172.21.15.5:0/1263499746 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a6419f5d0 con 0x7f4a641049c0 2024-09-10T09:54:13.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:13.836+0000 7f4a61400640 1 --2- 172.21.15.5:0/1263499746 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4a641049c0 0x7f4a6419ab60 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f4a5000e9f0 tx=0x7f4a5000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:54:13.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:13.836+0000 7f4a5aa00640 1 -- 172.21.15.5:0/1263499746 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a5000ccc0 con 0x7f4a641049c0 2024-09-10T09:54:13.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:13.836+0000 7f4a5aa00640 1 -- 172.21.15.5:0/1263499746 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4a50005020 con 0x7f4a641049c0 2024-09-10T09:54:13.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:13.836+0000 7f4a68a00640 1 -- 172.21.15.5:0/1263499746 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a6419f8c0 con 0x7f4a641049c0 2024-09-10T09:54:13.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:13.836+0000 7f4a5aa00640 1 -- 172.21.15.5:0/1263499746 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a500049e0 con 0x7f4a641049c0 2024-09-10T09:54:13.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:13.836+0000 7f4a68a00640 1 -- 172.21.15.5:0/1263499746 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a641a71a0 con 0x7f4a641049c0 2024-09-10T09:54:13.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:13.837+0000 7f4a6b606740 1 Processor -- start 2024-09-10T09:54:13.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:13.837+0000 7f4a5aa00640 1 -- 172.21.15.5:0/1263499746 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f4a50005560 con 0x7f4a641049c0 2024-09-10T09:54:13.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:13.837+0000 7f4a6b606740 1 -- start start 2024-09-10T09:54:13.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:13.837+0000 7f4a6b606740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5626989ba310 0x5626989ba6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:13.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:13.837+0000 7f4a6b606740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5626989bac30 0x5626989c4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:13.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:13.837+0000 7f4a6b606740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5626989c54f0 0x5626989c78e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:13.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:13.837+0000 7f4a6b606740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x562698920650 con 0x5626989bac30 2024-09-10T09:54:13.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:13.837+0000 7f4a6b606740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x5626988c7f10 con 0x5626989c54f0 2024-09-10T09:54:13.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:13.837+0000 7f4a6b606740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x56269891c610 con 0x5626989ba310 2024-09-10T09:54:13.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:13.837+0000 7f4a62800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5626989ba310 0x5626989ba6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:13.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:13.837+0000 7f4a62800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5626989ba310 0x5626989ba6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:53188/0 (socket says 172.21.15.5:53188) 2024-09-10T09:54:13.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:13.837+0000 7f4a62800640 1 -- 172.21.15.5:0/960478015 learned_addr learned my addr 172.21.15.5:0/960478015 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:54:13.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:13.837+0000 7f4a61400640 1 --2- 172.21.15.5:0/960478015 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5626989c54f0 0x5626989c78e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:13.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:13.837+0000 7f4a61e00640 1 --2- 172.21.15.5:0/960478015 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5626989bac30 0x5626989c4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:13.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:13.837+0000 7f4a62800640 1 -- 172.21.15.5:0/960478015 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5626989c54f0 msgr2=0x5626989c78e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:54:13.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:13.837+0000 7f4a62800640 1 --2- 172.21.15.5:0/960478015 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5626989c54f0 0x5626989c78e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:13.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:13.837+0000 7f4a62800640 1 -- 172.21.15.5:0/960478015 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5626989bac30 msgr2=0x5626989c4fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:54:13.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:13.837+0000 7f4a62800640 1 --2- 172.21.15.5:0/960478015 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5626989bac30 0x5626989c4fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:13.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:13.837+0000 7f4a62800640 1 -- 172.21.15.5:0/960478015 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56269862f580 con 0x5626989ba310 2024-09-10T09:54:13.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:13.837+0000 7f4a61400640 1 --2- 172.21.15.5:0/960478015 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5626989c54f0 0x5626989c78e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:54:13.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:13.837+0000 7f4a61e00640 1 --2- 172.21.15.5:0/960478015 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5626989bac30 0x5626989c4fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:54:13.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:13.843+0000 7f4a62800640 1 --2- 172.21.15.5:0/960478015 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5626989ba310 0x5626989ba6f0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f4a5c009f50 tx=0x7f4a5c0118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7d2e1e07c8b48ef6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:54:13.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:13.843+0000 7f4a4f000640 1 -- 172.21.15.5:0/960478015 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a5c004b20 con 0x5626989ba310 2024-09-10T09:54:13.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:13.843+0000 7f4a4f000640 1 -- 172.21.15.5:0/960478015 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4a5c004cc0 con 0x5626989ba310 2024-09-10T09:54:13.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:13.843+0000 7f4a4f000640 1 -- 172.21.15.5:0/960478015 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a5c005020 con 0x5626989ba310 2024-09-10T09:54:13.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:13.843+0000 7f4a5aa00640 1 --2- 172.21.15.5:0/1263499746 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f4a480a2c00 0x7f4a480a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:13.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:13.843+0000 7f4a5aa00640 1 -- 172.21.15.5:0/1263499746 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f4a500cf300 con 0x7f4a641049c0 2024-09-10T09:54:13.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:13.843+0000 7f4a61e00640 1 --2- 172.21.15.5:0/1263499746 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f4a480a2c00 0x7f4a480a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:13.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:13.843+0000 7f4a6b606740 1 -- 172.21.15.5:0/960478015 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5626989ba310 msgr2=0x5626989ba6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:13.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:13.843+0000 7f4a6b606740 1 --2- 172.21.15.5:0/960478015 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5626989ba310 0x5626989ba6f0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f4a5c009f50 tx=0x7f4a5c0118d0 comp rx=0 tx=0).stop 2024-09-10T09:54:13.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:13.843+0000 7f4a6b606740 1 -- 172.21.15.5:0/960478015 shutdown_connections 2024-09-10T09:54:13.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:13.843+0000 7f4a6b606740 1 --2- 172.21.15.5:0/960478015 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5626989c54f0 0x5626989c78e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:13.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:13.843+0000 7f4a6b606740 1 --2- 172.21.15.5:0/960478015 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5626989bac30 0x5626989c4fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:13.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:13.843+0000 7f4a6b606740 1 --2- 172.21.15.5:0/960478015 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5626989ba310 0x5626989ba6f0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:13.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:13.843+0000 7f4a6b606740 1 -- 172.21.15.5:0/960478015 >> 172.21.15.5:0/960478015 conn(0x5626989b68a0 msgr2=0x5626989b6c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:54:13.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:13.843+0000 7f4a6b606740 1 -- 172.21.15.5:0/960478015 shutdown_connections 2024-09-10T09:54:13.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:13.843+0000 7f4a6b606740 1 -- 172.21.15.5:0/960478015 wait complete. 2024-09-10T09:54:13.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:13.843+0000 7f4a61e00640 1 --2- 172.21.15.5:0/1263499746 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f4a480a2c00 0x7f4a480a50c0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x5626989b7700 tx=0x7f4a54005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:54:13.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:13.843+0000 7f4a6b606740 1 -- start start 2024-09-10T09:54:13.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:13.843+0000 7f4a6b606740 1 Processor -- start 2024-09-10T09:54:13.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:13.843+0000 7f4a6b606740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5626989ba310 0x5626989c23c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:13.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:13.843+0000 7f4a6b606740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5626989bac30 0x5626989c2900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:13.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:13.843+0000 7f4a6b606740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5626989c54f0 0x5626989c2e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:13.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:13.843+0000 7f4a6b606740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5626989c80c0 con 0x5626989c54f0 2024-09-10T09:54:13.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:13.843+0000 7f4a6b606740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x5626988db350 con 0x5626989bac30 2024-09-10T09:54:13.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:13.843+0000 7f4a6b606740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5626989c8240 con 0x5626989ba310 2024-09-10T09:54:13.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:13.843+0000 7f4a6b606740 10 client.0 _ll_register_callbacks cb 0x5626988fe290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:54:13.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:13.843+0000 7f4a61e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5626989ba310 0x5626989c23c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:13.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:13.843+0000 7f4a61e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5626989ba310 0x5626989c23c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:53204/0 (socket says 172.21.15.5:53204) 2024-09-10T09:54:13.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:13.843+0000 7f4a61e00640 1 -- 172.21.15.5:0/2110628197 learned_addr learned my addr 172.21.15.5:0/2110628197 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:54:13.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:13.843+0000 7f4a61400640 1 --2- 172.21.15.5:0/2110628197 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5626989bac30 0x5626989c2900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:13.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:13.843+0000 7f4a62800640 1 --2- 172.21.15.5:0/2110628197 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5626989c54f0 0x5626989c2e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:13.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:13.843+0000 7f4a61400640 1 -- 172.21.15.5:0/2110628197 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5626989ba310 msgr2=0x5626989c23c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:54:13.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:13.843+0000 7f4a61400640 1 --2- 172.21.15.5:0/2110628197 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5626989ba310 0x5626989c23c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:13.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:13.843+0000 7f4a61400640 1 -- 172.21.15.5:0/2110628197 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5626989c54f0 msgr2=0x5626989c2e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:54:13.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:13.843+0000 7f4a61400640 1 --2- 172.21.15.5:0/2110628197 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5626989c54f0 0x5626989c2e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:13.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:13.843+0000 7f4a61400640 1 -- 172.21.15.5:0/2110628197 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5626986f9960 con 0x5626989bac30 2024-09-10T09:54:13.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:13.843+0000 7f4a61e00640 1 --2- 172.21.15.5:0/2110628197 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5626989ba310 0x5626989c23c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:54:13.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:13.843+0000 7f4a61400640 1 --2- 172.21.15.5:0/2110628197 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5626989bac30 0x5626989c2900 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f4a500cca60 tx=0x7f4a5009b630 comp rx=0 tx=0).ready entity=mon.2 client_cookie=15272a75a80cbdd0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:54:13.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:13.843+0000 7f4a43400640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-09-10T09:54:13.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:13.843+0000 7f4a43400640 1 -- 172.21.15.5:0/2110628197 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a5001c070 con 0x5626989bac30 2024-09-10T09:54:13.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:13.843+0000 7f4a6b606740 1 -- 172.21.15.5:0/2110628197 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4a5c004cc0 con 0x5626989bac30 2024-09-10T09:54:13.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:13.843+0000 7f4a43400640 1 -- 172.21.15.5:0/2110628197 <== mon.2 v2:172.21.15.102:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f4a50097de0 con 0x5626989bac30 2024-09-10T09:54:13.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:13.843+0000 7f4a43400640 1 -- 172.21.15.5:0/2110628197 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f4a500cf160 con 0x5626989bac30 2024-09-10T09:54:13.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:13.843+0000 7f4a6b606740 10 client.28520 fetch_fsmap learned FSMap version 37 2024-09-10T09:54:13.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:13.843+0000 7f4a6b606740 1 -- 172.21.15.5:0/2110628197 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x5626989d3d10 con 0x5626989bac30 2024-09-10T09:54:13.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:13.843+0000 7f4a43400640 1 -- 172.21.15.5:0/2110628197 <== mon.2 v2:172.21.15.102:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4a5c004cc0 con 0x5626989bac30 2024-09-10T09:54:13.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:13.848+0000 7f4a43400640 1 -- 172.21.15.5:0/2110628197 <== mon.2 v2:172.21.15.102:3300/0 5 ==== fsmap(e 37) ==== 8390+0+0 (secure 0 0 0) 0x7f4a500a3f50 con 0x5626989bac30 2024-09-10T09:54:13.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:13.848+0000 7f4a6b606740 10 client.28520 fetch_fsmap finished waiting for FSMap version 37 2024-09-10T09:54:13.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:13.848+0000 7f4a6b606740 10 client.28520 resolve_mds: validated gid 24542 aka daemon mds.b 2024-09-10T09:54:13.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:13.848+0000 7f4a6b606740 20 client.28520 populate_metadata read hostname 'smithi005' 2024-09-10T09:54:13.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:13.848+0000 7f4a6b606740 1 --2- 172.21.15.5:0/2110628197 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x5626989d5120 0x5626989d7510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:13.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:13.848+0000 7f4a6b606740 4 client.28520 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:54:13.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:13.848+0000 7f4a6b606740 1 -- 172.21.15.5:0/2110628197 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56269892dbc0 con 0x5626989d5120 2024-09-10T09:54:13.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:13.848+0000 7f4a61e00640 1 --2- 172.21.15.5:0/2110628197 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x5626989d5120 0x5626989d7510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:13.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:13.848+0000 7f4a61e00640 1 --2- 172.21.15.5:0/2110628197 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x5626989d5120 0x5626989d7510 crc :-1 s=READY pgs=231 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dfedd8ce70b5cd60 server_cookie=174cba4cff839acf in_seq=0 out_seq=0 2024-09-10T09:54:13.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:13.848+0000 7f4a43400640 10 client.28520 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T09:54:13.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:13.849+0000 7f4a43400640 1 -- 172.21.15.5:0/2110628197 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56269892dbc0 con 0x5626989d5120 2024-09-10T09:54:13.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:13.849+0000 7f4a43400640 10 client.28520 handle_command_reply: tid=0 2024-09-10T09:54:13.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:13.849+0000 7f4a6b606740 1 client.28520 shutdown 2024-09-10T09:54:13.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:13.849+0000 7f4a6b606740 1 -- 172.21.15.5:0/2110628197 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5626989bac30 msgr2=0x5626989c2900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:13.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:13.849+0000 7f4a6b606740 1 --2- 172.21.15.5:0/2110628197 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5626989bac30 0x5626989c2900 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f4a500cca60 tx=0x7f4a5009b630 comp rx=0 tx=0).stop 2024-09-10T09:54:13.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:13.849+0000 7f4a6b606740 1 -- 172.21.15.5:0/2110628197 shutdown_connections 2024-09-10T09:54:13.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:13.849+0000 7f4a6b606740 1 --2- 172.21.15.5:0/2110628197 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x5626989d5120 0x5626989d7510 crc :-1 s=READY pgs=231 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:13.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:13.849+0000 7f4a6b606740 1 --2- 172.21.15.5:0/2110628197 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5626989c54f0 0x5626989c2e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:13.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:13.849+0000 7f4a6b606740 1 --2- 172.21.15.5:0/2110628197 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5626989bac30 0x5626989c2900 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:13.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:13.849+0000 7f4a6b606740 1 --2- 172.21.15.5:0/2110628197 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5626989ba310 0x5626989c23c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:13.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:13.849+0000 7f4a43400640 0 client.28520 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T09:54:13.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:13.849+0000 7f4a6b606740 1 -- 172.21.15.5:0/2110628197 >> 172.21.15.5:0/2110628197 conn(0x5626989b68a0 msgr2=0x5626989b5aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:54:13.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:13.849+0000 7f4a6b606740 1 -- 172.21.15.5:0/2110628197 shutdown_connections 2024-09-10T09:54:13.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:13.849+0000 7f4a6b606740 1 -- 172.21.15.5:0/2110628197 wait complete. 2024-09-10T09:54:13.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:13.849+0000 7f4a6b606740 20 client.28520 trim_cache size 0 max 16384 2024-09-10T09:54:13.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:13.867+0000 7f4a6b606740 1 Processor -- start 2024-09-10T09:54:13.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:13.867+0000 7f4a6b606740 1 -- start start 2024-09-10T09:54:13.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:13.867+0000 7f4a6b606740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5626989ba310 0x5626989bcd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:13.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:13.867+0000 7f4a6b606740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5626989bac30 0x5626989c4330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:13.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:13.867+0000 7f4a6b606740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5626989c54f0 0x5626989dcd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:13.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:13.867+0000 7f4a6b606740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f4a641b10e0 con 0x5626989ba310 2024-09-10T09:54:13.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:13.867+0000 7f4a6b606740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f4a641b2d10 con 0x5626989bac30 2024-09-10T09:54:13.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:13.867+0000 7f4a6b606740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f4a641b17b0 con 0x5626989c54f0 2024-09-10T09:54:13.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:13.867+0000 7f4a61e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5626989ba310 0x5626989bcd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:13.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:13.867+0000 7f4a61400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5626989bac30 0x5626989c4330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:13.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:13.867+0000 7f4a62800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5626989c54f0 0x5626989dcd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:13.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:13.867+0000 7f4a61400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5626989bac30 0x5626989c4330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:58528/0 (socket says 172.21.15.5:58528) 2024-09-10T09:54:13.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:13.867+0000 7f4a61400640 1 -- 172.21.15.5:0/680529886 learned_addr learned my addr 172.21.15.5:0/680529886 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:54:13.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:13.867+0000 7f4a61400640 1 -- 172.21.15.5:0/680529886 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5626989c54f0 msgr2=0x5626989dcd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:54:13.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:13.867+0000 7f4a61400640 1 --2- 172.21.15.5:0/680529886 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5626989c54f0 0x5626989dcd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:13.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:13.867+0000 7f4a61400640 1 -- 172.21.15.5:0/680529886 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5626989ba310 msgr2=0x5626989bcd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:54:13.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:13.867+0000 7f4a61400640 1 --2- 172.21.15.5:0/680529886 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5626989ba310 0x5626989bcd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:13.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:13.867+0000 7f4a61400640 1 -- 172.21.15.5:0/680529886 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5626989b7950 con 0x5626989bac30 2024-09-10T09:54:13.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:13.867+0000 7f4a61400640 1 --2- 172.21.15.5:0/680529886 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5626989bac30 0x5626989c4330 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f4a50005480 tx=0x7f4a500a4310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c275b77272ea384 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:54:13.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:13.867+0000 7f4a4e600640 1 -- 172.21.15.5:0/680529886 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a5001c070 con 0x5626989bac30 2024-09-10T09:54:13.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:13.867+0000 7f4a4e600640 1 -- 172.21.15.5:0/680529886 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4a5009be70 con 0x5626989bac30 2024-09-10T09:54:13.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:13.867+0000 7f4a4e600640 1 -- 172.21.15.5:0/680529886 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a500a3f50 con 0x5626989bac30 2024-09-10T09:54:13.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:13.867+0000 7f4a6b606740 1 -- 172.21.15.5:0/680529886 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5626989bac30 msgr2=0x5626989c4330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:13.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:13.867+0000 7f4a6b606740 1 --2- 172.21.15.5:0/680529886 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5626989bac30 0x5626989c4330 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f4a50005480 tx=0x7f4a500a4310 comp rx=0 tx=0).stop 2024-09-10T09:54:13.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:13.867+0000 7f4a6b606740 1 -- 172.21.15.5:0/680529886 shutdown_connections 2024-09-10T09:54:13.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:13.867+0000 7f4a6b606740 1 --2- 172.21.15.5:0/680529886 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5626989c54f0 0x5626989dcd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:13.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:13.867+0000 7f4a6b606740 1 --2- 172.21.15.5:0/680529886 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5626989bac30 0x5626989c4330 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:13.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:13.867+0000 7f4a6b606740 1 --2- 172.21.15.5:0/680529886 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5626989ba310 0x5626989bcd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:13.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:13.867+0000 7f4a6b606740 1 -- 172.21.15.5:0/680529886 >> 172.21.15.5:0/680529886 conn(0x5626989b68a0 msgr2=0x5626989c1530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:54:13.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:13.867+0000 7f4a6b606740 1 -- 172.21.15.5:0/680529886 shutdown_connections 2024-09-10T09:54:13.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:13.867+0000 7f4a6b606740 1 -- 172.21.15.5:0/680529886 wait complete. 2024-09-10T09:54:13.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:13.867+0000 7f4a6b606740 1 -- start start 2024-09-10T09:54:13.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:13.867+0000 7f4a6b606740 1 Processor -- start 2024-09-10T09:54:13.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:13.867+0000 7f4a6b606740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5626989ba310 0x5626989cdb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:13.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:13.867+0000 7f4a6b606740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5626989bac30 0x5626989bf050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:13.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:13.867+0000 7f4a6b606740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5626989c54f0 0x5626989bf590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:13.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:13.867+0000 7f4a6b606740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f4a641b2610 con 0x5626989bac30 2024-09-10T09:54:13.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:13.867+0000 7f4a6b606740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f4a641b1f40 con 0x5626989c54f0 2024-09-10T09:54:13.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:13.867+0000 7f4a6b606740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f4a641af0c0 con 0x5626989ba310 2024-09-10T09:54:13.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:13.867+0000 7f4a6b606740 10 client.0 _ll_register_callbacks cb 0x7f4a500053d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:54:13.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:13.872+0000 7f4a61400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5626989bac30 0x5626989bf050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:13.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:13.872+0000 7f4a61e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5626989ba310 0x5626989cdb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:13.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:13.872+0000 7f4a61400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5626989bac30 0x5626989bf050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55022/0 (socket says 172.21.15.5:55022) 2024-09-10T09:54:13.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:13.872+0000 7f4a61400640 1 -- 172.21.15.5:0/2175328096 learned_addr learned my addr 172.21.15.5:0/2175328096 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:54:13.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:13.872+0000 7f4a62800640 1 --2- 172.21.15.5:0/2175328096 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5626989c54f0 0x5626989bf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:13.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:13.872+0000 7f4a61e00640 1 -- 172.21.15.5:0/2175328096 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5626989c54f0 msgr2=0x5626989bf590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:54:13.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:13.872+0000 7f4a61e00640 1 --2- 172.21.15.5:0/2175328096 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5626989c54f0 0x5626989bf590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:13.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:13.872+0000 7f4a61e00640 1 -- 172.21.15.5:0/2175328096 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5626989bac30 msgr2=0x5626989bf050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:54:13.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:13.872+0000 7f4a61e00640 1 --2- 172.21.15.5:0/2175328096 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5626989bac30 0x5626989bf050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:13.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:13.872+0000 7f4a61e00640 1 -- 172.21.15.5:0/2175328096 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5626989bfdc0 con 0x5626989ba310 2024-09-10T09:54:13.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:13.872+0000 7f4a62800640 1 --2- 172.21.15.5:0/2175328096 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5626989c54f0 0x5626989bf590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:54:13.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:13.872+0000 7f4a61400640 1 --2- 172.21.15.5:0/2175328096 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5626989bac30 0x5626989bf050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:54:13.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:13.872+0000 7f4a61e00640 1 --2- 172.21.15.5:0/2175328096 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5626989ba310 0x5626989cdb40 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x5626989d7cd0 tx=0x7f4a54009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bbee51bc46b9e3f3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:54:13.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:13.872+0000 7f4a43400640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-09-10T09:54:13.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:13.872+0000 7f4a43400640 1 -- 172.21.15.5:0/2175328096 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a54035040 con 0x5626989ba310 2024-09-10T09:54:13.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:13.872+0000 7f4a6b606740 1 -- 172.21.15.5:0/2175328096 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4a5009be70 con 0x5626989ba310 2024-09-10T09:54:13.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:13.872+0000 7f4a43400640 1 -- 172.21.15.5:0/2175328096 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f4a54004450 con 0x5626989ba310 2024-09-10T09:54:13.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:13.873+0000 7f4a43400640 1 -- 172.21.15.5:0/2175328096 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f4a54005040 con 0x5626989ba310 2024-09-10T09:54:13.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:13.873+0000 7f4a6b606740 10 client.28525 fetch_fsmap learned FSMap version 37 2024-09-10T09:54:13.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:13.873+0000 7f4a6b606740 1 -- 172.21.15.5:0/2175328096 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x562698a3fde0 con 0x5626989ba310 2024-09-10T09:54:13.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:13.873+0000 7f4a43400640 1 -- 172.21.15.5:0/2175328096 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4a5009be70 con 0x5626989ba310 2024-09-10T09:54:13.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:13.873+0000 7f4a43400640 1 -- 172.21.15.5:0/2175328096 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 37) ==== 8390+0+0 (secure 0 0 0) 0x7f4a540035f0 con 0x5626989ba310 2024-09-10T09:54:13.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:13.873+0000 7f4a6b606740 10 client.28525 fetch_fsmap finished waiting for FSMap version 37 2024-09-10T09:54:13.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:13.873+0000 7f4a6b606740 10 client.28525 resolve_mds: validated gid 24542 aka daemon mds.b 2024-09-10T09:54:13.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:13.873+0000 7f4a6b606740 20 client.28525 populate_metadata read hostname 'smithi005' 2024-09-10T09:54:13.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:13.873+0000 7f4a6b606740 1 --2- 172.21.15.5:0/2175328096 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x562698a404e0 0x562698a42900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:13.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:13.873+0000 7f4a6b606740 4 client.28525 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "6cbb0d55", "release": true, "await": true}] 2024-09-10T09:54:13.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:13.873+0000 7f4a6b606740 1 -- 172.21.15.5:0/2175328096 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "quiesce db", "set_id": "6cbb0d55", "release": true, "await": true}) -- 0x5626989280d0 con 0x562698a404e0 2024-09-10T09:54:13.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:13.873+0000 7f4a61400640 1 --2- 172.21.15.5:0/2175328096 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x562698a404e0 0x562698a42900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:13.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:13.873+0000 7f4a61400640 1 --2- 172.21.15.5:0/2175328096 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x562698a404e0 0x562698a42900 crc :-1 s=READY pgs=232 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7c197af8e94cbe84 server_cookie=dc37788e0abcf56c in_seq=0 out_seq=0 2024-09-10T09:54:13.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:13.873+0000 7f4a43400640 10 client.28525 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T09:54:13.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:13.933+0000 7f4a43400640 1 -- 172.21.15.5:0/2175328096 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5626989280d0 con 0x562698a404e0 2024-09-10T09:54:13.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:13.933+0000 7f4a43400640 10 client.28525 handle_command_reply: tid=0 2024-09-10T09:54:13.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:13.933+0000 7f4a6b606740 1 client.28525 shutdown 2024-09-10T09:54:13.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:13.933+0000 7f4a6b606740 1 -- 172.21.15.5:0/2175328096 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5626989ba310 msgr2=0x5626989cdb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:13.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:13.933+0000 7f4a6b606740 1 --2- 172.21.15.5:0/2175328096 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5626989ba310 0x5626989cdb40 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x5626989d7cd0 tx=0x7f4a54009260 comp rx=0 tx=0).stop 2024-09-10T09:54:13.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:13.933+0000 7f4a6b606740 1 -- 172.21.15.5:0/2175328096 shutdown_connections 2024-09-10T09:54:13.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:13.933+0000 7f4a6b606740 1 --2- 172.21.15.5:0/2175328096 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x562698a404e0 0x562698a42900 crc :-1 s=READY pgs=232 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:13.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:13.933+0000 7f4a6b606740 1 --2- 172.21.15.5:0/2175328096 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5626989c54f0 0x5626989bf590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:13.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:13.933+0000 7f4a6b606740 1 --2- 172.21.15.5:0/2175328096 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5626989bac30 0x5626989bf050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:13.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:13.933+0000 7f4a6b606740 1 --2- 172.21.15.5:0/2175328096 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5626989ba310 0x5626989cdb40 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:13.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:13.933+0000 7f4a43400640 0 client.28525 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T09:54:13.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:13.933+0000 7f4a6b606740 1 -- 172.21.15.5:0/2175328096 >> 172.21.15.5:0/2175328096 conn(0x5626989b68a0 msgr2=0x5626989b5aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:54:13.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:13.933+0000 7f4a6b606740 1 -- 172.21.15.5:0/2175328096 shutdown_connections 2024-09-10T09:54:13.989 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-10T09:54:13.989 INFO:teuthology.orchestra.run.smithi005.stdout: "epoch": 37, 2024-09-10T09:54:13.989 INFO:teuthology.orchestra.run.smithi005.stdout: "leader": 24542, 2024-09-10T09:54:13.989 INFO:teuthology.orchestra.run.smithi005.stdout: "set_version": 56, 2024-09-10T09:54:13.990 INFO:teuthology.orchestra.run.smithi005.stdout: "sets": { 2024-09-10T09:54:13.990 INFO:teuthology.orchestra.run.smithi005.stdout: "6cbb0d55": { 2024-09-10T09:54:13.990 INFO:teuthology.orchestra.run.smithi005.stdout: "version": 56, 2024-09-10T09:54:13.990 INFO:teuthology.orchestra.run.smithi005.stdout: "age_ref": 0.0, 2024-09-10T09:54:13.990 INFO:teuthology.orchestra.run.smithi005.stdout: "state": { 2024-09-10T09:54:13.990 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "RELEASED", 2024-09-10T09:54:13.990 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0 2024-09-10T09:54:13.990 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T09:54:13.990 INFO:teuthology.orchestra.run.smithi005.stdout: "timeout": 90.0, 2024-09-10T09:54:13.990 INFO:teuthology.orchestra.run.smithi005.stdout: "expiration": 139.8, 2024-09-10T09:54:13.990 INFO:teuthology.orchestra.run.smithi005.stdout: "members": { 2024-09-10T09:54:13.990 INFO:teuthology.orchestra.run.smithi005.stdout: "file:/": { 2024-09-10T09:54:13.990 INFO:teuthology.orchestra.run.smithi005.stdout: "excluded": false, 2024-09-10T09:54:13.990 INFO:teuthology.orchestra.run.smithi005.stdout: "state": { 2024-09-10T09:54:13.990 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "RELEASED", 2024-09-10T09:54:13.991 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0 2024-09-10T09:54:13.991 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T09:54:13.991 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T09:54:13.991 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T09:54:13.991 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T09:54:13.991 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T09:54:13.991 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-10T09:54:13.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:13.933+0000 7f4a6b606740 1 -- 172.21.15.5:0/2175328096 wait complete. 2024-09-10T09:54:13.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:13.933+0000 7f4a6b606740 20 client.28525 trim_cache size 0 max 16384 2024-09-10T09:54:13.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:13.933+0000 7f4a68a00640 1 -- 172.21.15.5:0/1263499746 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f4a480a2c00 msgr2=0x7f4a480a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:13.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:13.933+0000 7f4a68a00640 1 --2- 172.21.15.5:0/1263499746 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f4a480a2c00 0x7f4a480a50c0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x5626989b7700 tx=0x7f4a54005a20 comp rx=0 tx=0).stop 2024-09-10T09:54:13.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:13.933+0000 7f4a68a00640 1 -- 172.21.15.5:0/1263499746 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4a641049c0 msgr2=0x7f4a6419ab60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:13.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:13.933+0000 7f4a68a00640 1 --2- 172.21.15.5:0/1263499746 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4a641049c0 0x7f4a6419ab60 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f4a5000e9f0 tx=0x7f4a5000eeb0 comp rx=0 tx=0).stop 2024-09-10T09:54:13.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:13.933+0000 7f4a68a00640 1 -- 172.21.15.5:0/1263499746 shutdown_connections 2024-09-10T09:54:13.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:13.933+0000 7f4a68a00640 1 --2- 172.21.15.5:0/1263499746 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f4a480a2c00 0x7f4a480a50c0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:13.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:13.933+0000 7f4a68a00640 1 --2- 172.21.15.5:0/1263499746 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4a64109140 0x7f4a6419eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:13.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:13.933+0000 7f4a68a00640 1 --2- 172.21.15.5:0/1263499746 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4a641049c0 0x7f4a6419ab60 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:13.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:13.933+0000 7f4a68a00640 1 --2- 172.21.15.5:0/1263499746 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4a64104010 0x7f4a6419a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:13.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:13.933+0000 7f4a68a00640 1 -- 172.21.15.5:0/1263499746 >> 172.21.15.5:0/1263499746 conn(0x7f4a640ffa50 msgr2=0x7f4a64100e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:54:13.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:13.933+0000 7f4a68a00640 1 -- 172.21.15.5:0/1263499746 shutdown_connections 2024-09-10T09:54:13.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:13.933+0000 7f4a68a00640 1 -- 172.21.15.5:0/1263499746 wait complete. 2024-09-10T09:54:14.965 INFO:tasks.workunit.client.0.smithi005.stdout: 10 39050 20.15 MB/sec warmup 113 sec latency 24726.536 ms 2024-09-10T09:54:15.728 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:54:15 smithi005 ceph-mon[30350]: pgmap v870: 129 pgs: 129 active+clean; 568 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 213 KiB/s rd, 5.3 MiB/s wr, 201 op/s 2024-09-10T09:54:15.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:54:15 smithi159 ceph-mon[35513]: pgmap v870: 129 pgs: 129 active+clean; 568 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 213 KiB/s rd, 5.3 MiB/s wr, 201 op/s 2024-09-10T09:54:15.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:54:15 smithi102 ceph-mon[35442]: pgmap v870: 129 pgs: 129 active+clean; 568 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 213 KiB/s rd, 5.3 MiB/s wr, 201 op/s 2024-09-10T09:54:15.965 INFO:tasks.workunit.client.0.smithi005.stdout: 10 39050 19.97 MB/sec warmup 114 sec latency 25726.578 ms 2024-09-10T09:54:16.965 INFO:tasks.workunit.client.0.smithi005.stdout: 10 39050 19.79 MB/sec warmup 115 sec latency 26726.620 ms 2024-09-10T09:54:17.070 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:54:17.106 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '6cbb0d55', total seconds elapsed: 34.6 2024-09-10T09:54:17.106 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 19.85 2024-09-10T09:54:17.107 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T09:54:17.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:54:17 smithi005 ceph-mon[30350]: pgmap v871: 129 pgs: 129 active+clean; 564 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 270 KiB/s rd, 5.1 MiB/s wr, 207 op/s 2024-09-10T09:54:17.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:54:17 smithi159 ceph-mon[35513]: pgmap v871: 129 pgs: 129 active+clean; 564 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 270 KiB/s rd, 5.1 MiB/s wr, 207 op/s 2024-09-10T09:54:17.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:54:17 smithi102 ceph-mon[35442]: pgmap v871: 129 pgs: 129 active+clean; 564 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 270 KiB/s rd, 5.1 MiB/s wr, 207 op/s 2024-09-10T09:54:17.965 INFO:tasks.workunit.client.0.smithi005.stdout: 10 39050 19.62 MB/sec warmup 116 sec latency 27726.661 ms 2024-09-10T09:54:18.791 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:54:18 smithi005 ceph-mon[30350]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T09:54:18.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:54:18 smithi159 ceph-mon[35513]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T09:54:18.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:54:18 smithi102 ceph-mon[35442]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T09:54:18.965 INFO:tasks.workunit.client.0.smithi005.stdout: 10 39050 19.46 MB/sec warmup 117 sec latency 28726.718 ms 2024-09-10T09:54:19.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:18.936+0000 7f946ce00640 1 -- 172.21.15.5:0/2854493203 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9468074510 msgr2=0x7f94680748f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:19.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:18.936+0000 7f946ce00640 1 --2- 172.21.15.5:0/2854493203 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9468074510 0x7f94680748f0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f9458009900 tx=0x7f945802a480 comp rx=0 tx=0).stop 2024-09-10T09:54:19.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:18.937+0000 7f946ce00640 1 -- 172.21.15.5:0/2854493203 shutdown_connections 2024-09-10T09:54:19.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:18.937+0000 7f946ce00640 1 --2- 172.21.15.5:0/2854493203 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f946810e450 0x7f9468110840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:19.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:18.937+0000 7f946ce00640 1 --2- 172.21.15.5:0/2854493203 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9468074e30 0x7f946810df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:19.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:18.937+0000 7f946ce00640 1 --2- 172.21.15.5:0/2854493203 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9468074510 0x7f94680748f0 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:19.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:18.937+0000 7f946ce00640 1 -- 172.21.15.5:0/2854493203 >> 172.21.15.5:0/2854493203 conn(0x7f94680fd8b0 msgr2=0x7f94680ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:54:19.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:18.937+0000 7f946ce00640 1 -- 172.21.15.5:0/2854493203 shutdown_connections 2024-09-10T09:54:19.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:18.937+0000 7f946ce00640 1 -- 172.21.15.5:0/2854493203 wait complete. 2024-09-10T09:54:19.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:18.937+0000 7f946ce00640 1 Processor -- start 2024-09-10T09:54:19.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:18.937+0000 7f946ce00640 1 -- start start 2024-09-10T09:54:19.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:18.937+0000 7f946ce00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9468074510 0x7f946819a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:19.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:18.937+0000 7f946ce00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9468074e30 0x7f946819ac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:19.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:18.937+0000 7f946ce00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f946810e450 0x7f946819efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:19.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:18.937+0000 7f946ce00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f9468114160 con 0x7f946810e450 2024-09-10T09:54:19.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:18.937+0000 7f946ce00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f9468113fe0 con 0x7f9468074510 2024-09-10T09:54:19.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:18.937+0000 7f946ce00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f94681142e0 con 0x7f9468074e30 2024-09-10T09:54:19.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:18.937+0000 7f9466800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f946810e450 0x7f946819efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:19.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:18.937+0000 7f9465e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9468074510 0x7f946819a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:19.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:18.937+0000 7f9466800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f946810e450 0x7f946819efb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55040/0 (socket says 172.21.15.5:55040) 2024-09-10T09:54:19.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:18.937+0000 7f9466800640 1 -- 172.21.15.5:0/3394994491 learned_addr learned my addr 172.21.15.5:0/3394994491 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:54:19.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:18.937+0000 7f945d400640 1 --2- 172.21.15.5:0/3394994491 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9468074e30 0x7f946819ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:19.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:18.937+0000 7f9466800640 1 -- 172.21.15.5:0/3394994491 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9468074e30 msgr2=0x7f946819ac20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:19.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:18.937+0000 7f9466800640 1 --2- 172.21.15.5:0/3394994491 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9468074e30 0x7f946819ac20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:19.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:18.937+0000 7f9466800640 1 -- 172.21.15.5:0/3394994491 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9468074510 msgr2=0x7f946819a6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:19.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:18.937+0000 7f9466800640 1 --2- 172.21.15.5:0/3394994491 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9468074510 0x7f946819a6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:19.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:18.937+0000 7f9466800640 1 -- 172.21.15.5:0/3394994491 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f946819f690 con 0x7f946810e450 2024-09-10T09:54:19.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:18.937+0000 7f945d400640 1 --2- 172.21.15.5:0/3394994491 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9468074e30 0x7f946819ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:54:19.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:18.937+0000 7f9466800640 1 --2- 172.21.15.5:0/3394994491 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f946810e450 0x7f946819efb0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f9460009c90 tx=0x7f946000c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:54:19.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:18.937+0000 7f945f400640 1 -- 172.21.15.5:0/3394994491 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9460010630 con 0x7f946810e450 2024-09-10T09:54:19.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:18.937+0000 7f945f400640 1 -- 172.21.15.5:0/3394994491 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9460010cb0 con 0x7f946810e450 2024-09-10T09:54:19.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:18.937+0000 7f945f400640 1 -- 172.21.15.5:0/3394994491 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9460017d20 con 0x7f946810e450 2024-09-10T09:54:19.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:18.937+0000 7f946ce00640 1 -- 172.21.15.5:0/3394994491 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f946819f980 con 0x7f946810e450 2024-09-10T09:54:19.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:18.937+0000 7f946ce00640 1 -- 172.21.15.5:0/3394994491 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94681a7260 con 0x7f946810e450 2024-09-10T09:54:19.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:18.943+0000 7f946f823740 1 Processor -- start 2024-09-10T09:54:19.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:18.944+0000 7f945f400640 1 -- 172.21.15.5:0/3394994491 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f94600107d0 con 0x7f946810e450 2024-09-10T09:54:19.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:18.944+0000 7f946f823740 1 -- start start 2024-09-10T09:54:19.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:18.944+0000 7f946f823740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55de3b0bcc70 0x55de3b0bd050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:19.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:18.944+0000 7f946f823740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55de3b0bd590 0x55de3b0c7910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:19.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:18.944+0000 7f946f823740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55de3b0c7e50 0x55de3b0ca240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:19.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:18.944+0000 7f946f823740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55de3b0233e0 con 0x55de3b0c7e50 2024-09-10T09:54:19.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:18.944+0000 7f946f823740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55de3afcad80 con 0x55de3b0bd590 2024-09-10T09:54:19.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:18.944+0000 7f946f823740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55de3b01f3a0 con 0x55de3b0bcc70 2024-09-10T09:54:19.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:18.944+0000 7f945d400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55de3b0bcc70 0x55de3b0bd050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:19.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:18.944+0000 7f945d400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55de3b0bcc70 0x55de3b0bd050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:53246/0 (socket says 172.21.15.5:53246) 2024-09-10T09:54:19.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:18.944+0000 7f945d400640 1 -- 172.21.15.5:0/1023957137 learned_addr learned my addr 172.21.15.5:0/1023957137 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:54:19.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:18.944+0000 7f9465e00640 1 --2- 172.21.15.5:0/1023957137 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55de3b0bd590 0x55de3b0c7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:19.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:18.944+0000 7f945d400640 1 --2- 172.21.15.5:0/1023957137 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55de3b0c7e50 0x55de3b0ca240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:19.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:18.944+0000 7f9465e00640 1 -- 172.21.15.5:0/1023957137 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55de3b0bcc70 msgr2=0x55de3b0bd050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:54:19.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:18.944+0000 7f9465e00640 1 --2- 172.21.15.5:0/1023957137 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55de3b0bcc70 0x55de3b0bd050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:19.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:18.944+0000 7f9465e00640 1 -- 172.21.15.5:0/1023957137 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55de3b0c7e50 msgr2=0x55de3b0ca240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:54:19.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:18.944+0000 7f9465e00640 1 --2- 172.21.15.5:0/1023957137 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55de3b0c7e50 0x55de3b0ca240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:19.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:18.944+0000 7f9465e00640 1 -- 172.21.15.5:0/1023957137 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55de3afe9ae0 con 0x55de3b0bd590 2024-09-10T09:54:19.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:18.944+0000 7f9465e00640 1 --2- 172.21.15.5:0/1023957137 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55de3b0bd590 0x55de3b0c7910 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f9458005940 tx=0x7f9458005e00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cce4731220eedbdc server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:54:19.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:18.944+0000 7f9457400640 1 -- 172.21.15.5:0/1023957137 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9458040070 con 0x55de3b0bd590 2024-09-10T09:54:19.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:18.944+0000 7f9457400640 1 -- 172.21.15.5:0/1023957137 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f94580318f0 con 0x55de3b0bd590 2024-09-10T09:54:19.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:18.944+0000 7f9457400640 1 -- 172.21.15.5:0/1023957137 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9458035040 con 0x55de3b0bd590 2024-09-10T09:54:19.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:18.944+0000 7f945f400640 1 --2- 172.21.15.5:0/3394994491 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f944c0a2c00 0x7f944c0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:19.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:18.944+0000 7f945f400640 1 -- 172.21.15.5:0/3394994491 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f94600cfc10 con 0x7f946810e450 2024-09-10T09:54:19.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:18.944+0000 7f945d400640 1 --2- 172.21.15.5:0/3394994491 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f944c0a2c00 0x7f944c0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:19.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:18.944+0000 7f946f823740 1 -- 172.21.15.5:0/1023957137 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55de3b0bd590 msgr2=0x55de3b0c7910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:19.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:18.944+0000 7f946f823740 1 --2- 172.21.15.5:0/1023957137 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55de3b0bd590 0x55de3b0c7910 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f9458005940 tx=0x7f9458005e00 comp rx=0 tx=0).stop 2024-09-10T09:54:19.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:18.944+0000 7f946f823740 1 -- 172.21.15.5:0/1023957137 shutdown_connections 2024-09-10T09:54:19.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:18.944+0000 7f946f823740 1 --2- 172.21.15.5:0/1023957137 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55de3b0c7e50 0x55de3b0ca240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:19.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:18.944+0000 7f946f823740 1 --2- 172.21.15.5:0/1023957137 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55de3b0bd590 0x55de3b0c7910 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:19.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:18.944+0000 7f946f823740 1 --2- 172.21.15.5:0/1023957137 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55de3b0bcc70 0x55de3b0bd050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:19.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:18.944+0000 7f946f823740 1 -- 172.21.15.5:0/1023957137 >> 172.21.15.5:0/1023957137 conn(0x55de3b0b9200 msgr2=0x55de3b0b95e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:54:19.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:18.944+0000 7f946f823740 1 -- 172.21.15.5:0/1023957137 shutdown_connections 2024-09-10T09:54:19.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:18.944+0000 7f945d400640 1 --2- 172.21.15.5:0/3394994491 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f944c0a2c00 0x7f944c0a50c0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x55de3b0ba580 tx=0x7f945000b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:54:19.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:18.944+0000 7f946f823740 1 -- 172.21.15.5:0/1023957137 wait complete. 2024-09-10T09:54:19.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:18.944+0000 7f946f823740 1 -- start start 2024-09-10T09:54:19.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:18.944+0000 7f946f823740 1 Processor -- start 2024-09-10T09:54:19.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:18.944+0000 7f946f823740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55de3b0bcc70 0x55de3b0c4d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:19.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:18.944+0000 7f946f823740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55de3b0bd590 0x55de3b0c52d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:19.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:18.944+0000 7f946f823740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55de3b0c7e50 0x55de3b0c5810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:19.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:18.944+0000 7f946f823740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55de3b0caa20 con 0x55de3b0c7e50 2024-09-10T09:54:19.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:18.944+0000 7f946f823740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55de3afde0f0 con 0x55de3b0bcc70 2024-09-10T09:54:19.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:18.944+0000 7f946f823740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55de3b0caba0 con 0x55de3b0bd590 2024-09-10T09:54:19.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:18.944+0000 7f946f823740 10 client.0 _ll_register_callbacks cb 0x55de3afc5bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:54:19.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:18.944+0000 7f945d400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55de3b0bd590 0x55de3b0c52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:19.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:18.944+0000 7f9465e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55de3b0bcc70 0x55de3b0c4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:19.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:18.944+0000 7f945d400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55de3b0bd590 0x55de3b0c52d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:53248/0 (socket says 172.21.15.5:53248) 2024-09-10T09:54:19.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:18.944+0000 7f9465e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55de3b0bcc70 0x55de3b0c4d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:58588/0 (socket says 172.21.15.5:58588) 2024-09-10T09:54:19.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:18.944+0000 7f945d400640 1 -- 172.21.15.5:0/1800266184 learned_addr learned my addr 172.21.15.5:0/1800266184 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:54:19.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:18.944+0000 7f9466800640 1 --2- 172.21.15.5:0/1800266184 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55de3b0c7e50 0x55de3b0c5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:19.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:18.944+0000 7f9465e00640 1 -- 172.21.15.5:0/1800266184 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55de3b0bd590 msgr2=0x55de3b0c52d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:54:19.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:18.944+0000 7f9465e00640 1 --2- 172.21.15.5:0/1800266184 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55de3b0bd590 0x55de3b0c52d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:19.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:18.944+0000 7f9465e00640 1 -- 172.21.15.5:0/1800266184 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55de3b0c7e50 msgr2=0x55de3b0c5810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:54:19.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:18.944+0000 7f9465e00640 1 --2- 172.21.15.5:0/1800266184 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55de3b0c7e50 0x55de3b0c5810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:19.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:18.944+0000 7f9465e00640 1 -- 172.21.15.5:0/1800266184 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55de3ad943c0 con 0x55de3b0bcc70 2024-09-10T09:54:19.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:18.944+0000 7f9466800640 1 --2- 172.21.15.5:0/1800266184 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55de3b0c7e50 0x55de3b0c5810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:54:19.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:18.944+0000 7f9465e00640 1 --2- 172.21.15.5:0/1800266184 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55de3b0bcc70 0x55de3b0c4d90 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f945802a9a0 tx=0x7f9458005250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ffe2720857d32891 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:54:19.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:18.944+0000 7f944be00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-09-10T09:54:19.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:18.950+0000 7f944be00640 1 -- 172.21.15.5:0/1800266184 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9458040030 con 0x55de3b0bcc70 2024-09-10T09:54:19.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:18.950+0000 7f946f823740 1 -- 172.21.15.5:0/1800266184 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f94580318f0 con 0x55de3b0bcc70 2024-09-10T09:54:19.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:18.950+0000 7f944be00640 1 -- 172.21.15.5:0/1800266184 <== mon.2 v2:172.21.15.102:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f94580316f0 con 0x55de3b0bcc70 2024-09-10T09:54:19.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:18.950+0000 7f944be00640 1 -- 172.21.15.5:0/1800266184 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f9458035070 con 0x55de3b0bcc70 2024-09-10T09:54:19.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:18.950+0000 7f946f823740 10 client.28538 fetch_fsmap learned FSMap version 38 2024-09-10T09:54:19.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:18.950+0000 7f946f823740 1 -- 172.21.15.5:0/1800266184 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55de3b0d6ab0 con 0x55de3b0bcc70 2024-09-10T09:54:19.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:18.950+0000 7f944be00640 1 -- 172.21.15.5:0/1800266184 <== mon.2 v2:172.21.15.102:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f94580318f0 con 0x55de3b0bcc70 2024-09-10T09:54:19.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:18.950+0000 7f944be00640 1 -- 172.21.15.5:0/1800266184 <== mon.2 v2:172.21.15.102:3300/0 5 ==== fsmap(e 38) ==== 8386+0+0 (secure 0 0 0) 0x7f9458004c10 con 0x55de3b0bcc70 2024-09-10T09:54:19.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:18.950+0000 7f946f823740 10 client.28538 fetch_fsmap finished waiting for FSMap version 38 2024-09-10T09:54:19.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:18.950+0000 7f946f823740 10 client.28538 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-09-10T09:54:19.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:18.950+0000 7f946f823740 20 client.28538 populate_metadata read hostname 'smithi005' 2024-09-10T09:54:19.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:18.950+0000 7f946f823740 1 --2- 172.21.15.5:0/1800266184 >> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] conn(0x55de3b0d7e00 0x55de3b0da1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:19.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:18.950+0000 7f946f823740 4 client.28538 mds_command: new command op to 24443 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:54:19.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:18.950+0000 7f946f823740 1 -- 172.21.15.5:0/1800266184 --> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55de3b030950 con 0x55de3b0d7e00 2024-09-10T09:54:19.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:18.950+0000 7f945d400640 1 --2- 172.21.15.5:0/1800266184 >> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] conn(0x55de3b0d7e00 0x55de3b0da1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:19.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:18.950+0000 7f945d400640 1 --2- 172.21.15.5:0/1800266184 >> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] conn(0x55de3b0d7e00 0x55de3b0da1f0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=9ca8b8b01f4a20c2 server_cookie=415a6b3428ffe2aa in_seq=0 out_seq=0 2024-09-10T09:54:19.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:18.950+0000 7f944be00640 10 client.28538 ms_handle_connect on v2:172.21.15.102:6832/2457007427 2024-09-10T09:54:19.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:18.950+0000 7f944be00640 1 -- 172.21.15.5:0/1800266184 <== mds.4 v2:172.21.15.102:6832/2457007427 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55de3b030950 con 0x55de3b0d7e00 2024-09-10T09:54:19.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:18.950+0000 7f944be00640 10 client.28538 handle_command_reply: tid=0 2024-09-10T09:54:19.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:18.950+0000 7f946f823740 1 client.28538 shutdown 2024-09-10T09:54:19.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:18.950+0000 7f946f823740 1 -- 172.21.15.5:0/1800266184 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55de3b0bcc70 msgr2=0x55de3b0c4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:19.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:18.950+0000 7f946f823740 1 --2- 172.21.15.5:0/1800266184 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55de3b0bcc70 0x55de3b0c4d90 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f945802a9a0 tx=0x7f9458005250 comp rx=0 tx=0).stop 2024-09-10T09:54:19.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:18.950+0000 7f946f823740 1 -- 172.21.15.5:0/1800266184 shutdown_connections 2024-09-10T09:54:19.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:18.950+0000 7f946f823740 1 --2- 172.21.15.5:0/1800266184 >> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] conn(0x55de3b0d7e00 0x55de3b0da1f0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:19.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:18.950+0000 7f946f823740 1 --2- 172.21.15.5:0/1800266184 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55de3b0c7e50 0x55de3b0c5810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:19.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:18.950+0000 7f946f823740 1 --2- 172.21.15.5:0/1800266184 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55de3b0bd590 0x55de3b0c52d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:19.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:18.950+0000 7f946f823740 1 --2- 172.21.15.5:0/1800266184 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55de3b0bcc70 0x55de3b0c4d90 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:19.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:18.950+0000 7f944be00640 0 client.28538 ms_handle_reset on v2:172.21.15.102:6832/2457007427 2024-09-10T09:54:19.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:18.950+0000 7f946f823740 1 -- 172.21.15.5:0/1800266184 >> 172.21.15.5:0/1800266184 conn(0x55de3b0b9200 msgr2=0x55de3b0beee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:54:19.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:18.950+0000 7f946f823740 1 -- 172.21.15.5:0/1800266184 shutdown_connections 2024-09-10T09:54:19.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:18.950+0000 7f946f823740 1 -- 172.21.15.5:0/1800266184 wait complete. 2024-09-10T09:54:19.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:18.950+0000 7f946f823740 20 client.28538 trim_cache size 0 max 16384 2024-09-10T09:54:19.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:18.967+0000 7f946f823740 1 Processor -- start 2024-09-10T09:54:19.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:18.967+0000 7f946f823740 1 -- start start 2024-09-10T09:54:19.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:18.967+0000 7f946f823740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55de3b0bcc70 0x55de3b0bf6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:19.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:18.967+0000 7f946f823740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55de3b0bd590 0x55de3b0df820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:19.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:18.967+0000 7f946f823740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55de3b0c7e50 0x55de3b0dfd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:19.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:18.967+0000 7f946f823740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f94681b0ff0 con 0x55de3b0c7e50 2024-09-10T09:54:19.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:18.967+0000 7f946f823740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f94681b2bf0 con 0x55de3b0bd590 2024-09-10T09:54:19.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:18.967+0000 7f946f823740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f94681b16c0 con 0x55de3b0bcc70 2024-09-10T09:54:19.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:18.967+0000 7f945d400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55de3b0bd590 0x55de3b0df820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:19.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:18.967+0000 7f9466800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55de3b0c7e50 0x55de3b0dfd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:19.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:18.967+0000 7f945d400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55de3b0bd590 0x55de3b0df820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:58604/0 (socket says 172.21.15.5:58604) 2024-09-10T09:54:19.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:18.967+0000 7f945d400640 1 -- 172.21.15.5:0/1045679223 learned_addr learned my addr 172.21.15.5:0/1045679223 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:54:19.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:18.967+0000 7f9465e00640 1 --2- 172.21.15.5:0/1045679223 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55de3b0bcc70 0x55de3b0bf6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:19.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:18.968+0000 7f945d400640 1 -- 172.21.15.5:0/1045679223 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55de3b0bcc70 msgr2=0x55de3b0bf6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:54:19.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:18.968+0000 7f945d400640 1 --2- 172.21.15.5:0/1045679223 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55de3b0bcc70 0x55de3b0bf6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:19.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:18.968+0000 7f945d400640 1 -- 172.21.15.5:0/1045679223 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55de3b0c7e50 msgr2=0x55de3b0dfd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:54:19.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:18.968+0000 7f945d400640 1 --2- 172.21.15.5:0/1045679223 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55de3b0c7e50 0x55de3b0dfd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:19.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:18.968+0000 7f945d400640 1 -- 172.21.15.5:0/1045679223 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55de3b0b4770 con 0x55de3b0bd590 2024-09-10T09:54:19.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:18.968+0000 7f9466800640 1 --2- 172.21.15.5:0/1045679223 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55de3b0c7e50 0x55de3b0dfd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:54:19.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:18.968+0000 7f945d400640 1 --2- 172.21.15.5:0/1045679223 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55de3b0bd590 0x55de3b0df820 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x55de3b0c63d0 tx=0x7f94500046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f79ee0ace0e94924 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:54:19.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:18.968+0000 7f9456a00640 1 -- 172.21.15.5:0/1045679223 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9450010dd0 con 0x55de3b0bd590 2024-09-10T09:54:19.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:18.968+0000 7f9456a00640 1 -- 172.21.15.5:0/1045679223 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9450012ce0 con 0x55de3b0bd590 2024-09-10T09:54:19.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:18.968+0000 7f9456a00640 1 -- 172.21.15.5:0/1045679223 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9450034460 con 0x55de3b0bd590 2024-09-10T09:54:19.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:18.968+0000 7f946f823740 1 -- 172.21.15.5:0/1045679223 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55de3b0bd590 msgr2=0x55de3b0df820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:19.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:18.968+0000 7f946f823740 1 --2- 172.21.15.5:0/1045679223 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55de3b0bd590 0x55de3b0df820 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x55de3b0c63d0 tx=0x7f94500046c0 comp rx=0 tx=0).stop 2024-09-10T09:54:19.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:18.968+0000 7f946f823740 1 -- 172.21.15.5:0/1045679223 shutdown_connections 2024-09-10T09:54:19.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:18.968+0000 7f946f823740 1 --2- 172.21.15.5:0/1045679223 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55de3b0c7e50 0x55de3b0dfd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:19.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:18.968+0000 7f946f823740 1 --2- 172.21.15.5:0/1045679223 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55de3b0bd590 0x55de3b0df820 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:19.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:18.968+0000 7f946f823740 1 --2- 172.21.15.5:0/1045679223 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55de3b0bcc70 0x55de3b0bf6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:19.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:18.968+0000 7f946f823740 1 -- 172.21.15.5:0/1045679223 >> 172.21.15.5:0/1045679223 conn(0x55de3b0b9200 msgr2=0x55de3b0c3e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:54:19.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:18.968+0000 7f946f823740 1 -- 172.21.15.5:0/1045679223 shutdown_connections 2024-09-10T09:54:19.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:18.968+0000 7f946f823740 1 -- 172.21.15.5:0/1045679223 wait complete. 2024-09-10T09:54:19.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:18.968+0000 7f946f823740 1 -- start start 2024-09-10T09:54:19.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:18.968+0000 7f946f823740 1 Processor -- start 2024-09-10T09:54:19.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:18.968+0000 7f946f823740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55de3b0bcc70 0x55de3b0d00c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:19.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:18.968+0000 7f946f823740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55de3b0bd590 0x55de3b0d0600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:19.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:18.968+0000 7f946f823740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55de3b0c7e50 0x55de3b0cb450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:19.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:18.968+0000 7f946f823740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f94681b2520 con 0x55de3b0bd590 2024-09-10T09:54:19.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:18.968+0000 7f946f823740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f94681b1e50 con 0x55de3b0bcc70 2024-09-10T09:54:19.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:18.968+0000 7f946f823740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f94681aefd0 con 0x55de3b0c7e50 2024-09-10T09:54:19.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:18.968+0000 7f946f823740 10 client.0 _ll_register_callbacks cb 0x7f9450012640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:54:19.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:18.968+0000 7f945d400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55de3b0bd590 0x55de3b0d0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:19.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:18.968+0000 7f9465e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55de3b0bcc70 0x55de3b0d00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:19.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:18.968+0000 7f945d400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55de3b0bd590 0x55de3b0d0600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55066/0 (socket says 172.21.15.5:55066) 2024-09-10T09:54:19.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:18.968+0000 7f945d400640 1 -- 172.21.15.5:0/2973615913 learned_addr learned my addr 172.21.15.5:0/2973615913 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:54:19.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:18.968+0000 7f9466800640 1 --2- 172.21.15.5:0/2973615913 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55de3b0c7e50 0x55de3b0cb450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:19.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:18.968+0000 7f945d400640 1 -- 172.21.15.5:0/2973615913 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55de3b0c7e50 msgr2=0x55de3b0cb450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:54:19.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:18.968+0000 7f945d400640 1 --2- 172.21.15.5:0/2973615913 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55de3b0c7e50 0x55de3b0cb450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:19.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:18.968+0000 7f945d400640 1 -- 172.21.15.5:0/2973615913 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55de3b0bcc70 msgr2=0x55de3b0d00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:54:19.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:18.968+0000 7f945d400640 1 --2- 172.21.15.5:0/2973615913 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55de3b0bcc70 0x55de3b0d00c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:19.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:18.968+0000 7f945d400640 1 -- 172.21.15.5:0/2973615913 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55de3b0cbc80 con 0x55de3b0bd590 2024-09-10T09:54:19.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:18.968+0000 7f945d400640 1 --2- 172.21.15.5:0/2973615913 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55de3b0bd590 0x55de3b0d0600 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f9450012af0 tx=0x7f9450012e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e44f8c1fb3b65bee server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:54:19.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:18.968+0000 7f944be00640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-10T09:54:19.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:18.968+0000 7f944be00640 1 -- 172.21.15.5:0/2973615913 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9450038020 con 0x55de3b0bd590 2024-09-10T09:54:19.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:18.968+0000 7f944be00640 1 -- 172.21.15.5:0/2973615913 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f945001b040 con 0x55de3b0bd590 2024-09-10T09:54:19.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:18.968+0000 7f946f823740 1 -- 172.21.15.5:0/2973615913 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9450012ce0 con 0x55de3b0bd590 2024-09-10T09:54:19.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:18.968+0000 7f944be00640 1 -- 172.21.15.5:0/2973615913 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f9450043050 con 0x55de3b0bd590 2024-09-10T09:54:19.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:18.968+0000 7f946f823740 10 client.18486 fetch_fsmap learned FSMap version 38 2024-09-10T09:54:19.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:18.968+0000 7f946f823740 1 -- 172.21.15.5:0/2973615913 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55de3b142370 con 0x55de3b0bd590 2024-09-10T09:54:19.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:18.968+0000 7f944be00640 1 -- 172.21.15.5:0/2973615913 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9450012ce0 con 0x55de3b0bd590 2024-09-10T09:54:19.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:18.968+0000 7f944be00640 1 -- 172.21.15.5:0/2973615913 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 38) ==== 8386+0+0 (secure 0 0 0) 0x7f945001d070 con 0x55de3b0bd590 2024-09-10T09:54:19.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:18.968+0000 7f946f823740 10 client.18486 fetch_fsmap finished waiting for FSMap version 38 2024-09-10T09:54:19.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:18.968+0000 7f946f823740 10 client.18486 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-09-10T09:54:19.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:18.968+0000 7f946f823740 20 client.18486 populate_metadata read hostname 'smithi005' 2024-09-10T09:54:19.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:18.968+0000 7f946f823740 1 --2- 172.21.15.5:0/2973615913 >> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] conn(0x55de3b142a80 0x55de3b144ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:19.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:18.968+0000 7f946f823740 4 client.18486 mds_command: new command op to 24443 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T09:54:19.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:18.968+0000 7f946f823740 1 -- 172.21.15.5:0/2973615913 --> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55de3b02ae60 con 0x55de3b142a80 2024-09-10T09:54:19.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:18.968+0000 7f9465e00640 1 --2- 172.21.15.5:0/2973615913 >> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] conn(0x55de3b142a80 0x55de3b144ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:19.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:18.968+0000 7f9465e00640 1 --2- 172.21.15.5:0/2973615913 >> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] conn(0x55de3b142a80 0x55de3b144ea0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=3050b6aac4cf034a server_cookie=3aa8c0b2cf640c36 in_seq=0 out_seq=0 2024-09-10T09:54:19.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:18.968+0000 7f944be00640 10 client.18486 ms_handle_connect on v2:172.21.15.102:6832/2457007427 2024-09-10T09:54:19.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:18.974+0000 7f944be00640 1 -- 172.21.15.5:0/2973615913 <== mds.4 v2:172.21.15.102:6832/2457007427 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55de3b02ae60 con 0x55de3b142a80 2024-09-10T09:54:19.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:18.974+0000 7f944be00640 10 client.18486 handle_command_reply: tid=0 2024-09-10T09:54:19.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:18.974+0000 7f946f823740 1 client.18486 shutdown 2024-09-10T09:54:19.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:18.974+0000 7f946f823740 1 -- 172.21.15.5:0/2973615913 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55de3b0bd590 msgr2=0x55de3b0d0600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:19.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:18.974+0000 7f946f823740 1 --2- 172.21.15.5:0/2973615913 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55de3b0bd590 0x55de3b0d0600 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f9450012af0 tx=0x7f9450012e90 comp rx=0 tx=0).stop 2024-09-10T09:54:19.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:18.974+0000 7f946f823740 1 -- 172.21.15.5:0/2973615913 shutdown_connections 2024-09-10T09:54:19.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:18.974+0000 7f946f823740 1 --2- 172.21.15.5:0/2973615913 >> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] conn(0x55de3b142a80 0x55de3b144ea0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:19.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:18.974+0000 7f946f823740 1 --2- 172.21.15.5:0/2973615913 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55de3b0c7e50 0x55de3b0cb450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:19.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:18.974+0000 7f946f823740 1 --2- 172.21.15.5:0/2973615913 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55de3b0bd590 0x55de3b0d0600 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:19.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:18.974+0000 7f946f823740 1 --2- 172.21.15.5:0/2973615913 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55de3b0bcc70 0x55de3b0d00c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:19.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:18.974+0000 7f946f823740 1 -- 172.21.15.5:0/2973615913 >> 172.21.15.5:0/2973615913 conn(0x55de3b0b9200 msgr2=0x55de3b0b7b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:54:19.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:18.974+0000 7f944be00640 0 client.18486 ms_handle_reset on v2:172.21.15.102:6832/2457007427 2024-09-10T09:54:19.059 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-10T09:54:19.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:18.974+0000 7f946f823740 1 -- 172.21.15.5:0/2973615913 shutdown_connections 2024-09-10T09:54:19.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:18.974+0000 7f946f823740 1 -- 172.21.15.5:0/2973615913 wait complete. 2024-09-10T09:54:19.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:18.974+0000 7f946f823740 20 client.18486 trim_cache size 0 max 16384 2024-09-10T09:54:19.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:18.974+0000 7f946ce00640 1 -- 172.21.15.5:0/3394994491 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f944c0a2c00 msgr2=0x7f944c0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:19.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:18.974+0000 7f946ce00640 1 --2- 172.21.15.5:0/3394994491 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f944c0a2c00 0x7f944c0a50c0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x55de3b0ba580 tx=0x7f945000b390 comp rx=0 tx=0).stop 2024-09-10T09:54:19.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:18.974+0000 7f946ce00640 1 -- 172.21.15.5:0/3394994491 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f946810e450 msgr2=0x7f946819efb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:19.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:18.974+0000 7f946ce00640 1 --2- 172.21.15.5:0/3394994491 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f946810e450 0x7f946819efb0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f9460009c90 tx=0x7f946000c6a0 comp rx=0 tx=0).stop 2024-09-10T09:54:19.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:18.974+0000 7f946ce00640 1 -- 172.21.15.5:0/3394994491 shutdown_connections 2024-09-10T09:54:19.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:18.974+0000 7f946ce00640 1 --2- 172.21.15.5:0/3394994491 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f944c0a2c00 0x7f944c0a50c0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:19.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:18.974+0000 7f946ce00640 1 --2- 172.21.15.5:0/3394994491 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f946810e450 0x7f946819efb0 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:19.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:18.974+0000 7f946ce00640 1 --2- 172.21.15.5:0/3394994491 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9468074e30 0x7f946819ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:19.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:18.974+0000 7f946ce00640 1 --2- 172.21.15.5:0/3394994491 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9468074510 0x7f946819a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:19.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:18.974+0000 7f946ce00640 1 -- 172.21.15.5:0/3394994491 >> 172.21.15.5:0/3394994491 conn(0x7f94680fd8b0 msgr2=0x7f94680fe5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:54:19.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:18.974+0000 7f946ce00640 1 -- 172.21.15.5:0/3394994491 shutdown_connections 2024-09-10T09:54:19.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:18.974+0000 7f946ce00640 1 -- 172.21.15.5:0/3394994491 wait complete. 2024-09-10T09:54:19.621 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:54:19 smithi005 ceph-mon[30350]: pgmap v872: 129 pgs: 129 active+clean; 559 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 311 KiB/s rd, 4.7 MiB/s wr, 200 op/s 2024-09-10T09:54:19.638 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:54:19.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:54:19 smithi159 ceph-mon[35513]: pgmap v872: 129 pgs: 129 active+clean; 559 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 311 KiB/s rd, 4.7 MiB/s wr, 200 op/s 2024-09-10T09:54:19.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:54:19 smithi102 ceph-mon[35442]: pgmap v872: 129 pgs: 129 active+clean; 559 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 311 KiB/s rd, 4.7 MiB/s wr, 200 op/s 2024-09-10T09:54:19.965 INFO:tasks.workunit.client.0.smithi005.stdout: 10 39050 19.29 MB/sec warmup 118 sec latency 29726.772 ms 2024-09-10T09:54:20.965 INFO:tasks.workunit.client.0.smithi005.stdout: 10 39050 19.13 MB/sec warmup 119 sec latency 30726.832 ms 2024-09-10T09:54:21.576 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:54:21 smithi005 ceph-mon[30350]: pgmap v873: 129 pgs: 129 active+clean; 547 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 328 KiB/s rd, 4.4 MiB/s wr, 228 op/s 2024-09-10T09:54:21.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:54:21 smithi159 ceph-mon[35513]: pgmap v873: 129 pgs: 129 active+clean; 547 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 328 KiB/s rd, 4.4 MiB/s wr, 228 op/s 2024-09-10T09:54:21.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:54:21 smithi102 ceph-mon[35442]: pgmap v873: 129 pgs: 129 active+clean; 547 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 328 KiB/s rd, 4.4 MiB/s wr, 228 op/s 2024-09-10T09:54:22.051 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-10T09:54:22.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:22.511+0000 7fb235c00640 1 -- 172.21.15.5:0/2274224793 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb23010e890 msgr2=0x7fb23010ed40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:22.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:22.511+0000 7fb235c00640 1 --2- 172.21.15.5:0/2274224793 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb23010e890 0x7fb23010ed40 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fb220009ca0 tx=0x7fb22002a8d0 comp rx=0 tx=0).stop 2024-09-10T09:54:22.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:22.517+0000 7fb235c00640 1 -- 172.21.15.5:0/2274224793 shutdown_connections 2024-09-10T09:54:22.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:22.517+0000 7fb235c00640 1 --2- 172.21.15.5:0/2274224793 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb23010e890 0x7fb23010ed40 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:22.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:22.517+0000 7fb235c00640 1 --2- 172.21.15.5:0/2274224793 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb230109b10 0x7fb23010e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:22.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:22.517+0000 7fb235c00640 1 --2- 172.21.15.5:0/2274224793 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb2301091f0 0x7fb2301095d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:22.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:22.517+0000 7fb235c00640 1 -- 172.21.15.5:0/2274224793 >> 172.21.15.5:0/2274224793 conn(0x7fb23006e2d0 msgr2=0x7fb23006e6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:54:22.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:22.517+0000 7fb235c00640 1 -- 172.21.15.5:0/2274224793 shutdown_connections 2024-09-10T09:54:22.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:22.517+0000 7fb235c00640 1 -- 172.21.15.5:0/2274224793 wait complete. 2024-09-10T09:54:22.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:22.517+0000 7fb235c00640 1 Processor -- start 2024-09-10T09:54:22.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:22.517+0000 7fb235c00640 1 -- start start 2024-09-10T09:54:22.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:22.517+0000 7fb235c00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb2301091f0 0x7fb23011c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:22.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:22.517+0000 7fb235c00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb230109b10 0x7fb230119380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:22.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:22.517+0000 7fb235c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb23010e890 0x7fb2301198c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:22.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:22.517+0000 7fb235c00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fb23011cc20 con 0x7fb230109b10 2024-09-10T09:54:22.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:22.517+0000 7fb235c00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fb230112520 con 0x7fb23010e890 2024-09-10T09:54:22.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:22.517+0000 7fb235c00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fb23011cda0 con 0x7fb2301091f0 2024-09-10T09:54:22.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:22.517+0000 7fb22ea00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb2301091f0 0x7fb23011c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:22.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:22.517+0000 7fb22ea00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb2301091f0 0x7fb23011c2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:37978/0 (socket says 172.21.15.5:37978) 2024-09-10T09:54:22.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:22.517+0000 7fb22ea00640 1 -- 172.21.15.5:0/315764297 learned_addr learned my addr 172.21.15.5:0/315764297 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:54:22.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:22.517+0000 7fb22f400640 1 --2- 172.21.15.5:0/315764297 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb23010e890 0x7fb2301198c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:22.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:22.517+0000 7fb22e000640 1 --2- 172.21.15.5:0/315764297 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb230109b10 0x7fb230119380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:22.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:22.522+0000 7fb22ea00640 1 -- 172.21.15.5:0/315764297 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb23010e890 msgr2=0x7fb2301198c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:22.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:22.522+0000 7fb22ea00640 1 --2- 172.21.15.5:0/315764297 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb23010e890 0x7fb2301198c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:22.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:22.522+0000 7fb22ea00640 1 -- 172.21.15.5:0/315764297 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb230109b10 msgr2=0x7fb230119380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:22.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:22.522+0000 7fb22ea00640 1 --2- 172.21.15.5:0/315764297 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb230109b10 0x7fb230119380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:22.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:22.522+0000 7fb22ea00640 1 -- 172.21.15.5:0/315764297 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb230119fa0 con 0x7fb2301091f0 2024-09-10T09:54:22.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:22.523+0000 7fb22ea00640 1 --2- 172.21.15.5:0/315764297 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb2301091f0 0x7fb23011c2d0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fb22800d950 tx=0x7fb22800de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:54:22.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:22.523+0000 7fb227400640 1 -- 172.21.15.5:0/315764297 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb228017070 con 0x7fb2301091f0 2024-09-10T09:54:22.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:22.523+0000 7fb227400640 1 -- 172.21.15.5:0/315764297 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb228009c30 con 0x7fb2301091f0 2024-09-10T09:54:22.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:22.523+0000 7fb227400640 1 -- 172.21.15.5:0/315764297 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb22800f6b0 con 0x7fb2301091f0 2024-09-10T09:54:22.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:22.523+0000 7fb235c00640 1 -- 172.21.15.5:0/315764297 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2301b3700 con 0x7fb2301091f0 2024-09-10T09:54:22.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:22.523+0000 7fb235c00640 1 -- 172.21.15.5:0/315764297 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2301b3aa0 con 0x7fb2301091f0 2024-09-10T09:54:22.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:22.523+0000 7fb235c00640 1 -- 172.21.15.5:0/315764297 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb230073f40 con 0x7fb2301091f0 2024-09-10T09:54:22.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:22.529+0000 7fb227400640 1 -- 172.21.15.5:0/315764297 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7fb2280109d0 con 0x7fb2301091f0 2024-09-10T09:54:22.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:22.529+0000 7fb227400640 1 --2- 172.21.15.5:0/315764297 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fb2140a2cd0 0x7fb2140a5190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:22.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:22.529+0000 7fb227400640 1 -- 172.21.15.5:0/315764297 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fb2280cee20 con 0x7fb2301091f0 2024-09-10T09:54:22.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:22.529+0000 7fb22e000640 1 --2- 172.21.15.5:0/315764297 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fb2140a2cd0 0x7fb2140a5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:22.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:22.534+0000 7fb22e000640 1 --2- 172.21.15.5:0/315764297 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fb2140a2cd0 0x7fb2140a5190 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fb23011ab40 tx=0x7fb21800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:54:22.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:22.534+0000 7fb227400640 1 -- 172.21.15.5:0/315764297 <== mon.1 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb228009de0 con 0x7fb2301091f0 2024-09-10T09:54:22.777 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:54:22.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:22.806+0000 7fb235c00640 1 -- 172.21.15.5:0/315764297 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb2301b3d30 con 0x7fb2301091f0 2024-09-10T09:54:22.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:22.807+0000 7fb227400640 1 -- 172.21.15.5:0/315764297 <== mon.1 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) ==== 94+0+11976 (secure 0 0 0) 0x7fb22809b340 con 0x7fb2301091f0 2024-09-10T09:54:22.808 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-10T09:54:22.808 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":38,"btime":"2024-09-10T09:54:17:539263+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14454,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/2970594311","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2970594311},{"type":"v1","addr":"172.21.15.5:6835","nonce":2970594311}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14514,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6839/3616502174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":3616502174},{"type":"v1","addr":"172.21.15.5:6839","nonce":3616502174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":38,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T09:35:29.626287+0000","modified":"2024-09-10T09:54:16.979953+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24542,"mds_1":24535,"mds_2":24511,"mds_3":24506,"mds_4":24443},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14520":{"gid":14520,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/2073599842","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":2073599842},{"type":"v1","addr":"172.21.15.5:6841","nonce":2073599842}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"k","rank":4,"incarnation":26,"state":"up:active","state_seq":24,"addr":"172.21.15.102:6833/2457007427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":2457007427},{"type":"v1","addr":"172.21.15.102:6833","nonce":2457007427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/356445159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":356445159},{"type":"v1","addr":"172.21.15.159:6833","nonce":356445159}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6837/2894314507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":2894314507},{"type":"v1","addr":"172.21.15.5:6837","nonce":2894314507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.102:6835/1647549111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":1647549111},{"type":"v1","addr":"172.21.15.102:6835","nonce":1647549111}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6835/952657242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":952657242},{"type":"v1","addr":"172.21.15.159:6835","nonce":952657242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":19,"addr":"172.21.15.102:6837/2689776764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":2689776764},{"type":"v1","addr":"172.21.15.102:6837","nonce":2689776764}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.159:6837/620627639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":620627639},{"type":"v1","addr":"172.21.15.159:6837","nonce":620627639}]},"join_fscid":-1,"export_targets":[0,1,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24535":{"gid":24535,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":18,"addr":"172.21.15.159:6839/2073739599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":2073739599},{"type":"v1","addr":"172.21.15.159:6839","nonce":2073739599}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24542":{"gid":24542,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.102:6839/335409749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6838","nonce":335409749},{"type":"v1","addr":"172.21.15.102:6839","nonce":335409749}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24542,"qdb_cluster":[24542,24535,24511,24506,24443]},"id":1}]} 2024-09-10T09:54:22.808 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 38 2024-09-10T09:54:22.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:22.807+0000 7fb235c00640 1 -- 172.21.15.5:0/315764297 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fb2140a2cd0 msgr2=0x7fb2140a5190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:22.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:22.807+0000 7fb235c00640 1 --2- 172.21.15.5:0/315764297 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fb2140a2cd0 0x7fb2140a5190 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fb23011ab40 tx=0x7fb21800b390 comp rx=0 tx=0).stop 2024-09-10T09:54:22.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:22.807+0000 7fb235c00640 1 -- 172.21.15.5:0/315764297 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb2301091f0 msgr2=0x7fb23011c2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:22.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:22.807+0000 7fb235c00640 1 --2- 172.21.15.5:0/315764297 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb2301091f0 0x7fb23011c2d0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fb22800d950 tx=0x7fb22800de10 comp rx=0 tx=0).stop 2024-09-10T09:54:22.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:22.807+0000 7fb235c00640 1 -- 172.21.15.5:0/315764297 shutdown_connections 2024-09-10T09:54:22.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:22.807+0000 7fb235c00640 1 --2- 172.21.15.5:0/315764297 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fb2140a2cd0 0x7fb2140a5190 secure :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fb23011ab40 tx=0x7fb21800b390 comp rx=0 tx=0).stop 2024-09-10T09:54:22.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:22.807+0000 7fb235c00640 1 --2- 172.21.15.5:0/315764297 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb23010e890 0x7fb2301198c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:22.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:22.807+0000 7fb235c00640 1 --2- 172.21.15.5:0/315764297 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb230109b10 0x7fb230119380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:22.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:22.807+0000 7fb235c00640 1 --2- 172.21.15.5:0/315764297 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb2301091f0 0x7fb23011c2d0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:22.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:22.807+0000 7fb235c00640 1 -- 172.21.15.5:0/315764297 >> 172.21.15.5:0/315764297 conn(0x7fb23006e2d0 msgr2=0x7fb23010c400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:54:22.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:22.807+0000 7fb235c00640 1 -- 172.21.15.5:0/315764297 shutdown_connections 2024-09-10T09:54:22.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:22.807+0000 7fb235c00640 1 -- 172.21.15.5:0/315764297 wait complete. 2024-09-10T09:54:22.965 INFO:tasks.workunit.client.0.smithi005.stdout: 10 39106 4.11 MB/sec execute 1 sec latency 31190.026 ms 2024-09-10T09:54:23.740 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24542 quiesce db / --timeout 90 --expiration 139.85 --await-for 45.0' 2024-09-10T09:54:23.740 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24542 quiesce db / --timeout 90 --expiration 139.85 --await-for 45.0 2024-09-10T09:54:23.830 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:54:23 smithi005 ceph-mon[30350]: pgmap v874: 129 pgs: 129 active+clean; 544 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 291 KiB/s rd, 2.8 MiB/s wr, 165 op/s 2024-09-10T09:54:23.830 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:54:23 smithi005 ceph-mon[30350]: from='client.? 172.21.15.5:0/315764297' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:54:23.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:54:23 smithi159 ceph-mon[35513]: pgmap v874: 129 pgs: 129 active+clean; 544 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 291 KiB/s rd, 2.8 MiB/s wr, 165 op/s 2024-09-10T09:54:23.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:54:23 smithi159 ceph-mon[35513]: from='client.? 172.21.15.5:0/315764297' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:54:23.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:54:23 smithi102 ceph-mon[35442]: pgmap v874: 129 pgs: 129 active+clean; 544 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 291 KiB/s rd, 2.8 MiB/s wr, 165 op/s 2024-09-10T09:54:23.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:54:23 smithi102 ceph-mon[35442]: from='client.? 172.21.15.5:0/315764297' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:54:23.965 INFO:tasks.workunit.client.0.smithi005.stdout: 10 39689 17.31 MB/sec execute 2 sec latency 43.341 ms 2024-09-10T09:54:24.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:24.147+0000 7f944ac00640 1 -- 172.21.15.5:0/1452529538 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f944411b140 msgr2=0x7f944411d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:24.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:24.147+0000 7f944ac00640 1 --2- 172.21.15.5:0/1452529538 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f944411b140 0x7f944411d530 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f943c009ca0 tx=0x7f943c02a8b0 comp rx=0 tx=0).stop 2024-09-10T09:54:24.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:24.147+0000 7f944ac00640 1 -- 172.21.15.5:0/1452529538 shutdown_connections 2024-09-10T09:54:24.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:24.147+0000 7f944ac00640 1 --2- 172.21.15.5:0/1452529538 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f944411b140 0x7f944411d530 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:24.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:24.147+0000 7f944ac00640 1 --2- 172.21.15.5:0/1452529538 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9444109830 0x7f9444109c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:24.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:24.147+0000 7f944ac00640 1 --2- 172.21.15.5:0/1452529538 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9444109450 0x7f9444110250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:24.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:24.147+0000 7f944ac00640 1 -- 172.21.15.5:0/1452529538 >> 172.21.15.5:0/1452529538 conn(0x7f944406e140 msgr2=0x7f944406e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:54:24.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:24.148+0000 7f944ac00640 1 -- 172.21.15.5:0/1452529538 shutdown_connections 2024-09-10T09:54:24.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:24.148+0000 7f944ac00640 1 -- 172.21.15.5:0/1452529538 wait complete. 2024-09-10T09:54:24.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:24.150+0000 7f944ac00640 1 Processor -- start 2024-09-10T09:54:24.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:24.150+0000 7f944ac00640 1 -- start start 2024-09-10T09:54:24.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:24.151+0000 7f944ac00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9444109450 0x7f94441a71f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:24.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:24.151+0000 7f944ac00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9444109830 0x7f94441a7730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:24.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:24.151+0000 7f944ac00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f944411b140 0x7f94441abac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:24.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:24.151+0000 7f944ac00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f9444120df0 con 0x7f9444109830 2024-09-10T09:54:24.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:24.151+0000 7f944ac00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f9444120c70 con 0x7f9444109450 2024-09-10T09:54:24.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:24.151+0000 7f944ac00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f9444120f70 con 0x7f944411b140 2024-09-10T09:54:24.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:24.151+0000 7f9443e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9444109450 0x7f94441a71f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:24.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:24.151+0000 7f9443e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9444109450 0x7f94441a71f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:38008/0 (socket says 172.21.15.5:38008) 2024-09-10T09:54:24.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:24.151+0000 7f9443e00640 1 -- 172.21.15.5:0/4809371 learned_addr learned my addr 172.21.15.5:0/4809371 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:54:24.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:24.153+0000 7f9448c00640 1 --2- 172.21.15.5:0/4809371 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f944411b140 0x7f94441abac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:24.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:24.153+0000 7f9443e00640 1 -- 172.21.15.5:0/4809371 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f944411b140 msgr2=0x7f94441abac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:24.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:24.153+0000 7f9443e00640 1 --2- 172.21.15.5:0/4809371 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f944411b140 0x7f94441abac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:24.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:24.153+0000 7f9443e00640 1 -- 172.21.15.5:0/4809371 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9444109830 msgr2=0x7f94441a7730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:24.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:24.154+0000 7f9443400640 1 --2- 172.21.15.5:0/4809371 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9444109830 0x7f94441a7730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:24.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:24.154+0000 7f9443e00640 1 --2- 172.21.15.5:0/4809371 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9444109830 0x7f94441a7730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:24.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:24.154+0000 7f9443e00640 1 -- 172.21.15.5:0/4809371 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94441ac1a0 con 0x7f9444109450 2024-09-10T09:54:24.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:24.154+0000 7f9443400640 1 --2- 172.21.15.5:0/4809371 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9444109830 0x7f94441a7730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T09:54:24.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:24.155+0000 7f9443e00640 1 --2- 172.21.15.5:0/4809371 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9444109450 0x7f94441a71f0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f943400e9f0 tx=0x7f943400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:54:24.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:24.155+0000 7f9440c00640 1 -- 172.21.15.5:0/4809371 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f943400cd80 con 0x7f9444109450 2024-09-10T09:54:24.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:24.155+0000 7f944ac00640 1 -- 172.21.15.5:0/4809371 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94441ac490 con 0x7f9444109450 2024-09-10T09:54:24.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:24.155+0000 7f9440c00640 1 -- 172.21.15.5:0/4809371 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9434010910 con 0x7f9444109450 2024-09-10T09:54:24.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:24.155+0000 7f9440c00640 1 -- 172.21.15.5:0/4809371 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94340188f0 con 0x7f9444109450 2024-09-10T09:54:24.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:24.155+0000 7f944ac00640 1 -- 172.21.15.5:0/4809371 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94441b3d70 con 0x7f9444109450 2024-09-10T09:54:24.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:24.160+0000 7f9440c00640 1 -- 172.21.15.5:0/4809371 <== mon.2 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f9434010ac0 con 0x7f9444109450 2024-09-10T09:54:24.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:24.161+0000 7f944d633740 1 Processor -- start 2024-09-10T09:54:24.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:24.162+0000 7f944d633740 1 -- start start 2024-09-10T09:54:24.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:24.162+0000 7f944d633740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556b3e10bc70 0x556b3e10c050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:24.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:24.162+0000 7f944d633740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556b3e10c590 0x556b3e116910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:24.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:24.163+0000 7f9448c00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556b3e10bc70 0x556b3e10c050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:24.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:24.163+0000 7f9448c00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556b3e10bc70 0x556b3e10c050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:38012/0 (socket says 172.21.15.5:38012) 2024-09-10T09:54:24.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:24.163+0000 7f9443e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556b3e10c590 0x556b3e116910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:24.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:24.163+0000 7f944d633740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556b3e116e50 0x556b3e119240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:24.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:24.163+0000 7f9448c00640 1 -- 172.21.15.5:0/3059138396 learned_addr learned my addr 172.21.15.5:0/3059138396 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:54:24.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:24.163+0000 7f944d633740 1 -- 172.21.15.5:0/3059138396 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x556b3e0723e0 con 0x556b3e116e50 2024-09-10T09:54:24.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:24.163+0000 7f944d633740 1 -- 172.21.15.5:0/3059138396 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x556b3e019d80 con 0x556b3e10c590 2024-09-10T09:54:24.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:24.163+0000 7f944d633740 1 -- 172.21.15.5:0/3059138396 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x556b3e06e3a0 con 0x556b3e10bc70 2024-09-10T09:54:24.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:24.164+0000 7f9448c00640 1 -- 172.21.15.5:0/3059138396 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556b3e10c590 msgr2=0x556b3e116910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:54:24.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:24.164+0000 7f9448c00640 1 --2- 172.21.15.5:0/3059138396 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556b3e10c590 0x556b3e116910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:24.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:24.164+0000 7f9448c00640 1 -- 172.21.15.5:0/3059138396 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556b3e116e50 msgr2=0x556b3e119240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:54:24.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:24.164+0000 7f9448c00640 1 --2- 172.21.15.5:0/3059138396 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556b3e116e50 0x556b3e119240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:24.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:24.164+0000 7f9448c00640 1 -- 172.21.15.5:0/3059138396 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556b3e038ae0 con 0x556b3e10bc70 2024-09-10T09:54:24.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:24.164+0000 7f9440c00640 1 --2- 172.21.15.5:0/4809371 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f94280a2b80 0x7f94280a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:24.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:24.164+0000 7f9440c00640 1 -- 172.21.15.5:0/4809371 <== mon.2 v2:172.21.15.102:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f94340cfa10 con 0x7f9444109450 2024-09-10T09:54:24.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:24.164+0000 7f9448c00640 1 --2- 172.21.15.5:0/3059138396 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556b3e10bc70 0x556b3e10c050 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f943c009ca0 tx=0x7f943c0093e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4fd394e3d85b0e63 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:54:24.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:24.165+0000 7f9439200640 1 -- 172.21.15.5:0/3059138396 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f943c0305a0 con 0x556b3e10bc70 2024-09-10T09:54:24.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:24.165+0000 7f9439200640 1 -- 172.21.15.5:0/3059138396 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f943c036030 con 0x556b3e10bc70 2024-09-10T09:54:24.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:24.165+0000 7f9439200640 1 -- 172.21.15.5:0/3059138396 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f943c031880 con 0x556b3e10bc70 2024-09-10T09:54:24.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:24.166+0000 7f9443e00640 1 --2- 172.21.15.5:0/4809371 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f94280a2b80 0x7f94280a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:24.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:24.166+0000 7f944d633740 1 -- 172.21.15.5:0/3059138396 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556b3e10bc70 msgr2=0x556b3e10c050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:24.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:24.166+0000 7f944d633740 1 --2- 172.21.15.5:0/3059138396 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556b3e10bc70 0x556b3e10c050 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f943c009ca0 tx=0x7f943c0093e0 comp rx=0 tx=0).stop 2024-09-10T09:54:24.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:24.167+0000 7f944d633740 1 -- 172.21.15.5:0/3059138396 shutdown_connections 2024-09-10T09:54:24.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:24.167+0000 7f944d633740 1 --2- 172.21.15.5:0/3059138396 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556b3e116e50 0x556b3e119240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:24.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:24.167+0000 7f944d633740 1 --2- 172.21.15.5:0/3059138396 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556b3e10c590 0x556b3e116910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:24.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:24.167+0000 7f944d633740 1 --2- 172.21.15.5:0/3059138396 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556b3e10bc70 0x556b3e10c050 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:24.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:24.167+0000 7f944d633740 1 -- 172.21.15.5:0/3059138396 >> 172.21.15.5:0/3059138396 conn(0x556b3e108200 msgr2=0x556b3e1085e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:54:24.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:24.167+0000 7f9443e00640 1 --2- 172.21.15.5:0/4809371 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f94280a2b80 0x7f94280a5040 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f94340987f0 tx=0x7f9434000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:54:24.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:24.167+0000 7f944d633740 1 -- 172.21.15.5:0/3059138396 shutdown_connections 2024-09-10T09:54:24.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:24.167+0000 7f944d633740 1 -- 172.21.15.5:0/3059138396 wait complete. 2024-09-10T09:54:24.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:24.168+0000 7f944d633740 1 -- start start 2024-09-10T09:54:24.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:24.168+0000 7f944d633740 1 Processor -- start 2024-09-10T09:54:24.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:24.169+0000 7f944d633740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556b3e10bc70 0x556b3e113d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:24.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:24.169+0000 7f944d633740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556b3e10c590 0x556b3e114290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:24.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:24.169+0000 7f944d633740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556b3e116e50 0x556b3e1147d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:24.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:24.169+0000 7f944d633740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x556b3e119a20 con 0x556b3e10bc70 2024-09-10T09:54:24.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:24.169+0000 7f944d633740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x556b3e02d0f0 con 0x556b3e116e50 2024-09-10T09:54:24.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:24.169+0000 7f944d633740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x556b3e119ba0 con 0x556b3e10c590 2024-09-10T09:54:24.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:24.169+0000 7f944d633740 10 client.0 _ll_register_callbacks cb 0x556b3e014bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:54:24.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:24.169+0000 7f9443400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556b3e10c590 0x556b3e114290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:24.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:24.170+0000 7f9443400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556b3e10c590 0x556b3e114290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:38024/0 (socket says 172.21.15.5:38024) 2024-09-10T09:54:24.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:24.170+0000 7f9443400640 1 -- 172.21.15.5:0/604545166 learned_addr learned my addr 172.21.15.5:0/604545166 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:54:24.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:24.170+0000 7f9448c00640 1 --2- 172.21.15.5:0/604545166 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556b3e116e50 0x556b3e1147d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:24.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:24.170+0000 7f9448c00640 1 --2- 172.21.15.5:0/604545166 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556b3e10bc70 0x556b3e113d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:24.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:24.170+0000 7f9443400640 1 -- 172.21.15.5:0/604545166 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556b3e116e50 msgr2=0x556b3e1147d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:54:24.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:24.170+0000 7f9443400640 1 --2- 172.21.15.5:0/604545166 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556b3e116e50 0x556b3e1147d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:24.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:24.170+0000 7f9443400640 1 -- 172.21.15.5:0/604545166 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556b3e10bc70 msgr2=0x556b3e113d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:54:24.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:24.170+0000 7f9443400640 1 --2- 172.21.15.5:0/604545166 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556b3e10bc70 0x556b3e113d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:24.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:24.170+0000 7f9443400640 1 -- 172.21.15.5:0/604545166 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556b3dde33c0 con 0x556b3e10c590 2024-09-10T09:54:24.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:24.171+0000 7f9448c00640 1 --2- 172.21.15.5:0/604545166 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556b3e10bc70 0x556b3e113d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T09:54:24.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:24.171+0000 7f9443400640 1 --2- 172.21.15.5:0/604545166 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556b3e10c590 0x556b3e114290 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f943000e320 tx=0x7f943000e7e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=27a458cf0ef444cc server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:54:24.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:24.171+0000 7f942d600640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-09-10T09:54:24.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:24.172+0000 7f942d600640 1 -- 172.21.15.5:0/604545166 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9430016070 con 0x556b3e10c590 2024-09-10T09:54:24.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:24.172+0000 7f942d600640 1 -- 172.21.15.5:0/604545166 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f94300108a0 con 0x556b3e10c590 2024-09-10T09:54:24.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:24.172+0000 7f944d633740 1 -- 172.21.15.5:0/604545166 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f943c036030 con 0x556b3e10c590 2024-09-10T09:54:24.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:24.172+0000 7f942d600640 1 -- 172.21.15.5:0/604545166 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f9430010aa0 con 0x556b3e10c590 2024-09-10T09:54:24.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:24.173+0000 7f944d633740 10 client.28561 fetch_fsmap learned FSMap version 38 2024-09-10T09:54:24.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:24.173+0000 7f944d633740 1 -- 172.21.15.5:0/604545166 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x556b3e125b50 con 0x556b3e10c590 2024-09-10T09:54:24.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:24.173+0000 7f942d600640 1 -- 172.21.15.5:0/604545166 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f943c036030 con 0x556b3e10c590 2024-09-10T09:54:24.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:24.174+0000 7f942d600640 1 -- 172.21.15.5:0/604545166 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 38) ==== 8386+0+0 (secure 0 0 0) 0x7f943000f040 con 0x556b3e10c590 2024-09-10T09:54:24.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:24.174+0000 7f944d633740 10 client.28561 fetch_fsmap finished waiting for FSMap version 38 2024-09-10T09:54:24.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:24.174+0000 7f944d633740 10 client.28561 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-09-10T09:54:24.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:24.174+0000 7f944d633740 20 client.28561 populate_metadata read hostname 'smithi005' 2024-09-10T09:54:24.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:24.174+0000 7f944d633740 1 --2- 172.21.15.5:0/604545166 >> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] conn(0x556b3e127010 0x556b3e129400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:24.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:24.174+0000 7f944d633740 4 client.28561 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:54:24.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:24.174+0000 7f944d633740 1 -- 172.21.15.5:0/604545166 --> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556b3e07f950 con 0x556b3e127010 2024-09-10T09:54:24.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:24.175+0000 7f9448c00640 1 --2- 172.21.15.5:0/604545166 >> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] conn(0x556b3e127010 0x556b3e129400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:24.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:24.176+0000 7f9448c00640 1 --2- 172.21.15.5:0/604545166 >> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] conn(0x556b3e127010 0x556b3e129400 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=b6f6f56542fbfe3 server_cookie=d616352c3fa49074 in_seq=0 out_seq=0 2024-09-10T09:54:24.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:24.176+0000 7f942d600640 10 client.28561 ms_handle_connect on v2:172.21.15.102:6836/2689776764 2024-09-10T09:54:24.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:24.177+0000 7f942d600640 1 -- 172.21.15.5:0/604545166 <== mds.3 v2:172.21.15.102:6836/2689776764 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556b3e07f950 con 0x556b3e127010 2024-09-10T09:54:24.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:24.177+0000 7f942d600640 10 client.28561 handle_command_reply: tid=0 2024-09-10T09:54:24.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:24.178+0000 7f944d633740 1 client.28561 shutdown 2024-09-10T09:54:24.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:24.178+0000 7f944d633740 1 -- 172.21.15.5:0/604545166 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556b3e10c590 msgr2=0x556b3e114290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:24.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:24.178+0000 7f944d633740 1 --2- 172.21.15.5:0/604545166 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556b3e10c590 0x556b3e114290 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f943000e320 tx=0x7f943000e7e0 comp rx=0 tx=0).stop 2024-09-10T09:54:24.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:24.178+0000 7f944d633740 1 -- 172.21.15.5:0/604545166 shutdown_connections 2024-09-10T09:54:24.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:24.178+0000 7f944d633740 1 --2- 172.21.15.5:0/604545166 >> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] conn(0x556b3e127010 0x556b3e129400 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:24.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:24.178+0000 7f944d633740 1 --2- 172.21.15.5:0/604545166 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556b3e116e50 0x556b3e1147d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:24.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:24.178+0000 7f944d633740 1 --2- 172.21.15.5:0/604545166 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556b3e10c590 0x556b3e114290 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:24.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:24.178+0000 7f944d633740 1 --2- 172.21.15.5:0/604545166 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556b3e10bc70 0x556b3e113d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:24.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:24.178+0000 7f944d633740 1 -- 172.21.15.5:0/604545166 >> 172.21.15.5:0/604545166 conn(0x556b3e108200 msgr2=0x556b3e107380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:54:24.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:24.178+0000 7f942d600640 0 client.28561 ms_handle_reset on v2:172.21.15.102:6836/2689776764 2024-09-10T09:54:24.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:24.178+0000 7f944d633740 1 -- 172.21.15.5:0/604545166 shutdown_connections 2024-09-10T09:54:24.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:24.178+0000 7f944d633740 1 -- 172.21.15.5:0/604545166 wait complete. 2024-09-10T09:54:24.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:24.179+0000 7f944d633740 20 client.28561 trim_cache size 0 max 16384 2024-09-10T09:54:24.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:24.198+0000 7f944d633740 1 Processor -- start 2024-09-10T09:54:24.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:24.198+0000 7f944d633740 1 -- start start 2024-09-10T09:54:24.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:24.199+0000 7f944d633740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556b3e10bc70 0x556b3e10ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:24.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:24.199+0000 7f944d633740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556b3e10c590 0x556b3e12ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:24.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:24.199+0000 7f944d633740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556b3e116e50 0x556b3e12f210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:24.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:24.199+0000 7f944d633740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f94441bdcf0 con 0x556b3e10c590 2024-09-10T09:54:24.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:24.199+0000 7f944d633740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f94441bf8f0 con 0x556b3e116e50 2024-09-10T09:54:24.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:24.199+0000 7f944d633740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f94441be3c0 con 0x556b3e10bc70 2024-09-10T09:54:24.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:24.199+0000 7f9448c00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556b3e10bc70 0x556b3e10ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:24.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:24.199+0000 7f9448c00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556b3e10bc70 0x556b3e10ddd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:38040/0 (socket says 172.21.15.5:38040) 2024-09-10T09:54:24.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:24.199+0000 7f9448c00640 1 -- 172.21.15.5:0/3061601310 learned_addr learned my addr 172.21.15.5:0/3061601310 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:54:24.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:24.199+0000 7f9448c00640 1 --2- 172.21.15.5:0/3061601310 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556b3e116e50 0x556b3e12f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:24.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:24.199+0000 7f9443400640 1 --2- 172.21.15.5:0/3061601310 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556b3e10c590 0x556b3e12ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:24.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:24.200+0000 7f9448c00640 1 -- 172.21.15.5:0/3061601310 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556b3e116e50 msgr2=0x556b3e12f210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:54:24.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:24.200+0000 7f9448c00640 1 --2- 172.21.15.5:0/3061601310 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556b3e116e50 0x556b3e12f210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:24.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:24.200+0000 7f9448c00640 1 -- 172.21.15.5:0/3061601310 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556b3e10c590 msgr2=0x556b3e12ecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:54:24.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:24.200+0000 7f9448c00640 1 --2- 172.21.15.5:0/3061601310 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556b3e10c590 0x556b3e12ecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:24.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:24.200+0000 7f9448c00640 1 -- 172.21.15.5:0/3061601310 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556b3e1195b0 con 0x556b3e10bc70 2024-09-10T09:54:24.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:24.200+0000 7f9443400640 1 --2- 172.21.15.5:0/3061601310 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556b3e10c590 0x556b3e12ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:54:24.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:24.201+0000 7f9448c00640 1 --2- 172.21.15.5:0/3061601310 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556b3e10bc70 0x556b3e10ddd0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f943c000f80 tx=0x7f943c02a8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a4f4b554e7895e51 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:54:24.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:24.201+0000 7f942fe00640 1 -- 172.21.15.5:0/3061601310 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f943c030e50 con 0x556b3e10bc70 2024-09-10T09:54:24.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:24.201+0000 7f942fe00640 1 -- 172.21.15.5:0/3061601310 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f943c036040 con 0x556b3e10bc70 2024-09-10T09:54:24.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:24.201+0000 7f942fe00640 1 -- 172.21.15.5:0/3061601310 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f943c031a80 con 0x556b3e10bc70 2024-09-10T09:54:24.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:24.202+0000 7f944d633740 1 -- 172.21.15.5:0/3061601310 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556b3e10bc70 msgr2=0x556b3e10ddd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:24.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:24.202+0000 7f944d633740 1 --2- 172.21.15.5:0/3061601310 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556b3e10bc70 0x556b3e10ddd0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f943c000f80 tx=0x7f943c02a8b0 comp rx=0 tx=0).stop 2024-09-10T09:54:24.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:24.202+0000 7f944d633740 1 -- 172.21.15.5:0/3061601310 shutdown_connections 2024-09-10T09:54:24.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:24.202+0000 7f944d633740 1 --2- 172.21.15.5:0/3061601310 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556b3e116e50 0x556b3e12f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:24.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:24.202+0000 7f944d633740 1 --2- 172.21.15.5:0/3061601310 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556b3e10c590 0x556b3e12ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:24.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:24.202+0000 7f944d633740 1 --2- 172.21.15.5:0/3061601310 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556b3e10bc70 0x556b3e10ddd0 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:24.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:24.202+0000 7f944d633740 1 -- 172.21.15.5:0/3061601310 >> 172.21.15.5:0/3061601310 conn(0x556b3e108200 msgr2=0x556b3e10d6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:54:24.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:24.202+0000 7f944d633740 1 -- 172.21.15.5:0/3061601310 shutdown_connections 2024-09-10T09:54:24.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:24.202+0000 7f944d633740 1 -- 172.21.15.5:0/3061601310 wait complete. 2024-09-10T09:54:24.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:24.203+0000 7f944d633740 1 -- start start 2024-09-10T09:54:24.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:24.203+0000 7f944d633740 1 Processor -- start 2024-09-10T09:54:24.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:24.203+0000 7f944d633740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556b3e10bc70 0x556b3e11ee70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:24.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:24.203+0000 7f944d633740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556b3e10c590 0x556b3e11f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:24.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:24.203+0000 7f944d633740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556b3e116e50 0x556b3e11be40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:24.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:24.203+0000 7f944d633740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f94441bf220 con 0x556b3e116e50 2024-09-10T09:54:24.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:24.203+0000 7f944d633740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f94441beb50 con 0x556b3e10bc70 2024-09-10T09:54:24.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:24.203+0000 7f944d633740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f94441bbcd0 con 0x556b3e10c590 2024-09-10T09:54:24.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:24.204+0000 7f944d633740 10 client.0 _ll_register_callbacks cb 0x7f94300092a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:54:24.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:24.204+0000 7f9443400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556b3e10c590 0x556b3e11f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:24.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:24.204+0000 7f9443400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556b3e10c590 0x556b3e11f3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:38042/0 (socket says 172.21.15.5:38042) 2024-09-10T09:54:24.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:24.204+0000 7f9443400640 1 -- 172.21.15.5:0/2594207370 learned_addr learned my addr 172.21.15.5:0/2594207370 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:54:24.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:24.204+0000 7f9448c00640 1 --2- 172.21.15.5:0/2594207370 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556b3e116e50 0x556b3e11be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:24.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:24.204+0000 7f9448c00640 1 --2- 172.21.15.5:0/2594207370 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556b3e10bc70 0x556b3e11ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:24.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:24.205+0000 7f9443400640 1 -- 172.21.15.5:0/2594207370 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556b3e10bc70 msgr2=0x556b3e11ee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:54:24.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:24.205+0000 7f9443400640 1 --2- 172.21.15.5:0/2594207370 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556b3e10bc70 0x556b3e11ee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:24.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:24.205+0000 7f9443400640 1 -- 172.21.15.5:0/2594207370 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556b3e116e50 msgr2=0x556b3e11be40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:54:24.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:24.205+0000 7f9443400640 1 --2- 172.21.15.5:0/2594207370 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556b3e116e50 0x556b3e11be40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:24.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:24.205+0000 7f9443400640 1 -- 172.21.15.5:0/2594207370 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556b3e11fa50 con 0x556b3e10c590 2024-09-10T09:54:24.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:24.205+0000 7f9448c00640 1 --2- 172.21.15.5:0/2594207370 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556b3e10bc70 0x556b3e11ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:54:24.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:24.205+0000 7f9443400640 1 --2- 172.21.15.5:0/2594207370 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556b3e10c590 0x556b3e11f3b0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f9430000c00 tx=0x7f9430009a90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c994955d4d652324 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:54:24.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:24.205+0000 7f942d600640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-09-10T09:54:24.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:24.205+0000 7f942d600640 1 -- 172.21.15.5:0/2594207370 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9430016070 con 0x556b3e10c590 2024-09-10T09:54:24.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:24.205+0000 7f944d633740 1 -- 172.21.15.5:0/2594207370 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f943c036040 con 0x556b3e10c590 2024-09-10T09:54:24.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:24.205+0000 7f942d600640 1 -- 172.21.15.5:0/2594207370 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f9430010e50 con 0x556b3e10c590 2024-09-10T09:54:24.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:24.206+0000 7f942d600640 1 -- 172.21.15.5:0/2594207370 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f9430022030 con 0x556b3e10c590 2024-09-10T09:54:24.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:24.206+0000 7f944d633740 10 client.28567 fetch_fsmap learned FSMap version 38 2024-09-10T09:54:24.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:24.206+0000 7f944d633740 1 -- 172.21.15.5:0/2594207370 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x556b3e192240 con 0x556b3e10c590 2024-09-10T09:54:24.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:24.206+0000 7f942d600640 1 -- 172.21.15.5:0/2594207370 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f943c036040 con 0x556b3e10c590 2024-09-10T09:54:24.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:24.207+0000 7f942d600640 1 -- 172.21.15.5:0/2594207370 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 38) ==== 8386+0+0 (secure 0 0 0) 0x7f9430004680 con 0x556b3e10c590 2024-09-10T09:54:24.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:24.207+0000 7f944d633740 10 client.28567 fetch_fsmap finished waiting for FSMap version 38 2024-09-10T09:54:24.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:24.207+0000 7f944d633740 10 client.28567 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-09-10T09:54:24.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:24.207+0000 7f944d633740 20 client.28567 populate_metadata read hostname 'smithi005' 2024-09-10T09:54:24.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:24.207+0000 7f944d633740 1 --2- 172.21.15.5:0/2594207370 >> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] conn(0x556b3e1929b0 0x556b3e194dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:24.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:24.207+0000 7f944d633740 4 client.28567 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T09:54:24.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:24.207+0000 7f944d633740 1 -- 172.21.15.5:0/2594207370 --> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556b3e079e60 con 0x556b3e1929b0 2024-09-10T09:54:24.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:24.207+0000 7f9448c00640 1 --2- 172.21.15.5:0/2594207370 >> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] conn(0x556b3e1929b0 0x556b3e194dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:24.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:24.208+0000 7f9448c00640 1 --2- 172.21.15.5:0/2594207370 >> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] conn(0x556b3e1929b0 0x556b3e194dd0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=13cc6739d1e8daba server_cookie=9dd5d94e8e06ab48 in_seq=0 out_seq=0 2024-09-10T09:54:24.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:24.208+0000 7f942d600640 10 client.28567 ms_handle_connect on v2:172.21.15.102:6836/2689776764 2024-09-10T09:54:24.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:24.208+0000 7f942d600640 1 -- 172.21.15.5:0/2594207370 <== mds.3 v2:172.21.15.102:6836/2689776764 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556b3e079e60 con 0x556b3e1929b0 2024-09-10T09:54:24.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:24.208+0000 7f942d600640 10 client.28567 handle_command_reply: tid=0 2024-09-10T09:54:24.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:24.208+0000 7f944d633740 1 client.28567 shutdown 2024-09-10T09:54:24.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:24.209+0000 7f944d633740 1 -- 172.21.15.5:0/2594207370 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556b3e10c590 msgr2=0x556b3e11f3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:24.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:24.209+0000 7f944d633740 1 --2- 172.21.15.5:0/2594207370 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556b3e10c590 0x556b3e11f3b0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f9430000c00 tx=0x7f9430009a90 comp rx=0 tx=0).stop 2024-09-10T09:54:24.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:24.209+0000 7f944d633740 1 -- 172.21.15.5:0/2594207370 shutdown_connections 2024-09-10T09:54:24.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:24.209+0000 7f944d633740 1 --2- 172.21.15.5:0/2594207370 >> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] conn(0x556b3e1929b0 0x556b3e194dd0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:24.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:24.210+0000 7f942d600640 0 client.28567 ms_handle_reset on v2:172.21.15.102:6836/2689776764 2024-09-10T09:54:24.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:24.210+0000 7f944d633740 1 --2- 172.21.15.5:0/2594207370 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556b3e116e50 0x556b3e11be40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:24.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:24.210+0000 7f944d633740 1 --2- 172.21.15.5:0/2594207370 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556b3e10c590 0x556b3e11f3b0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:24.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:24.210+0000 7f944d633740 1 --2- 172.21.15.5:0/2594207370 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556b3e10bc70 0x556b3e11ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:24.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:24.210+0000 7f944d633740 1 -- 172.21.15.5:0/2594207370 >> 172.21.15.5:0/2594207370 conn(0x556b3e108200 msgr2=0x556b3e10dee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:54:24.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:24.210+0000 7f944d633740 1 -- 172.21.15.5:0/2594207370 shutdown_connections 2024-09-10T09:54:24.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:24.210+0000 7f944d633740 1 -- 172.21.15.5:0/2594207370 wait complete. 2024-09-10T09:54:24.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:24.210+0000 7f944d633740 20 client.28567 trim_cache size 0 max 16384 2024-09-10T09:54:24.212 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-10T09:54:24.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:24.211+0000 7f944ac00640 1 -- 172.21.15.5:0/4809371 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f94280a2b80 msgr2=0x7f94280a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:24.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:24.211+0000 7f944ac00640 1 --2- 172.21.15.5:0/4809371 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f94280a2b80 0x7f94280a5040 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f94340987f0 tx=0x7f9434000bc0 comp rx=0 tx=0).stop 2024-09-10T09:54:24.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:24.211+0000 7f944ac00640 1 -- 172.21.15.5:0/4809371 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9444109450 msgr2=0x7f94441a71f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:24.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:24.211+0000 7f944ac00640 1 --2- 172.21.15.5:0/4809371 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9444109450 0x7f94441a71f0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f943400e9f0 tx=0x7f943400eeb0 comp rx=0 tx=0).stop 2024-09-10T09:54:24.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:24.212+0000 7f944ac00640 1 -- 172.21.15.5:0/4809371 shutdown_connections 2024-09-10T09:54:24.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:24.212+0000 7f944ac00640 1 --2- 172.21.15.5:0/4809371 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f94280a2b80 0x7f94280a5040 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:24.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:24.212+0000 7f944ac00640 1 --2- 172.21.15.5:0/4809371 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f944411b140 0x7f94441abac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:24.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:24.212+0000 7f944ac00640 1 --2- 172.21.15.5:0/4809371 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9444109830 0x7f94441a7730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:24.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:24.212+0000 7f944ac00640 1 --2- 172.21.15.5:0/4809371 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9444109450 0x7f94441a71f0 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:24.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:24.212+0000 7f944ac00640 1 -- 172.21.15.5:0/4809371 >> 172.21.15.5:0/4809371 conn(0x7f944406e140 msgr2=0x7f944411bd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:54:24.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:24.212+0000 7f944ac00640 1 -- 172.21.15.5:0/4809371 shutdown_connections 2024-09-10T09:54:24.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:24.212+0000 7f944ac00640 1 -- 172.21.15.5:0/4809371 wait complete. 2024-09-10T09:54:24.504 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:54:24.965 INFO:tasks.workunit.client.0.smithi005.stdout: 10 40422 21.50 MB/sec execute 3 sec latency 49.296 ms 2024-09-10T09:54:25.108 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-10T09:54:25.677 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:54:25 smithi005 ceph-mon[30350]: pgmap v875: 129 pgs: 129 active+clean; 541 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 258 KiB/s rd, 2.5 MiB/s wr, 147 op/s 2024-09-10T09:54:25.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:54:25 smithi159 ceph-mon[35513]: pgmap v875: 129 pgs: 129 active+clean; 541 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 258 KiB/s rd, 2.5 MiB/s wr, 147 op/s 2024-09-10T09:54:25.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:54:25 smithi102 ceph-mon[35442]: pgmap v875: 129 pgs: 129 active+clean; 541 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 258 KiB/s rd, 2.5 MiB/s wr, 147 op/s 2024-09-10T09:54:25.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:25.882+0000 7fb7a2c00640 1 -- 172.21.15.5:0/1189848104 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb79c102780 msgr2=0x7fb79c1067d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:25.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:25.882+0000 7fb7a2c00640 1 --2- 172.21.15.5:0/1189848104 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb79c102780 0x7fb79c1067d0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fb788009860 tx=0x7fb78802a340 comp rx=0 tx=0).stop 2024-09-10T09:54:25.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:25.882+0000 7fb7a2c00640 1 -- 172.21.15.5:0/1189848104 shutdown_connections 2024-09-10T09:54:25.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:25.882+0000 7fb7a2c00640 1 --2- 172.21.15.5:0/1189848104 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb79c106f00 0x7fb79c10ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:25.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:25.882+0000 7fb7a2c00640 1 --2- 172.21.15.5:0/1189848104 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb79c102780 0x7fb79c1067d0 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:25.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:25.882+0000 7fb7a2c00640 1 --2- 172.21.15.5:0/1189848104 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb79c101dd0 0x7fb79c1021b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:25.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:25.882+0000 7fb7a2c00640 1 -- 172.21.15.5:0/1189848104 >> 172.21.15.5:0/1189848104 conn(0x7fb79c0fd8b0 msgr2=0x7fb79c0ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:54:25.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:25.884+0000 7fb7a2c00640 1 -- 172.21.15.5:0/1189848104 shutdown_connections 2024-09-10T09:54:25.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:25.884+0000 7fb7a2c00640 1 -- 172.21.15.5:0/1189848104 wait complete. 2024-09-10T09:54:25.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:25.885+0000 7fb7a2c00640 1 Processor -- start 2024-09-10T09:54:25.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:25.886+0000 7fb7a2c00640 1 -- start start 2024-09-10T09:54:25.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:25.886+0000 7fb7a2c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb79c101dd0 0x7fb79c073a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:25.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:25.887+0000 7fb7a2c00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb79c102780 0x7fb79c06caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:25.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:25.887+0000 7fb7a2c00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb79c106f00 0x7fb79c06cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:25.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:25.887+0000 7fb7a2c00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fb79c111f60 con 0x7fb79c106f00 2024-09-10T09:54:25.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:25.887+0000 7fb79be00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb79c101dd0 0x7fb79c073a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:25.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:25.887+0000 7fb79be00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb79c101dd0 0x7fb79c073a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:38072/0 (socket says 172.21.15.5:38072) 2024-09-10T09:54:25.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:25.887+0000 7fb79be00640 1 -- 172.21.15.5:0/3547620595 learned_addr learned my addr 172.21.15.5:0/3547620595 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:54:25.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:25.887+0000 7fb7a2c00640 1 -- 172.21.15.5:0/3547620595 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fb79c111de0 con 0x7fb79c101dd0 2024-09-10T09:54:25.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:25.887+0000 7fb7a2c00640 1 -- 172.21.15.5:0/3547620595 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fb79c1120e0 con 0x7fb79c102780 2024-09-10T09:54:25.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:25.887+0000 7fb79b400640 1 --2- 172.21.15.5:0/3547620595 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb79c102780 0x7fb79c06caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:25.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:25.887+0000 7fb7a0c00640 1 --2- 172.21.15.5:0/3547620595 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb79c106f00 0x7fb79c06cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:25.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:25.888+0000 7fb7a0c00640 1 -- 172.21.15.5:0/3547620595 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb79c102780 msgr2=0x7fb79c06caa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:25.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:25.888+0000 7fb7a0c00640 1 --2- 172.21.15.5:0/3547620595 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb79c102780 0x7fb79c06caa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:25.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:25.888+0000 7fb7a0c00640 1 -- 172.21.15.5:0/3547620595 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb79c101dd0 msgr2=0x7fb79c073a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:25.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:25.888+0000 7fb7a0c00640 1 --2- 172.21.15.5:0/3547620595 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb79c101dd0 0x7fb79c073a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:25.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:25.888+0000 7fb7a0c00640 1 -- 172.21.15.5:0/3547620595 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb79c06d5d0 con 0x7fb79c106f00 2024-09-10T09:54:25.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:25.888+0000 7fb79be00640 1 --2- 172.21.15.5:0/3547620595 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb79c101dd0 0x7fb79c073a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:54:25.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:25.888+0000 7fb7a0c00640 1 --2- 172.21.15.5:0/3547620595 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb79c106f00 0x7fb79c06cfe0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7fb79400efc0 tx=0x7fb79400c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:54:25.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:25.889+0000 7fb798c00640 1 -- 172.21.15.5:0/3547620595 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb794003970 con 0x7fb79c106f00 2024-09-10T09:54:25.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:25.889+0000 7fb798c00640 1 -- 172.21.15.5:0/3547620595 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb794004020 con 0x7fb79c106f00 2024-09-10T09:54:25.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:25.889+0000 7fb798c00640 1 -- 172.21.15.5:0/3547620595 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb79400b9f0 con 0x7fb79c106f00 2024-09-10T09:54:25.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:25.889+0000 7fb7a2c00640 1 -- 172.21.15.5:0/3547620595 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb79c06d8c0 con 0x7fb79c106f00 2024-09-10T09:54:25.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:25.889+0000 7fb7a2c00640 1 -- 172.21.15.5:0/3547620595 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb79c071830 con 0x7fb79c106f00 2024-09-10T09:54:25.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:25.893+0000 7fb7a5872740 1 Processor -- start 2024-09-10T09:54:25.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:25.894+0000 7fb798c00640 1 -- 172.21.15.5:0/3547620595 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7fb79401d020 con 0x7fb79c106f00 2024-09-10T09:54:25.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:25.894+0000 7fb7a5872740 1 -- start start 2024-09-10T09:54:25.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:25.895+0000 7fb7a5872740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e9593b14e0 0x55e9593b18c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:25.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:25.895+0000 7fb7a5872740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e9593b1e00 0x55e9593bc180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:25.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:25.895+0000 7fb7a5872740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e9593bc6c0 0x55e9593beab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:25.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:25.895+0000 7fb7a5872740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55e959316260 con 0x55e9593b1e00 2024-09-10T09:54:25.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:25.895+0000 7fb7a5872740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55e9592bd500 con 0x55e9593b14e0 2024-09-10T09:54:25.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:25.895+0000 7fb7a5872740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55e959312220 con 0x55e9593bc6c0 2024-09-10T09:54:25.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:25.895+0000 7fb79b400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e9593b14e0 0x55e9593b18c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:25.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:25.895+0000 7fb79b400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e9593b14e0 0x55e9593b18c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:38078/0 (socket says 172.21.15.5:38078) 2024-09-10T09:54:25.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:25.895+0000 7fb79b400640 1 -- 172.21.15.5:0/4139265580 learned_addr learned my addr 172.21.15.5:0/4139265580 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:54:25.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:25.895+0000 7fb79b400640 1 --2- 172.21.15.5:0/4139265580 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e9593bc6c0 0x55e9593beab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:25.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:25.895+0000 7fb79be00640 1 --2- 172.21.15.5:0/4139265580 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e9593b1e00 0x55e9593bc180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:25.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:25.896+0000 7fb79b400640 1 -- 172.21.15.5:0/4139265580 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e9593bc6c0 msgr2=0x55e9593beab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:54:25.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:25.896+0000 7fb79b400640 1 --2- 172.21.15.5:0/4139265580 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e9593bc6c0 0x55e9593beab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:25.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:25.896+0000 7fb79b400640 1 -- 172.21.15.5:0/4139265580 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e9593b1e00 msgr2=0x55e9593bc180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:54:25.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:25.896+0000 7fb798c00640 1 --2- 172.21.15.5:0/3547620595 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fb7800a2c00 0x7fb7800a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:25.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:25.896+0000 7fb79b400640 1 --2- 172.21.15.5:0/4139265580 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e9593b1e00 0x55e9593bc180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:25.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:25.896+0000 7fb79b400640 1 -- 172.21.15.5:0/4139265580 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e959087ba0 con 0x55e9593b14e0 2024-09-10T09:54:25.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:25.897+0000 7fb798c00640 1 -- 172.21.15.5:0/3547620595 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fb7940cf390 con 0x7fb79c106f00 2024-09-10T09:54:25.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:25.896+0000 7fb79be00640 1 --2- 172.21.15.5:0/4139265580 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e9593b1e00 0x55e9593bc180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:54:25.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:25.897+0000 7fb79b400640 1 --2- 172.21.15.5:0/3547620595 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fb7800a2c00 0x7fb7800a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:25.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:25.898+0000 7fb79b400640 1 --2- 172.21.15.5:0/4139265580 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e9593b14e0 0x55e9593b18c0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fb7880095f0 tx=0x7fb788009490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7a008e100cdd0603 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:54:25.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:25.899+0000 7fb79b400640 1 --2- 172.21.15.5:0/3547620595 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fb7800a2c00 0x7fb7800a50c0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fb788030e50 tx=0x7fb78802b000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:54:25.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:25.899+0000 7fb791200640 1 -- 172.21.15.5:0/4139265580 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb788037040 con 0x55e9593b14e0 2024-09-10T09:54:25.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:25.899+0000 7fb791200640 1 -- 172.21.15.5:0/4139265580 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb788052070 con 0x55e9593b14e0 2024-09-10T09:54:25.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:25.899+0000 7fb791200640 1 -- 172.21.15.5:0/4139265580 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb788037040 con 0x55e9593b14e0 2024-09-10T09:54:25.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:25.900+0000 7fb7a5872740 1 -- 172.21.15.5:0/4139265580 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e9593b14e0 msgr2=0x55e9593b18c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:25.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:25.900+0000 7fb7a5872740 1 --2- 172.21.15.5:0/4139265580 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e9593b14e0 0x55e9593b18c0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fb7880095f0 tx=0x7fb788009490 comp rx=0 tx=0).stop 2024-09-10T09:54:25.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:25.900+0000 7fb7a5872740 1 -- 172.21.15.5:0/4139265580 shutdown_connections 2024-09-10T09:54:25.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:25.900+0000 7fb7a5872740 1 --2- 172.21.15.5:0/4139265580 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e9593bc6c0 0x55e9593beab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:25.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:25.900+0000 7fb7a5872740 1 --2- 172.21.15.5:0/4139265580 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e9593b1e00 0x55e9593bc180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:25.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:25.900+0000 7fb7a5872740 1 --2- 172.21.15.5:0/4139265580 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e9593b14e0 0x55e9593b18c0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:25.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:25.900+0000 7fb7a5872740 1 -- 172.21.15.5:0/4139265580 >> 172.21.15.5:0/4139265580 conn(0x55e9593ada70 msgr2=0x55e9593ade50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:54:25.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:25.900+0000 7fb7a5872740 1 -- 172.21.15.5:0/4139265580 shutdown_connections 2024-09-10T09:54:25.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:25.901+0000 7fb7a5872740 1 -- 172.21.15.5:0/4139265580 wait complete. 2024-09-10T09:54:25.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:25.901+0000 7fb7a5872740 1 -- start start 2024-09-10T09:54:25.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:25.902+0000 7fb7a5872740 1 Processor -- start 2024-09-10T09:54:25.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:25.902+0000 7fb7a5872740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e9593b14e0 0x55e9593b9610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:25.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:25.902+0000 7fb7a5872740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e9593b1e00 0x55e9593b9b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:25.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:25.902+0000 7fb7a5872740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e9593bc6c0 0x55e9593ba090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:25.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:25.902+0000 7fb7a5872740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55e9593bf290 con 0x55e9593b14e0 2024-09-10T09:54:25.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:25.902+0000 7fb7a5872740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55e9592d0bd0 con 0x55e9593b1e00 2024-09-10T09:54:25.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:25.902+0000 7fb7a5872740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55e9593bf410 con 0x55e9593bc6c0 2024-09-10T09:54:25.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:25.902+0000 7fb7a5872740 10 client.0 _ll_register_callbacks cb 0x55e9590250a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:54:25.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:25.903+0000 7fb7a0c00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e9593bc6c0 0x55e9593ba090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:25.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:25.903+0000 7fb7a0c00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e9593bc6c0 0x55e9593ba090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:38086/0 (socket says 172.21.15.5:38086) 2024-09-10T09:54:25.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:25.903+0000 7fb7a0c00640 1 -- 172.21.15.5:0/1916911017 learned_addr learned my addr 172.21.15.5:0/1916911017 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:54:25.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:25.903+0000 7fb79b400640 1 --2- 172.21.15.5:0/1916911017 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e9593b1e00 0x55e9593b9b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:25.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:25.903+0000 7fb79be00640 1 --2- 172.21.15.5:0/1916911017 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e9593b14e0 0x55e9593b9610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:25.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:25.904+0000 7fb7a0c00640 1 -- 172.21.15.5:0/1916911017 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e9593b1e00 msgr2=0x55e9593b9b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:54:25.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:25.904+0000 7fb7a0c00640 1 --2- 172.21.15.5:0/1916911017 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e9593b1e00 0x55e9593b9b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:25.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:25.904+0000 7fb7a0c00640 1 -- 172.21.15.5:0/1916911017 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e9593b14e0 msgr2=0x55e9593b9610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:54:25.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:25.904+0000 7fb7a0c00640 1 --2- 172.21.15.5:0/1916911017 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e9593b14e0 0x55e9593b9610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:25.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:25.904+0000 7fb7a0c00640 1 -- 172.21.15.5:0/1916911017 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e9593c8750 con 0x55e9593bc6c0 2024-09-10T09:54:25.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:25.904+0000 7fb7a0c00640 1 --2- 172.21.15.5:0/1916911017 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e9593bc6c0 0x55e9593ba090 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fb7940a3ee0 tx=0x7fb79400a3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c89aad8223830bd6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:54:25.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:25.904+0000 7fb785600640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-09-10T09:54:25.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:25.905+0000 7fb785600640 1 -- 172.21.15.5:0/1916911017 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7940cf160 con 0x55e9593bc6c0 2024-09-10T09:54:25.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:25.905+0000 7fb785600640 1 -- 172.21.15.5:0/1916911017 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fb794004560 con 0x55e9593bc6c0 2024-09-10T09:54:25.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:25.905+0000 7fb7a5872740 1 -- 172.21.15.5:0/1916911017 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb788052070 con 0x55e9593bc6c0 2024-09-10T09:54:25.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:25.905+0000 7fb785600640 1 -- 172.21.15.5:0/1916911017 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7fb79409fb50 con 0x55e9593bc6c0 2024-09-10T09:54:25.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:25.905+0000 7fb7a5872740 10 client.28579 fetch_fsmap learned FSMap version 38 2024-09-10T09:54:25.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:25.905+0000 7fb7a5872740 1 -- 172.21.15.5:0/1916911017 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55e9593cb0e0 con 0x55e9593bc6c0 2024-09-10T09:54:25.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:25.907+0000 7fb785600640 1 -- 172.21.15.5:0/1916911017 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb788052070 con 0x55e9593bc6c0 2024-09-10T09:54:25.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:25.907+0000 7fb785600640 1 -- 172.21.15.5:0/1916911017 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 38) ==== 8386+0+0 (secure 0 0 0) 0x7fb79400b040 con 0x55e9593bc6c0 2024-09-10T09:54:25.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:25.907+0000 7fb7a5872740 10 client.28579 fetch_fsmap finished waiting for FSMap version 38 2024-09-10T09:54:25.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:25.907+0000 7fb7a5872740 10 client.28579 resolve_mds: validated gid 24542 aka daemon mds.b 2024-09-10T09:54:25.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:25.907+0000 7fb7a5872740 20 client.28579 populate_metadata read hostname 'smithi005' 2024-09-10T09:54:25.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:25.907+0000 7fb7a5872740 1 --2- 172.21.15.5:0/1916911017 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55e9593cc5a0 0x55e9593ce9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:25.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:25.907+0000 7fb7a5872740 4 client.28579 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:54:25.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:25.907+0000 7fb7a5872740 1 -- 172.21.15.5:0/1916911017 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e959323990 con 0x55e9593cc5a0 2024-09-10T09:54:25.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:25.908+0000 7fb79be00640 1 --2- 172.21.15.5:0/1916911017 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55e9593cc5a0 0x55e9593ce9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:25.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:25.909+0000 7fb79be00640 1 --2- 172.21.15.5:0/1916911017 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55e9593cc5a0 0x55e9593ce9d0 crc :-1 s=READY pgs=233 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=476eeacee8391cb5 server_cookie=d0d250a17c1159df in_seq=0 out_seq=0 2024-09-10T09:54:25.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:25.909+0000 7fb785600640 10 client.28579 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T09:54:25.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:25.911+0000 7fb785600640 1 -- 172.21.15.5:0/1916911017 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e959323990 con 0x55e9593cc5a0 2024-09-10T09:54:25.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:25.911+0000 7fb785600640 10 client.28579 handle_command_reply: tid=0 2024-09-10T09:54:25.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:25.911+0000 7fb7a5872740 1 client.28579 shutdown 2024-09-10T09:54:25.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:25.914+0000 7fb7a5872740 1 -- 172.21.15.5:0/1916911017 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e9593bc6c0 msgr2=0x55e9593ba090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:25.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:25.914+0000 7fb7a5872740 1 --2- 172.21.15.5:0/1916911017 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e9593bc6c0 0x55e9593ba090 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fb7940a3ee0 tx=0x7fb79400a3b0 comp rx=0 tx=0).stop 2024-09-10T09:54:25.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:25.914+0000 7fb7a5872740 1 -- 172.21.15.5:0/1916911017 shutdown_connections 2024-09-10T09:54:25.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:25.914+0000 7fb7a5872740 1 --2- 172.21.15.5:0/1916911017 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55e9593cc5a0 0x55e9593ce9d0 crc :-1 s=READY pgs=233 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:25.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:25.914+0000 7fb7a5872740 1 --2- 172.21.15.5:0/1916911017 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e9593bc6c0 0x55e9593ba090 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:25.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:25.914+0000 7fb7a5872740 1 --2- 172.21.15.5:0/1916911017 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e9593b1e00 0x55e9593b9b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:25.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:25.914+0000 7fb7a5872740 1 --2- 172.21.15.5:0/1916911017 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e9593b14e0 0x55e9593b9610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:25.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:25.914+0000 7fb785600640 0 client.28579 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T09:54:25.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:25.914+0000 7fb7a5872740 1 -- 172.21.15.5:0/1916911017 >> 172.21.15.5:0/1916911017 conn(0x55e9593ada70 msgr2=0x55e9593acbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:54:25.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:25.915+0000 7fb7a5872740 1 -- 172.21.15.5:0/1916911017 shutdown_connections 2024-09-10T09:54:25.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:25.915+0000 7fb7a5872740 1 -- 172.21.15.5:0/1916911017 wait complete. 2024-09-10T09:54:25.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:25.915+0000 7fb7a5872740 20 client.28579 trim_cache size 0 max 16384 2024-09-10T09:54:25.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:25.944+0000 7fb7a5872740 1 Processor -- start 2024-09-10T09:54:25.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:25.944+0000 7fb7a5872740 1 -- start start 2024-09-10T09:54:25.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:25.945+0000 7fb7a5872740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e9593b14e0 0x55e9593d42f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:25.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:25.945+0000 7fb7a5872740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e9593b1e00 0x55e9593d4830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:25.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:25.945+0000 7fb7a5872740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e9593bc6c0 0x55e9593b6870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:25.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:25.945+0000 7fb7a5872740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fb79c1b54b0 con 0x55e9593bc6c0 2024-09-10T09:54:25.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:25.945+0000 7fb7a5872740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fb79c1b70b0 con 0x55e9593b1e00 2024-09-10T09:54:25.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:25.945+0000 7fb7a5872740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fb79c1b5b80 con 0x55e9593b14e0 2024-09-10T09:54:25.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:25.945+0000 7fb79b400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e9593b1e00 0x55e9593d4830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:25.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:25.945+0000 7fb79b400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e9593b1e00 0x55e9593d4830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:38100/0 (socket says 172.21.15.5:38100) 2024-09-10T09:54:25.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:25.945+0000 7fb79b400640 1 -- 172.21.15.5:0/232536349 learned_addr learned my addr 172.21.15.5:0/232536349 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:54:25.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:25.945+0000 7fb79be00640 1 --2- 172.21.15.5:0/232536349 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e9593b14e0 0x55e9593d42f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:25.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:25.946+0000 7fb79b400640 1 -- 172.21.15.5:0/232536349 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e9593b14e0 msgr2=0x55e9593d42f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:54:25.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:25.946+0000 7fb79b400640 1 --2- 172.21.15.5:0/232536349 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e9593b14e0 0x55e9593d42f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:25.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:25.946+0000 7fb79b400640 1 -- 172.21.15.5:0/232536349 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e9593bc6c0 msgr2=0x55e9593b6870 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T09:54:25.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:25.946+0000 7fb79b400640 1 --2- 172.21.15.5:0/232536349 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e9593bc6c0 0x55e9593b6870 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:25.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:25.946+0000 7fb79b400640 1 -- 172.21.15.5:0/232536349 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e9593a7fe0 con 0x55e9593b1e00 2024-09-10T09:54:25.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:25.946+0000 7fb79be00640 1 --2- 172.21.15.5:0/232536349 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e9593b14e0 0x55e9593d42f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:54:25.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:25.946+0000 7fb79b400640 1 --2- 172.21.15.5:0/232536349 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e9593b1e00 0x55e9593d4830 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fb7880304f0 tx=0x7fb788003920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cb19e73bf6d3ec2a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:54:25.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:25.946+0000 7fb787e00640 1 -- 172.21.15.5:0/232536349 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb78805b070 con 0x55e9593b1e00 2024-09-10T09:54:25.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:25.948+0000 7fb787e00640 1 -- 172.21.15.5:0/232536349 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb788052040 con 0x55e9593b1e00 2024-09-10T09:54:25.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:25.948+0000 7fb787e00640 1 -- 172.21.15.5:0/232536349 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb788037070 con 0x55e9593b1e00 2024-09-10T09:54:25.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:25.949+0000 7fb7a5872740 1 -- 172.21.15.5:0/232536349 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e9593b1e00 msgr2=0x55e9593d4830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:25.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:25.949+0000 7fb7a5872740 1 --2- 172.21.15.5:0/232536349 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e9593b1e00 0x55e9593d4830 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fb7880304f0 tx=0x7fb788003920 comp rx=0 tx=0).stop 2024-09-10T09:54:25.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:25.950+0000 7fb7a5872740 1 -- 172.21.15.5:0/232536349 shutdown_connections 2024-09-10T09:54:25.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:25.950+0000 7fb7a5872740 1 --2- 172.21.15.5:0/232536349 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e9593bc6c0 0x55e9593b6870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:25.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:25.950+0000 7fb7a5872740 1 --2- 172.21.15.5:0/232536349 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e9593b1e00 0x55e9593d4830 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:25.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:25.950+0000 7fb7a5872740 1 --2- 172.21.15.5:0/232536349 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e9593b14e0 0x55e9593d42f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:25.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:25.950+0000 7fb7a5872740 1 -- 172.21.15.5:0/232536349 >> 172.21.15.5:0/232536349 conn(0x55e9593ada70 msgr2=0x55e9593bb580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:54:25.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:25.950+0000 7fb7a5872740 1 -- 172.21.15.5:0/232536349 shutdown_connections 2024-09-10T09:54:25.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:25.950+0000 7fb7a5872740 1 -- 172.21.15.5:0/232536349 wait complete. 2024-09-10T09:54:25.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:25.951+0000 7fb7a5872740 1 -- start start 2024-09-10T09:54:25.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:25.951+0000 7fb7a5872740 1 Processor -- start 2024-09-10T09:54:25.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:25.953+0000 7fb7a5872740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e9593b14e0 0x55e9593c0a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:25.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:25.953+0000 7fb7a5872740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e9593b1e00 0x55e9593c0fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:25.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:25.953+0000 7fb7a5872740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e9593bc6c0 0x55e9594362c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:25.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:25.954+0000 7fb79be00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e9593b14e0 0x55e9593c0a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:25.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:25.954+0000 7fb7a5872740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fb79c1b69e0 con 0x55e9593bc6c0 2024-09-10T09:54:25.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:25.954+0000 7fb79b400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e9593b1e00 0x55e9593c0fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:25.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:25.954+0000 7fb7a5872740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fb79c1b6310 con 0x55e9593b14e0 2024-09-10T09:54:25.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:25.954+0000 7fb7a5872740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fb79c1b3490 con 0x55e9593b1e00 2024-09-10T09:54:25.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:25.954+0000 7fb79b400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e9593b1e00 0x55e9593c0fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:38106/0 (socket says 172.21.15.5:38106) 2024-09-10T09:54:25.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:25.954+0000 7fb79b400640 1 -- 172.21.15.5:0/231399521 learned_addr learned my addr 172.21.15.5:0/231399521 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:54:25.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:25.954+0000 7fb7a0c00640 1 --2- 172.21.15.5:0/231399521 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e9593bc6c0 0x55e9594362c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:25.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:25.954+0000 7fb79be00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e9593b14e0 0x55e9593c0a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:38102/0 (socket says 172.21.15.5:38102) 2024-09-10T09:54:25.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:25.955+0000 7fb7a5872740 10 client.0 _ll_register_callbacks cb 0x7fb7940099a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:54:25.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:25.955+0000 7fb79be00640 1 -- 172.21.15.5:0/231399521 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e9593b1e00 msgr2=0x55e9593c0fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:54:25.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:25.955+0000 7fb79be00640 1 --2- 172.21.15.5:0/231399521 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e9593b1e00 0x55e9593c0fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:25.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:25.955+0000 7fb79be00640 1 -- 172.21.15.5:0/231399521 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e9593bc6c0 msgr2=0x55e9594362c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:54:25.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:25.955+0000 7fb79be00640 1 --2- 172.21.15.5:0/231399521 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e9593bc6c0 0x55e9594362c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:25.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:25.955+0000 7fb79be00640 1 -- 172.21.15.5:0/231399521 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e959436800 con 0x55e9593b14e0 2024-09-10T09:54:25.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:25.956+0000 7fb7a0c00640 1 --2- 172.21.15.5:0/231399521 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e9593bc6c0 0x55e9594362c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:54:25.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:25.956+0000 7fb79b400640 1 --2- 172.21.15.5:0/231399521 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e9593b1e00 0x55e9593c0fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:54:25.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:25.956+0000 7fb79be00640 1 --2- 172.21.15.5:0/231399521 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e9593b14e0 0x55e9593c0a90 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x55e9593cf1e0 tx=0x7fb78c005630 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4fa29632b7dc9a61 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:54:25.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:25.956+0000 7fb785600640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-09-10T09:54:25.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:25.957+0000 7fb785600640 1 -- 172.21.15.5:0/231399521 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb78c012650 con 0x55e9593b14e0 2024-09-10T09:54:25.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:25.957+0000 7fb785600640 1 -- 172.21.15.5:0/231399521 <== mon.2 v2:172.21.15.102:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fb78c0127f0 con 0x55e9593b14e0 2024-09-10T09:54:25.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:25.958+0000 7fb7a5872740 1 -- 172.21.15.5:0/231399521 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb788052040 con 0x55e9593b14e0 2024-09-10T09:54:25.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:25.958+0000 7fb785600640 1 -- 172.21.15.5:0/231399521 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7fb78c0129f0 con 0x55e9593b14e0 2024-09-10T09:54:25.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:25.958+0000 7fb7a5872740 10 client.28583 fetch_fsmap learned FSMap version 38 2024-09-10T09:54:25.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:25.958+0000 7fb7a5872740 1 -- 172.21.15.5:0/231399521 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55e959437a80 con 0x55e9593b14e0 2024-09-10T09:54:25.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:25.958+0000 7fb785600640 1 -- 172.21.15.5:0/231399521 <== mon.2 v2:172.21.15.102:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb788052040 con 0x55e9593b14e0 2024-09-10T09:54:25.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:25.959+0000 7fb785600640 1 -- 172.21.15.5:0/231399521 <== mon.2 v2:172.21.15.102:3300/0 5 ==== fsmap(e 38) ==== 8386+0+0 (secure 0 0 0) 0x7fb78c022070 con 0x55e9593b14e0 2024-09-10T09:54:25.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:25.959+0000 7fb7a5872740 10 client.28583 fetch_fsmap finished waiting for FSMap version 38 2024-09-10T09:54:25.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:25.959+0000 7fb7a5872740 10 client.28583 resolve_mds: validated gid 24542 aka daemon mds.b 2024-09-10T09:54:25.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:25.959+0000 7fb7a5872740 20 client.28583 populate_metadata read hostname 'smithi005' 2024-09-10T09:54:25.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:25.959+0000 7fb7a5872740 1 --2- 172.21.15.5:0/231399521 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55e9594381c0 0x55e95943a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:25.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:25.959+0000 7fb7a5872740 4 client.28583 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 139.85, "await_for": 45.0}] 2024-09-10T09:54:25.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:25.959+0000 7fb7a5872740 1 -- 172.21.15.5:0/231399521 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 139.85, "await_for": 45.0}) -- 0x55e95931dce0 con 0x55e9594381c0 2024-09-10T09:54:25.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:25.960+0000 7fb79b400640 1 --2- 172.21.15.5:0/231399521 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55e9594381c0 0x55e95943a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:25.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:25.960+0000 7fb79b400640 1 --2- 172.21.15.5:0/231399521 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55e9594381c0 0x55e95943a5b0 crc :-1 s=READY pgs=234 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=af288fb563462bd3 server_cookie=c5451e46647bed12 in_seq=0 out_seq=0 2024-09-10T09:54:25.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:25.960+0000 7fb785600640 10 client.28583 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T09:54:25.965 INFO:tasks.workunit.client.0.smithi005.stdout: 10 41242 26.77 MB/sec execute 4 sec latency 227.485 ms 2024-09-10T09:54:26.017 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:54:26.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:26.617+0000 7fb785600640 1 -- 172.21.15.5:0/231399521 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55e95931dce0 con 0x55e9594381c0 2024-09-10T09:54:26.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:26.617+0000 7fb785600640 10 client.28583 handle_command_reply: tid=0 2024-09-10T09:54:26.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:26.617+0000 7fb7a5872740 1 client.28583 shutdown 2024-09-10T09:54:26.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:26.617+0000 7fb7a5872740 1 -- 172.21.15.5:0/231399521 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e9593b14e0 msgr2=0x55e9593c0a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:26.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:26.617+0000 7fb7a5872740 1 --2- 172.21.15.5:0/231399521 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e9593b14e0 0x55e9593c0a90 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x55e9593cf1e0 tx=0x7fb78c005630 comp rx=0 tx=0).stop 2024-09-10T09:54:26.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:26.617+0000 7fb7a5872740 1 -- 172.21.15.5:0/231399521 shutdown_connections 2024-09-10T09:54:26.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:26.617+0000 7fb7a5872740 1 --2- 172.21.15.5:0/231399521 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55e9594381c0 0x55e95943a5b0 crc :-1 s=READY pgs=234 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:26.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:26.617+0000 7fb7a5872740 1 --2- 172.21.15.5:0/231399521 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e9593bc6c0 0x55e9594362c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:26.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:26.617+0000 7fb7a5872740 1 --2- 172.21.15.5:0/231399521 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e9593b1e00 0x55e9593c0fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:26.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:26.617+0000 7fb7a5872740 1 --2- 172.21.15.5:0/231399521 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e9593b14e0 0x55e9593c0a90 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:26.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:26.617+0000 7fb785600640 0 client.28583 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T09:54:26.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:26.617+0000 7fb7a5872740 1 -- 172.21.15.5:0/231399521 >> 172.21.15.5:0/231399521 conn(0x55e9593ada70 msgr2=0x55e9593ac3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:54:26.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:26.618+0000 7fb7a5872740 1 -- 172.21.15.5:0/231399521 shutdown_connections 2024-09-10T09:54:26.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:26.618+0000 7fb7a5872740 1 -- 172.21.15.5:0/231399521 wait complete. 2024-09-10T09:54:26.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:26.618+0000 7fb7a5872740 20 client.28583 trim_cache size 0 max 16384 2024-09-10T09:54:26.621 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-10T09:54:26.622 INFO:teuthology.orchestra.run.smithi005.stdout: "epoch": 38, 2024-09-10T09:54:26.622 INFO:teuthology.orchestra.run.smithi005.stdout: "leader": 24542, 2024-09-10T09:54:26.622 INFO:teuthology.orchestra.run.smithi005.stdout: "set_version": 58, 2024-09-10T09:54:26.622 INFO:teuthology.orchestra.run.smithi005.stdout: "sets": { 2024-09-10T09:54:26.622 INFO:teuthology.orchestra.run.smithi005.stdout: "87c9f91d": { 2024-09-10T09:54:26.622 INFO:teuthology.orchestra.run.smithi005.stdout: "version": 58, 2024-09-10T09:54:26.622 INFO:teuthology.orchestra.run.smithi005.stdout: "age_ref": 0.0, 2024-09-10T09:54:26.622 INFO:teuthology.orchestra.run.smithi005.stdout: "state": { 2024-09-10T09:54:26.623 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "QUIESCED", 2024-09-10T09:54:26.623 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0 2024-09-10T09:54:26.623 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T09:54:26.623 INFO:teuthology.orchestra.run.smithi005.stdout: "timeout": 90.0, 2024-09-10T09:54:26.623 INFO:teuthology.orchestra.run.smithi005.stdout: "expiration": 139.8, 2024-09-10T09:54:26.623 INFO:teuthology.orchestra.run.smithi005.stdout: "members": { 2024-09-10T09:54:26.623 INFO:teuthology.orchestra.run.smithi005.stdout: "file:/": { 2024-09-10T09:54:26.623 INFO:teuthology.orchestra.run.smithi005.stdout: "excluded": false, 2024-09-10T09:54:26.623 INFO:teuthology.orchestra.run.smithi005.stdout: "state": { 2024-09-10T09:54:26.623 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "QUIESCED", 2024-09-10T09:54:26.623 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0 2024-09-10T09:54:26.623 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T09:54:26.623 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T09:54:26.624 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T09:54:26.624 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T09:54:26.624 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T09:54:26.624 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-10T09:54:26.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:26.619+0000 7fb7a2c00640 1 -- 172.21.15.5:0/3547620595 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fb7800a2c00 msgr2=0x7fb7800a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:26.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:26.619+0000 7fb7a2c00640 1 --2- 172.21.15.5:0/3547620595 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fb7800a2c00 0x7fb7800a50c0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fb788030e50 tx=0x7fb78802b000 comp rx=0 tx=0).stop 2024-09-10T09:54:26.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:26.619+0000 7fb7a2c00640 1 -- 172.21.15.5:0/3547620595 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb79c106f00 msgr2=0x7fb79c06cfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:26.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:26.619+0000 7fb7a2c00640 1 --2- 172.21.15.5:0/3547620595 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb79c106f00 0x7fb79c06cfe0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7fb79400efc0 tx=0x7fb79400c800 comp rx=0 tx=0).stop 2024-09-10T09:54:26.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:26.619+0000 7fb7a2c00640 1 -- 172.21.15.5:0/3547620595 shutdown_connections 2024-09-10T09:54:26.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:26.619+0000 7fb7a2c00640 1 --2- 172.21.15.5:0/3547620595 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fb7800a2c00 0x7fb7800a50c0 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:26.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:26.619+0000 7fb7a2c00640 1 --2- 172.21.15.5:0/3547620595 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb79c106f00 0x7fb79c06cfe0 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:26.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:26.619+0000 7fb7a2c00640 1 --2- 172.21.15.5:0/3547620595 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fb79c102780 0x7fb79c06caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:26.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:26.619+0000 7fb7a2c00640 1 --2- 172.21.15.5:0/3547620595 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fb79c101dd0 0x7fb79c073a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:26.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:26.619+0000 7fb7a2c00640 1 -- 172.21.15.5:0/3547620595 >> 172.21.15.5:0/3547620595 conn(0x7fb79c0fd8b0 msgr2=0x7fb79c0fed30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:54:26.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:26.619+0000 7fb7a2c00640 1 -- 172.21.15.5:0/3547620595 shutdown_connections 2024-09-10T09:54:26.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:26.620+0000 7fb7a2c00640 1 -- 172.21.15.5:0/3547620595 wait complete. 2024-09-10T09:54:26.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:26.681+0000 7f9d5c400640 1 -- 172.21.15.5:0/1447442223 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9d54074510 msgr2=0x7f9d540748f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:26.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:26.681+0000 7f9d5c400640 1 --2- 172.21.15.5:0/1447442223 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9d54074510 0x7f9d540748f0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f9d48009900 tx=0x7f9d4802a480 comp rx=0 tx=0).stop 2024-09-10T09:54:26.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:26.681+0000 7f9d5c400640 1 -- 172.21.15.5:0/1447442223 shutdown_connections 2024-09-10T09:54:26.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:26.681+0000 7f9d5c400640 1 --2- 172.21.15.5:0/1447442223 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9d5410e440 0x7f9d54110830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:26.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:26.681+0000 7f9d5c400640 1 --2- 172.21.15.5:0/1447442223 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9d54074e30 0x7f9d5410df00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:26.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:26.681+0000 7f9d5c400640 1 --2- 172.21.15.5:0/1447442223 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9d54074510 0x7f9d540748f0 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:26.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:26.681+0000 7f9d5c400640 1 -- 172.21.15.5:0/1447442223 >> 172.21.15.5:0/1447442223 conn(0x7f9d540fd8c0 msgr2=0x7f9d540ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:54:26.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:26.681+0000 7f9d5c400640 1 -- 172.21.15.5:0/1447442223 shutdown_connections 2024-09-10T09:54:26.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:26.681+0000 7f9d5c400640 1 -- 172.21.15.5:0/1447442223 wait complete. 2024-09-10T09:54:26.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:26.682+0000 7f9d5c400640 1 Processor -- start 2024-09-10T09:54:26.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:26.682+0000 7f9d5c400640 1 -- start start 2024-09-10T09:54:26.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:26.683+0000 7f9d5c400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9d54074510 0x7f9d5419a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:26.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:26.683+0000 7f9d5c400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9d54074e30 0x7f9d5419ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:26.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:26.683+0000 7f9d5c400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9d5410e440 0x7f9d5419ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:26.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:26.683+0000 7f9d5c400640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f9d541141a0 con 0x7f9d5410e440 2024-09-10T09:54:26.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:26.683+0000 7f9d5c400640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f9d54114020 con 0x7f9d54074510 2024-09-10T09:54:26.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:26.683+0000 7f9d5c400640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f9d54114320 con 0x7f9d54074e30 2024-09-10T09:54:26.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:26.683+0000 7f9d59a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9d54074510 0x7f9d5419a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:26.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:26.683+0000 7f9d59a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9d54074510 0x7f9d5419a640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:38120/0 (socket says 172.21.15.5:38120) 2024-09-10T09:54:26.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:26.683+0000 7f9d59a00640 1 -- 172.21.15.5:0/3852958 learned_addr learned my addr 172.21.15.5:0/3852958 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:54:26.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:26.683+0000 7f9d5a400640 1 --2- 172.21.15.5:0/3852958 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9d5410e440 0x7f9d5419ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:26.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:26.683+0000 7f9d59000640 1 --2- 172.21.15.5:0/3852958 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9d54074e30 0x7f9d5419ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:26.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:26.683+0000 7f9d59a00640 1 -- 172.21.15.5:0/3852958 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9d54074e30 msgr2=0x7f9d5419ab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:26.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:26.683+0000 7f9d59a00640 1 --2- 172.21.15.5:0/3852958 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9d54074e30 0x7f9d5419ab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:26.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:26.683+0000 7f9d59a00640 1 -- 172.21.15.5:0/3852958 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9d5410e440 msgr2=0x7f9d5419ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:26.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:26.683+0000 7f9d59a00640 1 --2- 172.21.15.5:0/3852958 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9d5410e440 0x7f9d5419ef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:26.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:26.684+0000 7f9d59a00640 1 -- 172.21.15.5:0/3852958 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d5419f5f0 con 0x7f9d54074510 2024-09-10T09:54:26.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:26.684+0000 7f9d59a00640 1 --2- 172.21.15.5:0/3852958 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9d54074510 0x7f9d5419a640 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f9d48004770 tx=0x7f9d48003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:54:26.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:26.684+0000 7f9d52000640 1 -- 172.21.15.5:0/3852958 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d4803f070 con 0x7f9d54074510 2024-09-10T09:54:26.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:26.684+0000 7f9d52000640 1 -- 172.21.15.5:0/3852958 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9d48031ca0 con 0x7f9d54074510 2024-09-10T09:54:26.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:26.684+0000 7f9d52000640 1 -- 172.21.15.5:0/3852958 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d48035030 con 0x7f9d54074510 2024-09-10T09:54:26.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:26.684+0000 7f9d5c400640 1 -- 172.21.15.5:0/3852958 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d5419f880 con 0x7f9d54074510 2024-09-10T09:54:26.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:26.684+0000 7f9d5c400640 1 -- 172.21.15.5:0/3852958 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d5419fd10 con 0x7f9d54074510 2024-09-10T09:54:26.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:26.687+0000 7f9d5ee9c740 1 Processor -- start 2024-09-10T09:54:26.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:26.688+0000 7f9d5ee9c740 1 -- start start 2024-09-10T09:54:26.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:26.688+0000 7f9d52000640 1 -- 172.21.15.5:0/3852958 <== mon.2 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f9d48042070 con 0x7f9d54074510 2024-09-10T09:54:26.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:26.688+0000 7f9d5ee9c740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cea6299c70 0x55cea629a050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:26.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:26.688+0000 7f9d5ee9c740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cea629a590 0x55cea62a4910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:26.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:26.688+0000 7f9d5ee9c740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cea62a4e50 0x55cea62a7240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:26.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:26.688+0000 7f9d5ee9c740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55cea62003e0 con 0x55cea6299c70 2024-09-10T09:54:26.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:26.688+0000 7f9d5ee9c740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55cea61a7d80 con 0x55cea62a4e50 2024-09-10T09:54:26.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:26.688+0000 7f9d5ee9c740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55cea61fc3a0 con 0x55cea629a590 2024-09-10T09:54:26.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:26.688+0000 7f9d5a400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cea6299c70 0x55cea629a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:26.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:26.688+0000 7f9d59000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cea62a4e50 0x55cea62a7240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:26.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:26.688+0000 7f9d59000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cea62a4e50 0x55cea62a7240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:38130/0 (socket says 172.21.15.5:38130) 2024-09-10T09:54:26.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:26.688+0000 7f9d59000640 1 -- 172.21.15.5:0/288688016 learned_addr learned my addr 172.21.15.5:0/288688016 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:54:26.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:26.688+0000 7f9d5a400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cea6299c70 0x55cea629a050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56382/0 (socket says 172.21.15.5:56382) 2024-09-10T09:54:26.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:26.688+0000 7f9d59a00640 1 --2- 172.21.15.5:0/288688016 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cea629a590 0x55cea62a4910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:26.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:26.689+0000 7f9d59000640 1 -- 172.21.15.5:0/288688016 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cea629a590 msgr2=0x55cea62a4910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:54:26.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:26.689+0000 7f9d59000640 1 --2- 172.21.15.5:0/288688016 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cea629a590 0x55cea62a4910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:26.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:26.689+0000 7f9d59000640 1 -- 172.21.15.5:0/288688016 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cea6299c70 msgr2=0x55cea629a050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:54:26.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:26.689+0000 7f9d59000640 1 --2- 172.21.15.5:0/288688016 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cea6299c70 0x55cea629a050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:26.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:26.689+0000 7f9d59000640 1 -- 172.21.15.5:0/288688016 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cea61c6ae0 con 0x55cea62a4e50 2024-09-10T09:54:26.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:26.689+0000 7f9d5a400640 1 --2- 172.21.15.5:0/288688016 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cea6299c70 0x55cea629a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:54:26.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:26.689+0000 7f9d59000640 1 --2- 172.21.15.5:0/288688016 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cea62a4e50 0x55cea62a7240 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f9d4000cc40 tx=0x7f9d4000e720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b0bf188d60d043e4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:54:26.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:26.689+0000 7f9d46a00640 1 -- 172.21.15.5:0/288688016 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d40014070 con 0x55cea62a4e50 2024-09-10T09:54:26.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:26.689+0000 7f9d46a00640 1 -- 172.21.15.5:0/288688016 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9d400047a0 con 0x55cea62a4e50 2024-09-10T09:54:26.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:26.689+0000 7f9d46a00640 1 -- 172.21.15.5:0/288688016 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d40010760 con 0x55cea62a4e50 2024-09-10T09:54:26.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:26.690+0000 7f9d52000640 1 --2- 172.21.15.5:0/3852958 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f9d3c0a2c00 0x7f9d3c0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:26.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:26.690+0000 7f9d52000640 1 -- 172.21.15.5:0/3852958 <== mon.2 v2:172.21.15.102:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f9d480efb40 con 0x7f9d54074510 2024-09-10T09:54:26.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:26.690+0000 7f9d5a400640 1 --2- 172.21.15.5:0/3852958 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f9d3c0a2c00 0x7f9d3c0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:26.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:26.690+0000 7f9d5ee9c740 1 -- 172.21.15.5:0/288688016 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cea62a4e50 msgr2=0x55cea62a7240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:26.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:26.690+0000 7f9d5ee9c740 1 --2- 172.21.15.5:0/288688016 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cea62a4e50 0x55cea62a7240 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f9d4000cc40 tx=0x7f9d4000e720 comp rx=0 tx=0).stop 2024-09-10T09:54:26.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:26.690+0000 7f9d5ee9c740 1 -- 172.21.15.5:0/288688016 shutdown_connections 2024-09-10T09:54:26.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:26.690+0000 7f9d5ee9c740 1 --2- 172.21.15.5:0/288688016 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cea62a4e50 0x55cea62a7240 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:26.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:26.690+0000 7f9d5ee9c740 1 --2- 172.21.15.5:0/288688016 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cea629a590 0x55cea62a4910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:26.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:26.690+0000 7f9d5ee9c740 1 --2- 172.21.15.5:0/288688016 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cea6299c70 0x55cea629a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:26.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:26.690+0000 7f9d5ee9c740 1 -- 172.21.15.5:0/288688016 >> 172.21.15.5:0/288688016 conn(0x55cea6296200 msgr2=0x55cea62965e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:54:26.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:26.691+0000 7f9d5ee9c740 1 -- 172.21.15.5:0/288688016 shutdown_connections 2024-09-10T09:54:26.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:26.691+0000 7f9d5ee9c740 1 -- 172.21.15.5:0/288688016 wait complete. 2024-09-10T09:54:26.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:26.691+0000 7f9d5a400640 1 --2- 172.21.15.5:0/3852958 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f9d3c0a2c00 0x7f9d3c0a50c0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f9d4c009d40 tx=0x7f9d4c009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:54:26.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:26.691+0000 7f9d5ee9c740 1 -- start start 2024-09-10T09:54:26.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:26.691+0000 7f9d5ee9c740 1 Processor -- start 2024-09-10T09:54:26.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:26.692+0000 7f9d5ee9c740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cea6299c70 0x55cea62a1c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:26.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:26.692+0000 7f9d5ee9c740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cea629a590 0x55cea62a21c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:26.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:26.692+0000 7f9d5ee9c740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cea62a4e50 0x55cea62a2700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:26.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:26.692+0000 7f9d5ee9c740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55cea62a7a20 con 0x55cea62a4e50 2024-09-10T09:54:26.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:26.692+0000 7f9d5ee9c740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55cea61bb0f0 con 0x55cea629a590 2024-09-10T09:54:26.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:26.692+0000 7f9d5ee9c740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55cea62a7ba0 con 0x55cea6299c70 2024-09-10T09:54:26.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:26.692+0000 7f9d5ee9c740 10 client.0 _ll_register_callbacks cb 0x55cea61a2bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:54:26.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:26.692+0000 7f9d59a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cea6299c70 0x55cea62a1c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:26.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:26.692+0000 7f9d5a400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cea62a4e50 0x55cea62a2700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:26.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:26.692+0000 7f9d59a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cea6299c70 0x55cea62a1c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:38144/0 (socket says 172.21.15.5:38144) 2024-09-10T09:54:26.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:26.692+0000 7f9d59a00640 1 -- 172.21.15.5:0/3886715580 learned_addr learned my addr 172.21.15.5:0/3886715580 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:54:26.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:26.692+0000 7f9d59000640 1 --2- 172.21.15.5:0/3886715580 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cea629a590 0x55cea62a21c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:26.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:26.693+0000 7f9d59a00640 1 -- 172.21.15.5:0/3886715580 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cea629a590 msgr2=0x55cea62a21c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:54:26.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:26.693+0000 7f9d59a00640 1 --2- 172.21.15.5:0/3886715580 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cea629a590 0x55cea62a21c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:26.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:26.693+0000 7f9d59a00640 1 -- 172.21.15.5:0/3886715580 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cea62a4e50 msgr2=0x55cea62a2700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:54:26.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:26.693+0000 7f9d59a00640 1 --2- 172.21.15.5:0/3886715580 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cea62a4e50 0x55cea62a2700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:26.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:26.693+0000 7f9d59a00640 1 -- 172.21.15.5:0/3886715580 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cea5f713c0 con 0x55cea6299c70 2024-09-10T09:54:26.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:26.693+0000 7f9d59a00640 1 --2- 172.21.15.5:0/3886715580 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cea6299c70 0x55cea62a1c80 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f9d480bbc50 tx=0x7f9d4802a480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2c99fcc6b0dd55ed server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:54:26.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:26.693+0000 7f9d3b400640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-09-10T09:54:26.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:26.693+0000 7f9d3b400640 1 -- 172.21.15.5:0/3886715580 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d48049070 con 0x55cea6299c70 2024-09-10T09:54:26.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:26.693+0000 7f9d3b400640 1 -- 172.21.15.5:0/3886715580 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f9d480b85c0 con 0x55cea6299c70 2024-09-10T09:54:26.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:26.693+0000 7f9d5ee9c740 1 -- 172.21.15.5:0/3886715580 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9d400047a0 con 0x55cea6299c70 2024-09-10T09:54:26.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:26.693+0000 7f9d3b400640 1 -- 172.21.15.5:0/3886715580 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f9d480bc650 con 0x55cea6299c70 2024-09-10T09:54:26.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:26.693+0000 7f9d5ee9c740 10 client.28594 fetch_fsmap learned FSMap version 38 2024-09-10T09:54:26.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:26.694+0000 7f9d5ee9c740 1 -- 172.21.15.5:0/3886715580 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55cea62a7a80 con 0x55cea6299c70 2024-09-10T09:54:26.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:26.694+0000 7f9d3b400640 1 -- 172.21.15.5:0/3886715580 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9d400047a0 con 0x55cea6299c70 2024-09-10T09:54:26.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:26.694+0000 7f9d3b400640 1 -- 172.21.15.5:0/3886715580 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 38) ==== 8386+0+0 (secure 0 0 0) 0x7f9d4803b4f0 con 0x55cea6299c70 2024-09-10T09:54:26.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:26.694+0000 7f9d5ee9c740 10 client.28594 fetch_fsmap finished waiting for FSMap version 38 2024-09-10T09:54:26.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:26.694+0000 7f9d5ee9c740 10 client.28594 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-10T09:54:26.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:26.694+0000 7f9d5ee9c740 20 client.28594 populate_metadata read hostname 'smithi005' 2024-09-10T09:54:26.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:26.694+0000 7f9d5ee9c740 1 --2- 172.21.15.5:0/3886715580 >> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] conn(0x55cea62b4f20 0x55cea62b73a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:26.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:26.694+0000 7f9d5ee9c740 4 client.28594 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:54:26.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:26.694+0000 7f9d5ee9c740 1 -- 172.21.15.5:0/3886715580 --> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cea620d950 con 0x55cea62b4f20 2024-09-10T09:54:26.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:26.695+0000 7f9d59000640 1 --2- 172.21.15.5:0/3886715580 >> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] conn(0x55cea62b4f20 0x55cea62b73a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:26.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:26.695+0000 7f9d59000640 1 --2- 172.21.15.5:0/3886715580 >> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] conn(0x55cea62b4f20 0x55cea62b73a0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=74535d1748bdaedc server_cookie=5840b36c9e85c3fc in_seq=0 out_seq=0 2024-09-10T09:54:26.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:26.695+0000 7f9d3b400640 10 client.28594 ms_handle_connect on v2:172.21.15.159:6836/620627639 2024-09-10T09:54:26.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:26.696+0000 7f9d3b400640 1 -- 172.21.15.5:0/3886715580 <== mds.2 v2:172.21.15.159:6836/620627639 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55cea620d950 con 0x55cea62b4f20 2024-09-10T09:54:26.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:26.696+0000 7f9d3b400640 10 client.28594 handle_command_reply: tid=0 2024-09-10T09:54:26.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:26.696+0000 7f9d5ee9c740 1 client.28594 shutdown 2024-09-10T09:54:26.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:26.696+0000 7f9d5ee9c740 1 -- 172.21.15.5:0/3886715580 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cea6299c70 msgr2=0x55cea62a1c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:26.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:26.696+0000 7f9d5ee9c740 1 --2- 172.21.15.5:0/3886715580 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cea6299c70 0x55cea62a1c80 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f9d480bbc50 tx=0x7f9d4802a480 comp rx=0 tx=0).stop 2024-09-10T09:54:26.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:26.696+0000 7f9d5ee9c740 1 -- 172.21.15.5:0/3886715580 shutdown_connections 2024-09-10T09:54:26.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:26.696+0000 7f9d5ee9c740 1 --2- 172.21.15.5:0/3886715580 >> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] conn(0x55cea62b4f20 0x55cea62b73a0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:26.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:26.696+0000 7f9d5ee9c740 1 --2- 172.21.15.5:0/3886715580 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cea62a4e50 0x55cea62a2700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:26.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:26.696+0000 7f9d5ee9c740 1 --2- 172.21.15.5:0/3886715580 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cea629a590 0x55cea62a21c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:26.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:26.696+0000 7f9d5ee9c740 1 --2- 172.21.15.5:0/3886715580 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cea6299c70 0x55cea62a1c80 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:26.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:26.696+0000 7f9d5ee9c740 1 -- 172.21.15.5:0/3886715580 >> 172.21.15.5:0/3886715580 conn(0x55cea6296200 msgr2=0x55cea629bf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:54:26.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:26.696+0000 7f9d3b400640 0 client.28594 ms_handle_reset on v2:172.21.15.159:6836/620627639 2024-09-10T09:54:26.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:26.697+0000 7f9d5ee9c740 1 -- 172.21.15.5:0/3886715580 shutdown_connections 2024-09-10T09:54:26.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:26.697+0000 7f9d5ee9c740 1 -- 172.21.15.5:0/3886715580 wait complete. 2024-09-10T09:54:26.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:26.697+0000 7f9d5ee9c740 20 client.28594 trim_cache size 0 max 16384 2024-09-10T09:54:26.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:26.711+0000 7f9d5ee9c740 1 Processor -- start 2024-09-10T09:54:26.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:26.711+0000 7f9d5ee9c740 1 -- start start 2024-09-10T09:54:26.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:26.711+0000 7f9d5ee9c740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cea6299c70 0x55cea62bcc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:26.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:26.711+0000 7f9d5ee9c740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cea629a590 0x55cea62bd1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:26.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:26.711+0000 7f9d5ee9c740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cea62a4e50 0x55cea6294b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:26.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:26.711+0000 7f9d5ee9c740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f9d541b1060 con 0x55cea629a590 2024-09-10T09:54:26.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:26.711+0000 7f9d5ee9c740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f9d541b2c60 con 0x55cea62a4e50 2024-09-10T09:54:26.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:26.711+0000 7f9d5ee9c740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f9d541b1730 con 0x55cea6299c70 2024-09-10T09:54:26.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:26.711+0000 7f9d59a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cea6299c70 0x55cea62bcc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:26.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:26.711+0000 7f9d59000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cea629a590 0x55cea62bd1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:26.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:26.711+0000 7f9d59a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cea6299c70 0x55cea62bcc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:38152/0 (socket says 172.21.15.5:38152) 2024-09-10T09:54:26.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:26.711+0000 7f9d59a00640 1 -- 172.21.15.5:0/369213494 learned_addr learned my addr 172.21.15.5:0/369213494 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:54:26.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:26.712+0000 7f9d5a400640 1 --2- 172.21.15.5:0/369213494 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cea62a4e50 0x55cea6294b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:26.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:26.712+0000 7f9d59a00640 1 -- 172.21.15.5:0/369213494 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cea62a4e50 msgr2=0x55cea6294b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:54:26.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:26.712+0000 7f9d59a00640 1 --2- 172.21.15.5:0/369213494 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cea62a4e50 0x55cea6294b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:26.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:26.712+0000 7f9d59a00640 1 -- 172.21.15.5:0/369213494 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cea629a590 msgr2=0x55cea62bd1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:54:26.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:26.712+0000 7f9d59a00640 1 --2- 172.21.15.5:0/369213494 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cea629a590 0x55cea62bd1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:26.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:26.712+0000 7f9d59a00640 1 -- 172.21.15.5:0/369213494 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cea629a3c0 con 0x55cea6299c70 2024-09-10T09:54:26.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:26.712+0000 7f9d59a00640 1 --2- 172.21.15.5:0/369213494 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cea6299c70 0x55cea62bcc70 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f9d48032fd0 tx=0x7f9d480b81f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c20071954e436b8e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:54:26.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:26.712+0000 7f9d46000640 1 -- 172.21.15.5:0/369213494 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d480d4030 con 0x55cea6299c70 2024-09-10T09:54:26.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:26.712+0000 7f9d46000640 1 -- 172.21.15.5:0/369213494 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9d480c4bc0 con 0x55cea6299c70 2024-09-10T09:54:26.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:26.712+0000 7f9d46000640 1 -- 172.21.15.5:0/369213494 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d48049070 con 0x55cea6299c70 2024-09-10T09:54:26.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:26.713+0000 7f9d5ee9c740 1 -- 172.21.15.5:0/369213494 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cea6299c70 msgr2=0x55cea62bcc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:26.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:26.713+0000 7f9d5ee9c740 1 --2- 172.21.15.5:0/369213494 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cea6299c70 0x55cea62bcc70 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f9d48032fd0 tx=0x7f9d480b81f0 comp rx=0 tx=0).stop 2024-09-10T09:54:26.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:26.713+0000 7f9d5ee9c740 1 -- 172.21.15.5:0/369213494 shutdown_connections 2024-09-10T09:54:26.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:26.713+0000 7f9d5ee9c740 1 --2- 172.21.15.5:0/369213494 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cea62a4e50 0x55cea6294b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:26.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:26.713+0000 7f9d5ee9c740 1 --2- 172.21.15.5:0/369213494 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cea629a590 0x55cea62bd1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:26.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:26.713+0000 7f9d5ee9c740 1 --2- 172.21.15.5:0/369213494 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cea6299c70 0x55cea62bcc70 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:26.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:26.713+0000 7f9d5ee9c740 1 -- 172.21.15.5:0/369213494 >> 172.21.15.5:0/369213494 conn(0x55cea6296200 msgr2=0x55cea62a0d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:54:26.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:26.713+0000 7f9d5ee9c740 1 -- 172.21.15.5:0/369213494 shutdown_connections 2024-09-10T09:54:26.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:26.713+0000 7f9d5ee9c740 1 -- 172.21.15.5:0/369213494 wait complete. 2024-09-10T09:54:26.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:26.714+0000 7f9d5ee9c740 1 -- start start 2024-09-10T09:54:26.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:26.714+0000 7f9d5ee9c740 1 Processor -- start 2024-09-10T09:54:26.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:26.714+0000 7f9d5ee9c740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cea6299c70 0x55cea62ad2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:26.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:26.714+0000 7f9d5ee9c740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cea629a590 0x55cea62a8620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:26.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:26.714+0000 7f9d5ee9c740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cea62a4e50 0x55cea62a8b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:26.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:26.714+0000 7f9d5ee9c740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f9d541b2590 con 0x55cea629a590 2024-09-10T09:54:26.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:26.714+0000 7f9d5ee9c740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f9d541b1ec0 con 0x55cea6299c70 2024-09-10T09:54:26.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:26.714+0000 7f9d5ee9c740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f9d541af040 con 0x55cea62a4e50 2024-09-10T09:54:26.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:26.714+0000 7f9d5ee9c740 10 client.0 _ll_register_callbacks cb 0x7f9d48009260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:54:26.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:26.715+0000 7f9d59a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cea6299c70 0x55cea62ad2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:26.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:26.715+0000 7f9d59a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cea6299c70 0x55cea62ad2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:38162/0 (socket says 172.21.15.5:38162) 2024-09-10T09:54:26.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:26.715+0000 7f9d59a00640 1 -- 172.21.15.5:0/1248575612 learned_addr learned my addr 172.21.15.5:0/1248575612 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:54:26.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:26.715+0000 7f9d5a400640 1 --2- 172.21.15.5:0/1248575612 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cea62a4e50 0x55cea62a8b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:26.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:26.715+0000 7f9d59000640 1 --2- 172.21.15.5:0/1248575612 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cea629a590 0x55cea62a8620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:26.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:26.715+0000 7f9d59a00640 1 -- 172.21.15.5:0/1248575612 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cea62a4e50 msgr2=0x55cea62a8b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:54:26.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:26.715+0000 7f9d59a00640 1 --2- 172.21.15.5:0/1248575612 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cea62a4e50 0x55cea62a8b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:26.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:26.715+0000 7f9d59a00640 1 -- 172.21.15.5:0/1248575612 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cea629a590 msgr2=0x55cea62a8620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:54:26.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:26.715+0000 7f9d59a00640 1 --2- 172.21.15.5:0/1248575612 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cea629a590 0x55cea62a8620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:26.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:26.715+0000 7f9d59a00640 1 -- 172.21.15.5:0/1248575612 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cea62a90a0 con 0x55cea6299c70 2024-09-10T09:54:26.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:26.715+0000 7f9d59000640 1 --2- 172.21.15.5:0/1248575612 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cea629a590 0x55cea62a8620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T09:54:26.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:26.715+0000 7f9d59a00640 1 --2- 172.21.15.5:0/1248575612 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cea6299c70 0x55cea62ad2f0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f9d480b83b0 tx=0x7f9d48005e80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9d2c60c22459ec7d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:54:26.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:26.715+0000 7f9d3b400640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-09-10T09:54:26.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:26.715+0000 7f9d3b400640 1 -- 172.21.15.5:0/1248575612 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d48049030 con 0x55cea6299c70 2024-09-10T09:54:26.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:26.715+0000 7f9d3b400640 1 -- 172.21.15.5:0/1248575612 <== mon.2 v2:172.21.15.102:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f9d480bb5a0 con 0x55cea6299c70 2024-09-10T09:54:26.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:26.715+0000 7f9d5ee9c740 1 -- 172.21.15.5:0/1248575612 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9d480c4bc0 con 0x55cea6299c70 2024-09-10T09:54:26.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:26.716+0000 7f9d3b400640 1 -- 172.21.15.5:0/1248575612 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f9d480f6030 con 0x55cea6299c70 2024-09-10T09:54:26.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:26.716+0000 7f9d5ee9c740 10 client.28601 fetch_fsmap learned FSMap version 38 2024-09-10T09:54:26.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:26.716+0000 7f9d5ee9c740 1 -- 172.21.15.5:0/1248575612 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55cea62a90a0 con 0x55cea6299c70 2024-09-10T09:54:26.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:26.716+0000 7f9d3b400640 1 -- 172.21.15.5:0/1248575612 <== mon.2 v2:172.21.15.102:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9d480c4bc0 con 0x55cea6299c70 2024-09-10T09:54:26.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:26.716+0000 7f9d3b400640 1 -- 172.21.15.5:0/1248575612 <== mon.2 v2:172.21.15.102:3300/0 5 ==== fsmap(e 38) ==== 8386+0+0 (secure 0 0 0) 0x7f9d480b6ac0 con 0x55cea6299c70 2024-09-10T09:54:26.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:26.716+0000 7f9d5ee9c740 10 client.28601 fetch_fsmap finished waiting for FSMap version 38 2024-09-10T09:54:26.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:26.716+0000 7f9d5ee9c740 10 client.28601 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-10T09:54:26.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:26.716+0000 7f9d5ee9c740 20 client.28601 populate_metadata read hostname 'smithi005' 2024-09-10T09:54:26.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:26.716+0000 7f9d5ee9c740 1 --2- 172.21.15.5:0/1248575612 >> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] conn(0x55cea631fcd0 0x55cea63220f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:26.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:26.716+0000 7f9d5ee9c740 4 client.28601 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T09:54:26.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:26.716+0000 7f9d5ee9c740 1 -- 172.21.15.5:0/1248575612 --> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55cea6207e60 con 0x55cea631fcd0 2024-09-10T09:54:26.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:26.717+0000 7f9d59000640 1 --2- 172.21.15.5:0/1248575612 >> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] conn(0x55cea631fcd0 0x55cea63220f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:26.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:26.717+0000 7f9d59000640 1 --2- 172.21.15.5:0/1248575612 >> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] conn(0x55cea631fcd0 0x55cea63220f0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=be7d576433f68123 server_cookie=fe1300e32af68304 in_seq=0 out_seq=0 2024-09-10T09:54:26.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:26.717+0000 7f9d3b400640 10 client.28601 ms_handle_connect on v2:172.21.15.159:6836/620627639 2024-09-10T09:54:26.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:26.718+0000 7f9d3b400640 1 -- 172.21.15.5:0/1248575612 <== mds.2 v2:172.21.15.159:6836/620627639 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55cea6207e60 con 0x55cea631fcd0 2024-09-10T09:54:26.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:26.718+0000 7f9d3b400640 10 client.28601 handle_command_reply: tid=0 2024-09-10T09:54:26.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:26.718+0000 7f9d5ee9c740 1 client.28601 shutdown 2024-09-10T09:54:26.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:26.718+0000 7f9d5ee9c740 1 -- 172.21.15.5:0/1248575612 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cea6299c70 msgr2=0x55cea62ad2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:26.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:26.718+0000 7f9d5ee9c740 1 --2- 172.21.15.5:0/1248575612 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cea6299c70 0x55cea62ad2f0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f9d480b83b0 tx=0x7f9d48005e80 comp rx=0 tx=0).stop 2024-09-10T09:54:26.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:26.718+0000 7f9d5ee9c740 1 -- 172.21.15.5:0/1248575612 shutdown_connections 2024-09-10T09:54:26.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:26.718+0000 7f9d5ee9c740 1 --2- 172.21.15.5:0/1248575612 >> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] conn(0x55cea631fcd0 0x55cea63220f0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:26.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:26.718+0000 7f9d5ee9c740 1 --2- 172.21.15.5:0/1248575612 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cea62a4e50 0x55cea62a8b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:26.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:26.718+0000 7f9d5ee9c740 1 --2- 172.21.15.5:0/1248575612 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cea629a590 0x55cea62a8620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:26.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:26.718+0000 7f9d5ee9c740 1 --2- 172.21.15.5:0/1248575612 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cea6299c70 0x55cea62ad2f0 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:26.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:26.718+0000 7f9d3b400640 0 client.28601 ms_handle_reset on v2:172.21.15.159:6836/620627639 2024-09-10T09:54:26.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:26.718+0000 7f9d5ee9c740 1 -- 172.21.15.5:0/1248575612 >> 172.21.15.5:0/1248575612 conn(0x55cea6296200 msgr2=0x55cea6294b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:54:26.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:26.718+0000 7f9d5ee9c740 1 -- 172.21.15.5:0/1248575612 shutdown_connections 2024-09-10T09:54:26.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:26.718+0000 7f9d5ee9c740 1 -- 172.21.15.5:0/1248575612 wait complete. 2024-09-10T09:54:26.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:26.718+0000 7f9d5ee9c740 20 client.28601 trim_cache size 0 max 16384 2024-09-10T09:54:26.725 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-10T09:54:26.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:26.719+0000 7f9d5c400640 1 -- 172.21.15.5:0/3852958 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f9d3c0a2c00 msgr2=0x7f9d3c0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:26.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:26.719+0000 7f9d5c400640 1 --2- 172.21.15.5:0/3852958 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f9d3c0a2c00 0x7f9d3c0a50c0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f9d4c009d40 tx=0x7f9d4c009500 comp rx=0 tx=0).stop 2024-09-10T09:54:26.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:26.719+0000 7f9d5c400640 1 -- 172.21.15.5:0/3852958 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9d54074510 msgr2=0x7f9d5419a640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:26.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:26.719+0000 7f9d5c400640 1 --2- 172.21.15.5:0/3852958 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9d54074510 0x7f9d5419a640 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f9d48004770 tx=0x7f9d48003c10 comp rx=0 tx=0).stop 2024-09-10T09:54:26.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:26.719+0000 7f9d5c400640 1 -- 172.21.15.5:0/3852958 shutdown_connections 2024-09-10T09:54:26.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:26.719+0000 7f9d5c400640 1 --2- 172.21.15.5:0/3852958 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f9d3c0a2c00 0x7f9d3c0a50c0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:26.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:26.719+0000 7f9d5c400640 1 --2- 172.21.15.5:0/3852958 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9d5410e440 0x7f9d5419ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:26.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:26.719+0000 7f9d5c400640 1 --2- 172.21.15.5:0/3852958 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9d54074e30 0x7f9d5419ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:26.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:26.719+0000 7f9d5c400640 1 --2- 172.21.15.5:0/3852958 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9d54074510 0x7f9d5419a640 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:26.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:26.719+0000 7f9d5c400640 1 -- 172.21.15.5:0/3852958 >> 172.21.15.5:0/3852958 conn(0x7f9d540fd8c0 msgr2=0x7f9d540fe520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:54:26.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:26.719+0000 7f9d5c400640 1 -- 172.21.15.5:0/3852958 shutdown_connections 2024-09-10T09:54:26.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:26.719+0000 7f9d5c400640 1 -- 172.21.15.5:0/3852958 wait complete. 2024-09-10T09:54:26.965 INFO:tasks.workunit.client.0.smithi005.stdout: 10 41284 22.33 MB/sec execute 5 sec latency 987.289 ms 2024-09-10T09:54:27.703 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '87c9f91d', quiesce took 10.6 seconds. Will release after: 9.250000000000002 2024-09-10T09:54:27.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:54:27 smithi159 ceph-mon[35513]: pgmap v876: 129 pgs: 129 active+clean; 539 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 700 KiB/s rd, 2.5 MiB/s wr, 185 op/s 2024-09-10T09:54:27.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:54:27 smithi102 ceph-mon[35442]: pgmap v876: 129 pgs: 129 active+clean; 539 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 700 KiB/s rd, 2.5 MiB/s wr, 185 op/s 2024-09-10T09:54:27.965 INFO:tasks.workunit.client.0.smithi005.stdout: 10 41284 18.60 MB/sec execute 6 sec latency 1987.403 ms 2024-09-10T09:54:28.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:54:27 smithi005 ceph-mon[30350]: pgmap v876: 129 pgs: 129 active+clean; 539 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 700 KiB/s rd, 2.5 MiB/s wr, 185 op/s 2024-09-10T09:54:28.470 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-10T09:54:28.848 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:54:28.965 INFO:tasks.workunit.client.0.smithi005.stdout: 10 41284 15.95 MB/sec execute 7 sec latency 2987.557 ms 2024-09-10T09:54:29.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:29.444+0000 7fc060600640 1 -- 172.21.15.5:0/2636289043 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc058109120 msgr2=0x7fc058110ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:29.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:29.444+0000 7fc060600640 1 --2- 172.21.15.5:0/2636289043 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc058109120 0x7fc058110ca0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fc050009900 tx=0x7fc05002a410 comp rx=0 tx=0).stop 2024-09-10T09:54:29.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:29.445+0000 7fc060600640 1 -- 172.21.15.5:0/2636289043 shutdown_connections 2024-09-10T09:54:29.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:29.445+0000 7fc060600640 1 --2- 172.21.15.5:0/2636289043 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc058109120 0x7fc058110ca0 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:29.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:29.445+0000 7fc060600640 1 --2- 172.21.15.5:0/2636289043 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc0581049a0 0x7fc0581089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:29.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:29.445+0000 7fc060600640 1 --2- 172.21.15.5:0/2636289043 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc058103ff0 0x7fc0581043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:29.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:29.445+0000 7fc060600640 1 -- 172.21.15.5:0/2636289043 >> 172.21.15.5:0/2636289043 conn(0x7fc0580ffa90 msgr2=0x7fc058101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:54:29.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:29.445+0000 7fc060600640 1 -- 172.21.15.5:0/2636289043 shutdown_connections 2024-09-10T09:54:29.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:29.445+0000 7fc060600640 1 -- 172.21.15.5:0/2636289043 wait complete. 2024-09-10T09:54:29.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:29.445+0000 7fc060600640 1 Processor -- start 2024-09-10T09:54:29.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:29.446+0000 7fc060600640 1 -- start start 2024-09-10T09:54:29.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:29.446+0000 7fc060600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc058103ff0 0x7fc05819a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:29.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:29.446+0000 7fc060600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc0581049a0 0x7fc05819aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:29.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:29.446+0000 7fc060600640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc058109120 0x7fc05819ee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:29.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:29.446+0000 7fc060600640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc058114130 con 0x7fc058103ff0 2024-09-10T09:54:29.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:29.446+0000 7fc060600640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fc058113fb0 con 0x7fc0581049a0 2024-09-10T09:54:29.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:29.446+0000 7fc060600640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fc0581142b0 con 0x7fc058109120 2024-09-10T09:54:29.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:29.446+0000 7fc05d200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc0581049a0 0x7fc05819aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:29.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:29.446+0000 7fc05d200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc0581049a0 0x7fc05819aad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:38188/0 (socket says 172.21.15.5:38188) 2024-09-10T09:54:29.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:29.446+0000 7fc05d200640 1 -- 172.21.15.5:0/3057141117 learned_addr learned my addr 172.21.15.5:0/3057141117 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:54:29.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:29.446+0000 7fc05dc00640 1 --2- 172.21.15.5:0/3057141117 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc058103ff0 0x7fc05819a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:29.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:29.447+0000 7fc05e600640 1 --2- 172.21.15.5:0/3057141117 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc058109120 0x7fc05819ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:29.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:29.447+0000 7fc05d200640 1 -- 172.21.15.5:0/3057141117 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc058109120 msgr2=0x7fc05819ee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:29.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:29.447+0000 7fc05d200640 1 --2- 172.21.15.5:0/3057141117 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc058109120 0x7fc05819ee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:29.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:29.447+0000 7fc05d200640 1 -- 172.21.15.5:0/3057141117 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc058103ff0 msgr2=0x7fc05819a590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:29.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:29.447+0000 7fc05d200640 1 --2- 172.21.15.5:0/3057141117 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc058103ff0 0x7fc05819a590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:29.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:29.447+0000 7fc05d200640 1 -- 172.21.15.5:0/3057141117 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc05819f540 con 0x7fc0581049a0 2024-09-10T09:54:29.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:29.447+0000 7fc05e600640 1 --2- 172.21.15.5:0/3057141117 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc058109120 0x7fc05819ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:54:29.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:29.447+0000 7fc05d200640 1 --2- 172.21.15.5:0/3057141117 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc0581049a0 0x7fc05819aad0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fc04400e9f0 tx=0x7fc04400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:54:29.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:29.447+0000 7fc056000640 1 -- 172.21.15.5:0/3057141117 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc04400cd80 con 0x7fc0581049a0 2024-09-10T09:54:29.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:29.447+0000 7fc056000640 1 -- 172.21.15.5:0/3057141117 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc044010910 con 0x7fc0581049a0 2024-09-10T09:54:29.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:29.447+0000 7fc056000640 1 -- 172.21.15.5:0/3057141117 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc044018990 con 0x7fc0581049a0 2024-09-10T09:54:29.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:29.447+0000 7fc060600640 1 -- 172.21.15.5:0/3057141117 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc05819f830 con 0x7fc0581049a0 2024-09-10T09:54:29.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:29.448+0000 7fc060600640 1 -- 172.21.15.5:0/3057141117 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0581a7110 con 0x7fc0581049a0 2024-09-10T09:54:29.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:29.451+0000 7fc0630d5740 1 Processor -- start 2024-09-10T09:54:29.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:29.451+0000 7fc0630d5740 1 -- start start 2024-09-10T09:54:29.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:29.451+0000 7fc0630d5740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558e25933c70 0x558e25934050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:29.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:29.451+0000 7fc0630d5740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558e25934590 0x558e2593e910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:29.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:29.451+0000 7fc0630d5740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558e2593ee50 0x558e25941240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:29.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:29.451+0000 7fc0630d5740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x558e2589a3e0 con 0x558e25933c70 2024-09-10T09:54:29.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:29.451+0000 7fc0630d5740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x558e25841d80 con 0x558e25934590 2024-09-10T09:54:29.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:29.451+0000 7fc0630d5740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x558e258963a0 con 0x558e2593ee50 2024-09-10T09:54:29.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:29.451+0000 7fc056000640 1 -- 172.21.15.5:0/3057141117 <== mon.2 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7fc044010ac0 con 0x7fc0581049a0 2024-09-10T09:54:29.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:29.451+0000 7fc05dc00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558e25934590 0x558e2593e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:29.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:29.451+0000 7fc05e600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558e25933c70 0x558e25934050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:29.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:29.451+0000 7fc05dc00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558e25934590 0x558e2593e910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:38198/0 (socket says 172.21.15.5:38198) 2024-09-10T09:54:29.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:29.452+0000 7fc05dc00640 1 -- 172.21.15.5:0/181551889 learned_addr learned my addr 172.21.15.5:0/181551889 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:54:29.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:29.452+0000 7fc05d200640 1 --2- 172.21.15.5:0/181551889 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558e2593ee50 0x558e25941240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:29.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:29.452+0000 7fc05dc00640 1 -- 172.21.15.5:0/181551889 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558e2593ee50 msgr2=0x558e25941240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:54:29.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:29.452+0000 7fc05dc00640 1 --2- 172.21.15.5:0/181551889 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558e2593ee50 0x558e25941240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:29.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:29.452+0000 7fc05dc00640 1 -- 172.21.15.5:0/181551889 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558e25933c70 msgr2=0x558e25934050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:54:29.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:29.452+0000 7fc05dc00640 1 --2- 172.21.15.5:0/181551889 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558e25933c70 0x558e25934050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:29.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:29.452+0000 7fc05dc00640 1 -- 172.21.15.5:0/181551889 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558e25860ae0 con 0x558e25934590 2024-09-10T09:54:29.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:29.452+0000 7fc05dc00640 1 --2- 172.21.15.5:0/181551889 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558e25934590 0x558e2593e910 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fc04c010480 tx=0x7fc04c010940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=17b028967db118f4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:54:29.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:29.452+0000 7fc04aa00640 1 -- 172.21.15.5:0/181551889 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc04c017070 con 0x558e25934590 2024-09-10T09:54:29.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:29.453+0000 7fc04aa00640 1 -- 172.21.15.5:0/181551889 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc04c011c40 con 0x558e25934590 2024-09-10T09:54:29.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:29.453+0000 7fc04aa00640 1 -- 172.21.15.5:0/181551889 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc04c01bd60 con 0x558e25934590 2024-09-10T09:54:29.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:29.453+0000 7fc056000640 1 --2- 172.21.15.5:0/3057141117 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fc0400a2b80 0x7fc0400a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:29.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:29.453+0000 7fc056000640 1 -- 172.21.15.5:0/3057141117 <== mon.2 v2:172.21.15.102:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fc0440cf8e0 con 0x7fc0581049a0 2024-09-10T09:54:29.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:29.453+0000 7fc05e600640 1 --2- 172.21.15.5:0/3057141117 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fc0400a2b80 0x7fc0400a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:29.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:29.453+0000 7fc0630d5740 1 -- 172.21.15.5:0/181551889 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558e25934590 msgr2=0x558e2593e910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:29.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:29.453+0000 7fc0630d5740 1 --2- 172.21.15.5:0/181551889 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558e25934590 0x558e2593e910 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fc04c010480 tx=0x7fc04c010940 comp rx=0 tx=0).stop 2024-09-10T09:54:29.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:29.454+0000 7fc0630d5740 1 -- 172.21.15.5:0/181551889 shutdown_connections 2024-09-10T09:54:29.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:29.454+0000 7fc0630d5740 1 --2- 172.21.15.5:0/181551889 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558e2593ee50 0x558e25941240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:29.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:29.454+0000 7fc0630d5740 1 --2- 172.21.15.5:0/181551889 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558e25934590 0x558e2593e910 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:29.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:29.454+0000 7fc0630d5740 1 --2- 172.21.15.5:0/181551889 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558e25933c70 0x558e25934050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:29.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:29.454+0000 7fc0630d5740 1 -- 172.21.15.5:0/181551889 >> 172.21.15.5:0/181551889 conn(0x558e25930200 msgr2=0x558e259305e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:54:29.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:29.454+0000 7fc0630d5740 1 -- 172.21.15.5:0/181551889 shutdown_connections 2024-09-10T09:54:29.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:29.454+0000 7fc0630d5740 1 -- 172.21.15.5:0/181551889 wait complete. 2024-09-10T09:54:29.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:29.454+0000 7fc05e600640 1 --2- 172.21.15.5:0/3057141117 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fc0400a2b80 0x7fc0400a5040 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fc050005940 tx=0x7fc0500058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:54:29.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:29.454+0000 7fc0630d5740 1 -- start start 2024-09-10T09:54:29.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:29.455+0000 7fc0630d5740 1 Processor -- start 2024-09-10T09:54:29.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:29.455+0000 7fc0630d5740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558e25933c70 0x558e2593bc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:29.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:29.455+0000 7fc0630d5740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558e25934590 0x558e2593c190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:29.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:29.455+0000 7fc0630d5740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558e2593ee50 0x558e2593c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:29.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:29.455+0000 7fc0630d5740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x558e25941a20 con 0x558e2593ee50 2024-09-10T09:54:29.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:29.455+0000 7fc0630d5740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x558e258550f0 con 0x558e25934590 2024-09-10T09:54:29.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:29.455+0000 7fc0630d5740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x558e25941ba0 con 0x558e25933c70 2024-09-10T09:54:29.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:29.455+0000 7fc0630d5740 10 client.0 _ll_register_callbacks cb 0x558e2583cbd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:54:29.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:29.455+0000 7fc05e600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558e2593ee50 0x558e2593c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:29.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:29.456+0000 7fc05d200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558e25934590 0x558e2593c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:29.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:29.456+0000 7fc05e600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558e2593ee50 0x558e2593c6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56428/0 (socket says 172.21.15.5:56428) 2024-09-10T09:54:29.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:29.456+0000 7fc05e600640 1 -- 172.21.15.5:0/1776734746 learned_addr learned my addr 172.21.15.5:0/1776734746 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:54:29.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:29.456+0000 7fc05dc00640 1 --2- 172.21.15.5:0/1776734746 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558e25933c70 0x558e2593bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:29.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:29.456+0000 7fc05d200640 1 -- 172.21.15.5:0/1776734746 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558e25933c70 msgr2=0x558e2593bc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:54:29.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:29.456+0000 7fc05d200640 1 --2- 172.21.15.5:0/1776734746 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558e25933c70 0x558e2593bc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:29.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:29.456+0000 7fc05d200640 1 -- 172.21.15.5:0/1776734746 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558e2593ee50 msgr2=0x558e2593c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:54:29.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:29.456+0000 7fc05d200640 1 --2- 172.21.15.5:0/1776734746 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558e2593ee50 0x558e2593c6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:29.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:29.456+0000 7fc05d200640 1 -- 172.21.15.5:0/1776734746 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558e2560b3c0 con 0x558e25934590 2024-09-10T09:54:29.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:29.456+0000 7fc05d200640 1 --2- 172.21.15.5:0/1776734746 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558e25934590 0x558e2593c190 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fc0440cd040 tx=0x7fc04409b950 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f5c457d883b8fc50 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:54:29.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:29.456+0000 7fc03f400640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-09-10T09:54:29.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:29.456+0000 7fc03f400640 1 -- 172.21.15.5:0/1776734746 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0440a0100 con 0x558e25934590 2024-09-10T09:54:29.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:29.456+0000 7fc03f400640 1 -- 172.21.15.5:0/1776734746 <== mon.2 v2:172.21.15.102:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fc044018b30 con 0x558e25934590 2024-09-10T09:54:29.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:29.456+0000 7fc0630d5740 1 -- 172.21.15.5:0/1776734746 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc04c011c40 con 0x558e25934590 2024-09-10T09:54:29.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:29.457+0000 7fc03f400640 1 -- 172.21.15.5:0/1776734746 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7fc0440a49c0 con 0x558e25934590 2024-09-10T09:54:29.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:29.457+0000 7fc0630d5740 10 client.28613 fetch_fsmap learned FSMap version 38 2024-09-10T09:54:29.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:29.457+0000 7fc0630d5740 1 -- 172.21.15.5:0/1776734746 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x558e2594d980 con 0x558e25934590 2024-09-10T09:54:29.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:29.457+0000 7fc03f400640 1 -- 172.21.15.5:0/1776734746 <== mon.2 v2:172.21.15.102:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc04c011c40 con 0x558e25934590 2024-09-10T09:54:29.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:29.457+0000 7fc03f400640 1 -- 172.21.15.5:0/1776734746 <== mon.2 v2:172.21.15.102:3300/0 5 ==== fsmap(e 38) ==== 8386+0+0 (secure 0 0 0) 0x7fc044097aa0 con 0x558e25934590 2024-09-10T09:54:29.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:29.457+0000 7fc0630d5740 10 client.28613 fetch_fsmap finished waiting for FSMap version 38 2024-09-10T09:54:29.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:29.457+0000 7fc0630d5740 10 client.28613 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-10T09:54:29.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:29.457+0000 7fc0630d5740 20 client.28613 populate_metadata read hostname 'smithi005' 2024-09-10T09:54:29.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:29.457+0000 7fc0630d5740 1 --2- 172.21.15.5:0/1776734746 >> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] conn(0x558e2594ed90 0x558e25951180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:29.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:29.457+0000 7fc0630d5740 4 client.28613 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:54:29.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:29.457+0000 7fc0630d5740 1 -- 172.21.15.5:0/1776734746 --> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558e258a7950 con 0x558e2594ed90 2024-09-10T09:54:29.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:29.458+0000 7fc05dc00640 1 --2- 172.21.15.5:0/1776734746 >> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] conn(0x558e2594ed90 0x558e25951180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:29.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:29.458+0000 7fc05dc00640 1 --2- 172.21.15.5:0/1776734746 >> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] conn(0x558e2594ed90 0x558e25951180 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=80ea19938ec42eea server_cookie=dc7fc7e5203dab82 in_seq=0 out_seq=0 2024-09-10T09:54:29.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:29.458+0000 7fc03f400640 10 client.28613 ms_handle_connect on v2:172.21.15.159:6838/2073739599 2024-09-10T09:54:29.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:29.459+0000 7fc03f400640 1 -- 172.21.15.5:0/1776734746 <== mds.1 v2:172.21.15.159:6838/2073739599 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558e258a7950 con 0x558e2594ed90 2024-09-10T09:54:29.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:29.459+0000 7fc03f400640 10 client.28613 handle_command_reply: tid=0 2024-09-10T09:54:29.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:29.459+0000 7fc0630d5740 1 client.28613 shutdown 2024-09-10T09:54:29.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:29.459+0000 7fc0630d5740 1 -- 172.21.15.5:0/1776734746 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558e25934590 msgr2=0x558e2593c190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:29.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:29.459+0000 7fc0630d5740 1 --2- 172.21.15.5:0/1776734746 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558e25934590 0x558e2593c190 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fc0440cd040 tx=0x7fc04409b950 comp rx=0 tx=0).stop 2024-09-10T09:54:29.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:29.459+0000 7fc0630d5740 1 -- 172.21.15.5:0/1776734746 shutdown_connections 2024-09-10T09:54:29.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:29.459+0000 7fc0630d5740 1 --2- 172.21.15.5:0/1776734746 >> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] conn(0x558e2594ed90 0x558e25951180 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:29.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:29.459+0000 7fc0630d5740 1 --2- 172.21.15.5:0/1776734746 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558e2593ee50 0x558e2593c6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:29.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:29.459+0000 7fc0630d5740 1 --2- 172.21.15.5:0/1776734746 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558e25934590 0x558e2593c190 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:29.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:29.459+0000 7fc0630d5740 1 --2- 172.21.15.5:0/1776734746 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558e25933c70 0x558e2593bc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:29.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:29.459+0000 7fc03f400640 0 client.28613 ms_handle_reset on v2:172.21.15.159:6838/2073739599 2024-09-10T09:54:29.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:29.459+0000 7fc0630d5740 1 -- 172.21.15.5:0/1776734746 >> 172.21.15.5:0/1776734746 conn(0x558e25930200 msgr2=0x558e2592f380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:54:29.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:29.460+0000 7fc0630d5740 1 -- 172.21.15.5:0/1776734746 shutdown_connections 2024-09-10T09:54:29.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:29.460+0000 7fc0630d5740 1 -- 172.21.15.5:0/1776734746 wait complete. 2024-09-10T09:54:29.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:29.460+0000 7fc0630d5740 20 client.28613 trim_cache size 0 max 16384 2024-09-10T09:54:29.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:29.473+0000 7fc0630d5740 1 Processor -- start 2024-09-10T09:54:29.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:29.474+0000 7fc0630d5740 1 -- start start 2024-09-10T09:54:29.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:29.474+0000 7fc0630d5740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558e25933c70 0x558e25956a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:29.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:29.474+0000 7fc0630d5740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558e25934590 0x558e25956f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:29.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:29.474+0000 7fc0630d5740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558e2593ee50 0x558e25938eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:29.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:29.474+0000 7fc0630d5740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc0581b10e0 con 0x558e2593ee50 2024-09-10T09:54:29.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:29.474+0000 7fc0630d5740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fc0581b2ce0 con 0x558e25934590 2024-09-10T09:54:29.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:29.474+0000 7fc0630d5740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fc0581b17b0 con 0x558e25933c70 2024-09-10T09:54:29.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:29.474+0000 7fc05e600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558e2593ee50 0x558e25938eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:29.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:29.474+0000 7fc05e600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558e2593ee50 0x558e25938eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56436/0 (socket says 172.21.15.5:56436) 2024-09-10T09:54:29.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:29.474+0000 7fc05e600640 1 -- 172.21.15.5:0/1444845451 learned_addr learned my addr 172.21.15.5:0/1444845451 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:54:29.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:29.474+0000 7fc05dc00640 1 --2- 172.21.15.5:0/1444845451 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558e25933c70 0x558e25956a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:29.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:29.474+0000 7fc05d200640 1 --2- 172.21.15.5:0/1444845451 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558e25934590 0x558e25956f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:29.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:29.475+0000 7fc05e600640 1 -- 172.21.15.5:0/1444845451 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558e25933c70 msgr2=0x558e25956a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:54:29.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:29.475+0000 7fc05e600640 1 --2- 172.21.15.5:0/1444845451 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558e25933c70 0x558e25956a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:29.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:29.475+0000 7fc05e600640 1 -- 172.21.15.5:0/1444845451 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558e25934590 msgr2=0x558e25956f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:54:29.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:29.475+0000 7fc05e600640 1 --2- 172.21.15.5:0/1444845451 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558e25934590 0x558e25956f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:29.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:29.475+0000 7fc05e600640 1 -- 172.21.15.5:0/1444845451 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558e2592b770 con 0x558e2593ee50 2024-09-10T09:54:29.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:29.475+0000 7fc05e600640 1 --2- 172.21.15.5:0/1444845451 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558e2593ee50 0x558e25938eb0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fc0500318d0 tx=0x7fc05003e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f99856211a4132b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:54:29.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:29.475+0000 7fc04a000640 1 -- 172.21.15.5:0/1444845451 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc050035040 con 0x558e2593ee50 2024-09-10T09:54:29.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:29.475+0000 7fc04a000640 1 -- 172.21.15.5:0/1444845451 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc05004d070 con 0x558e2593ee50 2024-09-10T09:54:29.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:29.475+0000 7fc04a000640 1 -- 172.21.15.5:0/1444845451 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc050004ae0 con 0x558e2593ee50 2024-09-10T09:54:29.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:29.476+0000 7fc0630d5740 1 -- 172.21.15.5:0/1444845451 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558e2593ee50 msgr2=0x558e25938eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:29.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:29.476+0000 7fc0630d5740 1 --2- 172.21.15.5:0/1444845451 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558e2593ee50 0x558e25938eb0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fc0500318d0 tx=0x7fc05003e000 comp rx=0 tx=0).stop 2024-09-10T09:54:29.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:29.476+0000 7fc0630d5740 1 -- 172.21.15.5:0/1444845451 shutdown_connections 2024-09-10T09:54:29.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:29.476+0000 7fc0630d5740 1 --2- 172.21.15.5:0/1444845451 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558e2593ee50 0x558e25938eb0 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:29.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:29.476+0000 7fc0630d5740 1 --2- 172.21.15.5:0/1444845451 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558e25934590 0x558e25956f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:29.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:29.476+0000 7fc0630d5740 1 --2- 172.21.15.5:0/1444845451 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558e25933c70 0x558e25956a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:29.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:29.476+0000 7fc0630d5740 1 -- 172.21.15.5:0/1444845451 >> 172.21.15.5:0/1444845451 conn(0x558e25930200 msgr2=0x558e2593dcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:54:29.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:29.476+0000 7fc0630d5740 1 -- 172.21.15.5:0/1444845451 shutdown_connections 2024-09-10T09:54:29.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:29.476+0000 7fc0630d5740 1 -- 172.21.15.5:0/1444845451 wait complete. 2024-09-10T09:54:29.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:29.476+0000 7fc0630d5740 1 -- start start 2024-09-10T09:54:29.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:29.477+0000 7fc0630d5740 1 Processor -- start 2024-09-10T09:54:29.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:29.477+0000 7fc0630d5740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558e25933c70 0x558e25942ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:29.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:29.477+0000 7fc0630d5740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558e25934590 0x558e25943220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:29.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:29.477+0000 7fc0630d5740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558e2593ee50 0x558e25943760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:29.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:29.477+0000 7fc0630d5740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc0581b2610 con 0x558e25933c70 2024-09-10T09:54:29.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:29.477+0000 7fc0630d5740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fc0581b1f40 con 0x558e2593ee50 2024-09-10T09:54:29.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:29.477+0000 7fc0630d5740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fc0581af0c0 con 0x558e25934590 2024-09-10T09:54:29.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:29.477+0000 7fc0630d5740 10 client.0 _ll_register_callbacks cb 0x7fc04400e940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:54:29.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:29.477+0000 7fc05dc00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558e25933c70 0x558e25942ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:29.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:29.477+0000 7fc05dc00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558e25933c70 0x558e25942ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56448/0 (socket says 172.21.15.5:56448) 2024-09-10T09:54:29.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:29.477+0000 7fc05dc00640 1 -- 172.21.15.5:0/4276179306 learned_addr learned my addr 172.21.15.5:0/4276179306 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:54:29.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:29.477+0000 7fc05e600640 1 --2- 172.21.15.5:0/4276179306 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558e2593ee50 0x558e25943760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:29.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:29.478+0000 7fc05d200640 1 --2- 172.21.15.5:0/4276179306 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558e25934590 0x558e25943220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:29.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:29.478+0000 7fc05dc00640 1 -- 172.21.15.5:0/4276179306 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558e25934590 msgr2=0x558e25943220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:54:29.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:29.478+0000 7fc05dc00640 1 --2- 172.21.15.5:0/4276179306 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558e25934590 0x558e25943220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:29.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:29.478+0000 7fc05dc00640 1 -- 172.21.15.5:0/4276179306 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558e2593ee50 msgr2=0x558e25943760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:54:29.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:29.478+0000 7fc05dc00640 1 --2- 172.21.15.5:0/4276179306 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558e2593ee50 0x558e25943760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:29.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:29.478+0000 7fc05dc00640 1 -- 172.21.15.5:0/4276179306 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558e259b8bc0 con 0x558e25933c70 2024-09-10T09:54:29.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:29.478+0000 7fc05dc00640 1 --2- 172.21.15.5:0/4276179306 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558e25933c70 0x558e25942ce0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fc04c010450 tx=0x7fc04c01bab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53dea0baaa9b4209 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:54:29.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:29.478+0000 7fc03f400640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-10T09:54:29.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:29.478+0000 7fc03f400640 1 -- 172.21.15.5:0/4276179306 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc04c017040 con 0x558e25933c70 2024-09-10T09:54:29.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:29.478+0000 7fc03f400640 1 -- 172.21.15.5:0/4276179306 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fc04c01b400 con 0x558e25933c70 2024-09-10T09:54:29.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:29.478+0000 7fc0630d5740 1 -- 172.21.15.5:0/4276179306 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc05004d070 con 0x558e25933c70 2024-09-10T09:54:29.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:29.479+0000 7fc03f400640 1 -- 172.21.15.5:0/4276179306 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7fc04c03d050 con 0x558e25933c70 2024-09-10T09:54:29.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:29.479+0000 7fc0630d5740 10 client.18549 fetch_fsmap learned FSMap version 38 2024-09-10T09:54:29.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:29.479+0000 7fc0630d5740 1 -- 172.21.15.5:0/4276179306 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x558e259b9ea0 con 0x558e25933c70 2024-09-10T09:54:29.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:29.479+0000 7fc03f400640 1 -- 172.21.15.5:0/4276179306 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc05004d070 con 0x558e25933c70 2024-09-10T09:54:29.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:29.479+0000 7fc03f400640 1 -- 172.21.15.5:0/4276179306 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 38) ==== 8386+0+0 (secure 0 0 0) 0x7fc04c003b00 con 0x558e25933c70 2024-09-10T09:54:29.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:29.479+0000 7fc0630d5740 10 client.18549 fetch_fsmap finished waiting for FSMap version 38 2024-09-10T09:54:29.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:29.479+0000 7fc0630d5740 10 client.18549 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-10T09:54:29.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:29.479+0000 7fc0630d5740 20 client.18549 populate_metadata read hostname 'smithi005' 2024-09-10T09:54:29.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:29.479+0000 7fc0630d5740 1 --2- 172.21.15.5:0/4276179306 >> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] conn(0x558e259ba2e0 0x558e259bc6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:29.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:29.479+0000 7fc0630d5740 4 client.18549 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T09:54:29.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:29.479+0000 7fc0630d5740 1 -- 172.21.15.5:0/4276179306 --> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558e258a1e60 con 0x558e259ba2e0 2024-09-10T09:54:29.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:29.480+0000 7fc05d200640 1 --2- 172.21.15.5:0/4276179306 >> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] conn(0x558e259ba2e0 0x558e259bc6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:29.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:29.480+0000 7fc05d200640 1 --2- 172.21.15.5:0/4276179306 >> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] conn(0x558e259ba2e0 0x558e259bc6d0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d96cf66a9b9b6b72 server_cookie=c1dfc2228b091cd4 in_seq=0 out_seq=0 2024-09-10T09:54:29.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:29.480+0000 7fc03f400640 10 client.18549 ms_handle_connect on v2:172.21.15.159:6838/2073739599 2024-09-10T09:54:29.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:29.481+0000 7fc03f400640 1 -- 172.21.15.5:0/4276179306 <== mds.1 v2:172.21.15.159:6838/2073739599 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558e258a1e60 con 0x558e259ba2e0 2024-09-10T09:54:29.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:29.481+0000 7fc03f400640 10 client.18549 handle_command_reply: tid=0 2024-09-10T09:54:29.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:29.481+0000 7fc0630d5740 1 client.18549 shutdown 2024-09-10T09:54:29.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:29.481+0000 7fc0630d5740 1 -- 172.21.15.5:0/4276179306 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558e25933c70 msgr2=0x558e25942ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:29.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:29.481+0000 7fc0630d5740 1 --2- 172.21.15.5:0/4276179306 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558e25933c70 0x558e25942ce0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fc04c010450 tx=0x7fc04c01bab0 comp rx=0 tx=0).stop 2024-09-10T09:54:29.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:29.481+0000 7fc0630d5740 1 -- 172.21.15.5:0/4276179306 shutdown_connections 2024-09-10T09:54:29.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:29.481+0000 7fc0630d5740 1 --2- 172.21.15.5:0/4276179306 >> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] conn(0x558e259ba2e0 0x558e259bc6d0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:29.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:29.481+0000 7fc0630d5740 1 --2- 172.21.15.5:0/4276179306 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558e2593ee50 0x558e25943760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:29.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:29.481+0000 7fc0630d5740 1 --2- 172.21.15.5:0/4276179306 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558e25934590 0x558e25943220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:29.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:29.481+0000 7fc0630d5740 1 --2- 172.21.15.5:0/4276179306 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558e25933c70 0x558e25942ce0 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:29.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:29.481+0000 7fc0630d5740 1 -- 172.21.15.5:0/4276179306 >> 172.21.15.5:0/4276179306 conn(0x558e25930200 msgr2=0x558e2592eb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:54:29.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:29.481+0000 7fc03f400640 0 client.18549 ms_handle_reset on v2:172.21.15.159:6838/2073739599 2024-09-10T09:54:29.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:29.481+0000 7fc0630d5740 1 -- 172.21.15.5:0/4276179306 shutdown_connections 2024-09-10T09:54:29.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:29.481+0000 7fc0630d5740 1 -- 172.21.15.5:0/4276179306 wait complete. 2024-09-10T09:54:29.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:29.481+0000 7fc0630d5740 20 client.18549 trim_cache size 0 max 16384 2024-09-10T09:54:29.487 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-10T09:54:29.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:29.482+0000 7fc060600640 1 -- 172.21.15.5:0/3057141117 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fc0400a2b80 msgr2=0x7fc0400a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:29.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:29.482+0000 7fc060600640 1 --2- 172.21.15.5:0/3057141117 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fc0400a2b80 0x7fc0400a5040 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fc050005940 tx=0x7fc0500058d0 comp rx=0 tx=0).stop 2024-09-10T09:54:29.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:29.482+0000 7fc060600640 1 -- 172.21.15.5:0/3057141117 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc0581049a0 msgr2=0x7fc05819aad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:29.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:29.482+0000 7fc060600640 1 --2- 172.21.15.5:0/3057141117 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc0581049a0 0x7fc05819aad0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fc04400e9f0 tx=0x7fc04400eeb0 comp rx=0 tx=0).stop 2024-09-10T09:54:29.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:29.482+0000 7fc060600640 1 -- 172.21.15.5:0/3057141117 shutdown_connections 2024-09-10T09:54:29.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:29.482+0000 7fc060600640 1 --2- 172.21.15.5:0/3057141117 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fc0400a2b80 0x7fc0400a5040 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:29.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:29.482+0000 7fc060600640 1 --2- 172.21.15.5:0/3057141117 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc058109120 0x7fc05819ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:29.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:29.482+0000 7fc060600640 1 --2- 172.21.15.5:0/3057141117 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc0581049a0 0x7fc05819aad0 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:29.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:29.482+0000 7fc060600640 1 --2- 172.21.15.5:0/3057141117 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc058103ff0 0x7fc05819a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:29.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:29.482+0000 7fc060600640 1 -- 172.21.15.5:0/3057141117 >> 172.21.15.5:0/3057141117 conn(0x7fc0580ffa90 msgr2=0x7fc058101ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:54:29.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:29.482+0000 7fc060600640 1 -- 172.21.15.5:0/3057141117 shutdown_connections 2024-09-10T09:54:29.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:29.482+0000 7fc060600640 1 -- 172.21.15.5:0/3057141117 wait complete. 2024-09-10T09:54:29.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:54:29 smithi005 ceph-mon[30350]: pgmap v877: 129 pgs: 129 active+clean; 534 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.3 MiB/s wr, 188 op/s 2024-09-10T09:54:29.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:54:29 smithi159 ceph-mon[35513]: pgmap v877: 129 pgs: 129 active+clean; 534 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.3 MiB/s wr, 188 op/s 2024-09-10T09:54:29.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:54:29 smithi102 ceph-mon[35442]: pgmap v877: 129 pgs: 129 active+clean; 534 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.3 MiB/s wr, 188 op/s 2024-09-10T09:54:29.965 INFO:tasks.workunit.client.0.smithi005.stdout: 10 41284 13.95 MB/sec execute 8 sec latency 3987.621 ms 2024-09-10T09:54:30.241 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-10T09:54:30.622 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:54:30.966 INFO:tasks.workunit.client.0.smithi005.stdout: 10 41284 12.40 MB/sec execute 9 sec latency 4987.685 ms 2024-09-10T09:54:31.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:31.213+0000 7f1d19800640 1 -- 172.21.15.5:0/689409804 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1d1410e260 msgr2=0x7f1d14110650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:31.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:31.213+0000 7f1d19800640 1 --2- 172.21.15.5:0/689409804 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1d1410e260 0x7f1d14110650 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f1d0c009ca0 tx=0x7f1d0c02a950 comp rx=0 tx=0).stop 2024-09-10T09:54:31.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:31.213+0000 7f1d19800640 1 -- 172.21.15.5:0/689409804 shutdown_connections 2024-09-10T09:54:31.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:31.213+0000 7f1d19800640 1 --2- 172.21.15.5:0/689409804 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1d1410e260 0x7f1d14110650 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:31.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:31.213+0000 7f1d19800640 1 --2- 172.21.15.5:0/689409804 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1d14074ee0 0x7f1d1410dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:31.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:31.213+0000 7f1d19800640 1 --2- 172.21.15.5:0/689409804 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1d140745c0 0x7f1d140749a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:31.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:31.213+0000 7f1d19800640 1 -- 172.21.15.5:0/689409804 >> 172.21.15.5:0/689409804 conn(0x7f1d140fd740 msgr2=0x7f1d140ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:54:31.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:31.214+0000 7f1d19800640 1 -- 172.21.15.5:0/689409804 shutdown_connections 2024-09-10T09:54:31.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:31.214+0000 7f1d19800640 1 -- 172.21.15.5:0/689409804 wait complete. 2024-09-10T09:54:31.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:31.214+0000 7f1d19800640 1 Processor -- start 2024-09-10T09:54:31.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:31.215+0000 7f1d19800640 1 -- start start 2024-09-10T09:54:31.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:31.215+0000 7f1d19800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1d140745c0 0x7f1d1419a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:31.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:31.215+0000 7f1d19800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1d14074ee0 0x7f1d1419a980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:31.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:31.215+0000 7f1d19800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1d1410e260 0x7f1d1419ed10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:31.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:31.215+0000 7f1d19800640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1d14113fb0 con 0x7f1d1410e260 2024-09-10T09:54:31.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:31.215+0000 7f1d19800640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f1d14113e30 con 0x7f1d140745c0 2024-09-10T09:54:31.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:31.215+0000 7f1d19800640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f1d14114130 con 0x7f1d14074ee0 2024-09-10T09:54:31.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:31.215+0000 7f1d13e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1d140745c0 0x7f1d1419a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:31.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:31.215+0000 7f1d0be00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1d14074ee0 0x7f1d1419a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:31.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:31.215+0000 7f1d13e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1d140745c0 0x7f1d1419a440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:37304/0 (socket says 172.21.15.5:37304) 2024-09-10T09:54:31.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:31.215+0000 7f1d13e00640 1 -- 172.21.15.5:0/1709334630 learned_addr learned my addr 172.21.15.5:0/1709334630 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:54:31.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:31.215+0000 7f1d18e00640 1 --2- 172.21.15.5:0/1709334630 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1d1410e260 0x7f1d1419ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:31.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:31.216+0000 7f1d0be00640 1 -- 172.21.15.5:0/1709334630 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1d140745c0 msgr2=0x7f1d1419a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:31.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:31.216+0000 7f1d0be00640 1 --2- 172.21.15.5:0/1709334630 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1d140745c0 0x7f1d1419a440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:31.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:31.216+0000 7f1d0be00640 1 -- 172.21.15.5:0/1709334630 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1d1410e260 msgr2=0x7f1d1419ed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:31.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:31.216+0000 7f1d0be00640 1 --2- 172.21.15.5:0/1709334630 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1d1410e260 0x7f1d1419ed10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:31.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:31.216+0000 7f1d0be00640 1 -- 172.21.15.5:0/1709334630 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d1419f3f0 con 0x7f1d14074ee0 2024-09-10T09:54:31.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:31.216+0000 7f1d13e00640 1 --2- 172.21.15.5:0/1709334630 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1d140745c0 0x7f1d1419a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:54:31.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:31.216+0000 7f1d0be00640 1 --2- 172.21.15.5:0/1709334630 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1d14074ee0 0x7f1d1419a980 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f1cfc00e9f0 tx=0x7f1cfc00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:54:31.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:31.217+0000 7f1d11600640 1 -- 172.21.15.5:0/1709334630 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cfc00ccc0 con 0x7f1d14074ee0 2024-09-10T09:54:31.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:31.217+0000 7f1d11600640 1 -- 172.21.15.5:0/1709334630 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1cfc010910 con 0x7f1d14074ee0 2024-09-10T09:54:31.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:31.217+0000 7f1d11600640 1 -- 172.21.15.5:0/1709334630 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cfc018960 con 0x7f1d14074ee0 2024-09-10T09:54:31.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:31.217+0000 7f1d19800640 1 -- 172.21.15.5:0/1709334630 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d1419f6e0 con 0x7f1d14074ee0 2024-09-10T09:54:31.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:31.217+0000 7f1d19800640 1 -- 172.21.15.5:0/1709334630 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d141a6fc0 con 0x7f1d14074ee0 2024-09-10T09:54:31.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:31.220+0000 7f1d1d3fd740 1 Processor -- start 2024-09-10T09:54:31.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:31.220+0000 7f1d1d3fd740 1 -- start start 2024-09-10T09:54:31.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:31.220+0000 7f1d1d3fd740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d694209e40 0x55d69420a220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:31.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:31.220+0000 7f1d1d3fd740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d69420a760 0x55d694214ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:31.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:31.220+0000 7f1d1d3fd740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d694215020 0x55d694217410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:31.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:31.220+0000 7f1d1d3fd740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55d6941703e0 con 0x55d69420a760 2024-09-10T09:54:31.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:31.220+0000 7f1d1d3fd740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55d694117d80 con 0x55d694209e40 2024-09-10T09:54:31.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:31.220+0000 7f1d1d3fd740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55d69416c3a0 con 0x55d694215020 2024-09-10T09:54:31.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:31.220+0000 7f1d13e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d69420a760 0x55d694214ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:31.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:31.220+0000 7f1d11600640 1 -- 172.21.15.5:0/1709334630 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f1cfc010ac0 con 0x7f1d14074ee0 2024-09-10T09:54:31.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:31.220+0000 7f1d18e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d694209e40 0x55d69420a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:31.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:31.220+0000 7f1d13e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d69420a760 0x55d694214ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38440/0 (socket says 172.21.15.5:38440) 2024-09-10T09:54:31.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:31.221+0000 7f1d13e00640 1 -- 172.21.15.5:0/261455242 learned_addr learned my addr 172.21.15.5:0/261455242 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:54:31.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:31.220+0000 7f1d18e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d694209e40 0x55d69420a220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:37320/0 (socket says 172.21.15.5:37320) 2024-09-10T09:54:31.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:31.221+0000 7f1d0be00640 1 --2- 172.21.15.5:0/261455242 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d694215020 0x55d694217410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:31.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:31.221+0000 7f1d18e00640 1 -- 172.21.15.5:0/261455242 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d694215020 msgr2=0x55d694217410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:54:31.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:31.221+0000 7f1d18e00640 1 --2- 172.21.15.5:0/261455242 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d694215020 0x55d694217410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:31.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:31.221+0000 7f1d18e00640 1 -- 172.21.15.5:0/261455242 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d69420a760 msgr2=0x55d694214ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:54:31.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:31.221+0000 7f1d18e00640 1 --2- 172.21.15.5:0/261455242 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d69420a760 0x55d694214ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:31.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:31.221+0000 7f1d18e00640 1 -- 172.21.15.5:0/261455242 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d694136ae0 con 0x55d694209e40 2024-09-10T09:54:31.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:31.221+0000 7f1d13e00640 1 --2- 172.21.15.5:0/261455242 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d69420a760 0x55d694214ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:54:31.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:31.221+0000 7f1d0be00640 1 --2- 172.21.15.5:0/261455242 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d694215020 0x55d694217410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T09:54:31.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:31.221+0000 7f1d18e00640 1 --2- 172.21.15.5:0/261455242 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d694209e40 0x55d69420a220 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f1d0c02ae70 tx=0x7f1d0c0093e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8e2fc6ce8c8d8095 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:54:31.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:31.222+0000 7f1d09600640 1 -- 172.21.15.5:0/261455242 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d0c048070 con 0x55d694209e40 2024-09-10T09:54:31.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:31.222+0000 7f1d09600640 1 -- 172.21.15.5:0/261455242 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1d0c02f940 con 0x55d694209e40 2024-09-10T09:54:31.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:31.222+0000 7f1d09600640 1 -- 172.21.15.5:0/261455242 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d0c034040 con 0x55d694209e40 2024-09-10T09:54:31.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:31.222+0000 7f1d1d3fd740 1 -- 172.21.15.5:0/261455242 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d694209e40 msgr2=0x55d69420a220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:31.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:31.222+0000 7f1d1d3fd740 1 --2- 172.21.15.5:0/261455242 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d694209e40 0x55d69420a220 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f1d0c02ae70 tx=0x7f1d0c0093e0 comp rx=0 tx=0).stop 2024-09-10T09:54:31.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:31.222+0000 7f1d1d3fd740 1 -- 172.21.15.5:0/261455242 shutdown_connections 2024-09-10T09:54:31.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:31.222+0000 7f1d1d3fd740 1 --2- 172.21.15.5:0/261455242 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d694215020 0x55d694217410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:31.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:31.222+0000 7f1d1d3fd740 1 --2- 172.21.15.5:0/261455242 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d69420a760 0x55d694214ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:31.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:31.222+0000 7f1d1d3fd740 1 --2- 172.21.15.5:0/261455242 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d694209e40 0x55d69420a220 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:31.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:31.222+0000 7f1d1d3fd740 1 -- 172.21.15.5:0/261455242 >> 172.21.15.5:0/261455242 conn(0x55d6942063d0 msgr2=0x55d6942067b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:54:31.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:31.223+0000 7f1d1d3fd740 1 -- 172.21.15.5:0/261455242 shutdown_connections 2024-09-10T09:54:31.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:31.223+0000 7f1d1d3fd740 1 -- 172.21.15.5:0/261455242 wait complete. 2024-09-10T09:54:31.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:31.223+0000 7f1d11600640 1 --2- 172.21.15.5:0/1709334630 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f1cf80ab320 0x7f1cf80ad7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:31.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:31.223+0000 7f1d11600640 1 -- 172.21.15.5:0/1709334630 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f1cfc0cf740 con 0x7f1d14074ee0 2024-09-10T09:54:31.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:31.223+0000 7f1d1d3fd740 1 -- start start 2024-09-10T09:54:31.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:31.223+0000 7f1d13e00640 1 --2- 172.21.15.5:0/1709334630 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f1cf80ab320 0x7f1cf80ad7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:31.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:31.224+0000 7f1d1d3fd740 1 Processor -- start 2024-09-10T09:54:31.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:31.224+0000 7f1d13e00640 1 --2- 172.21.15.5:0/1709334630 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f1cf80ab320 0x7f1cf80ad7e0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f1d040096e0 tx=0x7f1d04005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:54:31.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:31.224+0000 7f1d1d3fd740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d694209e40 0x55d694211f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:31.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:31.224+0000 7f1d1d3fd740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d69420a760 0x55d6942124b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:31.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:31.224+0000 7f1d1d3fd740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d694215020 0x55d6942129f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:31.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:31.224+0000 7f1d1d3fd740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55d694217bf0 con 0x55d69420a760 2024-09-10T09:54:31.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:31.224+0000 7f1d1d3fd740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55d69412b0f0 con 0x55d694209e40 2024-09-10T09:54:31.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:31.224+0000 7f1d1d3fd740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55d694217d70 con 0x55d694215020 2024-09-10T09:54:31.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:31.224+0000 7f1d1d3fd740 10 client.0 _ll_register_callbacks cb 0x55d694112bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:54:31.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:31.224+0000 7f1d13e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d694209e40 0x55d694211f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:31.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:31.224+0000 7f1d0be00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d69420a760 0x55d6942124b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:31.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:31.225+0000 7f1d13e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d694209e40 0x55d694211f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:37334/0 (socket says 172.21.15.5:37334) 2024-09-10T09:54:31.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:31.225+0000 7f1d13e00640 1 -- 172.21.15.5:0/3420863663 learned_addr learned my addr 172.21.15.5:0/3420863663 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:54:31.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:31.225+0000 7f1d18e00640 1 --2- 172.21.15.5:0/3420863663 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d694215020 0x55d6942129f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:31.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:31.225+0000 7f1d13e00640 1 -- 172.21.15.5:0/3420863663 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d694215020 msgr2=0x55d6942129f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:54:31.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:31.225+0000 7f1d13e00640 1 --2- 172.21.15.5:0/3420863663 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d694215020 0x55d6942129f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:31.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:31.225+0000 7f1d13e00640 1 -- 172.21.15.5:0/3420863663 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d69420a760 msgr2=0x55d6942124b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:54:31.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:31.225+0000 7f1d13e00640 1 --2- 172.21.15.5:0/3420863663 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d69420a760 0x55d6942124b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:31.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:31.225+0000 7f1d13e00640 1 -- 172.21.15.5:0/3420863663 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d693ee13c0 con 0x55d694209e40 2024-09-10T09:54:31.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:31.225+0000 7f1d13e00640 1 --2- 172.21.15.5:0/3420863663 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d694209e40 0x55d694211f70 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f1d0400f8a0 tx=0x7f1d04004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ecc3e8c0b0bf8e24 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:54:31.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:31.225+0000 7f1d02000640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-09-10T09:54:31.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:31.225+0000 7f1d02000640 1 -- 172.21.15.5:0/3420863663 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d04011bd0 con 0x55d694209e40 2024-09-10T09:54:31.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:31.226+0000 7f1d02000640 1 -- 172.21.15.5:0/3420863663 <== mon.2 v2:172.21.15.102:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f1d04011d70 con 0x55d694209e40 2024-09-10T09:54:31.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:31.226+0000 7f1d1d3fd740 1 -- 172.21.15.5:0/3420863663 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1d0c02f940 con 0x55d694209e40 2024-09-10T09:54:31.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:31.226+0000 7f1d02000640 1 -- 172.21.15.5:0/3420863663 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f1d04032050 con 0x55d694209e40 2024-09-10T09:54:31.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:31.226+0000 7f1d1d3fd740 10 client.28631 fetch_fsmap learned FSMap version 38 2024-09-10T09:54:31.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:31.226+0000 7f1d1d3fd740 1 -- 172.21.15.5:0/3420863663 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55d694223cb0 con 0x55d694209e40 2024-09-10T09:54:31.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:31.226+0000 7f1d02000640 1 -- 172.21.15.5:0/3420863663 <== mon.2 v2:172.21.15.102:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1d0c02f940 con 0x55d694209e40 2024-09-10T09:54:31.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:31.226+0000 7f1d02000640 1 -- 172.21.15.5:0/3420863663 <== mon.2 v2:172.21.15.102:3300/0 5 ==== fsmap(e 38) ==== 8386+0+0 (secure 0 0 0) 0x7f1d04027020 con 0x55d694209e40 2024-09-10T09:54:31.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:31.226+0000 7f1d1d3fd740 10 client.28631 fetch_fsmap finished waiting for FSMap version 38 2024-09-10T09:54:31.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:31.226+0000 7f1d1d3fd740 10 client.28631 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T09:54:31.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:31.226+0000 7f1d1d3fd740 20 client.28631 populate_metadata read hostname 'smithi005' 2024-09-10T09:54:31.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:31.227+0000 7f1d1d3fd740 1 --2- 172.21.15.5:0/3420863663 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55d694225170 0x55d6942275c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:31.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:31.227+0000 7f1d1d3fd740 4 client.28631 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:54:31.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:31.227+0000 7f1d1d3fd740 1 -- 172.21.15.5:0/3420863663 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d69417d950 con 0x55d694225170 2024-09-10T09:54:31.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:31.227+0000 7f1d0be00640 1 --2- 172.21.15.5:0/3420863663 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55d694225170 0x55d6942275c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:31.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:31.227+0000 7f1d0be00640 1 --2- 172.21.15.5:0/3420863663 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55d694225170 0x55d6942275c0 crc :-1 s=READY pgs=235 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=81cc55d89500eac4 server_cookie=b735566fc0a9157e in_seq=0 out_seq=0 2024-09-10T09:54:31.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:31.227+0000 7f1d02000640 10 client.28631 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T09:54:31.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:31.228+0000 7f1d02000640 1 -- 172.21.15.5:0/3420863663 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d69417d950 con 0x55d694225170 2024-09-10T09:54:31.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:31.228+0000 7f1d02000640 10 client.28631 handle_command_reply: tid=0 2024-09-10T09:54:31.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:31.228+0000 7f1d1d3fd740 1 client.28631 shutdown 2024-09-10T09:54:31.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:31.228+0000 7f1d1d3fd740 1 -- 172.21.15.5:0/3420863663 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d694209e40 msgr2=0x55d694211f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:31.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:31.228+0000 7f1d1d3fd740 1 --2- 172.21.15.5:0/3420863663 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d694209e40 0x55d694211f70 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f1d0400f8a0 tx=0x7f1d04004730 comp rx=0 tx=0).stop 2024-09-10T09:54:31.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:31.229+0000 7f1d1d3fd740 1 -- 172.21.15.5:0/3420863663 shutdown_connections 2024-09-10T09:54:31.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:31.229+0000 7f1d1d3fd740 1 --2- 172.21.15.5:0/3420863663 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55d694225170 0x55d6942275c0 crc :-1 s=READY pgs=235 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:31.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:31.229+0000 7f1d1d3fd740 1 --2- 172.21.15.5:0/3420863663 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d694215020 0x55d6942129f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:31.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:31.229+0000 7f1d1d3fd740 1 --2- 172.21.15.5:0/3420863663 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d69420a760 0x55d6942124b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:31.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:31.229+0000 7f1d1d3fd740 1 --2- 172.21.15.5:0/3420863663 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d694209e40 0x55d694211f70 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:31.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:31.229+0000 7f1d02000640 0 client.28631 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T09:54:31.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:31.229+0000 7f1d1d3fd740 1 -- 172.21.15.5:0/3420863663 >> 172.21.15.5:0/3420863663 conn(0x55d6942063d0 msgr2=0x55d69420c100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:54:31.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:31.229+0000 7f1d1d3fd740 1 -- 172.21.15.5:0/3420863663 shutdown_connections 2024-09-10T09:54:31.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:31.229+0000 7f1d1d3fd740 1 -- 172.21.15.5:0/3420863663 wait complete. 2024-09-10T09:54:31.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:31.229+0000 7f1d1d3fd740 20 client.28631 trim_cache size 0 max 16384 2024-09-10T09:54:31.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:31.243+0000 7f1d1d3fd740 1 Processor -- start 2024-09-10T09:54:31.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:31.243+0000 7f1d1d3fd740 1 -- start start 2024-09-10T09:54:31.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:31.243+0000 7f1d1d3fd740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d694209e40 0x55d69420c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:31.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:31.243+0000 7f1d1d3fd740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d69420a760 0x55d69422ce90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:31.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:31.243+0000 7f1d1d3fd740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d694215020 0x55d69422d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:31.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:31.243+0000 7f1d1d3fd740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1d141ac9b0 con 0x55d694215020 2024-09-10T09:54:31.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:31.243+0000 7f1d1d3fd740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f1d141ae5b0 con 0x55d694209e40 2024-09-10T09:54:31.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:31.243+0000 7f1d1d3fd740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f1d141ad080 con 0x55d69420a760 2024-09-10T09:54:31.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:31.243+0000 7f1d18e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d694215020 0x55d69422d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:31.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:31.243+0000 7f1d13e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d694209e40 0x55d69420c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:31.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:31.243+0000 7f1d18e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d694215020 0x55d69422d3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38460/0 (socket says 172.21.15.5:38460) 2024-09-10T09:54:31.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:31.243+0000 7f1d18e00640 1 -- 172.21.15.5:0/4197767947 learned_addr learned my addr 172.21.15.5:0/4197767947 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:54:31.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:31.243+0000 7f1d13e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d694209e40 0x55d69420c8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:37338/0 (socket says 172.21.15.5:37338) 2024-09-10T09:54:31.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:31.243+0000 7f1d0be00640 1 --2- 172.21.15.5:0/4197767947 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d69420a760 0x55d69422ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:31.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:31.244+0000 7f1d13e00640 1 -- 172.21.15.5:0/4197767947 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d69420a760 msgr2=0x55d69422ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:54:31.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:31.244+0000 7f1d13e00640 1 --2- 172.21.15.5:0/4197767947 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d69420a760 0x55d69422ce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:31.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:31.244+0000 7f1d13e00640 1 -- 172.21.15.5:0/4197767947 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d694215020 msgr2=0x55d69422d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:54:31.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:31.244+0000 7f1d13e00640 1 --2- 172.21.15.5:0/4197767947 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d694215020 0x55d69422d3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:31.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:31.244+0000 7f1d13e00640 1 -- 172.21.15.5:0/4197767947 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d6942148e0 con 0x55d694209e40 2024-09-10T09:54:31.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:31.244+0000 7f1d18e00640 1 --2- 172.21.15.5:0/4197767947 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d694215020 0x55d69422d3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:54:31.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:31.244+0000 7f1d13e00640 1 --2- 172.21.15.5:0/4197767947 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d694209e40 0x55d69420c8f0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f1d0401f670 tx=0x7f1d04000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ddc1ff735fcf65d8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:54:31.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:31.244+0000 7f1d08c00640 1 -- 172.21.15.5:0/4197767947 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d0403a020 con 0x55d694209e40 2024-09-10T09:54:31.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:31.244+0000 7f1d08c00640 1 -- 172.21.15.5:0/4197767947 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1d0401f8f0 con 0x55d694209e40 2024-09-10T09:54:31.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:31.244+0000 7f1d08c00640 1 -- 172.21.15.5:0/4197767947 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d04011bd0 con 0x55d694209e40 2024-09-10T09:54:31.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:31.245+0000 7f1d1d3fd740 1 -- 172.21.15.5:0/4197767947 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d694209e40 msgr2=0x55d69420c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:31.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:31.245+0000 7f1d1d3fd740 1 --2- 172.21.15.5:0/4197767947 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d694209e40 0x55d69420c8f0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f1d0401f670 tx=0x7f1d04000bc0 comp rx=0 tx=0).stop 2024-09-10T09:54:31.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:31.245+0000 7f1d1d3fd740 1 -- 172.21.15.5:0/4197767947 shutdown_connections 2024-09-10T09:54:31.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:31.245+0000 7f1d1d3fd740 1 --2- 172.21.15.5:0/4197767947 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d694215020 0x55d69422d3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:31.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:31.245+0000 7f1d1d3fd740 1 --2- 172.21.15.5:0/4197767947 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d69420a760 0x55d69422ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:31.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:31.245+0000 7f1d1d3fd740 1 --2- 172.21.15.5:0/4197767947 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d694209e40 0x55d69420c8f0 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:31.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:31.245+0000 7f1d1d3fd740 1 -- 172.21.15.5:0/4197767947 >> 172.21.15.5:0/4197767947 conn(0x55d6942063d0 msgr2=0x55d6942054b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:54:31.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:31.245+0000 7f1d1d3fd740 1 -- 172.21.15.5:0/4197767947 shutdown_connections 2024-09-10T09:54:31.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:31.245+0000 7f1d1d3fd740 1 -- 172.21.15.5:0/4197767947 wait complete. 2024-09-10T09:54:31.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:31.246+0000 7f1d1d3fd740 1 -- start start 2024-09-10T09:54:31.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:31.246+0000 7f1d1d3fd740 1 Processor -- start 2024-09-10T09:54:31.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:31.246+0000 7f1d1d3fd740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d694209e40 0x55d69420ef80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:31.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:31.246+0000 7f1d1d3fd740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d69420a760 0x55d69420f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:31.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:31.246+0000 7f1d1d3fd740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d694215020 0x55d69420fa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:31.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:31.246+0000 7f1d1d3fd740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1d141adee0 con 0x55d69420a760 2024-09-10T09:54:31.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:31.246+0000 7f1d1d3fd740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f1d141ad810 con 0x55d694215020 2024-09-10T09:54:31.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:31.246+0000 7f1d1d3fd740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f1d141aa9b0 con 0x55d694209e40 2024-09-10T09:54:31.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:31.246+0000 7f1d1d3fd740 10 client.0 _ll_register_callbacks cb 0x7f1d04012050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:54:31.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:31.247+0000 7f1d0be00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d69420a760 0x55d69420f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:31.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:31.247+0000 7f1d13e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d694209e40 0x55d69420ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:31.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:31.247+0000 7f1d0be00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d69420a760 0x55d69420f4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38470/0 (socket says 172.21.15.5:38470) 2024-09-10T09:54:31.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:31.247+0000 7f1d13e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d694209e40 0x55d69420ef80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:38256/0 (socket says 172.21.15.5:38256) 2024-09-10T09:54:31.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:31.247+0000 7f1d0be00640 1 -- 172.21.15.5:0/3825225907 learned_addr learned my addr 172.21.15.5:0/3825225907 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:54:31.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:31.247+0000 7f1d18e00640 1 --2- 172.21.15.5:0/3825225907 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d694215020 0x55d69420fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:31.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:31.247+0000 7f1d0be00640 1 -- 172.21.15.5:0/3825225907 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d694209e40 msgr2=0x55d69420ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:54:31.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:31.247+0000 7f1d0be00640 1 --2- 172.21.15.5:0/3825225907 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d694209e40 0x55d69420ef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:31.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:31.247+0000 7f1d0be00640 1 -- 172.21.15.5:0/3825225907 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d694215020 msgr2=0x55d69420fa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:54:31.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:31.247+0000 7f1d0be00640 1 --2- 172.21.15.5:0/3825225907 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d694215020 0x55d69420fa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:31.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:31.247+0000 7f1d0be00640 1 -- 172.21.15.5:0/3825225907 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d69428ee90 con 0x55d69420a760 2024-09-10T09:54:31.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:31.247+0000 7f1d0be00640 1 --2- 172.21.15.5:0/3825225907 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d69420a760 0x55d69420f4c0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f1cfc09bb90 tx=0x7f1cfc00cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c3dc01b032819e5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:54:31.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:31.247+0000 7f1d02000640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-10T09:54:31.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:31.248+0000 7f1d02000640 1 -- 172.21.15.5:0/3825225907 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cfc0b3070 con 0x55d69420a760 2024-09-10T09:54:31.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:31.248+0000 7f1d02000640 1 -- 172.21.15.5:0/3825225907 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f1cfc098270 con 0x55d69420a760 2024-09-10T09:54:31.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:31.248+0000 7f1d1d3fd740 1 -- 172.21.15.5:0/3825225907 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1d0401f8f0 con 0x55d69420a760 2024-09-10T09:54:31.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:31.248+0000 7f1d02000640 1 -- 172.21.15.5:0/3825225907 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f1cfc0cf390 con 0x55d69420a760 2024-09-10T09:54:31.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:31.248+0000 7f1d1d3fd740 10 client.18567 fetch_fsmap learned FSMap version 38 2024-09-10T09:54:31.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:31.248+0000 7f1d1d3fd740 1 -- 172.21.15.5:0/3825225907 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55d6942901b0 con 0x55d69420a760 2024-09-10T09:54:31.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:31.248+0000 7f1d02000640 1 -- 172.21.15.5:0/3825225907 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1d0401f8f0 con 0x55d69420a760 2024-09-10T09:54:31.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:31.248+0000 7f1d02000640 1 -- 172.21.15.5:0/3825225907 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 38) ==== 8386+0+0 (secure 0 0 0) 0x7f1cfc09b050 con 0x55d69420a760 2024-09-10T09:54:31.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:31.248+0000 7f1d1d3fd740 10 client.18567 fetch_fsmap finished waiting for FSMap version 38 2024-09-10T09:54:31.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:31.248+0000 7f1d1d3fd740 10 client.18567 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T09:54:31.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:31.248+0000 7f1d1d3fd740 20 client.18567 populate_metadata read hostname 'smithi005' 2024-09-10T09:54:31.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:31.249+0000 7f1d1d3fd740 1 --2- 172.21.15.5:0/3825225907 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55d6942908d0 0x55d694292cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:31.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:31.249+0000 7f1d1d3fd740 4 client.18567 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T09:54:31.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:31.249+0000 7f1d1d3fd740 1 -- 172.21.15.5:0/3825225907 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d694177e60 con 0x55d6942908d0 2024-09-10T09:54:31.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:31.249+0000 7f1d13e00640 1 --2- 172.21.15.5:0/3825225907 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55d6942908d0 0x55d694292cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:31.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:31.250+0000 7f1d13e00640 1 --2- 172.21.15.5:0/3825225907 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55d6942908d0 0x55d694292cc0 crc :-1 s=READY pgs=236 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ce7d8320ea652ed6 server_cookie=f9a71ecc2f155ed5 in_seq=0 out_seq=0 2024-09-10T09:54:31.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:31.250+0000 7f1d02000640 10 client.18567 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T09:54:31.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:31.250+0000 7f1d02000640 1 -- 172.21.15.5:0/3825225907 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d694177e60 con 0x55d6942908d0 2024-09-10T09:54:31.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:31.250+0000 7f1d02000640 10 client.18567 handle_command_reply: tid=0 2024-09-10T09:54:31.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:31.250+0000 7f1d1d3fd740 1 client.18567 shutdown 2024-09-10T09:54:31.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:31.250+0000 7f1d1d3fd740 1 -- 172.21.15.5:0/3825225907 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d69420a760 msgr2=0x55d69420f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:31.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:31.250+0000 7f1d1d3fd740 1 --2- 172.21.15.5:0/3825225907 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d69420a760 0x55d69420f4c0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f1cfc09bb90 tx=0x7f1cfc00cf60 comp rx=0 tx=0).stop 2024-09-10T09:54:31.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:31.250+0000 7f1d1d3fd740 1 -- 172.21.15.5:0/3825225907 shutdown_connections 2024-09-10T09:54:31.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:31.250+0000 7f1d1d3fd740 1 --2- 172.21.15.5:0/3825225907 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55d6942908d0 0x55d694292cc0 crc :-1 s=READY pgs=236 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:31.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:31.250+0000 7f1d1d3fd740 1 --2- 172.21.15.5:0/3825225907 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d694215020 0x55d69420fa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:31.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:31.250+0000 7f1d1d3fd740 1 --2- 172.21.15.5:0/3825225907 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d69420a760 0x55d69420f4c0 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:31.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:31.250+0000 7f1d1d3fd740 1 --2- 172.21.15.5:0/3825225907 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d694209e40 0x55d69420ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:31.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:31.250+0000 7f1d02000640 0 client.18567 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T09:54:31.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:31.250+0000 7f1d1d3fd740 1 -- 172.21.15.5:0/3825225907 >> 172.21.15.5:0/3825225907 conn(0x55d6942063d0 msgr2=0x55d6942137d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:54:31.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:31.250+0000 7f1d1d3fd740 1 -- 172.21.15.5:0/3825225907 shutdown_connections 2024-09-10T09:54:31.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:31.251+0000 7f1d1d3fd740 1 -- 172.21.15.5:0/3825225907 wait complete. 2024-09-10T09:54:31.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:31.251+0000 7f1d1d3fd740 20 client.18567 trim_cache size 0 max 16384 2024-09-10T09:54:31.257 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-10T09:54:31.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:31.251+0000 7f1d19800640 1 -- 172.21.15.5:0/1709334630 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f1cf80ab320 msgr2=0x7f1cf80ad7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:31.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:31.251+0000 7f1d19800640 1 --2- 172.21.15.5:0/1709334630 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f1cf80ab320 0x7f1cf80ad7e0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f1d040096e0 tx=0x7f1d04005f90 comp rx=0 tx=0).stop 2024-09-10T09:54:31.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:31.251+0000 7f1d19800640 1 -- 172.21.15.5:0/1709334630 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1d14074ee0 msgr2=0x7f1d1419a980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:31.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:31.251+0000 7f1d19800640 1 --2- 172.21.15.5:0/1709334630 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1d14074ee0 0x7f1d1419a980 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f1cfc00e9f0 tx=0x7f1cfc00eeb0 comp rx=0 tx=0).stop 2024-09-10T09:54:31.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:31.251+0000 7f1d19800640 1 -- 172.21.15.5:0/1709334630 shutdown_connections 2024-09-10T09:54:31.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:31.251+0000 7f1d19800640 1 --2- 172.21.15.5:0/1709334630 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f1cf80ab320 0x7f1cf80ad7e0 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:31.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:31.251+0000 7f1d19800640 1 --2- 172.21.15.5:0/1709334630 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1d1410e260 0x7f1d1419ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:31.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:31.251+0000 7f1d19800640 1 --2- 172.21.15.5:0/1709334630 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1d14074ee0 0x7f1d1419a980 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:31.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:31.251+0000 7f1d19800640 1 --2- 172.21.15.5:0/1709334630 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1d140745c0 0x7f1d1419a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:31.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:31.251+0000 7f1d19800640 1 -- 172.21.15.5:0/1709334630 >> 172.21.15.5:0/1709334630 conn(0x7f1d140fd740 msgr2=0x7f1d140fe3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:54:31.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:31.252+0000 7f1d19800640 1 -- 172.21.15.5:0/1709334630 shutdown_connections 2024-09-10T09:54:31.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:31.252+0000 7f1d19800640 1 -- 172.21.15.5:0/1709334630 wait complete. 2024-09-10T09:54:31.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:54:31 smithi159 ceph-mon[35513]: pgmap v878: 129 pgs: 129 active+clean; 535 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.1 MiB/s wr, 266 op/s 2024-09-10T09:54:31.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:54:31 smithi102 ceph-mon[35442]: pgmap v878: 129 pgs: 129 active+clean; 535 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.1 MiB/s wr, 266 op/s 2024-09-10T09:54:31.966 INFO:tasks.workunit.client.0.smithi005.stdout: 10 41284 11.16 MB/sec execute 10 sec latency 5987.750 ms 2024-09-10T09:54:32.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:54:31 smithi005 ceph-mon[30350]: pgmap v878: 129 pgs: 129 active+clean; 535 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.1 MiB/s wr, 266 op/s 2024-09-10T09:54:32.966 INFO:tasks.workunit.client.0.smithi005.stdout: 10 41284 10.15 MB/sec execute 11 sec latency 6987.895 ms 2024-09-10T09:54:33.215 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-10T09:54:33.216 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-10T09:54:33.604 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:54:33.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:54:33 smithi159 ceph-mon[35513]: pgmap v879: 129 pgs: 129 active+clean; 534 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 1.3 MiB/s wr, 212 op/s 2024-09-10T09:54:33.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:54:33 smithi102 ceph-mon[35442]: pgmap v879: 129 pgs: 129 active+clean; 534 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 1.3 MiB/s wr, 212 op/s 2024-09-10T09:54:33.921 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:54:33 smithi005 ceph-mon[30350]: pgmap v879: 129 pgs: 129 active+clean; 534 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 1.3 MiB/s wr, 212 op/s 2024-09-10T09:54:33.966 INFO:tasks.workunit.client.0.smithi005.stdout: 10 41284 9.30 MB/sec execute 12 sec latency 7987.962 ms 2024-09-10T09:54:34.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:34.191+0000 7f1794a00640 1 -- 172.21.15.5:0/3315748193 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f17901049a0 msgr2=0x7f17901089f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:34.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:34.191+0000 7f1794a00640 1 --2- 172.21.15.5:0/3315748193 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f17901049a0 0x7f17901089f0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f1778009900 tx=0x7f177802a3f0 comp rx=0 tx=0).stop 2024-09-10T09:54:34.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:34.191+0000 7f1794a00640 1 -- 172.21.15.5:0/3315748193 shutdown_connections 2024-09-10T09:54:34.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:34.191+0000 7f1794a00640 1 --2- 172.21.15.5:0/3315748193 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1790109120 0x7f1790110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:34.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:34.191+0000 7f1794a00640 1 --2- 172.21.15.5:0/3315748193 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f17901049a0 0x7f17901089f0 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:34.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:34.191+0000 7f1794a00640 1 --2- 172.21.15.5:0/3315748193 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1790103ff0 0x7f17901043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:34.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:34.191+0000 7f1794a00640 1 -- 172.21.15.5:0/3315748193 >> 172.21.15.5:0/3315748193 conn(0x7f17900ffa90 msgr2=0x7f1790101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:54:34.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:34.192+0000 7f1794a00640 1 -- 172.21.15.5:0/3315748193 shutdown_connections 2024-09-10T09:54:34.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:34.192+0000 7f1794a00640 1 -- 172.21.15.5:0/3315748193 wait complete. 2024-09-10T09:54:34.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:34.192+0000 7f1794a00640 1 Processor -- start 2024-09-10T09:54:34.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:34.193+0000 7f1794a00640 1 -- start start 2024-09-10T09:54:34.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:34.193+0000 7f1794a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1790103ff0 0x7f179019a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:34.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:34.193+0000 7f1794a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f17901049a0 0x7f179019abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:34.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:34.193+0000 7f1794a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1790109120 0x7f179019ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:34.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:34.193+0000 7f1794a00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1790114120 con 0x7f1790103ff0 2024-09-10T09:54:34.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:34.193+0000 7f1794a00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f1790113fa0 con 0x7f1790109120 2024-09-10T09:54:34.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:34.193+0000 7f1794a00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f17901142a0 con 0x7f17901049a0 2024-09-10T09:54:34.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:34.193+0000 7f178de00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1790103ff0 0x7f179019a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:34.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:34.193+0000 7f178de00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1790103ff0 0x7f179019a6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38492/0 (socket says 172.21.15.5:38492) 2024-09-10T09:54:34.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:34.193+0000 7f178de00640 1 -- 172.21.15.5:0/3006749761 learned_addr learned my addr 172.21.15.5:0/3006749761 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:54:34.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:34.193+0000 7f178e800640 1 --2- 172.21.15.5:0/3006749761 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1790109120 0x7f179019ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:34.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:34.193+0000 7f1785400640 1 --2- 172.21.15.5:0/3006749761 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f17901049a0 0x7f179019abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:34.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:34.194+0000 7f178de00640 1 -- 172.21.15.5:0/3006749761 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f17901049a0 msgr2=0x7f179019abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:34.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:34.194+0000 7f178de00640 1 --2- 172.21.15.5:0/3006749761 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f17901049a0 0x7f179019abe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:34.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:34.194+0000 7f178de00640 1 -- 172.21.15.5:0/3006749761 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1790109120 msgr2=0x7f179019ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:34.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:34.194+0000 7f178de00640 1 --2- 172.21.15.5:0/3006749761 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1790109120 0x7f179019ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:34.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:34.194+0000 7f178de00640 1 -- 172.21.15.5:0/3006749761 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f179019f650 con 0x7f1790103ff0 2024-09-10T09:54:34.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:34.194+0000 7f178e800640 1 --2- 172.21.15.5:0/3006749761 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1790109120 0x7f179019ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:54:34.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:34.194+0000 7f178de00640 1 --2- 172.21.15.5:0/3006749761 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1790103ff0 0x7f179019a6a0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f178000d950 tx=0x7f178000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:54:34.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:34.194+0000 7f1787400640 1 -- 172.21.15.5:0/3006749761 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1780017030 con 0x7f1790103ff0 2024-09-10T09:54:34.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:34.194+0000 7f1787400640 1 -- 172.21.15.5:0/3006749761 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1780004020 con 0x7f1790103ff0 2024-09-10T09:54:34.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:34.195+0000 7f1794a00640 1 -- 172.21.15.5:0/3006749761 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f179019f940 con 0x7f1790103ff0 2024-09-10T09:54:34.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:34.195+0000 7f1787400640 1 -- 172.21.15.5:0/3006749761 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17800047a0 con 0x7f1790103ff0 2024-09-10T09:54:34.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:34.195+0000 7f1794a00640 1 -- 172.21.15.5:0/3006749761 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17901a7220 con 0x7f1790103ff0 2024-09-10T09:54:34.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:34.198+0000 7f17974cf740 1 Processor -- start 2024-09-10T09:54:34.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:34.199+0000 7f17974cf740 1 -- start start 2024-09-10T09:54:34.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:34.199+0000 7f17974cf740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b7c11e0ac0 0x55b7c11e0ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:34.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:34.199+0000 7f17974cf740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b7c11e13e0 0x55b7c11eb760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:34.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:34.199+0000 7f1787400640 1 -- 172.21.15.5:0/3006749761 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f17800041d0 con 0x7f1790103ff0 2024-09-10T09:54:34.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:34.199+0000 7f17974cf740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b7c11ebca0 0x55b7c11ee090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:34.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:34.199+0000 7f17974cf740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55b7c1147290 con 0x55b7c11e13e0 2024-09-10T09:54:34.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:34.199+0000 7f17974cf740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55b7c10eeb40 con 0x55b7c11e0ac0 2024-09-10T09:54:34.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:34.199+0000 7f17974cf740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55b7c1143250 con 0x55b7c11ebca0 2024-09-10T09:54:34.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:34.200+0000 7f178e800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b7c11e0ac0 0x55b7c11e0ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:34.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:34.200+0000 7f178de00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b7c11e13e0 0x55b7c11eb760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:34.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:34.200+0000 7f178e800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b7c11e0ac0 0x55b7c11e0ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:37392/0 (socket says 172.21.15.5:37392) 2024-09-10T09:54:34.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:34.200+0000 7f178e800640 1 -- 172.21.15.5:0/842009995 learned_addr learned my addr 172.21.15.5:0/842009995 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:54:34.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:34.200+0000 7f1785400640 1 --2- 172.21.15.5:0/842009995 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b7c11ebca0 0x55b7c11ee090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:34.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:34.200+0000 7f178e800640 1 -- 172.21.15.5:0/842009995 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b7c11ebca0 msgr2=0x55b7c11ee090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:54:34.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:34.200+0000 7f178e800640 1 --2- 172.21.15.5:0/842009995 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b7c11ebca0 0x55b7c11ee090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:34.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:34.200+0000 7f178e800640 1 -- 172.21.15.5:0/842009995 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b7c11e13e0 msgr2=0x55b7c11eb760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:54:34.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:34.200+0000 7f178e800640 1 --2- 172.21.15.5:0/842009995 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b7c11e13e0 0x55b7c11eb760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:34.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:34.200+0000 7f178e800640 1 -- 172.21.15.5:0/842009995 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b7c0eb86a0 con 0x55b7c11e0ac0 2024-09-10T09:54:34.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:34.200+0000 7f178e800640 1 --2- 172.21.15.5:0/842009995 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b7c11e0ac0 0x55b7c11e0ea0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f1788010710 tx=0x7f1788010bd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ffebb7d7193a4371 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:54:34.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:34.201+0000 7f177f000640 1 -- 172.21.15.5:0/842009995 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f178800e8d0 con 0x55b7c11e0ac0 2024-09-10T09:54:34.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:34.201+0000 7f177f000640 1 -- 172.21.15.5:0/842009995 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1788012450 con 0x55b7c11e0ac0 2024-09-10T09:54:34.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:34.201+0000 7f177f000640 1 -- 172.21.15.5:0/842009995 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f178801c670 con 0x55b7c11e0ac0 2024-09-10T09:54:34.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:34.201+0000 7f17974cf740 1 -- 172.21.15.5:0/842009995 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b7c11e0ac0 msgr2=0x55b7c11e0ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:34.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:34.201+0000 7f17974cf740 1 --2- 172.21.15.5:0/842009995 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b7c11e0ac0 0x55b7c11e0ea0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f1788010710 tx=0x7f1788010bd0 comp rx=0 tx=0).stop 2024-09-10T09:54:34.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:34.201+0000 7f1787400640 1 --2- 172.21.15.5:0/3006749761 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f17740a2c00 0x7f17740a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:34.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:34.202+0000 7f17974cf740 1 -- 172.21.15.5:0/842009995 shutdown_connections 2024-09-10T09:54:34.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:34.202+0000 7f17974cf740 1 --2- 172.21.15.5:0/842009995 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b7c11ebca0 0x55b7c11ee090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:34.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:34.202+0000 7f17974cf740 1 --2- 172.21.15.5:0/842009995 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b7c11e13e0 0x55b7c11eb760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:34.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:34.202+0000 7f17974cf740 1 --2- 172.21.15.5:0/842009995 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b7c11e0ac0 0x55b7c11e0ea0 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:34.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:34.202+0000 7f17974cf740 1 -- 172.21.15.5:0/842009995 >> 172.21.15.5:0/842009995 conn(0x55b7c11dd050 msgr2=0x55b7c11dd430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:54:34.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:34.202+0000 7f1787400640 1 -- 172.21.15.5:0/3006749761 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f17800ced80 con 0x7f1790103ff0 2024-09-10T09:54:34.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:34.202+0000 7f178e800640 1 --2- 172.21.15.5:0/3006749761 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f17740a2c00 0x7f17740a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:34.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:34.202+0000 7f17974cf740 1 -- 172.21.15.5:0/842009995 shutdown_connections 2024-09-10T09:54:34.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:34.202+0000 7f17974cf740 1 -- 172.21.15.5:0/842009995 wait complete. 2024-09-10T09:54:34.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:34.202+0000 7f17974cf740 1 -- start start 2024-09-10T09:54:34.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:34.203+0000 7f178e800640 1 --2- 172.21.15.5:0/3006749761 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f17740a2c00 0x7f17740a50c0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f1788012d70 tx=0x7f1788012dd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:54:34.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:34.203+0000 7f17974cf740 1 Processor -- start 2024-09-10T09:54:34.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:34.203+0000 7f17974cf740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b7c11e0ac0 0x55b7c11eb590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:34.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:34.203+0000 7f17974cf740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b7c11e13e0 0x55b7c11e8840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:34.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:34.203+0000 7f17974cf740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b7c11ebca0 0x55b7c11e8d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:34.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:34.203+0000 7f17974cf740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55b7c11ee910 con 0x55b7c11e13e0 2024-09-10T09:54:34.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:34.203+0000 7f17974cf740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55b7c1101fa0 con 0x55b7c11ebca0 2024-09-10T09:54:34.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:34.203+0000 7f17974cf740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55b7c11eea90 con 0x55b7c11e0ac0 2024-09-10T09:54:34.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:34.203+0000 7f17974cf740 10 client.0 _ll_register_callbacks cb 0x55b7c0e55ba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:54:34.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:34.204+0000 7f1785400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b7c11e13e0 0x55b7c11e8840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:34.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:34.204+0000 7f1785400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b7c11e13e0 0x55b7c11e8840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38504/0 (socket says 172.21.15.5:38504) 2024-09-10T09:54:34.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:34.204+0000 7f1785400640 1 -- 172.21.15.5:0/3834088769 learned_addr learned my addr 172.21.15.5:0/3834088769 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:54:34.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:34.204+0000 7f178e800640 1 --2- 172.21.15.5:0/3834088769 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b7c11ebca0 0x55b7c11e8d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:34.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:34.204+0000 7f178de00640 1 --2- 172.21.15.5:0/3834088769 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b7c11e0ac0 0x55b7c11eb590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:34.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:34.204+0000 7f1785400640 1 -- 172.21.15.5:0/3834088769 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b7c11e0ac0 msgr2=0x55b7c11eb590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:54:34.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:34.204+0000 7f1785400640 1 --2- 172.21.15.5:0/3834088769 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b7c11e0ac0 0x55b7c11eb590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:34.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:34.204+0000 7f1785400640 1 -- 172.21.15.5:0/3834088769 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b7c11ebca0 msgr2=0x55b7c11e8d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:54:34.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:34.204+0000 7f1785400640 1 --2- 172.21.15.5:0/3834088769 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b7c11ebca0 0x55b7c11e8d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:34.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:34.204+0000 7f1785400640 1 -- 172.21.15.5:0/3834088769 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b7c11e9560 con 0x55b7c11e13e0 2024-09-10T09:54:34.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:34.204+0000 7f178e800640 1 --2- 172.21.15.5:0/3834088769 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b7c11ebca0 0x55b7c11e8d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:54:34.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:34.204+0000 7f1785400640 1 --2- 172.21.15.5:0/3834088769 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b7c11e13e0 0x55b7c11e8840 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f177802a910 tx=0x7f1778031850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a14a01a47e3e59af server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:54:34.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:34.204+0000 7f1773400640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-10T09:54:34.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:34.205+0000 7f1773400640 1 -- 172.21.15.5:0/3834088769 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1778035040 con 0x55b7c11e13e0 2024-09-10T09:54:34.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:34.205+0000 7f1773400640 1 -- 172.21.15.5:0/3834088769 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f1778004240 con 0x55b7c11e13e0 2024-09-10T09:54:34.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:34.205+0000 7f17974cf740 1 -- 172.21.15.5:0/3834088769 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1788012450 con 0x55b7c11e13e0 2024-09-10T09:54:34.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:34.205+0000 7f1773400640 1 -- 172.21.15.5:0/3834088769 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f1778004440 con 0x55b7c11e13e0 2024-09-10T09:54:34.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:34.205+0000 7f17974cf740 10 client.18579 fetch_fsmap learned FSMap version 38 2024-09-10T09:54:34.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:34.205+0000 7f17974cf740 1 -- 172.21.15.5:0/3834088769 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55b7c11fa7d0 con 0x55b7c11e13e0 2024-09-10T09:54:34.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:34.205+0000 7f1773400640 1 -- 172.21.15.5:0/3834088769 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1788012450 con 0x55b7c11e13e0 2024-09-10T09:54:34.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:34.206+0000 7f1773400640 1 -- 172.21.15.5:0/3834088769 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 38) ==== 8386+0+0 (secure 0 0 0) 0x7f177803c5e0 con 0x55b7c11e13e0 2024-09-10T09:54:34.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:34.206+0000 7f17974cf740 10 client.18579 fetch_fsmap finished waiting for FSMap version 38 2024-09-10T09:54:34.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:34.206+0000 7f17974cf740 10 client.18579 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T09:54:34.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:34.206+0000 7f17974cf740 20 client.18579 populate_metadata read hostname 'smithi005' 2024-09-10T09:54:34.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:34.206+0000 7f17974cf740 1 --2- 172.21.15.5:0/3834088769 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55b7c11fbce0 0x55b7c11fe120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:34.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:34.206+0000 7f17974cf740 4 client.18579 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:54:34.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:34.206+0000 7f17974cf740 1 -- 172.21.15.5:0/3834088769 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b7c1154800 con 0x55b7c11fbce0 2024-09-10T09:54:34.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:34.206+0000 7f178de00640 1 --2- 172.21.15.5:0/3834088769 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55b7c11fbce0 0x55b7c11fe120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:34.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:34.207+0000 7f178de00640 1 --2- 172.21.15.5:0/3834088769 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55b7c11fbce0 0x55b7c11fe120 crc :-1 s=READY pgs=237 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8d1f73e7e36be44 server_cookie=ec1fcd2787304842 in_seq=0 out_seq=0 2024-09-10T09:54:34.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:34.207+0000 7f1773400640 10 client.18579 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T09:54:34.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:34.208+0000 7f1773400640 1 -- 172.21.15.5:0/3834088769 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b7c1154800 con 0x55b7c11fbce0 2024-09-10T09:54:34.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:34.208+0000 7f1773400640 10 client.18579 handle_command_reply: tid=0 2024-09-10T09:54:34.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:34.208+0000 7f17974cf740 1 client.18579 shutdown 2024-09-10T09:54:34.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:34.208+0000 7f17974cf740 1 -- 172.21.15.5:0/3834088769 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b7c11e13e0 msgr2=0x55b7c11e8840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:34.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:34.208+0000 7f17974cf740 1 --2- 172.21.15.5:0/3834088769 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b7c11e13e0 0x55b7c11e8840 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f177802a910 tx=0x7f1778031850 comp rx=0 tx=0).stop 2024-09-10T09:54:34.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:34.208+0000 7f17974cf740 1 -- 172.21.15.5:0/3834088769 shutdown_connections 2024-09-10T09:54:34.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:34.208+0000 7f17974cf740 1 --2- 172.21.15.5:0/3834088769 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55b7c11fbce0 0x55b7c11fe120 crc :-1 s=READY pgs=237 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:34.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:34.208+0000 7f17974cf740 1 --2- 172.21.15.5:0/3834088769 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b7c11ebca0 0x55b7c11e8d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:34.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:34.208+0000 7f17974cf740 1 --2- 172.21.15.5:0/3834088769 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b7c11e13e0 0x55b7c11e8840 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:34.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:34.208+0000 7f17974cf740 1 --2- 172.21.15.5:0/3834088769 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b7c11e0ac0 0x55b7c11eb590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:34.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:34.208+0000 7f17974cf740 1 -- 172.21.15.5:0/3834088769 >> 172.21.15.5:0/3834088769 conn(0x55b7c11dd050 msgr2=0x55b7c11dc1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:54:34.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:34.208+0000 7f1773400640 0 client.18579 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T09:54:34.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:34.208+0000 7f17974cf740 1 -- 172.21.15.5:0/3834088769 shutdown_connections 2024-09-10T09:54:34.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:34.208+0000 7f17974cf740 1 -- 172.21.15.5:0/3834088769 wait complete. 2024-09-10T09:54:34.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:34.208+0000 7f17974cf740 20 client.18579 trim_cache size 0 max 16384 2024-09-10T09:54:34.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:34.226+0000 7f17974cf740 1 Processor -- start 2024-09-10T09:54:34.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:34.226+0000 7f17974cf740 1 -- start start 2024-09-10T09:54:34.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:34.227+0000 7f17974cf740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b7c11e0ac0 0x55b7c12039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:34.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:34.227+0000 7f17974cf740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b7c11e13e0 0x55b7c1203f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:34.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:34.227+0000 7f17974cf740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b7c11ebca0 0x55b7c11e5aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:34.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:34.227+0000 7f17974cf740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f17901b1120 con 0x55b7c11ebca0 2024-09-10T09:54:34.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:34.227+0000 7f17974cf740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f17901b2d20 con 0x55b7c11e0ac0 2024-09-10T09:54:34.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:34.227+0000 7f17974cf740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f17901b17f0 con 0x55b7c11e13e0 2024-09-10T09:54:34.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:34.227+0000 7f178e800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b7c11ebca0 0x55b7c11e5aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:34.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:34.227+0000 7f178e800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b7c11ebca0 0x55b7c11e5aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38514/0 (socket says 172.21.15.5:38514) 2024-09-10T09:54:34.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:34.227+0000 7f178e800640 1 -- 172.21.15.5:0/623894455 learned_addr learned my addr 172.21.15.5:0/623894455 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:54:34.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:34.227+0000 7f178de00640 1 --2- 172.21.15.5:0/623894455 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b7c11e0ac0 0x55b7c12039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:34.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:34.227+0000 7f1785400640 1 --2- 172.21.15.5:0/623894455 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b7c11e13e0 0x55b7c1203f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:34.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:34.227+0000 7f178de00640 1 -- 172.21.15.5:0/623894455 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b7c11e13e0 msgr2=0x55b7c1203f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:54:34.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:34.227+0000 7f178de00640 1 --2- 172.21.15.5:0/623894455 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b7c11e13e0 0x55b7c1203f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:34.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:34.227+0000 7f178de00640 1 -- 172.21.15.5:0/623894455 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b7c11ebca0 msgr2=0x55b7c11e5aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:54:34.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:34.227+0000 7f178de00640 1 --2- 172.21.15.5:0/623894455 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b7c11ebca0 0x55b7c11e5aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:34.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:34.227+0000 7f178de00640 1 -- 172.21.15.5:0/623894455 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b7c11e0840 con 0x55b7c11e0ac0 2024-09-10T09:54:34.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:34.227+0000 7f178e800640 1 --2- 172.21.15.5:0/623894455 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b7c11ebca0 0x55b7c11e5aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:54:34.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:34.228+0000 7f178de00640 1 --2- 172.21.15.5:0/623894455 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b7c11e0ac0 0x55b7c12039f0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x55b7c11ddeb0 tx=0x7f17800bb040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=32c13aa60cde9d58 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:54:34.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:34.228+0000 7f177e600640 1 -- 172.21.15.5:0/623894455 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17800d0040 con 0x55b7c11e0ac0 2024-09-10T09:54:34.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:34.228+0000 7f177e600640 1 -- 172.21.15.5:0/623894455 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f178009b890 con 0x55b7c11e0ac0 2024-09-10T09:54:34.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:34.228+0000 7f177e600640 1 -- 172.21.15.5:0/623894455 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f178009f540 con 0x55b7c11e0ac0 2024-09-10T09:54:34.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:34.229+0000 7f17974cf740 1 -- 172.21.15.5:0/623894455 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b7c11e0ac0 msgr2=0x55b7c12039f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:34.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:34.229+0000 7f17974cf740 1 --2- 172.21.15.5:0/623894455 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b7c11e0ac0 0x55b7c12039f0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x55b7c11ddeb0 tx=0x7f17800bb040 comp rx=0 tx=0).stop 2024-09-10T09:54:34.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:34.229+0000 7f17974cf740 1 -- 172.21.15.5:0/623894455 shutdown_connections 2024-09-10T09:54:34.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:34.229+0000 7f17974cf740 1 --2- 172.21.15.5:0/623894455 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b7c11ebca0 0x55b7c11e5aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:34.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:34.229+0000 7f17974cf740 1 --2- 172.21.15.5:0/623894455 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b7c11e13e0 0x55b7c1203f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:34.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:34.229+0000 7f17974cf740 1 --2- 172.21.15.5:0/623894455 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b7c11e0ac0 0x55b7c12039f0 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:34.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:34.229+0000 7f17974cf740 1 -- 172.21.15.5:0/623894455 >> 172.21.15.5:0/623894455 conn(0x55b7c11dd050 msgr2=0x55b7c11ea8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:54:34.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:34.229+0000 7f17974cf740 1 -- 172.21.15.5:0/623894455 shutdown_connections 2024-09-10T09:54:34.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:34.229+0000 7f17974cf740 1 -- 172.21.15.5:0/623894455 wait complete. 2024-09-10T09:54:34.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:34.229+0000 7f17974cf740 1 -- start start 2024-09-10T09:54:34.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:34.230+0000 7f17974cf740 1 Processor -- start 2024-09-10T09:54:34.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:34.230+0000 7f17974cf740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b7c11e0ac0 0x55b7c11f01f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:34.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:34.230+0000 7f17974cf740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b7c11e13e0 0x55b7c11f0730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:34.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:34.230+0000 7f17974cf740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b7c11ebca0 0x55b7c12659c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:34.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:34.230+0000 7f17974cf740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f17901b2650 con 0x55b7c11e0ac0 2024-09-10T09:54:34.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:34.230+0000 7f17974cf740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f17901b1f80 con 0x55b7c11e13e0 2024-09-10T09:54:34.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:34.230+0000 7f17974cf740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f17901af100 con 0x55b7c11ebca0 2024-09-10T09:54:34.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:34.230+0000 7f17974cf740 10 client.0 _ll_register_callbacks cb 0x7f1778005de0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:54:34.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:34.230+0000 7f178de00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b7c11e0ac0 0x55b7c11f01f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:34.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:34.230+0000 7f1785400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b7c11e13e0 0x55b7c11f0730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:34.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:34.230+0000 7f1785400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b7c11e13e0 0x55b7c11f0730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:37424/0 (socket says 172.21.15.5:37424) 2024-09-10T09:54:34.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:34.230+0000 7f1785400640 1 -- 172.21.15.5:0/1901643507 learned_addr learned my addr 172.21.15.5:0/1901643507 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:54:34.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:34.230+0000 7f178de00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b7c11e0ac0 0x55b7c11f01f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38520/0 (socket says 172.21.15.5:38520) 2024-09-10T09:54:34.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:34.230+0000 7f178e800640 1 --2- 172.21.15.5:0/1901643507 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b7c11ebca0 0x55b7c12659c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:34.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:34.231+0000 7f1785400640 1 -- 172.21.15.5:0/1901643507 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b7c11ebca0 msgr2=0x55b7c12659c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:54:34.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:34.231+0000 7f1785400640 1 --2- 172.21.15.5:0/1901643507 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b7c11ebca0 0x55b7c12659c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:34.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:34.231+0000 7f1785400640 1 -- 172.21.15.5:0/1901643507 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b7c11e0ac0 msgr2=0x55b7c11f01f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:54:34.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:34.231+0000 7f1785400640 1 --2- 172.21.15.5:0/1901643507 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b7c11e0ac0 0x55b7c11f01f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:34.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:34.231+0000 7f1785400640 1 -- 172.21.15.5:0/1901643507 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b7c1265fb0 con 0x55b7c11e13e0 2024-09-10T09:54:34.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:34.231+0000 7f1785400640 1 --2- 172.21.15.5:0/1901643507 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b7c11e13e0 0x55b7c11f0730 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x55b7c11e80d0 tx=0x7f17780315b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=97c2d5aa11aa285a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:54:34.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:34.231+0000 7f1773400640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-09-10T09:54:34.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:34.231+0000 7f1773400640 1 -- 172.21.15.5:0/1901643507 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1778057020 con 0x55b7c11e13e0 2024-09-10T09:54:34.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:34.231+0000 7f17974cf740 1 -- 172.21.15.5:0/1901643507 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f178009b890 con 0x55b7c11e13e0 2024-09-10T09:54:34.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:34.231+0000 7f1773400640 1 -- 172.21.15.5:0/1901643507 <== mon.2 v2:172.21.15.102:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f177803c640 con 0x55b7c11e13e0 2024-09-10T09:54:34.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:34.232+0000 7f1773400640 1 -- 172.21.15.5:0/1901643507 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f1778062050 con 0x55b7c11e13e0 2024-09-10T09:54:34.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:34.232+0000 7f17974cf740 10 client.28655 fetch_fsmap learned FSMap version 38 2024-09-10T09:54:34.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:34.232+0000 7f17974cf740 1 -- 172.21.15.5:0/1901643507 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55b7c12672b0 con 0x55b7c11e13e0 2024-09-10T09:54:34.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:34.232+0000 7f1773400640 1 -- 172.21.15.5:0/1901643507 <== mon.2 v2:172.21.15.102:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f178009b890 con 0x55b7c11e13e0 2024-09-10T09:54:34.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:34.232+0000 7f1773400640 1 -- 172.21.15.5:0/1901643507 <== mon.2 v2:172.21.15.102:3300/0 5 ==== fsmap(e 38) ==== 8386+0+0 (secure 0 0 0) 0x7f1778004a30 con 0x55b7c11e13e0 2024-09-10T09:54:34.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:34.232+0000 7f17974cf740 10 client.28655 fetch_fsmap finished waiting for FSMap version 38 2024-09-10T09:54:34.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:34.232+0000 7f17974cf740 10 client.28655 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T09:54:34.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:34.232+0000 7f17974cf740 20 client.28655 populate_metadata read hostname 'smithi005' 2024-09-10T09:54:34.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:34.232+0000 7f17974cf740 1 --2- 172.21.15.5:0/1901643507 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55b7c1267970 0x55b7c1269d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:34.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:34.232+0000 7f17974cf740 4 client.28655 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-10T09:54:34.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:34.232+0000 7f17974cf740 1 -- 172.21.15.5:0/1901643507 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55b7c114ed10 con 0x55b7c1267970 2024-09-10T09:54:34.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:34.233+0000 7f178de00640 1 --2- 172.21.15.5:0/1901643507 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55b7c1267970 0x55b7c1269d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:34.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:34.233+0000 7f178de00640 1 --2- 172.21.15.5:0/1901643507 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55b7c1267970 0x55b7c1269d90 crc :-1 s=READY pgs=238 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=61ef05189eb4af48 server_cookie=c264d5544a52fe2b in_seq=0 out_seq=0 2024-09-10T09:54:34.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:34.233+0000 7f1773400640 10 client.28655 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T09:54:34.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:34.234+0000 7f1773400640 1 -- 172.21.15.5:0/1901643507 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55b7c114ed10 con 0x55b7c1267970 2024-09-10T09:54:34.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:34.234+0000 7f1773400640 10 client.28655 handle_command_reply: tid=0 2024-09-10T09:54:34.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:34.234+0000 7f17974cf740 1 client.28655 shutdown 2024-09-10T09:54:34.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:34.234+0000 7f17974cf740 1 -- 172.21.15.5:0/1901643507 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b7c11e13e0 msgr2=0x55b7c11f0730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:34.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:34.234+0000 7f17974cf740 1 --2- 172.21.15.5:0/1901643507 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b7c11e13e0 0x55b7c11f0730 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x55b7c11e80d0 tx=0x7f17780315b0 comp rx=0 tx=0).stop 2024-09-10T09:54:34.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:34.234+0000 7f17974cf740 1 -- 172.21.15.5:0/1901643507 shutdown_connections 2024-09-10T09:54:34.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:34.234+0000 7f17974cf740 1 --2- 172.21.15.5:0/1901643507 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55b7c1267970 0x55b7c1269d90 crc :-1 s=READY pgs=238 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:34.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:34.235+0000 7f17974cf740 1 --2- 172.21.15.5:0/1901643507 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b7c11ebca0 0x55b7c12659c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:34.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:34.235+0000 7f17974cf740 1 --2- 172.21.15.5:0/1901643507 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b7c11e13e0 0x55b7c11f0730 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:34.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:34.235+0000 7f17974cf740 1 --2- 172.21.15.5:0/1901643507 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b7c11e0ac0 0x55b7c11f01f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:34.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:34.235+0000 7f17974cf740 1 -- 172.21.15.5:0/1901643507 >> 172.21.15.5:0/1901643507 conn(0x55b7c11dd050 msgr2=0x55b7c11ea0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:54:34.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:34.235+0000 7f1773400640 0 client.28655 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T09:54:34.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:34.235+0000 7f17974cf740 1 -- 172.21.15.5:0/1901643507 shutdown_connections 2024-09-10T09:54:34.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:34.235+0000 7f17974cf740 1 -- 172.21.15.5:0/1901643507 wait complete. 2024-09-10T09:54:34.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:34.235+0000 7f17974cf740 20 client.28655 trim_cache size 0 max 16384 2024-09-10T09:54:34.242 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-10T09:54:34.242 INFO:teuthology.orchestra.run.smithi005.stdout: "return_code": 0, 2024-09-10T09:54:34.242 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_tag": "3ae056c8-5927-4ca5-b8d3-cff88e53e371", 2024-09-10T09:54:34.242 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": "asynchronous" 2024-09-10T09:54:34.242 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-10T09:54:34.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:34.235+0000 7f1794a00640 1 -- 172.21.15.5:0/3006749761 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f17740a2c00 msgr2=0x7f17740a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:34.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:34.236+0000 7f1794a00640 1 --2- 172.21.15.5:0/3006749761 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f17740a2c00 0x7f17740a50c0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f1788012d70 tx=0x7f1788012dd0 comp rx=0 tx=0).stop 2024-09-10T09:54:34.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:34.236+0000 7f1794a00640 1 -- 172.21.15.5:0/3006749761 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1790103ff0 msgr2=0x7f179019a6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:34.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:34.236+0000 7f1794a00640 1 --2- 172.21.15.5:0/3006749761 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1790103ff0 0x7f179019a6a0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f178000d950 tx=0x7f178000de10 comp rx=0 tx=0).stop 2024-09-10T09:54:34.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:34.236+0000 7f1794a00640 1 -- 172.21.15.5:0/3006749761 shutdown_connections 2024-09-10T09:54:34.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:34.236+0000 7f1794a00640 1 --2- 172.21.15.5:0/3006749761 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f17740a2c00 0x7f17740a50c0 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:34.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:34.236+0000 7f1794a00640 1 --2- 172.21.15.5:0/3006749761 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1790109120 0x7f179019ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:34.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:34.236+0000 7f1794a00640 1 --2- 172.21.15.5:0/3006749761 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f17901049a0 0x7f179019abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:34.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:34.236+0000 7f1794a00640 1 --2- 172.21.15.5:0/3006749761 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1790103ff0 0x7f179019a6a0 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:34.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:34.236+0000 7f1794a00640 1 -- 172.21.15.5:0/3006749761 >> 172.21.15.5:0/3006749761 conn(0x7f17900ffa90 msgr2=0x7f1790100d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:54:34.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:34.236+0000 7f1794a00640 1 -- 172.21.15.5:0/3006749761 shutdown_connections 2024-09-10T09:54:34.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:34.236+0000 7f1794a00640 1 -- 172.21.15.5:0/3006749761 wait complete. 2024-09-10T09:54:34.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:54:34 smithi159 ceph-mon[35513]: scrub summary: active paths [/] 2024-09-10T09:54:34.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:54:34 smithi159 ceph-mon[35513]: scrub summary: idle+waiting paths [/] 2024-09-10T09:54:34.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:54:34 smithi102 ceph-mon[35442]: scrub summary: active paths [/] 2024-09-10T09:54:34.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:54:34 smithi102 ceph-mon[35442]: scrub summary: idle+waiting paths [/] 2024-09-10T09:54:34.966 INFO:tasks.workunit.client.0.smithi005.stdout: 10 41284 8.59 MB/sec execute 13 sec latency 8988.072 ms 2024-09-10T09:54:35.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:54:34 smithi005 ceph-mon[30350]: scrub summary: active paths [/] 2024-09-10T09:54:35.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:54:34 smithi005 ceph-mon[30350]: scrub summary: idle+waiting paths [/] 2024-09-10T09:54:35.111 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T09:54:35.489 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:54:35.849 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:54:35 smithi005 ceph-mon[30350]: pgmap v880: 129 pgs: 129 active+clean; 534 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 1.3 MiB/s wr, 206 op/s 2024-09-10T09:54:35.849 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:54:35 smithi005 ceph-mon[30350]: scrub queued for path: / 2024-09-10T09:54:35.849 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:54:35 smithi005 ceph-mon[30350]: scrub summary: idle+waiting paths [/] 2024-09-10T09:54:35.849 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:54:35 smithi005 ceph-mon[30350]: scrub summary: active paths [/] 2024-09-10T09:54:35.849 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:54:35 smithi005 ceph-mon[30350]: scrub summary: active paths [/] 2024-09-10T09:54:35.849 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:54:35 smithi005 ceph-mon[30350]: scrub summary: active paths [/] 2024-09-10T09:54:35.849 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:54:35 smithi005 ceph-mon[30350]: scrub summary: idle+waiting paths [/] 2024-09-10T09:54:35.849 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:54:35 smithi005 ceph-mon[30350]: scrub summary: idle+waiting paths [/] 2024-09-10T09:54:35.849 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:54:35 smithi005 ceph-mon[30350]: scrub summary: active paths [/] 2024-09-10T09:54:35.849 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:54:35 smithi005 ceph-mon[30350]: scrub summary: idle+waiting paths [/] 2024-09-10T09:54:35.849 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:54:35 smithi005 ceph-mon[30350]: scrub summary: active paths [/] 2024-09-10T09:54:35.850 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:54:35 smithi005 ceph-mon[30350]: scrub summary: idle+waiting paths [/] 2024-09-10T09:54:35.850 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:54:35 smithi005 ceph-mon[30350]: scrub summary: idle+waiting paths [/] 2024-09-10T09:54:35.850 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:54:35 smithi005 ceph-mon[30350]: scrub summary: active paths [/] 2024-09-10T09:54:35.850 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:54:35 smithi005 ceph-mon[30350]: scrub summary: idle+waiting paths [/] 2024-09-10T09:54:35.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:54:35 smithi159 ceph-mon[35513]: pgmap v880: 129 pgs: 129 active+clean; 534 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 1.3 MiB/s wr, 206 op/s 2024-09-10T09:54:35.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:54:35 smithi159 ceph-mon[35513]: scrub queued for path: / 2024-09-10T09:54:35.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:54:35 smithi159 ceph-mon[35513]: scrub summary: idle+waiting paths [/] 2024-09-10T09:54:35.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:54:35 smithi159 ceph-mon[35513]: scrub summary: active paths [/] 2024-09-10T09:54:35.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:54:35 smithi159 ceph-mon[35513]: scrub summary: active paths [/] 2024-09-10T09:54:35.851 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:54:35 smithi159 ceph-mon[35513]: scrub summary: active paths [/] 2024-09-10T09:54:35.851 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:54:35 smithi159 ceph-mon[35513]: scrub summary: idle+waiting paths [/] 2024-09-10T09:54:35.851 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:54:35 smithi159 ceph-mon[35513]: scrub summary: idle+waiting paths [/] 2024-09-10T09:54:35.851 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:54:35 smithi159 ceph-mon[35513]: scrub summary: active paths [/] 2024-09-10T09:54:35.851 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:54:35 smithi159 ceph-mon[35513]: scrub summary: idle+waiting paths [/] 2024-09-10T09:54:35.851 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:54:35 smithi159 ceph-mon[35513]: scrub summary: active paths [/] 2024-09-10T09:54:35.851 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:54:35 smithi159 ceph-mon[35513]: scrub summary: idle+waiting paths [/] 2024-09-10T09:54:35.851 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:54:35 smithi159 ceph-mon[35513]: scrub summary: idle+waiting paths [/] 2024-09-10T09:54:35.851 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:54:35 smithi159 ceph-mon[35513]: scrub summary: active paths [/] 2024-09-10T09:54:35.851 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:54:35 smithi159 ceph-mon[35513]: scrub summary: idle+waiting paths [/] 2024-09-10T09:54:35.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:54:35 smithi102 ceph-mon[35442]: pgmap v880: 129 pgs: 129 active+clean; 534 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 1.3 MiB/s wr, 206 op/s 2024-09-10T09:54:35.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:54:35 smithi102 ceph-mon[35442]: scrub queued for path: / 2024-09-10T09:54:35.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:54:35 smithi102 ceph-mon[35442]: scrub summary: idle+waiting paths [/] 2024-09-10T09:54:35.885 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:54:35 smithi102 ceph-mon[35442]: scrub summary: active paths [/] 2024-09-10T09:54:35.885 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:54:35 smithi102 ceph-mon[35442]: scrub summary: active paths [/] 2024-09-10T09:54:35.885 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:54:35 smithi102 ceph-mon[35442]: scrub summary: active paths [/] 2024-09-10T09:54:35.885 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:54:35 smithi102 ceph-mon[35442]: scrub summary: idle+waiting paths [/] 2024-09-10T09:54:35.885 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:54:35 smithi102 ceph-mon[35442]: scrub summary: idle+waiting paths [/] 2024-09-10T09:54:35.885 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:54:35 smithi102 ceph-mon[35442]: scrub summary: active paths [/] 2024-09-10T09:54:35.885 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:54:35 smithi102 ceph-mon[35442]: scrub summary: idle+waiting paths [/] 2024-09-10T09:54:35.885 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:54:35 smithi102 ceph-mon[35442]: scrub summary: active paths [/] 2024-09-10T09:54:35.885 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:54:35 smithi102 ceph-mon[35442]: scrub summary: idle+waiting paths [/] 2024-09-10T09:54:35.885 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:54:35 smithi102 ceph-mon[35442]: scrub summary: idle+waiting paths [/] 2024-09-10T09:54:35.885 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:54:35 smithi102 ceph-mon[35442]: scrub summary: active paths [/] 2024-09-10T09:54:35.885 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:54:35 smithi102 ceph-mon[35442]: scrub summary: idle+waiting paths [/] 2024-09-10T09:54:35.966 INFO:tasks.workunit.client.0.smithi005.stdout: 10 41284 7.97 MB/sec execute 14 sec latency 9988.178 ms 2024-09-10T09:54:36.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:36.104+0000 7f0f25c00640 1 -- 172.21.15.5:0/3537117561 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0f20112a50 msgr2=0x7f0f20114e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:36.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:36.104+0000 7f0f25c00640 1 --2- 172.21.15.5:0/3537117561 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0f20112a50 0x7f0f20114e40 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f0f18009ca0 tx=0x7f0f1802a8c0 comp rx=0 tx=0).stop 2024-09-10T09:54:36.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:36.104+0000 7f0f25c00640 1 -- 172.21.15.5:0/3537117561 shutdown_connections 2024-09-10T09:54:36.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:36.104+0000 7f0f25c00640 1 --2- 172.21.15.5:0/3537117561 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0f20112a50 0x7f0f20114e40 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:36.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:36.104+0000 7f0f25c00640 1 --2- 172.21.15.5:0/3537117561 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0f20077040 0x7f0f20112510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:36.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:36.104+0000 7f0f25c00640 1 --2- 172.21.15.5:0/3537117561 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0f20076720 0x7f0f20076b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:36.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:36.104+0000 7f0f25c00640 1 -- 172.21.15.5:0/3537117561 >> 172.21.15.5:0/3537117561 conn(0x7f0f200ffa50 msgr2=0x7f0f20101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:54:36.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:36.104+0000 7f0f25c00640 1 -- 172.21.15.5:0/3537117561 shutdown_connections 2024-09-10T09:54:36.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:36.104+0000 7f0f25c00640 1 -- 172.21.15.5:0/3537117561 wait complete. 2024-09-10T09:54:36.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:36.105+0000 7f0f25c00640 1 Processor -- start 2024-09-10T09:54:36.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:36.105+0000 7f0f25c00640 1 -- start start 2024-09-10T09:54:36.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:36.106+0000 7f0f25c00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0f20076720 0x7f0f201a46b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:36.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:36.106+0000 7f0f25c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0f20077040 0x7f0f201a4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:36.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:36.106+0000 7f0f25c00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0f20112a50 0x7f0f2019e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:36.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:36.106+0000 7f0f25c00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f0f20118590 con 0x7f0f20076720 2024-09-10T09:54:36.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:36.106+0000 7f0f25c00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f0f20118410 con 0x7f0f20077040 2024-09-10T09:54:36.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:36.106+0000 7f0f25c00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f0f20118710 con 0x7f0f20112a50 2024-09-10T09:54:36.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:36.106+0000 7f0f1ea00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0f20076720 0x7f0f201a46b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:36.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:36.106+0000 7f0f1e000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0f20077040 0x7f0f201a4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:36.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:36.106+0000 7f0f1e000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0f20077040 0x7f0f201a4bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:37444/0 (socket says 172.21.15.5:37444) 2024-09-10T09:54:36.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:36.106+0000 7f0f1e000640 1 -- 172.21.15.5:0/2215940796 learned_addr learned my addr 172.21.15.5:0/2215940796 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:54:36.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:36.106+0000 7f0f1f400640 1 --2- 172.21.15.5:0/2215940796 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0f20112a50 0x7f0f2019e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:36.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:36.107+0000 7f0f1e000640 1 -- 172.21.15.5:0/2215940796 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0f20112a50 msgr2=0x7f0f2019e870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:36.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:36.107+0000 7f0f1e000640 1 --2- 172.21.15.5:0/2215940796 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0f20112a50 0x7f0f2019e870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:36.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:36.107+0000 7f0f1e000640 1 -- 172.21.15.5:0/2215940796 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0f20076720 msgr2=0x7f0f201a46b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:36.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:36.107+0000 7f0f1e000640 1 --2- 172.21.15.5:0/2215940796 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0f20076720 0x7f0f201a46b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:36.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:36.107+0000 7f0f1e000640 1 -- 172.21.15.5:0/2215940796 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f2019efa0 con 0x7f0f20077040 2024-09-10T09:54:36.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:36.107+0000 7f0f1e000640 1 --2- 172.21.15.5:0/2215940796 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0f20077040 0x7f0f201a4bf0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f0f0c00e9f0 tx=0x7f0f0c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:54:36.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:36.107+0000 7f0f17400640 1 -- 172.21.15.5:0/2215940796 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f0c00cd80 con 0x7f0f20077040 2024-09-10T09:54:36.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:36.107+0000 7f0f17400640 1 -- 172.21.15.5:0/2215940796 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0f0c010910 con 0x7f0f20077040 2024-09-10T09:54:36.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:36.107+0000 7f0f25c00640 1 -- 172.21.15.5:0/2215940796 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f2019f290 con 0x7f0f20077040 2024-09-10T09:54:36.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:36.107+0000 7f0f17400640 1 -- 172.21.15.5:0/2215940796 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f0c018990 con 0x7f0f20077040 2024-09-10T09:54:36.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:36.107+0000 7f0f25c00640 1 -- 172.21.15.5:0/2215940796 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f201ab580 con 0x7f0f20077040 2024-09-10T09:54:36.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:36.110+0000 7f0f287bc740 1 Processor -- start 2024-09-10T09:54:36.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:36.111+0000 7f0f287bc740 1 -- start start 2024-09-10T09:54:36.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:36.111+0000 7f0f17400640 1 -- 172.21.15.5:0/2215940796 <== mon.2 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f0f0c010ac0 con 0x7f0f20077040 2024-09-10T09:54:36.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:36.111+0000 7f0f287bc740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b66f165b90 0x55b66f165f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:36.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:36.111+0000 7f0f287bc740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b66f1664b0 0x55b66f170830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:36.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:36.111+0000 7f0f287bc740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b66f170d70 0x55b66f173160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:36.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:36.111+0000 7f0f287bc740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55b66f0cc3e0 con 0x55b66f170d70 2024-09-10T09:54:36.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:36.111+0000 7f0f287bc740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55b66f073d80 con 0x55b66f1664b0 2024-09-10T09:54:36.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:36.111+0000 7f0f287bc740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55b66f0c83a0 con 0x55b66f165b90 2024-09-10T09:54:36.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:36.111+0000 7f0f1ea00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b66f1664b0 0x55b66f170830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:36.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:36.111+0000 7f0f1f400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b66f165b90 0x55b66f165f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:36.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:36.111+0000 7f0f1ea00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b66f1664b0 0x55b66f170830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:37452/0 (socket says 172.21.15.5:37452) 2024-09-10T09:54:36.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:36.111+0000 7f0f1f400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b66f165b90 0x55b66f165f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:38346/0 (socket says 172.21.15.5:38346) 2024-09-10T09:54:36.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:36.111+0000 7f0f1ea00640 1 -- 172.21.15.5:0/29378541 learned_addr learned my addr 172.21.15.5:0/29378541 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:54:36.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:36.111+0000 7f0f1e000640 1 --2- 172.21.15.5:0/29378541 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b66f170d70 0x55b66f173160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:36.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:36.112+0000 7f0f1f400640 1 -- 172.21.15.5:0/29378541 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b66f1664b0 msgr2=0x55b66f170830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:54:36.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:36.112+0000 7f0f1f400640 1 --2- 172.21.15.5:0/29378541 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b66f1664b0 0x55b66f170830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:36.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:36.112+0000 7f0f1f400640 1 -- 172.21.15.5:0/29378541 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b66f170d70 msgr2=0x55b66f173160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:54:36.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:36.112+0000 7f0f1f400640 1 --2- 172.21.15.5:0/29378541 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b66f170d70 0x55b66f173160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:36.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:36.112+0000 7f0f1f400640 1 -- 172.21.15.5:0/29378541 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b66f092ae0 con 0x55b66f165b90 2024-09-10T09:54:36.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:36.112+0000 7f0f1ea00640 1 --2- 172.21.15.5:0/29378541 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b66f1664b0 0x55b66f170830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:54:36.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:36.112+0000 7f0f1f400640 1 --2- 172.21.15.5:0/29378541 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b66f165b90 0x55b66f165f70 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f0f1802ade0 tx=0x7f0f18005fa0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a32e0eb7bcdd2b0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:54:36.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:36.113+0000 7f0f0be00640 1 -- 172.21.15.5:0/29378541 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f18031db0 con 0x55b66f165b90 2024-09-10T09:54:36.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:36.113+0000 7f0f0be00640 1 -- 172.21.15.5:0/29378541 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0f18036040 con 0x55b66f165b90 2024-09-10T09:54:36.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:36.113+0000 7f0f0be00640 1 -- 172.21.15.5:0/29378541 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f180042a0 con 0x55b66f165b90 2024-09-10T09:54:36.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:36.113+0000 7f0f17400640 1 --2- 172.21.15.5:0/2215940796 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f0f040a2b80 0x7f0f040a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:36.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:36.113+0000 7f0f17400640 1 -- 172.21.15.5:0/2215940796 <== mon.2 v2:172.21.15.102:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f0f0c0cf6c0 con 0x7f0f20077040 2024-09-10T09:54:36.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:36.113+0000 7f0f1ea00640 1 --2- 172.21.15.5:0/2215940796 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f0f040a2b80 0x7f0f040a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:36.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:36.113+0000 7f0f287bc740 1 -- 172.21.15.5:0/29378541 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b66f165b90 msgr2=0x55b66f165f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:36.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:36.113+0000 7f0f287bc740 1 --2- 172.21.15.5:0/29378541 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b66f165b90 0x55b66f165f70 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f0f1802ade0 tx=0x7f0f18005fa0 comp rx=0 tx=0).stop 2024-09-10T09:54:36.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:36.113+0000 7f0f287bc740 1 -- 172.21.15.5:0/29378541 shutdown_connections 2024-09-10T09:54:36.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:36.113+0000 7f0f287bc740 1 --2- 172.21.15.5:0/29378541 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b66f170d70 0x55b66f173160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:36.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:36.113+0000 7f0f287bc740 1 --2- 172.21.15.5:0/29378541 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b66f1664b0 0x55b66f170830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:36.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:36.113+0000 7f0f287bc740 1 --2- 172.21.15.5:0/29378541 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b66f165b90 0x55b66f165f70 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:36.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:36.113+0000 7f0f287bc740 1 -- 172.21.15.5:0/29378541 >> 172.21.15.5:0/29378541 conn(0x55b66f162120 msgr2=0x55b66f162500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:54:36.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:36.114+0000 7f0f287bc740 1 -- 172.21.15.5:0/29378541 shutdown_connections 2024-09-10T09:54:36.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:36.114+0000 7f0f287bc740 1 -- 172.21.15.5:0/29378541 wait complete. 2024-09-10T09:54:36.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:36.114+0000 7f0f1ea00640 1 --2- 172.21.15.5:0/2215940796 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f0f040a2b80 0x7f0f040a5040 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x55b66f162f80 tx=0x7f0f1000b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:54:36.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:36.114+0000 7f0f287bc740 1 -- start start 2024-09-10T09:54:36.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:36.115+0000 7f0f287bc740 1 Processor -- start 2024-09-10T09:54:36.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:36.115+0000 7f0f287bc740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b66f165b90 0x55b66f16dd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:36.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:36.115+0000 7f0f287bc740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b66f1664b0 0x55b66f16e280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:36.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:36.115+0000 7f0f287bc740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b66f170d70 0x55b66f16e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:36.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:36.115+0000 7f0f287bc740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55b66f173860 con 0x55b66f1664b0 2024-09-10T09:54:36.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:36.115+0000 7f0f287bc740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55b66f0870f0 con 0x55b66f170d70 2024-09-10T09:54:36.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:36.115+0000 7f0f287bc740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55b66f1739e0 con 0x55b66f165b90 2024-09-10T09:54:36.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:36.115+0000 7f0f287bc740 10 client.0 _ll_register_callbacks cb 0x55b66f0789e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:54:36.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:36.115+0000 7f0f1e000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b66f1664b0 0x55b66f16e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:36.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:36.115+0000 7f0f1e000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b66f1664b0 0x55b66f16e280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38540/0 (socket says 172.21.15.5:38540) 2024-09-10T09:54:36.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:36.115+0000 7f0f1e000640 1 -- 172.21.15.5:0/2281133750 learned_addr learned my addr 172.21.15.5:0/2281133750 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:54:36.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:36.115+0000 7f0f1ea00640 1 --2- 172.21.15.5:0/2281133750 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b66f165b90 0x55b66f16dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:36.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:36.116+0000 7f0f1f400640 1 --2- 172.21.15.5:0/2281133750 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b66f170d70 0x55b66f16e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:36.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:36.116+0000 7f0f1e000640 1 -- 172.21.15.5:0/2281133750 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b66f165b90 msgr2=0x55b66f16dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:54:36.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:36.116+0000 7f0f1e000640 1 --2- 172.21.15.5:0/2281133750 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b66f165b90 0x55b66f16dd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:36.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:36.116+0000 7f0f1e000640 1 -- 172.21.15.5:0/2281133750 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b66f170d70 msgr2=0x55b66f16e7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:54:36.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:36.116+0000 7f0f1e000640 1 --2- 172.21.15.5:0/2281133750 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b66f170d70 0x55b66f16e7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:36.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:36.116+0000 7f0f1e000640 1 -- 172.21.15.5:0/2281133750 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b66ee3d3c0 con 0x55b66f1664b0 2024-09-10T09:54:36.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:36.116+0000 7f0f1e000640 1 --2- 172.21.15.5:0/2281133750 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b66f1664b0 0x55b66f16e280 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f0f0c09b8e0 tx=0x7f0f0c09b540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28963b8b1be6041c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:54:36.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:36.116+0000 7f0f08c00640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-10T09:54:36.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:36.116+0000 7f0f08c00640 1 -- 172.21.15.5:0/2281133750 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f0c09ce60 con 0x55b66f1664b0 2024-09-10T09:54:36.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:36.116+0000 7f0f08c00640 1 -- 172.21.15.5:0/2281133750 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f0f0c0cfb70 con 0x55b66f1664b0 2024-09-10T09:54:36.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:36.116+0000 7f0f287bc740 1 -- 172.21.15.5:0/2281133750 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0f18036040 con 0x55b66f1664b0 2024-09-10T09:54:36.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:36.117+0000 7f0f08c00640 1 -- 172.21.15.5:0/2281133750 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f0f0c086070 con 0x55b66f1664b0 2024-09-10T09:54:36.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:36.117+0000 7f0f287bc740 10 client.18597 fetch_fsmap learned FSMap version 38 2024-09-10T09:54:36.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:36.117+0000 7f0f287bc740 1 -- 172.21.15.5:0/2281133750 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55b66f17f5e0 con 0x55b66f1664b0 2024-09-10T09:54:36.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:36.117+0000 7f0f08c00640 1 -- 172.21.15.5:0/2281133750 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0f18036040 con 0x55b66f1664b0 2024-09-10T09:54:36.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:36.117+0000 7f0f08c00640 1 -- 172.21.15.5:0/2281133750 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 38) ==== 8386+0+0 (secure 0 0 0) 0x7f0f0c018cc0 con 0x55b66f1664b0 2024-09-10T09:54:36.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:36.117+0000 7f0f287bc740 10 client.18597 fetch_fsmap finished waiting for FSMap version 38 2024-09-10T09:54:36.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:36.117+0000 7f0f287bc740 10 client.18597 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T09:54:36.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:36.117+0000 7f0f287bc740 20 client.18597 populate_metadata read hostname 'smithi005' 2024-09-10T09:54:36.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:36.117+0000 7f0f287bc740 1 --2- 172.21.15.5:0/2281133750 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55b66f180aa0 0x55b66f182e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:36.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:36.117+0000 7f0f287bc740 4 client.18597 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:54:36.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:36.117+0000 7f0f287bc740 1 -- 172.21.15.5:0/2281133750 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b66f0d9950 con 0x55b66f180aa0 2024-09-10T09:54:36.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:36.118+0000 7f0f1ea00640 1 --2- 172.21.15.5:0/2281133750 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55b66f180aa0 0x55b66f182e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:36.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:36.118+0000 7f0f1ea00640 1 --2- 172.21.15.5:0/2281133750 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55b66f180aa0 0x55b66f182e90 crc :-1 s=READY pgs=239 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=47b68535a0ee321f server_cookie=bc52371981f8c83b in_seq=0 out_seq=0 2024-09-10T09:54:36.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:36.118+0000 7f0f08c00640 10 client.18597 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T09:54:36.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:36.119+0000 7f0f08c00640 1 -- 172.21.15.5:0/2281133750 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b66f0d9950 con 0x55b66f180aa0 2024-09-10T09:54:36.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:36.119+0000 7f0f08c00640 10 client.18597 handle_command_reply: tid=0 2024-09-10T09:54:36.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:36.119+0000 7f0f287bc740 1 client.18597 shutdown 2024-09-10T09:54:36.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:36.119+0000 7f0f287bc740 1 -- 172.21.15.5:0/2281133750 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b66f1664b0 msgr2=0x55b66f16e280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:36.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:36.119+0000 7f0f287bc740 1 --2- 172.21.15.5:0/2281133750 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b66f1664b0 0x55b66f16e280 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f0f0c09b8e0 tx=0x7f0f0c09b540 comp rx=0 tx=0).stop 2024-09-10T09:54:36.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:36.119+0000 7f0f287bc740 1 -- 172.21.15.5:0/2281133750 shutdown_connections 2024-09-10T09:54:36.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:36.119+0000 7f0f287bc740 1 --2- 172.21.15.5:0/2281133750 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55b66f180aa0 0x55b66f182e90 crc :-1 s=READY pgs=239 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:36.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:36.119+0000 7f0f287bc740 1 --2- 172.21.15.5:0/2281133750 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b66f170d70 0x55b66f16e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:36.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:36.119+0000 7f0f287bc740 1 --2- 172.21.15.5:0/2281133750 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b66f1664b0 0x55b66f16e280 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:36.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:36.119+0000 7f0f287bc740 1 --2- 172.21.15.5:0/2281133750 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b66f165b90 0x55b66f16dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:36.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:36.119+0000 7f0f287bc740 1 -- 172.21.15.5:0/2281133750 >> 172.21.15.5:0/2281133750 conn(0x55b66f162120 msgr2=0x55b66f167e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:54:36.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:36.119+0000 7f0f08c00640 0 client.18597 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T09:54:36.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:36.120+0000 7f0f287bc740 1 -- 172.21.15.5:0/2281133750 shutdown_connections 2024-09-10T09:54:36.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:36.120+0000 7f0f287bc740 1 -- 172.21.15.5:0/2281133750 wait complete. 2024-09-10T09:54:36.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:36.120+0000 7f0f287bc740 20 client.18597 trim_cache size 0 max 16384 2024-09-10T09:54:36.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:36.138+0000 7f0f287bc740 1 Processor -- start 2024-09-10T09:54:36.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:36.138+0000 7f0f287bc740 1 -- start start 2024-09-10T09:54:36.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:36.138+0000 7f0f287bc740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b66f165b90 0x55b66f162500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:36.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:36.138+0000 7f0f287bc740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b66f1664b0 0x55b66f16fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:36.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:36.138+0000 7f0f287bc740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b66f170d70 0x55b66f188740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:36.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:36.138+0000 7f0f287bc740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f0f201b5400 con 0x55b66f1664b0 2024-09-10T09:54:36.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:36.138+0000 7f0f287bc740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f0f201b7000 con 0x55b66f165b90 2024-09-10T09:54:36.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:36.138+0000 7f0f287bc740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f0f201b5ad0 con 0x55b66f170d70 2024-09-10T09:54:36.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:36.138+0000 7f0f1e000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b66f1664b0 0x55b66f16fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:36.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:36.138+0000 7f0f1e000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b66f1664b0 0x55b66f16fcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38552/0 (socket says 172.21.15.5:38552) 2024-09-10T09:54:36.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:36.138+0000 7f0f1ea00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b66f165b90 0x55b66f162500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:36.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:36.138+0000 7f0f1e000640 1 -- 172.21.15.5:0/3350589609 learned_addr learned my addr 172.21.15.5:0/3350589609 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:54:36.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:36.139+0000 7f0f1f400640 1 --2- 172.21.15.5:0/3350589609 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b66f170d70 0x55b66f188740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:36.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:36.139+0000 7f0f1e000640 1 -- 172.21.15.5:0/3350589609 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b66f170d70 msgr2=0x55b66f188740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:54:36.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:36.139+0000 7f0f1e000640 1 --2- 172.21.15.5:0/3350589609 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b66f170d70 0x55b66f188740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:36.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:36.139+0000 7f0f1e000640 1 -- 172.21.15.5:0/3350589609 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b66f165b90 msgr2=0x55b66f162500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:54:36.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:36.139+0000 7f0f1e000640 1 --2- 172.21.15.5:0/3350589609 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b66f165b90 0x55b66f162500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:36.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:36.139+0000 7f0f1e000640 1 -- 172.21.15.5:0/3350589609 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b66f16ccf0 con 0x55b66f1664b0 2024-09-10T09:54:36.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:36.139+0000 7f0f1ea00640 1 --2- 172.21.15.5:0/3350589609 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b66f165b90 0x55b66f162500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:54:36.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:36.139+0000 7f0f1e000640 1 --2- 172.21.15.5:0/3350589609 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b66f1664b0 0x55b66f16fcb0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f0f0c0985b0 tx=0x7f0f0c0a0060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e23ec6c985e24cf4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:54:36.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:36.139+0000 7f0f0b400640 1 -- 172.21.15.5:0/3350589609 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f0c09ce60 con 0x55b66f1664b0 2024-09-10T09:54:36.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:36.140+0000 7f0f0b400640 1 -- 172.21.15.5:0/3350589609 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0f0c097a80 con 0x55b66f1664b0 2024-09-10T09:54:36.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:36.140+0000 7f0f0b400640 1 -- 172.21.15.5:0/3350589609 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f0c09aaa0 con 0x55b66f1664b0 2024-09-10T09:54:36.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:36.140+0000 7f0f287bc740 1 -- 172.21.15.5:0/3350589609 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b66f1664b0 msgr2=0x55b66f16fcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:36.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:36.140+0000 7f0f287bc740 1 --2- 172.21.15.5:0/3350589609 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b66f1664b0 0x55b66f16fcb0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f0f0c0985b0 tx=0x7f0f0c0a0060 comp rx=0 tx=0).stop 2024-09-10T09:54:36.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:36.140+0000 7f0f287bc740 1 -- 172.21.15.5:0/3350589609 shutdown_connections 2024-09-10T09:54:36.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:36.140+0000 7f0f287bc740 1 --2- 172.21.15.5:0/3350589609 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b66f170d70 0x55b66f188740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:36.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:36.140+0000 7f0f287bc740 1 --2- 172.21.15.5:0/3350589609 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b66f1664b0 0x55b66f16fcb0 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:36.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:36.140+0000 7f0f287bc740 1 --2- 172.21.15.5:0/3350589609 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b66f165b90 0x55b66f162500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:36.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:36.140+0000 7f0f287bc740 1 -- 172.21.15.5:0/3350589609 >> 172.21.15.5:0/3350589609 conn(0x55b66f162120 msgr2=0x55b66f16f5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:54:36.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:36.140+0000 7f0f287bc740 1 -- 172.21.15.5:0/3350589609 shutdown_connections 2024-09-10T09:54:36.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:36.140+0000 7f0f287bc740 1 -- 172.21.15.5:0/3350589609 wait complete. 2024-09-10T09:54:36.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:36.141+0000 7f0f287bc740 1 -- start start 2024-09-10T09:54:36.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:36.141+0000 7f0f287bc740 1 Processor -- start 2024-09-10T09:54:36.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:36.141+0000 7f0f287bc740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b66f165b90 0x55b66f179400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:36.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:36.141+0000 7f0f287bc740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b66f1664b0 0x55b66f16ab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:36.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:36.141+0000 7f0f287bc740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b66f170d70 0x55b66f16b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:36.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:36.141+0000 7f0f287bc740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f0f201b6930 con 0x55b66f1664b0 2024-09-10T09:54:36.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:36.141+0000 7f0f287bc740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f0f201b6260 con 0x55b66f170d70 2024-09-10T09:54:36.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:36.141+0000 7f0f287bc740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f0f201b33e0 con 0x55b66f165b90 2024-09-10T09:54:36.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:36.141+0000 7f0f287bc740 10 client.0 _ll_register_callbacks cb 0x7f0f0c00cf20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:54:36.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:36.142+0000 7f0f1e000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b66f1664b0 0x55b66f16ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:36.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:36.142+0000 7f0f1e000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b66f1664b0 0x55b66f16ab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38562/0 (socket says 172.21.15.5:38562) 2024-09-10T09:54:36.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:36.142+0000 7f0f1e000640 1 -- 172.21.15.5:0/3736872501 learned_addr learned my addr 172.21.15.5:0/3736872501 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:54:36.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:36.142+0000 7f0f1ea00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b66f165b90 0x55b66f179400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:36.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:36.142+0000 7f0f1f400640 1 --2- 172.21.15.5:0/3736872501 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b66f170d70 0x55b66f16b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:36.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:36.142+0000 7f0f1e000640 1 -- 172.21.15.5:0/3736872501 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b66f165b90 msgr2=0x55b66f179400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:54:36.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:36.142+0000 7f0f1e000640 1 --2- 172.21.15.5:0/3736872501 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b66f165b90 0x55b66f179400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:36.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:36.142+0000 7f0f1e000640 1 -- 172.21.15.5:0/3736872501 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b66f170d70 msgr2=0x55b66f16b0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:54:36.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:36.142+0000 7f0f1e000640 1 --2- 172.21.15.5:0/3736872501 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b66f170d70 0x55b66f16b0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:36.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:36.142+0000 7f0f1e000640 1 -- 172.21.15.5:0/3736872501 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b66f16b910 con 0x55b66f1664b0 2024-09-10T09:54:36.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:36.142+0000 7f0f1ea00640 1 --2- 172.21.15.5:0/3736872501 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b66f165b90 0x55b66f179400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:54:36.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:36.142+0000 7f0f1e000640 1 --2- 172.21.15.5:0/3736872501 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b66f1664b0 0x55b66f16ab40 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f0f0c00c6a0 tx=0x7f0f0c09b540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71114d1e8978a28a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:54:36.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:36.142+0000 7f0f08c00640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-10T09:54:36.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:36.143+0000 7f0f08c00640 1 -- 172.21.15.5:0/3736872501 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f0c0b8070 con 0x55b66f1664b0 2024-09-10T09:54:36.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:36.143+0000 7f0f08c00640 1 -- 172.21.15.5:0/3736872501 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f0f0c010430 con 0x55b66f1664b0 2024-09-10T09:54:36.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:36.143+0000 7f0f287bc740 1 -- 172.21.15.5:0/3736872501 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0f0c097a80 con 0x55b66f1664b0 2024-09-10T09:54:36.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:36.143+0000 7f0f08c00640 1 -- 172.21.15.5:0/3736872501 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f0f0c09ce60 con 0x55b66f1664b0 2024-09-10T09:54:36.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:36.143+0000 7f0f287bc740 10 client.18603 fetch_fsmap learned FSMap version 38 2024-09-10T09:54:36.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:36.143+0000 7f0f287bc740 1 -- 172.21.15.5:0/3736872501 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55b66f16b910 con 0x55b66f1664b0 2024-09-10T09:54:36.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:36.143+0000 7f0f08c00640 1 -- 172.21.15.5:0/3736872501 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0f0c097a80 con 0x55b66f1664b0 2024-09-10T09:54:36.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:36.143+0000 7f0f08c00640 1 -- 172.21.15.5:0/3736872501 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 38) ==== 8386+0+0 (secure 0 0 0) 0x7f0f0c0a1170 con 0x55b66f1664b0 2024-09-10T09:54:36.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:36.143+0000 7f0f287bc740 10 client.18603 fetch_fsmap finished waiting for FSMap version 38 2024-09-10T09:54:36.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:36.143+0000 7f0f287bc740 10 client.18603 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T09:54:36.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:36.143+0000 7f0f287bc740 20 client.18603 populate_metadata read hostname 'smithi005' 2024-09-10T09:54:36.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:36.143+0000 7f0f287bc740 1 --2- 172.21.15.5:0/3736872501 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55b66f1eb9a0 0x55b66f1edd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:36.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:36.143+0000 7f0f287bc740 4 client.18603 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T09:54:36.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:36.143+0000 7f0f287bc740 1 -- 172.21.15.5:0/3736872501 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55b66f0d3e60 con 0x55b66f1eb9a0 2024-09-10T09:54:36.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:36.144+0000 7f0f1ea00640 1 --2- 172.21.15.5:0/3736872501 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55b66f1eb9a0 0x55b66f1edd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:36.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:36.144+0000 7f0f1ea00640 1 --2- 172.21.15.5:0/3736872501 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55b66f1eb9a0 0x55b66f1edd90 crc :-1 s=READY pgs=240 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3d6315c9f2cb6ca8 server_cookie=e31aef2a6e2f5b3a in_seq=0 out_seq=0 2024-09-10T09:54:36.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:36.144+0000 7f0f08c00640 10 client.18603 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T09:54:36.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:36.145+0000 7f0f08c00640 1 -- 172.21.15.5:0/3736872501 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55b66f0d3e60 con 0x55b66f1eb9a0 2024-09-10T09:54:36.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:36.145+0000 7f0f08c00640 10 client.18603 handle_command_reply: tid=0 2024-09-10T09:54:36.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:36.145+0000 7f0f287bc740 1 client.18603 shutdown 2024-09-10T09:54:36.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:36.145+0000 7f0f287bc740 1 -- 172.21.15.5:0/3736872501 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b66f1664b0 msgr2=0x55b66f16ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:36.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:36.145+0000 7f0f287bc740 1 --2- 172.21.15.5:0/3736872501 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b66f1664b0 0x55b66f16ab40 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f0f0c00c6a0 tx=0x7f0f0c09b540 comp rx=0 tx=0).stop 2024-09-10T09:54:36.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:36.145+0000 7f0f287bc740 1 -- 172.21.15.5:0/3736872501 shutdown_connections 2024-09-10T09:54:36.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:36.145+0000 7f0f287bc740 1 --2- 172.21.15.5:0/3736872501 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55b66f1eb9a0 0x55b66f1edd90 crc :-1 s=READY pgs=240 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:36.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:36.145+0000 7f0f287bc740 1 --2- 172.21.15.5:0/3736872501 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b66f170d70 0x55b66f16b0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:36.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:36.145+0000 7f0f287bc740 1 --2- 172.21.15.5:0/3736872501 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b66f1664b0 0x55b66f16ab40 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:36.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:36.145+0000 7f0f287bc740 1 --2- 172.21.15.5:0/3736872501 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b66f165b90 0x55b66f179400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:36.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:36.145+0000 7f0f287bc740 1 -- 172.21.15.5:0/3736872501 >> 172.21.15.5:0/3736872501 conn(0x55b66f162120 msgr2=0x55b66f16fdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:54:36.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:36.145+0000 7f0f08c00640 0 client.18603 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T09:54:36.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:36.145+0000 7f0f287bc740 1 -- 172.21.15.5:0/3736872501 shutdown_connections 2024-09-10T09:54:36.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:36.145+0000 7f0f287bc740 1 -- 172.21.15.5:0/3736872501 wait complete. 2024-09-10T09:54:36.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:36.145+0000 7f0f287bc740 20 client.18603 trim_cache size 0 max 16384 2024-09-10T09:54:36.152 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-10T09:54:36.152 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-10T09:54:36.152 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2024-09-10T09:54:36.152 INFO:teuthology.orchestra.run.smithi005.stdout: "3ae056c8-5927-4ca5-b8d3-cff88e53e371": { 2024-09-10T09:54:36.152 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2024-09-10T09:54:36.152 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "3ae056c8-5927-4ca5-b8d3-cff88e53e371", 2024-09-10T09:54:36.152 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2024-09-10T09:54:36.152 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T09:54:36.152 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T09:54:36.152 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-10T09:54:36.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:36.146+0000 7f0f25c00640 1 -- 172.21.15.5:0/2215940796 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f0f040a2b80 msgr2=0x7f0f040a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:36.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:36.146+0000 7f0f25c00640 1 --2- 172.21.15.5:0/2215940796 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f0f040a2b80 0x7f0f040a5040 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x55b66f162f80 tx=0x7f0f1000b390 comp rx=0 tx=0).stop 2024-09-10T09:54:36.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:36.146+0000 7f0f25c00640 1 -- 172.21.15.5:0/2215940796 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0f20077040 msgr2=0x7f0f201a4bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:36.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:36.146+0000 7f0f25c00640 1 --2- 172.21.15.5:0/2215940796 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0f20077040 0x7f0f201a4bf0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f0f0c00e9f0 tx=0x7f0f0c00eeb0 comp rx=0 tx=0).stop 2024-09-10T09:54:36.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:36.146+0000 7f0f25c00640 1 -- 172.21.15.5:0/2215940796 shutdown_connections 2024-09-10T09:54:36.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:36.146+0000 7f0f25c00640 1 --2- 172.21.15.5:0/2215940796 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f0f040a2b80 0x7f0f040a5040 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:36.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:36.146+0000 7f0f25c00640 1 --2- 172.21.15.5:0/2215940796 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0f20112a50 0x7f0f2019e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:36.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:36.146+0000 7f0f25c00640 1 --2- 172.21.15.5:0/2215940796 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0f20077040 0x7f0f201a4bf0 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:36.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:36.146+0000 7f0f25c00640 1 --2- 172.21.15.5:0/2215940796 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0f20076720 0x7f0f201a46b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:36.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:36.146+0000 7f0f25c00640 1 -- 172.21.15.5:0/2215940796 >> 172.21.15.5:0/2215940796 conn(0x7f0f200ffa50 msgr2=0x7f0f20101e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:54:36.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:36.146+0000 7f0f25c00640 1 -- 172.21.15.5:0/2215940796 shutdown_connections 2024-09-10T09:54:36.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:36.146+0000 7f0f25c00640 1 -- 172.21.15.5:0/2215940796 wait complete. 2024-09-10T09:54:36.954 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T09:54:36.966 INFO:tasks.workunit.client.0.smithi005.stdout: 10 41284 7.44 MB/sec execute 15 sec latency 10988.228 ms 2024-09-10T09:54:37.010 INFO:tasks.cephfs.filesystem:scrub status for tag:3ae056c8-5927-4ca5-b8d3-cff88e53e371 - {'path': '/', 'tag': '3ae056c8-5927-4ca5-b8d3-cff88e53e371', 'options': 'recursive,force'} 2024-09-10T09:54:37.337 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:54:37.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:54:37 smithi005 ceph-mon[30350]: pgmap v881: 129 pgs: 129 active+clean; 535 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 1.2 MiB/s wr, 210 op/s 2024-09-10T09:54:37.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:54:37 smithi005 ceph-mon[30350]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T09:54:37.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:54:37 smithi159 ceph-mon[35513]: pgmap v881: 129 pgs: 129 active+clean; 535 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 1.2 MiB/s wr, 210 op/s 2024-09-10T09:54:37.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:54:37 smithi159 ceph-mon[35513]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T09:54:37.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:54:37 smithi102 ceph-mon[35442]: pgmap v881: 129 pgs: 129 active+clean; 535 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 1.2 MiB/s wr, 210 op/s 2024-09-10T09:54:37.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:54:37 smithi102 ceph-mon[35442]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T09:54:37.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:37.936+0000 7f15c8000640 1 -- 172.21.15.5:0/2845158183 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f15c0109120 msgr2=0x7f15c0110ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:37.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:37.936+0000 7f15c8000640 1 --2- 172.21.15.5:0/2845158183 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f15c0109120 0x7f15c0110ca0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f15b8009ca0 tx=0x7f15b802a8c0 comp rx=0 tx=0).stop 2024-09-10T09:54:37.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:37.936+0000 7f15c8000640 1 -- 172.21.15.5:0/2845158183 shutdown_connections 2024-09-10T09:54:37.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:37.936+0000 7f15c8000640 1 --2- 172.21.15.5:0/2845158183 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f15c0109120 0x7f15c0110ca0 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:37.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:37.936+0000 7f15c8000640 1 --2- 172.21.15.5:0/2845158183 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f15c01049a0 0x7f15c01089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:37.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:37.936+0000 7f15c8000640 1 --2- 172.21.15.5:0/2845158183 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f15c0103ff0 0x7f15c01043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:37.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:37.936+0000 7f15c8000640 1 -- 172.21.15.5:0/2845158183 >> 172.21.15.5:0/2845158183 conn(0x7f15c00ffa90 msgr2=0x7f15c0101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:54:37.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:37.936+0000 7f15c8000640 1 -- 172.21.15.5:0/2845158183 shutdown_connections 2024-09-10T09:54:37.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:37.936+0000 7f15c8000640 1 -- 172.21.15.5:0/2845158183 wait complete. 2024-09-10T09:54:37.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:37.937+0000 7f15c8000640 1 Processor -- start 2024-09-10T09:54:37.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:37.937+0000 7f15c8000640 1 -- start start 2024-09-10T09:54:37.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:37.938+0000 7f15c8000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f15c0103ff0 0x7f15c019a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:37.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:37.938+0000 7f15c8000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f15c01049a0 0x7f15c019ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:37.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:37.938+0000 7f15c8000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f15c0109120 0x7f15c019eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:37.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:37.938+0000 7f15c8000640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f15c01141a0 con 0x7f15c01049a0 2024-09-10T09:54:37.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:37.938+0000 7f15c8000640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f15c0114020 con 0x7f15c0103ff0 2024-09-10T09:54:37.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:37.938+0000 7f15c8000640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f15c0114320 con 0x7f15c0109120 2024-09-10T09:54:37.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:37.938+0000 7f15c4c00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f15c01049a0 0x7f15c019ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:37.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:37.938+0000 7f15c5600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f15c0103ff0 0x7f15c019a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:37.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:37.938+0000 7f15c5600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f15c0103ff0 0x7f15c019a5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:37486/0 (socket says 172.21.15.5:37486) 2024-09-10T09:54:37.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:37.938+0000 7f15c4c00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f15c01049a0 0x7f15c019ab20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38588/0 (socket says 172.21.15.5:38588) 2024-09-10T09:54:37.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:37.938+0000 7f15c5600640 1 -- 172.21.15.5:0/1061472057 learned_addr learned my addr 172.21.15.5:0/1061472057 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:54:37.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:37.938+0000 7f15c6000640 1 --2- 172.21.15.5:0/1061472057 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f15c0109120 0x7f15c019eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:37.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:37.939+0000 7f15c5600640 1 -- 172.21.15.5:0/1061472057 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f15c0109120 msgr2=0x7f15c019eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:37.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:37.939+0000 7f15c5600640 1 --2- 172.21.15.5:0/1061472057 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f15c0109120 0x7f15c019eeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:37.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:37.939+0000 7f15c5600640 1 -- 172.21.15.5:0/1061472057 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f15c01049a0 msgr2=0x7f15c019ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:37.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:37.939+0000 7f15c5600640 1 --2- 172.21.15.5:0/1061472057 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f15c01049a0 0x7f15c019ab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:37.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:37.939+0000 7f15c5600640 1 -- 172.21.15.5:0/1061472057 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15c019f590 con 0x7f15c0103ff0 2024-09-10T09:54:37.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:37.939+0000 7f15c6000640 1 --2- 172.21.15.5:0/1061472057 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f15c0109120 0x7f15c019eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:54:37.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:37.939+0000 7f15c5600640 1 --2- 172.21.15.5:0/1061472057 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f15c0103ff0 0x7f15c019a5e0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f15b400d950 tx=0x7f15b400de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:54:37.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:37.939+0000 7f15be000640 1 -- 172.21.15.5:0/1061472057 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15b4017030 con 0x7f15c0103ff0 2024-09-10T09:54:37.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:37.939+0000 7f15be000640 1 -- 172.21.15.5:0/1061472057 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f15b4004020 con 0x7f15c0103ff0 2024-09-10T09:54:37.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:37.939+0000 7f15be000640 1 -- 172.21.15.5:0/1061472057 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15b40047a0 con 0x7f15c0103ff0 2024-09-10T09:54:37.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:37.939+0000 7f15c8000640 1 -- 172.21.15.5:0/1061472057 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15c019f880 con 0x7f15c0103ff0 2024-09-10T09:54:37.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:37.940+0000 7f15c8000640 1 -- 172.21.15.5:0/1061472057 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15c01a7160 con 0x7f15c0103ff0 2024-09-10T09:54:37.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:37.941+0000 7f15c8000640 1 -- 172.21.15.5:0/1061472057 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15c01043d0 con 0x7f15c0103ff0 2024-09-10T09:54:37.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:37.942+0000 7f15be000640 1 -- 172.21.15.5:0/1061472057 <== mon.2 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f15b40041d0 con 0x7f15c0103ff0 2024-09-10T09:54:37.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:37.944+0000 7f15be000640 1 --2- 172.21.15.5:0/1061472057 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f15a80a2b80 0x7f15a80a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:37.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:37.944+0000 7f15be000640 1 -- 172.21.15.5:0/1061472057 <== mon.2 v2:172.21.15.102:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f15b40cf110 con 0x7f15c0103ff0 2024-09-10T09:54:37.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:37.944+0000 7f15c4c00640 1 --2- 172.21.15.5:0/1061472057 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f15a80a2b80 0x7f15a80a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:37.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:37.945+0000 7f15c4c00640 1 --2- 172.21.15.5:0/1061472057 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f15a80a2b80 0x7f15a80a5040 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f15c010ab00 tx=0x7f15ac005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:54:37.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:37.946+0000 7f15be000640 1 -- 172.21.15.5:0/1061472057 <== mon.2 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f15b409bc20 con 0x7f15c0103ff0 2024-09-10T09:54:37.966 INFO:tasks.workunit.client.0.smithi005.stdout: 10 41284 6.98 MB/sec execute 16 sec latency 11988.296 ms 2024-09-10T09:54:38.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:38.171+0000 7f15c8000640 1 -- 172.21.15.5:0/1061472057 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f15c019b940 con 0x7f15c0103ff0 2024-09-10T09:54:38.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:38.172+0000 7f15be000640 1 -- 172.21.15.5:0/1061472057 <== mon.2 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) ==== 94+0+11974 (secure 0 0 0) 0x7f15b409b520 con 0x7f15c0103ff0 2024-09-10T09:54:38.175 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-10T09:54:38.175 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":39,"btime":"2024-09-10T09:54:36:582547+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14454,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/2970594311","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2970594311},{"type":"v1","addr":"172.21.15.5:6835","nonce":2970594311}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14514,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6839/3616502174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":3616502174},{"type":"v1","addr":"172.21.15.5:6839","nonce":3616502174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":39,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T09:35:29.626287+0000","modified":"2024-09-10T09:54:36.412746+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24542,"mds_1":24535,"mds_2":24511,"mds_3":24506,"mds_4":24443},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14520":{"gid":14520,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/2073599842","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":2073599842},{"type":"v1","addr":"172.21.15.5:6841","nonce":2073599842}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"k","rank":4,"incarnation":26,"state":"up:active","state_seq":24,"addr":"172.21.15.102:6833/2457007427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":2457007427},{"type":"v1","addr":"172.21.15.102:6833","nonce":2457007427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/356445159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":356445159},{"type":"v1","addr":"172.21.15.159:6833","nonce":356445159}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6837/2894314507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":2894314507},{"type":"v1","addr":"172.21.15.5:6837","nonce":2894314507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.102:6835/1647549111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":1647549111},{"type":"v1","addr":"172.21.15.102:6835","nonce":1647549111}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6835/952657242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":952657242},{"type":"v1","addr":"172.21.15.159:6835","nonce":952657242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":19,"addr":"172.21.15.102:6837/2689776764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":2689776764},{"type":"v1","addr":"172.21.15.102:6837","nonce":2689776764}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.159:6837/620627639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":620627639},{"type":"v1","addr":"172.21.15.159:6837","nonce":620627639}]},"join_fscid":-1,"export_targets":[0,1,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24535":{"gid":24535,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":18,"addr":"172.21.15.159:6839/2073739599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":2073739599},{"type":"v1","addr":"172.21.15.159:6839","nonce":2073739599}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24542":{"gid":24542,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.102:6839/335409749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6838","nonce":335409749},{"type":"v1","addr":"172.21.15.102:6839","nonce":335409749}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24542,"qdb_cluster":[24443,24506,24511,24535,24542]},"id":1}]} 2024-09-10T09:54:38.175 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 39 2024-09-10T09:54:38.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:38.175+0000 7f15c8000640 1 -- 172.21.15.5:0/1061472057 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f15a80a2b80 msgr2=0x7f15a80a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:38.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:38.175+0000 7f15c8000640 1 --2- 172.21.15.5:0/1061472057 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f15a80a2b80 0x7f15a80a5040 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f15c010ab00 tx=0x7f15ac005f90 comp rx=0 tx=0).stop 2024-09-10T09:54:38.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:38.175+0000 7f15c8000640 1 -- 172.21.15.5:0/1061472057 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f15c0103ff0 msgr2=0x7f15c019a5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:38.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:38.175+0000 7f15c8000640 1 --2- 172.21.15.5:0/1061472057 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f15c0103ff0 0x7f15c019a5e0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f15b400d950 tx=0x7f15b400de10 comp rx=0 tx=0).stop 2024-09-10T09:54:38.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:38.175+0000 7f15c8000640 1 -- 172.21.15.5:0/1061472057 shutdown_connections 2024-09-10T09:54:38.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:38.175+0000 7f15c8000640 1 --2- 172.21.15.5:0/1061472057 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f15a80a2b80 0x7f15a80a5040 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:38.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:38.175+0000 7f15c8000640 1 --2- 172.21.15.5:0/1061472057 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f15c0109120 0x7f15c019eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:38.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:38.175+0000 7f15c8000640 1 --2- 172.21.15.5:0/1061472057 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f15c01049a0 0x7f15c019ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:38.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:38.175+0000 7f15c8000640 1 --2- 172.21.15.5:0/1061472057 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f15c0103ff0 0x7f15c019a5e0 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:38.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:38.175+0000 7f15c8000640 1 -- 172.21.15.5:0/1061472057 >> 172.21.15.5:0/1061472057 conn(0x7f15c00ffa90 msgr2=0x7f15c01005b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:54:38.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:38.175+0000 7f15c8000640 1 -- 172.21.15.5:0/1061472057 shutdown_connections 2024-09-10T09:54:38.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:38.175+0000 7f15c8000640 1 -- 172.21.15.5:0/1061472057 wait complete. 2024-09-10T09:54:38.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:54:38 smithi159 ceph-mon[35513]: from='client.? 172.21.15.5:0/1061472057' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:54:38.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:54:38 smithi102 ceph-mon[35442]: from='client.? 172.21.15.5:0/1061472057' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:54:38.967 INFO:tasks.workunit.client.0.smithi005.stdout: 10 41284 6.57 MB/sec execute 17 sec latency 12988.362 ms 2024-09-10T09:54:39.050 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:54:38 smithi005 ceph-mon[30350]: from='client.? 172.21.15.5:0/1061472057' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:54:39.089 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24542 quiesce db --set-id 87c9f91d --release --await' 2024-09-10T09:54:39.089 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24542 quiesce db --set-id 87c9f91d --release --await 2024-09-10T09:54:39.466 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:54:39.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:54:39 smithi159 ceph-mon[35513]: pgmap v882: 129 pgs: 129 active+clean; 535 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 953 KiB/s wr, 155 op/s 2024-09-10T09:54:39.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:54:39 smithi102 ceph-mon[35442]: pgmap v882: 129 pgs: 129 active+clean; 535 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 953 KiB/s wr, 155 op/s 2024-09-10T09:54:39.929 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:54:39 smithi005 ceph-mon[30350]: pgmap v882: 129 pgs: 129 active+clean; 535 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 953 KiB/s wr, 155 op/s 2024-09-10T09:54:39.966 INFO:tasks.workunit.client.0.smithi005.stdout: 10 41284 6.20 MB/sec execute 18 sec latency 13988.437 ms 2024-09-10T09:54:40.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:40.049+0000 7f06dfc00640 1 -- 172.21.15.5:0/1704410276 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f06d8106f10 msgr2=0x7f06d810ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:40.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:40.049+0000 7f06dfc00640 1 --2- 172.21.15.5:0/1704410276 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f06d8106f10 0x7f06d810ea90 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f06d0009ca0 tx=0x7f06d002a8c0 comp rx=0 tx=0).stop 2024-09-10T09:54:40.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:40.049+0000 7f06dfc00640 1 -- 172.21.15.5:0/1704410276 shutdown_connections 2024-09-10T09:54:40.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:40.049+0000 7f06dfc00640 1 --2- 172.21.15.5:0/1704410276 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f06d8106f10 0x7f06d810ea90 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:40.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:40.049+0000 7f06dfc00640 1 --2- 172.21.15.5:0/1704410276 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f06d8102790 0x7f06d81067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:40.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:40.049+0000 7f06dfc00640 1 --2- 172.21.15.5:0/1704410276 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f06d8101de0 0x7f06d81021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:40.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:40.049+0000 7f06dfc00640 1 -- 172.21.15.5:0/1704410276 >> 172.21.15.5:0/1704410276 conn(0x7f06d80fd8c0 msgr2=0x7f06d80ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:54:40.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:40.050+0000 7f06dfc00640 1 -- 172.21.15.5:0/1704410276 shutdown_connections 2024-09-10T09:54:40.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:40.050+0000 7f06dfc00640 1 -- 172.21.15.5:0/1704410276 wait complete. 2024-09-10T09:54:40.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:40.052+0000 7f06dfc00640 1 Processor -- start 2024-09-10T09:54:40.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:40.052+0000 7f06dfc00640 1 -- start start 2024-09-10T09:54:40.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:40.052+0000 7f06dfc00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f06d8101de0 0x7f06d819a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:40.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:40.052+0000 7f06dfc00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f06d8102790 0x7f06d819ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:40.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:40.052+0000 7f06dfc00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f06d8106f10 0x7f06d819eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:40.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:40.052+0000 7f06dfc00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f06d8111fc0 con 0x7f06d8102790 2024-09-10T09:54:40.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:40.052+0000 7f06dfc00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f06d8111e40 con 0x7f06d8101de0 2024-09-10T09:54:40.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:40.052+0000 7f06dfc00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f06d8112140 con 0x7f06d8106f10 2024-09-10T09:54:40.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:40.053+0000 7f06dd200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f06d8101de0 0x7f06d819a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:40.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:40.053+0000 7f06dd200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f06d8101de0 0x7f06d819a600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:37508/0 (socket says 172.21.15.5:37508) 2024-09-10T09:54:40.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:40.053+0000 7f06dd200640 1 -- 172.21.15.5:0/3087752849 learned_addr learned my addr 172.21.15.5:0/3087752849 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:54:40.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:40.053+0000 7f06ddc00640 1 --2- 172.21.15.5:0/3087752849 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f06d8106f10 0x7f06d819eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:40.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:40.053+0000 7f06cfe00640 1 --2- 172.21.15.5:0/3087752849 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f06d8102790 0x7f06d819ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:40.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:40.053+0000 7f06dd200640 1 -- 172.21.15.5:0/3087752849 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f06d8106f10 msgr2=0x7f06d819eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:40.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:40.053+0000 7f06dd200640 1 --2- 172.21.15.5:0/3087752849 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f06d8106f10 0x7f06d819eed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:40.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:40.053+0000 7f06dd200640 1 -- 172.21.15.5:0/3087752849 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f06d8102790 msgr2=0x7f06d819ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:40.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:40.053+0000 7f06dd200640 1 --2- 172.21.15.5:0/3087752849 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f06d8102790 0x7f06d819ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:40.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:40.053+0000 7f06dd200640 1 -- 172.21.15.5:0/3087752849 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06d819f5b0 con 0x7f06d8101de0 2024-09-10T09:54:40.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:40.053+0000 7f06dd200640 1 --2- 172.21.15.5:0/3087752849 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f06d8101de0 0x7f06d819a600 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f06c800e9f0 tx=0x7f06c800edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:54:40.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:40.053+0000 7f06d6000640 1 -- 172.21.15.5:0/3087752849 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06c800cd80 con 0x7f06d8101de0 2024-09-10T09:54:40.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:40.053+0000 7f06dfc00640 1 -- 172.21.15.5:0/3087752849 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06d819f8a0 con 0x7f06d8101de0 2024-09-10T09:54:40.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:40.053+0000 7f06d6000640 1 -- 172.21.15.5:0/3087752849 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f06c8005500 con 0x7f06d8101de0 2024-09-10T09:54:40.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:40.053+0000 7f06d6000640 1 -- 172.21.15.5:0/3087752849 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06c8010690 con 0x7f06d8101de0 2024-09-10T09:54:40.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:40.053+0000 7f06dfc00640 1 -- 172.21.15.5:0/3087752849 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06d81a7180 con 0x7f06d8101de0 2024-09-10T09:54:40.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:40.056+0000 7f06e2689740 1 Processor -- start 2024-09-10T09:54:40.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:40.056+0000 7f06e2689740 1 -- start start 2024-09-10T09:54:40.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:40.056+0000 7f06d6000640 1 -- 172.21.15.5:0/3087752849 <== mon.2 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f06c80046c0 con 0x7f06d8101de0 2024-09-10T09:54:40.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:40.057+0000 7f06e2689740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555d39039310 0x555d390396f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:40.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:40.057+0000 7f06e2689740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555d39039c30 0x555d39043fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:40.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:40.057+0000 7f06e2689740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555d390444f0 0x555d390468e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:40.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:40.057+0000 7f06e2689740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x555d38f9f650 con 0x555d39039310 2024-09-10T09:54:40.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:40.057+0000 7f06e2689740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x555d38f46f10 con 0x555d39039c30 2024-09-10T09:54:40.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:40.057+0000 7f06e2689740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x555d38f9b610 con 0x555d390444f0 2024-09-10T09:54:40.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:40.057+0000 7f06ddc00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555d39039310 0x555d390396f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:40.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:40.057+0000 7f06ddc00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555d39039310 0x555d390396f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38622/0 (socket says 172.21.15.5:38622) 2024-09-10T09:54:40.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:40.057+0000 7f06dd200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555d39039c30 0x555d39043fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:40.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:40.057+0000 7f06ddc00640 1 -- 172.21.15.5:0/1581207497 learned_addr learned my addr 172.21.15.5:0/1581207497 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:54:40.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:40.057+0000 7f06cfe00640 1 --2- 172.21.15.5:0/1581207497 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555d390444f0 0x555d390468e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:40.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:40.057+0000 7f06ddc00640 1 -- 172.21.15.5:0/1581207497 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555d390444f0 msgr2=0x555d390468e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:54:40.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:40.057+0000 7f06ddc00640 1 --2- 172.21.15.5:0/1581207497 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555d390444f0 0x555d390468e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:40.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:40.057+0000 7f06ddc00640 1 -- 172.21.15.5:0/1581207497 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555d39039c30 msgr2=0x555d39043fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:54:40.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:40.057+0000 7f06ddc00640 1 --2- 172.21.15.5:0/1581207497 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555d39039c30 0x555d39043fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:40.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:40.057+0000 7f06ddc00640 1 -- 172.21.15.5:0/1581207497 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555d38cae580 con 0x555d39039310 2024-09-10T09:54:40.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:40.058+0000 7f06ddc00640 1 --2- 172.21.15.5:0/1581207497 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555d39039310 0x555d390396f0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f06d0009ca0 tx=0x7f06d00093e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac976965deb667b6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:54:40.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:40.058+0000 7f06ce000640 1 -- 172.21.15.5:0/1581207497 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06d0031db0 con 0x555d39039310 2024-09-10T09:54:40.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:40.058+0000 7f06ce000640 1 -- 172.21.15.5:0/1581207497 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f06d0036030 con 0x555d39039310 2024-09-10T09:54:40.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:40.058+0000 7f06ce000640 1 -- 172.21.15.5:0/1581207497 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06d00042a0 con 0x555d39039310 2024-09-10T09:54:40.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:40.058+0000 7f06d6000640 1 --2- 172.21.15.5:0/3087752849 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f06bc0a2b80 0x7f06bc0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:40.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:40.058+0000 7f06d6000640 1 -- 172.21.15.5:0/3087752849 <== mon.2 v2:172.21.15.102:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f06c80cf870 con 0x7f06d8101de0 2024-09-10T09:54:40.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:40.058+0000 7f06dd200640 1 --2- 172.21.15.5:0/3087752849 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f06bc0a2b80 0x7f06bc0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:40.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:40.059+0000 7f06e2689740 1 -- 172.21.15.5:0/1581207497 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555d39039310 msgr2=0x555d390396f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:40.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:40.059+0000 7f06e2689740 1 --2- 172.21.15.5:0/1581207497 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555d39039310 0x555d390396f0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f06d0009ca0 tx=0x7f06d00093e0 comp rx=0 tx=0).stop 2024-09-10T09:54:40.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:40.059+0000 7f06e2689740 1 -- 172.21.15.5:0/1581207497 shutdown_connections 2024-09-10T09:54:40.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:40.059+0000 7f06e2689740 1 --2- 172.21.15.5:0/1581207497 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555d390444f0 0x555d390468e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:40.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:40.059+0000 7f06e2689740 1 --2- 172.21.15.5:0/1581207497 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555d39039c30 0x555d39043fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:40.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:40.059+0000 7f06e2689740 1 --2- 172.21.15.5:0/1581207497 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555d39039310 0x555d390396f0 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:40.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:40.059+0000 7f06e2689740 1 -- 172.21.15.5:0/1581207497 >> 172.21.15.5:0/1581207497 conn(0x555d390358a0 msgr2=0x555d39035c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:54:40.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:40.059+0000 7f06e2689740 1 -- 172.21.15.5:0/1581207497 shutdown_connections 2024-09-10T09:54:40.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:40.059+0000 7f06dd200640 1 --2- 172.21.15.5:0/3087752849 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f06bc0a2b80 0x7f06bc0a5040 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f06c80980e0 tx=0x7f06c80d5530 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:54:40.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:40.059+0000 7f06e2689740 1 -- 172.21.15.5:0/1581207497 wait complete. 2024-09-10T09:54:40.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:40.060+0000 7f06e2689740 1 -- start start 2024-09-10T09:54:40.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:40.060+0000 7f06e2689740 1 Processor -- start 2024-09-10T09:54:40.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:40.060+0000 7f06e2689740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555d39039310 0x555d390413e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:40.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:40.060+0000 7f06e2689740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555d39039c30 0x555d39041920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:40.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:40.060+0000 7f06e2689740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555d390444f0 0x555d39041e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:40.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:40.060+0000 7f06e2689740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x555d39046f40 con 0x555d390444f0 2024-09-10T09:54:40.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:40.060+0000 7f06e2689740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x555d38f5a350 con 0x555d39039310 2024-09-10T09:54:40.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:40.060+0000 7f06e2689740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x555d390470c0 con 0x555d39039c30 2024-09-10T09:54:40.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:40.060+0000 7f06e2689740 10 client.0 _ll_register_callbacks cb 0x555d38f7d290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:54:40.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:40.061+0000 7f06ddc00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555d39039310 0x555d390413e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:40.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:40.061+0000 7f06cfe00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555d39039c30 0x555d39041920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:40.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:40.061+0000 7f06ddc00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555d39039310 0x555d390413e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:37530/0 (socket says 172.21.15.5:37530) 2024-09-10T09:54:40.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:40.061+0000 7f06ddc00640 1 -- 172.21.15.5:0/1394602779 learned_addr learned my addr 172.21.15.5:0/1394602779 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:54:40.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:40.061+0000 7f06ddc00640 1 --2- 172.21.15.5:0/1394602779 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555d390444f0 0x555d39041e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:40.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:40.061+0000 7f06cfe00640 1 -- 172.21.15.5:0/1394602779 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555d39039310 msgr2=0x555d390413e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:54:40.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:40.061+0000 7f06cfe00640 1 --2- 172.21.15.5:0/1394602779 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555d39039310 0x555d390413e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:40.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:40.061+0000 7f06cfe00640 1 -- 172.21.15.5:0/1394602779 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555d390444f0 msgr2=0x555d39041e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:54:40.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:40.061+0000 7f06cfe00640 1 --2- 172.21.15.5:0/1394602779 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555d390444f0 0x555d39041e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:40.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:40.061+0000 7f06cfe00640 1 -- 172.21.15.5:0/1394602779 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555d38d78960 con 0x555d39039c30 2024-09-10T09:54:40.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:40.062+0000 7f06cfe00640 1 --2- 172.21.15.5:0/1394602779 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555d39039c30 0x555d39041920 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f06d810eeb0 tx=0x7f06c0012b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=19313d3e7a930b0f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:54:40.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:40.062+0000 7f06c6a00640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-09-10T09:54:40.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:40.062+0000 7f06c6a00640 1 -- 172.21.15.5:0/1394602779 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06c0004050 con 0x555d39039c30 2024-09-10T09:54:40.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:40.062+0000 7f06e2689740 1 -- 172.21.15.5:0/1394602779 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f06d0036030 con 0x555d39039c30 2024-09-10T09:54:40.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:40.062+0000 7f06c6a00640 1 -- 172.21.15.5:0/1394602779 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f06c00041f0 con 0x555d39039c30 2024-09-10T09:54:40.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:40.062+0000 7f06c6a00640 1 -- 172.21.15.5:0/1394602779 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f06c00043f0 con 0x555d39039c30 2024-09-10T09:54:40.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:40.062+0000 7f06e2689740 10 client.28687 fetch_fsmap learned FSMap version 39 2024-09-10T09:54:40.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:40.062+0000 7f06e2689740 1 -- 172.21.15.5:0/1394602779 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x555d39052c00 con 0x555d39039c30 2024-09-10T09:54:40.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:40.062+0000 7f06c6a00640 1 -- 172.21.15.5:0/1394602779 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f06d0036030 con 0x555d39039c30 2024-09-10T09:54:40.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:40.063+0000 7f06c6a00640 1 -- 172.21.15.5:0/1394602779 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 39) ==== 8382+0+0 (secure 0 0 0) 0x7f06c0003b80 con 0x555d39039c30 2024-09-10T09:54:40.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:40.063+0000 7f06e2689740 10 client.28687 fetch_fsmap finished waiting for FSMap version 39 2024-09-10T09:54:40.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:40.063+0000 7f06e2689740 10 client.28687 resolve_mds: validated gid 24542 aka daemon mds.b 2024-09-10T09:54:40.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:40.063+0000 7f06e2689740 20 client.28687 populate_metadata read hostname 'smithi005' 2024-09-10T09:54:40.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:40.063+0000 7f06e2689740 1 --2- 172.21.15.5:0/1394602779 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x555d39054060 0x555d39056480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:40.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:40.063+0000 7f06e2689740 4 client.28687 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:54:40.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:40.063+0000 7f06e2689740 1 -- 172.21.15.5:0/1394602779 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555d38facbc0 con 0x555d39054060 2024-09-10T09:54:40.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:40.063+0000 7f06ddc00640 1 --2- 172.21.15.5:0/1394602779 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x555d39054060 0x555d39056480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:40.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:40.064+0000 7f06ddc00640 1 --2- 172.21.15.5:0/1394602779 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x555d39054060 0x555d39056480 crc :-1 s=READY pgs=241 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d8d6aeb78979d674 server_cookie=d1173fd0d663cc61 in_seq=0 out_seq=0 2024-09-10T09:54:40.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:40.064+0000 7f06c6a00640 10 client.28687 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T09:54:40.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:40.064+0000 7f06c6a00640 1 -- 172.21.15.5:0/1394602779 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555d38facbc0 con 0x555d39054060 2024-09-10T09:54:40.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:40.066+0000 7f06c6a00640 10 client.28687 handle_command_reply: tid=0 2024-09-10T09:54:40.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:40.066+0000 7f06e2689740 1 client.28687 shutdown 2024-09-10T09:54:40.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:40.066+0000 7f06e2689740 1 -- 172.21.15.5:0/1394602779 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555d39039c30 msgr2=0x555d39041920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:40.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:40.066+0000 7f06e2689740 1 --2- 172.21.15.5:0/1394602779 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555d39039c30 0x555d39041920 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f06d810eeb0 tx=0x7f06c0012b60 comp rx=0 tx=0).stop 2024-09-10T09:54:40.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:40.066+0000 7f06e2689740 1 -- 172.21.15.5:0/1394602779 shutdown_connections 2024-09-10T09:54:40.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:40.066+0000 7f06e2689740 1 --2- 172.21.15.5:0/1394602779 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x555d39054060 0x555d39056480 crc :-1 s=READY pgs=241 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:40.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:40.066+0000 7f06e2689740 1 --2- 172.21.15.5:0/1394602779 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555d390444f0 0x555d39041e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:40.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:40.066+0000 7f06e2689740 1 --2- 172.21.15.5:0/1394602779 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555d39039c30 0x555d39041920 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:40.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:40.066+0000 7f06e2689740 1 --2- 172.21.15.5:0/1394602779 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555d39039310 0x555d390413e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:40.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:40.066+0000 7f06c6a00640 0 client.28687 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T09:54:40.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:40.066+0000 7f06e2689740 1 -- 172.21.15.5:0/1394602779 >> 172.21.15.5:0/1394602779 conn(0x555d390358a0 msgr2=0x555d39034aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:54:40.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:40.066+0000 7f06e2689740 1 -- 172.21.15.5:0/1394602779 shutdown_connections 2024-09-10T09:54:40.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:40.066+0000 7f06e2689740 1 -- 172.21.15.5:0/1394602779 wait complete. 2024-09-10T09:54:40.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:40.066+0000 7f06e2689740 20 client.28687 trim_cache size 0 max 16384 2024-09-10T09:54:40.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:40.083+0000 7f06e2689740 1 Processor -- start 2024-09-10T09:54:40.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:40.083+0000 7f06e2689740 1 -- start start 2024-09-10T09:54:40.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:40.084+0000 7f06e2689740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555d39039310 0x555d39034990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:40.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:40.084+0000 7f06e2689740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555d39039c30 0x555d3905bd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:40.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:40.084+0000 7f06e2689740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555d390444f0 0x555d3905c270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:40.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:40.084+0000 7f06e2689740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f06d81b0fd0 con 0x555d390444f0 2024-09-10T09:54:40.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:40.084+0000 7f06e2689740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f06d81b2ba0 con 0x555d39039310 2024-09-10T09:54:40.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:40.084+0000 7f06e2689740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f06d81b16a0 con 0x555d39039c30 2024-09-10T09:54:40.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:40.084+0000 7f06ddc00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555d39039310 0x555d39034990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:40.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:40.084+0000 7f06ddc00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555d39039310 0x555d39034990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:37546/0 (socket says 172.21.15.5:37546) 2024-09-10T09:54:40.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:40.084+0000 7f06ddc00640 1 -- 172.21.15.5:0/551607495 learned_addr learned my addr 172.21.15.5:0/551607495 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:54:40.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:40.084+0000 7f06ddc00640 1 --2- 172.21.15.5:0/551607495 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555d390444f0 0x555d3905c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:40.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:40.084+0000 7f06cfe00640 1 --2- 172.21.15.5:0/551607495 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555d39039c30 0x555d3905bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:40.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:40.084+0000 7f06ddc00640 1 -- 172.21.15.5:0/551607495 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555d39039c30 msgr2=0x555d3905bd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:54:40.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:40.084+0000 7f06ddc00640 1 --2- 172.21.15.5:0/551607495 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555d39039c30 0x555d3905bd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:40.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:40.084+0000 7f06ddc00640 1 -- 172.21.15.5:0/551607495 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555d390444f0 msgr2=0x555d3905c270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:54:40.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:40.084+0000 7f06ddc00640 1 --2- 172.21.15.5:0/551607495 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555d390444f0 0x555d3905c270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:40.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:40.084+0000 7f06ddc00640 1 -- 172.21.15.5:0/551607495 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555d39039140 con 0x555d39039310 2024-09-10T09:54:40.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:40.085+0000 7f06ddc00640 1 --2- 172.21.15.5:0/551607495 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555d39039310 0x555d39034990 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f06d0006010 tx=0x7f06d0005df0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e31322306e0a2cee server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:54:40.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:40.085+0000 7f06cd600640 1 -- 172.21.15.5:0/551607495 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06d0031bb0 con 0x555d39039310 2024-09-10T09:54:40.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:40.085+0000 7f06cd600640 1 -- 172.21.15.5:0/551607495 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f06d0036040 con 0x555d39039310 2024-09-10T09:54:40.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:40.085+0000 7f06cd600640 1 -- 172.21.15.5:0/551607495 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06d0004320 con 0x555d39039310 2024-09-10T09:54:40.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:40.086+0000 7f06e2689740 1 -- 172.21.15.5:0/551607495 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555d39039310 msgr2=0x555d39034990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:40.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:40.086+0000 7f06e2689740 1 --2- 172.21.15.5:0/551607495 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555d39039310 0x555d39034990 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f06d0006010 tx=0x7f06d0005df0 comp rx=0 tx=0).stop 2024-09-10T09:54:40.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:40.086+0000 7f06e2689740 1 -- 172.21.15.5:0/551607495 shutdown_connections 2024-09-10T09:54:40.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:40.086+0000 7f06e2689740 1 --2- 172.21.15.5:0/551607495 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555d390444f0 0x555d3905c270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:40.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:40.086+0000 7f06e2689740 1 --2- 172.21.15.5:0/551607495 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555d39039c30 0x555d3905bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:40.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:40.086+0000 7f06e2689740 1 --2- 172.21.15.5:0/551607495 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555d39039310 0x555d39034990 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:40.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:40.086+0000 7f06e2689740 1 -- 172.21.15.5:0/551607495 >> 172.21.15.5:0/551607495 conn(0x555d390358a0 msgr2=0x555d39034280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:54:40.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:40.086+0000 7f06e2689740 1 -- 172.21.15.5:0/551607495 shutdown_connections 2024-09-10T09:54:40.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:40.086+0000 7f06e2689740 1 -- 172.21.15.5:0/551607495 wait complete. 2024-09-10T09:54:40.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:40.086+0000 7f06e2689740 1 -- start start 2024-09-10T09:54:40.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:40.086+0000 7f06e2689740 1 Processor -- start 2024-09-10T09:54:40.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:40.087+0000 7f06e2689740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555d39039310 0x555d3903e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:40.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:40.087+0000 7f06e2689740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555d39039c30 0x555d3903e930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:40.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:40.087+0000 7f06e2689740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555d390444f0 0x555d3903ee70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:40.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:40.087+0000 7f06e2689740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f06d81b24d0 con 0x555d39039c30 2024-09-10T09:54:40.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:40.087+0000 7f06e2689740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f06d81b1e00 con 0x555d390444f0 2024-09-10T09:54:40.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:40.087+0000 7f06e2689740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f06d81aefb0 con 0x555d39039310 2024-09-10T09:54:40.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:40.087+0000 7f06e2689740 10 client.0 _ll_register_callbacks cb 0x7f06c00097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:54:40.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:40.087+0000 7f06ddc00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555d390444f0 0x555d3903ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:40.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:40.087+0000 7f06cfe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555d39039c30 0x555d3903e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:40.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:40.087+0000 7f06ddc00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555d390444f0 0x555d3903ee70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:37548/0 (socket says 172.21.15.5:37548) 2024-09-10T09:54:40.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:40.087+0000 7f06ddc00640 1 -- 172.21.15.5:0/3592765140 learned_addr learned my addr 172.21.15.5:0/3592765140 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:54:40.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:40.087+0000 7f06ddc00640 1 --2- 172.21.15.5:0/3592765140 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555d39039310 0x555d3903e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:40.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:40.087+0000 7f06cfe00640 1 -- 172.21.15.5:0/3592765140 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555d39039310 msgr2=0x555d3903e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:54:40.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:40.087+0000 7f06cfe00640 1 --2- 172.21.15.5:0/3592765140 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555d39039310 0x555d3903e3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:40.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:40.087+0000 7f06cfe00640 1 -- 172.21.15.5:0/3592765140 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555d390444f0 msgr2=0x555d3903ee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:54:40.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:40.087+0000 7f06cfe00640 1 --2- 172.21.15.5:0/3592765140 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555d390444f0 0x555d3903ee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:40.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:40.087+0000 7f06cfe00640 1 -- 172.21.15.5:0/3592765140 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555d390bdd00 con 0x555d39039c30 2024-09-10T09:54:40.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:40.087+0000 7f06ddc00640 1 --2- 172.21.15.5:0/3592765140 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555d390444f0 0x555d3903ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:54:40.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:40.088+0000 7f06cfe00640 1 --2- 172.21.15.5:0/3592765140 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555d39039c30 0x555d3903e930 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f06c00105c0 tx=0x7f06c000fad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a80e04944067c85a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:54:40.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:40.088+0000 7f06c6a00640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-10T09:54:40.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:40.088+0000 7f06c6a00640 1 -- 172.21.15.5:0/3592765140 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06c0009bf0 con 0x555d39039c30 2024-09-10T09:54:40.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:40.088+0000 7f06c6a00640 1 -- 172.21.15.5:0/3592765140 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f06c0005040 con 0x555d39039c30 2024-09-10T09:54:40.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:40.088+0000 7f06e2689740 1 -- 172.21.15.5:0/3592765140 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f06d0036040 con 0x555d39039c30 2024-09-10T09:54:40.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:40.088+0000 7f06c6a00640 1 -- 172.21.15.5:0/3592765140 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f06c0005240 con 0x555d39039c30 2024-09-10T09:54:40.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:40.088+0000 7f06e2689740 10 client.18627 fetch_fsmap learned FSMap version 39 2024-09-10T09:54:40.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:40.088+0000 7f06e2689740 1 -- 172.21.15.5:0/3592765140 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x555d390bf070 con 0x555d39039c30 2024-09-10T09:54:40.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:40.089+0000 7f06c6a00640 1 -- 172.21.15.5:0/3592765140 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f06d0036040 con 0x555d39039c30 2024-09-10T09:54:40.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:40.089+0000 7f06c6a00640 1 -- 172.21.15.5:0/3592765140 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 39) ==== 8382+0+0 (secure 0 0 0) 0x7f06c0003bd0 con 0x555d39039c30 2024-09-10T09:54:40.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:40.089+0000 7f06e2689740 10 client.18627 fetch_fsmap finished waiting for FSMap version 39 2024-09-10T09:54:40.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:40.089+0000 7f06e2689740 10 client.18627 resolve_mds: validated gid 24542 aka daemon mds.b 2024-09-10T09:54:40.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:40.089+0000 7f06e2689740 20 client.18627 populate_metadata read hostname 'smithi005' 2024-09-10T09:54:40.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:40.089+0000 7f06e2689740 1 --2- 172.21.15.5:0/3592765140 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x555d390bf7f0 0x555d390c1c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:54:40.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:40.089+0000 7f06e2689740 4 client.18627 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "87c9f91d", "release": true, "await": true}] 2024-09-10T09:54:40.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:40.089+0000 7f06e2689740 1 -- 172.21.15.5:0/3592765140 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "quiesce db", "set_id": "87c9f91d", "release": true, "await": true}) -- 0x555d38fa70d0 con 0x555d390bf7f0 2024-09-10T09:54:40.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:40.089+0000 7f06ddc00640 1 --2- 172.21.15.5:0/3592765140 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x555d390bf7f0 0x555d390c1c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:54:40.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:40.090+0000 7f06ddc00640 1 --2- 172.21.15.5:0/3592765140 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x555d390bf7f0 0x555d390c1c10 crc :-1 s=READY pgs=242 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a6c78276cd711204 server_cookie=8bcbf53c4774f248 in_seq=0 out_seq=0 2024-09-10T09:54:40.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:40.090+0000 7f06c6a00640 10 client.18627 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T09:54:40.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:40.146+0000 7f06c6a00640 1 -- 172.21.15.5:0/3592765140 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x555d38fa70d0 con 0x555d390bf7f0 2024-09-10T09:54:40.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:40.146+0000 7f06c6a00640 10 client.18627 handle_command_reply: tid=0 2024-09-10T09:54:40.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:40.146+0000 7f06e2689740 1 client.18627 shutdown 2024-09-10T09:54:40.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:40.146+0000 7f06e2689740 1 -- 172.21.15.5:0/3592765140 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555d39039c30 msgr2=0x555d3903e930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:40.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:40.146+0000 7f06e2689740 1 --2- 172.21.15.5:0/3592765140 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555d39039c30 0x555d3903e930 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f06c00105c0 tx=0x7f06c000fad0 comp rx=0 tx=0).stop 2024-09-10T09:54:40.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:40.146+0000 7f06e2689740 1 -- 172.21.15.5:0/3592765140 shutdown_connections 2024-09-10T09:54:40.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:40.146+0000 7f06e2689740 1 --2- 172.21.15.5:0/3592765140 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x555d390bf7f0 0x555d390c1c10 crc :-1 s=READY pgs=242 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:40.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:40.146+0000 7f06e2689740 1 --2- 172.21.15.5:0/3592765140 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555d390444f0 0x555d3903ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:40.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:40.146+0000 7f06e2689740 1 --2- 172.21.15.5:0/3592765140 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555d39039c30 0x555d3903e930 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:40.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:40.146+0000 7f06e2689740 1 --2- 172.21.15.5:0/3592765140 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555d39039310 0x555d3903e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:40.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:40.146+0000 7f06e2689740 1 -- 172.21.15.5:0/3592765140 >> 172.21.15.5:0/3592765140 conn(0x555d390358a0 msgr2=0x555d39034aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:54:40.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:40.146+0000 7f06c6a00640 0 client.18627 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T09:54:40.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:40.146+0000 7f06e2689740 1 -- 172.21.15.5:0/3592765140 shutdown_connections 2024-09-10T09:54:40.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:40.147+0000 7f06e2689740 1 -- 172.21.15.5:0/3592765140 wait complete. 2024-09-10T09:54:40.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:40.147+0000 7f06e2689740 20 client.18627 trim_cache size 0 max 16384 2024-09-10T09:54:40.150 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-10T09:54:40.150 INFO:teuthology.orchestra.run.smithi005.stdout: "epoch": 39, 2024-09-10T09:54:40.150 INFO:teuthology.orchestra.run.smithi005.stdout: "leader": 24542, 2024-09-10T09:54:40.150 INFO:teuthology.orchestra.run.smithi005.stdout: "set_version": 60, 2024-09-10T09:54:40.150 INFO:teuthology.orchestra.run.smithi005.stdout: "sets": { 2024-09-10T09:54:40.150 INFO:teuthology.orchestra.run.smithi005.stdout: "87c9f91d": { 2024-09-10T09:54:40.151 INFO:teuthology.orchestra.run.smithi005.stdout: "version": 60, 2024-09-10T09:54:40.151 INFO:teuthology.orchestra.run.smithi005.stdout: "age_ref": 0.0, 2024-09-10T09:54:40.151 INFO:teuthology.orchestra.run.smithi005.stdout: "state": { 2024-09-10T09:54:40.151 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "RELEASED", 2024-09-10T09:54:40.151 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0 2024-09-10T09:54:40.151 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T09:54:40.151 INFO:teuthology.orchestra.run.smithi005.stdout: "timeout": 90.0, 2024-09-10T09:54:40.151 INFO:teuthology.orchestra.run.smithi005.stdout: "expiration": 139.8, 2024-09-10T09:54:40.151 INFO:teuthology.orchestra.run.smithi005.stdout: "members": { 2024-09-10T09:54:40.151 INFO:teuthology.orchestra.run.smithi005.stdout: "file:/": { 2024-09-10T09:54:40.151 INFO:teuthology.orchestra.run.smithi005.stdout: "excluded": false, 2024-09-10T09:54:40.151 INFO:teuthology.orchestra.run.smithi005.stdout: "state": { 2024-09-10T09:54:40.151 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "RELEASED", 2024-09-10T09:54:40.152 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0 2024-09-10T09:54:40.152 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T09:54:40.152 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T09:54:40.152 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T09:54:40.152 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T09:54:40.152 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T09:54:40.152 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-10T09:54:40.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:40.148+0000 7f06dfc00640 1 -- 172.21.15.5:0/3087752849 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f06bc0a2b80 msgr2=0x7f06bc0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:40.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:40.148+0000 7f06dfc00640 1 --2- 172.21.15.5:0/3087752849 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f06bc0a2b80 0x7f06bc0a5040 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f06c80980e0 tx=0x7f06c80d5530 comp rx=0 tx=0).stop 2024-09-10T09:54:40.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:40.148+0000 7f06dfc00640 1 -- 172.21.15.5:0/3087752849 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f06d8101de0 msgr2=0x7f06d819a600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:54:40.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:40.148+0000 7f06dfc00640 1 --2- 172.21.15.5:0/3087752849 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f06d8101de0 0x7f06d819a600 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f06c800e9f0 tx=0x7f06c800edf0 comp rx=0 tx=0).stop 2024-09-10T09:54:40.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:40.148+0000 7f06dfc00640 1 -- 172.21.15.5:0/3087752849 shutdown_connections 2024-09-10T09:54:40.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:40.148+0000 7f06dfc00640 1 --2- 172.21.15.5:0/3087752849 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f06bc0a2b80 0x7f06bc0a5040 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:40.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:40.148+0000 7f06dfc00640 1 --2- 172.21.15.5:0/3087752849 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f06d8106f10 0x7f06d819eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:40.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:40.148+0000 7f06dfc00640 1 --2- 172.21.15.5:0/3087752849 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f06d8102790 0x7f06d819ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:40.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:40.148+0000 7f06dfc00640 1 --2- 172.21.15.5:0/3087752849 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f06d8101de0 0x7f06d819a600 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:54:40.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:40.148+0000 7f06dfc00640 1 -- 172.21.15.5:0/3087752849 >> 172.21.15.5:0/3087752849 conn(0x7f06d80fd8c0 msgr2=0x7f06d80fe2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:54:40.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:40.148+0000 7f06dfc00640 1 -- 172.21.15.5:0/3087752849 shutdown_connections 2024-09-10T09:54:40.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:54:40.148+0000 7f06dfc00640 1 -- 172.21.15.5:0/3087752849 wait complete. 2024-09-10T09:54:40.966 INFO:tasks.workunit.client.0.smithi005.stdout: 10 41831 7.64 MB/sec execute 19 sec latency 14315.915 ms 2024-09-10T09:54:41.179 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '87c9f91d', total seconds elapsed: 24.1 2024-09-10T09:54:41.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:54:41 smithi102 ceph-mon[35442]: pgmap v883: 129 pgs: 129 active+clean; 535 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 634 KiB/s wr, 128 op/s 2024-09-10T09:54:41.966 INFO:tasks.workunit.client.0.smithi005.stdout: 10 42595 9.87 MB/sec execute 20 sec latency 42.315 ms 2024-09-10T09:54:42.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:54:41 smithi005 ceph-mon[30350]: pgmap v883: 129 pgs: 129 active+clean; 535 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 634 KiB/s wr, 128 op/s 2024-09-10T09:54:42.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:54:41 smithi159 ceph-mon[35513]: pgmap v883: 129 pgs: 129 active+clean; 535 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 634 KiB/s wr, 128 op/s 2024-09-10T09:54:42.967 INFO:tasks.workunit.client.0.smithi005.stdout: 10 43124 10.58 MB/sec execute 21 sec latency 261.090 ms 2024-09-10T09:54:43.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:54:43 smithi005 ceph-mon[30350]: pgmap v884: 129 pgs: 129 active+clean; 535 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 526 KiB/s rd, 78 KiB/s wr, 24 op/s 2024-09-10T09:54:43.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:54:43 smithi102 ceph-mon[35442]: pgmap v884: 129 pgs: 129 active+clean; 535 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 526 KiB/s rd, 78 KiB/s wr, 24 op/s 2024-09-10T09:54:43.967 INFO:tasks.workunit.client.0.smithi005.stdout: 10 43719 11.40 MB/sec execute 22 sec latency 104.003 ms 2024-09-10T09:54:44.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:54:43 smithi159 ceph-mon[35513]: pgmap v884: 129 pgs: 129 active+clean; 535 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 526 KiB/s rd, 78 KiB/s wr, 24 op/s 2024-09-10T09:54:44.967 INFO:tasks.workunit.client.0.smithi005.stdout: 10 44475 12.68 MB/sec execute 23 sec latency 52.061 ms 2024-09-10T09:54:45.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:54:45 smithi102 ceph-mon[35442]: pgmap v885: 129 pgs: 129 active+clean; 536 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 209 KiB/s wr, 35 op/s 2024-09-10T09:54:45.967 INFO:tasks.workunit.client.0.smithi005.stdout: 10 45211 13.58 MB/sec execute 24 sec latency 51.628 ms 2024-09-10T09:54:46.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:54:45 smithi005 ceph-mon[30350]: pgmap v885: 129 pgs: 129 active+clean; 536 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 209 KiB/s wr, 35 op/s 2024-09-10T09:54:46.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:54:45 smithi159 ceph-mon[35513]: pgmap v885: 129 pgs: 129 active+clean; 536 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 209 KiB/s wr, 35 op/s 2024-09-10T09:54:46.967 INFO:tasks.workunit.client.0.smithi005.stdout: 10 46218 15.72 MB/sec execute 25 sec latency 50.406 ms 2024-09-10T09:54:47.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:54:47 smithi102 ceph-mon[35442]: pgmap v886: 129 pgs: 129 active+clean; 547 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 1.3 MiB/s wr, 198 op/s 2024-09-10T09:54:47.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:54:47 smithi102 ceph-mon[35442]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T09:54:47.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:54:47 smithi102 ceph-mon[35442]: scrub summary: idle 2024-09-10T09:54:47.885 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:54:47 smithi102 ceph-mon[35442]: scrub summary: idle 2024-09-10T09:54:47.885 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:54:47 smithi102 ceph-mon[35442]: scrub summary: idle 2024-09-10T09:54:47.885 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:54:47 smithi102 ceph-mon[35442]: scrub summary: idle 2024-09-10T09:54:47.885 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:54:47 smithi102 ceph-mon[35442]: scrub summary: idle 2024-09-10T09:54:47.967 INFO:tasks.workunit.client.0.smithi005.stdout: 10 46863 16.20 MB/sec execute 26 sec latency 56.275 ms 2024-09-10T09:54:48.057 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:54:47 smithi005 ceph-mon[30350]: pgmap v886: 129 pgs: 129 active+clean; 547 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 1.3 MiB/s wr, 198 op/s 2024-09-10T09:54:48.057 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:54:47 smithi005 ceph-mon[30350]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T09:54:48.057 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:54:47 smithi005 ceph-mon[30350]: scrub summary: idle 2024-09-10T09:54:48.057 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:54:47 smithi005 ceph-mon[30350]: scrub summary: idle 2024-09-10T09:54:48.057 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:54:47 smithi005 ceph-mon[30350]: scrub summary: idle 2024-09-10T09:54:48.057 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:54:47 smithi005 ceph-mon[30350]: scrub summary: idle 2024-09-10T09:54:48.057 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:54:47 smithi005 ceph-mon[30350]: scrub summary: idle 2024-09-10T09:54:48.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:54:47 smithi159 ceph-mon[35513]: pgmap v886: 129 pgs: 129 active+clean; 547 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 1.3 MiB/s wr, 198 op/s 2024-09-10T09:54:48.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:54:47 smithi159 ceph-mon[35513]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T09:54:48.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:54:47 smithi159 ceph-mon[35513]: scrub summary: idle 2024-09-10T09:54:48.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:54:47 smithi159 ceph-mon[35513]: scrub summary: idle 2024-09-10T09:54:48.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:54:47 smithi159 ceph-mon[35513]: scrub summary: idle 2024-09-10T09:54:48.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:54:47 smithi159 ceph-mon[35513]: scrub summary: idle 2024-09-10T09:54:48.101 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:54:47 smithi159 ceph-mon[35513]: scrub summary: idle 2024-09-10T09:54:48.967 INFO:tasks.workunit.client.0.smithi005.stdout: 10 47498 16.66 MB/sec execute 27 sec latency 47.321 ms 2024-09-10T09:54:49.619 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:54:49 smithi005 ceph-mon[30350]: pgmap v887: 129 pgs: 129 active+clean; 550 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 1.6 MiB/s wr, 217 op/s 2024-09-10T09:54:49.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:54:49 smithi102 ceph-mon[35442]: pgmap v887: 129 pgs: 129 active+clean; 550 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 1.6 MiB/s wr, 217 op/s 2024-09-10T09:54:49.967 INFO:tasks.workunit.client.0.smithi005.stdout: 10 48339 18.05 MB/sec execute 28 sec latency 53.398 ms 2024-09-10T09:54:50.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:54:49 smithi159 ceph-mon[35513]: pgmap v887: 129 pgs: 129 active+clean; 550 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 1.6 MiB/s wr, 217 op/s 2024-09-10T09:54:50.968 INFO:tasks.workunit.client.0.smithi005.stdout: 10 49036 18.40 MB/sec execute 29 sec latency 50.054 ms 2024-09-10T09:54:51.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:54:51 smithi102 ceph-mon[35442]: pgmap v888: 129 pgs: 129 active+clean; 564 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 3.0 MiB/s wr, 424 op/s 2024-09-10T09:54:51.968 INFO:tasks.workunit.client.0.smithi005.stdout: 10 49775 19.07 MB/sec execute 30 sec latency 106.055 ms 2024-09-10T09:54:52.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:54:51 smithi005 ceph-mon[30350]: pgmap v888: 129 pgs: 129 active+clean; 564 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 3.0 MiB/s wr, 424 op/s 2024-09-10T09:54:52.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:54:51 smithi159 ceph-mon[35513]: pgmap v888: 129 pgs: 129 active+clean; 564 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 3.0 MiB/s wr, 424 op/s 2024-09-10T09:54:52.968 INFO:tasks.workunit.client.0.smithi005.stdout: 10 50527 20.06 MB/sec execute 31 sec latency 57.094 ms 2024-09-10T09:54:53.001 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:54:52 smithi159 ceph-mon[35513]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T09:54:53.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:54:52 smithi005 ceph-mon[30350]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T09:54:53.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:54:52 smithi102 ceph-mon[35442]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T09:54:53.968 INFO:tasks.workunit.client.0.smithi005.stdout: 10 51019 20.34 MB/sec execute 32 sec latency 45.925 ms 2024-09-10T09:54:54.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:54:53 smithi005 ceph-mon[30350]: pgmap v889: 129 pgs: 129 active+clean; 566 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 3.3 MiB/s wr, 437 op/s 2024-09-10T09:54:54.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:54:53 smithi159 ceph-mon[35513]: pgmap v889: 129 pgs: 129 active+clean; 566 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 3.3 MiB/s wr, 437 op/s 2024-09-10T09:54:54.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:54:53 smithi102 ceph-mon[35442]: pgmap v889: 129 pgs: 129 active+clean; 566 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 3.3 MiB/s wr, 437 op/s 2024-09-10T09:54:54.968 INFO:tasks.workunit.client.0.smithi005.stdout: 10 51688 20.53 MB/sec execute 33 sec latency 146.472 ms 2024-09-10T09:54:55.968 INFO:tasks.workunit.client.0.smithi005.stdout: 10 52543 21.20 MB/sec execute 34 sec latency 48.195 ms 2024-09-10T09:54:56.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:54:55 smithi005 ceph-mon[30350]: pgmap v890: 129 pgs: 129 active+clean; 568 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 3.6 MiB/s wr, 451 op/s 2024-09-10T09:54:56.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:54:55 smithi159 ceph-mon[35513]: pgmap v890: 129 pgs: 129 active+clean; 568 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 3.6 MiB/s wr, 451 op/s 2024-09-10T09:54:56.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:54:55 smithi102 ceph-mon[35442]: pgmap v890: 129 pgs: 129 active+clean; 568 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 3.6 MiB/s wr, 451 op/s 2024-09-10T09:54:56.968 INFO:tasks.workunit.client.0.smithi005.stdout: 10 53223 21.80 MB/sec execute 35 sec latency 331.872 ms 2024-09-10T09:54:57.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:54:57 smithi005 ceph-mon[30350]: pgmap v891: 129 pgs: 129 active+clean; 580 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 4.6 MiB/s wr, 593 op/s 2024-09-10T09:54:57.968 INFO:tasks.workunit.client.0.smithi005.stdout: 10 53250 21.20 MB/sec execute 36 sec latency 1255.358 ms 2024-09-10T09:54:58.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:54:57 smithi159 ceph-mon[35513]: pgmap v891: 129 pgs: 129 active+clean; 580 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 4.6 MiB/s wr, 593 op/s 2024-09-10T09:54:58.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:54:57 smithi102 ceph-mon[35442]: pgmap v891: 129 pgs: 129 active+clean; 580 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 4.6 MiB/s wr, 593 op/s 2024-09-10T09:54:58.968 INFO:tasks.workunit.client.0.smithi005.stdout: 10 53251 20.63 MB/sec execute 37 sec latency 2255.416 ms 2024-09-10T09:54:59.751 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:54:59 smithi005 ceph-mon[30350]: pgmap v892: 129 pgs: 129 active+clean; 585 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 4.2 MiB/s wr, 480 op/s 2024-09-10T09:54:59.968 INFO:tasks.workunit.client.0.smithi005.stdout: 10 53480 20.40 MB/sec execute 38 sec latency 2812.875 ms 2024-09-10T09:55:00.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:54:59 smithi159 ceph-mon[35513]: pgmap v892: 129 pgs: 129 active+clean; 585 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 4.2 MiB/s wr, 480 op/s 2024-09-10T09:55:00.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:54:59 smithi102 ceph-mon[35442]: pgmap v892: 129 pgs: 129 active+clean; 585 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 4.2 MiB/s wr, 480 op/s 2024-09-10T09:55:00.968 INFO:tasks.workunit.client.0.smithi005.stdout: 10 54183 20.94 MB/sec execute 39 sec latency 208.301 ms 2024-09-10T09:55:01.936 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:55:01 smithi102 ceph-mon[35442]: pgmap v893: 129 pgs: 129 active+clean; 592 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 5.7 MiB/s wr, 575 op/s 2024-09-10T09:55:01.968 INFO:tasks.workunit.client.0.smithi005.stdout: 10 54708 21.14 MB/sec execute 40 sec latency 530.465 ms 2024-09-10T09:55:02.014 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:55:01 smithi005 ceph-mon[30350]: pgmap v893: 129 pgs: 129 active+clean; 592 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 5.7 MiB/s wr, 575 op/s 2024-09-10T09:55:02.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:55:01 smithi159 ceph-mon[35513]: pgmap v893: 129 pgs: 129 active+clean; 592 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 5.7 MiB/s wr, 575 op/s 2024-09-10T09:55:02.968 INFO:tasks.workunit.client.0.smithi005.stdout: 10 55347 21.49 MB/sec execute 41 sec latency 62.939 ms 2024-09-10T09:55:03.968 INFO:tasks.workunit.client.0.smithi005.stdout: 10 55948 21.62 MB/sec execute 42 sec latency 93.644 ms 2024-09-10T09:55:04.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:55:03 smithi005 ceph-mon[30350]: pgmap v894: 129 pgs: 129 active+clean; 588 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 4.9 MiB/s wr, 404 op/s 2024-09-10T09:55:04.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:55:03 smithi159 ceph-mon[35513]: pgmap v894: 129 pgs: 129 active+clean; 588 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 4.9 MiB/s wr, 404 op/s 2024-09-10T09:55:04.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:55:03 smithi102 ceph-mon[35442]: pgmap v894: 129 pgs: 129 active+clean; 588 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 4.9 MiB/s wr, 404 op/s 2024-09-10T09:55:04.968 INFO:tasks.workunit.client.0.smithi005.stdout: 10 56720 21.98 MB/sec execute 43 sec latency 77.454 ms 2024-09-10T09:55:05.968 INFO:tasks.workunit.client.0.smithi005.stdout: 10 57675 22.72 MB/sec execute 44 sec latency 60.186 ms 2024-09-10T09:55:06.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:55:05 smithi005 ceph-mon[30350]: pgmap v895: 129 pgs: 129 active+clean; 588 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 5.0 MiB/s wr, 430 op/s 2024-09-10T09:55:06.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:55:05 smithi159 ceph-mon[35513]: pgmap v895: 129 pgs: 129 active+clean; 588 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 5.0 MiB/s wr, 430 op/s 2024-09-10T09:55:06.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:55:05 smithi102 ceph-mon[35442]: pgmap v895: 129 pgs: 129 active+clean; 588 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 5.0 MiB/s wr, 430 op/s 2024-09-10T09:55:06.968 INFO:tasks.workunit.client.0.smithi005.stdout: 10 58341 23.10 MB/sec execute 45 sec latency 71.832 ms 2024-09-10T09:55:07.011 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T09:55:07.486 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:55:07.969 INFO:tasks.workunit.client.0.smithi005.stdout: 10 59064 23.69 MB/sec execute 46 sec latency 49.155 ms 2024-09-10T09:55:07.999 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:55:07 smithi005 ceph-mon[30350]: pgmap v896: 129 pgs: 129 active+clean; 575 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 5.5 MiB/s wr, 515 op/s 2024-09-10T09:55:07.999 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:55:07 smithi005 ceph-mon[30350]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T09:55:08.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:55:07 smithi159 ceph-mon[35513]: pgmap v896: 129 pgs: 129 active+clean; 575 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 5.5 MiB/s wr, 515 op/s 2024-09-10T09:55:08.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:55:07 smithi159 ceph-mon[35513]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T09:55:08.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:55:07 smithi102 ceph-mon[35442]: pgmap v896: 129 pgs: 129 active+clean; 575 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 5.5 MiB/s wr, 515 op/s 2024-09-10T09:55:08.135 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:55:07 smithi102 ceph-mon[35442]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T09:55:08.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:08.157+0000 7f701ee00640 1 -- 172.21.15.5:0/125336189 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f70181049a0 msgr2=0x7f70181089f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:08.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:08.157+0000 7f701ee00640 1 --2- 172.21.15.5:0/125336189 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f70181049a0 0x7f70181089f0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f7004009900 tx=0x7f700402a400 comp rx=0 tx=0).stop 2024-09-10T09:55:08.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:08.158+0000 7f701ee00640 1 -- 172.21.15.5:0/125336189 shutdown_connections 2024-09-10T09:55:08.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:08.158+0000 7f701ee00640 1 --2- 172.21.15.5:0/125336189 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7018109120 0x7f7018110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:08.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:08.158+0000 7f701ee00640 1 --2- 172.21.15.5:0/125336189 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f70181049a0 0x7f70181089f0 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:08.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:08.158+0000 7f701ee00640 1 --2- 172.21.15.5:0/125336189 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7018103ff0 0x7f70181043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:08.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:08.158+0000 7f701ee00640 1 -- 172.21.15.5:0/125336189 >> 172.21.15.5:0/125336189 conn(0x7f70180ffa90 msgr2=0x7f7018101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:55:08.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:08.158+0000 7f701ee00640 1 -- 172.21.15.5:0/125336189 shutdown_connections 2024-09-10T09:55:08.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:08.158+0000 7f701ee00640 1 -- 172.21.15.5:0/125336189 wait complete. 2024-09-10T09:55:08.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:08.160+0000 7f701ee00640 1 Processor -- start 2024-09-10T09:55:08.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:08.160+0000 7f701ee00640 1 -- start start 2024-09-10T09:55:08.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:08.161+0000 7f701ee00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7018103ff0 0x7f701819a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:08.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:08.161+0000 7f701ee00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f70181049a0 0x7f701819ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:08.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:08.161+0000 7f701ee00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7018109120 0x7f701819efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:08.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:08.161+0000 7f701ee00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f7018114170 con 0x7f7018109120 2024-09-10T09:55:08.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:08.161+0000 7f701ee00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f7018113ff0 con 0x7f70181049a0 2024-09-10T09:55:08.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:08.161+0000 7f701ee00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f70181142f0 con 0x7f7018103ff0 2024-09-10T09:55:08.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:08.161+0000 7f7017400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f70181049a0 0x7f701819ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:08.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:08.162+0000 7f7017400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f70181049a0 0x7f701819ac10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:51230/0 (socket says 172.21.15.5:51230) 2024-09-10T09:55:08.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:08.162+0000 7f7017400640 1 -- 172.21.15.5:0/3497246436 learned_addr learned my addr 172.21.15.5:0/3497246436 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:55:08.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:08.162+0000 7f7017e00640 1 --2- 172.21.15.5:0/3497246436 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7018103ff0 0x7f701819a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:08.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:08.162+0000 7f701ce00640 1 --2- 172.21.15.5:0/3497246436 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7018109120 0x7f701819efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:08.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:08.162+0000 7f7017400640 1 -- 172.21.15.5:0/3497246436 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7018103ff0 msgr2=0x7f701819a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:08.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:08.162+0000 7f7017400640 1 --2- 172.21.15.5:0/3497246436 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7018103ff0 0x7f701819a6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:08.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:08.162+0000 7f7017400640 1 -- 172.21.15.5:0/3497246436 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7018109120 msgr2=0x7f701819efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:08.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:08.162+0000 7f7017400640 1 --2- 172.21.15.5:0/3497246436 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7018109120 0x7f701819efa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:08.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:08.162+0000 7f7017400640 1 -- 172.21.15.5:0/3497246436 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f701819f680 con 0x7f70181049a0 2024-09-10T09:55:08.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:08.163+0000 7f7017400640 1 --2- 172.21.15.5:0/3497246436 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f70181049a0 0x7f701819ac10 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f7004004770 tx=0x7f7004003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:55:08.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:08.163+0000 7f7014c00640 1 -- 172.21.15.5:0/3497246436 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f700403f070 con 0x7f70181049a0 2024-09-10T09:55:08.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:08.163+0000 7f701ee00640 1 -- 172.21.15.5:0/3497246436 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f701819f910 con 0x7f70181049a0 2024-09-10T09:55:08.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:08.163+0000 7f701ee00640 1 -- 172.21.15.5:0/3497246436 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70181a7250 con 0x7f70181049a0 2024-09-10T09:55:08.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:08.164+0000 7f7014c00640 1 -- 172.21.15.5:0/3497246436 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7004031ca0 con 0x7f70181049a0 2024-09-10T09:55:08.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:08.164+0000 7f7014c00640 1 -- 172.21.15.5:0/3497246436 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7004035030 con 0x7f70181049a0 2024-09-10T09:55:08.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:08.169+0000 7f70217f0740 1 Processor -- start 2024-09-10T09:55:08.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:08.169+0000 7f7014c00640 1 -- 172.21.15.5:0/3497246436 <== mon.2 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f7004042070 con 0x7f70181049a0 2024-09-10T09:55:08.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:08.170+0000 7f70217f0740 1 -- start start 2024-09-10T09:55:08.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:08.170+0000 7f70217f0740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556558fc7b90 0x556558fc7f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:08.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:08.170+0000 7f70217f0740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556558fc84b0 0x556558fd2830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:08.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:08.171+0000 7f70217f0740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556558fd2d70 0x556558fd5160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:08.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:08.171+0000 7f70217f0740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x556558f2e3e0 con 0x556558fd2d70 2024-09-10T09:55:08.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:08.171+0000 7f70217f0740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x556558ed5d80 con 0x556558fc84b0 2024-09-10T09:55:08.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:08.171+0000 7f70217f0740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x556558f2a3a0 con 0x556558fc7b90 2024-09-10T09:55:08.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:08.171+0000 7f7014c00640 1 --2- 172.21.15.5:0/3497246436 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f6ffc0a2c80 0x7f6ffc0a5140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:08.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:08.171+0000 7f7014c00640 1 -- 172.21.15.5:0/3497246436 <== mon.2 v2:172.21.15.102:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f70040ef960 con 0x7f70181049a0 2024-09-10T09:55:08.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:08.171+0000 7f701ce00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556558fc7b90 0x556558fc7f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:08.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:08.171+0000 7f701ce00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556558fc7b90 0x556558fc7f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:49338/0 (socket says 172.21.15.5:49338) 2024-09-10T09:55:08.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:08.171+0000 7f701ce00640 1 -- 172.21.15.5:0/3242004951 learned_addr learned my addr 172.21.15.5:0/3242004951 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:55:08.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:08.171+0000 7f7017e00640 1 --2- 172.21.15.5:0/3242004951 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556558fc84b0 0x556558fd2830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:08.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:08.172+0000 7f7017400640 1 --2- 172.21.15.5:0/3242004951 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556558fd2d70 0x556558fd5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:08.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:08.172+0000 7f7017e00640 1 -- 172.21.15.5:0/3242004951 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556558fc7b90 msgr2=0x556558fc7f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:55:08.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:08.172+0000 7f7017e00640 1 --2- 172.21.15.5:0/3242004951 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556558fc7b90 0x556558fc7f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:08.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:08.172+0000 7f7017e00640 1 -- 172.21.15.5:0/3242004951 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556558fd2d70 msgr2=0x556558fd5160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:55:08.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:08.172+0000 7f7017e00640 1 --2- 172.21.15.5:0/3242004951 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556558fd2d70 0x556558fd5160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:08.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:08.172+0000 7f7017e00640 1 -- 172.21.15.5:0/3242004951 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556558ef4ae0 con 0x556558fc84b0 2024-09-10T09:55:08.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:08.172+0000 7f7017e00640 1 --2- 172.21.15.5:0/3242004951 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556558fc84b0 0x556558fd2830 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f7008009b70 tx=0x7f70080117b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fd845b3802f3dc5e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:55:08.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:08.173+0000 7f701ce00640 1 --2- 172.21.15.5:0/3497246436 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f6ffc0a2c80 0x7f6ffc0a5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:08.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:08.173+0000 7f700d600640 1 -- 172.21.15.5:0/3242004951 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7008009e60 con 0x556558fc84b0 2024-09-10T09:55:08.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:08.173+0000 7f700d600640 1 -- 172.21.15.5:0/3242004951 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7008012930 con 0x556558fc84b0 2024-09-10T09:55:08.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:08.173+0000 7f700d600640 1 -- 172.21.15.5:0/3242004951 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f700800fb10 con 0x556558fc84b0 2024-09-10T09:55:08.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:08.174+0000 7f70217f0740 1 -- 172.21.15.5:0/3242004951 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556558fc84b0 msgr2=0x556558fd2830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:08.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:08.174+0000 7f70217f0740 1 --2- 172.21.15.5:0/3242004951 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556558fc84b0 0x556558fd2830 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f7008009b70 tx=0x7f70080117b0 comp rx=0 tx=0).stop 2024-09-10T09:55:08.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:08.174+0000 7f70217f0740 1 -- 172.21.15.5:0/3242004951 shutdown_connections 2024-09-10T09:55:08.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:08.174+0000 7f70217f0740 1 --2- 172.21.15.5:0/3242004951 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556558fd2d70 0x556558fd5160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:08.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:08.174+0000 7f70217f0740 1 --2- 172.21.15.5:0/3242004951 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556558fc84b0 0x556558fd2830 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:08.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:08.174+0000 7f70217f0740 1 --2- 172.21.15.5:0/3242004951 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556558fc7b90 0x556558fc7f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:08.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:08.174+0000 7f70217f0740 1 -- 172.21.15.5:0/3242004951 >> 172.21.15.5:0/3242004951 conn(0x556558fc4120 msgr2=0x556558fc4500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:55:08.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:08.175+0000 7f701ce00640 1 --2- 172.21.15.5:0/3497246436 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f6ffc0a2c80 0x7f6ffc0a5140 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f7010009370 tx=0x7f7010009240 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:55:08.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:08.175+0000 7f70217f0740 1 -- 172.21.15.5:0/3242004951 shutdown_connections 2024-09-10T09:55:08.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:08.175+0000 7f70217f0740 1 -- 172.21.15.5:0/3242004951 wait complete. 2024-09-10T09:55:08.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:08.175+0000 7f70217f0740 1 -- start start 2024-09-10T09:55:08.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:08.176+0000 7f70217f0740 1 Processor -- start 2024-09-10T09:55:08.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:08.176+0000 7f70217f0740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556558fc7b90 0x556558fcfb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:08.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:08.176+0000 7f70217f0740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556558fc84b0 0x556558fd00c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:08.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:08.176+0000 7f70217f0740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556558fd2d70 0x556558fd0600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:08.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:08.176+0000 7f70217f0740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x556558fd59e0 con 0x556558fc7b90 2024-09-10T09:55:08.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:08.176+0000 7f70217f0740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x556558ee90f0 con 0x556558fc84b0 2024-09-10T09:55:08.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:08.176+0000 7f70217f0740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x556558fd5b60 con 0x556558fd2d70 2024-09-10T09:55:08.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:08.177+0000 7f70217f0740 10 client.0 _ll_register_callbacks cb 0x556558eda9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:55:08.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:08.177+0000 7f7017400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556558fc84b0 0x556558fd00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:08.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:08.177+0000 7f7017400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556558fc84b0 0x556558fd00c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:51248/0 (socket says 172.21.15.5:51248) 2024-09-10T09:55:08.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:08.177+0000 7f7017400640 1 -- 172.21.15.5:0/2725179635 learned_addr learned my addr 172.21.15.5:0/2725179635 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:55:08.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:08.177+0000 7f7017e00640 1 --2- 172.21.15.5:0/2725179635 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556558fc7b90 0x556558fcfb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:08.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:08.177+0000 7f7017400640 1 -- 172.21.15.5:0/2725179635 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556558fd2d70 msgr2=0x556558fd0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:55:08.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:08.177+0000 7f7017400640 1 --2- 172.21.15.5:0/2725179635 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556558fd2d70 0x556558fd0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:08.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:08.177+0000 7f7017400640 1 -- 172.21.15.5:0/2725179635 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556558fc7b90 msgr2=0x556558fcfb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:55:08.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:08.177+0000 7f7017400640 1 --2- 172.21.15.5:0/2725179635 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556558fc7b90 0x556558fcfb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:08.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:08.178+0000 7f7017400640 1 -- 172.21.15.5:0/2725179635 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556558c9f3c0 con 0x556558fc84b0 2024-09-10T09:55:08.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:08.178+0000 7f7017400640 1 --2- 172.21.15.5:0/2725179635 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556558fc84b0 0x556558fd00c0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f70040b81d0 tx=0x7f7004005e10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7970e74de1018b5c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:55:08.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:08.178+0000 7f7002000640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-09-10T09:55:08.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:08.178+0000 7f7002000640 1 -- 172.21.15.5:0/2725179635 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70040c0120 con 0x556558fc84b0 2024-09-10T09:55:08.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:08.178+0000 7f7002000640 1 -- 172.21.15.5:0/2725179635 <== mon.2 v2:172.21.15.102:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f70040b83e0 con 0x556558fc84b0 2024-09-10T09:55:08.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:08.178+0000 7f70217f0740 1 -- 172.21.15.5:0/2725179635 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7008012930 con 0x556558fc84b0 2024-09-10T09:55:08.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:08.179+0000 7f7002000640 1 -- 172.21.15.5:0/2725179635 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f70040c49e0 con 0x556558fc84b0 2024-09-10T09:55:08.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:08.179+0000 7f70217f0740 10 client.28709 fetch_fsmap learned FSMap version 42 2024-09-10T09:55:08.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:08.179+0000 7f70217f0740 1 -- 172.21.15.5:0/2725179635 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x556558fe1610 con 0x556558fc84b0 2024-09-10T09:55:08.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:08.179+0000 7f7002000640 1 -- 172.21.15.5:0/2725179635 <== mon.2 v2:172.21.15.102:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7008012930 con 0x556558fc84b0 2024-09-10T09:55:08.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:08.180+0000 7f7002000640 1 -- 172.21.15.5:0/2725179635 <== mon.2 v2:172.21.15.102:3300/0 5 ==== fsmap(e 42) ==== 8370+0+0 (secure 0 0 0) 0x7f70040bb1c0 con 0x556558fc84b0 2024-09-10T09:55:08.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:08.180+0000 7f70217f0740 10 client.28709 fetch_fsmap finished waiting for FSMap version 42 2024-09-10T09:55:08.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:08.180+0000 7f70217f0740 10 client.28709 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T09:55:08.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:08.180+0000 7f70217f0740 20 client.28709 populate_metadata read hostname 'smithi005' 2024-09-10T09:55:08.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:08.180+0000 7f70217f0740 1 --2- 172.21.15.5:0/2725179635 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x556558fe2b60 0x556558fe5020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:08.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:08.180+0000 7f70217f0740 4 client.28709 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:55:08.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:08.180+0000 7f70217f0740 1 -- 172.21.15.5:0/2725179635 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556558f3b950 con 0x556558fe2b60 2024-09-10T09:55:08.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:08.180+0000 7f7017e00640 1 --2- 172.21.15.5:0/2725179635 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x556558fe2b60 0x556558fe5020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:08.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:08.181+0000 7f7017e00640 1 --2- 172.21.15.5:0/2725179635 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x556558fe2b60 0x556558fe5020 crc :-1 s=READY pgs=243 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=da0d2e80c69ee472 server_cookie=92af9f1ec5333df8 in_seq=0 out_seq=0 2024-09-10T09:55:08.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:08.181+0000 7f7002000640 10 client.28709 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T09:55:08.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:08.182+0000 7f7002000640 1 -- 172.21.15.5:0/2725179635 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556558f3b950 con 0x556558fe2b60 2024-09-10T09:55:08.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:08.182+0000 7f7002000640 10 client.28709 handle_command_reply: tid=0 2024-09-10T09:55:08.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:08.182+0000 7f70217f0740 1 client.28709 shutdown 2024-09-10T09:55:08.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:08.183+0000 7f70217f0740 1 -- 172.21.15.5:0/2725179635 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556558fc84b0 msgr2=0x556558fd00c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:08.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:08.183+0000 7f70217f0740 1 --2- 172.21.15.5:0/2725179635 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556558fc84b0 0x556558fd00c0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f70040b81d0 tx=0x7f7004005e10 comp rx=0 tx=0).stop 2024-09-10T09:55:08.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:08.183+0000 7f70217f0740 1 -- 172.21.15.5:0/2725179635 shutdown_connections 2024-09-10T09:55:08.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:08.183+0000 7f70217f0740 1 --2- 172.21.15.5:0/2725179635 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x556558fe2b60 0x556558fe5020 crc :-1 s=READY pgs=243 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:08.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:08.183+0000 7f70217f0740 1 --2- 172.21.15.5:0/2725179635 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556558fd2d70 0x556558fd0600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:08.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:08.183+0000 7f70217f0740 1 --2- 172.21.15.5:0/2725179635 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556558fc84b0 0x556558fd00c0 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:08.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:08.183+0000 7f70217f0740 1 --2- 172.21.15.5:0/2725179635 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556558fc7b90 0x556558fcfb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:08.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:08.183+0000 7f70217f0740 1 -- 172.21.15.5:0/2725179635 >> 172.21.15.5:0/2725179635 conn(0x556558fc4120 msgr2=0x556558fc9e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:55:08.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:08.183+0000 7f7002000640 0 client.28709 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T09:55:08.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:08.183+0000 7f70217f0740 1 -- 172.21.15.5:0/2725179635 shutdown_connections 2024-09-10T09:55:08.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:08.183+0000 7f70217f0740 1 -- 172.21.15.5:0/2725179635 wait complete. 2024-09-10T09:55:08.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:08.183+0000 7f70217f0740 20 client.28709 trim_cache size 0 max 16384 2024-09-10T09:55:08.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:08.211+0000 7f70217f0740 1 Processor -- start 2024-09-10T09:55:08.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:08.211+0000 7f70217f0740 1 -- start start 2024-09-10T09:55:08.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:08.211+0000 7f70217f0740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556558fc7b90 0x556558fea930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:08.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:08.211+0000 7f70217f0740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556558fc84b0 0x556558feae70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:08.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:08.211+0000 7f70217f0740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556558fd2d70 0x556558fce320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:08.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:08.212+0000 7f70217f0740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f6ff800fd20 con 0x556558fc7b90 2024-09-10T09:55:08.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:08.212+0000 7f70217f0740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f6ff8011860 con 0x556558fc84b0 2024-09-10T09:55:08.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:08.212+0000 7f70217f0740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f6ff80103f0 con 0x556558fd2d70 2024-09-10T09:55:08.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:08.212+0000 7f7017400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556558fc84b0 0x556558feae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:08.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:08.212+0000 7f7017e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556558fc7b90 0x556558fea930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:08.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:08.212+0000 7f7017400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556558fc84b0 0x556558feae70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:51264/0 (socket says 172.21.15.5:51264) 2024-09-10T09:55:08.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:08.212+0000 7f7017400640 1 -- 172.21.15.5:0/1309720470 learned_addr learned my addr 172.21.15.5:0/1309720470 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:55:08.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:08.212+0000 7f701ce00640 1 --2- 172.21.15.5:0/1309720470 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556558fd2d70 0x556558fce320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:08.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:08.212+0000 7f7017400640 1 -- 172.21.15.5:0/1309720470 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556558fd2d70 msgr2=0x556558fce320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:55:08.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:08.212+0000 7f7017400640 1 --2- 172.21.15.5:0/1309720470 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556558fd2d70 0x556558fce320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:08.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:08.212+0000 7f7017400640 1 -- 172.21.15.5:0/1309720470 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556558fc7b90 msgr2=0x556558fea930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:55:08.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:08.212+0000 7f7017400640 1 --2- 172.21.15.5:0/1309720470 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556558fc7b90 0x556558fea930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:08.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:08.212+0000 7f7017400640 1 -- 172.21.15.5:0/1309720470 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556558fce0d0 con 0x556558fc84b0 2024-09-10T09:55:08.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:08.213+0000 7f7017400640 1 --2- 172.21.15.5:0/1309720470 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556558fc84b0 0x556558feae70 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f70040bbd00 tx=0x7f70040ba280 comp rx=0 tx=0).ready entity=mon.2 client_cookie=19a97759a30fe9fe server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:55:08.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:08.213+0000 7f700cc00640 1 -- 172.21.15.5:0/1309720470 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70040b83e0 con 0x556558fc84b0 2024-09-10T09:55:08.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:08.213+0000 7f700cc00640 1 -- 172.21.15.5:0/1309720470 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f70040b7c50 con 0x556558fc84b0 2024-09-10T09:55:08.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:08.213+0000 7f700cc00640 1 -- 172.21.15.5:0/1309720470 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70040bb1c0 con 0x556558fc84b0 2024-09-10T09:55:08.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:08.214+0000 7f70217f0740 1 -- 172.21.15.5:0/1309720470 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556558fc84b0 msgr2=0x556558feae70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:08.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:08.214+0000 7f70217f0740 1 --2- 172.21.15.5:0/1309720470 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556558fc84b0 0x556558feae70 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f70040bbd00 tx=0x7f70040ba280 comp rx=0 tx=0).stop 2024-09-10T09:55:08.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:08.214+0000 7f70217f0740 1 -- 172.21.15.5:0/1309720470 shutdown_connections 2024-09-10T09:55:08.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:08.214+0000 7f70217f0740 1 --2- 172.21.15.5:0/1309720470 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556558fd2d70 0x556558fce320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:08.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:08.214+0000 7f70217f0740 1 --2- 172.21.15.5:0/1309720470 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556558fc84b0 0x556558feae70 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:08.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:08.214+0000 7f70217f0740 1 --2- 172.21.15.5:0/1309720470 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556558fc7b90 0x556558fea930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:08.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:08.214+0000 7f70217f0740 1 -- 172.21.15.5:0/1309720470 >> 172.21.15.5:0/1309720470 conn(0x556558fc4120 msgr2=0x556558fc9e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:55:08.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:08.214+0000 7f70217f0740 1 -- 172.21.15.5:0/1309720470 shutdown_connections 2024-09-10T09:55:08.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:08.215+0000 7f70217f0740 1 -- 172.21.15.5:0/1309720470 wait complete. 2024-09-10T09:55:08.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:08.215+0000 7f70217f0740 1 -- start start 2024-09-10T09:55:08.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:08.215+0000 7f70217f0740 1 Processor -- start 2024-09-10T09:55:08.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:08.216+0000 7f70217f0740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556558fc7b90 0x556558fdb060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:08.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:08.216+0000 7f70217f0740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556558fc84b0 0x556558fdb5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:08.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:08.216+0000 7f70217f0740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556558fd2d70 0x556558fcc390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:08.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:08.216+0000 7f70217f0740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f6ff8011190 con 0x556558fd2d70 2024-09-10T09:55:08.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:08.216+0000 7f70217f0740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f6ff8010ac0 con 0x556558fc7b90 2024-09-10T09:55:08.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:08.216+0000 7f70217f0740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f6ff800dd00 con 0x556558fc84b0 2024-09-10T09:55:08.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:08.216+0000 7f70217f0740 10 client.0 _ll_register_callbacks cb 0x7f70040bb940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:55:08.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:08.217+0000 7f7017e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556558fc7b90 0x556558fdb060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:08.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:08.217+0000 7f7017e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556558fc7b90 0x556558fdb060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:51274/0 (socket says 172.21.15.5:51274) 2024-09-10T09:55:08.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:08.217+0000 7f7017e00640 1 -- 172.21.15.5:0/2698088756 learned_addr learned my addr 172.21.15.5:0/2698088756 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:55:08.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:08.217+0000 7f7017400640 1 --2- 172.21.15.5:0/2698088756 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556558fc84b0 0x556558fdb5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:08.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:08.217+0000 7f701ce00640 1 --2- 172.21.15.5:0/2698088756 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556558fd2d70 0x556558fcc390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:08.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:08.218+0000 7f7017400640 1 -- 172.21.15.5:0/2698088756 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556558fc7b90 msgr2=0x556558fdb060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:55:08.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:08.218+0000 7f7017400640 1 --2- 172.21.15.5:0/2698088756 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556558fc7b90 0x556558fdb060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:08.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:08.218+0000 7f7017400640 1 -- 172.21.15.5:0/2698088756 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556558fd2d70 msgr2=0x556558fcc390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:55:08.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:08.218+0000 7f7017400640 1 --2- 172.21.15.5:0/2698088756 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556558fd2d70 0x556558fcc390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:08.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:08.218+0000 7f7017400640 1 -- 172.21.15.5:0/2698088756 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556558fccbc0 con 0x556558fc84b0 2024-09-10T09:55:08.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:08.218+0000 7f7017e00640 1 --2- 172.21.15.5:0/2698088756 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556558fc7b90 0x556558fdb060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:55:08.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:08.218+0000 7f701ce00640 1 --2- 172.21.15.5:0/2698088756 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556558fd2d70 0x556558fcc390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:55:08.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:08.219+0000 7f7017400640 1 --2- 172.21.15.5:0/2698088756 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556558fc84b0 0x556558fdb5a0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f70040b81d0 tx=0x7f70040092e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a7b6e1a71058687f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:55:08.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:08.219+0000 7f7002000640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-09-10T09:55:08.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:08.219+0000 7f7002000640 1 -- 172.21.15.5:0/2698088756 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70040bb3c0 con 0x556558fc84b0 2024-09-10T09:55:08.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:08.219+0000 7f7002000640 1 -- 172.21.15.5:0/2698088756 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f70040bbfd0 con 0x556558fc84b0 2024-09-10T09:55:08.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:08.219+0000 7f70217f0740 1 -- 172.21.15.5:0/2698088756 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f70040b7c50 con 0x556558fc84b0 2024-09-10T09:55:08.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:08.220+0000 7f7002000640 1 -- 172.21.15.5:0/2698088756 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f70040b7e00 con 0x556558fc84b0 2024-09-10T09:55:08.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:08.220+0000 7f70217f0740 10 client.28708 fetch_fsmap learned FSMap version 42 2024-09-10T09:55:08.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:08.220+0000 7f70217f0740 1 -- 172.21.15.5:0/2698088756 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x556558fccbc0 con 0x556558fc84b0 2024-09-10T09:55:08.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:08.222+0000 7f7002000640 1 -- 172.21.15.5:0/2698088756 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f70040b7c50 con 0x556558fc84b0 2024-09-10T09:55:08.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:08.222+0000 7f7002000640 1 -- 172.21.15.5:0/2698088756 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 42) ==== 8370+0+0 (secure 0 0 0) 0x7f70040c12b0 con 0x556558fc84b0 2024-09-10T09:55:08.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:08.223+0000 7f70217f0740 10 client.28708 fetch_fsmap finished waiting for FSMap version 42 2024-09-10T09:55:08.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:08.223+0000 7f70217f0740 10 client.28708 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T09:55:08.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:08.223+0000 7f70217f0740 20 client.28708 populate_metadata read hostname 'smithi005' 2024-09-10T09:55:08.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:08.223+0000 7f70217f0740 1 --2- 172.21.15.5:0/2698088756 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55655904d630 0x55655904fa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:08.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:08.223+0000 7f70217f0740 4 client.28708 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T09:55:08.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:08.223+0000 7f70217f0740 1 -- 172.21.15.5:0/2698088756 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "scrub status"}) -- 0x556558f35e60 con 0x55655904d630 2024-09-10T09:55:08.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:08.223+0000 7f7017e00640 1 --2- 172.21.15.5:0/2698088756 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55655904d630 0x55655904fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:08.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:08.224+0000 7f7017e00640 1 --2- 172.21.15.5:0/2698088756 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55655904d630 0x55655904fa20 crc :-1 s=READY pgs=244 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3e4ee7dce375e018 server_cookie=b648c6bdeebf7517 in_seq=0 out_seq=0 2024-09-10T09:55:08.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:08.224+0000 7f7002000640 10 client.28708 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T09:55:08.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:08.224+0000 7f7002000640 1 -- 172.21.15.5:0/2698088756 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x556558f35e60 con 0x55655904d630 2024-09-10T09:55:08.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:08.224+0000 7f7002000640 10 client.28708 handle_command_reply: tid=0 2024-09-10T09:55:08.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:08.224+0000 7f70217f0740 1 client.28708 shutdown 2024-09-10T09:55:08.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:08.225+0000 7f70217f0740 1 -- 172.21.15.5:0/2698088756 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556558fc84b0 msgr2=0x556558fdb5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:08.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:08.225+0000 7f70217f0740 1 --2- 172.21.15.5:0/2698088756 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556558fc84b0 0x556558fdb5a0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f70040b81d0 tx=0x7f70040092e0 comp rx=0 tx=0).stop 2024-09-10T09:55:08.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:08.225+0000 7f70217f0740 1 -- 172.21.15.5:0/2698088756 shutdown_connections 2024-09-10T09:55:08.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:08.225+0000 7f70217f0740 1 --2- 172.21.15.5:0/2698088756 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55655904d630 0x55655904fa20 crc :-1 s=READY pgs=244 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:08.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:08.225+0000 7f70217f0740 1 --2- 172.21.15.5:0/2698088756 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556558fd2d70 0x556558fcc390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:08.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:08.225+0000 7f70217f0740 1 --2- 172.21.15.5:0/2698088756 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556558fc84b0 0x556558fdb5a0 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:08.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:08.225+0000 7f70217f0740 1 --2- 172.21.15.5:0/2698088756 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556558fc7b90 0x556558fdb060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:08.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:08.225+0000 7f70217f0740 1 -- 172.21.15.5:0/2698088756 >> 172.21.15.5:0/2698088756 conn(0x556558fc4120 msgr2=0x556558fcfb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:55:08.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:08.225+0000 7f7002000640 0 client.28708 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T09:55:08.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:08.225+0000 7f70217f0740 1 -- 172.21.15.5:0/2698088756 shutdown_connections 2024-09-10T09:55:08.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:08.226+0000 7f70217f0740 1 -- 172.21.15.5:0/2698088756 wait complete. 2024-09-10T09:55:08.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:08.226+0000 7f70217f0740 20 client.28708 trim_cache size 0 max 16384 2024-09-10T09:55:08.228 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-10T09:55:08.228 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "no active scrubs running", 2024-09-10T09:55:08.228 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": {} 2024-09-10T09:55:08.228 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-10T09:55:08.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:08.227+0000 7f701ee00640 1 -- 172.21.15.5:0/3497246436 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f6ffc0a2c80 msgr2=0x7f6ffc0a5140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:08.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:08.227+0000 7f701ee00640 1 --2- 172.21.15.5:0/3497246436 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f6ffc0a2c80 0x7f6ffc0a5140 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f7010009370 tx=0x7f7010009240 comp rx=0 tx=0).stop 2024-09-10T09:55:08.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:08.228+0000 7f701ee00640 1 -- 172.21.15.5:0/3497246436 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f70181049a0 msgr2=0x7f701819ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:08.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:08.228+0000 7f701ee00640 1 --2- 172.21.15.5:0/3497246436 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f70181049a0 0x7f701819ac10 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f7004004770 tx=0x7f7004003c10 comp rx=0 tx=0).stop 2024-09-10T09:55:08.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:08.228+0000 7f701ee00640 1 -- 172.21.15.5:0/3497246436 shutdown_connections 2024-09-10T09:55:08.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:08.228+0000 7f701ee00640 1 --2- 172.21.15.5:0/3497246436 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f6ffc0a2c80 0x7f6ffc0a5140 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:08.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:08.228+0000 7f701ee00640 1 --2- 172.21.15.5:0/3497246436 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7018109120 0x7f701819efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:08.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:08.228+0000 7f701ee00640 1 --2- 172.21.15.5:0/3497246436 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f70181049a0 0x7f701819ac10 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:08.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:08.228+0000 7f701ee00640 1 --2- 172.21.15.5:0/3497246436 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7018103ff0 0x7f701819a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:08.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:08.228+0000 7f701ee00640 1 -- 172.21.15.5:0/3497246436 >> 172.21.15.5:0/3497246436 conn(0x7f70180ffa90 msgr2=0x7f7018100d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:55:08.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:08.228+0000 7f701ee00640 1 -- 172.21.15.5:0/3497246436 shutdown_connections 2024-09-10T09:55:08.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:08.228+0000 7f701ee00640 1 -- 172.21.15.5:0/3497246436 wait complete. 2024-09-10T09:55:08.969 INFO:tasks.workunit.client.0.smithi005.stdout: 10 59490 23.56 MB/sec execute 47 sec latency 70.305 ms 2024-09-10T09:55:09.058 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:55:08 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:55:09.058 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:55:08 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:55:09.058 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:55:08 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:55:09.058 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:55:08 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:55:09.059 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:55:08 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:55:09.059 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:55:08 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:55:09.059 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:55:08 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:55:09.059 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:55:08 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:55:09.059 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:55:08 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:55:09.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:55:08 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:55:09.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:55:08 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:55:09.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:55:08 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:55:09.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:55:08 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:55:09.101 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:55:08 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:55:09.101 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:55:08 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:55:09.101 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:55:08 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:55:09.101 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:55:08 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:55:09.101 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:55:08 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:55:09.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:55:08 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:55:09.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:55:08 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:55:09.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:55:08 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:55:09.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:55:08 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:55:09.135 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:55:08 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:55:09.135 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:55:08 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:55:09.135 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:55:08 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:55:09.135 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:55:08 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:55:09.135 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:55:08 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-10T09:55:09.323 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-10T09:55:09.324 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T09:55:09.717 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:55:09 smithi005 ceph-mon[30350]: pgmap v897: 129 pgs: 129 active+clean; 577 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 5.0 MiB/s wr, 435 op/s 2024-09-10T09:55:09.717 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:55:09 smithi005 ceph-mon[30350]: Adjusting osd_memory_target on smithi102 to 176.0M 2024-09-10T09:55:09.869 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:55:09.969 INFO:tasks.workunit.client.0.smithi005.stdout: 10 60048 23.51 MB/sec execute 48 sec latency 195.239 ms 2024-09-10T09:55:10.017 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:55:09 smithi005 ceph-mon[30350]: Unable to set osd_memory_target on smithi102 to 184589619: error parsing value: Value '184589619' is below minimum 939524096 2024-09-10T09:55:10.018 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:55:09 smithi005 ceph-mon[30350]: Adjusting osd_memory_target on smithi159 to 176.0M 2024-09-10T09:55:10.018 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:55:09 smithi005 ceph-mon[30350]: Unable to set osd_memory_target on smithi159 to 184591052: error parsing value: Value '184591052' is below minimum 939524096 2024-09-10T09:55:10.018 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:55:09 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:55:10.018 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:55:09 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:55:10.018 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:55:09 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:55:10.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:55:09 smithi159 ceph-mon[35513]: pgmap v897: 129 pgs: 129 active+clean; 577 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 5.0 MiB/s wr, 435 op/s 2024-09-10T09:55:10.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:55:09 smithi159 ceph-mon[35513]: Adjusting osd_memory_target on smithi102 to 176.0M 2024-09-10T09:55:10.101 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:55:09 smithi159 ceph-mon[35513]: Unable to set osd_memory_target on smithi102 to 184589619: error parsing value: Value '184589619' is below minimum 939524096 2024-09-10T09:55:10.101 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:55:09 smithi159 ceph-mon[35513]: Adjusting osd_memory_target on smithi159 to 176.0M 2024-09-10T09:55:10.101 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:55:09 smithi159 ceph-mon[35513]: Unable to set osd_memory_target on smithi159 to 184591052: error parsing value: Value '184591052' is below minimum 939524096 2024-09-10T09:55:10.101 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:55:09 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:55:10.101 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:55:09 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:55:10.101 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:55:09 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:55:10.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:55:09 smithi102 ceph-mon[35442]: pgmap v897: 129 pgs: 129 active+clean; 577 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 5.0 MiB/s wr, 435 op/s 2024-09-10T09:55:10.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:55:09 smithi102 ceph-mon[35442]: Adjusting osd_memory_target on smithi102 to 176.0M 2024-09-10T09:55:10.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:55:09 smithi102 ceph-mon[35442]: Unable to set osd_memory_target on smithi102 to 184589619: error parsing value: Value '184589619' is below minimum 939524096 2024-09-10T09:55:10.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:55:09 smithi102 ceph-mon[35442]: Adjusting osd_memory_target on smithi159 to 176.0M 2024-09-10T09:55:10.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:55:09 smithi102 ceph-mon[35442]: Unable to set osd_memory_target on smithi159 to 184591052: error parsing value: Value '184591052' is below minimum 939524096 2024-09-10T09:55:10.135 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:55:09 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:55:10.135 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:55:09 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:55:10.135 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:55:09 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:55:10.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:10.554+0000 7f8a2ec00640 1 -- 172.21.15.5:0/1078460107 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8a281091f0 msgr2=0x7f8a281095d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:10.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:10.554+0000 7f8a2ec00640 1 --2- 172.21.15.5:0/1078460107 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8a281091f0 0x7f8a281095d0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f8a18009900 tx=0x7f8a1802a3f0 comp rx=0 tx=0).stop 2024-09-10T09:55:10.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:10.555+0000 7f8a2ec00640 1 -- 172.21.15.5:0/1078460107 shutdown_connections 2024-09-10T09:55:10.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:10.555+0000 7f8a2ec00640 1 --2- 172.21.15.5:0/1078460107 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8a28116ea0 0x7f8a28119290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:10.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:10.555+0000 7f8a2ec00640 1 --2- 172.21.15.5:0/1078460107 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8a28109b10 0x7f8a28116960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:10.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:10.555+0000 7f8a2ec00640 1 --2- 172.21.15.5:0/1078460107 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8a281091f0 0x7f8a281095d0 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:10.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:10.555+0000 7f8a2ec00640 1 -- 172.21.15.5:0/1078460107 >> 172.21.15.5:0/1078460107 conn(0x7f8a2806e140 msgr2=0x7f8a2806e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:55:10.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:10.556+0000 7f8a2ec00640 1 -- 172.21.15.5:0/1078460107 shutdown_connections 2024-09-10T09:55:10.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:10.556+0000 7f8a2ec00640 1 -- 172.21.15.5:0/1078460107 wait complete. 2024-09-10T09:55:10.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:10.557+0000 7f8a2ec00640 1 Processor -- start 2024-09-10T09:55:10.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:10.558+0000 7f8a2ec00640 1 -- start start 2024-09-10T09:55:10.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:10.558+0000 7f8a2ec00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8a281091f0 0x7f8a2810ff80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:10.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:10.558+0000 7f8a2ec00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8a28109b10 0x7f8a281104c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:10.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:10.558+0000 7f8a2ec00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8a28116ea0 0x7f8a281a9230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:10.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:10.559+0000 7f8a27400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8a28109b10 0x7f8a281104c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:10.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:10.559+0000 7f8a2ec00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f8a2811ca00 con 0x7f8a281091f0 2024-09-10T09:55:10.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:10.559+0000 7f8a27400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8a28109b10 0x7f8a281104c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:51292/0 (socket says 172.21.15.5:51292) 2024-09-10T09:55:10.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:10.559+0000 7f8a27400640 1 -- 172.21.15.5:0/151864220 learned_addr learned my addr 172.21.15.5:0/151864220 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:55:10.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:10.559+0000 7f8a2cc00640 1 --2- 172.21.15.5:0/151864220 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8a28116ea0 0x7f8a281a9230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:10.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:10.559+0000 7f8a2ec00640 1 -- 172.21.15.5:0/151864220 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f8a2811c880 con 0x7f8a28109b10 2024-09-10T09:55:10.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:10.559+0000 7f8a2ec00640 1 -- 172.21.15.5:0/151864220 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f8a2811cb80 con 0x7f8a28116ea0 2024-09-10T09:55:10.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:10.559+0000 7f8a27e00640 1 --2- 172.21.15.5:0/151864220 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8a281091f0 0x7f8a2810ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:10.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:10.560+0000 7f8a27400640 1 -- 172.21.15.5:0/151864220 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8a28116ea0 msgr2=0x7f8a281a9230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:10.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:10.560+0000 7f8a27400640 1 --2- 172.21.15.5:0/151864220 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8a28116ea0 0x7f8a281a9230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:10.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:10.560+0000 7f8a27400640 1 -- 172.21.15.5:0/151864220 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8a281091f0 msgr2=0x7f8a2810ff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:10.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:10.560+0000 7f8a27400640 1 --2- 172.21.15.5:0/151864220 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8a281091f0 0x7f8a2810ff80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:10.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:10.560+0000 7f8a27400640 1 -- 172.21.15.5:0/151864220 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a281a9af0 con 0x7f8a28109b10 2024-09-10T09:55:10.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:10.560+0000 7f8a2cc00640 1 --2- 172.21.15.5:0/151864220 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8a28116ea0 0x7f8a281a9230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:55:10.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:10.560+0000 7f8a27e00640 1 --2- 172.21.15.5:0/151864220 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8a281091f0 0x7f8a2810ff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T09:55:10.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:10.560+0000 7f8a27400640 1 --2- 172.21.15.5:0/151864220 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8a28109b10 0x7f8a281104c0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f8a1000ea60 tx=0x7f8a1000ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:55:10.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:10.561+0000 7f8a24c00640 1 -- 172.21.15.5:0/151864220 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a1000cde0 con 0x7f8a28109b10 2024-09-10T09:55:10.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:10.562+0000 7f8a24c00640 1 -- 172.21.15.5:0/151864220 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8a100035f0 con 0x7f8a28109b10 2024-09-10T09:55:10.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:10.562+0000 7f8a2ec00640 1 -- 172.21.15.5:0/151864220 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a281a9de0 con 0x7f8a28109b10 2024-09-10T09:55:10.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:10.562+0000 7f8a2ec00640 1 -- 172.21.15.5:0/151864220 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a28115e70 con 0x7f8a28109b10 2024-09-10T09:55:10.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:10.562+0000 7f8a24c00640 1 -- 172.21.15.5:0/151864220 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a10010670 con 0x7f8a28109b10 2024-09-10T09:55:10.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:10.565+0000 7f8a2ec00640 1 -- 172.21.15.5:0/151864220 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a28073b00 con 0x7f8a28109b10 2024-09-10T09:55:10.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:10.566+0000 7f8a24c00640 1 -- 172.21.15.5:0/151864220 <== mon.2 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f8a100037a0 con 0x7f8a28109b10 2024-09-10T09:55:10.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:10.568+0000 7f8a24c00640 1 --2- 172.21.15.5:0/151864220 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f8a0c0a2bc0 0x7f8a0c0a5080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:10.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:10.568+0000 7f8a24c00640 1 -- 172.21.15.5:0/151864220 <== mon.2 v2:172.21.15.102:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f8a100cf810 con 0x7f8a28109b10 2024-09-10T09:55:10.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:10.569+0000 7f8a27e00640 1 --2- 172.21.15.5:0/151864220 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f8a0c0a2bc0 0x7f8a0c0a5080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:10.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:10.570+0000 7f8a27e00640 1 --2- 172.21.15.5:0/151864220 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f8a0c0a2bc0 0x7f8a0c0a5080 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f8a180096c0 tx=0x7f8a18000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:55:10.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:10.572+0000 7f8a24c00640 1 -- 172.21.15.5:0/151864220 <== mon.2 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f8a1009c320 con 0x7f8a28109b10 2024-09-10T09:55:10.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:10.936+0000 7f8a2ec00640 1 -- 172.21.15.5:0/151864220 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8a28116190 con 0x7f8a28109b10 2024-09-10T09:55:10.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:10.937+0000 7f8a24c00640 1 -- 172.21.15.5:0/151864220 <== mon.2 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 42 v42) ==== 94+0+11969 (secure 0 0 0) 0x7f8a10086090 con 0x7f8a28109b10 2024-09-10T09:55:10.938 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-10T09:55:10.938 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":42,"btime":"2024-09-10T09:55:06:670916+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14454,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/2970594311","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2970594311},{"type":"v1","addr":"172.21.15.5:6835","nonce":2970594311}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14514,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6839/3616502174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":3616502174},{"type":"v1","addr":"172.21.15.5:6839","nonce":3616502174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":42,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T09:35:29.626287+0000","modified":"2024-09-10T09:55:06.419829+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24542,"mds_1":24535,"mds_2":24511,"mds_3":24506,"mds_4":24443},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14520":{"gid":14520,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/2073599842","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":2073599842},{"type":"v1","addr":"172.21.15.5:6841","nonce":2073599842}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"k","rank":4,"incarnation":26,"state":"up:active","state_seq":24,"addr":"172.21.15.102:6833/2457007427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":2457007427},{"type":"v1","addr":"172.21.15.102:6833","nonce":2457007427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/356445159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":356445159},{"type":"v1","addr":"172.21.15.159:6833","nonce":356445159}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6837/2894314507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":2894314507},{"type":"v1","addr":"172.21.15.5:6837","nonce":2894314507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.102:6835/1647549111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":1647549111},{"type":"v1","addr":"172.21.15.102:6835","nonce":1647549111}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6835/952657242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":952657242},{"type":"v1","addr":"172.21.15.159:6835","nonce":952657242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":19,"addr":"172.21.15.102:6837/2689776764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":2689776764},{"type":"v1","addr":"172.21.15.102:6837","nonce":2689776764}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.159:6837/620627639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":620627639},{"type":"v1","addr":"172.21.15.159:6837","nonce":620627639}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24535":{"gid":24535,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":18,"addr":"172.21.15.159:6839/2073739599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":2073739599},{"type":"v1","addr":"172.21.15.159:6839","nonce":2073739599}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24542":{"gid":24542,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.102:6839/335409749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6838","nonce":335409749},{"type":"v1","addr":"172.21.15.102:6839","nonce":335409749}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24542,"qdb_cluster":[24542,24535,24511,24506,24443]},"id":1}]} 2024-09-10T09:55:10.938 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 42 2024-09-10T09:55:10.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:10.939+0000 7f8a2ec00640 1 -- 172.21.15.5:0/151864220 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f8a0c0a2bc0 msgr2=0x7f8a0c0a5080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:10.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:10.939+0000 7f8a2ec00640 1 --2- 172.21.15.5:0/151864220 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f8a0c0a2bc0 0x7f8a0c0a5080 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f8a180096c0 tx=0x7f8a18000bc0 comp rx=0 tx=0).stop 2024-09-10T09:55:10.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:10.939+0000 7f8a2ec00640 1 -- 172.21.15.5:0/151864220 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8a28109b10 msgr2=0x7f8a281104c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:10.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:10.939+0000 7f8a2ec00640 1 --2- 172.21.15.5:0/151864220 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8a28109b10 0x7f8a281104c0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f8a1000ea60 tx=0x7f8a1000ef20 comp rx=0 tx=0).stop 2024-09-10T09:55:10.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:10.939+0000 7f8a2ec00640 1 -- 172.21.15.5:0/151864220 shutdown_connections 2024-09-10T09:55:10.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:10.939+0000 7f8a2ec00640 1 --2- 172.21.15.5:0/151864220 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f8a0c0a2bc0 0x7f8a0c0a5080 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:10.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:10.939+0000 7f8a2ec00640 1 --2- 172.21.15.5:0/151864220 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8a28116ea0 0x7f8a281a9230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:10.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:10.939+0000 7f8a2ec00640 1 --2- 172.21.15.5:0/151864220 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8a28109b10 0x7f8a281104c0 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:10.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:10.939+0000 7f8a2ec00640 1 --2- 172.21.15.5:0/151864220 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8a281091f0 0x7f8a2810ff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:10.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:10.939+0000 7f8a2ec00640 1 -- 172.21.15.5:0/151864220 >> 172.21.15.5:0/151864220 conn(0x7f8a2806e140 msgr2=0x7f8a28117380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:55:10.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:10.940+0000 7f8a2ec00640 1 -- 172.21.15.5:0/151864220 shutdown_connections 2024-09-10T09:55:10.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:10.940+0000 7f8a2ec00640 1 -- 172.21.15.5:0/151864220 wait complete. 2024-09-10T09:55:10.969 INFO:tasks.workunit.client.0.smithi005.stdout: 10 60653 23.61 MB/sec execute 49 sec latency 63.620 ms 2024-09-10T09:55:11.872 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-10T09:55:11.969 INFO:tasks.workunit.client.0.smithi005.stdout: 10 61422 24.09 MB/sec execute 50 sec latency 67.536 ms 2024-09-10T09:55:12.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:55:11 smithi005 ceph-mon[30350]: pgmap v898: 129 pgs: 129 active+clean; 584 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 5.1 MiB/s wr, 492 op/s 2024-09-10T09:55:12.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:55:11 smithi005 ceph-mon[30350]: from='client.? 172.21.15.5:0/151864220' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:55:12.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:55:11 smithi159 ceph-mon[35513]: pgmap v898: 129 pgs: 129 active+clean; 584 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 5.1 MiB/s wr, 492 op/s 2024-09-10T09:55:12.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:55:11 smithi159 ceph-mon[35513]: from='client.? 172.21.15.5:0/151864220' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:55:12.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:55:11 smithi102 ceph-mon[35442]: pgmap v898: 129 pgs: 129 active+clean; 584 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 5.1 MiB/s wr, 492 op/s 2024-09-10T09:55:12.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:55:11 smithi102 ceph-mon[35442]: from='client.? 172.21.15.5:0/151864220' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:55:12.420 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:55:12.979 INFO:tasks.workunit.client.0.smithi005.stdout: 10 61484 23.66 MB/sec execute 51 sec latency 921.025 ms 2024-09-10T09:55:13.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:13.367+0000 7f1914a00640 1 -- 172.21.15.5:0/3344559974 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1910102360 msgr2=0x7f191010e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:13.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:13.367+0000 7f1914a00640 1 --2- 172.21.15.5:0/3344559974 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1910102360 0x7f191010e890 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f1908009cd0 tx=0x7f190802a8e0 comp rx=0 tx=0).stop 2024-09-10T09:55:13.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:13.367+0000 7f1914a00640 1 -- 172.21.15.5:0/3344559974 shutdown_connections 2024-09-10T09:55:13.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:13.367+0000 7f1914a00640 1 --2- 172.21.15.5:0/3344559974 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1910102360 0x7f191010e890 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:13.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:13.367+0000 7f1914a00640 1 --2- 172.21.15.5:0/3344559974 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f19101019c0 0x7f1910101e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:13.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:13.367+0000 7f1914a00640 1 --2- 172.21.15.5:0/3344559974 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f19101079c0 0x7f1910107da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:13.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:13.367+0000 7f1914a00640 1 -- 172.21.15.5:0/3344559974 >> 172.21.15.5:0/3344559974 conn(0x7f19100fd740 msgr2=0x7f19100ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:55:13.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:13.368+0000 7f1914a00640 1 -- 172.21.15.5:0/3344559974 shutdown_connections 2024-09-10T09:55:13.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:13.368+0000 7f1914a00640 1 -- 172.21.15.5:0/3344559974 wait complete. 2024-09-10T09:55:13.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:13.369+0000 7f1914a00640 1 Processor -- start 2024-09-10T09:55:13.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:13.369+0000 7f1914a00640 1 -- start start 2024-09-10T09:55:13.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:13.369+0000 7f1914a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f19101019c0 0x7f191019a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:13.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:13.369+0000 7f1914a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1910102360 0x7f191019a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:13.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:13.369+0000 7f1914a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f19101079c0 0x7f191019ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:13.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:13.369+0000 7f1914a00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1910111dc0 con 0x7f19101079c0 2024-09-10T09:55:13.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:13.369+0000 7f1914a00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f1910111c40 con 0x7f19101019c0 2024-09-10T09:55:13.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:13.369+0000 7f1914a00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f1910111f40 con 0x7f1910102360 2024-09-10T09:55:13.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:13.369+0000 7f190fe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f19101079c0 0x7f191019ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:13.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:13.369+0000 7f190fe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f19101079c0 0x7f191019ed50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53198/0 (socket says 172.21.15.5:53198) 2024-09-10T09:55:13.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:13.369+0000 7f190fe00640 1 -- 172.21.15.5:0/999964381 learned_addr learned my addr 172.21.15.5:0/999964381 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:55:13.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:13.369+0000 7f190f400640 1 --2- 172.21.15.5:0/999964381 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f19101019c0 0x7f191019a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:13.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:13.370+0000 7f190ea00640 1 --2- 172.21.15.5:0/999964381 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1910102360 0x7f191019a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:13.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:13.370+0000 7f190f400640 1 -- 172.21.15.5:0/999964381 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1910102360 msgr2=0x7f191019a9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:13.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:13.370+0000 7f190f400640 1 --2- 172.21.15.5:0/999964381 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1910102360 0x7f191019a9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:13.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:13.370+0000 7f190f400640 1 -- 172.21.15.5:0/999964381 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f19101079c0 msgr2=0x7f191019ed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:13.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:13.370+0000 7f190f400640 1 --2- 172.21.15.5:0/999964381 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f19101079c0 0x7f191019ed50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:13.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:13.370+0000 7f190f400640 1 -- 172.21.15.5:0/999964381 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f191019f430 con 0x7f19101019c0 2024-09-10T09:55:13.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:13.370+0000 7f190f400640 1 --2- 172.21.15.5:0/999964381 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f19101019c0 0x7f191019a480 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f190000e9f0 tx=0x7f190000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:55:13.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:13.371+0000 7f1907e00640 1 -- 172.21.15.5:0/999964381 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f190000cd80 con 0x7f19101019c0 2024-09-10T09:55:13.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:13.371+0000 7f1907e00640 1 -- 172.21.15.5:0/999964381 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f19000035f0 con 0x7f19101019c0 2024-09-10T09:55:13.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:13.371+0000 7f1914a00640 1 -- 172.21.15.5:0/999964381 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f191019f720 con 0x7f19101019c0 2024-09-10T09:55:13.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:13.371+0000 7f1907e00640 1 -- 172.21.15.5:0/999964381 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1900010690 con 0x7f19101019c0 2024-09-10T09:55:13.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:13.371+0000 7f1914a00640 1 -- 172.21.15.5:0/999964381 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19101a7000 con 0x7f19101019c0 2024-09-10T09:55:13.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:13.374+0000 7f1918ab0740 1 Processor -- start 2024-09-10T09:55:13.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:13.374+0000 7f1907e00640 1 -- 172.21.15.5:0/999964381 <== mon.2 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f1900003b30 con 0x7f19101019c0 2024-09-10T09:55:13.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:13.374+0000 7f1918ab0740 1 -- start start 2024-09-10T09:55:13.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:13.374+0000 7f1918ab0740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d281239e40 0x55d28123a220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:13.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:13.375+0000 7f1918ab0740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d28123a760 0x55d281244ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:13.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:13.375+0000 7f1918ab0740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d281245020 0x55d281247410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:13.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:13.375+0000 7f1918ab0740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55d2811a03e0 con 0x55d281245020 2024-09-10T09:55:13.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:13.375+0000 7f1918ab0740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55d281147d80 con 0x55d281239e40 2024-09-10T09:55:13.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:13.375+0000 7f1918ab0740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55d28119c3a0 con 0x55d28123a760 2024-09-10T09:55:13.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:13.375+0000 7f190fe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d281239e40 0x55d28123a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:13.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:13.375+0000 7f190fe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d281239e40 0x55d28123a220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:35216/0 (socket says 172.21.15.5:35216) 2024-09-10T09:55:13.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:13.375+0000 7f190fe00640 1 -- 172.21.15.5:0/1427152975 learned_addr learned my addr 172.21.15.5:0/1427152975 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:55:13.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:13.375+0000 7f190ea00640 1 --2- 172.21.15.5:0/1427152975 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d281245020 0x55d281247410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:13.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:13.375+0000 7f190fe00640 1 -- 172.21.15.5:0/1427152975 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d28123a760 msgr2=0x55d281244ae0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T09:55:13.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:13.375+0000 7f190fe00640 1 --2- 172.21.15.5:0/1427152975 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d28123a760 0x55d281244ae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:13.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:13.375+0000 7f190fe00640 1 -- 172.21.15.5:0/1427152975 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d281245020 msgr2=0x55d281247410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:55:13.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:13.375+0000 7f190fe00640 1 --2- 172.21.15.5:0/1427152975 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d281245020 0x55d281247410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:13.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:13.375+0000 7f190fe00640 1 -- 172.21.15.5:0/1427152975 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d281166ae0 con 0x55d281239e40 2024-09-10T09:55:13.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:13.376+0000 7f190fe00640 1 --2- 172.21.15.5:0/1427152975 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d281239e40 0x55d28123a220 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f1908009a20 tx=0x7f1908005ef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e0b355a138abdc29 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:55:13.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:13.376+0000 7f18fbe00640 1 -- 172.21.15.5:0/1427152975 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1908048070 con 0x55d281239e40 2024-09-10T09:55:13.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:13.376+0000 7f1907e00640 1 --2- 172.21.15.5:0/999964381 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f18f40a2b80 0x7f18f40a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:13.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:13.376+0000 7f18fbe00640 1 -- 172.21.15.5:0/1427152975 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1908036070 con 0x55d281239e40 2024-09-10T09:55:13.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:13.376+0000 7f18fbe00640 1 -- 172.21.15.5:0/1427152975 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1908004390 con 0x55d281239e40 2024-09-10T09:55:13.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:13.376+0000 7f1907e00640 1 -- 172.21.15.5:0/999964381 <== mon.2 v2:172.21.15.102:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f19000cf710 con 0x7f19101019c0 2024-09-10T09:55:13.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:13.377+0000 7f190f400640 1 --2- 172.21.15.5:0/999964381 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f18f40a2b80 0x7f18f40a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:13.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:13.377+0000 7f1918ab0740 1 -- 172.21.15.5:0/1427152975 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d281239e40 msgr2=0x55d28123a220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:13.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:13.377+0000 7f1918ab0740 1 --2- 172.21.15.5:0/1427152975 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d281239e40 0x55d28123a220 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f1908009a20 tx=0x7f1908005ef0 comp rx=0 tx=0).stop 2024-09-10T09:55:13.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:13.377+0000 7f1918ab0740 1 -- 172.21.15.5:0/1427152975 shutdown_connections 2024-09-10T09:55:13.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:13.377+0000 7f1918ab0740 1 --2- 172.21.15.5:0/1427152975 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d281245020 0x55d281247410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:13.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:13.377+0000 7f1918ab0740 1 --2- 172.21.15.5:0/1427152975 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d28123a760 0x55d281244ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:13.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:13.377+0000 7f1918ab0740 1 --2- 172.21.15.5:0/1427152975 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d281239e40 0x55d28123a220 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:13.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:13.377+0000 7f1918ab0740 1 -- 172.21.15.5:0/1427152975 >> 172.21.15.5:0/1427152975 conn(0x55d2812363d0 msgr2=0x55d2812367b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:55:13.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:13.377+0000 7f190f400640 1 --2- 172.21.15.5:0/999964381 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f18f40a2b80 0x7f18f40a5040 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f19000cf9b0 tx=0x7f19000977c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:55:13.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:13.377+0000 7f1918ab0740 1 -- 172.21.15.5:0/1427152975 shutdown_connections 2024-09-10T09:55:13.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:13.377+0000 7f1918ab0740 1 -- 172.21.15.5:0/1427152975 wait complete. 2024-09-10T09:55:13.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:13.378+0000 7f1918ab0740 1 -- start start 2024-09-10T09:55:13.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:13.378+0000 7f1918ab0740 1 Processor -- start 2024-09-10T09:55:13.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:13.378+0000 7f1918ab0740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d281239e40 0x55d281241f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:13.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:13.378+0000 7f1918ab0740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d28123a760 0x55d2812424a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:13.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:13.379+0000 7f1918ab0740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d281245020 0x55d2812429e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:13.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:13.379+0000 7f1918ab0740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55d281247bf0 con 0x55d281245020 2024-09-10T09:55:13.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:13.379+0000 7f1918ab0740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55d28115b0f0 con 0x55d28123a760 2024-09-10T09:55:13.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:13.379+0000 7f1918ab0740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55d281247d70 con 0x55d281239e40 2024-09-10T09:55:13.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:13.379+0000 7f1918ab0740 10 client.0 _ll_register_callbacks cb 0x55d281142bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:55:13.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:13.379+0000 7f190fe00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d281239e40 0x55d281241f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:13.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:13.379+0000 7f190fe00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d281239e40 0x55d281241f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:42288/0 (socket says 172.21.15.5:42288) 2024-09-10T09:55:13.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:13.379+0000 7f190fe00640 1 -- 172.21.15.5:0/4263323176 learned_addr learned my addr 172.21.15.5:0/4263323176 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:55:13.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:13.379+0000 7f190fe00640 1 --2- 172.21.15.5:0/4263323176 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d281245020 0x55d2812429e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:13.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:13.380+0000 7f190fe00640 1 -- 172.21.15.5:0/4263323176 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d281239e40 msgr2=0x55d281241f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:55:13.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:13.380+0000 7f190fe00640 1 --2- 172.21.15.5:0/4263323176 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d281239e40 0x55d281241f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:13.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:13.380+0000 7f190fe00640 1 -- 172.21.15.5:0/4263323176 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d28123a760 msgr2=0x55d2812424a0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T09:55:13.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:13.380+0000 7f190fe00640 1 --2- 172.21.15.5:0/4263323176 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d28123a760 0x55d2812424a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:13.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:13.380+0000 7f190fe00640 1 -- 172.21.15.5:0/4263323176 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d280f113c0 con 0x55d281245020 2024-09-10T09:55:13.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:13.380+0000 7f190fe00640 1 --2- 172.21.15.5:0/4263323176 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d281245020 0x55d2812429e0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f1908046040 tx=0x7f190802a8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9057e108319083ea server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:55:13.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:13.380+0000 7f18f8c00640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-10T09:55:13.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:13.380+0000 7f18f8c00640 1 -- 172.21.15.5:0/4263323176 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1908056060 con 0x55d281245020 2024-09-10T09:55:13.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:13.380+0000 7f18f8c00640 1 -- 172.21.15.5:0/4263323176 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f1908003b80 con 0x55d281245020 2024-09-10T09:55:13.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:13.380+0000 7f1918ab0740 1 -- 172.21.15.5:0/4263323176 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1908036070 con 0x55d281245020 2024-09-10T09:55:13.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:13.381+0000 7f18f8c00640 1 -- 172.21.15.5:0/4263323176 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f190805d050 con 0x55d281245020 2024-09-10T09:55:13.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:13.381+0000 7f1918ab0740 10 client.18660 fetch_fsmap learned FSMap version 42 2024-09-10T09:55:13.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:13.381+0000 7f1918ab0740 1 -- 172.21.15.5:0/4263323176 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55d281247bf0 con 0x55d281245020 2024-09-10T09:55:13.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:13.381+0000 7f18f8c00640 1 -- 172.21.15.5:0/4263323176 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1908036070 con 0x55d281245020 2024-09-10T09:55:13.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:13.381+0000 7f18f8c00640 1 -- 172.21.15.5:0/4263323176 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 42) ==== 8370+0+0 (secure 0 0 0) 0x7f190804e040 con 0x55d281245020 2024-09-10T09:55:13.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:13.381+0000 7f1918ab0740 10 client.18660 fetch_fsmap finished waiting for FSMap version 42 2024-09-10T09:55:13.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:13.381+0000 7f1918ab0740 10 client.18660 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-09-10T09:55:13.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:13.381+0000 7f1918ab0740 20 client.18660 populate_metadata read hostname 'smithi005' 2024-09-10T09:55:13.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:13.381+0000 7f1918ab0740 1 --2- 172.21.15.5:0/4263323176 >> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] conn(0x55d281254e00 0x55d2812571f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:13.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:13.381+0000 7f1918ab0740 4 client.18660 mds_command: new command op to 24443 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:55:13.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:13.381+0000 7f1918ab0740 1 -- 172.21.15.5:0/4263323176 --> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d2811ad950 con 0x55d281254e00 2024-09-10T09:55:13.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:13.382+0000 7f190ea00640 1 --2- 172.21.15.5:0/4263323176 >> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] conn(0x55d281254e00 0x55d2812571f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:13.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:13.382+0000 7f190ea00640 1 --2- 172.21.15.5:0/4263323176 >> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] conn(0x55d281254e00 0x55d2812571f0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=2456eeecfd650015 server_cookie=47830592b7bae3cc in_seq=0 out_seq=0 2024-09-10T09:55:13.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:13.382+0000 7f18f8c00640 10 client.18660 ms_handle_connect on v2:172.21.15.102:6832/2457007427 2024-09-10T09:55:13.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:13.383+0000 7f18f8c00640 1 -- 172.21.15.5:0/4263323176 <== mds.4 v2:172.21.15.102:6832/2457007427 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d2811ad950 con 0x55d281254e00 2024-09-10T09:55:13.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:13.383+0000 7f18f8c00640 10 client.18660 handle_command_reply: tid=0 2024-09-10T09:55:13.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:13.383+0000 7f1918ab0740 1 client.18660 shutdown 2024-09-10T09:55:13.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:13.383+0000 7f1918ab0740 1 -- 172.21.15.5:0/4263323176 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d281245020 msgr2=0x55d2812429e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:13.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:13.383+0000 7f1918ab0740 1 --2- 172.21.15.5:0/4263323176 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d281245020 0x55d2812429e0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f1908046040 tx=0x7f190802a8e0 comp rx=0 tx=0).stop 2024-09-10T09:55:13.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:13.383+0000 7f1918ab0740 1 -- 172.21.15.5:0/4263323176 shutdown_connections 2024-09-10T09:55:13.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:13.383+0000 7f1918ab0740 1 --2- 172.21.15.5:0/4263323176 >> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] conn(0x55d281254e00 0x55d2812571f0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:13.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:13.383+0000 7f1918ab0740 1 --2- 172.21.15.5:0/4263323176 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d281245020 0x55d2812429e0 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:13.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:13.383+0000 7f1918ab0740 1 --2- 172.21.15.5:0/4263323176 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d28123a760 0x55d2812424a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:13.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:13.383+0000 7f1918ab0740 1 --2- 172.21.15.5:0/4263323176 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d281239e40 0x55d281241f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:13.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:13.383+0000 7f1918ab0740 1 -- 172.21.15.5:0/4263323176 >> 172.21.15.5:0/4263323176 conn(0x55d2812363d0 msgr2=0x55d28123c0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:55:13.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:13.383+0000 7f18f8c00640 0 client.18660 ms_handle_reset on v2:172.21.15.102:6832/2457007427 2024-09-10T09:55:13.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:13.384+0000 7f1918ab0740 1 -- 172.21.15.5:0/4263323176 shutdown_connections 2024-09-10T09:55:13.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:13.384+0000 7f1918ab0740 1 -- 172.21.15.5:0/4263323176 wait complete. 2024-09-10T09:55:13.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:13.384+0000 7f1918ab0740 20 client.18660 trim_cache size 0 max 16384 2024-09-10T09:55:13.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:13.409+0000 7f1918ab0740 1 Processor -- start 2024-09-10T09:55:13.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:13.410+0000 7f1918ab0740 1 -- start start 2024-09-10T09:55:13.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:13.410+0000 7f1918ab0740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d281239e40 0x55d28125c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:13.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:13.410+0000 7f1918ab0740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d28123a760 0x55d28125ce20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:13.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:13.410+0000 7f1918ab0740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d281245020 0x55d28123b890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:13.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:13.410+0000 7f1918ab0740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f19101b0e70 con 0x55d281245020 2024-09-10T09:55:13.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:13.410+0000 7f1918ab0740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f19101b29b0 con 0x55d28123a760 2024-09-10T09:55:13.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:13.410+0000 7f1918ab0740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f19101b1540 con 0x55d281239e40 2024-09-10T09:55:13.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:13.410+0000 7f190fe00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d281239e40 0x55d28125c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:13.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:13.410+0000 7f190fe00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d281239e40 0x55d28125c8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:42292/0 (socket says 172.21.15.5:42292) 2024-09-10T09:55:13.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:13.410+0000 7f190fe00640 1 -- 172.21.15.5:0/1575812979 learned_addr learned my addr 172.21.15.5:0/1575812979 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:55:13.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:13.410+0000 7f190fe00640 1 --2- 172.21.15.5:0/1575812979 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d281245020 0x55d28123b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:13.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:13.411+0000 7f190fe00640 1 -- 172.21.15.5:0/1575812979 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d28123a760 msgr2=0x55d28125ce20 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T09:55:13.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:13.411+0000 7f190fe00640 1 --2- 172.21.15.5:0/1575812979 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d28123a760 0x55d28125ce20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:13.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:13.411+0000 7f190fe00640 1 -- 172.21.15.5:0/1575812979 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d281245020 msgr2=0x55d28123b890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:55:13.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:13.411+0000 7f190fe00640 1 --2- 172.21.15.5:0/1575812979 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d281245020 0x55d28123b890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:13.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:13.411+0000 7f190fe00640 1 -- 172.21.15.5:0/1575812979 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d28123a590 con 0x55d281239e40 2024-09-10T09:55:13.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:13.411+0000 7f190fe00640 1 --2- 172.21.15.5:0/1575812979 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d281239e40 0x55d28125c8e0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f1908009620 tx=0x7f1908004040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=14bb8720745b07d0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:55:13.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:13.411+0000 7f18fb400640 1 -- 172.21.15.5:0/1575812979 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1908056040 con 0x55d281239e40 2024-09-10T09:55:13.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:13.412+0000 7f18fb400640 1 -- 172.21.15.5:0/1575812979 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1908036070 con 0x55d281239e40 2024-09-10T09:55:13.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:13.412+0000 7f18fb400640 1 -- 172.21.15.5:0/1575812979 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1908048040 con 0x55d281239e40 2024-09-10T09:55:13.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:13.413+0000 7f1918ab0740 1 -- 172.21.15.5:0/1575812979 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d281239e40 msgr2=0x55d28125c8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:13.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:13.413+0000 7f1918ab0740 1 --2- 172.21.15.5:0/1575812979 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d281239e40 0x55d28125c8e0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f1908009620 tx=0x7f1908004040 comp rx=0 tx=0).stop 2024-09-10T09:55:13.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:13.413+0000 7f1918ab0740 1 -- 172.21.15.5:0/1575812979 shutdown_connections 2024-09-10T09:55:13.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:13.413+0000 7f1918ab0740 1 --2- 172.21.15.5:0/1575812979 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d281245020 0x55d28123b890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:13.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:13.413+0000 7f1918ab0740 1 --2- 172.21.15.5:0/1575812979 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d28123a760 0x55d28125ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:13.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:13.413+0000 7f1918ab0740 1 --2- 172.21.15.5:0/1575812979 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d281239e40 0x55d28125c8e0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:13.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:13.413+0000 7f1918ab0740 1 -- 172.21.15.5:0/1575812979 >> 172.21.15.5:0/1575812979 conn(0x55d2812363d0 msgr2=0x55d281243fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:55:13.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:13.413+0000 7f1918ab0740 1 -- 172.21.15.5:0/1575812979 shutdown_connections 2024-09-10T09:55:13.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:13.413+0000 7f1918ab0740 1 -- 172.21.15.5:0/1575812979 wait complete. 2024-09-10T09:55:13.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:13.414+0000 7f1918ab0740 1 -- start start 2024-09-10T09:55:13.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:13.414+0000 7f1918ab0740 1 Processor -- start 2024-09-10T09:55:13.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:13.415+0000 7f1918ab0740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d281239e40 0x55d28124d490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:13.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:13.415+0000 7f1918ab0740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d28123a760 0x55d281248810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:13.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:13.415+0000 7f1918ab0740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d281245020 0x55d281248d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:13.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:13.415+0000 7f1918ab0740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f19101b22e0 con 0x55d281245020 2024-09-10T09:55:13.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:13.415+0000 7f1918ab0740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f19101b1c10 con 0x55d28123a760 2024-09-10T09:55:13.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:13.415+0000 7f1918ab0740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f19101aee50 con 0x55d281239e40 2024-09-10T09:55:13.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:13.415+0000 7f1918ab0740 10 client.0 _ll_register_callbacks cb 0x7f1908009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:55:13.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:13.415+0000 7f190fe00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d281239e40 0x55d28124d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:13.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:13.415+0000 7f190fe00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d281239e40 0x55d28124d490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:42298/0 (socket says 172.21.15.5:42298) 2024-09-10T09:55:13.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:13.415+0000 7f190fe00640 1 -- 172.21.15.5:0/2443698138 learned_addr learned my addr 172.21.15.5:0/2443698138 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:55:13.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:13.415+0000 7f190fe00640 1 --2- 172.21.15.5:0/2443698138 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d281245020 0x55d281248d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:13.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:13.416+0000 7f190fe00640 1 -- 172.21.15.5:0/2443698138 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d28123a760 msgr2=0x55d281248810 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T09:55:13.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:13.416+0000 7f190fe00640 1 --2- 172.21.15.5:0/2443698138 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d28123a760 0x55d281248810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:13.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:13.416+0000 7f190fe00640 1 -- 172.21.15.5:0/2443698138 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d281245020 msgr2=0x55d281248d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:55:13.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:13.416+0000 7f190fe00640 1 --2- 172.21.15.5:0/2443698138 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d281245020 0x55d281248d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:13.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:13.416+0000 7f190fe00640 1 -- 172.21.15.5:0/2443698138 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d281249310 con 0x55d281239e40 2024-09-10T09:55:13.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:13.416+0000 7f190fe00640 1 --2- 172.21.15.5:0/2443698138 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d281239e40 0x55d28124d490 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f1908006010 tx=0x7f1908053040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c34bcf113ee2e6ba server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:55:13.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:13.416+0000 7f18f8c00640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-09-10T09:55:13.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:13.416+0000 7f18f8c00640 1 -- 172.21.15.5:0/2443698138 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1908056070 con 0x55d281239e40 2024-09-10T09:55:13.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:13.417+0000 7f18f8c00640 1 -- 172.21.15.5:0/2443698138 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f19080044e0 con 0x55d281239e40 2024-09-10T09:55:13.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:13.417+0000 7f1918ab0740 1 -- 172.21.15.5:0/2443698138 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1908036070 con 0x55d281239e40 2024-09-10T09:55:13.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:13.417+0000 7f18f8c00640 1 -- 172.21.15.5:0/2443698138 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f19080046e0 con 0x55d281239e40 2024-09-10T09:55:13.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:13.418+0000 7f1918ab0740 10 client.28729 fetch_fsmap learned FSMap version 42 2024-09-10T09:55:13.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:13.418+0000 7f1918ab0740 1 -- 172.21.15.5:0/2443698138 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55d2812bf620 con 0x55d281239e40 2024-09-10T09:55:13.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:13.419+0000 7f18f8c00640 1 -- 172.21.15.5:0/2443698138 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1908036070 con 0x55d281239e40 2024-09-10T09:55:13.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:13.419+0000 7f18f8c00640 1 -- 172.21.15.5:0/2443698138 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 42) ==== 8370+0+0 (secure 0 0 0) 0x7f1908042030 con 0x55d281239e40 2024-09-10T09:55:13.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:13.419+0000 7f1918ab0740 10 client.28729 fetch_fsmap finished waiting for FSMap version 42 2024-09-10T09:55:13.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:13.419+0000 7f1918ab0740 10 client.28729 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-09-10T09:55:13.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:13.420+0000 7f1918ab0740 20 client.28729 populate_metadata read hostname 'smithi005' 2024-09-10T09:55:13.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:13.420+0000 7f1918ab0740 1 --2- 172.21.15.5:0/2443698138 >> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] conn(0x55d2812bfce0 0x55d2812c2100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:13.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:13.420+0000 7f1918ab0740 4 client.28729 mds_command: new command op to 24443 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T09:55:13.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:13.420+0000 7f1918ab0740 1 -- 172.21.15.5:0/2443698138 --> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d2811a7e60 con 0x55d2812bfce0 2024-09-10T09:55:13.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:13.442+0000 7f190ea00640 1 --2- 172.21.15.5:0/2443698138 >> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] conn(0x55d2812bfce0 0x55d2812c2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:13.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:13.501+0000 7f190ea00640 1 --2- 172.21.15.5:0/2443698138 >> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] conn(0x55d2812bfce0 0x55d2812c2100 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=1dcfc84bf18f2a33 server_cookie=630bde8b34512305 in_seq=0 out_seq=0 2024-09-10T09:55:13.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:13.501+0000 7f18f8c00640 10 client.28729 ms_handle_connect on v2:172.21.15.102:6832/2457007427 2024-09-10T09:55:13.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:13.513+0000 7f18f8c00640 1 -- 172.21.15.5:0/2443698138 <== mds.4 v2:172.21.15.102:6832/2457007427 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d2811a7e60 con 0x55d2812bfce0 2024-09-10T09:55:13.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:13.513+0000 7f18f8c00640 10 client.28729 handle_command_reply: tid=0 2024-09-10T09:55:13.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:13.513+0000 7f1918ab0740 1 client.28729 shutdown 2024-09-10T09:55:13.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:13.513+0000 7f1918ab0740 1 -- 172.21.15.5:0/2443698138 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d281239e40 msgr2=0x55d28124d490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:13.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:13.513+0000 7f1918ab0740 1 --2- 172.21.15.5:0/2443698138 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d281239e40 0x55d28124d490 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f1908006010 tx=0x7f1908053040 comp rx=0 tx=0).stop 2024-09-10T09:55:13.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:13.514+0000 7f1918ab0740 1 -- 172.21.15.5:0/2443698138 shutdown_connections 2024-09-10T09:55:13.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:13.514+0000 7f1918ab0740 1 --2- 172.21.15.5:0/2443698138 >> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] conn(0x55d2812bfce0 0x55d2812c2100 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:13.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:13.514+0000 7f1918ab0740 1 --2- 172.21.15.5:0/2443698138 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d281245020 0x55d281248d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:13.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:13.514+0000 7f1918ab0740 1 --2- 172.21.15.5:0/2443698138 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d28123a760 0x55d281248810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:13.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:13.514+0000 7f18f8c00640 0 client.28729 ms_handle_reset on v2:172.21.15.102:6832/2457007427 2024-09-10T09:55:13.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:13.514+0000 7f1918ab0740 1 --2- 172.21.15.5:0/2443698138 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d281239e40 0x55d28124d490 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:13.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:13.514+0000 7f1918ab0740 1 -- 172.21.15.5:0/2443698138 >> 172.21.15.5:0/2443698138 conn(0x55d2812363d0 msgr2=0x55d281240700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:55:13.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:13.514+0000 7f1918ab0740 1 -- 172.21.15.5:0/2443698138 shutdown_connections 2024-09-10T09:55:13.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:13.514+0000 7f1918ab0740 1 -- 172.21.15.5:0/2443698138 wait complete. 2024-09-10T09:55:13.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:13.514+0000 7f1918ab0740 20 client.28729 trim_cache size 0 max 16384 2024-09-10T09:55:13.516 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-10T09:55:13.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:13.515+0000 7f1914a00640 1 -- 172.21.15.5:0/999964381 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f18f40a2b80 msgr2=0x7f18f40a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:13.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:13.515+0000 7f1914a00640 1 --2- 172.21.15.5:0/999964381 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f18f40a2b80 0x7f18f40a5040 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f19000cf9b0 tx=0x7f19000977c0 comp rx=0 tx=0).stop 2024-09-10T09:55:13.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:13.515+0000 7f1914a00640 1 -- 172.21.15.5:0/999964381 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f19101019c0 msgr2=0x7f191019a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:13.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:13.515+0000 7f1914a00640 1 --2- 172.21.15.5:0/999964381 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f19101019c0 0x7f191019a480 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f190000e9f0 tx=0x7f190000eeb0 comp rx=0 tx=0).stop 2024-09-10T09:55:13.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:13.515+0000 7f1914a00640 1 -- 172.21.15.5:0/999964381 shutdown_connections 2024-09-10T09:55:13.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:13.515+0000 7f1914a00640 1 --2- 172.21.15.5:0/999964381 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f18f40a2b80 0x7f18f40a5040 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:13.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:13.515+0000 7f1914a00640 1 --2- 172.21.15.5:0/999964381 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f19101079c0 0x7f191019ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:13.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:13.515+0000 7f1914a00640 1 --2- 172.21.15.5:0/999964381 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1910102360 0x7f191019a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:13.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:13.515+0000 7f1914a00640 1 --2- 172.21.15.5:0/999964381 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f19101019c0 0x7f191019a480 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:13.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:13.515+0000 7f1914a00640 1 -- 172.21.15.5:0/999964381 >> 172.21.15.5:0/999964381 conn(0x7f19100fd740 msgr2=0x7f191010d010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:55:13.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:13.515+0000 7f1914a00640 1 -- 172.21.15.5:0/999964381 shutdown_connections 2024-09-10T09:55:13.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:13.515+0000 7f1914a00640 1 -- 172.21.15.5:0/999964381 wait complete. 2024-09-10T09:55:13.979 INFO:tasks.workunit.client.0.smithi005.stdout: 10 61504 23.20 MB/sec execute 52 sec latency 1921.086 ms 2024-09-10T09:55:14.025 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:55:13 smithi005 ceph-mon[30350]: pgmap v899: 129 pgs: 129 active+clean; 591 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 4.0 MiB/s wr, 457 op/s 2024-09-10T09:55:14.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:55:13 smithi159 ceph-mon[35513]: pgmap v899: 129 pgs: 129 active+clean; 591 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 4.0 MiB/s wr, 457 op/s 2024-09-10T09:55:14.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:55:13 smithi102 ceph-mon[35442]: pgmap v899: 129 pgs: 129 active+clean; 591 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 4.0 MiB/s wr, 457 op/s 2024-09-10T09:55:14.690 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-10T09:55:14.979 INFO:tasks.workunit.client.0.smithi005.stdout: 10 61989 23.19 MB/sec execute 53 sec latency 2366.490 ms 2024-09-10T09:55:15.231 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:55:15.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:55:15 smithi005 ceph-mon[30350]: pgmap v900: 129 pgs: 129 active+clean; 593 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 3.8 MiB/s wr, 442 op/s 2024-09-10T09:55:15.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:15.893+0000 7f95d7200640 1 -- 172.21.15.5:0/1316034883 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f95d8103e30 msgr2=0x7f95d8104210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:15.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:15.893+0000 7f95d7200640 1 --2- 172.21.15.5:0/1316034883 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f95d8103e30 0x7f95d8104210 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f95c8009860 tx=0x7f95c802a340 comp rx=0 tx=0).stop 2024-09-10T09:55:15.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:15.893+0000 7f95d7200640 1 -- 172.21.15.5:0/1316034883 shutdown_connections 2024-09-10T09:55:15.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:15.893+0000 7f95d7200640 1 --2- 172.21.15.5:0/1316034883 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f95d8108f60 0x7f95d8110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:15.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:15.893+0000 7f95d7200640 1 --2- 172.21.15.5:0/1316034883 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f95d81047e0 0x7f95d8108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:15.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:15.893+0000 7f95d7200640 1 --2- 172.21.15.5:0/1316034883 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f95d8103e30 0x7f95d8104210 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:15.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:15.893+0000 7f95d7200640 1 -- 172.21.15.5:0/1316034883 >> 172.21.15.5:0/1316034883 conn(0x7f95d80ff8d0 msgr2=0x7f95d8101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:55:15.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:15.893+0000 7f95d7200640 1 -- 172.21.15.5:0/1316034883 shutdown_connections 2024-09-10T09:55:15.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:15.893+0000 7f95d7200640 1 -- 172.21.15.5:0/1316034883 wait complete. 2024-09-10T09:55:15.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:15.894+0000 7f95d7200640 1 Processor -- start 2024-09-10T09:55:15.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:15.894+0000 7f95d7200640 1 -- start start 2024-09-10T09:55:15.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:15.895+0000 7f95d7200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f95d8103e30 0x7f95d819a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:15.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:15.895+0000 7f95d7200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f95d81047e0 0x7f95d819aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:15.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:15.895+0000 7f95d7200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f95d8108f60 0x7f95d819edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:15.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:15.895+0000 7f95d7200640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f95d8113fc0 con 0x7f95d81047e0 2024-09-10T09:55:15.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:15.895+0000 7f95d7200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f95d8113e40 con 0x7f95d8103e30 2024-09-10T09:55:15.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:15.895+0000 7f95d7200640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f95d8114140 con 0x7f95d8108f60 2024-09-10T09:55:15.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:15.895+0000 7f95d5e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f95d8103e30 0x7f95d819a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:15.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:15.895+0000 7f95d6800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f95d8108f60 0x7f95d819edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:15.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:15.895+0000 7f95d5e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f95d8103e30 0x7f95d819a4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:35260/0 (socket says 172.21.15.5:35260) 2024-09-10T09:55:15.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:15.895+0000 7f95d5e00640 1 -- 172.21.15.5:0/2599836801 learned_addr learned my addr 172.21.15.5:0/2599836801 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:55:15.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:15.896+0000 7f95cd400640 1 --2- 172.21.15.5:0/2599836801 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f95d81047e0 0x7f95d819aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:15.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:15.896+0000 7f95d6800640 1 -- 172.21.15.5:0/2599836801 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f95d8103e30 msgr2=0x7f95d819a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:15.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:15.896+0000 7f95d6800640 1 --2- 172.21.15.5:0/2599836801 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f95d8103e30 0x7f95d819a4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:15.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:15.896+0000 7f95d6800640 1 -- 172.21.15.5:0/2599836801 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f95d81047e0 msgr2=0x7f95d819aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:15.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:15.896+0000 7f95d6800640 1 --2- 172.21.15.5:0/2599836801 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f95d81047e0 0x7f95d819aa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:15.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:15.896+0000 7f95d6800640 1 -- 172.21.15.5:0/2599836801 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95d819f490 con 0x7f95d8108f60 2024-09-10T09:55:15.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:15.896+0000 7f95cd400640 1 --2- 172.21.15.5:0/2599836801 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f95d81047e0 0x7f95d819aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:55:15.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:15.897+0000 7f95d6800640 1 --2- 172.21.15.5:0/2599836801 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f95d8108f60 0x7f95d819edb0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f95d0009fd0 tx=0x7f95d000ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:55:15.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:15.897+0000 7f95cf400640 1 -- 172.21.15.5:0/2599836801 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95d000ce20 con 0x7f95d8108f60 2024-09-10T09:55:15.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:15.897+0000 7f95cf400640 1 -- 172.21.15.5:0/2599836801 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f95d0005020 con 0x7f95d8108f60 2024-09-10T09:55:15.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:15.897+0000 7f95cf400640 1 -- 172.21.15.5:0/2599836801 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95d0010720 con 0x7f95d8108f60 2024-09-10T09:55:15.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:15.897+0000 7f95d7200640 1 -- 172.21.15.5:0/2599836801 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95d819f780 con 0x7f95d8108f60 2024-09-10T09:55:15.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:15.897+0000 7f95d7200640 1 -- 172.21.15.5:0/2599836801 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95d81a7060 con 0x7f95d8108f60 2024-09-10T09:55:15.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:15.901+0000 7f95dfa0b740 1 Processor -- start 2024-09-10T09:55:15.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:15.901+0000 7f95cf400640 1 -- 172.21.15.5:0/2599836801 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f95d0005540 con 0x7f95d8108f60 2024-09-10T09:55:15.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:15.901+0000 7f95dfa0b740 1 -- start start 2024-09-10T09:55:15.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:15.902+0000 7f95dfa0b740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563e2577de40 0x563e2577e220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:15.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:15.902+0000 7f95dfa0b740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563e2577e760 0x563e25788ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:15.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:15.902+0000 7f95dfa0b740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563e25789020 0x563e2578b410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:15.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:15.902+0000 7f95dfa0b740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x563e256e43e0 con 0x563e2577e760 2024-09-10T09:55:15.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:15.902+0000 7f95dfa0b740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x563e2568bd80 con 0x563e2577de40 2024-09-10T09:55:15.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:15.902+0000 7f95dfa0b740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x563e256e03a0 con 0x563e25789020 2024-09-10T09:55:15.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:15.902+0000 7f95cd400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563e2577de40 0x563e2577e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:15.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:15.902+0000 7f95cd400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563e2577de40 0x563e2577e220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:35274/0 (socket says 172.21.15.5:35274) 2024-09-10T09:55:15.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:15.902+0000 7f95cd400640 1 -- 172.21.15.5:0/1401571693 learned_addr learned my addr 172.21.15.5:0/1401571693 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:55:15.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:15.902+0000 7f95d5e00640 1 --2- 172.21.15.5:0/1401571693 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563e2577e760 0x563e25788ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:15.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:15.902+0000 7f95cd400640 1 --2- 172.21.15.5:0/1401571693 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563e25789020 0x563e2578b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:15.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:15.903+0000 7f95cd400640 1 -- 172.21.15.5:0/1401571693 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563e25789020 msgr2=0x563e2578b410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:55:15.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:15.903+0000 7f95cd400640 1 --2- 172.21.15.5:0/1401571693 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563e25789020 0x563e2578b410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:15.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:15.903+0000 7f95cd400640 1 -- 172.21.15.5:0/1401571693 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563e2577e760 msgr2=0x563e25788ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:55:15.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:15.903+0000 7f95cf400640 1 --2- 172.21.15.5:0/2599836801 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f95bc0a2cd0 0x7f95bc0a5190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:15.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:15.903+0000 7f95cf400640 1 -- 172.21.15.5:0/2599836801 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f95d00cfa80 con 0x7f95d8108f60 2024-09-10T09:55:15.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:15.903+0000 7f95cd400640 1 --2- 172.21.15.5:0/1401571693 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563e2577e760 0x563e25788ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:15.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:15.903+0000 7f95cd400640 1 -- 172.21.15.5:0/1401571693 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563e256aaae0 con 0x563e2577de40 2024-09-10T09:55:15.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:15.903+0000 7f95d5e00640 1 --2- 172.21.15.5:0/1401571693 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563e2577e760 0x563e25788ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:55:15.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:15.904+0000 7f95cd400640 1 --2- 172.21.15.5:0/2599836801 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f95bc0a2cd0 0x7f95bc0a5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:15.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:15.904+0000 7f95cd400640 1 --2- 172.21.15.5:0/1401571693 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563e2577de40 0x563e2577e220 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f95c00137e0 tx=0x7f95c00096e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1505a4b7f6eff53 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:55:15.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:15.904+0000 7f95c7400640 1 -- 172.21.15.5:0/1401571693 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95c0013250 con 0x563e2577de40 2024-09-10T09:55:15.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:15.905+0000 7f95c7400640 1 -- 172.21.15.5:0/1401571693 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f95c0011b60 con 0x563e2577de40 2024-09-10T09:55:15.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:15.905+0000 7f95c7400640 1 -- 172.21.15.5:0/1401571693 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95c0024a80 con 0x563e2577de40 2024-09-10T09:55:15.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:15.905+0000 7f95dfa0b740 1 -- 172.21.15.5:0/1401571693 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563e2577de40 msgr2=0x563e2577e220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:15.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:15.905+0000 7f95dfa0b740 1 --2- 172.21.15.5:0/1401571693 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563e2577de40 0x563e2577e220 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f95c00137e0 tx=0x7f95c00096e0 comp rx=0 tx=0).stop 2024-09-10T09:55:15.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:15.905+0000 7f95cd400640 1 --2- 172.21.15.5:0/2599836801 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f95bc0a2cd0 0x7f95bc0a5190 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f95c0023000 tx=0x7f95c0020000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:55:15.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:15.906+0000 7f95dfa0b740 1 -- 172.21.15.5:0/1401571693 shutdown_connections 2024-09-10T09:55:15.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:15.906+0000 7f95dfa0b740 1 --2- 172.21.15.5:0/1401571693 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563e25789020 0x563e2578b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:15.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:15.906+0000 7f95dfa0b740 1 --2- 172.21.15.5:0/1401571693 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563e2577e760 0x563e25788ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:15.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:15.906+0000 7f95dfa0b740 1 --2- 172.21.15.5:0/1401571693 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563e2577de40 0x563e2577e220 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:15.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:15.906+0000 7f95dfa0b740 1 -- 172.21.15.5:0/1401571693 >> 172.21.15.5:0/1401571693 conn(0x563e2577a3d0 msgr2=0x563e2577a7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:55:15.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:15.906+0000 7f95dfa0b740 1 -- 172.21.15.5:0/1401571693 shutdown_connections 2024-09-10T09:55:15.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:15.907+0000 7f95dfa0b740 1 -- 172.21.15.5:0/1401571693 wait complete. 2024-09-10T09:55:15.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:15.908+0000 7f95dfa0b740 1 -- start start 2024-09-10T09:55:15.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:15.908+0000 7f95dfa0b740 1 Processor -- start 2024-09-10T09:55:15.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:15.909+0000 7f95dfa0b740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563e2577de40 0x563e25785ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:15.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:15.909+0000 7f95dfa0b740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563e2577e760 0x563e257863e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:15.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:15.909+0000 7f95dfa0b740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563e25789020 0x563e25786920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:15.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:15.909+0000 7f95dfa0b740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x563e2578bbf0 con 0x563e2577e760 2024-09-10T09:55:15.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:15.909+0000 7f95dfa0b740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x563e2569f0f0 con 0x563e25789020 2024-09-10T09:55:15.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:15.909+0000 7f95dfa0b740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x563e2578bd70 con 0x563e2577de40 2024-09-10T09:55:15.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:15.909+0000 7f95dfa0b740 10 client.0 _ll_register_callbacks cb 0x563e25686bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:55:15.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:15.910+0000 7f95d5e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563e2577de40 0x563e25785ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:15.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:15.910+0000 7f95d6800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563e25789020 0x563e25786920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:15.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:15.910+0000 7f95d5e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563e2577de40 0x563e25785ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:42338/0 (socket says 172.21.15.5:42338) 2024-09-10T09:55:15.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:15.910+0000 7f95d6800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563e25789020 0x563e25786920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:35288/0 (socket says 172.21.15.5:35288) 2024-09-10T09:55:15.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:15.910+0000 7f95d5e00640 1 -- 172.21.15.5:0/679475083 learned_addr learned my addr 172.21.15.5:0/679475083 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:55:15.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:15.910+0000 7f95cd400640 1 --2- 172.21.15.5:0/679475083 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563e2577e760 0x563e257863e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:15.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:15.910+0000 7f95cd400640 1 -- 172.21.15.5:0/679475083 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563e2577de40 msgr2=0x563e25785ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:55:15.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:15.910+0000 7f95cd400640 1 --2- 172.21.15.5:0/679475083 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563e2577de40 0x563e25785ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:15.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:15.910+0000 7f95cd400640 1 -- 172.21.15.5:0/679475083 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563e25789020 msgr2=0x563e25786920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:55:15.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:15.910+0000 7f95cd400640 1 --2- 172.21.15.5:0/679475083 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563e25789020 0x563e25786920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:15.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:15.910+0000 7f95cd400640 1 -- 172.21.15.5:0/679475083 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563e254553c0 con 0x563e2577e760 2024-09-10T09:55:15.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:15.910+0000 7f95d5e00640 1 --2- 172.21.15.5:0/679475083 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563e2577de40 0x563e25785ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:55:15.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:15.911+0000 7f95cd400640 1 --2- 172.21.15.5:0/679475083 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563e2577e760 0x563e257863e0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f95c00099b0 tx=0x7f95c0024940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61bce920ebe8b164 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:55:15.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:15.911+0000 7f95bbe00640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-10T09:55:15.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:15.911+0000 7f95bbe00640 1 -- 172.21.15.5:0/679475083 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95c0033060 con 0x563e2577e760 2024-09-10T09:55:15.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:15.912+0000 7f95dfa0b740 1 -- 172.21.15.5:0/679475083 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f95c0011b60 con 0x563e2577e760 2024-09-10T09:55:15.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:15.912+0000 7f95bbe00640 1 -- 172.21.15.5:0/679475083 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f95c0013050 con 0x563e2577e760 2024-09-10T09:55:15.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:15.912+0000 7f95bbe00640 1 -- 172.21.15.5:0/679475083 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f95c003a050 con 0x563e2577e760 2024-09-10T09:55:15.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:15.912+0000 7f95dfa0b740 10 client.18678 fetch_fsmap learned FSMap version 42 2024-09-10T09:55:15.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:15.912+0000 7f95dfa0b740 1 -- 172.21.15.5:0/679475083 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x563e257a03d0 con 0x563e2577e760 2024-09-10T09:55:15.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:15.913+0000 7f95bbe00640 1 -- 172.21.15.5:0/679475083 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f95c0011b60 con 0x563e2577e760 2024-09-10T09:55:15.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:15.913+0000 7f95bbe00640 1 -- 172.21.15.5:0/679475083 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 42) ==== 8370+0+0 (secure 0 0 0) 0x7f95c00035f0 con 0x563e2577e760 2024-09-10T09:55:15.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:15.913+0000 7f95dfa0b740 10 client.18678 fetch_fsmap finished waiting for FSMap version 42 2024-09-10T09:55:15.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:15.913+0000 7f95dfa0b740 10 client.18678 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-09-10T09:55:15.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:15.913+0000 7f95dfa0b740 20 client.18678 populate_metadata read hostname 'smithi005' 2024-09-10T09:55:15.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:15.913+0000 7f95dfa0b740 1 --2- 172.21.15.5:0/679475083 >> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] conn(0x563e257a17e0 0x563e257a3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:15.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:15.913+0000 7f95dfa0b740 4 client.18678 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:55:15.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:15.913+0000 7f95dfa0b740 1 -- 172.21.15.5:0/679475083 --> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563e256f1950 con 0x563e257a17e0 2024-09-10T09:55:15.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:15.913+0000 7f95d5e00640 1 --2- 172.21.15.5:0/679475083 >> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] conn(0x563e257a17e0 0x563e257a3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:15.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:15.914+0000 7f95bbe00640 10 client.18678 ms_handle_connect on v2:172.21.15.102:6836/2689776764 2024-09-10T09:55:15.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:15.914+0000 7f95d5e00640 1 --2- 172.21.15.5:0/679475083 >> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] conn(0x563e257a17e0 0x563e257a3bd0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=f38bd2b0f33f461e server_cookie=f98504c531e843ac in_seq=0 out_seq=0 2024-09-10T09:55:15.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:15.916+0000 7f95bbe00640 1 -- 172.21.15.5:0/679475083 <== mds.3 v2:172.21.15.102:6836/2689776764 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563e256f1950 con 0x563e257a17e0 2024-09-10T09:55:15.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:15.916+0000 7f95bbe00640 10 client.18678 handle_command_reply: tid=0 2024-09-10T09:55:15.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:15.917+0000 7f95dfa0b740 1 client.18678 shutdown 2024-09-10T09:55:15.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:15.918+0000 7f95dfa0b740 1 -- 172.21.15.5:0/679475083 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563e2577e760 msgr2=0x563e257863e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:15.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:15.918+0000 7f95dfa0b740 1 --2- 172.21.15.5:0/679475083 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563e2577e760 0x563e257863e0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f95c00099b0 tx=0x7f95c0024940 comp rx=0 tx=0).stop 2024-09-10T09:55:15.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:15.918+0000 7f95dfa0b740 1 -- 172.21.15.5:0/679475083 shutdown_connections 2024-09-10T09:55:15.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:15.918+0000 7f95dfa0b740 1 --2- 172.21.15.5:0/679475083 >> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] conn(0x563e257a17e0 0x563e257a3bd0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:15.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:15.918+0000 7f95dfa0b740 1 --2- 172.21.15.5:0/679475083 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563e25789020 0x563e25786920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:15.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:15.918+0000 7f95dfa0b740 1 --2- 172.21.15.5:0/679475083 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563e2577e760 0x563e257863e0 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:15.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:15.918+0000 7f95dfa0b740 1 --2- 172.21.15.5:0/679475083 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563e2577de40 0x563e25785ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:15.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:15.918+0000 7f95dfa0b740 1 -- 172.21.15.5:0/679475083 >> 172.21.15.5:0/679475083 conn(0x563e2577a3d0 msgr2=0x563e25780100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:55:15.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:15.918+0000 7f95bbe00640 0 client.18678 ms_handle_reset on v2:172.21.15.102:6836/2689776764 2024-09-10T09:55:15.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:15.918+0000 7f95dfa0b740 1 -- 172.21.15.5:0/679475083 shutdown_connections 2024-09-10T09:55:15.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:15.918+0000 7f95dfa0b740 1 -- 172.21.15.5:0/679475083 wait complete. 2024-09-10T09:55:15.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:15.918+0000 7f95dfa0b740 20 client.18678 trim_cache size 0 max 16384 2024-09-10T09:55:15.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:15.935+0000 7f95dfa0b740 1 Processor -- start 2024-09-10T09:55:15.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:15.935+0000 7f95dfa0b740 1 -- start start 2024-09-10T09:55:15.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:15.935+0000 7f95dfa0b740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563e2577de40 0x563e257808f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:15.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:15.935+0000 7f95dfa0b740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563e2577e760 0x563e257a9430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:15.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:15.935+0000 7f95dfa0b740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563e25789020 0x563e257a9970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:15.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:15.935+0000 7f95dfa0b740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f95d81acae0 con 0x563e2577de40 2024-09-10T09:55:15.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:15.935+0000 7f95dfa0b740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f95d81ae620 con 0x563e25789020 2024-09-10T09:55:15.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:15.935+0000 7f95dfa0b740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f95d81ad1b0 con 0x563e2577e760 2024-09-10T09:55:15.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:15.936+0000 7f95d6800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563e25789020 0x563e257a9970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:15.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:15.936+0000 7f95d5e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563e2577de40 0x563e257808f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:15.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:15.936+0000 7f95d6800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563e25789020 0x563e257a9970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:35292/0 (socket says 172.21.15.5:35292) 2024-09-10T09:55:15.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:15.936+0000 7f95d6800640 1 -- 172.21.15.5:0/3226596982 learned_addr learned my addr 172.21.15.5:0/3226596982 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:55:15.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:15.936+0000 7f95cd400640 1 --2- 172.21.15.5:0/3226596982 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563e2577e760 0x563e257a9430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:15.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:15.936+0000 7f95d6800640 1 -- 172.21.15.5:0/3226596982 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563e2577e760 msgr2=0x563e257a9430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:55:15.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:15.936+0000 7f95d6800640 1 --2- 172.21.15.5:0/3226596982 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563e2577e760 0x563e257a9430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:15.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:15.936+0000 7f95d6800640 1 -- 172.21.15.5:0/3226596982 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563e2577de40 msgr2=0x563e257808f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:55:15.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:15.936+0000 7f95d6800640 1 --2- 172.21.15.5:0/3226596982 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563e2577de40 0x563e257808f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:15.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:15.936+0000 7f95d6800640 1 -- 172.21.15.5:0/3226596982 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563e257843f0 con 0x563e25789020 2024-09-10T09:55:15.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:15.936+0000 7f95d6800640 1 --2- 172.21.15.5:0/3226596982 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563e25789020 0x563e257a9970 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f95d001b000 tx=0x7f95d00cd000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e226e487c6331fc7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:55:15.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:15.937+0000 7f95c6a00640 1 -- 172.21.15.5:0/3226596982 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95d009a190 con 0x563e25789020 2024-09-10T09:55:15.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:15.937+0000 7f95c6a00640 1 -- 172.21.15.5:0/3226596982 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f95d00051d0 con 0x563e25789020 2024-09-10T09:55:15.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:15.937+0000 7f95c6a00640 1 -- 172.21.15.5:0/3226596982 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95d00a48d0 con 0x563e25789020 2024-09-10T09:55:15.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:15.938+0000 7f95dfa0b740 1 -- 172.21.15.5:0/3226596982 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563e25789020 msgr2=0x563e257a9970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:15.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:15.938+0000 7f95dfa0b740 1 --2- 172.21.15.5:0/3226596982 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563e25789020 0x563e257a9970 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f95d001b000 tx=0x7f95d00cd000 comp rx=0 tx=0).stop 2024-09-10T09:55:15.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:15.938+0000 7f95dfa0b740 1 -- 172.21.15.5:0/3226596982 shutdown_connections 2024-09-10T09:55:15.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:15.938+0000 7f95dfa0b740 1 --2- 172.21.15.5:0/3226596982 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563e25789020 0x563e257a9970 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:15.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:15.938+0000 7f95dfa0b740 1 --2- 172.21.15.5:0/3226596982 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563e2577e760 0x563e257a9430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:15.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:15.938+0000 7f95dfa0b740 1 --2- 172.21.15.5:0/3226596982 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563e2577de40 0x563e257808f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:15.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:15.938+0000 7f95dfa0b740 1 -- 172.21.15.5:0/3226596982 >> 172.21.15.5:0/3226596982 conn(0x563e2577a3d0 msgr2=0x563e25787f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:55:15.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:15.938+0000 7f95dfa0b740 1 -- 172.21.15.5:0/3226596982 shutdown_connections 2024-09-10T09:55:15.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:15.938+0000 7f95dfa0b740 1 -- 172.21.15.5:0/3226596982 wait complete. 2024-09-10T09:55:15.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:15.938+0000 7f95dfa0b740 1 -- start start 2024-09-10T09:55:15.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:15.939+0000 7f95dfa0b740 1 Processor -- start 2024-09-10T09:55:15.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:15.939+0000 7f95dfa0b740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563e2577de40 0x563e25790fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:15.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:15.939+0000 7f95dfa0b740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563e2577e760 0x563e25791500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:15.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:15.939+0000 7f95dfa0b740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563e25789020 0x563e25782110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:15.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:15.939+0000 7f95dfa0b740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f95d81adf50 con 0x563e25789020 2024-09-10T09:55:15.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:15.939+0000 7f95dfa0b740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f95d81ad880 con 0x563e2577e760 2024-09-10T09:55:15.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:15.939+0000 7f95dfa0b740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f95d81aaac0 con 0x563e2577de40 2024-09-10T09:55:15.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:15.939+0000 7f95dfa0b740 10 client.0 _ll_register_callbacks cb 0x7f95d00093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:55:15.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:15.939+0000 7f95cd400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563e2577e760 0x563e25791500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:15.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:15.939+0000 7f95cd400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563e2577e760 0x563e25791500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:35296/0 (socket says 172.21.15.5:35296) 2024-09-10T09:55:15.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:15.939+0000 7f95cd400640 1 -- 172.21.15.5:0/1859663464 learned_addr learned my addr 172.21.15.5:0/1859663464 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:55:15.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:15.940+0000 7f95d6800640 1 --2- 172.21.15.5:0/1859663464 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563e25789020 0x563e25782110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:15.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:15.940+0000 7f95d5e00640 1 --2- 172.21.15.5:0/1859663464 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563e2577de40 0x563e25790fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:15.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:15.940+0000 7f95cd400640 1 -- 172.21.15.5:0/1859663464 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563e2577de40 msgr2=0x563e25790fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:55:15.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:15.940+0000 7f95cd400640 1 --2- 172.21.15.5:0/1859663464 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563e2577de40 0x563e25790fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:15.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:15.940+0000 7f95cd400640 1 -- 172.21.15.5:0/1859663464 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563e25789020 msgr2=0x563e25782110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:55:15.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:15.940+0000 7f95cd400640 1 --2- 172.21.15.5:0/1859663464 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563e25789020 0x563e25782110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:15.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:15.940+0000 7f95cd400640 1 -- 172.21.15.5:0/1859663464 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563e25791bf0 con 0x563e2577e760 2024-09-10T09:55:15.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:15.940+0000 7f95d6800640 1 --2- 172.21.15.5:0/1859663464 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563e25789020 0x563e25782110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:55:15.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:15.940+0000 7f95cd400640 1 --2- 172.21.15.5:0/1859663464 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563e2577e760 0x563e25791500 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f95c0001eb0 tx=0x7f95c00136e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1a56804060a270d0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:55:15.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:15.940+0000 7f95bbe00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-09-10T09:55:15.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:15.941+0000 7f95bbe00640 1 -- 172.21.15.5:0/1859663464 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95c0043060 con 0x563e2577e760 2024-09-10T09:55:15.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:15.941+0000 7f95bbe00640 1 -- 172.21.15.5:0/1859663464 <== mon.2 v2:172.21.15.102:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f95c00035f0 con 0x563e2577e760 2024-09-10T09:55:15.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:15.941+0000 7f95dfa0b740 1 -- 172.21.15.5:0/1859663464 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f95d00051d0 con 0x563e2577e760 2024-09-10T09:55:15.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:15.941+0000 7f95bbe00640 1 -- 172.21.15.5:0/1859663464 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f95c0033040 con 0x563e2577e760 2024-09-10T09:55:15.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:15.941+0000 7f95dfa0b740 10 client.28748 fetch_fsmap learned FSMap version 42 2024-09-10T09:55:15.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:15.941+0000 7f95dfa0b740 1 -- 172.21.15.5:0/1859663464 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x563e2580b980 con 0x563e2577e760 2024-09-10T09:55:15.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:15.941+0000 7f95bbe00640 1 -- 172.21.15.5:0/1859663464 <== mon.2 v2:172.21.15.102:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f95d00051d0 con 0x563e2577e760 2024-09-10T09:55:15.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:15.942+0000 7f95bbe00640 1 -- 172.21.15.5:0/1859663464 <== mon.2 v2:172.21.15.102:3300/0 5 ==== fsmap(e 42) ==== 8370+0+0 (secure 0 0 0) 0x7f95c0003be0 con 0x563e2577e760 2024-09-10T09:55:15.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:15.942+0000 7f95dfa0b740 10 client.28748 fetch_fsmap finished waiting for FSMap version 42 2024-09-10T09:55:15.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:15.942+0000 7f95dfa0b740 10 client.28748 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-09-10T09:55:15.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:15.942+0000 7f95dfa0b740 20 client.28748 populate_metadata read hostname 'smithi005' 2024-09-10T09:55:15.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:15.942+0000 7f95dfa0b740 1 --2- 172.21.15.5:0/1859663464 >> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] conn(0x563e2580bfc0 0x563e2580e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:15.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:15.942+0000 7f95dfa0b740 4 client.28748 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T09:55:15.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:15.942+0000 7f95dfa0b740 1 -- 172.21.15.5:0/1859663464 --> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563e256ebe60 con 0x563e2580bfc0 2024-09-10T09:55:15.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:15.942+0000 7f95d5e00640 1 --2- 172.21.15.5:0/1859663464 >> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] conn(0x563e2580bfc0 0x563e2580e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:15.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:15.942+0000 7f95bbe00640 10 client.28748 ms_handle_connect on v2:172.21.15.102:6836/2689776764 2024-09-10T09:55:15.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:15.942+0000 7f95d5e00640 1 --2- 172.21.15.5:0/1859663464 >> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] conn(0x563e2580bfc0 0x563e2580e3e0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=f246a879b0a6aa45 server_cookie=aaa044c5d587ef72 in_seq=0 out_seq=0 2024-09-10T09:55:15.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:15.943+0000 7f95bbe00640 1 -- 172.21.15.5:0/1859663464 <== mds.3 v2:172.21.15.102:6836/2689776764 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563e256ebe60 con 0x563e2580bfc0 2024-09-10T09:55:15.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:15.943+0000 7f95bbe00640 10 client.28748 handle_command_reply: tid=0 2024-09-10T09:55:15.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:15.943+0000 7f95dfa0b740 1 client.28748 shutdown 2024-09-10T09:55:15.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:15.943+0000 7f95dfa0b740 1 -- 172.21.15.5:0/1859663464 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563e2577e760 msgr2=0x563e25791500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:15.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:15.943+0000 7f95dfa0b740 1 --2- 172.21.15.5:0/1859663464 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563e2577e760 0x563e25791500 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f95c0001eb0 tx=0x7f95c00136e0 comp rx=0 tx=0).stop 2024-09-10T09:55:15.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:15.943+0000 7f95dfa0b740 1 -- 172.21.15.5:0/1859663464 shutdown_connections 2024-09-10T09:55:15.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:15.943+0000 7f95dfa0b740 1 --2- 172.21.15.5:0/1859663464 >> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] conn(0x563e2580bfc0 0x563e2580e3e0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:15.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:15.943+0000 7f95dfa0b740 1 --2- 172.21.15.5:0/1859663464 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563e25789020 0x563e25782110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:15.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:15.943+0000 7f95dfa0b740 1 --2- 172.21.15.5:0/1859663464 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563e2577e760 0x563e25791500 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:15.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:15.943+0000 7f95dfa0b740 1 --2- 172.21.15.5:0/1859663464 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563e2577de40 0x563e25790fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:15.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:15.943+0000 7f95bbe00640 0 client.28748 ms_handle_reset on v2:172.21.15.102:6836/2689776764 2024-09-10T09:55:15.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:15.943+0000 7f95dfa0b740 1 -- 172.21.15.5:0/1859663464 >> 172.21.15.5:0/1859663464 conn(0x563e2577a3d0 msgr2=0x563e25784640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:55:15.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:15.944+0000 7f95dfa0b740 1 -- 172.21.15.5:0/1859663464 shutdown_connections 2024-09-10T09:55:15.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:15.945+0000 7f95dfa0b740 1 -- 172.21.15.5:0/1859663464 wait complete. 2024-09-10T09:55:15.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:15.945+0000 7f95dfa0b740 20 client.28748 trim_cache size 0 max 16384 2024-09-10T09:55:15.949 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-10T09:55:15.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:15.946+0000 7f95d7200640 1 -- 172.21.15.5:0/2599836801 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f95bc0a2cd0 msgr2=0x7f95bc0a5190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:15.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:15.946+0000 7f95d7200640 1 --2- 172.21.15.5:0/2599836801 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f95bc0a2cd0 0x7f95bc0a5190 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f95c0023000 tx=0x7f95c0020000 comp rx=0 tx=0).stop 2024-09-10T09:55:15.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:15.947+0000 7f95d7200640 1 -- 172.21.15.5:0/2599836801 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f95d8108f60 msgr2=0x7f95d819edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:15.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:15.947+0000 7f95d7200640 1 --2- 172.21.15.5:0/2599836801 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f95d8108f60 0x7f95d819edb0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f95d0009fd0 tx=0x7f95d000ef10 comp rx=0 tx=0).stop 2024-09-10T09:55:15.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:15.947+0000 7f95d7200640 1 -- 172.21.15.5:0/2599836801 shutdown_connections 2024-09-10T09:55:15.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:15.947+0000 7f95d7200640 1 --2- 172.21.15.5:0/2599836801 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f95bc0a2cd0 0x7f95bc0a5190 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:15.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:15.947+0000 7f95d7200640 1 --2- 172.21.15.5:0/2599836801 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f95d8108f60 0x7f95d819edb0 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:15.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:15.947+0000 7f95d7200640 1 --2- 172.21.15.5:0/2599836801 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f95d81047e0 0x7f95d819aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:15.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:15.947+0000 7f95d7200640 1 --2- 172.21.15.5:0/2599836801 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f95d8103e30 0x7f95d819a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:15.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:15.947+0000 7f95d7200640 1 -- 172.21.15.5:0/2599836801 >> 172.21.15.5:0/2599836801 conn(0x7f95d80ff8d0 msgr2=0x7f95d8100c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:55:15.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:15.947+0000 7f95d7200640 1 -- 172.21.15.5:0/2599836801 shutdown_connections 2024-09-10T09:55:15.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:15.948+0000 7f95d7200640 1 -- 172.21.15.5:0/2599836801 wait complete. 2024-09-10T09:55:15.979 INFO:tasks.workunit.client.0.smithi005.stdout: 10 62852 23.68 MB/sec execute 54 sec latency 46.333 ms 2024-09-10T09:55:16.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:55:15 smithi159 ceph-mon[35513]: pgmap v900: 129 pgs: 129 active+clean; 593 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 3.8 MiB/s wr, 442 op/s 2024-09-10T09:55:16.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:55:15 smithi102 ceph-mon[35442]: pgmap v900: 129 pgs: 129 active+clean; 593 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 3.8 MiB/s wr, 442 op/s 2024-09-10T09:55:16.980 INFO:tasks.workunit.client.0.smithi005.stdout: 10 63552 24.19 MB/sec execute 55 sec latency 69.599 ms 2024-09-10T09:55:17.645 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-10T09:55:17.841 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:55:17 smithi005 ceph-mon[30350]: pgmap v901: 129 pgs: 129 active+clean; 597 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 4.5 MiB/s wr, 477 op/s 2024-09-10T09:55:17.980 INFO:tasks.workunit.client.0.smithi005.stdout: 10 63679 23.92 MB/sec execute 56 sec latency 863.107 ms 2024-09-10T09:55:18.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:55:17 smithi159 ceph-mon[35513]: pgmap v901: 129 pgs: 129 active+clean; 597 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 4.5 MiB/s wr, 477 op/s 2024-09-10T09:55:18.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:55:17 smithi102 ceph-mon[35442]: pgmap v901: 129 pgs: 129 active+clean; 597 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 4.5 MiB/s wr, 477 op/s 2024-09-10T09:55:18.186 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:55:18.980 INFO:tasks.workunit.client.0.smithi005.stdout: 10 63709 23.50 MB/sec execute 57 sec latency 1863.157 ms 2024-09-10T09:55:19.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:19.178+0000 7f24c9400640 1 -- 172.21.15.5:0/175524009 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f24c41019c0 msgr2=0x7f24c4101e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:19.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:19.178+0000 7f24c9400640 1 --2- 172.21.15.5:0/175524009 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f24c41019c0 0x7f24c4101e20 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f24b0009860 tx=0x7f24b002a340 comp rx=0 tx=0).stop 2024-09-10T09:55:19.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:19.178+0000 7f24c9400640 1 -- 172.21.15.5:0/175524009 shutdown_connections 2024-09-10T09:55:19.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:19.178+0000 7f24c9400640 1 --2- 172.21.15.5:0/175524009 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f24c4102360 0x7f24c410e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:19.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:19.178+0000 7f24c9400640 1 --2- 172.21.15.5:0/175524009 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f24c41019c0 0x7f24c4101e20 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:19.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:19.178+0000 7f24c9400640 1 --2- 172.21.15.5:0/175524009 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f24c41079c0 0x7f24c4107da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:19.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:19.178+0000 7f24c9400640 1 -- 172.21.15.5:0/175524009 >> 172.21.15.5:0/175524009 conn(0x7f24c40fd740 msgr2=0x7f24c40ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:55:19.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:19.179+0000 7f24c9400640 1 -- 172.21.15.5:0/175524009 shutdown_connections 2024-09-10T09:55:19.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:19.179+0000 7f24c9400640 1 -- 172.21.15.5:0/175524009 wait complete. 2024-09-10T09:55:19.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:19.180+0000 7f24c9400640 1 Processor -- start 2024-09-10T09:55:19.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:19.180+0000 7f24c9400640 1 -- start start 2024-09-10T09:55:19.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:19.180+0000 7f24c9400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f24c41019c0 0x7f24c419a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:19.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:19.180+0000 7f24c9400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f24c4102360 0x7f24c419a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:19.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:19.180+0000 7f24c9400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f24c41079c0 0x7f24c419ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:19.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:19.180+0000 7f24c9400640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f24c4111d90 con 0x7f24c41079c0 2024-09-10T09:55:19.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:19.180+0000 7f24c9400640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f24c4111c10 con 0x7f24c41019c0 2024-09-10T09:55:19.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:19.180+0000 7f24c9400640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f24c4111f10 con 0x7f24c4102360 2024-09-10T09:55:19.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:19.181+0000 7f24c3e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f24c41019c0 0x7f24c419a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:19.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:19.181+0000 7f24c3e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f24c41019c0 0x7f24c419a4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:35310/0 (socket says 172.21.15.5:35310) 2024-09-10T09:55:19.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:19.181+0000 7f24c3e00640 1 -- 172.21.15.5:0/3999040334 learned_addr learned my addr 172.21.15.5:0/3999040334 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:55:19.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:19.181+0000 7f24c8a00640 1 --2- 172.21.15.5:0/3999040334 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f24c41079c0 0x7f24c419ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:19.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:19.181+0000 7f24c3400640 1 --2- 172.21.15.5:0/3999040334 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f24c4102360 0x7f24c419a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:19.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:19.181+0000 7f24c3e00640 1 -- 172.21.15.5:0/3999040334 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f24c4102360 msgr2=0x7f24c419a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:19.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:19.181+0000 7f24c3e00640 1 --2- 172.21.15.5:0/3999040334 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f24c4102360 0x7f24c419a9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:19.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:19.181+0000 7f24c3e00640 1 -- 172.21.15.5:0/3999040334 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f24c41079c0 msgr2=0x7f24c419ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:19.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:19.181+0000 7f24c3e00640 1 --2- 172.21.15.5:0/3999040334 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f24c41079c0 0x7f24c419ed70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:19.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:19.181+0000 7f24c3e00640 1 -- 172.21.15.5:0/3999040334 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24c419f450 con 0x7f24c41019c0 2024-09-10T09:55:19.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:19.181+0000 7f24c8a00640 1 --2- 172.21.15.5:0/3999040334 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f24c41079c0 0x7f24c419ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:55:19.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:19.181+0000 7f24c3e00640 1 --2- 172.21.15.5:0/3999040334 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f24c41019c0 0x7f24c419a4a0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f24b400e9c0 tx=0x7f24b400ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:55:19.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:19.182+0000 7f24c0c00640 1 -- 172.21.15.5:0/3999040334 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24b400cde0 con 0x7f24c41019c0 2024-09-10T09:55:19.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:19.182+0000 7f24c0c00640 1 -- 172.21.15.5:0/3999040334 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f24b40035f0 con 0x7f24c41019c0 2024-09-10T09:55:19.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:19.182+0000 7f24c0c00640 1 -- 172.21.15.5:0/3999040334 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24b4010690 con 0x7f24c41019c0 2024-09-10T09:55:19.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:19.182+0000 7f24c9400640 1 -- 172.21.15.5:0/3999040334 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24c419f740 con 0x7f24c41019c0 2024-09-10T09:55:19.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:19.182+0000 7f24c9400640 1 -- 172.21.15.5:0/3999040334 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24c41a7020 con 0x7f24c41019c0 2024-09-10T09:55:19.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:19.185+0000 7f24c0c00640 1 -- 172.21.15.5:0/3999040334 <== mon.2 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f24b40037a0 con 0x7f24c41019c0 2024-09-10T09:55:19.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:19.185+0000 7f24cd3bf740 1 Processor -- start 2024-09-10T09:55:19.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:19.185+0000 7f24cd3bf740 1 -- start start 2024-09-10T09:55:19.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:19.185+0000 7f24cd3bf740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563e5ea51e40 0x563e5ea52220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:19.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:19.185+0000 7f24cd3bf740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563e5ea52760 0x563e5ea5cae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:19.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:19.186+0000 7f24cd3bf740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563e5ea5d020 0x563e5ea5f410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:19.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:19.186+0000 7f24cd3bf740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x563e5e9b83e0 con 0x563e5ea51e40 2024-09-10T09:55:19.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:19.186+0000 7f24cd3bf740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x563e5e95fd80 con 0x563e5ea5d020 2024-09-10T09:55:19.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:19.186+0000 7f24cd3bf740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x563e5e9b43a0 con 0x563e5ea52760 2024-09-10T09:55:19.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:19.186+0000 7f24c3e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563e5ea52760 0x563e5ea5cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:19.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:19.186+0000 7f24c8a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563e5ea51e40 0x563e5ea52220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:19.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:19.186+0000 7f24c3e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563e5ea52760 0x563e5ea5cae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:42382/0 (socket says 172.21.15.5:42382) 2024-09-10T09:55:19.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:19.186+0000 7f24c3e00640 1 -- 172.21.15.5:0/2779873854 learned_addr learned my addr 172.21.15.5:0/2779873854 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:55:19.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:19.186+0000 7f24c3400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563e5ea5d020 0x563e5ea5f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:19.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:19.187+0000 7f24c3e00640 1 -- 172.21.15.5:0/2779873854 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563e5ea5d020 msgr2=0x563e5ea5f410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:55:19.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:19.187+0000 7f24c3e00640 1 --2- 172.21.15.5:0/2779873854 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563e5ea5d020 0x563e5ea5f410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:19.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:19.187+0000 7f24c3e00640 1 -- 172.21.15.5:0/2779873854 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563e5ea51e40 msgr2=0x563e5ea52220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:55:19.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:19.187+0000 7f24c3e00640 1 --2- 172.21.15.5:0/2779873854 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563e5ea51e40 0x563e5ea52220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:19.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:19.187+0000 7f24c3e00640 1 -- 172.21.15.5:0/2779873854 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563e5e97eae0 con 0x563e5ea52760 2024-09-10T09:55:19.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:19.187+0000 7f24c0c00640 1 --2- 172.21.15.5:0/3999040334 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f24a80a2c00 0x7f24a80a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:19.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:19.187+0000 7f24c3e00640 1 --2- 172.21.15.5:0/2779873854 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563e5ea52760 0x563e5ea5cae0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f24b40d5a90 tx=0x7f24b40d5f50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b57f28568868b3dd server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:55:19.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:19.187+0000 7f24c0c00640 1 -- 172.21.15.5:0/3999040334 <== mon.2 v2:172.21.15.102:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f24b40cf7e0 con 0x7f24c41019c0 2024-09-10T09:55:19.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:19.187+0000 7f24c8a00640 1 --2- 172.21.15.5:0/3999040334 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f24a80a2c00 0x7f24a80a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:19.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:19.187+0000 7f24b9200640 1 -- 172.21.15.5:0/2779873854 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24b4097e90 con 0x563e5ea52760 2024-09-10T09:55:19.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:19.187+0000 7f24b9200640 1 -- 172.21.15.5:0/2779873854 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f24b4086020 con 0x563e5ea52760 2024-09-10T09:55:19.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:19.187+0000 7f24b9200640 1 -- 172.21.15.5:0/2779873854 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24b409b550 con 0x563e5ea52760 2024-09-10T09:55:19.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:19.188+0000 7f24cd3bf740 1 -- 172.21.15.5:0/2779873854 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563e5ea52760 msgr2=0x563e5ea5cae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:19.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:19.188+0000 7f24c8a00640 1 --2- 172.21.15.5:0/3999040334 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f24a80a2c00 0x7f24a80a50c0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f24bc005fd0 tx=0x7f24bc009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:55:19.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:19.188+0000 7f24cd3bf740 1 --2- 172.21.15.5:0/2779873854 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563e5ea52760 0x563e5ea5cae0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f24b40d5a90 tx=0x7f24b40d5f50 comp rx=0 tx=0).stop 2024-09-10T09:55:19.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:19.188+0000 7f24cd3bf740 1 -- 172.21.15.5:0/2779873854 shutdown_connections 2024-09-10T09:55:19.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:19.188+0000 7f24cd3bf740 1 --2- 172.21.15.5:0/2779873854 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563e5ea5d020 0x563e5ea5f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:19.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:19.188+0000 7f24cd3bf740 1 --2- 172.21.15.5:0/2779873854 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563e5ea52760 0x563e5ea5cae0 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:19.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:19.188+0000 7f24cd3bf740 1 --2- 172.21.15.5:0/2779873854 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563e5ea51e40 0x563e5ea52220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:19.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:19.188+0000 7f24cd3bf740 1 -- 172.21.15.5:0/2779873854 >> 172.21.15.5:0/2779873854 conn(0x563e5ea4e3d0 msgr2=0x563e5ea4e7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:55:19.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:19.188+0000 7f24cd3bf740 1 -- 172.21.15.5:0/2779873854 shutdown_connections 2024-09-10T09:55:19.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:19.188+0000 7f24cd3bf740 1 -- 172.21.15.5:0/2779873854 wait complete. 2024-09-10T09:55:19.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:19.189+0000 7f24cd3bf740 1 -- start start 2024-09-10T09:55:19.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:19.189+0000 7f24cd3bf740 1 Processor -- start 2024-09-10T09:55:19.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:19.189+0000 7f24cd3bf740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563e5ea51e40 0x563e5ea59f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:19.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:19.189+0000 7f24cd3bf740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563e5ea52760 0x563e5ea5a440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:19.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:19.189+0000 7f24cd3bf740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563e5ea5d020 0x563e5ea5a980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:19.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:19.190+0000 7f24cd3bf740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x563e5ea5fbf0 con 0x563e5ea52760 2024-09-10T09:55:19.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:19.190+0000 7f24cd3bf740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x563e5e9730f0 con 0x563e5ea51e40 2024-09-10T09:55:19.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:19.190+0000 7f24cd3bf740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x563e5ea5fd70 con 0x563e5ea5d020 2024-09-10T09:55:19.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:19.190+0000 7f24cd3bf740 10 client.0 _ll_register_callbacks cb 0x563e5e95abd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:55:19.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:19.190+0000 7f24c3e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563e5ea51e40 0x563e5ea59f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:19.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:19.190+0000 7f24c3e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563e5ea51e40 0x563e5ea59f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:35330/0 (socket says 172.21.15.5:35330) 2024-09-10T09:55:19.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:19.190+0000 7f24c3400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563e5ea52760 0x563e5ea5a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:19.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:19.190+0000 7f24c3e00640 1 -- 172.21.15.5:0/3440286239 learned_addr learned my addr 172.21.15.5:0/3440286239 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:55:19.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:19.190+0000 7f24c8a00640 1 --2- 172.21.15.5:0/3440286239 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563e5ea5d020 0x563e5ea5a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:19.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:19.190+0000 7f24c3e00640 1 -- 172.21.15.5:0/3440286239 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563e5ea5d020 msgr2=0x563e5ea5a980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:55:19.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:19.190+0000 7f24c3e00640 1 --2- 172.21.15.5:0/3440286239 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563e5ea5d020 0x563e5ea5a980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:19.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:19.190+0000 7f24c3e00640 1 -- 172.21.15.5:0/3440286239 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563e5ea52760 msgr2=0x563e5ea5a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:55:19.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:19.190+0000 7f24c3e00640 1 --2- 172.21.15.5:0/3440286239 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563e5ea52760 0x563e5ea5a440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:19.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:19.190+0000 7f24c3e00640 1 -- 172.21.15.5:0/3440286239 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563e5e7293c0 con 0x563e5ea51e40 2024-09-10T09:55:19.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:19.190+0000 7f24c8a00640 1 --2- 172.21.15.5:0/3440286239 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563e5ea5d020 0x563e5ea5a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T09:55:19.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:19.191+0000 7f24c3e00640 1 --2- 172.21.15.5:0/3440286239 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563e5ea51e40 0x563e5ea59f00 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f24b40d5a60 tx=0x7f24b4004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=87bfeb0d72c43131 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:55:19.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:19.191+0000 7f24ad600640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-09-10T09:55:19.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:19.191+0000 7f24ad600640 1 -- 172.21.15.5:0/3440286239 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24b409b730 con 0x563e5ea51e40 2024-09-10T09:55:19.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:19.191+0000 7f24cd3bf740 1 -- 172.21.15.5:0/3440286239 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f24b4086020 con 0x563e5ea51e40 2024-09-10T09:55:19.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:19.191+0000 7f24ad600640 1 -- 172.21.15.5:0/3440286239 <== mon.2 v2:172.21.15.102:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f24b4099ef0 con 0x563e5ea51e40 2024-09-10T09:55:19.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:19.191+0000 7f24ad600640 1 -- 172.21.15.5:0/3440286239 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f24b409b9c0 con 0x563e5ea51e40 2024-09-10T09:55:19.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:19.191+0000 7f24cd3bf740 10 client.28760 fetch_fsmap learned FSMap version 42 2024-09-10T09:55:19.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:19.191+0000 7f24cd3bf740 1 -- 172.21.15.5:0/3440286239 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x563e5ea6bc30 con 0x563e5ea51e40 2024-09-10T09:55:19.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:19.191+0000 7f24ad600640 1 -- 172.21.15.5:0/3440286239 <== mon.2 v2:172.21.15.102:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f24b4086020 con 0x563e5ea51e40 2024-09-10T09:55:19.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:19.191+0000 7f24ad600640 1 -- 172.21.15.5:0/3440286239 <== mon.2 v2:172.21.15.102:3300/0 5 ==== fsmap(e 42) ==== 8370+0+0 (secure 0 0 0) 0x7f24b40cfbd0 con 0x563e5ea51e40 2024-09-10T09:55:19.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:19.191+0000 7f24cd3bf740 10 client.28760 fetch_fsmap finished waiting for FSMap version 42 2024-09-10T09:55:19.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:19.191+0000 7f24cd3bf740 10 client.28760 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-10T09:55:19.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:19.192+0000 7f24cd3bf740 20 client.28760 populate_metadata read hostname 'smithi005' 2024-09-10T09:55:19.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:19.192+0000 7f24cd3bf740 1 --2- 172.21.15.5:0/3440286239 >> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] conn(0x563e5ea6cf80 0x563e5ea6f370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:19.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:19.192+0000 7f24cd3bf740 4 client.28760 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:55:19.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:19.192+0000 7f24cd3bf740 1 -- 172.21.15.5:0/3440286239 --> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563e5e9c5950 con 0x563e5ea6cf80 2024-09-10T09:55:19.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:19.192+0000 7f24c3400640 1 --2- 172.21.15.5:0/3440286239 >> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] conn(0x563e5ea6cf80 0x563e5ea6f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:19.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:19.192+0000 7f24c3400640 1 --2- 172.21.15.5:0/3440286239 >> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] conn(0x563e5ea6cf80 0x563e5ea6f370 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f0e58120936ebe21 server_cookie=e942286dc7d3798d in_seq=0 out_seq=0 2024-09-10T09:55:19.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:19.192+0000 7f24ad600640 10 client.28760 ms_handle_connect on v2:172.21.15.159:6836/620627639 2024-09-10T09:55:19.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:19.193+0000 7f24ad600640 1 -- 172.21.15.5:0/3440286239 <== mds.2 v2:172.21.15.159:6836/620627639 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563e5e9c5950 con 0x563e5ea6cf80 2024-09-10T09:55:19.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:19.193+0000 7f24ad600640 10 client.28760 handle_command_reply: tid=0 2024-09-10T09:55:19.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:19.193+0000 7f24cd3bf740 1 client.28760 shutdown 2024-09-10T09:55:19.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:19.193+0000 7f24cd3bf740 1 -- 172.21.15.5:0/3440286239 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563e5ea51e40 msgr2=0x563e5ea59f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:19.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:19.193+0000 7f24cd3bf740 1 --2- 172.21.15.5:0/3440286239 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563e5ea51e40 0x563e5ea59f00 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f24b40d5a60 tx=0x7f24b4004730 comp rx=0 tx=0).stop 2024-09-10T09:55:19.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:19.193+0000 7f24cd3bf740 1 -- 172.21.15.5:0/3440286239 shutdown_connections 2024-09-10T09:55:19.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:19.193+0000 7f24cd3bf740 1 --2- 172.21.15.5:0/3440286239 >> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] conn(0x563e5ea6cf80 0x563e5ea6f370 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:19.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:19.193+0000 7f24cd3bf740 1 --2- 172.21.15.5:0/3440286239 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563e5ea5d020 0x563e5ea5a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:19.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:19.193+0000 7f24cd3bf740 1 --2- 172.21.15.5:0/3440286239 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563e5ea52760 0x563e5ea5a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:19.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:19.193+0000 7f24cd3bf740 1 --2- 172.21.15.5:0/3440286239 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563e5ea51e40 0x563e5ea59f00 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:19.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:19.193+0000 7f24ad600640 0 client.28760 ms_handle_reset on v2:172.21.15.159:6836/620627639 2024-09-10T09:55:19.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:19.193+0000 7f24cd3bf740 1 -- 172.21.15.5:0/3440286239 >> 172.21.15.5:0/3440286239 conn(0x563e5ea4e3d0 msgr2=0x563e5ea54100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:55:19.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:19.194+0000 7f24cd3bf740 1 -- 172.21.15.5:0/3440286239 shutdown_connections 2024-09-10T09:55:19.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:19.194+0000 7f24cd3bf740 1 -- 172.21.15.5:0/3440286239 wait complete. 2024-09-10T09:55:19.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:19.194+0000 7f24cd3bf740 20 client.28760 trim_cache size 0 max 16384 2024-09-10T09:55:19.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:19.219+0000 7f24cd3bf740 1 Processor -- start 2024-09-10T09:55:19.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:19.219+0000 7f24cd3bf740 1 -- start start 2024-09-10T09:55:19.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:19.220+0000 7f24cd3bf740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563e5ea51e40 0x563e5ea5b760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:19.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:19.220+0000 7f24cd3bf740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563e5ea52760 0x563e5ea5a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:19.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:19.220+0000 7f24cd3bf740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563e5ea5d020 0x563e5ea72390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:19.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:19.220+0000 7f24cd3bf740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f24c41b0f80 con 0x563e5ea51e40 2024-09-10T09:55:19.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:19.220+0000 7f24cd3bf740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f24c41b2ac0 con 0x563e5ea5d020 2024-09-10T09:55:19.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:19.220+0000 7f24cd3bf740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f24c41b1650 con 0x563e5ea52760 2024-09-10T09:55:19.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:19.220+0000 7f24c3400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563e5ea52760 0x563e5ea5a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:19.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:19.220+0000 7f24c3e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563e5ea51e40 0x563e5ea5b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:19.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:19.220+0000 7f24c3e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563e5ea51e40 0x563e5ea5b760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53358/0 (socket says 172.21.15.5:53358) 2024-09-10T09:55:19.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:19.220+0000 7f24c8a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563e5ea5d020 0x563e5ea72390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:19.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:19.220+0000 7f24c3e00640 1 -- 172.21.15.5:0/3712256747 learned_addr learned my addr 172.21.15.5:0/3712256747 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:55:19.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:19.220+0000 7f24c3400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563e5ea52760 0x563e5ea5a7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:42396/0 (socket says 172.21.15.5:42396) 2024-09-10T09:55:19.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:19.220+0000 7f24c3400640 1 -- 172.21.15.5:0/3712256747 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563e5ea5d020 msgr2=0x563e5ea72390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:55:19.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:19.220+0000 7f24c3400640 1 --2- 172.21.15.5:0/3712256747 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563e5ea5d020 0x563e5ea72390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:19.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:19.221+0000 7f24c3400640 1 -- 172.21.15.5:0/3712256747 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563e5ea51e40 msgr2=0x563e5ea5b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:55:19.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:19.221+0000 7f24c3400640 1 --2- 172.21.15.5:0/3712256747 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563e5ea51e40 0x563e5ea5b760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:19.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:19.221+0000 7f24c3400640 1 -- 172.21.15.5:0/3712256747 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563e5ea4e200 con 0x563e5ea52760 2024-09-10T09:55:19.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:19.221+0000 7f24c3400640 1 --2- 172.21.15.5:0/3712256747 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563e5ea52760 0x563e5ea5a7d0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f24b0035040 tx=0x7f24b0030460 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3c7b9e1cbda6536 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:55:19.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:19.221+0000 7f24afe00640 1 -- 172.21.15.5:0/3712256747 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24b0037040 con 0x563e5ea52760 2024-09-10T09:55:19.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:19.221+0000 7f24afe00640 1 -- 172.21.15.5:0/3712256747 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f24b0030780 con 0x563e5ea52760 2024-09-10T09:55:19.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:19.221+0000 7f24afe00640 1 -- 172.21.15.5:0/3712256747 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24b00046c0 con 0x563e5ea52760 2024-09-10T09:55:19.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:19.222+0000 7f24cd3bf740 1 -- 172.21.15.5:0/3712256747 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563e5ea52760 msgr2=0x563e5ea5a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:19.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:19.222+0000 7f24cd3bf740 1 --2- 172.21.15.5:0/3712256747 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563e5ea52760 0x563e5ea5a7d0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f24b0035040 tx=0x7f24b0030460 comp rx=0 tx=0).stop 2024-09-10T09:55:19.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:19.222+0000 7f24cd3bf740 1 -- 172.21.15.5:0/3712256747 shutdown_connections 2024-09-10T09:55:19.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:19.222+0000 7f24cd3bf740 1 --2- 172.21.15.5:0/3712256747 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563e5ea5d020 0x563e5ea72390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:19.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:19.222+0000 7f24cd3bf740 1 --2- 172.21.15.5:0/3712256747 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563e5ea52760 0x563e5ea5a7d0 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:19.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:19.222+0000 7f24cd3bf740 1 --2- 172.21.15.5:0/3712256747 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563e5ea51e40 0x563e5ea5b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:19.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:19.222+0000 7f24cd3bf740 1 -- 172.21.15.5:0/3712256747 >> 172.21.15.5:0/3712256747 conn(0x563e5ea4e3d0 msgr2=0x563e5ea687e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:55:19.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:19.222+0000 7f24cd3bf740 1 -- 172.21.15.5:0/3712256747 shutdown_connections 2024-09-10T09:55:19.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:19.222+0000 7f24cd3bf740 1 -- 172.21.15.5:0/3712256747 wait complete. 2024-09-10T09:55:19.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:19.223+0000 7f24cd3bf740 1 -- start start 2024-09-10T09:55:19.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:19.223+0000 7f24cd3bf740 1 Processor -- start 2024-09-10T09:55:19.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:19.223+0000 7f24cd3bf740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563e5ea51e40 0x563e5ea72a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:19.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:19.223+0000 7f24cd3bf740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563e5ea52760 0x563e5ea6fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:19.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:19.223+0000 7f24cd3bf740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563e5ea5d020 0x563e5ea702e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:19.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:19.223+0000 7f24cd3bf740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f24c41b23f0 con 0x563e5ea51e40 2024-09-10T09:55:19.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:19.223+0000 7f24cd3bf740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f24c41b1d20 con 0x563e5ea52760 2024-09-10T09:55:19.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:19.223+0000 7f24cd3bf740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f24c41aef60 con 0x563e5ea5d020 2024-09-10T09:55:19.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:19.223+0000 7f24cd3bf740 10 client.0 _ll_register_callbacks cb 0x7f24bc0099e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:55:19.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:19.224+0000 7f24c3e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563e5ea51e40 0x563e5ea72a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:19.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:19.224+0000 7f24c8a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563e5ea5d020 0x563e5ea702e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:19.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:19.224+0000 7f24c8a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563e5ea5d020 0x563e5ea702e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:42398/0 (socket says 172.21.15.5:42398) 2024-09-10T09:55:19.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:19.224+0000 7f24c8a00640 1 -- 172.21.15.5:0/736110958 learned_addr learned my addr 172.21.15.5:0/736110958 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:55:19.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:19.224+0000 7f24c3400640 1 --2- 172.21.15.5:0/736110958 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563e5ea52760 0x563e5ea6fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:19.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:19.224+0000 7f24c8a00640 1 -- 172.21.15.5:0/736110958 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563e5ea52760 msgr2=0x563e5ea6fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:55:19.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:19.224+0000 7f24c8a00640 1 --2- 172.21.15.5:0/736110958 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563e5ea52760 0x563e5ea6fda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:19.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:19.224+0000 7f24c8a00640 1 -- 172.21.15.5:0/736110958 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563e5ea51e40 msgr2=0x563e5ea72a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:55:19.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:19.224+0000 7f24c8a00640 1 --2- 172.21.15.5:0/736110958 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563e5ea51e40 0x563e5ea72a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:19.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:19.224+0000 7f24c8a00640 1 -- 172.21.15.5:0/736110958 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563e5ea70ae0 con 0x563e5ea5d020 2024-09-10T09:55:19.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:19.224+0000 7f24c3400640 1 --2- 172.21.15.5:0/736110958 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563e5ea52760 0x563e5ea6fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:55:19.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:19.224+0000 7f24c3e00640 1 --2- 172.21.15.5:0/736110958 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563e5ea51e40 0x563e5ea72a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:55:19.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:19.224+0000 7f24c8a00640 1 --2- 172.21.15.5:0/736110958 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563e5ea5d020 0x563e5ea702e0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f24bc010da0 tx=0x7f24bc011f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6150b975f03e6887 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:55:19.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:19.224+0000 7f24ad600640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-09-10T09:55:19.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:19.225+0000 7f24ad600640 1 -- 172.21.15.5:0/736110958 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24bc02e060 con 0x563e5ea5d020 2024-09-10T09:55:19.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:19.225+0000 7f24ad600640 1 -- 172.21.15.5:0/736110958 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f24bc01e0d0 con 0x563e5ea5d020 2024-09-10T09:55:19.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:19.225+0000 7f24cd3bf740 1 -- 172.21.15.5:0/736110958 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f24b0030780 con 0x563e5ea5d020 2024-09-10T09:55:19.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:19.225+0000 7f24ad600640 1 -- 172.21.15.5:0/736110958 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f24bc035050 con 0x563e5ea5d020 2024-09-10T09:55:19.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:19.225+0000 7f24cd3bf740 10 client.28759 fetch_fsmap learned FSMap version 42 2024-09-10T09:55:19.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:19.225+0000 7f24cd3bf740 1 -- 172.21.15.5:0/736110958 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x563e5ead7890 con 0x563e5ea5d020 2024-09-10T09:55:19.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:19.225+0000 7f24ad600640 1 -- 172.21.15.5:0/736110958 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f24b0030780 con 0x563e5ea5d020 2024-09-10T09:55:19.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:19.226+0000 7f24ad600640 1 -- 172.21.15.5:0/736110958 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 42) ==== 8370+0+0 (secure 0 0 0) 0x7f24bc01c590 con 0x563e5ea5d020 2024-09-10T09:55:19.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:19.226+0000 7f24cd3bf740 10 client.28759 fetch_fsmap finished waiting for FSMap version 42 2024-09-10T09:55:19.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:19.226+0000 7f24cd3bf740 10 client.28759 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-10T09:55:19.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:19.226+0000 7f24cd3bf740 20 client.28759 populate_metadata read hostname 'smithi005' 2024-09-10T09:55:19.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:19.226+0000 7f24cd3bf740 1 --2- 172.21.15.5:0/736110958 >> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] conn(0x563e5ead8040 0x563e5eada460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:19.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:19.226+0000 7f24cd3bf740 4 client.28759 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T09:55:19.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:19.226+0000 7f24cd3bf740 1 -- 172.21.15.5:0/736110958 --> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563e5ea499a0 con 0x563e5ead8040 2024-09-10T09:55:19.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:19.226+0000 7f24c3e00640 1 --2- 172.21.15.5:0/736110958 >> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] conn(0x563e5ead8040 0x563e5eada460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:19.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:19.226+0000 7f24c3e00640 1 --2- 172.21.15.5:0/736110958 >> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] conn(0x563e5ead8040 0x563e5eada460 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=36bc7ed94766b2d1 server_cookie=c4cb7e9a849f0283 in_seq=0 out_seq=0 2024-09-10T09:55:19.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:19.227+0000 7f24ad600640 10 client.28759 ms_handle_connect on v2:172.21.15.159:6836/620627639 2024-09-10T09:55:19.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:19.227+0000 7f24ad600640 1 -- 172.21.15.5:0/736110958 <== mds.2 v2:172.21.15.159:6836/620627639 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563e5ea499a0 con 0x563e5ead8040 2024-09-10T09:55:19.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:19.227+0000 7f24ad600640 10 client.28759 handle_command_reply: tid=0 2024-09-10T09:55:19.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:19.227+0000 7f24cd3bf740 1 client.28759 shutdown 2024-09-10T09:55:19.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:19.227+0000 7f24cd3bf740 1 -- 172.21.15.5:0/736110958 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563e5ea5d020 msgr2=0x563e5ea702e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:19.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:19.227+0000 7f24cd3bf740 1 --2- 172.21.15.5:0/736110958 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563e5ea5d020 0x563e5ea702e0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f24bc010da0 tx=0x7f24bc011f90 comp rx=0 tx=0).stop 2024-09-10T09:55:19.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:19.227+0000 7f24cd3bf740 1 -- 172.21.15.5:0/736110958 shutdown_connections 2024-09-10T09:55:19.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:19.227+0000 7f24cd3bf740 1 --2- 172.21.15.5:0/736110958 >> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] conn(0x563e5ead8040 0x563e5eada460 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:19.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:19.227+0000 7f24cd3bf740 1 --2- 172.21.15.5:0/736110958 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563e5ea5d020 0x563e5ea702e0 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:19.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:19.227+0000 7f24ad600640 0 client.28759 ms_handle_reset on v2:172.21.15.159:6836/620627639 2024-09-10T09:55:19.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:19.227+0000 7f24cd3bf740 1 --2- 172.21.15.5:0/736110958 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563e5ea52760 0x563e5ea6fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:19.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:19.227+0000 7f24cd3bf740 1 --2- 172.21.15.5:0/736110958 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563e5ea51e40 0x563e5ea72a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:19.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:19.227+0000 7f24cd3bf740 1 -- 172.21.15.5:0/736110958 >> 172.21.15.5:0/736110958 conn(0x563e5ea4e3d0 msgr2=0x563e5ea53890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:55:19.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:19.227+0000 7f24cd3bf740 1 -- 172.21.15.5:0/736110958 shutdown_connections 2024-09-10T09:55:19.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:19.227+0000 7f24cd3bf740 1 -- 172.21.15.5:0/736110958 wait complete. 2024-09-10T09:55:19.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:19.227+0000 7f24cd3bf740 20 client.28759 trim_cache size 0 max 16384 2024-09-10T09:55:19.283 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-10T09:55:19.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:19.228+0000 7f24c9400640 1 -- 172.21.15.5:0/3999040334 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f24a80a2c00 msgr2=0x7f24a80a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:19.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:19.228+0000 7f24c9400640 1 --2- 172.21.15.5:0/3999040334 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f24a80a2c00 0x7f24a80a50c0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f24bc005fd0 tx=0x7f24bc009500 comp rx=0 tx=0).stop 2024-09-10T09:55:19.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:19.228+0000 7f24c9400640 1 -- 172.21.15.5:0/3999040334 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f24c41019c0 msgr2=0x7f24c419a4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:19.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:19.228+0000 7f24c9400640 1 --2- 172.21.15.5:0/3999040334 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f24c41019c0 0x7f24c419a4a0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f24b400e9c0 tx=0x7f24b400ee80 comp rx=0 tx=0).stop 2024-09-10T09:55:19.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:19.228+0000 7f24c9400640 1 -- 172.21.15.5:0/3999040334 shutdown_connections 2024-09-10T09:55:19.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:19.228+0000 7f24c9400640 1 --2- 172.21.15.5:0/3999040334 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f24a80a2c00 0x7f24a80a50c0 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:19.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:19.228+0000 7f24c9400640 1 --2- 172.21.15.5:0/3999040334 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f24c41079c0 0x7f24c419ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:19.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:19.228+0000 7f24c9400640 1 --2- 172.21.15.5:0/3999040334 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f24c4102360 0x7f24c419a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:19.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:19.228+0000 7f24c9400640 1 --2- 172.21.15.5:0/3999040334 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f24c41019c0 0x7f24c419a4a0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:19.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:19.228+0000 7f24c9400640 1 -- 172.21.15.5:0/3999040334 >> 172.21.15.5:0/3999040334 conn(0x7f24c40fd740 msgr2=0x7f24c40fdd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:55:19.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:19.228+0000 7f24c9400640 1 -- 172.21.15.5:0/3999040334 shutdown_connections 2024-09-10T09:55:19.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:19.228+0000 7f24c9400640 1 -- 172.21.15.5:0/3999040334 wait complete. 2024-09-10T09:55:19.980 INFO:tasks.workunit.client.0.smithi005.stdout: 10 63709 23.10 MB/sec execute 58 sec latency 2863.206 ms 2024-09-10T09:55:20.011 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:55:19 smithi005 ceph-mon[30350]: pgmap v902: 129 pgs: 129 active+clean; 602 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 4.7 MiB/s wr, 430 op/s 2024-09-10T09:55:20.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:55:19 smithi159 ceph-mon[35513]: pgmap v902: 129 pgs: 129 active+clean; 602 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 4.7 MiB/s wr, 430 op/s 2024-09-10T09:55:20.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:55:19 smithi102 ceph-mon[35442]: pgmap v902: 129 pgs: 129 active+clean; 602 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 4.7 MiB/s wr, 430 op/s 2024-09-10T09:55:20.487 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-10T09:55:20.912 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:55:20.980 INFO:tasks.workunit.client.0.smithi005.stdout: 10 63731 22.76 MB/sec execute 59 sec latency 3840.938 ms 2024-09-10T09:55:21.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:21.524+0000 7fabf6200640 1 -- 172.21.15.5:0/797913758 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fabf0101de0 msgr2=0x7fabf01021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:21.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:21.524+0000 7fabf6200640 1 --2- 172.21.15.5:0/797913758 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fabf0101de0 0x7fabf01021c0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fabe0009900 tx=0x7fabe002a410 comp rx=0 tx=0).stop 2024-09-10T09:55:21.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:21.524+0000 7fabf6200640 1 -- 172.21.15.5:0/797913758 shutdown_connections 2024-09-10T09:55:21.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:21.524+0000 7fabf6200640 1 --2- 172.21.15.5:0/797913758 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fabf0106f10 0x7fabf010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:21.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:21.524+0000 7fabf6200640 1 --2- 172.21.15.5:0/797913758 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fabf0102790 0x7fabf01067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:21.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:21.524+0000 7fabf6200640 1 --2- 172.21.15.5:0/797913758 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fabf0101de0 0x7fabf01021c0 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:21.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:21.524+0000 7fabf6200640 1 -- 172.21.15.5:0/797913758 >> 172.21.15.5:0/797913758 conn(0x7fabf00fd8c0 msgr2=0x7fabf00ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:55:21.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:21.524+0000 7fabf6200640 1 -- 172.21.15.5:0/797913758 shutdown_connections 2024-09-10T09:55:21.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:21.524+0000 7fabf6200640 1 -- 172.21.15.5:0/797913758 wait complete. 2024-09-10T09:55:21.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:21.525+0000 7fabf6200640 1 Processor -- start 2024-09-10T09:55:21.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:21.525+0000 7fabf6200640 1 -- start start 2024-09-10T09:55:21.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:21.526+0000 7fabf6200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fabf0101de0 0x7fabf00744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:21.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:21.526+0000 7fabf6200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fabf0102790 0x7fabf0074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:21.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:21.526+0000 7fabf6200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fabf0106f10 0x7fabf0074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:21.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:21.526+0000 7fabf6200640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fabf0111fc0 con 0x7fabf0102790 2024-09-10T09:55:21.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:21.526+0000 7fabf6200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fabf0111e40 con 0x7fabf0106f10 2024-09-10T09:55:21.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:21.526+0000 7fabf6200640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fabf0112140 con 0x7fabf0101de0 2024-09-10T09:55:21.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:21.526+0000 7fabef400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fabf0101de0 0x7fabf00744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:21.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:21.526+0000 7fabeea00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fabf0102790 0x7fabf0074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:21.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:21.526+0000 7fabef400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fabf0101de0 0x7fabf00744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:60890/0 (socket says 172.21.15.5:60890) 2024-09-10T09:55:21.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:21.526+0000 7fabef400640 1 -- 172.21.15.5:0/1329217789 learned_addr learned my addr 172.21.15.5:0/1329217789 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:55:21.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:21.526+0000 7fabefe00640 1 --2- 172.21.15.5:0/1329217789 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fabf0106f10 0x7fabf0074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:21.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:21.527+0000 7fabef400640 1 -- 172.21.15.5:0/1329217789 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fabf0106f10 msgr2=0x7fabf0074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:21.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:21.527+0000 7fabef400640 1 --2- 172.21.15.5:0/1329217789 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fabf0106f10 0x7fabf0074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:21.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:21.527+0000 7fabef400640 1 -- 172.21.15.5:0/1329217789 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fabf0102790 msgr2=0x7fabf0074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:21.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:21.527+0000 7fabef400640 1 --2- 172.21.15.5:0/1329217789 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fabf0102790 0x7fabf0074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:21.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:21.527+0000 7fabef400640 1 -- 172.21.15.5:0/1329217789 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabf01a6980 con 0x7fabf0101de0 2024-09-10T09:55:21.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:21.527+0000 7fabef400640 1 --2- 172.21.15.5:0/1329217789 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fabf0101de0 0x7fabf00744f0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fabe0004770 tx=0x7fabe0003c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:55:21.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:21.527+0000 7fabe7e00640 1 -- 172.21.15.5:0/1329217789 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabe003f070 con 0x7fabf0101de0 2024-09-10T09:55:21.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:21.527+0000 7fabf6200640 1 -- 172.21.15.5:0/1329217789 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fabf01a6c10 con 0x7fabf0101de0 2024-09-10T09:55:21.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:21.527+0000 7fabe7e00640 1 -- 172.21.15.5:0/1329217789 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fabe0031ca0 con 0x7fabf0101de0 2024-09-10T09:55:21.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:21.527+0000 7fabe7e00640 1 -- 172.21.15.5:0/1329217789 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabe0035030 con 0x7fabf0101de0 2024-09-10T09:55:21.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:21.527+0000 7fabf6200640 1 -- 172.21.15.5:0/1329217789 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fabf01a7140 con 0x7fabf0101de0 2024-09-10T09:55:21.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:21.530+0000 7fabf8922740 1 Processor -- start 2024-09-10T09:55:21.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:21.531+0000 7fabf8922740 1 -- start start 2024-09-10T09:55:21.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:21.531+0000 7fabe7e00640 1 -- 172.21.15.5:0/1329217789 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7fabe0042070 con 0x7fabf0101de0 2024-09-10T09:55:21.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:21.531+0000 7fabf8922740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558751ec0c70 0x558751ec1050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:21.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:21.531+0000 7fabf8922740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558751ec1590 0x558751ecb910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:21.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:21.531+0000 7fabf8922740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558751ecbe50 0x558751ece240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:21.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:21.531+0000 7fabf8922740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x558751e273e0 con 0x558751ec0c70 2024-09-10T09:55:21.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:21.531+0000 7fabf8922740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x558751dced80 con 0x558751ec1590 2024-09-10T09:55:21.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:21.531+0000 7fabf8922740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x558751e233a0 con 0x558751ecbe50 2024-09-10T09:55:21.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:21.531+0000 7fabefe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558751ec0c70 0x558751ec1050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:21.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:21.532+0000 7fabefe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558751ec0c70 0x558751ec1050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58150/0 (socket says 172.21.15.5:58150) 2024-09-10T09:55:21.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:21.532+0000 7fabefe00640 1 -- 172.21.15.5:0/125733687 learned_addr learned my addr 172.21.15.5:0/125733687 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:55:21.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:21.532+0000 7fabef400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558751ec1590 0x558751ecb910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:21.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:21.532+0000 7fabeea00640 1 --2- 172.21.15.5:0/125733687 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558751ecbe50 0x558751ece240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:21.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:21.532+0000 7fabefe00640 1 -- 172.21.15.5:0/125733687 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558751ecbe50 msgr2=0x558751ece240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:55:21.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:21.532+0000 7fabefe00640 1 --2- 172.21.15.5:0/125733687 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558751ecbe50 0x558751ece240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:21.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:21.532+0000 7fabefe00640 1 -- 172.21.15.5:0/125733687 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558751ec1590 msgr2=0x558751ecb910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:55:21.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:21.532+0000 7fabefe00640 1 --2- 172.21.15.5:0/125733687 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558751ec1590 0x558751ecb910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:21.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:21.532+0000 7fabefe00640 1 -- 172.21.15.5:0/125733687 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558751dedae0 con 0x558751ec0c70 2024-09-10T09:55:21.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:21.532+0000 7fabefe00640 1 --2- 172.21.15.5:0/125733687 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558751ec0c70 0x558751ec1050 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7fabe8009f50 tx=0x7fabe80118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=daa9ed47298d9df4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:55:21.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:21.533+0000 7fabdbe00640 1 -- 172.21.15.5:0/125733687 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabe8004b20 con 0x558751ec0c70 2024-09-10T09:55:21.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:21.533+0000 7fabdbe00640 1 -- 172.21.15.5:0/125733687 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fabe8004cc0 con 0x558751ec0c70 2024-09-10T09:55:21.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:21.533+0000 7fabdbe00640 1 -- 172.21.15.5:0/125733687 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabe8005020 con 0x558751ec0c70 2024-09-10T09:55:21.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:21.533+0000 7fabe7e00640 1 --2- 172.21.15.5:0/1329217789 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fabd40a2b80 0x7fabd40a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:21.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:21.533+0000 7fabe7e00640 1 -- 172.21.15.5:0/1329217789 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fabe00efa50 con 0x7fabf0101de0 2024-09-10T09:55:21.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:21.533+0000 7fabf8922740 1 -- 172.21.15.5:0/125733687 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558751ec0c70 msgr2=0x558751ec1050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:21.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:21.533+0000 7fabf8922740 1 --2- 172.21.15.5:0/125733687 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558751ec0c70 0x558751ec1050 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7fabe8009f50 tx=0x7fabe80118d0 comp rx=0 tx=0).stop 2024-09-10T09:55:21.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:21.534+0000 7fabf8922740 1 -- 172.21.15.5:0/125733687 shutdown_connections 2024-09-10T09:55:21.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:21.534+0000 7fabf8922740 1 --2- 172.21.15.5:0/125733687 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558751ecbe50 0x558751ece240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:21.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:21.534+0000 7fabf8922740 1 --2- 172.21.15.5:0/125733687 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558751ec1590 0x558751ecb910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:21.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:21.534+0000 7fabf8922740 1 --2- 172.21.15.5:0/125733687 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558751ec0c70 0x558751ec1050 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:21.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:21.534+0000 7fabf8922740 1 -- 172.21.15.5:0/125733687 >> 172.21.15.5:0/125733687 conn(0x558751ebd200 msgr2=0x558751ebd5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:55:21.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:21.534+0000 7fabef400640 1 --2- 172.21.15.5:0/1329217789 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fabd40a2b80 0x7fabd40a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:21.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:21.534+0000 7fabf8922740 1 -- 172.21.15.5:0/125733687 shutdown_connections 2024-09-10T09:55:21.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:21.534+0000 7fabf8922740 1 -- 172.21.15.5:0/125733687 wait complete. 2024-09-10T09:55:21.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:21.534+0000 7fabef400640 1 --2- 172.21.15.5:0/1329217789 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fabd40a2b80 0x7fabd40a5040 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fabe00b8430 tx=0x7fabe003b890 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:55:21.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:21.535+0000 7fabf8922740 1 -- start start 2024-09-10T09:55:21.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:21.535+0000 7fabf8922740 1 Processor -- start 2024-09-10T09:55:21.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:21.535+0000 7fabf8922740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558751ec0c70 0x558751ec8ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:21.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:21.535+0000 7fabf8922740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558751ec1590 0x558751ec9220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:21.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:21.535+0000 7fabf8922740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558751ecbe50 0x558751ec9760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:21.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:21.535+0000 7fabf8922740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x558751ecea20 con 0x558751ecbe50 2024-09-10T09:55:21.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:21.535+0000 7fabf8922740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x558751de20f0 con 0x558751ec1590 2024-09-10T09:55:21.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:21.535+0000 7fabf8922740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x558751eceba0 con 0x558751ec0c70 2024-09-10T09:55:21.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:21.535+0000 7fabf8922740 10 client.0 _ll_register_callbacks cb 0x558751dc9bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:55:21.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:21.536+0000 7fabefe00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558751ec0c70 0x558751ec8ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:21.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:21.536+0000 7fabeea00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558751ec1590 0x558751ec9220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:21.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:21.536+0000 7fabefe00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558751ec0c70 0x558751ec8ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:60896/0 (socket says 172.21.15.5:60896) 2024-09-10T09:55:21.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:21.536+0000 7fabefe00640 1 -- 172.21.15.5:0/2836394558 learned_addr learned my addr 172.21.15.5:0/2836394558 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:55:21.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:21.536+0000 7fabefe00640 1 --2- 172.21.15.5:0/2836394558 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558751ecbe50 0x558751ec9760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:21.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:21.536+0000 7fabeea00640 1 -- 172.21.15.5:0/2836394558 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558751ec0c70 msgr2=0x558751ec8ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:55:21.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:21.536+0000 7fabeea00640 1 --2- 172.21.15.5:0/2836394558 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558751ec0c70 0x558751ec8ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:21.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:21.536+0000 7fabeea00640 1 -- 172.21.15.5:0/2836394558 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558751ecbe50 msgr2=0x558751ec9760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:55:21.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:21.536+0000 7fabeea00640 1 --2- 172.21.15.5:0/2836394558 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558751ecbe50 0x558751ec9760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:21.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:21.536+0000 7fabeea00640 1 -- 172.21.15.5:0/2836394558 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558751b983c0 con 0x558751ec1590 2024-09-10T09:55:21.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:21.537+0000 7fabeea00640 1 --2- 172.21.15.5:0/2836394558 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558751ec1590 0x558751ec9220 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fabf0075f80 tx=0x7fabdc012b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=43f494294c851b27 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:55:21.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:21.537+0000 7fabd8c00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-09-10T09:55:21.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:21.537+0000 7fabd8c00640 1 -- 172.21.15.5:0/2836394558 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabdc004030 con 0x558751ec1590 2024-09-10T09:55:21.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:21.537+0000 7fabf8922740 1 -- 172.21.15.5:0/2836394558 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fabe8004cc0 con 0x558751ec1590 2024-09-10T09:55:21.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:21.537+0000 7fabd8c00640 1 -- 172.21.15.5:0/2836394558 <== mon.2 v2:172.21.15.102:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fabdc0041d0 con 0x558751ec1590 2024-09-10T09:55:21.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:21.537+0000 7fabd8c00640 1 -- 172.21.15.5:0/2836394558 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7fabdc0043d0 con 0x558751ec1590 2024-09-10T09:55:21.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:21.537+0000 7fabf8922740 10 client.28778 fetch_fsmap learned FSMap version 42 2024-09-10T09:55:21.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:21.537+0000 7fabf8922740 1 -- 172.21.15.5:0/2836394558 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x558751edaaa0 con 0x558751ec1590 2024-09-10T09:55:21.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:21.537+0000 7fabd8c00640 1 -- 172.21.15.5:0/2836394558 <== mon.2 v2:172.21.15.102:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fabe8004cc0 con 0x558751ec1590 2024-09-10T09:55:21.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:21.538+0000 7fabd8c00640 1 -- 172.21.15.5:0/2836394558 <== mon.2 v2:172.21.15.102:3300/0 5 ==== fsmap(e 42) ==== 8370+0+0 (secure 0 0 0) 0x7fabdc003bb0 con 0x558751ec1590 2024-09-10T09:55:21.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:21.538+0000 7fabf8922740 10 client.28778 fetch_fsmap finished waiting for FSMap version 42 2024-09-10T09:55:21.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:21.538+0000 7fabf8922740 10 client.28778 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-10T09:55:21.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:21.538+0000 7fabf8922740 20 client.28778 populate_metadata read hostname 'smithi005' 2024-09-10T09:55:21.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:21.538+0000 7fabf8922740 1 --2- 172.21.15.5:0/2836394558 >> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] conn(0x558751edbf00 0x558751ede320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:21.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:21.538+0000 7fabf8922740 4 client.28778 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:55:21.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:21.538+0000 7fabf8922740 1 -- 172.21.15.5:0/2836394558 --> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558751e34950 con 0x558751edbf00 2024-09-10T09:55:21.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:21.538+0000 7fabefe00640 1 --2- 172.21.15.5:0/2836394558 >> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] conn(0x558751edbf00 0x558751ede320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:21.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:21.539+0000 7fabefe00640 1 --2- 172.21.15.5:0/2836394558 >> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] conn(0x558751edbf00 0x558751ede320 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c1afe774004c9ab2 server_cookie=59d8a2cc53ef175f in_seq=0 out_seq=0 2024-09-10T09:55:21.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:21.539+0000 7fabd8c00640 10 client.28778 ms_handle_connect on v2:172.21.15.159:6838/2073739599 2024-09-10T09:55:21.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:21.539+0000 7fabd8c00640 1 -- 172.21.15.5:0/2836394558 <== mds.1 v2:172.21.15.159:6838/2073739599 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558751e34950 con 0x558751edbf00 2024-09-10T09:55:21.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:21.539+0000 7fabd8c00640 10 client.28778 handle_command_reply: tid=0 2024-09-10T09:55:21.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:21.539+0000 7fabf8922740 1 client.28778 shutdown 2024-09-10T09:55:21.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:21.540+0000 7fabf8922740 1 -- 172.21.15.5:0/2836394558 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558751ec1590 msgr2=0x558751ec9220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:21.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:21.540+0000 7fabf8922740 1 --2- 172.21.15.5:0/2836394558 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558751ec1590 0x558751ec9220 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fabf0075f80 tx=0x7fabdc012b60 comp rx=0 tx=0).stop 2024-09-10T09:55:21.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:21.540+0000 7fabf8922740 1 -- 172.21.15.5:0/2836394558 shutdown_connections 2024-09-10T09:55:21.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:21.540+0000 7fabf8922740 1 --2- 172.21.15.5:0/2836394558 >> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] conn(0x558751edbf00 0x558751ede320 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:21.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:21.540+0000 7fabf8922740 1 --2- 172.21.15.5:0/2836394558 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558751ecbe50 0x558751ec9760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:21.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:21.540+0000 7fabf8922740 1 --2- 172.21.15.5:0/2836394558 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558751ec1590 0x558751ec9220 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:21.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:21.540+0000 7fabf8922740 1 --2- 172.21.15.5:0/2836394558 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558751ec0c70 0x558751ec8ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:21.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:21.540+0000 7fabd8c00640 0 client.28778 ms_handle_reset on v2:172.21.15.159:6838/2073739599 2024-09-10T09:55:21.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:21.540+0000 7fabf8922740 1 -- 172.21.15.5:0/2836394558 >> 172.21.15.5:0/2836394558 conn(0x558751ebd200 msgr2=0x558751ec2f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:55:21.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:21.540+0000 7fabf8922740 1 -- 172.21.15.5:0/2836394558 shutdown_connections 2024-09-10T09:55:21.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:21.540+0000 7fabf8922740 1 -- 172.21.15.5:0/2836394558 wait complete. 2024-09-10T09:55:21.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:21.540+0000 7fabf8922740 20 client.28778 trim_cache size 0 max 16384 2024-09-10T09:55:21.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:21.554+0000 7fabf8922740 1 Processor -- start 2024-09-10T09:55:21.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:21.554+0000 7fabf8922740 1 -- start start 2024-09-10T09:55:21.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:21.554+0000 7fabf8922740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558751ec0c70 0x558751ee3bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:21.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:21.554+0000 7fabf8922740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558751ec1590 0x558751ee40f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:21.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:21.554+0000 7fabf8922740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558751ecbe50 0x558751ebbb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:21.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:21.554+0000 7fabf8922740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fabf01b1040 con 0x558751ec0c70 2024-09-10T09:55:21.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:21.554+0000 7fabf8922740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fabf01b2b80 con 0x558751ec1590 2024-09-10T09:55:21.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:21.554+0000 7fabf8922740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fabf01b1710 con 0x558751ecbe50 2024-09-10T09:55:21.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:21.555+0000 7fabefe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558751ec0c70 0x558751ee3bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:21.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:21.555+0000 7fabefe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558751ec0c70 0x558751ee3bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58164/0 (socket says 172.21.15.5:58164) 2024-09-10T09:55:21.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:21.555+0000 7fabefe00640 1 -- 172.21.15.5:0/3394294635 learned_addr learned my addr 172.21.15.5:0/3394294635 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:55:21.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:21.555+0000 7fabeea00640 1 --2- 172.21.15.5:0/3394294635 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558751ec1590 0x558751ee40f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:21.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:21.555+0000 7fabefe00640 1 --2- 172.21.15.5:0/3394294635 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558751ecbe50 0x558751ebbb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:21.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:21.555+0000 7fabeea00640 1 -- 172.21.15.5:0/3394294635 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558751ecbe50 msgr2=0x558751ebbb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:55:21.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:21.555+0000 7fabeea00640 1 --2- 172.21.15.5:0/3394294635 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558751ecbe50 0x558751ebbb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:21.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:21.555+0000 7fabeea00640 1 -- 172.21.15.5:0/3394294635 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558751ec0c70 msgr2=0x558751ee3bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:55:21.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:21.555+0000 7fabeea00640 1 --2- 172.21.15.5:0/3394294635 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558751ec0c70 0x558751ee3bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:21.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:21.555+0000 7fabeea00640 1 -- 172.21.15.5:0/3394294635 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558751ede980 con 0x558751ec1590 2024-09-10T09:55:21.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:21.555+0000 7fabeea00640 1 --2- 172.21.15.5:0/3394294635 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558751ec1590 0x558751ee40f0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fabdc0105f0 tx=0x7fabdc0096a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a10f579fe848a500 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:55:21.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:21.556+0000 7fabdb400640 1 -- 172.21.15.5:0/3394294635 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabdc010670 con 0x558751ec1590 2024-09-10T09:55:21.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:21.556+0000 7fabdb400640 1 -- 172.21.15.5:0/3394294635 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fabdc02e070 con 0x558751ec1590 2024-09-10T09:55:21.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:21.556+0000 7fabdb400640 1 -- 172.21.15.5:0/3394294635 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabdc00fba0 con 0x558751ec1590 2024-09-10T09:55:21.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:21.556+0000 7fabf8922740 1 -- 172.21.15.5:0/3394294635 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558751ec1590 msgr2=0x558751ee40f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:21.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:21.556+0000 7fabf8922740 1 --2- 172.21.15.5:0/3394294635 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558751ec1590 0x558751ee40f0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fabdc0105f0 tx=0x7fabdc0096a0 comp rx=0 tx=0).stop 2024-09-10T09:55:21.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:21.556+0000 7fabf8922740 1 -- 172.21.15.5:0/3394294635 shutdown_connections 2024-09-10T09:55:21.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:21.556+0000 7fabf8922740 1 --2- 172.21.15.5:0/3394294635 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558751ecbe50 0x558751ebbb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:21.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:21.556+0000 7fabf8922740 1 --2- 172.21.15.5:0/3394294635 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558751ec1590 0x558751ee40f0 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:21.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:21.556+0000 7fabf8922740 1 --2- 172.21.15.5:0/3394294635 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558751ec0c70 0x558751ee3bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:21.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:21.556+0000 7fabf8922740 1 -- 172.21.15.5:0/3394294635 >> 172.21.15.5:0/3394294635 conn(0x558751ebd200 msgr2=0x558751ec7dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:55:21.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:21.557+0000 7fabf8922740 1 -- 172.21.15.5:0/3394294635 shutdown_connections 2024-09-10T09:55:21.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:21.557+0000 7fabf8922740 1 -- 172.21.15.5:0/3394294635 wait complete. 2024-09-10T09:55:21.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:21.557+0000 7fabf8922740 1 -- start start 2024-09-10T09:55:21.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:21.557+0000 7fabf8922740 1 Processor -- start 2024-09-10T09:55:21.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:21.558+0000 7fabf8922740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558751ec0c70 0x558751ec5eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:21.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:21.558+0000 7fabf8922740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558751ec1590 0x558751ec63f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:21.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:21.558+0000 7fabf8922740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558751ecbe50 0x558751ec6930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:21.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:21.558+0000 7fabf8922740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fabf01b24b0 con 0x558751ec1590 2024-09-10T09:55:21.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:21.558+0000 7fabf8922740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fabf01b1de0 con 0x558751ec0c70 2024-09-10T09:55:21.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:21.558+0000 7fabf8922740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fabf01af020 con 0x558751ecbe50 2024-09-10T09:55:21.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:21.558+0000 7fabf8922740 10 client.0 _ll_register_callbacks cb 0x7fabdc009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:55:21.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:21.558+0000 7fabefe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558751ec0c70 0x558751ec5eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:21.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:21.558+0000 7fabeea00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558751ec1590 0x558751ec63f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:21.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:21.558+0000 7fabefe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558751ec0c70 0x558751ec5eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:53608/0 (socket says 172.21.15.5:53608) 2024-09-10T09:55:21.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:21.558+0000 7fabefe00640 1 -- 172.21.15.5:0/1699642391 learned_addr learned my addr 172.21.15.5:0/1699642391 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:55:21.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:21.558+0000 7fabefe00640 1 --2- 172.21.15.5:0/1699642391 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558751ecbe50 0x558751ec6930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:21.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:21.558+0000 7fabefe00640 1 -- 172.21.15.5:0/1699642391 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558751ecbe50 msgr2=0x558751ec6930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:55:21.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:21.558+0000 7fabefe00640 1 --2- 172.21.15.5:0/1699642391 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558751ecbe50 0x558751ec6930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:21.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:21.559+0000 7fabefe00640 1 -- 172.21.15.5:0/1699642391 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558751ec1590 msgr2=0x558751ec63f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:55:21.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:21.559+0000 7fabefe00640 1 --2- 172.21.15.5:0/1699642391 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558751ec1590 0x558751ec63f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:21.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:21.559+0000 7fabefe00640 1 -- 172.21.15.5:0/1699642391 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558751ebe2b0 con 0x558751ec0c70 2024-09-10T09:55:21.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:21.559+0000 7fabeea00640 1 --2- 172.21.15.5:0/1699642391 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558751ec1590 0x558751ec63f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:55:21.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:21.559+0000 7fabefe00640 1 --2- 172.21.15.5:0/1699642391 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558751ec0c70 0x558751ec5eb0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fabe8009270 tx=0x7fabe8005fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5df5f6a406509e62 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:55:21.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:21.559+0000 7fabd8c00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-09-10T09:55:21.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:21.559+0000 7fabd8c00640 1 -- 172.21.15.5:0/1699642391 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabe802e050 con 0x558751ec0c70 2024-09-10T09:55:21.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:21.559+0000 7fabd8c00640 1 -- 172.21.15.5:0/1699642391 <== mon.2 v2:172.21.15.102:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fabe8016040 con 0x558751ec0c70 2024-09-10T09:55:21.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:21.559+0000 7fabf8922740 1 -- 172.21.15.5:0/1699642391 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fabdc02e070 con 0x558751ec0c70 2024-09-10T09:55:21.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:21.560+0000 7fabd8c00640 1 -- 172.21.15.5:0/1699642391 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7fabe8044050 con 0x558751ec0c70 2024-09-10T09:55:21.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:21.560+0000 7fabf8922740 10 client.28784 fetch_fsmap learned FSMap version 42 2024-09-10T09:55:21.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:21.560+0000 7fabf8922740 1 -- 172.21.15.5:0/1699642391 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x558751eb8770 con 0x558751ec0c70 2024-09-10T09:55:21.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:21.560+0000 7fabd8c00640 1 -- 172.21.15.5:0/1699642391 <== mon.2 v2:172.21.15.102:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fabdc02e070 con 0x558751ec0c70 2024-09-10T09:55:21.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:21.560+0000 7fabd8c00640 1 -- 172.21.15.5:0/1699642391 <== mon.2 v2:172.21.15.102:3300/0 5 ==== fsmap(e 42) ==== 8370+0+0 (secure 0 0 0) 0x7fabe803b8c0 con 0x558751ec0c70 2024-09-10T09:55:21.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:21.561+0000 7fabf8922740 10 client.28784 fetch_fsmap finished waiting for FSMap version 42 2024-09-10T09:55:21.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:21.561+0000 7fabf8922740 10 client.28784 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-10T09:55:21.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:21.561+0000 7fabf8922740 20 client.28784 populate_metadata read hostname 'smithi005' 2024-09-10T09:55:21.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:21.561+0000 7fabf8922740 1 --2- 172.21.15.5:0/1699642391 >> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] conn(0x558751f471b0 0x558751f495a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:21.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:21.561+0000 7fabf8922740 4 client.28784 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T09:55:21.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:21.561+0000 7fabf8922740 1 -- 172.21.15.5:0/1699642391 --> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558751e2ee60 con 0x558751f471b0 2024-09-10T09:55:21.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:21.561+0000 7fabeea00640 1 --2- 172.21.15.5:0/1699642391 >> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] conn(0x558751f471b0 0x558751f495a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:21.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:21.561+0000 7fabeea00640 1 --2- 172.21.15.5:0/1699642391 >> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] conn(0x558751f471b0 0x558751f495a0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=37b6ade895ccfb27 server_cookie=825a779379595c64 in_seq=0 out_seq=0 2024-09-10T09:55:21.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:21.561+0000 7fabd8c00640 10 client.28784 ms_handle_connect on v2:172.21.15.159:6838/2073739599 2024-09-10T09:55:21.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:21.562+0000 7fabd8c00640 1 -- 172.21.15.5:0/1699642391 <== mds.1 v2:172.21.15.159:6838/2073739599 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558751e2ee60 con 0x558751f471b0 2024-09-10T09:55:21.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:21.562+0000 7fabd8c00640 10 client.28784 handle_command_reply: tid=0 2024-09-10T09:55:21.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:21.562+0000 7fabf8922740 1 client.28784 shutdown 2024-09-10T09:55:21.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:21.562+0000 7fabf8922740 1 -- 172.21.15.5:0/1699642391 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558751ec0c70 msgr2=0x558751ec5eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:21.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:21.562+0000 7fabf8922740 1 --2- 172.21.15.5:0/1699642391 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558751ec0c70 0x558751ec5eb0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fabe8009270 tx=0x7fabe8005fd0 comp rx=0 tx=0).stop 2024-09-10T09:55:21.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:21.562+0000 7fabf8922740 1 -- 172.21.15.5:0/1699642391 shutdown_connections 2024-09-10T09:55:21.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:21.562+0000 7fabf8922740 1 --2- 172.21.15.5:0/1699642391 >> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] conn(0x558751f471b0 0x558751f495a0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:21.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:21.562+0000 7fabf8922740 1 --2- 172.21.15.5:0/1699642391 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558751ecbe50 0x558751ec6930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:21.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:21.562+0000 7fabf8922740 1 --2- 172.21.15.5:0/1699642391 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558751ec1590 0x558751ec63f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:21.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:21.562+0000 7fabf8922740 1 --2- 172.21.15.5:0/1699642391 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558751ec0c70 0x558751ec5eb0 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:21.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:21.562+0000 7fabf8922740 1 -- 172.21.15.5:0/1699642391 >> 172.21.15.5:0/1699642391 conn(0x558751ebd200 msgr2=0x558751ebbb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:55:21.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:21.562+0000 7fabd8c00640 0 client.28784 ms_handle_reset on v2:172.21.15.159:6838/2073739599 2024-09-10T09:55:21.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:21.562+0000 7fabf8922740 1 -- 172.21.15.5:0/1699642391 shutdown_connections 2024-09-10T09:55:21.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:21.563+0000 7fabf8922740 1 -- 172.21.15.5:0/1699642391 wait complete. 2024-09-10T09:55:21.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:21.563+0000 7fabf8922740 20 client.28784 trim_cache size 0 max 16384 2024-09-10T09:55:21.568 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-10T09:55:21.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:21.563+0000 7fabf6200640 1 -- 172.21.15.5:0/1329217789 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fabd40a2b80 msgr2=0x7fabd40a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:21.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:21.563+0000 7fabf6200640 1 --2- 172.21.15.5:0/1329217789 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fabd40a2b80 0x7fabd40a5040 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fabe00b8430 tx=0x7fabe003b890 comp rx=0 tx=0).stop 2024-09-10T09:55:21.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:21.564+0000 7fabf6200640 1 -- 172.21.15.5:0/1329217789 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fabf0101de0 msgr2=0x7fabf00744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:21.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:21.564+0000 7fabf6200640 1 --2- 172.21.15.5:0/1329217789 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fabf0101de0 0x7fabf00744f0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fabe0004770 tx=0x7fabe0003c10 comp rx=0 tx=0).stop 2024-09-10T09:55:21.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:21.564+0000 7fabf6200640 1 -- 172.21.15.5:0/1329217789 shutdown_connections 2024-09-10T09:55:21.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:21.564+0000 7fabf6200640 1 --2- 172.21.15.5:0/1329217789 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fabd40a2b80 0x7fabd40a5040 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:21.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:21.564+0000 7fabf6200640 1 --2- 172.21.15.5:0/1329217789 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fabf0106f10 0x7fabf0074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:21.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:21.564+0000 7fabf6200640 1 --2- 172.21.15.5:0/1329217789 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fabf0102790 0x7fabf0074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:21.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:21.564+0000 7fabf6200640 1 --2- 172.21.15.5:0/1329217789 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fabf0101de0 0x7fabf00744f0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:21.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:21.564+0000 7fabf6200640 1 -- 172.21.15.5:0/1329217789 >> 172.21.15.5:0/1329217789 conn(0x7fabf00fd8c0 msgr2=0x7fabf00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:55:21.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:21.564+0000 7fabf6200640 1 -- 172.21.15.5:0/1329217789 shutdown_connections 2024-09-10T09:55:21.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:21.564+0000 7fabf6200640 1 -- 172.21.15.5:0/1329217789 wait complete. 2024-09-10T09:55:21.908 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:55:21 smithi005 ceph-mon[30350]: pgmap v903: 129 pgs: 129 active+clean; 613 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 5.7 MiB/s wr, 471 op/s 2024-09-10T09:55:21.980 INFO:tasks.workunit.client.0.smithi005.stdout: 10 63760 22.40 MB/sec execute 60 sec latency 4840.992 ms 2024-09-10T09:55:22.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:55:21 smithi159 ceph-mon[35513]: pgmap v903: 129 pgs: 129 active+clean; 613 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 5.7 MiB/s wr, 471 op/s 2024-09-10T09:55:22.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:55:21 smithi102 ceph-mon[35442]: pgmap v903: 129 pgs: 129 active+clean; 613 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 5.7 MiB/s wr, 471 op/s 2024-09-10T09:55:22.980 INFO:tasks.workunit.client.0.smithi005.stdout: 10 63797 22.05 MB/sec execute 61 sec latency 5274.670 ms 2024-09-10T09:55:23.075 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-10T09:55:23.478 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:55:23.980 INFO:tasks.workunit.client.0.smithi005.stdout: 10 63797 21.69 MB/sec execute 62 sec latency 1574.944 ms 2024-09-10T09:55:24.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:55:23 smithi005 ceph-mon[30350]: pgmap v904: 129 pgs: 129 active+clean; 615 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 5.6 MiB/s wr, 398 op/s 2024-09-10T09:55:24.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:55:23 smithi159 ceph-mon[35513]: pgmap v904: 129 pgs: 129 active+clean; 615 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 5.6 MiB/s wr, 398 op/s 2024-09-10T09:55:24.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:55:23 smithi102 ceph-mon[35442]: pgmap v904: 129 pgs: 129 active+clean; 615 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 5.6 MiB/s wr, 398 op/s 2024-09-10T09:55:24.980 INFO:tasks.workunit.client.0.smithi005.stdout: 10 63797 21.35 MB/sec execute 63 sec latency 2575.005 ms 2024-09-10T09:55:25.980 INFO:tasks.workunit.client.0.smithi005.stdout: 10 63797 21.01 MB/sec execute 64 sec latency 3575.068 ms 2024-09-10T09:55:26.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:55:25 smithi005 ceph-mon[30350]: pgmap v905: 129 pgs: 129 active+clean; 614 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 5.5 MiB/s wr, 335 op/s 2024-09-10T09:55:26.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:55:25 smithi159 ceph-mon[35513]: pgmap v905: 129 pgs: 129 active+clean; 614 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 5.5 MiB/s wr, 335 op/s 2024-09-10T09:55:26.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:55:25 smithi102 ceph-mon[35442]: pgmap v905: 129 pgs: 129 active+clean; 614 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 5.5 MiB/s wr, 335 op/s 2024-09-10T09:55:26.980 INFO:tasks.workunit.client.0.smithi005.stdout: 10 63797 20.69 MB/sec execute 65 sec latency 4575.130 ms 2024-09-10T09:55:27.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:27.944+0000 7f662c400640 1 -- 172.21.15.5:0/831470361 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6624100510 msgr2=0x7f6624100990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:27.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:27.944+0000 7f662c400640 1 --2- 172.21.15.5:0/831470361 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6624100510 0x7f6624100990 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f661000aa40 tx=0x7f661002a650 comp rx=0 tx=0).stop 2024-09-10T09:55:27.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:27.944+0000 7f662c400640 1 -- 172.21.15.5:0/831470361 shutdown_connections 2024-09-10T09:55:27.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:27.944+0000 7f662c400640 1 --2- 172.21.15.5:0/831470361 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6624100ed0 0x7f662410d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:27.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:27.944+0000 7f662c400640 1 --2- 172.21.15.5:0/831470361 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6624100510 0x7f6624100990 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:27.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:27.944+0000 7f662c400640 1 --2- 172.21.15.5:0/831470361 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f66241051f0 0x7f66241055d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:27.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:27.944+0000 7f662c400640 1 -- 172.21.15.5:0/831470361 >> 172.21.15.5:0/831470361 conn(0x7f66240fba90 msgr2=0x7f66240fdeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:55:27.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:27.944+0000 7f662c400640 1 -- 172.21.15.5:0/831470361 shutdown_connections 2024-09-10T09:55:27.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:27.945+0000 7f662c400640 1 -- 172.21.15.5:0/831470361 wait complete. 2024-09-10T09:55:27.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:27.946+0000 7f662c400640 1 Processor -- start 2024-09-10T09:55:27.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:27.946+0000 7f662c400640 1 -- start start 2024-09-10T09:55:27.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:27.947+0000 7f662c400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6624100510 0x7f662419cf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:27.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:27.947+0000 7f662c400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6624100ed0 0x7f662419d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:27.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:27.947+0000 7f662c400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f66241051f0 0x7f66241961a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:27.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:27.947+0000 7f662c400640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f662410fd90 con 0x7f66241051f0 2024-09-10T09:55:27.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:27.947+0000 7f662c400640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f662410fc10 con 0x7f6624100510 2024-09-10T09:55:27.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:27.947+0000 7f662c400640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f662410ff10 con 0x7f6624100ed0 2024-09-10T09:55:27.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:27.947+0000 7f6629a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6624100510 0x7f662419cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:27.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:27.947+0000 7f6629a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6624100510 0x7f662419cf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:53630/0 (socket says 172.21.15.5:53630) 2024-09-10T09:55:27.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:27.947+0000 7f6629a00640 1 -- 172.21.15.5:0/421242669 learned_addr learned my addr 172.21.15.5:0/421242669 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:55:27.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:27.947+0000 7f6629000640 1 --2- 172.21.15.5:0/421242669 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6624100ed0 0x7f662419d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:27.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:27.947+0000 7f662a400640 1 --2- 172.21.15.5:0/421242669 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f66241051f0 0x7f66241961a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:27.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:27.948+0000 7f6629a00640 1 -- 172.21.15.5:0/421242669 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6624100ed0 msgr2=0x7f662419d470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:27.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:27.948+0000 7f6629a00640 1 --2- 172.21.15.5:0/421242669 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6624100ed0 0x7f662419d470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:27.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:27.948+0000 7f6629a00640 1 -- 172.21.15.5:0/421242669 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f66241051f0 msgr2=0x7f66241961a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:27.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:27.948+0000 7f6629a00640 1 --2- 172.21.15.5:0/421242669 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f66241051f0 0x7f66241961a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:27.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:27.948+0000 7f6629a00640 1 -- 172.21.15.5:0/421242669 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6624196a60 con 0x7f6624100510 2024-09-10T09:55:27.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:27.948+0000 7f6629a00640 1 --2- 172.21.15.5:0/421242669 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6624100510 0x7f662419cf30 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f661800c920 tx=0x7f661800cde0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:55:27.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:27.949+0000 7f6622000640 1 -- 172.21.15.5:0/421242669 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6618007c70 con 0x7f6624100510 2024-09-10T09:55:27.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:27.949+0000 7f6622000640 1 -- 172.21.15.5:0/421242669 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6618007e10 con 0x7f6624100510 2024-09-10T09:55:27.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:27.949+0000 7f6622000640 1 -- 172.21.15.5:0/421242669 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6618018890 con 0x7f6624100510 2024-09-10T09:55:27.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:27.949+0000 7f662c400640 1 -- 172.21.15.5:0/421242669 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6624196d50 con 0x7f6624100510 2024-09-10T09:55:27.952 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:55:27 smithi005 ceph-mon[30350]: pgmap v906: 129 pgs: 129 active+clean; 612 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 6.2 MiB/s wr, 368 op/s 2024-09-10T09:55:27.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:27.950+0000 7f662c400640 1 -- 172.21.15.5:0/421242669 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f662419ace0 con 0x7f6624100510 2024-09-10T09:55:27.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:27.954+0000 7f6622000640 1 -- 172.21.15.5:0/421242669 <== mon.2 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f6618018a70 con 0x7f6624100510 2024-09-10T09:55:27.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:27.955+0000 7f662ee06740 1 Processor -- start 2024-09-10T09:55:27.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:27.955+0000 7f662ee06740 1 -- start start 2024-09-10T09:55:27.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:27.955+0000 7f662ee06740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f6cc601c70 0x55f6cc602050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:27.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:27.955+0000 7f662ee06740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f6cc602590 0x55f6cc60c910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:27.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:27.955+0000 7f662ee06740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f6cc60ce50 0x55f6cc60f240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:27.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:27.955+0000 7f662ee06740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55f6cc5683e0 con 0x55f6cc60ce50 2024-09-10T09:55:27.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:27.955+0000 7f662ee06740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55f6cc50fd80 con 0x55f6cc602590 2024-09-10T09:55:27.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:27.955+0000 7f662ee06740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55f6cc5643a0 con 0x55f6cc601c70 2024-09-10T09:55:27.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:27.956+0000 7f662a400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f6cc601c70 0x55f6cc602050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:27.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:27.956+0000 7f662a400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f6cc601c70 0x55f6cc602050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:60950/0 (socket says 172.21.15.5:60950) 2024-09-10T09:55:27.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:27.956+0000 7f662a400640 1 -- 172.21.15.5:0/1560856839 learned_addr learned my addr 172.21.15.5:0/1560856839 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:55:27.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:27.957+0000 7f6629000640 1 --2- 172.21.15.5:0/1560856839 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f6cc60ce50 0x55f6cc60f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:27.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:27.957+0000 7f6629a00640 1 --2- 172.21.15.5:0/1560856839 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f6cc602590 0x55f6cc60c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:27.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:27.957+0000 7f662a400640 1 -- 172.21.15.5:0/1560856839 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f6cc602590 msgr2=0x55f6cc60c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:55:27.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:27.957+0000 7f662a400640 1 --2- 172.21.15.5:0/1560856839 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f6cc602590 0x55f6cc60c910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:27.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:27.957+0000 7f662a400640 1 -- 172.21.15.5:0/1560856839 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f6cc60ce50 msgr2=0x55f6cc60f240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:55:27.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:27.957+0000 7f662a400640 1 --2- 172.21.15.5:0/1560856839 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f6cc60ce50 0x55f6cc60f240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:27.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:27.957+0000 7f662a400640 1 -- 172.21.15.5:0/1560856839 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f6cc52eae0 con 0x55f6cc601c70 2024-09-10T09:55:27.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:27.957+0000 7f6629a00640 1 --2- 172.21.15.5:0/1560856839 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f6cc602590 0x55f6cc60c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T09:55:27.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:27.958+0000 7f662a400640 1 --2- 172.21.15.5:0/1560856839 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f6cc601c70 0x55f6cc602050 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f661c009fc0 tx=0x7f661c009500 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e3ddc91e5c0a388a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:55:27.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:27.958+0000 7f6616a00640 1 -- 172.21.15.5:0/1560856839 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f661c004d20 con 0x55f6cc601c70 2024-09-10T09:55:27.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:27.958+0000 7f6616a00640 1 -- 172.21.15.5:0/1560856839 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f661c003e50 con 0x55f6cc601c70 2024-09-10T09:55:27.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:27.958+0000 7f6616a00640 1 -- 172.21.15.5:0/1560856839 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f661c005280 con 0x55f6cc601c70 2024-09-10T09:55:27.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:27.959+0000 7f6622000640 1 --2- 172.21.15.5:0/421242669 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f660c0a2c00 0x7f660c0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:27.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:27.959+0000 7f6622000640 1 -- 172.21.15.5:0/421242669 <== mon.2 v2:172.21.15.102:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f66180cf4e0 con 0x7f6624100510 2024-09-10T09:55:27.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:27.960+0000 7f6629a00640 1 --2- 172.21.15.5:0/421242669 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f660c0a2c00 0x7f660c0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:27.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:27.962+0000 7f662ee06740 1 -- 172.21.15.5:0/1560856839 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f6cc601c70 msgr2=0x55f6cc602050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:27.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:27.962+0000 7f662ee06740 1 --2- 172.21.15.5:0/1560856839 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f6cc601c70 0x55f6cc602050 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f661c009fc0 tx=0x7f661c009500 comp rx=0 tx=0).stop 2024-09-10T09:55:27.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:27.963+0000 7f662ee06740 1 -- 172.21.15.5:0/1560856839 shutdown_connections 2024-09-10T09:55:27.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:27.963+0000 7f662ee06740 1 --2- 172.21.15.5:0/1560856839 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f6cc60ce50 0x55f6cc60f240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:27.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:27.963+0000 7f662ee06740 1 --2- 172.21.15.5:0/1560856839 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f6cc602590 0x55f6cc60c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:27.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:27.963+0000 7f6629a00640 1 --2- 172.21.15.5:0/421242669 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f660c0a2c00 0x7f660c0a50c0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f66180ccf60 tx=0x7f6618000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:55:27.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:27.963+0000 7f662ee06740 1 --2- 172.21.15.5:0/1560856839 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f6cc601c70 0x55f6cc602050 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:27.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:27.963+0000 7f662ee06740 1 -- 172.21.15.5:0/1560856839 >> 172.21.15.5:0/1560856839 conn(0x55f6cc5fe200 msgr2=0x55f6cc5fe5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:55:27.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:27.963+0000 7f662ee06740 1 -- 172.21.15.5:0/1560856839 shutdown_connections 2024-09-10T09:55:27.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:27.963+0000 7f662ee06740 1 -- 172.21.15.5:0/1560856839 wait complete. 2024-09-10T09:55:27.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:27.964+0000 7f662ee06740 1 -- start start 2024-09-10T09:55:27.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:27.964+0000 7f662ee06740 1 Processor -- start 2024-09-10T09:55:27.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:27.965+0000 7f662ee06740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f6cc601c70 0x55f6cc609d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:27.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:27.965+0000 7f662ee06740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f6cc602590 0x55f6cc60a270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:27.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:27.965+0000 7f662ee06740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f6cc60ce50 0x55f6cc60a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:27.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:27.965+0000 7f662ee06740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55f6cc60fa70 con 0x55f6cc60ce50 2024-09-10T09:55:27.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:27.965+0000 7f662ee06740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55f6cc5230f0 con 0x55f6cc602590 2024-09-10T09:55:27.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:27.965+0000 7f662ee06740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55f6cc60fbf0 con 0x55f6cc601c70 2024-09-10T09:55:27.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:27.965+0000 7f662ee06740 10 client.0 _ll_register_callbacks cb 0x55f6cc50abd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:55:27.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:27.966+0000 7f662a400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f6cc601c70 0x55f6cc609d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:27.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:27.966+0000 7f662a400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f6cc601c70 0x55f6cc609d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:60966/0 (socket says 172.21.15.5:60966) 2024-09-10T09:55:27.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:27.966+0000 7f662a400640 1 -- 172.21.15.5:0/2575443161 learned_addr learned my addr 172.21.15.5:0/2575443161 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:55:27.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:27.966+0000 7f662a400640 1 -- 172.21.15.5:0/2575443161 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f6cc602590 msgr2=0x55f6cc60a270 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T09:55:27.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:27.966+0000 7f662a400640 1 --2- 172.21.15.5:0/2575443161 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f6cc602590 0x55f6cc60a270 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:27.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:27.966+0000 7f662a400640 1 -- 172.21.15.5:0/2575443161 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f6cc60ce50 msgr2=0x55f6cc60a7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:55:27.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:27.966+0000 7f662a400640 1 --2- 172.21.15.5:0/2575443161 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f6cc60ce50 0x55f6cc60a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:27.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:27.966+0000 7f662a400640 1 -- 172.21.15.5:0/2575443161 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f6cc2d93c0 con 0x55f6cc601c70 2024-09-10T09:55:27.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:27.966+0000 7f662a400640 1 --2- 172.21.15.5:0/2575443161 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f6cc601c70 0x55f6cc609d30 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f661c004530 tx=0x7f661c00df90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5692106ec4a2ec7c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:55:27.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:27.967+0000 7f660b400640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-09-10T09:55:27.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:27.967+0000 7f660b400640 1 -- 172.21.15.5:0/2575443161 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f661c028030 con 0x55f6cc601c70 2024-09-10T09:55:27.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:27.967+0000 7f660b400640 1 -- 172.21.15.5:0/2575443161 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f661c01d040 con 0x55f6cc601c70 2024-09-10T09:55:27.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:27.967+0000 7f662ee06740 1 -- 172.21.15.5:0/2575443161 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f661c003e50 con 0x55f6cc601c70 2024-09-10T09:55:27.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:27.968+0000 7f660b400640 1 -- 172.21.15.5:0/2575443161 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f661c032050 con 0x55f6cc601c70 2024-09-10T09:55:27.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:27.968+0000 7f662ee06740 10 client.28789 fetch_fsmap learned FSMap version 42 2024-09-10T09:55:27.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:27.968+0000 7f662ee06740 1 -- 172.21.15.5:0/2575443161 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55f6cc62cab0 con 0x55f6cc601c70 2024-09-10T09:55:27.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:27.969+0000 7f660b400640 1 -- 172.21.15.5:0/2575443161 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f661c003e50 con 0x55f6cc601c70 2024-09-10T09:55:27.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:27.970+0000 7f660b400640 1 -- 172.21.15.5:0/2575443161 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 42) ==== 8370+0+0 (secure 0 0 0) 0x7f661c005400 con 0x55f6cc601c70 2024-09-10T09:55:27.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:27.970+0000 7f662ee06740 10 client.28789 fetch_fsmap finished waiting for FSMap version 42 2024-09-10T09:55:27.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:27.970+0000 7f662ee06740 10 client.28789 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T09:55:27.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:27.970+0000 7f662ee06740 20 client.28789 populate_metadata read hostname 'smithi005' 2024-09-10T09:55:27.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:27.970+0000 7f662ee06740 1 --2- 172.21.15.5:0/2575443161 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55f6cc62df10 0x55f6cc630300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:27.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:27.970+0000 7f662ee06740 4 client.28789 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:55:27.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:27.970+0000 7f662ee06740 1 -- 172.21.15.5:0/2575443161 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f6cc575950 con 0x55f6cc62df10 2024-09-10T09:55:27.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:27.971+0000 7f6629000640 1 --2- 172.21.15.5:0/2575443161 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55f6cc62df10 0x55f6cc630300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:27.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:27.972+0000 7f6629000640 1 --2- 172.21.15.5:0/2575443161 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55f6cc62df10 0x55f6cc630300 crc :-1 s=READY pgs=245 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=72520c3607f8e71e server_cookie=8b491348a80006df in_seq=0 out_seq=0 2024-09-10T09:55:27.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:27.973+0000 7f660b400640 10 client.28789 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T09:55:27.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:27.973+0000 7f660b400640 1 -- 172.21.15.5:0/2575443161 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f6cc575950 con 0x55f6cc62df10 2024-09-10T09:55:27.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:27.973+0000 7f660b400640 10 client.28789 handle_command_reply: tid=0 2024-09-10T09:55:27.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:27.974+0000 7f662ee06740 1 client.28789 shutdown 2024-09-10T09:55:27.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:27.975+0000 7f662ee06740 1 -- 172.21.15.5:0/2575443161 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f6cc601c70 msgr2=0x55f6cc609d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:27.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:27.975+0000 7f662ee06740 1 --2- 172.21.15.5:0/2575443161 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f6cc601c70 0x55f6cc609d30 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f661c004530 tx=0x7f661c00df90 comp rx=0 tx=0).stop 2024-09-10T09:55:27.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:27.975+0000 7f662ee06740 1 -- 172.21.15.5:0/2575443161 shutdown_connections 2024-09-10T09:55:27.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:27.975+0000 7f662ee06740 1 --2- 172.21.15.5:0/2575443161 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55f6cc62df10 0x55f6cc630300 crc :-1 s=READY pgs=245 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:27.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:27.975+0000 7f662ee06740 1 --2- 172.21.15.5:0/2575443161 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f6cc60ce50 0x55f6cc60a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:27.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:27.975+0000 7f662ee06740 1 --2- 172.21.15.5:0/2575443161 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f6cc602590 0x55f6cc60a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:27.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:27.975+0000 7f662ee06740 1 --2- 172.21.15.5:0/2575443161 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f6cc601c70 0x55f6cc609d30 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:27.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:27.975+0000 7f662ee06740 1 -- 172.21.15.5:0/2575443161 >> 172.21.15.5:0/2575443161 conn(0x55f6cc5fe200 msgr2=0x55f6cc603ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:55:27.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:27.975+0000 7f660b400640 0 client.28789 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T09:55:27.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:27.976+0000 7f662ee06740 1 -- 172.21.15.5:0/2575443161 shutdown_connections 2024-09-10T09:55:27.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:27.976+0000 7f662ee06740 1 -- 172.21.15.5:0/2575443161 wait complete. 2024-09-10T09:55:27.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:27.976+0000 7f662ee06740 20 client.28789 trim_cache size 0 max 16384 2024-09-10T09:55:27.980 INFO:tasks.workunit.client.0.smithi005.stdout: 10 63797 20.38 MB/sec execute 66 sec latency 5575.180 ms 2024-09-10T09:55:28.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:28.001+0000 7f662ee06740 1 Processor -- start 2024-09-10T09:55:28.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:28.002+0000 7f662ee06740 1 -- start start 2024-09-10T09:55:28.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:28.002+0000 7f662ee06740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f6cc601c70 0x55f6cc60a600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:28.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:28.002+0000 7f662ee06740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f6cc602590 0x55f6cc6358d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:28.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:28.002+0000 7f662ee06740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f6cc60ce50 0x55f6cc635e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:28.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:28.002+0000 7f662ee06740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f66241b5440 con 0x55f6cc601c70 2024-09-10T09:55:28.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:28.002+0000 7f662ee06740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f66241b6f80 con 0x55f6cc60ce50 2024-09-10T09:55:28.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:28.002+0000 7f662ee06740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f66241b5b10 con 0x55f6cc602590 2024-09-10T09:55:28.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:28.003+0000 7f6629000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f6cc602590 0x55f6cc6358d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:28.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:28.003+0000 7f6629000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f6cc602590 0x55f6cc6358d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:60968/0 (socket says 172.21.15.5:60968) 2024-09-10T09:55:28.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:28.003+0000 7f6629000640 1 -- 172.21.15.5:0/2423048756 learned_addr learned my addr 172.21.15.5:0/2423048756 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:55:28.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:28.003+0000 7f662a400640 1 --2- 172.21.15.5:0/2423048756 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f6cc60ce50 0x55f6cc635e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:28.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:28.003+0000 7f662a400640 1 --2- 172.21.15.5:0/2423048756 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f6cc601c70 0x55f6cc60a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:28.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:28.003+0000 7f662a400640 1 -- 172.21.15.5:0/2423048756 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f6cc602590 msgr2=0x55f6cc6358d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:55:28.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:28.003+0000 7f662a400640 1 --2- 172.21.15.5:0/2423048756 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f6cc602590 0x55f6cc6358d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:28.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:28.003+0000 7f662a400640 1 -- 172.21.15.5:0/2423048756 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f6cc601c70 msgr2=0x55f6cc60a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:55:28.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:28.003+0000 7f662a400640 1 --2- 172.21.15.5:0/2423048756 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f6cc601c70 0x55f6cc60a600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:28.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:28.003+0000 7f662a400640 1 -- 172.21.15.5:0/2423048756 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f6cc608280 con 0x55f6cc60ce50 2024-09-10T09:55:28.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:28.004+0000 7f662a400640 1 --2- 172.21.15.5:0/2423048756 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f6cc60ce50 0x55f6cc635e10 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f661c021520 tx=0x7f661c005fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2d5a543c553800cf server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:55:28.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:28.004+0000 7f6616000640 1 -- 172.21.15.5:0/2423048756 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f661c028040 con 0x55f6cc60ce50 2024-09-10T09:55:28.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:28.004+0000 7f6616000640 1 -- 172.21.15.5:0/2423048756 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f661c009570 con 0x55f6cc60ce50 2024-09-10T09:55:28.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:28.004+0000 7f6616000640 1 -- 172.21.15.5:0/2423048756 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f661c023c50 con 0x55f6cc60ce50 2024-09-10T09:55:28.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:28.005+0000 7f662ee06740 1 -- 172.21.15.5:0/2423048756 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f6cc60ce50 msgr2=0x55f6cc635e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:28.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:28.005+0000 7f662ee06740 1 --2- 172.21.15.5:0/2423048756 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f6cc60ce50 0x55f6cc635e10 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f661c021520 tx=0x7f661c005fd0 comp rx=0 tx=0).stop 2024-09-10T09:55:28.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:28.005+0000 7f662ee06740 1 -- 172.21.15.5:0/2423048756 shutdown_connections 2024-09-10T09:55:28.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:28.005+0000 7f662ee06740 1 --2- 172.21.15.5:0/2423048756 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f6cc60ce50 0x55f6cc635e10 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:28.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:28.005+0000 7f662ee06740 1 --2- 172.21.15.5:0/2423048756 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f6cc602590 0x55f6cc6358d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:28.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:28.005+0000 7f662ee06740 1 --2- 172.21.15.5:0/2423048756 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f6cc601c70 0x55f6cc60a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:28.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:28.005+0000 7f662ee06740 1 -- 172.21.15.5:0/2423048756 >> 172.21.15.5:0/2423048756 conn(0x55f6cc5fe200 msgr2=0x55f6cc60bdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:55:28.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:28.006+0000 7f662ee06740 1 -- 172.21.15.5:0/2423048756 shutdown_connections 2024-09-10T09:55:28.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:28.006+0000 7f662ee06740 1 -- 172.21.15.5:0/2423048756 wait complete. 2024-09-10T09:55:28.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:28.006+0000 7f662ee06740 1 -- start start 2024-09-10T09:55:28.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:28.007+0000 7f662ee06740 1 Processor -- start 2024-09-10T09:55:28.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:28.008+0000 7f662ee06740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f6cc601c70 0x55f6cc615510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:28.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:28.009+0000 7f662a400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f6cc601c70 0x55f6cc615510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:28.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:28.009+0000 7f662a400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f6cc601c70 0x55f6cc615510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:60976/0 (socket says 172.21.15.5:60976) 2024-09-10T09:55:28.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:28.009+0000 7f662a400640 1 -- 172.21.15.5:0/1465479769 learned_addr learned my addr 172.21.15.5:0/1465479769 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:55:28.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:28.009+0000 7f662ee06740 1 --2- 172.21.15.5:0/1465479769 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f6cc602590 0x55f6cc610840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:28.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:28.010+0000 7f6629000640 1 --2- 172.21.15.5:0/1465479769 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f6cc602590 0x55f6cc610840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:28.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:28.010+0000 7f662ee06740 1 --2- 172.21.15.5:0/1465479769 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f6cc60ce50 0x55f6cc610d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:28.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:28.010+0000 7f662ee06740 1 -- 172.21.15.5:0/1465479769 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f66241b68b0 con 0x55f6cc60ce50 2024-09-10T09:55:28.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:28.010+0000 7f662ee06740 1 -- 172.21.15.5:0/1465479769 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f66241b61e0 con 0x55f6cc602590 2024-09-10T09:55:28.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:28.010+0000 7f662ee06740 1 -- 172.21.15.5:0/1465479769 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f6624072dd0 con 0x55f6cc601c70 2024-09-10T09:55:28.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:28.011+0000 7f662ee06740 10 client.0 _ll_register_callbacks cb 0x7f661c009e90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:55:28.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:28.012+0000 7f6629000640 1 -- 172.21.15.5:0/1465479769 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f6cc601c70 msgr2=0x55f6cc615510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:55:28.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:28.012+0000 7f6629000640 1 --2- 172.21.15.5:0/1465479769 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f6cc601c70 0x55f6cc615510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:28.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:28.012+0000 7f6629000640 1 -- 172.21.15.5:0/1465479769 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f6cc60ce50 msgr2=0x55f6cc610d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:55:28.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:28.012+0000 7f6629000640 1 --2- 172.21.15.5:0/1465479769 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f6cc60ce50 0x55f6cc610d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:28.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:28.012+0000 7f6629000640 1 -- 172.21.15.5:0/1465479769 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f6cc6115b0 con 0x55f6cc602590 2024-09-10T09:55:28.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:28.013+0000 7f6629000640 1 --2- 172.21.15.5:0/1465479769 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f6cc602590 0x55f6cc610840 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f6610045fd0 tx=0x7f6610004540 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1e01647862d114c3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:55:28.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:28.013+0000 7f660b400640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-09-10T09:55:28.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:28.013+0000 7f660b400640 1 -- 172.21.15.5:0/1465479769 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6610035040 con 0x55f6cc602590 2024-09-10T09:55:28.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:28.014+0000 7f662ee06740 1 -- 172.21.15.5:0/1465479769 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f661c009570 con 0x55f6cc602590 2024-09-10T09:55:28.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:28.014+0000 7f660b400640 1 -- 172.21.15.5:0/1465479769 <== mon.2 v2:172.21.15.102:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f6610005510 con 0x55f6cc602590 2024-09-10T09:55:28.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:28.015+0000 7f660b400640 1 -- 172.21.15.5:0/1465479769 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f6610055050 con 0x55f6cc602590 2024-09-10T09:55:28.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:28.015+0000 7f662ee06740 10 client.28796 fetch_fsmap learned FSMap version 42 2024-09-10T09:55:28.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:28.015+0000 7f662ee06740 1 -- 172.21.15.5:0/1465479769 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55f6cc6987c0 con 0x55f6cc602590 2024-09-10T09:55:28.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:28.015+0000 7f660b400640 1 -- 172.21.15.5:0/1465479769 <== mon.2 v2:172.21.15.102:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f661c009570 con 0x55f6cc602590 2024-09-10T09:55:28.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:28.015+0000 7f660b400640 1 -- 172.21.15.5:0/1465479769 <== mon.2 v2:172.21.15.102:3300/0 5 ==== fsmap(e 42) ==== 8370+0+0 (secure 0 0 0) 0x7f661000b040 con 0x55f6cc602590 2024-09-10T09:55:28.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:28.017+0000 7f662ee06740 10 client.28796 fetch_fsmap finished waiting for FSMap version 42 2024-09-10T09:55:28.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:28.017+0000 7f662ee06740 10 client.28796 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T09:55:28.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:28.017+0000 7f662ee06740 20 client.28796 populate_metadata read hostname 'smithi005' 2024-09-10T09:55:28.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:28.017+0000 7f662ee06740 1 --2- 172.21.15.5:0/1465479769 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55f6cc698ef0 0x55f6cc69b310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:28.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:28.017+0000 7f662ee06740 4 client.28796 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T09:55:28.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:28.017+0000 7f662ee06740 1 -- 172.21.15.5:0/1465479769 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f6cc56fe60 con 0x55f6cc698ef0 2024-09-10T09:55:28.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:28.018+0000 7f662a400640 1 --2- 172.21.15.5:0/1465479769 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55f6cc698ef0 0x55f6cc69b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:28.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:28.019+0000 7f662a400640 1 --2- 172.21.15.5:0/1465479769 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55f6cc698ef0 0x55f6cc69b310 crc :-1 s=READY pgs=246 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d7ad5f338fd84d1b server_cookie=9e7b1870cd1951c5 in_seq=0 out_seq=0 2024-09-10T09:55:28.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:28.020+0000 7f660b400640 10 client.28796 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T09:55:28.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:28.020+0000 7f660b400640 1 -- 172.21.15.5:0/1465479769 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f6cc56fe60 con 0x55f6cc698ef0 2024-09-10T09:55:28.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:28.020+0000 7f660b400640 10 client.28796 handle_command_reply: tid=0 2024-09-10T09:55:28.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:28.020+0000 7f662ee06740 1 client.28796 shutdown 2024-09-10T09:55:28.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:28.021+0000 7f662ee06740 1 -- 172.21.15.5:0/1465479769 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f6cc602590 msgr2=0x55f6cc610840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:28.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:28.023+0000 7f662ee06740 1 --2- 172.21.15.5:0/1465479769 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f6cc602590 0x55f6cc610840 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f6610045fd0 tx=0x7f6610004540 comp rx=0 tx=0).stop 2024-09-10T09:55:28.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:28.023+0000 7f662ee06740 1 -- 172.21.15.5:0/1465479769 shutdown_connections 2024-09-10T09:55:28.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:28.023+0000 7f662ee06740 1 --2- 172.21.15.5:0/1465479769 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55f6cc698ef0 0x55f6cc69b310 crc :-1 s=READY pgs=246 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:28.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:28.023+0000 7f662ee06740 1 --2- 172.21.15.5:0/1465479769 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f6cc60ce50 0x55f6cc610d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:28.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:28.023+0000 7f662ee06740 1 --2- 172.21.15.5:0/1465479769 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f6cc602590 0x55f6cc610840 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:28.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:28.023+0000 7f662ee06740 1 --2- 172.21.15.5:0/1465479769 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f6cc601c70 0x55f6cc615510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:28.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:28.023+0000 7f662ee06740 1 -- 172.21.15.5:0/1465479769 >> 172.21.15.5:0/1465479769 conn(0x55f6cc5fe200 msgr2=0x55f6cc60a4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:55:28.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:28.023+0000 7f660b400640 0 client.28796 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T09:55:28.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:28.024+0000 7f662ee06740 1 -- 172.21.15.5:0/1465479769 shutdown_connections 2024-09-10T09:55:28.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:28.024+0000 7f662ee06740 1 -- 172.21.15.5:0/1465479769 wait complete. 2024-09-10T09:55:28.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:28.024+0000 7f662ee06740 20 client.28796 trim_cache size 0 max 16384 2024-09-10T09:55:28.027 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-10T09:55:28.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:28.026+0000 7f662c400640 1 -- 172.21.15.5:0/421242669 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f660c0a2c00 msgr2=0x7f660c0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:28.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:28.026+0000 7f662c400640 1 --2- 172.21.15.5:0/421242669 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f660c0a2c00 0x7f660c0a50c0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f66180ccf60 tx=0x7f6618000bc0 comp rx=0 tx=0).stop 2024-09-10T09:55:28.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:28.026+0000 7f662c400640 1 -- 172.21.15.5:0/421242669 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6624100510 msgr2=0x7f662419cf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:28.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:28.026+0000 7f662c400640 1 --2- 172.21.15.5:0/421242669 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6624100510 0x7f662419cf30 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f661800c920 tx=0x7f661800cde0 comp rx=0 tx=0).stop 2024-09-10T09:55:28.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:28.026+0000 7f662c400640 1 -- 172.21.15.5:0/421242669 shutdown_connections 2024-09-10T09:55:28.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:28.026+0000 7f662c400640 1 --2- 172.21.15.5:0/421242669 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f660c0a2c00 0x7f660c0a50c0 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:28.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:28.026+0000 7f662c400640 1 --2- 172.21.15.5:0/421242669 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f66241051f0 0x7f66241961a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:28.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:28.026+0000 7f662c400640 1 --2- 172.21.15.5:0/421242669 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6624100ed0 0x7f662419d470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:28.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:28.026+0000 7f662c400640 1 --2- 172.21.15.5:0/421242669 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6624100510 0x7f662419cf30 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:28.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:28.026+0000 7f662c400640 1 -- 172.21.15.5:0/421242669 >> 172.21.15.5:0/421242669 conn(0x7f66240fba90 msgr2=0x7f6624101e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:55:28.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:28.027+0000 7f662c400640 1 -- 172.21.15.5:0/421242669 shutdown_connections 2024-09-10T09:55:28.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:28.027+0000 7f662c400640 1 -- 172.21.15.5:0/421242669 wait complete. 2024-09-10T09:55:28.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:55:27 smithi159 ceph-mon[35513]: pgmap v906: 129 pgs: 129 active+clean; 612 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 6.2 MiB/s wr, 368 op/s 2024-09-10T09:55:28.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:55:27 smithi102 ceph-mon[35442]: pgmap v906: 129 pgs: 129 active+clean; 612 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 6.2 MiB/s wr, 368 op/s 2024-09-10T09:55:28.980 INFO:tasks.workunit.client.0.smithi005.stdout: 10 63797 20.07 MB/sec execute 67 sec latency 6575.228 ms 2024-09-10T09:55:29.980 INFO:tasks.workunit.client.0.smithi005.stdout: 10 63797 19.78 MB/sec execute 68 sec latency 7575.289 ms 2024-09-10T09:55:30.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:55:29 smithi005 ceph-mon[30350]: pgmap v907: 129 pgs: 129 active+clean; 610 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 6.3 MiB/s wr, 324 op/s 2024-09-10T09:55:30.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:55:29 smithi159 ceph-mon[35513]: pgmap v907: 129 pgs: 129 active+clean; 610 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 6.3 MiB/s wr, 324 op/s 2024-09-10T09:55:30.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:55:29 smithi102 ceph-mon[35442]: pgmap v907: 129 pgs: 129 active+clean; 610 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 6.3 MiB/s wr, 324 op/s 2024-09-10T09:55:30.980 INFO:tasks.workunit.client.0.smithi005.stdout: 10 63797 19.49 MB/sec execute 69 sec latency 8575.331 ms 2024-09-10T09:55:31.876 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-10T09:55:31.876 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-10T09:55:31.981 INFO:tasks.workunit.client.0.smithi005.stdout: 10 63798 19.21 MB/sec execute 70 sec latency 9566.489 ms 2024-09-10T09:55:32.075 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:55:31 smithi005 ceph-mon[30350]: pgmap v908: 129 pgs: 129 active+clean; 619 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 6.8 MiB/s wr, 312 op/s 2024-09-10T09:55:32.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:55:31 smithi159 ceph-mon[35513]: pgmap v908: 129 pgs: 129 active+clean; 619 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 6.8 MiB/s wr, 312 op/s 2024-09-10T09:55:32.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:55:31 smithi102 ceph-mon[35442]: pgmap v908: 129 pgs: 129 active+clean; 619 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 6.8 MiB/s wr, 312 op/s 2024-09-10T09:55:32.272 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:55:32.981 INFO:tasks.workunit.client.0.smithi005.stdout: 10 63801 18.94 MB/sec execute 71 sec latency 10276.104 ms 2024-09-10T09:55:33.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:33.214+0000 7f30f3000640 1 -- 172.21.15.5:0/12891249 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f30ec074eb0 msgr2=0x7f30ec1124a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:33.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:33.214+0000 7f30f3000640 1 --2- 172.21.15.5:0/12891249 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f30ec074eb0 0x7f30ec1124a0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f30d8009900 tx=0x7f30d802a480 comp rx=0 tx=0).stop 2024-09-10T09:55:33.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:33.214+0000 7f30f3000640 1 -- 172.21.15.5:0/12891249 shutdown_connections 2024-09-10T09:55:33.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:33.214+0000 7f30f3000640 1 --2- 172.21.15.5:0/12891249 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f30ec1129e0 0x7f30ec114dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:33.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:33.214+0000 7f30f3000640 1 --2- 172.21.15.5:0/12891249 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f30ec074eb0 0x7f30ec1124a0 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:33.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:33.214+0000 7f30f3000640 1 --2- 172.21.15.5:0/12891249 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f30ec074590 0x7f30ec074970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:33.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:33.214+0000 7f30f3000640 1 -- 172.21.15.5:0/12891249 >> 172.21.15.5:0/12891249 conn(0x7f30ec0fd710 msgr2=0x7f30ec0ffb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:55:33.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:33.214+0000 7f30f3000640 1 -- 172.21.15.5:0/12891249 shutdown_connections 2024-09-10T09:55:33.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:33.214+0000 7f30f3000640 1 -- 172.21.15.5:0/12891249 wait complete. 2024-09-10T09:55:33.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:33.215+0000 7f30f3000640 1 Processor -- start 2024-09-10T09:55:33.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:33.215+0000 7f30f3000640 1 -- start start 2024-09-10T09:55:33.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:33.216+0000 7f30f3000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f30ec074590 0x7f30ec1a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:33.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:33.216+0000 7f30f3000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f30ec074eb0 0x7f30ec1a4b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:33.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:33.216+0000 7f30f3000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f30ec1129e0 0x7f30ec19e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:33.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:33.216+0000 7f30f3000640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f30ec118400 con 0x7f30ec074eb0 2024-09-10T09:55:33.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:33.216+0000 7f30f3000640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f30ec118280 con 0x7f30ec074590 2024-09-10T09:55:33.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:33.216+0000 7f30f3000640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f30ec118580 con 0x7f30ec1129e0 2024-09-10T09:55:33.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:33.216+0000 7f30f1200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f30ec074eb0 0x7f30ec1a4b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:33.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:33.216+0000 7f30f1c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f30ec074590 0x7f30ec1a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:33.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:33.216+0000 7f30f1c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f30ec074590 0x7f30ec1a4610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:39116/0 (socket says 172.21.15.5:39116) 2024-09-10T09:55:33.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:33.216+0000 7f30f1c00640 1 -- 172.21.15.5:0/3841526162 learned_addr learned my addr 172.21.15.5:0/3841526162 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:55:33.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:33.216+0000 7f30f2600640 1 --2- 172.21.15.5:0/3841526162 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f30ec1129e0 0x7f30ec19e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:33.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:33.216+0000 7f30f1c00640 1 -- 172.21.15.5:0/3841526162 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f30ec1129e0 msgr2=0x7f30ec19e6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:33.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:33.216+0000 7f30f1c00640 1 --2- 172.21.15.5:0/3841526162 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f30ec1129e0 0x7f30ec19e6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:33.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:33.217+0000 7f30f1c00640 1 -- 172.21.15.5:0/3841526162 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f30ec074eb0 msgr2=0x7f30ec1a4b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:33.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:33.217+0000 7f30f1c00640 1 --2- 172.21.15.5:0/3841526162 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f30ec074eb0 0x7f30ec1a4b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:33.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:33.217+0000 7f30f1c00640 1 -- 172.21.15.5:0/3841526162 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30ec19ef50 con 0x7f30ec074590 2024-09-10T09:55:33.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:33.217+0000 7f30f1c00640 1 --2- 172.21.15.5:0/3841526162 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f30ec074590 0x7f30ec1a4610 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f30e000e9f0 tx=0x7f30e000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:55:33.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:33.217+0000 7f30ea000640 1 -- 172.21.15.5:0/3841526162 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30e000cd80 con 0x7f30ec074590 2024-09-10T09:55:33.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:33.217+0000 7f30ea000640 1 -- 172.21.15.5:0/3841526162 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f30e0010910 con 0x7f30ec074590 2024-09-10T09:55:33.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:33.217+0000 7f30ea000640 1 -- 172.21.15.5:0/3841526162 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30e0018990 con 0x7f30ec074590 2024-09-10T09:55:33.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:33.217+0000 7f30f3000640 1 -- 172.21.15.5:0/3841526162 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30ec19f240 con 0x7f30ec074590 2024-09-10T09:55:33.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:33.217+0000 7f30f3000640 1 -- 172.21.15.5:0/3841526162 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30ec1ab3c0 con 0x7f30ec074590 2024-09-10T09:55:33.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:33.220+0000 7f30f70b8740 1 Processor -- start 2024-09-10T09:55:33.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:33.221+0000 7f30f70b8740 1 -- start start 2024-09-10T09:55:33.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:33.221+0000 7f30f70b8740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555585b84cb0 0x555585b85090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:33.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:33.221+0000 7f30f70b8740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555585b855d0 0x555585b8f950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:33.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:33.221+0000 7f30f70b8740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555585b8fe90 0x555585b92280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:33.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:33.221+0000 7f30f70b8740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x555585aeb290 con 0x555585b8fe90 2024-09-10T09:55:33.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:33.221+0000 7f30f70b8740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x555585a92b40 con 0x555585b84cb0 2024-09-10T09:55:33.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:33.221+0000 7f30f70b8740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x555585ae7250 con 0x555585b855d0 2024-09-10T09:55:33.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:33.221+0000 7f30ea000640 1 -- 172.21.15.5:0/3841526162 <== mon.2 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f30e0010ac0 con 0x7f30ec074590 2024-09-10T09:55:33.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:33.221+0000 7f30f2600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555585b84cb0 0x555585b85090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:33.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:33.221+0000 7f30f2600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555585b84cb0 0x555585b85090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:39122/0 (socket says 172.21.15.5:39122) 2024-09-10T09:55:33.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:33.221+0000 7f30f2600640 1 -- 172.21.15.5:0/3154992337 learned_addr learned my addr 172.21.15.5:0/3154992337 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:55:33.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:33.221+0000 7f30f1200640 1 --2- 172.21.15.5:0/3154992337 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555585b8fe90 0x555585b92280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:33.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:33.222+0000 7f30f1c00640 1 --2- 172.21.15.5:0/3154992337 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555585b855d0 0x555585b8f950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:33.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:33.222+0000 7f30f2600640 1 -- 172.21.15.5:0/3154992337 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555585b855d0 msgr2=0x555585b8f950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:55:33.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:33.222+0000 7f30f2600640 1 --2- 172.21.15.5:0/3154992337 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555585b855d0 0x555585b8f950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:33.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:33.222+0000 7f30f2600640 1 -- 172.21.15.5:0/3154992337 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555585b8fe90 msgr2=0x555585b92280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:55:33.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:33.222+0000 7f30f2600640 1 --2- 172.21.15.5:0/3154992337 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555585b8fe90 0x555585b92280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:33.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:33.222+0000 7f30f2600640 1 -- 172.21.15.5:0/3154992337 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55558585c6a0 con 0x555585b84cb0 2024-09-10T09:55:33.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:33.222+0000 7f30f1200640 1 --2- 172.21.15.5:0/3154992337 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555585b8fe90 0x555585b92280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:55:33.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:33.222+0000 7f30f2600640 1 --2- 172.21.15.5:0/3154992337 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555585b84cb0 0x555585b85090 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f30e4009f00 tx=0x7f30e40107a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=62c11fcfdf1c0e05 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:55:33.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:33.223+0000 7f30dea00640 1 -- 172.21.15.5:0/3154992337 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30e4003b70 con 0x555585b84cb0 2024-09-10T09:55:33.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:33.223+0000 7f30dea00640 1 -- 172.21.15.5:0/3154992337 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f30e4003d10 con 0x555585b84cb0 2024-09-10T09:55:33.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:33.223+0000 7f30dea00640 1 -- 172.21.15.5:0/3154992337 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30e4011a40 con 0x555585b84cb0 2024-09-10T09:55:33.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:33.223+0000 7f30ea000640 1 --2- 172.21.15.5:0/3841526162 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f30d40a2c00 0x7f30d40a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:33.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:33.223+0000 7f30f70b8740 1 -- 172.21.15.5:0/3154992337 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555585b84cb0 msgr2=0x555585b85090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:33.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:33.223+0000 7f30f70b8740 1 --2- 172.21.15.5:0/3154992337 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555585b84cb0 0x555585b85090 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f30e4009f00 tx=0x7f30e40107a0 comp rx=0 tx=0).stop 2024-09-10T09:55:33.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:33.223+0000 7f30ea000640 1 -- 172.21.15.5:0/3841526162 <== mon.2 v2:172.21.15.102:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f30e00cf7b0 con 0x7f30ec074590 2024-09-10T09:55:33.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:33.224+0000 7f30f1c00640 1 --2- 172.21.15.5:0/3841526162 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f30d40a2c00 0x7f30d40a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:33.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:33.224+0000 7f30f70b8740 1 -- 172.21.15.5:0/3154992337 shutdown_connections 2024-09-10T09:55:33.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:33.224+0000 7f30f70b8740 1 --2- 172.21.15.5:0/3154992337 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555585b8fe90 0x555585b92280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:33.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:33.224+0000 7f30f70b8740 1 --2- 172.21.15.5:0/3154992337 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555585b855d0 0x555585b8f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:33.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:33.224+0000 7f30f70b8740 1 --2- 172.21.15.5:0/3154992337 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555585b84cb0 0x555585b85090 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:33.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:33.224+0000 7f30f70b8740 1 -- 172.21.15.5:0/3154992337 >> 172.21.15.5:0/3154992337 conn(0x555585b81240 msgr2=0x555585b81620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:55:33.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:33.224+0000 7f30f70b8740 1 -- 172.21.15.5:0/3154992337 shutdown_connections 2024-09-10T09:55:33.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:33.224+0000 7f30f70b8740 1 -- 172.21.15.5:0/3154992337 wait complete. 2024-09-10T09:55:33.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:33.225+0000 7f30f1c00640 1 --2- 172.21.15.5:0/3841526162 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f30d40a2c00 0x7f30d40a50c0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f30e0098020 tx=0x7f30e0000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:55:33.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:33.225+0000 7f30f70b8740 1 -- start start 2024-09-10T09:55:33.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:33.225+0000 7f30f70b8740 1 Processor -- start 2024-09-10T09:55:33.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:33.226+0000 7f30f70b8740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555585b84cb0 0x555585b8f740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:33.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:33.226+0000 7f30f70b8740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555585b855d0 0x555585b8c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:33.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:33.226+0000 7f30f70b8740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555585b8fe90 0x555585b8cf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:33.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:33.226+0000 7f30f70b8740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x555585b92b00 con 0x555585b8fe90 2024-09-10T09:55:33.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:33.226+0000 7f30f70b8740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x555585aa5fa0 con 0x555585b855d0 2024-09-10T09:55:33.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:33.226+0000 7f30f70b8740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x555585b92c80 con 0x555585b84cb0 2024-09-10T09:55:33.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:33.226+0000 7f30f70b8740 10 client.0 _ll_register_callbacks cb 0x5555857f9ba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:55:33.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:33.226+0000 7f30f2600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555585b8fe90 0x555585b8cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:33.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:33.226+0000 7f30f2600640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555585b84cb0 0x555585b8f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:33.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:33.226+0000 7f30f2600640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555585b84cb0 0x555585b8f740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:39030/0 (socket says 172.21.15.5:39030) 2024-09-10T09:55:33.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:33.226+0000 7f30f2600640 1 -- 172.21.15.5:0/2169145989 learned_addr learned my addr 172.21.15.5:0/2169145989 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:55:33.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:33.227+0000 7f30f1200640 1 --2- 172.21.15.5:0/2169145989 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555585b855d0 0x555585b8c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:33.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:33.227+0000 7f30f2600640 1 -- 172.21.15.5:0/2169145989 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555585b855d0 msgr2=0x555585b8c9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:55:33.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:33.227+0000 7f30f2600640 1 --2- 172.21.15.5:0/2169145989 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555585b855d0 0x555585b8c9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:33.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:33.227+0000 7f30f2600640 1 -- 172.21.15.5:0/2169145989 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555585b8fe90 msgr2=0x555585b8cf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:55:33.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:33.227+0000 7f30f2600640 1 --2- 172.21.15.5:0/2169145989 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555585b8fe90 0x555585b8cf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:33.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:33.227+0000 7f30f2600640 1 -- 172.21.15.5:0/2169145989 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555585b8d730 con 0x555585b84cb0 2024-09-10T09:55:33.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:33.227+0000 7f30f2600640 1 --2- 172.21.15.5:0/2169145989 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555585b84cb0 0x555585b8f740 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f30e4009ed0 tx=0x7f30e4010fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=617af1c79744a667 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:55:33.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:33.227+0000 7f30d3400640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-09-10T09:55:33.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:33.227+0000 7f30d3400640 1 -- 172.21.15.5:0/2169145989 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30e4025040 con 0x555585b84cb0 2024-09-10T09:55:33.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:33.227+0000 7f30f70b8740 1 -- 172.21.15.5:0/2169145989 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f30e4003d10 con 0x555585b84cb0 2024-09-10T09:55:33.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:33.227+0000 7f30d3400640 1 -- 172.21.15.5:0/2169145989 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f30e40117e0 con 0x555585b84cb0 2024-09-10T09:55:33.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:33.228+0000 7f30d3400640 1 -- 172.21.15.5:0/2169145989 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f30e40119e0 con 0x555585b84cb0 2024-09-10T09:55:33.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:33.228+0000 7f30f70b8740 10 client.28807 fetch_fsmap learned FSMap version 42 2024-09-10T09:55:33.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:33.228+0000 7f30f70b8740 1 -- 172.21.15.5:0/2169145989 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x555585b9e9e0 con 0x555585b84cb0 2024-09-10T09:55:33.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:33.228+0000 7f30d3400640 1 -- 172.21.15.5:0/2169145989 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f30e4003d10 con 0x555585b84cb0 2024-09-10T09:55:33.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:33.228+0000 7f30d3400640 1 -- 172.21.15.5:0/2169145989 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 42) ==== 8370+0+0 (secure 0 0 0) 0x7f30e402cad0 con 0x555585b84cb0 2024-09-10T09:55:33.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:33.228+0000 7f30f70b8740 10 client.28807 fetch_fsmap finished waiting for FSMap version 42 2024-09-10T09:55:33.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:33.228+0000 7f30f70b8740 10 client.28807 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T09:55:33.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:33.228+0000 7f30f70b8740 20 client.28807 populate_metadata read hostname 'smithi005' 2024-09-10T09:55:33.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:33.228+0000 7f30f70b8740 1 --2- 172.21.15.5:0/2169145989 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x555585b9fdf0 0x555585ba21e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:33.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:33.228+0000 7f30f70b8740 4 client.28807 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:55:33.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:33.228+0000 7f30f70b8740 1 -- 172.21.15.5:0/2169145989 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555585af8800 con 0x555585b9fdf0 2024-09-10T09:55:33.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:33.229+0000 7f30f1200640 1 --2- 172.21.15.5:0/2169145989 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x555585b9fdf0 0x555585ba21e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:33.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:33.229+0000 7f30f1200640 1 --2- 172.21.15.5:0/2169145989 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x555585b9fdf0 0x555585ba21e0 crc :-1 s=READY pgs=247 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=498a8cfb28328396 server_cookie=adcd3d12c396de50 in_seq=0 out_seq=0 2024-09-10T09:55:33.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:33.229+0000 7f30d3400640 10 client.28807 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T09:55:33.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:33.230+0000 7f30d3400640 1 -- 172.21.15.5:0/2169145989 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555585af8800 con 0x555585b9fdf0 2024-09-10T09:55:33.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:33.230+0000 7f30d3400640 10 client.28807 handle_command_reply: tid=0 2024-09-10T09:55:33.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:33.230+0000 7f30f70b8740 1 client.28807 shutdown 2024-09-10T09:55:33.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:33.231+0000 7f30f70b8740 1 -- 172.21.15.5:0/2169145989 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555585b84cb0 msgr2=0x555585b8f740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:33.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:33.231+0000 7f30f70b8740 1 --2- 172.21.15.5:0/2169145989 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555585b84cb0 0x555585b8f740 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f30e4009ed0 tx=0x7f30e4010fd0 comp rx=0 tx=0).stop 2024-09-10T09:55:33.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:33.231+0000 7f30f70b8740 1 -- 172.21.15.5:0/2169145989 shutdown_connections 2024-09-10T09:55:33.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:33.231+0000 7f30f70b8740 1 --2- 172.21.15.5:0/2169145989 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x555585b9fdf0 0x555585ba21e0 crc :-1 s=READY pgs=247 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:33.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:33.231+0000 7f30f70b8740 1 --2- 172.21.15.5:0/2169145989 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555585b8fe90 0x555585b8cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:33.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:33.231+0000 7f30f70b8740 1 --2- 172.21.15.5:0/2169145989 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555585b855d0 0x555585b8c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:33.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:33.231+0000 7f30d3400640 0 client.28807 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T09:55:33.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:33.231+0000 7f30f70b8740 1 --2- 172.21.15.5:0/2169145989 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555585b84cb0 0x555585b8f740 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:33.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:33.231+0000 7f30f70b8740 1 -- 172.21.15.5:0/2169145989 >> 172.21.15.5:0/2169145989 conn(0x555585b81240 msgr2=0x555585b86f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:55:33.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:33.231+0000 7f30f70b8740 1 -- 172.21.15.5:0/2169145989 shutdown_connections 2024-09-10T09:55:33.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:33.231+0000 7f30f70b8740 1 -- 172.21.15.5:0/2169145989 wait complete. 2024-09-10T09:55:33.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:33.231+0000 7f30f70b8740 20 client.28807 trim_cache size 0 max 16384 2024-09-10T09:55:33.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:33.249+0000 7f30f70b8740 1 Processor -- start 2024-09-10T09:55:33.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:33.249+0000 7f30f70b8740 1 -- start start 2024-09-10T09:55:33.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:33.250+0000 7f30f70b8740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555585b81240 0x555585ba7900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:33.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:33.250+0000 7f30f70b8740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555585b84cb0 0x555585ba7e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:33.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:33.250+0000 7f30f70b8740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555585b855d0 0x555585b89c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:33.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:33.250+0000 7f30f70b8740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f30ec1b5340 con 0x555585b81240 2024-09-10T09:55:33.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:33.250+0000 7f30f70b8740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f30ec1b6e80 con 0x555585b84cb0 2024-09-10T09:55:33.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:33.250+0000 7f30f70b8740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f30ec1b5a10 con 0x555585b855d0 2024-09-10T09:55:33.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:33.250+0000 7f30f1200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555585b84cb0 0x555585ba7e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:33.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:33.250+0000 7f30f2600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555585b81240 0x555585ba7900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:33.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:33.250+0000 7f30f1200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555585b84cb0 0x555585ba7e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:39148/0 (socket says 172.21.15.5:39148) 2024-09-10T09:55:33.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:33.250+0000 7f30f1200640 1 -- 172.21.15.5:0/2387376998 learned_addr learned my addr 172.21.15.5:0/2387376998 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:55:33.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:33.250+0000 7f30f1200640 1 -- 172.21.15.5:0/2387376998 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555585b855d0 msgr2=0x555585b89c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:55:33.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:33.250+0000 7f30f1200640 1 --2- 172.21.15.5:0/2387376998 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555585b855d0 0x555585b89c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:33.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:33.250+0000 7f30f1200640 1 -- 172.21.15.5:0/2387376998 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555585b81240 msgr2=0x555585ba7900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:55:33.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:33.250+0000 7f30f1200640 1 --2- 172.21.15.5:0/2387376998 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555585b81240 0x555585ba7900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:33.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:33.250+0000 7f30f1200640 1 -- 172.21.15.5:0/2387376998 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555585b8af40 con 0x555585b84cb0 2024-09-10T09:55:33.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:33.251+0000 7f30f1200640 1 --2- 172.21.15.5:0/2387376998 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555585b84cb0 0x555585ba7e40 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f30d8009230 tx=0x7f30d802ac00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=64aa765b53d876ce server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:55:33.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:33.251+0000 7f30de000640 1 -- 172.21.15.5:0/2387376998 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30d8035040 con 0x555585b84cb0 2024-09-10T09:55:33.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:33.251+0000 7f30de000640 1 -- 172.21.15.5:0/2387376998 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f30d8005520 con 0x555585b84cb0 2024-09-10T09:55:33.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:33.251+0000 7f30de000640 1 -- 172.21.15.5:0/2387376998 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30d8004e00 con 0x555585b84cb0 2024-09-10T09:55:33.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:33.252+0000 7f30f70b8740 1 -- 172.21.15.5:0/2387376998 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555585b84cb0 msgr2=0x555585ba7e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:33.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:33.252+0000 7f30f70b8740 1 --2- 172.21.15.5:0/2387376998 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555585b84cb0 0x555585ba7e40 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f30d8009230 tx=0x7f30d802ac00 comp rx=0 tx=0).stop 2024-09-10T09:55:33.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:33.252+0000 7f30f70b8740 1 -- 172.21.15.5:0/2387376998 shutdown_connections 2024-09-10T09:55:33.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:33.252+0000 7f30f70b8740 1 --2- 172.21.15.5:0/2387376998 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555585b855d0 0x555585b89c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:33.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:33.252+0000 7f30f70b8740 1 --2- 172.21.15.5:0/2387376998 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555585b84cb0 0x555585ba7e40 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:33.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:33.252+0000 7f30f70b8740 1 --2- 172.21.15.5:0/2387376998 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555585b81240 0x555585ba7900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:33.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:33.252+0000 7f30f70b8740 1 -- 172.21.15.5:0/2387376998 >> 172.21.15.5:0/2387376998 conn(0x555585b8d090 msgr2=0x555585b86f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:55:33.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:33.252+0000 7f30f70b8740 1 -- 172.21.15.5:0/2387376998 shutdown_connections 2024-09-10T09:55:33.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:33.252+0000 7f30f70b8740 1 -- 172.21.15.5:0/2387376998 wait complete. 2024-09-10T09:55:33.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:33.252+0000 7f30f70b8740 1 -- start start 2024-09-10T09:55:33.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:33.253+0000 7f30f70b8740 1 Processor -- start 2024-09-10T09:55:33.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:33.253+0000 7f30f70b8740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555585b81240 0x555585b93cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:33.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:33.253+0000 7f30f70b8740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555585b84cb0 0x555585b94210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:33.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:33.253+0000 7f30f70b8740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555585b855d0 0x555585b94750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:33.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:33.253+0000 7f30f70b8740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f30ec1b67b0 con 0x555585b855d0 2024-09-10T09:55:33.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:33.253+0000 7f30f70b8740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f30ec1b60e0 con 0x555585b84cb0 2024-09-10T09:55:33.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:33.253+0000 7f30f70b8740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f30ec1b3320 con 0x555585b81240 2024-09-10T09:55:33.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:33.253+0000 7f30f70b8740 10 client.0 _ll_register_callbacks cb 0x7f30e4009dd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:55:33.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:33.253+0000 7f30f1200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555585b84cb0 0x555585b94210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:33.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:33.253+0000 7f30f1200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555585b84cb0 0x555585b94210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:39150/0 (socket says 172.21.15.5:39150) 2024-09-10T09:55:33.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:33.253+0000 7f30f1200640 1 -- 172.21.15.5:0/335967775 learned_addr learned my addr 172.21.15.5:0/335967775 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:55:33.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:33.253+0000 7f30f2600640 1 --2- 172.21.15.5:0/335967775 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555585b855d0 0x555585b94750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:33.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:33.254+0000 7f30f2600640 1 --2- 172.21.15.5:0/335967775 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555585b81240 0x555585b93cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:33.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:33.254+0000 7f30f1200640 1 -- 172.21.15.5:0/335967775 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555585b81240 msgr2=0x555585b93cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:55:33.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:33.254+0000 7f30f1200640 1 --2- 172.21.15.5:0/335967775 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555585b81240 0x555585b93cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:33.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:33.254+0000 7f30f1200640 1 -- 172.21.15.5:0/335967775 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555585b855d0 msgr2=0x555585b94750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:55:33.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:33.254+0000 7f30f1200640 1 --2- 172.21.15.5:0/335967775 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555585b855d0 0x555585b94750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:33.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:33.254+0000 7f30f1200640 1 -- 172.21.15.5:0/335967775 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555585c09980 con 0x555585b84cb0 2024-09-10T09:55:33.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:33.254+0000 7f30f2600640 1 --2- 172.21.15.5:0/335967775 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555585b855d0 0x555585b94750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:55:33.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:33.254+0000 7f30f1200640 1 --2- 172.21.15.5:0/335967775 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555585b84cb0 0x555585b94210 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f30d8004ab0 tx=0x7f30d8009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2e44e29e9fe15013 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:55:33.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:33.254+0000 7f30d3400640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-09-10T09:55:33.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:33.254+0000 7f30d3400640 1 -- 172.21.15.5:0/335967775 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30d8050070 con 0x555585b84cb0 2024-09-10T09:55:33.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:33.254+0000 7f30f70b8740 1 -- 172.21.15.5:0/335967775 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f30d8005520 con 0x555585b84cb0 2024-09-10T09:55:33.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:33.254+0000 7f30d3400640 1 -- 172.21.15.5:0/335967775 <== mon.2 v2:172.21.15.102:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f30d8004360 con 0x555585b84cb0 2024-09-10T09:55:33.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:33.255+0000 7f30d3400640 1 -- 172.21.15.5:0/335967775 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f30d8035040 con 0x555585b84cb0 2024-09-10T09:55:33.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:33.255+0000 7f30f70b8740 10 client.28811 fetch_fsmap learned FSMap version 42 2024-09-10T09:55:33.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:33.255+0000 7f30f70b8740 1 -- 172.21.15.5:0/335967775 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x555585c0aca0 con 0x555585b84cb0 2024-09-10T09:55:33.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:33.255+0000 7f30d3400640 1 -- 172.21.15.5:0/335967775 <== mon.2 v2:172.21.15.102:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f30d8005520 con 0x555585b84cb0 2024-09-10T09:55:33.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:33.255+0000 7f30d3400640 1 -- 172.21.15.5:0/335967775 <== mon.2 v2:172.21.15.102:3300/0 5 ==== fsmap(e 42) ==== 8370+0+0 (secure 0 0 0) 0x7f30d80318b0 con 0x555585b84cb0 2024-09-10T09:55:33.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:33.255+0000 7f30f70b8740 10 client.28811 fetch_fsmap finished waiting for FSMap version 42 2024-09-10T09:55:33.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:33.255+0000 7f30f70b8740 10 client.28811 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T09:55:33.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:33.255+0000 7f30f70b8740 20 client.28811 populate_metadata read hostname 'smithi005' 2024-09-10T09:55:33.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:33.255+0000 7f30f70b8740 1 --2- 172.21.15.5:0/335967775 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x555585b8fe90 0x555585c0d470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:33.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:33.255+0000 7f30f70b8740 4 client.28811 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-10T09:55:33.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:33.255+0000 7f30f70b8740 1 -- 172.21.15.5:0/335967775 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x555585af2d10 con 0x555585b8fe90 2024-09-10T09:55:33.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:33.256+0000 7f30f2600640 1 --2- 172.21.15.5:0/335967775 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x555585b8fe90 0x555585c0d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:33.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:33.257+0000 7f30f2600640 1 --2- 172.21.15.5:0/335967775 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x555585b8fe90 0x555585c0d470 crc :-1 s=READY pgs=248 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=320707140ea7fe65 server_cookie=d66cb9fd9e9de836 in_seq=0 out_seq=0 2024-09-10T09:55:33.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:33.257+0000 7f30d3400640 10 client.28811 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T09:55:33.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:33.257+0000 7f30d3400640 1 -- 172.21.15.5:0/335967775 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x555585af2d10 con 0x555585b8fe90 2024-09-10T09:55:33.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:33.257+0000 7f30d3400640 10 client.28811 handle_command_reply: tid=0 2024-09-10T09:55:33.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:33.257+0000 7f30f70b8740 1 client.28811 shutdown 2024-09-10T09:55:33.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:33.258+0000 7f30f70b8740 1 -- 172.21.15.5:0/335967775 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555585b84cb0 msgr2=0x555585b94210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:33.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:33.258+0000 7f30f70b8740 1 --2- 172.21.15.5:0/335967775 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555585b84cb0 0x555585b94210 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f30d8004ab0 tx=0x7f30d8009260 comp rx=0 tx=0).stop 2024-09-10T09:55:33.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:33.258+0000 7f30f70b8740 1 -- 172.21.15.5:0/335967775 shutdown_connections 2024-09-10T09:55:33.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:33.258+0000 7f30f70b8740 1 --2- 172.21.15.5:0/335967775 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x555585b8fe90 0x555585c0d470 crc :-1 s=READY pgs=248 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:33.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:33.258+0000 7f30f70b8740 1 --2- 172.21.15.5:0/335967775 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555585b855d0 0x555585b94750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:33.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:33.258+0000 7f30f70b8740 1 --2- 172.21.15.5:0/335967775 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555585b84cb0 0x555585b94210 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:33.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:33.258+0000 7f30f70b8740 1 --2- 172.21.15.5:0/335967775 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555585b81240 0x555585b93cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:33.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:33.258+0000 7f30f70b8740 1 -- 172.21.15.5:0/335967775 >> 172.21.15.5:0/335967775 conn(0x555585b8d090 msgr2=0x555585b87870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:55:33.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:33.258+0000 7f30d3400640 0 client.28811 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T09:55:33.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:33.258+0000 7f30f70b8740 1 -- 172.21.15.5:0/335967775 shutdown_connections 2024-09-10T09:55:33.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:33.258+0000 7f30f70b8740 1 -- 172.21.15.5:0/335967775 wait complete. 2024-09-10T09:55:33.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:33.258+0000 7f30f70b8740 20 client.28811 trim_cache size 0 max 16384 2024-09-10T09:55:33.350 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-10T09:55:33.350 INFO:teuthology.orchestra.run.smithi005.stdout: "return_code": 0, 2024-09-10T09:55:33.350 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_tag": "8f3269bd-aa14-4ff3-9f8e-6f800d5b9677", 2024-09-10T09:55:33.350 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": "asynchronous" 2024-09-10T09:55:33.350 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-10T09:55:33.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:33.259+0000 7f30f3000640 1 -- 172.21.15.5:0/3841526162 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f30d40a2c00 msgr2=0x7f30d40a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:33.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:33.259+0000 7f30f3000640 1 --2- 172.21.15.5:0/3841526162 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f30d40a2c00 0x7f30d40a50c0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f30e0098020 tx=0x7f30e0000bc0 comp rx=0 tx=0).stop 2024-09-10T09:55:33.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:33.259+0000 7f30f3000640 1 -- 172.21.15.5:0/3841526162 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f30ec074590 msgr2=0x7f30ec1a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:33.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:33.259+0000 7f30f3000640 1 --2- 172.21.15.5:0/3841526162 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f30ec074590 0x7f30ec1a4610 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f30e000e9f0 tx=0x7f30e000eeb0 comp rx=0 tx=0).stop 2024-09-10T09:55:33.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:33.259+0000 7f30f3000640 1 -- 172.21.15.5:0/3841526162 shutdown_connections 2024-09-10T09:55:33.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:33.259+0000 7f30f3000640 1 --2- 172.21.15.5:0/3841526162 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f30d40a2c00 0x7f30d40a50c0 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:33.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:33.259+0000 7f30f3000640 1 --2- 172.21.15.5:0/3841526162 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f30ec1129e0 0x7f30ec19e6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:33.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:33.259+0000 7f30f3000640 1 --2- 172.21.15.5:0/3841526162 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f30ec074eb0 0x7f30ec1a4b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:33.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:33.259+0000 7f30f3000640 1 --2- 172.21.15.5:0/3841526162 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f30ec074590 0x7f30ec1a4610 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:33.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:33.259+0000 7f30f3000640 1 -- 172.21.15.5:0/3841526162 >> 172.21.15.5:0/3841526162 conn(0x7f30ec0fd710 msgr2=0x7f30ec0feea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:55:33.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:33.259+0000 7f30f3000640 1 -- 172.21.15.5:0/3841526162 shutdown_connections 2024-09-10T09:55:33.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:33.259+0000 7f30f3000640 1 -- 172.21.15.5:0/3841526162 wait complete. 2024-09-10T09:55:33.981 INFO:tasks.workunit.client.0.smithi005.stdout: 10 63801 18.68 MB/sec execute 72 sec latency 11276.155 ms 2024-09-10T09:55:34.080 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:55:33 smithi005 ceph-mon[30350]: pgmap v909: 129 pgs: 129 active+clean; 619 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 459 KiB/s rd, 5.5 MiB/s wr, 214 op/s 2024-09-10T09:55:34.080 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:55:33 smithi005 ceph-mon[30350]: scrub summary: active paths [/] 2024-09-10T09:55:34.080 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:55:33 smithi005 ceph-mon[30350]: scrub summary: idle+waiting paths [/] 2024-09-10T09:55:34.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:55:33 smithi159 ceph-mon[35513]: pgmap v909: 129 pgs: 129 active+clean; 619 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 459 KiB/s rd, 5.5 MiB/s wr, 214 op/s 2024-09-10T09:55:34.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:55:33 smithi159 ceph-mon[35513]: scrub summary: active paths [/] 2024-09-10T09:55:34.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:55:33 smithi159 ceph-mon[35513]: scrub summary: idle+waiting paths [/] 2024-09-10T09:55:34.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:55:33 smithi102 ceph-mon[35442]: pgmap v909: 129 pgs: 129 active+clean; 619 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 459 KiB/s rd, 5.5 MiB/s wr, 214 op/s 2024-09-10T09:55:34.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:55:33 smithi102 ceph-mon[35442]: scrub summary: active paths [/] 2024-09-10T09:55:34.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:55:33 smithi102 ceph-mon[35442]: scrub summary: idle+waiting paths [/] 2024-09-10T09:55:34.951 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:55:34 smithi005 ceph-mon[30350]: scrub queued for path: / 2024-09-10T09:55:34.951 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:55:34 smithi005 ceph-mon[30350]: scrub summary: idle+waiting paths [/] 2024-09-10T09:55:34.952 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:55:34 smithi005 ceph-mon[30350]: scrub summary: active paths [/] 2024-09-10T09:55:34.952 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:55:34 smithi005 ceph-mon[30350]: scrub summary: active paths [/] 2024-09-10T09:55:34.952 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:55:34 smithi005 ceph-mon[30350]: scrub summary: active paths [/] 2024-09-10T09:55:34.952 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:55:34 smithi005 ceph-mon[30350]: scrub summary: active paths [/] 2024-09-10T09:55:34.952 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:55:34 smithi005 ceph-mon[30350]: scrub summary: idle+waiting paths [/] 2024-09-10T09:55:34.952 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:55:34 smithi005 ceph-mon[30350]: scrub summary: idle+waiting paths [/] 2024-09-10T09:55:34.980 INFO:tasks.workunit.client.0.smithi005.stdout: 10 63801 18.42 MB/sec execute 73 sec latency 12276.198 ms 2024-09-10T09:55:35.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:55:34 smithi159 ceph-mon[35513]: scrub queued for path: / 2024-09-10T09:55:35.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:55:34 smithi159 ceph-mon[35513]: scrub summary: idle+waiting paths [/] 2024-09-10T09:55:35.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:55:34 smithi159 ceph-mon[35513]: scrub summary: active paths [/] 2024-09-10T09:55:35.101 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:55:34 smithi159 ceph-mon[35513]: scrub summary: active paths [/] 2024-09-10T09:55:35.101 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:55:34 smithi159 ceph-mon[35513]: scrub summary: active paths [/] 2024-09-10T09:55:35.101 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:55:34 smithi159 ceph-mon[35513]: scrub summary: active paths [/] 2024-09-10T09:55:35.101 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:55:34 smithi159 ceph-mon[35513]: scrub summary: idle+waiting paths [/] 2024-09-10T09:55:35.101 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:55:34 smithi159 ceph-mon[35513]: scrub summary: idle+waiting paths [/] 2024-09-10T09:55:35.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:55:34 smithi102 ceph-mon[35442]: scrub queued for path: / 2024-09-10T09:55:35.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:55:34 smithi102 ceph-mon[35442]: scrub summary: idle+waiting paths [/] 2024-09-10T09:55:35.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:55:34 smithi102 ceph-mon[35442]: scrub summary: active paths [/] 2024-09-10T09:55:35.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:55:34 smithi102 ceph-mon[35442]: scrub summary: active paths [/] 2024-09-10T09:55:35.135 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:55:34 smithi102 ceph-mon[35442]: scrub summary: active paths [/] 2024-09-10T09:55:35.135 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:55:34 smithi102 ceph-mon[35442]: scrub summary: active paths [/] 2024-09-10T09:55:35.135 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:55:34 smithi102 ceph-mon[35442]: scrub summary: idle+waiting paths [/] 2024-09-10T09:55:35.135 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:55:34 smithi102 ceph-mon[35442]: scrub summary: idle+waiting paths [/] 2024-09-10T09:55:35.308 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T09:55:35.690 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:55:35.981 INFO:tasks.workunit.client.0.smithi005.stdout: 10 63801 18.17 MB/sec execute 74 sec latency 13276.237 ms 2024-09-10T09:55:36.072 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:55:35 smithi005 ceph-mon[30350]: pgmap v910: 129 pgs: 129 active+clean; 617 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 258 KiB/s rd, 5.5 MiB/s wr, 196 op/s 2024-09-10T09:55:36.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:55:35 smithi159 ceph-mon[35513]: pgmap v910: 129 pgs: 129 active+clean; 617 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 258 KiB/s rd, 5.5 MiB/s wr, 196 op/s 2024-09-10T09:55:36.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:55:35 smithi102 ceph-mon[35442]: pgmap v910: 129 pgs: 129 active+clean; 617 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 258 KiB/s rd, 5.5 MiB/s wr, 196 op/s 2024-09-10T09:55:36.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:36.412+0000 7fd44c600640 1 -- 172.21.15.5:0/2492219363 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd4441049a0 msgr2=0x7fd4441089f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:36.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:36.412+0000 7fd44c600640 1 --2- 172.21.15.5:0/2492219363 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd4441049a0 0x7fd4441089f0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fd430009900 tx=0x7fd43002a3f0 comp rx=0 tx=0).stop 2024-09-10T09:55:36.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:36.412+0000 7fd44c600640 1 -- 172.21.15.5:0/2492219363 shutdown_connections 2024-09-10T09:55:36.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:36.412+0000 7fd44c600640 1 --2- 172.21.15.5:0/2492219363 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd444109120 0x7fd444110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:36.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:36.412+0000 7fd44c600640 1 --2- 172.21.15.5:0/2492219363 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd4441049a0 0x7fd4441089f0 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:36.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:36.412+0000 7fd44c600640 1 --2- 172.21.15.5:0/2492219363 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd444103ff0 0x7fd4441043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:36.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:36.412+0000 7fd44c600640 1 -- 172.21.15.5:0/2492219363 >> 172.21.15.5:0/2492219363 conn(0x7fd4440ffa90 msgr2=0x7fd444101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:55:36.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:36.413+0000 7fd44c600640 1 -- 172.21.15.5:0/2492219363 shutdown_connections 2024-09-10T09:55:36.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:36.413+0000 7fd44c600640 1 -- 172.21.15.5:0/2492219363 wait complete. 2024-09-10T09:55:36.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:36.414+0000 7fd44c600640 1 Processor -- start 2024-09-10T09:55:36.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:36.414+0000 7fd44c600640 1 -- start start 2024-09-10T09:55:36.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:36.414+0000 7fd44c600640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd444103ff0 0x7fd44419a760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:36.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:36.415+0000 7fd44c600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd4441049a0 0x7fd44419aca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:36.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:36.415+0000 7fd44c600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd444109120 0x7fd44419df80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:36.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:36.415+0000 7fd44c600640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd444114120 con 0x7fd444109120 2024-09-10T09:55:36.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:36.415+0000 7fd44c600640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fd444113fa0 con 0x7fd4441049a0 2024-09-10T09:55:36.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:36.415+0000 7fd44c600640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fd4441142a0 con 0x7fd444103ff0 2024-09-10T09:55:36.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:36.415+0000 7fd449c00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd444103ff0 0x7fd44419a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:36.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:36.415+0000 7fd449c00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd444103ff0 0x7fd44419a760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:39054/0 (socket says 172.21.15.5:39054) 2024-09-10T09:55:36.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:36.415+0000 7fd449c00640 1 -- 172.21.15.5:0/1011632778 learned_addr learned my addr 172.21.15.5:0/1011632778 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:55:36.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:36.415+0000 7fd449200640 1 --2- 172.21.15.5:0/1011632778 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd4441049a0 0x7fd44419aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:36.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:36.415+0000 7fd44a600640 1 --2- 172.21.15.5:0/1011632778 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd444109120 0x7fd44419df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:36.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:36.415+0000 7fd449c00640 1 -- 172.21.15.5:0/1011632778 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd4441049a0 msgr2=0x7fd44419aca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:36.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:36.415+0000 7fd449c00640 1 --2- 172.21.15.5:0/1011632778 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd4441049a0 0x7fd44419aca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:36.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:36.415+0000 7fd449c00640 1 -- 172.21.15.5:0/1011632778 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd444109120 msgr2=0x7fd44419df80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:36.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:36.415+0000 7fd449c00640 1 --2- 172.21.15.5:0/1011632778 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd444109120 0x7fd44419df80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:36.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:36.415+0000 7fd449c00640 1 -- 172.21.15.5:0/1011632778 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd44419e6b0 con 0x7fd444103ff0 2024-09-10T09:55:36.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:36.416+0000 7fd449c00640 1 --2- 172.21.15.5:0/1011632778 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd444103ff0 0x7fd44419a760 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fd43800e9f0 tx=0x7fd43800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:55:36.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:36.416+0000 7fd442000640 1 -- 172.21.15.5:0/1011632778 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd43800cd80 con 0x7fd444103ff0 2024-09-10T09:55:36.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:36.416+0000 7fd442000640 1 -- 172.21.15.5:0/1011632778 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd438010910 con 0x7fd444103ff0 2024-09-10T09:55:36.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:36.416+0000 7fd44c600640 1 -- 172.21.15.5:0/1011632778 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd44419e9a0 con 0x7fd444103ff0 2024-09-10T09:55:36.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:36.416+0000 7fd442000640 1 -- 172.21.15.5:0/1011632778 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd438018990 con 0x7fd444103ff0 2024-09-10T09:55:36.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:36.416+0000 7fd44c600640 1 -- 172.21.15.5:0/1011632778 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4441a7310 con 0x7fd444103ff0 2024-09-10T09:55:36.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:36.419+0000 7fd442000640 1 -- 172.21.15.5:0/1011632778 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7fd438010ac0 con 0x7fd444103ff0 2024-09-10T09:55:36.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:36.419+0000 7fd44efe8740 1 Processor -- start 2024-09-10T09:55:36.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:36.420+0000 7fd44efe8740 1 -- start start 2024-09-10T09:55:36.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:36.420+0000 7fd44efe8740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564e6bac8b90 0x564e6bac8f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:36.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:36.420+0000 7fd44efe8740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564e6bac94b0 0x564e6bad3830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:36.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:36.420+0000 7fd44efe8740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564e6bad3d70 0x564e6bad6160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:36.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:36.420+0000 7fd44efe8740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x564e6ba2f3e0 con 0x564e6bad3d70 2024-09-10T09:55:36.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:36.420+0000 7fd44efe8740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x564e6b9d6d80 con 0x564e6bac94b0 2024-09-10T09:55:36.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:36.420+0000 7fd44efe8740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x564e6ba2b3a0 con 0x564e6bac8b90 2024-09-10T09:55:36.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:36.420+0000 7fd449c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564e6bac94b0 0x564e6bad3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:36.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:36.420+0000 7fd44a600640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564e6bac8b90 0x564e6bac8f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:36.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:36.420+0000 7fd44a600640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564e6bac8b90 0x564e6bac8f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:39066/0 (socket says 172.21.15.5:39066) 2024-09-10T09:55:36.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:36.420+0000 7fd44a600640 1 -- 172.21.15.5:0/2251842013 learned_addr learned my addr 172.21.15.5:0/2251842013 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:55:36.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:36.420+0000 7fd449200640 1 --2- 172.21.15.5:0/2251842013 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564e6bad3d70 0x564e6bad6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:36.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:36.421+0000 7fd44a600640 1 -- 172.21.15.5:0/2251842013 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564e6bac94b0 msgr2=0x564e6bad3830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:55:36.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:36.421+0000 7fd44a600640 1 --2- 172.21.15.5:0/2251842013 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564e6bac94b0 0x564e6bad3830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:36.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:36.421+0000 7fd44a600640 1 -- 172.21.15.5:0/2251842013 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564e6bad3d70 msgr2=0x564e6bad6160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:55:36.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:36.421+0000 7fd44a600640 1 --2- 172.21.15.5:0/2251842013 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564e6bad3d70 0x564e6bad6160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:36.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:36.421+0000 7fd44a600640 1 -- 172.21.15.5:0/2251842013 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564e6b9f5ae0 con 0x564e6bac8b90 2024-09-10T09:55:36.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:36.421+0000 7fd44a600640 1 --2- 172.21.15.5:0/2251842013 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564e6bac8b90 0x564e6bac8f70 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fd43c009f20 tx=0x7fd43c010860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=faebf404c616ebf7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:55:36.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:36.421+0000 7fd436a00640 1 -- 172.21.15.5:0/2251842013 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd43c011700 con 0x564e6bac8b90 2024-09-10T09:55:36.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:36.421+0000 7fd436a00640 1 -- 172.21.15.5:0/2251842013 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd43c011d80 con 0x564e6bac8b90 2024-09-10T09:55:36.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:36.421+0000 7fd436a00640 1 -- 172.21.15.5:0/2251842013 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd43c01a380 con 0x564e6bac8b90 2024-09-10T09:55:36.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:36.422+0000 7fd442000640 1 --2- 172.21.15.5:0/1011632778 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fd42c0a2c00 0x7fd42c0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:36.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:36.422+0000 7fd442000640 1 -- 172.21.15.5:0/1011632778 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fd4380cf7b0 con 0x7fd444103ff0 2024-09-10T09:55:36.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:36.422+0000 7fd449c00640 1 --2- 172.21.15.5:0/1011632778 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fd42c0a2c00 0x7fd42c0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:36.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:36.422+0000 7fd44efe8740 1 -- 172.21.15.5:0/2251842013 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564e6bac8b90 msgr2=0x564e6bac8f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:36.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:36.422+0000 7fd44efe8740 1 --2- 172.21.15.5:0/2251842013 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564e6bac8b90 0x564e6bac8f70 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fd43c009f20 tx=0x7fd43c010860 comp rx=0 tx=0).stop 2024-09-10T09:55:36.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:36.422+0000 7fd44efe8740 1 -- 172.21.15.5:0/2251842013 shutdown_connections 2024-09-10T09:55:36.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:36.422+0000 7fd44efe8740 1 --2- 172.21.15.5:0/2251842013 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564e6bad3d70 0x564e6bad6160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:36.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:36.422+0000 7fd44efe8740 1 --2- 172.21.15.5:0/2251842013 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564e6bac94b0 0x564e6bad3830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:36.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:36.422+0000 7fd44efe8740 1 --2- 172.21.15.5:0/2251842013 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564e6bac8b90 0x564e6bac8f70 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:36.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:36.422+0000 7fd44efe8740 1 -- 172.21.15.5:0/2251842013 >> 172.21.15.5:0/2251842013 conn(0x564e6bac5120 msgr2=0x564e6bac5500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:55:36.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:36.423+0000 7fd44efe8740 1 -- 172.21.15.5:0/2251842013 shutdown_connections 2024-09-10T09:55:36.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:36.423+0000 7fd44efe8740 1 -- 172.21.15.5:0/2251842013 wait complete. 2024-09-10T09:55:36.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:36.423+0000 7fd449c00640 1 --2- 172.21.15.5:0/1011632778 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fd42c0a2c00 0x7fd42c0a50c0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fd438098020 tx=0x7fd4380d5530 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:55:36.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:36.423+0000 7fd44efe8740 1 -- start start 2024-09-10T09:55:36.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:36.423+0000 7fd44efe8740 1 Processor -- start 2024-09-10T09:55:36.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:36.424+0000 7fd44efe8740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564e6bac8b90 0x564e6bad0ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:36.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:36.424+0000 7fd44efe8740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564e6bac94b0 0x564e6bad1220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:36.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:36.424+0000 7fd44efe8740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564e6bad3d70 0x564e6bad1760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:36.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:36.424+0000 7fd44efe8740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x564e6bad69e0 con 0x564e6bac8b90 2024-09-10T09:55:36.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:36.424+0000 7fd44efe8740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x564e6b9ea0f0 con 0x564e6bad3d70 2024-09-10T09:55:36.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:36.424+0000 7fd44efe8740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x564e6bad6b60 con 0x564e6bac94b0 2024-09-10T09:55:36.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:36.424+0000 7fd44efe8740 10 client.0 _ll_register_callbacks cb 0x564e6b9db9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:55:36.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:36.424+0000 7fd449200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564e6bac94b0 0x564e6bad1220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:36.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:36.424+0000 7fd44a600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564e6bac8b90 0x564e6bad0ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:36.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:36.424+0000 7fd449200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564e6bac94b0 0x564e6bad1220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:39074/0 (socket says 172.21.15.5:39074) 2024-09-10T09:55:36.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:36.424+0000 7fd44a600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564e6bac8b90 0x564e6bad0ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56044/0 (socket says 172.21.15.5:56044) 2024-09-10T09:55:36.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:36.424+0000 7fd449200640 1 -- 172.21.15.5:0/925774554 learned_addr learned my addr 172.21.15.5:0/925774554 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:55:36.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:36.424+0000 7fd44a600640 1 --2- 172.21.15.5:0/925774554 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564e6bad3d70 0x564e6bad1760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:36.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:36.425+0000 7fd449200640 1 -- 172.21.15.5:0/925774554 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564e6bad3d70 msgr2=0x564e6bad1760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:55:36.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:36.425+0000 7fd449200640 1 --2- 172.21.15.5:0/925774554 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564e6bad3d70 0x564e6bad1760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:36.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:36.425+0000 7fd449200640 1 -- 172.21.15.5:0/925774554 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564e6bac8b90 msgr2=0x564e6bad0ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:55:36.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:36.425+0000 7fd449200640 1 --2- 172.21.15.5:0/925774554 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564e6bac8b90 0x564e6bad0ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:36.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:36.425+0000 7fd449200640 1 -- 172.21.15.5:0/925774554 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564e6b7a03c0 con 0x564e6bac94b0 2024-09-10T09:55:36.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:36.425+0000 7fd449200640 1 --2- 172.21.15.5:0/925774554 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564e6bac94b0 0x564e6bad1220 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fd430009900 tx=0x7fd430031850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4347b37aa392dc85 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:55:36.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:36.425+0000 7fd42b400640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-09-10T09:55:36.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:36.425+0000 7fd42b400640 1 -- 172.21.15.5:0/925774554 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd430035040 con 0x564e6bac94b0 2024-09-10T09:55:36.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:36.426+0000 7fd44efe8740 1 -- 172.21.15.5:0/925774554 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd43c011d80 con 0x564e6bac94b0 2024-09-10T09:55:36.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:36.426+0000 7fd42b400640 1 -- 172.21.15.5:0/925774554 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fd430004460 con 0x564e6bac94b0 2024-09-10T09:55:36.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:36.426+0000 7fd42b400640 1 -- 172.21.15.5:0/925774554 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7fd43004f050 con 0x564e6bac94b0 2024-09-10T09:55:36.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:36.426+0000 7fd44efe8740 10 client.28822 fetch_fsmap learned FSMap version 42 2024-09-10T09:55:36.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:36.426+0000 7fd44efe8740 1 -- 172.21.15.5:0/925774554 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x564e6bae2760 con 0x564e6bac94b0 2024-09-10T09:55:36.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:36.426+0000 7fd42b400640 1 -- 172.21.15.5:0/925774554 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd43c011d80 con 0x564e6bac94b0 2024-09-10T09:55:36.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:36.426+0000 7fd42b400640 1 -- 172.21.15.5:0/925774554 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 42) ==== 8370+0+0 (secure 0 0 0) 0x7fd430046710 con 0x564e6bac94b0 2024-09-10T09:55:36.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:36.426+0000 7fd44efe8740 10 client.28822 fetch_fsmap finished waiting for FSMap version 42 2024-09-10T09:55:36.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:36.426+0000 7fd44efe8740 10 client.28822 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T09:55:36.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:36.426+0000 7fd44efe8740 20 client.28822 populate_metadata read hostname 'smithi005' 2024-09-10T09:55:36.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:36.427+0000 7fd44efe8740 1 --2- 172.21.15.5:0/925774554 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x564e6bae3b70 0x564e6bae5fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:36.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:36.427+0000 7fd44efe8740 4 client.28822 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:55:36.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:36.427+0000 7fd44efe8740 1 -- 172.21.15.5:0/925774554 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564e6ba3c950 con 0x564e6bae3b70 2024-09-10T09:55:36.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:36.427+0000 7fd44a600640 1 --2- 172.21.15.5:0/925774554 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x564e6bae3b70 0x564e6bae5fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:36.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:36.427+0000 7fd44a600640 1 --2- 172.21.15.5:0/925774554 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x564e6bae3b70 0x564e6bae5fa0 crc :-1 s=READY pgs=249 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=30a0c15751685ef9 server_cookie=18f216a0978f0722 in_seq=0 out_seq=0 2024-09-10T09:55:36.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:36.427+0000 7fd42b400640 10 client.28822 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T09:55:36.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:36.428+0000 7fd42b400640 1 -- 172.21.15.5:0/925774554 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564e6ba3c950 con 0x564e6bae3b70 2024-09-10T09:55:36.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:36.428+0000 7fd42b400640 10 client.28822 handle_command_reply: tid=0 2024-09-10T09:55:36.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:36.428+0000 7fd44efe8740 1 client.28822 shutdown 2024-09-10T09:55:36.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:36.428+0000 7fd44efe8740 1 -- 172.21.15.5:0/925774554 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564e6bac94b0 msgr2=0x564e6bad1220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:36.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:36.428+0000 7fd44efe8740 1 --2- 172.21.15.5:0/925774554 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564e6bac94b0 0x564e6bad1220 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fd430009900 tx=0x7fd430031850 comp rx=0 tx=0).stop 2024-09-10T09:55:36.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:36.428+0000 7fd44efe8740 1 -- 172.21.15.5:0/925774554 shutdown_connections 2024-09-10T09:55:36.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:36.428+0000 7fd44efe8740 1 --2- 172.21.15.5:0/925774554 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x564e6bae3b70 0x564e6bae5fa0 crc :-1 s=READY pgs=249 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:36.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:36.428+0000 7fd44efe8740 1 --2- 172.21.15.5:0/925774554 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564e6bad3d70 0x564e6bad1760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:36.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:36.428+0000 7fd44efe8740 1 --2- 172.21.15.5:0/925774554 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564e6bac94b0 0x564e6bad1220 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:36.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:36.428+0000 7fd44efe8740 1 --2- 172.21.15.5:0/925774554 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564e6bac8b90 0x564e6bad0ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:36.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:36.428+0000 7fd44efe8740 1 -- 172.21.15.5:0/925774554 >> 172.21.15.5:0/925774554 conn(0x564e6bac5120 msgr2=0x564e6bacae00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:55:36.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:36.428+0000 7fd42b400640 0 client.28822 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T09:55:36.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:36.429+0000 7fd44efe8740 1 -- 172.21.15.5:0/925774554 shutdown_connections 2024-09-10T09:55:36.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:36.429+0000 7fd44efe8740 1 -- 172.21.15.5:0/925774554 wait complete. 2024-09-10T09:55:36.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:36.429+0000 7fd44efe8740 20 client.28822 trim_cache size 0 max 16384 2024-09-10T09:55:36.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:36.450+0000 7fd44efe8740 1 Processor -- start 2024-09-10T09:55:36.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:36.450+0000 7fd44efe8740 1 -- start start 2024-09-10T09:55:36.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:36.450+0000 7fd44efe8740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564e6bac5120 0x564e6bac40b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:36.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:36.450+0000 7fd44efe8740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564e6bac8b90 0x564e6baeb870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:36.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:36.450+0000 7fd44efe8740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564e6bac94b0 0x564e6baebdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:36.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:36.450+0000 7fd44efe8740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd4441b1070 con 0x564e6bac8b90 2024-09-10T09:55:36.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:36.450+0000 7fd44efe8740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fd4441b2bb0 con 0x564e6bac94b0 2024-09-10T09:55:36.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:36.450+0000 7fd44efe8740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fd4441b1740 con 0x564e6bac5120 2024-09-10T09:55:36.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:36.451+0000 7fd44a600640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564e6bac5120 0x564e6bac40b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:36.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:36.451+0000 7fd44a600640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564e6bac5120 0x564e6bac40b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:39084/0 (socket says 172.21.15.5:39084) 2024-09-10T09:55:36.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:36.451+0000 7fd449200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564e6bac8b90 0x564e6baeb870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:36.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:36.451+0000 7fd44a600640 1 -- 172.21.15.5:0/1334663781 learned_addr learned my addr 172.21.15.5:0/1334663781 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:55:36.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:36.451+0000 7fd44a600640 1 --2- 172.21.15.5:0/1334663781 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564e6bac94b0 0x564e6baebdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:36.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:36.451+0000 7fd44a600640 1 -- 172.21.15.5:0/1334663781 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564e6bac94b0 msgr2=0x564e6baebdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:55:36.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:36.451+0000 7fd44a600640 1 --2- 172.21.15.5:0/1334663781 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564e6bac94b0 0x564e6baebdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:36.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:36.451+0000 7fd44a600640 1 -- 172.21.15.5:0/1334663781 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564e6bac8b90 msgr2=0x564e6baeb870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:55:36.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:36.451+0000 7fd44a600640 1 --2- 172.21.15.5:0/1334663781 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564e6bac8b90 0x564e6baeb870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:36.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:36.451+0000 7fd44a600640 1 -- 172.21.15.5:0/1334663781 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564e6bac61d0 con 0x564e6bac5120 2024-09-10T09:55:36.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:36.451+0000 7fd449200640 1 --2- 172.21.15.5:0/1334663781 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564e6bac8b90 0x564e6baeb870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:55:36.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:36.451+0000 7fd44a600640 1 --2- 172.21.15.5:0/1334663781 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564e6bac5120 0x564e6bac40b0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fd43c000f80 tx=0x7fd43c01cad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=730078267ba02cd6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:55:36.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:36.452+0000 7fd436000640 1 -- 172.21.15.5:0/1334663781 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd43c025040 con 0x564e6bac5120 2024-09-10T09:55:36.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:36.452+0000 7fd436000640 1 -- 172.21.15.5:0/1334663781 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd43c026040 con 0x564e6bac5120 2024-09-10T09:55:36.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:36.452+0000 7fd436000640 1 -- 172.21.15.5:0/1334663781 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd43c011b70 con 0x564e6bac5120 2024-09-10T09:55:36.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:36.452+0000 7fd44efe8740 1 -- 172.21.15.5:0/1334663781 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564e6bac5120 msgr2=0x564e6bac40b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:36.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:36.453+0000 7fd44efe8740 1 --2- 172.21.15.5:0/1334663781 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564e6bac5120 0x564e6bac40b0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fd43c000f80 tx=0x7fd43c01cad0 comp rx=0 tx=0).stop 2024-09-10T09:55:36.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:36.453+0000 7fd44efe8740 1 -- 172.21.15.5:0/1334663781 shutdown_connections 2024-09-10T09:55:36.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:36.453+0000 7fd44efe8740 1 --2- 172.21.15.5:0/1334663781 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564e6bac94b0 0x564e6baebdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:36.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:36.453+0000 7fd44efe8740 1 --2- 172.21.15.5:0/1334663781 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564e6bac8b90 0x564e6baeb870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:36.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:36.453+0000 7fd44efe8740 1 --2- 172.21.15.5:0/1334663781 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564e6bac5120 0x564e6bac40b0 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:36.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:36.453+0000 7fd44efe8740 1 -- 172.21.15.5:0/1334663781 >> 172.21.15.5:0/1334663781 conn(0x564e6bacc6d0 msgr2=0x564e6bac39a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:55:36.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:36.453+0000 7fd44efe8740 1 -- 172.21.15.5:0/1334663781 shutdown_connections 2024-09-10T09:55:36.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:36.453+0000 7fd44efe8740 1 -- 172.21.15.5:0/1334663781 wait complete. 2024-09-10T09:55:36.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:36.453+0000 7fd44efe8740 1 -- start start 2024-09-10T09:55:36.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:36.453+0000 7fd44efe8740 1 Processor -- start 2024-09-10T09:55:36.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:36.454+0000 7fd44efe8740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564e6bac5120 0x564e6bad7de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:36.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:36.454+0000 7fd44efe8740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564e6bac8b90 0x564e6bad8320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:36.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:36.454+0000 7fd44efe8740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564e6bac94b0 0x564e6bad8880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:36.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:36.454+0000 7fd44efe8740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd4441b24e0 con 0x564e6bac8b90 2024-09-10T09:55:36.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:36.454+0000 7fd44efe8740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fd4441b1e10 con 0x564e6bac94b0 2024-09-10T09:55:36.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:36.454+0000 7fd44efe8740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fd4441af050 con 0x564e6bac5120 2024-09-10T09:55:36.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:36.454+0000 7fd44efe8740 10 client.0 _ll_register_callbacks cb 0x7fd430005de0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:55:36.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:36.454+0000 7fd44a600640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564e6bac5120 0x564e6bad7de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:36.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:36.454+0000 7fd44a600640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564e6bac5120 0x564e6bad7de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:39100/0 (socket says 172.21.15.5:39100) 2024-09-10T09:55:36.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:36.454+0000 7fd449200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564e6bac8b90 0x564e6bad8320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:36.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:36.454+0000 7fd44a600640 1 -- 172.21.15.5:0/884582777 learned_addr learned my addr 172.21.15.5:0/884582777 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:55:36.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:36.454+0000 7fd44a600640 1 --2- 172.21.15.5:0/884582777 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564e6bac94b0 0x564e6bad8880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:36.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:36.455+0000 7fd44a600640 1 -- 172.21.15.5:0/884582777 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564e6bac94b0 msgr2=0x564e6bad8880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:55:36.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:36.455+0000 7fd44a600640 1 --2- 172.21.15.5:0/884582777 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564e6bac94b0 0x564e6bad8880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:36.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:36.455+0000 7fd44a600640 1 -- 172.21.15.5:0/884582777 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564e6bac8b90 msgr2=0x564e6bad8320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:55:36.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:36.455+0000 7fd44a600640 1 --2- 172.21.15.5:0/884582777 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564e6bac8b90 0x564e6bad8320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:36.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:36.455+0000 7fd44a600640 1 -- 172.21.15.5:0/884582777 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564e6bac5db0 con 0x564e6bac5120 2024-09-10T09:55:36.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:36.455+0000 7fd449200640 1 --2- 172.21.15.5:0/884582777 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564e6bac8b90 0x564e6bad8320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:55:36.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:36.455+0000 7fd44a600640 1 --2- 172.21.15.5:0/884582777 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564e6bac5120 0x564e6bad7de0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fd43c01c540 tx=0x7fd43c009590 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d238c7cad2aa09ce server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:55:36.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:36.455+0000 7fd42b400640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-09-10T09:55:36.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:36.455+0000 7fd42b400640 1 -- 172.21.15.5:0/884582777 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd43c025040 con 0x564e6bac5120 2024-09-10T09:55:36.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:36.455+0000 7fd42b400640 1 -- 172.21.15.5:0/884582777 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fd43c011500 con 0x564e6bac5120 2024-09-10T09:55:36.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:36.455+0000 7fd44efe8740 1 -- 172.21.15.5:0/884582777 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd43c026040 con 0x564e6bac5120 2024-09-10T09:55:36.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:36.456+0000 7fd42b400640 1 -- 172.21.15.5:0/884582777 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7fd43c0118f0 con 0x564e6bac5120 2024-09-10T09:55:36.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:36.456+0000 7fd44efe8740 10 client.28828 fetch_fsmap learned FSMap version 42 2024-09-10T09:55:36.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:36.456+0000 7fd44efe8740 1 -- 172.21.15.5:0/884582777 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x564e6bb4ec50 con 0x564e6bac5120 2024-09-10T09:55:36.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:36.456+0000 7fd42b400640 1 -- 172.21.15.5:0/884582777 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd43c026040 con 0x564e6bac5120 2024-09-10T09:55:36.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:36.456+0000 7fd42b400640 1 -- 172.21.15.5:0/884582777 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 42) ==== 8370+0+0 (secure 0 0 0) 0x7fd43c042020 con 0x564e6bac5120 2024-09-10T09:55:36.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:36.456+0000 7fd44efe8740 10 client.28828 fetch_fsmap finished waiting for FSMap version 42 2024-09-10T09:55:36.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:36.456+0000 7fd44efe8740 10 client.28828 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T09:55:36.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:36.456+0000 7fd44efe8740 20 client.28828 populate_metadata read hostname 'smithi005' 2024-09-10T09:55:36.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:36.456+0000 7fd44efe8740 1 --2- 172.21.15.5:0/884582777 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x564e6bad3d70 0x564e6bb51080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:36.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:36.456+0000 7fd44efe8740 4 client.28828 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T09:55:36.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:36.456+0000 7fd44efe8740 1 -- 172.21.15.5:0/884582777 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "scrub status"}) -- 0x564e6ba36e60 con 0x564e6bad3d70 2024-09-10T09:55:36.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:36.457+0000 7fd449200640 1 --2- 172.21.15.5:0/884582777 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x564e6bad3d70 0x564e6bb51080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:36.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:36.457+0000 7fd449200640 1 --2- 172.21.15.5:0/884582777 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x564e6bad3d70 0x564e6bb51080 crc :-1 s=READY pgs=250 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4ef7b58f1583ffef server_cookie=6c59b73abab39dcb in_seq=0 out_seq=0 2024-09-10T09:55:36.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:36.457+0000 7fd42b400640 10 client.28828 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T09:55:36.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:36.457+0000 7fd42b400640 1 -- 172.21.15.5:0/884582777 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+265 (crc 0 0 0) 0x564e6ba36e60 con 0x564e6bad3d70 2024-09-10T09:55:36.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:36.457+0000 7fd42b400640 10 client.28828 handle_command_reply: tid=0 2024-09-10T09:55:36.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:36.457+0000 7fd44efe8740 1 client.28828 shutdown 2024-09-10T09:55:36.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:36.458+0000 7fd44efe8740 1 -- 172.21.15.5:0/884582777 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564e6bac5120 msgr2=0x564e6bad7de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:36.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:36.458+0000 7fd44efe8740 1 --2- 172.21.15.5:0/884582777 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564e6bac5120 0x564e6bad7de0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fd43c01c540 tx=0x7fd43c009590 comp rx=0 tx=0).stop 2024-09-10T09:55:36.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:36.458+0000 7fd44efe8740 1 -- 172.21.15.5:0/884582777 shutdown_connections 2024-09-10T09:55:36.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:36.458+0000 7fd44efe8740 1 --2- 172.21.15.5:0/884582777 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x564e6bad3d70 0x564e6bb51080 crc :-1 s=READY pgs=250 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:36.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:36.458+0000 7fd44efe8740 1 --2- 172.21.15.5:0/884582777 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564e6bac94b0 0x564e6bad8880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:36.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:36.458+0000 7fd44efe8740 1 --2- 172.21.15.5:0/884582777 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564e6bac8b90 0x564e6bad8320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:36.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:36.458+0000 7fd44efe8740 1 --2- 172.21.15.5:0/884582777 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564e6bac5120 0x564e6bad7de0 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:36.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:36.458+0000 7fd44efe8740 1 -- 172.21.15.5:0/884582777 >> 172.21.15.5:0/884582777 conn(0x564e6bacc6d0 msgr2=0x564e6bad2d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:55:36.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:36.458+0000 7fd42b400640 0 client.28828 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T09:55:36.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:36.458+0000 7fd44efe8740 1 -- 172.21.15.5:0/884582777 shutdown_connections 2024-09-10T09:55:36.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:36.458+0000 7fd44efe8740 1 -- 172.21.15.5:0/884582777 wait complete. 2024-09-10T09:55:36.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:36.458+0000 7fd44efe8740 20 client.28828 trim_cache size 0 max 16384 2024-09-10T09:55:36.463 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-10T09:55:36.463 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "scrub active (0 inodes in the stack)", 2024-09-10T09:55:36.463 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2024-09-10T09:55:36.463 INFO:teuthology.orchestra.run.smithi005.stdout: "8f3269bd-aa14-4ff3-9f8e-6f800d5b9677": { 2024-09-10T09:55:36.463 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2024-09-10T09:55:36.463 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "8f3269bd-aa14-4ff3-9f8e-6f800d5b9677", 2024-09-10T09:55:36.463 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2024-09-10T09:55:36.463 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T09:55:36.464 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T09:55:36.464 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-10T09:55:36.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:36.459+0000 7fd44c600640 1 -- 172.21.15.5:0/1011632778 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fd42c0a2c00 msgr2=0x7fd42c0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:36.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:36.459+0000 7fd44c600640 1 --2- 172.21.15.5:0/1011632778 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fd42c0a2c00 0x7fd42c0a50c0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fd438098020 tx=0x7fd4380d5530 comp rx=0 tx=0).stop 2024-09-10T09:55:36.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:36.459+0000 7fd44c600640 1 -- 172.21.15.5:0/1011632778 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd444103ff0 msgr2=0x7fd44419a760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:36.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:36.459+0000 7fd44c600640 1 --2- 172.21.15.5:0/1011632778 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd444103ff0 0x7fd44419a760 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fd43800e9f0 tx=0x7fd43800eeb0 comp rx=0 tx=0).stop 2024-09-10T09:55:36.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:36.459+0000 7fd44c600640 1 -- 172.21.15.5:0/1011632778 shutdown_connections 2024-09-10T09:55:36.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:36.459+0000 7fd44c600640 1 --2- 172.21.15.5:0/1011632778 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fd42c0a2c00 0x7fd42c0a50c0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:36.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:36.459+0000 7fd44c600640 1 --2- 172.21.15.5:0/1011632778 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd444109120 0x7fd44419df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:36.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:36.459+0000 7fd44c600640 1 --2- 172.21.15.5:0/1011632778 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd4441049a0 0x7fd44419aca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:36.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:36.459+0000 7fd44c600640 1 --2- 172.21.15.5:0/1011632778 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd444103ff0 0x7fd44419a760 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:36.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:36.459+0000 7fd44c600640 1 -- 172.21.15.5:0/1011632778 >> 172.21.15.5:0/1011632778 conn(0x7fd4440ffa90 msgr2=0x7fd444100d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:55:36.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:36.459+0000 7fd44c600640 1 -- 172.21.15.5:0/1011632778 shutdown_connections 2024-09-10T09:55:36.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:36.459+0000 7fd44c600640 1 -- 172.21.15.5:0/1011632778 wait complete. 2024-09-10T09:55:36.981 INFO:tasks.workunit.client.0.smithi005.stdout: 10 63849 17.96 MB/sec execute 75 sec latency 14114.125 ms 2024-09-10T09:55:37.445 INFO:tasks.cephfs.filesystem:scrub status for tag:8f3269bd-aa14-4ff3-9f8e-6f800d5b9677 - {'path': '/', 'tag': '8f3269bd-aa14-4ff3-9f8e-6f800d5b9677', 'options': 'recursive,force'} 2024-09-10T09:55:37.981 INFO:tasks.workunit.client.0.smithi005.stdout: 10 63852 17.72 MB/sec execute 76 sec latency 14919.438 ms 2024-09-10T09:55:38.070 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:55:37 smithi005 ceph-mon[30350]: pgmap v911: 129 pgs: 129 active+clean; 608 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 384 KiB/s rd, 5.7 MiB/s wr, 216 op/s 2024-09-10T09:55:38.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:55:37 smithi102 ceph-mon[35442]: pgmap v911: 129 pgs: 129 active+clean; 608 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 384 KiB/s rd, 5.7 MiB/s wr, 216 op/s 2024-09-10T09:55:38.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:55:37 smithi159 ceph-mon[35513]: pgmap v911: 129 pgs: 129 active+clean; 608 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 384 KiB/s rd, 5.7 MiB/s wr, 216 op/s 2024-09-10T09:55:38.954 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:55:38 smithi005 ceph-mon[30350]: pgmap v912: 129 pgs: 129 active+clean; 602 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 270 KiB/s rd, 5.4 MiB/s wr, 192 op/s 2024-09-10T09:55:38.981 INFO:tasks.workunit.client.0.smithi005.stdout: 10 63852 17.49 MB/sec execute 77 sec latency 6750.723 ms 2024-09-10T09:55:39.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:55:38 smithi102 ceph-mon[35442]: pgmap v912: 129 pgs: 129 active+clean; 602 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 270 KiB/s rd, 5.4 MiB/s wr, 192 op/s 2024-09-10T09:55:39.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:55:38 smithi159 ceph-mon[35513]: pgmap v912: 129 pgs: 129 active+clean; 602 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 270 KiB/s rd, 5.4 MiB/s wr, 192 op/s 2024-09-10T09:55:39.981 INFO:tasks.workunit.client.0.smithi005.stdout: 10 63853 17.27 MB/sec execute 78 sec latency 7750.782 ms 2024-09-10T09:55:40.981 INFO:tasks.workunit.client.0.smithi005.stdout: 10 63855 17.05 MB/sec execute 79 sec latency 8655.487 ms 2024-09-10T09:55:41.306 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:55:41 smithi005 ceph-mon[30350]: pgmap v913: 129 pgs: 129 active+clean; 609 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 315 KiB/s rd, 6.0 MiB/s wr, 210 op/s 2024-09-10T09:55:41.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:55:41 smithi159 ceph-mon[35513]: pgmap v913: 129 pgs: 129 active+clean; 609 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 315 KiB/s rd, 6.0 MiB/s wr, 210 op/s 2024-09-10T09:55:41.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:55:41 smithi102 ceph-mon[35442]: pgmap v913: 129 pgs: 129 active+clean; 609 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 315 KiB/s rd, 6.0 MiB/s wr, 210 op/s 2024-09-10T09:55:41.983 INFO:tasks.workunit.client.0.smithi005.stdout: 10 63855 16.84 MB/sec execute 80 sec latency 4194.900 ms 2024-09-10T09:55:42.306 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:55:42 smithi005 ceph-mon[30350]: scrub summary: idle+waiting paths [/] 2024-09-10T09:55:42.306 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:55:42 smithi005 ceph-mon[30350]: scrub summary: idle+waiting paths [/] 2024-09-10T09:55:42.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:55:42 smithi159 ceph-mon[35513]: scrub summary: idle+waiting paths [/] 2024-09-10T09:55:42.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:55:42 smithi159 ceph-mon[35513]: scrub summary: idle+waiting paths [/] 2024-09-10T09:55:42.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:55:42 smithi102 ceph-mon[35442]: scrub summary: idle+waiting paths [/] 2024-09-10T09:55:42.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:55:42 smithi102 ceph-mon[35442]: scrub summary: idle+waiting paths [/] 2024-09-10T09:55:42.981 INFO:tasks.workunit.client.0.smithi005.stdout: 10 63855 16.63 MB/sec execute 81 sec latency 5194.966 ms 2024-09-10T09:55:43.196 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:55:43 smithi005 ceph-mon[30350]: pgmap v914: 129 pgs: 129 active+clean; 610 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 242 KiB/s rd, 5.3 MiB/s wr, 189 op/s 2024-09-10T09:55:43.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:55:43 smithi159 ceph-mon[35513]: pgmap v914: 129 pgs: 129 active+clean; 610 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 242 KiB/s rd, 5.3 MiB/s wr, 189 op/s 2024-09-10T09:55:43.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:55:43 smithi102 ceph-mon[35442]: pgmap v914: 129 pgs: 129 active+clean; 610 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 242 KiB/s rd, 5.3 MiB/s wr, 189 op/s 2024-09-10T09:55:43.981 INFO:tasks.workunit.client.0.smithi005.stdout: 10 63855 16.43 MB/sec execute 82 sec latency 6195.041 ms 2024-09-10T09:55:44.981 INFO:tasks.workunit.client.0.smithi005.stdout: 10 63858 16.23 MB/sec execute 83 sec latency 6682.154 ms 2024-09-10T09:55:45.165 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:55:45 smithi005 ceph-mon[30350]: pgmap v915: 129 pgs: 129 active+clean; 604 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 242 KiB/s rd, 5.3 MiB/s wr, 198 op/s 2024-09-10T09:55:45.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:55:45 smithi159 ceph-mon[35513]: pgmap v915: 129 pgs: 129 active+clean; 604 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 242 KiB/s rd, 5.3 MiB/s wr, 198 op/s 2024-09-10T09:55:45.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:55:45 smithi102 ceph-mon[35442]: pgmap v915: 129 pgs: 129 active+clean; 604 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 242 KiB/s rd, 5.3 MiB/s wr, 198 op/s 2024-09-10T09:55:45.981 INFO:tasks.workunit.client.0.smithi005.stdout: 10 63858 16.04 MB/sec execute 84 sec latency 6400.520 ms 2024-09-10T09:55:46.982 INFO:tasks.workunit.client.0.smithi005.stdout: 10 63859 15.85 MB/sec execute 85 sec latency 6655.804 ms 2024-09-10T09:55:47.306 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:55:47 smithi005 ceph-mon[30350]: pgmap v916: 129 pgs: 129 active+clean; 598 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 383 KiB/s rd, 5.5 MiB/s wr, 224 op/s 2024-09-10T09:55:47.306 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:55:47 smithi005 ceph-mon[30350]: scrub summary: idle 2024-09-10T09:55:47.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:55:47 smithi159 ceph-mon[35513]: pgmap v916: 129 pgs: 129 active+clean; 598 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 383 KiB/s rd, 5.5 MiB/s wr, 224 op/s 2024-09-10T09:55:47.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:55:47 smithi159 ceph-mon[35513]: scrub summary: idle 2024-09-10T09:55:47.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:55:47 smithi102 ceph-mon[35442]: pgmap v916: 129 pgs: 129 active+clean; 598 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 383 KiB/s rd, 5.5 MiB/s wr, 224 op/s 2024-09-10T09:55:47.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:55:47 smithi102 ceph-mon[35442]: scrub summary: idle 2024-09-10T09:55:47.981 INFO:tasks.workunit.client.0.smithi005.stdout: 10 63860 15.66 MB/sec execute 86 sec latency 7089.496 ms 2024-09-10T09:55:48.217 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:55:48 smithi005 ceph-mon[30350]: scrub summary: idle 2024-09-10T09:55:48.217 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:55:48 smithi005 ceph-mon[30350]: scrub summary: idle 2024-09-10T09:55:48.217 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:55:48 smithi005 ceph-mon[30350]: scrub summary: idle 2024-09-10T09:55:48.217 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:55:48 smithi005 ceph-mon[30350]: scrub summary: idle 2024-09-10T09:55:48.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:55:48 smithi159 ceph-mon[35513]: scrub summary: idle 2024-09-10T09:55:48.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:55:48 smithi159 ceph-mon[35513]: scrub summary: idle 2024-09-10T09:55:48.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:55:48 smithi159 ceph-mon[35513]: scrub summary: idle 2024-09-10T09:55:48.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:55:48 smithi159 ceph-mon[35513]: scrub summary: idle 2024-09-10T09:55:48.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:55:48 smithi102 ceph-mon[35442]: scrub summary: idle 2024-09-10T09:55:48.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:55:48 smithi102 ceph-mon[35442]: scrub summary: idle 2024-09-10T09:55:48.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:55:48 smithi102 ceph-mon[35442]: scrub summary: idle 2024-09-10T09:55:48.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:55:48 smithi102 ceph-mon[35442]: scrub summary: idle 2024-09-10T09:55:48.981 INFO:tasks.workunit.client.0.smithi005.stdout: 10 63868 15.49 MB/sec execute 87 sec latency 7560.756 ms 2024-09-10T09:55:49.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:55:49 smithi159 ceph-mon[35513]: pgmap v917: 129 pgs: 129 active+clean; 594 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 236 KiB/s rd, 5.0 MiB/s wr, 209 op/s 2024-09-10T09:55:49.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:55:49 smithi102 ceph-mon[35442]: pgmap v917: 129 pgs: 129 active+clean; 594 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 236 KiB/s rd, 5.0 MiB/s wr, 209 op/s 2024-09-10T09:55:49.394 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:55:49 smithi005 ceph-mon[30350]: pgmap v917: 129 pgs: 129 active+clean; 594 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 236 KiB/s rd, 5.0 MiB/s wr, 209 op/s 2024-09-10T09:55:49.982 INFO:tasks.workunit.client.0.smithi005.stdout: 10 63868 15.31 MB/sec execute 88 sec latency 5513.202 ms 2024-09-10T09:55:50.982 INFO:tasks.workunit.client.0.smithi005.stdout: 10 64362 15.37 MB/sec execute 89 sec latency 5645.786 ms 2024-09-10T09:55:51.306 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:55:51 smithi005 ceph-mon[30350]: pgmap v918: 129 pgs: 129 active+clean; 596 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 389 KiB/s rd, 5.5 MiB/s wr, 224 op/s 2024-09-10T09:55:51.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:55:51 smithi159 ceph-mon[35513]: pgmap v918: 129 pgs: 129 active+clean; 596 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 389 KiB/s rd, 5.5 MiB/s wr, 224 op/s 2024-09-10T09:55:51.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:55:51 smithi102 ceph-mon[35442]: pgmap v918: 129 pgs: 129 active+clean; 596 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 389 KiB/s rd, 5.5 MiB/s wr, 224 op/s 2024-09-10T09:55:51.982 INFO:tasks.workunit.client.0.smithi005.stdout: 10 64858 15.43 MB/sec execute 90 sec latency 77.468 ms 2024-09-10T09:55:52.982 INFO:tasks.workunit.client.0.smithi005.stdout: 10 65536 15.69 MB/sec execute 91 sec latency 55.492 ms 2024-09-10T09:55:53.292 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:55:53 smithi005 ceph-mon[30350]: pgmap v919: 129 pgs: 129 active+clean; 597 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 699 KiB/s rd, 4.2 MiB/s wr, 190 op/s 2024-09-10T09:55:53.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:55:53 smithi159 ceph-mon[35513]: pgmap v919: 129 pgs: 129 active+clean; 597 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 699 KiB/s rd, 4.2 MiB/s wr, 190 op/s 2024-09-10T09:55:53.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:55:53 smithi102 ceph-mon[35442]: pgmap v919: 129 pgs: 129 active+clean; 597 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 699 KiB/s rd, 4.2 MiB/s wr, 190 op/s 2024-09-10T09:55:53.982 INFO:tasks.workunit.client.0.smithi005.stdout: 10 66184 15.83 MB/sec execute 92 sec latency 128.289 ms 2024-09-10T09:55:54.908 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 30.5 2024-09-10T09:55:54.909 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T09:55:54.982 INFO:tasks.workunit.client.0.smithi005.stdout: 10 67107 16.14 MB/sec execute 93 sec latency 50.013 ms 2024-09-10T09:55:55.307 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:55:55 smithi005 ceph-mon[30350]: pgmap v920: 129 pgs: 129 active+clean; 595 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.0 MiB/s wr, 188 op/s 2024-09-10T09:55:55.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:55:55 smithi159 ceph-mon[35513]: pgmap v920: 129 pgs: 129 active+clean; 595 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.0 MiB/s wr, 188 op/s 2024-09-10T09:55:55.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:55:55 smithi102 ceph-mon[35442]: pgmap v920: 129 pgs: 129 active+clean; 595 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.0 MiB/s wr, 188 op/s 2024-09-10T09:55:55.526 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:55:55.982 INFO:tasks.workunit.client.0.smithi005.stdout: 10 67961 16.66 MB/sec execute 94 sec latency 53.628 ms 2024-09-10T09:55:56.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:56.393+0000 7f5ca1600640 1 -- 172.21.15.5:0/4193545 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5c9c100500 msgr2=0x7f5c9c1008e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:56.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:56.393+0000 7f5ca1600640 1 --2- 172.21.15.5:0/4193545 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5c9c100500 0x7f5c9c1008e0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f5c94009900 tx=0x7f5c9402a480 comp rx=0 tx=0).stop 2024-09-10T09:55:56.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:56.394+0000 7f5ca1600640 1 -- 172.21.15.5:0/4193545 shutdown_connections 2024-09-10T09:55:56.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:56.394+0000 7f5ca1600640 1 --2- 172.21.15.5:0/4193545 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5c9c112780 0x7f5c9c114bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:56.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:56.394+0000 7f5ca1600640 1 --2- 172.21.15.5:0/4193545 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5c9c100e20 0x7f5c9c1121f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:56.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:56.394+0000 7f5ca1600640 1 --2- 172.21.15.5:0/4193545 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5c9c100500 0x7f5c9c1008e0 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:56.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:56.394+0000 7f5ca1600640 1 -- 172.21.15.5:0/4193545 >> 172.21.15.5:0/4193545 conn(0x7f5c9c0fc5c0 msgr2=0x7f5c9c0fe9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:55:56.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:56.395+0000 7f5ca1600640 1 -- 172.21.15.5:0/4193545 shutdown_connections 2024-09-10T09:55:56.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:56.396+0000 7f5ca1600640 1 -- 172.21.15.5:0/4193545 wait complete. 2024-09-10T09:55:56.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:56.397+0000 7f5ca1600640 1 Processor -- start 2024-09-10T09:55:56.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:56.397+0000 7f5ca1600640 1 -- start start 2024-09-10T09:55:56.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:56.398+0000 7f5ca1600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5c9c100500 0x7f5c9c1a46d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:56.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:56.398+0000 7f5ca1600640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5c9c100e20 0x7f5c9c1a4c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:56.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:56.398+0000 7f5ca1600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5c9c112780 0x7f5c9c19e850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:56.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:56.398+0000 7f5ca1600640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f5c9c118560 con 0x7f5c9c100500 2024-09-10T09:55:56.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:56.398+0000 7f5ca1600640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f5c9c1183e0 con 0x7f5c9c112780 2024-09-10T09:55:56.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:56.398+0000 7f5ca1600640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f5c9c1186e0 con 0x7f5c9c100e20 2024-09-10T09:55:56.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:56.399+0000 7f5c9b400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5c9c112780 0x7f5c9c19e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:56.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:56.399+0000 7f5c9b400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5c9c112780 0x7f5c9c19e850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:42650/0 (socket says 172.21.15.5:42650) 2024-09-10T09:55:56.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:56.399+0000 7f5c9b400640 1 -- 172.21.15.5:0/879724687 learned_addr learned my addr 172.21.15.5:0/879724687 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:55:56.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:56.399+0000 7f5c9aa00640 1 --2- 172.21.15.5:0/879724687 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5c9c100500 0x7f5c9c1a46d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:56.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:56.400+0000 7f5c9a000640 1 --2- 172.21.15.5:0/879724687 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5c9c100e20 0x7f5c9c1a4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:56.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:56.400+0000 7f5c9b400640 1 -- 172.21.15.5:0/879724687 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5c9c100e20 msgr2=0x7f5c9c1a4c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:56.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:56.400+0000 7f5c9b400640 1 --2- 172.21.15.5:0/879724687 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5c9c100e20 0x7f5c9c1a4c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:56.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:56.400+0000 7f5c9b400640 1 -- 172.21.15.5:0/879724687 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5c9c100500 msgr2=0x7f5c9c1a46d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:56.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:56.400+0000 7f5c9b400640 1 --2- 172.21.15.5:0/879724687 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5c9c100500 0x7f5c9c1a46d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:56.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:56.400+0000 7f5c9b400640 1 -- 172.21.15.5:0/879724687 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c9c19eff0 con 0x7f5c9c112780 2024-09-10T09:55:56.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:56.400+0000 7f5c9aa00640 1 --2- 172.21.15.5:0/879724687 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5c9c100500 0x7f5c9c1a46d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:55:56.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:56.400+0000 7f5c9b400640 1 --2- 172.21.15.5:0/879724687 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5c9c112780 0x7f5c9c19e850 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f5c8c00eb10 tx=0x7f5c8c00c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:55:56.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:56.401+0000 7f5c93400640 1 -- 172.21.15.5:0/879724687 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c8c004b20 con 0x7f5c9c112780 2024-09-10T09:55:56.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:56.401+0000 7f5c93400640 1 -- 172.21.15.5:0/879724687 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5c8c004cc0 con 0x7f5c9c112780 2024-09-10T09:55:56.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:56.401+0000 7f5c93400640 1 -- 172.21.15.5:0/879724687 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c8c010790 con 0x7f5c9c112780 2024-09-10T09:55:56.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:56.401+0000 7f5ca1600640 1 -- 172.21.15.5:0/879724687 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c9c19f2e0 con 0x7f5c9c112780 2024-09-10T09:55:56.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:56.401+0000 7f5ca1600640 1 -- 172.21.15.5:0/879724687 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c9c1079d0 con 0x7f5c9c112780 2024-09-10T09:55:56.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:56.404+0000 7f5c90c00640 1 -- 172.21.15.5:0/879724687 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5c9c06a580 con 0x7f5c9c112780 2024-09-10T09:55:56.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:56.410+0000 7f5c93400640 1 -- 172.21.15.5:0/879724687 <== mon.2 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f5c8c005020 con 0x7f5c9c112780 2024-09-10T09:55:56.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:56.412+0000 7f5c93400640 1 --2- 172.21.15.5:0/879724687 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f5c800a2d10 0x7f5c800a51d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:56.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:56.413+0000 7f5c93400640 1 -- 172.21.15.5:0/879724687 <== mon.2 v2:172.21.15.102:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f5c8c099ea0 con 0x7f5c9c112780 2024-09-10T09:55:56.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:56.414+0000 7f5c9aa00640 1 --2- 172.21.15.5:0/879724687 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f5c800a2d10 0x7f5c800a51d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:56.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:56.416+0000 7f5c9aa00640 1 --2- 172.21.15.5:0/879724687 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f5c800a2d10 0x7f5c800a51d0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f5c94000c00 tx=0x7f5c9402a980 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:55:56.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:56.421+0000 7f5c93400640 1 -- 172.21.15.5:0/879724687 <== mon.2 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f5c8c09c500 con 0x7f5c9c112780 2024-09-10T09:55:56.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:56.812+0000 7f5c90c00640 1 -- 172.21.15.5:0/879724687 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5c9c062e90 con 0x7f5c9c112780 2024-09-10T09:55:56.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:56.813+0000 7f5c93400640 1 -- 172.21.15.5:0/879724687 <== mon.2 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 42 v42) ==== 94+0+11969 (secure 0 0 0) 0x7f5c8c0a01b0 con 0x7f5c9c112780 2024-09-10T09:55:56.815 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-10T09:55:56.815 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":42,"btime":"2024-09-10T09:55:06:670916+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14454,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/2970594311","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2970594311},{"type":"v1","addr":"172.21.15.5:6835","nonce":2970594311}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14514,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6839/3616502174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":3616502174},{"type":"v1","addr":"172.21.15.5:6839","nonce":3616502174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":42,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T09:35:29.626287+0000","modified":"2024-09-10T09:55:06.419829+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24542,"mds_1":24535,"mds_2":24511,"mds_3":24506,"mds_4":24443},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14520":{"gid":14520,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/2073599842","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":2073599842},{"type":"v1","addr":"172.21.15.5:6841","nonce":2073599842}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"k","rank":4,"incarnation":26,"state":"up:active","state_seq":24,"addr":"172.21.15.102:6833/2457007427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":2457007427},{"type":"v1","addr":"172.21.15.102:6833","nonce":2457007427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/356445159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":356445159},{"type":"v1","addr":"172.21.15.159:6833","nonce":356445159}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6837/2894314507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":2894314507},{"type":"v1","addr":"172.21.15.5:6837","nonce":2894314507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.102:6835/1647549111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":1647549111},{"type":"v1","addr":"172.21.15.102:6835","nonce":1647549111}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6835/952657242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":952657242},{"type":"v1","addr":"172.21.15.159:6835","nonce":952657242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":19,"addr":"172.21.15.102:6837/2689776764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":2689776764},{"type":"v1","addr":"172.21.15.102:6837","nonce":2689776764}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.159:6837/620627639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":620627639},{"type":"v1","addr":"172.21.15.159:6837","nonce":620627639}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24535":{"gid":24535,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":18,"addr":"172.21.15.159:6839/2073739599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":2073739599},{"type":"v1","addr":"172.21.15.159:6839","nonce":2073739599}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24542":{"gid":24542,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.102:6839/335409749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6838","nonce":335409749},{"type":"v1","addr":"172.21.15.102:6839","nonce":335409749}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24542,"qdb_cluster":[24542,24535,24511,24506,24443]},"id":1}]} 2024-09-10T09:55:56.816 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 42 2024-09-10T09:55:56.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:56.819+0000 7f5c90c00640 1 -- 172.21.15.5:0/879724687 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f5c800a2d10 msgr2=0x7f5c800a51d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:56.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:56.819+0000 7f5c90c00640 1 --2- 172.21.15.5:0/879724687 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f5c800a2d10 0x7f5c800a51d0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f5c94000c00 tx=0x7f5c9402a980 comp rx=0 tx=0).stop 2024-09-10T09:55:56.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:56.819+0000 7f5c90c00640 1 -- 172.21.15.5:0/879724687 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5c9c112780 msgr2=0x7f5c9c19e850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:56.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:56.819+0000 7f5c90c00640 1 --2- 172.21.15.5:0/879724687 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5c9c112780 0x7f5c9c19e850 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f5c8c00eb10 tx=0x7f5c8c00c2d0 comp rx=0 tx=0).stop 2024-09-10T09:55:56.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:56.819+0000 7f5c90c00640 1 -- 172.21.15.5:0/879724687 shutdown_connections 2024-09-10T09:55:56.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:56.819+0000 7f5c90c00640 1 --2- 172.21.15.5:0/879724687 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f5c800a2d10 0x7f5c800a51d0 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:56.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:56.819+0000 7f5c90c00640 1 --2- 172.21.15.5:0/879724687 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5c9c112780 0x7f5c9c19e850 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:56.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:56.819+0000 7f5c90c00640 1 --2- 172.21.15.5:0/879724687 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5c9c100e20 0x7f5c9c1a4c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:56.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:56.819+0000 7f5c90c00640 1 --2- 172.21.15.5:0/879724687 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5c9c100500 0x7f5c9c1a46d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:56.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:56.819+0000 7f5c90c00640 1 -- 172.21.15.5:0/879724687 >> 172.21.15.5:0/879724687 conn(0x7f5c9c0fc5c0 msgr2=0x7f5c9c111020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:55:56.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:56.819+0000 7f5c90c00640 1 -- 172.21.15.5:0/879724687 shutdown_connections 2024-09-10T09:55:56.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:56.819+0000 7f5c90c00640 1 -- 172.21.15.5:0/879724687 wait complete. 2024-09-10T09:55:56.982 INFO:tasks.workunit.client.0.smithi005.stdout: 10 68532 16.82 MB/sec execute 95 sec latency 83.995 ms 2024-09-10T09:55:57.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:55:57 smithi159 ceph-mon[35513]: pgmap v921: 129 pgs: 129 active+clean; 584 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 4.5 MiB/s wr, 314 op/s 2024-09-10T09:55:57.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:55:57 smithi159 ceph-mon[35513]: from='client.? 172.21.15.5:0/879724687' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:55:57.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:55:57 smithi102 ceph-mon[35442]: pgmap v921: 129 pgs: 129 active+clean; 584 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 4.5 MiB/s wr, 314 op/s 2024-09-10T09:55:57.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:55:57 smithi102 ceph-mon[35442]: from='client.? 172.21.15.5:0/879724687' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:55:57.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:55:57 smithi005 ceph-mon[30350]: pgmap v921: 129 pgs: 129 active+clean; 584 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 4.5 MiB/s wr, 314 op/s 2024-09-10T09:55:57.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:55:57 smithi005 ceph-mon[30350]: from='client.? 172.21.15.5:0/879724687' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:55:57.821 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24542 quiesce db / --timeout 90 --expiration 150.5 --await-for 45.0' 2024-09-10T09:55:57.822 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24542 quiesce db / --timeout 90 --expiration 150.5 --await-for 45.0 2024-09-10T09:55:57.982 INFO:tasks.workunit.client.0.smithi005.stdout: 10 69057 16.85 MB/sec execute 96 sec latency 74.886 ms 2024-09-10T09:55:58.280 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:55:58.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:58.930+0000 7f0bb7600640 1 -- 172.21.15.5:0/1886585670 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0bb01008e0 msgr2=0x7f0bb0100cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:58.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:58.930+0000 7f0bb7600640 1 --2- 172.21.15.5:0/1886585670 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0bb01008e0 0x7f0bb0100cc0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f0ba4009900 tx=0x7f0ba402a3f0 comp rx=0 tx=0).stop 2024-09-10T09:55:58.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:58.931+0000 7f0bb7600640 1 -- 172.21.15.5:0/1886585670 shutdown_connections 2024-09-10T09:55:58.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:58.931+0000 7f0bb7600640 1 --2- 172.21.15.5:0/1886585670 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0bb0112b20 0x7f0bb0114f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:58.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:58.931+0000 7f0bb7600640 1 --2- 172.21.15.5:0/1886585670 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0bb0101290 0x7f0bb01125e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:58.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:58.931+0000 7f0bb7600640 1 --2- 172.21.15.5:0/1886585670 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0bb01008e0 0x7f0bb0100cc0 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:58.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:58.931+0000 7f0bb7600640 1 -- 172.21.15.5:0/1886585670 >> 172.21.15.5:0/1886585670 conn(0x7f0bb0077210 msgr2=0x7f0bb0077620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:55:58.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:58.931+0000 7f0bb7600640 1 -- 172.21.15.5:0/1886585670 shutdown_connections 2024-09-10T09:55:58.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:58.931+0000 7f0bb7600640 1 -- 172.21.15.5:0/1886585670 wait complete. 2024-09-10T09:55:58.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:58.932+0000 7f0bb7600640 1 Processor -- start 2024-09-10T09:55:58.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:58.933+0000 7f0bb7600640 1 -- start start 2024-09-10T09:55:58.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:58.934+0000 7f0bb7600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0bb01008e0 0x7f0bb01121b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:58.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:58.934+0000 7f0bb7600640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0bb0101290 0x7f0bb010e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:58.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:58.934+0000 7f0bb7600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0bb0112b20 0x7f0bb010e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:58.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:58.934+0000 7f0bb7600640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f0bb010c770 con 0x7f0bb0112b20 2024-09-10T09:55:58.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:58.934+0000 7f0bb7600640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f0bb010c5f0 con 0x7f0bb01008e0 2024-09-10T09:55:58.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:58.934+0000 7f0bb7600640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f0bb010c8f0 con 0x7f0bb0101290 2024-09-10T09:55:58.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:58.934+0000 7f0bb4c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0bb01008e0 0x7f0bb01121b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:58.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:58.934+0000 7f0bb4c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0bb01008e0 0x7f0bb01121b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:42668/0 (socket says 172.21.15.5:42668) 2024-09-10T09:55:58.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:58.934+0000 7f0bb4c00640 1 -- 172.21.15.5:0/1063542786 learned_addr learned my addr 172.21.15.5:0/1063542786 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:55:58.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:58.934+0000 7f0bb5600640 1 --2- 172.21.15.5:0/1063542786 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0bb0112b20 0x7f0bb010e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:58.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:58.935+0000 7f0bb4c00640 1 -- 172.21.15.5:0/1063542786 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0bb0101290 msgr2=0x7f0bb010e2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:58.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:58.935+0000 7f0bb4c00640 1 --2- 172.21.15.5:0/1063542786 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0bb0101290 0x7f0bb010e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:58.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:58.935+0000 7f0bb4c00640 1 -- 172.21.15.5:0/1063542786 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0bb0112b20 msgr2=0x7f0bb010e7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:58.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:58.935+0000 7f0bb4c00640 1 --2- 172.21.15.5:0/1063542786 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0bb0112b20 0x7f0bb010e7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:58.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:58.935+0000 7f0bb4c00640 1 -- 172.21.15.5:0/1063542786 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0bb010ef10 con 0x7f0bb01008e0 2024-09-10T09:55:58.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:58.935+0000 7f0bb5600640 1 --2- 172.21.15.5:0/1063542786 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0bb0112b20 0x7f0bb010e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:55:58.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:58.935+0000 7f0bb4c00640 1 --2- 172.21.15.5:0/1063542786 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0bb01008e0 0x7f0bb01121b0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f0ba4009230 tx=0x7f0ba4032d90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:55:58.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:58.936+0000 7f0bad600640 1 -- 172.21.15.5:0/1063542786 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ba403f030 con 0x7f0bb01008e0 2024-09-10T09:55:58.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:58.936+0000 7f0bad600640 1 -- 172.21.15.5:0/1063542786 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0ba4003ce0 con 0x7f0bb01008e0 2024-09-10T09:55:58.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:58.936+0000 7f0bad600640 1 -- 172.21.15.5:0/1063542786 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ba4035070 con 0x7f0bb01008e0 2024-09-10T09:55:58.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:58.936+0000 7f0bb7600640 1 -- 172.21.15.5:0/1063542786 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0bb01a2970 con 0x7f0bb01008e0 2024-09-10T09:55:58.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:58.936+0000 7f0bb7600640 1 -- 172.21.15.5:0/1063542786 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0bb01a2d10 con 0x7f0bb01008e0 2024-09-10T09:55:58.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:58.942+0000 7f0bba25a740 1 Processor -- start 2024-09-10T09:55:58.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:58.942+0000 7f0bba25a740 1 -- start start 2024-09-10T09:55:58.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:58.942+0000 7f0bad600640 1 -- 172.21.15.5:0/1063542786 <== mon.2 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f0ba4042030 con 0x7f0bb01008e0 2024-09-10T09:55:58.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:58.942+0000 7f0bba25a740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d2d8f953b0 0x55d2d8f95790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:58.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:58.942+0000 7f0bba25a740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d2d8f95cd0 0x55d2d8fa0050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:58.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:58.942+0000 7f0bba25a740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d2d8fa0590 0x55d2d8fa2980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:58.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:58.942+0000 7f0bba25a740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55d2d8efb530 con 0x55d2d8fa0590 2024-09-10T09:55:58.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:58.942+0000 7f0bba25a740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55d2d8ea2860 con 0x55d2d8f953b0 2024-09-10T09:55:58.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:58.942+0000 7f0bba25a740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55d2d8ef74f0 con 0x55d2d8f95cd0 2024-09-10T09:55:58.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:58.943+0000 7f0bb5600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d2d8f953b0 0x55d2d8f95790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:58.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:58.943+0000 7f0bb5600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d2d8f953b0 0x55d2d8f95790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:42684/0 (socket says 172.21.15.5:42684) 2024-09-10T09:55:58.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:58.943+0000 7f0bb5600640 1 -- 172.21.15.5:0/1333283594 learned_addr learned my addr 172.21.15.5:0/1333283594 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:55:58.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:58.943+0000 7f0bafe00640 1 --2- 172.21.15.5:0/1333283594 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d2d8fa0590 0x55d2d8fa2980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:58.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:58.944+0000 7f0bb5600640 1 -- 172.21.15.5:0/1333283594 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d2d8f95cd0 msgr2=0x55d2d8fa0050 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T09:55:58.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:58.944+0000 7f0bb5600640 1 --2- 172.21.15.5:0/1333283594 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d2d8f95cd0 0x55d2d8fa0050 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:58.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:58.944+0000 7f0bb5600640 1 -- 172.21.15.5:0/1333283594 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d2d8fa0590 msgr2=0x55d2d8fa2980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:55:58.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:58.944+0000 7f0bb5600640 1 --2- 172.21.15.5:0/1333283594 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d2d8fa0590 0x55d2d8fa2980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:58.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:58.944+0000 7f0bb5600640 1 -- 172.21.15.5:0/1333283594 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d2d8c6cb90 con 0x55d2d8f953b0 2024-09-10T09:55:58.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:58.944+0000 7f0bb5600640 1 --2- 172.21.15.5:0/1333283594 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d2d8f953b0 0x55d2d8f95790 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f0ba8010480 tx=0x7f0ba8010940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=78957226549e7690 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:55:58.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:58.944+0000 7f0bad600640 1 --2- 172.21.15.5:0/1063542786 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f0b980a2d50 0x7f0b980a5210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:58.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:58.945+0000 7f0ba1c00640 1 -- 172.21.15.5:0/1333283594 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ba8017070 con 0x55d2d8f953b0 2024-09-10T09:55:58.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:58.945+0000 7f0bad600640 1 -- 172.21.15.5:0/1063542786 <== mon.2 v2:172.21.15.102:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f0ba40ef960 con 0x7f0bb01008e0 2024-09-10T09:55:58.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:58.945+0000 7f0ba1c00640 1 -- 172.21.15.5:0/1333283594 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0ba8011c40 con 0x55d2d8f953b0 2024-09-10T09:55:58.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:58.945+0000 7f0ba1c00640 1 -- 172.21.15.5:0/1333283594 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ba801be20 con 0x55d2d8f953b0 2024-09-10T09:55:58.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:58.946+0000 7f0bb4c00640 1 --2- 172.21.15.5:0/1063542786 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f0b980a2d50 0x7f0b980a5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:58.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:58.946+0000 7f0bba25a740 1 -- 172.21.15.5:0/1333283594 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d2d8f953b0 msgr2=0x55d2d8f95790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:58.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:58.946+0000 7f0bba25a740 1 --2- 172.21.15.5:0/1333283594 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d2d8f953b0 0x55d2d8f95790 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f0ba8010480 tx=0x7f0ba8010940 comp rx=0 tx=0).stop 2024-09-10T09:55:58.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:58.946+0000 7f0bba25a740 1 -- 172.21.15.5:0/1333283594 shutdown_connections 2024-09-10T09:55:58.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:58.946+0000 7f0bba25a740 1 --2- 172.21.15.5:0/1333283594 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d2d8fa0590 0x55d2d8fa2980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:58.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:58.946+0000 7f0bba25a740 1 --2- 172.21.15.5:0/1333283594 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d2d8f95cd0 0x55d2d8fa0050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:58.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:58.946+0000 7f0bba25a740 1 --2- 172.21.15.5:0/1333283594 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d2d8f953b0 0x55d2d8f95790 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:58.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:58.946+0000 7f0bba25a740 1 -- 172.21.15.5:0/1333283594 >> 172.21.15.5:0/1333283594 conn(0x55d2d8f91940 msgr2=0x55d2d8f91d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:55:58.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:58.947+0000 7f0bba25a740 1 -- 172.21.15.5:0/1333283594 shutdown_connections 2024-09-10T09:55:58.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:58.947+0000 7f0bba25a740 1 -- 172.21.15.5:0/1333283594 wait complete. 2024-09-10T09:55:58.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:58.947+0000 7f0bb4c00640 1 --2- 172.21.15.5:0/1063542786 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f0b980a2d50 0x7f0b980a5210 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f0ba40c44b0 tx=0x7f0ba402a3f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:55:58.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:58.948+0000 7f0bba25a740 1 -- start start 2024-09-10T09:55:58.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:58.948+0000 7f0bba25a740 1 Processor -- start 2024-09-10T09:55:58.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:58.950+0000 7f0bba25a740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d2d8f953b0 0x55d2d8f9d620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:58.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:58.950+0000 7f0bba25a740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d2d8f95cd0 0x55d2d8f9db60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:58.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:58.950+0000 7f0bba25a740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d2d8fa0590 0x55d2d8f9e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:58.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:58.950+0000 7f0bba25a740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55d2d8fa2fb0 con 0x55d2d8f95cd0 2024-09-10T09:55:58.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:58.950+0000 7f0bba25a740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55d2d8eb5e60 con 0x55d2d8fa0590 2024-09-10T09:55:58.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:58.950+0000 7f0bba25a740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55d2d8fa3130 con 0x55d2d8f953b0 2024-09-10T09:55:58.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:58.950+0000 7f0bba25a740 10 client.0 _ll_register_callbacks cb 0x55d2d8ebacc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:55:58.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:58.951+0000 7f0bb5600640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d2d8f953b0 0x55d2d8f9d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:58.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:58.951+0000 7f0bb5600640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d2d8f953b0 0x55d2d8f9d620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:42844/0 (socket says 172.21.15.5:42844) 2024-09-10T09:55:58.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:58.951+0000 7f0bb5600640 1 -- 172.21.15.5:0/3863006728 learned_addr learned my addr 172.21.15.5:0/3863006728 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:55:58.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:58.951+0000 7f0bafe00640 1 --2- 172.21.15.5:0/3863006728 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d2d8f95cd0 0x55d2d8f9db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:58.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:58.951+0000 7f0bb5600640 1 --2- 172.21.15.5:0/3863006728 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d2d8fa0590 0x55d2d8f9e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:58.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:58.952+0000 7f0bafe00640 1 -- 172.21.15.5:0/3863006728 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d2d8f953b0 msgr2=0x55d2d8f9d620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:55:58.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:58.952+0000 7f0bafe00640 1 --2- 172.21.15.5:0/3863006728 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d2d8f953b0 0x55d2d8f9d620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:58.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:58.952+0000 7f0bafe00640 1 -- 172.21.15.5:0/3863006728 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d2d8fa0590 msgr2=0x55d2d8f9e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:55:58.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:58.952+0000 7f0bafe00640 1 --2- 172.21.15.5:0/3863006728 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d2d8fa0590 0x55d2d8f9e0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:58.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:58.952+0000 7f0bafe00640 1 -- 172.21.15.5:0/3863006728 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d2d8ec1990 con 0x55d2d8f95cd0 2024-09-10T09:55:58.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:58.952+0000 7f0bb5600640 1 --2- 172.21.15.5:0/3863006728 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d2d8f953b0 0x55d2d8f9d620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:55:58.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:58.953+0000 7f0bafe00640 1 --2- 172.21.15.5:0/3863006728 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d2d8f95cd0 0x55d2d8f9db60 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x55d2d8f92cc0 tx=0x7f0b9c022930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ea8400b756b15d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:55:58.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:58.953+0000 7f0b96000640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-10T09:55:58.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:58.953+0000 7f0b96000640 1 -- 172.21.15.5:0/3863006728 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b9c004030 con 0x55d2d8f95cd0 2024-09-10T09:55:58.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:58.953+0000 7f0bba25a740 1 -- 172.21.15.5:0/3863006728 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0ba8011c40 con 0x55d2d8f95cd0 2024-09-10T09:55:58.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:58.954+0000 7f0b96000640 1 -- 172.21.15.5:0/3863006728 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f0b9c027070 con 0x55d2d8f95cd0 2024-09-10T09:55:58.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:58.954+0000 7f0b96000640 1 -- 172.21.15.5:0/3863006728 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f0b9c035050 con 0x55d2d8f95cd0 2024-09-10T09:55:58.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:58.954+0000 7f0bba25a740 10 client.18777 fetch_fsmap learned FSMap version 42 2024-09-10T09:55:58.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:58.954+0000 7f0bba25a740 1 -- 172.21.15.5:0/3863006728 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55d2d8faef00 con 0x55d2d8f95cd0 2024-09-10T09:55:58.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:58.955+0000 7f0b96000640 1 -- 172.21.15.5:0/3863006728 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0ba8011c40 con 0x55d2d8f95cd0 2024-09-10T09:55:58.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:58.955+0000 7f0b96000640 1 -- 172.21.15.5:0/3863006728 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 42) ==== 8370+0+0 (secure 0 0 0) 0x7f0b9c003660 con 0x55d2d8f95cd0 2024-09-10T09:55:58.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:58.957+0000 7f0bba25a740 10 client.18777 fetch_fsmap finished waiting for FSMap version 42 2024-09-10T09:55:58.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:58.957+0000 7f0bba25a740 10 client.18777 resolve_mds: validated gid 24542 aka daemon mds.b 2024-09-10T09:55:58.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:58.957+0000 7f0bba25a740 20 client.18777 populate_metadata read hostname 'smithi005' 2024-09-10T09:55:58.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:58.957+0000 7f0bba25a740 1 --2- 172.21.15.5:0/3863006728 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55d2d8fb0360 0x55d2d8fb2780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:58.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:58.957+0000 7f0bba25a740 4 client.18777 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:55:58.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:58.957+0000 7f0bba25a740 1 -- 172.21.15.5:0/3863006728 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d2d8e9e310 con 0x55d2d8fb0360 2024-09-10T09:55:58.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:58.957+0000 7f0bb5600640 1 --2- 172.21.15.5:0/3863006728 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55d2d8fb0360 0x55d2d8fb2780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:58.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:58.958+0000 7f0bb5600640 1 --2- 172.21.15.5:0/3863006728 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55d2d8fb0360 0x55d2d8fb2780 crc :-1 s=READY pgs=251 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b3e6e8a6a9cf395a server_cookie=8da312b2e3b75d92 in_seq=0 out_seq=0 2024-09-10T09:55:58.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:58.958+0000 7f0b96000640 10 client.18777 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T09:55:58.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:58.959+0000 7f0b96000640 1 -- 172.21.15.5:0/3863006728 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d2d8e9e310 con 0x55d2d8fb0360 2024-09-10T09:55:58.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:58.959+0000 7f0b96000640 10 client.18777 handle_command_reply: tid=0 2024-09-10T09:55:58.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:58.959+0000 7f0bba25a740 1 client.18777 shutdown 2024-09-10T09:55:58.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:58.960+0000 7f0bba25a740 1 -- 172.21.15.5:0/3863006728 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d2d8f95cd0 msgr2=0x55d2d8f9db60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:58.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:58.960+0000 7f0bba25a740 1 --2- 172.21.15.5:0/3863006728 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d2d8f95cd0 0x55d2d8f9db60 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x55d2d8f92cc0 tx=0x7f0b9c022930 comp rx=0 tx=0).stop 2024-09-10T09:55:58.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:58.960+0000 7f0bba25a740 1 -- 172.21.15.5:0/3863006728 shutdown_connections 2024-09-10T09:55:58.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:58.960+0000 7f0bba25a740 1 --2- 172.21.15.5:0/3863006728 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55d2d8fb0360 0x55d2d8fb2780 crc :-1 s=READY pgs=251 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:58.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:58.960+0000 7f0bba25a740 1 --2- 172.21.15.5:0/3863006728 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d2d8fa0590 0x55d2d8f9e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:58.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:58.960+0000 7f0bba25a740 1 --2- 172.21.15.5:0/3863006728 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d2d8f95cd0 0x55d2d8f9db60 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:58.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:58.960+0000 7f0bba25a740 1 --2- 172.21.15.5:0/3863006728 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d2d8f953b0 0x55d2d8f9d620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:58.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:58.960+0000 7f0b96000640 0 client.18777 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T09:55:58.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:58.960+0000 7f0bba25a740 1 -- 172.21.15.5:0/3863006728 >> 172.21.15.5:0/3863006728 conn(0x55d2d8f91940 msgr2=0x55d2d8f90c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:55:58.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:58.960+0000 7f0bba25a740 1 -- 172.21.15.5:0/3863006728 shutdown_connections 2024-09-10T09:55:58.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:58.960+0000 7f0bba25a740 1 -- 172.21.15.5:0/3863006728 wait complete. 2024-09-10T09:55:58.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:58.961+0000 7f0bba25a740 20 client.18777 trim_cache size 0 max 16384 2024-09-10T09:55:58.982 INFO:tasks.workunit.client.0.smithi005.stdout: 10 69498 16.91 MB/sec execute 97 sec latency 203.738 ms 2024-09-10T09:55:58.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:58.986+0000 7f0bba25a740 1 Processor -- start 2024-09-10T09:55:58.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:58.987+0000 7f0bba25a740 1 -- start start 2024-09-10T09:55:58.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:58.987+0000 7f0bba25a740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d2d8f953b0 0x55d2d8fb8030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:58.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:58.987+0000 7f0bba25a740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d2d8f95cd0 0x55d2d8fb8570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:58.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:58.987+0000 7f0bba25a740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d2d8fa0590 0x55d2d8f9a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:58.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:58.987+0000 7f0bba25a740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f0bb01acc60 con 0x55d2d8f953b0 2024-09-10T09:55:58.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:58.987+0000 7f0bba25a740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f0bb01ae7a0 con 0x55d2d8f95cd0 2024-09-10T09:55:58.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:58.987+0000 7f0bba25a740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f0bb01ad330 con 0x55d2d8fa0590 2024-09-10T09:55:58.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:58.988+0000 7f0bb5600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d2d8f953b0 0x55d2d8fb8030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:58.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:58.988+0000 7f0bafe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d2d8f95cd0 0x55d2d8fb8570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:58.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:58.988+0000 7f0bafe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d2d8f95cd0 0x55d2d8fb8570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:42698/0 (socket says 172.21.15.5:42698) 2024-09-10T09:55:58.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:58.988+0000 7f0bafe00640 1 -- 172.21.15.5:0/86677905 learned_addr learned my addr 172.21.15.5:0/86677905 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:55:58.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:58.989+0000 7f0bb5600640 1 --2- 172.21.15.5:0/86677905 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d2d8fa0590 0x55d2d8f9a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:58.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:58.989+0000 7f0bafe00640 1 -- 172.21.15.5:0/86677905 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d2d8fa0590 msgr2=0x55d2d8f9a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:55:58.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:58.989+0000 7f0bafe00640 1 --2- 172.21.15.5:0/86677905 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d2d8fa0590 0x55d2d8f9a8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:58.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:58.989+0000 7f0bafe00640 1 -- 172.21.15.5:0/86677905 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d2d8f953b0 msgr2=0x55d2d8fb8030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:55:58.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:58.989+0000 7f0bafe00640 1 --2- 172.21.15.5:0/86677905 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d2d8f953b0 0x55d2d8fb8030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:58.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:58.989+0000 7f0bafe00640 1 -- 172.21.15.5:0/86677905 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d2d8f925d0 con 0x55d2d8f95cd0 2024-09-10T09:55:58.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:58.989+0000 7f0bb5600640 1 --2- 172.21.15.5:0/86677905 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d2d8f953b0 0x55d2d8fb8030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:55:58.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:58.990+0000 7f0bafe00640 1 --2- 172.21.15.5:0/86677905 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d2d8f95cd0 0x55d2d8fb8570 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f0b9c000c00 tx=0x7f0b9c01a5c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c8f198bcb701fa98 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:55:58.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:58.990+0000 7f0ba1200640 1 -- 172.21.15.5:0/86677905 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b9c0035f0 con 0x55d2d8f95cd0 2024-09-10T09:55:58.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:58.990+0000 7f0ba1200640 1 -- 172.21.15.5:0/86677905 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0b9c003790 con 0x55d2d8f95cd0 2024-09-10T09:55:58.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:58.991+0000 7f0ba1200640 1 -- 172.21.15.5:0/86677905 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b9c004b10 con 0x55d2d8f95cd0 2024-09-10T09:55:58.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:58.991+0000 7f0bba25a740 1 -- 172.21.15.5:0/86677905 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d2d8f95cd0 msgr2=0x55d2d8fb8570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:58.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:58.991+0000 7f0bba25a740 1 --2- 172.21.15.5:0/86677905 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d2d8f95cd0 0x55d2d8fb8570 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f0b9c000c00 tx=0x7f0b9c01a5c0 comp rx=0 tx=0).stop 2024-09-10T09:55:58.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:58.991+0000 7f0bba25a740 1 -- 172.21.15.5:0/86677905 shutdown_connections 2024-09-10T09:55:58.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:58.991+0000 7f0bba25a740 1 --2- 172.21.15.5:0/86677905 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d2d8fa0590 0x55d2d8f9a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:58.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:58.991+0000 7f0bba25a740 1 --2- 172.21.15.5:0/86677905 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d2d8f95cd0 0x55d2d8fb8570 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:58.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:58.991+0000 7f0bba25a740 1 --2- 172.21.15.5:0/86677905 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d2d8f953b0 0x55d2d8fb8030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:58.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:58.991+0000 7f0bba25a740 1 -- 172.21.15.5:0/86677905 >> 172.21.15.5:0/86677905 conn(0x55d2d8f91940 msgr2=0x55d2d8f90420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:55:58.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:58.992+0000 7f0bba25a740 1 -- 172.21.15.5:0/86677905 shutdown_connections 2024-09-10T09:55:58.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:58.992+0000 7f0bba25a740 1 -- 172.21.15.5:0/86677905 wait complete. 2024-09-10T09:55:58.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:58.992+0000 7f0bba25a740 1 -- start start 2024-09-10T09:55:58.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:58.992+0000 7f0bba25a740 1 Processor -- start 2024-09-10T09:55:58.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:58.993+0000 7f0bba25a740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d2d8f953b0 0x55d2d8fa8b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:58.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:58.993+0000 7f0bba25a740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d2d8f95cd0 0x55d2d8fa5480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:58.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:58.993+0000 7f0bba25a740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d2d8fa0590 0x55d2d8fa59c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:58.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:58.993+0000 7f0bba25a740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f0bb01ae0d0 con 0x55d2d8f95cd0 2024-09-10T09:55:58.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:58.993+0000 7f0bba25a740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f0bb01ada00 con 0x55d2d8fa0590 2024-09-10T09:55:58.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:58.993+0000 7f0bba25a740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f0bb01aac40 con 0x55d2d8f953b0 2024-09-10T09:55:58.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:58.993+0000 7f0bba25a740 10 client.0 _ll_register_callbacks cb 0x7f0b9c01a7b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:55:58.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:58.994+0000 7f0bb5600640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d2d8f953b0 0x55d2d8fa8b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:58.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:58.994+0000 7f0bb5600640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d2d8f953b0 0x55d2d8fa8b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:42868/0 (socket says 172.21.15.5:42868) 2024-09-10T09:55:58.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:58.994+0000 7f0bb5600640 1 -- 172.21.15.5:0/3294814546 learned_addr learned my addr 172.21.15.5:0/3294814546 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:55:58.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:58.994+0000 7f0bafe00640 1 --2- 172.21.15.5:0/3294814546 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d2d8f95cd0 0x55d2d8fa5480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:58.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:58.995+0000 7f0bb5600640 1 --2- 172.21.15.5:0/3294814546 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d2d8fa0590 0x55d2d8fa59c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:58.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:58.995+0000 7f0bafe00640 1 -- 172.21.15.5:0/3294814546 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d2d8f953b0 msgr2=0x55d2d8fa8b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:55:58.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:58.995+0000 7f0bafe00640 1 --2- 172.21.15.5:0/3294814546 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d2d8f953b0 0x55d2d8fa8b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:58.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:58.995+0000 7f0bafe00640 1 -- 172.21.15.5:0/3294814546 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d2d8fa0590 msgr2=0x55d2d8fa59c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:55:58.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:58.995+0000 7f0bafe00640 1 --2- 172.21.15.5:0/3294814546 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d2d8fa0590 0x55d2d8fa59c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:58.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:58.995+0000 7f0bafe00640 1 -- 172.21.15.5:0/3294814546 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d2d8fac340 con 0x55d2d8f95cd0 2024-09-10T09:55:58.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:58.995+0000 7f0bb5600640 1 --2- 172.21.15.5:0/3294814546 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d2d8fa0590 0x55d2d8fa59c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T09:55:58.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:58.996+0000 7f0bafe00640 1 --2- 172.21.15.5:0/3294814546 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d2d8f95cd0 0x55d2d8fa5480 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f0b9c003c70 tx=0x7f0b9c004f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=635cba5c2c2ca0c2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:55:58.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:58.996+0000 7f0b96000640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-10T09:55:58.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:58.996+0000 7f0b96000640 1 -- 172.21.15.5:0/3294814546 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b9c004b10 con 0x55d2d8f95cd0 2024-09-10T09:55:58.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:58.996+0000 7f0bba25a740 1 -- 172.21.15.5:0/3294814546 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0b9c003790 con 0x55d2d8f95cd0 2024-09-10T09:55:58.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:58.996+0000 7f0b96000640 1 -- 172.21.15.5:0/3294814546 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f0b9c03e030 con 0x55d2d8f95cd0 2024-09-10T09:55:58.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:58.996+0000 7f0b96000640 1 -- 172.21.15.5:0/3294814546 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f0b9c004b10 con 0x55d2d8f95cd0 2024-09-10T09:55:58.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:58.996+0000 7f0bba25a740 10 client.18783 fetch_fsmap learned FSMap version 42 2024-09-10T09:55:58.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:58.996+0000 7f0bba25a740 1 -- 172.21.15.5:0/3294814546 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55d2d8f9e5f0 con 0x55d2d8f95cd0 2024-09-10T09:55:58.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:58.997+0000 7f0b96000640 1 -- 172.21.15.5:0/3294814546 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0b9c003790 con 0x55d2d8f95cd0 2024-09-10T09:55:58.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:58.997+0000 7f0b96000640 1 -- 172.21.15.5:0/3294814546 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 42) ==== 8370+0+0 (secure 0 0 0) 0x7f0b9c020be0 con 0x55d2d8f95cd0 2024-09-10T09:55:58.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:58.997+0000 7f0bba25a740 10 client.18783 fetch_fsmap finished waiting for FSMap version 42 2024-09-10T09:55:58.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:58.997+0000 7f0bba25a740 10 client.18783 resolve_mds: validated gid 24542 aka daemon mds.b 2024-09-10T09:55:58.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:58.997+0000 7f0bba25a740 20 client.18783 populate_metadata read hostname 'smithi005' 2024-09-10T09:55:58.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:58.997+0000 7f0bba25a740 1 --2- 172.21.15.5:0/3294814546 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55d2d901c160 0x55d2d901e580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:55:58.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:58.997+0000 7f0bba25a740 4 client.18783 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 150.5, "await_for": 45.0}] 2024-09-10T09:55:58.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:58.997+0000 7f0bba25a740 1 -- 172.21.15.5:0/3294814546 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 150.5, "await_for": 45.0}) -- 0x55d2d8f08c60 con 0x55d2d901c160 2024-09-10T09:55:58.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:58.998+0000 7f0bb5600640 1 --2- 172.21.15.5:0/3294814546 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55d2d901c160 0x55d2d901e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:55:59.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:58.998+0000 7f0bb5600640 1 --2- 172.21.15.5:0/3294814546 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55d2d901c160 0x55d2d901e580 crc :-1 s=READY pgs=252 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d164886ef8e3a45c server_cookie=db48871d801e9872 in_seq=0 out_seq=0 2024-09-10T09:55:59.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:58.998+0000 7f0b96000640 10 client.18783 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T09:55:59.306 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:55:59 smithi005 ceph-mon[30350]: pgmap v922: 129 pgs: 129 active+clean; 579 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 4.2 MiB/s wr, 330 op/s 2024-09-10T09:55:59.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:55:59 smithi159 ceph-mon[35513]: pgmap v922: 129 pgs: 129 active+clean; 579 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 4.2 MiB/s wr, 330 op/s 2024-09-10T09:55:59.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:55:59 smithi102 ceph-mon[35442]: pgmap v922: 129 pgs: 129 active+clean; 579 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 4.2 MiB/s wr, 330 op/s 2024-09-10T09:55:59.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:59.610+0000 7f0b96000640 1 -- 172.21.15.5:0/3294814546 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55d2d8f08c60 con 0x55d2d901c160 2024-09-10T09:55:59.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:59.610+0000 7f0b96000640 10 client.18783 handle_command_reply: tid=0 2024-09-10T09:55:59.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:59.610+0000 7f0bba25a740 1 client.18783 shutdown 2024-09-10T09:55:59.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:59.610+0000 7f0bba25a740 1 -- 172.21.15.5:0/3294814546 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d2d8f95cd0 msgr2=0x55d2d8fa5480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:59.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:59.610+0000 7f0bba25a740 1 --2- 172.21.15.5:0/3294814546 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d2d8f95cd0 0x55d2d8fa5480 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f0b9c003c70 tx=0x7f0b9c004f60 comp rx=0 tx=0).stop 2024-09-10T09:55:59.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:59.611+0000 7f0bba25a740 1 -- 172.21.15.5:0/3294814546 shutdown_connections 2024-09-10T09:55:59.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:59.611+0000 7f0bba25a740 1 --2- 172.21.15.5:0/3294814546 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55d2d901c160 0x55d2d901e580 crc :-1 s=READY pgs=252 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:59.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:59.611+0000 7f0bba25a740 1 --2- 172.21.15.5:0/3294814546 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d2d8fa0590 0x55d2d8fa59c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:59.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:59.611+0000 7f0bba25a740 1 --2- 172.21.15.5:0/3294814546 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d2d8f95cd0 0x55d2d8fa5480 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:59.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:59.611+0000 7f0bba25a740 1 --2- 172.21.15.5:0/3294814546 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d2d8f953b0 0x55d2d8fa8b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:59.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:59.611+0000 7f0b96000640 0 client.18783 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T09:55:59.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:59.611+0000 7f0bba25a740 1 -- 172.21.15.5:0/3294814546 >> 172.21.15.5:0/3294814546 conn(0x55d2d8f91940 msgr2=0x55d2d8fab330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:55:59.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:59.611+0000 7f0bba25a740 1 -- 172.21.15.5:0/3294814546 shutdown_connections 2024-09-10T09:55:59.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:59.611+0000 7f0bba25a740 1 -- 172.21.15.5:0/3294814546 wait complete. 2024-09-10T09:55:59.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:59.611+0000 7f0bba25a740 20 client.18783 trim_cache size 0 max 16384 2024-09-10T09:55:59.614 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-10T09:55:59.614 INFO:teuthology.orchestra.run.smithi005.stdout: "epoch": 42, 2024-09-10T09:55:59.614 INFO:teuthology.orchestra.run.smithi005.stdout: "leader": 24542, 2024-09-10T09:55:59.614 INFO:teuthology.orchestra.run.smithi005.stdout: "set_version": 62, 2024-09-10T09:55:59.615 INFO:teuthology.orchestra.run.smithi005.stdout: "sets": { 2024-09-10T09:55:59.615 INFO:teuthology.orchestra.run.smithi005.stdout: "16796d73": { 2024-09-10T09:55:59.615 INFO:teuthology.orchestra.run.smithi005.stdout: "version": 62, 2024-09-10T09:55:59.615 INFO:teuthology.orchestra.run.smithi005.stdout: "age_ref": 0.0, 2024-09-10T09:55:59.615 INFO:teuthology.orchestra.run.smithi005.stdout: "state": { 2024-09-10T09:55:59.615 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "QUIESCED", 2024-09-10T09:55:59.615 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0 2024-09-10T09:55:59.615 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T09:55:59.615 INFO:teuthology.orchestra.run.smithi005.stdout: "timeout": 90.0, 2024-09-10T09:55:59.615 INFO:teuthology.orchestra.run.smithi005.stdout: "expiration": 150.5, 2024-09-10T09:55:59.615 INFO:teuthology.orchestra.run.smithi005.stdout: "members": { 2024-09-10T09:55:59.615 INFO:teuthology.orchestra.run.smithi005.stdout: "file:/": { 2024-09-10T09:55:59.616 INFO:teuthology.orchestra.run.smithi005.stdout: "excluded": false, 2024-09-10T09:55:59.616 INFO:teuthology.orchestra.run.smithi005.stdout: "state": { 2024-09-10T09:55:59.616 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "QUIESCED", 2024-09-10T09:55:59.616 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0 2024-09-10T09:55:59.616 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T09:55:59.616 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T09:55:59.616 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T09:55:59.616 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T09:55:59.616 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T09:55:59.616 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-10T09:55:59.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:59.612+0000 7f0bb7600640 1 -- 172.21.15.5:0/1063542786 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f0b980a2d50 msgr2=0x7f0b980a5210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:59.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:59.612+0000 7f0bb7600640 1 --2- 172.21.15.5:0/1063542786 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f0b980a2d50 0x7f0b980a5210 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f0ba40c44b0 tx=0x7f0ba402a3f0 comp rx=0 tx=0).stop 2024-09-10T09:55:59.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:59.612+0000 7f0bb7600640 1 -- 172.21.15.5:0/1063542786 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0bb01008e0 msgr2=0x7f0bb01121b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:55:59.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:59.612+0000 7f0bb7600640 1 --2- 172.21.15.5:0/1063542786 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0bb01008e0 0x7f0bb01121b0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f0ba4009230 tx=0x7f0ba4032d90 comp rx=0 tx=0).stop 2024-09-10T09:55:59.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:59.612+0000 7f0bb7600640 1 -- 172.21.15.5:0/1063542786 shutdown_connections 2024-09-10T09:55:59.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:59.612+0000 7f0bb7600640 1 --2- 172.21.15.5:0/1063542786 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f0b980a2d50 0x7f0b980a5210 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:59.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:59.612+0000 7f0bb7600640 1 --2- 172.21.15.5:0/1063542786 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0bb0112b20 0x7f0bb010e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:59.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:59.612+0000 7f0bb7600640 1 --2- 172.21.15.5:0/1063542786 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0bb0101290 0x7f0bb010e2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:59.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:59.612+0000 7f0bb7600640 1 --2- 172.21.15.5:0/1063542786 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0bb01008e0 0x7f0bb01121b0 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:55:59.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:59.612+0000 7f0bb7600640 1 -- 172.21.15.5:0/1063542786 >> 172.21.15.5:0/1063542786 conn(0x7f0bb0077210 msgr2=0x7f0bb0108180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:55:59.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:59.613+0000 7f0bb7600640 1 -- 172.21.15.5:0/1063542786 shutdown_connections 2024-09-10T09:55:59.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:55:59.613+0000 7f0bb7600640 1 -- 172.21.15.5:0/1063542786 wait complete. 2024-09-10T09:55:59.983 INFO:tasks.workunit.client.0.smithi005.stdout: 10 69576 16.80 MB/sec execute 98 sec latency 833.080 ms 2024-09-10T09:56:00.524 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '16796d73', quiesce took 5.6 seconds. Will release after: 24.9 2024-09-10T09:56:00.983 INFO:tasks.workunit.client.0.smithi005.stdout: 10 69576 16.63 MB/sec execute 99 sec latency 1833.154 ms 2024-09-10T09:56:01.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:56:01 smithi159 ceph-mon[35513]: pgmap v923: 129 pgs: 129 active+clean; 578 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 4.9 MiB/s wr, 473 op/s 2024-09-10T09:56:01.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:56:01 smithi102 ceph-mon[35442]: pgmap v923: 129 pgs: 129 active+clean; 578 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 4.9 MiB/s wr, 473 op/s 2024-09-10T09:56:01.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:56:01 smithi005 ceph-mon[30350]: pgmap v923: 129 pgs: 129 active+clean; 578 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 4.9 MiB/s wr, 473 op/s 2024-09-10T09:56:01.983 INFO:tasks.workunit.client.0.smithi005.stdout: 10 69576 16.46 MB/sec execute 100 sec latency 2833.265 ms 2024-09-10T09:56:02.983 INFO:tasks.workunit.client.0.smithi005.stdout: 10 69576 16.30 MB/sec execute 101 sec latency 3833.428 ms 2024-09-10T09:56:03.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:56:03 smithi159 ceph-mon[35513]: pgmap v924: 129 pgs: 129 active+clean; 579 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 3.8 MiB/s wr, 451 op/s 2024-09-10T09:56:03.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:56:03 smithi102 ceph-mon[35442]: pgmap v924: 129 pgs: 129 active+clean; 579 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 3.8 MiB/s wr, 451 op/s 2024-09-10T09:56:03.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:56:03 smithi005 ceph-mon[30350]: pgmap v924: 129 pgs: 129 active+clean; 579 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 3.8 MiB/s wr, 451 op/s 2024-09-10T09:56:03.983 INFO:tasks.workunit.client.0.smithi005.stdout: 10 69576 16.14 MB/sec execute 102 sec latency 4833.550 ms 2024-09-10T09:56:04.983 INFO:tasks.workunit.client.0.smithi005.stdout: 10 69576 15.98 MB/sec execute 103 sec latency 5833.625 ms 2024-09-10T09:56:05.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:56:05 smithi159 ceph-mon[35513]: pgmap v925: 129 pgs: 129 active+clean; 579 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 3.3 MiB/s wr, 434 op/s 2024-09-10T09:56:05.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:56:05 smithi102 ceph-mon[35442]: pgmap v925: 129 pgs: 129 active+clean; 579 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 3.3 MiB/s wr, 434 op/s 2024-09-10T09:56:05.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:56:05 smithi005 ceph-mon[30350]: pgmap v925: 129 pgs: 129 active+clean; 579 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 3.3 MiB/s wr, 434 op/s 2024-09-10T09:56:05.983 INFO:tasks.workunit.client.0.smithi005.stdout: 10 69576 15.83 MB/sec execute 104 sec latency 6833.725 ms 2024-09-10T09:56:06.983 INFO:tasks.workunit.client.0.smithi005.stdout: 10 69576 15.68 MB/sec execute 105 sec latency 7833.839 ms 2024-09-10T09:56:07.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:56:07 smithi102 ceph-mon[35442]: pgmap v926: 129 pgs: 129 active+clean; 579 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 2.8 MiB/s wr, 421 op/s 2024-09-10T09:56:07.446 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T09:56:07.502 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:56:07 smithi005 ceph-mon[30350]: pgmap v926: 129 pgs: 129 active+clean; 579 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 2.8 MiB/s wr, 421 op/s 2024-09-10T09:56:07.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:56:07 smithi159 ceph-mon[35513]: pgmap v926: 129 pgs: 129 active+clean; 579 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 2.8 MiB/s wr, 421 op/s 2024-09-10T09:56:07.833 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:56:07.983 INFO:tasks.workunit.client.0.smithi005.stdout: 10 69576 15.53 MB/sec execute 106 sec latency 8833.955 ms 2024-09-10T09:56:08.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:08.431+0000 7f3238200640 1 -- 172.21.15.5:0/2331383283 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3230109140 msgr2=0x7f3230110cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:08.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:08.431+0000 7f3238200640 1 --2- 172.21.15.5:0/2331383283 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3230109140 0x7f3230110cc0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f3228009ca0 tx=0x7f322802a8c0 comp rx=0 tx=0).stop 2024-09-10T09:56:08.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:08.431+0000 7f3238200640 1 -- 172.21.15.5:0/2331383283 shutdown_connections 2024-09-10T09:56:08.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:08.431+0000 7f3238200640 1 --2- 172.21.15.5:0/2331383283 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3230109140 0x7f3230110cc0 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:08.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:08.431+0000 7f3238200640 1 --2- 172.21.15.5:0/2331383283 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f32301049c0 0x7f3230108a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:08.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:08.431+0000 7f3238200640 1 --2- 172.21.15.5:0/2331383283 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3230104010 0x7f32301043f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:08.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:08.431+0000 7f3238200640 1 -- 172.21.15.5:0/2331383283 >> 172.21.15.5:0/2331383283 conn(0x7f32300ffa50 msgr2=0x7f3230101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:56:08.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:08.431+0000 7f3238200640 1 -- 172.21.15.5:0/2331383283 shutdown_connections 2024-09-10T09:56:08.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:08.431+0000 7f3238200640 1 -- 172.21.15.5:0/2331383283 wait complete. 2024-09-10T09:56:08.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:08.432+0000 7f3238200640 1 Processor -- start 2024-09-10T09:56:08.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:08.432+0000 7f3238200640 1 -- start start 2024-09-10T09:56:08.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:08.433+0000 7f3238200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3230104010 0x7f3230076700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:08.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:08.433+0000 7f3238200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f32301049c0 0x7f3230076c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:08.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:08.433+0000 7f3238200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3230109140 0x7f3230077180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:08.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:08.433+0000 7f3238200640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f32301141c0 con 0x7f3230104010 2024-09-10T09:56:08.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:08.433+0000 7f3238200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f3230114040 con 0x7f32301049c0 2024-09-10T09:56:08.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:08.433+0000 7f3238200640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f3230114340 con 0x7f3230109140 2024-09-10T09:56:08.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:08.433+0000 7f3235800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3230104010 0x7f3230076700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:08.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:08.433+0000 7f3235800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3230104010 0x7f3230076700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55516/0 (socket says 172.21.15.5:55516) 2024-09-10T09:56:08.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:08.433+0000 7f3235800640 1 -- 172.21.15.5:0/3102068841 learned_addr learned my addr 172.21.15.5:0/3102068841 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:56:08.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:08.433+0000 7f3234e00640 1 --2- 172.21.15.5:0/3102068841 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f32301049c0 0x7f3230076c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:08.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:08.433+0000 7f3236200640 1 --2- 172.21.15.5:0/3102068841 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3230109140 0x7f3230077180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:08.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:08.433+0000 7f3234e00640 1 -- 172.21.15.5:0/3102068841 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3230109140 msgr2=0x7f3230077180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:08.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:08.433+0000 7f3234e00640 1 --2- 172.21.15.5:0/3102068841 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3230109140 0x7f3230077180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:08.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:08.433+0000 7f3234e00640 1 -- 172.21.15.5:0/3102068841 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3230104010 msgr2=0x7f3230076700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:08.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:08.433+0000 7f3234e00640 1 --2- 172.21.15.5:0/3102068841 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3230104010 0x7f3230076700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:08.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:08.433+0000 7f3234e00640 1 -- 172.21.15.5:0/3102068841 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32301aad40 con 0x7f32301049c0 2024-09-10T09:56:08.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:08.433+0000 7f3235800640 1 --2- 172.21.15.5:0/3102068841 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3230104010 0x7f3230076700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:56:08.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:08.434+0000 7f3234e00640 1 --2- 172.21.15.5:0/3102068841 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f32301049c0 0x7f3230076c40 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f321c00ea50 tx=0x7f321c00ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:56:08.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:08.434+0000 7f322e000640 1 -- 172.21.15.5:0/3102068841 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f321c00cd00 con 0x7f32301049c0 2024-09-10T09:56:08.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:08.434+0000 7f322e000640 1 -- 172.21.15.5:0/3102068841 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f321c010910 con 0x7f32301049c0 2024-09-10T09:56:08.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:08.434+0000 7f3238200640 1 -- 172.21.15.5:0/3102068841 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32301aafd0 con 0x7f32301049c0 2024-09-10T09:56:08.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:08.434+0000 7f322e000640 1 -- 172.21.15.5:0/3102068841 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f321c018920 con 0x7f32301049c0 2024-09-10T09:56:08.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:08.434+0000 7f3238200640 1 -- 172.21.15.5:0/3102068841 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32301ab4e0 con 0x7f32301049c0 2024-09-10T09:56:08.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:08.437+0000 7f323abbc740 1 Processor -- start 2024-09-10T09:56:08.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:08.437+0000 7f323abbc740 1 -- start start 2024-09-10T09:56:08.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:08.437+0000 7f322e000640 1 -- 172.21.15.5:0/3102068841 <== mon.2 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f321c010ac0 con 0x7f32301049c0 2024-09-10T09:56:08.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:08.437+0000 7f323abbc740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cd2238eb90 0x55cd2238ef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:08.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:08.438+0000 7f323abbc740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cd2238f4b0 0x55cd22399830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:08.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:08.438+0000 7f323abbc740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cd22399d70 0x55cd2239c160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:08.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:08.438+0000 7f323abbc740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55cd222f53e0 con 0x55cd2238eb90 2024-09-10T09:56:08.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:08.438+0000 7f323abbc740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55cd2229cd80 con 0x55cd2238f4b0 2024-09-10T09:56:08.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:08.438+0000 7f323abbc740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55cd222f13a0 con 0x55cd22399d70 2024-09-10T09:56:08.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:08.438+0000 7f3236200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cd2238eb90 0x55cd2238ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:08.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:08.438+0000 7f3236200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cd2238eb90 0x55cd2238ef70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55530/0 (socket says 172.21.15.5:55530) 2024-09-10T09:56:08.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:08.438+0000 7f3236200640 1 -- 172.21.15.5:0/3746673648 learned_addr learned my addr 172.21.15.5:0/3746673648 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:56:08.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:08.438+0000 7f3235800640 1 --2- 172.21.15.5:0/3746673648 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cd2238f4b0 0x55cd22399830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:08.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:08.438+0000 7f3234e00640 1 --2- 172.21.15.5:0/3746673648 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cd22399d70 0x55cd2239c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:08.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:08.438+0000 7f3236200640 1 -- 172.21.15.5:0/3746673648 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cd22399d70 msgr2=0x55cd2239c160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:56:08.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:08.438+0000 7f3236200640 1 --2- 172.21.15.5:0/3746673648 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cd22399d70 0x55cd2239c160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:08.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:08.438+0000 7f3236200640 1 -- 172.21.15.5:0/3746673648 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cd2238f4b0 msgr2=0x55cd22399830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:56:08.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:08.438+0000 7f3236200640 1 --2- 172.21.15.5:0/3746673648 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cd2238f4b0 0x55cd22399830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:08.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:08.438+0000 7f3236200640 1 -- 172.21.15.5:0/3746673648 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cd222bbae0 con 0x55cd2238eb90 2024-09-10T09:56:08.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:08.438+0000 7f3234e00640 1 --2- 172.21.15.5:0/3746673648 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cd22399d70 0x55cd2239c160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:56:08.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:08.438+0000 7f3235800640 1 --2- 172.21.15.5:0/3746673648 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cd2238f4b0 0x55cd22399830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:56:08.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:08.439+0000 7f3236200640 1 --2- 172.21.15.5:0/3746673648 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cd2238eb90 0x55cd2238ef70 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f3228009ca0 tx=0x7f32280093e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11a41707113790c5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:56:08.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:08.439+0000 7f3222a00640 1 -- 172.21.15.5:0/3746673648 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3228031db0 con 0x55cd2238eb90 2024-09-10T09:56:08.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:08.439+0000 7f3222a00640 1 -- 172.21.15.5:0/3746673648 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3228036030 con 0x55cd2238eb90 2024-09-10T09:56:08.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:08.439+0000 7f3222a00640 1 -- 172.21.15.5:0/3746673648 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32280042a0 con 0x55cd2238eb90 2024-09-10T09:56:08.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:08.439+0000 7f322e000640 1 --2- 172.21.15.5:0/3102068841 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f32180a2b80 0x7f32180a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:08.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:08.440+0000 7f322e000640 1 -- 172.21.15.5:0/3102068841 <== mon.2 v2:172.21.15.102:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f321c0cf720 con 0x7f32301049c0 2024-09-10T09:56:08.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:08.440+0000 7f3235800640 1 --2- 172.21.15.5:0/3102068841 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f32180a2b80 0x7f32180a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:08.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:08.440+0000 7f323abbc740 1 -- 172.21.15.5:0/3746673648 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cd2238eb90 msgr2=0x55cd2238ef70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:08.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:08.440+0000 7f323abbc740 1 --2- 172.21.15.5:0/3746673648 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cd2238eb90 0x55cd2238ef70 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f3228009ca0 tx=0x7f32280093e0 comp rx=0 tx=0).stop 2024-09-10T09:56:08.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:08.440+0000 7f323abbc740 1 -- 172.21.15.5:0/3746673648 shutdown_connections 2024-09-10T09:56:08.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:08.440+0000 7f323abbc740 1 --2- 172.21.15.5:0/3746673648 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cd22399d70 0x55cd2239c160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:08.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:08.440+0000 7f323abbc740 1 --2- 172.21.15.5:0/3746673648 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cd2238f4b0 0x55cd22399830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:08.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:08.440+0000 7f323abbc740 1 --2- 172.21.15.5:0/3746673648 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cd2238eb90 0x55cd2238ef70 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:08.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:08.440+0000 7f323abbc740 1 -- 172.21.15.5:0/3746673648 >> 172.21.15.5:0/3746673648 conn(0x55cd2238b120 msgr2=0x55cd2238b500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:56:08.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:08.440+0000 7f323abbc740 1 -- 172.21.15.5:0/3746673648 shutdown_connections 2024-09-10T09:56:08.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:08.440+0000 7f323abbc740 1 -- 172.21.15.5:0/3746673648 wait complete. 2024-09-10T09:56:08.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:08.440+0000 7f3235800640 1 --2- 172.21.15.5:0/3102068841 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f32180a2b80 0x7f32180a5040 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x55cd2238bf80 tx=0x7f3224005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:56:08.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:08.441+0000 7f323abbc740 1 -- start start 2024-09-10T09:56:08.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:08.441+0000 7f323abbc740 1 Processor -- start 2024-09-10T09:56:08.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:08.441+0000 7f323abbc740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cd2238eb90 0x55cd22396c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:08.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:08.441+0000 7f323abbc740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cd2238f4b0 0x55cd22397190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:08.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:08.441+0000 7f323abbc740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cd22399d70 0x55cd223976d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:08.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:08.441+0000 7f323abbc740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55cd2239c860 con 0x55cd2238f4b0 2024-09-10T09:56:08.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:08.441+0000 7f323abbc740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55cd222b00f0 con 0x55cd2238eb90 2024-09-10T09:56:08.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:08.441+0000 7f323abbc740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55cd2239c9e0 con 0x55cd22399d70 2024-09-10T09:56:08.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:08.441+0000 7f323abbc740 10 client.0 _ll_register_callbacks cb 0x55cd222a19e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:56:08.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:08.442+0000 7f3234e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cd2238f4b0 0x55cd22397190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:08.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:08.442+0000 7f3236200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cd22399d70 0x55cd223976d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:08.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:08.442+0000 7f3235800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cd2238eb90 0x55cd22396c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:08.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:08.442+0000 7f3234e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cd2238f4b0 0x55cd22397190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55532/0 (socket says 172.21.15.5:55532) 2024-09-10T09:56:08.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:08.442+0000 7f3234e00640 1 -- 172.21.15.5:0/1861858359 learned_addr learned my addr 172.21.15.5:0/1861858359 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:56:08.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:08.442+0000 7f3236200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cd22399d70 0x55cd223976d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:57942/0 (socket says 172.21.15.5:57942) 2024-09-10T09:56:08.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:08.442+0000 7f3235800640 1 -- 172.21.15.5:0/1861858359 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cd22399d70 msgr2=0x55cd223976d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:56:08.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:08.442+0000 7f3235800640 1 --2- 172.21.15.5:0/1861858359 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cd22399d70 0x55cd223976d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:08.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:08.442+0000 7f3235800640 1 -- 172.21.15.5:0/1861858359 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cd2238f4b0 msgr2=0x55cd22397190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:56:08.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:08.442+0000 7f3235800640 1 --2- 172.21.15.5:0/1861858359 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cd2238f4b0 0x55cd22397190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:08.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:08.442+0000 7f3235800640 1 -- 172.21.15.5:0/1861858359 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cd220663c0 con 0x55cd2238eb90 2024-09-10T09:56:08.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:08.442+0000 7f3236200640 1 --2- 172.21.15.5:0/1861858359 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cd22399d70 0x55cd223976d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:56:08.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:08.442+0000 7f3235800640 1 --2- 172.21.15.5:0/1861858359 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cd2238eb90 0x55cd22396c50 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f322400fe40 tx=0x7f3224004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ca2e0a9d80c109e7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:56:08.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:08.442+0000 7f3217400640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-09-10T09:56:08.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:08.443+0000 7f3217400640 1 -- 172.21.15.5:0/1861858359 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f322402e060 con 0x55cd2238eb90 2024-09-10T09:56:08.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:08.443+0000 7f323abbc740 1 -- 172.21.15.5:0/1861858359 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3228036030 con 0x55cd2238eb90 2024-09-10T09:56:08.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:08.443+0000 7f3217400640 1 -- 172.21.15.5:0/1861858359 <== mon.2 v2:172.21.15.102:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f3224018030 con 0x55cd2238eb90 2024-09-10T09:56:08.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:08.443+0000 7f3217400640 1 -- 172.21.15.5:0/1861858359 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f3224035050 con 0x55cd2238eb90 2024-09-10T09:56:08.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:08.443+0000 7f323abbc740 10 client.28862 fetch_fsmap learned FSMap version 42 2024-09-10T09:56:08.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:08.443+0000 7f323abbc740 1 -- 172.21.15.5:0/1861858359 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55cd223a85c0 con 0x55cd2238eb90 2024-09-10T09:56:08.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:08.443+0000 7f3217400640 1 -- 172.21.15.5:0/1861858359 <== mon.2 v2:172.21.15.102:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3228036030 con 0x55cd2238eb90 2024-09-10T09:56:08.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:08.443+0000 7f3217400640 1 -- 172.21.15.5:0/1861858359 <== mon.2 v2:172.21.15.102:3300/0 5 ==== fsmap(e 42) ==== 8370+0+0 (secure 0 0 0) 0x7f322401ca80 con 0x55cd2238eb90 2024-09-10T09:56:08.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:08.444+0000 7f323abbc740 10 client.28862 fetch_fsmap finished waiting for FSMap version 42 2024-09-10T09:56:08.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:08.444+0000 7f323abbc740 10 client.28862 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T09:56:08.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:08.444+0000 7f323abbc740 20 client.28862 populate_metadata read hostname 'smithi005' 2024-09-10T09:56:08.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:08.444+0000 7f323abbc740 1 --2- 172.21.15.5:0/1861858359 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55cd223a99d0 0x55cd223abe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:08.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:08.444+0000 7f323abbc740 4 client.28862 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:56:08.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:08.444+0000 7f323abbc740 1 -- 172.21.15.5:0/1861858359 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cd22302950 con 0x55cd223a99d0 2024-09-10T09:56:08.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:08.444+0000 7f3234e00640 1 --2- 172.21.15.5:0/1861858359 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55cd223a99d0 0x55cd223abe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:08.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:08.444+0000 7f3234e00640 1 --2- 172.21.15.5:0/1861858359 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55cd223a99d0 0x55cd223abe60 crc :-1 s=READY pgs=253 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=81103f171efd5194 server_cookie=4aa0496597e57afc in_seq=0 out_seq=0 2024-09-10T09:56:08.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:08.445+0000 7f3217400640 10 client.28862 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T09:56:08.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:08.445+0000 7f3217400640 1 -- 172.21.15.5:0/1861858359 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55cd22302950 con 0x55cd223a99d0 2024-09-10T09:56:08.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:08.445+0000 7f3217400640 10 client.28862 handle_command_reply: tid=0 2024-09-10T09:56:08.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:08.445+0000 7f323abbc740 1 client.28862 shutdown 2024-09-10T09:56:08.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:08.446+0000 7f323abbc740 1 -- 172.21.15.5:0/1861858359 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cd2238eb90 msgr2=0x55cd22396c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:08.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:08.446+0000 7f323abbc740 1 --2- 172.21.15.5:0/1861858359 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cd2238eb90 0x55cd22396c50 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f322400fe40 tx=0x7f3224004730 comp rx=0 tx=0).stop 2024-09-10T09:56:08.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:08.446+0000 7f323abbc740 1 -- 172.21.15.5:0/1861858359 shutdown_connections 2024-09-10T09:56:08.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:08.446+0000 7f323abbc740 1 --2- 172.21.15.5:0/1861858359 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55cd223a99d0 0x55cd223abe60 crc :-1 s=READY pgs=253 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:08.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:08.446+0000 7f323abbc740 1 --2- 172.21.15.5:0/1861858359 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cd22399d70 0x55cd223976d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:08.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:08.446+0000 7f323abbc740 1 --2- 172.21.15.5:0/1861858359 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cd2238f4b0 0x55cd22397190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:08.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:08.446+0000 7f323abbc740 1 --2- 172.21.15.5:0/1861858359 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cd2238eb90 0x55cd22396c50 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:08.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:08.446+0000 7f323abbc740 1 -- 172.21.15.5:0/1861858359 >> 172.21.15.5:0/1861858359 conn(0x55cd2238b120 msgr2=0x55cd22390e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:56:08.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:08.446+0000 7f3217400640 0 client.28862 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T09:56:08.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:08.446+0000 7f323abbc740 1 -- 172.21.15.5:0/1861858359 shutdown_connections 2024-09-10T09:56:08.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:08.446+0000 7f323abbc740 1 -- 172.21.15.5:0/1861858359 wait complete. 2024-09-10T09:56:08.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:08.446+0000 7f323abbc740 20 client.28862 trim_cache size 0 max 16384 2024-09-10T09:56:08.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:08.464+0000 7f323abbc740 1 Processor -- start 2024-09-10T09:56:08.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:08.464+0000 7f323abbc740 1 -- start start 2024-09-10T09:56:08.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:08.464+0000 7f323abbc740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cd2238eb90 0x55cd223915f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:08.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:08.464+0000 7f323abbc740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cd2238f4b0 0x55cd223b1730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:08.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:08.464+0000 7f323abbc740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cd22399d70 0x55cd223b1c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:08.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:08.464+0000 7f323abbc740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f32301b5490 con 0x55cd2238f4b0 2024-09-10T09:56:08.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:08.464+0000 7f323abbc740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f32301b6fd0 con 0x55cd2238eb90 2024-09-10T09:56:08.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:08.464+0000 7f323abbc740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f32301b5b60 con 0x55cd22399d70 2024-09-10T09:56:08.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:08.465+0000 7f3234e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cd2238f4b0 0x55cd223b1730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:08.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:08.465+0000 7f3234e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cd2238f4b0 0x55cd223b1730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55544/0 (socket says 172.21.15.5:55544) 2024-09-10T09:56:08.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:08.465+0000 7f3234e00640 1 -- 172.21.15.5:0/1931942896 learned_addr learned my addr 172.21.15.5:0/1931942896 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:56:08.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:08.465+0000 7f3235800640 1 --2- 172.21.15.5:0/1931942896 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cd2238eb90 0x55cd223915f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:08.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:08.465+0000 7f3236200640 1 --2- 172.21.15.5:0/1931942896 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cd22399d70 0x55cd223b1c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:08.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:08.465+0000 7f3234e00640 1 -- 172.21.15.5:0/1931942896 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cd22399d70 msgr2=0x55cd223b1c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:56:08.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:08.465+0000 7f3234e00640 1 --2- 172.21.15.5:0/1931942896 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cd22399d70 0x55cd223b1c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:08.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:08.465+0000 7f3234e00640 1 -- 172.21.15.5:0/1931942896 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cd2238eb90 msgr2=0x55cd223915f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:56:08.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:08.465+0000 7f3234e00640 1 --2- 172.21.15.5:0/1931942896 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cd2238eb90 0x55cd223915f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:08.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:08.465+0000 7f3234e00640 1 -- 172.21.15.5:0/1931942896 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cd2238c1d0 con 0x55cd2238f4b0 2024-09-10T09:56:08.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:08.465+0000 7f3234e00640 1 --2- 172.21.15.5:0/1931942896 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cd2238f4b0 0x55cd223b1730 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f321c00c6a0 tx=0x7f321c009700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=274f54dcf55c4a81 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:56:08.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:08.466+0000 7f3222000640 1 -- 172.21.15.5:0/1931942896 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f321c09ce60 con 0x55cd2238f4b0 2024-09-10T09:56:08.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:08.466+0000 7f3222000640 1 -- 172.21.15.5:0/1931942896 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f321c0a47a0 con 0x55cd2238f4b0 2024-09-10T09:56:08.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:08.466+0000 7f3222000640 1 -- 172.21.15.5:0/1931942896 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f321c086040 con 0x55cd2238f4b0 2024-09-10T09:56:08.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:08.466+0000 7f323abbc740 1 -- 172.21.15.5:0/1931942896 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cd2238f4b0 msgr2=0x55cd223b1730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:08.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:08.466+0000 7f323abbc740 1 --2- 172.21.15.5:0/1931942896 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cd2238f4b0 0x55cd223b1730 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f321c00c6a0 tx=0x7f321c009700 comp rx=0 tx=0).stop 2024-09-10T09:56:08.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:08.466+0000 7f323abbc740 1 -- 172.21.15.5:0/1931942896 shutdown_connections 2024-09-10T09:56:08.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:08.466+0000 7f323abbc740 1 --2- 172.21.15.5:0/1931942896 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cd22399d70 0x55cd223b1c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:08.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:08.466+0000 7f323abbc740 1 --2- 172.21.15.5:0/1931942896 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cd2238f4b0 0x55cd223b1730 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:08.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:08.466+0000 7f323abbc740 1 --2- 172.21.15.5:0/1931942896 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cd2238eb90 0x55cd223915f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:08.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:08.466+0000 7f323abbc740 1 -- 172.21.15.5:0/1931942896 >> 172.21.15.5:0/1931942896 conn(0x55cd2238b120 msgr2=0x55cd22398cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:56:08.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:08.466+0000 7f323abbc740 1 -- 172.21.15.5:0/1931942896 shutdown_connections 2024-09-10T09:56:08.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:08.466+0000 7f323abbc740 1 -- 172.21.15.5:0/1931942896 wait complete. 2024-09-10T09:56:08.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:08.467+0000 7f323abbc740 1 -- start start 2024-09-10T09:56:08.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:08.467+0000 7f323abbc740 1 Processor -- start 2024-09-10T09:56:08.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:08.467+0000 7f323abbc740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cd2238eb90 0x55cd223a2470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:08.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:08.467+0000 7f323abbc740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cd2238f4b0 0x55cd2239d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:08.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:08.467+0000 7f323abbc740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cd22399d70 0x55cd2239dde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:08.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:08.467+0000 7f323abbc740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f32301b6900 con 0x55cd2238eb90 2024-09-10T09:56:08.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:08.467+0000 7f323abbc740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f32301b6230 con 0x55cd2238f4b0 2024-09-10T09:56:08.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:08.467+0000 7f323abbc740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f32301b3470 con 0x55cd22399d70 2024-09-10T09:56:08.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:08.467+0000 7f323abbc740 10 client.0 _ll_register_callbacks cb 0x7f322400fd00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:56:08.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:08.468+0000 7f3235800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cd2238eb90 0x55cd223a2470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:08.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:08.468+0000 7f3235800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cd2238eb90 0x55cd223a2470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55556/0 (socket says 172.21.15.5:55556) 2024-09-10T09:56:08.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:08.468+0000 7f3235800640 1 -- 172.21.15.5:0/3325329446 learned_addr learned my addr 172.21.15.5:0/3325329446 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:56:08.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:08.468+0000 7f3234e00640 1 --2- 172.21.15.5:0/3325329446 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cd2238f4b0 0x55cd2239d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:08.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:08.468+0000 7f3236200640 1 --2- 172.21.15.5:0/3325329446 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cd22399d70 0x55cd2239dde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:08.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:08.468+0000 7f3235800640 1 -- 172.21.15.5:0/3325329446 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cd22399d70 msgr2=0x55cd2239dde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:56:08.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:08.468+0000 7f3235800640 1 --2- 172.21.15.5:0/3325329446 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cd22399d70 0x55cd2239dde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:08.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:08.468+0000 7f3235800640 1 -- 172.21.15.5:0/3325329446 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cd2238f4b0 msgr2=0x55cd2239d7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:56:08.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:08.468+0000 7f3235800640 1 --2- 172.21.15.5:0/3325329446 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cd2238f4b0 0x55cd2239d7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:08.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:08.468+0000 7f3235800640 1 -- 172.21.15.5:0/3325329446 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cd2238bdb0 con 0x55cd2238eb90 2024-09-10T09:56:08.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:08.468+0000 7f3236200640 1 --2- 172.21.15.5:0/3325329446 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cd22399d70 0x55cd2239dde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:56:08.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:08.468+0000 7f3235800640 1 --2- 172.21.15.5:0/3325329446 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cd2238eb90 0x55cd223a2470 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f3224009640 tx=0x7f322400e5e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3a71d79f266b8bc server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:56:08.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:08.468+0000 7f3217400640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-10T09:56:08.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:08.469+0000 7f3217400640 1 -- 172.21.15.5:0/3325329446 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f322400ea60 con 0x55cd2238eb90 2024-09-10T09:56:08.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:08.469+0000 7f3217400640 1 -- 172.21.15.5:0/3325329446 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f322400ec00 con 0x55cd2238eb90 2024-09-10T09:56:08.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:08.469+0000 7f323abbc740 1 -- 172.21.15.5:0/3325329446 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f321c0a47a0 con 0x55cd2238eb90 2024-09-10T09:56:08.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:08.469+0000 7f3217400640 1 -- 172.21.15.5:0/3325329446 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f3224046050 con 0x55cd2238eb90 2024-09-10T09:56:08.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:08.469+0000 7f323abbc740 10 client.18801 fetch_fsmap learned FSMap version 42 2024-09-10T09:56:08.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:08.469+0000 7f323abbc740 1 -- 172.21.15.5:0/3325329446 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55cd22414620 con 0x55cd2238eb90 2024-09-10T09:56:08.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:08.469+0000 7f3217400640 1 -- 172.21.15.5:0/3325329446 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f321c0a47a0 con 0x55cd2238eb90 2024-09-10T09:56:08.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:08.469+0000 7f3217400640 1 -- 172.21.15.5:0/3325329446 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 42) ==== 8370+0+0 (secure 0 0 0) 0x7f3224041030 con 0x55cd2238eb90 2024-09-10T09:56:08.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:08.469+0000 7f323abbc740 10 client.18801 fetch_fsmap finished waiting for FSMap version 42 2024-09-10T09:56:08.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:08.469+0000 7f323abbc740 10 client.18801 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T09:56:08.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:08.469+0000 7f323abbc740 20 client.18801 populate_metadata read hostname 'smithi005' 2024-09-10T09:56:08.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:08.469+0000 7f323abbc740 1 --2- 172.21.15.5:0/3325329446 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55cd22414d90 0x55cd224171b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:08.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:08.469+0000 7f323abbc740 4 client.18801 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T09:56:08.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:08.469+0000 7f323abbc740 1 -- 172.21.15.5:0/3325329446 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55cd222fce60 con 0x55cd22414d90 2024-09-10T09:56:08.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:08.470+0000 7f3234e00640 1 --2- 172.21.15.5:0/3325329446 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55cd22414d90 0x55cd224171b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:08.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:08.470+0000 7f3234e00640 1 --2- 172.21.15.5:0/3325329446 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55cd22414d90 0x55cd224171b0 crc :-1 s=READY pgs=254 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=23f7a6479fad4f5c server_cookie=f48f8fb1280ae928 in_seq=0 out_seq=0 2024-09-10T09:56:08.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:08.470+0000 7f3217400640 10 client.18801 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T09:56:08.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:08.470+0000 7f3217400640 1 -- 172.21.15.5:0/3325329446 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55cd222fce60 con 0x55cd22414d90 2024-09-10T09:56:08.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:08.470+0000 7f3217400640 10 client.18801 handle_command_reply: tid=0 2024-09-10T09:56:08.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:08.470+0000 7f323abbc740 1 client.18801 shutdown 2024-09-10T09:56:08.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:08.471+0000 7f323abbc740 1 -- 172.21.15.5:0/3325329446 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cd2238eb90 msgr2=0x55cd223a2470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:08.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:08.471+0000 7f323abbc740 1 --2- 172.21.15.5:0/3325329446 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cd2238eb90 0x55cd223a2470 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f3224009640 tx=0x7f322400e5e0 comp rx=0 tx=0).stop 2024-09-10T09:56:08.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:08.471+0000 7f323abbc740 1 -- 172.21.15.5:0/3325329446 shutdown_connections 2024-09-10T09:56:08.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:08.471+0000 7f323abbc740 1 --2- 172.21.15.5:0/3325329446 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55cd22414d90 0x55cd224171b0 crc :-1 s=READY pgs=254 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:08.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:08.471+0000 7f323abbc740 1 --2- 172.21.15.5:0/3325329446 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cd22399d70 0x55cd2239dde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:08.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:08.471+0000 7f323abbc740 1 --2- 172.21.15.5:0/3325329446 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cd2238f4b0 0x55cd2239d7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:08.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:08.471+0000 7f3217400640 0 client.18801 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T09:56:08.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:08.471+0000 7f323abbc740 1 --2- 172.21.15.5:0/3325329446 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cd2238eb90 0x55cd223a2470 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:08.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:08.471+0000 7f323abbc740 1 -- 172.21.15.5:0/3325329446 >> 172.21.15.5:0/3325329446 conn(0x55cd2238b120 msgr2=0x55cd223899a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:56:08.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:08.471+0000 7f323abbc740 1 -- 172.21.15.5:0/3325329446 shutdown_connections 2024-09-10T09:56:08.559 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-10T09:56:08.559 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "no active scrubs running", 2024-09-10T09:56:08.559 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": {} 2024-09-10T09:56:08.559 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-10T09:56:08.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:08.471+0000 7f323abbc740 1 -- 172.21.15.5:0/3325329446 wait complete. 2024-09-10T09:56:08.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:08.471+0000 7f323abbc740 20 client.18801 trim_cache size 0 max 16384 2024-09-10T09:56:08.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:08.472+0000 7f3238200640 1 -- 172.21.15.5:0/3102068841 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f32180a2b80 msgr2=0x7f32180a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:08.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:08.472+0000 7f3238200640 1 --2- 172.21.15.5:0/3102068841 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f32180a2b80 0x7f32180a5040 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x55cd2238bf80 tx=0x7f3224005f90 comp rx=0 tx=0).stop 2024-09-10T09:56:08.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:08.472+0000 7f3238200640 1 -- 172.21.15.5:0/3102068841 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f32301049c0 msgr2=0x7f3230076c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:08.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:08.472+0000 7f3238200640 1 --2- 172.21.15.5:0/3102068841 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f32301049c0 0x7f3230076c40 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f321c00ea50 tx=0x7f321c00ef10 comp rx=0 tx=0).stop 2024-09-10T09:56:08.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:08.472+0000 7f3238200640 1 -- 172.21.15.5:0/3102068841 shutdown_connections 2024-09-10T09:56:08.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:08.472+0000 7f3238200640 1 --2- 172.21.15.5:0/3102068841 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f32180a2b80 0x7f32180a5040 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:08.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:08.472+0000 7f3238200640 1 --2- 172.21.15.5:0/3102068841 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3230109140 0x7f3230077180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:08.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:08.472+0000 7f3238200640 1 --2- 172.21.15.5:0/3102068841 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f32301049c0 0x7f3230076c40 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:08.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:08.472+0000 7f3238200640 1 --2- 172.21.15.5:0/3102068841 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3230104010 0x7f3230076700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:08.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:08.472+0000 7f3238200640 1 -- 172.21.15.5:0/3102068841 >> 172.21.15.5:0/3102068841 conn(0x7f32300ffa50 msgr2=0x7f3230100d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:56:08.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:08.472+0000 7f3238200640 1 -- 172.21.15.5:0/3102068841 shutdown_connections 2024-09-10T09:56:08.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:08.472+0000 7f3238200640 1 -- 172.21.15.5:0/3102068841 wait complete. 2024-09-10T09:56:08.984 INFO:tasks.workunit.client.0.smithi005.stdout: 10 69576 15.39 MB/sec execute 107 sec latency 9834.098 ms 2024-09-10T09:56:09.277 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:56:09 smithi005 ceph-mon[30350]: pgmap v927: 129 pgs: 129 active+clean; 579 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 1.9 MiB/s wr, 271 op/s 2024-09-10T09:56:09.315 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-10T09:56:09.315 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T09:56:09.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:56:09 smithi102 ceph-mon[35442]: pgmap v927: 129 pgs: 129 active+clean; 579 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 1.9 MiB/s wr, 271 op/s 2024-09-10T09:56:09.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:56:09 smithi159 ceph-mon[35513]: pgmap v927: 129 pgs: 129 active+clean; 579 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 1.9 MiB/s wr, 271 op/s 2024-09-10T09:56:09.707 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:56:09.984 INFO:tasks.workunit.client.0.smithi005.stdout: 10 69576 15.24 MB/sec execute 108 sec latency 10834.171 ms 2024-09-10T09:56:10.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:10.282+0000 7f6ffca00640 1 -- 172.21.15.5:0/3906089643 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6ff81079d0 msgr2=0x7f6ff8107db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:10.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:10.282+0000 7f6ffca00640 1 --2- 172.21.15.5:0/3906089643 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6ff81079d0 0x7f6ff8107db0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f6fe8009930 tx=0x7f6fe802a440 comp rx=0 tx=0).stop 2024-09-10T09:56:10.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:10.282+0000 7f6ffca00640 1 -- 172.21.15.5:0/3906089643 shutdown_connections 2024-09-10T09:56:10.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:10.282+0000 7f6ffca00640 1 --2- 172.21.15.5:0/3906089643 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6ff8102370 0x7f6ff810e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:10.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:10.282+0000 7f6ffca00640 1 --2- 172.21.15.5:0/3906089643 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6ff81019d0 0x7f6ff8101e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:10.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:10.282+0000 7f6ffca00640 1 --2- 172.21.15.5:0/3906089643 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6ff81079d0 0x7f6ff8107db0 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:10.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:10.282+0000 7f6ffca00640 1 -- 172.21.15.5:0/3906089643 >> 172.21.15.5:0/3906089643 conn(0x7f6ff80fd710 msgr2=0x7f6ff80ffb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:56:10.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:10.282+0000 7f6ffca00640 1 -- 172.21.15.5:0/3906089643 shutdown_connections 2024-09-10T09:56:10.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:10.283+0000 7f6ffca00640 1 -- 172.21.15.5:0/3906089643 wait complete. 2024-09-10T09:56:10.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:10.283+0000 7f6ffca00640 1 Processor -- start 2024-09-10T09:56:10.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:10.284+0000 7f6ffca00640 1 -- start start 2024-09-10T09:56:10.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:10.284+0000 7f6ffca00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6ff81019d0 0x7f6ff80744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:10.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:10.284+0000 7f6ffca00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6ff8102370 0x7f6ff8074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:10.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:10.284+0000 7f6ffca00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6ff81079d0 0x7f6ff8074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:10.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:10.284+0000 7f6ffca00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f6ff8111da0 con 0x7f6ff81019d0 2024-09-10T09:56:10.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:10.284+0000 7f6ffca00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f6ff8111c20 con 0x7f6ff81079d0 2024-09-10T09:56:10.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:10.284+0000 7f6ffca00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f6ff8111f20 con 0x7f6ff8102370 2024-09-10T09:56:10.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:10.284+0000 7f6ff7400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6ff81019d0 0x7f6ff80744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:10.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:10.284+0000 7f6ff7e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6ff81079d0 0x7f6ff8074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:10.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:10.284+0000 7f6ff6a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6ff8102370 0x7f6ff8074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:10.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:10.284+0000 7f6ff7e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6ff81079d0 0x7f6ff8074f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:38100/0 (socket says 172.21.15.5:38100) 2024-09-10T09:56:10.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:10.284+0000 7f6ff7e00640 1 -- 172.21.15.5:0/2563854434 learned_addr learned my addr 172.21.15.5:0/2563854434 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:56:10.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:10.284+0000 7f6ff6a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6ff8102370 0x7f6ff8074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:57970/0 (socket says 172.21.15.5:57970) 2024-09-10T09:56:10.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:10.285+0000 7f6ff6a00640 1 -- 172.21.15.5:0/2563854434 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6ff81079d0 msgr2=0x7f6ff8074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:10.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:10.285+0000 7f6ff6a00640 1 --2- 172.21.15.5:0/2563854434 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6ff81079d0 0x7f6ff8074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:10.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:10.285+0000 7f6ff6a00640 1 -- 172.21.15.5:0/2563854434 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6ff81019d0 msgr2=0x7f6ff80744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:10.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:10.285+0000 7f6ff6a00640 1 --2- 172.21.15.5:0/2563854434 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6ff81019d0 0x7f6ff80744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:10.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:10.285+0000 7f6ff6a00640 1 -- 172.21.15.5:0/2563854434 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ff81a6780 con 0x7f6ff8102370 2024-09-10T09:56:10.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:10.285+0000 7f6ff6a00640 1 --2- 172.21.15.5:0/2563854434 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6ff8102370 0x7f6ff8074a30 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f6fe400e9f0 tx=0x7f6fe400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:56:10.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:10.286+0000 7f6fefe00640 1 -- 172.21.15.5:0/2563854434 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fe400ccc0 con 0x7f6ff8102370 2024-09-10T09:56:10.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:10.286+0000 7f6ffca00640 1 -- 172.21.15.5:0/2563854434 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ff81a6a70 con 0x7f6ff8102370 2024-09-10T09:56:10.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:10.286+0000 7f6fefe00640 1 -- 172.21.15.5:0/2563854434 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6fe4010910 con 0x7f6ff8102370 2024-09-10T09:56:10.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:10.286+0000 7f6fefe00640 1 -- 172.21.15.5:0/2563854434 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fe4018960 con 0x7f6ff8102370 2024-09-10T09:56:10.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:10.286+0000 7f6ffca00640 1 -- 172.21.15.5:0/2563854434 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ff81a7000 con 0x7f6ff8102370 2024-09-10T09:56:10.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:10.287+0000 7f6ffca00640 1 -- 172.21.15.5:0/2563854434 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ff8101e30 con 0x7f6ff8102370 2024-09-10T09:56:10.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:10.289+0000 7f6fefe00640 1 -- 172.21.15.5:0/2563854434 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f6fe4010ac0 con 0x7f6ff8102370 2024-09-10T09:56:10.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:10.291+0000 7f6fefe00640 1 --2- 172.21.15.5:0/2563854434 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f6fdc0a2b80 0x7f6fdc0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:10.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:10.291+0000 7f6fefe00640 1 -- 172.21.15.5:0/2563854434 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f6fe40cfbc0 con 0x7f6ff8102370 2024-09-10T09:56:10.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:10.291+0000 7f6ff7400640 1 --2- 172.21.15.5:0/2563854434 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f6fdc0a2b80 0x7f6fdc0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:10.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:10.292+0000 7f6ff7400640 1 --2- 172.21.15.5:0/2563854434 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f6fdc0a2b80 0x7f6fdc0a5040 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f6fe8009900 tx=0x7f6fe80058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:56:10.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:10.292+0000 7f6fefe00640 1 -- 172.21.15.5:0/2563854434 <== mon.1 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f6fe409c080 con 0x7f6ff8102370 2024-09-10T09:56:10.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:56:10 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:56:10.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:10.517+0000 7f6ffca00640 1 -- 172.21.15.5:0/2563854434 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6ff8109f40 con 0x7f6ff8102370 2024-09-10T09:56:10.520 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:56:10 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:56:10.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:10.518+0000 7f6fefe00640 1 -- 172.21.15.5:0/2563854434 <== mon.1 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 42 v42) ==== 94+0+11969 (secure 0 0 0) 0x7f6fe40a0380 con 0x7f6ff8102370 2024-09-10T09:56:10.521 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-10T09:56:10.521 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":42,"btime":"2024-09-10T09:55:06:670916+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14454,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/2970594311","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2970594311},{"type":"v1","addr":"172.21.15.5:6835","nonce":2970594311}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14514,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6839/3616502174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":3616502174},{"type":"v1","addr":"172.21.15.5:6839","nonce":3616502174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":42,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T09:35:29.626287+0000","modified":"2024-09-10T09:55:06.419829+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24542,"mds_1":24535,"mds_2":24511,"mds_3":24506,"mds_4":24443},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14520":{"gid":14520,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/2073599842","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":2073599842},{"type":"v1","addr":"172.21.15.5:6841","nonce":2073599842}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"k","rank":4,"incarnation":26,"state":"up:active","state_seq":24,"addr":"172.21.15.102:6833/2457007427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":2457007427},{"type":"v1","addr":"172.21.15.102:6833","nonce":2457007427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/356445159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":356445159},{"type":"v1","addr":"172.21.15.159:6833","nonce":356445159}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6837/2894314507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":2894314507},{"type":"v1","addr":"172.21.15.5:6837","nonce":2894314507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.102:6835/1647549111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":1647549111},{"type":"v1","addr":"172.21.15.102:6835","nonce":1647549111}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6835/952657242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":952657242},{"type":"v1","addr":"172.21.15.159:6835","nonce":952657242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":19,"addr":"172.21.15.102:6837/2689776764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":2689776764},{"type":"v1","addr":"172.21.15.102:6837","nonce":2689776764}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.159:6837/620627639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":620627639},{"type":"v1","addr":"172.21.15.159:6837","nonce":620627639}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24535":{"gid":24535,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":18,"addr":"172.21.15.159:6839/2073739599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":2073739599},{"type":"v1","addr":"172.21.15.159:6839","nonce":2073739599}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24542":{"gid":24542,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.102:6839/335409749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6838","nonce":335409749},{"type":"v1","addr":"172.21.15.102:6839","nonce":335409749}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24542,"qdb_cluster":[24542,24535,24511,24506,24443]},"id":1}]} 2024-09-10T09:56:10.521 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 42 2024-09-10T09:56:10.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:10.521+0000 7f6ffca00640 1 -- 172.21.15.5:0/2563854434 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f6fdc0a2b80 msgr2=0x7f6fdc0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:10.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:10.521+0000 7f6ffca00640 1 --2- 172.21.15.5:0/2563854434 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f6fdc0a2b80 0x7f6fdc0a5040 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f6fe8009900 tx=0x7f6fe80058d0 comp rx=0 tx=0).stop 2024-09-10T09:56:10.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:10.521+0000 7f6ffca00640 1 -- 172.21.15.5:0/2563854434 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6ff8102370 msgr2=0x7f6ff8074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:10.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:10.521+0000 7f6ffca00640 1 --2- 172.21.15.5:0/2563854434 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6ff8102370 0x7f6ff8074a30 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f6fe400e9f0 tx=0x7f6fe400eeb0 comp rx=0 tx=0).stop 2024-09-10T09:56:10.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:10.521+0000 7f6ffca00640 1 -- 172.21.15.5:0/2563854434 shutdown_connections 2024-09-10T09:56:10.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:10.521+0000 7f6ffca00640 1 --2- 172.21.15.5:0/2563854434 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f6fdc0a2b80 0x7f6fdc0a5040 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:10.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:10.521+0000 7f6ffca00640 1 --2- 172.21.15.5:0/2563854434 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6ff81079d0 0x7f6ff8074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:10.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:10.521+0000 7f6ffca00640 1 --2- 172.21.15.5:0/2563854434 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6ff8102370 0x7f6ff8074a30 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:10.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:10.521+0000 7f6ffca00640 1 --2- 172.21.15.5:0/2563854434 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6ff81019d0 0x7f6ff80744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:10.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:10.521+0000 7f6ffca00640 1 -- 172.21.15.5:0/2563854434 >> 172.21.15.5:0/2563854434 conn(0x7f6ff80fd710 msgr2=0x7f6ff80fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:56:10.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:10.521+0000 7f6ffca00640 1 -- 172.21.15.5:0/2563854434 shutdown_connections 2024-09-10T09:56:10.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:10.521+0000 7f6ffca00640 1 -- 172.21.15.5:0/2563854434 wait complete. 2024-09-10T09:56:10.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:56:10 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:56:10.984 INFO:tasks.workunit.client.0.smithi005.stdout: 10 69576 15.10 MB/sec execute 109 sec latency 11834.286 ms 2024-09-10T09:56:11.182 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:56:11 smithi005 ceph-mon[30350]: pgmap v928: 129 pgs: 129 active+clean; 579 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.5 MiB/s wr, 221 op/s 2024-09-10T09:56:11.183 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:56:11 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:56:11.183 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:56:11 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:56:11.183 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:56:11 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:56:11.183 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:56:11 smithi005 ceph-mon[30350]: from='client.? 172.21.15.5:0/2563854434' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:56:11.219 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-10T09:56:11.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:56:11 smithi102 ceph-mon[35442]: pgmap v928: 129 pgs: 129 active+clean; 579 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.5 MiB/s wr, 221 op/s 2024-09-10T09:56:11.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:56:11 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:56:11.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:56:11 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:56:11.385 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:56:11 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:56:11.385 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:56:11 smithi102 ceph-mon[35442]: from='client.? 172.21.15.5:0/2563854434' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:56:11.599 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:56:11.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:56:11 smithi159 ceph-mon[35513]: pgmap v928: 129 pgs: 129 active+clean; 579 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.5 MiB/s wr, 221 op/s 2024-09-10T09:56:11.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:56:11 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:56:11.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:56:11 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:56:11.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:56:11 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:56:11.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:56:11 smithi159 ceph-mon[35513]: from='client.? 172.21.15.5:0/2563854434' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:56:11.984 INFO:tasks.workunit.client.0.smithi005.stdout: 10 69576 14.97 MB/sec execute 110 sec latency 12834.398 ms 2024-09-10T09:56:12.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:12.149+0000 7f818ec00640 1 -- 172.21.15.5:0/2082996831 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8188103e30 msgr2=0x7f8188104210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:12.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:12.149+0000 7f818ec00640 1 --2- 172.21.15.5:0/2082996831 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8188103e30 0x7f8188104210 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f817c009900 tx=0x7f817c02a410 comp rx=0 tx=0).stop 2024-09-10T09:56:12.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:12.149+0000 7f818ec00640 1 -- 172.21.15.5:0/2082996831 shutdown_connections 2024-09-10T09:56:12.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:12.149+0000 7f818ec00640 1 --2- 172.21.15.5:0/2082996831 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8188108f60 0x7f8188110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:12.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:12.149+0000 7f818ec00640 1 --2- 172.21.15.5:0/2082996831 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f81881047e0 0x7f8188108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:12.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:12.149+0000 7f818ec00640 1 --2- 172.21.15.5:0/2082996831 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8188103e30 0x7f8188104210 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:12.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:12.149+0000 7f818ec00640 1 -- 172.21.15.5:0/2082996831 >> 172.21.15.5:0/2082996831 conn(0x7f81880ff8d0 msgr2=0x7f8188101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:56:12.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:12.149+0000 7f818ec00640 1 -- 172.21.15.5:0/2082996831 shutdown_connections 2024-09-10T09:56:12.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:12.149+0000 7f818ec00640 1 -- 172.21.15.5:0/2082996831 wait complete. 2024-09-10T09:56:12.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:12.150+0000 7f818ec00640 1 Processor -- start 2024-09-10T09:56:12.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:12.150+0000 7f818ec00640 1 -- start start 2024-09-10T09:56:12.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:12.150+0000 7f818ec00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8188103e30 0x7f818819a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:12.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:12.150+0000 7f818ec00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f81881047e0 0x7f818819a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:12.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:12.150+0000 7f818ec00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8188108f60 0x7f818819ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:12.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:12.150+0000 7f818ec00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f8188114010 con 0x7f8188108f60 2024-09-10T09:56:12.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:12.150+0000 7f818ec00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f8188113e90 con 0x7f81881047e0 2024-09-10T09:56:12.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:12.150+0000 7f818ec00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f8188114190 con 0x7f8188103e30 2024-09-10T09:56:12.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:12.151+0000 7f818d800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8188103e30 0x7f818819a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:12.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:12.151+0000 7f818e200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8188108f60 0x7f818819ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:12.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:12.151+0000 7f818d800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8188103e30 0x7f818819a4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:53878/0 (socket says 172.21.15.5:53878) 2024-09-10T09:56:12.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:12.151+0000 7f818d800640 1 -- 172.21.15.5:0/2330784785 learned_addr learned my addr 172.21.15.5:0/2330784785 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:56:12.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:12.151+0000 7f818ce00640 1 --2- 172.21.15.5:0/2330784785 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f81881047e0 0x7f818819a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:12.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:12.151+0000 7f818d800640 1 -- 172.21.15.5:0/2330784785 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f81881047e0 msgr2=0x7f818819a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:12.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:12.151+0000 7f818d800640 1 --2- 172.21.15.5:0/2330784785 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f81881047e0 0x7f818819a9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:12.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:12.151+0000 7f818d800640 1 -- 172.21.15.5:0/2330784785 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8188108f60 msgr2=0x7f818819ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:12.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:12.151+0000 7f818d800640 1 --2- 172.21.15.5:0/2330784785 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8188108f60 0x7f818819ed70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:12.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:12.151+0000 7f818d800640 1 -- 172.21.15.5:0/2330784785 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f818819f450 con 0x7f8188103e30 2024-09-10T09:56:12.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:12.151+0000 7f818e200640 1 --2- 172.21.15.5:0/2330784785 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8188108f60 0x7f818819ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:56:12.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:12.152+0000 7f818d800640 1 --2- 172.21.15.5:0/2330784785 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8188103e30 0x7f818819a4a0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f817c004770 tx=0x7f817c003c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:56:12.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:12.152+0000 7f8186000640 1 -- 172.21.15.5:0/2330784785 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f817c03f070 con 0x7f8188103e30 2024-09-10T09:56:12.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:12.152+0000 7f818ec00640 1 -- 172.21.15.5:0/2330784785 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f818819f6e0 con 0x7f8188103e30 2024-09-10T09:56:12.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:12.152+0000 7f8186000640 1 -- 172.21.15.5:0/2330784785 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f817c031ca0 con 0x7f8188103e30 2024-09-10T09:56:12.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:12.152+0000 7f8186000640 1 -- 172.21.15.5:0/2330784785 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f817c035030 con 0x7f8188103e30 2024-09-10T09:56:12.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:12.152+0000 7f818ec00640 1 -- 172.21.15.5:0/2330784785 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81881a7020 con 0x7f8188103e30 2024-09-10T09:56:12.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:12.156+0000 7f8192cd7740 1 Processor -- start 2024-09-10T09:56:12.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:12.156+0000 7f8192cd7740 1 -- start start 2024-09-10T09:56:12.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:12.157+0000 7f8186000640 1 -- 172.21.15.5:0/2330784785 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f817c042070 con 0x7f8188103e30 2024-09-10T09:56:12.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:12.157+0000 7f8192cd7740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f14ab90e40 0x55f14ab91220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:12.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:12.157+0000 7f8192cd7740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f14ab91760 0x55f14ab9bae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:12.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:12.157+0000 7f8192cd7740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f14ab9c020 0x55f14ab9e410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:12.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:12.157+0000 7f8192cd7740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55f14aaf73e0 con 0x55f14ab90e40 2024-09-10T09:56:12.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:12.157+0000 7f8192cd7740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55f14aa9ed80 con 0x55f14ab91760 2024-09-10T09:56:12.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:12.157+0000 7f8192cd7740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55f14aaf33a0 con 0x55f14ab9c020 2024-09-10T09:56:12.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:12.157+0000 7f818d800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f14ab91760 0x55f14ab9bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:12.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:12.157+0000 7f818e200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f14ab90e40 0x55f14ab91220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:12.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:12.157+0000 7f818d800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f14ab91760 0x55f14ab9bae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:43868/0 (socket says 172.21.15.5:43868) 2024-09-10T09:56:12.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:12.157+0000 7f818d800640 1 -- 172.21.15.5:0/40964162 learned_addr learned my addr 172.21.15.5:0/40964162 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:56:12.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:12.157+0000 7f818ce00640 1 --2- 172.21.15.5:0/40964162 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f14ab9c020 0x55f14ab9e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:12.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:12.158+0000 7f818d800640 1 -- 172.21.15.5:0/40964162 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f14ab9c020 msgr2=0x55f14ab9e410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:56:12.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:12.158+0000 7f818d800640 1 --2- 172.21.15.5:0/40964162 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f14ab9c020 0x55f14ab9e410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:12.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:12.158+0000 7f818d800640 1 -- 172.21.15.5:0/40964162 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f14ab90e40 msgr2=0x55f14ab91220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:56:12.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:12.158+0000 7f818d800640 1 --2- 172.21.15.5:0/40964162 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f14ab90e40 0x55f14ab91220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:12.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:12.158+0000 7f818d800640 1 -- 172.21.15.5:0/40964162 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f14aabdae0 con 0x55f14ab91760 2024-09-10T09:56:12.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:12.158+0000 7f818e200640 1 --2- 172.21.15.5:0/40964162 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f14ab90e40 0x55f14ab91220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:56:12.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:12.158+0000 7f818d800640 1 --2- 172.21.15.5:0/40964162 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f14ab91760 0x55f14ab9bae0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f817c0f3c60 tx=0x7f817c0f3d50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f2ae23264256a7b2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:56:12.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:12.158+0000 7f817aa00640 1 -- 172.21.15.5:0/40964162 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f817c0f87c0 con 0x55f14ab91760 2024-09-10T09:56:12.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:12.158+0000 7f817aa00640 1 -- 172.21.15.5:0/40964162 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f817c0f8e40 con 0x55f14ab91760 2024-09-10T09:56:12.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:12.158+0000 7f817aa00640 1 -- 172.21.15.5:0/40964162 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f817c0fc3d0 con 0x55f14ab91760 2024-09-10T09:56:12.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:12.159+0000 7f8186000640 1 --2- 172.21.15.5:0/2330784785 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f81700a2b80 0x7f81700a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:12.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:12.159+0000 7f8186000640 1 -- 172.21.15.5:0/2330784785 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f817c0efb30 con 0x7f8188103e30 2024-09-10T09:56:12.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:12.159+0000 7f818e200640 1 --2- 172.21.15.5:0/2330784785 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f81700a2b80 0x7f81700a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:12.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:12.159+0000 7f8192cd7740 1 -- 172.21.15.5:0/40964162 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f14ab91760 msgr2=0x55f14ab9bae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:12.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:12.159+0000 7f8192cd7740 1 --2- 172.21.15.5:0/40964162 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f14ab91760 0x55f14ab9bae0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f817c0f3c60 tx=0x7f817c0f3d50 comp rx=0 tx=0).stop 2024-09-10T09:56:12.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:12.159+0000 7f8192cd7740 1 -- 172.21.15.5:0/40964162 shutdown_connections 2024-09-10T09:56:12.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:12.159+0000 7f8192cd7740 1 --2- 172.21.15.5:0/40964162 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f14ab9c020 0x55f14ab9e410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:12.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:12.159+0000 7f8192cd7740 1 --2- 172.21.15.5:0/40964162 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f14ab91760 0x55f14ab9bae0 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:12.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:12.159+0000 7f8192cd7740 1 --2- 172.21.15.5:0/40964162 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f14ab90e40 0x55f14ab91220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:12.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:12.159+0000 7f8192cd7740 1 -- 172.21.15.5:0/40964162 >> 172.21.15.5:0/40964162 conn(0x55f14ab8d3d0 msgr2=0x55f14ab8d7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:56:12.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:12.159+0000 7f8192cd7740 1 -- 172.21.15.5:0/40964162 shutdown_connections 2024-09-10T09:56:12.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:12.160+0000 7f8192cd7740 1 -- 172.21.15.5:0/40964162 wait complete. 2024-09-10T09:56:12.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:12.160+0000 7f818e200640 1 --2- 172.21.15.5:0/2330784785 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f81700a2b80 0x7f81700a5040 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f8180005fd0 tx=0x7f81800096a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:56:12.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:12.160+0000 7f8192cd7740 1 -- start start 2024-09-10T09:56:12.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:12.160+0000 7f8192cd7740 1 Processor -- start 2024-09-10T09:56:12.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:12.161+0000 7f8192cd7740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f14ab90e40 0x55f14ab98e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:12.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:12.161+0000 7f8192cd7740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f14ab91760 0x55f14ab99380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:12.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:12.161+0000 7f8192cd7740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f14ab9c020 0x55f14ab998c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:12.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:12.161+0000 7f8192cd7740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55f14ab9e9b0 con 0x55f14ab91760 2024-09-10T09:56:12.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:12.161+0000 7f8192cd7740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55f14aab20f0 con 0x55f14ab90e40 2024-09-10T09:56:12.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:12.161+0000 7f8192cd7740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55f14ab9eb30 con 0x55f14ab9c020 2024-09-10T09:56:12.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:12.161+0000 7f8192cd7740 10 client.0 _ll_register_callbacks cb 0x55f14aa99bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:56:12.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:12.161+0000 7f818d800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f14ab90e40 0x55f14ab98e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:12.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:12.161+0000 7f818d800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f14ab90e40 0x55f14ab98e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:43878/0 (socket says 172.21.15.5:43878) 2024-09-10T09:56:12.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:12.161+0000 7f818d800640 1 -- 172.21.15.5:0/580693152 learned_addr learned my addr 172.21.15.5:0/580693152 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:56:12.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:12.161+0000 7f818ce00640 1 --2- 172.21.15.5:0/580693152 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f14ab91760 0x55f14ab99380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:12.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:12.161+0000 7f818e200640 1 --2- 172.21.15.5:0/580693152 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f14ab9c020 0x55f14ab998c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:12.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:12.161+0000 7f818d800640 1 -- 172.21.15.5:0/580693152 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f14ab9c020 msgr2=0x55f14ab998c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:56:12.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:12.161+0000 7f818d800640 1 --2- 172.21.15.5:0/580693152 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f14ab9c020 0x55f14ab998c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:12.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:12.161+0000 7f818d800640 1 -- 172.21.15.5:0/580693152 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f14ab91760 msgr2=0x55f14ab99380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:56:12.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:12.161+0000 7f818d800640 1 --2- 172.21.15.5:0/580693152 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f14ab91760 0x55f14ab99380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:12.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:12.161+0000 7f818d800640 1 -- 172.21.15.5:0/580693152 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f14a8683c0 con 0x55f14ab90e40 2024-09-10T09:56:12.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:12.162+0000 7f818d800640 1 --2- 172.21.15.5:0/580693152 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f14ab90e40 0x55f14ab98e40 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f817c0b8450 tx=0x7f817c03b890 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ba2a0901d2c56cba server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:56:12.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:12.162+0000 7f816f400640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-09-10T09:56:12.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:12.162+0000 7f816f400640 1 -- 172.21.15.5:0/580693152 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f817c049070 con 0x55f14ab90e40 2024-09-10T09:56:12.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:12.162+0000 7f816f400640 1 -- 172.21.15.5:0/580693152 <== mon.2 v2:172.21.15.102:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f817c0ed2f0 con 0x55f14ab90e40 2024-09-10T09:56:12.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:12.162+0000 7f8192cd7740 1 -- 172.21.15.5:0/580693152 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55f14ab04950 con 0x55f14ab90e40 2024-09-10T09:56:12.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:12.162+0000 7f816f400640 1 -- 172.21.15.5:0/580693152 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f817c0b8290 con 0x55f14ab90e40 2024-09-10T09:56:12.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:12.162+0000 7f8192cd7740 10 client.28886 fetch_fsmap learned FSMap version 42 2024-09-10T09:56:12.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:12.162+0000 7f8192cd7740 1 -- 172.21.15.5:0/580693152 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55f14ab9e9b0 con 0x55f14ab90e40 2024-09-10T09:56:12.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:12.163+0000 7f816f400640 1 -- 172.21.15.5:0/580693152 <== mon.2 v2:172.21.15.102:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55f14ab04950 con 0x55f14ab90e40 2024-09-10T09:56:12.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:12.163+0000 7f816f400640 1 -- 172.21.15.5:0/580693152 <== mon.2 v2:172.21.15.102:3300/0 5 ==== fsmap(e 42) ==== 8370+0+0 (secure 0 0 0) 0x7f817c0b8660 con 0x55f14ab90e40 2024-09-10T09:56:12.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:12.163+0000 7f8192cd7740 10 client.28886 fetch_fsmap finished waiting for FSMap version 42 2024-09-10T09:56:12.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:12.163+0000 7f8192cd7740 10 client.28886 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-09-10T09:56:12.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:12.163+0000 7f8192cd7740 20 client.28886 populate_metadata read hostname 'smithi005' 2024-09-10T09:56:12.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:12.163+0000 7f8192cd7740 1 --2- 172.21.15.5:0/580693152 >> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] conn(0x55f14ababb00 0x55f14abadf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:12.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:12.163+0000 7f8192cd7740 4 client.28886 mds_command: new command op to 24443 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:56:12.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:12.163+0000 7f8192cd7740 1 -- 172.21.15.5:0/580693152 --> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f14aafee60 con 0x55f14ababb00 2024-09-10T09:56:12.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:12.163+0000 7f818ce00640 1 --2- 172.21.15.5:0/580693152 >> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] conn(0x55f14ababb00 0x55f14abadf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:12.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:12.164+0000 7f818ce00640 1 --2- 172.21.15.5:0/580693152 >> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] conn(0x55f14ababb00 0x55f14abadf50 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=d7767948e852d79a server_cookie=a0943f8f47633e3d in_seq=0 out_seq=0 2024-09-10T09:56:12.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:12.164+0000 7f816f400640 10 client.28886 ms_handle_connect on v2:172.21.15.102:6832/2457007427 2024-09-10T09:56:12.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:12.165+0000 7f816f400640 1 -- 172.21.15.5:0/580693152 <== mds.4 v2:172.21.15.102:6832/2457007427 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f14aafee60 con 0x55f14ababb00 2024-09-10T09:56:12.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:12.165+0000 7f816f400640 10 client.28886 handle_command_reply: tid=0 2024-09-10T09:56:12.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:12.165+0000 7f8192cd7740 1 client.28886 shutdown 2024-09-10T09:56:12.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:12.165+0000 7f8192cd7740 1 -- 172.21.15.5:0/580693152 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f14ab90e40 msgr2=0x55f14ab98e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:12.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:12.165+0000 7f8192cd7740 1 --2- 172.21.15.5:0/580693152 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f14ab90e40 0x55f14ab98e40 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f817c0b8450 tx=0x7f817c03b890 comp rx=0 tx=0).stop 2024-09-10T09:56:12.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:12.165+0000 7f8192cd7740 1 -- 172.21.15.5:0/580693152 shutdown_connections 2024-09-10T09:56:12.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:12.165+0000 7f8192cd7740 1 --2- 172.21.15.5:0/580693152 >> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] conn(0x55f14ababb00 0x55f14abadf50 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:12.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:12.165+0000 7f8192cd7740 1 --2- 172.21.15.5:0/580693152 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f14ab9c020 0x55f14ab998c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:12.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:12.165+0000 7f8192cd7740 1 --2- 172.21.15.5:0/580693152 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f14ab91760 0x55f14ab99380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:12.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:12.165+0000 7f8192cd7740 1 --2- 172.21.15.5:0/580693152 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f14ab90e40 0x55f14ab98e40 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:12.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:12.165+0000 7f8192cd7740 1 -- 172.21.15.5:0/580693152 >> 172.21.15.5:0/580693152 conn(0x55f14ab8d3d0 msgr2=0x55f14ab93100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:56:12.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:12.165+0000 7f816f400640 0 client.28886 ms_handle_reset on v2:172.21.15.102:6832/2457007427 2024-09-10T09:56:12.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:12.165+0000 7f8192cd7740 1 -- 172.21.15.5:0/580693152 shutdown_connections 2024-09-10T09:56:12.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:12.165+0000 7f8192cd7740 1 -- 172.21.15.5:0/580693152 wait complete. 2024-09-10T09:56:12.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:12.166+0000 7f8192cd7740 20 client.28886 trim_cache size 0 max 16384 2024-09-10T09:56:12.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:12.187+0000 7f8192cd7740 1 Processor -- start 2024-09-10T09:56:12.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:12.188+0000 7f8192cd7740 1 -- start start 2024-09-10T09:56:12.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:12.188+0000 7f8192cd7740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f14ab90e40 0x55f14abb3820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:12.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:12.188+0000 7f8192cd7740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f14ab91760 0x55f14abb3d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:12.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:12.188+0000 7f8192cd7740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f14ab9c020 0x55f14ab975e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:12.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:12.188+0000 7f8192cd7740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f81881b0f00 con 0x55f14ab91760 2024-09-10T09:56:12.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:12.188+0000 7f8192cd7740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f81881b2a40 con 0x55f14ab90e40 2024-09-10T09:56:12.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:12.188+0000 7f8192cd7740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f81881b15d0 con 0x55f14ab9c020 2024-09-10T09:56:12.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:12.188+0000 7f818ce00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f14ab91760 0x55f14abb3d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:12.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:12.188+0000 7f818ce00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f14ab91760 0x55f14abb3d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58286/0 (socket says 172.21.15.5:58286) 2024-09-10T09:56:12.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:12.188+0000 7f818ce00640 1 -- 172.21.15.5:0/838678734 learned_addr learned my addr 172.21.15.5:0/838678734 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:56:12.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:12.188+0000 7f818e200640 1 --2- 172.21.15.5:0/838678734 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f14ab9c020 0x55f14ab975e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:12.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:12.188+0000 7f818d800640 1 --2- 172.21.15.5:0/838678734 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f14ab90e40 0x55f14abb3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:12.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:12.189+0000 7f818ce00640 1 -- 172.21.15.5:0/838678734 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f14ab9c020 msgr2=0x55f14ab975e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:56:12.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:12.189+0000 7f818ce00640 1 --2- 172.21.15.5:0/838678734 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f14ab9c020 0x55f14ab975e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:12.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:12.189+0000 7f818ce00640 1 -- 172.21.15.5:0/838678734 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f14ab90e40 msgr2=0x55f14abb3820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:56:12.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:12.189+0000 7f818ce00640 1 --2- 172.21.15.5:0/838678734 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f14ab90e40 0x55f14abb3820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:12.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:12.189+0000 7f818ce00640 1 -- 172.21.15.5:0/838678734 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f14ab99e10 con 0x55f14ab91760 2024-09-10T09:56:12.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:12.189+0000 7f818ce00640 1 --2- 172.21.15.5:0/838678734 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f14ab91760 0x55f14abb3d60 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f81880ffd60 tx=0x7f8174005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69e3f043bfd7b2df server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:56:12.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:12.189+0000 7f817a000640 1 -- 172.21.15.5:0/838678734 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8174003cc0 con 0x55f14ab91760 2024-09-10T09:56:12.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:12.189+0000 7f817a000640 1 -- 172.21.15.5:0/838678734 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8174010910 con 0x55f14ab91760 2024-09-10T09:56:12.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:12.189+0000 7f817a000640 1 -- 172.21.15.5:0/838678734 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f817401e9b0 con 0x55f14ab91760 2024-09-10T09:56:12.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:12.190+0000 7f8192cd7740 1 -- 172.21.15.5:0/838678734 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f14ab91760 msgr2=0x55f14abb3d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:12.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:12.190+0000 7f8192cd7740 1 --2- 172.21.15.5:0/838678734 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f14ab91760 0x55f14abb3d60 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f81880ffd60 tx=0x7f8174005f90 comp rx=0 tx=0).stop 2024-09-10T09:56:12.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:12.190+0000 7f8192cd7740 1 -- 172.21.15.5:0/838678734 shutdown_connections 2024-09-10T09:56:12.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:12.190+0000 7f8192cd7740 1 --2- 172.21.15.5:0/838678734 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f14ab9c020 0x55f14ab975e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:12.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:12.190+0000 7f8192cd7740 1 --2- 172.21.15.5:0/838678734 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f14ab91760 0x55f14abb3d60 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:12.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:12.190+0000 7f8192cd7740 1 --2- 172.21.15.5:0/838678734 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f14ab90e40 0x55f14abb3820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:12.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:12.190+0000 7f8192cd7740 1 -- 172.21.15.5:0/838678734 >> 172.21.15.5:0/838678734 conn(0x55f14ab8d3d0 msgr2=0x55f14ab9aec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:56:12.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:12.190+0000 7f8192cd7740 1 -- 172.21.15.5:0/838678734 shutdown_connections 2024-09-10T09:56:12.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:12.190+0000 7f8192cd7740 1 -- 172.21.15.5:0/838678734 wait complete. 2024-09-10T09:56:12.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:12.191+0000 7f8192cd7740 1 -- start start 2024-09-10T09:56:12.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:12.191+0000 7f8192cd7740 1 Processor -- start 2024-09-10T09:56:12.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:12.191+0000 7f8192cd7740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f14ab90e40 0x55f14ab95f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:12.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:12.191+0000 7f8192cd7740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f14ab91760 0x55f14ab96480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:12.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:12.191+0000 7f8192cd7740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f14ab9c020 0x55f14ab969c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:12.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:12.191+0000 7f8192cd7740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f81881b2370 con 0x55f14ab91760 2024-09-10T09:56:12.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:12.191+0000 7f8192cd7740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f81881b1ca0 con 0x55f14ab90e40 2024-09-10T09:56:12.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:12.191+0000 7f8192cd7740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f81881aeee0 con 0x55f14ab9c020 2024-09-10T09:56:12.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:12.191+0000 7f8192cd7740 10 client.0 _ll_register_callbacks cb 0x7f8180009db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:56:12.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:12.192+0000 7f818ce00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f14ab91760 0x55f14ab96480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:12.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:12.192+0000 7f818ce00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f14ab91760 0x55f14ab96480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58296/0 (socket says 172.21.15.5:58296) 2024-09-10T09:56:12.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:12.192+0000 7f818ce00640 1 -- 172.21.15.5:0/2623676712 learned_addr learned my addr 172.21.15.5:0/2623676712 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:56:12.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:12.192+0000 7f818e200640 1 --2- 172.21.15.5:0/2623676712 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f14ab9c020 0x55f14ab969c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:12.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:12.192+0000 7f818d800640 1 --2- 172.21.15.5:0/2623676712 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f14ab90e40 0x55f14ab95f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:12.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:12.192+0000 7f818ce00640 1 -- 172.21.15.5:0/2623676712 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f14ab9c020 msgr2=0x55f14ab969c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:56:12.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:12.192+0000 7f818ce00640 1 --2- 172.21.15.5:0/2623676712 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f14ab9c020 0x55f14ab969c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:12.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:12.192+0000 7f818ce00640 1 -- 172.21.15.5:0/2623676712 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f14ab90e40 msgr2=0x55f14ab95f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:56:12.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:12.192+0000 7f818ce00640 1 --2- 172.21.15.5:0/2623676712 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f14ab90e40 0x55f14ab95f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:12.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:12.192+0000 7f818ce00640 1 -- 172.21.15.5:0/2623676712 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f14aba2340 con 0x55f14ab91760 2024-09-10T09:56:12.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:12.192+0000 7f818ce00640 1 --2- 172.21.15.5:0/2623676712 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f14ab91760 0x55f14ab96480 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f817401e4e0 tx=0x7f8174010430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3582daeaedda431e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:56:12.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:12.192+0000 7f816f400640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-10T09:56:12.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:12.193+0000 7f816f400640 1 -- 172.21.15.5:0/2623676712 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8174010cd0 con 0x55f14ab91760 2024-09-10T09:56:12.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:12.193+0000 7f816f400640 1 -- 172.21.15.5:0/2623676712 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f8174003ac0 con 0x55f14ab91760 2024-09-10T09:56:12.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:12.193+0000 7f8192cd7740 1 -- 172.21.15.5:0/2623676712 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8174010910 con 0x55f14ab91760 2024-09-10T09:56:12.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:12.193+0000 7f816f400640 1 -- 172.21.15.5:0/2623676712 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f8174003d00 con 0x55f14ab91760 2024-09-10T09:56:12.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:12.193+0000 7f8192cd7740 10 client.18825 fetch_fsmap learned FSMap version 42 2024-09-10T09:56:12.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:12.193+0000 7f8192cd7740 1 -- 172.21.15.5:0/2623676712 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55f14ac16ba0 con 0x55f14ab91760 2024-09-10T09:56:12.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:12.193+0000 7f816f400640 1 -- 172.21.15.5:0/2623676712 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8174010910 con 0x55f14ab91760 2024-09-10T09:56:12.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:12.194+0000 7f816f400640 1 -- 172.21.15.5:0/2623676712 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 42) ==== 8370+0+0 (secure 0 0 0) 0x7f8174030070 con 0x55f14ab91760 2024-09-10T09:56:12.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:12.194+0000 7f8192cd7740 10 client.18825 fetch_fsmap finished waiting for FSMap version 42 2024-09-10T09:56:12.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:12.194+0000 7f8192cd7740 10 client.18825 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-09-10T09:56:12.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:12.194+0000 7f8192cd7740 20 client.18825 populate_metadata read hostname 'smithi005' 2024-09-10T09:56:12.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:12.194+0000 7f8192cd7740 1 --2- 172.21.15.5:0/2623676712 >> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] conn(0x55f14ac172f0 0x55f14ac19710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:12.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:12.194+0000 7f8192cd7740 4 client.18825 mds_command: new command op to 24443 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T09:56:12.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:12.194+0000 7f8192cd7740 1 -- 172.21.15.5:0/2623676712 --> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f14aa7b010 con 0x55f14ac172f0 2024-09-10T09:56:12.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:12.194+0000 7f818d800640 1 --2- 172.21.15.5:0/2623676712 >> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] conn(0x55f14ac172f0 0x55f14ac19710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:12.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:12.194+0000 7f818d800640 1 --2- 172.21.15.5:0/2623676712 >> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] conn(0x55f14ac172f0 0x55f14ac19710 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=3d0652cbe0349ed server_cookie=c3cb440e1c982f5f in_seq=0 out_seq=0 2024-09-10T09:56:12.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:12.194+0000 7f816f400640 10 client.18825 ms_handle_connect on v2:172.21.15.102:6832/2457007427 2024-09-10T09:56:12.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:12.195+0000 7f816f400640 1 -- 172.21.15.5:0/2623676712 <== mds.4 v2:172.21.15.102:6832/2457007427 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f14aa7b010 con 0x55f14ac172f0 2024-09-10T09:56:12.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:12.195+0000 7f816f400640 10 client.18825 handle_command_reply: tid=0 2024-09-10T09:56:12.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:12.195+0000 7f8192cd7740 1 client.18825 shutdown 2024-09-10T09:56:12.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:12.195+0000 7f8192cd7740 1 -- 172.21.15.5:0/2623676712 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f14ab91760 msgr2=0x55f14ab96480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:12.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:12.195+0000 7f8192cd7740 1 --2- 172.21.15.5:0/2623676712 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f14ab91760 0x55f14ab96480 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f817401e4e0 tx=0x7f8174010430 comp rx=0 tx=0).stop 2024-09-10T09:56:12.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:12.195+0000 7f8192cd7740 1 -- 172.21.15.5:0/2623676712 shutdown_connections 2024-09-10T09:56:12.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:12.195+0000 7f8192cd7740 1 --2- 172.21.15.5:0/2623676712 >> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] conn(0x55f14ac172f0 0x55f14ac19710 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:12.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:12.195+0000 7f8192cd7740 1 --2- 172.21.15.5:0/2623676712 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f14ab9c020 0x55f14ab969c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:12.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:12.195+0000 7f8192cd7740 1 --2- 172.21.15.5:0/2623676712 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f14ab91760 0x55f14ab96480 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:12.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:12.195+0000 7f8192cd7740 1 --2- 172.21.15.5:0/2623676712 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f14ab90e40 0x55f14ab95f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:12.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:12.195+0000 7f8192cd7740 1 -- 172.21.15.5:0/2623676712 >> 172.21.15.5:0/2623676712 conn(0x55f14ab8d3d0 msgr2=0x55f14ab8bc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:56:12.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:12.195+0000 7f816f400640 0 client.18825 ms_handle_reset on v2:172.21.15.102:6832/2457007427 2024-09-10T09:56:12.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:12.195+0000 7f8192cd7740 1 -- 172.21.15.5:0/2623676712 shutdown_connections 2024-09-10T09:56:12.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:12.195+0000 7f8192cd7740 1 -- 172.21.15.5:0/2623676712 wait complete. 2024-09-10T09:56:12.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:12.195+0000 7f8192cd7740 20 client.18825 trim_cache size 0 max 16384 2024-09-10T09:56:12.201 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-10T09:56:12.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:12.196+0000 7f818ec00640 1 -- 172.21.15.5:0/2330784785 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f81700a2b80 msgr2=0x7f81700a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:12.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:12.196+0000 7f818ec00640 1 --2- 172.21.15.5:0/2330784785 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f81700a2b80 0x7f81700a5040 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f8180005fd0 tx=0x7f81800096a0 comp rx=0 tx=0).stop 2024-09-10T09:56:12.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:12.196+0000 7f818ec00640 1 -- 172.21.15.5:0/2330784785 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8188103e30 msgr2=0x7f818819a4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:12.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:12.196+0000 7f818ec00640 1 --2- 172.21.15.5:0/2330784785 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8188103e30 0x7f818819a4a0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f817c004770 tx=0x7f817c003c10 comp rx=0 tx=0).stop 2024-09-10T09:56:12.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:12.196+0000 7f818ec00640 1 -- 172.21.15.5:0/2330784785 shutdown_connections 2024-09-10T09:56:12.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:12.196+0000 7f818ec00640 1 --2- 172.21.15.5:0/2330784785 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f81700a2b80 0x7f81700a5040 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:12.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:12.196+0000 7f818ec00640 1 --2- 172.21.15.5:0/2330784785 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8188108f60 0x7f818819ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:12.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:12.196+0000 7f818ec00640 1 --2- 172.21.15.5:0/2330784785 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f81881047e0 0x7f818819a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:12.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:12.196+0000 7f818ec00640 1 --2- 172.21.15.5:0/2330784785 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8188103e30 0x7f818819a4a0 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:12.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:12.196+0000 7f818ec00640 1 -- 172.21.15.5:0/2330784785 >> 172.21.15.5:0/2330784785 conn(0x7f81880ff8d0 msgr2=0x7f8188100b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:56:12.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:12.196+0000 7f818ec00640 1 -- 172.21.15.5:0/2330784785 shutdown_connections 2024-09-10T09:56:12.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:12.196+0000 7f818ec00640 1 -- 172.21.15.5:0/2330784785 wait complete. 2024-09-10T09:56:12.938 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-10T09:56:12.984 INFO:tasks.workunit.client.0.smithi005.stdout: 10 69576 14.83 MB/sec execute 111 sec latency 13834.470 ms 2024-09-10T09:56:13.306 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:56:13 smithi005 ceph-mon[30350]: pgmap v929: 129 pgs: 129 active+clean; 579 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 543 KiB/s rd, 308 KiB/s wr, 49 op/s 2024-09-10T09:56:13.313 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:56:13.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:56:13 smithi102 ceph-mon[35442]: pgmap v929: 129 pgs: 129 active+clean; 579 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 543 KiB/s rd, 308 KiB/s wr, 49 op/s 2024-09-10T09:56:13.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:56:13 smithi159 ceph-mon[35513]: pgmap v929: 129 pgs: 129 active+clean; 579 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 543 KiB/s rd, 308 KiB/s wr, 49 op/s 2024-09-10T09:56:13.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:13.892+0000 7fc2caa00640 1 -- 172.21.15.5:0/2695250937 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc2c4109140 msgr2=0x7fc2c4110cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:13.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:13.892+0000 7fc2caa00640 1 --2- 172.21.15.5:0/2695250937 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc2c4109140 0x7fc2c4110cc0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7fc2bc009ca0 tx=0x7fc2bc02a8c0 comp rx=0 tx=0).stop 2024-09-10T09:56:13.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:13.892+0000 7fc2caa00640 1 -- 172.21.15.5:0/2695250937 shutdown_connections 2024-09-10T09:56:13.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:13.892+0000 7fc2caa00640 1 --2- 172.21.15.5:0/2695250937 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc2c4109140 0x7fc2c4110cc0 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:13.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:13.892+0000 7fc2caa00640 1 --2- 172.21.15.5:0/2695250937 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc2c41049c0 0x7fc2c4108a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:13.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:13.892+0000 7fc2caa00640 1 --2- 172.21.15.5:0/2695250937 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc2c4104010 0x7fc2c41043f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:13.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:13.892+0000 7fc2caa00640 1 -- 172.21.15.5:0/2695250937 >> 172.21.15.5:0/2695250937 conn(0x7fc2c40ffa50 msgr2=0x7fc2c4101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:56:13.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:13.892+0000 7fc2caa00640 1 -- 172.21.15.5:0/2695250937 shutdown_connections 2024-09-10T09:56:13.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:13.892+0000 7fc2caa00640 1 -- 172.21.15.5:0/2695250937 wait complete. 2024-09-10T09:56:13.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:13.893+0000 7fc2caa00640 1 Processor -- start 2024-09-10T09:56:13.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:13.893+0000 7fc2caa00640 1 -- start start 2024-09-10T09:56:13.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:13.894+0000 7fc2caa00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc2c4104010 0x7fc2c419a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:13.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:13.894+0000 7fc2caa00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc2c41049c0 0x7fc2c419ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:13.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:13.894+0000 7fc2caa00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc2c4109140 0x7fc2c419ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:13.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:13.894+0000 7fc2caa00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc2c4114170 con 0x7fc2c4109140 2024-09-10T09:56:13.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:13.894+0000 7fc2caa00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fc2c4113ff0 con 0x7fc2c4104010 2024-09-10T09:56:13.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:13.894+0000 7fc2caa00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fc2c41142f0 con 0x7fc2c41049c0 2024-09-10T09:56:13.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:13.894+0000 7fc2c3e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc2c4104010 0x7fc2c419a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:13.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:13.894+0000 7fc2c3e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc2c4104010 0x7fc2c419a630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:43910/0 (socket says 172.21.15.5:43910) 2024-09-10T09:56:13.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:13.894+0000 7fc2c3e00640 1 -- 172.21.15.5:0/819338541 learned_addr learned my addr 172.21.15.5:0/819338541 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:56:13.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:13.894+0000 7fc2bbe00640 1 --2- 172.21.15.5:0/819338541 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc2c41049c0 0x7fc2c419ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:13.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:13.894+0000 7fc2c8a00640 1 --2- 172.21.15.5:0/819338541 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc2c4109140 0x7fc2c419ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:13.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:13.895+0000 7fc2c3e00640 1 -- 172.21.15.5:0/819338541 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc2c41049c0 msgr2=0x7fc2c419ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:13.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:13.895+0000 7fc2c3e00640 1 --2- 172.21.15.5:0/819338541 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc2c41049c0 0x7fc2c419ab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:13.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:13.895+0000 7fc2c3e00640 1 -- 172.21.15.5:0/819338541 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc2c4109140 msgr2=0x7fc2c419ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:13.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:13.895+0000 7fc2c3e00640 1 --2- 172.21.15.5:0/819338541 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc2c4109140 0x7fc2c419ef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:13.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:13.895+0000 7fc2c3e00640 1 -- 172.21.15.5:0/819338541 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2c419f5e0 con 0x7fc2c4104010 2024-09-10T09:56:13.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:13.895+0000 7fc2bbe00640 1 --2- 172.21.15.5:0/819338541 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc2c41049c0 0x7fc2c419ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:56:13.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:13.895+0000 7fc2c3e00640 1 --2- 172.21.15.5:0/819338541 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc2c4104010 0x7fc2c419a630 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fc2b400ea60 tx=0x7fc2b400ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:56:13.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:13.895+0000 7fc2c1600640 1 -- 172.21.15.5:0/819338541 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2b400cde0 con 0x7fc2c4104010 2024-09-10T09:56:13.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:13.895+0000 7fc2c1600640 1 -- 172.21.15.5:0/819338541 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc2b40035f0 con 0x7fc2c4104010 2024-09-10T09:56:13.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:13.895+0000 7fc2caa00640 1 -- 172.21.15.5:0/819338541 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2c419f8d0 con 0x7fc2c4104010 2024-09-10T09:56:13.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:13.895+0000 7fc2c1600640 1 -- 172.21.15.5:0/819338541 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2b4010670 con 0x7fc2c4104010 2024-09-10T09:56:13.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:13.896+0000 7fc2caa00640 1 -- 172.21.15.5:0/819338541 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2c41a71b0 con 0x7fc2c4104010 2024-09-10T09:56:13.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:13.899+0000 7fc2c1600640 1 -- 172.21.15.5:0/819338541 <== mon.2 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7fc2b40037a0 con 0x7fc2c4104010 2024-09-10T09:56:13.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:13.899+0000 7fc2cd48b740 1 Processor -- start 2024-09-10T09:56:13.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:13.899+0000 7fc2cd48b740 1 -- start start 2024-09-10T09:56:13.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:13.899+0000 7fc2cd48b740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55715a28bc70 0x55715a28c050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:13.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:13.899+0000 7fc2cd48b740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55715a28c590 0x55715a296910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:13.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:13.899+0000 7fc2cd48b740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55715a296e50 0x55715a299240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:13.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:13.899+0000 7fc2cd48b740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55715a1f23e0 con 0x55715a296e50 2024-09-10T09:56:13.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:13.899+0000 7fc2cd48b740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55715a199d80 con 0x55715a28c590 2024-09-10T09:56:13.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:13.899+0000 7fc2cd48b740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55715a1ee3a0 con 0x55715a28bc70 2024-09-10T09:56:13.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:13.900+0000 7fc2bbe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55715a296e50 0x55715a299240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:13.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:13.900+0000 7fc2c3e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55715a28c590 0x55715a296910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:13.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:13.900+0000 7fc2bbe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55715a296e50 0x55715a299240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58318/0 (socket says 172.21.15.5:58318) 2024-09-10T09:56:13.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:13.900+0000 7fc2bbe00640 1 -- 172.21.15.5:0/88078187 learned_addr learned my addr 172.21.15.5:0/88078187 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:56:13.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:13.900+0000 7fc2c8a00640 1 --2- 172.21.15.5:0/88078187 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55715a28bc70 0x55715a28c050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:13.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:13.900+0000 7fc2bbe00640 1 -- 172.21.15.5:0/88078187 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55715a28bc70 msgr2=0x55715a28c050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:56:13.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:13.900+0000 7fc2bbe00640 1 --2- 172.21.15.5:0/88078187 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55715a28bc70 0x55715a28c050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:13.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:13.900+0000 7fc2bbe00640 1 -- 172.21.15.5:0/88078187 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55715a28c590 msgr2=0x55715a296910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:56:13.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:13.900+0000 7fc2bbe00640 1 --2- 172.21.15.5:0/88078187 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55715a28c590 0x55715a296910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:13.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:13.900+0000 7fc2bbe00640 1 -- 172.21.15.5:0/88078187 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55715a1b8ae0 con 0x55715a296e50 2024-09-10T09:56:13.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:13.900+0000 7fc2c1600640 1 --2- 172.21.15.5:0/819338541 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fc2a80a2c00 0x7fc2a80a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:13.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:13.900+0000 7fc2c1600640 1 -- 172.21.15.5:0/819338541 <== mon.2 v2:172.21.15.102:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fc2b40cf720 con 0x7fc2c4104010 2024-09-10T09:56:13.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:13.901+0000 7fc2bbe00640 1 --2- 172.21.15.5:0/88078187 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55715a296e50 0x55715a299240 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7fc2ac0103d0 tx=0x7fc2ac010890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f5bf9dac171bc89 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:56:13.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:13.901+0000 7fc2c8a00640 1 --2- 172.21.15.5:0/819338541 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fc2a80a2c00 0x7fc2a80a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:13.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:13.901+0000 7fc2b9200640 1 -- 172.21.15.5:0/88078187 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2ac011700 con 0x55715a296e50 2024-09-10T09:56:13.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:13.901+0000 7fc2b9200640 1 -- 172.21.15.5:0/88078187 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc2ac011d80 con 0x55715a296e50 2024-09-10T09:56:13.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:13.901+0000 7fc2b9200640 1 -- 172.21.15.5:0/88078187 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2ac01a380 con 0x55715a296e50 2024-09-10T09:56:13.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:13.902+0000 7fc2cd48b740 1 -- 172.21.15.5:0/88078187 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55715a296e50 msgr2=0x55715a299240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:13.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:13.902+0000 7fc2cd48b740 1 --2- 172.21.15.5:0/88078187 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55715a296e50 0x55715a299240 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7fc2ac0103d0 tx=0x7fc2ac010890 comp rx=0 tx=0).stop 2024-09-10T09:56:13.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:13.902+0000 7fc2c8a00640 1 --2- 172.21.15.5:0/819338541 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fc2a80a2c00 0x7fc2a80a50c0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fc2bc004af0 tx=0x7fc2bc004a80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:56:13.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:13.902+0000 7fc2cd48b740 1 -- 172.21.15.5:0/88078187 shutdown_connections 2024-09-10T09:56:13.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:13.902+0000 7fc2cd48b740 1 --2- 172.21.15.5:0/88078187 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55715a296e50 0x55715a299240 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:13.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:13.902+0000 7fc2cd48b740 1 --2- 172.21.15.5:0/88078187 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55715a28c590 0x55715a296910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:13.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:13.902+0000 7fc2cd48b740 1 --2- 172.21.15.5:0/88078187 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55715a28bc70 0x55715a28c050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:13.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:13.902+0000 7fc2cd48b740 1 -- 172.21.15.5:0/88078187 >> 172.21.15.5:0/88078187 conn(0x55715a288200 msgr2=0x55715a2885e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:56:13.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:13.902+0000 7fc2cd48b740 1 -- 172.21.15.5:0/88078187 shutdown_connections 2024-09-10T09:56:13.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:13.902+0000 7fc2cd48b740 1 -- 172.21.15.5:0/88078187 wait complete. 2024-09-10T09:56:13.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:13.902+0000 7fc2cd48b740 1 -- start start 2024-09-10T09:56:13.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:13.903+0000 7fc2cd48b740 1 Processor -- start 2024-09-10T09:56:13.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:13.903+0000 7fc2cd48b740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55715a28bc70 0x55715a293d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:13.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:13.903+0000 7fc2cd48b740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55715a28c590 0x55715a294250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:13.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:13.903+0000 7fc2cd48b740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55715a296e50 0x55715a294790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:13.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:13.903+0000 7fc2cd48b740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55715a299a20 con 0x55715a28c590 2024-09-10T09:56:13.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:13.903+0000 7fc2cd48b740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55715a1ad0f0 con 0x55715a296e50 2024-09-10T09:56:13.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:13.903+0000 7fc2cd48b740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55715a299ba0 con 0x55715a28bc70 2024-09-10T09:56:13.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:13.903+0000 7fc2cd48b740 10 client.0 _ll_register_callbacks cb 0x55715a194bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:56:13.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:13.903+0000 7fc2bbe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55715a28c590 0x55715a294250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:13.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:13.904+0000 7fc2bbe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55715a28c590 0x55715a294250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58324/0 (socket says 172.21.15.5:58324) 2024-09-10T09:56:13.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:13.904+0000 7fc2bbe00640 1 -- 172.21.15.5:0/2385605365 learned_addr learned my addr 172.21.15.5:0/2385605365 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:56:13.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:13.904+0000 7fc2c3e00640 1 --2- 172.21.15.5:0/2385605365 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55715a28bc70 0x55715a293d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:13.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:13.904+0000 7fc2c8a00640 1 --2- 172.21.15.5:0/2385605365 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55715a296e50 0x55715a294790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:13.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:13.904+0000 7fc2bbe00640 1 -- 172.21.15.5:0/2385605365 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55715a28bc70 msgr2=0x55715a293d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:56:13.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:13.904+0000 7fc2bbe00640 1 --2- 172.21.15.5:0/2385605365 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55715a28bc70 0x55715a293d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:13.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:13.904+0000 7fc2bbe00640 1 -- 172.21.15.5:0/2385605365 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55715a296e50 msgr2=0x55715a294790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:56:13.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:13.904+0000 7fc2bbe00640 1 --2- 172.21.15.5:0/2385605365 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55715a296e50 0x55715a294790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:13.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:13.904+0000 7fc2bbe00640 1 -- 172.21.15.5:0/2385605365 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557159f633c0 con 0x55715a28c590 2024-09-10T09:56:13.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:13.904+0000 7fc2c8a00640 1 --2- 172.21.15.5:0/2385605365 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55715a296e50 0x55715a294790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:56:13.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:13.904+0000 7fc2bbe00640 1 --2- 172.21.15.5:0/2385605365 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55715a28c590 0x55715a294250 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7fc2ac01cab0 tx=0x7fc2ac01c6b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a23c6649cddc722 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:56:13.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:13.904+0000 7fc2b1600640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-10T09:56:13.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:13.905+0000 7fc2b1600640 1 -- 172.21.15.5:0/2385605365 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2ac02b060 con 0x55715a28c590 2024-09-10T09:56:13.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:13.905+0000 7fc2b1600640 1 -- 172.21.15.5:0/2385605365 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fc2ac004250 con 0x55715a28c590 2024-09-10T09:56:13.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:13.905+0000 7fc2cd48b740 1 -- 172.21.15.5:0/2385605365 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc2ac011d80 con 0x55715a28c590 2024-09-10T09:56:13.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:13.905+0000 7fc2b1600640 1 -- 172.21.15.5:0/2385605365 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7fc2ac032050 con 0x55715a28c590 2024-09-10T09:56:13.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:13.905+0000 7fc2cd48b740 10 client.18837 fetch_fsmap learned FSMap version 42 2024-09-10T09:56:13.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:13.905+0000 7fc2cd48b740 1 -- 172.21.15.5:0/2385605365 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55715a2a5a30 con 0x55715a28c590 2024-09-10T09:56:13.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:13.905+0000 7fc2b1600640 1 -- 172.21.15.5:0/2385605365 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc2ac011d80 con 0x55715a28c590 2024-09-10T09:56:13.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:13.905+0000 7fc2b1600640 1 -- 172.21.15.5:0/2385605365 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 42) ==== 8370+0+0 (secure 0 0 0) 0x7fc2ac003680 con 0x55715a28c590 2024-09-10T09:56:13.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:13.906+0000 7fc2cd48b740 10 client.18837 fetch_fsmap finished waiting for FSMap version 42 2024-09-10T09:56:13.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:13.906+0000 7fc2cd48b740 10 client.18837 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-09-10T09:56:13.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:13.906+0000 7fc2cd48b740 20 client.18837 populate_metadata read hostname 'smithi005' 2024-09-10T09:56:13.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:13.906+0000 7fc2cd48b740 1 --2- 172.21.15.5:0/2385605365 >> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] conn(0x55715a2a6ef0 0x55715a2a92e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:13.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:13.906+0000 7fc2cd48b740 4 client.18837 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:56:13.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:13.906+0000 7fc2cd48b740 1 -- 172.21.15.5:0/2385605365 --> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55715a1ff950 con 0x55715a2a6ef0 2024-09-10T09:56:13.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:13.906+0000 7fc2c3e00640 1 --2- 172.21.15.5:0/2385605365 >> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] conn(0x55715a2a6ef0 0x55715a2a92e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:13.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:13.907+0000 7fc2c3e00640 1 --2- 172.21.15.5:0/2385605365 >> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] conn(0x55715a2a6ef0 0x55715a2a92e0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=a6fc80317a991b49 server_cookie=fd651255f79d24ea in_seq=0 out_seq=0 2024-09-10T09:56:13.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:13.907+0000 7fc2b1600640 10 client.18837 ms_handle_connect on v2:172.21.15.102:6836/2689776764 2024-09-10T09:56:13.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:13.907+0000 7fc2b1600640 1 -- 172.21.15.5:0/2385605365 <== mds.3 v2:172.21.15.102:6836/2689776764 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55715a1ff950 con 0x55715a2a6ef0 2024-09-10T09:56:13.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:13.907+0000 7fc2b1600640 10 client.18837 handle_command_reply: tid=0 2024-09-10T09:56:13.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:13.907+0000 7fc2cd48b740 1 client.18837 shutdown 2024-09-10T09:56:13.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:13.908+0000 7fc2cd48b740 1 -- 172.21.15.5:0/2385605365 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55715a28c590 msgr2=0x55715a294250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:13.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:13.908+0000 7fc2cd48b740 1 --2- 172.21.15.5:0/2385605365 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55715a28c590 0x55715a294250 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7fc2ac01cab0 tx=0x7fc2ac01c6b0 comp rx=0 tx=0).stop 2024-09-10T09:56:13.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:13.908+0000 7fc2cd48b740 1 -- 172.21.15.5:0/2385605365 shutdown_connections 2024-09-10T09:56:13.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:13.908+0000 7fc2cd48b740 1 --2- 172.21.15.5:0/2385605365 >> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] conn(0x55715a2a6ef0 0x55715a2a92e0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:13.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:13.908+0000 7fc2cd48b740 1 --2- 172.21.15.5:0/2385605365 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55715a296e50 0x55715a294790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:13.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:13.908+0000 7fc2cd48b740 1 --2- 172.21.15.5:0/2385605365 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55715a28c590 0x55715a294250 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:13.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:13.908+0000 7fc2cd48b740 1 --2- 172.21.15.5:0/2385605365 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55715a28bc70 0x55715a293d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:13.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:13.908+0000 7fc2cd48b740 1 -- 172.21.15.5:0/2385605365 >> 172.21.15.5:0/2385605365 conn(0x55715a288200 msgr2=0x55715a28dee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:56:13.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:13.908+0000 7fc2b1600640 0 client.18837 ms_handle_reset on v2:172.21.15.102:6836/2689776764 2024-09-10T09:56:13.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:13.908+0000 7fc2cd48b740 1 -- 172.21.15.5:0/2385605365 shutdown_connections 2024-09-10T09:56:13.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:13.908+0000 7fc2cd48b740 1 -- 172.21.15.5:0/2385605365 wait complete. 2024-09-10T09:56:13.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:13.908+0000 7fc2cd48b740 20 client.18837 trim_cache size 0 max 16384 2024-09-10T09:56:13.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:13.922+0000 7fc2cd48b740 1 Processor -- start 2024-09-10T09:56:13.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:13.922+0000 7fc2cd48b740 1 -- start start 2024-09-10T09:56:13.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:13.922+0000 7fc2cd48b740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55715a28bc70 0x55715a2ae8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:13.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:13.922+0000 7fc2cd48b740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55715a28c590 0x55715a2aedf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:13.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:13.922+0000 7fc2cd48b740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55715a296e50 0x55715a28d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:13.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:13.923+0000 7fc2cd48b740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc2c41b10d0 con 0x55715a296e50 2024-09-10T09:56:13.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:13.923+0000 7fc2cd48b740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fc2c41b2c10 con 0x55715a28c590 2024-09-10T09:56:13.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:13.923+0000 7fc2cd48b740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fc2c41b17a0 con 0x55715a28bc70 2024-09-10T09:56:13.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:13.923+0000 7fc2c3e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55715a28bc70 0x55715a2ae8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:13.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:13.923+0000 7fc2c8a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55715a296e50 0x55715a28d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:13.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:13.923+0000 7fc2bbe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55715a28c590 0x55715a2aedf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:13.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:13.923+0000 7fc2c3e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55715a28bc70 0x55715a2ae8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:53974/0 (socket says 172.21.15.5:53974) 2024-09-10T09:56:13.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:13.923+0000 7fc2c3e00640 1 -- 172.21.15.5:0/3867972675 learned_addr learned my addr 172.21.15.5:0/3867972675 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:56:13.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:13.923+0000 7fc2bbe00640 1 -- 172.21.15.5:0/3867972675 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55715a28bc70 msgr2=0x55715a2ae8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:56:13.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:13.923+0000 7fc2bbe00640 1 --2- 172.21.15.5:0/3867972675 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55715a28bc70 0x55715a2ae8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:13.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:13.923+0000 7fc2bbe00640 1 -- 172.21.15.5:0/3867972675 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55715a296e50 msgr2=0x55715a28d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:56:13.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:13.923+0000 7fc2bbe00640 1 --2- 172.21.15.5:0/3867972675 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55715a296e50 0x55715a28d6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:13.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:13.923+0000 7fc2bbe00640 1 -- 172.21.15.5:0/3867972675 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55715a288e90 con 0x55715a28c590 2024-09-10T09:56:13.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:13.923+0000 7fc2c3e00640 1 --2- 172.21.15.5:0/3867972675 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55715a28bc70 0x55715a2ae8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:56:13.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:13.924+0000 7fc2bbe00640 1 --2- 172.21.15.5:0/3867972675 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55715a28c590 0x55715a2aedf0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fc2ac003a50 tx=0x7fc2ac00ba80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3e04cc21e1d92abe server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:56:13.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:13.924+0000 7fc2b3e00640 1 -- 172.21.15.5:0/3867972675 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2ac01c8a0 con 0x55715a28c590 2024-09-10T09:56:13.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:13.924+0000 7fc2b3e00640 1 -- 172.21.15.5:0/3867972675 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc2ac003cc0 con 0x55715a28c590 2024-09-10T09:56:13.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:13.924+0000 7fc2b3e00640 1 -- 172.21.15.5:0/3867972675 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2ac011880 con 0x55715a28c590 2024-09-10T09:56:13.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:13.925+0000 7fc2cd48b740 1 -- 172.21.15.5:0/3867972675 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55715a28c590 msgr2=0x55715a2aedf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:13.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:13.925+0000 7fc2cd48b740 1 --2- 172.21.15.5:0/3867972675 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55715a28c590 0x55715a2aedf0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fc2ac003a50 tx=0x7fc2ac00ba80 comp rx=0 tx=0).stop 2024-09-10T09:56:13.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:13.925+0000 7fc2cd48b740 1 -- 172.21.15.5:0/3867972675 shutdown_connections 2024-09-10T09:56:13.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:13.925+0000 7fc2cd48b740 1 --2- 172.21.15.5:0/3867972675 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55715a296e50 0x55715a28d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:13.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:13.925+0000 7fc2cd48b740 1 --2- 172.21.15.5:0/3867972675 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55715a28c590 0x55715a2aedf0 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:13.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:13.925+0000 7fc2cd48b740 1 --2- 172.21.15.5:0/3867972675 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55715a28bc70 0x55715a2ae8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:13.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:13.925+0000 7fc2cd48b740 1 -- 172.21.15.5:0/3867972675 >> 172.21.15.5:0/3867972675 conn(0x55715a288200 msgr2=0x55715a292e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:56:13.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:13.925+0000 7fc2cd48b740 1 -- 172.21.15.5:0/3867972675 shutdown_connections 2024-09-10T09:56:13.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:13.925+0000 7fc2cd48b740 1 -- 172.21.15.5:0/3867972675 wait complete. 2024-09-10T09:56:13.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:13.925+0000 7fc2cd48b740 1 -- start start 2024-09-10T09:56:13.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:13.925+0000 7fc2cd48b740 1 Processor -- start 2024-09-10T09:56:13.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:13.926+0000 7fc2cd48b740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55715a28bc70 0x55715a29f230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:13.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:13.926+0000 7fc2cd48b740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55715a28c590 0x55715a29a560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:13.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:13.926+0000 7fc2cd48b740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55715a296e50 0x55715a29aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:13.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:13.926+0000 7fc2cd48b740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc2c41b2540 con 0x55715a28c590 2024-09-10T09:56:13.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:13.926+0000 7fc2cd48b740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fc2c41b1e70 con 0x55715a296e50 2024-09-10T09:56:13.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:13.926+0000 7fc2cd48b740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fc2c41af0b0 con 0x55715a28bc70 2024-09-10T09:56:13.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:13.926+0000 7fc2cd48b740 10 client.0 _ll_register_callbacks cb 0x7fc2b40d5400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:56:13.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:13.926+0000 7fc2bbe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55715a28c590 0x55715a29a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:13.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:13.926+0000 7fc2c3e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55715a28bc70 0x55715a29f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:13.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:13.926+0000 7fc2bbe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55715a28c590 0x55715a29a560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58354/0 (socket says 172.21.15.5:58354) 2024-09-10T09:56:13.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:13.926+0000 7fc2bbe00640 1 -- 172.21.15.5:0/671170793 learned_addr learned my addr 172.21.15.5:0/671170793 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:56:13.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:13.926+0000 7fc2c8a00640 1 --2- 172.21.15.5:0/671170793 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55715a296e50 0x55715a29aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:13.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:13.927+0000 7fc2bbe00640 1 -- 172.21.15.5:0/671170793 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55715a28bc70 msgr2=0x55715a29f230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:56:13.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:13.927+0000 7fc2bbe00640 1 --2- 172.21.15.5:0/671170793 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55715a28bc70 0x55715a29f230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:13.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:13.927+0000 7fc2bbe00640 1 -- 172.21.15.5:0/671170793 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55715a296e50 msgr2=0x55715a29aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:56:13.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:13.927+0000 7fc2bbe00640 1 --2- 172.21.15.5:0/671170793 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55715a296e50 0x55715a29aaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:13.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:13.927+0000 7fc2bbe00640 1 -- 172.21.15.5:0/671170793 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55715a29afe0 con 0x55715a28c590 2024-09-10T09:56:13.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:13.927+0000 7fc2bbe00640 1 --2- 172.21.15.5:0/671170793 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55715a28c590 0x55715a29a560 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7fc2ac0103d0 tx=0x7fc2ac01cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3fba832cdd48425 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:56:13.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:13.927+0000 7fc2b1600640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-10T09:56:13.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:13.927+0000 7fc2b1600640 1 -- 172.21.15.5:0/671170793 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2ac0052c0 con 0x55715a28c590 2024-09-10T09:56:13.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:13.927+0000 7fc2b1600640 1 -- 172.21.15.5:0/671170793 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fc2ac005460 con 0x55715a28c590 2024-09-10T09:56:13.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:13.927+0000 7fc2cd48b740 1 -- 172.21.15.5:0/671170793 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc2ac003cc0 con 0x55715a28c590 2024-09-10T09:56:13.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:13.927+0000 7fc2b1600640 1 -- 172.21.15.5:0/671170793 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7fc2ac049050 con 0x55715a28c590 2024-09-10T09:56:13.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:13.928+0000 7fc2cd48b740 10 client.18843 fetch_fsmap learned FSMap version 42 2024-09-10T09:56:13.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:13.928+0000 7fc2cd48b740 1 -- 172.21.15.5:0/671170793 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55715a311570 con 0x55715a28c590 2024-09-10T09:56:13.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:13.928+0000 7fc2b1600640 1 -- 172.21.15.5:0/671170793 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc2ac003cc0 con 0x55715a28c590 2024-09-10T09:56:13.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:13.928+0000 7fc2b1600640 1 -- 172.21.15.5:0/671170793 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 42) ==== 8370+0+0 (secure 0 0 0) 0x7fc2ac037480 con 0x55715a28c590 2024-09-10T09:56:13.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:13.928+0000 7fc2cd48b740 10 client.18843 fetch_fsmap finished waiting for FSMap version 42 2024-09-10T09:56:13.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:13.928+0000 7fc2cd48b740 10 client.18843 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-09-10T09:56:13.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:13.928+0000 7fc2cd48b740 20 client.18843 populate_metadata read hostname 'smithi005' 2024-09-10T09:56:13.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:13.928+0000 7fc2cd48b740 1 --2- 172.21.15.5:0/671170793 >> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] conn(0x55715a311bb0 0x55715a313fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:13.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:13.928+0000 7fc2cd48b740 4 client.18843 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T09:56:13.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:13.928+0000 7fc2cd48b740 1 -- 172.21.15.5:0/671170793 --> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55715a1f9e60 con 0x55715a311bb0 2024-09-10T09:56:13.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:13.929+0000 7fc2c3e00640 1 --2- 172.21.15.5:0/671170793 >> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] conn(0x55715a311bb0 0x55715a313fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:13.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:13.929+0000 7fc2c3e00640 1 --2- 172.21.15.5:0/671170793 >> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] conn(0x55715a311bb0 0x55715a313fd0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=ecd6c7a15e411e42 server_cookie=a1e1f3fe6b2e34a9 in_seq=0 out_seq=0 2024-09-10T09:56:13.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:13.929+0000 7fc2b1600640 10 client.18843 ms_handle_connect on v2:172.21.15.102:6836/2689776764 2024-09-10T09:56:13.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:13.929+0000 7fc2b1600640 1 -- 172.21.15.5:0/671170793 <== mds.3 v2:172.21.15.102:6836/2689776764 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55715a1f9e60 con 0x55715a311bb0 2024-09-10T09:56:13.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:13.929+0000 7fc2b1600640 10 client.18843 handle_command_reply: tid=0 2024-09-10T09:56:13.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:13.929+0000 7fc2cd48b740 1 client.18843 shutdown 2024-09-10T09:56:13.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:13.929+0000 7fc2cd48b740 1 -- 172.21.15.5:0/671170793 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55715a28c590 msgr2=0x55715a29a560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:13.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:13.929+0000 7fc2cd48b740 1 --2- 172.21.15.5:0/671170793 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55715a28c590 0x55715a29a560 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7fc2ac0103d0 tx=0x7fc2ac01cb70 comp rx=0 tx=0).stop 2024-09-10T09:56:13.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:13.929+0000 7fc2cd48b740 1 -- 172.21.15.5:0/671170793 shutdown_connections 2024-09-10T09:56:13.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:13.929+0000 7fc2cd48b740 1 --2- 172.21.15.5:0/671170793 >> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] conn(0x55715a311bb0 0x55715a313fd0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:13.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:13.929+0000 7fc2cd48b740 1 --2- 172.21.15.5:0/671170793 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55715a296e50 0x55715a29aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:13.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:13.930+0000 7fc2cd48b740 1 --2- 172.21.15.5:0/671170793 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55715a28c590 0x55715a29a560 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:13.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:13.930+0000 7fc2cd48b740 1 --2- 172.21.15.5:0/671170793 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55715a28bc70 0x55715a29f230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:13.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:13.930+0000 7fc2cd48b740 1 -- 172.21.15.5:0/671170793 >> 172.21.15.5:0/671170793 conn(0x55715a288200 msgr2=0x55715a286b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:56:13.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:13.930+0000 7fc2b1600640 0 client.18843 ms_handle_reset on v2:172.21.15.102:6836/2689776764 2024-09-10T09:56:13.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:13.930+0000 7fc2cd48b740 1 -- 172.21.15.5:0/671170793 shutdown_connections 2024-09-10T09:56:13.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:13.930+0000 7fc2cd48b740 1 -- 172.21.15.5:0/671170793 wait complete. 2024-09-10T09:56:13.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:13.930+0000 7fc2cd48b740 20 client.18843 trim_cache size 0 max 16384 2024-09-10T09:56:13.935 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-10T09:56:13.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:13.930+0000 7fc2caa00640 1 -- 172.21.15.5:0/819338541 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fc2a80a2c00 msgr2=0x7fc2a80a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:13.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:13.930+0000 7fc2caa00640 1 --2- 172.21.15.5:0/819338541 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fc2a80a2c00 0x7fc2a80a50c0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fc2bc004af0 tx=0x7fc2bc004a80 comp rx=0 tx=0).stop 2024-09-10T09:56:13.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:13.930+0000 7fc2caa00640 1 -- 172.21.15.5:0/819338541 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc2c4104010 msgr2=0x7fc2c419a630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:13.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:13.930+0000 7fc2caa00640 1 --2- 172.21.15.5:0/819338541 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc2c4104010 0x7fc2c419a630 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fc2b400ea60 tx=0x7fc2b400ef20 comp rx=0 tx=0).stop 2024-09-10T09:56:13.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:13.931+0000 7fc2caa00640 1 -- 172.21.15.5:0/819338541 shutdown_connections 2024-09-10T09:56:13.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:13.931+0000 7fc2caa00640 1 --2- 172.21.15.5:0/819338541 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fc2a80a2c00 0x7fc2a80a50c0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:13.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:13.931+0000 7fc2caa00640 1 --2- 172.21.15.5:0/819338541 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc2c4109140 0x7fc2c419ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:13.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:13.931+0000 7fc2caa00640 1 --2- 172.21.15.5:0/819338541 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc2c41049c0 0x7fc2c419ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:13.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:13.931+0000 7fc2caa00640 1 --2- 172.21.15.5:0/819338541 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc2c4104010 0x7fc2c419a630 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:13.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:13.931+0000 7fc2caa00640 1 -- 172.21.15.5:0/819338541 >> 172.21.15.5:0/819338541 conn(0x7fc2c40ffa50 msgr2=0x7fc2c4100e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:56:13.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:13.931+0000 7fc2caa00640 1 -- 172.21.15.5:0/819338541 shutdown_connections 2024-09-10T09:56:13.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:13.931+0000 7fc2caa00640 1 -- 172.21.15.5:0/819338541 wait complete. 2024-09-10T09:56:13.984 INFO:tasks.workunit.client.0.smithi005.stdout: 10 69576 14.70 MB/sec execute 112 sec latency 14834.527 ms 2024-09-10T09:56:14.984 INFO:tasks.workunit.client.0.smithi005.stdout: 10 69576 14.57 MB/sec execute 113 sec latency 15834.643 ms 2024-09-10T09:56:15.026 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-10T09:56:15.306 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:56:15 smithi005 ceph-mon[30350]: pgmap v930: 129 pgs: 129 active+clean; 579 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 198 KiB/s rd, 125 KiB/s wr, 24 op/s 2024-09-10T09:56:15.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:56:15 smithi102 ceph-mon[35442]: pgmap v930: 129 pgs: 129 active+clean; 579 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 198 KiB/s rd, 125 KiB/s wr, 24 op/s 2024-09-10T09:56:15.405 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:56:15.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:56:15 smithi159 ceph-mon[35513]: pgmap v930: 129 pgs: 129 active+clean; 579 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 198 KiB/s rd, 125 KiB/s wr, 24 op/s 2024-09-10T09:56:15.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:15.977+0000 7fd333800640 1 -- 172.21.15.5:0/1364730628 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd32c109140 msgr2=0x7fd32c110cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:15.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:15.977+0000 7fd333800640 1 --2- 172.21.15.5:0/1364730628 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd32c109140 0x7fd32c110cc0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7fd324009ca0 tx=0x7fd32402a8c0 comp rx=0 tx=0).stop 2024-09-10T09:56:15.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:15.977+0000 7fd333800640 1 -- 172.21.15.5:0/1364730628 shutdown_connections 2024-09-10T09:56:15.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:15.977+0000 7fd333800640 1 --2- 172.21.15.5:0/1364730628 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd32c109140 0x7fd32c110cc0 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:15.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:15.977+0000 7fd333800640 1 --2- 172.21.15.5:0/1364730628 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd32c1049c0 0x7fd32c108a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:15.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:15.977+0000 7fd333800640 1 --2- 172.21.15.5:0/1364730628 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd32c104010 0x7fd32c1043f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:15.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:15.977+0000 7fd333800640 1 -- 172.21.15.5:0/1364730628 >> 172.21.15.5:0/1364730628 conn(0x7fd32c0ffa50 msgr2=0x7fd32c101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:56:15.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:15.977+0000 7fd333800640 1 -- 172.21.15.5:0/1364730628 shutdown_connections 2024-09-10T09:56:15.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:15.977+0000 7fd333800640 1 -- 172.21.15.5:0/1364730628 wait complete. 2024-09-10T09:56:15.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:15.978+0000 7fd333800640 1 Processor -- start 2024-09-10T09:56:15.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:15.978+0000 7fd333800640 1 -- start start 2024-09-10T09:56:15.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:15.979+0000 7fd333800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd32c104010 0x7fd32c19a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:15.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:15.979+0000 7fd333800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd32c1049c0 0x7fd32c19ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:15.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:15.979+0000 7fd333800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd32c109140 0x7fd32c19ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:15.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:15.979+0000 7fd333800640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd32c114170 con 0x7fd32c109140 2024-09-10T09:56:15.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:15.979+0000 7fd333800640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fd32c113ff0 con 0x7fd32c104010 2024-09-10T09:56:15.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:15.979+0000 7fd333800640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fd32c1142f0 con 0x7fd32c1049c0 2024-09-10T09:56:15.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:15.979+0000 7fd330e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd32c104010 0x7fd32c19a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:15.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:15.979+0000 7fd330e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd32c104010 0x7fd32c19a630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:43942/0 (socket says 172.21.15.5:43942) 2024-09-10T09:56:15.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:15.979+0000 7fd330e00640 1 -- 172.21.15.5:0/3318485441 learned_addr learned my addr 172.21.15.5:0/3318485441 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:56:15.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:15.979+0000 7fd323e00640 1 --2- 172.21.15.5:0/3318485441 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd32c1049c0 0x7fd32c19ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:15.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:15.979+0000 7fd331800640 1 --2- 172.21.15.5:0/3318485441 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd32c109140 0x7fd32c19ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:15.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:15.980+0000 7fd323e00640 1 -- 172.21.15.5:0/3318485441 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd32c104010 msgr2=0x7fd32c19a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:15.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:15.980+0000 7fd323e00640 1 --2- 172.21.15.5:0/3318485441 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd32c104010 0x7fd32c19a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:15.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:15.980+0000 7fd323e00640 1 -- 172.21.15.5:0/3318485441 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd32c109140 msgr2=0x7fd32c19ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:15.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:15.980+0000 7fd323e00640 1 --2- 172.21.15.5:0/3318485441 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd32c109140 0x7fd32c19ef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:15.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:15.980+0000 7fd323e00640 1 -- 172.21.15.5:0/3318485441 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd32c19f5e0 con 0x7fd32c1049c0 2024-09-10T09:56:15.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:15.980+0000 7fd330e00640 1 --2- 172.21.15.5:0/3318485441 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd32c104010 0x7fd32c19a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:56:15.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:15.980+0000 7fd323e00640 1 --2- 172.21.15.5:0/3318485441 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd32c1049c0 0x7fd32c19ab70 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fd31400e9f0 tx=0x7fd31400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:56:15.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:15.980+0000 7fd32a000640 1 -- 172.21.15.5:0/3318485441 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd31400ccc0 con 0x7fd32c1049c0 2024-09-10T09:56:15.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:15.980+0000 7fd333800640 1 -- 172.21.15.5:0/3318485441 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd32c19f8d0 con 0x7fd32c1049c0 2024-09-10T09:56:15.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:15.980+0000 7fd32a000640 1 -- 172.21.15.5:0/3318485441 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd314005020 con 0x7fd32c1049c0 2024-09-10T09:56:15.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:15.980+0000 7fd32a000640 1 -- 172.21.15.5:0/3318485441 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3140049e0 con 0x7fd32c1049c0 2024-09-10T09:56:15.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:15.980+0000 7fd333800640 1 -- 172.21.15.5:0/3318485441 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd32c1a71b0 con 0x7fd32c1049c0 2024-09-10T09:56:15.984 INFO:tasks.workunit.client.0.smithi005.stdout: 10 69576 14.44 MB/sec execute 114 sec latency 16834.709 ms 2024-09-10T09:56:15.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:15.983+0000 7fd33625e740 1 Processor -- start 2024-09-10T09:56:15.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:15.983+0000 7fd33625e740 1 -- start start 2024-09-10T09:56:15.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:15.984+0000 7fd32a000640 1 -- 172.21.15.5:0/3318485441 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7fd314005560 con 0x7fd32c1049c0 2024-09-10T09:56:15.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:15.984+0000 7fd33625e740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5653b93c4b90 0x5653b93c4f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:15.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:15.984+0000 7fd33625e740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5653b93c54b0 0x5653b93d8120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:15.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:15.984+0000 7fd33625e740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5653b93d8660 0x5653b93daa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:15.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:15.984+0000 7fd33625e740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5653b932b3e0 con 0x5653b93d8660 2024-09-10T09:56:15.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:15.984+0000 7fd33625e740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x5653b92d2d80 con 0x5653b93c4b90 2024-09-10T09:56:15.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:15.984+0000 7fd33625e740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5653b93273a0 con 0x5653b93c54b0 2024-09-10T09:56:15.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:15.984+0000 7fd323e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5653b93d8660 0x5653b93daa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:15.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:15.984+0000 7fd323e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5653b93d8660 0x5653b93daa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58394/0 (socket says 172.21.15.5:58394) 2024-09-10T09:56:15.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:15.984+0000 7fd323e00640 1 -- 172.21.15.5:0/1528174966 learned_addr learned my addr 172.21.15.5:0/1528174966 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:56:15.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:15.984+0000 7fd331800640 1 --2- 172.21.15.5:0/1528174966 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5653b93c4b90 0x5653b93c4f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:15.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:15.985+0000 7fd330e00640 1 --2- 172.21.15.5:0/1528174966 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5653b93c54b0 0x5653b93d8120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:15.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:15.985+0000 7fd323e00640 1 -- 172.21.15.5:0/1528174966 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5653b93c54b0 msgr2=0x5653b93d8120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:56:15.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:15.985+0000 7fd323e00640 1 --2- 172.21.15.5:0/1528174966 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5653b93c54b0 0x5653b93d8120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:15.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:15.985+0000 7fd323e00640 1 -- 172.21.15.5:0/1528174966 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5653b93c4b90 msgr2=0x5653b93c4f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:56:15.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:15.985+0000 7fd323e00640 1 --2- 172.21.15.5:0/1528174966 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5653b93c4b90 0x5653b93c4f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:15.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:15.985+0000 7fd323e00640 1 -- 172.21.15.5:0/1528174966 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5653b92f1ae0 con 0x5653b93d8660 2024-09-10T09:56:15.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:15.985+0000 7fd331800640 1 --2- 172.21.15.5:0/1528174966 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5653b93c4b90 0x5653b93c4f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:56:15.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:15.985+0000 7fd323e00640 1 --2- 172.21.15.5:0/1528174966 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5653b93d8660 0x5653b93daa50 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7fd3140d7520 tx=0x7fd3140d7550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73b63017ad47ac3a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:56:15.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:15.986+0000 7fd322000640 1 -- 172.21.15.5:0/1528174966 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3140d1310 con 0x5653b93d8660 2024-09-10T09:56:15.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:15.986+0000 7fd322000640 1 -- 172.21.15.5:0/1528174966 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd3140d14b0 con 0x5653b93d8660 2024-09-10T09:56:15.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:15.986+0000 7fd322000640 1 -- 172.21.15.5:0/1528174966 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3140dea80 con 0x5653b93d8660 2024-09-10T09:56:15.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:15.986+0000 7fd32a000640 1 --2- 172.21.15.5:0/3318485441 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fd3100a2b80 0x7fd3100a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:15.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:15.986+0000 7fd32a000640 1 -- 172.21.15.5:0/3318485441 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fd3140cf320 con 0x7fd32c1049c0 2024-09-10T09:56:15.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:15.986+0000 7fd331800640 1 --2- 172.21.15.5:0/3318485441 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fd3100a2b80 0x7fd3100a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:15.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:15.986+0000 7fd33625e740 1 -- 172.21.15.5:0/1528174966 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5653b93d8660 msgr2=0x5653b93daa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:15.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:15.986+0000 7fd33625e740 1 --2- 172.21.15.5:0/1528174966 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5653b93d8660 0x5653b93daa50 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7fd3140d7520 tx=0x7fd3140d7550 comp rx=0 tx=0).stop 2024-09-10T09:56:15.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:15.987+0000 7fd33625e740 1 -- 172.21.15.5:0/1528174966 shutdown_connections 2024-09-10T09:56:15.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:15.987+0000 7fd33625e740 1 --2- 172.21.15.5:0/1528174966 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5653b93d8660 0x5653b93daa50 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:15.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:15.987+0000 7fd33625e740 1 --2- 172.21.15.5:0/1528174966 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5653b93c54b0 0x5653b93d8120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:15.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:15.987+0000 7fd33625e740 1 --2- 172.21.15.5:0/1528174966 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5653b93c4b90 0x5653b93c4f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:15.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:15.987+0000 7fd33625e740 1 -- 172.21.15.5:0/1528174966 >> 172.21.15.5:0/1528174966 conn(0x5653b93c1200 msgr2=0x5653b93c15e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:56:15.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:15.987+0000 7fd33625e740 1 -- 172.21.15.5:0/1528174966 shutdown_connections 2024-09-10T09:56:15.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:15.987+0000 7fd331800640 1 --2- 172.21.15.5:0/3318485441 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fd3100a2b80 0x7fd3100a5040 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fd324004af0 tx=0x7fd324004a80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:56:15.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:15.987+0000 7fd33625e740 1 -- 172.21.15.5:0/1528174966 wait complete. 2024-09-10T09:56:15.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:15.987+0000 7fd33625e740 1 -- start start 2024-09-10T09:56:15.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:15.987+0000 7fd33625e740 1 Processor -- start 2024-09-10T09:56:15.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:15.988+0000 7fd33625e740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5653b93c4b90 0x5653b93d55a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:15.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:15.988+0000 7fd33625e740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5653b93c54b0 0x5653b93d5ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:15.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:15.988+0000 7fd33625e740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5653b93d8660 0x5653b93d6020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:15.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:15.988+0000 7fd33625e740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5653b93db1d0 con 0x5653b93d8660 2024-09-10T09:56:15.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:15.988+0000 7fd33625e740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x5653b92e60f0 con 0x5653b93c4b90 2024-09-10T09:56:15.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:15.988+0000 7fd33625e740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5653b93db350 con 0x5653b93c54b0 2024-09-10T09:56:15.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:15.988+0000 7fd33625e740 10 client.0 _ll_register_callbacks cb 0x5653b92cdbd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:56:15.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:15.988+0000 7fd331800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5653b93d8660 0x5653b93d6020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:15.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:15.988+0000 7fd323e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5653b93c54b0 0x5653b93d5ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:15.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:15.988+0000 7fd331800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5653b93d8660 0x5653b93d6020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58408/0 (socket says 172.21.15.5:58408) 2024-09-10T09:56:15.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:15.988+0000 7fd331800640 1 -- 172.21.15.5:0/2144822331 learned_addr learned my addr 172.21.15.5:0/2144822331 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:56:15.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:15.988+0000 7fd330e00640 1 --2- 172.21.15.5:0/2144822331 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5653b93c4b90 0x5653b93d55a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:15.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:15.989+0000 7fd323e00640 1 -- 172.21.15.5:0/2144822331 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5653b93c4b90 msgr2=0x5653b93d55a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:56:15.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:15.989+0000 7fd323e00640 1 --2- 172.21.15.5:0/2144822331 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5653b93c4b90 0x5653b93d55a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:15.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:15.989+0000 7fd323e00640 1 -- 172.21.15.5:0/2144822331 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5653b93d8660 msgr2=0x5653b93d6020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:56:15.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:15.989+0000 7fd323e00640 1 --2- 172.21.15.5:0/2144822331 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5653b93d8660 0x5653b93d6020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:15.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:15.989+0000 7fd323e00640 1 -- 172.21.15.5:0/2144822331 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5653b909c3c0 con 0x5653b93c54b0 2024-09-10T09:56:15.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:15.989+0000 7fd323e00640 1 --2- 172.21.15.5:0/2144822331 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5653b93c54b0 0x5653b93d5ae0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fd31400c6a0 tx=0x7fd3140046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5d3dc5240fbe104b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:56:15.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:15.989+0000 7fd31aa00640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-09-10T09:56:15.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:15.989+0000 7fd31aa00640 1 -- 172.21.15.5:0/2144822331 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd31401c070 con 0x5653b93c54b0 2024-09-10T09:56:15.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:15.989+0000 7fd33625e740 1 -- 172.21.15.5:0/2144822331 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd3140d14b0 con 0x5653b93c54b0 2024-09-10T09:56:15.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:15.989+0000 7fd31aa00640 1 -- 172.21.15.5:0/2144822331 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fd314099a90 con 0x5653b93c54b0 2024-09-10T09:56:15.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:15.990+0000 7fd31aa00640 1 -- 172.21.15.5:0/2144822331 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7fd314097a30 con 0x5653b93c54b0 2024-09-10T09:56:15.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:15.990+0000 7fd33625e740 10 client.28915 fetch_fsmap learned FSMap version 42 2024-09-10T09:56:15.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:15.990+0000 7fd33625e740 1 -- 172.21.15.5:0/2144822331 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x5653b93db1d0 con 0x5653b93c54b0 2024-09-10T09:56:15.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:15.990+0000 7fd31aa00640 1 -- 172.21.15.5:0/2144822331 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd3140d14b0 con 0x5653b93c54b0 2024-09-10T09:56:15.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:15.990+0000 7fd31aa00640 1 -- 172.21.15.5:0/2144822331 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 42) ==== 8370+0+0 (secure 0 0 0) 0x7fd3140d1110 con 0x5653b93c54b0 2024-09-10T09:56:15.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:15.990+0000 7fd33625e740 10 client.28915 fetch_fsmap finished waiting for FSMap version 42 2024-09-10T09:56:15.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:15.990+0000 7fd33625e740 10 client.28915 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-10T09:56:15.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:15.990+0000 7fd33625e740 20 client.28915 populate_metadata read hostname 'smithi005' 2024-09-10T09:56:15.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:15.990+0000 7fd33625e740 1 --2- 172.21.15.5:0/2144822331 >> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] conn(0x5653b93e8420 0x5653b93ea810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:15.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:15.990+0000 7fd33625e740 4 client.28915 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:56:15.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:15.990+0000 7fd33625e740 1 -- 172.21.15.5:0/2144822331 --> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5653b9338950 con 0x5653b93e8420 2024-09-10T09:56:15.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:15.991+0000 7fd330e00640 1 --2- 172.21.15.5:0/2144822331 >> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] conn(0x5653b93e8420 0x5653b93ea810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:16.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:15.991+0000 7fd330e00640 1 --2- 172.21.15.5:0/2144822331 >> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] conn(0x5653b93e8420 0x5653b93ea810 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d5e31f700cc0d242 server_cookie=324902d8e590949b in_seq=0 out_seq=0 2024-09-10T09:56:16.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:15.991+0000 7fd31aa00640 10 client.28915 ms_handle_connect on v2:172.21.15.159:6836/620627639 2024-09-10T09:56:16.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:15.992+0000 7fd31aa00640 1 -- 172.21.15.5:0/2144822331 <== mds.2 v2:172.21.15.159:6836/620627639 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5653b9338950 con 0x5653b93e8420 2024-09-10T09:56:16.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:15.992+0000 7fd31aa00640 10 client.28915 handle_command_reply: tid=0 2024-09-10T09:56:16.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:15.992+0000 7fd33625e740 1 client.28915 shutdown 2024-09-10T09:56:16.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:15.993+0000 7fd33625e740 1 -- 172.21.15.5:0/2144822331 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5653b93c54b0 msgr2=0x5653b93d5ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:16.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:15.993+0000 7fd33625e740 1 --2- 172.21.15.5:0/2144822331 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5653b93c54b0 0x5653b93d5ae0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fd31400c6a0 tx=0x7fd3140046c0 comp rx=0 tx=0).stop 2024-09-10T09:56:16.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:15.993+0000 7fd33625e740 1 -- 172.21.15.5:0/2144822331 shutdown_connections 2024-09-10T09:56:16.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:15.993+0000 7fd33625e740 1 --2- 172.21.15.5:0/2144822331 >> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] conn(0x5653b93e8420 0x5653b93ea810 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:16.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:15.993+0000 7fd33625e740 1 --2- 172.21.15.5:0/2144822331 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5653b93d8660 0x5653b93d6020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:16.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:15.993+0000 7fd33625e740 1 --2- 172.21.15.5:0/2144822331 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5653b93c54b0 0x5653b93d5ae0 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:16.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:15.993+0000 7fd33625e740 1 --2- 172.21.15.5:0/2144822331 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5653b93c4b90 0x5653b93d55a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:16.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:15.993+0000 7fd31aa00640 0 client.28915 ms_handle_reset on v2:172.21.15.159:6836/620627639 2024-09-10T09:56:16.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:15.993+0000 7fd33625e740 1 -- 172.21.15.5:0/2144822331 >> 172.21.15.5:0/2144822331 conn(0x5653b93c1200 msgr2=0x5653b93c0380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:56:16.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:15.993+0000 7fd33625e740 1 -- 172.21.15.5:0/2144822331 shutdown_connections 2024-09-10T09:56:16.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:15.993+0000 7fd33625e740 1 -- 172.21.15.5:0/2144822331 wait complete. 2024-09-10T09:56:16.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:15.993+0000 7fd33625e740 20 client.28915 trim_cache size 0 max 16384 2024-09-10T09:56:16.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:16.007+0000 7fd33625e740 1 Processor -- start 2024-09-10T09:56:16.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:16.007+0000 7fd33625e740 1 -- start start 2024-09-10T09:56:16.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:16.007+0000 7fd33625e740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5653b93c4b90 0x5653b93d78d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:16.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:16.007+0000 7fd33625e740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5653b93c54b0 0x5653b93efe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:16.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:16.007+0000 7fd33625e740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5653b93d8660 0x5653b93f03b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:16.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:16.007+0000 7fd33625e740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd32c1acc10 con 0x5653b93c54b0 2024-09-10T09:56:16.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:16.007+0000 7fd33625e740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fd32c1ae750 con 0x5653b93c4b90 2024-09-10T09:56:16.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:16.007+0000 7fd33625e740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fd32c1ad2e0 con 0x5653b93d8660 2024-09-10T09:56:16.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:16.007+0000 7fd323e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5653b93c54b0 0x5653b93efe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:16.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:16.007+0000 7fd323e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5653b93c54b0 0x5653b93efe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58412/0 (socket says 172.21.15.5:58412) 2024-09-10T09:56:16.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:16.007+0000 7fd323e00640 1 -- 172.21.15.5:0/3930315500 learned_addr learned my addr 172.21.15.5:0/3930315500 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:56:16.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:16.007+0000 7fd330e00640 1 --2- 172.21.15.5:0/3930315500 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5653b93c4b90 0x5653b93d78d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:16.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:16.007+0000 7fd331800640 1 --2- 172.21.15.5:0/3930315500 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5653b93d8660 0x5653b93f03b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:16.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:16.008+0000 7fd323e00640 1 -- 172.21.15.5:0/3930315500 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5653b93d8660 msgr2=0x5653b93f03b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:56:16.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:16.008+0000 7fd323e00640 1 --2- 172.21.15.5:0/3930315500 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5653b93d8660 0x5653b93f03b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:16.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:16.008+0000 7fd323e00640 1 -- 172.21.15.5:0/3930315500 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5653b93c4b90 msgr2=0x5653b93d78d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:56:16.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:16.008+0000 7fd323e00640 1 --2- 172.21.15.5:0/3930315500 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5653b93c4b90 0x5653b93d78d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:16.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:16.008+0000 7fd323e00640 1 -- 172.21.15.5:0/3930315500 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5653b93bc770 con 0x5653b93c54b0 2024-09-10T09:56:16.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:16.008+0000 7fd323e00640 1 --2- 172.21.15.5:0/3930315500 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5653b93c54b0 0x5653b93efe70 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7fd31409fd00 tx=0x7fd314004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db765071fb61088e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:56:16.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:16.008+0000 7fd321600640 1 -- 172.21.15.5:0/3930315500 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd31401c070 con 0x5653b93c54b0 2024-09-10T09:56:16.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:16.008+0000 7fd321600640 1 -- 172.21.15.5:0/3930315500 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd3140d7150 con 0x5653b93c54b0 2024-09-10T09:56:16.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:16.008+0000 7fd321600640 1 -- 172.21.15.5:0/3930315500 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd31409b0f0 con 0x5653b93c54b0 2024-09-10T09:56:16.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:16.009+0000 7fd33625e740 1 -- 172.21.15.5:0/3930315500 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5653b93c54b0 msgr2=0x5653b93efe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:16.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:16.009+0000 7fd33625e740 1 --2- 172.21.15.5:0/3930315500 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5653b93c54b0 0x5653b93efe70 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7fd31409fd00 tx=0x7fd314004730 comp rx=0 tx=0).stop 2024-09-10T09:56:16.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:16.009+0000 7fd33625e740 1 -- 172.21.15.5:0/3930315500 shutdown_connections 2024-09-10T09:56:16.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:16.009+0000 7fd33625e740 1 --2- 172.21.15.5:0/3930315500 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5653b93d8660 0x5653b93f03b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:16.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:16.009+0000 7fd33625e740 1 --2- 172.21.15.5:0/3930315500 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5653b93c54b0 0x5653b93efe70 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:16.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:16.009+0000 7fd33625e740 1 --2- 172.21.15.5:0/3930315500 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5653b93c4b90 0x5653b93d78d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:16.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:16.009+0000 7fd33625e740 1 -- 172.21.15.5:0/3930315500 >> 172.21.15.5:0/3930315500 conn(0x5653b93c1200 msgr2=0x5653b93d4560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:56:16.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:16.009+0000 7fd33625e740 1 -- 172.21.15.5:0/3930315500 shutdown_connections 2024-09-10T09:56:16.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:16.009+0000 7fd33625e740 1 -- 172.21.15.5:0/3930315500 wait complete. 2024-09-10T09:56:16.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:16.010+0000 7fd33625e740 1 -- start start 2024-09-10T09:56:16.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:16.010+0000 7fd33625e740 1 Processor -- start 2024-09-10T09:56:16.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:16.010+0000 7fd33625e740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5653b93c4b90 0x5653b93e0e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:16.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:16.010+0000 7fd33625e740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5653b93c54b0 0x5653b93dc1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:16.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:16.010+0000 7fd33625e740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5653b93d8660 0x5653b93dc790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:16.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:16.010+0000 7fd33625e740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd32c1ae080 con 0x5653b93d8660 2024-09-10T09:56:16.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:16.010+0000 7fd33625e740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fd32c1ad9b0 con 0x5653b93c54b0 2024-09-10T09:56:16.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:16.010+0000 7fd33625e740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fd32c1aabf0 con 0x5653b93c4b90 2024-09-10T09:56:16.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:16.010+0000 7fd33625e740 10 client.0 _ll_register_callbacks cb 0x7fd324009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:56:16.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:16.011+0000 7fd331800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5653b93d8660 0x5653b93dc790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:16.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:16.011+0000 7fd331800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5653b93d8660 0x5653b93dc790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58416/0 (socket says 172.21.15.5:58416) 2024-09-10T09:56:16.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:16.011+0000 7fd331800640 1 -- 172.21.15.5:0/2647394148 learned_addr learned my addr 172.21.15.5:0/2647394148 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:56:16.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:16.011+0000 7fd323e00640 1 --2- 172.21.15.5:0/2647394148 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5653b93c54b0 0x5653b93dc1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:16.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:16.011+0000 7fd330e00640 1 --2- 172.21.15.5:0/2647394148 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5653b93c4b90 0x5653b93e0e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:16.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:16.011+0000 7fd331800640 1 -- 172.21.15.5:0/2647394148 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5653b93c4b90 msgr2=0x5653b93e0e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:56:16.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:16.011+0000 7fd331800640 1 --2- 172.21.15.5:0/2647394148 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5653b93c4b90 0x5653b93e0e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:16.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:16.011+0000 7fd331800640 1 -- 172.21.15.5:0/2647394148 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5653b93c54b0 msgr2=0x5653b93dc1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:56:16.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:16.011+0000 7fd331800640 1 --2- 172.21.15.5:0/2647394148 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5653b93c54b0 0x5653b93dc1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:16.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:16.011+0000 7fd331800640 1 -- 172.21.15.5:0/2647394148 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5653b93dcf90 con 0x5653b93d8660 2024-09-10T09:56:16.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:16.011+0000 7fd331800640 1 --2- 172.21.15.5:0/2647394148 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5653b93d8660 0x5653b93dc790 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7fd324009ca0 tx=0x7fd32402a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67710dcf40c09d2f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:56:16.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:16.011+0000 7fd31aa00640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-10T09:56:16.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:16.012+0000 7fd31aa00640 1 -- 172.21.15.5:0/2647394148 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd324059060 con 0x5653b93d8660 2024-09-10T09:56:16.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:16.012+0000 7fd31aa00640 1 -- 172.21.15.5:0/2647394148 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fd324004490 con 0x5653b93d8660 2024-09-10T09:56:16.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:16.012+0000 7fd33625e740 1 -- 172.21.15.5:0/2647394148 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5653b9332e60 con 0x5653b93d8660 2024-09-10T09:56:16.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:16.012+0000 7fd31aa00640 1 -- 172.21.15.5:0/2647394148 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7fd324004690 con 0x5653b93d8660 2024-09-10T09:56:16.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:16.012+0000 7fd33625e740 10 client.18861 fetch_fsmap learned FSMap version 42 2024-09-10T09:56:16.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:16.012+0000 7fd33625e740 1 -- 172.21.15.5:0/2647394148 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x5653b9452f20 con 0x5653b93d8660 2024-09-10T09:56:16.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:16.012+0000 7fd31aa00640 1 -- 172.21.15.5:0/2647394148 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x5653b9332e60 con 0x5653b93d8660 2024-09-10T09:56:16.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:16.012+0000 7fd31aa00640 1 -- 172.21.15.5:0/2647394148 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 42) ==== 8370+0+0 (secure 0 0 0) 0x7fd324053070 con 0x5653b93d8660 2024-09-10T09:56:16.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:16.012+0000 7fd33625e740 10 client.18861 fetch_fsmap finished waiting for FSMap version 42 2024-09-10T09:56:16.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:16.012+0000 7fd33625e740 10 client.18861 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-10T09:56:16.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:16.012+0000 7fd33625e740 20 client.18861 populate_metadata read hostname 'smithi005' 2024-09-10T09:56:16.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:16.012+0000 7fd33625e740 1 --2- 172.21.15.5:0/2647394148 >> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] conn(0x5653b9453640 0x5653b9455a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:16.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:16.012+0000 7fd33625e740 4 client.18861 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T09:56:16.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:16.012+0000 7fd33625e740 1 -- 172.21.15.5:0/2647394148 --> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5653b92af010 con 0x5653b9453640 2024-09-10T09:56:16.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:16.013+0000 7fd330e00640 1 --2- 172.21.15.5:0/2647394148 >> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] conn(0x5653b9453640 0x5653b9455a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:16.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:16.013+0000 7fd330e00640 1 --2- 172.21.15.5:0/2647394148 >> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] conn(0x5653b9453640 0x5653b9455a30 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=41a80a0af7b7907a server_cookie=dda2cd306012955a in_seq=0 out_seq=0 2024-09-10T09:56:16.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:16.013+0000 7fd31aa00640 10 client.18861 ms_handle_connect on v2:172.21.15.159:6836/620627639 2024-09-10T09:56:16.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:16.013+0000 7fd31aa00640 1 -- 172.21.15.5:0/2647394148 <== mds.2 v2:172.21.15.159:6836/620627639 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5653b92af010 con 0x5653b9453640 2024-09-10T09:56:16.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:16.013+0000 7fd31aa00640 10 client.18861 handle_command_reply: tid=0 2024-09-10T09:56:16.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:16.014+0000 7fd33625e740 1 client.18861 shutdown 2024-09-10T09:56:16.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:16.014+0000 7fd33625e740 1 -- 172.21.15.5:0/2647394148 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5653b93d8660 msgr2=0x5653b93dc790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:16.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:16.014+0000 7fd33625e740 1 --2- 172.21.15.5:0/2647394148 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5653b93d8660 0x5653b93dc790 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7fd324009ca0 tx=0x7fd32402a8c0 comp rx=0 tx=0).stop 2024-09-10T09:56:16.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:16.014+0000 7fd33625e740 1 -- 172.21.15.5:0/2647394148 shutdown_connections 2024-09-10T09:56:16.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:16.014+0000 7fd33625e740 1 --2- 172.21.15.5:0/2647394148 >> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] conn(0x5653b9453640 0x5653b9455a30 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:16.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:16.014+0000 7fd33625e740 1 --2- 172.21.15.5:0/2647394148 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5653b93d8660 0x5653b93dc790 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:16.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:16.014+0000 7fd33625e740 1 --2- 172.21.15.5:0/2647394148 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5653b93c54b0 0x5653b93dc1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:16.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:16.014+0000 7fd33625e740 1 --2- 172.21.15.5:0/2647394148 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5653b93c4b90 0x5653b93e0e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:16.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:16.014+0000 7fd33625e740 1 -- 172.21.15.5:0/2647394148 >> 172.21.15.5:0/2647394148 conn(0x5653b93c1200 msgr2=0x5653b93d3d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:56:16.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:16.014+0000 7fd31aa00640 0 client.18861 ms_handle_reset on v2:172.21.15.159:6836/620627639 2024-09-10T09:56:16.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:16.014+0000 7fd33625e740 1 -- 172.21.15.5:0/2647394148 shutdown_connections 2024-09-10T09:56:16.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:16.014+0000 7fd33625e740 1 -- 172.21.15.5:0/2647394148 wait complete. 2024-09-10T09:56:16.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:16.014+0000 7fd33625e740 20 client.18861 trim_cache size 0 max 16384 2024-09-10T09:56:16.023 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-10T09:56:16.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:16.015+0000 7fd333800640 1 -- 172.21.15.5:0/3318485441 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fd3100a2b80 msgr2=0x7fd3100a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:16.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:16.015+0000 7fd333800640 1 --2- 172.21.15.5:0/3318485441 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fd3100a2b80 0x7fd3100a5040 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fd324004af0 tx=0x7fd324004a80 comp rx=0 tx=0).stop 2024-09-10T09:56:16.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:16.015+0000 7fd333800640 1 -- 172.21.15.5:0/3318485441 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd32c1049c0 msgr2=0x7fd32c19ab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:16.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:16.015+0000 7fd333800640 1 --2- 172.21.15.5:0/3318485441 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd32c1049c0 0x7fd32c19ab70 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fd31400e9f0 tx=0x7fd31400eeb0 comp rx=0 tx=0).stop 2024-09-10T09:56:16.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:16.015+0000 7fd333800640 1 -- 172.21.15.5:0/3318485441 shutdown_connections 2024-09-10T09:56:16.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:16.015+0000 7fd333800640 1 --2- 172.21.15.5:0/3318485441 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fd3100a2b80 0x7fd3100a5040 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:16.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:16.015+0000 7fd333800640 1 --2- 172.21.15.5:0/3318485441 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd32c109140 0x7fd32c19ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:16.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:16.015+0000 7fd333800640 1 --2- 172.21.15.5:0/3318485441 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd32c1049c0 0x7fd32c19ab70 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:16.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:16.015+0000 7fd333800640 1 --2- 172.21.15.5:0/3318485441 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd32c104010 0x7fd32c19a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:16.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:16.015+0000 7fd333800640 1 -- 172.21.15.5:0/3318485441 >> 172.21.15.5:0/3318485441 conn(0x7fd32c0ffa50 msgr2=0x7fd32c100e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:56:16.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:16.015+0000 7fd333800640 1 -- 172.21.15.5:0/3318485441 shutdown_connections 2024-09-10T09:56:16.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:16.015+0000 7fd333800640 1 -- 172.21.15.5:0/3318485441 wait complete. 2024-09-10T09:56:16.908 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-10T09:56:16.984 INFO:tasks.workunit.client.0.smithi005.stdout: 10 69576 14.32 MB/sec execute 115 sec latency 17834.769 ms 2024-09-10T09:56:17.287 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:56:17.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:56:17 smithi102 ceph-mon[35442]: pgmap v931: 129 pgs: 129 active+clean; 579 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 98 KiB/s wr, 26 op/s 2024-09-10T09:56:17.426 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:56:17 smithi005 ceph-mon[30350]: pgmap v931: 129 pgs: 129 active+clean; 579 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 98 KiB/s wr, 26 op/s 2024-09-10T09:56:17.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:56:17 smithi159 ceph-mon[35513]: pgmap v931: 129 pgs: 129 active+clean; 579 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 98 KiB/s wr, 26 op/s 2024-09-10T09:56:17.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:17.837+0000 7fde93e00640 1 -- 172.21.15.5:0/2701229610 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fde94103e30 msgr2=0x7fde94104210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:17.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:17.837+0000 7fde93e00640 1 --2- 172.21.15.5:0/2701229610 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fde94103e30 0x7fde94104210 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7fde84009900 tx=0x7fde8402a410 comp rx=0 tx=0).stop 2024-09-10T09:56:17.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:17.837+0000 7fde93e00640 1 -- 172.21.15.5:0/2701229610 shutdown_connections 2024-09-10T09:56:17.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:17.837+0000 7fde93e00640 1 --2- 172.21.15.5:0/2701229610 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fde94108f60 0x7fde94110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:17.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:17.837+0000 7fde93e00640 1 --2- 172.21.15.5:0/2701229610 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fde941047e0 0x7fde94108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:17.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:17.837+0000 7fde93e00640 1 --2- 172.21.15.5:0/2701229610 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fde94103e30 0x7fde94104210 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:17.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:17.837+0000 7fde93e00640 1 -- 172.21.15.5:0/2701229610 >> 172.21.15.5:0/2701229610 conn(0x7fde940ff8d0 msgr2=0x7fde94101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:56:17.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:17.837+0000 7fde93e00640 1 -- 172.21.15.5:0/2701229610 shutdown_connections 2024-09-10T09:56:17.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:17.837+0000 7fde93e00640 1 -- 172.21.15.5:0/2701229610 wait complete. 2024-09-10T09:56:17.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:17.838+0000 7fde93e00640 1 Processor -- start 2024-09-10T09:56:17.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:17.838+0000 7fde93e00640 1 -- start start 2024-09-10T09:56:17.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:17.838+0000 7fde93e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fde94103e30 0x7fde9419a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:17.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:17.838+0000 7fde93e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fde941047e0 0x7fde9419aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:17.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:17.838+0000 7fde93e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fde94108f60 0x7fde9419edc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:17.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:17.838+0000 7fde93e00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fde94113fe0 con 0x7fde941047e0 2024-09-10T09:56:17.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:17.838+0000 7fde93e00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fde94113e60 con 0x7fde94103e30 2024-09-10T09:56:17.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:17.838+0000 7fde93e00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fde94114160 con 0x7fde94108f60 2024-09-10T09:56:17.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:17.839+0000 7fde8be00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fde941047e0 0x7fde9419aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:17.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:17.839+0000 7fde93400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fde94108f60 0x7fde9419edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:17.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:17.839+0000 7fde8be00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fde941047e0 0x7fde9419aa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58438/0 (socket says 172.21.15.5:58438) 2024-09-10T09:56:17.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:17.839+0000 7fde8be00640 1 -- 172.21.15.5:0/3453108394 learned_addr learned my addr 172.21.15.5:0/3453108394 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:56:17.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:17.839+0000 7fde92a00640 1 --2- 172.21.15.5:0/3453108394 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fde94103e30 0x7fde9419a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:17.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:17.839+0000 7fde93400640 1 -- 172.21.15.5:0/3453108394 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fde94103e30 msgr2=0x7fde9419a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:17.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:17.839+0000 7fde93400640 1 --2- 172.21.15.5:0/3453108394 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fde94103e30 0x7fde9419a4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:17.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:17.839+0000 7fde93400640 1 -- 172.21.15.5:0/3453108394 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fde941047e0 msgr2=0x7fde9419aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:17.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:17.839+0000 7fde93400640 1 --2- 172.21.15.5:0/3453108394 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fde941047e0 0x7fde9419aa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:17.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:17.839+0000 7fde93400640 1 -- 172.21.15.5:0/3453108394 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde9419f4a0 con 0x7fde94108f60 2024-09-10T09:56:17.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:17.839+0000 7fde92a00640 1 --2- 172.21.15.5:0/3453108394 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fde94103e30 0x7fde9419a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:56:17.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:17.839+0000 7fde93400640 1 --2- 172.21.15.5:0/3453108394 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fde94108f60 0x7fde9419edc0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fde8c00da10 tx=0x7fde8c00ded0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:56:17.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:17.840+0000 7fde8b400640 1 -- 172.21.15.5:0/3453108394 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde8c00bd70 con 0x7fde94108f60 2024-09-10T09:56:17.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:17.840+0000 7fde93e00640 1 -- 172.21.15.5:0/3453108394 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde9419f790 con 0x7fde94108f60 2024-09-10T09:56:17.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:17.840+0000 7fde8b400640 1 -- 172.21.15.5:0/3453108394 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fde8c003e50 con 0x7fde94108f60 2024-09-10T09:56:17.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:17.840+0000 7fde8b400640 1 -- 172.21.15.5:0/3453108394 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde8c0106a0 con 0x7fde94108f60 2024-09-10T09:56:17.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:17.840+0000 7fde93e00640 1 -- 172.21.15.5:0/3453108394 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde941a7070 con 0x7fde94108f60 2024-09-10T09:56:17.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:17.844+0000 7fde9c423740 1 Processor -- start 2024-09-10T09:56:17.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:17.844+0000 7fde8b400640 1 -- 172.21.15.5:0/3453108394 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7fde8c003970 con 0x7fde94108f60 2024-09-10T09:56:17.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:17.844+0000 7fde9c423740 1 -- start start 2024-09-10T09:56:17.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:17.845+0000 7fde9c423740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f0486bce40 0x55f0486bd220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:17.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:17.845+0000 7fde9c423740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f0486bd760 0x55f0486c7ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:17.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:17.845+0000 7fde9c423740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f0486c8020 0x55f0486ca410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:17.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:17.845+0000 7fde9c423740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55f0486233e0 con 0x55f0486c8020 2024-09-10T09:56:17.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:17.845+0000 7fde9c423740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55f0485cad80 con 0x55f0486bce40 2024-09-10T09:56:17.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:17.845+0000 7fde9c423740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55f04861f3a0 con 0x55f0486bd760 2024-09-10T09:56:17.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:17.845+0000 7fde8be00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f0486bce40 0x55f0486bd220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:17.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:17.845+0000 7fde8be00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f0486c8020 0x55f0486ca410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:17.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:17.845+0000 7fde92a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f0486bd760 0x55f0486c7ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:17.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:17.845+0000 7fde92a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f0486bd760 0x55f0486c7ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:54060/0 (socket says 172.21.15.5:54060) 2024-09-10T09:56:17.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:17.845+0000 7fde92a00640 1 -- 172.21.15.5:0/433953588 learned_addr learned my addr 172.21.15.5:0/433953588 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:56:17.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:17.846+0000 7fde92a00640 1 -- 172.21.15.5:0/433953588 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f0486bce40 msgr2=0x55f0486bd220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:56:17.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:17.846+0000 7fde92a00640 1 --2- 172.21.15.5:0/433953588 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f0486bce40 0x55f0486bd220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:17.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:17.846+0000 7fde92a00640 1 -- 172.21.15.5:0/433953588 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f0486c8020 msgr2=0x55f0486ca410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:56:17.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:17.846+0000 7fde92a00640 1 --2- 172.21.15.5:0/433953588 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f0486c8020 0x55f0486ca410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:17.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:17.846+0000 7fde92a00640 1 -- 172.21.15.5:0/433953588 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f0485e9ae0 con 0x55f0486bd760 2024-09-10T09:56:17.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:17.846+0000 7fde8b400640 1 --2- 172.21.15.5:0/3453108394 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fde780a2b80 0x7fde780a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:17.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:17.846+0000 7fde8b400640 1 -- 172.21.15.5:0/3453108394 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fde8c0cf870 con 0x7fde94108f60 2024-09-10T09:56:17.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:17.846+0000 7fde8be00640 1 --2- 172.21.15.5:0/3453108394 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fde780a2b80 0x7fde780a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:17.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:17.846+0000 7fde92a00640 1 --2- 172.21.15.5:0/433953588 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f0486bd760 0x55f0486c7ae0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fde8402a930 tx=0x7fde84031630 comp rx=0 tx=0).ready entity=mon.1 client_cookie=adf1267fae6b1ef7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:56:17.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:17.846+0000 7fde83e00640 1 -- 172.21.15.5:0/433953588 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde84035040 con 0x55f0486bd760 2024-09-10T09:56:17.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:17.846+0000 7fde83e00640 1 -- 172.21.15.5:0/433953588 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fde84031a20 con 0x55f0486bd760 2024-09-10T09:56:17.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:17.847+0000 7fde83e00640 1 -- 172.21.15.5:0/433953588 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde84004a00 con 0x55f0486bd760 2024-09-10T09:56:17.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:17.847+0000 7fde9c423740 1 -- 172.21.15.5:0/433953588 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f0486bd760 msgr2=0x55f0486c7ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:17.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:17.847+0000 7fde9c423740 1 --2- 172.21.15.5:0/433953588 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f0486bd760 0x55f0486c7ae0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fde8402a930 tx=0x7fde84031630 comp rx=0 tx=0).stop 2024-09-10T09:56:17.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:17.847+0000 7fde8be00640 1 --2- 172.21.15.5:0/3453108394 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fde780a2b80 0x7fde780a5040 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fde7c009fd0 tx=0x7fde7c008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:56:17.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:17.847+0000 7fde9c423740 1 -- 172.21.15.5:0/433953588 shutdown_connections 2024-09-10T09:56:17.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:17.847+0000 7fde9c423740 1 --2- 172.21.15.5:0/433953588 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f0486c8020 0x55f0486ca410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:17.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:17.847+0000 7fde9c423740 1 --2- 172.21.15.5:0/433953588 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f0486bd760 0x55f0486c7ae0 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:17.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:17.847+0000 7fde9c423740 1 --2- 172.21.15.5:0/433953588 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f0486bce40 0x55f0486bd220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:17.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:17.847+0000 7fde9c423740 1 -- 172.21.15.5:0/433953588 >> 172.21.15.5:0/433953588 conn(0x55f0486b93d0 msgr2=0x55f0486b97b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:56:17.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:17.847+0000 7fde9c423740 1 -- 172.21.15.5:0/433953588 shutdown_connections 2024-09-10T09:56:17.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:17.847+0000 7fde9c423740 1 -- 172.21.15.5:0/433953588 wait complete. 2024-09-10T09:56:17.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:17.848+0000 7fde9c423740 1 -- start start 2024-09-10T09:56:17.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:17.848+0000 7fde9c423740 1 Processor -- start 2024-09-10T09:56:17.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:17.849+0000 7fde9c423740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f0486bce40 0x55f0486c4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:17.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:17.849+0000 7fde9c423740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f0486bd760 0x55f0486c5310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:17.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:17.849+0000 7fde9c423740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f0486c8020 0x55f0486c5850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:17.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:17.849+0000 7fde9c423740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55f0486cabf0 con 0x55f0486c8020 2024-09-10T09:56:17.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:17.849+0000 7fde9c423740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55f0485de0f0 con 0x55f0486bce40 2024-09-10T09:56:17.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:17.849+0000 7fde9c423740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55f0486cad70 con 0x55f0486bd760 2024-09-10T09:56:17.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:17.849+0000 7fde93400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f0486c8020 0x55f0486c5850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:17.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:17.849+0000 7fde8be00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f0486bd760 0x55f0486c5310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:17.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:17.849+0000 7fde9c423740 10 client.0 _ll_register_callbacks cb 0x55f0485c5bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:56:17.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:17.849+0000 7fde92a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f0486bce40 0x55f0486c4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:17.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:17.849+0000 7fde8be00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f0486bd760 0x55f0486c5310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:54076/0 (socket says 172.21.15.5:54076) 2024-09-10T09:56:17.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:17.849+0000 7fde8be00640 1 -- 172.21.15.5:0/262803649 learned_addr learned my addr 172.21.15.5:0/262803649 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:56:17.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:17.850+0000 7fde8be00640 1 -- 172.21.15.5:0/262803649 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f0486bce40 msgr2=0x55f0486c4dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:56:17.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:17.850+0000 7fde8be00640 1 --2- 172.21.15.5:0/262803649 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f0486bce40 0x55f0486c4dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:17.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:17.850+0000 7fde8be00640 1 -- 172.21.15.5:0/262803649 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f0486c8020 msgr2=0x55f0486c5850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:56:17.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:17.850+0000 7fde8be00640 1 --2- 172.21.15.5:0/262803649 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f0486c8020 0x55f0486c5850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:17.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:17.850+0000 7fde8be00640 1 -- 172.21.15.5:0/262803649 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f0483943c0 con 0x55f0486bd760 2024-09-10T09:56:17.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:17.850+0000 7fde93400640 1 --2- 172.21.15.5:0/262803649 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f0486c8020 0x55f0486c5850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:56:17.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:17.850+0000 7fde8be00640 1 --2- 172.21.15.5:0/262803649 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f0486bd760 0x55f0486c5310 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fde7c015ab0 tx=0x7fde7c0046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6fb1cebf46337cac server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:56:17.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:17.850+0000 7fde80c00640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-09-10T09:56:17.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:17.850+0000 7fde80c00640 1 -- 172.21.15.5:0/262803649 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde7c029020 con 0x55f0486bd760 2024-09-10T09:56:17.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:17.850+0000 7fde9c423740 1 -- 172.21.15.5:0/262803649 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fde84031a20 con 0x55f0486bd760 2024-09-10T09:56:17.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:17.851+0000 7fde80c00640 1 -- 172.21.15.5:0/262803649 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fde7c018c90 con 0x55f0486bd760 2024-09-10T09:56:17.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:17.851+0000 7fde80c00640 1 -- 172.21.15.5:0/262803649 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7fde7c034050 con 0x55f0486bd760 2024-09-10T09:56:17.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:17.851+0000 7fde9c423740 10 client.28930 fetch_fsmap learned FSMap version 42 2024-09-10T09:56:17.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:17.851+0000 7fde9c423740 1 -- 172.21.15.5:0/262803649 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55f0486d6b60 con 0x55f0486bd760 2024-09-10T09:56:17.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:17.851+0000 7fde80c00640 1 -- 172.21.15.5:0/262803649 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fde84031a20 con 0x55f0486bd760 2024-09-10T09:56:17.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:17.851+0000 7fde80c00640 1 -- 172.21.15.5:0/262803649 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 42) ==== 8370+0+0 (secure 0 0 0) 0x7fde7c0243a0 con 0x55f0486bd760 2024-09-10T09:56:17.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:17.851+0000 7fde9c423740 10 client.28930 fetch_fsmap finished waiting for FSMap version 42 2024-09-10T09:56:17.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:17.851+0000 7fde9c423740 10 client.28930 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-10T09:56:17.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:17.851+0000 7fde9c423740 20 client.28930 populate_metadata read hostname 'smithi005' 2024-09-10T09:56:17.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:17.851+0000 7fde9c423740 1 --2- 172.21.15.5:0/262803649 >> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] conn(0x55f0486d8030 0x55f0486da470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:17.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:17.852+0000 7fde9c423740 4 client.28930 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:56:17.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:17.852+0000 7fde9c423740 1 -- 172.21.15.5:0/262803649 --> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f048630950 con 0x55f0486d8030 2024-09-10T09:56:17.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:17.852+0000 7fde92a00640 1 --2- 172.21.15.5:0/262803649 >> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] conn(0x55f0486d8030 0x55f0486da470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:17.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:17.853+0000 7fde92a00640 1 --2- 172.21.15.5:0/262803649 >> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] conn(0x55f0486d8030 0x55f0486da470 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ae5d22c00fb70b8b server_cookie=c678ba08261b3a62 in_seq=0 out_seq=0 2024-09-10T09:56:17.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:17.853+0000 7fde80c00640 10 client.28930 ms_handle_connect on v2:172.21.15.159:6838/2073739599 2024-09-10T09:56:17.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:17.853+0000 7fde80c00640 1 -- 172.21.15.5:0/262803649 <== mds.1 v2:172.21.15.159:6838/2073739599 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f048630950 con 0x55f0486d8030 2024-09-10T09:56:17.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:17.853+0000 7fde80c00640 10 client.28930 handle_command_reply: tid=0 2024-09-10T09:56:17.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:17.853+0000 7fde9c423740 1 client.28930 shutdown 2024-09-10T09:56:17.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:17.854+0000 7fde9c423740 1 -- 172.21.15.5:0/262803649 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f0486bd760 msgr2=0x55f0486c5310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:17.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:17.854+0000 7fde9c423740 1 --2- 172.21.15.5:0/262803649 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f0486bd760 0x55f0486c5310 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fde7c015ab0 tx=0x7fde7c0046c0 comp rx=0 tx=0).stop 2024-09-10T09:56:17.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:17.854+0000 7fde9c423740 1 -- 172.21.15.5:0/262803649 shutdown_connections 2024-09-10T09:56:17.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:17.854+0000 7fde9c423740 1 --2- 172.21.15.5:0/262803649 >> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] conn(0x55f0486d8030 0x55f0486da470 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:17.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:17.854+0000 7fde9c423740 1 --2- 172.21.15.5:0/262803649 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f0486c8020 0x55f0486c5850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:17.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:17.854+0000 7fde9c423740 1 --2- 172.21.15.5:0/262803649 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f0486bd760 0x55f0486c5310 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:17.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:17.854+0000 7fde9c423740 1 --2- 172.21.15.5:0/262803649 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f0486bce40 0x55f0486c4dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:17.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:17.854+0000 7fde80c00640 0 client.28930 ms_handle_reset on v2:172.21.15.159:6838/2073739599 2024-09-10T09:56:17.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:17.854+0000 7fde9c423740 1 -- 172.21.15.5:0/262803649 >> 172.21.15.5:0/262803649 conn(0x55f0486b93d0 msgr2=0x55f0486bf0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:56:17.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:17.854+0000 7fde9c423740 1 -- 172.21.15.5:0/262803649 shutdown_connections 2024-09-10T09:56:17.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:17.854+0000 7fde9c423740 1 -- 172.21.15.5:0/262803649 wait complete. 2024-09-10T09:56:17.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:17.854+0000 7fde9c423740 20 client.28930 trim_cache size 0 max 16384 2024-09-10T09:56:17.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:17.868+0000 7fde9c423740 1 Processor -- start 2024-09-10T09:56:17.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:17.868+0000 7fde9c423740 1 -- start start 2024-09-10T09:56:17.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:17.868+0000 7fde9c423740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f0486bce40 0x55f0486b7c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:17.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:17.868+0000 7fde9c423740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f0486bd760 0x55f0486dfd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:17.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:17.868+0000 7fde9c423740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f0486c8020 0x55f0486e0280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:17.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:17.868+0000 7fde9c423740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fde941b0f10 con 0x55f0486bd760 2024-09-10T09:56:17.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:17.868+0000 7fde9c423740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fde941b2a50 con 0x55f0486bce40 2024-09-10T09:56:17.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:17.868+0000 7fde9c423740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fde941b15e0 con 0x55f0486c8020 2024-09-10T09:56:17.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:17.869+0000 7fde8be00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f0486bd760 0x55f0486dfd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:17.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:17.869+0000 7fde8be00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f0486bd760 0x55f0486dfd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58476/0 (socket says 172.21.15.5:58476) 2024-09-10T09:56:17.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:17.869+0000 7fde8be00640 1 -- 172.21.15.5:0/4000684650 learned_addr learned my addr 172.21.15.5:0/4000684650 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:56:17.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:17.869+0000 7fde92a00640 1 --2- 172.21.15.5:0/4000684650 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f0486bce40 0x55f0486b7c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:17.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:17.869+0000 7fde93400640 1 --2- 172.21.15.5:0/4000684650 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f0486c8020 0x55f0486e0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:17.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:17.869+0000 7fde8be00640 1 -- 172.21.15.5:0/4000684650 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f0486c8020 msgr2=0x55f0486e0280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:56:17.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:17.869+0000 7fde8be00640 1 --2- 172.21.15.5:0/4000684650 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f0486c8020 0x55f0486e0280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:17.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:17.869+0000 7fde8be00640 1 -- 172.21.15.5:0/4000684650 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f0486bce40 msgr2=0x55f0486b7c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:56:17.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:17.869+0000 7fde8be00640 1 --2- 172.21.15.5:0/4000684650 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f0486bce40 0x55f0486b7c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:17.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:17.869+0000 7fde8be00640 1 -- 172.21.15.5:0/4000684650 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f0486ba060 con 0x55f0486bd760 2024-09-10T09:56:17.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:17.869+0000 7fde8be00640 1 --2- 172.21.15.5:0/4000684650 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f0486bd760 0x55f0486dfd40 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7fde7c009850 tx=0x7fde7c009a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa64a61fc21cdf93 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:56:17.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:17.870+0000 7fde83400640 1 -- 172.21.15.5:0/4000684650 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde7c029040 con 0x55f0486bd760 2024-09-10T09:56:17.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:17.870+0000 7fde83400640 1 -- 172.21.15.5:0/4000684650 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fde7c01e030 con 0x55f0486bd760 2024-09-10T09:56:17.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:17.870+0000 7fde83400640 1 -- 172.21.15.5:0/4000684650 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde7c024e20 con 0x55f0486bd760 2024-09-10T09:56:17.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:17.870+0000 7fde9c423740 1 -- 172.21.15.5:0/4000684650 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f0486bd760 msgr2=0x55f0486dfd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:17.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:17.870+0000 7fde9c423740 1 --2- 172.21.15.5:0/4000684650 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f0486bd760 0x55f0486dfd40 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7fde7c009850 tx=0x7fde7c009a30 comp rx=0 tx=0).stop 2024-09-10T09:56:17.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:17.870+0000 7fde9c423740 1 -- 172.21.15.5:0/4000684650 shutdown_connections 2024-09-10T09:56:17.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:17.870+0000 7fde9c423740 1 --2- 172.21.15.5:0/4000684650 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f0486c8020 0x55f0486e0280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:17.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:17.870+0000 7fde9c423740 1 --2- 172.21.15.5:0/4000684650 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f0486bd760 0x55f0486dfd40 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:17.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:17.870+0000 7fde9c423740 1 --2- 172.21.15.5:0/4000684650 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f0486bce40 0x55f0486b7c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:17.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:17.871+0000 7fde9c423740 1 -- 172.21.15.5:0/4000684650 >> 172.21.15.5:0/4000684650 conn(0x55f0486b93d0 msgr2=0x55f0486c6e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:56:17.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:17.871+0000 7fde9c423740 1 -- 172.21.15.5:0/4000684650 shutdown_connections 2024-09-10T09:56:17.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:17.871+0000 7fde9c423740 1 -- 172.21.15.5:0/4000684650 wait complete. 2024-09-10T09:56:17.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:17.871+0000 7fde9c423740 1 -- start start 2024-09-10T09:56:17.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:17.871+0000 7fde9c423740 1 Processor -- start 2024-09-10T09:56:17.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:17.872+0000 7fde9c423740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f0486bce40 0x55f0486cffc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:17.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:17.872+0000 7fde9c423740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f0486bd760 0x55f0486d0500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:17.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:17.872+0000 7fde9c423740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f0486c8020 0x55f0486c1040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:17.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:17.872+0000 7fde9c423740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fde941b2380 con 0x55f0486bd760 2024-09-10T09:56:17.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:17.872+0000 7fde9c423740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fde941b1cb0 con 0x55f0486bce40 2024-09-10T09:56:17.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:17.872+0000 7fde9c423740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fde941aeef0 con 0x55f0486c8020 2024-09-10T09:56:17.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:17.872+0000 7fde9c423740 10 client.0 _ll_register_callbacks cb 0x7fde7c015760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:56:17.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:17.872+0000 7fde92a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f0486bce40 0x55f0486cffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:17.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:17.872+0000 7fde8be00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f0486bd760 0x55f0486d0500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:17.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:17.872+0000 7fde92a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f0486bce40 0x55f0486cffc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:44040/0 (socket says 172.21.15.5:44040) 2024-09-10T09:56:17.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:17.872+0000 7fde92a00640 1 -- 172.21.15.5:0/3644618575 learned_addr learned my addr 172.21.15.5:0/3644618575 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:56:17.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:17.872+0000 7fde93400640 1 --2- 172.21.15.5:0/3644618575 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f0486c8020 0x55f0486c1040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:17.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:17.872+0000 7fde8be00640 1 -- 172.21.15.5:0/3644618575 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f0486c8020 msgr2=0x55f0486c1040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:56:17.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:17.872+0000 7fde8be00640 1 --2- 172.21.15.5:0/3644618575 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f0486c8020 0x55f0486c1040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:17.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:17.872+0000 7fde8be00640 1 -- 172.21.15.5:0/3644618575 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f0486bce40 msgr2=0x55f0486cffc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:56:17.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:17.872+0000 7fde8be00640 1 --2- 172.21.15.5:0/3644618575 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f0486bce40 0x55f0486cffc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:17.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:17.873+0000 7fde8be00640 1 -- 172.21.15.5:0/3644618575 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f0486b8c80 con 0x55f0486bd760 2024-09-10T09:56:17.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:17.873+0000 7fde93400640 1 --2- 172.21.15.5:0/3644618575 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f0486c8020 0x55f0486c1040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:56:17.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:17.873+0000 7fde92a00640 1 --2- 172.21.15.5:0/3644618575 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f0486bce40 0x55f0486cffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:56:17.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:17.873+0000 7fde8be00640 1 --2- 172.21.15.5:0/3644618575 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f0486bd760 0x55f0486d0500 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7fde7c000c00 tx=0x7fde7c0245d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40402b0768924d86 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:56:17.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:17.873+0000 7fde80c00640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-10T09:56:17.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:17.873+0000 7fde80c00640 1 -- 172.21.15.5:0/3644618575 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde7c046070 con 0x55f0486bd760 2024-09-10T09:56:17.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:17.873+0000 7fde9c423740 1 -- 172.21.15.5:0/3644618575 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fde7c01e030 con 0x55f0486bd760 2024-09-10T09:56:17.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:17.873+0000 7fde80c00640 1 -- 172.21.15.5:0/3644618575 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fde7c018ca0 con 0x55f0486bd760 2024-09-10T09:56:17.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:17.874+0000 7fde80c00640 1 -- 172.21.15.5:0/3644618575 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7fde7c04e050 con 0x55f0486bd760 2024-09-10T09:56:17.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:17.874+0000 7fde9c423740 10 client.18879 fetch_fsmap learned FSMap version 42 2024-09-10T09:56:17.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:17.874+0000 7fde9c423740 1 -- 172.21.15.5:0/3644618575 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55f0486c2410 con 0x55f0486bd760 2024-09-10T09:56:17.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:17.874+0000 7fde80c00640 1 -- 172.21.15.5:0/3644618575 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fde7c01e030 con 0x55f0486bd760 2024-09-10T09:56:17.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:17.874+0000 7fde80c00640 1 -- 172.21.15.5:0/3644618575 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 42) ==== 8370+0+0 (secure 0 0 0) 0x7fde7c049030 con 0x55f0486bd760 2024-09-10T09:56:17.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:17.874+0000 7fde9c423740 10 client.18879 fetch_fsmap finished waiting for FSMap version 42 2024-09-10T09:56:17.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:17.874+0000 7fde9c423740 10 client.18879 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-10T09:56:17.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:17.874+0000 7fde9c423740 20 client.18879 populate_metadata read hostname 'smithi005' 2024-09-10T09:56:17.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:17.874+0000 7fde9c423740 1 --2- 172.21.15.5:0/3644618575 >> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] conn(0x55f0486c2b90 0x55f048745d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:17.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:17.874+0000 7fde9c423740 4 client.18879 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T09:56:17.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:17.874+0000 7fde9c423740 1 -- 172.21.15.5:0/3644618575 --> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f0486b49a0 con 0x55f0486c2b90 2024-09-10T09:56:17.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:17.875+0000 7fde92a00640 1 --2- 172.21.15.5:0/3644618575 >> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] conn(0x55f0486c2b90 0x55f048745d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:17.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:17.875+0000 7fde92a00640 1 --2- 172.21.15.5:0/3644618575 >> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] conn(0x55f0486c2b90 0x55f048745d30 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d7a773888689aca7 server_cookie=8e98c927e71bfb60 in_seq=0 out_seq=0 2024-09-10T09:56:17.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:17.875+0000 7fde80c00640 10 client.18879 ms_handle_connect on v2:172.21.15.159:6838/2073739599 2024-09-10T09:56:17.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:17.875+0000 7fde80c00640 1 -- 172.21.15.5:0/3644618575 <== mds.1 v2:172.21.15.159:6838/2073739599 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f0486b49a0 con 0x55f0486c2b90 2024-09-10T09:56:17.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:17.875+0000 7fde80c00640 10 client.18879 handle_command_reply: tid=0 2024-09-10T09:56:17.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:17.875+0000 7fde9c423740 1 client.18879 shutdown 2024-09-10T09:56:17.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:17.876+0000 7fde9c423740 1 -- 172.21.15.5:0/3644618575 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f0486bd760 msgr2=0x55f0486d0500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:17.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:17.876+0000 7fde9c423740 1 --2- 172.21.15.5:0/3644618575 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f0486bd760 0x55f0486d0500 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7fde7c000c00 tx=0x7fde7c0245d0 comp rx=0 tx=0).stop 2024-09-10T09:56:17.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:17.876+0000 7fde9c423740 1 -- 172.21.15.5:0/3644618575 shutdown_connections 2024-09-10T09:56:17.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:17.876+0000 7fde9c423740 1 --2- 172.21.15.5:0/3644618575 >> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] conn(0x55f0486c2b90 0x55f048745d30 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:17.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:17.876+0000 7fde9c423740 1 --2- 172.21.15.5:0/3644618575 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f0486c8020 0x55f0486c1040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:17.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:17.876+0000 7fde80c00640 0 client.18879 ms_handle_reset on v2:172.21.15.159:6838/2073739599 2024-09-10T09:56:17.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:17.876+0000 7fde9c423740 1 --2- 172.21.15.5:0/3644618575 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f0486bd760 0x55f0486d0500 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:17.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:17.876+0000 7fde9c423740 1 --2- 172.21.15.5:0/3644618575 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f0486bce40 0x55f0486cffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:17.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:17.876+0000 7fde9c423740 1 -- 172.21.15.5:0/3644618575 >> 172.21.15.5:0/3644618575 conn(0x55f0486b93d0 msgr2=0x55f0486c3570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:56:17.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:17.876+0000 7fde9c423740 1 -- 172.21.15.5:0/3644618575 shutdown_connections 2024-09-10T09:56:17.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:17.876+0000 7fde9c423740 1 -- 172.21.15.5:0/3644618575 wait complete. 2024-09-10T09:56:17.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:17.876+0000 7fde9c423740 20 client.18879 trim_cache size 0 max 16384 2024-09-10T09:56:17.880 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-10T09:56:17.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:17.877+0000 7fde93e00640 1 -- 172.21.15.5:0/3453108394 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fde780a2b80 msgr2=0x7fde780a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:17.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:17.877+0000 7fde93e00640 1 --2- 172.21.15.5:0/3453108394 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fde780a2b80 0x7fde780a5040 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fde7c009fd0 tx=0x7fde7c008040 comp rx=0 tx=0).stop 2024-09-10T09:56:17.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:17.877+0000 7fde93e00640 1 -- 172.21.15.5:0/3453108394 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fde94108f60 msgr2=0x7fde9419edc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:17.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:17.877+0000 7fde93e00640 1 --2- 172.21.15.5:0/3453108394 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fde94108f60 0x7fde9419edc0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fde8c00da10 tx=0x7fde8c00ded0 comp rx=0 tx=0).stop 2024-09-10T09:56:17.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:17.877+0000 7fde93e00640 1 -- 172.21.15.5:0/3453108394 shutdown_connections 2024-09-10T09:56:17.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:17.877+0000 7fde93e00640 1 --2- 172.21.15.5:0/3453108394 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fde780a2b80 0x7fde780a5040 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:17.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:17.877+0000 7fde93e00640 1 --2- 172.21.15.5:0/3453108394 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fde94108f60 0x7fde9419edc0 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:17.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:17.877+0000 7fde93e00640 1 --2- 172.21.15.5:0/3453108394 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fde941047e0 0x7fde9419aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:17.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:17.877+0000 7fde93e00640 1 --2- 172.21.15.5:0/3453108394 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fde94103e30 0x7fde9419a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:17.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:17.877+0000 7fde93e00640 1 -- 172.21.15.5:0/3453108394 >> 172.21.15.5:0/3453108394 conn(0x7fde940ff8d0 msgr2=0x7fde94100cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:56:17.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:17.877+0000 7fde93e00640 1 -- 172.21.15.5:0/3453108394 shutdown_connections 2024-09-10T09:56:17.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:17.877+0000 7fde93e00640 1 -- 172.21.15.5:0/3453108394 wait complete. 2024-09-10T09:56:17.984 INFO:tasks.workunit.client.0.smithi005.stdout: 10 69576 14.19 MB/sec execute 116 sec latency 18834.832 ms 2024-09-10T09:56:18.984 INFO:tasks.workunit.client.0.smithi005.stdout: 10 69576 14.07 MB/sec execute 117 sec latency 19834.887 ms 2024-09-10T09:56:19.095 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-10T09:56:19.179 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:56:19 smithi005 ceph-mon[30350]: pgmap v932: 129 pgs: 129 active+clean; 579 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 87 KiB/s wr, 16 op/s 2024-09-10T09:56:19.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:56:19 smithi102 ceph-mon[35442]: pgmap v932: 129 pgs: 129 active+clean; 579 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 87 KiB/s wr, 16 op/s 2024-09-10T09:56:19.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:56:19 smithi159 ceph-mon[35513]: pgmap v932: 129 pgs: 129 active+clean; 579 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 87 KiB/s wr, 16 op/s 2024-09-10T09:56:19.619 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:56:19.984 INFO:tasks.workunit.client.0.smithi005.stdout: 10 69576 13.95 MB/sec execute 118 sec latency 20834.967 ms 2024-09-10T09:56:20.984 INFO:tasks.workunit.client.0.smithi005.stdout: 10 69576 13.83 MB/sec execute 119 sec latency 21835.029 ms 2024-09-10T09:56:21.335 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:56:21 smithi005 ceph-mon[30350]: pgmap v933: 129 pgs: 129 active+clean; 593 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 1.6 MiB/s wr, 48 op/s 2024-09-10T09:56:21.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:56:21 smithi102 ceph-mon[35442]: pgmap v933: 129 pgs: 129 active+clean; 593 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 1.6 MiB/s wr, 48 op/s 2024-09-10T09:56:21.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:56:21 smithi159 ceph-mon[35513]: pgmap v933: 129 pgs: 129 active+clean; 593 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 1.6 MiB/s wr, 48 op/s 2024-09-10T09:56:21.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:21.487+0000 7fc62b200640 1 -- 172.21.15.5:0/2428491576 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc62c102370 msgr2=0x7fc62c10e8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:21.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:21.487+0000 7fc62b200640 1 --2- 172.21.15.5:0/2428491576 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc62c102370 0x7fc62c10e8a0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7fc624009cd0 tx=0x7fc62402a8f0 comp rx=0 tx=0).stop 2024-09-10T09:56:21.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:21.487+0000 7fc62b200640 1 -- 172.21.15.5:0/2428491576 shutdown_connections 2024-09-10T09:56:21.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:21.487+0000 7fc62b200640 1 --2- 172.21.15.5:0/2428491576 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc62c102370 0x7fc62c10e8a0 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:21.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:21.487+0000 7fc62b200640 1 --2- 172.21.15.5:0/2428491576 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc62c1019d0 0x7fc62c101e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:21.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:21.487+0000 7fc62b200640 1 --2- 172.21.15.5:0/2428491576 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc62c1079d0 0x7fc62c107db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:21.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:21.487+0000 7fc62b200640 1 -- 172.21.15.5:0/2428491576 >> 172.21.15.5:0/2428491576 conn(0x7fc62c0fd710 msgr2=0x7fc62c0ffb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:56:21.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:21.487+0000 7fc62b200640 1 -- 172.21.15.5:0/2428491576 shutdown_connections 2024-09-10T09:56:21.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:21.487+0000 7fc62b200640 1 -- 172.21.15.5:0/2428491576 wait complete. 2024-09-10T09:56:21.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:21.488+0000 7fc62b200640 1 Processor -- start 2024-09-10T09:56:21.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:21.488+0000 7fc62b200640 1 -- start start 2024-09-10T09:56:21.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:21.489+0000 7fc62b200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc62c1019d0 0x7fc62c19a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:21.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:21.489+0000 7fc62b200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc62c102370 0x7fc62c19aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:21.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:21.489+0000 7fc62b200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc62c1079d0 0x7fc62c19edc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:21.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:21.489+0000 7fc62b200640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc62c111dd0 con 0x7fc62c102370 2024-09-10T09:56:21.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:21.489+0000 7fc62b200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fc62c111c50 con 0x7fc62c1019d0 2024-09-10T09:56:21.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:21.489+0000 7fc62b200640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fc62c111f50 con 0x7fc62c1079d0 2024-09-10T09:56:21.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:21.489+0000 7fc629400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc62c102370 0x7fc62c19aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:21.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:21.489+0000 7fc629400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc62c102370 0x7fc62c19aa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41944/0 (socket says 172.21.15.5:41944) 2024-09-10T09:56:21.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:21.489+0000 7fc629400640 1 -- 172.21.15.5:0/472861682 learned_addr learned my addr 172.21.15.5:0/472861682 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:56:21.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:21.489+0000 7fc629400640 1 -- 172.21.15.5:0/472861682 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc62c1079d0 msgr2=0x7fc62c19edc0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-10T09:56:21.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:21.489+0000 7fc629400640 1 --2- 172.21.15.5:0/472861682 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc62c1079d0 0x7fc62c19edc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:21.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:21.489+0000 7fc629400640 1 -- 172.21.15.5:0/472861682 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc62c1019d0 msgr2=0x7fc62c19a4f0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-10T09:56:21.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:21.489+0000 7fc629400640 1 --2- 172.21.15.5:0/472861682 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc62c1019d0 0x7fc62c19a4f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:21.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:21.489+0000 7fc629400640 1 -- 172.21.15.5:0/472861682 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc62c19f4a0 con 0x7fc62c102370 2024-09-10T09:56:21.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:21.489+0000 7fc629400640 1 --2- 172.21.15.5:0/472861682 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc62c102370 0x7fc62c19aa30 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7fc61800e9f0 tx=0x7fc61800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:56:21.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:21.489+0000 7fc622a00640 1 -- 172.21.15.5:0/472861682 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc61800cd80 con 0x7fc62c102370 2024-09-10T09:56:21.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:21.489+0000 7fc622a00640 1 -- 172.21.15.5:0/472861682 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc618005500 con 0x7fc62c102370 2024-09-10T09:56:21.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:21.489+0000 7fc62b200640 1 -- 172.21.15.5:0/472861682 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc62c19f790 con 0x7fc62c102370 2024-09-10T09:56:21.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:21.489+0000 7fc622a00640 1 -- 172.21.15.5:0/472861682 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc618010690 con 0x7fc62c102370 2024-09-10T09:56:21.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:21.489+0000 7fc62b200640 1 -- 172.21.15.5:0/472861682 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc62c1a7070 con 0x7fc62c102370 2024-09-10T09:56:21.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:21.489+0000 7fc633843740 1 Processor -- start 2024-09-10T09:56:21.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:21.489+0000 7fc622a00640 1 -- 172.21.15.5:0/472861682 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7fc6180046c0 con 0x7fc62c102370 2024-09-10T09:56:21.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:21.489+0000 7fc633843740 1 -- start start 2024-09-10T09:56:21.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:21.489+0000 7fc633843740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564cce391e40 0x564cce392220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:21.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:21.489+0000 7fc633843740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564cce392760 0x564cce39cae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:21.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:21.489+0000 7fc633843740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564cce39d020 0x564cce39f410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:21.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:21.489+0000 7fc633843740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x564cce2f83e0 con 0x564cce39d020 2024-09-10T09:56:21.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:21.489+0000 7fc633843740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x564cce29fd80 con 0x564cce391e40 2024-09-10T09:56:21.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:21.489+0000 7fc633843740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x564cce2f43a0 con 0x564cce392760 2024-09-10T09:56:21.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:21.495+0000 7fc629400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564cce39d020 0x564cce39f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:21.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:21.495+0000 7fc629e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564cce392760 0x564cce39cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:21.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:21.495+0000 7fc629e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564cce392760 0x564cce39cae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:37252/0 (socket says 172.21.15.5:37252) 2024-09-10T09:56:21.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:21.495+0000 7fc629e00640 1 -- 172.21.15.5:0/3436238899 learned_addr learned my addr 172.21.15.5:0/3436238899 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:56:21.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:21.495+0000 7fc629400640 1 -- 172.21.15.5:0/3436238899 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564cce392760 msgr2=0x564cce39cae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:56:21.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:21.495+0000 7fc629400640 1 --2- 172.21.15.5:0/3436238899 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564cce392760 0x564cce39cae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:21.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:21.495+0000 7fc629400640 1 -- 172.21.15.5:0/3436238899 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564cce391e40 msgr2=0x564cce392220 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T09:56:21.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:21.495+0000 7fc629400640 1 --2- 172.21.15.5:0/3436238899 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564cce391e40 0x564cce392220 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:21.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:21.495+0000 7fc629400640 1 -- 172.21.15.5:0/3436238899 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564cce2beae0 con 0x564cce39d020 2024-09-10T09:56:21.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:21.495+0000 7fc622a00640 1 --2- 172.21.15.5:0/472861682 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fc6100a2c00 0x7fc6100a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:21.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:21.495+0000 7fc629400640 1 --2- 172.21.15.5:0/3436238899 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564cce39d020 0x564cce39f410 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7fc6180ccfb0 tx=0x7fc6180d5b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c22b1d07305450ae server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:56:21.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:21.495+0000 7fc622a00640 1 -- 172.21.15.5:0/472861682 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fc6180cf960 con 0x7fc62c102370 2024-09-10T09:56:21.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:21.495+0000 7fc62a800640 1 --2- 172.21.15.5:0/472861682 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fc6100a2c00 0x7fc6100a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:21.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:21.495+0000 7fc617000640 1 -- 172.21.15.5:0/3436238899 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc618098010 con 0x564cce39d020 2024-09-10T09:56:21.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:21.495+0000 7fc617000640 1 -- 172.21.15.5:0/3436238899 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc61809c470 con 0x564cce39d020 2024-09-10T09:56:21.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:21.495+0000 7fc617000640 1 -- 172.21.15.5:0/3436238899 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc61809b6d0 con 0x564cce39d020 2024-09-10T09:56:21.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:21.495+0000 7fc633843740 1 -- 172.21.15.5:0/3436238899 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564cce39d020 msgr2=0x564cce39f410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:21.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:21.495+0000 7fc62a800640 1 --2- 172.21.15.5:0/472861682 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fc6100a2c00 0x7fc6100a50c0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fc624004af0 tx=0x7fc624009360 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:56:21.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:21.495+0000 7fc633843740 1 --2- 172.21.15.5:0/3436238899 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564cce39d020 0x564cce39f410 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7fc6180ccfb0 tx=0x7fc6180d5b20 comp rx=0 tx=0).stop 2024-09-10T09:56:21.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:21.495+0000 7fc633843740 1 -- 172.21.15.5:0/3436238899 shutdown_connections 2024-09-10T09:56:21.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:21.495+0000 7fc633843740 1 --2- 172.21.15.5:0/3436238899 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564cce39d020 0x564cce39f410 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:21.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:21.495+0000 7fc633843740 1 --2- 172.21.15.5:0/3436238899 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564cce392760 0x564cce39cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:21.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:21.495+0000 7fc633843740 1 --2- 172.21.15.5:0/3436238899 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564cce391e40 0x564cce392220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:21.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:21.495+0000 7fc633843740 1 -- 172.21.15.5:0/3436238899 >> 172.21.15.5:0/3436238899 conn(0x564cce38e3d0 msgr2=0x564cce38e7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:56:21.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:21.495+0000 7fc633843740 1 -- 172.21.15.5:0/3436238899 shutdown_connections 2024-09-10T09:56:21.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:21.495+0000 7fc633843740 1 -- 172.21.15.5:0/3436238899 wait complete. 2024-09-10T09:56:21.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:21.495+0000 7fc633843740 1 -- start start 2024-09-10T09:56:21.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:21.495+0000 7fc633843740 1 Processor -- start 2024-09-10T09:56:21.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:21.495+0000 7fc633843740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564cce391e40 0x564cce39a090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:21.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:21.495+0000 7fc633843740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564cce392760 0x564cce39a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:21.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:21.495+0000 7fc633843740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564cce39d020 0x564cce39ab10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:21.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:21.495+0000 7fc633843740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x564cce3a82b0 con 0x564cce391e40 2024-09-10T09:56:21.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:21.495+0000 7fc633843740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x564cce2b30f0 con 0x564cce392760 2024-09-10T09:56:21.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:21.495+0000 7fc633843740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x564cce3a8430 con 0x564cce39d020 2024-09-10T09:56:21.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:21.495+0000 7fc633843740 10 client.0 _ll_register_callbacks cb 0x564cce29abd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:56:21.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:21.499+0000 7fc629e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564cce391e40 0x564cce39a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:21.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:21.499+0000 7fc629e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564cce391e40 0x564cce39a090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41968/0 (socket says 172.21.15.5:41968) 2024-09-10T09:56:21.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:21.499+0000 7fc629e00640 1 -- 172.21.15.5:0/2375852854 learned_addr learned my addr 172.21.15.5:0/2375852854 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:56:21.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:21.499+0000 7fc629e00640 1 -- 172.21.15.5:0/2375852854 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564cce39d020 msgr2=0x564cce39ab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:56:21.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:21.499+0000 7fc629e00640 1 --2- 172.21.15.5:0/2375852854 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564cce39d020 0x564cce39ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:21.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:21.499+0000 7fc629e00640 1 -- 172.21.15.5:0/2375852854 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564cce392760 msgr2=0x564cce39a5d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T09:56:21.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:21.499+0000 7fc629e00640 1 --2- 172.21.15.5:0/2375852854 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564cce392760 0x564cce39a5d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:21.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:21.499+0000 7fc629e00640 1 -- 172.21.15.5:0/2375852854 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564cce0693c0 con 0x564cce391e40 2024-09-10T09:56:21.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:21.499+0000 7fc629e00640 1 --2- 172.21.15.5:0/2375852854 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564cce391e40 0x564cce39a090 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7fc61c011450 tx=0x7fc61c011910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=beafc35f8fcb6a8a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:56:21.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:21.499+0000 7fc60b400640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-10T09:56:21.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:21.499+0000 7fc60b400640 1 -- 172.21.15.5:0/2375852854 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc61c004460 con 0x564cce391e40 2024-09-10T09:56:21.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:21.499+0000 7fc60b400640 1 -- 172.21.15.5:0/2375852854 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fc61c0035f0 con 0x564cce391e40 2024-09-10T09:56:21.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:21.499+0000 7fc633843740 1 -- 172.21.15.5:0/2375852854 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc61809c470 con 0x564cce391e40 2024-09-10T09:56:21.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:21.499+0000 7fc60b400640 1 -- 172.21.15.5:0/2375852854 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7fc61c0037f0 con 0x564cce391e40 2024-09-10T09:56:21.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:21.499+0000 7fc633843740 10 client.18891 fetch_fsmap learned FSMap version 42 2024-09-10T09:56:21.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:21.499+0000 7fc633843740 1 -- 172.21.15.5:0/2375852854 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x564cce3b4590 con 0x564cce391e40 2024-09-10T09:56:21.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:21.499+0000 7fc60b400640 1 -- 172.21.15.5:0/2375852854 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc61809c470 con 0x564cce391e40 2024-09-10T09:56:21.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:21.499+0000 7fc60b400640 1 -- 172.21.15.5:0/2375852854 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 42) ==== 8370+0+0 (secure 0 0 0) 0x7fc61c01b470 con 0x564cce391e40 2024-09-10T09:56:21.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:21.499+0000 7fc633843740 10 client.18891 fetch_fsmap finished waiting for FSMap version 42 2024-09-10T09:56:21.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:21.499+0000 7fc633843740 10 client.18891 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T09:56:21.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:21.499+0000 7fc633843740 20 client.18891 populate_metadata read hostname 'smithi005' 2024-09-10T09:56:21.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:21.499+0000 7fc633843740 1 --2- 172.21.15.5:0/2375852854 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x564cce3b5a50 0x564cce3b7ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:21.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:21.499+0000 7fc633843740 4 client.18891 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:56:21.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:21.499+0000 7fc633843740 1 -- 172.21.15.5:0/2375852854 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564cce305950 con 0x564cce3b5a50 2024-09-10T09:56:21.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:21.534+0000 7fc629400640 1 --2- 172.21.15.5:0/2375852854 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x564cce3b5a50 0x564cce3b7ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:21.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:21.628+0000 7fc629400640 1 --2- 172.21.15.5:0/2375852854 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x564cce3b5a50 0x564cce3b7ec0 crc :-1 s=READY pgs=255 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e45aa4c5e4886df5 server_cookie=e67f8af4e4fd06be in_seq=0 out_seq=0 2024-09-10T09:56:21.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:21.628+0000 7fc60b400640 10 client.18891 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T09:56:21.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:21.652+0000 7fc60b400640 1 -- 172.21.15.5:0/2375852854 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564cce305950 con 0x564cce3b5a50 2024-09-10T09:56:21.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:21.652+0000 7fc60b400640 10 client.18891 handle_command_reply: tid=0 2024-09-10T09:56:21.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:21.652+0000 7fc633843740 1 client.18891 shutdown 2024-09-10T09:56:21.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:21.652+0000 7fc633843740 1 -- 172.21.15.5:0/2375852854 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564cce391e40 msgr2=0x564cce39a090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:21.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:21.652+0000 7fc633843740 1 --2- 172.21.15.5:0/2375852854 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564cce391e40 0x564cce39a090 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7fc61c011450 tx=0x7fc61c011910 comp rx=0 tx=0).stop 2024-09-10T09:56:21.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:21.652+0000 7fc633843740 1 -- 172.21.15.5:0/2375852854 shutdown_connections 2024-09-10T09:56:21.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:21.652+0000 7fc633843740 1 --2- 172.21.15.5:0/2375852854 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x564cce3b5a50 0x564cce3b7ec0 crc :-1 s=READY pgs=255 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:21.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:21.652+0000 7fc633843740 1 --2- 172.21.15.5:0/2375852854 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564cce39d020 0x564cce39ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:21.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:21.652+0000 7fc633843740 1 --2- 172.21.15.5:0/2375852854 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564cce392760 0x564cce39a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:21.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:21.652+0000 7fc633843740 1 --2- 172.21.15.5:0/2375852854 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564cce391e40 0x564cce39a090 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:21.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:21.652+0000 7fc633843740 1 -- 172.21.15.5:0/2375852854 >> 172.21.15.5:0/2375852854 conn(0x564cce38e3d0 msgr2=0x564cce3940b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:56:21.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:21.652+0000 7fc60b400640 0 client.18891 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T09:56:21.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:21.652+0000 7fc633843740 1 -- 172.21.15.5:0/2375852854 shutdown_connections 2024-09-10T09:56:21.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:21.652+0000 7fc633843740 1 -- 172.21.15.5:0/2375852854 wait complete. 2024-09-10T09:56:21.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:21.652+0000 7fc633843740 20 client.18891 trim_cache size 0 max 16384 2024-09-10T09:56:21.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:21.676+0000 7fc633843740 1 Processor -- start 2024-09-10T09:56:21.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:21.676+0000 7fc633843740 1 -- start start 2024-09-10T09:56:21.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:21.676+0000 7fc633843740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564cce391e40 0x564cce39a760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:21.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:21.676+0000 7fc633843740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564cce392760 0x564cce3bd790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:21.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:21.676+0000 7fc633843740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564cce39d020 0x564cce3bdcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:21.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:21.676+0000 7fc633843740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc62c1acb90 con 0x564cce392760 2024-09-10T09:56:21.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:21.676+0000 7fc633843740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fc62c1ae6d0 con 0x564cce391e40 2024-09-10T09:56:21.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:21.676+0000 7fc633843740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fc62c1ad260 con 0x564cce39d020 2024-09-10T09:56:21.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:21.676+0000 7fc629e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564cce391e40 0x564cce39a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:21.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:21.676+0000 7fc629e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564cce391e40 0x564cce39a760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:59022/0 (socket says 172.21.15.5:59022) 2024-09-10T09:56:21.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:21.676+0000 7fc629e00640 1 -- 172.21.15.5:0/2030278249 learned_addr learned my addr 172.21.15.5:0/2030278249 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:56:21.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:21.676+0000 7fc629400640 1 --2- 172.21.15.5:0/2030278249 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564cce392760 0x564cce3bd790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:21.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:21.676+0000 7fc629400640 1 -- 172.21.15.5:0/2030278249 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564cce39d020 msgr2=0x564cce3bdcd0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T09:56:21.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:21.676+0000 7fc629400640 1 --2- 172.21.15.5:0/2030278249 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564cce39d020 0x564cce3bdcd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:21.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:21.676+0000 7fc629400640 1 -- 172.21.15.5:0/2030278249 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564cce391e40 msgr2=0x564cce39a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:56:21.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:21.676+0000 7fc629400640 1 --2- 172.21.15.5:0/2030278249 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564cce391e40 0x564cce39a760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:21.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:21.676+0000 7fc629400640 1 -- 172.21.15.5:0/2030278249 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564cce39f780 con 0x564cce392760 2024-09-10T09:56:21.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:21.676+0000 7fc629400640 1 --2- 172.21.15.5:0/2030278249 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564cce392760 0x564cce3bd790 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fc61809c620 tx=0x7fc6180d55a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9e7ab2d0992f203 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:56:21.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:21.676+0000 7fc616600640 1 -- 172.21.15.5:0/2030278249 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc61809b8b0 con 0x564cce392760 2024-09-10T09:56:21.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:21.676+0000 7fc616600640 1 -- 172.21.15.5:0/2030278249 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc61809c470 con 0x564cce392760 2024-09-10T09:56:21.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:21.676+0000 7fc616600640 1 -- 172.21.15.5:0/2030278249 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc61809bb40 con 0x564cce392760 2024-09-10T09:56:21.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:21.681+0000 7fc633843740 1 -- 172.21.15.5:0/2030278249 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564cce392760 msgr2=0x564cce3bd790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:21.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:21.681+0000 7fc633843740 1 --2- 172.21.15.5:0/2030278249 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564cce392760 0x564cce3bd790 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fc61809c620 tx=0x7fc6180d55a0 comp rx=0 tx=0).stop 2024-09-10T09:56:21.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:21.681+0000 7fc633843740 1 -- 172.21.15.5:0/2030278249 shutdown_connections 2024-09-10T09:56:21.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:21.681+0000 7fc633843740 1 --2- 172.21.15.5:0/2030278249 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564cce39d020 0x564cce3bdcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:21.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:21.681+0000 7fc633843740 1 --2- 172.21.15.5:0/2030278249 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564cce392760 0x564cce3bd790 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:21.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:21.681+0000 7fc633843740 1 --2- 172.21.15.5:0/2030278249 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564cce391e40 0x564cce39a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:21.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:21.681+0000 7fc633843740 1 -- 172.21.15.5:0/2030278249 >> 172.21.15.5:0/2030278249 conn(0x564cce38e3d0 msgr2=0x564cce39a050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:56:21.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:21.681+0000 7fc633843740 1 -- 172.21.15.5:0/2030278249 shutdown_connections 2024-09-10T09:56:21.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:21.681+0000 7fc633843740 1 -- 172.21.15.5:0/2030278249 wait complete. 2024-09-10T09:56:21.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:21.681+0000 7fc633843740 1 -- start start 2024-09-10T09:56:21.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:21.681+0000 7fc633843740 1 Processor -- start 2024-09-10T09:56:21.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:21.681+0000 7fc633843740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564cce391e40 0x564cce3ad7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:21.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:21.681+0000 7fc633843740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564cce392760 0x564cce3add30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:21.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:21.681+0000 7fc633843740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564cce39d020 0x564cce3aa7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:21.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:21.681+0000 7fc633843740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc62c1ae000 con 0x564cce391e40 2024-09-10T09:56:21.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:21.681+0000 7fc633843740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fc62c1ad930 con 0x564cce39d020 2024-09-10T09:56:21.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:21.681+0000 7fc633843740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fc62c1aabb0 con 0x564cce392760 2024-09-10T09:56:21.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:21.681+0000 7fc633843740 10 client.0 _ll_register_callbacks cb 0x7fc61c0094f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:56:21.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:21.681+0000 7fc629e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564cce391e40 0x564cce3ad7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:21.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:21.681+0000 7fc629e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564cce391e40 0x564cce3ad7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41994/0 (socket says 172.21.15.5:41994) 2024-09-10T09:56:21.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:21.681+0000 7fc629e00640 1 -- 172.21.15.5:0/2790535829 learned_addr learned my addr 172.21.15.5:0/2790535829 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:56:21.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:21.681+0000 7fc629400640 1 --2- 172.21.15.5:0/2790535829 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564cce392760 0x564cce3add30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:21.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:21.681+0000 7fc629e00640 1 -- 172.21.15.5:0/2790535829 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564cce392760 msgr2=0x564cce3add30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:56:21.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:21.681+0000 7fc629e00640 1 --2- 172.21.15.5:0/2790535829 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564cce392760 0x564cce3add30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:21.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:21.681+0000 7fc629e00640 1 -- 172.21.15.5:0/2790535829 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564cce39d020 msgr2=0x564cce3aa7c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T09:56:21.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:21.681+0000 7fc629e00640 1 --2- 172.21.15.5:0/2790535829 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564cce39d020 0x564cce3aa7c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:21.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:21.681+0000 7fc629e00640 1 -- 172.21.15.5:0/2790535829 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564cce3ae320 con 0x564cce391e40 2024-09-10T09:56:21.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:21.681+0000 7fc629e00640 1 --2- 172.21.15.5:0/2790535829 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564cce391e40 0x564cce3ad7f0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fc61c005bc0 tx=0x7fc61c01b470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53cc7705da03d64a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:56:21.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:21.681+0000 7fc60b400640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-10T09:56:21.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:21.681+0000 7fc60b400640 1 -- 172.21.15.5:0/2790535829 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc61c0052d0 con 0x564cce391e40 2024-09-10T09:56:21.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:21.681+0000 7fc60b400640 1 -- 172.21.15.5:0/2790535829 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fc61c005470 con 0x564cce391e40 2024-09-10T09:56:21.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:21.681+0000 7fc633843740 1 -- 172.21.15.5:0/2790535829 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc61809c470 con 0x564cce391e40 2024-09-10T09:56:21.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:21.681+0000 7fc60b400640 1 -- 172.21.15.5:0/2790535829 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7fc61c038050 con 0x564cce391e40 2024-09-10T09:56:21.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:21.681+0000 7fc633843740 10 client.18897 fetch_fsmap learned FSMap version 42 2024-09-10T09:56:21.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:21.681+0000 7fc633843740 1 -- 172.21.15.5:0/2790535829 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x564cce420ce0 con 0x564cce391e40 2024-09-10T09:56:21.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:21.681+0000 7fc60b400640 1 -- 172.21.15.5:0/2790535829 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc61809c470 con 0x564cce391e40 2024-09-10T09:56:21.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:21.681+0000 7fc60b400640 1 -- 172.21.15.5:0/2790535829 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 42) ==== 8370+0+0 (secure 0 0 0) 0x7fc61c012a80 con 0x564cce391e40 2024-09-10T09:56:21.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:21.681+0000 7fc633843740 10 client.18897 fetch_fsmap finished waiting for FSMap version 42 2024-09-10T09:56:21.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:21.681+0000 7fc633843740 10 client.18897 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T09:56:21.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:21.681+0000 7fc633843740 20 client.18897 populate_metadata read hostname 'smithi005' 2024-09-10T09:56:21.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:21.681+0000 7fc633843740 1 --2- 172.21.15.5:0/2790535829 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x564cce4214b0 0x564cce4238d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:21.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:21.681+0000 7fc633843740 4 client.18897 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T09:56:21.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:21.681+0000 7fc633843740 1 -- 172.21.15.5:0/2790535829 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564cce2ffe60 con 0x564cce4214b0 2024-09-10T09:56:21.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:21.681+0000 7fc629400640 1 --2- 172.21.15.5:0/2790535829 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x564cce4214b0 0x564cce4238d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:21.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:21.686+0000 7fc629400640 1 --2- 172.21.15.5:0/2790535829 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x564cce4214b0 0x564cce4238d0 crc :-1 s=READY pgs=256 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dc882e16eef68a59 server_cookie=5a5d2fbece5fed80 in_seq=0 out_seq=0 2024-09-10T09:56:21.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:21.686+0000 7fc60b400640 10 client.18897 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T09:56:21.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:21.686+0000 7fc60b400640 1 -- 172.21.15.5:0/2790535829 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564cce2ffe60 con 0x564cce4214b0 2024-09-10T09:56:21.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:21.686+0000 7fc60b400640 10 client.18897 handle_command_reply: tid=0 2024-09-10T09:56:21.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:21.686+0000 7fc633843740 1 client.18897 shutdown 2024-09-10T09:56:21.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:21.686+0000 7fc633843740 1 -- 172.21.15.5:0/2790535829 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564cce391e40 msgr2=0x564cce3ad7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:21.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:21.686+0000 7fc633843740 1 --2- 172.21.15.5:0/2790535829 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564cce391e40 0x564cce3ad7f0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fc61c005bc0 tx=0x7fc61c01b470 comp rx=0 tx=0).stop 2024-09-10T09:56:21.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:21.686+0000 7fc633843740 1 -- 172.21.15.5:0/2790535829 shutdown_connections 2024-09-10T09:56:21.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:21.686+0000 7fc633843740 1 --2- 172.21.15.5:0/2790535829 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x564cce4214b0 0x564cce4238d0 crc :-1 s=READY pgs=256 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:21.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:21.686+0000 7fc633843740 1 --2- 172.21.15.5:0/2790535829 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564cce39d020 0x564cce3aa7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:21.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:21.686+0000 7fc633843740 1 --2- 172.21.15.5:0/2790535829 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564cce392760 0x564cce3add30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:21.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:21.686+0000 7fc633843740 1 --2- 172.21.15.5:0/2790535829 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564cce391e40 0x564cce3ad7f0 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:21.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:21.686+0000 7fc633843740 1 -- 172.21.15.5:0/2790535829 >> 172.21.15.5:0/2790535829 conn(0x564cce38e3d0 msgr2=0x564cce39a030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:56:21.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:21.686+0000 7fc60b400640 0 client.18897 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T09:56:21.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:21.686+0000 7fc633843740 1 -- 172.21.15.5:0/2790535829 shutdown_connections 2024-09-10T09:56:21.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:21.686+0000 7fc633843740 1 -- 172.21.15.5:0/2790535829 wait complete. 2024-09-10T09:56:21.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:21.686+0000 7fc633843740 20 client.18897 trim_cache size 0 max 16384 2024-09-10T09:56:21.690 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-10T09:56:21.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:21.687+0000 7fc62b200640 1 -- 172.21.15.5:0/472861682 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fc6100a2c00 msgr2=0x7fc6100a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:21.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:21.687+0000 7fc62b200640 1 --2- 172.21.15.5:0/472861682 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fc6100a2c00 0x7fc6100a50c0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fc624004af0 tx=0x7fc624009360 comp rx=0 tx=0).stop 2024-09-10T09:56:21.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:21.687+0000 7fc62b200640 1 -- 172.21.15.5:0/472861682 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc62c102370 msgr2=0x7fc62c19aa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:21.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:21.687+0000 7fc62b200640 1 --2- 172.21.15.5:0/472861682 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc62c102370 0x7fc62c19aa30 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7fc61800e9f0 tx=0x7fc61800edf0 comp rx=0 tx=0).stop 2024-09-10T09:56:21.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:21.687+0000 7fc62b200640 1 -- 172.21.15.5:0/472861682 shutdown_connections 2024-09-10T09:56:21.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:21.687+0000 7fc62b200640 1 --2- 172.21.15.5:0/472861682 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fc6100a2c00 0x7fc6100a50c0 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:21.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:21.687+0000 7fc62b200640 1 --2- 172.21.15.5:0/472861682 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc62c1079d0 0x7fc62c19edc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:21.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:21.687+0000 7fc62b200640 1 --2- 172.21.15.5:0/472861682 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc62c102370 0x7fc62c19aa30 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:21.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:21.687+0000 7fc62b200640 1 --2- 172.21.15.5:0/472861682 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc62c1019d0 0x7fc62c19a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:21.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:21.687+0000 7fc62b200640 1 -- 172.21.15.5:0/472861682 >> 172.21.15.5:0/472861682 conn(0x7fc62c0fd710 msgr2=0x7fc62c10d180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:56:21.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:21.687+0000 7fc62b200640 1 -- 172.21.15.5:0/472861682 shutdown_connections 2024-09-10T09:56:21.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:21.687+0000 7fc62b200640 1 -- 172.21.15.5:0/472861682 wait complete. 2024-09-10T09:56:21.985 INFO:tasks.workunit.client.0.smithi005.stdout: 10 69576 13.72 MB/sec execute 120 sec latency 22835.082 ms 2024-09-10T09:56:22.985 INFO:tasks.workunit.client.0.smithi005.stdout: 10 69576 13.61 MB/sec execute 121 sec latency 23835.154 ms 2024-09-10T09:56:23.273 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:56:23 smithi005 ceph-mon[30350]: pgmap v934: 129 pgs: 129 active+clean; 604 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 2.7 MiB/s wr, 61 op/s 2024-09-10T09:56:23.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:56:23 smithi102 ceph-mon[35442]: pgmap v934: 129 pgs: 129 active+clean; 604 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 2.7 MiB/s wr, 61 op/s 2024-09-10T09:56:23.402 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:56:23 smithi159 ceph-mon[35513]: pgmap v934: 129 pgs: 129 active+clean; 604 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 2.7 MiB/s wr, 61 op/s 2024-09-10T09:56:23.985 INFO:tasks.workunit.client.0.smithi005.stdout: 10 69576 13.49 MB/sec execute 122 sec latency 24835.198 ms 2024-09-10T09:56:24.985 INFO:tasks.workunit.client.0.smithi005.stdout: 10 69576 13.38 MB/sec execute 123 sec latency 25835.241 ms 2024-09-10T09:56:25.057 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-10T09:56:25.057 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-10T09:56:25.243 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:56:25 smithi005 ceph-mon[30350]: pgmap v935: 129 pgs: 129 active+clean; 606 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 3.6 MiB/s wr, 87 op/s 2024-09-10T09:56:25.424 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T09:56:25.502 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:56:25.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:56:25 smithi159 ceph-mon[35513]: pgmap v935: 129 pgs: 129 active+clean; 606 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 3.6 MiB/s wr, 87 op/s 2024-09-10T09:56:25.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:56:25 smithi102 ceph-mon[35442]: pgmap v935: 129 pgs: 129 active+clean; 606 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 3.6 MiB/s wr, 87 op/s 2024-09-10T09:56:25.985 INFO:tasks.workunit.client.0.smithi005.stdout: 10 69576 13.28 MB/sec execute 124 sec latency 26835.282 ms 2024-09-10T09:56:26.985 INFO:tasks.workunit.client.0.smithi005.stdout: 10 69576 13.17 MB/sec execute 125 sec latency 27835.328 ms 2024-09-10T09:56:27.424 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:56:27 smithi005 ceph-mon[30350]: pgmap v936: 129 pgs: 129 active+clean; 598 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 4.0 MiB/s wr, 117 op/s 2024-09-10T09:56:27.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:56:27 smithi159 ceph-mon[35513]: pgmap v936: 129 pgs: 129 active+clean; 598 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 4.0 MiB/s wr, 117 op/s 2024-09-10T09:56:27.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:56:27 smithi102 ceph-mon[35442]: pgmap v936: 129 pgs: 129 active+clean; 598 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 4.0 MiB/s wr, 117 op/s 2024-09-10T09:56:27.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:27.738+0000 7fed39400640 1 -- 172.21.15.5:0/1958283423 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fed341049c0 msgr2=0x7fed34108a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:27.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:27.738+0000 7fed39400640 1 --2- 172.21.15.5:0/1958283423 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fed341049c0 0x7fed34108a10 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7fed20009860 tx=0x7fed2002a360 comp rx=0 tx=0).stop 2024-09-10T09:56:27.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:27.738+0000 7fed39400640 1 -- 172.21.15.5:0/1958283423 shutdown_connections 2024-09-10T09:56:27.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:27.738+0000 7fed39400640 1 --2- 172.21.15.5:0/1958283423 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fed34109140 0x7fed34110cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:27.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:27.738+0000 7fed39400640 1 --2- 172.21.15.5:0/1958283423 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fed341049c0 0x7fed34108a10 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:27.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:27.738+0000 7fed39400640 1 --2- 172.21.15.5:0/1958283423 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fed34104010 0x7fed341043f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:27.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:27.738+0000 7fed39400640 1 -- 172.21.15.5:0/1958283423 >> 172.21.15.5:0/1958283423 conn(0x7fed340ffa50 msgr2=0x7fed34101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:56:27.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:27.738+0000 7fed39400640 1 -- 172.21.15.5:0/1958283423 shutdown_connections 2024-09-10T09:56:27.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:27.738+0000 7fed39400640 1 -- 172.21.15.5:0/1958283423 wait complete. 2024-09-10T09:56:27.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:27.738+0000 7fed39400640 1 Processor -- start 2024-09-10T09:56:27.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:27.738+0000 7fed39400640 1 -- start start 2024-09-10T09:56:27.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:27.738+0000 7fed39400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fed34104010 0x7fed341a03f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:27.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:27.738+0000 7fed39400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fed341049c0 0x7fed341a0930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:27.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:27.738+0000 7fed39400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fed34109140 0x7fed3419a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:27.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:27.738+0000 7fed39400640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fed341141e0 con 0x7fed341049c0 2024-09-10T09:56:27.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:27.738+0000 7fed39400640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fed34114060 con 0x7fed34104010 2024-09-10T09:56:27.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:27.738+0000 7fed39400640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fed34114360 con 0x7fed34109140 2024-09-10T09:56:27.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:27.738+0000 7fed31e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fed341049c0 0x7fed341a0930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:27.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:27.738+0000 7fed31e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fed341049c0 0x7fed341a0930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42016/0 (socket says 172.21.15.5:42016) 2024-09-10T09:56:27.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:27.738+0000 7fed31e00640 1 -- 172.21.15.5:0/3164207638 learned_addr learned my addr 172.21.15.5:0/3164207638 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:56:27.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:27.738+0000 7fed32800640 1 --2- 172.21.15.5:0/3164207638 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fed34104010 0x7fed341a03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:27.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:27.738+0000 7fed33200640 1 --2- 172.21.15.5:0/3164207638 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fed34109140 0x7fed3419a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:27.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:27.738+0000 7fed31e00640 1 -- 172.21.15.5:0/3164207638 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fed34109140 msgr2=0x7fed3419a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:27.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:27.738+0000 7fed31e00640 1 --2- 172.21.15.5:0/3164207638 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fed34109140 0x7fed3419a5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:27.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:27.738+0000 7fed31e00640 1 -- 172.21.15.5:0/3164207638 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fed34104010 msgr2=0x7fed341a03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:27.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:27.738+0000 7fed31e00640 1 --2- 172.21.15.5:0/3164207638 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fed34104010 0x7fed341a03f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:27.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:27.738+0000 7fed31e00640 1 -- 172.21.15.5:0/3164207638 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed3419ace0 con 0x7fed341049c0 2024-09-10T09:56:27.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:27.738+0000 7fed33200640 1 --2- 172.21.15.5:0/3164207638 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fed34109140 0x7fed3419a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:56:27.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:27.738+0000 7fed32800640 1 --2- 172.21.15.5:0/3164207638 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fed34104010 0x7fed341a03f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:56:27.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:27.738+0000 7fed31e00640 1 --2- 172.21.15.5:0/3164207638 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fed341049c0 0x7fed341a0930 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7fed20035040 tx=0x7fed20032780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:56:27.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:27.738+0000 7fed2b400640 1 -- 172.21.15.5:0/3164207638 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed2003f020 con 0x7fed341049c0 2024-09-10T09:56:27.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:27.738+0000 7fed39400640 1 -- 172.21.15.5:0/3164207638 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed3419af70 con 0x7fed341049c0 2024-09-10T09:56:27.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:27.738+0000 7fed2b400640 1 -- 172.21.15.5:0/3164207638 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fed20033d60 con 0x7fed341049c0 2024-09-10T09:56:27.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:27.743+0000 7fed2b400640 1 -- 172.21.15.5:0/3164207638 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed20037040 con 0x7fed341049c0 2024-09-10T09:56:27.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:27.743+0000 7fed39400640 1 -- 172.21.15.5:0/3164207638 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed341a72c0 con 0x7fed341049c0 2024-09-10T09:56:27.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:27.744+0000 7fed3bdde740 1 Processor -- start 2024-09-10T09:56:27.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:27.744+0000 7fed2b400640 1 -- 172.21.15.5:0/3164207638 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7fed20042020 con 0x7fed341049c0 2024-09-10T09:56:27.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:27.744+0000 7fed3bdde740 1 -- start start 2024-09-10T09:56:27.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:27.744+0000 7fed3bdde740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a18fa7dac0 0x55a18fa7dea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:27.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:27.744+0000 7fed3bdde740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a18fa7e3e0 0x55a18fa88760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:27.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:27.744+0000 7fed3bdde740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a18fa88ca0 0x55a18fa8b090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:27.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:27.744+0000 7fed3bdde740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55a18f9e4290 con 0x55a18fa7e3e0 2024-09-10T09:56:27.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:27.744+0000 7fed3bdde740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55a18f98bb40 con 0x55a18fa7dac0 2024-09-10T09:56:27.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:27.744+0000 7fed3bdde740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55a18f9e0250 con 0x55a18fa88ca0 2024-09-10T09:56:27.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:27.744+0000 7fed32800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a18fa7e3e0 0x55a18fa88760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:27.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:27.744+0000 7fed33200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a18fa7dac0 0x55a18fa7dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:27.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:27.744+0000 7fed32800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a18fa7e3e0 0x55a18fa88760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42032/0 (socket says 172.21.15.5:42032) 2024-09-10T09:56:27.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:27.744+0000 7fed32800640 1 -- 172.21.15.5:0/2763924763 learned_addr learned my addr 172.21.15.5:0/2763924763 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:56:27.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:27.744+0000 7fed31e00640 1 --2- 172.21.15.5:0/2763924763 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a18fa88ca0 0x55a18fa8b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:27.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:27.744+0000 7fed32800640 1 -- 172.21.15.5:0/2763924763 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a18fa88ca0 msgr2=0x55a18fa8b090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:56:27.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:27.744+0000 7fed32800640 1 --2- 172.21.15.5:0/2763924763 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a18fa88ca0 0x55a18fa8b090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:27.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:27.744+0000 7fed32800640 1 -- 172.21.15.5:0/2763924763 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a18fa7dac0 msgr2=0x55a18fa7dea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:56:27.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:27.744+0000 7fed32800640 1 --2- 172.21.15.5:0/2763924763 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a18fa7dac0 0x55a18fa7dea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:27.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:27.744+0000 7fed32800640 1 -- 172.21.15.5:0/2763924763 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a18f7556a0 con 0x55a18fa7e3e0 2024-09-10T09:56:27.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:27.744+0000 7fed31e00640 1 --2- 172.21.15.5:0/2763924763 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a18fa88ca0 0x55a18fa8b090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:56:27.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:27.744+0000 7fed32800640 1 --2- 172.21.15.5:0/2763924763 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a18fa7e3e0 0x55a18fa88760 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fed24009df0 tx=0x7fed2400e5a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d6fd186ba8c0c77 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:56:27.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:27.744+0000 7fed2b400640 1 --2- 172.21.15.5:0/3164207638 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fed180a2b80 0x7fed180a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:27.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:27.749+0000 7fed2b400640 1 -- 172.21.15.5:0/3164207638 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fed200ef2e0 con 0x7fed341049c0 2024-09-10T09:56:27.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:27.749+0000 7fed33200640 1 --2- 172.21.15.5:0/3164207638 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fed180a2b80 0x7fed180a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:27.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:27.749+0000 7fed1f400640 1 -- 172.21.15.5:0/2763924763 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed24017070 con 0x55a18fa7e3e0 2024-09-10T09:56:27.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:27.749+0000 7fed1f400640 1 -- 172.21.15.5:0/2763924763 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fed24013950 con 0x55a18fa7e3e0 2024-09-10T09:56:27.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:27.749+0000 7fed1f400640 1 -- 172.21.15.5:0/2763924763 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed24012900 con 0x55a18fa7e3e0 2024-09-10T09:56:27.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:27.749+0000 7fed3bdde740 1 -- 172.21.15.5:0/2763924763 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a18fa7e3e0 msgr2=0x55a18fa88760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:27.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:27.749+0000 7fed33200640 1 --2- 172.21.15.5:0/3164207638 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fed180a2b80 0x7fed180a5040 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fed2c005fd0 tx=0x7fed2c010040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:56:27.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:27.749+0000 7fed3bdde740 1 --2- 172.21.15.5:0/2763924763 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a18fa7e3e0 0x55a18fa88760 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fed24009df0 tx=0x7fed2400e5a0 comp rx=0 tx=0).stop 2024-09-10T09:56:27.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:27.749+0000 7fed3bdde740 1 -- 172.21.15.5:0/2763924763 shutdown_connections 2024-09-10T09:56:27.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:27.749+0000 7fed3bdde740 1 --2- 172.21.15.5:0/2763924763 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a18fa88ca0 0x55a18fa8b090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:27.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:27.749+0000 7fed3bdde740 1 --2- 172.21.15.5:0/2763924763 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a18fa7e3e0 0x55a18fa88760 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:27.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:27.749+0000 7fed3bdde740 1 --2- 172.21.15.5:0/2763924763 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a18fa7dac0 0x55a18fa7dea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:27.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:27.749+0000 7fed3bdde740 1 -- 172.21.15.5:0/2763924763 >> 172.21.15.5:0/2763924763 conn(0x55a18fa7a050 msgr2=0x55a18fa7a430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:56:27.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:27.749+0000 7fed3bdde740 1 -- 172.21.15.5:0/2763924763 shutdown_connections 2024-09-10T09:56:27.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:27.749+0000 7fed3bdde740 1 -- 172.21.15.5:0/2763924763 wait complete. 2024-09-10T09:56:27.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:27.749+0000 7fed3bdde740 1 -- start start 2024-09-10T09:56:27.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:27.749+0000 7fed3bdde740 1 Processor -- start 2024-09-10T09:56:27.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:27.749+0000 7fed3bdde740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a18fa7dac0 0x55a18fa885b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:27.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:27.749+0000 7fed3bdde740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a18fa7e3e0 0x55a18fa85860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:27.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:27.749+0000 7fed3bdde740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a18fa88ca0 0x55a18fa85da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:27.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:27.749+0000 7fed3bdde740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55a18fa8b910 con 0x55a18fa88ca0 2024-09-10T09:56:27.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:27.749+0000 7fed3bdde740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55a18f99efa0 con 0x55a18fa7e3e0 2024-09-10T09:56:27.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:27.749+0000 7fed3bdde740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55a18fa8ba90 con 0x55a18fa7dac0 2024-09-10T09:56:27.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:27.749+0000 7fed3bdde740 10 client.0 _ll_register_callbacks cb 0x55a18f6f2ba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:56:27.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:27.749+0000 7fed32800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a18fa7dac0 0x55a18fa885b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:27.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:27.749+0000 7fed32800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a18fa7dac0 0x55a18fa885b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:37308/0 (socket says 172.21.15.5:37308) 2024-09-10T09:56:27.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:27.749+0000 7fed32800640 1 -- 172.21.15.5:0/1150651638 learned_addr learned my addr 172.21.15.5:0/1150651638 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:56:27.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:27.749+0000 7fed33200640 1 --2- 172.21.15.5:0/1150651638 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a18fa88ca0 0x55a18fa85da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:27.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:27.749+0000 7fed31e00640 1 --2- 172.21.15.5:0/1150651638 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a18fa7e3e0 0x55a18fa85860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:27.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:27.749+0000 7fed32800640 1 -- 172.21.15.5:0/1150651638 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a18fa7e3e0 msgr2=0x55a18fa85860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:56:27.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:27.749+0000 7fed32800640 1 --2- 172.21.15.5:0/1150651638 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a18fa7e3e0 0x55a18fa85860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:27.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:27.749+0000 7fed32800640 1 -- 172.21.15.5:0/1150651638 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a18fa88ca0 msgr2=0x55a18fa85da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:56:27.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:27.749+0000 7fed32800640 1 --2- 172.21.15.5:0/1150651638 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a18fa88ca0 0x55a18fa85da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:27.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:27.749+0000 7fed32800640 1 -- 172.21.15.5:0/1150651638 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a18fa865a0 con 0x55a18fa7dac0 2024-09-10T09:56:27.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:27.749+0000 7fed32800640 1 --2- 172.21.15.5:0/1150651638 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a18fa7dac0 0x55a18fa885b0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fed24000c00 tx=0x7fed24004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7175d3cd1772dce2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:56:27.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:27.749+0000 7fed13e00640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-09-10T09:56:27.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:27.749+0000 7fed13e00640 1 -- 172.21.15.5:0/1150651638 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed24029060 con 0x55a18fa7dac0 2024-09-10T09:56:27.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:27.749+0000 7fed13e00640 1 -- 172.21.15.5:0/1150651638 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fed24012680 con 0x55a18fa7dac0 2024-09-10T09:56:27.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:27.749+0000 7fed3bdde740 1 -- 172.21.15.5:0/1150651638 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fed24013950 con 0x55a18fa7dac0 2024-09-10T09:56:27.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:27.754+0000 7fed13e00640 1 -- 172.21.15.5:0/1150651638 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7fed24030050 con 0x55a18fa7dac0 2024-09-10T09:56:27.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:27.754+0000 7fed3bdde740 10 client.28957 fetch_fsmap learned FSMap version 42 2024-09-10T09:56:27.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:27.754+0000 7fed3bdde740 1 -- 172.21.15.5:0/1150651638 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55a18fa977f0 con 0x55a18fa7dac0 2024-09-10T09:56:27.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:27.754+0000 7fed13e00640 1 -- 172.21.15.5:0/1150651638 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fed24013950 con 0x55a18fa7dac0 2024-09-10T09:56:27.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:27.754+0000 7fed13e00640 1 -- 172.21.15.5:0/1150651638 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 42) ==== 8370+0+0 (secure 0 0 0) 0x7fed2401c7f0 con 0x55a18fa7dac0 2024-09-10T09:56:27.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:27.754+0000 7fed3bdde740 10 client.28957 fetch_fsmap finished waiting for FSMap version 42 2024-09-10T09:56:27.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:27.754+0000 7fed3bdde740 10 client.28957 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T09:56:27.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:27.754+0000 7fed3bdde740 20 client.28957 populate_metadata read hostname 'smithi005' 2024-09-10T09:56:27.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:27.754+0000 7fed3bdde740 1 --2- 172.21.15.5:0/1150651638 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55a18fa98b40 0x55a18fa9af30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:27.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:27.754+0000 7fed3bdde740 4 client.28957 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:56:27.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:27.754+0000 7fed3bdde740 1 -- 172.21.15.5:0/1150651638 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a18f9f1800 con 0x55a18fa98b40 2024-09-10T09:56:27.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:27.754+0000 7fed31e00640 1 --2- 172.21.15.5:0/1150651638 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55a18fa98b40 0x55a18fa9af30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:27.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:27.754+0000 7fed31e00640 1 --2- 172.21.15.5:0/1150651638 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55a18fa98b40 0x55a18fa9af30 crc :-1 s=READY pgs=257 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=71e7bd9d0305e377 server_cookie=588ad19721627db7 in_seq=0 out_seq=0 2024-09-10T09:56:27.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:27.754+0000 7fed13e00640 10 client.28957 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T09:56:27.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:27.756+0000 7fed13e00640 1 -- 172.21.15.5:0/1150651638 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a18f9f1800 con 0x55a18fa98b40 2024-09-10T09:56:27.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:27.756+0000 7fed13e00640 10 client.28957 handle_command_reply: tid=0 2024-09-10T09:56:27.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:27.756+0000 7fed3bdde740 1 client.28957 shutdown 2024-09-10T09:56:27.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:27.756+0000 7fed3bdde740 1 -- 172.21.15.5:0/1150651638 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a18fa7dac0 msgr2=0x55a18fa885b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:27.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:27.756+0000 7fed3bdde740 1 --2- 172.21.15.5:0/1150651638 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a18fa7dac0 0x55a18fa885b0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fed24000c00 tx=0x7fed24004750 comp rx=0 tx=0).stop 2024-09-10T09:56:27.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:27.756+0000 7fed3bdde740 1 -- 172.21.15.5:0/1150651638 shutdown_connections 2024-09-10T09:56:27.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:27.756+0000 7fed3bdde740 1 --2- 172.21.15.5:0/1150651638 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55a18fa98b40 0x55a18fa9af30 crc :-1 s=READY pgs=257 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:27.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:27.756+0000 7fed3bdde740 1 --2- 172.21.15.5:0/1150651638 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a18fa88ca0 0x55a18fa85da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:27.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:27.756+0000 7fed3bdde740 1 --2- 172.21.15.5:0/1150651638 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a18fa7e3e0 0x55a18fa85860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:27.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:27.756+0000 7fed3bdde740 1 --2- 172.21.15.5:0/1150651638 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a18fa7dac0 0x55a18fa885b0 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:27.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:27.756+0000 7fed13e00640 0 client.28957 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T09:56:27.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:27.756+0000 7fed3bdde740 1 -- 172.21.15.5:0/1150651638 >> 172.21.15.5:0/1150651638 conn(0x55a18fa7a050 msgr2=0x55a18fa7fd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:56:27.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:27.756+0000 7fed3bdde740 1 -- 172.21.15.5:0/1150651638 shutdown_connections 2024-09-10T09:56:27.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:27.756+0000 7fed3bdde740 1 -- 172.21.15.5:0/1150651638 wait complete. 2024-09-10T09:56:27.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:27.756+0000 7fed3bdde740 20 client.28957 trim_cache size 0 max 16384 2024-09-10T09:56:27.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:27.773+0000 7fed3bdde740 1 Processor -- start 2024-09-10T09:56:27.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:27.773+0000 7fed3bdde740 1 -- start start 2024-09-10T09:56:27.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:27.773+0000 7fed3bdde740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a18fa7dac0 0x55a18fa7fc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:27.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:27.773+0000 7fed3bdde740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a18fa7e3e0 0x55a18fa880d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:27.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:27.773+0000 7fed3bdde740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a18fa88ca0 0x55a18faa0590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:27.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:27.773+0000 7fed3bdde740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fed341b1120 con 0x55a18fa7e3e0 2024-09-10T09:56:27.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:27.773+0000 7fed3bdde740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fed341b2c60 con 0x55a18fa88ca0 2024-09-10T09:56:27.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:27.773+0000 7fed3bdde740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fed341b17f0 con 0x55a18fa7dac0 2024-09-10T09:56:27.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:27.773+0000 7fed31e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a18fa7e3e0 0x55a18fa880d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:27.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:27.773+0000 7fed32800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a18fa7dac0 0x55a18fa7fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:27.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:27.778+0000 7fed32800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a18fa7dac0 0x55a18fa7fc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:37318/0 (socket says 172.21.15.5:37318) 2024-09-10T09:56:27.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:27.778+0000 7fed32800640 1 -- 172.21.15.5:0/1857111313 learned_addr learned my addr 172.21.15.5:0/1857111313 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:56:27.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:27.778+0000 7fed32800640 1 -- 172.21.15.5:0/1857111313 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a18fa88ca0 msgr2=0x55a18faa0590 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T09:56:27.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:27.778+0000 7fed32800640 1 --2- 172.21.15.5:0/1857111313 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a18fa88ca0 0x55a18faa0590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:27.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:27.778+0000 7fed32800640 1 -- 172.21.15.5:0/1857111313 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a18fa7e3e0 msgr2=0x55a18fa880d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:56:27.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:27.778+0000 7fed32800640 1 --2- 172.21.15.5:0/1857111313 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a18fa7e3e0 0x55a18fa880d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:27.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:27.778+0000 7fed32800640 1 -- 172.21.15.5:0/1857111313 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a18fa79e80 con 0x55a18fa7dac0 2024-09-10T09:56:27.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:27.778+0000 7fed31e00640 1 --2- 172.21.15.5:0/1857111313 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a18fa7e3e0 0x55a18fa880d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:56:27.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:27.778+0000 7fed32800640 1 --2- 172.21.15.5:0/1857111313 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a18fa7dac0 0x55a18fa7fc70 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fed24010010 tx=0x7fed24012510 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b1fe2353ff8cf60c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:56:27.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:27.778+0000 7fed1ea00640 1 -- 172.21.15.5:0/1857111313 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed24029040 con 0x55a18fa7dac0 2024-09-10T09:56:27.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:27.778+0000 7fed1ea00640 1 -- 172.21.15.5:0/1857111313 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fed2401c930 con 0x55a18fa7dac0 2024-09-10T09:56:27.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:27.778+0000 7fed1ea00640 1 -- 172.21.15.5:0/1857111313 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed24017040 con 0x55a18fa7dac0 2024-09-10T09:56:27.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:27.779+0000 7fed3bdde740 1 -- 172.21.15.5:0/1857111313 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a18fa7dac0 msgr2=0x55a18fa7fc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:27.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:27.779+0000 7fed3bdde740 1 --2- 172.21.15.5:0/1857111313 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a18fa7dac0 0x55a18fa7fc70 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fed24010010 tx=0x7fed24012510 comp rx=0 tx=0).stop 2024-09-10T09:56:27.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:27.779+0000 7fed3bdde740 1 -- 172.21.15.5:0/1857111313 shutdown_connections 2024-09-10T09:56:27.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:27.779+0000 7fed3bdde740 1 --2- 172.21.15.5:0/1857111313 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a18fa88ca0 0x55a18faa0590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:27.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:27.779+0000 7fed3bdde740 1 --2- 172.21.15.5:0/1857111313 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a18fa7e3e0 0x55a18fa880d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:27.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:27.779+0000 7fed3bdde740 1 --2- 172.21.15.5:0/1857111313 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a18fa7dac0 0x55a18fa7fc70 secure :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fed24010010 tx=0x7fed24012510 comp rx=0 tx=0).stop 2024-09-10T09:56:27.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:27.779+0000 7fed3bdde740 1 -- 172.21.15.5:0/1857111313 >> 172.21.15.5:0/1857111313 conn(0x55a18fa7a050 msgr2=0x55a18fa7f560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:56:27.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:27.779+0000 7fed3bdde740 1 -- 172.21.15.5:0/1857111313 shutdown_connections 2024-09-10T09:56:27.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:27.779+0000 7fed3bdde740 1 -- 172.21.15.5:0/1857111313 wait complete. 2024-09-10T09:56:27.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:27.779+0000 7fed3bdde740 1 -- start start 2024-09-10T09:56:27.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:27.779+0000 7fed3bdde740 1 Processor -- start 2024-09-10T09:56:27.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:27.779+0000 7fed3bdde740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a18fa7e3e0 0x55a18fa8d570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:27.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:27.779+0000 7fed3bdde740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a18fa88ca0 0x55a18fa8dab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:27.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:27.779+0000 7fed3bdde740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a18fb03d30 0x55a18fb041b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:27.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:27.779+0000 7fed3bdde740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fed341b2590 con 0x55a18fa7e3e0 2024-09-10T09:56:27.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:27.779+0000 7fed3bdde740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fed341b1ec0 con 0x55a18fb03d30 2024-09-10T09:56:27.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:27.779+0000 7fed3bdde740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fed341af100 con 0x55a18fa88ca0 2024-09-10T09:56:27.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:27.779+0000 7fed3bdde740 10 client.0 _ll_register_callbacks cb 0x7fed2c00e450 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:56:27.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:27.779+0000 7fed33200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a18fb03d30 0x55a18fb041b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:27.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:27.779+0000 7fed31e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a18fa88ca0 0x55a18fa8dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:27.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:27.779+0000 7fed33200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a18fb03d30 0x55a18fb041b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:59098/0 (socket says 172.21.15.5:59098) 2024-09-10T09:56:27.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:27.779+0000 7fed33200640 1 -- 172.21.15.5:0/3077197373 learned_addr learned my addr 172.21.15.5:0/3077197373 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:56:27.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:27.779+0000 7fed32800640 1 --2- 172.21.15.5:0/3077197373 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a18fa7e3e0 0x55a18fa8d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:27.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:27.779+0000 7fed33200640 1 -- 172.21.15.5:0/3077197373 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a18fa88ca0 msgr2=0x55a18fa8dab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:56:27.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:27.779+0000 7fed33200640 1 --2- 172.21.15.5:0/3077197373 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a18fa88ca0 0x55a18fa8dab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:27.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:27.779+0000 7fed33200640 1 -- 172.21.15.5:0/3077197373 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a18fa7e3e0 msgr2=0x55a18fa8d570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:56:27.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:27.779+0000 7fed33200640 1 --2- 172.21.15.5:0/3077197373 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a18fa7e3e0 0x55a18fa8d570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:27.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:27.779+0000 7fed33200640 1 -- 172.21.15.5:0/3077197373 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a18fb048f0 con 0x55a18fb03d30 2024-09-10T09:56:27.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:27.779+0000 7fed32800640 1 --2- 172.21.15.5:0/3077197373 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a18fa7e3e0 0x55a18fa8d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:56:27.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:27.779+0000 7fed33200640 1 --2- 172.21.15.5:0/3077197373 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a18fb03d30 0x55a18fb041b0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fed2c003ef0 tx=0x7fed2c004ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8af14536a02b3f51 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:56:27.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:27.779+0000 7fed13e00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-09-10T09:56:27.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:27.779+0000 7fed13e00640 1 -- 172.21.15.5:0/3077197373 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed2c02a030 con 0x55a18fb03d30 2024-09-10T09:56:27.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:27.779+0000 7fed13e00640 1 -- 172.21.15.5:0/3077197373 <== mon.2 v2:172.21.15.102:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fed2c004b20 con 0x55a18fb03d30 2024-09-10T09:56:27.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:27.779+0000 7fed3bdde740 1 -- 172.21.15.5:0/3077197373 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fed2401c930 con 0x55a18fb03d30 2024-09-10T09:56:27.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:27.779+0000 7fed13e00640 1 -- 172.21.15.5:0/3077197373 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7fed2c034050 con 0x55a18fb03d30 2024-09-10T09:56:27.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:27.779+0000 7fed3bdde740 10 client.28964 fetch_fsmap learned FSMap version 42 2024-09-10T09:56:27.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:27.779+0000 7fed3bdde740 1 -- 172.21.15.5:0/3077197373 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55a18fb0cce0 con 0x55a18fb03d30 2024-09-10T09:56:27.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:27.779+0000 7fed13e00640 1 -- 172.21.15.5:0/3077197373 <== mon.2 v2:172.21.15.102:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fed2401c930 con 0x55a18fb03d30 2024-09-10T09:56:27.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:27.785+0000 7fed13e00640 1 -- 172.21.15.5:0/3077197373 <== mon.2 v2:172.21.15.102:3300/0 5 ==== fsmap(e 42) ==== 8370+0+0 (secure 0 0 0) 0x7fed2c00b040 con 0x55a18fb03d30 2024-09-10T09:56:27.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:27.785+0000 7fed3bdde740 10 client.28964 fetch_fsmap finished waiting for FSMap version 42 2024-09-10T09:56:27.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:27.785+0000 7fed3bdde740 10 client.28964 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T09:56:27.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:27.785+0000 7fed3bdde740 20 client.28964 populate_metadata read hostname 'smithi005' 2024-09-10T09:56:27.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:27.785+0000 7fed3bdde740 1 --2- 172.21.15.5:0/3077197373 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55a18fb0d4b0 0x55a18fb0f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:27.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:27.785+0000 7fed3bdde740 4 client.28964 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-10T09:56:27.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:27.785+0000 7fed3bdde740 1 -- 172.21.15.5:0/3077197373 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55a18f9ebd10 con 0x55a18fb0d4b0 2024-09-10T09:56:27.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:27.803+0000 7fed32800640 1 --2- 172.21.15.5:0/3077197373 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55a18fb0d4b0 0x55a18fb0f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:27.814 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:56:27.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:27.839+0000 7fed32800640 1 --2- 172.21.15.5:0/3077197373 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55a18fb0d4b0 0x55a18fb0f8d0 crc :-1 s=READY pgs=258 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=53adac9d8f89da75 server_cookie=fa207851eaf89068 in_seq=0 out_seq=0 2024-09-10T09:56:27.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:27.839+0000 7fed13e00640 10 client.28964 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T09:56:27.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:27.841+0000 7fed13e00640 1 -- 172.21.15.5:0/3077197373 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55a18f9ebd10 con 0x55a18fb0d4b0 2024-09-10T09:56:27.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:27.841+0000 7fed13e00640 10 client.28964 handle_command_reply: tid=0 2024-09-10T09:56:27.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:27.841+0000 7fed3bdde740 1 client.28964 shutdown 2024-09-10T09:56:27.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:27.841+0000 7fed3bdde740 1 -- 172.21.15.5:0/3077197373 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a18fb03d30 msgr2=0x55a18fb041b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:27.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:27.841+0000 7fed3bdde740 1 --2- 172.21.15.5:0/3077197373 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a18fb03d30 0x55a18fb041b0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fed2c003ef0 tx=0x7fed2c004ab0 comp rx=0 tx=0).stop 2024-09-10T09:56:27.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:27.841+0000 7fed3bdde740 1 -- 172.21.15.5:0/3077197373 shutdown_connections 2024-09-10T09:56:27.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:27.841+0000 7fed3bdde740 1 --2- 172.21.15.5:0/3077197373 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55a18fb0d4b0 0x55a18fb0f8d0 crc :-1 s=READY pgs=258 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:27.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:27.841+0000 7fed3bdde740 1 --2- 172.21.15.5:0/3077197373 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55a18fb03d30 0x55a18fb041b0 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:27.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:27.841+0000 7fed3bdde740 1 --2- 172.21.15.5:0/3077197373 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55a18fa88ca0 0x55a18fa8dab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:27.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:27.841+0000 7fed3bdde740 1 --2- 172.21.15.5:0/3077197373 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a18fa7e3e0 0x55a18fa8d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:27.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:27.841+0000 7fed3bdde740 1 -- 172.21.15.5:0/3077197373 >> 172.21.15.5:0/3077197373 conn(0x55a18fa7a050 msgr2=0x55a18fa7fd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:56:27.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:27.841+0000 7fed13e00640 0 client.28964 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T09:56:27.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:27.841+0000 7fed3bdde740 1 -- 172.21.15.5:0/3077197373 shutdown_connections 2024-09-10T09:56:27.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:27.841+0000 7fed3bdde740 1 -- 172.21.15.5:0/3077197373 wait complete. 2024-09-10T09:56:27.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:27.842+0000 7fed3bdde740 20 client.28964 trim_cache size 0 max 16384 2024-09-10T09:56:27.844 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-10T09:56:27.844 INFO:teuthology.orchestra.run.smithi005.stdout: "return_code": 0, 2024-09-10T09:56:27.844 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_tag": "7b6a3c33-cc02-48c1-a924-daf7fcbb8f82", 2024-09-10T09:56:27.844 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": "asynchronous" 2024-09-10T09:56:27.844 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-10T09:56:27.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:27.843+0000 7fed39400640 1 -- 172.21.15.5:0/3164207638 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fed180a2b80 msgr2=0x7fed180a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:27.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:27.843+0000 7fed39400640 1 --2- 172.21.15.5:0/3164207638 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fed180a2b80 0x7fed180a5040 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fed2c005fd0 tx=0x7fed2c010040 comp rx=0 tx=0).stop 2024-09-10T09:56:27.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:27.843+0000 7fed39400640 1 -- 172.21.15.5:0/3164207638 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fed341049c0 msgr2=0x7fed341a0930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:27.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:27.843+0000 7fed39400640 1 --2- 172.21.15.5:0/3164207638 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fed341049c0 0x7fed341a0930 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7fed20035040 tx=0x7fed20032780 comp rx=0 tx=0).stop 2024-09-10T09:56:27.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:27.843+0000 7fed39400640 1 -- 172.21.15.5:0/3164207638 shutdown_connections 2024-09-10T09:56:27.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:27.843+0000 7fed39400640 1 --2- 172.21.15.5:0/3164207638 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fed180a2b80 0x7fed180a5040 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:27.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:27.843+0000 7fed39400640 1 --2- 172.21.15.5:0/3164207638 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fed34109140 0x7fed3419a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:27.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:27.843+0000 7fed39400640 1 --2- 172.21.15.5:0/3164207638 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fed341049c0 0x7fed341a0930 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:27.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:27.843+0000 7fed39400640 1 --2- 172.21.15.5:0/3164207638 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fed34104010 0x7fed341a03f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:27.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:27.843+0000 7fed39400640 1 -- 172.21.15.5:0/3164207638 >> 172.21.15.5:0/3164207638 conn(0x7fed340ffa50 msgr2=0x7fed34100e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:56:27.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:27.843+0000 7fed39400640 1 -- 172.21.15.5:0/3164207638 shutdown_connections 2024-09-10T09:56:27.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:27.843+0000 7fed39400640 1 -- 172.21.15.5:0/3164207638 wait complete. 2024-09-10T09:56:27.985 INFO:tasks.workunit.client.0.smithi005.stdout: 10 69576 13.07 MB/sec execute 126 sec latency 28835.449 ms 2024-09-10T09:56:28.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:56:28 smithi005 ceph-mon[30350]: scrub queued for path: / 2024-09-10T09:56:28.557 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:56:28 smithi005 ceph-mon[30350]: scrub summary: idle+waiting paths [/] 2024-09-10T09:56:28.557 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:56:28 smithi005 ceph-mon[30350]: scrub summary: active paths [/] 2024-09-10T09:56:28.557 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:56:28 smithi005 ceph-mon[30350]: scrub summary: active paths [/] 2024-09-10T09:56:28.557 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:56:28 smithi005 ceph-mon[30350]: scrub summary: idle+waiting paths [/] 2024-09-10T09:56:28.557 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:56:28 smithi005 ceph-mon[30350]: scrub summary: idle+waiting paths [/] 2024-09-10T09:56:28.557 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:56:28 smithi005 ceph-mon[30350]: scrub summary: active paths [/] 2024-09-10T09:56:28.557 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:56:28 smithi005 ceph-mon[30350]: scrub summary: idle+waiting paths [/] 2024-09-10T09:56:28.557 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:56:28 smithi005 ceph-mon[30350]: scrub summary: active paths [/] 2024-09-10T09:56:28.557 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:56:28 smithi005 ceph-mon[30350]: scrub summary: idle+waiting paths [/] 2024-09-10T09:56:28.557 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:56:28 smithi005 ceph-mon[30350]: scrub summary: active paths [/] 2024-09-10T09:56:28.557 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:56:28 smithi005 ceph-mon[30350]: scrub summary: idle+waiting paths [/] 2024-09-10T09:56:28.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:56:28 smithi159 ceph-mon[35513]: scrub queued for path: / 2024-09-10T09:56:28.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:56:28 smithi159 ceph-mon[35513]: scrub summary: idle+waiting paths [/] 2024-09-10T09:56:28.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:56:28 smithi159 ceph-mon[35513]: scrub summary: active paths [/] 2024-09-10T09:56:28.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:56:28 smithi159 ceph-mon[35513]: scrub summary: active paths [/] 2024-09-10T09:56:28.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:56:28 smithi159 ceph-mon[35513]: scrub summary: idle+waiting paths [/] 2024-09-10T09:56:28.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:56:28 smithi159 ceph-mon[35513]: scrub summary: idle+waiting paths [/] 2024-09-10T09:56:28.601 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:56:28 smithi159 ceph-mon[35513]: scrub summary: active paths [/] 2024-09-10T09:56:28.601 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:56:28 smithi159 ceph-mon[35513]: scrub summary: idle+waiting paths [/] 2024-09-10T09:56:28.601 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:56:28 smithi159 ceph-mon[35513]: scrub summary: active paths [/] 2024-09-10T09:56:28.601 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:56:28 smithi159 ceph-mon[35513]: scrub summary: idle+waiting paths [/] 2024-09-10T09:56:28.601 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:56:28 smithi159 ceph-mon[35513]: scrub summary: active paths [/] 2024-09-10T09:56:28.601 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:56:28 smithi159 ceph-mon[35513]: scrub summary: idle+waiting paths [/] 2024-09-10T09:56:28.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:56:28 smithi102 ceph-mon[35442]: scrub queued for path: / 2024-09-10T09:56:28.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:56:28 smithi102 ceph-mon[35442]: scrub summary: idle+waiting paths [/] 2024-09-10T09:56:28.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:56:28 smithi102 ceph-mon[35442]: scrub summary: active paths [/] 2024-09-10T09:56:28.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:56:28 smithi102 ceph-mon[35442]: scrub summary: active paths [/] 2024-09-10T09:56:28.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:56:28 smithi102 ceph-mon[35442]: scrub summary: idle+waiting paths [/] 2024-09-10T09:56:28.635 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:56:28 smithi102 ceph-mon[35442]: scrub summary: idle+waiting paths [/] 2024-09-10T09:56:28.635 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:56:28 smithi102 ceph-mon[35442]: scrub summary: active paths [/] 2024-09-10T09:56:28.635 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:56:28 smithi102 ceph-mon[35442]: scrub summary: idle+waiting paths [/] 2024-09-10T09:56:28.635 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:56:28 smithi102 ceph-mon[35442]: scrub summary: active paths [/] 2024-09-10T09:56:28.635 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:56:28 smithi102 ceph-mon[35442]: scrub summary: idle+waiting paths [/] 2024-09-10T09:56:28.635 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:56:28 smithi102 ceph-mon[35442]: scrub summary: active paths [/] 2024-09-10T09:56:28.635 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:56:28 smithi102 ceph-mon[35442]: scrub summary: idle+waiting paths [/] 2024-09-10T09:56:28.702 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T09:56:28.985 INFO:tasks.workunit.client.0.smithi005.stdout: 10 69576 12.96 MB/sec execute 127 sec latency 29835.561 ms 2024-09-10T09:56:29.295 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:56:29 smithi005 ceph-mon[30350]: scrub summary: active paths [/] 2024-09-10T09:56:29.295 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:56:29 smithi005 ceph-mon[30350]: scrub summary: active paths [/] 2024-09-10T09:56:29.296 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:56:29 smithi005 ceph-mon[30350]: scrub summary: idle+waiting paths [/] 2024-09-10T09:56:29.296 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:56:29 smithi005 ceph-mon[30350]: scrub summary: idle+waiting paths [/] 2024-09-10T09:56:29.296 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:56:29 smithi005 ceph-mon[30350]: pgmap v937: 129 pgs: 129 active+clean; 587 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 4.1 MiB/s wr, 134 op/s 2024-09-10T09:56:29.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:29.409+0000 7fe89d400640 1 -- 172.21.15.5:0/1779755375 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe8981019c0 msgr2=0x7fe898101e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:29.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:29.409+0000 7fe89d400640 1 --2- 172.21.15.5:0/1779755375 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe8981019c0 0x7fe898101e20 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7fe884009930 tx=0x7fe88402a420 comp rx=0 tx=0).stop 2024-09-10T09:56:29.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:29.409+0000 7fe89d400640 1 -- 172.21.15.5:0/1779755375 shutdown_connections 2024-09-10T09:56:29.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:29.409+0000 7fe89d400640 1 --2- 172.21.15.5:0/1779755375 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe898102360 0x7fe89810e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:29.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:29.409+0000 7fe89d400640 1 --2- 172.21.15.5:0/1779755375 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe8981019c0 0x7fe898101e20 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:29.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:29.409+0000 7fe89d400640 1 --2- 172.21.15.5:0/1779755375 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe8981079c0 0x7fe898107da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:29.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:29.409+0000 7fe89d400640 1 -- 172.21.15.5:0/1779755375 >> 172.21.15.5:0/1779755375 conn(0x7fe8980fd740 msgr2=0x7fe8980ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:56:29.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:29.409+0000 7fe89d400640 1 -- 172.21.15.5:0/1779755375 shutdown_connections 2024-09-10T09:56:29.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:29.409+0000 7fe89d400640 1 -- 172.21.15.5:0/1779755375 wait complete. 2024-09-10T09:56:29.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:29.410+0000 7fe89d400640 1 Processor -- start 2024-09-10T09:56:29.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:29.410+0000 7fe89d400640 1 -- start start 2024-09-10T09:56:29.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:29.410+0000 7fe89d400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe8981019c0 0x7fe898108910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:29.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:29.411+0000 7fe89d400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe898102360 0x7fe898108e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:29.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:29.411+0000 7fe89d400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe8981079c0 0x7fe898109390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:29.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:29.411+0000 7fe89d400640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe898111dc0 con 0x7fe8981079c0 2024-09-10T09:56:29.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:29.411+0000 7fe89d400640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fe898111c40 con 0x7fe898102360 2024-09-10T09:56:29.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:29.411+0000 7fe89d400640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fe898111f40 con 0x7fe8981019c0 2024-09-10T09:56:29.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:29.411+0000 7fe897e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe8981019c0 0x7fe898108910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:29.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:29.411+0000 7fe897e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe8981019c0 0x7fe898108910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:37332/0 (socket says 172.21.15.5:37332) 2024-09-10T09:56:29.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:29.411+0000 7fe897e00640 1 -- 172.21.15.5:0/2622372499 learned_addr learned my addr 172.21.15.5:0/2622372499 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:56:29.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:29.411+0000 7fe89ca00640 1 --2- 172.21.15.5:0/2622372499 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe8981079c0 0x7fe898109390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:29.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:29.411+0000 7fe897400640 1 --2- 172.21.15.5:0/2622372499 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe898102360 0x7fe898108e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:29.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:29.411+0000 7fe89ca00640 1 -- 172.21.15.5:0/2622372499 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe8981019c0 msgr2=0x7fe898108910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:29.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:29.411+0000 7fe89ca00640 1 --2- 172.21.15.5:0/2622372499 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe8981019c0 0x7fe898108910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:29.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:29.411+0000 7fe89ca00640 1 -- 172.21.15.5:0/2622372499 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe898102360 msgr2=0x7fe898108e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:29.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:29.411+0000 7fe89ca00640 1 --2- 172.21.15.5:0/2622372499 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe898102360 0x7fe898108e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:29.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:29.411+0000 7fe89ca00640 1 -- 172.21.15.5:0/2622372499 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8981aab20 con 0x7fe8981079c0 2024-09-10T09:56:29.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:29.411+0000 7fe897e00640 1 --2- 172.21.15.5:0/2622372499 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe8981019c0 0x7fe898108910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:56:29.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:29.412+0000 7fe89ca00640 1 --2- 172.21.15.5:0/2622372499 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe8981079c0 0x7fe898109390 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fe89000ea80 tx=0x7fe89000ef40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:56:29.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:29.412+0000 7fe894c00640 1 -- 172.21.15.5:0/2622372499 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe89000cea0 con 0x7fe8981079c0 2024-09-10T09:56:29.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:29.412+0000 7fe894c00640 1 -- 172.21.15.5:0/2622372499 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe890005020 con 0x7fe8981079c0 2024-09-10T09:56:29.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:29.412+0000 7fe89d400640 1 -- 172.21.15.5:0/2622372499 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8981aae10 con 0x7fe8981079c0 2024-09-10T09:56:29.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:29.412+0000 7fe894c00640 1 -- 172.21.15.5:0/2622372499 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe890010690 con 0x7fe8981079c0 2024-09-10T09:56:29.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:29.412+0000 7fe89d400640 1 -- 172.21.15.5:0/2622372499 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8981ab2d0 con 0x7fe8981079c0 2024-09-10T09:56:29.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:29.413+0000 7fe89d400640 1 -- 172.21.15.5:0/2622372499 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe898103100 con 0x7fe8981079c0 2024-09-10T09:56:29.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:29.415+0000 7fe894c00640 1 -- 172.21.15.5:0/2622372499 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7fe890005540 con 0x7fe8981079c0 2024-09-10T09:56:29.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:29.417+0000 7fe894c00640 1 --2- 172.21.15.5:0/2622372499 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fe87c0a2b80 0x7fe87c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:29.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:29.417+0000 7fe894c00640 1 -- 172.21.15.5:0/2622372499 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fe8900cfb20 con 0x7fe8981079c0 2024-09-10T09:56:29.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:29.417+0000 7fe897e00640 1 --2- 172.21.15.5:0/2622372499 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fe87c0a2b80 0x7fe87c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:29.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:29.418+0000 7fe897e00640 1 --2- 172.21.15.5:0/2622372499 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fe87c0a2b80 0x7fe87c0a5040 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fe888005fd0 tx=0x7fe88800b4a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:56:29.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:29.423+0000 7fe894c00640 1 -- 172.21.15.5:0/2622372499 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe89009c630 con 0x7fe8981079c0 2024-09-10T09:56:29.493 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:56:29.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:56:29 smithi159 ceph-mon[35513]: scrub summary: active paths [/] 2024-09-10T09:56:29.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:56:29 smithi159 ceph-mon[35513]: scrub summary: active paths [/] 2024-09-10T09:56:29.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:56:29 smithi159 ceph-mon[35513]: scrub summary: idle+waiting paths [/] 2024-09-10T09:56:29.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:56:29 smithi159 ceph-mon[35513]: scrub summary: idle+waiting paths [/] 2024-09-10T09:56:29.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:56:29 smithi159 ceph-mon[35513]: pgmap v937: 129 pgs: 129 active+clean; 587 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 4.1 MiB/s wr, 134 op/s 2024-09-10T09:56:29.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:56:29 smithi102 ceph-mon[35442]: scrub summary: active paths [/] 2024-09-10T09:56:29.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:56:29 smithi102 ceph-mon[35442]: scrub summary: active paths [/] 2024-09-10T09:56:29.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:56:29 smithi102 ceph-mon[35442]: scrub summary: idle+waiting paths [/] 2024-09-10T09:56:29.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:56:29 smithi102 ceph-mon[35442]: scrub summary: idle+waiting paths [/] 2024-09-10T09:56:29.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:56:29 smithi102 ceph-mon[35442]: pgmap v937: 129 pgs: 129 active+clean; 587 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 4.1 MiB/s wr, 134 op/s 2024-09-10T09:56:29.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:29.666+0000 7fe89d400640 1 -- 172.21.15.5:0/2622372499 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe898101e20 con 0x7fe8981079c0 2024-09-10T09:56:29.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:29.666+0000 7fe894c00640 1 -- 172.21.15.5:0/2622372499 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 42 v42) ==== 94+0+11969 (secure 0 0 0) 0x7fe89009bf30 con 0x7fe8981079c0 2024-09-10T09:56:29.668 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-10T09:56:29.668 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":42,"btime":"2024-09-10T09:55:06:670916+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14454,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/2970594311","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2970594311},{"type":"v1","addr":"172.21.15.5:6835","nonce":2970594311}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14514,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6839/3616502174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":3616502174},{"type":"v1","addr":"172.21.15.5:6839","nonce":3616502174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":42,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T09:35:29.626287+0000","modified":"2024-09-10T09:55:06.419829+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24542,"mds_1":24535,"mds_2":24511,"mds_3":24506,"mds_4":24443},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14520":{"gid":14520,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/2073599842","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":2073599842},{"type":"v1","addr":"172.21.15.5:6841","nonce":2073599842}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"k","rank":4,"incarnation":26,"state":"up:active","state_seq":24,"addr":"172.21.15.102:6833/2457007427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":2457007427},{"type":"v1","addr":"172.21.15.102:6833","nonce":2457007427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/356445159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":356445159},{"type":"v1","addr":"172.21.15.159:6833","nonce":356445159}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6837/2894314507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":2894314507},{"type":"v1","addr":"172.21.15.5:6837","nonce":2894314507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.102:6835/1647549111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":1647549111},{"type":"v1","addr":"172.21.15.102:6835","nonce":1647549111}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6835/952657242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":952657242},{"type":"v1","addr":"172.21.15.159:6835","nonce":952657242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":19,"addr":"172.21.15.102:6837/2689776764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":2689776764},{"type":"v1","addr":"172.21.15.102:6837","nonce":2689776764}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.159:6837/620627639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":620627639},{"type":"v1","addr":"172.21.15.159:6837","nonce":620627639}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24535":{"gid":24535,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":18,"addr":"172.21.15.159:6839/2073739599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":2073739599},{"type":"v1","addr":"172.21.15.159:6839","nonce":2073739599}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24542":{"gid":24542,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.102:6839/335409749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6838","nonce":335409749},{"type":"v1","addr":"172.21.15.102:6839","nonce":335409749}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24542,"qdb_cluster":[24542,24535,24511,24506,24443]},"id":1}]} 2024-09-10T09:56:29.668 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 42 2024-09-10T09:56:29.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:29.669+0000 7fe89d400640 1 -- 172.21.15.5:0/2622372499 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fe87c0a2b80 msgr2=0x7fe87c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:29.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:29.669+0000 7fe89d400640 1 --2- 172.21.15.5:0/2622372499 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fe87c0a2b80 0x7fe87c0a5040 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fe888005fd0 tx=0x7fe88800b4a0 comp rx=0 tx=0).stop 2024-09-10T09:56:29.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:29.669+0000 7fe89d400640 1 -- 172.21.15.5:0/2622372499 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe8981079c0 msgr2=0x7fe898109390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:29.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:29.669+0000 7fe89d400640 1 --2- 172.21.15.5:0/2622372499 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe8981079c0 0x7fe898109390 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fe89000ea80 tx=0x7fe89000ef40 comp rx=0 tx=0).stop 2024-09-10T09:56:29.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:29.669+0000 7fe89d400640 1 -- 172.21.15.5:0/2622372499 shutdown_connections 2024-09-10T09:56:29.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:29.669+0000 7fe89d400640 1 --2- 172.21.15.5:0/2622372499 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fe87c0a2b80 0x7fe87c0a5040 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:29.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:29.669+0000 7fe89d400640 1 --2- 172.21.15.5:0/2622372499 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe8981079c0 0x7fe898109390 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:29.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:29.669+0000 7fe89d400640 1 --2- 172.21.15.5:0/2622372499 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe898102360 0x7fe898108e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:29.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:29.669+0000 7fe89d400640 1 --2- 172.21.15.5:0/2622372499 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe8981019c0 0x7fe898108910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:29.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:29.669+0000 7fe89d400640 1 -- 172.21.15.5:0/2622372499 >> 172.21.15.5:0/2622372499 conn(0x7fe8980fd740 msgr2=0x7fe89810d0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:56:29.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:29.669+0000 7fe89d400640 1 -- 172.21.15.5:0/2622372499 shutdown_connections 2024-09-10T09:56:29.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:29.669+0000 7fe89d400640 1 -- 172.21.15.5:0/2622372499 wait complete. 2024-09-10T09:56:29.985 INFO:tasks.workunit.client.0.smithi005.stdout: 10 69576 12.86 MB/sec execute 128 sec latency 30835.683 ms 2024-09-10T09:56:30.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:30.232+0000 7fc022000640 1 -- 172.21.15.5:0/1108609589 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc01c100a80 msgr2=0x7fc01c100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:30.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:30.232+0000 7fc022000640 1 --2- 172.21.15.5:0/1108609589 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc01c100a80 0x7fc01c100ee0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7fc008009930 tx=0x7fc00802a420 comp rx=0 tx=0).stop 2024-09-10T09:56:30.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:30.233+0000 7fc022000640 1 -- 172.21.15.5:0/1108609589 shutdown_connections 2024-09-10T09:56:30.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:30.233+0000 7fc022000640 1 --2- 172.21.15.5:0/1108609589 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc01c101420 0x7fc01c10d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:30.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:30.233+0000 7fc022000640 1 --2- 172.21.15.5:0/1108609589 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc01c100a80 0x7fc01c100ee0 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:30.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:30.233+0000 7fc022000640 1 --2- 172.21.15.5:0/1108609589 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc01c106a80 0x7fc01c106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:30.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:30.233+0000 7fc022000640 1 -- 172.21.15.5:0/1108609589 >> 172.21.15.5:0/1108609589 conn(0x7fc01c0fc7a0 msgr2=0x7fc01c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:56:30.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:30.233+0000 7fc022000640 1 -- 172.21.15.5:0/1108609589 shutdown_connections 2024-09-10T09:56:30.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:30.233+0000 7fc022000640 1 -- 172.21.15.5:0/1108609589 wait complete. 2024-09-10T09:56:30.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:30.234+0000 7fc022000640 1 Processor -- start 2024-09-10T09:56:30.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:30.234+0000 7fc022000640 1 -- start start 2024-09-10T09:56:30.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:30.235+0000 7fc022000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc01c100a80 0x7fc01c198470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:30.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:30.235+0000 7fc022000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc01c101420 0x7fc01c1989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:30.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:30.235+0000 7fc022000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc01c106a80 0x7fc01c19cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:30.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:30.235+0000 7fc022000640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc01c110e80 con 0x7fc01c106a80 2024-09-10T09:56:30.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:30.235+0000 7fc022000640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fc01c110d00 con 0x7fc01c100a80 2024-09-10T09:56:30.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:30.235+0000 7fc022000640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fc01c111000 con 0x7fc01c101420 2024-09-10T09:56:30.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:30.235+0000 7fc01b400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc01c100a80 0x7fc01c198470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:30.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:30.235+0000 7fc01be00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc01c106a80 0x7fc01c19cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:30.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:30.235+0000 7fc01b400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc01c100a80 0x7fc01c198470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:59148/0 (socket says 172.21.15.5:59148) 2024-09-10T09:56:30.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:30.235+0000 7fc01be00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc01c106a80 0x7fc01c19cd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42088/0 (socket says 172.21.15.5:42088) 2024-09-10T09:56:30.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:30.235+0000 7fc01b400640 1 -- 172.21.15.5:0/2375919772 learned_addr learned my addr 172.21.15.5:0/2375919772 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:56:30.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:30.235+0000 7fc01aa00640 1 --2- 172.21.15.5:0/2375919772 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc01c101420 0x7fc01c1989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:30.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:30.236+0000 7fc01b400640 1 -- 172.21.15.5:0/2375919772 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc01c101420 msgr2=0x7fc01c1989b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:30.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:30.236+0000 7fc01b400640 1 --2- 172.21.15.5:0/2375919772 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc01c101420 0x7fc01c1989b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:30.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:30.236+0000 7fc01b400640 1 -- 172.21.15.5:0/2375919772 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc01c106a80 msgr2=0x7fc01c19cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:30.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:30.236+0000 7fc01b400640 1 --2- 172.21.15.5:0/2375919772 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc01c106a80 0x7fc01c19cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:30.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:30.236+0000 7fc01b400640 1 -- 172.21.15.5:0/2375919772 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc01c19d420 con 0x7fc01c100a80 2024-09-10T09:56:30.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:30.236+0000 7fc01be00640 1 --2- 172.21.15.5:0/2375919772 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc01c106a80 0x7fc01c19cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:56:30.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:30.236+0000 7fc01b400640 1 --2- 172.21.15.5:0/2375919772 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc01c100a80 0x7fc01c198470 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fc00c00e9f0 tx=0x7fc00c00edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:56:30.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:30.236+0000 7fc013e00640 1 -- 172.21.15.5:0/2375919772 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc00c00cd80 con 0x7fc01c100a80 2024-09-10T09:56:30.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:30.236+0000 7fc013e00640 1 -- 172.21.15.5:0/2375919772 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc00c005500 con 0x7fc01c100a80 2024-09-10T09:56:30.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:30.236+0000 7fc022000640 1 -- 172.21.15.5:0/2375919772 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc01c19d710 con 0x7fc01c100a80 2024-09-10T09:56:30.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:30.236+0000 7fc013e00640 1 -- 172.21.15.5:0/2375919772 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc00c010690 con 0x7fc01c100a80 2024-09-10T09:56:30.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:30.236+0000 7fc022000640 1 -- 172.21.15.5:0/2375919772 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc01c1a4ff0 con 0x7fc01c100a80 2024-09-10T09:56:30.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:30.239+0000 7fc013e00640 1 -- 172.21.15.5:0/2375919772 <== mon.2 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7fc00c0046c0 con 0x7fc01c100a80 2024-09-10T09:56:30.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:30.239+0000 7fc0249f9740 1 Processor -- start 2024-09-10T09:56:30.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:30.240+0000 7fc0249f9740 1 -- start start 2024-09-10T09:56:30.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:30.240+0000 7fc0249f9740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56315a210a90 0x56315a210e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:30.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:30.240+0000 7fc0249f9740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56315a2113b0 0x56315a21fc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:30.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:30.240+0000 7fc0249f9740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56315a220180 0x56315a222570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:30.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:30.240+0000 7fc0249f9740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x56315a1773e0 con 0x56315a2113b0 2024-09-10T09:56:30.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:30.240+0000 7fc0249f9740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x56315a11ed80 con 0x56315a220180 2024-09-10T09:56:30.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:30.240+0000 7fc0249f9740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x56315a1733a0 con 0x56315a210a90 2024-09-10T09:56:30.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:30.240+0000 7fc01aa00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56315a220180 0x56315a222570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:30.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:30.240+0000 7fc01b400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56315a2113b0 0x56315a21fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:30.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:30.240+0000 7fc01aa00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56315a220180 0x56315a222570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:59162/0 (socket says 172.21.15.5:59162) 2024-09-10T09:56:30.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:30.240+0000 7fc01aa00640 1 -- 172.21.15.5:0/3453453765 learned_addr learned my addr 172.21.15.5:0/3453453765 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:56:30.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:30.240+0000 7fc01be00640 1 --2- 172.21.15.5:0/3453453765 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56315a210a90 0x56315a210e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:30.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:30.241+0000 7fc01aa00640 1 -- 172.21.15.5:0/3453453765 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56315a210a90 msgr2=0x56315a210e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:56:30.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:30.241+0000 7fc01aa00640 1 --2- 172.21.15.5:0/3453453765 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56315a210a90 0x56315a210e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:30.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:30.241+0000 7fc01aa00640 1 -- 172.21.15.5:0/3453453765 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56315a2113b0 msgr2=0x56315a21fc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:56:30.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:30.241+0000 7fc01aa00640 1 --2- 172.21.15.5:0/3453453765 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56315a2113b0 0x56315a21fc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:30.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:30.241+0000 7fc01aa00640 1 -- 172.21.15.5:0/3453453765 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56315a13dae0 con 0x56315a220180 2024-09-10T09:56:30.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:30.241+0000 7fc01b400640 1 --2- 172.21.15.5:0/3453453765 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56315a2113b0 0x56315a21fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:56:30.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:30.241+0000 7fc01aa00640 1 --2- 172.21.15.5:0/3453453765 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56315a220180 0x56315a222570 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fc00802a940 tx=0x7fc008005bd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8df7d8ee5da91d9c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:56:30.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:30.241+0000 7fc013e00640 1 --2- 172.21.15.5:0/2375919772 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fc0000a2c00 0x7fc0000a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:30.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:30.242+0000 7fc007e00640 1 -- 172.21.15.5:0/3453453765 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc008040070 con 0x56315a220180 2024-09-10T09:56:30.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:30.242+0000 7fc013e00640 1 -- 172.21.15.5:0/2375919772 <== mon.2 v2:172.21.15.102:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fc00c0cf780 con 0x7fc01c100a80 2024-09-10T09:56:30.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:30.242+0000 7fc01be00640 1 --2- 172.21.15.5:0/2375919772 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fc0000a2c00 0x7fc0000a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:30.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:30.242+0000 7fc007e00640 1 -- 172.21.15.5:0/3453453765 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc008031870 con 0x56315a220180 2024-09-10T09:56:30.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:30.242+0000 7fc007e00640 1 -- 172.21.15.5:0/3453453765 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc008035040 con 0x56315a220180 2024-09-10T09:56:30.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:30.243+0000 7fc0249f9740 1 -- 172.21.15.5:0/3453453765 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56315a220180 msgr2=0x56315a222570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:30.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:30.243+0000 7fc0249f9740 1 --2- 172.21.15.5:0/3453453765 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56315a220180 0x56315a222570 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fc00802a940 tx=0x7fc008005bd0 comp rx=0 tx=0).stop 2024-09-10T09:56:30.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:30.243+0000 7fc01be00640 1 --2- 172.21.15.5:0/2375919772 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fc0000a2c00 0x7fc0000a50c0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fc014009fd0 tx=0x7fc014009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:56:30.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:30.243+0000 7fc0249f9740 1 -- 172.21.15.5:0/3453453765 shutdown_connections 2024-09-10T09:56:30.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:30.243+0000 7fc0249f9740 1 --2- 172.21.15.5:0/3453453765 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56315a220180 0x56315a222570 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:30.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:30.243+0000 7fc0249f9740 1 --2- 172.21.15.5:0/3453453765 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56315a2113b0 0x56315a21fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:30.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:30.243+0000 7fc0249f9740 1 --2- 172.21.15.5:0/3453453765 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56315a210a90 0x56315a210e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:30.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:30.243+0000 7fc0249f9740 1 -- 172.21.15.5:0/3453453765 >> 172.21.15.5:0/3453453765 conn(0x56315a20d120 msgr2=0x56315a20d500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:56:30.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:30.243+0000 7fc0249f9740 1 -- 172.21.15.5:0/3453453765 shutdown_connections 2024-09-10T09:56:30.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:30.243+0000 7fc0249f9740 1 -- 172.21.15.5:0/3453453765 wait complete. 2024-09-10T09:56:30.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:30.243+0000 7fc0249f9740 1 -- start start 2024-09-10T09:56:30.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:30.244+0000 7fc0249f9740 1 Processor -- start 2024-09-10T09:56:30.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:30.244+0000 7fc0249f9740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56315a210a90 0x56315a21cf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:30.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:30.244+0000 7fc0249f9740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56315a2113b0 0x56315a21d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:30.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:30.244+0000 7fc0249f9740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56315a220180 0x56315a21da10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:30.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:30.244+0000 7fc0249f9740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x56315a222df0 con 0x56315a2113b0 2024-09-10T09:56:30.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:30.244+0000 7fc0249f9740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x56315a1320f0 con 0x56315a220180 2024-09-10T09:56:30.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:30.244+0000 7fc0249f9740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x56315a222f70 con 0x56315a210a90 2024-09-10T09:56:30.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:30.244+0000 7fc0249f9740 10 client.0 _ll_register_callbacks cb 0x56315a1239e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:56:30.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:30.245+0000 7fc01aa00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56315a2113b0 0x56315a21d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:30.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:30.245+0000 7fc01b400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56315a210a90 0x56315a21cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:30.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:30.245+0000 7fc01aa00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56315a2113b0 0x56315a21d4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42102/0 (socket says 172.21.15.5:42102) 2024-09-10T09:56:30.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:30.245+0000 7fc01aa00640 1 -- 172.21.15.5:0/3648124624 learned_addr learned my addr 172.21.15.5:0/3648124624 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:56:30.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:30.245+0000 7fc01be00640 1 --2- 172.21.15.5:0/3648124624 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56315a220180 0x56315a21da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:30.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:30.245+0000 7fc01b400640 1 -- 172.21.15.5:0/3648124624 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56315a220180 msgr2=0x56315a21da10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:56:30.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:30.245+0000 7fc01b400640 1 --2- 172.21.15.5:0/3648124624 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56315a220180 0x56315a21da10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:30.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:30.245+0000 7fc01b400640 1 -- 172.21.15.5:0/3648124624 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56315a2113b0 msgr2=0x56315a21d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:56:30.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:30.245+0000 7fc01b400640 1 --2- 172.21.15.5:0/3648124624 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56315a2113b0 0x56315a21d4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:30.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:30.245+0000 7fc01b400640 1 -- 172.21.15.5:0/3648124624 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563159ee83c0 con 0x56315a210a90 2024-09-10T09:56:30.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:30.245+0000 7fc01b400640 1 --2- 172.21.15.5:0/3648124624 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56315a210a90 0x56315a21cf90 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x56315a213780 tx=0x7fc00c004ba0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b07385b11ed0fa59 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:56:30.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:30.245+0000 7fc004c00640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-09-10T09:56:30.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:30.246+0000 7fc004c00640 1 -- 172.21.15.5:0/3648124624 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc00c09b4f0 con 0x56315a210a90 2024-09-10T09:56:30.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:30.246+0000 7fc004c00640 1 -- 172.21.15.5:0/3648124624 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fc00c098200 con 0x56315a210a90 2024-09-10T09:56:30.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:30.246+0000 7fc0249f9740 1 -- 172.21.15.5:0/3648124624 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc008031870 con 0x56315a210a90 2024-09-10T09:56:30.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:30.246+0000 7fc004c00640 1 -- 172.21.15.5:0/3648124624 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7fc00c09c290 con 0x56315a210a90 2024-09-10T09:56:30.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:30.246+0000 7fc0249f9740 10 client.28981 fetch_fsmap learned FSMap version 42 2024-09-10T09:56:30.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:30.246+0000 7fc0249f9740 1 -- 172.21.15.5:0/3648124624 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x56315a22ea90 con 0x56315a210a90 2024-09-10T09:56:30.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:30.246+0000 7fc004c00640 1 -- 172.21.15.5:0/3648124624 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc008031870 con 0x56315a210a90 2024-09-10T09:56:30.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:30.246+0000 7fc004c00640 1 -- 172.21.15.5:0/3648124624 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 42) ==== 8370+0+0 (secure 0 0 0) 0x7fc00c0ce070 con 0x56315a210a90 2024-09-10T09:56:30.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:30.246+0000 7fc0249f9740 10 client.28981 fetch_fsmap finished waiting for FSMap version 42 2024-09-10T09:56:30.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:30.246+0000 7fc0249f9740 10 client.28981 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T09:56:30.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:30.246+0000 7fc0249f9740 20 client.28981 populate_metadata read hostname 'smithi005' 2024-09-10T09:56:30.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:30.246+0000 7fc0249f9740 1 --2- 172.21.15.5:0/3648124624 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x56315a22fef0 0x56315a232360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:30.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:30.246+0000 7fc0249f9740 4 client.28981 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:56:30.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:30.246+0000 7fc0249f9740 1 -- 172.21.15.5:0/3648124624 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56315a184950 con 0x56315a22fef0 2024-09-10T09:56:30.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:30.247+0000 7fc01aa00640 1 --2- 172.21.15.5:0/3648124624 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x56315a22fef0 0x56315a232360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:30.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:30.247+0000 7fc004c00640 10 client.28981 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T09:56:30.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:30.247+0000 7fc01aa00640 1 --2- 172.21.15.5:0/3648124624 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x56315a22fef0 0x56315a232360 crc :-1 s=READY pgs=259 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a52162c6513495ce server_cookie=29ec929d2d6966af in_seq=0 out_seq=0 2024-09-10T09:56:30.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:30.248+0000 7fc004c00640 1 -- 172.21.15.5:0/3648124624 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56315a184950 con 0x56315a22fef0 2024-09-10T09:56:30.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:30.248+0000 7fc004c00640 10 client.28981 handle_command_reply: tid=0 2024-09-10T09:56:30.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:30.248+0000 7fc0249f9740 1 client.28981 shutdown 2024-09-10T09:56:30.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:30.249+0000 7fc0249f9740 1 -- 172.21.15.5:0/3648124624 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56315a210a90 msgr2=0x56315a21cf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:30.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:30.249+0000 7fc0249f9740 1 --2- 172.21.15.5:0/3648124624 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56315a210a90 0x56315a21cf90 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x56315a213780 tx=0x7fc00c004ba0 comp rx=0 tx=0).stop 2024-09-10T09:56:30.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:30.249+0000 7fc0249f9740 1 -- 172.21.15.5:0/3648124624 shutdown_connections 2024-09-10T09:56:30.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:30.249+0000 7fc0249f9740 1 --2- 172.21.15.5:0/3648124624 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x56315a22fef0 0x56315a232360 crc :-1 s=READY pgs=259 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:30.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:30.249+0000 7fc0249f9740 1 --2- 172.21.15.5:0/3648124624 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56315a220180 0x56315a21da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:30.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:30.249+0000 7fc0249f9740 1 --2- 172.21.15.5:0/3648124624 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56315a2113b0 0x56315a21d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:30.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:30.249+0000 7fc0249f9740 1 --2- 172.21.15.5:0/3648124624 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56315a210a90 0x56315a21cf90 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:30.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:30.249+0000 7fc0249f9740 1 -- 172.21.15.5:0/3648124624 >> 172.21.15.5:0/3648124624 conn(0x56315a20d120 msgr2=0x56315a212e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:56:30.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:30.249+0000 7fc004c00640 0 client.28981 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T09:56:30.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:30.249+0000 7fc0249f9740 1 -- 172.21.15.5:0/3648124624 shutdown_connections 2024-09-10T09:56:30.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:30.249+0000 7fc0249f9740 1 -- 172.21.15.5:0/3648124624 wait complete. 2024-09-10T09:56:30.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:30.249+0000 7fc0249f9740 20 client.28981 trim_cache size 0 max 16384 2024-09-10T09:56:30.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:30.268+0000 7fc0249f9740 1 Processor -- start 2024-09-10T09:56:30.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:30.268+0000 7fc0249f9740 1 -- start start 2024-09-10T09:56:30.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:30.268+0000 7fc0249f9740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56315a210a90 0x56315a237c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:30.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:30.268+0000 7fc0249f9740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56315a2113b0 0x56315a238170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:30.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:30.268+0000 7fc0249f9740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56315a220180 0x56315a20b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:30.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:30.268+0000 7fc0249f9740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc01c1acc70 con 0x56315a210a90 2024-09-10T09:56:30.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:30.268+0000 7fc0249f9740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fc01c1ae7b0 con 0x56315a220180 2024-09-10T09:56:30.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:30.268+0000 7fc0249f9740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fc01c1ad340 con 0x56315a2113b0 2024-09-10T09:56:30.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:30.269+0000 7fc01b400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56315a210a90 0x56315a237c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:30.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:30.269+0000 7fc01be00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56315a220180 0x56315a20b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:30.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:30.269+0000 7fc01aa00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56315a2113b0 0x56315a238170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:30.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:30.269+0000 7fc01be00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56315a220180 0x56315a20b9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:59168/0 (socket says 172.21.15.5:59168) 2024-09-10T09:56:30.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:30.269+0000 7fc01be00640 1 -- 172.21.15.5:0/1144006684 learned_addr learned my addr 172.21.15.5:0/1144006684 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:56:30.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:30.269+0000 7fc01aa00640 1 -- 172.21.15.5:0/1144006684 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56315a220180 msgr2=0x56315a20b9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:56:30.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:30.269+0000 7fc01aa00640 1 --2- 172.21.15.5:0/1144006684 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56315a220180 0x56315a20b9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:30.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:30.269+0000 7fc01aa00640 1 -- 172.21.15.5:0/1144006684 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56315a210a90 msgr2=0x56315a237c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:56:30.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:30.269+0000 7fc01aa00640 1 --2- 172.21.15.5:0/1144006684 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56315a210a90 0x56315a237c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:30.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:30.269+0000 7fc01aa00640 1 -- 172.21.15.5:0/1144006684 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56315a20cf50 con 0x56315a2113b0 2024-09-10T09:56:30.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:30.269+0000 7fc01aa00640 1 --2- 172.21.15.5:0/1144006684 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56315a2113b0 0x56315a238170 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fc008009900 tx=0x7fc00802a420 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bd19573620f6e86a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:56:30.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:30.270+0000 7fc007400640 1 -- 172.21.15.5:0/1144006684 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc008040040 con 0x56315a2113b0 2024-09-10T09:56:30.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:30.270+0000 7fc007400640 1 -- 172.21.15.5:0/1144006684 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc0080318b0 con 0x56315a2113b0 2024-09-10T09:56:30.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:30.270+0000 7fc007400640 1 -- 172.21.15.5:0/1144006684 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc008035040 con 0x56315a2113b0 2024-09-10T09:56:30.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:30.270+0000 7fc0249f9740 1 -- 172.21.15.5:0/1144006684 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56315a2113b0 msgr2=0x56315a238170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:30.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:30.270+0000 7fc0249f9740 1 --2- 172.21.15.5:0/1144006684 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56315a2113b0 0x56315a238170 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fc008009900 tx=0x7fc00802a420 comp rx=0 tx=0).stop 2024-09-10T09:56:30.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:30.270+0000 7fc0249f9740 1 -- 172.21.15.5:0/1144006684 shutdown_connections 2024-09-10T09:56:30.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:30.271+0000 7fc0249f9740 1 --2- 172.21.15.5:0/1144006684 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56315a220180 0x56315a20b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:30.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:30.271+0000 7fc0249f9740 1 --2- 172.21.15.5:0/1144006684 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56315a2113b0 0x56315a238170 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:30.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:30.271+0000 7fc0249f9740 1 --2- 172.21.15.5:0/1144006684 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56315a210a90 0x56315a237c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:30.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:30.271+0000 7fc0249f9740 1 -- 172.21.15.5:0/1144006684 >> 172.21.15.5:0/1144006684 conn(0x56315a20d120 msgr2=0x56315a21c080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:56:30.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:30.271+0000 7fc0249f9740 1 -- 172.21.15.5:0/1144006684 shutdown_connections 2024-09-10T09:56:30.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:30.271+0000 7fc0249f9740 1 -- 172.21.15.5:0/1144006684 wait complete. 2024-09-10T09:56:30.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:30.271+0000 7fc0249f9740 1 -- start start 2024-09-10T09:56:30.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:30.271+0000 7fc0249f9740 1 Processor -- start 2024-09-10T09:56:30.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:30.272+0000 7fc0249f9740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56315a210a90 0x56315a2288a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:30.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:30.272+0000 7fc0249f9740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56315a2113b0 0x56315a223c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:30.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:30.272+0000 7fc0249f9740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56315a220180 0x56315a2241b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:30.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:30.272+0000 7fc0249f9740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc01c1ae0e0 con 0x56315a2113b0 2024-09-10T09:56:30.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:30.272+0000 7fc0249f9740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fc01c1ada10 con 0x56315a210a90 2024-09-10T09:56:30.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:30.272+0000 7fc0249f9740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fc01c1aac50 con 0x56315a220180 2024-09-10T09:56:30.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:30.272+0000 7fc0249f9740 10 client.0 _ll_register_callbacks cb 0x7fc00c00cf20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:56:30.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:30.272+0000 7fc01b400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56315a210a90 0x56315a2288a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:30.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:30.272+0000 7fc01b400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56315a210a90 0x56315a2288a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:59172/0 (socket says 172.21.15.5:59172) 2024-09-10T09:56:30.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:30.272+0000 7fc01aa00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56315a2113b0 0x56315a223c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:30.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:30.272+0000 7fc01b400640 1 -- 172.21.15.5:0/2698621732 learned_addr learned my addr 172.21.15.5:0/2698621732 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:56:30.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:30.272+0000 7fc01be00640 1 --2- 172.21.15.5:0/2698621732 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56315a220180 0x56315a2241b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:30.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:30.272+0000 7fc01be00640 1 -- 172.21.15.5:0/2698621732 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56315a210a90 msgr2=0x56315a2288a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:56:30.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:30.272+0000 7fc01be00640 1 --2- 172.21.15.5:0/2698621732 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56315a210a90 0x56315a2288a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:30.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:30.272+0000 7fc01be00640 1 -- 172.21.15.5:0/2698621732 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56315a2113b0 msgr2=0x56315a223c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:56:30.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:30.272+0000 7fc01be00640 1 --2- 172.21.15.5:0/2698621732 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56315a2113b0 0x56315a223c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:30.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:30.272+0000 7fc01be00640 1 -- 172.21.15.5:0/2698621732 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56315a2249e0 con 0x56315a220180 2024-09-10T09:56:30.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:30.272+0000 7fc01b400640 1 --2- 172.21.15.5:0/2698621732 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56315a210a90 0x56315a2288a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:56:30.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:30.272+0000 7fc01aa00640 1 --2- 172.21.15.5:0/2698621732 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56315a2113b0 0x56315a223c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:56:30.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:30.273+0000 7fc01be00640 1 --2- 172.21.15.5:0/2698621732 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56315a220180 0x56315a2241b0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fc014005ee0 tx=0x7fc014004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1783cc45bda13802 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:56:30.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:30.273+0000 7fc004c00640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-09-10T09:56:30.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:30.273+0000 7fc004c00640 1 -- 172.21.15.5:0/2698621732 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc014004b20 con 0x56315a220180 2024-09-10T09:56:30.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:30.273+0000 7fc0249f9740 1 -- 172.21.15.5:0/2698621732 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc0080318b0 con 0x56315a220180 2024-09-10T09:56:30.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:30.273+0000 7fc004c00640 1 -- 172.21.15.5:0/2698621732 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fc014004cc0 con 0x56315a220180 2024-09-10T09:56:30.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:30.274+0000 7fc004c00640 1 -- 172.21.15.5:0/2698621732 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7fc014034050 con 0x56315a220180 2024-09-10T09:56:30.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:30.274+0000 7fc0249f9740 10 client.28987 fetch_fsmap learned FSMap version 42 2024-09-10T09:56:30.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:30.274+0000 7fc0249f9740 1 -- 172.21.15.5:0/2698621732 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x56315a29abf0 con 0x56315a220180 2024-09-10T09:56:30.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:30.274+0000 7fc004c00640 1 -- 172.21.15.5:0/2698621732 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc0080318b0 con 0x56315a220180 2024-09-10T09:56:30.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:30.274+0000 7fc004c00640 1 -- 172.21.15.5:0/2698621732 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 42) ==== 8370+0+0 (secure 0 0 0) 0x7fc014005720 con 0x56315a220180 2024-09-10T09:56:30.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:30.274+0000 7fc0249f9740 10 client.28987 fetch_fsmap finished waiting for FSMap version 42 2024-09-10T09:56:30.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:30.274+0000 7fc0249f9740 10 client.28987 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T09:56:30.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:30.274+0000 7fc0249f9740 20 client.28987 populate_metadata read hostname 'smithi005' 2024-09-10T09:56:30.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:30.274+0000 7fc0249f9740 1 --2- 172.21.15.5:0/2698621732 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x56315a29b370 0x56315a29d790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:30.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:30.274+0000 7fc0249f9740 4 client.28987 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T09:56:30.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:30.274+0000 7fc0249f9740 1 -- 172.21.15.5:0/2698621732 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56315a17ee60 con 0x56315a29b370 2024-09-10T09:56:30.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:30.275+0000 7fc01b400640 1 --2- 172.21.15.5:0/2698621732 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x56315a29b370 0x56315a29d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:30.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:30.275+0000 7fc01b400640 1 --2- 172.21.15.5:0/2698621732 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x56315a29b370 0x56315a29d790 crc :-1 s=READY pgs=260 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f5f4d12fdfd3d64d server_cookie=2aaa06b9d697ced4 in_seq=0 out_seq=0 2024-09-10T09:56:30.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:30.275+0000 7fc004c00640 10 client.28987 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T09:56:30.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:30.275+0000 7fc004c00640 1 -- 172.21.15.5:0/2698621732 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x56315a17ee60 con 0x56315a29b370 2024-09-10T09:56:30.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:30.275+0000 7fc004c00640 10 client.28987 handle_command_reply: tid=0 2024-09-10T09:56:30.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:30.275+0000 7fc0249f9740 1 client.28987 shutdown 2024-09-10T09:56:30.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:30.276+0000 7fc0249f9740 1 -- 172.21.15.5:0/2698621732 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56315a220180 msgr2=0x56315a2241b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:30.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:30.276+0000 7fc0249f9740 1 --2- 172.21.15.5:0/2698621732 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56315a220180 0x56315a2241b0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fc014005ee0 tx=0x7fc014004a40 comp rx=0 tx=0).stop 2024-09-10T09:56:30.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:30.276+0000 7fc0249f9740 1 -- 172.21.15.5:0/2698621732 shutdown_connections 2024-09-10T09:56:30.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:30.276+0000 7fc0249f9740 1 --2- 172.21.15.5:0/2698621732 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x56315a29b370 0x56315a29d790 crc :-1 s=READY pgs=260 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:30.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:30.276+0000 7fc0249f9740 1 --2- 172.21.15.5:0/2698621732 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56315a220180 0x56315a2241b0 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:30.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:30.276+0000 7fc0249f9740 1 --2- 172.21.15.5:0/2698621732 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56315a2113b0 0x56315a223c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:30.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:30.276+0000 7fc004c00640 0 client.28987 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T09:56:30.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:30.276+0000 7fc0249f9740 1 --2- 172.21.15.5:0/2698621732 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56315a210a90 0x56315a2288a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:30.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:30.276+0000 7fc0249f9740 1 -- 172.21.15.5:0/2698621732 >> 172.21.15.5:0/2698621732 conn(0x56315a20d120 msgr2=0x56315a20b9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:56:30.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:30.276+0000 7fc0249f9740 1 -- 172.21.15.5:0/2698621732 shutdown_connections 2024-09-10T09:56:30.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:30.276+0000 7fc0249f9740 1 -- 172.21.15.5:0/2698621732 wait complete. 2024-09-10T09:56:30.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:30.276+0000 7fc0249f9740 20 client.28987 trim_cache size 0 max 16384 2024-09-10T09:56:30.281 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-10T09:56:30.281 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-10T09:56:30.281 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2024-09-10T09:56:30.281 INFO:teuthology.orchestra.run.smithi005.stdout: "7b6a3c33-cc02-48c1-a924-daf7fcbb8f82": { 2024-09-10T09:56:30.281 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2024-09-10T09:56:30.281 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "7b6a3c33-cc02-48c1-a924-daf7fcbb8f82", 2024-09-10T09:56:30.281 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2024-09-10T09:56:30.281 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T09:56:30.281 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T09:56:30.281 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-10T09:56:30.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:30.277+0000 7fc022000640 1 -- 172.21.15.5:0/2375919772 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fc0000a2c00 msgr2=0x7fc0000a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:30.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:30.277+0000 7fc022000640 1 --2- 172.21.15.5:0/2375919772 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fc0000a2c00 0x7fc0000a50c0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fc014009fd0 tx=0x7fc014009500 comp rx=0 tx=0).stop 2024-09-10T09:56:30.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:30.277+0000 7fc022000640 1 -- 172.21.15.5:0/2375919772 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc01c100a80 msgr2=0x7fc01c198470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:30.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:30.277+0000 7fc022000640 1 --2- 172.21.15.5:0/2375919772 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc01c100a80 0x7fc01c198470 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fc00c00e9f0 tx=0x7fc00c00edf0 comp rx=0 tx=0).stop 2024-09-10T09:56:30.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:30.277+0000 7fc022000640 1 -- 172.21.15.5:0/2375919772 shutdown_connections 2024-09-10T09:56:30.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:30.277+0000 7fc022000640 1 --2- 172.21.15.5:0/2375919772 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fc0000a2c00 0x7fc0000a50c0 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:30.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:30.277+0000 7fc022000640 1 --2- 172.21.15.5:0/2375919772 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc01c106a80 0x7fc01c19cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:30.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:30.277+0000 7fc022000640 1 --2- 172.21.15.5:0/2375919772 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc01c101420 0x7fc01c1989b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:30.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:30.277+0000 7fc022000640 1 --2- 172.21.15.5:0/2375919772 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc01c100a80 0x7fc01c198470 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:30.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:30.277+0000 7fc022000640 1 -- 172.21.15.5:0/2375919772 >> 172.21.15.5:0/2375919772 conn(0x7fc01c0fc7a0 msgr2=0x7fc01c06aad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:56:30.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:30.277+0000 7fc022000640 1 -- 172.21.15.5:0/2375919772 shutdown_connections 2024-09-10T09:56:30.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:30.277+0000 7fc022000640 1 -- 172.21.15.5:0/2375919772 wait complete. 2024-09-10T09:56:30.370 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:56:30 smithi005 ceph-mon[30350]: from='client.? 172.21.15.5:0/2622372499' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:56:30.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:56:30 smithi159 ceph-mon[35513]: from='client.? 172.21.15.5:0/2622372499' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:56:30.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:56:30 smithi102 ceph-mon[35442]: from='client.? 172.21.15.5:0/2622372499' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:56:30.985 INFO:tasks.workunit.client.0.smithi005.stdout: 10 69576 12.76 MB/sec execute 129 sec latency 31835.745 ms 2024-09-10T09:56:31.048 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24542 quiesce db --set-id 16796d73 --release --await' 2024-09-10T09:56:31.048 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24542 quiesce db --set-id 16796d73 --release --await 2024-09-10T09:56:31.306 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:56:31 smithi005 ceph-mon[30350]: pgmap v938: 129 pgs: 129 active+clean; 582 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 205 KiB/s rd, 4.4 MiB/s wr, 159 op/s 2024-09-10T09:56:31.536 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:56:31 smithi102 ceph-mon[35442]: pgmap v938: 129 pgs: 129 active+clean; 582 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 205 KiB/s rd, 4.4 MiB/s wr, 159 op/s 2024-09-10T09:56:31.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:56:31 smithi159 ceph-mon[35513]: pgmap v938: 129 pgs: 129 active+clean; 582 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 205 KiB/s rd, 4.4 MiB/s wr, 159 op/s 2024-09-10T09:56:31.701 INFO:tasks.cephfs.filesystem:scrub status for tag:7b6a3c33-cc02-48c1-a924-daf7fcbb8f82 - {'path': '/', 'tag': '7b6a3c33-cc02-48c1-a924-daf7fcbb8f82', 'options': 'recursive,force'} 2024-09-10T09:56:31.741 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:56:31.985 INFO:tasks.workunit.client.0.smithi005.stdout: 10 69576 12.66 MB/sec execute 130 sec latency 32835.828 ms 2024-09-10T09:56:32.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:32.298+0000 7f731da00640 1 -- 172.21.15.5:0/4250757494 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7318103ff0 msgr2=0x7f73181043d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:32.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:32.298+0000 7f731da00640 1 --2- 172.21.15.5:0/4250757494 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7318103ff0 0x7f73181043d0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f7308009ca0 tx=0x7f730802a8c0 comp rx=0 tx=0).stop 2024-09-10T09:56:32.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:32.298+0000 7f731da00640 1 -- 172.21.15.5:0/4250757494 shutdown_connections 2024-09-10T09:56:32.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:32.298+0000 7f731da00640 1 --2- 172.21.15.5:0/4250757494 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7318109120 0x7f7318110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:32.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:32.298+0000 7f731da00640 1 --2- 172.21.15.5:0/4250757494 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f73181049a0 0x7f73181089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:32.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:32.298+0000 7f731da00640 1 --2- 172.21.15.5:0/4250757494 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7318103ff0 0x7f73181043d0 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:32.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:32.298+0000 7f731da00640 1 -- 172.21.15.5:0/4250757494 >> 172.21.15.5:0/4250757494 conn(0x7f73180ffa90 msgr2=0x7f7318101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:56:32.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:32.298+0000 7f731da00640 1 -- 172.21.15.5:0/4250757494 shutdown_connections 2024-09-10T09:56:32.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:32.298+0000 7f731da00640 1 -- 172.21.15.5:0/4250757494 wait complete. 2024-09-10T09:56:32.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:32.299+0000 7f731da00640 1 Processor -- start 2024-09-10T09:56:32.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:32.299+0000 7f731da00640 1 -- start start 2024-09-10T09:56:32.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:32.300+0000 7f731da00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7318103ff0 0x7f731819a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:32.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:32.300+0000 7f731da00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f73181049a0 0x7f731819ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:32.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:32.300+0000 7f731da00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7318109120 0x7f731819eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:32.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:32.300+0000 7f731da00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f7318114120 con 0x7f7318109120 2024-09-10T09:56:32.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:32.300+0000 7f731da00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f7318113fa0 con 0x7f7318103ff0 2024-09-10T09:56:32.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:32.300+0000 7f731da00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f73181142a0 con 0x7f73181049a0 2024-09-10T09:56:32.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:32.300+0000 7f7316a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7318103ff0 0x7f731819a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:32.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:32.300+0000 7f7316a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7318103ff0 0x7f731819a5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:45966/0 (socket says 172.21.15.5:45966) 2024-09-10T09:56:32.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:32.300+0000 7f7316a00640 1 -- 172.21.15.5:0/3205639448 learned_addr learned my addr 172.21.15.5:0/3205639448 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:56:32.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:32.300+0000 7f730e000640 1 --2- 172.21.15.5:0/3205639448 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f73181049a0 0x7f731819ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:32.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:32.300+0000 7f7317400640 1 --2- 172.21.15.5:0/3205639448 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7318109120 0x7f731819eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:32.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:32.300+0000 7f7316a00640 1 -- 172.21.15.5:0/3205639448 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f73181049a0 msgr2=0x7f731819ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:32.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:32.300+0000 7f7316a00640 1 --2- 172.21.15.5:0/3205639448 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f73181049a0 0x7f731819ab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:32.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:32.300+0000 7f7316a00640 1 -- 172.21.15.5:0/3205639448 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7318109120 msgr2=0x7f731819eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:32.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:32.300+0000 7f7316a00640 1 --2- 172.21.15.5:0/3205639448 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7318109120 0x7f731819eeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:32.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:32.300+0000 7f7316a00640 1 -- 172.21.15.5:0/3205639448 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f731819f590 con 0x7f7318103ff0 2024-09-10T09:56:32.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:32.300+0000 7f7317400640 1 --2- 172.21.15.5:0/3205639448 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7318109120 0x7f731819eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:56:32.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:32.301+0000 7f7316a00640 1 --2- 172.21.15.5:0/3205639448 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7318103ff0 0x7f731819a5e0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f730802ade0 tx=0x7f73080319f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:56:32.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:32.301+0000 7f730fe00640 1 -- 172.21.15.5:0/3205639448 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f730802fa20 con 0x7f7318103ff0 2024-09-10T09:56:32.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:32.301+0000 7f731da00640 1 -- 172.21.15.5:0/3205639448 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f731819f820 con 0x7f7318103ff0 2024-09-10T09:56:32.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:32.301+0000 7f730fe00640 1 -- 172.21.15.5:0/3205639448 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f730802fbc0 con 0x7f7318103ff0 2024-09-10T09:56:32.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:32.301+0000 7f730fe00640 1 -- 172.21.15.5:0/3205639448 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73080466e0 con 0x7f7318103ff0 2024-09-10T09:56:32.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:32.301+0000 7f731da00640 1 -- 172.21.15.5:0/3205639448 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73181a7160 con 0x7f7318103ff0 2024-09-10T09:56:32.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:32.304+0000 7f73205c3740 1 Processor -- start 2024-09-10T09:56:32.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:32.304+0000 7f730fe00640 1 -- 172.21.15.5:0/3205639448 <== mon.2 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f7308046940 con 0x7f7318103ff0 2024-09-10T09:56:32.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:32.305+0000 7f73205c3740 1 -- start start 2024-09-10T09:56:32.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:32.305+0000 7f73205c3740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560dd89cf310 0x560dd89cf6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:32.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:32.305+0000 7f73205c3740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560dd89cfc30 0x560dd89d9fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:32.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:32.305+0000 7f73205c3740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560dd89da4f0 0x560dd89dc8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:32.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:32.305+0000 7f73205c3740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x560dd8935650 con 0x560dd89da4f0 2024-09-10T09:56:32.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:32.305+0000 7f73205c3740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x560dd88dcf10 con 0x560dd89cfc30 2024-09-10T09:56:32.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:32.305+0000 7f73205c3740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x560dd8931610 con 0x560dd89cf310 2024-09-10T09:56:32.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:32.305+0000 7f7316a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560dd89cfc30 0x560dd89d9fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:32.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:32.305+0000 7f7316a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560dd89cfc30 0x560dd89d9fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:45978/0 (socket says 172.21.15.5:45978) 2024-09-10T09:56:32.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:32.305+0000 7f730e000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560dd89da4f0 0x560dd89dc8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:32.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:32.305+0000 7f7316a00640 1 -- 172.21.15.5:0/2726309550 learned_addr learned my addr 172.21.15.5:0/2726309550 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:56:32.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:32.305+0000 7f7317400640 1 --2- 172.21.15.5:0/2726309550 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560dd89cf310 0x560dd89cf6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:32.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:32.306+0000 7f7316a00640 1 -- 172.21.15.5:0/2726309550 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560dd89cf310 msgr2=0x560dd89cf6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:56:32.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:32.306+0000 7f7316a00640 1 --2- 172.21.15.5:0/2726309550 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560dd89cf310 0x560dd89cf6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:32.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:32.306+0000 7f7316a00640 1 -- 172.21.15.5:0/2726309550 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560dd89da4f0 msgr2=0x560dd89dc8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:56:32.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:32.306+0000 7f7316a00640 1 --2- 172.21.15.5:0/2726309550 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560dd89da4f0 0x560dd89dc8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:32.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:32.306+0000 7f7316a00640 1 -- 172.21.15.5:0/2726309550 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560dd8644580 con 0x560dd89cfc30 2024-09-10T09:56:32.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:32.306+0000 7f730e000640 1 --2- 172.21.15.5:0/2726309550 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560dd89da4f0 0x560dd89dc8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:56:32.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:32.306+0000 7f730fe00640 1 --2- 172.21.15.5:0/3205639448 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f72fc0a2b80 0x7f72fc0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:32.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:32.306+0000 7f7316a00640 1 --2- 172.21.15.5:0/2726309550 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560dd89cfc30 0x560dd89d9fb0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f73080f4a00 tx=0x7f73080f4af0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=841d3809242120a9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:56:32.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:32.306+0000 7f730fe00640 1 -- 172.21.15.5:0/3205639448 <== mon.2 v2:172.21.15.102:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f73080f0ae0 con 0x7f7318103ff0 2024-09-10T09:56:32.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:32.306+0000 7f7317400640 1 --2- 172.21.15.5:0/3205639448 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f72fc0a2b80 0x7f72fc0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:32.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:32.306+0000 7f7307e00640 1 -- 172.21.15.5:0/2726309550 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73080b9190 con 0x560dd89cfc30 2024-09-10T09:56:32.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:32.307+0000 7f7307e00640 1 -- 172.21.15.5:0/2726309550 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7308036020 con 0x560dd89cfc30 2024-09-10T09:56:32.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:32.307+0000 7f7307e00640 1 -- 172.21.15.5:0/2726309550 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73080bc850 con 0x560dd89cfc30 2024-09-10T09:56:32.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:32.307+0000 7f73205c3740 1 -- 172.21.15.5:0/2726309550 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560dd89cfc30 msgr2=0x560dd89d9fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:32.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:32.307+0000 7f73205c3740 1 --2- 172.21.15.5:0/2726309550 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560dd89cfc30 0x560dd89d9fb0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f73080f4a00 tx=0x7f73080f4af0 comp rx=0 tx=0).stop 2024-09-10T09:56:32.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:32.307+0000 7f7317400640 1 --2- 172.21.15.5:0/3205639448 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f72fc0a2b80 0x7f72fc0a5040 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f7310009cb0 tx=0x7f7310005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:56:32.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:32.307+0000 7f73205c3740 1 -- 172.21.15.5:0/2726309550 shutdown_connections 2024-09-10T09:56:32.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:32.307+0000 7f73205c3740 1 --2- 172.21.15.5:0/2726309550 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560dd89da4f0 0x560dd89dc8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:32.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:32.307+0000 7f73205c3740 1 --2- 172.21.15.5:0/2726309550 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560dd89cfc30 0x560dd89d9fb0 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:32.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:32.307+0000 7f73205c3740 1 --2- 172.21.15.5:0/2726309550 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560dd89cf310 0x560dd89cf6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:32.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:32.307+0000 7f73205c3740 1 -- 172.21.15.5:0/2726309550 >> 172.21.15.5:0/2726309550 conn(0x560dd89cb8a0 msgr2=0x560dd89cbc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:56:32.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:32.307+0000 7f73205c3740 1 -- 172.21.15.5:0/2726309550 shutdown_connections 2024-09-10T09:56:32.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:32.307+0000 7f73205c3740 1 -- 172.21.15.5:0/2726309550 wait complete. 2024-09-10T09:56:32.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:32.308+0000 7f73205c3740 1 -- start start 2024-09-10T09:56:32.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:32.308+0000 7f73205c3740 1 Processor -- start 2024-09-10T09:56:32.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:32.308+0000 7f73205c3740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560dd89cf310 0x560dd89d7450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:32.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:32.309+0000 7f73205c3740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560dd89cfc30 0x560dd89d7990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:32.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:32.309+0000 7f73205c3740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560dd89da4f0 0x560dd89d7ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:32.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:32.309+0000 7f73205c3740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x560dd88dcf10 con 0x560dd89da4f0 2024-09-10T09:56:32.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:32.309+0000 7f73205c3740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x560dd88f0350 con 0x560dd89cfc30 2024-09-10T09:56:32.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:32.309+0000 7f73205c3740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x560dd89dce80 con 0x560dd89cf310 2024-09-10T09:56:32.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:32.309+0000 7f73205c3740 10 client.0 _ll_register_callbacks cb 0x560dd8913290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:56:32.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:32.309+0000 7f7316a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560dd89cf310 0x560dd89d7450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:32.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:32.309+0000 7f7317400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560dd89da4f0 0x560dd89d7ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:32.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:32.309+0000 7f7316a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560dd89cf310 0x560dd89d7450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:56472/0 (socket says 172.21.15.5:56472) 2024-09-10T09:56:32.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:32.309+0000 7f7316a00640 1 -- 172.21.15.5:0/1951471570 learned_addr learned my addr 172.21.15.5:0/1951471570 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:56:32.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:32.309+0000 7f730e000640 1 --2- 172.21.15.5:0/1951471570 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560dd89cfc30 0x560dd89d7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:32.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:32.309+0000 7f7316a00640 1 -- 172.21.15.5:0/1951471570 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560dd89cfc30 msgr2=0x560dd89d7990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:56:32.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:32.309+0000 7f7316a00640 1 --2- 172.21.15.5:0/1951471570 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560dd89cfc30 0x560dd89d7990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:32.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:32.309+0000 7f7316a00640 1 -- 172.21.15.5:0/1951471570 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560dd89da4f0 msgr2=0x560dd89d7ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:56:32.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:32.309+0000 7f7316a00640 1 --2- 172.21.15.5:0/1951471570 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560dd89da4f0 0x560dd89d7ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:32.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:32.309+0000 7f7316a00640 1 -- 172.21.15.5:0/1951471570 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560dd870e960 con 0x560dd89cf310 2024-09-10T09:56:32.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:32.309+0000 7f730e000640 1 --2- 172.21.15.5:0/1951471570 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560dd89cfc30 0x560dd89d7990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:56:32.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:32.310+0000 7f7316a00640 1 --2- 172.21.15.5:0/1951471570 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560dd89cf310 0x560dd89d7450 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f73080f49d0 tx=0x7f73080bd5f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=23c6e6349f4c10ac server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:56:32.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:32.310+0000 7f7304c00640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-09-10T09:56:32.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:32.310+0000 7f7304c00640 1 -- 172.21.15.5:0/1951471570 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73080bca30 con 0x560dd89cf310 2024-09-10T09:56:32.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:32.310+0000 7f7304c00640 1 -- 172.21.15.5:0/1951471570 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f73080bd150 con 0x560dd89cf310 2024-09-10T09:56:32.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:32.310+0000 7f73205c3740 1 -- 172.21.15.5:0/1951471570 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7308036020 con 0x560dd89cf310 2024-09-10T09:56:32.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:32.310+0000 7f7304c00640 1 -- 172.21.15.5:0/1951471570 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f73080bccc0 con 0x560dd89cf310 2024-09-10T09:56:32.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:32.310+0000 7f73205c3740 10 client.28999 fetch_fsmap learned FSMap version 42 2024-09-10T09:56:32.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:32.310+0000 7f73205c3740 1 -- 172.21.15.5:0/1951471570 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x560dd89e88f0 con 0x560dd89cf310 2024-09-10T09:56:32.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:32.310+0000 7f7304c00640 1 -- 172.21.15.5:0/1951471570 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7308036020 con 0x560dd89cf310 2024-09-10T09:56:32.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:32.310+0000 7f7304c00640 1 -- 172.21.15.5:0/1951471570 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 42) ==== 8370+0+0 (secure 0 0 0) 0x7f73080c56d0 con 0x560dd89cf310 2024-09-10T09:56:32.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:32.311+0000 7f73205c3740 10 client.28999 fetch_fsmap finished waiting for FSMap version 42 2024-09-10T09:56:32.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:32.311+0000 7f73205c3740 10 client.28999 resolve_mds: validated gid 24542 aka daemon mds.b 2024-09-10T09:56:32.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:32.311+0000 7f73205c3740 20 client.28999 populate_metadata read hostname 'smithi005' 2024-09-10T09:56:32.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:32.311+0000 7f73205c3740 1 --2- 172.21.15.5:0/1951471570 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x560dd89e9a70 0x560dd89ebe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:32.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:32.311+0000 7f73205c3740 4 client.28999 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:56:32.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:32.311+0000 7f73205c3740 1 -- 172.21.15.5:0/1951471570 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560dd8942bc0 con 0x560dd89e9a70 2024-09-10T09:56:32.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:32.311+0000 7f730e000640 1 --2- 172.21.15.5:0/1951471570 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x560dd89e9a70 0x560dd89ebe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:32.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:32.311+0000 7f730e000640 1 --2- 172.21.15.5:0/1951471570 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x560dd89e9a70 0x560dd89ebe80 crc :-1 s=READY pgs=261 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=50b87f1fbe6378a6 server_cookie=368f16407dc02b68 in_seq=0 out_seq=0 2024-09-10T09:56:32.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:32.311+0000 7f7304c00640 10 client.28999 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T09:56:32.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:32.312+0000 7f7304c00640 1 -- 172.21.15.5:0/1951471570 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560dd8942bc0 con 0x560dd89e9a70 2024-09-10T09:56:32.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:32.312+0000 7f7304c00640 10 client.28999 handle_command_reply: tid=0 2024-09-10T09:56:32.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:32.312+0000 7f73205c3740 1 client.28999 shutdown 2024-09-10T09:56:32.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:32.313+0000 7f73205c3740 1 -- 172.21.15.5:0/1951471570 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560dd89cf310 msgr2=0x560dd89d7450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:32.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:32.313+0000 7f73205c3740 1 --2- 172.21.15.5:0/1951471570 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560dd89cf310 0x560dd89d7450 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f73080f49d0 tx=0x7f73080bd5f0 comp rx=0 tx=0).stop 2024-09-10T09:56:32.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:32.313+0000 7f73205c3740 1 -- 172.21.15.5:0/1951471570 shutdown_connections 2024-09-10T09:56:32.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:32.313+0000 7f73205c3740 1 --2- 172.21.15.5:0/1951471570 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x560dd89e9a70 0x560dd89ebe80 crc :-1 s=READY pgs=261 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:32.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:32.313+0000 7f73205c3740 1 --2- 172.21.15.5:0/1951471570 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560dd89da4f0 0x560dd89d7ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:32.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:32.313+0000 7f73205c3740 1 --2- 172.21.15.5:0/1951471570 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560dd89cfc30 0x560dd89d7990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:32.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:32.313+0000 7f73205c3740 1 --2- 172.21.15.5:0/1951471570 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560dd89cf310 0x560dd89d7450 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:32.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:32.313+0000 7f7304c00640 0 client.28999 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T09:56:32.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:32.313+0000 7f73205c3740 1 -- 172.21.15.5:0/1951471570 >> 172.21.15.5:0/1951471570 conn(0x560dd89cb8a0 msgr2=0x560dd89caaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:56:32.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:32.313+0000 7f73205c3740 1 -- 172.21.15.5:0/1951471570 shutdown_connections 2024-09-10T09:56:32.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:32.313+0000 7f73205c3740 1 -- 172.21.15.5:0/1951471570 wait complete. 2024-09-10T09:56:32.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:32.313+0000 7f73205c3740 20 client.28999 trim_cache size 0 max 16384 2024-09-10T09:56:32.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:32.331+0000 7f73205c3740 1 Processor -- start 2024-09-10T09:56:32.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:32.331+0000 7f73205c3740 1 -- start start 2024-09-10T09:56:32.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:32.331+0000 7f73205c3740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560dd89cf310 0x560dd89d6390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:32.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:32.331+0000 7f73205c3740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560dd89cfc30 0x560dd89f1750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:32.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:32.331+0000 7f73205c3740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560dd89da4f0 0x560dd89f1c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:32.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:32.331+0000 7f73205c3740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f73181b1020 con 0x560dd89cfc30 2024-09-10T09:56:32.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:32.331+0000 7f73205c3740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f73181b2b60 con 0x560dd89cf310 2024-09-10T09:56:32.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:32.331+0000 7f73205c3740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f73181b16f0 con 0x560dd89da4f0 2024-09-10T09:56:32.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:32.331+0000 7f730e000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560dd89cfc30 0x560dd89f1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:32.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:32.331+0000 7f7316a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560dd89cf310 0x560dd89d6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:32.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:32.331+0000 7f730e000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560dd89cfc30 0x560dd89f1750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33318/0 (socket says 172.21.15.5:33318) 2024-09-10T09:56:32.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:32.331+0000 7f730e000640 1 -- 172.21.15.5:0/4017223369 learned_addr learned my addr 172.21.15.5:0/4017223369 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:56:32.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:32.332+0000 7f7317400640 1 --2- 172.21.15.5:0/4017223369 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560dd89da4f0 0x560dd89f1c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:32.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:32.332+0000 7f7316a00640 1 -- 172.21.15.5:0/4017223369 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560dd89da4f0 msgr2=0x560dd89f1c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:56:32.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:32.332+0000 7f7316a00640 1 --2- 172.21.15.5:0/4017223369 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560dd89da4f0 0x560dd89f1c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:32.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:32.332+0000 7f7316a00640 1 -- 172.21.15.5:0/4017223369 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560dd89cfc30 msgr2=0x560dd89f1750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:56:32.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:32.332+0000 7f7316a00640 1 --2- 172.21.15.5:0/4017223369 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560dd89cfc30 0x560dd89f1750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:32.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:32.332+0000 7f7316a00640 1 -- 172.21.15.5:0/4017223369 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560dd89cc910 con 0x560dd89cf310 2024-09-10T09:56:32.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:32.332+0000 7f7316a00640 1 --2- 172.21.15.5:0/4017223369 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560dd89cf310 0x560dd89d6390 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f7308006010 tx=0x7f73080bb400 comp rx=0 tx=0).ready entity=mon.2 client_cookie=da83d59a206d3ecb server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:56:32.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:32.332+0000 7f7307400640 1 -- 172.21.15.5:0/4017223369 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73080bca30 con 0x560dd89cf310 2024-09-10T09:56:32.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:32.332+0000 7f7307400640 1 -- 172.21.15.5:0/4017223369 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7308036070 con 0x560dd89cf310 2024-09-10T09:56:32.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:32.332+0000 7f7307400640 1 -- 172.21.15.5:0/4017223369 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73080bc850 con 0x560dd89cf310 2024-09-10T09:56:32.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:32.333+0000 7f73205c3740 1 -- 172.21.15.5:0/4017223369 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560dd89cf310 msgr2=0x560dd89d6390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:32.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:32.333+0000 7f73205c3740 1 --2- 172.21.15.5:0/4017223369 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560dd89cf310 0x560dd89d6390 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f7308006010 tx=0x7f73080bb400 comp rx=0 tx=0).stop 2024-09-10T09:56:32.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:32.333+0000 7f73205c3740 1 -- 172.21.15.5:0/4017223369 shutdown_connections 2024-09-10T09:56:32.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:32.333+0000 7f73205c3740 1 --2- 172.21.15.5:0/4017223369 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560dd89da4f0 0x560dd89f1c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:32.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:32.333+0000 7f73205c3740 1 --2- 172.21.15.5:0/4017223369 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560dd89cfc30 0x560dd89f1750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:32.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:32.333+0000 7f73205c3740 1 --2- 172.21.15.5:0/4017223369 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560dd89cf310 0x560dd89d6390 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:32.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:32.333+0000 7f73205c3740 1 -- 172.21.15.5:0/4017223369 >> 172.21.15.5:0/4017223369 conn(0x560dd89cb8a0 msgr2=0x560dd89d93c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:56:32.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:32.333+0000 7f73205c3740 1 -- 172.21.15.5:0/4017223369 shutdown_connections 2024-09-10T09:56:32.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:32.333+0000 7f73205c3740 1 -- 172.21.15.5:0/4017223369 wait complete. 2024-09-10T09:56:32.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:32.334+0000 7f73205c3740 1 -- start start 2024-09-10T09:56:32.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:32.334+0000 7f73205c3740 1 Processor -- start 2024-09-10T09:56:32.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:32.334+0000 7f73205c3740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560dd89cf310 0x560dd89e2560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:32.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:32.334+0000 7f73205c3740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560dd89cfc30 0x560dd89dd8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:32.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:32.334+0000 7f73205c3740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560dd89da4f0 0x560dd89dde20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:32.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:32.334+0000 7f73205c3740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f73181b2490 con 0x560dd89cfc30 2024-09-10T09:56:32.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:32.334+0000 7f73205c3740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f73181b1dc0 con 0x560dd89da4f0 2024-09-10T09:56:32.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:32.334+0000 7f73205c3740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f73181af000 con 0x560dd89cf310 2024-09-10T09:56:32.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:32.334+0000 7f73205c3740 10 client.0 _ll_register_callbacks cb 0x7f730000b810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:56:32.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:32.335+0000 7f730e000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560dd89cfc30 0x560dd89dd8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:32.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:32.335+0000 7f7317400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560dd89da4f0 0x560dd89dde20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:32.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:32.335+0000 7f730e000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560dd89cfc30 0x560dd89dd8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33326/0 (socket says 172.21.15.5:33326) 2024-09-10T09:56:32.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:32.335+0000 7f730e000640 1 -- 172.21.15.5:0/3856549682 learned_addr learned my addr 172.21.15.5:0/3856549682 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:56:32.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:32.335+0000 7f7317400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560dd89da4f0 0x560dd89dde20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:46002/0 (socket says 172.21.15.5:46002) 2024-09-10T09:56:32.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:32.335+0000 7f7316a00640 1 --2- 172.21.15.5:0/3856549682 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560dd89cf310 0x560dd89e2560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:32.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:32.335+0000 7f7316a00640 1 -- 172.21.15.5:0/3856549682 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560dd89da4f0 msgr2=0x560dd89dde20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:56:32.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:32.335+0000 7f7316a00640 1 --2- 172.21.15.5:0/3856549682 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560dd89da4f0 0x560dd89dde20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:32.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:32.335+0000 7f7316a00640 1 -- 172.21.15.5:0/3856549682 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560dd89cfc30 msgr2=0x560dd89dd8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:56:32.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:32.335+0000 7f7316a00640 1 --2- 172.21.15.5:0/3856549682 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560dd89cfc30 0x560dd89dd8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:32.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:32.335+0000 7f7316a00640 1 -- 172.21.15.5:0/3856549682 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560dd89de460 con 0x560dd89cf310 2024-09-10T09:56:32.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:32.335+0000 7f7316a00640 1 --2- 172.21.15.5:0/3856549682 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560dd89cf310 0x560dd89e2560 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f73080c13c0 tx=0x7f7308005c50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f255a42cdfc846b8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:56:32.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:32.335+0000 7f7304c00640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-09-10T09:56:32.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:32.336+0000 7f7304c00640 1 -- 172.21.15.5:0/3856549682 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73080b9190 con 0x560dd89cf310 2024-09-10T09:56:32.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:32.336+0000 7f7304c00640 1 -- 172.21.15.5:0/3856549682 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f7308004b40 con 0x560dd89cf310 2024-09-10T09:56:32.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:32.336+0000 7f73205c3740 1 -- 172.21.15.5:0/3856549682 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7308036070 con 0x560dd89cf310 2024-09-10T09:56:32.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:32.336+0000 7f7304c00640 1 -- 172.21.15.5:0/3856549682 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f73080bc850 con 0x560dd89cf310 2024-09-10T09:56:32.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:32.336+0000 7f73205c3740 10 client.29005 fetch_fsmap learned FSMap version 42 2024-09-10T09:56:32.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:32.336+0000 7f73205c3740 1 -- 172.21.15.5:0/3856549682 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x560dd89de460 con 0x560dd89cf310 2024-09-10T09:56:32.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:32.336+0000 7f7304c00640 1 -- 172.21.15.5:0/3856549682 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7308036070 con 0x560dd89cf310 2024-09-10T09:56:32.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:32.336+0000 7f7304c00640 1 -- 172.21.15.5:0/3856549682 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 42) ==== 8370+0+0 (secure 0 0 0) 0x7f73080dbbc0 con 0x560dd89cf310 2024-09-10T09:56:32.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:32.336+0000 7f73205c3740 10 client.29005 fetch_fsmap finished waiting for FSMap version 42 2024-09-10T09:56:32.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:32.336+0000 7f73205c3740 10 client.29005 resolve_mds: validated gid 24542 aka daemon mds.b 2024-09-10T09:56:32.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:32.336+0000 7f73205c3740 20 client.29005 populate_metadata read hostname 'smithi005' 2024-09-10T09:56:32.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:32.336+0000 7f73205c3740 1 --2- 172.21.15.5:0/3856549682 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x560dd8a54870 0x560dd8a56c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:56:32.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:32.336+0000 7f73205c3740 4 client.29005 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "16796d73", "release": true, "await": true}] 2024-09-10T09:56:32.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:32.336+0000 7f73205c3740 1 -- 172.21.15.5:0/3856549682 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "quiesce db", "set_id": "16796d73", "release": true, "await": true}) -- 0x560dd893d0d0 con 0x560dd8a54870 2024-09-10T09:56:32.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:32.337+0000 7f730e000640 1 --2- 172.21.15.5:0/3856549682 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x560dd8a54870 0x560dd8a56c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:56:32.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:32.337+0000 7f730e000640 1 --2- 172.21.15.5:0/3856549682 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x560dd8a54870 0x560dd8a56c60 crc :-1 s=READY pgs=262 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6443659bf87be22f server_cookie=449fa75188c8adf in_seq=0 out_seq=0 2024-09-10T09:56:32.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:32.337+0000 7f7304c00640 10 client.29005 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T09:56:32.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:32.405+0000 7f7304c00640 1 -- 172.21.15.5:0/3856549682 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x560dd893d0d0 con 0x560dd8a54870 2024-09-10T09:56:32.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:32.405+0000 7f7304c00640 10 client.29005 handle_command_reply: tid=0 2024-09-10T09:56:32.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:32.405+0000 7f73205c3740 1 client.29005 shutdown 2024-09-10T09:56:32.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:32.405+0000 7f73205c3740 1 -- 172.21.15.5:0/3856549682 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560dd89cf310 msgr2=0x560dd89e2560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:32.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:32.406+0000 7f73205c3740 1 --2- 172.21.15.5:0/3856549682 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560dd89cf310 0x560dd89e2560 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f73080c13c0 tx=0x7f7308005c50 comp rx=0 tx=0).stop 2024-09-10T09:56:32.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:32.406+0000 7f73205c3740 1 -- 172.21.15.5:0/3856549682 shutdown_connections 2024-09-10T09:56:32.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:32.406+0000 7f73205c3740 1 --2- 172.21.15.5:0/3856549682 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x560dd8a54870 0x560dd8a56c60 crc :-1 s=READY pgs=262 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:32.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:32.406+0000 7f73205c3740 1 --2- 172.21.15.5:0/3856549682 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560dd89da4f0 0x560dd89dde20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:32.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:32.406+0000 7f73205c3740 1 --2- 172.21.15.5:0/3856549682 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560dd89cfc30 0x560dd89dd8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:32.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:32.406+0000 7f73205c3740 1 --2- 172.21.15.5:0/3856549682 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560dd89cf310 0x560dd89e2560 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:32.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:32.406+0000 7f7304c00640 0 client.29005 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T09:56:32.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:32.406+0000 7f73205c3740 1 -- 172.21.15.5:0/3856549682 >> 172.21.15.5:0/3856549682 conn(0x560dd89cb8a0 msgr2=0x560dd89d0d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:56:32.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:32.406+0000 7f73205c3740 1 -- 172.21.15.5:0/3856549682 shutdown_connections 2024-09-10T09:56:32.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:32.406+0000 7f73205c3740 1 -- 172.21.15.5:0/3856549682 wait complete. 2024-09-10T09:56:32.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:32.406+0000 7f73205c3740 20 client.29005 trim_cache size 0 max 16384 2024-09-10T09:56:32.409 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-10T09:56:32.409 INFO:teuthology.orchestra.run.smithi005.stdout: "epoch": 42, 2024-09-10T09:56:32.409 INFO:teuthology.orchestra.run.smithi005.stdout: "leader": 24542, 2024-09-10T09:56:32.409 INFO:teuthology.orchestra.run.smithi005.stdout: "set_version": 64, 2024-09-10T09:56:32.409 INFO:teuthology.orchestra.run.smithi005.stdout: "sets": { 2024-09-10T09:56:32.409 INFO:teuthology.orchestra.run.smithi005.stdout: "16796d73": { 2024-09-10T09:56:32.409 INFO:teuthology.orchestra.run.smithi005.stdout: "version": 64, 2024-09-10T09:56:32.409 INFO:teuthology.orchestra.run.smithi005.stdout: "age_ref": 0.0, 2024-09-10T09:56:32.409 INFO:teuthology.orchestra.run.smithi005.stdout: "state": { 2024-09-10T09:56:32.409 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "RELEASED", 2024-09-10T09:56:32.409 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0 2024-09-10T09:56:32.409 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T09:56:32.410 INFO:teuthology.orchestra.run.smithi005.stdout: "timeout": 90.0, 2024-09-10T09:56:32.410 INFO:teuthology.orchestra.run.smithi005.stdout: "expiration": 150.5, 2024-09-10T09:56:32.410 INFO:teuthology.orchestra.run.smithi005.stdout: "members": { 2024-09-10T09:56:32.410 INFO:teuthology.orchestra.run.smithi005.stdout: "file:/": { 2024-09-10T09:56:32.410 INFO:teuthology.orchestra.run.smithi005.stdout: "excluded": false, 2024-09-10T09:56:32.410 INFO:teuthology.orchestra.run.smithi005.stdout: "state": { 2024-09-10T09:56:32.411 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "RELEASED", 2024-09-10T09:56:32.411 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0 2024-09-10T09:56:32.411 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T09:56:32.411 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T09:56:32.411 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T09:56:32.412 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T09:56:32.412 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T09:56:32.412 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-10T09:56:32.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:32.407+0000 7f731da00640 1 -- 172.21.15.5:0/3205639448 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f72fc0a2b80 msgr2=0x7f72fc0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:32.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:32.407+0000 7f731da00640 1 --2- 172.21.15.5:0/3205639448 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f72fc0a2b80 0x7f72fc0a5040 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f7310009cb0 tx=0x7f7310005f90 comp rx=0 tx=0).stop 2024-09-10T09:56:32.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:32.407+0000 7f731da00640 1 -- 172.21.15.5:0/3205639448 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7318103ff0 msgr2=0x7f731819a5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:56:32.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:32.407+0000 7f731da00640 1 --2- 172.21.15.5:0/3205639448 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7318103ff0 0x7f731819a5e0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f730802ade0 tx=0x7f73080319f0 comp rx=0 tx=0).stop 2024-09-10T09:56:32.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:32.407+0000 7f731da00640 1 -- 172.21.15.5:0/3205639448 shutdown_connections 2024-09-10T09:56:32.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:32.407+0000 7f731da00640 1 --2- 172.21.15.5:0/3205639448 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f72fc0a2b80 0x7f72fc0a5040 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:32.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:32.407+0000 7f731da00640 1 --2- 172.21.15.5:0/3205639448 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7318109120 0x7f731819eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:32.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:32.407+0000 7f731da00640 1 --2- 172.21.15.5:0/3205639448 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f73181049a0 0x7f731819ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:32.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:32.407+0000 7f731da00640 1 --2- 172.21.15.5:0/3205639448 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7318103ff0 0x7f731819a5e0 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:56:32.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:32.407+0000 7f731da00640 1 -- 172.21.15.5:0/3205639448 >> 172.21.15.5:0/3205639448 conn(0x7f73180ffa90 msgr2=0x7f7318100e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:56:32.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:32.407+0000 7f731da00640 1 -- 172.21.15.5:0/3205639448 shutdown_connections 2024-09-10T09:56:32.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:56:32.407+0000 7f731da00640 1 -- 172.21.15.5:0/3205639448 wait complete. 2024-09-10T09:56:32.985 INFO:tasks.workunit.client.0.smithi005.stdout: 10 69954 12.71 MB/sec execute 131 sec latency 33252.754 ms 2024-09-10T09:56:33.453 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:56:33 smithi005 ceph-mon[30350]: pgmap v939: 129 pgs: 129 active+clean; 581 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 212 KiB/s rd, 2.9 MiB/s wr, 129 op/s 2024-09-10T09:56:33.459 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '16796d73', total seconds elapsed: 38.6 2024-09-10T09:56:33.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:56:33 smithi159 ceph-mon[35513]: pgmap v939: 129 pgs: 129 active+clean; 581 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 212 KiB/s rd, 2.9 MiB/s wr, 129 op/s 2024-09-10T09:56:33.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:56:33 smithi102 ceph-mon[35442]: pgmap v939: 129 pgs: 129 active+clean; 581 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 212 KiB/s rd, 2.9 MiB/s wr, 129 op/s 2024-09-10T09:56:33.985 INFO:tasks.workunit.client.0.smithi005.stdout: 10 70632 12.89 MB/sec execute 132 sec latency 46.159 ms 2024-09-10T09:56:34.985 INFO:tasks.workunit.client.0.smithi005.stdout: 10 71446 13.05 MB/sec execute 133 sec latency 92.358 ms 2024-09-10T09:56:35.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:56:35 smithi005 ceph-mon[30350]: pgmap v940: 129 pgs: 129 active+clean; 583 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 512 KiB/s rd, 1.8 MiB/s wr, 122 op/s 2024-09-10T09:56:35.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:56:35 smithi159 ceph-mon[35513]: pgmap v940: 129 pgs: 129 active+clean; 583 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 512 KiB/s rd, 1.8 MiB/s wr, 122 op/s 2024-09-10T09:56:35.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:56:35 smithi102 ceph-mon[35442]: pgmap v940: 129 pgs: 129 active+clean; 583 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 512 KiB/s rd, 1.8 MiB/s wr, 122 op/s 2024-09-10T09:56:35.985 INFO:tasks.workunit.client.0.smithi005.stdout: 10 72308 13.43 MB/sec execute 134 sec latency 55.530 ms 2024-09-10T09:56:36.985 INFO:tasks.workunit.client.0.smithi005.stdout: 10 72969 13.59 MB/sec execute 135 sec latency 85.390 ms 2024-09-10T09:56:37.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:56:37 smithi005 ceph-mon[30350]: pgmap v941: 129 pgs: 129 active+clean; 585 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.3 MiB/s wr, 154 op/s 2024-09-10T09:56:37.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:56:37 smithi159 ceph-mon[35513]: pgmap v941: 129 pgs: 129 active+clean; 585 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.3 MiB/s wr, 154 op/s 2024-09-10T09:56:37.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:56:37 smithi102 ceph-mon[35442]: pgmap v941: 129 pgs: 129 active+clean; 585 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.3 MiB/s wr, 154 op/s 2024-09-10T09:56:37.986 INFO:tasks.workunit.client.0.smithi005.stdout: 10 73442 13.61 MB/sec execute 136 sec latency 132.054 ms 2024-09-10T09:56:38.986 INFO:tasks.workunit.client.0.smithi005.stdout: 10 74097 13.78 MB/sec execute 137 sec latency 103.914 ms 2024-09-10T09:56:39.365 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:56:39 smithi005 ceph-mon[30350]: pgmap v942: 129 pgs: 129 active+clean; 586 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 1.3 MiB/s wr, 168 op/s 2024-09-10T09:56:39.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:56:39 smithi159 ceph-mon[35513]: pgmap v942: 129 pgs: 129 active+clean; 586 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 1.3 MiB/s wr, 168 op/s 2024-09-10T09:56:39.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:56:39 smithi102 ceph-mon[35442]: pgmap v942: 129 pgs: 129 active+clean; 586 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 1.3 MiB/s wr, 168 op/s 2024-09-10T09:56:39.985 INFO:tasks.workunit.client.0.smithi005.stdout: 10 74902 13.98 MB/sec execute 138 sec latency 46.190 ms 2024-09-10T09:56:40.986 INFO:tasks.workunit.client.0.smithi005.stdout: 10 75717 14.20 MB/sec execute 139 sec latency 61.038 ms 2024-09-10T09:56:41.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:56:41 smithi005 ceph-mon[30350]: pgmap v943: 129 pgs: 129 active+clean; 596 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 2.5 MiB/s wr, 337 op/s 2024-09-10T09:56:41.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:56:41 smithi159 ceph-mon[35513]: pgmap v943: 129 pgs: 129 active+clean; 596 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 2.5 MiB/s wr, 337 op/s 2024-09-10T09:56:41.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:56:41 smithi102 ceph-mon[35442]: pgmap v943: 129 pgs: 129 active+clean; 596 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 2.5 MiB/s wr, 337 op/s 2024-09-10T09:56:41.986 INFO:tasks.workunit.client.0.smithi005.stdout: 10 76576 14.49 MB/sec execute 140 sec latency 106.383 ms 2024-09-10T09:56:42.557 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:56:42 smithi005 ceph-mon[30350]: scrub summary: idle 2024-09-10T09:56:42.557 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:56:42 smithi005 ceph-mon[30350]: scrub summary: idle 2024-09-10T09:56:42.557 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:56:42 smithi005 ceph-mon[30350]: scrub summary: idle 2024-09-10T09:56:42.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:56:42 smithi159 ceph-mon[35513]: scrub summary: idle 2024-09-10T09:56:42.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:56:42 smithi159 ceph-mon[35513]: scrub summary: idle 2024-09-10T09:56:42.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:56:42 smithi159 ceph-mon[35513]: scrub summary: idle 2024-09-10T09:56:42.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:56:42 smithi102 ceph-mon[35442]: scrub summary: idle 2024-09-10T09:56:42.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:56:42 smithi102 ceph-mon[35442]: scrub summary: idle 2024-09-10T09:56:42.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:56:42 smithi102 ceph-mon[35442]: scrub summary: idle 2024-09-10T09:56:42.986 INFO:tasks.workunit.client.0.smithi005.stdout: 10 77201 14.62 MB/sec execute 141 sec latency 105.027 ms 2024-09-10T09:56:43.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:56:43 smithi005 ceph-mon[30350]: pgmap v944: 129 pgs: 129 active+clean; 596 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 2.5 MiB/s wr, 327 op/s 2024-09-10T09:56:43.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:56:43 smithi005 ceph-mon[30350]: scrub summary: idle 2024-09-10T09:56:43.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:56:43 smithi005 ceph-mon[30350]: scrub summary: idle 2024-09-10T09:56:43.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:56:43 smithi159 ceph-mon[35513]: pgmap v944: 129 pgs: 129 active+clean; 596 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 2.5 MiB/s wr, 327 op/s 2024-09-10T09:56:43.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:56:43 smithi159 ceph-mon[35513]: scrub summary: idle 2024-09-10T09:56:43.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:56:43 smithi159 ceph-mon[35513]: scrub summary: idle 2024-09-10T09:56:43.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:56:43 smithi102 ceph-mon[35442]: pgmap v944: 129 pgs: 129 active+clean; 596 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 2.5 MiB/s wr, 327 op/s 2024-09-10T09:56:43.635 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:56:43 smithi102 ceph-mon[35442]: scrub summary: idle 2024-09-10T09:56:43.635 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:56:43 smithi102 ceph-mon[35442]: scrub summary: idle 2024-09-10T09:56:43.986 INFO:tasks.workunit.client.0.smithi005.stdout: 10 77750 14.71 MB/sec execute 142 sec latency 88.522 ms 2024-09-10T09:56:44.986 INFO:tasks.workunit.client.0.smithi005.stdout: 10 78248 14.75 MB/sec execute 143 sec latency 130.740 ms 2024-09-10T09:56:45.207 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:56:45 smithi005 ceph-mon[30350]: pgmap v945: 129 pgs: 129 active+clean; 605 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 3.4 MiB/s wr, 432 op/s 2024-09-10T09:56:45.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:56:45 smithi159 ceph-mon[35513]: pgmap v945: 129 pgs: 129 active+clean; 605 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 3.4 MiB/s wr, 432 op/s 2024-09-10T09:56:45.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:56:45 smithi102 ceph-mon[35442]: pgmap v945: 129 pgs: 129 active+clean; 605 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 3.4 MiB/s wr, 432 op/s 2024-09-10T09:56:45.986 INFO:tasks.workunit.client.0.smithi005.stdout: 10 79149 14.95 MB/sec execute 144 sec latency 51.366 ms 2024-09-10T09:56:46.986 INFO:tasks.workunit.client.0.smithi005.stdout: 10 79931 15.14 MB/sec execute 145 sec latency 53.129 ms 2024-09-10T09:56:47.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:56:47 smithi005 ceph-mon[30350]: pgmap v946: 129 pgs: 129 active+clean; 609 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 3.7 MiB/s wr, 487 op/s 2024-09-10T09:56:47.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:56:47 smithi159 ceph-mon[35513]: pgmap v946: 129 pgs: 129 active+clean; 609 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 3.7 MiB/s wr, 487 op/s 2024-09-10T09:56:47.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:56:47 smithi102 ceph-mon[35442]: pgmap v946: 129 pgs: 129 active+clean; 609 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 3.7 MiB/s wr, 487 op/s 2024-09-10T09:56:47.986 INFO:tasks.workunit.client.0.smithi005.stdout: 10 80716 15.41 MB/sec execute 146 sec latency 83.526 ms 2024-09-10T09:56:48.986 INFO:tasks.workunit.client.0.smithi005.stdout: 10 81442 15.60 MB/sec execute 147 sec latency 59.932 ms 2024-09-10T09:56:49.458 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:56:49 smithi005 ceph-mon[30350]: pgmap v947: 129 pgs: 129 active+clean; 614 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 3.9 MiB/s wr, 484 op/s 2024-09-10T09:56:49.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:56:49 smithi159 ceph-mon[35513]: pgmap v947: 129 pgs: 129 active+clean; 614 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 3.9 MiB/s wr, 484 op/s 2024-09-10T09:56:49.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:56:49 smithi102 ceph-mon[35442]: pgmap v947: 129 pgs: 129 active+clean; 614 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 3.9 MiB/s wr, 484 op/s 2024-09-10T09:56:49.986 INFO:tasks.workunit.client.0.smithi005.stdout: 10 81930 15.64 MB/sec execute 148 sec latency 89.755 ms 2024-09-10T09:56:50.987 INFO:tasks.workunit.client.0.smithi005.stdout: 10 82513 15.71 MB/sec execute 149 sec latency 91.227 ms 2024-09-10T09:56:51.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:56:51 smithi005 ceph-mon[30350]: pgmap v948: 129 pgs: 129 active+clean; 623 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 4.5 MiB/s wr, 583 op/s 2024-09-10T09:56:51.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:56:51 smithi159 ceph-mon[35513]: pgmap v948: 129 pgs: 129 active+clean; 623 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 4.5 MiB/s wr, 583 op/s 2024-09-10T09:56:51.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:56:51 smithi102 ceph-mon[35442]: pgmap v948: 129 pgs: 129 active+clean; 623 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 4.5 MiB/s wr, 583 op/s 2024-09-10T09:56:51.987 INFO:tasks.workunit.client.0.smithi005.stdout: 10 83279 15.88 MB/sec execute 150 sec latency 74.123 ms 2024-09-10T09:56:52.987 INFO:tasks.workunit.client.0.smithi005.stdout: 10 84143 16.03 MB/sec execute 151 sec latency 47.264 ms 2024-09-10T09:56:53.545 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:56:53 smithi005 ceph-mon[30350]: pgmap v949: 129 pgs: 129 active+clean; 626 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 3.4 MiB/s wr, 421 op/s 2024-09-10T09:56:53.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:56:53 smithi159 ceph-mon[35513]: pgmap v949: 129 pgs: 129 active+clean; 626 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 3.4 MiB/s wr, 421 op/s 2024-09-10T09:56:53.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:56:53 smithi102 ceph-mon[35442]: pgmap v949: 129 pgs: 129 active+clean; 626 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 3.4 MiB/s wr, 421 op/s 2024-09-10T09:56:53.987 INFO:tasks.workunit.client.0.smithi005.stdout: 10 84977 16.26 MB/sec execute 152 sec latency 43.649 ms 2024-09-10T09:56:54.987 INFO:tasks.workunit.client.0.smithi005.stdout: 10 85713 16.49 MB/sec execute 153 sec latency 102.959 ms 2024-09-10T09:56:55.532 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:56:55 smithi005 ceph-mon[30350]: pgmap v950: 129 pgs: 129 active+clean; 634 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 4.0 MiB/s wr, 497 op/s 2024-09-10T09:56:55.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:56:55 smithi159 ceph-mon[35513]: pgmap v950: 129 pgs: 129 active+clean; 634 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 4.0 MiB/s wr, 497 op/s 2024-09-10T09:56:55.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:56:55 smithi102 ceph-mon[35442]: pgmap v950: 129 pgs: 129 active+clean; 634 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 4.0 MiB/s wr, 497 op/s 2024-09-10T09:56:55.987 INFO:tasks.workunit.client.0.smithi005.stdout: 10 86237 16.56 MB/sec execute 154 sec latency 65.899 ms 2024-09-10T09:56:56.987 INFO:tasks.workunit.client.0.smithi005.stdout: 10 86807 16.60 MB/sec execute 155 sec latency 53.131 ms 2024-09-10T09:56:57.473 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:56:57 smithi005 ceph-mon[30350]: pgmap v951: 129 pgs: 129 active+clean; 623 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 4.0 MiB/s wr, 497 op/s 2024-09-10T09:56:57.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:56:57 smithi159 ceph-mon[35513]: pgmap v951: 129 pgs: 129 active+clean; 623 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 4.0 MiB/s wr, 497 op/s 2024-09-10T09:56:57.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:56:57 smithi102 ceph-mon[35442]: pgmap v951: 129 pgs: 129 active+clean; 623 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 4.0 MiB/s wr, 497 op/s 2024-09-10T09:56:57.988 INFO:tasks.workunit.client.0.smithi005.stdout: 10 87506 16.73 MB/sec execute 156 sec latency 81.202 ms 2024-09-10T09:56:58.988 INFO:tasks.workunit.client.0.smithi005.stdout: 10 87520 16.62 MB/sec execute 157 sec latency 959.064 ms 2024-09-10T09:56:59.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:56:59 smithi005 ceph-mon[30350]: pgmap v952: 129 pgs: 129 active+clean; 622 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 3.9 MiB/s wr, 485 op/s 2024-09-10T09:56:59.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:56:59 smithi159 ceph-mon[35513]: pgmap v952: 129 pgs: 129 active+clean; 622 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 3.9 MiB/s wr, 485 op/s 2024-09-10T09:56:59.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:56:59 smithi102 ceph-mon[35442]: pgmap v952: 129 pgs: 129 active+clean; 622 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 3.9 MiB/s wr, 485 op/s 2024-09-10T09:56:59.988 INFO:tasks.workunit.client.0.smithi005.stdout: 10 87520 16.52 MB/sec execute 158 sec latency 1959.173 ms 2024-09-10T09:57:00.988 INFO:tasks.workunit.client.0.smithi005.stdout: 10 87521 16.41 MB/sec execute 159 sec latency 2959.242 ms 2024-09-10T09:57:01.350 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:57:01 smithi005 ceph-mon[30350]: pgmap v953: 129 pgs: 129 active+clean; 652 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 6.6 MiB/s wr, 582 op/s 2024-09-10T09:57:01.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:57:01 smithi159 ceph-mon[35513]: pgmap v953: 129 pgs: 129 active+clean; 652 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 6.6 MiB/s wr, 582 op/s 2024-09-10T09:57:01.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:57:01 smithi102 ceph-mon[35442]: pgmap v953: 129 pgs: 129 active+clean; 652 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 6.6 MiB/s wr, 582 op/s 2024-09-10T09:57:01.702 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T09:57:01.988 INFO:tasks.workunit.client.0.smithi005.stdout: 10 87522 16.31 MB/sec execute 160 sec latency 3959.310 ms 2024-09-10T09:57:02.124 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:57:02.988 INFO:tasks.workunit.client.0.smithi005.stdout: 10 87526 16.21 MB/sec execute 161 sec latency 4804.988 ms 2024-09-10T09:57:03.423 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:57:03 smithi005 ceph-mon[30350]: pgmap v954: 129 pgs: 129 active+clean; 655 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 6.6 MiB/s wr, 468 op/s 2024-09-10T09:57:03.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:57:03 smithi159 ceph-mon[35513]: pgmap v954: 129 pgs: 129 active+clean; 655 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 6.6 MiB/s wr, 468 op/s 2024-09-10T09:57:03.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:57:03 smithi102 ceph-mon[35442]: pgmap v954: 129 pgs: 129 active+clean; 655 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 6.6 MiB/s wr, 468 op/s 2024-09-10T09:57:03.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:03.674+0000 7fbff8800640 1 -- 172.21.15.5:0/2014866137 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbff010ef30 msgr2=0x7fbff010d2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:57:03.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:03.674+0000 7fbff8800640 1 --2- 172.21.15.5:0/2014866137 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbff010ef30 0x7fbff010d2a0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fbfe00074e0 tx=0x7fbfe002b070 comp rx=0 tx=0).stop 2024-09-10T09:57:03.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:03.674+0000 7fbff8800640 1 -- 172.21.15.5:0/2014866137 shutdown_connections 2024-09-10T09:57:03.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:03.674+0000 7fbff8800640 1 --2- 172.21.15.5:0/2014866137 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbff010d9b0 0x7fbff0115510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:03.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:03.674+0000 7fbff8800640 1 --2- 172.21.15.5:0/2014866137 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbff010ef30 0x7fbff010d2a0 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:03.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:03.674+0000 7fbff8800640 1 --2- 172.21.15.5:0/2014866137 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbff010e580 0x7fbff010e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:03.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:03.674+0000 7fbff8800640 1 -- 172.21.15.5:0/2014866137 >> 172.21.15.5:0/2014866137 conn(0x7fbff006e140 msgr2=0x7fbff006e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:57:03.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:03.674+0000 7fbff8800640 1 -- 172.21.15.5:0/2014866137 shutdown_connections 2024-09-10T09:57:03.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:03.674+0000 7fbff8800640 1 -- 172.21.15.5:0/2014866137 wait complete. 2024-09-10T09:57:03.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:03.679+0000 7fbff8800640 1 Processor -- start 2024-09-10T09:57:03.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:03.679+0000 7fbff8800640 1 -- start start 2024-09-10T09:57:03.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:03.679+0000 7fbff8800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbff010d9b0 0x7fbff01a36e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:03.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:03.679+0000 7fbff8800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbff010e580 0x7fbff01a3c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:03.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:03.679+0000 7fbff8800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbff010ef30 0x7fbff01a7740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:03.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:03.684+0000 7fbff6800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbff010ef30 0x7fbff01a7740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:03.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:03.684+0000 7fbff5400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbff010e580 0x7fbff01a3c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:03.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:03.684+0000 7fbff5400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbff010e580 0x7fbff01a3c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:37776/0 (socket says 172.21.15.5:37776) 2024-09-10T09:57:03.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:03.685+0000 7fbff6800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbff010ef30 0x7fbff01a7740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:55800/0 (socket says 172.21.15.5:55800) 2024-09-10T09:57:03.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:03.685+0000 7fbff5e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbff010d9b0 0x7fbff01a36e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:03.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:03.685+0000 7fbff5e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbff010d9b0 0x7fbff01a36e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51098/0 (socket says 172.21.15.5:51098) 2024-09-10T09:57:03.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:03.685+0000 7fbff5400640 1 -- 172.21.15.5:0/2922577058 learned_addr learned my addr 172.21.15.5:0/2922577058 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:57:03.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:03.685+0000 7fbff8800640 1 -- 172.21.15.5:0/2922577058 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fbff0118040 con 0x7fbff010d9b0 2024-09-10T09:57:03.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:03.685+0000 7fbff8800640 1 -- 172.21.15.5:0/2922577058 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fbff0117ec0 con 0x7fbff010ef30 2024-09-10T09:57:03.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:03.685+0000 7fbff8800640 1 -- 172.21.15.5:0/2922577058 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fbff01181c0 con 0x7fbff010e580 2024-09-10T09:57:03.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:03.685+0000 7fbff5e00640 1 -- 172.21.15.5:0/2922577058 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbff010e580 msgr2=0x7fbff01a3c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:57:03.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:03.685+0000 7fbff5e00640 1 --2- 172.21.15.5:0/2922577058 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbff010e580 0x7fbff01a3c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:03.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:03.685+0000 7fbff5e00640 1 -- 172.21.15.5:0/2922577058 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbff010ef30 msgr2=0x7fbff01a7740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:57:03.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:03.685+0000 7fbff5e00640 1 --2- 172.21.15.5:0/2922577058 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbff010ef30 0x7fbff01a7740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:03.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:03.685+0000 7fbff5e00640 1 -- 172.21.15.5:0/2922577058 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbff01a7e20 con 0x7fbff010d9b0 2024-09-10T09:57:03.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:03.690+0000 7fbff5e00640 1 --2- 172.21.15.5:0/2922577058 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbff010d9b0 0x7fbff01a36e0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7fbfe400e9f0 tx=0x7fbfe400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:57:03.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:03.690+0000 7fbfeaa00640 1 -- 172.21.15.5:0/2922577058 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfe400ccc0 con 0x7fbff010d9b0 2024-09-10T09:57:03.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:03.690+0000 7fbfeaa00640 1 -- 172.21.15.5:0/2922577058 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbfe4010910 con 0x7fbff010d9b0 2024-09-10T09:57:03.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:03.690+0000 7fbff8800640 1 -- 172.21.15.5:0/2922577058 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbff01a8110 con 0x7fbff010d9b0 2024-09-10T09:57:03.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:03.690+0000 7fbff8800640 1 -- 172.21.15.5:0/2922577058 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbff01af9f0 con 0x7fbff010d9b0 2024-09-10T09:57:03.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:03.690+0000 7fbfeaa00640 1 -- 172.21.15.5:0/2922577058 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfe4018960 con 0x7fbff010d9b0 2024-09-10T09:57:03.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:03.700+0000 7fbffb3bb740 1 Processor -- start 2024-09-10T09:57:03.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:03.700+0000 7fbfeaa00640 1 -- 172.21.15.5:0/2922577058 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7fbfe4010ac0 con 0x7fbff010d9b0 2024-09-10T09:57:03.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:03.700+0000 7fbffb3bb740 1 -- start start 2024-09-10T09:57:03.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:03.706+0000 7fbffb3bb740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5571fec16b90 0x5571fec16f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:03.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:03.706+0000 7fbffb3bb740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5571fec174b0 0x5571fec21830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:03.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:03.706+0000 7fbffb3bb740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5571fec21d70 0x5571fec24160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:03.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:03.706+0000 7fbffb3bb740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5571feb7d3e0 con 0x5571fec16b90 2024-09-10T09:57:03.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:03.706+0000 7fbffb3bb740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x5571feb24d80 con 0x5571fec174b0 2024-09-10T09:57:03.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:03.706+0000 7fbffb3bb740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5571feb793a0 con 0x5571fec21d70 2024-09-10T09:57:03.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:03.706+0000 7fbff5e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5571fec174b0 0x5571fec21830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:03.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:03.709+0000 7fbff5e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5571fec174b0 0x5571fec21830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:55808/0 (socket says 172.21.15.5:55808) 2024-09-10T09:57:03.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:03.709+0000 7fbff5e00640 1 -- 172.21.15.5:0/3354138518 learned_addr learned my addr 172.21.15.5:0/3354138518 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:57:03.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:03.709+0000 7fbff5400640 1 --2- 172.21.15.5:0/3354138518 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5571fec21d70 0x5571fec24160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:03.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:03.709+0000 7fbff5e00640 1 -- 172.21.15.5:0/3354138518 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5571fec21d70 msgr2=0x5571fec24160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:57:03.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:03.709+0000 7fbff5e00640 1 --2- 172.21.15.5:0/3354138518 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5571fec21d70 0x5571fec24160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:03.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:03.709+0000 7fbff5e00640 1 -- 172.21.15.5:0/3354138518 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5571fec16b90 msgr2=0x5571fec16f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:57:03.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:03.709+0000 7fbff5e00640 1 --2- 172.21.15.5:0/3354138518 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5571fec16b90 0x5571fec16f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:03.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:03.709+0000 7fbff5e00640 1 -- 172.21.15.5:0/3354138518 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5571feb43ae0 con 0x5571fec174b0 2024-09-10T09:57:03.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:03.709+0000 7fbff5400640 1 --2- 172.21.15.5:0/3354138518 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5571fec21d70 0x5571fec24160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:57:03.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:03.709+0000 7fbfeaa00640 1 --2- 172.21.15.5:0/2922577058 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fbfd80a2c80 0x7fbfd80a5140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:03.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:03.709+0000 7fbff5e00640 1 --2- 172.21.15.5:0/3354138518 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5571fec174b0 0x5571fec21830 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fbfe40d5c20 tx=0x7fbfe40d5d10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bc4b070352499fd8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:57:03.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:03.709+0000 7fbfeaa00640 1 -- 172.21.15.5:0/2922577058 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fbfe40cf740 con 0x7fbff010d9b0 2024-09-10T09:57:03.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:03.709+0000 7fbff6800640 1 --2- 172.21.15.5:0/2922577058 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fbfd80a2c80 0x7fbfd80a5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:03.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:03.709+0000 7fbfdf000640 1 -- 172.21.15.5:0/3354138518 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfe4097ea0 con 0x5571fec174b0 2024-09-10T09:57:03.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:03.709+0000 7fbfdf000640 1 -- 172.21.15.5:0/3354138518 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbfe409bc00 con 0x5571fec174b0 2024-09-10T09:57:03.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:03.709+0000 7fbfdf000640 1 -- 172.21.15.5:0/3354138518 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfe409b560 con 0x5571fec174b0 2024-09-10T09:57:03.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:03.709+0000 7fbffb3bb740 1 -- 172.21.15.5:0/3354138518 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5571fec174b0 msgr2=0x5571fec21830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:57:03.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:03.709+0000 7fbffb3bb740 1 --2- 172.21.15.5:0/3354138518 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5571fec174b0 0x5571fec21830 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fbfe40d5c20 tx=0x7fbfe40d5d10 comp rx=0 tx=0).stop 2024-09-10T09:57:03.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:03.709+0000 7fbffb3bb740 1 -- 172.21.15.5:0/3354138518 shutdown_connections 2024-09-10T09:57:03.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:03.709+0000 7fbffb3bb740 1 --2- 172.21.15.5:0/3354138518 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5571fec21d70 0x5571fec24160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:03.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:03.709+0000 7fbff6800640 1 --2- 172.21.15.5:0/2922577058 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fbfd80a2c80 0x7fbfd80a5140 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fbfec005fd0 tx=0x7fbfec009b00 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:57:03.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:03.709+0000 7fbffb3bb740 1 --2- 172.21.15.5:0/3354138518 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5571fec174b0 0x5571fec21830 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:03.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:03.709+0000 7fbffb3bb740 1 --2- 172.21.15.5:0/3354138518 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5571fec16b90 0x5571fec16f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:03.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:03.709+0000 7fbffb3bb740 1 -- 172.21.15.5:0/3354138518 >> 172.21.15.5:0/3354138518 conn(0x5571fec13120 msgr2=0x5571fec13500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:57:03.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:03.709+0000 7fbffb3bb740 1 -- 172.21.15.5:0/3354138518 shutdown_connections 2024-09-10T09:57:03.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:03.709+0000 7fbffb3bb740 1 -- 172.21.15.5:0/3354138518 wait complete. 2024-09-10T09:57:03.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:03.714+0000 7fbffb3bb740 1 -- start start 2024-09-10T09:57:03.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:03.714+0000 7fbffb3bb740 1 Processor -- start 2024-09-10T09:57:03.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:03.714+0000 7fbffb3bb740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5571fec16b90 0x5571fec1ec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:03.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:03.714+0000 7fbffb3bb740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5571fec174b0 0x5571fec1f180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:03.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:03.714+0000 7fbffb3bb740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5571fec21d70 0x5571fec1f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:03.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:03.714+0000 7fbffb3bb740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5571fec2d1f0 con 0x5571fec16b90 2024-09-10T09:57:03.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:03.714+0000 7fbffb3bb740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x5571feb380f0 con 0x5571fec21d70 2024-09-10T09:57:03.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:03.714+0000 7fbffb3bb740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5571fec2d370 con 0x5571fec174b0 2024-09-10T09:57:03.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:03.714+0000 7fbffb3bb740 10 client.0 _ll_register_callbacks cb 0x5571feb299e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:57:03.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:03.714+0000 7fbff5400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5571fec174b0 0x5571fec1f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:03.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:03.714+0000 7fbff5400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5571fec174b0 0x5571fec1f180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:37792/0 (socket says 172.21.15.5:37792) 2024-09-10T09:57:03.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:03.714+0000 7fbff5400640 1 -- 172.21.15.5:0/1101438765 learned_addr learned my addr 172.21.15.5:0/1101438765 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:57:03.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:03.714+0000 7fbff6800640 1 --2- 172.21.15.5:0/1101438765 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5571fec21d70 0x5571fec1f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:03.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:03.719+0000 7fbff5e00640 1 --2- 172.21.15.5:0/1101438765 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5571fec16b90 0x5571fec1ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:03.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:03.719+0000 7fbff6800640 1 -- 172.21.15.5:0/1101438765 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5571fec174b0 msgr2=0x5571fec1f180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:57:03.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:03.719+0000 7fbff6800640 1 --2- 172.21.15.5:0/1101438765 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5571fec174b0 0x5571fec1f180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:03.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:03.719+0000 7fbff6800640 1 -- 172.21.15.5:0/1101438765 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5571fec16b90 msgr2=0x5571fec1ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:57:03.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:03.719+0000 7fbff6800640 1 --2- 172.21.15.5:0/1101438765 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5571fec16b90 0x5571fec1ec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:03.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:03.719+0000 7fbff6800640 1 -- 172.21.15.5:0/1101438765 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5571fe8ee3c0 con 0x5571fec21d70 2024-09-10T09:57:03.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:03.721+0000 7fbff6800640 1 --2- 172.21.15.5:0/1101438765 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5571fec21d70 0x5571fec1f6c0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fbfec0049d0 tx=0x7fbfec0096a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1688d19686cd752 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:57:03.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:03.721+0000 7fbfd3400640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-09-10T09:57:03.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:03.721+0000 7fbfd3400640 1 -- 172.21.15.5:0/1101438765 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfec01e830 con 0x5571fec21d70 2024-09-10T09:57:03.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:03.721+0000 7fbfd3400640 1 -- 172.21.15.5:0/1101438765 <== mon.2 v2:172.21.15.102:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fbfec01e9d0 con 0x5571fec21d70 2024-09-10T09:57:03.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:03.721+0000 7fbffb3bb740 1 -- 172.21.15.5:0/1101438765 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbfe409bc00 con 0x5571fec21d70 2024-09-10T09:57:03.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:03.721+0000 7fbfd3400640 1 -- 172.21.15.5:0/1101438765 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7fbfec01ebd0 con 0x5571fec21d70 2024-09-10T09:57:03.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:03.721+0000 7fbffb3bb740 10 client.29018 fetch_fsmap learned FSMap version 42 2024-09-10T09:57:03.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:03.721+0000 7fbffb3bb740 1 -- 172.21.15.5:0/1101438765 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x5571fec38f10 con 0x5571fec21d70 2024-09-10T09:57:03.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:03.721+0000 7fbfd3400640 1 -- 172.21.15.5:0/1101438765 <== mon.2 v2:172.21.15.102:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbfe409bc00 con 0x5571fec21d70 2024-09-10T09:57:03.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:03.721+0000 7fbfd3400640 1 -- 172.21.15.5:0/1101438765 <== mon.2 v2:172.21.15.102:3300/0 5 ==== fsmap(e 42) ==== 8370+0+0 (secure 0 0 0) 0x7fbfec026400 con 0x5571fec21d70 2024-09-10T09:57:03.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:03.721+0000 7fbffb3bb740 10 client.29018 fetch_fsmap finished waiting for FSMap version 42 2024-09-10T09:57:03.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:03.721+0000 7fbffb3bb740 10 client.29018 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T09:57:03.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:03.721+0000 7fbffb3bb740 20 client.29018 populate_metadata read hostname 'smithi005' 2024-09-10T09:57:03.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:03.721+0000 7fbffb3bb740 1 --2- 172.21.15.5:0/1101438765 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x5571fec3a320 0x5571fec3c730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:03.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:03.721+0000 7fbffb3bb740 4 client.29018 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:57:03.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:03.721+0000 7fbffb3bb740 1 -- 172.21.15.5:0/1101438765 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5571feb8a950 con 0x5571fec3a320 2024-09-10T09:57:03.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:03.726+0000 7fbff5e00640 1 --2- 172.21.15.5:0/1101438765 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x5571fec3a320 0x5571fec3c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:03.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:03.726+0000 7fbff5e00640 1 --2- 172.21.15.5:0/1101438765 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x5571fec3a320 0x5571fec3c730 crc :-1 s=READY pgs=263 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=443912966d036def server_cookie=343862fef2f5d393 in_seq=0 out_seq=0 2024-09-10T09:57:03.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:03.726+0000 7fbfd3400640 10 client.29018 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T09:57:03.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:03.731+0000 7fbfd3400640 1 -- 172.21.15.5:0/1101438765 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5571feb8a950 con 0x5571fec3a320 2024-09-10T09:57:03.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:03.731+0000 7fbfd3400640 10 client.29018 handle_command_reply: tid=0 2024-09-10T09:57:03.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:03.731+0000 7fbffb3bb740 1 client.29018 shutdown 2024-09-10T09:57:03.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:03.733+0000 7fbffb3bb740 1 -- 172.21.15.5:0/1101438765 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5571fec21d70 msgr2=0x5571fec1f6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:57:03.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:03.733+0000 7fbffb3bb740 1 --2- 172.21.15.5:0/1101438765 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5571fec21d70 0x5571fec1f6c0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fbfec0049d0 tx=0x7fbfec0096a0 comp rx=0 tx=0).stop 2024-09-10T09:57:03.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:03.733+0000 7fbffb3bb740 1 -- 172.21.15.5:0/1101438765 shutdown_connections 2024-09-10T09:57:03.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:03.733+0000 7fbffb3bb740 1 --2- 172.21.15.5:0/1101438765 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x5571fec3a320 0x5571fec3c730 crc :-1 s=READY pgs=263 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:03.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:03.733+0000 7fbffb3bb740 1 --2- 172.21.15.5:0/1101438765 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5571fec21d70 0x5571fec1f6c0 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:03.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:03.733+0000 7fbffb3bb740 1 --2- 172.21.15.5:0/1101438765 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5571fec174b0 0x5571fec1f180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:03.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:03.733+0000 7fbfd3400640 0 client.29018 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T09:57:03.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:03.733+0000 7fbffb3bb740 1 --2- 172.21.15.5:0/1101438765 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5571fec16b90 0x5571fec1ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:03.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:03.733+0000 7fbffb3bb740 1 -- 172.21.15.5:0/1101438765 >> 172.21.15.5:0/1101438765 conn(0x5571fec13120 msgr2=0x5571fec18e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:57:03.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:03.733+0000 7fbffb3bb740 1 -- 172.21.15.5:0/1101438765 shutdown_connections 2024-09-10T09:57:03.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:03.733+0000 7fbffb3bb740 1 -- 172.21.15.5:0/1101438765 wait complete. 2024-09-10T09:57:03.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:03.733+0000 7fbffb3bb740 20 client.29018 trim_cache size 0 max 16384 2024-09-10T09:57:03.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:03.769+0000 7fbffb3bb740 1 Processor -- start 2024-09-10T09:57:03.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:03.769+0000 7fbffb3bb740 1 -- start start 2024-09-10T09:57:03.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:03.769+0000 7fbffb3bb740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5571fec13120 0x5571fec19640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:03.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:03.769+0000 7fbffb3bb740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5571fec16b90 0x5571fec1f510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:03.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:03.769+0000 7fbffb3bb740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5571fec174b0 0x5571fec42000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:03.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:03.769+0000 7fbffb3bb740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fbff01b55a0 con 0x5571fec13120 2024-09-10T09:57:03.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:03.769+0000 7fbffb3bb740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fbff01b70e0 con 0x5571fec16b90 2024-09-10T09:57:03.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:03.769+0000 7fbffb3bb740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fbff01b5c70 con 0x5571fec174b0 2024-09-10T09:57:03.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:03.769+0000 7fbff6800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5571fec174b0 0x5571fec42000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:03.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:03.769+0000 7fbff6800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5571fec174b0 0x5571fec42000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:37808/0 (socket says 172.21.15.5:37808) 2024-09-10T09:57:03.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:03.769+0000 7fbff6800640 1 -- 172.21.15.5:0/2373992899 learned_addr learned my addr 172.21.15.5:0/2373992899 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:57:03.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:03.769+0000 7fbff5400640 1 --2- 172.21.15.5:0/2373992899 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5571fec16b90 0x5571fec1f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:03.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:03.769+0000 7fbff5e00640 1 --2- 172.21.15.5:0/2373992899 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5571fec13120 0x5571fec19640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:03.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:03.775+0000 7fbff6800640 1 -- 172.21.15.5:0/2373992899 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5571fec16b90 msgr2=0x5571fec1f510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:57:03.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:03.775+0000 7fbff6800640 1 --2- 172.21.15.5:0/2373992899 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5571fec16b90 0x5571fec1f510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:03.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:03.775+0000 7fbff6800640 1 -- 172.21.15.5:0/2373992899 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5571fec13120 msgr2=0x5571fec19640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:57:03.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:03.775+0000 7fbff6800640 1 --2- 172.21.15.5:0/2373992899 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5571fec13120 0x5571fec19640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:03.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:03.775+0000 7fbff6800640 1 -- 172.21.15.5:0/2373992899 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5571fec3cdb0 con 0x5571fec174b0 2024-09-10T09:57:03.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:03.775+0000 7fbff5e00640 1 --2- 172.21.15.5:0/2373992899 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5571fec13120 0x5571fec19640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T09:57:03.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:03.775+0000 7fbff6800640 1 --2- 172.21.15.5:0/2373992899 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5571fec174b0 0x5571fec42000 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fbfec0267d0 tx=0x7fbfec005b70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1c7628c1b1036ced server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:57:03.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:03.780+0000 7fbfde600640 1 -- 172.21.15.5:0/2373992899 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfec01ebd0 con 0x5571fec174b0 2024-09-10T09:57:03.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:03.780+0000 7fbfde600640 1 -- 172.21.15.5:0/2373992899 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbfec037040 con 0x5571fec174b0 2024-09-10T09:57:03.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:03.780+0000 7fbfde600640 1 -- 172.21.15.5:0/2373992899 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfec03b900 con 0x5571fec174b0 2024-09-10T09:57:03.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:03.780+0000 7fbffb3bb740 1 -- 172.21.15.5:0/2373992899 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5571fec174b0 msgr2=0x5571fec42000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:57:03.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:03.780+0000 7fbffb3bb740 1 --2- 172.21.15.5:0/2373992899 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5571fec174b0 0x5571fec42000 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fbfec0267d0 tx=0x7fbfec005b70 comp rx=0 tx=0).stop 2024-09-10T09:57:03.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:03.780+0000 7fbffb3bb740 1 -- 172.21.15.5:0/2373992899 shutdown_connections 2024-09-10T09:57:03.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:03.780+0000 7fbffb3bb740 1 --2- 172.21.15.5:0/2373992899 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5571fec174b0 0x5571fec42000 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:03.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:03.780+0000 7fbffb3bb740 1 --2- 172.21.15.5:0/2373992899 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5571fec16b90 0x5571fec1f510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:03.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:03.780+0000 7fbffb3bb740 1 --2- 172.21.15.5:0/2373992899 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5571fec13120 0x5571fec19640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:03.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:03.780+0000 7fbffb3bb740 1 -- 172.21.15.5:0/2373992899 >> 172.21.15.5:0/2373992899 conn(0x5571fec1a630 msgr2=0x5571fec20cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:57:03.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:03.780+0000 7fbffb3bb740 1 -- 172.21.15.5:0/2373992899 shutdown_connections 2024-09-10T09:57:03.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:03.780+0000 7fbffb3bb740 1 -- 172.21.15.5:0/2373992899 wait complete. 2024-09-10T09:57:03.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:03.780+0000 7fbffb3bb740 1 -- start start 2024-09-10T09:57:03.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:03.780+0000 7fbffb3bb740 1 Processor -- start 2024-09-10T09:57:03.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:03.785+0000 7fbffb3bb740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5571fec13120 0x5571fec2e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:03.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:03.785+0000 7fbffb3bb740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5571fec16b90 0x5571fec2e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:03.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:03.785+0000 7fbffb3bb740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5571fec174b0 0x5571fec2ee20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:03.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:03.785+0000 7fbff5e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5571fec13120 0x5571fec2e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:03.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:03.785+0000 7fbff5400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5571fec16b90 0x5571fec2e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:03.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:03.785+0000 7fbff5400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5571fec16b90 0x5571fec2e8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:55844/0 (socket says 172.21.15.5:55844) 2024-09-10T09:57:03.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:03.785+0000 7fbff5e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5571fec13120 0x5571fec2e3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51126/0 (socket says 172.21.15.5:51126) 2024-09-10T09:57:03.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:03.785+0000 7fbff6800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5571fec174b0 0x5571fec2ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:03.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:03.785+0000 7fbff6800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5571fec174b0 0x5571fec2ee20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:37822/0 (socket says 172.21.15.5:37822) 2024-09-10T09:57:03.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:03.785+0000 7fbffb3bb740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fbff01b6a10 con 0x5571fec13120 2024-09-10T09:57:03.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:03.785+0000 7fbffb3bb740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fbff01b6340 con 0x5571fec16b90 2024-09-10T09:57:03.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:03.785+0000 7fbffb3bb740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fbff01b3560 con 0x5571fec174b0 2024-09-10T09:57:03.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:03.785+0000 7fbffb3bb740 10 client.0 _ll_register_callbacks cb 0x7fbfec004680 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:57:03.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:03.785+0000 7fbff5400640 1 -- 172.21.15.5:0/1726542591 learned_addr learned my addr 172.21.15.5:0/1726542591 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:57:03.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:03.785+0000 7fbff5400640 1 -- 172.21.15.5:0/1726542591 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5571fec174b0 msgr2=0x5571fec2ee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:57:03.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:03.785+0000 7fbff5400640 1 --2- 172.21.15.5:0/1726542591 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5571fec174b0 0x5571fec2ee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:03.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:03.785+0000 7fbff5400640 1 -- 172.21.15.5:0/1726542591 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5571fec13120 msgr2=0x5571fec2e3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:57:03.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:03.790+0000 7fbff5400640 1 --2- 172.21.15.5:0/1726542591 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5571fec13120 0x5571fec2e3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:03.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:03.790+0000 7fbff5400640 1 -- 172.21.15.5:0/1726542591 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5571feca4020 con 0x5571fec16b90 2024-09-10T09:57:03.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:03.790+0000 7fbff5e00640 1 --2- 172.21.15.5:0/1726542591 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5571fec13120 0x5571fec2e3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:57:03.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:03.792+0000 7fbff5400640 1 --2- 172.21.15.5:0/1726542591 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5571fec16b90 0x5571fec2e8e0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fbfe0000c00 tx=0x7fbfe0004cb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c37c88c4bbff6f4f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:57:03.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:03.792+0000 7fbfd3400640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-09-10T09:57:03.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:03.792+0000 7fbfd3400640 1 -- 172.21.15.5:0/1726542591 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfe0007c60 con 0x5571fec16b90 2024-09-10T09:57:03.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:03.792+0000 7fbfd3400640 1 -- 172.21.15.5:0/1726542591 <== mon.2 v2:172.21.15.102:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fbfe0007e00 con 0x5571fec16b90 2024-09-10T09:57:03.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:03.792+0000 7fbffb3bb740 1 -- 172.21.15.5:0/1726542591 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbfec037040 con 0x5571fec16b90 2024-09-10T09:57:03.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:03.798+0000 7fbfd3400640 1 -- 172.21.15.5:0/1726542591 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7fbfe0004030 con 0x5571fec16b90 2024-09-10T09:57:03.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:03.798+0000 7fbffb3bb740 10 client.29024 fetch_fsmap learned FSMap version 42 2024-09-10T09:57:03.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:03.798+0000 7fbffb3bb740 1 -- 172.21.15.5:0/1726542591 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x5571feca52b0 con 0x5571fec16b90 2024-09-10T09:57:03.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:03.798+0000 7fbfd3400640 1 -- 172.21.15.5:0/1726542591 <== mon.2 v2:172.21.15.102:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbfec037040 con 0x5571fec16b90 2024-09-10T09:57:03.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:03.798+0000 7fbfd3400640 1 -- 172.21.15.5:0/1726542591 <== mon.2 v2:172.21.15.102:3300/0 5 ==== fsmap(e 42) ==== 8370+0+0 (secure 0 0 0) 0x7fbfe0050020 con 0x5571fec16b90 2024-09-10T09:57:03.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:03.798+0000 7fbffb3bb740 10 client.29024 fetch_fsmap finished waiting for FSMap version 42 2024-09-10T09:57:03.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:03.798+0000 7fbffb3bb740 10 client.29024 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T09:57:03.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:03.798+0000 7fbffb3bb740 20 client.29024 populate_metadata read hostname 'smithi005' 2024-09-10T09:57:03.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:03.798+0000 7fbffb3bb740 1 --2- 172.21.15.5:0/1726542591 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x5571fec21d70 0x5571feca7a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:03.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:03.798+0000 7fbffb3bb740 4 client.29024 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T09:57:03.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:03.798+0000 7fbffb3bb740 1 -- 172.21.15.5:0/1726542591 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5571feb84e60 con 0x5571fec21d70 2024-09-10T09:57:03.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:03.798+0000 7fbff5e00640 1 --2- 172.21.15.5:0/1726542591 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x5571fec21d70 0x5571feca7a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:03.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:03.798+0000 7fbff5e00640 1 --2- 172.21.15.5:0/1726542591 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x5571fec21d70 0x5571feca7a40 crc :-1 s=READY pgs=264 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=44b6eab676fc8956 server_cookie=62398308d2aee6f4 in_seq=0 out_seq=0 2024-09-10T09:57:03.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:03.798+0000 7fbfd3400640 10 client.29024 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T09:57:03.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:03.798+0000 7fbfd3400640 1 -- 172.21.15.5:0/1726542591 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5571feb84e60 con 0x5571fec21d70 2024-09-10T09:57:03.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:03.798+0000 7fbfd3400640 10 client.29024 handle_command_reply: tid=0 2024-09-10T09:57:03.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:03.798+0000 7fbffb3bb740 1 client.29024 shutdown 2024-09-10T09:57:03.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:03.798+0000 7fbffb3bb740 1 -- 172.21.15.5:0/1726542591 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5571fec16b90 msgr2=0x5571fec2e8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:57:03.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:03.798+0000 7fbffb3bb740 1 --2- 172.21.15.5:0/1726542591 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5571fec16b90 0x5571fec2e8e0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fbfe0000c00 tx=0x7fbfe0004cb0 comp rx=0 tx=0).stop 2024-09-10T09:57:03.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:03.798+0000 7fbffb3bb740 1 -- 172.21.15.5:0/1726542591 shutdown_connections 2024-09-10T09:57:03.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:03.798+0000 7fbffb3bb740 1 --2- 172.21.15.5:0/1726542591 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x5571fec21d70 0x5571feca7a40 crc :-1 s=READY pgs=264 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:03.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:03.803+0000 7fbffb3bb740 1 --2- 172.21.15.5:0/1726542591 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5571fec174b0 0x5571fec2ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:03.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:03.803+0000 7fbfd3400640 0 client.29024 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T09:57:03.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:03.803+0000 7fbffb3bb740 1 --2- 172.21.15.5:0/1726542591 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5571fec16b90 0x5571fec2e8e0 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:03.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:03.803+0000 7fbffb3bb740 1 --2- 172.21.15.5:0/1726542591 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5571fec13120 0x5571fec2e3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:03.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:03.803+0000 7fbffb3bb740 1 -- 172.21.15.5:0/1726542591 >> 172.21.15.5:0/1726542591 conn(0x5571fec1a630 msgr2=0x5571fec119a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:57:03.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:03.803+0000 7fbffb3bb740 1 -- 172.21.15.5:0/1726542591 shutdown_connections 2024-09-10T09:57:03.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:03.803+0000 7fbffb3bb740 1 -- 172.21.15.5:0/1726542591 wait complete. 2024-09-10T09:57:03.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:03.803+0000 7fbffb3bb740 20 client.29024 trim_cache size 0 max 16384 2024-09-10T09:57:03.807 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-10T09:57:03.807 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "no active scrubs running", 2024-09-10T09:57:03.807 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": {} 2024-09-10T09:57:03.807 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-10T09:57:03.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:03.803+0000 7fbff8800640 1 -- 172.21.15.5:0/2922577058 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fbfd80a2c80 msgr2=0x7fbfd80a5140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:57:03.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:03.803+0000 7fbff8800640 1 --2- 172.21.15.5:0/2922577058 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fbfd80a2c80 0x7fbfd80a5140 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fbfec005fd0 tx=0x7fbfec009b00 comp rx=0 tx=0).stop 2024-09-10T09:57:03.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:03.803+0000 7fbff8800640 1 -- 172.21.15.5:0/2922577058 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbff010d9b0 msgr2=0x7fbff01a36e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:57:03.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:03.803+0000 7fbff8800640 1 --2- 172.21.15.5:0/2922577058 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbff010d9b0 0x7fbff01a36e0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7fbfe400e9f0 tx=0x7fbfe400eeb0 comp rx=0 tx=0).stop 2024-09-10T09:57:03.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:03.803+0000 7fbff8800640 1 -- 172.21.15.5:0/2922577058 shutdown_connections 2024-09-10T09:57:03.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:03.803+0000 7fbff8800640 1 --2- 172.21.15.5:0/2922577058 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fbfd80a2c80 0x7fbfd80a5140 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:03.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:03.803+0000 7fbff8800640 1 --2- 172.21.15.5:0/2922577058 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbff010ef30 0x7fbff01a7740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:03.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:03.803+0000 7fbff8800640 1 --2- 172.21.15.5:0/2922577058 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbff010e580 0x7fbff01a3c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:03.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:03.803+0000 7fbff8800640 1 --2- 172.21.15.5:0/2922577058 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbff010d9b0 0x7fbff01a36e0 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:03.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:03.803+0000 7fbff8800640 1 -- 172.21.15.5:0/2922577058 >> 172.21.15.5:0/2922577058 conn(0x7fbff006e140 msgr2=0x7fbff00712b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:57:03.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:03.803+0000 7fbff8800640 1 -- 172.21.15.5:0/2922577058 shutdown_connections 2024-09-10T09:57:03.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:03.803+0000 7fbff8800640 1 -- 172.21.15.5:0/2922577058 wait complete. 2024-09-10T09:57:03.989 INFO:tasks.workunit.client.0.smithi005.stdout: 10 87526 16.11 MB/sec execute 162 sec latency 5805.468 ms 2024-09-10T09:57:04.989 INFO:tasks.workunit.client.0.smithi005.stdout: 10 87526 16.01 MB/sec execute 163 sec latency 6805.538 ms 2024-09-10T09:57:05.239 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-10T09:57:05.239 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T09:57:05.557 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:57:05 smithi005 ceph-mon[30350]: pgmap v955: 129 pgs: 129 active+clean; 667 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 8.0 MiB/s wr, 485 op/s 2024-09-10T09:57:05.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:57:05 smithi159 ceph-mon[35513]: pgmap v955: 129 pgs: 129 active+clean; 667 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 8.0 MiB/s wr, 485 op/s 2024-09-10T09:57:05.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:57:05 smithi102 ceph-mon[35442]: pgmap v955: 129 pgs: 129 active+clean; 667 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 8.0 MiB/s wr, 485 op/s 2024-09-10T09:57:05.682 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:57:05.989 INFO:tasks.workunit.client.0.smithi005.stdout: 10 87528 15.91 MB/sec execute 164 sec latency 7805.621 ms 2024-09-10T09:57:06.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:06.328+0000 7fe6bf600640 1 -- 172.21.15.5:0/2640617945 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe6b8112a30 msgr2=0x7fe6b8114e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:57:06.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:06.328+0000 7fe6bf600640 1 --2- 172.21.15.5:0/2640617945 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe6b8112a30 0x7fe6b8114e20 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fe6b0009ca0 tx=0x7fe6b002a8c0 comp rx=0 tx=0).stop 2024-09-10T09:57:06.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:06.328+0000 7fe6bf600640 1 -- 172.21.15.5:0/2640617945 shutdown_connections 2024-09-10T09:57:06.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:06.328+0000 7fe6bf600640 1 --2- 172.21.15.5:0/2640617945 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe6b8112a30 0x7fe6b8114e20 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:06.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:06.328+0000 7fe6bf600640 1 --2- 172.21.15.5:0/2640617945 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe6b8077040 0x7fe6b81124f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:06.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:06.328+0000 7fe6bf600640 1 --2- 172.21.15.5:0/2640617945 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe6b8076720 0x7fe6b8076b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:06.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:06.328+0000 7fe6bf600640 1 -- 172.21.15.5:0/2640617945 >> 172.21.15.5:0/2640617945 conn(0x7fe6b80ffa90 msgr2=0x7fe6b8101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:57:06.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:06.328+0000 7fe6bf600640 1 -- 172.21.15.5:0/2640617945 shutdown_connections 2024-09-10T09:57:06.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:06.328+0000 7fe6bf600640 1 -- 172.21.15.5:0/2640617945 wait complete. 2024-09-10T09:57:06.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:06.328+0000 7fe6bf600640 1 Processor -- start 2024-09-10T09:57:06.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:06.328+0000 7fe6bf600640 1 -- start start 2024-09-10T09:57:06.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:06.328+0000 7fe6bf600640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe6b8076720 0x7fe6b81a4710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:06.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:06.328+0000 7fe6bf600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe6b8077040 0x7fe6b81a4c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:06.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:06.328+0000 7fe6bf600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe6b8112a30 0x7fe6b819e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:06.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:06.328+0000 7fe6bf600640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe6b8118590 con 0x7fe6b8077040 2024-09-10T09:57:06.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:06.328+0000 7fe6bf600640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fe6b8118410 con 0x7fe6b8112a30 2024-09-10T09:57:06.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:06.328+0000 7fe6bf600640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fe6b8118710 con 0x7fe6b8076720 2024-09-10T09:57:06.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:06.328+0000 7fe6b7e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe6b8077040 0x7fe6b81a4c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:06.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:06.328+0000 7fe6bcc00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe6b8076720 0x7fe6b81a4710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:06.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:06.328+0000 7fe6bcc00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe6b8076720 0x7fe6b81a4710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:37842/0 (socket says 172.21.15.5:37842) 2024-09-10T09:57:06.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:06.328+0000 7fe6bcc00640 1 -- 172.21.15.5:0/3895649129 learned_addr learned my addr 172.21.15.5:0/3895649129 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:57:06.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:06.328+0000 7fe6bd600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe6b8112a30 0x7fe6b819e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:06.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:06.328+0000 7fe6bcc00640 1 -- 172.21.15.5:0/3895649129 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe6b8112a30 msgr2=0x7fe6b819e8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:57:06.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:06.328+0000 7fe6bcc00640 1 --2- 172.21.15.5:0/3895649129 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe6b8112a30 0x7fe6b819e8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:06.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:06.328+0000 7fe6bcc00640 1 -- 172.21.15.5:0/3895649129 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe6b8077040 msgr2=0x7fe6b81a4c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:57:06.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:06.328+0000 7fe6bcc00640 1 --2- 172.21.15.5:0/3895649129 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe6b8077040 0x7fe6b81a4c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:06.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:06.328+0000 7fe6bcc00640 1 -- 172.21.15.5:0/3895649129 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6b819f000 con 0x7fe6b8076720 2024-09-10T09:57:06.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:06.333+0000 7fe6bcc00640 1 --2- 172.21.15.5:0/3895649129 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe6b8076720 0x7fe6b81a4710 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fe6ac00e9f0 tx=0x7fe6ac00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:57:06.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:06.333+0000 7fe6b5600640 1 -- 172.21.15.5:0/3895649129 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6ac00cd80 con 0x7fe6b8076720 2024-09-10T09:57:06.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:06.333+0000 7fe6b5600640 1 -- 172.21.15.5:0/3895649129 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe6ac010910 con 0x7fe6b8076720 2024-09-10T09:57:06.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:06.333+0000 7fe6b5600640 1 -- 172.21.15.5:0/3895649129 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6ac018990 con 0x7fe6b8076720 2024-09-10T09:57:06.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:06.333+0000 7fe6bf600640 1 -- 172.21.15.5:0/3895649129 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6b819f2f0 con 0x7fe6b8076720 2024-09-10T09:57:06.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:06.333+0000 7fe6bf600640 1 -- 172.21.15.5:0/3895649129 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6b81ab5e0 con 0x7fe6b8076720 2024-09-10T09:57:06.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:06.333+0000 7fe6bf600640 1 -- 172.21.15.5:0/3895649129 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe6b8076b00 con 0x7fe6b8076720 2024-09-10T09:57:06.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:06.333+0000 7fe6b5600640 1 -- 172.21.15.5:0/3895649129 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7fe6ac010ac0 con 0x7fe6b8076720 2024-09-10T09:57:06.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:06.337+0000 7fe6b5600640 1 --2- 172.21.15.5:0/3895649129 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fe6a00a2c00 0x7fe6a00a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:06.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:06.337+0000 7fe6b5600640 1 -- 172.21.15.5:0/3895649129 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fe6ac0cfcb0 con 0x7fe6b8076720 2024-09-10T09:57:06.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:06.337+0000 7fe6b7e00640 1 --2- 172.21.15.5:0/3895649129 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fe6a00a2c00 0x7fe6a00a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:06.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:06.337+0000 7fe6b7e00640 1 --2- 172.21.15.5:0/3895649129 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fe6a00a2c00 0x7fe6a00a50c0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fe6b80793e0 tx=0x7fe6a4005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:57:06.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:06.337+0000 7fe6b5600640 1 -- 172.21.15.5:0/3895649129 <== mon.1 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe6ac09c170 con 0x7fe6b8076720 2024-09-10T09:57:06.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:06.567+0000 7fe6bf600640 1 -- 172.21.15.5:0/3895649129 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe6b819fe00 con 0x7fe6b8076720 2024-09-10T09:57:06.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:06.568+0000 7fe6b5600640 1 -- 172.21.15.5:0/3895649129 <== mon.1 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 42 v42) ==== 94+0+11969 (secure 0 0 0) 0x7fe6ac0a0470 con 0x7fe6b8076720 2024-09-10T09:57:06.569 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-10T09:57:06.569 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":42,"btime":"2024-09-10T09:55:06:670916+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14454,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/2970594311","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2970594311},{"type":"v1","addr":"172.21.15.5:6835","nonce":2970594311}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14514,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6839/3616502174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":3616502174},{"type":"v1","addr":"172.21.15.5:6839","nonce":3616502174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":42,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T09:35:29.626287+0000","modified":"2024-09-10T09:55:06.419829+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24542,"mds_1":24535,"mds_2":24511,"mds_3":24506,"mds_4":24443},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14520":{"gid":14520,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/2073599842","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":2073599842},{"type":"v1","addr":"172.21.15.5:6841","nonce":2073599842}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"k","rank":4,"incarnation":26,"state":"up:active","state_seq":24,"addr":"172.21.15.102:6833/2457007427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":2457007427},{"type":"v1","addr":"172.21.15.102:6833","nonce":2457007427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/356445159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":356445159},{"type":"v1","addr":"172.21.15.159:6833","nonce":356445159}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6837/2894314507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":2894314507},{"type":"v1","addr":"172.21.15.5:6837","nonce":2894314507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.102:6835/1647549111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":1647549111},{"type":"v1","addr":"172.21.15.102:6835","nonce":1647549111}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6835/952657242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":952657242},{"type":"v1","addr":"172.21.15.159:6835","nonce":952657242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":19,"addr":"172.21.15.102:6837/2689776764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":2689776764},{"type":"v1","addr":"172.21.15.102:6837","nonce":2689776764}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.159:6837/620627639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":620627639},{"type":"v1","addr":"172.21.15.159:6837","nonce":620627639}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24535":{"gid":24535,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":18,"addr":"172.21.15.159:6839/2073739599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":2073739599},{"type":"v1","addr":"172.21.15.159:6839","nonce":2073739599}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24542":{"gid":24542,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.102:6839/335409749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6838","nonce":335409749},{"type":"v1","addr":"172.21.15.102:6839","nonce":335409749}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24542,"qdb_cluster":[24542,24535,24511,24506,24443]},"id":1}]} 2024-09-10T09:57:06.570 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 42 2024-09-10T09:57:06.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:06.570+0000 7fe6bf600640 1 -- 172.21.15.5:0/3895649129 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fe6a00a2c00 msgr2=0x7fe6a00a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:57:06.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:06.570+0000 7fe6bf600640 1 --2- 172.21.15.5:0/3895649129 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fe6a00a2c00 0x7fe6a00a50c0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fe6b80793e0 tx=0x7fe6a4005f90 comp rx=0 tx=0).stop 2024-09-10T09:57:06.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:06.570+0000 7fe6bf600640 1 -- 172.21.15.5:0/3895649129 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe6b8076720 msgr2=0x7fe6b81a4710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:57:06.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:06.570+0000 7fe6bf600640 1 --2- 172.21.15.5:0/3895649129 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe6b8076720 0x7fe6b81a4710 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fe6ac00e9f0 tx=0x7fe6ac00eeb0 comp rx=0 tx=0).stop 2024-09-10T09:57:06.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:06.570+0000 7fe6bf600640 1 -- 172.21.15.5:0/3895649129 shutdown_connections 2024-09-10T09:57:06.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:06.570+0000 7fe6bf600640 1 --2- 172.21.15.5:0/3895649129 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fe6a00a2c00 0x7fe6a00a50c0 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:06.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:06.570+0000 7fe6bf600640 1 --2- 172.21.15.5:0/3895649129 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe6b8112a30 0x7fe6b819e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:06.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:06.570+0000 7fe6bf600640 1 --2- 172.21.15.5:0/3895649129 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe6b8077040 0x7fe6b81a4c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:06.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:06.570+0000 7fe6bf600640 1 --2- 172.21.15.5:0/3895649129 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe6b8076720 0x7fe6b81a4710 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:06.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:06.570+0000 7fe6bf600640 1 -- 172.21.15.5:0/3895649129 >> 172.21.15.5:0/3895649129 conn(0x7fe6b80ffa90 msgr2=0x7fe6b8101e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:57:06.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:06.570+0000 7fe6bf600640 1 -- 172.21.15.5:0/3895649129 shutdown_connections 2024-09-10T09:57:06.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:06.570+0000 7fe6bf600640 1 -- 172.21.15.5:0/3895649129 wait complete. 2024-09-10T09:57:06.989 INFO:tasks.workunit.client.0.smithi005.stdout: 10 87528 15.82 MB/sec execute 165 sec latency 8805.678 ms 2024-09-10T09:57:07.331 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:57:07 smithi005 ceph-mon[30350]: pgmap v956: 129 pgs: 129 active+clean; 694 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 10 MiB/s wr, 467 op/s 2024-09-10T09:57:07.331 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:57:07 smithi005 ceph-mon[30350]: from='client.? 172.21.15.5:0/3895649129' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:57:07.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:57:07 smithi159 ceph-mon[35513]: pgmap v956: 129 pgs: 129 active+clean; 694 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 10 MiB/s wr, 467 op/s 2024-09-10T09:57:07.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:57:07 smithi159 ceph-mon[35513]: from='client.? 172.21.15.5:0/3895649129' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:57:07.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:57:07 smithi102 ceph-mon[35442]: pgmap v956: 129 pgs: 129 active+clean; 694 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 10 MiB/s wr, 467 op/s 2024-09-10T09:57:07.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:57:07 smithi102 ceph-mon[35442]: from='client.? 172.21.15.5:0/3895649129' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:57:07.989 INFO:tasks.workunit.client.0.smithi005.stdout: 10 87528 15.72 MB/sec execute 166 sec latency 9805.744 ms 2024-09-10T09:57:08.254 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-10T09:57:08.637 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:57:08.989 INFO:tasks.workunit.client.0.smithi005.stdout: 10 87528 15.63 MB/sec execute 167 sec latency 10805.806 ms 2024-09-10T09:57:09.280 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:57:09 smithi005 ceph-mon[30350]: pgmap v957: 129 pgs: 129 active+clean; 721 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 12 MiB/s wr, 433 op/s 2024-09-10T09:57:09.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:09.294+0000 7f7aaee00640 1 -- 172.21.15.5:0/355541255 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7aa81049a0 msgr2=0x7f7aa81089f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:57:09.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:09.294+0000 7f7aaee00640 1 --2- 172.21.15.5:0/355541255 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7aa81049a0 0x7f7aa81089f0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f7a94009860 tx=0x7f7a9402a340 comp rx=0 tx=0).stop 2024-09-10T09:57:09.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:09.294+0000 7f7aaee00640 1 -- 172.21.15.5:0/355541255 shutdown_connections 2024-09-10T09:57:09.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:09.294+0000 7f7aaee00640 1 --2- 172.21.15.5:0/355541255 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7aa8109120 0x7f7aa8110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:09.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:09.294+0000 7f7aaee00640 1 --2- 172.21.15.5:0/355541255 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7aa81049a0 0x7f7aa81089f0 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:09.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:09.294+0000 7f7aaee00640 1 --2- 172.21.15.5:0/355541255 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7aa8103ff0 0x7f7aa81043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:09.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:09.294+0000 7f7aaee00640 1 -- 172.21.15.5:0/355541255 >> 172.21.15.5:0/355541255 conn(0x7f7aa80ffa90 msgr2=0x7f7aa8101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:57:09.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:09.294+0000 7f7aaee00640 1 -- 172.21.15.5:0/355541255 shutdown_connections 2024-09-10T09:57:09.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:09.294+0000 7f7aaee00640 1 -- 172.21.15.5:0/355541255 wait complete. 2024-09-10T09:57:09.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:09.294+0000 7f7aaee00640 1 Processor -- start 2024-09-10T09:57:09.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:09.294+0000 7f7aaee00640 1 -- start start 2024-09-10T09:57:09.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:09.294+0000 7f7aaee00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7aa8103ff0 0x7f7aa819a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:09.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:09.294+0000 7f7aaee00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7aa81049a0 0x7f7aa819abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:09.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:09.294+0000 7f7aaee00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7aa8109120 0x7f7aa819ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:09.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:09.294+0000 7f7aaee00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f7aa8114120 con 0x7f7aa81049a0 2024-09-10T09:57:09.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:09.294+0000 7f7aaee00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f7aa8113fa0 con 0x7f7aa8103ff0 2024-09-10T09:57:09.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:09.294+0000 7f7aaee00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f7aa81142a0 con 0x7f7aa8109120 2024-09-10T09:57:09.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:09.294+0000 7f7aa7400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7aa81049a0 0x7f7aa819abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:09.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:09.294+0000 7f7aace00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7aa8109120 0x7f7aa819ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:09.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:09.294+0000 7f7aa7400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7aa81049a0 0x7f7aa819abe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51160/0 (socket says 172.21.15.5:51160) 2024-09-10T09:57:09.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:09.294+0000 7f7aace00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7aa8109120 0x7f7aa819ef70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:37862/0 (socket says 172.21.15.5:37862) 2024-09-10T09:57:09.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:09.294+0000 7f7aa7400640 1 -- 172.21.15.5:0/9168759 learned_addr learned my addr 172.21.15.5:0/9168759 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:57:09.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:09.294+0000 7f7aa7e00640 1 --2- 172.21.15.5:0/9168759 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7aa8103ff0 0x7f7aa819a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:09.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:09.294+0000 7f7aa7400640 1 -- 172.21.15.5:0/9168759 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7aa8109120 msgr2=0x7f7aa819ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:57:09.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:09.294+0000 7f7aa7400640 1 --2- 172.21.15.5:0/9168759 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7aa8109120 0x7f7aa819ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:09.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:09.294+0000 7f7aa7400640 1 -- 172.21.15.5:0/9168759 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7aa8103ff0 msgr2=0x7f7aa819a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:57:09.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:09.294+0000 7f7aa7400640 1 --2- 172.21.15.5:0/9168759 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7aa8103ff0 0x7f7aa819a6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:09.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:09.294+0000 7f7aa7400640 1 -- 172.21.15.5:0/9168759 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7aa819f650 con 0x7f7aa81049a0 2024-09-10T09:57:09.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:09.294+0000 7f7aace00640 1 --2- 172.21.15.5:0/9168759 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7aa8109120 0x7f7aa819ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:57:09.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:09.294+0000 7f7aa7e00640 1 --2- 172.21.15.5:0/9168759 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7aa8103ff0 0x7f7aa819a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:57:09.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:09.294+0000 7f7aa7400640 1 --2- 172.21.15.5:0/9168759 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7aa81049a0 0x7f7aa819abe0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f7a94035040 tx=0x7f7a94033d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:57:09.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:09.300+0000 7f7aa4c00640 1 -- 172.21.15.5:0/9168759 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a9403f020 con 0x7f7aa81049a0 2024-09-10T09:57:09.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:09.300+0000 7f7aa4c00640 1 -- 172.21.15.5:0/9168759 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7a94004240 con 0x7f7aa81049a0 2024-09-10T09:57:09.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:09.300+0000 7f7aaee00640 1 -- 172.21.15.5:0/9168759 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7aa819f8e0 con 0x7f7aa81049a0 2024-09-10T09:57:09.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:09.300+0000 7f7aa4c00640 1 -- 172.21.15.5:0/9168759 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a94037040 con 0x7f7aa81049a0 2024-09-10T09:57:09.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:09.300+0000 7f7aaee00640 1 -- 172.21.15.5:0/9168759 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7aa81a7220 con 0x7f7aa81049a0 2024-09-10T09:57:09.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:09.302+0000 7f7ab17dd740 1 Processor -- start 2024-09-10T09:57:09.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:09.302+0000 7f7ab17dd740 1 -- start start 2024-09-10T09:57:09.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:09.302+0000 7f7aa4c00640 1 -- 172.21.15.5:0/9168759 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f7a94042020 con 0x7f7aa81049a0 2024-09-10T09:57:09.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:09.302+0000 7f7ab17dd740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f479526c70 0x55f479527050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:09.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:09.302+0000 7f7ab17dd740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f479527590 0x55f479531910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:09.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:09.302+0000 7f7ab17dd740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f479531e50 0x55f479534240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:09.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:09.302+0000 7f7ab17dd740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55f47948d3e0 con 0x55f479531e50 2024-09-10T09:57:09.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:09.302+0000 7f7ab17dd740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55f479434d80 con 0x55f479526c70 2024-09-10T09:57:09.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:09.302+0000 7f7ab17dd740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55f4794893a0 con 0x55f479527590 2024-09-10T09:57:09.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:09.302+0000 7f7aace00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f479526c70 0x55f479527050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:09.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:09.302+0000 7f7aace00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f479526c70 0x55f479527050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:55888/0 (socket says 172.21.15.5:55888) 2024-09-10T09:57:09.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:09.302+0000 7f7aace00640 1 -- 172.21.15.5:0/2414522248 learned_addr learned my addr 172.21.15.5:0/2414522248 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:57:09.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:09.302+0000 7f7aa7400640 1 --2- 172.21.15.5:0/2414522248 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f479531e50 0x55f479534240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:09.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:09.302+0000 7f7aace00640 1 -- 172.21.15.5:0/2414522248 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f479527590 msgr2=0x55f479531910 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T09:57:09.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:09.302+0000 7f7aace00640 1 --2- 172.21.15.5:0/2414522248 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f479527590 0x55f479531910 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:09.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:09.302+0000 7f7aace00640 1 -- 172.21.15.5:0/2414522248 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f479531e50 msgr2=0x55f479534240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:57:09.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:09.302+0000 7f7aace00640 1 --2- 172.21.15.5:0/2414522248 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f479531e50 0x55f479534240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:09.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:09.302+0000 7f7aace00640 1 -- 172.21.15.5:0/2414522248 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f479453ae0 con 0x55f479526c70 2024-09-10T09:57:09.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:09.302+0000 7f7aa7400640 1 --2- 172.21.15.5:0/2414522248 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f479531e50 0x55f479534240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T09:57:09.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:09.302+0000 7f7aace00640 1 --2- 172.21.15.5:0/2414522248 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f479526c70 0x55f479527050 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f7aa0010710 tx=0x7f7aa0010bd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=94e5cda4a2ab0353 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:57:09.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:09.302+0000 7f7a9d600640 1 -- 172.21.15.5:0/2414522248 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7aa000e8d0 con 0x55f479526c70 2024-09-10T09:57:09.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:09.302+0000 7f7a9d600640 1 -- 172.21.15.5:0/2414522248 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7aa0012450 con 0x55f479526c70 2024-09-10T09:57:09.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:09.302+0000 7f7a9d600640 1 -- 172.21.15.5:0/2414522248 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7aa001c650 con 0x55f479526c70 2024-09-10T09:57:09.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:09.302+0000 7f7aa4c00640 1 --2- 172.21.15.5:0/9168759 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f7a8c0a2b80 0x7f7a8c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:09.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:09.302+0000 7f7aa4c00640 1 -- 172.21.15.5:0/9168759 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f7a940ef640 con 0x7f7aa81049a0 2024-09-10T09:57:09.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:09.302+0000 7f7aa7e00640 1 --2- 172.21.15.5:0/9168759 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f7a8c0a2b80 0x7f7a8c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:09.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:09.302+0000 7f7ab17dd740 1 -- 172.21.15.5:0/2414522248 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f479526c70 msgr2=0x55f479527050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:57:09.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:09.302+0000 7f7ab17dd740 1 --2- 172.21.15.5:0/2414522248 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f479526c70 0x55f479527050 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f7aa0010710 tx=0x7f7aa0010bd0 comp rx=0 tx=0).stop 2024-09-10T09:57:09.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:09.302+0000 7f7ab17dd740 1 -- 172.21.15.5:0/2414522248 shutdown_connections 2024-09-10T09:57:09.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:09.302+0000 7f7ab17dd740 1 --2- 172.21.15.5:0/2414522248 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f479531e50 0x55f479534240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:09.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:09.302+0000 7f7ab17dd740 1 --2- 172.21.15.5:0/2414522248 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f479527590 0x55f479531910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:09.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:09.302+0000 7f7ab17dd740 1 --2- 172.21.15.5:0/2414522248 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f479526c70 0x55f479527050 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:09.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:09.302+0000 7f7ab17dd740 1 -- 172.21.15.5:0/2414522248 >> 172.21.15.5:0/2414522248 conn(0x55f479523200 msgr2=0x55f4795235e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:57:09.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:09.302+0000 7f7ab17dd740 1 -- 172.21.15.5:0/2414522248 shutdown_connections 2024-09-10T09:57:09.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:09.302+0000 7f7ab17dd740 1 -- 172.21.15.5:0/2414522248 wait complete. 2024-09-10T09:57:09.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:09.302+0000 7f7aa7e00640 1 --2- 172.21.15.5:0/9168759 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f7a8c0a2b80 0x7f7a8c0a5040 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x55f479524060 tx=0x7f7a98009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:57:09.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:09.302+0000 7f7ab17dd740 1 -- start start 2024-09-10T09:57:09.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:09.302+0000 7f7ab17dd740 1 Processor -- start 2024-09-10T09:57:09.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:09.308+0000 7f7ab17dd740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f479526c70 0x55f47952ed10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:09.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:09.308+0000 7f7ab17dd740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f479527590 0x55f47952f250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:09.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:09.308+0000 7f7ab17dd740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f479531e50 0x55f47952f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:09.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:09.308+0000 7f7ab17dd740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55f479534a20 con 0x55f479526c70 2024-09-10T09:57:09.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:09.308+0000 7f7ab17dd740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55f4794480f0 con 0x55f479527590 2024-09-10T09:57:09.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:09.308+0000 7f7ab17dd740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55f479534ba0 con 0x55f479531e50 2024-09-10T09:57:09.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:09.308+0000 7f7ab17dd740 10 client.0 _ll_register_callbacks cb 0x55f47942fbd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:57:09.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:09.308+0000 7f7aa7e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f479526c70 0x55f47952ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:09.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:09.308+0000 7f7aa7e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f479526c70 0x55f47952ed10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51180/0 (socket says 172.21.15.5:51180) 2024-09-10T09:57:09.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:09.308+0000 7f7aa7e00640 1 -- 172.21.15.5:0/2367113986 learned_addr learned my addr 172.21.15.5:0/2367113986 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:57:09.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:09.308+0000 7f7aace00640 1 --2- 172.21.15.5:0/2367113986 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f479531e50 0x55f47952f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:09.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:09.308+0000 7f7aa7e00640 1 -- 172.21.15.5:0/2367113986 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f479531e50 msgr2=0x55f47952f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:57:09.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:09.308+0000 7f7aa7e00640 1 --2- 172.21.15.5:0/2367113986 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f479531e50 0x55f47952f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:09.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:09.308+0000 7f7aa7e00640 1 -- 172.21.15.5:0/2367113986 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f479527590 msgr2=0x55f47952f250 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T09:57:09.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:09.308+0000 7f7aa7e00640 1 --2- 172.21.15.5:0/2367113986 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f479527590 0x55f47952f250 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:09.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:09.308+0000 7f7aa7e00640 1 -- 172.21.15.5:0/2367113986 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f4791fe3c0 con 0x55f479526c70 2024-09-10T09:57:09.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:09.308+0000 7f7aace00640 1 --2- 172.21.15.5:0/2367113986 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f479531e50 0x55f47952f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:57:09.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:09.308+0000 7f7aa7e00640 1 --2- 172.21.15.5:0/2367113986 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f479526c70 0x55f47952ed10 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f7a98003870 tx=0x7f7a98004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0a0ec19275870b7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:57:09.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:09.308+0000 7f7a92000640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-10T09:57:09.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:09.308+0000 7f7a92000640 1 -- 172.21.15.5:0/2367113986 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a9800fd50 con 0x55f479526c70 2024-09-10T09:57:09.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:09.308+0000 7f7a92000640 1 -- 172.21.15.5:0/2367113986 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f7a9801e3f0 con 0x55f479526c70 2024-09-10T09:57:09.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:09.308+0000 7f7ab17dd740 1 -- 172.21.15.5:0/2367113986 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7aa0012450 con 0x55f479526c70 2024-09-10T09:57:09.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:09.308+0000 7f7a92000640 1 -- 172.21.15.5:0/2367113986 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f7a98003e50 con 0x55f479526c70 2024-09-10T09:57:09.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:09.308+0000 7f7ab17dd740 10 client.18984 fetch_fsmap learned FSMap version 42 2024-09-10T09:57:09.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:09.308+0000 7f7ab17dd740 1 -- 172.21.15.5:0/2367113986 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55f479540a30 con 0x55f479526c70 2024-09-10T09:57:09.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:09.308+0000 7f7a92000640 1 -- 172.21.15.5:0/2367113986 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7aa0012450 con 0x55f479526c70 2024-09-10T09:57:09.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:09.308+0000 7f7a92000640 1 -- 172.21.15.5:0/2367113986 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 42) ==== 8370+0+0 (secure 0 0 0) 0x7f7a98026720 con 0x55f479526c70 2024-09-10T09:57:09.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:09.308+0000 7f7ab17dd740 10 client.18984 fetch_fsmap finished waiting for FSMap version 42 2024-09-10T09:57:09.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:09.308+0000 7f7ab17dd740 10 client.18984 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-09-10T09:57:09.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:09.308+0000 7f7ab17dd740 20 client.18984 populate_metadata read hostname 'smithi005' 2024-09-10T09:57:09.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:09.308+0000 7f7ab17dd740 1 --2- 172.21.15.5:0/2367113986 >> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] conn(0x55f479541ef0 0x55f4795443b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:09.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:09.308+0000 7f7ab17dd740 4 client.18984 mds_command: new command op to 24443 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:57:09.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:09.308+0000 7f7ab17dd740 1 -- 172.21.15.5:0/2367113986 --> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f47949a950 con 0x55f479541ef0 2024-09-10T09:57:09.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:09.308+0000 7f7aa7400640 1 --2- 172.21.15.5:0/2367113986 >> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] conn(0x55f479541ef0 0x55f4795443b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:09.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:09.308+0000 7f7aa7400640 1 --2- 172.21.15.5:0/2367113986 >> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] conn(0x55f479541ef0 0x55f4795443b0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=3f412fa9243c47ae server_cookie=668fe027c54cb35d in_seq=0 out_seq=0 2024-09-10T09:57:09.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:09.308+0000 7f7a92000640 10 client.18984 ms_handle_connect on v2:172.21.15.102:6832/2457007427 2024-09-10T09:57:09.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:09.308+0000 7f7a92000640 1 -- 172.21.15.5:0/2367113986 <== mds.4 v2:172.21.15.102:6832/2457007427 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f47949a950 con 0x55f479541ef0 2024-09-10T09:57:09.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:09.308+0000 7f7a92000640 10 client.18984 handle_command_reply: tid=0 2024-09-10T09:57:09.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:09.308+0000 7f7ab17dd740 1 client.18984 shutdown 2024-09-10T09:57:09.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:09.308+0000 7f7ab17dd740 1 -- 172.21.15.5:0/2367113986 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f479526c70 msgr2=0x55f47952ed10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:57:09.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:09.308+0000 7f7ab17dd740 1 --2- 172.21.15.5:0/2367113986 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f479526c70 0x55f47952ed10 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f7a98003870 tx=0x7f7a98004730 comp rx=0 tx=0).stop 2024-09-10T09:57:09.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:09.308+0000 7f7ab17dd740 1 -- 172.21.15.5:0/2367113986 shutdown_connections 2024-09-10T09:57:09.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:09.308+0000 7f7ab17dd740 1 --2- 172.21.15.5:0/2367113986 >> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] conn(0x55f479541ef0 0x55f4795443b0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:09.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:09.308+0000 7f7ab17dd740 1 --2- 172.21.15.5:0/2367113986 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f479531e50 0x55f47952f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:09.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:09.308+0000 7f7ab17dd740 1 --2- 172.21.15.5:0/2367113986 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f479527590 0x55f47952f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:09.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:09.308+0000 7f7ab17dd740 1 --2- 172.21.15.5:0/2367113986 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f479526c70 0x55f47952ed10 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:09.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:09.308+0000 7f7ab17dd740 1 -- 172.21.15.5:0/2367113986 >> 172.21.15.5:0/2367113986 conn(0x55f479523200 msgr2=0x55f479522380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:57:09.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:09.308+0000 7f7a92000640 0 client.18984 ms_handle_reset on v2:172.21.15.102:6832/2457007427 2024-09-10T09:57:09.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:09.308+0000 7f7ab17dd740 1 -- 172.21.15.5:0/2367113986 shutdown_connections 2024-09-10T09:57:09.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:09.308+0000 7f7ab17dd740 1 -- 172.21.15.5:0/2367113986 wait complete. 2024-09-10T09:57:09.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:09.313+0000 7f7ab17dd740 20 client.18984 trim_cache size 0 max 16384 2024-09-10T09:57:09.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:09.326+0000 7f7ab17dd740 1 Processor -- start 2024-09-10T09:57:09.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:09.326+0000 7f7ab17dd740 1 -- start start 2024-09-10T09:57:09.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:09.326+0000 7f7ab17dd740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f479526c70 0x55f4795296d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:09.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:09.326+0000 7f7ab17dd740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f479527590 0x55f479549cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:09.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:09.326+0000 7f7ab17dd740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f479531e50 0x55f47954a200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:09.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:09.326+0000 7f7ab17dd740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f7aa81b1020 con 0x55f479531e50 2024-09-10T09:57:09.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:09.326+0000 7f7ab17dd740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f7aa81b2b60 con 0x55f479526c70 2024-09-10T09:57:09.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:09.326+0000 7f7ab17dd740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f7aa81b16f0 con 0x55f479527590 2024-09-10T09:57:09.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:09.326+0000 7f7aace00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f479531e50 0x55f47954a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:09.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:09.326+0000 7f7aa7400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f479527590 0x55f479549cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:09.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:09.326+0000 7f7aace00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f479531e50 0x55f47954a200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51182/0 (socket says 172.21.15.5:51182) 2024-09-10T09:57:09.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:09.326+0000 7f7aace00640 1 -- 172.21.15.5:0/2236557155 learned_addr learned my addr 172.21.15.5:0/2236557155 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:57:09.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:09.326+0000 7f7aa7e00640 1 --2- 172.21.15.5:0/2236557155 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f479526c70 0x55f4795296d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:09.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:09.326+0000 7f7aa7400640 1 -- 172.21.15.5:0/2236557155 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f479526c70 msgr2=0x55f4795296d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:57:09.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:09.326+0000 7f7aa7400640 1 --2- 172.21.15.5:0/2236557155 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f479526c70 0x55f4795296d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:09.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:09.326+0000 7f7aa7400640 1 -- 172.21.15.5:0/2236557155 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f479531e50 msgr2=0x55f47954a200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:57:09.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:09.326+0000 7f7aa7400640 1 --2- 172.21.15.5:0/2236557155 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f479531e50 0x55f47954a200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:09.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:09.326+0000 7f7aa7400640 1 -- 172.21.15.5:0/2236557155 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f47951e770 con 0x55f479527590 2024-09-10T09:57:09.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:09.326+0000 7f7aace00640 1 --2- 172.21.15.5:0/2236557155 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f479531e50 0x55f47954a200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:57:09.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:09.326+0000 7f7aa7400640 1 --2- 172.21.15.5:0/2236557155 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f479527590 0x55f479549cc0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x55f479524580 tx=0x7f7a94005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3f5d13cbf209e143 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:57:09.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:09.326+0000 7f7a9cc00640 1 -- 172.21.15.5:0/2236557155 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a940bba50 con 0x55f479527590 2024-09-10T09:57:09.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:09.326+0000 7f7a9cc00640 1 -- 172.21.15.5:0/2236557155 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7a940bfe00 con 0x55f479527590 2024-09-10T09:57:09.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:09.326+0000 7f7a9cc00640 1 -- 172.21.15.5:0/2236557155 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a940ef2b0 con 0x55f479527590 2024-09-10T09:57:09.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:09.326+0000 7f7ab17dd740 1 -- 172.21.15.5:0/2236557155 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f479527590 msgr2=0x55f479549cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:57:09.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:09.326+0000 7f7ab17dd740 1 --2- 172.21.15.5:0/2236557155 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f479527590 0x55f479549cc0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x55f479524580 tx=0x7f7a94005f90 comp rx=0 tx=0).stop 2024-09-10T09:57:09.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:09.326+0000 7f7ab17dd740 1 -- 172.21.15.5:0/2236557155 shutdown_connections 2024-09-10T09:57:09.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:09.326+0000 7f7ab17dd740 1 --2- 172.21.15.5:0/2236557155 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f479531e50 0x55f47954a200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:09.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:09.326+0000 7f7ab17dd740 1 --2- 172.21.15.5:0/2236557155 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f479527590 0x55f479549cc0 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:09.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:09.326+0000 7f7ab17dd740 1 --2- 172.21.15.5:0/2236557155 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f479526c70 0x55f4795296d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:09.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:09.326+0000 7f7ab17dd740 1 -- 172.21.15.5:0/2236557155 >> 172.21.15.5:0/2236557155 conn(0x55f479523200 msgr2=0x55f47952a040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:57:09.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:09.326+0000 7f7ab17dd740 1 -- 172.21.15.5:0/2236557155 shutdown_connections 2024-09-10T09:57:09.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:09.326+0000 7f7ab17dd740 1 -- 172.21.15.5:0/2236557155 wait complete. 2024-09-10T09:57:09.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:09.326+0000 7f7ab17dd740 1 -- start start 2024-09-10T09:57:09.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:09.326+0000 7f7ab17dd740 1 Processor -- start 2024-09-10T09:57:09.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:09.326+0000 7f7ab17dd740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f479526c70 0x55f479535340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:09.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:09.326+0000 7f7ab17dd740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f479527590 0x55f479535880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:09.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:09.326+0000 7f7ab17dd740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f479531e50 0x55f4795aac80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:09.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:09.326+0000 7f7ab17dd740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f7aa81b2490 con 0x55f479527590 2024-09-10T09:57:09.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:09.326+0000 7f7ab17dd740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f7aa81b1dc0 con 0x55f479531e50 2024-09-10T09:57:09.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:09.326+0000 7f7ab17dd740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f7aa81af000 con 0x55f479526c70 2024-09-10T09:57:09.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:09.326+0000 7f7ab17dd740 10 client.0 _ll_register_callbacks cb 0x7f7a98009d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:57:09.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:09.326+0000 7f7aa7400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f479527590 0x55f479535880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:09.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:09.326+0000 7f7aa7400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f479527590 0x55f479535880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51190/0 (socket says 172.21.15.5:51190) 2024-09-10T09:57:09.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:09.326+0000 7f7aa7400640 1 -- 172.21.15.5:0/3102181665 learned_addr learned my addr 172.21.15.5:0/3102181665 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:57:09.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:09.326+0000 7f7aa7e00640 1 --2- 172.21.15.5:0/3102181665 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f479526c70 0x55f479535340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:09.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:09.326+0000 7f7aace00640 1 --2- 172.21.15.5:0/3102181665 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f479531e50 0x55f4795aac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:09.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:09.332+0000 7f7aa7e00640 1 -- 172.21.15.5:0/3102181665 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f479531e50 msgr2=0x55f4795aac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:57:09.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:09.332+0000 7f7aa7e00640 1 --2- 172.21.15.5:0/3102181665 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f479531e50 0x55f4795aac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:09.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:09.332+0000 7f7aa7e00640 1 -- 172.21.15.5:0/3102181665 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f479527590 msgr2=0x55f479535880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:57:09.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:09.332+0000 7f7aa7e00640 1 --2- 172.21.15.5:0/3102181665 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f479527590 0x55f479535880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:09.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:09.332+0000 7f7aa7e00640 1 -- 172.21.15.5:0/3102181665 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f4795ab270 con 0x55f479526c70 2024-09-10T09:57:09.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:09.332+0000 7f7aa7400640 1 --2- 172.21.15.5:0/3102181665 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f479527590 0x55f479535880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:57:09.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:09.332+0000 7f7aa7e00640 1 --2- 172.21.15.5:0/3102181665 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f479526c70 0x55f479535340 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f7a98026b40 tx=0x7f7a980269b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8de3318446057af server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:57:09.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:09.332+0000 7f7a92000640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-09-10T09:57:09.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:09.332+0000 7f7a92000640 1 -- 172.21.15.5:0/3102181665 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a98003e50 con 0x55f479526c70 2024-09-10T09:57:09.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:09.332+0000 7f7a92000640 1 -- 172.21.15.5:0/3102181665 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f7a9800fb50 con 0x55f479526c70 2024-09-10T09:57:09.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:09.332+0000 7f7ab17dd740 1 -- 172.21.15.5:0/3102181665 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7a940bfe00 con 0x55f479526c70 2024-09-10T09:57:09.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:09.332+0000 7f7a92000640 1 -- 172.21.15.5:0/3102181665 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f7a9800f790 con 0x55f479526c70 2024-09-10T09:57:09.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:09.332+0000 7f7ab17dd740 10 client.29044 fetch_fsmap learned FSMap version 42 2024-09-10T09:57:09.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:09.332+0000 7f7ab17dd740 1 -- 172.21.15.5:0/3102181665 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55f4795ac5b0 con 0x55f479526c70 2024-09-10T09:57:09.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:09.332+0000 7f7a92000640 1 -- 172.21.15.5:0/3102181665 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7a940bfe00 con 0x55f479526c70 2024-09-10T09:57:09.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:09.332+0000 7f7a92000640 1 -- 172.21.15.5:0/3102181665 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 42) ==== 8370+0+0 (secure 0 0 0) 0x7f7a9801e8a0 con 0x55f479526c70 2024-09-10T09:57:09.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:09.332+0000 7f7ab17dd740 10 client.29044 fetch_fsmap finished waiting for FSMap version 42 2024-09-10T09:57:09.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:09.332+0000 7f7ab17dd740 10 client.29044 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-09-10T09:57:09.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:09.332+0000 7f7ab17dd740 20 client.29044 populate_metadata read hostname 'smithi005' 2024-09-10T09:57:09.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:09.332+0000 7f7ab17dd740 1 --2- 172.21.15.5:0/3102181665 >> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] conn(0x55f4795acd00 0x55f4795af120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:09.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:09.332+0000 7f7ab17dd740 4 client.29044 mds_command: new command op to 24443 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T09:57:09.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:09.332+0000 7f7ab17dd740 1 -- 172.21.15.5:0/3102181665 --> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f479494e60 con 0x55f4795acd00 2024-09-10T09:57:09.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:09.332+0000 7f7aa7400640 1 --2- 172.21.15.5:0/3102181665 >> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] conn(0x55f4795acd00 0x55f4795af120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:09.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:09.332+0000 7f7aa7400640 1 --2- 172.21.15.5:0/3102181665 >> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] conn(0x55f4795acd00 0x55f4795af120 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=7441cb23c5f9e3e7 server_cookie=34fc2086d58735ac in_seq=0 out_seq=0 2024-09-10T09:57:09.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:09.332+0000 7f7a92000640 10 client.29044 ms_handle_connect on v2:172.21.15.102:6832/2457007427 2024-09-10T09:57:09.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:09.332+0000 7f7a92000640 1 -- 172.21.15.5:0/3102181665 <== mds.4 v2:172.21.15.102:6832/2457007427 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f479494e60 con 0x55f4795acd00 2024-09-10T09:57:09.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:09.332+0000 7f7a92000640 10 client.29044 handle_command_reply: tid=0 2024-09-10T09:57:09.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:09.332+0000 7f7ab17dd740 1 client.29044 shutdown 2024-09-10T09:57:09.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:09.332+0000 7f7ab17dd740 1 -- 172.21.15.5:0/3102181665 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f479526c70 msgr2=0x55f479535340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:57:09.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:09.332+0000 7f7ab17dd740 1 --2- 172.21.15.5:0/3102181665 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f479526c70 0x55f479535340 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f7a98026b40 tx=0x7f7a980269b0 comp rx=0 tx=0).stop 2024-09-10T09:57:09.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:09.332+0000 7f7ab17dd740 1 -- 172.21.15.5:0/3102181665 shutdown_connections 2024-09-10T09:57:09.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:09.332+0000 7f7ab17dd740 1 --2- 172.21.15.5:0/3102181665 >> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] conn(0x55f4795acd00 0x55f4795af120 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:09.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:09.332+0000 7f7ab17dd740 1 --2- 172.21.15.5:0/3102181665 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f479531e50 0x55f4795aac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:09.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:09.332+0000 7f7ab17dd740 1 --2- 172.21.15.5:0/3102181665 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f479527590 0x55f479535880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:09.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:09.332+0000 7f7ab17dd740 1 --2- 172.21.15.5:0/3102181665 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f479526c70 0x55f479535340 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:09.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:09.332+0000 7f7a92000640 0 client.29044 ms_handle_reset on v2:172.21.15.102:6832/2457007427 2024-09-10T09:57:09.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:09.332+0000 7f7ab17dd740 1 -- 172.21.15.5:0/3102181665 >> 172.21.15.5:0/3102181665 conn(0x55f479523200 msgr2=0x55f479530570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:57:09.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:09.332+0000 7f7ab17dd740 1 -- 172.21.15.5:0/3102181665 shutdown_connections 2024-09-10T09:57:09.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:09.332+0000 7f7ab17dd740 1 -- 172.21.15.5:0/3102181665 wait complete. 2024-09-10T09:57:09.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:09.332+0000 7f7ab17dd740 20 client.29044 trim_cache size 0 max 16384 2024-09-10T09:57:09.351 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-10T09:57:09.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:09.332+0000 7f7aaee00640 1 -- 172.21.15.5:0/9168759 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f7a8c0a2b80 msgr2=0x7f7a8c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:57:09.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:09.332+0000 7f7aaee00640 1 --2- 172.21.15.5:0/9168759 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f7a8c0a2b80 0x7f7a8c0a5040 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x55f479524060 tx=0x7f7a98009260 comp rx=0 tx=0).stop 2024-09-10T09:57:09.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:09.332+0000 7f7aaee00640 1 -- 172.21.15.5:0/9168759 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7aa81049a0 msgr2=0x7f7aa819abe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:57:09.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:09.332+0000 7f7aaee00640 1 --2- 172.21.15.5:0/9168759 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7aa81049a0 0x7f7aa819abe0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f7a94035040 tx=0x7f7a94033d90 comp rx=0 tx=0).stop 2024-09-10T09:57:09.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:09.332+0000 7f7aaee00640 1 -- 172.21.15.5:0/9168759 shutdown_connections 2024-09-10T09:57:09.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:09.332+0000 7f7aaee00640 1 --2- 172.21.15.5:0/9168759 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f7a8c0a2b80 0x7f7a8c0a5040 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:09.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:09.332+0000 7f7aaee00640 1 --2- 172.21.15.5:0/9168759 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7aa8109120 0x7f7aa819ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:09.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:09.332+0000 7f7aaee00640 1 --2- 172.21.15.5:0/9168759 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7aa81049a0 0x7f7aa819abe0 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:09.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:09.332+0000 7f7aaee00640 1 --2- 172.21.15.5:0/9168759 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7aa8103ff0 0x7f7aa819a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:09.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:09.332+0000 7f7aaee00640 1 -- 172.21.15.5:0/9168759 >> 172.21.15.5:0/9168759 conn(0x7f7aa80ffa90 msgr2=0x7f7aa8100d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:57:09.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:09.332+0000 7f7aaee00640 1 -- 172.21.15.5:0/9168759 shutdown_connections 2024-09-10T09:57:09.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:09.332+0000 7f7aaee00640 1 -- 172.21.15.5:0/9168759 wait complete. 2024-09-10T09:57:09.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:57:09 smithi159 ceph-mon[35513]: pgmap v957: 129 pgs: 129 active+clean; 721 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 12 MiB/s wr, 433 op/s 2024-09-10T09:57:09.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:57:09 smithi102 ceph-mon[35442]: pgmap v957: 129 pgs: 129 active+clean; 721 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 12 MiB/s wr, 433 op/s 2024-09-10T09:57:09.989 INFO:tasks.workunit.client.0.smithi005.stdout: 10 87528 15.54 MB/sec execute 168 sec latency 11805.878 ms 2024-09-10T09:57:10.497 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-10T09:57:10.900 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:57:10.989 INFO:tasks.workunit.client.0.smithi005.stdout: 10 87528 15.44 MB/sec execute 169 sec latency 12805.946 ms 2024-09-10T09:57:11.525 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:57:11 smithi005 ceph-mon[30350]: pgmap v958: 129 pgs: 129 active+clean; 721 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 13 MiB/s wr, 429 op/s 2024-09-10T09:57:11.525 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:57:11 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:57:11.525 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:57:11 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:57:11.525 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:57:11 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:57:11.525 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:57:11 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:57:11.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:57:11 smithi159 ceph-mon[35513]: pgmap v958: 129 pgs: 129 active+clean; 721 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 13 MiB/s wr, 429 op/s 2024-09-10T09:57:11.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:57:11 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:57:11.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:57:11 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:57:11.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:57:11 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:57:11.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:57:11 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:57:11.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:11.603+0000 7f9e96200640 1 -- 172.21.15.5:0/610248812 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9e90100520 msgr2=0x7f9e90104860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:57:11.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:11.603+0000 7f9e96200640 1 --2- 172.21.15.5:0/610248812 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9e90100520 0x7f9e90104860 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f9e74009900 tx=0x7f9e7402a440 comp rx=0 tx=0).stop 2024-09-10T09:57:11.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:11.603+0000 7f9e96200640 1 -- 172.21.15.5:0/610248812 shutdown_connections 2024-09-10T09:57:11.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:11.603+0000 7f9e96200640 1 --2- 172.21.15.5:0/610248812 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9e90104da0 0x7f9e90105250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:11.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:11.603+0000 7f9e96200640 1 --2- 172.21.15.5:0/610248812 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9e90100520 0x7f9e90104860 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:11.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:11.603+0000 7f9e96200640 1 --2- 172.21.15.5:0/610248812 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9e900ffc00 0x7f9e900fffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:11.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:11.603+0000 7f9e96200640 1 -- 172.21.15.5:0/610248812 >> 172.21.15.5:0/610248812 conn(0x7f9e900fb920 msgr2=0x7f9e900fdd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:57:11.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:11.603+0000 7f9e96200640 1 -- 172.21.15.5:0/610248812 shutdown_connections 2024-09-10T09:57:11.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:11.603+0000 7f9e96200640 1 -- 172.21.15.5:0/610248812 wait complete. 2024-09-10T09:57:11.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:11.603+0000 7f9e96200640 1 Processor -- start 2024-09-10T09:57:11.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:11.607+0000 7f9e96200640 1 -- start start 2024-09-10T09:57:11.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:11.607+0000 7f9e96200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9e900ffc00 0x7f9e901a12c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:11.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:11.607+0000 7f9e96200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9e90100520 0x7f9e901a1800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:11.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:11.607+0000 7f9e96200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9e90104da0 0x7f9e9019f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:11.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:11.607+0000 7f9e96200640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f9e90114240 con 0x7f9e900ffc00 2024-09-10T09:57:11.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:11.607+0000 7f9e96200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f9e90109c00 con 0x7f9e90100520 2024-09-10T09:57:11.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:11.607+0000 7f9e96200640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f9e901143c0 con 0x7f9e90104da0 2024-09-10T09:57:11.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:11.607+0000 7f9e94e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9e900ffc00 0x7f9e901a12c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:11.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:11.607+0000 7f9e87e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9e90100520 0x7f9e901a1800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:11.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:11.607+0000 7f9e87e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9e90100520 0x7f9e901a1800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:34676/0 (socket says 172.21.15.5:34676) 2024-09-10T09:57:11.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:11.607+0000 7f9e87e00640 1 -- 172.21.15.5:0/3075990412 learned_addr learned my addr 172.21.15.5:0/3075990412 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:57:11.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:11.607+0000 7f9e94e00640 1 -- 172.21.15.5:0/3075990412 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9e90104da0 msgr2=0x7f9e9019f5d0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-10T09:57:11.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:11.607+0000 7f9e94e00640 1 --2- 172.21.15.5:0/3075990412 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9e90104da0 0x7f9e9019f5d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:11.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:11.607+0000 7f9e94e00640 1 -- 172.21.15.5:0/3075990412 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9e90100520 msgr2=0x7f9e901a1800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:57:11.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:11.607+0000 7f9e94e00640 1 --2- 172.21.15.5:0/3075990412 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9e90100520 0x7f9e901a1800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:11.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:11.607+0000 7f9e94e00640 1 -- 172.21.15.5:0/3075990412 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e9019fe90 con 0x7f9e900ffc00 2024-09-10T09:57:11.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:11.607+0000 7f9e87e00640 1 --2- 172.21.15.5:0/3075990412 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9e90100520 0x7f9e901a1800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:57:11.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:11.607+0000 7f9e94e00640 1 --2- 172.21.15.5:0/3075990412 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9e900ffc00 0x7f9e901a12c0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f9e7c009fd0 tx=0x7f9e7c00b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:57:11.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:11.607+0000 7f9e8e000640 1 -- 172.21.15.5:0/3075990412 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e7c015020 con 0x7f9e900ffc00 2024-09-10T09:57:11.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:11.607+0000 7f9e8e000640 1 -- 172.21.15.5:0/3075990412 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9e7c011d10 con 0x7f9e900ffc00 2024-09-10T09:57:11.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:11.607+0000 7f9e96200640 1 -- 172.21.15.5:0/3075990412 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e901a0180 con 0x7f9e900ffc00 2024-09-10T09:57:11.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:11.607+0000 7f9e8e000640 1 -- 172.21.15.5:0/3075990412 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e7c00f2e0 con 0x7f9e900ffc00 2024-09-10T09:57:11.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:11.607+0000 7f9e96200640 1 -- 172.21.15.5:0/3075990412 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e901131f0 con 0x7f9e900ffc00 2024-09-10T09:57:11.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:11.618+0000 7f9e8e000640 1 -- 172.21.15.5:0/3075990412 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f9e7c018020 con 0x7f9e900ffc00 2024-09-10T09:57:11.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:11.620+0000 7f9e9a433740 1 Processor -- start 2024-09-10T09:57:11.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:11.620+0000 7f9e8e000640 1 --2- 172.21.15.5:0/3075990412 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f9e6c0a2c40 0x7f9e6c0a5100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:11.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:11.620+0000 7f9e8e000640 1 -- 172.21.15.5:0/3075990412 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f9e7c0ce970 con 0x7f9e900ffc00 2024-09-10T09:57:11.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:11.620+0000 7f9e9a433740 1 -- start start 2024-09-10T09:57:11.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:11.620+0000 7f9e95800640 1 --2- 172.21.15.5:0/3075990412 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f9e6c0a2c40 0x7f9e6c0a5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:11.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:11.620+0000 7f9e9a433740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x565007b16e40 0x565007b17220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:11.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:11.620+0000 7f9e9a433740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x565007b17760 0x565007b21ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:11.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:11.620+0000 7f9e9a433740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x565007b22020 0x565007b24410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:11.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:11.620+0000 7f9e9a433740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x565007a7d3e0 con 0x565007b22020 2024-09-10T09:57:11.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:11.620+0000 7f9e9a433740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x565007a24d80 con 0x565007b17760 2024-09-10T09:57:11.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:11.620+0000 7f9e9a433740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x565007a793a0 con 0x565007b16e40 2024-09-10T09:57:11.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:11.625+0000 7f9e87e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x565007b17760 0x565007b21ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:11.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:11.625+0000 7f9e87e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x565007b17760 0x565007b21ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:34692/0 (socket says 172.21.15.5:34692) 2024-09-10T09:57:11.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:11.625+0000 7f9e87e00640 1 -- 172.21.15.5:0/2353540057 learned_addr learned my addr 172.21.15.5:0/2353540057 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:57:11.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:11.625+0000 7f9e95800640 1 --2- 172.21.15.5:0/2353540057 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x565007b22020 0x565007b24410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:11.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:11.625+0000 7f9e94e00640 1 --2- 172.21.15.5:0/2353540057 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x565007b16e40 0x565007b17220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:11.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:11.625+0000 7f9e87e00640 1 -- 172.21.15.5:0/2353540057 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x565007b16e40 msgr2=0x565007b17220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:57:11.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:11.625+0000 7f9e87e00640 1 --2- 172.21.15.5:0/2353540057 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x565007b16e40 0x565007b17220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:11.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:11.625+0000 7f9e87e00640 1 -- 172.21.15.5:0/2353540057 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x565007b22020 msgr2=0x565007b24410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:57:11.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:11.625+0000 7f9e87e00640 1 --2- 172.21.15.5:0/2353540057 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x565007b22020 0x565007b24410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:11.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:11.625+0000 7f9e87e00640 1 -- 172.21.15.5:0/2353540057 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565007a43ae0 con 0x565007b17760 2024-09-10T09:57:11.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:11.625+0000 7f9e95800640 1 --2- 172.21.15.5:0/2353540057 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x565007b22020 0x565007b24410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:57:11.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:11.625+0000 7f9e87e00640 1 --2- 172.21.15.5:0/2353540057 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x565007b17760 0x565007b21ae0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f9e74009a20 tx=0x7f9e74004610 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6ce77e8c70ae3211 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:57:11.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:11.625+0000 7f9e95800640 1 --2- 172.21.15.5:0/3075990412 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f9e6c0a2c40 0x7f9e6c0a5100 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x565007b14750 tx=0x7f9e8800a500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:57:11.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:11.625+0000 7f9e86000640 1 -- 172.21.15.5:0/2353540057 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e74040070 con 0x565007b17760 2024-09-10T09:57:11.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:11.625+0000 7f9e86000640 1 -- 172.21.15.5:0/2353540057 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9e740047a0 con 0x565007b17760 2024-09-10T09:57:11.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:11.625+0000 7f9e86000640 1 -- 172.21.15.5:0/2353540057 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e74035040 con 0x565007b17760 2024-09-10T09:57:11.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:11.625+0000 7f9e9a433740 1 -- 172.21.15.5:0/2353540057 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x565007b17760 msgr2=0x565007b21ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:57:11.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:11.625+0000 7f9e9a433740 1 --2- 172.21.15.5:0/2353540057 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x565007b17760 0x565007b21ae0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f9e74009a20 tx=0x7f9e74004610 comp rx=0 tx=0).stop 2024-09-10T09:57:11.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:11.625+0000 7f9e9a433740 1 -- 172.21.15.5:0/2353540057 shutdown_connections 2024-09-10T09:57:11.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:11.625+0000 7f9e9a433740 1 --2- 172.21.15.5:0/2353540057 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x565007b22020 0x565007b24410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:11.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:11.625+0000 7f9e9a433740 1 --2- 172.21.15.5:0/2353540057 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x565007b17760 0x565007b21ae0 secure :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f9e74009a20 tx=0x7f9e74004610 comp rx=0 tx=0).stop 2024-09-10T09:57:11.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:11.625+0000 7f9e9a433740 1 --2- 172.21.15.5:0/2353540057 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x565007b16e40 0x565007b17220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:11.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:11.625+0000 7f9e9a433740 1 -- 172.21.15.5:0/2353540057 >> 172.21.15.5:0/2353540057 conn(0x565007b133d0 msgr2=0x565007b137b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:57:11.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:11.625+0000 7f9e9a433740 1 -- 172.21.15.5:0/2353540057 shutdown_connections 2024-09-10T09:57:11.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:11.630+0000 7f9e9a433740 1 -- 172.21.15.5:0/2353540057 wait complete. 2024-09-10T09:57:11.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:11.630+0000 7f9e9a433740 1 -- start start 2024-09-10T09:57:11.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:11.630+0000 7f9e9a433740 1 Processor -- start 2024-09-10T09:57:11.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:11.631+0000 7f9e9a433740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x565007b16e40 0x565007b1f200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:11.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:11.631+0000 7f9e9a433740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x565007b22020 0x565007b1f740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:11.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:11.631+0000 7f9e9a433740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x565007b1fcd0 0x565007b2e090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:11.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:11.632+0000 7f9e95800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x565007b1fcd0 0x565007b2e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:11.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:11.632+0000 7f9e94e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x565007b16e40 0x565007b1f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:11.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:11.632+0000 7f9e94e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x565007b16e40 0x565007b1f200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:46792/0 (socket says 172.21.15.5:46792) 2024-09-10T09:57:11.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:11.632+0000 7f9e95800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x565007b1fcd0 0x565007b2e090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59916/0 (socket says 172.21.15.5:59916) 2024-09-10T09:57:11.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:11.632+0000 7f9e94e00640 1 -- 172.21.15.5:0/1148409693 learned_addr learned my addr 172.21.15.5:0/1148409693 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:57:11.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:57:11 smithi102 ceph-mon[35442]: pgmap v958: 129 pgs: 129 active+clean; 721 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 13 MiB/s wr, 429 op/s 2024-09-10T09:57:11.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:57:11 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:57:11.635 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:57:11 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:57:11.635 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:57:11 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:57:11.635 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:57:11 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:57:11.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:11.632+0000 7f9e87e00640 1 --2- 172.21.15.5:0/1148409693 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x565007b22020 0x565007b1f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:11.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:11.632+0000 7f9e9a433740 1 -- 172.21.15.5:0/1148409693 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x565007b24bf0 con 0x565007b1fcd0 2024-09-10T09:57:11.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:11.632+0000 7f9e9a433740 1 -- 172.21.15.5:0/1148409693 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x565007a380f0 con 0x565007b22020 2024-09-10T09:57:11.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:11.632+0000 7f9e9a433740 1 -- 172.21.15.5:0/1148409693 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x565007b24d70 con 0x565007b16e40 2024-09-10T09:57:11.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:11.632+0000 7f9e9a433740 10 client.0 _ll_register_callbacks cb 0x565007a1fbd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:57:11.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:11.632+0000 7f9e94e00640 1 -- 172.21.15.5:0/1148409693 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x565007b22020 msgr2=0x565007b1f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:57:11.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:11.632+0000 7f9e94e00640 1 --2- 172.21.15.5:0/1148409693 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x565007b22020 0x565007b1f740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:11.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:11.632+0000 7f9e94e00640 1 -- 172.21.15.5:0/1148409693 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x565007b1fcd0 msgr2=0x565007b2e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:57:11.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:11.632+0000 7f9e94e00640 1 --2- 172.21.15.5:0/1148409693 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x565007b1fcd0 0x565007b2e090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:11.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:11.632+0000 7f9e94e00640 1 -- 172.21.15.5:0/1148409693 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5650077ee3c0 con 0x565007b16e40 2024-09-10T09:57:11.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:11.632+0000 7f9e94e00640 1 --2- 172.21.15.5:0/1148409693 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x565007b16e40 0x565007b1f200 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f9e7c000c00 tx=0x7f9e7c09b480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d2b6d7e7f95989b1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:57:11.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:11.632+0000 7f9e95800640 1 --2- 172.21.15.5:0/1148409693 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x565007b1fcd0 0x565007b2e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello state changed while learned_addr, mark_down or replacing must be happened just now 2024-09-10T09:57:11.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:11.632+0000 7f9e82e00640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-09-10T09:57:11.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:11.632+0000 7f9e82e00640 1 -- 172.21.15.5:0/1148409693 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e7c0a39f0 con 0x565007b16e40 2024-09-10T09:57:11.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:11.632+0000 7f9e9a433740 1 -- 172.21.15.5:0/1148409693 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9e740047a0 con 0x565007b16e40 2024-09-10T09:57:11.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:11.632+0000 7f9e82e00640 1 -- 172.21.15.5:0/1148409693 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f9e7c0973f0 con 0x565007b16e40 2024-09-10T09:57:11.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:11.632+0000 7f9e87e00640 1 --2- 172.21.15.5:0/1148409693 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x565007b22020 0x565007b1f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:57:11.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:11.632+0000 7f9e82e00640 1 -- 172.21.15.5:0/1148409693 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f9e7c09a1d0 con 0x565007b16e40 2024-09-10T09:57:11.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:11.632+0000 7f9e9a433740 10 client.29050 fetch_fsmap learned FSMap version 42 2024-09-10T09:57:11.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:11.632+0000 7f9e9a433740 1 -- 172.21.15.5:0/1148409693 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x565007b31390 con 0x565007b16e40 2024-09-10T09:57:11.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:11.632+0000 7f9e82e00640 1 -- 172.21.15.5:0/1148409693 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9e740047a0 con 0x565007b16e40 2024-09-10T09:57:11.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:11.637+0000 7f9e82e00640 1 -- 172.21.15.5:0/1148409693 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 42) ==== 8370+0+0 (secure 0 0 0) 0x7f9e7c099c40 con 0x565007b16e40 2024-09-10T09:57:11.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:11.637+0000 7f9e9a433740 10 client.29050 fetch_fsmap finished waiting for FSMap version 42 2024-09-10T09:57:11.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:11.637+0000 7f9e9a433740 10 client.29050 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-09-10T09:57:11.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:11.637+0000 7f9e9a433740 20 client.29050 populate_metadata read hostname 'smithi005' 2024-09-10T09:57:11.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:11.637+0000 7f9e9a433740 1 --2- 172.21.15.5:0/1148409693 >> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] conn(0x565007b327a0 0x565007b34b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:11.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:11.637+0000 7f9e9a433740 4 client.29050 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:57:11.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:11.637+0000 7f9e9a433740 1 -- 172.21.15.5:0/1148409693 --> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x565007a8a950 con 0x565007b327a0 2024-09-10T09:57:11.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:11.637+0000 7f9e87e00640 1 --2- 172.21.15.5:0/1148409693 >> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] conn(0x565007b327a0 0x565007b34b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:11.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:11.637+0000 7f9e87e00640 1 --2- 172.21.15.5:0/1148409693 >> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] conn(0x565007b327a0 0x565007b34b90 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=3c5402906d9cc5f server_cookie=3038f05813733633 in_seq=0 out_seq=0 2024-09-10T09:57:11.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:11.637+0000 7f9e82e00640 10 client.29050 ms_handle_connect on v2:172.21.15.102:6836/2689776764 2024-09-10T09:57:11.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:11.637+0000 7f9e82e00640 1 -- 172.21.15.5:0/1148409693 <== mds.3 v2:172.21.15.102:6836/2689776764 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x565007a8a950 con 0x565007b327a0 2024-09-10T09:57:11.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:11.637+0000 7f9e82e00640 10 client.29050 handle_command_reply: tid=0 2024-09-10T09:57:11.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:11.637+0000 7f9e9a433740 1 client.29050 shutdown 2024-09-10T09:57:11.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:11.637+0000 7f9e9a433740 1 -- 172.21.15.5:0/1148409693 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x565007b16e40 msgr2=0x565007b1f200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:57:11.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:11.637+0000 7f9e9a433740 1 --2- 172.21.15.5:0/1148409693 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x565007b16e40 0x565007b1f200 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f9e7c000c00 tx=0x7f9e7c09b480 comp rx=0 tx=0).stop 2024-09-10T09:57:11.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:11.642+0000 7f9e9a433740 1 -- 172.21.15.5:0/1148409693 shutdown_connections 2024-09-10T09:57:11.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:11.642+0000 7f9e9a433740 1 --2- 172.21.15.5:0/1148409693 >> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] conn(0x565007b327a0 0x565007b34b90 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:11.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:11.642+0000 7f9e9a433740 1 --2- 172.21.15.5:0/1148409693 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x565007b1fcd0 0x565007b2e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:11.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:11.642+0000 7f9e9a433740 1 --2- 172.21.15.5:0/1148409693 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x565007b22020 0x565007b1f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:11.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:11.642+0000 7f9e9a433740 1 --2- 172.21.15.5:0/1148409693 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x565007b16e40 0x565007b1f200 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:11.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:11.642+0000 7f9e9a433740 1 -- 172.21.15.5:0/1148409693 >> 172.21.15.5:0/1148409693 conn(0x565007b133d0 msgr2=0x565007b19100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:57:11.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:11.642+0000 7f9e82e00640 0 client.29050 ms_handle_reset on v2:172.21.15.102:6836/2689776764 2024-09-10T09:57:11.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:11.642+0000 7f9e9a433740 1 -- 172.21.15.5:0/1148409693 shutdown_connections 2024-09-10T09:57:11.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:11.642+0000 7f9e9a433740 1 -- 172.21.15.5:0/1148409693 wait complete. 2024-09-10T09:57:11.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:11.642+0000 7f9e9a433740 20 client.29050 trim_cache size 0 max 16384 2024-09-10T09:57:11.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:11.655+0000 7f9e9a433740 1 Processor -- start 2024-09-10T09:57:11.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:11.655+0000 7f9e9a433740 1 -- start start 2024-09-10T09:57:11.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:11.655+0000 7f9e9a433740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x565007b16e40 0x565007b2de80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:11.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:11.655+0000 7f9e9a433740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x565007b1fcd0 0x565007b123a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:11.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:11.655+0000 7f9e9a433740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x565007b22020 0x565007b3a430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:11.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:11.655+0000 7f9e9a433740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f9e901bdbd0 con 0x565007b1fcd0 2024-09-10T09:57:11.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:11.655+0000 7f9e9a433740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f9e901bf710 con 0x565007b22020 2024-09-10T09:57:11.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:11.655+0000 7f9e9a433740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f9e901be2a0 con 0x565007b16e40 2024-09-10T09:57:11.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:11.655+0000 7f9e94e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x565007b16e40 0x565007b2de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:11.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:11.655+0000 7f9e87e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x565007b1fcd0 0x565007b123a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:11.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:11.655+0000 7f9e87e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x565007b1fcd0 0x565007b123a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59920/0 (socket says 172.21.15.5:59920) 2024-09-10T09:57:11.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:11.655+0000 7f9e87e00640 1 -- 172.21.15.5:0/3344775064 learned_addr learned my addr 172.21.15.5:0/3344775064 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:57:11.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:11.660+0000 7f9e94e00640 1 -- 172.21.15.5:0/3344775064 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x565007b22020 msgr2=0x565007b3a430 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T09:57:11.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:11.660+0000 7f9e94e00640 1 --2- 172.21.15.5:0/3344775064 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x565007b22020 0x565007b3a430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:11.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:11.660+0000 7f9e94e00640 1 -- 172.21.15.5:0/3344775064 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x565007b1fcd0 msgr2=0x565007b123a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:57:11.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:11.660+0000 7f9e94e00640 1 --2- 172.21.15.5:0/3344775064 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x565007b1fcd0 0x565007b123a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:11.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:11.660+0000 7f9e94e00640 1 -- 172.21.15.5:0/3344775064 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565007b24780 con 0x565007b16e40 2024-09-10T09:57:11.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:11.660+0000 7f9e94e00640 1 --2- 172.21.15.5:0/3344775064 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x565007b16e40 0x565007b2de80 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f9e7c099030 tx=0x7f9e7c010930 comp rx=0 tx=0).ready entity=mon.1 client_cookie=130e76cae8f88ce1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:57:11.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:11.660+0000 7f9e85600640 1 -- 172.21.15.5:0/3344775064 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e7c09a8c0 con 0x565007b16e40 2024-09-10T09:57:11.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:11.660+0000 7f9e85600640 1 -- 172.21.15.5:0/3344775064 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9e7c09a370 con 0x565007b16e40 2024-09-10T09:57:11.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:11.660+0000 7f9e85600640 1 -- 172.21.15.5:0/3344775064 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e7c09a8c0 con 0x565007b16e40 2024-09-10T09:57:11.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:11.660+0000 7f9e9a433740 1 -- 172.21.15.5:0/3344775064 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x565007b16e40 msgr2=0x565007b2de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:57:11.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:11.660+0000 7f9e9a433740 1 --2- 172.21.15.5:0/3344775064 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x565007b16e40 0x565007b2de80 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f9e7c099030 tx=0x7f9e7c010930 comp rx=0 tx=0).stop 2024-09-10T09:57:11.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:11.664+0000 7f9e9a433740 1 -- 172.21.15.5:0/3344775064 shutdown_connections 2024-09-10T09:57:11.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:11.664+0000 7f9e9a433740 1 --2- 172.21.15.5:0/3344775064 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x565007b22020 0x565007b3a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:11.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:11.664+0000 7f9e9a433740 1 --2- 172.21.15.5:0/3344775064 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x565007b1fcd0 0x565007b123a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:11.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:11.664+0000 7f9e9a433740 1 --2- 172.21.15.5:0/3344775064 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x565007b16e40 0x565007b2de80 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:11.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:11.664+0000 7f9e9a433740 1 -- 172.21.15.5:0/3344775064 >> 172.21.15.5:0/3344775064 conn(0x565007b133d0 msgr2=0x565007b11c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:57:11.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:11.665+0000 7f9e9a433740 1 -- 172.21.15.5:0/3344775064 shutdown_connections 2024-09-10T09:57:11.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:11.665+0000 7f9e9a433740 1 -- 172.21.15.5:0/3344775064 wait complete. 2024-09-10T09:57:11.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:11.665+0000 7f9e9a433740 1 -- start start 2024-09-10T09:57:11.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:11.665+0000 7f9e9a433740 1 Processor -- start 2024-09-10T09:57:11.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:11.665+0000 7f9e9a433740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x565007b16e40 0x565007b2a780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:11.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:11.665+0000 7f9e9a433740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x565007b1fcd0 0x565007b25ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:11.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:11.665+0000 7f9e9a433740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x565007b22020 0x565007b25ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:11.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:11.665+0000 7f9e9a433740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f9e901bf040 con 0x565007b22020 2024-09-10T09:57:11.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:11.665+0000 7f9e9a433740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f9e901be970 con 0x565007b1fcd0 2024-09-10T09:57:11.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:11.665+0000 7f9e9a433740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f9e901bbbb0 con 0x565007b16e40 2024-09-10T09:57:11.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:11.665+0000 7f9e94e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x565007b16e40 0x565007b2a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:11.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:11.665+0000 7f9e94e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x565007b16e40 0x565007b2a780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:46808/0 (socket says 172.21.15.5:46808) 2024-09-10T09:57:11.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:11.665+0000 7f9e94e00640 1 -- 172.21.15.5:0/2177741238 learned_addr learned my addr 172.21.15.5:0/2177741238 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:57:11.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:11.668+0000 7f9e87e00640 1 --2- 172.21.15.5:0/2177741238 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x565007b1fcd0 0x565007b25ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:11.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:11.668+0000 7f9e9a433740 10 client.0 _ll_register_callbacks cb 0x7f9e7c097020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:57:11.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:11.668+0000 7f9e95800640 1 --2- 172.21.15.5:0/2177741238 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x565007b22020 0x565007b25ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:11.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:11.668+0000 7f9e94e00640 1 -- 172.21.15.5:0/2177741238 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x565007b1fcd0 msgr2=0x565007b25ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:57:11.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:11.668+0000 7f9e94e00640 1 --2- 172.21.15.5:0/2177741238 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x565007b1fcd0 0x565007b25ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:11.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:11.668+0000 7f9e94e00640 1 -- 172.21.15.5:0/2177741238 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x565007b22020 msgr2=0x565007b25ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:57:11.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:11.668+0000 7f9e94e00640 1 --2- 172.21.15.5:0/2177741238 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x565007b22020 0x565007b25ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:11.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:11.668+0000 7f9e94e00640 1 -- 172.21.15.5:0/2177741238 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565007b267f0 con 0x565007b16e40 2024-09-10T09:57:11.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:11.668+0000 7f9e94e00640 1 --2- 172.21.15.5:0/2177741238 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x565007b16e40 0x565007b2a780 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f9e7c09ad10 tx=0x7f9e7c09f270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4367493656531758 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:57:11.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:11.668+0000 7f9e82e00640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-09-10T09:57:11.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:11.668+0000 7f9e82e00640 1 -- 172.21.15.5:0/2177741238 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e7c09a8c0 con 0x565007b16e40 2024-09-10T09:57:11.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:11.668+0000 7f9e82e00640 1 -- 172.21.15.5:0/2177741238 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f9e7c011a30 con 0x565007b16e40 2024-09-10T09:57:11.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:11.668+0000 7f9e9a433740 1 -- 172.21.15.5:0/2177741238 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9e7c09a370 con 0x565007b16e40 2024-09-10T09:57:11.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:11.668+0000 7f9e82e00640 1 -- 172.21.15.5:0/2177741238 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f9e7c0a0340 con 0x565007b16e40 2024-09-10T09:57:11.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:11.668+0000 7f9e9a433740 10 client.29056 fetch_fsmap learned FSMap version 42 2024-09-10T09:57:11.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:11.668+0000 7f9e9a433740 1 -- 172.21.15.5:0/2177741238 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x565007b9d430 con 0x565007b16e40 2024-09-10T09:57:11.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:11.668+0000 7f9e82e00640 1 -- 172.21.15.5:0/2177741238 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9e7c09a370 con 0x565007b16e40 2024-09-10T09:57:11.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:11.668+0000 7f9e82e00640 1 -- 172.21.15.5:0/2177741238 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 42) ==== 8370+0+0 (secure 0 0 0) 0x7f9e7c0853f0 con 0x565007b16e40 2024-09-10T09:57:11.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:11.668+0000 7f9e9a433740 10 client.29056 fetch_fsmap finished waiting for FSMap version 42 2024-09-10T09:57:11.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:11.668+0000 7f9e9a433740 10 client.29056 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-09-10T09:57:11.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:11.668+0000 7f9e9a433740 20 client.29056 populate_metadata read hostname 'smithi005' 2024-09-10T09:57:11.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:11.668+0000 7f9e9a433740 1 --2- 172.21.15.5:0/2177741238 >> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] conn(0x565007b9d900 0x565007b9fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:11.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:11.668+0000 7f9e9a433740 4 client.29056 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T09:57:11.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:11.668+0000 7f9e9a433740 1 -- 172.21.15.5:0/2177741238 --> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] -- command(tid 0: {"prefix": "damage ls"}) -- 0x565007a84e60 con 0x565007b9d900 2024-09-10T09:57:11.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:11.668+0000 7f9e87e00640 1 --2- 172.21.15.5:0/2177741238 >> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] conn(0x565007b9d900 0x565007b9fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:11.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:11.668+0000 7f9e87e00640 1 --2- 172.21.15.5:0/2177741238 >> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] conn(0x565007b9d900 0x565007b9fcf0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=91b9e994c2472518 server_cookie=35ce6e3830138053 in_seq=0 out_seq=0 2024-09-10T09:57:11.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:11.668+0000 7f9e82e00640 10 client.29056 ms_handle_connect on v2:172.21.15.102:6836/2689776764 2024-09-10T09:57:11.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:11.668+0000 7f9e82e00640 1 -- 172.21.15.5:0/2177741238 <== mds.3 v2:172.21.15.102:6836/2689776764 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x565007a84e60 con 0x565007b9d900 2024-09-10T09:57:11.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:11.668+0000 7f9e82e00640 10 client.29056 handle_command_reply: tid=0 2024-09-10T09:57:11.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:11.668+0000 7f9e9a433740 1 client.29056 shutdown 2024-09-10T09:57:11.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:11.668+0000 7f9e9a433740 1 -- 172.21.15.5:0/2177741238 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x565007b16e40 msgr2=0x565007b2a780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:57:11.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:11.668+0000 7f9e9a433740 1 --2- 172.21.15.5:0/2177741238 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x565007b16e40 0x565007b2a780 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f9e7c09ad10 tx=0x7f9e7c09f270 comp rx=0 tx=0).stop 2024-09-10T09:57:11.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:11.674+0000 7f9e9a433740 1 -- 172.21.15.5:0/2177741238 shutdown_connections 2024-09-10T09:57:11.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:11.674+0000 7f9e9a433740 1 --2- 172.21.15.5:0/2177741238 >> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] conn(0x565007b9d900 0x565007b9fcf0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:11.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:11.674+0000 7f9e9a433740 1 --2- 172.21.15.5:0/2177741238 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x565007b22020 0x565007b25ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:11.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:11.674+0000 7f9e9a433740 1 --2- 172.21.15.5:0/2177741238 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x565007b1fcd0 0x565007b25ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:11.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:11.674+0000 7f9e9a433740 1 --2- 172.21.15.5:0/2177741238 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x565007b16e40 0x565007b2a780 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:11.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:11.674+0000 7f9e9a433740 1 -- 172.21.15.5:0/2177741238 >> 172.21.15.5:0/2177741238 conn(0x565007b133d0 msgr2=0x565007b124b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:57:11.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:11.674+0000 7f9e82e00640 0 client.29056 ms_handle_reset on v2:172.21.15.102:6836/2689776764 2024-09-10T09:57:11.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:11.674+0000 7f9e9a433740 1 -- 172.21.15.5:0/2177741238 shutdown_connections 2024-09-10T09:57:11.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:11.674+0000 7f9e9a433740 1 -- 172.21.15.5:0/2177741238 wait complete. 2024-09-10T09:57:11.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:11.674+0000 7f9e9a433740 20 client.29056 trim_cache size 0 max 16384 2024-09-10T09:57:11.677 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-10T09:57:11.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:11.674+0000 7f9e96200640 1 -- 172.21.15.5:0/3075990412 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f9e6c0a2c40 msgr2=0x7f9e6c0a5100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:57:11.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:11.674+0000 7f9e96200640 1 --2- 172.21.15.5:0/3075990412 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f9e6c0a2c40 0x7f9e6c0a5100 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x565007b14750 tx=0x7f9e8800a500 comp rx=0 tx=0).stop 2024-09-10T09:57:11.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:11.674+0000 7f9e96200640 1 -- 172.21.15.5:0/3075990412 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9e900ffc00 msgr2=0x7f9e901a12c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:57:11.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:11.674+0000 7f9e96200640 1 --2- 172.21.15.5:0/3075990412 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9e900ffc00 0x7f9e901a12c0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f9e7c009fd0 tx=0x7f9e7c00b870 comp rx=0 tx=0).stop 2024-09-10T09:57:11.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:11.679+0000 7f9e96200640 1 -- 172.21.15.5:0/3075990412 shutdown_connections 2024-09-10T09:57:11.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:11.679+0000 7f9e96200640 1 --2- 172.21.15.5:0/3075990412 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f9e6c0a2c40 0x7f9e6c0a5100 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:11.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:11.679+0000 7f9e96200640 1 --2- 172.21.15.5:0/3075990412 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9e90104da0 0x7f9e9019f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:11.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:11.679+0000 7f9e96200640 1 --2- 172.21.15.5:0/3075990412 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9e90100520 0x7f9e901a1800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:11.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:11.679+0000 7f9e96200640 1 --2- 172.21.15.5:0/3075990412 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9e900ffc00 0x7f9e901a12c0 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:11.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:11.679+0000 7f9e96200640 1 -- 172.21.15.5:0/3075990412 >> 172.21.15.5:0/3075990412 conn(0x7f9e900fb920 msgr2=0x7f9e900fbd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:57:11.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:11.679+0000 7f9e96200640 1 -- 172.21.15.5:0/3075990412 shutdown_connections 2024-09-10T09:57:11.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:11.679+0000 7f9e96200640 1 -- 172.21.15.5:0/3075990412 wait complete. 2024-09-10T09:57:11.989 INFO:tasks.workunit.client.0.smithi005.stdout: 10 87528 15.35 MB/sec execute 170 sec latency 13806.003 ms 2024-09-10T09:57:12.950 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-10T09:57:12.989 INFO:tasks.workunit.client.0.smithi005.stdout: 10 87528 15.26 MB/sec execute 171 sec latency 14806.060 ms 2024-09-10T09:57:13.334 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:57:13.460 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:57:13 smithi005 ceph-mon[30350]: pgmap v959: 129 pgs: 129 active+clean; 714 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 499 KiB/s rd, 10 MiB/s wr, 292 op/s 2024-09-10T09:57:13.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:57:13 smithi159 ceph-mon[35513]: pgmap v959: 129 pgs: 129 active+clean; 714 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 499 KiB/s rd, 10 MiB/s wr, 292 op/s 2024-09-10T09:57:13.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:57:13 smithi102 ceph-mon[35442]: pgmap v959: 129 pgs: 129 active+clean; 714 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 499 KiB/s rd, 10 MiB/s wr, 292 op/s 2024-09-10T09:57:13.990 INFO:tasks.workunit.client.0.smithi005.stdout: 10 87528 15.17 MB/sec execute 172 sec latency 15806.112 ms 2024-09-10T09:57:14.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:14.144+0000 7fc2ca800640 1 -- 172.21.15.5:0/4062497519 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc2c41019c0 msgr2=0x7fc2c4101e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:57:14.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:14.144+0000 7fc2ca800640 1 --2- 172.21.15.5:0/4062497519 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc2c41019c0 0x7fc2c4101e20 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fc2b0009900 tx=0x7fc2b002a410 comp rx=0 tx=0).stop 2024-09-10T09:57:14.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:14.144+0000 7fc2ca800640 1 -- 172.21.15.5:0/4062497519 shutdown_connections 2024-09-10T09:57:14.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:14.144+0000 7fc2ca800640 1 --2- 172.21.15.5:0/4062497519 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc2c4102360 0x7fc2c410e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:14.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:14.144+0000 7fc2ca800640 1 --2- 172.21.15.5:0/4062497519 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc2c41019c0 0x7fc2c4101e20 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:14.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:14.144+0000 7fc2ca800640 1 --2- 172.21.15.5:0/4062497519 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc2c41079c0 0x7fc2c4107da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:14.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:14.144+0000 7fc2ca800640 1 -- 172.21.15.5:0/4062497519 >> 172.21.15.5:0/4062497519 conn(0x7fc2c40fd740 msgr2=0x7fc2c40ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:57:14.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:14.144+0000 7fc2ca800640 1 -- 172.21.15.5:0/4062497519 shutdown_connections 2024-09-10T09:57:14.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:14.144+0000 7fc2ca800640 1 -- 172.21.15.5:0/4062497519 wait complete. 2024-09-10T09:57:14.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:14.150+0000 7fc2ca800640 1 Processor -- start 2024-09-10T09:57:14.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:14.150+0000 7fc2ca800640 1 -- start start 2024-09-10T09:57:14.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:14.150+0000 7fc2ca800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc2c41019c0 0x7fc2c419a420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:14.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:14.150+0000 7fc2ca800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc2c4102360 0x7fc2c419a960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:14.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:14.150+0000 7fc2ca800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc2c41079c0 0x7fc2c419ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:14.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:14.150+0000 7fc2ca800640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc2c4111e10 con 0x7fc2c41079c0 2024-09-10T09:57:14.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:14.150+0000 7fc2ca800640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fc2c4111c90 con 0x7fc2c4102360 2024-09-10T09:57:14.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:14.150+0000 7fc2ca800640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fc2c4111f90 con 0x7fc2c41019c0 2024-09-10T09:57:14.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:14.150+0000 7fc2c9400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc2c41019c0 0x7fc2c419a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:14.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:14.150+0000 7fc2c9400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc2c41019c0 0x7fc2c419a420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:46830/0 (socket says 172.21.15.5:46830) 2024-09-10T09:57:14.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:14.150+0000 7fc2c9400640 1 -- 172.21.15.5:0/3684385172 learned_addr learned my addr 172.21.15.5:0/3684385172 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:57:14.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:14.150+0000 7fc2c8a00640 1 --2- 172.21.15.5:0/3684385172 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc2c4102360 0x7fc2c419a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:14.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:14.150+0000 7fc2c8a00640 1 -- 172.21.15.5:0/3684385172 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc2c41019c0 msgr2=0x7fc2c419a420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:57:14.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:14.150+0000 7fc2c8a00640 1 --2- 172.21.15.5:0/3684385172 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc2c41019c0 0x7fc2c419a420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:14.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:14.150+0000 7fc2c8a00640 1 -- 172.21.15.5:0/3684385172 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc2c41079c0 msgr2=0x7fc2c419ecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:57:14.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:14.150+0000 7fc2c8a00640 1 --2- 172.21.15.5:0/3684385172 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc2c41079c0 0x7fc2c419ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:14.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:14.150+0000 7fc2c8a00640 1 -- 172.21.15.5:0/3684385172 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2c419f3d0 con 0x7fc2c4102360 2024-09-10T09:57:14.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:14.155+0000 7fc2c8a00640 1 --2- 172.21.15.5:0/3684385172 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc2c4102360 0x7fc2c419a960 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fc2b002aa00 tx=0x7fc2b0003b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:57:14.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:14.155+0000 7fc2c2000640 1 -- 172.21.15.5:0/3684385172 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2b003f070 con 0x7fc2c4102360 2024-09-10T09:57:14.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:14.155+0000 7fc2c2000640 1 -- 172.21.15.5:0/3684385172 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc2b0031c20 con 0x7fc2c4102360 2024-09-10T09:57:14.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:14.155+0000 7fc2c2000640 1 -- 172.21.15.5:0/3684385172 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2b0035030 con 0x7fc2c4102360 2024-09-10T09:57:14.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:14.155+0000 7fc2ca800640 1 -- 172.21.15.5:0/3684385172 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2c419f660 con 0x7fc2c4102360 2024-09-10T09:57:14.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:14.156+0000 7fc2ca800640 1 -- 172.21.15.5:0/3684385172 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2c419fac0 con 0x7fc2c4102360 2024-09-10T09:57:14.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:14.162+0000 7fc2cea65740 1 Processor -- start 2024-09-10T09:57:14.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:14.162+0000 7fc2cea65740 1 -- start start 2024-09-10T09:57:14.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:14.162+0000 7fc2c2000640 1 -- 172.21.15.5:0/3684385172 <== mon.2 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7fc2b0042070 con 0x7fc2c4102360 2024-09-10T09:57:14.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:14.162+0000 7fc2cea65740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557966089650 0x557966089a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:14.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:14.162+0000 7fc2c9e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557966089650 0x557966089a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:14.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:14.162+0000 7fc2cea65740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557966089f70 0x5579660942f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:14.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:14.162+0000 7fc2cea65740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557966094830 0x557966096c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:14.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:14.162+0000 7fc2cea65740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x557965fe73e0 con 0x557966089650 2024-09-10T09:57:14.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:14.162+0000 7fc2cea65740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x557965f8ed80 con 0x557966094830 2024-09-10T09:57:14.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:14.162+0000 7fc2cea65740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x557965fe33a0 con 0x557966089f70 2024-09-10T09:57:14.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:14.162+0000 7fc2c9e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557966089650 0x557966089a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59966/0 (socket says 172.21.15.5:59966) 2024-09-10T09:57:14.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:14.162+0000 7fc2c9e00640 1 -- 172.21.15.5:0/1063432758 learned_addr learned my addr 172.21.15.5:0/1063432758 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:57:14.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:14.162+0000 7fc2c2000640 1 --2- 172.21.15.5:0/3684385172 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fc2ac0a2c40 0x7fc2ac0a5100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:14.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:14.162+0000 7fc2c9400640 1 --2- 172.21.15.5:0/1063432758 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557966089f70 0x5579660942f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:14.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:14.162+0000 7fc2c8a00640 1 --2- 172.21.15.5:0/1063432758 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557966094830 0x557966096c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:14.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:14.167+0000 7fc2c2000640 1 -- 172.21.15.5:0/3684385172 <== mon.2 v2:172.21.15.102:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fc2b00efb60 con 0x7fc2c4102360 2024-09-10T09:57:14.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:14.167+0000 7fc2c9e00640 1 --2- 172.21.15.5:0/3684385172 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fc2ac0a2c40 0x7fc2ac0a5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:14.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:14.168+0000 7fc2c9400640 1 -- 172.21.15.5:0/1063432758 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557966094830 msgr2=0x557966096c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:57:14.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:14.168+0000 7fc2c9400640 1 --2- 172.21.15.5:0/1063432758 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557966094830 0x557966096c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:14.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:14.168+0000 7fc2c9400640 1 -- 172.21.15.5:0/1063432758 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557966089650 msgr2=0x557966089a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:57:14.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:14.168+0000 7fc2c9400640 1 --2- 172.21.15.5:0/1063432758 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557966089650 0x557966089a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:14.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:14.168+0000 7fc2c9400640 1 -- 172.21.15.5:0/1063432758 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557965fadae0 con 0x557966089f70 2024-09-10T09:57:14.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:14.168+0000 7fc2c9400640 1 --2- 172.21.15.5:0/1063432758 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557966089f70 0x5579660942f0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fc2b80058c0 tx=0x7fc2b80107a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=99ca3883ef624b73 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:57:14.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:14.168+0000 7fc2b6600640 1 -- 172.21.15.5:0/1063432758 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2b8004020 con 0x557966089f70 2024-09-10T09:57:14.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:14.168+0000 7fc2b6600640 1 -- 172.21.15.5:0/1063432758 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc2b80041c0 con 0x557966089f70 2024-09-10T09:57:14.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:14.168+0000 7fc2b6600640 1 -- 172.21.15.5:0/1063432758 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2b80117c0 con 0x557966089f70 2024-09-10T09:57:14.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:14.168+0000 7fc2c9e00640 1 --2- 172.21.15.5:0/3684385172 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fc2ac0a2c40 0x7fc2ac0a5100 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fc2bc013b70 tx=0x7fc2bc005e90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:57:14.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:14.168+0000 7fc2cea65740 1 -- 172.21.15.5:0/1063432758 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557966089f70 msgr2=0x5579660942f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:57:14.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:14.168+0000 7fc2cea65740 1 --2- 172.21.15.5:0/1063432758 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557966089f70 0x5579660942f0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fc2b80058c0 tx=0x7fc2b80107a0 comp rx=0 tx=0).stop 2024-09-10T09:57:14.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:14.168+0000 7fc2cea65740 1 -- 172.21.15.5:0/1063432758 shutdown_connections 2024-09-10T09:57:14.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:14.168+0000 7fc2cea65740 1 --2- 172.21.15.5:0/1063432758 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557966094830 0x557966096c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:14.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:14.168+0000 7fc2cea65740 1 --2- 172.21.15.5:0/1063432758 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557966089f70 0x5579660942f0 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:14.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:14.168+0000 7fc2cea65740 1 --2- 172.21.15.5:0/1063432758 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557966089650 0x557966089a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:14.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:14.168+0000 7fc2cea65740 1 -- 172.21.15.5:0/1063432758 >> 172.21.15.5:0/1063432758 conn(0x557966085be0 msgr2=0x557966085fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:57:14.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:14.168+0000 7fc2cea65740 1 -- 172.21.15.5:0/1063432758 shutdown_connections 2024-09-10T09:57:14.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:14.168+0000 7fc2cea65740 1 -- 172.21.15.5:0/1063432758 wait complete. 2024-09-10T09:57:14.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:14.168+0000 7fc2cea65740 1 -- start start 2024-09-10T09:57:14.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:14.174+0000 7fc2cea65740 1 Processor -- start 2024-09-10T09:57:14.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:14.174+0000 7fc2cea65740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557966089650 0x557966091650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:14.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:14.174+0000 7fc2cea65740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557966089f70 0x557966091b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:14.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:14.174+0000 7fc2cea65740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557966094830 0x5579660920d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:14.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:14.174+0000 7fc2cea65740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x557966097400 con 0x557966089f70 2024-09-10T09:57:14.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:14.174+0000 7fc2cea65740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x557965fa20f0 con 0x557966089650 2024-09-10T09:57:14.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:14.174+0000 7fc2c9400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557966089650 0x557966091650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:14.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:14.174+0000 7fc2c9400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557966089650 0x557966091650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:34768/0 (socket says 172.21.15.5:34768) 2024-09-10T09:57:14.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:14.174+0000 7fc2c9400640 1 -- 172.21.15.5:0/3022074221 learned_addr learned my addr 172.21.15.5:0/3022074221 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:57:14.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:14.174+0000 7fc2c8a00640 1 --2- 172.21.15.5:0/3022074221 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557966089f70 0x557966091b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:14.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:14.174+0000 7fc2cea65740 1 -- 172.21.15.5:0/3022074221 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x557966097580 con 0x557966094830 2024-09-10T09:57:14.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:14.174+0000 7fc2c9e00640 1 --2- 172.21.15.5:0/3022074221 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557966094830 0x5579660920d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:14.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:14.174+0000 7fc2c9400640 1 -- 172.21.15.5:0/3022074221 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557966094830 msgr2=0x5579660920d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:57:14.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:14.174+0000 7fc2c9400640 1 --2- 172.21.15.5:0/3022074221 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557966094830 0x5579660920d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:14.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:14.174+0000 7fc2c9400640 1 -- 172.21.15.5:0/3022074221 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557966089f70 msgr2=0x557966091b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:57:14.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:14.174+0000 7fc2c9400640 1 --2- 172.21.15.5:0/3022074221 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557966089f70 0x557966091b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:14.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:14.174+0000 7fc2c9400640 1 -- 172.21.15.5:0/3022074221 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557965d583c0 con 0x557966089650 2024-09-10T09:57:14.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:14.174+0000 7fc2c9e00640 1 --2- 172.21.15.5:0/3022074221 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557966094830 0x5579660920d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:57:14.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:14.174+0000 7fc2c8a00640 1 --2- 172.21.15.5:0/3022074221 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557966089f70 0x557966091b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:57:14.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:14.174+0000 7fc2cea65740 10 client.0 _ll_register_callbacks cb 0x557965f89bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:57:14.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:14.174+0000 7fc2cea65740 1 -- 172.21.15.5:0/3022074221 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc2b80041c0 con 0x557966089650 2024-09-10T09:57:14.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:14.179+0000 7fc2c9400640 1 --2- 172.21.15.5:0/3022074221 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557966089650 0x557966091650 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fc2b8006010 tx=0x7fc2b8003cc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=20f9059075d4f0be server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:57:14.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:14.179+0000 7fc2aaa00640 10 client.29069 ms_handle_connect on v2:172.21.15.102:3300/0 2024-09-10T09:57:14.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:14.179+0000 7fc2aaa00640 1 -- 172.21.15.5:0/3022074221 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2b802b060 con 0x557966089650 2024-09-10T09:57:14.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:14.179+0000 7fc2aaa00640 1 -- 172.21.15.5:0/3022074221 <== mon.2 v2:172.21.15.102:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fc2b8004370 con 0x557966089650 2024-09-10T09:57:14.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:14.179+0000 7fc2aaa00640 1 -- 172.21.15.5:0/3022074221 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7fc2b80035f0 con 0x557966089650 2024-09-10T09:57:14.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:14.179+0000 7fc2cea65740 10 client.29069 fetch_fsmap learned FSMap version 42 2024-09-10T09:57:14.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:14.179+0000 7fc2cea65740 1 -- 172.21.15.5:0/3022074221 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({fsmap=42}) -- 0x5579660a3240 con 0x557966089650 2024-09-10T09:57:14.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:14.180+0000 7fc2aaa00640 1 -- 172.21.15.5:0/3022074221 <== mon.2 v2:172.21.15.102:3300/0 4 ==== fsmap(e 42) ==== 8370+0+0 (secure 0 0 0) 0x7fc2b8003790 con 0x557966089650 2024-09-10T09:57:14.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:14.180+0000 7fc2cea65740 10 client.29069 fetch_fsmap finished waiting for FSMap version 42 2024-09-10T09:57:14.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:14.180+0000 7fc2cea65740 10 client.29069 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-10T09:57:14.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:14.180+0000 7fc2cea65740 20 client.29069 populate_metadata read hostname 'smithi005' 2024-09-10T09:57:14.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:14.180+0000 7fc2cea65740 1 --2- 172.21.15.5:0/3022074221 >> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] conn(0x5579660a4970 0x5579660a6e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:14.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:14.180+0000 7fc2cea65740 4 client.29069 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:57:14.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:14.180+0000 7fc2cea65740 1 -- 172.21.15.5:0/3022074221 --> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557965ff4950 con 0x5579660a4970 2024-09-10T09:57:14.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:14.180+0000 7fc2c8a00640 1 --2- 172.21.15.5:0/3022074221 >> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] conn(0x5579660a4970 0x5579660a6e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:14.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:14.180+0000 7fc2c8a00640 1 --2- 172.21.15.5:0/3022074221 >> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] conn(0x5579660a4970 0x5579660a6e30 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=99cf1035eb28baec server_cookie=2996884d8d43bcdc in_seq=0 out_seq=0 2024-09-10T09:57:14.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:14.180+0000 7fc2aaa00640 10 client.29069 ms_handle_connect on v2:172.21.15.159:6836/620627639 2024-09-10T09:57:14.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:14.180+0000 7fc2aaa00640 1 -- 172.21.15.5:0/3022074221 <== mds.2 v2:172.21.15.159:6836/620627639 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557965ff4950 con 0x5579660a4970 2024-09-10T09:57:14.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:14.180+0000 7fc2aaa00640 10 client.29069 handle_command_reply: tid=0 2024-09-10T09:57:14.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:14.180+0000 7fc2cea65740 1 client.29069 shutdown 2024-09-10T09:57:14.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:14.180+0000 7fc2cea65740 1 -- 172.21.15.5:0/3022074221 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557966089650 msgr2=0x557966091650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:57:14.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:14.180+0000 7fc2cea65740 1 --2- 172.21.15.5:0/3022074221 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557966089650 0x557966091650 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fc2b8006010 tx=0x7fc2b8003cc0 comp rx=0 tx=0).stop 2024-09-10T09:57:14.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:14.180+0000 7fc2cea65740 1 -- 172.21.15.5:0/3022074221 shutdown_connections 2024-09-10T09:57:14.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:14.180+0000 7fc2cea65740 1 --2- 172.21.15.5:0/3022074221 >> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] conn(0x5579660a4970 0x5579660a6e30 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:14.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:14.180+0000 7fc2cea65740 1 --2- 172.21.15.5:0/3022074221 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557966094830 0x5579660920d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:14.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:14.180+0000 7fc2cea65740 1 --2- 172.21.15.5:0/3022074221 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557966089f70 0x557966091b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:14.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:14.180+0000 7fc2cea65740 1 --2- 172.21.15.5:0/3022074221 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557966089650 0x557966091650 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:14.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:14.180+0000 7fc2cea65740 1 -- 172.21.15.5:0/3022074221 >> 172.21.15.5:0/3022074221 conn(0x557966085be0 msgr2=0x55796607c490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:57:14.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:14.180+0000 7fc2aaa00640 0 client.29069 ms_handle_reset on v2:172.21.15.159:6836/620627639 2024-09-10T09:57:14.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:14.180+0000 7fc2cea65740 1 -- 172.21.15.5:0/3022074221 shutdown_connections 2024-09-10T09:57:14.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:14.180+0000 7fc2cea65740 1 -- 172.21.15.5:0/3022074221 wait complete. 2024-09-10T09:57:14.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:14.180+0000 7fc2cea65740 20 client.29069 trim_cache size 0 max 16384 2024-09-10T09:57:14.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:14.210+0000 7fc2cea65740 1 Processor -- start 2024-09-10T09:57:14.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:14.210+0000 7fc2cea65740 1 -- start start 2024-09-10T09:57:14.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:14.210+0000 7fc2cea65740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557966089650 0x557966078300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:14.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:14.210+0000 7fc2cea65740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557966089f70 0x5579660ac740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:14.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:14.210+0000 7fc2cea65740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557966094830 0x5579660acc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:14.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:14.210+0000 7fc2cea65740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc2c41acae0 con 0x557966089650 2024-09-10T09:57:14.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:14.210+0000 7fc2cea65740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fc2c41ae620 con 0x557966089f70 2024-09-10T09:57:14.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:14.210+0000 7fc2cea65740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fc2c41ad1b0 con 0x557966094830 2024-09-10T09:57:14.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:14.210+0000 7fc2c9e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557966094830 0x5579660acc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:14.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:14.210+0000 7fc2c9e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557966094830 0x5579660acc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:46858/0 (socket says 172.21.15.5:46858) 2024-09-10T09:57:14.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:14.210+0000 7fc2c9e00640 1 -- 172.21.15.5:0/2561879031 learned_addr learned my addr 172.21.15.5:0/2561879031 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:57:14.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:14.210+0000 7fc2c8a00640 1 --2- 172.21.15.5:0/2561879031 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557966089f70 0x5579660ac740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:14.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:14.210+0000 7fc2c9400640 1 --2- 172.21.15.5:0/2561879031 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557966089650 0x557966078300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:14.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:14.210+0000 7fc2c9e00640 1 -- 172.21.15.5:0/2561879031 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557966089f70 msgr2=0x5579660ac740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:57:14.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:14.210+0000 7fc2c9e00640 1 --2- 172.21.15.5:0/2561879031 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557966089f70 0x5579660ac740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:14.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:14.210+0000 7fc2c9e00640 1 -- 172.21.15.5:0/2561879031 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557966089650 msgr2=0x557966078300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:57:14.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:14.210+0000 7fc2c9e00640 1 --2- 172.21.15.5:0/2561879031 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557966089650 0x557966078300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:14.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:14.210+0000 7fc2c9e00640 1 -- 172.21.15.5:0/2561879031 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55796608c4c0 con 0x557966094830 2024-09-10T09:57:14.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:14.210+0000 7fc2c9400640 1 --2- 172.21.15.5:0/2561879031 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557966089650 0x557966078300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T09:57:14.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:14.210+0000 7fc2c9e00640 1 --2- 172.21.15.5:0/2561879031 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557966094830 0x5579660acc80 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fc2bc0137e0 tx=0x7fc2bc007970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cb39b1da44b357f6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:57:14.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:14.210+0000 7fc2b5c00640 1 -- 172.21.15.5:0/2561879031 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2bc0108d0 con 0x557966094830 2024-09-10T09:57:14.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:14.215+0000 7fc2b5c00640 1 -- 172.21.15.5:0/2561879031 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc2bc025070 con 0x557966094830 2024-09-10T09:57:14.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:14.215+0000 7fc2b5c00640 1 -- 172.21.15.5:0/2561879031 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2bc020400 con 0x557966094830 2024-09-10T09:57:14.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:14.215+0000 7fc2cea65740 1 -- 172.21.15.5:0/2561879031 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557966094830 msgr2=0x5579660acc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:57:14.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:14.215+0000 7fc2cea65740 1 --2- 172.21.15.5:0/2561879031 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557966094830 0x5579660acc80 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fc2bc0137e0 tx=0x7fc2bc007970 comp rx=0 tx=0).stop 2024-09-10T09:57:14.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:14.216+0000 7fc2cea65740 1 -- 172.21.15.5:0/2561879031 shutdown_connections 2024-09-10T09:57:14.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:14.216+0000 7fc2cea65740 1 --2- 172.21.15.5:0/2561879031 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557966094830 0x5579660acc80 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:14.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:14.216+0000 7fc2cea65740 1 --2- 172.21.15.5:0/2561879031 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557966089f70 0x5579660ac740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:14.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:14.216+0000 7fc2cea65740 1 --2- 172.21.15.5:0/2561879031 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557966089650 0x557966078300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:14.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:14.216+0000 7fc2cea65740 1 -- 172.21.15.5:0/2561879031 >> 172.21.15.5:0/2561879031 conn(0x557966085be0 msgr2=0x557966090740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:57:14.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:14.216+0000 7fc2cea65740 1 -- 172.21.15.5:0/2561879031 shutdown_connections 2024-09-10T09:57:14.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:14.216+0000 7fc2cea65740 1 -- 172.21.15.5:0/2561879031 wait complete. 2024-09-10T09:57:14.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:14.216+0000 7fc2cea65740 1 -- start start 2024-09-10T09:57:14.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:14.216+0000 7fc2cea65740 1 Processor -- start 2024-09-10T09:57:14.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:14.216+0000 7fc2cea65740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557966089650 0x55796610a980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:14.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:14.216+0000 7fc2cea65740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557966089f70 0x55796610aec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:14.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:14.216+0000 7fc2cea65740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557966094830 0x55796609c800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:14.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:14.216+0000 7fc2cea65740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc2c41adf50 con 0x557966089650 2024-09-10T09:57:14.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:14.216+0000 7fc2cea65740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fc2c41ad880 con 0x557966094830 2024-09-10T09:57:14.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:14.216+0000 7fc2cea65740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fc2c41aaae0 con 0x557966089f70 2024-09-10T09:57:14.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:14.216+0000 7fc2cea65740 10 client.0 _ll_register_callbacks cb 0x7fc2bc007480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:57:14.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:14.216+0000 7fc2c8a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557966089f70 0x55796610aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:14.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:14.216+0000 7fc2c9e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557966094830 0x55796609c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:14.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:14.216+0000 7fc2c8a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557966089f70 0x55796610aec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:46860/0 (socket says 172.21.15.5:46860) 2024-09-10T09:57:14.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:14.216+0000 7fc2c8a00640 1 -- 172.21.15.5:0/2908159439 learned_addr learned my addr 172.21.15.5:0/2908159439 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:57:14.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:14.216+0000 7fc2c9400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557966089650 0x55796610a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:14.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:14.216+0000 7fc2c8a00640 1 -- 172.21.15.5:0/2908159439 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557966094830 msgr2=0x55796609c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:57:14.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:14.216+0000 7fc2c8a00640 1 --2- 172.21.15.5:0/2908159439 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557966094830 0x55796609c800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:14.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:14.216+0000 7fc2c8a00640 1 -- 172.21.15.5:0/2908159439 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557966089650 msgr2=0x55796610a980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:57:14.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:14.216+0000 7fc2c8a00640 1 --2- 172.21.15.5:0/2908159439 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557966089650 0x55796610a980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:14.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:14.216+0000 7fc2c8a00640 1 -- 172.21.15.5:0/2908159439 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55796609cf40 con 0x557966089f70 2024-09-10T09:57:14.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:14.228+0000 7fc2c8a00640 1 --2- 172.21.15.5:0/2908159439 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557966089f70 0x55796610aec0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fc2b0004650 tx=0x7fc2b003b890 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4a9c2c0ffbb23fc1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:57:14.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:14.228+0000 7fc2aaa00640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-09-10T09:57:14.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:14.228+0000 7fc2aaa00640 1 -- 172.21.15.5:0/2908159439 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2b00d4070 con 0x557966089f70 2024-09-10T09:57:14.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:14.228+0000 7fc2aaa00640 1 -- 172.21.15.5:0/2908159439 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fc2b00ba2d0 con 0x557966089f70 2024-09-10T09:57:14.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:14.228+0000 7fc2cea65740 1 -- 172.21.15.5:0/2908159439 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc2bc025070 con 0x557966089f70 2024-09-10T09:57:14.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:14.228+0000 7fc2aaa00640 1 -- 172.21.15.5:0/2908159439 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7fc2b0049040 con 0x557966089f70 2024-09-10T09:57:14.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:14.228+0000 7fc2cea65740 10 client.29071 fetch_fsmap learned FSMap version 42 2024-09-10T09:57:14.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:14.228+0000 7fc2cea65740 1 -- 172.21.15.5:0/2908159439 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55796608eab0 con 0x557966089f70 2024-09-10T09:57:14.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:14.228+0000 7fc2aaa00640 1 -- 172.21.15.5:0/2908159439 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc2bc025070 con 0x557966089f70 2024-09-10T09:57:14.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:14.228+0000 7fc2aaa00640 1 -- 172.21.15.5:0/2908159439 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 42) ==== 8370+0+0 (secure 0 0 0) 0x7fc2b00b8640 con 0x557966089f70 2024-09-10T09:57:14.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:14.228+0000 7fc2cea65740 10 client.29071 fetch_fsmap finished waiting for FSMap version 42 2024-09-10T09:57:14.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:14.228+0000 7fc2cea65740 10 client.29071 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-10T09:57:14.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:14.228+0000 7fc2cea65740 20 client.29071 populate_metadata read hostname 'smithi005' 2024-09-10T09:57:14.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:14.228+0000 7fc2cea65740 1 --2- 172.21.15.5:0/2908159439 >> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] conn(0x557966080fb0 0x5579660833d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:14.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:14.228+0000 7fc2cea65740 4 client.29071 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T09:57:14.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:14.228+0000 7fc2cea65740 1 -- 172.21.15.5:0/2908159439 --> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557965feee60 con 0x557966080fb0 2024-09-10T09:57:14.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:14.234+0000 7fc2c9400640 1 --2- 172.21.15.5:0/2908159439 >> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] conn(0x557966080fb0 0x5579660833d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:14.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:14.234+0000 7fc2c9400640 1 --2- 172.21.15.5:0/2908159439 >> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] conn(0x557966080fb0 0x5579660833d0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d5e81081158a3431 server_cookie=a3b85c202a71b660 in_seq=0 out_seq=0 2024-09-10T09:57:14.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:14.234+0000 7fc2aaa00640 10 client.29071 ms_handle_connect on v2:172.21.15.159:6836/620627639 2024-09-10T09:57:14.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:14.234+0000 7fc2aaa00640 1 -- 172.21.15.5:0/2908159439 <== mds.2 v2:172.21.15.159:6836/620627639 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557965feee60 con 0x557966080fb0 2024-09-10T09:57:14.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:14.234+0000 7fc2aaa00640 10 client.29071 handle_command_reply: tid=0 2024-09-10T09:57:14.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:14.234+0000 7fc2cea65740 1 client.29071 shutdown 2024-09-10T09:57:14.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:14.234+0000 7fc2cea65740 1 -- 172.21.15.5:0/2908159439 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557966089f70 msgr2=0x55796610aec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:57:14.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:14.234+0000 7fc2cea65740 1 --2- 172.21.15.5:0/2908159439 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557966089f70 0x55796610aec0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fc2b0004650 tx=0x7fc2b003b890 comp rx=0 tx=0).stop 2024-09-10T09:57:14.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:14.234+0000 7fc2cea65740 1 -- 172.21.15.5:0/2908159439 shutdown_connections 2024-09-10T09:57:14.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:14.234+0000 7fc2cea65740 1 --2- 172.21.15.5:0/2908159439 >> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] conn(0x557966080fb0 0x5579660833d0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:14.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:14.234+0000 7fc2cea65740 1 --2- 172.21.15.5:0/2908159439 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x557966094830 0x55796609c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:14.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:14.234+0000 7fc2cea65740 1 --2- 172.21.15.5:0/2908159439 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x557966089f70 0x55796610aec0 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:14.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:14.234+0000 7fc2cea65740 1 --2- 172.21.15.5:0/2908159439 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557966089650 0x55796610a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:14.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:14.234+0000 7fc2aaa00640 0 client.29071 ms_handle_reset on v2:172.21.15.159:6836/620627639 2024-09-10T09:57:14.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:14.234+0000 7fc2cea65740 1 -- 172.21.15.5:0/2908159439 >> 172.21.15.5:0/2908159439 conn(0x557966085be0 msgr2=0x55796607bc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:57:14.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:14.234+0000 7fc2cea65740 1 -- 172.21.15.5:0/2908159439 shutdown_connections 2024-09-10T09:57:14.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:14.234+0000 7fc2cea65740 1 -- 172.21.15.5:0/2908159439 wait complete. 2024-09-10T09:57:14.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:14.234+0000 7fc2cea65740 20 client.29071 trim_cache size 0 max 16384 2024-09-10T09:57:14.241 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-10T09:57:14.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:14.240+0000 7fc2ca800640 1 -- 172.21.15.5:0/3684385172 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fc2ac0a2c40 msgr2=0x7fc2ac0a5100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:57:14.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:14.240+0000 7fc2ca800640 1 --2- 172.21.15.5:0/3684385172 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fc2ac0a2c40 0x7fc2ac0a5100 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fc2bc013b70 tx=0x7fc2bc005e90 comp rx=0 tx=0).stop 2024-09-10T09:57:14.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:14.240+0000 7fc2ca800640 1 -- 172.21.15.5:0/3684385172 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc2c4102360 msgr2=0x7fc2c419a960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:57:14.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:14.240+0000 7fc2ca800640 1 --2- 172.21.15.5:0/3684385172 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc2c4102360 0x7fc2c419a960 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fc2b002aa00 tx=0x7fc2b0003b90 comp rx=0 tx=0).stop 2024-09-10T09:57:14.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:14.240+0000 7fc2ca800640 1 -- 172.21.15.5:0/3684385172 shutdown_connections 2024-09-10T09:57:14.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:14.240+0000 7fc2ca800640 1 --2- 172.21.15.5:0/3684385172 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fc2ac0a2c40 0x7fc2ac0a5100 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:14.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:14.240+0000 7fc2ca800640 1 --2- 172.21.15.5:0/3684385172 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc2c41079c0 0x7fc2c419ecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:14.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:14.240+0000 7fc2ca800640 1 --2- 172.21.15.5:0/3684385172 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc2c4102360 0x7fc2c419a960 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:14.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:14.240+0000 7fc2ca800640 1 --2- 172.21.15.5:0/3684385172 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc2c41019c0 0x7fc2c419a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:14.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:14.240+0000 7fc2ca800640 1 -- 172.21.15.5:0/3684385172 >> 172.21.15.5:0/3684385172 conn(0x7fc2c40fd740 msgr2=0x7fc2c410d070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:57:14.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:14.240+0000 7fc2ca800640 1 -- 172.21.15.5:0/3684385172 shutdown_connections 2024-09-10T09:57:14.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:14.240+0000 7fc2ca800640 1 -- 172.21.15.5:0/3684385172 wait complete. 2024-09-10T09:57:14.990 INFO:tasks.workunit.client.0.smithi005.stdout: 10 87528 15.09 MB/sec execute 173 sec latency 16806.555 ms 2024-09-10T09:57:15.407 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:57:15 smithi005 ceph-mon[30350]: pgmap v960: 129 pgs: 129 active+clean; 704 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 360 KiB/s rd, 9.3 MiB/s wr, 276 op/s 2024-09-10T09:57:15.414 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-10T09:57:15.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:57:15 smithi159 ceph-mon[35513]: pgmap v960: 129 pgs: 129 active+clean; 704 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 360 KiB/s rd, 9.3 MiB/s wr, 276 op/s 2024-09-10T09:57:15.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:57:15 smithi102 ceph-mon[35442]: pgmap v960: 129 pgs: 129 active+clean; 704 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 360 KiB/s rd, 9.3 MiB/s wr, 276 op/s 2024-09-10T09:57:15.990 INFO:tasks.workunit.client.0.smithi005.stdout: 10 87530 15.00 MB/sec execute 174 sec latency 17799.018 ms 2024-09-10T09:57:16.017 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:57:16.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:16.769+0000 7f5455400640 1 -- 172.21.15.5:0/2174147329 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5450106f20 msgr2=0x7f545010eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:57:16.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:16.769+0000 7f5455400640 1 --2- 172.21.15.5:0/2174147329 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5450106f20 0x7f545010eaa0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f5448009be0 tx=0x7f544802a6c0 comp rx=0 tx=0).stop 2024-09-10T09:57:16.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:16.769+0000 7f5455400640 1 -- 172.21.15.5:0/2174147329 shutdown_connections 2024-09-10T09:57:16.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:16.769+0000 7f5455400640 1 --2- 172.21.15.5:0/2174147329 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5450106f20 0x7f545010eaa0 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:16.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:16.769+0000 7f5455400640 1 --2- 172.21.15.5:0/2174147329 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f54501027a0 0x7f54501067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:16.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:16.769+0000 7f5455400640 1 --2- 172.21.15.5:0/2174147329 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5450101df0 0x7f54501021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:16.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:16.769+0000 7f5455400640 1 -- 172.21.15.5:0/2174147329 >> 172.21.15.5:0/2174147329 conn(0x7f54500fd8b0 msgr2=0x7f54500ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:57:16.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:16.769+0000 7f5455400640 1 -- 172.21.15.5:0/2174147329 shutdown_connections 2024-09-10T09:57:16.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:16.769+0000 7f5455400640 1 -- 172.21.15.5:0/2174147329 wait complete. 2024-09-10T09:57:16.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:16.769+0000 7f5455400640 1 Processor -- start 2024-09-10T09:57:16.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:16.769+0000 7f5455400640 1 -- start start 2024-09-10T09:57:16.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:16.772+0000 7f5455400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5450101df0 0x7f545019a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:16.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:16.772+0000 7f5455400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f54501027a0 0x7f545019aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:16.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:16.772+0000 7f5455400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5450106f20 0x7f545019ee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:16.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:16.772+0000 7f5455400640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f5450111f80 con 0x7f54501027a0 2024-09-10T09:57:16.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:16.772+0000 7f5455400640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f5450111e00 con 0x7f5450101df0 2024-09-10T09:57:16.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:16.772+0000 7f5455400640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f5450112100 con 0x7f5450106f20 2024-09-10T09:57:16.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:16.772+0000 7f5447e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f54501027a0 0x7f545019aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:16.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:16.772+0000 7f5447e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f54501027a0 0x7f545019aad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60012/0 (socket says 172.21.15.5:60012) 2024-09-10T09:57:16.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:16.772+0000 7f5447e00640 1 -- 172.21.15.5:0/4017306704 learned_addr learned my addr 172.21.15.5:0/4017306704 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:57:16.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:16.772+0000 7f544f200640 1 --2- 172.21.15.5:0/4017306704 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5450106f20 0x7f545019ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:16.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:16.772+0000 7f5447e00640 1 -- 172.21.15.5:0/4017306704 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5450106f20 msgr2=0x7f545019ee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:57:16.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:16.772+0000 7f5447e00640 1 --2- 172.21.15.5:0/4017306704 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5450106f20 0x7f545019ee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:16.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:16.772+0000 7f5447e00640 1 -- 172.21.15.5:0/4017306704 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5450101df0 msgr2=0x7f545019a590 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-10T09:57:16.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:16.772+0000 7f5447e00640 1 --2- 172.21.15.5:0/4017306704 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5450101df0 0x7f545019a590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:16.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:16.772+0000 7f5447e00640 1 -- 172.21.15.5:0/4017306704 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f545019f540 con 0x7f54501027a0 2024-09-10T09:57:16.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:16.772+0000 7f5447e00640 1 --2- 172.21.15.5:0/4017306704 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f54501027a0 0x7f545019aad0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f543800e9f0 tx=0x7f543800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:57:16.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:16.774+0000 7f5447400640 1 -- 172.21.15.5:0/4017306704 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f543800cd80 con 0x7f54501027a0 2024-09-10T09:57:16.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:16.774+0000 7f5447400640 1 -- 172.21.15.5:0/4017306704 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5438010910 con 0x7f54501027a0 2024-09-10T09:57:16.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:16.774+0000 7f5447400640 1 -- 172.21.15.5:0/4017306704 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5438018990 con 0x7f54501027a0 2024-09-10T09:57:16.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:16.774+0000 7f5455400640 1 -- 172.21.15.5:0/4017306704 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f545019f830 con 0x7f54501027a0 2024-09-10T09:57:16.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:16.774+0000 7f5455400640 1 -- 172.21.15.5:0/4017306704 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54501a7110 con 0x7f54501027a0 2024-09-10T09:57:16.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:16.774+0000 7f5457e90740 1 Processor -- start 2024-09-10T09:57:16.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:16.774+0000 7f5457e90740 1 -- start start 2024-09-10T09:57:16.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:16.774+0000 7f5457e90740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c967dcac70 0x55c967dcb050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:16.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:16.774+0000 7f5457e90740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c967dcb590 0x55c967dd5910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:16.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:16.774+0000 7f5457e90740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c967dd5e50 0x55c967dd8240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:16.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:16.774+0000 7f5457e90740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55c967d313e0 con 0x55c967dcac70 2024-09-10T09:57:16.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:16.774+0000 7f5457e90740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55c967cd8d80 con 0x55c967dcb590 2024-09-10T09:57:16.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:16.774+0000 7f5457e90740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55c967d2d3a0 con 0x55c967dd5e50 2024-09-10T09:57:16.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:16.774+0000 7f5447400640 1 -- 172.21.15.5:0/4017306704 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f5438010ac0 con 0x7f54501027a0 2024-09-10T09:57:16.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:16.774+0000 7f544f200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c967dcac70 0x55c967dcb050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:16.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:16.774+0000 7f544f200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c967dcac70 0x55c967dcb050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60016/0 (socket says 172.21.15.5:60016) 2024-09-10T09:57:16.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:16.774+0000 7f544f200640 1 -- 172.21.15.5:0/2873413815 learned_addr learned my addr 172.21.15.5:0/2873413815 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:57:16.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:16.774+0000 7f544e800640 1 --2- 172.21.15.5:0/2873413815 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c967dcb590 0x55c967dd5910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:16.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:16.774+0000 7f5447e00640 1 --2- 172.21.15.5:0/2873413815 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c967dd5e50 0x55c967dd8240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:16.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:16.774+0000 7f544e800640 1 -- 172.21.15.5:0/2873413815 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c967dd5e50 msgr2=0x55c967dd8240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:57:16.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:16.774+0000 7f544e800640 1 --2- 172.21.15.5:0/2873413815 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c967dd5e50 0x55c967dd8240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:16.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:16.774+0000 7f544e800640 1 -- 172.21.15.5:0/2873413815 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c967dcac70 msgr2=0x55c967dcb050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:57:16.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:16.774+0000 7f544e800640 1 --2- 172.21.15.5:0/2873413815 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c967dcac70 0x55c967dcb050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:16.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:16.774+0000 7f544e800640 1 -- 172.21.15.5:0/2873413815 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c967cf7ae0 con 0x55c967dcb590 2024-09-10T09:57:16.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:16.774+0000 7f544e800640 1 --2- 172.21.15.5:0/2873413815 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c967dcb590 0x55c967dd5910 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f5440009ab0 tx=0x7f5440009f70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c3e9595e8a328a43 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:57:16.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:16.774+0000 7f543f400640 1 -- 172.21.15.5:0/2873413815 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f544000ecb0 con 0x55c967dcb590 2024-09-10T09:57:16.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:16.774+0000 7f543f400640 1 -- 172.21.15.5:0/2873413815 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f544000ee50 con 0x55c967dcb590 2024-09-10T09:57:16.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:16.780+0000 7f543f400640 1 -- 172.21.15.5:0/2873413815 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5440010690 con 0x55c967dcb590 2024-09-10T09:57:16.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:16.780+0000 7f5447400640 1 --2- 172.21.15.5:0/4017306704 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f54340a2c00 0x7f54340a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:16.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:16.780+0000 7f5447400640 1 -- 172.21.15.5:0/4017306704 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f54380cf710 con 0x7f54501027a0 2024-09-10T09:57:16.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:16.780+0000 7f5457e90740 1 -- 172.21.15.5:0/2873413815 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c967dcb590 msgr2=0x55c967dd5910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:57:16.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:16.780+0000 7f5457e90740 1 --2- 172.21.15.5:0/2873413815 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c967dcb590 0x55c967dd5910 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f5440009ab0 tx=0x7f5440009f70 comp rx=0 tx=0).stop 2024-09-10T09:57:16.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:16.780+0000 7f544f200640 1 --2- 172.21.15.5:0/4017306704 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f54340a2c00 0x7f54340a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:16.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:16.780+0000 7f5457e90740 1 -- 172.21.15.5:0/2873413815 shutdown_connections 2024-09-10T09:57:16.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:16.780+0000 7f5457e90740 1 --2- 172.21.15.5:0/2873413815 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c967dd5e50 0x55c967dd8240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:16.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:16.780+0000 7f5457e90740 1 --2- 172.21.15.5:0/2873413815 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c967dcb590 0x55c967dd5910 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:16.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:16.780+0000 7f5457e90740 1 --2- 172.21.15.5:0/2873413815 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c967dcac70 0x55c967dcb050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:16.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:16.780+0000 7f5457e90740 1 -- 172.21.15.5:0/2873413815 >> 172.21.15.5:0/2873413815 conn(0x55c967dc7200 msgr2=0x55c967dc75e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:57:16.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:16.780+0000 7f5457e90740 1 -- 172.21.15.5:0/2873413815 shutdown_connections 2024-09-10T09:57:16.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:16.780+0000 7f5457e90740 1 -- 172.21.15.5:0/2873413815 wait complete. 2024-09-10T09:57:16.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:16.780+0000 7f544f200640 1 --2- 172.21.15.5:0/4017306704 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f54340a2c00 0x7f54340a50c0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f54480099a0 tx=0x7f5448005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:57:16.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:16.780+0000 7f5457e90740 1 -- start start 2024-09-10T09:57:16.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:16.780+0000 7f5457e90740 1 Processor -- start 2024-09-10T09:57:16.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:16.780+0000 7f5457e90740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c967dcac70 0x55c967dd2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:16.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:16.780+0000 7f5457e90740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c967dcb590 0x55c967dd31f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:16.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:16.780+0000 7f5457e90740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c967dd5e50 0x55c967dd3730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:16.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:16.780+0000 7f5457e90740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55c967dd8a20 con 0x55c967dcac70 2024-09-10T09:57:16.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:16.780+0000 7f5457e90740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55c967cec0f0 con 0x55c967dd5e50 2024-09-10T09:57:16.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:16.780+0000 7f5457e90740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55c967dd8ba0 con 0x55c967dcb590 2024-09-10T09:57:16.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:16.780+0000 7f5457e90740 10 client.0 _ll_register_callbacks cb 0x55c967cd3bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:57:16.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:16.780+0000 7f5447e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c967dcb590 0x55c967dd31f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:16.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:16.780+0000 7f5447e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c967dcb590 0x55c967dd31f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:46874/0 (socket says 172.21.15.5:46874) 2024-09-10T09:57:16.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:16.780+0000 7f5447e00640 1 -- 172.21.15.5:0/642252831 learned_addr learned my addr 172.21.15.5:0/642252831 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:57:16.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:16.780+0000 7f544e800640 1 --2- 172.21.15.5:0/642252831 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c967dcac70 0x55c967dd2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:16.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:16.780+0000 7f5447e00640 1 -- 172.21.15.5:0/642252831 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c967dd5e50 msgr2=0x55c967dd3730 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T09:57:16.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:16.780+0000 7f5447e00640 1 --2- 172.21.15.5:0/642252831 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c967dd5e50 0x55c967dd3730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:16.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:16.780+0000 7f5447e00640 1 -- 172.21.15.5:0/642252831 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c967dcac70 msgr2=0x55c967dd2cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:57:16.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:16.780+0000 7f5447e00640 1 --2- 172.21.15.5:0/642252831 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c967dcac70 0x55c967dd2cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:16.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:16.780+0000 7f5447e00640 1 -- 172.21.15.5:0/642252831 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c967aa23c0 con 0x55c967dcb590 2024-09-10T09:57:16.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:16.780+0000 7f5447e00640 1 --2- 172.21.15.5:0/642252831 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c967dcb590 0x55c967dd31f0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f5438097f80 tx=0x7f543809b480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e845c269d6a9ef62 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:57:16.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:16.780+0000 7f5433e00640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-09-10T09:57:16.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:16.780+0000 7f5433e00640 1 -- 172.21.15.5:0/642252831 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f543809bb20 con 0x55c967dcb590 2024-09-10T09:57:16.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:16.780+0000 7f5433e00640 1 -- 172.21.15.5:0/642252831 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f5438018b30 con 0x55c967dcb590 2024-09-10T09:57:16.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:16.780+0000 7f5457e90740 1 -- 172.21.15.5:0/642252831 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f544000ee50 con 0x55c967dcb590 2024-09-10T09:57:16.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:16.780+0000 7f5433e00640 1 -- 172.21.15.5:0/642252831 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f543809fed0 con 0x55c967dcb590 2024-09-10T09:57:16.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:16.780+0000 7f5457e90740 10 client.29083 fetch_fsmap learned FSMap version 42 2024-09-10T09:57:16.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:16.780+0000 7f5457e90740 1 -- 172.21.15.5:0/642252831 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55c967ded1f0 con 0x55c967dcb590 2024-09-10T09:57:16.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:16.780+0000 7f5433e00640 1 -- 172.21.15.5:0/642252831 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f544000ee50 con 0x55c967dcb590 2024-09-10T09:57:16.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:16.780+0000 7f5433e00640 1 -- 172.21.15.5:0/642252831 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 42) ==== 8370+0+0 (secure 0 0 0) 0x7f543809af70 con 0x55c967dcb590 2024-09-10T09:57:16.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:16.780+0000 7f5457e90740 10 client.29083 fetch_fsmap finished waiting for FSMap version 42 2024-09-10T09:57:16.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:16.780+0000 7f5457e90740 10 client.29083 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-10T09:57:16.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:16.780+0000 7f5457e90740 20 client.29083 populate_metadata read hostname 'smithi005' 2024-09-10T09:57:16.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:16.780+0000 7f5457e90740 1 --2- 172.21.15.5:0/642252831 >> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] conn(0x55c967dee650 0x55c967df0ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:16.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:16.780+0000 7f5457e90740 4 client.29083 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:57:16.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:16.780+0000 7f5457e90740 1 -- 172.21.15.5:0/642252831 --> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c967d3e950 con 0x55c967dee650 2024-09-10T09:57:16.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:16.785+0000 7f544e800640 1 --2- 172.21.15.5:0/642252831 >> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] conn(0x55c967dee650 0x55c967df0ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:16.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:16.785+0000 7f544e800640 1 --2- 172.21.15.5:0/642252831 >> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] conn(0x55c967dee650 0x55c967df0ae0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=bdac07085b5631b4 server_cookie=508375bd0b4118d1 in_seq=0 out_seq=0 2024-09-10T09:57:16.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:16.785+0000 7f5433e00640 10 client.29083 ms_handle_connect on v2:172.21.15.159:6838/2073739599 2024-09-10T09:57:16.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:16.786+0000 7f5433e00640 1 -- 172.21.15.5:0/642252831 <== mds.1 v2:172.21.15.159:6838/2073739599 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c967d3e950 con 0x55c967dee650 2024-09-10T09:57:16.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:16.786+0000 7f5433e00640 10 client.29083 handle_command_reply: tid=0 2024-09-10T09:57:16.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:16.786+0000 7f5457e90740 1 client.29083 shutdown 2024-09-10T09:57:16.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:16.786+0000 7f5457e90740 1 -- 172.21.15.5:0/642252831 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c967dcb590 msgr2=0x55c967dd31f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:57:16.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:16.786+0000 7f5457e90740 1 --2- 172.21.15.5:0/642252831 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c967dcb590 0x55c967dd31f0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f5438097f80 tx=0x7f543809b480 comp rx=0 tx=0).stop 2024-09-10T09:57:16.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:16.786+0000 7f5457e90740 1 -- 172.21.15.5:0/642252831 shutdown_connections 2024-09-10T09:57:16.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:16.786+0000 7f5457e90740 1 --2- 172.21.15.5:0/642252831 >> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] conn(0x55c967dee650 0x55c967df0ae0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:16.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:16.786+0000 7f5457e90740 1 --2- 172.21.15.5:0/642252831 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c967dd5e50 0x55c967dd3730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:16.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:16.786+0000 7f5457e90740 1 --2- 172.21.15.5:0/642252831 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c967dcb590 0x55c967dd31f0 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:16.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:16.786+0000 7f5433e00640 0 client.29083 ms_handle_reset on v2:172.21.15.159:6838/2073739599 2024-09-10T09:57:16.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:16.786+0000 7f5457e90740 1 --2- 172.21.15.5:0/642252831 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c967dcac70 0x55c967dd2cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:16.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:16.786+0000 7f5457e90740 1 -- 172.21.15.5:0/642252831 >> 172.21.15.5:0/642252831 conn(0x55c967dc7200 msgr2=0x55c967dccf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:57:16.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:16.786+0000 7f5457e90740 1 -- 172.21.15.5:0/642252831 shutdown_connections 2024-09-10T09:57:16.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:16.786+0000 7f5457e90740 1 -- 172.21.15.5:0/642252831 wait complete. 2024-09-10T09:57:16.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:16.786+0000 7f5457e90740 20 client.29083 trim_cache size 0 max 16384 2024-09-10T09:57:16.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:16.798+0000 7f5457e90740 1 Processor -- start 2024-09-10T09:57:16.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:16.798+0000 7f5457e90740 1 -- start start 2024-09-10T09:57:16.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:16.798+0000 7f5457e90740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c967dcac70 0x55c967dd4c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:16.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:16.798+0000 7f5457e90740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c967dcb590 0x55c967df6360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:16.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:16.798+0000 7f5457e90740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c967dd5e50 0x55c967df68a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:16.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:16.798+0000 7f5457e90740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f54501acc70 con 0x55c967dcb590 2024-09-10T09:57:16.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:16.798+0000 7f5457e90740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f54501ae7b0 con 0x55c967dd5e50 2024-09-10T09:57:16.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:16.798+0000 7f5457e90740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f54501ad340 con 0x55c967dcac70 2024-09-10T09:57:16.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:16.798+0000 7f544e800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c967dcac70 0x55c967dd4c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:16.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:16.798+0000 7f544e800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c967dcac70 0x55c967dd4c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:46888/0 (socket says 172.21.15.5:46888) 2024-09-10T09:57:16.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:16.798+0000 7f544e800640 1 -- 172.21.15.5:0/3216886109 learned_addr learned my addr 172.21.15.5:0/3216886109 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:57:16.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:16.798+0000 7f544f200640 1 --2- 172.21.15.5:0/3216886109 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c967dd5e50 0x55c967df68a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:16.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:16.798+0000 7f5447e00640 1 --2- 172.21.15.5:0/3216886109 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c967dcb590 0x55c967df6360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:16.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:16.798+0000 7f544f200640 1 -- 172.21.15.5:0/3216886109 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c967dcac70 msgr2=0x55c967dd4c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:57:16.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:16.798+0000 7f544f200640 1 --2- 172.21.15.5:0/3216886109 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c967dcac70 0x55c967dd4c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:16.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:16.798+0000 7f544f200640 1 -- 172.21.15.5:0/3216886109 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c967dcb590 msgr2=0x55c967df6360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:57:16.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:16.798+0000 7f544f200640 1 --2- 172.21.15.5:0/3216886109 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c967dcb590 0x55c967df6360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:16.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:16.798+0000 7f544f200640 1 -- 172.21.15.5:0/3216886109 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c967dc7030 con 0x55c967dd5e50 2024-09-10T09:57:16.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:16.798+0000 7f544e800640 1 --2- 172.21.15.5:0/3216886109 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c967dcac70 0x55c967dd4c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:57:16.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:16.798+0000 7f544f200640 1 --2- 172.21.15.5:0/3216886109 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c967dd5e50 0x55c967df68a0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f5448009b30 tx=0x7f5448040000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4d19ec728d7e4648 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:57:16.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:16.798+0000 7f543ea00640 1 -- 172.21.15.5:0/3216886109 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5448036040 con 0x55c967dd5e50 2024-09-10T09:57:16.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:16.798+0000 7f543ea00640 1 -- 172.21.15.5:0/3216886109 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5448003b70 con 0x55c967dd5e50 2024-09-10T09:57:16.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:16.798+0000 7f543ea00640 1 -- 172.21.15.5:0/3216886109 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f544804b6b0 con 0x55c967dd5e50 2024-09-10T09:57:16.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:16.798+0000 7f5457e90740 1 -- 172.21.15.5:0/3216886109 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c967dd5e50 msgr2=0x55c967df68a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:57:16.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:16.798+0000 7f5457e90740 1 --2- 172.21.15.5:0/3216886109 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c967dd5e50 0x55c967df68a0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f5448009b30 tx=0x7f5448040000 comp rx=0 tx=0).stop 2024-09-10T09:57:16.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:16.798+0000 7f5457e90740 1 -- 172.21.15.5:0/3216886109 shutdown_connections 2024-09-10T09:57:16.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:16.798+0000 7f5457e90740 1 --2- 172.21.15.5:0/3216886109 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c967dd5e50 0x55c967df68a0 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:16.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:16.798+0000 7f5457e90740 1 --2- 172.21.15.5:0/3216886109 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c967dcb590 0x55c967df6360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:16.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:16.798+0000 7f5457e90740 1 --2- 172.21.15.5:0/3216886109 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c967dcac70 0x55c967dd4c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:16.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:16.798+0000 7f5457e90740 1 -- 172.21.15.5:0/3216886109 >> 172.21.15.5:0/3216886109 conn(0x55c967dc7200 msgr2=0x55c967dd4510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:57:16.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:16.798+0000 7f5457e90740 1 -- 172.21.15.5:0/3216886109 shutdown_connections 2024-09-10T09:57:16.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:16.798+0000 7f5457e90740 1 -- 172.21.15.5:0/3216886109 wait complete. 2024-09-10T09:57:16.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:16.798+0000 7f5457e90740 1 -- start start 2024-09-10T09:57:16.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:16.804+0000 7f5457e90740 1 Processor -- start 2024-09-10T09:57:16.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:16.804+0000 7f5457e90740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c967dcac70 0x55c967de6690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:16.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:16.804+0000 7f5457e90740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c967dcb590 0x55c967de6bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:16.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:16.804+0000 7f5457e90740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c967dd5e50 0x55c967de1640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:16.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:16.804+0000 7f5457e90740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f54501ae0e0 con 0x55c967dcb590 2024-09-10T09:57:16.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:16.804+0000 7f5457e90740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f54501ada10 con 0x55c967dd5e50 2024-09-10T09:57:16.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:16.804+0000 7f5457e90740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f54501aac50 con 0x55c967dcac70 2024-09-10T09:57:16.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:16.804+0000 7f5457e90740 10 client.0 _ll_register_callbacks cb 0x7f543800e940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:57:16.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:16.804+0000 7f5447e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c967dcb590 0x55c967de6bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:16.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:16.804+0000 7f5447e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c967dcb590 0x55c967de6bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60034/0 (socket says 172.21.15.5:60034) 2024-09-10T09:57:16.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:16.804+0000 7f5447e00640 1 -- 172.21.15.5:0/691575312 learned_addr learned my addr 172.21.15.5:0/691575312 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:57:16.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:16.804+0000 7f544e800640 1 --2- 172.21.15.5:0/691575312 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c967dcac70 0x55c967de6690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:16.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:16.804+0000 7f544f200640 1 --2- 172.21.15.5:0/691575312 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c967dd5e50 0x55c967de1640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:16.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:16.804+0000 7f5447e00640 1 -- 172.21.15.5:0/691575312 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c967dcac70 msgr2=0x55c967de6690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:57:16.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:16.804+0000 7f5447e00640 1 --2- 172.21.15.5:0/691575312 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c967dcac70 0x55c967de6690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:16.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:16.804+0000 7f5447e00640 1 -- 172.21.15.5:0/691575312 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c967dd5e50 msgr2=0x55c967de1640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:57:16.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:16.804+0000 7f5447e00640 1 --2- 172.21.15.5:0/691575312 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c967dd5e50 0x55c967de1640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:16.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:16.804+0000 7f5447e00640 1 -- 172.21.15.5:0/691575312 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c967dc7e90 con 0x55c967dcb590 2024-09-10T09:57:16.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:16.804+0000 7f544e800640 1 --2- 172.21.15.5:0/691575312 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c967dcac70 0x55c967de6690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:57:16.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:16.804+0000 7f5447e00640 1 --2- 172.21.15.5:0/691575312 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c967dcb590 0x55c967de6bd0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f543809bab0 tx=0x7f5438005c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f6ecbde485a2bb6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:57:16.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:16.804+0000 7f5433e00640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-10T09:57:16.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:16.804+0000 7f5433e00640 1 -- 172.21.15.5:0/691575312 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54380a9060 con 0x55c967dcb590 2024-09-10T09:57:16.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:16.804+0000 7f5433e00640 1 -- 172.21.15.5:0/691575312 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f5438018b30 con 0x55c967dcb590 2024-09-10T09:57:16.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:16.804+0000 7f5457e90740 1 -- 172.21.15.5:0/691575312 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5448003b70 con 0x55c967dcb590 2024-09-10T09:57:16.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:16.804+0000 7f5433e00640 1 -- 172.21.15.5:0/691575312 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f543809bb20 con 0x55c967dcb590 2024-09-10T09:57:16.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:16.804+0000 7f5457e90740 10 client.19035 fetch_fsmap learned FSMap version 42 2024-09-10T09:57:16.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:16.804+0000 7f5457e90740 1 -- 172.21.15.5:0/691575312 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55c967dc2770 con 0x55c967dcb590 2024-09-10T09:57:16.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:16.804+0000 7f5433e00640 1 -- 172.21.15.5:0/691575312 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5448003b70 con 0x55c967dcb590 2024-09-10T09:57:16.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:16.804+0000 7f5433e00640 1 -- 172.21.15.5:0/691575312 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 42) ==== 8370+0+0 (secure 0 0 0) 0x7f5438097870 con 0x55c967dcb590 2024-09-10T09:57:16.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:16.804+0000 7f5457e90740 10 client.19035 fetch_fsmap finished waiting for FSMap version 42 2024-09-10T09:57:16.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:16.804+0000 7f5457e90740 10 client.19035 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-10T09:57:16.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:16.804+0000 7f5457e90740 20 client.19035 populate_metadata read hostname 'smithi005' 2024-09-10T09:57:16.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:16.804+0000 7f5457e90740 1 --2- 172.21.15.5:0/691575312 >> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] conn(0x55c967e59cf0 0x55c967e5c110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:16.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:16.804+0000 7f5457e90740 4 client.19035 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T09:57:16.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:16.804+0000 7f5457e90740 1 -- 172.21.15.5:0/691575312 --> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c967d38e60 con 0x55c967e59cf0 2024-09-10T09:57:16.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:16.804+0000 7f544e800640 1 --2- 172.21.15.5:0/691575312 >> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] conn(0x55c967e59cf0 0x55c967e5c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:16.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:16.804+0000 7f544e800640 1 --2- 172.21.15.5:0/691575312 >> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] conn(0x55c967e59cf0 0x55c967e5c110 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=aa3f1cd78205a78 server_cookie=c61704bf0a1a852 in_seq=0 out_seq=0 2024-09-10T09:57:16.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:16.804+0000 7f5433e00640 10 client.19035 ms_handle_connect on v2:172.21.15.159:6838/2073739599 2024-09-10T09:57:16.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:16.804+0000 7f5433e00640 1 -- 172.21.15.5:0/691575312 <== mds.1 v2:172.21.15.159:6838/2073739599 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c967d38e60 con 0x55c967e59cf0 2024-09-10T09:57:16.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:16.804+0000 7f5433e00640 10 client.19035 handle_command_reply: tid=0 2024-09-10T09:57:16.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:16.804+0000 7f5457e90740 1 client.19035 shutdown 2024-09-10T09:57:16.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:16.804+0000 7f5457e90740 1 -- 172.21.15.5:0/691575312 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c967dcb590 msgr2=0x55c967de6bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:57:16.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:16.804+0000 7f5457e90740 1 --2- 172.21.15.5:0/691575312 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c967dcb590 0x55c967de6bd0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f543809bab0 tx=0x7f5438005c80 comp rx=0 tx=0).stop 2024-09-10T09:57:16.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:16.804+0000 7f5457e90740 1 -- 172.21.15.5:0/691575312 shutdown_connections 2024-09-10T09:57:16.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:16.804+0000 7f5457e90740 1 --2- 172.21.15.5:0/691575312 >> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] conn(0x55c967e59cf0 0x55c967e5c110 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:16.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:16.804+0000 7f5457e90740 1 --2- 172.21.15.5:0/691575312 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c967dd5e50 0x55c967de1640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:16.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:16.804+0000 7f5457e90740 1 --2- 172.21.15.5:0/691575312 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c967dcb590 0x55c967de6bd0 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:16.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:16.804+0000 7f5457e90740 1 --2- 172.21.15.5:0/691575312 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c967dcac70 0x55c967de6690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:16.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:16.804+0000 7f5457e90740 1 -- 172.21.15.5:0/691575312 >> 172.21.15.5:0/691575312 conn(0x55c967dc7200 msgr2=0x55c967dc6380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:57:16.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:16.804+0000 7f5433e00640 0 client.19035 ms_handle_reset on v2:172.21.15.159:6838/2073739599 2024-09-10T09:57:16.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:16.804+0000 7f5457e90740 1 -- 172.21.15.5:0/691575312 shutdown_connections 2024-09-10T09:57:16.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:16.804+0000 7f5457e90740 1 -- 172.21.15.5:0/691575312 wait complete. 2024-09-10T09:57:16.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:16.804+0000 7f5457e90740 20 client.19035 trim_cache size 0 max 16384 2024-09-10T09:57:16.812 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-10T09:57:16.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:16.809+0000 7f5455400640 1 -- 172.21.15.5:0/4017306704 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f54340a2c00 msgr2=0x7f54340a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:57:16.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:16.809+0000 7f5455400640 1 --2- 172.21.15.5:0/4017306704 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f54340a2c00 0x7f54340a50c0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f54480099a0 tx=0x7f5448005fd0 comp rx=0 tx=0).stop 2024-09-10T09:57:16.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:16.809+0000 7f5455400640 1 -- 172.21.15.5:0/4017306704 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f54501027a0 msgr2=0x7f545019aad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:57:16.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:16.809+0000 7f5455400640 1 --2- 172.21.15.5:0/4017306704 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f54501027a0 0x7f545019aad0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f543800e9f0 tx=0x7f543800eeb0 comp rx=0 tx=0).stop 2024-09-10T09:57:16.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:16.809+0000 7f5455400640 1 -- 172.21.15.5:0/4017306704 shutdown_connections 2024-09-10T09:57:16.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:16.809+0000 7f5455400640 1 --2- 172.21.15.5:0/4017306704 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f54340a2c00 0x7f54340a50c0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:16.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:16.809+0000 7f5455400640 1 --2- 172.21.15.5:0/4017306704 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5450106f20 0x7f545019ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:16.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:16.809+0000 7f5455400640 1 --2- 172.21.15.5:0/4017306704 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f54501027a0 0x7f545019aad0 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:16.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:16.809+0000 7f5455400640 1 --2- 172.21.15.5:0/4017306704 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5450101df0 0x7f545019a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:16.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:16.809+0000 7f5455400640 1 -- 172.21.15.5:0/4017306704 >> 172.21.15.5:0/4017306704 conn(0x7f54500fd8b0 msgr2=0x7f54500fe800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:57:16.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:16.809+0000 7f5455400640 1 -- 172.21.15.5:0/4017306704 shutdown_connections 2024-09-10T09:57:16.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:16.809+0000 7f5455400640 1 -- 172.21.15.5:0/4017306704 wait complete. 2024-09-10T09:57:16.990 INFO:tasks.workunit.client.0.smithi005.stdout: 10 87530 14.91 MB/sec execute 175 sec latency 18360.798 ms 2024-09-10T09:57:17.306 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:57:17 smithi005 ceph-mon[30350]: pgmap v961: 129 pgs: 129 active+clean; 688 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 319 KiB/s rd, 7.9 MiB/s wr, 265 op/s 2024-09-10T09:57:17.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:57:17 smithi159 ceph-mon[35513]: pgmap v961: 129 pgs: 129 active+clean; 688 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 319 KiB/s rd, 7.9 MiB/s wr, 265 op/s 2024-09-10T09:57:17.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:57:17 smithi102 ceph-mon[35442]: pgmap v961: 129 pgs: 129 active+clean; 688 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 319 KiB/s rd, 7.9 MiB/s wr, 265 op/s 2024-09-10T09:57:17.708 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-10T09:57:17.990 INFO:tasks.workunit.client.0.smithi005.stdout: 10 87530 14.83 MB/sec execute 176 sec latency 19360.841 ms 2024-09-10T09:57:18.088 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:57:18.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:18.729+0000 7f8137e00640 1 -- 172.21.15.5:0/1108218522 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f81381019c0 msgr2=0x7f8138101e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:57:18.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:18.729+0000 7f8137e00640 1 --2- 172.21.15.5:0/1108218522 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f81381019c0 0x7f8138101e20 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f8124009930 tx=0x7f812402a460 comp rx=0 tx=0).stop 2024-09-10T09:57:18.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:18.729+0000 7f8137e00640 1 -- 172.21.15.5:0/1108218522 shutdown_connections 2024-09-10T09:57:18.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:18.729+0000 7f8137e00640 1 --2- 172.21.15.5:0/1108218522 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8138102360 0x7f813810e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:18.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:18.729+0000 7f8137e00640 1 --2- 172.21.15.5:0/1108218522 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f81381019c0 0x7f8138101e20 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:18.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:18.729+0000 7f8137e00640 1 --2- 172.21.15.5:0/1108218522 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f81381079c0 0x7f8138107da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:18.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:18.729+0000 7f8137e00640 1 -- 172.21.15.5:0/1108218522 >> 172.21.15.5:0/1108218522 conn(0x7f81380fd740 msgr2=0x7f81380ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:57:18.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:18.729+0000 7f8137e00640 1 -- 172.21.15.5:0/1108218522 shutdown_connections 2024-09-10T09:57:18.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:18.729+0000 7f8137e00640 1 -- 172.21.15.5:0/1108218522 wait complete. 2024-09-10T09:57:18.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:18.729+0000 7f8137e00640 1 Processor -- start 2024-09-10T09:57:18.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:18.729+0000 7f8137e00640 1 -- start start 2024-09-10T09:57:18.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:18.729+0000 7f8137e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f81381019c0 0x7f813819a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:18.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:18.729+0000 7f8137e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8138102360 0x7f813819aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:18.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:18.729+0000 7f8137e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f81381079c0 0x7f813819ee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:18.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:18.729+0000 7f8137e00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f8138111e50 con 0x7f81381079c0 2024-09-10T09:57:18.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:18.729+0000 7f8137e00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f8138111cd0 con 0x7f81381019c0 2024-09-10T09:57:18.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:18.729+0000 7f8137e00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f8138111fd0 con 0x7f8138102360 2024-09-10T09:57:18.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:18.729+0000 7f8136a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f81381019c0 0x7f813819a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:18.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:18.729+0000 7f8136a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f81381019c0 0x7f813819a550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:34824/0 (socket says 172.21.15.5:34824) 2024-09-10T09:57:18.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:18.729+0000 7f8136a00640 1 -- 172.21.15.5:0/1862089254 learned_addr learned my addr 172.21.15.5:0/1862089254 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:57:18.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:18.729+0000 7f8137400640 1 --2- 172.21.15.5:0/1862089254 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f81381079c0 0x7f813819ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:18.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:18.729+0000 7f8136000640 1 --2- 172.21.15.5:0/1862089254 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8138102360 0x7f813819aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:18.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:18.729+0000 7f8136000640 1 -- 172.21.15.5:0/1862089254 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f81381019c0 msgr2=0x7f813819a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:57:18.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:18.729+0000 7f8136000640 1 --2- 172.21.15.5:0/1862089254 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f81381019c0 0x7f813819a550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:18.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:18.729+0000 7f8136000640 1 -- 172.21.15.5:0/1862089254 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f81381079c0 msgr2=0x7f813819ee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:57:18.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:18.729+0000 7f8136000640 1 --2- 172.21.15.5:0/1862089254 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f81381079c0 0x7f813819ee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:18.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:18.729+0000 7f8136000640 1 -- 172.21.15.5:0/1862089254 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f813819f500 con 0x7f8138102360 2024-09-10T09:57:18.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:18.729+0000 7f8137400640 1 --2- 172.21.15.5:0/1862089254 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f81381079c0 0x7f813819ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:57:18.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:18.735+0000 7f8136000640 1 --2- 172.21.15.5:0/1862089254 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8138102360 0x7f813819aa90 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f812402a980 tx=0x7f8124003b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:57:18.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:18.735+0000 7f812f400640 1 -- 172.21.15.5:0/1862089254 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f812403f070 con 0x7f8138102360 2024-09-10T09:57:18.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:18.735+0000 7f8137e00640 1 -- 172.21.15.5:0/1862089254 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f813819f790 con 0x7f8138102360 2024-09-10T09:57:18.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:18.735+0000 7f812f400640 1 -- 172.21.15.5:0/1862089254 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8124031c90 con 0x7f8138102360 2024-09-10T09:57:18.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:18.735+0000 7f812f400640 1 -- 172.21.15.5:0/1862089254 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8124035030 con 0x7f8138102360 2024-09-10T09:57:18.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:18.735+0000 7f8137e00640 1 -- 172.21.15.5:0/1862089254 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81381a70d0 con 0x7f8138102360 2024-09-10T09:57:18.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:18.739+0000 7f812f400640 1 -- 172.21.15.5:0/1862089254 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f8124042070 con 0x7f8138102360 2024-09-10T09:57:18.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:18.739+0000 7f81403f5740 1 Processor -- start 2024-09-10T09:57:18.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:18.740+0000 7f81403f5740 1 -- start start 2024-09-10T09:57:18.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:18.740+0000 7f81403f5740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563a64012e40 0x563a64013220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:18.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:18.740+0000 7f81403f5740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563a64013760 0x563a6401dae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:18.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:18.740+0000 7f81403f5740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563a6401e020 0x563a64020410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:18.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:18.740+0000 7f81403f5740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x563a63f793e0 con 0x563a64012e40 2024-09-10T09:57:18.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:18.740+0000 7f81403f5740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x563a63f20d80 con 0x563a6401e020 2024-09-10T09:57:18.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:18.740+0000 7f81403f5740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x563a63f753a0 con 0x563a64013760 2024-09-10T09:57:18.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:18.741+0000 7f8136a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563a64013760 0x563a6401dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:18.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:18.741+0000 7f8137400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563a64012e40 0x563a64013220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:18.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:18.741+0000 7f8137400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563a64012e40 0x563a64013220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60060/0 (socket says 172.21.15.5:60060) 2024-09-10T09:57:18.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:18.741+0000 7f8137400640 1 -- 172.21.15.5:0/3684754069 learned_addr learned my addr 172.21.15.5:0/3684754069 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:57:18.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:18.741+0000 7f8136000640 1 --2- 172.21.15.5:0/3684754069 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563a6401e020 0x563a64020410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:18.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:18.741+0000 7f8137400640 1 -- 172.21.15.5:0/3684754069 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563a64013760 msgr2=0x563a6401dae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:57:18.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:18.741+0000 7f8137400640 1 --2- 172.21.15.5:0/3684754069 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563a64013760 0x563a6401dae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:18.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:18.741+0000 7f8137400640 1 -- 172.21.15.5:0/3684754069 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563a6401e020 msgr2=0x563a64020410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:57:18.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:18.741+0000 7f8137400640 1 --2- 172.21.15.5:0/3684754069 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563a6401e020 0x563a64020410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:18.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:18.741+0000 7f8137400640 1 -- 172.21.15.5:0/3684754069 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563a63f3fae0 con 0x563a64012e40 2024-09-10T09:57:18.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:18.741+0000 7f812f400640 1 --2- 172.21.15.5:0/1862089254 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f811c0a2c00 0x7f811c0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:18.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:18.741+0000 7f8137400640 1 --2- 172.21.15.5:0/3684754069 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563a64012e40 0x563a64013220 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f8130010750 tx=0x7f8130010c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d284d874d5381b3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:57:18.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:18.741+0000 7f812f400640 1 -- 172.21.15.5:0/1862089254 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f81240efb40 con 0x7f8138102360 2024-09-10T09:57:18.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:18.741+0000 7f8136a00640 1 --2- 172.21.15.5:0/1862089254 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f811c0a2c00 0x7f811c0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:18.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:18.741+0000 7f8123e00640 1 -- 172.21.15.5:0/3684754069 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f813000ead0 con 0x563a64012e40 2024-09-10T09:57:18.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:18.741+0000 7f8123e00640 1 -- 172.21.15.5:0/3684754069 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8130010e60 con 0x563a64012e40 2024-09-10T09:57:18.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:18.741+0000 7f8123e00640 1 -- 172.21.15.5:0/3684754069 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8130011640 con 0x563a64012e40 2024-09-10T09:57:18.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:18.741+0000 7f81403f5740 1 -- 172.21.15.5:0/3684754069 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563a64012e40 msgr2=0x563a64013220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:57:18.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:18.741+0000 7f8136a00640 1 --2- 172.21.15.5:0/1862089254 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f811c0a2c00 0x7f811c0a50c0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f8128006fd0 tx=0x7f8128008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:57:18.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:18.741+0000 7f81403f5740 1 --2- 172.21.15.5:0/3684754069 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563a64012e40 0x563a64013220 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f8130010750 tx=0x7f8130010c10 comp rx=0 tx=0).stop 2024-09-10T09:57:18.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:18.741+0000 7f81403f5740 1 -- 172.21.15.5:0/3684754069 shutdown_connections 2024-09-10T09:57:18.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:18.741+0000 7f81403f5740 1 --2- 172.21.15.5:0/3684754069 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563a6401e020 0x563a64020410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:18.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:18.741+0000 7f81403f5740 1 --2- 172.21.15.5:0/3684754069 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563a64013760 0x563a6401dae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:18.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:18.741+0000 7f81403f5740 1 --2- 172.21.15.5:0/3684754069 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563a64012e40 0x563a64013220 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:18.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:18.741+0000 7f81403f5740 1 -- 172.21.15.5:0/3684754069 >> 172.21.15.5:0/3684754069 conn(0x563a6400f3d0 msgr2=0x563a6400f7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:57:18.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:18.741+0000 7f81403f5740 1 -- 172.21.15.5:0/3684754069 shutdown_connections 2024-09-10T09:57:18.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:18.741+0000 7f81403f5740 1 -- 172.21.15.5:0/3684754069 wait complete. 2024-09-10T09:57:18.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:18.741+0000 7f81403f5740 1 -- start start 2024-09-10T09:57:18.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:18.741+0000 7f81403f5740 1 Processor -- start 2024-09-10T09:57:18.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:18.741+0000 7f81403f5740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563a64012e40 0x563a6401af20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:18.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:18.741+0000 7f81403f5740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563a64013760 0x563a6401b460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:18.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:18.741+0000 7f81403f5740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563a6401e020 0x563a6401b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:18.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:18.741+0000 7f81403f5740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x563a64020bf0 con 0x563a6401e020 2024-09-10T09:57:18.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:18.741+0000 7f81403f5740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x563a63f340f0 con 0x563a64013760 2024-09-10T09:57:18.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:18.741+0000 7f81403f5740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x563a64020d70 con 0x563a64012e40 2024-09-10T09:57:18.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:18.741+0000 7f81403f5740 10 client.0 _ll_register_callbacks cb 0x563a63f1bbd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:57:18.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:18.741+0000 7f8136000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563a64013760 0x563a6401b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:18.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:18.741+0000 7f8137400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563a6401e020 0x563a6401b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:18.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:18.741+0000 7f8136000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563a64013760 0x563a6401b460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:34844/0 (socket says 172.21.15.5:34844) 2024-09-10T09:57:18.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:18.741+0000 7f8136000640 1 -- 172.21.15.5:0/3159726099 learned_addr learned my addr 172.21.15.5:0/3159726099 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:57:18.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:18.746+0000 7f8136000640 1 -- 172.21.15.5:0/3159726099 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563a64012e40 msgr2=0x563a6401af20 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T09:57:18.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:18.746+0000 7f8136000640 1 --2- 172.21.15.5:0/3159726099 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563a64012e40 0x563a6401af20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:18.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:18.746+0000 7f8136000640 1 -- 172.21.15.5:0/3159726099 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563a6401e020 msgr2=0x563a6401b9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:57:18.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:18.746+0000 7f8136000640 1 --2- 172.21.15.5:0/3159726099 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563a6401e020 0x563a6401b9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:18.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:18.746+0000 7f8136000640 1 -- 172.21.15.5:0/3159726099 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563a63cea3c0 con 0x563a64013760 2024-09-10T09:57:18.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:18.746+0000 7f8136000640 1 --2- 172.21.15.5:0/3159726099 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563a64013760 0x563a6401b460 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f81240c4690 tx=0x7f8124031490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5a036e37c665e71e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:57:18.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:18.746+0000 7f8120c00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-09-10T09:57:18.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:18.746+0000 7f8120c00640 1 -- 172.21.15.5:0/3159726099 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8124049070 con 0x563a64013760 2024-09-10T09:57:18.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:18.746+0000 7f81403f5740 1 -- 172.21.15.5:0/3159726099 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8130010e60 con 0x563a64013760 2024-09-10T09:57:18.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:18.746+0000 7f8120c00640 1 -- 172.21.15.5:0/3159726099 <== mon.2 v2:172.21.15.102:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f81240b85c0 con 0x563a64013760 2024-09-10T09:57:18.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:18.746+0000 7f8120c00640 1 -- 172.21.15.5:0/3159726099 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f81240bbd20 con 0x563a64013760 2024-09-10T09:57:18.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:18.746+0000 7f81403f5740 10 client.29099 fetch_fsmap learned FSMap version 42 2024-09-10T09:57:18.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:18.746+0000 7f81403f5740 1 -- 172.21.15.5:0/3159726099 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x563a6402cc40 con 0x563a64013760 2024-09-10T09:57:18.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:18.746+0000 7f8120c00640 1 -- 172.21.15.5:0/3159726099 <== mon.2 v2:172.21.15.102:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8130010e60 con 0x563a64013760 2024-09-10T09:57:18.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:18.746+0000 7f8120c00640 1 -- 172.21.15.5:0/3159726099 <== mon.2 v2:172.21.15.102:3300/0 5 ==== fsmap(e 42) ==== 8370+0+0 (secure 0 0 0) 0x7f812403b4f0 con 0x563a64013760 2024-09-10T09:57:18.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:18.746+0000 7f81403f5740 10 client.29099 fetch_fsmap finished waiting for FSMap version 42 2024-09-10T09:57:18.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:18.746+0000 7f81403f5740 10 client.29099 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T09:57:18.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:18.746+0000 7f81403f5740 20 client.29099 populate_metadata read hostname 'smithi005' 2024-09-10T09:57:18.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:18.746+0000 7f81403f5740 1 --2- 172.21.15.5:0/3159726099 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x563a6402e0f0 0x563a64030540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:18.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:18.746+0000 7f81403f5740 4 client.29099 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:57:18.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:18.746+0000 7f81403f5740 1 -- 172.21.15.5:0/3159726099 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563a63f86950 con 0x563a6402e0f0 2024-09-10T09:57:18.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:18.746+0000 7f8136a00640 1 --2- 172.21.15.5:0/3159726099 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x563a6402e0f0 0x563a64030540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:18.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:18.746+0000 7f8136a00640 1 --2- 172.21.15.5:0/3159726099 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x563a6402e0f0 0x563a64030540 crc :-1 s=READY pgs=265 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b2e1946124254bb7 server_cookie=3de08d774b4dcdd0 in_seq=0 out_seq=0 2024-09-10T09:57:18.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:18.746+0000 7f8120c00640 10 client.29099 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T09:57:18.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:18.746+0000 7f8120c00640 1 -- 172.21.15.5:0/3159726099 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563a63f86950 con 0x563a6402e0f0 2024-09-10T09:57:18.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:18.746+0000 7f8120c00640 10 client.29099 handle_command_reply: tid=0 2024-09-10T09:57:18.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:18.746+0000 7f81403f5740 1 client.29099 shutdown 2024-09-10T09:57:18.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:18.746+0000 7f81403f5740 1 -- 172.21.15.5:0/3159726099 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563a64013760 msgr2=0x563a6401b460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:57:18.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:18.746+0000 7f81403f5740 1 --2- 172.21.15.5:0/3159726099 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563a64013760 0x563a6401b460 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f81240c4690 tx=0x7f8124031490 comp rx=0 tx=0).stop 2024-09-10T09:57:18.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:18.746+0000 7f81403f5740 1 -- 172.21.15.5:0/3159726099 shutdown_connections 2024-09-10T09:57:18.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:18.746+0000 7f81403f5740 1 --2- 172.21.15.5:0/3159726099 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x563a6402e0f0 0x563a64030540 crc :-1 s=READY pgs=265 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:18.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:18.746+0000 7f81403f5740 1 --2- 172.21.15.5:0/3159726099 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563a6401e020 0x563a6401b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:18.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:18.746+0000 7f8120c00640 0 client.29099 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T09:57:18.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:18.746+0000 7f81403f5740 1 --2- 172.21.15.5:0/3159726099 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563a64013760 0x563a6401b460 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:18.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:18.746+0000 7f81403f5740 1 --2- 172.21.15.5:0/3159726099 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563a64012e40 0x563a6401af20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:18.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:18.746+0000 7f81403f5740 1 -- 172.21.15.5:0/3159726099 >> 172.21.15.5:0/3159726099 conn(0x563a6400f3d0 msgr2=0x563a640150b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:57:18.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:18.746+0000 7f81403f5740 1 -- 172.21.15.5:0/3159726099 shutdown_connections 2024-09-10T09:57:18.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:18.746+0000 7f81403f5740 1 -- 172.21.15.5:0/3159726099 wait complete. 2024-09-10T09:57:18.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:18.746+0000 7f81403f5740 20 client.29099 trim_cache size 0 max 16384 2024-09-10T09:57:18.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:18.763+0000 7f81403f5740 1 Processor -- start 2024-09-10T09:57:18.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:18.763+0000 7f81403f5740 1 -- start start 2024-09-10T09:57:18.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:18.765+0000 7f81403f5740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563a64012e40 0x563a640158a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:18.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:18.765+0000 7f81403f5740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563a64013760 0x563a6401ce90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:18.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:18.765+0000 7f81403f5740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563a6401e020 0x563a64035e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:18.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:18.765+0000 7f81403f5740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f81381b0ee0 con 0x563a64013760 2024-09-10T09:57:18.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:18.765+0000 7f81403f5740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f81381b2a20 con 0x563a6401e020 2024-09-10T09:57:18.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:18.765+0000 7f81403f5740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f81381b15b0 con 0x563a64012e40 2024-09-10T09:57:18.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:18.765+0000 7f8136000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563a64013760 0x563a6401ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:18.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:18.765+0000 7f8136000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563a64013760 0x563a6401ce90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60084/0 (socket says 172.21.15.5:60084) 2024-09-10T09:57:18.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:18.765+0000 7f8136000640 1 -- 172.21.15.5:0/33686708 learned_addr learned my addr 172.21.15.5:0/33686708 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:57:18.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:18.765+0000 7f8137400640 1 --2- 172.21.15.5:0/33686708 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563a6401e020 0x563a64035e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:18.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:18.765+0000 7f8136a00640 1 --2- 172.21.15.5:0/33686708 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563a64012e40 0x563a640158a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:18.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:18.765+0000 7f8137400640 1 -- 172.21.15.5:0/33686708 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563a64012e40 msgr2=0x563a640158a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:57:18.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:18.765+0000 7f8137400640 1 --2- 172.21.15.5:0/33686708 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563a64012e40 0x563a640158a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:18.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:18.765+0000 7f8137400640 1 -- 172.21.15.5:0/33686708 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563a64013760 msgr2=0x563a6401ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:57:18.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:18.765+0000 7f8137400640 1 --2- 172.21.15.5:0/33686708 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563a64013760 0x563a6401ce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:18.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:18.765+0000 7f8137400640 1 -- 172.21.15.5:0/33686708 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563a64030b70 con 0x563a6401e020 2024-09-10T09:57:18.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:18.765+0000 7f8136000640 1 --2- 172.21.15.5:0/33686708 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563a64013760 0x563a6401ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:57:18.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:18.765+0000 7f8137400640 1 --2- 172.21.15.5:0/33686708 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563a6401e020 0x563a64035e00 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f8130012e10 tx=0x7f8130005bd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9fa64e8b40468621 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:57:18.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:18.765+0000 7f8123400640 1 -- 172.21.15.5:0/33686708 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8130010e60 con 0x563a6401e020 2024-09-10T09:57:18.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:18.765+0000 7f8123400640 1 -- 172.21.15.5:0/33686708 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f813000e8d0 con 0x563a6401e020 2024-09-10T09:57:18.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:18.765+0000 7f8123400640 1 -- 172.21.15.5:0/33686708 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8130012650 con 0x563a6401e020 2024-09-10T09:57:18.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:18.765+0000 7f81403f5740 1 -- 172.21.15.5:0/33686708 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563a6401e020 msgr2=0x563a64035e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:57:18.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:18.765+0000 7f81403f5740 1 --2- 172.21.15.5:0/33686708 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563a6401e020 0x563a64035e00 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f8130012e10 tx=0x7f8130005bd0 comp rx=0 tx=0).stop 2024-09-10T09:57:18.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:18.765+0000 7f81403f5740 1 -- 172.21.15.5:0/33686708 shutdown_connections 2024-09-10T09:57:18.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:18.765+0000 7f81403f5740 1 --2- 172.21.15.5:0/33686708 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563a6401e020 0x563a64035e00 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:18.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:18.765+0000 7f81403f5740 1 --2- 172.21.15.5:0/33686708 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563a64013760 0x563a6401ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:18.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:18.765+0000 7f81403f5740 1 --2- 172.21.15.5:0/33686708 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563a64012e40 0x563a640158a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:18.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:18.765+0000 7f81403f5740 1 -- 172.21.15.5:0/33686708 >> 172.21.15.5:0/33686708 conn(0x563a6400f3d0 msgr2=0x563a6401c780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:57:18.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:18.765+0000 7f81403f5740 1 -- 172.21.15.5:0/33686708 shutdown_connections 2024-09-10T09:57:18.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:18.765+0000 7f81403f5740 1 -- 172.21.15.5:0/33686708 wait complete. 2024-09-10T09:57:18.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:18.765+0000 7f81403f5740 1 -- start start 2024-09-10T09:57:18.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:18.765+0000 7f81403f5740 1 Processor -- start 2024-09-10T09:57:18.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:18.765+0000 7f81403f5740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563a64012e40 0x563a64021c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:18.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:18.765+0000 7f81403f5740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563a64013760 0x563a640221d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:18.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:18.765+0000 7f81403f5740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563a6401e020 0x563a64022710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:18.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:18.765+0000 7f81403f5740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f81381b2350 con 0x563a64012e40 2024-09-10T09:57:18.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:18.765+0000 7f81403f5740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f81381b1c80 con 0x563a6401e020 2024-09-10T09:57:18.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:18.765+0000 7f81403f5740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f81381aeec0 con 0x563a64013760 2024-09-10T09:57:18.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:18.765+0000 7f81403f5740 10 client.0 _ll_register_callbacks cb 0x7f81240f3270 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:57:18.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:18.765+0000 7f8136a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563a64012e40 0x563a64021c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:18.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:18.765+0000 7f8136000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563a64013760 0x563a640221d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:18.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:18.765+0000 7f8136a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563a64012e40 0x563a64021c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60094/0 (socket says 172.21.15.5:60094) 2024-09-10T09:57:18.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:18.765+0000 7f8136a00640 1 -- 172.21.15.5:0/62622379 learned_addr learned my addr 172.21.15.5:0/62622379 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:57:18.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:18.765+0000 7f8137400640 1 --2- 172.21.15.5:0/62622379 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563a6401e020 0x563a64022710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:18.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:18.765+0000 7f8136a00640 1 -- 172.21.15.5:0/62622379 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563a64013760 msgr2=0x563a640221d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:57:18.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:18.765+0000 7f8136a00640 1 --2- 172.21.15.5:0/62622379 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563a64013760 0x563a640221d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:18.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:18.765+0000 7f8136a00640 1 -- 172.21.15.5:0/62622379 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563a6401e020 msgr2=0x563a64022710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:57:18.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:18.765+0000 7f8136a00640 1 --2- 172.21.15.5:0/62622379 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563a6401e020 0x563a64022710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:18.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:18.765+0000 7f8136a00640 1 -- 172.21.15.5:0/62622379 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563a64020ad0 con 0x563a64012e40 2024-09-10T09:57:18.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:18.765+0000 7f8136000640 1 --2- 172.21.15.5:0/62622379 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563a64013760 0x563a640221d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:57:18.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:18.765+0000 7f8137400640 1 --2- 172.21.15.5:0/62622379 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563a6401e020 0x563a64022710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:57:18.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:18.765+0000 7f8136a00640 1 --2- 172.21.15.5:0/62622379 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563a64012e40 0x563a64021c90 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x563a64030da0 tx=0x7f81280046e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f2221f0373ca514 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:57:18.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:18.765+0000 7f8120c00640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-10T09:57:18.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:18.765+0000 7f8120c00640 1 -- 172.21.15.5:0/62622379 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8128030060 con 0x563a64012e40 2024-09-10T09:57:18.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:18.765+0000 7f81403f5740 1 -- 172.21.15.5:0/62622379 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f813000e8d0 con 0x563a64012e40 2024-09-10T09:57:18.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:18.765+0000 7f8120c00640 1 -- 172.21.15.5:0/62622379 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f8128017720 con 0x563a64012e40 2024-09-10T09:57:18.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:18.770+0000 7f8120c00640 1 -- 172.21.15.5:0/62622379 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f8128037050 con 0x563a64012e40 2024-09-10T09:57:18.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:18.770+0000 7f81403f5740 10 client.19053 fetch_fsmap learned FSMap version 42 2024-09-10T09:57:18.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:18.770+0000 7f81403f5740 1 -- 172.21.15.5:0/62622379 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x563a64098f10 con 0x563a64012e40 2024-09-10T09:57:18.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:18.770+0000 7f8120c00640 1 -- 172.21.15.5:0/62622379 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f813000e8d0 con 0x563a64012e40 2024-09-10T09:57:18.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:18.770+0000 7f8120c00640 1 -- 172.21.15.5:0/62622379 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 42) ==== 8370+0+0 (secure 0 0 0) 0x7f8128012030 con 0x563a64012e40 2024-09-10T09:57:18.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:18.770+0000 7f81403f5740 10 client.19053 fetch_fsmap finished waiting for FSMap version 42 2024-09-10T09:57:18.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:18.770+0000 7f81403f5740 10 client.19053 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T09:57:18.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:18.770+0000 7f81403f5740 20 client.19053 populate_metadata read hostname 'smithi005' 2024-09-10T09:57:18.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:18.770+0000 7f81403f5740 1 --2- 172.21.15.5:0/62622379 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x563a64099630 0x563a6409ba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:18.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:18.770+0000 7f81403f5740 4 client.19053 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T09:57:18.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:18.770+0000 7f81403f5740 1 -- 172.21.15.5:0/62622379 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563a63f80e60 con 0x563a64099630 2024-09-10T09:57:18.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:18.770+0000 7f8136000640 1 --2- 172.21.15.5:0/62622379 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x563a64099630 0x563a6409ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:18.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:18.770+0000 7f8136000640 1 --2- 172.21.15.5:0/62622379 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x563a64099630 0x563a6409ba20 crc :-1 s=READY pgs=266 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a55d0fc66bfeabe server_cookie=a830ce8017c9eab6 in_seq=0 out_seq=0 2024-09-10T09:57:18.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:18.770+0000 7f8120c00640 10 client.19053 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T09:57:18.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:18.770+0000 7f8120c00640 1 -- 172.21.15.5:0/62622379 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563a63f80e60 con 0x563a64099630 2024-09-10T09:57:18.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:18.770+0000 7f8120c00640 10 client.19053 handle_command_reply: tid=0 2024-09-10T09:57:18.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:18.770+0000 7f81403f5740 1 client.19053 shutdown 2024-09-10T09:57:18.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:18.770+0000 7f81403f5740 1 -- 172.21.15.5:0/62622379 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563a64012e40 msgr2=0x563a64021c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:57:18.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:18.770+0000 7f81403f5740 1 --2- 172.21.15.5:0/62622379 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563a64012e40 0x563a64021c90 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x563a64030da0 tx=0x7f81280046e0 comp rx=0 tx=0).stop 2024-09-10T09:57:18.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:18.770+0000 7f81403f5740 1 -- 172.21.15.5:0/62622379 shutdown_connections 2024-09-10T09:57:18.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:18.770+0000 7f81403f5740 1 --2- 172.21.15.5:0/62622379 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x563a64099630 0x563a6409ba20 crc :-1 s=READY pgs=266 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:18.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:18.770+0000 7f81403f5740 1 --2- 172.21.15.5:0/62622379 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563a6401e020 0x563a64022710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:18.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:18.770+0000 7f81403f5740 1 --2- 172.21.15.5:0/62622379 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563a64013760 0x563a640221d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:18.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:18.770+0000 7f81403f5740 1 --2- 172.21.15.5:0/62622379 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563a64012e40 0x563a64021c90 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:18.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:18.770+0000 7f81403f5740 1 -- 172.21.15.5:0/62622379 >> 172.21.15.5:0/62622379 conn(0x563a6400f3d0 msgr2=0x563a6401b700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:57:18.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:18.770+0000 7f8120c00640 0 client.19053 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T09:57:18.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:18.770+0000 7f81403f5740 1 -- 172.21.15.5:0/62622379 shutdown_connections 2024-09-10T09:57:18.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:18.770+0000 7f81403f5740 1 -- 172.21.15.5:0/62622379 wait complete. 2024-09-10T09:57:18.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:18.770+0000 7f81403f5740 20 client.19053 trim_cache size 0 max 16384 2024-09-10T09:57:18.776 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-10T09:57:18.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:18.770+0000 7f8137e00640 1 -- 172.21.15.5:0/1862089254 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f811c0a2c00 msgr2=0x7f811c0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:57:18.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:18.770+0000 7f8137e00640 1 --2- 172.21.15.5:0/1862089254 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f811c0a2c00 0x7f811c0a50c0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f8128006fd0 tx=0x7f8128008040 comp rx=0 tx=0).stop 2024-09-10T09:57:18.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:18.770+0000 7f8137e00640 1 -- 172.21.15.5:0/1862089254 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8138102360 msgr2=0x7f813819aa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:57:18.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:18.770+0000 7f8137e00640 1 --2- 172.21.15.5:0/1862089254 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8138102360 0x7f813819aa90 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f812402a980 tx=0x7f8124003b90 comp rx=0 tx=0).stop 2024-09-10T09:57:18.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:18.770+0000 7f8137e00640 1 -- 172.21.15.5:0/1862089254 shutdown_connections 2024-09-10T09:57:18.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:18.770+0000 7f8137e00640 1 --2- 172.21.15.5:0/1862089254 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f811c0a2c00 0x7f811c0a50c0 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:18.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:18.770+0000 7f8137e00640 1 --2- 172.21.15.5:0/1862089254 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f81381079c0 0x7f813819ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:18.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:18.770+0000 7f8137e00640 1 --2- 172.21.15.5:0/1862089254 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8138102360 0x7f813819aa90 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:18.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:18.770+0000 7f8137e00640 1 --2- 172.21.15.5:0/1862089254 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f81381019c0 0x7f813819a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:18.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:18.770+0000 7f8137e00640 1 -- 172.21.15.5:0/1862089254 >> 172.21.15.5:0/1862089254 conn(0x7f81380fd740 msgr2=0x7f81380fdd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:57:18.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:18.770+0000 7f8137e00640 1 -- 172.21.15.5:0/1862089254 shutdown_connections 2024-09-10T09:57:18.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:18.770+0000 7f8137e00640 1 -- 172.21.15.5:0/1862089254 wait complete. 2024-09-10T09:57:18.990 INFO:tasks.workunit.client.0.smithi005.stdout: 10 87534 14.75 MB/sec execute 177 sec latency 20360.885 ms 2024-09-10T09:57:19.532 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:57:19 smithi005 ceph-mon[30350]: pgmap v962: 129 pgs: 129 active+clean; 662 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 272 KiB/s rd, 4.9 MiB/s wr, 232 op/s 2024-09-10T09:57:19.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:57:19 smithi159 ceph-mon[35513]: pgmap v962: 129 pgs: 129 active+clean; 662 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 272 KiB/s rd, 4.9 MiB/s wr, 232 op/s 2024-09-10T09:57:19.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:57:19 smithi102 ceph-mon[35442]: pgmap v962: 129 pgs: 129 active+clean; 662 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 272 KiB/s rd, 4.9 MiB/s wr, 232 op/s 2024-09-10T09:57:19.990 INFO:tasks.workunit.client.0.smithi005.stdout: 10 87534 14.66 MB/sec execute 178 sec latency 21360.940 ms 2024-09-10T09:57:20.935 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-10T09:57:20.935 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-10T09:57:20.990 INFO:tasks.workunit.client.0.smithi005.stdout: 10 87534 14.58 MB/sec execute 179 sec latency 22360.978 ms 2024-09-10T09:57:21.317 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:57:21.516 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:57:21 smithi005 ceph-mon[30350]: pgmap v963: 129 pgs: 129 active+clean; 618 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 500 KiB/s rd, 2.1 MiB/s wr, 222 op/s 2024-09-10T09:57:21.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:57:21 smithi159 ceph-mon[35513]: pgmap v963: 129 pgs: 129 active+clean; 618 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 500 KiB/s rd, 2.1 MiB/s wr, 222 op/s 2024-09-10T09:57:21.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:57:21 smithi102 ceph-mon[35442]: pgmap v963: 129 pgs: 129 active+clean; 618 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 500 KiB/s rd, 2.1 MiB/s wr, 222 op/s 2024-09-10T09:57:21.990 INFO:tasks.workunit.client.0.smithi005.stdout: 10 87534 14.50 MB/sec execute 180 sec latency 23361.035 ms 2024-09-10T09:57:22.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:22.608+0000 7fef54000640 1 -- 172.21.15.5:0/750864257 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fef4c1049c0 msgr2=0x7fef4c108a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:57:22.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:22.608+0000 7fef54000640 1 --2- 172.21.15.5:0/750864257 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fef4c1049c0 0x7fef4c108a10 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7fef38009900 tx=0x7fef3802a3f0 comp rx=0 tx=0).stop 2024-09-10T09:57:22.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:22.608+0000 7fef54000640 1 -- 172.21.15.5:0/750864257 shutdown_connections 2024-09-10T09:57:22.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:22.608+0000 7fef54000640 1 --2- 172.21.15.5:0/750864257 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fef4c109140 0x7fef4c110cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:22.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:22.608+0000 7fef54000640 1 --2- 172.21.15.5:0/750864257 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fef4c1049c0 0x7fef4c108a10 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:22.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:22.608+0000 7fef54000640 1 --2- 172.21.15.5:0/750864257 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fef4c104010 0x7fef4c1043f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:22.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:22.608+0000 7fef54000640 1 -- 172.21.15.5:0/750864257 >> 172.21.15.5:0/750864257 conn(0x7fef4c0ffa50 msgr2=0x7fef4c101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:57:22.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:22.608+0000 7fef54000640 1 -- 172.21.15.5:0/750864257 shutdown_connections 2024-09-10T09:57:22.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:22.608+0000 7fef54000640 1 -- 172.21.15.5:0/750864257 wait complete. 2024-09-10T09:57:22.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:22.608+0000 7fef54000640 1 Processor -- start 2024-09-10T09:57:22.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:22.608+0000 7fef54000640 1 -- start start 2024-09-10T09:57:22.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:22.608+0000 7fef54000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fef4c104010 0x7fef4c19a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:22.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:22.608+0000 7fef54000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fef4c1049c0 0x7fef4c19ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:22.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:22.608+0000 7fef54000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fef4c109140 0x7fef4c19efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:22.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:22.608+0000 7fef54000640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fef4c114190 con 0x7fef4c104010 2024-09-10T09:57:22.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:22.608+0000 7fef54000640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fef4c114010 con 0x7fef4c109140 2024-09-10T09:57:22.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:22.608+0000 7fef54000640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fef4c114310 con 0x7fef4c1049c0 2024-09-10T09:57:22.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:22.608+0000 7fef51600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fef4c104010 0x7fef4c19a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:22.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:22.608+0000 7fef50c00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fef4c1049c0 0x7fef4c19ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:22.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:22.608+0000 7fef51600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fef4c104010 0x7fef4c19a6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51304/0 (socket says 172.21.15.5:51304) 2024-09-10T09:57:22.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:22.608+0000 7fef51600640 1 -- 172.21.15.5:0/1124338024 learned_addr learned my addr 172.21.15.5:0/1124338024 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:57:22.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:22.608+0000 7fef52000640 1 --2- 172.21.15.5:0/1124338024 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fef4c109140 0x7fef4c19efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:22.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:22.608+0000 7fef51600640 1 -- 172.21.15.5:0/1124338024 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fef4c1049c0 msgr2=0x7fef4c19ac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:57:22.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:22.608+0000 7fef51600640 1 --2- 172.21.15.5:0/1124338024 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fef4c1049c0 0x7fef4c19ac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:22.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:22.608+0000 7fef51600640 1 -- 172.21.15.5:0/1124338024 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fef4c109140 msgr2=0x7fef4c19efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:57:22.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:22.608+0000 7fef51600640 1 --2- 172.21.15.5:0/1124338024 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fef4c109140 0x7fef4c19efc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:22.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:22.608+0000 7fef51600640 1 -- 172.21.15.5:0/1124338024 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef4c19f6a0 con 0x7fef4c104010 2024-09-10T09:57:22.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:22.608+0000 7fef51600640 1 --2- 172.21.15.5:0/1124338024 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fef4c104010 0x7fef4c19a6f0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7fef4000d950 tx=0x7fef4000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:57:22.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:22.608+0000 7fef4a000640 1 -- 172.21.15.5:0/1124338024 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef40017070 con 0x7fef4c104010 2024-09-10T09:57:22.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:22.608+0000 7fef4a000640 1 -- 172.21.15.5:0/1124338024 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fef40009c30 con 0x7fef4c104010 2024-09-10T09:57:22.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:22.608+0000 7fef54000640 1 -- 172.21.15.5:0/1124338024 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef4c19f990 con 0x7fef4c104010 2024-09-10T09:57:22.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:22.608+0000 7fef4a000640 1 -- 172.21.15.5:0/1124338024 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef4000f6b0 con 0x7fef4c104010 2024-09-10T09:57:22.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:22.608+0000 7fef54000640 1 -- 172.21.15.5:0/1124338024 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef4c1a7270 con 0x7fef4c104010 2024-09-10T09:57:22.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:22.612+0000 7fef56a8b740 1 Processor -- start 2024-09-10T09:57:22.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:22.612+0000 7fef4a000640 1 -- 172.21.15.5:0/1124338024 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7fef400109d0 con 0x7fef4c104010 2024-09-10T09:57:22.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:22.612+0000 7fef56a8b740 1 -- start start 2024-09-10T09:57:22.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:22.612+0000 7fef56a8b740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55655eb20ac0 0x55655eb20ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:22.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:22.612+0000 7fef56a8b740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55655eb213e0 0x55655eb2b760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:22.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:22.612+0000 7fef56a8b740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55655eb2bca0 0x55655eb2e090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:22.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:22.612+0000 7fef56a8b740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55655ea87290 con 0x55655eb2bca0 2024-09-10T09:57:22.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:22.612+0000 7fef56a8b740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55655ea2eb40 con 0x55655eb20ac0 2024-09-10T09:57:22.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:22.612+0000 7fef56a8b740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55655ea83250 con 0x55655eb213e0 2024-09-10T09:57:22.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:22.612+0000 7fef51600640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55655eb213e0 0x55655eb2b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:22.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:22.612+0000 7fef50c00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55655eb2bca0 0x55655eb2e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:22.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:22.612+0000 7fef52000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55655eb20ac0 0x55655eb20ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:22.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:22.612+0000 7fef50c00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55655eb2bca0 0x55655eb2e090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51320/0 (socket says 172.21.15.5:51320) 2024-09-10T09:57:22.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:22.612+0000 7fef50c00640 1 -- 172.21.15.5:0/1478520995 learned_addr learned my addr 172.21.15.5:0/1478520995 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:57:22.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:22.612+0000 7fef52000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55655eb20ac0 0x55655eb20ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:38214/0 (socket says 172.21.15.5:38214) 2024-09-10T09:57:22.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:22.612+0000 7fef52000640 1 -- 172.21.15.5:0/1478520995 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55655eb213e0 msgr2=0x55655eb2b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:57:22.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:22.612+0000 7fef52000640 1 --2- 172.21.15.5:0/1478520995 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55655eb213e0 0x55655eb2b760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:22.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:22.612+0000 7fef52000640 1 -- 172.21.15.5:0/1478520995 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55655eb2bca0 msgr2=0x55655eb2e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:57:22.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:22.612+0000 7fef52000640 1 --2- 172.21.15.5:0/1478520995 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55655eb2bca0 0x55655eb2e090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:22.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:22.612+0000 7fef52000640 1 -- 172.21.15.5:0/1478520995 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55655e7f86a0 con 0x55655eb20ac0 2024-09-10T09:57:22.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:22.612+0000 7fef4a000640 1 --2- 172.21.15.5:0/1124338024 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fef340a2b80 0x7fef340a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:22.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:22.612+0000 7fef52000640 1 --2- 172.21.15.5:0/1478520995 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55655eb20ac0 0x55655eb20ea0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fef44009eb0 tx=0x7fef44010740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ce27a306dfe518fa server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:57:22.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:22.612+0000 7fef4a000640 1 -- 172.21.15.5:0/1124338024 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fef400ce960 con 0x7fef4c104010 2024-09-10T09:57:22.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:22.612+0000 7fef51600640 1 --2- 172.21.15.5:0/1124338024 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fef340a2b80 0x7fef340a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:22.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:22.612+0000 7fef3e600640 1 -- 172.21.15.5:0/1478520995 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef44016070 con 0x55655eb20ac0 2024-09-10T09:57:22.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:22.612+0000 7fef3e600640 1 -- 172.21.15.5:0/1478520995 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fef44004d20 con 0x55655eb20ac0 2024-09-10T09:57:22.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:22.612+0000 7fef3e600640 1 -- 172.21.15.5:0/1478520995 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef44012810 con 0x55655eb20ac0 2024-09-10T09:57:22.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:22.612+0000 7fef56a8b740 1 -- 172.21.15.5:0/1478520995 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55655eb20ac0 msgr2=0x55655eb20ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:57:22.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:22.612+0000 7fef51600640 1 --2- 172.21.15.5:0/1124338024 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fef340a2b80 0x7fef340a5040 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fef400cec00 tx=0x7fef400d4530 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:57:22.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:22.612+0000 7fef56a8b740 1 --2- 172.21.15.5:0/1478520995 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55655eb20ac0 0x55655eb20ea0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fef44009eb0 tx=0x7fef44010740 comp rx=0 tx=0).stop 2024-09-10T09:57:22.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:22.612+0000 7fef56a8b740 1 -- 172.21.15.5:0/1478520995 shutdown_connections 2024-09-10T09:57:22.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:22.612+0000 7fef56a8b740 1 --2- 172.21.15.5:0/1478520995 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55655eb2bca0 0x55655eb2e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:22.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:22.612+0000 7fef56a8b740 1 --2- 172.21.15.5:0/1478520995 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55655eb213e0 0x55655eb2b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:22.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:22.612+0000 7fef56a8b740 1 --2- 172.21.15.5:0/1478520995 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55655eb20ac0 0x55655eb20ea0 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:22.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:22.612+0000 7fef56a8b740 1 -- 172.21.15.5:0/1478520995 >> 172.21.15.5:0/1478520995 conn(0x55655eb1d050 msgr2=0x55655eb1d430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:57:22.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:22.612+0000 7fef56a8b740 1 -- 172.21.15.5:0/1478520995 shutdown_connections 2024-09-10T09:57:22.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:22.618+0000 7fef56a8b740 1 -- 172.21.15.5:0/1478520995 wait complete. 2024-09-10T09:57:22.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:22.618+0000 7fef56a8b740 1 -- start start 2024-09-10T09:57:22.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:22.618+0000 7fef56a8b740 1 Processor -- start 2024-09-10T09:57:22.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:22.618+0000 7fef56a8b740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55655eb20ac0 0x55655eb2b660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:22.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:22.618+0000 7fef56a8b740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55655eb213e0 0x55655eb28910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:22.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:22.618+0000 7fef56a8b740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55655eb2bca0 0x55655eb28f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:22.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:22.618+0000 7fef56a8b740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55655eb2e910 con 0x55655eb213e0 2024-09-10T09:57:22.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:22.618+0000 7fef56a8b740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55655ea41fa0 con 0x55655eb20ac0 2024-09-10T09:57:22.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:22.618+0000 7fef56a8b740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55655eb2ea90 con 0x55655eb2bca0 2024-09-10T09:57:22.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:22.618+0000 7fef56a8b740 10 client.0 _ll_register_callbacks cb 0x55655e795ba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:57:22.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:22.618+0000 7fef52000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55655eb20ac0 0x55655eb2b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:22.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:22.618+0000 7fef52000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55655eb20ac0 0x55655eb2b660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:38216/0 (socket says 172.21.15.5:38216) 2024-09-10T09:57:22.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:22.618+0000 7fef50c00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55655eb213e0 0x55655eb28910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:22.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:22.618+0000 7fef52000640 1 -- 172.21.15.5:0/219895459 learned_addr learned my addr 172.21.15.5:0/219895459 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:57:22.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:22.618+0000 7fef52000640 1 --2- 172.21.15.5:0/219895459 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55655eb2bca0 0x55655eb28f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:22.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:22.618+0000 7fef52000640 1 -- 172.21.15.5:0/219895459 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55655eb2bca0 msgr2=0x55655eb28f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:57:22.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:22.618+0000 7fef52000640 1 --2- 172.21.15.5:0/219895459 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55655eb2bca0 0x55655eb28f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:22.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:22.618+0000 7fef52000640 1 -- 172.21.15.5:0/219895459 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55655eb213e0 msgr2=0x55655eb28910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:57:22.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:22.618+0000 7fef52000640 1 --2- 172.21.15.5:0/219895459 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55655eb213e0 0x55655eb28910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:22.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:22.618+0000 7fef52000640 1 -- 172.21.15.5:0/219895459 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55655eb29730 con 0x55655eb20ac0 2024-09-10T09:57:22.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:22.618+0000 7fef52000640 1 --2- 172.21.15.5:0/219895459 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55655eb20ac0 0x55655eb2b660 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fef44005a40 tx=0x7fef44012450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ddbeb66fb07f8052 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:57:22.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:22.618+0000 7fef32a00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-09-10T09:57:22.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:22.618+0000 7fef32a00640 1 -- 172.21.15.5:0/219895459 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef44016070 con 0x55655eb20ac0 2024-09-10T09:57:22.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:22.618+0000 7fef56a8b740 1 -- 172.21.15.5:0/219895459 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fef44004d20 con 0x55655eb20ac0 2024-09-10T09:57:22.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:22.618+0000 7fef32a00640 1 -- 172.21.15.5:0/219895459 <== mon.2 v2:172.21.15.102:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fef44012bf0 con 0x55655eb20ac0 2024-09-10T09:57:22.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:22.618+0000 7fef32a00640 1 -- 172.21.15.5:0/219895459 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7fef4402f050 con 0x55655eb20ac0 2024-09-10T09:57:22.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:22.618+0000 7fef56a8b740 10 client.29117 fetch_fsmap learned FSMap version 42 2024-09-10T09:57:22.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:22.618+0000 7fef56a8b740 1 -- 172.21.15.5:0/219895459 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55655eb3a8a0 con 0x55655eb20ac0 2024-09-10T09:57:22.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:22.618+0000 7fef32a00640 1 -- 172.21.15.5:0/219895459 <== mon.2 v2:172.21.15.102:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fef44004d20 con 0x55655eb20ac0 2024-09-10T09:57:22.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:22.618+0000 7fef32a00640 1 -- 172.21.15.5:0/219895459 <== mon.2 v2:172.21.15.102:3300/0 5 ==== fsmap(e 42) ==== 8370+0+0 (secure 0 0 0) 0x7fef4400f040 con 0x55655eb20ac0 2024-09-10T09:57:22.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:22.618+0000 7fef56a8b740 10 client.29117 fetch_fsmap finished waiting for FSMap version 42 2024-09-10T09:57:22.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:22.618+0000 7fef56a8b740 10 client.29117 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T09:57:22.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:22.618+0000 7fef56a8b740 20 client.29117 populate_metadata read hostname 'smithi005' 2024-09-10T09:57:22.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:22.618+0000 7fef56a8b740 1 --2- 172.21.15.5:0/219895459 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55655eb3bd00 0x55655eb3e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:22.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:22.618+0000 7fef56a8b740 4 client.29117 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:57:22.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:22.618+0000 7fef56a8b740 1 -- 172.21.15.5:0/219895459 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55655ea94800 con 0x55655eb3bd00 2024-09-10T09:57:22.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:22.618+0000 7fef50c00640 1 --2- 172.21.15.5:0/219895459 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55655eb3bd00 0x55655eb3e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:22.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:22.618+0000 7fef50c00640 1 --2- 172.21.15.5:0/219895459 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55655eb3bd00 0x55655eb3e0f0 crc :-1 s=READY pgs=267 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c8f691f7c3fb36ab server_cookie=bb91b1a8460961cc in_seq=0 out_seq=0 2024-09-10T09:57:22.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:22.618+0000 7fef32a00640 10 client.29117 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T09:57:22.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:22.623+0000 7fef32a00640 1 -- 172.21.15.5:0/219895459 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55655ea94800 con 0x55655eb3bd00 2024-09-10T09:57:22.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:22.623+0000 7fef32a00640 10 client.29117 handle_command_reply: tid=0 2024-09-10T09:57:22.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:22.623+0000 7fef56a8b740 1 client.29117 shutdown 2024-09-10T09:57:22.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:22.623+0000 7fef56a8b740 1 -- 172.21.15.5:0/219895459 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55655eb20ac0 msgr2=0x55655eb2b660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:57:22.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:22.623+0000 7fef56a8b740 1 --2- 172.21.15.5:0/219895459 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55655eb20ac0 0x55655eb2b660 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fef44005a40 tx=0x7fef44012450 comp rx=0 tx=0).stop 2024-09-10T09:57:22.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:22.623+0000 7fef56a8b740 1 -- 172.21.15.5:0/219895459 shutdown_connections 2024-09-10T09:57:22.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:22.623+0000 7fef56a8b740 1 --2- 172.21.15.5:0/219895459 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55655eb3bd00 0x55655eb3e0f0 crc :-1 s=READY pgs=267 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:22.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:22.623+0000 7fef56a8b740 1 --2- 172.21.15.5:0/219895459 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55655eb2bca0 0x55655eb28f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:22.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:22.623+0000 7fef32a00640 0 client.29117 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T09:57:22.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:22.623+0000 7fef56a8b740 1 --2- 172.21.15.5:0/219895459 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55655eb213e0 0x55655eb28910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:22.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:22.623+0000 7fef56a8b740 1 --2- 172.21.15.5:0/219895459 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55655eb20ac0 0x55655eb2b660 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:22.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:22.623+0000 7fef56a8b740 1 -- 172.21.15.5:0/219895459 >> 172.21.15.5:0/219895459 conn(0x55655eb1d050 msgr2=0x55655eb1c1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:57:22.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:22.623+0000 7fef56a8b740 1 -- 172.21.15.5:0/219895459 shutdown_connections 2024-09-10T09:57:22.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:22.623+0000 7fef56a8b740 1 -- 172.21.15.5:0/219895459 wait complete. 2024-09-10T09:57:22.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:22.624+0000 7fef56a8b740 20 client.29117 trim_cache size 0 max 16384 2024-09-10T09:57:22.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:22.642+0000 7fef56a8b740 1 Processor -- start 2024-09-10T09:57:22.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:22.642+0000 7fef56a8b740 1 -- start start 2024-09-10T09:57:22.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:22.642+0000 7fef56a8b740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55655eb20ac0 0x55655eb1c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:22.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:22.642+0000 7fef56a8b740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55655eb213e0 0x55655eb43690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:22.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:22.642+0000 7fef56a8b740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55655eb2bca0 0x55655eb43bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:22.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:22.642+0000 7fef56a8b740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fef4c1b54e0 con 0x55655eb20ac0 2024-09-10T09:57:22.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:22.642+0000 7fef56a8b740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fef4c1b7020 con 0x55655eb213e0 2024-09-10T09:57:22.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:22.642+0000 7fef56a8b740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fef4c1b5bb0 con 0x55655eb2bca0 2024-09-10T09:57:22.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:22.642+0000 7fef52000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55655eb20ac0 0x55655eb1c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:22.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:22.642+0000 7fef52000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55655eb20ac0 0x55655eb1c9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51336/0 (socket says 172.21.15.5:51336) 2024-09-10T09:57:22.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:22.642+0000 7fef52000640 1 -- 172.21.15.5:0/1317025506 learned_addr learned my addr 172.21.15.5:0/1317025506 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:57:22.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:22.642+0000 7fef50c00640 1 --2- 172.21.15.5:0/1317025506 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55655eb213e0 0x55655eb43690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:22.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:22.642+0000 7fef52000640 1 --2- 172.21.15.5:0/1317025506 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55655eb2bca0 0x55655eb43bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:22.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:22.642+0000 7fef52000640 1 -- 172.21.15.5:0/1317025506 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55655eb2bca0 msgr2=0x55655eb43bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:57:22.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:22.642+0000 7fef52000640 1 --2- 172.21.15.5:0/1317025506 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55655eb2bca0 0x55655eb43bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:22.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:22.642+0000 7fef52000640 1 -- 172.21.15.5:0/1317025506 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55655eb213e0 msgr2=0x55655eb43690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:57:22.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:22.642+0000 7fef52000640 1 --2- 172.21.15.5:0/1317025506 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55655eb213e0 0x55655eb43690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:22.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:22.642+0000 7fef52000640 1 -- 172.21.15.5:0/1317025506 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55655eb3e460 con 0x55655eb20ac0 2024-09-10T09:57:22.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:22.642+0000 7fef50c00640 1 --2- 172.21.15.5:0/1317025506 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55655eb213e0 0x55655eb43690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:57:22.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:22.642+0000 7fef52000640 1 --2- 172.21.15.5:0/1317025506 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55655eb20ac0 0x55655eb1c9b0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7fef44009770 tx=0x7fef44010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7a9c540d705bf50 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:57:22.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:22.642+0000 7fef3dc00640 1 -- 172.21.15.5:0/1317025506 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef44016040 con 0x55655eb20ac0 2024-09-10T09:57:22.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:22.642+0000 7fef3dc00640 1 -- 172.21.15.5:0/1317025506 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fef4401f030 con 0x55655eb20ac0 2024-09-10T09:57:22.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:22.642+0000 7fef3dc00640 1 -- 172.21.15.5:0/1317025506 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef4401ac70 con 0x55655eb20ac0 2024-09-10T09:57:22.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:22.642+0000 7fef56a8b740 1 -- 172.21.15.5:0/1317025506 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55655eb20ac0 msgr2=0x55655eb1c9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:57:22.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:22.642+0000 7fef56a8b740 1 --2- 172.21.15.5:0/1317025506 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55655eb20ac0 0x55655eb1c9b0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7fef44009770 tx=0x7fef44010740 comp rx=0 tx=0).stop 2024-09-10T09:57:22.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:22.642+0000 7fef56a8b740 1 -- 172.21.15.5:0/1317025506 shutdown_connections 2024-09-10T09:57:22.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:22.642+0000 7fef56a8b740 1 --2- 172.21.15.5:0/1317025506 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55655eb2bca0 0x55655eb43bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:22.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:22.642+0000 7fef56a8b740 1 --2- 172.21.15.5:0/1317025506 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55655eb213e0 0x55655eb43690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:22.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:22.642+0000 7fef56a8b740 1 --2- 172.21.15.5:0/1317025506 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55655eb20ac0 0x55655eb1c9b0 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:22.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:22.642+0000 7fef56a8b740 1 -- 172.21.15.5:0/1317025506 >> 172.21.15.5:0/1317025506 conn(0x55655eb1d050 msgr2=0x55655eb23f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:57:22.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:22.642+0000 7fef56a8b740 1 -- 172.21.15.5:0/1317025506 shutdown_connections 2024-09-10T09:57:22.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:22.642+0000 7fef56a8b740 1 -- 172.21.15.5:0/1317025506 wait complete. 2024-09-10T09:57:22.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:22.642+0000 7fef56a8b740 1 -- start start 2024-09-10T09:57:22.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:22.642+0000 7fef56a8b740 1 Processor -- start 2024-09-10T09:57:22.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:22.642+0000 7fef56a8b740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55655eb20ac0 0x55655eb2fd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:22.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:22.642+0000 7fef56a8b740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55655eb213e0 0x55655eb30250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:22.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:22.642+0000 7fef56a8b740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55655eb2bca0 0x55655eb30790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:22.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:22.642+0000 7fef56a8b740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fef4c1b6950 con 0x55655eb2bca0 2024-09-10T09:57:22.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:22.642+0000 7fef56a8b740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fef4c1b6280 con 0x55655eb213e0 2024-09-10T09:57:22.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:22.642+0000 7fef56a8b740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fef4c1b34c0 con 0x55655eb20ac0 2024-09-10T09:57:22.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:22.642+0000 7fef56a8b740 10 client.0 _ll_register_callbacks cb 0x7fef38005b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:57:22.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:22.642+0000 7fef52000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55655eb2bca0 0x55655eb30790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:22.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:22.642+0000 7fef50c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55655eb213e0 0x55655eb30250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:22.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:22.642+0000 7fef52000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55655eb20ac0 0x55655eb2fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:22.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:22.642+0000 7fef50c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55655eb213e0 0x55655eb30250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:38244/0 (socket says 172.21.15.5:38244) 2024-09-10T09:57:22.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:22.642+0000 7fef50c00640 1 -- 172.21.15.5:0/3148483414 learned_addr learned my addr 172.21.15.5:0/3148483414 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:57:22.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:22.642+0000 7fef50c00640 1 -- 172.21.15.5:0/3148483414 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55655eb20ac0 msgr2=0x55655eb2fd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:57:22.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:22.642+0000 7fef50c00640 1 --2- 172.21.15.5:0/3148483414 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55655eb20ac0 0x55655eb2fd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:22.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:22.642+0000 7fef50c00640 1 -- 172.21.15.5:0/3148483414 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55655eb2bca0 msgr2=0x55655eb30790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:57:22.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:22.642+0000 7fef50c00640 1 --2- 172.21.15.5:0/3148483414 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55655eb2bca0 0x55655eb30790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:22.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:22.642+0000 7fef50c00640 1 -- 172.21.15.5:0/3148483414 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55655eb29270 con 0x55655eb213e0 2024-09-10T09:57:22.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:22.647+0000 7fef52000640 1 --2- 172.21.15.5:0/3148483414 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55655eb20ac0 0x55655eb2fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:57:22.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:22.647+0000 7fef50c00640 1 --2- 172.21.15.5:0/3148483414 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55655eb213e0 0x55655eb30250 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fef38000c00 tx=0x7fef38005e40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=82127bea8ec32240 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:57:22.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:22.647+0000 7fef32a00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-09-10T09:57:22.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:22.647+0000 7fef32a00640 1 -- 172.21.15.5:0/3148483414 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef38035040 con 0x55655eb213e0 2024-09-10T09:57:22.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:22.647+0000 7fef32a00640 1 -- 172.21.15.5:0/3148483414 <== mon.2 v2:172.21.15.102:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fef38005450 con 0x55655eb213e0 2024-09-10T09:57:22.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:22.647+0000 7fef56a8b740 1 -- 172.21.15.5:0/3148483414 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fef4401f030 con 0x55655eb213e0 2024-09-10T09:57:22.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:22.648+0000 7fef32a00640 1 -- 172.21.15.5:0/3148483414 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7fef38055050 con 0x55655eb213e0 2024-09-10T09:57:22.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:22.648+0000 7fef56a8b740 10 client.29123 fetch_fsmap learned FSMap version 42 2024-09-10T09:57:22.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:22.648+0000 7fef56a8b740 1 -- 172.21.15.5:0/3148483414 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55655eba6a40 con 0x55655eb213e0 2024-09-10T09:57:22.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:22.648+0000 7fef32a00640 1 -- 172.21.15.5:0/3148483414 <== mon.2 v2:172.21.15.102:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fef4401f030 con 0x55655eb213e0 2024-09-10T09:57:22.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:22.648+0000 7fef32a00640 1 -- 172.21.15.5:0/3148483414 <== mon.2 v2:172.21.15.102:3300/0 5 ==== fsmap(e 42) ==== 8370+0+0 (secure 0 0 0) 0x7fef380317f0 con 0x55655eb213e0 2024-09-10T09:57:22.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:22.648+0000 7fef56a8b740 10 client.29123 fetch_fsmap finished waiting for FSMap version 42 2024-09-10T09:57:22.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:22.648+0000 7fef56a8b740 10 client.29123 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T09:57:22.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:22.648+0000 7fef56a8b740 20 client.29123 populate_metadata read hostname 'smithi005' 2024-09-10T09:57:22.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:22.648+0000 7fef56a8b740 1 --2- 172.21.15.5:0/3148483414 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55655eba7210 0x55655eba9630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:22.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:22.648+0000 7fef56a8b740 4 client.29123 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-10T09:57:22.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:22.648+0000 7fef56a8b740 1 -- 172.21.15.5:0/3148483414 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55655ea8ed10 con 0x55655eba7210 2024-09-10T09:57:22.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:22.671+0000 7fef52000640 1 --2- 172.21.15.5:0/3148483414 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55655eba7210 0x55655eba9630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:22.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:22.730+0000 7fef52000640 1 --2- 172.21.15.5:0/3148483414 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55655eba7210 0x55655eba9630 crc :-1 s=READY pgs=268 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ea9d6337db4818fb server_cookie=9df14a2720b41379 in_seq=0 out_seq=0 2024-09-10T09:57:22.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:22.730+0000 7fef32a00640 10 client.29123 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T09:57:22.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:22.742+0000 7fef32a00640 1 -- 172.21.15.5:0/3148483414 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55655ea8ed10 con 0x55655eba7210 2024-09-10T09:57:22.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:22.742+0000 7fef32a00640 10 client.29123 handle_command_reply: tid=0 2024-09-10T09:57:22.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:22.742+0000 7fef56a8b740 1 client.29123 shutdown 2024-09-10T09:57:22.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:22.742+0000 7fef56a8b740 1 -- 172.21.15.5:0/3148483414 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55655eb213e0 msgr2=0x55655eb30250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:57:22.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:22.742+0000 7fef56a8b740 1 --2- 172.21.15.5:0/3148483414 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55655eb213e0 0x55655eb30250 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fef38000c00 tx=0x7fef38005e40 comp rx=0 tx=0).stop 2024-09-10T09:57:22.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:22.742+0000 7fef56a8b740 1 -- 172.21.15.5:0/3148483414 shutdown_connections 2024-09-10T09:57:22.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:22.742+0000 7fef56a8b740 1 --2- 172.21.15.5:0/3148483414 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55655eba7210 0x55655eba9630 crc :-1 s=READY pgs=268 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:22.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:22.742+0000 7fef56a8b740 1 --2- 172.21.15.5:0/3148483414 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55655eb2bca0 0x55655eb30790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:22.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:22.742+0000 7fef56a8b740 1 --2- 172.21.15.5:0/3148483414 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55655eb213e0 0x55655eb30250 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:22.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:22.742+0000 7fef56a8b740 1 --2- 172.21.15.5:0/3148483414 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55655eb20ac0 0x55655eb2fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:22.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:22.742+0000 7fef56a8b740 1 -- 172.21.15.5:0/3148483414 >> 172.21.15.5:0/3148483414 conn(0x55655eb1d050 msgr2=0x55655eb36b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:57:22.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:22.742+0000 7fef32a00640 0 client.29123 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T09:57:22.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:22.742+0000 7fef56a8b740 1 -- 172.21.15.5:0/3148483414 shutdown_connections 2024-09-10T09:57:22.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:22.742+0000 7fef56a8b740 1 -- 172.21.15.5:0/3148483414 wait complete. 2024-09-10T09:57:22.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:22.742+0000 7fef56a8b740 20 client.29123 trim_cache size 0 max 16384 2024-09-10T09:57:22.746 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-10T09:57:22.746 INFO:teuthology.orchestra.run.smithi005.stdout: "return_code": 0, 2024-09-10T09:57:22.746 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_tag": "27348b77-ddea-4e99-9b6b-686849ae2c0d", 2024-09-10T09:57:22.746 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": "asynchronous" 2024-09-10T09:57:22.746 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-10T09:57:22.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:22.742+0000 7fef54000640 1 -- 172.21.15.5:0/1124338024 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fef340a2b80 msgr2=0x7fef340a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:57:22.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:22.742+0000 7fef54000640 1 --2- 172.21.15.5:0/1124338024 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fef340a2b80 0x7fef340a5040 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fef400cec00 tx=0x7fef400d4530 comp rx=0 tx=0).stop 2024-09-10T09:57:22.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:22.742+0000 7fef54000640 1 -- 172.21.15.5:0/1124338024 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fef4c104010 msgr2=0x7fef4c19a6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:57:22.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:22.742+0000 7fef54000640 1 --2- 172.21.15.5:0/1124338024 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fef4c104010 0x7fef4c19a6f0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7fef4000d950 tx=0x7fef4000de10 comp rx=0 tx=0).stop 2024-09-10T09:57:22.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:22.742+0000 7fef54000640 1 -- 172.21.15.5:0/1124338024 shutdown_connections 2024-09-10T09:57:22.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:22.742+0000 7fef54000640 1 --2- 172.21.15.5:0/1124338024 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fef340a2b80 0x7fef340a5040 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:22.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:22.742+0000 7fef54000640 1 --2- 172.21.15.5:0/1124338024 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fef4c109140 0x7fef4c19efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:22.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:22.742+0000 7fef54000640 1 --2- 172.21.15.5:0/1124338024 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fef4c1049c0 0x7fef4c19ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:22.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:22.742+0000 7fef54000640 1 --2- 172.21.15.5:0/1124338024 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fef4c104010 0x7fef4c19a6f0 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:22.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:22.742+0000 7fef54000640 1 -- 172.21.15.5:0/1124338024 >> 172.21.15.5:0/1124338024 conn(0x7fef4c0ffa50 msgr2=0x7fef4c100dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:57:22.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:22.742+0000 7fef54000640 1 -- 172.21.15.5:0/1124338024 shutdown_connections 2024-09-10T09:57:22.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:22.742+0000 7fef54000640 1 -- 172.21.15.5:0/1124338024 wait complete. 2024-09-10T09:57:22.990 INFO:tasks.workunit.client.0.smithi005.stdout: 10 87534 14.42 MB/sec execute 181 sec latency 24361.080 ms 2024-09-10T09:57:23.462 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:57:23 smithi005 ceph-mon[30350]: pgmap v964: 129 pgs: 129 active+clean; 616 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 420 KiB/s rd, 394 KiB/s wr, 172 op/s 2024-09-10T09:57:23.462 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:57:23 smithi005 ceph-mon[30350]: scrub queued for path: / 2024-09-10T09:57:23.462 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:57:23 smithi005 ceph-mon[30350]: scrub summary: idle+waiting paths [/] 2024-09-10T09:57:23.462 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:57:23 smithi005 ceph-mon[30350]: scrub summary: active paths [/] 2024-09-10T09:57:23.462 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:57:23 smithi005 ceph-mon[30350]: scrub summary: active paths [/] 2024-09-10T09:57:23.462 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:57:23 smithi005 ceph-mon[30350]: scrub summary: active paths [/] 2024-09-10T09:57:23.462 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:57:23 smithi005 ceph-mon[30350]: scrub summary: active paths [/] 2024-09-10T09:57:23.462 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:57:23 smithi005 ceph-mon[30350]: scrub summary: idle+waiting paths [/] 2024-09-10T09:57:23.462 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:57:23 smithi005 ceph-mon[30350]: scrub summary: active paths [/] 2024-09-10T09:57:23.462 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:57:23 smithi005 ceph-mon[30350]: scrub summary: idle+waiting paths [/] 2024-09-10T09:57:23.462 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:57:23 smithi005 ceph-mon[30350]: scrub summary: idle+waiting paths [/] 2024-09-10T09:57:23.462 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:57:23 smithi005 ceph-mon[30350]: scrub summary: idle+waiting paths [/] 2024-09-10T09:57:23.463 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:57:23 smithi005 ceph-mon[30350]: scrub summary: idle+waiting paths [/] 2024-09-10T09:57:23.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:57:23 smithi102 ceph-mon[35442]: pgmap v964: 129 pgs: 129 active+clean; 616 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 420 KiB/s rd, 394 KiB/s wr, 172 op/s 2024-09-10T09:57:23.635 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:57:23 smithi102 ceph-mon[35442]: scrub queued for path: / 2024-09-10T09:57:23.635 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:57:23 smithi102 ceph-mon[35442]: scrub summary: idle+waiting paths [/] 2024-09-10T09:57:23.635 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:57:23 smithi102 ceph-mon[35442]: scrub summary: active paths [/] 2024-09-10T09:57:23.635 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:57:23 smithi102 ceph-mon[35442]: scrub summary: active paths [/] 2024-09-10T09:57:23.635 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:57:23 smithi102 ceph-mon[35442]: scrub summary: active paths [/] 2024-09-10T09:57:23.635 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:57:23 smithi102 ceph-mon[35442]: scrub summary: active paths [/] 2024-09-10T09:57:23.635 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:57:23 smithi102 ceph-mon[35442]: scrub summary: idle+waiting paths [/] 2024-09-10T09:57:23.635 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:57:23 smithi102 ceph-mon[35442]: scrub summary: active paths [/] 2024-09-10T09:57:23.635 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:57:23 smithi102 ceph-mon[35442]: scrub summary: idle+waiting paths [/] 2024-09-10T09:57:23.635 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:57:23 smithi102 ceph-mon[35442]: scrub summary: idle+waiting paths [/] 2024-09-10T09:57:23.635 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:57:23 smithi102 ceph-mon[35442]: scrub summary: idle+waiting paths [/] 2024-09-10T09:57:23.635 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:57:23 smithi102 ceph-mon[35442]: scrub summary: idle+waiting paths [/] 2024-09-10T09:57:23.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:57:23 smithi159 ceph-mon[35513]: pgmap v964: 129 pgs: 129 active+clean; 616 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 420 KiB/s rd, 394 KiB/s wr, 172 op/s 2024-09-10T09:57:23.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:57:23 smithi159 ceph-mon[35513]: scrub queued for path: / 2024-09-10T09:57:23.851 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:57:23 smithi159 ceph-mon[35513]: scrub summary: idle+waiting paths [/] 2024-09-10T09:57:23.851 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:57:23 smithi159 ceph-mon[35513]: scrub summary: active paths [/] 2024-09-10T09:57:23.851 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:57:23 smithi159 ceph-mon[35513]: scrub summary: active paths [/] 2024-09-10T09:57:23.851 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:57:23 smithi159 ceph-mon[35513]: scrub summary: active paths [/] 2024-09-10T09:57:23.851 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:57:23 smithi159 ceph-mon[35513]: scrub summary: active paths [/] 2024-09-10T09:57:23.851 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:57:23 smithi159 ceph-mon[35513]: scrub summary: idle+waiting paths [/] 2024-09-10T09:57:23.851 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:57:23 smithi159 ceph-mon[35513]: scrub summary: active paths [/] 2024-09-10T09:57:23.851 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:57:23 smithi159 ceph-mon[35513]: scrub summary: idle+waiting paths [/] 2024-09-10T09:57:23.851 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:57:23 smithi159 ceph-mon[35513]: scrub summary: idle+waiting paths [/] 2024-09-10T09:57:23.851 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:57:23 smithi159 ceph-mon[35513]: scrub summary: idle+waiting paths [/] 2024-09-10T09:57:23.851 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:57:23 smithi159 ceph-mon[35513]: scrub summary: idle+waiting paths [/] 2024-09-10T09:57:23.990 INFO:tasks.workunit.client.0.smithi005.stdout: 10 87534 14.34 MB/sec execute 182 sec latency 25361.129 ms 2024-09-10T09:57:23.993 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T09:57:24.377 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:57:24.586 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:57:24 smithi005 ceph-mon[30350]: scrub summary: active paths [/] 2024-09-10T09:57:24.586 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:57:24 smithi005 ceph-mon[30350]: scrub summary: idle+waiting paths [/] 2024-09-10T09:57:24.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:57:24 smithi159 ceph-mon[35513]: scrub summary: active paths [/] 2024-09-10T09:57:24.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:57:24 smithi159 ceph-mon[35513]: scrub summary: idle+waiting paths [/] 2024-09-10T09:57:24.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:57:24 smithi102 ceph-mon[35442]: scrub summary: active paths [/] 2024-09-10T09:57:24.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:57:24 smithi102 ceph-mon[35442]: scrub summary: idle+waiting paths [/] 2024-09-10T09:57:24.990 INFO:tasks.workunit.client.0.smithi005.stdout: 10 87534 14.26 MB/sec execute 183 sec latency 26361.193 ms 2024-09-10T09:57:25.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:25.310+0000 7f0f5f200640 1 -- 172.21.15.5:0/1007642860 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0f581019d0 msgr2=0x7f0f58101e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:57:25.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:25.310+0000 7f0f5f200640 1 --2- 172.21.15.5:0/1007642860 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0f581019d0 0x7f0f58101e30 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f0f48009930 tx=0x7f0f4802a420 comp rx=0 tx=0).stop 2024-09-10T09:57:25.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:25.310+0000 7f0f5f200640 1 -- 172.21.15.5:0/1007642860 shutdown_connections 2024-09-10T09:57:25.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:25.310+0000 7f0f5f200640 1 --2- 172.21.15.5:0/1007642860 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0f58102370 0x7f0f5810e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:25.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:25.310+0000 7f0f5f200640 1 --2- 172.21.15.5:0/1007642860 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0f581019d0 0x7f0f58101e30 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:25.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:25.310+0000 7f0f5f200640 1 --2- 172.21.15.5:0/1007642860 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0f581079d0 0x7f0f58107db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:25.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:25.310+0000 7f0f5f200640 1 -- 172.21.15.5:0/1007642860 >> 172.21.15.5:0/1007642860 conn(0x7f0f580fd710 msgr2=0x7f0f580ffb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:57:25.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:25.310+0000 7f0f5f200640 1 -- 172.21.15.5:0/1007642860 shutdown_connections 2024-09-10T09:57:25.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:25.310+0000 7f0f5f200640 1 -- 172.21.15.5:0/1007642860 wait complete. 2024-09-10T09:57:25.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:25.310+0000 7f0f5f200640 1 Processor -- start 2024-09-10T09:57:25.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:25.310+0000 7f0f5f200640 1 -- start start 2024-09-10T09:57:25.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:25.310+0000 7f0f5f200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0f581019d0 0x7f0f581982a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:25.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:25.310+0000 7f0f5f200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0f58102370 0x7f0f581987e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:25.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:25.310+0000 7f0f5f200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0f581079d0 0x7f0f5819cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:25.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:25.310+0000 7f0f5f200640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f0f58111e20 con 0x7f0f58102370 2024-09-10T09:57:25.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:25.310+0000 7f0f5f200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f0f58111ca0 con 0x7f0f581019d0 2024-09-10T09:57:25.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:25.310+0000 7f0f5f200640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f0f58111fa0 con 0x7f0f581079d0 2024-09-10T09:57:25.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:25.310+0000 7f0f5d400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0f58102370 0x7f0f581987e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:25.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:25.310+0000 7f0f5d400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0f58102370 0x7f0f581987e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51370/0 (socket says 172.21.15.5:51370) 2024-09-10T09:57:25.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:25.310+0000 7f0f5de00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0f581019d0 0x7f0f581982a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:25.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:25.310+0000 7f0f5d400640 1 -- 172.21.15.5:0/2859957599 learned_addr learned my addr 172.21.15.5:0/2859957599 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:57:25.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:25.310+0000 7f0f5e800640 1 --2- 172.21.15.5:0/2859957599 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0f581079d0 0x7f0f5819cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:25.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:25.310+0000 7f0f5d400640 1 -- 172.21.15.5:0/2859957599 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0f581079d0 msgr2=0x7f0f5819cb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:57:25.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:25.310+0000 7f0f5d400640 1 --2- 172.21.15.5:0/2859957599 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0f581079d0 0x7f0f5819cb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:25.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:25.310+0000 7f0f5d400640 1 -- 172.21.15.5:0/2859957599 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0f581019d0 msgr2=0x7f0f581982a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:57:25.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:25.310+0000 7f0f5d400640 1 --2- 172.21.15.5:0/2859957599 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0f581019d0 0x7f0f581982a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:25.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:25.310+0000 7f0f5d400640 1 -- 172.21.15.5:0/2859957599 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f5819d250 con 0x7f0f58102370 2024-09-10T09:57:25.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:25.310+0000 7f0f5e800640 1 --2- 172.21.15.5:0/2859957599 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0f581079d0 0x7f0f5819cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:57:25.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:25.310+0000 7f0f5d400640 1 --2- 172.21.15.5:0/2859957599 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0f58102370 0x7f0f581987e0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f0f4802a940 tx=0x7f0f48003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:57:25.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:25.310+0000 7f0f52a00640 1 -- 172.21.15.5:0/2859957599 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f4803f070 con 0x7f0f58102370 2024-09-10T09:57:25.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:25.310+0000 7f0f52a00640 1 -- 172.21.15.5:0/2859957599 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0f48031c90 con 0x7f0f58102370 2024-09-10T09:57:25.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:25.310+0000 7f0f5f200640 1 -- 172.21.15.5:0/2859957599 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f5819d4e0 con 0x7f0f58102370 2024-09-10T09:57:25.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:25.310+0000 7f0f5f200640 1 -- 172.21.15.5:0/2859957599 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f581a4e20 con 0x7f0f58102370 2024-09-10T09:57:25.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:25.310+0000 7f0f52a00640 1 -- 172.21.15.5:0/2859957599 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f48035030 con 0x7f0f58102370 2024-09-10T09:57:25.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:25.316+0000 7f0f633ca740 1 Processor -- start 2024-09-10T09:57:25.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:25.316+0000 7f0f633ca740 1 -- start start 2024-09-10T09:57:25.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:25.316+0000 7f0f633ca740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55914f401c50 0x55914f402030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:25.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:25.316+0000 7f0f633ca740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55914f402570 0x55914f410e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:25.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:25.316+0000 7f0f633ca740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55914f411340 0x55914f413730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:25.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:25.316+0000 7f0f633ca740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55914f3683e0 con 0x55914f401c50 2024-09-10T09:57:25.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:25.316+0000 7f0f633ca740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55914f30fd80 con 0x55914f402570 2024-09-10T09:57:25.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:25.316+0000 7f0f633ca740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55914f3643a0 con 0x55914f411340 2024-09-10T09:57:25.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:25.316+0000 7f0f52a00640 1 -- 172.21.15.5:0/2859957599 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f0f48042070 con 0x7f0f58102370 2024-09-10T09:57:25.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:25.316+0000 7f0f5e800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55914f401c50 0x55914f402030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:25.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:25.316+0000 7f0f5e800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55914f401c50 0x55914f402030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51386/0 (socket says 172.21.15.5:51386) 2024-09-10T09:57:25.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:25.316+0000 7f0f5e800640 1 -- 172.21.15.5:0/3863349476 learned_addr learned my addr 172.21.15.5:0/3863349476 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:57:25.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:25.316+0000 7f0f5de00640 1 --2- 172.21.15.5:0/3863349476 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55914f402570 0x55914f410e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:25.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:25.316+0000 7f0f5d400640 1 --2- 172.21.15.5:0/3863349476 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55914f411340 0x55914f413730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:25.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:25.316+0000 7f0f5de00640 1 -- 172.21.15.5:0/3863349476 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55914f411340 msgr2=0x55914f413730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:57:25.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:25.316+0000 7f0f5de00640 1 --2- 172.21.15.5:0/3863349476 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55914f411340 0x55914f413730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:25.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:25.316+0000 7f0f5de00640 1 -- 172.21.15.5:0/3863349476 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55914f401c50 msgr2=0x55914f402030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:57:25.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:25.316+0000 7f0f5de00640 1 --2- 172.21.15.5:0/3863349476 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55914f401c50 0x55914f402030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:25.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:25.316+0000 7f0f5de00640 1 -- 172.21.15.5:0/3863349476 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55914f32eae0 con 0x55914f402570 2024-09-10T09:57:25.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:25.316+0000 7f0f5de00640 1 --2- 172.21.15.5:0/3863349476 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55914f402570 0x55914f410e00 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f0f4c005fd0 tx=0x7f0f4c0107a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1069c8f8e06bd577 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:57:25.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:25.316+0000 7f0f47000640 1 -- 172.21.15.5:0/3863349476 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f4c01c480 con 0x55914f402570 2024-09-10T09:57:25.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:25.316+0000 7f0f47000640 1 -- 172.21.15.5:0/3863349476 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0f4c01cb00 con 0x55914f402570 2024-09-10T09:57:25.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:25.316+0000 7f0f47000640 1 -- 172.21.15.5:0/3863349476 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f4c011b00 con 0x55914f402570 2024-09-10T09:57:25.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:25.316+0000 7f0f52a00640 1 --2- 172.21.15.5:0/2859957599 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f0f400a2c00 0x7f0f400a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:25.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:25.316+0000 7f0f52a00640 1 -- 172.21.15.5:0/2859957599 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f0f480efdd0 con 0x7f0f58102370 2024-09-10T09:57:25.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:25.316+0000 7f0f5e800640 1 --2- 172.21.15.5:0/2859957599 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f0f400a2c00 0x7f0f400a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:25.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:25.316+0000 7f0f633ca740 1 -- 172.21.15.5:0/3863349476 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55914f402570 msgr2=0x55914f410e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:57:25.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:25.316+0000 7f0f633ca740 1 --2- 172.21.15.5:0/3863349476 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55914f402570 0x55914f410e00 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f0f4c005fd0 tx=0x7f0f4c0107a0 comp rx=0 tx=0).stop 2024-09-10T09:57:25.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:25.321+0000 7f0f633ca740 1 -- 172.21.15.5:0/3863349476 shutdown_connections 2024-09-10T09:57:25.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:25.321+0000 7f0f633ca740 1 --2- 172.21.15.5:0/3863349476 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55914f411340 0x55914f413730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:25.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:25.321+0000 7f0f633ca740 1 --2- 172.21.15.5:0/3863349476 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55914f402570 0x55914f410e00 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:25.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:25.321+0000 7f0f633ca740 1 --2- 172.21.15.5:0/3863349476 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55914f401c50 0x55914f402030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:25.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:25.321+0000 7f0f633ca740 1 -- 172.21.15.5:0/3863349476 >> 172.21.15.5:0/3863349476 conn(0x55914f3fe2e0 msgr2=0x55914f3fe6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:57:25.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:25.321+0000 7f0f633ca740 1 -- 172.21.15.5:0/3863349476 shutdown_connections 2024-09-10T09:57:25.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:25.321+0000 7f0f5e800640 1 --2- 172.21.15.5:0/2859957599 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f0f400a2c00 0x7f0f400a50c0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f0f54013570 tx=0x7f0f540098a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:57:25.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:25.321+0000 7f0f633ca740 1 -- 172.21.15.5:0/3863349476 wait complete. 2024-09-10T09:57:25.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:25.321+0000 7f0f633ca740 1 -- start start 2024-09-10T09:57:25.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:25.321+0000 7f0f633ca740 1 Processor -- start 2024-09-10T09:57:25.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:25.321+0000 7f0f633ca740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55914f401c50 0x55914f40e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:25.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:25.321+0000 7f0f633ca740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55914f402570 0x55914f40e800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:25.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:25.321+0000 7f0f633ca740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55914f411340 0x55914f40ed40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:25.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:25.321+0000 7f0f633ca740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55914f414010 con 0x55914f402570 2024-09-10T09:57:25.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:25.321+0000 7f0f633ca740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55914f3230f0 con 0x55914f411340 2024-09-10T09:57:25.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:25.321+0000 7f0f633ca740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55914f414190 con 0x55914f401c50 2024-09-10T09:57:25.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:25.321+0000 7f0f633ca740 10 client.0 _ll_register_callbacks cb 0x55914f3149e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:57:25.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:25.321+0000 7f0f5d400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55914f402570 0x55914f40e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:25.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:25.321+0000 7f0f5d400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55914f402570 0x55914f40e800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51402/0 (socket says 172.21.15.5:51402) 2024-09-10T09:57:25.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:25.321+0000 7f0f5d400640 1 -- 172.21.15.5:0/2787813501 learned_addr learned my addr 172.21.15.5:0/2787813501 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:57:25.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:25.321+0000 7f0f5d400640 1 -- 172.21.15.5:0/2787813501 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55914f401c50 msgr2=0x55914f40e2c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T09:57:25.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:25.321+0000 7f0f5d400640 1 --2- 172.21.15.5:0/2787813501 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55914f401c50 0x55914f40e2c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:25.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:25.321+0000 7f0f5d400640 1 -- 172.21.15.5:0/2787813501 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55914f411340 msgr2=0x55914f40ed40 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T09:57:25.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:25.321+0000 7f0f5d400640 1 --2- 172.21.15.5:0/2787813501 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55914f411340 0x55914f40ed40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:25.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:25.321+0000 7f0f5d400640 1 -- 172.21.15.5:0/2787813501 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55914f0d93c0 con 0x55914f402570 2024-09-10T09:57:25.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:25.321+0000 7f0f5d400640 1 --2- 172.21.15.5:0/2787813501 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55914f402570 0x55914f40e800 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f0f48031fd0 tx=0x7f0f48000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8bb46885b3a45eb server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:57:25.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:25.321+0000 7f0f3b400640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-10T09:57:25.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:25.321+0000 7f0f3b400640 1 -- 172.21.15.5:0/2787813501 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f48049040 con 0x55914f402570 2024-09-10T09:57:25.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:25.321+0000 7f0f3b400640 1 -- 172.21.15.5:0/2787813501 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f0f480dc070 con 0x55914f402570 2024-09-10T09:57:25.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:25.321+0000 7f0f633ca740 1 -- 172.21.15.5:0/2787813501 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0f4c01cb00 con 0x55914f402570 2024-09-10T09:57:25.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:25.321+0000 7f0f3b400640 1 -- 172.21.15.5:0/2787813501 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f0f480bc8e0 con 0x55914f402570 2024-09-10T09:57:25.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:25.321+0000 7f0f633ca740 10 client.19083 fetch_fsmap learned FSMap version 42 2024-09-10T09:57:25.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:25.321+0000 7f0f633ca740 1 -- 172.21.15.5:0/2787813501 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55914f41fe10 con 0x55914f402570 2024-09-10T09:57:25.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:25.321+0000 7f0f3b400640 1 -- 172.21.15.5:0/2787813501 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0f4c01cb00 con 0x55914f402570 2024-09-10T09:57:25.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:25.321+0000 7f0f3b400640 1 -- 172.21.15.5:0/2787813501 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 42) ==== 8370+0+0 (secure 0 0 0) 0x7f0f480c0150 con 0x55914f402570 2024-09-10T09:57:25.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:25.321+0000 7f0f633ca740 10 client.19083 fetch_fsmap finished waiting for FSMap version 42 2024-09-10T09:57:25.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:25.321+0000 7f0f633ca740 10 client.19083 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T09:57:25.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:25.321+0000 7f0f633ca740 20 client.19083 populate_metadata read hostname 'smithi005' 2024-09-10T09:57:25.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:25.321+0000 7f0f633ca740 1 --2- 172.21.15.5:0/2787813501 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55914f4212d0 0x55914f423730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:25.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:25.321+0000 7f0f633ca740 4 client.19083 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:57:25.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:25.321+0000 7f0f633ca740 1 -- 172.21.15.5:0/2787813501 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55914f375950 con 0x55914f4212d0 2024-09-10T09:57:25.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:25.321+0000 7f0f5de00640 1 --2- 172.21.15.5:0/2787813501 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55914f4212d0 0x55914f423730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:25.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:25.326+0000 7f0f5de00640 1 --2- 172.21.15.5:0/2787813501 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55914f4212d0 0x55914f423730 crc :-1 s=READY pgs=269 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=374619f8d991bcd server_cookie=9b715e4f720a5c78 in_seq=0 out_seq=0 2024-09-10T09:57:25.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:25.326+0000 7f0f3b400640 10 client.19083 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T09:57:25.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:25.326+0000 7f0f3b400640 1 -- 172.21.15.5:0/2787813501 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55914f375950 con 0x55914f4212d0 2024-09-10T09:57:25.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:25.326+0000 7f0f3b400640 10 client.19083 handle_command_reply: tid=0 2024-09-10T09:57:25.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:25.326+0000 7f0f633ca740 1 client.19083 shutdown 2024-09-10T09:57:25.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:25.326+0000 7f0f633ca740 1 -- 172.21.15.5:0/2787813501 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55914f402570 msgr2=0x55914f40e800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:57:25.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:25.326+0000 7f0f633ca740 1 --2- 172.21.15.5:0/2787813501 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55914f402570 0x55914f40e800 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f0f48031fd0 tx=0x7f0f48000bc0 comp rx=0 tx=0).stop 2024-09-10T09:57:25.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:25.326+0000 7f0f633ca740 1 -- 172.21.15.5:0/2787813501 shutdown_connections 2024-09-10T09:57:25.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:25.326+0000 7f0f633ca740 1 --2- 172.21.15.5:0/2787813501 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55914f4212d0 0x55914f423730 crc :-1 s=READY pgs=269 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:25.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:25.326+0000 7f0f633ca740 1 --2- 172.21.15.5:0/2787813501 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55914f411340 0x55914f40ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:25.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:25.326+0000 7f0f633ca740 1 --2- 172.21.15.5:0/2787813501 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55914f402570 0x55914f40e800 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:25.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:25.326+0000 7f0f633ca740 1 --2- 172.21.15.5:0/2787813501 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55914f401c50 0x55914f40e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:25.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:25.326+0000 7f0f633ca740 1 -- 172.21.15.5:0/2787813501 >> 172.21.15.5:0/2787813501 conn(0x55914f3fe2e0 msgr2=0x55914f404040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:57:25.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:25.326+0000 7f0f3b400640 0 client.19083 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T09:57:25.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:25.326+0000 7f0f633ca740 1 -- 172.21.15.5:0/2787813501 shutdown_connections 2024-09-10T09:57:25.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:25.326+0000 7f0f633ca740 1 -- 172.21.15.5:0/2787813501 wait complete. 2024-09-10T09:57:25.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:25.326+0000 7f0f633ca740 20 client.19083 trim_cache size 0 max 16384 2024-09-10T09:57:25.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:25.340+0000 7f0f633ca740 1 Processor -- start 2024-09-10T09:57:25.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:25.340+0000 7f0f633ca740 1 -- start start 2024-09-10T09:57:25.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:25.346+0000 7f0f633ca740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55914f401c50 0x55914f410230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:25.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:25.346+0000 7f0f633ca740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55914f402570 0x55914f429040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:25.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:25.346+0000 7f0f633ca740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55914f411340 0x55914f429580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:25.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:25.346+0000 7f0f633ca740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f0f3c00fc80 con 0x55914f402570 2024-09-10T09:57:25.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:25.346+0000 7f0f633ca740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f0f3c0117c0 con 0x55914f401c50 2024-09-10T09:57:25.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:25.346+0000 7f0f633ca740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f0f3c010350 con 0x55914f411340 2024-09-10T09:57:25.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:25.346+0000 7f0f5d400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55914f402570 0x55914f429040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:25.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:25.346+0000 7f0f5de00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55914f401c50 0x55914f410230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:25.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:25.346+0000 7f0f5d400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55914f402570 0x55914f429040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51414/0 (socket says 172.21.15.5:51414) 2024-09-10T09:57:25.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:25.346+0000 7f0f5d400640 1 -- 172.21.15.5:0/234369187 learned_addr learned my addr 172.21.15.5:0/234369187 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:57:25.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:25.346+0000 7f0f5de00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55914f401c50 0x55914f410230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:38280/0 (socket says 172.21.15.5:38280) 2024-09-10T09:57:25.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:25.346+0000 7f0f5e800640 1 --2- 172.21.15.5:0/234369187 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55914f411340 0x55914f429580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:25.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:25.346+0000 7f0f5de00640 1 -- 172.21.15.5:0/234369187 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55914f411340 msgr2=0x55914f429580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:57:25.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:25.346+0000 7f0f5de00640 1 --2- 172.21.15.5:0/234369187 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55914f411340 0x55914f429580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:25.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:25.346+0000 7f0f5de00640 1 -- 172.21.15.5:0/234369187 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55914f402570 msgr2=0x55914f429040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:57:25.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:25.346+0000 7f0f5de00640 1 --2- 172.21.15.5:0/234369187 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55914f402570 0x55914f429040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:25.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:25.346+0000 7f0f5de00640 1 -- 172.21.15.5:0/234369187 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55914f4023a0 con 0x55914f401c50 2024-09-10T09:57:25.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:25.346+0000 7f0f5d400640 1 --2- 172.21.15.5:0/234369187 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55914f402570 0x55914f429040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:57:25.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:25.352+0000 7f0f5de00640 1 --2- 172.21.15.5:0/234369187 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55914f401c50 0x55914f410230 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f0f4c000c00 tx=0x7f0f4c010fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9b5fdcce77b5ea1c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:57:25.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:25.352+0000 7f0f46600640 1 -- 172.21.15.5:0/234369187 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f4c005240 con 0x55914f401c50 2024-09-10T09:57:25.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:25.352+0000 7f0f46600640 1 -- 172.21.15.5:0/234369187 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0f4c01c960 con 0x55914f401c50 2024-09-10T09:57:25.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:25.352+0000 7f0f46600640 1 -- 172.21.15.5:0/234369187 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f4c0048e0 con 0x55914f401c50 2024-09-10T09:57:25.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:25.352+0000 7f0f633ca740 1 -- 172.21.15.5:0/234369187 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55914f401c50 msgr2=0x55914f410230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:57:25.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:25.352+0000 7f0f633ca740 1 --2- 172.21.15.5:0/234369187 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55914f401c50 0x55914f410230 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f0f4c000c00 tx=0x7f0f4c010fd0 comp rx=0 tx=0).stop 2024-09-10T09:57:25.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:25.352+0000 7f0f633ca740 1 -- 172.21.15.5:0/234369187 shutdown_connections 2024-09-10T09:57:25.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:25.352+0000 7f0f633ca740 1 --2- 172.21.15.5:0/234369187 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55914f411340 0x55914f429580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:25.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:25.352+0000 7f0f633ca740 1 --2- 172.21.15.5:0/234369187 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55914f402570 0x55914f429040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:25.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:25.352+0000 7f0f633ca740 1 --2- 172.21.15.5:0/234369187 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55914f401c50 0x55914f410230 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:25.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:25.352+0000 7f0f633ca740 1 -- 172.21.15.5:0/234369187 >> 172.21.15.5:0/234369187 conn(0x55914f3fe2e0 msgr2=0x55914f40fb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:57:25.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:25.352+0000 7f0f633ca740 1 -- 172.21.15.5:0/234369187 shutdown_connections 2024-09-10T09:57:25.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:25.352+0000 7f0f633ca740 1 -- 172.21.15.5:0/234369187 wait complete. 2024-09-10T09:57:25.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:25.352+0000 7f0f633ca740 1 -- start start 2024-09-10T09:57:25.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:25.352+0000 7f0f633ca740 1 Processor -- start 2024-09-10T09:57:25.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:25.352+0000 7f0f633ca740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55914f401c50 0x55914f419910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:25.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:25.352+0000 7f0f633ca740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55914f402570 0x55914f40ade0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:25.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:25.352+0000 7f0f633ca740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55914f411340 0x55914f40b320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:25.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:25.352+0000 7f0f633ca740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f0f3c0110f0 con 0x55914f401c50 2024-09-10T09:57:25.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:25.352+0000 7f0f633ca740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f0f3c010a20 con 0x55914f411340 2024-09-10T09:57:25.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:25.352+0000 7f0f633ca740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f0f3c00dc60 con 0x55914f402570 2024-09-10T09:57:25.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:25.352+0000 7f0f633ca740 10 client.0 _ll_register_callbacks cb 0x7f0f480b8480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:57:25.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:25.352+0000 7f0f5de00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55914f401c50 0x55914f419910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:25.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:25.352+0000 7f0f5de00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55914f401c50 0x55914f419910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51420/0 (socket says 172.21.15.5:51420) 2024-09-10T09:57:25.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:25.352+0000 7f0f5de00640 1 -- 172.21.15.5:0/3220516317 learned_addr learned my addr 172.21.15.5:0/3220516317 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:57:25.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:25.352+0000 7f0f5d400640 1 --2- 172.21.15.5:0/3220516317 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55914f402570 0x55914f40ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:25.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:25.352+0000 7f0f5e800640 1 --2- 172.21.15.5:0/3220516317 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55914f411340 0x55914f40b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:25.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:25.352+0000 7f0f5de00640 1 -- 172.21.15.5:0/3220516317 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55914f402570 msgr2=0x55914f40ade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:57:25.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:25.352+0000 7f0f5de00640 1 --2- 172.21.15.5:0/3220516317 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55914f402570 0x55914f40ade0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:25.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:25.352+0000 7f0f5de00640 1 -- 172.21.15.5:0/3220516317 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55914f411340 msgr2=0x55914f40b320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:57:25.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:25.352+0000 7f0f5de00640 1 --2- 172.21.15.5:0/3220516317 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55914f411340 0x55914f40b320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:25.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:25.352+0000 7f0f5de00640 1 -- 172.21.15.5:0/3220516317 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55914f40b890 con 0x55914f401c50 2024-09-10T09:57:25.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:25.352+0000 7f0f5d400640 1 --2- 172.21.15.5:0/3220516317 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55914f402570 0x55914f40ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:57:25.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:25.352+0000 7f0f5de00640 1 --2- 172.21.15.5:0/3220516317 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55914f401c50 0x55914f419910 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f0f4c0055a0 tx=0x7f0f4c0058e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be0a26de20d8ba2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:57:25.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:25.352+0000 7f0f3b400640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-10T09:57:25.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:25.352+0000 7f0f3b400640 1 -- 172.21.15.5:0/3220516317 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f4c035070 con 0x55914f401c50 2024-09-10T09:57:25.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:25.352+0000 7f0f3b400640 1 -- 172.21.15.5:0/3220516317 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f0f4c01c690 con 0x55914f401c50 2024-09-10T09:57:25.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:25.352+0000 7f0f633ca740 1 -- 172.21.15.5:0/3220516317 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0f4c01c960 con 0x55914f401c50 2024-09-10T09:57:25.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:25.352+0000 7f0f3b400640 1 -- 172.21.15.5:0/3220516317 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f0f4c00bd20 con 0x55914f401c50 2024-09-10T09:57:25.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:25.352+0000 7f0f633ca740 10 client.19089 fetch_fsmap learned FSMap version 42 2024-09-10T09:57:25.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:25.352+0000 7f0f633ca740 1 -- 172.21.15.5:0/3220516317 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55914f48bdc0 con 0x55914f401c50 2024-09-10T09:57:25.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:25.352+0000 7f0f3b400640 1 -- 172.21.15.5:0/3220516317 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0f4c01c960 con 0x55914f401c50 2024-09-10T09:57:25.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:25.352+0000 7f0f3b400640 1 -- 172.21.15.5:0/3220516317 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 42) ==== 8370+0+0 (secure 0 0 0) 0x7f0f4c003b50 con 0x55914f401c50 2024-09-10T09:57:25.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:25.352+0000 7f0f633ca740 10 client.19089 fetch_fsmap finished waiting for FSMap version 42 2024-09-10T09:57:25.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:25.352+0000 7f0f633ca740 10 client.19089 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T09:57:25.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:25.352+0000 7f0f633ca740 20 client.19089 populate_metadata read hostname 'smithi005' 2024-09-10T09:57:25.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:25.352+0000 7f0f633ca740 1 --2- 172.21.15.5:0/3220516317 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55914f48c280 0x55914f48e670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:25.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:25.352+0000 7f0f633ca740 4 client.19089 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T09:57:25.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:25.352+0000 7f0f633ca740 1 -- 172.21.15.5:0/3220516317 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55914f36fe60 con 0x55914f48c280 2024-09-10T09:57:25.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:25.352+0000 7f0f5d400640 1 --2- 172.21.15.5:0/3220516317 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55914f48c280 0x55914f48e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:25.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:25.358+0000 7f0f5d400640 1 --2- 172.21.15.5:0/3220516317 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55914f48c280 0x55914f48e670 crc :-1 s=READY pgs=270 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dd381b5d283c34ef server_cookie=b141cd1168f7fe92 in_seq=0 out_seq=0 2024-09-10T09:57:25.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:25.358+0000 7f0f3b400640 10 client.19089 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T09:57:25.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:25.358+0000 7f0f3b400640 1 -- 172.21.15.5:0/3220516317 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55914f36fe60 con 0x55914f48c280 2024-09-10T09:57:25.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:25.358+0000 7f0f3b400640 10 client.19089 handle_command_reply: tid=0 2024-09-10T09:57:25.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:25.358+0000 7f0f633ca740 1 client.19089 shutdown 2024-09-10T09:57:25.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:25.358+0000 7f0f633ca740 1 -- 172.21.15.5:0/3220516317 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55914f401c50 msgr2=0x55914f419910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:57:25.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:25.358+0000 7f0f633ca740 1 --2- 172.21.15.5:0/3220516317 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55914f401c50 0x55914f419910 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f0f4c0055a0 tx=0x7f0f4c0058e0 comp rx=0 tx=0).stop 2024-09-10T09:57:25.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:25.358+0000 7f0f633ca740 1 -- 172.21.15.5:0/3220516317 shutdown_connections 2024-09-10T09:57:25.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:25.358+0000 7f0f633ca740 1 --2- 172.21.15.5:0/3220516317 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55914f48c280 0x55914f48e670 crc :-1 s=READY pgs=270 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:25.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:25.358+0000 7f0f633ca740 1 --2- 172.21.15.5:0/3220516317 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55914f411340 0x55914f40b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:25.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:25.358+0000 7f0f633ca740 1 --2- 172.21.15.5:0/3220516317 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55914f402570 0x55914f40ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:25.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:25.358+0000 7f0f633ca740 1 --2- 172.21.15.5:0/3220516317 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55914f401c50 0x55914f419910 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:25.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:25.358+0000 7f0f633ca740 1 -- 172.21.15.5:0/3220516317 >> 172.21.15.5:0/3220516317 conn(0x55914f3fe2e0 msgr2=0x55914f403fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:57:25.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:25.358+0000 7f0f3b400640 0 client.19089 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T09:57:25.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:25.358+0000 7f0f633ca740 1 -- 172.21.15.5:0/3220516317 shutdown_connections 2024-09-10T09:57:25.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:25.358+0000 7f0f633ca740 1 -- 172.21.15.5:0/3220516317 wait complete. 2024-09-10T09:57:25.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:25.358+0000 7f0f633ca740 20 client.19089 trim_cache size 0 max 16384 2024-09-10T09:57:25.363 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-10T09:57:25.363 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-10T09:57:25.363 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2024-09-10T09:57:25.363 INFO:teuthology.orchestra.run.smithi005.stdout: "27348b77-ddea-4e99-9b6b-686849ae2c0d": { 2024-09-10T09:57:25.363 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2024-09-10T09:57:25.363 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "27348b77-ddea-4e99-9b6b-686849ae2c0d", 2024-09-10T09:57:25.363 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2024-09-10T09:57:25.363 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T09:57:25.363 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T09:57:25.363 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-10T09:57:25.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:25.358+0000 7f0f5f200640 1 -- 172.21.15.5:0/2859957599 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f0f400a2c00 msgr2=0x7f0f400a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:57:25.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:25.358+0000 7f0f5f200640 1 --2- 172.21.15.5:0/2859957599 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f0f400a2c00 0x7f0f400a50c0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f0f54013570 tx=0x7f0f540098a0 comp rx=0 tx=0).stop 2024-09-10T09:57:25.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:25.358+0000 7f0f5f200640 1 -- 172.21.15.5:0/2859957599 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0f58102370 msgr2=0x7f0f581987e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:57:25.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:25.358+0000 7f0f5f200640 1 --2- 172.21.15.5:0/2859957599 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0f58102370 0x7f0f581987e0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f0f4802a940 tx=0x7f0f48003b90 comp rx=0 tx=0).stop 2024-09-10T09:57:25.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:25.358+0000 7f0f5f200640 1 -- 172.21.15.5:0/2859957599 shutdown_connections 2024-09-10T09:57:25.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:25.358+0000 7f0f5f200640 1 --2- 172.21.15.5:0/2859957599 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f0f400a2c00 0x7f0f400a50c0 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:25.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:25.358+0000 7f0f5f200640 1 --2- 172.21.15.5:0/2859957599 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0f581079d0 0x7f0f5819cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:25.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:25.358+0000 7f0f5f200640 1 --2- 172.21.15.5:0/2859957599 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0f58102370 0x7f0f581987e0 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:25.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:25.358+0000 7f0f5f200640 1 --2- 172.21.15.5:0/2859957599 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0f581019d0 0x7f0f581982a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:25.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:25.358+0000 7f0f5f200640 1 -- 172.21.15.5:0/2859957599 >> 172.21.15.5:0/2859957599 conn(0x7f0f580fd710 msgr2=0x7f0f580fdea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:57:25.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:25.358+0000 7f0f5f200640 1 -- 172.21.15.5:0/2859957599 shutdown_connections 2024-09-10T09:57:25.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:25.358+0000 7f0f5f200640 1 -- 172.21.15.5:0/2859957599 wait complete. 2024-09-10T09:57:25.436 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:57:25 smithi005 ceph-mon[30350]: pgmap v965: 129 pgs: 129 active+clean; 616 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 419 KiB/s rd, 361 KiB/s wr, 158 op/s 2024-09-10T09:57:25.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:57:25 smithi159 ceph-mon[35513]: pgmap v965: 129 pgs: 129 active+clean; 616 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 419 KiB/s rd, 361 KiB/s wr, 158 op/s 2024-09-10T09:57:25.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:57:25 smithi102 ceph-mon[35442]: pgmap v965: 129 pgs: 129 active+clean; 616 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 419 KiB/s rd, 361 KiB/s wr, 158 op/s 2024-09-10T09:57:25.990 INFO:tasks.workunit.client.0.smithi005.stdout: 10 87534 14.19 MB/sec execute 184 sec latency 27361.252 ms 2024-09-10T09:57:26.259 INFO:tasks.cephfs.filesystem:scrub status for tag:27348b77-ddea-4e99-9b6b-686849ae2c0d - {'path': '/', 'tag': '27348b77-ddea-4e99-9b6b-686849ae2c0d', 'options': 'recursive,force'} 2024-09-10T09:57:26.990 INFO:tasks.workunit.client.0.smithi005.stdout: 10 87534 14.11 MB/sec execute 185 sec latency 28361.302 ms 2024-09-10T09:57:27.452 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:57:27 smithi005 ceph-mon[30350]: pgmap v966: 129 pgs: 129 active+clean; 616 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 394 KiB/s rd, 334 KiB/s wr, 149 op/s 2024-09-10T09:57:27.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:57:27 smithi159 ceph-mon[35513]: pgmap v966: 129 pgs: 129 active+clean; 616 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 394 KiB/s rd, 334 KiB/s wr, 149 op/s 2024-09-10T09:57:27.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:57:27 smithi102 ceph-mon[35442]: pgmap v966: 129 pgs: 129 active+clean; 616 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 394 KiB/s rd, 334 KiB/s wr, 149 op/s 2024-09-10T09:57:27.990 INFO:tasks.workunit.client.0.smithi005.stdout: 10 87534 14.03 MB/sec execute 186 sec latency 29361.368 ms 2024-09-10T09:57:28.991 INFO:tasks.workunit.client.0.smithi005.stdout: 10 87534 13.96 MB/sec execute 187 sec latency 30361.446 ms 2024-09-10T09:57:29.447 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:57:29 smithi005 ceph-mon[30350]: pgmap v967: 129 pgs: 129 active+clean; 616 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 295 KiB/s rd, 84 KiB/s wr, 116 op/s 2024-09-10T09:57:29.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:57:29 smithi159 ceph-mon[35513]: pgmap v967: 129 pgs: 129 active+clean; 616 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 295 KiB/s rd, 84 KiB/s wr, 116 op/s 2024-09-10T09:57:29.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:57:29 smithi102 ceph-mon[35442]: pgmap v967: 129 pgs: 129 active+clean; 616 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 295 KiB/s rd, 84 KiB/s wr, 116 op/s 2024-09-10T09:57:29.990 INFO:tasks.workunit.client.0.smithi005.stdout: 10 87534 13.88 MB/sec execute 188 sec latency 31361.494 ms 2024-09-10T09:57:30.103 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 10 2024-09-10T09:57:30.103 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T09:57:30.576 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:57:30.991 INFO:tasks.workunit.client.0.smithi005.stdout: 10 88140 13.99 MB/sec execute 189 sec latency 31408.779 ms 2024-09-10T09:57:31.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:31.238+0000 7fe1d3000640 1 -- 172.21.15.5:0/2139881583 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe1cc103e30 msgr2=0x7fe1cc104210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:57:31.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:31.238+0000 7fe1d3000640 1 --2- 172.21.15.5:0/2139881583 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe1cc103e30 0x7fe1cc104210 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fe1c0009900 tx=0x7fe1c002a410 comp rx=0 tx=0).stop 2024-09-10T09:57:31.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:31.238+0000 7fe1d3000640 1 -- 172.21.15.5:0/2139881583 shutdown_connections 2024-09-10T09:57:31.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:31.238+0000 7fe1d3000640 1 --2- 172.21.15.5:0/2139881583 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe1cc108f60 0x7fe1cc110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:31.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:31.238+0000 7fe1d3000640 1 --2- 172.21.15.5:0/2139881583 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe1cc1047e0 0x7fe1cc108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:31.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:31.238+0000 7fe1d3000640 1 --2- 172.21.15.5:0/2139881583 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe1cc103e30 0x7fe1cc104210 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:31.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:31.238+0000 7fe1d3000640 1 -- 172.21.15.5:0/2139881583 >> 172.21.15.5:0/2139881583 conn(0x7fe1cc0ff910 msgr2=0x7fe1cc101d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:57:31.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:31.238+0000 7fe1d3000640 1 -- 172.21.15.5:0/2139881583 shutdown_connections 2024-09-10T09:57:31.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:31.238+0000 7fe1d3000640 1 -- 172.21.15.5:0/2139881583 wait complete. 2024-09-10T09:57:31.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:31.239+0000 7fe1d3000640 1 Processor -- start 2024-09-10T09:57:31.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:31.239+0000 7fe1d3000640 1 -- start start 2024-09-10T09:57:31.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:31.240+0000 7fe1d3000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe1cc103e30 0x7fe1cc19a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:31.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:31.240+0000 7fe1d3000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe1cc1047e0 0x7fe1cc19a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:31.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:31.240+0000 7fe1d3000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe1cc108f60 0x7fe1cc19ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:31.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:31.240+0000 7fe1d3000640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe1cc113fd0 con 0x7fe1cc108f60 2024-09-10T09:57:31.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:31.240+0000 7fe1d3000640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fe1cc113e50 con 0x7fe1cc103e30 2024-09-10T09:57:31.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:31.240+0000 7fe1d3000640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fe1cc114150 con 0x7fe1cc1047e0 2024-09-10T09:57:31.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:31.240+0000 7fe1d1200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe1cc1047e0 0x7fe1cc19a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:31.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:31.240+0000 7fe1d1200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe1cc1047e0 0x7fe1cc19a9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:40456/0 (socket says 172.21.15.5:40456) 2024-09-10T09:57:31.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:31.240+0000 7fe1d1200640 1 -- 172.21.15.5:0/3510531965 learned_addr learned my addr 172.21.15.5:0/3510531965 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:57:31.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:31.240+0000 7fe1d2600640 1 --2- 172.21.15.5:0/3510531965 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe1cc108f60 0x7fe1cc19ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:31.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:31.240+0000 7fe1d1c00640 1 --2- 172.21.15.5:0/3510531965 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe1cc103e30 0x7fe1cc19a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:31.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:31.241+0000 7fe1d1200640 1 -- 172.21.15.5:0/3510531965 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe1cc103e30 msgr2=0x7fe1cc19a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:57:31.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:31.241+0000 7fe1d1200640 1 --2- 172.21.15.5:0/3510531965 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe1cc103e30 0x7fe1cc19a470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:31.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:31.241+0000 7fe1d1200640 1 -- 172.21.15.5:0/3510531965 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe1cc108f60 msgr2=0x7fe1cc19ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:57:31.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:31.241+0000 7fe1d1200640 1 --2- 172.21.15.5:0/3510531965 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe1cc108f60 0x7fe1cc19ed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:31.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:31.241+0000 7fe1d1200640 1 -- 172.21.15.5:0/3510531965 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1cc19f420 con 0x7fe1cc1047e0 2024-09-10T09:57:31.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:31.242+0000 7fe1d1200640 1 --2- 172.21.15.5:0/3510531965 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe1cc1047e0 0x7fe1cc19a9b0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fe1b800e9f0 tx=0x7fe1b800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:57:31.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:31.242+0000 7fe1ca000640 1 -- 172.21.15.5:0/3510531965 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1b800cd80 con 0x7fe1cc1047e0 2024-09-10T09:57:31.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:31.242+0000 7fe1ca000640 1 -- 172.21.15.5:0/3510531965 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe1b8010910 con 0x7fe1cc1047e0 2024-09-10T09:57:31.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:31.242+0000 7fe1ca000640 1 -- 172.21.15.5:0/3510531965 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1b8018990 con 0x7fe1cc1047e0 2024-09-10T09:57:31.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:31.243+0000 7fe1d3000640 1 -- 172.21.15.5:0/3510531965 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1cc19f710 con 0x7fe1cc1047e0 2024-09-10T09:57:31.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:31.243+0000 7fe1d3000640 1 -- 172.21.15.5:0/3510531965 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1cc076630 con 0x7fe1cc1047e0 2024-09-10T09:57:31.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:31.245+0000 7fe1d3000640 1 -- 172.21.15.5:0/3510531965 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe1cc105310 con 0x7fe1cc1047e0 2024-09-10T09:57:31.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:31.248+0000 7fe1ca000640 1 -- 172.21.15.5:0/3510531965 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7fe1b8010ac0 con 0x7fe1cc1047e0 2024-09-10T09:57:31.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:31.249+0000 7fe1ca000640 1 --2- 172.21.15.5:0/3510531965 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fe1b40a2b80 0x7fe1b40a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:31.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:31.249+0000 7fe1ca000640 1 -- 172.21.15.5:0/3510531965 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fe1b80cfad0 con 0x7fe1cc1047e0 2024-09-10T09:57:31.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:31.250+0000 7fe1d1c00640 1 --2- 172.21.15.5:0/3510531965 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fe1b40a2b80 0x7fe1b40a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:31.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:31.250+0000 7fe1d1c00640 1 --2- 172.21.15.5:0/3510531965 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fe1b40a2b80 0x7fe1b40a5040 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fe1c0000c00 tx=0x7fe1c0005b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:57:31.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:31.253+0000 7fe1ca000640 1 -- 172.21.15.5:0/3510531965 <== mon.1 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe1b809c5e0 con 0x7fe1cc1047e0 2024-09-10T09:57:31.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:31.621+0000 7fe1d3000640 1 -- 172.21.15.5:0/3510531965 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe1cc104210 con 0x7fe1cc1047e0 2024-09-10T09:57:31.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:31.623+0000 7fe1ca000640 1 -- 172.21.15.5:0/3510531965 <== mon.1 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 42 v42) ==== 94+0+11969 (secure 0 0 0) 0x7fe1cc104210 con 0x7fe1cc1047e0 2024-09-10T09:57:31.624 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-10T09:57:31.625 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":42,"btime":"2024-09-10T09:55:06:670916+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14454,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/2970594311","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2970594311},{"type":"v1","addr":"172.21.15.5:6835","nonce":2970594311}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14514,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6839/3616502174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":3616502174},{"type":"v1","addr":"172.21.15.5:6839","nonce":3616502174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":42,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T09:35:29.626287+0000","modified":"2024-09-10T09:55:06.419829+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24542,"mds_1":24535,"mds_2":24511,"mds_3":24506,"mds_4":24443},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14520":{"gid":14520,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/2073599842","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":2073599842},{"type":"v1","addr":"172.21.15.5:6841","nonce":2073599842}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"k","rank":4,"incarnation":26,"state":"up:active","state_seq":24,"addr":"172.21.15.102:6833/2457007427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":2457007427},{"type":"v1","addr":"172.21.15.102:6833","nonce":2457007427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/356445159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":356445159},{"type":"v1","addr":"172.21.15.159:6833","nonce":356445159}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6837/2894314507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":2894314507},{"type":"v1","addr":"172.21.15.5:6837","nonce":2894314507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.102:6835/1647549111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":1647549111},{"type":"v1","addr":"172.21.15.102:6835","nonce":1647549111}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6835/952657242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":952657242},{"type":"v1","addr":"172.21.15.159:6835","nonce":952657242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":19,"addr":"172.21.15.102:6837/2689776764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":2689776764},{"type":"v1","addr":"172.21.15.102:6837","nonce":2689776764}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.159:6837/620627639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":620627639},{"type":"v1","addr":"172.21.15.159:6837","nonce":620627639}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24535":{"gid":24535,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":18,"addr":"172.21.15.159:6839/2073739599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":2073739599},{"type":"v1","addr":"172.21.15.159:6839","nonce":2073739599}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24542":{"gid":24542,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.102:6839/335409749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6838","nonce":335409749},{"type":"v1","addr":"172.21.15.102:6839","nonce":335409749}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24542,"qdb_cluster":[24542,24535,24511,24506,24443]},"id":1}]} 2024-09-10T09:57:31.625 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 42 2024-09-10T09:57:31.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:31.627+0000 7fe1d3000640 1 -- 172.21.15.5:0/3510531965 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fe1b40a2b80 msgr2=0x7fe1b40a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:57:31.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:31.627+0000 7fe1d3000640 1 --2- 172.21.15.5:0/3510531965 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fe1b40a2b80 0x7fe1b40a5040 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fe1c0000c00 tx=0x7fe1c0005b50 comp rx=0 tx=0).stop 2024-09-10T09:57:31.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:31.627+0000 7fe1d3000640 1 -- 172.21.15.5:0/3510531965 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe1cc1047e0 msgr2=0x7fe1cc19a9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:57:31.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:31.627+0000 7fe1d3000640 1 --2- 172.21.15.5:0/3510531965 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe1cc1047e0 0x7fe1cc19a9b0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fe1b800e9f0 tx=0x7fe1b800eeb0 comp rx=0 tx=0).stop 2024-09-10T09:57:31.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:31.628+0000 7fe1d3000640 1 -- 172.21.15.5:0/3510531965 shutdown_connections 2024-09-10T09:57:31.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:31.628+0000 7fe1d3000640 1 --2- 172.21.15.5:0/3510531965 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fe1b40a2b80 0x7fe1b40a5040 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:31.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:31.628+0000 7fe1d3000640 1 --2- 172.21.15.5:0/3510531965 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe1cc108f60 0x7fe1cc19ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:31.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:31.628+0000 7fe1d3000640 1 --2- 172.21.15.5:0/3510531965 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe1cc1047e0 0x7fe1cc19a9b0 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:31.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:31.628+0000 7fe1d3000640 1 --2- 172.21.15.5:0/3510531965 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe1cc103e30 0x7fe1cc19a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:31.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:31.628+0000 7fe1d3000640 1 -- 172.21.15.5:0/3510531965 >> 172.21.15.5:0/3510531965 conn(0x7fe1cc0ff910 msgr2=0x7fe1cc100c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:57:31.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:31.628+0000 7fe1d3000640 1 -- 172.21.15.5:0/3510531965 shutdown_connections 2024-09-10T09:57:31.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:31.628+0000 7fe1d3000640 1 -- 172.21.15.5:0/3510531965 wait complete. 2024-09-10T09:57:31.686 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:57:31 smithi102 ceph-mon[35442]: pgmap v968: 129 pgs: 129 active+clean; 616 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 266 KiB/s rd, 92 KiB/s wr, 79 op/s 2024-09-10T09:57:31.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:57:31 smithi005 ceph-mon[30350]: pgmap v968: 129 pgs: 129 active+clean; 616 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 266 KiB/s rd, 92 KiB/s wr, 79 op/s 2024-09-10T09:57:31.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:57:31 smithi159 ceph-mon[35513]: pgmap v968: 129 pgs: 129 active+clean; 616 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 266 KiB/s rd, 92 KiB/s wr, 79 op/s 2024-09-10T09:57:31.991 INFO:tasks.workunit.client.0.smithi005.stdout: 10 88898 14.10 MB/sec execute 190 sec latency 57.963 ms 2024-09-10T09:57:32.597 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24542 quiesce db / --timeout 90 --expiration 130 --await-for 45.0' 2024-09-10T09:57:32.597 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24542 quiesce db / --timeout 90 --expiration 130 --await-for 45.0 2024-09-10T09:57:32.666 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:57:32 smithi005 ceph-mon[30350]: from='client.? 172.21.15.5:0/3510531965' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:57:32.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:57:32 smithi159 ceph-mon[35513]: from='client.? 172.21.15.5:0/3510531965' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:57:32.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:57:32 smithi102 ceph-mon[35442]: from='client.? 172.21.15.5:0/3510531965' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:57:32.991 INFO:tasks.workunit.client.0.smithi005.stdout: 10 89808 14.33 MB/sec execute 191 sec latency 100.575 ms 2024-09-10T09:57:33.152 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:57:33.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:57:33 smithi005 ceph-mon[30350]: pgmap v969: 129 pgs: 129 active+clean; 616 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 231 KiB/s rd, 122 KiB/s wr, 30 op/s 2024-09-10T09:57:33.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:57:33 smithi159 ceph-mon[35513]: pgmap v969: 129 pgs: 129 active+clean; 616 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 231 KiB/s rd, 122 KiB/s wr, 30 op/s 2024-09-10T09:57:33.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:57:33 smithi102 ceph-mon[35442]: pgmap v969: 129 pgs: 129 active+clean; 616 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 231 KiB/s rd, 122 KiB/s wr, 30 op/s 2024-09-10T09:57:33.992 INFO:tasks.workunit.client.0.smithi005.stdout: 10 90324 14.42 MB/sec execute 192 sec latency 104.551 ms 2024-09-10T09:57:34.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:34.054+0000 7f7cef600640 1 -- 172.21.15.5:0/2443106893 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7ce810d9b0 msgr2=0x7f7ce8115510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:57:34.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:34.054+0000 7f7cef600640 1 --2- 172.21.15.5:0/2443106893 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7ce810d9b0 0x7f7ce8115510 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f7ce0009cd0 tx=0x7f7ce002a8f0 comp rx=0 tx=0).stop 2024-09-10T09:57:34.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:34.055+0000 7f7cef600640 1 -- 172.21.15.5:0/2443106893 shutdown_connections 2024-09-10T09:57:34.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:34.055+0000 7f7cef600640 1 --2- 172.21.15.5:0/2443106893 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7ce810d9b0 0x7f7ce8115510 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:34.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:34.055+0000 7f7cef600640 1 --2- 172.21.15.5:0/2443106893 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7ce810ef30 0x7f7ce810d2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:34.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:34.055+0000 7f7cef600640 1 --2- 172.21.15.5:0/2443106893 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7ce810e580 0x7f7ce810e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:34.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:34.055+0000 7f7cef600640 1 -- 172.21.15.5:0/2443106893 >> 172.21.15.5:0/2443106893 conn(0x7f7ce806e140 msgr2=0x7f7ce806e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:57:34.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:34.055+0000 7f7cef600640 1 -- 172.21.15.5:0/2443106893 shutdown_connections 2024-09-10T09:57:34.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:34.056+0000 7f7cef600640 1 -- 172.21.15.5:0/2443106893 wait complete. 2024-09-10T09:57:34.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:34.060+0000 7f7cef600640 1 Processor -- start 2024-09-10T09:57:34.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:34.060+0000 7f7cef600640 1 -- start start 2024-09-10T09:57:34.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:34.061+0000 7f7cef600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7ce810d9b0 0x7f7ce810fd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:34.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:34.061+0000 7f7cef600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7ce810e580 0x7f7ce8110250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:34.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:34.061+0000 7f7cef600640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7ce810ef30 0x7f7ce81acf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:34.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:34.062+0000 7f7cecc00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7ce810d9b0 0x7f7ce810fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:34.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:34.062+0000 7f7cecc00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7ce810d9b0 0x7f7ce810fd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:49766/0 (socket says 172.21.15.5:49766) 2024-09-10T09:57:34.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:34.062+0000 7f7ced600640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7ce810ef30 0x7f7ce81acf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:34.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:34.062+0000 7f7ce7e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7ce810e580 0x7f7ce8110250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:34.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:34.062+0000 7f7ced600640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7ce810ef30 0x7f7ce81acf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:40478/0 (socket says 172.21.15.5:40478) 2024-09-10T09:57:34.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:34.062+0000 7f7ce7e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7ce810e580 0x7f7ce8110250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37674/0 (socket says 172.21.15.5:37674) 2024-09-10T09:57:34.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:34.062+0000 7f7cef600640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f7ce81185d0 con 0x7f7ce810e580 2024-09-10T09:57:34.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:34.062+0000 7f7cef600640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f7ce8118450 con 0x7f7ce810d9b0 2024-09-10T09:57:34.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:34.062+0000 7f7cef600640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f7ce8118750 con 0x7f7ce810ef30 2024-09-10T09:57:34.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:34.063+0000 7f7cecc00640 1 -- 172.21.15.5:0/3143265502 learned_addr learned my addr 172.21.15.5:0/3143265502 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:57:34.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:34.063+0000 7f7ced600640 1 -- 172.21.15.5:0/3143265502 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7ce810d9b0 msgr2=0x7f7ce810fd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:57:34.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:34.063+0000 7f7ced600640 1 --2- 172.21.15.5:0/3143265502 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7ce810d9b0 0x7f7ce810fd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:34.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:34.063+0000 7f7ced600640 1 -- 172.21.15.5:0/3143265502 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7ce810e580 msgr2=0x7f7ce8110250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:57:34.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:34.063+0000 7f7ced600640 1 --2- 172.21.15.5:0/3143265502 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7ce810e580 0x7f7ce8110250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:34.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:34.063+0000 7f7ced600640 1 -- 172.21.15.5:0/3143265502 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ce81ad630 con 0x7f7ce810ef30 2024-09-10T09:57:34.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:34.064+0000 7f7ced600640 1 --2- 172.21.15.5:0/3143265502 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7ce810ef30 0x7f7ce81acf00 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f7ce0009bf0 tx=0x7f7ce0003f80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:57:34.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:34.064+0000 7f7ce5600640 1 -- 172.21.15.5:0/3143265502 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ce0030dc0 con 0x7f7ce810ef30 2024-09-10T09:57:34.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:34.065+0000 7f7cef600640 1 -- 172.21.15.5:0/3143265502 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ce81ad8c0 con 0x7f7ce810ef30 2024-09-10T09:57:34.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:34.065+0000 7f7cef600640 1 -- 172.21.15.5:0/3143265502 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ce81b3fb0 con 0x7f7ce810ef30 2024-09-10T09:57:34.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:34.065+0000 7f7ce5600640 1 -- 172.21.15.5:0/3143265502 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7ce0040070 con 0x7f7ce810ef30 2024-09-10T09:57:34.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:34.065+0000 7f7ce5600640 1 -- 172.21.15.5:0/3143265502 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ce00459d0 con 0x7f7ce810ef30 2024-09-10T09:57:34.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:34.071+0000 7f7ce5600640 1 -- 172.21.15.5:0/3143265502 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f7ce0045cb0 con 0x7f7ce810ef30 2024-09-10T09:57:34.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:34.072+0000 7f7cf21bd740 1 Processor -- start 2024-09-10T09:57:34.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:34.072+0000 7f7cf21bd740 1 -- start start 2024-09-10T09:57:34.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:34.073+0000 7f7cf21bd740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563e932fe3b0 0x563e932fe790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:34.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:34.073+0000 7f7cf21bd740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563e932fecd0 0x563e93309050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:34.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:34.073+0000 7f7cf21bd740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563e93309590 0x563e9330b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:34.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:34.073+0000 7f7cf21bd740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x563e93264530 con 0x563e93309590 2024-09-10T09:57:34.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:34.073+0000 7f7cf21bd740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x563e9320b860 con 0x563e932fecd0 2024-09-10T09:57:34.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:34.073+0000 7f7cf21bd740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x563e932604f0 con 0x563e932fe3b0 2024-09-10T09:57:34.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:34.073+0000 7f7cecc00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563e932fecd0 0x563e93309050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:34.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:34.073+0000 7f7cecc00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563e932fecd0 0x563e93309050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:49778/0 (socket says 172.21.15.5:49778) 2024-09-10T09:57:34.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:34.073+0000 7f7cecc00640 1 -- 172.21.15.5:0/2076004560 learned_addr learned my addr 172.21.15.5:0/2076004560 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:57:34.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:34.074+0000 7f7cecc00640 1 -- 172.21.15.5:0/2076004560 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563e932fe3b0 msgr2=0x563e932fe790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:57:34.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:34.074+0000 7f7cecc00640 1 --2- 172.21.15.5:0/2076004560 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563e932fe3b0 0x563e932fe790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:34.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:34.074+0000 7f7cecc00640 1 -- 172.21.15.5:0/2076004560 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563e93309590 msgr2=0x563e9330b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:57:34.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:34.074+0000 7f7cecc00640 1 --2- 172.21.15.5:0/2076004560 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563e93309590 0x563e9330b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:34.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:34.074+0000 7f7cecc00640 1 -- 172.21.15.5:0/2076004560 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563e92fd5b90 con 0x563e932fecd0 2024-09-10T09:57:34.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:34.075+0000 7f7ce5600640 1 --2- 172.21.15.5:0/3143265502 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f7cd00a2cc0 0x7f7cd00a5180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:34.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:34.075+0000 7f7ce5600640 1 -- 172.21.15.5:0/3143265502 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f7ce00f0550 con 0x7f7ce810ef30 2024-09-10T09:57:34.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:34.076+0000 7f7cecc00640 1 --2- 172.21.15.5:0/2076004560 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563e932fecd0 0x563e93309050 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f7cdc007890 tx=0x7f7cdc007d50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a329bc0a054c2d72 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:57:34.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:34.076+0000 7f7ce7e00640 1 --2- 172.21.15.5:0/3143265502 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f7cd00a2cc0 0x7f7cd00a5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:34.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:34.076+0000 7f7cd9c00640 1 -- 172.21.15.5:0/2076004560 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cdc00d950 con 0x563e932fecd0 2024-09-10T09:57:34.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:34.077+0000 7f7cd9c00640 1 -- 172.21.15.5:0/2076004560 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7cdc018070 con 0x563e932fecd0 2024-09-10T09:57:34.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:34.077+0000 7f7cd9c00640 1 -- 172.21.15.5:0/2076004560 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cdc0123e0 con 0x563e932fecd0 2024-09-10T09:57:34.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:34.078+0000 7f7cf21bd740 1 -- 172.21.15.5:0/2076004560 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563e932fecd0 msgr2=0x563e93309050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:57:34.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:34.078+0000 7f7cf21bd740 1 --2- 172.21.15.5:0/2076004560 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563e932fecd0 0x563e93309050 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f7cdc007890 tx=0x7f7cdc007d50 comp rx=0 tx=0).stop 2024-09-10T09:57:34.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:34.079+0000 7f7ce7e00640 1 --2- 172.21.15.5:0/3143265502 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f7cd00a2cc0 0x7f7cd00a5180 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x563e932fbcc0 tx=0x7f7cd400a500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:57:34.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:34.079+0000 7f7cf21bd740 1 -- 172.21.15.5:0/2076004560 shutdown_connections 2024-09-10T09:57:34.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:34.079+0000 7f7cf21bd740 1 --2- 172.21.15.5:0/2076004560 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563e93309590 0x563e9330b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:34.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:34.079+0000 7f7cf21bd740 1 --2- 172.21.15.5:0/2076004560 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563e932fecd0 0x563e93309050 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:34.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:34.079+0000 7f7cf21bd740 1 --2- 172.21.15.5:0/2076004560 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563e932fe3b0 0x563e932fe790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:34.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:34.079+0000 7f7cf21bd740 1 -- 172.21.15.5:0/2076004560 >> 172.21.15.5:0/2076004560 conn(0x563e932fa940 msgr2=0x563e932fad20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:57:34.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:34.080+0000 7f7cf21bd740 1 -- 172.21.15.5:0/2076004560 shutdown_connections 2024-09-10T09:57:34.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:34.080+0000 7f7cf21bd740 1 -- 172.21.15.5:0/2076004560 wait complete. 2024-09-10T09:57:34.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:34.081+0000 7f7cf21bd740 1 -- start start 2024-09-10T09:57:34.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:34.082+0000 7f7cf21bd740 1 Processor -- start 2024-09-10T09:57:34.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:34.084+0000 7f7cf21bd740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563e932fe3b0 0x563e93306650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:34.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:34.084+0000 7f7cf21bd740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563e932fecd0 0x563e93306b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:34.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:34.085+0000 7f7cf21bd740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563e93309590 0x563e933070d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:34.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:34.085+0000 7f7ce7e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563e932fecd0 0x563e93306b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:34.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:34.085+0000 7f7cf21bd740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x563e9330bfb0 con 0x563e932fecd0 2024-09-10T09:57:34.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:34.086+0000 7f7cf21bd740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x563e9321ee60 con 0x563e93309590 2024-09-10T09:57:34.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:34.086+0000 7f7cf21bd740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x563e9330c130 con 0x563e932fe3b0 2024-09-10T09:57:34.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:34.086+0000 7f7cf21bd740 10 client.0 _ll_register_callbacks cb 0x563e93223cc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:57:34.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:34.086+0000 7f7ce7e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563e932fecd0 0x563e93306b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37700/0 (socket says 172.21.15.5:37700) 2024-09-10T09:57:34.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:34.086+0000 7f7ce7e00640 1 -- 172.21.15.5:0/977987303 learned_addr learned my addr 172.21.15.5:0/977987303 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:57:34.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:34.088+0000 7f7ce7e00640 1 -- 172.21.15.5:0/977987303 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563e932fe3b0 msgr2=0x563e93306650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:57:34.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:34.085+0000 7f7cecc00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563e932fe3b0 0x563e93306650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:34.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:34.089+0000 7f7ced600640 1 --2- 172.21.15.5:0/977987303 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563e93309590 0x563e933070d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:34.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:34.089+0000 7f7ce7e00640 1 --2- 172.21.15.5:0/977987303 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563e932fe3b0 0x563e93306650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:34.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:34.089+0000 7f7ce7e00640 1 -- 172.21.15.5:0/977987303 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563e93309590 msgr2=0x563e933070d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:57:34.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:34.089+0000 7f7ce7e00640 1 --2- 172.21.15.5:0/977987303 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563e93309590 0x563e933070d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:34.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:34.089+0000 7f7ce7e00640 1 -- 172.21.15.5:0/977987303 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563e9322a990 con 0x563e932fecd0 2024-09-10T09:57:34.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:34.089+0000 7f7cecc00640 1 --2- 172.21.15.5:0/977987303 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563e932fe3b0 0x563e93306650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T09:57:34.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:34.090+0000 7f7ce7e00640 1 --2- 172.21.15.5:0/977987303 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563e932fecd0 0x563e93306b90 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f7cd400a800 tx=0x7f7cd400c040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35975163ec646d2d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:57:34.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:34.090+0000 7f7cce000640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-10T09:57:34.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:34.090+0000 7f7cce000640 1 -- 172.21.15.5:0/977987303 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cd4004b90 con 0x563e932fecd0 2024-09-10T09:57:34.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:34.090+0000 7f7cce000640 1 -- 172.21.15.5:0/977987303 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f7cd4004d30 con 0x563e932fecd0 2024-09-10T09:57:34.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:34.090+0000 7f7cf21bd740 1 -- 172.21.15.5:0/977987303 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7cdc018070 con 0x563e932fecd0 2024-09-10T09:57:34.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:34.091+0000 7f7cce000640 1 -- 172.21.15.5:0/977987303 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f7cd4018470 con 0x563e932fecd0 2024-09-10T09:57:34.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:34.092+0000 7f7cf21bd740 10 client.19101 fetch_fsmap learned FSMap version 42 2024-09-10T09:57:34.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:34.092+0000 7f7cf21bd740 1 -- 172.21.15.5:0/977987303 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x563e93317df0 con 0x563e932fecd0 2024-09-10T09:57:34.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:34.093+0000 7f7cce000640 1 -- 172.21.15.5:0/977987303 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7cdc018070 con 0x563e932fecd0 2024-09-10T09:57:34.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:34.094+0000 7f7cce000640 1 -- 172.21.15.5:0/977987303 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 42) ==== 8370+0+0 (secure 0 0 0) 0x7f7cd402b030 con 0x563e932fecd0 2024-09-10T09:57:34.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:34.094+0000 7f7cf21bd740 10 client.19101 fetch_fsmap finished waiting for FSMap version 42 2024-09-10T09:57:34.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:34.094+0000 7f7cf21bd740 10 client.19101 resolve_mds: validated gid 24542 aka daemon mds.b 2024-09-10T09:57:34.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:34.094+0000 7f7cf21bd740 20 client.19101 populate_metadata read hostname 'smithi005' 2024-09-10T09:57:34.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:34.094+0000 7f7cf21bd740 1 --2- 172.21.15.5:0/977987303 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x563e933192b0 0x563e9331b770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:34.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:34.094+0000 7f7cf21bd740 4 client.19101 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:57:34.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:34.094+0000 7f7cf21bd740 1 -- 172.21.15.5:0/977987303 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563e93207310 con 0x563e933192b0 2024-09-10T09:57:34.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:34.094+0000 7f7cecc00640 1 --2- 172.21.15.5:0/977987303 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x563e933192b0 0x563e9331b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:34.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:34.095+0000 7f7cce000640 10 client.19101 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T09:57:34.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:34.096+0000 7f7cecc00640 1 --2- 172.21.15.5:0/977987303 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x563e933192b0 0x563e9331b770 crc :-1 s=READY pgs=271 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=be3b0ce3b4ad363a server_cookie=e6df7e739a5366d7 in_seq=0 out_seq=0 2024-09-10T09:57:34.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:34.097+0000 7f7cce000640 1 -- 172.21.15.5:0/977987303 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563e93207310 con 0x563e933192b0 2024-09-10T09:57:34.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:34.097+0000 7f7cce000640 10 client.19101 handle_command_reply: tid=0 2024-09-10T09:57:34.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:34.097+0000 7f7cf21bd740 1 client.19101 shutdown 2024-09-10T09:57:34.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:34.098+0000 7f7cf21bd740 1 -- 172.21.15.5:0/977987303 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563e932fecd0 msgr2=0x563e93306b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:57:34.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:34.098+0000 7f7cf21bd740 1 --2- 172.21.15.5:0/977987303 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563e932fecd0 0x563e93306b90 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f7cd400a800 tx=0x7f7cd400c040 comp rx=0 tx=0).stop 2024-09-10T09:57:34.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:34.098+0000 7f7cf21bd740 1 -- 172.21.15.5:0/977987303 shutdown_connections 2024-09-10T09:57:34.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:34.098+0000 7f7cf21bd740 1 --2- 172.21.15.5:0/977987303 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x563e933192b0 0x563e9331b770 crc :-1 s=READY pgs=271 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:34.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:34.098+0000 7f7cf21bd740 1 --2- 172.21.15.5:0/977987303 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563e93309590 0x563e933070d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:34.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:34.098+0000 7f7cf21bd740 1 --2- 172.21.15.5:0/977987303 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563e932fecd0 0x563e93306b90 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:34.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:34.098+0000 7f7cf21bd740 1 --2- 172.21.15.5:0/977987303 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563e932fe3b0 0x563e93306650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:34.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:34.098+0000 7f7cf21bd740 1 -- 172.21.15.5:0/977987303 >> 172.21.15.5:0/977987303 conn(0x563e932fa940 msgr2=0x563e932f9c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:57:34.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:34.098+0000 7f7cce000640 0 client.19101 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T09:57:34.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:34.099+0000 7f7cf21bd740 1 -- 172.21.15.5:0/977987303 shutdown_connections 2024-09-10T09:57:34.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:34.099+0000 7f7cf21bd740 1 -- 172.21.15.5:0/977987303 wait complete. 2024-09-10T09:57:34.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:34.099+0000 7f7cf21bd740 20 client.19101 trim_cache size 0 max 16384 2024-09-10T09:57:34.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:34.136+0000 7f7cf21bd740 1 Processor -- start 2024-09-10T09:57:34.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:34.136+0000 7f7cf21bd740 1 -- start start 2024-09-10T09:57:34.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:34.136+0000 7f7cf21bd740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563e932fe3b0 0x563e932f9a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:34.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:34.136+0000 7f7cf21bd740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563e93309590 0x563e93300e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:34.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:34.136+0000 7f7cf21bd740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563e93321080 0x563e93321500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:34.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:34.137+0000 7f7cf21bd740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f7ce81bded0 con 0x563e93309590 2024-09-10T09:57:34.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:34.137+0000 7f7cf21bd740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f7ce81bfa10 con 0x563e93321080 2024-09-10T09:57:34.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:34.137+0000 7f7cf21bd740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f7ce81be5a0 con 0x563e932fe3b0 2024-09-10T09:57:34.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:34.137+0000 7f7ced600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563e93321080 0x563e93321500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:34.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:34.137+0000 7f7cecc00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563e932fe3b0 0x563e932f9a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:34.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:34.137+0000 7f7ced600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563e93321080 0x563e93321500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:49804/0 (socket says 172.21.15.5:49804) 2024-09-10T09:57:34.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:34.137+0000 7f7ced600640 1 -- 172.21.15.5:0/1352728663 learned_addr learned my addr 172.21.15.5:0/1352728663 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:57:34.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:34.138+0000 7f7ced600640 1 -- 172.21.15.5:0/1352728663 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563e932fe3b0 msgr2=0x563e932f9a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:57:34.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:34.138+0000 7f7ced600640 1 --2- 172.21.15.5:0/1352728663 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563e932fe3b0 0x563e932f9a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:34.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:34.138+0000 7f7ced600640 1 -- 172.21.15.5:0/1352728663 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563e93309590 msgr2=0x563e93300e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:57:34.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:34.138+0000 7f7ced600640 1 --2- 172.21.15.5:0/1352728663 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563e93309590 0x563e93300e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:34.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:34.138+0000 7f7ced600640 1 -- 172.21.15.5:0/1352728663 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563e932fbab0 con 0x563e93321080 2024-09-10T09:57:34.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:34.138+0000 7f7ced600640 1 --2- 172.21.15.5:0/1352728663 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563e93321080 0x563e93321500 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f7ce00ba290 tx=0x7f7ce0045470 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3ace7ff82b2de6f4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:57:34.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:34.138+0000 7f7cd9200640 1 -- 172.21.15.5:0/1352728663 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ce00c1c70 con 0x563e93321080 2024-09-10T09:57:34.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:34.138+0000 7f7cd9200640 1 -- 172.21.15.5:0/1352728663 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7ce0036040 con 0x563e93321080 2024-09-10T09:57:34.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:34.139+0000 7f7cd9200640 1 -- 172.21.15.5:0/1352728663 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ce00a80e0 con 0x563e93321080 2024-09-10T09:57:34.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:34.140+0000 7f7cf21bd740 1 -- 172.21.15.5:0/1352728663 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563e93321080 msgr2=0x563e93321500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:57:34.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:34.140+0000 7f7cf21bd740 1 --2- 172.21.15.5:0/1352728663 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563e93321080 0x563e93321500 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f7ce00ba290 tx=0x7f7ce0045470 comp rx=0 tx=0).stop 2024-09-10T09:57:34.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:34.140+0000 7f7cf21bd740 1 -- 172.21.15.5:0/1352728663 shutdown_connections 2024-09-10T09:57:34.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:34.140+0000 7f7cf21bd740 1 --2- 172.21.15.5:0/1352728663 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563e93321080 0x563e93321500 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:34.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:34.140+0000 7f7cf21bd740 1 --2- 172.21.15.5:0/1352728663 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563e93309590 0x563e93300e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:34.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:34.140+0000 7f7cf21bd740 1 --2- 172.21.15.5:0/1352728663 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563e932fe3b0 0x563e932f9a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:34.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:34.140+0000 7f7cf21bd740 1 -- 172.21.15.5:0/1352728663 >> 172.21.15.5:0/1352728663 conn(0x563e932fa940 msgr2=0x563e933085c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:57:34.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:34.140+0000 7f7cf21bd740 1 -- 172.21.15.5:0/1352728663 shutdown_connections 2024-09-10T09:57:34.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:34.141+0000 7f7cf21bd740 1 -- 172.21.15.5:0/1352728663 wait complete. 2024-09-10T09:57:34.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:34.141+0000 7f7cf21bd740 1 -- start start 2024-09-10T09:57:34.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:34.142+0000 7f7cf21bd740 1 Processor -- start 2024-09-10T09:57:34.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:34.142+0000 7f7cf21bd740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563e932fe3b0 0x563e9330e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:34.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:34.142+0000 7f7cf21bd740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563e93309590 0x563e9330ee30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:34.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:34.143+0000 7f7cf21bd740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563e93321080 0x563e93384060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:34.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:34.143+0000 7f7cf21bd740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f7ce81bf340 con 0x563e93321080 2024-09-10T09:57:34.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:34.143+0000 7f7cf21bd740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f7ce81bec70 con 0x563e932fe3b0 2024-09-10T09:57:34.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:34.143+0000 7f7cf21bd740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f7ce81bbe50 con 0x563e93309590 2024-09-10T09:57:34.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:34.143+0000 7f7cf21bd740 10 client.0 _ll_register_callbacks cb 0x7f7cd400aa10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:57:34.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:34.143+0000 7f7cecc00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563e932fe3b0 0x563e9330e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:34.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:34.143+0000 7f7cecc00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563e932fe3b0 0x563e9330e8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:49814/0 (socket says 172.21.15.5:49814) 2024-09-10T09:57:34.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:34.143+0000 7f7cecc00640 1 -- 172.21.15.5:0/1639085559 learned_addr learned my addr 172.21.15.5:0/1639085559 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:57:34.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:34.144+0000 7f7ced600640 1 --2- 172.21.15.5:0/1639085559 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563e93321080 0x563e93384060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:34.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:34.144+0000 7f7cecc00640 1 -- 172.21.15.5:0/1639085559 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563e93309590 msgr2=0x563e9330ee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:57:34.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:34.144+0000 7f7ce7e00640 1 --2- 172.21.15.5:0/1639085559 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563e93309590 0x563e9330ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:34.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:34.144+0000 7f7cecc00640 1 --2- 172.21.15.5:0/1639085559 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563e93309590 0x563e9330ee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:34.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:34.144+0000 7f7cecc00640 1 -- 172.21.15.5:0/1639085559 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563e93321080 msgr2=0x563e93384060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:57:34.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:34.144+0000 7f7cecc00640 1 --2- 172.21.15.5:0/1639085559 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563e93321080 0x563e93384060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:34.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:34.144+0000 7f7cecc00640 1 -- 172.21.15.5:0/1639085559 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563e933845d0 con 0x563e932fe3b0 2024-09-10T09:57:34.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:34.144+0000 7f7ce7e00640 1 --2- 172.21.15.5:0/1639085559 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563e93309590 0x563e9330ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T09:57:34.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:34.146+0000 7f7cecc00640 1 --2- 172.21.15.5:0/1639085559 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563e932fe3b0 0x563e9330e8f0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x563e932fb7a0 tx=0x7f7cdc013b30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=52440fba1bba14bf server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:57:34.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:34.146+0000 7f7cce000640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-09-10T09:57:34.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:34.147+0000 7f7cce000640 1 -- 172.21.15.5:0/1639085559 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cdc02a060 con 0x563e932fe3b0 2024-09-10T09:57:34.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:34.147+0000 7f7cce000640 1 -- 172.21.15.5:0/1639085559 <== mon.2 v2:172.21.15.102:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f7cdc013820 con 0x563e932fe3b0 2024-09-10T09:57:34.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:34.147+0000 7f7cf21bd740 1 -- 172.21.15.5:0/1639085559 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7ce0036040 con 0x563e932fe3b0 2024-09-10T09:57:34.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:34.148+0000 7f7cce000640 1 -- 172.21.15.5:0/1639085559 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f7cdc031050 con 0x563e932fe3b0 2024-09-10T09:57:34.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:34.148+0000 7f7cf21bd740 10 client.29159 fetch_fsmap learned FSMap version 42 2024-09-10T09:57:34.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:34.148+0000 7f7cf21bd740 1 -- 172.21.15.5:0/1639085559 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x563e933858b0 con 0x563e932fe3b0 2024-09-10T09:57:34.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:34.149+0000 7f7cce000640 1 -- 172.21.15.5:0/1639085559 <== mon.2 v2:172.21.15.102:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7ce0036040 con 0x563e932fe3b0 2024-09-10T09:57:34.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:34.149+0000 7f7cce000640 1 -- 172.21.15.5:0/1639085559 <== mon.2 v2:172.21.15.102:3300/0 5 ==== fsmap(e 42) ==== 8370+0+0 (secure 0 0 0) 0x7f7cdc024910 con 0x563e932fe3b0 2024-09-10T09:57:34.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:34.149+0000 7f7cf21bd740 10 client.29159 fetch_fsmap finished waiting for FSMap version 42 2024-09-10T09:57:34.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:34.149+0000 7f7cf21bd740 10 client.29159 resolve_mds: validated gid 24542 aka daemon mds.b 2024-09-10T09:57:34.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:34.149+0000 7f7cf21bd740 20 client.29159 populate_metadata read hostname 'smithi005' 2024-09-10T09:57:34.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:34.149+0000 7f7cf21bd740 1 --2- 172.21.15.5:0/1639085559 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x563e93385e90 0x563e933882b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:34.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:34.149+0000 7f7cf21bd740 4 client.29159 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 130.0, "await_for": 45.0}] 2024-09-10T09:57:34.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:34.149+0000 7f7cf21bd740 1 -- 172.21.15.5:0/1639085559 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 130.0, "await_for": 45.0}) -- 0x563e93271c60 con 0x563e93385e90 2024-09-10T09:57:34.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:34.149+0000 7f7ce7e00640 1 --2- 172.21.15.5:0/1639085559 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x563e93385e90 0x563e933882b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:34.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:34.151+0000 7f7cce000640 10 client.29159 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T09:57:34.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:34.152+0000 7f7ce7e00640 1 --2- 172.21.15.5:0/1639085559 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x563e93385e90 0x563e933882b0 crc :-1 s=READY pgs=272 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8fa53e5914adef9d server_cookie=935d6ab2e69cd372 in_seq=0 out_seq=0 2024-09-10T09:57:34.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:34.837+0000 7f7cce000640 1 -- 172.21.15.5:0/1639085559 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x563e93271c60 con 0x563e93385e90 2024-09-10T09:57:34.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:34.837+0000 7f7cce000640 10 client.29159 handle_command_reply: tid=0 2024-09-10T09:57:34.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:34.837+0000 7f7cf21bd740 1 client.29159 shutdown 2024-09-10T09:57:34.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:34.837+0000 7f7cf21bd740 1 -- 172.21.15.5:0/1639085559 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563e932fe3b0 msgr2=0x563e9330e8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:57:34.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:34.837+0000 7f7cf21bd740 1 --2- 172.21.15.5:0/1639085559 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563e932fe3b0 0x563e9330e8f0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x563e932fb7a0 tx=0x7f7cdc013b30 comp rx=0 tx=0).stop 2024-09-10T09:57:34.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:34.838+0000 7f7cf21bd740 1 -- 172.21.15.5:0/1639085559 shutdown_connections 2024-09-10T09:57:34.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:34.838+0000 7f7cf21bd740 1 --2- 172.21.15.5:0/1639085559 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x563e93385e90 0x563e933882b0 crc :-1 s=READY pgs=272 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:34.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:34.838+0000 7f7cf21bd740 1 --2- 172.21.15.5:0/1639085559 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563e93321080 0x563e93384060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:34.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:34.838+0000 7f7cce000640 0 client.29159 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T09:57:34.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:34.838+0000 7f7cf21bd740 1 --2- 172.21.15.5:0/1639085559 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563e93309590 0x563e9330ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:34.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:34.838+0000 7f7cf21bd740 1 --2- 172.21.15.5:0/1639085559 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563e932fe3b0 0x563e9330e8f0 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:34.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:34.838+0000 7f7cf21bd740 1 -- 172.21.15.5:0/1639085559 >> 172.21.15.5:0/1639085559 conn(0x563e932fa940 msgr2=0x563e93320070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:57:34.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:34.838+0000 7f7cf21bd740 1 -- 172.21.15.5:0/1639085559 shutdown_connections 2024-09-10T09:57:34.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:34.838+0000 7f7cf21bd740 1 -- 172.21.15.5:0/1639085559 wait complete. 2024-09-10T09:57:34.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:34.838+0000 7f7cf21bd740 20 client.29159 trim_cache size 0 max 16384 2024-09-10T09:57:34.840 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-10T09:57:34.841 INFO:teuthology.orchestra.run.smithi005.stdout: "epoch": 42, 2024-09-10T09:57:34.841 INFO:teuthology.orchestra.run.smithi005.stdout: "leader": 24542, 2024-09-10T09:57:34.841 INFO:teuthology.orchestra.run.smithi005.stdout: "set_version": 66, 2024-09-10T09:57:34.841 INFO:teuthology.orchestra.run.smithi005.stdout: "sets": { 2024-09-10T09:57:34.841 INFO:teuthology.orchestra.run.smithi005.stdout: "5b4e4e18": { 2024-09-10T09:57:34.841 INFO:teuthology.orchestra.run.smithi005.stdout: "version": 66, 2024-09-10T09:57:34.841 INFO:teuthology.orchestra.run.smithi005.stdout: "age_ref": 0.0, 2024-09-10T09:57:34.841 INFO:teuthology.orchestra.run.smithi005.stdout: "state": { 2024-09-10T09:57:34.841 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "QUIESCED", 2024-09-10T09:57:34.841 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0 2024-09-10T09:57:34.841 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T09:57:34.841 INFO:teuthology.orchestra.run.smithi005.stdout: "timeout": 90.0, 2024-09-10T09:57:34.841 INFO:teuthology.orchestra.run.smithi005.stdout: "expiration": 130.0, 2024-09-10T09:57:34.841 INFO:teuthology.orchestra.run.smithi005.stdout: "members": { 2024-09-10T09:57:34.841 INFO:teuthology.orchestra.run.smithi005.stdout: "file:/": { 2024-09-10T09:57:34.842 INFO:teuthology.orchestra.run.smithi005.stdout: "excluded": false, 2024-09-10T09:57:34.842 INFO:teuthology.orchestra.run.smithi005.stdout: "state": { 2024-09-10T09:57:34.842 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "QUIESCED", 2024-09-10T09:57:34.842 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0 2024-09-10T09:57:34.842 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T09:57:34.842 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T09:57:34.842 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T09:57:34.842 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T09:57:34.842 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T09:57:34.842 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-10T09:57:34.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:34.839+0000 7f7cef600640 1 -- 172.21.15.5:0/3143265502 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f7cd00a2cc0 msgr2=0x7f7cd00a5180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:57:34.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:34.839+0000 7f7cef600640 1 --2- 172.21.15.5:0/3143265502 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f7cd00a2cc0 0x7f7cd00a5180 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x563e932fbcc0 tx=0x7f7cd400a500 comp rx=0 tx=0).stop 2024-09-10T09:57:34.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:34.839+0000 7f7cef600640 1 -- 172.21.15.5:0/3143265502 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7ce810ef30 msgr2=0x7f7ce81acf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:57:34.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:34.839+0000 7f7cef600640 1 --2- 172.21.15.5:0/3143265502 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7ce810ef30 0x7f7ce81acf00 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f7ce0009bf0 tx=0x7f7ce0003f80 comp rx=0 tx=0).stop 2024-09-10T09:57:34.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:34.839+0000 7f7cef600640 1 -- 172.21.15.5:0/3143265502 shutdown_connections 2024-09-10T09:57:34.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:34.839+0000 7f7cef600640 1 --2- 172.21.15.5:0/3143265502 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f7cd00a2cc0 0x7f7cd00a5180 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:34.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:34.839+0000 7f7cef600640 1 --2- 172.21.15.5:0/3143265502 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7ce810ef30 0x7f7ce81acf00 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:34.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:34.839+0000 7f7cef600640 1 --2- 172.21.15.5:0/3143265502 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7ce810e580 0x7f7ce8110250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:34.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:34.839+0000 7f7cef600640 1 --2- 172.21.15.5:0/3143265502 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7ce810d9b0 0x7f7ce810fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:34.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:34.839+0000 7f7cef600640 1 -- 172.21.15.5:0/3143265502 >> 172.21.15.5:0/3143265502 conn(0x7f7ce806e140 msgr2=0x7f7ce8113960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:57:34.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:34.839+0000 7f7cef600640 1 -- 172.21.15.5:0/3143265502 shutdown_connections 2024-09-10T09:57:34.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:34.839+0000 7f7cef600640 1 -- 172.21.15.5:0/3143265502 wait complete. 2024-09-10T09:57:34.992 INFO:tasks.workunit.client.0.smithi005.stdout: 10 90419 14.39 MB/sec execute 193 sec latency 837.311 ms 2024-09-10T09:57:35.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:57:35 smithi005 ceph-mon[30350]: pgmap v970: 129 pgs: 129 active+clean; 616 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 815 KiB/s rd, 308 KiB/s wr, 40 op/s 2024-09-10T09:57:35.639 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '5b4e4e18', quiesce took 5.5 seconds. Will release after: 4.5 2024-09-10T09:57:35.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:57:35 smithi159 ceph-mon[35513]: pgmap v970: 129 pgs: 129 active+clean; 616 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 815 KiB/s rd, 308 KiB/s wr, 40 op/s 2024-09-10T09:57:35.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:57:35 smithi102 ceph-mon[35442]: pgmap v970: 129 pgs: 129 active+clean; 616 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 815 KiB/s rd, 308 KiB/s wr, 40 op/s 2024-09-10T09:57:35.992 INFO:tasks.workunit.client.0.smithi005.stdout: 10 90419 14.32 MB/sec execute 194 sec latency 1837.411 ms 2024-09-10T09:57:36.992 INFO:tasks.workunit.client.0.smithi005.stdout: 10 90419 14.24 MB/sec execute 195 sec latency 2837.471 ms 2024-09-10T09:57:37.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:57:37 smithi005 ceph-mon[30350]: pgmap v971: 129 pgs: 129 active+clean; 622 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.1 MiB/s wr, 168 op/s 2024-09-10T09:57:37.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:57:37 smithi005 ceph-mon[30350]: scrub summary: idle 2024-09-10T09:57:37.807 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:57:37 smithi005 ceph-mon[30350]: scrub summary: idle 2024-09-10T09:57:37.807 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:57:37 smithi005 ceph-mon[30350]: scrub summary: idle 2024-09-10T09:57:37.807 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:57:37 smithi005 ceph-mon[30350]: scrub summary: idle 2024-09-10T09:57:37.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:57:37 smithi159 ceph-mon[35513]: pgmap v971: 129 pgs: 129 active+clean; 622 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.1 MiB/s wr, 168 op/s 2024-09-10T09:57:37.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:57:37 smithi159 ceph-mon[35513]: scrub summary: idle 2024-09-10T09:57:37.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:57:37 smithi159 ceph-mon[35513]: scrub summary: idle 2024-09-10T09:57:37.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:57:37 smithi159 ceph-mon[35513]: scrub summary: idle 2024-09-10T09:57:37.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:57:37 smithi159 ceph-mon[35513]: scrub summary: idle 2024-09-10T09:57:37.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:57:37 smithi102 ceph-mon[35442]: pgmap v971: 129 pgs: 129 active+clean; 622 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.1 MiB/s wr, 168 op/s 2024-09-10T09:57:37.885 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:57:37 smithi102 ceph-mon[35442]: scrub summary: idle 2024-09-10T09:57:37.885 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:57:37 smithi102 ceph-mon[35442]: scrub summary: idle 2024-09-10T09:57:37.885 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:57:37 smithi102 ceph-mon[35442]: scrub summary: idle 2024-09-10T09:57:37.885 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:57:37 smithi102 ceph-mon[35442]: scrub summary: idle 2024-09-10T09:57:37.992 INFO:tasks.workunit.client.0.smithi005.stdout: 10 90419 14.17 MB/sec execute 196 sec latency 3837.624 ms 2024-09-10T09:57:38.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:57:38 smithi005 ceph-mon[30350]: scrub summary: idle 2024-09-10T09:57:38.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:57:38 smithi159 ceph-mon[35513]: scrub summary: idle 2024-09-10T09:57:38.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:57:38 smithi102 ceph-mon[35442]: scrub summary: idle 2024-09-10T09:57:38.992 INFO:tasks.workunit.client.0.smithi005.stdout: 10 90419 14.10 MB/sec execute 197 sec latency 4837.774 ms 2024-09-10T09:57:39.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:57:39 smithi005 ceph-mon[30350]: pgmap v972: 129 pgs: 129 active+clean; 623 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 1.3 MiB/s wr, 191 op/s 2024-09-10T09:57:39.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:57:39 smithi159 ceph-mon[35513]: pgmap v972: 129 pgs: 129 active+clean; 623 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 1.3 MiB/s wr, 191 op/s 2024-09-10T09:57:39.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:57:39 smithi102 ceph-mon[35442]: pgmap v972: 129 pgs: 129 active+clean; 623 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 1.3 MiB/s wr, 191 op/s 2024-09-10T09:57:39.992 INFO:tasks.workunit.client.0.smithi005.stdout: 10 90419 14.03 MB/sec execute 198 sec latency 5837.870 ms 2024-09-10T09:57:40.140 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T09:57:40.532 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:57:40.992 INFO:tasks.workunit.client.0.smithi005.stdout: 10 90419 13.96 MB/sec execute 199 sec latency 6837.940 ms 2024-09-10T09:57:41.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:41.070+0000 7f35e7200640 1 -- 172.21.15.5:0/2140058720 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f35e81047e0 msgr2=0x7f35e8108830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:57:41.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:41.070+0000 7f35e7200640 1 --2- 172.21.15.5:0/2140058720 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f35e81047e0 0x7f35e8108830 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f35d0009900 tx=0x7f35d002a410 comp rx=0 tx=0).stop 2024-09-10T09:57:41.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:41.070+0000 7f35e7200640 1 -- 172.21.15.5:0/2140058720 shutdown_connections 2024-09-10T09:57:41.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:41.070+0000 7f35e7200640 1 --2- 172.21.15.5:0/2140058720 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f35e8108f60 0x7f35e8110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:41.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:41.070+0000 7f35e7200640 1 --2- 172.21.15.5:0/2140058720 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f35e81047e0 0x7f35e8108830 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:41.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:41.070+0000 7f35e7200640 1 --2- 172.21.15.5:0/2140058720 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f35e8103e30 0x7f35e8104210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:41.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:41.070+0000 7f35e7200640 1 -- 172.21.15.5:0/2140058720 >> 172.21.15.5:0/2140058720 conn(0x7f35e80ff8d0 msgr2=0x7f35e8101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:57:41.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:41.070+0000 7f35e7200640 1 -- 172.21.15.5:0/2140058720 shutdown_connections 2024-09-10T09:57:41.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:41.070+0000 7f35e7200640 1 -- 172.21.15.5:0/2140058720 wait complete. 2024-09-10T09:57:41.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:41.071+0000 7f35e7200640 1 Processor -- start 2024-09-10T09:57:41.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:41.071+0000 7f35e7200640 1 -- start start 2024-09-10T09:57:41.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:41.072+0000 7f35e7200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f35e8103e30 0x7f35e819a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:41.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:41.072+0000 7f35e7200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f35e81047e0 0x7f35e819a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:41.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:41.072+0000 7f35e7200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f35e8108f60 0x7f35e819ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:41.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:41.072+0000 7f35e7200640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f35e8114010 con 0x7f35e8108f60 2024-09-10T09:57:41.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:41.072+0000 7f35e7200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f35e8113e90 con 0x7f35e8103e30 2024-09-10T09:57:41.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:41.072+0000 7f35e7200640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f35e8114190 con 0x7f35e81047e0 2024-09-10T09:57:41.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:41.072+0000 7f35dd400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f35e81047e0 0x7f35e819a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:41.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:41.072+0000 7f35dd400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f35e81047e0 0x7f35e819a9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:47046/0 (socket says 172.21.15.5:47046) 2024-09-10T09:57:41.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:41.072+0000 7f35e6800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f35e8108f60 0x7f35e819ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:41.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:41.072+0000 7f35dd400640 1 -- 172.21.15.5:0/2952949859 learned_addr learned my addr 172.21.15.5:0/2952949859 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:57:41.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:41.072+0000 7f35e5e00640 1 --2- 172.21.15.5:0/2952949859 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f35e8103e30 0x7f35e819a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:41.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:41.072+0000 7f35dd400640 1 -- 172.21.15.5:0/2952949859 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f35e8103e30 msgr2=0x7f35e819a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:57:41.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:41.072+0000 7f35dd400640 1 --2- 172.21.15.5:0/2952949859 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f35e8103e30 0x7f35e819a4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:41.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:41.072+0000 7f35dd400640 1 -- 172.21.15.5:0/2952949859 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f35e8108f60 msgr2=0x7f35e819ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:57:41.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:41.072+0000 7f35dd400640 1 --2- 172.21.15.5:0/2952949859 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f35e8108f60 0x7f35e819ed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:41.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:41.072+0000 7f35dd400640 1 -- 172.21.15.5:0/2952949859 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35e819f460 con 0x7f35e81047e0 2024-09-10T09:57:41.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:41.073+0000 7f35e5e00640 1 --2- 172.21.15.5:0/2952949859 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f35e8103e30 0x7f35e819a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:57:41.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:41.073+0000 7f35e6800640 1 --2- 172.21.15.5:0/2952949859 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f35e8108f60 0x7f35e819ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:57:41.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:41.073+0000 7f35dd400640 1 --2- 172.21.15.5:0/2952949859 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f35e81047e0 0x7f35e819a9f0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f35d0004770 tx=0x7f35d0003c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:57:41.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:41.073+0000 7f35df400640 1 -- 172.21.15.5:0/2952949859 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35d003f070 con 0x7f35e81047e0 2024-09-10T09:57:41.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:41.073+0000 7f35e7200640 1 -- 172.21.15.5:0/2952949859 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35e819f6f0 con 0x7f35e81047e0 2024-09-10T09:57:41.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:41.073+0000 7f35df400640 1 -- 172.21.15.5:0/2952949859 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f35d0031ca0 con 0x7f35e81047e0 2024-09-10T09:57:41.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:41.073+0000 7f35df400640 1 -- 172.21.15.5:0/2952949859 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35d0035030 con 0x7f35e81047e0 2024-09-10T09:57:41.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:41.073+0000 7f35e7200640 1 -- 172.21.15.5:0/2952949859 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35e81a7030 con 0x7f35e81047e0 2024-09-10T09:57:41.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:41.074+0000 7f35e7200640 1 -- 172.21.15.5:0/2952949859 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f35e8104210 con 0x7f35e81047e0 2024-09-10T09:57:41.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:41.076+0000 7f35df400640 1 -- 172.21.15.5:0/2952949859 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f35d0042070 con 0x7f35e81047e0 2024-09-10T09:57:41.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:41.078+0000 7f35df400640 1 --2- 172.21.15.5:0/2952949859 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f35cc0a2b80 0x7f35cc0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:41.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:41.078+0000 7f35df400640 1 -- 172.21.15.5:0/2952949859 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f35d00f00d0 con 0x7f35e81047e0 2024-09-10T09:57:41.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:41.078+0000 7f35e5e00640 1 --2- 172.21.15.5:0/2952949859 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f35cc0a2b80 0x7f35cc0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:41.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:41.079+0000 7f35e5e00640 1 --2- 172.21.15.5:0/2952949859 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f35cc0a2b80 0x7f35cc0a5040 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f35d8009fc0 tx=0x7f35d8005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:57:41.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:41.080+0000 7f35df400640 1 -- 172.21.15.5:0/2952949859 <== mon.1 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f35d00bc590 con 0x7f35e81047e0 2024-09-10T09:57:41.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:41.314+0000 7f35e7200640 1 -- 172.21.15.5:0/2952949859 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f35e819b810 con 0x7f35e81047e0 2024-09-10T09:57:41.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:41.315+0000 7f35df400640 1 -- 172.21.15.5:0/2952949859 <== mon.1 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 42 v42) ==== 94+0+11969 (secure 0 0 0) 0x7f35e819b810 con 0x7f35e81047e0 2024-09-10T09:57:41.318 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-10T09:57:41.318 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":42,"btime":"2024-09-10T09:55:06:670916+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14454,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/2970594311","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2970594311},{"type":"v1","addr":"172.21.15.5:6835","nonce":2970594311}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14514,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6839/3616502174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":3616502174},{"type":"v1","addr":"172.21.15.5:6839","nonce":3616502174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":42,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T09:35:29.626287+0000","modified":"2024-09-10T09:55:06.419829+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24542,"mds_1":24535,"mds_2":24511,"mds_3":24506,"mds_4":24443},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14520":{"gid":14520,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/2073599842","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":2073599842},{"type":"v1","addr":"172.21.15.5:6841","nonce":2073599842}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"k","rank":4,"incarnation":26,"state":"up:active","state_seq":24,"addr":"172.21.15.102:6833/2457007427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":2457007427},{"type":"v1","addr":"172.21.15.102:6833","nonce":2457007427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/356445159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":356445159},{"type":"v1","addr":"172.21.15.159:6833","nonce":356445159}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6837/2894314507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":2894314507},{"type":"v1","addr":"172.21.15.5:6837","nonce":2894314507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.102:6835/1647549111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":1647549111},{"type":"v1","addr":"172.21.15.102:6835","nonce":1647549111}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6835/952657242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":952657242},{"type":"v1","addr":"172.21.15.159:6835","nonce":952657242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":19,"addr":"172.21.15.102:6837/2689776764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":2689776764},{"type":"v1","addr":"172.21.15.102:6837","nonce":2689776764}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.159:6837/620627639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":620627639},{"type":"v1","addr":"172.21.15.159:6837","nonce":620627639}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24535":{"gid":24535,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":18,"addr":"172.21.15.159:6839/2073739599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":2073739599},{"type":"v1","addr":"172.21.15.159:6839","nonce":2073739599}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24542":{"gid":24542,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.102:6839/335409749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6838","nonce":335409749},{"type":"v1","addr":"172.21.15.102:6839","nonce":335409749}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24542,"qdb_cluster":[24542,24535,24511,24506,24443]},"id":1}]} 2024-09-10T09:57:41.319 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 42 2024-09-10T09:57:41.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:41.318+0000 7f35e7200640 1 -- 172.21.15.5:0/2952949859 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f35cc0a2b80 msgr2=0x7f35cc0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:57:41.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:41.318+0000 7f35e7200640 1 --2- 172.21.15.5:0/2952949859 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f35cc0a2b80 0x7f35cc0a5040 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f35d8009fc0 tx=0x7f35d8005f90 comp rx=0 tx=0).stop 2024-09-10T09:57:41.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:41.318+0000 7f35e7200640 1 -- 172.21.15.5:0/2952949859 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f35e81047e0 msgr2=0x7f35e819a9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:57:41.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:41.318+0000 7f35e7200640 1 --2- 172.21.15.5:0/2952949859 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f35e81047e0 0x7f35e819a9f0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f35d0004770 tx=0x7f35d0003c10 comp rx=0 tx=0).stop 2024-09-10T09:57:41.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:41.318+0000 7f35e7200640 1 -- 172.21.15.5:0/2952949859 shutdown_connections 2024-09-10T09:57:41.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:41.318+0000 7f35e7200640 1 --2- 172.21.15.5:0/2952949859 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f35cc0a2b80 0x7f35cc0a5040 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:41.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:41.318+0000 7f35e7200640 1 --2- 172.21.15.5:0/2952949859 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f35e8108f60 0x7f35e819ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:41.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:41.318+0000 7f35e7200640 1 --2- 172.21.15.5:0/2952949859 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f35e81047e0 0x7f35e819a9f0 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:41.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:41.318+0000 7f35e7200640 1 --2- 172.21.15.5:0/2952949859 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f35e8103e30 0x7f35e819a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:41.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:41.318+0000 7f35e7200640 1 -- 172.21.15.5:0/2952949859 >> 172.21.15.5:0/2952949859 conn(0x7f35e80ff8d0 msgr2=0x7f35e8100ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:57:41.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:41.318+0000 7f35e7200640 1 -- 172.21.15.5:0/2952949859 shutdown_connections 2024-09-10T09:57:41.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:41.318+0000 7f35e7200640 1 -- 172.21.15.5:0/2952949859 wait complete. 2024-09-10T09:57:41.668 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:57:41 smithi005 ceph-mon[30350]: pgmap v973: 129 pgs: 129 active+clean; 621 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 1.4 MiB/s wr, 225 op/s 2024-09-10T09:57:41.668 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:57:41 smithi005 ceph-mon[30350]: from='client.? 172.21.15.5:0/2952949859' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:57:41.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:57:41 smithi159 ceph-mon[35513]: pgmap v973: 129 pgs: 129 active+clean; 621 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 1.4 MiB/s wr, 225 op/s 2024-09-10T09:57:41.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:57:41 smithi159 ceph-mon[35513]: from='client.? 172.21.15.5:0/2952949859' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:57:41.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:57:41 smithi102 ceph-mon[35442]: pgmap v973: 129 pgs: 129 active+clean; 621 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 1.4 MiB/s wr, 225 op/s 2024-09-10T09:57:41.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:57:41 smithi102 ceph-mon[35442]: from='client.? 172.21.15.5:0/2952949859' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:57:41.992 INFO:tasks.workunit.client.0.smithi005.stdout: 10 90419 13.89 MB/sec execute 200 sec latency 7838.004 ms 2024-09-10T09:57:41.994 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24542 quiesce db --set-id 5b4e4e18 --release --await' 2024-09-10T09:57:41.994 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24542 quiesce db --set-id 5b4e4e18 --release --await 2024-09-10T09:57:42.372 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:57:42.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:42.927+0000 7f8014400640 1 -- 172.21.15.5:0/3405918037 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f800c074620 msgr2=0x7f800c074a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:57:42.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:42.927+0000 7f8014400640 1 --2- 172.21.15.5:0/3405918037 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f800c074620 0x7f800c074a00 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f8000009900 tx=0x7f800002a480 comp rx=0 tx=0).stop 2024-09-10T09:57:42.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:42.928+0000 7f8014400640 1 -- 172.21.15.5:0/3405918037 shutdown_connections 2024-09-10T09:57:42.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:42.928+0000 7f8014400640 1 --2- 172.21.15.5:0/3405918037 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f800c10e3c0 0x7f800c110810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:42.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:42.928+0000 7f8014400640 1 --2- 172.21.15.5:0/3405918037 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f800c074f40 0x7f800c10de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:42.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:42.928+0000 7f8014400640 1 --2- 172.21.15.5:0/3405918037 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f800c074620 0x7f800c074a00 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:42.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:42.928+0000 7f8014400640 1 -- 172.21.15.5:0/3405918037 >> 172.21.15.5:0/3405918037 conn(0x7f800c0fd8c0 msgr2=0x7f800c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:57:42.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:42.928+0000 7f8014400640 1 -- 172.21.15.5:0/3405918037 shutdown_connections 2024-09-10T09:57:42.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:42.928+0000 7f8014400640 1 -- 172.21.15.5:0/3405918037 wait complete. 2024-09-10T09:57:42.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:42.929+0000 7f8014400640 1 Processor -- start 2024-09-10T09:57:42.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:42.929+0000 7f8014400640 1 -- start start 2024-09-10T09:57:42.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:42.930+0000 7f8014400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f800c074620 0x7f800c19a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:42.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:42.930+0000 7f8014400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f800c074f40 0x7f800c19abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:42.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:42.930+0000 7f8014400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f800c10e3c0 0x7f800c19ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:42.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:42.930+0000 7f8014400640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f800c1141c0 con 0x7f800c074620 2024-09-10T09:57:42.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:42.930+0000 7f8014400640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f800c114040 con 0x7f800c074f40 2024-09-10T09:57:42.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:42.930+0000 7f8014400640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f800c114340 con 0x7f800c10e3c0 2024-09-10T09:57:42.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:42.930+0000 7f8011a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f800c074620 0x7f800c19a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:42.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:42.930+0000 7f8011a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f800c074620 0x7f800c19a680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52598/0 (socket says 172.21.15.5:52598) 2024-09-10T09:57:42.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:42.930+0000 7f8011a00640 1 -- 172.21.15.5:0/871022712 learned_addr learned my addr 172.21.15.5:0/871022712 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:57:42.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:42.930+0000 7f8012400640 1 --2- 172.21.15.5:0/871022712 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f800c10e3c0 0x7f800c19ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:42.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:42.930+0000 7f8011000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f800c074f40 0x7f800c19abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:42.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:42.930+0000 7f8011a00640 1 -- 172.21.15.5:0/871022712 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f800c10e3c0 msgr2=0x7f800c19ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:57:42.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:42.930+0000 7f8011a00640 1 --2- 172.21.15.5:0/871022712 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f800c10e3c0 0x7f800c19ef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:42.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:42.930+0000 7f8011a00640 1 -- 172.21.15.5:0/871022712 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f800c074f40 msgr2=0x7f800c19abc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:57:42.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:42.930+0000 7f8011a00640 1 --2- 172.21.15.5:0/871022712 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f800c074f40 0x7f800c19abc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:42.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:42.930+0000 7f8011a00640 1 -- 172.21.15.5:0/871022712 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f800c19f630 con 0x7f800c074620 2024-09-10T09:57:42.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:42.931+0000 7f8011a00640 1 --2- 172.21.15.5:0/871022712 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f800c074620 0x7f800c19a680 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f8000004770 tx=0x7f8000003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:57:42.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:42.931+0000 7f800a000640 1 -- 172.21.15.5:0/871022712 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f800003f070 con 0x7f800c074620 2024-09-10T09:57:42.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:42.931+0000 7f8014400640 1 -- 172.21.15.5:0/871022712 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f800c19f8c0 con 0x7f800c074620 2024-09-10T09:57:42.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:42.931+0000 7f800a000640 1 -- 172.21.15.5:0/871022712 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8000031ca0 con 0x7f800c074620 2024-09-10T09:57:42.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:42.931+0000 7f800a000640 1 -- 172.21.15.5:0/871022712 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8000035030 con 0x7f800c074620 2024-09-10T09:57:42.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:42.931+0000 7f8014400640 1 -- 172.21.15.5:0/871022712 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f800c1a7200 con 0x7f800c074620 2024-09-10T09:57:42.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:42.934+0000 7f8016df4740 1 Processor -- start 2024-09-10T09:57:42.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:42.934+0000 7f800a000640 1 -- 172.21.15.5:0/871022712 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f8000042070 con 0x7f800c074620 2024-09-10T09:57:42.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:42.934+0000 7f8016df4740 1 -- start start 2024-09-10T09:57:42.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:42.935+0000 7f8016df4740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5629033e7310 0x5629033e76f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:42.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:42.935+0000 7f8016df4740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5629033e7c30 0x5629033f1fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:42.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:42.935+0000 7f8016df4740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5629033f24f0 0x5629033f48e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:42.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:42.935+0000 7f8016df4740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x56290334d650 con 0x5629033e7c30 2024-09-10T09:57:42.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:42.935+0000 7f8016df4740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x5629032f4f10 con 0x5629033e7310 2024-09-10T09:57:42.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:42.935+0000 7f8016df4740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x562903349610 con 0x5629033f24f0 2024-09-10T09:57:42.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:42.935+0000 7f8011000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5629033f24f0 0x5629033f48e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:42.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:42.935+0000 7f8011a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5629033e7c30 0x5629033f1fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:42.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:42.935+0000 7f8011000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5629033f24f0 0x5629033f48e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:47082/0 (socket says 172.21.15.5:47082) 2024-09-10T09:57:42.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:42.935+0000 7f8011000640 1 -- 172.21.15.5:0/1976361064 learned_addr learned my addr 172.21.15.5:0/1976361064 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:57:42.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:42.935+0000 7f8012400640 1 --2- 172.21.15.5:0/1976361064 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5629033e7310 0x5629033e76f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:42.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:42.935+0000 7f8011000640 1 -- 172.21.15.5:0/1976361064 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5629033e7310 msgr2=0x5629033e76f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:57:42.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:42.935+0000 7f8011000640 1 --2- 172.21.15.5:0/1976361064 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5629033e7310 0x5629033e76f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:42.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:42.935+0000 7f8011000640 1 -- 172.21.15.5:0/1976361064 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5629033e7c30 msgr2=0x5629033f1fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:57:42.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:42.935+0000 7f8011000640 1 --2- 172.21.15.5:0/1976361064 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5629033e7c30 0x5629033f1fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:42.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:42.936+0000 7f8011000640 1 -- 172.21.15.5:0/1976361064 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56290305c580 con 0x5629033f24f0 2024-09-10T09:57:42.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:42.936+0000 7f8011000640 1 --2- 172.21.15.5:0/1976361064 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5629033f24f0 0x5629033f48e0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f7ff8009ca0 tx=0x7f7ff8011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=63a9c7343004df81 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:57:42.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:42.936+0000 7f800a000640 1 --2- 172.21.15.5:0/871022712 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f7ff40a2b80 0x7f7ff40a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:42.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:42.936+0000 7f7ffea00640 1 -- 172.21.15.5:0/1976361064 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ff801e070 con 0x5629033f24f0 2024-09-10T09:57:42.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:42.936+0000 7f800a000640 1 -- 172.21.15.5:0/871022712 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f80000efa10 con 0x7f800c074620 2024-09-10T09:57:42.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:42.936+0000 7f7ffea00640 1 -- 172.21.15.5:0/1976361064 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7ff8019b10 con 0x5629033f24f0 2024-09-10T09:57:42.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:42.936+0000 7f7ffea00640 1 -- 172.21.15.5:0/1976361064 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ff800ea30 con 0x5629033f24f0 2024-09-10T09:57:42.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:42.936+0000 7f8012400640 1 --2- 172.21.15.5:0/871022712 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f7ff40a2b80 0x7f7ff40a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:42.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:42.937+0000 7f8016df4740 1 -- 172.21.15.5:0/1976361064 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5629033f24f0 msgr2=0x5629033f48e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:57:42.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:42.937+0000 7f8016df4740 1 --2- 172.21.15.5:0/1976361064 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5629033f24f0 0x5629033f48e0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f7ff8009ca0 tx=0x7f7ff8011810 comp rx=0 tx=0).stop 2024-09-10T09:57:42.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:42.937+0000 7f8012400640 1 --2- 172.21.15.5:0/871022712 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f7ff40a2b80 0x7f7ff40a5040 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f8004005fd0 tx=0x7f8004009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:57:42.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:42.937+0000 7f8016df4740 1 -- 172.21.15.5:0/1976361064 shutdown_connections 2024-09-10T09:57:42.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:42.937+0000 7f8016df4740 1 --2- 172.21.15.5:0/1976361064 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5629033f24f0 0x5629033f48e0 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:42.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:42.937+0000 7f8016df4740 1 --2- 172.21.15.5:0/1976361064 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5629033e7c30 0x5629033f1fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:42.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:42.937+0000 7f8016df4740 1 --2- 172.21.15.5:0/1976361064 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5629033e7310 0x5629033e76f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:42.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:42.937+0000 7f8016df4740 1 -- 172.21.15.5:0/1976361064 >> 172.21.15.5:0/1976361064 conn(0x5629033e38a0 msgr2=0x5629033e3c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:57:42.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:42.937+0000 7f8016df4740 1 -- 172.21.15.5:0/1976361064 shutdown_connections 2024-09-10T09:57:42.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:42.938+0000 7f8016df4740 1 -- 172.21.15.5:0/1976361064 wait complete. 2024-09-10T09:57:42.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:42.938+0000 7f8016df4740 1 -- start start 2024-09-10T09:57:42.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:42.938+0000 7f8016df4740 1 Processor -- start 2024-09-10T09:57:42.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:42.939+0000 7f8016df4740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5629033e7310 0x5629033ef3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:42.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:42.939+0000 7f8016df4740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5629033e7c30 0x5629033ef930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:42.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:42.939+0000 7f8016df4740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5629033f24f0 0x5629033efe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:42.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:42.939+0000 7f8016df4740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5629033f50c0 con 0x5629033f24f0 2024-09-10T09:57:42.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:42.939+0000 7f8016df4740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x562903308350 con 0x5629033e7310 2024-09-10T09:57:42.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:42.939+0000 7f8016df4740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5629033f5240 con 0x5629033e7c30 2024-09-10T09:57:42.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:42.939+0000 7f8016df4740 10 client.0 _ll_register_callbacks cb 0x56290332b290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:57:42.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:42.939+0000 7f8012400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5629033f24f0 0x5629033efe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:42.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:42.939+0000 7f8012400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5629033f24f0 0x5629033efe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52606/0 (socket says 172.21.15.5:52606) 2024-09-10T09:57:42.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:42.939+0000 7f8012400640 1 -- 172.21.15.5:0/2728641511 learned_addr learned my addr 172.21.15.5:0/2728641511 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:57:42.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:42.939+0000 7f8011a00640 1 --2- 172.21.15.5:0/2728641511 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5629033e7310 0x5629033ef3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:42.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:42.939+0000 7f8011000640 1 --2- 172.21.15.5:0/2728641511 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5629033e7c30 0x5629033ef930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:42.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:42.940+0000 7f8012400640 1 -- 172.21.15.5:0/2728641511 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5629033e7c30 msgr2=0x5629033ef930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:57:42.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:42.940+0000 7f8012400640 1 --2- 172.21.15.5:0/2728641511 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5629033e7c30 0x5629033ef930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:42.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:42.940+0000 7f8012400640 1 -- 172.21.15.5:0/2728641511 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5629033e7310 msgr2=0x5629033ef3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:57:42.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:42.940+0000 7f8012400640 1 --2- 172.21.15.5:0/2728641511 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5629033e7310 0x5629033ef3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:42.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:42.940+0000 7f8012400640 1 -- 172.21.15.5:0/2728641511 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562903126960 con 0x5629033f24f0 2024-09-10T09:57:42.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:42.940+0000 7f8011a00640 1 --2- 172.21.15.5:0/2728641511 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5629033e7310 0x5629033ef3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:57:42.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:42.940+0000 7f8012400640 1 --2- 172.21.15.5:0/2728641511 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5629033f24f0 0x5629033efe70 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f8004004040 tx=0x7f8004004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a043ce83064f10c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:57:42.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:42.940+0000 7f7ff3400640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-10T09:57:42.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:42.940+0000 7f7ff3400640 1 -- 172.21.15.5:0/2728641511 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8004004b20 con 0x5629033f24f0 2024-09-10T09:57:42.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:42.940+0000 7f7ff3400640 1 -- 172.21.15.5:0/2728641511 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f8004004cc0 con 0x5629033f24f0 2024-09-10T09:57:42.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:42.940+0000 7f8016df4740 1 -- 172.21.15.5:0/2728641511 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7ff8019b10 con 0x5629033f24f0 2024-09-10T09:57:42.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:42.941+0000 7f7ff3400640 1 -- 172.21.15.5:0/2728641511 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f800400f7d0 con 0x5629033f24f0 2024-09-10T09:57:42.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:42.941+0000 7f8016df4740 10 client.19119 fetch_fsmap learned FSMap version 42 2024-09-10T09:57:42.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:42.941+0000 7f8016df4740 1 -- 172.21.15.5:0/2728641511 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x562903400cf0 con 0x5629033f24f0 2024-09-10T09:57:42.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:42.941+0000 7f7ff3400640 1 -- 172.21.15.5:0/2728641511 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7ff8019b10 con 0x5629033f24f0 2024-09-10T09:57:42.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:42.941+0000 7f7ff3400640 1 -- 172.21.15.5:0/2728641511 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 42) ==== 8370+0+0 (secure 0 0 0) 0x7f8004026030 con 0x5629033f24f0 2024-09-10T09:57:42.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:42.941+0000 7f8016df4740 10 client.19119 fetch_fsmap finished waiting for FSMap version 42 2024-09-10T09:57:42.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:42.941+0000 7f8016df4740 10 client.19119 resolve_mds: validated gid 24542 aka daemon mds.b 2024-09-10T09:57:42.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:42.941+0000 7f8016df4740 20 client.19119 populate_metadata read hostname 'smithi005' 2024-09-10T09:57:42.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:42.941+0000 7f8016df4740 1 --2- 172.21.15.5:0/2728641511 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x5629034021b0 0x562903404650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:42.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:42.941+0000 7f8016df4740 4 client.19119 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:57:42.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:42.941+0000 7f8016df4740 1 -- 172.21.15.5:0/2728641511 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56290335abc0 con 0x5629034021b0 2024-09-10T09:57:42.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:42.942+0000 7f8011a00640 1 --2- 172.21.15.5:0/2728641511 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x5629034021b0 0x562903404650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:42.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:42.942+0000 7f8011a00640 1 --2- 172.21.15.5:0/2728641511 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x5629034021b0 0x562903404650 crc :-1 s=READY pgs=273 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=926a16d09c59ab7f server_cookie=55b11947652243aa in_seq=0 out_seq=0 2024-09-10T09:57:42.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:42.942+0000 7f7ff3400640 10 client.19119 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T09:57:42.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:42.943+0000 7f7ff3400640 1 -- 172.21.15.5:0/2728641511 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56290335abc0 con 0x5629034021b0 2024-09-10T09:57:42.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:42.943+0000 7f7ff3400640 10 client.19119 handle_command_reply: tid=0 2024-09-10T09:57:42.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:42.943+0000 7f8016df4740 1 client.19119 shutdown 2024-09-10T09:57:42.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:42.943+0000 7f8016df4740 1 -- 172.21.15.5:0/2728641511 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5629033f24f0 msgr2=0x5629033efe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:57:42.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:42.943+0000 7f8016df4740 1 --2- 172.21.15.5:0/2728641511 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5629033f24f0 0x5629033efe70 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f8004004040 tx=0x7f8004004ab0 comp rx=0 tx=0).stop 2024-09-10T09:57:42.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:42.943+0000 7f8016df4740 1 -- 172.21.15.5:0/2728641511 shutdown_connections 2024-09-10T09:57:42.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:42.943+0000 7f8016df4740 1 --2- 172.21.15.5:0/2728641511 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x5629034021b0 0x562903404650 crc :-1 s=READY pgs=273 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:42.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:42.943+0000 7f8016df4740 1 --2- 172.21.15.5:0/2728641511 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5629033f24f0 0x5629033efe70 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:42.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:42.943+0000 7f8016df4740 1 --2- 172.21.15.5:0/2728641511 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5629033e7c30 0x5629033ef930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:42.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:42.943+0000 7f7ff3400640 0 client.19119 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T09:57:42.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:42.943+0000 7f8016df4740 1 --2- 172.21.15.5:0/2728641511 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5629033e7310 0x5629033ef3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:42.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:42.943+0000 7f8016df4740 1 -- 172.21.15.5:0/2728641511 >> 172.21.15.5:0/2728641511 conn(0x5629033e38a0 msgr2=0x5629033e95d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:57:42.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:42.944+0000 7f8016df4740 1 -- 172.21.15.5:0/2728641511 shutdown_connections 2024-09-10T09:57:42.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:42.944+0000 7f8016df4740 1 -- 172.21.15.5:0/2728641511 wait complete. 2024-09-10T09:57:42.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:42.944+0000 7f8016df4740 20 client.19119 trim_cache size 0 max 16384 2024-09-10T09:57:42.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:42.962+0000 7f8016df4740 1 Processor -- start 2024-09-10T09:57:42.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:42.962+0000 7f8016df4740 1 -- start start 2024-09-10T09:57:42.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:42.962+0000 7f8016df4740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5629033e7310 0x562903409f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:42.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:42.962+0000 7f8016df4740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5629033e7c30 0x56290340a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:42.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:42.962+0000 7f8016df4740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5629033f24f0 0x5629033ec650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:42.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:42.962+0000 7f8016df4740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f800c1b10c0 con 0x5629033e7c30 2024-09-10T09:57:42.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:42.962+0000 7f8016df4740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f800c1b2c00 con 0x5629033e7310 2024-09-10T09:57:42.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:42.962+0000 7f8016df4740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f800c1b1790 con 0x5629033f24f0 2024-09-10T09:57:42.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:42.962+0000 7f8011000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5629033e7c30 0x56290340a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:42.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:42.963+0000 7f8011000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5629033e7c30 0x56290340a4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52612/0 (socket says 172.21.15.5:52612) 2024-09-10T09:57:42.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:42.963+0000 7f8011000640 1 -- 172.21.15.5:0/3880677627 learned_addr learned my addr 172.21.15.5:0/3880677627 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:57:42.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:42.963+0000 7f8011a00640 1 --2- 172.21.15.5:0/3880677627 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5629033e7310 0x562903409f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:42.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:42.963+0000 7f8012400640 1 --2- 172.21.15.5:0/3880677627 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5629033f24f0 0x5629033ec650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:42.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:42.963+0000 7f8011000640 1 -- 172.21.15.5:0/3880677627 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5629033f24f0 msgr2=0x5629033ec650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:57:42.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:42.963+0000 7f8011000640 1 --2- 172.21.15.5:0/3880677627 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5629033f24f0 0x5629033ec650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:42.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:42.963+0000 7f8011000640 1 -- 172.21.15.5:0/3880677627 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5629033e7310 msgr2=0x562903409f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:57:42.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:42.963+0000 7f8011000640 1 --2- 172.21.15.5:0/3880677627 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5629033e7310 0x562903409f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:42.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:42.963+0000 7f8011000640 1 -- 172.21.15.5:0/3880677627 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5629033e36d0 con 0x5629033e7c30 2024-09-10T09:57:42.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:42.963+0000 7f8011a00640 1 --2- 172.21.15.5:0/3880677627 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5629033e7310 0x562903409f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:57:42.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:42.963+0000 7f8012400640 1 --2- 172.21.15.5:0/3880677627 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5629033f24f0 0x5629033ec650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:57:42.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:42.963+0000 7f8011000640 1 --2- 172.21.15.5:0/3880677627 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5629033e7c30 0x56290340a4b0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f7ff8009ca0 tx=0x7f7ff80096c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac710fb433ba641a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:57:42.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:42.964+0000 7f7ffe000640 1 -- 172.21.15.5:0/3880677627 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ff802a070 con 0x5629033e7c30 2024-09-10T09:57:42.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:42.964+0000 7f7ffe000640 1 -- 172.21.15.5:0/3880677627 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7ff8005520 con 0x5629033e7c30 2024-09-10T09:57:42.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:42.964+0000 7f7ffe000640 1 -- 172.21.15.5:0/3880677627 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ff801e040 con 0x5629033e7c30 2024-09-10T09:57:42.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:42.964+0000 7f8016df4740 1 -- 172.21.15.5:0/3880677627 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5629033e7c30 msgr2=0x56290340a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:57:42.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:42.964+0000 7f8016df4740 1 --2- 172.21.15.5:0/3880677627 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5629033e7c30 0x56290340a4b0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f7ff8009ca0 tx=0x7f7ff80096c0 comp rx=0 tx=0).stop 2024-09-10T09:57:42.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:42.964+0000 7f8016df4740 1 -- 172.21.15.5:0/3880677627 shutdown_connections 2024-09-10T09:57:42.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:42.964+0000 7f8016df4740 1 --2- 172.21.15.5:0/3880677627 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5629033f24f0 0x5629033ec650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:42.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:42.964+0000 7f8016df4740 1 --2- 172.21.15.5:0/3880677627 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5629033e7c30 0x56290340a4b0 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:42.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:42.964+0000 7f8016df4740 1 --2- 172.21.15.5:0/3880677627 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5629033e7310 0x562903409f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:42.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:42.964+0000 7f8016df4740 1 -- 172.21.15.5:0/3880677627 >> 172.21.15.5:0/3880677627 conn(0x5629033e38a0 msgr2=0x5629033e95d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:57:42.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:42.965+0000 7f8016df4740 1 -- 172.21.15.5:0/3880677627 shutdown_connections 2024-09-10T09:57:42.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:42.965+0000 7f8016df4740 1 -- 172.21.15.5:0/3880677627 wait complete. 2024-09-10T09:57:42.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:42.965+0000 7f8016df4740 1 -- start start 2024-09-10T09:57:42.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:42.965+0000 7f8016df4740 1 Processor -- start 2024-09-10T09:57:42.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:42.966+0000 7f8016df4740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5629033e7310 0x5629033f6870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:42.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:42.966+0000 7f8016df4740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5629033e7c30 0x5629033f6db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:42.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:42.966+0000 7f8016df4740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5629033f24f0 0x56290346bf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:42.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:42.966+0000 7f8016df4740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f800c1b2530 con 0x5629033f24f0 2024-09-10T09:57:42.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:42.966+0000 7f8016df4740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f800c1b1e60 con 0x5629033e7310 2024-09-10T09:57:42.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:42.966+0000 7f8016df4740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f800c1af0a0 con 0x5629033e7c30 2024-09-10T09:57:42.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:42.966+0000 7f8016df4740 10 client.0 _ll_register_callbacks cb 0x7f8004009e40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:57:42.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:42.966+0000 7f8011a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5629033e7310 0x5629033f6870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:42.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:42.966+0000 7f8011a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5629033e7310 0x5629033f6870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:52160/0 (socket says 172.21.15.5:52160) 2024-09-10T09:57:42.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:42.966+0000 7f8011a00640 1 -- 172.21.15.5:0/800438957 learned_addr learned my addr 172.21.15.5:0/800438957 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:57:42.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:42.966+0000 7f8011000640 1 --2- 172.21.15.5:0/800438957 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5629033e7c30 0x5629033f6db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:42.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:42.966+0000 7f8012400640 1 --2- 172.21.15.5:0/800438957 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5629033f24f0 0x56290346bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:42.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:42.966+0000 7f8011a00640 1 -- 172.21.15.5:0/800438957 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5629033e7c30 msgr2=0x5629033f6db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:57:42.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:42.966+0000 7f8011a00640 1 --2- 172.21.15.5:0/800438957 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5629033e7c30 0x5629033f6db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:42.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:42.966+0000 7f8011a00640 1 -- 172.21.15.5:0/800438957 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5629033f24f0 msgr2=0x56290346bf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:57:42.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:42.966+0000 7f8011a00640 1 --2- 172.21.15.5:0/800438957 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5629033f24f0 0x56290346bf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:42.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:42.966+0000 7f8011a00640 1 -- 172.21.15.5:0/800438957 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56290346c480 con 0x5629033e7310 2024-09-10T09:57:42.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:42.967+0000 7f8011a00640 1 --2- 172.21.15.5:0/800438957 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5629033e7310 0x5629033f6870 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f80000b89c0 tx=0x7f80000b83c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=12a6584983437687 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:57:42.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:42.967+0000 7f7ff3400640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-09-10T09:57:42.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:42.967+0000 7f7ff3400640 1 -- 172.21.15.5:0/800438957 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8000049040 con 0x5629033e7310 2024-09-10T09:57:42.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:42.967+0000 7f7ff3400640 1 -- 172.21.15.5:0/800438957 <== mon.2 v2:172.21.15.102:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f80000ba2f0 con 0x5629033e7310 2024-09-10T09:57:42.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:42.967+0000 7f8016df4740 1 -- 172.21.15.5:0/800438957 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7ff8005520 con 0x5629033e7310 2024-09-10T09:57:42.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:42.967+0000 7f7ff3400640 1 -- 172.21.15.5:0/800438957 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f8000005d50 con 0x5629033e7310 2024-09-10T09:57:42.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:42.967+0000 7f8016df4740 10 client.29183 fetch_fsmap learned FSMap version 42 2024-09-10T09:57:42.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:42.967+0000 7f8016df4740 1 -- 172.21.15.5:0/800438957 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x56290346d760 con 0x5629033e7310 2024-09-10T09:57:42.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:42.967+0000 7f7ff3400640 1 -- 172.21.15.5:0/800438957 <== mon.2 v2:172.21.15.102:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7ff8005520 con 0x5629033e7310 2024-09-10T09:57:42.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:42.968+0000 7f7ff3400640 1 -- 172.21.15.5:0/800438957 <== mon.2 v2:172.21.15.102:3300/0 5 ==== fsmap(e 42) ==== 8370+0+0 (secure 0 0 0) 0x7f800003b4f0 con 0x5629033e7310 2024-09-10T09:57:42.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:42.968+0000 7f8016df4740 10 client.29183 fetch_fsmap finished waiting for FSMap version 42 2024-09-10T09:57:42.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:42.968+0000 7f8016df4740 10 client.29183 resolve_mds: validated gid 24542 aka daemon mds.b 2024-09-10T09:57:42.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:42.968+0000 7f8016df4740 20 client.29183 populate_metadata read hostname 'smithi005' 2024-09-10T09:57:42.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:42.968+0000 7f8016df4740 1 --2- 172.21.15.5:0/800438957 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x56290346de80 0x562903470290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:42.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:42.968+0000 7f8016df4740 4 client.29183 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "5b4e4e18", "release": true, "await": true}] 2024-09-10T09:57:42.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:42.968+0000 7f8016df4740 1 -- 172.21.15.5:0/800438957 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "quiesce db", "set_id": "5b4e4e18", "release": true, "await": true}) -- 0x5629033550d0 con 0x56290346de80 2024-09-10T09:57:42.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:42.968+0000 7f8011000640 1 --2- 172.21.15.5:0/800438957 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x56290346de80 0x562903470290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:42.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:42.969+0000 7f8011000640 1 --2- 172.21.15.5:0/800438957 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x56290346de80 0x562903470290 crc :-1 s=READY pgs=274 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a3c8ef362a807ccd server_cookie=44661b2c056cab7a in_seq=0 out_seq=0 2024-09-10T09:57:42.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:42.969+0000 7f7ff3400640 10 client.29183 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T09:57:42.992 INFO:tasks.workunit.client.0.smithi005.stdout: 10 90420 13.82 MB/sec execute 201 sec latency 8837.337 ms 2024-09-10T09:57:43.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:43.036+0000 7f7ff3400640 1 -- 172.21.15.5:0/800438957 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5629033550d0 con 0x56290346de80 2024-09-10T09:57:43.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:43.036+0000 7f7ff3400640 10 client.29183 handle_command_reply: tid=0 2024-09-10T09:57:43.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:43.036+0000 7f8016df4740 1 client.29183 shutdown 2024-09-10T09:57:43.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:43.036+0000 7f8016df4740 1 -- 172.21.15.5:0/800438957 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5629033e7310 msgr2=0x5629033f6870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:57:43.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:43.036+0000 7f8016df4740 1 --2- 172.21.15.5:0/800438957 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5629033e7310 0x5629033f6870 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f80000b89c0 tx=0x7f80000b83c0 comp rx=0 tx=0).stop 2024-09-10T09:57:43.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:43.036+0000 7f8016df4740 1 -- 172.21.15.5:0/800438957 shutdown_connections 2024-09-10T09:57:43.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:43.036+0000 7f8016df4740 1 --2- 172.21.15.5:0/800438957 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x56290346de80 0x562903470290 crc :-1 s=READY pgs=274 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:43.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:43.036+0000 7f8016df4740 1 --2- 172.21.15.5:0/800438957 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5629033f24f0 0x56290346bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:43.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:43.036+0000 7f8016df4740 1 --2- 172.21.15.5:0/800438957 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5629033e7c30 0x5629033f6db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:43.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:43.036+0000 7f7ff3400640 0 client.29183 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T09:57:43.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:43.036+0000 7f8016df4740 1 --2- 172.21.15.5:0/800438957 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5629033e7310 0x5629033f6870 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:43.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:43.036+0000 7f8016df4740 1 -- 172.21.15.5:0/800438957 >> 172.21.15.5:0/800438957 conn(0x5629033e38a0 msgr2=0x5629033e2280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:57:43.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:43.037+0000 7f8016df4740 1 -- 172.21.15.5:0/800438957 shutdown_connections 2024-09-10T09:57:43.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:43.037+0000 7f8016df4740 1 -- 172.21.15.5:0/800438957 wait complete. 2024-09-10T09:57:43.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:43.037+0000 7f8016df4740 20 client.29183 trim_cache size 0 max 16384 2024-09-10T09:57:43.040 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-10T09:57:43.041 INFO:teuthology.orchestra.run.smithi005.stdout: "epoch": 42, 2024-09-10T09:57:43.041 INFO:teuthology.orchestra.run.smithi005.stdout: "leader": 24542, 2024-09-10T09:57:43.041 INFO:teuthology.orchestra.run.smithi005.stdout: "set_version": 68, 2024-09-10T09:57:43.041 INFO:teuthology.orchestra.run.smithi005.stdout: "sets": { 2024-09-10T09:57:43.041 INFO:teuthology.orchestra.run.smithi005.stdout: "5b4e4e18": { 2024-09-10T09:57:43.041 INFO:teuthology.orchestra.run.smithi005.stdout: "version": 68, 2024-09-10T09:57:43.041 INFO:teuthology.orchestra.run.smithi005.stdout: "age_ref": 0.0, 2024-09-10T09:57:43.041 INFO:teuthology.orchestra.run.smithi005.stdout: "state": { 2024-09-10T09:57:43.041 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "RELEASED", 2024-09-10T09:57:43.041 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0 2024-09-10T09:57:43.041 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T09:57:43.041 INFO:teuthology.orchestra.run.smithi005.stdout: "timeout": 90.0, 2024-09-10T09:57:43.041 INFO:teuthology.orchestra.run.smithi005.stdout: "expiration": 130.0, 2024-09-10T09:57:43.041 INFO:teuthology.orchestra.run.smithi005.stdout: "members": { 2024-09-10T09:57:43.042 INFO:teuthology.orchestra.run.smithi005.stdout: "file:/": { 2024-09-10T09:57:43.042 INFO:teuthology.orchestra.run.smithi005.stdout: "excluded": false, 2024-09-10T09:57:43.042 INFO:teuthology.orchestra.run.smithi005.stdout: "state": { 2024-09-10T09:57:43.042 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "RELEASED", 2024-09-10T09:57:43.042 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0 2024-09-10T09:57:43.042 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T09:57:43.042 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T09:57:43.042 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T09:57:43.042 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T09:57:43.042 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T09:57:43.042 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-10T09:57:43.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:43.038+0000 7f8014400640 1 -- 172.21.15.5:0/871022712 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f7ff40a2b80 msgr2=0x7f7ff40a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:57:43.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:43.038+0000 7f8014400640 1 --2- 172.21.15.5:0/871022712 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f7ff40a2b80 0x7f7ff40a5040 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f8004005fd0 tx=0x7f8004009500 comp rx=0 tx=0).stop 2024-09-10T09:57:43.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:43.038+0000 7f8014400640 1 -- 172.21.15.5:0/871022712 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f800c074620 msgr2=0x7f800c19a680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:57:43.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:43.038+0000 7f8014400640 1 --2- 172.21.15.5:0/871022712 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f800c074620 0x7f800c19a680 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f8000004770 tx=0x7f8000003c10 comp rx=0 tx=0).stop 2024-09-10T09:57:43.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:43.038+0000 7f8014400640 1 -- 172.21.15.5:0/871022712 shutdown_connections 2024-09-10T09:57:43.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:43.038+0000 7f8014400640 1 --2- 172.21.15.5:0/871022712 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f7ff40a2b80 0x7f7ff40a5040 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:43.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:43.038+0000 7f8014400640 1 --2- 172.21.15.5:0/871022712 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f800c10e3c0 0x7f800c19ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:43.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:43.038+0000 7f8014400640 1 --2- 172.21.15.5:0/871022712 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f800c074f40 0x7f800c19abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:43.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:43.038+0000 7f8014400640 1 --2- 172.21.15.5:0/871022712 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f800c074620 0x7f800c19a680 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:43.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:43.038+0000 7f8014400640 1 -- 172.21.15.5:0/871022712 >> 172.21.15.5:0/871022712 conn(0x7f800c0fd8c0 msgr2=0x7f800c0fe5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:57:43.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:43.038+0000 7f8014400640 1 -- 172.21.15.5:0/871022712 shutdown_connections 2024-09-10T09:57:43.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:43.038+0000 7f8014400640 1 -- 172.21.15.5:0/871022712 wait complete. 2024-09-10T09:57:43.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:57:43 smithi005 ceph-mon[30350]: pgmap v974: 129 pgs: 129 active+clean; 621 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 1.4 MiB/s wr, 222 op/s 2024-09-10T09:57:43.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:57:43 smithi159 ceph-mon[35513]: pgmap v974: 129 pgs: 129 active+clean; 621 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 1.4 MiB/s wr, 222 op/s 2024-09-10T09:57:43.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:57:43 smithi102 ceph-mon[35442]: pgmap v974: 129 pgs: 129 active+clean; 621 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 1.4 MiB/s wr, 222 op/s 2024-09-10T09:57:43.993 INFO:tasks.workunit.client.0.smithi005.stdout: 10 91000 13.88 MB/sec execute 202 sec latency 8900.211 ms 2024-09-10T09:57:44.116 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '5b4e4e18', total seconds elapsed: 14.0 2024-09-10T09:57:44.993 INFO:tasks.workunit.client.0.smithi005.stdout: 10 91731 13.98 MB/sec execute 203 sec latency 51.849 ms 2024-09-10T09:57:45.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:57:45 smithi005 ceph-mon[30350]: pgmap v975: 129 pgs: 129 active+clean; 622 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 1.3 MiB/s wr, 210 op/s 2024-09-10T09:57:45.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:57:45 smithi159 ceph-mon[35513]: pgmap v975: 129 pgs: 129 active+clean; 622 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 1.3 MiB/s wr, 210 op/s 2024-09-10T09:57:45.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:57:45 smithi102 ceph-mon[35442]: pgmap v975: 129 pgs: 129 active+clean; 622 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 1.3 MiB/s wr, 210 op/s 2024-09-10T09:57:45.993 INFO:tasks.workunit.client.0.smithi005.stdout: 10 92349 14.06 MB/sec execute 204 sec latency 105.688 ms 2024-09-10T09:57:46.993 INFO:tasks.workunit.client.0.smithi005.stdout: 10 93017 14.18 MB/sec execute 205 sec latency 49.120 ms 2024-09-10T09:57:47.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:57:47 smithi005 ceph-mon[30350]: pgmap v976: 129 pgs: 129 active+clean; 624 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.4 MiB/s wr, 234 op/s 2024-09-10T09:57:47.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:57:47 smithi159 ceph-mon[35513]: pgmap v976: 129 pgs: 129 active+clean; 624 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.4 MiB/s wr, 234 op/s 2024-09-10T09:57:47.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:57:47 smithi102 ceph-mon[35442]: pgmap v976: 129 pgs: 129 active+clean; 624 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.4 MiB/s wr, 234 op/s 2024-09-10T09:57:47.993 INFO:tasks.workunit.client.0.smithi005.stdout: 10 93719 14.30 MB/sec execute 206 sec latency 61.290 ms 2024-09-10T09:57:48.993 INFO:tasks.workunit.client.0.smithi005.stdout: 10 94431 14.41 MB/sec execute 207 sec latency 78.345 ms 2024-09-10T09:57:49.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:57:49 smithi005 ceph-mon[30350]: pgmap v977: 129 pgs: 129 active+clean; 631 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.2 MiB/s wr, 184 op/s 2024-09-10T09:57:49.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:57:49 smithi159 ceph-mon[35513]: pgmap v977: 129 pgs: 129 active+clean; 631 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.2 MiB/s wr, 184 op/s 2024-09-10T09:57:49.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:57:49 smithi102 ceph-mon[35442]: pgmap v977: 129 pgs: 129 active+clean; 631 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.2 MiB/s wr, 184 op/s 2024-09-10T09:57:49.993 INFO:tasks.workunit.client.0.smithi005.stdout: 10 95051 14.51 MB/sec execute 208 sec latency 58.422 ms 2024-09-10T09:57:50.993 INFO:tasks.workunit.client.0.smithi005.stdout: 10 95734 14.63 MB/sec execute 209 sec latency 59.180 ms 2024-09-10T09:57:51.807 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:57:51 smithi005 ceph-mon[30350]: pgmap v978: 129 pgs: 129 active+clean; 640 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 2.0 MiB/s wr, 280 op/s 2024-09-10T09:57:51.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:57:51 smithi159 ceph-mon[35513]: pgmap v978: 129 pgs: 129 active+clean; 640 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 2.0 MiB/s wr, 280 op/s 2024-09-10T09:57:51.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:57:51 smithi102 ceph-mon[35442]: pgmap v978: 129 pgs: 129 active+clean; 640 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 2.0 MiB/s wr, 280 op/s 2024-09-10T09:57:51.993 INFO:tasks.workunit.client.0.smithi005.stdout: 10 96385 14.71 MB/sec execute 210 sec latency 55.356 ms 2024-09-10T09:57:52.993 INFO:tasks.workunit.client.0.smithi005.stdout: 10 97024 14.78 MB/sec execute 211 sec latency 79.897 ms 2024-09-10T09:57:53.807 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:57:53 smithi005 ceph-mon[30350]: pgmap v979: 129 pgs: 129 active+clean; 643 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 2.3 MiB/s wr, 291 op/s 2024-09-10T09:57:53.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:57:53 smithi159 ceph-mon[35513]: pgmap v979: 129 pgs: 129 active+clean; 643 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 2.3 MiB/s wr, 291 op/s 2024-09-10T09:57:53.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:57:53 smithi102 ceph-mon[35442]: pgmap v979: 129 pgs: 129 active+clean; 643 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 2.3 MiB/s wr, 291 op/s 2024-09-10T09:57:53.993 INFO:tasks.workunit.client.0.smithi005.stdout: 10 97893 14.98 MB/sec execute 212 sec latency 49.585 ms 2024-09-10T09:57:54.994 INFO:tasks.workunit.client.0.smithi005.stdout: 10 98503 15.06 MB/sec execute 213 sec latency 39.023 ms 2024-09-10T09:57:55.739 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:57:55 smithi005 ceph-mon[30350]: pgmap v980: 129 pgs: 129 active+clean; 647 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 2.7 MiB/s wr, 332 op/s 2024-09-10T09:57:55.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:57:55 smithi159 ceph-mon[35513]: pgmap v980: 129 pgs: 129 active+clean; 647 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 2.7 MiB/s wr, 332 op/s 2024-09-10T09:57:55.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:57:55 smithi102 ceph-mon[35442]: pgmap v980: 129 pgs: 129 active+clean; 647 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 2.7 MiB/s wr, 332 op/s 2024-09-10T09:57:55.994 INFO:tasks.workunit.client.0.smithi005.stdout: 10 99069 15.10 MB/sec execute 214 sec latency 82.085 ms 2024-09-10T09:57:56.261 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T09:57:56.835 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:57:56.994 INFO:tasks.workunit.client.0.smithi005.stdout: 10 99686 15.19 MB/sec execute 215 sec latency 54.081 ms 2024-09-10T09:57:57.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:57.515+0000 7fde6be00640 1 -- 172.21.15.5:0/231579346 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fde6c1019c0 msgr2=0x7fde6c101e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:57:57.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:57.515+0000 7fde6be00640 1 --2- 172.21.15.5:0/231579346 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fde6c1019c0 0x7fde6c101e20 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fde58009900 tx=0x7fde5802a3f0 comp rx=0 tx=0).stop 2024-09-10T09:57:57.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:57.515+0000 7fde6be00640 1 -- 172.21.15.5:0/231579346 shutdown_connections 2024-09-10T09:57:57.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:57.515+0000 7fde6be00640 1 --2- 172.21.15.5:0/231579346 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fde6c102360 0x7fde6c10e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:57.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:57.515+0000 7fde6be00640 1 --2- 172.21.15.5:0/231579346 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fde6c1019c0 0x7fde6c101e20 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:57.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:57.515+0000 7fde6be00640 1 --2- 172.21.15.5:0/231579346 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fde6c1079c0 0x7fde6c107da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:57.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:57.515+0000 7fde6be00640 1 -- 172.21.15.5:0/231579346 >> 172.21.15.5:0/231579346 conn(0x7fde6c0fd740 msgr2=0x7fde6c0ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:57:57.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:57.515+0000 7fde6be00640 1 -- 172.21.15.5:0/231579346 shutdown_connections 2024-09-10T09:57:57.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:57.515+0000 7fde6be00640 1 -- 172.21.15.5:0/231579346 wait complete. 2024-09-10T09:57:57.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:57.516+0000 7fde6be00640 1 Processor -- start 2024-09-10T09:57:57.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:57.517+0000 7fde6be00640 1 -- start start 2024-09-10T09:57:57.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:57.517+0000 7fde6be00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fde6c1019c0 0x7fde6c1a4660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:57.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:57.517+0000 7fde6be00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fde6c102360 0x7fde6c1a4ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:57.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:57.517+0000 7fde6be00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fde6c1079c0 0x7fde6c10b920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:57.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:57.517+0000 7fde6be00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fde6c111dc0 con 0x7fde6c102360 2024-09-10T09:57:57.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:57.517+0000 7fde6be00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fde6c111c40 con 0x7fde6c1019c0 2024-09-10T09:57:57.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:57.517+0000 7fde6be00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fde6c111f40 con 0x7fde6c1079c0 2024-09-10T09:57:57.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:57.517+0000 7fde6aa00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fde6c1019c0 0x7fde6c1a4660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:57.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:57.517+0000 7fde6aa00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fde6c1019c0 0x7fde6c1a4660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:54296/0 (socket says 172.21.15.5:54296) 2024-09-10T09:57:57.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:57.517+0000 7fde6aa00640 1 -- 172.21.15.5:0/1272254696 learned_addr learned my addr 172.21.15.5:0/1272254696 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:57:57.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:57.517+0000 7fde6b400640 1 --2- 172.21.15.5:0/1272254696 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fde6c1079c0 0x7fde6c10b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:57.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:57.518+0000 7fde6a000640 1 --2- 172.21.15.5:0/1272254696 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fde6c102360 0x7fde6c1a4ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:57.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:57.518+0000 7fde6aa00640 1 -- 172.21.15.5:0/1272254696 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fde6c1079c0 msgr2=0x7fde6c10b920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:57:57.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:57.518+0000 7fde6aa00640 1 --2- 172.21.15.5:0/1272254696 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fde6c1079c0 0x7fde6c10b920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:57.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:57.518+0000 7fde6aa00640 1 -- 172.21.15.5:0/1272254696 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fde6c102360 msgr2=0x7fde6c1a4ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:57:57.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:57.518+0000 7fde6aa00640 1 --2- 172.21.15.5:0/1272254696 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fde6c102360 0x7fde6c1a4ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:57.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:57.518+0000 7fde6aa00640 1 -- 172.21.15.5:0/1272254696 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde6c10c1c0 con 0x7fde6c1019c0 2024-09-10T09:57:57.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:57.518+0000 7fde6a000640 1 --2- 172.21.15.5:0/1272254696 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fde6c102360 0x7fde6c1a4ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T09:57:57.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:57.518+0000 7fde6aa00640 1 --2- 172.21.15.5:0/1272254696 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fde6c1019c0 0x7fde6c1a4660 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fde5c00e9f0 tx=0x7fde5c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:57:57.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:57.518+0000 7fde70a00640 1 -- 172.21.15.5:0/1272254696 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde5c00cd80 con 0x7fde6c1019c0 2024-09-10T09:57:57.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:57.518+0000 7fde70a00640 1 -- 172.21.15.5:0/1272254696 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fde5c010910 con 0x7fde6c1019c0 2024-09-10T09:57:57.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:57.518+0000 7fde70a00640 1 -- 172.21.15.5:0/1272254696 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde5c018990 con 0x7fde6c1019c0 2024-09-10T09:57:57.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:57.519+0000 7fde6be00640 1 -- 172.21.15.5:0/1272254696 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde6c10c4b0 con 0x7fde6c1019c0 2024-09-10T09:57:57.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:57.519+0000 7fde6be00640 1 -- 172.21.15.5:0/1272254696 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde6c1ab5e0 con 0x7fde6c1019c0 2024-09-10T09:57:57.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:57.522+0000 7fde70a00640 1 -- 172.21.15.5:0/1272254696 <== mon.2 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7fde5c010ac0 con 0x7fde6c1019c0 2024-09-10T09:57:57.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:57.522+0000 7fde7483e740 1 Processor -- start 2024-09-10T09:57:57.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:57.522+0000 7fde7483e740 1 -- start start 2024-09-10T09:57:57.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:57.522+0000 7fde7483e740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5604f5f14d50 0x5604f5f15130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:57.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:57.522+0000 7fde7483e740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5604f5f15670 0x5604f5f1f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:57.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:57.522+0000 7fde7483e740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5604f5f1ff30 0x5604f5f22320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:57.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:57.522+0000 7fde7483e740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5604f5e7b3e0 con 0x5604f5f1ff30 2024-09-10T09:57:57.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:57.522+0000 7fde7483e740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x5604f5e22d80 con 0x5604f5f15670 2024-09-10T09:57:57.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:57.522+0000 7fde7483e740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5604f5e773a0 con 0x5604f5f14d50 2024-09-10T09:57:57.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:57.523+0000 7fde6aa00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5604f5f15670 0x5604f5f1f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:57.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:57.523+0000 7fde6b400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5604f5f14d50 0x5604f5f15130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:57.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:57.523+0000 7fde6a000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5604f5f1ff30 0x5604f5f22320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:57.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:57.523+0000 7fde6b400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5604f5f14d50 0x5604f5f15130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:48034/0 (socket says 172.21.15.5:48034) 2024-09-10T09:57:57.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:57.523+0000 7fde6b400640 1 -- 172.21.15.5:0/3549797420 learned_addr learned my addr 172.21.15.5:0/3549797420 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:57:57.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:57.523+0000 7fde6b400640 1 -- 172.21.15.5:0/3549797420 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5604f5f15670 msgr2=0x5604f5f1f9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:57:57.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:57.523+0000 7fde6b400640 1 --2- 172.21.15.5:0/3549797420 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5604f5f15670 0x5604f5f1f9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:57.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:57.523+0000 7fde6b400640 1 -- 172.21.15.5:0/3549797420 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5604f5f1ff30 msgr2=0x5604f5f22320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:57:57.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:57.523+0000 7fde6b400640 1 --2- 172.21.15.5:0/3549797420 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5604f5f1ff30 0x5604f5f22320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:57.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:57.523+0000 7fde6b400640 1 -- 172.21.15.5:0/3549797420 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5604f5e41ae0 con 0x5604f5f14d50 2024-09-10T09:57:57.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:57.524+0000 7fde6b400640 1 --2- 172.21.15.5:0/3549797420 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5604f5f14d50 0x5604f5f15130 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fde640039f0 tx=0x7fde64003eb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cf6e236dd1faed0f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:57:57.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:57.524+0000 7fde70a00640 1 --2- 172.21.15.5:0/1272254696 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fde500a2c00 0x7fde500a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:57.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:57.524+0000 7fde70a00640 1 -- 172.21.15.5:0/1272254696 <== mon.2 v2:172.21.15.102:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fde5c0cf720 con 0x7fde6c1019c0 2024-09-10T09:57:57.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:57.524+0000 7fde57e00640 1 -- 172.21.15.5:0/3549797420 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde6401a470 con 0x5604f5f14d50 2024-09-10T09:57:57.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:57.524+0000 7fde6aa00640 1 --2- 172.21.15.5:0/1272254696 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fde500a2c00 0x7fde500a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:57.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:57.524+0000 7fde57e00640 1 -- 172.21.15.5:0/3549797420 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fde64015070 con 0x5604f5f14d50 2024-09-10T09:57:57.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:57.524+0000 7fde57e00640 1 -- 172.21.15.5:0/3549797420 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde64010900 con 0x5604f5f14d50 2024-09-10T09:57:57.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:57.525+0000 7fde7483e740 1 -- 172.21.15.5:0/3549797420 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5604f5f14d50 msgr2=0x5604f5f15130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:57:57.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:57.525+0000 7fde6aa00640 1 --2- 172.21.15.5:0/1272254696 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fde500a2c00 0x7fde500a50c0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fde5c09bac0 tx=0x7fde5c0d5530 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:57:57.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:57.525+0000 7fde7483e740 1 --2- 172.21.15.5:0/3549797420 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5604f5f14d50 0x5604f5f15130 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fde640039f0 tx=0x7fde64003eb0 comp rx=0 tx=0).stop 2024-09-10T09:57:57.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:57.525+0000 7fde7483e740 1 -- 172.21.15.5:0/3549797420 shutdown_connections 2024-09-10T09:57:57.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:57.525+0000 7fde7483e740 1 --2- 172.21.15.5:0/3549797420 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5604f5f1ff30 0x5604f5f22320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:57.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:57.525+0000 7fde7483e740 1 --2- 172.21.15.5:0/3549797420 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5604f5f15670 0x5604f5f1f9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:57.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:57.525+0000 7fde7483e740 1 --2- 172.21.15.5:0/3549797420 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5604f5f14d50 0x5604f5f15130 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:57.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:57.525+0000 7fde7483e740 1 -- 172.21.15.5:0/3549797420 >> 172.21.15.5:0/3549797420 conn(0x5604f5f112e0 msgr2=0x5604f5f116c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:57:57.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:57.525+0000 7fde7483e740 1 -- 172.21.15.5:0/3549797420 shutdown_connections 2024-09-10T09:57:57.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:57.525+0000 7fde7483e740 1 -- 172.21.15.5:0/3549797420 wait complete. 2024-09-10T09:57:57.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:57.526+0000 7fde7483e740 1 -- start start 2024-09-10T09:57:57.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:57.526+0000 7fde7483e740 1 Processor -- start 2024-09-10T09:57:57.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:57.527+0000 7fde7483e740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5604f5f14d50 0x5604f5f1cee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:57.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:57.527+0000 7fde7483e740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5604f5f15670 0x5604f5f1d420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:57.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:57.527+0000 7fde7483e740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5604f5f1ff30 0x5604f5f1d960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:57.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:57.527+0000 7fde7483e740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5604f5f22bf0 con 0x5604f5f1ff30 2024-09-10T09:57:57.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:57.527+0000 7fde7483e740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x5604f5e360f0 con 0x5604f5f14d50 2024-09-10T09:57:57.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:57.527+0000 7fde7483e740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5604f5f22d70 con 0x5604f5f15670 2024-09-10T09:57:57.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:57.527+0000 7fde7483e740 10 client.0 _ll_register_callbacks cb 0x5604f5e279e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:57:57.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:57.527+0000 7fde6a000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5604f5f15670 0x5604f5f1d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:57.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:57.527+0000 7fde6a000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5604f5f15670 0x5604f5f1d420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:48040/0 (socket says 172.21.15.5:48040) 2024-09-10T09:57:57.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:57.527+0000 7fde6a000640 1 -- 172.21.15.5:0/4197340857 learned_addr learned my addr 172.21.15.5:0/4197340857 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:57:57.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:57.527+0000 7fde6b400640 1 --2- 172.21.15.5:0/4197340857 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5604f5f14d50 0x5604f5f1cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:57.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:57.527+0000 7fde6b400640 1 --2- 172.21.15.5:0/4197340857 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5604f5f1ff30 0x5604f5f1d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:57.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:57.527+0000 7fde6a000640 1 -- 172.21.15.5:0/4197340857 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5604f5f14d50 msgr2=0x5604f5f1cee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:57:57.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:57.527+0000 7fde6a000640 1 --2- 172.21.15.5:0/4197340857 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5604f5f14d50 0x5604f5f1cee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:57.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:57.527+0000 7fde6a000640 1 -- 172.21.15.5:0/4197340857 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5604f5f1ff30 msgr2=0x5604f5f1d960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:57:57.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:57.527+0000 7fde6a000640 1 --2- 172.21.15.5:0/4197340857 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5604f5f1ff30 0x5604f5f1d960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:57.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:57.528+0000 7fde6a000640 1 -- 172.21.15.5:0/4197340857 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5604f5bec3c0 con 0x5604f5f15670 2024-09-10T09:57:57.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:57.528+0000 7fde6a000640 1 --2- 172.21.15.5:0/4197340857 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5604f5f15670 0x5604f5f1d420 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fde58009230 tx=0x7fde580043b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=192f1ede646b234d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:57:57.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:57.528+0000 7fde54c00640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-09-10T09:57:57.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:57.528+0000 7fde54c00640 1 -- 172.21.15.5:0/4197340857 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde58035040 con 0x5604f5f15670 2024-09-10T09:57:57.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:57.528+0000 7fde7483e740 1 -- 172.21.15.5:0/4197340857 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fde64015070 con 0x5604f5f15670 2024-09-10T09:57:57.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:57.528+0000 7fde54c00640 1 -- 172.21.15.5:0/4197340857 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fde58005450 con 0x5604f5f15670 2024-09-10T09:57:57.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:57.529+0000 7fde54c00640 1 -- 172.21.15.5:0/4197340857 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7fde5804d050 con 0x5604f5f15670 2024-09-10T09:57:57.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:57.529+0000 7fde7483e740 10 client.29182 fetch_fsmap learned FSMap version 42 2024-09-10T09:57:57.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:57.529+0000 7fde7483e740 1 -- 172.21.15.5:0/4197340857 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x5604f5f2e9e0 con 0x5604f5f15670 2024-09-10T09:57:57.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:57.529+0000 7fde54c00640 1 -- 172.21.15.5:0/4197340857 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fde64015070 con 0x5604f5f15670 2024-09-10T09:57:57.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:57.529+0000 7fde54c00640 1 -- 172.21.15.5:0/4197340857 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 42) ==== 8370+0+0 (secure 0 0 0) 0x7fde58003820 con 0x5604f5f15670 2024-09-10T09:57:57.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:57.529+0000 7fde7483e740 10 client.29182 fetch_fsmap finished waiting for FSMap version 42 2024-09-10T09:57:57.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:57.529+0000 7fde7483e740 10 client.29182 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T09:57:57.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:57.529+0000 7fde7483e740 20 client.29182 populate_metadata read hostname 'smithi005' 2024-09-10T09:57:57.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:57.529+0000 7fde7483e740 1 --2- 172.21.15.5:0/4197340857 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x5604f5f2fe40 0x5604f5f32260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:57.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:57.529+0000 7fde7483e740 4 client.29182 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:57:57.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:57.529+0000 7fde7483e740 1 -- 172.21.15.5:0/4197340857 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5604f5e88950 con 0x5604f5f2fe40 2024-09-10T09:57:57.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:57.530+0000 7fde6b400640 1 --2- 172.21.15.5:0/4197340857 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x5604f5f2fe40 0x5604f5f32260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:57.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:57.530+0000 7fde6b400640 1 --2- 172.21.15.5:0/4197340857 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x5604f5f2fe40 0x5604f5f32260 crc :-1 s=READY pgs=275 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e81e929dbde0ba42 server_cookie=f660acb130548eab in_seq=0 out_seq=0 2024-09-10T09:57:57.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:57.530+0000 7fde54c00640 10 client.29182 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T09:57:57.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:57.531+0000 7fde54c00640 1 -- 172.21.15.5:0/4197340857 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5604f5e88950 con 0x5604f5f2fe40 2024-09-10T09:57:57.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:57.531+0000 7fde54c00640 10 client.29182 handle_command_reply: tid=0 2024-09-10T09:57:57.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:57.531+0000 7fde7483e740 1 client.29182 shutdown 2024-09-10T09:57:57.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:57.531+0000 7fde7483e740 1 -- 172.21.15.5:0/4197340857 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5604f5f15670 msgr2=0x5604f5f1d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:57:57.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:57.531+0000 7fde7483e740 1 --2- 172.21.15.5:0/4197340857 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5604f5f15670 0x5604f5f1d420 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fde58009230 tx=0x7fde580043b0 comp rx=0 tx=0).stop 2024-09-10T09:57:57.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:57.531+0000 7fde7483e740 1 -- 172.21.15.5:0/4197340857 shutdown_connections 2024-09-10T09:57:57.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:57.531+0000 7fde7483e740 1 --2- 172.21.15.5:0/4197340857 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x5604f5f2fe40 0x5604f5f32260 crc :-1 s=READY pgs=275 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:57.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:57.532+0000 7fde7483e740 1 --2- 172.21.15.5:0/4197340857 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5604f5f1ff30 0x5604f5f1d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:57.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:57.532+0000 7fde54c00640 0 client.29182 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T09:57:57.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:57.532+0000 7fde7483e740 1 --2- 172.21.15.5:0/4197340857 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5604f5f15670 0x5604f5f1d420 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:57.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:57.532+0000 7fde7483e740 1 --2- 172.21.15.5:0/4197340857 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5604f5f14d50 0x5604f5f1cee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:57.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:57.532+0000 7fde7483e740 1 -- 172.21.15.5:0/4197340857 >> 172.21.15.5:0/4197340857 conn(0x5604f5f112e0 msgr2=0x5604f5f16fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:57:57.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:57.532+0000 7fde7483e740 1 -- 172.21.15.5:0/4197340857 shutdown_connections 2024-09-10T09:57:57.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:57.532+0000 7fde7483e740 1 -- 172.21.15.5:0/4197340857 wait complete. 2024-09-10T09:57:57.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:57.532+0000 7fde7483e740 20 client.29182 trim_cache size 0 max 16384 2024-09-10T09:57:57.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:57.552+0000 7fde7483e740 1 Processor -- start 2024-09-10T09:57:57.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:57.553+0000 7fde7483e740 1 -- start start 2024-09-10T09:57:57.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:57.553+0000 7fde7483e740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5604f5f112e0 0x5604f5f1b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:57.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:57.553+0000 7fde7483e740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5604f5f14d50 0x5604f5f1d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:57.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:57.553+0000 7fde7483e740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5604f5f15670 0x5604f5f10290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:57.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:57.553+0000 7fde7483e740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fde6c1b5440 con 0x5604f5f15670 2024-09-10T09:57:57.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:57.553+0000 7fde7483e740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fde6c1b6f80 con 0x5604f5f112e0 2024-09-10T09:57:57.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:57.553+0000 7fde7483e740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fde6c1b5b10 con 0x5604f5f14d50 2024-09-10T09:57:57.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:57.553+0000 7fde6a000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5604f5f14d50 0x5604f5f1d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:57.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:57.553+0000 7fde6b400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5604f5f112e0 0x5604f5f1b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:57.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:57.553+0000 7fde6a000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5604f5f14d50 0x5604f5f1d7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:48048/0 (socket says 172.21.15.5:48048) 2024-09-10T09:57:57.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:57.553+0000 7fde6a000640 1 -- 172.21.15.5:0/2800755358 learned_addr learned my addr 172.21.15.5:0/2800755358 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:57:57.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:57.553+0000 7fde6b400640 1 --2- 172.21.15.5:0/2800755358 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5604f5f15670 0x5604f5f10290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:57.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:57.554+0000 7fde6a000640 1 -- 172.21.15.5:0/2800755358 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5604f5f112e0 msgr2=0x5604f5f1b7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:57:57.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:57.554+0000 7fde6a000640 1 --2- 172.21.15.5:0/2800755358 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5604f5f112e0 0x5604f5f1b7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:57.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:57.554+0000 7fde6a000640 1 -- 172.21.15.5:0/2800755358 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5604f5f15670 msgr2=0x5604f5f10290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:57:57.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:57.554+0000 7fde6a000640 1 --2- 172.21.15.5:0/2800755358 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5604f5f15670 0x5604f5f10290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:57.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:57.554+0000 7fde6a000640 1 -- 172.21.15.5:0/2800755358 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5604f5f14b80 con 0x5604f5f14d50 2024-09-10T09:57:57.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:57.554+0000 7fde6b400640 1 --2- 172.21.15.5:0/2800755358 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5604f5f112e0 0x5604f5f1b7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:57:57.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:57.554+0000 7fde6a000640 1 --2- 172.21.15.5:0/2800755358 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5604f5f14d50 0x5604f5f1d7b0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fde580092a0 tx=0x7fde58009830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=efd7ac32b1796640 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:57:57.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:57.555+0000 7fde57400640 1 -- 172.21.15.5:0/2800755358 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde58035070 con 0x5604f5f14d50 2024-09-10T09:57:57.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:57.555+0000 7fde57400640 1 -- 172.21.15.5:0/2800755358 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fde58003de0 con 0x5604f5f14d50 2024-09-10T09:57:57.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:57.555+0000 7fde57400640 1 -- 172.21.15.5:0/2800755358 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde580039c0 con 0x5604f5f14d50 2024-09-10T09:57:57.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:57.555+0000 7fde7483e740 1 -- 172.21.15.5:0/2800755358 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5604f5f14d50 msgr2=0x5604f5f1d7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:57:57.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:57.555+0000 7fde7483e740 1 --2- 172.21.15.5:0/2800755358 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5604f5f14d50 0x5604f5f1d7b0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fde580092a0 tx=0x7fde58009830 comp rx=0 tx=0).stop 2024-09-10T09:57:57.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:57.556+0000 7fde7483e740 1 -- 172.21.15.5:0/2800755358 shutdown_connections 2024-09-10T09:57:57.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:57.556+0000 7fde7483e740 1 --2- 172.21.15.5:0/2800755358 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5604f5f15670 0x5604f5f10290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:57.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:57.556+0000 7fde7483e740 1 --2- 172.21.15.5:0/2800755358 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5604f5f14d50 0x5604f5f1d7b0 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:57.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:57.556+0000 7fde7483e740 1 --2- 172.21.15.5:0/2800755358 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5604f5f112e0 0x5604f5f1b7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:57.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:57.556+0000 7fde7483e740 1 -- 172.21.15.5:0/2800755358 >> 172.21.15.5:0/2800755358 conn(0x5604f5f11d60 msgr2=0x5604f5f0fb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:57:57.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:57.556+0000 7fde7483e740 1 -- 172.21.15.5:0/2800755358 shutdown_connections 2024-09-10T09:57:57.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:57.556+0000 7fde7483e740 1 -- 172.21.15.5:0/2800755358 wait complete. 2024-09-10T09:57:57.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:57.556+0000 7fde7483e740 1 -- start start 2024-09-10T09:57:57.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:57.557+0000 7fde7483e740 1 Processor -- start 2024-09-10T09:57:57.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:57.557+0000 7fde7483e740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5604f5f112e0 0x5604f5f243e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:57.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:57.557+0000 7fde7483e740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5604f5f14d50 0x5604f5f24920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:57.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:57.557+0000 7fde7483e740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5604f5f15670 0x5604f5f99ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:57.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:57.557+0000 7fde7483e740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fde6c1b68b0 con 0x5604f5f14d50 2024-09-10T09:57:57.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:57.557+0000 7fde7483e740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fde6c1b61e0 con 0x5604f5f15670 2024-09-10T09:57:57.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:57.557+0000 7fde7483e740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fde6c1b3420 con 0x5604f5f112e0 2024-09-10T09:57:57.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:57.557+0000 7fde7483e740 10 client.0 _ll_register_callbacks cb 0x7fde58004040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:57:57.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:57.558+0000 7fde6b400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5604f5f112e0 0x5604f5f243e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:57.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:57.558+0000 7fde6a000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5604f5f14d50 0x5604f5f24920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:57.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:57.558+0000 7fde6b400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5604f5f112e0 0x5604f5f243e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:48056/0 (socket says 172.21.15.5:48056) 2024-09-10T09:57:57.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:57.558+0000 7fde6b400640 1 -- 172.21.15.5:0/3432673978 learned_addr learned my addr 172.21.15.5:0/3432673978 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:57:57.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:57.558+0000 7fde6b400640 1 --2- 172.21.15.5:0/3432673978 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5604f5f15670 0x5604f5f99ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:57.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:57.558+0000 7fde6a000640 1 -- 172.21.15.5:0/3432673978 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5604f5f112e0 msgr2=0x5604f5f243e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:57:57.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:57.558+0000 7fde6a000640 1 --2- 172.21.15.5:0/3432673978 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5604f5f112e0 0x5604f5f243e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:57.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:57.558+0000 7fde6a000640 1 -- 172.21.15.5:0/3432673978 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5604f5f15670 msgr2=0x5604f5f99ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:57:57.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:57.558+0000 7fde6a000640 1 --2- 172.21.15.5:0/3432673978 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5604f5f15670 0x5604f5f99ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:57.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:57.558+0000 7fde6a000640 1 -- 172.21.15.5:0/3432673978 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5604f5f9a020 con 0x5604f5f14d50 2024-09-10T09:57:57.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:57.558+0000 7fde6b400640 1 --2- 172.21.15.5:0/3432673978 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5604f5f112e0 0x5604f5f243e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:57:57.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:57.559+0000 7fde6a000640 1 --2- 172.21.15.5:0/3432673978 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5604f5f14d50 0x5604f5f24920 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x5604f5f22ef0 tx=0x7fde58004560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e9d8357b3bec2f7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:57:57.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:57.559+0000 7fde54c00640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-10T09:57:57.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:57.559+0000 7fde54c00640 1 -- 172.21.15.5:0/3432673978 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde5805f060 con 0x5604f5f14d50 2024-09-10T09:57:57.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:57.559+0000 7fde54c00640 1 -- 172.21.15.5:0/3432673978 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fde580035f0 con 0x5604f5f14d50 2024-09-10T09:57:57.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:57.559+0000 7fde7483e740 1 -- 172.21.15.5:0/3432673978 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fde58003de0 con 0x5604f5f14d50 2024-09-10T09:57:57.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:57.559+0000 7fde54c00640 1 -- 172.21.15.5:0/3432673978 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7fde58066050 con 0x5604f5f14d50 2024-09-10T09:57:57.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:57.559+0000 7fde7483e740 10 client.19140 fetch_fsmap learned FSMap version 42 2024-09-10T09:57:57.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:57.559+0000 7fde7483e740 1 -- 172.21.15.5:0/3432673978 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x5604f5f9b290 con 0x5604f5f14d50 2024-09-10T09:57:57.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:57.560+0000 7fde54c00640 1 -- 172.21.15.5:0/3432673978 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fde58003de0 con 0x5604f5f14d50 2024-09-10T09:57:57.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:57.560+0000 7fde54c00640 1 -- 172.21.15.5:0/3432673978 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 42) ==== 8370+0+0 (secure 0 0 0) 0x7fde58004850 con 0x5604f5f14d50 2024-09-10T09:57:57.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:57.560+0000 7fde7483e740 10 client.19140 fetch_fsmap finished waiting for FSMap version 42 2024-09-10T09:57:57.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:57.560+0000 7fde7483e740 10 client.19140 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T09:57:57.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:57.560+0000 7fde7483e740 20 client.19140 populate_metadata read hostname 'smithi005' 2024-09-10T09:57:57.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:57.560+0000 7fde7483e740 1 --2- 172.21.15.5:0/3432673978 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x5604f5f1ff30 0x5604f5f9d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:57.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:57.560+0000 7fde7483e740 4 client.19140 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T09:57:57.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:57.560+0000 7fde7483e740 1 -- 172.21.15.5:0/3432673978 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5604f5e82e60 con 0x5604f5f1ff30 2024-09-10T09:57:57.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:57.561+0000 7fde6b400640 1 --2- 172.21.15.5:0/3432673978 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x5604f5f1ff30 0x5604f5f9d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:57.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:57.561+0000 7fde6b400640 1 --2- 172.21.15.5:0/3432673978 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x5604f5f1ff30 0x5604f5f9d8b0 crc :-1 s=READY pgs=276 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=241d57e8f4784cb2 server_cookie=1cf867937cceca81 in_seq=0 out_seq=0 2024-09-10T09:57:57.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:57.561+0000 7fde54c00640 10 client.19140 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T09:57:57.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:57.561+0000 7fde54c00640 1 -- 172.21.15.5:0/3432673978 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5604f5e82e60 con 0x5604f5f1ff30 2024-09-10T09:57:57.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:57.561+0000 7fde54c00640 10 client.19140 handle_command_reply: tid=0 2024-09-10T09:57:57.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:57.561+0000 7fde7483e740 1 client.19140 shutdown 2024-09-10T09:57:57.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:57.562+0000 7fde7483e740 1 -- 172.21.15.5:0/3432673978 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5604f5f14d50 msgr2=0x5604f5f24920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:57:57.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:57.562+0000 7fde7483e740 1 --2- 172.21.15.5:0/3432673978 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5604f5f14d50 0x5604f5f24920 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x5604f5f22ef0 tx=0x7fde58004560 comp rx=0 tx=0).stop 2024-09-10T09:57:57.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:57.562+0000 7fde7483e740 1 -- 172.21.15.5:0/3432673978 shutdown_connections 2024-09-10T09:57:57.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:57.562+0000 7fde7483e740 1 --2- 172.21.15.5:0/3432673978 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x5604f5f1ff30 0x5604f5f9d8b0 crc :-1 s=READY pgs=276 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:57.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:57.562+0000 7fde7483e740 1 --2- 172.21.15.5:0/3432673978 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5604f5f15670 0x5604f5f99ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:57.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:57.562+0000 7fde7483e740 1 --2- 172.21.15.5:0/3432673978 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5604f5f14d50 0x5604f5f24920 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:57.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:57.562+0000 7fde7483e740 1 --2- 172.21.15.5:0/3432673978 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5604f5f112e0 0x5604f5f243e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:57.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:57.562+0000 7fde7483e740 1 -- 172.21.15.5:0/3432673978 >> 172.21.15.5:0/3432673978 conn(0x5604f5f11d60 msgr2=0x5604f5f16fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:57:57.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:57.562+0000 7fde54c00640 0 client.19140 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T09:57:57.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:57.562+0000 7fde7483e740 1 -- 172.21.15.5:0/3432673978 shutdown_connections 2024-09-10T09:57:57.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:57.562+0000 7fde7483e740 1 -- 172.21.15.5:0/3432673978 wait complete. 2024-09-10T09:57:57.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:57.562+0000 7fde7483e740 20 client.19140 trim_cache size 0 max 16384 2024-09-10T09:57:57.566 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-10T09:57:57.566 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "no active scrubs running", 2024-09-10T09:57:57.566 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": {} 2024-09-10T09:57:57.566 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-10T09:57:57.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:57.563+0000 7fde6be00640 1 -- 172.21.15.5:0/1272254696 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fde500a2c00 msgr2=0x7fde500a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:57:57.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:57.563+0000 7fde6be00640 1 --2- 172.21.15.5:0/1272254696 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fde500a2c00 0x7fde500a50c0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fde5c09bac0 tx=0x7fde5c0d5530 comp rx=0 tx=0).stop 2024-09-10T09:57:57.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:57.563+0000 7fde6be00640 1 -- 172.21.15.5:0/1272254696 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fde6c1019c0 msgr2=0x7fde6c1a4660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:57:57.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:57.563+0000 7fde6be00640 1 --2- 172.21.15.5:0/1272254696 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fde6c1019c0 0x7fde6c1a4660 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fde5c00e9f0 tx=0x7fde5c00eeb0 comp rx=0 tx=0).stop 2024-09-10T09:57:57.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:57.564+0000 7fde6be00640 1 -- 172.21.15.5:0/1272254696 shutdown_connections 2024-09-10T09:57:57.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:57.564+0000 7fde6be00640 1 --2- 172.21.15.5:0/1272254696 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fde500a2c00 0x7fde500a50c0 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:57.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:57.564+0000 7fde6be00640 1 --2- 172.21.15.5:0/1272254696 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fde6c1079c0 0x7fde6c10b920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:57.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:57.564+0000 7fde6be00640 1 --2- 172.21.15.5:0/1272254696 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fde6c102360 0x7fde6c1a4ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:57.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:57.564+0000 7fde6be00640 1 --2- 172.21.15.5:0/1272254696 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fde6c1019c0 0x7fde6c1a4660 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:57.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:57.564+0000 7fde6be00640 1 -- 172.21.15.5:0/1272254696 >> 172.21.15.5:0/1272254696 conn(0x7fde6c0fd740 msgr2=0x7fde6c10ccc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:57:57.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:57.564+0000 7fde6be00640 1 -- 172.21.15.5:0/1272254696 shutdown_connections 2024-09-10T09:57:57.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:57.564+0000 7fde6be00640 1 -- 172.21.15.5:0/1272254696 wait complete. 2024-09-10T09:57:57.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:57:57 smithi005 ceph-mon[30350]: pgmap v981: 129 pgs: 129 active+clean; 644 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 3.6 MiB/s wr, 459 op/s 2024-09-10T09:57:57.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:57:57 smithi159 ceph-mon[35513]: pgmap v981: 129 pgs: 129 active+clean; 644 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 3.6 MiB/s wr, 459 op/s 2024-09-10T09:57:57.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:57:57 smithi102 ceph-mon[35442]: pgmap v981: 129 pgs: 129 active+clean; 644 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 3.6 MiB/s wr, 459 op/s 2024-09-10T09:57:57.994 INFO:tasks.workunit.client.0.smithi005.stdout: 10 100517 15.33 MB/sec execute 216 sec latency 54.528 ms 2024-09-10T09:57:58.359 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-10T09:57:58.359 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T09:57:58.967 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:57:58.995 INFO:tasks.workunit.client.0.smithi005.stdout: 10 101142 15.41 MB/sec execute 217 sec latency 58.313 ms 2024-09-10T09:57:59.728 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:57:59 smithi005 ceph-mon[30350]: pgmap v982: 129 pgs: 129 active+clean; 634 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 3.6 MiB/s wr, 444 op/s 2024-09-10T09:57:59.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:59.792+0000 7f6dd2800640 1 -- 172.21.15.5:0/3580780588 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6dcc077040 msgr2=0x7f6dcc112510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:57:59.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:59.792+0000 7f6dd2800640 1 --2- 172.21.15.5:0/3580780588 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6dcc077040 0x7f6dcc112510 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f6db4009900 tx=0x7f6db402a3f0 comp rx=0 tx=0).stop 2024-09-10T09:57:59.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:59.793+0000 7f6dd2800640 1 -- 172.21.15.5:0/3580780588 shutdown_connections 2024-09-10T09:57:59.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:59.793+0000 7f6dd2800640 1 --2- 172.21.15.5:0/3580780588 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6dcc112a50 0x7f6dcc114e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:59.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:59.793+0000 7f6dd2800640 1 --2- 172.21.15.5:0/3580780588 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6dcc077040 0x7f6dcc112510 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:59.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:59.793+0000 7f6dd2800640 1 --2- 172.21.15.5:0/3580780588 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6dcc076720 0x7f6dcc076b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:59.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:59.793+0000 7f6dd2800640 1 -- 172.21.15.5:0/3580780588 >> 172.21.15.5:0/3580780588 conn(0x7f6dcc0ffa50 msgr2=0x7f6dcc101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:57:59.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:59.793+0000 7f6dd2800640 1 -- 172.21.15.5:0/3580780588 shutdown_connections 2024-09-10T09:57:59.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:59.793+0000 7f6dd2800640 1 -- 172.21.15.5:0/3580780588 wait complete. 2024-09-10T09:57:59.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:59.794+0000 7f6dd2800640 1 Processor -- start 2024-09-10T09:57:59.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:59.794+0000 7f6dd2800640 1 -- start start 2024-09-10T09:57:59.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:59.795+0000 7f6dd2800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6dcc076720 0x7f6dcc1a4730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:59.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:59.795+0000 7f6dd2800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6dcc077040 0x7f6dcc1a4c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:59.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:59.795+0000 7f6dd2800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6dcc112a50 0x7f6dcc19e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:59.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:59.795+0000 7f6dd2800640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f6dcc118610 con 0x7f6dcc112a50 2024-09-10T09:57:59.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:59.795+0000 7f6dd2800640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f6dcc118490 con 0x7f6dcc077040 2024-09-10T09:57:59.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:59.795+0000 7f6dd2800640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f6dcc118790 con 0x7f6dcc076720 2024-09-10T09:57:59.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:59.796+0000 7f6dcaa00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6dcc077040 0x7f6dcc1a4c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:59.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:59.796+0000 7f6dcaa00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6dcc077040 0x7f6dcc1a4c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:54352/0 (socket says 172.21.15.5:54352) 2024-09-10T09:57:59.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:59.796+0000 7f6dcaa00640 1 -- 172.21.15.5:0/3470813056 learned_addr learned my addr 172.21.15.5:0/3470813056 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:57:59.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:59.796+0000 7f6dcb400640 1 --2- 172.21.15.5:0/3470813056 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6dcc076720 0x7f6dcc1a4730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:59.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:59.796+0000 7f6dcbe00640 1 --2- 172.21.15.5:0/3470813056 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6dcc112a50 0x7f6dcc19e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:59.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:59.796+0000 7f6dcaa00640 1 -- 172.21.15.5:0/3470813056 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6dcc076720 msgr2=0x7f6dcc1a4730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:57:59.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:59.796+0000 7f6dcaa00640 1 --2- 172.21.15.5:0/3470813056 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6dcc076720 0x7f6dcc1a4730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:59.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:59.796+0000 7f6dcaa00640 1 -- 172.21.15.5:0/3470813056 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6dcc112a50 msgr2=0x7f6dcc19e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:57:59.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:59.796+0000 7f6dcaa00640 1 --2- 172.21.15.5:0/3470813056 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6dcc112a50 0x7f6dcc19e8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:57:59.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:59.796+0000 7f6dcaa00640 1 -- 172.21.15.5:0/3470813056 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6dcc19f020 con 0x7f6dcc077040 2024-09-10T09:57:59.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:59.797+0000 7f6dcaa00640 1 --2- 172.21.15.5:0/3470813056 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6dcc077040 0x7f6dcc1a4c70 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f6db402a960 tx=0x7f6db4003b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:57:59.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:59.797+0000 7f6dc3e00640 1 -- 172.21.15.5:0/3470813056 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6db403f070 con 0x7f6dcc077040 2024-09-10T09:57:59.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:59.797+0000 7f6dc3e00640 1 -- 172.21.15.5:0/3470813056 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6db4031c90 con 0x7f6dcc077040 2024-09-10T09:57:59.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:59.797+0000 7f6dc3e00640 1 -- 172.21.15.5:0/3470813056 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6db4035030 con 0x7f6dcc077040 2024-09-10T09:57:59.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:59.797+0000 7f6dd2800640 1 -- 172.21.15.5:0/3470813056 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6dcc19f2b0 con 0x7f6dcc077040 2024-09-10T09:57:59.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:59.798+0000 7f6dd2800640 1 -- 172.21.15.5:0/3470813056 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6dcc1ab600 con 0x7f6dcc077040 2024-09-10T09:57:59.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:59.799+0000 7f6dd2800640 1 -- 172.21.15.5:0/3470813056 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6dcc076b00 con 0x7f6dcc077040 2024-09-10T09:57:59.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:59.802+0000 7f6dc3e00640 1 -- 172.21.15.5:0/3470813056 <== mon.2 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f6db4042070 con 0x7f6dcc077040 2024-09-10T09:57:59.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:59.803+0000 7f6dc3e00640 1 --2- 172.21.15.5:0/3470813056 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f6db00a2cc0 0x7f6db00a5180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:57:59.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:59.803+0000 7f6dc3e00640 1 -- 172.21.15.5:0/3470813056 <== mon.2 v2:172.21.15.102:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f6db40ef810 con 0x7f6dcc077040 2024-09-10T09:57:59.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:59.803+0000 7f6dcb400640 1 --2- 172.21.15.5:0/3470813056 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f6db00a2cc0 0x7f6db00a5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:57:59.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:59.805+0000 7f6dcb400640 1 --2- 172.21.15.5:0/3470813056 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f6db00a2cc0 0x7f6db00a5180 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f6dcc19fa30 tx=0x7f6dbc00b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:57:59.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:57:59.808+0000 7f6dc3e00640 1 -- 172.21.15.5:0/3470813056 <== mon.2 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f6db40bbcd0 con 0x7f6dcc077040 2024-09-10T09:57:59.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:57:59 smithi159 ceph-mon[35513]: pgmap v982: 129 pgs: 129 active+clean; 634 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 3.6 MiB/s wr, 444 op/s 2024-09-10T09:57:59.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:57:59 smithi102 ceph-mon[35442]: pgmap v982: 129 pgs: 129 active+clean; 634 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 3.6 MiB/s wr, 444 op/s 2024-09-10T09:57:59.996 INFO:tasks.workunit.client.0.smithi005.stdout: 10 101973 15.54 MB/sec execute 218 sec latency 63.684 ms 2024-09-10T09:58:00.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:00.185+0000 7f6dd2800640 1 -- 172.21.15.5:0/3470813056 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6dcc1ab830 con 0x7f6dcc077040 2024-09-10T09:58:00.187 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-10T09:58:00.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:00.186+0000 7f6dc3e00640 1 -- 172.21.15.5:0/3470813056 <== mon.2 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 42 v42) ==== 94+0+11969 (secure 0 0 0) 0x7f6db40bffd0 con 0x7f6dcc077040 2024-09-10T09:58:00.188 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 42 2024-09-10T09:58:00.188 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":42,"btime":"2024-09-10T09:55:06:670916+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14454,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/2970594311","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2970594311},{"type":"v1","addr":"172.21.15.5:6835","nonce":2970594311}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14514,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6839/3616502174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":3616502174},{"type":"v1","addr":"172.21.15.5:6839","nonce":3616502174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":42,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T09:35:29.626287+0000","modified":"2024-09-10T09:55:06.419829+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24542,"mds_1":24535,"mds_2":24511,"mds_3":24506,"mds_4":24443},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14520":{"gid":14520,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/2073599842","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":2073599842},{"type":"v1","addr":"172.21.15.5:6841","nonce":2073599842}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"k","rank":4,"incarnation":26,"state":"up:active","state_seq":24,"addr":"172.21.15.102:6833/2457007427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":2457007427},{"type":"v1","addr":"172.21.15.102:6833","nonce":2457007427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/356445159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":356445159},{"type":"v1","addr":"172.21.15.159:6833","nonce":356445159}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6837/2894314507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":2894314507},{"type":"v1","addr":"172.21.15.5:6837","nonce":2894314507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.102:6835/1647549111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":1647549111},{"type":"v1","addr":"172.21.15.102:6835","nonce":1647549111}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6835/952657242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":952657242},{"type":"v1","addr":"172.21.15.159:6835","nonce":952657242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":19,"addr":"172.21.15.102:6837/2689776764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":2689776764},{"type":"v1","addr":"172.21.15.102:6837","nonce":2689776764}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.159:6837/620627639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":620627639},{"type":"v1","addr":"172.21.15.159:6837","nonce":620627639}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24535":{"gid":24535,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":18,"addr":"172.21.15.159:6839/2073739599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":2073739599},{"type":"v1","addr":"172.21.15.159:6839","nonce":2073739599}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24542":{"gid":24542,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.102:6839/335409749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6838","nonce":335409749},{"type":"v1","addr":"172.21.15.102:6839","nonce":335409749}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24542,"qdb_cluster":[24542,24535,24511,24506,24443]},"id":1}]} 2024-09-10T09:58:00.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:00.190+0000 7f6dd2800640 1 -- 172.21.15.5:0/3470813056 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f6db00a2cc0 msgr2=0x7f6db00a5180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:00.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:00.190+0000 7f6dd2800640 1 --2- 172.21.15.5:0/3470813056 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f6db00a2cc0 0x7f6db00a5180 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f6dcc19fa30 tx=0x7f6dbc00b410 comp rx=0 tx=0).stop 2024-09-10T09:58:00.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:00.190+0000 7f6dd2800640 1 -- 172.21.15.5:0/3470813056 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6dcc077040 msgr2=0x7f6dcc1a4c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:00.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:00.190+0000 7f6dd2800640 1 --2- 172.21.15.5:0/3470813056 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6dcc077040 0x7f6dcc1a4c70 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f6db402a960 tx=0x7f6db4003b90 comp rx=0 tx=0).stop 2024-09-10T09:58:00.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:00.191+0000 7f6dd2800640 1 -- 172.21.15.5:0/3470813056 shutdown_connections 2024-09-10T09:58:00.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:00.191+0000 7f6dd2800640 1 --2- 172.21.15.5:0/3470813056 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f6db00a2cc0 0x7f6db00a5180 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:00.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:00.191+0000 7f6dd2800640 1 --2- 172.21.15.5:0/3470813056 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6dcc112a50 0x7f6dcc19e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:00.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:00.191+0000 7f6dd2800640 1 --2- 172.21.15.5:0/3470813056 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6dcc077040 0x7f6dcc1a4c70 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:00.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:00.191+0000 7f6dd2800640 1 --2- 172.21.15.5:0/3470813056 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6dcc076720 0x7f6dcc1a4730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:00.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:00.191+0000 7f6dd2800640 1 -- 172.21.15.5:0/3470813056 >> 172.21.15.5:0/3470813056 conn(0x7f6dcc0ffa50 msgr2=0x7f6dcc101e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:58:00.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:00.192+0000 7f6dd2800640 1 -- 172.21.15.5:0/3470813056 shutdown_connections 2024-09-10T09:58:00.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:00.192+0000 7f6dd2800640 1 -- 172.21.15.5:0/3470813056 wait complete. 2024-09-10T09:58:00.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:58:00 smithi005 ceph-mon[30350]: from='client.? 172.21.15.5:0/3470813056' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:58:00.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:58:00 smithi159 ceph-mon[35513]: from='client.? 172.21.15.5:0/3470813056' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:58:00.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:58:00 smithi102 ceph-mon[35442]: from='client.? 172.21.15.5:0/3470813056' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:58:01.010 INFO:tasks.workunit.client.0.smithi005.stdout: 10 102589 15.68 MB/sec execute 219 sec latency 72.635 ms 2024-09-10T09:58:01.524 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:58:01 smithi005 ceph-mon[30350]: pgmap v983: 129 pgs: 129 active+clean; 647 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 4.3 MiB/s wr, 551 op/s 2024-09-10T09:58:01.529 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-10T09:58:01.785 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:58:01 smithi102 ceph-mon[35442]: pgmap v983: 129 pgs: 129 active+clean; 647 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 4.3 MiB/s wr, 551 op/s 2024-09-10T09:58:01.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:58:01 smithi159 ceph-mon[35513]: pgmap v983: 129 pgs: 129 active+clean; 647 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 4.3 MiB/s wr, 551 op/s 2024-09-10T09:58:01.984 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:58:02.011 INFO:tasks.workunit.client.0.smithi005.stdout: 10 102671 15.64 MB/sec execute 220 sec latency 984.278 ms 2024-09-10T09:58:02.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:02.629+0000 7f720ee00640 1 -- 172.21.15.5:0/995895568 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f720810e280 msgr2=0x7f7208110670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:02.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:02.629+0000 7f720ee00640 1 --2- 172.21.15.5:0/995895568 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f720810e280 0x7f7208110670 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f7200009ca0 tx=0x7f720002a950 comp rx=0 tx=0).stop 2024-09-10T09:58:02.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:02.629+0000 7f720ee00640 1 -- 172.21.15.5:0/995895568 shutdown_connections 2024-09-10T09:58:02.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:02.629+0000 7f720ee00640 1 --2- 172.21.15.5:0/995895568 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f720810e280 0x7f7208110670 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:02.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:02.629+0000 7f720ee00640 1 --2- 172.21.15.5:0/995895568 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7208074e30 0x7f720810dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:02.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:02.629+0000 7f720ee00640 1 --2- 172.21.15.5:0/995895568 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7208074510 0x7f72080748f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:02.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:02.629+0000 7f720ee00640 1 -- 172.21.15.5:0/995895568 >> 172.21.15.5:0/995895568 conn(0x7f72080fd740 msgr2=0x7f72080ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:58:02.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:02.629+0000 7f720ee00640 1 -- 172.21.15.5:0/995895568 shutdown_connections 2024-09-10T09:58:02.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:02.629+0000 7f720ee00640 1 -- 172.21.15.5:0/995895568 wait complete. 2024-09-10T09:58:02.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:02.630+0000 7f720ee00640 1 Processor -- start 2024-09-10T09:58:02.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:02.631+0000 7f720ee00640 1 -- start start 2024-09-10T09:58:02.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:02.631+0000 7f720ee00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7208074510 0x7f720819a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:02.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:02.631+0000 7f720ee00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7208074e30 0x7f720819a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:02.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:02.631+0000 7f720ee00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f720810e280 0x7f720819ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:02.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:02.631+0000 7f720ee00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f7208113fc0 con 0x7f7208074e30 2024-09-10T09:58:02.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:02.631+0000 7f720ee00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f7208113e40 con 0x7f7208074510 2024-09-10T09:58:02.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:02.631+0000 7f720ee00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f7208114140 con 0x7f720810e280 2024-09-10T09:58:02.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:02.632+0000 7f720da00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7208074510 0x7f720819a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:02.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:02.632+0000 7f720e400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f720810e280 0x7f720819ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:02.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:02.632+0000 7f720e400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f720810e280 0x7f720819ed50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:40644/0 (socket says 172.21.15.5:40644) 2024-09-10T09:58:02.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:02.632+0000 7f720e400640 1 -- 172.21.15.5:0/2046060397 learned_addr learned my addr 172.21.15.5:0/2046060397 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:58:02.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:02.632+0000 7f720da00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7208074510 0x7f720819a480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:56518/0 (socket says 172.21.15.5:56518) 2024-09-10T09:58:02.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:02.632+0000 7f720d000640 1 --2- 172.21.15.5:0/2046060397 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7208074e30 0x7f720819a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:02.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:02.632+0000 7f720e400640 1 -- 172.21.15.5:0/2046060397 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7208074510 msgr2=0x7f720819a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:02.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:02.632+0000 7f720e400640 1 --2- 172.21.15.5:0/2046060397 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7208074510 0x7f720819a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:02.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:02.632+0000 7f720e400640 1 -- 172.21.15.5:0/2046060397 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7208074e30 msgr2=0x7f720819a9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:02.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:02.632+0000 7f720e400640 1 --2- 172.21.15.5:0/2046060397 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7208074e30 0x7f720819a9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:02.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:02.632+0000 7f720e400640 1 -- 172.21.15.5:0/2046060397 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f720819f430 con 0x7f720810e280 2024-09-10T09:58:02.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:02.632+0000 7f720d000640 1 --2- 172.21.15.5:0/2046060397 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7208074e30 0x7f720819a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:58:02.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:02.633+0000 7f720e400640 1 --2- 172.21.15.5:0/2046060397 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f720810e280 0x7f720819ed50 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f7200005e30 tx=0x7f7200003b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:02.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:02.633+0000 7f7206000640 1 -- 172.21.15.5:0/2046060397 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7200034040 con 0x7f720810e280 2024-09-10T09:58:02.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:02.633+0000 7f7206000640 1 -- 172.21.15.5:0/2046060397 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f720002f940 con 0x7f720810e280 2024-09-10T09:58:02.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:02.633+0000 7f720ee00640 1 -- 172.21.15.5:0/2046060397 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f720819f6c0 con 0x7f720810e280 2024-09-10T09:58:02.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:02.634+0000 7f7206000640 1 -- 172.21.15.5:0/2046060397 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72000045f0 con 0x7f720810e280 2024-09-10T09:58:02.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:02.634+0000 7f720ee00640 1 -- 172.21.15.5:0/2046060397 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72081a7000 con 0x7f720810e280 2024-09-10T09:58:02.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:02.638+0000 7f7206000640 1 -- 172.21.15.5:0/2046060397 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f7200037020 con 0x7f720810e280 2024-09-10T09:58:02.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:02.638+0000 7f7213043740 1 Processor -- start 2024-09-10T09:58:02.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:02.639+0000 7f7213043740 1 -- start start 2024-09-10T09:58:02.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:02.639+0000 7f7213043740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bf8046ee40 0x55bf8046f220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:02.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:02.639+0000 7f7213043740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bf8046f760 0x55bf80479ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:02.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:02.639+0000 7f7213043740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bf8047a020 0x55bf8047c410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:02.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:02.639+0000 7f7213043740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55bf803d53e0 con 0x55bf8046f760 2024-09-10T09:58:02.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:02.639+0000 7f7213043740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55bf8037cd80 con 0x55bf8046ee40 2024-09-10T09:58:02.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:02.639+0000 7f7213043740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55bf803d13a0 con 0x55bf8047a020 2024-09-10T09:58:02.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:02.640+0000 7f720d000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bf8047a020 0x55bf8047c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:02.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:02.640+0000 7f720da00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bf8046f760 0x55bf80479ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:02.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:02.640+0000 7f720da00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bf8046f760 0x55bf80479ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42126/0 (socket says 172.21.15.5:42126) 2024-09-10T09:58:02.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:02.640+0000 7f720da00640 1 -- 172.21.15.5:0/1041727618 learned_addr learned my addr 172.21.15.5:0/1041727618 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:58:02.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:02.640+0000 7f720d000640 1 --2- 172.21.15.5:0/1041727618 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bf8046ee40 0x55bf8046f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:02.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:02.641+0000 7f7206000640 1 --2- 172.21.15.5:0/2046060397 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f71f00a2d50 0x7f71f00a5210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:02.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:02.641+0000 7f7206000640 1 -- 172.21.15.5:0/2046060397 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f72000f01b0 con 0x7f720810e280 2024-09-10T09:58:02.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:02.641+0000 7f720da00640 1 -- 172.21.15.5:0/1041727618 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bf8047a020 msgr2=0x55bf8047c410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:58:02.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:02.641+0000 7f720da00640 1 --2- 172.21.15.5:0/1041727618 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bf8047a020 0x55bf8047c410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:02.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:02.641+0000 7f720da00640 1 -- 172.21.15.5:0/1041727618 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bf8046ee40 msgr2=0x55bf8046f220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:58:02.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:02.641+0000 7f720da00640 1 --2- 172.21.15.5:0/1041727618 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bf8046ee40 0x55bf8046f220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:02.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:02.641+0000 7f720da00640 1 -- 172.21.15.5:0/1041727618 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bf8039bae0 con 0x55bf8046f760 2024-09-10T09:58:02.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:02.641+0000 7f720da00640 1 --2- 172.21.15.5:0/1041727618 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bf8046f760 0x55bf80479ae0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f71fc009cf0 tx=0x7f71fc011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8112f19d011f7e53 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:02.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:02.642+0000 7f71faa00640 1 -- 172.21.15.5:0/1041727618 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71fc004230 con 0x55bf8046f760 2024-09-10T09:58:02.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:02.642+0000 7f71faa00640 1 -- 172.21.15.5:0/1041727618 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f71fc0043d0 con 0x55bf8046f760 2024-09-10T09:58:02.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:02.642+0000 7f71faa00640 1 -- 172.21.15.5:0/1041727618 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71fc019840 con 0x55bf8046f760 2024-09-10T09:58:02.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:02.643+0000 7f720e400640 1 --2- 172.21.15.5:0/2046060397 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f71f00a2d50 0x7f71f00a5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:02.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:02.643+0000 7f7213043740 1 -- 172.21.15.5:0/1041727618 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bf8046f760 msgr2=0x55bf80479ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:02.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:02.643+0000 7f7213043740 1 --2- 172.21.15.5:0/1041727618 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bf8046f760 0x55bf80479ae0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f71fc009cf0 tx=0x7f71fc011810 comp rx=0 tx=0).stop 2024-09-10T09:58:02.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:02.643+0000 7f7213043740 1 -- 172.21.15.5:0/1041727618 shutdown_connections 2024-09-10T09:58:02.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:02.643+0000 7f7213043740 1 --2- 172.21.15.5:0/1041727618 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bf8047a020 0x55bf8047c410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:02.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:02.643+0000 7f7213043740 1 --2- 172.21.15.5:0/1041727618 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bf8046f760 0x55bf80479ae0 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:02.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:02.643+0000 7f7213043740 1 --2- 172.21.15.5:0/1041727618 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bf8046ee40 0x55bf8046f220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:02.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:02.643+0000 7f7213043740 1 -- 172.21.15.5:0/1041727618 >> 172.21.15.5:0/1041727618 conn(0x55bf8046b3d0 msgr2=0x55bf8046b7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:58:02.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:02.644+0000 7f720e400640 1 --2- 172.21.15.5:0/2046060397 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f71f00a2d50 0x7f71f00a5210 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f72000b8a20 tx=0x7f720003c040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:02.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:02.644+0000 7f7213043740 1 -- 172.21.15.5:0/1041727618 shutdown_connections 2024-09-10T09:58:02.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:02.645+0000 7f7213043740 1 -- 172.21.15.5:0/1041727618 wait complete. 2024-09-10T09:58:02.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:02.645+0000 7f7213043740 1 -- start start 2024-09-10T09:58:02.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:02.646+0000 7f7213043740 1 Processor -- start 2024-09-10T09:58:02.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:02.647+0000 7f7213043740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bf8046ee40 0x55bf80476f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:02.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:02.647+0000 7f7213043740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bf8046f760 0x55bf804774c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:02.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:02.647+0000 7f7213043740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bf8047a020 0x55bf80477a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:02.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:02.647+0000 7f7213043740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55bf8048dc00 con 0x55bf8046f760 2024-09-10T09:58:02.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:02.647+0000 7f7213043740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55bf803900f0 con 0x55bf8046ee40 2024-09-10T09:58:02.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:02.647+0000 7f7213043740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55bf8048dd80 con 0x55bf8047a020 2024-09-10T09:58:02.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:02.647+0000 7f7213043740 10 client.0 _ll_register_callbacks cb 0x55bf80377bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:58:02.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:02.647+0000 7f720d000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bf8046ee40 0x55bf80476f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:02.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:02.647+0000 7f720d000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bf8046ee40 0x55bf80476f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:56542/0 (socket says 172.21.15.5:56542) 2024-09-10T09:58:02.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:02.647+0000 7f720d000640 1 -- 172.21.15.5:0/2371411785 learned_addr learned my addr 172.21.15.5:0/2371411785 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:58:02.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:02.647+0000 7f720d000640 1 --2- 172.21.15.5:0/2371411785 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bf8047a020 0x55bf80477a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:02.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:02.648+0000 7f720da00640 1 --2- 172.21.15.5:0/2371411785 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bf8046f760 0x55bf804774c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:02.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:02.648+0000 7f720da00640 1 -- 172.21.15.5:0/2371411785 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bf8047a020 msgr2=0x55bf80477a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:58:02.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:02.648+0000 7f720da00640 1 --2- 172.21.15.5:0/2371411785 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bf8047a020 0x55bf80477a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:02.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:02.648+0000 7f720da00640 1 -- 172.21.15.5:0/2371411785 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bf8046ee40 msgr2=0x55bf80476f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:58:02.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:02.648+0000 7f720da00640 1 --2- 172.21.15.5:0/2371411785 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bf8046ee40 0x55bf80476f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:02.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:02.648+0000 7f720da00640 1 -- 172.21.15.5:0/2371411785 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bf801463c0 con 0x55bf8046f760 2024-09-10T09:58:02.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:02.649+0000 7f720da00640 1 --2- 172.21.15.5:0/2371411785 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bf8046f760 0x55bf804774c0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f71fc009cf0 tx=0x7f71fc003820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e27c920836c008c4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:02.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:02.649+0000 7f71ef400640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-10T09:58:02.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:02.649+0000 7f71ef400640 1 -- 172.21.15.5:0/2371411785 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71fc027070 con 0x55bf8046f760 2024-09-10T09:58:02.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:02.649+0000 7f71ef400640 1 -- 172.21.15.5:0/2371411785 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f71fc005240 con 0x55bf8046f760 2024-09-10T09:58:02.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:02.650+0000 7f7213043740 1 -- 172.21.15.5:0/2371411785 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f71fc0043d0 con 0x55bf8046f760 2024-09-10T09:58:02.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:02.650+0000 7f71ef400640 1 -- 172.21.15.5:0/2371411785 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f71fc033050 con 0x55bf8046f760 2024-09-10T09:58:02.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:02.650+0000 7f7213043740 10 client.19158 fetch_fsmap learned FSMap version 42 2024-09-10T09:58:02.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:02.650+0000 7f7213043740 1 -- 172.21.15.5:0/2371411785 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55bf804a2500 con 0x55bf8046f760 2024-09-10T09:58:02.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:02.650+0000 7f71ef400640 1 -- 172.21.15.5:0/2371411785 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f71fc0043d0 con 0x55bf8046f760 2024-09-10T09:58:02.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:02.651+0000 7f71ef400640 1 -- 172.21.15.5:0/2371411785 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 42) ==== 8370+0+0 (secure 0 0 0) 0x7f71fc004ba0 con 0x55bf8046f760 2024-09-10T09:58:02.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:02.651+0000 7f7213043740 10 client.19158 fetch_fsmap finished waiting for FSMap version 42 2024-09-10T09:58:02.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:02.651+0000 7f7213043740 10 client.19158 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-09-10T09:58:02.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:02.651+0000 7f7213043740 20 client.19158 populate_metadata read hostname 'smithi005' 2024-09-10T09:58:02.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:02.651+0000 7f7213043740 1 --2- 172.21.15.5:0/2371411785 >> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] conn(0x55bf804a38b0 0x55bf804a5ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:02.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:02.651+0000 7f7213043740 4 client.19158 mds_command: new command op to 24443 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:58:02.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:02.651+0000 7f7213043740 1 -- 172.21.15.5:0/2371411785 --> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bf803e2950 con 0x55bf804a38b0 2024-09-10T09:58:02.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:02.651+0000 7f720d000640 1 --2- 172.21.15.5:0/2371411785 >> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] conn(0x55bf804a38b0 0x55bf804a5ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:02.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:02.652+0000 7f720d000640 1 --2- 172.21.15.5:0/2371411785 >> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] conn(0x55bf804a38b0 0x55bf804a5ca0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=fd1307cf64834635 server_cookie=d98c42cc28bd6441 in_seq=0 out_seq=0 2024-09-10T09:58:02.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:02.652+0000 7f71ef400640 10 client.19158 ms_handle_connect on v2:172.21.15.102:6832/2457007427 2024-09-10T09:58:02.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:02.652+0000 7f71ef400640 1 -- 172.21.15.5:0/2371411785 <== mds.4 v2:172.21.15.102:6832/2457007427 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55bf803e2950 con 0x55bf804a38b0 2024-09-10T09:58:02.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:02.652+0000 7f71ef400640 10 client.19158 handle_command_reply: tid=0 2024-09-10T09:58:02.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:02.652+0000 7f7213043740 1 client.19158 shutdown 2024-09-10T09:58:02.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:02.653+0000 7f7213043740 1 -- 172.21.15.5:0/2371411785 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bf8046f760 msgr2=0x55bf804774c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:02.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:02.653+0000 7f7213043740 1 --2- 172.21.15.5:0/2371411785 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bf8046f760 0x55bf804774c0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f71fc009cf0 tx=0x7f71fc003820 comp rx=0 tx=0).stop 2024-09-10T09:58:02.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:02.653+0000 7f7213043740 1 -- 172.21.15.5:0/2371411785 shutdown_connections 2024-09-10T09:58:02.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:02.653+0000 7f7213043740 1 --2- 172.21.15.5:0/2371411785 >> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] conn(0x55bf804a38b0 0x55bf804a5ca0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:02.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:02.653+0000 7f7213043740 1 --2- 172.21.15.5:0/2371411785 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bf8047a020 0x55bf80477a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:02.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:02.653+0000 7f7213043740 1 --2- 172.21.15.5:0/2371411785 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bf8046f760 0x55bf804774c0 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:02.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:02.653+0000 7f7213043740 1 --2- 172.21.15.5:0/2371411785 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bf8046ee40 0x55bf80476f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:02.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:02.653+0000 7f71ef400640 0 client.19158 ms_handle_reset on v2:172.21.15.102:6832/2457007427 2024-09-10T09:58:02.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:02.653+0000 7f7213043740 1 -- 172.21.15.5:0/2371411785 >> 172.21.15.5:0/2371411785 conn(0x55bf8046b3d0 msgr2=0x55bf80471100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:58:02.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:02.653+0000 7f7213043740 1 -- 172.21.15.5:0/2371411785 shutdown_connections 2024-09-10T09:58:02.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:02.653+0000 7f7213043740 1 -- 172.21.15.5:0/2371411785 wait complete. 2024-09-10T09:58:02.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:02.653+0000 7f7213043740 20 client.19158 trim_cache size 0 max 16384 2024-09-10T09:58:02.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:02.671+0000 7f7213043740 1 Processor -- start 2024-09-10T09:58:02.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:02.672+0000 7f7213043740 1 -- start start 2024-09-10T09:58:02.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:02.672+0000 7f7213043740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bf8046ee40 0x55bf80477850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:02.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:02.672+0000 7f7213043740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bf8046f760 0x55bf804ab3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:02.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:02.672+0000 7f7213043740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bf8047a020 0x55bf804ab920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:02.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:02.672+0000 7f7213043740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f7208071460 con 0x55bf8047a020 2024-09-10T09:58:02.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:02.672+0000 7f7213043740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f7208072fa0 con 0x55bf8046ee40 2024-09-10T09:58:02.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:02.672+0000 7f7213043740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f7208071b30 con 0x55bf8046f760 2024-09-10T09:58:02.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:02.672+0000 7f720da00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bf8046f760 0x55bf804ab3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:02.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:02.672+0000 7f720d000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bf8046ee40 0x55bf80477850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:02.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:02.672+0000 7f720da00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bf8046f760 0x55bf804ab3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:40660/0 (socket says 172.21.15.5:40660) 2024-09-10T09:58:02.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:02.672+0000 7f720da00640 1 -- 172.21.15.5:0/2852699954 learned_addr learned my addr 172.21.15.5:0/2852699954 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:58:02.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:02.672+0000 7f720d000640 1 --2- 172.21.15.5:0/2852699954 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bf8047a020 0x55bf804ab920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:02.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:02.673+0000 7f720da00640 1 -- 172.21.15.5:0/2852699954 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bf8046ee40 msgr2=0x55bf80477850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:58:02.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:02.673+0000 7f720da00640 1 --2- 172.21.15.5:0/2852699954 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bf8046ee40 0x55bf80477850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:02.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:02.673+0000 7f720da00640 1 -- 172.21.15.5:0/2852699954 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bf8047a020 msgr2=0x55bf804ab920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:58:02.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:02.673+0000 7f720da00640 1 --2- 172.21.15.5:0/2852699954 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bf8047a020 0x55bf804ab920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:02.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:02.673+0000 7f720da00640 1 -- 172.21.15.5:0/2852699954 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bf804798f0 con 0x55bf8046f760 2024-09-10T09:58:02.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:02.673+0000 7f720d000640 1 --2- 172.21.15.5:0/2852699954 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bf8046ee40 0x55bf80477850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:58:02.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:02.673+0000 7f720da00640 1 --2- 172.21.15.5:0/2852699954 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bf8046f760 0x55bf804ab3e0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f71fc011850 tx=0x7f71fc005970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fc982b096206227b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:02.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:02.673+0000 7f71fa000640 1 -- 172.21.15.5:0/2852699954 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71fc030020 con 0x55bf8046f760 2024-09-10T09:58:02.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:02.673+0000 7f71fa000640 1 -- 172.21.15.5:0/2852699954 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f71fc011d10 con 0x55bf8046f760 2024-09-10T09:58:02.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:02.673+0000 7f71fa000640 1 -- 172.21.15.5:0/2852699954 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71fc027070 con 0x55bf8046f760 2024-09-10T09:58:02.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:02.674+0000 7f7213043740 1 -- 172.21.15.5:0/2852699954 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bf8046f760 msgr2=0x55bf804ab3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:02.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:02.674+0000 7f7213043740 1 --2- 172.21.15.5:0/2852699954 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bf8046f760 0x55bf804ab3e0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f71fc011850 tx=0x7f71fc005970 comp rx=0 tx=0).stop 2024-09-10T09:58:02.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:02.674+0000 7f7213043740 1 -- 172.21.15.5:0/2852699954 shutdown_connections 2024-09-10T09:58:02.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:02.674+0000 7f7213043740 1 --2- 172.21.15.5:0/2852699954 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bf8047a020 0x55bf804ab920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:02.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:02.674+0000 7f7213043740 1 --2- 172.21.15.5:0/2852699954 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bf8046f760 0x55bf804ab3e0 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:02.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:02.674+0000 7f7213043740 1 --2- 172.21.15.5:0/2852699954 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bf8046ee40 0x55bf80477850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:02.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:02.674+0000 7f7213043740 1 -- 172.21.15.5:0/2852699954 >> 172.21.15.5:0/2852699954 conn(0x55bf8046b3d0 msgr2=0x55bf80471100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:58:02.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:02.674+0000 7f7213043740 1 -- 172.21.15.5:0/2852699954 shutdown_connections 2024-09-10T09:58:02.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:02.675+0000 7f7213043740 1 -- 172.21.15.5:0/2852699954 wait complete. 2024-09-10T09:58:02.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:02.675+0000 7f7213043740 1 -- start start 2024-09-10T09:58:02.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:02.675+0000 7f7213043740 1 Processor -- start 2024-09-10T09:58:02.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:02.676+0000 7f7213043740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bf8046ee40 0x55bf8049bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:02.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:02.676+0000 7f7213043740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bf8046f760 0x55bf8049c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:02.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:02.676+0000 7f7213043740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bf8047a020 0x55bf804989a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:02.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:02.676+0000 7f7213043740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f72080728d0 con 0x55bf8047a020 2024-09-10T09:58:02.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:02.676+0000 7f7213043740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f7208072200 con 0x55bf8046ee40 2024-09-10T09:58:02.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:02.676+0000 7f7213043740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f720806f440 con 0x55bf8046f760 2024-09-10T09:58:02.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:02.676+0000 7f7213043740 10 client.0 _ll_register_callbacks cb 0x7f71f40099b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:58:02.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:02.676+0000 7f720d000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bf8046ee40 0x55bf8049bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:02.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:02.676+0000 7f720d000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bf8046ee40 0x55bf8049bbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:56564/0 (socket says 172.21.15.5:56564) 2024-09-10T09:58:02.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:02.676+0000 7f720da00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bf8046f760 0x55bf8049c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:02.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:02.676+0000 7f720d000640 1 -- 172.21.15.5:0/3390209279 learned_addr learned my addr 172.21.15.5:0/3390209279 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:58:02.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:02.676+0000 7f720d000640 1 --2- 172.21.15.5:0/3390209279 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bf8047a020 0x55bf804989a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:02.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:02.677+0000 7f720da00640 1 -- 172.21.15.5:0/3390209279 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bf8046ee40 msgr2=0x55bf8049bbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:58:02.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:02.677+0000 7f720da00640 1 --2- 172.21.15.5:0/3390209279 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bf8046ee40 0x55bf8049bbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:02.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:02.677+0000 7f720da00640 1 -- 172.21.15.5:0/3390209279 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bf8047a020 msgr2=0x55bf804989a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:58:02.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:02.677+0000 7f720da00640 1 --2- 172.21.15.5:0/3390209279 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bf8047a020 0x55bf804989a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:02.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:02.677+0000 7f720da00640 1 -- 172.21.15.5:0/3390209279 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bf804991d0 con 0x55bf8046f760 2024-09-10T09:58:02.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:02.677+0000 7f720d000640 1 --2- 172.21.15.5:0/3390209279 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bf8046ee40 0x55bf8049bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:58:02.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:02.677+0000 7f720da00640 1 --2- 172.21.15.5:0/3390209279 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bf8046f760 0x55bf8049c0f0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x55bf804784a0 tx=0x7f71fc0198a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9353a02e61298f7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:02.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:02.677+0000 7f71ef400640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-09-10T09:58:02.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:02.677+0000 7f71ef400640 1 -- 172.21.15.5:0/3390209279 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71fc030070 con 0x55bf8046f760 2024-09-10T09:58:02.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:02.677+0000 7f7213043740 1 -- 172.21.15.5:0/3390209279 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f71fc011d10 con 0x55bf8046f760 2024-09-10T09:58:02.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:02.677+0000 7f71ef400640 1 -- 172.21.15.5:0/3390209279 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f71fc019c90 con 0x55bf8046f760 2024-09-10T09:58:02.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:02.677+0000 7f71ef400640 1 -- 172.21.15.5:0/3390209279 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f71fc027040 con 0x55bf8046f760 2024-09-10T09:58:02.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:02.677+0000 7f7213043740 10 client.29212 fetch_fsmap learned FSMap version 42 2024-09-10T09:58:02.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:02.677+0000 7f7213043740 1 -- 172.21.15.5:0/3390209279 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55bf8050f020 con 0x55bf8046f760 2024-09-10T09:58:02.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:02.678+0000 7f71ef400640 1 -- 172.21.15.5:0/3390209279 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f71fc011d10 con 0x55bf8046f760 2024-09-10T09:58:02.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:02.678+0000 7f71ef400640 1 -- 172.21.15.5:0/3390209279 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 42) ==== 8370+0+0 (secure 0 0 0) 0x7f71fc03e060 con 0x55bf8046f760 2024-09-10T09:58:02.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:02.678+0000 7f7213043740 10 client.29212 fetch_fsmap finished waiting for FSMap version 42 2024-09-10T09:58:02.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:02.678+0000 7f7213043740 10 client.29212 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-09-10T09:58:02.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:02.678+0000 7f7213043740 20 client.29212 populate_metadata read hostname 'smithi005' 2024-09-10T09:58:02.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:02.678+0000 7f7213043740 1 --2- 172.21.15.5:0/3390209279 >> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] conn(0x55bf8050f680 0x55bf80511aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:02.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:02.678+0000 7f7213043740 4 client.29212 mds_command: new command op to 24443 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T09:58:02.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:02.678+0000 7f7213043740 1 -- 172.21.15.5:0/3390209279 --> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55bf803dce60 con 0x55bf8050f680 2024-09-10T09:58:02.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:02.678+0000 7f720d000640 1 --2- 172.21.15.5:0/3390209279 >> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] conn(0x55bf8050f680 0x55bf80511aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:02.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:02.679+0000 7f720d000640 1 --2- 172.21.15.5:0/3390209279 >> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] conn(0x55bf8050f680 0x55bf80511aa0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=a24f146be2519d40 server_cookie=9ced43fbc662075b in_seq=0 out_seq=0 2024-09-10T09:58:02.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:02.679+0000 7f71ef400640 10 client.29212 ms_handle_connect on v2:172.21.15.102:6832/2457007427 2024-09-10T09:58:02.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:02.679+0000 7f71ef400640 1 -- 172.21.15.5:0/3390209279 <== mds.4 v2:172.21.15.102:6832/2457007427 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55bf803dce60 con 0x55bf8050f680 2024-09-10T09:58:02.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:02.679+0000 7f71ef400640 10 client.29212 handle_command_reply: tid=0 2024-09-10T09:58:02.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:02.679+0000 7f7213043740 1 client.29212 shutdown 2024-09-10T09:58:02.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:02.680+0000 7f7213043740 1 -- 172.21.15.5:0/3390209279 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bf8046f760 msgr2=0x55bf8049c0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:02.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:02.680+0000 7f7213043740 1 --2- 172.21.15.5:0/3390209279 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bf8046f760 0x55bf8049c0f0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x55bf804784a0 tx=0x7f71fc0198a0 comp rx=0 tx=0).stop 2024-09-10T09:58:02.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:02.680+0000 7f7213043740 1 -- 172.21.15.5:0/3390209279 shutdown_connections 2024-09-10T09:58:02.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:02.680+0000 7f7213043740 1 --2- 172.21.15.5:0/3390209279 >> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] conn(0x55bf8050f680 0x55bf80511aa0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:02.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:02.680+0000 7f7213043740 1 --2- 172.21.15.5:0/3390209279 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bf8047a020 0x55bf804989a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:02.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:02.680+0000 7f7213043740 1 --2- 172.21.15.5:0/3390209279 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bf8046f760 0x55bf8049c0f0 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:02.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:02.680+0000 7f7213043740 1 --2- 172.21.15.5:0/3390209279 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bf8046ee40 0x55bf8049bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:02.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:02.680+0000 7f7213043740 1 -- 172.21.15.5:0/3390209279 >> 172.21.15.5:0/3390209279 conn(0x55bf8046b3d0 msgr2=0x55bf804787e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:58:02.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:02.680+0000 7f71ef400640 0 client.29212 ms_handle_reset on v2:172.21.15.102:6832/2457007427 2024-09-10T09:58:02.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:02.680+0000 7f7213043740 1 -- 172.21.15.5:0/3390209279 shutdown_connections 2024-09-10T09:58:02.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:02.680+0000 7f7213043740 1 -- 172.21.15.5:0/3390209279 wait complete. 2024-09-10T09:58:02.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:02.681+0000 7f7213043740 20 client.29212 trim_cache size 0 max 16384 2024-09-10T09:58:02.685 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-10T09:58:02.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:02.682+0000 7f720ee00640 1 -- 172.21.15.5:0/2046060397 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f71f00a2d50 msgr2=0x7f71f00a5210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:02.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:02.682+0000 7f720ee00640 1 --2- 172.21.15.5:0/2046060397 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f71f00a2d50 0x7f71f00a5210 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f72000b8a20 tx=0x7f720003c040 comp rx=0 tx=0).stop 2024-09-10T09:58:02.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:02.682+0000 7f720ee00640 1 -- 172.21.15.5:0/2046060397 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f720810e280 msgr2=0x7f720819ed50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:02.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:02.682+0000 7f720ee00640 1 --2- 172.21.15.5:0/2046060397 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f720810e280 0x7f720819ed50 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f7200005e30 tx=0x7f7200003b80 comp rx=0 tx=0).stop 2024-09-10T09:58:02.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:02.682+0000 7f720ee00640 1 -- 172.21.15.5:0/2046060397 shutdown_connections 2024-09-10T09:58:02.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:02.682+0000 7f720ee00640 1 --2- 172.21.15.5:0/2046060397 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f71f00a2d50 0x7f71f00a5210 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:02.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:02.682+0000 7f720ee00640 1 --2- 172.21.15.5:0/2046060397 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f720810e280 0x7f720819ed50 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:02.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:02.682+0000 7f720ee00640 1 --2- 172.21.15.5:0/2046060397 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7208074e30 0x7f720819a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:02.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:02.682+0000 7f720ee00640 1 --2- 172.21.15.5:0/2046060397 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7208074510 0x7f720819a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:02.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:02.682+0000 7f720ee00640 1 -- 172.21.15.5:0/2046060397 >> 172.21.15.5:0/2046060397 conn(0x7f72080fd740 msgr2=0x7f72080fe440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:58:02.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:02.683+0000 7f720ee00640 1 -- 172.21.15.5:0/2046060397 shutdown_connections 2024-09-10T09:58:02.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:02.683+0000 7f720ee00640 1 -- 172.21.15.5:0/2046060397 wait complete. 2024-09-10T09:58:02.689 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 12.8 2024-09-10T09:58:02.689 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T09:58:03.011 INFO:tasks.workunit.client.0.smithi005.stdout: 10 103167 15.66 MB/sec execute 221 sec latency 57.859 ms 2024-09-10T09:58:03.807 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:58:03 smithi005 ceph-mon[30350]: pgmap v984: 129 pgs: 129 active+clean; 650 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 4.0 MiB/s wr, 446 op/s 2024-09-10T09:58:03.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:58:03 smithi159 ceph-mon[35513]: pgmap v984: 129 pgs: 129 active+clean; 650 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 4.0 MiB/s wr, 446 op/s 2024-09-10T09:58:03.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:58:03 smithi102 ceph-mon[35442]: pgmap v984: 129 pgs: 129 active+clean; 650 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 4.0 MiB/s wr, 446 op/s 2024-09-10T09:58:03.955 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-10T09:58:04.012 INFO:tasks.workunit.client.0.smithi005.stdout: 10 103726 15.68 MB/sec execute 222 sec latency 61.530 ms 2024-09-10T09:58:04.046 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:58:04.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:04.767+0000 7fd279600640 1 -- 172.21.15.5:0/2898195158 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd274077040 msgr2=0x7fd274112510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:04.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:04.767+0000 7fd279600640 1 --2- 172.21.15.5:0/2898195158 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd274077040 0x7fd274112510 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fd25c009860 tx=0x7fd25c02a340 comp rx=0 tx=0).stop 2024-09-10T09:58:04.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:04.767+0000 7fd279600640 1 -- 172.21.15.5:0/2898195158 shutdown_connections 2024-09-10T09:58:04.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:04.767+0000 7fd279600640 1 --2- 172.21.15.5:0/2898195158 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd274112a50 0x7fd274114e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:04.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:04.767+0000 7fd279600640 1 --2- 172.21.15.5:0/2898195158 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd274077040 0x7fd274112510 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:04.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:04.767+0000 7fd279600640 1 --2- 172.21.15.5:0/2898195158 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd274076720 0x7fd274076b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:04.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:04.767+0000 7fd279600640 1 -- 172.21.15.5:0/2898195158 >> 172.21.15.5:0/2898195158 conn(0x7fd2740ffa50 msgr2=0x7fd274101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:58:04.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:04.768+0000 7fd279600640 1 -- 172.21.15.5:0/2898195158 shutdown_connections 2024-09-10T09:58:04.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:04.768+0000 7fd279600640 1 -- 172.21.15.5:0/2898195158 wait complete. 2024-09-10T09:58:04.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:04.769+0000 7fd279600640 1 Processor -- start 2024-09-10T09:58:04.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:04.769+0000 7fd279600640 1 -- start start 2024-09-10T09:58:04.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:04.770+0000 7fd279600640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd274076720 0x7fd27419eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:04.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:04.770+0000 7fd279600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd274077040 0x7fd27419f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:04.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:04.770+0000 7fd279600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd274112a50 0x7fd2741a3460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:04.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:04.770+0000 7fd279600640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd2741185c0 con 0x7fd274112a50 2024-09-10T09:58:04.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:04.770+0000 7fd279600640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fd274118440 con 0x7fd274077040 2024-09-10T09:58:04.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:04.770+0000 7fd279600640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fd274118740 con 0x7fd274076720 2024-09-10T09:58:04.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:04.770+0000 7fd272a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd274076720 0x7fd27419eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:04.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:04.770+0000 7fd272a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd274076720 0x7fd27419eb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:40690/0 (socket says 172.21.15.5:40690) 2024-09-10T09:58:04.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:04.770+0000 7fd272a00640 1 -- 172.21.15.5:0/3507838495 learned_addr learned my addr 172.21.15.5:0/3507838495 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:58:04.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:04.771+0000 7fd272a00640 1 -- 172.21.15.5:0/3507838495 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd274077040 msgr2=0x7fd27419f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:04.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:04.771+0000 7fd272a00640 1 --2- 172.21.15.5:0/3507838495 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd274077040 0x7fd27419f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:04.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:04.771+0000 7fd272a00640 1 -- 172.21.15.5:0/3507838495 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd274112a50 msgr2=0x7fd2741a3460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:04.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:04.771+0000 7fd272a00640 1 --2- 172.21.15.5:0/3507838495 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd274112a50 0x7fd2741a3460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:04.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:04.771+0000 7fd272a00640 1 -- 172.21.15.5:0/3507838495 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2741a3b90 con 0x7fd274076720 2024-09-10T09:58:04.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:04.771+0000 7fd272a00640 1 --2- 172.21.15.5:0/3507838495 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd274076720 0x7fd27419eb60 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fd26400ebc0 tx=0x7fd26400c6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:04.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:04.772+0000 7fd26be00640 1 -- 172.21.15.5:0/3507838495 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd264010630 con 0x7fd274076720 2024-09-10T09:58:04.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:04.772+0000 7fd26be00640 1 -- 172.21.15.5:0/3507838495 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd264010cb0 con 0x7fd274076720 2024-09-10T09:58:04.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:04.772+0000 7fd26be00640 1 -- 172.21.15.5:0/3507838495 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd264018c00 con 0x7fd274076720 2024-09-10T09:58:04.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:04.772+0000 7fd279600640 1 -- 172.21.15.5:0/3507838495 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2741a3e80 con 0x7fd274076720 2024-09-10T09:58:04.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:04.772+0000 7fd279600640 1 -- 172.21.15.5:0/3507838495 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2741ab710 con 0x7fd274076720 2024-09-10T09:58:04.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:04.775+0000 7fd279600640 1 -- 172.21.15.5:0/3507838495 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd274076b00 con 0x7fd274076720 2024-09-10T09:58:04.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:04.779+0000 7fd26be00640 1 -- 172.21.15.5:0/3507838495 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7fd2640107d0 con 0x7fd274076720 2024-09-10T09:58:04.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:04.780+0000 7fd26be00640 1 --2- 172.21.15.5:0/3507838495 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fd2580a2cd0 0x7fd2580a5190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:04.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:04.781+0000 7fd26be00640 1 -- 172.21.15.5:0/3507838495 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fd2640cfb30 con 0x7fd274076720 2024-09-10T09:58:04.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:04.782+0000 7fd26a000640 1 --2- 172.21.15.5:0/3507838495 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fd2580a2cd0 0x7fd2580a5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:04.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:04.784+0000 7fd26a000640 1 --2- 172.21.15.5:0/3507838495 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fd2580a2cd0 0x7fd2580a5190 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fd25c009780 tx=0x7fd25c030460 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:04.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:04.786+0000 7fd26be00640 1 -- 172.21.15.5:0/3507838495 <== mon.1 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd26409c640 con 0x7fd274076720 2024-09-10T09:58:04.818 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:58:05.012 INFO:tasks.workunit.client.0.smithi005.stdout: 10 104279 15.75 MB/sec execute 223 sec latency 76.879 ms 2024-09-10T09:58:05.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:05.141+0000 7fd279600640 1 -- 172.21.15.5:0/3507838495 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd27419fec0 con 0x7fd274076720 2024-09-10T09:58:05.144 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-10T09:58:05.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:05.143+0000 7fd26be00640 1 -- 172.21.15.5:0/3507838495 <== mon.1 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 42 v42) ==== 94+0+11969 (secure 0 0 0) 0x7fd26409bf40 con 0x7fd274076720 2024-09-10T09:58:05.144 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 42 2024-09-10T09:58:05.145 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":42,"btime":"2024-09-10T09:55:06:670916+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14454,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/2970594311","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2970594311},{"type":"v1","addr":"172.21.15.5:6835","nonce":2970594311}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14514,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6839/3616502174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":3616502174},{"type":"v1","addr":"172.21.15.5:6839","nonce":3616502174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":42,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T09:35:29.626287+0000","modified":"2024-09-10T09:55:06.419829+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24542,"mds_1":24535,"mds_2":24511,"mds_3":24506,"mds_4":24443},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14520":{"gid":14520,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/2073599842","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":2073599842},{"type":"v1","addr":"172.21.15.5:6841","nonce":2073599842}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"k","rank":4,"incarnation":26,"state":"up:active","state_seq":24,"addr":"172.21.15.102:6833/2457007427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":2457007427},{"type":"v1","addr":"172.21.15.102:6833","nonce":2457007427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/356445159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":356445159},{"type":"v1","addr":"172.21.15.159:6833","nonce":356445159}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6837/2894314507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":2894314507},{"type":"v1","addr":"172.21.15.5:6837","nonce":2894314507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.102:6835/1647549111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":1647549111},{"type":"v1","addr":"172.21.15.102:6835","nonce":1647549111}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6835/952657242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":952657242},{"type":"v1","addr":"172.21.15.159:6835","nonce":952657242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":19,"addr":"172.21.15.102:6837/2689776764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":2689776764},{"type":"v1","addr":"172.21.15.102:6837","nonce":2689776764}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.159:6837/620627639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":620627639},{"type":"v1","addr":"172.21.15.159:6837","nonce":620627639}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24535":{"gid":24535,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":18,"addr":"172.21.15.159:6839/2073739599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":2073739599},{"type":"v1","addr":"172.21.15.159:6839","nonce":2073739599}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24542":{"gid":24542,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.102:6839/335409749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6838","nonce":335409749},{"type":"v1","addr":"172.21.15.102:6839","nonce":335409749}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24542,"qdb_cluster":[24542,24535,24511,24506,24443]},"id":1}]} 2024-09-10T09:58:05.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:05.147+0000 7fd279600640 1 -- 172.21.15.5:0/3507838495 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fd2580a2cd0 msgr2=0x7fd2580a5190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:05.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:05.147+0000 7fd279600640 1 --2- 172.21.15.5:0/3507838495 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fd2580a2cd0 0x7fd2580a5190 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fd25c009780 tx=0x7fd25c030460 comp rx=0 tx=0).stop 2024-09-10T09:58:05.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:05.147+0000 7fd279600640 1 -- 172.21.15.5:0/3507838495 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd274076720 msgr2=0x7fd27419eb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:05.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:05.147+0000 7fd279600640 1 --2- 172.21.15.5:0/3507838495 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd274076720 0x7fd27419eb60 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fd26400ebc0 tx=0x7fd26400c6a0 comp rx=0 tx=0).stop 2024-09-10T09:58:05.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:05.150+0000 7fd279600640 1 -- 172.21.15.5:0/3507838495 shutdown_connections 2024-09-10T09:58:05.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:05.150+0000 7fd279600640 1 --2- 172.21.15.5:0/3507838495 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fd2580a2cd0 0x7fd2580a5190 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:05.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:05.150+0000 7fd279600640 1 --2- 172.21.15.5:0/3507838495 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd274112a50 0x7fd2741a3460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:05.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:05.150+0000 7fd279600640 1 --2- 172.21.15.5:0/3507838495 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd274077040 0x7fd27419f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:05.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:05.150+0000 7fd279600640 1 --2- 172.21.15.5:0/3507838495 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd274076720 0x7fd27419eb60 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:05.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:05.150+0000 7fd279600640 1 -- 172.21.15.5:0/3507838495 >> 172.21.15.5:0/3507838495 conn(0x7fd2740ffa50 msgr2=0x7fd274101e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:58:05.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:05.150+0000 7fd279600640 1 -- 172.21.15.5:0/3507838495 shutdown_connections 2024-09-10T09:58:05.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:05.150+0000 7fd279600640 1 -- 172.21.15.5:0/3507838495 wait complete. 2024-09-10T09:58:05.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:05.723+0000 7f2d3ea00640 1 -- 172.21.15.5:0/3037289986 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2d38074e30 msgr2=0x7f2d3810dd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:05.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:05.723+0000 7f2d3ea00640 1 --2- 172.21.15.5:0/3037289986 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2d38074e30 0x7f2d3810dd40 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f2d24009900 tx=0x7f2d2402a480 comp rx=0 tx=0).stop 2024-09-10T09:58:05.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:05.723+0000 7f2d3ea00640 1 -- 172.21.15.5:0/3037289986 shutdown_connections 2024-09-10T09:58:05.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:05.723+0000 7f2d3ea00640 1 --2- 172.21.15.5:0/3037289986 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2d3810e280 0x7f2d38110670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:05.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:05.723+0000 7f2d3ea00640 1 --2- 172.21.15.5:0/3037289986 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2d38074e30 0x7f2d3810dd40 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:05.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:05.723+0000 7f2d3ea00640 1 --2- 172.21.15.5:0/3037289986 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2d38074510 0x7f2d380748f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:05.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:05.723+0000 7f2d3ea00640 1 -- 172.21.15.5:0/3037289986 >> 172.21.15.5:0/3037289986 conn(0x7f2d380fd740 msgr2=0x7f2d380ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:58:05.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:05.723+0000 7f2d3ea00640 1 -- 172.21.15.5:0/3037289986 shutdown_connections 2024-09-10T09:58:05.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:05.723+0000 7f2d3ea00640 1 -- 172.21.15.5:0/3037289986 wait complete. 2024-09-10T09:58:05.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:05.724+0000 7f2d3ea00640 1 Processor -- start 2024-09-10T09:58:05.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:05.725+0000 7f2d3ea00640 1 -- start start 2024-09-10T09:58:05.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:05.725+0000 7f2d3ea00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2d38074510 0x7f2d3819a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:05.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:05.725+0000 7f2d3ea00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2d38074e30 0x7f2d3819aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:05.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:05.725+0000 7f2d3ea00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2d3810e280 0x7f2d3819eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:05.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:05.725+0000 7f2d3ea00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f2d38113fb0 con 0x7f2d38074e30 2024-09-10T09:58:05.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:05.725+0000 7f2d3ea00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f2d38113e30 con 0x7f2d3810e280 2024-09-10T09:58:05.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:05.725+0000 7f2d3ea00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f2d38114130 con 0x7f2d38074510 2024-09-10T09:58:05.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:05.725+0000 7f2d3d600640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2d38074510 0x7f2d3819a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:05.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:05.725+0000 7f2d3e000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2d3810e280 0x7f2d3819eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:05.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:05.726+0000 7f2d3d600640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2d38074510 0x7f2d3819a4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:40712/0 (socket says 172.21.15.5:40712) 2024-09-10T09:58:05.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:05.726+0000 7f2d3d600640 1 -- 172.21.15.5:0/294630744 learned_addr learned my addr 172.21.15.5:0/294630744 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:58:05.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:05.726+0000 7f2d3e000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2d3810e280 0x7f2d3819eda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:56602/0 (socket says 172.21.15.5:56602) 2024-09-10T09:58:05.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:05.726+0000 7f2d3cc00640 1 --2- 172.21.15.5:0/294630744 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2d38074e30 0x7f2d3819aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:05.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:05.726+0000 7f2d3d600640 1 -- 172.21.15.5:0/294630744 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2d3810e280 msgr2=0x7f2d3819eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:05.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:05.726+0000 7f2d3d600640 1 --2- 172.21.15.5:0/294630744 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2d3810e280 0x7f2d3819eda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:05.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:05.726+0000 7f2d3d600640 1 -- 172.21.15.5:0/294630744 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2d38074e30 msgr2=0x7f2d3819aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:05.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:05.726+0000 7f2d3d600640 1 --2- 172.21.15.5:0/294630744 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2d38074e30 0x7f2d3819aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:05.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:05.727+0000 7f2d3d600640 1 -- 172.21.15.5:0/294630744 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d3819f480 con 0x7f2d38074510 2024-09-10T09:58:05.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:05.727+0000 7f2d3e000640 1 --2- 172.21.15.5:0/294630744 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2d3810e280 0x7f2d3819eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:58:05.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:05.727+0000 7f2d3d600640 1 --2- 172.21.15.5:0/294630744 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2d38074510 0x7f2d3819a4d0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f2d2800e9f0 tx=0x7f2d2800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:05.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:05.728+0000 7f2d36000640 1 -- 172.21.15.5:0/294630744 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d2800cd80 con 0x7f2d38074510 2024-09-10T09:58:05.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:05.728+0000 7f2d36000640 1 -- 172.21.15.5:0/294630744 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2d28010910 con 0x7f2d38074510 2024-09-10T09:58:05.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:05.728+0000 7f2d36000640 1 -- 172.21.15.5:0/294630744 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d28018990 con 0x7f2d38074510 2024-09-10T09:58:05.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:05.728+0000 7f2d3ea00640 1 -- 172.21.15.5:0/294630744 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d3819f770 con 0x7f2d38074510 2024-09-10T09:58:05.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:05.728+0000 7f2d3ea00640 1 -- 172.21.15.5:0/294630744 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d381a7050 con 0x7f2d38074510 2024-09-10T09:58:05.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:05.733+0000 7f2d42beb740 1 Processor -- start 2024-09-10T09:58:05.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:05.734+0000 7f2d36000640 1 -- 172.21.15.5:0/294630744 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f2d28010ac0 con 0x7f2d38074510 2024-09-10T09:58:05.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:05.734+0000 7f2d42beb740 1 -- start start 2024-09-10T09:58:05.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:05.734+0000 7f2d42beb740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ed45d26d40 0x55ed45d27120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:05.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:05.734+0000 7f2d42beb740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ed45d27660 0x55ed45d3a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:05.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:05.734+0000 7f2d42beb740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ed45d3a830 0x55ed45d3cc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:05.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:05.734+0000 7f2d42beb740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55ed45c8d3e0 con 0x55ed45d3a830 2024-09-10T09:58:05.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:05.734+0000 7f2d42beb740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55ed45c34d80 con 0x55ed45d27660 2024-09-10T09:58:05.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:05.734+0000 7f2d42beb740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55ed45c893a0 con 0x55ed45d26d40 2024-09-10T09:58:05.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:05.734+0000 7f2d3e000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ed45d26d40 0x55ed45d27120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:05.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:05.734+0000 7f2d3e000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ed45d26d40 0x55ed45d27120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:40724/0 (socket says 172.21.15.5:40724) 2024-09-10T09:58:05.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:05.734+0000 7f2d3e000640 1 -- 172.21.15.5:0/830147384 learned_addr learned my addr 172.21.15.5:0/830147384 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:58:05.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:05.735+0000 7f2d3e000640 1 -- 172.21.15.5:0/830147384 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ed45d27660 msgr2=0x55ed45d3a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:58:05.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:05.735+0000 7f2d3d600640 1 --2- 172.21.15.5:0/830147384 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ed45d27660 0x55ed45d3a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:05.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:05.735+0000 7f2d3e000640 1 --2- 172.21.15.5:0/830147384 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ed45d27660 0x55ed45d3a2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:05.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:05.735+0000 7f2d3cc00640 1 --2- 172.21.15.5:0/830147384 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ed45d3a830 0x55ed45d3cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:05.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:05.735+0000 7f2d3e000640 1 -- 172.21.15.5:0/830147384 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ed45d3a830 msgr2=0x55ed45d3cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:58:05.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:05.735+0000 7f2d3e000640 1 --2- 172.21.15.5:0/830147384 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ed45d3a830 0x55ed45d3cc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:05.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:05.735+0000 7f2d3e000640 1 -- 172.21.15.5:0/830147384 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ed45c53ae0 con 0x55ed45d26d40 2024-09-10T09:58:05.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:05.735+0000 7f2d3d600640 1 --2- 172.21.15.5:0/830147384 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ed45d27660 0x55ed45d3a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T09:58:05.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:05.735+0000 7f2d3cc00640 1 --2- 172.21.15.5:0/830147384 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ed45d3a830 0x55ed45d3cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T09:58:05.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:05.736+0000 7f2d3e000640 1 --2- 172.21.15.5:0/830147384 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ed45d26d40 0x55ed45d27120 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f2d30011760 tx=0x7f2d30011c20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7f166bc135a0f50a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:05.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:05.736+0000 7f2d36000640 1 --2- 172.21.15.5:0/294630744 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f2d1c0a2cd0 0x7f2d1c0a5190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:05.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:05.736+0000 7f2d2e600640 1 -- 172.21.15.5:0/830147384 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d30004d20 con 0x55ed45d26d40 2024-09-10T09:58:05.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:05.736+0000 7f2d2e600640 1 -- 172.21.15.5:0/830147384 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2d3000f9a0 con 0x55ed45d26d40 2024-09-10T09:58:05.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:05.736+0000 7f2d2e600640 1 -- 172.21.15.5:0/830147384 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d30004590 con 0x55ed45d26d40 2024-09-10T09:58:05.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:05.736+0000 7f2d36000640 1 -- 172.21.15.5:0/294630744 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f2d280cf6c0 con 0x7f2d38074510 2024-09-10T09:58:05.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:05.736+0000 7f2d3d600640 1 --2- 172.21.15.5:0/294630744 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f2d1c0a2cd0 0x7f2d1c0a5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:05.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:05.737+0000 7f2d42beb740 1 -- 172.21.15.5:0/830147384 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ed45d26d40 msgr2=0x55ed45d27120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:05.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:05.737+0000 7f2d42beb740 1 --2- 172.21.15.5:0/830147384 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ed45d26d40 0x55ed45d27120 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f2d30011760 tx=0x7f2d30011c20 comp rx=0 tx=0).stop 2024-09-10T09:58:05.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:05.738+0000 7f2d42beb740 1 -- 172.21.15.5:0/830147384 shutdown_connections 2024-09-10T09:58:05.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:05.738+0000 7f2d42beb740 1 --2- 172.21.15.5:0/830147384 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ed45d3a830 0x55ed45d3cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:05.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:05.738+0000 7f2d42beb740 1 --2- 172.21.15.5:0/830147384 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ed45d27660 0x55ed45d3a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:05.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:05.738+0000 7f2d42beb740 1 --2- 172.21.15.5:0/830147384 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ed45d26d40 0x55ed45d27120 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:05.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:05.738+0000 7f2d42beb740 1 -- 172.21.15.5:0/830147384 >> 172.21.15.5:0/830147384 conn(0x55ed45d233d0 msgr2=0x55ed45d237b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:58:05.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:05.738+0000 7f2d42beb740 1 -- 172.21.15.5:0/830147384 shutdown_connections 2024-09-10T09:58:05.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:05.738+0000 7f2d42beb740 1 -- 172.21.15.5:0/830147384 wait complete. 2024-09-10T09:58:05.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:05.738+0000 7f2d3d600640 1 --2- 172.21.15.5:0/294630744 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f2d1c0a2cd0 0x7f2d1c0a5190 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f2d280cd140 tx=0x7f2d28018420 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:05.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:05.739+0000 7f2d42beb740 1 -- start start 2024-09-10T09:58:05.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:05.739+0000 7f2d42beb740 1 Processor -- start 2024-09-10T09:58:05.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:05.740+0000 7f2d42beb740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ed45d26d40 0x55ed45d377a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:05.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:05.740+0000 7f2d42beb740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ed45d27660 0x55ed45d37ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:05.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:05.740+0000 7f2d42beb740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ed45d3a830 0x55ed45d38220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:05.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:05.740+0000 7f2d42beb740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55ed45d3d460 con 0x55ed45d27660 2024-09-10T09:58:05.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:05.740+0000 7f2d42beb740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55ed45c480f0 con 0x55ed45d26d40 2024-09-10T09:58:05.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:05.740+0000 7f2d42beb740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55ed45d3d5e0 con 0x55ed45d3a830 2024-09-10T09:58:05.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:05.740+0000 7f2d42beb740 10 client.0 _ll_register_callbacks cb 0x55ed45c2fbd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:58:05.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:05.740+0000 7f2d3cc00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ed45d27660 0x55ed45d37ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:05.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:05.740+0000 7f2d3e000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ed45d26d40 0x55ed45d377a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:05.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:05.740+0000 7f2d3cc00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ed45d27660 0x55ed45d37ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42190/0 (socket says 172.21.15.5:42190) 2024-09-10T09:58:05.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:05.740+0000 7f2d3cc00640 1 -- 172.21.15.5:0/2673671760 learned_addr learned my addr 172.21.15.5:0/2673671760 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:58:05.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:05.741+0000 7f2d3e000640 1 --2- 172.21.15.5:0/2673671760 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ed45d3a830 0x55ed45d38220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:05.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:05.741+0000 7f2d3cc00640 1 -- 172.21.15.5:0/2673671760 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ed45d3a830 msgr2=0x55ed45d38220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:58:05.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:05.741+0000 7f2d3cc00640 1 --2- 172.21.15.5:0/2673671760 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ed45d3a830 0x55ed45d38220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:05.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:05.741+0000 7f2d3cc00640 1 -- 172.21.15.5:0/2673671760 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ed45d26d40 msgr2=0x55ed45d377a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:58:05.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:05.741+0000 7f2d3cc00640 1 --2- 172.21.15.5:0/2673671760 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ed45d26d40 0x55ed45d377a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:05.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:05.741+0000 7f2d3cc00640 1 -- 172.21.15.5:0/2673671760 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ed459fe3c0 con 0x55ed45d27660 2024-09-10T09:58:05.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:05.741+0000 7f2d3e000640 1 --2- 172.21.15.5:0/2673671760 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ed45d3a830 0x55ed45d38220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:58:05.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:05.741+0000 7f2d3cc00640 1 --2- 172.21.15.5:0/2673671760 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ed45d27660 0x55ed45d37ce0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f2d2402a9a0 tx=0x7f2d24031860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7269aed327420ce server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:05.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:05.741+0000 7f2d22a00640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-10T09:58:05.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:05.742+0000 7f2d22a00640 1 -- 172.21.15.5:0/2673671760 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d2403f070 con 0x55ed45d27660 2024-09-10T09:58:05.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:05.742+0000 7f2d22a00640 1 -- 172.21.15.5:0/2673671760 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f2d24004450 con 0x55ed45d27660 2024-09-10T09:58:05.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:05.742+0000 7f2d42beb740 1 -- 172.21.15.5:0/2673671760 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2d3000f9a0 con 0x55ed45d27660 2024-09-10T09:58:05.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:05.744+0000 7f2d22a00640 1 -- 172.21.15.5:0/2673671760 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f2d2404c050 con 0x55ed45d27660 2024-09-10T09:58:05.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:05.745+0000 7f2d42beb740 10 client.19176 fetch_fsmap learned FSMap version 42 2024-09-10T09:58:05.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:05.745+0000 7f2d42beb740 1 -- 172.21.15.5:0/2673671760 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55ed45d494d0 con 0x55ed45d27660 2024-09-10T09:58:05.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:05.746+0000 7f2d22a00640 1 -- 172.21.15.5:0/2673671760 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2d3000f9a0 con 0x55ed45d27660 2024-09-10T09:58:05.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:05.746+0000 7f2d22a00640 1 -- 172.21.15.5:0/2673671760 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 42) ==== 8370+0+0 (secure 0 0 0) 0x7f2d24004990 con 0x55ed45d27660 2024-09-10T09:58:05.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:05.746+0000 7f2d42beb740 10 client.19176 fetch_fsmap finished waiting for FSMap version 42 2024-09-10T09:58:05.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:05.746+0000 7f2d42beb740 10 client.19176 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-09-10T09:58:05.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:05.746+0000 7f2d42beb740 20 client.19176 populate_metadata read hostname 'smithi005' 2024-09-10T09:58:05.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:05.746+0000 7f2d42beb740 1 --2- 172.21.15.5:0/2673671760 >> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] conn(0x55ed45d4a990 0x55ed45d4cd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:05.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:05.746+0000 7f2d42beb740 4 client.19176 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:58:05.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:05.746+0000 7f2d42beb740 1 -- 172.21.15.5:0/2673671760 --> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ed45c9a950 con 0x55ed45d4a990 2024-09-10T09:58:05.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:05.747+0000 7f2d3e000640 1 --2- 172.21.15.5:0/2673671760 >> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] conn(0x55ed45d4a990 0x55ed45d4cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:05.749 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:58:05 smithi005 ceph-mon[30350]: pgmap v985: 129 pgs: 129 active+clean; 653 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 4.2 MiB/s wr, 439 op/s 2024-09-10T09:58:05.749 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:58:05 smithi005 ceph-mon[30350]: from='client.? 172.21.15.5:0/3507838495' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:58:05.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:05.748+0000 7f2d22a00640 10 client.19176 ms_handle_connect on v2:172.21.15.102:6836/2689776764 2024-09-10T09:58:05.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:05.748+0000 7f2d3e000640 1 --2- 172.21.15.5:0/2673671760 >> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] conn(0x55ed45d4a990 0x55ed45d4cd80 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=33b4cef7782d3261 server_cookie=d1cc992c5789740e in_seq=0 out_seq=0 2024-09-10T09:58:05.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:05.749+0000 7f2d22a00640 1 -- 172.21.15.5:0/2673671760 <== mds.3 v2:172.21.15.102:6836/2689776764 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ed45c9a950 con 0x55ed45d4a990 2024-09-10T09:58:05.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:05.749+0000 7f2d22a00640 10 client.19176 handle_command_reply: tid=0 2024-09-10T09:58:05.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:05.749+0000 7f2d42beb740 1 client.19176 shutdown 2024-09-10T09:58:05.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:05.750+0000 7f2d42beb740 1 -- 172.21.15.5:0/2673671760 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ed45d27660 msgr2=0x55ed45d37ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:05.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:05.750+0000 7f2d42beb740 1 --2- 172.21.15.5:0/2673671760 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ed45d27660 0x55ed45d37ce0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f2d2402a9a0 tx=0x7f2d24031860 comp rx=0 tx=0).stop 2024-09-10T09:58:05.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:05.750+0000 7f2d42beb740 1 -- 172.21.15.5:0/2673671760 shutdown_connections 2024-09-10T09:58:05.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:05.750+0000 7f2d42beb740 1 --2- 172.21.15.5:0/2673671760 >> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] conn(0x55ed45d4a990 0x55ed45d4cd80 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:05.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:05.750+0000 7f2d42beb740 1 --2- 172.21.15.5:0/2673671760 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ed45d3a830 0x55ed45d38220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:05.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:05.750+0000 7f2d42beb740 1 --2- 172.21.15.5:0/2673671760 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ed45d27660 0x55ed45d37ce0 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:05.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:05.750+0000 7f2d42beb740 1 --2- 172.21.15.5:0/2673671760 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ed45d26d40 0x55ed45d377a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:05.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:05.750+0000 7f2d22a00640 0 client.19176 ms_handle_reset on v2:172.21.15.102:6836/2689776764 2024-09-10T09:58:05.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:05.750+0000 7f2d42beb740 1 -- 172.21.15.5:0/2673671760 >> 172.21.15.5:0/2673671760 conn(0x55ed45d233d0 msgr2=0x55ed45d29100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:58:05.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:05.751+0000 7f2d42beb740 1 -- 172.21.15.5:0/2673671760 shutdown_connections 2024-09-10T09:58:05.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:05.751+0000 7f2d42beb740 1 -- 172.21.15.5:0/2673671760 wait complete. 2024-09-10T09:58:05.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:05.751+0000 7f2d42beb740 20 client.19176 trim_cache size 0 max 16384 2024-09-10T09:58:05.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:05.767+0000 7f2d42beb740 1 Processor -- start 2024-09-10T09:58:05.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:05.767+0000 7f2d42beb740 1 -- start start 2024-09-10T09:58:05.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:05.768+0000 7f2d42beb740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ed45d26d40 0x55ed45d525e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:05.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:05.768+0000 7f2d42beb740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ed45d27660 0x55ed45d52b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:05.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:05.768+0000 7f2d42beb740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ed45d3a830 0x55ed45d28870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:05.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:05.768+0000 7f2d42beb740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f2d381acca0 con 0x55ed45d3a830 2024-09-10T09:58:05.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:05.768+0000 7f2d42beb740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f2d381ae7e0 con 0x55ed45d26d40 2024-09-10T09:58:05.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:05.768+0000 7f2d42beb740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f2d381ad370 con 0x55ed45d27660 2024-09-10T09:58:05.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:05.768+0000 7f2d3cc00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ed45d27660 0x55ed45d52b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:05.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:05.768+0000 7f2d3cc00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ed45d27660 0x55ed45d52b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:40744/0 (socket says 172.21.15.5:40744) 2024-09-10T09:58:05.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:05.768+0000 7f2d3cc00640 1 -- 172.21.15.5:0/710009132 learned_addr learned my addr 172.21.15.5:0/710009132 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:58:05.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:05.768+0000 7f2d3e000640 1 --2- 172.21.15.5:0/710009132 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ed45d26d40 0x55ed45d525e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:05.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:05.768+0000 7f2d3e000640 1 --2- 172.21.15.5:0/710009132 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ed45d3a830 0x55ed45d28870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:05.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:05.768+0000 7f2d3cc00640 1 -- 172.21.15.5:0/710009132 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ed45d26d40 msgr2=0x55ed45d525e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:58:05.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:05.768+0000 7f2d3cc00640 1 --2- 172.21.15.5:0/710009132 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ed45d26d40 0x55ed45d525e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:05.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:05.768+0000 7f2d3cc00640 1 -- 172.21.15.5:0/710009132 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ed45d3a830 msgr2=0x55ed45d28870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:58:05.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:05.768+0000 7f2d3cc00640 1 --2- 172.21.15.5:0/710009132 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ed45d3a830 0x55ed45d28870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:05.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:05.768+0000 7f2d3cc00640 1 -- 172.21.15.5:0/710009132 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ed45d27450 con 0x55ed45d27660 2024-09-10T09:58:05.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:05.768+0000 7f2d3e000640 1 --2- 172.21.15.5:0/710009132 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ed45d26d40 0x55ed45d525e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:58:05.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:05.769+0000 7f2d3cc00640 1 --2- 172.21.15.5:0/710009132 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ed45d27660 0x55ed45d52b20 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f2d24000c00 tx=0x7f2d2402a480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=61eba9f9e61b7b4e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:05.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:05.769+0000 7f2d2dc00640 1 -- 172.21.15.5:0/710009132 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d2403f030 con 0x55ed45d27660 2024-09-10T09:58:05.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:05.769+0000 7f2d2dc00640 1 -- 172.21.15.5:0/710009132 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2d24055070 con 0x55ed45d27660 2024-09-10T09:58:05.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:05.769+0000 7f2d2dc00640 1 -- 172.21.15.5:0/710009132 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d24035070 con 0x55ed45d27660 2024-09-10T09:58:05.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:05.770+0000 7f2d42beb740 1 -- 172.21.15.5:0/710009132 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ed45d27660 msgr2=0x55ed45d52b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:05.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:05.770+0000 7f2d42beb740 1 --2- 172.21.15.5:0/710009132 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ed45d27660 0x55ed45d52b20 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f2d24000c00 tx=0x7f2d2402a480 comp rx=0 tx=0).stop 2024-09-10T09:58:05.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:05.770+0000 7f2d42beb740 1 -- 172.21.15.5:0/710009132 shutdown_connections 2024-09-10T09:58:05.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:05.770+0000 7f2d42beb740 1 --2- 172.21.15.5:0/710009132 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ed45d3a830 0x55ed45d28870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:05.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:05.770+0000 7f2d42beb740 1 --2- 172.21.15.5:0/710009132 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ed45d27660 0x55ed45d52b20 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:05.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:05.770+0000 7f2d42beb740 1 --2- 172.21.15.5:0/710009132 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ed45d26d40 0x55ed45d525e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:05.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:05.770+0000 7f2d42beb740 1 -- 172.21.15.5:0/710009132 >> 172.21.15.5:0/710009132 conn(0x55ed45d233d0 msgr2=0x55ed45d237b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:58:05.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:05.770+0000 7f2d42beb740 1 -- 172.21.15.5:0/710009132 shutdown_connections 2024-09-10T09:58:05.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:05.770+0000 7f2d42beb740 1 -- 172.21.15.5:0/710009132 wait complete. 2024-09-10T09:58:05.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:05.771+0000 7f2d42beb740 1 -- start start 2024-09-10T09:58:05.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:05.771+0000 7f2d42beb740 1 Processor -- start 2024-09-10T09:58:05.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:05.771+0000 7f2d42beb740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ed45d26d40 0x55ed45d42a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:05.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:05.771+0000 7f2d42beb740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ed45d27660 0x55ed45d42f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:05.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:05.771+0000 7f2d42beb740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ed45d3a830 0x55ed45d3de90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:05.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:05.771+0000 7f2d42beb740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f2d381ae110 con 0x55ed45d26d40 2024-09-10T09:58:05.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:05.771+0000 7f2d42beb740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f2d381ada40 con 0x55ed45d3a830 2024-09-10T09:58:05.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:05.771+0000 7f2d42beb740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f2d381aac80 con 0x55ed45d27660 2024-09-10T09:58:05.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:05.772+0000 7f2d42beb740 10 client.0 _ll_register_callbacks cb 0x7f2d28005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:58:05.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:05.772+0000 7f2d3cc00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ed45d27660 0x55ed45d42f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:05.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:05.772+0000 7f2d3cc00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ed45d27660 0x55ed45d42f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:40748/0 (socket says 172.21.15.5:40748) 2024-09-10T09:58:05.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:05.772+0000 7f2d3cc00640 1 -- 172.21.15.5:0/361312560 learned_addr learned my addr 172.21.15.5:0/361312560 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:58:05.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:05.772+0000 7f2d3e000640 1 --2- 172.21.15.5:0/361312560 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ed45d26d40 0x55ed45d42a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:05.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:05.772+0000 7f2d3e000640 1 --2- 172.21.15.5:0/361312560 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ed45d3a830 0x55ed45d3de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:05.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:05.772+0000 7f2d3cc00640 1 -- 172.21.15.5:0/361312560 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ed45d3a830 msgr2=0x55ed45d3de90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:58:05.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:05.772+0000 7f2d3cc00640 1 --2- 172.21.15.5:0/361312560 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ed45d3a830 0x55ed45d3de90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:05.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:05.772+0000 7f2d3cc00640 1 -- 172.21.15.5:0/361312560 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ed45d26d40 msgr2=0x55ed45d42a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:58:05.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:05.772+0000 7f2d3cc00640 1 --2- 172.21.15.5:0/361312560 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ed45d26d40 0x55ed45d42a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:05.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:05.772+0000 7f2d3cc00640 1 -- 172.21.15.5:0/361312560 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ed45d3e6c0 con 0x55ed45d27660 2024-09-10T09:58:05.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:05.773+0000 7f2d3cc00640 1 --2- 172.21.15.5:0/361312560 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ed45d27660 0x55ed45d42f70 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f2d2402a9a0 tx=0x7f2d240095f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=69b2117dac6dfb70 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:05.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:05.773+0000 7f2d22a00640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-09-10T09:58:05.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:05.773+0000 7f2d22a00640 1 -- 172.21.15.5:0/361312560 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d2405e060 con 0x55ed45d27660 2024-09-10T09:58:05.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:05.773+0000 7f2d22a00640 1 -- 172.21.15.5:0/361312560 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f2d24004460 con 0x55ed45d27660 2024-09-10T09:58:05.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:05.773+0000 7f2d42beb740 1 -- 172.21.15.5:0/361312560 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2d24055070 con 0x55ed45d27660 2024-09-10T09:58:05.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:05.773+0000 7f2d22a00640 1 -- 172.21.15.5:0/361312560 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f2d24035070 con 0x55ed45d27660 2024-09-10T09:58:05.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:05.773+0000 7f2d42beb740 10 client.29236 fetch_fsmap learned FSMap version 42 2024-09-10T09:58:05.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:05.773+0000 7f2d42beb740 1 -- 172.21.15.5:0/361312560 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55ed45db50e0 con 0x55ed45d27660 2024-09-10T09:58:05.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:05.774+0000 7f2d22a00640 1 -- 172.21.15.5:0/361312560 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2d24055070 con 0x55ed45d27660 2024-09-10T09:58:05.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:05.774+0000 7f2d22a00640 1 -- 172.21.15.5:0/361312560 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 42) ==== 8370+0+0 (secure 0 0 0) 0x7f2d24004fe0 con 0x55ed45d27660 2024-09-10T09:58:05.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:05.774+0000 7f2d42beb740 10 client.29236 fetch_fsmap finished waiting for FSMap version 42 2024-09-10T09:58:05.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:05.774+0000 7f2d42beb740 10 client.29236 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-09-10T09:58:05.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:05.774+0000 7f2d42beb740 20 client.29236 populate_metadata read hostname 'smithi005' 2024-09-10T09:58:05.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:05.774+0000 7f2d42beb740 1 --2- 172.21.15.5:0/361312560 >> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] conn(0x55ed45db5850 0x55ed45db7c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:05.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:05.774+0000 7f2d42beb740 4 client.29236 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T09:58:05.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:05.774+0000 7f2d42beb740 1 -- 172.21.15.5:0/361312560 --> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ed45c94e60 con 0x55ed45db5850 2024-09-10T09:58:05.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:05.775+0000 7f2d3e000640 1 --2- 172.21.15.5:0/361312560 >> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] conn(0x55ed45db5850 0x55ed45db7c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:05.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:05.775+0000 7f2d3e000640 1 --2- 172.21.15.5:0/361312560 >> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] conn(0x55ed45db5850 0x55ed45db7c70 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=2b1097a07063a55e server_cookie=be441da7dae05aeb in_seq=0 out_seq=0 2024-09-10T09:58:05.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:05.775+0000 7f2d22a00640 10 client.29236 ms_handle_connect on v2:172.21.15.102:6836/2689776764 2024-09-10T09:58:05.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:05.775+0000 7f2d22a00640 1 -- 172.21.15.5:0/361312560 <== mds.3 v2:172.21.15.102:6836/2689776764 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ed45c94e60 con 0x55ed45db5850 2024-09-10T09:58:05.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:05.775+0000 7f2d22a00640 10 client.29236 handle_command_reply: tid=0 2024-09-10T09:58:05.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:05.775+0000 7f2d42beb740 1 client.29236 shutdown 2024-09-10T09:58:05.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:05.776+0000 7f2d42beb740 1 -- 172.21.15.5:0/361312560 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ed45d27660 msgr2=0x55ed45d42f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:05.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:05.776+0000 7f2d42beb740 1 --2- 172.21.15.5:0/361312560 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ed45d27660 0x55ed45d42f70 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f2d2402a9a0 tx=0x7f2d240095f0 comp rx=0 tx=0).stop 2024-09-10T09:58:05.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:05.776+0000 7f2d42beb740 1 -- 172.21.15.5:0/361312560 shutdown_connections 2024-09-10T09:58:05.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:05.776+0000 7f2d42beb740 1 --2- 172.21.15.5:0/361312560 >> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] conn(0x55ed45db5850 0x55ed45db7c70 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:05.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:05.776+0000 7f2d22a00640 0 client.29236 ms_handle_reset on v2:172.21.15.102:6836/2689776764 2024-09-10T09:58:05.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:05.776+0000 7f2d42beb740 1 --2- 172.21.15.5:0/361312560 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ed45d3a830 0x55ed45d3de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:05.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:05.776+0000 7f2d42beb740 1 --2- 172.21.15.5:0/361312560 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ed45d27660 0x55ed45d42f70 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:05.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:05.776+0000 7f2d42beb740 1 --2- 172.21.15.5:0/361312560 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ed45d26d40 0x55ed45d42a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:05.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:05.776+0000 7f2d42beb740 1 -- 172.21.15.5:0/361312560 >> 172.21.15.5:0/361312560 conn(0x55ed45d233d0 msgr2=0x55ed45d28870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:58:05.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:05.776+0000 7f2d42beb740 1 -- 172.21.15.5:0/361312560 shutdown_connections 2024-09-10T09:58:05.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:05.776+0000 7f2d42beb740 1 -- 172.21.15.5:0/361312560 wait complete. 2024-09-10T09:58:05.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:05.776+0000 7f2d42beb740 20 client.29236 trim_cache size 0 max 16384 2024-09-10T09:58:05.780 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-10T09:58:05.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:05.777+0000 7f2d3ea00640 1 -- 172.21.15.5:0/294630744 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f2d1c0a2cd0 msgr2=0x7f2d1c0a5190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:05.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:05.777+0000 7f2d3ea00640 1 --2- 172.21.15.5:0/294630744 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f2d1c0a2cd0 0x7f2d1c0a5190 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f2d280cd140 tx=0x7f2d28018420 comp rx=0 tx=0).stop 2024-09-10T09:58:05.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:05.777+0000 7f2d3ea00640 1 -- 172.21.15.5:0/294630744 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2d38074510 msgr2=0x7f2d3819a4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:05.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:05.777+0000 7f2d3ea00640 1 --2- 172.21.15.5:0/294630744 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2d38074510 0x7f2d3819a4d0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f2d2800e9f0 tx=0x7f2d2800eeb0 comp rx=0 tx=0).stop 2024-09-10T09:58:05.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:05.778+0000 7f2d3ea00640 1 -- 172.21.15.5:0/294630744 shutdown_connections 2024-09-10T09:58:05.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:05.778+0000 7f2d3ea00640 1 --2- 172.21.15.5:0/294630744 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f2d1c0a2cd0 0x7f2d1c0a5190 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:05.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:05.778+0000 7f2d3ea00640 1 --2- 172.21.15.5:0/294630744 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2d3810e280 0x7f2d3819eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:05.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:05.778+0000 7f2d3ea00640 1 --2- 172.21.15.5:0/294630744 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2d38074e30 0x7f2d3819aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:05.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:05.778+0000 7f2d3ea00640 1 --2- 172.21.15.5:0/294630744 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2d38074510 0x7f2d3819a4d0 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:05.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:05.778+0000 7f2d3ea00640 1 -- 172.21.15.5:0/294630744 >> 172.21.15.5:0/294630744 conn(0x7f2d380fd740 msgr2=0x7f2d380fe440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:58:05.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:05.778+0000 7f2d3ea00640 1 -- 172.21.15.5:0/294630744 shutdown_connections 2024-09-10T09:58:05.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:05.778+0000 7f2d3ea00640 1 -- 172.21.15.5:0/294630744 wait complete. 2024-09-10T09:58:05.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:58:05 smithi159 ceph-mon[35513]: pgmap v985: 129 pgs: 129 active+clean; 653 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 4.2 MiB/s wr, 439 op/s 2024-09-10T09:58:05.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:58:05 smithi159 ceph-mon[35513]: from='client.? 172.21.15.5:0/3507838495' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:58:05.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:58:05 smithi102 ceph-mon[35442]: pgmap v985: 129 pgs: 129 active+clean; 653 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 4.2 MiB/s wr, 439 op/s 2024-09-10T09:58:05.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:58:05 smithi102 ceph-mon[35442]: from='client.? 172.21.15.5:0/3507838495' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:58:06.012 INFO:tasks.workunit.client.0.smithi005.stdout: 10 105042 15.87 MB/sec execute 224 sec latency 80.393 ms 2024-09-10T09:58:07.018 INFO:tasks.workunit.client.0.smithi005.stdout: 10 105120 15.81 MB/sec execute 225 sec latency 941.636 ms 2024-09-10T09:58:07.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:58:07 smithi005 ceph-mon[30350]: pgmap v986: 129 pgs: 129 active+clean; 659 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 4.8 MiB/s wr, 503 op/s 2024-09-10T09:58:07.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:58:07 smithi159 ceph-mon[35513]: pgmap v986: 129 pgs: 129 active+clean; 659 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 4.8 MiB/s wr, 503 op/s 2024-09-10T09:58:07.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:58:07 smithi102 ceph-mon[35442]: pgmap v986: 129 pgs: 129 active+clean; 659 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 4.8 MiB/s wr, 503 op/s 2024-09-10T09:58:08.018 INFO:tasks.workunit.client.0.smithi005.stdout: 10 105232 15.78 MB/sec execute 226 sec latency 1941.689 ms 2024-09-10T09:58:09.018 INFO:tasks.workunit.client.0.smithi005.stdout: 10 105641 15.78 MB/sec execute 227 sec latency 2409.567 ms 2024-09-10T09:58:09.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:58:09 smithi005 ceph-mon[30350]: pgmap v987: 129 pgs: 129 active+clean; 670 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 5.1 MiB/s wr, 454 op/s 2024-09-10T09:58:09.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:58:09 smithi159 ceph-mon[35513]: pgmap v987: 129 pgs: 129 active+clean; 670 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 5.1 MiB/s wr, 454 op/s 2024-09-10T09:58:09.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:58:09 smithi102 ceph-mon[35442]: pgmap v987: 129 pgs: 129 active+clean; 670 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 5.1 MiB/s wr, 454 op/s 2024-09-10T09:58:10.019 INFO:tasks.workunit.client.0.smithi005.stdout: 10 106404 15.90 MB/sec execute 228 sec latency 54.666 ms 2024-09-10T09:58:10.161 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24542 quiesce db / --timeout 90 --expiration 132.8 --await-for 45.0' 2024-09-10T09:58:10.162 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24542 quiesce db / --timeout 90 --expiration 132.8 --await-for 45.0 2024-09-10T09:58:10.769 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-10T09:58:10.851 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:58:11.019 INFO:tasks.workunit.client.0.smithi005.stdout: 10 107219 16.05 MB/sec execute 229 sec latency 59.463 ms 2024-09-10T09:58:11.527 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:58:11 smithi005 ceph-mon[30350]: pgmap v988: 129 pgs: 129 active+clean; 673 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 6.5 MiB/s wr, 541 op/s 2024-09-10T09:58:11.527 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:58:11 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:58:11.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:11.594+0000 7f4ee2600640 1 -- 172.21.15.5:0/307472256 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4edc106f20 msgr2=0x7f4edc10eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:11.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:11.594+0000 7f4ee2600640 1 --2- 172.21.15.5:0/307472256 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4edc106f20 0x7f4edc10eaa0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f4ed4009ca0 tx=0x7f4ed402a8c0 comp rx=0 tx=0).stop 2024-09-10T09:58:11.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:11.594+0000 7f4ee2600640 1 -- 172.21.15.5:0/307472256 shutdown_connections 2024-09-10T09:58:11.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:11.594+0000 7f4ee2600640 1 --2- 172.21.15.5:0/307472256 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4edc106f20 0x7f4edc10eaa0 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:11.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:11.594+0000 7f4ee2600640 1 --2- 172.21.15.5:0/307472256 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4edc1027a0 0x7f4edc1067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:11.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:11.594+0000 7f4ee2600640 1 --2- 172.21.15.5:0/307472256 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4edc101df0 0x7f4edc1021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:11.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:11.594+0000 7f4ee2600640 1 -- 172.21.15.5:0/307472256 >> 172.21.15.5:0/307472256 conn(0x7f4edc0fd8b0 msgr2=0x7f4edc0ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:58:11.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:11.594+0000 7f4ee2600640 1 -- 172.21.15.5:0/307472256 shutdown_connections 2024-09-10T09:58:11.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:11.596+0000 7f4ee2600640 1 -- 172.21.15.5:0/307472256 wait complete. 2024-09-10T09:58:11.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:11.597+0000 7f4ee2600640 1 Processor -- start 2024-09-10T09:58:11.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:11.598+0000 7f4ee2600640 1 -- start start 2024-09-10T09:58:11.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:11.598+0000 7f4ee2600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4edc101df0 0x7f4edc117200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:11.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:11.598+0000 7f4ee2600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4edc1027a0 0x7f4edc117740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:11.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:11.598+0000 7f4ee2600640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4edc106f20 0x7f4edc1112c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:11.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:11.598+0000 7f4ee2600640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f4edc10fdc0 con 0x7f4edc101df0 2024-09-10T09:58:11.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:11.598+0000 7f4ee2600640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f4edc10fc40 con 0x7f4edc1027a0 2024-09-10T09:58:11.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:11.598+0000 7f4ee2600640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f4edc10ff40 con 0x7f4edc106f20 2024-09-10T09:58:11.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:11.599+0000 7f4edaa00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4edc1027a0 0x7f4edc117740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:11.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:11.599+0000 7f4edbe00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4edc106f20 0x7f4edc1112c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:11.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:11.599+0000 7f4edaa00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4edc1027a0 0x7f4edc117740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:57078/0 (socket says 172.21.15.5:57078) 2024-09-10T09:58:11.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:11.599+0000 7f4edaa00640 1 -- 172.21.15.5:0/559346138 learned_addr learned my addr 172.21.15.5:0/559346138 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:58:11.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:11.599+0000 7f4edb400640 1 --2- 172.21.15.5:0/559346138 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4edc101df0 0x7f4edc117200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:11.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:11.599+0000 7f4edbe00640 1 -- 172.21.15.5:0/559346138 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4edc1027a0 msgr2=0x7f4edc117740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:11.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:11.599+0000 7f4edbe00640 1 --2- 172.21.15.5:0/559346138 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4edc1027a0 0x7f4edc117740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:11.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:11.599+0000 7f4edbe00640 1 -- 172.21.15.5:0/559346138 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4edc101df0 msgr2=0x7f4edc117200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:11.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:11.599+0000 7f4edbe00640 1 --2- 172.21.15.5:0/559346138 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4edc101df0 0x7f4edc117200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:11.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:11.599+0000 7f4edbe00640 1 -- 172.21.15.5:0/559346138 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4edc111b80 con 0x7f4edc106f20 2024-09-10T09:58:11.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:11.599+0000 7f4edb400640 1 --2- 172.21.15.5:0/559346138 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4edc101df0 0x7f4edc117200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:58:11.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:11.600+0000 7f4edbe00640 1 --2- 172.21.15.5:0/559346138 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4edc106f20 0x7f4edc1112c0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f4ed4005e30 tx=0x7f4ed4003f10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:11.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:11.600+0000 7f4ed3e00640 1 -- 172.21.15.5:0/559346138 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ed4031650 con 0x7f4edc106f20 2024-09-10T09:58:11.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:11.600+0000 7f4ed3e00640 1 -- 172.21.15.5:0/559346138 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4ed40317f0 con 0x7f4edc106f20 2024-09-10T09:58:11.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:11.600+0000 7f4ed3e00640 1 -- 172.21.15.5:0/559346138 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ed402fa30 con 0x7f4edc106f20 2024-09-10T09:58:11.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:11.601+0000 7f4ee2600640 1 -- 172.21.15.5:0/559346138 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4edc111e10 con 0x7f4edc106f20 2024-09-10T09:58:11.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:11.601+0000 7f4ee2600640 1 -- 172.21.15.5:0/559346138 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4edc1b3d20 con 0x7f4edc106f20 2024-09-10T09:58:11.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:11.606+0000 7f4ed3e00640 1 -- 172.21.15.5:0/559346138 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f4ed402fbd0 con 0x7f4edc106f20 2024-09-10T09:58:11.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:11.606+0000 7f4ee5089740 1 Processor -- start 2024-09-10T09:58:11.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:11.606+0000 7f4ee5089740 1 -- start start 2024-09-10T09:58:11.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:11.607+0000 7f4ee5089740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560e345293b0 0x560e34529790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:11.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:11.607+0000 7f4ee5089740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560e34529cd0 0x560e34534050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:11.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:11.607+0000 7f4ee5089740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560e34534590 0x560e34536980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:11.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:11.607+0000 7f4ee5089740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x560e3448f530 con 0x560e345293b0 2024-09-10T09:58:11.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:11.607+0000 7f4ee5089740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x560e34436860 con 0x560e34529cd0 2024-09-10T09:58:11.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:11.607+0000 7f4ee5089740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x560e3448b4f0 con 0x560e34534590 2024-09-10T09:58:11.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:11.607+0000 7f4edaa00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560e345293b0 0x560e34529790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:11.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:11.607+0000 7f4edaa00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560e34534590 0x560e34536980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:11.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:11.607+0000 7f4edaa00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560e34534590 0x560e34536980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:46810/0 (socket says 172.21.15.5:46810) 2024-09-10T09:58:11.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:11.607+0000 7f4edaa00640 1 -- 172.21.15.5:0/3527373976 learned_addr learned my addr 172.21.15.5:0/3527373976 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:58:11.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:11.608+0000 7f4edb400640 1 --2- 172.21.15.5:0/3527373976 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560e34529cd0 0x560e34534050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:11.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:11.608+0000 7f4edaa00640 1 -- 172.21.15.5:0/3527373976 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560e34529cd0 msgr2=0x560e34534050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:58:11.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:11.608+0000 7f4edaa00640 1 --2- 172.21.15.5:0/3527373976 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560e34529cd0 0x560e34534050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:11.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:11.608+0000 7f4edaa00640 1 -- 172.21.15.5:0/3527373976 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560e345293b0 msgr2=0x560e34529790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:58:11.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:11.608+0000 7f4edaa00640 1 --2- 172.21.15.5:0/3527373976 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560e345293b0 0x560e34529790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:11.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:11.608+0000 7f4edaa00640 1 -- 172.21.15.5:0/3527373976 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560e34200b90 con 0x560e34534590 2024-09-10T09:58:11.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:11.608+0000 7f4edaa00640 1 --2- 172.21.15.5:0/3527373976 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560e34534590 0x560e34536980 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f4ec8009fd0 tx=0x7f4ec8004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f5661d663093c0fc server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:11.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:11.608+0000 7f4ed3e00640 1 --2- 172.21.15.5:0/559346138 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f4ec00a2c00 0x7f4ec00a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:11.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:11.609+0000 7f4ed0c00640 1 -- 172.21.15.5:0/3527373976 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ec8005020 con 0x560e34534590 2024-09-10T09:58:11.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:11.609+0000 7f4ed3e00640 1 -- 172.21.15.5:0/559346138 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f4ed40f09e0 con 0x7f4edc106f20 2024-09-10T09:58:11.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:11.609+0000 7f4ed0c00640 1 -- 172.21.15.5:0/3527373976 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4ec80047a0 con 0x560e34534590 2024-09-10T09:58:11.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:11.609+0000 7f4ed0c00640 1 -- 172.21.15.5:0/3527373976 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ec8010770 con 0x560e34534590 2024-09-10T09:58:11.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:11.609+0000 7f4edb400640 1 --2- 172.21.15.5:0/559346138 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f4ec00a2c00 0x7f4ec00a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:11.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:11.610+0000 7f4ee5089740 1 -- 172.21.15.5:0/3527373976 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560e34534590 msgr2=0x560e34536980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:11.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:11.610+0000 7f4ee5089740 1 --2- 172.21.15.5:0/3527373976 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560e34534590 0x560e34536980 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f4ec8009fd0 tx=0x7f4ec8004730 comp rx=0 tx=0).stop 2024-09-10T09:58:11.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:11.610+0000 7f4ee5089740 1 -- 172.21.15.5:0/3527373976 shutdown_connections 2024-09-10T09:58:11.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:11.610+0000 7f4ee5089740 1 --2- 172.21.15.5:0/3527373976 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560e34534590 0x560e34536980 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:11.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:11.610+0000 7f4ee5089740 1 --2- 172.21.15.5:0/3527373976 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560e34529cd0 0x560e34534050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:11.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:11.610+0000 7f4ee5089740 1 --2- 172.21.15.5:0/3527373976 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560e345293b0 0x560e34529790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:11.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:11.610+0000 7f4ee5089740 1 -- 172.21.15.5:0/3527373976 >> 172.21.15.5:0/3527373976 conn(0x560e34525940 msgr2=0x560e34525d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:58:11.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:11.610+0000 7f4ee5089740 1 -- 172.21.15.5:0/3527373976 shutdown_connections 2024-09-10T09:58:11.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:11.611+0000 7f4ee5089740 1 -- 172.21.15.5:0/3527373976 wait complete. 2024-09-10T09:58:11.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:11.611+0000 7f4edb400640 1 --2- 172.21.15.5:0/559346138 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f4ec00a2c00 0x7f4ec00a50c0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x560e345267a0 tx=0x7f4ecc00b5a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:11.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:11.611+0000 7f4ee5089740 1 -- start start 2024-09-10T09:58:11.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:11.612+0000 7f4ee5089740 1 Processor -- start 2024-09-10T09:58:11.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:11.612+0000 7f4ee5089740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560e345293b0 0x560e34531570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:11.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:11.613+0000 7f4ee5089740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560e34529cd0 0x560e34531ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:11.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:11.613+0000 7f4ee5089740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560e34534590 0x560e34531ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:11.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:11.613+0000 7f4ee5089740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x560e34537010 con 0x560e34529cd0 2024-09-10T09:58:11.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:11.613+0000 7f4ee5089740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x560e34449e60 con 0x560e34534590 2024-09-10T09:58:11.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:11.613+0000 7f4ee5089740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x560e34537190 con 0x560e345293b0 2024-09-10T09:58:11.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:11.613+0000 7f4ee5089740 10 client.0 _ll_register_callbacks cb 0x560e3444ecc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:58:11.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:11.613+0000 7f4edb400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560e345293b0 0x560e34531570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:11.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:11.613+0000 7f4edbe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560e34534590 0x560e34531ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:11.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:11.613+0000 7f4edbe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560e34534590 0x560e34531ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:57082/0 (socket says 172.21.15.5:57082) 2024-09-10T09:58:11.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:11.613+0000 7f4edb400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560e345293b0 0x560e34531570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:46818/0 (socket says 172.21.15.5:46818) 2024-09-10T09:58:11.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:11.613+0000 7f4edbe00640 1 -- 172.21.15.5:0/796801130 learned_addr learned my addr 172.21.15.5:0/796801130 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:58:11.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:11.614+0000 7f4edaa00640 1 --2- 172.21.15.5:0/796801130 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560e34529cd0 0x560e34531ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:11.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:11.614+0000 7f4edb400640 1 -- 172.21.15.5:0/796801130 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560e34534590 msgr2=0x560e34531ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:58:11.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:11.614+0000 7f4edb400640 1 --2- 172.21.15.5:0/796801130 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560e34534590 0x560e34531ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:11.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:11.614+0000 7f4edb400640 1 -- 172.21.15.5:0/796801130 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560e34529cd0 msgr2=0x560e34531ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:58:11.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:11.614+0000 7f4edb400640 1 --2- 172.21.15.5:0/796801130 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560e34529cd0 0x560e34531ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:11.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:11.614+0000 7f4edb400640 1 -- 172.21.15.5:0/796801130 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560e34455990 con 0x560e345293b0 2024-09-10T09:58:11.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:11.615+0000 7f4edb400640 1 --2- 172.21.15.5:0/796801130 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560e345293b0 0x560e34531570 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f4ecc01d8e0 tx=0x7f4ecc0046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7e970b95e298e574 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:11.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:11.615+0000 7f4ec5600640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-09-10T09:58:11.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:11.615+0000 7f4ec5600640 1 -- 172.21.15.5:0/796801130 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ecc01de60 con 0x560e345293b0 2024-09-10T09:58:11.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:11.615+0000 7f4ec5600640 1 -- 172.21.15.5:0/796801130 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f4ecc010d60 con 0x560e345293b0 2024-09-10T09:58:11.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:11.615+0000 7f4ee5089740 1 -- 172.21.15.5:0/796801130 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4ec80047a0 con 0x560e345293b0 2024-09-10T09:58:11.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:11.615+0000 7f4ec5600640 1 -- 172.21.15.5:0/796801130 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f4ecc01b460 con 0x560e345293b0 2024-09-10T09:58:11.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:11.615+0000 7f4ee5089740 10 client.29248 fetch_fsmap learned FSMap version 42 2024-09-10T09:58:11.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:11.616+0000 7f4ee5089740 1 -- 172.21.15.5:0/796801130 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x560e34542ee0 con 0x560e345293b0 2024-09-10T09:58:11.618 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:58:11.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:11.616+0000 7f4ec5600640 1 -- 172.21.15.5:0/796801130 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4ec80047a0 con 0x560e345293b0 2024-09-10T09:58:11.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:11.616+0000 7f4ec5600640 1 -- 172.21.15.5:0/796801130 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 42) ==== 8370+0+0 (secure 0 0 0) 0x7f4ecc024070 con 0x560e345293b0 2024-09-10T09:58:11.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:11.617+0000 7f4ee5089740 10 client.29248 fetch_fsmap finished waiting for FSMap version 42 2024-09-10T09:58:11.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:11.617+0000 7f4ee5089740 10 client.29248 resolve_mds: validated gid 24542 aka daemon mds.b 2024-09-10T09:58:11.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:11.617+0000 7f4ee5089740 20 client.29248 populate_metadata read hostname 'smithi005' 2024-09-10T09:58:11.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:11.617+0000 7f4ee5089740 1 --2- 172.21.15.5:0/796801130 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x560e345443a0 0x560e34546860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:11.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:11.617+0000 7f4ee5089740 4 client.29248 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:58:11.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:11.617+0000 7f4ee5089740 1 -- 172.21.15.5:0/796801130 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560e34432310 con 0x560e345443a0 2024-09-10T09:58:11.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:11.617+0000 7f4edaa00640 1 --2- 172.21.15.5:0/796801130 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x560e345443a0 0x560e34546860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:11.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:11.618+0000 7f4edaa00640 1 --2- 172.21.15.5:0/796801130 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x560e345443a0 0x560e34546860 crc :-1 s=READY pgs=277 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a0f439f880459d9f server_cookie=d4d71c5f99b67cea in_seq=0 out_seq=0 2024-09-10T09:58:11.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:11.618+0000 7f4ec5600640 10 client.29248 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T09:58:11.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:11.619+0000 7f4ec5600640 1 -- 172.21.15.5:0/796801130 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560e34432310 con 0x560e345443a0 2024-09-10T09:58:11.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:11.619+0000 7f4ec5600640 10 client.29248 handle_command_reply: tid=0 2024-09-10T09:58:11.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:11.619+0000 7f4ee5089740 1 client.29248 shutdown 2024-09-10T09:58:11.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:11.620+0000 7f4ee5089740 1 -- 172.21.15.5:0/796801130 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560e345293b0 msgr2=0x560e34531570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:11.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:11.620+0000 7f4ee5089740 1 --2- 172.21.15.5:0/796801130 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560e345293b0 0x560e34531570 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f4ecc01d8e0 tx=0x7f4ecc0046c0 comp rx=0 tx=0).stop 2024-09-10T09:58:11.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:11.620+0000 7f4ee5089740 1 -- 172.21.15.5:0/796801130 shutdown_connections 2024-09-10T09:58:11.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:11.620+0000 7f4ee5089740 1 --2- 172.21.15.5:0/796801130 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x560e345443a0 0x560e34546860 crc :-1 s=READY pgs=277 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:11.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:11.620+0000 7f4ee5089740 1 --2- 172.21.15.5:0/796801130 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560e34534590 0x560e34531ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:11.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:11.620+0000 7f4ee5089740 1 --2- 172.21.15.5:0/796801130 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560e34529cd0 0x560e34531ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:11.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:11.621+0000 7f4ec5600640 0 client.29248 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T09:58:11.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:11.620+0000 7f4ee5089740 1 --2- 172.21.15.5:0/796801130 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560e345293b0 0x560e34531570 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:11.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:11.621+0000 7f4ee5089740 1 -- 172.21.15.5:0/796801130 >> 172.21.15.5:0/796801130 conn(0x560e34525940 msgr2=0x560e34524c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:58:11.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:11.622+0000 7f4ee5089740 1 -- 172.21.15.5:0/796801130 shutdown_connections 2024-09-10T09:58:11.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:11.622+0000 7f4ee5089740 1 -- 172.21.15.5:0/796801130 wait complete. 2024-09-10T09:58:11.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:11.622+0000 7f4ee5089740 20 client.29248 trim_cache size 0 max 16384 2024-09-10T09:58:11.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:11.645+0000 7f4ee5089740 1 Processor -- start 2024-09-10T09:58:11.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:11.645+0000 7f4ee5089740 1 -- start start 2024-09-10T09:58:11.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:11.645+0000 7f4ee5089740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560e345293b0 0x560e34530e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:11.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:11.645+0000 7f4ee5089740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560e34529cd0 0x560e3452b560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:11.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:11.645+0000 7f4ee5089740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560e34534590 0x560e3454c180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:11.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:11.645+0000 7f4ee5089740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f4edc1bdbe0 con 0x560e34529cd0 2024-09-10T09:58:11.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:11.645+0000 7f4ee5089740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f4edc1bf720 con 0x560e345293b0 2024-09-10T09:58:11.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:11.645+0000 7f4ee5089740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f4edc1be2b0 con 0x560e34534590 2024-09-10T09:58:11.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:11.646+0000 7f4edb400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560e345293b0 0x560e34530e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:11.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:11.646+0000 7f4edb400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560e345293b0 0x560e34530e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:57092/0 (socket says 172.21.15.5:57092) 2024-09-10T09:58:11.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:11.646+0000 7f4edb400640 1 -- 172.21.15.5:0/1007862054 learned_addr learned my addr 172.21.15.5:0/1007862054 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:58:11.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:11.646+0000 7f4edbe00640 1 --2- 172.21.15.5:0/1007862054 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560e34534590 0x560e3454c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:11.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:11.646+0000 7f4edaa00640 1 --2- 172.21.15.5:0/1007862054 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560e34529cd0 0x560e3452b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:11.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:11.646+0000 7f4edb400640 1 -- 172.21.15.5:0/1007862054 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560e34534590 msgr2=0x560e3454c180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:58:11.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:11.646+0000 7f4edb400640 1 --2- 172.21.15.5:0/1007862054 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560e34534590 0x560e3454c180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:11.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:11.646+0000 7f4edb400640 1 -- 172.21.15.5:0/1007862054 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560e34529cd0 msgr2=0x560e3452b560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:58:11.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:11.646+0000 7f4edb400640 1 --2- 172.21.15.5:0/1007862054 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560e34529cd0 0x560e3452b560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:11.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:11.646+0000 7f4edb400640 1 -- 172.21.15.5:0/1007862054 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560e34529b00 con 0x560e345293b0 2024-09-10T09:58:11.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:11.647+0000 7f4edaa00640 1 --2- 172.21.15.5:0/1007862054 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560e34529cd0 0x560e3452b560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:58:11.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:11.647+0000 7f4edb400640 1 --2- 172.21.15.5:0/1007862054 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560e345293b0 0x560e34530e50 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f4ecc026fd0 tx=0x7f4ecc00bc20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d9a05daff84f9498 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:11.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:11.647+0000 7f4ec7e00640 1 -- 172.21.15.5:0/1007862054 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ecc038020 con 0x560e345293b0 2024-09-10T09:58:11.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:11.648+0000 7f4ec7e00640 1 -- 172.21.15.5:0/1007862054 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4ecc026440 con 0x560e345293b0 2024-09-10T09:58:11.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:11.648+0000 7f4ec7e00640 1 -- 172.21.15.5:0/1007862054 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ecc038020 con 0x560e345293b0 2024-09-10T09:58:11.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:11.649+0000 7f4ee5089740 1 -- 172.21.15.5:0/1007862054 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560e345293b0 msgr2=0x560e34530e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:11.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:11.649+0000 7f4ee5089740 1 --2- 172.21.15.5:0/1007862054 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560e345293b0 0x560e34530e50 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f4ecc026fd0 tx=0x7f4ecc00bc20 comp rx=0 tx=0).stop 2024-09-10T09:58:11.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:11.650+0000 7f4ee5089740 1 -- 172.21.15.5:0/1007862054 shutdown_connections 2024-09-10T09:58:11.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:11.650+0000 7f4ee5089740 1 --2- 172.21.15.5:0/1007862054 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560e34534590 0x560e3454c180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:11.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:11.650+0000 7f4ee5089740 1 --2- 172.21.15.5:0/1007862054 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560e34529cd0 0x560e3452b560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:11.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:11.650+0000 7f4ee5089740 1 --2- 172.21.15.5:0/1007862054 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560e345293b0 0x560e34530e50 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:11.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:11.650+0000 7f4ee5089740 1 -- 172.21.15.5:0/1007862054 >> 172.21.15.5:0/1007862054 conn(0x560e34525940 msgr2=0x560e3452ae50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:58:11.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:11.650+0000 7f4ee5089740 1 -- 172.21.15.5:0/1007862054 shutdown_connections 2024-09-10T09:58:11.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:11.650+0000 7f4ee5089740 1 -- 172.21.15.5:0/1007862054 wait complete. 2024-09-10T09:58:11.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:11.651+0000 7f4ee5089740 1 -- start start 2024-09-10T09:58:11.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:11.651+0000 7f4ee5089740 1 Processor -- start 2024-09-10T09:58:11.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:11.651+0000 7f4ee5089740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560e345293b0 0x560e3453c730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:11.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:11.652+0000 7f4ee5089740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560e34529cd0 0x560e3453cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:11.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:11.652+0000 7f4ee5089740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560e34534590 0x560e34539520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:11.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:11.652+0000 7f4ee5089740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f4edc1bf050 con 0x560e34529cd0 2024-09-10T09:58:11.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:11.652+0000 7f4ee5089740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f4edc1be980 con 0x560e345293b0 2024-09-10T09:58:11.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:11.652+0000 7f4ee5089740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f4edc1bbbc0 con 0x560e34534590 2024-09-10T09:58:11.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:11.652+0000 7f4ee5089740 10 client.0 _ll_register_callbacks cb 0x7f4ecc01d560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:58:11.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:11.652+0000 7f4edb400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560e345293b0 0x560e3453c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:11.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:11.652+0000 7f4edb400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560e345293b0 0x560e3453c730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:57094/0 (socket says 172.21.15.5:57094) 2024-09-10T09:58:11.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:11.652+0000 7f4edb400640 1 -- 172.21.15.5:0/1565857965 learned_addr learned my addr 172.21.15.5:0/1565857965 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:58:11.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:11.652+0000 7f4edbe00640 1 --2- 172.21.15.5:0/1565857965 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560e34534590 0x560e34539520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:11.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:11.653+0000 7f4edb400640 1 -- 172.21.15.5:0/1565857965 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560e34534590 msgr2=0x560e34539520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:58:11.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:11.653+0000 7f4edb400640 1 --2- 172.21.15.5:0/1565857965 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560e34534590 0x560e34539520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:11.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:11.653+0000 7f4edb400640 1 -- 172.21.15.5:0/1565857965 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560e34529cd0 msgr2=0x560e3453cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:58:11.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:11.653+0000 7f4edb400640 1 --2- 172.21.15.5:0/1565857965 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560e34529cd0 0x560e3453cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:11.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:11.653+0000 7f4edb400640 1 -- 172.21.15.5:0/1565857965 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560e34539bb0 con 0x560e345293b0 2024-09-10T09:58:11.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:11.653+0000 7f4edb400640 1 --2- 172.21.15.5:0/1565857965 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560e345293b0 0x560e3453c730 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f4ecc0059f0 tx=0x7f4ecc026820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c50e7f6b6167f08c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:11.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:11.653+0000 7f4ec5600640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-09-10T09:58:11.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:11.654+0000 7f4ec5600640 1 -- 172.21.15.5:0/1565857965 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ecc038030 con 0x560e345293b0 2024-09-10T09:58:11.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:11.654+0000 7f4ec5600640 1 -- 172.21.15.5:0/1565857965 <== mon.2 v2:172.21.15.102:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f4ecc01b600 con 0x560e345293b0 2024-09-10T09:58:11.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:11.654+0000 7f4ee5089740 1 -- 172.21.15.5:0/1565857965 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4ecc026440 con 0x560e345293b0 2024-09-10T09:58:11.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:11.654+0000 7f4ec5600640 1 -- 172.21.15.5:0/1565857965 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f4ecc04e050 con 0x560e345293b0 2024-09-10T09:58:11.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:11.654+0000 7f4ee5089740 10 client.29261 fetch_fsmap learned FSMap version 42 2024-09-10T09:58:11.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:11.654+0000 7f4ee5089740 1 -- 172.21.15.5:0/1565857965 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x560e345afac0 con 0x560e345293b0 2024-09-10T09:58:11.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:11.654+0000 7f4ec5600640 1 -- 172.21.15.5:0/1565857965 <== mon.2 v2:172.21.15.102:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4ecc026440 con 0x560e345293b0 2024-09-10T09:58:11.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:11.654+0000 7f4ec5600640 1 -- 172.21.15.5:0/1565857965 <== mon.2 v2:172.21.15.102:3300/0 5 ==== fsmap(e 42) ==== 8370+0+0 (secure 0 0 0) 0x7f4ecc024050 con 0x560e345293b0 2024-09-10T09:58:11.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:11.655+0000 7f4ee5089740 10 client.29261 fetch_fsmap finished waiting for FSMap version 42 2024-09-10T09:58:11.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:11.655+0000 7f4ee5089740 10 client.29261 resolve_mds: validated gid 24542 aka daemon mds.b 2024-09-10T09:58:11.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:11.655+0000 7f4ee5089740 20 client.29261 populate_metadata read hostname 'smithi005' 2024-09-10T09:58:11.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:11.655+0000 7f4ee5089740 1 --2- 172.21.15.5:0/1565857965 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x560e345443a0 0x560e345b1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:11.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:11.655+0000 7f4ee5089740 4 client.29261 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 132.8, "await_for": 45.0}] 2024-09-10T09:58:11.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:11.655+0000 7f4ee5089740 1 -- 172.21.15.5:0/1565857965 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 132.8, "await_for": 45.0}) -- 0x560e3449cc60 con 0x560e345443a0 2024-09-10T09:58:11.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:11.655+0000 7f4edaa00640 1 --2- 172.21.15.5:0/1565857965 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x560e345443a0 0x560e345b1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:11.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:11.656+0000 7f4edaa00640 1 --2- 172.21.15.5:0/1565857965 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x560e345443a0 0x560e345b1ff0 crc :-1 s=READY pgs=278 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e3186f03a5e29c97 server_cookie=5a39870b04bd0d53 in_seq=0 out_seq=0 2024-09-10T09:58:11.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:11.656+0000 7f4ec5600640 10 client.29261 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T09:58:11.851 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:58:11 smithi159 ceph-mon[35513]: pgmap v988: 129 pgs: 129 active+clean; 673 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 6.5 MiB/s wr, 541 op/s 2024-09-10T09:58:11.851 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:58:11 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:58:11.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:58:11 smithi102 ceph-mon[35442]: pgmap v988: 129 pgs: 129 active+clean; 673 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 6.5 MiB/s wr, 541 op/s 2024-09-10T09:58:11.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:58:11 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:58:12.019 INFO:tasks.workunit.client.0.smithi005.stdout: 10 107573 16.08 MB/sec execute 230 sec latency 318.618 ms 2024-09-10T09:58:12.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:12.360+0000 7ffa2b200640 1 -- 172.21.15.5:0/1961422333 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ffa2c102b90 msgr2=0x7ffa2c102f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:12.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:12.360+0000 7ffa2b200640 1 --2- 172.21.15.5:0/1961422333 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ffa2c102b90 0x7ffa2c102f70 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7ffa1c009900 tx=0x7ffa1c02a480 comp rx=0 tx=0).stop 2024-09-10T09:58:12.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:12.360+0000 7ffa2b200640 1 -- 172.21.15.5:0/1961422333 shutdown_connections 2024-09-10T09:58:12.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:12.360+0000 7ffa2b200640 1 --2- 172.21.15.5:0/1961422333 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ffa2c10bfc0 0x7ffa2c10e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:12.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:12.360+0000 7ffa2b200640 1 --2- 172.21.15.5:0/1961422333 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffa2c1034b0 0x7ffa2c10b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:12.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:12.360+0000 7ffa2b200640 1 --2- 172.21.15.5:0/1961422333 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ffa2c102b90 0x7ffa2c102f70 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:12.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:12.360+0000 7ffa2b200640 1 -- 172.21.15.5:0/1961422333 >> 172.21.15.5:0/1961422333 conn(0x7ffa2c0fc610 msgr2=0x7ffa2c0fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:58:12.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:12.360+0000 7ffa2b200640 1 -- 172.21.15.5:0/1961422333 shutdown_connections 2024-09-10T09:58:12.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:12.361+0000 7ffa2b200640 1 -- 172.21.15.5:0/1961422333 wait complete. 2024-09-10T09:58:12.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:12.362+0000 7ffa2b200640 1 Processor -- start 2024-09-10T09:58:12.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:12.363+0000 7ffa2b200640 1 -- start start 2024-09-10T09:58:12.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:12.364+0000 7ffa2b200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ffa2c102b90 0x7ffa2c19a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:12.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:12.364+0000 7ffa2b200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ffa2c1034b0 0x7ffa2c19a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:12.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:12.364+0000 7ffa2b200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffa2c10bfc0 0x7ffa2c19ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:12.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:12.364+0000 7ffa2b200640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ffa2c111e10 con 0x7ffa2c10bfc0 2024-09-10T09:58:12.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:12.365+0000 7ffa29e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ffa2c102b90 0x7ffa2c19a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:12.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:12.364+0000 7ffa2b200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7ffa2c111c90 con 0x7ffa2c102b90 2024-09-10T09:58:12.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:12.365+0000 7ffa2b200640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7ffa2c111f90 con 0x7ffa2c1034b0 2024-09-10T09:58:12.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:12.365+0000 7ffa29400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ffa2c1034b0 0x7ffa2c19a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:12.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:12.365+0000 7ffa29e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ffa2c102b90 0x7ffa2c19a490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:57110/0 (socket says 172.21.15.5:57110) 2024-09-10T09:58:12.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:12.365+0000 7ffa29e00640 1 -- 172.21.15.5:0/806998825 learned_addr learned my addr 172.21.15.5:0/806998825 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:58:12.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:12.366+0000 7ffa29e00640 1 -- 172.21.15.5:0/806998825 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ffa2c1034b0 msgr2=0x7ffa2c19a9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:12.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:12.366+0000 7ffa29e00640 1 --2- 172.21.15.5:0/806998825 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ffa2c1034b0 0x7ffa2c19a9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:12.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:12.366+0000 7ffa29e00640 1 -- 172.21.15.5:0/806998825 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffa2c10bfc0 msgr2=0x7ffa2c19ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:12.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:12.366+0000 7ffa29e00640 1 --2- 172.21.15.5:0/806998825 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffa2c10bfc0 0x7ffa2c19ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:12.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:12.366+0000 7ffa29e00640 1 -- 172.21.15.5:0/806998825 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa2c19f440 con 0x7ffa2c102b90 2024-09-10T09:58:12.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:12.366+0000 7ffa29e00640 1 --2- 172.21.15.5:0/806998825 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ffa2c102b90 0x7ffa2c19a490 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7ffa1c02aa70 tx=0x7ffa1c003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:12.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:12.368+0000 7ffa22a00640 1 -- 172.21.15.5:0/806998825 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa1c03f070 con 0x7ffa2c102b90 2024-09-10T09:58:12.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:12.368+0000 7ffa22a00640 1 -- 172.21.15.5:0/806998825 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa1c031ca0 con 0x7ffa2c102b90 2024-09-10T09:58:12.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:12.368+0000 7ffa22a00640 1 -- 172.21.15.5:0/806998825 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa1c035030 con 0x7ffa2c102b90 2024-09-10T09:58:12.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:12.368+0000 7ffa2b200640 1 -- 172.21.15.5:0/806998825 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa2c19f6d0 con 0x7ffa2c102b90 2024-09-10T09:58:12.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:12.368+0000 7ffa2b200640 1 -- 172.21.15.5:0/806998825 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa2c19fb60 con 0x7ffa2c102b90 2024-09-10T09:58:12.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:12.372+0000 7ffa33879740 1 Processor -- start 2024-09-10T09:58:12.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:12.372+0000 7ffa22a00640 1 -- 172.21.15.5:0/806998825 <== mon.2 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7ffa1c042070 con 0x7ffa2c102b90 2024-09-10T09:58:12.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:12.372+0000 7ffa33879740 1 -- start start 2024-09-10T09:58:12.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:12.373+0000 7ffa33879740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c0c936dd40 0x55c0c936e120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:12.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:12.373+0000 7ffa33879740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c0c936e660 0x55c0c93812f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:12.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:12.373+0000 7ffa33879740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c0c9381830 0x55c0c9383c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:12.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:12.373+0000 7ffa33879740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55c0c92d43e0 con 0x55c0c936e660 2024-09-10T09:58:12.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:12.373+0000 7ffa33879740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55c0c927bd80 con 0x55c0c936dd40 2024-09-10T09:58:12.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:12.373+0000 7ffa33879740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55c0c92d03a0 con 0x55c0c9381830 2024-09-10T09:58:12.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:12.373+0000 7ffa2a800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c0c936dd40 0x55c0c936e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:12.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:12.373+0000 7ffa2a800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c0c936dd40 0x55c0c936e120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:57118/0 (socket says 172.21.15.5:57118) 2024-09-10T09:58:12.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:12.373+0000 7ffa2a800640 1 -- 172.21.15.5:0/983872554 learned_addr learned my addr 172.21.15.5:0/983872554 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:58:12.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:12.373+0000 7ffa29400640 1 --2- 172.21.15.5:0/983872554 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c0c9381830 0x55c0c9383c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:12.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:12.374+0000 7ffa2a800640 1 -- 172.21.15.5:0/983872554 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c0c9381830 msgr2=0x55c0c9383c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:58:12.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:12.374+0000 7ffa29e00640 1 --2- 172.21.15.5:0/983872554 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c0c936e660 0x55c0c93812f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:12.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:12.374+0000 7ffa2a800640 1 --2- 172.21.15.5:0/983872554 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c0c9381830 0x55c0c9383c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:12.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:12.374+0000 7ffa2a800640 1 -- 172.21.15.5:0/983872554 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c0c936e660 msgr2=0x55c0c93812f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:58:12.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:12.374+0000 7ffa2a800640 1 --2- 172.21.15.5:0/983872554 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c0c936e660 0x55c0c93812f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:12.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:12.374+0000 7ffa2a800640 1 -- 172.21.15.5:0/983872554 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c0c929aae0 con 0x55c0c936dd40 2024-09-10T09:58:12.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:12.374+0000 7ffa29400640 1 --2- 172.21.15.5:0/983872554 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c0c9381830 0x55c0c9383c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T09:58:12.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:12.374+0000 7ffa29e00640 1 --2- 172.21.15.5:0/983872554 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c0c936e660 0x55c0c93812f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T09:58:12.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:12.374+0000 7ffa22a00640 1 --2- 172.21.15.5:0/806998825 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7ffa100a2cd0 0x7ffa100a5190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:12.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:12.374+0000 7ffa2a800640 1 --2- 172.21.15.5:0/983872554 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c0c936dd40 0x55c0c936e120 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7ffa24009e20 tx=0x7ffa2400e720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=36607e7f18a048c9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:12.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:12.374+0000 7ffa22a00640 1 -- 172.21.15.5:0/806998825 <== mon.2 v2:172.21.15.102:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7ffa1c0efb20 con 0x7ffa2c102b90 2024-09-10T09:58:12.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:12.375+0000 7ffa29e00640 1 --2- 172.21.15.5:0/806998825 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7ffa100a2cd0 0x7ffa100a5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:12.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:12.375+0000 7ffa17000640 1 -- 172.21.15.5:0/983872554 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa24004d20 con 0x55c0c936dd40 2024-09-10T09:58:12.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:12.375+0000 7ffa17000640 1 -- 172.21.15.5:0/983872554 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa2400ee70 con 0x55c0c936dd40 2024-09-10T09:58:12.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:12.375+0000 7ffa17000640 1 -- 172.21.15.5:0/983872554 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa24018900 con 0x55c0c936dd40 2024-09-10T09:58:12.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:12.376+0000 7ffa33879740 1 -- 172.21.15.5:0/983872554 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c0c936dd40 msgr2=0x55c0c936e120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:12.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:12.376+0000 7ffa29e00640 1 --2- 172.21.15.5:0/806998825 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7ffa100a2cd0 0x7ffa100a5190 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7ffa1c0ed5a0 tx=0x7ffa1c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:12.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:12.376+0000 7ffa33879740 1 --2- 172.21.15.5:0/983872554 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c0c936dd40 0x55c0c936e120 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7ffa24009e20 tx=0x7ffa2400e720 comp rx=0 tx=0).stop 2024-09-10T09:58:12.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:12.376+0000 7ffa33879740 1 -- 172.21.15.5:0/983872554 shutdown_connections 2024-09-10T09:58:12.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:12.376+0000 7ffa33879740 1 --2- 172.21.15.5:0/983872554 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c0c9381830 0x55c0c9383c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:12.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:12.376+0000 7ffa33879740 1 --2- 172.21.15.5:0/983872554 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c0c936e660 0x55c0c93812f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:12.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:12.376+0000 7ffa33879740 1 --2- 172.21.15.5:0/983872554 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c0c936dd40 0x55c0c936e120 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:12.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:12.376+0000 7ffa33879740 1 -- 172.21.15.5:0/983872554 >> 172.21.15.5:0/983872554 conn(0x55c0c936a3d0 msgr2=0x55c0c936a7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:58:12.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:12.376+0000 7ffa33879740 1 -- 172.21.15.5:0/983872554 shutdown_connections 2024-09-10T09:58:12.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:12.376+0000 7ffa33879740 1 -- 172.21.15.5:0/983872554 wait complete. 2024-09-10T09:58:12.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:12.377+0000 7ffa33879740 1 -- start start 2024-09-10T09:58:12.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:12.377+0000 7ffa33879740 1 Processor -- start 2024-09-10T09:58:12.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:12.378+0000 7ffa33879740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c0c936dd40 0x55c0c937e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:12.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:12.378+0000 7ffa33879740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c0c936e660 0x55c0c937eca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:12.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:12.378+0000 7ffa33879740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c0c9381830 0x55c0c937f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:12.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:12.378+0000 7ffa33879740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55c0c9384460 con 0x55c0c936dd40 2024-09-10T09:58:12.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:12.378+0000 7ffa33879740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55c0c928f0f0 con 0x55c0c936e660 2024-09-10T09:58:12.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:12.378+0000 7ffa33879740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55c0c93845e0 con 0x55c0c9381830 2024-09-10T09:58:12.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:12.378+0000 7ffa33879740 10 client.0 _ll_register_callbacks cb 0x55c0c9276bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:58:12.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:12.379+0000 7ffa2a800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c0c936dd40 0x55c0c937e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:12.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:12.379+0000 7ffa2a800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c0c936dd40 0x55c0c937e760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49430/0 (socket says 172.21.15.5:49430) 2024-09-10T09:58:12.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:12.379+0000 7ffa2a800640 1 -- 172.21.15.5:0/3929767079 learned_addr learned my addr 172.21.15.5:0/3929767079 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:58:12.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:12.379+0000 7ffa2a800640 1 --2- 172.21.15.5:0/3929767079 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c0c9381830 0x55c0c937f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:12.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:12.379+0000 7ffa29400640 1 --2- 172.21.15.5:0/3929767079 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c0c936e660 0x55c0c937eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:12.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:12.380+0000 7ffa29400640 1 -- 172.21.15.5:0/3929767079 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c0c9381830 msgr2=0x55c0c937f1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:58:12.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:12.380+0000 7ffa29400640 1 --2- 172.21.15.5:0/3929767079 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c0c9381830 0x55c0c937f1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:12.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:12.380+0000 7ffa29400640 1 -- 172.21.15.5:0/3929767079 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c0c936dd40 msgr2=0x55c0c937e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:58:12.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:12.380+0000 7ffa29400640 1 --2- 172.21.15.5:0/3929767079 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c0c936dd40 0x55c0c937e760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:12.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:12.380+0000 7ffa29400640 1 -- 172.21.15.5:0/3929767079 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c0c90453c0 con 0x55c0c936e660 2024-09-10T09:58:12.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:12.380+0000 7ffa2a800640 1 --2- 172.21.15.5:0/3929767079 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c0c9381830 0x55c0c937f1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:58:12.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:12.380+0000 7ffa29400640 1 --2- 172.21.15.5:0/3929767079 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c0c936e660 0x55c0c937eca0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7ffa2c10e910 tx=0x7ffa180117b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a0625c3d31a49256 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:12.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:12.380+0000 7ffa0b400640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-09-10T09:58:12.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:12.381+0000 7ffa0b400640 1 -- 172.21.15.5:0/3929767079 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa18004220 con 0x55c0c936e660 2024-09-10T09:58:12.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:12.381+0000 7ffa33879740 1 -- 172.21.15.5:0/3929767079 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffa2400ee70 con 0x55c0c936e660 2024-09-10T09:58:12.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:12.381+0000 7ffa0b400640 1 -- 172.21.15.5:0/3929767079 <== mon.2 v2:172.21.15.102:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7ffa180043c0 con 0x55c0c936e660 2024-09-10T09:58:12.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:12.381+0000 7ffa0b400640 1 -- 172.21.15.5:0/3929767079 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7ffa18025050 con 0x55c0c936e660 2024-09-10T09:58:12.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:12.381+0000 7ffa33879740 10 client.29273 fetch_fsmap learned FSMap version 42 2024-09-10T09:58:12.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:12.381+0000 7ffa33879740 1 -- 172.21.15.5:0/3929767079 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55c0c9390580 con 0x55c0c936e660 2024-09-10T09:58:12.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:12.382+0000 7ffa0b400640 1 -- 172.21.15.5:0/3929767079 <== mon.2 v2:172.21.15.102:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa2400ee70 con 0x55c0c936e660 2024-09-10T09:58:12.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:12.383+0000 7ffa0b400640 1 -- 172.21.15.5:0/3929767079 <== mon.2 v2:172.21.15.102:3300/0 5 ==== fsmap(e 42) ==== 8370+0+0 (secure 0 0 0) 0x7ffa18003980 con 0x55c0c936e660 2024-09-10T09:58:12.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:12.383+0000 7ffa33879740 10 client.29273 fetch_fsmap finished waiting for FSMap version 42 2024-09-10T09:58:12.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:12.383+0000 7ffa33879740 10 client.29273 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-10T09:58:12.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:12.383+0000 7ffa33879740 20 client.29273 populate_metadata read hostname 'smithi005' 2024-09-10T09:58:12.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:12.383+0000 7ffa33879740 1 --2- 172.21.15.5:0/3929767079 >> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] conn(0x55c0c9391990 0x55c0c9393dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:12.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:12.383+0000 7ffa33879740 4 client.29273 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:58:12.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:12.383+0000 7ffa33879740 1 -- 172.21.15.5:0/3929767079 --> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c0c92e1950 con 0x55c0c9391990 2024-09-10T09:58:12.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:12.383+0000 7ffa2a800640 1 --2- 172.21.15.5:0/3929767079 >> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] conn(0x55c0c9391990 0x55c0c9393dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:12.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:12.384+0000 7ffa2a800640 1 --2- 172.21.15.5:0/3929767079 >> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] conn(0x55c0c9391990 0x55c0c9393dd0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ea03d91c1d39d845 server_cookie=3e9f7e4737b9dab4 in_seq=0 out_seq=0 2024-09-10T09:58:12.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:12.384+0000 7ffa0b400640 10 client.29273 ms_handle_connect on v2:172.21.15.159:6836/620627639 2024-09-10T09:58:12.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:12.387+0000 7ffa0b400640 1 -- 172.21.15.5:0/3929767079 <== mds.2 v2:172.21.15.159:6836/620627639 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c0c92e1950 con 0x55c0c9391990 2024-09-10T09:58:12.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:12.387+0000 7ffa0b400640 10 client.29273 handle_command_reply: tid=0 2024-09-10T09:58:12.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:12.387+0000 7ffa33879740 1 client.29273 shutdown 2024-09-10T09:58:12.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:12.387+0000 7ffa33879740 1 -- 172.21.15.5:0/3929767079 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c0c936e660 msgr2=0x55c0c937eca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:12.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:12.387+0000 7ffa33879740 1 --2- 172.21.15.5:0/3929767079 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c0c936e660 0x55c0c937eca0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7ffa2c10e910 tx=0x7ffa180117b0 comp rx=0 tx=0).stop 2024-09-10T09:58:12.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:12.387+0000 7ffa33879740 1 -- 172.21.15.5:0/3929767079 shutdown_connections 2024-09-10T09:58:12.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:12.387+0000 7ffa33879740 1 --2- 172.21.15.5:0/3929767079 >> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] conn(0x55c0c9391990 0x55c0c9393dd0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:12.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:12.387+0000 7ffa33879740 1 --2- 172.21.15.5:0/3929767079 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c0c9381830 0x55c0c937f1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:12.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:12.387+0000 7ffa33879740 1 --2- 172.21.15.5:0/3929767079 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c0c936e660 0x55c0c937eca0 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:12.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:12.387+0000 7ffa33879740 1 --2- 172.21.15.5:0/3929767079 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c0c936dd40 0x55c0c937e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:12.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:12.387+0000 7ffa33879740 1 -- 172.21.15.5:0/3929767079 >> 172.21.15.5:0/3929767079 conn(0x55c0c936a3d0 msgr2=0x55c0c9370130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:58:12.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:12.387+0000 7ffa0b400640 0 client.29273 ms_handle_reset on v2:172.21.15.159:6836/620627639 2024-09-10T09:58:12.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:12.388+0000 7ffa33879740 1 -- 172.21.15.5:0/3929767079 shutdown_connections 2024-09-10T09:58:12.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:12.388+0000 7ffa33879740 1 -- 172.21.15.5:0/3929767079 wait complete. 2024-09-10T09:58:12.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:12.388+0000 7ffa33879740 20 client.29273 trim_cache size 0 max 16384 2024-09-10T09:58:12.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:12.407+0000 7ffa33879740 1 Processor -- start 2024-09-10T09:58:12.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:12.408+0000 7ffa33879740 1 -- start start 2024-09-10T09:58:12.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:12.409+0000 7ffa33879740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c0c936dd40 0x55c0c9365300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:12.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:12.409+0000 7ffa33879740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c0c936e660 0x55c0c93996e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:12.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:12.409+0000 7ffa33879740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c0c9381830 0x55c0c9399c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:12.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:12.409+0000 7ffa33879740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ffa2c1acae0 con 0x55c0c936e660 2024-09-10T09:58:12.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:12.409+0000 7ffa33879740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7ffa2c1ae620 con 0x55c0c9381830 2024-09-10T09:58:12.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:12.409+0000 7ffa33879740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7ffa2c1ad1b0 con 0x55c0c936dd40 2024-09-10T09:58:12.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:12.409+0000 7ffa2a800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c0c936dd40 0x55c0c9365300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:12.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:12.409+0000 7ffa2a800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c0c936dd40 0x55c0c9365300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:46866/0 (socket says 172.21.15.5:46866) 2024-09-10T09:58:12.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:12.409+0000 7ffa2a800640 1 -- 172.21.15.5:0/1337343504 learned_addr learned my addr 172.21.15.5:0/1337343504 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:58:12.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:12.409+0000 7ffa2a800640 1 --2- 172.21.15.5:0/1337343504 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c0c9381830 0x55c0c9399c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:12.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:12.409+0000 7ffa29400640 1 --2- 172.21.15.5:0/1337343504 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c0c936e660 0x55c0c93996e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:12.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:12.410+0000 7ffa29400640 1 -- 172.21.15.5:0/1337343504 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c0c936dd40 msgr2=0x55c0c9365300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:58:12.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:12.410+0000 7ffa29400640 1 --2- 172.21.15.5:0/1337343504 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c0c936dd40 0x55c0c9365300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:12.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:12.410+0000 7ffa29400640 1 -- 172.21.15.5:0/1337343504 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c0c9381830 msgr2=0x55c0c9399c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:58:12.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:12.410+0000 7ffa29400640 1 --2- 172.21.15.5:0/1337343504 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c0c9381830 0x55c0c9399c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:12.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:12.410+0000 7ffa29400640 1 -- 172.21.15.5:0/1337343504 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c0c937d710 con 0x55c0c936e660 2024-09-10T09:58:12.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:12.410+0000 7ffa29400640 1 --2- 172.21.15.5:0/1337343504 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c0c936e660 0x55c0c93996e0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7ffa18005d00 tx=0x7ffa18005bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=380b87b399c6d12c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:12.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:12.411+0000 7ffa16600640 1 -- 172.21.15.5:0/1337343504 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa18004020 con 0x55c0c936e660 2024-09-10T09:58:12.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:12.411+0000 7ffa16600640 1 -- 172.21.15.5:0/1337343504 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa1802d070 con 0x55c0c936e660 2024-09-10T09:58:12.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:12.411+0000 7ffa16600640 1 -- 172.21.15.5:0/1337343504 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa180052a0 con 0x55c0c936e660 2024-09-10T09:58:12.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:12.412+0000 7ffa33879740 1 -- 172.21.15.5:0/1337343504 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c0c936e660 msgr2=0x55c0c93996e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:12.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:12.412+0000 7ffa33879740 1 --2- 172.21.15.5:0/1337343504 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c0c936e660 0x55c0c93996e0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7ffa18005d00 tx=0x7ffa18005bb0 comp rx=0 tx=0).stop 2024-09-10T09:58:12.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:12.412+0000 7ffa33879740 1 -- 172.21.15.5:0/1337343504 shutdown_connections 2024-09-10T09:58:12.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:12.412+0000 7ffa33879740 1 --2- 172.21.15.5:0/1337343504 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c0c9381830 0x55c0c9399c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:12.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:12.412+0000 7ffa33879740 1 --2- 172.21.15.5:0/1337343504 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c0c936e660 0x55c0c93996e0 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:12.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:12.412+0000 7ffa33879740 1 --2- 172.21.15.5:0/1337343504 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c0c936dd40 0x55c0c9365300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:12.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:12.412+0000 7ffa33879740 1 -- 172.21.15.5:0/1337343504 >> 172.21.15.5:0/1337343504 conn(0x55c0c936a3d0 msgr2=0x55c0c93807e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:58:12.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:12.413+0000 7ffa33879740 1 -- 172.21.15.5:0/1337343504 shutdown_connections 2024-09-10T09:58:12.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:12.413+0000 7ffa33879740 1 -- 172.21.15.5:0/1337343504 wait complete. 2024-09-10T09:58:12.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:12.414+0000 7ffa33879740 1 -- start start 2024-09-10T09:58:12.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:12.414+0000 7ffa33879740 1 Processor -- start 2024-09-10T09:58:12.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:12.414+0000 7ffa33879740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c0c936dd40 0x55c0c9385620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:12.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:12.415+0000 7ffa33879740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c0c936e660 0x55c0c9385b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:12.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:12.415+0000 7ffa33879740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c0c9381830 0x55c0c93860a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:12.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:12.415+0000 7ffa33879740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ffa2c1adf50 con 0x55c0c9381830 2024-09-10T09:58:12.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:12.415+0000 7ffa33879740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7ffa2c1ad880 con 0x55c0c936e660 2024-09-10T09:58:12.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:12.415+0000 7ffa33879740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7ffa2c1aab20 con 0x55c0c936dd40 2024-09-10T09:58:12.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:12.415+0000 7ffa33879740 10 client.0 _ll_register_callbacks cb 0x7ffa1800bb40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:58:12.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:12.415+0000 7ffa29400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c0c936e660 0x55c0c9385b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:12.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:12.415+0000 7ffa2a800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c0c9381830 0x55c0c93860a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:12.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:12.415+0000 7ffa29400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c0c936e660 0x55c0c9385b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:57142/0 (socket says 172.21.15.5:57142) 2024-09-10T09:58:12.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:12.415+0000 7ffa29400640 1 -- 172.21.15.5:0/3632947005 learned_addr learned my addr 172.21.15.5:0/3632947005 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:58:12.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:12.416+0000 7ffa2a800640 1 --2- 172.21.15.5:0/3632947005 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c0c936dd40 0x55c0c9385620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:12.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:12.416+0000 7ffa29400640 1 -- 172.21.15.5:0/3632947005 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c0c936dd40 msgr2=0x55c0c9385620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:58:12.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:12.416+0000 7ffa29400640 1 --2- 172.21.15.5:0/3632947005 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c0c936dd40 0x55c0c9385620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:12.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:12.416+0000 7ffa29400640 1 -- 172.21.15.5:0/3632947005 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c0c9381830 msgr2=0x55c0c93860a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:58:12.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:12.416+0000 7ffa29400640 1 --2- 172.21.15.5:0/3632947005 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c0c9381830 0x55c0c93860a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:12.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:12.416+0000 7ffa29400640 1 -- 172.21.15.5:0/3632947005 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c0c93fb6e0 con 0x55c0c936e660 2024-09-10T09:58:12.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:12.416+0000 7ffa2a800640 1 --2- 172.21.15.5:0/3632947005 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c0c9381830 0x55c0c93860a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:58:12.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:12.416+0000 7ffa29400640 1 --2- 172.21.15.5:0/3632947005 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c0c936e660 0x55c0c9385b60 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7ffa18000c00 tx=0x7ffa180042d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=70b885c201caabf2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:12.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:12.417+0000 7ffa0b400640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-09-10T09:58:12.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:12.417+0000 7ffa0b400640 1 -- 172.21.15.5:0/3632947005 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa18035070 con 0x55c0c936e660 2024-09-10T09:58:12.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:12.417+0000 7ffa0b400640 1 -- 172.21.15.5:0/3632947005 <== mon.2 v2:172.21.15.102:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7ffa18003a00 con 0x55c0c936e660 2024-09-10T09:58:12.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:12.417+0000 7ffa33879740 1 -- 172.21.15.5:0/3632947005 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffa1802d070 con 0x55c0c936e660 2024-09-10T09:58:12.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:12.417+0000 7ffa0b400640 1 -- 172.21.15.5:0/3632947005 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7ffa1803f050 con 0x55c0c936e660 2024-09-10T09:58:12.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:12.418+0000 7ffa33879740 10 client.29279 fetch_fsmap learned FSMap version 42 2024-09-10T09:58:12.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:12.418+0000 7ffa33879740 1 -- 172.21.15.5:0/3632947005 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55c0c93fc9a0 con 0x55c0c936e660 2024-09-10T09:58:12.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:12.418+0000 7ffa0b400640 1 -- 172.21.15.5:0/3632947005 <== mon.2 v2:172.21.15.102:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa1802d070 con 0x55c0c936e660 2024-09-10T09:58:12.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:12.418+0000 7ffa0b400640 1 -- 172.21.15.5:0/3632947005 <== mon.2 v2:172.21.15.102:3300/0 5 ==== fsmap(e 42) ==== 8370+0+0 (secure 0 0 0) 0x7ffa180047a0 con 0x55c0c936e660 2024-09-10T09:58:12.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:12.418+0000 7ffa33879740 10 client.29279 fetch_fsmap finished waiting for FSMap version 42 2024-09-10T09:58:12.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:12.418+0000 7ffa33879740 10 client.29279 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-10T09:58:12.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:12.418+0000 7ffa33879740 20 client.29279 populate_metadata read hostname 'smithi005' 2024-09-10T09:58:12.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:12.418+0000 7ffa33879740 1 --2- 172.21.15.5:0/3632947005 >> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] conn(0x55c0c93fd000 0x55c0c93ff420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:12.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:12.418+0000 7ffa33879740 4 client.29279 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T09:58:12.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:12.418+0000 7ffa33879740 1 -- 172.21.15.5:0/3632947005 --> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c0c92dbe60 con 0x55c0c93fd000 2024-09-10T09:58:12.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:12.419+0000 7ffa2a800640 1 --2- 172.21.15.5:0/3632947005 >> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] conn(0x55c0c93fd000 0x55c0c93ff420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:12.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:12.420+0000 7ffa2a800640 1 --2- 172.21.15.5:0/3632947005 >> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] conn(0x55c0c93fd000 0x55c0c93ff420 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e2f21015f837a349 server_cookie=5f81823ae1153e9e in_seq=0 out_seq=0 2024-09-10T09:58:12.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:12.420+0000 7ffa0b400640 10 client.29279 ms_handle_connect on v2:172.21.15.159:6836/620627639 2024-09-10T09:58:12.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:12.422+0000 7ffa0b400640 1 -- 172.21.15.5:0/3632947005 <== mds.2 v2:172.21.15.159:6836/620627639 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c0c92dbe60 con 0x55c0c93fd000 2024-09-10T09:58:12.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:12.422+0000 7ffa0b400640 10 client.29279 handle_command_reply: tid=0 2024-09-10T09:58:12.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:12.422+0000 7ffa33879740 1 client.29279 shutdown 2024-09-10T09:58:12.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:12.423+0000 7ffa33879740 1 -- 172.21.15.5:0/3632947005 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c0c936e660 msgr2=0x55c0c9385b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:12.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:12.423+0000 7ffa33879740 1 --2- 172.21.15.5:0/3632947005 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c0c936e660 0x55c0c9385b60 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7ffa18000c00 tx=0x7ffa180042d0 comp rx=0 tx=0).stop 2024-09-10T09:58:12.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:12.423+0000 7ffa33879740 1 -- 172.21.15.5:0/3632947005 shutdown_connections 2024-09-10T09:58:12.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:12.423+0000 7ffa33879740 1 --2- 172.21.15.5:0/3632947005 >> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] conn(0x55c0c93fd000 0x55c0c93ff420 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:12.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:12.423+0000 7ffa33879740 1 --2- 172.21.15.5:0/3632947005 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c0c9381830 0x55c0c93860a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:12.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:12.423+0000 7ffa33879740 1 --2- 172.21.15.5:0/3632947005 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c0c936e660 0x55c0c9385b60 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:12.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:12.423+0000 7ffa33879740 1 --2- 172.21.15.5:0/3632947005 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c0c936dd40 0x55c0c9385620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:12.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:12.423+0000 7ffa33879740 1 -- 172.21.15.5:0/3632947005 >> 172.21.15.5:0/3632947005 conn(0x55c0c936a3d0 msgr2=0x55c0c936f8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:58:12.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:12.423+0000 7ffa0b400640 0 client.29279 ms_handle_reset on v2:172.21.15.159:6836/620627639 2024-09-10T09:58:12.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:12.424+0000 7ffa33879740 1 -- 172.21.15.5:0/3632947005 shutdown_connections 2024-09-10T09:58:12.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:12.424+0000 7ffa33879740 1 -- 172.21.15.5:0/3632947005 wait complete. 2024-09-10T09:58:12.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:12.424+0000 7ffa33879740 20 client.29279 trim_cache size 0 max 16384 2024-09-10T09:58:12.426 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-10T09:58:12.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:12.425+0000 7ffa2b200640 1 -- 172.21.15.5:0/806998825 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7ffa100a2cd0 msgr2=0x7ffa100a5190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:12.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:12.425+0000 7ffa2b200640 1 --2- 172.21.15.5:0/806998825 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7ffa100a2cd0 0x7ffa100a5190 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7ffa1c0ed5a0 tx=0x7ffa1c009260 comp rx=0 tx=0).stop 2024-09-10T09:58:12.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:12.425+0000 7ffa2b200640 1 -- 172.21.15.5:0/806998825 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ffa2c102b90 msgr2=0x7ffa2c19a490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:12.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:12.425+0000 7ffa2b200640 1 --2- 172.21.15.5:0/806998825 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ffa2c102b90 0x7ffa2c19a490 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7ffa1c02aa70 tx=0x7ffa1c003c10 comp rx=0 tx=0).stop 2024-09-10T09:58:12.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:12.425+0000 7ffa2b200640 1 -- 172.21.15.5:0/806998825 shutdown_connections 2024-09-10T09:58:12.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:12.425+0000 7ffa2b200640 1 --2- 172.21.15.5:0/806998825 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7ffa100a2cd0 0x7ffa100a5190 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:12.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:12.425+0000 7ffa2b200640 1 --2- 172.21.15.5:0/806998825 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffa2c10bfc0 0x7ffa2c19ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:12.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:12.425+0000 7ffa2b200640 1 --2- 172.21.15.5:0/806998825 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ffa2c1034b0 0x7ffa2c19a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:12.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:12.425+0000 7ffa2b200640 1 --2- 172.21.15.5:0/806998825 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ffa2c102b90 0x7ffa2c19a490 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:12.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:12.425+0000 7ffa2b200640 1 -- 172.21.15.5:0/806998825 >> 172.21.15.5:0/806998825 conn(0x7ffa2c0fc610 msgr2=0x7ffa2c10d2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:58:12.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:12.425+0000 7ffa2b200640 1 -- 172.21.15.5:0/806998825 shutdown_connections 2024-09-10T09:58:12.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:12.425+0000 7ffa2b200640 1 -- 172.21.15.5:0/806998825 wait complete. 2024-09-10T09:58:12.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:12.744+0000 7f4ec5600640 1 -- 172.21.15.5:0/1565857965 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x560e3449cc60 con 0x560e345443a0 2024-09-10T09:58:12.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:12.744+0000 7f4ec5600640 10 client.29261 handle_command_reply: tid=0 2024-09-10T09:58:12.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:12.744+0000 7f4ee5089740 1 client.29261 shutdown 2024-09-10T09:58:12.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:12.744+0000 7f4ee5089740 1 -- 172.21.15.5:0/1565857965 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560e345293b0 msgr2=0x560e3453c730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:12.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:12.744+0000 7f4ee5089740 1 --2- 172.21.15.5:0/1565857965 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560e345293b0 0x560e3453c730 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f4ecc0059f0 tx=0x7f4ecc026820 comp rx=0 tx=0).stop 2024-09-10T09:58:12.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:12.745+0000 7f4ee5089740 1 -- 172.21.15.5:0/1565857965 shutdown_connections 2024-09-10T09:58:12.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:12.745+0000 7f4ee5089740 1 --2- 172.21.15.5:0/1565857965 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x560e345443a0 0x560e345b1ff0 crc :-1 s=READY pgs=278 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:12.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:12.745+0000 7f4ee5089740 1 --2- 172.21.15.5:0/1565857965 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560e34534590 0x560e34539520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:12.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:12.745+0000 7f4ee5089740 1 --2- 172.21.15.5:0/1565857965 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560e34529cd0 0x560e3453cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:12.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:12.745+0000 7f4ee5089740 1 --2- 172.21.15.5:0/1565857965 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560e345293b0 0x560e3453c730 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:12.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:12.745+0000 7f4ee5089740 1 -- 172.21.15.5:0/1565857965 >> 172.21.15.5:0/1565857965 conn(0x560e34525940 msgr2=0x560e3452b670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:58:12.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:12.745+0000 7f4ec5600640 0 client.29261 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T09:58:12.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:12.745+0000 7f4ee5089740 1 -- 172.21.15.5:0/1565857965 shutdown_connections 2024-09-10T09:58:12.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:12.745+0000 7f4ee5089740 1 -- 172.21.15.5:0/1565857965 wait complete. 2024-09-10T09:58:12.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:12.745+0000 7f4ee5089740 20 client.29261 trim_cache size 0 max 16384 2024-09-10T09:58:12.748 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-10T09:58:12.748 INFO:teuthology.orchestra.run.smithi005.stdout: "epoch": 42, 2024-09-10T09:58:12.748 INFO:teuthology.orchestra.run.smithi005.stdout: "leader": 24542, 2024-09-10T09:58:12.748 INFO:teuthology.orchestra.run.smithi005.stdout: "set_version": 70, 2024-09-10T09:58:12.748 INFO:teuthology.orchestra.run.smithi005.stdout: "sets": { 2024-09-10T09:58:12.748 INFO:teuthology.orchestra.run.smithi005.stdout: "b9f7a64f": { 2024-09-10T09:58:12.748 INFO:teuthology.orchestra.run.smithi005.stdout: "version": 70, 2024-09-10T09:58:12.748 INFO:teuthology.orchestra.run.smithi005.stdout: "age_ref": 0.0, 2024-09-10T09:58:12.748 INFO:teuthology.orchestra.run.smithi005.stdout: "state": { 2024-09-10T09:58:12.749 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "QUIESCED", 2024-09-10T09:58:12.749 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0 2024-09-10T09:58:12.749 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T09:58:12.749 INFO:teuthology.orchestra.run.smithi005.stdout: "timeout": 90.0, 2024-09-10T09:58:12.749 INFO:teuthology.orchestra.run.smithi005.stdout: "expiration": 132.8, 2024-09-10T09:58:12.749 INFO:teuthology.orchestra.run.smithi005.stdout: "members": { 2024-09-10T09:58:12.749 INFO:teuthology.orchestra.run.smithi005.stdout: "file:/": { 2024-09-10T09:58:12.749 INFO:teuthology.orchestra.run.smithi005.stdout: "excluded": false, 2024-09-10T09:58:12.749 INFO:teuthology.orchestra.run.smithi005.stdout: "state": { 2024-09-10T09:58:12.749 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "QUIESCED", 2024-09-10T09:58:12.749 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0 2024-09-10T09:58:12.749 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T09:58:12.749 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T09:58:12.749 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T09:58:12.749 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T09:58:12.750 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T09:58:12.750 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-10T09:58:12.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:12.746+0000 7f4ee2600640 1 -- 172.21.15.5:0/559346138 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f4ec00a2c00 msgr2=0x7f4ec00a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:12.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:12.746+0000 7f4ee2600640 1 --2- 172.21.15.5:0/559346138 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f4ec00a2c00 0x7f4ec00a50c0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x560e345267a0 tx=0x7f4ecc00b5a0 comp rx=0 tx=0).stop 2024-09-10T09:58:12.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:12.746+0000 7f4ee2600640 1 -- 172.21.15.5:0/559346138 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4edc106f20 msgr2=0x7f4edc1112c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:12.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:12.746+0000 7f4ee2600640 1 --2- 172.21.15.5:0/559346138 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4edc106f20 0x7f4edc1112c0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f4ed4005e30 tx=0x7f4ed4003f10 comp rx=0 tx=0).stop 2024-09-10T09:58:12.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:12.746+0000 7f4ee2600640 1 -- 172.21.15.5:0/559346138 shutdown_connections 2024-09-10T09:58:12.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:12.746+0000 7f4ee2600640 1 --2- 172.21.15.5:0/559346138 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f4ec00a2c00 0x7f4ec00a50c0 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:12.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:12.746+0000 7f4ee2600640 1 --2- 172.21.15.5:0/559346138 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4edc106f20 0x7f4edc1112c0 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:12.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:12.746+0000 7f4ee2600640 1 --2- 172.21.15.5:0/559346138 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4edc1027a0 0x7f4edc117740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:12.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:12.746+0000 7f4ee2600640 1 --2- 172.21.15.5:0/559346138 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4edc101df0 0x7f4edc117200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:12.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:12.746+0000 7f4ee2600640 1 -- 172.21.15.5:0/559346138 >> 172.21.15.5:0/559346138 conn(0x7f4edc0fd8b0 msgr2=0x7f4edc0fe460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:58:12.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:12.747+0000 7f4ee2600640 1 -- 172.21.15.5:0/559346138 shutdown_connections 2024-09-10T09:58:12.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:12.747+0000 7f4ee2600640 1 -- 172.21.15.5:0/559346138 wait complete. 2024-09-10T09:58:13.019 INFO:tasks.workunit.client.0.smithi005.stdout: 10 107580 16.01 MB/sec execute 231 sec latency 1318.680 ms 2024-09-10T09:58:13.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:58:13 smithi005 ceph-mon[30350]: pgmap v989: 129 pgs: 129 active+clean; 669 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 5.5 MiB/s wr, 400 op/s 2024-09-10T09:58:13.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:58:13 smithi159 ceph-mon[35513]: pgmap v989: 129 pgs: 129 active+clean; 669 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 5.5 MiB/s wr, 400 op/s 2024-09-10T09:58:13.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:58:13 smithi102 ceph-mon[35442]: pgmap v989: 129 pgs: 129 active+clean; 669 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 5.5 MiB/s wr, 400 op/s 2024-09-10T09:58:13.968 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'b9f7a64f', quiesce took 11.3 seconds. Will release after: 1.5 2024-09-10T09:58:14.019 INFO:tasks.workunit.client.0.smithi005.stdout: 10 107580 15.94 MB/sec execute 232 sec latency 2318.729 ms 2024-09-10T09:58:14.499 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-10T09:58:15.019 INFO:tasks.workunit.client.0.smithi005.stdout: 10 107580 15.87 MB/sec execute 233 sec latency 3318.776 ms 2024-09-10T09:58:15.307 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:58:15 smithi005 ceph-mon[30350]: pgmap v990: 129 pgs: 129 active+clean; 669 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 5.0 MiB/s wr, 392 op/s 2024-09-10T09:58:15.307 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:58:15 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:58:15.307 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:58:15 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:58:15.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:58:15 smithi159 ceph-mon[35513]: pgmap v990: 129 pgs: 129 active+clean; 669 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 5.0 MiB/s wr, 392 op/s 2024-09-10T09:58:15.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:58:15 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:58:15.351 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:58:15 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:58:15.469 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T09:58:15.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:58:15 smithi102 ceph-mon[35442]: pgmap v990: 129 pgs: 129 active+clean; 669 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 5.0 MiB/s wr, 392 op/s 2024-09-10T09:58:15.635 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:58:15 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:58:15.635 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:58:15 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:58:15.664 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:58:16.019 INFO:tasks.workunit.client.0.smithi005.stdout: 10 107580 15.81 MB/sec execute 234 sec latency 4318.820 ms 2024-09-10T09:58:16.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:16.254+0000 7efdaf800640 1 -- 172.21.15.5:0/1146592216 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efda8109140 msgr2=0x7efda8110cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:16.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:16.254+0000 7efdaf800640 1 --2- 172.21.15.5:0/1146592216 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efda8109140 0x7efda8110cc0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7efda0009be0 tx=0x7efda002a6c0 comp rx=0 tx=0).stop 2024-09-10T09:58:16.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:16.254+0000 7efdaf800640 1 -- 172.21.15.5:0/1146592216 shutdown_connections 2024-09-10T09:58:16.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:16.254+0000 7efdaf800640 1 --2- 172.21.15.5:0/1146592216 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efda8109140 0x7efda8110cc0 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:16.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:16.254+0000 7efdaf800640 1 --2- 172.21.15.5:0/1146592216 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efda81049c0 0x7efda8108a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:16.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:16.254+0000 7efdaf800640 1 --2- 172.21.15.5:0/1146592216 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efda8104010 0x7efda81043f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:16.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:16.254+0000 7efdaf800640 1 -- 172.21.15.5:0/1146592216 >> 172.21.15.5:0/1146592216 conn(0x7efda80ffa50 msgr2=0x7efda8101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:58:16.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:16.255+0000 7efdaf800640 1 -- 172.21.15.5:0/1146592216 shutdown_connections 2024-09-10T09:58:16.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:16.255+0000 7efdaf800640 1 -- 172.21.15.5:0/1146592216 wait complete. 2024-09-10T09:58:16.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:16.256+0000 7efdaf800640 1 Processor -- start 2024-09-10T09:58:16.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:16.256+0000 7efdaf800640 1 -- start start 2024-09-10T09:58:16.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:16.256+0000 7efdaf800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efda8104010 0x7efda8076700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:16.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:16.256+0000 7efdaf800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efda81049c0 0x7efda8076c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:16.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:16.257+0000 7efdaf800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efda8109140 0x7efda8077180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:16.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:16.257+0000 7efdaf800640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7efda8114170 con 0x7efda81049c0 2024-09-10T09:58:16.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:16.257+0000 7efdaf800640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7efda8113ff0 con 0x7efda8104010 2024-09-10T09:58:16.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:16.257+0000 7efdaf800640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7efda81142f0 con 0x7efda8109140 2024-09-10T09:58:16.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:16.257+0000 7efdace00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efda8104010 0x7efda8076700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:16.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:16.257+0000 7efdace00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efda8104010 0x7efda8076700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:57164/0 (socket says 172.21.15.5:57164) 2024-09-10T09:58:16.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:16.257+0000 7efdace00640 1 -- 172.21.15.5:0/2091923839 learned_addr learned my addr 172.21.15.5:0/2091923839 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:58:16.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:16.257+0000 7efdad800640 1 --2- 172.21.15.5:0/2091923839 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efda8109140 0x7efda8077180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:16.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:16.257+0000 7efda7e00640 1 --2- 172.21.15.5:0/2091923839 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efda81049c0 0x7efda8076c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:16.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:16.257+0000 7efdace00640 1 -- 172.21.15.5:0/2091923839 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efda8109140 msgr2=0x7efda8077180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:16.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:16.257+0000 7efdace00640 1 --2- 172.21.15.5:0/2091923839 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efda8109140 0x7efda8077180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:16.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:16.257+0000 7efdace00640 1 -- 172.21.15.5:0/2091923839 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efda81049c0 msgr2=0x7efda8076c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:16.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:16.257+0000 7efdace00640 1 --2- 172.21.15.5:0/2091923839 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efda81049c0 0x7efda8076c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:16.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:16.258+0000 7efdace00640 1 -- 172.21.15.5:0/2091923839 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efda81aacd0 con 0x7efda8104010 2024-09-10T09:58:16.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:16.258+0000 7efdace00640 1 --2- 172.21.15.5:0/2091923839 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efda8104010 0x7efda8076700 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7efd9c00ea20 tx=0x7efd9c00eee0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:16.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:16.258+0000 7efda5600640 1 -- 172.21.15.5:0/2091923839 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd9c00cd80 con 0x7efda8104010 2024-09-10T09:58:16.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:16.259+0000 7efda5600640 1 -- 172.21.15.5:0/2091923839 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efd9c005500 con 0x7efda8104010 2024-09-10T09:58:16.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:16.259+0000 7efda5600640 1 -- 172.21.15.5:0/2091923839 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd9c010690 con 0x7efda8104010 2024-09-10T09:58:16.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:16.259+0000 7efdaf800640 1 -- 172.21.15.5:0/2091923839 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efda81aafc0 con 0x7efda8104010 2024-09-10T09:58:16.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:16.259+0000 7efdaf800640 1 -- 172.21.15.5:0/2091923839 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efda81ab420 con 0x7efda8104010 2024-09-10T09:58:16.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:16.263+0000 7efdb23bd740 1 Processor -- start 2024-09-10T09:58:16.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:16.263+0000 7efdb23bd740 1 -- start start 2024-09-10T09:58:16.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:16.264+0000 7efda5600640 1 -- 172.21.15.5:0/2091923839 <== mon.2 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7efd9c0046c0 con 0x7efda8104010 2024-09-10T09:58:16.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:16.264+0000 7efdb23bd740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b00514bc70 0x55b00514c050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:16.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:16.264+0000 7efdb23bd740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b00514c590 0x55b005156910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:16.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:16.264+0000 7efdb23bd740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b005156e50 0x55b005159240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:16.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:16.264+0000 7efdb23bd740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55b0050b23e0 con 0x55b00514bc70 2024-09-10T09:58:16.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:16.264+0000 7efdb23bd740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55b005059d80 con 0x55b005156e50 2024-09-10T09:58:16.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:16.264+0000 7efdb23bd740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55b0050ae3a0 con 0x55b00514c590 2024-09-10T09:58:16.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:16.264+0000 7efdad800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b00514bc70 0x55b00514c050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:16.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:16.264+0000 7efdad800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b00514bc70 0x55b00514c050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49464/0 (socket says 172.21.15.5:49464) 2024-09-10T09:58:16.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:16.264+0000 7efdad800640 1 -- 172.21.15.5:0/4192099650 learned_addr learned my addr 172.21.15.5:0/4192099650 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:58:16.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:16.265+0000 7efdace00640 1 --2- 172.21.15.5:0/4192099650 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b00514c590 0x55b005156910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:16.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:16.265+0000 7efdad800640 1 -- 172.21.15.5:0/4192099650 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b00514c590 msgr2=0x55b005156910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:58:16.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:16.265+0000 7efdad800640 1 --2- 172.21.15.5:0/4192099650 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b00514c590 0x55b005156910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:16.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:16.265+0000 7efdad800640 1 -- 172.21.15.5:0/4192099650 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b005156e50 msgr2=0x55b005159240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:58:16.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:16.265+0000 7efdad800640 1 --2- 172.21.15.5:0/4192099650 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b005156e50 0x55b005159240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:16.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:16.265+0000 7efdad800640 1 -- 172.21.15.5:0/4192099650 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b005078ae0 con 0x55b00514bc70 2024-09-10T09:58:16.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:16.266+0000 7efdad800640 1 --2- 172.21.15.5:0/4192099650 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b00514bc70 0x55b00514c050 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7efda0009be0 tx=0x7efda0030a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9fc116d1fca8c0f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:16.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:16.266+0000 7efda5600640 1 --2- 172.21.15.5:0/2091923839 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7efd900a2c00 0x7efd900a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:16.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:16.267+0000 7efda5600640 1 -- 172.21.15.5:0/2091923839 <== mon.2 v2:172.21.15.102:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7efd9c0cf6b0 con 0x7efda8104010 2024-09-10T09:58:16.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:16.267+0000 7efd9a000640 1 -- 172.21.15.5:0/4192099650 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efda0041030 con 0x55b00514bc70 2024-09-10T09:58:16.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:16.267+0000 7efd9a000640 1 -- 172.21.15.5:0/4192099650 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efda0030c60 con 0x55b00514bc70 2024-09-10T09:58:16.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:16.267+0000 7efd9a000640 1 -- 172.21.15.5:0/4192099650 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efda0036040 con 0x55b00514bc70 2024-09-10T09:58:16.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:16.267+0000 7efdace00640 1 --2- 172.21.15.5:0/2091923839 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7efd900a2c00 0x7efd900a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:16.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:16.268+0000 7efdb23bd740 1 -- 172.21.15.5:0/4192099650 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b00514bc70 msgr2=0x55b00514c050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:16.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:16.268+0000 7efdb23bd740 1 --2- 172.21.15.5:0/4192099650 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b00514bc70 0x55b00514c050 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7efda0009be0 tx=0x7efda0030a30 comp rx=0 tx=0).stop 2024-09-10T09:58:16.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:16.268+0000 7efdb23bd740 1 -- 172.21.15.5:0/4192099650 shutdown_connections 2024-09-10T09:58:16.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:16.268+0000 7efdb23bd740 1 --2- 172.21.15.5:0/4192099650 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b005156e50 0x55b005159240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:16.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:16.268+0000 7efdb23bd740 1 --2- 172.21.15.5:0/4192099650 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b00514c590 0x55b005156910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:16.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:16.268+0000 7efdb23bd740 1 --2- 172.21.15.5:0/4192099650 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b00514bc70 0x55b00514c050 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:16.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:16.268+0000 7efdb23bd740 1 -- 172.21.15.5:0/4192099650 >> 172.21.15.5:0/4192099650 conn(0x55b005148200 msgr2=0x55b0051485e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:58:16.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:16.269+0000 7efdb23bd740 1 -- 172.21.15.5:0/4192099650 shutdown_connections 2024-09-10T09:58:16.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:16.269+0000 7efdb23bd740 1 -- 172.21.15.5:0/4192099650 wait complete. 2024-09-10T09:58:16.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:16.269+0000 7efdace00640 1 --2- 172.21.15.5:0/2091923839 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7efd900a2c00 0x7efd900a50c0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7efd9c0cd130 tx=0x7efd9c0d5550 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:16.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:16.269+0000 7efdb23bd740 1 -- start start 2024-09-10T09:58:16.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:16.270+0000 7efdb23bd740 1 Processor -- start 2024-09-10T09:58:16.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:16.270+0000 7efdb23bd740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b00514bc70 0x55b005153d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:16.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:16.270+0000 7efdb23bd740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b00514c590 0x55b0051542d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:16.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:16.270+0000 7efdb23bd740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b005156e50 0x55b005154810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:16.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:16.270+0000 7efdb23bd740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55b005159a20 con 0x55b00514bc70 2024-09-10T09:58:16.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:16.270+0000 7efdb23bd740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55b00506d0f0 con 0x55b005156e50 2024-09-10T09:58:16.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:16.270+0000 7efdb23bd740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55b005159ba0 con 0x55b00514c590 2024-09-10T09:58:16.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:16.270+0000 7efdb23bd740 10 client.0 _ll_register_callbacks cb 0x55b005054bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:58:16.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:16.271+0000 7efdad800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b00514bc70 0x55b005153d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:16.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:16.271+0000 7efda7e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b00514c590 0x55b0051542d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:16.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:16.271+0000 7efdad800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b00514bc70 0x55b005153d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49478/0 (socket says 172.21.15.5:49478) 2024-09-10T09:58:16.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:16.271+0000 7efdad800640 1 -- 172.21.15.5:0/955888048 learned_addr learned my addr 172.21.15.5:0/955888048 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:58:16.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:16.271+0000 7efdad800640 1 --2- 172.21.15.5:0/955888048 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b005156e50 0x55b005154810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:16.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:16.271+0000 7efdad800640 1 -- 172.21.15.5:0/955888048 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b00514c590 msgr2=0x55b0051542d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:58:16.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:16.271+0000 7efdad800640 1 --2- 172.21.15.5:0/955888048 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b00514c590 0x55b0051542d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:16.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:16.272+0000 7efdad800640 1 -- 172.21.15.5:0/955888048 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b005156e50 msgr2=0x55b005154810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:58:16.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:16.272+0000 7efdad800640 1 --2- 172.21.15.5:0/955888048 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b005156e50 0x55b005154810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:16.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:16.272+0000 7efdad800640 1 -- 172.21.15.5:0/955888048 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b004e233c0 con 0x55b00514bc70 2024-09-10T09:58:16.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:16.272+0000 7efdad800640 1 --2- 172.21.15.5:0/955888048 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b00514bc70 0x55b005153d90 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7efda0003dd0 tx=0x7efda0003cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9ebf3b87beea8bb server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:16.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:16.272+0000 7efd8ea00640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-10T09:58:16.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:16.272+0000 7efd8ea00640 1 -- 172.21.15.5:0/955888048 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efda0036070 con 0x55b00514bc70 2024-09-10T09:58:16.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:16.273+0000 7efd8ea00640 1 -- 172.21.15.5:0/955888048 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7efda00305c0 con 0x55b00514bc70 2024-09-10T09:58:16.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:16.273+0000 7efdb23bd740 1 -- 172.21.15.5:0/955888048 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efda0030c60 con 0x55b00514bc70 2024-09-10T09:58:16.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:16.273+0000 7efd8ea00640 1 -- 172.21.15.5:0/955888048 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7efda0058050 con 0x55b00514bc70 2024-09-10T09:58:16.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:16.273+0000 7efdb23bd740 10 client.19221 fetch_fsmap learned FSMap version 42 2024-09-10T09:58:16.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:16.273+0000 7efdb23bd740 1 -- 172.21.15.5:0/955888048 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55b005165ab0 con 0x55b00514bc70 2024-09-10T09:58:16.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:16.273+0000 7efd8ea00640 1 -- 172.21.15.5:0/955888048 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efda0030c60 con 0x55b00514bc70 2024-09-10T09:58:16.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:16.274+0000 7efd8ea00640 1 -- 172.21.15.5:0/955888048 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 42) ==== 8370+0+0 (secure 0 0 0) 0x7efda003d460 con 0x55b00514bc70 2024-09-10T09:58:16.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:16.274+0000 7efdb23bd740 10 client.19221 fetch_fsmap finished waiting for FSMap version 42 2024-09-10T09:58:16.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:16.274+0000 7efdb23bd740 10 client.19221 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-10T09:58:16.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:16.274+0000 7efdb23bd740 20 client.19221 populate_metadata read hostname 'smithi005' 2024-09-10T09:58:16.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:16.274+0000 7efdb23bd740 1 --2- 172.21.15.5:0/955888048 >> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] conn(0x55b005166e00 0x55b0051691f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:16.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:16.274+0000 7efdb23bd740 4 client.19221 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:58:16.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:16.274+0000 7efdb23bd740 1 -- 172.21.15.5:0/955888048 --> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b0050bf950 con 0x55b005166e00 2024-09-10T09:58:16.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:16.275+0000 7efda7e00640 1 --2- 172.21.15.5:0/955888048 >> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] conn(0x55b005166e00 0x55b0051691f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:16.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:16.275+0000 7efda7e00640 1 --2- 172.21.15.5:0/955888048 >> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] conn(0x55b005166e00 0x55b0051691f0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c60ed869d90be6a4 server_cookie=b615ef12dbb09c75 in_seq=0 out_seq=0 2024-09-10T09:58:16.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:16.276+0000 7efd8ea00640 10 client.19221 ms_handle_connect on v2:172.21.15.159:6838/2073739599 2024-09-10T09:58:16.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:16.276+0000 7efd8ea00640 1 -- 172.21.15.5:0/955888048 <== mds.1 v2:172.21.15.159:6838/2073739599 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b0050bf950 con 0x55b005166e00 2024-09-10T09:58:16.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:16.276+0000 7efd8ea00640 10 client.19221 handle_command_reply: tid=0 2024-09-10T09:58:16.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:16.276+0000 7efdb23bd740 1 client.19221 shutdown 2024-09-10T09:58:16.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:16.277+0000 7efdb23bd740 1 -- 172.21.15.5:0/955888048 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b00514bc70 msgr2=0x55b005153d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:16.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:16.277+0000 7efdb23bd740 1 --2- 172.21.15.5:0/955888048 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b00514bc70 0x55b005153d90 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7efda0003dd0 tx=0x7efda0003cd0 comp rx=0 tx=0).stop 2024-09-10T09:58:16.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:16.277+0000 7efdb23bd740 1 -- 172.21.15.5:0/955888048 shutdown_connections 2024-09-10T09:58:16.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:16.277+0000 7efdb23bd740 1 --2- 172.21.15.5:0/955888048 >> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] conn(0x55b005166e00 0x55b0051691f0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:16.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:16.277+0000 7efdb23bd740 1 --2- 172.21.15.5:0/955888048 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b005156e50 0x55b005154810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:16.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:16.277+0000 7efdb23bd740 1 --2- 172.21.15.5:0/955888048 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b00514c590 0x55b0051542d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:16.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:16.277+0000 7efdb23bd740 1 --2- 172.21.15.5:0/955888048 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b00514bc70 0x55b005153d90 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:16.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:16.277+0000 7efd8ea00640 0 client.19221 ms_handle_reset on v2:172.21.15.159:6838/2073739599 2024-09-10T09:58:16.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:16.277+0000 7efdb23bd740 1 -- 172.21.15.5:0/955888048 >> 172.21.15.5:0/955888048 conn(0x55b005148200 msgr2=0x55b00514df30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:58:16.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:16.277+0000 7efdb23bd740 1 -- 172.21.15.5:0/955888048 shutdown_connections 2024-09-10T09:58:16.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:16.278+0000 7efdb23bd740 1 -- 172.21.15.5:0/955888048 wait complete. 2024-09-10T09:58:16.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:16.278+0000 7efdb23bd740 20 client.19221 trim_cache size 0 max 16384 2024-09-10T09:58:16.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:16.295+0000 7efdb23bd740 1 Processor -- start 2024-09-10T09:58:16.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:16.295+0000 7efdb23bd740 1 -- start start 2024-09-10T09:58:16.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:16.296+0000 7efdb23bd740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b00514bc70 0x55b00516c210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:16.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:16.296+0000 7efdb23bd740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b00514c590 0x55b00516c750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:16.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:16.296+0000 7efdb23bd740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b005156e50 0x55b005150ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:16.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:16.296+0000 7efdb23bd740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7efda81b55f0 con 0x55b00514bc70 2024-09-10T09:58:16.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:16.296+0000 7efdb23bd740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7efda81b7130 con 0x55b00514c590 2024-09-10T09:58:16.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:16.296+0000 7efdb23bd740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7efda81b5cc0 con 0x55b005156e50 2024-09-10T09:58:16.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:16.296+0000 7efdad800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b00514bc70 0x55b00516c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:16.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:16.296+0000 7efda7e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b00514c590 0x55b00516c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:16.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:16.296+0000 7efda7e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b00514c590 0x55b00516c750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:57184/0 (socket says 172.21.15.5:57184) 2024-09-10T09:58:16.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:16.296+0000 7efda7e00640 1 -- 172.21.15.5:0/3679487264 learned_addr learned my addr 172.21.15.5:0/3679487264 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:58:16.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:16.296+0000 7efdad800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b00514bc70 0x55b00516c210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49486/0 (socket says 172.21.15.5:49486) 2024-09-10T09:58:16.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:16.296+0000 7efdad800640 1 --2- 172.21.15.5:0/3679487264 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b005156e50 0x55b005150ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:16.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:16.297+0000 7efda7e00640 1 -- 172.21.15.5:0/3679487264 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b005156e50 msgr2=0x55b005150ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:58:16.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:16.297+0000 7efda7e00640 1 --2- 172.21.15.5:0/3679487264 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b005156e50 0x55b005150ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:16.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:16.297+0000 7efda7e00640 1 -- 172.21.15.5:0/3679487264 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b00514bc70 msgr2=0x55b00516c210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:58:16.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:16.297+0000 7efda7e00640 1 --2- 172.21.15.5:0/3679487264 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b00514bc70 0x55b00516c210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:16.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:16.297+0000 7efda7e00640 1 -- 172.21.15.5:0/3679487264 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b005148e90 con 0x55b00514c590 2024-09-10T09:58:16.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:16.297+0000 7efda7e00640 1 --2- 172.21.15.5:0/3679487264 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b00514c590 0x55b00516c750 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x55b005149580 tx=0x7efd94004410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c8a37f8d298b640d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:16.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:16.297+0000 7efd99600640 1 -- 172.21.15.5:0/3679487264 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd94021070 con 0x55b00514c590 2024-09-10T09:58:16.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:16.297+0000 7efd99600640 1 -- 172.21.15.5:0/3679487264 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efd94003d60 con 0x55b00514c590 2024-09-10T09:58:16.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:16.297+0000 7efd99600640 1 -- 172.21.15.5:0/3679487264 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd94010990 con 0x55b00514c590 2024-09-10T09:58:16.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:16.298+0000 7efdb23bd740 1 -- 172.21.15.5:0/3679487264 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b00514c590 msgr2=0x55b00516c750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:16.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:16.298+0000 7efdb23bd740 1 --2- 172.21.15.5:0/3679487264 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b00514c590 0x55b00516c750 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x55b005149580 tx=0x7efd94004410 comp rx=0 tx=0).stop 2024-09-10T09:58:16.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:16.298+0000 7efdb23bd740 1 -- 172.21.15.5:0/3679487264 shutdown_connections 2024-09-10T09:58:16.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:16.298+0000 7efdb23bd740 1 --2- 172.21.15.5:0/3679487264 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b005156e50 0x55b005150ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:16.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:16.298+0000 7efdb23bd740 1 --2- 172.21.15.5:0/3679487264 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b00514c590 0x55b00516c750 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:16.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:16.298+0000 7efdb23bd740 1 --2- 172.21.15.5:0/3679487264 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b00514bc70 0x55b00516c210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:16.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:16.298+0000 7efdb23bd740 1 -- 172.21.15.5:0/3679487264 >> 172.21.15.5:0/3679487264 conn(0x55b005148200 msgr2=0x55b00514dee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:58:16.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:16.299+0000 7efdb23bd740 1 -- 172.21.15.5:0/3679487264 shutdown_connections 2024-09-10T09:58:16.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:16.299+0000 7efdb23bd740 1 -- 172.21.15.5:0/3679487264 wait complete. 2024-09-10T09:58:16.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:16.299+0000 7efdb23bd740 1 -- start start 2024-09-10T09:58:16.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:16.299+0000 7efdb23bd740 1 Processor -- start 2024-09-10T09:58:16.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:16.300+0000 7efdb23bd740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b00514bc70 0x55b00516c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:16.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:16.300+0000 7efdb23bd740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b00514c590 0x55b00516cb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:16.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:16.300+0000 7efdb23bd740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b005156e50 0x55b005169b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:16.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:16.300+0000 7efdb23bd740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7efda81b6a60 con 0x55b005156e50 2024-09-10T09:58:16.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:16.300+0000 7efdb23bd740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7efda81b6390 con 0x55b00514c590 2024-09-10T09:58:16.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:16.300+0000 7efdb23bd740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7efda81b35d0 con 0x55b00514bc70 2024-09-10T09:58:16.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:16.300+0000 7efdb23bd740 10 client.0 _ll_register_callbacks cb 0x7efd9c0d5420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:58:16.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:16.301+0000 7efda7e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b00514c590 0x55b00516cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:16.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:16.301+0000 7efda7e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b00514c590 0x55b00516cb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:57198/0 (socket says 172.21.15.5:57198) 2024-09-10T09:58:16.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:16.301+0000 7efda7e00640 1 -- 172.21.15.5:0/249173913 learned_addr learned my addr 172.21.15.5:0/249173913 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:58:16.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:16.301+0000 7efdad800640 1 --2- 172.21.15.5:0/249173913 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b00514bc70 0x55b00516c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:16.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:16.301+0000 7efdad800640 1 --2- 172.21.15.5:0/249173913 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b005156e50 0x55b005169b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:16.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:16.301+0000 7efda7e00640 1 -- 172.21.15.5:0/249173913 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b00514bc70 msgr2=0x55b00516c5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:58:16.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:16.301+0000 7efda7e00640 1 --2- 172.21.15.5:0/249173913 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b00514bc70 0x55b00516c5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:16.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:16.301+0000 7efda7e00640 1 -- 172.21.15.5:0/249173913 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b005156e50 msgr2=0x55b005169b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:58:16.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:16.301+0000 7efda7e00640 1 --2- 172.21.15.5:0/249173913 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b005156e50 0x55b005169b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:16.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:16.301+0000 7efda7e00640 1 -- 172.21.15.5:0/249173913 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b00516a380 con 0x55b00514c590 2024-09-10T09:58:16.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:16.302+0000 7efda7e00640 1 --2- 172.21.15.5:0/249173913 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b00514c590 0x55b00516cb20 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7efd94010470 tx=0x7efd94009810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9bb8808d6d1ffcad server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:16.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:16.302+0000 7efd8ea00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-09-10T09:58:16.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:16.302+0000 7efd8ea00640 1 -- 172.21.15.5:0/249173913 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd9402d060 con 0x55b00514c590 2024-09-10T09:58:16.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:16.302+0000 7efd8ea00640 1 -- 172.21.15.5:0/249173913 <== mon.2 v2:172.21.15.102:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7efd94005240 con 0x55b00514c590 2024-09-10T09:58:16.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:16.302+0000 7efdb23bd740 1 -- 172.21.15.5:0/249173913 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efd94003d60 con 0x55b00514c590 2024-09-10T09:58:16.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:16.303+0000 7efd8ea00640 1 -- 172.21.15.5:0/249173913 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7efd94021070 con 0x55b00514c590 2024-09-10T09:58:16.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:16.303+0000 7efdb23bd740 10 client.29294 fetch_fsmap learned FSMap version 42 2024-09-10T09:58:16.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:16.303+0000 7efdb23bd740 1 -- 172.21.15.5:0/249173913 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55b0051d13c0 con 0x55b00514c590 2024-09-10T09:58:16.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:16.303+0000 7efd8ea00640 1 -- 172.21.15.5:0/249173913 <== mon.2 v2:172.21.15.102:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efd94003d60 con 0x55b00514c590 2024-09-10T09:58:16.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:16.303+0000 7efd8ea00640 1 -- 172.21.15.5:0/249173913 <== mon.2 v2:172.21.15.102:3300/0 5 ==== fsmap(e 42) ==== 8370+0+0 (secure 0 0 0) 0x7efd940046c0 con 0x55b00514c590 2024-09-10T09:58:16.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:16.303+0000 7efdb23bd740 10 client.29294 fetch_fsmap finished waiting for FSMap version 42 2024-09-10T09:58:16.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:16.303+0000 7efdb23bd740 10 client.29294 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-10T09:58:16.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:16.303+0000 7efdb23bd740 20 client.29294 populate_metadata read hostname 'smithi005' 2024-09-10T09:58:16.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:16.304+0000 7efdb23bd740 1 --2- 172.21.15.5:0/249173913 >> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] conn(0x55b0051d1a30 0x55b0051d3e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:16.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:16.304+0000 7efdb23bd740 4 client.29294 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T09:58:16.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:16.304+0000 7efdb23bd740 1 -- 172.21.15.5:0/249173913 --> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b0050b9e60 con 0x55b0051d1a30 2024-09-10T09:58:16.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:16.304+0000 7efdad800640 1 --2- 172.21.15.5:0/249173913 >> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] conn(0x55b0051d1a30 0x55b0051d3e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:16.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:16.304+0000 7efdad800640 1 --2- 172.21.15.5:0/249173913 >> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] conn(0x55b0051d1a30 0x55b0051d3e50 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1158397a28ea5eb5 server_cookie=2841db0e562228a3 in_seq=0 out_seq=0 2024-09-10T09:58:16.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:16.304+0000 7efd8ea00640 10 client.29294 ms_handle_connect on v2:172.21.15.159:6838/2073739599 2024-09-10T09:58:16.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:16.305+0000 7efd8ea00640 1 -- 172.21.15.5:0/249173913 <== mds.1 v2:172.21.15.159:6838/2073739599 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b0050b9e60 con 0x55b0051d1a30 2024-09-10T09:58:16.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:16.305+0000 7efd8ea00640 10 client.29294 handle_command_reply: tid=0 2024-09-10T09:58:16.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:16.305+0000 7efdb23bd740 1 client.29294 shutdown 2024-09-10T09:58:16.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:16.305+0000 7efdb23bd740 1 -- 172.21.15.5:0/249173913 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b00514c590 msgr2=0x55b00516cb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:16.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:16.305+0000 7efdb23bd740 1 --2- 172.21.15.5:0/249173913 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b00514c590 0x55b00516cb20 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7efd94010470 tx=0x7efd94009810 comp rx=0 tx=0).stop 2024-09-10T09:58:16.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:16.305+0000 7efdb23bd740 1 -- 172.21.15.5:0/249173913 shutdown_connections 2024-09-10T09:58:16.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:16.305+0000 7efdb23bd740 1 --2- 172.21.15.5:0/249173913 >> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] conn(0x55b0051d1a30 0x55b0051d3e50 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:16.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:16.305+0000 7efdb23bd740 1 --2- 172.21.15.5:0/249173913 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b005156e50 0x55b005169b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:16.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:16.305+0000 7efdb23bd740 1 --2- 172.21.15.5:0/249173913 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b00514c590 0x55b00516cb20 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:16.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:16.305+0000 7efd8ea00640 0 client.29294 ms_handle_reset on v2:172.21.15.159:6838/2073739599 2024-09-10T09:58:16.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:16.305+0000 7efdb23bd740 1 --2- 172.21.15.5:0/249173913 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b00514bc70 0x55b00516c5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:16.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:16.305+0000 7efdb23bd740 1 -- 172.21.15.5:0/249173913 >> 172.21.15.5:0/249173913 conn(0x55b005148200 msgr2=0x55b00514d6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:58:16.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:16.305+0000 7efdb23bd740 1 -- 172.21.15.5:0/249173913 shutdown_connections 2024-09-10T09:58:16.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:16.305+0000 7efdb23bd740 1 -- 172.21.15.5:0/249173913 wait complete. 2024-09-10T09:58:16.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:16.306+0000 7efdb23bd740 20 client.29294 trim_cache size 0 max 16384 2024-09-10T09:58:16.309 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-10T09:58:16.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:16.307+0000 7efdaf800640 1 -- 172.21.15.5:0/2091923839 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7efd900a2c00 msgr2=0x7efd900a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:16.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:16.307+0000 7efdaf800640 1 --2- 172.21.15.5:0/2091923839 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7efd900a2c00 0x7efd900a50c0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7efd9c0cd130 tx=0x7efd9c0d5550 comp rx=0 tx=0).stop 2024-09-10T09:58:16.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:16.307+0000 7efdaf800640 1 -- 172.21.15.5:0/2091923839 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efda8104010 msgr2=0x7efda8076700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:16.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:16.307+0000 7efdaf800640 1 --2- 172.21.15.5:0/2091923839 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efda8104010 0x7efda8076700 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7efd9c00ea20 tx=0x7efd9c00eee0 comp rx=0 tx=0).stop 2024-09-10T09:58:16.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:16.307+0000 7efdaf800640 1 -- 172.21.15.5:0/2091923839 shutdown_connections 2024-09-10T09:58:16.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:16.307+0000 7efdaf800640 1 --2- 172.21.15.5:0/2091923839 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7efd900a2c00 0x7efd900a50c0 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:16.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:16.307+0000 7efdaf800640 1 --2- 172.21.15.5:0/2091923839 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efda8109140 0x7efda8077180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:16.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:16.307+0000 7efdaf800640 1 --2- 172.21.15.5:0/2091923839 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efda81049c0 0x7efda8076c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:16.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:16.307+0000 7efdaf800640 1 --2- 172.21.15.5:0/2091923839 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efda8104010 0x7efda8076700 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:16.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:16.307+0000 7efdaf800640 1 -- 172.21.15.5:0/2091923839 >> 172.21.15.5:0/2091923839 conn(0x7efda80ffa50 msgr2=0x7efda8100e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:58:16.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:16.307+0000 7efdaf800640 1 -- 172.21.15.5:0/2091923839 shutdown_connections 2024-09-10T09:58:16.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:16.307+0000 7efdaf800640 1 -- 172.21.15.5:0/2091923839 wait complete. 2024-09-10T09:58:16.964 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:58:17.019 INFO:tasks.workunit.client.0.smithi005.stdout: 10 107580 15.74 MB/sec execute 235 sec latency 5318.874 ms 2024-09-10T09:58:17.559 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:58:17 smithi005 ceph-mon[30350]: pgmap v991: 129 pgs: 129 active+clean; 673 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 4.9 MiB/s wr, 449 op/s 2024-09-10T09:58:17.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:58:17 smithi159 ceph-mon[35513]: pgmap v991: 129 pgs: 129 active+clean; 673 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 4.9 MiB/s wr, 449 op/s 2024-09-10T09:58:17.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:58:17 smithi102 ceph-mon[35442]: pgmap v991: 129 pgs: 129 active+clean; 673 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 4.9 MiB/s wr, 449 op/s 2024-09-10T09:58:17.817 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-10T09:58:18.019 INFO:tasks.workunit.client.0.smithi005.stdout: 10 107580 15.67 MB/sec execute 236 sec latency 6318.925 ms 2024-09-10T09:58:19.019 INFO:tasks.workunit.client.0.smithi005.stdout: 10 107580 15.61 MB/sec execute 237 sec latency 7318.982 ms 2024-09-10T09:58:19.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:58:19 smithi005 ceph-mon[30350]: pgmap v992: 129 pgs: 129 active+clean; 690 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 5.3 MiB/s wr, 378 op/s 2024-09-10T09:58:19.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:58:19 smithi159 ceph-mon[35513]: pgmap v992: 129 pgs: 129 active+clean; 690 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 5.3 MiB/s wr, 378 op/s 2024-09-10T09:58:19.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:58:19 smithi102 ceph-mon[35442]: pgmap v992: 129 pgs: 129 active+clean; 690 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 5.3 MiB/s wr, 378 op/s 2024-09-10T09:58:20.019 INFO:tasks.workunit.client.0.smithi005.stdout: 10 107580 15.54 MB/sec execute 238 sec latency 8319.040 ms 2024-09-10T09:58:21.019 INFO:tasks.workunit.client.0.smithi005.stdout: 10 107580 15.47 MB/sec execute 239 sec latency 9319.094 ms 2024-09-10T09:58:21.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:58:20 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:58:21.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:58:20 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:58:21.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:58:20 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:58:21.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:58:20 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:58:21.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:58:20 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:58:21.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:58:20 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:58:22.022 INFO:tasks.workunit.client.0.smithi005.stdout: 10 107580 15.41 MB/sec execute 240 sec latency 10321.550 ms 2024-09-10T09:58:22.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:58:21 smithi005 ceph-mon[30350]: pgmap v993: 129 pgs: 129 active+clean; 733 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 8.4 MiB/s wr, 398 op/s 2024-09-10T09:58:22.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:58:21 smithi159 ceph-mon[35513]: pgmap v993: 129 pgs: 129 active+clean; 733 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 8.4 MiB/s wr, 398 op/s 2024-09-10T09:58:22.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:58:21 smithi102 ceph-mon[35442]: pgmap v993: 129 pgs: 129 active+clean; 733 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 8.4 MiB/s wr, 398 op/s 2024-09-10T09:58:23.022 INFO:tasks.workunit.client.0.smithi005.stdout: 10 107580 15.35 MB/sec execute 241 sec latency 11321.639 ms 2024-09-10T09:58:23.819 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:58:23 smithi005 ceph-mon[30350]: pgmap v994: 129 pgs: 129 active+clean; 726 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 7.4 MiB/s wr, 331 op/s 2024-09-10T09:58:23.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:23.579+0000 7f837fe00640 1 -- 172.21.15.5:0/2440940180 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f838010e280 msgr2=0x7f8380110670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:23.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:23.579+0000 7f837fe00640 1 --2- 172.21.15.5:0/2440940180 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f838010e280 0x7f8380110670 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f8378009ca0 tx=0x7f837802a950 comp rx=0 tx=0).stop 2024-09-10T09:58:23.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:23.579+0000 7f837fe00640 1 -- 172.21.15.5:0/2440940180 shutdown_connections 2024-09-10T09:58:23.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:23.579+0000 7f837fe00640 1 --2- 172.21.15.5:0/2440940180 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f838010e280 0x7f8380110670 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:23.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:23.579+0000 7f837fe00640 1 --2- 172.21.15.5:0/2440940180 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8380074e30 0x7f838010dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:23.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:23.579+0000 7f837fe00640 1 --2- 172.21.15.5:0/2440940180 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8380074510 0x7f83800748f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:23.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:23.579+0000 7f837fe00640 1 -- 172.21.15.5:0/2440940180 >> 172.21.15.5:0/2440940180 conn(0x7f83800fd740 msgr2=0x7f83800ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:58:23.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:23.579+0000 7f837fe00640 1 -- 172.21.15.5:0/2440940180 shutdown_connections 2024-09-10T09:58:23.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:23.579+0000 7f837fe00640 1 -- 172.21.15.5:0/2440940180 wait complete. 2024-09-10T09:58:23.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:23.579+0000 7f837fe00640 1 Processor -- start 2024-09-10T09:58:23.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:23.579+0000 7f837fe00640 1 -- start start 2024-09-10T09:58:23.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:23.579+0000 7f837fe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8380074510 0x7f838019a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:23.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:23.579+0000 7f837fe00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8380074e30 0x7f838019a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:23.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:23.579+0000 7f837fe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f838010e280 0x7f838019ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:23.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:23.579+0000 7f837fe00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f8380113fb0 con 0x7f8380074510 2024-09-10T09:58:23.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:23.579+0000 7f837fe00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f8380113e30 con 0x7f838010e280 2024-09-10T09:58:23.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:23.579+0000 7f837fe00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f8380114130 con 0x7f8380074e30 2024-09-10T09:58:23.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:23.579+0000 7f837ea00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8380074510 0x7f838019a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:23.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:23.579+0000 7f837ea00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8380074510 0x7f838019a490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42638/0 (socket says 172.21.15.5:42638) 2024-09-10T09:58:23.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:23.579+0000 7f837ea00640 1 -- 172.21.15.5:0/1790190839 learned_addr learned my addr 172.21.15.5:0/1790190839 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:58:23.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:23.579+0000 7f837f400640 1 --2- 172.21.15.5:0/1790190839 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f838010e280 0x7f838019ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:23.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:23.579+0000 7f837ea00640 1 -- 172.21.15.5:0/1790190839 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8380074e30 msgr2=0x7f838019a9d0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-10T09:58:23.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:23.579+0000 7f837ea00640 1 --2- 172.21.15.5:0/1790190839 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8380074e30 0x7f838019a9d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:23.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:23.579+0000 7f837ea00640 1 -- 172.21.15.5:0/1790190839 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f838010e280 msgr2=0x7f838019ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:23.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:23.579+0000 7f837ea00640 1 --2- 172.21.15.5:0/1790190839 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f838010e280 0x7f838019ed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:23.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:23.579+0000 7f837ea00640 1 -- 172.21.15.5:0/1790190839 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f838019f440 con 0x7f8380074510 2024-09-10T09:58:23.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:23.584+0000 7f837ea00640 1 --2- 172.21.15.5:0/1790190839 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8380074510 0x7f838019a490 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f837000d950 tx=0x7f837000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:23.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:23.584+0000 7f8377400640 1 -- 172.21.15.5:0/1790190839 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8370017070 con 0x7f8380074510 2024-09-10T09:58:23.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:23.584+0000 7f8377400640 1 -- 172.21.15.5:0/1790190839 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8370009c30 con 0x7f8380074510 2024-09-10T09:58:23.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:23.584+0000 7f8377400640 1 -- 172.21.15.5:0/1790190839 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f837000f6b0 con 0x7f8380074510 2024-09-10T09:58:23.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:23.584+0000 7f837fe00640 1 -- 172.21.15.5:0/1790190839 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f838019f730 con 0x7f8380074510 2024-09-10T09:58:23.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:23.584+0000 7f837fe00640 1 -- 172.21.15.5:0/1790190839 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83801a7010 con 0x7f8380074510 2024-09-10T09:58:23.990 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-10T09:58:23.990 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":42,"btime":"2024-09-10T09:55:06:670916+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14454,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/2970594311","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2970594311},{"type":"v1","addr":"172.21.15.5:6835","nonce":2970594311}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14514,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6839/3616502174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":3616502174},{"type":"v1","addr":"172.21.15.5:6839","nonce":3616502174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":42,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T09:35:29.626287+0000","modified":"2024-09-10T09:55:06.419829+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24542,"mds_1":24535,"mds_2":24511,"mds_3":24506,"mds_4":24443},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14520":{"gid":14520,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/2073599842","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":2073599842},{"type":"v1","addr":"172.21.15.5:6841","nonce":2073599842}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"k","rank":4,"incarnation":26,"state":"up:active","state_seq":24,"addr":"172.21.15.102:6833/2457007427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":2457007427},{"type":"v1","addr":"172.21.15.102:6833","nonce":2457007427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/356445159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":356445159},{"type":"v1","addr":"172.21.15.159:6833","nonce":356445159}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6837/2894314507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":2894314507},{"type":"v1","addr":"172.21.15.5:6837","nonce":2894314507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.102:6835/1647549111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":1647549111},{"type":"v1","addr":"172.21.15.102:6835","nonce":1647549111}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6835/952657242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":952657242},{"type":"v1","addr":"172.21.15.159:6835","nonce":952657242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":19,"addr":"172.21.15.102:6837/2689776764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":2689776764},{"type":"v1","addr":"172.21.15.102:6837","nonce":2689776764}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.159:6837/620627639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":620627639},{"type":"v1","addr":"172.21.15.159:6837","nonce":620627639}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24535":{"gid":24535,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":18,"addr":"172.21.15.159:6839/2073739599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":2073739599},{"type":"v1","addr":"172.21.15.159:6839","nonce":2073739599}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24542":{"gid":24542,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.102:6839/335409749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6838","nonce":335409749},{"type":"v1","addr":"172.21.15.102:6839","nonce":335409749}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24542,"qdb_cluster":[24542,24535,24511,24506,24443]},"id":1}]} 2024-09-10T09:58:23.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:23.584+0000 7f837fe00640 1 -- 172.21.15.5:0/1790190839 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f83800748f0 con 0x7f8380074510 2024-09-10T09:58:23.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:23.584+0000 7f8377400640 1 -- 172.21.15.5:0/1790190839 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f83700109d0 con 0x7f8380074510 2024-09-10T09:58:23.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:23.589+0000 7f8377400640 1 --2- 172.21.15.5:0/1790190839 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f83640a2c00 0x7f83640a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:23.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:23.589+0000 7f8377400640 1 -- 172.21.15.5:0/1790190839 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f83700cefe0 con 0x7f8380074510 2024-09-10T09:58:23.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:23.589+0000 7f837e000640 1 --2- 172.21.15.5:0/1790190839 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f83640a2c00 0x7f83640a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:23.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:23.589+0000 7f837e000640 1 --2- 172.21.15.5:0/1790190839 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f83640a2c00 0x7f83640a50c0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f838010f610 tx=0x7f836c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:23.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:23.589+0000 7f8377400640 1 -- 172.21.15.5:0/1790190839 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f8370009de0 con 0x7f8380074510 2024-09-10T09:58:23.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:23.818+0000 7f837fe00640 1 -- 172.21.15.5:0/1790190839 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f838019b7f0 con 0x7f8380074510 2024-09-10T09:58:23.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:23.819+0000 7f8377400640 1 -- 172.21.15.5:0/1790190839 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 42 v42) ==== 94+0+11969 (secure 0 0 0) 0x7f837009bb70 con 0x7f8380074510 2024-09-10T09:58:23.991 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 42 2024-09-10T09:58:23.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:23.821+0000 7f837fe00640 1 -- 172.21.15.5:0/1790190839 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f83640a2c00 msgr2=0x7f83640a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:23.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:23.821+0000 7f837fe00640 1 --2- 172.21.15.5:0/1790190839 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f83640a2c00 0x7f83640a50c0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f838010f610 tx=0x7f836c005f90 comp rx=0 tx=0).stop 2024-09-10T09:58:23.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:23.821+0000 7f837fe00640 1 -- 172.21.15.5:0/1790190839 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8380074510 msgr2=0x7f838019a490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:23.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:23.821+0000 7f837fe00640 1 --2- 172.21.15.5:0/1790190839 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8380074510 0x7f838019a490 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f837000d950 tx=0x7f837000de10 comp rx=0 tx=0).stop 2024-09-10T09:58:23.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:23.821+0000 7f837fe00640 1 -- 172.21.15.5:0/1790190839 shutdown_connections 2024-09-10T09:58:23.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:23.821+0000 7f837fe00640 1 --2- 172.21.15.5:0/1790190839 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f83640a2c00 0x7f83640a50c0 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:23.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:23.821+0000 7f837fe00640 1 --2- 172.21.15.5:0/1790190839 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f838010e280 0x7f838019ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:23.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:23.821+0000 7f837fe00640 1 --2- 172.21.15.5:0/1790190839 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8380074e30 0x7f838019a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:23.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:23.821+0000 7f837fe00640 1 --2- 172.21.15.5:0/1790190839 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8380074510 0x7f838019a490 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:23.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:23.821+0000 7f837fe00640 1 -- 172.21.15.5:0/1790190839 >> 172.21.15.5:0/1790190839 conn(0x7f83800fd740 msgr2=0x7f83800fe3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:58:23.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:23.821+0000 7f837fe00640 1 -- 172.21.15.5:0/1790190839 shutdown_connections 2024-09-10T09:58:23.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:23.821+0000 7f837fe00640 1 -- 172.21.15.5:0/1790190839 wait complete. 2024-09-10T09:58:24.022 INFO:tasks.workunit.client.0.smithi005.stdout: 10 107580 15.28 MB/sec execute 242 sec latency 12321.680 ms 2024-09-10T09:58:24.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:58:23 smithi159 ceph-mon[35513]: pgmap v994: 129 pgs: 129 active+clean; 726 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 7.4 MiB/s wr, 331 op/s 2024-09-10T09:58:24.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:58:23 smithi102 ceph-mon[35442]: pgmap v994: 129 pgs: 129 active+clean; 726 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 7.4 MiB/s wr, 331 op/s 2024-09-10T09:58:24.277 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:58:24.921 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:58:24 smithi005 ceph-mon[30350]: from='client.? 172.21.15.5:0/1790190839' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:58:24.922 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:58:24 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:58:24.922 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:58:24 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:58:25.022 INFO:tasks.workunit.client.0.smithi005.stdout: 10 107580 15.22 MB/sec execute 243 sec latency 13321.718 ms 2024-09-10T09:58:25.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:58:24 smithi159 ceph-mon[35513]: from='client.? 172.21.15.5:0/1790190839' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:58:25.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:58:24 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:58:25.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:58:24 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:58:25.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:58:24 smithi102 ceph-mon[35442]: from='client.? 172.21.15.5:0/1790190839' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:58:25.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:58:24 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:58:25.135 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:58:24 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:58:25.172 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24542 quiesce db --set-id b9f7a64f --release --await' 2024-09-10T09:58:25.172 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24542 quiesce db --set-id b9f7a64f --release --await 2024-09-10T09:58:25.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:25.691+0000 7f8781e00640 1 -- 172.21.15.5:0/2325218629 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f877c102b90 msgr2=0x7f877c102f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:25.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:25.691+0000 7f8781e00640 1 --2- 172.21.15.5:0/2325218629 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f877c102b90 0x7f877c102f70 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f8770009900 tx=0x7f877002a4a0 comp rx=0 tx=0).stop 2024-09-10T09:58:25.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:25.694+0000 7f8781e00640 1 -- 172.21.15.5:0/2325218629 shutdown_connections 2024-09-10T09:58:25.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:25.694+0000 7f8781e00640 1 --2- 172.21.15.5:0/2325218629 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f877c10bfc0 0x7f877c10e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:25.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:25.694+0000 7f8781e00640 1 --2- 172.21.15.5:0/2325218629 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f877c1034b0 0x7f877c10b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:25.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:25.694+0000 7f8781e00640 1 --2- 172.21.15.5:0/2325218629 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f877c102b90 0x7f877c102f70 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:25.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:25.694+0000 7f8781e00640 1 -- 172.21.15.5:0/2325218629 >> 172.21.15.5:0/2325218629 conn(0x7f877c0fc610 msgr2=0x7f877c0fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:58:25.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:25.694+0000 7f8781e00640 1 -- 172.21.15.5:0/2325218629 shutdown_connections 2024-09-10T09:58:25.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:25.694+0000 7f8781e00640 1 -- 172.21.15.5:0/2325218629 wait complete. 2024-09-10T09:58:25.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:25.694+0000 7f8781e00640 1 Processor -- start 2024-09-10T09:58:25.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:25.694+0000 7f8781e00640 1 -- start start 2024-09-10T09:58:25.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:25.694+0000 7f8781e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f877c102b90 0x7f877c19a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:25.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:25.694+0000 7f8781e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f877c1034b0 0x7f877c19aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:25.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:25.694+0000 7f8781e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f877c10bfc0 0x7f877c19ee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:25.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:25.694+0000 7f8781e00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f877c111e40 con 0x7f877c1034b0 2024-09-10T09:58:25.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:25.694+0000 7f8781e00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f877c111cc0 con 0x7f877c102b90 2024-09-10T09:58:25.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:25.694+0000 7f8781e00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f877c111fc0 con 0x7f877c10bfc0 2024-09-10T09:58:25.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:25.694+0000 7f877be00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f877c1034b0 0x7f877c19aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:25.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:25.694+0000 7f877be00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f877c1034b0 0x7f877c19aa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42642/0 (socket says 172.21.15.5:42642) 2024-09-10T09:58:25.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:25.694+0000 7f877be00640 1 -- 172.21.15.5:0/1634157590 learned_addr learned my addr 172.21.15.5:0/1634157590 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:58:25.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:25.694+0000 7f8781400640 1 --2- 172.21.15.5:0/1634157590 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f877c10bfc0 0x7f877c19ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:25.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:25.694+0000 7f8780a00640 1 --2- 172.21.15.5:0/1634157590 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f877c102b90 0x7f877c19a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:25.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:25.694+0000 7f877be00640 1 -- 172.21.15.5:0/1634157590 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f877c10bfc0 msgr2=0x7f877c19ee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:25.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:25.694+0000 7f877be00640 1 --2- 172.21.15.5:0/1634157590 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f877c10bfc0 0x7f877c19ee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:25.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:25.694+0000 7f877be00640 1 -- 172.21.15.5:0/1634157590 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f877c102b90 msgr2=0x7f877c19a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:25.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:25.694+0000 7f877be00640 1 --2- 172.21.15.5:0/1634157590 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f877c102b90 0x7f877c19a540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:25.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:25.694+0000 7f877be00640 1 -- 172.21.15.5:0/1634157590 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f877c19f4f0 con 0x7f877c1034b0 2024-09-10T09:58:25.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:25.694+0000 7f877be00640 1 --2- 172.21.15.5:0/1634157590 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f877c1034b0 0x7f877c19aa80 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f876800b3c0 tx=0x7f876800b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:25.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:25.694+0000 7f8779600640 1 -- 172.21.15.5:0/1634157590 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8768011830 con 0x7f877c1034b0 2024-09-10T09:58:25.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:25.694+0000 7f8779600640 1 -- 172.21.15.5:0/1634157590 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8768016070 con 0x7f877c1034b0 2024-09-10T09:58:25.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:25.694+0000 7f8781e00640 1 -- 172.21.15.5:0/1634157590 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f877c19f7e0 con 0x7f877c1034b0 2024-09-10T09:58:25.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:25.694+0000 7f8779600640 1 -- 172.21.15.5:0/1634157590 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f876800f2e0 con 0x7f877c1034b0 2024-09-10T09:58:25.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:25.694+0000 7f8781e00640 1 -- 172.21.15.5:0/1634157590 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f877c1a70c0 con 0x7f877c1034b0 2024-09-10T09:58:25.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:25.700+0000 7f8786053740 1 Processor -- start 2024-09-10T09:58:25.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:25.700+0000 7f8786053740 1 -- start start 2024-09-10T09:58:25.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:25.700+0000 7f8779600640 1 -- 172.21.15.5:0/1634157590 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f87680119d0 con 0x7f877c1034b0 2024-09-10T09:58:25.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:25.700+0000 7f8786053740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5608f4d05e40 0x5608f4d06220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:25.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:25.700+0000 7f8786053740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5608f4d06760 0x5608f4d10ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:25.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:25.700+0000 7f8786053740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5608f4d11020 0x5608f4d13410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:25.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:25.700+0000 7f8786053740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5608f4c6c3e0 con 0x5608f4d11020 2024-09-10T09:58:25.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:25.700+0000 7f8786053740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x5608f4c13d80 con 0x5608f4d05e40 2024-09-10T09:58:25.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:25.700+0000 7f8786053740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5608f4c683a0 con 0x5608f4d06760 2024-09-10T09:58:25.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:25.700+0000 7f877be00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5608f4d11020 0x5608f4d13410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:25.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:25.700+0000 7f877be00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5608f4d11020 0x5608f4d13410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42654/0 (socket says 172.21.15.5:42654) 2024-09-10T09:58:25.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:25.700+0000 7f877be00640 1 -- 172.21.15.5:0/1211296413 learned_addr learned my addr 172.21.15.5:0/1211296413 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:58:25.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:25.700+0000 7f8781400640 1 --2- 172.21.15.5:0/1211296413 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5608f4d05e40 0x5608f4d06220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:25.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:25.700+0000 7f877be00640 1 -- 172.21.15.5:0/1211296413 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5608f4d06760 msgr2=0x5608f4d10ae0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T09:58:25.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:25.700+0000 7f877be00640 1 --2- 172.21.15.5:0/1211296413 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5608f4d06760 0x5608f4d10ae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:25.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:25.700+0000 7f877be00640 1 -- 172.21.15.5:0/1211296413 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5608f4d05e40 msgr2=0x5608f4d06220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:58:25.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:25.700+0000 7f877be00640 1 --2- 172.21.15.5:0/1211296413 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5608f4d05e40 0x5608f4d06220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:25.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:25.700+0000 7f877be00640 1 -- 172.21.15.5:0/1211296413 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5608f4c32ae0 con 0x5608f4d11020 2024-09-10T09:58:25.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:25.700+0000 7f8781400640 1 --2- 172.21.15.5:0/1211296413 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5608f4d05e40 0x5608f4d06220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:58:25.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:25.700+0000 7f877be00640 1 --2- 172.21.15.5:0/1211296413 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5608f4d11020 0x5608f4d13410 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f87680d4a90 tx=0x7f87680d4f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33a9b58d46f69bb2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:25.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:25.700+0000 7f8779600640 1 --2- 172.21.15.5:0/1634157590 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f87640a2c00 0x7f87640a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:25.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:25.700+0000 7f8781400640 1 --2- 172.21.15.5:0/1634157590 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f87640a2c00 0x7f87640a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:25.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:25.700+0000 7f8779600640 1 -- 172.21.15.5:0/1634157590 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f87680ce940 con 0x7f877c1034b0 2024-09-10T09:58:25.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:25.700+0000 7f876dc00640 1 -- 172.21.15.5:0/1211296413 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87680dd070 con 0x5608f4d11020 2024-09-10T09:58:25.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:25.700+0000 7f876dc00640 1 -- 172.21.15.5:0/1211296413 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f876809b450 con 0x5608f4d11020 2024-09-10T09:58:25.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:25.700+0000 7f876dc00640 1 -- 172.21.15.5:0/1211296413 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f876809a890 con 0x5608f4d11020 2024-09-10T09:58:25.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:25.700+0000 7f8786053740 1 -- 172.21.15.5:0/1211296413 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5608f4d11020 msgr2=0x5608f4d13410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:25.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:25.700+0000 7f8781400640 1 --2- 172.21.15.5:0/1634157590 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f87640a2c00 0x7f87640a50c0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f8774009fd0 tx=0x7f8774009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:25.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:25.700+0000 7f8786053740 1 --2- 172.21.15.5:0/1211296413 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5608f4d11020 0x5608f4d13410 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f87680d4a90 tx=0x7f87680d4f50 comp rx=0 tx=0).stop 2024-09-10T09:58:25.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:25.700+0000 7f8786053740 1 -- 172.21.15.5:0/1211296413 shutdown_connections 2024-09-10T09:58:25.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:25.700+0000 7f8786053740 1 --2- 172.21.15.5:0/1211296413 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5608f4d11020 0x5608f4d13410 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:25.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:25.700+0000 7f8786053740 1 --2- 172.21.15.5:0/1211296413 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5608f4d06760 0x5608f4d10ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:25.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:25.700+0000 7f8786053740 1 --2- 172.21.15.5:0/1211296413 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5608f4d05e40 0x5608f4d06220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:25.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:25.700+0000 7f8786053740 1 -- 172.21.15.5:0/1211296413 >> 172.21.15.5:0/1211296413 conn(0x5608f4d023d0 msgr2=0x5608f4d027b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:58:25.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:25.700+0000 7f8786053740 1 -- 172.21.15.5:0/1211296413 shutdown_connections 2024-09-10T09:58:25.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:25.700+0000 7f8786053740 1 -- 172.21.15.5:0/1211296413 wait complete. 2024-09-10T09:58:25.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:25.700+0000 7f8786053740 1 -- start start 2024-09-10T09:58:25.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:25.705+0000 7f8786053740 1 Processor -- start 2024-09-10T09:58:25.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:25.705+0000 7f8786053740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5608f4d05e40 0x5608f4d0dff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:25.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:25.705+0000 7f8786053740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5608f4d06760 0x5608f4d0e530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:25.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:25.705+0000 7f8786053740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5608f4d11020 0x5608f4d0ea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:25.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:25.705+0000 7f8786053740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5608f4d13bf0 con 0x5608f4d05e40 2024-09-10T09:58:25.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:25.705+0000 7f8786053740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x5608f4c270f0 con 0x5608f4d06760 2024-09-10T09:58:25.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:25.705+0000 7f8786053740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5608f4d13d70 con 0x5608f4d11020 2024-09-10T09:58:25.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:25.705+0000 7f8786053740 10 client.0 _ll_register_callbacks cb 0x5608f4c0ebd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:58:25.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:25.705+0000 7f8780a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5608f4d05e40 0x5608f4d0dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:25.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:25.705+0000 7f8780a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5608f4d05e40 0x5608f4d0dff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42664/0 (socket says 172.21.15.5:42664) 2024-09-10T09:58:25.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:25.705+0000 7f8780a00640 1 -- 172.21.15.5:0/3571153025 learned_addr learned my addr 172.21.15.5:0/3571153025 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:58:25.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:25.705+0000 7f877be00640 1 --2- 172.21.15.5:0/3571153025 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5608f4d06760 0x5608f4d0e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:25.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:25.705+0000 7f8781400640 1 --2- 172.21.15.5:0/3571153025 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5608f4d11020 0x5608f4d0ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:25.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:25.705+0000 7f8780a00640 1 -- 172.21.15.5:0/3571153025 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5608f4d11020 msgr2=0x5608f4d0ea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:58:25.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:25.705+0000 7f8780a00640 1 --2- 172.21.15.5:0/3571153025 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5608f4d11020 0x5608f4d0ea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:25.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:25.705+0000 7f8780a00640 1 -- 172.21.15.5:0/3571153025 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5608f4d06760 msgr2=0x5608f4d0e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:58:25.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:25.705+0000 7f8780a00640 1 --2- 172.21.15.5:0/3571153025 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5608f4d06760 0x5608f4d0e530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:25.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:25.705+0000 7f8780a00640 1 -- 172.21.15.5:0/3571153025 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5608f49dd3c0 con 0x5608f4d05e40 2024-09-10T09:58:25.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:25.705+0000 7f8781400640 1 --2- 172.21.15.5:0/3571153025 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5608f4d11020 0x5608f4d0ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:58:25.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:25.705+0000 7f8780a00640 1 --2- 172.21.15.5:0/3571153025 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5608f4d05e40 0x5608f4d0dff0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f8770009900 tx=0x7f87700317e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aeb22ad42b6d9daa server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:25.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:25.705+0000 7f8762000640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-10T09:58:25.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:25.705+0000 7f8762000640 1 -- 172.21.15.5:0/3571153025 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8770040070 con 0x5608f4d05e40 2024-09-10T09:58:25.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:25.705+0000 7f8786053740 1 -- 172.21.15.5:0/3571153025 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f876809b450 con 0x5608f4d05e40 2024-09-10T09:58:25.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:25.705+0000 7f8762000640 1 -- 172.21.15.5:0/3571153025 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f8770004450 con 0x5608f4d05e40 2024-09-10T09:58:25.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:25.705+0000 7f8762000640 1 -- 172.21.15.5:0/3571153025 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f877004d050 con 0x5608f4d05e40 2024-09-10T09:58:25.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:25.705+0000 7f8786053740 10 client.19245 fetch_fsmap learned FSMap version 42 2024-09-10T09:58:25.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:25.705+0000 7f8786053740 1 -- 172.21.15.5:0/3571153025 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x5608f4d1fd50 con 0x5608f4d05e40 2024-09-10T09:58:25.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:25.705+0000 7f8762000640 1 -- 172.21.15.5:0/3571153025 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f876809b450 con 0x5608f4d05e40 2024-09-10T09:58:25.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:25.705+0000 7f8762000640 1 -- 172.21.15.5:0/3571153025 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 42) ==== 8370+0+0 (secure 0 0 0) 0x7f8770004cc0 con 0x5608f4d05e40 2024-09-10T09:58:25.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:25.705+0000 7f8786053740 10 client.19245 fetch_fsmap finished waiting for FSMap version 42 2024-09-10T09:58:25.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:25.705+0000 7f8786053740 10 client.19245 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T09:58:25.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:25.705+0000 7f8786053740 20 client.19245 populate_metadata read hostname 'smithi005' 2024-09-10T09:58:25.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:25.705+0000 7f8786053740 1 --2- 172.21.15.5:0/3571153025 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x5608f4d21160 0x5608f4d23550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:25.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:25.705+0000 7f8786053740 4 client.19245 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:58:25.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:25.705+0000 7f8786053740 1 -- 172.21.15.5:0/3571153025 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5608f4c79950 con 0x5608f4d21160 2024-09-10T09:58:25.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:25.705+0000 7f877be00640 1 --2- 172.21.15.5:0/3571153025 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x5608f4d21160 0x5608f4d23550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:25.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:25.705+0000 7f877be00640 1 --2- 172.21.15.5:0/3571153025 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x5608f4d21160 0x5608f4d23550 crc :-1 s=READY pgs=279 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3d8144d2ed98ada server_cookie=6e4f7ab58ec6125c in_seq=0 out_seq=0 2024-09-10T09:58:25.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:25.705+0000 7f8762000640 10 client.19245 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T09:58:25.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:25.705+0000 7f8762000640 1 -- 172.21.15.5:0/3571153025 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5608f4c79950 con 0x5608f4d21160 2024-09-10T09:58:25.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:25.705+0000 7f8762000640 10 client.19245 handle_command_reply: tid=0 2024-09-10T09:58:25.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:25.705+0000 7f8786053740 1 client.19245 shutdown 2024-09-10T09:58:25.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:25.710+0000 7f8786053740 1 -- 172.21.15.5:0/3571153025 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5608f4d05e40 msgr2=0x5608f4d0dff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:25.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:25.710+0000 7f8786053740 1 --2- 172.21.15.5:0/3571153025 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5608f4d05e40 0x5608f4d0dff0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f8770009900 tx=0x7f87700317e0 comp rx=0 tx=0).stop 2024-09-10T09:58:25.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:25.710+0000 7f8786053740 1 -- 172.21.15.5:0/3571153025 shutdown_connections 2024-09-10T09:58:25.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:25.710+0000 7f8786053740 1 --2- 172.21.15.5:0/3571153025 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x5608f4d21160 0x5608f4d23550 crc :-1 s=READY pgs=279 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:25.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:25.710+0000 7f8786053740 1 --2- 172.21.15.5:0/3571153025 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5608f4d11020 0x5608f4d0ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:25.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:25.710+0000 7f8786053740 1 --2- 172.21.15.5:0/3571153025 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5608f4d06760 0x5608f4d0e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:25.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:25.710+0000 7f8786053740 1 --2- 172.21.15.5:0/3571153025 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5608f4d05e40 0x5608f4d0dff0 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:25.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:25.710+0000 7f8786053740 1 -- 172.21.15.5:0/3571153025 >> 172.21.15.5:0/3571153025 conn(0x5608f4d023d0 msgr2=0x5608f4d080b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:58:25.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:25.710+0000 7f8762000640 0 client.19245 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T09:58:25.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:25.710+0000 7f8786053740 1 -- 172.21.15.5:0/3571153025 shutdown_connections 2024-09-10T09:58:25.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:25.710+0000 7f8786053740 1 -- 172.21.15.5:0/3571153025 wait complete. 2024-09-10T09:58:25.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:25.710+0000 7f8786053740 20 client.19245 trim_cache size 0 max 16384 2024-09-10T09:58:25.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:25.736+0000 7f8786053740 1 Processor -- start 2024-09-10T09:58:25.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:25.736+0000 7f8786053740 1 -- start start 2024-09-10T09:58:25.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:25.736+0000 7f8786053740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5608f4d05e40 0x5608f4d28e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:25.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:25.736+0000 7f8786053740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5608f4d06760 0x5608f4d29340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:25.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:25.736+0000 7f8786053740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5608f4d11020 0x5608f4d0b250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:25.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:25.736+0000 7f8786053740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f877c1b0fa0 con 0x5608f4d05e40 2024-09-10T09:58:25.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:25.736+0000 7f8786053740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f877c1b2ae0 con 0x5608f4d11020 2024-09-10T09:58:25.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:25.736+0000 7f8786053740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f877c1b1670 con 0x5608f4d06760 2024-09-10T09:58:25.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:25.736+0000 7f8780a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5608f4d05e40 0x5608f4d28e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:25.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:25.736+0000 7f8780a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5608f4d05e40 0x5608f4d28e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42670/0 (socket says 172.21.15.5:42670) 2024-09-10T09:58:25.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:25.736+0000 7f8780a00640 1 -- 172.21.15.5:0/3813910825 learned_addr learned my addr 172.21.15.5:0/3813910825 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:58:25.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:25.736+0000 7f877be00640 1 --2- 172.21.15.5:0/3813910825 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5608f4d06760 0x5608f4d29340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:25.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:25.736+0000 7f8781400640 1 --2- 172.21.15.5:0/3813910825 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5608f4d11020 0x5608f4d0b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:25.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:25.736+0000 7f8780a00640 1 -- 172.21.15.5:0/3813910825 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5608f4d06760 msgr2=0x5608f4d29340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:58:25.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:25.736+0000 7f8780a00640 1 --2- 172.21.15.5:0/3813910825 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5608f4d06760 0x5608f4d29340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:25.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:25.736+0000 7f8780a00640 1 -- 172.21.15.5:0/3813910825 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5608f4d11020 msgr2=0x5608f4d0b250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:58:25.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:25.736+0000 7f8780a00640 1 --2- 172.21.15.5:0/3813910825 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5608f4d11020 0x5608f4d0b250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:25.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:25.736+0000 7f8780a00640 1 -- 172.21.15.5:0/3813910825 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5608f4d03060 con 0x5608f4d05e40 2024-09-10T09:58:25.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:25.736+0000 7f877be00640 1 --2- 172.21.15.5:0/3813910825 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5608f4d06760 0x5608f4d29340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:58:25.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:25.736+0000 7f8780a00640 1 --2- 172.21.15.5:0/3813910825 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5608f4d05e40 0x5608f4d28e00 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f8770009850 tx=0x7f877002a4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f218677678035638 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:25.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:25.736+0000 7f876d200640 1 -- 172.21.15.5:0/3813910825 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f877004a040 con 0x5608f4d05e40 2024-09-10T09:58:25.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:25.736+0000 7f876d200640 1 -- 172.21.15.5:0/3813910825 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8770005510 con 0x5608f4d05e40 2024-09-10T09:58:25.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:25.736+0000 7f876d200640 1 -- 172.21.15.5:0/3813910825 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8770040040 con 0x5608f4d05e40 2024-09-10T09:58:25.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:25.738+0000 7f8786053740 1 -- 172.21.15.5:0/3813910825 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5608f4d05e40 msgr2=0x5608f4d28e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:25.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:25.738+0000 7f8786053740 1 --2- 172.21.15.5:0/3813910825 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5608f4d05e40 0x5608f4d28e00 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f8770009850 tx=0x7f877002a4a0 comp rx=0 tx=0).stop 2024-09-10T09:58:25.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:25.738+0000 7f8786053740 1 -- 172.21.15.5:0/3813910825 shutdown_connections 2024-09-10T09:58:25.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:25.738+0000 7f8786053740 1 --2- 172.21.15.5:0/3813910825 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5608f4d11020 0x5608f4d0b250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:25.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:25.738+0000 7f8786053740 1 --2- 172.21.15.5:0/3813910825 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5608f4d06760 0x5608f4d29340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:25.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:25.738+0000 7f8786053740 1 --2- 172.21.15.5:0/3813910825 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5608f4d05e40 0x5608f4d28e00 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:25.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:25.738+0000 7f8786053740 1 -- 172.21.15.5:0/3813910825 >> 172.21.15.5:0/3813910825 conn(0x5608f4d023d0 msgr2=0x5608f4d09300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:58:25.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:25.738+0000 7f8786053740 1 -- 172.21.15.5:0/3813910825 shutdown_connections 2024-09-10T09:58:25.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:25.738+0000 7f8786053740 1 -- 172.21.15.5:0/3813910825 wait complete. 2024-09-10T09:58:25.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:25.738+0000 7f8786053740 1 -- start start 2024-09-10T09:58:25.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:25.738+0000 7f8786053740 1 Processor -- start 2024-09-10T09:58:25.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:25.738+0000 7f8786053740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5608f4d05e40 0x5608f4d19710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:25.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:25.738+0000 7f8786053740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5608f4d06760 0x5608f4d14a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:25.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:25.738+0000 7f8786053740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5608f4d11020 0x5608f4d14f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:25.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:25.738+0000 7f8786053740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f877c1b2410 con 0x5608f4d05e40 2024-09-10T09:58:25.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:25.738+0000 7f8786053740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f877c1b1d40 con 0x5608f4d06760 2024-09-10T09:58:25.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:25.738+0000 7f8786053740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f877c1aef80 con 0x5608f4d11020 2024-09-10T09:58:25.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:25.738+0000 7f8786053740 10 client.0 _ll_register_callbacks cb 0x7f8770009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:58:25.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:25.738+0000 7f8780a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5608f4d05e40 0x5608f4d19710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:25.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:25.738+0000 7f8780a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5608f4d05e40 0x5608f4d19710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42682/0 (socket says 172.21.15.5:42682) 2024-09-10T09:58:25.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:25.738+0000 7f877be00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5608f4d06760 0x5608f4d14a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:25.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:25.738+0000 7f8780a00640 1 -- 172.21.15.5:0/520220440 learned_addr learned my addr 172.21.15.5:0/520220440 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:58:25.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:25.738+0000 7f877be00640 1 -- 172.21.15.5:0/520220440 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5608f4d11020 msgr2=0x5608f4d14f80 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T09:58:25.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:25.738+0000 7f877be00640 1 --2- 172.21.15.5:0/520220440 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5608f4d11020 0x5608f4d14f80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:25.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:25.738+0000 7f877be00640 1 -- 172.21.15.5:0/520220440 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5608f4d05e40 msgr2=0x5608f4d19710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:58:25.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:25.738+0000 7f877be00640 1 --2- 172.21.15.5:0/520220440 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5608f4d05e40 0x5608f4d19710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:25.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:25.738+0000 7f877be00640 1 -- 172.21.15.5:0/520220440 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5608f4d15760 con 0x5608f4d06760 2024-09-10T09:58:25.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:25.738+0000 7f8780a00640 1 --2- 172.21.15.5:0/520220440 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5608f4d05e40 0x5608f4d19710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:58:25.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:25.738+0000 7f877be00640 1 --2- 172.21.15.5:0/520220440 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5608f4d06760 0x5608f4d14a40 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x5608f4d03750 tx=0x7f876800beb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=be748608ae482a27 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:25.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:25.738+0000 7f8762000640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-09-10T09:58:25.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:25.738+0000 7f8762000640 1 -- 172.21.15.5:0/520220440 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87680dd040 con 0x5608f4d06760 2024-09-10T09:58:25.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:25.738+0000 7f8762000640 1 -- 172.21.15.5:0/520220440 <== mon.2 v2:172.21.15.102:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f8768099050 con 0x5608f4d06760 2024-09-10T09:58:25.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:25.738+0000 7f8786053740 1 -- 172.21.15.5:0/520220440 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8770005510 con 0x5608f4d06760 2024-09-10T09:58:25.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:25.738+0000 7f8762000640 1 -- 172.21.15.5:0/520220440 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f876809a6b0 con 0x5608f4d06760 2024-09-10T09:58:25.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:25.738+0000 7f8786053740 10 client.29315 fetch_fsmap learned FSMap version 42 2024-09-10T09:58:25.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:25.738+0000 7f8786053740 1 -- 172.21.15.5:0/520220440 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x5608f4d8bde0 con 0x5608f4d06760 2024-09-10T09:58:25.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:25.738+0000 7f8762000640 1 -- 172.21.15.5:0/520220440 <== mon.2 v2:172.21.15.102:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8770005510 con 0x5608f4d06760 2024-09-10T09:58:25.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:25.738+0000 7f8762000640 1 -- 172.21.15.5:0/520220440 <== mon.2 v2:172.21.15.102:3300/0 5 ==== fsmap(e 42) ==== 8370+0+0 (secure 0 0 0) 0x7f87680cc070 con 0x5608f4d06760 2024-09-10T09:58:25.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:25.738+0000 7f8786053740 10 client.29315 fetch_fsmap finished waiting for FSMap version 42 2024-09-10T09:58:25.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:25.738+0000 7f8786053740 10 client.29315 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T09:58:25.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:25.738+0000 7f8786053740 20 client.29315 populate_metadata read hostname 'smithi005' 2024-09-10T09:58:25.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:25.738+0000 7f8786053740 1 --2- 172.21.15.5:0/520220440 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x5608f4d8c140 0x5608f4d8e580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:25.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:25.738+0000 7f8786053740 4 client.29315 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T09:58:25.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:25.738+0000 7f8786053740 1 -- 172.21.15.5:0/520220440 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5608f4c73e60 con 0x5608f4d8c140 2024-09-10T09:58:25.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:25.738+0000 7f8780a00640 1 --2- 172.21.15.5:0/520220440 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x5608f4d8c140 0x5608f4d8e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:25.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:25.738+0000 7f8780a00640 1 --2- 172.21.15.5:0/520220440 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x5608f4d8c140 0x5608f4d8e580 crc :-1 s=READY pgs=280 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9d1abe6f1930876d server_cookie=8debe035b9ec8d15 in_seq=0 out_seq=0 2024-09-10T09:58:25.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:25.738+0000 7f8762000640 10 client.29315 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T09:58:25.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:25.738+0000 7f8762000640 1 -- 172.21.15.5:0/520220440 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5608f4c73e60 con 0x5608f4d8c140 2024-09-10T09:58:25.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:25.738+0000 7f8762000640 10 client.29315 handle_command_reply: tid=0 2024-09-10T09:58:25.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:25.738+0000 7f8786053740 1 client.29315 shutdown 2024-09-10T09:58:25.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:25.738+0000 7f8786053740 1 -- 172.21.15.5:0/520220440 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5608f4d06760 msgr2=0x5608f4d14a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:25.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:25.738+0000 7f8786053740 1 --2- 172.21.15.5:0/520220440 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5608f4d06760 0x5608f4d14a40 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x5608f4d03750 tx=0x7f876800beb0 comp rx=0 tx=0).stop 2024-09-10T09:58:25.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:25.738+0000 7f8786053740 1 -- 172.21.15.5:0/520220440 shutdown_connections 2024-09-10T09:58:25.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:25.738+0000 7f8786053740 1 --2- 172.21.15.5:0/520220440 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x5608f4d8c140 0x5608f4d8e580 crc :-1 s=READY pgs=280 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:25.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:25.738+0000 7f8786053740 1 --2- 172.21.15.5:0/520220440 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5608f4d11020 0x5608f4d14f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:25.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:25.738+0000 7f8786053740 1 --2- 172.21.15.5:0/520220440 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5608f4d06760 0x5608f4d14a40 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:25.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:25.738+0000 7f8762000640 0 client.29315 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T09:58:25.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:25.738+0000 7f8786053740 1 --2- 172.21.15.5:0/520220440 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5608f4d05e40 0x5608f4d19710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:25.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:25.738+0000 7f8786053740 1 -- 172.21.15.5:0/520220440 >> 172.21.15.5:0/520220440 conn(0x5608f4d023d0 msgr2=0x5608f4d00c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:58:25.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:25.744+0000 7f8786053740 1 -- 172.21.15.5:0/520220440 shutdown_connections 2024-09-10T09:58:25.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:25.744+0000 7f8786053740 1 -- 172.21.15.5:0/520220440 wait complete. 2024-09-10T09:58:25.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:25.744+0000 7f8786053740 20 client.29315 trim_cache size 0 max 16384 2024-09-10T09:58:25.748 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-10T09:58:25.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:25.744+0000 7f8781e00640 1 -- 172.21.15.5:0/1634157590 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f87640a2c00 msgr2=0x7f87640a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:25.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:25.744+0000 7f8781e00640 1 --2- 172.21.15.5:0/1634157590 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f87640a2c00 0x7f87640a50c0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f8774009fd0 tx=0x7f8774009500 comp rx=0 tx=0).stop 2024-09-10T09:58:25.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:25.744+0000 7f8781e00640 1 -- 172.21.15.5:0/1634157590 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f877c1034b0 msgr2=0x7f877c19aa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:25.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:25.744+0000 7f8781e00640 1 --2- 172.21.15.5:0/1634157590 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f877c1034b0 0x7f877c19aa80 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f876800b3c0 tx=0x7f876800b880 comp rx=0 tx=0).stop 2024-09-10T09:58:25.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:25.744+0000 7f8781e00640 1 -- 172.21.15.5:0/1634157590 shutdown_connections 2024-09-10T09:58:25.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:25.744+0000 7f8781e00640 1 --2- 172.21.15.5:0/1634157590 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f87640a2c00 0x7f87640a50c0 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:25.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:25.744+0000 7f8781e00640 1 --2- 172.21.15.5:0/1634157590 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f877c10bfc0 0x7f877c19ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:25.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:25.744+0000 7f8781e00640 1 --2- 172.21.15.5:0/1634157590 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f877c1034b0 0x7f877c19aa80 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:25.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:25.744+0000 7f8781e00640 1 --2- 172.21.15.5:0/1634157590 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f877c102b90 0x7f877c19a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:25.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:25.744+0000 7f8781e00640 1 -- 172.21.15.5:0/1634157590 >> 172.21.15.5:0/1634157590 conn(0x7f877c0fc610 msgr2=0x7f877c10d2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:58:25.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:25.744+0000 7f8781e00640 1 -- 172.21.15.5:0/1634157590 shutdown_connections 2024-09-10T09:58:25.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:25.744+0000 7f8781e00640 1 -- 172.21.15.5:0/1634157590 wait complete. 2024-09-10T09:58:25.762 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:58:25 smithi005 ceph-mon[30350]: pgmap v995: 129 pgs: 129 active+clean; 722 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 7.5 MiB/s wr, 310 op/s 2024-09-10T09:58:26.022 INFO:tasks.workunit.client.0.smithi005.stdout: 10 107580 15.16 MB/sec execute 244 sec latency 14321.759 ms 2024-09-10T09:58:26.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:58:25 smithi159 ceph-mon[35513]: pgmap v995: 129 pgs: 129 active+clean; 722 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 7.5 MiB/s wr, 310 op/s 2024-09-10T09:58:26.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:58:25 smithi102 ceph-mon[35442]: pgmap v995: 129 pgs: 129 active+clean; 722 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 7.5 MiB/s wr, 310 op/s 2024-09-10T09:58:26.872 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:58:27.022 INFO:tasks.workunit.client.0.smithi005.stdout: 10 107580 15.10 MB/sec execute 245 sec latency 15321.811 ms 2024-09-10T09:58:27.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:27.722+0000 7ffa76400640 1 -- 172.21.15.5:0/2098012482 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ffa70074510 msgr2=0x7ffa700748f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:27.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:27.722+0000 7ffa76400640 1 --2- 172.21.15.5:0/2098012482 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ffa70074510 0x7ffa700748f0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7ffa64009860 tx=0x7ffa6402a3d0 comp rx=0 tx=0).stop 2024-09-10T09:58:27.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:27.722+0000 7ffa76400640 1 -- 172.21.15.5:0/2098012482 shutdown_connections 2024-09-10T09:58:27.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:27.722+0000 7ffa76400640 1 --2- 172.21.15.5:0/2098012482 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ffa7010e280 0x7ffa70110670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:27.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:27.722+0000 7ffa76400640 1 --2- 172.21.15.5:0/2098012482 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffa70074e30 0x7ffa7010dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:27.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:27.722+0000 7ffa76400640 1 --2- 172.21.15.5:0/2098012482 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ffa70074510 0x7ffa700748f0 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:27.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:27.722+0000 7ffa76400640 1 -- 172.21.15.5:0/2098012482 >> 172.21.15.5:0/2098012482 conn(0x7ffa700fd740 msgr2=0x7ffa700ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:58:27.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:27.722+0000 7ffa76400640 1 -- 172.21.15.5:0/2098012482 shutdown_connections 2024-09-10T09:58:27.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:27.722+0000 7ffa76400640 1 -- 172.21.15.5:0/2098012482 wait complete. 2024-09-10T09:58:27.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:27.722+0000 7ffa76400640 1 Processor -- start 2024-09-10T09:58:27.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:27.722+0000 7ffa76400640 1 -- start start 2024-09-10T09:58:27.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:27.722+0000 7ffa76400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffa70074510 0x7ffa7019a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:27.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:27.722+0000 7ffa76400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ffa70074e30 0x7ffa7019aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:27.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:27.722+0000 7ffa76400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ffa7010e280 0x7ffa7019ed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:27.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:27.722+0000 7ffa76400640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ffa70114000 con 0x7ffa70074510 2024-09-10T09:58:27.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:27.722+0000 7ffa76400640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7ffa70113e80 con 0x7ffa70074e30 2024-09-10T09:58:27.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:27.722+0000 7ffa76400640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7ffa70114180 con 0x7ffa7010e280 2024-09-10T09:58:27.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:27.722+0000 7ffa75000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffa70074510 0x7ffa7019a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:27.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:27.722+0000 7ffa75000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffa70074510 0x7ffa7019a4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42698/0 (socket says 172.21.15.5:42698) 2024-09-10T09:58:27.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:27.722+0000 7ffa75000640 1 -- 172.21.15.5:0/4074161775 learned_addr learned my addr 172.21.15.5:0/4074161775 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:58:27.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:27.722+0000 7ffa6fe00640 1 --2- 172.21.15.5:0/4074161775 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ffa70074e30 0x7ffa7019aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:27.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:27.722+0000 7ffa75a00640 1 --2- 172.21.15.5:0/4074161775 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ffa7010e280 0x7ffa7019ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:27.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:27.722+0000 7ffa6fe00640 1 -- 172.21.15.5:0/4074161775 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ffa7010e280 msgr2=0x7ffa7019ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:27.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:27.722+0000 7ffa6fe00640 1 --2- 172.21.15.5:0/4074161775 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ffa7010e280 0x7ffa7019ed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:27.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:27.722+0000 7ffa6fe00640 1 -- 172.21.15.5:0/4074161775 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffa70074510 msgr2=0x7ffa7019a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:27.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:27.722+0000 7ffa6fe00640 1 --2- 172.21.15.5:0/4074161775 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffa70074510 0x7ffa7019a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:27.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:27.722+0000 7ffa6fe00640 1 -- 172.21.15.5:0/4074161775 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa7019f470 con 0x7ffa70074e30 2024-09-10T09:58:27.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:27.734+0000 7ffa6fe00640 1 --2- 172.21.15.5:0/4074161775 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ffa70074e30 0x7ffa7019aa00 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7ffa5c005fd0 tx=0x7ffa5c00ee50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:27.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:27.734+0000 7ffa6d600640 1 -- 172.21.15.5:0/4074161775 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa5c00cd00 con 0x7ffa70074e30 2024-09-10T09:58:27.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:27.734+0000 7ffa76400640 1 -- 172.21.15.5:0/4074161775 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa7019f760 con 0x7ffa70074e30 2024-09-10T09:58:27.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:27.734+0000 7ffa6d600640 1 -- 172.21.15.5:0/4074161775 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa5c0035f0 con 0x7ffa70074e30 2024-09-10T09:58:27.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:27.734+0000 7ffa6d600640 1 -- 172.21.15.5:0/4074161775 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa5c010670 con 0x7ffa70074e30 2024-09-10T09:58:27.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:27.734+0000 7ffa76400640 1 -- 172.21.15.5:0/4074161775 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa701a7040 con 0x7ffa70074e30 2024-09-10T09:58:27.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:27.734+0000 7ffa7a412740 1 Processor -- start 2024-09-10T09:58:27.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:27.734+0000 7ffa7a412740 1 -- start start 2024-09-10T09:58:27.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:27.734+0000 7ffa6d600640 1 -- 172.21.15.5:0/4074161775 <== mon.2 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7ffa5c0037a0 con 0x7ffa70074e30 2024-09-10T09:58:27.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:27.734+0000 7ffa7a412740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55db4673e520 0x55db4673e900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:27.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:27.734+0000 7ffa7a412740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55db4673ee40 0x55db467491c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:27.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:27.739+0000 7ffa7a412740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55db46749700 0x55db4674baf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:27.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:27.739+0000 7ffa7a412740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55db466a4650 con 0x55db46749700 2024-09-10T09:58:27.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:27.739+0000 7ffa7a412740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55db4664bf10 con 0x55db4673e520 2024-09-10T09:58:27.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:27.739+0000 7ffa7a412740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55db466a0610 con 0x55db4673ee40 2024-09-10T09:58:27.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:27.739+0000 7ffa75a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55db4673e520 0x55db4673e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:27.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:27.739+0000 7ffa75a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55db4673e520 0x55db4673e900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:32928/0 (socket says 172.21.15.5:32928) 2024-09-10T09:58:27.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:27.739+0000 7ffa75a00640 1 -- 172.21.15.5:0/3254179651 learned_addr learned my addr 172.21.15.5:0/3254179651 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:58:27.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:27.739+0000 7ffa6fe00640 1 --2- 172.21.15.5:0/3254179651 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55db46749700 0x55db4674baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:27.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:27.739+0000 7ffa75a00640 1 -- 172.21.15.5:0/3254179651 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55db4673ee40 msgr2=0x55db467491c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T09:58:27.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:27.739+0000 7ffa75a00640 1 --2- 172.21.15.5:0/3254179651 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55db4673ee40 0x55db467491c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:27.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:27.739+0000 7ffa75a00640 1 -- 172.21.15.5:0/3254179651 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55db46749700 msgr2=0x55db4674baf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:58:27.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:27.739+0000 7ffa75a00640 1 --2- 172.21.15.5:0/3254179651 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55db46749700 0x55db4674baf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:27.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:27.739+0000 7ffa75a00640 1 -- 172.21.15.5:0/3254179651 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55db463b3580 con 0x55db4673e520 2024-09-10T09:58:27.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:27.739+0000 7ffa6fe00640 1 --2- 172.21.15.5:0/3254179651 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55db46749700 0x55db4674baf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:58:27.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:27.739+0000 7ffa75a00640 1 --2- 172.21.15.5:0/3254179651 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55db4673e520 0x55db4673e900 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7ffa68009f00 tx=0x7ffa680107a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7b4e51dab8888108 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:27.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:27.739+0000 7ffa62000640 1 -- 172.21.15.5:0/3254179651 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa68003b70 con 0x55db4673e520 2024-09-10T09:58:27.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:27.739+0000 7ffa62000640 1 -- 172.21.15.5:0/3254179651 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa68003d10 con 0x55db4673e520 2024-09-10T09:58:27.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:27.739+0000 7ffa62000640 1 -- 172.21.15.5:0/3254179651 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa68011a40 con 0x55db4673e520 2024-09-10T09:58:27.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:27.739+0000 7ffa6d600640 1 --2- 172.21.15.5:0/4074161775 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7ffa580a2b80 0x7ffa580a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:27.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:27.739+0000 7ffa6d600640 1 -- 172.21.15.5:0/4074161775 <== mon.2 v2:172.21.15.102:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7ffa5c0cf6a0 con 0x7ffa70074e30 2024-09-10T09:58:27.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:27.739+0000 7ffa75000640 1 --2- 172.21.15.5:0/4074161775 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7ffa580a2b80 0x7ffa580a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:27.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:27.739+0000 7ffa7a412740 1 -- 172.21.15.5:0/3254179651 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55db4673e520 msgr2=0x55db4673e900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:27.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:27.739+0000 7ffa7a412740 1 --2- 172.21.15.5:0/3254179651 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55db4673e520 0x55db4673e900 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7ffa68009f00 tx=0x7ffa680107a0 comp rx=0 tx=0).stop 2024-09-10T09:58:27.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:27.739+0000 7ffa7a412740 1 -- 172.21.15.5:0/3254179651 shutdown_connections 2024-09-10T09:58:27.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:27.739+0000 7ffa7a412740 1 --2- 172.21.15.5:0/3254179651 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55db46749700 0x55db4674baf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:27.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:27.739+0000 7ffa7a412740 1 --2- 172.21.15.5:0/3254179651 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55db4673ee40 0x55db467491c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:27.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:27.739+0000 7ffa7a412740 1 --2- 172.21.15.5:0/3254179651 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55db4673e520 0x55db4673e900 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:27.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:27.739+0000 7ffa7a412740 1 -- 172.21.15.5:0/3254179651 >> 172.21.15.5:0/3254179651 conn(0x55db4673aab0 msgr2=0x55db4673ae90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:58:27.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:27.739+0000 7ffa7a412740 1 -- 172.21.15.5:0/3254179651 shutdown_connections 2024-09-10T09:58:27.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:27.739+0000 7ffa7a412740 1 -- 172.21.15.5:0/3254179651 wait complete. 2024-09-10T09:58:27.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:27.739+0000 7ffa75000640 1 --2- 172.21.15.5:0/4074161775 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7ffa580a2b80 0x7ffa580a5040 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7ffa640095f0 tx=0x7ffa64005cb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:27.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:27.739+0000 7ffa7a412740 1 -- start start 2024-09-10T09:58:27.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:27.739+0000 7ffa7a412740 1 Processor -- start 2024-09-10T09:58:27.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:27.739+0000 7ffa7a412740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55db4673e520 0x55db46746660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:27.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:27.739+0000 7ffa7a412740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55db4673ee40 0x55db46746ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:27.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:27.739+0000 7ffa7a412740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55db46749700 0x55db467470e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:27.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:27.739+0000 7ffa7a412740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55db4674c2d0 con 0x55db46749700 2024-09-10T09:58:27.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:27.739+0000 7ffa7a412740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55db4665f350 con 0x55db4673ee40 2024-09-10T09:58:27.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:27.739+0000 7ffa7a412740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55db4674c450 con 0x55db4673e520 2024-09-10T09:58:27.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:27.739+0000 7ffa7a412740 10 client.0 _ll_register_callbacks cb 0x55db46682290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:58:27.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:27.739+0000 7ffa6fe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55db4673ee40 0x55db46746ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:27.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:27.739+0000 7ffa75000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55db4673e520 0x55db46746660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:27.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:27.739+0000 7ffa6fe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55db4673ee40 0x55db46746ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:32934/0 (socket says 172.21.15.5:32934) 2024-09-10T09:58:27.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:27.739+0000 7ffa75a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55db46749700 0x55db467470e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:27.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:27.739+0000 7ffa6fe00640 1 -- 172.21.15.5:0/778929341 learned_addr learned my addr 172.21.15.5:0/778929341 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:58:27.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:27.739+0000 7ffa6fe00640 1 -- 172.21.15.5:0/778929341 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55db4673e520 msgr2=0x55db46746660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:58:27.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:27.739+0000 7ffa6fe00640 1 --2- 172.21.15.5:0/778929341 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55db4673e520 0x55db46746660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:27.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:27.739+0000 7ffa6fe00640 1 -- 172.21.15.5:0/778929341 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55db46749700 msgr2=0x55db467470e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:58:27.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:27.739+0000 7ffa6fe00640 1 --2- 172.21.15.5:0/778929341 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55db46749700 0x55db467470e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:27.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:27.739+0000 7ffa6fe00640 1 -- 172.21.15.5:0/778929341 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55db4647d960 con 0x55db4673ee40 2024-09-10T09:58:27.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:27.739+0000 7ffa75a00640 1 --2- 172.21.15.5:0/778929341 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55db46749700 0x55db467470e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:58:27.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:27.739+0000 7ffa6fe00640 1 --2- 172.21.15.5:0/778929341 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55db4673ee40 0x55db46746ba0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7ffa5c0ccdb0 tx=0x7ffa5c09b470 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9af84128551cf9b6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:27.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:27.739+0000 7ffa56a00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-09-10T09:58:27.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:27.744+0000 7ffa56a00640 1 -- 172.21.15.5:0/778929341 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa5c09bb10 con 0x55db4673ee40 2024-09-10T09:58:27.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:27.744+0000 7ffa7a412740 1 -- 172.21.15.5:0/778929341 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffa68003d10 con 0x55db4673ee40 2024-09-10T09:58:27.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:27.744+0000 7ffa56a00640 1 -- 172.21.15.5:0/778929341 <== mon.2 v2:172.21.15.102:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7ffa5c0cf370 con 0x55db4673ee40 2024-09-10T09:58:27.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:27.744+0000 7ffa56a00640 1 -- 172.21.15.5:0/778929341 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7ffa5c0b9050 con 0x55db4673ee40 2024-09-10T09:58:27.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:27.744+0000 7ffa7a412740 10 client.29327 fetch_fsmap learned FSMap version 42 2024-09-10T09:58:27.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:27.744+0000 7ffa7a412740 1 -- 172.21.15.5:0/778929341 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55db46757fa0 con 0x55db4673ee40 2024-09-10T09:58:27.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:27.744+0000 7ffa56a00640 1 -- 172.21.15.5:0/778929341 <== mon.2 v2:172.21.15.102:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa68003d10 con 0x55db4673ee40 2024-09-10T09:58:27.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:27.744+0000 7ffa56a00640 1 -- 172.21.15.5:0/778929341 <== mon.2 v2:172.21.15.102:3300/0 5 ==== fsmap(e 42) ==== 8370+0+0 (secure 0 0 0) 0x7ffa5c0cfb50 con 0x55db4673ee40 2024-09-10T09:58:27.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:27.744+0000 7ffa7a412740 10 client.29327 fetch_fsmap finished waiting for FSMap version 42 2024-09-10T09:58:27.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:27.744+0000 7ffa7a412740 10 client.29327 resolve_mds: validated gid 24542 aka daemon mds.b 2024-09-10T09:58:27.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:27.744+0000 7ffa7a412740 20 client.29327 populate_metadata read hostname 'smithi005' 2024-09-10T09:58:27.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:27.744+0000 7ffa7a412740 1 --2- 172.21.15.5:0/778929341 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55db467593b0 0x55db4675b800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:27.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:27.744+0000 7ffa7a412740 4 client.29327 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:58:27.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:27.744+0000 7ffa7a412740 1 -- 172.21.15.5:0/778929341 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55db466b1bc0 con 0x55db467593b0 2024-09-10T09:58:27.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:27.745+0000 7ffa75000640 1 --2- 172.21.15.5:0/778929341 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55db467593b0 0x55db4675b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:27.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:27.745+0000 7ffa75000640 1 --2- 172.21.15.5:0/778929341 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55db467593b0 0x55db4675b800 crc :-1 s=READY pgs=281 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=78a9af16b5a725aa server_cookie=9b92eb57a6e87a90 in_seq=0 out_seq=0 2024-09-10T09:58:27.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:27.745+0000 7ffa56a00640 10 client.29327 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T09:58:27.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:27.745+0000 7ffa56a00640 1 -- 172.21.15.5:0/778929341 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55db466b1bc0 con 0x55db467593b0 2024-09-10T09:58:27.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:27.745+0000 7ffa56a00640 10 client.29327 handle_command_reply: tid=0 2024-09-10T09:58:27.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:27.745+0000 7ffa7a412740 1 client.29327 shutdown 2024-09-10T09:58:27.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:27.745+0000 7ffa7a412740 1 -- 172.21.15.5:0/778929341 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55db4673ee40 msgr2=0x55db46746ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:27.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:27.745+0000 7ffa7a412740 1 --2- 172.21.15.5:0/778929341 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55db4673ee40 0x55db46746ba0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7ffa5c0ccdb0 tx=0x7ffa5c09b470 comp rx=0 tx=0).stop 2024-09-10T09:58:27.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:27.745+0000 7ffa7a412740 1 -- 172.21.15.5:0/778929341 shutdown_connections 2024-09-10T09:58:27.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:27.745+0000 7ffa7a412740 1 --2- 172.21.15.5:0/778929341 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55db467593b0 0x55db4675b800 crc :-1 s=READY pgs=281 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:27.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:27.745+0000 7ffa7a412740 1 --2- 172.21.15.5:0/778929341 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55db46749700 0x55db467470e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:27.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:27.745+0000 7ffa7a412740 1 --2- 172.21.15.5:0/778929341 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55db4673ee40 0x55db46746ba0 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:27.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:27.745+0000 7ffa7a412740 1 --2- 172.21.15.5:0/778929341 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55db4673e520 0x55db46746660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:27.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:27.745+0000 7ffa56a00640 0 client.29327 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T09:58:27.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:27.745+0000 7ffa7a412740 1 -- 172.21.15.5:0/778929341 >> 172.21.15.5:0/778929341 conn(0x55db4673aab0 msgr2=0x55db46739cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:58:27.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:27.745+0000 7ffa7a412740 1 -- 172.21.15.5:0/778929341 shutdown_connections 2024-09-10T09:58:27.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:27.745+0000 7ffa7a412740 1 -- 172.21.15.5:0/778929341 wait complete. 2024-09-10T09:58:27.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:27.745+0000 7ffa7a412740 20 client.29327 trim_cache size 0 max 16384 2024-09-10T09:58:27.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:27.779+0000 7ffa7a412740 1 Processor -- start 2024-09-10T09:58:27.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:27.779+0000 7ffa7a412740 1 -- start start 2024-09-10T09:58:27.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:27.779+0000 7ffa7a412740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55db4673e520 0x55db46740f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:27.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:27.779+0000 7ffa7a412740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55db4673ee40 0x55db467610d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:27.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:27.779+0000 7ffa7a412740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55db46749700 0x55db46761610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:27.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:27.779+0000 7ffa7a412740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ffa701acb80 con 0x55db4673e520 2024-09-10T09:58:27.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:27.779+0000 7ffa7a412740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7ffa701ae6c0 con 0x55db46749700 2024-09-10T09:58:27.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:27.779+0000 7ffa7a412740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7ffa701ad250 con 0x55db4673ee40 2024-09-10T09:58:27.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:27.781+0000 7ffa75000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55db4673e520 0x55db46740f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:27.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:27.781+0000 7ffa75000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55db4673e520 0x55db46740f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42732/0 (socket says 172.21.15.5:42732) 2024-09-10T09:58:27.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:27.781+0000 7ffa75000640 1 -- 172.21.15.5:0/300299759 learned_addr learned my addr 172.21.15.5:0/300299759 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:58:27.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:27.781+0000 7ffa75a00640 1 --2- 172.21.15.5:0/300299759 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55db46749700 0x55db46761610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:27.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:27.781+0000 7ffa75000640 1 -- 172.21.15.5:0/300299759 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55db4673ee40 msgr2=0x55db467610d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:58:27.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:27.781+0000 7ffa75000640 1 --2- 172.21.15.5:0/300299759 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55db4673ee40 0x55db467610d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:27.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:27.781+0000 7ffa75000640 1 -- 172.21.15.5:0/300299759 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55db46749700 msgr2=0x55db46761610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:58:27.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:27.781+0000 7ffa75000640 1 --2- 172.21.15.5:0/300299759 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55db46749700 0x55db46761610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:27.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:27.781+0000 7ffa75000640 1 -- 172.21.15.5:0/300299759 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55db4674be60 con 0x55db4673e520 2024-09-10T09:58:27.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:27.781+0000 7ffa75000640 1 --2- 172.21.15.5:0/300299759 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55db4673e520 0x55db46740f80 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7ffa64000c00 tx=0x7ffa6403f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82b11767662cca5f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:27.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:27.781+0000 7ffa61600640 1 -- 172.21.15.5:0/300299759 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa640035f0 con 0x55db4673e520 2024-09-10T09:58:27.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:27.781+0000 7ffa61600640 1 -- 172.21.15.5:0/300299759 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa64059070 con 0x55db4673e520 2024-09-10T09:58:27.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:27.781+0000 7ffa61600640 1 -- 172.21.15.5:0/300299759 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa64054950 con 0x55db4673e520 2024-09-10T09:58:27.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:27.781+0000 7ffa7a412740 1 -- 172.21.15.5:0/300299759 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55db4673e520 msgr2=0x55db46740f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:27.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:27.781+0000 7ffa7a412740 1 --2- 172.21.15.5:0/300299759 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55db4673e520 0x55db46740f80 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7ffa64000c00 tx=0x7ffa6403f000 comp rx=0 tx=0).stop 2024-09-10T09:58:27.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:27.781+0000 7ffa7a412740 1 -- 172.21.15.5:0/300299759 shutdown_connections 2024-09-10T09:58:27.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:27.781+0000 7ffa7a412740 1 --2- 172.21.15.5:0/300299759 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55db46749700 0x55db46761610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:27.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:27.781+0000 7ffa7a412740 1 --2- 172.21.15.5:0/300299759 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55db4673ee40 0x55db467610d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:27.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:27.781+0000 7ffa7a412740 1 --2- 172.21.15.5:0/300299759 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55db4673e520 0x55db46740f80 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:27.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:27.781+0000 7ffa7a412740 1 -- 172.21.15.5:0/300299759 >> 172.21.15.5:0/300299759 conn(0x55db4673aab0 msgr2=0x55db46739cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:58:27.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:27.781+0000 7ffa7a412740 1 -- 172.21.15.5:0/300299759 shutdown_connections 2024-09-10T09:58:27.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:27.781+0000 7ffa7a412740 1 -- 172.21.15.5:0/300299759 wait complete. 2024-09-10T09:58:27.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:27.781+0000 7ffa7a412740 1 -- start start 2024-09-10T09:58:27.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:27.781+0000 7ffa7a412740 1 Processor -- start 2024-09-10T09:58:27.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:27.781+0000 7ffa7a412740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55db4673e520 0x55db467516a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:27.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:27.781+0000 7ffa7a412740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55db4673ee40 0x55db46751be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:27.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:27.781+0000 7ffa7a412740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55db46749700 0x55db4674e670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:27.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:27.781+0000 7ffa7a412740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ffa701adff0 con 0x55db4673e520 2024-09-10T09:58:27.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:27.781+0000 7ffa7a412740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7ffa701ad920 con 0x55db4673ee40 2024-09-10T09:58:27.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:27.781+0000 7ffa7a412740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7ffa701aab60 con 0x55db46749700 2024-09-10T09:58:27.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:27.781+0000 7ffa7a412740 10 client.0 _ll_register_callbacks cb 0x7ffa5c003dc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:58:27.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:27.781+0000 7ffa75a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55db46749700 0x55db4674e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:27.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:27.781+0000 7ffa6fe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55db4673ee40 0x55db46751be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:27.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:27.781+0000 7ffa75a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55db46749700 0x55db4674e670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:60838/0 (socket says 172.21.15.5:60838) 2024-09-10T09:58:27.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:27.781+0000 7ffa75a00640 1 -- 172.21.15.5:0/1856312911 learned_addr learned my addr 172.21.15.5:0/1856312911 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:58:27.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:27.781+0000 7ffa75000640 1 --2- 172.21.15.5:0/1856312911 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55db4673e520 0x55db467516a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:27.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:27.781+0000 7ffa6fe00640 1 -- 172.21.15.5:0/1856312911 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55db46749700 msgr2=0x55db4674e670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:58:27.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:27.781+0000 7ffa6fe00640 1 --2- 172.21.15.5:0/1856312911 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55db46749700 0x55db4674e670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:27.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:27.781+0000 7ffa6fe00640 1 -- 172.21.15.5:0/1856312911 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55db4673e520 msgr2=0x55db467516a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:58:27.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:27.781+0000 7ffa6fe00640 1 --2- 172.21.15.5:0/1856312911 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55db4673e520 0x55db467516a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:27.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:27.781+0000 7ffa6fe00640 1 -- 172.21.15.5:0/1856312911 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55db4674ebb0 con 0x55db4673ee40 2024-09-10T09:58:27.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:27.781+0000 7ffa6fe00640 1 --2- 172.21.15.5:0/1856312911 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55db4673ee40 0x55db46751be0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x55db4673f660 tx=0x7ffa5c003d20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b268fd204be39117 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:27.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:27.781+0000 7ffa56a00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-09-10T09:58:27.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:27.781+0000 7ffa56a00640 1 -- 172.21.15.5:0/1856312911 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa5c0b4070 con 0x55db4673ee40 2024-09-10T09:58:27.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:27.781+0000 7ffa56a00640 1 -- 172.21.15.5:0/1856312911 <== mon.2 v2:172.21.15.102:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7ffa5c0ba240 con 0x55db4673ee40 2024-09-10T09:58:27.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:27.781+0000 7ffa7a412740 1 -- 172.21.15.5:0/1856312911 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffa64059070 con 0x55db4673ee40 2024-09-10T09:58:27.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:27.781+0000 7ffa56a00640 1 -- 172.21.15.5:0/1856312911 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7ffa5c09bb10 con 0x55db4673ee40 2024-09-10T09:58:27.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:27.781+0000 7ffa7a412740 10 client.29333 fetch_fsmap learned FSMap version 42 2024-09-10T09:58:27.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:27.781+0000 7ffa7a412740 1 -- 172.21.15.5:0/1856312911 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55db467c4850 con 0x55db4673ee40 2024-09-10T09:58:27.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:27.781+0000 7ffa56a00640 1 -- 172.21.15.5:0/1856312911 <== mon.2 v2:172.21.15.102:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa64059070 con 0x55db4673ee40 2024-09-10T09:58:27.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:27.781+0000 7ffa56a00640 1 -- 172.21.15.5:0/1856312911 <== mon.2 v2:172.21.15.102:3300/0 5 ==== fsmap(e 42) ==== 8370+0+0 (secure 0 0 0) 0x7ffa5c097860 con 0x55db4673ee40 2024-09-10T09:58:27.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:27.781+0000 7ffa7a412740 10 client.29333 fetch_fsmap finished waiting for FSMap version 42 2024-09-10T09:58:27.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:27.786+0000 7ffa7a412740 10 client.29333 resolve_mds: validated gid 24542 aka daemon mds.b 2024-09-10T09:58:27.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:27.786+0000 7ffa7a412740 20 client.29333 populate_metadata read hostname 'smithi005' 2024-09-10T09:58:27.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:27.786+0000 7ffa7a412740 1 --2- 172.21.15.5:0/1856312911 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55db467c4f40 0x55db467c7360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:27.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:27.786+0000 7ffa7a412740 4 client.29333 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "b9f7a64f", "release": true, "await": true}] 2024-09-10T09:58:27.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:27.786+0000 7ffa7a412740 1 -- 172.21.15.5:0/1856312911 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "quiesce db", "set_id": "b9f7a64f", "release": true, "await": true}) -- 0x55db466ac0d0 con 0x55db467c4f40 2024-09-10T09:58:27.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:27.786+0000 7ffa75000640 1 --2- 172.21.15.5:0/1856312911 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55db467c4f40 0x55db467c7360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:27.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:27.786+0000 7ffa75000640 1 --2- 172.21.15.5:0/1856312911 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55db467c4f40 0x55db467c7360 crc :-1 s=READY pgs=282 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4f83d015ef96c085 server_cookie=19a0e935bfda678d in_seq=0 out_seq=0 2024-09-10T09:58:27.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:27.786+0000 7ffa56a00640 10 client.29333 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T09:58:27.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:27.844+0000 7ffa56a00640 1 -- 172.21.15.5:0/1856312911 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55db466ac0d0 con 0x55db467c4f40 2024-09-10T09:58:27.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:27.844+0000 7ffa56a00640 10 client.29333 handle_command_reply: tid=0 2024-09-10T09:58:27.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:27.844+0000 7ffa7a412740 1 client.29333 shutdown 2024-09-10T09:58:27.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:27.844+0000 7ffa7a412740 1 -- 172.21.15.5:0/1856312911 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55db4673ee40 msgr2=0x55db46751be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:27.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:27.844+0000 7ffa7a412740 1 --2- 172.21.15.5:0/1856312911 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55db4673ee40 0x55db46751be0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x55db4673f660 tx=0x7ffa5c003d20 comp rx=0 tx=0).stop 2024-09-10T09:58:27.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:27.844+0000 7ffa7a412740 1 -- 172.21.15.5:0/1856312911 shutdown_connections 2024-09-10T09:58:27.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:27.844+0000 7ffa7a412740 1 --2- 172.21.15.5:0/1856312911 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55db467c4f40 0x55db467c7360 crc :-1 s=READY pgs=282 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:27.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:27.844+0000 7ffa7a412740 1 --2- 172.21.15.5:0/1856312911 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55db46749700 0x55db4674e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:27.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:27.844+0000 7ffa7a412740 1 --2- 172.21.15.5:0/1856312911 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55db4673ee40 0x55db46751be0 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:27.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:27.844+0000 7ffa7a412740 1 --2- 172.21.15.5:0/1856312911 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55db4673e520 0x55db467516a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:27.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:27.844+0000 7ffa56a00640 0 client.29333 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T09:58:27.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:27.844+0000 7ffa7a412740 1 -- 172.21.15.5:0/1856312911 >> 172.21.15.5:0/1856312911 conn(0x55db4673aab0 msgr2=0x55db46747cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:58:27.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:27.844+0000 7ffa7a412740 1 -- 172.21.15.5:0/1856312911 shutdown_connections 2024-09-10T09:58:27.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:27.844+0000 7ffa7a412740 1 -- 172.21.15.5:0/1856312911 wait complete. 2024-09-10T09:58:27.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:27.844+0000 7ffa7a412740 20 client.29333 trim_cache size 0 max 16384 2024-09-10T09:58:27.847 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-10T09:58:27.847 INFO:teuthology.orchestra.run.smithi005.stdout: "epoch": 42, 2024-09-10T09:58:27.847 INFO:teuthology.orchestra.run.smithi005.stdout: "leader": 24542, 2024-09-10T09:58:27.847 INFO:teuthology.orchestra.run.smithi005.stdout: "set_version": 72, 2024-09-10T09:58:27.847 INFO:teuthology.orchestra.run.smithi005.stdout: "sets": { 2024-09-10T09:58:27.847 INFO:teuthology.orchestra.run.smithi005.stdout: "b9f7a64f": { 2024-09-10T09:58:27.847 INFO:teuthology.orchestra.run.smithi005.stdout: "version": 72, 2024-09-10T09:58:27.847 INFO:teuthology.orchestra.run.smithi005.stdout: "age_ref": 0.0, 2024-09-10T09:58:27.847 INFO:teuthology.orchestra.run.smithi005.stdout: "state": { 2024-09-10T09:58:27.847 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "RELEASED", 2024-09-10T09:58:27.847 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0 2024-09-10T09:58:27.848 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T09:58:27.848 INFO:teuthology.orchestra.run.smithi005.stdout: "timeout": 90.0, 2024-09-10T09:58:27.848 INFO:teuthology.orchestra.run.smithi005.stdout: "expiration": 132.8, 2024-09-10T09:58:27.848 INFO:teuthology.orchestra.run.smithi005.stdout: "members": { 2024-09-10T09:58:27.848 INFO:teuthology.orchestra.run.smithi005.stdout: "file:/": { 2024-09-10T09:58:27.848 INFO:teuthology.orchestra.run.smithi005.stdout: "excluded": false, 2024-09-10T09:58:27.848 INFO:teuthology.orchestra.run.smithi005.stdout: "state": { 2024-09-10T09:58:27.848 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "RELEASED", 2024-09-10T09:58:27.848 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0 2024-09-10T09:58:27.848 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T09:58:27.848 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T09:58:27.848 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T09:58:27.848 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T09:58:27.848 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T09:58:27.849 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-10T09:58:27.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:27.844+0000 7ffa76400640 1 -- 172.21.15.5:0/4074161775 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7ffa580a2b80 msgr2=0x7ffa580a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:27.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:27.844+0000 7ffa76400640 1 --2- 172.21.15.5:0/4074161775 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7ffa580a2b80 0x7ffa580a5040 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7ffa640095f0 tx=0x7ffa64005cb0 comp rx=0 tx=0).stop 2024-09-10T09:58:27.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:27.844+0000 7ffa76400640 1 -- 172.21.15.5:0/4074161775 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ffa70074e30 msgr2=0x7ffa7019aa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:27.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:27.844+0000 7ffa76400640 1 --2- 172.21.15.5:0/4074161775 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ffa70074e30 0x7ffa7019aa00 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7ffa5c005fd0 tx=0x7ffa5c00ee50 comp rx=0 tx=0).stop 2024-09-10T09:58:27.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:27.844+0000 7ffa76400640 1 -- 172.21.15.5:0/4074161775 shutdown_connections 2024-09-10T09:58:27.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:27.844+0000 7ffa76400640 1 --2- 172.21.15.5:0/4074161775 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7ffa580a2b80 0x7ffa580a5040 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:27.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:27.844+0000 7ffa76400640 1 --2- 172.21.15.5:0/4074161775 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ffa7010e280 0x7ffa7019ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:27.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:27.844+0000 7ffa76400640 1 --2- 172.21.15.5:0/4074161775 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ffa70074e30 0x7ffa7019aa00 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:27.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:27.844+0000 7ffa76400640 1 --2- 172.21.15.5:0/4074161775 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffa70074510 0x7ffa7019a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:27.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:27.844+0000 7ffa76400640 1 -- 172.21.15.5:0/4074161775 >> 172.21.15.5:0/4074161775 conn(0x7ffa700fd740 msgr2=0x7ffa700fee50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:58:27.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:27.844+0000 7ffa76400640 1 -- 172.21.15.5:0/4074161775 shutdown_connections 2024-09-10T09:58:27.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:27.844+0000 7ffa76400640 1 -- 172.21.15.5:0/4074161775 wait complete. 2024-09-10T09:58:27.900 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-10T09:58:27.900 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-10T09:58:27.952 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:58:27 smithi005 ceph-mon[30350]: pgmap v996: 129 pgs: 129 active+clean; 730 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 9.0 MiB/s wr, 343 op/s 2024-09-10T09:58:28.022 INFO:tasks.workunit.client.0.smithi005.stdout: 10 107580 15.03 MB/sec execute 246 sec latency 16321.863 ms 2024-09-10T09:58:28.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:58:27 smithi159 ceph-mon[35513]: pgmap v996: 129 pgs: 129 active+clean; 730 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 9.0 MiB/s wr, 343 op/s 2024-09-10T09:58:28.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:58:27 smithi102 ceph-mon[35442]: pgmap v996: 129 pgs: 129 active+clean; 730 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 9.0 MiB/s wr, 343 op/s 2024-09-10T09:58:29.022 INFO:tasks.workunit.client.0.smithi005.stdout: 10 107580 14.97 MB/sec execute 247 sec latency 17321.926 ms 2024-09-10T09:58:29.086 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'b9f7a64f', total seconds elapsed: 26.4 2024-09-10T09:58:29.123 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:58:29.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:29.810+0000 7fdb6ca00640 1 -- 172.21.15.5:0/738958402 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdb68077040 msgr2=0x7fdb681124f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:29.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:29.810+0000 7fdb6ca00640 1 --2- 172.21.15.5:0/738958402 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdb68077040 0x7fdb681124f0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fdb54009900 tx=0x7fdb5402a410 comp rx=0 tx=0).stop 2024-09-10T09:58:29.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:29.811+0000 7fdb6ca00640 1 -- 172.21.15.5:0/738958402 shutdown_connections 2024-09-10T09:58:29.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:29.811+0000 7fdb6ca00640 1 --2- 172.21.15.5:0/738958402 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdb68112a30 0x7fdb68114e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:29.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:29.811+0000 7fdb6ca00640 1 --2- 172.21.15.5:0/738958402 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdb68077040 0x7fdb681124f0 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:29.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:29.811+0000 7fdb6ca00640 1 --2- 172.21.15.5:0/738958402 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdb68076720 0x7fdb68076b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:29.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:29.811+0000 7fdb6ca00640 1 -- 172.21.15.5:0/738958402 >> 172.21.15.5:0/738958402 conn(0x7fdb680ffa90 msgr2=0x7fdb68101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:58:29.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:29.811+0000 7fdb6ca00640 1 -- 172.21.15.5:0/738958402 shutdown_connections 2024-09-10T09:58:29.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:29.811+0000 7fdb6ca00640 1 -- 172.21.15.5:0/738958402 wait complete. 2024-09-10T09:58:29.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:29.811+0000 7fdb6ca00640 1 Processor -- start 2024-09-10T09:58:29.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:29.811+0000 7fdb6ca00640 1 -- start start 2024-09-10T09:58:29.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:29.811+0000 7fdb6ca00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdb68076720 0x7fdb6819ee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:29.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:29.811+0000 7fdb6ca00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdb68112a30 0x7fdb6819f350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:29.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:29.811+0000 7fdb6ca00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdb681a36e0 0x7fdb681a3b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:29.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:29.811+0000 7fdb6ca00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fdb68118640 con 0x7fdb68076720 2024-09-10T09:58:29.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:29.811+0000 7fdb6ca00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fdb681184c0 con 0x7fdb681a36e0 2024-09-10T09:58:29.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:29.811+0000 7fdb6ca00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fdb681187c0 con 0x7fdb68112a30 2024-09-10T09:58:29.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:29.811+0000 7fdb66800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdb681a36e0 0x7fdb681a3b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:29.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:29.811+0000 7fdb66800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdb681a36e0 0x7fdb681a3b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:32960/0 (socket says 172.21.15.5:32960) 2024-09-10T09:58:29.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:29.811+0000 7fdb66800640 1 -- 172.21.15.5:0/4028429496 learned_addr learned my addr 172.21.15.5:0/4028429496 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:58:29.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:29.811+0000 7fdb65400640 1 --2- 172.21.15.5:0/4028429496 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdb68112a30 0x7fdb6819f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:29.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:29.811+0000 7fdb65e00640 1 --2- 172.21.15.5:0/4028429496 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdb68076720 0x7fdb6819ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:29.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:29.811+0000 7fdb66800640 1 -- 172.21.15.5:0/4028429496 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdb68112a30 msgr2=0x7fdb6819f350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:29.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:29.811+0000 7fdb66800640 1 --2- 172.21.15.5:0/4028429496 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdb68112a30 0x7fdb6819f350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:29.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:29.811+0000 7fdb66800640 1 -- 172.21.15.5:0/4028429496 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdb68076720 msgr2=0x7fdb6819ee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:29.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:29.811+0000 7fdb66800640 1 --2- 172.21.15.5:0/4028429496 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdb68076720 0x7fdb6819ee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:29.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:29.811+0000 7fdb66800640 1 -- 172.21.15.5:0/4028429496 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb681a41a0 con 0x7fdb681a36e0 2024-09-10T09:58:29.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:29.811+0000 7fdb66800640 1 --2- 172.21.15.5:0/4028429496 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdb681a36e0 0x7fdb681a3b90 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fdb5800b390 tx=0x7fdb5800b850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:29.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:29.816+0000 7fdb5ea00640 1 -- 172.21.15.5:0/4028429496 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb58015020 con 0x7fdb681a36e0 2024-09-10T09:58:29.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:29.816+0000 7fdb6ca00640 1 -- 172.21.15.5:0/4028429496 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb681a4490 con 0x7fdb681a36e0 2024-09-10T09:58:29.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:29.816+0000 7fdb6ca00640 1 -- 172.21.15.5:0/4028429496 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb681abcc0 con 0x7fdb681a36e0 2024-09-10T09:58:29.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:29.816+0000 7fdb5ea00640 1 -- 172.21.15.5:0/4028429496 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdb58011d10 con 0x7fdb681a36e0 2024-09-10T09:58:29.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:29.816+0000 7fdb5ea00640 1 -- 172.21.15.5:0/4028429496 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb5800f2e0 con 0x7fdb681a36e0 2024-09-10T09:58:29.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:29.816+0000 7fdb5ea00640 1 -- 172.21.15.5:0/4028429496 <== mon.2 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7fdb58018020 con 0x7fdb681a36e0 2024-09-10T09:58:29.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:29.821+0000 7fdb6f5f6740 1 Processor -- start 2024-09-10T09:58:29.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:29.821+0000 7fdb6f5f6740 1 -- start start 2024-09-10T09:58:29.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:29.821+0000 7fdb6f5f6740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56480ae68ac0 0x56480ae68ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:29.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:29.821+0000 7fdb6f5f6740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56480ae693e0 0x56480ae73760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:29.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:29.821+0000 7fdb6f5f6740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56480ae73ca0 0x56480ae76090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:29.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:29.821+0000 7fdb6f5f6740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x56480adcf290 con 0x56480ae73ca0 2024-09-10T09:58:29.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:29.821+0000 7fdb6f5f6740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x56480ad76b40 con 0x56480ae693e0 2024-09-10T09:58:29.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:29.821+0000 7fdb6f5f6740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x56480adcb250 con 0x56480ae68ac0 2024-09-10T09:58:29.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:29.821+0000 7fdb5ea00640 1 --2- 172.21.15.5:0/4028429496 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fdb4c0a2cc0 0x7fdb4c0a5180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:29.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:29.821+0000 7fdb65e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56480ae693e0 0x56480ae73760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:29.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:29.821+0000 7fdb65400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56480ae73ca0 0x56480ae76090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:29.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:29.821+0000 7fdb65400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56480ae73ca0 0x56480ae76090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42780/0 (socket says 172.21.15.5:42780) 2024-09-10T09:58:29.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:29.821+0000 7fdb65400640 1 -- 172.21.15.5:0/1322841298 learned_addr learned my addr 172.21.15.5:0/1322841298 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:58:29.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:29.821+0000 7fdb65400640 1 --2- 172.21.15.5:0/1322841298 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56480ae68ac0 0x56480ae68ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:29.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:29.821+0000 7fdb66800640 1 --2- 172.21.15.5:0/4028429496 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fdb4c0a2cc0 0x7fdb4c0a5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:29.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:29.821+0000 7fdb65e00640 1 -- 172.21.15.5:0/1322841298 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56480ae68ac0 msgr2=0x56480ae68ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:58:29.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:29.821+0000 7fdb65e00640 1 --2- 172.21.15.5:0/1322841298 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56480ae68ac0 0x56480ae68ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:29.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:29.821+0000 7fdb65e00640 1 -- 172.21.15.5:0/1322841298 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56480ae73ca0 msgr2=0x56480ae76090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:58:29.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:29.821+0000 7fdb65e00640 1 --2- 172.21.15.5:0/1322841298 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56480ae73ca0 0x56480ae76090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:29.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:29.821+0000 7fdb5ea00640 1 -- 172.21.15.5:0/4028429496 <== mon.2 v2:172.21.15.102:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fdb580ce640 con 0x7fdb681a36e0 2024-09-10T09:58:29.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:29.821+0000 7fdb65e00640 1 -- 172.21.15.5:0/1322841298 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56480ab406a0 con 0x56480ae693e0 2024-09-10T09:58:29.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:29.821+0000 7fdb65e00640 1 --2- 172.21.15.5:0/1322841298 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56480ae693e0 0x56480ae73760 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7fdb60009ee0 tx=0x7fdb6000f730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b2eabf78d7d908b1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:29.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:29.826+0000 7fdb53000640 1 -- 172.21.15.5:0/1322841298 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb60011640 con 0x56480ae693e0 2024-09-10T09:58:29.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:29.826+0000 7fdb53000640 1 -- 172.21.15.5:0/1322841298 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdb60011cc0 con 0x56480ae693e0 2024-09-10T09:58:29.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:29.826+0000 7fdb53000640 1 -- 172.21.15.5:0/1322841298 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb6001bd20 con 0x56480ae693e0 2024-09-10T09:58:29.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:29.826+0000 7fdb6f5f6740 1 -- 172.21.15.5:0/1322841298 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56480ae693e0 msgr2=0x56480ae73760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:29.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:29.826+0000 7fdb6f5f6740 1 --2- 172.21.15.5:0/1322841298 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56480ae693e0 0x56480ae73760 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7fdb60009ee0 tx=0x7fdb6000f730 comp rx=0 tx=0).stop 2024-09-10T09:58:29.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:29.826+0000 7fdb66800640 1 --2- 172.21.15.5:0/4028429496 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fdb4c0a2cc0 0x7fdb4c0a5180 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fdb58006fd0 tx=0x7fdb580d1d10 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:29.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:29.829+0000 7fdb6f5f6740 1 -- 172.21.15.5:0/1322841298 shutdown_connections 2024-09-10T09:58:29.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:29.829+0000 7fdb6f5f6740 1 --2- 172.21.15.5:0/1322841298 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56480ae73ca0 0x56480ae76090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:29.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:29.829+0000 7fdb6f5f6740 1 --2- 172.21.15.5:0/1322841298 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56480ae693e0 0x56480ae73760 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:29.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:29.829+0000 7fdb6f5f6740 1 --2- 172.21.15.5:0/1322841298 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56480ae68ac0 0x56480ae68ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:29.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:29.829+0000 7fdb6f5f6740 1 -- 172.21.15.5:0/1322841298 >> 172.21.15.5:0/1322841298 conn(0x56480ae65050 msgr2=0x56480ae65430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:58:29.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:29.831+0000 7fdb6f5f6740 1 -- 172.21.15.5:0/1322841298 shutdown_connections 2024-09-10T09:58:29.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:29.831+0000 7fdb6f5f6740 1 -- 172.21.15.5:0/1322841298 wait complete. 2024-09-10T09:58:29.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:29.831+0000 7fdb6f5f6740 1 -- start start 2024-09-10T09:58:29.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:29.831+0000 7fdb6f5f6740 1 Processor -- start 2024-09-10T09:58:29.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:29.831+0000 7fdb6f5f6740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56480ae68ac0 0x56480ae735b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:29.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:29.831+0000 7fdb6f5f6740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56480ae693e0 0x56480ae70860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:29.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:29.831+0000 7fdb6f5f6740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56480ae73ca0 0x56480ae70da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:29.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:29.831+0000 7fdb6f5f6740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x56480ae76910 con 0x56480ae693e0 2024-09-10T09:58:29.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:29.831+0000 7fdb6f5f6740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x56480ad89fa0 con 0x56480ae73ca0 2024-09-10T09:58:29.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:29.831+0000 7fdb6f5f6740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x56480ae76a90 con 0x56480ae68ac0 2024-09-10T09:58:29.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:29.831+0000 7fdb6f5f6740 10 client.0 _ll_register_callbacks cb 0x56480aaddba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:58:29.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:29.831+0000 7fdb65400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56480ae73ca0 0x56480ae70da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:29.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:29.831+0000 7fdb65400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56480ae68ac0 0x56480ae735b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:29.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:29.831+0000 7fdb65400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56480ae68ac0 0x56480ae735b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:60876/0 (socket says 172.21.15.5:60876) 2024-09-10T09:58:29.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:29.831+0000 7fdb65400640 1 -- 172.21.15.5:0/784511806 learned_addr learned my addr 172.21.15.5:0/784511806 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:58:29.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:29.831+0000 7fdb65e00640 1 --2- 172.21.15.5:0/784511806 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56480ae693e0 0x56480ae70860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:29.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:29.831+0000 7fdb65400640 1 -- 172.21.15.5:0/784511806 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56480ae73ca0 msgr2=0x56480ae70da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:58:29.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:29.831+0000 7fdb65400640 1 --2- 172.21.15.5:0/784511806 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56480ae73ca0 0x56480ae70da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:29.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:29.831+0000 7fdb65400640 1 -- 172.21.15.5:0/784511806 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56480ae693e0 msgr2=0x56480ae70860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:58:29.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:29.831+0000 7fdb65400640 1 --2- 172.21.15.5:0/784511806 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56480ae693e0 0x56480ae70860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:29.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:29.831+0000 7fdb65400640 1 -- 172.21.15.5:0/784511806 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56480ae715a0 con 0x56480ae68ac0 2024-09-10T09:58:29.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:29.831+0000 7fdb65400640 1 --2- 172.21.15.5:0/784511806 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56480ae68ac0 0x56480ae735b0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fdb54005a70 tx=0x7fdb54005d50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=62d78e5e602aeecd server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:29.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:29.831+0000 7fdb47400640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-09-10T09:58:29.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:29.831+0000 7fdb47400640 1 -- 172.21.15.5:0/784511806 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb54035040 con 0x56480ae68ac0 2024-09-10T09:58:29.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:29.831+0000 7fdb47400640 1 -- 172.21.15.5:0/784511806 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fdb54003a00 con 0x56480ae68ac0 2024-09-10T09:58:29.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:29.836+0000 7fdb6f5f6740 1 -- 172.21.15.5:0/784511806 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdb60011cc0 con 0x56480ae68ac0 2024-09-10T09:58:29.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:29.836+0000 7fdb47400640 1 -- 172.21.15.5:0/784511806 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7fdb54032e10 con 0x56480ae68ac0 2024-09-10T09:58:29.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:29.837+0000 7fdb6f5f6740 10 client.29320 fetch_fsmap learned FSMap version 42 2024-09-10T09:58:29.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:29.837+0000 7fdb6f5f6740 1 -- 172.21.15.5:0/784511806 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x56480ae82850 con 0x56480ae68ac0 2024-09-10T09:58:29.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:29.837+0000 7fdb47400640 1 -- 172.21.15.5:0/784511806 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdb60011cc0 con 0x56480ae68ac0 2024-09-10T09:58:29.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:29.837+0000 7fdb47400640 1 -- 172.21.15.5:0/784511806 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 42) ==== 8370+0+0 (secure 0 0 0) 0x7fdb5403a040 con 0x56480ae68ac0 2024-09-10T09:58:29.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:29.837+0000 7fdb6f5f6740 10 client.29320 fetch_fsmap finished waiting for FSMap version 42 2024-09-10T09:58:29.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:29.837+0000 7fdb6f5f6740 10 client.29320 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T09:58:29.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:29.840+0000 7fdb6f5f6740 20 client.29320 populate_metadata read hostname 'smithi005' 2024-09-10T09:58:29.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:29.840+0000 7fdb6f5f6740 1 --2- 172.21.15.5:0/784511806 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x56480ae83d60 0x56480ae861c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:29.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:29.840+0000 7fdb6f5f6740 4 client.29320 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:58:29.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:29.840+0000 7fdb6f5f6740 1 -- 172.21.15.5:0/784511806 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56480addc800 con 0x56480ae83d60 2024-09-10T09:58:29.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:29.840+0000 7fdb65e00640 1 --2- 172.21.15.5:0/784511806 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x56480ae83d60 0x56480ae861c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:29.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:29.842+0000 7fdb65e00640 1 --2- 172.21.15.5:0/784511806 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x56480ae83d60 0x56480ae861c0 crc :-1 s=READY pgs=283 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6b3019c9191b054b server_cookie=35825368285b0e3f in_seq=0 out_seq=0 2024-09-10T09:58:29.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:29.842+0000 7fdb47400640 10 client.29320 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T09:58:29.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:29.843+0000 7fdb47400640 1 -- 172.21.15.5:0/784511806 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56480addc800 con 0x56480ae83d60 2024-09-10T09:58:29.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:29.843+0000 7fdb47400640 10 client.29320 handle_command_reply: tid=0 2024-09-10T09:58:29.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:29.843+0000 7fdb6f5f6740 1 client.29320 shutdown 2024-09-10T09:58:29.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:29.843+0000 7fdb6f5f6740 1 -- 172.21.15.5:0/784511806 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56480ae68ac0 msgr2=0x56480ae735b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:29.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:29.843+0000 7fdb6f5f6740 1 --2- 172.21.15.5:0/784511806 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56480ae68ac0 0x56480ae735b0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fdb54005a70 tx=0x7fdb54005d50 comp rx=0 tx=0).stop 2024-09-10T09:58:29.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:29.843+0000 7fdb6f5f6740 1 -- 172.21.15.5:0/784511806 shutdown_connections 2024-09-10T09:58:29.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:29.843+0000 7fdb6f5f6740 1 --2- 172.21.15.5:0/784511806 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x56480ae83d60 0x56480ae861c0 crc :-1 s=READY pgs=283 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:29.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:29.843+0000 7fdb6f5f6740 1 --2- 172.21.15.5:0/784511806 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56480ae73ca0 0x56480ae70da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:29.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:29.843+0000 7fdb6f5f6740 1 --2- 172.21.15.5:0/784511806 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56480ae693e0 0x56480ae70860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:29.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:29.843+0000 7fdb6f5f6740 1 --2- 172.21.15.5:0/784511806 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56480ae68ac0 0x56480ae735b0 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:29.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:29.843+0000 7fdb6f5f6740 1 -- 172.21.15.5:0/784511806 >> 172.21.15.5:0/784511806 conn(0x56480ae65050 msgr2=0x56480ae641c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:58:29.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:29.843+0000 7fdb47400640 0 client.29320 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T09:58:29.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:29.843+0000 7fdb6f5f6740 1 -- 172.21.15.5:0/784511806 shutdown_connections 2024-09-10T09:58:29.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:29.843+0000 7fdb6f5f6740 1 -- 172.21.15.5:0/784511806 wait complete. 2024-09-10T09:58:29.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:29.843+0000 7fdb6f5f6740 20 client.29320 trim_cache size 0 max 16384 2024-09-10T09:58:29.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:29.876+0000 7fdb6f5f6740 1 Processor -- start 2024-09-10T09:58:29.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:29.876+0000 7fdb6f5f6740 1 -- start start 2024-09-10T09:58:29.878 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:58:29 smithi005 ceph-mon[30350]: pgmap v997: 129 pgs: 129 active+clean; 732 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 232 KiB/s rd, 8.8 MiB/s wr, 254 op/s 2024-09-10T09:58:29.878 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:58:29 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:58:29.878 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:58:29 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:58:29.878 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:58:29 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:58:29.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:58:29 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:58:29.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:58:29 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:58:29.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:29.876+0000 7fdb6f5f6740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56480ae68ac0 0x56480ae640b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:29.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:29.876+0000 7fdb6f5f6740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56480ae693e0 0x56480ae7bd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:29.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:29.876+0000 7fdb6f5f6740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56480ae73ca0 0x56480ae7c280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:29.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:29.876+0000 7fdb6f5f6740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fdb4800fd20 con 0x56480ae73ca0 2024-09-10T09:58:29.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:29.876+0000 7fdb6f5f6740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fdb48011860 con 0x56480ae693e0 2024-09-10T09:58:29.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:29.876+0000 7fdb6f5f6740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fdb480103f0 con 0x56480ae68ac0 2024-09-10T09:58:29.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:29.876+0000 7fdb65400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56480ae73ca0 0x56480ae7c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:29.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:29.876+0000 7fdb65400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56480ae68ac0 0x56480ae640b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:29.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:29.876+0000 7fdb65400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56480ae68ac0 0x56480ae640b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:60882/0 (socket says 172.21.15.5:60882) 2024-09-10T09:58:29.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:29.876+0000 7fdb65400640 1 -- 172.21.15.5:0/3016796777 learned_addr learned my addr 172.21.15.5:0/3016796777 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:58:29.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:29.876+0000 7fdb65400640 1 -- 172.21.15.5:0/3016796777 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56480ae693e0 msgr2=0x56480ae7bd40 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T09:58:29.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:29.876+0000 7fdb65400640 1 --2- 172.21.15.5:0/3016796777 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56480ae693e0 0x56480ae7bd40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:29.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:29.876+0000 7fdb65400640 1 -- 172.21.15.5:0/3016796777 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56480ae73ca0 msgr2=0x56480ae7c280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:58:29.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:29.876+0000 7fdb65400640 1 --2- 172.21.15.5:0/3016796777 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56480ae73ca0 0x56480ae7c280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:29.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:29.876+0000 7fdb65400640 1 -- 172.21.15.5:0/3016796777 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56480ae65b90 con 0x56480ae68ac0 2024-09-10T09:58:29.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:29.888+0000 7fdb65400640 1 --2- 172.21.15.5:0/3016796777 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56480ae68ac0 0x56480ae640b0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fdb54033000 tx=0x7fdb54000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=738ccaea728bf00 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:29.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:29.888+0000 7fdb52600640 1 -- 172.21.15.5:0/3016796777 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb54035070 con 0x56480ae68ac0 2024-09-10T09:58:29.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:29.888+0000 7fdb52600640 1 -- 172.21.15.5:0/3016796777 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdb54009520 con 0x56480ae68ac0 2024-09-10T09:58:29.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:29.888+0000 7fdb52600640 1 -- 172.21.15.5:0/3016796777 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb54003ee0 con 0x56480ae68ac0 2024-09-10T09:58:29.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:29.888+0000 7fdb6f5f6740 1 -- 172.21.15.5:0/3016796777 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56480ae68ac0 msgr2=0x56480ae640b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:29.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:29.888+0000 7fdb6f5f6740 1 --2- 172.21.15.5:0/3016796777 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56480ae68ac0 0x56480ae640b0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fdb54033000 tx=0x7fdb54000bc0 comp rx=0 tx=0).stop 2024-09-10T09:58:29.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:29.888+0000 7fdb6f5f6740 1 -- 172.21.15.5:0/3016796777 shutdown_connections 2024-09-10T09:58:29.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:29.888+0000 7fdb6f5f6740 1 --2- 172.21.15.5:0/3016796777 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56480ae73ca0 0x56480ae7c280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:29.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:29.888+0000 7fdb6f5f6740 1 --2- 172.21.15.5:0/3016796777 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56480ae693e0 0x56480ae7bd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:29.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:29.888+0000 7fdb6f5f6740 1 --2- 172.21.15.5:0/3016796777 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56480ae68ac0 0x56480ae640b0 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:29.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:29.888+0000 7fdb6f5f6740 1 -- 172.21.15.5:0/3016796777 >> 172.21.15.5:0/3016796777 conn(0x56480ae65050 msgr2=0x56480ae639a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:58:29.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:29.888+0000 7fdb6f5f6740 1 -- 172.21.15.5:0/3016796777 shutdown_connections 2024-09-10T09:58:29.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:29.888+0000 7fdb6f5f6740 1 -- 172.21.15.5:0/3016796777 wait complete. 2024-09-10T09:58:29.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:29.888+0000 7fdb6f5f6740 1 -- start start 2024-09-10T09:58:29.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:29.888+0000 7fdb6f5f6740 1 Processor -- start 2024-09-10T09:58:29.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:29.894+0000 7fdb6f5f6740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56480ae68ac0 0x56480ae7c550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:29.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:29.894+0000 7fdb6f5f6740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56480ae693e0 0x56480ae7a900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:29.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:29.894+0000 7fdb6f5f6740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56480ae73ca0 0x56480ae7af80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:29.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:29.894+0000 7fdb6f5f6740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fdb48011190 con 0x56480ae68ac0 2024-09-10T09:58:29.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:29.894+0000 7fdb6f5f6740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fdb48010ac0 con 0x56480ae693e0 2024-09-10T09:58:29.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:29.894+0000 7fdb6f5f6740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fdb4800dd00 con 0x56480ae73ca0 2024-09-10T09:58:29.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:29.894+0000 7fdb6f5f6740 10 client.0 _ll_register_callbacks cb 0x7fdb5402aa90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:58:29.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:29.894+0000 7fdb65e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56480ae693e0 0x56480ae7a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:29.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:29.894+0000 7fdb65e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56480ae693e0 0x56480ae7a900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:33004/0 (socket says 172.21.15.5:33004) 2024-09-10T09:58:29.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:29.894+0000 7fdb65e00640 1 -- 172.21.15.5:0/843500547 learned_addr learned my addr 172.21.15.5:0/843500547 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:58:29.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:29.894+0000 7fdb65400640 1 --2- 172.21.15.5:0/843500547 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56480ae73ca0 0x56480ae7af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:29.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:29.894+0000 7fdb65e00640 1 -- 172.21.15.5:0/843500547 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56480ae73ca0 msgr2=0x56480ae7af80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:58:29.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:29.894+0000 7fdb65e00640 1 --2- 172.21.15.5:0/843500547 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56480ae73ca0 0x56480ae7af80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:29.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:29.894+0000 7fdb65e00640 1 -- 172.21.15.5:0/843500547 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56480ae68ac0 msgr2=0x56480ae7c550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:58:29.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:29.894+0000 7fdb65e00640 1 --2- 172.21.15.5:0/843500547 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56480ae68ac0 0x56480ae7c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:29.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:29.894+0000 7fdb65e00640 1 -- 172.21.15.5:0/843500547 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56480aeffa90 con 0x56480ae693e0 2024-09-10T09:58:29.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:29.894+0000 7fdb65400640 1 --2- 172.21.15.5:0/843500547 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56480ae73ca0 0x56480ae7af80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:58:29.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:29.894+0000 7fdb65e00640 1 --2- 172.21.15.5:0/843500547 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56480ae693e0 0x56480ae7a900 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7fdb60009ee0 tx=0x7fdb6001b800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=27c8b51634af4677 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:29.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:29.894+0000 7fdb47400640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-09-10T09:58:29.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:29.899+0000 7fdb47400640 1 -- 172.21.15.5:0/843500547 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb60004be0 con 0x56480ae693e0 2024-09-10T09:58:29.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:29.899+0000 7fdb47400640 1 -- 172.21.15.5:0/843500547 <== mon.2 v2:172.21.15.102:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fdb60004d80 con 0x56480ae693e0 2024-09-10T09:58:29.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:29.899+0000 7fdb6f5f6740 1 -- 172.21.15.5:0/843500547 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdb54009520 con 0x56480ae693e0 2024-09-10T09:58:29.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:29.900+0000 7fdb47400640 1 -- 172.21.15.5:0/843500547 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7fdb60009b00 con 0x56480ae693e0 2024-09-10T09:58:29.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:29.900+0000 7fdb6f5f6740 10 client.29348 fetch_fsmap learned FSMap version 42 2024-09-10T09:58:29.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:29.900+0000 7fdb6f5f6740 1 -- 172.21.15.5:0/843500547 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x56480af00c40 con 0x56480ae693e0 2024-09-10T09:58:29.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:29.900+0000 7fdb47400640 1 -- 172.21.15.5:0/843500547 <== mon.2 v2:172.21.15.102:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdb54009520 con 0x56480ae693e0 2024-09-10T09:58:29.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:29.900+0000 7fdb47400640 1 -- 172.21.15.5:0/843500547 <== mon.2 v2:172.21.15.102:3300/0 5 ==== fsmap(e 42) ==== 8370+0+0 (secure 0 0 0) 0x7fdb600119a0 con 0x56480ae693e0 2024-09-10T09:58:29.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:29.900+0000 7fdb6f5f6740 10 client.29348 fetch_fsmap finished waiting for FSMap version 42 2024-09-10T09:58:29.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:29.900+0000 7fdb6f5f6740 10 client.29348 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T09:58:29.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:29.900+0000 7fdb6f5f6740 20 client.29348 populate_metadata read hostname 'smithi005' 2024-09-10T09:58:29.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:29.900+0000 7fdb6f5f6740 1 --2- 172.21.15.5:0/843500547 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x56480af01170 0x56480af03590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:29.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:29.900+0000 7fdb6f5f6740 4 client.29348 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-10T09:58:29.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:29.900+0000 7fdb6f5f6740 1 -- 172.21.15.5:0/843500547 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x56480add6d10 con 0x56480af01170 2024-09-10T09:58:29.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:29.900+0000 7fdb65400640 1 --2- 172.21.15.5:0/843500547 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x56480af01170 0x56480af03590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:29.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:29.906+0000 7fdb65400640 1 --2- 172.21.15.5:0/843500547 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x56480af01170 0x56480af03590 crc :-1 s=READY pgs=284 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bc59460d20a9f6cb server_cookie=9cc0d4ebfcb4fe4 in_seq=0 out_seq=0 2024-09-10T09:58:29.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:29.906+0000 7fdb47400640 10 client.29348 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T09:58:29.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:29.906+0000 7fdb47400640 1 -- 172.21.15.5:0/843500547 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x56480add6d10 con 0x56480af01170 2024-09-10T09:58:29.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:29.906+0000 7fdb47400640 10 client.29348 handle_command_reply: tid=0 2024-09-10T09:58:29.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:29.906+0000 7fdb6f5f6740 1 client.29348 shutdown 2024-09-10T09:58:29.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:29.906+0000 7fdb6f5f6740 1 -- 172.21.15.5:0/843500547 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56480ae693e0 msgr2=0x56480ae7a900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:29.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:29.906+0000 7fdb6f5f6740 1 --2- 172.21.15.5:0/843500547 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56480ae693e0 0x56480ae7a900 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7fdb60009ee0 tx=0x7fdb6001b800 comp rx=0 tx=0).stop 2024-09-10T09:58:29.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:29.906+0000 7fdb6f5f6740 1 -- 172.21.15.5:0/843500547 shutdown_connections 2024-09-10T09:58:29.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:29.906+0000 7fdb6f5f6740 1 --2- 172.21.15.5:0/843500547 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x56480af01170 0x56480af03590 crc :-1 s=READY pgs=284 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:29.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:29.906+0000 7fdb6f5f6740 1 --2- 172.21.15.5:0/843500547 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56480ae73ca0 0x56480ae7af80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:29.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:29.906+0000 7fdb6f5f6740 1 --2- 172.21.15.5:0/843500547 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56480ae693e0 0x56480ae7a900 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:29.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:29.906+0000 7fdb6f5f6740 1 --2- 172.21.15.5:0/843500547 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56480ae68ac0 0x56480ae7c550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:29.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:29.906+0000 7fdb47400640 0 client.29348 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T09:58:29.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:29.906+0000 7fdb6f5f6740 1 -- 172.21.15.5:0/843500547 >> 172.21.15.5:0/843500547 conn(0x56480ae65050 msgr2=0x56480ae641c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:58:29.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:29.906+0000 7fdb6f5f6740 1 -- 172.21.15.5:0/843500547 shutdown_connections 2024-09-10T09:58:29.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:29.906+0000 7fdb6f5f6740 1 -- 172.21.15.5:0/843500547 wait complete. 2024-09-10T09:58:29.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:29.906+0000 7fdb6f5f6740 20 client.29348 trim_cache size 0 max 16384 2024-09-10T09:58:29.911 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-10T09:58:29.911 INFO:teuthology.orchestra.run.smithi005.stdout: "return_code": 0, 2024-09-10T09:58:29.911 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_tag": "305147c2-efd3-48fb-9a5d-0630485285b8", 2024-09-10T09:58:29.911 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": "asynchronous" 2024-09-10T09:58:29.911 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-10T09:58:29.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:29.906+0000 7fdb6ca00640 1 -- 172.21.15.5:0/4028429496 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fdb4c0a2cc0 msgr2=0x7fdb4c0a5180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:29.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:29.906+0000 7fdb6ca00640 1 --2- 172.21.15.5:0/4028429496 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fdb4c0a2cc0 0x7fdb4c0a5180 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fdb58006fd0 tx=0x7fdb580d1d10 comp rx=0 tx=0).stop 2024-09-10T09:58:29.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:29.906+0000 7fdb6ca00640 1 -- 172.21.15.5:0/4028429496 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdb681a36e0 msgr2=0x7fdb681a3b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:29.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:29.906+0000 7fdb6ca00640 1 --2- 172.21.15.5:0/4028429496 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdb681a36e0 0x7fdb681a3b90 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fdb5800b390 tx=0x7fdb5800b850 comp rx=0 tx=0).stop 2024-09-10T09:58:29.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:29.906+0000 7fdb6ca00640 1 -- 172.21.15.5:0/4028429496 shutdown_connections 2024-09-10T09:58:29.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:29.906+0000 7fdb6ca00640 1 --2- 172.21.15.5:0/4028429496 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fdb4c0a2cc0 0x7fdb4c0a5180 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:29.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:29.906+0000 7fdb6ca00640 1 --2- 172.21.15.5:0/4028429496 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdb681a36e0 0x7fdb681a3b90 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:29.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:29.906+0000 7fdb6ca00640 1 --2- 172.21.15.5:0/4028429496 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdb68112a30 0x7fdb6819f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:29.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:29.906+0000 7fdb6ca00640 1 --2- 172.21.15.5:0/4028429496 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdb68076720 0x7fdb6819ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:29.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:29.906+0000 7fdb6ca00640 1 -- 172.21.15.5:0/4028429496 >> 172.21.15.5:0/4028429496 conn(0x7fdb680ffa90 msgr2=0x7fdb681140e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:58:29.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:29.906+0000 7fdb6ca00640 1 -- 172.21.15.5:0/4028429496 shutdown_connections 2024-09-10T09:58:29.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:29.912+0000 7fdb6ca00640 1 -- 172.21.15.5:0/4028429496 wait complete. 2024-09-10T09:58:30.022 INFO:tasks.workunit.client.0.smithi005.stdout: 10 107580 14.91 MB/sec execute 248 sec latency 18321.967 ms 2024-09-10T09:58:30.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:58:29 smithi159 ceph-mon[35513]: pgmap v997: 129 pgs: 129 active+clean; 732 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 232 KiB/s rd, 8.8 MiB/s wr, 254 op/s 2024-09-10T09:58:30.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:58:29 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:58:30.101 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:58:29 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:58:30.101 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:58:29 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:58:30.101 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:58:29 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:58:30.101 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:58:29 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:58:30.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:58:29 smithi102 ceph-mon[35442]: pgmap v997: 129 pgs: 129 active+clean; 732 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 232 KiB/s rd, 8.8 MiB/s wr, 254 op/s 2024-09-10T09:58:30.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:58:29 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:58:30.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:58:29 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:58:30.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:58:29 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:58:30.135 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:58:29 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:58:30.135 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:58:29 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:58:30.991 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T09:58:31.001 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:58:30 smithi005 ceph-mon[30350]: scrub queued for path: / 2024-09-10T09:58:31.001 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:58:30 smithi005 ceph-mon[30350]: scrub summary: idle+waiting paths [/] 2024-09-10T09:58:31.001 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:58:30 smithi005 ceph-mon[30350]: scrub summary: active paths [/] 2024-09-10T09:58:31.001 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:58:30 smithi005 ceph-mon[30350]: scrub summary: active paths [/] 2024-09-10T09:58:31.001 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:58:30 smithi005 ceph-mon[30350]: scrub summary: active paths [/] 2024-09-10T09:58:31.001 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:58:30 smithi005 ceph-mon[30350]: scrub summary: active paths [/] 2024-09-10T09:58:31.001 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:58:30 smithi005 ceph-mon[30350]: scrub summary: active paths [/] 2024-09-10T09:58:31.001 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:58:30 smithi005 ceph-mon[30350]: scrub summary: idle+waiting paths [/] 2024-09-10T09:58:31.001 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:58:30 smithi005 ceph-mon[30350]: scrub summary: idle+waiting paths [/] 2024-09-10T09:58:31.001 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:58:30 smithi005 ceph-mon[30350]: scrub summary: idle+waiting paths [/] 2024-09-10T09:58:31.001 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:58:30 smithi005 ceph-mon[30350]: scrub summary: active paths [/] 2024-09-10T09:58:31.001 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:58:30 smithi005 ceph-mon[30350]: scrub summary: idle+waiting paths [/] 2024-09-10T09:58:31.001 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:58:30 smithi005 ceph-mon[30350]: scrub summary: idle+waiting paths [/] 2024-09-10T09:58:31.022 INFO:tasks.workunit.client.0.smithi005.stdout: 10 107580 14.85 MB/sec execute 249 sec latency 19322.027 ms 2024-09-10T09:58:31.035 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:58:30 smithi102 ceph-mon[35442]: scrub queued for path: / 2024-09-10T09:58:31.035 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:58:30 smithi102 ceph-mon[35442]: scrub summary: idle+waiting paths [/] 2024-09-10T09:58:31.035 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:58:30 smithi102 ceph-mon[35442]: scrub summary: active paths [/] 2024-09-10T09:58:31.035 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:58:30 smithi102 ceph-mon[35442]: scrub summary: active paths [/] 2024-09-10T09:58:31.036 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:58:30 smithi102 ceph-mon[35442]: scrub summary: active paths [/] 2024-09-10T09:58:31.036 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:58:30 smithi102 ceph-mon[35442]: scrub summary: active paths [/] 2024-09-10T09:58:31.036 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:58:30 smithi102 ceph-mon[35442]: scrub summary: active paths [/] 2024-09-10T09:58:31.036 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:58:30 smithi102 ceph-mon[35442]: scrub summary: idle+waiting paths [/] 2024-09-10T09:58:31.036 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:58:30 smithi102 ceph-mon[35442]: scrub summary: idle+waiting paths [/] 2024-09-10T09:58:31.036 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:58:30 smithi102 ceph-mon[35442]: scrub summary: idle+waiting paths [/] 2024-09-10T09:58:31.036 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:58:30 smithi102 ceph-mon[35442]: scrub summary: active paths [/] 2024-09-10T09:58:31.036 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:58:30 smithi102 ceph-mon[35442]: scrub summary: idle+waiting paths [/] 2024-09-10T09:58:31.036 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:58:30 smithi102 ceph-mon[35442]: scrub summary: idle+waiting paths [/] 2024-09-10T09:58:31.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:58:30 smithi159 ceph-mon[35513]: scrub queued for path: / 2024-09-10T09:58:31.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:58:30 smithi159 ceph-mon[35513]: scrub summary: idle+waiting paths [/] 2024-09-10T09:58:31.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:58:30 smithi159 ceph-mon[35513]: scrub summary: active paths [/] 2024-09-10T09:58:31.101 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:58:30 smithi159 ceph-mon[35513]: scrub summary: active paths [/] 2024-09-10T09:58:31.101 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:58:30 smithi159 ceph-mon[35513]: scrub summary: active paths [/] 2024-09-10T09:58:31.101 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:58:30 smithi159 ceph-mon[35513]: scrub summary: active paths [/] 2024-09-10T09:58:31.101 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:58:30 smithi159 ceph-mon[35513]: scrub summary: active paths [/] 2024-09-10T09:58:31.101 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:58:30 smithi159 ceph-mon[35513]: scrub summary: idle+waiting paths [/] 2024-09-10T09:58:31.101 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:58:30 smithi159 ceph-mon[35513]: scrub summary: idle+waiting paths [/] 2024-09-10T09:58:31.101 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:58:30 smithi159 ceph-mon[35513]: scrub summary: idle+waiting paths [/] 2024-09-10T09:58:31.101 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:58:30 smithi159 ceph-mon[35513]: scrub summary: active paths [/] 2024-09-10T09:58:31.101 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:58:30 smithi159 ceph-mon[35513]: scrub summary: idle+waiting paths [/] 2024-09-10T09:58:31.101 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:58:30 smithi159 ceph-mon[35513]: scrub summary: idle+waiting paths [/] 2024-09-10T09:58:31.381 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:58:31.830 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:58:31 smithi005 ceph-mon[30350]: pgmap v998: 129 pgs: 129 active+clean; 750 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 326 KiB/s rd, 9.5 MiB/s wr, 260 op/s 2024-09-10T09:58:32.022 INFO:tasks.workunit.client.0.smithi005.stdout: 10 107580 14.79 MB/sec execute 250 sec latency 20322.087 ms 2024-09-10T09:58:32.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:32.050+0000 7fc06a600640 1 -- 172.21.15.5:0/3582434741 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc064102360 msgr2=0x7fc06410e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:32.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:32.050+0000 7fc06a600640 1 --2- 172.21.15.5:0/3582434741 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc064102360 0x7fc06410e890 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7fc05c009cd0 tx=0x7fc05c02a910 comp rx=0 tx=0).stop 2024-09-10T09:58:32.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:32.052+0000 7fc06a600640 1 -- 172.21.15.5:0/3582434741 shutdown_connections 2024-09-10T09:58:32.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:32.052+0000 7fc06a600640 1 --2- 172.21.15.5:0/3582434741 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc064102360 0x7fc06410e890 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:32.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:32.052+0000 7fc06a600640 1 --2- 172.21.15.5:0/3582434741 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc0641019c0 0x7fc064101e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:32.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:32.052+0000 7fc06a600640 1 --2- 172.21.15.5:0/3582434741 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc0641079c0 0x7fc064107da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:32.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:32.052+0000 7fc06a600640 1 -- 172.21.15.5:0/3582434741 >> 172.21.15.5:0/3582434741 conn(0x7fc0640fd740 msgr2=0x7fc0640ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:58:32.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:32.052+0000 7fc06a600640 1 -- 172.21.15.5:0/3582434741 shutdown_connections 2024-09-10T09:58:32.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:32.052+0000 7fc06a600640 1 -- 172.21.15.5:0/3582434741 wait complete. 2024-09-10T09:58:32.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:32.052+0000 7fc06a600640 1 Processor -- start 2024-09-10T09:58:32.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:32.052+0000 7fc06a600640 1 -- start start 2024-09-10T09:58:32.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:32.052+0000 7fc06a600640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc0641019c0 0x7fc06419a510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:32.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:32.052+0000 7fc06a600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc064102360 0x7fc06419aa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:32.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:32.052+0000 7fc06a600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc0641079c0 0x7fc06419ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:32.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:32.052+0000 7fc06a600640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc064111e70 con 0x7fc064102360 2024-09-10T09:58:32.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:32.052+0000 7fc06a600640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fc064111cf0 con 0x7fc0641079c0 2024-09-10T09:58:32.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:32.052+0000 7fc06a600640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fc064111ff0 con 0x7fc0641019c0 2024-09-10T09:58:32.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:32.052+0000 7fc063e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc064102360 0x7fc06419aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:32.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:32.052+0000 7fc069200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc0641019c0 0x7fc06419a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:32.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:32.052+0000 7fc063e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc064102360 0x7fc06419aa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54958/0 (socket says 172.21.15.5:54958) 2024-09-10T09:58:32.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:32.052+0000 7fc069200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc0641019c0 0x7fc06419a510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:39664/0 (socket says 172.21.15.5:39664) 2024-09-10T09:58:32.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:32.052+0000 7fc063e00640 1 -- 172.21.15.5:0/868285339 learned_addr learned my addr 172.21.15.5:0/868285339 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:58:32.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:32.052+0000 7fc069c00640 1 --2- 172.21.15.5:0/868285339 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc0641079c0 0x7fc06419ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:32.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:32.052+0000 7fc069200640 1 -- 172.21.15.5:0/868285339 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc0641079c0 msgr2=0x7fc06419ede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:32.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:32.052+0000 7fc069200640 1 --2- 172.21.15.5:0/868285339 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc0641079c0 0x7fc06419ede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:32.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:32.052+0000 7fc069200640 1 -- 172.21.15.5:0/868285339 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc064102360 msgr2=0x7fc06419aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:32.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:32.052+0000 7fc069200640 1 --2- 172.21.15.5:0/868285339 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc064102360 0x7fc06419aa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:32.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:32.052+0000 7fc069200640 1 -- 172.21.15.5:0/868285339 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc06419f4c0 con 0x7fc0641019c0 2024-09-10T09:58:32.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:32.052+0000 7fc069c00640 1 --2- 172.21.15.5:0/868285339 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc0641079c0 0x7fc06419ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:58:32.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:32.052+0000 7fc069200640 1 --2- 172.21.15.5:0/868285339 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc0641019c0 0x7fc06419a510 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fc058009fd0 tx=0x7fc05800b870 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:32.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:32.052+0000 7fc061600640 1 -- 172.21.15.5:0/868285339 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc058015020 con 0x7fc0641019c0 2024-09-10T09:58:32.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:32.052+0000 7fc061600640 1 -- 172.21.15.5:0/868285339 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc058011d10 con 0x7fc0641019c0 2024-09-10T09:58:32.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:32.052+0000 7fc061600640 1 -- 172.21.15.5:0/868285339 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc05800f2e0 con 0x7fc0641019c0 2024-09-10T09:58:32.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:32.052+0000 7fc06a600640 1 -- 172.21.15.5:0/868285339 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc06419f7b0 con 0x7fc0641019c0 2024-09-10T09:58:32.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:32.052+0000 7fc06a600640 1 -- 172.21.15.5:0/868285339 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0641a7090 con 0x7fc0641019c0 2024-09-10T09:58:32.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:32.057+0000 7fc06e63f740 1 Processor -- start 2024-09-10T09:58:32.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:32.057+0000 7fc06e63f740 1 -- start start 2024-09-10T09:58:32.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:32.057+0000 7fc06e63f740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559cae1efd50 0x559cae1f0130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:32.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:32.057+0000 7fc06e63f740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559cae1f0670 0x559cae1fa9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:32.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:32.057+0000 7fc06e63f740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559cae1faf30 0x559cae1fd320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:32.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:32.057+0000 7fc06e63f740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x559cae1563e0 con 0x559cae1faf30 2024-09-10T09:58:32.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:32.057+0000 7fc06e63f740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x559cae0fdd80 con 0x559cae1efd50 2024-09-10T09:58:32.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:32.057+0000 7fc06e63f740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x559cae1523a0 con 0x559cae1f0670 2024-09-10T09:58:32.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:32.057+0000 7fc063e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559cae1faf30 0x559cae1fd320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:32.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:32.057+0000 7fc063e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559cae1faf30 0x559cae1fd320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54960/0 (socket says 172.21.15.5:54960) 2024-09-10T09:58:32.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:32.057+0000 7fc063e00640 1 -- 172.21.15.5:0/2212241089 learned_addr learned my addr 172.21.15.5:0/2212241089 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:58:32.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:32.057+0000 7fc069c00640 1 --2- 172.21.15.5:0/2212241089 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559cae1efd50 0x559cae1f0130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:32.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:32.057+0000 7fc061600640 1 -- 172.21.15.5:0/868285339 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7fc058018020 con 0x7fc0641019c0 2024-09-10T09:58:32.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:32.057+0000 7fc069200640 1 --2- 172.21.15.5:0/2212241089 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559cae1f0670 0x559cae1fa9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:32.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:32.057+0000 7fc063e00640 1 -- 172.21.15.5:0/2212241089 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559cae1f0670 msgr2=0x559cae1fa9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:58:32.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:32.057+0000 7fc063e00640 1 --2- 172.21.15.5:0/2212241089 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559cae1f0670 0x559cae1fa9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:32.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:32.057+0000 7fc063e00640 1 -- 172.21.15.5:0/2212241089 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559cae1efd50 msgr2=0x559cae1f0130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:58:32.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:32.057+0000 7fc063e00640 1 --2- 172.21.15.5:0/2212241089 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559cae1efd50 0x559cae1f0130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:32.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:32.057+0000 7fc063e00640 1 -- 172.21.15.5:0/2212241089 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559cae11cae0 con 0x559cae1faf30 2024-09-10T09:58:32.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:32.057+0000 7fc063e00640 1 --2- 172.21.15.5:0/2212241089 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559cae1faf30 0x559cae1fd320 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7fc050009d80 tx=0x7fc050011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcb0e4c15034974c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:32.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:32.057+0000 7fc056000640 1 -- 172.21.15.5:0/2212241089 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc050005220 con 0x559cae1faf30 2024-09-10T09:58:32.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:32.057+0000 7fc056000640 1 -- 172.21.15.5:0/2212241089 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc050004c80 con 0x559cae1faf30 2024-09-10T09:58:32.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:32.057+0000 7fc056000640 1 -- 172.21.15.5:0/2212241089 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0500199a0 con 0x559cae1faf30 2024-09-10T09:58:32.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:32.057+0000 7fc061600640 1 --2- 172.21.15.5:0/868285339 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fc04c0a2c00 0x7fc04c0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:32.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:32.062+0000 7fc06e63f740 1 -- 172.21.15.5:0/2212241089 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559cae1faf30 msgr2=0x559cae1fd320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:32.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:32.062+0000 7fc06e63f740 1 --2- 172.21.15.5:0/2212241089 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559cae1faf30 0x559cae1fd320 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7fc050009d80 tx=0x7fc050011810 comp rx=0 tx=0).stop 2024-09-10T09:58:32.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:32.062+0000 7fc061600640 1 -- 172.21.15.5:0/868285339 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fc0580ced10 con 0x7fc0641019c0 2024-09-10T09:58:32.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:32.062+0000 7fc069c00640 1 --2- 172.21.15.5:0/868285339 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fc04c0a2c00 0x7fc04c0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:32.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:32.062+0000 7fc06e63f740 1 -- 172.21.15.5:0/2212241089 shutdown_connections 2024-09-10T09:58:32.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:32.062+0000 7fc06e63f740 1 --2- 172.21.15.5:0/2212241089 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559cae1faf30 0x559cae1fd320 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:32.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:32.062+0000 7fc06e63f740 1 --2- 172.21.15.5:0/2212241089 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559cae1f0670 0x559cae1fa9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:32.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:32.062+0000 7fc06e63f740 1 --2- 172.21.15.5:0/2212241089 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559cae1efd50 0x559cae1f0130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:32.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:32.062+0000 7fc06e63f740 1 -- 172.21.15.5:0/2212241089 >> 172.21.15.5:0/2212241089 conn(0x559cae1ec2e0 msgr2=0x559cae1ec6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:58:32.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:32.062+0000 7fc06e63f740 1 -- 172.21.15.5:0/2212241089 shutdown_connections 2024-09-10T09:58:32.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:32.062+0000 7fc06e63f740 1 -- 172.21.15.5:0/2212241089 wait complete. 2024-09-10T09:58:32.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:32.062+0000 7fc069c00640 1 --2- 172.21.15.5:0/868285339 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fc04c0a2c00 0x7fc04c0a50c0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fc05c006010 tx=0x7fc05c005eb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:32.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:32.062+0000 7fc06e63f740 1 -- start start 2024-09-10T09:58:32.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:32.062+0000 7fc06e63f740 1 Processor -- start 2024-09-10T09:58:32.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:32.062+0000 7fc06e63f740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559cae1efd50 0x559cae1f7d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:32.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:32.062+0000 7fc06e63f740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559cae1f0670 0x559cae1f82a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:32.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:32.062+0000 7fc06e63f740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559cae1faf30 0x559cae1f87e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:32.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:32.062+0000 7fc06e63f740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x559cae1fdba0 con 0x559cae1faf30 2024-09-10T09:58:32.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:32.062+0000 7fc06e63f740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x559cae1110f0 con 0x559cae1f0670 2024-09-10T09:58:32.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:32.062+0000 7fc06e63f740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x559cae1fdd20 con 0x559cae1efd50 2024-09-10T09:58:32.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:32.062+0000 7fc06e63f740 10 client.0 _ll_register_callbacks cb 0x559cae1029e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:58:32.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:32.062+0000 7fc069c00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559cae1faf30 0x559cae1f87e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:32.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:32.062+0000 7fc063e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559cae1f0670 0x559cae1f82a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:32.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:32.062+0000 7fc069200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559cae1efd50 0x559cae1f7d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:32.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:32.062+0000 7fc069c00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559cae1faf30 0x559cae1f87e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54962/0 (socket says 172.21.15.5:54962) 2024-09-10T09:58:32.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:32.062+0000 7fc069c00640 1 -- 172.21.15.5:0/3302183302 learned_addr learned my addr 172.21.15.5:0/3302183302 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:58:32.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:32.062+0000 7fc063e00640 1 -- 172.21.15.5:0/3302183302 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559cae1efd50 msgr2=0x559cae1f7d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:58:32.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:32.062+0000 7fc063e00640 1 --2- 172.21.15.5:0/3302183302 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559cae1efd50 0x559cae1f7d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:32.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:32.062+0000 7fc063e00640 1 -- 172.21.15.5:0/3302183302 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559cae1faf30 msgr2=0x559cae1f87e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:58:32.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:32.062+0000 7fc063e00640 1 --2- 172.21.15.5:0/3302183302 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559cae1faf30 0x559cae1f87e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:32.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:32.062+0000 7fc063e00640 1 -- 172.21.15.5:0/3302183302 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559cadec73c0 con 0x559cae1f0670 2024-09-10T09:58:32.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:32.062+0000 7fc069c00640 1 --2- 172.21.15.5:0/3302183302 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559cae1faf30 0x559cae1f87e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:58:32.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:32.062+0000 7fc063e00640 1 --2- 172.21.15.5:0/3302183302 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559cae1f0670 0x559cae1f82a0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7fc050011d10 tx=0x7fc050019450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c7b9286f7b3a1c16 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:32.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:32.062+0000 7fc04aa00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-09-10T09:58:32.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:32.062+0000 7fc04aa00640 1 -- 172.21.15.5:0/3302183302 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc050029070 con 0x559cae1f0670 2024-09-10T09:58:32.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:32.062+0000 7fc06e63f740 1 -- 172.21.15.5:0/3302183302 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc050004c80 con 0x559cae1f0670 2024-09-10T09:58:32.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:32.062+0000 7fc04aa00640 1 -- 172.21.15.5:0/3302183302 <== mon.2 v2:172.21.15.102:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fc0500196e0 con 0x559cae1f0670 2024-09-10T09:58:32.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:32.062+0000 7fc04aa00640 1 -- 172.21.15.5:0/3302183302 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7fc050035050 con 0x559cae1f0670 2024-09-10T09:58:32.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:32.062+0000 7fc06e63f740 10 client.29357 fetch_fsmap learned FSMap version 42 2024-09-10T09:58:32.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:32.062+0000 7fc06e63f740 1 -- 172.21.15.5:0/3302183302 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x559cae2097a0 con 0x559cae1f0670 2024-09-10T09:58:32.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:32.062+0000 7fc04aa00640 1 -- 172.21.15.5:0/3302183302 <== mon.2 v2:172.21.15.102:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc050004c80 con 0x559cae1f0670 2024-09-10T09:58:32.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:32.062+0000 7fc04aa00640 1 -- 172.21.15.5:0/3302183302 <== mon.2 v2:172.21.15.102:3300/0 5 ==== fsmap(e 42) ==== 8370+0+0 (secure 0 0 0) 0x7fc050023880 con 0x559cae1f0670 2024-09-10T09:58:32.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:32.062+0000 7fc06e63f740 10 client.29357 fetch_fsmap finished waiting for FSMap version 42 2024-09-10T09:58:32.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:32.062+0000 7fc06e63f740 10 client.29357 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T09:58:32.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:32.062+0000 7fc06e63f740 20 client.29357 populate_metadata read hostname 'smithi005' 2024-09-10T09:58:32.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:32.062+0000 7fc06e63f740 1 --2- 172.21.15.5:0/3302183302 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x559cae20ab50 0x559cae20cf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:32.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:32.062+0000 7fc06e63f740 4 client.29357 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:58:32.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:32.062+0000 7fc06e63f740 1 -- 172.21.15.5:0/3302183302 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559cae163950 con 0x559cae20ab50 2024-09-10T09:58:32.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:32.082+0000 7fc069200640 1 --2- 172.21.15.5:0/3302183302 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x559cae20ab50 0x559cae20cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:32.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:58:31 smithi159 ceph-mon[35513]: pgmap v998: 129 pgs: 129 active+clean; 750 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 326 KiB/s rd, 9.5 MiB/s wr, 260 op/s 2024-09-10T09:58:32.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:58:31 smithi102 ceph-mon[35442]: pgmap v998: 129 pgs: 129 active+clean; 750 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 326 KiB/s rd, 9.5 MiB/s wr, 260 op/s 2024-09-10T09:58:32.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:32.141+0000 7fc069200640 1 --2- 172.21.15.5:0/3302183302 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x559cae20ab50 0x559cae20cf40 crc :-1 s=READY pgs=285 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bd3ec6b6d3d7e535 server_cookie=11c874e7b7c4954f in_seq=0 out_seq=0 2024-09-10T09:58:32.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:32.141+0000 7fc04aa00640 10 client.29357 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T09:58:32.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:32.164+0000 7fc04aa00640 1 -- 172.21.15.5:0/3302183302 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559cae163950 con 0x559cae20ab50 2024-09-10T09:58:32.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:32.164+0000 7fc04aa00640 10 client.29357 handle_command_reply: tid=0 2024-09-10T09:58:32.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:32.164+0000 7fc06e63f740 1 client.29357 shutdown 2024-09-10T09:58:32.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:32.164+0000 7fc06e63f740 1 -- 172.21.15.5:0/3302183302 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559cae1f0670 msgr2=0x559cae1f82a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:32.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:32.164+0000 7fc06e63f740 1 --2- 172.21.15.5:0/3302183302 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559cae1f0670 0x559cae1f82a0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7fc050011d10 tx=0x7fc050019450 comp rx=0 tx=0).stop 2024-09-10T09:58:32.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:32.164+0000 7fc06e63f740 1 -- 172.21.15.5:0/3302183302 shutdown_connections 2024-09-10T09:58:32.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:32.164+0000 7fc06e63f740 1 --2- 172.21.15.5:0/3302183302 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x559cae20ab50 0x559cae20cf40 crc :-1 s=READY pgs=285 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:32.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:32.165+0000 7fc06e63f740 1 --2- 172.21.15.5:0/3302183302 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559cae1faf30 0x559cae1f87e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:32.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:32.165+0000 7fc06e63f740 1 --2- 172.21.15.5:0/3302183302 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559cae1f0670 0x559cae1f82a0 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:32.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:32.165+0000 7fc06e63f740 1 --2- 172.21.15.5:0/3302183302 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559cae1efd50 0x559cae1f7d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:32.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:32.165+0000 7fc04aa00640 0 client.29357 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T09:58:32.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:32.165+0000 7fc06e63f740 1 -- 172.21.15.5:0/3302183302 >> 172.21.15.5:0/3302183302 conn(0x559cae1ec2e0 msgr2=0x559cae1f1fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:58:32.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:32.165+0000 7fc06e63f740 1 -- 172.21.15.5:0/3302183302 shutdown_connections 2024-09-10T09:58:32.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:32.165+0000 7fc06e63f740 1 -- 172.21.15.5:0/3302183302 wait complete. 2024-09-10T09:58:32.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:32.165+0000 7fc06e63f740 20 client.29357 trim_cache size 0 max 16384 2024-09-10T09:58:32.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:32.181+0000 7fc06e63f740 1 Processor -- start 2024-09-10T09:58:32.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:32.181+0000 7fc06e63f740 1 -- start start 2024-09-10T09:58:32.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:32.181+0000 7fc06e63f740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559cae1efd50 0x559cae1f6fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:32.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:32.181+0000 7fc06e63f740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559cae1f0670 0x559cae212720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:32.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:32.181+0000 7fc06e63f740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559cae1faf30 0x559cae212c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:32.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:32.181+0000 7fc06e63f740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc0641b0f70 con 0x559cae1faf30 2024-09-10T09:58:32.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:32.181+0000 7fc06e63f740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fc0641b2ab0 con 0x559cae1f0670 2024-09-10T09:58:32.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:32.181+0000 7fc06e63f740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fc0641b1640 con 0x559cae1efd50 2024-09-10T09:58:32.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:32.181+0000 7fc069c00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559cae1faf30 0x559cae212c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:32.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:32.181+0000 7fc069c00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559cae1faf30 0x559cae212c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54968/0 (socket says 172.21.15.5:54968) 2024-09-10T09:58:32.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:32.181+0000 7fc069c00640 1 -- 172.21.15.5:0/593337281 learned_addr learned my addr 172.21.15.5:0/593337281 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:58:32.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:32.181+0000 7fc069200640 1 --2- 172.21.15.5:0/593337281 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559cae1efd50 0x559cae1f6fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:32.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:32.181+0000 7fc063e00640 1 --2- 172.21.15.5:0/593337281 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559cae1f0670 0x559cae212720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:32.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:32.181+0000 7fc069200640 1 -- 172.21.15.5:0/593337281 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559cae1f0670 msgr2=0x559cae212720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:58:32.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:32.181+0000 7fc069200640 1 --2- 172.21.15.5:0/593337281 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559cae1f0670 0x559cae212720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:32.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:32.181+0000 7fc069200640 1 -- 172.21.15.5:0/593337281 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559cae1faf30 msgr2=0x559cae212c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:58:32.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:32.181+0000 7fc069200640 1 --2- 172.21.15.5:0/593337281 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559cae1faf30 0x559cae212c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:32.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:32.181+0000 7fc069200640 1 -- 172.21.15.5:0/593337281 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559cae20d4d0 con 0x559cae1efd50 2024-09-10T09:58:32.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:32.181+0000 7fc069c00640 1 --2- 172.21.15.5:0/593337281 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559cae1faf30 0x559cae212c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:58:32.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:32.188+0000 7fc069200640 1 --2- 172.21.15.5:0/593337281 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559cae1efd50 0x559cae1f6fe0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x559cae1ed140 tx=0x7fc058010790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=54185df64721a99d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:32.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:32.188+0000 7fc055600640 1 -- 172.21.15.5:0/593337281 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc05809b820 con 0x559cae1efd50 2024-09-10T09:58:32.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:32.188+0000 7fc055600640 1 -- 172.21.15.5:0/593337281 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc05809b120 con 0x559cae1efd50 2024-09-10T09:58:32.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:32.188+0000 7fc055600640 1 -- 172.21.15.5:0/593337281 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0580bb0b0 con 0x559cae1efd50 2024-09-10T09:58:32.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:32.188+0000 7fc06e63f740 1 -- 172.21.15.5:0/593337281 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559cae1efd50 msgr2=0x559cae1f6fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:32.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:32.188+0000 7fc06e63f740 1 --2- 172.21.15.5:0/593337281 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559cae1efd50 0x559cae1f6fe0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x559cae1ed140 tx=0x7fc058010790 comp rx=0 tx=0).stop 2024-09-10T09:58:32.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:32.188+0000 7fc06e63f740 1 -- 172.21.15.5:0/593337281 shutdown_connections 2024-09-10T09:58:32.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:32.188+0000 7fc06e63f740 1 --2- 172.21.15.5:0/593337281 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559cae1faf30 0x559cae212c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:32.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:32.188+0000 7fc06e63f740 1 --2- 172.21.15.5:0/593337281 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559cae1f0670 0x559cae212720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:32.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:32.188+0000 7fc06e63f740 1 --2- 172.21.15.5:0/593337281 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559cae1efd50 0x559cae1f6fe0 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:32.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:32.188+0000 7fc06e63f740 1 -- 172.21.15.5:0/593337281 >> 172.21.15.5:0/593337281 conn(0x559cae1ec2e0 msgr2=0x559cae1f9de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:58:32.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:32.188+0000 7fc06e63f740 1 -- 172.21.15.5:0/593337281 shutdown_connections 2024-09-10T09:58:32.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:32.188+0000 7fc06e63f740 1 -- 172.21.15.5:0/593337281 wait complete. 2024-09-10T09:58:32.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:32.188+0000 7fc06e63f740 1 -- start start 2024-09-10T09:58:32.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:32.188+0000 7fc06e63f740 1 Processor -- start 2024-09-10T09:58:32.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:32.188+0000 7fc06e63f740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559cae1efd50 0x559cae203150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:32.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:32.188+0000 7fc06e63f740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559cae1f0670 0x559cae203690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:32.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:32.188+0000 7fc06e63f740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559cae1faf30 0x559cae1f45a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:32.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:32.188+0000 7fc06e63f740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc0641b23e0 con 0x559cae1f0670 2024-09-10T09:58:32.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:32.188+0000 7fc06e63f740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fc0641b1d10 con 0x559cae1efd50 2024-09-10T09:58:32.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:32.188+0000 7fc06e63f740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fc0641aef50 con 0x559cae1faf30 2024-09-10T09:58:32.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:32.188+0000 7fc06e63f740 10 client.0 _ll_register_callbacks cb 0x7fc0500053e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:58:32.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:32.188+0000 7fc063e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559cae1f0670 0x559cae203690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:32.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:32.188+0000 7fc069200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559cae1efd50 0x559cae203150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:32.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:32.188+0000 7fc069c00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559cae1faf30 0x559cae1f45a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:32.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:32.188+0000 7fc063e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559cae1f0670 0x559cae203690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54980/0 (socket says 172.21.15.5:54980) 2024-09-10T09:58:32.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:32.188+0000 7fc063e00640 1 -- 172.21.15.5:0/1836988733 learned_addr learned my addr 172.21.15.5:0/1836988733 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:58:32.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:32.188+0000 7fc063e00640 1 -- 172.21.15.5:0/1836988733 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559cae1faf30 msgr2=0x559cae1f45a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:58:32.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:32.188+0000 7fc063e00640 1 --2- 172.21.15.5:0/1836988733 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559cae1faf30 0x559cae1f45a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:32.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:32.188+0000 7fc063e00640 1 -- 172.21.15.5:0/1836988733 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559cae1efd50 msgr2=0x559cae203150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:58:32.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:32.188+0000 7fc063e00640 1 --2- 172.21.15.5:0/1836988733 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559cae1efd50 0x559cae203150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:32.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:32.188+0000 7fc063e00640 1 -- 172.21.15.5:0/1836988733 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559cae1f4c90 con 0x559cae1f0670 2024-09-10T09:58:32.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:32.188+0000 7fc069200640 1 --2- 172.21.15.5:0/1836988733 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559cae1efd50 0x559cae203150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:58:32.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:32.188+0000 7fc063e00640 1 --2- 172.21.15.5:0/1836988733 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559cae1f0670 0x559cae203690 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7fc050023e40 tx=0x7fc050005490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1524546d33b4c77e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:32.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:32.188+0000 7fc04aa00640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-10T09:58:32.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:32.188+0000 7fc04aa00640 1 -- 172.21.15.5:0/1836988733 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc050032070 con 0x559cae1f0670 2024-09-10T09:58:32.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:32.188+0000 7fc04aa00640 1 -- 172.21.15.5:0/1836988733 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fc050004ce0 con 0x559cae1f0670 2024-09-10T09:58:32.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:32.188+0000 7fc06e63f740 1 -- 172.21.15.5:0/1836988733 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc05809b120 con 0x559cae1f0670 2024-09-10T09:58:32.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:32.188+0000 7fc04aa00640 1 -- 172.21.15.5:0/1836988733 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7fc050029050 con 0x559cae1f0670 2024-09-10T09:58:32.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:32.188+0000 7fc06e63f740 10 client.19302 fetch_fsmap learned FSMap version 42 2024-09-10T09:58:32.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:32.188+0000 7fc06e63f740 1 -- 172.21.15.5:0/1836988733 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x559cae2750e0 con 0x559cae1f0670 2024-09-10T09:58:32.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:32.188+0000 7fc04aa00640 1 -- 172.21.15.5:0/1836988733 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc05809b120 con 0x559cae1f0670 2024-09-10T09:58:32.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:32.188+0000 7fc04aa00640 1 -- 172.21.15.5:0/1836988733 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 42) ==== 8370+0+0 (secure 0 0 0) 0x7fc0500198e0 con 0x559cae1f0670 2024-09-10T09:58:32.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:32.188+0000 7fc06e63f740 10 client.19302 fetch_fsmap finished waiting for FSMap version 42 2024-09-10T09:58:32.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:32.188+0000 7fc06e63f740 10 client.19302 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T09:58:32.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:32.188+0000 7fc06e63f740 20 client.19302 populate_metadata read hostname 'smithi005' 2024-09-10T09:58:32.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:32.188+0000 7fc06e63f740 1 --2- 172.21.15.5:0/1836988733 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x559cae2757c0 0x559cae277be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:32.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:32.188+0000 7fc06e63f740 4 client.19302 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T09:58:32.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:32.188+0000 7fc06e63f740 1 -- 172.21.15.5:0/1836988733 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "scrub status"}) -- 0x559cae15de60 con 0x559cae2757c0 2024-09-10T09:58:32.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:32.188+0000 7fc069200640 1 --2- 172.21.15.5:0/1836988733 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x559cae2757c0 0x559cae277be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:32.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:32.194+0000 7fc04aa00640 10 client.19302 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T09:58:32.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:32.194+0000 7fc069200640 1 --2- 172.21.15.5:0/1836988733 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x559cae2757c0 0x559cae277be0 crc :-1 s=READY pgs=286 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=274bdc92adcd6574 server_cookie=a0bed4eb9e20c0a6 in_seq=0 out_seq=0 2024-09-10T09:58:32.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:32.200+0000 7fc04aa00640 1 -- 172.21.15.5:0/1836988733 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x559cae15de60 con 0x559cae2757c0 2024-09-10T09:58:32.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:32.200+0000 7fc04aa00640 10 client.19302 handle_command_reply: tid=0 2024-09-10T09:58:32.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:32.200+0000 7fc06e63f740 1 client.19302 shutdown 2024-09-10T09:58:32.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:32.200+0000 7fc06e63f740 1 -- 172.21.15.5:0/1836988733 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559cae1f0670 msgr2=0x559cae203690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:32.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:32.200+0000 7fc06e63f740 1 --2- 172.21.15.5:0/1836988733 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559cae1f0670 0x559cae203690 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7fc050023e40 tx=0x7fc050005490 comp rx=0 tx=0).stop 2024-09-10T09:58:32.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:32.200+0000 7fc06e63f740 1 -- 172.21.15.5:0/1836988733 shutdown_connections 2024-09-10T09:58:32.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:32.200+0000 7fc06e63f740 1 --2- 172.21.15.5:0/1836988733 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x559cae2757c0 0x559cae277be0 crc :-1 s=READY pgs=286 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:32.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:32.200+0000 7fc06e63f740 1 --2- 172.21.15.5:0/1836988733 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559cae1faf30 0x559cae1f45a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:32.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:32.200+0000 7fc06e63f740 1 --2- 172.21.15.5:0/1836988733 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559cae1f0670 0x559cae203690 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:32.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:32.200+0000 7fc06e63f740 1 --2- 172.21.15.5:0/1836988733 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559cae1efd50 0x559cae203150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:32.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:32.200+0000 7fc06e63f740 1 -- 172.21.15.5:0/1836988733 >> 172.21.15.5:0/1836988733 conn(0x559cae1ec2e0 msgr2=0x559cae1f17a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:58:32.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:32.200+0000 7fc04aa00640 0 client.19302 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T09:58:32.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:32.200+0000 7fc06e63f740 1 -- 172.21.15.5:0/1836988733 shutdown_connections 2024-09-10T09:58:32.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:32.200+0000 7fc06e63f740 1 -- 172.21.15.5:0/1836988733 wait complete. 2024-09-10T09:58:32.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:32.200+0000 7fc06e63f740 20 client.19302 trim_cache size 0 max 16384 2024-09-10T09:58:32.204 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-10T09:58:32.204 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-10T09:58:32.204 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2024-09-10T09:58:32.204 INFO:teuthology.orchestra.run.smithi005.stdout: "305147c2-efd3-48fb-9a5d-0630485285b8": { 2024-09-10T09:58:32.204 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2024-09-10T09:58:32.204 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "305147c2-efd3-48fb-9a5d-0630485285b8", 2024-09-10T09:58:32.204 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2024-09-10T09:58:32.204 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T09:58:32.204 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T09:58:32.204 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-10T09:58:32.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:32.200+0000 7fc06a600640 1 -- 172.21.15.5:0/868285339 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fc04c0a2c00 msgr2=0x7fc04c0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:32.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:32.200+0000 7fc06a600640 1 --2- 172.21.15.5:0/868285339 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fc04c0a2c00 0x7fc04c0a50c0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fc05c006010 tx=0x7fc05c005eb0 comp rx=0 tx=0).stop 2024-09-10T09:58:32.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:32.200+0000 7fc06a600640 1 -- 172.21.15.5:0/868285339 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc0641019c0 msgr2=0x7fc06419a510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:32.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:32.200+0000 7fc06a600640 1 --2- 172.21.15.5:0/868285339 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc0641019c0 0x7fc06419a510 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fc058009fd0 tx=0x7fc05800b870 comp rx=0 tx=0).stop 2024-09-10T09:58:32.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:32.200+0000 7fc06a600640 1 -- 172.21.15.5:0/868285339 shutdown_connections 2024-09-10T09:58:32.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:32.200+0000 7fc06a600640 1 --2- 172.21.15.5:0/868285339 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fc04c0a2c00 0x7fc04c0a50c0 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:32.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:32.200+0000 7fc06a600640 1 --2- 172.21.15.5:0/868285339 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc0641079c0 0x7fc06419ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:32.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:32.200+0000 7fc06a600640 1 --2- 172.21.15.5:0/868285339 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc064102360 0x7fc06419aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:32.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:32.200+0000 7fc06a600640 1 --2- 172.21.15.5:0/868285339 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc0641019c0 0x7fc06419a510 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:32.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:32.200+0000 7fc06a600640 1 -- 172.21.15.5:0/868285339 >> 172.21.15.5:0/868285339 conn(0x7fc0640fd740 msgr2=0x7fc06410d1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:58:32.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:32.200+0000 7fc06a600640 1 -- 172.21.15.5:0/868285339 shutdown_connections 2024-09-10T09:58:32.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:32.200+0000 7fc06a600640 1 -- 172.21.15.5:0/868285339 wait complete. 2024-09-10T09:58:33.022 INFO:tasks.workunit.client.0.smithi005.stdout: 10 107580 14.74 MB/sec execute 251 sec latency 21322.141 ms 2024-09-10T09:58:33.274 INFO:tasks.cephfs.filesystem:scrub status for tag:305147c2-efd3-48fb-9a5d-0630485285b8 - {'path': '/', 'tag': '305147c2-efd3-48fb-9a5d-0630485285b8', 'options': 'recursive,force'} 2024-09-10T09:58:34.022 INFO:tasks.workunit.client.0.smithi005.stdout: 10 107580 14.68 MB/sec execute 252 sec latency 22322.216 ms 2024-09-10T09:58:34.026 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:58:33 smithi005 ceph-mon[30350]: pgmap v999: 129 pgs: 129 active+clean; 756 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 326 KiB/s rd, 6.4 MiB/s wr, 187 op/s 2024-09-10T09:58:34.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:58:33 smithi159 ceph-mon[35513]: pgmap v999: 129 pgs: 129 active+clean; 756 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 326 KiB/s rd, 6.4 MiB/s wr, 187 op/s 2024-09-10T09:58:34.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:58:33 smithi102 ceph-mon[35442]: pgmap v999: 129 pgs: 129 active+clean; 756 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 326 KiB/s rd, 6.4 MiB/s wr, 187 op/s 2024-09-10T09:58:35.022 INFO:tasks.workunit.client.0.smithi005.stdout: 10 107580 14.62 MB/sec execute 253 sec latency 23322.282 ms 2024-09-10T09:58:36.020 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:58:35 smithi005 ceph-mon[30350]: pgmap v1000: 129 pgs: 129 active+clean; 743 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 297 KiB/s rd, 5.7 MiB/s wr, 159 op/s 2024-09-10T09:58:36.022 INFO:tasks.workunit.client.0.smithi005.stdout: 10 107580 14.56 MB/sec execute 254 sec latency 24322.347 ms 2024-09-10T09:58:36.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:58:35 smithi159 ceph-mon[35513]: pgmap v1000: 129 pgs: 129 active+clean; 743 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 297 KiB/s rd, 5.7 MiB/s wr, 159 op/s 2024-09-10T09:58:36.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:58:35 smithi102 ceph-mon[35442]: pgmap v1000: 129 pgs: 129 active+clean; 743 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 297 KiB/s rd, 5.7 MiB/s wr, 159 op/s 2024-09-10T09:58:37.022 INFO:tasks.workunit.client.0.smithi005.stdout: 10 107580 14.50 MB/sec execute 255 sec latency 25322.410 ms 2024-09-10T09:58:37.985 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:58:37 smithi005 ceph-mon[30350]: pgmap v1001: 129 pgs: 129 active+clean; 732 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 352 KiB/s rd, 5.2 MiB/s wr, 156 op/s 2024-09-10T09:58:38.023 INFO:tasks.workunit.client.0.smithi005.stdout: 10 107580 14.45 MB/sec execute 256 sec latency 26322.465 ms 2024-09-10T09:58:38.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:58:37 smithi159 ceph-mon[35513]: pgmap v1001: 129 pgs: 129 active+clean; 732 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 352 KiB/s rd, 5.2 MiB/s wr, 156 op/s 2024-09-10T09:58:38.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:58:37 smithi102 ceph-mon[35442]: pgmap v1001: 129 pgs: 129 active+clean; 732 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 352 KiB/s rd, 5.2 MiB/s wr, 156 op/s 2024-09-10T09:58:39.023 INFO:tasks.workunit.client.0.smithi005.stdout: 10 107580 14.39 MB/sec execute 257 sec latency 27322.513 ms 2024-09-10T09:58:39.992 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:58:39 smithi005 ceph-mon[30350]: pgmap v1002: 129 pgs: 129 active+clean; 727 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 224 KiB/s rd, 3.6 MiB/s wr, 120 op/s 2024-09-10T09:58:40.023 INFO:tasks.workunit.client.0.smithi005.stdout: 10 107580 14.34 MB/sec execute 258 sec latency 28322.558 ms 2024-09-10T09:58:40.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:58:39 smithi159 ceph-mon[35513]: pgmap v1002: 129 pgs: 129 active+clean; 727 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 224 KiB/s rd, 3.6 MiB/s wr, 120 op/s 2024-09-10T09:58:40.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:58:39 smithi102 ceph-mon[35442]: pgmap v1002: 129 pgs: 129 active+clean; 727 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 224 KiB/s rd, 3.6 MiB/s wr, 120 op/s 2024-09-10T09:58:41.023 INFO:tasks.workunit.client.0.smithi005.stdout: 10 107580 14.28 MB/sec execute 259 sec latency 29322.605 ms 2024-09-10T09:58:41.956 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:58:41 smithi005 ceph-mon[30350]: pgmap v1003: 129 pgs: 129 active+clean; 724 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 243 KiB/s rd, 3.3 MiB/s wr, 122 op/s 2024-09-10T09:58:42.023 INFO:tasks.workunit.client.0.smithi005.stdout: 10 107587 14.23 MB/sec execute 260 sec latency 30322.649 ms 2024-09-10T09:58:42.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:58:41 smithi159 ceph-mon[35513]: pgmap v1003: 129 pgs: 129 active+clean; 724 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 243 KiB/s rd, 3.3 MiB/s wr, 122 op/s 2024-09-10T09:58:42.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:58:41 smithi102 ceph-mon[35442]: pgmap v1003: 129 pgs: 129 active+clean; 724 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 243 KiB/s rd, 3.3 MiB/s wr, 122 op/s 2024-09-10T09:58:43.023 INFO:tasks.workunit.client.0.smithi005.stdout: 10 107589 14.17 MB/sec execute 261 sec latency 31282.659 ms 2024-09-10T09:58:43.943 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:58:43 smithi005 ceph-mon[30350]: pgmap v1004: 129 pgs: 129 active+clean; 723 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 1.2 MiB/s wr, 87 op/s 2024-09-10T09:58:44.023 INFO:tasks.workunit.client.0.smithi005.stdout: 10 107589 14.12 MB/sec execute 262 sec latency 32282.699 ms 2024-09-10T09:58:44.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:58:43 smithi159 ceph-mon[35513]: pgmap v1004: 129 pgs: 129 active+clean; 723 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 1.2 MiB/s wr, 87 op/s 2024-09-10T09:58:44.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:58:43 smithi102 ceph-mon[35442]: pgmap v1004: 129 pgs: 129 active+clean; 723 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 1.2 MiB/s wr, 87 op/s 2024-09-10T09:58:45.023 INFO:tasks.workunit.client.0.smithi005.stdout: 10 107589 14.06 MB/sec execute 263 sec latency 33282.740 ms 2024-09-10T09:58:45.950 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:58:45 smithi005 ceph-mon[30350]: pgmap v1005: 129 pgs: 129 active+clean; 720 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 82 KiB/s wr, 64 op/s 2024-09-10T09:58:46.023 INFO:tasks.workunit.client.0.smithi005.stdout: 10 107589 14.01 MB/sec execute 264 sec latency 34282.796 ms 2024-09-10T09:58:46.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:58:45 smithi159 ceph-mon[35513]: pgmap v1005: 129 pgs: 129 active+clean; 720 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 82 KiB/s wr, 64 op/s 2024-09-10T09:58:46.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:58:45 smithi102 ceph-mon[35442]: pgmap v1005: 129 pgs: 129 active+clean; 720 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 82 KiB/s wr, 64 op/s 2024-09-10T09:58:47.023 INFO:tasks.workunit.client.0.smithi005.stdout: 10 107589 13.96 MB/sec execute 265 sec latency 35282.836 ms 2024-09-10T09:58:47.933 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:58:47 smithi005 ceph-mon[30350]: pgmap v1006: 129 pgs: 129 active+clean; 717 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 139 KiB/s rd, 134 KiB/s wr, 62 op/s 2024-09-10T09:58:48.023 INFO:tasks.workunit.client.0.smithi005.stdout: 10 107589 13.91 MB/sec execute 266 sec latency 36282.895 ms 2024-09-10T09:58:48.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:58:47 smithi159 ceph-mon[35513]: pgmap v1006: 129 pgs: 129 active+clean; 717 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 139 KiB/s rd, 134 KiB/s wr, 62 op/s 2024-09-10T09:58:48.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:58:47 smithi102 ceph-mon[35442]: pgmap v1006: 129 pgs: 129 active+clean; 717 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 139 KiB/s rd, 134 KiB/s wr, 62 op/s 2024-09-10T09:58:49.023 INFO:tasks.workunit.client.0.smithi005.stdout: 10 107589 13.85 MB/sec execute 267 sec latency 37282.956 ms 2024-09-10T09:58:49.921 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:58:49 smithi005 ceph-mon[30350]: pgmap v1007: 129 pgs: 129 active+clean; 717 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 83 KiB/s wr, 41 op/s 2024-09-10T09:58:50.023 INFO:tasks.workunit.client.0.smithi005.stdout: 10 107589 13.80 MB/sec execute 268 sec latency 38283.005 ms 2024-09-10T09:58:50.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:58:49 smithi159 ceph-mon[35513]: pgmap v1007: 129 pgs: 129 active+clean; 717 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 83 KiB/s wr, 41 op/s 2024-09-10T09:58:50.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:58:49 smithi102 ceph-mon[35442]: pgmap v1007: 129 pgs: 129 active+clean; 717 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 83 KiB/s wr, 41 op/s 2024-09-10T09:58:51.023 INFO:tasks.workunit.client.0.smithi005.stdout: 10 107589 13.75 MB/sec execute 269 sec latency 39283.093 ms 2024-09-10T09:58:51.031 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:58:50 smithi005 ceph-mon[30350]: scrub summary: idle+waiting paths [/] 2024-09-10T09:58:51.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:58:50 smithi159 ceph-mon[35513]: scrub summary: idle+waiting paths [/] 2024-09-10T09:58:51.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:58:50 smithi102 ceph-mon[35442]: scrub summary: idle+waiting paths [/] 2024-09-10T09:58:51.840 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:58:51 smithi005 ceph-mon[30350]: pgmap v1008: 129 pgs: 129 active+clean; 717 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 75 KiB/s wr, 37 op/s 2024-09-10T09:58:52.023 INFO:tasks.workunit.client.0.smithi005.stdout: 10 107592 13.70 MB/sec execute 270 sec latency 40242.286 ms 2024-09-10T09:58:52.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:58:51 smithi159 ceph-mon[35513]: pgmap v1008: 129 pgs: 129 active+clean; 717 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 75 KiB/s wr, 37 op/s 2024-09-10T09:58:52.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:58:51 smithi102 ceph-mon[35442]: pgmap v1008: 129 pgs: 129 active+clean; 717 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 75 KiB/s wr, 37 op/s 2024-09-10T09:58:52.858 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 33.2 2024-09-10T09:58:52.859 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T09:58:53.024 INFO:tasks.workunit.client.0.smithi005.stdout: 10 108124 13.75 MB/sec execute 271 sec latency 9146.286 ms 2024-09-10T09:58:53.416 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:58:53.802 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:58:53 smithi005 ceph-mon[30350]: pgmap v1009: 129 pgs: 129 active+clean; 706 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 58 KiB/s wr, 42 op/s 2024-09-10T09:58:54.024 INFO:tasks.workunit.client.0.smithi005.stdout: 10 108623 13.78 MB/sec execute 272 sec latency 85.720 ms 2024-09-10T09:58:54.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:58:53 smithi159 ceph-mon[35513]: pgmap v1009: 129 pgs: 129 active+clean; 706 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 58 KiB/s wr, 42 op/s 2024-09-10T09:58:54.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:58:53 smithi102 ceph-mon[35442]: pgmap v1009: 129 pgs: 129 active+clean; 706 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 58 KiB/s wr, 42 op/s 2024-09-10T09:58:54.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:54.175+0000 7f0f86e00640 1 -- 172.21.15.5:0/898870524 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0f801036e0 msgr2=0x7f0f80107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:54.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:54.175+0000 7f0f86e00640 1 --2- 172.21.15.5:0/898870524 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0f801036e0 0x7f0f80107b90 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f0f6c009890 tx=0x7f0f6c02a380 comp rx=0 tx=0).stop 2024-09-10T09:58:54.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:54.175+0000 7f0f86e00640 1 -- 172.21.15.5:0/898870524 shutdown_connections 2024-09-10T09:58:54.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:54.175+0000 7f0f86e00640 1 --2- 172.21.15.5:0/898870524 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0f801082a0 0x7f0f8010ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:54.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:54.175+0000 7f0f86e00640 1 --2- 172.21.15.5:0/898870524 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0f801036e0 0x7f0f80107b90 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:54.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:54.175+0000 7f0f86e00640 1 --2- 172.21.15.5:0/898870524 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0f80102d30 0x7f0f80103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:54.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:54.175+0000 7f0f86e00640 1 -- 172.21.15.5:0/898870524 >> 172.21.15.5:0/898870524 conn(0x7f0f800fc7d0 msgr2=0x7f0f800febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:58:54.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:54.176+0000 7f0f86e00640 1 -- 172.21.15.5:0/898870524 shutdown_connections 2024-09-10T09:58:54.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:54.176+0000 7f0f86e00640 1 -- 172.21.15.5:0/898870524 wait complete. 2024-09-10T09:58:54.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:54.177+0000 7f0f86e00640 1 Processor -- start 2024-09-10T09:58:54.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:54.178+0000 7f0f86e00640 1 -- start start 2024-09-10T09:58:54.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:54.178+0000 7f0f86e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0f80102d30 0x7f0f80100990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:54.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:54.178+0000 7f0f86e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0f801036e0 0x7f0f80100ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:54.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:54.178+0000 7f0f86e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0f801082a0 0x7f0f80101410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:54.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:54.178+0000 7f0f86e00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f0f801120a0 con 0x7f0f80102d30 2024-09-10T09:58:54.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:54.178+0000 7f0f86e00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f0f80111f20 con 0x7f0f801036e0 2024-09-10T09:58:54.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:54.178+0000 7f0f86e00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f0f80112220 con 0x7f0f801082a0 2024-09-10T09:58:54.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:54.179+0000 7f0f7f400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0f801036e0 0x7f0f80100ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:54.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:54.179+0000 7f0f7f400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0f801036e0 0x7f0f80100ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:50718/0 (socket says 172.21.15.5:50718) 2024-09-10T09:58:54.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:54.179+0000 7f0f7f400640 1 -- 172.21.15.5:0/3504311884 learned_addr learned my addr 172.21.15.5:0/3504311884 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:58:54.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:54.179+0000 7f0f84e00640 1 --2- 172.21.15.5:0/3504311884 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0f801082a0 0x7f0f80101410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:54.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:54.179+0000 7f0f7f400640 1 -- 172.21.15.5:0/3504311884 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0f801082a0 msgr2=0x7f0f80101410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:54.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:54.179+0000 7f0f7f400640 1 --2- 172.21.15.5:0/3504311884 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0f801082a0 0x7f0f80101410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:54.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:54.179+0000 7f0f7f400640 1 -- 172.21.15.5:0/3504311884 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0f80102d30 msgr2=0x7f0f80100990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:54.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:54.179+0000 7f0f7f400640 1 --2- 172.21.15.5:0/3504311884 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0f80102d30 0x7f0f80100990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:54.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:54.179+0000 7f0f7f400640 1 -- 172.21.15.5:0/3504311884 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f801a6ae0 con 0x7f0f801036e0 2024-09-10T09:58:54.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:54.179+0000 7f0f84e00640 1 --2- 172.21.15.5:0/3504311884 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0f801082a0 0x7f0f80101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T09:58:54.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:54.180+0000 7f0f7f400640 1 --2- 172.21.15.5:0/3504311884 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0f801036e0 0x7f0f80100ed0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f0f6c0057c0 tx=0x7f0f6c033ae0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:54.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:54.180+0000 7f0f7cc00640 1 -- 172.21.15.5:0/3504311884 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f6c037040 con 0x7f0f801036e0 2024-09-10T09:58:54.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:54.180+0000 7f0f7cc00640 1 -- 172.21.15.5:0/3504311884 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0f6c040070 con 0x7f0f801036e0 2024-09-10T09:58:54.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:54.180+0000 7f0f7cc00640 1 -- 172.21.15.5:0/3504311884 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f6c003890 con 0x7f0f801036e0 2024-09-10T09:58:54.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:54.181+0000 7f0f86e00640 1 -- 172.21.15.5:0/3504311884 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f801a6d70 con 0x7f0f801036e0 2024-09-10T09:58:54.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:54.181+0000 7f0f86e00640 1 -- 172.21.15.5:0/3504311884 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f801a71a0 con 0x7f0f801036e0 2024-09-10T09:58:54.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:54.182+0000 7f0f86e00640 1 -- 172.21.15.5:0/3504311884 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f80103110 con 0x7f0f801036e0 2024-09-10T09:58:54.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:54.185+0000 7f0f7cc00640 1 -- 172.21.15.5:0/3504311884 <== mon.2 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f0f6c030550 con 0x7f0f801036e0 2024-09-10T09:58:54.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:54.190+0000 7f0f7cc00640 1 --2- 172.21.15.5:0/3504311884 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f0f640a2d50 0x7f0f640a5210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:54.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:54.190+0000 7f0f7cc00640 1 -- 172.21.15.5:0/3504311884 <== mon.2 v2:172.21.15.102:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f0f6c0ef8b0 con 0x7f0f801036e0 2024-09-10T09:58:54.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:54.190+0000 7f0f7cc00640 1 -- 172.21.15.5:0/3504311884 <== mon.2 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f0f6c11e1b0 con 0x7f0f801036e0 2024-09-10T09:58:54.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:54.191+0000 7f0f7fe00640 1 --2- 172.21.15.5:0/3504311884 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f0f640a2d50 0x7f0f640a5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:54.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:54.194+0000 7f0f7fe00640 1 --2- 172.21.15.5:0/3504311884 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f0f640a2d50 0x7f0f640a5210 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f0f700058c0 tx=0x7f0f70005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:54.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:54.607+0000 7f0f86e00640 1 -- 172.21.15.5:0/3504311884 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0f8010a260 con 0x7f0f801036e0 2024-09-10T09:58:54.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:54.609+0000 7f0f7cc00640 1 -- 172.21.15.5:0/3504311884 <== mon.2 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 42 v42) ==== 94+0+11969 (secure 0 0 0) 0x7f0f6c0bc440 con 0x7f0f801036e0 2024-09-10T09:58:54.611 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-10T09:58:54.611 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":42,"btime":"2024-09-10T09:55:06:670916+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14454,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/2970594311","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2970594311},{"type":"v1","addr":"172.21.15.5:6835","nonce":2970594311}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14514,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6839/3616502174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":3616502174},{"type":"v1","addr":"172.21.15.5:6839","nonce":3616502174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":42,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T09:35:29.626287+0000","modified":"2024-09-10T09:55:06.419829+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24542,"mds_1":24535,"mds_2":24511,"mds_3":24506,"mds_4":24443},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14520":{"gid":14520,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/2073599842","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":2073599842},{"type":"v1","addr":"172.21.15.5:6841","nonce":2073599842}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"k","rank":4,"incarnation":26,"state":"up:active","state_seq":24,"addr":"172.21.15.102:6833/2457007427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":2457007427},{"type":"v1","addr":"172.21.15.102:6833","nonce":2457007427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/356445159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":356445159},{"type":"v1","addr":"172.21.15.159:6833","nonce":356445159}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6837/2894314507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":2894314507},{"type":"v1","addr":"172.21.15.5:6837","nonce":2894314507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.102:6835/1647549111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":1647549111},{"type":"v1","addr":"172.21.15.102:6835","nonce":1647549111}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6835/952657242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":952657242},{"type":"v1","addr":"172.21.15.159:6835","nonce":952657242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":19,"addr":"172.21.15.102:6837/2689776764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":2689776764},{"type":"v1","addr":"172.21.15.102:6837","nonce":2689776764}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.159:6837/620627639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":620627639},{"type":"v1","addr":"172.21.15.159:6837","nonce":620627639}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24535":{"gid":24535,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":18,"addr":"172.21.15.159:6839/2073739599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":2073739599},{"type":"v1","addr":"172.21.15.159:6839","nonce":2073739599}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24542":{"gid":24542,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.102:6839/335409749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6838","nonce":335409749},{"type":"v1","addr":"172.21.15.102:6839","nonce":335409749}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24542,"qdb_cluster":[24542,24535,24511,24506,24443]},"id":1}]} 2024-09-10T09:58:54.611 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 42 2024-09-10T09:58:54.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:54.613+0000 7f0f86e00640 1 -- 172.21.15.5:0/3504311884 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f0f640a2d50 msgr2=0x7f0f640a5210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:54.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:54.613+0000 7f0f86e00640 1 --2- 172.21.15.5:0/3504311884 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f0f640a2d50 0x7f0f640a5210 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f0f700058c0 tx=0x7f0f70005850 comp rx=0 tx=0).stop 2024-09-10T09:58:54.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:54.613+0000 7f0f86e00640 1 -- 172.21.15.5:0/3504311884 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0f801036e0 msgr2=0x7f0f80100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:54.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:54.613+0000 7f0f86e00640 1 --2- 172.21.15.5:0/3504311884 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0f801036e0 0x7f0f80100ed0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f0f6c0057c0 tx=0x7f0f6c033ae0 comp rx=0 tx=0).stop 2024-09-10T09:58:54.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:54.613+0000 7f0f86e00640 1 -- 172.21.15.5:0/3504311884 shutdown_connections 2024-09-10T09:58:54.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:54.613+0000 7f0f86e00640 1 --2- 172.21.15.5:0/3504311884 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f0f640a2d50 0x7f0f640a5210 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:54.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:54.613+0000 7f0f86e00640 1 --2- 172.21.15.5:0/3504311884 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0f801082a0 0x7f0f80101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:54.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:54.613+0000 7f0f86e00640 1 --2- 172.21.15.5:0/3504311884 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0f801036e0 0x7f0f80100ed0 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:54.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:54.613+0000 7f0f86e00640 1 --2- 172.21.15.5:0/3504311884 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0f80102d30 0x7f0f80100990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:54.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:54.613+0000 7f0f86e00640 1 -- 172.21.15.5:0/3504311884 >> 172.21.15.5:0/3504311884 conn(0x7f0f800fc7d0 msgr2=0x7f0f80103e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:58:54.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:54.614+0000 7f0f86e00640 1 -- 172.21.15.5:0/3504311884 shutdown_connections 2024-09-10T09:58:54.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:54.614+0000 7f0f86e00640 1 -- 172.21.15.5:0/3504311884 wait complete. 2024-09-10T09:58:55.023 INFO:tasks.workunit.client.0.smithi005.stdout: 10 109208 13.84 MB/sec execute 273 sec latency 76.933 ms 2024-09-10T09:58:55.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:58:54 smithi159 ceph-mon[35513]: from='client.? 172.21.15.5:0/3504311884' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:58:55.107 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:58:54 smithi005 ceph-mon[30350]: from='client.? 172.21.15.5:0/3504311884' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:58:55.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:58:54 smithi102 ceph-mon[35442]: from='client.? 172.21.15.5:0/3504311884' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:58:55.922 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24542 quiesce db / --timeout 90 --expiration 153.2 --await-for 45.0' 2024-09-10T09:58:55.922 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24542 quiesce db / --timeout 90 --expiration 153.2 --await-for 45.0 2024-09-10T09:58:56.024 INFO:tasks.workunit.client.0.smithi005.stdout: 10 109858 13.92 MB/sec execute 274 sec latency 74.003 ms 2024-09-10T09:58:56.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:58:55 smithi159 ceph-mon[35513]: pgmap v1010: 129 pgs: 129 active+clean; 691 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 764 KiB/s rd, 116 KiB/s wr, 59 op/s 2024-09-10T09:58:56.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:58:55 smithi102 ceph-mon[35442]: pgmap v1010: 129 pgs: 129 active+clean; 691 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 764 KiB/s rd, 116 KiB/s wr, 59 op/s 2024-09-10T09:58:56.307 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:58:55 smithi005 ceph-mon[30350]: pgmap v1010: 129 pgs: 129 active+clean; 691 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 764 KiB/s rd, 116 KiB/s wr, 59 op/s 2024-09-10T09:58:56.531 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:58:57.024 INFO:tasks.workunit.client.0.smithi005.stdout: 10 110561 14.01 MB/sec execute 275 sec latency 136.955 ms 2024-09-10T09:58:57.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:57.264+0000 7ff870a00640 1 -- 172.21.15.5:0/2910360575 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff860030950 con 0x7ff8741008d0 2024-09-10T09:58:57.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:57.265+0000 7ff873200640 1 -- 172.21.15.5:0/2910360575 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff8741008d0 msgr2=0x7ff874100d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:57.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:57.265+0000 7ff873200640 1 --2- 172.21.15.5:0/2910360575 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff8741008d0 0x7ff874100d30 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7ff860009890 tx=0x7ff86002a380 comp rx=0 tx=0).stop 2024-09-10T09:58:57.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:57.265+0000 7ff873200640 1 -- 172.21.15.5:0/2910360575 shutdown_connections 2024-09-10T09:58:57.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:57.265+0000 7ff873200640 1 --2- 172.21.15.5:0/2910360575 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff874101270 0x7ff87410d7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:57.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:57.265+0000 7ff873200640 1 --2- 172.21.15.5:0/2910360575 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff8741008d0 0x7ff874100d30 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:57.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:57.265+0000 7ff873200640 1 --2- 172.21.15.5:0/2910360575 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff8741068d0 0x7ff874106cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:57.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:57.265+0000 7ff873200640 1 -- 172.21.15.5:0/2910360575 >> 172.21.15.5:0/2910360575 conn(0x7ff8740fc610 msgr2=0x7ff8740fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:58:57.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:57.265+0000 7ff873200640 1 -- 172.21.15.5:0/2910360575 shutdown_connections 2024-09-10T09:58:57.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:57.265+0000 7ff873200640 1 -- 172.21.15.5:0/2910360575 wait complete. 2024-09-10T09:58:57.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:57.266+0000 7ff873200640 1 Processor -- start 2024-09-10T09:58:57.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:57.267+0000 7ff873200640 1 -- start start 2024-09-10T09:58:57.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:57.267+0000 7ff873200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff8741008d0 0x7ff8740fccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:57.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:57.267+0000 7ff873200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff874101270 0x7ff8740fd1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:57.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:57.267+0000 7ff873200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff8741068d0 0x7ff8740fd730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:57.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:57.267+0000 7ff873200640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff874110ca0 con 0x7ff874101270 2024-09-10T09:58:57.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:57.267+0000 7ff873200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7ff874110b20 con 0x7ff8741068d0 2024-09-10T09:58:57.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:57.267+0000 7ff873200640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7ff874110e20 con 0x7ff8741008d0 2024-09-10T09:58:57.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:57.267+0000 7ff871e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff8741008d0 0x7ff8740fccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:57.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:57.267+0000 7ff871e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff8741008d0 0x7ff8740fccb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:45758/0 (socket says 172.21.15.5:45758) 2024-09-10T09:58:57.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:57.267+0000 7ff871e00640 1 -- 172.21.15.5:0/1379795465 learned_addr learned my addr 172.21.15.5:0/1379795465 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:58:57.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:57.268+0000 7ff871e00640 1 -- 172.21.15.5:0/1379795465 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff8741068d0 msgr2=0x7ff8740fd730 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T09:58:57.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:57.268+0000 7ff871e00640 1 --2- 172.21.15.5:0/1379795465 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff8741068d0 0x7ff8740fd730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:57.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:57.268+0000 7ff871e00640 1 -- 172.21.15.5:0/1379795465 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff874101270 msgr2=0x7ff8740fd1f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T09:58:57.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:57.268+0000 7ff871e00640 1 --2- 172.21.15.5:0/1379795465 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff874101270 0x7ff8740fd1f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:57.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:57.268+0000 7ff871e00640 1 -- 172.21.15.5:0/1379795465 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8741a67c0 con 0x7ff8741008d0 2024-09-10T09:58:57.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:57.268+0000 7ff871e00640 1 --2- 172.21.15.5:0/1379795465 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff8741008d0 0x7ff8740fccb0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7ff86400e9c0 tx=0x7ff86400ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:57.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:57.269+0000 7ff86aa00640 1 -- 172.21.15.5:0/1379795465 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff86400ccc0 con 0x7ff8741008d0 2024-09-10T09:58:57.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:57.269+0000 7ff86aa00640 1 -- 172.21.15.5:0/1379795465 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff864005500 con 0x7ff8741008d0 2024-09-10T09:58:57.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:57.269+0000 7ff873200640 1 -- 172.21.15.5:0/1379795465 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8741a6ab0 con 0x7ff8741008d0 2024-09-10T09:58:57.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:57.269+0000 7ff873200640 1 -- 172.21.15.5:0/1379795465 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8741a6ff0 con 0x7ff8741008d0 2024-09-10T09:58:57.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:57.269+0000 7ff86aa00640 1 -- 172.21.15.5:0/1379795465 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff864010430 con 0x7ff8741008d0 2024-09-10T09:58:57.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:57.273+0000 7ff86aa00640 1 -- 172.21.15.5:0/1379795465 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7ff8640046c0 con 0x7ff8741008d0 2024-09-10T09:58:57.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:57.275+0000 7ff87b6b8740 1 Processor -- start 2024-09-10T09:58:57.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:57.275+0000 7ff86aa00640 1 --2- 172.21.15.5:0/1379795465 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7ff8580a2b80 0x7ff8580a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:57.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:57.275+0000 7ff86aa00640 1 -- 172.21.15.5:0/1379795465 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7ff8640cf820 con 0x7ff8741008d0 2024-09-10T09:58:57.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:57.276+0000 7ff872800640 1 --2- 172.21.15.5:0/1379795465 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7ff8580a2b80 0x7ff8580a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:57.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:57.276+0000 7ff87b6b8740 1 -- start start 2024-09-10T09:58:57.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:57.276+0000 7ff87b6b8740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b32737dd70 0x55b32737e150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:57.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:57.276+0000 7ff87b6b8740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b32737e690 0x55b327388a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:57.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:57.276+0000 7ff87b6b8740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b327388f50 0x55b32738b340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:57.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:57.276+0000 7ff87b6b8740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55b3272db530 con 0x55b32737e690 2024-09-10T09:58:57.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:57.276+0000 7ff87b6b8740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55b327282860 con 0x55b327388f50 2024-09-10T09:58:57.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:57.276+0000 7ff87b6b8740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55b3272d74f0 con 0x55b32737dd70 2024-09-10T09:58:57.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:57.277+0000 7ff872800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b327388f50 0x55b32738b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:57.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:57.277+0000 7ff872800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b327388f50 0x55b32738b340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:50722/0 (socket says 172.21.15.5:50722) 2024-09-10T09:58:57.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:57.277+0000 7ff872800640 1 -- 172.21.15.5:0/616479957 learned_addr learned my addr 172.21.15.5:0/616479957 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:58:57.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:57.277+0000 7ff871e00640 1 --2- 172.21.15.5:0/616479957 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b32737dd70 0x55b32737e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:57.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:57.277+0000 7ff871400640 1 --2- 172.21.15.5:0/616479957 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b32737e690 0x55b327388a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:57.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:57.277+0000 7ff872800640 1 -- 172.21.15.5:0/616479957 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b32737dd70 msgr2=0x55b32737e150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:58:57.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:57.277+0000 7ff872800640 1 --2- 172.21.15.5:0/616479957 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b32737dd70 0x55b32737e150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:57.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:57.277+0000 7ff872800640 1 -- 172.21.15.5:0/616479957 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b32737e690 msgr2=0x55b327388a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:58:57.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:57.277+0000 7ff872800640 1 --2- 172.21.15.5:0/616479957 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b32737e690 0x55b327388a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:57.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:57.277+0000 7ff872800640 1 -- 172.21.15.5:0/616479957 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b32704cb90 con 0x55b327388f50 2024-09-10T09:58:57.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:57.278+0000 7ff872800640 1 --2- 172.21.15.5:0/1379795465 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7ff8580a2b80 0x7ff8580a5040 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7ff86c012a80 tx=0x7ff86c012600 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:57.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:57.278+0000 7ff872800640 1 --2- 172.21.15.5:0/616479957 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b327388f50 0x55b32738b340 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7ff86c00ec50 tx=0x7ff86c01a000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d3a19e0a609a781b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:57.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:57.278+0000 7ff85f000640 1 -- 172.21.15.5:0/616479957 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff86c0126f0 con 0x55b327388f50 2024-09-10T09:58:57.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:57.279+0000 7ff85f000640 1 -- 172.21.15.5:0/616479957 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff86c003e50 con 0x55b327388f50 2024-09-10T09:58:57.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:57.279+0000 7ff85f000640 1 -- 172.21.15.5:0/616479957 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff86c004690 con 0x55b327388f50 2024-09-10T09:58:57.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:57.280+0000 7ff87b6b8740 1 -- 172.21.15.5:0/616479957 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b327388f50 msgr2=0x55b32738b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:57.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:57.280+0000 7ff87b6b8740 1 --2- 172.21.15.5:0/616479957 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b327388f50 0x55b32738b340 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7ff86c00ec50 tx=0x7ff86c01a000 comp rx=0 tx=0).stop 2024-09-10T09:58:57.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:57.280+0000 7ff87b6b8740 1 -- 172.21.15.5:0/616479957 shutdown_connections 2024-09-10T09:58:57.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:57.280+0000 7ff87b6b8740 1 --2- 172.21.15.5:0/616479957 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b327388f50 0x55b32738b340 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:57.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:57.280+0000 7ff87b6b8740 1 --2- 172.21.15.5:0/616479957 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b32737e690 0x55b327388a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:57.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:57.280+0000 7ff87b6b8740 1 --2- 172.21.15.5:0/616479957 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b32737dd70 0x55b32737e150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:57.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:57.280+0000 7ff87b6b8740 1 -- 172.21.15.5:0/616479957 >> 172.21.15.5:0/616479957 conn(0x55b32737a300 msgr2=0x55b32737a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:58:57.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:57.280+0000 7ff87b6b8740 1 -- 172.21.15.5:0/616479957 shutdown_connections 2024-09-10T09:58:57.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:57.280+0000 7ff87b6b8740 1 -- 172.21.15.5:0/616479957 wait complete. 2024-09-10T09:58:57.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:57.281+0000 7ff87b6b8740 1 -- start start 2024-09-10T09:58:57.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:57.281+0000 7ff87b6b8740 1 Processor -- start 2024-09-10T09:58:57.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:57.282+0000 7ff87b6b8740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b32737dd70 0x55b327385e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:57.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:57.282+0000 7ff87b6b8740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b32737e690 0x55b327386340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:57.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:57.282+0000 7ff87b6b8740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b327388f50 0x55b327386880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:57.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:57.282+0000 7ff87b6b8740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55b32738bb20 con 0x55b32737e690 2024-09-10T09:58:57.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:57.282+0000 7ff87b6b8740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55b327295e60 con 0x55b32737dd70 2024-09-10T09:58:57.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:57.282+0000 7ff87b6b8740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55b32738bca0 con 0x55b327388f50 2024-09-10T09:58:57.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:57.282+0000 7ff87b6b8740 10 client.0 _ll_register_callbacks cb 0x55b32729acc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:58:57.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:57.282+0000 7ff871e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b32737dd70 0x55b327385e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:57.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:57.283+0000 7ff871e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b32737dd70 0x55b327385e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:50724/0 (socket says 172.21.15.5:50724) 2024-09-10T09:58:57.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:57.283+0000 7ff871e00640 1 -- 172.21.15.5:0/4581225 learned_addr learned my addr 172.21.15.5:0/4581225 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:58:57.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:57.283+0000 7ff872800640 1 --2- 172.21.15.5:0/4581225 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b327388f50 0x55b327386880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:57.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:57.284+0000 7ff872800640 1 -- 172.21.15.5:0/4581225 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b32737dd70 msgr2=0x55b327385e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:58:57.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:57.284+0000 7ff872800640 1 --2- 172.21.15.5:0/4581225 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b32737dd70 0x55b327385e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:57.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:57.284+0000 7ff872800640 1 -- 172.21.15.5:0/4581225 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b32737e690 msgr2=0x55b327386340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:58:57.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:57.284+0000 7ff872800640 1 --2- 172.21.15.5:0/4581225 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b32737e690 0x55b327386340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:57.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:57.284+0000 7ff872800640 1 -- 172.21.15.5:0/4581225 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b3272a1990 con 0x55b327388f50 2024-09-10T09:58:57.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:57.284+0000 7ff871e00640 1 --2- 172.21.15.5:0/4581225 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b32737dd70 0x55b327385e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:58:57.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:57.284+0000 7ff872800640 1 --2- 172.21.15.5:0/4581225 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b327388f50 0x55b327386880 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7ff86c015040 tx=0x7ff86c005b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4b905efab856f804 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:57.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:57.284+0000 7ff853400640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-09-10T09:58:57.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:57.284+0000 7ff853400640 1 -- 172.21.15.5:0/4581225 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff86c02f060 con 0x55b327388f50 2024-09-10T09:58:57.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:57.285+0000 7ff87b6b8740 1 -- 172.21.15.5:0/4581225 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff86c003e50 con 0x55b327388f50 2024-09-10T09:58:57.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:57.285+0000 7ff853400640 1 -- 172.21.15.5:0/4581225 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7ff86c003970 con 0x55b327388f50 2024-09-10T09:58:57.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:57.285+0000 7ff853400640 1 -- 172.21.15.5:0/4581225 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7ff86c036050 con 0x55b327388f50 2024-09-10T09:58:57.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:57.285+0000 7ff87b6b8740 10 client.29359 fetch_fsmap learned FSMap version 42 2024-09-10T09:58:57.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:57.285+0000 7ff87b6b8740 1 -- 172.21.15.5:0/4581225 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55b327397780 con 0x55b327388f50 2024-09-10T09:58:57.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:57.286+0000 7ff853400640 1 -- 172.21.15.5:0/4581225 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff86c003e50 con 0x55b327388f50 2024-09-10T09:58:57.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:57.287+0000 7ff853400640 1 -- 172.21.15.5:0/4581225 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 42) ==== 8370+0+0 (secure 0 0 0) 0x7ff86c01e070 con 0x55b327388f50 2024-09-10T09:58:57.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:57.287+0000 7ff87b6b8740 10 client.29359 fetch_fsmap finished waiting for FSMap version 42 2024-09-10T09:58:57.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:57.287+0000 7ff87b6b8740 10 client.29359 resolve_mds: validated gid 24542 aka daemon mds.b 2024-09-10T09:58:57.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:57.287+0000 7ff87b6b8740 20 client.29359 populate_metadata read hostname 'smithi005' 2024-09-10T09:58:57.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:57.287+0000 7ff87b6b8740 1 --2- 172.21.15.5:0/4581225 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55b327398b30 0x55b32739af20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:57.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:57.287+0000 7ff87b6b8740 4 client.29359 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:58:57.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:57.287+0000 7ff87b6b8740 1 -- 172.21.15.5:0/4581225 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b32727e310 con 0x55b327398b30 2024-09-10T09:58:57.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:57.288+0000 7ff871e00640 1 --2- 172.21.15.5:0/4581225 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55b327398b30 0x55b32739af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:57.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:57.291+0000 7ff871e00640 1 --2- 172.21.15.5:0/4581225 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55b327398b30 0x55b32739af20 crc :-1 s=READY pgs=287 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4ca9303ebc9942cc server_cookie=b97db297f0bc3365 in_seq=0 out_seq=0 2024-09-10T09:58:57.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:57.291+0000 7ff853400640 10 client.29359 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T09:58:57.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:57.292+0000 7ff853400640 1 -- 172.21.15.5:0/4581225 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b32727e310 con 0x55b327398b30 2024-09-10T09:58:57.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:57.292+0000 7ff853400640 10 client.29359 handle_command_reply: tid=0 2024-09-10T09:58:57.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:57.292+0000 7ff87b6b8740 1 client.29359 shutdown 2024-09-10T09:58:57.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:57.292+0000 7ff87b6b8740 1 -- 172.21.15.5:0/4581225 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b327388f50 msgr2=0x55b327386880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:57.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:57.292+0000 7ff87b6b8740 1 --2- 172.21.15.5:0/4581225 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b327388f50 0x55b327386880 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7ff86c015040 tx=0x7ff86c005b90 comp rx=0 tx=0).stop 2024-09-10T09:58:57.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:57.292+0000 7ff87b6b8740 1 -- 172.21.15.5:0/4581225 shutdown_connections 2024-09-10T09:58:57.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:57.292+0000 7ff87b6b8740 1 --2- 172.21.15.5:0/4581225 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55b327398b30 0x55b32739af20 crc :-1 s=READY pgs=287 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:57.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:57.292+0000 7ff87b6b8740 1 --2- 172.21.15.5:0/4581225 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b327388f50 0x55b327386880 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:57.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:57.292+0000 7ff87b6b8740 1 --2- 172.21.15.5:0/4581225 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b32737e690 0x55b327386340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:57.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:57.292+0000 7ff87b6b8740 1 --2- 172.21.15.5:0/4581225 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b32737dd70 0x55b327385e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:57.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:57.292+0000 7ff853400640 0 client.29359 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T09:58:57.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:57.292+0000 7ff87b6b8740 1 -- 172.21.15.5:0/4581225 >> 172.21.15.5:0/4581225 conn(0x55b32737a300 msgr2=0x55b327370c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:58:57.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:57.293+0000 7ff87b6b8740 1 -- 172.21.15.5:0/4581225 shutdown_connections 2024-09-10T09:58:57.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:57.293+0000 7ff87b6b8740 1 -- 172.21.15.5:0/4581225 wait complete. 2024-09-10T09:58:57.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:57.293+0000 7ff87b6b8740 20 client.29359 trim_cache size 0 max 16384 2024-09-10T09:58:57.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:57.323+0000 7ff87b6b8740 1 Processor -- start 2024-09-10T09:58:57.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:57.324+0000 7ff87b6b8740 1 -- start start 2024-09-10T09:58:57.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:57.324+0000 7ff87b6b8740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b32737dd70 0x55b3273a0520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:57.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:57.324+0000 7ff87b6b8740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b32737e690 0x55b3273a0a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:57.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:57.324+0000 7ff87b6b8740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b327388f50 0x55b32737f810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:57.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:57.324+0000 7ff87b6b8740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff85400fdf0 con 0x55b327388f50 2024-09-10T09:58:57.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:57.324+0000 7ff87b6b8740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7ff854011930 con 0x55b32737dd70 2024-09-10T09:58:57.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:57.324+0000 7ff87b6b8740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7ff8540104c0 con 0x55b32737e690 2024-09-10T09:58:57.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:57.324+0000 7ff871400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b32737e690 0x55b3273a0a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:57.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:57.324+0000 7ff871400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b32737e690 0x55b3273a0a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:45782/0 (socket says 172.21.15.5:45782) 2024-09-10T09:58:57.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:57.324+0000 7ff871400640 1 -- 172.21.15.5:0/3132192974 learned_addr learned my addr 172.21.15.5:0/3132192974 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:58:57.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:57.325+0000 7ff872800640 1 --2- 172.21.15.5:0/3132192974 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b327388f50 0x55b32737f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:57.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:57.325+0000 7ff871400640 1 -- 172.21.15.5:0/3132192974 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b32737dd70 msgr2=0x55b3273a0520 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T09:58:57.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:57.325+0000 7ff871400640 1 --2- 172.21.15.5:0/3132192974 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b32737dd70 0x55b3273a0520 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:57.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:57.325+0000 7ff871400640 1 -- 172.21.15.5:0/3132192974 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b327388f50 msgr2=0x55b32737f810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:58:57.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:57.325+0000 7ff871400640 1 --2- 172.21.15.5:0/3132192974 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b327388f50 0x55b32737f810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:57.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:57.325+0000 7ff871400640 1 -- 172.21.15.5:0/3132192974 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b32737a130 con 0x55b32737e690 2024-09-10T09:58:57.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:57.325+0000 7ff871400640 1 --2- 172.21.15.5:0/3132192974 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b32737e690 0x55b3273a0a60 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7ff8740fe740 tx=0x7ff860009450 comp rx=0 tx=0).ready entity=mon.1 client_cookie=70a7598eb3358a7d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:57.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:57.326+0000 7ff85e600640 1 -- 172.21.15.5:0/3132192974 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff860040020 con 0x55b32737e690 2024-09-10T09:58:57.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:57.326+0000 7ff85e600640 1 -- 172.21.15.5:0/3132192974 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff860030340 con 0x55b32737e690 2024-09-10T09:58:57.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:57.326+0000 7ff85e600640 1 -- 172.21.15.5:0/3132192974 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff860037040 con 0x55b32737e690 2024-09-10T09:58:57.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:57.327+0000 7ff87b6b8740 1 -- 172.21.15.5:0/3132192974 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b32737e690 msgr2=0x55b3273a0a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:57.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:57.327+0000 7ff87b6b8740 1 --2- 172.21.15.5:0/3132192974 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b32737e690 0x55b3273a0a60 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7ff8740fe740 tx=0x7ff860009450 comp rx=0 tx=0).stop 2024-09-10T09:58:57.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:57.327+0000 7ff87b6b8740 1 -- 172.21.15.5:0/3132192974 shutdown_connections 2024-09-10T09:58:57.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:57.327+0000 7ff87b6b8740 1 --2- 172.21.15.5:0/3132192974 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b327388f50 0x55b32737f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:57.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:57.327+0000 7ff87b6b8740 1 --2- 172.21.15.5:0/3132192974 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b32737e690 0x55b3273a0a60 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:57.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:57.327+0000 7ff87b6b8740 1 --2- 172.21.15.5:0/3132192974 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b32737dd70 0x55b3273a0520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:57.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:57.327+0000 7ff87b6b8740 1 -- 172.21.15.5:0/3132192974 >> 172.21.15.5:0/3132192974 conn(0x55b32737a300 msgr2=0x55b327370c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:58:57.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:57.327+0000 7ff87b6b8740 1 -- 172.21.15.5:0/3132192974 shutdown_connections 2024-09-10T09:58:57.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:57.327+0000 7ff87b6b8740 1 -- 172.21.15.5:0/3132192974 wait complete. 2024-09-10T09:58:57.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:57.327+0000 7ff87b6b8740 1 -- start start 2024-09-10T09:58:57.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:57.328+0000 7ff87b6b8740 1 Processor -- start 2024-09-10T09:58:57.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:57.328+0000 7ff87b6b8740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b32737dd70 0x55b32738d050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:57.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:57.328+0000 7ff87b6b8740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b32737e690 0x55b32738d590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:57.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:57.328+0000 7ff87b6b8740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b327388f50 0x55b3274024f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:57.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:57.328+0000 7ff87b6b8740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff854011260 con 0x55b32737e690 2024-09-10T09:58:57.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:57.328+0000 7ff87b6b8740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7ff854010b90 con 0x55b327388f50 2024-09-10T09:58:57.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:57.328+0000 7ff87b6b8740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7ff85400ddd0 con 0x55b32737dd70 2024-09-10T09:58:57.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:57.328+0000 7ff87b6b8740 10 client.0 _ll_register_callbacks cb 0x7ff86c015070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:58:57.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:57.328+0000 7ff871e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b32737dd70 0x55b32738d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:57.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:57.329+0000 7ff872800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b327388f50 0x55b3274024f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:57.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:57.329+0000 7ff871e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b32737dd70 0x55b32738d050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:45788/0 (socket says 172.21.15.5:45788) 2024-09-10T09:58:57.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:57.329+0000 7ff871e00640 1 -- 172.21.15.5:0/2017231819 learned_addr learned my addr 172.21.15.5:0/2017231819 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:58:57.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:57.329+0000 7ff871400640 1 --2- 172.21.15.5:0/2017231819 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b32737e690 0x55b32738d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:57.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:57.329+0000 7ff871e00640 1 -- 172.21.15.5:0/2017231819 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b327388f50 msgr2=0x55b3274024f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:58:57.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:57.329+0000 7ff871e00640 1 --2- 172.21.15.5:0/2017231819 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b327388f50 0x55b3274024f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:57.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:57.329+0000 7ff871e00640 1 -- 172.21.15.5:0/2017231819 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b32737e690 msgr2=0x55b32738d590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:58:57.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:57.329+0000 7ff871e00640 1 --2- 172.21.15.5:0/2017231819 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b32737e690 0x55b32738d590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:57.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:57.329+0000 7ff871e00640 1 -- 172.21.15.5:0/2017231819 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b32738b970 con 0x55b32737dd70 2024-09-10T09:58:57.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:57.329+0000 7ff872800640 1 --2- 172.21.15.5:0/2017231819 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b327388f50 0x55b3274024f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:58:57.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:57.329+0000 7ff871e00640 1 --2- 172.21.15.5:0/2017231819 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b32737dd70 0x55b32738d050 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7ff864098600 tx=0x7ff8640a00a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f790bce120818c1d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:58:57.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:57.329+0000 7ff853400640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-09-10T09:58:57.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:57.330+0000 7ff853400640 1 -- 172.21.15.5:0/2017231819 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff864016d90 con 0x55b32737dd70 2024-09-10T09:58:57.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:57.330+0000 7ff853400640 1 -- 172.21.15.5:0/2017231819 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7ff8640a4410 con 0x55b32737dd70 2024-09-10T09:58:57.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:57.330+0000 7ff87b6b8740 1 -- 172.21.15.5:0/2017231819 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff860030340 con 0x55b32737dd70 2024-09-10T09:58:57.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:57.330+0000 7ff853400640 1 -- 172.21.15.5:0/2017231819 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7ff8640a4610 con 0x55b32737dd70 2024-09-10T09:58:57.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:57.330+0000 7ff87b6b8740 10 client.29365 fetch_fsmap learned FSMap version 42 2024-09-10T09:58:57.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:57.330+0000 7ff87b6b8740 1 -- 172.21.15.5:0/2017231819 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55b327403960 con 0x55b32737dd70 2024-09-10T09:58:57.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:57.330+0000 7ff853400640 1 -- 172.21.15.5:0/2017231819 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff860030340 con 0x55b32737dd70 2024-09-10T09:58:57.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:57.330+0000 7ff853400640 1 -- 172.21.15.5:0/2017231819 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 42) ==== 8370+0+0 (secure 0 0 0) 0x7ff864097980 con 0x55b32737dd70 2024-09-10T09:58:57.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:57.331+0000 7ff87b6b8740 10 client.29365 fetch_fsmap finished waiting for FSMap version 42 2024-09-10T09:58:57.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:57.331+0000 7ff87b6b8740 10 client.29365 resolve_mds: validated gid 24542 aka daemon mds.b 2024-09-10T09:58:57.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:57.331+0000 7ff87b6b8740 20 client.29365 populate_metadata read hostname 'smithi005' 2024-09-10T09:58:57.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:57.331+0000 7ff87b6b8740 1 --2- 172.21.15.5:0/2017231819 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55b3274040a0 0x55b327406490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:58:57.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:57.331+0000 7ff87b6b8740 4 client.29365 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 153.2, "await_for": 45.0}] 2024-09-10T09:58:57.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:57.331+0000 7ff87b6b8740 1 -- 172.21.15.5:0/2017231819 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 153.2, "await_for": 45.0}) -- 0x55b3272e8c60 con 0x55b3274040a0 2024-09-10T09:58:57.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:57.331+0000 7ff871400640 1 --2- 172.21.15.5:0/2017231819 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55b3274040a0 0x55b327406490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:58:57.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:57.331+0000 7ff871400640 1 --2- 172.21.15.5:0/2017231819 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55b3274040a0 0x55b327406490 crc :-1 s=READY pgs=288 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9f4e5a8724fbf09f server_cookie=2939a8616bd53e45 in_seq=0 out_seq=0 2024-09-10T09:58:57.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:57.331+0000 7ff853400640 10 client.29365 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T09:58:58.025 INFO:tasks.workunit.client.0.smithi005.stdout: 10 110874 14.02 MB/sec execute 276 sec latency 689.202 ms 2024-09-10T09:58:58.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:58:57 smithi159 ceph-mon[35513]: pgmap v1011: 129 pgs: 129 active+clean; 658 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 773 KiB/s wr, 173 op/s 2024-09-10T09:58:58.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:58:57 smithi159 ceph-mon[35513]: scrub summary: idle 2024-09-10T09:58:58.101 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:58:57 smithi159 ceph-mon[35513]: scrub summary: idle 2024-09-10T09:58:58.101 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:58:57 smithi159 ceph-mon[35513]: scrub summary: idle 2024-09-10T09:58:58.101 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:58:57 smithi159 ceph-mon[35513]: scrub summary: idle 2024-09-10T09:58:58.101 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:58:57 smithi159 ceph-mon[35513]: scrub summary: idle 2024-09-10T09:58:58.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:58:57 smithi102 ceph-mon[35442]: pgmap v1011: 129 pgs: 129 active+clean; 658 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 773 KiB/s wr, 173 op/s 2024-09-10T09:58:58.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:58:57 smithi102 ceph-mon[35442]: scrub summary: idle 2024-09-10T09:58:58.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:58:57 smithi102 ceph-mon[35442]: scrub summary: idle 2024-09-10T09:58:58.135 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:58:57 smithi102 ceph-mon[35442]: scrub summary: idle 2024-09-10T09:58:58.135 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:58:57 smithi102 ceph-mon[35442]: scrub summary: idle 2024-09-10T09:58:58.135 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:58:57 smithi102 ceph-mon[35442]: scrub summary: idle 2024-09-10T09:58:58.307 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:58:57 smithi005 ceph-mon[30350]: pgmap v1011: 129 pgs: 129 active+clean; 658 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 773 KiB/s wr, 173 op/s 2024-09-10T09:58:58.307 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:58:57 smithi005 ceph-mon[30350]: scrub summary: idle 2024-09-10T09:58:58.307 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:58:57 smithi005 ceph-mon[30350]: scrub summary: idle 2024-09-10T09:58:58.307 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:58:57 smithi005 ceph-mon[30350]: scrub summary: idle 2024-09-10T09:58:58.307 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:58:57 smithi005 ceph-mon[30350]: scrub summary: idle 2024-09-10T09:58:58.307 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:58:57 smithi005 ceph-mon[30350]: scrub summary: idle 2024-09-10T09:58:58.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:58.691+0000 7ff853400640 1 -- 172.21.15.5:0/2017231819 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55b3272e8c60 con 0x55b3274040a0 2024-09-10T09:58:58.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:58.691+0000 7ff853400640 10 client.29365 handle_command_reply: tid=0 2024-09-10T09:58:58.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:58.691+0000 7ff87b6b8740 1 client.29365 shutdown 2024-09-10T09:58:58.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:58.692+0000 7ff87b6b8740 1 -- 172.21.15.5:0/2017231819 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b32737dd70 msgr2=0x55b32738d050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:58.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:58.692+0000 7ff87b6b8740 1 --2- 172.21.15.5:0/2017231819 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b32737dd70 0x55b32738d050 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7ff864098600 tx=0x7ff8640a00a0 comp rx=0 tx=0).stop 2024-09-10T09:58:58.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:58.692+0000 7ff87b6b8740 1 -- 172.21.15.5:0/2017231819 shutdown_connections 2024-09-10T09:58:58.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:58.692+0000 7ff87b6b8740 1 --2- 172.21.15.5:0/2017231819 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55b3274040a0 0x55b327406490 crc :-1 s=READY pgs=288 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:58.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:58.692+0000 7ff87b6b8740 1 --2- 172.21.15.5:0/2017231819 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b327388f50 0x55b3274024f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:58.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:58.692+0000 7ff87b6b8740 1 --2- 172.21.15.5:0/2017231819 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b32737e690 0x55b32738d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:58.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:58.692+0000 7ff853400640 0 client.29365 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T09:58:58.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:58.692+0000 7ff87b6b8740 1 --2- 172.21.15.5:0/2017231819 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b32737dd70 0x55b32738d050 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:58.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:58.692+0000 7ff87b6b8740 1 -- 172.21.15.5:0/2017231819 >> 172.21.15.5:0/2017231819 conn(0x55b32737a300 msgr2=0x55b327385da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:58:58.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:58.692+0000 7ff87b6b8740 1 -- 172.21.15.5:0/2017231819 shutdown_connections 2024-09-10T09:58:58.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:58.692+0000 7ff87b6b8740 1 -- 172.21.15.5:0/2017231819 wait complete. 2024-09-10T09:58:58.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:58.693+0000 7ff87b6b8740 20 client.29365 trim_cache size 0 max 16384 2024-09-10T09:58:58.695 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-10T09:58:58.695 INFO:teuthology.orchestra.run.smithi005.stdout: "epoch": 42, 2024-09-10T09:58:58.695 INFO:teuthology.orchestra.run.smithi005.stdout: "leader": 24542, 2024-09-10T09:58:58.695 INFO:teuthology.orchestra.run.smithi005.stdout: "set_version": 74, 2024-09-10T09:58:58.696 INFO:teuthology.orchestra.run.smithi005.stdout: "sets": { 2024-09-10T09:58:58.696 INFO:teuthology.orchestra.run.smithi005.stdout: "830978ed": { 2024-09-10T09:58:58.696 INFO:teuthology.orchestra.run.smithi005.stdout: "version": 74, 2024-09-10T09:58:58.696 INFO:teuthology.orchestra.run.smithi005.stdout: "age_ref": 0.0, 2024-09-10T09:58:58.696 INFO:teuthology.orchestra.run.smithi005.stdout: "state": { 2024-09-10T09:58:58.696 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "QUIESCED", 2024-09-10T09:58:58.696 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0 2024-09-10T09:58:58.696 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T09:58:58.696 INFO:teuthology.orchestra.run.smithi005.stdout: "timeout": 90.0, 2024-09-10T09:58:58.696 INFO:teuthology.orchestra.run.smithi005.stdout: "expiration": 153.2, 2024-09-10T09:58:58.696 INFO:teuthology.orchestra.run.smithi005.stdout: "members": { 2024-09-10T09:58:58.696 INFO:teuthology.orchestra.run.smithi005.stdout: "file:/": { 2024-09-10T09:58:58.697 INFO:teuthology.orchestra.run.smithi005.stdout: "excluded": false, 2024-09-10T09:58:58.697 INFO:teuthology.orchestra.run.smithi005.stdout: "state": { 2024-09-10T09:58:58.697 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "QUIESCED", 2024-09-10T09:58:58.697 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0 2024-09-10T09:58:58.697 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T09:58:58.697 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T09:58:58.697 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T09:58:58.697 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T09:58:58.697 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T09:58:58.697 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-10T09:58:58.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:58.694+0000 7ff873200640 1 -- 172.21.15.5:0/1379795465 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7ff8580a2b80 msgr2=0x7ff8580a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:58.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:58.694+0000 7ff873200640 1 --2- 172.21.15.5:0/1379795465 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7ff8580a2b80 0x7ff8580a5040 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7ff86c012a80 tx=0x7ff86c012600 comp rx=0 tx=0).stop 2024-09-10T09:58:58.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:58.694+0000 7ff873200640 1 -- 172.21.15.5:0/1379795465 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff8741008d0 msgr2=0x7ff8740fccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:58:58.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:58.694+0000 7ff873200640 1 --2- 172.21.15.5:0/1379795465 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff8741008d0 0x7ff8740fccb0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7ff86400e9c0 tx=0x7ff86400ee80 comp rx=0 tx=0).stop 2024-09-10T09:58:58.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:58.694+0000 7ff873200640 1 -- 172.21.15.5:0/1379795465 shutdown_connections 2024-09-10T09:58:58.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:58.695+0000 7ff873200640 1 --2- 172.21.15.5:0/1379795465 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7ff8580a2b80 0x7ff8580a5040 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:58.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:58.695+0000 7ff873200640 1 --2- 172.21.15.5:0/1379795465 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff8741068d0 0x7ff8740fd730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:58.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:58.695+0000 7ff873200640 1 --2- 172.21.15.5:0/1379795465 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff874101270 0x7ff8740fd1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:58.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:58.695+0000 7ff873200640 1 --2- 172.21.15.5:0/1379795465 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff8741008d0 0x7ff8740fccb0 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:58:58.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:58.695+0000 7ff873200640 1 -- 172.21.15.5:0/1379795465 >> 172.21.15.5:0/1379795465 conn(0x7ff8740fc610 msgr2=0x7ff87406a270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:58:58.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:58.695+0000 7ff873200640 1 -- 172.21.15.5:0/1379795465 shutdown_connections 2024-09-10T09:58:58.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:58:58.695+0000 7ff873200640 1 -- 172.21.15.5:0/1379795465 wait complete. 2024-09-10T09:58:59.024 INFO:tasks.workunit.client.0.smithi005.stdout: 10 110875 13.97 MB/sec execute 277 sec latency 1689.654 ms 2024-09-10T09:58:59.780 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '830978ed', quiesce took 6.9 seconds. Will release after: 26.300000000000004 2024-09-10T09:59:00.024 INFO:tasks.workunit.client.0.smithi005.stdout: 10 110875 13.92 MB/sec execute 278 sec latency 2689.718 ms 2024-09-10T09:59:00.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:58:59 smithi005 ceph-mon[30350]: pgmap v1012: 129 pgs: 129 active+clean; 653 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 1.1 MiB/s wr, 219 op/s 2024-09-10T09:59:00.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:58:59 smithi159 ceph-mon[35513]: pgmap v1012: 129 pgs: 129 active+clean; 653 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 1.1 MiB/s wr, 219 op/s 2024-09-10T09:59:00.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:58:59 smithi102 ceph-mon[35442]: pgmap v1012: 129 pgs: 129 active+clean; 653 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 1.1 MiB/s wr, 219 op/s 2024-09-10T09:59:01.024 INFO:tasks.workunit.client.0.smithi005.stdout: 10 110875 13.87 MB/sec execute 279 sec latency 3689.824 ms 2024-09-10T09:59:02.025 INFO:tasks.workunit.client.0.smithi005.stdout: 10 110875 13.82 MB/sec execute 280 sec latency 4689.934 ms 2024-09-10T09:59:02.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:59:01 smithi159 ceph-mon[35513]: pgmap v1013: 129 pgs: 129 active+clean; 645 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 1.9 MiB/s wr, 352 op/s 2024-09-10T09:59:02.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:59:01 smithi102 ceph-mon[35442]: pgmap v1013: 129 pgs: 129 active+clean; 645 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 1.9 MiB/s wr, 352 op/s 2024-09-10T09:59:02.306 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:59:01 smithi005 ceph-mon[30350]: pgmap v1013: 129 pgs: 129 active+clean; 645 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 1.9 MiB/s wr, 352 op/s 2024-09-10T09:59:03.025 INFO:tasks.workunit.client.0.smithi005.stdout: 10 110875 13.77 MB/sec execute 281 sec latency 5690.053 ms 2024-09-10T09:59:03.275 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T09:59:03.662 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:59:04.025 INFO:tasks.workunit.client.0.smithi005.stdout: 10 110875 13.73 MB/sec execute 282 sec latency 6690.176 ms 2024-09-10T09:59:04.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:59:03 smithi005 ceph-mon[30350]: pgmap v1014: 129 pgs: 129 active+clean; 646 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 1.9 MiB/s wr, 357 op/s 2024-09-10T09:59:04.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:59:03 smithi159 ceph-mon[35513]: pgmap v1014: 129 pgs: 129 active+clean; 646 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 1.9 MiB/s wr, 357 op/s 2024-09-10T09:59:04.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:59:03 smithi102 ceph-mon[35442]: pgmap v1014: 129 pgs: 129 active+clean; 646 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 1.9 MiB/s wr, 357 op/s 2024-09-10T09:59:04.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:04.362+0000 7f45e1200640 1 -- 172.21.15.5:0/411712107 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f45dc109140 msgr2=0x7f45dc110cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:04.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:04.362+0000 7f45e1200640 1 --2- 172.21.15.5:0/411712107 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f45dc109140 0x7f45dc110cc0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f45d4009ca0 tx=0x7f45d402a8c0 comp rx=0 tx=0).stop 2024-09-10T09:59:04.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:04.362+0000 7f45e1200640 1 -- 172.21.15.5:0/411712107 shutdown_connections 2024-09-10T09:59:04.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:04.362+0000 7f45e1200640 1 --2- 172.21.15.5:0/411712107 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f45dc109140 0x7f45dc110cc0 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:04.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:04.362+0000 7f45e1200640 1 --2- 172.21.15.5:0/411712107 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f45dc1049c0 0x7f45dc108a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:04.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:04.362+0000 7f45e1200640 1 --2- 172.21.15.5:0/411712107 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f45dc104010 0x7f45dc1043f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:04.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:04.362+0000 7f45e1200640 1 -- 172.21.15.5:0/411712107 >> 172.21.15.5:0/411712107 conn(0x7f45dc0ffa50 msgr2=0x7f45dc101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:04.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:04.362+0000 7f45e1200640 1 -- 172.21.15.5:0/411712107 shutdown_connections 2024-09-10T09:59:04.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:04.362+0000 7f45e1200640 1 -- 172.21.15.5:0/411712107 wait complete. 2024-09-10T09:59:04.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:04.363+0000 7f45e1200640 1 Processor -- start 2024-09-10T09:59:04.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:04.363+0000 7f45e1200640 1 -- start start 2024-09-10T09:59:04.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:04.364+0000 7f45e1200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f45dc104010 0x7f45dc19a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:04.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:04.364+0000 7f45e1200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f45dc1049c0 0x7f45dc19ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:04.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:04.364+0000 7f45e1200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f45dc109140 0x7f45dc19eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:04.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:04.364+0000 7f45e1200640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f45dc114150 con 0x7f45dc109140 2024-09-10T09:59:04.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:04.364+0000 7f45e1200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f45dc113fd0 con 0x7f45dc104010 2024-09-10T09:59:04.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:04.364+0000 7f45e1200640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f45dc1142d0 con 0x7f45dc1049c0 2024-09-10T09:59:04.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:04.364+0000 7f45d3e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f45dc1049c0 0x7f45dc19ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:04.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:04.364+0000 7f45d9e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f45dc104010 0x7f45dc19a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:04.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:04.364+0000 7f45d3e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f45dc1049c0 0x7f45dc19ab30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:59892/0 (socket says 172.21.15.5:59892) 2024-09-10T09:59:04.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:04.364+0000 7f45d3e00640 1 -- 172.21.15.5:0/3928712606 learned_addr learned my addr 172.21.15.5:0/3928712606 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:59:04.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:04.364+0000 7f45da800640 1 --2- 172.21.15.5:0/3928712606 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f45dc109140 0x7f45dc19eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:04.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:04.364+0000 7f45d9e00640 1 -- 172.21.15.5:0/3928712606 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f45dc1049c0 msgr2=0x7f45dc19ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:04.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:04.364+0000 7f45d9e00640 1 --2- 172.21.15.5:0/3928712606 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f45dc1049c0 0x7f45dc19ab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:04.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:04.365+0000 7f45d9e00640 1 -- 172.21.15.5:0/3928712606 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f45dc109140 msgr2=0x7f45dc19eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:04.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:04.365+0000 7f45d9e00640 1 --2- 172.21.15.5:0/3928712606 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f45dc109140 0x7f45dc19eec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:04.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:04.365+0000 7f45d9e00640 1 -- 172.21.15.5:0/3928712606 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45dc19f5a0 con 0x7f45dc104010 2024-09-10T09:59:04.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:04.365+0000 7f45d9e00640 1 --2- 172.21.15.5:0/3928712606 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f45dc104010 0x7f45dc19a5f0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f45cc00d6d0 tx=0x7f45cc00db90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:04.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:04.365+0000 7f45d2a00640 1 -- 172.21.15.5:0/3928712606 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45cc017070 con 0x7f45dc104010 2024-09-10T09:59:04.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:04.365+0000 7f45e1200640 1 -- 172.21.15.5:0/3928712606 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45dc19f890 con 0x7f45dc104010 2024-09-10T09:59:04.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:04.365+0000 7f45d2a00640 1 -- 172.21.15.5:0/3928712606 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f45cc009820 con 0x7f45dc104010 2024-09-10T09:59:04.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:04.365+0000 7f45d2a00640 1 -- 172.21.15.5:0/3928712606 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45cc004cd0 con 0x7f45dc104010 2024-09-10T09:59:04.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:04.365+0000 7f45e1200640 1 -- 172.21.15.5:0/3928712606 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45dc1a7170 con 0x7f45dc104010 2024-09-10T09:59:04.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:04.369+0000 7f45d2a00640 1 -- 172.21.15.5:0/3928712606 <== mon.2 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f45cc0099d0 con 0x7f45dc104010 2024-09-10T09:59:04.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:04.369+0000 7f45e3c93740 1 Processor -- start 2024-09-10T09:59:04.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:04.369+0000 7f45e3c93740 1 -- start start 2024-09-10T09:59:04.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:04.369+0000 7f45e3c93740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562cee877b90 0x562cee877f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:04.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:04.369+0000 7f45e3c93740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562cee8784b0 0x562cee882830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:04.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:04.369+0000 7f45e3c93740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562cee882d70 0x562cee885160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:04.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:04.369+0000 7f45e3c93740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x562cee7de3e0 con 0x562cee882d70 2024-09-10T09:59:04.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:04.369+0000 7f45e3c93740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x562cee785d80 con 0x562cee8784b0 2024-09-10T09:59:04.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:04.369+0000 7f45e3c93740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x562cee7da3a0 con 0x562cee877b90 2024-09-10T09:59:04.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:04.369+0000 7f45d9e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562cee8784b0 0x562cee882830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:04.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:04.369+0000 7f45d9e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562cee8784b0 0x562cee882830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:47032/0 (socket says 172.21.15.5:47032) 2024-09-10T09:59:04.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:04.369+0000 7f45d9e00640 1 -- 172.21.15.5:0/2816998680 learned_addr learned my addr 172.21.15.5:0/2816998680 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:59:04.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:04.370+0000 7f45da800640 1 --2- 172.21.15.5:0/2816998680 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562cee877b90 0x562cee877f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:04.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:04.370+0000 7f45d3e00640 1 --2- 172.21.15.5:0/2816998680 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562cee882d70 0x562cee885160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:04.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:04.370+0000 7f45d9e00640 1 -- 172.21.15.5:0/2816998680 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562cee877b90 msgr2=0x562cee877f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:59:04.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:04.370+0000 7f45d9e00640 1 --2- 172.21.15.5:0/2816998680 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562cee877b90 0x562cee877f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:04.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:04.370+0000 7f45d9e00640 1 -- 172.21.15.5:0/2816998680 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562cee882d70 msgr2=0x562cee885160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:59:04.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:04.370+0000 7f45d9e00640 1 --2- 172.21.15.5:0/2816998680 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562cee882d70 0x562cee885160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:04.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:04.370+0000 7f45d9e00640 1 -- 172.21.15.5:0/2816998680 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562cee7a4ae0 con 0x562cee8784b0 2024-09-10T09:59:04.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:04.370+0000 7f45d9e00640 1 --2- 172.21.15.5:0/2816998680 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562cee8784b0 0x562cee882830 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f45cc0d2fd0 tx=0x7f45cc0d7570 comp rx=0 tx=0).ready entity=mon.2 client_cookie=14d216d6866ab483 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:04.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:04.371+0000 7f45d2a00640 1 --2- 172.21.15.5:0/3928712606 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f45c00a2c00 0x7f45c00a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:04.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:04.371+0000 7f45cb400640 1 -- 172.21.15.5:0/2816998680 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45cc0e2070 con 0x562cee8784b0 2024-09-10T09:59:04.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:04.371+0000 7f45d2a00640 1 -- 172.21.15.5:0/3928712606 <== mon.2 v2:172.21.15.102:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f45cc0cef30 con 0x7f45dc104010 2024-09-10T09:59:04.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:04.371+0000 7f45cb400640 1 -- 172.21.15.5:0/2816998680 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f45cc0b9f80 con 0x562cee8784b0 2024-09-10T09:59:04.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:04.371+0000 7f45da800640 1 --2- 172.21.15.5:0/3928712606 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f45c00a2c00 0x7f45c00a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:04.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:04.371+0000 7f45cb400640 1 -- 172.21.15.5:0/2816998680 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45cc005540 con 0x562cee8784b0 2024-09-10T09:59:04.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:04.372+0000 7f45e3c93740 1 -- 172.21.15.5:0/2816998680 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562cee8784b0 msgr2=0x562cee882830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:04.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:04.372+0000 7f45da800640 1 --2- 172.21.15.5:0/3928712606 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f45c00a2c00 0x7f45c00a50c0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f45d4004af0 tx=0x7f45d4004a80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:04.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:04.372+0000 7f45e3c93740 1 --2- 172.21.15.5:0/2816998680 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562cee8784b0 0x562cee882830 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f45cc0d2fd0 tx=0x7f45cc0d7570 comp rx=0 tx=0).stop 2024-09-10T09:59:04.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:04.372+0000 7f45e3c93740 1 -- 172.21.15.5:0/2816998680 shutdown_connections 2024-09-10T09:59:04.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:04.372+0000 7f45e3c93740 1 --2- 172.21.15.5:0/2816998680 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562cee882d70 0x562cee885160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:04.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:04.372+0000 7f45e3c93740 1 --2- 172.21.15.5:0/2816998680 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562cee8784b0 0x562cee882830 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:04.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:04.372+0000 7f45e3c93740 1 --2- 172.21.15.5:0/2816998680 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562cee877b90 0x562cee877f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:04.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:04.372+0000 7f45e3c93740 1 -- 172.21.15.5:0/2816998680 >> 172.21.15.5:0/2816998680 conn(0x562cee874120 msgr2=0x562cee874500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:04.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:04.372+0000 7f45e3c93740 1 -- 172.21.15.5:0/2816998680 shutdown_connections 2024-09-10T09:59:04.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:04.372+0000 7f45e3c93740 1 -- 172.21.15.5:0/2816998680 wait complete. 2024-09-10T09:59:04.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:04.372+0000 7f45e3c93740 1 -- start start 2024-09-10T09:59:04.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:04.373+0000 7f45e3c93740 1 Processor -- start 2024-09-10T09:59:04.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:04.373+0000 7f45e3c93740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562cee877b90 0x562cee87fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:04.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:04.373+0000 7f45e3c93740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562cee8784b0 0x562cee880130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:04.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:04.373+0000 7f45e3c93740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562cee882d70 0x562cee880670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:04.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:04.373+0000 7f45e3c93740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x562cee8859e0 con 0x562cee8784b0 2024-09-10T09:59:04.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:04.373+0000 7f45e3c93740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x562cee7990f0 con 0x562cee877b90 2024-09-10T09:59:04.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:04.373+0000 7f45e3c93740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x562cee885b60 con 0x562cee882d70 2024-09-10T09:59:04.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:04.373+0000 7f45e3c93740 10 client.0 _ll_register_callbacks cb 0x562cee78a9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:59:04.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:04.373+0000 7f45d3e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562cee8784b0 0x562cee880130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:04.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:04.373+0000 7f45d3e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562cee8784b0 0x562cee880130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59608/0 (socket says 172.21.15.5:59608) 2024-09-10T09:59:04.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:04.373+0000 7f45d3e00640 1 -- 172.21.15.5:0/3534562232 learned_addr learned my addr 172.21.15.5:0/3534562232 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:59:04.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:04.373+0000 7f45da800640 1 --2- 172.21.15.5:0/3534562232 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562cee882d70 0x562cee880670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:04.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:04.373+0000 7f45d9e00640 1 --2- 172.21.15.5:0/3534562232 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562cee877b90 0x562cee87fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:04.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:04.374+0000 7f45d3e00640 1 -- 172.21.15.5:0/3534562232 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562cee882d70 msgr2=0x562cee880670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:59:04.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:04.374+0000 7f45d3e00640 1 --2- 172.21.15.5:0/3534562232 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562cee882d70 0x562cee880670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:04.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:04.374+0000 7f45d3e00640 1 -- 172.21.15.5:0/3534562232 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562cee877b90 msgr2=0x562cee87fbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:59:04.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:04.374+0000 7f45d3e00640 1 --2- 172.21.15.5:0/3534562232 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562cee877b90 0x562cee87fbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:04.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:04.374+0000 7f45d3e00640 1 -- 172.21.15.5:0/3534562232 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562cee54f3c0 con 0x562cee8784b0 2024-09-10T09:59:04.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:04.374+0000 7f45d3e00640 1 --2- 172.21.15.5:0/3534562232 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562cee8784b0 0x562cee880130 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f45c40127d0 tx=0x7f45c4012c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cbac4ae60a28510 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:04.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:04.374+0000 7f45bfe00640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-10T09:59:04.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:04.374+0000 7f45bfe00640 1 -- 172.21.15.5:0/3534562232 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45c4004020 con 0x562cee8784b0 2024-09-10T09:59:04.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:04.374+0000 7f45bfe00640 1 -- 172.21.15.5:0/3534562232 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f45c40041c0 con 0x562cee8784b0 2024-09-10T09:59:04.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:04.374+0000 7f45e3c93740 1 -- 172.21.15.5:0/3534562232 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f45cc0b9f80 con 0x562cee8784b0 2024-09-10T09:59:04.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:04.375+0000 7f45bfe00640 1 -- 172.21.15.5:0/3534562232 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f45c4025050 con 0x562cee8784b0 2024-09-10T09:59:04.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:04.375+0000 7f45e3c93740 10 client.19326 fetch_fsmap learned FSMap version 42 2024-09-10T09:59:04.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:04.375+0000 7f45e3c93740 1 -- 172.21.15.5:0/3534562232 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x562cee8916b0 con 0x562cee8784b0 2024-09-10T09:59:04.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:04.375+0000 7f45bfe00640 1 -- 172.21.15.5:0/3534562232 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f45cc0b9f80 con 0x562cee8784b0 2024-09-10T09:59:04.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:04.375+0000 7f45bfe00640 1 -- 172.21.15.5:0/3534562232 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 42) ==== 8370+0+0 (secure 0 0 0) 0x7f45c4003980 con 0x562cee8784b0 2024-09-10T09:59:04.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:04.375+0000 7f45e3c93740 10 client.19326 fetch_fsmap finished waiting for FSMap version 42 2024-09-10T09:59:04.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:04.375+0000 7f45e3c93740 10 client.19326 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T09:59:04.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:04.375+0000 7f45e3c93740 20 client.19326 populate_metadata read hostname 'smithi005' 2024-09-10T09:59:04.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:04.375+0000 7f45e3c93740 1 --2- 172.21.15.5:0/3534562232 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x562cee892b70 0x562cee894fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:04.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:04.375+0000 7f45e3c93740 4 client.19326 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:59:04.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:04.375+0000 7f45e3c93740 1 -- 172.21.15.5:0/3534562232 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562cee7eb950 con 0x562cee892b70 2024-09-10T09:59:04.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:04.376+0000 7f45d9e00640 1 --2- 172.21.15.5:0/3534562232 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x562cee892b70 0x562cee894fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:04.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:04.376+0000 7f45d9e00640 1 --2- 172.21.15.5:0/3534562232 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x562cee892b70 0x562cee894fa0 crc :-1 s=READY pgs=289 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=41b5a1256ff7cdb1 server_cookie=5c91cfdfdf871c4d in_seq=0 out_seq=0 2024-09-10T09:59:04.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:04.376+0000 7f45bfe00640 10 client.19326 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T09:59:04.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:04.377+0000 7f45bfe00640 1 -- 172.21.15.5:0/3534562232 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562cee7eb950 con 0x562cee892b70 2024-09-10T09:59:04.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:04.377+0000 7f45bfe00640 10 client.19326 handle_command_reply: tid=0 2024-09-10T09:59:04.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:04.377+0000 7f45e3c93740 1 client.19326 shutdown 2024-09-10T09:59:04.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:04.377+0000 7f45e3c93740 1 -- 172.21.15.5:0/3534562232 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562cee8784b0 msgr2=0x562cee880130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:04.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:04.377+0000 7f45e3c93740 1 --2- 172.21.15.5:0/3534562232 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562cee8784b0 0x562cee880130 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f45c40127d0 tx=0x7f45c4012c90 comp rx=0 tx=0).stop 2024-09-10T09:59:04.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:04.378+0000 7f45e3c93740 1 -- 172.21.15.5:0/3534562232 shutdown_connections 2024-09-10T09:59:04.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:04.378+0000 7f45e3c93740 1 --2- 172.21.15.5:0/3534562232 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x562cee892b70 0x562cee894fa0 crc :-1 s=READY pgs=289 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:04.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:04.378+0000 7f45e3c93740 1 --2- 172.21.15.5:0/3534562232 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562cee882d70 0x562cee880670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:04.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:04.378+0000 7f45e3c93740 1 --2- 172.21.15.5:0/3534562232 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562cee8784b0 0x562cee880130 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:04.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:04.378+0000 7f45e3c93740 1 --2- 172.21.15.5:0/3534562232 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562cee877b90 0x562cee87fbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:04.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:04.378+0000 7f45e3c93740 1 -- 172.21.15.5:0/3534562232 >> 172.21.15.5:0/3534562232 conn(0x562cee874120 msgr2=0x562cee879e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:04.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:04.378+0000 7f45bfe00640 0 client.19326 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T09:59:04.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:04.378+0000 7f45e3c93740 1 -- 172.21.15.5:0/3534562232 shutdown_connections 2024-09-10T09:59:04.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:04.378+0000 7f45e3c93740 1 -- 172.21.15.5:0/3534562232 wait complete. 2024-09-10T09:59:04.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:04.378+0000 7f45e3c93740 20 client.19326 trim_cache size 0 max 16384 2024-09-10T09:59:04.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:04.396+0000 7f45e3c93740 1 Processor -- start 2024-09-10T09:59:04.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:04.396+0000 7f45e3c93740 1 -- start start 2024-09-10T09:59:04.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:04.396+0000 7f45e3c93740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562cee877b90 0x562cee881ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:04.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:04.396+0000 7f45e3c93740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562cee8784b0 0x562cee89a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:04.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:04.396+0000 7f45e3c93740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562cee882d70 0x562cee89adf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:04.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:04.396+0000 7f45e3c93740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f45dc1b1180 con 0x562cee882d70 2024-09-10T09:59:04.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:04.396+0000 7f45e3c93740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f45dc1b2cc0 con 0x562cee8784b0 2024-09-10T09:59:04.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:04.396+0000 7f45e3c93740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f45dc1b1850 con 0x562cee877b90 2024-09-10T09:59:04.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:04.397+0000 7f45da800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562cee882d70 0x562cee89adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:04.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:04.397+0000 7f45d3e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562cee8784b0 0x562cee89a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:04.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:04.397+0000 7f45d9e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562cee877b90 0x562cee881ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:04.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:04.397+0000 7f45d3e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562cee8784b0 0x562cee89a8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:47058/0 (socket says 172.21.15.5:47058) 2024-09-10T09:59:04.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:04.397+0000 7f45d3e00640 1 -- 172.21.15.5:0/3854037261 learned_addr learned my addr 172.21.15.5:0/3854037261 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:59:04.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:04.397+0000 7f45d3e00640 1 -- 172.21.15.5:0/3854037261 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562cee877b90 msgr2=0x562cee881ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:59:04.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:04.397+0000 7f45d3e00640 1 --2- 172.21.15.5:0/3854037261 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562cee877b90 0x562cee881ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:04.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:04.397+0000 7f45d3e00640 1 -- 172.21.15.5:0/3854037261 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562cee882d70 msgr2=0x562cee89adf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:59:04.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:04.397+0000 7f45d3e00640 1 --2- 172.21.15.5:0/3854037261 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562cee882d70 0x562cee89adf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:04.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:04.397+0000 7f45d3e00640 1 -- 172.21.15.5:0/3854037261 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562cee873f50 con 0x562cee8784b0 2024-09-10T09:59:04.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:04.397+0000 7f45d9e00640 1 --2- 172.21.15.5:0/3854037261 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562cee877b90 0x562cee881ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:59:04.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:04.397+0000 7f45d3e00640 1 --2- 172.21.15.5:0/3854037261 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562cee8784b0 0x562cee89a8b0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f45c4003dd0 tx=0x7f45c401c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=833657bcbc9d241f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:04.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:04.397+0000 7f45caa00640 1 -- 172.21.15.5:0/3854037261 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45c4022040 con 0x562cee8784b0 2024-09-10T09:59:04.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:04.398+0000 7f45caa00640 1 -- 172.21.15.5:0/3854037261 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f45c4003ad0 con 0x562cee8784b0 2024-09-10T09:59:04.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:04.398+0000 7f45caa00640 1 -- 172.21.15.5:0/3854037261 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45c4005350 con 0x562cee8784b0 2024-09-10T09:59:04.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:04.398+0000 7f45e3c93740 1 -- 172.21.15.5:0/3854037261 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562cee8784b0 msgr2=0x562cee89a8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:04.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:04.398+0000 7f45e3c93740 1 --2- 172.21.15.5:0/3854037261 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562cee8784b0 0x562cee89a8b0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f45c4003dd0 tx=0x7f45c401c6a0 comp rx=0 tx=0).stop 2024-09-10T09:59:04.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:04.398+0000 7f45e3c93740 1 -- 172.21.15.5:0/3854037261 shutdown_connections 2024-09-10T09:59:04.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:04.398+0000 7f45e3c93740 1 --2- 172.21.15.5:0/3854037261 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562cee882d70 0x562cee89adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:04.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:04.398+0000 7f45e3c93740 1 --2- 172.21.15.5:0/3854037261 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562cee8784b0 0x562cee89a8b0 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:04.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:04.398+0000 7f45e3c93740 1 --2- 172.21.15.5:0/3854037261 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562cee877b90 0x562cee881ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:04.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:04.398+0000 7f45e3c93740 1 -- 172.21.15.5:0/3854037261 >> 172.21.15.5:0/3854037261 conn(0x562cee874120 msgr2=0x562cee8731c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:04.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:04.398+0000 7f45e3c93740 1 -- 172.21.15.5:0/3854037261 shutdown_connections 2024-09-10T09:59:04.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:04.398+0000 7f45e3c93740 1 -- 172.21.15.5:0/3854037261 wait complete. 2024-09-10T09:59:04.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:04.399+0000 7f45e3c93740 1 -- start start 2024-09-10T09:59:04.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:04.399+0000 7f45e3c93740 1 Processor -- start 2024-09-10T09:59:04.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:04.399+0000 7f45e3c93740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562cee877b90 0x562cee88b250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:04.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:04.399+0000 7f45e3c93740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562cee8784b0 0x562cee887db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:04.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:04.399+0000 7f45e3c93740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562cee882d70 0x562cee8882f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:04.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:04.399+0000 7f45e3c93740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f45dc1b25f0 con 0x562cee877b90 2024-09-10T09:59:04.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:04.399+0000 7f45e3c93740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f45dc1b1f20 con 0x562cee8784b0 2024-09-10T09:59:04.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:04.399+0000 7f45e3c93740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f45dc1af160 con 0x562cee882d70 2024-09-10T09:59:04.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:04.399+0000 7f45e3c93740 10 client.0 _ll_register_callbacks cb 0x7f45c4012450 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:59:04.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:04.400+0000 7f45d9e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562cee877b90 0x562cee88b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:04.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:04.400+0000 7f45d3e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562cee8784b0 0x562cee887db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:04.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:04.400+0000 7f45d3e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562cee8784b0 0x562cee887db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:47070/0 (socket says 172.21.15.5:47070) 2024-09-10T09:59:04.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:04.400+0000 7f45d3e00640 1 -- 172.21.15.5:0/3644629217 learned_addr learned my addr 172.21.15.5:0/3644629217 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:59:04.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:04.400+0000 7f45da800640 1 --2- 172.21.15.5:0/3644629217 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562cee882d70 0x562cee8882f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:04.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:04.400+0000 7f45d3e00640 1 -- 172.21.15.5:0/3644629217 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562cee882d70 msgr2=0x562cee8882f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:59:04.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:04.400+0000 7f45d3e00640 1 --2- 172.21.15.5:0/3644629217 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562cee882d70 0x562cee8882f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:04.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:04.400+0000 7f45d3e00640 1 -- 172.21.15.5:0/3644629217 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562cee877b90 msgr2=0x562cee88b250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:59:04.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:04.400+0000 7f45d3e00640 1 --2- 172.21.15.5:0/3644629217 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562cee877b90 0x562cee88b250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:04.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:04.400+0000 7f45d3e00640 1 -- 172.21.15.5:0/3644629217 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562cee87aa00 con 0x562cee8784b0 2024-09-10T09:59:04.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:04.400+0000 7f45d3e00640 1 --2- 172.21.15.5:0/3644629217 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562cee8784b0 0x562cee887db0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f45c4003d50 tx=0x7f45c400f820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3468f8025d41b148 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:04.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:04.400+0000 7f45bfe00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-09-10T09:59:04.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:04.401+0000 7f45bfe00640 1 -- 172.21.15.5:0/3644629217 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45c401c470 con 0x562cee8784b0 2024-09-10T09:59:04.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:04.401+0000 7f45e3c93740 1 -- 172.21.15.5:0/3644629217 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f45c4003ad0 con 0x562cee8784b0 2024-09-10T09:59:04.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:04.401+0000 7f45bfe00640 1 -- 172.21.15.5:0/3644629217 <== mon.2 v2:172.21.15.102:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f45c402f070 con 0x562cee8784b0 2024-09-10T09:59:04.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:04.401+0000 7f45bfe00640 1 -- 172.21.15.5:0/3644629217 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f45c403c050 con 0x562cee8784b0 2024-09-10T09:59:04.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:04.401+0000 7f45e3c93740 10 client.29399 fetch_fsmap learned FSMap version 42 2024-09-10T09:59:04.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:04.401+0000 7f45e3c93740 1 -- 172.21.15.5:0/3644629217 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x562cee8fe420 con 0x562cee8784b0 2024-09-10T09:59:04.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:04.401+0000 7f45bfe00640 1 -- 172.21.15.5:0/3644629217 <== mon.2 v2:172.21.15.102:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f45c4003ad0 con 0x562cee8784b0 2024-09-10T09:59:04.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:04.402+0000 7f45bfe00640 1 -- 172.21.15.5:0/3644629217 <== mon.2 v2:172.21.15.102:3300/0 5 ==== fsmap(e 42) ==== 8370+0+0 (secure 0 0 0) 0x7f45c401c790 con 0x562cee8784b0 2024-09-10T09:59:04.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:04.402+0000 7f45e3c93740 10 client.29399 fetch_fsmap finished waiting for FSMap version 42 2024-09-10T09:59:04.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:04.402+0000 7f45e3c93740 10 client.29399 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T09:59:04.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:04.402+0000 7f45e3c93740 20 client.29399 populate_metadata read hostname 'smithi005' 2024-09-10T09:59:04.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:04.402+0000 7f45e3c93740 1 --2- 172.21.15.5:0/3644629217 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x562cee8feb90 0x562cee900fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:04.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:04.402+0000 7f45e3c93740 4 client.29399 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T09:59:04.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:04.402+0000 7f45e3c93740 1 -- 172.21.15.5:0/3644629217 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "scrub status"}) -- 0x562cee7e5e60 con 0x562cee8feb90 2024-09-10T09:59:04.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:04.402+0000 7f45d9e00640 1 --2- 172.21.15.5:0/3644629217 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x562cee8feb90 0x562cee900fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:04.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:04.402+0000 7f45d9e00640 1 --2- 172.21.15.5:0/3644629217 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x562cee8feb90 0x562cee900fb0 crc :-1 s=READY pgs=290 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9c7358e43fd70ba server_cookie=8fc53fef7ebe68bf in_seq=0 out_seq=0 2024-09-10T09:59:04.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:04.402+0000 7f45bfe00640 10 client.29399 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T09:59:04.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:04.403+0000 7f45bfe00640 1 -- 172.21.15.5:0/3644629217 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x562cee7e5e60 con 0x562cee8feb90 2024-09-10T09:59:04.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:04.403+0000 7f45bfe00640 10 client.29399 handle_command_reply: tid=0 2024-09-10T09:59:04.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:04.403+0000 7f45e3c93740 1 client.29399 shutdown 2024-09-10T09:59:04.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:04.403+0000 7f45e3c93740 1 -- 172.21.15.5:0/3644629217 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562cee8784b0 msgr2=0x562cee887db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:04.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:04.403+0000 7f45e3c93740 1 --2- 172.21.15.5:0/3644629217 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562cee8784b0 0x562cee887db0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f45c4003d50 tx=0x7f45c400f820 comp rx=0 tx=0).stop 2024-09-10T09:59:04.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:04.403+0000 7f45e3c93740 1 -- 172.21.15.5:0/3644629217 shutdown_connections 2024-09-10T09:59:04.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:04.403+0000 7f45e3c93740 1 --2- 172.21.15.5:0/3644629217 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x562cee8feb90 0x562cee900fb0 crc :-1 s=READY pgs=290 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:04.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:04.403+0000 7f45e3c93740 1 --2- 172.21.15.5:0/3644629217 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562cee882d70 0x562cee8882f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:04.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:04.403+0000 7f45e3c93740 1 --2- 172.21.15.5:0/3644629217 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562cee8784b0 0x562cee887db0 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:04.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:04.403+0000 7f45e3c93740 1 --2- 172.21.15.5:0/3644629217 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562cee877b90 0x562cee88b250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:04.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:04.403+0000 7f45e3c93740 1 -- 172.21.15.5:0/3644629217 >> 172.21.15.5:0/3644629217 conn(0x562cee874120 msgr2=0x562cee8729a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:04.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:04.403+0000 7f45bfe00640 0 client.29399 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T09:59:04.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:04.403+0000 7f45e3c93740 1 -- 172.21.15.5:0/3644629217 shutdown_connections 2024-09-10T09:59:04.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:04.403+0000 7f45e3c93740 1 -- 172.21.15.5:0/3644629217 wait complete. 2024-09-10T09:59:04.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:04.403+0000 7f45e3c93740 20 client.29399 trim_cache size 0 max 16384 2024-09-10T09:59:04.409 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-10T09:59:04.409 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "no active scrubs running", 2024-09-10T09:59:04.409 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": {} 2024-09-10T09:59:04.409 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-10T09:59:04.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:04.404+0000 7f45e1200640 1 -- 172.21.15.5:0/3928712606 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f45c00a2c00 msgr2=0x7f45c00a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:04.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:04.404+0000 7f45e1200640 1 --2- 172.21.15.5:0/3928712606 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f45c00a2c00 0x7f45c00a50c0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f45d4004af0 tx=0x7f45d4004a80 comp rx=0 tx=0).stop 2024-09-10T09:59:04.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:04.404+0000 7f45e1200640 1 -- 172.21.15.5:0/3928712606 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f45dc104010 msgr2=0x7f45dc19a5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:04.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:04.404+0000 7f45e1200640 1 --2- 172.21.15.5:0/3928712606 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f45dc104010 0x7f45dc19a5f0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f45cc00d6d0 tx=0x7f45cc00db90 comp rx=0 tx=0).stop 2024-09-10T09:59:04.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:04.404+0000 7f45e1200640 1 -- 172.21.15.5:0/3928712606 shutdown_connections 2024-09-10T09:59:04.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:04.404+0000 7f45e1200640 1 --2- 172.21.15.5:0/3928712606 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f45c00a2c00 0x7f45c00a50c0 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:04.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:04.404+0000 7f45e1200640 1 --2- 172.21.15.5:0/3928712606 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f45dc109140 0x7f45dc19eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:04.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:04.404+0000 7f45e1200640 1 --2- 172.21.15.5:0/3928712606 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f45dc1049c0 0x7f45dc19ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:04.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:04.404+0000 7f45e1200640 1 --2- 172.21.15.5:0/3928712606 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f45dc104010 0x7f45dc19a5f0 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:04.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:04.404+0000 7f45e1200640 1 -- 172.21.15.5:0/3928712606 >> 172.21.15.5:0/3928712606 conn(0x7f45dc0ffa50 msgr2=0x7f45dc100de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:04.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:04.404+0000 7f45e1200640 1 -- 172.21.15.5:0/3928712606 shutdown_connections 2024-09-10T09:59:04.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:04.404+0000 7f45e1200640 1 -- 172.21.15.5:0/3928712606 wait complete. 2024-09-10T09:59:05.025 INFO:tasks.workunit.client.0.smithi005.stdout: 10 110875 13.68 MB/sec execute 283 sec latency 7690.278 ms 2024-09-10T09:59:05.621 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-10T09:59:05.621 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T09:59:06.008 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:59:06.025 INFO:tasks.workunit.client.0.smithi005.stdout: 10 110875 13.63 MB/sec execute 284 sec latency 8690.317 ms 2024-09-10T09:59:06.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:59:05 smithi159 ceph-mon[35513]: pgmap v1015: 129 pgs: 129 active+clean; 646 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 1.9 MiB/s wr, 342 op/s 2024-09-10T09:59:06.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:59:05 smithi102 ceph-mon[35442]: pgmap v1015: 129 pgs: 129 active+clean; 646 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 1.9 MiB/s wr, 342 op/s 2024-09-10T09:59:06.148 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:59:05 smithi005 ceph-mon[30350]: pgmap v1015: 129 pgs: 129 active+clean; 646 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 1.9 MiB/s wr, 342 op/s 2024-09-10T09:59:07.028 INFO:tasks.workunit.client.0.smithi005.stdout: 10 110875 13.58 MB/sec execute 285 sec latency 9690.389 ms 2024-09-10T09:59:07.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:07.106+0000 7effda200640 1 -- 172.21.15.5:0/3811557475 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7effd4109140 msgr2=0x7effd4110cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:07.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:07.106+0000 7effda200640 1 --2- 172.21.15.5:0/3811557475 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7effd4109140 0x7effd4110cc0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7effcc009ca0 tx=0x7effcc02a8c0 comp rx=0 tx=0).stop 2024-09-10T09:59:07.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:07.106+0000 7effda200640 1 -- 172.21.15.5:0/3811557475 shutdown_connections 2024-09-10T09:59:07.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:07.106+0000 7effda200640 1 --2- 172.21.15.5:0/3811557475 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7effd4109140 0x7effd4110cc0 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:07.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:07.106+0000 7effda200640 1 --2- 172.21.15.5:0/3811557475 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7effd41049c0 0x7effd4108a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:07.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:07.106+0000 7effda200640 1 --2- 172.21.15.5:0/3811557475 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7effd4104010 0x7effd41043f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:07.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:07.106+0000 7effda200640 1 -- 172.21.15.5:0/3811557475 >> 172.21.15.5:0/3811557475 conn(0x7effd40ffa50 msgr2=0x7effd4101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:07.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:07.106+0000 7effda200640 1 -- 172.21.15.5:0/3811557475 shutdown_connections 2024-09-10T09:59:07.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:07.106+0000 7effda200640 1 -- 172.21.15.5:0/3811557475 wait complete. 2024-09-10T09:59:07.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:07.107+0000 7effda200640 1 Processor -- start 2024-09-10T09:59:07.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:07.107+0000 7effda200640 1 -- start start 2024-09-10T09:59:07.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:07.108+0000 7effda200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7effd4104010 0x7effd419a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:07.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:07.108+0000 7effda200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7effd41049c0 0x7effd419abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:07.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:07.108+0000 7effda200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7effd4109140 0x7effd419ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:07.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:07.108+0000 7effda200640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7effd4114150 con 0x7effd41049c0 2024-09-10T09:59:07.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:07.108+0000 7effda200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7effd4113fd0 con 0x7effd4109140 2024-09-10T09:59:07.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:07.108+0000 7effda200640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7effd41142d0 con 0x7effd4104010 2024-09-10T09:59:07.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:07.108+0000 7effcbe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7effd41049c0 0x7effd419abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:07.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:07.108+0000 7effd3400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7effd4104010 0x7effd419a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:07.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:07.108+0000 7effd3e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7effd4109140 0x7effd419ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:07.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:07.108+0000 7effd3400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7effd4104010 0x7effd419a690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:59960/0 (socket says 172.21.15.5:59960) 2024-09-10T09:59:07.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:07.108+0000 7effd3400640 1 -- 172.21.15.5:0/3425561842 learned_addr learned my addr 172.21.15.5:0/3425561842 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:59:07.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:07.109+0000 7effd3400640 1 -- 172.21.15.5:0/3425561842 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7effd4109140 msgr2=0x7effd419ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:07.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:07.109+0000 7effd3400640 1 --2- 172.21.15.5:0/3425561842 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7effd4109140 0x7effd419ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:07.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:07.109+0000 7effd3400640 1 -- 172.21.15.5:0/3425561842 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7effd41049c0 msgr2=0x7effd419abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:07.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:07.109+0000 7effd3400640 1 --2- 172.21.15.5:0/3425561842 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7effd41049c0 0x7effd419abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:07.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:07.109+0000 7effd3400640 1 -- 172.21.15.5:0/3425561842 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7effd419f640 con 0x7effd4104010 2024-09-10T09:59:07.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:07.109+0000 7effd3400640 1 --2- 172.21.15.5:0/3425561842 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7effd4104010 0x7effd419a690 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7effc400e9f0 tx=0x7effc400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:07.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:07.109+0000 7effd0c00640 1 -- 172.21.15.5:0/3425561842 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effc400cd80 con 0x7effd4104010 2024-09-10T09:59:07.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:07.109+0000 7effda200640 1 -- 172.21.15.5:0/3425561842 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7effd419f930 con 0x7effd4104010 2024-09-10T09:59:07.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:07.110+0000 7effda200640 1 -- 172.21.15.5:0/3425561842 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7effd41a7210 con 0x7effd4104010 2024-09-10T09:59:07.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:07.110+0000 7effd0c00640 1 -- 172.21.15.5:0/3425561842 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7effc4010910 con 0x7effd4104010 2024-09-10T09:59:07.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:07.110+0000 7effd0c00640 1 -- 172.21.15.5:0/3425561842 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effc4018980 con 0x7effd4104010 2024-09-10T09:59:07.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:07.111+0000 7effda200640 1 -- 172.21.15.5:0/3425561842 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7effd41043f0 con 0x7effd4104010 2024-09-10T09:59:07.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:07.113+0000 7effd0c00640 1 -- 172.21.15.5:0/3425561842 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7effc4010ac0 con 0x7effd4104010 2024-09-10T09:59:07.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:07.114+0000 7effd0c00640 1 --2- 172.21.15.5:0/3425561842 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7effb80a2b80 0x7effb80a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:07.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:07.114+0000 7effd0c00640 1 -- 172.21.15.5:0/3425561842 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7effc40cfa40 con 0x7effd4104010 2024-09-10T09:59:07.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:07.114+0000 7effcbe00640 1 --2- 172.21.15.5:0/3425561842 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7effb80a2b80 0x7effb80a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:07.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:07.115+0000 7effcbe00640 1 --2- 172.21.15.5:0/3425561842 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7effb80a2b80 0x7effb80a5040 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7effd40ffe70 tx=0x7effbc005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:07.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:07.116+0000 7effd0c00640 1 -- 172.21.15.5:0/3425561842 <== mon.1 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7effc409bf00 con 0x7effd4104010 2024-09-10T09:59:07.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:07.342+0000 7effda200640 1 -- 172.21.15.5:0/3425561842 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7effd410c380 con 0x7effd4104010 2024-09-10T09:59:07.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:07.343+0000 7effd0c00640 1 -- 172.21.15.5:0/3425561842 <== mon.1 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 42 v42) ==== 94+0+11969 (secure 0 0 0) 0x7effc40a0200 con 0x7effd4104010 2024-09-10T09:59:07.345 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-10T09:59:07.345 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":42,"btime":"2024-09-10T09:55:06:670916+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14454,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/2970594311","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2970594311},{"type":"v1","addr":"172.21.15.5:6835","nonce":2970594311}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14514,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6839/3616502174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":3616502174},{"type":"v1","addr":"172.21.15.5:6839","nonce":3616502174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":42,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T09:35:29.626287+0000","modified":"2024-09-10T09:55:06.419829+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24542,"mds_1":24535,"mds_2":24511,"mds_3":24506,"mds_4":24443},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14520":{"gid":14520,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/2073599842","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":2073599842},{"type":"v1","addr":"172.21.15.5:6841","nonce":2073599842}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"k","rank":4,"incarnation":26,"state":"up:active","state_seq":24,"addr":"172.21.15.102:6833/2457007427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":2457007427},{"type":"v1","addr":"172.21.15.102:6833","nonce":2457007427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/356445159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":356445159},{"type":"v1","addr":"172.21.15.159:6833","nonce":356445159}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6837/2894314507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":2894314507},{"type":"v1","addr":"172.21.15.5:6837","nonce":2894314507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.102:6835/1647549111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":1647549111},{"type":"v1","addr":"172.21.15.102:6835","nonce":1647549111}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6835/952657242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":952657242},{"type":"v1","addr":"172.21.15.159:6835","nonce":952657242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":19,"addr":"172.21.15.102:6837/2689776764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":2689776764},{"type":"v1","addr":"172.21.15.102:6837","nonce":2689776764}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.159:6837/620627639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":620627639},{"type":"v1","addr":"172.21.15.159:6837","nonce":620627639}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24535":{"gid":24535,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":18,"addr":"172.21.15.159:6839/2073739599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":2073739599},{"type":"v1","addr":"172.21.15.159:6839","nonce":2073739599}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24542":{"gid":24542,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.102:6839/335409749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6838","nonce":335409749},{"type":"v1","addr":"172.21.15.102:6839","nonce":335409749}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24542,"qdb_cluster":[24542,24535,24511,24506,24443]},"id":1}]} 2024-09-10T09:59:07.345 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 42 2024-09-10T09:59:07.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:07.346+0000 7effda200640 1 -- 172.21.15.5:0/3425561842 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7effb80a2b80 msgr2=0x7effb80a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:07.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:07.346+0000 7effda200640 1 --2- 172.21.15.5:0/3425561842 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7effb80a2b80 0x7effb80a5040 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7effd40ffe70 tx=0x7effbc005f90 comp rx=0 tx=0).stop 2024-09-10T09:59:07.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:07.346+0000 7effda200640 1 -- 172.21.15.5:0/3425561842 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7effd4104010 msgr2=0x7effd419a690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:07.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:07.346+0000 7effda200640 1 --2- 172.21.15.5:0/3425561842 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7effd4104010 0x7effd419a690 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7effc400e9f0 tx=0x7effc400eeb0 comp rx=0 tx=0).stop 2024-09-10T09:59:07.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:07.346+0000 7effda200640 1 -- 172.21.15.5:0/3425561842 shutdown_connections 2024-09-10T09:59:07.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:07.346+0000 7effda200640 1 --2- 172.21.15.5:0/3425561842 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7effb80a2b80 0x7effb80a5040 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:07.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:07.346+0000 7effda200640 1 --2- 172.21.15.5:0/3425561842 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7effd4109140 0x7effd419ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:07.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:07.346+0000 7effda200640 1 --2- 172.21.15.5:0/3425561842 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7effd41049c0 0x7effd419abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:07.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:07.346+0000 7effda200640 1 --2- 172.21.15.5:0/3425561842 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7effd4104010 0x7effd419a690 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:07.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:07.346+0000 7effda200640 1 -- 172.21.15.5:0/3425561842 >> 172.21.15.5:0/3425561842 conn(0x7effd40ffa50 msgr2=0x7effd4100d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:07.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:07.346+0000 7effda200640 1 -- 172.21.15.5:0/3425561842 shutdown_connections 2024-09-10T09:59:07.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:07.346+0000 7effda200640 1 -- 172.21.15.5:0/3425561842 wait complete. 2024-09-10T09:59:08.025 INFO:tasks.workunit.client.0.smithi005.stdout: 10 110875 13.53 MB/sec execute 286 sec latency 10690.437 ms 2024-09-10T09:59:08.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:59:07 smithi005 ceph-mon[30350]: pgmap v1016: 129 pgs: 129 active+clean; 646 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 1.9 MiB/s wr, 327 op/s 2024-09-10T09:59:08.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:59:07 smithi005 ceph-mon[30350]: from='client.? 172.21.15.5:0/3425561842' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:59:08.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:59:07 smithi159 ceph-mon[35513]: pgmap v1016: 129 pgs: 129 active+clean; 646 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 1.9 MiB/s wr, 327 op/s 2024-09-10T09:59:08.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:59:07 smithi159 ceph-mon[35513]: from='client.? 172.21.15.5:0/3425561842' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:59:08.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:59:07 smithi102 ceph-mon[35442]: pgmap v1016: 129 pgs: 129 active+clean; 646 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 1.9 MiB/s wr, 327 op/s 2024-09-10T09:59:08.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:59:07 smithi102 ceph-mon[35442]: from='client.? 172.21.15.5:0/3425561842' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:59:08.269 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-10T09:59:08.848 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:59:09.025 INFO:tasks.workunit.client.0.smithi005.stdout: 10 110875 13.49 MB/sec execute 287 sec latency 11690.478 ms 2024-09-10T09:59:10.025 INFO:tasks.workunit.client.0.smithi005.stdout: 10 110875 13.44 MB/sec execute 288 sec latency 12690.542 ms 2024-09-10T09:59:10.098 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:59:09 smithi005 ceph-mon[30350]: pgmap v1017: 129 pgs: 129 active+clean; 646 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 1.3 MiB/s wr, 210 op/s 2024-09-10T09:59:10.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:59:09 smithi102 ceph-mon[35442]: pgmap v1017: 129 pgs: 129 active+clean; 646 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 1.3 MiB/s wr, 210 op/s 2024-09-10T09:59:10.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:10.150+0000 7fdbf2800640 1 -- 172.21.15.5:0/584228436 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdbec103e30 msgr2=0x7fdbec104210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:10.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:10.150+0000 7fdbf2800640 1 --2- 172.21.15.5:0/584228436 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdbec103e30 0x7fdbec104210 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fdbe4009ca0 tx=0x7fdbe402a8b0 comp rx=0 tx=0).stop 2024-09-10T09:59:10.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:10.151+0000 7fdbf2800640 1 -- 172.21.15.5:0/584228436 shutdown_connections 2024-09-10T09:59:10.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:10.151+0000 7fdbf2800640 1 --2- 172.21.15.5:0/584228436 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdbec108f60 0x7fdbec110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:10.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:10.151+0000 7fdbf2800640 1 --2- 172.21.15.5:0/584228436 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdbec1047e0 0x7fdbec108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:10.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:10.151+0000 7fdbf2800640 1 --2- 172.21.15.5:0/584228436 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdbec103e30 0x7fdbec104210 secure :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fdbe4009ca0 tx=0x7fdbe402a8b0 comp rx=0 tx=0).stop 2024-09-10T09:59:10.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:10.151+0000 7fdbf2800640 1 -- 172.21.15.5:0/584228436 >> 172.21.15.5:0/584228436 conn(0x7fdbec0ff8d0 msgr2=0x7fdbec101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:10.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:10.152+0000 7fdbf2800640 1 -- 172.21.15.5:0/584228436 shutdown_connections 2024-09-10T09:59:10.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:10.152+0000 7fdbf2800640 1 -- 172.21.15.5:0/584228436 wait complete. 2024-09-10T09:59:10.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:10.153+0000 7fdbf2800640 1 Processor -- start 2024-09-10T09:59:10.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:10.154+0000 7fdbf2800640 1 -- start start 2024-09-10T09:59:10.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:10.154+0000 7fdbf2800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdbec1047e0 0x7fdbec19a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:10.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:10.154+0000 7fdbf2800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdbec108f60 0x7fdbec19ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:10.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:10.154+0000 7fdbf2800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdbec19eee0 0x7fdbec19f390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:10.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:10.154+0000 7fdbf2800640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fdbec114010 con 0x7fdbec108f60 2024-09-10T09:59:10.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:10.154+0000 7fdbf2800640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fdbec113e90 con 0x7fdbec19eee0 2024-09-10T09:59:10.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:10.154+0000 7fdbf2800640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fdbec114190 con 0x7fdbec1047e0 2024-09-10T09:59:10.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:10.154+0000 7fdbf0a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdbec108f60 0x7fdbec19ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:10.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:10.154+0000 7fdbf1e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdbec19eee0 0x7fdbec19f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:10.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:10.154+0000 7fdbf0a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdbec108f60 0x7fdbec19ab50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59656/0 (socket says 172.21.15.5:59656) 2024-09-10T09:59:10.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:10.154+0000 7fdbf0a00640 1 -- 172.21.15.5:0/2379561573 learned_addr learned my addr 172.21.15.5:0/2379561573 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:59:10.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:10.155+0000 7fdbf0a00640 1 -- 172.21.15.5:0/2379561573 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdbec1047e0 msgr2=0x7fdbec19a610 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-10T09:59:10.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:10.155+0000 7fdbf0a00640 1 --2- 172.21.15.5:0/2379561573 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdbec1047e0 0x7fdbec19a610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:10.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:10.155+0000 7fdbf0a00640 1 -- 172.21.15.5:0/2379561573 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdbec19eee0 msgr2=0x7fdbec19f390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:10.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:10.155+0000 7fdbf0a00640 1 --2- 172.21.15.5:0/2379561573 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdbec19eee0 0x7fdbec19f390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:10.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:10.155+0000 7fdbf0a00640 1 -- 172.21.15.5:0/2379561573 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbec19f9a0 con 0x7fdbec108f60 2024-09-10T09:59:10.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:10.156+0000 7fdbf0a00640 1 --2- 172.21.15.5:0/2379561573 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdbec108f60 0x7fdbec19ab50 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7fdbd800d950 tx=0x7fdbd800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:10.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:10.156+0000 7fdbea000640 1 -- 172.21.15.5:0/2379561573 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbd8017070 con 0x7fdbec108f60 2024-09-10T09:59:10.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:10.156+0000 7fdbea000640 1 -- 172.21.15.5:0/2379561573 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdbd8009c30 con 0x7fdbec108f60 2024-09-10T09:59:10.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:10.156+0000 7fdbea000640 1 -- 172.21.15.5:0/2379561573 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbd800f6b0 con 0x7fdbec108f60 2024-09-10T09:59:10.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:10.158+0000 7fdbf2800640 1 -- 172.21.15.5:0/2379561573 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdbec19fc90 con 0x7fdbec108f60 2024-09-10T09:59:10.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:10.159+0000 7fdbf2800640 1 -- 172.21.15.5:0/2379561573 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdbec1a7420 con 0x7fdbec108f60 2024-09-10T09:59:10.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:10.164+0000 7fdbea000640 1 -- 172.21.15.5:0/2379561573 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7fdbd80109d0 con 0x7fdbec108f60 2024-09-10T09:59:10.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:10.166+0000 7fdbea000640 1 --2- 172.21.15.5:0/2379561573 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fdbd40a2cd0 0x7fdbd40a5190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:10.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:10.167+0000 7fdbea000640 1 -- 172.21.15.5:0/2379561573 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fdbd80ceab0 con 0x7fdbec108f60 2024-09-10T09:59:10.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:10.167+0000 7fdbf683d740 1 Processor -- start 2024-09-10T09:59:10.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:10.167+0000 7fdbf683d740 1 -- start start 2024-09-10T09:59:10.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:10.167+0000 7fdbf683d740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556a6a424650 0x556a6a424a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:10.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:10.167+0000 7fdbf683d740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556a6a424f70 0x556a6a42f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:10.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:10.167+0000 7fdbf683d740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556a6a42f830 0x556a6a431c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:10.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:10.167+0000 7fdbf683d740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x556a6a3823e0 con 0x556a6a42f830 2024-09-10T09:59:10.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:10.167+0000 7fdbf683d740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x556a6a329d80 con 0x556a6a424650 2024-09-10T09:59:10.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:10.167+0000 7fdbf683d740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x556a6a37e3a0 con 0x556a6a424f70 2024-09-10T09:59:10.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:10.168+0000 7fdbf1400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556a6a424650 0x556a6a424a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:10.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:10.168+0000 7fdbf1400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556a6a424650 0x556a6a424a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:47120/0 (socket says 172.21.15.5:47120) 2024-09-10T09:59:10.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:10.168+0000 7fdbf1e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556a6a42f830 0x556a6a431c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:10.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:10.168+0000 7fdbf1400640 1 -- 172.21.15.5:0/3449762454 learned_addr learned my addr 172.21.15.5:0/3449762454 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:59:10.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:10.168+0000 7fdbf0a00640 1 --2- 172.21.15.5:0/3449762454 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556a6a424f70 0x556a6a42f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:10.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:10.169+0000 7fdbf0a00640 1 -- 172.21.15.5:0/3449762454 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556a6a424650 msgr2=0x556a6a424a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:59:10.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:10.169+0000 7fdbf1400640 1 --2- 172.21.15.5:0/2379561573 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fdbd40a2cd0 0x7fdbd40a5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:10.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:10.170+0000 7fdbf0a00640 1 --2- 172.21.15.5:0/3449762454 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556a6a424650 0x556a6a424a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:10.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:10.170+0000 7fdbf0a00640 1 -- 172.21.15.5:0/3449762454 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556a6a42f830 msgr2=0x556a6a431c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:59:10.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:10.170+0000 7fdbf0a00640 1 --2- 172.21.15.5:0/3449762454 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556a6a42f830 0x556a6a431c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:10.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:10.170+0000 7fdbf0a00640 1 -- 172.21.15.5:0/3449762454 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556a6a348ae0 con 0x556a6a424f70 2024-09-10T09:59:10.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:10.170+0000 7fdbf1400640 1 --2- 172.21.15.5:0/3449762454 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556a6a424650 0x556a6a424a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:59:10.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:10.170+0000 7fdbf1e00640 1 --2- 172.21.15.5:0/3449762454 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556a6a42f830 0x556a6a431c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:59:10.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:10.171+0000 7fdbf1400640 1 --2- 172.21.15.5:0/2379561573 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fdbd40a2cd0 0x7fdbd40a5190 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fdbe4004af0 tx=0x7fdbe4000f40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:10.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:10.181+0000 7fdbf0a00640 1 --2- 172.21.15.5:0/3449762454 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556a6a424f70 0x556a6a42f2f0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fdbd80cc210 tx=0x7fdbd8010d70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2ac3e6e81bca263a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:10.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:10.181+0000 7fdbde600640 1 -- 172.21.15.5:0/3449762454 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbd80b3070 con 0x556a6a424f70 2024-09-10T09:59:10.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:10.182+0000 7fdbde600640 1 -- 172.21.15.5:0/3449762454 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdbd80a3c10 con 0x556a6a424f70 2024-09-10T09:59:10.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:10.182+0000 7fdbde600640 1 -- 172.21.15.5:0/3449762454 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbd8096cf0 con 0x556a6a424f70 2024-09-10T09:59:10.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:10.185+0000 7fdbf683d740 1 -- 172.21.15.5:0/3449762454 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556a6a424f70 msgr2=0x556a6a42f2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:10.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:10.185+0000 7fdbf683d740 1 --2- 172.21.15.5:0/3449762454 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556a6a424f70 0x556a6a42f2f0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fdbd80cc210 tx=0x7fdbd8010d70 comp rx=0 tx=0).stop 2024-09-10T09:59:10.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:10.186+0000 7fdbf683d740 1 -- 172.21.15.5:0/3449762454 shutdown_connections 2024-09-10T09:59:10.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:10.186+0000 7fdbf683d740 1 --2- 172.21.15.5:0/3449762454 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556a6a42f830 0x556a6a431c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:10.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:10.186+0000 7fdbf683d740 1 --2- 172.21.15.5:0/3449762454 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556a6a424f70 0x556a6a42f2f0 secure :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fdbd80cc210 tx=0x7fdbd8010d70 comp rx=0 tx=0).stop 2024-09-10T09:59:10.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:10.186+0000 7fdbf683d740 1 --2- 172.21.15.5:0/3449762454 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556a6a424650 0x556a6a424a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:10.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:10.186+0000 7fdbf683d740 1 -- 172.21.15.5:0/3449762454 >> 172.21.15.5:0/3449762454 conn(0x556a6a420be0 msgr2=0x556a6a420fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:10.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:10.187+0000 7fdbf683d740 1 -- 172.21.15.5:0/3449762454 shutdown_connections 2024-09-10T09:59:10.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:10.188+0000 7fdbf683d740 1 -- 172.21.15.5:0/3449762454 wait complete. 2024-09-10T09:59:10.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:10.189+0000 7fdbf683d740 1 -- start start 2024-09-10T09:59:10.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:10.190+0000 7fdbf683d740 1 Processor -- start 2024-09-10T09:59:10.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:10.191+0000 7fdbf683d740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556a6a424650 0x556a6a42c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:10.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:10.191+0000 7fdbf683d740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556a6a42f830 0x556a6a42cf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:10.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:10.191+0000 7fdbf683d740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556a6a42d4c0 0x556a6a43b870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:10.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:10.191+0000 7fdbf683d740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x556a6a432400 con 0x556a6a42f830 2024-09-10T09:59:10.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:10.191+0000 7fdbf683d740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x556a6a33d0f0 con 0x556a6a424650 2024-09-10T09:59:10.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:10.191+0000 7fdbf683d740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x556a6a432580 con 0x556a6a42d4c0 2024-09-10T09:59:10.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:10.191+0000 7fdbf683d740 10 client.0 _ll_register_callbacks cb 0x556a6a324bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:59:10.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:10.191+0000 7fdbf1400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556a6a424650 0x556a6a42c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:10.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:10.191+0000 7fdbf1400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556a6a424650 0x556a6a42c9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:47124/0 (socket says 172.21.15.5:47124) 2024-09-10T09:59:10.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:10.191+0000 7fdbf1400640 1 -- 172.21.15.5:0/85443762 learned_addr learned my addr 172.21.15.5:0/85443762 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:59:10.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:10.192+0000 7fdbf1400640 1 -- 172.21.15.5:0/85443762 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556a6a42d4c0 msgr2=0x556a6a43b870 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T09:59:10.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:10.192+0000 7fdbf1400640 1 --2- 172.21.15.5:0/85443762 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556a6a42d4c0 0x556a6a43b870 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:10.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:10.192+0000 7fdbf1400640 1 -- 172.21.15.5:0/85443762 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556a6a42f830 msgr2=0x556a6a42cf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:59:10.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:10.192+0000 7fdbf0a00640 1 --2- 172.21.15.5:0/85443762 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556a6a42f830 0x556a6a42cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:10.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:10.192+0000 7fdbf1400640 1 --2- 172.21.15.5:0/85443762 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556a6a42f830 0x556a6a42cf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:10.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:10.192+0000 7fdbf1400640 1 -- 172.21.15.5:0/85443762 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556a6a0f33c0 con 0x556a6a424650 2024-09-10T09:59:10.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:10.192+0000 7fdbf0a00640 1 --2- 172.21.15.5:0/85443762 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556a6a42f830 0x556a6a42cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T09:59:10.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:10.193+0000 7fdbd2a00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-09-10T09:59:10.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:10.193+0000 7fdbf1400640 1 --2- 172.21.15.5:0/85443762 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556a6a424650 0x556a6a42c9f0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fdbe4003ba0 tx=0x7fdbe40095c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3797641a09fcf9d6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:10.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:10.193+0000 7fdbd2a00640 1 -- 172.21.15.5:0/85443762 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbe404d020 con 0x556a6a424650 2024-09-10T09:59:10.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:10.193+0000 7fdbd2a00640 1 -- 172.21.15.5:0/85443762 <== mon.2 v2:172.21.15.102:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fdbe4030b30 con 0x556a6a424650 2024-09-10T09:59:10.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:10.194+0000 7fdbf683d740 1 -- 172.21.15.5:0/85443762 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdbd80a3c10 con 0x556a6a424650 2024-09-10T09:59:10.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:10.194+0000 7fdbd2a00640 1 -- 172.21.15.5:0/85443762 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7fdbe4030d30 con 0x556a6a424650 2024-09-10T09:59:10.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:10.194+0000 7fdbf683d740 10 client.29417 fetch_fsmap learned FSMap version 42 2024-09-10T09:59:10.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:10.194+0000 7fdbf683d740 1 -- 172.21.15.5:0/85443762 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x556a6a44fb00 con 0x556a6a424650 2024-09-10T09:59:10.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:10.195+0000 7fdbd2a00640 1 -- 172.21.15.5:0/85443762 <== mon.2 v2:172.21.15.102:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdbd80a3c10 con 0x556a6a424650 2024-09-10T09:59:10.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:10.196+0000 7fdbd2a00640 1 -- 172.21.15.5:0/85443762 <== mon.2 v2:172.21.15.102:3300/0 5 ==== fsmap(e 42) ==== 8370+0+0 (secure 0 0 0) 0x7fdbe403a8c0 con 0x556a6a424650 2024-09-10T09:59:10.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:10.196+0000 7fdbf683d740 10 client.29417 fetch_fsmap finished waiting for FSMap version 42 2024-09-10T09:59:10.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:10.196+0000 7fdbf683d740 10 client.29417 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-09-10T09:59:10.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:10.196+0000 7fdbf683d740 20 client.29417 populate_metadata read hostname 'smithi005' 2024-09-10T09:59:10.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:10.196+0000 7fdbf683d740 1 --2- 172.21.15.5:0/85443762 >> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] conn(0x556a6a450fc0 0x556a6a453450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:10.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:10.196+0000 7fdbf683d740 4 client.29417 mds_command: new command op to 24443 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:59:10.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:10.196+0000 7fdbf683d740 1 -- 172.21.15.5:0/85443762 --> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556a6a38f950 con 0x556a6a450fc0 2024-09-10T09:59:10.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:10.199+0000 7fdbf0a00640 1 --2- 172.21.15.5:0/85443762 >> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] conn(0x556a6a450fc0 0x556a6a453450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:10.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:10.200+0000 7fdbd2a00640 10 client.29417 ms_handle_connect on v2:172.21.15.102:6832/2457007427 2024-09-10T09:59:10.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:10.200+0000 7fdbf0a00640 1 --2- 172.21.15.5:0/85443762 >> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] conn(0x556a6a450fc0 0x556a6a453450 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=301292826657d5d5 server_cookie=9105efaf75607788 in_seq=0 out_seq=0 2024-09-10T09:59:10.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:10.201+0000 7fdbd2a00640 1 -- 172.21.15.5:0/85443762 <== mds.4 v2:172.21.15.102:6832/2457007427 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556a6a38f950 con 0x556a6a450fc0 2024-09-10T09:59:10.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:10.201+0000 7fdbd2a00640 10 client.29417 handle_command_reply: tid=0 2024-09-10T09:59:10.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:10.201+0000 7fdbf683d740 1 client.29417 shutdown 2024-09-10T09:59:10.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:10.202+0000 7fdbf683d740 1 -- 172.21.15.5:0/85443762 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556a6a424650 msgr2=0x556a6a42c9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:10.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:10.202+0000 7fdbf683d740 1 --2- 172.21.15.5:0/85443762 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556a6a424650 0x556a6a42c9f0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fdbe4003ba0 tx=0x7fdbe40095c0 comp rx=0 tx=0).stop 2024-09-10T09:59:10.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:10.202+0000 7fdbf683d740 1 -- 172.21.15.5:0/85443762 shutdown_connections 2024-09-10T09:59:10.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:10.202+0000 7fdbf683d740 1 --2- 172.21.15.5:0/85443762 >> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] conn(0x556a6a450fc0 0x556a6a453450 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:10.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:10.202+0000 7fdbf683d740 1 --2- 172.21.15.5:0/85443762 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556a6a42d4c0 0x556a6a43b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:10.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:10.202+0000 7fdbf683d740 1 --2- 172.21.15.5:0/85443762 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556a6a42f830 0x556a6a42cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:10.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:10.202+0000 7fdbf683d740 1 --2- 172.21.15.5:0/85443762 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556a6a424650 0x556a6a42c9f0 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:10.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:10.202+0000 7fdbf683d740 1 -- 172.21.15.5:0/85443762 >> 172.21.15.5:0/85443762 conn(0x556a6a420be0 msgr2=0x556a6a4174b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:10.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:10.203+0000 7fdbd2a00640 0 client.29417 ms_handle_reset on v2:172.21.15.102:6832/2457007427 2024-09-10T09:59:10.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:10.203+0000 7fdbf683d740 1 -- 172.21.15.5:0/85443762 shutdown_connections 2024-09-10T09:59:10.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:10.203+0000 7fdbf683d740 1 -- 172.21.15.5:0/85443762 wait complete. 2024-09-10T09:59:10.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:10.204+0000 7fdbf683d740 20 client.29417 trim_cache size 0 max 16384 2024-09-10T09:59:10.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:10.224+0000 7fdbf683d740 1 Processor -- start 2024-09-10T09:59:10.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:10.224+0000 7fdbf683d740 1 -- start start 2024-09-10T09:59:10.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:10.225+0000 7fdbf683d740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556a6a42d4c0 0x556a6a427100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:10.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:10.225+0000 7fdbf683d740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556a6a42f830 0x556a6a43b660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:10.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:10.225+0000 7fdbf683d740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556a6a428300 0x556a6a458d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:10.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:10.225+0000 7fdbf683d740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fdbec071620 con 0x556a6a428300 2024-09-10T09:59:10.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:10.225+0000 7fdbf683d740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fdbec073160 con 0x556a6a42f830 2024-09-10T09:59:10.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:10.225+0000 7fdbf683d740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fdbec071cf0 con 0x556a6a42d4c0 2024-09-10T09:59:10.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:10.225+0000 7fdbf0a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556a6a42f830 0x556a6a43b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:10.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:10.225+0000 7fdbf1e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556a6a428300 0x556a6a458d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:10.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:10.225+0000 7fdbf1e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556a6a428300 0x556a6a458d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59674/0 (socket says 172.21.15.5:59674) 2024-09-10T09:59:10.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:10.225+0000 7fdbf1e00640 1 -- 172.21.15.5:0/3253690785 learned_addr learned my addr 172.21.15.5:0/3253690785 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:59:10.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:10.226+0000 7fdbf1e00640 1 -- 172.21.15.5:0/3253690785 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556a6a42d4c0 msgr2=0x556a6a427100 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T09:59:10.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:10.226+0000 7fdbf1e00640 1 --2- 172.21.15.5:0/3253690785 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556a6a42d4c0 0x556a6a427100 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:10.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:10.226+0000 7fdbf1e00640 1 -- 172.21.15.5:0/3253690785 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556a6a42f830 msgr2=0x556a6a43b660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:59:10.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:10.226+0000 7fdbf1e00640 1 --2- 172.21.15.5:0/3253690785 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556a6a42f830 0x556a6a43b660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:10.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:10.226+0000 7fdbf1e00640 1 -- 172.21.15.5:0/3253690785 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556a6a42af40 con 0x556a6a428300 2024-09-10T09:59:10.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:10.226+0000 7fdbf0a00640 1 --2- 172.21.15.5:0/3253690785 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556a6a42f830 0x556a6a43b660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:59:10.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:10.226+0000 7fdbf1e00640 1 --2- 172.21.15.5:0/3253690785 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556a6a428300 0x556a6a458d20 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7fdbe000aad0 tx=0x7fdbe000af90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cae819b96c4aa2b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:10.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:10.227+0000 7fdbddc00640 1 -- 172.21.15.5:0/3253690785 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbe0004050 con 0x556a6a428300 2024-09-10T09:59:10.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:10.227+0000 7fdbddc00640 1 -- 172.21.15.5:0/3253690785 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdbe00041f0 con 0x556a6a428300 2024-09-10T09:59:10.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:10.227+0000 7fdbddc00640 1 -- 172.21.15.5:0/3253690785 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbe001b8e0 con 0x556a6a428300 2024-09-10T09:59:10.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:10.227+0000 7fdbf683d740 1 -- 172.21.15.5:0/3253690785 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556a6a428300 msgr2=0x556a6a458d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:10.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:10.227+0000 7fdbf683d740 1 --2- 172.21.15.5:0/3253690785 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556a6a428300 0x556a6a458d20 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7fdbe000aad0 tx=0x7fdbe000af90 comp rx=0 tx=0).stop 2024-09-10T09:59:10.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:10.227+0000 7fdbf683d740 1 -- 172.21.15.5:0/3253690785 shutdown_connections 2024-09-10T09:59:10.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:10.227+0000 7fdbf683d740 1 --2- 172.21.15.5:0/3253690785 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556a6a428300 0x556a6a458d20 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:10.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:10.227+0000 7fdbf683d740 1 --2- 172.21.15.5:0/3253690785 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556a6a42f830 0x556a6a43b660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:10.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:10.227+0000 7fdbf683d740 1 --2- 172.21.15.5:0/3253690785 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556a6a42d4c0 0x556a6a427100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:10.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:10.228+0000 7fdbf683d740 1 -- 172.21.15.5:0/3253690785 >> 172.21.15.5:0/3253690785 conn(0x556a6a420be0 msgr2=0x556a6a427bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:10.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:10.228+0000 7fdbf683d740 1 -- 172.21.15.5:0/3253690785 shutdown_connections 2024-09-10T09:59:10.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:10.228+0000 7fdbf683d740 1 -- 172.21.15.5:0/3253690785 wait complete. 2024-09-10T09:59:10.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:10.228+0000 7fdbf683d740 1 -- start start 2024-09-10T09:59:10.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:10.228+0000 7fdbf683d740 1 Processor -- start 2024-09-10T09:59:10.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:10.229+0000 7fdbf683d740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556a6a428300 0x556a6a433930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:10.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:10.229+0000 7fdbf683d740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556a6a42d4c0 0x556a6a433e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:10.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:10.229+0000 7fdbf683d740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556a6a42f830 0x556a6a4bacf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:10.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:10.229+0000 7fdbf683d740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fdbec072a90 con 0x556a6a42d4c0 2024-09-10T09:59:10.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:10.229+0000 7fdbf683d740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fdbec0723c0 con 0x556a6a42f830 2024-09-10T09:59:10.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:10.229+0000 7fdbf683d740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fdbec06f600 con 0x556a6a428300 2024-09-10T09:59:10.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:10.229+0000 7fdbf683d740 10 client.0 _ll_register_callbacks cb 0x7fdbe4009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:59:10.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:10.230+0000 7fdbf0a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556a6a42d4c0 0x556a6a433e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:10.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:10.230+0000 7fdbf0a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556a6a42d4c0 0x556a6a433e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59678/0 (socket says 172.21.15.5:59678) 2024-09-10T09:59:10.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:10.230+0000 7fdbf0a00640 1 -- 172.21.15.5:0/3508340456 learned_addr learned my addr 172.21.15.5:0/3508340456 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:59:10.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:10.230+0000 7fdbf1e00640 1 --2- 172.21.15.5:0/3508340456 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556a6a42f830 0x556a6a4bacf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:10.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:10.230+0000 7fdbf1e00640 1 -- 172.21.15.5:0/3508340456 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556a6a428300 msgr2=0x556a6a433930 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T09:59:10.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:10.230+0000 7fdbf1e00640 1 --2- 172.21.15.5:0/3508340456 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556a6a428300 0x556a6a433930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:10.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:10.230+0000 7fdbf1e00640 1 -- 172.21.15.5:0/3508340456 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556a6a42d4c0 msgr2=0x556a6a433e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:59:10.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:10.230+0000 7fdbf1e00640 1 --2- 172.21.15.5:0/3508340456 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556a6a42d4c0 0x556a6a433e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:10.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:10.230+0000 7fdbf1e00640 1 -- 172.21.15.5:0/3508340456 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556a6a4bb230 con 0x556a6a42f830 2024-09-10T09:59:10.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:10.231+0000 7fdbf1e00640 1 --2- 172.21.15.5:0/3508340456 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556a6a42f830 0x556a6a4bacf0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7fdbe0003fd0 tx=0x7fdbe00058f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1c05885cee616047 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:10.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:10.231+0000 7fdbd2a00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-09-10T09:59:10.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:10.232+0000 7fdbd2a00640 1 -- 172.21.15.5:0/3508340456 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbe0003ad0 con 0x556a6a42f830 2024-09-10T09:59:10.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:10.232+0000 7fdbd2a00640 1 -- 172.21.15.5:0/3508340456 <== mon.2 v2:172.21.15.102:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fdbe0005040 con 0x556a6a42f830 2024-09-10T09:59:10.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:10.232+0000 7fdbf683d740 1 -- 172.21.15.5:0/3508340456 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdbe00041f0 con 0x556a6a42f830 2024-09-10T09:59:10.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:10.233+0000 7fdbd2a00640 1 -- 172.21.15.5:0/3508340456 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7fdbe0005240 con 0x556a6a42f830 2024-09-10T09:59:10.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:10.233+0000 7fdbf683d740 10 client.29423 fetch_fsmap learned FSMap version 42 2024-09-10T09:59:10.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:10.233+0000 7fdbf683d740 1 -- 172.21.15.5:0/3508340456 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x556a6a4bc3d0 con 0x556a6a42f830 2024-09-10T09:59:10.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:10.234+0000 7fdbd2a00640 1 -- 172.21.15.5:0/3508340456 <== mon.2 v2:172.21.15.102:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdbe00041f0 con 0x556a6a42f830 2024-09-10T09:59:10.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:10.234+0000 7fdbd2a00640 1 -- 172.21.15.5:0/3508340456 <== mon.2 v2:172.21.15.102:3300/0 5 ==== fsmap(e 42) ==== 8370+0+0 (secure 0 0 0) 0x7fdbe0019020 con 0x556a6a42f830 2024-09-10T09:59:10.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:10.234+0000 7fdbf683d740 10 client.29423 fetch_fsmap finished waiting for FSMap version 42 2024-09-10T09:59:10.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:10.234+0000 7fdbf683d740 10 client.29423 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-09-10T09:59:10.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:10.234+0000 7fdbf683d740 20 client.29423 populate_metadata read hostname 'smithi005' 2024-09-10T09:59:10.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:10.234+0000 7fdbf683d740 1 --2- 172.21.15.5:0/3508340456 >> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] conn(0x556a6a4bc940 0x556a6a4bed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:10.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:10.234+0000 7fdbf683d740 4 client.29423 mds_command: new command op to 24443 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T09:59:10.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:10.234+0000 7fdbf683d740 1 -- 172.21.15.5:0/3508340456 --> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556a6a389e60 con 0x556a6a4bc940 2024-09-10T09:59:10.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:10.235+0000 7fdbf1400640 1 --2- 172.21.15.5:0/3508340456 >> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] conn(0x556a6a4bc940 0x556a6a4bed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:10.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:10.235+0000 7fdbd2a00640 10 client.29423 ms_handle_connect on v2:172.21.15.102:6832/2457007427 2024-09-10T09:59:10.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:10.235+0000 7fdbf1400640 1 --2- 172.21.15.5:0/3508340456 >> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] conn(0x556a6a4bc940 0x556a6a4bed60 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=2fe44f792caa9866 server_cookie=119e00d3f2190d01 in_seq=0 out_seq=0 2024-09-10T09:59:10.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:10.235+0000 7fdbd2a00640 1 -- 172.21.15.5:0/3508340456 <== mds.4 v2:172.21.15.102:6832/2457007427 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556a6a389e60 con 0x556a6a4bc940 2024-09-10T09:59:10.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:10.236+0000 7fdbd2a00640 10 client.29423 handle_command_reply: tid=0 2024-09-10T09:59:10.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:10.236+0000 7fdbf683d740 1 client.29423 shutdown 2024-09-10T09:59:10.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:10.236+0000 7fdbf683d740 1 -- 172.21.15.5:0/3508340456 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556a6a42f830 msgr2=0x556a6a4bacf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:10.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:10.236+0000 7fdbf683d740 1 --2- 172.21.15.5:0/3508340456 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556a6a42f830 0x556a6a4bacf0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7fdbe0003fd0 tx=0x7fdbe00058f0 comp rx=0 tx=0).stop 2024-09-10T09:59:10.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:10.236+0000 7fdbf683d740 1 -- 172.21.15.5:0/3508340456 shutdown_connections 2024-09-10T09:59:10.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:10.236+0000 7fdbf683d740 1 --2- 172.21.15.5:0/3508340456 >> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] conn(0x556a6a4bc940 0x556a6a4bed60 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:10.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:10.236+0000 7fdbf683d740 1 --2- 172.21.15.5:0/3508340456 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556a6a42f830 0x556a6a4bacf0 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:10.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:10.236+0000 7fdbf683d740 1 --2- 172.21.15.5:0/3508340456 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556a6a42d4c0 0x556a6a433e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:10.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:10.236+0000 7fdbf683d740 1 --2- 172.21.15.5:0/3508340456 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556a6a428300 0x556a6a433930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:10.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:10.236+0000 7fdbd2a00640 0 client.29423 ms_handle_reset on v2:172.21.15.102:6832/2457007427 2024-09-10T09:59:10.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:10.236+0000 7fdbf683d740 1 -- 172.21.15.5:0/3508340456 >> 172.21.15.5:0/3508340456 conn(0x556a6a420be0 msgr2=0x556a6a416be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:10.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:10.238+0000 7fdbf683d740 1 -- 172.21.15.5:0/3508340456 shutdown_connections 2024-09-10T09:59:10.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:10.238+0000 7fdbf683d740 1 -- 172.21.15.5:0/3508340456 wait complete. 2024-09-10T09:59:10.330 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-10T09:59:10.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:10.240+0000 7fdbf683d740 20 client.29423 trim_cache size 0 max 16384 2024-09-10T09:59:10.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:10.241+0000 7fdbf2800640 1 -- 172.21.15.5:0/2379561573 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fdbd40a2cd0 msgr2=0x7fdbd40a5190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:10.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:10.241+0000 7fdbf2800640 1 --2- 172.21.15.5:0/2379561573 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fdbd40a2cd0 0x7fdbd40a5190 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fdbe4004af0 tx=0x7fdbe4000f40 comp rx=0 tx=0).stop 2024-09-10T09:59:10.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:10.241+0000 7fdbf2800640 1 -- 172.21.15.5:0/2379561573 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdbec108f60 msgr2=0x7fdbec19ab50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:10.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:10.241+0000 7fdbf2800640 1 --2- 172.21.15.5:0/2379561573 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdbec108f60 0x7fdbec19ab50 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7fdbd800d950 tx=0x7fdbd800de10 comp rx=0 tx=0).stop 2024-09-10T09:59:10.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:10.242+0000 7fdbf2800640 1 -- 172.21.15.5:0/2379561573 shutdown_connections 2024-09-10T09:59:10.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:10.242+0000 7fdbf2800640 1 --2- 172.21.15.5:0/2379561573 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fdbd40a2cd0 0x7fdbd40a5190 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:10.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:10.242+0000 7fdbf2800640 1 --2- 172.21.15.5:0/2379561573 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdbec19eee0 0x7fdbec19f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:10.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:10.242+0000 7fdbf2800640 1 --2- 172.21.15.5:0/2379561573 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdbec108f60 0x7fdbec19ab50 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:10.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:10.242+0000 7fdbf2800640 1 --2- 172.21.15.5:0/2379561573 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdbec1047e0 0x7fdbec19a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:10.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:10.242+0000 7fdbf2800640 1 -- 172.21.15.5:0/2379561573 >> 172.21.15.5:0/2379561573 conn(0x7fdbec0ff8d0 msgr2=0x7fdbec100cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:10.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:10.242+0000 7fdbf2800640 1 -- 172.21.15.5:0/2379561573 shutdown_connections 2024-09-10T09:59:10.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:10.242+0000 7fdbf2800640 1 -- 172.21.15.5:0/2379561573 wait complete. 2024-09-10T09:59:10.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:59:09 smithi159 ceph-mon[35513]: pgmap v1017: 129 pgs: 129 active+clean; 646 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 1.3 MiB/s wr, 210 op/s 2024-09-10T09:59:11.025 INFO:tasks.workunit.client.0.smithi005.stdout: 10 110875 13.39 MB/sec execute 289 sec latency 13690.585 ms 2024-09-10T09:59:11.543 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-10T09:59:11.946 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:59:12.025 INFO:tasks.workunit.client.0.smithi005.stdout: 10 110875 13.35 MB/sec execute 290 sec latency 14690.690 ms 2024-09-10T09:59:12.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:59:11 smithi102 ceph-mon[35442]: pgmap v1018: 129 pgs: 129 active+clean; 650 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 1.8 MiB/s wr, 178 op/s 2024-09-10T09:59:12.243 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:59:11 smithi005 ceph-mon[30350]: pgmap v1018: 129 pgs: 129 active+clean; 650 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 1.8 MiB/s wr, 178 op/s 2024-09-10T09:59:12.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:59:11 smithi159 ceph-mon[35513]: pgmap v1018: 129 pgs: 129 active+clean; 650 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 1.8 MiB/s wr, 178 op/s 2024-09-10T09:59:12.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:12.519+0000 7f832ca00640 1 -- 172.21.15.5:0/4272936778 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8328112870 msgr2=0x7f8328114c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:12.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:12.519+0000 7f832ca00640 1 --2- 172.21.15.5:0/4272936778 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8328112870 0x7f8328114c60 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f8320009ca0 tx=0x7f832002a8c0 comp rx=0 tx=0).stop 2024-09-10T09:59:12.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:12.519+0000 7f832ca00640 1 -- 172.21.15.5:0/4272936778 shutdown_connections 2024-09-10T09:59:12.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:12.519+0000 7f832ca00640 1 --2- 172.21.15.5:0/4272936778 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8328112870 0x7f8328114c60 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:12.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:12.519+0000 7f832ca00640 1 --2- 172.21.15.5:0/4272936778 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8328077040 0x7f8328112330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:12.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:12.519+0000 7f832ca00640 1 --2- 172.21.15.5:0/4272936778 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8328076720 0x7f8328076b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:12.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:12.519+0000 7f832ca00640 1 -- 172.21.15.5:0/4272936778 >> 172.21.15.5:0/4272936778 conn(0x7f83280ff8d0 msgr2=0x7f8328101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:12.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:12.520+0000 7f832ca00640 1 -- 172.21.15.5:0/4272936778 shutdown_connections 2024-09-10T09:59:12.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:12.520+0000 7f832ca00640 1 -- 172.21.15.5:0/4272936778 wait complete. 2024-09-10T09:59:12.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:12.520+0000 7f832ca00640 1 Processor -- start 2024-09-10T09:59:12.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:12.521+0000 7f832ca00640 1 -- start start 2024-09-10T09:59:12.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:12.521+0000 7f832ca00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8328076720 0x7f83281a45d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:12.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:12.521+0000 7f832ca00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8328077040 0x7f83281a4b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:12.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:12.521+0000 7f832ca00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8328112870 0x7f832819e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:12.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:12.521+0000 7f832ca00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f83281183c0 con 0x7f8328112870 2024-09-10T09:59:12.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:12.521+0000 7f832ca00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f8328118240 con 0x7f8328077040 2024-09-10T09:59:12.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:12.521+0000 7f832ca00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f8328118540 con 0x7f8328076720 2024-09-10T09:59:12.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:12.521+0000 7f8327e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8328112870 0x7f832819e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:12.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:12.521+0000 7f8326a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8328077040 0x7f83281a4b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:12.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:12.521+0000 7f8326a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8328077040 0x7f83281a4b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:35774/0 (socket says 172.21.15.5:35774) 2024-09-10T09:59:12.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:12.521+0000 7f8326a00640 1 -- 172.21.15.5:0/1176903986 learned_addr learned my addr 172.21.15.5:0/1176903986 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:59:12.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:12.521+0000 7f8327e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8328112870 0x7f832819e6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60356/0 (socket says 172.21.15.5:60356) 2024-09-10T09:59:12.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:12.521+0000 7f8327400640 1 --2- 172.21.15.5:0/1176903986 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8328076720 0x7f83281a45d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:12.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:12.522+0000 7f8326a00640 1 -- 172.21.15.5:0/1176903986 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8328076720 msgr2=0x7f83281a45d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:12.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:12.522+0000 7f8326a00640 1 --2- 172.21.15.5:0/1176903986 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8328076720 0x7f83281a45d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:12.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:12.522+0000 7f8326a00640 1 -- 172.21.15.5:0/1176903986 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8328112870 msgr2=0x7f832819e6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:12.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:12.522+0000 7f8326a00640 1 --2- 172.21.15.5:0/1176903986 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8328112870 0x7f832819e6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:12.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:12.522+0000 7f8326a00640 1 -- 172.21.15.5:0/1176903986 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f832819ef10 con 0x7f8328077040 2024-09-10T09:59:12.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:12.522+0000 7f8326a00640 1 --2- 172.21.15.5:0/1176903986 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8328077040 0x7f83281a4b10 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f831400d950 tx=0x7f831400de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:12.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:12.522+0000 7f831fe00640 1 -- 172.21.15.5:0/1176903986 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8314017030 con 0x7f8328077040 2024-09-10T09:59:12.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:12.523+0000 7f832ca00640 1 -- 172.21.15.5:0/1176903986 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f832819f200 con 0x7f8328077040 2024-09-10T09:59:12.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:12.523+0000 7f831fe00640 1 -- 172.21.15.5:0/1176903986 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8314004020 con 0x7f8328077040 2024-09-10T09:59:12.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:12.523+0000 7f831fe00640 1 -- 172.21.15.5:0/1176903986 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83140047a0 con 0x7f8328077040 2024-09-10T09:59:12.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:12.523+0000 7f832ca00640 1 -- 172.21.15.5:0/1176903986 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83281ab550 con 0x7f8328077040 2024-09-10T09:59:12.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:12.526+0000 7f83309bb740 1 Processor -- start 2024-09-10T09:59:12.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:12.526+0000 7f83309bb740 1 -- start start 2024-09-10T09:59:12.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:12.526+0000 7f83309bb740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55980cd16e40 0x55980cd17220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:12.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:12.526+0000 7f83309bb740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55980cd17760 0x55980cd21ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:12.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:12.526+0000 7f83309bb740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55980cd22020 0x55980cd24410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:12.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:12.526+0000 7f83309bb740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55980cc7d3e0 con 0x55980cd16e40 2024-09-10T09:59:12.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:12.526+0000 7f83309bb740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55980cc24d80 con 0x55980cd22020 2024-09-10T09:59:12.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:12.526+0000 7f83309bb740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55980cc793a0 con 0x55980cd17760 2024-09-10T09:59:12.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:12.526+0000 7f8327e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55980cd16e40 0x55980cd17220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:12.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:12.526+0000 7f831fe00640 1 -- 172.21.15.5:0/1176903986 <== mon.2 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f83140041d0 con 0x7f8328077040 2024-09-10T09:59:12.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:12.526+0000 7f8327e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55980cd16e40 0x55980cd17220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60358/0 (socket says 172.21.15.5:60358) 2024-09-10T09:59:12.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:12.526+0000 7f8327e00640 1 -- 172.21.15.5:0/1964635602 learned_addr learned my addr 172.21.15.5:0/1964635602 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:59:12.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:12.527+0000 7f8327400640 1 --2- 172.21.15.5:0/1964635602 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55980cd17760 0x55980cd21ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:12.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:12.527+0000 7f8326a00640 1 --2- 172.21.15.5:0/1964635602 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55980cd22020 0x55980cd24410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:12.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:12.527+0000 7f8327e00640 1 -- 172.21.15.5:0/1964635602 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55980cd17760 msgr2=0x55980cd21ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:59:12.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:12.527+0000 7f8327e00640 1 --2- 172.21.15.5:0/1964635602 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55980cd17760 0x55980cd21ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:12.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:12.527+0000 7f8327e00640 1 -- 172.21.15.5:0/1964635602 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55980cd22020 msgr2=0x55980cd24410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:59:12.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:12.527+0000 7f8327e00640 1 --2- 172.21.15.5:0/1964635602 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55980cd22020 0x55980cd24410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:12.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:12.527+0000 7f8327e00640 1 -- 172.21.15.5:0/1964635602 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55980cc43ae0 con 0x55980cd16e40 2024-09-10T09:59:12.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:12.527+0000 7f8326a00640 1 --2- 172.21.15.5:0/1964635602 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55980cd22020 0x55980cd24410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T09:59:12.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:12.527+0000 7f8327e00640 1 --2- 172.21.15.5:0/1964635602 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55980cd16e40 0x55980cd17220 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f8320005e30 tx=0x7f8320009800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70d4661acda710c9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:12.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:12.528+0000 7f8313e00640 1 -- 172.21.15.5:0/1964635602 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8320031db0 con 0x55980cd16e40 2024-09-10T09:59:12.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:12.528+0000 7f8313e00640 1 -- 172.21.15.5:0/1964635602 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8320036040 con 0x55980cd16e40 2024-09-10T09:59:12.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:12.528+0000 7f8313e00640 1 -- 172.21.15.5:0/1964635602 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83200043c0 con 0x55980cd16e40 2024-09-10T09:59:12.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:12.528+0000 7f831fe00640 1 --2- 172.21.15.5:0/1176903986 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f830c0a2b80 0x7f830c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:12.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:12.529+0000 7f831fe00640 1 -- 172.21.15.5:0/1176903986 <== mon.2 v2:172.21.15.102:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f83140ced20 con 0x7f8328077040 2024-09-10T09:59:12.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:12.529+0000 7f83309bb740 1 -- 172.21.15.5:0/1964635602 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55980cd16e40 msgr2=0x55980cd17220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:12.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:12.529+0000 7f83309bb740 1 --2- 172.21.15.5:0/1964635602 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55980cd16e40 0x55980cd17220 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f8320005e30 tx=0x7f8320009800 comp rx=0 tx=0).stop 2024-09-10T09:59:12.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:12.529+0000 7f8327400640 1 --2- 172.21.15.5:0/1176903986 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f830c0a2b80 0x7f830c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:12.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:12.529+0000 7f83309bb740 1 -- 172.21.15.5:0/1964635602 shutdown_connections 2024-09-10T09:59:12.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:12.529+0000 7f83309bb740 1 --2- 172.21.15.5:0/1964635602 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55980cd22020 0x55980cd24410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:12.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:12.529+0000 7f83309bb740 1 --2- 172.21.15.5:0/1964635602 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55980cd17760 0x55980cd21ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:12.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:12.529+0000 7f83309bb740 1 --2- 172.21.15.5:0/1964635602 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55980cd16e40 0x55980cd17220 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:12.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:12.529+0000 7f83309bb740 1 -- 172.21.15.5:0/1964635602 >> 172.21.15.5:0/1964635602 conn(0x55980cd133d0 msgr2=0x55980cd137b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:12.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:12.529+0000 7f83309bb740 1 -- 172.21.15.5:0/1964635602 shutdown_connections 2024-09-10T09:59:12.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:12.529+0000 7f83309bb740 1 -- 172.21.15.5:0/1964635602 wait complete. 2024-09-10T09:59:12.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:12.530+0000 7f8327400640 1 --2- 172.21.15.5:0/1176903986 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f830c0a2b80 0x7f830c0a5040 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x55980cd14230 tx=0x7f8318005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:12.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:12.530+0000 7f83309bb740 1 -- start start 2024-09-10T09:59:12.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:12.530+0000 7f83309bb740 1 Processor -- start 2024-09-10T09:59:12.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:12.531+0000 7f83309bb740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55980cd16e40 0x55980cd1ef80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:12.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:12.531+0000 7f83309bb740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55980cd17760 0x55980cd1f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:12.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:12.531+0000 7f83309bb740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55980cd22020 0x55980cd1fa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:12.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:12.531+0000 7f83309bb740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55980cd24a70 con 0x55980cd17760 2024-09-10T09:59:12.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:12.531+0000 7f83309bb740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55980cc380f0 con 0x55980cd16e40 2024-09-10T09:59:12.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:12.531+0000 7f83309bb740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55980cd24bf0 con 0x55980cd22020 2024-09-10T09:59:12.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:12.531+0000 7f83309bb740 10 client.0 _ll_register_callbacks cb 0x55980cc1fbd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:59:12.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:12.531+0000 7f8327400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55980cd16e40 0x55980cd1ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:12.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:12.531+0000 7f8327400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55980cd16e40 0x55980cd1ef80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:35798/0 (socket says 172.21.15.5:35798) 2024-09-10T09:59:12.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:12.531+0000 7f8327400640 1 -- 172.21.15.5:0/1146396402 learned_addr learned my addr 172.21.15.5:0/1146396402 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:59:12.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:12.531+0000 7f8327e00640 1 --2- 172.21.15.5:0/1146396402 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55980cd22020 0x55980cd1fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:12.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:12.531+0000 7f8326a00640 1 --2- 172.21.15.5:0/1146396402 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55980cd17760 0x55980cd1f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:12.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:12.531+0000 7f8327e00640 1 -- 172.21.15.5:0/1146396402 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55980cd16e40 msgr2=0x55980cd1ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:59:12.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:12.531+0000 7f8327e00640 1 --2- 172.21.15.5:0/1146396402 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55980cd16e40 0x55980cd1ef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:12.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:12.531+0000 7f8327e00640 1 -- 172.21.15.5:0/1146396402 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55980cd17760 msgr2=0x55980cd1f4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:59:12.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:12.531+0000 7f8327e00640 1 --2- 172.21.15.5:0/1146396402 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55980cd17760 0x55980cd1f4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:12.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:12.532+0000 7f8327e00640 1 -- 172.21.15.5:0/1146396402 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55980c9ee3c0 con 0x55980cd22020 2024-09-10T09:59:12.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:12.532+0000 7f8327400640 1 --2- 172.21.15.5:0/1146396402 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55980cd16e40 0x55980cd1ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:59:12.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:12.532+0000 7f8327e00640 1 --2- 172.21.15.5:0/1146396402 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55980cd22020 0x55980cd1fa00 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f8320009bf0 tx=0x7f8320031700 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cc5737b6b2bbe058 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:12.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:12.532+0000 7f8310c00640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-09-10T09:59:12.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:12.532+0000 7f8310c00640 1 -- 172.21.15.5:0/1146396402 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8320003b80 con 0x55980cd22020 2024-09-10T09:59:12.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:12.532+0000 7f83309bb740 1 -- 172.21.15.5:0/1146396402 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8320036040 con 0x55980cd22020 2024-09-10T09:59:12.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:12.532+0000 7f8310c00640 1 -- 172.21.15.5:0/1146396402 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f8320003d20 con 0x55980cd22020 2024-09-10T09:59:12.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:12.533+0000 7f8310c00640 1 -- 172.21.15.5:0/1146396402 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f832005b050 con 0x55980cd22020 2024-09-10T09:59:12.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:12.533+0000 7f83309bb740 10 client.29407 fetch_fsmap learned FSMap version 42 2024-09-10T09:59:12.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:12.533+0000 7f83309bb740 1 -- 172.21.15.5:0/1146396402 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55980cd24a70 con 0x55980cd22020 2024-09-10T09:59:12.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:12.533+0000 7f8310c00640 1 -- 172.21.15.5:0/1146396402 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8320036040 con 0x55980cd22020 2024-09-10T09:59:12.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:12.533+0000 7f8310c00640 1 -- 172.21.15.5:0/1146396402 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 42) ==== 8370+0+0 (secure 0 0 0) 0x7f832004c070 con 0x55980cd22020 2024-09-10T09:59:12.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:12.534+0000 7f83309bb740 10 client.29407 fetch_fsmap finished waiting for FSMap version 42 2024-09-10T09:59:12.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:12.534+0000 7f83309bb740 10 client.29407 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-09-10T09:59:12.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:12.534+0000 7f83309bb740 20 client.29407 populate_metadata read hostname 'smithi005' 2024-09-10T09:59:12.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:12.534+0000 7f83309bb740 1 --2- 172.21.15.5:0/1146396402 >> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] conn(0x55980cd31c20 0x55980cd34010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:12.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:12.534+0000 7f83309bb740 4 client.29407 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:59:12.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:12.534+0000 7f83309bb740 1 -- 172.21.15.5:0/1146396402 --> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55980cc8a950 con 0x55980cd31c20 2024-09-10T09:59:12.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:12.534+0000 7f8327400640 1 --2- 172.21.15.5:0/1146396402 >> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] conn(0x55980cd31c20 0x55980cd34010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:12.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:12.535+0000 7f8327400640 1 --2- 172.21.15.5:0/1146396402 >> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] conn(0x55980cd31c20 0x55980cd34010 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=ecf8ed1fe150f8c server_cookie=d3171bef57fd0abf in_seq=0 out_seq=0 2024-09-10T09:59:12.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:12.535+0000 7f8310c00640 10 client.29407 ms_handle_connect on v2:172.21.15.102:6836/2689776764 2024-09-10T09:59:12.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:12.536+0000 7f8310c00640 1 -- 172.21.15.5:0/1146396402 <== mds.3 v2:172.21.15.102:6836/2689776764 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55980cc8a950 con 0x55980cd31c20 2024-09-10T09:59:12.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:12.536+0000 7f8310c00640 10 client.29407 handle_command_reply: tid=0 2024-09-10T09:59:12.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:12.536+0000 7f83309bb740 1 client.29407 shutdown 2024-09-10T09:59:12.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:12.536+0000 7f83309bb740 1 -- 172.21.15.5:0/1146396402 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55980cd22020 msgr2=0x55980cd1fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:12.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:12.536+0000 7f83309bb740 1 --2- 172.21.15.5:0/1146396402 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55980cd22020 0x55980cd1fa00 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f8320009bf0 tx=0x7f8320031700 comp rx=0 tx=0).stop 2024-09-10T09:59:12.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:12.536+0000 7f83309bb740 1 -- 172.21.15.5:0/1146396402 shutdown_connections 2024-09-10T09:59:12.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:12.536+0000 7f83309bb740 1 --2- 172.21.15.5:0/1146396402 >> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] conn(0x55980cd31c20 0x55980cd34010 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:12.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:12.536+0000 7f83309bb740 1 --2- 172.21.15.5:0/1146396402 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55980cd22020 0x55980cd1fa00 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:12.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:12.536+0000 7f83309bb740 1 --2- 172.21.15.5:0/1146396402 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55980cd17760 0x55980cd1f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:12.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:12.536+0000 7f83309bb740 1 --2- 172.21.15.5:0/1146396402 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55980cd16e40 0x55980cd1ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:12.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:12.536+0000 7f8310c00640 0 client.29407 ms_handle_reset on v2:172.21.15.102:6836/2689776764 2024-09-10T09:59:12.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:12.536+0000 7f83309bb740 1 -- 172.21.15.5:0/1146396402 >> 172.21.15.5:0/1146396402 conn(0x55980cd133d0 msgr2=0x55980cd124b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:12.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:12.536+0000 7f83309bb740 1 -- 172.21.15.5:0/1146396402 shutdown_connections 2024-09-10T09:59:12.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:12.536+0000 7f83309bb740 1 -- 172.21.15.5:0/1146396402 wait complete. 2024-09-10T09:59:12.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:12.537+0000 7f83309bb740 20 client.29407 trim_cache size 0 max 16384 2024-09-10T09:59:12.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:12.551+0000 7f83309bb740 1 Processor -- start 2024-09-10T09:59:12.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:12.551+0000 7f83309bb740 1 -- start start 2024-09-10T09:59:12.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:12.552+0000 7f83309bb740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55980cd16e40 0x55980cd39890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:12.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:12.552+0000 7f83309bb740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55980cd17760 0x55980cd39dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:12.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:12.552+0000 7f83309bb740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55980cd22020 0x55980cd1d720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:12.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:12.552+0000 7f83309bb740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f83281b5230 con 0x55980cd17760 2024-09-10T09:59:12.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:12.552+0000 7f83309bb740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f83281b6d70 con 0x55980cd16e40 2024-09-10T09:59:12.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:12.552+0000 7f83309bb740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f83281b5900 con 0x55980cd22020 2024-09-10T09:59:12.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:12.552+0000 7f8326a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55980cd17760 0x55980cd39dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:12.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:12.552+0000 7f8326a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55980cd17760 0x55980cd39dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60372/0 (socket says 172.21.15.5:60372) 2024-09-10T09:59:12.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:12.552+0000 7f8326a00640 1 -- 172.21.15.5:0/874454437 learned_addr learned my addr 172.21.15.5:0/874454437 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:59:12.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:12.552+0000 7f8327400640 1 --2- 172.21.15.5:0/874454437 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55980cd16e40 0x55980cd39890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:12.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:12.552+0000 7f8327e00640 1 --2- 172.21.15.5:0/874454437 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55980cd22020 0x55980cd1d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:12.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:12.552+0000 7f8326a00640 1 -- 172.21.15.5:0/874454437 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55980cd22020 msgr2=0x55980cd1d720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:59:12.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:12.552+0000 7f8326a00640 1 --2- 172.21.15.5:0/874454437 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55980cd22020 0x55980cd1d720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:12.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:12.552+0000 7f8326a00640 1 -- 172.21.15.5:0/874454437 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55980cd16e40 msgr2=0x55980cd39890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:59:12.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:12.552+0000 7f8326a00640 1 --2- 172.21.15.5:0/874454437 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55980cd16e40 0x55980cd39890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:12.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:12.552+0000 7f8326a00640 1 -- 172.21.15.5:0/874454437 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55980cd17590 con 0x55980cd17760 2024-09-10T09:59:12.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:12.553+0000 7f8326a00640 1 --2- 172.21.15.5:0/874454437 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55980cd17760 0x55980cd39dd0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f831409b0c0 tx=0x7f8314000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f3c39f25910394a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:12.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:12.553+0000 7f8313400640 1 -- 172.21.15.5:0/874454437 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f831409b130 con 0x55980cd17760 2024-09-10T09:59:12.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:12.553+0000 7f8313400640 1 -- 172.21.15.5:0/874454437 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f831409f4e0 con 0x55980cd17760 2024-09-10T09:59:12.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:12.553+0000 7f8313400640 1 -- 172.21.15.5:0/874454437 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f831409f130 con 0x55980cd17760 2024-09-10T09:59:12.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:12.554+0000 7f83309bb740 1 -- 172.21.15.5:0/874454437 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55980cd17760 msgr2=0x55980cd39dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:12.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:12.554+0000 7f83309bb740 1 --2- 172.21.15.5:0/874454437 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55980cd17760 0x55980cd39dd0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f831409b0c0 tx=0x7f8314000bc0 comp rx=0 tx=0).stop 2024-09-10T09:59:12.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:12.554+0000 7f83309bb740 1 -- 172.21.15.5:0/874454437 shutdown_connections 2024-09-10T09:59:12.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:12.554+0000 7f83309bb740 1 --2- 172.21.15.5:0/874454437 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55980cd22020 0x55980cd1d720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:12.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:12.554+0000 7f83309bb740 1 --2- 172.21.15.5:0/874454437 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55980cd17760 0x55980cd39dd0 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:12.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:12.554+0000 7f83309bb740 1 --2- 172.21.15.5:0/874454437 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55980cd16e40 0x55980cd39890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:12.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:12.554+0000 7f83309bb740 1 -- 172.21.15.5:0/874454437 >> 172.21.15.5:0/874454437 conn(0x55980cd133d0 msgr2=0x55980cd18890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:12.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:12.554+0000 7f83309bb740 1 -- 172.21.15.5:0/874454437 shutdown_connections 2024-09-10T09:59:12.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:12.554+0000 7f83309bb740 1 -- 172.21.15.5:0/874454437 wait complete. 2024-09-10T09:59:12.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:12.554+0000 7f83309bb740 1 -- start start 2024-09-10T09:59:12.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:12.554+0000 7f83309bb740 1 Processor -- start 2024-09-10T09:59:12.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:12.555+0000 7f83309bb740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55980cd16e40 0x55980cd2a390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:12.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:12.555+0000 7f83309bb740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55980cd17760 0x55980cd1baa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:12.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:12.555+0000 7f83309bb740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55980cd22020 0x55980cd1bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:12.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:12.555+0000 7f83309bb740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f83281b66a0 con 0x55980cd22020 2024-09-10T09:59:12.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:12.555+0000 7f83309bb740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f83281b5fd0 con 0x55980cd16e40 2024-09-10T09:59:12.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:12.555+0000 7f83309bb740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f83281b3210 con 0x55980cd17760 2024-09-10T09:59:12.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:12.555+0000 7f83309bb740 10 client.0 _ll_register_callbacks cb 0x7f8318009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:59:12.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:12.555+0000 7f8327400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55980cd16e40 0x55980cd2a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:12.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:12.555+0000 7f8326a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55980cd17760 0x55980cd1baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:12.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:12.555+0000 7f8327e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55980cd22020 0x55980cd1bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:12.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:12.555+0000 7f8326a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55980cd17760 0x55980cd1baa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:57468/0 (socket says 172.21.15.5:57468) 2024-09-10T09:59:12.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:12.555+0000 7f8327400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55980cd16e40 0x55980cd2a390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:35816/0 (socket says 172.21.15.5:35816) 2024-09-10T09:59:12.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:12.555+0000 7f8326a00640 1 -- 172.21.15.5:0/3991433511 learned_addr learned my addr 172.21.15.5:0/3991433511 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:59:12.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:12.556+0000 7f8326a00640 1 -- 172.21.15.5:0/3991433511 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55980cd16e40 msgr2=0x55980cd2a390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:59:12.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:12.556+0000 7f8326a00640 1 --2- 172.21.15.5:0/3991433511 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55980cd16e40 0x55980cd2a390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:12.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:12.556+0000 7f8326a00640 1 -- 172.21.15.5:0/3991433511 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55980cd22020 msgr2=0x55980cd1bfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:59:12.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:12.556+0000 7f8326a00640 1 --2- 172.21.15.5:0/3991433511 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55980cd22020 0x55980cd1bfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:12.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:12.556+0000 7f8326a00640 1 -- 172.21.15.5:0/3991433511 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55980cd1c5d0 con 0x55980cd17760 2024-09-10T09:59:12.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:12.556+0000 7f8327e00640 1 --2- 172.21.15.5:0/3991433511 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55980cd22020 0x55980cd1bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:59:12.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:12.556+0000 7f8327400640 1 --2- 172.21.15.5:0/3991433511 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55980cd16e40 0x55980cd2a390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:59:12.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:12.556+0000 7f8326a00640 1 --2- 172.21.15.5:0/3991433511 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55980cd17760 0x55980cd1baa0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f8314097540 tx=0x7f83140046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a6ef0cb6149d8708 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:12.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:12.556+0000 7f8310c00640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-09-10T09:59:12.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:12.556+0000 7f8310c00640 1 -- 172.21.15.5:0/3991433511 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83140cc4e0 con 0x55980cd17760 2024-09-10T09:59:12.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:12.556+0000 7f8310c00640 1 -- 172.21.15.5:0/3991433511 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f83140bb0b0 con 0x55980cd17760 2024-09-10T09:59:12.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:12.556+0000 7f83309bb740 1 -- 172.21.15.5:0/3991433511 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f831409f4e0 con 0x55980cd17760 2024-09-10T09:59:12.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:12.557+0000 7f8310c00640 1 -- 172.21.15.5:0/3991433511 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f831409f130 con 0x55980cd17760 2024-09-10T09:59:12.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:12.557+0000 7f83309bb740 10 client.29413 fetch_fsmap learned FSMap version 42 2024-09-10T09:59:12.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:12.557+0000 7f83309bb740 1 -- 172.21.15.5:0/3991433511 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55980cd9c660 con 0x55980cd17760 2024-09-10T09:59:12.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:12.557+0000 7f8310c00640 1 -- 172.21.15.5:0/3991433511 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f831409f4e0 con 0x55980cd17760 2024-09-10T09:59:12.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:12.557+0000 7f8310c00640 1 -- 172.21.15.5:0/3991433511 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 42) ==== 8370+0+0 (secure 0 0 0) 0x7f8314099ff0 con 0x55980cd17760 2024-09-10T09:59:12.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:12.557+0000 7f83309bb740 10 client.29413 fetch_fsmap finished waiting for FSMap version 42 2024-09-10T09:59:12.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:12.557+0000 7f83309bb740 10 client.29413 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-09-10T09:59:12.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:12.557+0000 7f83309bb740 20 client.29413 populate_metadata read hostname 'smithi005' 2024-09-10T09:59:12.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:12.557+0000 7f83309bb740 1 --2- 172.21.15.5:0/3991433511 >> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] conn(0x55980cd9ccb0 0x55980cd9f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:12.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:12.557+0000 7f83309bb740 4 client.29413 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T09:59:12.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:12.557+0000 7f83309bb740 1 -- 172.21.15.5:0/3991433511 --> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55980cd194b0 con 0x55980cd9ccb0 2024-09-10T09:59:12.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:12.558+0000 7f8327400640 1 --2- 172.21.15.5:0/3991433511 >> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] conn(0x55980cd9ccb0 0x55980cd9f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:12.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:12.558+0000 7f8327400640 1 --2- 172.21.15.5:0/3991433511 >> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] conn(0x55980cd9ccb0 0x55980cd9f0d0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=e6a1e0d780de8d7 server_cookie=32fabdfc8bb3d814 in_seq=0 out_seq=0 2024-09-10T09:59:12.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:12.558+0000 7f8310c00640 10 client.29413 ms_handle_connect on v2:172.21.15.102:6836/2689776764 2024-09-10T09:59:12.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:12.558+0000 7f8310c00640 1 -- 172.21.15.5:0/3991433511 <== mds.3 v2:172.21.15.102:6836/2689776764 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55980cd194b0 con 0x55980cd9ccb0 2024-09-10T09:59:12.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:12.558+0000 7f8310c00640 10 client.29413 handle_command_reply: tid=0 2024-09-10T09:59:12.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:12.558+0000 7f83309bb740 1 client.29413 shutdown 2024-09-10T09:59:12.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:12.559+0000 7f83309bb740 1 -- 172.21.15.5:0/3991433511 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55980cd17760 msgr2=0x55980cd1baa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:12.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:12.559+0000 7f83309bb740 1 --2- 172.21.15.5:0/3991433511 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55980cd17760 0x55980cd1baa0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f8314097540 tx=0x7f83140046c0 comp rx=0 tx=0).stop 2024-09-10T09:59:12.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:12.559+0000 7f83309bb740 1 -- 172.21.15.5:0/3991433511 shutdown_connections 2024-09-10T09:59:12.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:12.559+0000 7f83309bb740 1 --2- 172.21.15.5:0/3991433511 >> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] conn(0x55980cd9ccb0 0x55980cd9f0d0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:12.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:12.559+0000 7f83309bb740 1 --2- 172.21.15.5:0/3991433511 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55980cd22020 0x55980cd1bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:12.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:12.559+0000 7f8310c00640 0 client.29413 ms_handle_reset on v2:172.21.15.102:6836/2689776764 2024-09-10T09:59:12.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:12.559+0000 7f83309bb740 1 --2- 172.21.15.5:0/3991433511 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55980cd17760 0x55980cd1baa0 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:12.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:12.559+0000 7f83309bb740 1 --2- 172.21.15.5:0/3991433511 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55980cd16e40 0x55980cd2a390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:12.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:12.559+0000 7f83309bb740 1 -- 172.21.15.5:0/3991433511 >> 172.21.15.5:0/3991433511 conn(0x55980cd133d0 msgr2=0x55980cd1ef20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:12.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:12.559+0000 7f83309bb740 1 -- 172.21.15.5:0/3991433511 shutdown_connections 2024-09-10T09:59:12.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:12.559+0000 7f83309bb740 1 -- 172.21.15.5:0/3991433511 wait complete. 2024-09-10T09:59:12.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:12.559+0000 7f83309bb740 20 client.29413 trim_cache size 0 max 16384 2024-09-10T09:59:12.566 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-10T09:59:12.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:12.560+0000 7f832ca00640 1 -- 172.21.15.5:0/1176903986 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f830c0a2b80 msgr2=0x7f830c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:12.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:12.560+0000 7f832ca00640 1 --2- 172.21.15.5:0/1176903986 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f830c0a2b80 0x7f830c0a5040 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x55980cd14230 tx=0x7f8318005f90 comp rx=0 tx=0).stop 2024-09-10T09:59:12.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:12.560+0000 7f832ca00640 1 -- 172.21.15.5:0/1176903986 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8328077040 msgr2=0x7f83281a4b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:12.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:12.560+0000 7f832ca00640 1 --2- 172.21.15.5:0/1176903986 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8328077040 0x7f83281a4b10 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f831400d950 tx=0x7f831400de10 comp rx=0 tx=0).stop 2024-09-10T09:59:12.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:12.560+0000 7f832ca00640 1 -- 172.21.15.5:0/1176903986 shutdown_connections 2024-09-10T09:59:12.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:12.560+0000 7f832ca00640 1 --2- 172.21.15.5:0/1176903986 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f830c0a2b80 0x7f830c0a5040 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:12.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:12.560+0000 7f832ca00640 1 --2- 172.21.15.5:0/1176903986 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8328112870 0x7f832819e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:12.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:12.560+0000 7f832ca00640 1 --2- 172.21.15.5:0/1176903986 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8328077040 0x7f83281a4b10 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:12.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:12.560+0000 7f832ca00640 1 --2- 172.21.15.5:0/1176903986 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8328076720 0x7f83281a45d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:12.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:12.560+0000 7f832ca00640 1 -- 172.21.15.5:0/1176903986 >> 172.21.15.5:0/1176903986 conn(0x7f83280ff8d0 msgr2=0x7f8328101cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:12.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:12.560+0000 7f832ca00640 1 -- 172.21.15.5:0/1176903986 shutdown_connections 2024-09-10T09:59:12.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:12.560+0000 7f832ca00640 1 -- 172.21.15.5:0/1176903986 wait complete. 2024-09-10T09:59:13.025 INFO:tasks.workunit.client.0.smithi005.stdout: 10 110875 13.30 MB/sec execute 291 sec latency 15690.847 ms 2024-09-10T09:59:13.491 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-10T09:59:14.012 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:59:13 smithi005 ceph-mon[30350]: pgmap v1019: 129 pgs: 129 active+clean; 649 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 279 KiB/s rd, 1.5 MiB/s wr, 58 op/s 2024-09-10T09:59:14.025 INFO:tasks.workunit.client.0.smithi005.stdout: 10 110875 13.26 MB/sec execute 292 sec latency 16690.928 ms 2024-09-10T09:59:14.034 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:59:14.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:59:13 smithi159 ceph-mon[35513]: pgmap v1019: 129 pgs: 129 active+clean; 649 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 279 KiB/s rd, 1.5 MiB/s wr, 58 op/s 2024-09-10T09:59:14.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:59:13 smithi102 ceph-mon[35442]: pgmap v1019: 129 pgs: 129 active+clean; 649 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 279 KiB/s rd, 1.5 MiB/s wr, 58 op/s 2024-09-10T09:59:14.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:14.878+0000 7f586b600640 1 -- 172.21.15.5:0/4140962508 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f58641049c0 msgr2=0x7f5864108a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:14.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:14.878+0000 7f586b600640 1 --2- 172.21.15.5:0/4140962508 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f58641049c0 0x7f5864108a10 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f5850009900 tx=0x7f585002a410 comp rx=0 tx=0).stop 2024-09-10T09:59:14.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:14.879+0000 7f586b600640 1 -- 172.21.15.5:0/4140962508 shutdown_connections 2024-09-10T09:59:14.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:14.879+0000 7f586b600640 1 --2- 172.21.15.5:0/4140962508 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5864109140 0x7f5864110cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:14.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:14.879+0000 7f586b600640 1 --2- 172.21.15.5:0/4140962508 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f58641049c0 0x7f5864108a10 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:14.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:14.879+0000 7f586b600640 1 --2- 172.21.15.5:0/4140962508 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5864104010 0x7f58641043f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:14.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:14.879+0000 7f586b600640 1 -- 172.21.15.5:0/4140962508 >> 172.21.15.5:0/4140962508 conn(0x7f58640ffa50 msgr2=0x7f5864101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:14.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:14.879+0000 7f586b600640 1 -- 172.21.15.5:0/4140962508 shutdown_connections 2024-09-10T09:59:14.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:14.879+0000 7f586b600640 1 -- 172.21.15.5:0/4140962508 wait complete. 2024-09-10T09:59:14.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:14.880+0000 7f586b600640 1 Processor -- start 2024-09-10T09:59:14.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:14.880+0000 7f586b600640 1 -- start start 2024-09-10T09:59:14.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:14.880+0000 7f586b600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5864104010 0x7f586419a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:14.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:14.880+0000 7f586b600640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f58641049c0 0x7f586419ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:14.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:14.880+0000 7f586b600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5864109140 0x7f586419efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:14.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:14.880+0000 7f586b600640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f5864114190 con 0x7f5864109140 2024-09-10T09:59:14.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:14.880+0000 7f586b600640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f5864114010 con 0x7f5864104010 2024-09-10T09:59:14.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:14.880+0000 7f586b600640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f5864114310 con 0x7f58641049c0 2024-09-10T09:59:14.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:14.881+0000 7f5863e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f58641049c0 0x7f586419ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:14.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:14.881+0000 7f5868c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5864104010 0x7f586419a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:14.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:14.881+0000 7f5863e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f58641049c0 0x7f586419ac30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:57482/0 (socket says 172.21.15.5:57482) 2024-09-10T09:59:14.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:14.881+0000 7f5863e00640 1 -- 172.21.15.5:0/1644232895 learned_addr learned my addr 172.21.15.5:0/1644232895 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:59:14.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:14.881+0000 7f5869600640 1 --2- 172.21.15.5:0/1644232895 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5864109140 0x7f586419efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:14.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:14.881+0000 7f5863e00640 1 -- 172.21.15.5:0/1644232895 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5864104010 msgr2=0x7f586419a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:14.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:14.881+0000 7f5863e00640 1 --2- 172.21.15.5:0/1644232895 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5864104010 0x7f586419a6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:14.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:14.881+0000 7f5863e00640 1 -- 172.21.15.5:0/1644232895 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5864109140 msgr2=0x7f586419efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:14.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:14.881+0000 7f5863e00640 1 --2- 172.21.15.5:0/1644232895 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5864109140 0x7f586419efc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:14.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:14.881+0000 7f5863e00640 1 -- 172.21.15.5:0/1644232895 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f586419f6a0 con 0x7f58641049c0 2024-09-10T09:59:14.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:14.881+0000 7f5868c00640 1 --2- 172.21.15.5:0/1644232895 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5864104010 0x7f586419a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:59:14.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:14.885+0000 7f5863e00640 1 --2- 172.21.15.5:0/1644232895 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f58641049c0 0x7f586419ac30 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f5850004770 tx=0x7f5850003c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:14.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:14.886+0000 7f5861600640 1 -- 172.21.15.5:0/1644232895 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f585003f070 con 0x7f58641049c0 2024-09-10T09:59:14.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:14.886+0000 7f5861600640 1 -- 172.21.15.5:0/1644232895 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5850031ca0 con 0x7f58641049c0 2024-09-10T09:59:14.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:14.886+0000 7f586b600640 1 -- 172.21.15.5:0/1644232895 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f586419f930 con 0x7f58641049c0 2024-09-10T09:59:14.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:14.886+0000 7f5861600640 1 -- 172.21.15.5:0/1644232895 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5850035030 con 0x7f58641049c0 2024-09-10T09:59:14.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:14.886+0000 7f586b600640 1 -- 172.21.15.5:0/1644232895 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58641a7270 con 0x7f58641049c0 2024-09-10T09:59:14.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:14.889+0000 7f586e08f740 1 Processor -- start 2024-09-10T09:59:14.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:14.889+0000 7f586e08f740 1 -- start start 2024-09-10T09:59:14.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:14.889+0000 7f586e08f740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561f60bbcc70 0x561f60bbd050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:14.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:14.889+0000 7f586e08f740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561f60bbd590 0x561f60bc7910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:14.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:14.889+0000 7f586e08f740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561f60bc7e50 0x561f60bca240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:14.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:14.889+0000 7f586e08f740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x561f60b233e0 con 0x561f60bc7e50 2024-09-10T09:59:14.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:14.889+0000 7f586e08f740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x561f60acad80 con 0x561f60bbcc70 2024-09-10T09:59:14.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:14.889+0000 7f586e08f740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x561f60b1f3a0 con 0x561f60bbd590 2024-09-10T09:59:14.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:14.890+0000 7f5868c00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561f60bbd590 0x561f60bc7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:14.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:14.890+0000 7f5863e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561f60bc7e50 0x561f60bca240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:14.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:14.890+0000 7f5868c00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561f60bbd590 0x561f60bc7910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:57488/0 (socket says 172.21.15.5:57488) 2024-09-10T09:59:14.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:14.890+0000 7f5868c00640 1 -- 172.21.15.5:0/1809317318 learned_addr learned my addr 172.21.15.5:0/1809317318 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:59:14.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:14.890+0000 7f5868c00640 1 -- 172.21.15.5:0/1809317318 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561f60bbcc70 msgr2=0x561f60bbd050 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T09:59:14.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:14.890+0000 7f5868c00640 1 --2- 172.21.15.5:0/1809317318 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561f60bbcc70 0x561f60bbd050 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:14.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:14.890+0000 7f5868c00640 1 -- 172.21.15.5:0/1809317318 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561f60bc7e50 msgr2=0x561f60bca240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:59:14.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:14.890+0000 7f5868c00640 1 --2- 172.21.15.5:0/1809317318 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561f60bc7e50 0x561f60bca240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:14.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:14.890+0000 7f5868c00640 1 -- 172.21.15.5:0/1809317318 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561f60ae9ae0 con 0x561f60bbd590 2024-09-10T09:59:14.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:14.890+0000 7f5863e00640 1 --2- 172.21.15.5:0/1809317318 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561f60bc7e50 0x561f60bca240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:59:14.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:14.890+0000 7f5868c00640 1 --2- 172.21.15.5:0/1809317318 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561f60bbd590 0x561f60bc7910 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f58580115f0 tx=0x7f5858011ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1ca6c2489d9fe325 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:14.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:14.891+0000 7f5855c00640 1 -- 172.21.15.5:0/1809317318 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f585801e070 con 0x561f60bbd590 2024-09-10T09:59:14.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:14.891+0000 7f5855c00640 1 -- 172.21.15.5:0/1809317318 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f585800fb20 con 0x561f60bbd590 2024-09-10T09:59:14.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:14.891+0000 7f5855c00640 1 -- 172.21.15.5:0/1809317318 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5858019c40 con 0x561f60bbd590 2024-09-10T09:59:14.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:14.891+0000 7f586e08f740 1 -- 172.21.15.5:0/1809317318 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561f60bbd590 msgr2=0x561f60bc7910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:14.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:14.891+0000 7f586e08f740 1 --2- 172.21.15.5:0/1809317318 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561f60bbd590 0x561f60bc7910 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f58580115f0 tx=0x7f5858011ab0 comp rx=0 tx=0).stop 2024-09-10T09:59:14.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:14.892+0000 7f586e08f740 1 -- 172.21.15.5:0/1809317318 shutdown_connections 2024-09-10T09:59:14.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:14.892+0000 7f586e08f740 1 --2- 172.21.15.5:0/1809317318 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561f60bc7e50 0x561f60bca240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:14.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:14.892+0000 7f586e08f740 1 --2- 172.21.15.5:0/1809317318 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561f60bbd590 0x561f60bc7910 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:14.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:14.892+0000 7f586e08f740 1 --2- 172.21.15.5:0/1809317318 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561f60bbcc70 0x561f60bbd050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:14.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:14.892+0000 7f586e08f740 1 -- 172.21.15.5:0/1809317318 >> 172.21.15.5:0/1809317318 conn(0x561f60bb9200 msgr2=0x561f60bb95e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:14.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:14.892+0000 7f586e08f740 1 -- 172.21.15.5:0/1809317318 shutdown_connections 2024-09-10T09:59:14.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:14.892+0000 7f586e08f740 1 -- 172.21.15.5:0/1809317318 wait complete. 2024-09-10T09:59:14.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:14.892+0000 7f586e08f740 1 -- start start 2024-09-10T09:59:14.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:14.892+0000 7f586e08f740 1 Processor -- start 2024-09-10T09:59:14.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:14.893+0000 7f586e08f740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561f60bbcc70 0x561f60bc4d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:14.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:14.893+0000 7f586e08f740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561f60bbd590 0x561f60bc52d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:14.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:14.893+0000 7f586e08f740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561f60bc7e50 0x561f60bc5810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:14.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:14.893+0000 7f586e08f740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x561f60bcaa20 con 0x561f60bbcc70 2024-09-10T09:59:14.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:14.893+0000 7f586e08f740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x561f60ade0f0 con 0x561f60bc7e50 2024-09-10T09:59:14.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:14.893+0000 7f586e08f740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x561f60bcaba0 con 0x561f60bbd590 2024-09-10T09:59:14.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:14.893+0000 7f586e08f740 10 client.0 _ll_register_callbacks cb 0x561f60ac5bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:59:14.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:14.893+0000 7f5868c00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561f60bbd590 0x561f60bc52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:14.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:14.893+0000 7f5868c00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561f60bbd590 0x561f60bc52d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:57496/0 (socket says 172.21.15.5:57496) 2024-09-10T09:59:14.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:14.893+0000 7f5868c00640 1 -- 172.21.15.5:0/3669587532 learned_addr learned my addr 172.21.15.5:0/3669587532 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:59:14.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:14.893+0000 7f5869600640 1 --2- 172.21.15.5:0/3669587532 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561f60bbcc70 0x561f60bc4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:14.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:14.893+0000 7f5863e00640 1 --2- 172.21.15.5:0/3669587532 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561f60bc7e50 0x561f60bc5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:14.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:14.894+0000 7f5868c00640 1 -- 172.21.15.5:0/3669587532 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561f60bc7e50 msgr2=0x561f60bc5810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:59:14.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:14.894+0000 7f5868c00640 1 --2- 172.21.15.5:0/3669587532 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561f60bc7e50 0x561f60bc5810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:14.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:14.894+0000 7f5868c00640 1 -- 172.21.15.5:0/3669587532 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561f60bbcc70 msgr2=0x561f60bc4d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:59:14.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:14.894+0000 7f5868c00640 1 --2- 172.21.15.5:0/3669587532 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561f60bbcc70 0x561f60bc4d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:14.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:14.894+0000 7f5868c00640 1 -- 172.21.15.5:0/3669587532 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561f608943c0 con 0x561f60bbd590 2024-09-10T09:59:14.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:14.897+0000 7f5868c00640 1 --2- 172.21.15.5:0/3669587532 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561f60bbd590 0x561f60bc52d0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f585801c040 tx=0x7f585800f870 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1b963b5dd9cf662 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:14.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:14.897+0000 7f584a000640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-09-10T09:59:14.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:14.898+0000 7f584a000640 1 -- 172.21.15.5:0/3669587532 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5858027060 con 0x561f60bbd590 2024-09-10T09:59:14.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:14.898+0000 7f584a000640 1 -- 172.21.15.5:0/3669587532 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f58580199c0 con 0x561f60bbd590 2024-09-10T09:59:14.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:14.898+0000 7f586e08f740 1 -- 172.21.15.5:0/3669587532 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f585800fb20 con 0x561f60bbd590 2024-09-10T09:59:14.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:14.898+0000 7f584a000640 1 -- 172.21.15.5:0/3669587532 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f585802e050 con 0x561f60bbd590 2024-09-10T09:59:14.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:14.898+0000 7f586e08f740 10 client.29425 fetch_fsmap learned FSMap version 42 2024-09-10T09:59:14.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:14.898+0000 7f586e08f740 1 -- 172.21.15.5:0/3669587532 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x561f60bd6ab0 con 0x561f60bbd590 2024-09-10T09:59:14.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:14.898+0000 7f584a000640 1 -- 172.21.15.5:0/3669587532 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f585800fb20 con 0x561f60bbd590 2024-09-10T09:59:14.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:14.899+0000 7f584a000640 1 -- 172.21.15.5:0/3669587532 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 42) ==== 8370+0+0 (secure 0 0 0) 0x7f58580035f0 con 0x561f60bbd590 2024-09-10T09:59:14.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:14.899+0000 7f586e08f740 10 client.29425 fetch_fsmap finished waiting for FSMap version 42 2024-09-10T09:59:14.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:14.899+0000 7f586e08f740 10 client.29425 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-10T09:59:14.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:14.899+0000 7f586e08f740 20 client.29425 populate_metadata read hostname 'smithi005' 2024-09-10T09:59:14.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:14.899+0000 7f586e08f740 1 --2- 172.21.15.5:0/3669587532 >> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] conn(0x561f60bd7f70 0x561f60bda360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:14.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:14.899+0000 7f586e08f740 4 client.29425 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:59:14.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:14.899+0000 7f586e08f740 1 -- 172.21.15.5:0/3669587532 --> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561f60b30950 con 0x561f60bd7f70 2024-09-10T09:59:14.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:14.899+0000 7f5869600640 1 --2- 172.21.15.5:0/3669587532 >> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] conn(0x561f60bd7f70 0x561f60bda360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:14.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:14.899+0000 7f5861600640 1 -- 172.21.15.5:0/1644232895 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f5850042070 con 0x7f58641049c0 2024-09-10T09:59:14.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:14.901+0000 7f5861600640 1 --2- 172.21.15.5:0/1644232895 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f584c0a2b10 0x7f584c0a4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:14.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:14.901+0000 7f5861600640 1 -- 172.21.15.5:0/1644232895 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f58500efe40 con 0x7f58641049c0 2024-09-10T09:59:14.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:14.901+0000 7f5863e00640 1 --2- 172.21.15.5:0/1644232895 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f584c0a2b10 0x7f584c0a4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:14.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:14.902+0000 7f5863e00640 1 --2- 172.21.15.5:0/1644232895 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f584c0a2b10 0x7f584c0a4fd0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f5850031fb0 tx=0x7f58500dc040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:14.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:14.909+0000 7f5869600640 1 --2- 172.21.15.5:0/3669587532 >> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] conn(0x561f60bd7f70 0x561f60bda360 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7a2cf049cff518f5 server_cookie=686ca3a7c727b846 in_seq=0 out_seq=0 2024-09-10T09:59:14.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:14.909+0000 7f584a000640 10 client.29425 ms_handle_connect on v2:172.21.15.159:6836/620627639 2024-09-10T09:59:14.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:14.910+0000 7f584a000640 1 -- 172.21.15.5:0/3669587532 <== mds.2 v2:172.21.15.159:6836/620627639 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561f60b30950 con 0x561f60bd7f70 2024-09-10T09:59:14.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:14.910+0000 7f584a000640 10 client.29425 handle_command_reply: tid=0 2024-09-10T09:59:14.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:14.910+0000 7f586e08f740 1 client.29425 shutdown 2024-09-10T09:59:14.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:14.910+0000 7f586e08f740 1 -- 172.21.15.5:0/3669587532 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561f60bbd590 msgr2=0x561f60bc52d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:14.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:14.910+0000 7f586e08f740 1 --2- 172.21.15.5:0/3669587532 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561f60bbd590 0x561f60bc52d0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f585801c040 tx=0x7f585800f870 comp rx=0 tx=0).stop 2024-09-10T09:59:14.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:14.910+0000 7f586e08f740 1 -- 172.21.15.5:0/3669587532 shutdown_connections 2024-09-10T09:59:14.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:14.910+0000 7f586e08f740 1 --2- 172.21.15.5:0/3669587532 >> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] conn(0x561f60bd7f70 0x561f60bda360 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:14.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:14.910+0000 7f586e08f740 1 --2- 172.21.15.5:0/3669587532 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561f60bc7e50 0x561f60bc5810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:14.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:14.910+0000 7f586e08f740 1 --2- 172.21.15.5:0/3669587532 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561f60bbd590 0x561f60bc52d0 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:14.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:14.910+0000 7f586e08f740 1 --2- 172.21.15.5:0/3669587532 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561f60bbcc70 0x561f60bc4d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:14.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:14.910+0000 7f586e08f740 1 -- 172.21.15.5:0/3669587532 >> 172.21.15.5:0/3669587532 conn(0x561f60bb9200 msgr2=0x561f60bb8380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:14.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:14.910+0000 7f584a000640 0 client.29425 ms_handle_reset on v2:172.21.15.159:6836/620627639 2024-09-10T09:59:14.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:14.910+0000 7f586e08f740 1 -- 172.21.15.5:0/3669587532 shutdown_connections 2024-09-10T09:59:14.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:14.910+0000 7f586e08f740 1 -- 172.21.15.5:0/3669587532 wait complete. 2024-09-10T09:59:14.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:14.911+0000 7f586e08f740 20 client.29425 trim_cache size 0 max 16384 2024-09-10T09:59:14.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:14.936+0000 7f586e08f740 1 Processor -- start 2024-09-10T09:59:14.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:14.936+0000 7f586e08f740 1 -- start start 2024-09-10T09:59:14.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:14.936+0000 7f586e08f740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561f60bbcc70 0x561f60bbedd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:14.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:14.936+0000 7f586e08f740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561f60bbd590 0x561f60bdfa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:14.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:14.936+0000 7f586e08f740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561f60bc7e50 0x561f60bdff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:14.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:14.936+0000 7f586e08f740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f58641b11d0 con 0x561f60bbcc70 2024-09-10T09:59:14.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:14.936+0000 7f586e08f740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f58641b2d10 con 0x561f60bc7e50 2024-09-10T09:59:14.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:14.936+0000 7f586e08f740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f58641b18a0 con 0x561f60bbd590 2024-09-10T09:59:14.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:14.937+0000 7f5869600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561f60bbcc70 0x561f60bbedd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:14.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:14.937+0000 7f5869600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561f60bbcc70 0x561f60bbedd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60436/0 (socket says 172.21.15.5:60436) 2024-09-10T09:59:14.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:14.937+0000 7f5869600640 1 -- 172.21.15.5:0/2899213852 learned_addr learned my addr 172.21.15.5:0/2899213852 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:59:14.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:14.937+0000 7f5868c00640 1 --2- 172.21.15.5:0/2899213852 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561f60bbd590 0x561f60bdfa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:14.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:14.937+0000 7f5869600640 1 --2- 172.21.15.5:0/2899213852 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561f60bc7e50 0x561f60bdff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:14.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:14.937+0000 7f5868c00640 1 -- 172.21.15.5:0/2899213852 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561f60bc7e50 msgr2=0x561f60bdff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:59:14.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:14.937+0000 7f5868c00640 1 --2- 172.21.15.5:0/2899213852 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561f60bc7e50 0x561f60bdff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:14.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:14.937+0000 7f5868c00640 1 -- 172.21.15.5:0/2899213852 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561f60bbcc70 msgr2=0x561f60bbedd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:59:14.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:14.937+0000 7f5868c00640 1 --2- 172.21.15.5:0/2899213852 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561f60bbcc70 0x561f60bbedd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:14.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:14.937+0000 7f5868c00640 1 -- 172.21.15.5:0/2899213852 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561f60bb4770 con 0x561f60bbd590 2024-09-10T09:59:14.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:14.937+0000 7f5869600640 1 --2- 172.21.15.5:0/2899213852 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561f60bbcc70 0x561f60bbedd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:59:14.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:14.938+0000 7f5868c00640 1 --2- 172.21.15.5:0/2899213852 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561f60bbd590 0x561f60bdfa50 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f58580039f0 tx=0x7f5858011ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7663879f272d944a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:14.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:14.938+0000 7f5855200640 1 -- 172.21.15.5:0/2899213852 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5858015020 con 0x561f60bbd590 2024-09-10T09:59:14.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:14.938+0000 7f5855200640 1 -- 172.21.15.5:0/2899213852 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f58580037f0 con 0x561f60bbd590 2024-09-10T09:59:14.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:14.938+0000 7f5855200640 1 -- 172.21.15.5:0/2899213852 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f585801e040 con 0x561f60bbd590 2024-09-10T09:59:14.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:14.939+0000 7f586e08f740 1 -- 172.21.15.5:0/2899213852 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561f60bbd590 msgr2=0x561f60bdfa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:14.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:14.939+0000 7f586e08f740 1 --2- 172.21.15.5:0/2899213852 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561f60bbd590 0x561f60bdfa50 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f58580039f0 tx=0x7f5858011ab0 comp rx=0 tx=0).stop 2024-09-10T09:59:14.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:14.939+0000 7f586e08f740 1 -- 172.21.15.5:0/2899213852 shutdown_connections 2024-09-10T09:59:14.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:14.939+0000 7f586e08f740 1 --2- 172.21.15.5:0/2899213852 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561f60bc7e50 0x561f60bdff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:14.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:14.939+0000 7f586e08f740 1 --2- 172.21.15.5:0/2899213852 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561f60bbd590 0x561f60bdfa50 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:14.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:14.939+0000 7f586e08f740 1 --2- 172.21.15.5:0/2899213852 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561f60bbcc70 0x561f60bbedd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:14.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:14.939+0000 7f586e08f740 1 -- 172.21.15.5:0/2899213852 >> 172.21.15.5:0/2899213852 conn(0x561f60bb9200 msgr2=0x561f60bbe6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:14.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:14.939+0000 7f586e08f740 1 -- 172.21.15.5:0/2899213852 shutdown_connections 2024-09-10T09:59:14.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:14.939+0000 7f586e08f740 1 -- 172.21.15.5:0/2899213852 wait complete. 2024-09-10T09:59:14.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:14.939+0000 7f586e08f740 1 -- start start 2024-09-10T09:59:14.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:14.940+0000 7f586e08f740 1 Processor -- start 2024-09-10T09:59:14.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:14.940+0000 7f586e08f740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561f60bbcc70 0x561f60bd0490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:14.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:14.940+0000 7f586e08f740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561f60bbd590 0x561f60bcb7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:14.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:14.940+0000 7f586e08f740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561f60bc7e50 0x561f60bcbd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:14.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:14.940+0000 7f586e08f740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f58641b2640 con 0x561f60bc7e50 2024-09-10T09:59:14.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:14.940+0000 7f586e08f740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f58641b1f70 con 0x561f60bbd590 2024-09-10T09:59:14.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:14.940+0000 7f586e08f740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f58641af1b0 con 0x561f60bbcc70 2024-09-10T09:59:14.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:14.940+0000 7f586e08f740 10 client.0 _ll_register_callbacks cb 0x7f585801c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:59:14.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:14.941+0000 7f5869600640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561f60bbcc70 0x561f60bd0490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:14.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:14.941+0000 7f5869600640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561f60bbcc70 0x561f60bd0490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:57518/0 (socket says 172.21.15.5:57518) 2024-09-10T09:59:14.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:14.941+0000 7f5869600640 1 -- 172.21.15.5:0/3571433297 learned_addr learned my addr 172.21.15.5:0/3571433297 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:59:14.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:14.941+0000 7f5869600640 1 --2- 172.21.15.5:0/3571433297 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561f60bc7e50 0x561f60bcbd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:14.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:14.941+0000 7f5868c00640 1 --2- 172.21.15.5:0/3571433297 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561f60bbd590 0x561f60bcb7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:14.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:14.941+0000 7f5869600640 1 -- 172.21.15.5:0/3571433297 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561f60bbd590 msgr2=0x561f60bcb7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:59:14.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:14.941+0000 7f5869600640 1 --2- 172.21.15.5:0/3571433297 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561f60bbd590 0x561f60bcb7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:14.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:14.941+0000 7f5869600640 1 -- 172.21.15.5:0/3571433297 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561f60bc7e50 msgr2=0x561f60bcbd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:59:14.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:14.941+0000 7f5869600640 1 --2- 172.21.15.5:0/3571433297 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561f60bc7e50 0x561f60bcbd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:14.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:14.941+0000 7f5869600640 1 -- 172.21.15.5:0/3571433297 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561f60bcc530 con 0x561f60bbcc70 2024-09-10T09:59:14.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:14.941+0000 7f5869600640 1 --2- 172.21.15.5:0/3571433297 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561f60bbcc70 0x561f60bd0490 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f585c012ac0 tx=0x7f585c009500 comp rx=0 tx=0).ready entity=mon.1 client_cookie=63bce4537d726930 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:14.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:14.941+0000 7f584a000640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-09-10T09:59:14.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:14.942+0000 7f584a000640 1 -- 172.21.15.5:0/3571433297 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f585c01e850 con 0x561f60bbcc70 2024-09-10T09:59:14.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:14.942+0000 7f584a000640 1 -- 172.21.15.5:0/3571433297 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f585c01b040 con 0x561f60bbcc70 2024-09-10T09:59:14.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:14.942+0000 7f586e08f740 1 -- 172.21.15.5:0/3571433297 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f58580037f0 con 0x561f60bbcc70 2024-09-10T09:59:14.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:14.942+0000 7f584a000640 1 -- 172.21.15.5:0/3571433297 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f585c01e9f0 con 0x561f60bbcc70 2024-09-10T09:59:14.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:14.942+0000 7f586e08f740 10 client.29431 fetch_fsmap learned FSMap version 42 2024-09-10T09:59:14.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:14.942+0000 7f586e08f740 1 -- 172.21.15.5:0/3571433297 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x561f60c429f0 con 0x561f60bbcc70 2024-09-10T09:59:14.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:14.942+0000 7f584a000640 1 -- 172.21.15.5:0/3571433297 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f58580037f0 con 0x561f60bbcc70 2024-09-10T09:59:14.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:14.943+0000 7f584a000640 1 -- 172.21.15.5:0/3571433297 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 42) ==== 8370+0+0 (secure 0 0 0) 0x7f585c004430 con 0x561f60bbcc70 2024-09-10T09:59:14.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:14.943+0000 7f586e08f740 10 client.29431 fetch_fsmap finished waiting for FSMap version 42 2024-09-10T09:59:14.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:14.943+0000 7f586e08f740 10 client.29431 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-10T09:59:14.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:14.943+0000 7f586e08f740 20 client.29431 populate_metadata read hostname 'smithi005' 2024-09-10T09:59:14.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:14.943+0000 7f586e08f740 1 --2- 172.21.15.5:0/3571433297 >> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] conn(0x561f60c43170 0x561f60c45590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:14.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:14.943+0000 7f586e08f740 4 client.29431 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T09:59:14.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:14.943+0000 7f586e08f740 1 -- 172.21.15.5:0/3571433297 --> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561f60b2ae60 con 0x561f60c43170 2024-09-10T09:59:14.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:14.943+0000 7f5868c00640 1 --2- 172.21.15.5:0/3571433297 >> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] conn(0x561f60c43170 0x561f60c45590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:14.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:14.944+0000 7f5868c00640 1 --2- 172.21.15.5:0/3571433297 >> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] conn(0x561f60c43170 0x561f60c45590 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3efa073ea5fc191c server_cookie=cf7a46288b426684 in_seq=0 out_seq=0 2024-09-10T09:59:14.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:14.944+0000 7f584a000640 10 client.29431 ms_handle_connect on v2:172.21.15.159:6836/620627639 2024-09-10T09:59:14.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:14.944+0000 7f584a000640 1 -- 172.21.15.5:0/3571433297 <== mds.2 v2:172.21.15.159:6836/620627639 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561f60b2ae60 con 0x561f60c43170 2024-09-10T09:59:14.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:14.944+0000 7f584a000640 10 client.29431 handle_command_reply: tid=0 2024-09-10T09:59:14.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:14.944+0000 7f586e08f740 1 client.29431 shutdown 2024-09-10T09:59:14.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:14.944+0000 7f586e08f740 1 -- 172.21.15.5:0/3571433297 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561f60bbcc70 msgr2=0x561f60bd0490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:14.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:14.944+0000 7f586e08f740 1 --2- 172.21.15.5:0/3571433297 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561f60bbcc70 0x561f60bd0490 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f585c012ac0 tx=0x7f585c009500 comp rx=0 tx=0).stop 2024-09-10T09:59:14.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:14.944+0000 7f586e08f740 1 -- 172.21.15.5:0/3571433297 shutdown_connections 2024-09-10T09:59:14.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:14.944+0000 7f586e08f740 1 --2- 172.21.15.5:0/3571433297 >> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] conn(0x561f60c43170 0x561f60c45590 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:14.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:14.944+0000 7f586e08f740 1 --2- 172.21.15.5:0/3571433297 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561f60bc7e50 0x561f60bcbd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:14.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:14.944+0000 7f586e08f740 1 --2- 172.21.15.5:0/3571433297 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x561f60bbd590 0x561f60bcb7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:14.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:14.944+0000 7f586e08f740 1 --2- 172.21.15.5:0/3571433297 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x561f60bbcc70 0x561f60bd0490 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:14.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:14.944+0000 7f586e08f740 1 -- 172.21.15.5:0/3571433297 >> 172.21.15.5:0/3571433297 conn(0x561f60bb9200 msgr2=0x561f60bbeee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:14.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:14.944+0000 7f584a000640 0 client.29431 ms_handle_reset on v2:172.21.15.159:6836/620627639 2024-09-10T09:59:14.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:14.944+0000 7f586e08f740 1 -- 172.21.15.5:0/3571433297 shutdown_connections 2024-09-10T09:59:14.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:14.944+0000 7f586e08f740 1 -- 172.21.15.5:0/3571433297 wait complete. 2024-09-10T09:59:14.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:14.945+0000 7f586e08f740 20 client.29431 trim_cache size 0 max 16384 2024-09-10T09:59:14.950 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-10T09:59:14.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:14.945+0000 7f586b600640 1 -- 172.21.15.5:0/1644232895 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f584c0a2b10 msgr2=0x7f584c0a4fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:14.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:14.945+0000 7f586b600640 1 --2- 172.21.15.5:0/1644232895 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f584c0a2b10 0x7f584c0a4fd0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f5850031fb0 tx=0x7f58500dc040 comp rx=0 tx=0).stop 2024-09-10T09:59:14.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:14.945+0000 7f586b600640 1 -- 172.21.15.5:0/1644232895 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f58641049c0 msgr2=0x7f586419ac30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:14.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:14.945+0000 7f586b600640 1 --2- 172.21.15.5:0/1644232895 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f58641049c0 0x7f586419ac30 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f5850004770 tx=0x7f5850003c10 comp rx=0 tx=0).stop 2024-09-10T09:59:14.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:14.945+0000 7f586b600640 1 -- 172.21.15.5:0/1644232895 shutdown_connections 2024-09-10T09:59:14.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:14.945+0000 7f586b600640 1 --2- 172.21.15.5:0/1644232895 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f584c0a2b10 0x7f584c0a4fd0 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:14.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:14.945+0000 7f586b600640 1 --2- 172.21.15.5:0/1644232895 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5864109140 0x7f586419efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:14.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:14.945+0000 7f586b600640 1 --2- 172.21.15.5:0/1644232895 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f58641049c0 0x7f586419ac30 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:14.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:14.945+0000 7f586b600640 1 --2- 172.21.15.5:0/1644232895 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5864104010 0x7f586419a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:14.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:14.945+0000 7f586b600640 1 -- 172.21.15.5:0/1644232895 >> 172.21.15.5:0/1644232895 conn(0x7f58640ffa50 msgr2=0x7f5864100dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:14.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:14.946+0000 7f586b600640 1 -- 172.21.15.5:0/1644232895 shutdown_connections 2024-09-10T09:59:14.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:14.946+0000 7f586b600640 1 -- 172.21.15.5:0/1644232895 wait complete. 2024-09-10T09:59:15.026 INFO:tasks.workunit.client.0.smithi005.stdout: 10 110875 13.21 MB/sec execute 293 sec latency 17690.992 ms 2024-09-10T09:59:15.945 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-10T09:59:16.025 INFO:tasks.workunit.client.0.smithi005.stdout: 10 110875 13.17 MB/sec execute 294 sec latency 18691.035 ms 2024-09-10T09:59:16.306 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:59:15 smithi005 ceph-mon[30350]: pgmap v1020: 129 pgs: 129 active+clean; 648 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 1.7 MiB/s wr, 56 op/s 2024-09-10T09:59:16.320 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:59:16.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:59:15 smithi159 ceph-mon[35513]: pgmap v1020: 129 pgs: 129 active+clean; 648 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 1.7 MiB/s wr, 56 op/s 2024-09-10T09:59:16.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:59:15 smithi102 ceph-mon[35442]: pgmap v1020: 129 pgs: 129 active+clean; 648 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 1.7 MiB/s wr, 56 op/s 2024-09-10T09:59:16.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:16.899+0000 7f93d2400640 1 -- 172.21.15.5:0/730340623 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f93cc103e30 msgr2=0x7f93cc104210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:16.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:16.899+0000 7f93d2400640 1 --2- 172.21.15.5:0/730340623 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f93cc103e30 0x7f93cc104210 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f93c0009900 tx=0x7f93c002a3f0 comp rx=0 tx=0).stop 2024-09-10T09:59:16.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:16.899+0000 7f93d2400640 1 -- 172.21.15.5:0/730340623 shutdown_connections 2024-09-10T09:59:16.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:16.899+0000 7f93d2400640 1 --2- 172.21.15.5:0/730340623 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f93cc108f60 0x7f93cc110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:16.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:16.899+0000 7f93d2400640 1 --2- 172.21.15.5:0/730340623 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f93cc1047e0 0x7f93cc108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:16.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:16.899+0000 7f93d2400640 1 --2- 172.21.15.5:0/730340623 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f93cc103e30 0x7f93cc104210 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:16.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:16.899+0000 7f93d2400640 1 -- 172.21.15.5:0/730340623 >> 172.21.15.5:0/730340623 conn(0x7f93cc0ff910 msgr2=0x7f93cc101d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:16.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:16.899+0000 7f93d2400640 1 -- 172.21.15.5:0/730340623 shutdown_connections 2024-09-10T09:59:16.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:16.899+0000 7f93d2400640 1 -- 172.21.15.5:0/730340623 wait complete. 2024-09-10T09:59:16.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:16.900+0000 7f93d2400640 1 Processor -- start 2024-09-10T09:59:16.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:16.900+0000 7f93d2400640 1 -- start start 2024-09-10T09:59:16.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:16.901+0000 7f93d2400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f93cc103e30 0x7f93cc19a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:16.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:16.901+0000 7f93d2400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f93cc1047e0 0x7f93cc19a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:16.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:16.901+0000 7f93d2400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f93cc108f60 0x7f93cc19ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:16.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:16.901+0000 7f93d2400640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f93cc113fc0 con 0x7f93cc108f60 2024-09-10T09:59:16.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:16.901+0000 7f93d2400640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f93cc113e40 con 0x7f93cc103e30 2024-09-10T09:59:16.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:16.901+0000 7f93d2400640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f93cc114140 con 0x7f93cc1047e0 2024-09-10T09:59:16.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:16.901+0000 7f93d1000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f93cc103e30 0x7f93cc19a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:16.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:16.901+0000 7f93d1a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f93cc108f60 0x7f93cc19ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:16.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:16.901+0000 7f93cbe00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f93cc1047e0 0x7f93cc19a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:16.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:16.901+0000 7f93d1000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f93cc103e30 0x7f93cc19a480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:35870/0 (socket says 172.21.15.5:35870) 2024-09-10T09:59:16.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:16.901+0000 7f93d1000640 1 -- 172.21.15.5:0/377016074 learned_addr learned my addr 172.21.15.5:0/377016074 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:59:16.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:16.901+0000 7f93d1000640 1 -- 172.21.15.5:0/377016074 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f93cc1047e0 msgr2=0x7f93cc19a9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:16.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:16.902+0000 7f93d1000640 1 --2- 172.21.15.5:0/377016074 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f93cc1047e0 0x7f93cc19a9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:16.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:16.902+0000 7f93d1000640 1 -- 172.21.15.5:0/377016074 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f93cc108f60 msgr2=0x7f93cc19ed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:16.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:16.902+0000 7f93d1000640 1 --2- 172.21.15.5:0/377016074 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f93cc108f60 0x7f93cc19ed50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:16.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:16.902+0000 7f93d1000640 1 -- 172.21.15.5:0/377016074 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93cc19f430 con 0x7f93cc103e30 2024-09-10T09:59:16.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:16.902+0000 7f93cbe00640 1 --2- 172.21.15.5:0/377016074 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f93cc1047e0 0x7f93cc19a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:59:16.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:16.902+0000 7f93d1000640 1 --2- 172.21.15.5:0/377016074 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f93cc103e30 0x7f93cc19a480 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f93c0006010 tx=0x7f93c0004040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:16.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:16.902+0000 7f93c9600640 1 -- 172.21.15.5:0/377016074 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93c003f030 con 0x7f93cc103e30 2024-09-10T09:59:16.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:16.902+0000 7f93c9600640 1 -- 172.21.15.5:0/377016074 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f93c0004440 con 0x7f93cc103e30 2024-09-10T09:59:16.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:16.902+0000 7f93d2400640 1 -- 172.21.15.5:0/377016074 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93cc19f6c0 con 0x7f93cc103e30 2024-09-10T09:59:16.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:16.902+0000 7f93c9600640 1 -- 172.21.15.5:0/377016074 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93c0035070 con 0x7f93cc103e30 2024-09-10T09:59:16.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:16.902+0000 7f93d2400640 1 -- 172.21.15.5:0/377016074 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93cc1a7000 con 0x7f93cc103e30 2024-09-10T09:59:16.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:16.905+0000 7f93c9600640 1 -- 172.21.15.5:0/377016074 <== mon.2 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f93c0042030 con 0x7f93cc103e30 2024-09-10T09:59:16.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:16.905+0000 7f93d6436740 1 Processor -- start 2024-09-10T09:59:16.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:16.906+0000 7f93d6436740 1 -- start start 2024-09-10T09:59:16.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:16.906+0000 7f93d6436740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e109513e40 0x55e109514220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:16.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:16.906+0000 7f93d6436740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e109514760 0x55e10951eae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:16.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:16.906+0000 7f93d6436740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e10951f020 0x55e109521410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:16.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:16.906+0000 7f93d6436740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55e10947a3e0 con 0x55e10951f020 2024-09-10T09:59:16.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:16.906+0000 7f93d6436740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55e109421d80 con 0x55e109514760 2024-09-10T09:59:16.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:16.906+0000 7f93d6436740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55e1094763a0 con 0x55e109513e40 2024-09-10T09:59:16.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:16.906+0000 7f93cbe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e10951f020 0x55e109521410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:16.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:16.906+0000 7f93d1000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e109514760 0x55e10951eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:16.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:16.906+0000 7f93cbe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e10951f020 0x55e109521410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60464/0 (socket says 172.21.15.5:60464) 2024-09-10T09:59:16.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:16.906+0000 7f93cbe00640 1 -- 172.21.15.5:0/1432322312 learned_addr learned my addr 172.21.15.5:0/1432322312 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:59:16.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:16.906+0000 7f93d1a00640 1 --2- 172.21.15.5:0/1432322312 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e109513e40 0x55e109514220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:16.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:16.907+0000 7f93d1000640 1 -- 172.21.15.5:0/1432322312 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e109513e40 msgr2=0x55e109514220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:59:16.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:16.907+0000 7f93d1000640 1 --2- 172.21.15.5:0/1432322312 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e109513e40 0x55e109514220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:16.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:16.907+0000 7f93d1000640 1 -- 172.21.15.5:0/1432322312 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e10951f020 msgr2=0x55e109521410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:59:16.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:16.907+0000 7f93d1000640 1 --2- 172.21.15.5:0/1432322312 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e10951f020 0x55e109521410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:16.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:16.907+0000 7f93d1000640 1 -- 172.21.15.5:0/1432322312 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e109440ae0 con 0x55e109514760 2024-09-10T09:59:16.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:16.907+0000 7f93cbe00640 1 --2- 172.21.15.5:0/1432322312 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e10951f020 0x55e109521410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:59:16.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:16.907+0000 7f93c9600640 1 --2- 172.21.15.5:0/377016074 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f93b40a2c00 0x7f93b40a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:16.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:16.907+0000 7f93d1a00640 1 --2- 172.21.15.5:0/377016074 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f93b40a2c00 0x7f93b40a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:16.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:16.907+0000 7f93c9600640 1 -- 172.21.15.5:0/377016074 <== mon.2 v2:172.21.15.102:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f93c00ef860 con 0x7f93cc103e30 2024-09-10T09:59:16.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:16.907+0000 7f93d1000640 1 --2- 172.21.15.5:0/1432322312 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e109514760 0x55e10951eae0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f93c00eceb0 tx=0x7f93c00f5cb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f7e5d9a346d3642e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:16.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:16.907+0000 7f93be000640 1 -- 172.21.15.5:0/1432322312 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93c0049020 con 0x55e109514760 2024-09-10T09:59:16.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:16.908+0000 7f93be000640 1 -- 172.21.15.5:0/1432322312 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f93c00f5e40 con 0x55e109514760 2024-09-10T09:59:16.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:16.908+0000 7f93be000640 1 -- 172.21.15.5:0/1432322312 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93c00bb680 con 0x55e109514760 2024-09-10T09:59:16.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:16.908+0000 7f93d6436740 1 -- 172.21.15.5:0/1432322312 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e109514760 msgr2=0x55e10951eae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:16.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:16.908+0000 7f93d6436740 1 --2- 172.21.15.5:0/1432322312 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e109514760 0x55e10951eae0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f93c00eceb0 tx=0x7f93c00f5cb0 comp rx=0 tx=0).stop 2024-09-10T09:59:16.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:16.908+0000 7f93d1a00640 1 --2- 172.21.15.5:0/377016074 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f93b40a2c00 0x7f93b40a50c0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f93c4005fd0 tx=0x7f93c4009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:16.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:16.908+0000 7f93d6436740 1 -- 172.21.15.5:0/1432322312 shutdown_connections 2024-09-10T09:59:16.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:16.908+0000 7f93d6436740 1 --2- 172.21.15.5:0/1432322312 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e10951f020 0x55e109521410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:16.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:16.908+0000 7f93d6436740 1 --2- 172.21.15.5:0/1432322312 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e109514760 0x55e10951eae0 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:16.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:16.908+0000 7f93d6436740 1 --2- 172.21.15.5:0/1432322312 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e109513e40 0x55e109514220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:16.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:16.908+0000 7f93d6436740 1 -- 172.21.15.5:0/1432322312 >> 172.21.15.5:0/1432322312 conn(0x55e1095103d0 msgr2=0x55e1095107b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:16.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:16.909+0000 7f93d6436740 1 -- 172.21.15.5:0/1432322312 shutdown_connections 2024-09-10T09:59:16.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:16.909+0000 7f93d6436740 1 -- 172.21.15.5:0/1432322312 wait complete. 2024-09-10T09:59:16.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:16.909+0000 7f93d6436740 1 -- start start 2024-09-10T09:59:16.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:16.909+0000 7f93d6436740 1 Processor -- start 2024-09-10T09:59:16.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:16.910+0000 7f93d6436740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e109513e40 0x55e10951bef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:16.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:16.910+0000 7f93d6436740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e109514760 0x55e10951c430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:16.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:16.910+0000 7f93d6436740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e10951f020 0x55e10951c970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:16.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:16.910+0000 7f93d6436740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55e1095219b0 con 0x55e109513e40 2024-09-10T09:59:16.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:16.910+0000 7f93d6436740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55e1094350f0 con 0x55e109514760 2024-09-10T09:59:16.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:16.910+0000 7f93d6436740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55e109521b30 con 0x55e10951f020 2024-09-10T09:59:16.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:16.910+0000 7f93d6436740 10 client.0 _ll_register_callbacks cb 0x55e10941cbd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:59:16.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:16.910+0000 7f93d1000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e109513e40 0x55e10951bef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:16.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:16.910+0000 7f93cbe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e109514760 0x55e10951c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:16.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:16.910+0000 7f93d1000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e109513e40 0x55e10951bef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60474/0 (socket says 172.21.15.5:60474) 2024-09-10T09:59:16.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:16.910+0000 7f93d1000640 1 -- 172.21.15.5:0/3247560021 learned_addr learned my addr 172.21.15.5:0/3247560021 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:59:16.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:16.910+0000 7f93d1a00640 1 --2- 172.21.15.5:0/3247560021 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e10951f020 0x55e10951c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:16.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:16.910+0000 7f93cbe00640 1 -- 172.21.15.5:0/3247560021 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e10951f020 msgr2=0x55e10951c970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:59:16.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:16.910+0000 7f93cbe00640 1 --2- 172.21.15.5:0/3247560021 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e10951f020 0x55e10951c970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:16.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:16.910+0000 7f93cbe00640 1 -- 172.21.15.5:0/3247560021 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e109513e40 msgr2=0x55e10951bef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:59:16.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:16.910+0000 7f93cbe00640 1 --2- 172.21.15.5:0/3247560021 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e109513e40 0x55e10951bef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:16.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:16.910+0000 7f93cbe00640 1 -- 172.21.15.5:0/3247560021 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e1091eb3c0 con 0x55e109514760 2024-09-10T09:59:16.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:16.910+0000 7f93d1000640 1 --2- 172.21.15.5:0/3247560021 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e109513e40 0x55e10951bef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:59:16.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:16.911+0000 7f93cbe00640 1 --2- 172.21.15.5:0/3247560021 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e109514760 0x55e10951c430 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f93b8012cf0 tx=0x7f93b800f790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6320239b99848b5f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:16.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:16.911+0000 7f93b2a00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-09-10T09:59:16.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:16.911+0000 7f93b2a00640 1 -- 172.21.15.5:0/3247560021 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93b80120d0 con 0x55e109514760 2024-09-10T09:59:16.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:16.911+0000 7f93b2a00640 1 -- 172.21.15.5:0/3247560021 <== mon.2 v2:172.21.15.102:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f93b80047a0 con 0x55e109514760 2024-09-10T09:59:16.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:16.911+0000 7f93d6436740 1 -- 172.21.15.5:0/3247560021 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55e109487950 con 0x55e109514760 2024-09-10T09:59:16.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:16.911+0000 7f93b2a00640 1 -- 172.21.15.5:0/3247560021 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f93b8025050 con 0x55e109514760 2024-09-10T09:59:16.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:16.911+0000 7f93d6436740 10 client.29465 fetch_fsmap learned FSMap version 42 2024-09-10T09:59:16.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:16.911+0000 7f93d6436740 1 -- 172.21.15.5:0/3247560021 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55e10952d9a0 con 0x55e109514760 2024-09-10T09:59:16.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:16.912+0000 7f93b2a00640 1 -- 172.21.15.5:0/3247560021 <== mon.2 v2:172.21.15.102:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55e109487950 con 0x55e109514760 2024-09-10T09:59:16.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:16.912+0000 7f93b2a00640 1 -- 172.21.15.5:0/3247560021 <== mon.2 v2:172.21.15.102:3300/0 5 ==== fsmap(e 42) ==== 8370+0+0 (secure 0 0 0) 0x7f93b8010690 con 0x55e109514760 2024-09-10T09:59:16.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:16.912+0000 7f93d6436740 10 client.29465 fetch_fsmap finished waiting for FSMap version 42 2024-09-10T09:59:16.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:16.912+0000 7f93d6436740 10 client.29465 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-10T09:59:16.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:16.912+0000 7f93d6436740 20 client.29465 populate_metadata read hostname 'smithi005' 2024-09-10T09:59:16.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:16.912+0000 7f93d6436740 1 --2- 172.21.15.5:0/3247560021 >> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] conn(0x55e10952ee60 0x55e109531270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:16.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:16.912+0000 7f93d6436740 4 client.29465 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:59:16.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:16.912+0000 7f93d6436740 1 -- 172.21.15.5:0/3247560021 --> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e109481e60 con 0x55e10952ee60 2024-09-10T09:59:16.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:16.913+0000 7f93d1000640 1 --2- 172.21.15.5:0/3247560021 >> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] conn(0x55e10952ee60 0x55e109531270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:16.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:16.913+0000 7f93d1000640 1 --2- 172.21.15.5:0/3247560021 >> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] conn(0x55e10952ee60 0x55e109531270 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=33298421089dcc51 server_cookie=b524ec80c1b5274a in_seq=0 out_seq=0 2024-09-10T09:59:16.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:16.913+0000 7f93b2a00640 10 client.29465 ms_handle_connect on v2:172.21.15.159:6838/2073739599 2024-09-10T09:59:16.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:16.914+0000 7f93b2a00640 1 -- 172.21.15.5:0/3247560021 <== mds.1 v2:172.21.15.159:6838/2073739599 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e109481e60 con 0x55e10952ee60 2024-09-10T09:59:16.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:16.914+0000 7f93b2a00640 10 client.29465 handle_command_reply: tid=0 2024-09-10T09:59:16.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:16.914+0000 7f93d6436740 1 client.29465 shutdown 2024-09-10T09:59:16.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:16.914+0000 7f93d6436740 1 -- 172.21.15.5:0/3247560021 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e109514760 msgr2=0x55e10951c430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:16.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:16.914+0000 7f93d6436740 1 --2- 172.21.15.5:0/3247560021 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e109514760 0x55e10951c430 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f93b8012cf0 tx=0x7f93b800f790 comp rx=0 tx=0).stop 2024-09-10T09:59:16.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:16.915+0000 7f93d6436740 1 -- 172.21.15.5:0/3247560021 shutdown_connections 2024-09-10T09:59:16.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:16.915+0000 7f93d6436740 1 --2- 172.21.15.5:0/3247560021 >> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] conn(0x55e10952ee60 0x55e109531270 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:16.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:16.915+0000 7f93d6436740 1 --2- 172.21.15.5:0/3247560021 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e10951f020 0x55e10951c970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:16.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:16.915+0000 7f93d6436740 1 --2- 172.21.15.5:0/3247560021 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e109514760 0x55e10951c430 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:16.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:16.915+0000 7f93d6436740 1 --2- 172.21.15.5:0/3247560021 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e109513e40 0x55e10951bef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:16.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:16.915+0000 7f93b2a00640 0 client.29465 ms_handle_reset on v2:172.21.15.159:6838/2073739599 2024-09-10T09:59:16.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:16.915+0000 7f93d6436740 1 -- 172.21.15.5:0/3247560021 >> 172.21.15.5:0/3247560021 conn(0x55e1095103d0 msgr2=0x55e1095160b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:16.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:16.915+0000 7f93d6436740 1 -- 172.21.15.5:0/3247560021 shutdown_connections 2024-09-10T09:59:16.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:16.915+0000 7f93d6436740 1 -- 172.21.15.5:0/3247560021 wait complete. 2024-09-10T09:59:16.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:16.915+0000 7f93d6436740 20 client.29465 trim_cache size 0 max 16384 2024-09-10T09:59:16.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:16.929+0000 7f93d6436740 1 Processor -- start 2024-09-10T09:59:16.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:16.929+0000 7f93d6436740 1 -- start start 2024-09-10T09:59:16.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:16.929+0000 7f93d6436740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e109513e40 0x55e109515fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:16.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:16.929+0000 7f93d6436740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e109514760 0x55e109536b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:16.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:16.929+0000 7f93d6436740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e10951f020 0x55e1095370c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:16.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:16.929+0000 7f93d6436740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f93cc1b0e60 con 0x55e10951f020 2024-09-10T09:59:16.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:16.929+0000 7f93d6436740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f93cc1b29a0 con 0x55e109514760 2024-09-10T09:59:16.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:16.929+0000 7f93d6436740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f93cc1b1530 con 0x55e109513e40 2024-09-10T09:59:16.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:16.930+0000 7f93d1a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e10951f020 0x55e1095370c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:16.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:16.930+0000 7f93d1a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e10951f020 0x55e1095370c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60480/0 (socket says 172.21.15.5:60480) 2024-09-10T09:59:16.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:16.930+0000 7f93d1a00640 1 -- 172.21.15.5:0/3708420159 learned_addr learned my addr 172.21.15.5:0/3708420159 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:59:16.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:16.930+0000 7f93d1000640 1 --2- 172.21.15.5:0/3708420159 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e109513e40 0x55e109515fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:16.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:16.930+0000 7f93cbe00640 1 --2- 172.21.15.5:0/3708420159 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e109514760 0x55e109536b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:16.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:16.930+0000 7f93d1a00640 1 -- 172.21.15.5:0/3708420159 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e109513e40 msgr2=0x55e109515fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:59:16.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:16.930+0000 7f93d1a00640 1 --2- 172.21.15.5:0/3708420159 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e109513e40 0x55e109515fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:16.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:16.930+0000 7f93d1a00640 1 -- 172.21.15.5:0/3708420159 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e109514760 msgr2=0x55e109536b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:59:16.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:16.930+0000 7f93d1a00640 1 --2- 172.21.15.5:0/3708420159 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e109514760 0x55e109536b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:16.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:16.930+0000 7f93d1a00640 1 -- 172.21.15.5:0/3708420159 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e109510200 con 0x55e10951f020 2024-09-10T09:59:16.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:16.930+0000 7f93d1000640 1 --2- 172.21.15.5:0/3708420159 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e109513e40 0x55e109515fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:59:16.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:16.930+0000 7f93d1a00640 1 --2- 172.21.15.5:0/3708420159 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e10951f020 0x55e1095370c0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f93c4003fc0 tx=0x7f93c401a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b52951a902bddcde server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:16.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:16.931+0000 7f93bd600640 1 -- 172.21.15.5:0/3708420159 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93c4004b20 con 0x55e10951f020 2024-09-10T09:59:16.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:16.931+0000 7f93bd600640 1 -- 172.21.15.5:0/3708420159 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f93c4004cc0 con 0x55e10951f020 2024-09-10T09:59:16.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:16.931+0000 7f93bd600640 1 -- 172.21.15.5:0/3708420159 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93c4005520 con 0x55e10951f020 2024-09-10T09:59:16.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:16.931+0000 7f93d6436740 1 -- 172.21.15.5:0/3708420159 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e10951f020 msgr2=0x55e1095370c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:16.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:16.931+0000 7f93d6436740 1 --2- 172.21.15.5:0/3708420159 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e10951f020 0x55e1095370c0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f93c4003fc0 tx=0x7f93c401a000 comp rx=0 tx=0).stop 2024-09-10T09:59:16.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:16.931+0000 7f93d6436740 1 -- 172.21.15.5:0/3708420159 shutdown_connections 2024-09-10T09:59:16.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:16.931+0000 7f93d6436740 1 --2- 172.21.15.5:0/3708420159 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e10951f020 0x55e1095370c0 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:16.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:16.931+0000 7f93d6436740 1 --2- 172.21.15.5:0/3708420159 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e109514760 0x55e109536b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:16.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:16.931+0000 7f93d6436740 1 --2- 172.21.15.5:0/3708420159 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e109513e40 0x55e109515fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:16.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:16.931+0000 7f93d6436740 1 -- 172.21.15.5:0/3708420159 >> 172.21.15.5:0/3708420159 conn(0x55e1095103d0 msgr2=0x55e109515890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:16.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:16.932+0000 7f93d6436740 1 -- 172.21.15.5:0/3708420159 shutdown_connections 2024-09-10T09:59:16.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:16.932+0000 7f93d6436740 1 -- 172.21.15.5:0/3708420159 wait complete. 2024-09-10T09:59:16.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:16.932+0000 7f93d6436740 1 -- start start 2024-09-10T09:59:16.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:16.932+0000 7f93d6436740 1 Processor -- start 2024-09-10T09:59:16.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:16.933+0000 7f93d6436740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e109513e40 0x55e109522c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:16.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:16.933+0000 7f93d6436740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e109514760 0x55e109523160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:16.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:16.933+0000 7f93d6436740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e10951f020 0x55e1095236a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:16.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:16.933+0000 7f93d6436740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f93cc1b22d0 con 0x55e10951f020 2024-09-10T09:59:16.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:16.933+0000 7f93d6436740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f93cc1b1c00 con 0x55e109514760 2024-09-10T09:59:16.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:16.933+0000 7f93d6436740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f93cc1aee40 con 0x55e109513e40 2024-09-10T09:59:16.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:16.933+0000 7f93d6436740 10 client.0 _ll_register_callbacks cb 0x7f93b8009ce0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:59:16.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:16.933+0000 7f93d1000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e109513e40 0x55e109522c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:16.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:16.933+0000 7f93d1000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e109513e40 0x55e109522c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:57566/0 (socket says 172.21.15.5:57566) 2024-09-10T09:59:16.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:16.933+0000 7f93d1000640 1 -- 172.21.15.5:0/4265044689 learned_addr learned my addr 172.21.15.5:0/4265044689 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:59:16.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:16.933+0000 7f93d1a00640 1 --2- 172.21.15.5:0/4265044689 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e10951f020 0x55e1095236a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:16.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:16.933+0000 7f93cbe00640 1 --2- 172.21.15.5:0/4265044689 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e109514760 0x55e109523160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:16.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:16.934+0000 7f93d1000640 1 -- 172.21.15.5:0/4265044689 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e109514760 msgr2=0x55e109523160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:59:16.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:16.934+0000 7f93d1000640 1 --2- 172.21.15.5:0/4265044689 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e109514760 0x55e109523160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:16.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:16.934+0000 7f93d1000640 1 -- 172.21.15.5:0/4265044689 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e10951f020 msgr2=0x55e1095236a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:59:16.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:16.934+0000 7f93d1000640 1 --2- 172.21.15.5:0/4265044689 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e10951f020 0x55e1095236a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:16.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:16.934+0000 7f93d1000640 1 -- 172.21.15.5:0/4265044689 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e10950fc80 con 0x55e109513e40 2024-09-10T09:59:16.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:16.934+0000 7f93d1000640 1 --2- 172.21.15.5:0/4265044689 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e109513e40 0x55e109522c20 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f93c0032c00 tx=0x7f93c003b4f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f7a590de4f018bfa server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:16.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:16.934+0000 7f93b2a00640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-09-10T09:59:16.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:16.934+0000 7f93b2a00640 1 -- 172.21.15.5:0/4265044689 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93c0049040 con 0x55e109513e40 2024-09-10T09:59:16.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:16.934+0000 7f93b2a00640 1 -- 172.21.15.5:0/4265044689 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f93c0003cf0 con 0x55e109513e40 2024-09-10T09:59:16.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:16.934+0000 7f93d6436740 1 -- 172.21.15.5:0/4265044689 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f93c4004cc0 con 0x55e109513e40 2024-09-10T09:59:16.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:16.934+0000 7f93b2a00640 1 -- 172.21.15.5:0/4265044689 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f93c00bb680 con 0x55e109513e40 2024-09-10T09:59:16.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:16.934+0000 7f93d6436740 10 client.29449 fetch_fsmap learned FSMap version 42 2024-09-10T09:59:16.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:16.934+0000 7f93d6436740 1 -- 172.21.15.5:0/4265044689 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55e109599e90 con 0x55e109513e40 2024-09-10T09:59:16.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:16.935+0000 7f93b2a00640 1 -- 172.21.15.5:0/4265044689 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f93c4004cc0 con 0x55e109513e40 2024-09-10T09:59:16.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:16.935+0000 7f93b2a00640 1 -- 172.21.15.5:0/4265044689 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 42) ==== 8370+0+0 (secure 0 0 0) 0x7f93c00b8390 con 0x55e109513e40 2024-09-10T09:59:16.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:16.935+0000 7f93d6436740 10 client.29449 fetch_fsmap finished waiting for FSMap version 42 2024-09-10T09:59:16.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:16.935+0000 7f93d6436740 10 client.29449 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-10T09:59:16.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:16.935+0000 7f93d6436740 20 client.29449 populate_metadata read hostname 'smithi005' 2024-09-10T09:59:16.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:16.935+0000 7f93d6436740 1 --2- 172.21.15.5:0/4265044689 >> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] conn(0x55e10959a3c0 0x55e10959c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:16.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:16.935+0000 7f93d6436740 4 client.29449 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T09:59:16.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:16.935+0000 7f93d6436740 1 -- 172.21.15.5:0/4265044689 --> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e1093fe010 con 0x55e10959a3c0 2024-09-10T09:59:16.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:16.935+0000 7f93cbe00640 1 --2- 172.21.15.5:0/4265044689 >> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] conn(0x55e10959a3c0 0x55e10959c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:16.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:16.936+0000 7f93cbe00640 1 --2- 172.21.15.5:0/4265044689 >> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] conn(0x55e10959a3c0 0x55e10959c7b0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=fe433fde692328c3 server_cookie=17287987e7e412b8 in_seq=0 out_seq=0 2024-09-10T09:59:16.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:16.936+0000 7f93b2a00640 10 client.29449 ms_handle_connect on v2:172.21.15.159:6838/2073739599 2024-09-10T09:59:16.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:16.936+0000 7f93b2a00640 1 -- 172.21.15.5:0/4265044689 <== mds.1 v2:172.21.15.159:6838/2073739599 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e1093fe010 con 0x55e10959a3c0 2024-09-10T09:59:16.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:16.936+0000 7f93b2a00640 10 client.29449 handle_command_reply: tid=0 2024-09-10T09:59:16.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:16.936+0000 7f93d6436740 1 client.29449 shutdown 2024-09-10T09:59:16.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:16.936+0000 7f93d6436740 1 -- 172.21.15.5:0/4265044689 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e109513e40 msgr2=0x55e109522c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:16.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:16.936+0000 7f93d6436740 1 --2- 172.21.15.5:0/4265044689 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e109513e40 0x55e109522c20 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f93c0032c00 tx=0x7f93c003b4f0 comp rx=0 tx=0).stop 2024-09-10T09:59:16.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:16.936+0000 7f93d6436740 1 -- 172.21.15.5:0/4265044689 shutdown_connections 2024-09-10T09:59:16.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:16.936+0000 7f93d6436740 1 --2- 172.21.15.5:0/4265044689 >> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] conn(0x55e10959a3c0 0x55e10959c7b0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:16.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:16.936+0000 7f93d6436740 1 --2- 172.21.15.5:0/4265044689 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e10951f020 0x55e1095236a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:16.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:16.936+0000 7f93d6436740 1 --2- 172.21.15.5:0/4265044689 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e109514760 0x55e109523160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:16.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:16.936+0000 7f93d6436740 1 --2- 172.21.15.5:0/4265044689 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e109513e40 0x55e109522c20 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:16.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:16.936+0000 7f93b2a00640 0 client.29449 ms_handle_reset on v2:172.21.15.159:6838/2073739599 2024-09-10T09:59:16.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:16.936+0000 7f93d6436740 1 -- 172.21.15.5:0/4265044689 >> 172.21.15.5:0/4265044689 conn(0x55e1095103d0 msgr2=0x55e10950f490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:16.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:16.936+0000 7f93d6436740 1 -- 172.21.15.5:0/4265044689 shutdown_connections 2024-09-10T09:59:16.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:16.936+0000 7f93d6436740 1 -- 172.21.15.5:0/4265044689 wait complete. 2024-09-10T09:59:16.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:16.937+0000 7f93d6436740 20 client.29449 trim_cache size 0 max 16384 2024-09-10T09:59:16.942 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-10T09:59:16.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:16.937+0000 7f93d2400640 1 -- 172.21.15.5:0/377016074 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f93b40a2c00 msgr2=0x7f93b40a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:16.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:16.937+0000 7f93d2400640 1 --2- 172.21.15.5:0/377016074 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f93b40a2c00 0x7f93b40a50c0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f93c4005fd0 tx=0x7f93c4009500 comp rx=0 tx=0).stop 2024-09-10T09:59:16.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:16.937+0000 7f93d2400640 1 -- 172.21.15.5:0/377016074 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f93cc103e30 msgr2=0x7f93cc19a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:16.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:16.937+0000 7f93d2400640 1 --2- 172.21.15.5:0/377016074 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f93cc103e30 0x7f93cc19a480 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f93c0006010 tx=0x7f93c0004040 comp rx=0 tx=0).stop 2024-09-10T09:59:16.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:16.937+0000 7f93d2400640 1 -- 172.21.15.5:0/377016074 shutdown_connections 2024-09-10T09:59:16.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:16.937+0000 7f93d2400640 1 --2- 172.21.15.5:0/377016074 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f93b40a2c00 0x7f93b40a50c0 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:16.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:16.937+0000 7f93d2400640 1 --2- 172.21.15.5:0/377016074 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f93cc108f60 0x7f93cc19ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:16.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:16.937+0000 7f93d2400640 1 --2- 172.21.15.5:0/377016074 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f93cc1047e0 0x7f93cc19a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:16.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:16.937+0000 7f93d2400640 1 --2- 172.21.15.5:0/377016074 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f93cc103e30 0x7f93cc19a480 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:16.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:16.937+0000 7f93d2400640 1 -- 172.21.15.5:0/377016074 >> 172.21.15.5:0/377016074 conn(0x7f93cc0ff910 msgr2=0x7f93cc100d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:16.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:16.938+0000 7f93d2400640 1 -- 172.21.15.5:0/377016074 shutdown_connections 2024-09-10T09:59:16.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:16.938+0000 7f93d2400640 1 -- 172.21.15.5:0/377016074 wait complete. 2024-09-10T09:59:17.026 INFO:tasks.workunit.client.0.smithi005.stdout: 10 110875 13.12 MB/sec execute 295 sec latency 19691.104 ms 2024-09-10T09:59:17.829 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-10T09:59:18.026 INFO:tasks.workunit.client.0.smithi005.stdout: 10 110875 13.08 MB/sec execute 296 sec latency 20691.169 ms 2024-09-10T09:59:18.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:59:17 smithi005 ceph-mon[30350]: pgmap v1021: 129 pgs: 129 active+clean; 649 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 2.1 MiB/s wr, 80 op/s 2024-09-10T09:59:18.205 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:59:18.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:59:17 smithi159 ceph-mon[35513]: pgmap v1021: 129 pgs: 129 active+clean; 649 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 2.1 MiB/s wr, 80 op/s 2024-09-10T09:59:18.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:59:17 smithi102 ceph-mon[35442]: pgmap v1021: 129 pgs: 129 active+clean; 649 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 2.1 MiB/s wr, 80 op/s 2024-09-10T09:59:18.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:18.744+0000 7f8153c00640 1 -- 172.21.15.5:0/1470466533 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f814c077040 msgr2=0x7f814c1124f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:18.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:18.744+0000 7f8153c00640 1 --2- 172.21.15.5:0/1470466533 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f814c077040 0x7f814c1124f0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f8138009900 tx=0x7f813802a3f0 comp rx=0 tx=0).stop 2024-09-10T09:59:18.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:18.744+0000 7f8153c00640 1 -- 172.21.15.5:0/1470466533 shutdown_connections 2024-09-10T09:59:18.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:18.744+0000 7f8153c00640 1 --2- 172.21.15.5:0/1470466533 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f814c112a30 0x7f814c114e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:18.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:18.744+0000 7f8153c00640 1 --2- 172.21.15.5:0/1470466533 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f814c077040 0x7f814c1124f0 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:18.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:18.744+0000 7f8153c00640 1 --2- 172.21.15.5:0/1470466533 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f814c076720 0x7f814c076b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:18.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:18.744+0000 7f8153c00640 1 -- 172.21.15.5:0/1470466533 >> 172.21.15.5:0/1470466533 conn(0x7f814c0ffa90 msgr2=0x7f814c101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:18.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:18.744+0000 7f8153c00640 1 -- 172.21.15.5:0/1470466533 shutdown_connections 2024-09-10T09:59:18.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:18.745+0000 7f8153c00640 1 -- 172.21.15.5:0/1470466533 wait complete. 2024-09-10T09:59:18.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:18.745+0000 7f8153c00640 1 Processor -- start 2024-09-10T09:59:18.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:18.746+0000 7f8153c00640 1 -- start start 2024-09-10T09:59:18.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:18.746+0000 7f8153c00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f814c076720 0x7f814c1a4710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:18.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:18.746+0000 7f8153c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f814c077040 0x7f814c1a4c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:18.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:18.746+0000 7f8153c00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f814c112a30 0x7f814c19e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:18.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:18.746+0000 7f8153c00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f814c1185c0 con 0x7f814c112a30 2024-09-10T09:59:18.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:18.746+0000 7f8153c00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f814c118440 con 0x7f814c077040 2024-09-10T09:59:18.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:18.746+0000 7f8153c00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f814c118740 con 0x7f814c076720 2024-09-10T09:59:18.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:18.746+0000 7f8151c00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f814c112a30 0x7f814c19e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:18.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:18.746+0000 7f814be00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f814c077040 0x7f814c1a4c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:18.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:18.746+0000 7f814be00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f814c077040 0x7f814c1a4c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:35942/0 (socket says 172.21.15.5:35942) 2024-09-10T09:59:18.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:18.746+0000 7f814be00640 1 -- 172.21.15.5:0/4007212973 learned_addr learned my addr 172.21.15.5:0/4007212973 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:59:18.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:18.746+0000 7f8151200640 1 --2- 172.21.15.5:0/4007212973 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f814c076720 0x7f814c1a4710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:18.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:18.747+0000 7f814be00640 1 -- 172.21.15.5:0/4007212973 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f814c076720 msgr2=0x7f814c1a4710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:18.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:18.747+0000 7f814be00640 1 --2- 172.21.15.5:0/4007212973 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f814c076720 0x7f814c1a4710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:18.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:18.747+0000 7f814be00640 1 -- 172.21.15.5:0/4007212973 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f814c112a30 msgr2=0x7f814c19e8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:18.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:18.747+0000 7f814be00640 1 --2- 172.21.15.5:0/4007212973 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f814c112a30 0x7f814c19e8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:18.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:18.747+0000 7f814be00640 1 -- 172.21.15.5:0/4007212973 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f814c19f000 con 0x7f814c077040 2024-09-10T09:59:18.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:18.747+0000 7f814be00640 1 --2- 172.21.15.5:0/4007212973 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f814c077040 0x7f814c1a4c50 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f8138006010 tx=0x7f8138004040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:18.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:18.747+0000 7f8149600640 1 -- 172.21.15.5:0/4007212973 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f813803f030 con 0x7f814c077040 2024-09-10T09:59:18.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:18.747+0000 7f8153c00640 1 -- 172.21.15.5:0/4007212973 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f814c19f290 con 0x7f814c077040 2024-09-10T09:59:18.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:18.747+0000 7f8149600640 1 -- 172.21.15.5:0/4007212973 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8138004440 con 0x7f814c077040 2024-09-10T09:59:18.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:18.747+0000 7f8149600640 1 -- 172.21.15.5:0/4007212973 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8138035070 con 0x7f814c077040 2024-09-10T09:59:18.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:18.747+0000 7f8153c00640 1 -- 172.21.15.5:0/4007212973 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f814c1ab5e0 con 0x7f814c077040 2024-09-10T09:59:18.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:18.750+0000 7f815664e740 1 Processor -- start 2024-09-10T09:59:18.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:18.751+0000 7f8149600640 1 -- 172.21.15.5:0/4007212973 <== mon.2 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f8138042030 con 0x7f814c077040 2024-09-10T09:59:18.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:18.751+0000 7f815664e740 1 -- start start 2024-09-10T09:59:18.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:18.751+0000 7f815664e740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56052a8f3c70 0x56052a8f4050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:18.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:18.751+0000 7f815664e740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56052a8f4590 0x56052a8fe910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:18.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:18.751+0000 7f815664e740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56052a8fee50 0x56052a901240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:18.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:18.751+0000 7f815664e740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x56052a85a3e0 con 0x56052a8fee50 2024-09-10T09:59:18.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:18.751+0000 7f815664e740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x56052a801d80 con 0x56052a8f3c70 2024-09-10T09:59:18.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:18.751+0000 7f815664e740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x56052a8563a0 con 0x56052a8f4590 2024-09-10T09:59:18.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:18.751+0000 7f8151c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56052a8f3c70 0x56052a8f4050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:18.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:18.751+0000 7f814be00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56052a8fee50 0x56052a901240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:18.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:18.751+0000 7f8151c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56052a8f3c70 0x56052a8f4050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:35956/0 (socket says 172.21.15.5:35956) 2024-09-10T09:59:18.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:18.751+0000 7f8151c00640 1 -- 172.21.15.5:0/2821081076 learned_addr learned my addr 172.21.15.5:0/2821081076 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:59:18.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:18.752+0000 7f8151200640 1 --2- 172.21.15.5:0/2821081076 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56052a8f4590 0x56052a8fe910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:18.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:18.752+0000 7f814be00640 1 -- 172.21.15.5:0/2821081076 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56052a8f4590 msgr2=0x56052a8fe910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:59:18.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:18.752+0000 7f814be00640 1 --2- 172.21.15.5:0/2821081076 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56052a8f4590 0x56052a8fe910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:18.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:18.752+0000 7f814be00640 1 -- 172.21.15.5:0/2821081076 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56052a8f3c70 msgr2=0x56052a8f4050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:59:18.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:18.752+0000 7f814be00640 1 --2- 172.21.15.5:0/2821081076 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56052a8f3c70 0x56052a8f4050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:18.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:18.752+0000 7f814be00640 1 -- 172.21.15.5:0/2821081076 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56052a820ae0 con 0x56052a8fee50 2024-09-10T09:59:18.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:18.752+0000 7f814be00640 1 --2- 172.21.15.5:0/2821081076 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56052a8fee50 0x56052a901240 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f81380ecd40 tx=0x7f81380f5cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c66f9685f64b70c5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:18.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:18.753+0000 7f8149600640 1 --2- 172.21.15.5:0/4007212973 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f81340a2b80 0x7f81340a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:18.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:18.753+0000 7f813e000640 1 -- 172.21.15.5:0/2821081076 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81380fe070 con 0x56052a8fee50 2024-09-10T09:59:18.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:18.753+0000 7f813e000640 1 -- 172.21.15.5:0/2821081076 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f81380bc200 con 0x56052a8fee50 2024-09-10T09:59:18.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:18.753+0000 7f813e000640 1 -- 172.21.15.5:0/2821081076 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81380bb460 con 0x56052a8fee50 2024-09-10T09:59:18.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:18.753+0000 7f8149600640 1 -- 172.21.15.5:0/4007212973 <== mon.2 v2:172.21.15.102:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f81380ef6f0 con 0x7f814c077040 2024-09-10T09:59:18.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:18.753+0000 7f8151c00640 1 --2- 172.21.15.5:0/4007212973 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f81340a2b80 0x7f81340a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:18.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:18.754+0000 7f815664e740 1 -- 172.21.15.5:0/2821081076 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56052a8fee50 msgr2=0x56052a901240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:18.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:18.754+0000 7f815664e740 1 --2- 172.21.15.5:0/2821081076 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56052a8fee50 0x56052a901240 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f81380ecd40 tx=0x7f81380f5cb0 comp rx=0 tx=0).stop 2024-09-10T09:59:18.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:18.754+0000 7f8151c00640 1 --2- 172.21.15.5:0/4007212973 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f81340a2b80 0x7f81340a5040 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f8144012640 tx=0x7f8144009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:18.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:18.754+0000 7f815664e740 1 -- 172.21.15.5:0/2821081076 shutdown_connections 2024-09-10T09:59:18.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:18.754+0000 7f815664e740 1 --2- 172.21.15.5:0/2821081076 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56052a8fee50 0x56052a901240 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:18.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:18.754+0000 7f815664e740 1 --2- 172.21.15.5:0/2821081076 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56052a8f4590 0x56052a8fe910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:18.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:18.754+0000 7f815664e740 1 --2- 172.21.15.5:0/2821081076 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56052a8f3c70 0x56052a8f4050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:18.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:18.754+0000 7f815664e740 1 -- 172.21.15.5:0/2821081076 >> 172.21.15.5:0/2821081076 conn(0x56052a8f0200 msgr2=0x56052a8f05e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:18.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:18.754+0000 7f815664e740 1 -- 172.21.15.5:0/2821081076 shutdown_connections 2024-09-10T09:59:18.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:18.754+0000 7f815664e740 1 -- 172.21.15.5:0/2821081076 wait complete. 2024-09-10T09:59:18.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:18.754+0000 7f815664e740 1 -- start start 2024-09-10T09:59:18.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:18.755+0000 7f815664e740 1 Processor -- start 2024-09-10T09:59:18.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:18.755+0000 7f815664e740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56052a8f3c70 0x56052a8fbcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:18.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:18.755+0000 7f815664e740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56052a8f4590 0x56052a8fc1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:18.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:18.755+0000 7f815664e740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56052a8fee50 0x56052a8fc730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:18.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:18.755+0000 7f815664e740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x56052a9017e0 con 0x56052a8f3c70 2024-09-10T09:59:18.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:18.755+0000 7f815664e740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x56052a8150f0 con 0x56052a8fee50 2024-09-10T09:59:18.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:18.755+0000 7f815664e740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x56052a901960 con 0x56052a8f4590 2024-09-10T09:59:18.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:18.755+0000 7f815664e740 10 client.0 _ll_register_callbacks cb 0x56052a7fcbd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:59:18.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:18.755+0000 7f8151200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56052a8f3c70 0x56052a8fbcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:18.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:18.755+0000 7f8151c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56052a8fee50 0x56052a8fc730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:18.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:18.756+0000 7f8151200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56052a8f3c70 0x56052a8fbcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60538/0 (socket says 172.21.15.5:60538) 2024-09-10T09:59:18.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:18.756+0000 7f8151200640 1 -- 172.21.15.5:0/171914148 learned_addr learned my addr 172.21.15.5:0/171914148 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:59:18.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:18.756+0000 7f814be00640 1 --2- 172.21.15.5:0/171914148 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56052a8f4590 0x56052a8fc1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:18.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:18.756+0000 7f8151c00640 1 -- 172.21.15.5:0/171914148 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56052a8f4590 msgr2=0x56052a8fc1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:59:18.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:18.756+0000 7f8151c00640 1 --2- 172.21.15.5:0/171914148 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56052a8f4590 0x56052a8fc1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:18.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:18.756+0000 7f8151c00640 1 -- 172.21.15.5:0/171914148 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56052a8f3c70 msgr2=0x56052a8fbcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:59:18.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:18.756+0000 7f8151c00640 1 --2- 172.21.15.5:0/171914148 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56052a8f3c70 0x56052a8fbcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:18.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:18.756+0000 7f8151c00640 1 -- 172.21.15.5:0/171914148 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56052a5cb3c0 con 0x56052a8fee50 2024-09-10T09:59:18.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:18.756+0000 7f814be00640 1 --2- 172.21.15.5:0/171914148 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56052a8f4590 0x56052a8fc1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:59:18.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:18.756+0000 7f8151200640 1 --2- 172.21.15.5:0/171914148 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56052a8f3c70 0x56052a8fbcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:59:18.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:18.756+0000 7f8151c00640 1 --2- 172.21.15.5:0/171914148 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56052a8fee50 0x56052a8fc730 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f8144012ed0 tx=0x7f8144004ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=db8cec5513af82e9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:18.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:18.756+0000 7f8132a00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-09-10T09:59:18.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:18.757+0000 7f8132a00640 1 -- 172.21.15.5:0/171914148 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f814400f870 con 0x56052a8fee50 2024-09-10T09:59:18.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:18.757+0000 7f815664e740 1 -- 172.21.15.5:0/171914148 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f81380bc200 con 0x56052a8fee50 2024-09-10T09:59:18.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:18.757+0000 7f8132a00640 1 -- 172.21.15.5:0/171914148 <== mon.2 v2:172.21.15.102:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f814400fa10 con 0x56052a8fee50 2024-09-10T09:59:18.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:18.757+0000 7f8132a00640 1 -- 172.21.15.5:0/171914148 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f814400fc10 con 0x56052a8fee50 2024-09-10T09:59:18.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:18.757+0000 7f815664e740 10 client.29483 fetch_fsmap learned FSMap version 42 2024-09-10T09:59:18.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:18.757+0000 7f815664e740 1 -- 172.21.15.5:0/171914148 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x56052a90d760 con 0x56052a8fee50 2024-09-10T09:59:18.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:18.757+0000 7f8132a00640 1 -- 172.21.15.5:0/171914148 <== mon.2 v2:172.21.15.102:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f81380bc200 con 0x56052a8fee50 2024-09-10T09:59:18.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:18.758+0000 7f8132a00640 1 -- 172.21.15.5:0/171914148 <== mon.2 v2:172.21.15.102:3300/0 5 ==== fsmap(e 42) ==== 8370+0+0 (secure 0 0 0) 0x7f8144027020 con 0x56052a8fee50 2024-09-10T09:59:18.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:18.758+0000 7f815664e740 10 client.29483 fetch_fsmap finished waiting for FSMap version 42 2024-09-10T09:59:18.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:18.758+0000 7f815664e740 10 client.29483 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T09:59:18.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:18.758+0000 7f815664e740 20 client.29483 populate_metadata read hostname 'smithi005' 2024-09-10T09:59:18.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:18.758+0000 7f815664e740 1 --2- 172.21.15.5:0/171914148 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x56052a90ebc0 0x56052a911010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:18.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:18.758+0000 7f815664e740 4 client.29483 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:59:18.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:18.758+0000 7f815664e740 1 -- 172.21.15.5:0/171914148 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56052a867950 con 0x56052a90ebc0 2024-09-10T09:59:18.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:18.758+0000 7f8151200640 1 --2- 172.21.15.5:0/171914148 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x56052a90ebc0 0x56052a911010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:18.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:18.759+0000 7f8151200640 1 --2- 172.21.15.5:0/171914148 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x56052a90ebc0 0x56052a911010 crc :-1 s=READY pgs=291 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d8d0d4b7093a90c9 server_cookie=2881b71716305e7f in_seq=0 out_seq=0 2024-09-10T09:59:18.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:18.759+0000 7f8132a00640 10 client.29483 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T09:59:18.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:18.759+0000 7f8132a00640 1 -- 172.21.15.5:0/171914148 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56052a867950 con 0x56052a90ebc0 2024-09-10T09:59:18.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:18.759+0000 7f8132a00640 10 client.29483 handle_command_reply: tid=0 2024-09-10T09:59:18.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:18.760+0000 7f815664e740 1 client.29483 shutdown 2024-09-10T09:59:18.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:18.760+0000 7f815664e740 1 -- 172.21.15.5:0/171914148 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56052a8fee50 msgr2=0x56052a8fc730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:18.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:18.760+0000 7f815664e740 1 --2- 172.21.15.5:0/171914148 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56052a8fee50 0x56052a8fc730 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f8144012ed0 tx=0x7f8144004ab0 comp rx=0 tx=0).stop 2024-09-10T09:59:18.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:18.760+0000 7f815664e740 1 -- 172.21.15.5:0/171914148 shutdown_connections 2024-09-10T09:59:18.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:18.760+0000 7f815664e740 1 --2- 172.21.15.5:0/171914148 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x56052a90ebc0 0x56052a911010 crc :-1 s=READY pgs=291 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:18.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:18.760+0000 7f815664e740 1 --2- 172.21.15.5:0/171914148 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56052a8fee50 0x56052a8fc730 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:18.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:18.760+0000 7f815664e740 1 --2- 172.21.15.5:0/171914148 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56052a8f4590 0x56052a8fc1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:18.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:18.760+0000 7f815664e740 1 --2- 172.21.15.5:0/171914148 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56052a8f3c70 0x56052a8fbcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:18.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:18.760+0000 7f815664e740 1 -- 172.21.15.5:0/171914148 >> 172.21.15.5:0/171914148 conn(0x56052a8f0200 msgr2=0x56052a8ef380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:18.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:18.760+0000 7f8132a00640 0 client.29483 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T09:59:18.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:18.760+0000 7f815664e740 1 -- 172.21.15.5:0/171914148 shutdown_connections 2024-09-10T09:59:18.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:18.760+0000 7f815664e740 1 -- 172.21.15.5:0/171914148 wait complete. 2024-09-10T09:59:18.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:18.760+0000 7f815664e740 20 client.29483 trim_cache size 0 max 16384 2024-09-10T09:59:18.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:18.774+0000 7f815664e740 1 Processor -- start 2024-09-10T09:59:18.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:18.774+0000 7f815664e740 1 -- start start 2024-09-10T09:59:18.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:18.774+0000 7f815664e740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56052a8f3c70 0x56052a8fb570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:18.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:18.774+0000 7f815664e740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56052a8f4590 0x56052a8fdc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:18.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:18.774+0000 7f815664e740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56052a8fee50 0x56052a916930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:18.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:18.774+0000 7f815664e740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f814c1b53e0 con 0x56052a8f4590 2024-09-10T09:59:18.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:18.774+0000 7f815664e740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f814c1b6f20 con 0x56052a8fee50 2024-09-10T09:59:18.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:18.774+0000 7f815664e740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f814c1b5ab0 con 0x56052a8f3c70 2024-09-10T09:59:18.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:18.774+0000 7f814be00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56052a8f4590 0x56052a8fdc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:18.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:18.775+0000 7f814be00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56052a8f4590 0x56052a8fdc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60550/0 (socket says 172.21.15.5:60550) 2024-09-10T09:59:18.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:18.775+0000 7f814be00640 1 -- 172.21.15.5:0/3079103513 learned_addr learned my addr 172.21.15.5:0/3079103513 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:59:18.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:18.775+0000 7f8151200640 1 --2- 172.21.15.5:0/3079103513 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56052a8f3c70 0x56052a8fb570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:18.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:18.775+0000 7f8151c00640 1 --2- 172.21.15.5:0/3079103513 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56052a8fee50 0x56052a916930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:18.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:18.775+0000 7f814be00640 1 -- 172.21.15.5:0/3079103513 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56052a8f3c70 msgr2=0x56052a8fb570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:59:18.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:18.775+0000 7f814be00640 1 --2- 172.21.15.5:0/3079103513 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56052a8f3c70 0x56052a8fb570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:18.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:18.775+0000 7f814be00640 1 -- 172.21.15.5:0/3079103513 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56052a8fee50 msgr2=0x56052a916930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:59:18.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:18.775+0000 7f814be00640 1 --2- 172.21.15.5:0/3079103513 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56052a8fee50 0x56052a916930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:18.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:18.775+0000 7f814be00640 1 -- 172.21.15.5:0/3079103513 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56052a8f43c0 con 0x56052a8f4590 2024-09-10T09:59:18.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:18.775+0000 7f8151200640 1 --2- 172.21.15.5:0/3079103513 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56052a8f3c70 0x56052a8fb570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:59:18.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:18.775+0000 7f8151c00640 1 --2- 172.21.15.5:0/3079103513 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56052a8fee50 0x56052a916930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:59:18.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:18.775+0000 7f814be00640 1 --2- 172.21.15.5:0/3079103513 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56052a8f4590 0x56052a8fdc20 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f813808e9f0 tx=0x7f81380bb640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0ffcc7e2f98ece1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:18.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:18.776+0000 7f813d600640 1 -- 172.21.15.5:0/3079103513 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f813808e650 con 0x56052a8f4590 2024-09-10T09:59:18.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:18.776+0000 7f813d600640 1 -- 172.21.15.5:0/3079103513 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f813808b150 con 0x56052a8f4590 2024-09-10T09:59:18.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:18.776+0000 7f813d600640 1 -- 172.21.15.5:0/3079103513 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f813808e830 con 0x56052a8f4590 2024-09-10T09:59:18.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:18.776+0000 7f815664e740 1 -- 172.21.15.5:0/3079103513 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56052a8f4590 msgr2=0x56052a8fdc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:18.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:18.776+0000 7f815664e740 1 --2- 172.21.15.5:0/3079103513 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56052a8f4590 0x56052a8fdc20 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f813808e9f0 tx=0x7f81380bb640 comp rx=0 tx=0).stop 2024-09-10T09:59:18.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:18.776+0000 7f815664e740 1 -- 172.21.15.5:0/3079103513 shutdown_connections 2024-09-10T09:59:18.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:18.776+0000 7f815664e740 1 --2- 172.21.15.5:0/3079103513 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56052a8fee50 0x56052a916930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:18.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:18.776+0000 7f815664e740 1 --2- 172.21.15.5:0/3079103513 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56052a8f4590 0x56052a8fdc20 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:18.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:18.776+0000 7f815664e740 1 --2- 172.21.15.5:0/3079103513 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56052a8f3c70 0x56052a8fb570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:18.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:18.776+0000 7f815664e740 1 -- 172.21.15.5:0/3079103513 >> 172.21.15.5:0/3079103513 conn(0x56052a8f0200 msgr2=0x56052a8fd510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:18.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:18.776+0000 7f815664e740 1 -- 172.21.15.5:0/3079103513 shutdown_connections 2024-09-10T09:59:18.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:18.776+0000 7f815664e740 1 -- 172.21.15.5:0/3079103513 wait complete. 2024-09-10T09:59:18.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:18.777+0000 7f815664e740 1 -- start start 2024-09-10T09:59:18.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:18.777+0000 7f815664e740 1 Processor -- start 2024-09-10T09:59:18.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:18.777+0000 7f815664e740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56052a8f3c70 0x56052a906f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:18.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:18.777+0000 7f815664e740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56052a8f4590 0x56052a9022a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:18.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:18.777+0000 7f815664e740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56052a8fee50 0x56052a9027e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:18.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:18.777+0000 7f815664e740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f814c1b6850 con 0x56052a8f3c70 2024-09-10T09:59:18.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:18.777+0000 7f815664e740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f814c1b6180 con 0x56052a8fee50 2024-09-10T09:59:18.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:18.777+0000 7f815664e740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f814c1b33c0 con 0x56052a8f4590 2024-09-10T09:59:18.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:18.777+0000 7f815664e740 10 client.0 _ll_register_callbacks cb 0x7f8144009e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:59:18.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:18.778+0000 7f8151200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56052a8f3c70 0x56052a906f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:18.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:18.778+0000 7f8151200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56052a8f3c70 0x56052a906f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60552/0 (socket says 172.21.15.5:60552) 2024-09-10T09:59:18.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:18.778+0000 7f8151200640 1 -- 172.21.15.5:0/1959851873 learned_addr learned my addr 172.21.15.5:0/1959851873 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:59:18.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:18.778+0000 7f814be00640 1 --2- 172.21.15.5:0/1959851873 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56052a8f4590 0x56052a9022a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:18.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:18.778+0000 7f8151c00640 1 --2- 172.21.15.5:0/1959851873 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56052a8fee50 0x56052a9027e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:18.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:18.778+0000 7f8151200640 1 -- 172.21.15.5:0/1959851873 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56052a8f4590 msgr2=0x56052a9022a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:59:18.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:18.778+0000 7f8151200640 1 --2- 172.21.15.5:0/1959851873 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56052a8f4590 0x56052a9022a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:18.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:18.778+0000 7f8151200640 1 -- 172.21.15.5:0/1959851873 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56052a8fee50 msgr2=0x56052a9027e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:59:18.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:18.778+0000 7f8151200640 1 --2- 172.21.15.5:0/1959851873 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56052a8fee50 0x56052a9027e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:18.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:18.778+0000 7f8151200640 1 -- 172.21.15.5:0/1959851873 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56052a902d20 con 0x56052a8f3c70 2024-09-10T09:59:18.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:18.779+0000 7f8151200640 1 --2- 172.21.15.5:0/1959851873 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56052a8f3c70 0x56052a906f20 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f8140004700 tx=0x7f8140012b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d19c2767ff51038f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:18.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:18.779+0000 7f8132a00640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-10T09:59:18.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:18.779+0000 7f8132a00640 1 -- 172.21.15.5:0/1959851873 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f814002c060 con 0x56052a8f3c70 2024-09-10T09:59:18.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:18.779+0000 7f8132a00640 1 -- 172.21.15.5:0/1959851873 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f8140011c80 con 0x56052a8f3c70 2024-09-10T09:59:18.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:18.779+0000 7f815664e740 1 -- 172.21.15.5:0/1959851873 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x56052a861e60 con 0x56052a8f3c70 2024-09-10T09:59:18.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:18.779+0000 7f8132a00640 1 -- 172.21.15.5:0/1959851873 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f8140033050 con 0x56052a8f3c70 2024-09-10T09:59:18.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:18.779+0000 7f815664e740 10 client.19422 fetch_fsmap learned FSMap version 42 2024-09-10T09:59:18.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:18.779+0000 7f815664e740 1 -- 172.21.15.5:0/1959851873 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x56052a979610 con 0x56052a8f3c70 2024-09-10T09:59:18.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:18.780+0000 7f8132a00640 1 -- 172.21.15.5:0/1959851873 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x56052a861e60 con 0x56052a8f3c70 2024-09-10T09:59:18.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:18.780+0000 7f8132a00640 1 -- 172.21.15.5:0/1959851873 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 42) ==== 8370+0+0 (secure 0 0 0) 0x7f81400263b0 con 0x56052a8f3c70 2024-09-10T09:59:18.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:18.780+0000 7f815664e740 10 client.19422 fetch_fsmap finished waiting for FSMap version 42 2024-09-10T09:59:18.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:18.780+0000 7f815664e740 10 client.19422 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T09:59:18.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:18.780+0000 7f815664e740 20 client.19422 populate_metadata read hostname 'smithi005' 2024-09-10T09:59:18.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:18.780+0000 7f815664e740 1 --2- 172.21.15.5:0/1959851873 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x56052a979d10 0x56052a97c130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:18.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:18.780+0000 7f815664e740 4 client.19422 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T09:59:18.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:18.780+0000 7f815664e740 1 -- 172.21.15.5:0/1959851873 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56052a7de010 con 0x56052a979d10 2024-09-10T09:59:18.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:18.780+0000 7f814be00640 1 --2- 172.21.15.5:0/1959851873 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x56052a979d10 0x56052a97c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:18.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:18.781+0000 7f814be00640 1 --2- 172.21.15.5:0/1959851873 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x56052a979d10 0x56052a97c130 crc :-1 s=READY pgs=292 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7231bb14751d4ece server_cookie=7dbbbccd69e3851e in_seq=0 out_seq=0 2024-09-10T09:59:18.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:18.781+0000 7f8132a00640 10 client.19422 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T09:59:18.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:18.781+0000 7f8132a00640 1 -- 172.21.15.5:0/1959851873 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56052a7de010 con 0x56052a979d10 2024-09-10T09:59:18.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:18.781+0000 7f8132a00640 10 client.19422 handle_command_reply: tid=0 2024-09-10T09:59:18.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:18.781+0000 7f815664e740 1 client.19422 shutdown 2024-09-10T09:59:18.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:18.781+0000 7f815664e740 1 -- 172.21.15.5:0/1959851873 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56052a8f3c70 msgr2=0x56052a906f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:18.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:18.781+0000 7f815664e740 1 --2- 172.21.15.5:0/1959851873 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56052a8f3c70 0x56052a906f20 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f8140004700 tx=0x7f8140012b60 comp rx=0 tx=0).stop 2024-09-10T09:59:18.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:18.781+0000 7f815664e740 1 -- 172.21.15.5:0/1959851873 shutdown_connections 2024-09-10T09:59:18.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:18.781+0000 7f815664e740 1 --2- 172.21.15.5:0/1959851873 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x56052a979d10 0x56052a97c130 crc :-1 s=READY pgs=292 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:18.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:18.781+0000 7f815664e740 1 --2- 172.21.15.5:0/1959851873 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56052a8fee50 0x56052a9027e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:18.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:18.781+0000 7f815664e740 1 --2- 172.21.15.5:0/1959851873 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56052a8f4590 0x56052a9022a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:18.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:18.781+0000 7f815664e740 1 --2- 172.21.15.5:0/1959851873 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56052a8f3c70 0x56052a906f20 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:18.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:18.781+0000 7f815664e740 1 -- 172.21.15.5:0/1959851873 >> 172.21.15.5:0/1959851873 conn(0x56052a8f0200 msgr2=0x56052a8f56c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:18.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:18.781+0000 7f8132a00640 0 client.19422 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T09:59:18.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:18.781+0000 7f815664e740 1 -- 172.21.15.5:0/1959851873 shutdown_connections 2024-09-10T09:59:18.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:18.782+0000 7f815664e740 1 -- 172.21.15.5:0/1959851873 wait complete. 2024-09-10T09:59:18.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:18.782+0000 7f815664e740 20 client.19422 trim_cache size 0 max 16384 2024-09-10T09:59:18.786 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-10T09:59:18.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:18.782+0000 7f8153c00640 1 -- 172.21.15.5:0/4007212973 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f81340a2b80 msgr2=0x7f81340a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:18.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:18.782+0000 7f8153c00640 1 --2- 172.21.15.5:0/4007212973 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f81340a2b80 0x7f81340a5040 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f8144012640 tx=0x7f8144009500 comp rx=0 tx=0).stop 2024-09-10T09:59:18.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:18.782+0000 7f8153c00640 1 -- 172.21.15.5:0/4007212973 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f814c077040 msgr2=0x7f814c1a4c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:18.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:18.782+0000 7f8153c00640 1 --2- 172.21.15.5:0/4007212973 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f814c077040 0x7f814c1a4c50 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f8138006010 tx=0x7f8138004040 comp rx=0 tx=0).stop 2024-09-10T09:59:18.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:18.782+0000 7f8153c00640 1 -- 172.21.15.5:0/4007212973 shutdown_connections 2024-09-10T09:59:18.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:18.782+0000 7f8153c00640 1 --2- 172.21.15.5:0/4007212973 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f81340a2b80 0x7f81340a5040 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:18.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:18.782+0000 7f8153c00640 1 --2- 172.21.15.5:0/4007212973 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f814c112a30 0x7f814c19e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:18.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:18.782+0000 7f8153c00640 1 --2- 172.21.15.5:0/4007212973 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f814c077040 0x7f814c1a4c50 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:18.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:18.782+0000 7f8153c00640 1 --2- 172.21.15.5:0/4007212973 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f814c076720 0x7f814c1a4710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:18.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:18.782+0000 7f8153c00640 1 -- 172.21.15.5:0/4007212973 >> 172.21.15.5:0/4007212973 conn(0x7f814c0ffa90 msgr2=0x7f814c101e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:18.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:18.783+0000 7f8153c00640 1 -- 172.21.15.5:0/4007212973 shutdown_connections 2024-09-10T09:59:18.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:18.783+0000 7f8153c00640 1 -- 172.21.15.5:0/4007212973 wait complete. 2024-09-10T09:59:19.026 INFO:tasks.workunit.client.0.smithi005.stdout: 10 110875 13.03 MB/sec execute 297 sec latency 21691.211 ms 2024-09-10T09:59:20.026 INFO:tasks.workunit.client.0.smithi005.stdout: 10 110875 12.99 MB/sec execute 298 sec latency 22691.323 ms 2024-09-10T09:59:20.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:59:19 smithi005 ceph-mon[30350]: pgmap v1022: 129 pgs: 129 active+clean; 654 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 2.7 MiB/s wr, 89 op/s 2024-09-10T09:59:20.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:59:19 smithi159 ceph-mon[35513]: pgmap v1022: 129 pgs: 129 active+clean; 654 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 2.7 MiB/s wr, 89 op/s 2024-09-10T09:59:20.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:59:19 smithi102 ceph-mon[35442]: pgmap v1022: 129 pgs: 129 active+clean; 654 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 2.7 MiB/s wr, 89 op/s 2024-09-10T09:59:20.699 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-10T09:59:20.699 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-10T09:59:21.026 INFO:tasks.workunit.client.0.smithi005.stdout: 10 110875 12.95 MB/sec execute 299 sec latency 23691.393 ms 2024-09-10T09:59:21.091 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:59:21.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:21.662+0000 7f2b73e00640 1 -- 172.21.15.5:0/1420708262 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2b74103e30 msgr2=0x7f2b74104210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:21.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:21.662+0000 7f2b73e00640 1 --2- 172.21.15.5:0/1420708262 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2b74103e30 0x7f2b74104210 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f2b64009900 tx=0x7f2b6402a410 comp rx=0 tx=0).stop 2024-09-10T09:59:21.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:21.662+0000 7f2b73e00640 1 -- 172.21.15.5:0/1420708262 shutdown_connections 2024-09-10T09:59:21.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:21.662+0000 7f2b73e00640 1 --2- 172.21.15.5:0/1420708262 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2b74108f60 0x7f2b74110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:21.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:21.662+0000 7f2b73e00640 1 --2- 172.21.15.5:0/1420708262 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2b741047e0 0x7f2b74108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:21.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:21.662+0000 7f2b73e00640 1 --2- 172.21.15.5:0/1420708262 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2b74103e30 0x7f2b74104210 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:21.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:21.662+0000 7f2b73e00640 1 -- 172.21.15.5:0/1420708262 >> 172.21.15.5:0/1420708262 conn(0x7f2b740ff910 msgr2=0x7f2b74101d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:21.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:21.662+0000 7f2b73e00640 1 -- 172.21.15.5:0/1420708262 shutdown_connections 2024-09-10T09:59:21.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:21.662+0000 7f2b73e00640 1 -- 172.21.15.5:0/1420708262 wait complete. 2024-09-10T09:59:21.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:21.663+0000 7f2b73e00640 1 Processor -- start 2024-09-10T09:59:21.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:21.663+0000 7f2b73e00640 1 -- start start 2024-09-10T09:59:21.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:21.664+0000 7f2b73e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2b74103e30 0x7f2b7419a410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:21.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:21.664+0000 7f2b73e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2b741047e0 0x7f2b7419a950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:21.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:21.664+0000 7f2b73e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2b74108f60 0x7f2b7419ece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:21.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:21.664+0000 7f2b73e00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f2b74114010 con 0x7f2b74103e30 2024-09-10T09:59:21.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:21.664+0000 7f2b73e00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f2b74113e90 con 0x7f2b741047e0 2024-09-10T09:59:21.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:21.664+0000 7f2b73e00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f2b74114190 con 0x7f2b74108f60 2024-09-10T09:59:21.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:21.664+0000 7f2b72a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2b74103e30 0x7f2b7419a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:21.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:21.664+0000 7f2b72a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2b74103e30 0x7f2b7419a410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41284/0 (socket says 172.21.15.5:41284) 2024-09-10T09:59:21.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:21.664+0000 7f2b72a00640 1 -- 172.21.15.5:0/1068593145 learned_addr learned my addr 172.21.15.5:0/1068593145 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:59:21.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:21.664+0000 7f2b6be00640 1 --2- 172.21.15.5:0/1068593145 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2b741047e0 0x7f2b7419a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:21.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:21.664+0000 7f2b73400640 1 --2- 172.21.15.5:0/1068593145 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2b74108f60 0x7f2b7419ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:21.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:21.665+0000 7f2b72a00640 1 -- 172.21.15.5:0/1068593145 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2b74108f60 msgr2=0x7f2b7419ece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:21.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:21.665+0000 7f2b72a00640 1 --2- 172.21.15.5:0/1068593145 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2b74108f60 0x7f2b7419ece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:21.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:21.665+0000 7f2b72a00640 1 -- 172.21.15.5:0/1068593145 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2b741047e0 msgr2=0x7f2b7419a950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:21.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:21.665+0000 7f2b72a00640 1 --2- 172.21.15.5:0/1068593145 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2b741047e0 0x7f2b7419a950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:21.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:21.665+0000 7f2b72a00640 1 -- 172.21.15.5:0/1068593145 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b7419f3c0 con 0x7f2b74103e30 2024-09-10T09:59:21.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:21.665+0000 7f2b72a00640 1 --2- 172.21.15.5:0/1068593145 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2b74103e30 0x7f2b7419a410 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f2b64004770 tx=0x7f2b64003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:21.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:21.665+0000 7f2b6b400640 1 -- 172.21.15.5:0/1068593145 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b6403f070 con 0x7f2b74103e30 2024-09-10T09:59:21.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:21.665+0000 7f2b73e00640 1 -- 172.21.15.5:0/1068593145 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b7419f650 con 0x7f2b74103e30 2024-09-10T09:59:21.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:21.665+0000 7f2b6b400640 1 -- 172.21.15.5:0/1068593145 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2b64031ca0 con 0x7f2b74103e30 2024-09-10T09:59:21.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:21.665+0000 7f2b6b400640 1 -- 172.21.15.5:0/1068593145 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b64035030 con 0x7f2b74103e30 2024-09-10T09:59:21.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:21.665+0000 7f2b73e00640 1 -- 172.21.15.5:0/1068593145 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b741a6f90 con 0x7f2b74103e30 2024-09-10T09:59:21.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:21.668+0000 7f2b6b400640 1 -- 172.21.15.5:0/1068593145 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f2b64042070 con 0x7f2b74103e30 2024-09-10T09:59:21.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:21.669+0000 7f2b7c4d2740 1 Processor -- start 2024-09-10T09:59:21.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:21.669+0000 7f2b7c4d2740 1 -- start start 2024-09-10T09:59:21.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:21.669+0000 7f2b7c4d2740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563633d4fcb0 0x563633d50090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:21.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:21.669+0000 7f2b7c4d2740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563633d505d0 0x563633d5a950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:21.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:21.669+0000 7f2b7c4d2740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563633d5ae90 0x563633d5d280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:21.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:21.669+0000 7f2b7c4d2740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x563633cb6290 con 0x563633d4fcb0 2024-09-10T09:59:21.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:21.669+0000 7f2b7c4d2740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x563633c5db40 con 0x563633d5ae90 2024-09-10T09:59:21.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:21.669+0000 7f2b7c4d2740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x563633cb2250 con 0x563633d505d0 2024-09-10T09:59:21.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:21.669+0000 7f2b73400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563633d4fcb0 0x563633d50090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:21.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:21.669+0000 7f2b73400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563633d4fcb0 0x563633d50090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41286/0 (socket says 172.21.15.5:41286) 2024-09-10T09:59:21.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:21.669+0000 7f2b73400640 1 -- 172.21.15.5:0/627495175 learned_addr learned my addr 172.21.15.5:0/627495175 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:59:21.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:21.670+0000 7f2b72a00640 1 --2- 172.21.15.5:0/627495175 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563633d505d0 0x563633d5a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:21.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:21.670+0000 7f2b73400640 1 -- 172.21.15.5:0/627495175 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563633d505d0 msgr2=0x563633d5a950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:59:21.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:21.670+0000 7f2b73400640 1 --2- 172.21.15.5:0/627495175 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563633d505d0 0x563633d5a950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:21.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:21.670+0000 7f2b6be00640 1 --2- 172.21.15.5:0/627495175 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563633d5ae90 0x563633d5d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:21.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:21.670+0000 7f2b73400640 1 -- 172.21.15.5:0/627495175 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563633d5ae90 msgr2=0x563633d5d280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:59:21.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:21.670+0000 7f2b73400640 1 --2- 172.21.15.5:0/627495175 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563633d5ae90 0x563633d5d280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:21.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:21.670+0000 7f2b73400640 1 -- 172.21.15.5:0/627495175 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563633a276a0 con 0x563633d4fcb0 2024-09-10T09:59:21.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:21.670+0000 7f2b72a00640 1 --2- 172.21.15.5:0/627495175 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563633d505d0 0x563633d5a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:59:21.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:21.670+0000 7f2b6be00640 1 --2- 172.21.15.5:0/627495175 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563633d5ae90 0x563633d5d280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T09:59:21.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:21.670+0000 7f2b6b400640 1 --2- 172.21.15.5:0/1068593145 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f2b580a2b80 0x7f2b580a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:21.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:21.670+0000 7f2b73400640 1 --2- 172.21.15.5:0/627495175 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563633d4fcb0 0x563633d50090 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f2b6c009f50 tx=0x7f2b6c0118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7094790b84dd9b8c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:21.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:21.670+0000 7f2b6b400640 1 -- 172.21.15.5:0/1068593145 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f2b640efb10 con 0x7f2b74103e30 2024-09-10T09:59:21.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:21.671+0000 7f2b72a00640 1 --2- 172.21.15.5:0/1068593145 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f2b580a2b80 0x7f2b580a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:21.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:21.671+0000 7f2b63e00640 1 -- 172.21.15.5:0/627495175 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b6c004d20 con 0x563633d4fcb0 2024-09-10T09:59:21.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:21.671+0000 7f2b63e00640 1 -- 172.21.15.5:0/627495175 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2b6c004050 con 0x563633d4fcb0 2024-09-10T09:59:21.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:21.671+0000 7f2b63e00640 1 -- 172.21.15.5:0/627495175 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b6c005280 con 0x563633d4fcb0 2024-09-10T09:59:21.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:21.672+0000 7f2b7c4d2740 1 -- 172.21.15.5:0/627495175 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563633d4fcb0 msgr2=0x563633d50090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:21.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:21.672+0000 7f2b72a00640 1 --2- 172.21.15.5:0/1068593145 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f2b580a2b80 0x7f2b580a5040 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f2b640b8430 tx=0x7f2b6403b890 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:21.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:21.672+0000 7f2b7c4d2740 1 --2- 172.21.15.5:0/627495175 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563633d4fcb0 0x563633d50090 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f2b6c009f50 tx=0x7f2b6c0118d0 comp rx=0 tx=0).stop 2024-09-10T09:59:21.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:21.672+0000 7f2b7c4d2740 1 -- 172.21.15.5:0/627495175 shutdown_connections 2024-09-10T09:59:21.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:21.672+0000 7f2b7c4d2740 1 --2- 172.21.15.5:0/627495175 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563633d5ae90 0x563633d5d280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:21.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:21.672+0000 7f2b7c4d2740 1 --2- 172.21.15.5:0/627495175 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563633d505d0 0x563633d5a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:21.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:21.672+0000 7f2b7c4d2740 1 --2- 172.21.15.5:0/627495175 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563633d4fcb0 0x563633d50090 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:21.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:21.672+0000 7f2b7c4d2740 1 -- 172.21.15.5:0/627495175 >> 172.21.15.5:0/627495175 conn(0x563633d4c240 msgr2=0x563633d4c620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:21.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:21.672+0000 7f2b7c4d2740 1 -- 172.21.15.5:0/627495175 shutdown_connections 2024-09-10T09:59:21.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:21.672+0000 7f2b7c4d2740 1 -- 172.21.15.5:0/627495175 wait complete. 2024-09-10T09:59:21.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:21.672+0000 7f2b7c4d2740 1 -- start start 2024-09-10T09:59:21.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:21.673+0000 7f2b7c4d2740 1 Processor -- start 2024-09-10T09:59:21.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:21.673+0000 7f2b7c4d2740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563633d4fcb0 0x563633d5a760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:21.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:21.673+0000 7f2b7c4d2740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563633d505d0 0x563633d57a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:21.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:21.673+0000 7f2b7c4d2740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563633d5ae90 0x563633d57f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:21.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:21.673+0000 7f2b7c4d2740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x563633d5db00 con 0x563633d505d0 2024-09-10T09:59:21.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:21.673+0000 7f2b7c4d2740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x563633c70fa0 con 0x563633d5ae90 2024-09-10T09:59:21.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:21.673+0000 7f2b7c4d2740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x563633d5dc80 con 0x563633d4fcb0 2024-09-10T09:59:21.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:21.673+0000 7f2b7c4d2740 10 client.0 _ll_register_callbacks cb 0x5636339c4ba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:59:21.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:21.673+0000 7f2b6be00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563633d505d0 0x563633d57a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:21.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:21.673+0000 7f2b73400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563633d4fcb0 0x563633d5a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:21.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:21.674+0000 7f2b6be00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563633d505d0 0x563633d57a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41298/0 (socket says 172.21.15.5:41298) 2024-09-10T09:59:21.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:21.674+0000 7f2b6be00640 1 -- 172.21.15.5:0/2798663510 learned_addr learned my addr 172.21.15.5:0/2798663510 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:59:21.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:21.674+0000 7f2b73400640 1 --2- 172.21.15.5:0/2798663510 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563633d5ae90 0x563633d57f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:21.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:21.674+0000 7f2b6be00640 1 -- 172.21.15.5:0/2798663510 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563633d4fcb0 msgr2=0x563633d5a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:59:21.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:21.674+0000 7f2b6be00640 1 --2- 172.21.15.5:0/2798663510 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563633d4fcb0 0x563633d5a760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:21.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:21.674+0000 7f2b6be00640 1 -- 172.21.15.5:0/2798663510 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563633d5ae90 msgr2=0x563633d57f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:59:21.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:21.674+0000 7f2b6be00640 1 --2- 172.21.15.5:0/2798663510 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563633d5ae90 0x563633d57f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:21.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:21.674+0000 7f2b6be00640 1 -- 172.21.15.5:0/2798663510 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563633d58780 con 0x563633d505d0 2024-09-10T09:59:21.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:21.674+0000 7f2b73400640 1 --2- 172.21.15.5:0/2798663510 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563633d5ae90 0x563633d57f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:59:21.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:21.674+0000 7f2b6be00640 1 --2- 172.21.15.5:0/2798663510 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563633d505d0 0x563633d57a10 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f2b5c00bed0 tx=0x7f2b5c0117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac7215eaba34135c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:21.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:21.674+0000 7f2b60c00640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-10T09:59:21.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:21.675+0000 7f2b60c00640 1 -- 172.21.15.5:0/2798663510 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b5c012920 con 0x563633d505d0 2024-09-10T09:59:21.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:21.675+0000 7f2b7c4d2740 1 -- 172.21.15.5:0/2798663510 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2b6c004050 con 0x563633d505d0 2024-09-10T09:59:21.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:21.675+0000 7f2b60c00640 1 -- 172.21.15.5:0/2798663510 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f2b5c012ac0 con 0x563633d505d0 2024-09-10T09:59:21.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:21.675+0000 7f2b60c00640 1 -- 172.21.15.5:0/2798663510 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f2b5c026050 con 0x563633d505d0 2024-09-10T09:59:21.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:21.675+0000 7f2b7c4d2740 10 client.19434 fetch_fsmap learned FSMap version 42 2024-09-10T09:59:21.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:21.675+0000 7f2b7c4d2740 1 -- 172.21.15.5:0/2798663510 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x563633d699c0 con 0x563633d505d0 2024-09-10T09:59:21.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:21.675+0000 7f2b60c00640 1 -- 172.21.15.5:0/2798663510 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2b6c004050 con 0x563633d505d0 2024-09-10T09:59:21.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:21.676+0000 7f2b60c00640 1 -- 172.21.15.5:0/2798663510 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 42) ==== 8370+0+0 (secure 0 0 0) 0x7f2b5c00f3d0 con 0x563633d505d0 2024-09-10T09:59:21.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:21.676+0000 7f2b7c4d2740 10 client.19434 fetch_fsmap finished waiting for FSMap version 42 2024-09-10T09:59:21.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:21.676+0000 7f2b7c4d2740 10 client.19434 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T09:59:21.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:21.676+0000 7f2b7c4d2740 20 client.19434 populate_metadata read hostname 'smithi005' 2024-09-10T09:59:21.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:21.676+0000 7f2b7c4d2740 1 --2- 172.21.15.5:0/2798663510 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x563633d6add0 0x563633d6d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:21.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:21.676+0000 7f2b7c4d2740 4 client.19434 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:59:21.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:21.676+0000 7f2b7c4d2740 1 -- 172.21.15.5:0/2798663510 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563633cc3800 con 0x563633d6add0 2024-09-10T09:59:21.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:21.676+0000 7f2b73400640 1 --2- 172.21.15.5:0/2798663510 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x563633d6add0 0x563633d6d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:21.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:21.676+0000 7f2b73400640 1 --2- 172.21.15.5:0/2798663510 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x563633d6add0 0x563633d6d1f0 crc :-1 s=READY pgs=293 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7584cda0ed9e9454 server_cookie=4eb0061b52032dc3 in_seq=0 out_seq=0 2024-09-10T09:59:21.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:21.676+0000 7f2b60c00640 10 client.19434 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T09:59:21.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:21.677+0000 7f2b60c00640 1 -- 172.21.15.5:0/2798663510 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563633cc3800 con 0x563633d6add0 2024-09-10T09:59:21.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:21.677+0000 7f2b60c00640 10 client.19434 handle_command_reply: tid=0 2024-09-10T09:59:21.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:21.677+0000 7f2b7c4d2740 1 client.19434 shutdown 2024-09-10T09:59:21.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:21.678+0000 7f2b7c4d2740 1 -- 172.21.15.5:0/2798663510 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563633d505d0 msgr2=0x563633d57a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:21.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:21.678+0000 7f2b7c4d2740 1 --2- 172.21.15.5:0/2798663510 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563633d505d0 0x563633d57a10 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f2b5c00bed0 tx=0x7f2b5c0117b0 comp rx=0 tx=0).stop 2024-09-10T09:59:21.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:21.678+0000 7f2b7c4d2740 1 -- 172.21.15.5:0/2798663510 shutdown_connections 2024-09-10T09:59:21.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:21.678+0000 7f2b7c4d2740 1 --2- 172.21.15.5:0/2798663510 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x563633d6add0 0x563633d6d1f0 crc :-1 s=READY pgs=293 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:21.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:21.678+0000 7f2b7c4d2740 1 --2- 172.21.15.5:0/2798663510 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563633d5ae90 0x563633d57f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:21.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:21.678+0000 7f2b7c4d2740 1 --2- 172.21.15.5:0/2798663510 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563633d505d0 0x563633d57a10 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:21.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:21.678+0000 7f2b7c4d2740 1 --2- 172.21.15.5:0/2798663510 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563633d4fcb0 0x563633d5a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:21.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:21.678+0000 7f2b7c4d2740 1 -- 172.21.15.5:0/2798663510 >> 172.21.15.5:0/2798663510 conn(0x563633d4c240 msgr2=0x563633d51f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:21.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:21.678+0000 7f2b60c00640 0 client.19434 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T09:59:21.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:21.678+0000 7f2b7c4d2740 1 -- 172.21.15.5:0/2798663510 shutdown_connections 2024-09-10T09:59:21.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:21.678+0000 7f2b7c4d2740 1 -- 172.21.15.5:0/2798663510 wait complete. 2024-09-10T09:59:21.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:21.678+0000 7f2b7c4d2740 20 client.19434 trim_cache size 0 max 16384 2024-09-10T09:59:21.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:21.696+0000 7f2b7c4d2740 1 Processor -- start 2024-09-10T09:59:21.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:21.696+0000 7f2b7c4d2740 1 -- start start 2024-09-10T09:59:21.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:21.697+0000 7f2b7c4d2740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563633d4fcb0 0x563633d59980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:21.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:21.697+0000 7f2b7c4d2740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563633d505d0 0x563633d72ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:21.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:21.697+0000 7f2b7c4d2740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563633d5ae90 0x563633d73000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:21.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:21.697+0000 7f2b7c4d2740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f2b741b0df0 con 0x563633d4fcb0 2024-09-10T09:59:21.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:21.697+0000 7f2b7c4d2740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f2b741b2930 con 0x563633d5ae90 2024-09-10T09:59:21.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:21.697+0000 7f2b7c4d2740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f2b741b14c0 con 0x563633d505d0 2024-09-10T09:59:21.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:21.697+0000 7f2b73400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563633d4fcb0 0x563633d59980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:21.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:21.697+0000 7f2b73400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563633d4fcb0 0x563633d59980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41304/0 (socket says 172.21.15.5:41304) 2024-09-10T09:59:21.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:21.697+0000 7f2b73400640 1 -- 172.21.15.5:0/3883735380 learned_addr learned my addr 172.21.15.5:0/3883735380 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:59:21.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:21.697+0000 7f2b6be00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563633d505d0 0x563633d72ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:21.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:21.697+0000 7f2b73400640 1 --2- 172.21.15.5:0/3883735380 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563633d5ae90 0x563633d73000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:21.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:21.697+0000 7f2b6be00640 1 -- 172.21.15.5:0/3883735380 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563633d5ae90 msgr2=0x563633d73000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:59:21.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:21.697+0000 7f2b6be00640 1 --2- 172.21.15.5:0/3883735380 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563633d5ae90 0x563633d73000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:21.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:21.698+0000 7f2b6be00640 1 -- 172.21.15.5:0/3883735380 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563633d4fcb0 msgr2=0x563633d59980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:59:21.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:21.698+0000 7f2b6be00640 1 --2- 172.21.15.5:0/3883735380 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563633d4fcb0 0x563633d59980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:21.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:21.698+0000 7f2b6be00640 1 -- 172.21.15.5:0/3883735380 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563633d4ced0 con 0x563633d505d0 2024-09-10T09:59:21.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:21.698+0000 7f2b73400640 1 --2- 172.21.15.5:0/3883735380 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563633d4fcb0 0x563633d59980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:59:21.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:21.698+0000 7f2b6be00640 1 --2- 172.21.15.5:0/3883735380 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563633d505d0 0x563633d72ac0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f2b5c00f7a0 tx=0x7f2b5c005d00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1f276c382dc69371 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:21.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:21.698+0000 7f2b63400640 1 -- 172.21.15.5:0/3883735380 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b5c02e070 con 0x563633d505d0 2024-09-10T09:59:21.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:21.698+0000 7f2b63400640 1 -- 172.21.15.5:0/3883735380 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2b5c005510 con 0x563633d505d0 2024-09-10T09:59:21.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:21.698+0000 7f2b63400640 1 -- 172.21.15.5:0/3883735380 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b5c023040 con 0x563633d505d0 2024-09-10T09:59:21.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:21.699+0000 7f2b7c4d2740 1 -- 172.21.15.5:0/3883735380 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563633d505d0 msgr2=0x563633d72ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:21.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:21.699+0000 7f2b7c4d2740 1 --2- 172.21.15.5:0/3883735380 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563633d505d0 0x563633d72ac0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f2b5c00f7a0 tx=0x7f2b5c005d00 comp rx=0 tx=0).stop 2024-09-10T09:59:21.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:21.699+0000 7f2b7c4d2740 1 -- 172.21.15.5:0/3883735380 shutdown_connections 2024-09-10T09:59:21.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:21.699+0000 7f2b7c4d2740 1 --2- 172.21.15.5:0/3883735380 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563633d5ae90 0x563633d73000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:21.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:21.699+0000 7f2b7c4d2740 1 --2- 172.21.15.5:0/3883735380 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563633d505d0 0x563633d72ac0 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:21.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:21.699+0000 7f2b7c4d2740 1 --2- 172.21.15.5:0/3883735380 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563633d4fcb0 0x563633d59980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:21.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:21.699+0000 7f2b7c4d2740 1 -- 172.21.15.5:0/3883735380 >> 172.21.15.5:0/3883735380 conn(0x563633d4c240 msgr2=0x563633d59270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:21.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:21.699+0000 7f2b7c4d2740 1 -- 172.21.15.5:0/3883735380 shutdown_connections 2024-09-10T09:59:21.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:21.699+0000 7f2b7c4d2740 1 -- 172.21.15.5:0/3883735380 wait complete. 2024-09-10T09:59:21.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:21.699+0000 7f2b7c4d2740 1 -- start start 2024-09-10T09:59:21.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:21.700+0000 7f2b7c4d2740 1 Processor -- start 2024-09-10T09:59:21.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:21.700+0000 7f2b7c4d2740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563633d4fcb0 0x563633d60440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:21.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:21.700+0000 7f2b7c4d2740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563633d505d0 0x563633d60980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:21.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:21.700+0000 7f2b7c4d2740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563633d5ae90 0x563633dd5aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:21.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:21.700+0000 7f2b7c4d2740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f2b741b2260 con 0x563633d505d0 2024-09-10T09:59:21.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:21.700+0000 7f2b7c4d2740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f2b741b1b90 con 0x563633d4fcb0 2024-09-10T09:59:21.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:21.700+0000 7f2b7c4d2740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f2b741aedd0 con 0x563633d5ae90 2024-09-10T09:59:21.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:21.700+0000 7f2b7c4d2740 10 client.0 _ll_register_callbacks cb 0x7f2b5c00b730 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:59:21.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:21.700+0000 7f2b6be00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563633d505d0 0x563633d60980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:21.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:21.700+0000 7f2b73400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563633d5ae90 0x563633dd5aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:21.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:21.701+0000 7f2b6be00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563633d505d0 0x563633d60980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41308/0 (socket says 172.21.15.5:41308) 2024-09-10T09:59:21.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:21.701+0000 7f2b6be00640 1 -- 172.21.15.5:0/1843257896 learned_addr learned my addr 172.21.15.5:0/1843257896 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:59:21.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:21.701+0000 7f2b73400640 1 --2- 172.21.15.5:0/1843257896 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563633d4fcb0 0x563633d60440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:21.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:21.701+0000 7f2b73400640 1 -- 172.21.15.5:0/1843257896 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563633d4fcb0 msgr2=0x563633d60440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:59:21.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:21.701+0000 7f2b73400640 1 --2- 172.21.15.5:0/1843257896 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563633d4fcb0 0x563633d60440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:21.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:21.701+0000 7f2b73400640 1 -- 172.21.15.5:0/1843257896 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563633d505d0 msgr2=0x563633d60980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:59:21.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:21.701+0000 7f2b73400640 1 --2- 172.21.15.5:0/1843257896 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563633d505d0 0x563633d60980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:21.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:21.701+0000 7f2b73400640 1 -- 172.21.15.5:0/1843257896 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563633dd5fe0 con 0x563633d5ae90 2024-09-10T09:59:21.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:21.701+0000 7f2b6be00640 1 --2- 172.21.15.5:0/1843257896 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563633d505d0 0x563633d60980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:59:21.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:21.701+0000 7f2b73400640 1 --2- 172.21.15.5:0/1843257896 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563633d5ae90 0x563633dd5aa0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f2b6c009d60 tx=0x7f2b6c005fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ae32812d32b61de1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:21.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:21.701+0000 7f2b60c00640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-09-10T09:59:21.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:21.701+0000 7f2b60c00640 1 -- 172.21.15.5:0/1843257896 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b6c003c40 con 0x563633d5ae90 2024-09-10T09:59:21.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:21.702+0000 7f2b7c4d2740 1 -- 172.21.15.5:0/1843257896 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2b5c005510 con 0x563633d5ae90 2024-09-10T09:59:21.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:21.702+0000 7f2b60c00640 1 -- 172.21.15.5:0/1843257896 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f2b6c003de0 con 0x563633d5ae90 2024-09-10T09:59:21.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:21.702+0000 7f2b60c00640 1 -- 172.21.15.5:0/1843257896 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f2b6c042050 con 0x563633d5ae90 2024-09-10T09:59:21.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:21.702+0000 7f2b7c4d2740 10 client.29485 fetch_fsmap learned FSMap version 42 2024-09-10T09:59:21.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:21.702+0000 7f2b7c4d2740 1 -- 172.21.15.5:0/1843257896 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x563633dd7320 con 0x563633d5ae90 2024-09-10T09:59:21.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:21.702+0000 7f2b60c00640 1 -- 172.21.15.5:0/1843257896 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2b5c005510 con 0x563633d5ae90 2024-09-10T09:59:21.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:21.702+0000 7f2b60c00640 1 -- 172.21.15.5:0/1843257896 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 42) ==== 8370+0+0 (secure 0 0 0) 0x7f2b6c039070 con 0x563633d5ae90 2024-09-10T09:59:21.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:21.702+0000 7f2b7c4d2740 10 client.29485 fetch_fsmap finished waiting for FSMap version 42 2024-09-10T09:59:21.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:21.702+0000 7f2b7c4d2740 10 client.29485 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T09:59:21.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:21.702+0000 7f2b7c4d2740 20 client.29485 populate_metadata read hostname 'smithi005' 2024-09-10T09:59:21.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:21.702+0000 7f2b7c4d2740 1 --2- 172.21.15.5:0/1843257896 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x563633dd7860 0x563633dd9c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:21.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:21.702+0000 7f2b7c4d2740 4 client.29485 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-10T09:59:21.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:21.702+0000 7f2b7c4d2740 1 -- 172.21.15.5:0/1843257896 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x563633cbdd10 con 0x563633dd7860 2024-09-10T09:59:21.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:21.703+0000 7f2b6be00640 1 --2- 172.21.15.5:0/1843257896 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x563633dd7860 0x563633dd9c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:21.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:21.703+0000 7f2b6be00640 1 --2- 172.21.15.5:0/1843257896 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x563633dd7860 0x563633dd9c80 crc :-1 s=READY pgs=294 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f51d6145e348bdcc server_cookie=7553c4e947fc2897 in_seq=0 out_seq=0 2024-09-10T09:59:21.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:21.703+0000 7f2b60c00640 10 client.29485 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T09:59:21.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:21.704+0000 7f2b60c00640 1 -- 172.21.15.5:0/1843257896 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x563633cbdd10 con 0x563633dd7860 2024-09-10T09:59:21.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:21.704+0000 7f2b60c00640 10 client.29485 handle_command_reply: tid=0 2024-09-10T09:59:21.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:21.704+0000 7f2b7c4d2740 1 client.29485 shutdown 2024-09-10T09:59:21.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:21.705+0000 7f2b7c4d2740 1 -- 172.21.15.5:0/1843257896 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563633d5ae90 msgr2=0x563633dd5aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:21.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:21.705+0000 7f2b7c4d2740 1 --2- 172.21.15.5:0/1843257896 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563633d5ae90 0x563633dd5aa0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f2b6c009d60 tx=0x7f2b6c005fd0 comp rx=0 tx=0).stop 2024-09-10T09:59:21.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:21.705+0000 7f2b7c4d2740 1 -- 172.21.15.5:0/1843257896 shutdown_connections 2024-09-10T09:59:21.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:21.705+0000 7f2b7c4d2740 1 --2- 172.21.15.5:0/1843257896 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x563633dd7860 0x563633dd9c80 crc :-1 s=READY pgs=294 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:21.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:21.705+0000 7f2b7c4d2740 1 --2- 172.21.15.5:0/1843257896 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563633d5ae90 0x563633dd5aa0 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:21.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:21.705+0000 7f2b60c00640 0 client.29485 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T09:59:21.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:21.705+0000 7f2b7c4d2740 1 --2- 172.21.15.5:0/1843257896 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563633d505d0 0x563633d60980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:21.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:21.705+0000 7f2b7c4d2740 1 --2- 172.21.15.5:0/1843257896 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563633d4fcb0 0x563633d60440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:21.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:21.705+0000 7f2b7c4d2740 1 -- 172.21.15.5:0/1843257896 >> 172.21.15.5:0/1843257896 conn(0x563633d4c240 msgr2=0x563633d4b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:21.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:21.705+0000 7f2b7c4d2740 1 -- 172.21.15.5:0/1843257896 shutdown_connections 2024-09-10T09:59:21.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:21.705+0000 7f2b7c4d2740 1 -- 172.21.15.5:0/1843257896 wait complete. 2024-09-10T09:59:21.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:21.705+0000 7f2b7c4d2740 20 client.29485 trim_cache size 0 max 16384 2024-09-10T09:59:21.711 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-10T09:59:21.711 INFO:teuthology.orchestra.run.smithi005.stdout: "return_code": 0, 2024-09-10T09:59:21.711 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_tag": "2c3987c4-1e83-487a-8209-5f7e4b276597", 2024-09-10T09:59:21.711 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": "asynchronous" 2024-09-10T09:59:21.711 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-10T09:59:21.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:21.706+0000 7f2b73e00640 1 -- 172.21.15.5:0/1068593145 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f2b580a2b80 msgr2=0x7f2b580a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:21.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:21.706+0000 7f2b73e00640 1 --2- 172.21.15.5:0/1068593145 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f2b580a2b80 0x7f2b580a5040 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f2b640b8430 tx=0x7f2b6403b890 comp rx=0 tx=0).stop 2024-09-10T09:59:21.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:21.706+0000 7f2b73e00640 1 -- 172.21.15.5:0/1068593145 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2b74103e30 msgr2=0x7f2b7419a410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:21.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:21.706+0000 7f2b73e00640 1 --2- 172.21.15.5:0/1068593145 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2b74103e30 0x7f2b7419a410 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f2b64004770 tx=0x7f2b64003c10 comp rx=0 tx=0).stop 2024-09-10T09:59:21.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:21.706+0000 7f2b73e00640 1 -- 172.21.15.5:0/1068593145 shutdown_connections 2024-09-10T09:59:21.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:21.706+0000 7f2b73e00640 1 --2- 172.21.15.5:0/1068593145 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f2b580a2b80 0x7f2b580a5040 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:21.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:21.706+0000 7f2b73e00640 1 --2- 172.21.15.5:0/1068593145 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2b74108f60 0x7f2b7419ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:21.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:21.706+0000 7f2b73e00640 1 --2- 172.21.15.5:0/1068593145 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2b741047e0 0x7f2b7419a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:21.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:21.706+0000 7f2b73e00640 1 --2- 172.21.15.5:0/1068593145 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2b74103e30 0x7f2b7419a410 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:21.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:21.706+0000 7f2b73e00640 1 -- 172.21.15.5:0/1068593145 >> 172.21.15.5:0/1068593145 conn(0x7f2b740ff910 msgr2=0x7f2b74100bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:21.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:21.706+0000 7f2b73e00640 1 -- 172.21.15.5:0/1068593145 shutdown_connections 2024-09-10T09:59:21.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:21.706+0000 7f2b73e00640 1 -- 172.21.15.5:0/1068593145 wait complete. 2024-09-10T09:59:22.026 INFO:tasks.workunit.client.0.smithi005.stdout: 10 110875 12.90 MB/sec execute 300 sec latency 24691.493 ms 2024-09-10T09:59:22.226 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:59:21 smithi005 ceph-mon[30350]: pgmap v1023: 129 pgs: 129 active+clean; 654 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 2.9 MiB/s wr, 105 op/s 2024-09-10T09:59:22.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:59:21 smithi159 ceph-mon[35513]: pgmap v1023: 129 pgs: 129 active+clean; 654 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 2.9 MiB/s wr, 105 op/s 2024-09-10T09:59:22.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:59:21 smithi102 ceph-mon[35442]: pgmap v1023: 129 pgs: 129 active+clean; 654 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 2.9 MiB/s wr, 105 op/s 2024-09-10T09:59:22.683 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T09:59:23.026 INFO:tasks.workunit.client.0.smithi005.stdout: 10 110875 12.86 MB/sec execute 301 sec latency 25691.555 ms 2024-09-10T09:59:23.062 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:59:23.201 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:59:22 smithi005 ceph-mon[30350]: scrub queued for path: / 2024-09-10T09:59:23.201 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:59:22 smithi005 ceph-mon[30350]: scrub summary: idle+waiting paths [/] 2024-09-10T09:59:23.201 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:59:22 smithi005 ceph-mon[30350]: scrub summary: active paths [/] 2024-09-10T09:59:23.201 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:59:22 smithi005 ceph-mon[30350]: scrub summary: active paths [/] 2024-09-10T09:59:23.201 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:59:22 smithi005 ceph-mon[30350]: scrub summary: active paths [/] 2024-09-10T09:59:23.202 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:59:22 smithi005 ceph-mon[30350]: scrub summary: idle+waiting paths [/] 2024-09-10T09:59:23.202 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:59:22 smithi005 ceph-mon[30350]: scrub summary: idle+waiting paths [/] 2024-09-10T09:59:23.202 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:59:22 smithi005 ceph-mon[30350]: scrub summary: active paths [/] 2024-09-10T09:59:23.202 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:59:22 smithi005 ceph-mon[30350]: scrub summary: active paths [/] 2024-09-10T09:59:23.202 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:59:22 smithi005 ceph-mon[30350]: scrub summary: idle+waiting paths [/] 2024-09-10T09:59:23.202 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:59:22 smithi005 ceph-mon[30350]: scrub summary: active paths [/] 2024-09-10T09:59:23.202 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:59:22 smithi005 ceph-mon[30350]: scrub summary: idle+waiting paths [/] 2024-09-10T09:59:23.202 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:59:22 smithi005 ceph-mon[30350]: scrub summary: idle+waiting paths [/] 2024-09-10T09:59:23.202 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:59:22 smithi005 ceph-mon[30350]: scrub summary: active paths [/] 2024-09-10T09:59:23.202 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:59:22 smithi005 ceph-mon[30350]: scrub summary: idle+waiting paths [/] 2024-09-10T09:59:23.202 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:59:22 smithi005 ceph-mon[30350]: scrub summary: idle+waiting paths [/] 2024-09-10T09:59:23.252 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:59:22 smithi159 ceph-mon[35513]: scrub queued for path: / 2024-09-10T09:59:23.252 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:59:22 smithi159 ceph-mon[35513]: scrub summary: idle+waiting paths [/] 2024-09-10T09:59:23.252 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:59:22 smithi159 ceph-mon[35513]: scrub summary: active paths [/] 2024-09-10T09:59:23.252 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:59:22 smithi159 ceph-mon[35513]: scrub summary: active paths [/] 2024-09-10T09:59:23.252 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:59:22 smithi159 ceph-mon[35513]: scrub summary: active paths [/] 2024-09-10T09:59:23.252 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:59:22 smithi159 ceph-mon[35513]: scrub summary: idle+waiting paths [/] 2024-09-10T09:59:23.252 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:59:22 smithi159 ceph-mon[35513]: scrub summary: idle+waiting paths [/] 2024-09-10T09:59:23.252 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:59:22 smithi159 ceph-mon[35513]: scrub summary: active paths [/] 2024-09-10T09:59:23.252 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:59:22 smithi159 ceph-mon[35513]: scrub summary: active paths [/] 2024-09-10T09:59:23.252 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:59:22 smithi159 ceph-mon[35513]: scrub summary: idle+waiting paths [/] 2024-09-10T09:59:23.252 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:59:22 smithi159 ceph-mon[35513]: scrub summary: active paths [/] 2024-09-10T09:59:23.252 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:59:22 smithi159 ceph-mon[35513]: scrub summary: idle+waiting paths [/] 2024-09-10T09:59:23.253 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:59:22 smithi159 ceph-mon[35513]: scrub summary: idle+waiting paths [/] 2024-09-10T09:59:23.253 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:59:22 smithi159 ceph-mon[35513]: scrub summary: active paths [/] 2024-09-10T09:59:23.253 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:59:22 smithi159 ceph-mon[35513]: scrub summary: idle+waiting paths [/] 2024-09-10T09:59:23.253 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:59:22 smithi159 ceph-mon[35513]: scrub summary: idle+waiting paths [/] 2024-09-10T09:59:23.385 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:59:22 smithi102 ceph-mon[35442]: scrub queued for path: / 2024-09-10T09:59:23.385 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:59:22 smithi102 ceph-mon[35442]: scrub summary: idle+waiting paths [/] 2024-09-10T09:59:23.385 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:59:22 smithi102 ceph-mon[35442]: scrub summary: active paths [/] 2024-09-10T09:59:23.385 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:59:22 smithi102 ceph-mon[35442]: scrub summary: active paths [/] 2024-09-10T09:59:23.385 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:59:22 smithi102 ceph-mon[35442]: scrub summary: active paths [/] 2024-09-10T09:59:23.385 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:59:22 smithi102 ceph-mon[35442]: scrub summary: idle+waiting paths [/] 2024-09-10T09:59:23.385 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:59:22 smithi102 ceph-mon[35442]: scrub summary: idle+waiting paths [/] 2024-09-10T09:59:23.385 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:59:22 smithi102 ceph-mon[35442]: scrub summary: active paths [/] 2024-09-10T09:59:23.385 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:59:22 smithi102 ceph-mon[35442]: scrub summary: active paths [/] 2024-09-10T09:59:23.385 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:59:22 smithi102 ceph-mon[35442]: scrub summary: idle+waiting paths [/] 2024-09-10T09:59:23.385 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:59:22 smithi102 ceph-mon[35442]: scrub summary: active paths [/] 2024-09-10T09:59:23.386 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:59:22 smithi102 ceph-mon[35442]: scrub summary: idle+waiting paths [/] 2024-09-10T09:59:23.386 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:59:22 smithi102 ceph-mon[35442]: scrub summary: idle+waiting paths [/] 2024-09-10T09:59:23.386 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:59:22 smithi102 ceph-mon[35442]: scrub summary: active paths [/] 2024-09-10T09:59:23.386 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:59:22 smithi102 ceph-mon[35442]: scrub summary: idle+waiting paths [/] 2024-09-10T09:59:23.386 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:59:22 smithi102 ceph-mon[35442]: scrub summary: idle+waiting paths [/] 2024-09-10T09:59:23.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:23.621+0000 7feadbe00640 1 -- 172.21.15.5:0/4217920822 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feadc074e30 msgr2=0x7feadc10dd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:23.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:23.621+0000 7feadbe00640 1 --2- 172.21.15.5:0/4217920822 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feadc074e30 0x7feadc10dd40 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7feac8009900 tx=0x7feac802a480 comp rx=0 tx=0).stop 2024-09-10T09:59:23.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:23.622+0000 7feadbe00640 1 -- 172.21.15.5:0/4217920822 shutdown_connections 2024-09-10T09:59:23.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:23.622+0000 7feadbe00640 1 --2- 172.21.15.5:0/4217920822 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7feadc10e280 0x7feadc110670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:23.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:23.622+0000 7feadbe00640 1 --2- 172.21.15.5:0/4217920822 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feadc074e30 0x7feadc10dd40 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:23.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:23.622+0000 7feadbe00640 1 --2- 172.21.15.5:0/4217920822 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7feadc074510 0x7feadc0748f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:23.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:23.622+0000 7feadbe00640 1 -- 172.21.15.5:0/4217920822 >> 172.21.15.5:0/4217920822 conn(0x7feadc0fd6e0 msgr2=0x7feadc0ffb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:23.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:23.622+0000 7feadbe00640 1 -- 172.21.15.5:0/4217920822 shutdown_connections 2024-09-10T09:59:23.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:23.622+0000 7feadbe00640 1 -- 172.21.15.5:0/4217920822 wait complete. 2024-09-10T09:59:23.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:23.623+0000 7feadbe00640 1 Processor -- start 2024-09-10T09:59:23.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:23.623+0000 7feadbe00640 1 -- start start 2024-09-10T09:59:23.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:23.623+0000 7feadbe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feadc074510 0x7feadc19a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:23.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:23.623+0000 7feadbe00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7feadc074e30 0x7feadc19a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:23.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:23.623+0000 7feadbe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7feadc10e280 0x7feadc19ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:23.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:23.623+0000 7feadbe00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7feadc113fb0 con 0x7feadc074510 2024-09-10T09:59:23.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:23.623+0000 7feadbe00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7feadc113e30 con 0x7feadc10e280 2024-09-10T09:59:23.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:23.623+0000 7feadbe00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7feadc114130 con 0x7feadc074e30 2024-09-10T09:59:23.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:23.623+0000 7feadaa00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feadc074510 0x7feadc19a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:23.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:23.624+0000 7feadaa00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feadc074510 0x7feadc19a480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41326/0 (socket says 172.21.15.5:41326) 2024-09-10T09:59:23.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:23.624+0000 7feadaa00640 1 -- 172.21.15.5:0/883071346 learned_addr learned my addr 172.21.15.5:0/883071346 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:59:23.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:23.624+0000 7feada000640 1 --2- 172.21.15.5:0/883071346 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7feadc074e30 0x7feadc19a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:23.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:23.624+0000 7feadb400640 1 --2- 172.21.15.5:0/883071346 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7feadc10e280 0x7feadc19ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:23.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:23.624+0000 7feadaa00640 1 -- 172.21.15.5:0/883071346 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7feadc074e30 msgr2=0x7feadc19a9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:23.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:23.624+0000 7feadaa00640 1 --2- 172.21.15.5:0/883071346 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7feadc074e30 0x7feadc19a9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:23.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:23.624+0000 7feadaa00640 1 -- 172.21.15.5:0/883071346 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7feadc10e280 msgr2=0x7feadc19ed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:23.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:23.624+0000 7feadaa00640 1 --2- 172.21.15.5:0/883071346 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7feadc10e280 0x7feadc19ed50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:23.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:23.624+0000 7feadaa00640 1 -- 172.21.15.5:0/883071346 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feadc19f430 con 0x7feadc074510 2024-09-10T09:59:23.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:23.624+0000 7feadb400640 1 --2- 172.21.15.5:0/883071346 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7feadc10e280 0x7feadc19ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:59:23.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:23.624+0000 7feada000640 1 --2- 172.21.15.5:0/883071346 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7feadc074e30 0x7feadc19a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:59:23.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:23.624+0000 7feadaa00640 1 --2- 172.21.15.5:0/883071346 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feadc074510 0x7feadc19a480 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7feacc00e9f0 tx=0x7feacc00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:23.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:23.625+0000 7fead3400640 1 -- 172.21.15.5:0/883071346 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feacc00cd80 con 0x7feadc074510 2024-09-10T09:59:23.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:23.625+0000 7feadbe00640 1 -- 172.21.15.5:0/883071346 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feadc19f720 con 0x7feadc074510 2024-09-10T09:59:23.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:23.625+0000 7fead3400640 1 -- 172.21.15.5:0/883071346 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feacc010910 con 0x7feadc074510 2024-09-10T09:59:23.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:23.625+0000 7fead3400640 1 -- 172.21.15.5:0/883071346 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feacc0188f0 con 0x7feadc074510 2024-09-10T09:59:23.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:23.625+0000 7feadbe00640 1 -- 172.21.15.5:0/883071346 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feadc1a7000 con 0x7feadc074510 2024-09-10T09:59:23.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:23.628+0000 7fead3400640 1 -- 172.21.15.5:0/883071346 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7feacc010ac0 con 0x7feadc074510 2024-09-10T09:59:23.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:23.628+0000 7feae43ee740 1 Processor -- start 2024-09-10T09:59:23.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:23.629+0000 7feae43ee740 1 -- start start 2024-09-10T09:59:23.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:23.629+0000 7feae43ee740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564aa2df7d50 0x564aa2df8130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:23.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:23.629+0000 7feae43ee740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564aa2df8670 0x564aa2e029f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:23.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:23.629+0000 7feae43ee740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564aa2e02f30 0x564aa2e05320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:23.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:23.629+0000 7feae43ee740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x564aa2d5e3e0 con 0x564aa2df7d50 2024-09-10T09:59:23.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:23.629+0000 7feae43ee740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x564aa2d05d80 con 0x564aa2e02f30 2024-09-10T09:59:23.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:23.629+0000 7feae43ee740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x564aa2d5a3a0 con 0x564aa2df8670 2024-09-10T09:59:23.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:23.629+0000 7feadb400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564aa2df7d50 0x564aa2df8130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:23.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:23.629+0000 7feadb400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564aa2df7d50 0x564aa2df8130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41332/0 (socket says 172.21.15.5:41332) 2024-09-10T09:59:23.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:23.629+0000 7feadb400640 1 -- 172.21.15.5:0/123375111 learned_addr learned my addr 172.21.15.5:0/123375111 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:59:23.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:23.629+0000 7feada000640 1 --2- 172.21.15.5:0/123375111 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564aa2e02f30 0x564aa2e05320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:23.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:23.630+0000 7feadaa00640 1 --2- 172.21.15.5:0/123375111 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564aa2df8670 0x564aa2e029f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:23.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:23.630+0000 7feadb400640 1 -- 172.21.15.5:0/123375111 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564aa2df8670 msgr2=0x564aa2e029f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:59:23.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:23.630+0000 7feadb400640 1 --2- 172.21.15.5:0/123375111 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564aa2df8670 0x564aa2e029f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:23.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:23.630+0000 7feadb400640 1 -- 172.21.15.5:0/123375111 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564aa2e02f30 msgr2=0x564aa2e05320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:59:23.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:23.630+0000 7feadb400640 1 --2- 172.21.15.5:0/123375111 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564aa2e02f30 0x564aa2e05320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:23.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:23.630+0000 7feadb400640 1 -- 172.21.15.5:0/123375111 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564aa2d24ae0 con 0x564aa2df7d50 2024-09-10T09:59:23.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:23.630+0000 7feadb400640 1 --2- 172.21.15.5:0/123375111 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564aa2df7d50 0x564aa2df8130 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7fead4009fc0 tx=0x7fead4010860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=997e871f9dbad83e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:23.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:23.630+0000 7fead3400640 1 --2- 172.21.15.5:0/883071346 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7feac00a2c00 0x7feac00a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:23.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:23.631+0000 7feac7e00640 1 -- 172.21.15.5:0/123375111 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fead4011700 con 0x564aa2df7d50 2024-09-10T09:59:23.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:23.631+0000 7fead3400640 1 -- 172.21.15.5:0/883071346 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7feacc0cf930 con 0x7feadc074510 2024-09-10T09:59:23.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:23.631+0000 7feac7e00640 1 -- 172.21.15.5:0/123375111 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fead4011d80 con 0x564aa2df7d50 2024-09-10T09:59:23.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:23.631+0000 7feadaa00640 1 --2- 172.21.15.5:0/883071346 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7feac00a2c00 0x7feac00a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:23.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:23.631+0000 7feac7e00640 1 -- 172.21.15.5:0/123375111 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fead401a380 con 0x564aa2df7d50 2024-09-10T09:59:23.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:23.632+0000 7feae43ee740 1 -- 172.21.15.5:0/123375111 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564aa2df7d50 msgr2=0x564aa2df8130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:23.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:23.632+0000 7feadaa00640 1 --2- 172.21.15.5:0/883071346 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7feac00a2c00 0x7feac00a50c0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7feacc09bcd0 tx=0x7feacc0d5530 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:23.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:23.632+0000 7feae43ee740 1 --2- 172.21.15.5:0/123375111 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564aa2df7d50 0x564aa2df8130 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7fead4009fc0 tx=0x7fead4010860 comp rx=0 tx=0).stop 2024-09-10T09:59:23.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:23.632+0000 7feae43ee740 1 -- 172.21.15.5:0/123375111 shutdown_connections 2024-09-10T09:59:23.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:23.632+0000 7feae43ee740 1 --2- 172.21.15.5:0/123375111 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564aa2e02f30 0x564aa2e05320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:23.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:23.632+0000 7feae43ee740 1 --2- 172.21.15.5:0/123375111 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564aa2df8670 0x564aa2e029f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:23.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:23.632+0000 7feae43ee740 1 --2- 172.21.15.5:0/123375111 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564aa2df7d50 0x564aa2df8130 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:23.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:23.632+0000 7feae43ee740 1 -- 172.21.15.5:0/123375111 >> 172.21.15.5:0/123375111 conn(0x564aa2df42e0 msgr2=0x564aa2df46c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:23.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:23.632+0000 7feae43ee740 1 -- 172.21.15.5:0/123375111 shutdown_connections 2024-09-10T09:59:23.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:23.632+0000 7feae43ee740 1 -- 172.21.15.5:0/123375111 wait complete. 2024-09-10T09:59:23.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:23.632+0000 7feae43ee740 1 -- start start 2024-09-10T09:59:23.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:23.633+0000 7feae43ee740 1 Processor -- start 2024-09-10T09:59:23.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:23.633+0000 7feae43ee740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564aa2df7d50 0x564aa2dffea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:23.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:23.633+0000 7feae43ee740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564aa2df8670 0x564aa2e003e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:23.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:23.633+0000 7feae43ee740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564aa2e02f30 0x564aa2e00920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:23.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:23.633+0000 7feae43ee740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x564aa2e05ba0 con 0x564aa2e02f30 2024-09-10T09:59:23.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:23.633+0000 7feae43ee740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x564aa2d190f0 con 0x564aa2df8670 2024-09-10T09:59:23.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:23.633+0000 7feae43ee740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x564aa2e05d20 con 0x564aa2df7d50 2024-09-10T09:59:23.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:23.633+0000 7feae43ee740 10 client.0 _ll_register_callbacks cb 0x564aa2d0a9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:59:23.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:23.633+0000 7feadb400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564aa2df7d50 0x564aa2dffea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:23.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:23.633+0000 7feadb400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564aa2e02f30 0x564aa2e00920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:23.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:23.633+0000 7feadb400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564aa2e02f30 0x564aa2e00920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41338/0 (socket says 172.21.15.5:41338) 2024-09-10T09:59:23.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:23.634+0000 7feadb400640 1 -- 172.21.15.5:0/4266604083 learned_addr learned my addr 172.21.15.5:0/4266604083 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:59:23.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:23.634+0000 7feada000640 1 --2- 172.21.15.5:0/4266604083 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564aa2df8670 0x564aa2e003e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:23.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:23.634+0000 7feada000640 1 -- 172.21.15.5:0/4266604083 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564aa2df7d50 msgr2=0x564aa2dffea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:59:23.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:23.634+0000 7feada000640 1 --2- 172.21.15.5:0/4266604083 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564aa2df7d50 0x564aa2dffea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:23.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:23.634+0000 7feada000640 1 -- 172.21.15.5:0/4266604083 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564aa2e02f30 msgr2=0x564aa2e00920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:59:23.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:23.634+0000 7feada000640 1 --2- 172.21.15.5:0/4266604083 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564aa2e02f30 0x564aa2e00920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:23.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:23.634+0000 7feada000640 1 -- 172.21.15.5:0/4266604083 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564aa2acf3c0 con 0x564aa2df8670 2024-09-10T09:59:23.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:23.634+0000 7feadb400640 1 --2- 172.21.15.5:0/4266604083 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564aa2e02f30 0x564aa2e00920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:59:23.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:23.634+0000 7feada000640 1 --2- 172.21.15.5:0/4266604083 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564aa2df8670 0x564aa2e003e0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7feac8006010 tx=0x7feac8031860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=64b3167840459639 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:23.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:23.634+0000 7feac4c00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-09-10T09:59:23.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:23.635+0000 7feac4c00640 1 -- 172.21.15.5:0/4266604083 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feac8035040 con 0x564aa2df8670 2024-09-10T09:59:23.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:23.635+0000 7feac4c00640 1 -- 172.21.15.5:0/4266604083 <== mon.2 v2:172.21.15.102:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7feac8004460 con 0x564aa2df8670 2024-09-10T09:59:23.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:23.635+0000 7feae43ee740 1 -- 172.21.15.5:0/4266604083 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fead4011d80 con 0x564aa2df8670 2024-09-10T09:59:23.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:23.635+0000 7feac4c00640 1 -- 172.21.15.5:0/4266604083 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7feac8005030 con 0x564aa2df8670 2024-09-10T09:59:23.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:23.635+0000 7feae43ee740 10 client.29516 fetch_fsmap learned FSMap version 42 2024-09-10T09:59:23.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:23.635+0000 7feae43ee740 1 -- 172.21.15.5:0/4266604083 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x564aa2e11920 con 0x564aa2df8670 2024-09-10T09:59:23.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:23.635+0000 7feac4c00640 1 -- 172.21.15.5:0/4266604083 <== mon.2 v2:172.21.15.102:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fead4011d80 con 0x564aa2df8670 2024-09-10T09:59:23.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:23.636+0000 7feac4c00640 1 -- 172.21.15.5:0/4266604083 <== mon.2 v2:172.21.15.102:3300/0 5 ==== fsmap(e 42) ==== 8370+0+0 (secure 0 0 0) 0x7feac8004c50 con 0x564aa2df8670 2024-09-10T09:59:23.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:23.636+0000 7feae43ee740 10 client.29516 fetch_fsmap finished waiting for FSMap version 42 2024-09-10T09:59:23.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:23.636+0000 7feae43ee740 10 client.29516 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T09:59:23.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:23.636+0000 7feae43ee740 20 client.29516 populate_metadata read hostname 'smithi005' 2024-09-10T09:59:23.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:23.636+0000 7feae43ee740 1 --2- 172.21.15.5:0/4266604083 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x564aa2e12de0 0x564aa2e151d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:23.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:23.636+0000 7feae43ee740 4 client.29516 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:59:23.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:23.636+0000 7feae43ee740 1 -- 172.21.15.5:0/4266604083 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564aa2d6b950 con 0x564aa2e12de0 2024-09-10T09:59:23.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:23.636+0000 7feadb400640 1 --2- 172.21.15.5:0/4266604083 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x564aa2e12de0 0x564aa2e151d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:23.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:23.637+0000 7feadb400640 1 --2- 172.21.15.5:0/4266604083 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x564aa2e12de0 0x564aa2e151d0 crc :-1 s=READY pgs=295 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=26c7d98a7a176c9d server_cookie=99ababebc49bdf64 in_seq=0 out_seq=0 2024-09-10T09:59:23.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:23.637+0000 7feac4c00640 10 client.29516 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T09:59:23.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:23.638+0000 7feac4c00640 1 -- 172.21.15.5:0/4266604083 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564aa2d6b950 con 0x564aa2e12de0 2024-09-10T09:59:23.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:23.638+0000 7feac4c00640 10 client.29516 handle_command_reply: tid=0 2024-09-10T09:59:23.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:23.638+0000 7feae43ee740 1 client.29516 shutdown 2024-09-10T09:59:23.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:23.638+0000 7feae43ee740 1 -- 172.21.15.5:0/4266604083 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564aa2df8670 msgr2=0x564aa2e003e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:23.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:23.638+0000 7feae43ee740 1 --2- 172.21.15.5:0/4266604083 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564aa2df8670 0x564aa2e003e0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7feac8006010 tx=0x7feac8031860 comp rx=0 tx=0).stop 2024-09-10T09:59:23.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:23.638+0000 7feae43ee740 1 -- 172.21.15.5:0/4266604083 shutdown_connections 2024-09-10T09:59:23.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:23.638+0000 7feae43ee740 1 --2- 172.21.15.5:0/4266604083 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x564aa2e12de0 0x564aa2e151d0 crc :-1 s=READY pgs=295 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:23.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:23.638+0000 7feae43ee740 1 --2- 172.21.15.5:0/4266604083 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564aa2e02f30 0x564aa2e00920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:23.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:23.638+0000 7feae43ee740 1 --2- 172.21.15.5:0/4266604083 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564aa2df8670 0x564aa2e003e0 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:23.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:23.638+0000 7feae43ee740 1 --2- 172.21.15.5:0/4266604083 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564aa2df7d50 0x564aa2dffea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:23.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:23.638+0000 7feac4c00640 0 client.29516 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T09:59:23.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:23.638+0000 7feae43ee740 1 -- 172.21.15.5:0/4266604083 >> 172.21.15.5:0/4266604083 conn(0x564aa2df42e0 msgr2=0x564aa2df9fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:23.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:23.638+0000 7feae43ee740 1 -- 172.21.15.5:0/4266604083 shutdown_connections 2024-09-10T09:59:23.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:23.638+0000 7feae43ee740 1 -- 172.21.15.5:0/4266604083 wait complete. 2024-09-10T09:59:23.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:23.638+0000 7feae43ee740 20 client.29516 trim_cache size 0 max 16384 2024-09-10T09:59:23.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:23.656+0000 7feae43ee740 1 Processor -- start 2024-09-10T09:59:23.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:23.657+0000 7feae43ee740 1 -- start start 2024-09-10T09:59:23.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:23.657+0000 7feae43ee740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564aa2df7d50 0x564aa2e00770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:23.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:23.657+0000 7feae43ee740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564aa2df8670 0x564aa2e01e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:23.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:23.657+0000 7feae43ee740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564aa2e02f30 0x564aa2e1aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:23.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:23.657+0000 7feae43ee740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7feadc1b0ea0 con 0x564aa2df8670 2024-09-10T09:59:23.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:23.657+0000 7feae43ee740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7feadc1b29e0 con 0x564aa2df7d50 2024-09-10T09:59:23.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:23.657+0000 7feae43ee740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7feadc1b1570 con 0x564aa2e02f30 2024-09-10T09:59:23.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:23.657+0000 7feada000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564aa2df8670 0x564aa2e01e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:23.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:23.657+0000 7feada000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564aa2df8670 0x564aa2e01e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41352/0 (socket says 172.21.15.5:41352) 2024-09-10T09:59:23.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:23.657+0000 7feadb400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564aa2df7d50 0x564aa2e00770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:23.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:23.657+0000 7feada000640 1 -- 172.21.15.5:0/1030318543 learned_addr learned my addr 172.21.15.5:0/1030318543 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:59:23.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:23.657+0000 7feadb400640 1 --2- 172.21.15.5:0/1030318543 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564aa2e02f30 0x564aa2e1aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:23.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:23.658+0000 7feada000640 1 -- 172.21.15.5:0/1030318543 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564aa2e02f30 msgr2=0x564aa2e1aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:59:23.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:23.658+0000 7feada000640 1 --2- 172.21.15.5:0/1030318543 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564aa2e02f30 0x564aa2e1aaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:23.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:23.658+0000 7feada000640 1 -- 172.21.15.5:0/1030318543 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564aa2df7d50 msgr2=0x564aa2e00770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:59:23.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:23.658+0000 7feada000640 1 --2- 172.21.15.5:0/1030318543 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564aa2df7d50 0x564aa2e00770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:23.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:23.658+0000 7feada000640 1 -- 172.21.15.5:0/1030318543 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564aa2e0ee00 con 0x564aa2df8670 2024-09-10T09:59:23.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:23.658+0000 7feada000640 1 --2- 172.21.15.5:0/1030318543 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564aa2df8670 0x564aa2e01e10 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7feac8003800 tx=0x7feac8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc6b9f6309ae8a14 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:23.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:23.658+0000 7feac7400640 1 -- 172.21.15.5:0/1030318543 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feac8056070 con 0x564aa2df8670 2024-09-10T09:59:23.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:23.658+0000 7feac7400640 1 -- 172.21.15.5:0/1030318543 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feac8040070 con 0x564aa2df8670 2024-09-10T09:59:23.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:23.658+0000 7feac7400640 1 -- 172.21.15.5:0/1030318543 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feac8035050 con 0x564aa2df8670 2024-09-10T09:59:23.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:23.659+0000 7feae43ee740 1 -- 172.21.15.5:0/1030318543 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564aa2df8670 msgr2=0x564aa2e01e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:23.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:23.659+0000 7feae43ee740 1 --2- 172.21.15.5:0/1030318543 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564aa2df8670 0x564aa2e01e10 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7feac8003800 tx=0x7feac8009250 comp rx=0 tx=0).stop 2024-09-10T09:59:23.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:23.659+0000 7feae43ee740 1 -- 172.21.15.5:0/1030318543 shutdown_connections 2024-09-10T09:59:23.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:23.659+0000 7feae43ee740 1 --2- 172.21.15.5:0/1030318543 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564aa2e02f30 0x564aa2e1aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:23.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:23.659+0000 7feae43ee740 1 --2- 172.21.15.5:0/1030318543 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564aa2df8670 0x564aa2e01e10 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:23.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:23.659+0000 7feae43ee740 1 --2- 172.21.15.5:0/1030318543 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564aa2df7d50 0x564aa2e00770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:23.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:23.659+0000 7feae43ee740 1 -- 172.21.15.5:0/1030318543 >> 172.21.15.5:0/1030318543 conn(0x564aa2df42e0 msgr2=0x564aa2e01700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:23.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:23.659+0000 7feae43ee740 1 -- 172.21.15.5:0/1030318543 shutdown_connections 2024-09-10T09:59:23.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:23.659+0000 7feae43ee740 1 -- 172.21.15.5:0/1030318543 wait complete. 2024-09-10T09:59:23.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:23.660+0000 7feae43ee740 1 -- start start 2024-09-10T09:59:23.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:23.660+0000 7feae43ee740 1 Processor -- start 2024-09-10T09:59:23.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:23.660+0000 7feae43ee740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564aa2df7d50 0x564aa2e0af70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:23.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:23.660+0000 7feae43ee740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564aa2df8670 0x564aa2e0b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:23.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:23.660+0000 7feae43ee740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564aa2e02f30 0x564aa2e07f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:23.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:23.660+0000 7feae43ee740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7feadc1b2310 con 0x564aa2df8670 2024-09-10T09:59:23.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:23.660+0000 7feae43ee740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7feadc1b1c40 con 0x564aa2df7d50 2024-09-10T09:59:23.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:23.660+0000 7feae43ee740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7feadc1aee80 con 0x564aa2e02f30 2024-09-10T09:59:23.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:23.660+0000 7feae43ee740 10 client.0 _ll_register_callbacks cb 0x7feac8005d50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:59:23.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:23.660+0000 7feada000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564aa2df8670 0x564aa2e0b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:23.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:23.661+0000 7feada000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564aa2df8670 0x564aa2e0b4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41358/0 (socket says 172.21.15.5:41358) 2024-09-10T09:59:23.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:23.661+0000 7feada000640 1 -- 172.21.15.5:0/1314341301 learned_addr learned my addr 172.21.15.5:0/1314341301 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:59:23.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:23.661+0000 7feadb400640 1 --2- 172.21.15.5:0/1314341301 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564aa2df7d50 0x564aa2e0af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:23.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:23.661+0000 7feadb400640 1 --2- 172.21.15.5:0/1314341301 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564aa2e02f30 0x564aa2e07f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:23.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:23.661+0000 7feada000640 1 -- 172.21.15.5:0/1314341301 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564aa2e02f30 msgr2=0x564aa2e07f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:59:23.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:23.661+0000 7feada000640 1 --2- 172.21.15.5:0/1314341301 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564aa2e02f30 0x564aa2e07f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:23.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:23.661+0000 7feada000640 1 -- 172.21.15.5:0/1314341301 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564aa2df7d50 msgr2=0x564aa2e0af70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:59:23.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:23.661+0000 7feada000640 1 --2- 172.21.15.5:0/1314341301 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564aa2df7d50 0x564aa2e0af70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:23.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:23.661+0000 7feada000640 1 -- 172.21.15.5:0/1314341301 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564aa2e0bba0 con 0x564aa2df8670 2024-09-10T09:59:23.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:23.661+0000 7feadb400640 1 --2- 172.21.15.5:0/1314341301 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564aa2df7d50 0x564aa2e0af70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:59:23.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:23.661+0000 7feada000640 1 --2- 172.21.15.5:0/1314341301 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564aa2df8670 0x564aa2e0b4b0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7feac80037d0 tx=0x7feac8032a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=837de06bec3c33fe server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:23.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:23.661+0000 7feac4c00640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-10T09:59:23.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:23.661+0000 7feac4c00640 1 -- 172.21.15.5:0/1314341301 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feac8035070 con 0x564aa2df8670 2024-09-10T09:59:23.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:23.662+0000 7feac4c00640 1 -- 172.21.15.5:0/1314341301 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7feac80051d0 con 0x564aa2df8670 2024-09-10T09:59:23.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:23.662+0000 7feae43ee740 1 -- 172.21.15.5:0/1314341301 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feac8040070 con 0x564aa2df8670 2024-09-10T09:59:23.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:23.662+0000 7feac4c00640 1 -- 172.21.15.5:0/1314341301 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7feac8063050 con 0x564aa2df8670 2024-09-10T09:59:23.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:23.662+0000 7feae43ee740 10 client.19458 fetch_fsmap learned FSMap version 42 2024-09-10T09:59:23.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:23.662+0000 7feae43ee740 1 -- 172.21.15.5:0/1314341301 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x564aa2e7e060 con 0x564aa2df8670 2024-09-10T09:59:23.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:23.662+0000 7feac4c00640 1 -- 172.21.15.5:0/1314341301 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feac8040070 con 0x564aa2df8670 2024-09-10T09:59:23.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:23.662+0000 7feac4c00640 1 -- 172.21.15.5:0/1314341301 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 42) ==== 8370+0+0 (secure 0 0 0) 0x7feac8004c30 con 0x564aa2df8670 2024-09-10T09:59:23.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:23.662+0000 7feae43ee740 10 client.19458 fetch_fsmap finished waiting for FSMap version 42 2024-09-10T09:59:23.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:23.662+0000 7feae43ee740 10 client.19458 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T09:59:23.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:23.662+0000 7feae43ee740 20 client.19458 populate_metadata read hostname 'smithi005' 2024-09-10T09:59:23.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:23.662+0000 7feae43ee740 1 --2- 172.21.15.5:0/1314341301 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x564aa2e7e720 0x564aa2e80b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:23.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:23.662+0000 7feae43ee740 4 client.19458 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T09:59:23.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:23.662+0000 7feae43ee740 1 -- 172.21.15.5:0/1314341301 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "scrub status"}) -- 0x564aa2d65e60 con 0x564aa2e7e720 2024-09-10T09:59:23.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:23.663+0000 7feadb400640 1 --2- 172.21.15.5:0/1314341301 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x564aa2e7e720 0x564aa2e80b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:23.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:23.663+0000 7feadb400640 1 --2- 172.21.15.5:0/1314341301 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x564aa2e7e720 0x564aa2e80b40 crc :-1 s=READY pgs=296 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4e00ffe28f3c3595 server_cookie=465b6be89ea67979 in_seq=0 out_seq=0 2024-09-10T09:59:23.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:23.663+0000 7feac4c00640 10 client.19458 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T09:59:23.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:23.663+0000 7feac4c00640 1 -- 172.21.15.5:0/1314341301 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x564aa2d65e60 con 0x564aa2e7e720 2024-09-10T09:59:23.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:23.663+0000 7feac4c00640 10 client.19458 handle_command_reply: tid=0 2024-09-10T09:59:23.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:23.663+0000 7feae43ee740 1 client.19458 shutdown 2024-09-10T09:59:23.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:23.664+0000 7feae43ee740 1 -- 172.21.15.5:0/1314341301 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564aa2df8670 msgr2=0x564aa2e0b4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:23.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:23.664+0000 7feae43ee740 1 --2- 172.21.15.5:0/1314341301 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564aa2df8670 0x564aa2e0b4b0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7feac80037d0 tx=0x7feac8032a00 comp rx=0 tx=0).stop 2024-09-10T09:59:23.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:23.664+0000 7feae43ee740 1 -- 172.21.15.5:0/1314341301 shutdown_connections 2024-09-10T09:59:23.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:23.664+0000 7feae43ee740 1 --2- 172.21.15.5:0/1314341301 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x564aa2e7e720 0x564aa2e80b40 crc :-1 s=READY pgs=296 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:23.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:23.664+0000 7feae43ee740 1 --2- 172.21.15.5:0/1314341301 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564aa2e02f30 0x564aa2e07f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:23.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:23.664+0000 7feae43ee740 1 --2- 172.21.15.5:0/1314341301 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564aa2df8670 0x564aa2e0b4b0 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:23.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:23.664+0000 7feae43ee740 1 --2- 172.21.15.5:0/1314341301 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564aa2df7d50 0x564aa2e0af70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:23.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:23.664+0000 7feae43ee740 1 -- 172.21.15.5:0/1314341301 >> 172.21.15.5:0/1314341301 conn(0x564aa2df42e0 msgr2=0x564aa2e0e610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:23.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:23.664+0000 7feac4c00640 0 client.19458 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T09:59:23.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:23.664+0000 7feae43ee740 1 -- 172.21.15.5:0/1314341301 shutdown_connections 2024-09-10T09:59:23.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:23.664+0000 7feae43ee740 1 -- 172.21.15.5:0/1314341301 wait complete. 2024-09-10T09:59:23.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:23.664+0000 7feae43ee740 20 client.19458 trim_cache size 0 max 16384 2024-09-10T09:59:23.669 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-10T09:59:23.669 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-10T09:59:23.669 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2024-09-10T09:59:23.669 INFO:teuthology.orchestra.run.smithi005.stdout: "2c3987c4-1e83-487a-8209-5f7e4b276597": { 2024-09-10T09:59:23.669 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2024-09-10T09:59:23.669 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "2c3987c4-1e83-487a-8209-5f7e4b276597", 2024-09-10T09:59:23.669 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2024-09-10T09:59:23.669 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T09:59:23.670 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T09:59:23.670 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-10T09:59:23.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:23.665+0000 7feadbe00640 1 -- 172.21.15.5:0/883071346 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7feac00a2c00 msgr2=0x7feac00a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:23.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:23.665+0000 7feadbe00640 1 --2- 172.21.15.5:0/883071346 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7feac00a2c00 0x7feac00a50c0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7feacc09bcd0 tx=0x7feacc0d5530 comp rx=0 tx=0).stop 2024-09-10T09:59:23.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:23.665+0000 7feadbe00640 1 -- 172.21.15.5:0/883071346 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feadc074510 msgr2=0x7feadc19a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:23.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:23.665+0000 7feadbe00640 1 --2- 172.21.15.5:0/883071346 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feadc074510 0x7feadc19a480 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7feacc00e9f0 tx=0x7feacc00eeb0 comp rx=0 tx=0).stop 2024-09-10T09:59:23.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:23.665+0000 7feadbe00640 1 -- 172.21.15.5:0/883071346 shutdown_connections 2024-09-10T09:59:23.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:23.665+0000 7feadbe00640 1 --2- 172.21.15.5:0/883071346 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7feac00a2c00 0x7feac00a50c0 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:23.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:23.665+0000 7feadbe00640 1 --2- 172.21.15.5:0/883071346 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7feadc10e280 0x7feadc19ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:23.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:23.665+0000 7feadbe00640 1 --2- 172.21.15.5:0/883071346 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7feadc074e30 0x7feadc19a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:23.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:23.665+0000 7feadbe00640 1 --2- 172.21.15.5:0/883071346 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feadc074510 0x7feadc19a480 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:23.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:23.665+0000 7feadbe00640 1 -- 172.21.15.5:0/883071346 >> 172.21.15.5:0/883071346 conn(0x7feadc0fd6e0 msgr2=0x7feadc0fee70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:23.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:23.665+0000 7feadbe00640 1 -- 172.21.15.5:0/883071346 shutdown_connections 2024-09-10T09:59:23.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:23.665+0000 7feadbe00640 1 -- 172.21.15.5:0/883071346 wait complete. 2024-09-10T09:59:24.026 INFO:tasks.workunit.client.0.smithi005.stdout: 10 110875 12.82 MB/sec execute 302 sec latency 26691.665 ms 2024-09-10T09:59:24.306 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:59:23 smithi005 ceph-mon[30350]: pgmap v1024: 129 pgs: 129 active+clean; 654 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 1.9 MiB/s wr, 83 op/s 2024-09-10T09:59:24.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:59:23 smithi159 ceph-mon[35513]: pgmap v1024: 129 pgs: 129 active+clean; 654 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 1.9 MiB/s wr, 83 op/s 2024-09-10T09:59:24.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:59:23 smithi102 ceph-mon[35442]: pgmap v1024: 129 pgs: 129 active+clean; 654 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 1.9 MiB/s wr, 83 op/s 2024-09-10T09:59:24.527 INFO:tasks.cephfs.filesystem:scrub status for tag:2c3987c4-1e83-487a-8209-5f7e4b276597 - {'path': '/', 'tag': '2c3987c4-1e83-487a-8209-5f7e4b276597', 'options': 'recursive,force'} 2024-09-10T09:59:25.026 INFO:tasks.workunit.client.0.smithi005.stdout: 10 110875 12.77 MB/sec execute 303 sec latency 27691.780 ms 2024-09-10T09:59:26.027 INFO:tasks.workunit.client.0.smithi005.stdout: 10 110875 12.73 MB/sec execute 304 sec latency 28691.942 ms 2024-09-10T09:59:26.081 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T09:59:26.306 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:59:25 smithi005 ceph-mon[30350]: pgmap v1025: 129 pgs: 129 active+clean; 654 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 1.5 MiB/s wr, 69 op/s 2024-09-10T09:59:26.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:59:25 smithi159 ceph-mon[35513]: pgmap v1025: 129 pgs: 129 active+clean; 654 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 1.5 MiB/s wr, 69 op/s 2024-09-10T09:59:26.385 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:59:25 smithi102 ceph-mon[35442]: pgmap v1025: 129 pgs: 129 active+clean; 654 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 1.5 MiB/s wr, 69 op/s 2024-09-10T09:59:26.475 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:59:27.027 INFO:tasks.workunit.client.0.smithi005.stdout: 10 110875 12.69 MB/sec execute 305 sec latency 29692.100 ms 2024-09-10T09:59:27.169 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:59:26 smithi005 ceph-mon[30350]: pgmap v1026: 129 pgs: 129 active+clean; 654 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 1.2 MiB/s wr, 66 op/s 2024-09-10T09:59:27.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:27.172+0000 7f0eda800640 1 -- 172.21.15.5:0/1050854992 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0ed41049a0 msgr2=0x7f0ed41089f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:27.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:27.172+0000 7f0eda800640 1 --2- 172.21.15.5:0/1050854992 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0ed41049a0 0x7f0ed41089f0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f0ec0009900 tx=0x7f0ec002a410 comp rx=0 tx=0).stop 2024-09-10T09:59:27.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:27.172+0000 7f0eda800640 1 -- 172.21.15.5:0/1050854992 shutdown_connections 2024-09-10T09:59:27.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:27.172+0000 7f0eda800640 1 --2- 172.21.15.5:0/1050854992 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0ed4109120 0x7f0ed4110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:27.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:27.172+0000 7f0eda800640 1 --2- 172.21.15.5:0/1050854992 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0ed41049a0 0x7f0ed41089f0 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:27.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:27.172+0000 7f0eda800640 1 --2- 172.21.15.5:0/1050854992 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0ed4103ff0 0x7f0ed41043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:27.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:27.172+0000 7f0eda800640 1 -- 172.21.15.5:0/1050854992 >> 172.21.15.5:0/1050854992 conn(0x7f0ed40ffa90 msgr2=0x7f0ed4101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:27.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:27.172+0000 7f0eda800640 1 -- 172.21.15.5:0/1050854992 shutdown_connections 2024-09-10T09:59:27.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:27.172+0000 7f0eda800640 1 -- 172.21.15.5:0/1050854992 wait complete. 2024-09-10T09:59:27.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:27.173+0000 7f0eda800640 1 Processor -- start 2024-09-10T09:59:27.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:27.173+0000 7f0eda800640 1 -- start start 2024-09-10T09:59:27.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:27.173+0000 7f0eda800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0ed4103ff0 0x7f0ed419a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:27.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:27.173+0000 7f0eda800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0ed41049a0 0x7f0ed419abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:27.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:27.173+0000 7f0eda800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0ed4109120 0x7f0ed419ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:27.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:27.173+0000 7f0eda800640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f0ed4114140 con 0x7f0ed4109120 2024-09-10T09:59:27.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:27.173+0000 7f0eda800640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f0ed4113fc0 con 0x7f0ed41049a0 2024-09-10T09:59:27.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:27.173+0000 7f0eda800640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f0ed41142c0 con 0x7f0ed4103ff0 2024-09-10T09:59:27.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:27.174+0000 7f0ed3e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0ed4109120 0x7f0ed419ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:27.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:27.174+0000 7f0ed3400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0ed4103ff0 0x7f0ed419a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:27.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:27.174+0000 7f0ed3e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0ed4109120 0x7f0ed419ef70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41374/0 (socket says 172.21.15.5:41374) 2024-09-10T09:59:27.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:27.174+0000 7f0ed3e00640 1 -- 172.21.15.5:0/3328354561 learned_addr learned my addr 172.21.15.5:0/3328354561 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:59:27.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:27.174+0000 7f0ed2a00640 1 --2- 172.21.15.5:0/3328354561 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0ed41049a0 0x7f0ed419abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:27.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:27.174+0000 7f0ed3e00640 1 -- 172.21.15.5:0/3328354561 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0ed4103ff0 msgr2=0x7f0ed419a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:27.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:27.174+0000 7f0ed3e00640 1 --2- 172.21.15.5:0/3328354561 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0ed4103ff0 0x7f0ed419a6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:27.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:27.174+0000 7f0ed3e00640 1 -- 172.21.15.5:0/3328354561 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0ed41049a0 msgr2=0x7f0ed419abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:27.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:27.174+0000 7f0ed3e00640 1 --2- 172.21.15.5:0/3328354561 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0ed41049a0 0x7f0ed419abe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:27.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:27.174+0000 7f0ed3e00640 1 -- 172.21.15.5:0/3328354561 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ed419f650 con 0x7f0ed4109120 2024-09-10T09:59:27.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:27.174+0000 7f0ed3400640 1 --2- 172.21.15.5:0/3328354561 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0ed4103ff0 0x7f0ed419a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:59:27.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:27.175+0000 7f0ed3e00640 1 --2- 172.21.15.5:0/3328354561 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0ed4109120 0x7f0ed419ef70 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f0ecc00eb10 tx=0x7f0ecc00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:27.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:27.175+0000 7f0ecbe00640 1 -- 172.21.15.5:0/3328354561 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ecc004b20 con 0x7f0ed4109120 2024-09-10T09:59:27.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:27.175+0000 7f0ecbe00640 1 -- 172.21.15.5:0/3328354561 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0ecc004cc0 con 0x7f0ed4109120 2024-09-10T09:59:27.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:27.175+0000 7f0ecbe00640 1 -- 172.21.15.5:0/3328354561 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ecc010760 con 0x7f0ed4109120 2024-09-10T09:59:27.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:27.175+0000 7f0eda800640 1 -- 172.21.15.5:0/3328354561 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ed419f940 con 0x7f0ed4109120 2024-09-10T09:59:27.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:27.175+0000 7f0eda800640 1 -- 172.21.15.5:0/3328354561 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ed41a7220 con 0x7f0ed4109120 2024-09-10T09:59:27.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:27.176+0000 7f0eda800640 1 -- 172.21.15.5:0/3328354561 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ed41043d0 con 0x7f0ed4109120 2024-09-10T09:59:27.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:27.180+0000 7f0ecbe00640 1 -- 172.21.15.5:0/3328354561 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f0ecc005000 con 0x7f0ed4109120 2024-09-10T09:59:27.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:27.181+0000 7f0ecbe00640 1 --2- 172.21.15.5:0/3328354561 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f0eb80a2c00 0x7f0eb80a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:27.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:27.181+0000 7f0ecbe00640 1 -- 172.21.15.5:0/3328354561 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f0ecc0cfaf0 con 0x7f0ed4109120 2024-09-10T09:59:27.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:27.181+0000 7f0ed3400640 1 --2- 172.21.15.5:0/3328354561 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f0eb80a2c00 0x7f0eb80a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:27.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:27.182+0000 7f0ed3400640 1 --2- 172.21.15.5:0/3328354561 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f0eb80a2c00 0x7f0eb80a50c0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f0ec40095f0 tx=0x7f0ec4005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:27.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:27.184+0000 7f0ecbe00640 1 -- 172.21.15.5:0/3328354561 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f0ecc0541b0 con 0x7f0ed4109120 2024-09-10T09:59:27.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:59:26 smithi159 ceph-mon[35513]: pgmap v1026: 129 pgs: 129 active+clean; 654 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 1.2 MiB/s wr, 66 op/s 2024-09-10T09:59:27.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:59:26 smithi102 ceph-mon[35442]: pgmap v1026: 129 pgs: 129 active+clean; 654 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 1.2 MiB/s wr, 66 op/s 2024-09-10T09:59:27.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:27.405+0000 7f0eda800640 1 -- 172.21.15.5:0/3328354561 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0ed419ba00 con 0x7f0ed4109120 2024-09-10T09:59:27.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:27.406+0000 7f0ecbe00640 1 -- 172.21.15.5:0/3328354561 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 42 v42) ==== 94+0+11969 (secure 0 0 0) 0x7f0ed419ba00 con 0x7f0ed4109120 2024-09-10T09:59:27.408 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-10T09:59:27.409 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":42,"btime":"2024-09-10T09:55:06:670916+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14454,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/2970594311","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2970594311},{"type":"v1","addr":"172.21.15.5:6835","nonce":2970594311}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14514,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6839/3616502174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":3616502174},{"type":"v1","addr":"172.21.15.5:6839","nonce":3616502174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":42,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T09:35:29.626287+0000","modified":"2024-09-10T09:55:06.419829+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24542,"mds_1":24535,"mds_2":24511,"mds_3":24506,"mds_4":24443},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14520":{"gid":14520,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/2073599842","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":2073599842},{"type":"v1","addr":"172.21.15.5:6841","nonce":2073599842}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"k","rank":4,"incarnation":26,"state":"up:active","state_seq":24,"addr":"172.21.15.102:6833/2457007427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":2457007427},{"type":"v1","addr":"172.21.15.102:6833","nonce":2457007427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/356445159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":356445159},{"type":"v1","addr":"172.21.15.159:6833","nonce":356445159}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6837/2894314507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":2894314507},{"type":"v1","addr":"172.21.15.5:6837","nonce":2894314507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.102:6835/1647549111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":1647549111},{"type":"v1","addr":"172.21.15.102:6835","nonce":1647549111}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6835/952657242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":952657242},{"type":"v1","addr":"172.21.15.159:6835","nonce":952657242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":19,"addr":"172.21.15.102:6837/2689776764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":2689776764},{"type":"v1","addr":"172.21.15.102:6837","nonce":2689776764}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.159:6837/620627639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":620627639},{"type":"v1","addr":"172.21.15.159:6837","nonce":620627639}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24535":{"gid":24535,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":18,"addr":"172.21.15.159:6839/2073739599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":2073739599},{"type":"v1","addr":"172.21.15.159:6839","nonce":2073739599}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24542":{"gid":24542,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.102:6839/335409749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6838","nonce":335409749},{"type":"v1","addr":"172.21.15.102:6839","nonce":335409749}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24542,"qdb_cluster":[24542,24535,24511,24506,24443]},"id":1}]} 2024-09-10T09:59:27.409 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 42 2024-09-10T09:59:27.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:27.409+0000 7f0eda800640 1 -- 172.21.15.5:0/3328354561 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f0eb80a2c00 msgr2=0x7f0eb80a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:27.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:27.409+0000 7f0eda800640 1 --2- 172.21.15.5:0/3328354561 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f0eb80a2c00 0x7f0eb80a50c0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f0ec40095f0 tx=0x7f0ec4005f90 comp rx=0 tx=0).stop 2024-09-10T09:59:27.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:27.409+0000 7f0eda800640 1 -- 172.21.15.5:0/3328354561 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0ed4109120 msgr2=0x7f0ed419ef70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:27.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:27.409+0000 7f0eda800640 1 --2- 172.21.15.5:0/3328354561 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0ed4109120 0x7f0ed419ef70 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f0ecc00eb10 tx=0x7f0ecc00c2d0 comp rx=0 tx=0).stop 2024-09-10T09:59:27.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:27.409+0000 7f0eda800640 1 -- 172.21.15.5:0/3328354561 shutdown_connections 2024-09-10T09:59:27.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:27.409+0000 7f0eda800640 1 --2- 172.21.15.5:0/3328354561 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f0eb80a2c00 0x7f0eb80a50c0 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:27.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:27.409+0000 7f0eda800640 1 --2- 172.21.15.5:0/3328354561 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0ed4109120 0x7f0ed419ef70 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:27.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:27.409+0000 7f0eda800640 1 --2- 172.21.15.5:0/3328354561 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0ed41049a0 0x7f0ed419abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:27.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:27.409+0000 7f0eda800640 1 --2- 172.21.15.5:0/3328354561 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0ed4103ff0 0x7f0ed419a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:27.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:27.409+0000 7f0eda800640 1 -- 172.21.15.5:0/3328354561 >> 172.21.15.5:0/3328354561 conn(0x7f0ed40ffa90 msgr2=0x7f0ed4100d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:27.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:27.409+0000 7f0eda800640 1 -- 172.21.15.5:0/3328354561 shutdown_connections 2024-09-10T09:59:27.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:27.409+0000 7f0eda800640 1 -- 172.21.15.5:0/3328354561 wait complete. 2024-09-10T09:59:28.027 INFO:tasks.workunit.client.0.smithi005.stdout: 10 110875 12.65 MB/sec execute 306 sec latency 30692.198 ms 2024-09-10T09:59:28.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:59:27 smithi005 ceph-mon[30350]: from='client.? 172.21.15.5:0/3328354561' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:59:28.280 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24542 quiesce db --set-id 830978ed --release --await' 2024-09-10T09:59:28.280 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24542 quiesce db --set-id 830978ed --release --await 2024-09-10T09:59:28.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:59:27 smithi159 ceph-mon[35513]: from='client.? 172.21.15.5:0/3328354561' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:59:28.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:59:27 smithi102 ceph-mon[35442]: from='client.? 172.21.15.5:0/3328354561' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:59:28.659 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:59:29.027 INFO:tasks.workunit.client.0.smithi005.stdout: 10 110875 12.61 MB/sec execute 307 sec latency 31692.291 ms 2024-09-10T09:59:29.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:59:28 smithi005 ceph-mon[30350]: pgmap v1027: 129 pgs: 129 active+clean; 654 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 831 KiB/s wr, 42 op/s 2024-09-10T09:59:29.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:29.221+0000 7fc853e00640 1 -- 172.21.15.5:0/2931455029 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc854108f60 msgr2=0x7fc854110ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:29.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:29.221+0000 7fc853e00640 1 --2- 172.21.15.5:0/2931455029 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc854108f60 0x7fc854110ae0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7fc84c009ca0 tx=0x7fc84c02a8b0 comp rx=0 tx=0).stop 2024-09-10T09:59:29.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:29.221+0000 7fc853e00640 1 -- 172.21.15.5:0/2931455029 shutdown_connections 2024-09-10T09:59:29.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:29.221+0000 7fc853e00640 1 --2- 172.21.15.5:0/2931455029 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc854108f60 0x7fc854110ae0 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:29.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:29.221+0000 7fc853e00640 1 --2- 172.21.15.5:0/2931455029 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc8541047e0 0x7fc854108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:29.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:29.221+0000 7fc853e00640 1 --2- 172.21.15.5:0/2931455029 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc854103e30 0x7fc854104210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:29.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:29.221+0000 7fc853e00640 1 -- 172.21.15.5:0/2931455029 >> 172.21.15.5:0/2931455029 conn(0x7fc8540ff910 msgr2=0x7fc854101d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:29.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:29.221+0000 7fc853e00640 1 -- 172.21.15.5:0/2931455029 shutdown_connections 2024-09-10T09:59:29.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:29.221+0000 7fc853e00640 1 -- 172.21.15.5:0/2931455029 wait complete. 2024-09-10T09:59:29.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:29.222+0000 7fc853e00640 1 Processor -- start 2024-09-10T09:59:29.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:29.222+0000 7fc853e00640 1 -- start start 2024-09-10T09:59:29.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:29.223+0000 7fc853e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc854103e30 0x7fc85419a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:29.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:29.223+0000 7fc853e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc8541047e0 0x7fc85419a970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:29.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:29.223+0000 7fc853e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc854108f60 0x7fc85419ed00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:29.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:29.223+0000 7fc853e00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc854114010 con 0x7fc854108f60 2024-09-10T09:59:29.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:29.223+0000 7fc853e00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fc854113e90 con 0x7fc854103e30 2024-09-10T09:59:29.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:29.223+0000 7fc853e00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fc854114190 con 0x7fc8541047e0 2024-09-10T09:59:29.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:29.223+0000 7fc853400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc854108f60 0x7fc85419ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:29.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:29.223+0000 7fc853400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc854108f60 0x7fc85419ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41380/0 (socket says 172.21.15.5:41380) 2024-09-10T09:59:29.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:29.223+0000 7fc852000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc8541047e0 0x7fc85419a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:29.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:29.223+0000 7fc853400640 1 -- 172.21.15.5:0/2478547338 learned_addr learned my addr 172.21.15.5:0/2478547338 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:59:29.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:29.223+0000 7fc852a00640 1 --2- 172.21.15.5:0/2478547338 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc854103e30 0x7fc85419a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:29.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:29.224+0000 7fc852000640 1 -- 172.21.15.5:0/2478547338 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc854103e30 msgr2=0x7fc85419a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:29.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:29.224+0000 7fc852000640 1 --2- 172.21.15.5:0/2478547338 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc854103e30 0x7fc85419a430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:29.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:29.224+0000 7fc852000640 1 -- 172.21.15.5:0/2478547338 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc854108f60 msgr2=0x7fc85419ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:29.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:29.224+0000 7fc852000640 1 --2- 172.21.15.5:0/2478547338 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc854108f60 0x7fc85419ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:29.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:29.224+0000 7fc852000640 1 -- 172.21.15.5:0/2478547338 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc85419f3e0 con 0x7fc8541047e0 2024-09-10T09:59:29.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:29.224+0000 7fc852a00640 1 --2- 172.21.15.5:0/2478547338 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc854103e30 0x7fc85419a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:59:29.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:29.224+0000 7fc853400640 1 --2- 172.21.15.5:0/2478547338 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc854108f60 0x7fc85419ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:59:29.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:29.224+0000 7fc852000640 1 --2- 172.21.15.5:0/2478547338 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc8541047e0 0x7fc85419a970 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fc84000e9f0 tx=0x7fc84000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:29.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:29.224+0000 7fc858a00640 1 -- 172.21.15.5:0/2478547338 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc84000ccc0 con 0x7fc8541047e0 2024-09-10T09:59:29.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:29.224+0000 7fc853e00640 1 -- 172.21.15.5:0/2478547338 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc85419f6d0 con 0x7fc8541047e0 2024-09-10T09:59:29.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:29.224+0000 7fc858a00640 1 -- 172.21.15.5:0/2478547338 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc840005020 con 0x7fc8541047e0 2024-09-10T09:59:29.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:29.224+0000 7fc858a00640 1 -- 172.21.15.5:0/2478547338 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8400049e0 con 0x7fc8541047e0 2024-09-10T09:59:29.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:29.224+0000 7fc853e00640 1 -- 172.21.15.5:0/2478547338 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8541a6fb0 con 0x7fc8541047e0 2024-09-10T09:59:29.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:29.227+0000 7fc85c9ff740 1 Processor -- start 2024-09-10T09:59:29.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:29.228+0000 7fc85c9ff740 1 -- start start 2024-09-10T09:59:29.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:29.228+0000 7fc858a00640 1 -- 172.21.15.5:0/2478547338 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7fc840005560 con 0x7fc8541047e0 2024-09-10T09:59:29.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:29.228+0000 7fc85c9ff740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55636e0bd520 0x55636e0bd900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:29.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:29.228+0000 7fc85c9ff740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55636e0bde40 0x55636e0c81c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:29.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:29.228+0000 7fc85c9ff740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55636e0c8700 0x55636e0caaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:29.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:29.228+0000 7fc85c9ff740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55636e023650 con 0x55636e0c8700 2024-09-10T09:59:29.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:29.228+0000 7fc85c9ff740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55636dfcaf10 con 0x55636e0bde40 2024-09-10T09:59:29.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:29.228+0000 7fc85c9ff740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55636e01f610 con 0x55636e0bd520 2024-09-10T09:59:29.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:29.228+0000 7fc853400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55636e0bd520 0x55636e0bd900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:29.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:29.228+0000 7fc852a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55636e0bde40 0x55636e0c81c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:29.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:29.228+0000 7fc853400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55636e0bd520 0x55636e0bd900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:38710/0 (socket says 172.21.15.5:38710) 2024-09-10T09:59:29.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:29.228+0000 7fc853400640 1 -- 172.21.15.5:0/2370560918 learned_addr learned my addr 172.21.15.5:0/2370560918 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:59:29.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:29.228+0000 7fc852000640 1 --2- 172.21.15.5:0/2370560918 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55636e0c8700 0x55636e0caaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:29.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:29.229+0000 7fc853400640 1 -- 172.21.15.5:0/2370560918 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55636e0bde40 msgr2=0x55636e0c81c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:59:29.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:29.229+0000 7fc853400640 1 --2- 172.21.15.5:0/2370560918 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55636e0bde40 0x55636e0c81c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:29.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:29.229+0000 7fc853400640 1 -- 172.21.15.5:0/2370560918 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55636e0c8700 msgr2=0x55636e0caaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:59:29.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:29.229+0000 7fc853400640 1 --2- 172.21.15.5:0/2370560918 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55636e0c8700 0x55636e0caaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:29.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:29.229+0000 7fc853400640 1 -- 172.21.15.5:0/2370560918 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55636dd32580 con 0x55636e0bd520 2024-09-10T09:59:29.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:29.229+0000 7fc853400640 1 --2- 172.21.15.5:0/2370560918 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55636e0bd520 0x55636e0bd900 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fc84c005fd0 tx=0x7fc84c009880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d1ec4be699284bf2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:29.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:29.229+0000 7fc83fe00640 1 -- 172.21.15.5:0/2370560918 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc84c003bf0 con 0x55636e0bd520 2024-09-10T09:59:29.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:29.229+0000 7fc83fe00640 1 -- 172.21.15.5:0/2370560918 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc84c036030 con 0x55636e0bd520 2024-09-10T09:59:29.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:29.229+0000 7fc83fe00640 1 -- 172.21.15.5:0/2370560918 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc84c004740 con 0x55636e0bd520 2024-09-10T09:59:29.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:29.230+0000 7fc858a00640 1 --2- 172.21.15.5:0/2478547338 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fc8380a2b80 0x7fc8380a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:29.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:29.230+0000 7fc858a00640 1 -- 172.21.15.5:0/2478547338 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fc8400cf320 con 0x7fc8541047e0 2024-09-10T09:59:29.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:29.230+0000 7fc852a00640 1 --2- 172.21.15.5:0/2478547338 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fc8380a2b80 0x7fc8380a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:29.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:29.230+0000 7fc85c9ff740 1 -- 172.21.15.5:0/2370560918 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55636e0bd520 msgr2=0x55636e0bd900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:29.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:29.230+0000 7fc85c9ff740 1 --2- 172.21.15.5:0/2370560918 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55636e0bd520 0x55636e0bd900 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fc84c005fd0 tx=0x7fc84c009880 comp rx=0 tx=0).stop 2024-09-10T09:59:29.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:29.230+0000 7fc85c9ff740 1 -- 172.21.15.5:0/2370560918 shutdown_connections 2024-09-10T09:59:29.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:29.230+0000 7fc85c9ff740 1 --2- 172.21.15.5:0/2370560918 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55636e0c8700 0x55636e0caaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:29.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:29.230+0000 7fc85c9ff740 1 --2- 172.21.15.5:0/2370560918 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55636e0bde40 0x55636e0c81c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:29.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:29.230+0000 7fc85c9ff740 1 --2- 172.21.15.5:0/2370560918 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55636e0bd520 0x55636e0bd900 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:29.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:29.230+0000 7fc85c9ff740 1 -- 172.21.15.5:0/2370560918 >> 172.21.15.5:0/2370560918 conn(0x55636e0b9ab0 msgr2=0x55636e0b9e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:29.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:29.230+0000 7fc85c9ff740 1 -- 172.21.15.5:0/2370560918 shutdown_connections 2024-09-10T09:59:29.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:29.231+0000 7fc85c9ff740 1 -- 172.21.15.5:0/2370560918 wait complete. 2024-09-10T09:59:29.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:29.231+0000 7fc852a00640 1 --2- 172.21.15.5:0/2478547338 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fc8380a2b80 0x7fc8380a5040 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x55636e0ba910 tx=0x7fc844005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:29.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:29.231+0000 7fc85c9ff740 1 -- start start 2024-09-10T09:59:29.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:29.231+0000 7fc85c9ff740 1 Processor -- start 2024-09-10T09:59:29.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:29.232+0000 7fc85c9ff740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55636e0bd520 0x55636e0c5660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:29.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:29.232+0000 7fc85c9ff740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55636e0bde40 0x55636e0c5ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:29.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:29.232+0000 7fc85c9ff740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55636e0c8700 0x55636e0c60e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:29.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:29.232+0000 7fc85c9ff740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55636e0cb2d0 con 0x55636e0bd520 2024-09-10T09:59:29.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:29.232+0000 7fc85c9ff740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55636dfde350 con 0x55636e0bde40 2024-09-10T09:59:29.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:29.232+0000 7fc85c9ff740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55636e0cb450 con 0x55636e0c8700 2024-09-10T09:59:29.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:29.232+0000 7fc85c9ff740 10 client.0 _ll_register_callbacks cb 0x55636e001290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:59:29.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:29.232+0000 7fc852a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55636e0bd520 0x55636e0c5660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:29.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:29.232+0000 7fc852000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55636e0bde40 0x55636e0c5ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:29.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:29.232+0000 7fc852a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55636e0bd520 0x55636e0c5660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41398/0 (socket says 172.21.15.5:41398) 2024-09-10T09:59:29.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:29.232+0000 7fc852a00640 1 -- 172.21.15.5:0/3543067209 learned_addr learned my addr 172.21.15.5:0/3543067209 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:59:29.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:29.232+0000 7fc853400640 1 --2- 172.21.15.5:0/3543067209 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55636e0c8700 0x55636e0c60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:29.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:29.232+0000 7fc852000640 1 -- 172.21.15.5:0/3543067209 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55636e0c8700 msgr2=0x55636e0c60e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:59:29.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:29.232+0000 7fc852000640 1 --2- 172.21.15.5:0/3543067209 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55636e0c8700 0x55636e0c60e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:29.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:29.232+0000 7fc852000640 1 -- 172.21.15.5:0/3543067209 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55636e0bd520 msgr2=0x55636e0c5660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:59:29.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:29.232+0000 7fc852000640 1 --2- 172.21.15.5:0/3543067209 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55636e0bd520 0x55636e0c5660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:29.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:29.232+0000 7fc852000640 1 -- 172.21.15.5:0/3543067209 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55636ddfc960 con 0x55636e0bde40 2024-09-10T09:59:29.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:29.233+0000 7fc852000640 1 --2- 172.21.15.5:0/3543067209 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55636e0bde40 0x55636e0c5ba0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fc8400cca80 tx=0x7fc840009f60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1ed7bc404fb1cd6f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:29.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:29.233+0000 7fc83cc00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-09-10T09:59:29.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:29.233+0000 7fc83cc00640 1 -- 172.21.15.5:0/3543067209 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc84001c070 con 0x55636e0bde40 2024-09-10T09:59:29.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:29.233+0000 7fc83cc00640 1 -- 172.21.15.5:0/3543067209 <== mon.2 v2:172.21.15.102:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fc840004c60 con 0x55636e0bde40 2024-09-10T09:59:29.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:29.233+0000 7fc85c9ff740 1 -- 172.21.15.5:0/3543067209 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc84c036030 con 0x55636e0bde40 2024-09-10T09:59:29.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:29.233+0000 7fc83cc00640 1 -- 172.21.15.5:0/3543067209 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7fc84009b2d0 con 0x55636e0bde40 2024-09-10T09:59:29.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:29.233+0000 7fc85c9ff740 10 client.29540 fetch_fsmap learned FSMap version 42 2024-09-10T09:59:29.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:29.233+0000 7fc85c9ff740 1 -- 172.21.15.5:0/3543067209 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55636e0d7000 con 0x55636e0bde40 2024-09-10T09:59:29.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:29.233+0000 7fc83cc00640 1 -- 172.21.15.5:0/3543067209 <== mon.2 v2:172.21.15.102:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc84c036030 con 0x55636e0bde40 2024-09-10T09:59:29.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:29.234+0000 7fc83cc00640 1 -- 172.21.15.5:0/3543067209 <== mon.2 v2:172.21.15.102:3300/0 5 ==== fsmap(e 42) ==== 8370+0+0 (secure 0 0 0) 0x7fc8400a3f70 con 0x55636e0bde40 2024-09-10T09:59:29.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:29.234+0000 7fc85c9ff740 10 client.29540 fetch_fsmap finished waiting for FSMap version 42 2024-09-10T09:59:29.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:29.234+0000 7fc85c9ff740 10 client.29540 resolve_mds: validated gid 24542 aka daemon mds.b 2024-09-10T09:59:29.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:29.234+0000 7fc85c9ff740 20 client.29540 populate_metadata read hostname 'smithi005' 2024-09-10T09:59:29.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:29.234+0000 7fc85c9ff740 1 --2- 172.21.15.5:0/3543067209 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55636e0d84c0 0x55636e0da8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:29.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:29.234+0000 7fc85c9ff740 4 client.29540 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:59:29.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:29.234+0000 7fc85c9ff740 1 -- 172.21.15.5:0/3543067209 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55636e030bc0 con 0x55636e0d84c0 2024-09-10T09:59:29.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:29.234+0000 7fc852a00640 1 --2- 172.21.15.5:0/3543067209 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55636e0d84c0 0x55636e0da8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:29.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:29.235+0000 7fc852a00640 1 --2- 172.21.15.5:0/3543067209 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55636e0d84c0 0x55636e0da8b0 crc :-1 s=READY pgs=297 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b0654a9d5278ec28 server_cookie=99cb0e685d140d01 in_seq=0 out_seq=0 2024-09-10T09:59:29.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:29.235+0000 7fc83cc00640 10 client.29540 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T09:59:29.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:29.236+0000 7fc83cc00640 1 -- 172.21.15.5:0/3543067209 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55636e030bc0 con 0x55636e0d84c0 2024-09-10T09:59:29.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:29.236+0000 7fc83cc00640 10 client.29540 handle_command_reply: tid=0 2024-09-10T09:59:29.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:29.236+0000 7fc85c9ff740 1 client.29540 shutdown 2024-09-10T09:59:29.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:29.236+0000 7fc85c9ff740 1 -- 172.21.15.5:0/3543067209 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55636e0bde40 msgr2=0x55636e0c5ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:29.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:29.236+0000 7fc85c9ff740 1 --2- 172.21.15.5:0/3543067209 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55636e0bde40 0x55636e0c5ba0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fc8400cca80 tx=0x7fc840009f60 comp rx=0 tx=0).stop 2024-09-10T09:59:29.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:29.236+0000 7fc85c9ff740 1 -- 172.21.15.5:0/3543067209 shutdown_connections 2024-09-10T09:59:29.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:29.236+0000 7fc85c9ff740 1 --2- 172.21.15.5:0/3543067209 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55636e0d84c0 0x55636e0da8b0 crc :-1 s=READY pgs=297 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:29.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:29.236+0000 7fc85c9ff740 1 --2- 172.21.15.5:0/3543067209 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55636e0c8700 0x55636e0c60e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:29.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:29.236+0000 7fc85c9ff740 1 --2- 172.21.15.5:0/3543067209 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55636e0bde40 0x55636e0c5ba0 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:29.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:29.236+0000 7fc85c9ff740 1 --2- 172.21.15.5:0/3543067209 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55636e0bd520 0x55636e0c5660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:29.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:29.236+0000 7fc83cc00640 0 client.29540 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T09:59:29.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:29.236+0000 7fc85c9ff740 1 -- 172.21.15.5:0/3543067209 >> 172.21.15.5:0/3543067209 conn(0x55636e0b9ab0 msgr2=0x55636e0b8cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:29.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:29.236+0000 7fc85c9ff740 1 -- 172.21.15.5:0/3543067209 shutdown_connections 2024-09-10T09:59:29.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:29.236+0000 7fc85c9ff740 1 -- 172.21.15.5:0/3543067209 wait complete. 2024-09-10T09:59:29.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:29.236+0000 7fc85c9ff740 20 client.29540 trim_cache size 0 max 16384 2024-09-10T09:59:29.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:29.254+0000 7fc85c9ff740 1 Processor -- start 2024-09-10T09:59:29.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:29.254+0000 7fc85c9ff740 1 -- start start 2024-09-10T09:59:29.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:29.255+0000 7fc85c9ff740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55636e0bd520 0x55636e0e0060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:29.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:29.255+0000 7fc85c9ff740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55636e0bde40 0x55636e0e05a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:29.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:29.255+0000 7fc85c9ff740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55636e0c8700 0x55636e0c2920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:29.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:29.255+0000 7fc85c9ff740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc8541b0d10 con 0x55636e0c8700 2024-09-10T09:59:29.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:29.255+0000 7fc85c9ff740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fc8541b2850 con 0x55636e0bde40 2024-09-10T09:59:29.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:29.255+0000 7fc85c9ff740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fc8541b13e0 con 0x55636e0bd520 2024-09-10T09:59:29.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:29.255+0000 7fc853400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55636e0c8700 0x55636e0c2920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:29.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:29.255+0000 7fc853400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55636e0c8700 0x55636e0c2920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41408/0 (socket says 172.21.15.5:41408) 2024-09-10T09:59:29.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:29.255+0000 7fc853400640 1 -- 172.21.15.5:0/3609967905 learned_addr learned my addr 172.21.15.5:0/3609967905 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:59:29.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:29.255+0000 7fc852a00640 1 --2- 172.21.15.5:0/3609967905 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55636e0bd520 0x55636e0e0060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:29.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:29.255+0000 7fc852000640 1 --2- 172.21.15.5:0/3609967905 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55636e0bde40 0x55636e0e05a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:29.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:29.255+0000 7fc853400640 1 -- 172.21.15.5:0/3609967905 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55636e0bd520 msgr2=0x55636e0e0060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:59:29.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:29.255+0000 7fc853400640 1 --2- 172.21.15.5:0/3609967905 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55636e0bd520 0x55636e0e0060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:29.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:29.255+0000 7fc853400640 1 -- 172.21.15.5:0/3609967905 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55636e0bde40 msgr2=0x55636e0e05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:59:29.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:29.255+0000 7fc853400640 1 --2- 172.21.15.5:0/3609967905 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55636e0bde40 0x55636e0e05a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:29.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:29.255+0000 7fc853400640 1 -- 172.21.15.5:0/3609967905 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55636e0c4650 con 0x55636e0c8700 2024-09-10T09:59:29.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:29.256+0000 7fc853400640 1 --2- 172.21.15.5:0/3609967905 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55636e0c8700 0x55636e0c2920 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7fc84c045000 tx=0x7fc84c02a8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30bdc978b4f59ba0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:29.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:29.256+0000 7fc83f400640 1 -- 172.21.15.5:0/3609967905 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc84c030750 con 0x55636e0c8700 2024-09-10T09:59:29.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:29.256+0000 7fc83f400640 1 -- 172.21.15.5:0/3609967905 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc84c036040 con 0x55636e0c8700 2024-09-10T09:59:29.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:29.256+0000 7fc83f400640 1 -- 172.21.15.5:0/3609967905 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc84c031cd0 con 0x55636e0c8700 2024-09-10T09:59:29.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:29.257+0000 7fc85c9ff740 1 -- 172.21.15.5:0/3609967905 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55636e0c8700 msgr2=0x55636e0c2920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:29.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:29.257+0000 7fc85c9ff740 1 --2- 172.21.15.5:0/3609967905 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55636e0c8700 0x55636e0c2920 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7fc84c045000 tx=0x7fc84c02a8b0 comp rx=0 tx=0).stop 2024-09-10T09:59:29.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:29.257+0000 7fc85c9ff740 1 -- 172.21.15.5:0/3609967905 shutdown_connections 2024-09-10T09:59:29.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:29.257+0000 7fc85c9ff740 1 --2- 172.21.15.5:0/3609967905 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55636e0c8700 0x55636e0c2920 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:29.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:29.257+0000 7fc85c9ff740 1 --2- 172.21.15.5:0/3609967905 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55636e0bde40 0x55636e0e05a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:29.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:29.257+0000 7fc85c9ff740 1 --2- 172.21.15.5:0/3609967905 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55636e0bd520 0x55636e0e0060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:29.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:29.257+0000 7fc85c9ff740 1 -- 172.21.15.5:0/3609967905 >> 172.21.15.5:0/3609967905 conn(0x55636e0b9ab0 msgr2=0x55636e0b8cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:29.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:29.257+0000 7fc85c9ff740 1 -- 172.21.15.5:0/3609967905 shutdown_connections 2024-09-10T09:59:29.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:29.257+0000 7fc85c9ff740 1 -- 172.21.15.5:0/3609967905 wait complete. 2024-09-10T09:59:29.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:29.257+0000 7fc85c9ff740 1 -- start start 2024-09-10T09:59:29.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:29.257+0000 7fc85c9ff740 1 Processor -- start 2024-09-10T09:59:29.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:29.258+0000 7fc85c9ff740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55636e0bd520 0x55636e0d0b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:29.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:29.258+0000 7fc85c9ff740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55636e0bde40 0x55636e0cd670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:29.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:29.258+0000 7fc85c9ff740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55636e0c8700 0x55636e0cdbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:29.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:29.258+0000 7fc85c9ff740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc8541b2180 con 0x55636e0c8700 2024-09-10T09:59:29.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:29.258+0000 7fc85c9ff740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fc8541b1ab0 con 0x55636e0bd520 2024-09-10T09:59:29.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:29.258+0000 7fc85c9ff740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fc8541aecf0 con 0x55636e0bde40 2024-09-10T09:59:29.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:29.258+0000 7fc85c9ff740 10 client.0 _ll_register_callbacks cb 0x7fc8400053d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:59:29.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:29.258+0000 7fc853400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55636e0c8700 0x55636e0cdbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:29.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:29.258+0000 7fc852000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55636e0bde40 0x55636e0cd670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:29.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:29.258+0000 7fc852000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55636e0bde40 0x55636e0cd670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:38724/0 (socket says 172.21.15.5:38724) 2024-09-10T09:59:29.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:29.258+0000 7fc852000640 1 -- 172.21.15.5:0/1941304259 learned_addr learned my addr 172.21.15.5:0/1941304259 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:59:29.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:29.258+0000 7fc852a00640 1 --2- 172.21.15.5:0/1941304259 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55636e0bd520 0x55636e0d0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:29.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:29.259+0000 7fc852000640 1 -- 172.21.15.5:0/1941304259 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55636e0bd520 msgr2=0x55636e0d0b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:59:29.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:29.259+0000 7fc852000640 1 --2- 172.21.15.5:0/1941304259 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55636e0bd520 0x55636e0d0b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:29.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:29.259+0000 7fc852000640 1 -- 172.21.15.5:0/1941304259 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55636e0c8700 msgr2=0x55636e0cdbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:59:29.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:29.259+0000 7fc852000640 1 --2- 172.21.15.5:0/1941304259 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55636e0c8700 0x55636e0cdbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:29.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:29.259+0000 7fc852000640 1 -- 172.21.15.5:0/1941304259 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55636e0ce120 con 0x55636e0bde40 2024-09-10T09:59:29.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:29.259+0000 7fc853400640 1 --2- 172.21.15.5:0/1941304259 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55636e0c8700 0x55636e0cdbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:59:29.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:29.259+0000 7fc852a00640 1 --2- 172.21.15.5:0/1941304259 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55636e0bd520 0x55636e0d0b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:59:29.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:29.259+0000 7fc852000640 1 --2- 172.21.15.5:0/1941304259 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55636e0bde40 0x55636e0cd670 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fc840099a40 tx=0x7fc8400054d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5248667f55623a5b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:29.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:29.259+0000 7fc83cc00640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-09-10T09:59:29.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:29.259+0000 7fc83cc00640 1 -- 172.21.15.5:0/1941304259 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc84001c040 con 0x55636e0bde40 2024-09-10T09:59:29.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:29.259+0000 7fc85c9ff740 1 -- 172.21.15.5:0/1941304259 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc84c036040 con 0x55636e0bde40 2024-09-10T09:59:29.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:29.259+0000 7fc83cc00640 1 -- 172.21.15.5:0/1941304259 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fc84009adf0 con 0x55636e0bde40 2024-09-10T09:59:29.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:29.260+0000 7fc83cc00640 1 -- 172.21.15.5:0/1941304259 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7fc840004e60 con 0x55636e0bde40 2024-09-10T09:59:29.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:29.260+0000 7fc85c9ff740 10 client.29527 fetch_fsmap learned FSMap version 42 2024-09-10T09:59:29.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:29.260+0000 7fc85c9ff740 1 -- 172.21.15.5:0/1941304259 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55636e143dc0 con 0x55636e0bde40 2024-09-10T09:59:29.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:29.260+0000 7fc83cc00640 1 -- 172.21.15.5:0/1941304259 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc84c036040 con 0x55636e0bde40 2024-09-10T09:59:29.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:29.260+0000 7fc83cc00640 1 -- 172.21.15.5:0/1941304259 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 42) ==== 8370+0+0 (secure 0 0 0) 0x7fc84009cac0 con 0x55636e0bde40 2024-09-10T09:59:29.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:29.260+0000 7fc85c9ff740 10 client.29527 fetch_fsmap finished waiting for FSMap version 42 2024-09-10T09:59:29.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:29.260+0000 7fc85c9ff740 10 client.29527 resolve_mds: validated gid 24542 aka daemon mds.b 2024-09-10T09:59:29.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:29.260+0000 7fc85c9ff740 20 client.29527 populate_metadata read hostname 'smithi005' 2024-09-10T09:59:29.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:29.260+0000 7fc85c9ff740 1 --2- 172.21.15.5:0/1941304259 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55636e144310 0x55636e146730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:29.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:29.260+0000 7fc85c9ff740 4 client.29527 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "830978ed", "release": true, "await": true}] 2024-09-10T09:59:29.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:29.260+0000 7fc85c9ff740 1 -- 172.21.15.5:0/1941304259 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "quiesce db", "set_id": "830978ed", "release": true, "await": true}) -- 0x55636e02b0d0 con 0x55636e144310 2024-09-10T09:59:29.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:29.261+0000 7fc852a00640 1 --2- 172.21.15.5:0/1941304259 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55636e144310 0x55636e146730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:29.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:29.261+0000 7fc852a00640 1 --2- 172.21.15.5:0/1941304259 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55636e144310 0x55636e146730 crc :-1 s=READY pgs=298 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5355de548798de01 server_cookie=ce38c453a8e2dea8 in_seq=0 out_seq=0 2024-09-10T09:59:29.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:29.261+0000 7fc83cc00640 10 client.29527 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T09:59:29.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:29.314+0000 7fc83cc00640 1 -- 172.21.15.5:0/1941304259 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55636e02b0d0 con 0x55636e144310 2024-09-10T09:59:29.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:29.314+0000 7fc83cc00640 10 client.29527 handle_command_reply: tid=0 2024-09-10T09:59:29.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:29.314+0000 7fc85c9ff740 1 client.29527 shutdown 2024-09-10T09:59:29.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:29.314+0000 7fc85c9ff740 1 -- 172.21.15.5:0/1941304259 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55636e0bde40 msgr2=0x55636e0cd670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:29.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:29.314+0000 7fc85c9ff740 1 --2- 172.21.15.5:0/1941304259 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55636e0bde40 0x55636e0cd670 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fc840099a40 tx=0x7fc8400054d0 comp rx=0 tx=0).stop 2024-09-10T09:59:29.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:29.314+0000 7fc85c9ff740 1 -- 172.21.15.5:0/1941304259 shutdown_connections 2024-09-10T09:59:29.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:29.314+0000 7fc85c9ff740 1 --2- 172.21.15.5:0/1941304259 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55636e144310 0x55636e146730 crc :-1 s=READY pgs=298 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:29.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:29.314+0000 7fc85c9ff740 1 --2- 172.21.15.5:0/1941304259 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55636e0c8700 0x55636e0cdbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:29.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:29.314+0000 7fc85c9ff740 1 --2- 172.21.15.5:0/1941304259 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55636e0bde40 0x55636e0cd670 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:29.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:29.314+0000 7fc85c9ff740 1 --2- 172.21.15.5:0/1941304259 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55636e0bd520 0x55636e0d0b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:29.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:29.314+0000 7fc85c9ff740 1 -- 172.21.15.5:0/1941304259 >> 172.21.15.5:0/1941304259 conn(0x55636e0b9ab0 msgr2=0x55636e0bef70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:29.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:29.314+0000 7fc83cc00640 0 client.29527 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T09:59:29.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:29.315+0000 7fc85c9ff740 1 -- 172.21.15.5:0/1941304259 shutdown_connections 2024-09-10T09:59:29.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:29.315+0000 7fc85c9ff740 1 -- 172.21.15.5:0/1941304259 wait complete. 2024-09-10T09:59:29.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:29.315+0000 7fc85c9ff740 20 client.29527 trim_cache size 0 max 16384 2024-09-10T09:59:29.318 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-10T09:59:29.318 INFO:teuthology.orchestra.run.smithi005.stdout: "epoch": 42, 2024-09-10T09:59:29.318 INFO:teuthology.orchestra.run.smithi005.stdout: "leader": 24542, 2024-09-10T09:59:29.318 INFO:teuthology.orchestra.run.smithi005.stdout: "set_version": 76, 2024-09-10T09:59:29.318 INFO:teuthology.orchestra.run.smithi005.stdout: "sets": { 2024-09-10T09:59:29.318 INFO:teuthology.orchestra.run.smithi005.stdout: "830978ed": { 2024-09-10T09:59:29.318 INFO:teuthology.orchestra.run.smithi005.stdout: "version": 76, 2024-09-10T09:59:29.319 INFO:teuthology.orchestra.run.smithi005.stdout: "age_ref": 0.0, 2024-09-10T09:59:29.319 INFO:teuthology.orchestra.run.smithi005.stdout: "state": { 2024-09-10T09:59:29.319 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "RELEASED", 2024-09-10T09:59:29.319 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0 2024-09-10T09:59:29.319 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T09:59:29.319 INFO:teuthology.orchestra.run.smithi005.stdout: "timeout": 90.0, 2024-09-10T09:59:29.319 INFO:teuthology.orchestra.run.smithi005.stdout: "expiration": 153.2, 2024-09-10T09:59:29.319 INFO:teuthology.orchestra.run.smithi005.stdout: "members": { 2024-09-10T09:59:29.319 INFO:teuthology.orchestra.run.smithi005.stdout: "file:/": { 2024-09-10T09:59:29.319 INFO:teuthology.orchestra.run.smithi005.stdout: "excluded": false, 2024-09-10T09:59:29.319 INFO:teuthology.orchestra.run.smithi005.stdout: "state": { 2024-09-10T09:59:29.319 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "RELEASED", 2024-09-10T09:59:29.319 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0 2024-09-10T09:59:29.319 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T09:59:29.319 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T09:59:29.320 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T09:59:29.320 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T09:59:29.320 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T09:59:29.320 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-10T09:59:29.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:29.316+0000 7fc853e00640 1 -- 172.21.15.5:0/2478547338 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fc8380a2b80 msgr2=0x7fc8380a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:29.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:29.316+0000 7fc853e00640 1 --2- 172.21.15.5:0/2478547338 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fc8380a2b80 0x7fc8380a5040 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x55636e0ba910 tx=0x7fc844005f90 comp rx=0 tx=0).stop 2024-09-10T09:59:29.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:29.316+0000 7fc853e00640 1 -- 172.21.15.5:0/2478547338 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc8541047e0 msgr2=0x7fc85419a970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:29.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:29.316+0000 7fc853e00640 1 --2- 172.21.15.5:0/2478547338 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc8541047e0 0x7fc85419a970 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fc84000e9f0 tx=0x7fc84000eeb0 comp rx=0 tx=0).stop 2024-09-10T09:59:29.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:29.316+0000 7fc853e00640 1 -- 172.21.15.5:0/2478547338 shutdown_connections 2024-09-10T09:59:29.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:29.316+0000 7fc853e00640 1 --2- 172.21.15.5:0/2478547338 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fc8380a2b80 0x7fc8380a5040 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:29.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:29.316+0000 7fc853e00640 1 --2- 172.21.15.5:0/2478547338 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc854108f60 0x7fc85419ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:29.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:29.316+0000 7fc853e00640 1 --2- 172.21.15.5:0/2478547338 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc8541047e0 0x7fc85419a970 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:29.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:29.316+0000 7fc853e00640 1 --2- 172.21.15.5:0/2478547338 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc854103e30 0x7fc85419a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:29.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:29.316+0000 7fc853e00640 1 -- 172.21.15.5:0/2478547338 >> 172.21.15.5:0/2478547338 conn(0x7fc8540ff910 msgr2=0x7fc854100ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:29.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:29.316+0000 7fc853e00640 1 -- 172.21.15.5:0/2478547338 shutdown_connections 2024-09-10T09:59:29.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:29.316+0000 7fc853e00640 1 -- 172.21.15.5:0/2478547338 wait complete. 2024-09-10T09:59:29.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:59:28 smithi159 ceph-mon[35513]: pgmap v1027: 129 pgs: 129 active+clean; 654 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 831 KiB/s wr, 42 op/s 2024-09-10T09:59:29.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:59:28 smithi102 ceph-mon[35442]: pgmap v1027: 129 pgs: 129 active+clean; 654 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 831 KiB/s wr, 42 op/s 2024-09-10T09:59:30.027 INFO:tasks.workunit.client.0.smithi005.stdout: 10 111147 12.61 MB/sec execute 308 sec latency 32144.702 ms 2024-09-10T09:59:30.256 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:59:29 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:59:30.256 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:59:29 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:59:30.256 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:59:29 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:59:30.256 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:59:29 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:59:30.257 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:59:29 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:59:30.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:59:29 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:59:30.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:59:29 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:59:30.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:59:29 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:59:30.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:59:29 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:59:30.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:59:29 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:59:30.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:59:29 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T09:59:30.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:59:29 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:59:30.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:59:29 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:59:30.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:59:29 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:59:30.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:59:29 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:59:30.474 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '830978ed', total seconds elapsed: 37.6 2024-09-10T09:59:31.027 INFO:tasks.workunit.client.0.smithi005.stdout: 10 111750 12.68 MB/sec execute 309 sec latency 138.154 ms 2024-09-10T09:59:31.286 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:59:30 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:59:31.286 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:59:30 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:59:31.286 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:59:30 smithi102 ceph-mon[35442]: pgmap v1028: 129 pgs: 129 active+clean; 654 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 247 KiB/s wr, 39 op/s 2024-09-10T09:59:31.286 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:59:30 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:59:31.286 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:59:30 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:59:31.286 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:59:30 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:59:31.286 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:59:30 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:59:31.306 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:59:30 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:59:31.306 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:59:30 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:59:31.306 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:59:30 smithi005 ceph-mon[30350]: pgmap v1028: 129 pgs: 129 active+clean; 654 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 247 KiB/s wr, 39 op/s 2024-09-10T09:59:31.306 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:59:30 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:59:31.307 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:59:30 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:59:31.307 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:59:30 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:59:31.307 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:59:30 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:59:31.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:59:30 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:59:31.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:59:30 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:59:31.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:59:30 smithi159 ceph-mon[35513]: pgmap v1028: 129 pgs: 129 active+clean; 654 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 247 KiB/s wr, 39 op/s 2024-09-10T09:59:31.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:59:30 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:59:31.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:59:30 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:59:31.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:59:30 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:59:31.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:59:30 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:59:32.027 INFO:tasks.workunit.client.0.smithi005.stdout: 10 112366 12.76 MB/sec execute 310 sec latency 43.734 ms 2024-09-10T09:59:32.204 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:59:31 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:59:32.204 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:59:31 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:59:32.204 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:59:31 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:59:32.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:59:31 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:59:32.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:59:31 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:59:32.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:59:31 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:59:32.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:59:31 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T09:59:32.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:59:31 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T09:59:32.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:59:31 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T09:59:33.027 INFO:tasks.workunit.client.0.smithi005.stdout: 10 113003 12.81 MB/sec execute 311 sec latency 74.479 ms 2024-09-10T09:59:33.307 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:59:32 smithi005 ceph-mon[30350]: scrub summary: idle 2024-09-10T09:59:33.307 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:59:32 smithi005 ceph-mon[30350]: scrub summary: idle 2024-09-10T09:59:33.307 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:59:32 smithi005 ceph-mon[30350]: scrub summary: idle 2024-09-10T09:59:33.307 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:59:32 smithi005 ceph-mon[30350]: scrub summary: idle 2024-09-10T09:59:33.307 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:59:32 smithi005 ceph-mon[30350]: scrub summary: idle 2024-09-10T09:59:33.307 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:59:32 smithi005 ceph-mon[30350]: pgmap v1029: 129 pgs: 129 active+clean; 655 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 836 KiB/s rd, 194 KiB/s wr, 42 op/s 2024-09-10T09:59:33.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:59:32 smithi159 ceph-mon[35513]: scrub summary: idle 2024-09-10T09:59:33.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:59:32 smithi159 ceph-mon[35513]: scrub summary: idle 2024-09-10T09:59:33.351 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:59:32 smithi159 ceph-mon[35513]: scrub summary: idle 2024-09-10T09:59:33.351 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:59:32 smithi159 ceph-mon[35513]: scrub summary: idle 2024-09-10T09:59:33.351 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:59:32 smithi159 ceph-mon[35513]: scrub summary: idle 2024-09-10T09:59:33.351 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:59:32 smithi159 ceph-mon[35513]: pgmap v1029: 129 pgs: 129 active+clean; 655 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 836 KiB/s rd, 194 KiB/s wr, 42 op/s 2024-09-10T09:59:33.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:59:33 smithi102 ceph-mon[35442]: scrub summary: idle 2024-09-10T09:59:33.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:59:33 smithi102 ceph-mon[35442]: scrub summary: idle 2024-09-10T09:59:33.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:59:33 smithi102 ceph-mon[35442]: scrub summary: idle 2024-09-10T09:59:33.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:59:33 smithi102 ceph-mon[35442]: scrub summary: idle 2024-09-10T09:59:33.385 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:59:33 smithi102 ceph-mon[35442]: scrub summary: idle 2024-09-10T09:59:33.385 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:59:33 smithi102 ceph-mon[35442]: pgmap v1029: 129 pgs: 129 active+clean; 655 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 836 KiB/s rd, 194 KiB/s wr, 42 op/s 2024-09-10T09:59:34.027 INFO:tasks.workunit.client.0.smithi005.stdout: 10 113520 12.86 MB/sec execute 312 sec latency 69.133 ms 2024-09-10T09:59:35.027 INFO:tasks.workunit.client.0.smithi005.stdout: 10 114305 12.96 MB/sec execute 313 sec latency 50.292 ms 2024-09-10T09:59:35.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:59:35 smithi102 ceph-mon[35442]: pgmap v1030: 129 pgs: 129 active+clean; 657 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 760 KiB/s wr, 107 op/s 2024-09-10T09:59:35.536 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:59:35 smithi005 ceph-mon[30350]: pgmap v1030: 129 pgs: 129 active+clean; 657 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 760 KiB/s wr, 107 op/s 2024-09-10T09:59:35.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:59:35 smithi159 ceph-mon[35513]: pgmap v1030: 129 pgs: 129 active+clean; 657 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 760 KiB/s wr, 107 op/s 2024-09-10T09:59:36.027 INFO:tasks.workunit.client.0.smithi005.stdout: 10 114902 13.02 MB/sec execute 314 sec latency 58.726 ms 2024-09-10T09:59:37.028 INFO:tasks.workunit.client.0.smithi005.stdout: 10 115559 13.08 MB/sec execute 315 sec latency 56.159 ms 2024-09-10T09:59:37.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:59:37 smithi102 ceph-mon[35442]: pgmap v1031: 129 pgs: 129 active+clean; 659 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 1.4 MiB/s wr, 213 op/s 2024-09-10T09:59:37.400 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:59:37 smithi005 ceph-mon[30350]: pgmap v1031: 129 pgs: 129 active+clean; 659 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 1.4 MiB/s wr, 213 op/s 2024-09-10T09:59:37.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:59:37 smithi159 ceph-mon[35513]: pgmap v1031: 129 pgs: 129 active+clean; 659 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 1.4 MiB/s wr, 213 op/s 2024-09-10T09:59:38.028 INFO:tasks.workunit.client.0.smithi005.stdout: 10 116267 13.16 MB/sec execute 316 sec latency 57.875 ms 2024-09-10T09:59:39.028 INFO:tasks.workunit.client.0.smithi005.stdout: 10 116915 13.24 MB/sec execute 317 sec latency 76.361 ms 2024-09-10T09:59:39.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:59:39 smithi102 ceph-mon[35442]: pgmap v1032: 129 pgs: 129 active+clean; 658 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 1.7 MiB/s wr, 257 op/s 2024-09-10T09:59:39.557 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:59:39 smithi005 ceph-mon[30350]: pgmap v1032: 129 pgs: 129 active+clean; 658 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 1.7 MiB/s wr, 257 op/s 2024-09-10T09:59:39.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:59:39 smithi159 ceph-mon[35513]: pgmap v1032: 129 pgs: 129 active+clean; 658 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 1.7 MiB/s wr, 257 op/s 2024-09-10T09:59:40.028 INFO:tasks.workunit.client.0.smithi005.stdout: 10 117508 13.30 MB/sec execute 318 sec latency 66.443 ms 2024-09-10T09:59:41.028 INFO:tasks.workunit.client.0.smithi005.stdout: 10 118182 13.37 MB/sec execute 319 sec latency 60.014 ms 2024-09-10T09:59:41.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:59:41 smithi102 ceph-mon[35442]: pgmap v1033: 129 pgs: 129 active+clean; 671 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 3.0 MiB/s wr, 418 op/s 2024-09-10T09:59:41.469 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:59:41 smithi005 ceph-mon[30350]: pgmap v1033: 129 pgs: 129 active+clean; 671 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 3.0 MiB/s wr, 418 op/s 2024-09-10T09:59:41.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:59:41 smithi159 ceph-mon[35513]: pgmap v1033: 129 pgs: 129 active+clean; 671 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 3.0 MiB/s wr, 418 op/s 2024-09-10T09:59:42.028 INFO:tasks.workunit.client.0.smithi005.stdout: 10 118993 13.49 MB/sec execute 320 sec latency 48.488 ms 2024-09-10T09:59:43.028 INFO:tasks.workunit.client.0.smithi005.stdout: 10 119326 13.49 MB/sec execute 321 sec latency 357.974 ms 2024-09-10T09:59:43.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:59:43 smithi102 ceph-mon[35442]: pgmap v1034: 129 pgs: 129 active+clean; 674 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 3.3 MiB/s wr, 445 op/s 2024-09-10T09:59:43.495 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:59:43 smithi005 ceph-mon[30350]: pgmap v1034: 129 pgs: 129 active+clean; 674 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 3.3 MiB/s wr, 445 op/s 2024-09-10T09:59:43.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:59:43 smithi159 ceph-mon[35513]: pgmap v1034: 129 pgs: 129 active+clean; 674 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 3.3 MiB/s wr, 445 op/s 2024-09-10T09:59:44.028 INFO:tasks.workunit.client.0.smithi005.stdout: 10 119992 13.54 MB/sec execute 322 sec latency 58.529 ms 2024-09-10T09:59:45.028 INFO:tasks.workunit.client.0.smithi005.stdout: 10 120664 13.60 MB/sec execute 323 sec latency 78.782 ms 2024-09-10T09:59:45.378 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:59:45 smithi005 ceph-mon[30350]: pgmap v1035: 129 pgs: 129 active+clean; 675 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 3.4 MiB/s wr, 462 op/s 2024-09-10T09:59:45.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:59:45 smithi102 ceph-mon[35442]: pgmap v1035: 129 pgs: 129 active+clean; 675 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 3.4 MiB/s wr, 462 op/s 2024-09-10T09:59:45.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:59:45 smithi159 ceph-mon[35513]: pgmap v1035: 129 pgs: 129 active+clean; 675 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 3.4 MiB/s wr, 462 op/s 2024-09-10T09:59:46.028 INFO:tasks.workunit.client.0.smithi005.stdout: 10 121426 13.70 MB/sec execute 324 sec latency 59.710 ms 2024-09-10T09:59:47.028 INFO:tasks.workunit.client.0.smithi005.stdout: 10 122005 13.76 MB/sec execute 325 sec latency 182.114 ms 2024-09-10T09:59:47.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:59:47 smithi102 ceph-mon[35442]: pgmap v1036: 129 pgs: 129 active+clean; 684 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 3.8 MiB/s wr, 567 op/s 2024-09-10T09:59:47.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:59:47 smithi005 ceph-mon[30350]: pgmap v1036: 129 pgs: 129 active+clean; 684 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 3.8 MiB/s wr, 567 op/s 2024-09-10T09:59:47.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:59:47 smithi159 ceph-mon[35513]: pgmap v1036: 129 pgs: 129 active+clean; 684 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 3.8 MiB/s wr, 567 op/s 2024-09-10T09:59:48.028 INFO:tasks.workunit.client.0.smithi005.stdout: 10 122903 13.85 MB/sec execute 326 sec latency 60.921 ms 2024-09-10T09:59:49.028 INFO:tasks.workunit.client.0.smithi005.stdout: 10 123441 13.93 MB/sec execute 327 sec latency 62.834 ms 2024-09-10T09:59:49.212 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:59:49 smithi005 ceph-mon[30350]: pgmap v1037: 129 pgs: 129 active+clean; 690 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 3.7 MiB/s wr, 584 op/s 2024-09-10T09:59:49.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:59:49 smithi102 ceph-mon[35442]: pgmap v1037: 129 pgs: 129 active+clean; 690 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 3.7 MiB/s wr, 584 op/s 2024-09-10T09:59:49.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:59:49 smithi159 ceph-mon[35513]: pgmap v1037: 129 pgs: 129 active+clean; 690 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 3.7 MiB/s wr, 584 op/s 2024-09-10T09:59:50.029 INFO:tasks.workunit.client.0.smithi005.stdout: 10 124047 13.99 MB/sec execute 328 sec latency 104.187 ms 2024-09-10T09:59:51.028 INFO:tasks.workunit.client.0.smithi005.stdout: 10 124456 13.99 MB/sec execute 329 sec latency 131.035 ms 2024-09-10T09:59:51.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:59:51 smithi102 ceph-mon[35442]: pgmap v1038: 129 pgs: 129 active+clean; 697 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 4.1 MiB/s wr, 635 op/s 2024-09-10T09:59:51.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:59:51 smithi005 ceph-mon[30350]: pgmap v1038: 129 pgs: 129 active+clean; 697 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 4.1 MiB/s wr, 635 op/s 2024-09-10T09:59:51.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:59:51 smithi159 ceph-mon[35513]: pgmap v1038: 129 pgs: 129 active+clean; 697 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 4.1 MiB/s wr, 635 op/s 2024-09-10T09:59:52.029 INFO:tasks.workunit.client.0.smithi005.stdout: 10 125265 14.06 MB/sec execute 330 sec latency 57.265 ms 2024-09-10T09:59:53.029 INFO:tasks.workunit.client.0.smithi005.stdout: 10 125975 14.15 MB/sec execute 331 sec latency 253.567 ms 2024-09-10T09:59:53.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:59:53 smithi102 ceph-mon[35442]: pgmap v1039: 129 pgs: 129 active+clean; 698 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 3.4 MiB/s wr, 526 op/s 2024-09-10T09:59:53.401 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:59:53 smithi159 ceph-mon[35513]: pgmap v1039: 129 pgs: 129 active+clean; 698 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 3.4 MiB/s wr, 526 op/s 2024-09-10T09:59:53.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:59:53 smithi005 ceph-mon[30350]: pgmap v1039: 129 pgs: 129 active+clean; 698 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 3.4 MiB/s wr, 526 op/s 2024-09-10T09:59:54.029 INFO:tasks.workunit.client.0.smithi005.stdout: 10 126728 14.24 MB/sec execute 332 sec latency 156.831 ms 2024-09-10T09:59:54.528 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T09:59:55.029 INFO:tasks.workunit.client.0.smithi005.stdout: 10 127498 14.31 MB/sec execute 333 sec latency 125.299 ms 2024-09-10T09:59:55.057 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:59:55.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:59:55 smithi102 ceph-mon[35442]: pgmap v1040: 129 pgs: 129 active+clean; 697 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 3.3 MiB/s wr, 528 op/s 2024-09-10T09:59:55.413 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:59:55 smithi005 ceph-mon[30350]: pgmap v1040: 129 pgs: 129 active+clean; 697 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 3.3 MiB/s wr, 528 op/s 2024-09-10T09:59:55.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:59:55 smithi159 ceph-mon[35513]: pgmap v1040: 129 pgs: 129 active+clean; 697 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 3.3 MiB/s wr, 528 op/s 2024-09-10T09:59:55.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:55.777+0000 7fa32fc00640 1 -- 172.21.15.5:0/1255032576 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa32810c590 msgr2=0x7fa32810c9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:55.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:55.778+0000 7fa32fc00640 1 --2- 172.21.15.5:0/1255032576 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa32810c590 0x7fa32810c9f0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7fa31c009930 tx=0x7fa31c02a440 comp rx=0 tx=0).stop 2024-09-10T09:59:55.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:55.779+0000 7fa32fc00640 1 -- 172.21.15.5:0/1255032576 shutdown_connections 2024-09-10T09:59:55.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:55.779+0000 7fa32fc00640 1 --2- 172.21.15.5:0/1255032576 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa32810cf30 0x7fa328119410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:55.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:55.779+0000 7fa32fc00640 1 --2- 172.21.15.5:0/1255032576 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa32810c590 0x7fa32810c9f0 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:55.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:55.779+0000 7fa32fc00640 1 --2- 172.21.15.5:0/1255032576 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa328112640 0x7fa328112a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:55.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:55.779+0000 7fa32fc00640 1 -- 172.21.15.5:0/1255032576 >> 172.21.15.5:0/1255032576 conn(0x7fa32806cdc0 msgr2=0x7fa32806d1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:55.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:55.781+0000 7fa32fc00640 1 -- 172.21.15.5:0/1255032576 shutdown_connections 2024-09-10T09:59:55.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:55.781+0000 7fa32fc00640 1 -- 172.21.15.5:0/1255032576 wait complete. 2024-09-10T09:59:55.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:55.783+0000 7fa32fc00640 1 Processor -- start 2024-09-10T09:59:55.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:55.783+0000 7fa32fc00640 1 -- start start 2024-09-10T09:59:55.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:55.783+0000 7fa32fc00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa32810c590 0x7fa32806db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:55.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:55.783+0000 7fa32fc00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa32810cf30 0x7fa32806e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:55.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:55.783+0000 7fa32fc00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa328112640 0x7fa32806e600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:55.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:55.783+0000 7fa32fc00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa32811c9b0 con 0x7fa32810c590 2024-09-10T09:59:55.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:55.784+0000 7fa32fc00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fa32811c830 con 0x7fa328112640 2024-09-10T09:59:55.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:55.784+0000 7fa32fc00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fa32811cb30 con 0x7fa32810cf30 2024-09-10T09:59:55.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:55.784+0000 7fa32d200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa32810c590 0x7fa32806db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:55.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:55.784+0000 7fa32dc00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa328112640 0x7fa32806e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:55.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:55.784+0000 7fa327e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa32810cf30 0x7fa32806e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:55.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:55.784+0000 7fa327e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa32810cf30 0x7fa32806e0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:45032/0 (socket says 172.21.15.5:45032) 2024-09-10T09:59:55.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:55.784+0000 7fa327e00640 1 -- 172.21.15.5:0/2611282711 learned_addr learned my addr 172.21.15.5:0/2611282711 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:59:55.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:55.784+0000 7fa32dc00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa328112640 0x7fa32806e600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:37170/0 (socket says 172.21.15.5:37170) 2024-09-10T09:59:55.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:55.785+0000 7fa32dc00640 1 -- 172.21.15.5:0/2611282711 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa32810cf30 msgr2=0x7fa32806e0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:55.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:55.785+0000 7fa32dc00640 1 --2- 172.21.15.5:0/2611282711 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa32810cf30 0x7fa32806e0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:55.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:55.785+0000 7fa32dc00640 1 -- 172.21.15.5:0/2611282711 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa32810c590 msgr2=0x7fa32806db80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:55.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:55.785+0000 7fa32dc00640 1 --2- 172.21.15.5:0/2611282711 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa32810c590 0x7fa32806db80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:55.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:55.785+0000 7fa32dc00640 1 -- 172.21.15.5:0/2611282711 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3281a2bd0 con 0x7fa328112640 2024-09-10T09:59:55.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:55.785+0000 7fa32dc00640 1 --2- 172.21.15.5:0/2611282711 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa328112640 0x7fa32806e600 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7fa32000ea80 tx=0x7fa32000ef40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:55.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:55.786+0000 7fa325600640 1 -- 172.21.15.5:0/2611282711 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa32000cea0 con 0x7fa328112640 2024-09-10T09:59:55.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:55.786+0000 7fa325600640 1 -- 172.21.15.5:0/2611282711 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa320005020 con 0x7fa328112640 2024-09-10T09:59:55.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:55.786+0000 7fa32fc00640 1 -- 172.21.15.5:0/2611282711 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3281a2ec0 con 0x7fa328112640 2024-09-10T09:59:55.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:55.786+0000 7fa325600640 1 -- 172.21.15.5:0/2611282711 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa320018710 con 0x7fa328112640 2024-09-10T09:59:55.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:55.786+0000 7fa32fc00640 1 -- 172.21.15.5:0/2611282711 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3281a3400 con 0x7fa328112640 2024-09-10T09:59:55.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:55.796+0000 7fa325600640 1 -- 172.21.15.5:0/2611282711 <== mon.2 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7fa3200051d0 con 0x7fa328112640 2024-09-10T09:59:55.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:55.800+0000 7fa33266d740 1 Processor -- start 2024-09-10T09:59:55.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:55.801+0000 7fa33266d740 1 -- start start 2024-09-10T09:59:55.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:55.801+0000 7fa33266d740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56014fd35b90 0x56014fd35f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:55.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:55.801+0000 7fa33266d740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56014fd364b0 0x56014fd40830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:55.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:55.801+0000 7fa33266d740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56014fd40d70 0x56014fd43160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:55.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:55.801+0000 7fa33266d740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x56014fc9c3e0 con 0x56014fd35b90 2024-09-10T09:59:55.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:55.801+0000 7fa33266d740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x56014fc43d80 con 0x56014fd364b0 2024-09-10T09:59:55.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:55.801+0000 7fa33266d740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x56014fc983a0 con 0x56014fd40d70 2024-09-10T09:59:55.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:55.801+0000 7fa325600640 1 --2- 172.21.15.5:0/2611282711 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fa3100a2c80 0x7fa3100a5140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:55.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:55.802+0000 7fa325600640 1 -- 172.21.15.5:0/2611282711 <== mon.2 v2:172.21.15.102:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fa3200cfa70 con 0x7fa328112640 2024-09-10T09:59:55.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:55.803+0000 7fa32d200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56014fd364b0 0x56014fd40830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:55.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:55.803+0000 7fa32dc00640 1 --2- 172.21.15.5:0/2611282711 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fa3100a2c80 0x7fa3100a5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:55.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:55.803+0000 7fa32d200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56014fd364b0 0x56014fd40830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:37186/0 (socket says 172.21.15.5:37186) 2024-09-10T09:59:55.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:55.803+0000 7fa32d200640 1 -- 172.21.15.5:0/515071579 learned_addr learned my addr 172.21.15.5:0/515071579 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:59:55.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:55.804+0000 7fa32dc00640 1 --2- 172.21.15.5:0/2611282711 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fa3100a2c80 0x7fa3100a5140 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fa32000c9a0 tx=0x7fa32000c910 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:55.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:55.804+0000 7fa327e00640 1 --2- 172.21.15.5:0/515071579 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56014fd40d70 0x56014fd43160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:55.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:55.805+0000 7fa32d200640 1 -- 172.21.15.5:0/515071579 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56014fd40d70 msgr2=0x56014fd43160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:59:55.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:55.805+0000 7fa32d200640 1 --2- 172.21.15.5:0/515071579 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56014fd40d70 0x56014fd43160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:55.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:55.805+0000 7fa32d200640 1 -- 172.21.15.5:0/515071579 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56014fd35b90 msgr2=0x56014fd35f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:59:55.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:55.805+0000 7fa32d200640 1 --2- 172.21.15.5:0/515071579 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56014fd35b90 0x56014fd35f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:55.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:55.805+0000 7fa327e00640 1 --2- 172.21.15.5:0/515071579 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56014fd40d70 0x56014fd43160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T09:59:55.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:55.805+0000 7fa32d200640 1 -- 172.21.15.5:0/515071579 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56014fc62ae0 con 0x56014fd364b0 2024-09-10T09:59:55.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:55.807+0000 7fa32d200640 1 --2- 172.21.15.5:0/515071579 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56014fd364b0 0x56014fd40830 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fa31400ac20 tx=0x7fa31400f790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=33e30bbbaea30f38 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:55.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:55.807+0000 7fa31a000640 1 -- 172.21.15.5:0/515071579 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa31401d070 con 0x56014fd364b0 2024-09-10T09:59:55.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:55.807+0000 7fa31a000640 1 -- 172.21.15.5:0/515071579 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa314018b20 con 0x56014fd364b0 2024-09-10T09:59:55.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:55.807+0000 7fa31a000640 1 -- 172.21.15.5:0/515071579 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa314010bc0 con 0x56014fd364b0 2024-09-10T09:59:55.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:55.808+0000 7fa33266d740 1 -- 172.21.15.5:0/515071579 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56014fd364b0 msgr2=0x56014fd40830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:55.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:55.808+0000 7fa33266d740 1 --2- 172.21.15.5:0/515071579 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56014fd364b0 0x56014fd40830 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fa31400ac20 tx=0x7fa31400f790 comp rx=0 tx=0).stop 2024-09-10T09:59:55.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:55.809+0000 7fa33266d740 1 -- 172.21.15.5:0/515071579 shutdown_connections 2024-09-10T09:59:55.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:55.809+0000 7fa33266d740 1 --2- 172.21.15.5:0/515071579 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56014fd40d70 0x56014fd43160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:55.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:55.809+0000 7fa33266d740 1 --2- 172.21.15.5:0/515071579 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56014fd364b0 0x56014fd40830 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:55.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:55.809+0000 7fa33266d740 1 --2- 172.21.15.5:0/515071579 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56014fd35b90 0x56014fd35f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:55.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:55.809+0000 7fa33266d740 1 -- 172.21.15.5:0/515071579 >> 172.21.15.5:0/515071579 conn(0x56014fd32120 msgr2=0x56014fd32500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:55.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:55.809+0000 7fa33266d740 1 -- 172.21.15.5:0/515071579 shutdown_connections 2024-09-10T09:59:55.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:55.811+0000 7fa33266d740 1 -- 172.21.15.5:0/515071579 wait complete. 2024-09-10T09:59:55.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:55.811+0000 7fa33266d740 1 -- start start 2024-09-10T09:59:55.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:55.812+0000 7fa33266d740 1 Processor -- start 2024-09-10T09:59:55.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:55.813+0000 7fa33266d740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56014fd35b90 0x56014fd3ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:55.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:55.813+0000 7fa33266d740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56014fd364b0 0x56014fd3e310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:55.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:55.813+0000 7fa327e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56014fd35b90 0x56014fd3ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:55.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:55.813+0000 7fa327e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56014fd35b90 0x56014fd3ddd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:45042/0 (socket says 172.21.15.5:45042) 2024-09-10T09:59:55.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:55.814+0000 7fa33266d740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56014fd40d70 0x56014fd3e850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:55.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:55.814+0000 7fa33266d740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x56014fd439e0 con 0x56014fd364b0 2024-09-10T09:59:55.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:55.814+0000 7fa33266d740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x56014fc570f0 con 0x56014fd40d70 2024-09-10T09:59:55.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:55.814+0000 7fa33266d740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x56014fd43b60 con 0x56014fd35b90 2024-09-10T09:59:55.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:55.814+0000 7fa33266d740 10 client.0 _ll_register_callbacks cb 0x56014fc489e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:59:55.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:55.814+0000 7fa32d200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56014fd364b0 0x56014fd3e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:55.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:55.815+0000 7fa32d200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56014fd364b0 0x56014fd3e310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59796/0 (socket says 172.21.15.5:59796) 2024-09-10T09:59:55.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:55.815+0000 7fa327e00640 1 -- 172.21.15.5:0/617143405 learned_addr learned my addr 172.21.15.5:0/617143405 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:59:55.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:55.816+0000 7fa327e00640 1 --2- 172.21.15.5:0/617143405 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56014fd40d70 0x56014fd3e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:55.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:55.816+0000 7fa32d200640 1 -- 172.21.15.5:0/617143405 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56014fd35b90 msgr2=0x56014fd3ddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:59:55.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:55.816+0000 7fa32d200640 1 --2- 172.21.15.5:0/617143405 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56014fd35b90 0x56014fd3ddd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:55.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:55.816+0000 7fa32d200640 1 -- 172.21.15.5:0/617143405 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56014fd40d70 msgr2=0x56014fd3e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:59:55.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:55.817+0000 7fa32d200640 1 --2- 172.21.15.5:0/617143405 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56014fd40d70 0x56014fd3e850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:55.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:55.817+0000 7fa32d200640 1 -- 172.21.15.5:0/617143405 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56014fa0d3c0 con 0x56014fd364b0 2024-09-10T09:59:55.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:55.817+0000 7fa327e00640 1 --2- 172.21.15.5:0/617143405 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56014fd35b90 0x56014fd3ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T09:59:55.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:55.818+0000 7fa32d200640 1 --2- 172.21.15.5:0/617143405 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56014fd364b0 0x56014fd3e310 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7fa314010700 tx=0x7fa314018670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51272226fbcb2e1d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:55.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:55.818+0000 7fa30ea00640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-10T09:59:55.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:55.818+0000 7fa30ea00640 1 -- 172.21.15.5:0/617143405 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa31401d050 con 0x56014fd364b0 2024-09-10T09:59:55.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:55.819+0000 7fa30ea00640 1 -- 172.21.15.5:0/617143405 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fa314010900 con 0x56014fd364b0 2024-09-10T09:59:55.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:55.819+0000 7fa33266d740 1 -- 172.21.15.5:0/617143405 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa314018b20 con 0x56014fd364b0 2024-09-10T09:59:55.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:55.819+0000 7fa30ea00640 1 -- 172.21.15.5:0/617143405 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7fa314031050 con 0x56014fd364b0 2024-09-10T09:59:55.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:55.819+0000 7fa33266d740 10 client.19488 fetch_fsmap learned FSMap version 42 2024-09-10T09:59:55.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:55.819+0000 7fa33266d740 1 -- 172.21.15.5:0/617143405 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x56014fd4f760 con 0x56014fd364b0 2024-09-10T09:59:55.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:55.820+0000 7fa30ea00640 1 -- 172.21.15.5:0/617143405 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa31401e040 con 0x56014fd364b0 2024-09-10T09:59:55.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:55.820+0000 7fa30ea00640 1 -- 172.21.15.5:0/617143405 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 42) ==== 8370+0+0 (secure 0 0 0) 0x7fa314005480 con 0x56014fd364b0 2024-09-10T09:59:55.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:55.821+0000 7fa33266d740 10 client.19488 fetch_fsmap finished waiting for FSMap version 42 2024-09-10T09:59:55.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:55.821+0000 7fa33266d740 10 client.19488 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T09:59:55.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:55.821+0000 7fa33266d740 20 client.19488 populate_metadata read hostname 'smithi005' 2024-09-10T09:59:55.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:55.821+0000 7fa33266d740 1 --2- 172.21.15.5:0/617143405 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x56014fd50b10 0x56014fd52f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:55.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:55.822+0000 7fa33266d740 4 client.19488 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T09:59:55.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:55.822+0000 7fa33266d740 1 -- 172.21.15.5:0/617143405 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fa314018b20 con 0x56014fd50b10 2024-09-10T09:59:55.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:55.822+0000 7fa327e00640 1 --2- 172.21.15.5:0/617143405 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x56014fd50b10 0x56014fd52f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:55.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:55.822+0000 7fa327e00640 1 --2- 172.21.15.5:0/617143405 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x56014fd50b10 0x56014fd52f00 crc :-1 s=READY pgs=299 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4ef4b8828d2974cf server_cookie=191fe3ea14d8fb10 in_seq=0 out_seq=0 2024-09-10T09:59:55.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:55.823+0000 7fa30ea00640 10 client.19488 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T09:59:55.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:55.823+0000 7fa30ea00640 1 -- 172.21.15.5:0/617143405 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7fa314018b20 con 0x56014fd50b10 2024-09-10T09:59:55.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:55.823+0000 7fa30ea00640 10 client.19488 handle_command_reply: tid=0 2024-09-10T09:59:55.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:55.824+0000 7fa33266d740 1 client.19488 shutdown 2024-09-10T09:59:55.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:55.824+0000 7fa33266d740 1 -- 172.21.15.5:0/617143405 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56014fd364b0 msgr2=0x56014fd3e310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:55.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:55.824+0000 7fa33266d740 1 --2- 172.21.15.5:0/617143405 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56014fd364b0 0x56014fd3e310 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7fa314010700 tx=0x7fa314018670 comp rx=0 tx=0).stop 2024-09-10T09:59:55.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:55.824+0000 7fa33266d740 1 -- 172.21.15.5:0/617143405 shutdown_connections 2024-09-10T09:59:55.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:55.824+0000 7fa33266d740 1 --2- 172.21.15.5:0/617143405 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x56014fd50b10 0x56014fd52f00 crc :-1 s=READY pgs=299 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:55.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:55.824+0000 7fa33266d740 1 --2- 172.21.15.5:0/617143405 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56014fd40d70 0x56014fd3e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:55.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:55.824+0000 7fa33266d740 1 --2- 172.21.15.5:0/617143405 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56014fd364b0 0x56014fd3e310 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:55.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:55.824+0000 7fa33266d740 1 --2- 172.21.15.5:0/617143405 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56014fd35b90 0x56014fd3ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:55.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:55.824+0000 7fa33266d740 1 -- 172.21.15.5:0/617143405 >> 172.21.15.5:0/617143405 conn(0x56014fd32120 msgr2=0x56014fd37e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:55.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:55.824+0000 7fa30ea00640 0 client.19488 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T09:59:55.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:55.825+0000 7fa33266d740 1 -- 172.21.15.5:0/617143405 shutdown_connections 2024-09-10T09:59:55.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:55.825+0000 7fa33266d740 1 -- 172.21.15.5:0/617143405 wait complete. 2024-09-10T09:59:55.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:55.825+0000 7fa33266d740 20 client.19488 trim_cache size 0 max 16384 2024-09-10T09:59:55.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:55.845+0000 7fa33266d740 1 Processor -- start 2024-09-10T09:59:55.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:55.845+0000 7fa33266d740 1 -- start start 2024-09-10T09:59:55.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:55.845+0000 7fa33266d740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56014fd35b90 0x56014fd4bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:55.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:55.845+0000 7fa33266d740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56014fd40d70 0x56014fd3e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:55.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:55.845+0000 7fa33266d740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56014fd55f20 0x56014fd563d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:55.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:55.845+0000 7fa33266d740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa3281c6530 con 0x56014fd40d70 2024-09-10T09:59:55.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:55.845+0000 7fa33266d740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fa3281c8070 con 0x56014fd35b90 2024-09-10T09:59:55.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:55.845+0000 7fa33266d740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fa3281c6c00 con 0x56014fd55f20 2024-09-10T09:59:55.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:55.845+0000 7fa327e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56014fd35b90 0x56014fd4bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:55.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:55.846+0000 7fa327e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56014fd35b90 0x56014fd4bdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:37206/0 (socket says 172.21.15.5:37206) 2024-09-10T09:59:55.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:55.846+0000 7fa327e00640 1 -- 172.21.15.5:0/3907371539 learned_addr learned my addr 172.21.15.5:0/3907371539 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:59:55.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:55.846+0000 7fa327e00640 1 --2- 172.21.15.5:0/3907371539 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56014fd55f20 0x56014fd563d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:55.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:55.846+0000 7fa32d200640 1 --2- 172.21.15.5:0/3907371539 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56014fd40d70 0x56014fd3e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:55.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:55.846+0000 7fa327e00640 1 -- 172.21.15.5:0/3907371539 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56014fd55f20 msgr2=0x56014fd563d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:59:55.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:55.846+0000 7fa327e00640 1 --2- 172.21.15.5:0/3907371539 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56014fd55f20 0x56014fd563d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:55.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:55.846+0000 7fa327e00640 1 -- 172.21.15.5:0/3907371539 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56014fd40d70 msgr2=0x56014fd3e6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:59:55.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:55.846+0000 7fa327e00640 1 --2- 172.21.15.5:0/3907371539 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56014fd40d70 0x56014fd3e6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:55.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:55.846+0000 7fa327e00640 1 -- 172.21.15.5:0/3907371539 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56014fd35910 con 0x56014fd35b90 2024-09-10T09:59:55.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:55.846+0000 7fa32d200640 1 --2- 172.21.15.5:0/3907371539 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56014fd40d70 0x56014fd3e6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T09:59:55.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:55.847+0000 7fa327e00640 1 --2- 172.21.15.5:0/3907371539 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56014fd35b90 0x56014fd4bdb0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7fa31c031550 tx=0x7fa31c009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c506d9787546e0ec server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:55.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:55.847+0000 7fa319600640 1 -- 172.21.15.5:0/3907371539 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa31c047070 con 0x56014fd35b90 2024-09-10T09:59:55.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:55.847+0000 7fa319600640 1 -- 172.21.15.5:0/3907371539 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa31c005410 con 0x56014fd35b90 2024-09-10T09:59:55.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:55.847+0000 7fa319600640 1 -- 172.21.15.5:0/3907371539 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa31c035040 con 0x56014fd35b90 2024-09-10T09:59:55.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:55.848+0000 7fa33266d740 1 -- 172.21.15.5:0/3907371539 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56014fd35b90 msgr2=0x56014fd4bdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:55.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:55.848+0000 7fa33266d740 1 --2- 172.21.15.5:0/3907371539 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56014fd35b90 0x56014fd4bdb0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7fa31c031550 tx=0x7fa31c009260 comp rx=0 tx=0).stop 2024-09-10T09:59:55.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:55.848+0000 7fa33266d740 1 -- 172.21.15.5:0/3907371539 shutdown_connections 2024-09-10T09:59:55.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:55.848+0000 7fa33266d740 1 --2- 172.21.15.5:0/3907371539 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56014fd55f20 0x56014fd563d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:55.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:55.848+0000 7fa33266d740 1 --2- 172.21.15.5:0/3907371539 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56014fd40d70 0x56014fd3e6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:55.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:55.848+0000 7fa33266d740 1 --2- 172.21.15.5:0/3907371539 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56014fd35b90 0x56014fd4bdb0 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:55.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:55.848+0000 7fa33266d740 1 -- 172.21.15.5:0/3907371539 >> 172.21.15.5:0/3907371539 conn(0x56014fd32120 msgr2=0x56014fd3fe50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:55.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:55.848+0000 7fa33266d740 1 -- 172.21.15.5:0/3907371539 shutdown_connections 2024-09-10T09:59:55.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:55.848+0000 7fa33266d740 1 -- 172.21.15.5:0/3907371539 wait complete. 2024-09-10T09:59:55.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:55.848+0000 7fa33266d740 1 -- start start 2024-09-10T09:59:55.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:55.849+0000 7fa33266d740 1 Processor -- start 2024-09-10T09:59:55.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:55.849+0000 7fa33266d740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56014fd35b90 0x56014fd53aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:55.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:55.849+0000 7fa33266d740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56014fd40d70 0x56014fd53fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:55.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:55.849+0000 7fa33266d740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56014fd55f20 0x56014fd54ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:55.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:55.849+0000 7fa33266d740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa3281c79a0 con 0x56014fd35b90 2024-09-10T09:59:55.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:55.849+0000 7fa33266d740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fa3281c72d0 con 0x56014fd40d70 2024-09-10T09:59:55.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:55.849+0000 7fa33266d740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fa3281c4510 con 0x56014fd55f20 2024-09-10T09:59:55.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:55.849+0000 7fa33266d740 10 client.0 _ll_register_callbacks cb 0x7fa314010430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T09:59:55.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:55.849+0000 7fa327e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56014fd35b90 0x56014fd53aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:55.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:55.850+0000 7fa32d200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56014fd40d70 0x56014fd53fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:55.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:55.850+0000 7fa327e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56014fd35b90 0x56014fd53aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59812/0 (socket says 172.21.15.5:59812) 2024-09-10T09:59:55.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:55.850+0000 7fa327e00640 1 -- 172.21.15.5:0/3472384998 learned_addr learned my addr 172.21.15.5:0/3472384998 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:59:55.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:55.850+0000 7fa327e00640 1 --2- 172.21.15.5:0/3472384998 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56014fd55f20 0x56014fd54ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:55.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:55.850+0000 7fa327e00640 1 -- 172.21.15.5:0/3472384998 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56014fd55f20 msgr2=0x56014fd54ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:59:55.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:55.850+0000 7fa327e00640 1 --2- 172.21.15.5:0/3472384998 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56014fd55f20 0x56014fd54ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:55.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:55.850+0000 7fa327e00640 1 -- 172.21.15.5:0/3472384998 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56014fd40d70 msgr2=0x56014fd53fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T09:59:55.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:55.850+0000 7fa327e00640 1 --2- 172.21.15.5:0/3472384998 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56014fd40d70 0x56014fd53fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:55.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:55.850+0000 7fa327e00640 1 -- 172.21.15.5:0/3472384998 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56014fd55720 con 0x56014fd35b90 2024-09-10T09:59:55.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:55.851+0000 7fa327e00640 1 --2- 172.21.15.5:0/3472384998 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56014fd35b90 0x56014fd53aa0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7fa31c02aa00 tx=0x7fa31c0315b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40a3d5b305b7a7a5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:55.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:55.851+0000 7fa30ea00640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-10T09:59:55.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:55.851+0000 7fa30ea00640 1 -- 172.21.15.5:0/3472384998 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa31c035070 con 0x56014fd35b90 2024-09-10T09:59:55.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:55.851+0000 7fa30ea00640 1 -- 172.21.15.5:0/3472384998 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fa31c004bb0 con 0x56014fd35b90 2024-09-10T09:59:55.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:55.851+0000 7fa33266d740 1 -- 172.21.15.5:0/3472384998 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa31c005410 con 0x56014fd35b90 2024-09-10T09:59:55.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:55.851+0000 7fa30ea00640 1 -- 172.21.15.5:0/3472384998 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7fa31c047030 con 0x56014fd35b90 2024-09-10T09:59:55.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:55.852+0000 7fa33266d740 10 client.19494 fetch_fsmap learned FSMap version 42 2024-09-10T09:59:55.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:55.852+0000 7fa33266d740 1 -- 172.21.15.5:0/3472384998 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x56014fdbc040 con 0x56014fd35b90 2024-09-10T09:59:55.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:55.852+0000 7fa30ea00640 1 -- 172.21.15.5:0/3472384998 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa31c005410 con 0x56014fd35b90 2024-09-10T09:59:55.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:55.852+0000 7fa30ea00640 1 -- 172.21.15.5:0/3472384998 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 42) ==== 8370+0+0 (secure 0 0 0) 0x7fa314019ce0 con 0x56014fd35b90 2024-09-10T09:59:55.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:55.853+0000 7fa33266d740 10 client.19494 fetch_fsmap finished waiting for FSMap version 42 2024-09-10T09:59:55.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:55.853+0000 7fa33266d740 10 client.19494 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T09:59:55.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:55.853+0000 7fa33266d740 20 client.19494 populate_metadata read hostname 'smithi005' 2024-09-10T09:59:55.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:55.853+0000 7fa33266d740 1 --2- 172.21.15.5:0/3472384998 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x56014fdbc6b0 0x56014fdbead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:55.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:55.853+0000 7fa33266d740 4 client.19494 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T09:59:55.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:55.853+0000 7fa33266d740 1 -- 172.21.15.5:0/3472384998 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56014fca9950 con 0x56014fdbc6b0 2024-09-10T09:59:55.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:55.854+0000 7fa32d200640 1 --2- 172.21.15.5:0/3472384998 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x56014fdbc6b0 0x56014fdbead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:55.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:55.858+0000 7fa32d200640 1 --2- 172.21.15.5:0/3472384998 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x56014fdbc6b0 0x56014fdbead0 crc :-1 s=READY pgs=300 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ddc04de28174deec server_cookie=68c73838bb5cdd8d in_seq=0 out_seq=0 2024-09-10T09:59:55.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:55.858+0000 7fa30ea00640 10 client.19494 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T09:59:55.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:55.858+0000 7fa30ea00640 1 -- 172.21.15.5:0/3472384998 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x56014fca9950 con 0x56014fdbc6b0 2024-09-10T09:59:55.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:55.858+0000 7fa30ea00640 10 client.19494 handle_command_reply: tid=0 2024-09-10T09:59:55.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:55.859+0000 7fa33266d740 1 client.19494 shutdown 2024-09-10T09:59:55.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:55.860+0000 7fa33266d740 1 -- 172.21.15.5:0/3472384998 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56014fd35b90 msgr2=0x56014fd53aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:55.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:55.860+0000 7fa33266d740 1 --2- 172.21.15.5:0/3472384998 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56014fd35b90 0x56014fd53aa0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7fa31c02aa00 tx=0x7fa31c0315b0 comp rx=0 tx=0).stop 2024-09-10T09:59:55.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:55.862+0000 7fa33266d740 1 -- 172.21.15.5:0/3472384998 shutdown_connections 2024-09-10T09:59:55.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:55.862+0000 7fa33266d740 1 --2- 172.21.15.5:0/3472384998 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x56014fdbc6b0 0x56014fdbead0 crc :-1 s=READY pgs=300 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:55.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:55.862+0000 7fa33266d740 1 --2- 172.21.15.5:0/3472384998 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56014fd55f20 0x56014fd54ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:55.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:55.862+0000 7fa33266d740 1 --2- 172.21.15.5:0/3472384998 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56014fd40d70 0x56014fd53fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:55.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:55.862+0000 7fa33266d740 1 --2- 172.21.15.5:0/3472384998 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56014fd35b90 0x56014fd53aa0 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:55.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:55.862+0000 7fa33266d740 1 -- 172.21.15.5:0/3472384998 >> 172.21.15.5:0/3472384998 conn(0x56014fd32120 msgr2=0x56014fd3fe50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:55.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:55.862+0000 7fa30ea00640 0 client.19494 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T09:59:55.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:55.863+0000 7fa33266d740 1 -- 172.21.15.5:0/3472384998 shutdown_connections 2024-09-10T09:59:55.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:55.863+0000 7fa33266d740 1 -- 172.21.15.5:0/3472384998 wait complete. 2024-09-10T09:59:55.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:55.863+0000 7fa33266d740 20 client.19494 trim_cache size 0 max 16384 2024-09-10T09:59:55.865 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-10T09:59:55.865 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "no active scrubs running", 2024-09-10T09:59:55.865 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": {} 2024-09-10T09:59:55.865 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-10T09:59:55.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:55.864+0000 7fa32fc00640 1 -- 172.21.15.5:0/2611282711 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fa3100a2c80 msgr2=0x7fa3100a5140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:55.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:55.864+0000 7fa32fc00640 1 --2- 172.21.15.5:0/2611282711 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fa3100a2c80 0x7fa3100a5140 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fa32000c9a0 tx=0x7fa32000c910 comp rx=0 tx=0).stop 2024-09-10T09:59:55.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:55.864+0000 7fa32fc00640 1 -- 172.21.15.5:0/2611282711 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa328112640 msgr2=0x7fa32806e600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:55.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:55.864+0000 7fa32fc00640 1 --2- 172.21.15.5:0/2611282711 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa328112640 0x7fa32806e600 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7fa32000ea80 tx=0x7fa32000ef40 comp rx=0 tx=0).stop 2024-09-10T09:59:55.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:55.865+0000 7fa32fc00640 1 -- 172.21.15.5:0/2611282711 shutdown_connections 2024-09-10T09:59:55.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:55.865+0000 7fa32fc00640 1 --2- 172.21.15.5:0/2611282711 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fa3100a2c80 0x7fa3100a5140 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:55.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:55.865+0000 7fa32fc00640 1 --2- 172.21.15.5:0/2611282711 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa328112640 0x7fa32806e600 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:55.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:55.865+0000 7fa32fc00640 1 --2- 172.21.15.5:0/2611282711 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa32810cf30 0x7fa32806e0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:55.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:55.865+0000 7fa32fc00640 1 --2- 172.21.15.5:0/2611282711 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa32810c590 0x7fa32806db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:55.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:55.865+0000 7fa32fc00640 1 -- 172.21.15.5:0/2611282711 >> 172.21.15.5:0/2611282711 conn(0x7fa32806cdc0 msgr2=0x7fa32810aeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:55.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:55.865+0000 7fa32fc00640 1 -- 172.21.15.5:0/2611282711 shutdown_connections 2024-09-10T09:59:55.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:55.865+0000 7fa32fc00640 1 -- 172.21.15.5:0/2611282711 wait complete. 2024-09-10T09:59:56.029 INFO:tasks.workunit.client.0.smithi005.stdout: 10 127912 14.35 MB/sec execute 334 sec latency 172.267 ms 2024-09-10T09:59:56.797 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-10T09:59:56.798 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T09:59:57.029 INFO:tasks.workunit.client.0.smithi005.stdout: 10 128448 14.40 MB/sec execute 335 sec latency 67.735 ms 2024-09-10T09:59:57.381 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T09:59:57.427 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:59:57 smithi005 ceph-mon[30350]: pgmap v1041: 129 pgs: 129 active+clean; 687 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 4.2 MiB/s wr, 632 op/s 2024-09-10T09:59:57.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:59:57 smithi159 ceph-mon[35513]: pgmap v1041: 129 pgs: 129 active+clean; 687 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 4.2 MiB/s wr, 632 op/s 2024-09-10T09:59:57.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:59:57 smithi102 ceph-mon[35442]: pgmap v1041: 129 pgs: 129 active+clean; 687 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 4.2 MiB/s wr, 632 op/s 2024-09-10T09:59:58.030 INFO:tasks.workunit.client.0.smithi005.stdout: 10 129011 14.43 MB/sec execute 336 sec latency 81.961 ms 2024-09-10T09:59:58.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:58.325+0000 7f5894c00640 1 -- 172.21.15.5:0/341061121 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5890109450 msgr2=0x7f5890110250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:58.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:58.325+0000 7f5894c00640 1 --2- 172.21.15.5:0/341061121 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5890109450 0x7f5890110250 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f5880009900 tx=0x7f588002a3f0 comp rx=0 tx=0).stop 2024-09-10T09:59:58.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:58.326+0000 7f5894c00640 1 -- 172.21.15.5:0/341061121 shutdown_connections 2024-09-10T09:59:58.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:58.326+0000 7f5894c00640 1 --2- 172.21.15.5:0/341061121 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f589011b140 0x7f589011d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:58.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:58.326+0000 7f5894c00640 1 --2- 172.21.15.5:0/341061121 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5890109830 0x7f5890109c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:58.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:58.326+0000 7f5894c00640 1 --2- 172.21.15.5:0/341061121 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5890109450 0x7f5890110250 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:58.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:58.326+0000 7f5894c00640 1 -- 172.21.15.5:0/341061121 >> 172.21.15.5:0/341061121 conn(0x7f589006e140 msgr2=0x7f589006e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:58.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:58.326+0000 7f5894c00640 1 -- 172.21.15.5:0/341061121 shutdown_connections 2024-09-10T09:59:58.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:58.327+0000 7f5894c00640 1 -- 172.21.15.5:0/341061121 wait complete. 2024-09-10T09:59:58.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:58.328+0000 7f5894c00640 1 Processor -- start 2024-09-10T09:59:58.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:58.328+0000 7f5894c00640 1 -- start start 2024-09-10T09:59:58.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:58.329+0000 7f5894c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5890109450 0x7f58901a7210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:58.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:58.329+0000 7f5894c00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5890109830 0x7f58901a7750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:58.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:58.329+0000 7f5894c00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f589011b140 0x7f58901abae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:58.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:58.329+0000 7f5894c00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f5890120df0 con 0x7f5890109830 2024-09-10T09:59:58.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:58.329+0000 7f5894c00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f5890120c70 con 0x7f5890109450 2024-09-10T09:59:58.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:58.329+0000 7f5894c00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f5890120f70 con 0x7f589011b140 2024-09-10T09:59:58.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:58.330+0000 7f588e800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f589011b140 0x7f58901abae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:58.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:58.330+0000 7f588e800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f589011b140 0x7f58901abae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:45058/0 (socket says 172.21.15.5:45058) 2024-09-10T09:59:58.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:58.330+0000 7f588e800640 1 -- 172.21.15.5:0/903996808 learned_addr learned my addr 172.21.15.5:0/903996808 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T09:59:58.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:58.330+0000 7f588de00640 1 --2- 172.21.15.5:0/903996808 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5890109450 0x7f58901a7210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:58.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:58.331+0000 7f588e800640 1 -- 172.21.15.5:0/903996808 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5890109450 msgr2=0x7f58901a7210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:58.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:58.331+0000 7f588e800640 1 --2- 172.21.15.5:0/903996808 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5890109450 0x7f58901a7210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:58.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:58.331+0000 7f588e800640 1 -- 172.21.15.5:0/903996808 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5890109830 msgr2=0x7f58901a7750 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T09:59:58.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:58.331+0000 7f588d400640 1 --2- 172.21.15.5:0/903996808 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5890109830 0x7f58901a7750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:58.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:58.331+0000 7f588e800640 1 --2- 172.21.15.5:0/903996808 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5890109830 0x7f58901a7750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:58.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:58.331+0000 7f588e800640 1 -- 172.21.15.5:0/903996808 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58901ac1c0 con 0x7f589011b140 2024-09-10T09:59:58.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:58.332+0000 7f588e800640 1 --2- 172.21.15.5:0/903996808 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f589011b140 0x7f58901abae0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f588800ebc0 tx=0x7f588800c2d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:58.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:58.333+0000 7f5886a00640 1 -- 172.21.15.5:0/903996808 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5888004b20 con 0x7f589011b140 2024-09-10T09:59:58.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:58.333+0000 7f5886a00640 1 -- 172.21.15.5:0/903996808 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5888004cc0 con 0x7f589011b140 2024-09-10T09:59:58.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:58.333+0000 7f5886a00640 1 -- 172.21.15.5:0/903996808 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58880107a0 con 0x7f589011b140 2024-09-10T09:59:58.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:58.333+0000 7f5894c00640 1 -- 172.21.15.5:0/903996808 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58901ac4b0 con 0x7f589011b140 2024-09-10T09:59:58.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:58.333+0000 7f5894c00640 1 -- 172.21.15.5:0/903996808 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58901b3d90 con 0x7f589011b140 2024-09-10T09:59:58.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:58.336+0000 7f5894c00640 1 -- 172.21.15.5:0/903996808 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f58901a8570 con 0x7f589011b140 2024-09-10T09:59:58.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:58.339+0000 7f5886a00640 1 -- 172.21.15.5:0/903996808 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f5888005020 con 0x7f589011b140 2024-09-10T09:59:58.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:58.341+0000 7f5886a00640 1 --2- 172.21.15.5:0/903996808 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f58740a2d10 0x7f58740a51d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T09:59:58.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:58.342+0000 7f5886a00640 1 -- 172.21.15.5:0/903996808 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f58880cfb70 con 0x7f589011b140 2024-09-10T09:59:58.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:58.342+0000 7f588de00640 1 --2- 172.21.15.5:0/903996808 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f58740a2d10 0x7f58740a51d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T09:59:58.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:58.344+0000 7f588de00640 1 --2- 172.21.15.5:0/903996808 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f58740a2d10 0x7f58740a51d0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f5880000c00 tx=0x7f58800040c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T09:59:58.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:58.349+0000 7f5886a00640 1 -- 172.21.15.5:0/903996808 <== mon.1 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f588809c6e0 con 0x7f589011b140 2024-09-10T09:59:58.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:58.668+0000 7f5894c00640 1 -- 172.21.15.5:0/903996808 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f589002d7c0 con 0x7f589011b140 2024-09-10T09:59:58.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:58.669+0000 7f5886a00640 1 -- 172.21.15.5:0/903996808 <== mon.1 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 42 v42) ==== 94+0+11969 (secure 0 0 0) 0x7f588809bfe0 con 0x7f589011b140 2024-09-10T09:59:58.670 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-10T09:59:58.670 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":42,"btime":"2024-09-10T09:55:06:670916+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14454,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/2970594311","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2970594311},{"type":"v1","addr":"172.21.15.5:6835","nonce":2970594311}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14514,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6839/3616502174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":3616502174},{"type":"v1","addr":"172.21.15.5:6839","nonce":3616502174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":42,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T09:35:29.626287+0000","modified":"2024-09-10T09:55:06.419829+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24542,"mds_1":24535,"mds_2":24511,"mds_3":24506,"mds_4":24443},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14520":{"gid":14520,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/2073599842","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":2073599842},{"type":"v1","addr":"172.21.15.5:6841","nonce":2073599842}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"k","rank":4,"incarnation":26,"state":"up:active","state_seq":24,"addr":"172.21.15.102:6833/2457007427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":2457007427},{"type":"v1","addr":"172.21.15.102:6833","nonce":2457007427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/356445159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":356445159},{"type":"v1","addr":"172.21.15.159:6833","nonce":356445159}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6837/2894314507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":2894314507},{"type":"v1","addr":"172.21.15.5:6837","nonce":2894314507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.102:6835/1647549111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":1647549111},{"type":"v1","addr":"172.21.15.102:6835","nonce":1647549111}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6835/952657242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":952657242},{"type":"v1","addr":"172.21.15.159:6835","nonce":952657242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":19,"addr":"172.21.15.102:6837/2689776764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":2689776764},{"type":"v1","addr":"172.21.15.102:6837","nonce":2689776764}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.159:6837/620627639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":620627639},{"type":"v1","addr":"172.21.15.159:6837","nonce":620627639}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24535":{"gid":24535,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":18,"addr":"172.21.15.159:6839/2073739599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":2073739599},{"type":"v1","addr":"172.21.15.159:6839","nonce":2073739599}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24542":{"gid":24542,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.102:6839/335409749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6838","nonce":335409749},{"type":"v1","addr":"172.21.15.102:6839","nonce":335409749}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24542,"qdb_cluster":[24542,24535,24511,24506,24443]},"id":1}]} 2024-09-10T09:59:58.670 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 42 2024-09-10T09:59:58.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:58.671+0000 7f5894c00640 1 -- 172.21.15.5:0/903996808 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f58740a2d10 msgr2=0x7f58740a51d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:58.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:58.671+0000 7f5894c00640 1 --2- 172.21.15.5:0/903996808 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f58740a2d10 0x7f58740a51d0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f5880000c00 tx=0x7f58800040c0 comp rx=0 tx=0).stop 2024-09-10T09:59:58.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:58.671+0000 7f5894c00640 1 -- 172.21.15.5:0/903996808 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f589011b140 msgr2=0x7f58901abae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T09:59:58.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:58.671+0000 7f5894c00640 1 --2- 172.21.15.5:0/903996808 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f589011b140 0x7f58901abae0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f588800ebc0 tx=0x7f588800c2d0 comp rx=0 tx=0).stop 2024-09-10T09:59:58.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:58.671+0000 7f5894c00640 1 -- 172.21.15.5:0/903996808 shutdown_connections 2024-09-10T09:59:58.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:58.671+0000 7f5894c00640 1 --2- 172.21.15.5:0/903996808 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f58740a2d10 0x7f58740a51d0 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:58.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:58.671+0000 7f5894c00640 1 --2- 172.21.15.5:0/903996808 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f589011b140 0x7f58901abae0 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:58.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:58.671+0000 7f5894c00640 1 --2- 172.21.15.5:0/903996808 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5890109830 0x7f58901a7750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:58.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:58.671+0000 7f5894c00640 1 --2- 172.21.15.5:0/903996808 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5890109450 0x7f58901a7210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T09:59:58.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:58.671+0000 7f5894c00640 1 -- 172.21.15.5:0/903996808 >> 172.21.15.5:0/903996808 conn(0x7f589006e140 msgr2=0x7f589011c1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T09:59:58.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:58.672+0000 7f5894c00640 1 -- 172.21.15.5:0/903996808 shutdown_connections 2024-09-10T09:59:58.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T09:59:58.672+0000 7f5894c00640 1 -- 172.21.15.5:0/903996808 wait complete. 2024-09-10T09:59:59.029 INFO:tasks.workunit.client.0.smithi005.stdout: 10 129634 14.48 MB/sec execute 337 sec latency 124.313 ms 2024-09-10T09:59:59.307 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:59:59 smithi005 ceph-mon[30350]: pgmap v1042: 129 pgs: 129 active+clean; 688 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 3.5 MiB/s wr, 485 op/s 2024-09-10T09:59:59.307 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 09:59:59 smithi005 ceph-mon[30350]: from='client.? 172.21.15.5:0/903996808' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:59:59.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:59:59 smithi159 ceph-mon[35513]: pgmap v1042: 129 pgs: 129 active+clean; 688 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 3.5 MiB/s wr, 485 op/s 2024-09-10T09:59:59.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 09:59:59 smithi159 ceph-mon[35513]: from='client.? 172.21.15.5:0/903996808' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:59:59.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:59:59 smithi102 ceph-mon[35442]: pgmap v1042: 129 pgs: 129 active+clean; 688 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 3.5 MiB/s wr, 485 op/s 2024-09-10T09:59:59.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 09:59:59 smithi102 ceph-mon[35442]: from='client.? 172.21.15.5:0/903996808' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T09:59:59.840 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-10T10:00:00.029 INFO:tasks.workunit.client.0.smithi005.stdout: 10 130296 14.54 MB/sec execute 338 sec latency 91.235 ms 2024-09-10T10:00:00.402 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:00:00 smithi005 ceph-mon[30350]: overall HEALTH_OK 2024-09-10T10:00:00.442 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:00:00.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:00:00 smithi159 ceph-mon[35513]: overall HEALTH_OK 2024-09-10T10:00:00.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:00:00 smithi102 ceph-mon[35442]: overall HEALTH_OK 2024-09-10T10:00:01.029 INFO:tasks.workunit.client.0.smithi005.stdout: 10 130485 14.53 MB/sec execute 339 sec latency 625.047 ms 2024-09-10T10:00:01.205 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:00:01 smithi005 ceph-mon[30350]: pgmap v1043: 129 pgs: 129 active+clean; 701 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 4.2 MiB/s wr, 545 op/s 2024-09-10T10:00:01.535 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:00:01 smithi102 ceph-mon[35442]: pgmap v1043: 129 pgs: 129 active+clean; 701 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 4.2 MiB/s wr, 545 op/s 2024-09-10T10:00:01.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:00:01 smithi159 ceph-mon[35513]: pgmap v1043: 129 pgs: 129 active+clean; 701 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 4.2 MiB/s wr, 545 op/s 2024-09-10T10:00:02.029 INFO:tasks.workunit.client.0.smithi005.stdout: 10 130495 14.49 MB/sec execute 340 sec latency 1625.102 ms 2024-09-10T10:00:03.029 INFO:tasks.workunit.client.0.smithi005.stdout: 10 130495 14.44 MB/sec execute 341 sec latency 2625.155 ms 2024-09-10T10:00:03.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:00:03 smithi005 ceph-mon[30350]: pgmap v1044: 129 pgs: 129 active+clean; 706 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 4.5 MiB/s wr, 478 op/s 2024-09-10T10:00:03.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:00:03 smithi159 ceph-mon[35513]: pgmap v1044: 129 pgs: 129 active+clean; 706 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 4.5 MiB/s wr, 478 op/s 2024-09-10T10:00:03.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:00:03 smithi102 ceph-mon[35442]: pgmap v1044: 129 pgs: 129 active+clean; 706 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 4.5 MiB/s wr, 478 op/s 2024-09-10T10:00:03.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:03.792+0000 7f59bd200640 1 -- 172.21.15.5:0/4049587272 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f59b8104010 msgr2=0x7f59b81043f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:03.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:03.792+0000 7f59bd200640 1 --2- 172.21.15.5:0/4049587272 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f59b8104010 0x7f59b81043f0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f59a8009900 tx=0x7f59a802a3f0 comp rx=0 tx=0).stop 2024-09-10T10:00:03.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:03.792+0000 7f59bd200640 1 -- 172.21.15.5:0/4049587272 shutdown_connections 2024-09-10T10:00:03.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:03.792+0000 7f59bd200640 1 --2- 172.21.15.5:0/4049587272 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f59b8109140 0x7f59b8110cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:03.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:03.792+0000 7f59bd200640 1 --2- 172.21.15.5:0/4049587272 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f59b81049c0 0x7f59b8108a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:03.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:03.792+0000 7f59bd200640 1 --2- 172.21.15.5:0/4049587272 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f59b8104010 0x7f59b81043f0 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:03.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:03.792+0000 7f59bd200640 1 -- 172.21.15.5:0/4049587272 >> 172.21.15.5:0/4049587272 conn(0x7f59b80ffa50 msgr2=0x7f59b8101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:00:03.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:03.792+0000 7f59bd200640 1 -- 172.21.15.5:0/4049587272 shutdown_connections 2024-09-10T10:00:03.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:03.792+0000 7f59bd200640 1 -- 172.21.15.5:0/4049587272 wait complete. 2024-09-10T10:00:03.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:03.793+0000 7f59bd200640 1 Processor -- start 2024-09-10T10:00:03.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:03.793+0000 7f59bd200640 1 -- start start 2024-09-10T10:00:03.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:03.793+0000 7f59bd200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f59b8104010 0x7f59b819a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:03.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:03.793+0000 7f59bd200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f59b81049c0 0x7f59b819ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:03.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:03.793+0000 7f59bd200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f59b8109140 0x7f59b819ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:03.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:03.793+0000 7f59bd200640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f59b81141b0 con 0x7f59b8104010 2024-09-10T10:00:03.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:03.794+0000 7f59bd200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f59b8114030 con 0x7f59b81049c0 2024-09-10T10:00:03.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:03.794+0000 7f59bd200640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f59b8114330 con 0x7f59b8109140 2024-09-10T10:00:03.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:03.794+0000 7f59b5e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f59b8104010 0x7f59b819a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:03.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:03.794+0000 7f59b6800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f59b8109140 0x7f59b819ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:03.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:03.794+0000 7f59b5e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f59b8104010 0x7f59b819a630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52404/0 (socket says 172.21.15.5:52404) 2024-09-10T10:00:03.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:03.794+0000 7f59b5e00640 1 -- 172.21.15.5:0/225071566 learned_addr learned my addr 172.21.15.5:0/225071566 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T10:00:03.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:03.794+0000 7f59afe00640 1 --2- 172.21.15.5:0/225071566 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f59b81049c0 0x7f59b819ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:03.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:03.794+0000 7f59b6800640 1 -- 172.21.15.5:0/225071566 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f59b81049c0 msgr2=0x7f59b819ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:03.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:03.794+0000 7f59b6800640 1 --2- 172.21.15.5:0/225071566 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f59b81049c0 0x7f59b819ab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:03.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:03.794+0000 7f59b6800640 1 -- 172.21.15.5:0/225071566 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f59b8104010 msgr2=0x7f59b819a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:03.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:03.794+0000 7f59b6800640 1 --2- 172.21.15.5:0/225071566 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f59b8104010 0x7f59b819a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:03.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:03.794+0000 7f59b6800640 1 -- 172.21.15.5:0/225071566 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59b819f5e0 con 0x7f59b8109140 2024-09-10T10:00:03.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:03.794+0000 7f59afe00640 1 --2- 172.21.15.5:0/225071566 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f59b81049c0 0x7f59b819ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:00:03.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:03.794+0000 7f59b5e00640 1 --2- 172.21.15.5:0/225071566 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f59b8104010 0x7f59b819a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:00:03.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:03.795+0000 7f59b6800640 1 --2- 172.21.15.5:0/225071566 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f59b8109140 0x7f59b819ef00 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f59b0009fd0 tx=0x7f59b000ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:00:03.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:03.795+0000 7f59aea00640 1 -- 172.21.15.5:0/225071566 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59b000ce20 con 0x7f59b8109140 2024-09-10T10:00:03.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:03.795+0000 7f59aea00640 1 -- 172.21.15.5:0/225071566 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f59b0005020 con 0x7f59b8109140 2024-09-10T10:00:03.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:03.795+0000 7f59aea00640 1 -- 172.21.15.5:0/225071566 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59b0010720 con 0x7f59b8109140 2024-09-10T10:00:03.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:03.795+0000 7f59bd200640 1 -- 172.21.15.5:0/225071566 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59b819f8d0 con 0x7f59b8109140 2024-09-10T10:00:03.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:03.795+0000 7f59bd200640 1 -- 172.21.15.5:0/225071566 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59b81a71b0 con 0x7f59b8109140 2024-09-10T10:00:03.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:03.798+0000 7f59bfc58740 1 Processor -- start 2024-09-10T10:00:03.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:03.799+0000 7f59aea00640 1 -- 172.21.15.5:0/225071566 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f59b0005540 con 0x7f59b8109140 2024-09-10T10:00:03.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:03.799+0000 7f59bfc58740 1 -- start start 2024-09-10T10:00:03.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:03.799+0000 7f59bfc58740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b771090c70 0x55b771091050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:03.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:03.799+0000 7f59bfc58740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b771091590 0x55b77109b910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:03.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:03.799+0000 7f59bfc58740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b77109be50 0x55b77109e240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:03.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:03.799+0000 7f59bfc58740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55b770ff73e0 con 0x55b771091590 2024-09-10T10:00:03.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:03.799+0000 7f59bfc58740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55b770f9ed80 con 0x55b77109be50 2024-09-10T10:00:03.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:03.799+0000 7f59bfc58740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55b770ff33a0 con 0x55b771090c70 2024-09-10T10:00:03.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:03.800+0000 7f59b5e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b771091590 0x55b77109b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:03.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:03.800+0000 7f59b5e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b771091590 0x55b77109b910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52410/0 (socket says 172.21.15.5:52410) 2024-09-10T10:00:03.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:03.800+0000 7f59b5e00640 1 -- 172.21.15.5:0/3588590063 learned_addr learned my addr 172.21.15.5:0/3588590063 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T10:00:03.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:03.800+0000 7f59afe00640 1 --2- 172.21.15.5:0/3588590063 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b771090c70 0x55b771091050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:03.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:03.800+0000 7f59afe00640 1 --2- 172.21.15.5:0/3588590063 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b77109be50 0x55b77109e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:03.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:03.800+0000 7f59b5e00640 1 -- 172.21.15.5:0/3588590063 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b771090c70 msgr2=0x55b771091050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:00:03.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:03.800+0000 7f59b5e00640 1 --2- 172.21.15.5:0/3588590063 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b771090c70 0x55b771091050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:03.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:03.800+0000 7f59b5e00640 1 -- 172.21.15.5:0/3588590063 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b77109be50 msgr2=0x55b77109e240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:00:03.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:03.800+0000 7f59b5e00640 1 --2- 172.21.15.5:0/3588590063 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b77109be50 0x55b77109e240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:03.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:03.800+0000 7f59b5e00640 1 -- 172.21.15.5:0/3588590063 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b770fbdae0 con 0x55b771091590 2024-09-10T10:00:03.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:03.800+0000 7f59afe00640 1 --2- 172.21.15.5:0/3588590063 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b77109be50 0x55b77109e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:00:03.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:03.800+0000 7f59b5e00640 1 --2- 172.21.15.5:0/3588590063 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b771091590 0x55b77109b910 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f59a8005c40 tx=0x7f59a8005dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c666bb886bac6af5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:00:03.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:03.800+0000 7f59aea00640 1 --2- 172.21.15.5:0/225071566 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f599c0a2c00 0x7f599c0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:03.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:03.801+0000 7f59aea00640 1 -- 172.21.15.5:0/225071566 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f59b00cfa50 con 0x7f59b8109140 2024-09-10T10:00:03.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:03.801+0000 7f59a7400640 1 -- 172.21.15.5:0/3588590063 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59a8040070 con 0x55b771091590 2024-09-10T10:00:03.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:03.801+0000 7f59a7400640 1 -- 172.21.15.5:0/3588590063 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f59a80047a0 con 0x55b771091590 2024-09-10T10:00:03.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:03.801+0000 7f59a7400640 1 -- 172.21.15.5:0/3588590063 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59a8035040 con 0x55b771091590 2024-09-10T10:00:03.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:03.801+0000 7f59afe00640 1 --2- 172.21.15.5:0/225071566 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f599c0a2c00 0x7f599c0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:03.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:03.802+0000 7f59bfc58740 1 -- 172.21.15.5:0/3588590063 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b771091590 msgr2=0x55b77109b910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:03.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:03.802+0000 7f59bfc58740 1 --2- 172.21.15.5:0/3588590063 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b771091590 0x55b77109b910 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f59a8005c40 tx=0x7f59a8005dd0 comp rx=0 tx=0).stop 2024-09-10T10:00:03.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:03.802+0000 7f59afe00640 1 --2- 172.21.15.5:0/225071566 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f599c0a2c00 0x7f599c0a50c0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x55b77108e580 tx=0x7f59a0013040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:00:03.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:03.802+0000 7f59bfc58740 1 -- 172.21.15.5:0/3588590063 shutdown_connections 2024-09-10T10:00:03.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:03.802+0000 7f59bfc58740 1 --2- 172.21.15.5:0/3588590063 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b77109be50 0x55b77109e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:03.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:03.802+0000 7f59bfc58740 1 --2- 172.21.15.5:0/3588590063 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b771091590 0x55b77109b910 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:03.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:03.802+0000 7f59bfc58740 1 --2- 172.21.15.5:0/3588590063 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b771090c70 0x55b771091050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:03.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:03.802+0000 7f59bfc58740 1 -- 172.21.15.5:0/3588590063 >> 172.21.15.5:0/3588590063 conn(0x55b77108d200 msgr2=0x55b77108d5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:00:03.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:03.802+0000 7f59bfc58740 1 -- 172.21.15.5:0/3588590063 shutdown_connections 2024-09-10T10:00:03.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:03.802+0000 7f59bfc58740 1 -- 172.21.15.5:0/3588590063 wait complete. 2024-09-10T10:00:03.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:03.802+0000 7f59bfc58740 1 -- start start 2024-09-10T10:00:03.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:03.803+0000 7f59bfc58740 1 Processor -- start 2024-09-10T10:00:03.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:03.803+0000 7f59bfc58740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b771090c70 0x55b771098d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:03.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:03.803+0000 7f59bfc58740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b771091590 0x55b7710992c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:03.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:03.803+0000 7f59bfc58740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b77109be50 0x55b771099800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:03.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:03.803+0000 7f59bfc58740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55b77109ea20 con 0x55b771091590 2024-09-10T10:00:03.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:03.803+0000 7f59bfc58740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55b770fb20f0 con 0x55b771090c70 2024-09-10T10:00:03.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:03.803+0000 7f59bfc58740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55b77109eba0 con 0x55b77109be50 2024-09-10T10:00:03.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:03.803+0000 7f59bfc58740 10 client.0 _ll_register_callbacks cb 0x55b770f99bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:00:03.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:03.803+0000 7f59afe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b771091590 0x55b7710992c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:03.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:03.803+0000 7f59afe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b771091590 0x55b7710992c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52412/0 (socket says 172.21.15.5:52412) 2024-09-10T10:00:03.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:03.803+0000 7f59afe00640 1 -- 172.21.15.5:0/1003628735 learned_addr learned my addr 172.21.15.5:0/1003628735 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T10:00:03.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:03.803+0000 7f59b5e00640 1 --2- 172.21.15.5:0/1003628735 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b771090c70 0x55b771098d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:03.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:03.804+0000 7f59b6800640 1 --2- 172.21.15.5:0/1003628735 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b77109be50 0x55b771099800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:03.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:03.804+0000 7f59b5e00640 1 -- 172.21.15.5:0/1003628735 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b77109be50 msgr2=0x55b771099800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:00:03.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:03.804+0000 7f59b5e00640 1 --2- 172.21.15.5:0/1003628735 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b77109be50 0x55b771099800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:03.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:03.804+0000 7f59b5e00640 1 -- 172.21.15.5:0/1003628735 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b771091590 msgr2=0x55b7710992c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:00:03.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:03.804+0000 7f59b5e00640 1 --2- 172.21.15.5:0/1003628735 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b771091590 0x55b7710992c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:03.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:03.804+0000 7f59b5e00640 1 -- 172.21.15.5:0/1003628735 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b770d683c0 con 0x55b771090c70 2024-09-10T10:00:03.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:03.804+0000 7f59afe00640 1 --2- 172.21.15.5:0/1003628735 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b771091590 0x55b7710992c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:00:03.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:03.804+0000 7f59b6800640 1 --2- 172.21.15.5:0/1003628735 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b77109be50 0x55b771099800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:00:03.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:03.804+0000 7f59b5e00640 1 --2- 172.21.15.5:0/1003628735 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b771090c70 0x55b771098d80 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f59a8031550 tx=0x7f59a8009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=46bb1e7eb2b2db37 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:00:03.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:03.804+0000 7f599be00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-09-10T10:00:03.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:03.805+0000 7f599be00640 1 -- 172.21.15.5:0/1003628735 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59a804c020 con 0x55b771090c70 2024-09-10T10:00:03.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:03.805+0000 7f59bfc58740 1 -- 172.21.15.5:0/1003628735 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f59a80047a0 con 0x55b771090c70 2024-09-10T10:00:03.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:03.805+0000 7f599be00640 1 -- 172.21.15.5:0/1003628735 <== mon.2 v2:172.21.15.102:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f59a80051a0 con 0x55b771090c70 2024-09-10T10:00:03.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:03.805+0000 7f599be00640 1 -- 172.21.15.5:0/1003628735 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f59a8057050 con 0x55b771090c70 2024-09-10T10:00:03.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:03.805+0000 7f59bfc58740 10 client.29582 fetch_fsmap learned FSMap version 42 2024-09-10T10:00:03.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:03.805+0000 7f59bfc58740 1 -- 172.21.15.5:0/1003628735 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55b7710aaab0 con 0x55b771090c70 2024-09-10T10:00:03.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:03.805+0000 7f599be00640 1 -- 172.21.15.5:0/1003628735 <== mon.2 v2:172.21.15.102:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f59a80047a0 con 0x55b771090c70 2024-09-10T10:00:03.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:03.805+0000 7f599be00640 1 -- 172.21.15.5:0/1003628735 <== mon.2 v2:172.21.15.102:3300/0 5 ==== fsmap(e 42) ==== 8370+0+0 (secure 0 0 0) 0x7f59a8031640 con 0x55b771090c70 2024-09-10T10:00:03.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:03.805+0000 7f59bfc58740 10 client.29582 fetch_fsmap finished waiting for FSMap version 42 2024-09-10T10:00:03.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:03.805+0000 7f59bfc58740 10 client.29582 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-09-10T10:00:03.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:03.805+0000 7f59bfc58740 20 client.29582 populate_metadata read hostname 'smithi005' 2024-09-10T10:00:03.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:03.805+0000 7f59bfc58740 1 --2- 172.21.15.5:0/1003628735 >> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] conn(0x55b7710abe00 0x55b7710ae1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:03.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:03.806+0000 7f59bfc58740 4 client.29582 mds_command: new command op to 24443 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:00:03.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:03.806+0000 7f59bfc58740 1 -- 172.21.15.5:0/1003628735 --> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b771004950 con 0x55b7710abe00 2024-09-10T10:00:03.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:03.806+0000 7f59afe00640 1 --2- 172.21.15.5:0/1003628735 >> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] conn(0x55b7710abe00 0x55b7710ae1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:03.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:03.806+0000 7f59afe00640 1 --2- 172.21.15.5:0/1003628735 >> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] conn(0x55b7710abe00 0x55b7710ae1f0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=7db14bae0eff8c4 server_cookie=acb4c269ebbf2efc in_seq=0 out_seq=0 2024-09-10T10:00:03.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:03.806+0000 7f599be00640 10 client.29582 ms_handle_connect on v2:172.21.15.102:6832/2457007427 2024-09-10T10:00:03.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:03.807+0000 7f599be00640 1 -- 172.21.15.5:0/1003628735 <== mds.4 v2:172.21.15.102:6832/2457007427 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b771004950 con 0x55b7710abe00 2024-09-10T10:00:03.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:03.807+0000 7f599be00640 10 client.29582 handle_command_reply: tid=0 2024-09-10T10:00:03.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:03.807+0000 7f59bfc58740 1 client.29582 shutdown 2024-09-10T10:00:03.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:03.808+0000 7f59bfc58740 1 -- 172.21.15.5:0/1003628735 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b771090c70 msgr2=0x55b771098d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:03.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:03.808+0000 7f59bfc58740 1 --2- 172.21.15.5:0/1003628735 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b771090c70 0x55b771098d80 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f59a8031550 tx=0x7f59a8009260 comp rx=0 tx=0).stop 2024-09-10T10:00:03.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:03.808+0000 7f59bfc58740 1 -- 172.21.15.5:0/1003628735 shutdown_connections 2024-09-10T10:00:03.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:03.808+0000 7f59bfc58740 1 --2- 172.21.15.5:0/1003628735 >> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] conn(0x55b7710abe00 0x55b7710ae1f0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:03.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:03.808+0000 7f59bfc58740 1 --2- 172.21.15.5:0/1003628735 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b77109be50 0x55b771099800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:03.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:03.808+0000 7f59bfc58740 1 --2- 172.21.15.5:0/1003628735 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b771091590 0x55b7710992c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:03.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:03.808+0000 7f59bfc58740 1 --2- 172.21.15.5:0/1003628735 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b771090c70 0x55b771098d80 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:03.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:03.808+0000 7f599be00640 0 client.29582 ms_handle_reset on v2:172.21.15.102:6832/2457007427 2024-09-10T10:00:03.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:03.808+0000 7f59bfc58740 1 -- 172.21.15.5:0/1003628735 >> 172.21.15.5:0/1003628735 conn(0x55b77108d200 msgr2=0x55b771092ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:00:03.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:03.808+0000 7f59bfc58740 1 -- 172.21.15.5:0/1003628735 shutdown_connections 2024-09-10T10:00:03.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:03.808+0000 7f59bfc58740 1 -- 172.21.15.5:0/1003628735 wait complete. 2024-09-10T10:00:03.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:03.808+0000 7f59bfc58740 20 client.29582 trim_cache size 0 max 16384 2024-09-10T10:00:03.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:03.823+0000 7f59bfc58740 1 Processor -- start 2024-09-10T10:00:03.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:03.823+0000 7f59bfc58740 1 -- start start 2024-09-10T10:00:03.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:03.823+0000 7f59bfc58740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b771090c70 0x55b771093190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:03.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:03.823+0000 7f59bfc58740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b771091590 0x55b7710b37f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:03.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:03.823+0000 7f59bfc58740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b77109be50 0x55b7710b3d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:03.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:03.823+0000 7f59bfc58740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f59b81b0ff0 con 0x55b77109be50 2024-09-10T10:00:03.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:03.823+0000 7f59bfc58740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f59b81b2b30 con 0x55b771090c70 2024-09-10T10:00:03.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:03.823+0000 7f59bfc58740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f59b81b16c0 con 0x55b771091590 2024-09-10T10:00:03.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:03.823+0000 7f59b6800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b77109be50 0x55b7710b3d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:03.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:03.823+0000 7f59b5e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b771090c70 0x55b771093190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:03.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:03.823+0000 7f59b6800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b77109be50 0x55b7710b3d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52414/0 (socket says 172.21.15.5:52414) 2024-09-10T10:00:03.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:03.824+0000 7f59b6800640 1 -- 172.21.15.5:0/3974991023 learned_addr learned my addr 172.21.15.5:0/3974991023 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T10:00:03.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:03.824+0000 7f59afe00640 1 --2- 172.21.15.5:0/3974991023 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b771091590 0x55b7710b37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:03.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:03.824+0000 7f59b6800640 1 -- 172.21.15.5:0/3974991023 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b771091590 msgr2=0x55b7710b37f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:00:03.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:03.824+0000 7f59b6800640 1 --2- 172.21.15.5:0/3974991023 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b771091590 0x55b7710b37f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:03.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:03.824+0000 7f59b6800640 1 -- 172.21.15.5:0/3974991023 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b771090c70 msgr2=0x55b771093190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:00:03.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:03.824+0000 7f59b6800640 1 --2- 172.21.15.5:0/3974991023 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b771090c70 0x55b771093190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:03.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:03.824+0000 7f59b6800640 1 -- 172.21.15.5:0/3974991023 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b77108d030 con 0x55b77109be50 2024-09-10T10:00:03.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:03.824+0000 7f59afe00640 1 --2- 172.21.15.5:0/3974991023 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b771091590 0x55b7710b37f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:00:03.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:03.824+0000 7f59b5e00640 1 --2- 172.21.15.5:0/3974991023 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b771090c70 0x55b771093190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:00:03.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:03.824+0000 7f59b6800640 1 --2- 172.21.15.5:0/3974991023 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b77109be50 0x55b7710b3d30 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f59b000c760 tx=0x7f59b0098100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff80a2ffae9bf638 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:00:03.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:03.825+0000 7f59a6a00640 1 -- 172.21.15.5:0/3974991023 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59b009a160 con 0x55b77109be50 2024-09-10T10:00:03.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:03.825+0000 7f59a6a00640 1 -- 172.21.15.5:0/3974991023 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f59b00051d0 con 0x55b77109be50 2024-09-10T10:00:03.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:03.825+0000 7f59a6a00640 1 -- 172.21.15.5:0/3974991023 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59b009b2b0 con 0x55b77109be50 2024-09-10T10:00:03.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:03.825+0000 7f59bfc58740 1 -- 172.21.15.5:0/3974991023 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b77109be50 msgr2=0x55b7710b3d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:03.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:03.825+0000 7f59bfc58740 1 --2- 172.21.15.5:0/3974991023 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b77109be50 0x55b7710b3d30 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f59b000c760 tx=0x7f59b0098100 comp rx=0 tx=0).stop 2024-09-10T10:00:03.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:03.825+0000 7f59bfc58740 1 -- 172.21.15.5:0/3974991023 shutdown_connections 2024-09-10T10:00:03.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:03.825+0000 7f59bfc58740 1 --2- 172.21.15.5:0/3974991023 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b77109be50 0x55b7710b3d30 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:03.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:03.825+0000 7f59bfc58740 1 --2- 172.21.15.5:0/3974991023 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b771091590 0x55b7710b37f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:03.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:03.825+0000 7f59bfc58740 1 --2- 172.21.15.5:0/3974991023 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b771090c70 0x55b771093190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:03.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:03.825+0000 7f59bfc58740 1 -- 172.21.15.5:0/3974991023 >> 172.21.15.5:0/3974991023 conn(0x55b77108d200 msgr2=0x55b771097d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:00:03.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:03.826+0000 7f59bfc58740 1 -- 172.21.15.5:0/3974991023 shutdown_connections 2024-09-10T10:00:03.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:03.826+0000 7f59bfc58740 1 -- 172.21.15.5:0/3974991023 wait complete. 2024-09-10T10:00:03.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:03.826+0000 7f59bfc58740 1 -- start start 2024-09-10T10:00:03.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:03.826+0000 7f59bfc58740 1 Processor -- start 2024-09-10T10:00:03.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:03.827+0000 7f59bfc58740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b771090c70 0x55b7710a45d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:03.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:03.827+0000 7f59bfc58740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b771091590 0x55b771095af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:03.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:03.827+0000 7f59bfc58740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b77109be50 0x55b771096030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:03.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:03.827+0000 7f59bfc58740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f59b81b2460 con 0x55b771090c70 2024-09-10T10:00:03.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:03.827+0000 7f59bfc58740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f59b81b1d90 con 0x55b77109be50 2024-09-10T10:00:03.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:03.827+0000 7f59bfc58740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f59b81aefd0 con 0x55b771091590 2024-09-10T10:00:03.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:03.827+0000 7f59bfc58740 10 client.0 _ll_register_callbacks cb 0x7f59a0009e00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:00:03.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:03.827+0000 7f59b5e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b771090c70 0x55b7710a45d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:03.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:03.827+0000 7f59afe00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b771091590 0x55b771095af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:03.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:03.827+0000 7f59b5e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b771090c70 0x55b7710a45d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52424/0 (socket says 172.21.15.5:52424) 2024-09-10T10:00:03.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:03.827+0000 7f59b5e00640 1 -- 172.21.15.5:0/740068381 learned_addr learned my addr 172.21.15.5:0/740068381 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T10:00:03.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:03.827+0000 7f59b6800640 1 --2- 172.21.15.5:0/740068381 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b77109be50 0x55b771096030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:03.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:03.827+0000 7f59afe00640 1 -- 172.21.15.5:0/740068381 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b77109be50 msgr2=0x55b771096030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:00:03.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:03.827+0000 7f59afe00640 1 --2- 172.21.15.5:0/740068381 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b77109be50 0x55b771096030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:03.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:03.827+0000 7f59afe00640 1 -- 172.21.15.5:0/740068381 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b771090c70 msgr2=0x55b7710a45d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:00:03.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:03.827+0000 7f59afe00640 1 --2- 172.21.15.5:0/740068381 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b771090c70 0x55b7710a45d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:03.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:03.827+0000 7f59afe00640 1 -- 172.21.15.5:0/740068381 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b771088770 con 0x55b771091590 2024-09-10T10:00:03.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:03.828+0000 7f59afe00640 1 --2- 172.21.15.5:0/740068381 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b771091590 0x55b771095af0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x55b7710ae760 tx=0x7f59a00046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7775784ca485f00a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:00:03.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:03.828+0000 7f599be00640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-09-10T10:00:03.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:03.828+0000 7f599be00640 1 -- 172.21.15.5:0/740068381 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59a0032020 con 0x55b771091590 2024-09-10T10:00:03.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:03.828+0000 7f599be00640 1 -- 172.21.15.5:0/740068381 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f59a0009040 con 0x55b771091590 2024-09-10T10:00:03.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:03.828+0000 7f59bfc58740 1 -- 172.21.15.5:0/740068381 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f59b00051d0 con 0x55b771091590 2024-09-10T10:00:03.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:03.828+0000 7f599be00640 1 -- 172.21.15.5:0/740068381 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f59a003d050 con 0x55b771091590 2024-09-10T10:00:03.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:03.828+0000 7f59bfc58740 10 client.29566 fetch_fsmap learned FSMap version 42 2024-09-10T10:00:03.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:03.828+0000 7f59bfc58740 1 -- 172.21.15.5:0/740068381 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55b771116650 con 0x55b771091590 2024-09-10T10:00:03.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:03.829+0000 7f599be00640 1 -- 172.21.15.5:0/740068381 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f59b00051d0 con 0x55b771091590 2024-09-10T10:00:03.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:03.829+0000 7f599be00640 1 -- 172.21.15.5:0/740068381 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 42) ==== 8370+0+0 (secure 0 0 0) 0x7f59a001f920 con 0x55b771091590 2024-09-10T10:00:03.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:03.829+0000 7f59bfc58740 10 client.29566 fetch_fsmap finished waiting for FSMap version 42 2024-09-10T10:00:03.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:03.829+0000 7f59bfc58740 10 client.29566 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-09-10T10:00:03.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:03.829+0000 7f59bfc58740 20 client.29566 populate_metadata read hostname 'smithi005' 2024-09-10T10:00:03.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:03.829+0000 7f59bfc58740 1 --2- 172.21.15.5:0/740068381 >> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] conn(0x55b771116dd0 0x55b7711191f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:03.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:03.829+0000 7f59bfc58740 4 client.29566 mds_command: new command op to 24443 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T10:00:03.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:03.829+0000 7f59bfc58740 1 -- 172.21.15.5:0/740068381 --> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b770ffee60 con 0x55b771116dd0 2024-09-10T10:00:03.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:03.829+0000 7f59b5e00640 1 --2- 172.21.15.5:0/740068381 >> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] conn(0x55b771116dd0 0x55b7711191f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:03.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:03.830+0000 7f59b5e00640 1 --2- 172.21.15.5:0/740068381 >> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] conn(0x55b771116dd0 0x55b7711191f0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=ac982cf4c300da67 server_cookie=16fc1cb62792097c in_seq=0 out_seq=0 2024-09-10T10:00:03.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:03.830+0000 7f599be00640 10 client.29566 ms_handle_connect on v2:172.21.15.102:6832/2457007427 2024-09-10T10:00:03.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:03.830+0000 7f599be00640 1 -- 172.21.15.5:0/740068381 <== mds.4 v2:172.21.15.102:6832/2457007427 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b770ffee60 con 0x55b771116dd0 2024-09-10T10:00:03.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:03.830+0000 7f599be00640 10 client.29566 handle_command_reply: tid=0 2024-09-10T10:00:03.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:03.830+0000 7f59bfc58740 1 client.29566 shutdown 2024-09-10T10:00:03.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:03.830+0000 7f59bfc58740 1 -- 172.21.15.5:0/740068381 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b771091590 msgr2=0x55b771095af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:03.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:03.830+0000 7f59bfc58740 1 --2- 172.21.15.5:0/740068381 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b771091590 0x55b771095af0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x55b7710ae760 tx=0x7f59a00046c0 comp rx=0 tx=0).stop 2024-09-10T10:00:03.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:03.830+0000 7f59bfc58740 1 -- 172.21.15.5:0/740068381 shutdown_connections 2024-09-10T10:00:03.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:03.830+0000 7f59bfc58740 1 --2- 172.21.15.5:0/740068381 >> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] conn(0x55b771116dd0 0x55b7711191f0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:03.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:03.831+0000 7f59bfc58740 1 --2- 172.21.15.5:0/740068381 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b77109be50 0x55b771096030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:03.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:03.831+0000 7f59bfc58740 1 --2- 172.21.15.5:0/740068381 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b771091590 0x55b771095af0 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:03.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:03.831+0000 7f59bfc58740 1 --2- 172.21.15.5:0/740068381 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b771090c70 0x55b7710a45d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:03.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:03.831+0000 7f599be00640 0 client.29566 ms_handle_reset on v2:172.21.15.102:6832/2457007427 2024-09-10T10:00:03.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:03.831+0000 7f59bfc58740 1 -- 172.21.15.5:0/740068381 >> 172.21.15.5:0/740068381 conn(0x55b77108d200 msgr2=0x55b771097520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:00:03.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:03.831+0000 7f59bfc58740 1 -- 172.21.15.5:0/740068381 shutdown_connections 2024-09-10T10:00:03.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:03.831+0000 7f59bfc58740 1 -- 172.21.15.5:0/740068381 wait complete. 2024-09-10T10:00:03.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:03.831+0000 7f59bfc58740 20 client.29566 trim_cache size 0 max 16384 2024-09-10T10:00:03.846 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-10T10:00:03.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:03.831+0000 7f59bd200640 1 -- 172.21.15.5:0/225071566 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f599c0a2c00 msgr2=0x7f599c0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:03.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:03.831+0000 7f59bd200640 1 --2- 172.21.15.5:0/225071566 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f599c0a2c00 0x7f599c0a50c0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x55b77108e580 tx=0x7f59a0013040 comp rx=0 tx=0).stop 2024-09-10T10:00:03.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:03.831+0000 7f59bd200640 1 -- 172.21.15.5:0/225071566 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f59b8109140 msgr2=0x7f59b819ef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:03.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:03.831+0000 7f59bd200640 1 --2- 172.21.15.5:0/225071566 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f59b8109140 0x7f59b819ef00 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f59b0009fd0 tx=0x7f59b000ef10 comp rx=0 tx=0).stop 2024-09-10T10:00:03.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:03.832+0000 7f59bd200640 1 -- 172.21.15.5:0/225071566 shutdown_connections 2024-09-10T10:00:03.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:03.832+0000 7f59bd200640 1 --2- 172.21.15.5:0/225071566 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f599c0a2c00 0x7f599c0a50c0 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:03.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:03.832+0000 7f59bd200640 1 --2- 172.21.15.5:0/225071566 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f59b8109140 0x7f59b819ef00 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:03.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:03.832+0000 7f59bd200640 1 --2- 172.21.15.5:0/225071566 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f59b81049c0 0x7f59b819ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:03.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:03.832+0000 7f59bd200640 1 --2- 172.21.15.5:0/225071566 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f59b8104010 0x7f59b819a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:03.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:03.832+0000 7f59bd200640 1 -- 172.21.15.5:0/225071566 >> 172.21.15.5:0/225071566 conn(0x7f59b80ffa50 msgr2=0x7f59b8100dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:00:03.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:03.832+0000 7f59bd200640 1 -- 172.21.15.5:0/225071566 shutdown_connections 2024-09-10T10:00:03.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:03.832+0000 7f59bd200640 1 -- 172.21.15.5:0/225071566 wait complete. 2024-09-10T10:00:04.029 INFO:tasks.workunit.client.0.smithi005.stdout: 10 130511 14.40 MB/sec execute 342 sec latency 3579.659 ms 2024-09-10T10:00:05.026 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-10T10:00:05.031 INFO:tasks.workunit.client.0.smithi005.stdout: 10 130511 14.36 MB/sec execute 343 sec latency 4579.706 ms 2024-09-10T10:00:05.189 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:00:05 smithi005 ceph-mon[30350]: pgmap v1045: 129 pgs: 129 active+clean; 709 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 4.9 MiB/s wr, 453 op/s 2024-09-10T10:00:05.413 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:00:05.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:00:05 smithi159 ceph-mon[35513]: pgmap v1045: 129 pgs: 129 active+clean; 709 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 4.9 MiB/s wr, 453 op/s 2024-09-10T10:00:05.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:00:05 smithi102 ceph-mon[35442]: pgmap v1045: 129 pgs: 129 active+clean; 709 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 4.9 MiB/s wr, 453 op/s 2024-09-10T10:00:06.030 INFO:tasks.workunit.client.0.smithi005.stdout: 10 130575 14.33 MB/sec execute 344 sec latency 5247.045 ms 2024-09-10T10:00:06.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:06.341+0000 7f185ca00640 1 -- 172.21.15.5:0/259253134 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1858105d90 msgr2=0x7f1858106220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:06.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:06.341+0000 7f1856000640 1 -- 172.21.15.5:0/259253134 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1850045020 con 0x7f1858105d90 2024-09-10T10:00:06.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:06.341+0000 7f185ca00640 1 --2- 172.21.15.5:0/259253134 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1858105d90 0x7f1858106220 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f185001acb0 tx=0x7f185003b970 comp rx=0 tx=0).stop 2024-09-10T10:00:06.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:06.341+0000 7f185ca00640 1 -- 172.21.15.5:0/259253134 shutdown_connections 2024-09-10T10:00:06.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:06.341+0000 7f185ca00640 1 --2- 172.21.15.5:0/259253134 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1858105d90 0x7f1858106220 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:06.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:06.341+0000 7f185ca00640 1 --2- 172.21.15.5:0/259253134 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f18581012a0 0x7f1858105770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:06.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:06.341+0000 7f185ca00640 1 --2- 172.21.15.5:0/259253134 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1858100980 0x7f1858100d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:06.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:06.341+0000 7f185ca00640 1 -- 172.21.15.5:0/259253134 >> 172.21.15.5:0/259253134 conn(0x7f18580fc610 msgr2=0x7f18580fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:00:06.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:06.342+0000 7f185ca00640 1 -- 172.21.15.5:0/259253134 shutdown_connections 2024-09-10T10:00:06.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:06.343+0000 7f185ca00640 1 -- 172.21.15.5:0/259253134 wait complete. 2024-09-10T10:00:06.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:06.347+0000 7f185ca00640 1 Processor -- start 2024-09-10T10:00:06.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:06.348+0000 7f185ca00640 1 -- start start 2024-09-10T10:00:06.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:06.349+0000 7f185ca00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1858100980 0x7f1858110cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:06.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:06.350+0000 7f185ca00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f18581012a0 0x7f1858111210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:06.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:06.350+0000 7f1857400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1858100980 0x7f1858110cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:06.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:06.350+0000 7f1857400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1858100980 0x7f1858110cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:39668/0 (socket says 172.21.15.5:39668) 2024-09-10T10:00:06.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:06.351+0000 7f1857400640 1 -- 172.21.15.5:0/3291846767 learned_addr learned my addr 172.21.15.5:0/3291846767 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T10:00:06.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:06.351+0000 7f185ca00640 1 --2- 172.21.15.5:0/3291846767 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1858105d90 0x7f1858114590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:06.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:06.352+0000 7f185ca00640 1 -- 172.21.15.5:0/3291846767 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1858109760 con 0x7f18581012a0 2024-09-10T10:00:06.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:06.352+0000 7f185ca00640 1 -- 172.21.15.5:0/3291846767 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f18581095e0 con 0x7f1858100980 2024-09-10T10:00:06.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:06.352+0000 7f185ca00640 1 -- 172.21.15.5:0/3291846767 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f18581098e0 con 0x7f1858105d90 2024-09-10T10:00:06.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:06.352+0000 7f1857e00640 1 --2- 172.21.15.5:0/3291846767 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1858105d90 0x7f1858114590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:06.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:06.352+0000 7f1856a00640 1 --2- 172.21.15.5:0/3291846767 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f18581012a0 0x7f1858111210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:06.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:06.353+0000 7f1857e00640 1 -- 172.21.15.5:0/3291846767 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1858100980 msgr2=0x7f1858110cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:06.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:06.353+0000 7f1857e00640 1 --2- 172.21.15.5:0/3291846767 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1858100980 0x7f1858110cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:06.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:06.353+0000 7f1857e00640 1 -- 172.21.15.5:0/3291846767 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f18581012a0 msgr2=0x7f1858111210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:06.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:06.353+0000 7f1857e00640 1 --2- 172.21.15.5:0/3291846767 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f18581012a0 0x7f1858111210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:06.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:06.353+0000 7f1857e00640 1 -- 172.21.15.5:0/3291846767 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1858114c70 con 0x7f1858105d90 2024-09-10T10:00:06.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:06.354+0000 7f1857e00640 1 --2- 172.21.15.5:0/3291846767 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1858105d90 0x7f1858114590 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f185001acb0 tx=0x7f18500419e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:00:06.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:06.354+0000 7f184fe00640 1 -- 172.21.15.5:0/3291846767 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1850045070 con 0x7f1858105d90 2024-09-10T10:00:06.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:06.355+0000 7f185ca00640 1 -- 172.21.15.5:0/3291846767 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1858114f00 con 0x7f1858105d90 2024-09-10T10:00:06.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:06.355+0000 7f185ca00640 1 -- 172.21.15.5:0/3291846767 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18581a2a00 con 0x7f1858105d90 2024-09-10T10:00:06.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:06.355+0000 7f184fe00640 1 -- 172.21.15.5:0/3291846767 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1850003970 con 0x7f1858105d90 2024-09-10T10:00:06.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:06.355+0000 7f184fe00640 1 -- 172.21.15.5:0/3291846767 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18500046c0 con 0x7f1858105d90 2024-09-10T10:00:06.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:06.359+0000 7f1860c36740 1 Processor -- start 2024-09-10T10:00:06.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:06.366+0000 7f184fe00640 1 -- 172.21.15.5:0/3291846767 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f1850048020 con 0x7f1858105d90 2024-09-10T10:00:06.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:06.366+0000 7f1860c36740 1 -- start start 2024-09-10T10:00:06.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:06.367+0000 7f1860c36740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55982eacee70 0x55982eacf250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:06.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:06.367+0000 7f1860c36740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55982eacf790 0x55982ead9b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:06.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:06.367+0000 7f1860c36740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55982eada050 0x55982eadc440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:06.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:06.367+0000 7f1860c36740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55982ea353e0 con 0x55982eada050 2024-09-10T10:00:06.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:06.368+0000 7f1860c36740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55982e9dcd80 con 0x55982eacee70 2024-09-10T10:00:06.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:06.368+0000 7f1860c36740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55982ea313a0 con 0x55982eacf790 2024-09-10T10:00:06.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:06.368+0000 7f1857400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55982eacf790 0x55982ead9b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:06.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:06.368+0000 7f1857400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55982eacf790 0x55982ead9b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:33898/0 (socket says 172.21.15.5:33898) 2024-09-10T10:00:06.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:06.368+0000 7f1857400640 1 -- 172.21.15.5:0/1071167827 learned_addr learned my addr 172.21.15.5:0/1071167827 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T10:00:06.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:06.368+0000 7f1856a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55982eacee70 0x55982eacf250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:06.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:06.370+0000 7f1857400640 1 -- 172.21.15.5:0/1071167827 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55982eacee70 msgr2=0x55982eacf250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:00:06.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:06.371+0000 7f184fe00640 1 --2- 172.21.15.5:0/3291846767 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f183c0a2c40 0x7f183c0a5100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:06.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:06.371+0000 7f1857400640 1 --2- 172.21.15.5:0/1071167827 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55982eacee70 0x55982eacf250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:06.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:06.371+0000 7f1857400640 1 -- 172.21.15.5:0/1071167827 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55982eada050 msgr2=0x55982eadc440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:00:06.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:06.371+0000 7f1857400640 1 --2- 172.21.15.5:0/1071167827 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55982eada050 0x55982eadc440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:06.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:06.371+0000 7f1857400640 1 -- 172.21.15.5:0/1071167827 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55982e9fbae0 con 0x55982eacf790 2024-09-10T10:00:06.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:06.371+0000 7f184fe00640 1 -- 172.21.15.5:0/3291846767 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f1850101330 con 0x7f1858105d90 2024-09-10T10:00:06.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:06.371+0000 7f1857400640 1 --2- 172.21.15.5:0/1071167827 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55982eacf790 0x55982ead9b10 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f1848007890 tx=0x7f1848007d50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=92ff9350c6456abe server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:00:06.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:06.372+0000 7f1843e00640 1 -- 172.21.15.5:0/1071167827 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f184800d950 con 0x55982eacf790 2024-09-10T10:00:06.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:06.372+0000 7f1843e00640 1 -- 172.21.15.5:0/1071167827 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1848018070 con 0x55982eacf790 2024-09-10T10:00:06.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:06.374+0000 7f1857e00640 1 --2- 172.21.15.5:0/3291846767 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f183c0a2c40 0x7f183c0a5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:06.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:06.377+0000 7f1843e00640 1 -- 172.21.15.5:0/1071167827 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18480123e0 con 0x55982eacf790 2024-09-10T10:00:06.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:06.377+0000 7f1860c36740 1 -- 172.21.15.5:0/1071167827 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55982eacf790 msgr2=0x55982ead9b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:06.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:06.377+0000 7f1860c36740 1 --2- 172.21.15.5:0/1071167827 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55982eacf790 0x55982ead9b10 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f1848007890 tx=0x7f1848007d50 comp rx=0 tx=0).stop 2024-09-10T10:00:06.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:06.378+0000 7f1857e00640 1 --2- 172.21.15.5:0/3291846767 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f183c0a2c40 0x7f183c0a5100 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f185001a630 tx=0x7f1850005f10 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:00:06.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:06.378+0000 7f1860c36740 1 -- 172.21.15.5:0/1071167827 shutdown_connections 2024-09-10T10:00:06.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:06.378+0000 7f1860c36740 1 --2- 172.21.15.5:0/1071167827 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55982eada050 0x55982eadc440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:06.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:06.378+0000 7f1860c36740 1 --2- 172.21.15.5:0/1071167827 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55982eacf790 0x55982ead9b10 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:06.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:06.378+0000 7f1860c36740 1 --2- 172.21.15.5:0/1071167827 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55982eacee70 0x55982eacf250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:06.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:06.378+0000 7f1860c36740 1 -- 172.21.15.5:0/1071167827 >> 172.21.15.5:0/1071167827 conn(0x55982eacb400 msgr2=0x55982eacb7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:00:06.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:06.379+0000 7f1860c36740 1 -- 172.21.15.5:0/1071167827 shutdown_connections 2024-09-10T10:00:06.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:06.380+0000 7f1860c36740 1 -- 172.21.15.5:0/1071167827 wait complete. 2024-09-10T10:00:06.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:06.380+0000 7f1860c36740 1 -- start start 2024-09-10T10:00:06.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:06.381+0000 7f1860c36740 1 Processor -- start 2024-09-10T10:00:06.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:06.381+0000 7f1860c36740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55982eacee70 0x55982ead9b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:06.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:06.381+0000 7f1860c36740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55982eacf790 0x55982ead6db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:06.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:06.381+0000 7f1860c36740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55982eada050 0x55982ead7540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:06.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:06.381+0000 7f1860c36740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55982eaedcd0 con 0x55982eada050 2024-09-10T10:00:06.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:06.381+0000 7f1860c36740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55982e9f00f0 con 0x55982eacf790 2024-09-10T10:00:06.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:06.381+0000 7f1860c36740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55982eaede50 con 0x55982eacee70 2024-09-10T10:00:06.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:06.381+0000 7f1860c36740 10 client.0 _ll_register_callbacks cb 0x55982e9d7bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:00:06.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:06.382+0000 7f1856a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55982eacee70 0x55982ead9b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:06.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:06.382+0000 7f1856a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55982eacee70 0x55982ead9b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:33904/0 (socket says 172.21.15.5:33904) 2024-09-10T10:00:06.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:06.382+0000 7f1856a00640 1 -- 172.21.15.5:0/2769573790 learned_addr learned my addr 172.21.15.5:0/2769573790 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T10:00:06.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:06.383+0000 7f1857400640 1 --2- 172.21.15.5:0/2769573790 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55982eacf790 0x55982ead6db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:06.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:06.383+0000 7f1856a00640 1 --2- 172.21.15.5:0/2769573790 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55982eada050 0x55982ead7540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:06.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:06.383+0000 7f1856a00640 1 -- 172.21.15.5:0/2769573790 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55982eacf790 msgr2=0x55982ead6db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:00:06.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:06.383+0000 7f1856a00640 1 --2- 172.21.15.5:0/2769573790 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55982eacf790 0x55982ead6db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:06.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:06.383+0000 7f1856a00640 1 -- 172.21.15.5:0/2769573790 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55982eada050 msgr2=0x55982ead7540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:00:06.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:06.383+0000 7f1856a00640 1 --2- 172.21.15.5:0/2769573790 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55982eada050 0x55982ead7540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:06.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:06.384+0000 7f1856a00640 1 -- 172.21.15.5:0/2769573790 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55982e7a63c0 con 0x55982eacee70 2024-09-10T10:00:06.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:06.385+0000 7f1856a00640 1 --2- 172.21.15.5:0/2769573790 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55982eacee70 0x55982ead9b00 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f1844005b90 tx=0x7f184400e610 comp rx=0 tx=0).ready entity=mon.1 client_cookie=53c82a07a562aeb server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:00:06.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:06.385+0000 7f1840c00640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-09-10T10:00:06.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:06.386+0000 7f1840c00640 1 -- 172.21.15.5:0/2769573790 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1844011d10 con 0x55982eacee70 2024-09-10T10:00:06.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:06.386+0000 7f1840c00640 1 -- 172.21.15.5:0/2769573790 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f1844005020 con 0x55982eacee70 2024-09-10T10:00:06.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:06.386+0000 7f1860c36740 1 -- 172.21.15.5:0/2769573790 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1848018070 con 0x55982eacee70 2024-09-10T10:00:06.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:06.386+0000 7f1840c00640 1 -- 172.21.15.5:0/2769573790 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f1844005220 con 0x55982eacee70 2024-09-10T10:00:06.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:06.387+0000 7f1860c36740 10 client.29578 fetch_fsmap learned FSMap version 42 2024-09-10T10:00:06.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:06.387+0000 7f1860c36740 1 -- 172.21.15.5:0/2769573790 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55982eaf9cd0 con 0x55982eacee70 2024-09-10T10:00:06.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:06.387+0000 7f1840c00640 1 -- 172.21.15.5:0/2769573790 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1848018070 con 0x55982eacee70 2024-09-10T10:00:06.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:06.388+0000 7f1840c00640 1 -- 172.21.15.5:0/2769573790 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 42) ==== 8370+0+0 (secure 0 0 0) 0x7f184400f040 con 0x55982eacee70 2024-09-10T10:00:06.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:06.389+0000 7f1860c36740 10 client.29578 fetch_fsmap finished waiting for FSMap version 42 2024-09-10T10:00:06.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:06.390+0000 7f1860c36740 10 client.29578 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-09-10T10:00:06.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:06.390+0000 7f1860c36740 20 client.29578 populate_metadata read hostname 'smithi005' 2024-09-10T10:00:06.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:06.390+0000 7f1860c36740 1 --2- 172.21.15.5:0/2769573790 >> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] conn(0x55982eafb190 0x55982eafd580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:06.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:06.391+0000 7f1860c36740 4 client.29578 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:00:06.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:06.391+0000 7f1860c36740 1 -- 172.21.15.5:0/2769573790 --> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55982ea42950 con 0x55982eafb190 2024-09-10T10:00:06.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:06.392+0000 7f1857400640 1 --2- 172.21.15.5:0/2769573790 >> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] conn(0x55982eafb190 0x55982eafd580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:06.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:06.394+0000 7f1857400640 1 --2- 172.21.15.5:0/2769573790 >> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] conn(0x55982eafb190 0x55982eafd580 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=d4bd67a26acbc8e server_cookie=424c62794079ac94 in_seq=0 out_seq=0 2024-09-10T10:00:06.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:06.394+0000 7f1840c00640 10 client.29578 ms_handle_connect on v2:172.21.15.102:6836/2689776764 2024-09-10T10:00:06.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:06.395+0000 7f1840c00640 1 -- 172.21.15.5:0/2769573790 <== mds.3 v2:172.21.15.102:6836/2689776764 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55982ea42950 con 0x55982eafb190 2024-09-10T10:00:06.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:06.395+0000 7f1840c00640 10 client.29578 handle_command_reply: tid=0 2024-09-10T10:00:06.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:06.396+0000 7f1860c36740 1 client.29578 shutdown 2024-09-10T10:00:06.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:06.402+0000 7f1860c36740 1 -- 172.21.15.5:0/2769573790 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55982eacee70 msgr2=0x55982ead9b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:06.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:06.402+0000 7f1860c36740 1 --2- 172.21.15.5:0/2769573790 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55982eacee70 0x55982ead9b00 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f1844005b90 tx=0x7f184400e610 comp rx=0 tx=0).stop 2024-09-10T10:00:06.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:06.402+0000 7f1860c36740 1 -- 172.21.15.5:0/2769573790 shutdown_connections 2024-09-10T10:00:06.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:06.402+0000 7f1860c36740 1 --2- 172.21.15.5:0/2769573790 >> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] conn(0x55982eafb190 0x55982eafd580 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:06.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:06.402+0000 7f1860c36740 1 --2- 172.21.15.5:0/2769573790 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55982eada050 0x55982ead7540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:06.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:06.402+0000 7f1860c36740 1 --2- 172.21.15.5:0/2769573790 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55982eacf790 0x55982ead6db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:06.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:06.402+0000 7f1860c36740 1 --2- 172.21.15.5:0/2769573790 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55982eacee70 0x55982ead9b00 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:06.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:06.402+0000 7f1860c36740 1 -- 172.21.15.5:0/2769573790 >> 172.21.15.5:0/2769573790 conn(0x55982eacb400 msgr2=0x55982eaca580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:00:06.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:06.402+0000 7f1840c00640 0 client.29578 ms_handle_reset on v2:172.21.15.102:6836/2689776764 2024-09-10T10:00:06.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:06.402+0000 7f1860c36740 1 -- 172.21.15.5:0/2769573790 shutdown_connections 2024-09-10T10:00:06.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:06.403+0000 7f1860c36740 1 -- 172.21.15.5:0/2769573790 wait complete. 2024-09-10T10:00:06.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:06.404+0000 7f1860c36740 20 client.29578 trim_cache size 0 max 16384 2024-09-10T10:00:06.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:06.443+0000 7f1860c36740 1 Processor -- start 2024-09-10T10:00:06.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:06.443+0000 7f1860c36740 1 -- start start 2024-09-10T10:00:06.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:06.443+0000 7f1860c36740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55982eacee70 0x55982ead9b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:06.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:06.443+0000 7f1860c36740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55982eacf790 0x55982ead6d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:06.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:06.443+0000 7f1860c36740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55982eada050 0x55982eb02e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:06.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:06.443+0000 7f1860c36740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f183800fd20 con 0x55982eacee70 2024-09-10T10:00:06.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:06.443+0000 7f1860c36740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f1838011860 con 0x55982eacf790 2024-09-10T10:00:06.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:06.443+0000 7f1860c36740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f18380103f0 con 0x55982eada050 2024-09-10T10:00:06.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:06.444+0000 7f1856a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55982eacee70 0x55982ead9b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:06.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:06.444+0000 7f1857400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55982eacf790 0x55982ead6d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:06.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:06.444+0000 7f1856a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55982eacee70 0x55982ead9b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52474/0 (socket says 172.21.15.5:52474) 2024-09-10T10:00:06.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:06.444+0000 7f1856a00640 1 -- 172.21.15.5:0/1292333037 learned_addr learned my addr 172.21.15.5:0/1292333037 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T10:00:06.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:06.445+0000 7f1856a00640 1 --2- 172.21.15.5:0/1292333037 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55982eada050 0x55982eb02e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:06.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:06.445+0000 7f1856a00640 1 -- 172.21.15.5:0/1292333037 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55982eada050 msgr2=0x55982eb02e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:00:06.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:06.445+0000 7f1856a00640 1 --2- 172.21.15.5:0/1292333037 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55982eada050 0x55982eb02e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:06.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:06.445+0000 7f1856a00640 1 -- 172.21.15.5:0/1292333037 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55982eacf790 msgr2=0x55982ead6d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:00:06.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:06.445+0000 7f1856a00640 1 --2- 172.21.15.5:0/1292333037 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55982eacf790 0x55982ead6d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:06.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:06.445+0000 7f1856a00640 1 -- 172.21.15.5:0/1292333037 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55982eacb230 con 0x55982eacee70 2024-09-10T10:00:06.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:06.446+0000 7f1856a00640 1 --2- 172.21.15.5:0/1292333037 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55982eacee70 0x55982ead9b00 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f18440118b0 tx=0x7f184400e700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb6eac1df0147bd5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:00:06.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:06.447+0000 7f1843400640 1 -- 172.21.15.5:0/1292333037 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18440055c0 con 0x55982eacee70 2024-09-10T10:00:06.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:06.448+0000 7f1843400640 1 -- 172.21.15.5:0/1292333037 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1844013040 con 0x55982eacee70 2024-09-10T10:00:06.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:06.448+0000 7f1843400640 1 -- 172.21.15.5:0/1292333037 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1844026e10 con 0x55982eacee70 2024-09-10T10:00:06.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:06.449+0000 7f1860c36740 1 -- 172.21.15.5:0/1292333037 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55982eacee70 msgr2=0x55982ead9b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:06.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:06.449+0000 7f1860c36740 1 --2- 172.21.15.5:0/1292333037 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55982eacee70 0x55982ead9b00 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f18440118b0 tx=0x7f184400e700 comp rx=0 tx=0).stop 2024-09-10T10:00:06.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:06.449+0000 7f1860c36740 1 -- 172.21.15.5:0/1292333037 shutdown_connections 2024-09-10T10:00:06.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:06.449+0000 7f1860c36740 1 --2- 172.21.15.5:0/1292333037 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55982eada050 0x55982eb02e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:06.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:06.449+0000 7f1860c36740 1 --2- 172.21.15.5:0/1292333037 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55982eacf790 0x55982ead6d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:06.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:06.450+0000 7f1860c36740 1 --2- 172.21.15.5:0/1292333037 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55982eacee70 0x55982ead9b00 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:06.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:06.450+0000 7f1860c36740 1 -- 172.21.15.5:0/1292333037 >> 172.21.15.5:0/1292333037 conn(0x55982eacb400 msgr2=0x55982ead6670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:00:06.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:06.450+0000 7f1860c36740 1 -- 172.21.15.5:0/1292333037 shutdown_connections 2024-09-10T10:00:06.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:06.451+0000 7f1860c36740 1 -- 172.21.15.5:0/1292333037 wait complete. 2024-09-10T10:00:06.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:06.451+0000 7f1860c36740 1 -- start start 2024-09-10T10:00:06.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:06.452+0000 7f1860c36740 1 Processor -- start 2024-09-10T10:00:06.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:06.454+0000 7f1860c36740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55982eacee70 0x55982eaf3190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:06.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:06.454+0000 7f1860c36740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55982eacf790 0x55982eaf36d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:06.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:06.455+0000 7f1860c36740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55982eada050 0x55982eaee140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:06.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:06.455+0000 7f1860c36740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1838011190 con 0x55982eacf790 2024-09-10T10:00:06.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:06.455+0000 7f1860c36740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f1838010ac0 con 0x55982eacee70 2024-09-10T10:00:06.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:06.455+0000 7f1860c36740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f183800dd00 con 0x55982eada050 2024-09-10T10:00:06.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:06.455+0000 7f1860c36740 10 client.0 _ll_register_callbacks cb 0x7f185806afd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:00:06.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:06.455+0000 7f1856a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55982eacee70 0x55982eaf3190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:06.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:06.455+0000 7f1856a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55982eacee70 0x55982eaf3190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:39706/0 (socket says 172.21.15.5:39706) 2024-09-10T10:00:06.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:06.455+0000 7f1856a00640 1 -- 172.21.15.5:0/826887463 learned_addr learned my addr 172.21.15.5:0/826887463 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T10:00:06.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:06.456+0000 7f1856a00640 1 --2- 172.21.15.5:0/826887463 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55982eada050 0x55982eaee140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:06.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:06.456+0000 7f1857400640 1 --2- 172.21.15.5:0/826887463 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55982eacf790 0x55982eaf36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:06.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:06.457+0000 7f1857400640 1 -- 172.21.15.5:0/826887463 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55982eada050 msgr2=0x55982eaee140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:00:06.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:06.458+0000 7f1857400640 1 --2- 172.21.15.5:0/826887463 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55982eada050 0x55982eaee140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:06.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:06.458+0000 7f1857400640 1 -- 172.21.15.5:0/826887463 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55982eacee70 msgr2=0x55982eaf3190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:00:06.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:06.458+0000 7f1857400640 1 --2- 172.21.15.5:0/826887463 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55982eacee70 0x55982eaf3190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:06.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:06.458+0000 7f1857400640 1 -- 172.21.15.5:0/826887463 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55982eaf3d40 con 0x55982eacf790 2024-09-10T10:00:06.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:06.458+0000 7f1856a00640 1 --2- 172.21.15.5:0/826887463 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55982eada050 0x55982eaee140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:00:06.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:06.459+0000 7f1857400640 1 --2- 172.21.15.5:0/826887463 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55982eacf790 0x55982eaf36d0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f1848013b60 tx=0x7f1848007430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed89568470fc9597 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:00:06.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:06.459+0000 7f1840c00640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-10T10:00:06.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:06.459+0000 7f1840c00640 1 -- 172.21.15.5:0/826887463 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f184800d950 con 0x55982eacf790 2024-09-10T10:00:06.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:06.462+0000 7f1840c00640 1 -- 172.21.15.5:0/826887463 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f184800daf0 con 0x55982eacf790 2024-09-10T10:00:06.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:06.462+0000 7f1860c36740 1 -- 172.21.15.5:0/826887463 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1844013040 con 0x55982eacf790 2024-09-10T10:00:06.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:06.473+0000 7f1840c00640 1 -- 172.21.15.5:0/826887463 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f184800dcf0 con 0x55982eacf790 2024-09-10T10:00:06.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:06.473+0000 7f1860c36740 10 client.19524 fetch_fsmap learned FSMap version 42 2024-09-10T10:00:06.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:06.473+0000 7f1860c36740 1 -- 172.21.15.5:0/826887463 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55982eb663b0 con 0x55982eacf790 2024-09-10T10:00:06.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:06.473+0000 7f1840c00640 1 -- 172.21.15.5:0/826887463 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1844013040 con 0x55982eacf790 2024-09-10T10:00:06.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:06.474+0000 7f1840c00640 1 -- 172.21.15.5:0/826887463 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 42) ==== 8370+0+0 (secure 0 0 0) 0x7f184802f430 con 0x55982eacf790 2024-09-10T10:00:06.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:06.475+0000 7f1860c36740 10 client.19524 fetch_fsmap finished waiting for FSMap version 42 2024-09-10T10:00:06.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:06.475+0000 7f1860c36740 10 client.19524 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-09-10T10:00:06.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:06.475+0000 7f1860c36740 20 client.19524 populate_metadata read hostname 'smithi005' 2024-09-10T10:00:06.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:06.475+0000 7f1860c36740 1 --2- 172.21.15.5:0/826887463 >> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] conn(0x55982eb667b0 0x55982eb68bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:06.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:06.475+0000 7f1860c36740 4 client.19524 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T10:00:06.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:06.475+0000 7f1860c36740 1 -- 172.21.15.5:0/826887463 --> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55982eac69e0 con 0x55982eb667b0 2024-09-10T10:00:06.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:06.476+0000 7f1856a00640 1 --2- 172.21.15.5:0/826887463 >> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] conn(0x55982eb667b0 0x55982eb68bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:06.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:06.477+0000 7f1856a00640 1 --2- 172.21.15.5:0/826887463 >> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] conn(0x55982eb667b0 0x55982eb68bd0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=133359b553c02450 server_cookie=3cef8ad9b14c72d2 in_seq=0 out_seq=0 2024-09-10T10:00:06.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:06.477+0000 7f1840c00640 10 client.19524 ms_handle_connect on v2:172.21.15.102:6836/2689776764 2024-09-10T10:00:06.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:06.477+0000 7f1840c00640 1 -- 172.21.15.5:0/826887463 <== mds.3 v2:172.21.15.102:6836/2689776764 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55982eac69e0 con 0x55982eb667b0 2024-09-10T10:00:06.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:06.477+0000 7f1840c00640 10 client.19524 handle_command_reply: tid=0 2024-09-10T10:00:06.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:06.478+0000 7f1860c36740 1 client.19524 shutdown 2024-09-10T10:00:06.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:06.479+0000 7f1860c36740 1 -- 172.21.15.5:0/826887463 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55982eacf790 msgr2=0x55982eaf36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:06.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:06.479+0000 7f1860c36740 1 --2- 172.21.15.5:0/826887463 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55982eacf790 0x55982eaf36d0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f1848013b60 tx=0x7f1848007430 comp rx=0 tx=0).stop 2024-09-10T10:00:06.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:06.480+0000 7f1860c36740 1 -- 172.21.15.5:0/826887463 shutdown_connections 2024-09-10T10:00:06.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:06.480+0000 7f1860c36740 1 --2- 172.21.15.5:0/826887463 >> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] conn(0x55982eb667b0 0x55982eb68bd0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:06.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:06.480+0000 7f1860c36740 1 --2- 172.21.15.5:0/826887463 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55982eada050 0x55982eaee140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:06.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:06.480+0000 7f1860c36740 1 --2- 172.21.15.5:0/826887463 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55982eacf790 0x55982eaf36d0 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:06.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:06.480+0000 7f1860c36740 1 --2- 172.21.15.5:0/826887463 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55982eacee70 0x55982eaf3190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:06.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:06.480+0000 7f1860c36740 1 -- 172.21.15.5:0/826887463 >> 172.21.15.5:0/826887463 conn(0x55982eacb400 msgr2=0x55982ead8610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:00:06.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:06.480+0000 7f1840c00640 0 client.19524 ms_handle_reset on v2:172.21.15.102:6836/2689776764 2024-09-10T10:00:06.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:06.480+0000 7f1860c36740 1 -- 172.21.15.5:0/826887463 shutdown_connections 2024-09-10T10:00:06.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:06.481+0000 7f1860c36740 1 -- 172.21.15.5:0/826887463 wait complete. 2024-09-10T10:00:06.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:06.482+0000 7f1860c36740 20 client.19524 trim_cache size 0 max 16384 2024-09-10T10:00:06.483 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-10T10:00:06.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:06.483+0000 7f185ca00640 1 -- 172.21.15.5:0/3291846767 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f183c0a2c40 msgr2=0x7f183c0a5100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:06.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:06.483+0000 7f185ca00640 1 --2- 172.21.15.5:0/3291846767 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f183c0a2c40 0x7f183c0a5100 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f185001a630 tx=0x7f1850005f10 comp rx=0 tx=0).stop 2024-09-10T10:00:06.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:06.483+0000 7f185ca00640 1 -- 172.21.15.5:0/3291846767 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1858105d90 msgr2=0x7f1858114590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:06.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:06.483+0000 7f185ca00640 1 --2- 172.21.15.5:0/3291846767 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1858105d90 0x7f1858114590 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f185001acb0 tx=0x7f18500419e0 comp rx=0 tx=0).stop 2024-09-10T10:00:06.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:06.484+0000 7f185ca00640 1 -- 172.21.15.5:0/3291846767 shutdown_connections 2024-09-10T10:00:06.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:06.484+0000 7f185ca00640 1 --2- 172.21.15.5:0/3291846767 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f183c0a2c40 0x7f183c0a5100 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:06.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:06.484+0000 7f185ca00640 1 --2- 172.21.15.5:0/3291846767 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1858105d90 0x7f1858114590 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:06.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:06.484+0000 7f185ca00640 1 --2- 172.21.15.5:0/3291846767 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f18581012a0 0x7f1858111210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:06.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:06.484+0000 7f185ca00640 1 --2- 172.21.15.5:0/3291846767 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1858100980 0x7f1858110cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:06.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:06.484+0000 7f185ca00640 1 -- 172.21.15.5:0/3291846767 >> 172.21.15.5:0/3291846767 conn(0x7f18580fc610 msgr2=0x7f18580fd8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:00:06.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:06.484+0000 7f185ca00640 1 -- 172.21.15.5:0/3291846767 shutdown_connections 2024-09-10T10:00:06.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:06.484+0000 7f185ca00640 1 -- 172.21.15.5:0/3291846767 wait complete. 2024-09-10T10:00:07.030 INFO:tasks.workunit.client.0.smithi005.stdout: 10 130577 14.29 MB/sec execute 345 sec latency 5941.357 ms 2024-09-10T10:00:07.308 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:00:07 smithi005 ceph-mon[30350]: pgmap v1046: 129 pgs: 129 active+clean; 713 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 6.1 MiB/s wr, 479 op/s 2024-09-10T10:00:07.533 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:00:07 smithi159 ceph-mon[35513]: pgmap v1046: 129 pgs: 129 active+clean; 713 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 6.1 MiB/s wr, 479 op/s 2024-09-10T10:00:07.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:00:07 smithi102 ceph-mon[35442]: pgmap v1046: 129 pgs: 129 active+clean; 713 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 6.1 MiB/s wr, 479 op/s 2024-09-10T10:00:08.030 INFO:tasks.workunit.client.0.smithi005.stdout: 10 130577 14.25 MB/sec execute 346 sec latency 6941.420 ms 2024-09-10T10:00:09.030 INFO:tasks.workunit.client.0.smithi005.stdout: 10 130577 14.21 MB/sec execute 347 sec latency 7941.472 ms 2024-09-10T10:00:09.352 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:00:09 smithi005 ceph-mon[30350]: pgmap v1047: 129 pgs: 129 active+clean; 746 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 8.2 MiB/s wr, 400 op/s 2024-09-10T10:00:09.601 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:00:09 smithi159 ceph-mon[35513]: pgmap v1047: 129 pgs: 129 active+clean; 746 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 8.2 MiB/s wr, 400 op/s 2024-09-10T10:00:09.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:00:09 smithi102 ceph-mon[35442]: pgmap v1047: 129 pgs: 129 active+clean; 746 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 8.2 MiB/s wr, 400 op/s 2024-09-10T10:00:09.930 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-10T10:00:10.032 INFO:tasks.workunit.client.0.smithi005.stdout: 10 130590 14.17 MB/sec execute 348 sec latency 8076.170 ms 2024-09-10T10:00:10.583 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:00:11.032 INFO:tasks.workunit.client.0.smithi005.stdout: 10 130590 14.13 MB/sec execute 349 sec latency 5334.762 ms 2024-09-10T10:00:11.289 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:00:11 smithi005 ceph-mon[30350]: pgmap v1048: 129 pgs: 129 active+clean; 797 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 14 MiB/s wr, 540 op/s 2024-09-10T10:00:11.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:00:11 smithi159 ceph-mon[35513]: pgmap v1048: 129 pgs: 129 active+clean; 797 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 14 MiB/s wr, 540 op/s 2024-09-10T10:00:11.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:00:11 smithi102 ceph-mon[35442]: pgmap v1048: 129 pgs: 129 active+clean; 797 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 14 MiB/s wr, 540 op/s 2024-09-10T10:00:11.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:11.719+0000 7f1938200640 1 -- 172.21.15.5:0/1560047671 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1930076720 msgr2=0x7f1930076b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:11.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:11.719+0000 7f1938200640 1 --2- 172.21.15.5:0/1560047671 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1930076720 0x7f1930076b00 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f1924009900 tx=0x7f192402a420 comp rx=0 tx=0).stop 2024-09-10T10:00:11.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:11.719+0000 7f1938200640 1 -- 172.21.15.5:0/1560047671 shutdown_connections 2024-09-10T10:00:11.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:11.719+0000 7f1938200640 1 --2- 172.21.15.5:0/1560047671 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1930112a30 0x7f1930114e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:11.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:11.719+0000 7f1938200640 1 --2- 172.21.15.5:0/1560047671 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1930077040 0x7f19301124f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:11.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:11.719+0000 7f1938200640 1 --2- 172.21.15.5:0/1560047671 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1930076720 0x7f1930076b00 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:11.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:11.719+0000 7f1938200640 1 -- 172.21.15.5:0/1560047671 >> 172.21.15.5:0/1560047671 conn(0x7f19300ffa90 msgr2=0x7f1930101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:00:11.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:11.719+0000 7f1938200640 1 -- 172.21.15.5:0/1560047671 shutdown_connections 2024-09-10T10:00:11.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:11.720+0000 7f1938200640 1 -- 172.21.15.5:0/1560047671 wait complete. 2024-09-10T10:00:11.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:11.721+0000 7f1938200640 1 Processor -- start 2024-09-10T10:00:11.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:11.721+0000 7f1938200640 1 -- start start 2024-09-10T10:00:11.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:11.722+0000 7f1938200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1930076720 0x7f19301a47d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:11.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:11.722+0000 7f1938200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1930077040 0x7f19301a4d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:11.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:11.722+0000 7f1938200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1930112a30 0x7f193019e990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:11.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:11.722+0000 7f1938200640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1930118600 con 0x7f1930076720 2024-09-10T10:00:11.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:11.722+0000 7f1938200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f1930118480 con 0x7f1930112a30 2024-09-10T10:00:11.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:11.722+0000 7f1938200640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f1930118780 con 0x7f1930077040 2024-09-10T10:00:11.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:11.723+0000 7f1935800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1930076720 0x7f19301a47d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:11.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:11.723+0000 7f1935800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1930076720 0x7f19301a47d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43006/0 (socket says 172.21.15.5:43006) 2024-09-10T10:00:11.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:11.723+0000 7f1935800640 1 -- 172.21.15.5:0/3809437683 learned_addr learned my addr 172.21.15.5:0/3809437683 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T10:00:11.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:11.723+0000 7f1935800640 1 -- 172.21.15.5:0/3809437683 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1930077040 msgr2=0x7f19301a4d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:11.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:11.723+0000 7f1935800640 1 --2- 172.21.15.5:0/3809437683 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1930077040 0x7f19301a4d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:11.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:11.723+0000 7f1935800640 1 -- 172.21.15.5:0/3809437683 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1930112a30 msgr2=0x7f193019e990 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-10T10:00:11.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:11.723+0000 7f1935800640 1 --2- 172.21.15.5:0/3809437683 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1930112a30 0x7f193019e990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:11.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:11.723+0000 7f1935800640 1 -- 172.21.15.5:0/3809437683 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f193019f0c0 con 0x7f1930076720 2024-09-10T10:00:11.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:11.724+0000 7f1935800640 1 --2- 172.21.15.5:0/3809437683 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1930076720 0x7f19301a47d0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f192402a940 tx=0x7f19240042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:00:11.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:11.726+0000 7f192e000640 1 -- 172.21.15.5:0/3809437683 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f192403f030 con 0x7f1930076720 2024-09-10T10:00:11.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:11.726+0000 7f192e000640 1 -- 172.21.15.5:0/3809437683 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1924003ce0 con 0x7f1930076720 2024-09-10T10:00:11.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:11.726+0000 7f192e000640 1 -- 172.21.15.5:0/3809437683 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1924035070 con 0x7f1930076720 2024-09-10T10:00:11.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:11.726+0000 7f1938200640 1 -- 172.21.15.5:0/3809437683 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f193019f350 con 0x7f1930076720 2024-09-10T10:00:11.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:11.726+0000 7f1938200640 1 -- 172.21.15.5:0/3809437683 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19301ab6a0 con 0x7f1930076720 2024-09-10T10:00:11.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:11.734+0000 7f192e000640 1 -- 172.21.15.5:0/3809437683 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f1924042030 con 0x7f1930076720 2024-09-10T10:00:11.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:11.734+0000 7f193ac80740 1 Processor -- start 2024-09-10T10:00:11.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:11.734+0000 7f193ac80740 1 -- start start 2024-09-10T10:00:11.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:11.735+0000 7f193ac80740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555836ea3c70 0x555836ea4050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:11.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:11.735+0000 7f193ac80740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555836ea4590 0x555836eae910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:11.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:11.735+0000 7f193ac80740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555836eaee50 0x555836eb1240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:11.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:11.735+0000 7f193ac80740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x555836e0a3e0 con 0x555836eaee50 2024-09-10T10:00:11.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:11.735+0000 7f193ac80740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x555836db1d80 con 0x555836ea3c70 2024-09-10T10:00:11.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:11.735+0000 7f193ac80740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x555836e063a0 con 0x555836ea4590 2024-09-10T10:00:11.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:11.737+0000 7f1934e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555836eaee50 0x555836eb1240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:11.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:11.738+0000 7f1934e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555836eaee50 0x555836eb1240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43010/0 (socket says 172.21.15.5:43010) 2024-09-10T10:00:11.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:11.738+0000 7f1934e00640 1 -- 172.21.15.5:0/3216239565 learned_addr learned my addr 172.21.15.5:0/3216239565 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T10:00:11.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:11.738+0000 7f192e000640 1 --2- 172.21.15.5:0/3809437683 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f19180a2cd0 0x7f19180a5190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:11.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:11.739+0000 7f1936200640 1 --2- 172.21.15.5:0/3809437683 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f19180a2cd0 0x7f19180a5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:11.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:11.739+0000 7f192e000640 1 -- 172.21.15.5:0/3809437683 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f19240ef860 con 0x7f1930076720 2024-09-10T10:00:11.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:11.739+0000 7f1934e00640 1 -- 172.21.15.5:0/3216239565 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555836ea4590 msgr2=0x555836eae910 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T10:00:11.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:11.739+0000 7f1934e00640 1 --2- 172.21.15.5:0/3216239565 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555836ea4590 0x555836eae910 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:11.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:11.739+0000 7f1934e00640 1 -- 172.21.15.5:0/3216239565 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555836ea3c70 msgr2=0x555836ea4050 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T10:00:11.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:11.739+0000 7f1934e00640 1 --2- 172.21.15.5:0/3216239565 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555836ea3c70 0x555836ea4050 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:11.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:11.740+0000 7f1934e00640 1 -- 172.21.15.5:0/3216239565 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555836dd0ae0 con 0x555836eaee50 2024-09-10T10:00:11.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:11.740+0000 7f1936200640 1 --2- 172.21.15.5:0/3809437683 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f19180a2cd0 0x7f19180a5190 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f193006b460 tx=0x7f19280015d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:00:11.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:11.741+0000 7f1934e00640 1 --2- 172.21.15.5:0/3216239565 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555836eaee50 0x555836eb1240 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f191c00baa0 tx=0x7f191c00bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcc3f9bb5974f81c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:00:11.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:11.742+0000 7f1922a00640 1 -- 172.21.15.5:0/3216239565 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f191c016070 con 0x555836eaee50 2024-09-10T10:00:11.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:11.742+0000 7f1922a00640 1 -- 172.21.15.5:0/3216239565 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f191c005500 con 0x555836eaee50 2024-09-10T10:00:11.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:11.742+0000 7f1922a00640 1 -- 172.21.15.5:0/3216239565 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f191c010460 con 0x555836eaee50 2024-09-10T10:00:11.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:11.744+0000 7f193ac80740 1 -- 172.21.15.5:0/3216239565 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555836eaee50 msgr2=0x555836eb1240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:11.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:11.744+0000 7f193ac80740 1 --2- 172.21.15.5:0/3216239565 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555836eaee50 0x555836eb1240 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f191c00baa0 tx=0x7f191c00bf60 comp rx=0 tx=0).stop 2024-09-10T10:00:11.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:11.744+0000 7f193ac80740 1 -- 172.21.15.5:0/3216239565 shutdown_connections 2024-09-10T10:00:11.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:11.744+0000 7f193ac80740 1 --2- 172.21.15.5:0/3216239565 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555836eaee50 0x555836eb1240 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:11.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:11.744+0000 7f193ac80740 1 --2- 172.21.15.5:0/3216239565 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555836ea4590 0x555836eae910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:11.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:11.744+0000 7f193ac80740 1 --2- 172.21.15.5:0/3216239565 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555836ea3c70 0x555836ea4050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:11.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:11.744+0000 7f193ac80740 1 -- 172.21.15.5:0/3216239565 >> 172.21.15.5:0/3216239565 conn(0x555836ea0200 msgr2=0x555836ea05e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:00:11.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:11.744+0000 7f193ac80740 1 -- 172.21.15.5:0/3216239565 shutdown_connections 2024-09-10T10:00:11.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:11.744+0000 7f193ac80740 1 -- 172.21.15.5:0/3216239565 wait complete. 2024-09-10T10:00:11.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:11.745+0000 7f193ac80740 1 -- start start 2024-09-10T10:00:11.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:11.745+0000 7f193ac80740 1 Processor -- start 2024-09-10T10:00:11.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:11.746+0000 7f193ac80740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555836ea3c70 0x555836eabdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:11.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:11.746+0000 7f193ac80740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555836ea4590 0x555836eac310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:11.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:11.747+0000 7f193ac80740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555836eaee50 0x555836eac850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:11.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:11.747+0000 7f1934e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555836ea4590 0x555836eac310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:11.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:11.747+0000 7f193ac80740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x555836ec2a30 con 0x555836ea4590 2024-09-10T10:00:11.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:11.747+0000 7f193ac80740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x555836dc50f0 con 0x555836ea3c70 2024-09-10T10:00:11.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:11.747+0000 7f193ac80740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x555836ec2bb0 con 0x555836eaee50 2024-09-10T10:00:11.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:11.748+0000 7f1934e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555836ea4590 0x555836eac310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43024/0 (socket says 172.21.15.5:43024) 2024-09-10T10:00:11.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:11.748+0000 7f1934e00640 1 -- 172.21.15.5:0/1013139901 learned_addr learned my addr 172.21.15.5:0/1013139901 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T10:00:11.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:11.748+0000 7f193ac80740 10 client.0 _ll_register_callbacks cb 0x555836dacbd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:00:11.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:11.747+0000 7f1936200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555836eaee50 0x555836eac850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:11.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:11.749+0000 7f1934e00640 1 -- 172.21.15.5:0/1013139901 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555836eaee50 msgr2=0x555836eac850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:00:11.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:11.749+0000 7f1934e00640 1 --2- 172.21.15.5:0/1013139901 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555836eaee50 0x555836eac850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:11.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:11.749+0000 7f1934e00640 1 -- 172.21.15.5:0/1013139901 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555836ea3c70 msgr2=0x555836eabdd0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T10:00:11.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:11.749+0000 7f1934e00640 1 --2- 172.21.15.5:0/1013139901 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555836ea3c70 0x555836eabdd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:11.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:11.749+0000 7f1934e00640 1 -- 172.21.15.5:0/1013139901 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555836b7b3c0 con 0x555836ea4590 2024-09-10T10:00:11.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:11.749+0000 7f1934e00640 1 --2- 172.21.15.5:0/1013139901 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555836ea4590 0x555836eac310 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f191c004b30 tx=0x7f191c004ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a7e0a2b27a6207 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:00:11.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:11.749+0000 7f1917400640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-10T10:00:11.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:11.750+0000 7f1917400640 1 -- 172.21.15.5:0/1013139901 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f191c016070 con 0x555836ea4590 2024-09-10T10:00:11.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:11.750+0000 7f193ac80740 1 -- 172.21.15.5:0/1013139901 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f191c005500 con 0x555836ea4590 2024-09-10T10:00:11.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:11.750+0000 7f1917400640 1 -- 172.21.15.5:0/1013139901 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f191c00da70 con 0x555836ea4590 2024-09-10T10:00:11.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:11.751+0000 7f1917400640 1 -- 172.21.15.5:0/1013139901 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f191c02d050 con 0x555836ea4590 2024-09-10T10:00:11.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:11.751+0000 7f193ac80740 10 client.19533 fetch_fsmap learned FSMap version 42 2024-09-10T10:00:11.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:11.751+0000 7f193ac80740 1 -- 172.21.15.5:0/1013139901 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x555836eceaf0 con 0x555836ea4590 2024-09-10T10:00:11.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:11.752+0000 7f1917400640 1 -- 172.21.15.5:0/1013139901 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f191c005500 con 0x555836ea4590 2024-09-10T10:00:11.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:11.752+0000 7f1917400640 1 -- 172.21.15.5:0/1013139901 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 42) ==== 8370+0+0 (secure 0 0 0) 0x7f191c022020 con 0x555836ea4590 2024-09-10T10:00:11.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:11.752+0000 7f193ac80740 10 client.19533 fetch_fsmap finished waiting for FSMap version 42 2024-09-10T10:00:11.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:11.752+0000 7f193ac80740 10 client.19533 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-10T10:00:11.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:11.752+0000 7f193ac80740 20 client.19533 populate_metadata read hostname 'smithi005' 2024-09-10T10:00:11.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:11.752+0000 7f193ac80740 1 --2- 172.21.15.5:0/1013139901 >> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] conn(0x555836ecff00 0x555836ed22f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:11.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:11.752+0000 7f193ac80740 4 client.19533 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:00:11.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:11.752+0000 7f193ac80740 1 -- 172.21.15.5:0/1013139901 --> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555836e17950 con 0x555836ecff00 2024-09-10T10:00:11.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:11.753+0000 7f1935800640 1 --2- 172.21.15.5:0/1013139901 >> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] conn(0x555836ecff00 0x555836ed22f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:11.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:11.754+0000 7f1935800640 1 --2- 172.21.15.5:0/1013139901 >> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] conn(0x555836ecff00 0x555836ed22f0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=729011bf56bbdd9d server_cookie=9d17fc2c2efc170c in_seq=0 out_seq=0 2024-09-10T10:00:11.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:11.754+0000 7f1917400640 10 client.19533 ms_handle_connect on v2:172.21.15.159:6836/620627639 2024-09-10T10:00:11.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:11.755+0000 7f1917400640 1 -- 172.21.15.5:0/1013139901 <== mds.2 v2:172.21.15.159:6836/620627639 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555836e17950 con 0x555836ecff00 2024-09-10T10:00:11.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:11.755+0000 7f1917400640 10 client.19533 handle_command_reply: tid=0 2024-09-10T10:00:11.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:11.755+0000 7f193ac80740 1 client.19533 shutdown 2024-09-10T10:00:11.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:11.755+0000 7f193ac80740 1 -- 172.21.15.5:0/1013139901 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555836ea4590 msgr2=0x555836eac310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:11.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:11.755+0000 7f193ac80740 1 --2- 172.21.15.5:0/1013139901 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555836ea4590 0x555836eac310 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f191c004b30 tx=0x7f191c004ec0 comp rx=0 tx=0).stop 2024-09-10T10:00:11.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:11.756+0000 7f193ac80740 1 -- 172.21.15.5:0/1013139901 shutdown_connections 2024-09-10T10:00:11.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:11.756+0000 7f193ac80740 1 --2- 172.21.15.5:0/1013139901 >> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] conn(0x555836ecff00 0x555836ed22f0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:11.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:11.756+0000 7f193ac80740 1 --2- 172.21.15.5:0/1013139901 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555836eaee50 0x555836eac850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:11.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:11.756+0000 7f193ac80740 1 --2- 172.21.15.5:0/1013139901 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555836ea4590 0x555836eac310 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:11.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:11.756+0000 7f1917400640 0 client.19533 ms_handle_reset on v2:172.21.15.159:6836/620627639 2024-09-10T10:00:11.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:11.756+0000 7f193ac80740 1 --2- 172.21.15.5:0/1013139901 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555836ea3c70 0x555836eabdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:11.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:11.756+0000 7f193ac80740 1 -- 172.21.15.5:0/1013139901 >> 172.21.15.5:0/1013139901 conn(0x555836ea0200 msgr2=0x555836ea5ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:00:11.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:11.757+0000 7f193ac80740 1 -- 172.21.15.5:0/1013139901 shutdown_connections 2024-09-10T10:00:11.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:11.757+0000 7f193ac80740 1 -- 172.21.15.5:0/1013139901 wait complete. 2024-09-10T10:00:11.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:11.757+0000 7f193ac80740 20 client.19533 trim_cache size 0 max 16384 2024-09-10T10:00:11.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:11.783+0000 7f193ac80740 1 Processor -- start 2024-09-10T10:00:11.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:11.783+0000 7f193ac80740 1 -- start start 2024-09-10T10:00:11.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:11.783+0000 7f193ac80740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555836ea3c70 0x555836eac6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:11.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:11.783+0000 7f193ac80740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555836ea4590 0x555836ed7ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:11.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:11.783+0000 7f193ac80740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555836eaee50 0x555836ed7ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:11.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:11.783+0000 7f193ac80740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f19301accf0 con 0x555836ea3c70 2024-09-10T10:00:11.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:11.783+0000 7f193ac80740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f19301ae830 con 0x555836ea4590 2024-09-10T10:00:11.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:11.783+0000 7f193ac80740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f19301ad3c0 con 0x555836eaee50 2024-09-10T10:00:11.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:11.784+0000 7f1936200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555836eaee50 0x555836ed7ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:11.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:11.784+0000 7f1936200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555836eaee50 0x555836ed7ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:40426/0 (socket says 172.21.15.5:40426) 2024-09-10T10:00:11.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:11.784+0000 7f1936200640 1 -- 172.21.15.5:0/1648614282 learned_addr learned my addr 172.21.15.5:0/1648614282 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T10:00:11.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:11.784+0000 7f1935800640 1 --2- 172.21.15.5:0/1648614282 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555836ea3c70 0x555836eac6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:11.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:11.784+0000 7f1936200640 1 -- 172.21.15.5:0/1648614282 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555836ea4590 msgr2=0x555836ed7ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:00:11.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:11.784+0000 7f1936200640 1 --2- 172.21.15.5:0/1648614282 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555836ea4590 0x555836ed7ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:11.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:11.784+0000 7f1936200640 1 -- 172.21.15.5:0/1648614282 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555836ea3c70 msgr2=0x555836eac6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:00:11.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:11.784+0000 7f1936200640 1 --2- 172.21.15.5:0/1648614282 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555836ea3c70 0x555836eac6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:11.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:11.784+0000 7f1936200640 1 -- 172.21.15.5:0/1648614282 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555836ed2880 con 0x555836eaee50 2024-09-10T10:00:11.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:11.786+0000 7f1936200640 1 --2- 172.21.15.5:0/1648614282 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555836eaee50 0x555836ed7ff0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f192800cf80 tx=0x7f1928012000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bb2296844b081cf2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:00:11.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:11.787+0000 7f1922000640 1 -- 172.21.15.5:0/1648614282 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1928014620 con 0x555836eaee50 2024-09-10T10:00:11.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:11.787+0000 7f1922000640 1 -- 172.21.15.5:0/1648614282 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1928025070 con 0x555836eaee50 2024-09-10T10:00:11.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:11.787+0000 7f1922000640 1 -- 172.21.15.5:0/1648614282 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1928020a60 con 0x555836eaee50 2024-09-10T10:00:11.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:11.789+0000 7f193ac80740 1 -- 172.21.15.5:0/1648614282 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555836eaee50 msgr2=0x555836ed7ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:11.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:11.789+0000 7f193ac80740 1 --2- 172.21.15.5:0/1648614282 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555836eaee50 0x555836ed7ff0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f192800cf80 tx=0x7f1928012000 comp rx=0 tx=0).stop 2024-09-10T10:00:11.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:11.789+0000 7f193ac80740 1 -- 172.21.15.5:0/1648614282 shutdown_connections 2024-09-10T10:00:11.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:11.789+0000 7f193ac80740 1 --2- 172.21.15.5:0/1648614282 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555836eaee50 0x555836ed7ff0 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:11.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:11.789+0000 7f193ac80740 1 --2- 172.21.15.5:0/1648614282 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555836ea4590 0x555836ed7ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:11.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:11.789+0000 7f193ac80740 1 --2- 172.21.15.5:0/1648614282 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555836ea3c70 0x555836eac6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:11.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:11.789+0000 7f193ac80740 1 -- 172.21.15.5:0/1648614282 >> 172.21.15.5:0/1648614282 conn(0x555836ea0200 msgr2=0x555836eade50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:00:11.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:11.790+0000 7f193ac80740 1 -- 172.21.15.5:0/1648614282 shutdown_connections 2024-09-10T10:00:11.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:11.790+0000 7f193ac80740 1 -- 172.21.15.5:0/1648614282 wait complete. 2024-09-10T10:00:11.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:11.790+0000 7f193ac80740 1 -- start start 2024-09-10T10:00:11.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:11.791+0000 7f193ac80740 1 Processor -- start 2024-09-10T10:00:11.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:11.792+0000 7f193ac80740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555836ea3c70 0x555836ec3d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:11.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:11.792+0000 7f193ac80740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555836ea4590 0x555836ec4260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:11.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:11.792+0000 7f193ac80740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555836eaee50 0x555836ec47a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:11.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:11.792+0000 7f1935800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555836ea3c70 0x555836ec3d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:11.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:11.792+0000 7f1935800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555836ea3c70 0x555836ec3d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43048/0 (socket says 172.21.15.5:43048) 2024-09-10T10:00:11.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:11.793+0000 7f1935800640 1 -- 172.21.15.5:0/364836315 learned_addr learned my addr 172.21.15.5:0/364836315 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T10:00:11.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:11.793+0000 7f1936200640 1 --2- 172.21.15.5:0/364836315 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555836eaee50 0x555836ec47a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:11.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:11.793+0000 7f193ac80740 1 -- 172.21.15.5:0/364836315 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f19301ae160 con 0x555836ea3c70 2024-09-10T10:00:11.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:11.793+0000 7f193ac80740 1 -- 172.21.15.5:0/364836315 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f19301ada90 con 0x555836eaee50 2024-09-10T10:00:11.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:11.793+0000 7f193ac80740 1 -- 172.21.15.5:0/364836315 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f193010d250 con 0x555836ea4590 2024-09-10T10:00:11.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:11.793+0000 7f1934e00640 1 --2- 172.21.15.5:0/364836315 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555836ea4590 0x555836ec4260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:11.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:11.794+0000 7f193ac80740 10 client.0 _ll_register_callbacks cb 0x7f191c00b8f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:00:11.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:11.794+0000 7f1936200640 1 -- 172.21.15.5:0/364836315 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555836ea4590 msgr2=0x555836ec4260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:00:11.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:11.794+0000 7f1936200640 1 --2- 172.21.15.5:0/364836315 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555836ea4590 0x555836ec4260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:11.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:11.794+0000 7f1936200640 1 -- 172.21.15.5:0/364836315 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555836ea3c70 msgr2=0x555836ec3d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:00:11.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:11.794+0000 7f1936200640 1 --2- 172.21.15.5:0/364836315 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555836ea3c70 0x555836ec3d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:11.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:11.794+0000 7f1936200640 1 -- 172.21.15.5:0/364836315 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555836ea0e90 con 0x555836eaee50 2024-09-10T10:00:11.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:11.795+0000 7f1917400640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-09-10T10:00:11.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:11.795+0000 7f1936200640 1 --2- 172.21.15.5:0/364836315 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555836eaee50 0x555836ec47a0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f1928020680 tx=0x7f1928020a10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=25c0bf8f8bbe7801 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:00:11.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:11.796+0000 7f1917400640 1 -- 172.21.15.5:0/364836315 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1928014620 con 0x555836eaee50 2024-09-10T10:00:11.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:11.796+0000 7f1917400640 1 -- 172.21.15.5:0/364836315 <== mon.2 v2:172.21.15.102:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f19280147c0 con 0x555836eaee50 2024-09-10T10:00:11.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:11.797+0000 7f193ac80740 1 -- 172.21.15.5:0/364836315 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1928025070 con 0x555836eaee50 2024-09-10T10:00:11.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:11.797+0000 7f1917400640 1 -- 172.21.15.5:0/364836315 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f1928020e10 con 0x555836eaee50 2024-09-10T10:00:11.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:11.797+0000 7f193ac80740 10 client.29609 fetch_fsmap learned FSMap version 42 2024-09-10T10:00:11.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:11.797+0000 7f193ac80740 1 -- 172.21.15.5:0/364836315 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x555836e9b770 con 0x555836eaee50 2024-09-10T10:00:11.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:11.798+0000 7f1917400640 1 -- 172.21.15.5:0/364836315 <== mon.2 v2:172.21.15.102:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1928025070 con 0x555836eaee50 2024-09-10T10:00:11.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:11.798+0000 7f1917400640 1 -- 172.21.15.5:0/364836315 <== mon.2 v2:172.21.15.102:3300/0 5 ==== fsmap(e 42) ==== 8370+0+0 (secure 0 0 0) 0x7f192803d940 con 0x555836eaee50 2024-09-10T10:00:11.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:11.799+0000 7f193ac80740 10 client.29609 fetch_fsmap finished waiting for FSMap version 42 2024-09-10T10:00:11.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:11.799+0000 7f193ac80740 10 client.29609 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-10T10:00:11.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:11.799+0000 7f193ac80740 20 client.29609 populate_metadata read hostname 'smithi005' 2024-09-10T10:00:11.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:11.799+0000 7f193ac80740 1 --2- 172.21.15.5:0/364836315 >> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] conn(0x555836f3b190 0x555836f3d580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:11.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:11.800+0000 7f193ac80740 4 client.29609 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T10:00:11.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:11.800+0000 7f193ac80740 1 -- 172.21.15.5:0/364836315 --> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555836e11e60 con 0x555836f3b190 2024-09-10T10:00:11.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:11.800+0000 7f1935800640 1 --2- 172.21.15.5:0/364836315 >> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] conn(0x555836f3b190 0x555836f3d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:11.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:11.801+0000 7f1935800640 1 --2- 172.21.15.5:0/364836315 >> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] conn(0x555836f3b190 0x555836f3d580 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6b4eba00cdf1bbf5 server_cookie=845622bc54ae91a1 in_seq=0 out_seq=0 2024-09-10T10:00:11.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:11.801+0000 7f1917400640 10 client.29609 ms_handle_connect on v2:172.21.15.159:6836/620627639 2024-09-10T10:00:11.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:11.801+0000 7f1917400640 1 -- 172.21.15.5:0/364836315 <== mds.2 v2:172.21.15.159:6836/620627639 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555836e11e60 con 0x555836f3b190 2024-09-10T10:00:11.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:11.801+0000 7f1917400640 10 client.29609 handle_command_reply: tid=0 2024-09-10T10:00:11.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:11.802+0000 7f193ac80740 1 client.29609 shutdown 2024-09-10T10:00:11.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:11.803+0000 7f193ac80740 1 -- 172.21.15.5:0/364836315 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555836eaee50 msgr2=0x555836ec47a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:11.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:11.803+0000 7f193ac80740 1 --2- 172.21.15.5:0/364836315 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555836eaee50 0x555836ec47a0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f1928020680 tx=0x7f1928020a10 comp rx=0 tx=0).stop 2024-09-10T10:00:11.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:11.803+0000 7f193ac80740 1 -- 172.21.15.5:0/364836315 shutdown_connections 2024-09-10T10:00:11.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:11.803+0000 7f193ac80740 1 --2- 172.21.15.5:0/364836315 >> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] conn(0x555836f3b190 0x555836f3d580 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:11.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:11.803+0000 7f193ac80740 1 --2- 172.21.15.5:0/364836315 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x555836eaee50 0x555836ec47a0 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:11.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:11.803+0000 7f193ac80740 1 --2- 172.21.15.5:0/364836315 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x555836ea4590 0x555836ec4260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:11.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:11.803+0000 7f193ac80740 1 --2- 172.21.15.5:0/364836315 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555836ea3c70 0x555836ec3d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:11.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:11.803+0000 7f1917400640 0 client.29609 ms_handle_reset on v2:172.21.15.159:6836/620627639 2024-09-10T10:00:11.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:11.803+0000 7f193ac80740 1 -- 172.21.15.5:0/364836315 >> 172.21.15.5:0/364836315 conn(0x555836ea0200 msgr2=0x555836ea56c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:00:11.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:11.804+0000 7f193ac80740 1 -- 172.21.15.5:0/364836315 shutdown_connections 2024-09-10T10:00:11.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:11.804+0000 7f193ac80740 1 -- 172.21.15.5:0/364836315 wait complete. 2024-09-10T10:00:11.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:11.804+0000 7f193ac80740 20 client.29609 trim_cache size 0 max 16384 2024-09-10T10:00:11.807 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-10T10:00:11.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:11.805+0000 7f1938200640 1 -- 172.21.15.5:0/3809437683 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f19180a2cd0 msgr2=0x7f19180a5190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:11.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:11.805+0000 7f1938200640 1 --2- 172.21.15.5:0/3809437683 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f19180a2cd0 0x7f19180a5190 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f193006b460 tx=0x7f19280015d0 comp rx=0 tx=0).stop 2024-09-10T10:00:11.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:11.805+0000 7f1938200640 1 -- 172.21.15.5:0/3809437683 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1930076720 msgr2=0x7f19301a47d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:11.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:11.805+0000 7f1938200640 1 --2- 172.21.15.5:0/3809437683 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1930076720 0x7f19301a47d0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f192402a940 tx=0x7f19240042f0 comp rx=0 tx=0).stop 2024-09-10T10:00:11.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:11.805+0000 7f1938200640 1 -- 172.21.15.5:0/3809437683 shutdown_connections 2024-09-10T10:00:11.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:11.805+0000 7f1938200640 1 --2- 172.21.15.5:0/3809437683 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f19180a2cd0 0x7f19180a5190 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:11.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:11.805+0000 7f1938200640 1 --2- 172.21.15.5:0/3809437683 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1930112a30 0x7f193019e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:11.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:11.805+0000 7f1938200640 1 --2- 172.21.15.5:0/3809437683 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1930077040 0x7f19301a4d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:11.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:11.805+0000 7f1938200640 1 --2- 172.21.15.5:0/3809437683 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1930076720 0x7f19301a47d0 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:11.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:11.806+0000 7f1938200640 1 -- 172.21.15.5:0/3809437683 >> 172.21.15.5:0/3809437683 conn(0x7f19300ffa90 msgr2=0x7f1930101e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:00:11.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:11.807+0000 7f1938200640 1 -- 172.21.15.5:0/3809437683 shutdown_connections 2024-09-10T10:00:11.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:11.807+0000 7f1938200640 1 -- 172.21.15.5:0/3809437683 wait complete. 2024-09-10T10:00:12.033 INFO:tasks.workunit.client.0.smithi005.stdout: 10 130591 14.09 MB/sec execute 350 sec latency 6335.957 ms 2024-09-10T10:00:13.033 INFO:tasks.workunit.client.0.smithi005.stdout: 10 130665 14.06 MB/sec execute 351 sec latency 6500.883 ms 2024-09-10T10:00:13.557 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:00:13 smithi005 ceph-mon[30350]: pgmap v1049: 129 pgs: 129 active+clean; 779 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 14 MiB/s wr, 414 op/s 2024-09-10T10:00:13.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:00:13 smithi159 ceph-mon[35513]: pgmap v1049: 129 pgs: 129 active+clean; 779 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 14 MiB/s wr, 414 op/s 2024-09-10T10:00:13.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:00:13 smithi102 ceph-mon[35442]: pgmap v1049: 129 pgs: 129 active+clean; 779 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 14 MiB/s wr, 414 op/s 2024-09-10T10:00:14.033 INFO:tasks.workunit.client.0.smithi005.stdout: 10 130665 14.02 MB/sec execute 352 sec latency 4713.673 ms 2024-09-10T10:00:15.033 INFO:tasks.workunit.client.0.smithi005.stdout: 10 130665 13.98 MB/sec execute 353 sec latency 5713.732 ms 2024-09-10T10:00:16.033 INFO:tasks.workunit.client.0.smithi005.stdout: 10 130665 13.94 MB/sec execute 354 sec latency 6713.794 ms 2024-09-10T10:00:16.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:00:15 smithi102 ceph-mon[35442]: pgmap v1050: 129 pgs: 129 active+clean; 750 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 944 KiB/s rd, 13 MiB/s wr, 429 op/s 2024-09-10T10:00:16.306 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:00:15 smithi005 ceph-mon[30350]: pgmap v1050: 129 pgs: 129 active+clean; 750 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 944 KiB/s rd, 13 MiB/s wr, 429 op/s 2024-09-10T10:00:16.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:00:15 smithi159 ceph-mon[35513]: pgmap v1050: 129 pgs: 129 active+clean; 750 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 944 KiB/s rd, 13 MiB/s wr, 429 op/s 2024-09-10T10:00:17.033 INFO:tasks.workunit.client.0.smithi005.stdout: 10 130665 13.90 MB/sec execute 355 sec latency 7713.856 ms 2024-09-10T10:00:18.033 INFO:tasks.workunit.client.0.smithi005.stdout: 10 130665 13.87 MB/sec execute 356 sec latency 8713.912 ms 2024-09-10T10:00:18.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:00:17 smithi102 ceph-mon[35442]: pgmap v1051: 129 pgs: 129 active+clean; 709 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 934 KiB/s rd, 12 MiB/s wr, 474 op/s 2024-09-10T10:00:18.306 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:00:17 smithi005 ceph-mon[30350]: pgmap v1051: 129 pgs: 129 active+clean; 709 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 934 KiB/s rd, 12 MiB/s wr, 474 op/s 2024-09-10T10:00:18.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:00:17 smithi159 ceph-mon[35513]: pgmap v1051: 129 pgs: 129 active+clean; 709 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 934 KiB/s rd, 12 MiB/s wr, 474 op/s 2024-09-10T10:00:19.033 INFO:tasks.workunit.client.0.smithi005.stdout: 10 130665 13.83 MB/sec execute 357 sec latency 9713.973 ms 2024-09-10T10:00:20.034 INFO:tasks.workunit.client.0.smithi005.stdout: 10 130665 13.79 MB/sec execute 358 sec latency 10714.035 ms 2024-09-10T10:00:20.306 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:00:19 smithi005 ceph-mon[30350]: pgmap v1052: 129 pgs: 129 active+clean; 690 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 640 KiB/s rd, 11 MiB/s wr, 431 op/s 2024-09-10T10:00:20.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:00:19 smithi159 ceph-mon[35513]: pgmap v1052: 129 pgs: 129 active+clean; 690 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 640 KiB/s rd, 11 MiB/s wr, 431 op/s 2024-09-10T10:00:20.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:00:19 smithi102 ceph-mon[35442]: pgmap v1052: 129 pgs: 129 active+clean; 690 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 640 KiB/s rd, 11 MiB/s wr, 431 op/s 2024-09-10T10:00:21.033 INFO:tasks.workunit.client.0.smithi005.stdout: 10 130665 13.75 MB/sec execute 359 sec latency 11714.097 ms 2024-09-10T10:00:21.306 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:00:21 smithi005 ceph-mon[30350]: pgmap v1053: 129 pgs: 129 active+clean; 689 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 660 KiB/s rd, 7.8 MiB/s wr, 377 op/s 2024-09-10T10:00:21.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:00:21 smithi159 ceph-mon[35513]: pgmap v1053: 129 pgs: 129 active+clean; 689 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 660 KiB/s rd, 7.8 MiB/s wr, 377 op/s 2024-09-10T10:00:21.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:00:21 smithi102 ceph-mon[35442]: pgmap v1053: 129 pgs: 129 active+clean; 689 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 660 KiB/s rd, 7.8 MiB/s wr, 377 op/s 2024-09-10T10:00:22.033 INFO:tasks.workunit.client.0.smithi005.stdout: 10 130665 13.71 MB/sec execute 360 sec latency 12714.159 ms 2024-09-10T10:00:23.034 INFO:tasks.workunit.client.0.smithi005.stdout: 10 130665 13.67 MB/sec execute 361 sec latency 13714.222 ms 2024-09-10T10:00:23.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:00:23 smithi005 ceph-mon[30350]: pgmap v1054: 129 pgs: 129 active+clean; 689 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 508 KiB/s rd, 1.4 MiB/s wr, 213 op/s 2024-09-10T10:00:23.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:00:23 smithi159 ceph-mon[35513]: pgmap v1054: 129 pgs: 129 active+clean; 689 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 508 KiB/s rd, 1.4 MiB/s wr, 213 op/s 2024-09-10T10:00:23.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:00:23 smithi102 ceph-mon[35442]: pgmap v1054: 129 pgs: 129 active+clean; 689 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 508 KiB/s rd, 1.4 MiB/s wr, 213 op/s 2024-09-10T10:00:24.034 INFO:tasks.workunit.client.0.smithi005.stdout: 10 130665 13.64 MB/sec execute 362 sec latency 14714.294 ms 2024-09-10T10:00:25.034 INFO:tasks.workunit.client.0.smithi005.stdout: 10 130665 13.60 MB/sec execute 363 sec latency 15714.335 ms 2024-09-10T10:00:25.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:00:25 smithi005 ceph-mon[30350]: pgmap v1055: 129 pgs: 129 active+clean; 689 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 279 KiB/s rd, 439 KiB/s wr, 155 op/s 2024-09-10T10:00:25.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:00:25 smithi159 ceph-mon[35513]: pgmap v1055: 129 pgs: 129 active+clean; 689 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 279 KiB/s rd, 439 KiB/s wr, 155 op/s 2024-09-10T10:00:25.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:00:25 smithi102 ceph-mon[35442]: pgmap v1055: 129 pgs: 129 active+clean; 689 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 279 KiB/s rd, 439 KiB/s wr, 155 op/s 2024-09-10T10:00:26.034 INFO:tasks.workunit.client.0.smithi005.stdout: 10 130665 13.56 MB/sec execute 364 sec latency 16714.378 ms 2024-09-10T10:00:27.034 INFO:tasks.workunit.client.0.smithi005.stdout: 10 130665 13.52 MB/sec execute 365 sec latency 17714.424 ms 2024-09-10T10:00:27.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:00:27 smithi005 ceph-mon[30350]: pgmap v1056: 129 pgs: 129 active+clean; 689 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 281 KiB/s rd, 190 KiB/s wr, 111 op/s 2024-09-10T10:00:27.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:00:27 smithi159 ceph-mon[35513]: pgmap v1056: 129 pgs: 129 active+clean; 689 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 281 KiB/s rd, 190 KiB/s wr, 111 op/s 2024-09-10T10:00:27.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:00:27 smithi102 ceph-mon[35442]: pgmap v1056: 129 pgs: 129 active+clean; 689 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 281 KiB/s rd, 190 KiB/s wr, 111 op/s 2024-09-10T10:00:28.034 INFO:tasks.workunit.client.0.smithi005.stdout: 10 130665 13.49 MB/sec execute 366 sec latency 18714.465 ms 2024-09-10T10:00:29.034 INFO:tasks.workunit.client.0.smithi005.stdout: 10 130665 13.45 MB/sec execute 367 sec latency 19714.508 ms 2024-09-10T10:00:29.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:00:29 smithi005 ceph-mon[30350]: pgmap v1057: 129 pgs: 129 active+clean; 689 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 37 KiB/s wr, 40 op/s 2024-09-10T10:00:29.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:00:29 smithi159 ceph-mon[35513]: pgmap v1057: 129 pgs: 129 active+clean; 689 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 37 KiB/s wr, 40 op/s 2024-09-10T10:00:29.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:00:29 smithi102 ceph-mon[35442]: pgmap v1057: 129 pgs: 129 active+clean; 689 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 37 KiB/s wr, 40 op/s 2024-09-10T10:00:30.034 INFO:tasks.workunit.client.0.smithi005.stdout: 10 130665 13.41 MB/sec execute 368 sec latency 20714.553 ms 2024-09-10T10:00:31.034 INFO:tasks.workunit.client.0.smithi005.stdout: 10 130687 13.39 MB/sec execute 369 sec latency 21714.595 ms 2024-09-10T10:00:31.712 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-10T10:00:32.034 INFO:tasks.workunit.client.0.smithi005.stdout: 10 131083 13.43 MB/sec execute 370 sec latency 22714.650 ms 2024-09-10T10:00:32.049 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:00:31 smithi005 ceph-mon[30350]: pgmap v1058: 129 pgs: 129 active+clean; 689 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 22 KiB/s wr, 18 op/s 2024-09-10T10:00:32.049 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:00:31 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:00:32.131 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 34.2 2024-09-10T10:00:32.132 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T10:00:32.205 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:00:32.336 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:00:31 smithi102 ceph-mon[35442]: pgmap v1058: 129 pgs: 129 active+clean; 689 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 22 KiB/s wr, 18 op/s 2024-09-10T10:00:32.337 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:00:31 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:00:32.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:00:31 smithi159 ceph-mon[35513]: pgmap v1058: 129 pgs: 129 active+clean; 689 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 22 KiB/s wr, 18 op/s 2024-09-10T10:00:32.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:00:31 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:00:32.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:32.848+0000 7f9ce6a00640 1 -- 172.21.15.5:0/1777181623 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9ce0103e30 msgr2=0x7f9ce0104210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:32.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:32.848+0000 7f9ce6a00640 1 --2- 172.21.15.5:0/1777181623 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9ce0103e30 0x7f9ce0104210 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f9cd4009900 tx=0x7f9cd402a3f0 comp rx=0 tx=0).stop 2024-09-10T10:00:32.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:32.848+0000 7f9ce6a00640 1 -- 172.21.15.5:0/1777181623 shutdown_connections 2024-09-10T10:00:32.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:32.848+0000 7f9ce6a00640 1 --2- 172.21.15.5:0/1777181623 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9ce0108f60 0x7f9ce0110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:32.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:32.848+0000 7f9ce6a00640 1 --2- 172.21.15.5:0/1777181623 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9ce01047e0 0x7f9ce0108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:32.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:32.848+0000 7f9ce6a00640 1 --2- 172.21.15.5:0/1777181623 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9ce0103e30 0x7f9ce0104210 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:32.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:32.848+0000 7f9ce6a00640 1 -- 172.21.15.5:0/1777181623 >> 172.21.15.5:0/1777181623 conn(0x7f9ce00ff910 msgr2=0x7f9ce0101d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:00:32.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:32.848+0000 7f9ce6a00640 1 -- 172.21.15.5:0/1777181623 shutdown_connections 2024-09-10T10:00:32.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:32.848+0000 7f9ce6a00640 1 -- 172.21.15.5:0/1777181623 wait complete. 2024-09-10T10:00:32.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:32.849+0000 7f9ce6a00640 1 Processor -- start 2024-09-10T10:00:32.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:32.849+0000 7f9ce6a00640 1 -- start start 2024-09-10T10:00:32.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:32.850+0000 7f9ce6a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9ce0103e30 0x7f9ce019a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:32.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:32.850+0000 7f9ce6a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9ce01047e0 0x7f9ce019a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:32.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:32.850+0000 7f9ce6a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9ce0108f60 0x7f9ce019ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:32.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:32.850+0000 7f9ce6a00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f9ce0114010 con 0x7f9ce0103e30 2024-09-10T10:00:32.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:32.850+0000 7f9ce6a00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f9ce0113e90 con 0x7f9ce01047e0 2024-09-10T10:00:32.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:32.850+0000 7f9ce6a00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f9ce0114190 con 0x7f9ce0108f60 2024-09-10T10:00:32.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:32.851+0000 7f9ce5600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9ce0103e30 0x7f9ce019a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:32.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:32.851+0000 7f9ce6000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9ce0108f60 0x7f9ce019ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:32.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:32.851+0000 7f9ce6000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9ce0108f60 0x7f9ce019ed50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:34428/0 (socket says 172.21.15.5:34428) 2024-09-10T10:00:32.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:32.851+0000 7f9ce6000640 1 -- 172.21.15.5:0/3294988383 learned_addr learned my addr 172.21.15.5:0/3294988383 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T10:00:32.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:32.851+0000 7f9ce6000640 1 -- 172.21.15.5:0/3294988383 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9ce01047e0 msgr2=0x7f9ce019a9c0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-10T10:00:32.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:32.851+0000 7f9ce6000640 1 --2- 172.21.15.5:0/3294988383 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9ce01047e0 0x7f9ce019a9c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:32.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:32.851+0000 7f9ce6000640 1 -- 172.21.15.5:0/3294988383 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9ce0103e30 msgr2=0x7f9ce019a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:32.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:32.851+0000 7f9ce6000640 1 --2- 172.21.15.5:0/3294988383 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9ce0103e30 0x7f9ce019a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:32.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:32.851+0000 7f9ce6000640 1 -- 172.21.15.5:0/3294988383 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ce019f430 con 0x7f9ce0108f60 2024-09-10T10:00:32.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:32.852+0000 7f9ce6000640 1 --2- 172.21.15.5:0/3294988383 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9ce0108f60 0x7f9ce019ed50 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f9cd8009fd0 tx=0x7f9cd800b260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:00:32.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:32.852+0000 7f9cde000640 1 -- 172.21.15.5:0/3294988383 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cd8017070 con 0x7f9ce0108f60 2024-09-10T10:00:32.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:32.852+0000 7f9ce6a00640 1 -- 172.21.15.5:0/3294988383 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ce019f720 con 0x7f9ce0108f60 2024-09-10T10:00:32.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:32.852+0000 7f9cde000640 1 -- 172.21.15.5:0/3294988383 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9cd8003e50 con 0x7f9ce0108f60 2024-09-10T10:00:32.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:32.852+0000 7f9cde000640 1 -- 172.21.15.5:0/3294988383 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cd80106c0 con 0x7f9ce0108f60 2024-09-10T10:00:32.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:32.852+0000 7f9ce6a00640 1 -- 172.21.15.5:0/3294988383 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ce01a7000 con 0x7f9ce0108f60 2024-09-10T10:00:32.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:32.856+0000 7f9cde000640 1 -- 172.21.15.5:0/3294988383 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f9cd8003970 con 0x7f9ce0108f60 2024-09-10T10:00:32.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:32.856+0000 7f9ceaa9d740 1 Processor -- start 2024-09-10T10:00:32.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:32.856+0000 7f9ceaa9d740 1 -- start start 2024-09-10T10:00:32.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:32.857+0000 7f9ceaa9d740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56403686fe40 0x564036870220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:32.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:32.857+0000 7f9ceaa9d740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564036870760 0x56403687aae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:32.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:32.857+0000 7f9ceaa9d740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56403687b020 0x56403687d410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:32.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:32.857+0000 7f9ceaa9d740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5640367d63e0 con 0x564036870760 2024-09-10T10:00:32.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:32.857+0000 7f9ceaa9d740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x56403677dd80 con 0x56403686fe40 2024-09-10T10:00:32.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:32.857+0000 7f9ceaa9d740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5640367d23a0 con 0x56403687b020 2024-09-10T10:00:32.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:32.857+0000 7f9ce4c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56403686fe40 0x564036870220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:32.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:32.857+0000 7f9ce5600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564036870760 0x56403687aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:32.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:32.857+0000 7f9ce4c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56403686fe40 0x564036870220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:34096/0 (socket says 172.21.15.5:34096) 2024-09-10T10:00:32.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:32.857+0000 7f9ce4c00640 1 -- 172.21.15.5:0/256671427 learned_addr learned my addr 172.21.15.5:0/256671427 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T10:00:32.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:32.857+0000 7f9ce4c00640 1 --2- 172.21.15.5:0/256671427 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56403687b020 0x56403687d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:32.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:32.857+0000 7f9cde000640 1 --2- 172.21.15.5:0/3294988383 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f9cc80a2b80 0x7f9cc80a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:32.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:32.858+0000 7f9cde000640 1 -- 172.21.15.5:0/3294988383 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f9cd80cf200 con 0x7f9ce0108f60 2024-09-10T10:00:32.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:32.858+0000 7f9ce4c00640 1 -- 172.21.15.5:0/256671427 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56403687b020 msgr2=0x56403687d410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:00:32.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:32.858+0000 7f9ce4c00640 1 --2- 172.21.15.5:0/256671427 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56403687b020 0x56403687d410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:32.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:32.858+0000 7f9ce4c00640 1 -- 172.21.15.5:0/256671427 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564036870760 msgr2=0x56403687aae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:00:32.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:32.858+0000 7f9ce4c00640 1 --2- 172.21.15.5:0/256671427 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564036870760 0x56403687aae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:32.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:32.858+0000 7f9ce4c00640 1 -- 172.21.15.5:0/256671427 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56403679cae0 con 0x56403686fe40 2024-09-10T10:00:32.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:32.858+0000 7f9ce6000640 1 --2- 172.21.15.5:0/3294988383 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f9cc80a2b80 0x7f9cc80a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:32.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:32.858+0000 7f9ce4c00640 1 --2- 172.21.15.5:0/256671427 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56403686fe40 0x564036870220 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f9ccc013780 tx=0x7f9ccc0096c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8c2de84403228cb server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:00:32.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:32.858+0000 7f9cd2600640 1 -- 172.21.15.5:0/256671427 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ccc013110 con 0x56403686fe40 2024-09-10T10:00:32.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:32.859+0000 7f9cd2600640 1 -- 172.21.15.5:0/256671427 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9ccc0132b0 con 0x56403686fe40 2024-09-10T10:00:32.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:32.859+0000 7f9cd2600640 1 -- 172.21.15.5:0/256671427 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ccc005650 con 0x56403686fe40 2024-09-10T10:00:32.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:32.860+0000 7f9ceaa9d740 1 -- 172.21.15.5:0/256671427 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56403686fe40 msgr2=0x564036870220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:32.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:32.860+0000 7f9ceaa9d740 1 --2- 172.21.15.5:0/256671427 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56403686fe40 0x564036870220 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f9ccc013780 tx=0x7f9ccc0096c0 comp rx=0 tx=0).stop 2024-09-10T10:00:32.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:32.860+0000 7f9ce6000640 1 --2- 172.21.15.5:0/3294988383 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f9cc80a2b80 0x7f9cc80a5040 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f9cd8004460 tx=0x7f9cd80043b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:00:32.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:32.860+0000 7f9ceaa9d740 1 -- 172.21.15.5:0/256671427 shutdown_connections 2024-09-10T10:00:32.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:32.860+0000 7f9ceaa9d740 1 --2- 172.21.15.5:0/256671427 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56403687b020 0x56403687d410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:32.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:32.860+0000 7f9ceaa9d740 1 --2- 172.21.15.5:0/256671427 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564036870760 0x56403687aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:32.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:32.860+0000 7f9ceaa9d740 1 --2- 172.21.15.5:0/256671427 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56403686fe40 0x564036870220 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:32.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:32.860+0000 7f9ceaa9d740 1 -- 172.21.15.5:0/256671427 >> 172.21.15.5:0/256671427 conn(0x56403686c3d0 msgr2=0x56403686c7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:00:32.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:32.860+0000 7f9ceaa9d740 1 -- 172.21.15.5:0/256671427 shutdown_connections 2024-09-10T10:00:32.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:32.860+0000 7f9ceaa9d740 1 -- 172.21.15.5:0/256671427 wait complete. 2024-09-10T10:00:32.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:32.860+0000 7f9ceaa9d740 1 -- start start 2024-09-10T10:00:32.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:32.861+0000 7f9ceaa9d740 1 Processor -- start 2024-09-10T10:00:32.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:32.861+0000 7f9ceaa9d740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56403686fe40 0x564036877fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:32.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:32.861+0000 7f9ceaa9d740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564036870760 0x5640368784e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:32.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:32.861+0000 7f9ceaa9d740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56403687b020 0x564036878a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:32.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:32.861+0000 7f9ceaa9d740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x56403687dbf0 con 0x56403686fe40 2024-09-10T10:00:32.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:32.861+0000 7f9ceaa9d740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x5640367910f0 con 0x564036870760 2024-09-10T10:00:32.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:32.861+0000 7f9ceaa9d740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x56403687dd70 con 0x56403687b020 2024-09-10T10:00:32.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:32.862+0000 7f9ceaa9d740 10 client.0 _ll_register_callbacks cb 0x564036778bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:00:32.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:32.862+0000 7f9ce4c00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56403686fe40 0x564036877fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:32.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:32.862+0000 7f9ce4c00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56403686fe40 0x564036877fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57780/0 (socket says 172.21.15.5:57780) 2024-09-10T10:00:32.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:32.862+0000 7f9ce4c00640 1 -- 172.21.15.5:0/3261098532 learned_addr learned my addr 172.21.15.5:0/3261098532 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T10:00:32.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:32.862+0000 7f9ce5600640 1 --2- 172.21.15.5:0/3261098532 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564036870760 0x5640368784e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:32.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:32.862+0000 7f9ce4c00640 1 --2- 172.21.15.5:0/3261098532 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56403687b020 0x564036878a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:32.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:32.862+0000 7f9ce5600640 1 -- 172.21.15.5:0/3261098532 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56403687b020 msgr2=0x564036878a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:00:32.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:32.863+0000 7f9ce5600640 1 --2- 172.21.15.5:0/3261098532 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56403687b020 0x564036878a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:32.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:32.863+0000 7f9ce5600640 1 -- 172.21.15.5:0/3261098532 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56403686fe40 msgr2=0x564036877fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:00:32.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:32.863+0000 7f9ce5600640 1 --2- 172.21.15.5:0/3261098532 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56403686fe40 0x564036877fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:32.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:32.863+0000 7f9ce5600640 1 -- 172.21.15.5:0/3261098532 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5640365473c0 con 0x564036870760 2024-09-10T10:00:32.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:32.863+0000 7f9ce4c00640 1 --2- 172.21.15.5:0/3261098532 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56403686fe40 0x564036877fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:00:32.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:32.863+0000 7f9ce5600640 1 --2- 172.21.15.5:0/3261098532 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564036870760 0x5640368784e0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f9cd402a9e0 tx=0x7f9cd4004420 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4fbd8bb70b4a604a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:00:32.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:32.863+0000 7f9cc6a00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-09-10T10:00:32.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:32.863+0000 7f9cc6a00640 1 -- 172.21.15.5:0/3261098532 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cd4035040 con 0x564036870760 2024-09-10T10:00:32.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:32.863+0000 7f9ceaa9d740 1 -- 172.21.15.5:0/3261098532 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9ccc0132b0 con 0x564036870760 2024-09-10T10:00:32.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:32.863+0000 7f9cc6a00640 1 -- 172.21.15.5:0/3261098532 <== mon.2 v2:172.21.15.102:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f9cd4005450 con 0x564036870760 2024-09-10T10:00:32.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:32.864+0000 7f9cc6a00640 1 -- 172.21.15.5:0/3261098532 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f9cd404f050 con 0x564036870760 2024-09-10T10:00:32.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:32.864+0000 7f9ceaa9d740 10 client.29618 fetch_fsmap learned FSMap version 42 2024-09-10T10:00:32.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:32.864+0000 7f9ceaa9d740 1 -- 172.21.15.5:0/3261098532 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x5640368924c0 con 0x564036870760 2024-09-10T10:00:32.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:32.864+0000 7f9cc6a00640 1 -- 172.21.15.5:0/3261098532 <== mon.2 v2:172.21.15.102:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9ccc0132b0 con 0x564036870760 2024-09-10T10:00:32.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:32.864+0000 7f9cc6a00640 1 -- 172.21.15.5:0/3261098532 <== mon.2 v2:172.21.15.102:3300/0 5 ==== fsmap(e 42) ==== 8370+0+0 (secure 0 0 0) 0x7f9cd40038f0 con 0x564036870760 2024-09-10T10:00:32.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:32.865+0000 7f9ceaa9d740 10 client.29618 fetch_fsmap finished waiting for FSMap version 42 2024-09-10T10:00:32.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:32.865+0000 7f9ceaa9d740 10 client.29618 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-10T10:00:32.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:32.865+0000 7f9ceaa9d740 20 client.29618 populate_metadata read hostname 'smithi005' 2024-09-10T10:00:32.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:32.865+0000 7f9ceaa9d740 1 --2- 172.21.15.5:0/3261098532 >> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] conn(0x564036893980 0x564036895d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:32.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:32.865+0000 7f9ceaa9d740 4 client.29618 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:00:32.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:32.865+0000 7f9ceaa9d740 1 -- 172.21.15.5:0/3261098532 --> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5640367e3950 con 0x564036893980 2024-09-10T10:00:32.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:32.865+0000 7f9ce4c00640 1 --2- 172.21.15.5:0/3261098532 >> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] conn(0x564036893980 0x564036895d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:32.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:32.865+0000 7f9ce4c00640 1 --2- 172.21.15.5:0/3261098532 >> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] conn(0x564036893980 0x564036895d70 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=40c0c54adcaf700d server_cookie=b509d0f8dac899de in_seq=0 out_seq=0 2024-09-10T10:00:32.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:32.865+0000 7f9cc6a00640 10 client.29618 ms_handle_connect on v2:172.21.15.159:6838/2073739599 2024-09-10T10:00:32.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:32.866+0000 7f9cc6a00640 1 -- 172.21.15.5:0/3261098532 <== mds.1 v2:172.21.15.159:6838/2073739599 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5640367e3950 con 0x564036893980 2024-09-10T10:00:32.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:32.866+0000 7f9cc6a00640 10 client.29618 handle_command_reply: tid=0 2024-09-10T10:00:32.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:32.866+0000 7f9ceaa9d740 1 client.29618 shutdown 2024-09-10T10:00:32.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:32.867+0000 7f9ceaa9d740 1 -- 172.21.15.5:0/3261098532 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564036870760 msgr2=0x5640368784e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:32.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:32.867+0000 7f9ceaa9d740 1 --2- 172.21.15.5:0/3261098532 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564036870760 0x5640368784e0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f9cd402a9e0 tx=0x7f9cd4004420 comp rx=0 tx=0).stop 2024-09-10T10:00:32.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:32.867+0000 7f9ceaa9d740 1 -- 172.21.15.5:0/3261098532 shutdown_connections 2024-09-10T10:00:32.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:32.867+0000 7f9ceaa9d740 1 --2- 172.21.15.5:0/3261098532 >> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] conn(0x564036893980 0x564036895d70 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:32.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:32.867+0000 7f9ceaa9d740 1 --2- 172.21.15.5:0/3261098532 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56403687b020 0x564036878a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:32.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:32.867+0000 7f9ceaa9d740 1 --2- 172.21.15.5:0/3261098532 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564036870760 0x5640368784e0 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:32.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:32.867+0000 7f9ceaa9d740 1 --2- 172.21.15.5:0/3261098532 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56403686fe40 0x564036877fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:32.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:32.867+0000 7f9cc6a00640 0 client.29618 ms_handle_reset on v2:172.21.15.159:6838/2073739599 2024-09-10T10:00:32.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:32.867+0000 7f9ceaa9d740 1 -- 172.21.15.5:0/3261098532 >> 172.21.15.5:0/3261098532 conn(0x56403686c3d0 msgr2=0x564036872100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:00:32.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:32.867+0000 7f9ceaa9d740 1 -- 172.21.15.5:0/3261098532 shutdown_connections 2024-09-10T10:00:32.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:32.868+0000 7f9ceaa9d740 1 -- 172.21.15.5:0/3261098532 wait complete. 2024-09-10T10:00:32.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:32.868+0000 7f9ceaa9d740 20 client.29618 trim_cache size 0 max 16384 2024-09-10T10:00:32.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:32.886+0000 7f9ceaa9d740 1 Processor -- start 2024-09-10T10:00:32.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:32.886+0000 7f9ceaa9d740 1 -- start start 2024-09-10T10:00:32.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:32.887+0000 7f9ceaa9d740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56403686fe40 0x564036879f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:32.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:32.887+0000 7f9ceaa9d740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564036870760 0x56403689b640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:32.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:32.887+0000 7f9ceaa9d740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56403687b020 0x56403689bb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:32.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:32.887+0000 7f9ceaa9d740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f9ce01aca80 con 0x564036870760 2024-09-10T10:00:32.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:32.887+0000 7f9ceaa9d740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f9ce01ae5c0 con 0x56403686fe40 2024-09-10T10:00:32.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:32.887+0000 7f9ceaa9d740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f9ce01ad150 con 0x56403687b020 2024-09-10T10:00:32.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:32.887+0000 7f9ce4c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56403686fe40 0x564036879f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:32.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:32.887+0000 7f9ce5600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564036870760 0x56403689b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:32.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:32.887+0000 7f9ce4c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56403686fe40 0x564036879f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:34108/0 (socket says 172.21.15.5:34108) 2024-09-10T10:00:32.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:32.888+0000 7f9ce4c00640 1 -- 172.21.15.5:0/481506241 learned_addr learned my addr 172.21.15.5:0/481506241 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T10:00:32.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:32.888+0000 7f9ce4c00640 1 --2- 172.21.15.5:0/481506241 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56403687b020 0x56403689bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:32.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:32.888+0000 7f9ce4c00640 1 -- 172.21.15.5:0/481506241 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56403687b020 msgr2=0x56403689bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:00:32.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:32.888+0000 7f9ce4c00640 1 --2- 172.21.15.5:0/481506241 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56403687b020 0x56403689bb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:32.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:32.888+0000 7f9ce4c00640 1 -- 172.21.15.5:0/481506241 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564036870760 msgr2=0x56403689b640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:00:32.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:32.888+0000 7f9ce4c00640 1 --2- 172.21.15.5:0/481506241 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564036870760 0x56403689b640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:32.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:32.888+0000 7f9ce4c00640 1 -- 172.21.15.5:0/481506241 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564036887990 con 0x56403686fe40 2024-09-10T10:00:32.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:32.888+0000 7f9ce5600640 1 --2- 172.21.15.5:0/481506241 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564036870760 0x56403689b640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:00:32.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:32.889+0000 7f9ce4c00640 1 --2- 172.21.15.5:0/481506241 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56403686fe40 0x564036879f10 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f9ccc009230 tx=0x7f9ccc009c20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bd46b8eee716916d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:00:32.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:32.891+0000 7f9cd1c00640 1 -- 172.21.15.5:0/481506241 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ccc028070 con 0x56403686fe40 2024-09-10T10:00:32.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:32.891+0000 7f9cd1c00640 1 -- 172.21.15.5:0/481506241 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9ccc029040 con 0x56403686fe40 2024-09-10T10:00:32.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:32.891+0000 7f9cd1c00640 1 -- 172.21.15.5:0/481506241 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ccc02de80 con 0x56403686fe40 2024-09-10T10:00:32.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:32.892+0000 7f9ceaa9d740 1 -- 172.21.15.5:0/481506241 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56403686fe40 msgr2=0x564036879f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:32.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:32.892+0000 7f9ceaa9d740 1 --2- 172.21.15.5:0/481506241 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56403686fe40 0x564036879f10 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f9ccc009230 tx=0x7f9ccc009c20 comp rx=0 tx=0).stop 2024-09-10T10:00:32.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:32.892+0000 7f9ceaa9d740 1 -- 172.21.15.5:0/481506241 shutdown_connections 2024-09-10T10:00:32.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:32.892+0000 7f9ceaa9d740 1 --2- 172.21.15.5:0/481506241 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56403687b020 0x56403689bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:32.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:32.892+0000 7f9ceaa9d740 1 --2- 172.21.15.5:0/481506241 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564036870760 0x56403689b640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:32.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:32.892+0000 7f9ceaa9d740 1 --2- 172.21.15.5:0/481506241 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56403686fe40 0x564036879f10 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:32.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:32.892+0000 7f9ceaa9d740 1 -- 172.21.15.5:0/481506241 >> 172.21.15.5:0/481506241 conn(0x56403686c3d0 msgr2=0x564036879800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:00:32.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:32.893+0000 7f9ceaa9d740 1 -- 172.21.15.5:0/481506241 shutdown_connections 2024-09-10T10:00:32.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:32.893+0000 7f9ceaa9d740 1 -- 172.21.15.5:0/481506241 wait complete. 2024-09-10T10:00:32.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:32.894+0000 7f9ceaa9d740 1 -- start start 2024-09-10T10:00:32.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:32.894+0000 7f9ceaa9d740 1 Processor -- start 2024-09-10T10:00:32.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:32.894+0000 7f9ceaa9d740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56403686fe40 0x564036882fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:32.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:32.894+0000 7f9ceaa9d740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564036870760 0x564036883500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:32.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:32.894+0000 7f9ceaa9d740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56403687b020 0x56403687ff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:32.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:32.894+0000 7f9ceaa9d740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f9ce01adef0 con 0x56403687b020 2024-09-10T10:00:32.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:32.894+0000 7f9ceaa9d740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f9ce01ad820 con 0x56403686fe40 2024-09-10T10:00:32.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:32.894+0000 7f9ceaa9d740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f9ce01aaa60 con 0x564036870760 2024-09-10T10:00:32.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:32.895+0000 7f9ceaa9d740 10 client.0 _ll_register_callbacks cb 0x7f9cd4009640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:00:32.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:32.895+0000 7f9ce4c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56403686fe40 0x564036882fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:32.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:32.895+0000 7f9ce4c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56403686fe40 0x564036882fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:34124/0 (socket says 172.21.15.5:34124) 2024-09-10T10:00:32.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:32.895+0000 7f9ce4c00640 1 -- 172.21.15.5:0/3369995453 learned_addr learned my addr 172.21.15.5:0/3369995453 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T10:00:32.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:32.895+0000 7f9ce5600640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564036870760 0x564036883500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:32.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:32.895+0000 7f9ce5600640 1 -- 172.21.15.5:0/3369995453 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56403686fe40 msgr2=0x564036882fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:00:32.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:32.895+0000 7f9ce4c00640 1 --2- 172.21.15.5:0/3369995453 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56403687b020 0x56403687ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:32.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:32.896+0000 7f9ce5600640 1 --2- 172.21.15.5:0/3369995453 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56403686fe40 0x564036882fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:32.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:32.896+0000 7f9ce5600640 1 -- 172.21.15.5:0/3369995453 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56403687b020 msgr2=0x56403687ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:00:32.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:32.896+0000 7f9ce5600640 1 --2- 172.21.15.5:0/3369995453 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56403687b020 0x56403687ff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:32.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:32.896+0000 7f9ce5600640 1 -- 172.21.15.5:0/3369995453 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5640368804d0 con 0x564036870760 2024-09-10T10:00:32.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:32.896+0000 7f9ce4c00640 1 --2- 172.21.15.5:0/3369995453 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56403687b020 0x56403687ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:00:32.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:32.896+0000 7f9ce5600640 1 --2- 172.21.15.5:0/3369995453 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564036870760 0x564036883500 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f9cd4031550 tx=0x7f9cd4004110 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a808618f5c5b19e2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:00:32.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:32.896+0000 7f9cc6a00640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-09-10T10:00:32.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:32.900+0000 7f9cc6a00640 1 -- 172.21.15.5:0/3369995453 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cd404c040 con 0x564036870760 2024-09-10T10:00:32.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:32.900+0000 7f9cc6a00640 1 -- 172.21.15.5:0/3369995453 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f9cd4005460 con 0x564036870760 2024-09-10T10:00:32.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:32.900+0000 7f9ceaa9d740 1 -- 172.21.15.5:0/3369995453 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9ccc029040 con 0x564036870760 2024-09-10T10:00:32.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:32.901+0000 7f9cc6a00640 1 -- 172.21.15.5:0/3369995453 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f9cd4035040 con 0x564036870760 2024-09-10T10:00:32.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:32.901+0000 7f9ceaa9d740 10 client.29611 fetch_fsmap learned FSMap version 42 2024-09-10T10:00:32.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:32.901+0000 7f9ceaa9d740 1 -- 172.21.15.5:0/3369995453 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x5640368fecd0 con 0x564036870760 2024-09-10T10:00:32.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:32.901+0000 7f9cc6a00640 1 -- 172.21.15.5:0/3369995453 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9ccc029040 con 0x564036870760 2024-09-10T10:00:32.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:32.902+0000 7f9cc6a00640 1 -- 172.21.15.5:0/3369995453 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 42) ==== 8370+0+0 (secure 0 0 0) 0x7f9cd4031b90 con 0x564036870760 2024-09-10T10:00:32.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:32.902+0000 7f9ceaa9d740 10 client.29611 fetch_fsmap finished waiting for FSMap version 42 2024-09-10T10:00:32.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:32.902+0000 7f9ceaa9d740 10 client.29611 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-10T10:00:32.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:32.902+0000 7f9ceaa9d740 20 client.29611 populate_metadata read hostname 'smithi005' 2024-09-10T10:00:32.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:32.902+0000 7f9ceaa9d740 1 --2- 172.21.15.5:0/3369995453 >> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] conn(0x5640368ff4a0 0x564036901890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:32.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:32.902+0000 7f9ceaa9d740 4 client.29611 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T10:00:32.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:32.902+0000 7f9ceaa9d740 1 -- 172.21.15.5:0/3369995453 --> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5640367dde60 con 0x5640368ff4a0 2024-09-10T10:00:32.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:32.903+0000 7f9ce4c00640 1 --2- 172.21.15.5:0/3369995453 >> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] conn(0x5640368ff4a0 0x564036901890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:32.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:32.904+0000 7f9ce4c00640 1 --2- 172.21.15.5:0/3369995453 >> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] conn(0x5640368ff4a0 0x564036901890 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2be30720dc46a2db server_cookie=1b6247b3b51a33c6 in_seq=0 out_seq=0 2024-09-10T10:00:32.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:32.904+0000 7f9cc6a00640 10 client.29611 ms_handle_connect on v2:172.21.15.159:6838/2073739599 2024-09-10T10:00:32.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:32.904+0000 7f9cc6a00640 1 -- 172.21.15.5:0/3369995453 <== mds.1 v2:172.21.15.159:6838/2073739599 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5640367dde60 con 0x5640368ff4a0 2024-09-10T10:00:32.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:32.904+0000 7f9cc6a00640 10 client.29611 handle_command_reply: tid=0 2024-09-10T10:00:32.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:32.904+0000 7f9ceaa9d740 1 client.29611 shutdown 2024-09-10T10:00:32.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:32.905+0000 7f9ceaa9d740 1 -- 172.21.15.5:0/3369995453 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564036870760 msgr2=0x564036883500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:32.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:32.905+0000 7f9ceaa9d740 1 --2- 172.21.15.5:0/3369995453 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564036870760 0x564036883500 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f9cd4031550 tx=0x7f9cd4004110 comp rx=0 tx=0).stop 2024-09-10T10:00:32.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:32.905+0000 7f9ceaa9d740 1 -- 172.21.15.5:0/3369995453 shutdown_connections 2024-09-10T10:00:32.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:32.905+0000 7f9ceaa9d740 1 --2- 172.21.15.5:0/3369995453 >> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] conn(0x5640368ff4a0 0x564036901890 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:32.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:32.905+0000 7f9ceaa9d740 1 --2- 172.21.15.5:0/3369995453 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56403687b020 0x56403687ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:32.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:32.905+0000 7f9ceaa9d740 1 --2- 172.21.15.5:0/3369995453 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564036870760 0x564036883500 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:32.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:32.905+0000 7f9ceaa9d740 1 --2- 172.21.15.5:0/3369995453 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56403686fe40 0x564036882fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:32.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:32.905+0000 7f9ceaa9d740 1 -- 172.21.15.5:0/3369995453 >> 172.21.15.5:0/3369995453 conn(0x56403686c3d0 msgr2=0x56403687a020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:00:32.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:32.906+0000 7f9cc6a00640 0 client.29611 ms_handle_reset on v2:172.21.15.159:6838/2073739599 2024-09-10T10:00:32.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:32.906+0000 7f9ceaa9d740 1 -- 172.21.15.5:0/3369995453 shutdown_connections 2024-09-10T10:00:32.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:32.906+0000 7f9ceaa9d740 1 -- 172.21.15.5:0/3369995453 wait complete. 2024-09-10T10:00:32.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:32.906+0000 7f9ceaa9d740 20 client.29611 trim_cache size 0 max 16384 2024-09-10T10:00:32.908 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-10T10:00:32.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:32.907+0000 7f9ce6a00640 1 -- 172.21.15.5:0/3294988383 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f9cc80a2b80 msgr2=0x7f9cc80a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:32.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:32.907+0000 7f9ce6a00640 1 --2- 172.21.15.5:0/3294988383 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f9cc80a2b80 0x7f9cc80a5040 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f9cd8004460 tx=0x7f9cd80043b0 comp rx=0 tx=0).stop 2024-09-10T10:00:32.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:32.907+0000 7f9ce6a00640 1 -- 172.21.15.5:0/3294988383 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9ce0108f60 msgr2=0x7f9ce019ed50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:32.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:32.907+0000 7f9ce6a00640 1 --2- 172.21.15.5:0/3294988383 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9ce0108f60 0x7f9ce019ed50 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f9cd8009fd0 tx=0x7f9cd800b260 comp rx=0 tx=0).stop 2024-09-10T10:00:32.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:32.907+0000 7f9ce6a00640 1 -- 172.21.15.5:0/3294988383 shutdown_connections 2024-09-10T10:00:32.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:32.907+0000 7f9ce6a00640 1 --2- 172.21.15.5:0/3294988383 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f9cc80a2b80 0x7f9cc80a5040 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:32.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:32.907+0000 7f9ce6a00640 1 --2- 172.21.15.5:0/3294988383 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9ce0108f60 0x7f9ce019ed50 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:32.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:32.907+0000 7f9ce6a00640 1 --2- 172.21.15.5:0/3294988383 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f9ce01047e0 0x7f9ce019a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:32.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:32.907+0000 7f9ce6a00640 1 --2- 172.21.15.5:0/3294988383 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9ce0103e30 0x7f9ce019a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:32.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:32.907+0000 7f9ce6a00640 1 -- 172.21.15.5:0/3294988383 >> 172.21.15.5:0/3294988383 conn(0x7f9ce00ff910 msgr2=0x7f9ce0100c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:00:32.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:32.908+0000 7f9ce6a00640 1 -- 172.21.15.5:0/3294988383 shutdown_connections 2024-09-10T10:00:32.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:32.908+0000 7f9ce6a00640 1 -- 172.21.15.5:0/3294988383 wait complete. 2024-09-10T10:00:33.034 INFO:tasks.workunit.client.0.smithi005.stdout: 10 131324 13.43 MB/sec execute 371 sec latency 23714.747 ms 2024-09-10T10:00:33.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:00:32 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:00:33.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:00:32 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:00:33.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:00:32 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T10:00:33.309 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:00:32 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:00:33.309 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:00:32 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:00:33.309 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:00:32 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T10:00:33.309 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:00:33.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:00:32 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:00:33.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:00:32 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:00:33.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:00:32 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T10:00:33.993 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-10T10:00:34.034 INFO:tasks.workunit.client.0.smithi005.stdout: 10 131793 13.43 MB/sec execute 372 sec latency 23743.691 ms 2024-09-10T10:00:34.109 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:00:33 smithi005 ceph-mon[30350]: pgmap v1059: 129 pgs: 129 active+clean; 690 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 72 KiB/s wr, 19 op/s 2024-09-10T10:00:34.285 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:00:33 smithi102 ceph-mon[35442]: pgmap v1059: 129 pgs: 129 active+clean; 690 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 72 KiB/s wr, 19 op/s 2024-09-10T10:00:34.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:00:33 smithi159 ceph-mon[35513]: pgmap v1059: 129 pgs: 129 active+clean; 690 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 72 KiB/s wr, 19 op/s 2024-09-10T10:00:34.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:34.612+0000 7fdf77000640 1 -- 172.21.15.5:0/3400088365 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdf70100b20 msgr2=0x7fdf70100f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:34.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:34.612+0000 7fdf77000640 1 --2- 172.21.15.5:0/3400088365 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdf70100b20 0x7fdf70100f00 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fdf60009860 tx=0x7fdf6002a340 comp rx=0 tx=0).stop 2024-09-10T10:00:34.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:34.612+0000 7fdf77000640 1 -- 172.21.15.5:0/3400088365 shutdown_connections 2024-09-10T10:00:34.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:34.612+0000 7fdf77000640 1 --2- 172.21.15.5:0/3400088365 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdf7010e500 0x7fdf70110910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:34.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:34.612+0000 7fdf77000640 1 --2- 172.21.15.5:0/3400088365 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdf70101440 0x7fdf7010df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:34.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:34.612+0000 7fdf77000640 1 --2- 172.21.15.5:0/3400088365 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdf70100b20 0x7fdf70100f00 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:34.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:34.612+0000 7fdf77000640 1 -- 172.21.15.5:0/3400088365 >> 172.21.15.5:0/3400088365 conn(0x7fdf700fc7d0 msgr2=0x7fdf700febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:00:34.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:34.613+0000 7fdf77000640 1 -- 172.21.15.5:0/3400088365 shutdown_connections 2024-09-10T10:00:34.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:34.613+0000 7fdf77000640 1 -- 172.21.15.5:0/3400088365 wait complete. 2024-09-10T10:00:34.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:34.615+0000 7fdf77000640 1 Processor -- start 2024-09-10T10:00:34.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:34.615+0000 7fdf77000640 1 -- start start 2024-09-10T10:00:34.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:34.616+0000 7fdf77000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdf70100b20 0x7fdf7010c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:34.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:34.617+0000 7fdf77000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdf70101440 0x7fdf7010c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:34.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:34.617+0000 7fdf77000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdf7010e500 0x7fdf701aaa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:34.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:34.617+0000 7fdf77000640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fdf701038d0 con 0x7fdf70100b20 2024-09-10T10:00:34.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:34.617+0000 7fdf77000640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fdf70103750 con 0x7fdf7010e500 2024-09-10T10:00:34.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:34.617+0000 7fdf77000640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fdf70103a50 con 0x7fdf70101440 2024-09-10T10:00:34.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:34.617+0000 7fdf6fe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdf70100b20 0x7fdf7010c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:34.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:34.618+0000 7fdf75000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdf7010e500 0x7fdf701aaa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:34.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:34.618+0000 7fdf75000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdf7010e500 0x7fdf701aaa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:34136/0 (socket says 172.21.15.5:34136) 2024-09-10T10:00:34.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:34.618+0000 7fdf75000640 1 -- 172.21.15.5:0/988153599 learned_addr learned my addr 172.21.15.5:0/988153599 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T10:00:34.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:34.618+0000 7fdf6f400640 1 --2- 172.21.15.5:0/988153599 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdf70101440 0x7fdf7010c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:34.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:34.620+0000 7fdf75000640 1 -- 172.21.15.5:0/988153599 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdf70101440 msgr2=0x7fdf7010c9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:34.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:34.620+0000 7fdf75000640 1 --2- 172.21.15.5:0/988153599 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdf70101440 0x7fdf7010c9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:34.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:34.620+0000 7fdf75000640 1 -- 172.21.15.5:0/988153599 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdf70100b20 msgr2=0x7fdf7010c4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:34.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:34.620+0000 7fdf75000640 1 --2- 172.21.15.5:0/988153599 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdf70100b20 0x7fdf7010c4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:34.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:34.620+0000 7fdf75000640 1 -- 172.21.15.5:0/988153599 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf701aaf50 con 0x7fdf7010e500 2024-09-10T10:00:34.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:34.620+0000 7fdf6f400640 1 --2- 172.21.15.5:0/988153599 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdf70101440 0x7fdf7010c9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:00:34.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:34.621+0000 7fdf75000640 1 --2- 172.21.15.5:0/988153599 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdf7010e500 0x7fdf701aaa10 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7fdf6800eb40 tx=0x7fdf6800c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:00:34.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:34.622+0000 7fdf6cc00640 1 -- 172.21.15.5:0/988153599 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf68004b20 con 0x7fdf7010e500 2024-09-10T10:00:34.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:34.622+0000 7fdf6cc00640 1 -- 172.21.15.5:0/988153599 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdf68004cc0 con 0x7fdf7010e500 2024-09-10T10:00:34.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:34.622+0000 7fdf6cc00640 1 -- 172.21.15.5:0/988153599 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf68010850 con 0x7fdf7010e500 2024-09-10T10:00:34.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:34.623+0000 7fdf77000640 1 -- 172.21.15.5:0/988153599 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf701ab1e0 con 0x7fdf7010e500 2024-09-10T10:00:34.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:34.623+0000 7fdf77000640 1 -- 172.21.15.5:0/988153599 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf701ab6a0 con 0x7fdf7010e500 2024-09-10T10:00:34.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:34.625+0000 7fdf77000640 1 -- 172.21.15.5:0/988153599 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf7006a850 con 0x7fdf7010e500 2024-09-10T10:00:34.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:34.629+0000 7fdf6cc00640 1 -- 172.21.15.5:0/988153599 <== mon.2 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7fdf68005000 con 0x7fdf7010e500 2024-09-10T10:00:34.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:34.632+0000 7fdf6cc00640 1 --2- 172.21.15.5:0/988153599 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fdf540a2d50 0x7fdf540a5210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:34.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:34.634+0000 7fdf6cc00640 1 -- 172.21.15.5:0/988153599 <== mon.2 v2:172.21.15.102:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fdf680d0260 con 0x7fdf7010e500 2024-09-10T10:00:34.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:34.634+0000 7fdf6fe00640 1 --2- 172.21.15.5:0/988153599 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fdf540a2d50 0x7fdf540a5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:34.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:34.637+0000 7fdf6cc00640 1 -- 172.21.15.5:0/988153599 <== mon.2 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fdf6801d060 con 0x7fdf7010e500 2024-09-10T10:00:34.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:34.638+0000 7fdf6fe00640 1 --2- 172.21.15.5:0/988153599 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fdf540a2d50 0x7fdf540a5210 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fdf6002a930 tx=0x7fdf600056d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:00:34.702 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:00:34.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:34.976+0000 7fdf77000640 1 -- 172.21.15.5:0/988153599 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdf70100f00 con 0x7fdf7010e500 2024-09-10T10:00:34.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:34.977+0000 7fdf6cc00640 1 -- 172.21.15.5:0/988153599 <== mon.2 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 42 v42) ==== 94+0+11969 (secure 0 0 0) 0x7fdf70100f00 con 0x7fdf7010e500 2024-09-10T10:00:34.978 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-10T10:00:34.979 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":42,"btime":"2024-09-10T09:55:06:670916+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14454,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/2970594311","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2970594311},{"type":"v1","addr":"172.21.15.5:6835","nonce":2970594311}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14514,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6839/3616502174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":3616502174},{"type":"v1","addr":"172.21.15.5:6839","nonce":3616502174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":42,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T09:35:29.626287+0000","modified":"2024-09-10T09:55:06.419829+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24542,"mds_1":24535,"mds_2":24511,"mds_3":24506,"mds_4":24443},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14520":{"gid":14520,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/2073599842","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":2073599842},{"type":"v1","addr":"172.21.15.5:6841","nonce":2073599842}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"k","rank":4,"incarnation":26,"state":"up:active","state_seq":24,"addr":"172.21.15.102:6833/2457007427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":2457007427},{"type":"v1","addr":"172.21.15.102:6833","nonce":2457007427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/356445159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":356445159},{"type":"v1","addr":"172.21.15.159:6833","nonce":356445159}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6837/2894314507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":2894314507},{"type":"v1","addr":"172.21.15.5:6837","nonce":2894314507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.102:6835/1647549111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":1647549111},{"type":"v1","addr":"172.21.15.102:6835","nonce":1647549111}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6835/952657242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":952657242},{"type":"v1","addr":"172.21.15.159:6835","nonce":952657242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":19,"addr":"172.21.15.102:6837/2689776764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":2689776764},{"type":"v1","addr":"172.21.15.102:6837","nonce":2689776764}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.159:6837/620627639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":620627639},{"type":"v1","addr":"172.21.15.159:6837","nonce":620627639}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24535":{"gid":24535,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":18,"addr":"172.21.15.159:6839/2073739599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":2073739599},{"type":"v1","addr":"172.21.15.159:6839","nonce":2073739599}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24542":{"gid":24542,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.102:6839/335409749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6838","nonce":335409749},{"type":"v1","addr":"172.21.15.102:6839","nonce":335409749}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24542,"qdb_cluster":[24542,24535,24511,24506,24443]},"id":1}]} 2024-09-10T10:00:34.979 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 42 2024-09-10T10:00:34.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:34.979+0000 7fdf77000640 1 -- 172.21.15.5:0/988153599 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fdf540a2d50 msgr2=0x7fdf540a5210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:34.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:34.979+0000 7fdf77000640 1 --2- 172.21.15.5:0/988153599 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fdf540a2d50 0x7fdf540a5210 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fdf6002a930 tx=0x7fdf600056d0 comp rx=0 tx=0).stop 2024-09-10T10:00:34.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:34.979+0000 7fdf77000640 1 -- 172.21.15.5:0/988153599 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdf7010e500 msgr2=0x7fdf701aaa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:34.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:34.979+0000 7fdf77000640 1 --2- 172.21.15.5:0/988153599 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdf7010e500 0x7fdf701aaa10 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7fdf6800eb40 tx=0x7fdf6800c2d0 comp rx=0 tx=0).stop 2024-09-10T10:00:34.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:34.979+0000 7fdf77000640 1 -- 172.21.15.5:0/988153599 shutdown_connections 2024-09-10T10:00:34.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:34.980+0000 7fdf77000640 1 --2- 172.21.15.5:0/988153599 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fdf540a2d50 0x7fdf540a5210 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:34.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:34.980+0000 7fdf77000640 1 --2- 172.21.15.5:0/988153599 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fdf7010e500 0x7fdf701aaa10 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:34.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:34.980+0000 7fdf77000640 1 --2- 172.21.15.5:0/988153599 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fdf70101440 0x7fdf7010c9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:34.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:34.980+0000 7fdf77000640 1 --2- 172.21.15.5:0/988153599 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdf70100b20 0x7fdf7010c4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:34.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:34.980+0000 7fdf77000640 1 -- 172.21.15.5:0/988153599 >> 172.21.15.5:0/988153599 conn(0x7fdf700fc7d0 msgr2=0x7fdf700febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:00:34.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:34.980+0000 7fdf77000640 1 -- 172.21.15.5:0/988153599 shutdown_connections 2024-09-10T10:00:34.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:34.980+0000 7fdf77000640 1 -- 172.21.15.5:0/988153599 wait complete. 2024-09-10T10:00:35.034 INFO:tasks.workunit.client.0.smithi005.stdout: 10 132436 13.48 MB/sec execute 373 sec latency 51.786 ms 2024-09-10T10:00:35.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:35.639+0000 7fc117e00640 1 -- 172.21.15.5:0/3942254356 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc110101df0 msgr2=0x7fc1101021d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:35.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:35.639+0000 7fc117e00640 1 --2- 172.21.15.5:0/3942254356 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc110101df0 0x7fc1101021d0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7fc104009900 tx=0x7fc10402a3f0 comp rx=0 tx=0).stop 2024-09-10T10:00:35.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:35.639+0000 7fc117e00640 1 -- 172.21.15.5:0/3942254356 shutdown_connections 2024-09-10T10:00:35.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:35.639+0000 7fc117e00640 1 --2- 172.21.15.5:0/3942254356 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc110106f20 0x7fc11010eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:35.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:35.639+0000 7fc117e00640 1 --2- 172.21.15.5:0/3942254356 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc1101027a0 0x7fc1101067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:35.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:35.639+0000 7fc117e00640 1 --2- 172.21.15.5:0/3942254356 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc110101df0 0x7fc1101021d0 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:35.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:35.639+0000 7fc117e00640 1 -- 172.21.15.5:0/3942254356 >> 172.21.15.5:0/3942254356 conn(0x7fc1100fd8b0 msgr2=0x7fc1100ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:00:35.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:35.639+0000 7fc117e00640 1 -- 172.21.15.5:0/3942254356 shutdown_connections 2024-09-10T10:00:35.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:35.639+0000 7fc117e00640 1 -- 172.21.15.5:0/3942254356 wait complete. 2024-09-10T10:00:35.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:35.640+0000 7fc117e00640 1 Processor -- start 2024-09-10T10:00:35.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:35.640+0000 7fc117e00640 1 -- start start 2024-09-10T10:00:35.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:35.641+0000 7fc117e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc110101df0 0x7fc1100744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:35.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:35.641+0000 7fc117e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc1101027a0 0x7fc110074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:35.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:35.641+0000 7fc117e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc110106f20 0x7fc110074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:35.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:35.641+0000 7fc117e00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc110111f90 con 0x7fc110101df0 2024-09-10T10:00:35.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:35.641+0000 7fc117e00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fc110111e10 con 0x7fc1101027a0 2024-09-10T10:00:35.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:35.641+0000 7fc117e00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fc110112110 con 0x7fc110106f20 2024-09-10T10:00:35.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:35.641+0000 7fc115400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc110101df0 0x7fc1100744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:35.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:35.641+0000 7fc114a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc1101027a0 0x7fc110074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:35.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:35.641+0000 7fc115e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc110106f20 0x7fc110074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:35.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:35.641+0000 7fc114a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc1101027a0 0x7fc110074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:34152/0 (socket says 172.21.15.5:34152) 2024-09-10T10:00:35.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:35.641+0000 7fc114a00640 1 -- 172.21.15.5:0/1611617915 learned_addr learned my addr 172.21.15.5:0/1611617915 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T10:00:35.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:35.641+0000 7fc115400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc110101df0 0x7fc1100744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57824/0 (socket says 172.21.15.5:57824) 2024-09-10T10:00:35.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:35.641+0000 7fc115e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc110106f20 0x7fc110074f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:34494/0 (socket says 172.21.15.5:34494) 2024-09-10T10:00:35.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:35.642+0000 7fc115400640 1 -- 172.21.15.5:0/1611617915 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc110106f20 msgr2=0x7fc110074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:35.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:35.642+0000 7fc115400640 1 --2- 172.21.15.5:0/1611617915 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc110106f20 0x7fc110074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:35.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:35.642+0000 7fc115400640 1 -- 172.21.15.5:0/1611617915 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc1101027a0 msgr2=0x7fc110074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:35.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:35.642+0000 7fc115400640 1 --2- 172.21.15.5:0/1611617915 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc1101027a0 0x7fc110074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:35.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:35.642+0000 7fc115400640 1 -- 172.21.15.5:0/1611617915 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1101a6930 con 0x7fc110101df0 2024-09-10T10:00:35.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:35.642+0000 7fc115400640 1 --2- 172.21.15.5:0/1611617915 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc110101df0 0x7fc1100744f0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7fc10402a910 tx=0x7fc104003b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:00:35.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:35.642+0000 7fc10e000640 1 -- 172.21.15.5:0/1611617915 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc10403f070 con 0x7fc110101df0 2024-09-10T10:00:35.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:35.643+0000 7fc117e00640 1 -- 172.21.15.5:0/1611617915 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1101a6bc0 con 0x7fc110101df0 2024-09-10T10:00:35.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:35.643+0000 7fc10e000640 1 -- 172.21.15.5:0/1611617915 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc104031c90 con 0x7fc110101df0 2024-09-10T10:00:35.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:35.643+0000 7fc10e000640 1 -- 172.21.15.5:0/1611617915 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc104035030 con 0x7fc110101df0 2024-09-10T10:00:35.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:35.643+0000 7fc117e00640 1 -- 172.21.15.5:0/1611617915 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1101a70a0 con 0x7fc110101df0 2024-09-10T10:00:35.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:35.646+0000 7fc10e000640 1 -- 172.21.15.5:0/1611617915 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7fc104042070 con 0x7fc110101df0 2024-09-10T10:00:35.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:35.647+0000 7fc11aa38740 1 Processor -- start 2024-09-10T10:00:35.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:35.647+0000 7fc11aa38740 1 -- start start 2024-09-10T10:00:35.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:35.648+0000 7fc11aa38740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dd589ecb90 0x55dd589ecf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:35.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:35.648+0000 7fc11aa38740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dd589ed4b0 0x55dd58a00120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:35.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:35.648+0000 7fc11aa38740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dd58a00660 0x55dd58a02a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:35.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:35.648+0000 7fc11aa38740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55dd589533e0 con 0x55dd589ecb90 2024-09-10T10:00:35.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:35.648+0000 7fc11aa38740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55dd588fad80 con 0x55dd589ed4b0 2024-09-10T10:00:35.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:35.648+0000 7fc11aa38740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55dd5894f3a0 con 0x55dd58a00660 2024-09-10T10:00:35.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:35.648+0000 7fc114a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dd58a00660 0x55dd58a02a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:35.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:35.648+0000 7fc114a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dd58a00660 0x55dd58a02a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:34496/0 (socket says 172.21.15.5:34496) 2024-09-10T10:00:35.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:35.648+0000 7fc114a00640 1 -- 172.21.15.5:0/1999604295 learned_addr learned my addr 172.21.15.5:0/1999604295 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T10:00:35.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:35.649+0000 7fc115e00640 1 --2- 172.21.15.5:0/1999604295 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dd589ecb90 0x55dd589ecf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:35.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:35.649+0000 7fc10e000640 1 --2- 172.21.15.5:0/1611617915 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fc0f80a2b80 0x7fc0f80a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:35.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:35.649+0000 7fc115400640 1 --2- 172.21.15.5:0/1999604295 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dd589ed4b0 0x55dd58a00120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:35.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:35.649+0000 7fc10e000640 1 -- 172.21.15.5:0/1611617915 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fc1040efdc0 con 0x7fc110101df0 2024-09-10T10:00:35.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:35.649+0000 7fc115e00640 1 --2- 172.21.15.5:0/1611617915 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fc0f80a2b80 0x7fc0f80a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:35.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:35.649+0000 7fc115e00640 1 -- 172.21.15.5:0/1999604295 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dd58a00660 msgr2=0x55dd58a02a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:00:35.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:35.649+0000 7fc115e00640 1 --2- 172.21.15.5:0/1999604295 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dd58a00660 0x55dd58a02a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:35.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:35.649+0000 7fc115e00640 1 -- 172.21.15.5:0/1999604295 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dd589ed4b0 msgr2=0x55dd58a00120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:00:35.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:35.649+0000 7fc115e00640 1 --2- 172.21.15.5:0/1999604295 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dd589ed4b0 0x55dd58a00120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:35.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:35.649+0000 7fc115e00640 1 -- 172.21.15.5:0/1999604295 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dd58919ae0 con 0x55dd589ecb90 2024-09-10T10:00:35.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:35.650+0000 7fc115e00640 1 --2- 172.21.15.5:0/1999604295 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dd589ecb90 0x55dd589ecf70 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7fc108005fd0 tx=0x7fc108013e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7884f1ffde45b7de server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:00:35.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:35.650+0000 7fc102600640 1 -- 172.21.15.5:0/1999604295 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc108023030 con 0x55dd589ecb90 2024-09-10T10:00:35.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:35.651+0000 7fc102600640 1 -- 172.21.15.5:0/1999604295 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc1080058d0 con 0x55dd589ecb90 2024-09-10T10:00:35.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:35.651+0000 7fc102600640 1 -- 172.21.15.5:0/1999604295 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc10801f9b0 con 0x55dd589ecb90 2024-09-10T10:00:35.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:35.651+0000 7fc11aa38740 1 -- 172.21.15.5:0/1999604295 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dd589ecb90 msgr2=0x55dd589ecf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:35.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:35.651+0000 7fc11aa38740 1 --2- 172.21.15.5:0/1999604295 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dd589ecb90 0x55dd589ecf70 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7fc108005fd0 tx=0x7fc108013e50 comp rx=0 tx=0).stop 2024-09-10T10:00:35.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:35.651+0000 7fc115e00640 1 --2- 172.21.15.5:0/1611617915 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fc0f80a2b80 0x7fc0f80a5040 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fc108023000 tx=0x7fc108022000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:00:35.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:35.652+0000 7fc11aa38740 1 -- 172.21.15.5:0/1999604295 shutdown_connections 2024-09-10T10:00:35.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:35.652+0000 7fc11aa38740 1 --2- 172.21.15.5:0/1999604295 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dd58a00660 0x55dd58a02a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:35.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:35.652+0000 7fc11aa38740 1 --2- 172.21.15.5:0/1999604295 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dd589ed4b0 0x55dd58a00120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:35.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:35.652+0000 7fc11aa38740 1 --2- 172.21.15.5:0/1999604295 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dd589ecb90 0x55dd589ecf70 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:35.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:35.652+0000 7fc11aa38740 1 -- 172.21.15.5:0/1999604295 >> 172.21.15.5:0/1999604295 conn(0x55dd589e9200 msgr2=0x55dd589e95e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:00:35.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:35.652+0000 7fc11aa38740 1 -- 172.21.15.5:0/1999604295 shutdown_connections 2024-09-10T10:00:35.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:35.652+0000 7fc11aa38740 1 -- 172.21.15.5:0/1999604295 wait complete. 2024-09-10T10:00:35.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:35.653+0000 7fc11aa38740 1 -- start start 2024-09-10T10:00:35.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:35.653+0000 7fc11aa38740 1 Processor -- start 2024-09-10T10:00:35.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:35.654+0000 7fc11aa38740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dd589ecb90 0x55dd589fd5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:35.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:35.654+0000 7fc11aa38740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dd589ed4b0 0x55dd589fdae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:35.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:35.654+0000 7fc11aa38740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dd58a00660 0x55dd589fe020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:35.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:35.654+0000 7fc11aa38740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55dd58a031d0 con 0x55dd58a00660 2024-09-10T10:00:35.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:35.654+0000 7fc11aa38740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55dd5890e0f0 con 0x55dd589ed4b0 2024-09-10T10:00:35.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:35.654+0000 7fc11aa38740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55dd58a03350 con 0x55dd589ecb90 2024-09-10T10:00:35.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:35.654+0000 7fc11aa38740 10 client.0 _ll_register_callbacks cb 0x55dd588f5bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:00:35.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:35.654+0000 7fc115e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dd58a00660 0x55dd589fe020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:35.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:35.654+0000 7fc115400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dd589ecb90 0x55dd589fd5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:35.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:35.654+0000 7fc115400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dd589ecb90 0x55dd589fd5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:34498/0 (socket says 172.21.15.5:34498) 2024-09-10T10:00:35.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:35.654+0000 7fc115400640 1 -- 172.21.15.5:0/2833027654 learned_addr learned my addr 172.21.15.5:0/2833027654 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T10:00:35.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:35.654+0000 7fc114a00640 1 --2- 172.21.15.5:0/2833027654 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dd589ed4b0 0x55dd589fdae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:35.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:35.655+0000 7fc115400640 1 -- 172.21.15.5:0/2833027654 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dd589ed4b0 msgr2=0x55dd589fdae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:00:35.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:35.655+0000 7fc115400640 1 --2- 172.21.15.5:0/2833027654 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dd589ed4b0 0x55dd589fdae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:35.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:35.655+0000 7fc115400640 1 -- 172.21.15.5:0/2833027654 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dd58a00660 msgr2=0x55dd589fe020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:00:35.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:35.655+0000 7fc115400640 1 --2- 172.21.15.5:0/2833027654 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dd58a00660 0x55dd589fe020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:35.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:35.655+0000 7fc115400640 1 -- 172.21.15.5:0/2833027654 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dd586c43c0 con 0x55dd589ecb90 2024-09-10T10:00:35.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:35.655+0000 7fc115400640 1 --2- 172.21.15.5:0/2833027654 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dd589ecb90 0x55dd589fd5a0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7fc1040bc210 tx=0x7fc104031490 comp rx=0 tx=0).ready entity=mon.1 client_cookie=614ad6aeeb7f05fb server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:00:35.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:35.655+0000 7fc0f6a00640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-09-10T10:00:35.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:35.655+0000 7fc0f6a00640 1 -- 172.21.15.5:0/2833027654 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc104049040 con 0x55dd589ecb90 2024-09-10T10:00:35.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:35.656+0000 7fc0f6a00640 1 -- 172.21.15.5:0/2833027654 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fc10403b460 con 0x55dd589ecb90 2024-09-10T10:00:35.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:35.656+0000 7fc11aa38740 1 -- 172.21.15.5:0/2833027654 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc1080058d0 con 0x55dd589ecb90 2024-09-10T10:00:35.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:35.656+0000 7fc0f6a00640 1 -- 172.21.15.5:0/2833027654 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7fc1040bc280 con 0x55dd589ecb90 2024-09-10T10:00:35.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:35.656+0000 7fc11aa38740 10 client.29626 fetch_fsmap learned FSMap version 42 2024-09-10T10:00:35.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:35.656+0000 7fc11aa38740 1 -- 172.21.15.5:0/2833027654 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55dd58a031d0 con 0x55dd589ecb90 2024-09-10T10:00:35.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:35.656+0000 7fc0f6a00640 1 -- 172.21.15.5:0/2833027654 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc1080058d0 con 0x55dd589ecb90 2024-09-10T10:00:35.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:35.656+0000 7fc0f6a00640 1 -- 172.21.15.5:0/2833027654 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 42) ==== 8370+0+0 (secure 0 0 0) 0x7fc1040c01b0 con 0x55dd589ecb90 2024-09-10T10:00:35.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:35.656+0000 7fc11aa38740 10 client.29626 fetch_fsmap finished waiting for FSMap version 42 2024-09-10T10:00:35.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:35.656+0000 7fc11aa38740 10 client.29626 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T10:00:35.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:35.657+0000 7fc11aa38740 20 client.29626 populate_metadata read hostname 'smithi005' 2024-09-10T10:00:35.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:35.657+0000 7fc11aa38740 1 --2- 172.21.15.5:0/2833027654 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55dd58a106d0 0x55dd58a12b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:35.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:35.657+0000 7fc11aa38740 4 client.29626 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:00:35.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:35.657+0000 7fc11aa38740 1 -- 172.21.15.5:0/2833027654 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dd58960950 con 0x55dd58a106d0 2024-09-10T10:00:35.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:35.657+0000 7fc114a00640 1 --2- 172.21.15.5:0/2833027654 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55dd58a106d0 0x55dd58a12b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:35.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:35.658+0000 7fc0f6a00640 10 client.29626 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T10:00:35.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:35.658+0000 7fc114a00640 1 --2- 172.21.15.5:0/2833027654 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55dd58a106d0 0x55dd58a12b10 crc :-1 s=READY pgs=301 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5665c0750ae6e00a server_cookie=bff127efe32a3ab in_seq=0 out_seq=0 2024-09-10T10:00:35.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:35.659+0000 7fc0f6a00640 1 -- 172.21.15.5:0/2833027654 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55dd58960950 con 0x55dd58a106d0 2024-09-10T10:00:35.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:35.659+0000 7fc0f6a00640 10 client.29626 handle_command_reply: tid=0 2024-09-10T10:00:35.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:35.659+0000 7fc11aa38740 1 client.29626 shutdown 2024-09-10T10:00:35.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:35.659+0000 7fc11aa38740 1 -- 172.21.15.5:0/2833027654 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dd589ecb90 msgr2=0x55dd589fd5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:35.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:35.659+0000 7fc11aa38740 1 --2- 172.21.15.5:0/2833027654 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dd589ecb90 0x55dd589fd5a0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7fc1040bc210 tx=0x7fc104031490 comp rx=0 tx=0).stop 2024-09-10T10:00:35.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:35.659+0000 7fc11aa38740 1 -- 172.21.15.5:0/2833027654 shutdown_connections 2024-09-10T10:00:35.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:35.659+0000 7fc11aa38740 1 --2- 172.21.15.5:0/2833027654 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55dd58a106d0 0x55dd58a12b10 crc :-1 s=READY pgs=301 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:35.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:35.659+0000 7fc11aa38740 1 --2- 172.21.15.5:0/2833027654 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dd58a00660 0x55dd589fe020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:35.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:35.659+0000 7fc11aa38740 1 --2- 172.21.15.5:0/2833027654 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dd589ed4b0 0x55dd589fdae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:35.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:35.659+0000 7fc0f6a00640 0 client.29626 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T10:00:35.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:35.660+0000 7fc11aa38740 1 --2- 172.21.15.5:0/2833027654 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dd589ecb90 0x55dd589fd5a0 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:35.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:35.660+0000 7fc11aa38740 1 -- 172.21.15.5:0/2833027654 >> 172.21.15.5:0/2833027654 conn(0x55dd589e9200 msgr2=0x55dd589e8380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:00:35.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:35.660+0000 7fc11aa38740 1 -- 172.21.15.5:0/2833027654 shutdown_connections 2024-09-10T10:00:35.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:35.660+0000 7fc11aa38740 1 -- 172.21.15.5:0/2833027654 wait complete. 2024-09-10T10:00:35.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:35.660+0000 7fc11aa38740 20 client.29626 trim_cache size 0 max 16384 2024-09-10T10:00:35.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:35.689+0000 7fc11aa38740 1 Processor -- start 2024-09-10T10:00:35.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:35.689+0000 7fc11aa38740 1 -- start start 2024-09-10T10:00:35.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:35.690+0000 7fc11aa38740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dd589ed4b0 0x55dd589eed40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:35.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:35.690+0000 7fc11aa38740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dd58a00660 0x55dd58a18420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:35.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:35.690+0000 7fc11aa38740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dd58a106d0 0x55dd58a18960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:35.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:35.690+0000 7fc11aa38740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc1101acb50 con 0x55dd58a00660 2024-09-10T10:00:35.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:35.690+0000 7fc11aa38740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fc1101ae690 con 0x55dd58a106d0 2024-09-10T10:00:35.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:35.690+0000 7fc11aa38740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fc1101ad220 con 0x55dd589ed4b0 2024-09-10T10:00:35.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:35.690+0000 7fc115400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dd589ed4b0 0x55dd589eed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:35.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:35.690+0000 7fc115400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dd589ed4b0 0x55dd589eed40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:34510/0 (socket says 172.21.15.5:34510) 2024-09-10T10:00:35.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:35.690+0000 7fc115400640 1 -- 172.21.15.5:0/1620227423 learned_addr learned my addr 172.21.15.5:0/1620227423 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T10:00:35.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:35.690+0000 7fc115e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dd58a106d0 0x55dd58a18960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:35.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:35.691+0000 7fc114a00640 1 --2- 172.21.15.5:0/1620227423 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dd58a00660 0x55dd58a18420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:35.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:35.691+0000 7fc115e00640 1 -- 172.21.15.5:0/1620227423 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dd589ed4b0 msgr2=0x55dd589eed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:00:35.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:35.691+0000 7fc115e00640 1 --2- 172.21.15.5:0/1620227423 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dd589ed4b0 0x55dd589eed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:35.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:35.691+0000 7fc115e00640 1 -- 172.21.15.5:0/1620227423 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dd58a00660 msgr2=0x55dd58a18420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:00:35.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:35.692+0000 7fc115e00640 1 --2- 172.21.15.5:0/1620227423 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dd58a00660 0x55dd58a18420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:35.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:35.692+0000 7fc115e00640 1 -- 172.21.15.5:0/1620227423 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dd589ed2e0 con 0x55dd58a106d0 2024-09-10T10:00:35.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:35.692+0000 7fc114a00640 1 --2- 172.21.15.5:0/1620227423 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dd58a00660 0x55dd58a18420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:00:35.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:35.693+0000 7fc115e00640 1 --2- 172.21.15.5:0/1620227423 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dd58a106d0 0x55dd58a18960 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7fc108009540 tx=0x7fc108005c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=19c6c0d62821dd99 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:00:35.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:35.694+0000 7fc101c00640 1 -- 172.21.15.5:0/1620227423 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc10800fc60 con 0x55dd58a106d0 2024-09-10T10:00:35.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:35.694+0000 7fc101c00640 1 -- 172.21.15.5:0/1620227423 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc10800fe00 con 0x55dd58a106d0 2024-09-10T10:00:35.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:35.694+0000 7fc101c00640 1 -- 172.21.15.5:0/1620227423 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc108013320 con 0x55dd58a106d0 2024-09-10T10:00:35.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:35.695+0000 7fc11aa38740 1 -- 172.21.15.5:0/1620227423 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dd58a106d0 msgr2=0x55dd58a18960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:35.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:35.695+0000 7fc11aa38740 1 --2- 172.21.15.5:0/1620227423 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dd58a106d0 0x55dd58a18960 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7fc108009540 tx=0x7fc108005c30 comp rx=0 tx=0).stop 2024-09-10T10:00:35.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:35.696+0000 7fc11aa38740 1 -- 172.21.15.5:0/1620227423 shutdown_connections 2024-09-10T10:00:35.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:35.696+0000 7fc11aa38740 1 --2- 172.21.15.5:0/1620227423 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dd58a106d0 0x55dd58a18960 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:35.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:35.696+0000 7fc11aa38740 1 --2- 172.21.15.5:0/1620227423 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dd58a00660 0x55dd58a18420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:35.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:35.696+0000 7fc11aa38740 1 --2- 172.21.15.5:0/1620227423 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dd589ed4b0 0x55dd589eed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:35.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:35.696+0000 7fc11aa38740 1 -- 172.21.15.5:0/1620227423 >> 172.21.15.5:0/1620227423 conn(0x55dd589e9200 msgr2=0x55dd589ee630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:00:35.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:35.697+0000 7fc11aa38740 1 -- 172.21.15.5:0/1620227423 shutdown_connections 2024-09-10T10:00:35.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:35.697+0000 7fc11aa38740 1 -- 172.21.15.5:0/1620227423 wait complete. 2024-09-10T10:00:35.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:35.698+0000 7fc11aa38740 1 -- start start 2024-09-10T10:00:35.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:35.698+0000 7fc11aa38740 1 Processor -- start 2024-09-10T10:00:35.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:35.699+0000 7fc11aa38740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dd589ed4b0 0x55dd58a04f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:35.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:35.699+0000 7fc11aa38740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dd58a00660 0x55dd58a054a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:35.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:35.699+0000 7fc11aa38740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dd58a106d0 0x55dd58a733b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:35.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:35.699+0000 7fc11aa38740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc1101adfc0 con 0x55dd589ed4b0 2024-09-10T10:00:35.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:35.699+0000 7fc11aa38740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fc1101ad8f0 con 0x55dd58a00660 2024-09-10T10:00:35.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:35.699+0000 7fc11aa38740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fc1101aab30 con 0x55dd58a106d0 2024-09-10T10:00:35.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:35.699+0000 7fc11aa38740 10 client.0 _ll_register_callbacks cb 0x7fc104009600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:00:35.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:35.700+0000 7fc114a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dd58a00660 0x55dd58a054a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:35.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:35.700+0000 7fc114a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dd58a00660 0x55dd58a054a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:34188/0 (socket says 172.21.15.5:34188) 2024-09-10T10:00:35.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:35.700+0000 7fc115e00640 1 --2- 172.21.15.5:0/1398148343 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dd58a106d0 0x55dd58a733b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:35.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:35.700+0000 7fc114a00640 1 -- 172.21.15.5:0/1398148343 learned_addr learned my addr 172.21.15.5:0/1398148343 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T10:00:35.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:35.701+0000 7fc115400640 1 --2- 172.21.15.5:0/1398148343 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dd589ed4b0 0x55dd58a04f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:35.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:35.701+0000 7fc114a00640 1 -- 172.21.15.5:0/1398148343 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dd58a106d0 msgr2=0x55dd58a733b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:00:35.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:35.701+0000 7fc114a00640 1 --2- 172.21.15.5:0/1398148343 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dd58a106d0 0x55dd58a733b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:35.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:35.701+0000 7fc114a00640 1 -- 172.21.15.5:0/1398148343 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dd589ed4b0 msgr2=0x55dd58a04f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:00:35.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:35.702+0000 7fc114a00640 1 --2- 172.21.15.5:0/1398148343 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dd589ed4b0 0x55dd58a04f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:35.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:35.702+0000 7fc114a00640 1 -- 172.21.15.5:0/1398148343 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dd58a73a70 con 0x55dd58a00660 2024-09-10T10:00:35.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:35.704+0000 7fc114a00640 1 --2- 172.21.15.5:0/1398148343 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dd58a00660 0x55dd58a054a0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x55dd589febe0 tx=0x7fc0fc0207e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=290d22d1169db0d2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:00:35.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:35.704+0000 7fc0f6a00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-09-10T10:00:35.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:35.704+0000 7fc0f6a00640 1 -- 172.21.15.5:0/1398148343 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0fc005220 con 0x55dd58a00660 2024-09-10T10:00:35.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:35.704+0000 7fc0f6a00640 1 -- 172.21.15.5:0/1398148343 <== mon.2 v2:172.21.15.102:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fc0fc0053c0 con 0x55dd58a00660 2024-09-10T10:00:35.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:35.705+0000 7fc11aa38740 1 -- 172.21.15.5:0/1398148343 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc10800fe00 con 0x55dd58a00660 2024-09-10T10:00:35.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:35.706+0000 7fc0f6a00640 1 -- 172.21.15.5:0/1398148343 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7fc0fc0055c0 con 0x55dd58a00660 2024-09-10T10:00:35.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:35.706+0000 7fc11aa38740 10 client.29648 fetch_fsmap learned FSMap version 42 2024-09-10T10:00:35.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:35.706+0000 7fc11aa38740 1 -- 172.21.15.5:0/1398148343 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55dd58a7bf70 con 0x55dd58a00660 2024-09-10T10:00:35.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:35.707+0000 7fc0f6a00640 1 -- 172.21.15.5:0/1398148343 <== mon.2 v2:172.21.15.102:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc10800fe00 con 0x55dd58a00660 2024-09-10T10:00:35.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:35.707+0000 7fc0f6a00640 1 -- 172.21.15.5:0/1398148343 <== mon.2 v2:172.21.15.102:3300/0 5 ==== fsmap(e 42) ==== 8370+0+0 (secure 0 0 0) 0x7fc0fc025030 con 0x55dd58a00660 2024-09-10T10:00:35.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:35.707+0000 7fc11aa38740 10 client.29648 fetch_fsmap finished waiting for FSMap version 42 2024-09-10T10:00:35.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:35.707+0000 7fc11aa38740 10 client.29648 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T10:00:35.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:35.707+0000 7fc11aa38740 20 client.29648 populate_metadata read hostname 'smithi005' 2024-09-10T10:00:35.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:35.708+0000 7fc11aa38740 1 --2- 172.21.15.5:0/1398148343 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55dd58a7c680 0x55dd58a7eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:35.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:35.708+0000 7fc11aa38740 4 client.29648 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T10:00:35.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:35.708+0000 7fc11aa38740 1 -- 172.21.15.5:0/1398148343 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55dd5895ae60 con 0x55dd58a7c680 2024-09-10T10:00:35.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:35.708+0000 7fc115400640 1 --2- 172.21.15.5:0/1398148343 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55dd58a7c680 0x55dd58a7eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:35.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:35.711+0000 7fc115400640 1 --2- 172.21.15.5:0/1398148343 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55dd58a7c680 0x55dd58a7eaa0 crc :-1 s=READY pgs=302 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ea6fc9ccf897fede server_cookie=1dd95cd1db54aaf9 in_seq=0 out_seq=0 2024-09-10T10:00:35.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:35.711+0000 7fc0f6a00640 10 client.29648 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T10:00:35.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:35.712+0000 7fc0f6a00640 1 -- 172.21.15.5:0/1398148343 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55dd5895ae60 con 0x55dd58a7c680 2024-09-10T10:00:35.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:35.712+0000 7fc0f6a00640 10 client.29648 handle_command_reply: tid=0 2024-09-10T10:00:35.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:35.712+0000 7fc11aa38740 1 client.29648 shutdown 2024-09-10T10:00:35.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:35.712+0000 7fc11aa38740 1 -- 172.21.15.5:0/1398148343 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dd58a00660 msgr2=0x55dd58a054a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:35.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:35.712+0000 7fc11aa38740 1 --2- 172.21.15.5:0/1398148343 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dd58a00660 0x55dd58a054a0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x55dd589febe0 tx=0x7fc0fc0207e0 comp rx=0 tx=0).stop 2024-09-10T10:00:35.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:35.712+0000 7fc11aa38740 1 -- 172.21.15.5:0/1398148343 shutdown_connections 2024-09-10T10:00:35.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:35.712+0000 7fc11aa38740 1 --2- 172.21.15.5:0/1398148343 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55dd58a7c680 0x55dd58a7eaa0 crc :-1 s=READY pgs=302 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:35.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:35.712+0000 7fc11aa38740 1 --2- 172.21.15.5:0/1398148343 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dd58a106d0 0x55dd58a733b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:35.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:35.712+0000 7fc11aa38740 1 --2- 172.21.15.5:0/1398148343 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dd58a00660 0x55dd58a054a0 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:35.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:35.712+0000 7fc11aa38740 1 --2- 172.21.15.5:0/1398148343 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dd589ed4b0 0x55dd58a04f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:35.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:35.713+0000 7fc11aa38740 1 -- 172.21.15.5:0/1398148343 >> 172.21.15.5:0/1398148343 conn(0x55dd589e9200 msgr2=0x55dd589fdd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:00:35.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:35.713+0000 7fc0f6a00640 0 client.29648 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T10:00:35.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:35.713+0000 7fc11aa38740 1 -- 172.21.15.5:0/1398148343 shutdown_connections 2024-09-10T10:00:35.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:35.713+0000 7fc11aa38740 1 -- 172.21.15.5:0/1398148343 wait complete. 2024-09-10T10:00:35.715 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-10T10:00:35.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:35.713+0000 7fc11aa38740 20 client.29648 trim_cache size 0 max 16384 2024-09-10T10:00:35.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:35.714+0000 7fc117e00640 1 -- 172.21.15.5:0/1611617915 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fc0f80a2b80 msgr2=0x7fc0f80a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:35.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:35.714+0000 7fc117e00640 1 --2- 172.21.15.5:0/1611617915 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fc0f80a2b80 0x7fc0f80a5040 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fc108023000 tx=0x7fc108022000 comp rx=0 tx=0).stop 2024-09-10T10:00:35.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:35.714+0000 7fc117e00640 1 -- 172.21.15.5:0/1611617915 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc110101df0 msgr2=0x7fc1100744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:35.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:35.714+0000 7fc117e00640 1 --2- 172.21.15.5:0/1611617915 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc110101df0 0x7fc1100744f0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7fc10402a910 tx=0x7fc104003b20 comp rx=0 tx=0).stop 2024-09-10T10:00:35.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:35.715+0000 7fc117e00640 1 -- 172.21.15.5:0/1611617915 shutdown_connections 2024-09-10T10:00:35.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:35.715+0000 7fc117e00640 1 --2- 172.21.15.5:0/1611617915 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fc0f80a2b80 0x7fc0f80a5040 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:35.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:35.715+0000 7fc117e00640 1 --2- 172.21.15.5:0/1611617915 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc110106f20 0x7fc110074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:35.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:35.715+0000 7fc117e00640 1 --2- 172.21.15.5:0/1611617915 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc1101027a0 0x7fc110074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:35.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:35.715+0000 7fc117e00640 1 --2- 172.21.15.5:0/1611617915 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc110101df0 0x7fc1100744f0 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:35.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:35.715+0000 7fc117e00640 1 -- 172.21.15.5:0/1611617915 >> 172.21.15.5:0/1611617915 conn(0x7fc1100fd8b0 msgr2=0x7fc1100feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:00:35.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:35.715+0000 7fc117e00640 1 -- 172.21.15.5:0/1611617915 shutdown_connections 2024-09-10T10:00:35.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:35.715+0000 7fc117e00640 1 -- 172.21.15.5:0/1611617915 wait complete. 2024-09-10T10:00:36.034 INFO:tasks.workunit.client.0.smithi005.stdout: 10 133271 13.57 MB/sec execute 374 sec latency 54.593 ms 2024-09-10T10:00:36.099 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:00:35 smithi005 ceph-mon[30350]: pgmap v1060: 129 pgs: 129 active+clean; 690 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 536 KiB/s rd, 133 KiB/s wr, 38 op/s 2024-09-10T10:00:36.099 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:00:35 smithi005 ceph-mon[30350]: from='client.? 172.21.15.5:0/988153599' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:00:36.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:00:35 smithi159 ceph-mon[35513]: pgmap v1060: 129 pgs: 129 active+clean; 690 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 536 KiB/s rd, 133 KiB/s wr, 38 op/s 2024-09-10T10:00:36.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:00:35 smithi159 ceph-mon[35513]: from='client.? 172.21.15.5:0/988153599' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:00:36.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:00:35 smithi102 ceph-mon[35442]: pgmap v1060: 129 pgs: 129 active+clean; 690 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 536 KiB/s rd, 133 KiB/s wr, 38 op/s 2024-09-10T10:00:36.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:00:35 smithi102 ceph-mon[35442]: from='client.? 172.21.15.5:0/988153599' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:00:36.951 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24542 quiesce db / --timeout 90 --expiration 154.2 --await-for 45.0' 2024-09-10T10:00:36.952 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24542 quiesce db / --timeout 90 --expiration 154.2 --await-for 45.0 2024-09-10T10:00:37.034 INFO:tasks.workunit.client.0.smithi005.stdout: 10 133922 13.64 MB/sec execute 375 sec latency 67.723 ms 2024-09-10T10:00:37.590 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:00:38.025 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:00:37 smithi005 ceph-mon[30350]: pgmap v1061: 129 pgs: 129 active+clean; 691 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 675 KiB/s wr, 122 op/s 2024-09-10T10:00:38.034 INFO:tasks.workunit.client.0.smithi005.stdout: 10 134269 13.65 MB/sec execute 376 sec latency 373.049 ms 2024-09-10T10:00:38.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:00:37 smithi159 ceph-mon[35513]: pgmap v1061: 129 pgs: 129 active+clean; 691 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 675 KiB/s wr, 122 op/s 2024-09-10T10:00:38.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:00:37 smithi102 ceph-mon[35442]: pgmap v1061: 129 pgs: 129 active+clean; 691 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 675 KiB/s wr, 122 op/s 2024-09-10T10:00:38.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:38.402+0000 7f53ab200640 1 -- 172.21.15.5:0/4169539183 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f53ac06ea30 msgr2=0x7f53ac06ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:38.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:38.402+0000 7f53ab200640 1 --2- 172.21.15.5:0/4169539183 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f53ac06ea30 0x7f53ac06ee70 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f5398009860 tx=0x7f539802a340 comp rx=0 tx=0).stop 2024-09-10T10:00:38.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:38.403+0000 7f53ab200640 1 -- 172.21.15.5:0/4169539183 shutdown_connections 2024-09-10T10:00:38.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:38.403+0000 7f53ab200640 1 --2- 172.21.15.5:0/4169539183 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f53ac11b140 0x7f53ac11d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:38.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:38.403+0000 7f53ab200640 1 --2- 172.21.15.5:0/4169539183 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f53ac06ea30 0x7f53ac06ee70 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:38.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:38.403+0000 7f53ab200640 1 --2- 172.21.15.5:0/4169539183 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f53ac110190 0x7f53ac06e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:38.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:38.403+0000 7f53ab200640 1 -- 172.21.15.5:0/4169539183 >> 172.21.15.5:0/4169539183 conn(0x7f53ac073180 msgr2=0x7f53ac073590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:00:38.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:38.403+0000 7f53ab200640 1 -- 172.21.15.5:0/4169539183 shutdown_connections 2024-09-10T10:00:38.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:38.403+0000 7f53ab200640 1 -- 172.21.15.5:0/4169539183 wait complete. 2024-09-10T10:00:38.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:38.405+0000 7f53ab200640 1 Processor -- start 2024-09-10T10:00:38.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:38.405+0000 7f53ab200640 1 -- start start 2024-09-10T10:00:38.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:38.406+0000 7f53ab200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f53ac06ea30 0x7f53ac1a70f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:38.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:38.406+0000 7f53ab200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f53ac110190 0x7f53ac1a7630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:38.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:38.406+0000 7f53ab200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f53ac11b140 0x7f53ac1ab9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:38.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:38.406+0000 7f53ab200640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f53ac120b90 con 0x7f53ac11b140 2024-09-10T10:00:38.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:38.406+0000 7f53ab200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f53ac120a10 con 0x7f53ac110190 2024-09-10T10:00:38.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:38.406+0000 7f53ab200640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f53ac120d10 con 0x7f53ac06ea30 2024-09-10T10:00:38.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:38.407+0000 7f53a9e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f53ac06ea30 0x7f53ac1a70f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:38.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:38.407+0000 7f53a9e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f53ac06ea30 0x7f53ac1a70f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:34538/0 (socket says 172.21.15.5:34538) 2024-09-10T10:00:38.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:38.407+0000 7f53a9e00640 1 -- 172.21.15.5:0/3143641335 learned_addr learned my addr 172.21.15.5:0/3143641335 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T10:00:38.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:38.408+0000 7f53a9400640 1 --2- 172.21.15.5:0/3143641335 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f53ac110190 0x7f53ac1a7630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:38.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:38.408+0000 7f53a9e00640 1 -- 172.21.15.5:0/3143641335 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f53ac110190 msgr2=0x7f53ac1a7630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:38.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:38.408+0000 7f53a9e00640 1 --2- 172.21.15.5:0/3143641335 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f53ac110190 0x7f53ac1a7630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:38.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:38.408+0000 7f53a9e00640 1 -- 172.21.15.5:0/3143641335 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f53ac11b140 msgr2=0x7f53ac1ab9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:38.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:38.408+0000 7f53a9e00640 1 --2- 172.21.15.5:0/3143641335 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f53ac11b140 0x7f53ac1ab9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:38.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:38.408+0000 7f53a9e00640 1 -- 172.21.15.5:0/3143641335 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53ac1ac0a0 con 0x7f53ac06ea30 2024-09-10T10:00:38.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:38.408+0000 7f53a9400640 1 --2- 172.21.15.5:0/3143641335 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f53ac110190 0x7f53ac1a7630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:00:38.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:38.409+0000 7f53a9e00640 1 --2- 172.21.15.5:0/3143641335 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f53ac06ea30 0x7f53ac1a70f0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f539c00d950 tx=0x7f539c00de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:00:38.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:38.409+0000 7f53a2a00640 1 -- 172.21.15.5:0/3143641335 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f539c017070 con 0x7f53ac06ea30 2024-09-10T10:00:38.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:38.409+0000 7f53a2a00640 1 -- 172.21.15.5:0/3143641335 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f539c009c30 con 0x7f53ac06ea30 2024-09-10T10:00:38.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:38.409+0000 7f53a2a00640 1 -- 172.21.15.5:0/3143641335 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f539c00f690 con 0x7f53ac06ea30 2024-09-10T10:00:38.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:38.409+0000 7f53ab200640 1 -- 172.21.15.5:0/3143641335 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53ac1ac390 con 0x7f53ac06ea30 2024-09-10T10:00:38.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:38.409+0000 7f53ab200640 1 -- 172.21.15.5:0/3143641335 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53ac1b3c70 con 0x7f53ac06ea30 2024-09-10T10:00:38.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:38.413+0000 7f53b3a4c740 1 Processor -- start 2024-09-10T10:00:38.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:38.413+0000 7f53b3a4c740 1 -- start start 2024-09-10T10:00:38.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:38.414+0000 7f53b3a4c740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d1c461c560 0x55d1c461c940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:38.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:38.414+0000 7f53b3a4c740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d1c461ce80 0x55d1c4627200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:38.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:38.415+0000 7f53aa800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d1c461c560 0x55d1c461c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:38.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:38.415+0000 7f53aa800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d1c461c560 0x55d1c461c940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57906/0 (socket says 172.21.15.5:57906) 2024-09-10T10:00:38.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:38.416+0000 7f53b3a4c740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d1c4627740 0x55d1c4629b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:38.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:38.416+0000 7f53b3a4c740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55d1c4582530 con 0x55d1c461c560 2024-09-10T10:00:38.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:38.416+0000 7f53b3a4c740 1 -- 172.21.15.5:0/171936596 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55d1c4529860 con 0x55d1c461ce80 2024-09-10T10:00:38.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:38.416+0000 7f53b3a4c740 1 -- 172.21.15.5:0/171936596 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55d1c457e4f0 con 0x55d1c4627740 2024-09-10T10:00:38.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:38.416+0000 7f53aa800640 1 -- 172.21.15.5:0/171936596 learned_addr learned my addr 172.21.15.5:0/171936596 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T10:00:38.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:38.416+0000 7f53a9400640 1 --2- 172.21.15.5:0/171936596 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d1c4627740 0x55d1c4629b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:38.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:38.417+0000 7f53aa800640 1 -- 172.21.15.5:0/171936596 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d1c4627740 msgr2=0x55d1c4629b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:00:38.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:38.417+0000 7f53aa800640 1 --2- 172.21.15.5:0/171936596 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d1c4627740 0x55d1c4629b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:38.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:38.417+0000 7f53aa800640 1 -- 172.21.15.5:0/171936596 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d1c461ce80 msgr2=0x55d1c4627200 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T10:00:38.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:38.417+0000 7f53aa800640 1 --2- 172.21.15.5:0/171936596 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d1c461ce80 0x55d1c4627200 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:38.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:38.417+0000 7f53aa800640 1 -- 172.21.15.5:0/171936596 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d1c42f3b90 con 0x55d1c461c560 2024-09-10T10:00:38.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:38.418+0000 7f53a2a00640 1 -- 172.21.15.5:0/3143641335 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f539c0109d0 con 0x7f53ac06ea30 2024-09-10T10:00:38.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:38.419+0000 7f53aa800640 1 --2- 172.21.15.5:0/171936596 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d1c461c560 0x55d1c461c940 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f53a4009de0 tx=0x7f53a400e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca0647751c64e689 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:00:38.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:38.420+0000 7f5397400640 1 -- 172.21.15.5:0/171936596 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53a4004d20 con 0x55d1c461c560 2024-09-10T10:00:38.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:38.420+0000 7f5397400640 1 -- 172.21.15.5:0/171936596 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f53a4003e50 con 0x55d1c461c560 2024-09-10T10:00:38.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:38.420+0000 7f53a2a00640 1 --2- 172.21.15.5:0/3143641335 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f53900a2b80 0x7f53900a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:38.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:38.420+0000 7f5397400640 1 -- 172.21.15.5:0/171936596 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53a4005080 con 0x55d1c461c560 2024-09-10T10:00:38.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:38.421+0000 7f53a2a00640 1 -- 172.21.15.5:0/3143641335 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f539c0cee40 con 0x7f53ac06ea30 2024-09-10T10:00:38.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:38.421+0000 7f53a9e00640 1 --2- 172.21.15.5:0/3143641335 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f53900a2b80 0x7f53900a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:38.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:38.421+0000 7f53b3a4c740 1 -- 172.21.15.5:0/171936596 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d1c461c560 msgr2=0x55d1c461c940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:38.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:38.421+0000 7f53b3a4c740 1 --2- 172.21.15.5:0/171936596 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d1c461c560 0x55d1c461c940 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f53a4009de0 tx=0x7f53a400e720 comp rx=0 tx=0).stop 2024-09-10T10:00:38.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:38.422+0000 7f53a9e00640 1 --2- 172.21.15.5:0/3143641335 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f53900a2b80 0x7f53900a5040 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f539c097710 tx=0x7f539c005880 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:00:38.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:38.422+0000 7f53b3a4c740 1 -- 172.21.15.5:0/171936596 shutdown_connections 2024-09-10T10:00:38.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:38.422+0000 7f53b3a4c740 1 --2- 172.21.15.5:0/171936596 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d1c4627740 0x55d1c4629b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:38.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:38.422+0000 7f53b3a4c740 1 --2- 172.21.15.5:0/171936596 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d1c461ce80 0x55d1c4627200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:38.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:38.422+0000 7f53b3a4c740 1 --2- 172.21.15.5:0/171936596 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d1c461c560 0x55d1c461c940 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:38.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:38.422+0000 7f53b3a4c740 1 -- 172.21.15.5:0/171936596 >> 172.21.15.5:0/171936596 conn(0x55d1c4618af0 msgr2=0x55d1c4618ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:00:38.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:38.424+0000 7f53b3a4c740 1 -- 172.21.15.5:0/171936596 shutdown_connections 2024-09-10T10:00:38.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:38.424+0000 7f53b3a4c740 1 -- 172.21.15.5:0/171936596 wait complete. 2024-09-10T10:00:38.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:38.424+0000 7f53b3a4c740 1 -- start start 2024-09-10T10:00:38.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:38.425+0000 7f53b3a4c740 1 Processor -- start 2024-09-10T10:00:38.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:38.426+0000 7f53b3a4c740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d1c461c560 0x55d1c46246d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:38.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:38.426+0000 7f53b3a4c740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d1c461ce80 0x55d1c4624c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:38.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:38.426+0000 7f53b3a4c740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d1c4627740 0x55d1c4625150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:38.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:38.426+0000 7f53b3a4c740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55d1c462a360 con 0x55d1c4627740 2024-09-10T10:00:38.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:38.426+0000 7f53b3a4c740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55d1c453ce60 con 0x55d1c461c560 2024-09-10T10:00:38.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:38.426+0000 7f53b3a4c740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55d1c462a4e0 con 0x55d1c461ce80 2024-09-10T10:00:38.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:38.426+0000 7f53b3a4c740 10 client.0 _ll_register_callbacks cb 0x55d1c4541cc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:00:38.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:38.426+0000 7f53a9400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d1c461ce80 0x55d1c4624c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:38.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:38.427+0000 7f53a9400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d1c461ce80 0x55d1c4624c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:34564/0 (socket says 172.21.15.5:34564) 2024-09-10T10:00:38.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:38.427+0000 7f53a9400640 1 -- 172.21.15.5:0/991373027 learned_addr learned my addr 172.21.15.5:0/991373027 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T10:00:38.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:38.427+0000 7f53aa800640 1 --2- 172.21.15.5:0/991373027 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d1c461c560 0x55d1c46246d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:38.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:38.427+0000 7f53aa800640 1 --2- 172.21.15.5:0/991373027 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d1c4627740 0x55d1c4625150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:38.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:38.428+0000 7f53a9400640 1 -- 172.21.15.5:0/991373027 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d1c461c560 msgr2=0x55d1c46246d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:00:38.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:38.428+0000 7f53a9400640 1 --2- 172.21.15.5:0/991373027 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d1c461c560 0x55d1c46246d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:38.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:38.428+0000 7f53a9400640 1 -- 172.21.15.5:0/991373027 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d1c4627740 msgr2=0x55d1c4625150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:00:38.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:38.428+0000 7f53a9400640 1 --2- 172.21.15.5:0/991373027 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d1c4627740 0x55d1c4625150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:38.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:38.428+0000 7f53a9400640 1 -- 172.21.15.5:0/991373027 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d1c4548990 con 0x55d1c461ce80 2024-09-10T10:00:38.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:38.429+0000 7f53a9400640 1 --2- 172.21.15.5:0/991373027 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d1c461ce80 0x55d1c4624c10 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f5398009780 tx=0x7f53980045b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3c9a48fc77f4584 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:00:38.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:38.429+0000 7f538be00640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-09-10T10:00:38.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:38.429+0000 7f538be00640 1 -- 172.21.15.5:0/991373027 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5398041070 con 0x55d1c461ce80 2024-09-10T10:00:38.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:38.429+0000 7f53b3a4c740 1 -- 172.21.15.5:0/991373027 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f53a4003e50 con 0x55d1c461ce80 2024-09-10T10:00:38.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:38.429+0000 7f538be00640 1 -- 172.21.15.5:0/991373027 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f5398030960 con 0x55d1c461ce80 2024-09-10T10:00:38.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:38.430+0000 7f538be00640 1 -- 172.21.15.5:0/991373027 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f539804d050 con 0x55d1c461ce80 2024-09-10T10:00:38.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:38.430+0000 7f53b3a4c740 10 client.29644 fetch_fsmap learned FSMap version 42 2024-09-10T10:00:38.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:38.430+0000 7f53b3a4c740 1 -- 172.21.15.5:0/991373027 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55d1c4636060 con 0x55d1c461ce80 2024-09-10T10:00:38.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:38.430+0000 7f538be00640 1 -- 172.21.15.5:0/991373027 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f53a4003e50 con 0x55d1c461ce80 2024-09-10T10:00:38.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:38.431+0000 7f538be00640 1 -- 172.21.15.5:0/991373027 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 42) ==== 8370+0+0 (secure 0 0 0) 0x7f539803d8e0 con 0x55d1c461ce80 2024-09-10T10:00:38.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:38.431+0000 7f53b3a4c740 10 client.29644 fetch_fsmap finished waiting for FSMap version 42 2024-09-10T10:00:38.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:38.431+0000 7f53b3a4c740 10 client.29644 resolve_mds: validated gid 24542 aka daemon mds.b 2024-09-10T10:00:38.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:38.431+0000 7f53b3a4c740 20 client.29644 populate_metadata read hostname 'smithi005' 2024-09-10T10:00:38.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:38.431+0000 7f53b3a4c740 1 --2- 172.21.15.5:0/991373027 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55d1c4637470 0x55d1c46398a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:38.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:38.432+0000 7f53b3a4c740 4 client.29644 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:00:38.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:38.432+0000 7f53b3a4c740 1 -- 172.21.15.5:0/991373027 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d1c4525310 con 0x55d1c4637470 2024-09-10T10:00:38.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:38.432+0000 7f53aa800640 1 --2- 172.21.15.5:0/991373027 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55d1c4637470 0x55d1c46398a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:38.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:38.433+0000 7f538be00640 10 client.29644 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T10:00:38.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:38.433+0000 7f53aa800640 1 --2- 172.21.15.5:0/991373027 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55d1c4637470 0x55d1c46398a0 crc :-1 s=READY pgs=303 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5eddec89fc3265cc server_cookie=b8a47718887ec12b in_seq=0 out_seq=0 2024-09-10T10:00:38.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:38.434+0000 7f538be00640 1 -- 172.21.15.5:0/991373027 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d1c4525310 con 0x55d1c4637470 2024-09-10T10:00:38.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:38.434+0000 7f538be00640 10 client.29644 handle_command_reply: tid=0 2024-09-10T10:00:38.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:38.435+0000 7f53b3a4c740 1 client.29644 shutdown 2024-09-10T10:00:38.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:38.436+0000 7f53b3a4c740 1 -- 172.21.15.5:0/991373027 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d1c461ce80 msgr2=0x55d1c4624c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:38.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:38.436+0000 7f53b3a4c740 1 --2- 172.21.15.5:0/991373027 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d1c461ce80 0x55d1c4624c10 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f5398009780 tx=0x7f53980045b0 comp rx=0 tx=0).stop 2024-09-10T10:00:38.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:38.436+0000 7f53b3a4c740 1 -- 172.21.15.5:0/991373027 shutdown_connections 2024-09-10T10:00:38.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:38.436+0000 7f53b3a4c740 1 --2- 172.21.15.5:0/991373027 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55d1c4637470 0x55d1c46398a0 crc :-1 s=READY pgs=303 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:38.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:38.436+0000 7f53b3a4c740 1 --2- 172.21.15.5:0/991373027 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d1c4627740 0x55d1c4625150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:38.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:38.436+0000 7f53b3a4c740 1 --2- 172.21.15.5:0/991373027 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d1c461ce80 0x55d1c4624c10 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:38.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:38.436+0000 7f53b3a4c740 1 --2- 172.21.15.5:0/991373027 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d1c461c560 0x55d1c46246d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:38.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:38.436+0000 7f53b3a4c740 1 -- 172.21.15.5:0/991373027 >> 172.21.15.5:0/991373027 conn(0x55d1c4618af0 msgr2=0x55d1c461e820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:00:38.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:38.437+0000 7f538be00640 0 client.29644 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T10:00:38.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:38.437+0000 7f53b3a4c740 1 -- 172.21.15.5:0/991373027 shutdown_connections 2024-09-10T10:00:38.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:38.437+0000 7f53b3a4c740 1 -- 172.21.15.5:0/991373027 wait complete. 2024-09-10T10:00:38.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:38.439+0000 7f53b3a4c740 20 client.29644 trim_cache size 0 max 16384 2024-09-10T10:00:38.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:38.469+0000 7f53b3a4c740 1 Processor -- start 2024-09-10T10:00:38.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:38.470+0000 7f53b3a4c740 1 -- start start 2024-09-10T10:00:38.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:38.470+0000 7f53b3a4c740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d1c461c560 0x55d1c4617b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:38.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:38.470+0000 7f53b3a4c740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d1c461ce80 0x55d1c463f170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:38.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:38.470+0000 7f53b3a4c740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d1c4627740 0x55d1c463f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:38.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:38.470+0000 7f53b3a4c740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f53ac1bdbf0 con 0x55d1c461c560 2024-09-10T10:00:38.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:38.470+0000 7f53b3a4c740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f53ac1bf730 con 0x55d1c4627740 2024-09-10T10:00:38.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:38.470+0000 7f53b3a4c740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f53ac1be2c0 con 0x55d1c461ce80 2024-09-10T10:00:38.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:38.471+0000 7f53aa800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d1c461c560 0x55d1c4617b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:38.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:38.471+0000 7f53aa800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d1c4627740 0x55d1c463f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:38.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:38.471+0000 7f53aa800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d1c4627740 0x55d1c463f6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:34222/0 (socket says 172.21.15.5:34222) 2024-09-10T10:00:38.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:38.471+0000 7f53aa800640 1 -- 172.21.15.5:0/4231809233 learned_addr learned my addr 172.21.15.5:0/4231809233 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T10:00:38.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:38.472+0000 7f53a9400640 1 --2- 172.21.15.5:0/4231809233 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d1c461ce80 0x55d1c463f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:38.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:38.472+0000 7f53aa800640 1 -- 172.21.15.5:0/4231809233 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d1c461ce80 msgr2=0x55d1c463f170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:00:38.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:38.472+0000 7f53aa800640 1 --2- 172.21.15.5:0/4231809233 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d1c461ce80 0x55d1c463f170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:38.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:38.472+0000 7f53aa800640 1 -- 172.21.15.5:0/4231809233 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d1c461c560 msgr2=0x55d1c4617b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:00:38.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:38.472+0000 7f53aa800640 1 --2- 172.21.15.5:0/4231809233 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d1c461c560 0x55d1c4617b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:38.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:38.472+0000 7f53aa800640 1 -- 172.21.15.5:0/4231809233 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d1c461f510 con 0x55d1c4627740 2024-09-10T10:00:38.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:38.473+0000 7f53aa800640 1 --2- 172.21.15.5:0/4231809233 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d1c4627740 0x55d1c463f6b0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f53a4009d80 tx=0x7f53a4000f40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4f247a450b1dfeb9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:00:38.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:38.473+0000 7f5396a00640 1 -- 172.21.15.5:0/4231809233 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53a4004b20 con 0x55d1c4627740 2024-09-10T10:00:38.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:38.473+0000 7f5396a00640 1 -- 172.21.15.5:0/4231809233 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f53a4004cc0 con 0x55d1c4627740 2024-09-10T10:00:38.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:38.473+0000 7f5396a00640 1 -- 172.21.15.5:0/4231809233 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53a4005310 con 0x55d1c4627740 2024-09-10T10:00:38.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:38.474+0000 7f53b3a4c740 1 -- 172.21.15.5:0/4231809233 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d1c4627740 msgr2=0x55d1c463f6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:38.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:38.474+0000 7f53b3a4c740 1 --2- 172.21.15.5:0/4231809233 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d1c4627740 0x55d1c463f6b0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f53a4009d80 tx=0x7f53a4000f40 comp rx=0 tx=0).stop 2024-09-10T10:00:38.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:38.474+0000 7f53b3a4c740 1 -- 172.21.15.5:0/4231809233 shutdown_connections 2024-09-10T10:00:38.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:38.474+0000 7f53b3a4c740 1 --2- 172.21.15.5:0/4231809233 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d1c4627740 0x55d1c463f6b0 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:38.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:38.474+0000 7f53b3a4c740 1 --2- 172.21.15.5:0/4231809233 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d1c461ce80 0x55d1c463f170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:38.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:38.474+0000 7f53b3a4c740 1 --2- 172.21.15.5:0/4231809233 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d1c461c560 0x55d1c4617b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:38.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:38.474+0000 7f53b3a4c740 1 -- 172.21.15.5:0/4231809233 >> 172.21.15.5:0/4231809233 conn(0x55d1c4618af0 msgr2=0x55d1c4617450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:00:38.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:38.476+0000 7f53b3a4c740 1 -- 172.21.15.5:0/4231809233 shutdown_connections 2024-09-10T10:00:38.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:38.476+0000 7f53b3a4c740 1 -- 172.21.15.5:0/4231809233 wait complete. 2024-09-10T10:00:38.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:38.476+0000 7f53b3a4c740 1 -- start start 2024-09-10T10:00:38.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:38.476+0000 7f53b3a4c740 1 Processor -- start 2024-09-10T10:00:38.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:38.477+0000 7f53b3a4c740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d1c461c560 0x55d1c462fa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:38.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:38.477+0000 7f53b3a4c740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d1c461ce80 0x55d1c462ff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:38.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:38.477+0000 7f53aa800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d1c461c560 0x55d1c462fa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:38.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:38.477+0000 7f53aa800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d1c461c560 0x55d1c462fa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:34232/0 (socket says 172.21.15.5:34232) 2024-09-10T10:00:38.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:38.478+0000 7f53b3a4c740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d1c4627740 0x55d1c462c820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:38.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:38.478+0000 7f53b3a4c740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f53ac1bf060 con 0x55d1c4627740 2024-09-10T10:00:38.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:38.478+0000 7f53aa800640 1 -- 172.21.15.5:0/1991820904 learned_addr learned my addr 172.21.15.5:0/1991820904 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T10:00:38.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:38.478+0000 7f53b3a4c740 1 -- 172.21.15.5:0/1991820904 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f53ac1be990 con 0x55d1c461c560 2024-09-10T10:00:38.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:38.478+0000 7f53b3a4c740 1 -- 172.21.15.5:0/1991820904 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f53ac1bbbd0 con 0x55d1c461ce80 2024-09-10T10:00:38.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:38.478+0000 7f53b3a4c740 10 client.0 _ll_register_callbacks cb 0x7f53980096b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:00:38.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:38.478+0000 7f53a9400640 1 --2- 172.21.15.5:0/1991820904 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d1c461ce80 0x55d1c462ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:38.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:38.478+0000 7f53aa800640 1 --2- 172.21.15.5:0/1991820904 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d1c4627740 0x55d1c462c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:38.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:38.479+0000 7f53aa800640 1 -- 172.21.15.5:0/1991820904 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d1c461ce80 msgr2=0x55d1c462ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:00:38.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:38.479+0000 7f53aa800640 1 --2- 172.21.15.5:0/1991820904 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d1c461ce80 0x55d1c462ff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:38.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:38.479+0000 7f53aa800640 1 -- 172.21.15.5:0/1991820904 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d1c4627740 msgr2=0x55d1c462c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:00:38.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:38.479+0000 7f53aa800640 1 --2- 172.21.15.5:0/1991820904 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d1c4627740 0x55d1c462c820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:38.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:38.479+0000 7f53aa800640 1 -- 172.21.15.5:0/1991820904 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d1c461ccb0 con 0x55d1c461c560 2024-09-10T10:00:38.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:38.479+0000 7f53a9400640 1 --2- 172.21.15.5:0/1991820904 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d1c461ce80 0x55d1c462ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:00:38.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:38.480+0000 7f53aa800640 1 --2- 172.21.15.5:0/1991820904 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d1c461c560 0x55d1c462fa50 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f53a400e760 tx=0x7f53a4029db0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7fbaf10695620c4a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:00:38.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:38.480+0000 7f538be00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-09-10T10:00:38.484 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-10T10:00:38.484 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-10T10:00:38.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:38.481+0000 7f538be00640 1 -- 172.21.15.5:0/1991820904 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53a403b060 con 0x55d1c461c560 2024-09-10T10:00:38.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:38.481+0000 7f538be00640 1 -- 172.21.15.5:0/1991820904 <== mon.2 v2:172.21.15.102:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f53a4029930 con 0x55d1c461c560 2024-09-10T10:00:38.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:38.481+0000 7f53b3a4c740 1 -- 172.21.15.5:0/1991820904 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f53a4004cc0 con 0x55d1c461c560 2024-09-10T10:00:38.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:38.482+0000 7f538be00640 1 -- 172.21.15.5:0/1991820904 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f53a4029b30 con 0x55d1c461c560 2024-09-10T10:00:38.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:38.482+0000 7f53b3a4c740 10 client.29663 fetch_fsmap learned FSMap version 42 2024-09-10T10:00:38.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:38.482+0000 7f53b3a4c740 1 -- 172.21.15.5:0/1991820904 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55d1c46a2b10 con 0x55d1c461c560 2024-09-10T10:00:38.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:38.483+0000 7f538be00640 1 -- 172.21.15.5:0/1991820904 <== mon.2 v2:172.21.15.102:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f53a4004cc0 con 0x55d1c461c560 2024-09-10T10:00:38.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:38.483+0000 7f538be00640 1 -- 172.21.15.5:0/1991820904 <== mon.2 v2:172.21.15.102:3300/0 5 ==== fsmap(e 42) ==== 8370+0+0 (secure 0 0 0) 0x7f53a402f070 con 0x55d1c461c560 2024-09-10T10:00:38.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:38.483+0000 7f53b3a4c740 10 client.29663 fetch_fsmap finished waiting for FSMap version 42 2024-09-10T10:00:38.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:38.483+0000 7f53b3a4c740 10 client.29663 resolve_mds: validated gid 24542 aka daemon mds.b 2024-09-10T10:00:38.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:38.483+0000 7f53b3a4c740 20 client.29663 populate_metadata read hostname 'smithi005' 2024-09-10T10:00:38.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:38.483+0000 7f53b3a4c740 1 --2- 172.21.15.5:0/1991820904 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55d1c46a2fa0 0x55d1c46a5390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:38.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:38.483+0000 7f53b3a4c740 4 client.29663 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 154.2, "await_for": 45.0}] 2024-09-10T10:00:38.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:38.483+0000 7f53b3a4c740 1 -- 172.21.15.5:0/1991820904 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 154.2, "await_for": 45.0}) -- 0x55d1c458fc60 con 0x55d1c46a2fa0 2024-09-10T10:00:38.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:38.484+0000 7f53a9400640 1 --2- 172.21.15.5:0/1991820904 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55d1c46a2fa0 0x55d1c46a5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:38.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:38.485+0000 7f53a9400640 1 --2- 172.21.15.5:0/1991820904 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55d1c46a2fa0 0x55d1c46a5390 crc :-1 s=READY pgs=304 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=47df806a2552acf6 server_cookie=2453eaf5f17e75ac in_seq=0 out_seq=0 2024-09-10T10:00:38.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:38.485+0000 7f538be00640 10 client.29663 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T10:00:39.035 INFO:tasks.workunit.client.0.smithi005.stdout: 10 134559 13.67 MB/sec execute 377 sec latency 549.650 ms 2024-09-10T10:00:39.044 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:00:39.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:39.319+0000 7f538be00640 1 -- 172.21.15.5:0/1991820904 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55d1c458fc60 con 0x55d1c46a2fa0 2024-09-10T10:00:39.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:39.319+0000 7f538be00640 10 client.29663 handle_command_reply: tid=0 2024-09-10T10:00:39.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:39.319+0000 7f53b3a4c740 1 client.29663 shutdown 2024-09-10T10:00:39.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:39.319+0000 7f53b3a4c740 1 -- 172.21.15.5:0/1991820904 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d1c461c560 msgr2=0x55d1c462fa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:39.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:39.319+0000 7f53b3a4c740 1 --2- 172.21.15.5:0/1991820904 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d1c461c560 0x55d1c462fa50 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f53a400e760 tx=0x7f53a4029db0 comp rx=0 tx=0).stop 2024-09-10T10:00:39.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:39.320+0000 7f53b3a4c740 1 -- 172.21.15.5:0/1991820904 shutdown_connections 2024-09-10T10:00:39.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:39.320+0000 7f53b3a4c740 1 --2- 172.21.15.5:0/1991820904 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55d1c46a2fa0 0x55d1c46a5390 crc :-1 s=READY pgs=304 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:39.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:39.320+0000 7f53b3a4c740 1 --2- 172.21.15.5:0/1991820904 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d1c4627740 0x55d1c462c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:39.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:39.320+0000 7f53b3a4c740 1 --2- 172.21.15.5:0/1991820904 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d1c461ce80 0x55d1c462ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:39.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:39.320+0000 7f53b3a4c740 1 --2- 172.21.15.5:0/1991820904 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d1c461c560 0x55d1c462fa50 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:39.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:39.320+0000 7f538be00640 0 client.29663 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T10:00:39.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:39.320+0000 7f53b3a4c740 1 -- 172.21.15.5:0/1991820904 >> 172.21.15.5:0/1991820904 conn(0x55d1c4618af0 msgr2=0x55d1c4617c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:00:39.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:39.320+0000 7f53b3a4c740 1 -- 172.21.15.5:0/1991820904 shutdown_connections 2024-09-10T10:00:39.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:39.320+0000 7f53b3a4c740 1 -- 172.21.15.5:0/1991820904 wait complete. 2024-09-10T10:00:39.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:39.320+0000 7f53b3a4c740 20 client.29663 trim_cache size 0 max 16384 2024-09-10T10:00:39.323 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-10T10:00:39.323 INFO:teuthology.orchestra.run.smithi005.stdout: "epoch": 42, 2024-09-10T10:00:39.324 INFO:teuthology.orchestra.run.smithi005.stdout: "leader": 24542, 2024-09-10T10:00:39.324 INFO:teuthology.orchestra.run.smithi005.stdout: "set_version": 78, 2024-09-10T10:00:39.324 INFO:teuthology.orchestra.run.smithi005.stdout: "sets": { 2024-09-10T10:00:39.324 INFO:teuthology.orchestra.run.smithi005.stdout: "8aeff82a": { 2024-09-10T10:00:39.324 INFO:teuthology.orchestra.run.smithi005.stdout: "version": 78, 2024-09-10T10:00:39.324 INFO:teuthology.orchestra.run.smithi005.stdout: "age_ref": 0.0, 2024-09-10T10:00:39.324 INFO:teuthology.orchestra.run.smithi005.stdout: "state": { 2024-09-10T10:00:39.324 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "QUIESCED", 2024-09-10T10:00:39.324 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0 2024-09-10T10:00:39.324 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:00:39.324 INFO:teuthology.orchestra.run.smithi005.stdout: "timeout": 90.0, 2024-09-10T10:00:39.324 INFO:teuthology.orchestra.run.smithi005.stdout: "expiration": 154.2, 2024-09-10T10:00:39.325 INFO:teuthology.orchestra.run.smithi005.stdout: "members": { 2024-09-10T10:00:39.325 INFO:teuthology.orchestra.run.smithi005.stdout: "file:/": { 2024-09-10T10:00:39.325 INFO:teuthology.orchestra.run.smithi005.stdout: "excluded": false, 2024-09-10T10:00:39.325 INFO:teuthology.orchestra.run.smithi005.stdout: "state": { 2024-09-10T10:00:39.325 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "QUIESCED", 2024-09-10T10:00:39.325 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0 2024-09-10T10:00:39.325 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T10:00:39.325 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T10:00:39.325 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T10:00:39.325 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T10:00:39.325 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T10:00:39.325 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-10T10:00:39.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:39.321+0000 7f53ab200640 1 -- 172.21.15.5:0/3143641335 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f53900a2b80 msgr2=0x7f53900a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:39.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:39.321+0000 7f53ab200640 1 --2- 172.21.15.5:0/3143641335 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f53900a2b80 0x7f53900a5040 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f539c097710 tx=0x7f539c005880 comp rx=0 tx=0).stop 2024-09-10T10:00:39.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:39.321+0000 7f53ab200640 1 -- 172.21.15.5:0/3143641335 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f53ac06ea30 msgr2=0x7f53ac1a70f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:39.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:39.321+0000 7f53ab200640 1 --2- 172.21.15.5:0/3143641335 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f53ac06ea30 0x7f53ac1a70f0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f539c00d950 tx=0x7f539c00de10 comp rx=0 tx=0).stop 2024-09-10T10:00:39.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:39.321+0000 7f53ab200640 1 -- 172.21.15.5:0/3143641335 shutdown_connections 2024-09-10T10:00:39.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:39.321+0000 7f53ab200640 1 --2- 172.21.15.5:0/3143641335 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f53900a2b80 0x7f53900a5040 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:39.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:39.321+0000 7f53ab200640 1 --2- 172.21.15.5:0/3143641335 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f53ac11b140 0x7f53ac1ab9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:39.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:39.321+0000 7f53ab200640 1 --2- 172.21.15.5:0/3143641335 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f53ac110190 0x7f53ac1a7630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:39.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:39.321+0000 7f53ab200640 1 --2- 172.21.15.5:0/3143641335 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f53ac06ea30 0x7f53ac1a70f0 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:39.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:39.321+0000 7f53ab200640 1 -- 172.21.15.5:0/3143641335 >> 172.21.15.5:0/3143641335 conn(0x7f53ac073180 msgr2=0x7f53ac119310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:00:39.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:39.322+0000 7f53ab200640 1 -- 172.21.15.5:0/3143641335 shutdown_connections 2024-09-10T10:00:39.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:39.322+0000 7f53ab200640 1 -- 172.21.15.5:0/3143641335 wait complete. 2024-09-10T10:00:40.034 INFO:tasks.workunit.client.0.smithi005.stdout: 10 134561 13.63 MB/sec execute 378 sec latency 1549.713 ms 2024-09-10T10:00:40.257 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:00:39 smithi005 ceph-mon[30350]: pgmap v1062: 129 pgs: 129 active+clean; 684 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 907 KiB/s wr, 139 op/s 2024-09-10T10:00:40.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:00:39 smithi159 ceph-mon[35513]: pgmap v1062: 129 pgs: 129 active+clean; 684 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 907 KiB/s wr, 139 op/s 2024-09-10T10:00:40.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:00:39 smithi102 ceph-mon[35442]: pgmap v1062: 129 pgs: 129 active+clean; 684 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 907 KiB/s wr, 139 op/s 2024-09-10T10:00:40.506 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '8aeff82a', quiesce took 8.4 seconds. Will release after: 25.800000000000004 2024-09-10T10:00:40.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:40.256+0000 7f554ae00640 1 -- 172.21.15.5:0/3722083916 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5544109140 msgr2=0x7f5544110cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:40.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:40.256+0000 7f554ae00640 1 --2- 172.21.15.5:0/3722083916 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5544109140 0x7f5544110cc0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f553c009ca0 tx=0x7f553c02a8c0 comp rx=0 tx=0).stop 2024-09-10T10:00:40.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:40.256+0000 7f554ae00640 1 -- 172.21.15.5:0/3722083916 shutdown_connections 2024-09-10T10:00:40.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:40.256+0000 7f554ae00640 1 --2- 172.21.15.5:0/3722083916 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5544109140 0x7f5544110cc0 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:40.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:40.256+0000 7f554ae00640 1 --2- 172.21.15.5:0/3722083916 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f55441049c0 0x7f5544108a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:40.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:40.256+0000 7f554ae00640 1 --2- 172.21.15.5:0/3722083916 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5544104010 0x7f55441043f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:40.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:40.256+0000 7f554ae00640 1 -- 172.21.15.5:0/3722083916 >> 172.21.15.5:0/3722083916 conn(0x7f55440ffa50 msgr2=0x7f5544101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:00:40.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:40.256+0000 7f554ae00640 1 -- 172.21.15.5:0/3722083916 shutdown_connections 2024-09-10T10:00:40.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:40.256+0000 7f554ae00640 1 -- 172.21.15.5:0/3722083916 wait complete. 2024-09-10T10:00:40.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:40.257+0000 7f554ae00640 1 Processor -- start 2024-09-10T10:00:40.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:40.257+0000 7f554ae00640 1 -- start start 2024-09-10T10:00:40.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:40.258+0000 7f554ae00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5544104010 0x7f554419a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:40.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:40.258+0000 7f554ae00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f55441049c0 0x7f554419abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:40.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:40.258+0000 7f554ae00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5544109140 0x7f554419ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:40.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:40.258+0000 7f554ae00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f5544114150 con 0x7f55441049c0 2024-09-10T10:00:40.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:40.258+0000 7f554ae00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f5544113fd0 con 0x7f5544109140 2024-09-10T10:00:40.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:40.258+0000 7f554ae00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f55441142d0 con 0x7f5544104010 2024-09-10T10:00:40.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:40.258+0000 7f5543400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f55441049c0 0x7f554419abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:40.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:40.258+0000 7f5543e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5544104010 0x7f554419a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:40.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:40.258+0000 7f5543e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5544104010 0x7f554419a690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:34608/0 (socket says 172.21.15.5:34608) 2024-09-10T10:00:40.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:40.258+0000 7f5543e00640 1 -- 172.21.15.5:0/3261742325 learned_addr learned my addr 172.21.15.5:0/3261742325 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T10:00:40.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:40.258+0000 7f5548e00640 1 --2- 172.21.15.5:0/3261742325 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5544109140 0x7f554419ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:40.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:40.259+0000 7f5543e00640 1 -- 172.21.15.5:0/3261742325 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5544109140 msgr2=0x7f554419ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:40.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:40.259+0000 7f5543e00640 1 --2- 172.21.15.5:0/3261742325 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5544109140 0x7f554419ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:40.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:40.259+0000 7f5543e00640 1 -- 172.21.15.5:0/3261742325 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f55441049c0 msgr2=0x7f554419abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:40.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:40.259+0000 7f5543e00640 1 --2- 172.21.15.5:0/3261742325 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f55441049c0 0x7f554419abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:40.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:40.259+0000 7f5543e00640 1 -- 172.21.15.5:0/3261742325 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f554419f640 con 0x7f5544104010 2024-09-10T10:00:40.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:40.259+0000 7f5548e00640 1 --2- 172.21.15.5:0/3261742325 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5544109140 0x7f554419ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:00:40.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:40.259+0000 7f5543e00640 1 --2- 172.21.15.5:0/3261742325 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5544104010 0x7f554419a690 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f553400e9f0 tx=0x7f553400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:00:40.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:40.259+0000 7f5540c00640 1 -- 172.21.15.5:0/3261742325 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f553400cd80 con 0x7f5544104010 2024-09-10T10:00:40.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:40.259+0000 7f554ae00640 1 -- 172.21.15.5:0/3261742325 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f554419f930 con 0x7f5544104010 2024-09-10T10:00:40.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:40.259+0000 7f5540c00640 1 -- 172.21.15.5:0/3261742325 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5534010910 con 0x7f5544104010 2024-09-10T10:00:40.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:40.259+0000 7f5540c00640 1 -- 172.21.15.5:0/3261742325 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55340188f0 con 0x7f5544104010 2024-09-10T10:00:40.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:40.259+0000 7f554ae00640 1 -- 172.21.15.5:0/3261742325 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55441a7210 con 0x7f5544104010 2024-09-10T10:00:40.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:40.262+0000 7f554d850740 1 Processor -- start 2024-09-10T10:00:40.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:40.262+0000 7f554d850740 1 -- start start 2024-09-10T10:00:40.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:40.263+0000 7f5540c00640 1 -- 172.21.15.5:0/3261742325 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f5534010ac0 con 0x7f5544104010 2024-09-10T10:00:40.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:40.263+0000 7f554d850740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556eac899ac0 0x556eac899ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:40.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:40.263+0000 7f554d850740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556eac89a3e0 0x556eac8a4760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:40.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:40.263+0000 7f554d850740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556eac8a4ca0 0x556eac8a7090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:40.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:40.263+0000 7f554d850740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x556eac800290 con 0x556eac89a3e0 2024-09-10T10:00:40.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:40.263+0000 7f554d850740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x556eac7a7b40 con 0x556eac899ac0 2024-09-10T10:00:40.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:40.263+0000 7f554d850740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x556eac7fc250 con 0x556eac8a4ca0 2024-09-10T10:00:40.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:40.263+0000 7f5543e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556eac89a3e0 0x556eac8a4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:40.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:40.263+0000 7f5543400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556eac8a4ca0 0x556eac8a7090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:40.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:40.263+0000 7f5543e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556eac89a3e0 0x556eac8a4760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57976/0 (socket says 172.21.15.5:57976) 2024-09-10T10:00:40.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:40.263+0000 7f5543e00640 1 -- 172.21.15.5:0/694906103 learned_addr learned my addr 172.21.15.5:0/694906103 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T10:00:40.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:40.263+0000 7f5548e00640 1 --2- 172.21.15.5:0/694906103 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556eac899ac0 0x556eac899ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:40.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:40.264+0000 7f5543400640 1 -- 172.21.15.5:0/694906103 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556eac899ac0 msgr2=0x556eac899ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:00:40.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:40.264+0000 7f5543400640 1 --2- 172.21.15.5:0/694906103 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556eac899ac0 0x556eac899ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:40.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:40.264+0000 7f5543400640 1 -- 172.21.15.5:0/694906103 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556eac89a3e0 msgr2=0x556eac8a4760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:00:40.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:40.264+0000 7f5543400640 1 --2- 172.21.15.5:0/694906103 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556eac89a3e0 0x556eac8a4760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:40.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:40.264+0000 7f5543400640 1 -- 172.21.15.5:0/694906103 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556eac5716a0 con 0x556eac8a4ca0 2024-09-10T10:00:40.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:40.264+0000 7f5543e00640 1 --2- 172.21.15.5:0/694906103 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556eac89a3e0 0x556eac8a4760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:00:40.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:40.264+0000 7f5543400640 1 --2- 172.21.15.5:0/694906103 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556eac8a4ca0 0x556eac8a7090 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f553000bb40 tx=0x7f55300103d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f51a4f6c5043486a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:00:40.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:40.264+0000 7f5539600640 1 -- 172.21.15.5:0/694906103 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5530004030 con 0x556eac8a4ca0 2024-09-10T10:00:40.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:40.264+0000 7f5539600640 1 -- 172.21.15.5:0/694906103 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f55300041d0 con 0x556eac8a4ca0 2024-09-10T10:00:40.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:40.264+0000 7f5539600640 1 -- 172.21.15.5:0/694906103 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55300116a0 con 0x556eac8a4ca0 2024-09-10T10:00:40.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:40.265+0000 7f5540c00640 1 --2- 172.21.15.5:0/3261742325 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f55280a2b80 0x7f55280a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:40.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:40.265+0000 7f5540c00640 1 -- 172.21.15.5:0/3261742325 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f55340cf820 con 0x7f5544104010 2024-09-10T10:00:40.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:40.265+0000 7f554d850740 1 -- 172.21.15.5:0/694906103 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556eac8a4ca0 msgr2=0x556eac8a7090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:40.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:40.265+0000 7f554d850740 1 --2- 172.21.15.5:0/694906103 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556eac8a4ca0 0x556eac8a7090 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f553000bb40 tx=0x7f55300103d0 comp rx=0 tx=0).stop 2024-09-10T10:00:40.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:40.265+0000 7f5548e00640 1 --2- 172.21.15.5:0/3261742325 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f55280a2b80 0x7f55280a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:40.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:40.265+0000 7f554d850740 1 -- 172.21.15.5:0/694906103 shutdown_connections 2024-09-10T10:00:40.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:40.265+0000 7f554d850740 1 --2- 172.21.15.5:0/694906103 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556eac8a4ca0 0x556eac8a7090 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:40.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:40.265+0000 7f554d850740 1 --2- 172.21.15.5:0/694906103 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556eac89a3e0 0x556eac8a4760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:40.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:40.265+0000 7f554d850740 1 --2- 172.21.15.5:0/694906103 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556eac899ac0 0x556eac899ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:40.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:40.265+0000 7f554d850740 1 -- 172.21.15.5:0/694906103 >> 172.21.15.5:0/694906103 conn(0x556eac896050 msgr2=0x556eac896430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:00:40.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:40.265+0000 7f554d850740 1 -- 172.21.15.5:0/694906103 shutdown_connections 2024-09-10T10:00:40.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:40.266+0000 7f554d850740 1 -- 172.21.15.5:0/694906103 wait complete. 2024-09-10T10:00:40.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:40.266+0000 7f5548e00640 1 --2- 172.21.15.5:0/3261742325 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f55280a2b80 0x7f55280a5040 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f553c031f80 tx=0x7f553c004a60 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:00:40.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:40.266+0000 7f554d850740 1 -- start start 2024-09-10T10:00:40.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:40.266+0000 7f554d850740 1 Processor -- start 2024-09-10T10:00:40.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:40.267+0000 7f554d850740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556eac899ac0 0x556eac8a45c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:40.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:40.267+0000 7f554d850740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556eac89a3e0 0x556eac8a1870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:40.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:40.267+0000 7f554d850740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556eac8a4ca0 0x556eac8a1db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:40.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:40.267+0000 7f554d850740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x556eac8a7910 con 0x556eac899ac0 2024-09-10T10:00:40.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:40.267+0000 7f554d850740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x556eac7bafa0 con 0x556eac8a4ca0 2024-09-10T10:00:40.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:40.267+0000 7f554d850740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x556eac8a7a90 con 0x556eac89a3e0 2024-09-10T10:00:40.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:40.267+0000 7f554d850740 10 client.0 _ll_register_callbacks cb 0x556eac50eba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:00:40.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:40.267+0000 7f5543e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556eac899ac0 0x556eac8a45c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:40.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:40.267+0000 7f5548e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556eac8a4ca0 0x556eac8a1db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:40.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:40.267+0000 7f5543e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556eac899ac0 0x556eac8a45c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57986/0 (socket says 172.21.15.5:57986) 2024-09-10T10:00:40.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:40.267+0000 7f5548e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556eac8a4ca0 0x556eac8a1db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:34278/0 (socket says 172.21.15.5:34278) 2024-09-10T10:00:40.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:40.267+0000 7f5543e00640 1 -- 172.21.15.5:0/63867800 learned_addr learned my addr 172.21.15.5:0/63867800 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T10:00:40.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:40.267+0000 7f5543400640 1 --2- 172.21.15.5:0/63867800 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556eac89a3e0 0x556eac8a1870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:40.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:40.267+0000 7f5543e00640 1 -- 172.21.15.5:0/63867800 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556eac89a3e0 msgr2=0x556eac8a1870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:00:40.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:40.267+0000 7f5543e00640 1 --2- 172.21.15.5:0/63867800 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556eac89a3e0 0x556eac8a1870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:40.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:40.267+0000 7f5543e00640 1 -- 172.21.15.5:0/63867800 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556eac8a4ca0 msgr2=0x556eac8a1db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:00:40.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:40.267+0000 7f5543e00640 1 --2- 172.21.15.5:0/63867800 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556eac8a4ca0 0x556eac8a1db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:40.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:40.268+0000 7f5543e00640 1 -- 172.21.15.5:0/63867800 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556eac8a25b0 con 0x556eac899ac0 2024-09-10T10:00:40.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:40.268+0000 7f5543400640 1 --2- 172.21.15.5:0/63867800 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556eac89a3e0 0x556eac8a1870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:00:40.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:40.268+0000 7f5543e00640 1 --2- 172.21.15.5:0/63867800 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556eac899ac0 0x556eac8a45c0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f553401f040 tx=0x7f553409bab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c5605e235e0f9b1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:00:40.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:40.268+0000 7f552e000640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-10T10:00:40.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:40.268+0000 7f552e000640 1 -- 172.21.15.5:0/63867800 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f553409b330 con 0x556eac899ac0 2024-09-10T10:00:40.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:40.268+0000 7f554d850740 1 -- 172.21.15.5:0/63867800 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f55300041d0 con 0x556eac899ac0 2024-09-10T10:00:40.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:40.268+0000 7f552e000640 1 -- 172.21.15.5:0/63867800 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f5534010430 con 0x556eac899ac0 2024-09-10T10:00:40.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:40.269+0000 7f552e000640 1 -- 172.21.15.5:0/63867800 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f553409bce0 con 0x556eac899ac0 2024-09-10T10:00:40.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:40.269+0000 7f554d850740 10 client.19602 fetch_fsmap learned FSMap version 42 2024-09-10T10:00:40.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:40.269+0000 7f554d850740 1 -- 172.21.15.5:0/63867800 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x556eac8b3840 con 0x556eac899ac0 2024-09-10T10:00:40.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:40.269+0000 7f552e000640 1 -- 172.21.15.5:0/63867800 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f55300041d0 con 0x556eac899ac0 2024-09-10T10:00:40.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:40.269+0000 7f552e000640 1 -- 172.21.15.5:0/63867800 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 42) ==== 8370+0+0 (secure 0 0 0) 0x7f55340d1ce0 con 0x556eac899ac0 2024-09-10T10:00:40.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:40.269+0000 7f554d850740 10 client.19602 fetch_fsmap finished waiting for FSMap version 42 2024-09-10T10:00:40.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:40.269+0000 7f554d850740 10 client.19602 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T10:00:40.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:40.269+0000 7f554d850740 20 client.19602 populate_metadata read hostname 'smithi005' 2024-09-10T10:00:40.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:40.269+0000 7f554d850740 1 --2- 172.21.15.5:0/63867800 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x556eac8b4c50 0x556eac8b7080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:40.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:40.269+0000 7f554d850740 4 client.19602 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:00:40.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:40.269+0000 7f554d850740 1 -- 172.21.15.5:0/63867800 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556eac80d800 con 0x556eac8b4c50 2024-09-10T10:00:40.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:40.270+0000 7f5543400640 1 --2- 172.21.15.5:0/63867800 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x556eac8b4c50 0x556eac8b7080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:40.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:40.270+0000 7f5543400640 1 --2- 172.21.15.5:0/63867800 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x556eac8b4c50 0x556eac8b7080 crc :-1 s=READY pgs=305 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2e8c069b9bdc2e7e server_cookie=5f04309d9c184fbc in_seq=0 out_seq=0 2024-09-10T10:00:40.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:40.270+0000 7f552e000640 10 client.19602 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T10:00:40.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:40.271+0000 7f552e000640 1 -- 172.21.15.5:0/63867800 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556eac80d800 con 0x556eac8b4c50 2024-09-10T10:00:40.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:40.271+0000 7f552e000640 10 client.19602 handle_command_reply: tid=0 2024-09-10T10:00:40.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:40.271+0000 7f554d850740 1 client.19602 shutdown 2024-09-10T10:00:40.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:40.271+0000 7f554d850740 1 -- 172.21.15.5:0/63867800 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556eac899ac0 msgr2=0x556eac8a45c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:40.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:40.271+0000 7f554d850740 1 --2- 172.21.15.5:0/63867800 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556eac899ac0 0x556eac8a45c0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f553401f040 tx=0x7f553409bab0 comp rx=0 tx=0).stop 2024-09-10T10:00:40.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:40.271+0000 7f554d850740 1 -- 172.21.15.5:0/63867800 shutdown_connections 2024-09-10T10:00:40.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:40.271+0000 7f554d850740 1 --2- 172.21.15.5:0/63867800 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x556eac8b4c50 0x556eac8b7080 crc :-1 s=READY pgs=305 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:40.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:40.272+0000 7f554d850740 1 --2- 172.21.15.5:0/63867800 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556eac8a4ca0 0x556eac8a1db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:40.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:40.272+0000 7f554d850740 1 --2- 172.21.15.5:0/63867800 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556eac89a3e0 0x556eac8a1870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:40.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:40.272+0000 7f554d850740 1 --2- 172.21.15.5:0/63867800 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556eac899ac0 0x556eac8a45c0 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:40.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:40.272+0000 7f554d850740 1 -- 172.21.15.5:0/63867800 >> 172.21.15.5:0/63867800 conn(0x556eac896050 msgr2=0x556eac89bd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:00:40.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:40.272+0000 7f552e000640 0 client.19602 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T10:00:40.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:40.272+0000 7f554d850740 1 -- 172.21.15.5:0/63867800 shutdown_connections 2024-09-10T10:00:40.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:40.272+0000 7f554d850740 1 -- 172.21.15.5:0/63867800 wait complete. 2024-09-10T10:00:40.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:40.272+0000 7f554d850740 20 client.19602 trim_cache size 0 max 16384 2024-09-10T10:00:40.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:40.290+0000 7f554d850740 1 Processor -- start 2024-09-10T10:00:40.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:40.290+0000 7f554d850740 1 -- start start 2024-09-10T10:00:40.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:40.290+0000 7f554d850740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556eac899ac0 0x556eac8a37e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:40.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:40.290+0000 7f554d850740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556eac89a3e0 0x556eac8bc920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:40.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:40.290+0000 7f554d850740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556eac8a4ca0 0x556eac8bce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:40.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:40.290+0000 7f554d850740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f55441b1110 con 0x556eac8a4ca0 2024-09-10T10:00:40.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:40.290+0000 7f554d850740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f55441b2c50 con 0x556eac89a3e0 2024-09-10T10:00:40.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:40.290+0000 7f554d850740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f55441b17e0 con 0x556eac899ac0 2024-09-10T10:00:40.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:40.291+0000 7f5548e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556eac8a4ca0 0x556eac8bce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:40.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:40.291+0000 7f5548e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556eac8a4ca0 0x556eac8bce60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57994/0 (socket says 172.21.15.5:57994) 2024-09-10T10:00:40.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:40.291+0000 7f5548e00640 1 -- 172.21.15.5:0/87691906 learned_addr learned my addr 172.21.15.5:0/87691906 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T10:00:40.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:40.291+0000 7f5543400640 1 --2- 172.21.15.5:0/87691906 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556eac89a3e0 0x556eac8bc920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:40.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:40.291+0000 7f5543e00640 1 --2- 172.21.15.5:0/87691906 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556eac899ac0 0x556eac8a37e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:40.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:40.291+0000 7f5543400640 1 -- 172.21.15.5:0/87691906 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556eac899ac0 msgr2=0x556eac8a37e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:00:40.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:40.291+0000 7f5543400640 1 --2- 172.21.15.5:0/87691906 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556eac899ac0 0x556eac8a37e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:40.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:40.291+0000 7f5543400640 1 -- 172.21.15.5:0/87691906 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556eac8a4ca0 msgr2=0x556eac8bce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:00:40.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:40.291+0000 7f5543400640 1 --2- 172.21.15.5:0/87691906 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556eac8a4ca0 0x556eac8bce60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:40.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:40.291+0000 7f5543400640 1 -- 172.21.15.5:0/87691906 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556eac8a0820 con 0x556eac89a3e0 2024-09-10T10:00:40.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:40.291+0000 7f5548e00640 1 --2- 172.21.15.5:0/87691906 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556eac8a4ca0 0x556eac8bce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:00:40.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:40.291+0000 7f5543400640 1 --2- 172.21.15.5:0/87691906 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556eac89a3e0 0x556eac8bc920 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f5530003d90 tx=0x7f5530003df0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eae28ece5494b643 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:00:40.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:40.292+0000 7f5538c00640 1 -- 172.21.15.5:0/87691906 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f553002c020 con 0x556eac89a3e0 2024-09-10T10:00:40.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:40.292+0000 7f5538c00640 1 -- 172.21.15.5:0/87691906 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5530003ad0 con 0x556eac89a3e0 2024-09-10T10:00:40.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:40.292+0000 7f5538c00640 1 -- 172.21.15.5:0/87691906 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55300054a0 con 0x556eac89a3e0 2024-09-10T10:00:40.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:40.292+0000 7f554d850740 1 -- 172.21.15.5:0/87691906 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556eac89a3e0 msgr2=0x556eac8bc920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:40.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:40.292+0000 7f554d850740 1 --2- 172.21.15.5:0/87691906 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556eac89a3e0 0x556eac8bc920 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f5530003d90 tx=0x7f5530003df0 comp rx=0 tx=0).stop 2024-09-10T10:00:40.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:40.292+0000 7f554d850740 1 -- 172.21.15.5:0/87691906 shutdown_connections 2024-09-10T10:00:40.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:40.292+0000 7f554d850740 1 --2- 172.21.15.5:0/87691906 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556eac8a4ca0 0x556eac8bce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:40.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:40.292+0000 7f554d850740 1 --2- 172.21.15.5:0/87691906 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556eac89a3e0 0x556eac8bc920 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:40.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:40.292+0000 7f554d850740 1 --2- 172.21.15.5:0/87691906 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556eac899ac0 0x556eac8a37e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:40.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:40.292+0000 7f554d850740 1 -- 172.21.15.5:0/87691906 >> 172.21.15.5:0/87691906 conn(0x556eac896050 msgr2=0x556eac8a30d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:00:40.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:40.292+0000 7f554d850740 1 -- 172.21.15.5:0/87691906 shutdown_connections 2024-09-10T10:00:40.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:40.292+0000 7f554d850740 1 -- 172.21.15.5:0/87691906 wait complete. 2024-09-10T10:00:40.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:40.293+0000 7f554d850740 1 -- start start 2024-09-10T10:00:40.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:40.293+0000 7f554d850740 1 Processor -- start 2024-09-10T10:00:40.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:40.293+0000 7f554d850740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556eac899ac0 0x556eac8aaa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:40.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:40.293+0000 7f554d850740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556eac89a3e0 0x556eac8aafa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:40.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:40.293+0000 7f554d850740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556eac8a4ca0 0x556eac8ab4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:40.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:40.293+0000 7f554d850740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f55441b2580 con 0x556eac8a4ca0 2024-09-10T10:00:40.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:40.293+0000 7f554d850740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f55441b1eb0 con 0x556eac89a3e0 2024-09-10T10:00:40.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:40.293+0000 7f554d850740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f55441af0f0 con 0x556eac899ac0 2024-09-10T10:00:40.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:40.294+0000 7f554d850740 10 client.0 _ll_register_callbacks cb 0x7f553400e940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:00:40.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:40.294+0000 7f5548e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556eac8a4ca0 0x556eac8ab4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:40.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:40.294+0000 7f5548e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556eac8a4ca0 0x556eac8ab4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58002/0 (socket says 172.21.15.5:58002) 2024-09-10T10:00:40.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:40.294+0000 7f5543400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556eac89a3e0 0x556eac8aafa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:40.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:40.294+0000 7f5548e00640 1 -- 172.21.15.5:0/2524056190 learned_addr learned my addr 172.21.15.5:0/2524056190 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T10:00:40.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:40.294+0000 7f5543e00640 1 --2- 172.21.15.5:0/2524056190 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556eac899ac0 0x556eac8aaa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:40.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:40.294+0000 7f5548e00640 1 -- 172.21.15.5:0/2524056190 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556eac899ac0 msgr2=0x556eac8aaa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:00:40.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:40.294+0000 7f5548e00640 1 --2- 172.21.15.5:0/2524056190 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556eac899ac0 0x556eac8aaa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:40.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:40.294+0000 7f5548e00640 1 -- 172.21.15.5:0/2524056190 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556eac89a3e0 msgr2=0x556eac8aafa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:00:40.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:40.294+0000 7f5548e00640 1 --2- 172.21.15.5:0/2524056190 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556eac89a3e0 0x556eac8aafa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:40.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:40.294+0000 7f5548e00640 1 -- 172.21.15.5:0/2524056190 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556eac91e900 con 0x556eac8a4ca0 2024-09-10T10:00:40.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:40.294+0000 7f5548e00640 1 --2- 172.21.15.5:0/2524056190 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556eac8a4ca0 0x556eac8ab4e0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f553c005e30 tx=0x7f553c02a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdaf87952721817a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:00:40.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:40.294+0000 7f552e000640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-10T10:00:40.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:40.295+0000 7f552e000640 1 -- 172.21.15.5:0/2524056190 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f553c059060 con 0x556eac8a4ca0 2024-09-10T10:00:40.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:40.295+0000 7f552e000640 1 -- 172.21.15.5:0/2524056190 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f553c004790 con 0x556eac8a4ca0 2024-09-10T10:00:40.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:40.295+0000 7f554d850740 1 -- 172.21.15.5:0/2524056190 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5530003ad0 con 0x556eac8a4ca0 2024-09-10T10:00:40.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:40.295+0000 7f552e000640 1 -- 172.21.15.5:0/2524056190 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f553c060050 con 0x556eac8a4ca0 2024-09-10T10:00:40.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:40.295+0000 7f554d850740 10 client.19608 fetch_fsmap learned FSMap version 42 2024-09-10T10:00:40.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:40.295+0000 7f554d850740 1 -- 172.21.15.5:0/2524056190 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x556eac91fc20 con 0x556eac8a4ca0 2024-09-10T10:00:40.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:40.295+0000 7f552e000640 1 -- 172.21.15.5:0/2524056190 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5530003ad0 con 0x556eac8a4ca0 2024-09-10T10:00:40.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:40.295+0000 7f552e000640 1 -- 172.21.15.5:0/2524056190 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 42) ==== 8370+0+0 (secure 0 0 0) 0x7f553c053070 con 0x556eac8a4ca0 2024-09-10T10:00:40.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:40.295+0000 7f554d850740 10 client.19608 fetch_fsmap finished waiting for FSMap version 42 2024-09-10T10:00:40.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:40.295+0000 7f554d850740 10 client.19608 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T10:00:40.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:40.295+0000 7f554d850740 20 client.19608 populate_metadata read hostname 'smithi005' 2024-09-10T10:00:40.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:40.295+0000 7f554d850740 1 --2- 172.21.15.5:0/2524056190 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x556eac920340 0x556eac922730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:40.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:40.295+0000 7f554d850740 4 client.19608 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-10T10:00:40.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:40.295+0000 7f554d850740 1 -- 172.21.15.5:0/2524056190 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x556eac807d10 con 0x556eac920340 2024-09-10T10:00:40.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:40.296+0000 7f5543e00640 1 --2- 172.21.15.5:0/2524056190 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x556eac920340 0x556eac922730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:40.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:40.296+0000 7f5543e00640 1 --2- 172.21.15.5:0/2524056190 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x556eac920340 0x556eac922730 crc :-1 s=READY pgs=306 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=25f23598697131ce server_cookie=6564346195c6b8ee in_seq=0 out_seq=0 2024-09-10T10:00:40.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:40.296+0000 7f552e000640 10 client.19608 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T10:00:40.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:40.297+0000 7f552e000640 1 -- 172.21.15.5:0/2524056190 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x556eac807d10 con 0x556eac920340 2024-09-10T10:00:40.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:40.297+0000 7f552e000640 10 client.19608 handle_command_reply: tid=0 2024-09-10T10:00:40.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:40.297+0000 7f554d850740 1 client.19608 shutdown 2024-09-10T10:00:40.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:40.298+0000 7f554d850740 1 -- 172.21.15.5:0/2524056190 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556eac8a4ca0 msgr2=0x556eac8ab4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:40.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:40.298+0000 7f554d850740 1 --2- 172.21.15.5:0/2524056190 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556eac8a4ca0 0x556eac8ab4e0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f553c005e30 tx=0x7f553c02a8c0 comp rx=0 tx=0).stop 2024-09-10T10:00:40.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:40.298+0000 7f554d850740 1 -- 172.21.15.5:0/2524056190 shutdown_connections 2024-09-10T10:00:40.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:40.298+0000 7f554d850740 1 --2- 172.21.15.5:0/2524056190 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x556eac920340 0x556eac922730 crc :-1 s=READY pgs=306 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:40.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:40.298+0000 7f554d850740 1 --2- 172.21.15.5:0/2524056190 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556eac8a4ca0 0x556eac8ab4e0 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:40.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:40.298+0000 7f554d850740 1 --2- 172.21.15.5:0/2524056190 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556eac89a3e0 0x556eac8aafa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:40.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:40.298+0000 7f554d850740 1 --2- 172.21.15.5:0/2524056190 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556eac899ac0 0x556eac8aaa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:40.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:40.298+0000 7f552e000640 0 client.19608 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T10:00:40.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:40.298+0000 7f554d850740 1 -- 172.21.15.5:0/2524056190 >> 172.21.15.5:0/2524056190 conn(0x556eac896050 msgr2=0x556eac8951c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:00:40.641 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-10T10:00:40.641 INFO:teuthology.orchestra.run.smithi005.stdout: "return_code": 0, 2024-09-10T10:00:40.641 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_tag": "baac0d07-e881-4a67-9f71-de0a5215e662", 2024-09-10T10:00:40.641 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": "asynchronous" 2024-09-10T10:00:40.641 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-10T10:00:40.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:40.298+0000 7f554d850740 1 -- 172.21.15.5:0/2524056190 shutdown_connections 2024-09-10T10:00:40.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:40.298+0000 7f554d850740 1 -- 172.21.15.5:0/2524056190 wait complete. 2024-09-10T10:00:40.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:40.298+0000 7f554d850740 20 client.19608 trim_cache size 0 max 16384 2024-09-10T10:00:40.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:40.299+0000 7f554ae00640 1 -- 172.21.15.5:0/3261742325 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f55280a2b80 msgr2=0x7f55280a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:40.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:40.299+0000 7f554ae00640 1 --2- 172.21.15.5:0/3261742325 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f55280a2b80 0x7f55280a5040 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f553c031f80 tx=0x7f553c004a60 comp rx=0 tx=0).stop 2024-09-10T10:00:40.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:40.299+0000 7f554ae00640 1 -- 172.21.15.5:0/3261742325 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5544104010 msgr2=0x7f554419a690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:40.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:40.299+0000 7f554ae00640 1 --2- 172.21.15.5:0/3261742325 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5544104010 0x7f554419a690 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f553400e9f0 tx=0x7f553400eeb0 comp rx=0 tx=0).stop 2024-09-10T10:00:40.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:40.299+0000 7f554ae00640 1 -- 172.21.15.5:0/3261742325 shutdown_connections 2024-09-10T10:00:40.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:40.299+0000 7f554ae00640 1 --2- 172.21.15.5:0/3261742325 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f55280a2b80 0x7f55280a5040 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:40.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:40.299+0000 7f554ae00640 1 --2- 172.21.15.5:0/3261742325 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5544109140 0x7f554419ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:40.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:40.299+0000 7f554ae00640 1 --2- 172.21.15.5:0/3261742325 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f55441049c0 0x7f554419abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:40.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:40.299+0000 7f554ae00640 1 --2- 172.21.15.5:0/3261742325 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5544104010 0x7f554419a690 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:40.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:40.299+0000 7f554ae00640 1 -- 172.21.15.5:0/3261742325 >> 172.21.15.5:0/3261742325 conn(0x7f55440ffa50 msgr2=0x7f5544100d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:00:40.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:40.299+0000 7f554ae00640 1 -- 172.21.15.5:0/3261742325 shutdown_connections 2024-09-10T10:00:40.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:40.299+0000 7f554ae00640 1 -- 172.21.15.5:0/3261742325 wait complete. 2024-09-10T10:00:41.034 INFO:tasks.workunit.client.0.smithi005.stdout: 10 134561 13.60 MB/sec execute 379 sec latency 2549.785 ms 2024-09-10T10:00:41.306 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:00:40 smithi005 ceph-mon[30350]: scrub summary: active paths [/] 2024-09-10T10:00:41.307 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:00:40 smithi005 ceph-mon[30350]: scrub summary: idle+waiting paths [/] 2024-09-10T10:00:41.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:00:40 smithi159 ceph-mon[35513]: scrub summary: active paths [/] 2024-09-10T10:00:41.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:00:40 smithi159 ceph-mon[35513]: scrub summary: idle+waiting paths [/] 2024-09-10T10:00:41.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:00:40 smithi102 ceph-mon[35442]: scrub summary: active paths [/] 2024-09-10T10:00:41.385 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:00:40 smithi102 ceph-mon[35442]: scrub summary: idle+waiting paths [/] 2024-09-10T10:00:41.828 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T10:00:42.034 INFO:tasks.workunit.client.0.smithi005.stdout: 10 134561 13.56 MB/sec execute 380 sec latency 3549.852 ms 2024-09-10T10:00:42.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:00:41 smithi005 ceph-mon[30350]: pgmap v1063: 129 pgs: 129 active+clean; 678 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 2.0 MiB/s wr, 313 op/s 2024-09-10T10:00:42.057 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:00:41 smithi005 ceph-mon[30350]: scrub queued for path: / 2024-09-10T10:00:42.057 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:00:41 smithi005 ceph-mon[30350]: scrub summary: idle+waiting paths [/] 2024-09-10T10:00:42.057 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:00:41 smithi005 ceph-mon[30350]: scrub summary: active paths [/] 2024-09-10T10:00:42.057 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:00:41 smithi005 ceph-mon[30350]: scrub summary: active paths [/] 2024-09-10T10:00:42.057 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:00:41 smithi005 ceph-mon[30350]: scrub summary: idle+waiting paths [/] 2024-09-10T10:00:42.057 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:00:41 smithi005 ceph-mon[30350]: scrub summary: idle+waiting paths [/] 2024-09-10T10:00:42.057 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:00:41 smithi005 ceph-mon[30350]: scrub summary: active paths [/] 2024-09-10T10:00:42.057 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:00:41 smithi005 ceph-mon[30350]: scrub summary: active paths [/] 2024-09-10T10:00:42.057 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:00:41 smithi005 ceph-mon[30350]: scrub summary: idle+waiting paths [/] 2024-09-10T10:00:42.057 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:00:41 smithi005 ceph-mon[30350]: scrub summary: active paths [/] 2024-09-10T10:00:42.057 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:00:41 smithi005 ceph-mon[30350]: scrub summary: idle+waiting paths [/] 2024-09-10T10:00:42.057 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:00:41 smithi005 ceph-mon[30350]: scrub summary: idle+waiting paths [/] 2024-09-10T10:00:42.057 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:00:41 smithi005 ceph-mon[30350]: scrub summary: active paths [/] 2024-09-10T10:00:42.058 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:00:41 smithi005 ceph-mon[30350]: scrub summary: idle+waiting paths [/] 2024-09-10T10:00:42.213 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:00:42.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:00:41 smithi159 ceph-mon[35513]: pgmap v1063: 129 pgs: 129 active+clean; 678 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 2.0 MiB/s wr, 313 op/s 2024-09-10T10:00:42.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:00:41 smithi159 ceph-mon[35513]: scrub queued for path: / 2024-09-10T10:00:42.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:00:41 smithi159 ceph-mon[35513]: scrub summary: idle+waiting paths [/] 2024-09-10T10:00:42.351 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:00:41 smithi159 ceph-mon[35513]: scrub summary: active paths [/] 2024-09-10T10:00:42.351 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:00:41 smithi159 ceph-mon[35513]: scrub summary: active paths [/] 2024-09-10T10:00:42.351 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:00:41 smithi159 ceph-mon[35513]: scrub summary: idle+waiting paths [/] 2024-09-10T10:00:42.351 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:00:41 smithi159 ceph-mon[35513]: scrub summary: idle+waiting paths [/] 2024-09-10T10:00:42.351 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:00:41 smithi159 ceph-mon[35513]: scrub summary: active paths [/] 2024-09-10T10:00:42.351 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:00:41 smithi159 ceph-mon[35513]: scrub summary: active paths [/] 2024-09-10T10:00:42.351 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:00:41 smithi159 ceph-mon[35513]: scrub summary: idle+waiting paths [/] 2024-09-10T10:00:42.351 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:00:41 smithi159 ceph-mon[35513]: scrub summary: active paths [/] 2024-09-10T10:00:42.351 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:00:41 smithi159 ceph-mon[35513]: scrub summary: idle+waiting paths [/] 2024-09-10T10:00:42.351 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:00:41 smithi159 ceph-mon[35513]: scrub summary: idle+waiting paths [/] 2024-09-10T10:00:42.351 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:00:41 smithi159 ceph-mon[35513]: scrub summary: active paths [/] 2024-09-10T10:00:42.351 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:00:41 smithi159 ceph-mon[35513]: scrub summary: idle+waiting paths [/] 2024-09-10T10:00:42.385 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:00:41 smithi102 ceph-mon[35442]: pgmap v1063: 129 pgs: 129 active+clean; 678 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 2.0 MiB/s wr, 313 op/s 2024-09-10T10:00:42.385 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:00:41 smithi102 ceph-mon[35442]: scrub queued for path: / 2024-09-10T10:00:42.385 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:00:41 smithi102 ceph-mon[35442]: scrub summary: idle+waiting paths [/] 2024-09-10T10:00:42.385 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:00:41 smithi102 ceph-mon[35442]: scrub summary: active paths [/] 2024-09-10T10:00:42.385 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:00:41 smithi102 ceph-mon[35442]: scrub summary: active paths [/] 2024-09-10T10:00:42.385 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:00:41 smithi102 ceph-mon[35442]: scrub summary: idle+waiting paths [/] 2024-09-10T10:00:42.385 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:00:41 smithi102 ceph-mon[35442]: scrub summary: idle+waiting paths [/] 2024-09-10T10:00:42.385 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:00:41 smithi102 ceph-mon[35442]: scrub summary: active paths [/] 2024-09-10T10:00:42.385 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:00:41 smithi102 ceph-mon[35442]: scrub summary: active paths [/] 2024-09-10T10:00:42.385 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:00:41 smithi102 ceph-mon[35442]: scrub summary: idle+waiting paths [/] 2024-09-10T10:00:42.385 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:00:41 smithi102 ceph-mon[35442]: scrub summary: active paths [/] 2024-09-10T10:00:42.385 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:00:41 smithi102 ceph-mon[35442]: scrub summary: idle+waiting paths [/] 2024-09-10T10:00:42.386 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:00:41 smithi102 ceph-mon[35442]: scrub summary: idle+waiting paths [/] 2024-09-10T10:00:42.386 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:00:41 smithi102 ceph-mon[35442]: scrub summary: active paths [/] 2024-09-10T10:00:42.386 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:00:41 smithi102 ceph-mon[35442]: scrub summary: idle+waiting paths [/] 2024-09-10T10:00:43.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:43.005+0000 7faa1dc00640 1 -- 172.21.15.5:0/949471641 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faa18102360 msgr2=0x7faa1810e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:43.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:43.005+0000 7faa1dc00640 1 --2- 172.21.15.5:0/949471641 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faa18102360 0x7faa1810e890 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7faa10009cd0 tx=0x7faa1002a8f0 comp rx=0 tx=0).stop 2024-09-10T10:00:43.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:43.005+0000 7faa1dc00640 1 -- 172.21.15.5:0/949471641 shutdown_connections 2024-09-10T10:00:43.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:43.005+0000 7faa1dc00640 1 --2- 172.21.15.5:0/949471641 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faa18102360 0x7faa1810e890 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:43.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:43.005+0000 7faa1dc00640 1 --2- 172.21.15.5:0/949471641 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7faa181019c0 0x7faa18101e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:43.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:43.005+0000 7faa1dc00640 1 --2- 172.21.15.5:0/949471641 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7faa181079c0 0x7faa18107da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:43.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:43.005+0000 7faa1dc00640 1 -- 172.21.15.5:0/949471641 >> 172.21.15.5:0/949471641 conn(0x7faa180fd740 msgr2=0x7faa180ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:00:43.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:43.005+0000 7faa1dc00640 1 -- 172.21.15.5:0/949471641 shutdown_connections 2024-09-10T10:00:43.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:43.005+0000 7faa1dc00640 1 -- 172.21.15.5:0/949471641 wait complete. 2024-09-10T10:00:43.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:43.006+0000 7faa1dc00640 1 Processor -- start 2024-09-10T10:00:43.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:43.006+0000 7faa1dc00640 1 -- start start 2024-09-10T10:00:43.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:43.007+0000 7faa1dc00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7faa181019c0 0x7faa180744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:43.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:43.007+0000 7faa1dc00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faa18102360 0x7faa18074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:43.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:43.007+0000 7faa1dc00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7faa181079c0 0x7faa18074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:43.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:43.007+0000 7faa1dc00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7faa18111e10 con 0x7faa18102360 2024-09-10T10:00:43.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:43.007+0000 7faa1dc00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7faa18111c90 con 0x7faa181019c0 2024-09-10T10:00:43.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:43.007+0000 7faa1dc00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7faa18111f90 con 0x7faa181079c0 2024-09-10T10:00:43.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:43.007+0000 7faa17e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7faa181019c0 0x7faa180744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:43.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:43.007+0000 7faa17400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faa18102360 0x7faa18074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:43.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:43.007+0000 7faa17e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7faa181019c0 0x7faa180744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:40638/0 (socket says 172.21.15.5:40638) 2024-09-10T10:00:43.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:43.007+0000 7faa17e00640 1 -- 172.21.15.5:0/837481548 learned_addr learned my addr 172.21.15.5:0/837481548 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T10:00:43.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:43.007+0000 7faa1d200640 1 --2- 172.21.15.5:0/837481548 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7faa181079c0 0x7faa18074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:43.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:43.008+0000 7faa17e00640 1 -- 172.21.15.5:0/837481548 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7faa181079c0 msgr2=0x7faa18074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:43.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:43.008+0000 7faa17e00640 1 --2- 172.21.15.5:0/837481548 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7faa181079c0 0x7faa18074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:43.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:43.008+0000 7faa17e00640 1 -- 172.21.15.5:0/837481548 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faa18102360 msgr2=0x7faa18074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:43.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:43.008+0000 7faa17e00640 1 --2- 172.21.15.5:0/837481548 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faa18102360 0x7faa18074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:43.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:43.008+0000 7faa17e00640 1 -- 172.21.15.5:0/837481548 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa181a6710 con 0x7faa181019c0 2024-09-10T10:00:43.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:43.008+0000 7faa17e00640 1 --2- 172.21.15.5:0/837481548 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7faa181019c0 0x7faa180744f0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7faa0800ea60 tx=0x7faa0800ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:00:43.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:43.008+0000 7faa14c00640 1 -- 172.21.15.5:0/837481548 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa0800cde0 con 0x7faa181019c0 2024-09-10T10:00:43.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:43.008+0000 7faa14c00640 1 -- 172.21.15.5:0/837481548 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faa080035f0 con 0x7faa181019c0 2024-09-10T10:00:43.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:43.008+0000 7faa1dc00640 1 -- 172.21.15.5:0/837481548 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa181a6a00 con 0x7faa181019c0 2024-09-10T10:00:43.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:43.008+0000 7faa14c00640 1 -- 172.21.15.5:0/837481548 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa08010670 con 0x7faa181019c0 2024-09-10T10:00:43.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:43.008+0000 7faa1dc00640 1 -- 172.21.15.5:0/837481548 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa181a6f90 con 0x7faa181019c0 2024-09-10T10:00:43.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:43.011+0000 7faa21c1c740 1 Processor -- start 2024-09-10T10:00:43.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:43.011+0000 7faa21c1c740 1 -- start start 2024-09-10T10:00:43.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:43.012+0000 7faa14c00640 1 -- 172.21.15.5:0/837481548 <== mon.2 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7faa080037a0 con 0x7faa181019c0 2024-09-10T10:00:43.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:43.012+0000 7faa21c1c740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e6ce453d50 0x55e6ce454130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:43.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:43.012+0000 7faa21c1c740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e6ce454670 0x55e6ce45e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:43.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:43.012+0000 7faa21c1c740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e6ce45ef30 0x55e6ce461320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:43.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:43.012+0000 7faa21c1c740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55e6ce3ba3e0 con 0x55e6ce45ef30 2024-09-10T10:00:43.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:43.012+0000 7faa21c1c740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55e6ce361d80 con 0x55e6ce453d50 2024-09-10T10:00:43.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:43.012+0000 7faa21c1c740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55e6ce3b63a0 con 0x55e6ce454670 2024-09-10T10:00:43.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:43.012+0000 7faa1d200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e6ce453d50 0x55e6ce454130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:43.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:43.012+0000 7faa17e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e6ce454670 0x55e6ce45e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:43.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:43.012+0000 7faa1d200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e6ce453d50 0x55e6ce454130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:40654/0 (socket says 172.21.15.5:40654) 2024-09-10T10:00:43.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:43.012+0000 7faa1d200640 1 -- 172.21.15.5:0/796072863 learned_addr learned my addr 172.21.15.5:0/796072863 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T10:00:43.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:43.012+0000 7faa17400640 1 --2- 172.21.15.5:0/796072863 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e6ce45ef30 0x55e6ce461320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:43.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:43.013+0000 7faa1d200640 1 -- 172.21.15.5:0/796072863 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e6ce454670 msgr2=0x55e6ce45e9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:00:43.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:43.013+0000 7faa1d200640 1 --2- 172.21.15.5:0/796072863 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e6ce454670 0x55e6ce45e9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:43.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:43.013+0000 7faa1d200640 1 -- 172.21.15.5:0/796072863 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e6ce45ef30 msgr2=0x55e6ce461320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:00:43.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:43.013+0000 7faa1d200640 1 --2- 172.21.15.5:0/796072863 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e6ce45ef30 0x55e6ce461320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:43.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:43.013+0000 7faa1d200640 1 -- 172.21.15.5:0/796072863 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e6ce380ae0 con 0x55e6ce453d50 2024-09-10T10:00:43.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:43.013+0000 7faa1d200640 1 --2- 172.21.15.5:0/796072863 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e6ce453d50 0x55e6ce454130 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7faa10035000 tx=0x7faa100303a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=76ac328deba70905 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:00:43.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:43.013+0000 7faa0d600640 1 -- 172.21.15.5:0/796072863 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa10030740 con 0x55e6ce453d50 2024-09-10T10:00:43.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:43.014+0000 7faa0d600640 1 -- 172.21.15.5:0/796072863 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faa10036040 con 0x55e6ce453d50 2024-09-10T10:00:43.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:43.014+0000 7faa0d600640 1 -- 172.21.15.5:0/796072863 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa10004430 con 0x55e6ce453d50 2024-09-10T10:00:43.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:43.014+0000 7faa14c00640 1 --2- 172.21.15.5:0/837481548 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fa9fc0a2c00 0x7fa9fc0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:43.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:43.014+0000 7faa14c00640 1 -- 172.21.15.5:0/837481548 <== mon.2 v2:172.21.15.102:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7faa080cf770 con 0x7faa181019c0 2024-09-10T10:00:43.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:43.014+0000 7faa17e00640 1 --2- 172.21.15.5:0/837481548 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fa9fc0a2c00 0x7fa9fc0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:43.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:43.014+0000 7faa21c1c740 1 -- 172.21.15.5:0/796072863 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e6ce453d50 msgr2=0x55e6ce454130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:43.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:43.014+0000 7faa21c1c740 1 --2- 172.21.15.5:0/796072863 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e6ce453d50 0x55e6ce454130 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7faa10035000 tx=0x7faa100303a0 comp rx=0 tx=0).stop 2024-09-10T10:00:43.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:43.014+0000 7faa21c1c740 1 -- 172.21.15.5:0/796072863 shutdown_connections 2024-09-10T10:00:43.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:43.014+0000 7faa21c1c740 1 --2- 172.21.15.5:0/796072863 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e6ce45ef30 0x55e6ce461320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:43.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:43.014+0000 7faa21c1c740 1 --2- 172.21.15.5:0/796072863 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e6ce454670 0x55e6ce45e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:43.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:43.014+0000 7faa21c1c740 1 --2- 172.21.15.5:0/796072863 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e6ce453d50 0x55e6ce454130 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:43.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:43.014+0000 7faa21c1c740 1 -- 172.21.15.5:0/796072863 >> 172.21.15.5:0/796072863 conn(0x55e6ce4502e0 msgr2=0x55e6ce4506c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:00:43.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:43.015+0000 7faa21c1c740 1 -- 172.21.15.5:0/796072863 shutdown_connections 2024-09-10T10:00:43.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:43.015+0000 7faa21c1c740 1 -- 172.21.15.5:0/796072863 wait complete. 2024-09-10T10:00:43.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:43.015+0000 7faa17e00640 1 --2- 172.21.15.5:0/837481548 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fa9fc0a2c00 0x7fa9fc0a50c0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7faa080a42c0 tx=0x7faa080d5530 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:00:43.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:43.015+0000 7faa21c1c740 1 -- start start 2024-09-10T10:00:43.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:43.015+0000 7faa21c1c740 1 Processor -- start 2024-09-10T10:00:43.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:43.016+0000 7faa21c1c740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e6ce453d50 0x55e6ce45be70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:43.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:43.016+0000 7faa21c1c740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e6ce454670 0x55e6ce45c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:43.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:43.016+0000 7faa21c1c740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e6ce45ef30 0x55e6ce45c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:43.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:43.016+0000 7faa21c1c740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55e6ce461ba0 con 0x55e6ce453d50 2024-09-10T10:00:43.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:43.016+0000 7faa21c1c740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55e6ce3750f0 con 0x55e6ce454670 2024-09-10T10:00:43.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:43.016+0000 7faa21c1c740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55e6ce461d20 con 0x55e6ce45ef30 2024-09-10T10:00:43.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:43.016+0000 7faa21c1c740 10 client.0 _ll_register_callbacks cb 0x55e6ce3669e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:00:43.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:43.016+0000 7faa1d200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e6ce453d50 0x55e6ce45be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:43.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:43.016+0000 7faa1d200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e6ce453d50 0x55e6ce45be70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41860/0 (socket says 172.21.15.5:41860) 2024-09-10T10:00:43.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:43.016+0000 7faa1d200640 1 -- 172.21.15.5:0/3059672833 learned_addr learned my addr 172.21.15.5:0/3059672833 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T10:00:43.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:43.016+0000 7faa17400640 1 --2- 172.21.15.5:0/3059672833 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e6ce454670 0x55e6ce45c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:43.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:43.016+0000 7faa1d200640 1 --2- 172.21.15.5:0/3059672833 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e6ce45ef30 0x55e6ce45c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:43.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:43.017+0000 7faa1d200640 1 -- 172.21.15.5:0/3059672833 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e6ce45ef30 msgr2=0x55e6ce45c8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:00:43.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:43.017+0000 7faa1d200640 1 --2- 172.21.15.5:0/3059672833 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e6ce45ef30 0x55e6ce45c8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:43.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:43.017+0000 7faa1d200640 1 -- 172.21.15.5:0/3059672833 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e6ce454670 msgr2=0x55e6ce45c3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:00:43.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:43.017+0000 7faa1d200640 1 --2- 172.21.15.5:0/3059672833 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e6ce454670 0x55e6ce45c3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:43.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:43.017+0000 7faa1d200640 1 -- 172.21.15.5:0/3059672833 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e6ce12b3c0 con 0x55e6ce453d50 2024-09-10T10:00:43.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:43.017+0000 7faa1d200640 1 --2- 172.21.15.5:0/3059672833 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e6ce453d50 0x55e6ce45be70 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7faa1002a930 tx=0x7faa10005fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0f593d551a6c584 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:00:43.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:43.017+0000 7faa02000640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-10T10:00:43.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:43.017+0000 7faa02000640 1 -- 172.21.15.5:0/3059672833 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa10030750 con 0x55e6ce453d50 2024-09-10T10:00:43.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:43.017+0000 7faa21c1c740 1 -- 172.21.15.5:0/3059672833 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faa10036040 con 0x55e6ce453d50 2024-09-10T10:00:43.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:43.017+0000 7faa02000640 1 -- 172.21.15.5:0/3059672833 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7faa10003970 con 0x55e6ce453d50 2024-09-10T10:00:43.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:43.018+0000 7faa02000640 1 -- 172.21.15.5:0/3059672833 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7faa1005e050 con 0x55e6ce453d50 2024-09-10T10:00:43.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:43.018+0000 7faa21c1c740 10 client.19620 fetch_fsmap learned FSMap version 42 2024-09-10T10:00:43.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:43.018+0000 7faa21c1c740 1 -- 172.21.15.5:0/3059672833 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55e6ce46d8d0 con 0x55e6ce453d50 2024-09-10T10:00:43.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:43.018+0000 7faa02000640 1 -- 172.21.15.5:0/3059672833 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faa10036040 con 0x55e6ce453d50 2024-09-10T10:00:43.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:43.018+0000 7faa02000640 1 -- 172.21.15.5:0/3059672833 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 42) ==== 8370+0+0 (secure 0 0 0) 0x7faa10031440 con 0x55e6ce453d50 2024-09-10T10:00:43.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:43.018+0000 7faa21c1c740 10 client.19620 fetch_fsmap finished waiting for FSMap version 42 2024-09-10T10:00:43.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:43.018+0000 7faa21c1c740 10 client.19620 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T10:00:43.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:43.018+0000 7faa21c1c740 20 client.19620 populate_metadata read hostname 'smithi005' 2024-09-10T10:00:43.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:43.018+0000 7faa21c1c740 1 --2- 172.21.15.5:0/3059672833 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55e6ce46ec20 0x55e6ce471010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:43.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:43.018+0000 7faa21c1c740 4 client.19620 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:00:43.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:43.018+0000 7faa21c1c740 1 -- 172.21.15.5:0/3059672833 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e6ce3c7950 con 0x55e6ce46ec20 2024-09-10T10:00:43.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:43.019+0000 7faa17400640 1 --2- 172.21.15.5:0/3059672833 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55e6ce46ec20 0x55e6ce471010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:43.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:43.019+0000 7faa17400640 1 --2- 172.21.15.5:0/3059672833 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55e6ce46ec20 0x55e6ce471010 crc :-1 s=READY pgs=307 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e5b6909bd2c91d36 server_cookie=53c3e3cd7433a02 in_seq=0 out_seq=0 2024-09-10T10:00:43.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:43.019+0000 7faa02000640 10 client.19620 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T10:00:43.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:43.020+0000 7faa02000640 1 -- 172.21.15.5:0/3059672833 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e6ce3c7950 con 0x55e6ce46ec20 2024-09-10T10:00:43.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:43.020+0000 7faa02000640 10 client.19620 handle_command_reply: tid=0 2024-09-10T10:00:43.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:43.020+0000 7faa21c1c740 1 client.19620 shutdown 2024-09-10T10:00:43.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:43.020+0000 7faa21c1c740 1 -- 172.21.15.5:0/3059672833 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e6ce453d50 msgr2=0x55e6ce45be70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:43.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:43.020+0000 7faa21c1c740 1 --2- 172.21.15.5:0/3059672833 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e6ce453d50 0x55e6ce45be70 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7faa1002a930 tx=0x7faa10005fb0 comp rx=0 tx=0).stop 2024-09-10T10:00:43.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:43.021+0000 7faa21c1c740 1 -- 172.21.15.5:0/3059672833 shutdown_connections 2024-09-10T10:00:43.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:43.021+0000 7faa21c1c740 1 --2- 172.21.15.5:0/3059672833 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55e6ce46ec20 0x55e6ce471010 crc :-1 s=READY pgs=307 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:43.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:43.021+0000 7faa21c1c740 1 --2- 172.21.15.5:0/3059672833 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e6ce45ef30 0x55e6ce45c8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:43.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:43.021+0000 7faa21c1c740 1 --2- 172.21.15.5:0/3059672833 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e6ce454670 0x55e6ce45c3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:43.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:43.021+0000 7faa21c1c740 1 --2- 172.21.15.5:0/3059672833 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e6ce453d50 0x55e6ce45be70 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:43.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:43.021+0000 7faa21c1c740 1 -- 172.21.15.5:0/3059672833 >> 172.21.15.5:0/3059672833 conn(0x55e6ce4502e0 msgr2=0x55e6ce455fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:00:43.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:43.021+0000 7faa02000640 0 client.19620 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T10:00:43.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:43.021+0000 7faa21c1c740 1 -- 172.21.15.5:0/3059672833 shutdown_connections 2024-09-10T10:00:43.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:43.021+0000 7faa21c1c740 1 -- 172.21.15.5:0/3059672833 wait complete. 2024-09-10T10:00:43.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:43.021+0000 7faa21c1c740 20 client.19620 trim_cache size 0 max 16384 2024-09-10T10:00:43.034 INFO:tasks.workunit.client.0.smithi005.stdout: 10 134561 13.52 MB/sec execute 381 sec latency 4549.896 ms 2024-09-10T10:00:43.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:43.039+0000 7faa21c1c740 1 Processor -- start 2024-09-10T10:00:43.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:43.039+0000 7faa21c1c740 1 -- start start 2024-09-10T10:00:43.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:43.040+0000 7faa21c1c740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e6ce453d50 0x55e6ce455eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:43.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:43.040+0000 7faa21c1c740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e6ce454670 0x55e6ce474030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:43.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:43.040+0000 7faa21c1c740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e6ce45ef30 0x55e6ce474570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:43.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:43.040+0000 7faa21c1c740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7faa181b0ef0 con 0x55e6ce454670 2024-09-10T10:00:43.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:43.040+0000 7faa21c1c740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7faa181b2a30 con 0x55e6ce453d50 2024-09-10T10:00:43.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:43.040+0000 7faa21c1c740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7faa181b15c0 con 0x55e6ce45ef30 2024-09-10T10:00:43.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:43.040+0000 7faa17400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e6ce454670 0x55e6ce474030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:43.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:43.040+0000 7faa17400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e6ce454670 0x55e6ce474030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41870/0 (socket says 172.21.15.5:41870) 2024-09-10T10:00:43.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:43.040+0000 7faa17400640 1 -- 172.21.15.5:0/1124291714 learned_addr learned my addr 172.21.15.5:0/1124291714 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T10:00:43.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:43.040+0000 7faa1d200640 1 --2- 172.21.15.5:0/1124291714 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e6ce453d50 0x55e6ce455eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:43.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:43.040+0000 7faa1d200640 1 --2- 172.21.15.5:0/1124291714 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e6ce45ef30 0x55e6ce474570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:43.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:43.040+0000 7faa17400640 1 -- 172.21.15.5:0/1124291714 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e6ce45ef30 msgr2=0x55e6ce474570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:00:43.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:43.040+0000 7faa17400640 1 --2- 172.21.15.5:0/1124291714 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e6ce45ef30 0x55e6ce474570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:43.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:43.040+0000 7faa17400640 1 -- 172.21.15.5:0/1124291714 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e6ce453d50 msgr2=0x55e6ce455eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:00:43.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:43.040+0000 7faa17400640 1 --2- 172.21.15.5:0/1124291714 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e6ce453d50 0x55e6ce455eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:43.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:43.040+0000 7faa17400640 1 -- 172.21.15.5:0/1124291714 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e6ce45a3e0 con 0x55e6ce454670 2024-09-10T10:00:43.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:43.040+0000 7faa1d200640 1 --2- 172.21.15.5:0/1124291714 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e6ce45ef30 0x55e6ce474570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:00:43.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:43.041+0000 7faa17400640 1 --2- 172.21.15.5:0/1124291714 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e6ce454670 0x55e6ce474030 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7faa04009dd0 tx=0x7faa04005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fdc549974fc69cc server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:00:43.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:43.041+0000 7faa0cc00640 1 -- 172.21.15.5:0/1124291714 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa04005220 con 0x55e6ce454670 2024-09-10T10:00:43.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:43.041+0000 7faa0cc00640 1 -- 172.21.15.5:0/1124291714 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faa04004c80 con 0x55e6ce454670 2024-09-10T10:00:43.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:43.041+0000 7faa0cc00640 1 -- 172.21.15.5:0/1124291714 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa0401ead0 con 0x55e6ce454670 2024-09-10T10:00:43.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:43.042+0000 7faa21c1c740 1 -- 172.21.15.5:0/1124291714 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e6ce454670 msgr2=0x55e6ce474030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:43.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:43.042+0000 7faa21c1c740 1 --2- 172.21.15.5:0/1124291714 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e6ce454670 0x55e6ce474030 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7faa04009dd0 tx=0x7faa04005f90 comp rx=0 tx=0).stop 2024-09-10T10:00:43.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:43.042+0000 7faa21c1c740 1 -- 172.21.15.5:0/1124291714 shutdown_connections 2024-09-10T10:00:43.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:43.042+0000 7faa21c1c740 1 --2- 172.21.15.5:0/1124291714 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e6ce45ef30 0x55e6ce474570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:43.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:43.042+0000 7faa21c1c740 1 --2- 172.21.15.5:0/1124291714 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e6ce454670 0x55e6ce474030 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:43.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:43.042+0000 7faa21c1c740 1 --2- 172.21.15.5:0/1124291714 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e6ce453d50 0x55e6ce455eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:43.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:43.042+0000 7faa21c1c740 1 -- 172.21.15.5:0/1124291714 >> 172.21.15.5:0/1124291714 conn(0x55e6ce4502e0 msgr2=0x55e6ce4557a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:00:43.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:43.042+0000 7faa21c1c740 1 -- 172.21.15.5:0/1124291714 shutdown_connections 2024-09-10T10:00:43.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:43.042+0000 7faa21c1c740 1 -- 172.21.15.5:0/1124291714 wait complete. 2024-09-10T10:00:43.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:43.042+0000 7faa21c1c740 1 -- start start 2024-09-10T10:00:43.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:43.042+0000 7faa21c1c740 1 Processor -- start 2024-09-10T10:00:43.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:43.043+0000 7faa21c1c740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e6ce453d50 0x55e6ce4721e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:43.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:43.043+0000 7faa21c1c740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e6ce454670 0x55e6ce472720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:43.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:43.043+0000 7faa21c1c740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e6ce45ef30 0x55e6ce4d8560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:43.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:43.043+0000 7faa21c1c740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7faa181b2360 con 0x55e6ce45ef30 2024-09-10T10:00:43.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:43.043+0000 7faa21c1c740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7faa181b1c90 con 0x55e6ce454670 2024-09-10T10:00:43.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:43.043+0000 7faa21c1c740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7faa181aeed0 con 0x55e6ce453d50 2024-09-10T10:00:43.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:43.043+0000 7faa21c1c740 10 client.0 _ll_register_callbacks cb 0x7faa04005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:00:43.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:43.043+0000 7faa1d200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e6ce453d50 0x55e6ce4721e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:43.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:43.043+0000 7faa1d200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e6ce45ef30 0x55e6ce4d8560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:43.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:43.043+0000 7faa1d200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e6ce453d50 0x55e6ce4721e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:49394/0 (socket says 172.21.15.5:49394) 2024-09-10T10:00:43.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:43.043+0000 7faa1d200640 1 -- 172.21.15.5:0/2763737386 learned_addr learned my addr 172.21.15.5:0/2763737386 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T10:00:43.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:43.043+0000 7faa17400640 1 --2- 172.21.15.5:0/2763737386 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e6ce454670 0x55e6ce472720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:43.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:43.044+0000 7faa17400640 1 -- 172.21.15.5:0/2763737386 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e6ce453d50 msgr2=0x55e6ce4721e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:00:43.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:43.044+0000 7faa17400640 1 --2- 172.21.15.5:0/2763737386 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e6ce453d50 0x55e6ce4721e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:43.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:43.044+0000 7faa17400640 1 -- 172.21.15.5:0/2763737386 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e6ce45ef30 msgr2=0x55e6ce4d8560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:00:43.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:43.044+0000 7faa17400640 1 --2- 172.21.15.5:0/2763737386 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e6ce45ef30 0x55e6ce4d8560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:43.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:43.044+0000 7faa17400640 1 -- 172.21.15.5:0/2763737386 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e6ce4d8aa0 con 0x55e6ce454670 2024-09-10T10:00:43.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:43.044+0000 7faa1d200640 1 --2- 172.21.15.5:0/2763737386 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e6ce45ef30 0x55e6ce4d8560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:00:43.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:43.044+0000 7faa17400640 1 --2- 172.21.15.5:0/2763737386 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e6ce454670 0x55e6ce472720 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7faa04009540 tx=0x7faa04004ec0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=96d0d926777fdedf server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:00:43.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:43.044+0000 7faa02000640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-09-10T10:00:43.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:43.044+0000 7faa02000640 1 -- 172.21.15.5:0/2763737386 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa04036060 con 0x55e6ce454670 2024-09-10T10:00:43.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:43.044+0000 7faa21c1c740 1 -- 172.21.15.5:0/2763737386 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faa04004c80 con 0x55e6ce454670 2024-09-10T10:00:43.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:43.045+0000 7faa02000640 1 -- 172.21.15.5:0/2763737386 <== mon.2 v2:172.21.15.102:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7faa04004a10 con 0x55e6ce454670 2024-09-10T10:00:43.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:43.045+0000 7faa02000640 1 -- 172.21.15.5:0/2763737386 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7faa0403d050 con 0x55e6ce454670 2024-09-10T10:00:43.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:43.045+0000 7faa21c1c740 10 client.29699 fetch_fsmap learned FSMap version 42 2024-09-10T10:00:43.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:43.045+0000 7faa21c1c740 1 -- 172.21.15.5:0/2763737386 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55e6ce4d9be0 con 0x55e6ce454670 2024-09-10T10:00:43.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:43.045+0000 7faa02000640 1 -- 172.21.15.5:0/2763737386 <== mon.2 v2:172.21.15.102:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faa04004c80 con 0x55e6ce454670 2024-09-10T10:00:43.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:43.045+0000 7faa02000640 1 -- 172.21.15.5:0/2763737386 <== mon.2 v2:172.21.15.102:3300/0 5 ==== fsmap(e 42) ==== 8370+0+0 (secure 0 0 0) 0x7faa0401ecd0 con 0x55e6ce454670 2024-09-10T10:00:43.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:43.045+0000 7faa21c1c740 10 client.29699 fetch_fsmap finished waiting for FSMap version 42 2024-09-10T10:00:43.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:43.045+0000 7faa21c1c740 10 client.29699 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T10:00:43.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:43.045+0000 7faa21c1c740 20 client.29699 populate_metadata read hostname 'smithi005' 2024-09-10T10:00:43.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:43.045+0000 7faa21c1c740 1 --2- 172.21.15.5:0/2763737386 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55e6ce4da2e0 0x55e6ce4dc700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:00:43.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:43.045+0000 7faa21c1c740 4 client.29699 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T10:00:43.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:43.045+0000 7faa21c1c740 1 -- 172.21.15.5:0/2763737386 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55e6ce3c1e60 con 0x55e6ce4da2e0 2024-09-10T10:00:43.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:43.046+0000 7faa1d200640 1 --2- 172.21.15.5:0/2763737386 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55e6ce4da2e0 0x55e6ce4dc700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:00:43.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:43.046+0000 7faa1d200640 1 --2- 172.21.15.5:0/2763737386 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55e6ce4da2e0 0x55e6ce4dc700 crc :-1 s=READY pgs=308 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f4fab1d1197926d6 server_cookie=cb716f8edae3cd82 in_seq=0 out_seq=0 2024-09-10T10:00:43.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:43.046+0000 7faa02000640 10 client.29699 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T10:00:43.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:43.047+0000 7faa02000640 1 -- 172.21.15.5:0/2763737386 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55e6ce3c1e60 con 0x55e6ce4da2e0 2024-09-10T10:00:43.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:43.047+0000 7faa02000640 10 client.29699 handle_command_reply: tid=0 2024-09-10T10:00:43.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:43.047+0000 7faa21c1c740 1 client.29699 shutdown 2024-09-10T10:00:43.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:43.047+0000 7faa21c1c740 1 -- 172.21.15.5:0/2763737386 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e6ce454670 msgr2=0x55e6ce472720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:43.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:43.047+0000 7faa21c1c740 1 --2- 172.21.15.5:0/2763737386 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e6ce454670 0x55e6ce472720 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7faa04009540 tx=0x7faa04004ec0 comp rx=0 tx=0).stop 2024-09-10T10:00:43.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:43.047+0000 7faa21c1c740 1 -- 172.21.15.5:0/2763737386 shutdown_connections 2024-09-10T10:00:43.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:43.047+0000 7faa21c1c740 1 --2- 172.21.15.5:0/2763737386 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55e6ce4da2e0 0x55e6ce4dc700 crc :-1 s=READY pgs=308 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:43.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:43.047+0000 7faa21c1c740 1 --2- 172.21.15.5:0/2763737386 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e6ce45ef30 0x55e6ce4d8560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:43.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:43.047+0000 7faa21c1c740 1 --2- 172.21.15.5:0/2763737386 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e6ce454670 0x55e6ce472720 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:43.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:43.047+0000 7faa21c1c740 1 --2- 172.21.15.5:0/2763737386 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e6ce453d50 0x55e6ce4721e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:43.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:43.047+0000 7faa21c1c740 1 -- 172.21.15.5:0/2763737386 >> 172.21.15.5:0/2763737386 conn(0x55e6ce4502e0 msgr2=0x55e6ce46a7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:00:43.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:43.047+0000 7faa02000640 0 client.29699 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T10:00:43.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:43.047+0000 7faa21c1c740 1 -- 172.21.15.5:0/2763737386 shutdown_connections 2024-09-10T10:00:43.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:43.047+0000 7faa21c1c740 1 -- 172.21.15.5:0/2763737386 wait complete. 2024-09-10T10:00:43.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:43.047+0000 7faa21c1c740 20 client.29699 trim_cache size 0 max 16384 2024-09-10T10:00:43.054 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-10T10:00:43.054 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-10T10:00:43.054 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2024-09-10T10:00:43.055 INFO:teuthology.orchestra.run.smithi005.stdout: "baac0d07-e881-4a67-9f71-de0a5215e662": { 2024-09-10T10:00:43.055 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2024-09-10T10:00:43.055 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "baac0d07-e881-4a67-9f71-de0a5215e662", 2024-09-10T10:00:43.055 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2024-09-10T10:00:43.055 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T10:00:43.055 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T10:00:43.055 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-10T10:00:43.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:43.048+0000 7faa1dc00640 1 -- 172.21.15.5:0/837481548 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fa9fc0a2c00 msgr2=0x7fa9fc0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:43.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:43.048+0000 7faa1dc00640 1 --2- 172.21.15.5:0/837481548 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fa9fc0a2c00 0x7fa9fc0a50c0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7faa080a42c0 tx=0x7faa080d5530 comp rx=0 tx=0).stop 2024-09-10T10:00:43.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:43.048+0000 7faa1dc00640 1 -- 172.21.15.5:0/837481548 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7faa181019c0 msgr2=0x7faa180744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:00:43.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:43.048+0000 7faa1dc00640 1 --2- 172.21.15.5:0/837481548 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7faa181019c0 0x7faa180744f0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7faa0800ea60 tx=0x7faa0800ef20 comp rx=0 tx=0).stop 2024-09-10T10:00:43.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:43.048+0000 7faa1dc00640 1 -- 172.21.15.5:0/837481548 shutdown_connections 2024-09-10T10:00:43.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:43.048+0000 7faa1dc00640 1 --2- 172.21.15.5:0/837481548 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fa9fc0a2c00 0x7fa9fc0a50c0 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:43.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:43.048+0000 7faa1dc00640 1 --2- 172.21.15.5:0/837481548 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7faa181079c0 0x7faa18074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:43.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:43.048+0000 7faa1dc00640 1 --2- 172.21.15.5:0/837481548 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faa18102360 0x7faa18074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:43.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:43.048+0000 7faa1dc00640 1 --2- 172.21.15.5:0/837481548 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7faa181019c0 0x7faa180744f0 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:00:43.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:43.048+0000 7faa1dc00640 1 -- 172.21.15.5:0/837481548 >> 172.21.15.5:0/837481548 conn(0x7faa180fd740 msgr2=0x7faa180feef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:00:43.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:43.048+0000 7faa1dc00640 1 -- 172.21.15.5:0/837481548 shutdown_connections 2024-09-10T10:00:43.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:00:43.048+0000 7faa1dc00640 1 -- 172.21.15.5:0/837481548 wait complete. 2024-09-10T10:00:43.947 INFO:tasks.cephfs.filesystem:scrub status for tag:baac0d07-e881-4a67-9f71-de0a5215e662 - {'path': '/', 'tag': 'baac0d07-e881-4a67-9f71-de0a5215e662', 'options': 'recursive,force'} 2024-09-10T10:00:44.035 INFO:tasks.workunit.client.0.smithi005.stdout: 10 134561 13.49 MB/sec execute 382 sec latency 5550.005 ms 2024-09-10T10:00:44.306 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:00:43 smithi005 ceph-mon[30350]: pgmap v1064: 129 pgs: 129 active+clean; 679 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 2.1 MiB/s wr, 328 op/s 2024-09-10T10:00:44.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:00:43 smithi159 ceph-mon[35513]: pgmap v1064: 129 pgs: 129 active+clean; 679 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 2.1 MiB/s wr, 328 op/s 2024-09-10T10:00:44.357 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:00:43 smithi102 ceph-mon[35442]: pgmap v1064: 129 pgs: 129 active+clean; 679 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 2.1 MiB/s wr, 328 op/s 2024-09-10T10:00:45.035 INFO:tasks.workunit.client.0.smithi005.stdout: 10 134561 13.45 MB/sec execute 383 sec latency 6550.176 ms 2024-09-10T10:00:46.035 INFO:tasks.workunit.client.0.smithi005.stdout: 10 134561 13.42 MB/sec execute 384 sec latency 7550.304 ms 2024-09-10T10:00:46.306 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:00:45 smithi005 ceph-mon[30350]: pgmap v1065: 129 pgs: 129 active+clean; 679 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 2.1 MiB/s wr, 332 op/s 2024-09-10T10:00:46.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:00:45 smithi159 ceph-mon[35513]: pgmap v1065: 129 pgs: 129 active+clean; 679 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 2.1 MiB/s wr, 332 op/s 2024-09-10T10:00:46.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:00:45 smithi102 ceph-mon[35442]: pgmap v1065: 129 pgs: 129 active+clean; 679 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 2.1 MiB/s wr, 332 op/s 2024-09-10T10:00:47.035 INFO:tasks.workunit.client.0.smithi005.stdout: 10 134561 13.38 MB/sec execute 385 sec latency 8550.436 ms 2024-09-10T10:00:47.306 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:00:46 smithi005 ceph-mon[30350]: pgmap v1066: 129 pgs: 129 active+clean; 679 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 2.0 MiB/s wr, 337 op/s 2024-09-10T10:00:47.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:00:46 smithi159 ceph-mon[35513]: pgmap v1066: 129 pgs: 129 active+clean; 679 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 2.0 MiB/s wr, 337 op/s 2024-09-10T10:00:47.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:00:46 smithi102 ceph-mon[35442]: pgmap v1066: 129 pgs: 129 active+clean; 679 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 2.0 MiB/s wr, 337 op/s 2024-09-10T10:00:48.035 INFO:tasks.workunit.client.0.smithi005.stdout: 10 134561 13.35 MB/sec execute 386 sec latency 9550.517 ms 2024-09-10T10:00:49.035 INFO:tasks.workunit.client.0.smithi005.stdout: 10 134561 13.32 MB/sec execute 387 sec latency 10550.639 ms 2024-09-10T10:00:49.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:00:49 smithi005 ceph-mon[30350]: pgmap v1067: 129 pgs: 129 active+clean; 679 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 1.6 MiB/s wr, 254 op/s 2024-09-10T10:00:49.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:00:49 smithi159 ceph-mon[35513]: pgmap v1067: 129 pgs: 129 active+clean; 679 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 1.6 MiB/s wr, 254 op/s 2024-09-10T10:00:49.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:00:49 smithi102 ceph-mon[35442]: pgmap v1067: 129 pgs: 129 active+clean; 679 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 1.6 MiB/s wr, 254 op/s 2024-09-10T10:00:50.036 INFO:tasks.workunit.client.0.smithi005.stdout: 10 134561 13.28 MB/sec execute 388 sec latency 11550.769 ms 2024-09-10T10:00:51.036 INFO:tasks.workunit.client.0.smithi005.stdout: 10 134561 13.25 MB/sec execute 389 sec latency 12550.946 ms 2024-09-10T10:00:51.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:00:51 smithi005 ceph-mon[30350]: pgmap v1068: 129 pgs: 129 active+clean; 680 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.4 MiB/s wr, 243 op/s 2024-09-10T10:00:51.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:00:51 smithi159 ceph-mon[35513]: pgmap v1068: 129 pgs: 129 active+clean; 680 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.4 MiB/s wr, 243 op/s 2024-09-10T10:00:51.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:00:51 smithi102 ceph-mon[35442]: pgmap v1068: 129 pgs: 129 active+clean; 680 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.4 MiB/s wr, 243 op/s 2024-09-10T10:00:52.036 INFO:tasks.workunit.client.0.smithi005.stdout: 10 134561 13.21 MB/sec execute 390 sec latency 13551.097 ms 2024-09-10T10:00:52.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:00:52 smithi005 ceph-mon[30350]: scrub summary: idle 2024-09-10T10:00:52.557 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:00:52 smithi005 ceph-mon[30350]: scrub summary: idle 2024-09-10T10:00:52.557 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:00:52 smithi005 ceph-mon[30350]: scrub summary: idle 2024-09-10T10:00:52.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:00:52 smithi159 ceph-mon[35513]: scrub summary: idle 2024-09-10T10:00:52.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:00:52 smithi159 ceph-mon[35513]: scrub summary: idle 2024-09-10T10:00:52.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:00:52 smithi159 ceph-mon[35513]: scrub summary: idle 2024-09-10T10:00:52.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:00:52 smithi102 ceph-mon[35442]: scrub summary: idle 2024-09-10T10:00:52.635 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:00:52 smithi102 ceph-mon[35442]: scrub summary: idle 2024-09-10T10:00:52.635 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:00:52 smithi102 ceph-mon[35442]: scrub summary: idle 2024-09-10T10:00:53.036 INFO:tasks.workunit.client.0.smithi005.stdout: 10 134561 13.18 MB/sec execute 391 sec latency 14551.202 ms 2024-09-10T10:00:53.502 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:00:53 smithi159 ceph-mon[35513]: scrub summary: idle 2024-09-10T10:00:53.502 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:00:53 smithi159 ceph-mon[35513]: scrub summary: idle 2024-09-10T10:00:53.502 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:00:53 smithi159 ceph-mon[35513]: pgmap v1069: 129 pgs: 129 active+clean; 679 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 282 KiB/s wr, 69 op/s 2024-09-10T10:00:53.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:00:53 smithi005 ceph-mon[30350]: scrub summary: idle 2024-09-10T10:00:53.557 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:00:53 smithi005 ceph-mon[30350]: scrub summary: idle 2024-09-10T10:00:53.557 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:00:53 smithi005 ceph-mon[30350]: pgmap v1069: 129 pgs: 129 active+clean; 679 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 282 KiB/s wr, 69 op/s 2024-09-10T10:00:53.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:00:53 smithi102 ceph-mon[35442]: scrub summary: idle 2024-09-10T10:00:53.635 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:00:53 smithi102 ceph-mon[35442]: scrub summary: idle 2024-09-10T10:00:53.635 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:00:53 smithi102 ceph-mon[35442]: pgmap v1069: 129 pgs: 129 active+clean; 679 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 282 KiB/s wr, 69 op/s 2024-09-10T10:00:54.036 INFO:tasks.workunit.client.0.smithi005.stdout: 10 134561 13.15 MB/sec execute 392 sec latency 15551.346 ms 2024-09-10T10:00:55.036 INFO:tasks.workunit.client.0.smithi005.stdout: 10 134561 13.11 MB/sec execute 393 sec latency 16551.512 ms 2024-09-10T10:00:55.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:00:55 smithi005 ceph-mon[30350]: pgmap v1070: 129 pgs: 129 active+clean; 679 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 201 KiB/s wr, 51 op/s 2024-09-10T10:00:55.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:00:55 smithi159 ceph-mon[35513]: pgmap v1070: 129 pgs: 129 active+clean; 679 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 201 KiB/s wr, 51 op/s 2024-09-10T10:00:55.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:00:55 smithi102 ceph-mon[35442]: pgmap v1070: 129 pgs: 129 active+clean; 679 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 201 KiB/s wr, 51 op/s 2024-09-10T10:00:56.037 INFO:tasks.workunit.client.0.smithi005.stdout: 10 134561 13.08 MB/sec execute 394 sec latency 17551.664 ms 2024-09-10T10:00:57.037 INFO:tasks.workunit.client.0.smithi005.stdout: 10 134561 13.05 MB/sec execute 395 sec latency 18551.821 ms 2024-09-10T10:00:57.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:00:57 smithi005 ceph-mon[30350]: pgmap v1071: 129 pgs: 129 active+clean; 679 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 172 KiB/s wr, 44 op/s 2024-09-10T10:00:57.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:00:57 smithi159 ceph-mon[35513]: pgmap v1071: 129 pgs: 129 active+clean; 679 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 172 KiB/s wr, 44 op/s 2024-09-10T10:00:57.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:00:57 smithi102 ceph-mon[35442]: pgmap v1071: 129 pgs: 129 active+clean; 679 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 172 KiB/s wr, 44 op/s 2024-09-10T10:00:58.037 INFO:tasks.workunit.client.0.smithi005.stdout: 10 134561 13.01 MB/sec execute 396 sec latency 19551.978 ms 2024-09-10T10:00:59.037 INFO:tasks.workunit.client.0.smithi005.stdout: 10 134561 12.98 MB/sec execute 397 sec latency 20552.101 ms 2024-09-10T10:00:59.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:00:59 smithi005 ceph-mon[30350]: pgmap v1072: 129 pgs: 129 active+clean; 679 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 157 KiB/s wr, 18 op/s 2024-09-10T10:00:59.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:00:59 smithi159 ceph-mon[35513]: pgmap v1072: 129 pgs: 129 active+clean; 679 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 157 KiB/s wr, 18 op/s 2024-09-10T10:00:59.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:00:59 smithi102 ceph-mon[35442]: pgmap v1072: 129 pgs: 129 active+clean; 679 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 157 KiB/s wr, 18 op/s 2024-09-10T10:01:00.037 INFO:tasks.workunit.client.0.smithi005.stdout: 10 134561 12.95 MB/sec execute 398 sec latency 21552.268 ms 2024-09-10T10:01:01.037 INFO:tasks.workunit.client.0.smithi005.stdout: 10 134561 12.91 MB/sec execute 399 sec latency 22552.433 ms 2024-09-10T10:01:01.434 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:01:01 smithi005 ceph-mon[30350]: pgmap v1073: 129 pgs: 129 active+clean; 679 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 106 KiB/s wr, 24 op/s 2024-09-10T10:01:01.552 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:01:01 smithi159 ceph-mon[35513]: pgmap v1073: 129 pgs: 129 active+clean; 679 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 106 KiB/s wr, 24 op/s 2024-09-10T10:01:01.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:01:01 smithi102 ceph-mon[35442]: pgmap v1073: 129 pgs: 129 active+clean; 679 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 106 KiB/s wr, 24 op/s 2024-09-10T10:01:02.037 INFO:tasks.workunit.client.0.smithi005.stdout: 10 134561 12.88 MB/sec execute 400 sec latency 23552.588 ms 2024-09-10T10:01:03.039 INFO:tasks.workunit.client.0.smithi005.stdout: 10 134561 12.85 MB/sec execute 401 sec latency 24553.815 ms 2024-09-10T10:01:03.297 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:01:03 smithi005 ceph-mon[30350]: pgmap v1074: 129 pgs: 129 active+clean; 679 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 17 KiB/s wr, 14 op/s 2024-09-10T10:01:03.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:01:03 smithi159 ceph-mon[35513]: pgmap v1074: 129 pgs: 129 active+clean; 679 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 17 KiB/s wr, 14 op/s 2024-09-10T10:01:03.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:01:03 smithi102 ceph-mon[35442]: pgmap v1074: 129 pgs: 129 active+clean; 679 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 17 KiB/s wr, 14 op/s 2024-09-10T10:01:04.039 INFO:tasks.workunit.client.0.smithi005.stdout: 10 134561 12.82 MB/sec execute 402 sec latency 25553.871 ms 2024-09-10T10:01:05.039 INFO:tasks.workunit.client.0.smithi005.stdout: 10 134561 12.79 MB/sec execute 403 sec latency 26553.935 ms 2024-09-10T10:01:05.277 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:01:05 smithi005 ceph-mon[30350]: pgmap v1075: 129 pgs: 129 active+clean; 687 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 786 KiB/s wr, 23 op/s 2024-09-10T10:01:05.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:01:05 smithi159 ceph-mon[35513]: pgmap v1075: 129 pgs: 129 active+clean; 687 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 786 KiB/s wr, 23 op/s 2024-09-10T10:01:05.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:01:05 smithi102 ceph-mon[35442]: pgmap v1075: 129 pgs: 129 active+clean; 687 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 786 KiB/s wr, 23 op/s 2024-09-10T10:01:06.039 INFO:tasks.workunit.client.0.smithi005.stdout: 10 134561 12.75 MB/sec execute 404 sec latency 27553.982 ms 2024-09-10T10:01:06.306 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T10:01:06.774 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:01:07.039 INFO:tasks.workunit.client.0.smithi005.stdout: 10 134561 12.72 MB/sec execute 405 sec latency 28554.027 ms 2024-09-10T10:01:07.290 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:01:07 smithi005 ceph-mon[30350]: pgmap v1076: 129 pgs: 129 active+clean; 696 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 2.2 MiB/s wr, 55 op/s 2024-09-10T10:01:07.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:01:07 smithi159 ceph-mon[35513]: pgmap v1076: 129 pgs: 129 active+clean; 696 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 2.2 MiB/s wr, 55 op/s 2024-09-10T10:01:07.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:01:07 smithi102 ceph-mon[35442]: pgmap v1076: 129 pgs: 129 active+clean; 696 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 2.2 MiB/s wr, 55 op/s 2024-09-10T10:01:07.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:07.708+0000 7f12aae00640 1 -- 172.21.15.5:0/410527550 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f12a4104000 msgr2=0x7f12a41043e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:07.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:07.708+0000 7f12aae00640 1 --2- 172.21.15.5:0/410527550 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f12a4104000 0x7f12a41043e0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f1290009900 tx=0x7f129002a400 comp rx=0 tx=0).stop 2024-09-10T10:01:07.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:07.708+0000 7f12aae00640 1 -- 172.21.15.5:0/410527550 shutdown_connections 2024-09-10T10:01:07.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:07.708+0000 7f12aae00640 1 --2- 172.21.15.5:0/410527550 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f12a4109130 0x7f12a4110cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:07.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:07.708+0000 7f12aae00640 1 --2- 172.21.15.5:0/410527550 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f12a41049b0 0x7f12a4108a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:07.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:07.708+0000 7f12aae00640 1 --2- 172.21.15.5:0/410527550 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f12a4104000 0x7f12a41043e0 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:07.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:07.708+0000 7f12aae00640 1 -- 172.21.15.5:0/410527550 >> 172.21.15.5:0/410527550 conn(0x7f12a40ffa20 msgr2=0x7f12a4101e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:07.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:07.708+0000 7f12aae00640 1 -- 172.21.15.5:0/410527550 shutdown_connections 2024-09-10T10:01:07.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:07.708+0000 7f12aae00640 1 -- 172.21.15.5:0/410527550 wait complete. 2024-09-10T10:01:07.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:07.709+0000 7f12aae00640 1 Processor -- start 2024-09-10T10:01:07.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:07.710+0000 7f12aae00640 1 -- start start 2024-09-10T10:01:07.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:07.710+0000 7f12aae00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f12a4104000 0x7f12a4110af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:07.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:07.710+0000 7f12aae00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f12a41049b0 0x7f12a410dba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:07.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:07.711+0000 7f12aae00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f12a4109130 0x7f12a410e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:07.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:07.711+0000 7f12a3e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f12a4104000 0x7f12a4110af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:07.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:07.711+0000 7f12a3e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f12a4104000 0x7f12a4110af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:59142/0 (socket says 172.21.15.5:59142) 2024-09-10T10:01:07.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:07.711+0000 7f12a3e00640 1 -- 172.21.15.5:0/1905332785 learned_addr learned my addr 172.21.15.5:0/1905332785 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T10:01:07.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:07.711+0000 7f12aae00640 1 -- 172.21.15.5:0/1905332785 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f12a4114180 con 0x7f12a4109130 2024-09-10T10:01:07.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:07.711+0000 7f12aae00640 1 -- 172.21.15.5:0/1905332785 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f12a4114000 con 0x7f12a4104000 2024-09-10T10:01:07.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:07.711+0000 7f12aae00640 1 -- 172.21.15.5:0/1905332785 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f12a4114300 con 0x7f12a41049b0 2024-09-10T10:01:07.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:07.712+0000 7f12a8e00640 1 --2- 172.21.15.5:0/1905332785 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f12a4109130 0x7f12a410e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:07.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:07.712+0000 7f12a3e00640 1 -- 172.21.15.5:0/1905332785 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f12a41049b0 msgr2=0x7f12a410dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:07.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:07.713+0000 7f12a3400640 1 --2- 172.21.15.5:0/1905332785 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f12a41049b0 0x7f12a410dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:07.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:07.713+0000 7f12a3e00640 1 --2- 172.21.15.5:0/1905332785 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f12a41049b0 0x7f12a410dba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:07.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:07.713+0000 7f12a3e00640 1 -- 172.21.15.5:0/1905332785 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f12a4109130 msgr2=0x7f12a410e0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:07.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:07.713+0000 7f12a3e00640 1 --2- 172.21.15.5:0/1905332785 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f12a4109130 0x7f12a410e0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:07.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:07.713+0000 7f12a3e00640 1 -- 172.21.15.5:0/1905332785 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12a410e950 con 0x7f12a4104000 2024-09-10T10:01:07.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:07.714+0000 7f12a8e00640 1 --2- 172.21.15.5:0/1905332785 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f12a4109130 0x7f12a410e0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:01:07.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:07.714+0000 7f12a3400640 1 --2- 172.21.15.5:0/1905332785 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f12a41049b0 0x7f12a410dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:01:07.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:07.714+0000 7f12a3e00640 1 --2- 172.21.15.5:0/1905332785 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f12a4104000 0x7f12a4110af0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f1290005ab0 tx=0x7f1290003bb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:07.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:07.714+0000 7f12a0c00640 1 -- 172.21.15.5:0/1905332785 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f129003f070 con 0x7f12a4104000 2024-09-10T10:01:07.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:07.715+0000 7f12a0c00640 1 -- 172.21.15.5:0/1905332785 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f129003b460 con 0x7f12a4104000 2024-09-10T10:01:07.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:07.715+0000 7f12a0c00640 1 -- 172.21.15.5:0/1905332785 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1290035030 con 0x7f12a4104000 2024-09-10T10:01:07.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:07.715+0000 7f12aae00640 1 -- 172.21.15.5:0/1905332785 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12a41aaf20 con 0x7f12a4104000 2024-09-10T10:01:07.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:07.715+0000 7f12aae00640 1 -- 172.21.15.5:0/1905332785 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12a41ab380 con 0x7f12a4104000 2024-09-10T10:01:07.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:07.717+0000 7f129a000640 1 -- 172.21.15.5:0/1905332785 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f127c005290 con 0x7f12a4104000 2024-09-10T10:01:07.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:07.721+0000 7f12a0c00640 1 -- 172.21.15.5:0/1905332785 <== mon.2 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f1290042070 con 0x7f12a4104000 2024-09-10T10:01:07.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:07.724+0000 7f12a0c00640 1 --2- 172.21.15.5:0/1905332785 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f12880a2cc0 0x7f12880a5180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:07.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:07.724+0000 7f12a0c00640 1 -- 172.21.15.5:0/1905332785 <== mon.2 v2:172.21.15.102:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f12900ef590 con 0x7f12a4104000 2024-09-10T10:01:07.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:07.725+0000 7f12a3400640 1 --2- 172.21.15.5:0/1905332785 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f12880a2cc0 0x7f12880a5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:07.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:07.727+0000 7f12a3400640 1 --2- 172.21.15.5:0/1905332785 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f12880a2cc0 0x7f12880a5180 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f12a410f360 tx=0x7f1294005d30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:07.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:07.728+0000 7f12a0c00640 1 -- 172.21.15.5:0/1905332785 <== mon.2 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f12900bc0a0 con 0x7f12a4104000 2024-09-10T10:01:08.039 INFO:tasks.workunit.client.0.smithi005.stdout: 10 134561 12.69 MB/sec execute 406 sec latency 29554.075 ms 2024-09-10T10:01:08.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:08.088+0000 7f129a000640 1 -- 172.21.15.5:0/1905332785 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f127c005580 con 0x7f12a4104000 2024-09-10T10:01:08.107 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-10T10:01:08.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:08.104+0000 7f12a0c00640 1 -- 172.21.15.5:0/1905332785 <== mon.2 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 42 v42) ==== 94+0+11969 (secure 0 0 0) 0x7f12900bb9a0 con 0x7f12a4104000 2024-09-10T10:01:08.108 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":42,"btime":"2024-09-10T09:55:06:670916+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14454,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/2970594311","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2970594311},{"type":"v1","addr":"172.21.15.5:6835","nonce":2970594311}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14514,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6839/3616502174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":3616502174},{"type":"v1","addr":"172.21.15.5:6839","nonce":3616502174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":42,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T09:35:29.626287+0000","modified":"2024-09-10T09:55:06.419829+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24542,"mds_1":24535,"mds_2":24511,"mds_3":24506,"mds_4":24443},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14520":{"gid":14520,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/2073599842","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":2073599842},{"type":"v1","addr":"172.21.15.5:6841","nonce":2073599842}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"k","rank":4,"incarnation":26,"state":"up:active","state_seq":24,"addr":"172.21.15.102:6833/2457007427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":2457007427},{"type":"v1","addr":"172.21.15.102:6833","nonce":2457007427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/356445159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":356445159},{"type":"v1","addr":"172.21.15.159:6833","nonce":356445159}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6837/2894314507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":2894314507},{"type":"v1","addr":"172.21.15.5:6837","nonce":2894314507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.102:6835/1647549111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":1647549111},{"type":"v1","addr":"172.21.15.102:6835","nonce":1647549111}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6835/952657242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":952657242},{"type":"v1","addr":"172.21.15.159:6835","nonce":952657242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":19,"addr":"172.21.15.102:6837/2689776764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":2689776764},{"type":"v1","addr":"172.21.15.102:6837","nonce":2689776764}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.159:6837/620627639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":620627639},{"type":"v1","addr":"172.21.15.159:6837","nonce":620627639}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24535":{"gid":24535,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":18,"addr":"172.21.15.159:6839/2073739599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":2073739599},{"type":"v1","addr":"172.21.15.159:6839","nonce":2073739599}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24542":{"gid":24542,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.102:6839/335409749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6838","nonce":335409749},{"type":"v1","addr":"172.21.15.102:6839","nonce":335409749}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24542,"qdb_cluster":[24542,24535,24511,24506,24443]},"id":1}]} 2024-09-10T10:01:08.108 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 42 2024-09-10T10:01:08.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:08.110+0000 7f129a000640 1 -- 172.21.15.5:0/1905332785 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f12880a2cc0 msgr2=0x7f12880a5180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:08.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:08.110+0000 7f129a000640 1 --2- 172.21.15.5:0/1905332785 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f12880a2cc0 0x7f12880a5180 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f12a410f360 tx=0x7f1294005d30 comp rx=0 tx=0).stop 2024-09-10T10:01:08.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:08.110+0000 7f129a000640 1 -- 172.21.15.5:0/1905332785 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f12a4104000 msgr2=0x7f12a4110af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:08.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:08.110+0000 7f129a000640 1 --2- 172.21.15.5:0/1905332785 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f12a4104000 0x7f12a4110af0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f1290005ab0 tx=0x7f1290003bb0 comp rx=0 tx=0).stop 2024-09-10T10:01:08.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:08.111+0000 7f129a000640 1 -- 172.21.15.5:0/1905332785 shutdown_connections 2024-09-10T10:01:08.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:08.111+0000 7f129a000640 1 --2- 172.21.15.5:0/1905332785 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f12880a2cc0 0x7f12880a5180 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:08.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:08.111+0000 7f129a000640 1 --2- 172.21.15.5:0/1905332785 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f12a4109130 0x7f12a410e0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:08.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:08.111+0000 7f129a000640 1 --2- 172.21.15.5:0/1905332785 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f12a41049b0 0x7f12a410dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:08.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:08.111+0000 7f129a000640 1 --2- 172.21.15.5:0/1905332785 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f12a4104000 0x7f12a4110af0 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:08.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:08.111+0000 7f129a000640 1 -- 172.21.15.5:0/1905332785 >> 172.21.15.5:0/1905332785 conn(0x7f12a40ffa20 msgr2=0x7f12a4100d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:08.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:08.111+0000 7f129a000640 1 -- 172.21.15.5:0/1905332785 shutdown_connections 2024-09-10T10:01:08.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:08.111+0000 7f129a000640 1 -- 172.21.15.5:0/1905332785 wait complete. 2024-09-10T10:01:08.191 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:01:08 smithi005 ceph-mon[30350]: from='client.? 172.21.15.5:0/1905332785' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:01:08.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:01:08 smithi159 ceph-mon[35513]: from='client.? 172.21.15.5:0/1905332785' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:01:08.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:01:08 smithi102 ceph-mon[35442]: from='client.? 172.21.15.5:0/1905332785' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:01:09.039 INFO:tasks.workunit.client.0.smithi005.stdout: 10 134561 12.66 MB/sec execute 407 sec latency 30554.131 ms 2024-09-10T10:01:09.056 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24542 quiesce db --set-id 8aeff82a --release --await' 2024-09-10T10:01:09.056 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24542 quiesce db --set-id 8aeff82a --release --await 2024-09-10T10:01:09.306 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:01:09 smithi005 ceph-mon[30350]: pgmap v1077: 129 pgs: 129 active+clean; 698 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 2.9 MiB/s wr, 68 op/s 2024-09-10T10:01:09.441 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:01:09.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:01:09 smithi159 ceph-mon[35513]: pgmap v1077: 129 pgs: 129 active+clean; 698 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 2.9 MiB/s wr, 68 op/s 2024-09-10T10:01:09.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:01:09 smithi102 ceph-mon[35442]: pgmap v1077: 129 pgs: 129 active+clean; 698 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 2.9 MiB/s wr, 68 op/s 2024-09-10T10:01:10.039 INFO:tasks.workunit.client.0.smithi005.stdout: 10 134561 12.63 MB/sec execute 408 sec latency 31554.269 ms 2024-09-10T10:01:10.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:10.240+0000 7ffb43e00640 1 -- 172.21.15.5:0/2028677050 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffb44108f60 msgr2=0x7ffb44110ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:10.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:10.240+0000 7ffb43e00640 1 --2- 172.21.15.5:0/2028677050 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffb44108f60 0x7ffb44110ae0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7ffb3c009900 tx=0x7ffb3c02a3f0 comp rx=0 tx=0).stop 2024-09-10T10:01:10.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:10.241+0000 7ffb43e00640 1 -- 172.21.15.5:0/2028677050 shutdown_connections 2024-09-10T10:01:10.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:10.241+0000 7ffb43e00640 1 --2- 172.21.15.5:0/2028677050 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffb44108f60 0x7ffb44110ae0 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:10.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:10.241+0000 7ffb43e00640 1 --2- 172.21.15.5:0/2028677050 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ffb441047e0 0x7ffb44108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:10.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:10.241+0000 7ffb43e00640 1 --2- 172.21.15.5:0/2028677050 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ffb44103e30 0x7ffb44104210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:10.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:10.241+0000 7ffb43e00640 1 -- 172.21.15.5:0/2028677050 >> 172.21.15.5:0/2028677050 conn(0x7ffb440ff8d0 msgr2=0x7ffb44101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:10.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:10.241+0000 7ffb43e00640 1 -- 172.21.15.5:0/2028677050 shutdown_connections 2024-09-10T10:01:10.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:10.241+0000 7ffb43e00640 1 -- 172.21.15.5:0/2028677050 wait complete. 2024-09-10T10:01:10.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:10.242+0000 7ffb43e00640 1 Processor -- start 2024-09-10T10:01:10.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:10.242+0000 7ffb43e00640 1 -- start start 2024-09-10T10:01:10.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:10.242+0000 7ffb43e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ffb44103e30 0x7ffb4419a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:10.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:10.242+0000 7ffb43e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ffb441047e0 0x7ffb4419a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:10.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:10.242+0000 7ffb43e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffb44108f60 0x7ffb4419eca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:10.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:10.242+0000 7ffb43e00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ffb44114010 con 0x7ffb44108f60 2024-09-10T10:01:10.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:10.242+0000 7ffb43e00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7ffb44113e90 con 0x7ffb44103e30 2024-09-10T10:01:10.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:10.242+0000 7ffb43e00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7ffb44114190 con 0x7ffb441047e0 2024-09-10T10:01:10.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:10.243+0000 7ffb43400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffb44108f60 0x7ffb4419eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:10.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:10.243+0000 7ffb3a000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ffb441047e0 0x7ffb4419a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:10.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:10.243+0000 7ffb43400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffb44108f60 0x7ffb4419eca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60368/0 (socket says 172.21.15.5:60368) 2024-09-10T10:01:10.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:10.243+0000 7ffb42a00640 1 --2- 172.21.15.5:0/1419142458 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ffb44103e30 0x7ffb4419a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:10.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:10.243+0000 7ffb43400640 1 -- 172.21.15.5:0/1419142458 learned_addr learned my addr 172.21.15.5:0/1419142458 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T10:01:10.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:10.243+0000 7ffb3a000640 1 -- 172.21.15.5:0/1419142458 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ffb44103e30 msgr2=0x7ffb4419a3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:10.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:10.243+0000 7ffb3a000640 1 --2- 172.21.15.5:0/1419142458 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ffb44103e30 0x7ffb4419a3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:10.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:10.243+0000 7ffb3a000640 1 -- 172.21.15.5:0/1419142458 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffb44108f60 msgr2=0x7ffb4419eca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:10.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:10.243+0000 7ffb3a000640 1 --2- 172.21.15.5:0/1419142458 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffb44108f60 0x7ffb4419eca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:10.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:10.243+0000 7ffb3a000640 1 -- 172.21.15.5:0/1419142458 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb4419f380 con 0x7ffb441047e0 2024-09-10T10:01:10.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:10.243+0000 7ffb42a00640 1 --2- 172.21.15.5:0/1419142458 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ffb44103e30 0x7ffb4419a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:01:10.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:10.243+0000 7ffb43400640 1 --2- 172.21.15.5:0/1419142458 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffb44108f60 0x7ffb4419eca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:01:10.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:10.243+0000 7ffb3a000640 1 --2- 172.21.15.5:0/1419142458 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ffb441047e0 0x7ffb4419a910 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7ffb2c00e9f0 tx=0x7ffb2c00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:10.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:10.244+0000 7ffb3be00640 1 -- 172.21.15.5:0/1419142458 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb2c00cd80 con 0x7ffb441047e0 2024-09-10T10:01:10.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:10.244+0000 7ffb3be00640 1 -- 172.21.15.5:0/1419142458 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffb2c0035f0 con 0x7ffb441047e0 2024-09-10T10:01:10.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:10.244+0000 7ffb3be00640 1 -- 172.21.15.5:0/1419142458 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb2c010690 con 0x7ffb441047e0 2024-09-10T10:01:10.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:10.244+0000 7ffb43e00640 1 -- 172.21.15.5:0/1419142458 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb4419f670 con 0x7ffb441047e0 2024-09-10T10:01:10.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:10.244+0000 7ffb43e00640 1 -- 172.21.15.5:0/1419142458 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb441a6f50 con 0x7ffb441047e0 2024-09-10T10:01:10.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:10.248+0000 7ffb4bfcb740 1 Processor -- start 2024-09-10T10:01:10.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:10.248+0000 7ffb4bfcb740 1 -- start start 2024-09-10T10:01:10.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:10.248+0000 7ffb3be00640 1 -- 172.21.15.5:0/1419142458 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7ffb2c003b30 con 0x7ffb441047e0 2024-09-10T10:01:10.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:10.249+0000 7ffb4bfcb740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56059a9e4520 0x56059a9e4900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:10.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:10.249+0000 7ffb4bfcb740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56059a9e4e40 0x56059a9ef1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:10.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:10.249+0000 7ffb4bfcb740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56059a9ef700 0x56059a9f1af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:10.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:10.249+0000 7ffb4bfcb740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x56059a94a650 con 0x56059a9e4e40 2024-09-10T10:01:10.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:10.249+0000 7ffb4bfcb740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x56059a8f1f10 con 0x56059a9ef700 2024-09-10T10:01:10.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:10.249+0000 7ffb4bfcb740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x56059a946610 con 0x56059a9e4520 2024-09-10T10:01:10.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:10.249+0000 7ffb42a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56059a9e4e40 0x56059a9ef1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:10.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:10.249+0000 7ffb42a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56059a9e4e40 0x56059a9ef1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60376/0 (socket says 172.21.15.5:60376) 2024-09-10T10:01:10.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:10.249+0000 7ffb42a00640 1 -- 172.21.15.5:0/2009865496 learned_addr learned my addr 172.21.15.5:0/2009865496 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T10:01:10.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:10.249+0000 7ffb43400640 1 --2- 172.21.15.5:0/2009865496 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56059a9e4520 0x56059a9e4900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:10.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:10.249+0000 7ffb3a000640 1 --2- 172.21.15.5:0/2009865496 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56059a9ef700 0x56059a9f1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:10.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:10.250+0000 7ffb42a00640 1 -- 172.21.15.5:0/2009865496 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56059a9e4520 msgr2=0x56059a9e4900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:01:10.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:10.250+0000 7ffb42a00640 1 --2- 172.21.15.5:0/2009865496 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56059a9e4520 0x56059a9e4900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:10.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:10.250+0000 7ffb42a00640 1 -- 172.21.15.5:0/2009865496 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56059a9ef700 msgr2=0x56059a9f1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:01:10.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:10.250+0000 7ffb42a00640 1 --2- 172.21.15.5:0/2009865496 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56059a9ef700 0x56059a9f1af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:10.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:10.250+0000 7ffb42a00640 1 -- 172.21.15.5:0/2009865496 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56059a659580 con 0x56059a9e4e40 2024-09-10T10:01:10.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:10.250+0000 7ffb3a000640 1 --2- 172.21.15.5:0/2009865496 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56059a9ef700 0x56059a9f1af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:01:10.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:10.250+0000 7ffb42a00640 1 --2- 172.21.15.5:0/2009865496 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56059a9e4e40 0x56059a9ef1c0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7ffb34011860 tx=0x7ffb34011d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71b7023c050d163b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:10.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:10.250+0000 7ffb33400640 1 -- 172.21.15.5:0/2009865496 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb340112f0 con 0x56059a9e4e40 2024-09-10T10:01:10.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:10.250+0000 7ffb33400640 1 -- 172.21.15.5:0/2009865496 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffb34003e50 con 0x56059a9e4e40 2024-09-10T10:01:10.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:10.250+0000 7ffb33400640 1 -- 172.21.15.5:0/2009865496 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb34004590 con 0x56059a9e4e40 2024-09-10T10:01:10.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:10.250+0000 7ffb3be00640 1 --2- 172.21.15.5:0/1419142458 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7ffb280a2c00 0x7ffb280a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:10.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:10.251+0000 7ffb3be00640 1 -- 172.21.15.5:0/1419142458 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7ffb2c0cf630 con 0x7ffb441047e0 2024-09-10T10:01:10.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:10.251+0000 7ffb43400640 1 --2- 172.21.15.5:0/1419142458 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7ffb280a2c00 0x7ffb280a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:10.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:10.251+0000 7ffb4bfcb740 1 -- 172.21.15.5:0/2009865496 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56059a9e4e40 msgr2=0x56059a9ef1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:10.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:10.251+0000 7ffb4bfcb740 1 --2- 172.21.15.5:0/2009865496 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56059a9e4e40 0x56059a9ef1c0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7ffb34011860 tx=0x7ffb34011d20 comp rx=0 tx=0).stop 2024-09-10T10:01:10.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:10.251+0000 7ffb4bfcb740 1 -- 172.21.15.5:0/2009865496 shutdown_connections 2024-09-10T10:01:10.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:10.251+0000 7ffb4bfcb740 1 --2- 172.21.15.5:0/2009865496 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56059a9ef700 0x56059a9f1af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:10.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:10.251+0000 7ffb4bfcb740 1 --2- 172.21.15.5:0/2009865496 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56059a9e4e40 0x56059a9ef1c0 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:10.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:10.251+0000 7ffb4bfcb740 1 --2- 172.21.15.5:0/2009865496 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56059a9e4520 0x56059a9e4900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:10.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:10.251+0000 7ffb4bfcb740 1 -- 172.21.15.5:0/2009865496 >> 172.21.15.5:0/2009865496 conn(0x56059a9e0ab0 msgr2=0x56059a9e0e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:10.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:10.251+0000 7ffb4bfcb740 1 -- 172.21.15.5:0/2009865496 shutdown_connections 2024-09-10T10:01:10.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:10.251+0000 7ffb4bfcb740 1 -- 172.21.15.5:0/2009865496 wait complete. 2024-09-10T10:01:10.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:10.252+0000 7ffb43400640 1 --2- 172.21.15.5:0/1419142458 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7ffb280a2c00 0x7ffb280a50c0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7ffb3c0096c0 tx=0x7ffb3c0046c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:10.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:10.252+0000 7ffb4bfcb740 1 -- start start 2024-09-10T10:01:10.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:10.252+0000 7ffb4bfcb740 1 Processor -- start 2024-09-10T10:01:10.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:10.253+0000 7ffb4bfcb740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56059a9e4520 0x56059a9ec600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:10.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:10.253+0000 7ffb4bfcb740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56059a9e4e40 0x56059a9ecb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:10.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:10.253+0000 7ffb4bfcb740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56059a9ef700 0x56059a9ed080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:10.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:10.253+0000 7ffb4bfcb740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x56059a9f22d0 con 0x56059a9e4e40 2024-09-10T10:01:10.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:10.253+0000 7ffb4bfcb740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x56059a905350 con 0x56059a9ef700 2024-09-10T10:01:10.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:10.253+0000 7ffb4bfcb740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x56059a9f2450 con 0x56059a9e4520 2024-09-10T10:01:10.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:10.253+0000 7ffb4bfcb740 10 client.0 _ll_register_callbacks cb 0x56059a928290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:01:10.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:10.253+0000 7ffb3a000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56059a9e4e40 0x56059a9ecb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:10.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:10.253+0000 7ffb43400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56059a9ef700 0x56059a9ed080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:10.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:10.253+0000 7ffb43400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56059a9ef700 0x56059a9ed080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:59190/0 (socket says 172.21.15.5:59190) 2024-09-10T10:01:10.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:10.253+0000 7ffb42a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56059a9e4520 0x56059a9ec600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:10.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:10.253+0000 7ffb43400640 1 -- 172.21.15.5:0/300923304 learned_addr learned my addr 172.21.15.5:0/300923304 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T10:01:10.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:10.253+0000 7ffb3a000640 1 -- 172.21.15.5:0/300923304 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56059a9e4520 msgr2=0x56059a9ec600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:01:10.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:10.253+0000 7ffb3a000640 1 --2- 172.21.15.5:0/300923304 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56059a9e4520 0x56059a9ec600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:10.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:10.254+0000 7ffb3a000640 1 -- 172.21.15.5:0/300923304 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56059a9ef700 msgr2=0x56059a9ed080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:01:10.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:10.254+0000 7ffb3a000640 1 --2- 172.21.15.5:0/300923304 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56059a9ef700 0x56059a9ed080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:10.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:10.254+0000 7ffb3a000640 1 -- 172.21.15.5:0/300923304 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56059a723960 con 0x56059a9e4e40 2024-09-10T10:01:10.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:10.254+0000 7ffb42a00640 1 --2- 172.21.15.5:0/300923304 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56059a9e4520 0x56059a9ec600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:01:10.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:10.254+0000 7ffb43400640 1 --2- 172.21.15.5:0/300923304 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56059a9ef700 0x56059a9ed080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:01:10.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:10.254+0000 7ffb3a000640 1 --2- 172.21.15.5:0/300923304 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56059a9e4e40 0x56059a9ecb40 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7ffb2c0ccd90 tx=0x7ffb2c09b5e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1c0a31a95290c64 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:10.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:10.254+0000 7ffb27e00640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-10T10:01:10.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:10.254+0000 7ffb27e00640 1 -- 172.21.15.5:0/300923304 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb2c09c1a0 con 0x56059a9e4e40 2024-09-10T10:01:10.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:10.254+0000 7ffb27e00640 1 -- 172.21.15.5:0/300923304 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7ffb2c0cf300 con 0x56059a9e4e40 2024-09-10T10:01:10.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:10.254+0000 7ffb4bfcb740 1 -- 172.21.15.5:0/300923304 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffb34003e50 con 0x56059a9e4e40 2024-09-10T10:01:10.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:10.254+0000 7ffb27e00640 1 -- 172.21.15.5:0/300923304 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7ffb2c09baa0 con 0x56059a9e4e40 2024-09-10T10:01:10.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:10.254+0000 7ffb4bfcb740 10 client.19641 fetch_fsmap learned FSMap version 42 2024-09-10T10:01:10.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:10.254+0000 7ffb4bfcb740 1 -- 172.21.15.5:0/300923304 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x56059a9fdf00 con 0x56059a9e4e40 2024-09-10T10:01:10.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:10.255+0000 7ffb27e00640 1 -- 172.21.15.5:0/300923304 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffb34003e50 con 0x56059a9e4e40 2024-09-10T10:01:10.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:10.255+0000 7ffb27e00640 1 -- 172.21.15.5:0/300923304 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 42) ==== 8370+0+0 (secure 0 0 0) 0x7ffb2c0ba2a0 con 0x56059a9e4e40 2024-09-10T10:01:10.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:10.255+0000 7ffb4bfcb740 10 client.19641 fetch_fsmap finished waiting for FSMap version 42 2024-09-10T10:01:10.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:10.255+0000 7ffb4bfcb740 10 client.19641 resolve_mds: validated gid 24542 aka daemon mds.b 2024-09-10T10:01:10.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:10.255+0000 7ffb4bfcb740 20 client.19641 populate_metadata read hostname 'smithi005' 2024-09-10T10:01:10.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:10.255+0000 7ffb4bfcb740 1 --2- 172.21.15.5:0/300923304 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x56059a9ff3b0 0x56059aa017a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:10.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:10.255+0000 7ffb4bfcb740 4 client.19641 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:01:10.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:10.255+0000 7ffb4bfcb740 1 -- 172.21.15.5:0/300923304 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56059a957bc0 con 0x56059a9ff3b0 2024-09-10T10:01:10.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:10.256+0000 7ffb42a00640 1 --2- 172.21.15.5:0/300923304 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x56059a9ff3b0 0x56059aa017a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:10.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:10.256+0000 7ffb42a00640 1 --2- 172.21.15.5:0/300923304 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x56059a9ff3b0 0x56059aa017a0 crc :-1 s=READY pgs=309 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6b4019404d74c2c1 server_cookie=46a40708db04b103 in_seq=0 out_seq=0 2024-09-10T10:01:10.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:10.256+0000 7ffb27e00640 10 client.19641 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T10:01:10.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:10.257+0000 7ffb27e00640 1 -- 172.21.15.5:0/300923304 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56059a957bc0 con 0x56059a9ff3b0 2024-09-10T10:01:10.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:10.257+0000 7ffb27e00640 10 client.19641 handle_command_reply: tid=0 2024-09-10T10:01:10.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:10.257+0000 7ffb4bfcb740 1 client.19641 shutdown 2024-09-10T10:01:10.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:10.257+0000 7ffb4bfcb740 1 -- 172.21.15.5:0/300923304 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56059a9e4e40 msgr2=0x56059a9ecb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:10.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:10.257+0000 7ffb4bfcb740 1 --2- 172.21.15.5:0/300923304 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56059a9e4e40 0x56059a9ecb40 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7ffb2c0ccd90 tx=0x7ffb2c09b5e0 comp rx=0 tx=0).stop 2024-09-10T10:01:10.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:10.257+0000 7ffb4bfcb740 1 -- 172.21.15.5:0/300923304 shutdown_connections 2024-09-10T10:01:10.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:10.257+0000 7ffb4bfcb740 1 --2- 172.21.15.5:0/300923304 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x56059a9ff3b0 0x56059aa017a0 crc :-1 s=READY pgs=309 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:10.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:10.257+0000 7ffb4bfcb740 1 --2- 172.21.15.5:0/300923304 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56059a9ef700 0x56059a9ed080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:10.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:10.257+0000 7ffb4bfcb740 1 --2- 172.21.15.5:0/300923304 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56059a9e4e40 0x56059a9ecb40 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:10.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:10.257+0000 7ffb4bfcb740 1 --2- 172.21.15.5:0/300923304 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56059a9e4520 0x56059a9ec600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:10.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:10.257+0000 7ffb27e00640 0 client.19641 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T10:01:10.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:10.257+0000 7ffb4bfcb740 1 -- 172.21.15.5:0/300923304 >> 172.21.15.5:0/300923304 conn(0x56059a9e0ab0 msgr2=0x56059a9dfcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:10.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:10.258+0000 7ffb4bfcb740 1 -- 172.21.15.5:0/300923304 shutdown_connections 2024-09-10T10:01:10.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:10.258+0000 7ffb4bfcb740 1 -- 172.21.15.5:0/300923304 wait complete. 2024-09-10T10:01:10.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:10.258+0000 7ffb4bfcb740 20 client.19641 trim_cache size 0 max 16384 2024-09-10T10:01:10.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:10.276+0000 7ffb4bfcb740 1 Processor -- start 2024-09-10T10:01:10.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:10.276+0000 7ffb4bfcb740 1 -- start start 2024-09-10T10:01:10.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:10.276+0000 7ffb4bfcb740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56059a9e4520 0x56059aa07070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:10.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:10.276+0000 7ffb4bfcb740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56059a9e4e40 0x56059aa075b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:10.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:10.276+0000 7ffb4bfcb740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56059a9ef700 0x56059a9e9860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:10.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:10.276+0000 7ffb4bfcb740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ffb441b0e40 con 0x56059a9ef700 2024-09-10T10:01:10.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:10.276+0000 7ffb4bfcb740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7ffb441b2980 con 0x56059a9e4e40 2024-09-10T10:01:10.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:10.276+0000 7ffb4bfcb740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7ffb441b1510 con 0x56059a9e4520 2024-09-10T10:01:10.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:10.277+0000 7ffb42a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56059a9e4520 0x56059aa07070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:10.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:10.277+0000 7ffb3a000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56059a9e4e40 0x56059aa075b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:10.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:10.277+0000 7ffb43400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56059a9ef700 0x56059a9e9860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:10.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:10.277+0000 7ffb3a000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56059a9e4e40 0x56059aa075b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:59192/0 (socket says 172.21.15.5:59192) 2024-09-10T10:01:10.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:10.277+0000 7ffb3a000640 1 -- 172.21.15.5:0/1744304629 learned_addr learned my addr 172.21.15.5:0/1744304629 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T10:01:10.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:10.277+0000 7ffb42a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56059a9e4520 0x56059aa07070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:46918/0 (socket says 172.21.15.5:46918) 2024-09-10T10:01:10.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:10.277+0000 7ffb3a000640 1 -- 172.21.15.5:0/1744304629 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56059a9e4520 msgr2=0x56059aa07070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:01:10.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:10.277+0000 7ffb3a000640 1 --2- 172.21.15.5:0/1744304629 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56059a9e4520 0x56059aa07070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:10.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:10.277+0000 7ffb3a000640 1 -- 172.21.15.5:0/1744304629 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56059a9ef700 msgr2=0x56059a9e9860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:01:10.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:10.277+0000 7ffb3a000640 1 --2- 172.21.15.5:0/1744304629 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56059a9ef700 0x56059a9e9860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:10.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:10.277+0000 7ffb3a000640 1 -- 172.21.15.5:0/1744304629 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56059a9e4c70 con 0x56059a9e4e40 2024-09-10T10:01:10.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:10.277+0000 7ffb3a000640 1 --2- 172.21.15.5:0/1744304629 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56059a9e4e40 0x56059aa075b0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7ffb2c097f00 tx=0x7ffb2c099fb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=693941e6b74b4b7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:10.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:10.277+0000 7ffb32a00640 1 -- 172.21.15.5:0/1744304629 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb2c09c1a0 con 0x56059a9e4e40 2024-09-10T10:01:10.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:10.278+0000 7ffb32a00640 1 -- 172.21.15.5:0/1744304629 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffb2c09fe50 con 0x56059a9e4e40 2024-09-10T10:01:10.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:10.278+0000 7ffb32a00640 1 -- 172.21.15.5:0/1744304629 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb2c09afc0 con 0x56059a9e4e40 2024-09-10T10:01:10.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:10.278+0000 7ffb4bfcb740 1 -- 172.21.15.5:0/1744304629 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56059a9e4e40 msgr2=0x56059aa075b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:10.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:10.278+0000 7ffb4bfcb740 1 --2- 172.21.15.5:0/1744304629 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56059a9e4e40 0x56059aa075b0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7ffb2c097f00 tx=0x7ffb2c099fb0 comp rx=0 tx=0).stop 2024-09-10T10:01:10.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:10.278+0000 7ffb4bfcb740 1 -- 172.21.15.5:0/1744304629 shutdown_connections 2024-09-10T10:01:10.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:10.278+0000 7ffb4bfcb740 1 --2- 172.21.15.5:0/1744304629 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56059a9ef700 0x56059a9e9860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:10.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:10.278+0000 7ffb4bfcb740 1 --2- 172.21.15.5:0/1744304629 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56059a9e4e40 0x56059aa075b0 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:10.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:10.278+0000 7ffb4bfcb740 1 --2- 172.21.15.5:0/1744304629 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56059a9e4520 0x56059aa07070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:10.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:10.278+0000 7ffb4bfcb740 1 -- 172.21.15.5:0/1744304629 >> 172.21.15.5:0/1744304629 conn(0x56059a9e0ab0 msgr2=0x56059a9dfcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:10.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:10.279+0000 7ffb4bfcb740 1 -- 172.21.15.5:0/1744304629 shutdown_connections 2024-09-10T10:01:10.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:10.279+0000 7ffb4bfcb740 1 -- 172.21.15.5:0/1744304629 wait complete. 2024-09-10T10:01:10.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:10.279+0000 7ffb4bfcb740 1 -- start start 2024-09-10T10:01:10.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:10.279+0000 7ffb4bfcb740 1 Processor -- start 2024-09-10T10:01:10.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:10.280+0000 7ffb4bfcb740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56059a9e4520 0x56059a9f7fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:10.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:10.280+0000 7ffb4bfcb740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56059a9e4e40 0x56059a9f33a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:10.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:10.280+0000 7ffb4bfcb740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56059a9ef700 0x56059a9f3b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:10.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:10.280+0000 7ffb4bfcb740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ffb441b22b0 con 0x56059a9e4520 2024-09-10T10:01:10.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:10.280+0000 7ffb4bfcb740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7ffb441b1be0 con 0x56059a9ef700 2024-09-10T10:01:10.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:10.280+0000 7ffb4bfcb740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7ffb441aee20 con 0x56059a9e4e40 2024-09-10T10:01:10.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:10.280+0000 7ffb4bfcb740 10 client.0 _ll_register_callbacks cb 0x7ffb2c097d40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:01:10.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:10.280+0000 7ffb42a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56059a9e4520 0x56059a9f7fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:10.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:10.280+0000 7ffb42a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56059a9e4520 0x56059a9f7fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60396/0 (socket says 172.21.15.5:60396) 2024-09-10T10:01:10.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:10.280+0000 7ffb42a00640 1 -- 172.21.15.5:0/1153519010 learned_addr learned my addr 172.21.15.5:0/1153519010 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T10:01:10.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:10.280+0000 7ffb3a000640 1 --2- 172.21.15.5:0/1153519010 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56059a9e4e40 0x56059a9f33a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:10.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:10.280+0000 7ffb43400640 1 --2- 172.21.15.5:0/1153519010 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56059a9ef700 0x56059a9f3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:10.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:10.280+0000 7ffb42a00640 1 -- 172.21.15.5:0/1153519010 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56059a9e4e40 msgr2=0x56059a9f33a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:01:10.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:10.280+0000 7ffb42a00640 1 --2- 172.21.15.5:0/1153519010 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56059a9e4e40 0x56059a9f33a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:10.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:10.280+0000 7ffb42a00640 1 -- 172.21.15.5:0/1153519010 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56059a9ef700 msgr2=0x56059a9f3b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:01:10.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:10.280+0000 7ffb42a00640 1 --2- 172.21.15.5:0/1153519010 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56059a9ef700 0x56059a9f3b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:10.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:10.280+0000 7ffb42a00640 1 -- 172.21.15.5:0/1153519010 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56059aa69040 con 0x56059a9e4520 2024-09-10T10:01:10.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:10.281+0000 7ffb42a00640 1 --2- 172.21.15.5:0/1153519010 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56059a9e4520 0x56059a9f7fd0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7ffb34011830 tx=0x7ffb34009ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b06426253c7ab7d1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:10.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:10.281+0000 7ffb27e00640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-10T10:01:10.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:10.281+0000 7ffb27e00640 1 -- 172.21.15.5:0/1153519010 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb34011400 con 0x56059a9e4520 2024-09-10T10:01:10.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:10.281+0000 7ffb4bfcb740 1 -- 172.21.15.5:0/1153519010 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffb2c09fe50 con 0x56059a9e4520 2024-09-10T10:01:10.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:10.281+0000 7ffb27e00640 1 -- 172.21.15.5:0/1153519010 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7ffb34023040 con 0x56059a9e4520 2024-09-10T10:01:10.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:10.281+0000 7ffb27e00640 1 -- 172.21.15.5:0/1153519010 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7ffb3403a050 con 0x56059a9e4520 2024-09-10T10:01:10.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:10.281+0000 7ffb4bfcb740 10 client.19647 fetch_fsmap learned FSMap version 42 2024-09-10T10:01:10.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:10.281+0000 7ffb4bfcb740 1 -- 172.21.15.5:0/1153519010 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x56059aa6a310 con 0x56059a9e4520 2024-09-10T10:01:10.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:10.282+0000 7ffb27e00640 1 -- 172.21.15.5:0/1153519010 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffb2c09fe50 con 0x56059a9e4520 2024-09-10T10:01:10.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:10.282+0000 7ffb27e00640 1 -- 172.21.15.5:0/1153519010 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 42) ==== 8370+0+0 (secure 0 0 0) 0x7ffb34005260 con 0x56059a9e4520 2024-09-10T10:01:10.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:10.282+0000 7ffb4bfcb740 10 client.19647 fetch_fsmap finished waiting for FSMap version 42 2024-09-10T10:01:10.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:10.282+0000 7ffb4bfcb740 10 client.19647 resolve_mds: validated gid 24542 aka daemon mds.b 2024-09-10T10:01:10.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:10.282+0000 7ffb4bfcb740 20 client.19647 populate_metadata read hostname 'smithi005' 2024-09-10T10:01:10.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:10.282+0000 7ffb4bfcb740 1 --2- 172.21.15.5:0/1153519010 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x56059aa6a8a0 0x56059aa6cc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:10.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:10.282+0000 7ffb4bfcb740 4 client.19647 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "8aeff82a", "release": true, "await": true}] 2024-09-10T10:01:10.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:10.282+0000 7ffb4bfcb740 1 -- 172.21.15.5:0/1153519010 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "quiesce db", "set_id": "8aeff82a", "release": true, "await": true}) -- 0x56059a9520d0 con 0x56059aa6a8a0 2024-09-10T10:01:10.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:10.282+0000 7ffb3a000640 1 --2- 172.21.15.5:0/1153519010 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x56059aa6a8a0 0x56059aa6cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:10.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:10.283+0000 7ffb3a000640 1 --2- 172.21.15.5:0/1153519010 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x56059aa6a8a0 0x56059aa6cc90 crc :-1 s=READY pgs=310 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ed98c0bfd8cdbd5a server_cookie=28f928a2d7d193cc in_seq=0 out_seq=0 2024-09-10T10:01:10.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:10.283+0000 7ffb27e00640 10 client.19647 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T10:01:10.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:10.332+0000 7ffb27e00640 1 -- 172.21.15.5:0/1153519010 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x56059a9520d0 con 0x56059aa6a8a0 2024-09-10T10:01:10.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:10.332+0000 7ffb27e00640 10 client.19647 handle_command_reply: tid=0 2024-09-10T10:01:10.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:10.332+0000 7ffb4bfcb740 1 client.19647 shutdown 2024-09-10T10:01:10.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:10.333+0000 7ffb4bfcb740 1 -- 172.21.15.5:0/1153519010 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56059a9e4520 msgr2=0x56059a9f7fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:10.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:10.333+0000 7ffb4bfcb740 1 --2- 172.21.15.5:0/1153519010 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56059a9e4520 0x56059a9f7fd0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7ffb34011830 tx=0x7ffb34009ed0 comp rx=0 tx=0).stop 2024-09-10T10:01:10.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:10.333+0000 7ffb4bfcb740 1 -- 172.21.15.5:0/1153519010 shutdown_connections 2024-09-10T10:01:10.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:10.333+0000 7ffb4bfcb740 1 --2- 172.21.15.5:0/1153519010 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x56059aa6a8a0 0x56059aa6cc90 crc :-1 s=READY pgs=310 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:10.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:10.333+0000 7ffb4bfcb740 1 --2- 172.21.15.5:0/1153519010 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56059a9ef700 0x56059a9f3b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:10.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:10.333+0000 7ffb4bfcb740 1 --2- 172.21.15.5:0/1153519010 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56059a9e4e40 0x56059a9f33a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:10.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:10.333+0000 7ffb4bfcb740 1 --2- 172.21.15.5:0/1153519010 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56059a9e4520 0x56059a9f7fd0 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:10.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:10.333+0000 7ffb27e00640 0 client.19647 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T10:01:10.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:10.333+0000 7ffb4bfcb740 1 -- 172.21.15.5:0/1153519010 >> 172.21.15.5:0/1153519010 conn(0x56059a9e0ab0 msgr2=0x56059a9e5f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:10.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:10.333+0000 7ffb4bfcb740 1 -- 172.21.15.5:0/1153519010 shutdown_connections 2024-09-10T10:01:10.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:10.333+0000 7ffb4bfcb740 1 -- 172.21.15.5:0/1153519010 wait complete. 2024-09-10T10:01:10.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:10.334+0000 7ffb4bfcb740 20 client.19647 trim_cache size 0 max 16384 2024-09-10T10:01:10.337 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-10T10:01:10.337 INFO:teuthology.orchestra.run.smithi005.stdout: "epoch": 42, 2024-09-10T10:01:10.337 INFO:teuthology.orchestra.run.smithi005.stdout: "leader": 24542, 2024-09-10T10:01:10.337 INFO:teuthology.orchestra.run.smithi005.stdout: "set_version": 80, 2024-09-10T10:01:10.337 INFO:teuthology.orchestra.run.smithi005.stdout: "sets": { 2024-09-10T10:01:10.337 INFO:teuthology.orchestra.run.smithi005.stdout: "8aeff82a": { 2024-09-10T10:01:10.337 INFO:teuthology.orchestra.run.smithi005.stdout: "version": 80, 2024-09-10T10:01:10.338 INFO:teuthology.orchestra.run.smithi005.stdout: "age_ref": 0.0, 2024-09-10T10:01:10.338 INFO:teuthology.orchestra.run.smithi005.stdout: "state": { 2024-09-10T10:01:10.338 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "RELEASED", 2024-09-10T10:01:10.338 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0 2024-09-10T10:01:10.338 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:01:10.338 INFO:teuthology.orchestra.run.smithi005.stdout: "timeout": 90.0, 2024-09-10T10:01:10.338 INFO:teuthology.orchestra.run.smithi005.stdout: "expiration": 154.2, 2024-09-10T10:01:10.338 INFO:teuthology.orchestra.run.smithi005.stdout: "members": { 2024-09-10T10:01:10.338 INFO:teuthology.orchestra.run.smithi005.stdout: "file:/": { 2024-09-10T10:01:10.338 INFO:teuthology.orchestra.run.smithi005.stdout: "excluded": false, 2024-09-10T10:01:10.338 INFO:teuthology.orchestra.run.smithi005.stdout: "state": { 2024-09-10T10:01:10.338 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "RELEASED", 2024-09-10T10:01:10.338 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0 2024-09-10T10:01:10.339 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T10:01:10.339 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T10:01:10.339 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T10:01:10.339 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T10:01:10.339 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T10:01:10.339 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-10T10:01:10.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:10.334+0000 7ffb43e00640 1 -- 172.21.15.5:0/1419142458 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7ffb280a2c00 msgr2=0x7ffb280a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:10.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:10.334+0000 7ffb43e00640 1 --2- 172.21.15.5:0/1419142458 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7ffb280a2c00 0x7ffb280a50c0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7ffb3c0096c0 tx=0x7ffb3c0046c0 comp rx=0 tx=0).stop 2024-09-10T10:01:10.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:10.334+0000 7ffb43e00640 1 -- 172.21.15.5:0/1419142458 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ffb441047e0 msgr2=0x7ffb4419a910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:10.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:10.334+0000 7ffb43e00640 1 --2- 172.21.15.5:0/1419142458 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ffb441047e0 0x7ffb4419a910 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7ffb2c00e9f0 tx=0x7ffb2c00eeb0 comp rx=0 tx=0).stop 2024-09-10T10:01:10.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:10.334+0000 7ffb43e00640 1 -- 172.21.15.5:0/1419142458 shutdown_connections 2024-09-10T10:01:10.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:10.334+0000 7ffb43e00640 1 --2- 172.21.15.5:0/1419142458 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7ffb280a2c00 0x7ffb280a50c0 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:10.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:10.334+0000 7ffb43e00640 1 --2- 172.21.15.5:0/1419142458 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffb44108f60 0x7ffb4419eca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:10.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:10.334+0000 7ffb43e00640 1 --2- 172.21.15.5:0/1419142458 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ffb441047e0 0x7ffb4419a910 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:10.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:10.334+0000 7ffb43e00640 1 --2- 172.21.15.5:0/1419142458 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ffb44103e30 0x7ffb4419a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:10.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:10.334+0000 7ffb43e00640 1 -- 172.21.15.5:0/1419142458 >> 172.21.15.5:0/1419142458 conn(0x7ffb440ff8d0 msgr2=0x7ffb44100330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:10.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:10.335+0000 7ffb43e00640 1 -- 172.21.15.5:0/1419142458 shutdown_connections 2024-09-10T10:01:10.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:10.335+0000 7ffb43e00640 1 -- 172.21.15.5:0/1419142458 wait complete. 2024-09-10T10:01:11.039 INFO:tasks.workunit.client.0.smithi005.stdout: 10 134974 12.64 MB/sec execute 409 sec latency 31863.395 ms 2024-09-10T10:01:11.356 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '8aeff82a', total seconds elapsed: 39.2 2024-09-10T10:01:11.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:01:11 smithi005 ceph-mon[30350]: pgmap v1078: 129 pgs: 129 active+clean; 688 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 3.3 MiB/s wr, 100 op/s 2024-09-10T10:01:11.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:01:11 smithi159 ceph-mon[35513]: pgmap v1078: 129 pgs: 129 active+clean; 688 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 3.3 MiB/s wr, 100 op/s 2024-09-10T10:01:11.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:01:11 smithi102 ceph-mon[35442]: pgmap v1078: 129 pgs: 129 active+clean; 688 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 3.3 MiB/s wr, 100 op/s 2024-09-10T10:01:12.039 INFO:tasks.workunit.client.0.smithi005.stdout: 10 135475 12.68 MB/sec execute 410 sec latency 88.630 ms 2024-09-10T10:01:13.040 INFO:tasks.workunit.client.0.smithi005.stdout: 10 136009 12.70 MB/sec execute 411 sec latency 149.422 ms 2024-09-10T10:01:13.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:01:13 smithi005 ceph-mon[30350]: pgmap v1079: 129 pgs: 129 active+clean; 689 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 851 KiB/s rd, 3.5 MiB/s wr, 107 op/s 2024-09-10T10:01:13.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:01:13 smithi159 ceph-mon[35513]: pgmap v1079: 129 pgs: 129 active+clean; 689 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 851 KiB/s rd, 3.5 MiB/s wr, 107 op/s 2024-09-10T10:01:13.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:01:13 smithi102 ceph-mon[35442]: pgmap v1079: 129 pgs: 129 active+clean; 689 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 851 KiB/s rd, 3.5 MiB/s wr, 107 op/s 2024-09-10T10:01:13.948 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T10:01:14.039 INFO:tasks.workunit.client.0.smithi005.stdout: 10 136635 12.75 MB/sec execute 412 sec latency 65.030 ms 2024-09-10T10:01:14.441 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:01:15.040 INFO:tasks.workunit.client.0.smithi005.stdout: 10 137443 12.85 MB/sec execute 413 sec latency 130.011 ms 2024-09-10T10:01:15.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:15.129+0000 7f4bb9800640 1 -- 172.21.15.5:0/1528036394 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4bb4074f40 msgr2=0x7f4bb410de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:15.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:15.129+0000 7f4bb9800640 1 --2- 172.21.15.5:0/1528036394 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4bb4074f40 0x7f4bb410de80 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f4ba0009900 tx=0x7f4ba002a480 comp rx=0 tx=0).stop 2024-09-10T10:01:15.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:15.129+0000 7f4bb9800640 1 -- 172.21.15.5:0/1528036394 shutdown_connections 2024-09-10T10:01:15.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:15.129+0000 7f4bb9800640 1 --2- 172.21.15.5:0/1528036394 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4bb410e3c0 0x7f4bb4110810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:15.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:15.129+0000 7f4bb9800640 1 --2- 172.21.15.5:0/1528036394 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4bb4074f40 0x7f4bb410de80 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:15.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:15.129+0000 7f4bb9800640 1 --2- 172.21.15.5:0/1528036394 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4bb4074620 0x7f4bb4074a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:15.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:15.129+0000 7f4bb9800640 1 -- 172.21.15.5:0/1528036394 >> 172.21.15.5:0/1528036394 conn(0x7f4bb40fd8c0 msgr2=0x7f4bb40ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:15.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:15.129+0000 7f4bb9800640 1 -- 172.21.15.5:0/1528036394 shutdown_connections 2024-09-10T10:01:15.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:15.129+0000 7f4bb9800640 1 -- 172.21.15.5:0/1528036394 wait complete. 2024-09-10T10:01:15.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:15.130+0000 7f4bb9800640 1 Processor -- start 2024-09-10T10:01:15.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:15.130+0000 7f4bb9800640 1 -- start start 2024-09-10T10:01:15.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:15.131+0000 7f4bb9800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4bb4074620 0x7f4bb419a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:15.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:15.131+0000 7f4bb9800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4bb4074f40 0x7f4bb419ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:15.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:15.131+0000 7f4bb9800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4bb410e3c0 0x7f4bb419ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:15.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:15.131+0000 7f4bb9800640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f4bb4114190 con 0x7f4bb4074f40 2024-09-10T10:01:15.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:15.131+0000 7f4bb9800640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f4bb4114010 con 0x7f4bb4074620 2024-09-10T10:01:15.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:15.131+0000 7f4bb9800640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f4bb4114310 con 0x7f4bb410e3c0 2024-09-10T10:01:15.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:15.131+0000 7f4bb2a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4bb4074620 0x7f4bb419a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:15.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:15.131+0000 7f4bb2a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4bb4074620 0x7f4bb419a650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:51508/0 (socket says 172.21.15.5:51508) 2024-09-10T10:01:15.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:15.131+0000 7f4bb2a00640 1 -- 172.21.15.5:0/3226482932 learned_addr learned my addr 172.21.15.5:0/3226482932 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T10:01:15.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:15.131+0000 7f4bb2000640 1 --2- 172.21.15.5:0/3226482932 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4bb4074f40 0x7f4bb419ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:15.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:15.132+0000 7f4bb2a00640 1 -- 172.21.15.5:0/3226482932 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4bb410e3c0 msgr2=0x7f4bb419ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:15.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:15.132+0000 7f4bb2a00640 1 --2- 172.21.15.5:0/3226482932 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4bb410e3c0 0x7f4bb419ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:15.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:15.132+0000 7f4bb2a00640 1 -- 172.21.15.5:0/3226482932 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4bb4074f40 msgr2=0x7f4bb419ab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:15.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:15.132+0000 7f4bb2a00640 1 --2- 172.21.15.5:0/3226482932 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4bb4074f40 0x7f4bb419ab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:15.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:15.132+0000 7f4bb2a00640 1 -- 172.21.15.5:0/3226482932 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4bb419f600 con 0x7f4bb4074620 2024-09-10T10:01:15.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:15.132+0000 7f4bb2000640 1 --2- 172.21.15.5:0/3226482932 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4bb4074f40 0x7f4bb419ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:01:15.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:15.133+0000 7f4bb2a00640 1 --2- 172.21.15.5:0/3226482932 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4bb4074620 0x7f4bb419a650 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f4ba400d950 tx=0x7f4ba400de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:15.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:15.133+0000 7f4bab400640 1 -- 172.21.15.5:0/3226482932 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ba4017030 con 0x7f4bb4074620 2024-09-10T10:01:15.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:15.133+0000 7f4bab400640 1 -- 172.21.15.5:0/3226482932 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4ba4004020 con 0x7f4bb4074620 2024-09-10T10:01:15.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:15.133+0000 7f4bab400640 1 -- 172.21.15.5:0/3226482932 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ba40047a0 con 0x7f4bb4074620 2024-09-10T10:01:15.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:15.133+0000 7f4bb9800640 1 -- 172.21.15.5:0/3226482932 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4bb419f8f0 con 0x7f4bb4074620 2024-09-10T10:01:15.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:15.133+0000 7f4bb9800640 1 -- 172.21.15.5:0/3226482932 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4bb41a71d0 con 0x7f4bb4074620 2024-09-10T10:01:15.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:15.137+0000 7f4bab400640 1 -- 172.21.15.5:0/3226482932 <== mon.2 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f4ba40041d0 con 0x7f4bb4074620 2024-09-10T10:01:15.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:15.137+0000 7f4bbc43f740 1 Processor -- start 2024-09-10T10:01:15.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:15.137+0000 7f4bbc43f740 1 -- start start 2024-09-10T10:01:15.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:15.138+0000 7f4bbc43f740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dbcc130b90 0x55dbcc130f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:15.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:15.138+0000 7f4bbc43f740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dbcc1314b0 0x55dbcc13b830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:15.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:15.138+0000 7f4bbc43f740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dbcc13bd70 0x55dbcc13e160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:15.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:15.138+0000 7f4bbc43f740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55dbcc0973e0 con 0x55dbcc130b90 2024-09-10T10:01:15.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:15.138+0000 7f4bbc43f740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55dbcc03ed80 con 0x55dbcc13bd70 2024-09-10T10:01:15.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:15.138+0000 7f4bbc43f740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55dbcc0933a0 con 0x55dbcc1314b0 2024-09-10T10:01:15.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:15.138+0000 7f4bb2a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dbcc1314b0 0x55dbcc13b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:15.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:15.138+0000 7f4bb2a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dbcc1314b0 0x55dbcc13b830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:41634/0 (socket says 172.21.15.5:41634) 2024-09-10T10:01:15.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:15.138+0000 7f4bb3400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dbcc130b90 0x55dbcc130f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:15.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:15.138+0000 7f4bb2a00640 1 -- 172.21.15.5:0/3364653674 learned_addr learned my addr 172.21.15.5:0/3364653674 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T10:01:15.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:15.138+0000 7f4bb2000640 1 --2- 172.21.15.5:0/3364653674 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dbcc13bd70 0x55dbcc13e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:15.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:15.139+0000 7f4bb2a00640 1 -- 172.21.15.5:0/3364653674 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dbcc13bd70 msgr2=0x55dbcc13e160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:01:15.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:15.139+0000 7f4bb2a00640 1 --2- 172.21.15.5:0/3364653674 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dbcc13bd70 0x55dbcc13e160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:15.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:15.139+0000 7f4bab400640 1 --2- 172.21.15.5:0/3226482932 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f4b980a2c00 0x7f4b980a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:15.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:15.139+0000 7f4bb2a00640 1 -- 172.21.15.5:0/3364653674 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dbcc130b90 msgr2=0x55dbcc130f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:01:15.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:15.139+0000 7f4bb2a00640 1 --2- 172.21.15.5:0/3364653674 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dbcc130b90 0x55dbcc130f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:15.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:15.139+0000 7f4bb2a00640 1 -- 172.21.15.5:0/3364653674 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dbcc05dae0 con 0x55dbcc1314b0 2024-09-10T10:01:15.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:15.139+0000 7f4bab400640 1 -- 172.21.15.5:0/3226482932 <== mon.2 v2:172.21.15.102:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f4ba40cee00 con 0x7f4bb4074620 2024-09-10T10:01:15.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:15.139+0000 7f4bb2a00640 1 --2- 172.21.15.5:0/3364653674 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dbcc1314b0 0x55dbcc13b830 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f4ba409e550 tx=0x7f4ba40d57d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2bf88020df669cf1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:15.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:15.140+0000 7f4b9fe00640 1 -- 172.21.15.5:0/3364653674 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ba409afe0 con 0x55dbcc1314b0 2024-09-10T10:01:15.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:15.140+0000 7f4b9fe00640 1 -- 172.21.15.5:0/3364653674 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4ba40b9e50 con 0x55dbcc1314b0 2024-09-10T10:01:15.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:15.140+0000 7f4b9fe00640 1 -- 172.21.15.5:0/3364653674 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ba409e5e0 con 0x55dbcc1314b0 2024-09-10T10:01:15.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:15.140+0000 7f4bb3400640 1 --2- 172.21.15.5:0/3226482932 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f4b980a2c00 0x7f4b980a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:15.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:15.141+0000 7f4bbc43f740 1 -- 172.21.15.5:0/3364653674 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dbcc1314b0 msgr2=0x55dbcc13b830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:15.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:15.141+0000 7f4bb3400640 1 --2- 172.21.15.5:0/3226482932 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f4b980a2c00 0x7f4b980a50c0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f4bac005fd0 tx=0x7f4bac004a40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:15.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:15.141+0000 7f4bbc43f740 1 --2- 172.21.15.5:0/3364653674 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dbcc1314b0 0x55dbcc13b830 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f4ba409e550 tx=0x7f4ba40d57d0 comp rx=0 tx=0).stop 2024-09-10T10:01:15.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:15.141+0000 7f4bbc43f740 1 -- 172.21.15.5:0/3364653674 shutdown_connections 2024-09-10T10:01:15.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:15.141+0000 7f4bbc43f740 1 --2- 172.21.15.5:0/3364653674 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dbcc13bd70 0x55dbcc13e160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:15.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:15.141+0000 7f4bbc43f740 1 --2- 172.21.15.5:0/3364653674 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dbcc1314b0 0x55dbcc13b830 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:15.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:15.141+0000 7f4bbc43f740 1 --2- 172.21.15.5:0/3364653674 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dbcc130b90 0x55dbcc130f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:15.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:15.141+0000 7f4bbc43f740 1 -- 172.21.15.5:0/3364653674 >> 172.21.15.5:0/3364653674 conn(0x55dbcc12d120 msgr2=0x55dbcc12d500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:15.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:15.141+0000 7f4bbc43f740 1 -- 172.21.15.5:0/3364653674 shutdown_connections 2024-09-10T10:01:15.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:15.141+0000 7f4bbc43f740 1 -- 172.21.15.5:0/3364653674 wait complete. 2024-09-10T10:01:15.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:15.142+0000 7f4bbc43f740 1 -- start start 2024-09-10T10:01:15.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:15.142+0000 7f4bbc43f740 1 Processor -- start 2024-09-10T10:01:15.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:15.143+0000 7f4bbc43f740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dbcc130b90 0x55dbcc138ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:15.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:15.143+0000 7f4bbc43f740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dbcc1314b0 0x55dbcc139220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:15.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:15.143+0000 7f4bbc43f740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dbcc13bd70 0x55dbcc139760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:15.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:15.143+0000 7f4bbc43f740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55dbcc13e9e0 con 0x55dbcc1314b0 2024-09-10T10:01:15.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:15.143+0000 7f4bbc43f740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55dbcc0520f0 con 0x55dbcc13bd70 2024-09-10T10:01:15.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:15.143+0000 7f4bbc43f740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55dbcc13eb60 con 0x55dbcc130b90 2024-09-10T10:01:15.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:15.143+0000 7f4bbc43f740 10 client.0 _ll_register_callbacks cb 0x55dbcc0439e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:01:15.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:15.143+0000 7f4bb2a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dbcc130b90 0x55dbcc138ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:15.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:15.143+0000 7f4bb2a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dbcc130b90 0x55dbcc138ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:41636/0 (socket says 172.21.15.5:41636) 2024-09-10T10:01:15.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:15.143+0000 7f4bb2000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dbcc1314b0 0x55dbcc139220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:15.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:15.143+0000 7f4bb2a00640 1 -- 172.21.15.5:0/2899550526 learned_addr learned my addr 172.21.15.5:0/2899550526 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T10:01:15.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:15.143+0000 7f4bb3400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dbcc13bd70 0x55dbcc139760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:15.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:15.143+0000 7f4bb2000640 1 -- 172.21.15.5:0/2899550526 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dbcc130b90 msgr2=0x55dbcc138ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:01:15.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:15.143+0000 7f4bb2000640 1 --2- 172.21.15.5:0/2899550526 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dbcc130b90 0x55dbcc138ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:15.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:15.143+0000 7f4bb2000640 1 -- 172.21.15.5:0/2899550526 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dbcc13bd70 msgr2=0x55dbcc139760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:01:15.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:15.143+0000 7f4bb2000640 1 --2- 172.21.15.5:0/2899550526 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dbcc13bd70 0x55dbcc139760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:15.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:15.143+0000 7f4bb2000640 1 -- 172.21.15.5:0/2899550526 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dbcbe083c0 con 0x55dbcc1314b0 2024-09-10T10:01:15.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:15.143+0000 7f4bb3400640 1 --2- 172.21.15.5:0/2899550526 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dbcc13bd70 0x55dbcc139760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:01:15.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:15.144+0000 7f4bb2000640 1 --2- 172.21.15.5:0/2899550526 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dbcc1314b0 0x55dbcc139220 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f4ba002aa70 tx=0x7f4ba0005270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22fdfef716e4bf97 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:15.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:15.144+0000 7f4b9cc00640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-10T10:01:15.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:15.144+0000 7f4b9cc00640 1 -- 172.21.15.5:0/2899550526 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ba0035040 con 0x55dbcc1314b0 2024-09-10T10:01:15.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:15.144+0000 7f4b9cc00640 1 -- 172.21.15.5:0/2899550526 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f4ba0004760 con 0x55dbcc1314b0 2024-09-10T10:01:15.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:15.144+0000 7f4bbc43f740 1 -- 172.21.15.5:0/2899550526 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4ba40b9e50 con 0x55dbcc1314b0 2024-09-10T10:01:15.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:15.145+0000 7f4b9cc00640 1 -- 172.21.15.5:0/2899550526 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f4ba0004960 con 0x55dbcc1314b0 2024-09-10T10:01:15.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:15.145+0000 7f4bbc43f740 10 client.19659 fetch_fsmap learned FSMap version 42 2024-09-10T10:01:15.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:15.145+0000 7f4bbc43f740 1 -- 172.21.15.5:0/2899550526 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55dbcc14a710 con 0x55dbcc1314b0 2024-09-10T10:01:15.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:15.145+0000 7f4b9cc00640 1 -- 172.21.15.5:0/2899550526 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4ba40b9e50 con 0x55dbcc1314b0 2024-09-10T10:01:15.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:15.145+0000 7f4b9cc00640 1 -- 172.21.15.5:0/2899550526 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 42) ==== 8370+0+0 (secure 0 0 0) 0x7f4ba0044020 con 0x55dbcc1314b0 2024-09-10T10:01:15.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:15.145+0000 7f4bbc43f740 10 client.19659 fetch_fsmap finished waiting for FSMap version 42 2024-09-10T10:01:15.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:15.145+0000 7f4bbc43f740 10 client.19659 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T10:01:15.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:15.145+0000 7f4bbc43f740 20 client.19659 populate_metadata read hostname 'smithi005' 2024-09-10T10:01:15.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:15.145+0000 7f4bbc43f740 1 --2- 172.21.15.5:0/2899550526 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55dbcc14bbd0 0x55dbcc14dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:15.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:15.145+0000 7f4bbc43f740 4 client.19659 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:01:15.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:15.145+0000 7f4bbc43f740 1 -- 172.21.15.5:0/2899550526 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dbcc0a4950 con 0x55dbcc14bbd0 2024-09-10T10:01:15.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:15.146+0000 7f4bb2a00640 1 --2- 172.21.15.5:0/2899550526 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55dbcc14bbd0 0x55dbcc14dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:15.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:15.146+0000 7f4bb2a00640 1 --2- 172.21.15.5:0/2899550526 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55dbcc14bbd0 0x55dbcc14dfc0 crc :-1 s=READY pgs=311 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3edb5d386c1d9f9b server_cookie=35b34b3bc5542dba in_seq=0 out_seq=0 2024-09-10T10:01:15.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:15.146+0000 7f4b9cc00640 10 client.19659 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T10:01:15.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:15.147+0000 7f4b9cc00640 1 -- 172.21.15.5:0/2899550526 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55dbcc0a4950 con 0x55dbcc14bbd0 2024-09-10T10:01:15.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:15.147+0000 7f4b9cc00640 10 client.19659 handle_command_reply: tid=0 2024-09-10T10:01:15.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:15.147+0000 7f4bbc43f740 1 client.19659 shutdown 2024-09-10T10:01:15.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:15.147+0000 7f4bbc43f740 1 -- 172.21.15.5:0/2899550526 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dbcc1314b0 msgr2=0x55dbcc139220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:15.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:15.147+0000 7f4bbc43f740 1 --2- 172.21.15.5:0/2899550526 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dbcc1314b0 0x55dbcc139220 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f4ba002aa70 tx=0x7f4ba0005270 comp rx=0 tx=0).stop 2024-09-10T10:01:15.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:15.147+0000 7f4bbc43f740 1 -- 172.21.15.5:0/2899550526 shutdown_connections 2024-09-10T10:01:15.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:15.147+0000 7f4bbc43f740 1 --2- 172.21.15.5:0/2899550526 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55dbcc14bbd0 0x55dbcc14dfc0 crc :-1 s=READY pgs=311 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:15.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:15.147+0000 7f4bbc43f740 1 --2- 172.21.15.5:0/2899550526 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dbcc13bd70 0x55dbcc139760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:15.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:15.147+0000 7f4bbc43f740 1 --2- 172.21.15.5:0/2899550526 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dbcc1314b0 0x55dbcc139220 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:15.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:15.147+0000 7f4bbc43f740 1 --2- 172.21.15.5:0/2899550526 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dbcc130b90 0x55dbcc138ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:15.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:15.147+0000 7f4bbc43f740 1 -- 172.21.15.5:0/2899550526 >> 172.21.15.5:0/2899550526 conn(0x55dbcc12d120 msgr2=0x55dbcc132e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:15.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:15.148+0000 7f4b9cc00640 0 client.19659 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T10:01:15.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:15.148+0000 7f4bbc43f740 1 -- 172.21.15.5:0/2899550526 shutdown_connections 2024-09-10T10:01:15.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:15.148+0000 7f4bbc43f740 1 -- 172.21.15.5:0/2899550526 wait complete. 2024-09-10T10:01:15.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:15.148+0000 7f4bbc43f740 20 client.19659 trim_cache size 0 max 16384 2024-09-10T10:01:15.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:15.177+0000 7f4bbc43f740 1 Processor -- start 2024-09-10T10:01:15.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:15.177+0000 7f4bbc43f740 1 -- start start 2024-09-10T10:01:15.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:15.178+0000 7f4bbc43f740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dbcc130b90 0x55dbcc1375b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:15.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:15.178+0000 7f4bbc43f740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dbcc1314b0 0x55dbcc137af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:15.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:15.178+0000 7f4bbc43f740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dbcc13bd70 0x55dbcc153870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:15.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:15.178+0000 7f4bbc43f740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f4bb41b10b0 con 0x55dbcc1314b0 2024-09-10T10:01:15.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:15.178+0000 7f4bbc43f740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f4bb41b2bf0 con 0x55dbcc130b90 2024-09-10T10:01:15.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:15.178+0000 7f4bbc43f740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f4bb41b1780 con 0x55dbcc13bd70 2024-09-10T10:01:15.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:15.178+0000 7f4bb3400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dbcc13bd70 0x55dbcc153870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:15.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:15.178+0000 7f4bb2000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dbcc1314b0 0x55dbcc137af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:15.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:15.178+0000 7f4bb3400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dbcc13bd70 0x55dbcc153870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:41642/0 (socket says 172.21.15.5:41642) 2024-09-10T10:01:15.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:15.178+0000 7f4bb3400640 1 -- 172.21.15.5:0/230531354 learned_addr learned my addr 172.21.15.5:0/230531354 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T10:01:15.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:15.178+0000 7f4bb2a00640 1 --2- 172.21.15.5:0/230531354 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dbcc130b90 0x55dbcc1375b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:15.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:15.178+0000 7f4bb3400640 1 -- 172.21.15.5:0/230531354 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dbcc130b90 msgr2=0x55dbcc1375b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:01:15.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:15.178+0000 7f4bb3400640 1 --2- 172.21.15.5:0/230531354 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dbcc130b90 0x55dbcc1375b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:15.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:15.178+0000 7f4bb3400640 1 -- 172.21.15.5:0/230531354 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dbcc1314b0 msgr2=0x55dbcc137af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:01:15.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:15.179+0000 7f4bb3400640 1 --2- 172.21.15.5:0/230531354 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dbcc1314b0 0x55dbcc137af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:15.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:15.179+0000 7f4bb3400640 1 -- 172.21.15.5:0/230531354 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dbcc148a30 con 0x55dbcc13bd70 2024-09-10T10:01:15.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:15.179+0000 7f4bb2000640 1 --2- 172.21.15.5:0/230531354 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dbcc1314b0 0x55dbcc137af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:01:15.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:15.179+0000 7f4bb3400640 1 --2- 172.21.15.5:0/230531354 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dbcc13bd70 0x55dbcc153870 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f4bac01f6f0 tx=0x7f4bac015000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c92cf40ea7e55689 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:15.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:15.179+0000 7f4b9f400640 1 -- 172.21.15.5:0/230531354 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bac012670 con 0x55dbcc13bd70 2024-09-10T10:01:15.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:15.179+0000 7f4b9f400640 1 -- 172.21.15.5:0/230531354 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4bac012cf0 con 0x55dbcc13bd70 2024-09-10T10:01:15.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:15.179+0000 7f4b9f400640 1 -- 172.21.15.5:0/230531354 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bac01ed80 con 0x55dbcc13bd70 2024-09-10T10:01:15.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:15.180+0000 7f4bbc43f740 1 -- 172.21.15.5:0/230531354 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dbcc13bd70 msgr2=0x55dbcc153870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:15.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:15.180+0000 7f4bbc43f740 1 --2- 172.21.15.5:0/230531354 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dbcc13bd70 0x55dbcc153870 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f4bac01f6f0 tx=0x7f4bac015000 comp rx=0 tx=0).stop 2024-09-10T10:01:15.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:15.181+0000 7f4bbc43f740 1 -- 172.21.15.5:0/230531354 shutdown_connections 2024-09-10T10:01:15.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:15.181+0000 7f4bbc43f740 1 --2- 172.21.15.5:0/230531354 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dbcc13bd70 0x55dbcc153870 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:15.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:15.181+0000 7f4bbc43f740 1 --2- 172.21.15.5:0/230531354 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dbcc1314b0 0x55dbcc137af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:15.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:15.181+0000 7f4bbc43f740 1 --2- 172.21.15.5:0/230531354 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dbcc130b90 0x55dbcc1375b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:15.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:15.181+0000 7f4bbc43f740 1 -- 172.21.15.5:0/230531354 >> 172.21.15.5:0/230531354 conn(0x55dbcc12d120 msgr2=0x55dbcc133fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:15.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:15.181+0000 7f4bbc43f740 1 -- 172.21.15.5:0/230531354 shutdown_connections 2024-09-10T10:01:15.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:15.181+0000 7f4bbc43f740 1 -- 172.21.15.5:0/230531354 wait complete. 2024-09-10T10:01:15.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:15.181+0000 7f4bbc43f740 1 -- start start 2024-09-10T10:01:15.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:15.182+0000 7f4bbc43f740 1 Processor -- start 2024-09-10T10:01:15.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:15.182+0000 7f4bbc43f740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dbcc130b90 0x55dbcc1aeb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:15.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:15.182+0000 7f4bbc43f740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dbcc1314b0 0x55dbcc1af070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:15.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:15.182+0000 7f4bbc43f740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dbcc13bd70 0x55dbcc13f390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:15.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:15.182+0000 7f4bbc43f740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f4bb41b2520 con 0x55dbcc1314b0 2024-09-10T10:01:15.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:15.182+0000 7f4bbc43f740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f4bb41b1e50 con 0x55dbcc130b90 2024-09-10T10:01:15.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:15.182+0000 7f4bbc43f740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f4bb41af090 con 0x55dbcc13bd70 2024-09-10T10:01:15.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:15.182+0000 7f4bbc43f740 10 client.0 _ll_register_callbacks cb 0x7f4ba0004460 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:01:15.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:15.183+0000 7f4bb2a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dbcc130b90 0x55dbcc1aeb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:15.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:15.183+0000 7f4bb2a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dbcc130b90 0x55dbcc1aeb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:51532/0 (socket says 172.21.15.5:51532) 2024-09-10T10:01:15.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:15.183+0000 7f4bb2a00640 1 -- 172.21.15.5:0/1034218106 learned_addr learned my addr 172.21.15.5:0/1034218106 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T10:01:15.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:15.183+0000 7f4bb3400640 1 --2- 172.21.15.5:0/1034218106 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dbcc13bd70 0x55dbcc13f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:15.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:15.183+0000 7f4bb2000640 1 --2- 172.21.15.5:0/1034218106 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dbcc1314b0 0x55dbcc1af070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:15.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:15.183+0000 7f4bb3400640 1 -- 172.21.15.5:0/1034218106 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dbcc130b90 msgr2=0x55dbcc1aeb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:01:15.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:15.183+0000 7f4bb3400640 1 --2- 172.21.15.5:0/1034218106 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dbcc130b90 0x55dbcc1aeb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:15.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:15.183+0000 7f4bb3400640 1 -- 172.21.15.5:0/1034218106 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dbcc1314b0 msgr2=0x55dbcc1af070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:01:15.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:15.183+0000 7f4bb3400640 1 --2- 172.21.15.5:0/1034218106 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dbcc1314b0 0x55dbcc1af070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:15.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:15.183+0000 7f4bb3400640 1 -- 172.21.15.5:0/1034218106 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dbcc13fa80 con 0x55dbcc13bd70 2024-09-10T10:01:15.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:15.183+0000 7f4bb2000640 1 --2- 172.21.15.5:0/1034218106 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dbcc1314b0 0x55dbcc1af070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:01:15.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:15.185+0000 7f4bb3400640 1 --2- 172.21.15.5:0/1034218106 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dbcc13bd70 0x55dbcc13f390 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f4bac01e880 tx=0x7f4bac01ebb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b978e5917a6e9bff server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:15.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:15.185+0000 7f4b9cc00640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-09-10T10:01:15.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:15.185+0000 7f4b9cc00640 1 -- 172.21.15.5:0/1034218106 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bac012af0 con 0x55dbcc13bd70 2024-09-10T10:01:15.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:15.186+0000 7f4b9cc00640 1 -- 172.21.15.5:0/1034218106 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f4bac025070 con 0x55dbcc13bd70 2024-09-10T10:01:15.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:15.186+0000 7f4bbc43f740 1 -- 172.21.15.5:0/1034218106 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4bac012cf0 con 0x55dbcc13bd70 2024-09-10T10:01:15.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:15.186+0000 7f4b9cc00640 1 -- 172.21.15.5:0/1034218106 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f4bac03a050 con 0x55dbcc13bd70 2024-09-10T10:01:15.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:15.186+0000 7f4bbc43f740 10 client.29716 fetch_fsmap learned FSMap version 42 2024-09-10T10:01:15.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:15.186+0000 7f4bbc43f740 1 -- 172.21.15.5:0/1034218106 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55dbcc1b63a0 con 0x55dbcc13bd70 2024-09-10T10:01:15.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:15.187+0000 7f4b9cc00640 1 -- 172.21.15.5:0/1034218106 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4bac012cf0 con 0x55dbcc13bd70 2024-09-10T10:01:15.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:15.187+0000 7f4b9cc00640 1 -- 172.21.15.5:0/1034218106 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 42) ==== 8370+0+0 (secure 0 0 0) 0x7f4bac029a10 con 0x55dbcc13bd70 2024-09-10T10:01:15.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:15.187+0000 7f4bbc43f740 10 client.29716 fetch_fsmap finished waiting for FSMap version 42 2024-09-10T10:01:15.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:15.187+0000 7f4bbc43f740 10 client.29716 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T10:01:15.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:15.187+0000 7f4bbc43f740 20 client.29716 populate_metadata read hostname 'smithi005' 2024-09-10T10:01:15.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:15.187+0000 7f4bbc43f740 1 --2- 172.21.15.5:0/1034218106 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55dbcc1b69b0 0x55dbcc1b8dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:15.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:15.187+0000 7f4bbc43f740 4 client.29716 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T10:01:15.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:15.187+0000 7f4bbc43f740 1 -- 172.21.15.5:0/1034218106 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55dbcc09ee60 con 0x55dbcc1b69b0 2024-09-10T10:01:15.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:15.188+0000 7f4bb2a00640 1 --2- 172.21.15.5:0/1034218106 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55dbcc1b69b0 0x55dbcc1b8dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:15.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:15.189+0000 7f4bb2a00640 1 --2- 172.21.15.5:0/1034218106 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55dbcc1b69b0 0x55dbcc1b8dd0 crc :-1 s=READY pgs=312 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2e2590760fc8d6be server_cookie=ccb649f5f48da595 in_seq=0 out_seq=0 2024-09-10T10:01:15.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:15.190+0000 7f4b9cc00640 10 client.29716 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T10:01:15.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:15.190+0000 7f4b9cc00640 1 -- 172.21.15.5:0/1034218106 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55dbcc09ee60 con 0x55dbcc1b69b0 2024-09-10T10:01:15.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:15.190+0000 7f4b9cc00640 10 client.29716 handle_command_reply: tid=0 2024-09-10T10:01:15.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:15.190+0000 7f4bbc43f740 1 client.29716 shutdown 2024-09-10T10:01:15.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:15.191+0000 7f4bbc43f740 1 -- 172.21.15.5:0/1034218106 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dbcc13bd70 msgr2=0x55dbcc13f390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:15.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:15.191+0000 7f4bbc43f740 1 --2- 172.21.15.5:0/1034218106 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dbcc13bd70 0x55dbcc13f390 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f4bac01e880 tx=0x7f4bac01ebb0 comp rx=0 tx=0).stop 2024-09-10T10:01:15.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:15.191+0000 7f4bbc43f740 1 -- 172.21.15.5:0/1034218106 shutdown_connections 2024-09-10T10:01:15.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:15.191+0000 7f4bbc43f740 1 --2- 172.21.15.5:0/1034218106 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55dbcc1b69b0 0x55dbcc1b8dd0 crc :-1 s=READY pgs=312 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:15.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:15.191+0000 7f4bbc43f740 1 --2- 172.21.15.5:0/1034218106 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55dbcc13bd70 0x55dbcc13f390 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:15.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:15.191+0000 7f4bbc43f740 1 --2- 172.21.15.5:0/1034218106 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dbcc1314b0 0x55dbcc1af070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:15.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:15.191+0000 7f4bbc43f740 1 --2- 172.21.15.5:0/1034218106 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55dbcc130b90 0x55dbcc1aeb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:15.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:15.191+0000 7f4b9cc00640 0 client.29716 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T10:01:15.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:15.191+0000 7f4bbc43f740 1 -- 172.21.15.5:0/1034218106 >> 172.21.15.5:0/1034218106 conn(0x55dbcc12d120 msgr2=0x55dbcc1374f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:15.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:15.192+0000 7f4bbc43f740 1 -- 172.21.15.5:0/1034218106 shutdown_connections 2024-09-10T10:01:15.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:15.192+0000 7f4bbc43f740 1 -- 172.21.15.5:0/1034218106 wait complete. 2024-09-10T10:01:15.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:15.192+0000 7f4bbc43f740 20 client.29716 trim_cache size 0 max 16384 2024-09-10T10:01:15.195 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-10T10:01:15.195 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "no active scrubs running", 2024-09-10T10:01:15.195 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": {} 2024-09-10T10:01:15.196 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-10T10:01:15.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:15.194+0000 7f4bb9800640 1 -- 172.21.15.5:0/3226482932 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f4b980a2c00 msgr2=0x7f4b980a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:15.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:15.194+0000 7f4bb9800640 1 --2- 172.21.15.5:0/3226482932 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f4b980a2c00 0x7f4b980a50c0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f4bac005fd0 tx=0x7f4bac004a40 comp rx=0 tx=0).stop 2024-09-10T10:01:15.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:15.195+0000 7f4bb9800640 1 -- 172.21.15.5:0/3226482932 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4bb4074620 msgr2=0x7f4bb419a650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:15.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:15.195+0000 7f4bb9800640 1 --2- 172.21.15.5:0/3226482932 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4bb4074620 0x7f4bb419a650 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f4ba400d950 tx=0x7f4ba400de10 comp rx=0 tx=0).stop 2024-09-10T10:01:15.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:15.195+0000 7f4bb9800640 1 -- 172.21.15.5:0/3226482932 shutdown_connections 2024-09-10T10:01:15.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:15.195+0000 7f4bb9800640 1 --2- 172.21.15.5:0/3226482932 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f4b980a2c00 0x7f4b980a50c0 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:15.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:15.195+0000 7f4bb9800640 1 --2- 172.21.15.5:0/3226482932 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4bb410e3c0 0x7f4bb419ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:15.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:15.195+0000 7f4bb9800640 1 --2- 172.21.15.5:0/3226482932 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4bb4074f40 0x7f4bb419ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:15.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:15.195+0000 7f4bb9800640 1 --2- 172.21.15.5:0/3226482932 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4bb4074620 0x7f4bb419a650 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:15.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:15.195+0000 7f4bb9800640 1 -- 172.21.15.5:0/3226482932 >> 172.21.15.5:0/3226482932 conn(0x7f4bb40fd8c0 msgr2=0x7f4bb40fe520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:15.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:15.196+0000 7f4bb9800640 1 -- 172.21.15.5:0/3226482932 shutdown_connections 2024-09-10T10:01:15.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:15.196+0000 7f4bb9800640 1 -- 172.21.15.5:0/3226482932 wait complete. 2024-09-10T10:01:15.241 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:01:15 smithi005 ceph-mon[30350]: pgmap v1080: 129 pgs: 129 active+clean; 685 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.6 MiB/s wr, 120 op/s 2024-09-10T10:01:15.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:01:15 smithi159 ceph-mon[35513]: pgmap v1080: 129 pgs: 129 active+clean; 685 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.6 MiB/s wr, 120 op/s 2024-09-10T10:01:15.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:01:15 smithi102 ceph-mon[35442]: pgmap v1080: 129 pgs: 129 active+clean; 685 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.6 MiB/s wr, 120 op/s 2024-09-10T10:01:16.040 INFO:tasks.workunit.client.0.smithi005.stdout: 10 138000 12.89 MB/sec execute 414 sec latency 65.309 ms 2024-09-10T10:01:16.269 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-10T10:01:16.269 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T10:01:16.856 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:01:17.040 INFO:tasks.workunit.client.0.smithi005.stdout: 10 138500 12.93 MB/sec execute 415 sec latency 77.791 ms 2024-09-10T10:01:17.277 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:01:17 smithi005 ceph-mon[30350]: pgmap v1081: 129 pgs: 129 active+clean; 691 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 3.8 MiB/s wr, 269 op/s 2024-09-10T10:01:17.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:17.476+0000 7fcb97200640 1 -- 172.21.15.5:0/164737656 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcb901009a0 msgr2=0x7fcb901049f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:17.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:17.476+0000 7fcb97200640 1 --2- 172.21.15.5:0/164737656 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcb901009a0 0x7fcb901049f0 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7fcb78009860 tx=0x7fcb7802a340 comp rx=0 tx=0).stop 2024-09-10T10:01:17.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:17.476+0000 7fcb97200640 1 -- 172.21.15.5:0/164737656 shutdown_connections 2024-09-10T10:01:17.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:17.476+0000 7fcb97200640 1 --2- 172.21.15.5:0/164737656 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcb90105120 0x7fcb9010cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:17.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:17.476+0000 7fcb97200640 1 --2- 172.21.15.5:0/164737656 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcb901009a0 0x7fcb901049f0 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:17.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:17.476+0000 7fcb97200640 1 --2- 172.21.15.5:0/164737656 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcb900ffff0 0x7fcb901003d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:17.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:17.476+0000 7fcb97200640 1 -- 172.21.15.5:0/164737656 >> 172.21.15.5:0/164737656 conn(0x7fcb900fba90 msgr2=0x7fcb900fdeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:17.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:17.476+0000 7fcb97200640 1 -- 172.21.15.5:0/164737656 shutdown_connections 2024-09-10T10:01:17.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:17.477+0000 7fcb97200640 1 -- 172.21.15.5:0/164737656 wait complete. 2024-09-10T10:01:17.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:17.479+0000 7fcb97200640 1 Processor -- start 2024-09-10T10:01:17.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:17.479+0000 7fcb97200640 1 -- start start 2024-09-10T10:01:17.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:17.480+0000 7fcb97200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcb900ffff0 0x7fcb9019a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:17.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:17.480+0000 7fcb97200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcb901009a0 0x7fcb9019ac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:17.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:17.480+0000 7fcb97200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcb90105120 0x7fcb9019f010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:17.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:17.480+0000 7fcb97200640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fcb9010fd60 con 0x7fcb901009a0 2024-09-10T10:01:17.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:17.480+0000 7fcb97200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fcb9010fbe0 con 0x7fcb900ffff0 2024-09-10T10:01:17.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:17.480+0000 7fcb97200640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fcb9010fee0 con 0x7fcb90105120 2024-09-10T10:01:17.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:17.480+0000 7fcb8fe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcb900ffff0 0x7fcb9019a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:17.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:17.480+0000 7fcb8fe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcb900ffff0 0x7fcb9019a740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:51544/0 (socket says 172.21.15.5:51544) 2024-09-10T10:01:17.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:17.480+0000 7fcb8fe00640 1 -- 172.21.15.5:0/2326758434 learned_addr learned my addr 172.21.15.5:0/2326758434 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T10:01:17.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:17.481+0000 7fcb8fe00640 1 -- 172.21.15.5:0/2326758434 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcb90105120 msgr2=0x7fcb9019f010 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T10:01:17.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:17.481+0000 7fcb8fe00640 1 --2- 172.21.15.5:0/2326758434 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcb90105120 0x7fcb9019f010 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:17.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:17.481+0000 7fcb8fe00640 1 -- 172.21.15.5:0/2326758434 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcb901009a0 msgr2=0x7fcb9019ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:17.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:17.481+0000 7fcb8fe00640 1 --2- 172.21.15.5:0/2326758434 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcb901009a0 0x7fcb9019ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:17.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:17.481+0000 7fcb8fe00640 1 -- 172.21.15.5:0/2326758434 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb9019f6f0 con 0x7fcb900ffff0 2024-09-10T10:01:17.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:17.482+0000 7fcb8fe00640 1 --2- 172.21.15.5:0/2326758434 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcb900ffff0 0x7fcb9019a740 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fcb8000fad0 tx=0x7fcb8000ff90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:17.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:17.482+0000 7fcb8cc00640 1 -- 172.21.15.5:0/2326758434 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb8000ddb0 con 0x7fcb900ffff0 2024-09-10T10:01:17.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:17.483+0000 7fcb97200640 1 -- 172.21.15.5:0/2326758434 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb9019f9e0 con 0x7fcb900ffff0 2024-09-10T10:01:17.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:17.483+0000 7fcb97200640 1 -- 172.21.15.5:0/2326758434 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb901a72c0 con 0x7fcb900ffff0 2024-09-10T10:01:17.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:17.483+0000 7fcb8cc00640 1 -- 172.21.15.5:0/2326758434 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcb800035f0 con 0x7fcb900ffff0 2024-09-10T10:01:17.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:17.483+0000 7fcb8cc00640 1 -- 172.21.15.5:0/2326758434 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb80017650 con 0x7fcb900ffff0 2024-09-10T10:01:17.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:17.486+0000 7fcb97200640 1 -- 172.21.15.5:0/2326758434 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb6c005290 con 0x7fcb900ffff0 2024-09-10T10:01:17.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:17.488+0000 7fcb8cc00640 1 -- 172.21.15.5:0/2326758434 <== mon.2 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7fcb8001e020 con 0x7fcb900ffff0 2024-09-10T10:01:17.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:17.489+0000 7fcb8cc00640 1 --2- 172.21.15.5:0/2326758434 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fcb740a2c40 0x7fcb740a5100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:17.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:17.489+0000 7fcb8cc00640 1 -- 172.21.15.5:0/2326758434 <== mon.2 v2:172.21.15.102:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fcb800cee60 con 0x7fcb900ffff0 2024-09-10T10:01:17.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:17.490+0000 7fcb8f400640 1 --2- 172.21.15.5:0/2326758434 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fcb740a2c40 0x7fcb740a5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:17.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:17.490+0000 7fcb8f400640 1 --2- 172.21.15.5:0/2326758434 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fcb740a2c40 0x7fcb740a5100 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fcb780095f0 tx=0x7fcb78005760 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:17.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:17.494+0000 7fcb8cc00640 1 -- 172.21.15.5:0/2326758434 <== mon.2 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fcb8009b9d0 con 0x7fcb900ffff0 2024-09-10T10:01:17.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:01:17 smithi159 ceph-mon[35513]: pgmap v1081: 129 pgs: 129 active+clean; 691 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 3.8 MiB/s wr, 269 op/s 2024-09-10T10:01:17.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:01:17 smithi102 ceph-mon[35442]: pgmap v1081: 129 pgs: 129 active+clean; 691 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 3.8 MiB/s wr, 269 op/s 2024-09-10T10:01:17.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:17.885+0000 7fcb97200640 1 -- 172.21.15.5:0/2326758434 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcb6c005580 con 0x7fcb900ffff0 2024-09-10T10:01:17.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:17.886+0000 7fcb8cc00640 1 -- 172.21.15.5:0/2326758434 <== mon.2 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 42 v42) ==== 94+0+11969 (secure 0 0 0) 0x7fcb8009b2d0 con 0x7fcb900ffff0 2024-09-10T10:01:17.888 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-10T10:01:17.888 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":42,"btime":"2024-09-10T09:55:06:670916+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14454,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/2970594311","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2970594311},{"type":"v1","addr":"172.21.15.5:6835","nonce":2970594311}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14514,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6839/3616502174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":3616502174},{"type":"v1","addr":"172.21.15.5:6839","nonce":3616502174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":42,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T09:35:29.626287+0000","modified":"2024-09-10T09:55:06.419829+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24542,"mds_1":24535,"mds_2":24511,"mds_3":24506,"mds_4":24443},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14520":{"gid":14520,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/2073599842","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":2073599842},{"type":"v1","addr":"172.21.15.5:6841","nonce":2073599842}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"k","rank":4,"incarnation":26,"state":"up:active","state_seq":24,"addr":"172.21.15.102:6833/2457007427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":2457007427},{"type":"v1","addr":"172.21.15.102:6833","nonce":2457007427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/356445159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":356445159},{"type":"v1","addr":"172.21.15.159:6833","nonce":356445159}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6837/2894314507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":2894314507},{"type":"v1","addr":"172.21.15.5:6837","nonce":2894314507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.102:6835/1647549111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":1647549111},{"type":"v1","addr":"172.21.15.102:6835","nonce":1647549111}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6835/952657242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":952657242},{"type":"v1","addr":"172.21.15.159:6835","nonce":952657242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":19,"addr":"172.21.15.102:6837/2689776764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":2689776764},{"type":"v1","addr":"172.21.15.102:6837","nonce":2689776764}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.159:6837/620627639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":620627639},{"type":"v1","addr":"172.21.15.159:6837","nonce":620627639}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24535":{"gid":24535,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":18,"addr":"172.21.15.159:6839/2073739599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":2073739599},{"type":"v1","addr":"172.21.15.159:6839","nonce":2073739599}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24542":{"gid":24542,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.102:6839/335409749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6838","nonce":335409749},{"type":"v1","addr":"172.21.15.102:6839","nonce":335409749}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24542,"qdb_cluster":[24542,24535,24511,24506,24443]},"id":1}]} 2024-09-10T10:01:17.889 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 42 2024-09-10T10:01:17.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:17.891+0000 7fcb97200640 1 -- 172.21.15.5:0/2326758434 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fcb740a2c40 msgr2=0x7fcb740a5100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:17.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:17.891+0000 7fcb97200640 1 --2- 172.21.15.5:0/2326758434 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fcb740a2c40 0x7fcb740a5100 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fcb780095f0 tx=0x7fcb78005760 comp rx=0 tx=0).stop 2024-09-10T10:01:17.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:17.891+0000 7fcb97200640 1 -- 172.21.15.5:0/2326758434 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcb900ffff0 msgr2=0x7fcb9019a740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:17.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:17.891+0000 7fcb97200640 1 --2- 172.21.15.5:0/2326758434 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcb900ffff0 0x7fcb9019a740 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fcb8000fad0 tx=0x7fcb8000ff90 comp rx=0 tx=0).stop 2024-09-10T10:01:17.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:17.892+0000 7fcb97200640 1 -- 172.21.15.5:0/2326758434 shutdown_connections 2024-09-10T10:01:17.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:17.892+0000 7fcb97200640 1 --2- 172.21.15.5:0/2326758434 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fcb740a2c40 0x7fcb740a5100 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:17.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:17.892+0000 7fcb97200640 1 --2- 172.21.15.5:0/2326758434 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fcb90105120 0x7fcb9019f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:17.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:17.892+0000 7fcb97200640 1 --2- 172.21.15.5:0/2326758434 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcb901009a0 0x7fcb9019ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:17.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:17.892+0000 7fcb97200640 1 --2- 172.21.15.5:0/2326758434 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fcb900ffff0 0x7fcb9019a740 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:17.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:17.892+0000 7fcb97200640 1 -- 172.21.15.5:0/2326758434 >> 172.21.15.5:0/2326758434 conn(0x7fcb900fba90 msgr2=0x7fcb900fce30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:17.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:17.892+0000 7fcb97200640 1 -- 172.21.15.5:0/2326758434 shutdown_connections 2024-09-10T10:01:17.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:17.892+0000 7fcb97200640 1 -- 172.21.15.5:0/2326758434 wait complete. 2024-09-10T10:01:18.040 INFO:tasks.workunit.client.0.smithi005.stdout: 10 138907 12.94 MB/sec execute 416 sec latency 266.761 ms 2024-09-10T10:01:18.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:01:18 smithi005 ceph-mon[30350]: from='client.? 172.21.15.5:0/2326758434' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:01:18.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:01:18 smithi159 ceph-mon[35513]: from='client.? 172.21.15.5:0/2326758434' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:01:18.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:01:18 smithi102 ceph-mon[35442]: from='client.? 172.21.15.5:0/2326758434' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:01:18.849 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-10T10:01:19.040 INFO:tasks.workunit.client.0.smithi005.stdout: 10 139518 12.98 MB/sec execute 417 sec latency 76.567 ms 2024-09-10T10:01:19.222 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:01:19 smithi005 ceph-mon[30350]: pgmap v1082: 129 pgs: 129 active+clean; 692 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 2.8 MiB/s wr, 288 op/s 2024-09-10T10:01:19.409 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:01:19.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:01:19 smithi159 ceph-mon[35513]: pgmap v1082: 129 pgs: 129 active+clean; 692 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 2.8 MiB/s wr, 288 op/s 2024-09-10T10:01:19.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:01:19 smithi102 ceph-mon[35442]: pgmap v1082: 129 pgs: 129 active+clean; 692 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 2.8 MiB/s wr, 288 op/s 2024-09-10T10:01:20.041 INFO:tasks.workunit.client.0.smithi005.stdout: 10 140320 13.06 MB/sec execute 418 sec latency 94.503 ms 2024-09-10T10:01:20.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:20.252+0000 7f1ffaa00640 1 -- 172.21.15.5:0/1689223682 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1ff4074e30 msgr2=0x7f1ff410dd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:20.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:20.252+0000 7f1ffaa00640 1 --2- 172.21.15.5:0/1689223682 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1ff4074e30 0x7f1ff410dd40 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f1fe0009900 tx=0x7f1fe002a480 comp rx=0 tx=0).stop 2024-09-10T10:01:20.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:20.252+0000 7f1ffaa00640 1 -- 172.21.15.5:0/1689223682 shutdown_connections 2024-09-10T10:01:20.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:20.252+0000 7f1ffaa00640 1 --2- 172.21.15.5:0/1689223682 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1ff410e280 0x7f1ff4110670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:20.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:20.252+0000 7f1ffaa00640 1 --2- 172.21.15.5:0/1689223682 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1ff4074e30 0x7f1ff410dd40 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:20.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:20.252+0000 7f1ffaa00640 1 --2- 172.21.15.5:0/1689223682 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1ff4074510 0x7f1ff40748f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:20.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:20.252+0000 7f1ffaa00640 1 -- 172.21.15.5:0/1689223682 >> 172.21.15.5:0/1689223682 conn(0x7f1ff40fd740 msgr2=0x7f1ff40ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:20.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:20.252+0000 7f1ffaa00640 1 -- 172.21.15.5:0/1689223682 shutdown_connections 2024-09-10T10:01:20.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:20.252+0000 7f1ffaa00640 1 -- 172.21.15.5:0/1689223682 wait complete. 2024-09-10T10:01:20.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:20.253+0000 7f1ffaa00640 1 Processor -- start 2024-09-10T10:01:20.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:20.253+0000 7f1ffaa00640 1 -- start start 2024-09-10T10:01:20.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:20.253+0000 7f1ffaa00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1ff4074510 0x7f1ff419a570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:20.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:20.253+0000 7f1ffaa00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1ff4074e30 0x7f1ff419aab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:20.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:20.254+0000 7f1ffaa00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1ff410e280 0x7f1ff419ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:20.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:20.254+0000 7f1ffaa00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1ff4113f70 con 0x7f1ff4074510 2024-09-10T10:01:20.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:20.254+0000 7f1ffaa00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f1ff4113df0 con 0x7f1ff4074e30 2024-09-10T10:01:20.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:20.254+0000 7f1ffaa00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f1ff41140f0 con 0x7f1ff410e280 2024-09-10T10:01:20.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:20.254+0000 7f1ff8c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1ff4074e30 0x7f1ff419aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:20.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:20.254+0000 7f1ff8c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1ff4074e30 0x7f1ff419aab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:51570/0 (socket says 172.21.15.5:51570) 2024-09-10T10:01:20.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:20.254+0000 7f1ff8c00640 1 -- 172.21.15.5:0/2069011210 learned_addr learned my addr 172.21.15.5:0/2069011210 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T10:01:20.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:20.254+0000 7f1ff9600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1ff4074510 0x7f1ff419a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:20.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:20.254+0000 7f1ffa000640 1 --2- 172.21.15.5:0/2069011210 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1ff410e280 0x7f1ff419ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:20.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:20.254+0000 7f1ff8c00640 1 -- 172.21.15.5:0/2069011210 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1ff410e280 msgr2=0x7f1ff419ee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:20.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:20.254+0000 7f1ff8c00640 1 --2- 172.21.15.5:0/2069011210 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1ff410e280 0x7f1ff419ee40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:20.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:20.254+0000 7f1ff8c00640 1 -- 172.21.15.5:0/2069011210 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1ff4074510 msgr2=0x7f1ff419a570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:20.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:20.254+0000 7f1ff8c00640 1 --2- 172.21.15.5:0/2069011210 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1ff4074510 0x7f1ff419a570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:20.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:20.254+0000 7f1ff8c00640 1 -- 172.21.15.5:0/2069011210 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ff419f520 con 0x7f1ff4074e30 2024-09-10T10:01:20.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:20.254+0000 7f1ffa000640 1 --2- 172.21.15.5:0/2069011210 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1ff410e280 0x7f1ff419ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:01:20.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:20.255+0000 7f1ff8c00640 1 --2- 172.21.15.5:0/2069011210 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1ff4074e30 0x7f1ff419aab0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f1fe0006010 tx=0x7f1fe0004040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:20.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:20.255+0000 7f1ff2000640 1 -- 172.21.15.5:0/2069011210 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fe003f030 con 0x7f1ff4074e30 2024-09-10T10:01:20.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:20.255+0000 7f1ff2000640 1 -- 172.21.15.5:0/2069011210 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1fe0004440 con 0x7f1ff4074e30 2024-09-10T10:01:20.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:20.255+0000 7f1ff2000640 1 -- 172.21.15.5:0/2069011210 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fe0035070 con 0x7f1ff4074e30 2024-09-10T10:01:20.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:20.255+0000 7f1ffaa00640 1 -- 172.21.15.5:0/2069011210 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ff419f7b0 con 0x7f1ff4074e30 2024-09-10T10:01:20.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:20.256+0000 7f1ffaa00640 1 -- 172.21.15.5:0/2069011210 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ff419fc40 con 0x7f1ff4074e30 2024-09-10T10:01:20.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:20.259+0000 7f1ffebdf740 1 Processor -- start 2024-09-10T10:01:20.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:20.259+0000 7f1ffebdf740 1 -- start start 2024-09-10T10:01:20.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:20.260+0000 7f1ffebdf740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55999db08e40 0x55999db09220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:20.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:20.260+0000 7f1ffebdf740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55999db09760 0x55999db13ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:20.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:20.260+0000 7f1ffebdf740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55999db14020 0x55999db16410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:20.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:20.260+0000 7f1ffebdf740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55999da6f3e0 con 0x55999db09760 2024-09-10T10:01:20.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:20.260+0000 7f1ffebdf740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55999da16d80 con 0x55999db08e40 2024-09-10T10:01:20.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:20.260+0000 7f1ffebdf740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55999da6b3a0 con 0x55999db14020 2024-09-10T10:01:20.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:20.260+0000 7f1ffa000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55999db08e40 0x55999db09220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:20.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:20.260+0000 7f1ffa000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55999db08e40 0x55999db09220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:51584/0 (socket says 172.21.15.5:51584) 2024-09-10T10:01:20.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:20.260+0000 7f1ffa000640 1 -- 172.21.15.5:0/3298686649 learned_addr learned my addr 172.21.15.5:0/3298686649 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T10:01:20.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:20.260+0000 7f1ff9600640 1 --2- 172.21.15.5:0/3298686649 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55999db09760 0x55999db13ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:20.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:20.261+0000 7f1ffa000640 1 -- 172.21.15.5:0/3298686649 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55999db14020 msgr2=0x55999db16410 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T10:01:20.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:20.261+0000 7f1ffa000640 1 --2- 172.21.15.5:0/3298686649 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55999db14020 0x55999db16410 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:20.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:20.261+0000 7f1ffa000640 1 -- 172.21.15.5:0/3298686649 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55999db09760 msgr2=0x55999db13ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:01:20.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:20.261+0000 7f1ffa000640 1 --2- 172.21.15.5:0/3298686649 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55999db09760 0x55999db13ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:20.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:20.261+0000 7f1ffa000640 1 -- 172.21.15.5:0/3298686649 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55999da35ae0 con 0x55999db08e40 2024-09-10T10:01:20.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:20.261+0000 7f1ff2000640 1 -- 172.21.15.5:0/2069011210 <== mon.2 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f1fe0042030 con 0x7f1ff4074e30 2024-09-10T10:01:20.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:20.261+0000 7f1ffa000640 1 --2- 172.21.15.5:0/3298686649 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55999db08e40 0x55999db09220 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f1fec011760 tx=0x7f1fec011c20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d138f04c9d87dd server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:20.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:20.261+0000 7f1fe6600640 1 -- 172.21.15.5:0/3298686649 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fec004d20 con 0x55999db08e40 2024-09-10T10:01:20.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:20.261+0000 7f1fe6600640 1 -- 172.21.15.5:0/3298686649 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1fec00f9a0 con 0x55999db08e40 2024-09-10T10:01:20.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:20.261+0000 7f1fe6600640 1 -- 172.21.15.5:0/3298686649 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fec004480 con 0x55999db08e40 2024-09-10T10:01:20.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:20.262+0000 7f1ffebdf740 1 -- 172.21.15.5:0/3298686649 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55999db08e40 msgr2=0x55999db09220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:20.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:20.262+0000 7f1ffebdf740 1 --2- 172.21.15.5:0/3298686649 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55999db08e40 0x55999db09220 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f1fec011760 tx=0x7f1fec011c20 comp rx=0 tx=0).stop 2024-09-10T10:01:20.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:20.263+0000 7f1ffebdf740 1 -- 172.21.15.5:0/3298686649 shutdown_connections 2024-09-10T10:01:20.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:20.263+0000 7f1ffebdf740 1 --2- 172.21.15.5:0/3298686649 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55999db14020 0x55999db16410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:20.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:20.263+0000 7f1ffebdf740 1 --2- 172.21.15.5:0/3298686649 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55999db09760 0x55999db13ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:20.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:20.263+0000 7f1ffebdf740 1 --2- 172.21.15.5:0/3298686649 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55999db08e40 0x55999db09220 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:20.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:20.263+0000 7f1ffebdf740 1 -- 172.21.15.5:0/3298686649 >> 172.21.15.5:0/3298686649 conn(0x55999db053d0 msgr2=0x55999db057b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:20.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:20.263+0000 7f1ffebdf740 1 -- 172.21.15.5:0/3298686649 shutdown_connections 2024-09-10T10:01:20.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:20.263+0000 7f1ff2000640 1 --2- 172.21.15.5:0/2069011210 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f1fdc0a2a90 0x7f1fdc0a4f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:20.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:20.263+0000 7f1ffebdf740 1 -- 172.21.15.5:0/3298686649 wait complete. 2024-09-10T10:01:20.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:20.263+0000 7f1ff2000640 1 -- 172.21.15.5:0/2069011210 <== mon.2 v2:172.21.15.102:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f1fe00ef790 con 0x7f1ff4074e30 2024-09-10T10:01:20.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:20.264+0000 7f1ffebdf740 1 -- start start 2024-09-10T10:01:20.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:20.265+0000 7f1ffebdf740 1 Processor -- start 2024-09-10T10:01:20.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:20.265+0000 7f1ff9600640 1 --2- 172.21.15.5:0/2069011210 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f1fdc0a2a90 0x7f1fdc0a4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:20.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:20.266+0000 7f1ffebdf740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55999db08e40 0x55999db10f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:20.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:20.266+0000 7f1ffebdf740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55999db09760 0x55999db114a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:20.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:20.266+0000 7f1ffebdf740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55999db14020 0x55999db119e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:20.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:20.266+0000 7f1ffebdf740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55999db16bf0 con 0x55999db08e40 2024-09-10T10:01:20.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:20.266+0000 7f1ffebdf740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55999da2a0f0 con 0x55999db14020 2024-09-10T10:01:20.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:20.266+0000 7f1ffebdf740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55999db16d70 con 0x55999db09760 2024-09-10T10:01:20.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:20.266+0000 7f1ffebdf740 10 client.0 _ll_register_callbacks cb 0x55999da11bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:01:20.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:20.267+0000 7f1ff9600640 1 --2- 172.21.15.5:0/2069011210 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f1fdc0a2a90 0x7f1fdc0a4f50 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x55999db06230 tx=0x7f1fe8005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:20.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:20.267+0000 7f1ff8c00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55999db09760 0x55999db114a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:20.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:20.267+0000 7f1ff8c00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55999db09760 0x55999db114a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:41702/0 (socket says 172.21.15.5:41702) 2024-09-10T10:01:20.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:20.267+0000 7f1ff8c00640 1 -- 172.21.15.5:0/415016053 learned_addr learned my addr 172.21.15.5:0/415016053 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T10:01:20.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:20.267+0000 7f1ff9600640 1 --2- 172.21.15.5:0/415016053 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55999db08e40 0x55999db10f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:20.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:20.267+0000 7f1ff8c00640 1 -- 172.21.15.5:0/415016053 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55999db14020 msgr2=0x55999db119e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T10:01:20.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:20.267+0000 7f1ff8c00640 1 --2- 172.21.15.5:0/415016053 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55999db14020 0x55999db119e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:20.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:20.267+0000 7f1ff8c00640 1 -- 172.21.15.5:0/415016053 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55999db08e40 msgr2=0x55999db10f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:01:20.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:20.267+0000 7f1ff8c00640 1 --2- 172.21.15.5:0/415016053 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55999db08e40 0x55999db10f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:20.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:20.267+0000 7f1ff8c00640 1 -- 172.21.15.5:0/415016053 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55999d7e03c0 con 0x55999db09760 2024-09-10T10:01:20.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:20.268+0000 7f1ff8c00640 1 --2- 172.21.15.5:0/415016053 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55999db09760 0x55999db114a0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f1fe00c4400 tx=0x7f1fe00bb8c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a0b45ed0c0aacbcd server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:20.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:20.268+0000 7f1fdaa00640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-09-10T10:01:20.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:20.269+0000 7f1fdaa00640 1 -- 172.21.15.5:0/415016053 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fe00c0070 con 0x55999db09760 2024-09-10T10:01:20.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:20.269+0000 7f1fdaa00640 1 -- 172.21.15.5:0/415016053 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f1fe0003cd0 con 0x55999db09760 2024-09-10T10:01:20.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:20.269+0000 7f1ffebdf740 1 -- 172.21.15.5:0/415016053 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1fec00f9a0 con 0x55999db09760 2024-09-10T10:01:20.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:20.269+0000 7f1fdaa00640 1 -- 172.21.15.5:0/415016053 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f1fe00c4930 con 0x55999db09760 2024-09-10T10:01:20.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:20.269+0000 7f1ffebdf740 10 client.29725 fetch_fsmap learned FSMap version 42 2024-09-10T10:01:20.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:20.269+0000 7f1ffebdf740 1 -- 172.21.15.5:0/415016053 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55999db16ca0 con 0x55999db09760 2024-09-10T10:01:20.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:20.270+0000 7f1fdaa00640 1 -- 172.21.15.5:0/415016053 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1fec00f9a0 con 0x55999db09760 2024-09-10T10:01:20.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:20.270+0000 7f1fdaa00640 1 -- 172.21.15.5:0/415016053 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 42) ==== 8370+0+0 (secure 0 0 0) 0x7f1fe00c44a0 con 0x55999db09760 2024-09-10T10:01:20.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:20.270+0000 7f1ffebdf740 10 client.29725 fetch_fsmap finished waiting for FSMap version 42 2024-09-10T10:01:20.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:20.270+0000 7f1ffebdf740 10 client.29725 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-09-10T10:01:20.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:20.270+0000 7f1ffebdf740 20 client.29725 populate_metadata read hostname 'smithi005' 2024-09-10T10:01:20.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:20.270+0000 7f1ffebdf740 1 --2- 172.21.15.5:0/415016053 >> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] conn(0x55999db2c950 0x55999db2ed40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:20.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:20.270+0000 7f1ffebdf740 4 client.29725 mds_command: new command op to 24443 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:01:20.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:20.270+0000 7f1ffebdf740 1 -- 172.21.15.5:0/415016053 --> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55999da7c950 con 0x55999db2c950 2024-09-10T10:01:20.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:20.271+0000 7f1ff9600640 1 --2- 172.21.15.5:0/415016053 >> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] conn(0x55999db2c950 0x55999db2ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:20.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:20.274+0000 7f1ff9600640 1 --2- 172.21.15.5:0/415016053 >> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] conn(0x55999db2c950 0x55999db2ed40 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=9b2974d390c66a56 server_cookie=8bd342a1862c4227 in_seq=0 out_seq=0 2024-09-10T10:01:20.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:20.274+0000 7f1fdaa00640 10 client.29725 ms_handle_connect on v2:172.21.15.102:6832/2457007427 2024-09-10T10:01:20.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:20.274+0000 7f1fdaa00640 1 -- 172.21.15.5:0/415016053 <== mds.4 v2:172.21.15.102:6832/2457007427 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55999da7c950 con 0x55999db2c950 2024-09-10T10:01:20.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:20.274+0000 7f1fdaa00640 10 client.29725 handle_command_reply: tid=0 2024-09-10T10:01:20.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:20.275+0000 7f1ffebdf740 1 client.29725 shutdown 2024-09-10T10:01:20.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:20.275+0000 7f1ffebdf740 1 -- 172.21.15.5:0/415016053 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55999db09760 msgr2=0x55999db114a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:20.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:20.275+0000 7f1ffebdf740 1 --2- 172.21.15.5:0/415016053 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55999db09760 0x55999db114a0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f1fe00c4400 tx=0x7f1fe00bb8c0 comp rx=0 tx=0).stop 2024-09-10T10:01:20.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:20.275+0000 7f1ffebdf740 1 -- 172.21.15.5:0/415016053 shutdown_connections 2024-09-10T10:01:20.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:20.275+0000 7f1ffebdf740 1 --2- 172.21.15.5:0/415016053 >> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] conn(0x55999db2c950 0x55999db2ed40 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:20.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:20.275+0000 7f1ffebdf740 1 --2- 172.21.15.5:0/415016053 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55999db14020 0x55999db119e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:20.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:20.275+0000 7f1ffebdf740 1 --2- 172.21.15.5:0/415016053 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55999db09760 0x55999db114a0 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:20.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:20.275+0000 7f1ffebdf740 1 --2- 172.21.15.5:0/415016053 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55999db08e40 0x55999db10f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:20.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:20.275+0000 7f1fdaa00640 0 client.29725 ms_handle_reset on v2:172.21.15.102:6832/2457007427 2024-09-10T10:01:20.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:20.275+0000 7f1ffebdf740 1 -- 172.21.15.5:0/415016053 >> 172.21.15.5:0/415016053 conn(0x55999db053d0 msgr2=0x55999db0b100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:20.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:20.276+0000 7f1ffebdf740 1 -- 172.21.15.5:0/415016053 shutdown_connections 2024-09-10T10:01:20.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:20.276+0000 7f1ffebdf740 1 -- 172.21.15.5:0/415016053 wait complete. 2024-09-10T10:01:20.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:20.276+0000 7f1ffebdf740 20 client.29725 trim_cache size 0 max 16384 2024-09-10T10:01:20.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:20.293+0000 7f1ffebdf740 1 Processor -- start 2024-09-10T10:01:20.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:20.293+0000 7f1ffebdf740 1 -- start start 2024-09-10T10:01:20.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:20.294+0000 7f1ffebdf740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55999db08e40 0x55999db057b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:20.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:20.294+0000 7f1ffebdf740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55999db09760 0x55999db34610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:20.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:20.294+0000 7f1ffebdf740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55999db14020 0x55999db34b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:20.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:20.294+0000 7f1ffebdf740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1ff41aca30 con 0x55999db09760 2024-09-10T10:01:20.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:20.294+0000 7f1ffebdf740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f1ff41ae570 con 0x55999db08e40 2024-09-10T10:01:20.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:20.294+0000 7f1ffebdf740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f1ff41ad100 con 0x55999db14020 2024-09-10T10:01:20.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:20.294+0000 7f1ff9600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55999db08e40 0x55999db057b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:20.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:20.294+0000 7f1ff9600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55999db08e40 0x55999db057b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:51606/0 (socket says 172.21.15.5:51606) 2024-09-10T10:01:20.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:20.294+0000 7f1ff9600640 1 -- 172.21.15.5:0/1856932292 learned_addr learned my addr 172.21.15.5:0/1856932292 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T10:01:20.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:20.294+0000 7f1ffa000640 1 --2- 172.21.15.5:0/1856932292 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55999db14020 0x55999db34b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:20.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:20.294+0000 7f1ff9600640 1 -- 172.21.15.5:0/1856932292 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55999db14020 msgr2=0x55999db34b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:01:20.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:20.294+0000 7f1ff9600640 1 --2- 172.21.15.5:0/1856932292 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55999db14020 0x55999db34b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:20.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:20.294+0000 7f1ff9600640 1 -- 172.21.15.5:0/1856932292 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55999db09760 msgr2=0x55999db34610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:01:20.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:20.294+0000 7f1ff9600640 1 --2- 172.21.15.5:0/1856932292 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55999db09760 0x55999db34610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:20.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:20.295+0000 7f1ff9600640 1 -- 172.21.15.5:0/1856932292 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55999db0ff30 con 0x55999db08e40 2024-09-10T10:01:20.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:20.295+0000 7f1ff9600640 1 --2- 172.21.15.5:0/1856932292 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55999db08e40 0x55999db057b0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x55999db2f580 tx=0x7f1fe8019000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fe5fb1a47f9771f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:20.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:20.295+0000 7f1fe5c00640 1 -- 172.21.15.5:0/1856932292 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fe802d070 con 0x55999db08e40 2024-09-10T10:01:20.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:20.295+0000 7f1fe5c00640 1 -- 172.21.15.5:0/1856932292 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1fe8032460 con 0x55999db08e40 2024-09-10T10:01:20.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:20.295+0000 7f1fe5c00640 1 -- 172.21.15.5:0/1856932292 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fe8029680 con 0x55999db08e40 2024-09-10T10:01:20.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:20.296+0000 7f1ffebdf740 1 -- 172.21.15.5:0/1856932292 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55999db08e40 msgr2=0x55999db057b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:20.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:20.296+0000 7f1ffebdf740 1 --2- 172.21.15.5:0/1856932292 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55999db08e40 0x55999db057b0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x55999db2f580 tx=0x7f1fe8019000 comp rx=0 tx=0).stop 2024-09-10T10:01:20.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:20.296+0000 7f1ffebdf740 1 -- 172.21.15.5:0/1856932292 shutdown_connections 2024-09-10T10:01:20.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:20.296+0000 7f1ffebdf740 1 --2- 172.21.15.5:0/1856932292 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55999db14020 0x55999db34b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:20.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:20.296+0000 7f1ffebdf740 1 --2- 172.21.15.5:0/1856932292 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55999db09760 0x55999db34610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:20.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:20.296+0000 7f1ffebdf740 1 --2- 172.21.15.5:0/1856932292 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55999db08e40 0x55999db057b0 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:20.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:20.296+0000 7f1ffebdf740 1 -- 172.21.15.5:0/1856932292 >> 172.21.15.5:0/1856932292 conn(0x55999db053d0 msgr2=0x55999db0b100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:20.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:20.296+0000 7f1ffebdf740 1 -- 172.21.15.5:0/1856932292 shutdown_connections 2024-09-10T10:01:20.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:20.297+0000 7f1ffebdf740 1 -- 172.21.15.5:0/1856932292 wait complete. 2024-09-10T10:01:20.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:20.297+0000 7f1ffebdf740 1 -- start start 2024-09-10T10:01:20.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:20.297+0000 7f1ffebdf740 1 Processor -- start 2024-09-10T10:01:20.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:20.298+0000 7f1ffebdf740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55999db08e40 0x55999db24880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:20.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:20.298+0000 7f1ffebdf740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55999db09760 0x55999db24dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:20.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:20.298+0000 7f1ffebdf740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55999db14020 0x55999db21850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:20.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:20.298+0000 7f1ffebdf740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1ff41adea0 con 0x55999db08e40 2024-09-10T10:01:20.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:20.298+0000 7f1ffebdf740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f1ff41ad7d0 con 0x55999db09760 2024-09-10T10:01:20.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:20.298+0000 7f1ffebdf740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f1ff41aaa10 con 0x55999db14020 2024-09-10T10:01:20.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:20.298+0000 7f1ffebdf740 10 client.0 _ll_register_callbacks cb 0x7f1fe0005b90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:01:20.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:20.298+0000 7f1ff8c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55999db09760 0x55999db24dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:20.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:20.298+0000 7f1ff9600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55999db08e40 0x55999db24880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:20.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:20.298+0000 7f1ff8c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55999db09760 0x55999db24dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:51616/0 (socket says 172.21.15.5:51616) 2024-09-10T10:01:20.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:20.298+0000 7f1ffa000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55999db14020 0x55999db21850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:20.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:20.298+0000 7f1ff8c00640 1 -- 172.21.15.5:0/2603924888 learned_addr learned my addr 172.21.15.5:0/2603924888 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T10:01:20.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:20.299+0000 7f1ff9600640 1 -- 172.21.15.5:0/2603924888 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55999db14020 msgr2=0x55999db21850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:01:20.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:20.299+0000 7f1ff9600640 1 --2- 172.21.15.5:0/2603924888 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55999db14020 0x55999db21850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:20.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:20.299+0000 7f1ff9600640 1 -- 172.21.15.5:0/2603924888 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55999db09760 msgr2=0x55999db24dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:01:20.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:20.299+0000 7f1ff9600640 1 --2- 172.21.15.5:0/2603924888 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55999db09760 0x55999db24dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:20.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:20.299+0000 7f1ff9600640 1 -- 172.21.15.5:0/2603924888 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55999db21d90 con 0x55999db08e40 2024-09-10T10:01:20.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:20.299+0000 7f1ff8c00640 1 --2- 172.21.15.5:0/2603924888 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55999db09760 0x55999db24dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:01:20.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:20.299+0000 7f1ff9600640 1 --2- 172.21.15.5:0/2603924888 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55999db08e40 0x55999db24880 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f1fe8012ea0 tx=0x7f1fe8032d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17f50f30539c8fbf server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:20.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:20.299+0000 7f1fdaa00640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-10T10:01:20.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:20.300+0000 7f1fdaa00640 1 -- 172.21.15.5:0/2603924888 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fe802d050 con 0x55999db08e40 2024-09-10T10:01:20.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:20.300+0000 7f1fdaa00640 1 -- 172.21.15.5:0/2603924888 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f1fe8032810 con 0x55999db08e40 2024-09-10T10:01:20.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:20.300+0000 7f1ffebdf740 1 -- 172.21.15.5:0/2603924888 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1fe8032460 con 0x55999db08e40 2024-09-10T10:01:20.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:20.300+0000 7f1fdaa00640 1 -- 172.21.15.5:0/2603924888 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f1fe8032a10 con 0x55999db08e40 2024-09-10T10:01:20.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:20.300+0000 7f1ffebdf740 10 client.19680 fetch_fsmap learned FSMap version 42 2024-09-10T10:01:20.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:20.300+0000 7f1ffebdf740 1 -- 172.21.15.5:0/2603924888 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55999db97db0 con 0x55999db08e40 2024-09-10T10:01:20.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:20.301+0000 7f1fdaa00640 1 -- 172.21.15.5:0/2603924888 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1fe8032460 con 0x55999db08e40 2024-09-10T10:01:20.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:20.301+0000 7f1fdaa00640 1 -- 172.21.15.5:0/2603924888 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 42) ==== 8370+0+0 (secure 0 0 0) 0x7f1fe80047e0 con 0x55999db08e40 2024-09-10T10:01:20.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:20.301+0000 7f1ffebdf740 10 client.19680 fetch_fsmap finished waiting for FSMap version 42 2024-09-10T10:01:20.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:20.301+0000 7f1ffebdf740 10 client.19680 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-09-10T10:01:20.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:20.301+0000 7f1ffebdf740 20 client.19680 populate_metadata read hostname 'smithi005' 2024-09-10T10:01:20.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:20.301+0000 7f1ffebdf740 1 --2- 172.21.15.5:0/2603924888 >> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] conn(0x55999db983c0 0x55999db9a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:20.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:20.301+0000 7f1ffebdf740 4 client.19680 mds_command: new command op to 24443 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T10:01:20.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:20.302+0000 7f1ffebdf740 1 -- 172.21.15.5:0/2603924888 --> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55999da76e60 con 0x55999db983c0 2024-09-10T10:01:20.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:20.302+0000 7f1ff8c00640 1 --2- 172.21.15.5:0/2603924888 >> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] conn(0x55999db983c0 0x55999db9a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:20.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:20.303+0000 7f1ff8c00640 1 --2- 172.21.15.5:0/2603924888 >> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] conn(0x55999db983c0 0x55999db9a7b0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=2e7ace4a47107403 server_cookie=647470b09f5493ce in_seq=0 out_seq=0 2024-09-10T10:01:20.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:20.303+0000 7f1fdaa00640 10 client.19680 ms_handle_connect on v2:172.21.15.102:6832/2457007427 2024-09-10T10:01:20.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:20.303+0000 7f1fdaa00640 1 -- 172.21.15.5:0/2603924888 <== mds.4 v2:172.21.15.102:6832/2457007427 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55999da76e60 con 0x55999db983c0 2024-09-10T10:01:20.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:20.303+0000 7f1fdaa00640 10 client.19680 handle_command_reply: tid=0 2024-09-10T10:01:20.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:20.303+0000 7f1ffebdf740 1 client.19680 shutdown 2024-09-10T10:01:20.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:20.303+0000 7f1ffebdf740 1 -- 172.21.15.5:0/2603924888 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55999db08e40 msgr2=0x55999db24880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:20.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:20.303+0000 7f1ffebdf740 1 --2- 172.21.15.5:0/2603924888 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55999db08e40 0x55999db24880 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f1fe8012ea0 tx=0x7f1fe8032d20 comp rx=0 tx=0).stop 2024-09-10T10:01:20.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:20.303+0000 7f1ffebdf740 1 -- 172.21.15.5:0/2603924888 shutdown_connections 2024-09-10T10:01:20.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:20.303+0000 7f1ffebdf740 1 --2- 172.21.15.5:0/2603924888 >> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] conn(0x55999db983c0 0x55999db9a7b0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:20.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:20.303+0000 7f1ffebdf740 1 --2- 172.21.15.5:0/2603924888 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55999db14020 0x55999db21850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:20.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:20.303+0000 7f1ffebdf740 1 --2- 172.21.15.5:0/2603924888 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55999db09760 0x55999db24dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:20.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:20.303+0000 7f1ffebdf740 1 --2- 172.21.15.5:0/2603924888 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55999db08e40 0x55999db24880 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:20.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:20.303+0000 7f1ffebdf740 1 -- 172.21.15.5:0/2603924888 >> 172.21.15.5:0/2603924888 conn(0x55999db053d0 msgr2=0x55999db0a8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:20.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:20.303+0000 7f1fdaa00640 0 client.19680 ms_handle_reset on v2:172.21.15.102:6832/2457007427 2024-09-10T10:01:20.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:20.304+0000 7f1ffebdf740 1 -- 172.21.15.5:0/2603924888 shutdown_connections 2024-09-10T10:01:20.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:20.304+0000 7f1ffebdf740 1 -- 172.21.15.5:0/2603924888 wait complete. 2024-09-10T10:01:20.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:20.304+0000 7f1ffebdf740 20 client.19680 trim_cache size 0 max 16384 2024-09-10T10:01:20.307 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-10T10:01:20.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:20.305+0000 7f1ffaa00640 1 -- 172.21.15.5:0/2069011210 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f1fdc0a2a90 msgr2=0x7f1fdc0a4f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:20.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:20.305+0000 7f1ffaa00640 1 --2- 172.21.15.5:0/2069011210 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f1fdc0a2a90 0x7f1fdc0a4f50 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x55999db06230 tx=0x7f1fe8005f90 comp rx=0 tx=0).stop 2024-09-10T10:01:20.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:20.305+0000 7f1ffaa00640 1 -- 172.21.15.5:0/2069011210 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1ff4074e30 msgr2=0x7f1ff419aab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:20.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:20.305+0000 7f1ffaa00640 1 --2- 172.21.15.5:0/2069011210 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1ff4074e30 0x7f1ff419aab0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f1fe0006010 tx=0x7f1fe0004040 comp rx=0 tx=0).stop 2024-09-10T10:01:20.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:20.305+0000 7f1ffaa00640 1 -- 172.21.15.5:0/2069011210 shutdown_connections 2024-09-10T10:01:20.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:20.305+0000 7f1ffaa00640 1 --2- 172.21.15.5:0/2069011210 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f1fdc0a2a90 0x7f1fdc0a4f50 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:20.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:20.305+0000 7f1ffaa00640 1 --2- 172.21.15.5:0/2069011210 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1ff410e280 0x7f1ff419ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:20.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:20.305+0000 7f1ffaa00640 1 --2- 172.21.15.5:0/2069011210 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1ff4074e30 0x7f1ff419aab0 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:20.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:20.305+0000 7f1ffaa00640 1 --2- 172.21.15.5:0/2069011210 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1ff4074510 0x7f1ff419a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:20.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:20.305+0000 7f1ffaa00640 1 -- 172.21.15.5:0/2069011210 >> 172.21.15.5:0/2069011210 conn(0x7f1ff40fd740 msgr2=0x7f1ff40fef00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:20.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:20.306+0000 7f1ffaa00640 1 -- 172.21.15.5:0/2069011210 shutdown_connections 2024-09-10T10:01:20.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:20.306+0000 7f1ffaa00640 1 -- 172.21.15.5:0/2069011210 wait complete. 2024-09-10T10:01:21.041 INFO:tasks.workunit.client.0.smithi005.stdout: 10 140922 13.09 MB/sec execute 419 sec latency 56.658 ms 2024-09-10T10:01:21.307 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:01:21 smithi005 ceph-mon[30350]: pgmap v1083: 129 pgs: 129 active+clean; 696 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 2.8 MiB/s wr, 370 op/s 2024-09-10T10:01:21.593 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-10T10:01:21.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:01:21 smithi159 ceph-mon[35513]: pgmap v1083: 129 pgs: 129 active+clean; 696 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 2.8 MiB/s wr, 370 op/s 2024-09-10T10:01:21.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:01:21 smithi102 ceph-mon[35442]: pgmap v1083: 129 pgs: 129 active+clean; 696 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 2.8 MiB/s wr, 370 op/s 2024-09-10T10:01:22.041 INFO:tasks.workunit.client.0.smithi005.stdout: 10 141527 13.14 MB/sec execute 420 sec latency 92.296 ms 2024-09-10T10:01:22.259 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:01:22.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:22.886+0000 7f4bf9400640 1 -- 172.21.15.5:0/384108411 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4bf4101fc0 msgr2=0x7f4bf410d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:22.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:22.886+0000 7f4bf9400640 1 --2- 172.21.15.5:0/384108411 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4bf4101fc0 0x7f4bf410d940 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f4bec009860 tx=0x7f4bec02a340 comp rx=0 tx=0).stop 2024-09-10T10:01:22.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:22.886+0000 7f4bf9400640 1 -- 172.21.15.5:0/384108411 shutdown_connections 2024-09-10T10:01:22.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:22.886+0000 7f4bf9400640 1 --2- 172.21.15.5:0/384108411 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4bf4101fc0 0x7f4bf410d940 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:22.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:22.886+0000 7f4bf9400640 1 --2- 172.21.15.5:0/384108411 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4bf4101620 0x7f4bf4101a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:22.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:22.886+0000 7f4bf9400640 1 --2- 172.21.15.5:0/384108411 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4bf4105aa0 0x7f4bf4105e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:22.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:22.886+0000 7f4bf9400640 1 -- 172.21.15.5:0/384108411 >> 172.21.15.5:0/384108411 conn(0x7f4bf40fba90 msgr2=0x7f4bf40fdeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:22.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:22.886+0000 7f4bf9400640 1 -- 172.21.15.5:0/384108411 shutdown_connections 2024-09-10T10:01:22.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:22.886+0000 7f4bf9400640 1 -- 172.21.15.5:0/384108411 wait complete. 2024-09-10T10:01:22.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:22.888+0000 7f4bf9400640 1 Processor -- start 2024-09-10T10:01:22.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:22.888+0000 7f4bf9400640 1 -- start start 2024-09-10T10:01:22.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:22.888+0000 7f4bf9400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4bf4101620 0x7f4bf41079c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:22.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:22.888+0000 7f4bf9400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4bf4101fc0 0x7f4bf4107f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:22.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:22.888+0000 7f4bf9400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4bf4105aa0 0x7f4bf4108440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:22.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:22.889+0000 7f4bf9400640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f4bf4110ec0 con 0x7f4bf4101fc0 2024-09-10T10:01:22.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:22.889+0000 7f4bf9400640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f4bf4110d40 con 0x7f4bf4101620 2024-09-10T10:01:22.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:22.889+0000 7f4bf9400640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f4bf4111040 con 0x7f4bf4105aa0 2024-09-10T10:01:22.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:22.889+0000 7f4bf2800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4bf4101620 0x7f4bf41079c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:22.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:22.889+0000 7f4bf2800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4bf4101620 0x7f4bf41079c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:38270/0 (socket says 172.21.15.5:38270) 2024-09-10T10:01:22.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:22.889+0000 7f4bf2800640 1 -- 172.21.15.5:0/1715076333 learned_addr learned my addr 172.21.15.5:0/1715076333 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T10:01:22.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:22.889+0000 7f4bf2800640 1 -- 172.21.15.5:0/1715076333 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4bf4105aa0 msgr2=0x7f4bf4108440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:22.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:22.889+0000 7f4bf2800640 1 --2- 172.21.15.5:0/1715076333 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4bf4105aa0 0x7f4bf4108440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:22.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:22.889+0000 7f4bf2800640 1 -- 172.21.15.5:0/1715076333 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4bf4101fc0 msgr2=0x7f4bf4107f00 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T10:01:22.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:22.890+0000 7f4be9e00640 1 --2- 172.21.15.5:0/1715076333 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4bf4101fc0 0x7f4bf4107f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:22.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:22.890+0000 7f4bf2800640 1 --2- 172.21.15.5:0/1715076333 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4bf4101fc0 0x7f4bf4107f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:22.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:22.890+0000 7f4bf2800640 1 -- 172.21.15.5:0/1715076333 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4bf41aac90 con 0x7f4bf4101620 2024-09-10T10:01:22.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:22.890+0000 7f4be9e00640 1 --2- 172.21.15.5:0/1715076333 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4bf4101fc0 0x7f4bf4107f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:01:22.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:22.890+0000 7f4bf2800640 1 --2- 172.21.15.5:0/1715076333 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4bf4101620 0x7f4bf41079c0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f4be400ec20 tx=0x7f4be400c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:22.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:22.891+0000 7f4bebe00640 1 -- 172.21.15.5:0/1715076333 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4be4010630 con 0x7f4bf4101620 2024-09-10T10:01:22.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:22.891+0000 7f4bebe00640 1 -- 172.21.15.5:0/1715076333 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4be4010cb0 con 0x7f4bf4101620 2024-09-10T10:01:22.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:22.891+0000 7f4bf9400640 1 -- 172.21.15.5:0/1715076333 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4bf41aaf80 con 0x7f4bf4101620 2024-09-10T10:01:22.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:22.891+0000 7f4bf9400640 1 -- 172.21.15.5:0/1715076333 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4bf41ab4b0 con 0x7f4bf4101620 2024-09-10T10:01:22.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:22.891+0000 7f4bebe00640 1 -- 172.21.15.5:0/1715076333 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4be4018bf0 con 0x7f4bf4101620 2024-09-10T10:01:22.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:22.896+0000 7f4bebe00640 1 -- 172.21.15.5:0/1715076333 <== mon.2 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f4be40107d0 con 0x7f4bf4101620 2024-09-10T10:01:22.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:22.896+0000 7f4bfbebe740 1 Processor -- start 2024-09-10T10:01:22.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:22.896+0000 7f4bfbebe740 1 -- start start 2024-09-10T10:01:22.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:22.897+0000 7f4bfbebe740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d2f8d72c70 0x55d2f8d73050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:22.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:22.897+0000 7f4bfbebe740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d2f8d73590 0x55d2f8d7d910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:22.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:22.897+0000 7f4bfbebe740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d2f8d7de50 0x55d2f8d80240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:22.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:22.897+0000 7f4bfbebe740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55d2f8cd93e0 con 0x55d2f8d7de50 2024-09-10T10:01:22.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:22.897+0000 7f4bfbebe740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55d2f8c80d80 con 0x55d2f8d73590 2024-09-10T10:01:22.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:22.897+0000 7f4bfbebe740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55d2f8cd53a0 con 0x55d2f8d72c70 2024-09-10T10:01:22.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:22.897+0000 7f4bf3200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d2f8d72c70 0x55d2f8d73050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:22.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:22.897+0000 7f4bf3200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d2f8d72c70 0x55d2f8d73050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:47660/0 (socket says 172.21.15.5:47660) 2024-09-10T10:01:22.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:22.897+0000 7f4bf3200640 1 -- 172.21.15.5:0/3743048409 learned_addr learned my addr 172.21.15.5:0/3743048409 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T10:01:22.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:22.897+0000 7f4bf2800640 1 --2- 172.21.15.5:0/3743048409 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d2f8d73590 0x55d2f8d7d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:22.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:22.897+0000 7f4bf3200640 1 -- 172.21.15.5:0/3743048409 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d2f8d73590 msgr2=0x55d2f8d7d910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:01:22.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:22.897+0000 7f4bebe00640 1 --2- 172.21.15.5:0/1715076333 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f4bd80a2c00 0x7f4bd80a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:22.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:22.897+0000 7f4bf3200640 1 --2- 172.21.15.5:0/3743048409 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d2f8d73590 0x55d2f8d7d910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:22.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:22.897+0000 7f4bf3200640 1 -- 172.21.15.5:0/3743048409 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d2f8d7de50 msgr2=0x55d2f8d80240 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T10:01:22.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:22.898+0000 7f4be9e00640 1 --2- 172.21.15.5:0/3743048409 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d2f8d7de50 0x55d2f8d80240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:22.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:22.898+0000 7f4bebe00640 1 -- 172.21.15.5:0/1715076333 <== mon.2 v2:172.21.15.102:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f4be40cfa20 con 0x7f4bf4101620 2024-09-10T10:01:22.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:22.898+0000 7f4bf3200640 1 --2- 172.21.15.5:0/3743048409 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d2f8d7de50 0x55d2f8d80240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:22.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:22.898+0000 7f4bf3200640 1 -- 172.21.15.5:0/3743048409 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d2f8c9fae0 con 0x55d2f8d72c70 2024-09-10T10:01:22.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:22.898+0000 7f4bf2800640 1 --2- 172.21.15.5:0/3743048409 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d2f8d73590 0x55d2f8d7d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:01:22.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:22.898+0000 7f4be9e00640 1 --2- 172.21.15.5:0/3743048409 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d2f8d7de50 0x55d2f8d80240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:01:22.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:22.898+0000 7f4bf3200640 1 --2- 172.21.15.5:0/1715076333 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f4bd80a2c00 0x7f4bd80a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:22.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:22.899+0000 7f4bf3200640 1 --2- 172.21.15.5:0/3743048409 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d2f8d72c70 0x55d2f8d73050 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f4bec009780 tx=0x7f4bec005d20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fc2b36df0195ce42 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:22.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:22.899+0000 7f4be3400640 1 -- 172.21.15.5:0/3743048409 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bec037040 con 0x55d2f8d72c70 2024-09-10T10:01:22.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:22.899+0000 7f4be3400640 1 -- 172.21.15.5:0/3743048409 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4bec004380 con 0x55d2f8d72c70 2024-09-10T10:01:22.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:22.899+0000 7f4be3400640 1 -- 172.21.15.5:0/3743048409 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bec047650 con 0x55d2f8d72c70 2024-09-10T10:01:22.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:22.900+0000 7f4bfbebe740 1 -- 172.21.15.5:0/3743048409 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d2f8d72c70 msgr2=0x55d2f8d73050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:22.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:22.900+0000 7f4bfbebe740 1 --2- 172.21.15.5:0/3743048409 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d2f8d72c70 0x55d2f8d73050 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f4bec009780 tx=0x7f4bec005d20 comp rx=0 tx=0).stop 2024-09-10T10:01:22.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:22.900+0000 7f4bf3200640 1 --2- 172.21.15.5:0/1715076333 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f4bd80a2c00 0x7f4bd80a50c0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f4bec040000 tx=0x55d2f8d73c60 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:22.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:22.900+0000 7f4bfbebe740 1 -- 172.21.15.5:0/3743048409 shutdown_connections 2024-09-10T10:01:22.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:22.900+0000 7f4bfbebe740 1 --2- 172.21.15.5:0/3743048409 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d2f8d7de50 0x55d2f8d80240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:22.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:22.900+0000 7f4bfbebe740 1 --2- 172.21.15.5:0/3743048409 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d2f8d73590 0x55d2f8d7d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:22.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:22.900+0000 7f4bfbebe740 1 --2- 172.21.15.5:0/3743048409 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d2f8d72c70 0x55d2f8d73050 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:22.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:22.900+0000 7f4bfbebe740 1 -- 172.21.15.5:0/3743048409 >> 172.21.15.5:0/3743048409 conn(0x55d2f8d6f200 msgr2=0x55d2f8d6f5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:22.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:22.900+0000 7f4bfbebe740 1 -- 172.21.15.5:0/3743048409 shutdown_connections 2024-09-10T10:01:22.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:22.900+0000 7f4bfbebe740 1 -- 172.21.15.5:0/3743048409 wait complete. 2024-09-10T10:01:22.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:22.901+0000 7f4bfbebe740 1 -- start start 2024-09-10T10:01:22.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:22.901+0000 7f4bfbebe740 1 Processor -- start 2024-09-10T10:01:22.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:22.901+0000 7f4bfbebe740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d2f8d72c70 0x55d2f8d7ad20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:22.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:22.901+0000 7f4bfbebe740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d2f8d73590 0x55d2f8d7b260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:22.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:22.901+0000 7f4bfbebe740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d2f8d7de50 0x55d2f8d7b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:22.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:22.901+0000 7f4bfbebe740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55d2f8d80a20 con 0x55d2f8d7de50 2024-09-10T10:01:22.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:22.901+0000 7f4bfbebe740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55d2f8c940f0 con 0x55d2f8d73590 2024-09-10T10:01:22.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:22.901+0000 7f4bfbebe740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55d2f8d80ba0 con 0x55d2f8d72c70 2024-09-10T10:01:22.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:22.902+0000 7f4bfbebe740 10 client.0 _ll_register_callbacks cb 0x55d2f8c7bbd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:01:22.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:22.902+0000 7f4bf2800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d2f8d72c70 0x55d2f8d7ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:22.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:22.902+0000 7f4bf3200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d2f8d7de50 0x55d2f8d7b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:22.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:22.902+0000 7f4bf2800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d2f8d72c70 0x55d2f8d7ad20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:47666/0 (socket says 172.21.15.5:47666) 2024-09-10T10:01:22.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:22.902+0000 7f4bf2800640 1 -- 172.21.15.5:0/3881008781 learned_addr learned my addr 172.21.15.5:0/3881008781 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T10:01:22.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:22.902+0000 7f4be9e00640 1 --2- 172.21.15.5:0/3881008781 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d2f8d73590 0x55d2f8d7b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:22.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:22.902+0000 7f4bf3200640 1 -- 172.21.15.5:0/3881008781 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d2f8d72c70 msgr2=0x55d2f8d7ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:01:22.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:22.902+0000 7f4bf3200640 1 --2- 172.21.15.5:0/3881008781 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d2f8d72c70 0x55d2f8d7ad20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:22.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:22.902+0000 7f4bf3200640 1 -- 172.21.15.5:0/3881008781 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d2f8d73590 msgr2=0x55d2f8d7b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:01:22.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:22.902+0000 7f4bf3200640 1 --2- 172.21.15.5:0/3881008781 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d2f8d73590 0x55d2f8d7b260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:22.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:22.902+0000 7f4bf3200640 1 -- 172.21.15.5:0/3881008781 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d2f8a4a3c0 con 0x55d2f8d7de50 2024-09-10T10:01:22.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:22.902+0000 7f4bf2800640 1 --2- 172.21.15.5:0/3881008781 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d2f8d72c70 0x55d2f8d7ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:01:22.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:22.903+0000 7f4bf3200640 1 --2- 172.21.15.5:0/3881008781 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d2f8d7de50 0x55d2f8d7b7a0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f4bec0057c0 tx=0x7f4bec005ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5939be92d737b19d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:22.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:22.903+0000 7f4bd7e00640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-10T10:01:22.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:22.903+0000 7f4bd7e00640 1 -- 172.21.15.5:0/3881008781 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bec037030 con 0x55d2f8d7de50 2024-09-10T10:01:22.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:22.903+0000 7f4bd7e00640 1 -- 172.21.15.5:0/3881008781 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f4bec003b10 con 0x55d2f8d7de50 2024-09-10T10:01:22.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:22.903+0000 7f4bfbebe740 1 -- 172.21.15.5:0/3881008781 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4bec004380 con 0x55d2f8d7de50 2024-09-10T10:01:22.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:22.904+0000 7f4bd7e00640 1 -- 172.21.15.5:0/3881008781 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f4bec05d050 con 0x55d2f8d7de50 2024-09-10T10:01:22.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:22.904+0000 7f4bfbebe740 10 client.19686 fetch_fsmap learned FSMap version 42 2024-09-10T10:01:22.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:22.904+0000 7f4bfbebe740 1 -- 172.21.15.5:0/3881008781 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55d2f8d8ca40 con 0x55d2f8d7de50 2024-09-10T10:01:22.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:22.904+0000 7f4bd7e00640 1 -- 172.21.15.5:0/3881008781 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4bec004380 con 0x55d2f8d7de50 2024-09-10T10:01:22.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:22.904+0000 7f4bd7e00640 1 -- 172.21.15.5:0/3881008781 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 42) ==== 8370+0+0 (secure 0 0 0) 0x7f4bec04e070 con 0x55d2f8d7de50 2024-09-10T10:01:22.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:22.904+0000 7f4bfbebe740 10 client.19686 fetch_fsmap finished waiting for FSMap version 42 2024-09-10T10:01:22.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:22.904+0000 7f4bfbebe740 10 client.19686 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-09-10T10:01:22.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:22.904+0000 7f4bfbebe740 20 client.19686 populate_metadata read hostname 'smithi005' 2024-09-10T10:01:22.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:22.904+0000 7f4bfbebe740 1 --2- 172.21.15.5:0/3881008781 >> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] conn(0x55d2f8d8dd90 0x55d2f8d90180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:22.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:22.904+0000 7f4bfbebe740 4 client.19686 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:01:22.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:22.904+0000 7f4bfbebe740 1 -- 172.21.15.5:0/3881008781 --> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d2f8ce6950 con 0x55d2f8d8dd90 2024-09-10T10:01:22.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:22.905+0000 7f4bf2800640 1 --2- 172.21.15.5:0/3881008781 >> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] conn(0x55d2f8d8dd90 0x55d2f8d90180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:22.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:22.905+0000 7f4bf2800640 1 --2- 172.21.15.5:0/3881008781 >> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] conn(0x55d2f8d8dd90 0x55d2f8d90180 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=9a126cdbf70a752d server_cookie=e8eff0821d71cca6 in_seq=0 out_seq=0 2024-09-10T10:01:22.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:22.905+0000 7f4bd7e00640 10 client.19686 ms_handle_connect on v2:172.21.15.102:6836/2689776764 2024-09-10T10:01:22.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:22.906+0000 7f4bd7e00640 1 -- 172.21.15.5:0/3881008781 <== mds.3 v2:172.21.15.102:6836/2689776764 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d2f8ce6950 con 0x55d2f8d8dd90 2024-09-10T10:01:22.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:22.906+0000 7f4bd7e00640 10 client.19686 handle_command_reply: tid=0 2024-09-10T10:01:22.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:22.906+0000 7f4bfbebe740 1 client.19686 shutdown 2024-09-10T10:01:22.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:22.906+0000 7f4bfbebe740 1 -- 172.21.15.5:0/3881008781 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d2f8d7de50 msgr2=0x55d2f8d7b7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:22.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:22.906+0000 7f4bfbebe740 1 --2- 172.21.15.5:0/3881008781 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d2f8d7de50 0x55d2f8d7b7a0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f4bec0057c0 tx=0x7f4bec005ad0 comp rx=0 tx=0).stop 2024-09-10T10:01:22.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:22.906+0000 7f4bfbebe740 1 -- 172.21.15.5:0/3881008781 shutdown_connections 2024-09-10T10:01:22.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:22.906+0000 7f4bfbebe740 1 --2- 172.21.15.5:0/3881008781 >> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] conn(0x55d2f8d8dd90 0x55d2f8d90180 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:22.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:22.907+0000 7f4bfbebe740 1 --2- 172.21.15.5:0/3881008781 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d2f8d7de50 0x55d2f8d7b7a0 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:22.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:22.907+0000 7f4bd7e00640 0 client.19686 ms_handle_reset on v2:172.21.15.102:6836/2689776764 2024-09-10T10:01:22.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:22.907+0000 7f4bfbebe740 1 --2- 172.21.15.5:0/3881008781 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d2f8d73590 0x55d2f8d7b260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:22.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:22.907+0000 7f4bfbebe740 1 --2- 172.21.15.5:0/3881008781 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d2f8d72c70 0x55d2f8d7ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:22.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:22.907+0000 7f4bfbebe740 1 -- 172.21.15.5:0/3881008781 >> 172.21.15.5:0/3881008781 conn(0x55d2f8d6f200 msgr2=0x55d2f8d74f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:22.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:22.907+0000 7f4bfbebe740 1 -- 172.21.15.5:0/3881008781 shutdown_connections 2024-09-10T10:01:22.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:22.907+0000 7f4bfbebe740 1 -- 172.21.15.5:0/3881008781 wait complete. 2024-09-10T10:01:22.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:22.907+0000 7f4bfbebe740 20 client.19686 trim_cache size 0 max 16384 2024-09-10T10:01:22.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:22.921+0000 7f4bfbebe740 1 Processor -- start 2024-09-10T10:01:22.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:22.921+0000 7f4bfbebe740 1 -- start start 2024-09-10T10:01:22.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:22.921+0000 7f4bfbebe740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d2f8d72c70 0x55d2f8d7a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:22.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:22.921+0000 7f4bfbebe740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d2f8d73590 0x55d2f8d95870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:22.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:22.921+0000 7f4bfbebe740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d2f8d7de50 0x55d2f8d95db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:22.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:22.921+0000 7f4bfbebe740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f4bd000fd20 con 0x55d2f8d7de50 2024-09-10T10:01:22.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:22.921+0000 7f4bfbebe740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f4bd0011860 con 0x55d2f8d72c70 2024-09-10T10:01:22.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:22.921+0000 7f4bfbebe740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f4bd00103f0 con 0x55d2f8d73590 2024-09-10T10:01:22.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:22.921+0000 7f4bf3200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d2f8d7de50 0x55d2f8d95db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:22.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:22.921+0000 7f4be9e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d2f8d73590 0x55d2f8d95870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:22.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:22.921+0000 7f4bf2800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d2f8d72c70 0x55d2f8d7a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:22.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:22.921+0000 7f4bf3200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d2f8d7de50 0x55d2f8d95db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59534/0 (socket says 172.21.15.5:59534) 2024-09-10T10:01:22.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:22.921+0000 7f4bf3200640 1 -- 172.21.15.5:0/2930060142 learned_addr learned my addr 172.21.15.5:0/2930060142 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T10:01:22.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:22.921+0000 7f4be9e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d2f8d73590 0x55d2f8d95870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:47670/0 (socket says 172.21.15.5:47670) 2024-09-10T10:01:22.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:22.922+0000 7f4bf2800640 1 -- 172.21.15.5:0/2930060142 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d2f8d73590 msgr2=0x55d2f8d95870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:01:22.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:22.922+0000 7f4bf2800640 1 --2- 172.21.15.5:0/2930060142 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d2f8d73590 0x55d2f8d95870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:22.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:22.922+0000 7f4bf2800640 1 -- 172.21.15.5:0/2930060142 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d2f8d7de50 msgr2=0x55d2f8d95db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:01:22.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:22.922+0000 7f4bf2800640 1 --2- 172.21.15.5:0/2930060142 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d2f8d7de50 0x55d2f8d95db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:22.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:22.922+0000 7f4bf2800640 1 -- 172.21.15.5:0/2930060142 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d2f8d6a770 con 0x55d2f8d72c70 2024-09-10T10:01:22.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:22.922+0000 7f4bf3200640 1 --2- 172.21.15.5:0/2930060142 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d2f8d7de50 0x55d2f8d95db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:01:22.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:22.922+0000 7f4bf2800640 1 --2- 172.21.15.5:0/2930060142 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d2f8d72c70 0x55d2f8d7a5e0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f4be400cba0 tx=0x7f4be40bc040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f014a99af8c72034 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:22.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:22.922+0000 7f4be2a00640 1 -- 172.21.15.5:0/2930060142 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4be4010e60 con 0x55d2f8d72c70 2024-09-10T10:01:22.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:22.922+0000 7f4be2a00640 1 -- 172.21.15.5:0/2930060142 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4be40a0240 con 0x55d2f8d72c70 2024-09-10T10:01:22.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:22.922+0000 7f4be2a00640 1 -- 172.21.15.5:0/2930060142 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4be409b2e0 con 0x55d2f8d72c70 2024-09-10T10:01:22.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:22.923+0000 7f4bfbebe740 1 -- 172.21.15.5:0/2930060142 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d2f8d72c70 msgr2=0x55d2f8d7a5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:22.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:22.923+0000 7f4bfbebe740 1 --2- 172.21.15.5:0/2930060142 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d2f8d72c70 0x55d2f8d7a5e0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f4be400cba0 tx=0x7f4be40bc040 comp rx=0 tx=0).stop 2024-09-10T10:01:22.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:22.923+0000 7f4bfbebe740 1 -- 172.21.15.5:0/2930060142 shutdown_connections 2024-09-10T10:01:22.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:22.923+0000 7f4bfbebe740 1 --2- 172.21.15.5:0/2930060142 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d2f8d7de50 0x55d2f8d95db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:22.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:22.923+0000 7f4bfbebe740 1 --2- 172.21.15.5:0/2930060142 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d2f8d73590 0x55d2f8d95870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:22.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:22.923+0000 7f4bfbebe740 1 --2- 172.21.15.5:0/2930060142 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d2f8d72c70 0x55d2f8d7a5e0 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:22.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:22.923+0000 7f4bfbebe740 1 -- 172.21.15.5:0/2930060142 >> 172.21.15.5:0/2930060142 conn(0x55d2f8d6f200 msgr2=0x55d2f8d6f5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:22.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:22.923+0000 7f4bfbebe740 1 -- 172.21.15.5:0/2930060142 shutdown_connections 2024-09-10T10:01:22.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:22.923+0000 7f4bfbebe740 1 -- 172.21.15.5:0/2930060142 wait complete. 2024-09-10T10:01:22.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:22.923+0000 7f4bfbebe740 1 -- start start 2024-09-10T10:01:22.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:22.924+0000 7f4bfbebe740 1 Processor -- start 2024-09-10T10:01:22.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:22.924+0000 7f4bfbebe740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d2f8d72c70 0x55d2f8d861c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:22.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:22.924+0000 7f4bfbebe740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d2f8d73590 0x55d2f8d86700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:22.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:22.924+0000 7f4bfbebe740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d2f8d7de50 0x55d2f8d81610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:22.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:22.924+0000 7f4bfbebe740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f4bd0011190 con 0x55d2f8d72c70 2024-09-10T10:01:22.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:22.924+0000 7f4bfbebe740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f4bd0010ac0 con 0x55d2f8d73590 2024-09-10T10:01:22.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:22.924+0000 7f4bfbebe740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f4bd000dd00 con 0x55d2f8d7de50 2024-09-10T10:01:22.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:22.924+0000 7f4bfbebe740 10 client.0 _ll_register_callbacks cb 0x7f4bec03e040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:01:22.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:22.924+0000 7f4bf2800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d2f8d72c70 0x55d2f8d861c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:22.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:22.925+0000 7f4bf3200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d2f8d7de50 0x55d2f8d81610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:22.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:22.925+0000 7f4bf2800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d2f8d72c70 0x55d2f8d861c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59548/0 (socket says 172.21.15.5:59548) 2024-09-10T10:01:22.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:22.925+0000 7f4bf2800640 1 -- 172.21.15.5:0/146587877 learned_addr learned my addr 172.21.15.5:0/146587877 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T10:01:22.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:22.925+0000 7f4be9e00640 1 --2- 172.21.15.5:0/146587877 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d2f8d73590 0x55d2f8d86700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:22.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:22.925+0000 7f4bf2800640 1 -- 172.21.15.5:0/146587877 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d2f8d7de50 msgr2=0x55d2f8d81610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:01:22.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:22.925+0000 7f4bf2800640 1 --2- 172.21.15.5:0/146587877 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d2f8d7de50 0x55d2f8d81610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:22.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:22.925+0000 7f4bf2800640 1 -- 172.21.15.5:0/146587877 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d2f8d73590 msgr2=0x55d2f8d86700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:01:22.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:22.925+0000 7f4bf2800640 1 --2- 172.21.15.5:0/146587877 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d2f8d73590 0x55d2f8d86700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:22.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:22.925+0000 7f4bf2800640 1 -- 172.21.15.5:0/146587877 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d2f8d75ae0 con 0x55d2f8d72c70 2024-09-10T10:01:22.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:22.925+0000 7f4be9e00640 1 --2- 172.21.15.5:0/146587877 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d2f8d73590 0x55d2f8d86700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:01:22.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:22.925+0000 7f4bf3200640 1 --2- 172.21.15.5:0/146587877 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d2f8d7de50 0x55d2f8d81610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:01:22.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:22.925+0000 7f4bf2800640 1 --2- 172.21.15.5:0/146587877 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d2f8d72c70 0x55d2f8d861c0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f4be409b960 tx=0x7f4be40a4b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24eb92d0b839caf0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:22.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:22.925+0000 7f4bd7e00640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-10T10:01:22.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:22.925+0000 7f4bd7e00640 1 -- 172.21.15.5:0/146587877 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4be409c590 con 0x55d2f8d72c70 2024-09-10T10:01:22.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:22.926+0000 7f4bd7e00640 1 -- 172.21.15.5:0/146587877 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f4be4018420 con 0x55d2f8d72c70 2024-09-10T10:01:22.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:22.926+0000 7f4bfbebe740 1 -- 172.21.15.5:0/146587877 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4be40a0240 con 0x55d2f8d72c70 2024-09-10T10:01:22.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:22.926+0000 7f4bd7e00640 1 -- 172.21.15.5:0/146587877 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f4be4010ac0 con 0x55d2f8d72c70 2024-09-10T10:01:22.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:22.926+0000 7f4bfbebe740 10 client.19692 fetch_fsmap learned FSMap version 42 2024-09-10T10:01:22.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:22.926+0000 7f4bfbebe740 1 -- 172.21.15.5:0/146587877 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55d2f8df8260 con 0x55d2f8d72c70 2024-09-10T10:01:22.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:22.926+0000 7f4bd7e00640 1 -- 172.21.15.5:0/146587877 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4be40a0240 con 0x55d2f8d72c70 2024-09-10T10:01:22.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:22.926+0000 7f4bd7e00640 1 -- 172.21.15.5:0/146587877 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 42) ==== 8370+0+0 (secure 0 0 0) 0x7f4be40bac90 con 0x55d2f8d72c70 2024-09-10T10:01:22.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:22.926+0000 7f4bfbebe740 10 client.19692 fetch_fsmap finished waiting for FSMap version 42 2024-09-10T10:01:22.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:22.926+0000 7f4bfbebe740 10 client.19692 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-09-10T10:01:22.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:22.926+0000 7f4bfbebe740 20 client.19692 populate_metadata read hostname 'smithi005' 2024-09-10T10:01:22.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:22.926+0000 7f4bfbebe740 1 --2- 172.21.15.5:0/146587877 >> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] conn(0x55d2f8df8870 0x55d2f8dfac60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:22.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:22.926+0000 7f4bfbebe740 4 client.19692 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T10:01:22.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:22.926+0000 7f4bfbebe740 1 -- 172.21.15.5:0/146587877 --> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d2f8ce0e60 con 0x55d2f8df8870 2024-09-10T10:01:22.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:22.927+0000 7f4be9e00640 1 --2- 172.21.15.5:0/146587877 >> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] conn(0x55d2f8df8870 0x55d2f8dfac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:22.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:22.927+0000 7f4be9e00640 1 --2- 172.21.15.5:0/146587877 >> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] conn(0x55d2f8df8870 0x55d2f8dfac60 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=41c60bcc5b4a1708 server_cookie=59dca2d6e11c79ec in_seq=0 out_seq=0 2024-09-10T10:01:22.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:22.927+0000 7f4bd7e00640 10 client.19692 ms_handle_connect on v2:172.21.15.102:6836/2689776764 2024-09-10T10:01:22.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:22.927+0000 7f4bd7e00640 1 -- 172.21.15.5:0/146587877 <== mds.3 v2:172.21.15.102:6836/2689776764 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d2f8ce0e60 con 0x55d2f8df8870 2024-09-10T10:01:22.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:22.927+0000 7f4bd7e00640 10 client.19692 handle_command_reply: tid=0 2024-09-10T10:01:22.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:22.927+0000 7f4bfbebe740 1 client.19692 shutdown 2024-09-10T10:01:22.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:22.927+0000 7f4bfbebe740 1 -- 172.21.15.5:0/146587877 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d2f8d72c70 msgr2=0x55d2f8d861c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:22.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:22.928+0000 7f4bfbebe740 1 --2- 172.21.15.5:0/146587877 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d2f8d72c70 0x55d2f8d861c0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f4be409b960 tx=0x7f4be40a4b00 comp rx=0 tx=0).stop 2024-09-10T10:01:22.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:22.928+0000 7f4bfbebe740 1 -- 172.21.15.5:0/146587877 shutdown_connections 2024-09-10T10:01:22.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:22.928+0000 7f4bfbebe740 1 --2- 172.21.15.5:0/146587877 >> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] conn(0x55d2f8df8870 0x55d2f8dfac60 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:22.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:22.928+0000 7f4bfbebe740 1 --2- 172.21.15.5:0/146587877 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d2f8d7de50 0x55d2f8d81610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:22.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:22.928+0000 7f4bfbebe740 1 --2- 172.21.15.5:0/146587877 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d2f8d73590 0x55d2f8d86700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:22.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:22.928+0000 7f4bfbebe740 1 --2- 172.21.15.5:0/146587877 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d2f8d72c70 0x55d2f8d861c0 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:22.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:22.928+0000 7f4bfbebe740 1 -- 172.21.15.5:0/146587877 >> 172.21.15.5:0/146587877 conn(0x55d2f8d6f200 msgr2=0x55d2f8d74710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:22.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:22.928+0000 7f4bd7e00640 0 client.19692 ms_handle_reset on v2:172.21.15.102:6836/2689776764 2024-09-10T10:01:22.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:22.928+0000 7f4bfbebe740 1 -- 172.21.15.5:0/146587877 shutdown_connections 2024-09-10T10:01:22.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:22.928+0000 7f4bfbebe740 1 -- 172.21.15.5:0/146587877 wait complete. 2024-09-10T10:01:22.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:22.928+0000 7f4bfbebe740 20 client.19692 trim_cache size 0 max 16384 2024-09-10T10:01:22.933 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-10T10:01:22.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:22.929+0000 7f4bf9400640 1 -- 172.21.15.5:0/1715076333 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f4bd80a2c00 msgr2=0x7f4bd80a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:22.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:22.929+0000 7f4bf9400640 1 --2- 172.21.15.5:0/1715076333 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f4bd80a2c00 0x7f4bd80a50c0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f4bec040000 tx=0x55d2f8d73c60 comp rx=0 tx=0).stop 2024-09-10T10:01:22.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:22.929+0000 7f4bf9400640 1 -- 172.21.15.5:0/1715076333 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4bf4101620 msgr2=0x7f4bf41079c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:22.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:22.929+0000 7f4bf9400640 1 --2- 172.21.15.5:0/1715076333 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4bf4101620 0x7f4bf41079c0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f4be400ec20 tx=0x7f4be400c6a0 comp rx=0 tx=0).stop 2024-09-10T10:01:22.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:22.929+0000 7f4bf9400640 1 -- 172.21.15.5:0/1715076333 shutdown_connections 2024-09-10T10:01:22.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:22.929+0000 7f4bf9400640 1 --2- 172.21.15.5:0/1715076333 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f4bd80a2c00 0x7f4bd80a50c0 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:22.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:22.929+0000 7f4bf9400640 1 --2- 172.21.15.5:0/1715076333 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4bf4105aa0 0x7f4bf4108440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:22.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:22.929+0000 7f4bf9400640 1 --2- 172.21.15.5:0/1715076333 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4bf4101fc0 0x7f4bf4107f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:22.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:22.929+0000 7f4bf9400640 1 --2- 172.21.15.5:0/1715076333 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4bf4101620 0x7f4bf41079c0 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:22.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:22.929+0000 7f4bf9400640 1 -- 172.21.15.5:0/1715076333 >> 172.21.15.5:0/1715076333 conn(0x7f4bf40fba90 msgr2=0x7f4bf4100570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:22.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:22.929+0000 7f4bf9400640 1 -- 172.21.15.5:0/1715076333 shutdown_connections 2024-09-10T10:01:22.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:22.929+0000 7f4bf9400640 1 -- 172.21.15.5:0/1715076333 wait complete. 2024-09-10T10:01:23.041 INFO:tasks.workunit.client.0.smithi005.stdout: 10 142180 13.21 MB/sec execute 421 sec latency 135.917 ms 2024-09-10T10:01:23.535 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:01:23 smithi005 ceph-mon[30350]: pgmap v1084: 129 pgs: 129 active+clean; 704 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 3.2 MiB/s wr, 428 op/s 2024-09-10T10:01:23.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:01:23 smithi159 ceph-mon[35513]: pgmap v1084: 129 pgs: 129 active+clean; 704 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 3.2 MiB/s wr, 428 op/s 2024-09-10T10:01:23.636 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:01:23 smithi102 ceph-mon[35442]: pgmap v1084: 129 pgs: 129 active+clean; 704 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 3.2 MiB/s wr, 428 op/s 2024-09-10T10:01:24.041 INFO:tasks.workunit.client.0.smithi005.stdout: 10 142760 13.26 MB/sec execute 422 sec latency 135.677 ms 2024-09-10T10:01:24.256 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-10T10:01:24.750 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:01:25.041 INFO:tasks.workunit.client.0.smithi005.stdout: 10 143112 13.26 MB/sec execute 423 sec latency 364.070 ms 2024-09-10T10:01:25.524 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:01:25 smithi005 ceph-mon[30350]: pgmap v1085: 129 pgs: 129 active+clean; 705 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 3.1 MiB/s wr, 430 op/s 2024-09-10T10:01:25.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:01:25 smithi159 ceph-mon[35513]: pgmap v1085: 129 pgs: 129 active+clean; 705 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 3.1 MiB/s wr, 430 op/s 2024-09-10T10:01:25.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:01:25 smithi102 ceph-mon[35442]: pgmap v1085: 129 pgs: 129 active+clean; 705 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 3.1 MiB/s wr, 430 op/s 2024-09-10T10:01:26.041 INFO:tasks.workunit.client.0.smithi005.stdout: 10 143666 13.30 MB/sec execute 424 sec latency 217.239 ms 2024-09-10T10:01:26.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:25.520+0000 7f06dfe00640 1 -- 172.21.15.5:0/1524930491 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f06e0108f60 msgr2=0x7f06e0110ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:26.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:25.520+0000 7f06dfe00640 1 --2- 172.21.15.5:0/1524930491 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f06e0108f60 0x7f06e0110ae0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f06d8009be0 tx=0x7f06d802a6d0 comp rx=0 tx=0).stop 2024-09-10T10:01:26.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:25.521+0000 7f06dfe00640 1 -- 172.21.15.5:0/1524930491 shutdown_connections 2024-09-10T10:01:26.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:25.521+0000 7f06dfe00640 1 --2- 172.21.15.5:0/1524930491 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f06e0108f60 0x7f06e0110ae0 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:26.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:25.521+0000 7f06dfe00640 1 --2- 172.21.15.5:0/1524930491 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f06e01047e0 0x7f06e0108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:26.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:25.521+0000 7f06dfe00640 1 --2- 172.21.15.5:0/1524930491 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f06e0103e30 0x7f06e0104210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:26.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:25.521+0000 7f06dfe00640 1 -- 172.21.15.5:0/1524930491 >> 172.21.15.5:0/1524930491 conn(0x7f06e00ff910 msgr2=0x7f06e0101d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:26.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:25.521+0000 7f06dfe00640 1 -- 172.21.15.5:0/1524930491 shutdown_connections 2024-09-10T10:01:26.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:25.521+0000 7f06dfe00640 1 -- 172.21.15.5:0/1524930491 wait complete. 2024-09-10T10:01:26.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:25.522+0000 7f06dfe00640 1 Processor -- start 2024-09-10T10:01:26.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:25.522+0000 7f06dfe00640 1 -- start start 2024-09-10T10:01:26.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:25.523+0000 7f06dfe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f06e0103e30 0x7f06e019a420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:26.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:25.523+0000 7f06dfe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f06e01047e0 0x7f06e019a960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:26.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:25.523+0000 7f06dfe00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f06e0108f60 0x7f06e019ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:26.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:25.523+0000 7f06dfe00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f06e0113fc0 con 0x7f06e0103e30 2024-09-10T10:01:26.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:25.523+0000 7f06dfe00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f06e0113e40 con 0x7f06e01047e0 2024-09-10T10:01:26.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:25.523+0000 7f06dfe00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f06e0114140 con 0x7f06e0108f60 2024-09-10T10:01:26.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:25.523+0000 7f06df400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f06e0108f60 0x7f06e019ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:26.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:25.524+0000 7f06df400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f06e0108f60 0x7f06e019ecf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:47706/0 (socket says 172.21.15.5:47706) 2024-09-10T10:01:26.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:25.524+0000 7f06df400640 1 -- 172.21.15.5:0/4175017923 learned_addr learned my addr 172.21.15.5:0/4175017923 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T10:01:26.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:25.524+0000 7f06dea00640 1 --2- 172.21.15.5:0/4175017923 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f06e0103e30 0x7f06e019a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:26.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:25.524+0000 7f06de000640 1 --2- 172.21.15.5:0/4175017923 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f06e01047e0 0x7f06e019a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:26.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:25.524+0000 7f06dea00640 1 -- 172.21.15.5:0/4175017923 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f06e0108f60 msgr2=0x7f06e019ecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:26.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:25.524+0000 7f06dea00640 1 --2- 172.21.15.5:0/4175017923 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f06e0108f60 0x7f06e019ecf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:26.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:25.524+0000 7f06dea00640 1 -- 172.21.15.5:0/4175017923 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f06e01047e0 msgr2=0x7f06e019a960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:26.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:25.524+0000 7f06dea00640 1 --2- 172.21.15.5:0/4175017923 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f06e01047e0 0x7f06e019a960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:26.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:25.524+0000 7f06dea00640 1 -- 172.21.15.5:0/4175017923 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06e019f3d0 con 0x7f06e0103e30 2024-09-10T10:01:26.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:25.524+0000 7f06de000640 1 --2- 172.21.15.5:0/4175017923 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f06e01047e0 0x7f06e019a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:01:26.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:25.525+0000 7f06dea00640 1 --2- 172.21.15.5:0/4175017923 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f06e0103e30 0x7f06e019a420 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f06d0005fd0 tx=0x7f06d000dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:26.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:25.525+0000 7f06d7400640 1 -- 172.21.15.5:0/4175017923 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06d0009c30 con 0x7f06e0103e30 2024-09-10T10:01:26.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:25.525+0000 7f06d7400640 1 -- 172.21.15.5:0/4175017923 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f06d0009dd0 con 0x7f06e0103e30 2024-09-10T10:01:26.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:25.525+0000 7f06dfe00640 1 -- 172.21.15.5:0/4175017923 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06e019f6c0 con 0x7f06e0103e30 2024-09-10T10:01:26.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:25.525+0000 7f06d7400640 1 -- 172.21.15.5:0/4175017923 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06d0010460 con 0x7f06e0103e30 2024-09-10T10:01:26.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:25.525+0000 7f06dfe00640 1 -- 172.21.15.5:0/4175017923 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06e01a6fa0 con 0x7f06e0103e30 2024-09-10T10:01:26.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:25.528+0000 7f06e8652740 1 Processor -- start 2024-09-10T10:01:26.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:25.529+0000 7f06e8652740 1 -- start start 2024-09-10T10:01:26.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:25.529+0000 7f06e8652740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e870152e40 0x55e870153220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:26.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:25.529+0000 7f06e8652740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e870153760 0x55e87015dae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:26.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:25.529+0000 7f06e8652740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e87015e020 0x55e870160410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:26.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:25.529+0000 7f06e8652740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55e8700b93e0 con 0x55e870152e40 2024-09-10T10:01:26.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:25.529+0000 7f06e8652740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55e870060d80 con 0x55e87015e020 2024-09-10T10:01:26.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:25.529+0000 7f06e8652740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55e8700b53a0 con 0x55e870153760 2024-09-10T10:01:26.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:25.529+0000 7f06d7400640 1 -- 172.21.15.5:0/4175017923 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f06d0003ef0 con 0x7f06e0103e30 2024-09-10T10:01:26.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:25.529+0000 7f06de000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e87015e020 0x55e870160410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:26.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:25.529+0000 7f06de000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e87015e020 0x55e870160410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:38338/0 (socket says 172.21.15.5:38338) 2024-09-10T10:01:26.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:25.529+0000 7f06de000640 1 -- 172.21.15.5:0/2217368477 learned_addr learned my addr 172.21.15.5:0/2217368477 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T10:01:26.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:25.530+0000 7f06dea00640 1 --2- 172.21.15.5:0/2217368477 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e870153760 0x55e87015dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:26.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:25.530+0000 7f06de000640 1 -- 172.21.15.5:0/2217368477 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e870153760 msgr2=0x55e87015dae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:01:26.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:25.530+0000 7f06de000640 1 --2- 172.21.15.5:0/2217368477 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e870153760 0x55e87015dae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:26.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:25.530+0000 7f06de000640 1 -- 172.21.15.5:0/2217368477 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e870152e40 msgr2=0x55e870153220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:01:26.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:25.530+0000 7f06df400640 1 --2- 172.21.15.5:0/2217368477 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e870152e40 0x55e870153220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:26.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:25.530+0000 7f06de000640 1 --2- 172.21.15.5:0/2217368477 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e870152e40 0x55e870153220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:26.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:25.530+0000 7f06de000640 1 -- 172.21.15.5:0/2217368477 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e87007fae0 con 0x55e87015e020 2024-09-10T10:01:26.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:25.530+0000 7f06df400640 1 --2- 172.21.15.5:0/2217368477 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e870152e40 0x55e870153220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:01:26.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:25.530+0000 7f06de000640 1 --2- 172.21.15.5:0/2217368477 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e87015e020 0x55e870160410 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f06cc009bd0 tx=0x7f06cc00f730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dc3d6329138058e6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:26.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:25.531+0000 7f06cbe00640 1 -- 172.21.15.5:0/2217368477 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06cc004020 con 0x55e87015e020 2024-09-10T10:01:26.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:25.531+0000 7f06cbe00640 1 -- 172.21.15.5:0/2217368477 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f06cc0041c0 con 0x55e87015e020 2024-09-10T10:01:26.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:25.531+0000 7f06cbe00640 1 -- 172.21.15.5:0/2217368477 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06cc011680 con 0x55e87015e020 2024-09-10T10:01:26.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:25.531+0000 7f06d7400640 1 --2- 172.21.15.5:0/4175017923 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f06c40a2c00 0x7f06c40a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:26.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:25.531+0000 7f06d7400640 1 -- 172.21.15.5:0/4175017923 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f06d00cefd0 con 0x7f06e0103e30 2024-09-10T10:01:26.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:25.531+0000 7f06e8652740 1 -- 172.21.15.5:0/2217368477 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e87015e020 msgr2=0x55e870160410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:26.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:25.531+0000 7f06e8652740 1 --2- 172.21.15.5:0/2217368477 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e87015e020 0x55e870160410 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f06cc009bd0 tx=0x7f06cc00f730 comp rx=0 tx=0).stop 2024-09-10T10:01:26.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:25.532+0000 7f06df400640 1 --2- 172.21.15.5:0/4175017923 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f06c40a2c00 0x7f06c40a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:26.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:25.532+0000 7f06e8652740 1 -- 172.21.15.5:0/2217368477 shutdown_connections 2024-09-10T10:01:26.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:25.532+0000 7f06e8652740 1 --2- 172.21.15.5:0/2217368477 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e87015e020 0x55e870160410 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:26.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:25.532+0000 7f06e8652740 1 --2- 172.21.15.5:0/2217368477 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e870153760 0x55e87015dae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:26.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:25.532+0000 7f06e8652740 1 --2- 172.21.15.5:0/2217368477 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e870152e40 0x55e870153220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:26.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:25.532+0000 7f06e8652740 1 -- 172.21.15.5:0/2217368477 >> 172.21.15.5:0/2217368477 conn(0x55e87014f3d0 msgr2=0x55e87014f7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:26.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:25.532+0000 7f06e8652740 1 -- 172.21.15.5:0/2217368477 shutdown_connections 2024-09-10T10:01:26.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:25.532+0000 7f06e8652740 1 -- 172.21.15.5:0/2217368477 wait complete. 2024-09-10T10:01:26.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:25.532+0000 7f06df400640 1 --2- 172.21.15.5:0/4175017923 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f06c40a2c00 0x7f06c40a50c0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f06d80099a0 tx=0x7f06d8009930 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:26.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:25.533+0000 7f06e8652740 1 -- start start 2024-09-10T10:01:26.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:25.533+0000 7f06e8652740 1 Processor -- start 2024-09-10T10:01:26.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:25.534+0000 7f06e8652740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e870152e40 0x55e87015aec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:26.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:25.534+0000 7f06e8652740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e870153760 0x55e87015b400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:26.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:25.534+0000 7f06e8652740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e87015e020 0x55e87015b940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:26.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:25.534+0000 7f06e8652740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55e870160bf0 con 0x55e870152e40 2024-09-10T10:01:26.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:25.534+0000 7f06e8652740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55e8700740f0 con 0x55e870153760 2024-09-10T10:01:26.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:25.534+0000 7f06e8652740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55e870160d70 con 0x55e87015e020 2024-09-10T10:01:26.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:25.534+0000 7f06e8652740 10 client.0 _ll_register_callbacks cb 0x55e87005bbd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:01:26.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:25.534+0000 7f06de000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e870153760 0x55e87015b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:26.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:25.534+0000 7f06dea00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e870152e40 0x55e87015aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:26.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:25.534+0000 7f06de000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e870153760 0x55e87015b400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:38350/0 (socket says 172.21.15.5:38350) 2024-09-10T10:01:26.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:25.534+0000 7f06de000640 1 -- 172.21.15.5:0/2527666810 learned_addr learned my addr 172.21.15.5:0/2527666810 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T10:01:26.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:25.534+0000 7f06df400640 1 --2- 172.21.15.5:0/2527666810 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e87015e020 0x55e87015b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:26.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:25.534+0000 7f06de000640 1 -- 172.21.15.5:0/2527666810 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e87015e020 msgr2=0x55e87015b940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:01:26.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:25.534+0000 7f06de000640 1 --2- 172.21.15.5:0/2527666810 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e87015e020 0x55e87015b940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:26.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:25.534+0000 7f06de000640 1 -- 172.21.15.5:0/2527666810 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e870152e40 msgr2=0x55e87015aec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:01:26.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:25.534+0000 7f06de000640 1 --2- 172.21.15.5:0/2527666810 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e870152e40 0x55e87015aec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:26.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:25.534+0000 7f06de000640 1 -- 172.21.15.5:0/2527666810 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e86fe2a3c0 con 0x55e870153760 2024-09-10T10:01:26.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:25.534+0000 7f06dea00640 1 --2- 172.21.15.5:0/2527666810 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e870152e40 0x55e87015aec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:01:26.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:25.535+0000 7f06de000640 1 --2- 172.21.15.5:0/2527666810 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e870153760 0x55e87015b400 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f06cc000c00 tx=0x7f06cc009800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2fbb4e56a53b9cc3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:26.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:25.535+0000 7f06c8c00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-09-10T10:01:26.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:25.535+0000 7f06c8c00640 1 -- 172.21.15.5:0/2527666810 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06cc02a060 con 0x55e870153760 2024-09-10T10:01:26.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:25.535+0000 7f06e8652740 1 -- 172.21.15.5:0/2527666810 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f06cc0041c0 con 0x55e870153760 2024-09-10T10:01:26.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:25.535+0000 7f06c8c00640 1 -- 172.21.15.5:0/2527666810 <== mon.2 v2:172.21.15.102:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f06cc004370 con 0x55e870153760 2024-09-10T10:01:26.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:25.535+0000 7f06c8c00640 1 -- 172.21.15.5:0/2527666810 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f06cc031050 con 0x55e870153760 2024-09-10T10:01:26.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:25.535+0000 7f06e8652740 10 client.29783 fetch_fsmap learned FSMap version 42 2024-09-10T10:01:26.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:25.535+0000 7f06e8652740 1 -- 172.21.15.5:0/2527666810 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55e87016cbb0 con 0x55e870153760 2024-09-10T10:01:26.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:25.536+0000 7f06c8c00640 1 -- 172.21.15.5:0/2527666810 <== mon.2 v2:172.21.15.102:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f06cc0041c0 con 0x55e870153760 2024-09-10T10:01:26.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:25.537+0000 7f06c8c00640 1 -- 172.21.15.5:0/2527666810 <== mon.2 v2:172.21.15.102:3300/0 5 ==== fsmap(e 42) ==== 8370+0+0 (secure 0 0 0) 0x7f06cc0035f0 con 0x55e870153760 2024-09-10T10:01:26.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:25.537+0000 7f06e8652740 10 client.29783 fetch_fsmap finished waiting for FSMap version 42 2024-09-10T10:01:26.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:25.537+0000 7f06e8652740 10 client.29783 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-10T10:01:26.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:25.537+0000 7f06e8652740 20 client.29783 populate_metadata read hostname 'smithi005' 2024-09-10T10:01:26.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:25.537+0000 7f06e8652740 1 --2- 172.21.15.5:0/2527666810 >> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] conn(0x55e87016dfc0 0x55e8701703b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:26.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:25.537+0000 7f06e8652740 4 client.29783 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:01:26.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:25.537+0000 7f06e8652740 1 -- 172.21.15.5:0/2527666810 --> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e8700c6950 con 0x55e87016dfc0 2024-09-10T10:01:26.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:25.537+0000 7f06dea00640 1 --2- 172.21.15.5:0/2527666810 >> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] conn(0x55e87016dfc0 0x55e8701703b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:26.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:25.538+0000 7f06dea00640 1 --2- 172.21.15.5:0/2527666810 >> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] conn(0x55e87016dfc0 0x55e8701703b0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f4b2f6b0cffbaf4d server_cookie=d73f1e3704b84e2d in_seq=0 out_seq=0 2024-09-10T10:01:26.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:25.538+0000 7f06c8c00640 10 client.29783 ms_handle_connect on v2:172.21.15.159:6836/620627639 2024-09-10T10:01:26.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:25.541+0000 7f06c8c00640 1 -- 172.21.15.5:0/2527666810 <== mds.2 v2:172.21.15.159:6836/620627639 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e8700c6950 con 0x55e87016dfc0 2024-09-10T10:01:26.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:25.541+0000 7f06c8c00640 10 client.29783 handle_command_reply: tid=0 2024-09-10T10:01:26.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:25.541+0000 7f06e8652740 1 client.29783 shutdown 2024-09-10T10:01:26.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:25.541+0000 7f06e8652740 1 -- 172.21.15.5:0/2527666810 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e870153760 msgr2=0x55e87015b400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:26.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:25.541+0000 7f06e8652740 1 --2- 172.21.15.5:0/2527666810 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e870153760 0x55e87015b400 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f06cc000c00 tx=0x7f06cc009800 comp rx=0 tx=0).stop 2024-09-10T10:01:26.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:25.541+0000 7f06e8652740 1 -- 172.21.15.5:0/2527666810 shutdown_connections 2024-09-10T10:01:26.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:25.541+0000 7f06e8652740 1 --2- 172.21.15.5:0/2527666810 >> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] conn(0x55e87016dfc0 0x55e8701703b0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:26.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:25.541+0000 7f06e8652740 1 --2- 172.21.15.5:0/2527666810 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e87015e020 0x55e87015b940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:26.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:25.541+0000 7f06e8652740 1 --2- 172.21.15.5:0/2527666810 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e870153760 0x55e87015b400 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:26.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:25.541+0000 7f06e8652740 1 --2- 172.21.15.5:0/2527666810 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e870152e40 0x55e87015aec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:26.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:25.541+0000 7f06c8c00640 0 client.29783 ms_handle_reset on v2:172.21.15.159:6836/620627639 2024-09-10T10:01:26.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:25.541+0000 7f06e8652740 1 -- 172.21.15.5:0/2527666810 >> 172.21.15.5:0/2527666810 conn(0x55e87014f3d0 msgr2=0x55e870155100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:26.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:25.542+0000 7f06e8652740 1 -- 172.21.15.5:0/2527666810 shutdown_connections 2024-09-10T10:01:26.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:25.542+0000 7f06e8652740 1 -- 172.21.15.5:0/2527666810 wait complete. 2024-09-10T10:01:26.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:25.542+0000 7f06e8652740 20 client.29783 trim_cache size 0 max 16384 2024-09-10T10:01:26.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:25.558+0000 7f06e8652740 1 Processor -- start 2024-09-10T10:01:26.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:25.558+0000 7f06e8652740 1 -- start start 2024-09-10T10:01:26.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:25.558+0000 7f06e8652740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e870152e40 0x55e87015ce30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:26.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:25.558+0000 7f06e8652740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e870153760 0x55e8701759b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:26.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:25.558+0000 7f06e8652740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e87015e020 0x55e870175ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:26.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:25.558+0000 7f06e8652740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f06e01acb40 con 0x55e870152e40 2024-09-10T10:01:26.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:25.558+0000 7f06e8652740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f06e01ae680 con 0x55e870153760 2024-09-10T10:01:26.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:25.558+0000 7f06e8652740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f06e01ad210 con 0x55e87015e020 2024-09-10T10:01:26.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:25.558+0000 7f06dea00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e870152e40 0x55e87015ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:26.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:25.559+0000 7f06de000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e870153760 0x55e8701759b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:26.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:25.559+0000 7f06dea00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e870152e40 0x55e87015ce30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59608/0 (socket says 172.21.15.5:59608) 2024-09-10T10:01:26.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:25.559+0000 7f06dea00640 1 -- 172.21.15.5:0/225326600 learned_addr learned my addr 172.21.15.5:0/225326600 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T10:01:26.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:25.559+0000 7f06df400640 1 --2- 172.21.15.5:0/225326600 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e87015e020 0x55e870175ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:26.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:25.559+0000 7f06dea00640 1 -- 172.21.15.5:0/225326600 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e87015e020 msgr2=0x55e870175ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:01:26.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:25.559+0000 7f06dea00640 1 --2- 172.21.15.5:0/225326600 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e87015e020 0x55e870175ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:26.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:25.559+0000 7f06dea00640 1 -- 172.21.15.5:0/225326600 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e870153760 msgr2=0x55e8701759b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:01:26.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:25.559+0000 7f06dea00640 1 --2- 172.21.15.5:0/225326600 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e870153760 0x55e8701759b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:26.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:25.559+0000 7f06dea00640 1 -- 172.21.15.5:0/225326600 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e870150060 con 0x55e870152e40 2024-09-10T10:01:26.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:25.559+0000 7f06de000640 1 --2- 172.21.15.5:0/225326600 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e870153760 0x55e8701759b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:01:26.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:25.559+0000 7f06dea00640 1 --2- 172.21.15.5:0/225326600 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e870152e40 0x55e87015ce30 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f06d000bfd0 tx=0x7f06d0097040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5c36828a8cc308d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:26.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:25.560+0000 7f06cb400640 1 -- 172.21.15.5:0/225326600 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06d0020040 con 0x55e870152e40 2024-09-10T10:01:26.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:25.560+0000 7f06cb400640 1 -- 172.21.15.5:0/225326600 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f06d009bae0 con 0x55e870152e40 2024-09-10T10:01:26.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:25.560+0000 7f06cb400640 1 -- 172.21.15.5:0/225326600 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06d0099510 con 0x55e870152e40 2024-09-10T10:01:26.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:25.561+0000 7f06e8652740 1 -- 172.21.15.5:0/225326600 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e870152e40 msgr2=0x55e87015ce30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:26.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:25.561+0000 7f06e8652740 1 --2- 172.21.15.5:0/225326600 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e870152e40 0x55e87015ce30 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f06d000bfd0 tx=0x7f06d0097040 comp rx=0 tx=0).stop 2024-09-10T10:01:26.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:25.561+0000 7f06e8652740 1 -- 172.21.15.5:0/225326600 shutdown_connections 2024-09-10T10:01:26.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:25.561+0000 7f06e8652740 1 --2- 172.21.15.5:0/225326600 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e87015e020 0x55e870175ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:26.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:25.561+0000 7f06e8652740 1 --2- 172.21.15.5:0/225326600 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e870153760 0x55e8701759b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:26.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:25.561+0000 7f06e8652740 1 --2- 172.21.15.5:0/225326600 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e870152e40 0x55e87015ce30 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:26.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:25.561+0000 7f06e8652740 1 -- 172.21.15.5:0/225326600 >> 172.21.15.5:0/225326600 conn(0x55e87014f3d0 msgr2=0x55e87015c720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:26.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:25.561+0000 7f06e8652740 1 -- 172.21.15.5:0/225326600 shutdown_connections 2024-09-10T10:01:26.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:25.561+0000 7f06e8652740 1 -- 172.21.15.5:0/225326600 wait complete. 2024-09-10T10:01:26.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:25.561+0000 7f06e8652740 1 -- start start 2024-09-10T10:01:26.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:25.562+0000 7f06e8652740 1 Processor -- start 2024-09-10T10:01:26.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:25.562+0000 7f06e8652740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e870152e40 0x55e870166600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:26.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:25.562+0000 7f06e8652740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e870153760 0x55e870161980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:26.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:25.562+0000 7f06e8652740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e87015e020 0x55e870161ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:26.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:25.562+0000 7f06e8652740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f06e01adfb0 con 0x55e87015e020 2024-09-10T10:01:26.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:25.562+0000 7f06e8652740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f06e01ad8e0 con 0x55e870153760 2024-09-10T10:01:26.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:25.562+0000 7f06e8652740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f06e01aab20 con 0x55e870152e40 2024-09-10T10:01:26.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:25.562+0000 7f06e8652740 10 client.0 _ll_register_callbacks cb 0x7f06cc015070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:01:26.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:25.563+0000 7f06dea00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e870152e40 0x55e870166600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:26.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:25.563+0000 7f06dea00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e870152e40 0x55e870166600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:47748/0 (socket says 172.21.15.5:47748) 2024-09-10T10:01:26.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:25.563+0000 7f06dea00640 1 -- 172.21.15.5:0/2730531483 learned_addr learned my addr 172.21.15.5:0/2730531483 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T10:01:26.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:25.563+0000 7f06df400640 1 --2- 172.21.15.5:0/2730531483 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e87015e020 0x55e870161ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:26.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:25.563+0000 7f06de000640 1 --2- 172.21.15.5:0/2730531483 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e870153760 0x55e870161980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:26.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:25.563+0000 7f06dea00640 1 -- 172.21.15.5:0/2730531483 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e870153760 msgr2=0x55e870161980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:01:26.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:25.563+0000 7f06dea00640 1 --2- 172.21.15.5:0/2730531483 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e870153760 0x55e870161980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:26.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:25.563+0000 7f06dea00640 1 -- 172.21.15.5:0/2730531483 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e87015e020 msgr2=0x55e870161ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:01:26.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:25.563+0000 7f06dea00640 1 --2- 172.21.15.5:0/2730531483 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e87015e020 0x55e870161ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:26.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:25.563+0000 7f06dea00640 1 -- 172.21.15.5:0/2730531483 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e8701625b0 con 0x55e870152e40 2024-09-10T10:01:26.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:25.563+0000 7f06dea00640 1 --2- 172.21.15.5:0/2730531483 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e870152e40 0x55e870166600 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f06d009b370 tx=0x7f06d009adb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=967f9fa84f99c620 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:26.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:25.563+0000 7f06c8c00640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-09-10T10:01:26.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:25.564+0000 7f06c8c00640 1 -- 172.21.15.5:0/2730531483 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06d00b3070 con 0x55e870152e40 2024-09-10T10:01:26.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:25.564+0000 7f06c8c00640 1 -- 172.21.15.5:0/2730531483 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f06d00cc790 con 0x55e870152e40 2024-09-10T10:01:26.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:25.564+0000 7f06e8652740 1 -- 172.21.15.5:0/2730531483 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f06d009bae0 con 0x55e870152e40 2024-09-10T10:01:26.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:25.564+0000 7f06c8c00640 1 -- 172.21.15.5:0/2730531483 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f06d0020070 con 0x55e870152e40 2024-09-10T10:01:26.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:25.564+0000 7f06e8652740 10 client.29764 fetch_fsmap learned FSMap version 42 2024-09-10T10:01:26.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:25.564+0000 7f06e8652740 1 -- 172.21.15.5:0/2730531483 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55e8701d8860 con 0x55e870152e40 2024-09-10T10:01:26.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:25.564+0000 7f06c8c00640 1 -- 172.21.15.5:0/2730531483 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f06d009bae0 con 0x55e870152e40 2024-09-10T10:01:26.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:25.564+0000 7f06c8c00640 1 -- 172.21.15.5:0/2730531483 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 42) ==== 8370+0+0 (secure 0 0 0) 0x7f06d00a3bc0 con 0x55e870152e40 2024-09-10T10:01:26.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:25.564+0000 7f06e8652740 10 client.29764 fetch_fsmap finished waiting for FSMap version 42 2024-09-10T10:01:26.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:25.564+0000 7f06e8652740 10 client.29764 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-10T10:01:26.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:25.564+0000 7f06e8652740 20 client.29764 populate_metadata read hostname 'smithi005' 2024-09-10T10:01:26.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:25.564+0000 7f06e8652740 1 --2- 172.21.15.5:0/2730531483 >> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] conn(0x55e8701d8d80 0x55e8701db170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:26.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:25.564+0000 7f06e8652740 4 client.29764 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T10:01:26.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:25.564+0000 7f06e8652740 1 -- 172.21.15.5:0/2730531483 --> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e8700c0e60 con 0x55e8701d8d80 2024-09-10T10:01:26.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:25.565+0000 7f06de000640 1 --2- 172.21.15.5:0/2730531483 >> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] conn(0x55e8701d8d80 0x55e8701db170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:26.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:25.565+0000 7f06c8c00640 10 client.29764 ms_handle_connect on v2:172.21.15.159:6836/620627639 2024-09-10T10:01:26.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:25.565+0000 7f06de000640 1 --2- 172.21.15.5:0/2730531483 >> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] conn(0x55e8701d8d80 0x55e8701db170 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5b0d9907473e76aa server_cookie=5c59266e50710cc4 in_seq=0 out_seq=0 2024-09-10T10:01:26.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:25.569+0000 7f06c8c00640 1 -- 172.21.15.5:0/2730531483 <== mds.2 v2:172.21.15.159:6836/620627639 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e8700c0e60 con 0x55e8701d8d80 2024-09-10T10:01:26.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:25.569+0000 7f06c8c00640 10 client.29764 handle_command_reply: tid=0 2024-09-10T10:01:26.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:25.569+0000 7f06e8652740 1 client.29764 shutdown 2024-09-10T10:01:26.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:25.569+0000 7f06e8652740 1 -- 172.21.15.5:0/2730531483 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e870152e40 msgr2=0x55e870166600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:26.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:25.569+0000 7f06e8652740 1 --2- 172.21.15.5:0/2730531483 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e870152e40 0x55e870166600 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f06d009b370 tx=0x7f06d009adb0 comp rx=0 tx=0).stop 2024-09-10T10:01:26.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:25.569+0000 7f06e8652740 1 -- 172.21.15.5:0/2730531483 shutdown_connections 2024-09-10T10:01:26.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:25.569+0000 7f06e8652740 1 --2- 172.21.15.5:0/2730531483 >> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] conn(0x55e8701d8d80 0x55e8701db170 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:26.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:25.569+0000 7f06e8652740 1 --2- 172.21.15.5:0/2730531483 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e87015e020 0x55e870161ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:26.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:25.569+0000 7f06e8652740 1 --2- 172.21.15.5:0/2730531483 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e870153760 0x55e870161980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:26.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:25.569+0000 7f06e8652740 1 --2- 172.21.15.5:0/2730531483 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e870152e40 0x55e870166600 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:26.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:25.569+0000 7f06e8652740 1 -- 172.21.15.5:0/2730531483 >> 172.21.15.5:0/2730531483 conn(0x55e87014f3d0 msgr2=0x55e87014e490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:26.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:25.569+0000 7f06c8c00640 0 client.29764 ms_handle_reset on v2:172.21.15.159:6836/620627639 2024-09-10T10:01:26.576 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-10T10:01:26.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:25.569+0000 7f06e8652740 1 -- 172.21.15.5:0/2730531483 shutdown_connections 2024-09-10T10:01:26.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:25.569+0000 7f06e8652740 1 -- 172.21.15.5:0/2730531483 wait complete. 2024-09-10T10:01:26.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:25.569+0000 7f06e8652740 20 client.29764 trim_cache size 0 max 16384 2024-09-10T10:01:26.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:25.570+0000 7f06dfe00640 1 -- 172.21.15.5:0/4175017923 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f06c40a2c00 msgr2=0x7f06c40a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:26.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:25.570+0000 7f06dfe00640 1 --2- 172.21.15.5:0/4175017923 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f06c40a2c00 0x7f06c40a50c0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f06d80099a0 tx=0x7f06d8009930 comp rx=0 tx=0).stop 2024-09-10T10:01:26.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:25.570+0000 7f06dfe00640 1 -- 172.21.15.5:0/4175017923 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f06e0103e30 msgr2=0x7f06e019a420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:26.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:25.570+0000 7f06dfe00640 1 --2- 172.21.15.5:0/4175017923 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f06e0103e30 0x7f06e019a420 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f06d0005fd0 tx=0x7f06d000dde0 comp rx=0 tx=0).stop 2024-09-10T10:01:26.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:25.570+0000 7f06dfe00640 1 -- 172.21.15.5:0/4175017923 shutdown_connections 2024-09-10T10:01:26.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:25.570+0000 7f06dfe00640 1 --2- 172.21.15.5:0/4175017923 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f06c40a2c00 0x7f06c40a50c0 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:26.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:25.570+0000 7f06dfe00640 1 --2- 172.21.15.5:0/4175017923 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f06e0108f60 0x7f06e019ecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:26.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:25.570+0000 7f06dfe00640 1 --2- 172.21.15.5:0/4175017923 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f06e01047e0 0x7f06e019a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:26.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:25.570+0000 7f06dfe00640 1 --2- 172.21.15.5:0/4175017923 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f06e0103e30 0x7f06e019a420 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:26.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:25.570+0000 7f06dfe00640 1 -- 172.21.15.5:0/4175017923 >> 172.21.15.5:0/4175017923 conn(0x7f06e00ff910 msgr2=0x7f06e0100c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:26.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:25.570+0000 7f06dfe00640 1 -- 172.21.15.5:0/4175017923 shutdown_connections 2024-09-10T10:01:26.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:25.571+0000 7f06dfe00640 1 -- 172.21.15.5:0/4175017923 wait complete. 2024-09-10T10:01:27.041 INFO:tasks.workunit.client.0.smithi005.stdout: 10 144427 13.35 MB/sec execute 425 sec latency 181.790 ms 2024-09-10T10:01:27.415 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-10T10:01:27.557 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:01:27 smithi005 ceph-mon[30350]: pgmap v1086: 129 pgs: 129 active+clean; 713 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 3.9 MiB/s wr, 533 op/s 2024-09-10T10:01:27.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:01:27 smithi159 ceph-mon[35513]: pgmap v1086: 129 pgs: 129 active+clean; 713 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 3.9 MiB/s wr, 533 op/s 2024-09-10T10:01:27.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:01:27 smithi102 ceph-mon[35442]: pgmap v1086: 129 pgs: 129 active+clean; 713 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 3.9 MiB/s wr, 533 op/s 2024-09-10T10:01:27.947 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:01:28.041 INFO:tasks.workunit.client.0.smithi005.stdout: 10 145179 13.42 MB/sec execute 426 sec latency 56.833 ms 2024-09-10T10:01:28.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:28.595+0000 7efc08c00640 1 -- 172.21.15.5:0/3179249310 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efc041049c0 msgr2=0x7efc04108a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:28.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:28.595+0000 7efc08c00640 1 --2- 172.21.15.5:0/3179249310 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efc041049c0 0x7efc04108a10 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7efbf0009900 tx=0x7efbf002a3f0 comp rx=0 tx=0).stop 2024-09-10T10:01:28.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:28.595+0000 7efc08c00640 1 -- 172.21.15.5:0/3179249310 shutdown_connections 2024-09-10T10:01:28.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:28.595+0000 7efc08c00640 1 --2- 172.21.15.5:0/3179249310 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efc04109140 0x7efc04110cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:28.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:28.595+0000 7efc08c00640 1 --2- 172.21.15.5:0/3179249310 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efc041049c0 0x7efc04108a10 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:28.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:28.595+0000 7efc08c00640 1 --2- 172.21.15.5:0/3179249310 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efc04104010 0x7efc041043f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:28.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:28.595+0000 7efc08c00640 1 -- 172.21.15.5:0/3179249310 >> 172.21.15.5:0/3179249310 conn(0x7efc040ffa50 msgr2=0x7efc04101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:28.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:28.595+0000 7efc08c00640 1 -- 172.21.15.5:0/3179249310 shutdown_connections 2024-09-10T10:01:28.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:28.595+0000 7efc08c00640 1 -- 172.21.15.5:0/3179249310 wait complete. 2024-09-10T10:01:28.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:28.597+0000 7efc08c00640 1 Processor -- start 2024-09-10T10:01:28.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:28.597+0000 7efc08c00640 1 -- start start 2024-09-10T10:01:28.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:28.597+0000 7efc08c00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efc04104010 0x7efc0419a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:28.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:28.597+0000 7efc08c00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efc041049c0 0x7efc0419abf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:28.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:28.597+0000 7efc08c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efc04109140 0x7efc0419ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:28.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:28.597+0000 7efc08c00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7efc041141b0 con 0x7efc04104010 2024-09-10T10:01:28.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:28.597+0000 7efc08c00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7efc04114030 con 0x7efc04109140 2024-09-10T10:01:28.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:28.597+0000 7efc08c00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7efc04114330 con 0x7efc041049c0 2024-09-10T10:01:28.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:28.597+0000 7efc01400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efc041049c0 0x7efc0419abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:28.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:28.597+0000 7efc01400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efc041049c0 0x7efc0419abf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:47776/0 (socket says 172.21.15.5:47776) 2024-09-10T10:01:28.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:28.598+0000 7efc01400640 1 -- 172.21.15.5:0/2004569877 learned_addr learned my addr 172.21.15.5:0/2004569877 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T10:01:28.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:28.598+0000 7efc01e00640 1 --2- 172.21.15.5:0/2004569877 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efc04104010 0x7efc0419a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:28.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:28.598+0000 7efc02800640 1 --2- 172.21.15.5:0/2004569877 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efc04109140 0x7efc0419ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:28.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:28.598+0000 7efc01400640 1 -- 172.21.15.5:0/2004569877 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efc04109140 msgr2=0x7efc0419ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:28.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:28.598+0000 7efc01400640 1 --2- 172.21.15.5:0/2004569877 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efc04109140 0x7efc0419ef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:28.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:28.598+0000 7efc01400640 1 -- 172.21.15.5:0/2004569877 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efc04104010 msgr2=0x7efc0419a6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:28.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:28.598+0000 7efc01400640 1 --2- 172.21.15.5:0/2004569877 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efc04104010 0x7efc0419a6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:28.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:28.598+0000 7efc01400640 1 -- 172.21.15.5:0/2004569877 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc0419f660 con 0x7efc041049c0 2024-09-10T10:01:28.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:28.598+0000 7efc01400640 1 --2- 172.21.15.5:0/2004569877 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efc041049c0 0x7efc0419abf0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7efbf0004770 tx=0x7efbf0003c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:28.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:28.599+0000 7efbfaa00640 1 -- 172.21.15.5:0/2004569877 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efbf003f070 con 0x7efc041049c0 2024-09-10T10:01:28.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:28.599+0000 7efc08c00640 1 -- 172.21.15.5:0/2004569877 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc0419f8f0 con 0x7efc041049c0 2024-09-10T10:01:28.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:28.599+0000 7efbfaa00640 1 -- 172.21.15.5:0/2004569877 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efbf0031ca0 con 0x7efc041049c0 2024-09-10T10:01:28.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:28.599+0000 7efbfaa00640 1 -- 172.21.15.5:0/2004569877 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efbf0035030 con 0x7efc041049c0 2024-09-10T10:01:28.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:28.599+0000 7efc08c00640 1 -- 172.21.15.5:0/2004569877 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc041a7230 con 0x7efc041049c0 2024-09-10T10:01:28.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:28.603+0000 7efc0b85d740 1 Processor -- start 2024-09-10T10:01:28.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:28.603+0000 7efc0b85d740 1 -- start start 2024-09-10T10:01:28.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:28.603+0000 7efc0b85d740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d2741d6c70 0x55d2741d7050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:28.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:28.603+0000 7efc0b85d740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d2741d7590 0x55d2741e1910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:28.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:28.603+0000 7efc0b85d740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d2741e1e50 0x55d2741e4240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:28.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:28.603+0000 7efc0b85d740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55d27413d3e0 con 0x55d2741e1e50 2024-09-10T10:01:28.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:28.603+0000 7efc0b85d740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55d2740e4d80 con 0x55d2741d7590 2024-09-10T10:01:28.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:28.603+0000 7efc0b85d740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55d2741393a0 con 0x55d2741d6c70 2024-09-10T10:01:28.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:28.603+0000 7efc01e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d2741d7590 0x55d2741e1910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:28.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:28.603+0000 7efc02800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d2741d6c70 0x55d2741d7050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:28.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:28.604+0000 7efc01e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d2741d7590 0x55d2741e1910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:38374/0 (socket says 172.21.15.5:38374) 2024-09-10T10:01:28.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:28.604+0000 7efc01e00640 1 -- 172.21.15.5:0/1939581014 learned_addr learned my addr 172.21.15.5:0/1939581014 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T10:01:28.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:28.604+0000 7efbfaa00640 1 -- 172.21.15.5:0/2004569877 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7efbf0042070 con 0x7efc041049c0 2024-09-10T10:01:28.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:28.604+0000 7efc01e00640 1 -- 172.21.15.5:0/1939581014 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d2741d6c70 msgr2=0x55d2741d7050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:01:28.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:28.604+0000 7efc01e00640 1 --2- 172.21.15.5:0/1939581014 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d2741d6c70 0x55d2741d7050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:28.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:28.604+0000 7efc01e00640 1 -- 172.21.15.5:0/1939581014 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d2741e1e50 msgr2=0x55d2741e4240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:01:28.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:28.604+0000 7efc01400640 1 --2- 172.21.15.5:0/1939581014 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d2741e1e50 0x55d2741e4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:28.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:28.604+0000 7efc01e00640 1 --2- 172.21.15.5:0/1939581014 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d2741e1e50 0x55d2741e4240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:28.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:28.604+0000 7efc01e00640 1 -- 172.21.15.5:0/1939581014 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d274103ae0 con 0x55d2741d7590 2024-09-10T10:01:28.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:28.604+0000 7efc02800640 1 --2- 172.21.15.5:0/1939581014 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d2741d6c70 0x55d2741d7050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:01:28.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:28.605+0000 7efc01e00640 1 --2- 172.21.15.5:0/1939581014 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d2741d7590 0x55d2741e1910 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7efbf4009d20 tx=0x7efbf4011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bb27e80b05498158 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:28.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:28.605+0000 7efbef000640 1 -- 172.21.15.5:0/1939581014 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efbf401e070 con 0x55d2741d7590 2024-09-10T10:01:28.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:28.605+0000 7efbef000640 1 -- 172.21.15.5:0/1939581014 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efbf4019b10 con 0x55d2741d7590 2024-09-10T10:01:28.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:28.605+0000 7efbef000640 1 -- 172.21.15.5:0/1939581014 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efbf400ebb0 con 0x55d2741d7590 2024-09-10T10:01:28.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:28.606+0000 7efbfaa00640 1 --2- 172.21.15.5:0/2004569877 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7efbe80a2c80 0x7efbe80a5140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:28.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:28.606+0000 7efc0b85d740 1 -- 172.21.15.5:0/1939581014 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d2741d7590 msgr2=0x55d2741e1910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:28.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:28.606+0000 7efc0b85d740 1 --2- 172.21.15.5:0/1939581014 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d2741d7590 0x55d2741e1910 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7efbf4009d20 tx=0x7efbf4011810 comp rx=0 tx=0).stop 2024-09-10T10:01:28.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:28.606+0000 7efbfaa00640 1 -- 172.21.15.5:0/2004569877 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7efbf00ef9a0 con 0x7efc041049c0 2024-09-10T10:01:28.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:28.606+0000 7efc02800640 1 --2- 172.21.15.5:0/2004569877 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7efbe80a2c80 0x7efbe80a5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:28.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:28.607+0000 7efc0b85d740 1 -- 172.21.15.5:0/1939581014 shutdown_connections 2024-09-10T10:01:28.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:28.607+0000 7efc0b85d740 1 --2- 172.21.15.5:0/1939581014 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d2741e1e50 0x55d2741e4240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:28.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:28.607+0000 7efc0b85d740 1 --2- 172.21.15.5:0/1939581014 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d2741d7590 0x55d2741e1910 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:28.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:28.607+0000 7efc0b85d740 1 --2- 172.21.15.5:0/1939581014 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d2741d6c70 0x55d2741d7050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:28.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:28.607+0000 7efc0b85d740 1 -- 172.21.15.5:0/1939581014 >> 172.21.15.5:0/1939581014 conn(0x55d2741d3200 msgr2=0x55d2741d35e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:28.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:28.607+0000 7efc0b85d740 1 -- 172.21.15.5:0/1939581014 shutdown_connections 2024-09-10T10:01:28.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:28.607+0000 7efc02800640 1 --2- 172.21.15.5:0/2004569877 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7efbe80a2c80 0x7efbe80a5140 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7efbfc009fd0 tx=0x7efbfc009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:28.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:28.608+0000 7efc0b85d740 1 -- 172.21.15.5:0/1939581014 wait complete. 2024-09-10T10:01:28.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:28.608+0000 7efc0b85d740 1 -- start start 2024-09-10T10:01:28.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:28.608+0000 7efc0b85d740 1 Processor -- start 2024-09-10T10:01:28.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:28.609+0000 7efc0b85d740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d2741d6c70 0x55d2741ded40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:28.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:28.609+0000 7efc0b85d740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d2741d7590 0x55d2741df280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:28.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:28.609+0000 7efc0b85d740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d2741e1e50 0x55d2741df7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:28.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:28.609+0000 7efc0b85d740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55d2741e4a20 con 0x55d2741d6c70 2024-09-10T10:01:28.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:28.609+0000 7efc0b85d740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55d2740f80f0 con 0x55d2741d7590 2024-09-10T10:01:28.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:28.609+0000 7efc0b85d740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55d2741e4ba0 con 0x55d2741e1e50 2024-09-10T10:01:28.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:28.609+0000 7efc0b85d740 10 client.0 _ll_register_callbacks cb 0x55d2740dfbd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:01:28.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:28.609+0000 7efc01e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d2741d6c70 0x55d2741ded40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:28.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:28.609+0000 7efc01400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d2741d7590 0x55d2741df280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:28.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:28.609+0000 7efc01400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d2741d7590 0x55d2741df280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:38384/0 (socket says 172.21.15.5:38384) 2024-09-10T10:01:28.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:28.609+0000 7efc01400640 1 -- 172.21.15.5:0/3211837993 learned_addr learned my addr 172.21.15.5:0/3211837993 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T10:01:28.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:28.609+0000 7efc02800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d2741e1e50 0x55d2741df7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:28.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:28.610+0000 7efc01400640 1 -- 172.21.15.5:0/3211837993 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d2741e1e50 msgr2=0x55d2741df7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:01:28.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:28.610+0000 7efc01400640 1 --2- 172.21.15.5:0/3211837993 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d2741e1e50 0x55d2741df7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:28.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:28.610+0000 7efc01400640 1 -- 172.21.15.5:0/3211837993 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d2741d6c70 msgr2=0x55d2741ded40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:01:28.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:28.610+0000 7efc01400640 1 --2- 172.21.15.5:0/3211837993 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d2741d6c70 0x55d2741ded40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:28.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:28.610+0000 7efc01400640 1 -- 172.21.15.5:0/3211837993 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d273eae3c0 con 0x55d2741d7590 2024-09-10T10:01:28.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:28.610+0000 7efc02800640 1 --2- 172.21.15.5:0/3211837993 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d2741e1e50 0x55d2741df7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:01:28.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:28.610+0000 7efc01e00640 1 --2- 172.21.15.5:0/3211837993 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d2741d6c70 0x55d2741ded40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:01:28.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:28.610+0000 7efc01400640 1 --2- 172.21.15.5:0/3211837993 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d2741d7590 0x55d2741df280 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7efbf00b88f0 tx=0x7efbf00bbb20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bcf2f37ef0d9bf4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:28.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:28.610+0000 7efbe3400640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-09-10T10:01:28.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:28.611+0000 7efbe3400640 1 -- 172.21.15.5:0/3211837993 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efbf0049070 con 0x55d2741d7590 2024-09-10T10:01:28.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:28.611+0000 7efc0b85d740 1 -- 172.21.15.5:0/3211837993 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efbf4019b10 con 0x55d2741d7590 2024-09-10T10:01:28.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:28.611+0000 7efbe3400640 1 -- 172.21.15.5:0/3211837993 <== mon.2 v2:172.21.15.102:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7efbf00b8590 con 0x55d2741d7590 2024-09-10T10:01:28.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:28.611+0000 7efbe3400640 1 -- 172.21.15.5:0/3211837993 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7efbf00c0220 con 0x55d2741d7590 2024-09-10T10:01:28.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:28.611+0000 7efc0b85d740 10 client.29801 fetch_fsmap learned FSMap version 42 2024-09-10T10:01:28.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:28.611+0000 7efc0b85d740 1 -- 172.21.15.5:0/3211837993 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55d2741f0a60 con 0x55d2741d7590 2024-09-10T10:01:28.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:28.611+0000 7efbe3400640 1 -- 172.21.15.5:0/3211837993 <== mon.2 v2:172.21.15.102:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efbf4019b10 con 0x55d2741d7590 2024-09-10T10:01:28.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:28.612+0000 7efbe3400640 1 -- 172.21.15.5:0/3211837993 <== mon.2 v2:172.21.15.102:3300/0 5 ==== fsmap(e 42) ==== 8370+0+0 (secure 0 0 0) 0x7efbf00c46e0 con 0x55d2741d7590 2024-09-10T10:01:28.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:28.612+0000 7efc0b85d740 10 client.29801 fetch_fsmap finished waiting for FSMap version 42 2024-09-10T10:01:28.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:28.612+0000 7efc0b85d740 10 client.29801 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-10T10:01:28.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:28.612+0000 7efc0b85d740 20 client.29801 populate_metadata read hostname 'smithi005' 2024-09-10T10:01:28.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:28.612+0000 7efc0b85d740 1 --2- 172.21.15.5:0/3211837993 >> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] conn(0x55d2741f1ec0 0x55d2741f4330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:28.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:28.612+0000 7efc0b85d740 4 client.29801 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:01:28.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:28.612+0000 7efc0b85d740 1 -- 172.21.15.5:0/3211837993 --> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d27414a950 con 0x55d2741f1ec0 2024-09-10T10:01:28.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:28.612+0000 7efc01e00640 1 --2- 172.21.15.5:0/3211837993 >> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] conn(0x55d2741f1ec0 0x55d2741f4330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:28.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:28.613+0000 7efc01e00640 1 --2- 172.21.15.5:0/3211837993 >> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] conn(0x55d2741f1ec0 0x55d2741f4330 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b01ed239ae7ef2f7 server_cookie=5348f44e681be4a in_seq=0 out_seq=0 2024-09-10T10:01:28.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:28.613+0000 7efbe3400640 10 client.29801 ms_handle_connect on v2:172.21.15.159:6838/2073739599 2024-09-10T10:01:28.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:28.614+0000 7efbe3400640 1 -- 172.21.15.5:0/3211837993 <== mds.1 v2:172.21.15.159:6838/2073739599 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d27414a950 con 0x55d2741f1ec0 2024-09-10T10:01:28.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:28.614+0000 7efbe3400640 10 client.29801 handle_command_reply: tid=0 2024-09-10T10:01:28.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:28.614+0000 7efc0b85d740 1 client.29801 shutdown 2024-09-10T10:01:28.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:28.615+0000 7efc0b85d740 1 -- 172.21.15.5:0/3211837993 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d2741d7590 msgr2=0x55d2741df280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:28.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:28.615+0000 7efc0b85d740 1 --2- 172.21.15.5:0/3211837993 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d2741d7590 0x55d2741df280 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7efbf00b88f0 tx=0x7efbf00bbb20 comp rx=0 tx=0).stop 2024-09-10T10:01:28.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:28.615+0000 7efc0b85d740 1 -- 172.21.15.5:0/3211837993 shutdown_connections 2024-09-10T10:01:28.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:28.615+0000 7efc0b85d740 1 --2- 172.21.15.5:0/3211837993 >> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] conn(0x55d2741f1ec0 0x55d2741f4330 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:28.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:28.615+0000 7efbe3400640 0 client.29801 ms_handle_reset on v2:172.21.15.159:6838/2073739599 2024-09-10T10:01:28.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:28.615+0000 7efc0b85d740 1 --2- 172.21.15.5:0/3211837993 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d2741e1e50 0x55d2741df7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:28.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:28.615+0000 7efc0b85d740 1 --2- 172.21.15.5:0/3211837993 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d2741d7590 0x55d2741df280 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:28.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:28.615+0000 7efc0b85d740 1 --2- 172.21.15.5:0/3211837993 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d2741d6c70 0x55d2741ded40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:28.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:28.615+0000 7efc0b85d740 1 -- 172.21.15.5:0/3211837993 >> 172.21.15.5:0/3211837993 conn(0x55d2741d3200 msgr2=0x55d2741d2380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:28.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:28.615+0000 7efc0b85d740 1 -- 172.21.15.5:0/3211837993 shutdown_connections 2024-09-10T10:01:28.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:28.615+0000 7efc0b85d740 1 -- 172.21.15.5:0/3211837993 wait complete. 2024-09-10T10:01:28.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:28.615+0000 7efc0b85d740 20 client.29801 trim_cache size 0 max 16384 2024-09-10T10:01:28.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:28.632+0000 7efc0b85d740 1 Processor -- start 2024-09-10T10:01:28.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:28.632+0000 7efc0b85d740 1 -- start start 2024-09-10T10:01:28.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:28.632+0000 7efc0b85d740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d2741d6c70 0x55d2741de600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:28.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:28.633+0000 7efc0b85d740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d2741d7590 0x55d2741f9c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:28.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:28.633+0000 7efc0b85d740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d2741e1e50 0x55d2741fa150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:28.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:28.633+0000 7efc0b85d740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7efc041b1020 con 0x55d2741e1e50 2024-09-10T10:01:28.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:28.633+0000 7efc0b85d740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7efc041b2b60 con 0x55d2741d7590 2024-09-10T10:01:28.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:28.633+0000 7efc0b85d740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7efc041b16f0 con 0x55d2741d6c70 2024-09-10T10:01:28.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:28.633+0000 7efc01e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d2741d6c70 0x55d2741de600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:28.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:28.633+0000 7efc01400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d2741d7590 0x55d2741f9c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:28.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:28.633+0000 7efc01e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d2741d6c70 0x55d2741de600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:47802/0 (socket says 172.21.15.5:47802) 2024-09-10T10:01:28.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:28.633+0000 7efc01e00640 1 -- 172.21.15.5:0/440092518 learned_addr learned my addr 172.21.15.5:0/440092518 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T10:01:28.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:28.633+0000 7efc02800640 1 --2- 172.21.15.5:0/440092518 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d2741e1e50 0x55d2741fa150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:28.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:28.633+0000 7efc01e00640 1 -- 172.21.15.5:0/440092518 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d2741d7590 msgr2=0x55d2741f9c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:01:28.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:28.633+0000 7efc01e00640 1 --2- 172.21.15.5:0/440092518 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d2741d7590 0x55d2741f9c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:28.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:28.633+0000 7efc01e00640 1 -- 172.21.15.5:0/440092518 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d2741e1e50 msgr2=0x55d2741fa150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:01:28.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:28.633+0000 7efc01e00640 1 --2- 172.21.15.5:0/440092518 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d2741e1e50 0x55d2741fa150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:28.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:28.633+0000 7efc01e00640 1 -- 172.21.15.5:0/440092518 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d2741ddcf0 con 0x55d2741d6c70 2024-09-10T10:01:28.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:28.633+0000 7efc01400640 1 --2- 172.21.15.5:0/440092518 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d2741d7590 0x55d2741f9c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:01:28.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:28.634+0000 7efc01e00640 1 --2- 172.21.15.5:0/440092518 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d2741d6c70 0x55d2741de600 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7efbf4000c00 tx=0x7efbf400e950 comp rx=0 tx=0).ready entity=mon.1 client_cookie=df12875776a05b19 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:28.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:28.634+0000 7efbee600640 1 -- 172.21.15.5:0/440092518 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efbf401e040 con 0x55d2741d6c70 2024-09-10T10:01:28.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:28.634+0000 7efbee600640 1 -- 172.21.15.5:0/440092518 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efbf401f040 con 0x55d2741d6c70 2024-09-10T10:01:28.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:28.634+0000 7efbee600640 1 -- 172.21.15.5:0/440092518 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efbf4005550 con 0x55d2741d6c70 2024-09-10T10:01:28.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:28.635+0000 7efc0b85d740 1 -- 172.21.15.5:0/440092518 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d2741d6c70 msgr2=0x55d2741de600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:28.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:28.635+0000 7efc0b85d740 1 --2- 172.21.15.5:0/440092518 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d2741d6c70 0x55d2741de600 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7efbf4000c00 tx=0x7efbf400e950 comp rx=0 tx=0).stop 2024-09-10T10:01:28.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:28.635+0000 7efc0b85d740 1 -- 172.21.15.5:0/440092518 shutdown_connections 2024-09-10T10:01:28.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:28.635+0000 7efc0b85d740 1 --2- 172.21.15.5:0/440092518 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d2741e1e50 0x55d2741fa150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:28.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:28.635+0000 7efc0b85d740 1 --2- 172.21.15.5:0/440092518 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d2741d7590 0x55d2741f9c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:28.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:28.635+0000 7efc0b85d740 1 --2- 172.21.15.5:0/440092518 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d2741d6c70 0x55d2741de600 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:28.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:28.635+0000 7efc0b85d740 1 -- 172.21.15.5:0/440092518 >> 172.21.15.5:0/440092518 conn(0x55d2741d3200 msgr2=0x55d2741e0dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:28.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:28.636+0000 7efc0b85d740 1 -- 172.21.15.5:0/440092518 shutdown_connections 2024-09-10T10:01:28.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:28.636+0000 7efc0b85d740 1 -- 172.21.15.5:0/440092518 wait complete. 2024-09-10T10:01:28.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:28.636+0000 7efc0b85d740 1 -- start start 2024-09-10T10:01:28.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:28.636+0000 7efc0b85d740 1 Processor -- start 2024-09-10T10:01:28.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:28.637+0000 7efc0b85d740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d2741d6c70 0x55d2741ea5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:28.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:28.637+0000 7efc0b85d740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d2741d7590 0x55d2741e5950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:28.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:28.637+0000 7efc0b85d740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d2741e1e50 0x55d2741e5eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:28.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:28.637+0000 7efc0b85d740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7efc041b2490 con 0x55d2741d7590 2024-09-10T10:01:28.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:28.637+0000 7efc0b85d740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7efc041b1dc0 con 0x55d2741d6c70 2024-09-10T10:01:28.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:28.637+0000 7efc0b85d740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7efc041af000 con 0x55d2741e1e50 2024-09-10T10:01:28.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:28.637+0000 7efc0b85d740 10 client.0 _ll_register_callbacks cb 0x7efbf0009260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:01:28.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:28.637+0000 7efc01e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d2741d6c70 0x55d2741ea5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:28.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:28.637+0000 7efc01e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d2741d6c70 0x55d2741ea5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:38394/0 (socket says 172.21.15.5:38394) 2024-09-10T10:01:28.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:28.637+0000 7efc01400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d2741d7590 0x55d2741e5950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:28.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:28.637+0000 7efc01e00640 1 -- 172.21.15.5:0/602452811 learned_addr learned my addr 172.21.15.5:0/602452811 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T10:01:28.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:28.637+0000 7efc02800640 1 --2- 172.21.15.5:0/602452811 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d2741e1e50 0x55d2741e5eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:28.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:28.638+0000 7efc01e00640 1 -- 172.21.15.5:0/602452811 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d2741e1e50 msgr2=0x55d2741e5eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:01:28.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:28.638+0000 7efc01e00640 1 --2- 172.21.15.5:0/602452811 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d2741e1e50 0x55d2741e5eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:28.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:28.638+0000 7efc01e00640 1 -- 172.21.15.5:0/602452811 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d2741d7590 msgr2=0x55d2741e5950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:01:28.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:28.638+0000 7efc01e00640 1 --2- 172.21.15.5:0/602452811 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d2741d7590 0x55d2741e5950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:28.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:28.638+0000 7efc01e00640 1 -- 172.21.15.5:0/602452811 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d2741d73c0 con 0x55d2741d6c70 2024-09-10T10:01:28.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:28.638+0000 7efc02800640 1 --2- 172.21.15.5:0/602452811 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d2741e1e50 0x55d2741e5eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:01:28.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:28.638+0000 7efc01e00640 1 --2- 172.21.15.5:0/602452811 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d2741d6c70 0x55d2741ea5d0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7efbf401c040 tx=0x7efbf4009620 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7b52a9c6f2c7af44 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:28.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:28.638+0000 7efbe3400640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-09-10T10:01:28.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:28.639+0000 7efbe3400640 1 -- 172.21.15.5:0/602452811 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efbf4034020 con 0x55d2741d6c70 2024-09-10T10:01:28.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:28.639+0000 7efc0b85d740 1 -- 172.21.15.5:0/602452811 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efbf401f040 con 0x55d2741d6c70 2024-09-10T10:01:28.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:28.639+0000 7efbe3400640 1 -- 172.21.15.5:0/602452811 <== mon.2 v2:172.21.15.102:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7efbf400ecd0 con 0x55d2741d6c70 2024-09-10T10:01:28.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:28.639+0000 7efbe3400640 1 -- 172.21.15.5:0/602452811 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7efbf401e040 con 0x55d2741d6c70 2024-09-10T10:01:28.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:28.639+0000 7efc0b85d740 10 client.29807 fetch_fsmap learned FSMap version 42 2024-09-10T10:01:28.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:28.639+0000 7efc0b85d740 1 -- 172.21.15.5:0/602452811 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55d27425ca90 con 0x55d2741d6c70 2024-09-10T10:01:28.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:28.639+0000 7efbe3400640 1 -- 172.21.15.5:0/602452811 <== mon.2 v2:172.21.15.102:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efbf401f040 con 0x55d2741d6c70 2024-09-10T10:01:28.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:28.639+0000 7efbe3400640 1 -- 172.21.15.5:0/602452811 <== mon.2 v2:172.21.15.102:3300/0 5 ==== fsmap(e 42) ==== 8370+0+0 (secure 0 0 0) 0x7efbf402f070 con 0x55d2741d6c70 2024-09-10T10:01:28.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:28.640+0000 7efc0b85d740 10 client.29807 fetch_fsmap finished waiting for FSMap version 42 2024-09-10T10:01:28.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:28.640+0000 7efc0b85d740 10 client.29807 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-10T10:01:28.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:28.640+0000 7efc0b85d740 20 client.29807 populate_metadata read hostname 'smithi005' 2024-09-10T10:01:28.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:28.640+0000 7efc0b85d740 1 --2- 172.21.15.5:0/602452811 >> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] conn(0x55d2741f1ec0 0x55d27425ef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:28.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:28.640+0000 7efc0b85d740 4 client.29807 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T10:01:28.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:28.640+0000 7efc0b85d740 1 -- 172.21.15.5:0/602452811 --> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d274144e60 con 0x55d2741f1ec0 2024-09-10T10:01:28.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:28.640+0000 7efc01400640 1 --2- 172.21.15.5:0/602452811 >> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] conn(0x55d2741f1ec0 0x55d27425ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:28.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:28.641+0000 7efc01400640 1 --2- 172.21.15.5:0/602452811 >> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] conn(0x55d2741f1ec0 0x55d27425ef70 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=368bdeb1d76430b4 server_cookie=8811b11862bf11b3 in_seq=0 out_seq=0 2024-09-10T10:01:28.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:28.641+0000 7efbe3400640 10 client.29807 ms_handle_connect on v2:172.21.15.159:6838/2073739599 2024-09-10T10:01:28.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:28.641+0000 7efbe3400640 1 -- 172.21.15.5:0/602452811 <== mds.1 v2:172.21.15.159:6838/2073739599 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d274144e60 con 0x55d2741f1ec0 2024-09-10T10:01:28.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:28.641+0000 7efbe3400640 10 client.29807 handle_command_reply: tid=0 2024-09-10T10:01:28.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:28.641+0000 7efc0b85d740 1 client.29807 shutdown 2024-09-10T10:01:28.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:28.641+0000 7efc0b85d740 1 -- 172.21.15.5:0/602452811 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d2741d6c70 msgr2=0x55d2741ea5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:28.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:28.641+0000 7efc0b85d740 1 --2- 172.21.15.5:0/602452811 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d2741d6c70 0x55d2741ea5d0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7efbf401c040 tx=0x7efbf4009620 comp rx=0 tx=0).stop 2024-09-10T10:01:28.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:28.641+0000 7efc0b85d740 1 -- 172.21.15.5:0/602452811 shutdown_connections 2024-09-10T10:01:28.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:28.641+0000 7efc0b85d740 1 --2- 172.21.15.5:0/602452811 >> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] conn(0x55d2741f1ec0 0x55d27425ef70 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:28.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:28.641+0000 7efc0b85d740 1 --2- 172.21.15.5:0/602452811 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d2741e1e50 0x55d2741e5eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:28.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:28.641+0000 7efc0b85d740 1 --2- 172.21.15.5:0/602452811 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d2741d7590 0x55d2741e5950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:28.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:28.641+0000 7efc0b85d740 1 --2- 172.21.15.5:0/602452811 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d2741d6c70 0x55d2741ea5d0 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:28.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:28.641+0000 7efc0b85d740 1 -- 172.21.15.5:0/602452811 >> 172.21.15.5:0/602452811 conn(0x55d2741d3200 msgr2=0x55d2741d1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:28.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:28.641+0000 7efbe3400640 0 client.29807 ms_handle_reset on v2:172.21.15.159:6838/2073739599 2024-09-10T10:01:28.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:28.642+0000 7efc0b85d740 1 -- 172.21.15.5:0/602452811 shutdown_connections 2024-09-10T10:01:28.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:28.642+0000 7efc0b85d740 1 -- 172.21.15.5:0/602452811 wait complete. 2024-09-10T10:01:28.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:28.642+0000 7efc0b85d740 20 client.29807 trim_cache size 0 max 16384 2024-09-10T10:01:28.645 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-10T10:01:28.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:28.643+0000 7efc08c00640 1 -- 172.21.15.5:0/2004569877 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7efbe80a2c80 msgr2=0x7efbe80a5140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:28.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:28.643+0000 7efc08c00640 1 --2- 172.21.15.5:0/2004569877 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7efbe80a2c80 0x7efbe80a5140 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7efbfc009fd0 tx=0x7efbfc009500 comp rx=0 tx=0).stop 2024-09-10T10:01:28.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:28.643+0000 7efc08c00640 1 -- 172.21.15.5:0/2004569877 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efc041049c0 msgr2=0x7efc0419abf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:28.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:28.643+0000 7efc08c00640 1 --2- 172.21.15.5:0/2004569877 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efc041049c0 0x7efc0419abf0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7efbf0004770 tx=0x7efbf0003c10 comp rx=0 tx=0).stop 2024-09-10T10:01:28.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:28.643+0000 7efc08c00640 1 -- 172.21.15.5:0/2004569877 shutdown_connections 2024-09-10T10:01:28.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:28.643+0000 7efc08c00640 1 --2- 172.21.15.5:0/2004569877 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7efbe80a2c80 0x7efbe80a5140 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:28.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:28.643+0000 7efc08c00640 1 --2- 172.21.15.5:0/2004569877 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efc04109140 0x7efc0419ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:28.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:28.643+0000 7efc08c00640 1 --2- 172.21.15.5:0/2004569877 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efc041049c0 0x7efc0419abf0 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:28.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:28.643+0000 7efc08c00640 1 --2- 172.21.15.5:0/2004569877 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efc04104010 0x7efc0419a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:28.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:28.643+0000 7efc08c00640 1 -- 172.21.15.5:0/2004569877 >> 172.21.15.5:0/2004569877 conn(0x7efc040ffa50 msgr2=0x7efc04100d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:28.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:28.643+0000 7efc08c00640 1 -- 172.21.15.5:0/2004569877 shutdown_connections 2024-09-10T10:01:28.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:28.643+0000 7efc08c00640 1 -- 172.21.15.5:0/2004569877 wait complete. 2024-09-10T10:01:29.041 INFO:tasks.workunit.client.0.smithi005.stdout: 10 145816 13.48 MB/sec execute 427 sec latency 54.691 ms 2024-09-10T10:01:29.436 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:01:29 smithi005 ceph-mon[30350]: pgmap v1087: 129 pgs: 129 active+clean; 715 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 3.1 MiB/s wr, 406 op/s 2024-09-10T10:01:29.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:01:29 smithi159 ceph-mon[35513]: pgmap v1087: 129 pgs: 129 active+clean; 715 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 3.1 MiB/s wr, 406 op/s 2024-09-10T10:01:29.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:01:29 smithi102 ceph-mon[35442]: pgmap v1087: 129 pgs: 129 active+clean; 715 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 3.1 MiB/s wr, 406 op/s 2024-09-10T10:01:29.639 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-10T10:01:30.041 INFO:tasks.workunit.client.0.smithi005.stdout: 10 146422 13.54 MB/sec execute 428 sec latency 62.168 ms 2024-09-10T10:01:30.230 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:01:30.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:30.877+0000 7f4a1aa00640 1 -- 172.21.15.5:0/1873288251 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4a14105d90 msgr2=0x7f4a14106220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:30.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:30.877+0000 7f4a1aa00640 1 --2- 172.21.15.5:0/1873288251 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4a14105d90 0x7f4a14106220 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f4a0c009ca0 tx=0x7f4a0c02a950 comp rx=0 tx=0).stop 2024-09-10T10:01:30.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:30.877+0000 7f4a1aa00640 1 -- 172.21.15.5:0/1873288251 shutdown_connections 2024-09-10T10:01:30.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:30.877+0000 7f4a1aa00640 1 --2- 172.21.15.5:0/1873288251 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4a14105d90 0x7f4a14106220 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:30.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:30.877+0000 7f4a1aa00640 1 --2- 172.21.15.5:0/1873288251 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4a141012a0 0x7f4a14105770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:30.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:30.877+0000 7f4a1aa00640 1 --2- 172.21.15.5:0/1873288251 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4a14100980 0x7f4a14100d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:30.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:30.877+0000 7f4a1aa00640 1 -- 172.21.15.5:0/1873288251 >> 172.21.15.5:0/1873288251 conn(0x7f4a140fc610 msgr2=0x7f4a140fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:30.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:30.878+0000 7f4a1aa00640 1 -- 172.21.15.5:0/1873288251 shutdown_connections 2024-09-10T10:01:30.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:30.878+0000 7f4a1aa00640 1 -- 172.21.15.5:0/1873288251 wait complete. 2024-09-10T10:01:30.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:30.879+0000 7f4a1aa00640 1 Processor -- start 2024-09-10T10:01:30.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:30.880+0000 7f4a1aa00640 1 -- start start 2024-09-10T10:01:30.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:30.880+0000 7f4a1aa00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4a14100980 0x7f4a141a4430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:30.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:30.880+0000 7f4a1aa00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4a141012a0 0x7f4a141a4970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:30.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:30.880+0000 7f4a1aa00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4a14105d90 0x7f4a1419e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:30.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:30.880+0000 7f4a1aa00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f4a141183a0 con 0x7f4a14105d90 2024-09-10T10:01:30.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:30.880+0000 7f4a1aa00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f4a14118220 con 0x7f4a14100980 2024-09-10T10:01:30.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:30.880+0000 7f4a1aa00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f4a14118520 con 0x7f4a141012a0 2024-09-10T10:01:30.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:30.880+0000 7f4a19600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4a14100980 0x7f4a141a4430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:30.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:30.881+0000 7f4a19600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4a14100980 0x7f4a141a4430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:54294/0 (socket says 172.21.15.5:54294) 2024-09-10T10:01:30.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:30.881+0000 7f4a19600640 1 -- 172.21.15.5:0/4042746985 learned_addr learned my addr 172.21.15.5:0/4042746985 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T10:01:30.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:30.881+0000 7f4a1a000640 1 --2- 172.21.15.5:0/4042746985 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4a14105d90 0x7f4a1419e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:30.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:30.881+0000 7f4a19600640 1 -- 172.21.15.5:0/4042746985 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4a141012a0 msgr2=0x7f4a141a4970 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T10:01:30.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:30.881+0000 7f4a19600640 1 --2- 172.21.15.5:0/4042746985 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4a141012a0 0x7f4a141a4970 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:30.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:30.881+0000 7f4a19600640 1 -- 172.21.15.5:0/4042746985 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4a14105d90 msgr2=0x7f4a1419e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:30.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:30.881+0000 7f4a19600640 1 --2- 172.21.15.5:0/4042746985 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4a14105d90 0x7f4a1419e5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:30.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:30.881+0000 7f4a19600640 1 -- 172.21.15.5:0/4042746985 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a1419ed20 con 0x7f4a14100980 2024-09-10T10:01:30.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:30.881+0000 7f4a1a000640 1 --2- 172.21.15.5:0/4042746985 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4a14105d90 0x7f4a1419e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:01:30.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:30.882+0000 7f4a19600640 1 --2- 172.21.15.5:0/4042746985 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4a14100980 0x7f4a141a4430 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f4a0000c920 tx=0x7f4a0000cde0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:30.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:30.882+0000 7f4a12000640 1 -- 172.21.15.5:0/4042746985 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a00007c70 con 0x7f4a14100980 2024-09-10T10:01:30.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:30.882+0000 7f4a12000640 1 -- 172.21.15.5:0/4042746985 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4a00007e10 con 0x7f4a14100980 2024-09-10T10:01:30.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:30.882+0000 7f4a1aa00640 1 -- 172.21.15.5:0/4042746985 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a1419f010 con 0x7f4a14100980 2024-09-10T10:01:30.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:30.882+0000 7f4a12000640 1 -- 172.21.15.5:0/4042746985 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a00018890 con 0x7f4a14100980 2024-09-10T10:01:30.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:30.882+0000 7f4a1aa00640 1 -- 172.21.15.5:0/4042746985 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a141ab300 con 0x7f4a14100980 2024-09-10T10:01:30.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:30.886+0000 7f4a1ec50740 1 Processor -- start 2024-09-10T10:01:30.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:30.886+0000 7f4a1ec50740 1 -- start start 2024-09-10T10:01:30.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:30.886+0000 7f4a12000640 1 -- 172.21.15.5:0/4042746985 <== mon.2 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f4a00018a70 con 0x7f4a14100980 2024-09-10T10:01:30.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:30.886+0000 7f4a1ec50740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d07eb7fe40 0x55d07eb80220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:30.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:30.886+0000 7f4a1ec50740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d07eb80760 0x55d07eb8aae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:30.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:30.886+0000 7f4a1ec50740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d07eb8b020 0x55d07eb8d410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:30.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:30.886+0000 7f4a1ec50740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55d07eae63e0 con 0x55d07eb8b020 2024-09-10T10:01:30.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:30.886+0000 7f4a1ec50740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55d07ea8dd80 con 0x55d07eb80760 2024-09-10T10:01:30.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:30.886+0000 7f4a1ec50740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55d07eae23a0 con 0x55d07eb7fe40 2024-09-10T10:01:30.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:30.887+0000 7f4a1a000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d07eb7fe40 0x55d07eb80220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:30.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:30.887+0000 7f4a1a000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d07eb7fe40 0x55d07eb80220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:47132/0 (socket says 172.21.15.5:47132) 2024-09-10T10:01:30.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:30.887+0000 7f4a1a000640 1 -- 172.21.15.5:0/271789894 learned_addr learned my addr 172.21.15.5:0/271789894 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T10:01:30.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:30.887+0000 7f4a19600640 1 --2- 172.21.15.5:0/271789894 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d07eb80760 0x55d07eb8aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:30.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:30.887+0000 7f4a18c00640 1 --2- 172.21.15.5:0/271789894 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d07eb8b020 0x55d07eb8d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:30.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:30.887+0000 7f4a1a000640 1 -- 172.21.15.5:0/271789894 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d07eb80760 msgr2=0x55d07eb8aae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:01:30.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:30.887+0000 7f4a1a000640 1 --2- 172.21.15.5:0/271789894 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d07eb80760 0x55d07eb8aae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:30.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:30.887+0000 7f4a1a000640 1 -- 172.21.15.5:0/271789894 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d07eb8b020 msgr2=0x55d07eb8d410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:01:30.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:30.887+0000 7f4a1a000640 1 --2- 172.21.15.5:0/271789894 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d07eb8b020 0x55d07eb8d410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:30.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:30.887+0000 7f4a1a000640 1 -- 172.21.15.5:0/271789894 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d07eaacae0 con 0x55d07eb7fe40 2024-09-10T10:01:30.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:30.887+0000 7f4a19600640 1 --2- 172.21.15.5:0/271789894 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d07eb80760 0x55d07eb8aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:01:30.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:30.887+0000 7f4a18c00640 1 --2- 172.21.15.5:0/271789894 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d07eb8b020 0x55d07eb8d410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:01:30.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:30.887+0000 7f4a1a000640 1 --2- 172.21.15.5:0/271789894 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d07eb7fe40 0x55d07eb80220 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f4a0c02fa40 tx=0x7f4a0c02f3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=85641f414af1a028 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:30.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:30.888+0000 7f4a06600640 1 -- 172.21.15.5:0/271789894 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a0c034040 con 0x55d07eb7fe40 2024-09-10T10:01:30.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:30.888+0000 7f4a06600640 1 -- 172.21.15.5:0/271789894 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4a0c003970 con 0x55d07eb7fe40 2024-09-10T10:01:30.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:30.888+0000 7f4a06600640 1 -- 172.21.15.5:0/271789894 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a0c0045a0 con 0x55d07eb7fe40 2024-09-10T10:01:30.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:30.888+0000 7f4a12000640 1 --2- 172.21.15.5:0/4042746985 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f49fc0a2c00 0x7f49fc0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:30.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:30.889+0000 7f4a12000640 1 -- 172.21.15.5:0/4042746985 <== mon.2 v2:172.21.15.102:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f4a000cf660 con 0x7f4a14100980 2024-09-10T10:01:30.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:30.889+0000 7f4a1ec50740 1 -- 172.21.15.5:0/271789894 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d07eb7fe40 msgr2=0x55d07eb80220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:30.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:30.889+0000 7f4a1ec50740 1 --2- 172.21.15.5:0/271789894 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d07eb7fe40 0x55d07eb80220 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f4a0c02fa40 tx=0x7f4a0c02f3a0 comp rx=0 tx=0).stop 2024-09-10T10:01:30.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:30.889+0000 7f4a19600640 1 --2- 172.21.15.5:0/4042746985 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f49fc0a2c00 0x7f49fc0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:30.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:30.889+0000 7f4a1ec50740 1 -- 172.21.15.5:0/271789894 shutdown_connections 2024-09-10T10:01:30.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:30.889+0000 7f4a1ec50740 1 --2- 172.21.15.5:0/271789894 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d07eb8b020 0x55d07eb8d410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:30.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:30.889+0000 7f4a1ec50740 1 --2- 172.21.15.5:0/271789894 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d07eb80760 0x55d07eb8aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:30.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:30.889+0000 7f4a1ec50740 1 --2- 172.21.15.5:0/271789894 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d07eb7fe40 0x55d07eb80220 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:30.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:30.889+0000 7f4a1ec50740 1 -- 172.21.15.5:0/271789894 >> 172.21.15.5:0/271789894 conn(0x55d07eb7c3d0 msgr2=0x55d07eb7c7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:30.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:30.889+0000 7f4a1ec50740 1 -- 172.21.15.5:0/271789894 shutdown_connections 2024-09-10T10:01:30.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:30.889+0000 7f4a1ec50740 1 -- 172.21.15.5:0/271789894 wait complete. 2024-09-10T10:01:30.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:30.890+0000 7f4a1ec50740 1 -- start start 2024-09-10T10:01:30.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:30.890+0000 7f4a19600640 1 --2- 172.21.15.5:0/4042746985 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f49fc0a2c00 0x7f49fc0a50c0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f4a00097ed0 tx=0x7f4a00000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:30.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:30.890+0000 7f4a1ec50740 1 Processor -- start 2024-09-10T10:01:30.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:30.891+0000 7f4a1ec50740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d07eb7fe40 0x55d07eb87ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:30.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:30.891+0000 7f4a1ec50740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d07eb80760 0x55d07eb88420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:30.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:30.891+0000 7f4a1ec50740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d07eb8b020 0x55d07eb88960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:30.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:30.891+0000 7f4a1ec50740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55d07eb8dbf0 con 0x55d07eb80760 2024-09-10T10:01:30.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:30.891+0000 7f4a1ec50740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55d07eaa10f0 con 0x55d07eb7fe40 2024-09-10T10:01:30.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:30.891+0000 7f4a1ec50740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55d07eb8dd70 con 0x55d07eb8b020 2024-09-10T10:01:30.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:30.891+0000 7f4a1ec50740 10 client.0 _ll_register_callbacks cb 0x55d07ea88bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:01:30.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:30.891+0000 7f4a1a000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d07eb7fe40 0x55d07eb87ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:30.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:30.891+0000 7f4a1a000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d07eb7fe40 0x55d07eb87ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:54308/0 (socket says 172.21.15.5:54308) 2024-09-10T10:01:30.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:30.891+0000 7f4a1a000640 1 -- 172.21.15.5:0/3585478661 learned_addr learned my addr 172.21.15.5:0/3585478661 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T10:01:30.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:30.891+0000 7f4a1a000640 1 --2- 172.21.15.5:0/3585478661 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d07eb8b020 0x55d07eb88960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:30.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:30.892+0000 7f4a18c00640 1 --2- 172.21.15.5:0/3585478661 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d07eb80760 0x55d07eb88420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:30.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:30.892+0000 7f4a1a000640 1 -- 172.21.15.5:0/3585478661 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d07eb8b020 msgr2=0x55d07eb88960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:01:30.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:30.892+0000 7f4a1a000640 1 --2- 172.21.15.5:0/3585478661 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d07eb8b020 0x55d07eb88960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:30.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:30.892+0000 7f4a1a000640 1 -- 172.21.15.5:0/3585478661 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d07eb80760 msgr2=0x55d07eb88420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:01:30.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:30.892+0000 7f4a1a000640 1 --2- 172.21.15.5:0/3585478661 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d07eb80760 0x55d07eb88420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:30.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:30.892+0000 7f4a1a000640 1 -- 172.21.15.5:0/3585478661 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d07e8573c0 con 0x55d07eb7fe40 2024-09-10T10:01:30.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:30.892+0000 7f4a18c00640 1 --2- 172.21.15.5:0/3585478661 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d07eb80760 0x55d07eb88420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:01:30.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:30.892+0000 7f4a1a000640 1 --2- 172.21.15.5:0/3585478661 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d07eb7fe40 0x55d07eb87ee0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f4a0c003f90 tx=0x7f4a0c004180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=258e1cde13da9bfe server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:30.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:30.892+0000 7f49faa00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-09-10T10:01:30.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:30.893+0000 7f49faa00640 1 -- 172.21.15.5:0/3585478661 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a0c004520 con 0x55d07eb7fe40 2024-09-10T10:01:30.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:30.893+0000 7f49faa00640 1 -- 172.21.15.5:0/3585478661 <== mon.2 v2:172.21.15.102:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f4a0c0046c0 con 0x55d07eb7fe40 2024-09-10T10:01:30.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:30.893+0000 7f4a1ec50740 1 -- 172.21.15.5:0/3585478661 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4a0c003970 con 0x55d07eb7fe40 2024-09-10T10:01:30.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:30.893+0000 7f49faa00640 1 -- 172.21.15.5:0/3585478661 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f4a0c053480 con 0x55d07eb7fe40 2024-09-10T10:01:30.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:30.894+0000 7f4a1ec50740 10 client.29819 fetch_fsmap learned FSMap version 42 2024-09-10T10:01:30.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:30.894+0000 7f4a1ec50740 1 -- 172.21.15.5:0/3585478661 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55d07eb8dbf0 con 0x55d07eb7fe40 2024-09-10T10:01:30.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:30.894+0000 7f49faa00640 1 -- 172.21.15.5:0/3585478661 <== mon.2 v2:172.21.15.102:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4a0c003970 con 0x55d07eb7fe40 2024-09-10T10:01:30.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:30.895+0000 7f49faa00640 1 -- 172.21.15.5:0/3585478661 <== mon.2 v2:172.21.15.102:3300/0 5 ==== fsmap(e 42) ==== 8370+0+0 (secure 0 0 0) 0x7f4a0c042070 con 0x55d07eb7fe40 2024-09-10T10:01:30.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:30.895+0000 7f4a1ec50740 10 client.29819 fetch_fsmap finished waiting for FSMap version 42 2024-09-10T10:01:30.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:30.895+0000 7f4a1ec50740 10 client.29819 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T10:01:30.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:30.895+0000 7f4a1ec50740 20 client.29819 populate_metadata read hostname 'smithi005' 2024-09-10T10:01:30.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:30.895+0000 7f4a1ec50740 1 --2- 172.21.15.5:0/3585478661 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55d07eb9ad80 0x55d07eb9d170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:30.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:30.895+0000 7f4a1ec50740 4 client.29819 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:01:30.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:30.895+0000 7f4a1ec50740 1 -- 172.21.15.5:0/3585478661 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d07eaf3950 con 0x55d07eb9ad80 2024-09-10T10:01:30.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:30.895+0000 7f4a18c00640 1 --2- 172.21.15.5:0/3585478661 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55d07eb9ad80 0x55d07eb9d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:30.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:30.896+0000 7f4a18c00640 1 --2- 172.21.15.5:0/3585478661 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55d07eb9ad80 0x55d07eb9d170 crc :-1 s=READY pgs=313 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bc84acd7e2cf4d00 server_cookie=dd54d2476222d2b0 in_seq=0 out_seq=0 2024-09-10T10:01:30.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:30.896+0000 7f49faa00640 10 client.29819 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T10:01:30.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:30.898+0000 7f49faa00640 1 -- 172.21.15.5:0/3585478661 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d07eaf3950 con 0x55d07eb9ad80 2024-09-10T10:01:30.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:30.898+0000 7f49faa00640 10 client.29819 handle_command_reply: tid=0 2024-09-10T10:01:30.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:30.898+0000 7f4a1ec50740 1 client.29819 shutdown 2024-09-10T10:01:30.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:30.898+0000 7f4a1ec50740 1 -- 172.21.15.5:0/3585478661 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d07eb7fe40 msgr2=0x55d07eb87ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:30.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:30.898+0000 7f4a1ec50740 1 --2- 172.21.15.5:0/3585478661 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d07eb7fe40 0x55d07eb87ee0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f4a0c003f90 tx=0x7f4a0c004180 comp rx=0 tx=0).stop 2024-09-10T10:01:30.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:30.899+0000 7f4a1ec50740 1 -- 172.21.15.5:0/3585478661 shutdown_connections 2024-09-10T10:01:30.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:30.899+0000 7f4a1ec50740 1 --2- 172.21.15.5:0/3585478661 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55d07eb9ad80 0x55d07eb9d170 crc :-1 s=READY pgs=313 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:30.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:30.899+0000 7f4a1ec50740 1 --2- 172.21.15.5:0/3585478661 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d07eb8b020 0x55d07eb88960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:30.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:30.899+0000 7f4a1ec50740 1 --2- 172.21.15.5:0/3585478661 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d07eb80760 0x55d07eb88420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:30.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:30.899+0000 7f4a1ec50740 1 --2- 172.21.15.5:0/3585478661 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d07eb7fe40 0x55d07eb87ee0 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:30.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:30.899+0000 7f4a1ec50740 1 -- 172.21.15.5:0/3585478661 >> 172.21.15.5:0/3585478661 conn(0x55d07eb7c3d0 msgr2=0x55d07eb820b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:30.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:30.899+0000 7f49faa00640 0 client.29819 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T10:01:30.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:30.899+0000 7f4a1ec50740 1 -- 172.21.15.5:0/3585478661 shutdown_connections 2024-09-10T10:01:30.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:30.900+0000 7f4a1ec50740 1 -- 172.21.15.5:0/3585478661 wait complete. 2024-09-10T10:01:30.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:30.900+0000 7f4a1ec50740 20 client.29819 trim_cache size 0 max 16384 2024-09-10T10:01:30.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:30.921+0000 7f4a1ec50740 1 Processor -- start 2024-09-10T10:01:30.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:30.921+0000 7f4a1ec50740 1 -- start start 2024-09-10T10:01:30.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:30.921+0000 7f4a1ec50740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d07eb7fe40 0x55d07eba2800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:30.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:30.921+0000 7f4a1ec50740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d07eb80760 0x55d07eba2d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:30.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:30.921+0000 7f4a1ec50740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d07eb8b020 0x55d07eb87ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:30.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:30.921+0000 7f4a1ec50740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f4a141b5250 con 0x55d07eb80760 2024-09-10T10:01:30.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:30.921+0000 7f4a1ec50740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f4a141b6d90 con 0x55d07eb7fe40 2024-09-10T10:01:30.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:30.921+0000 7f4a1ec50740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f4a141b5920 con 0x55d07eb8b020 2024-09-10T10:01:30.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:30.922+0000 7f4a1a000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d07eb7fe40 0x55d07eba2800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:30.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:30.922+0000 7f4a18c00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d07eb80760 0x55d07eba2d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:30.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:30.922+0000 7f4a1a000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d07eb7fe40 0x55d07eba2800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:54312/0 (socket says 172.21.15.5:54312) 2024-09-10T10:01:30.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:30.922+0000 7f4a1a000640 1 -- 172.21.15.5:0/1630188568 learned_addr learned my addr 172.21.15.5:0/1630188568 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T10:01:30.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:30.922+0000 7f4a1a000640 1 --2- 172.21.15.5:0/1630188568 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d07eb8b020 0x55d07eb87ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:30.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:30.922+0000 7f4a18c00640 1 -- 172.21.15.5:0/1630188568 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d07eb8b020 msgr2=0x55d07eb87ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:01:30.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:30.922+0000 7f4a18c00640 1 --2- 172.21.15.5:0/1630188568 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d07eb8b020 0x55d07eb87ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:30.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:30.922+0000 7f4a18c00640 1 -- 172.21.15.5:0/1630188568 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d07eb7fe40 msgr2=0x55d07eba2800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:01:30.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:30.922+0000 7f4a18c00640 1 --2- 172.21.15.5:0/1630188568 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d07eb7fe40 0x55d07eba2800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:30.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:30.922+0000 7f4a18c00640 1 -- 172.21.15.5:0/1630188568 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d07eb8a850 con 0x55d07eb80760 2024-09-10T10:01:30.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:30.923+0000 7f4a18c00640 1 --2- 172.21.15.5:0/1630188568 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d07eb80760 0x55d07eba2d40 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x55d07eb89520 tx=0x7f4a08005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7c0572d8a394beb server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:30.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:30.923+0000 7f4a05c00640 1 -- 172.21.15.5:0/1630188568 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a080140d0 con 0x55d07eb80760 2024-09-10T10:01:30.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:30.923+0000 7f4a05c00640 1 -- 172.21.15.5:0/1630188568 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4a08014270 con 0x55d07eb80760 2024-09-10T10:01:30.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:30.923+0000 7f4a05c00640 1 -- 172.21.15.5:0/1630188568 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a08012920 con 0x55d07eb80760 2024-09-10T10:01:30.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:30.924+0000 7f4a1ec50740 1 -- 172.21.15.5:0/1630188568 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d07eb80760 msgr2=0x55d07eba2d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:30.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:30.924+0000 7f4a1ec50740 1 --2- 172.21.15.5:0/1630188568 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d07eb80760 0x55d07eba2d40 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x55d07eb89520 tx=0x7f4a08005f90 comp rx=0 tx=0).stop 2024-09-10T10:01:30.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:30.924+0000 7f4a1ec50740 1 -- 172.21.15.5:0/1630188568 shutdown_connections 2024-09-10T10:01:30.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:30.924+0000 7f4a1ec50740 1 --2- 172.21.15.5:0/1630188568 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d07eb8b020 0x55d07eb87ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:30.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:30.924+0000 7f4a1ec50740 1 --2- 172.21.15.5:0/1630188568 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d07eb80760 0x55d07eba2d40 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:30.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:30.924+0000 7f4a1ec50740 1 --2- 172.21.15.5:0/1630188568 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d07eb7fe40 0x55d07eba2800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:30.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:30.924+0000 7f4a1ec50740 1 -- 172.21.15.5:0/1630188568 >> 172.21.15.5:0/1630188568 conn(0x55d07eb7c3d0 msgr2=0x55d07eb86fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:30.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:30.925+0000 7f4a1ec50740 1 -- 172.21.15.5:0/1630188568 shutdown_connections 2024-09-10T10:01:30.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:30.925+0000 7f4a1ec50740 1 -- 172.21.15.5:0/1630188568 wait complete. 2024-09-10T10:01:30.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:30.925+0000 7f4a1ec50740 1 -- start start 2024-09-10T10:01:30.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:30.925+0000 7f4a1ec50740 1 Processor -- start 2024-09-10T10:01:30.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:30.926+0000 7f4a1ec50740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d07eb7fe40 0x55d07eb93860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:30.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:30.926+0000 7f4a1ec50740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d07eb80760 0x55d07eb84d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:30.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:30.926+0000 7f4a1ec50740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d07eb8b020 0x55d07eb854e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:30.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:30.926+0000 7f4a1ec50740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f4a141b66c0 con 0x55d07eb8b020 2024-09-10T10:01:30.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:30.926+0000 7f4a1ec50740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f4a141b5ff0 con 0x55d07eb7fe40 2024-09-10T10:01:30.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:30.926+0000 7f4a1ec50740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f4a141b3230 con 0x55d07eb80760 2024-09-10T10:01:30.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:30.926+0000 7f4a1ec50740 10 client.0 _ll_register_callbacks cb 0x7f4a0c009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:01:30.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:30.926+0000 7f4a1a000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d07eb7fe40 0x55d07eb93860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:30.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:30.926+0000 7f4a18c00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d07eb80760 0x55d07eb84d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:30.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:30.927+0000 7f4a1a000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d07eb7fe40 0x55d07eb93860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:54318/0 (socket says 172.21.15.5:54318) 2024-09-10T10:01:30.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:30.927+0000 7f4a1a000640 1 -- 172.21.15.5:0/3449388338 learned_addr learned my addr 172.21.15.5:0/3449388338 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T10:01:30.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:30.927+0000 7f4a18c00640 1 --2- 172.21.15.5:0/3449388338 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d07eb80760 0x55d07eb84d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:47150/0 (socket says 172.21.15.5:47150) 2024-09-10T10:01:30.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:30.927+0000 7f4a1a000640 1 --2- 172.21.15.5:0/3449388338 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d07eb8b020 0x55d07eb854e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:30.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:30.927+0000 7f4a1a000640 1 -- 172.21.15.5:0/3449388338 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d07eb80760 msgr2=0x55d07eb84d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:01:30.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:30.927+0000 7f4a1a000640 1 --2- 172.21.15.5:0/3449388338 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d07eb80760 0x55d07eb84d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:30.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:30.927+0000 7f4a1a000640 1 -- 172.21.15.5:0/3449388338 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d07eb7fe40 msgr2=0x55d07eb93860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:01:30.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:30.927+0000 7f4a1a000640 1 --2- 172.21.15.5:0/3449388338 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d07eb7fe40 0x55d07eb93860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:30.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:30.927+0000 7f4a1a000640 1 -- 172.21.15.5:0/3449388338 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d07ec047d0 con 0x55d07eb8b020 2024-09-10T10:01:30.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:30.927+0000 7f4a18c00640 1 --2- 172.21.15.5:0/3449388338 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d07eb80760 0x55d07eb84d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:01:30.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:30.928+0000 7f4a1a000640 1 --2- 172.21.15.5:0/3449388338 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d07eb8b020 0x55d07eb854e0 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f4a0c009ca0 tx=0x7f4a0c02a950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58fe2ad34963fd85 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:30.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:30.928+0000 7f49faa00640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-10T10:01:30.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:30.928+0000 7f49faa00640 1 -- 172.21.15.5:0/3449388338 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a0c034030 con 0x55d07eb8b020 2024-09-10T10:01:30.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:30.928+0000 7f49faa00640 1 -- 172.21.15.5:0/3449388338 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f4a0c0041f0 con 0x55d07eb8b020 2024-09-10T10:01:30.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:30.928+0000 7f4a1ec50740 1 -- 172.21.15.5:0/3449388338 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4a08014270 con 0x55d07eb8b020 2024-09-10T10:01:30.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:30.929+0000 7f49faa00640 1 -- 172.21.15.5:0/3449388338 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f4a0c009540 con 0x55d07eb8b020 2024-09-10T10:01:30.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:30.929+0000 7f4a1ec50740 10 client.19737 fetch_fsmap learned FSMap version 42 2024-09-10T10:01:30.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:30.929+0000 7f4a1ec50740 1 -- 172.21.15.5:0/3449388338 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55d07ec05920 con 0x55d07eb8b020 2024-09-10T10:01:30.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:30.929+0000 7f49faa00640 1 -- 172.21.15.5:0/3449388338 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4a08014270 con 0x55d07eb8b020 2024-09-10T10:01:30.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:30.929+0000 7f49faa00640 1 -- 172.21.15.5:0/3449388338 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 42) ==== 8370+0+0 (secure 0 0 0) 0x7f4a0c067070 con 0x55d07eb8b020 2024-09-10T10:01:30.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:30.929+0000 7f4a1ec50740 10 client.19737 fetch_fsmap finished waiting for FSMap version 42 2024-09-10T10:01:30.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:30.929+0000 7f4a1ec50740 10 client.19737 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T10:01:30.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:30.929+0000 7f4a1ec50740 20 client.19737 populate_metadata read hostname 'smithi005' 2024-09-10T10:01:30.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:30.929+0000 7f4a1ec50740 1 --2- 172.21.15.5:0/3449388338 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55d07ec060e0 0x55d07ec08500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:30.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:30.930+0000 7f4a1ec50740 4 client.19737 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T10:01:30.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:30.930+0000 7f4a1ec50740 1 -- 172.21.15.5:0/3449388338 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d07eaede60 con 0x55d07ec060e0 2024-09-10T10:01:30.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:30.930+0000 7f4a18c00640 1 --2- 172.21.15.5:0/3449388338 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55d07ec060e0 0x55d07ec08500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:30.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:30.931+0000 7f4a18c00640 1 --2- 172.21.15.5:0/3449388338 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55d07ec060e0 0x55d07ec08500 crc :-1 s=READY pgs=314 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6df6e56320c1b0a8 server_cookie=61e60b8d046a6c2 in_seq=0 out_seq=0 2024-09-10T10:01:30.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:30.931+0000 7f49faa00640 10 client.19737 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T10:01:30.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:30.931+0000 7f49faa00640 1 -- 172.21.15.5:0/3449388338 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d07eaede60 con 0x55d07ec060e0 2024-09-10T10:01:30.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:30.932+0000 7f49faa00640 10 client.19737 handle_command_reply: tid=0 2024-09-10T10:01:30.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:30.932+0000 7f4a1ec50740 1 client.19737 shutdown 2024-09-10T10:01:30.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:30.932+0000 7f4a1ec50740 1 -- 172.21.15.5:0/3449388338 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d07eb8b020 msgr2=0x55d07eb854e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:30.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:30.932+0000 7f4a1ec50740 1 --2- 172.21.15.5:0/3449388338 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d07eb8b020 0x55d07eb854e0 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f4a0c009ca0 tx=0x7f4a0c02a950 comp rx=0 tx=0).stop 2024-09-10T10:01:30.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:30.932+0000 7f4a1ec50740 1 -- 172.21.15.5:0/3449388338 shutdown_connections 2024-09-10T10:01:30.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:30.932+0000 7f4a1ec50740 1 --2- 172.21.15.5:0/3449388338 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55d07ec060e0 0x55d07ec08500 crc :-1 s=READY pgs=314 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:30.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:30.932+0000 7f4a1ec50740 1 --2- 172.21.15.5:0/3449388338 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d07eb8b020 0x55d07eb854e0 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:30.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:30.932+0000 7f4a1ec50740 1 --2- 172.21.15.5:0/3449388338 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d07eb80760 0x55d07eb84d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:30.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:30.932+0000 7f4a1ec50740 1 --2- 172.21.15.5:0/3449388338 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d07eb7fe40 0x55d07eb93860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:30.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:30.932+0000 7f49faa00640 0 client.19737 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T10:01:30.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:30.932+0000 7f4a1ec50740 1 -- 172.21.15.5:0/3449388338 >> 172.21.15.5:0/3449388338 conn(0x55d07eb7c3d0 msgr2=0x55d07eb81800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:30.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:30.932+0000 7f4a1ec50740 1 -- 172.21.15.5:0/3449388338 shutdown_connections 2024-09-10T10:01:30.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:30.932+0000 7f4a1ec50740 1 -- 172.21.15.5:0/3449388338 wait complete. 2024-09-10T10:01:30.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:30.933+0000 7f4a1ec50740 20 client.19737 trim_cache size 0 max 16384 2024-09-10T10:01:30.935 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-10T10:01:30.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:30.934+0000 7f4a1aa00640 1 -- 172.21.15.5:0/4042746985 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f49fc0a2c00 msgr2=0x7f49fc0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:30.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:30.934+0000 7f4a1aa00640 1 --2- 172.21.15.5:0/4042746985 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f49fc0a2c00 0x7f49fc0a50c0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f4a00097ed0 tx=0x7f4a00000bc0 comp rx=0 tx=0).stop 2024-09-10T10:01:30.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:30.934+0000 7f4a1aa00640 1 -- 172.21.15.5:0/4042746985 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4a14100980 msgr2=0x7f4a141a4430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:30.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:30.934+0000 7f4a1aa00640 1 --2- 172.21.15.5:0/4042746985 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4a14100980 0x7f4a141a4430 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f4a0000c920 tx=0x7f4a0000cde0 comp rx=0 tx=0).stop 2024-09-10T10:01:30.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:30.934+0000 7f4a1aa00640 1 -- 172.21.15.5:0/4042746985 shutdown_connections 2024-09-10T10:01:30.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:30.934+0000 7f4a1aa00640 1 --2- 172.21.15.5:0/4042746985 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f49fc0a2c00 0x7f49fc0a50c0 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:30.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:30.934+0000 7f4a1aa00640 1 --2- 172.21.15.5:0/4042746985 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4a14105d90 0x7f4a1419e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:30.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:30.934+0000 7f4a1aa00640 1 --2- 172.21.15.5:0/4042746985 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4a141012a0 0x7f4a141a4970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:30.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:30.934+0000 7f4a1aa00640 1 --2- 172.21.15.5:0/4042746985 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4a14100980 0x7f4a141a4430 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:30.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:30.934+0000 7f4a1aa00640 1 -- 172.21.15.5:0/4042746985 >> 172.21.15.5:0/4042746985 conn(0x7f4a140fc610 msgr2=0x7f4a14103710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:30.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:30.934+0000 7f4a1aa00640 1 -- 172.21.15.5:0/4042746985 shutdown_connections 2024-09-10T10:01:30.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:30.935+0000 7f4a1aa00640 1 -- 172.21.15.5:0/4042746985 wait complete. 2024-09-10T10:01:31.041 INFO:tasks.workunit.client.0.smithi005.stdout: 10 147014 13.58 MB/sec execute 429 sec latency 56.708 ms 2024-09-10T10:01:31.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:01:31 smithi005 ceph-mon[30350]: pgmap v1088: 129 pgs: 129 active+clean; 727 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 4.0 MiB/s wr, 511 op/s 2024-09-10T10:01:31.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:01:31 smithi159 ceph-mon[35513]: pgmap v1088: 129 pgs: 129 active+clean; 727 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 4.0 MiB/s wr, 511 op/s 2024-09-10T10:01:31.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:01:31 smithi102 ceph-mon[35442]: pgmap v1088: 129 pgs: 129 active+clean; 727 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 4.0 MiB/s wr, 511 op/s 2024-09-10T10:01:32.041 INFO:tasks.workunit.client.0.smithi005.stdout: 10 147649 13.60 MB/sec execute 430 sec latency 66.426 ms 2024-09-10T10:01:33.042 INFO:tasks.workunit.client.0.smithi005.stdout: 10 148351 13.66 MB/sec execute 431 sec latency 61.754 ms 2024-09-10T10:01:33.287 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-10T10:01:33.287 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-10T10:01:33.557 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:01:33 smithi005 ceph-mon[30350]: pgmap v1089: 129 pgs: 129 active+clean; 730 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 3.6 MiB/s wr, 447 op/s 2024-09-10T10:01:33.558 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:01:33 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:01:33.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:01:33 smithi159 ceph-mon[35513]: pgmap v1089: 129 pgs: 129 active+clean; 730 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 3.6 MiB/s wr, 447 op/s 2024-09-10T10:01:33.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:01:33 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:01:33.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:01:33 smithi102 ceph-mon[35442]: pgmap v1089: 129 pgs: 129 active+clean; 730 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 3.6 MiB/s wr, 447 op/s 2024-09-10T10:01:33.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:01:33 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:01:33.875 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:01:34.042 INFO:tasks.workunit.client.0.smithi005.stdout: 10 148983 13.72 MB/sec execute 432 sec latency 160.685 ms 2024-09-10T10:01:34.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:34.511+0000 7ff5a7e00640 1 -- 172.21.15.5:0/697547302 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff5a8114ab0 msgr2=0x7ff5a8116ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:34.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:34.511+0000 7ff5a7e00640 1 --2- 172.21.15.5:0/697547302 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff5a8114ab0 0x7ff5a8116ea0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7ff5a0009c10 tx=0x7ff5a002a6f0 comp rx=0 tx=0).stop 2024-09-10T10:01:34.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:34.511+0000 7ff5a7e00640 1 -- 172.21.15.5:0/697547302 shutdown_connections 2024-09-10T10:01:34.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:34.511+0000 7ff5a7e00640 1 --2- 172.21.15.5:0/697547302 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff5a8114ab0 0x7ff5a8116ea0 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:34.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:34.511+0000 7ff5a7e00640 1 --2- 172.21.15.5:0/697547302 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff5a806ee10 0x7ff5a8114570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:34.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:34.511+0000 7ff5a7e00640 1 --2- 172.21.15.5:0/697547302 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff5a806e4f0 0x7ff5a806e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:34.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:34.511+0000 7ff5a7e00640 1 -- 172.21.15.5:0/697547302 >> 172.21.15.5:0/697547302 conn(0x7ff5a8073180 msgr2=0x7ff5a8073590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:34.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:34.511+0000 7ff5a7e00640 1 -- 172.21.15.5:0/697547302 shutdown_connections 2024-09-10T10:01:34.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:34.511+0000 7ff5a7e00640 1 -- 172.21.15.5:0/697547302 wait complete. 2024-09-10T10:01:34.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:34.513+0000 7ff5a7e00640 1 Processor -- start 2024-09-10T10:01:34.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:34.513+0000 7ff5a7e00640 1 -- start start 2024-09-10T10:01:34.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:34.513+0000 7ff5a7e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff5a806e4f0 0x7ff5a81a67f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:34.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:34.513+0000 7ff5a7e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff5a806ee10 0x7ff5a81a6d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:34.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:34.513+0000 7ff5a7e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff5a8114ab0 0x7ff5a81a09b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:34.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:34.513+0000 7ff5a7e00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff5a811a5c0 con 0x7ff5a8114ab0 2024-09-10T10:01:34.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:34.513+0000 7ff5a7e00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7ff5a811a440 con 0x7ff5a806ee10 2024-09-10T10:01:34.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:34.513+0000 7ff5a7e00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7ff5a811a740 con 0x7ff5a806e4f0 2024-09-10T10:01:34.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:34.513+0000 7ff5a6a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff5a806e4f0 0x7ff5a81a67f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:34.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:34.514+0000 7ff5a6a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff5a806e4f0 0x7ff5a81a67f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:47172/0 (socket says 172.21.15.5:47172) 2024-09-10T10:01:34.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:34.514+0000 7ff5a6a00640 1 -- 172.21.15.5:0/2115953498 learned_addr learned my addr 172.21.15.5:0/2115953498 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T10:01:34.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:34.514+0000 7ff5a7400640 1 --2- 172.21.15.5:0/2115953498 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff5a8114ab0 0x7ff5a81a09b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:34.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:34.514+0000 7ff5a6000640 1 --2- 172.21.15.5:0/2115953498 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff5a806ee10 0x7ff5a81a6d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:34.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:34.514+0000 7ff5a6a00640 1 -- 172.21.15.5:0/2115953498 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff5a806ee10 msgr2=0x7ff5a81a6d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:34.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:34.514+0000 7ff5a6a00640 1 --2- 172.21.15.5:0/2115953498 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff5a806ee10 0x7ff5a81a6d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:34.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:34.514+0000 7ff5a6a00640 1 -- 172.21.15.5:0/2115953498 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff5a8114ab0 msgr2=0x7ff5a81a09b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:34.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:34.514+0000 7ff5a6a00640 1 --2- 172.21.15.5:0/2115953498 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff5a8114ab0 0x7ff5a81a09b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:34.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:34.514+0000 7ff5a6a00640 1 -- 172.21.15.5:0/2115953498 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5a81a10e0 con 0x7ff5a806e4f0 2024-09-10T10:01:34.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:34.514+0000 7ff5a6000640 1 --2- 172.21.15.5:0/2115953498 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff5a806ee10 0x7ff5a81a6d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:01:34.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:34.515+0000 7ff5a6a00640 1 --2- 172.21.15.5:0/2115953498 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff5a806e4f0 0x7ff5a81a67f0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7ff59800e9c0 tx=0x7ff59800ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:34.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:34.515+0000 7ff59f400640 1 -- 172.21.15.5:0/2115953498 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff59800cde0 con 0x7ff5a806e4f0 2024-09-10T10:01:34.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:34.515+0000 7ff59f400640 1 -- 172.21.15.5:0/2115953498 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff5980035f0 con 0x7ff5a806e4f0 2024-09-10T10:01:34.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:34.515+0000 7ff5a7e00640 1 -- 172.21.15.5:0/2115953498 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5a81a13d0 con 0x7ff5a806e4f0 2024-09-10T10:01:34.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:34.515+0000 7ff59f400640 1 -- 172.21.15.5:0/2115953498 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff598010690 con 0x7ff5a806e4f0 2024-09-10T10:01:34.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:34.515+0000 7ff5a7e00640 1 -- 172.21.15.5:0/2115953498 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5a81ad6c0 con 0x7ff5a806e4f0 2024-09-10T10:01:34.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:34.518+0000 7ff59f400640 1 -- 172.21.15.5:0/2115953498 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7ff5980037a0 con 0x7ff5a806e4f0 2024-09-10T10:01:34.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:34.519+0000 7ff5b00d4740 1 Processor -- start 2024-09-10T10:01:34.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:34.519+0000 7ff5b00d4740 1 -- start start 2024-09-10T10:01:34.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:34.519+0000 7ff5b00d4740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b6ec072b50 0x55b6ec072f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:34.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:34.519+0000 7ff5b00d4740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b6ec073470 0x55b6ec081d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:34.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:34.519+0000 7ff5b00d4740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b6ec0822a0 0x55b6ec084690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:34.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:34.519+0000 7ff5b00d4740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55b6ebfd9290 con 0x55b6ec073470 2024-09-10T10:01:34.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:34.519+0000 7ff5b00d4740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55b6ebf80b40 con 0x55b6ec0822a0 2024-09-10T10:01:34.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:34.519+0000 7ff5b00d4740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55b6ebfd5250 con 0x55b6ec072b50 2024-09-10T10:01:34.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:34.520+0000 7ff5a7400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b6ec072b50 0x55b6ec072f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:34.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:34.520+0000 7ff5a7400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b6ec072b50 0x55b6ec072f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:47188/0 (socket says 172.21.15.5:47188) 2024-09-10T10:01:34.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:34.520+0000 7ff5a7400640 1 -- 172.21.15.5:0/4149884336 learned_addr learned my addr 172.21.15.5:0/4149884336 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T10:01:34.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:34.520+0000 7ff5a6000640 1 --2- 172.21.15.5:0/4149884336 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b6ec0822a0 0x55b6ec084690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:34.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:34.520+0000 7ff5a6a00640 1 --2- 172.21.15.5:0/4149884336 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b6ec073470 0x55b6ec081d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:34.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:34.520+0000 7ff5a7400640 1 -- 172.21.15.5:0/4149884336 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b6ec0822a0 msgr2=0x55b6ec084690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:01:34.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:34.520+0000 7ff5a7400640 1 --2- 172.21.15.5:0/4149884336 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b6ec0822a0 0x55b6ec084690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:34.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:34.520+0000 7ff5a7400640 1 -- 172.21.15.5:0/4149884336 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b6ec073470 msgr2=0x55b6ec081d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:01:34.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:34.520+0000 7ff5a7400640 1 --2- 172.21.15.5:0/4149884336 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b6ec073470 0x55b6ec081d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:34.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:34.520+0000 7ff5a7400640 1 -- 172.21.15.5:0/4149884336 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b6ebd4a6a0 con 0x55b6ec072b50 2024-09-10T10:01:34.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:34.520+0000 7ff5a6000640 1 --2- 172.21.15.5:0/4149884336 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b6ec0822a0 0x55b6ec084690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:01:34.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:34.520+0000 7ff59f400640 1 --2- 172.21.15.5:0/2115953498 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7ff58c0a2c00 0x7ff58c0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:34.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:34.521+0000 7ff5a7400640 1 --2- 172.21.15.5:0/4149884336 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b6ec072b50 0x55b6ec072f30 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7ff5a0009be0 tx=0x7ff5a0032e10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aa3e74fcabedc24e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:34.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:34.521+0000 7ff59f400640 1 -- 172.21.15.5:0/2115953498 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7ff5980cf7c0 con 0x7ff5a806e4f0 2024-09-10T10:01:34.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:34.521+0000 7ff5a6a00640 1 --2- 172.21.15.5:0/2115953498 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7ff58c0a2c00 0x7ff58c0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:34.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:34.521+0000 7ff593400640 1 -- 172.21.15.5:0/4149884336 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5a0041030 con 0x55b6ec072b50 2024-09-10T10:01:34.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:34.521+0000 7ff593400640 1 -- 172.21.15.5:0/4149884336 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff5a00303a0 con 0x55b6ec072b50 2024-09-10T10:01:34.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:34.521+0000 7ff593400640 1 -- 172.21.15.5:0/4149884336 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5a0036040 con 0x55b6ec072b50 2024-09-10T10:01:34.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:34.522+0000 7ff5b00d4740 1 -- 172.21.15.5:0/4149884336 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b6ec072b50 msgr2=0x55b6ec072f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:34.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:34.522+0000 7ff5b00d4740 1 --2- 172.21.15.5:0/4149884336 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b6ec072b50 0x55b6ec072f30 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7ff5a0009be0 tx=0x7ff5a0032e10 comp rx=0 tx=0).stop 2024-09-10T10:01:34.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:34.522+0000 7ff5a6a00640 1 --2- 172.21.15.5:0/2115953498 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7ff58c0a2c00 0x7ff58c0a50c0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7ff5980985a0 tx=0x7ff598000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:34.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:34.522+0000 7ff5b00d4740 1 -- 172.21.15.5:0/4149884336 shutdown_connections 2024-09-10T10:01:34.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:34.522+0000 7ff5b00d4740 1 --2- 172.21.15.5:0/4149884336 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b6ec0822a0 0x55b6ec084690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:34.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:34.522+0000 7ff5b00d4740 1 --2- 172.21.15.5:0/4149884336 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b6ec073470 0x55b6ec081d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:34.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:34.522+0000 7ff5b00d4740 1 --2- 172.21.15.5:0/4149884336 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b6ec072b50 0x55b6ec072f30 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:34.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:34.522+0000 7ff5b00d4740 1 -- 172.21.15.5:0/4149884336 >> 172.21.15.5:0/4149884336 conn(0x55b6ec06f240 msgr2=0x55b6ec06f620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:34.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:34.522+0000 7ff5b00d4740 1 -- 172.21.15.5:0/4149884336 shutdown_connections 2024-09-10T10:01:34.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:34.522+0000 7ff5b00d4740 1 -- 172.21.15.5:0/4149884336 wait complete. 2024-09-10T10:01:34.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:34.523+0000 7ff5b00d4740 1 -- start start 2024-09-10T10:01:34.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:34.523+0000 7ff5b00d4740 1 Processor -- start 2024-09-10T10:01:34.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:34.523+0000 7ff5b00d4740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b6ec072b50 0x55b6ec081bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:34.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:34.523+0000 7ff5b00d4740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b6ec073470 0x55b6ec07ee70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:34.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:34.523+0000 7ff5b00d4740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b6ec0822a0 0x55b6ec07f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:34.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:34.523+0000 7ff5b00d4740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55b6ec084f10 con 0x55b6ec072b50 2024-09-10T10:01:34.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:34.523+0000 7ff5b00d4740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55b6ebf93fa0 con 0x55b6ec0822a0 2024-09-10T10:01:34.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:34.523+0000 7ff5b00d4740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55b6ec085090 con 0x55b6ec073470 2024-09-10T10:01:34.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:34.524+0000 7ff5b00d4740 10 client.0 _ll_register_callbacks cb 0x55b6ebce7ba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:01:34.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:34.524+0000 7ff5a7400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b6ec072b50 0x55b6ec081bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:34.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:34.524+0000 7ff5a6000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b6ec073470 0x55b6ec07ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:34.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:34.524+0000 7ff5a6000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b6ec073470 0x55b6ec07ee70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:47204/0 (socket says 172.21.15.5:47204) 2024-09-10T10:01:34.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:34.524+0000 7ff5a6000640 1 -- 172.21.15.5:0/3095214783 learned_addr learned my addr 172.21.15.5:0/3095214783 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T10:01:34.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:34.524+0000 7ff5a7400640 1 --2- 172.21.15.5:0/3095214783 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b6ec0822a0 0x55b6ec07f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:34.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:34.524+0000 7ff5a6000640 1 -- 172.21.15.5:0/3095214783 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b6ec0822a0 msgr2=0x55b6ec07f3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:01:34.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:34.524+0000 7ff5a6000640 1 --2- 172.21.15.5:0/3095214783 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b6ec0822a0 0x55b6ec07f3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:34.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:34.524+0000 7ff5a6000640 1 -- 172.21.15.5:0/3095214783 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b6ec072b50 msgr2=0x55b6ec081bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:01:34.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:34.524+0000 7ff5a6000640 1 --2- 172.21.15.5:0/3095214783 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b6ec072b50 0x55b6ec081bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:34.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:34.524+0000 7ff5a6000640 1 -- 172.21.15.5:0/3095214783 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b6ec07fbb0 con 0x55b6ec073470 2024-09-10T10:01:34.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:34.525+0000 7ff5a7400640 1 --2- 172.21.15.5:0/3095214783 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b6ec072b50 0x55b6ec081bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:01:34.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:34.525+0000 7ff5a6000640 1 --2- 172.21.15.5:0/3095214783 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b6ec073470 0x55b6ec07ee70 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7ff59400f4f0 tx=0x7ff59400f9b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=67d82c6b23394ab1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:34.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:34.525+0000 7ff587e00640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-09-10T10:01:34.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:34.526+0000 7ff587e00640 1 -- 172.21.15.5:0/3095214783 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff594013850 con 0x55b6ec073470 2024-09-10T10:01:34.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:34.526+0000 7ff587e00640 1 -- 172.21.15.5:0/3095214783 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7ff5940139f0 con 0x55b6ec073470 2024-09-10T10:01:34.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:34.526+0000 7ff5b00d4740 1 -- 172.21.15.5:0/3095214783 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff5a00303a0 con 0x55b6ec073470 2024-09-10T10:01:34.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:34.526+0000 7ff587e00640 1 -- 172.21.15.5:0/3095214783 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7ff594024050 con 0x55b6ec073470 2024-09-10T10:01:34.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:34.526+0000 7ff5b00d4740 10 client.29806 fetch_fsmap learned FSMap version 42 2024-09-10T10:01:34.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:34.526+0000 7ff5b00d4740 1 -- 172.21.15.5:0/3095214783 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55b6ec0759a0 con 0x55b6ec073470 2024-09-10T10:01:34.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:34.527+0000 7ff587e00640 1 -- 172.21.15.5:0/3095214783 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff5a00303a0 con 0x55b6ec073470 2024-09-10T10:01:34.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:34.527+0000 7ff587e00640 1 -- 172.21.15.5:0/3095214783 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 42) ==== 8370+0+0 (secure 0 0 0) 0x7ff5940113e0 con 0x55b6ec073470 2024-09-10T10:01:34.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:34.527+0000 7ff5b00d4740 10 client.29806 fetch_fsmap finished waiting for FSMap version 42 2024-09-10T10:01:34.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:34.527+0000 7ff5b00d4740 10 client.29806 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T10:01:34.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:34.527+0000 7ff5b00d4740 20 client.29806 populate_metadata read hostname 'smithi005' 2024-09-10T10:01:34.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:34.527+0000 7ff5b00d4740 1 --2- 172.21.15.5:0/3095214783 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55b6ec076eb0 0x55b6ec0792d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:34.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:34.527+0000 7ff5b00d4740 4 client.29806 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:01:34.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:34.527+0000 7ff5b00d4740 1 -- 172.21.15.5:0/3095214783 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b6ebfe6800 con 0x55b6ec076eb0 2024-09-10T10:01:34.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:34.528+0000 7ff5a7400640 1 --2- 172.21.15.5:0/3095214783 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55b6ec076eb0 0x55b6ec0792d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:34.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:34.528+0000 7ff5a7400640 1 --2- 172.21.15.5:0/3095214783 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55b6ec076eb0 0x55b6ec0792d0 crc :-1 s=READY pgs=315 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=598c265a9a2509c2 server_cookie=619ad90a73d48740 in_seq=0 out_seq=0 2024-09-10T10:01:34.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:34.528+0000 7ff587e00640 10 client.29806 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T10:01:34.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:34.529+0000 7ff587e00640 1 -- 172.21.15.5:0/3095214783 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b6ebfe6800 con 0x55b6ec076eb0 2024-09-10T10:01:34.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:34.529+0000 7ff587e00640 10 client.29806 handle_command_reply: tid=0 2024-09-10T10:01:34.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:34.529+0000 7ff5b00d4740 1 client.29806 shutdown 2024-09-10T10:01:34.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:34.529+0000 7ff5b00d4740 1 -- 172.21.15.5:0/3095214783 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b6ec073470 msgr2=0x55b6ec07ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:34.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:34.529+0000 7ff5b00d4740 1 --2- 172.21.15.5:0/3095214783 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b6ec073470 0x55b6ec07ee70 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7ff59400f4f0 tx=0x7ff59400f9b0 comp rx=0 tx=0).stop 2024-09-10T10:01:34.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:34.529+0000 7ff5b00d4740 1 -- 172.21.15.5:0/3095214783 shutdown_connections 2024-09-10T10:01:34.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:34.529+0000 7ff5b00d4740 1 --2- 172.21.15.5:0/3095214783 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55b6ec076eb0 0x55b6ec0792d0 crc :-1 s=READY pgs=315 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:34.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:34.529+0000 7ff5b00d4740 1 --2- 172.21.15.5:0/3095214783 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b6ec0822a0 0x55b6ec07f3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:34.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:34.529+0000 7ff5b00d4740 1 --2- 172.21.15.5:0/3095214783 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b6ec073470 0x55b6ec07ee70 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:34.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:34.529+0000 7ff5b00d4740 1 --2- 172.21.15.5:0/3095214783 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b6ec072b50 0x55b6ec081bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:34.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:34.529+0000 7ff587e00640 0 client.29806 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T10:01:34.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:34.529+0000 7ff5b00d4740 1 -- 172.21.15.5:0/3095214783 >> 172.21.15.5:0/3095214783 conn(0x55b6ec06f240 msgr2=0x55b6ec074fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:34.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:34.529+0000 7ff5b00d4740 1 -- 172.21.15.5:0/3095214783 shutdown_connections 2024-09-10T10:01:34.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:34.529+0000 7ff5b00d4740 1 -- 172.21.15.5:0/3095214783 wait complete. 2024-09-10T10:01:34.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:34.530+0000 7ff5b00d4740 20 client.29806 trim_cache size 0 max 16384 2024-09-10T10:01:34.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:34.550+0000 7ff5b00d4740 1 Processor -- start 2024-09-10T10:01:34.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:34.550+0000 7ff5b00d4740 1 -- start start 2024-09-10T10:01:34.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:34.550+0000 7ff5b00d4740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b6ec072b50 0x55b6ec09e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:34.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:34.550+0000 7ff5b00d4740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b6ec073470 0x55b6ec09ec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:34.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:34.550+0000 7ff5b00d4740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b6ec0822a0 0x55b6ec07c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:34.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:34.550+0000 7ff5b00d4740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff5a81b53b0 con 0x55b6ec073470 2024-09-10T10:01:34.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:34.550+0000 7ff5b00d4740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7ff5a81b6ef0 con 0x55b6ec072b50 2024-09-10T10:01:34.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:34.550+0000 7ff5b00d4740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7ff5a81b5a80 con 0x55b6ec0822a0 2024-09-10T10:01:34.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:34.551+0000 7ff5a7400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b6ec072b50 0x55b6ec09e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:34.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:34.551+0000 7ff5a6000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b6ec073470 0x55b6ec09ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:34.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:34.551+0000 7ff5a7400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b6ec0822a0 0x55b6ec07c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:34.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:34.551+0000 7ff5a6000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b6ec073470 0x55b6ec09ec10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42724/0 (socket says 172.21.15.5:42724) 2024-09-10T10:01:34.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:34.551+0000 7ff5a6000640 1 -- 172.21.15.5:0/605289108 learned_addr learned my addr 172.21.15.5:0/605289108 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T10:01:34.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:34.551+0000 7ff5a6000640 1 -- 172.21.15.5:0/605289108 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b6ec0822a0 msgr2=0x55b6ec07c0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:01:34.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:34.551+0000 7ff5a6000640 1 --2- 172.21.15.5:0/605289108 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b6ec0822a0 0x55b6ec07c0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:34.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:34.551+0000 7ff5a6000640 1 -- 172.21.15.5:0/605289108 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b6ec072b50 msgr2=0x55b6ec09e6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:01:34.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:34.551+0000 7ff5a6000640 1 --2- 172.21.15.5:0/605289108 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b6ec072b50 0x55b6ec09e6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:34.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:34.551+0000 7ff5a6000640 1 -- 172.21.15.5:0/605289108 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b6ec081580 con 0x55b6ec073470 2024-09-10T10:01:34.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:34.552+0000 7ff5a6000640 1 --2- 172.21.15.5:0/605289108 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b6ec073470 0x55b6ec09ec10 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7ff594008010 tx=0x7ff59400feb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5205cde22e09840f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:34.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:34.552+0000 7ff592a00640 1 -- 172.21.15.5:0/605289108 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff59402e020 con 0x55b6ec073470 2024-09-10T10:01:34.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:34.552+0000 7ff592a00640 1 -- 172.21.15.5:0/605289108 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff59400a200 con 0x55b6ec073470 2024-09-10T10:01:34.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:34.552+0000 7ff592a00640 1 -- 172.21.15.5:0/605289108 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5940138b0 con 0x55b6ec073470 2024-09-10T10:01:34.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:34.553+0000 7ff5b00d4740 1 -- 172.21.15.5:0/605289108 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b6ec073470 msgr2=0x55b6ec09ec10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:34.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:34.553+0000 7ff5b00d4740 1 --2- 172.21.15.5:0/605289108 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b6ec073470 0x55b6ec09ec10 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7ff594008010 tx=0x7ff59400feb0 comp rx=0 tx=0).stop 2024-09-10T10:01:34.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:34.557+0000 7ff5b00d4740 1 -- 172.21.15.5:0/605289108 shutdown_connections 2024-09-10T10:01:34.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:34.557+0000 7ff5b00d4740 1 --2- 172.21.15.5:0/605289108 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b6ec0822a0 0x55b6ec07c0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:34.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:34.557+0000 7ff5b00d4740 1 --2- 172.21.15.5:0/605289108 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b6ec073470 0x55b6ec09ec10 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:34.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:34.557+0000 7ff5b00d4740 1 --2- 172.21.15.5:0/605289108 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b6ec072b50 0x55b6ec09e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:34.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:34.557+0000 7ff5b00d4740 1 -- 172.21.15.5:0/605289108 >> 172.21.15.5:0/605289108 conn(0x55b6ec06f240 msgr2=0x55b6ec06e270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:34.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:34.558+0000 7ff5b00d4740 1 -- 172.21.15.5:0/605289108 shutdown_connections 2024-09-10T10:01:34.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:34.558+0000 7ff5b00d4740 1 -- 172.21.15.5:0/605289108 wait complete. 2024-09-10T10:01:34.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:34.559+0000 7ff5b00d4740 1 -- start start 2024-09-10T10:01:34.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:34.559+0000 7ff5b00d4740 1 Processor -- start 2024-09-10T10:01:34.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:34.560+0000 7ff5b00d4740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b6ec072b50 0x55b6ec0867b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:34.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:34.560+0000 7ff5b00d4740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b6ec073470 0x55b6ec086cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:34.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:34.560+0000 7ff5b00d4740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b6ec0822a0 0x55b6ec1006a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:34.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:34.560+0000 7ff5b00d4740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff5a81b6820 con 0x55b6ec0822a0 2024-09-10T10:01:34.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:34.560+0000 7ff5b00d4740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7ff5a81b6150 con 0x55b6ec072b50 2024-09-10T10:01:34.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:34.560+0000 7ff5b00d4740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7ff5a81b3390 con 0x55b6ec073470 2024-09-10T10:01:34.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:34.560+0000 7ff5b00d4740 10 client.0 _ll_register_callbacks cb 0x7ff59400a510 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:01:34.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:34.560+0000 7ff5a7400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b6ec072b50 0x55b6ec0867b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:34.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:34.561+0000 7ff5a7400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b6ec072b50 0x55b6ec0867b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:54382/0 (socket says 172.21.15.5:54382) 2024-09-10T10:01:34.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:34.561+0000 7ff5a7400640 1 -- 172.21.15.5:0/1917723847 learned_addr learned my addr 172.21.15.5:0/1917723847 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T10:01:34.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:34.561+0000 7ff5a7400640 1 --2- 172.21.15.5:0/1917723847 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b6ec0822a0 0x55b6ec1006a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:34.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:34.561+0000 7ff5a7400640 1 -- 172.21.15.5:0/1917723847 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b6ec073470 msgr2=0x55b6ec086cf0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T10:01:34.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:34.561+0000 7ff5a7400640 1 --2- 172.21.15.5:0/1917723847 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b6ec073470 0x55b6ec086cf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:34.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:34.561+0000 7ff5a7400640 1 -- 172.21.15.5:0/1917723847 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b6ec072b50 msgr2=0x55b6ec0867b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:01:34.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:34.561+0000 7ff5a7400640 1 --2- 172.21.15.5:0/1917723847 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b6ec072b50 0x55b6ec0867b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:34.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:34.562+0000 7ff5a7400640 1 -- 172.21.15.5:0/1917723847 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b6ec100be0 con 0x55b6ec0822a0 2024-09-10T10:01:34.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:34.562+0000 7ff5a7400640 1 --2- 172.21.15.5:0/1917723847 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b6ec0822a0 0x55b6ec1006a0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7ff5a0009370 tx=0x7ff5a0003c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34ac1889ca4e6f38 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:34.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:34.562+0000 7ff587e00640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-10T10:01:34.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:34.562+0000 7ff587e00640 1 -- 172.21.15.5:0/1917723847 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5a0036040 con 0x55b6ec0822a0 2024-09-10T10:01:34.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:34.563+0000 7ff587e00640 1 -- 172.21.15.5:0/1917723847 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7ff5a0030bd0 con 0x55b6ec0822a0 2024-09-10T10:01:34.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:34.563+0000 7ff5b00d4740 1 -- 172.21.15.5:0/1917723847 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff59400a200 con 0x55b6ec0822a0 2024-09-10T10:01:34.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:34.563+0000 7ff587e00640 1 -- 172.21.15.5:0/1917723847 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7ff5a0041040 con 0x55b6ec0822a0 2024-09-10T10:01:34.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:34.563+0000 7ff5b00d4740 10 client.19755 fetch_fsmap learned FSMap version 42 2024-09-10T10:01:34.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:34.563+0000 7ff5b00d4740 1 -- 172.21.15.5:0/1917723847 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55b6ec101ea0 con 0x55b6ec0822a0 2024-09-10T10:01:34.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:34.564+0000 7ff587e00640 1 -- 172.21.15.5:0/1917723847 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff59400a200 con 0x55b6ec0822a0 2024-09-10T10:01:34.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:34.564+0000 7ff587e00640 1 -- 172.21.15.5:0/1917723847 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 42) ==== 8370+0+0 (secure 0 0 0) 0x7ff5a003d460 con 0x55b6ec0822a0 2024-09-10T10:01:34.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:34.564+0000 7ff5b00d4740 10 client.19755 fetch_fsmap finished waiting for FSMap version 42 2024-09-10T10:01:34.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:34.564+0000 7ff5b00d4740 10 client.19755 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T10:01:34.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:34.564+0000 7ff5b00d4740 20 client.19755 populate_metadata read hostname 'smithi005' 2024-09-10T10:01:34.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:34.564+0000 7ff5b00d4740 1 --2- 172.21.15.5:0/1917723847 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55b6ec1021e0 0x55b6ec1045d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:34.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:34.564+0000 7ff5b00d4740 4 client.19755 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-10T10:01:34.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:34.564+0000 7ff5b00d4740 1 -- 172.21.15.5:0/1917723847 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55b6ebfe0d10 con 0x55b6ec1021e0 2024-09-10T10:01:34.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:34.564+0000 7ff5a6000640 1 --2- 172.21.15.5:0/1917723847 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55b6ec1021e0 0x55b6ec1045d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:34.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:34.565+0000 7ff5a6000640 1 --2- 172.21.15.5:0/1917723847 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55b6ec1021e0 0x55b6ec1045d0 crc :-1 s=READY pgs=316 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d936c3b7cfff2a7c server_cookie=cbe210ab1de3b40d in_seq=0 out_seq=0 2024-09-10T10:01:34.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:34.565+0000 7ff587e00640 10 client.19755 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T10:01:34.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:34.566+0000 7ff587e00640 1 -- 172.21.15.5:0/1917723847 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55b6ebfe0d10 con 0x55b6ec1021e0 2024-09-10T10:01:34.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:34.566+0000 7ff587e00640 10 client.19755 handle_command_reply: tid=0 2024-09-10T10:01:34.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:34.566+0000 7ff5b00d4740 1 client.19755 shutdown 2024-09-10T10:01:34.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:34.566+0000 7ff5b00d4740 1 -- 172.21.15.5:0/1917723847 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b6ec0822a0 msgr2=0x55b6ec1006a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:34.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:34.566+0000 7ff5b00d4740 1 --2- 172.21.15.5:0/1917723847 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b6ec0822a0 0x55b6ec1006a0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7ff5a0009370 tx=0x7ff5a0003c00 comp rx=0 tx=0).stop 2024-09-10T10:01:34.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:34.566+0000 7ff5b00d4740 1 -- 172.21.15.5:0/1917723847 shutdown_connections 2024-09-10T10:01:34.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:34.566+0000 7ff5b00d4740 1 --2- 172.21.15.5:0/1917723847 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55b6ec1021e0 0x55b6ec1045d0 crc :-1 s=READY pgs=316 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:34.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:34.566+0000 7ff5b00d4740 1 --2- 172.21.15.5:0/1917723847 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b6ec0822a0 0x55b6ec1006a0 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:34.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:34.566+0000 7ff5b00d4740 1 --2- 172.21.15.5:0/1917723847 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55b6ec073470 0x55b6ec086cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:34.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:34.566+0000 7ff5b00d4740 1 --2- 172.21.15.5:0/1917723847 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55b6ec072b50 0x55b6ec0867b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:34.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:34.566+0000 7ff5b00d4740 1 -- 172.21.15.5:0/1917723847 >> 172.21.15.5:0/1917723847 conn(0x55b6ec06f240 msgr2=0x55b6ec074660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:34.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:34.566+0000 7ff587e00640 0 client.19755 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T10:01:34.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:34.567+0000 7ff5b00d4740 1 -- 172.21.15.5:0/1917723847 shutdown_connections 2024-09-10T10:01:34.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:34.567+0000 7ff5b00d4740 1 -- 172.21.15.5:0/1917723847 wait complete. 2024-09-10T10:01:34.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:34.567+0000 7ff5b00d4740 20 client.19755 trim_cache size 0 max 16384 2024-09-10T10:01:34.569 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-10T10:01:34.569 INFO:teuthology.orchestra.run.smithi005.stdout: "return_code": 0, 2024-09-10T10:01:34.569 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_tag": "6f4cd6da-dbe0-4b24-bc25-3ba9bffe3312", 2024-09-10T10:01:34.570 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": "asynchronous" 2024-09-10T10:01:34.570 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-10T10:01:34.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:34.568+0000 7ff5a7e00640 1 -- 172.21.15.5:0/2115953498 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7ff58c0a2c00 msgr2=0x7ff58c0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:34.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:34.568+0000 7ff5a7e00640 1 --2- 172.21.15.5:0/2115953498 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7ff58c0a2c00 0x7ff58c0a50c0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7ff5980985a0 tx=0x7ff598000bc0 comp rx=0 tx=0).stop 2024-09-10T10:01:34.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:34.568+0000 7ff5a7e00640 1 -- 172.21.15.5:0/2115953498 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff5a806e4f0 msgr2=0x7ff5a81a67f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:34.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:34.568+0000 7ff5a7e00640 1 --2- 172.21.15.5:0/2115953498 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff5a806e4f0 0x7ff5a81a67f0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7ff59800e9c0 tx=0x7ff59800ee80 comp rx=0 tx=0).stop 2024-09-10T10:01:34.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:34.568+0000 7ff5a7e00640 1 -- 172.21.15.5:0/2115953498 shutdown_connections 2024-09-10T10:01:34.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:34.568+0000 7ff5a7e00640 1 --2- 172.21.15.5:0/2115953498 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7ff58c0a2c00 0x7ff58c0a50c0 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:34.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:34.568+0000 7ff5a7e00640 1 --2- 172.21.15.5:0/2115953498 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff5a8114ab0 0x7ff5a81a09b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:34.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:34.568+0000 7ff5a7e00640 1 --2- 172.21.15.5:0/2115953498 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff5a806ee10 0x7ff5a81a6d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:34.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:34.568+0000 7ff5a7e00640 1 --2- 172.21.15.5:0/2115953498 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff5a806e4f0 0x7ff5a81a67f0 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:34.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:34.568+0000 7ff5a7e00640 1 -- 172.21.15.5:0/2115953498 >> 172.21.15.5:0/2115953498 conn(0x7ff5a8073180 msgr2=0x7ff5a806cab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:34.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:34.568+0000 7ff5a7e00640 1 -- 172.21.15.5:0/2115953498 shutdown_connections 2024-09-10T10:01:34.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:34.568+0000 7ff5a7e00640 1 -- 172.21.15.5:0/2115953498 wait complete. 2024-09-10T10:01:35.042 INFO:tasks.workunit.client.0.smithi005.stdout: 10 149511 13.75 MB/sec execute 433 sec latency 193.050 ms 2024-09-10T10:01:35.452 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T10:01:35.515 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:01:35 smithi005 ceph-mon[30350]: pgmap v1090: 129 pgs: 129 active+clean; 734 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 3.3 MiB/s wr, 404 op/s 2024-09-10T10:01:35.516 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:01:35 smithi005 ceph-mon[30350]: scrub queued for path: / 2024-09-10T10:01:35.516 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:01:35 smithi005 ceph-mon[30350]: scrub summary: idle+waiting paths [/] 2024-09-10T10:01:35.516 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:01:35 smithi005 ceph-mon[30350]: scrub summary: active paths [/] 2024-09-10T10:01:35.517 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:01:35 smithi005 ceph-mon[30350]: scrub summary: active paths [/] 2024-09-10T10:01:35.517 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:01:35 smithi005 ceph-mon[30350]: scrub summary: idle+waiting paths [/] 2024-09-10T10:01:35.517 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:01:35 smithi005 ceph-mon[30350]: scrub summary: active paths [/] 2024-09-10T10:01:35.517 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:01:35 smithi005 ceph-mon[30350]: scrub summary: idle+waiting paths [/] 2024-09-10T10:01:35.517 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:01:35 smithi005 ceph-mon[30350]: scrub summary: idle+waiting paths [/] 2024-09-10T10:01:35.521 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:01:35 smithi005 ceph-mon[30350]: scrub summary: active paths [/] 2024-09-10T10:01:35.521 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:01:35 smithi005 ceph-mon[30350]: scrub summary: idle+waiting paths [/] 2024-09-10T10:01:35.521 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:01:35 smithi005 ceph-mon[30350]: scrub summary: active paths [/] 2024-09-10T10:01:35.521 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:01:35 smithi005 ceph-mon[30350]: scrub summary: idle+waiting paths [/] 2024-09-10T10:01:35.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:01:35 smithi159 ceph-mon[35513]: pgmap v1090: 129 pgs: 129 active+clean; 734 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 3.3 MiB/s wr, 404 op/s 2024-09-10T10:01:35.601 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:01:35 smithi159 ceph-mon[35513]: scrub queued for path: / 2024-09-10T10:01:35.601 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:01:35 smithi159 ceph-mon[35513]: scrub summary: idle+waiting paths [/] 2024-09-10T10:01:35.601 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:01:35 smithi159 ceph-mon[35513]: scrub summary: active paths [/] 2024-09-10T10:01:35.601 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:01:35 smithi159 ceph-mon[35513]: scrub summary: active paths [/] 2024-09-10T10:01:35.601 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:01:35 smithi159 ceph-mon[35513]: scrub summary: idle+waiting paths [/] 2024-09-10T10:01:35.601 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:01:35 smithi159 ceph-mon[35513]: scrub summary: active paths [/] 2024-09-10T10:01:35.601 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:01:35 smithi159 ceph-mon[35513]: scrub summary: idle+waiting paths [/] 2024-09-10T10:01:35.601 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:01:35 smithi159 ceph-mon[35513]: scrub summary: idle+waiting paths [/] 2024-09-10T10:01:35.601 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:01:35 smithi159 ceph-mon[35513]: scrub summary: active paths [/] 2024-09-10T10:01:35.601 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:01:35 smithi159 ceph-mon[35513]: scrub summary: idle+waiting paths [/] 2024-09-10T10:01:35.601 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:01:35 smithi159 ceph-mon[35513]: scrub summary: active paths [/] 2024-09-10T10:01:35.601 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:01:35 smithi159 ceph-mon[35513]: scrub summary: idle+waiting paths [/] 2024-09-10T10:01:35.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:01:35 smithi102 ceph-mon[35442]: pgmap v1090: 129 pgs: 129 active+clean; 734 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 3.3 MiB/s wr, 404 op/s 2024-09-10T10:01:35.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:01:35 smithi102 ceph-mon[35442]: scrub queued for path: / 2024-09-10T10:01:35.635 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:01:35 smithi102 ceph-mon[35442]: scrub summary: idle+waiting paths [/] 2024-09-10T10:01:35.635 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:01:35 smithi102 ceph-mon[35442]: scrub summary: active paths [/] 2024-09-10T10:01:35.635 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:01:35 smithi102 ceph-mon[35442]: scrub summary: active paths [/] 2024-09-10T10:01:35.635 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:01:35 smithi102 ceph-mon[35442]: scrub summary: idle+waiting paths [/] 2024-09-10T10:01:35.635 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:01:35 smithi102 ceph-mon[35442]: scrub summary: active paths [/] 2024-09-10T10:01:35.635 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:01:35 smithi102 ceph-mon[35442]: scrub summary: idle+waiting paths [/] 2024-09-10T10:01:35.635 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:01:35 smithi102 ceph-mon[35442]: scrub summary: idle+waiting paths [/] 2024-09-10T10:01:35.635 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:01:35 smithi102 ceph-mon[35442]: scrub summary: active paths [/] 2024-09-10T10:01:35.635 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:01:35 smithi102 ceph-mon[35442]: scrub summary: idle+waiting paths [/] 2024-09-10T10:01:35.635 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:01:35 smithi102 ceph-mon[35442]: scrub summary: active paths [/] 2024-09-10T10:01:35.635 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:01:35 smithi102 ceph-mon[35442]: scrub summary: idle+waiting paths [/] 2024-09-10T10:01:35.950 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:01:36.042 INFO:tasks.workunit.client.0.smithi005.stdout: 10 150177 13.80 MB/sec execute 434 sec latency 54.016 ms 2024-09-10T10:01:36.455 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:01:36 smithi005 ceph-mon[30350]: scrub summary: active paths [/] 2024-09-10T10:01:36.455 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:01:36 smithi005 ceph-mon[30350]: scrub summary: active paths [/] 2024-09-10T10:01:36.455 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:01:36 smithi005 ceph-mon[30350]: scrub summary: idle+waiting paths [/] 2024-09-10T10:01:36.455 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:01:36 smithi005 ceph-mon[30350]: scrub summary: idle+waiting paths [/] 2024-09-10T10:01:36.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:01:36 smithi159 ceph-mon[35513]: scrub summary: active paths [/] 2024-09-10T10:01:36.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:01:36 smithi159 ceph-mon[35513]: scrub summary: active paths [/] 2024-09-10T10:01:36.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:01:36 smithi159 ceph-mon[35513]: scrub summary: idle+waiting paths [/] 2024-09-10T10:01:36.601 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:01:36 smithi159 ceph-mon[35513]: scrub summary: idle+waiting paths [/] 2024-09-10T10:01:36.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:01:36 smithi102 ceph-mon[35442]: scrub summary: active paths [/] 2024-09-10T10:01:36.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:01:36 smithi102 ceph-mon[35442]: scrub summary: active paths [/] 2024-09-10T10:01:36.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:01:36 smithi102 ceph-mon[35442]: scrub summary: idle+waiting paths [/] 2024-09-10T10:01:36.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:01:36 smithi102 ceph-mon[35442]: scrub summary: idle+waiting paths [/] 2024-09-10T10:01:37.042 INFO:tasks.workunit.client.0.smithi005.stdout: 10 150736 13.85 MB/sec execute 435 sec latency 68.353 ms 2024-09-10T10:01:37.424 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:01:37 smithi005 ceph-mon[30350]: pgmap v1091: 129 pgs: 129 active+clean; 732 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 3.8 MiB/s wr, 488 op/s 2024-09-10T10:01:37.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:36.714+0000 7fd61ce00640 1 -- 172.21.15.5:0/530750784 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd618109450 msgr2=0x7fd618109830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:37.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:36.714+0000 7fd61ce00640 1 --2- 172.21.15.5:0/530750784 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd618109450 0x7fd618109830 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7fd61000b8b0 tx=0x7fd610005f90 comp rx=0 tx=0).stop 2024-09-10T10:01:37.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:36.715+0000 7fd61ce00640 1 -- 172.21.15.5:0/530750784 shutdown_connections 2024-09-10T10:01:37.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:36.716+0000 7fd61ce00640 1 --2- 172.21.15.5:0/530750784 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd61810ea10 0x7fd618115360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:37.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:36.716+0000 7fd61ce00640 1 --2- 172.21.15.5:0/530750784 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd618109e00 0x7fd61810e2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:37.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:36.716+0000 7fd61ce00640 1 --2- 172.21.15.5:0/530750784 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd618109450 0x7fd618109830 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:37.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:36.716+0000 7fd61ce00640 1 -- 172.21.15.5:0/530750784 >> 172.21.15.5:0/530750784 conn(0x7fd618073660 msgr2=0x7fd618073a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:37.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:36.716+0000 7fd61ce00640 1 -- 172.21.15.5:0/530750784 shutdown_connections 2024-09-10T10:01:37.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:36.717+0000 7fd61ce00640 1 -- 172.21.15.5:0/530750784 wait complete. 2024-09-10T10:01:37.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:36.718+0000 7fd61ce00640 1 Processor -- start 2024-09-10T10:01:37.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:36.719+0000 7fd61ce00640 1 -- start start 2024-09-10T10:01:37.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:36.720+0000 7fd61ce00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd618109450 0x7fd6181a4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:37.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:36.720+0000 7fd61ce00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd618109e00 0x7fd6181a4b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:37.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:36.720+0000 7fd61ce00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd61810ea10 0x7fd61819e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:37.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:36.720+0000 7fd61ce00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd6181183c0 con 0x7fd618109450 2024-09-10T10:01:37.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:36.720+0000 7fd61ce00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fd618118240 con 0x7fd618109e00 2024-09-10T10:01:37.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:36.720+0000 7fd61ce00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fd618118540 con 0x7fd61810ea10 2024-09-10T10:01:37.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:36.720+0000 7fd616a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd618109e00 0x7fd6181a4b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:37.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:36.720+0000 7fd616a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd618109e00 0x7fd6181a4b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:54414/0 (socket says 172.21.15.5:54414) 2024-09-10T10:01:37.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:36.720+0000 7fd616a00640 1 -- 172.21.15.5:0/1274602990 learned_addr learned my addr 172.21.15.5:0/1274602990 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T10:01:37.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:36.721+0000 7fd617e00640 1 --2- 172.21.15.5:0/1274602990 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd61810ea10 0x7fd61819e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:37.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:36.721+0000 7fd617400640 1 --2- 172.21.15.5:0/1274602990 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd618109450 0x7fd6181a4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:37.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:36.721+0000 7fd616a00640 1 -- 172.21.15.5:0/1274602990 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd61810ea10 msgr2=0x7fd61819e6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:37.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:36.721+0000 7fd616a00640 1 --2- 172.21.15.5:0/1274602990 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd61810ea10 0x7fd61819e6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:37.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:36.722+0000 7fd616a00640 1 -- 172.21.15.5:0/1274602990 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd618109450 msgr2=0x7fd6181a4600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:37.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:36.722+0000 7fd616a00640 1 --2- 172.21.15.5:0/1274602990 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd618109450 0x7fd6181a4600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:37.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:36.722+0000 7fd616a00640 1 -- 172.21.15.5:0/1274602990 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd61819ef40 con 0x7fd618109e00 2024-09-10T10:01:37.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:36.722+0000 7fd617400640 1 --2- 172.21.15.5:0/1274602990 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd618109450 0x7fd6181a4600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:01:37.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:36.725+0000 7fd616a00640 1 --2- 172.21.15.5:0/1274602990 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd618109e00 0x7fd6181a4b40 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fd60800eab0 tx=0x7fd60800ef70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:37.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:36.725+0000 7fd60fe00640 1 -- 172.21.15.5:0/1274602990 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd60800ce40 con 0x7fd618109e00 2024-09-10T10:01:37.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:36.725+0000 7fd61ce00640 1 -- 172.21.15.5:0/1274602990 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd61819f1d0 con 0x7fd618109e00 2024-09-10T10:01:37.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:36.725+0000 7fd61ce00640 1 -- 172.21.15.5:0/1274602990 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6180745f0 con 0x7fd618109e00 2024-09-10T10:01:37.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:36.726+0000 7fd60fe00640 1 -- 172.21.15.5:0/1274602990 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd608033960 con 0x7fd618109e00 2024-09-10T10:01:37.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:36.726+0000 7fd60fe00640 1 -- 172.21.15.5:0/1274602990 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd608032af0 con 0x7fd618109e00 2024-09-10T10:01:37.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:36.730+0000 7fd620e4d740 1 Processor -- start 2024-09-10T10:01:37.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:36.733+0000 7fd60fe00640 1 -- 172.21.15.5:0/1274602990 <== mon.2 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7fd608032c90 con 0x7fd618109e00 2024-09-10T10:01:37.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:36.734+0000 7fd620e4d740 1 -- start start 2024-09-10T10:01:37.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:36.735+0000 7fd620e4d740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559ca9f6c560 0x559ca9f6c940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:37.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:36.735+0000 7fd60fe00640 1 --2- 172.21.15.5:0/1274602990 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fd5fc0a2c80 0x7fd5fc0a5140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:37.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:36.735+0000 7fd617e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559ca9f6c560 0x559ca9f6c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:37.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:36.735+0000 7fd617e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559ca9f6c560 0x559ca9f6c940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:47262/0 (socket says 172.21.15.5:47262) 2024-09-10T10:01:37.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:36.736+0000 7fd620e4d740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559ca9f6ce80 0x559ca9f77200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:37.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:36.736+0000 7fd620e4d740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559ca9f77740 0x559ca9f79b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:37.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:36.736+0000 7fd620e4d740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x559ca9eca3e0 con 0x559ca9f77740 2024-09-10T10:01:37.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:36.736+0000 7fd620e4d740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x559ca9e71d80 con 0x559ca9f6ce80 2024-09-10T10:01:37.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:36.736+0000 7fd620e4d740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x559ca9ec63a0 con 0x559ca9f6c560 2024-09-10T10:01:37.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:36.736+0000 7fd617e00640 1 -- 172.21.15.5:0/1450619868 learned_addr learned my addr 172.21.15.5:0/1450619868 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T10:01:37.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:36.736+0000 7fd617e00640 1 --2- 172.21.15.5:0/1450619868 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559ca9f77740 0x559ca9f79b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:37.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:36.737+0000 7fd60fe00640 1 -- 172.21.15.5:0/1274602990 <== mon.2 v2:172.21.15.102:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fd6080f19e0 con 0x7fd618109e00 2024-09-10T10:01:37.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:36.737+0000 7fd617e00640 1 -- 172.21.15.5:0/1450619868 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559ca9f6ce80 msgr2=0x559ca9f77200 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T10:01:37.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:36.737+0000 7fd617e00640 1 --2- 172.21.15.5:0/1450619868 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559ca9f6ce80 0x559ca9f77200 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:37.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:36.737+0000 7fd617e00640 1 -- 172.21.15.5:0/1450619868 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559ca9f77740 msgr2=0x559ca9f79b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:01:37.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:36.737+0000 7fd617e00640 1 --2- 172.21.15.5:0/1450619868 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559ca9f77740 0x559ca9f79b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:37.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:36.737+0000 7fd617e00640 1 -- 172.21.15.5:0/1450619868 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559ca9e90ae0 con 0x559ca9f6c560 2024-09-10T10:01:37.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:36.737+0000 7fd617400640 1 --2- 172.21.15.5:0/1274602990 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fd5fc0a2c80 0x7fd5fc0a5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:37.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:36.738+0000 7fd617e00640 1 --2- 172.21.15.5:0/1450619868 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559ca9f6c560 0x559ca9f6c940 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x559ca9f69e70 tx=0x7fd604011040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aeef1cfb6cf5d3d4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:37.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:36.739+0000 7fd617400640 1 --2- 172.21.15.5:0/1274602990 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fd5fc0a2c80 0x7fd5fc0a5140 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fd610032fd0 tx=0x7fd610000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:37.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:36.739+0000 7fd60cc00640 1 -- 172.21.15.5:0/1450619868 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd604004230 con 0x559ca9f6c560 2024-09-10T10:01:37.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:36.739+0000 7fd60cc00640 1 -- 172.21.15.5:0/1450619868 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd6040043d0 con 0x559ca9f6c560 2024-09-10T10:01:37.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:36.741+0000 7fd60cc00640 1 -- 172.21.15.5:0/1450619868 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd60401a9d0 con 0x559ca9f6c560 2024-09-10T10:01:37.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:36.741+0000 7fd620e4d740 1 -- 172.21.15.5:0/1450619868 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559ca9f6c560 msgr2=0x559ca9f6c940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:37.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:36.741+0000 7fd620e4d740 1 --2- 172.21.15.5:0/1450619868 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559ca9f6c560 0x559ca9f6c940 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x559ca9f69e70 tx=0x7fd604011040 comp rx=0 tx=0).stop 2024-09-10T10:01:37.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:36.741+0000 7fd620e4d740 1 -- 172.21.15.5:0/1450619868 shutdown_connections 2024-09-10T10:01:37.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:36.741+0000 7fd620e4d740 1 --2- 172.21.15.5:0/1450619868 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559ca9f77740 0x559ca9f79b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:37.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:36.741+0000 7fd620e4d740 1 --2- 172.21.15.5:0/1450619868 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559ca9f6ce80 0x559ca9f77200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:37.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:36.741+0000 7fd620e4d740 1 --2- 172.21.15.5:0/1450619868 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559ca9f6c560 0x559ca9f6c940 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:37.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:36.741+0000 7fd620e4d740 1 -- 172.21.15.5:0/1450619868 >> 172.21.15.5:0/1450619868 conn(0x559ca9f68af0 msgr2=0x559ca9f68ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:37.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:36.742+0000 7fd620e4d740 1 -- 172.21.15.5:0/1450619868 shutdown_connections 2024-09-10T10:01:37.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:36.743+0000 7fd620e4d740 1 -- 172.21.15.5:0/1450619868 wait complete. 2024-09-10T10:01:37.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:36.744+0000 7fd620e4d740 1 -- start start 2024-09-10T10:01:37.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:36.744+0000 7fd620e4d740 1 Processor -- start 2024-09-10T10:01:37.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:36.745+0000 7fd620e4d740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559ca9f6c560 0x559ca9f746f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:37.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:36.745+0000 7fd620e4d740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559ca9f6ce80 0x559ca9f74c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:37.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:36.745+0000 7fd620e4d740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559ca9f77740 0x559ca9f75170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:37.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:36.745+0000 7fd620e4d740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x559ca9f7a400 con 0x559ca9f6ce80 2024-09-10T10:01:37.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:36.745+0000 7fd620e4d740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x559ca9e850f0 con 0x559ca9f6c560 2024-09-10T10:01:37.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:36.745+0000 7fd620e4d740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x559ca9f7a580 con 0x559ca9f77740 2024-09-10T10:01:37.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:36.746+0000 7fd617400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559ca9f6c560 0x559ca9f746f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:37.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:36.746+0000 7fd617400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559ca9f6c560 0x559ca9f746f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:54444/0 (socket says 172.21.15.5:54444) 2024-09-10T10:01:37.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:36.746+0000 7fd617400640 1 -- 172.21.15.5:0/448877003 learned_addr learned my addr 172.21.15.5:0/448877003 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T10:01:37.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:36.746+0000 7fd616a00640 1 --2- 172.21.15.5:0/448877003 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559ca9f6ce80 0x559ca9f74c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:37.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:36.746+0000 7fd617400640 1 -- 172.21.15.5:0/448877003 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559ca9f77740 msgr2=0x559ca9f75170 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T10:01:37.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:36.746+0000 7fd617400640 1 --2- 172.21.15.5:0/448877003 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559ca9f77740 0x559ca9f75170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:37.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:36.746+0000 7fd617400640 1 -- 172.21.15.5:0/448877003 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559ca9f6ce80 msgr2=0x559ca9f74c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:01:37.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:36.746+0000 7fd617400640 1 --2- 172.21.15.5:0/448877003 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559ca9f6ce80 0x559ca9f74c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:37.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:36.746+0000 7fd617400640 1 -- 172.21.15.5:0/448877003 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559ca9c3b3c0 con 0x559ca9f6c560 2024-09-10T10:01:37.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:36.746+0000 7fd620e4d740 10 client.0 _ll_register_callbacks cb 0x559ca9e769e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:01:37.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:36.747+0000 7fd620e4d740 1 -- 172.21.15.5:0/448877003 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd6040043d0 con 0x559ca9f6c560 2024-09-10T10:01:37.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:36.747+0000 7fd617400640 1 --2- 172.21.15.5:0/448877003 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559ca9f6c560 0x559ca9f746f0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7fd610033000 tx=0x7fd61000d040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cb90988fd5e12aec server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:37.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:36.747+0000 7fd601600640 10 client.29849 ms_handle_connect on v2:172.21.15.102:3300/0 2024-09-10T10:01:37.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:36.747+0000 7fd601600640 1 -- 172.21.15.5:0/448877003 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd610044040 con 0x559ca9f6c560 2024-09-10T10:01:37.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:36.748+0000 7fd601600640 1 -- 172.21.15.5:0/448877003 <== mon.2 v2:172.21.15.102:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fd610004380 con 0x559ca9f6c560 2024-09-10T10:01:37.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:36.749+0000 7fd601600640 1 -- 172.21.15.5:0/448877003 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7fd610056050 con 0x559ca9f6c560 2024-09-10T10:01:37.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:36.749+0000 7fd620e4d740 10 client.29849 fetch_fsmap learned FSMap version 42 2024-09-10T10:01:37.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:36.749+0000 7fd620e4d740 1 -- 172.21.15.5:0/448877003 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({fsmap=42}) -- 0x559ca9f86180 con 0x559ca9f6c560 2024-09-10T10:01:37.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:36.750+0000 7fd601600640 1 -- 172.21.15.5:0/448877003 <== mon.2 v2:172.21.15.102:3300/0 4 ==== fsmap(e 42) ==== 8370+0+0 (secure 0 0 0) 0x7fd6100323f0 con 0x559ca9f6c560 2024-09-10T10:01:37.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:36.750+0000 7fd620e4d740 10 client.29849 fetch_fsmap finished waiting for FSMap version 42 2024-09-10T10:01:37.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:36.750+0000 7fd620e4d740 10 client.29849 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T10:01:37.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:36.750+0000 7fd620e4d740 20 client.29849 populate_metadata read hostname 'smithi005' 2024-09-10T10:01:37.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:36.750+0000 7fd620e4d740 1 --2- 172.21.15.5:0/448877003 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x559ca9f874d0 0x559ca9f89970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:37.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:36.750+0000 7fd620e4d740 4 client.29849 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:01:37.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:36.750+0000 7fd620e4d740 1 -- 172.21.15.5:0/448877003 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559ca9ed7950 con 0x559ca9f874d0 2024-09-10T10:01:37.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:36.751+0000 7fd616a00640 1 --2- 172.21.15.5:0/448877003 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x559ca9f874d0 0x559ca9f89970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:37.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:36.752+0000 7fd616a00640 1 --2- 172.21.15.5:0/448877003 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x559ca9f874d0 0x559ca9f89970 crc :-1 s=READY pgs=317 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f9249113bad2fd7a server_cookie=41099312b3084428 in_seq=0 out_seq=0 2024-09-10T10:01:37.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:36.752+0000 7fd601600640 10 client.29849 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T10:01:37.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:36.753+0000 7fd601600640 1 -- 172.21.15.5:0/448877003 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559ca9ed7950 con 0x559ca9f874d0 2024-09-10T10:01:37.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:36.753+0000 7fd601600640 10 client.29849 handle_command_reply: tid=0 2024-09-10T10:01:37.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:36.754+0000 7fd620e4d740 1 client.29849 shutdown 2024-09-10T10:01:37.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:36.754+0000 7fd620e4d740 1 -- 172.21.15.5:0/448877003 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559ca9f6c560 msgr2=0x559ca9f746f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:37.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:36.754+0000 7fd620e4d740 1 --2- 172.21.15.5:0/448877003 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559ca9f6c560 0x559ca9f746f0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7fd610033000 tx=0x7fd61000d040 comp rx=0 tx=0).stop 2024-09-10T10:01:37.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:36.754+0000 7fd620e4d740 1 -- 172.21.15.5:0/448877003 shutdown_connections 2024-09-10T10:01:37.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:36.754+0000 7fd620e4d740 1 --2- 172.21.15.5:0/448877003 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x559ca9f874d0 0x559ca9f89970 crc :-1 s=READY pgs=317 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:37.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:36.754+0000 7fd620e4d740 1 --2- 172.21.15.5:0/448877003 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559ca9f77740 0x559ca9f75170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:37.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:36.754+0000 7fd620e4d740 1 --2- 172.21.15.5:0/448877003 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559ca9f6ce80 0x559ca9f74c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:37.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:36.754+0000 7fd620e4d740 1 --2- 172.21.15.5:0/448877003 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559ca9f6c560 0x559ca9f746f0 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:37.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:36.754+0000 7fd620e4d740 1 -- 172.21.15.5:0/448877003 >> 172.21.15.5:0/448877003 conn(0x559ca9f68af0 msgr2=0x559ca9f6e820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:37.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:36.755+0000 7fd601600640 0 client.29849 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T10:01:37.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:36.755+0000 7fd620e4d740 1 -- 172.21.15.5:0/448877003 shutdown_connections 2024-09-10T10:01:37.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:36.755+0000 7fd620e4d740 1 -- 172.21.15.5:0/448877003 wait complete. 2024-09-10T10:01:37.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:36.756+0000 7fd620e4d740 20 client.29849 trim_cache size 0 max 16384 2024-09-10T10:01:37.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:36.789+0000 7fd620e4d740 1 Processor -- start 2024-09-10T10:01:37.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:36.789+0000 7fd620e4d740 1 -- start start 2024-09-10T10:01:37.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:36.789+0000 7fd620e4d740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559ca9f6c560 0x559ca9f6efc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:37.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:36.789+0000 7fd620e4d740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559ca9f6ce80 0x559ca9f76660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:37.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:36.789+0000 7fd620e4d740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559ca9f77740 0x559ca9f8f240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:37.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:36.789+0000 7fd620e4d740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd5f800fc90 con 0x559ca9f6c560 2024-09-10T10:01:37.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:36.789+0000 7fd620e4d740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fd5f80117d0 con 0x559ca9f77740 2024-09-10T10:01:37.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:36.789+0000 7fd620e4d740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fd5f8010360 con 0x559ca9f6ce80 2024-09-10T10:01:37.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:36.790+0000 7fd617e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559ca9f77740 0x559ca9f8f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:37.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:36.790+0000 7fd617400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559ca9f6c560 0x559ca9f6efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:37.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:36.791+0000 7fd616a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559ca9f6ce80 0x559ca9f76660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:37.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:36.791+0000 7fd616a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559ca9f6ce80 0x559ca9f76660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:47268/0 (socket says 172.21.15.5:47268) 2024-09-10T10:01:37.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:36.791+0000 7fd616a00640 1 -- 172.21.15.5:0/1406358511 learned_addr learned my addr 172.21.15.5:0/1406358511 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T10:01:37.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:36.791+0000 7fd616a00640 1 -- 172.21.15.5:0/1406358511 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559ca9f77740 msgr2=0x559ca9f8f240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:01:37.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:36.791+0000 7fd616a00640 1 --2- 172.21.15.5:0/1406358511 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559ca9f77740 0x559ca9f8f240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:37.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:36.791+0000 7fd616a00640 1 -- 172.21.15.5:0/1406358511 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559ca9f6c560 msgr2=0x559ca9f6efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:01:37.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:36.791+0000 7fd616a00640 1 --2- 172.21.15.5:0/1406358511 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559ca9f6c560 0x559ca9f6efc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:37.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:36.791+0000 7fd616a00640 1 -- 172.21.15.5:0/1406358511 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559ca9f6f500 con 0x559ca9f6ce80 2024-09-10T10:01:37.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:36.792+0000 7fd617e00640 1 --2- 172.21.15.5:0/1406358511 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559ca9f77740 0x559ca9f8f240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:01:37.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:36.792+0000 7fd617400640 1 --2- 172.21.15.5:0/1406358511 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559ca9f6c560 0x559ca9f6efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:01:37.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:36.792+0000 7fd616a00640 1 --2- 172.21.15.5:0/1406358511 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559ca9f6ce80 0x559ca9f76660 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fd60800c700 tx=0x7fd6080ba090 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b9f948c6524f68d5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:37.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:36.792+0000 7fd603e00640 1 -- 172.21.15.5:0/1406358511 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6080c6a60 con 0x559ca9f6ce80 2024-09-10T10:01:37.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:36.793+0000 7fd603e00640 1 -- 172.21.15.5:0/1406358511 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd6080dca30 con 0x559ca9f6ce80 2024-09-10T10:01:37.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:36.793+0000 7fd603e00640 1 -- 172.21.15.5:0/1406358511 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6080ba460 con 0x559ca9f6ce80 2024-09-10T10:01:37.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:36.795+0000 7fd620e4d740 1 -- 172.21.15.5:0/1406358511 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559ca9f6ce80 msgr2=0x559ca9f76660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:37.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:36.795+0000 7fd620e4d740 1 --2- 172.21.15.5:0/1406358511 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559ca9f6ce80 0x559ca9f76660 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fd60800c700 tx=0x7fd6080ba090 comp rx=0 tx=0).stop 2024-09-10T10:01:37.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:36.795+0000 7fd620e4d740 1 -- 172.21.15.5:0/1406358511 shutdown_connections 2024-09-10T10:01:37.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:36.795+0000 7fd620e4d740 1 --2- 172.21.15.5:0/1406358511 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559ca9f77740 0x559ca9f8f240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:37.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:36.795+0000 7fd620e4d740 1 --2- 172.21.15.5:0/1406358511 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559ca9f6ce80 0x559ca9f76660 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:37.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:36.795+0000 7fd620e4d740 1 --2- 172.21.15.5:0/1406358511 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559ca9f6c560 0x559ca9f6efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:37.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:36.795+0000 7fd620e4d740 1 -- 172.21.15.5:0/1406358511 >> 172.21.15.5:0/1406358511 conn(0x559ca9f68af0 msgr2=0x559ca9f75f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:37.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:36.795+0000 7fd620e4d740 1 -- 172.21.15.5:0/1406358511 shutdown_connections 2024-09-10T10:01:37.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:36.795+0000 7fd620e4d740 1 -- 172.21.15.5:0/1406358511 wait complete. 2024-09-10T10:01:37.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:36.796+0000 7fd620e4d740 1 -- start start 2024-09-10T10:01:37.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:36.796+0000 7fd620e4d740 1 Processor -- start 2024-09-10T10:01:37.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:36.797+0000 7fd620e4d740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559ca9f6c560 0x559ca9f800a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:37.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:36.797+0000 7fd620e4d740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559ca9f6ce80 0x559ca9f7b420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:37.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:36.797+0000 7fd620e4d740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559ca9f77740 0x559ca9f7b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:37.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:36.797+0000 7fd620e4d740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd5f8011100 con 0x559ca9f77740 2024-09-10T10:01:37.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:36.797+0000 7fd620e4d740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fd5f8010a30 con 0x559ca9f6c560 2024-09-10T10:01:37.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:36.797+0000 7fd620e4d740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fd5f800dc70 con 0x559ca9f6ce80 2024-09-10T10:01:37.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:36.797+0000 7fd620e4d740 10 client.0 _ll_register_callbacks cb 0x7fd61000b6d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:01:37.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:36.797+0000 7fd616a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559ca9f6ce80 0x559ca9f7b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:37.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:36.797+0000 7fd616a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559ca9f6ce80 0x559ca9f7b420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:47278/0 (socket says 172.21.15.5:47278) 2024-09-10T10:01:37.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:36.797+0000 7fd616a00640 1 -- 172.21.15.5:0/3203621162 learned_addr learned my addr 172.21.15.5:0/3203621162 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T10:01:37.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:36.797+0000 7fd617400640 1 --2- 172.21.15.5:0/3203621162 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559ca9f6c560 0x559ca9f800a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:37.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:36.797+0000 7fd617e00640 1 --2- 172.21.15.5:0/3203621162 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559ca9f77740 0x559ca9f7b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:37.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:36.798+0000 7fd617400640 1 -- 172.21.15.5:0/3203621162 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559ca9f6ce80 msgr2=0x559ca9f7b420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:01:37.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:36.798+0000 7fd617400640 1 --2- 172.21.15.5:0/3203621162 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559ca9f6ce80 0x559ca9f7b420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:37.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:36.798+0000 7fd617400640 1 -- 172.21.15.5:0/3203621162 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559ca9f77740 msgr2=0x559ca9f7b9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:01:37.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:36.798+0000 7fd617400640 1 --2- 172.21.15.5:0/3203621162 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559ca9f77740 0x559ca9f7b9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:37.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:36.798+0000 7fd617400640 1 -- 172.21.15.5:0/3203621162 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559ca9f7c1d0 con 0x559ca9f6c560 2024-09-10T10:01:37.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:36.798+0000 7fd617400640 1 --2- 172.21.15.5:0/3203621162 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559ca9f6c560 0x559ca9f800a0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7fd61000b610 tx=0x7fd610045000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8092302b78ea3cda server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:01:37.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:36.798+0000 7fd601600640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-09-10T10:01:37.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:36.798+0000 7fd601600640 1 -- 172.21.15.5:0/3203621162 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd610044040 con 0x559ca9f6c560 2024-09-10T10:01:37.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:36.798+0000 7fd601600640 1 -- 172.21.15.5:0/3203621162 <== mon.2 v2:172.21.15.102:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fd61000ad20 con 0x559ca9f6c560 2024-09-10T10:01:37.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:36.799+0000 7fd620e4d740 1 -- 172.21.15.5:0/3203621162 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd6080dca30 con 0x559ca9f6c560 2024-09-10T10:01:37.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:36.799+0000 7fd601600640 1 -- 172.21.15.5:0/3203621162 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7fd61006a050 con 0x559ca9f6c560 2024-09-10T10:01:37.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:36.799+0000 7fd620e4d740 10 client.29855 fetch_fsmap learned FSMap version 42 2024-09-10T10:01:37.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:36.799+0000 7fd620e4d740 1 -- 172.21.15.5:0/3203621162 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x559ca9f67da0 con 0x559ca9f6c560 2024-09-10T10:01:37.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:36.799+0000 7fd601600640 1 -- 172.21.15.5:0/3203621162 <== mon.2 v2:172.21.15.102:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd6080dca30 con 0x559ca9f6c560 2024-09-10T10:01:37.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:36.800+0000 7fd601600640 1 -- 172.21.15.5:0/3203621162 <== mon.2 v2:172.21.15.102:3300/0 5 ==== fsmap(e 42) ==== 8370+0+0 (secure 0 0 0) 0x7fd610003680 con 0x559ca9f6c560 2024-09-10T10:01:37.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:36.800+0000 7fd620e4d740 10 client.29855 fetch_fsmap finished waiting for FSMap version 42 2024-09-10T10:01:37.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:36.800+0000 7fd620e4d740 10 client.29855 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T10:01:37.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:36.800+0000 7fd620e4d740 20 client.29855 populate_metadata read hostname 'smithi005' 2024-09-10T10:01:37.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:36.800+0000 7fd620e4d740 1 --2- 172.21.15.5:0/3203621162 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x559ca9ffb230 0x559ca9ffd650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:01:37.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:36.800+0000 7fd620e4d740 4 client.29855 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T10:01:37.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:36.800+0000 7fd620e4d740 1 -- 172.21.15.5:0/3203621162 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "scrub status"}) -- 0x559ca9ed1e60 con 0x559ca9ffb230 2024-09-10T10:01:37.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:36.801+0000 7fd616a00640 1 --2- 172.21.15.5:0/3203621162 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x559ca9ffb230 0x559ca9ffd650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:01:37.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:36.802+0000 7fd616a00640 1 --2- 172.21.15.5:0/3203621162 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x559ca9ffb230 0x559ca9ffd650 crc :-1 s=READY pgs=318 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ab4a1acf6f26ab5f server_cookie=d4bb742ceaa606b5 in_seq=0 out_seq=0 2024-09-10T10:01:37.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:36.802+0000 7fd601600640 10 client.29855 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T10:01:37.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:36.802+0000 7fd601600640 1 -- 172.21.15.5:0/3203621162 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x559ca9ed1e60 con 0x559ca9ffb230 2024-09-10T10:01:37.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:36.802+0000 7fd601600640 10 client.29855 handle_command_reply: tid=0 2024-09-10T10:01:37.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:36.802+0000 7fd620e4d740 1 client.29855 shutdown 2024-09-10T10:01:37.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:36.803+0000 7fd620e4d740 1 -- 172.21.15.5:0/3203621162 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559ca9f6c560 msgr2=0x559ca9f800a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:37.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:36.803+0000 7fd620e4d740 1 --2- 172.21.15.5:0/3203621162 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559ca9f6c560 0x559ca9f800a0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7fd61000b610 tx=0x7fd610045000 comp rx=0 tx=0).stop 2024-09-10T10:01:37.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:36.803+0000 7fd620e4d740 1 -- 172.21.15.5:0/3203621162 shutdown_connections 2024-09-10T10:01:37.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:36.803+0000 7fd620e4d740 1 --2- 172.21.15.5:0/3203621162 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x559ca9ffb230 0x559ca9ffd650 crc :-1 s=READY pgs=318 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:37.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:36.803+0000 7fd620e4d740 1 --2- 172.21.15.5:0/3203621162 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559ca9f77740 0x559ca9f7b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:37.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:36.803+0000 7fd620e4d740 1 --2- 172.21.15.5:0/3203621162 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x559ca9f6ce80 0x559ca9f7b420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:37.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:36.803+0000 7fd620e4d740 1 --2- 172.21.15.5:0/3203621162 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x559ca9f6c560 0x559ca9f800a0 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:37.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:36.803+0000 7fd620e4d740 1 -- 172.21.15.5:0/3203621162 >> 172.21.15.5:0/3203621162 conn(0x559ca9f68af0 msgr2=0x559ca9f5f3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:37.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:36.803+0000 7fd601600640 0 client.29855 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T10:01:37.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:36.804+0000 7fd620e4d740 1 -- 172.21.15.5:0/3203621162 shutdown_connections 2024-09-10T10:01:37.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:36.804+0000 7fd620e4d740 1 -- 172.21.15.5:0/3203621162 wait complete. 2024-09-10T10:01:37.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:36.804+0000 7fd620e4d740 20 client.29855 trim_cache size 0 max 16384 2024-09-10T10:01:37.531 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-10T10:01:37.532 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-10T10:01:37.532 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2024-09-10T10:01:37.532 INFO:teuthology.orchestra.run.smithi005.stdout: "6f4cd6da-dbe0-4b24-bc25-3ba9bffe3312": { 2024-09-10T10:01:37.532 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2024-09-10T10:01:37.532 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "6f4cd6da-dbe0-4b24-bc25-3ba9bffe3312", 2024-09-10T10:01:37.532 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2024-09-10T10:01:37.532 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T10:01:37.532 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T10:01:37.532 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-10T10:01:37.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:36.808+0000 7fd61ce00640 1 -- 172.21.15.5:0/1274602990 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fd5fc0a2c80 msgr2=0x7fd5fc0a5140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:37.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:36.808+0000 7fd61ce00640 1 --2- 172.21.15.5:0/1274602990 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fd5fc0a2c80 0x7fd5fc0a5140 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fd610032fd0 tx=0x7fd610000bc0 comp rx=0 tx=0).stop 2024-09-10T10:01:37.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:36.808+0000 7fd61ce00640 1 -- 172.21.15.5:0/1274602990 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd618109e00 msgr2=0x7fd6181a4b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:01:37.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:36.808+0000 7fd61ce00640 1 --2- 172.21.15.5:0/1274602990 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd618109e00 0x7fd6181a4b40 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fd60800eab0 tx=0x7fd60800ef70 comp rx=0 tx=0).stop 2024-09-10T10:01:37.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:36.808+0000 7fd61ce00640 1 -- 172.21.15.5:0/1274602990 shutdown_connections 2024-09-10T10:01:37.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:36.808+0000 7fd61ce00640 1 --2- 172.21.15.5:0/1274602990 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fd5fc0a2c80 0x7fd5fc0a5140 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:37.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:36.808+0000 7fd61ce00640 1 --2- 172.21.15.5:0/1274602990 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd61810ea10 0x7fd61819e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:37.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:36.808+0000 7fd61ce00640 1 --2- 172.21.15.5:0/1274602990 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd618109e00 0x7fd6181a4b40 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:37.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:36.808+0000 7fd61ce00640 1 --2- 172.21.15.5:0/1274602990 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd618109450 0x7fd6181a4600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:01:37.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:36.808+0000 7fd61ce00640 1 -- 172.21.15.5:0/1274602990 >> 172.21.15.5:0/1274602990 conn(0x7fd618073660 msgr2=0x7fd61810c480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:01:37.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:36.809+0000 7fd61ce00640 1 -- 172.21.15.5:0/1274602990 shutdown_connections 2024-09-10T10:01:37.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:01:36.809+0000 7fd61ce00640 1 -- 172.21.15.5:0/1274602990 wait complete. 2024-09-10T10:01:37.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:01:37 smithi159 ceph-mon[35513]: pgmap v1091: 129 pgs: 129 active+clean; 732 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 3.8 MiB/s wr, 488 op/s 2024-09-10T10:01:37.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:01:37 smithi102 ceph-mon[35442]: pgmap v1091: 129 pgs: 129 active+clean; 732 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 3.8 MiB/s wr, 488 op/s 2024-09-10T10:01:38.042 INFO:tasks.workunit.client.0.smithi005.stdout: 10 151209 13.88 MB/sec execute 436 sec latency 108.552 ms 2024-09-10T10:01:38.259 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:01:38 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:01:38.259 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:01:38 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:01:38.259 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:01:38 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T10:01:38.378 INFO:tasks.cephfs.filesystem:scrub status for tag:6f4cd6da-dbe0-4b24-bc25-3ba9bffe3312 - {'path': '/', 'tag': '6f4cd6da-dbe0-4b24-bc25-3ba9bffe3312', 'options': 'recursive,force'} 2024-09-10T10:01:38.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:01:38 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:01:38.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:01:38 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:01:38.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:01:38 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T10:01:38.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:01:38 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:01:38.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:01:38 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:01:38.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:01:38 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T10:01:39.042 INFO:tasks.workunit.client.0.smithi005.stdout: 10 151597 13.88 MB/sec execute 437 sec latency 91.883 ms 2024-09-10T10:01:39.557 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:01:39 smithi005 ceph-mon[30350]: pgmap v1092: 129 pgs: 129 active+clean; 733 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 3.4 MiB/s wr, 449 op/s 2024-09-10T10:01:39.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:01:39 smithi159 ceph-mon[35513]: pgmap v1092: 129 pgs: 129 active+clean; 733 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 3.4 MiB/s wr, 449 op/s 2024-09-10T10:01:39.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:01:39 smithi102 ceph-mon[35442]: pgmap v1092: 129 pgs: 129 active+clean; 733 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 3.4 MiB/s wr, 449 op/s 2024-09-10T10:01:40.042 INFO:tasks.workunit.client.0.smithi005.stdout: 10 152312 13.92 MB/sec execute 438 sec latency 84.974 ms 2024-09-10T10:01:41.042 INFO:tasks.workunit.client.0.smithi005.stdout: 10 153066 13.98 MB/sec execute 439 sec latency 63.925 ms 2024-09-10T10:01:41.429 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:01:41 smithi005 ceph-mon[30350]: pgmap v1093: 129 pgs: 129 active+clean; 725 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 4.0 MiB/s wr, 548 op/s 2024-09-10T10:01:41.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:01:41 smithi102 ceph-mon[35442]: pgmap v1093: 129 pgs: 129 active+clean; 725 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 4.0 MiB/s wr, 548 op/s 2024-09-10T10:01:41.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:01:41 smithi159 ceph-mon[35513]: pgmap v1093: 129 pgs: 129 active+clean; 725 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 4.0 MiB/s wr, 548 op/s 2024-09-10T10:01:42.042 INFO:tasks.workunit.client.0.smithi005.stdout: 10 153244 13.98 MB/sec execute 440 sec latency 613.629 ms 2024-09-10T10:01:43.042 INFO:tasks.workunit.client.0.smithi005.stdout: 10 153959 14.06 MB/sec execute 441 sec latency 686.603 ms 2024-09-10T10:01:43.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:01:43 smithi005 ceph-mon[30350]: pgmap v1094: 129 pgs: 129 active+clean; 728 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 3.3 MiB/s wr, 447 op/s 2024-09-10T10:01:43.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:01:43 smithi102 ceph-mon[35442]: pgmap v1094: 129 pgs: 129 active+clean; 728 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 3.3 MiB/s wr, 447 op/s 2024-09-10T10:01:43.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:01:43 smithi159 ceph-mon[35513]: pgmap v1094: 129 pgs: 129 active+clean; 728 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 3.3 MiB/s wr, 447 op/s 2024-09-10T10:01:44.042 INFO:tasks.workunit.client.0.smithi005.stdout: 10 154464 14.09 MB/sec execute 442 sec latency 68.191 ms 2024-09-10T10:01:45.043 INFO:tasks.workunit.client.0.smithi005.stdout: 10 155091 14.12 MB/sec execute 443 sec latency 65.786 ms 2024-09-10T10:01:45.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:01:45 smithi102 ceph-mon[35442]: pgmap v1095: 129 pgs: 129 active+clean; 730 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 3.4 MiB/s wr, 427 op/s 2024-09-10T10:01:45.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:01:45 smithi005 ceph-mon[30350]: pgmap v1095: 129 pgs: 129 active+clean; 730 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 3.4 MiB/s wr, 427 op/s 2024-09-10T10:01:45.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:01:45 smithi159 ceph-mon[35513]: pgmap v1095: 129 pgs: 129 active+clean; 730 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 3.4 MiB/s wr, 427 op/s 2024-09-10T10:01:46.042 INFO:tasks.workunit.client.0.smithi005.stdout: 10 155639 14.16 MB/sec execute 444 sec latency 54.427 ms 2024-09-10T10:01:47.044 INFO:tasks.workunit.client.0.smithi005.stdout: 10 155772 14.14 MB/sec execute 445 sec latency 783.946 ms 2024-09-10T10:01:47.428 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:01:47 smithi005 ceph-mon[30350]: pgmap v1096: 129 pgs: 129 active+clean; 738 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 4.0 MiB/s wr, 506 op/s 2024-09-10T10:01:47.428 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:01:47 smithi005 ceph-mon[30350]: scrub summary: idle 2024-09-10T10:01:47.428 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:01:47 smithi005 ceph-mon[30350]: scrub summary: idle 2024-09-10T10:01:47.429 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:01:47 smithi005 ceph-mon[30350]: scrub summary: idle 2024-09-10T10:01:47.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:01:47 smithi102 ceph-mon[35442]: pgmap v1096: 129 pgs: 129 active+clean; 738 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 4.0 MiB/s wr, 506 op/s 2024-09-10T10:01:47.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:01:47 smithi102 ceph-mon[35442]: scrub summary: idle 2024-09-10T10:01:47.635 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:01:47 smithi102 ceph-mon[35442]: scrub summary: idle 2024-09-10T10:01:47.635 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:01:47 smithi102 ceph-mon[35442]: scrub summary: idle 2024-09-10T10:01:47.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:01:47 smithi159 ceph-mon[35513]: pgmap v1096: 129 pgs: 129 active+clean; 738 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 4.0 MiB/s wr, 506 op/s 2024-09-10T10:01:47.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:01:47 smithi159 ceph-mon[35513]: scrub summary: idle 2024-09-10T10:01:47.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:01:47 smithi159 ceph-mon[35513]: scrub summary: idle 2024-09-10T10:01:47.851 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:01:47 smithi159 ceph-mon[35513]: scrub summary: idle 2024-09-10T10:01:48.044 INFO:tasks.workunit.client.0.smithi005.stdout: 10 155776 14.11 MB/sec execute 446 sec latency 1784.059 ms 2024-09-10T10:01:48.578 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:01:48 smithi005 ceph-mon[30350]: scrub summary: idle 2024-09-10T10:01:48.579 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:01:48 smithi005 ceph-mon[30350]: scrub summary: idle 2024-09-10T10:01:48.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:01:48 smithi102 ceph-mon[35442]: scrub summary: idle 2024-09-10T10:01:48.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:01:48 smithi102 ceph-mon[35442]: scrub summary: idle 2024-09-10T10:01:48.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:01:48 smithi159 ceph-mon[35513]: scrub summary: idle 2024-09-10T10:01:48.851 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:01:48 smithi159 ceph-mon[35513]: scrub summary: idle 2024-09-10T10:01:49.044 INFO:tasks.workunit.client.0.smithi005.stdout: 10 155787 14.08 MB/sec execute 447 sec latency 2784.112 ms 2024-09-10T10:01:49.441 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:01:49 smithi005 ceph-mon[30350]: pgmap v1097: 129 pgs: 129 active+clean; 745 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 4.2 MiB/s wr, 459 op/s 2024-09-10T10:01:49.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:01:49 smithi102 ceph-mon[35442]: pgmap v1097: 129 pgs: 129 active+clean; 745 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 4.2 MiB/s wr, 459 op/s 2024-09-10T10:01:49.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:01:49 smithi159 ceph-mon[35513]: pgmap v1097: 129 pgs: 129 active+clean; 745 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 4.2 MiB/s wr, 459 op/s 2024-09-10T10:01:50.044 INFO:tasks.workunit.client.0.smithi005.stdout: 10 155816 14.05 MB/sec execute 448 sec latency 3784.158 ms 2024-09-10T10:01:51.044 INFO:tasks.workunit.client.0.smithi005.stdout: 10 155817 14.02 MB/sec execute 449 sec latency 4705.107 ms 2024-09-10T10:01:51.456 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:01:51 smithi005 ceph-mon[30350]: pgmap v1098: 129 pgs: 129 active+clean; 749 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 5.8 MiB/s wr, 489 op/s 2024-09-10T10:01:51.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:01:51 smithi159 ceph-mon[35513]: pgmap v1098: 129 pgs: 129 active+clean; 749 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 5.8 MiB/s wr, 489 op/s 2024-09-10T10:01:51.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:01:51 smithi102 ceph-mon[35442]: pgmap v1098: 129 pgs: 129 active+clean; 749 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 5.8 MiB/s wr, 489 op/s 2024-09-10T10:01:52.044 INFO:tasks.workunit.client.0.smithi005.stdout: 10 155822 13.99 MB/sec execute 450 sec latency 5236.458 ms 2024-09-10T10:01:53.044 INFO:tasks.workunit.client.0.smithi005.stdout: 10 155822 13.95 MB/sec execute 451 sec latency 2079.080 ms 2024-09-10T10:01:53.649 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:01:53 smithi005 ceph-mon[30350]: pgmap v1099: 129 pgs: 129 active+clean; 746 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 5.5 MiB/s wr, 383 op/s 2024-09-10T10:01:53.652 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:01:53 smithi159 ceph-mon[35513]: pgmap v1099: 129 pgs: 129 active+clean; 746 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 5.5 MiB/s wr, 383 op/s 2024-09-10T10:01:53.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:01:53 smithi102 ceph-mon[35442]: pgmap v1099: 129 pgs: 129 active+clean; 746 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 5.5 MiB/s wr, 383 op/s 2024-09-10T10:01:54.044 INFO:tasks.workunit.client.0.smithi005.stdout: 10 155822 13.92 MB/sec execute 452 sec latency 3079.144 ms 2024-09-10T10:01:55.044 INFO:tasks.workunit.client.0.smithi005.stdout: 10 155826 13.89 MB/sec execute 453 sec latency 4079.207 ms 2024-09-10T10:01:55.607 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:01:55 smithi005 ceph-mon[30350]: pgmap v1100: 129 pgs: 129 active+clean; 746 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 5.4 MiB/s wr, 347 op/s 2024-09-10T10:01:55.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:01:55 smithi159 ceph-mon[35513]: pgmap v1100: 129 pgs: 129 active+clean; 746 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 5.4 MiB/s wr, 347 op/s 2024-09-10T10:01:55.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:01:55 smithi102 ceph-mon[35442]: pgmap v1100: 129 pgs: 129 active+clean; 746 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 5.4 MiB/s wr, 347 op/s 2024-09-10T10:01:56.044 INFO:tasks.workunit.client.0.smithi005.stdout: 10 155826 13.86 MB/sec execute 454 sec latency 5079.247 ms 2024-09-10T10:01:57.044 INFO:tasks.workunit.client.0.smithi005.stdout: 10 155839 13.83 MB/sec execute 455 sec latency 5491.818 ms 2024-09-10T10:01:57.420 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:01:57 smithi005 ceph-mon[30350]: pgmap v1101: 129 pgs: 129 active+clean; 753 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 6.0 MiB/s wr, 370 op/s 2024-09-10T10:01:57.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:01:57 smithi159 ceph-mon[35513]: pgmap v1101: 129 pgs: 129 active+clean; 753 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 6.0 MiB/s wr, 370 op/s 2024-09-10T10:01:57.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:01:57 smithi102 ceph-mon[35442]: pgmap v1101: 129 pgs: 129 active+clean; 753 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 6.0 MiB/s wr, 370 op/s 2024-09-10T10:01:58.044 INFO:tasks.workunit.client.0.smithi005.stdout: 10 155843 13.80 MB/sec execute 456 sec latency 1575.447 ms 2024-09-10T10:01:59.044 INFO:tasks.workunit.client.0.smithi005.stdout: 10 155871 13.78 MB/sec execute 457 sec latency 2575.504 ms 2024-09-10T10:01:59.659 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:01:59 smithi005 ceph-mon[30350]: pgmap v1102: 129 pgs: 129 active+clean; 751 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 5.7 MiB/s wr, 268 op/s 2024-09-10T10:01:59.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:01:59 smithi159 ceph-mon[35513]: pgmap v1102: 129 pgs: 129 active+clean; 751 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 5.7 MiB/s wr, 268 op/s 2024-09-10T10:01:59.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:01:59 smithi102 ceph-mon[35442]: pgmap v1102: 129 pgs: 129 active+clean; 751 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 5.7 MiB/s wr, 268 op/s 2024-09-10T10:02:00.044 INFO:tasks.workunit.client.0.smithi005.stdout: 10 155871 13.75 MB/sec execute 458 sec latency 3575.558 ms 2024-09-10T10:02:01.044 INFO:tasks.workunit.client.0.smithi005.stdout: 10 155871 13.72 MB/sec execute 459 sec latency 4575.612 ms 2024-09-10T10:02:01.927 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:02:01 smithi005 ceph-mon[30350]: pgmap v1103: 129 pgs: 129 active+clean; 749 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 6.1 MiB/s wr, 270 op/s 2024-09-10T10:02:02.063 INFO:tasks.workunit.client.0.smithi005.stdout: 10 156035 13.71 MB/sec execute 460 sec latency 4759.110 ms 2024-09-10T10:02:02.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:02:01 smithi159 ceph-mon[35513]: pgmap v1103: 129 pgs: 129 active+clean; 749 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 6.1 MiB/s wr, 270 op/s 2024-09-10T10:02:02.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:02:01 smithi102 ceph-mon[35442]: pgmap v1103: 129 pgs: 129 active+clean; 749 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 6.1 MiB/s wr, 270 op/s 2024-09-10T10:02:03.063 INFO:tasks.workunit.client.0.smithi005.stdout: 10 156039 13.68 MB/sec execute 461 sec latency 1298.372 ms 2024-09-10T10:02:04.065 INFO:tasks.workunit.client.0.smithi005.stdout: 10 156073 13.65 MB/sec execute 462 sec latency 1930.467 ms 2024-09-10T10:02:04.187 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:02:03 smithi102 ceph-mon[35442]: pgmap v1104: 129 pgs: 129 active+clean; 751 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 246 KiB/s rd, 4.5 MiB/s wr, 178 op/s 2024-09-10T10:02:04.306 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:02:03 smithi005 ceph-mon[30350]: pgmap v1104: 129 pgs: 129 active+clean; 751 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 246 KiB/s rd, 4.5 MiB/s wr, 178 op/s 2024-09-10T10:02:04.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:02:03 smithi159 ceph-mon[35513]: pgmap v1104: 129 pgs: 129 active+clean; 751 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 246 KiB/s rd, 4.5 MiB/s wr, 178 op/s 2024-09-10T10:02:05.065 INFO:tasks.workunit.client.0.smithi005.stdout: 10 156073 13.62 MB/sec execute 463 sec latency 2930.539 ms 2024-09-10T10:02:06.065 INFO:tasks.workunit.client.0.smithi005.stdout: 10 156073 13.59 MB/sec execute 464 sec latency 3930.605 ms 2024-09-10T10:02:06.179 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:02:05 smithi005 ceph-mon[30350]: pgmap v1105: 129 pgs: 129 active+clean; 749 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 232 KiB/s rd, 4.8 MiB/s wr, 174 op/s 2024-09-10T10:02:06.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:02:05 smithi159 ceph-mon[35513]: pgmap v1105: 129 pgs: 129 active+clean; 749 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 232 KiB/s rd, 4.8 MiB/s wr, 174 op/s 2024-09-10T10:02:06.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:02:05 smithi102 ceph-mon[35442]: pgmap v1105: 129 pgs: 129 active+clean; 749 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 232 KiB/s rd, 4.8 MiB/s wr, 174 op/s 2024-09-10T10:02:07.067 INFO:tasks.workunit.client.0.smithi005.stdout: 10 156074 13.56 MB/sec execute 465 sec latency 4930.658 ms 2024-09-10T10:02:08.065 INFO:tasks.workunit.client.0.smithi005.stdout: 10 156088 13.54 MB/sec execute 466 sec latency 5930.732 ms 2024-09-10T10:02:08.132 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:02:07 smithi005 ceph-mon[30350]: pgmap v1106: 129 pgs: 129 active+clean; 744 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 400 KiB/s rd, 5.4 MiB/s wr, 225 op/s 2024-09-10T10:02:08.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:02:07 smithi159 ceph-mon[35513]: pgmap v1106: 129 pgs: 129 active+clean; 744 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 400 KiB/s rd, 5.4 MiB/s wr, 225 op/s 2024-09-10T10:02:08.379 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T10:02:08.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:02:07 smithi102 ceph-mon[35442]: pgmap v1106: 129 pgs: 129 active+clean; 744 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 400 KiB/s rd, 5.4 MiB/s wr, 225 op/s 2024-09-10T10:02:08.786 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:02:09.066 INFO:tasks.workunit.client.0.smithi005.stdout: 10 156089 13.51 MB/sec execute 467 sec latency 6930.793 ms 2024-09-10T10:02:09.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:09.651+0000 7f621c000640 1 -- 172.21.15.5:0/1873829162 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6214104010 msgr2=0x7f62141043f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:09.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:09.651+0000 7f621c000640 1 --2- 172.21.15.5:0/1873829162 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6214104010 0x7f62141043f0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f6208009900 tx=0x7f620802a410 comp rx=0 tx=0).stop 2024-09-10T10:02:09.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:09.651+0000 7f621c000640 1 -- 172.21.15.5:0/1873829162 shutdown_connections 2024-09-10T10:02:09.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:09.651+0000 7f621c000640 1 --2- 172.21.15.5:0/1873829162 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6214109140 0x7f6214110cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:09.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:09.651+0000 7f621c000640 1 --2- 172.21.15.5:0/1873829162 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f62141049c0 0x7f6214108a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:09.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:09.651+0000 7f621c000640 1 --2- 172.21.15.5:0/1873829162 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6214104010 0x7f62141043f0 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:09.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:09.651+0000 7f621c000640 1 -- 172.21.15.5:0/1873829162 >> 172.21.15.5:0/1873829162 conn(0x7f62140ffa50 msgr2=0x7f6214101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:09.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:09.651+0000 7f621c000640 1 -- 172.21.15.5:0/1873829162 shutdown_connections 2024-09-10T10:02:09.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:09.651+0000 7f621c000640 1 -- 172.21.15.5:0/1873829162 wait complete. 2024-09-10T10:02:09.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:09.651+0000 7f621c000640 1 Processor -- start 2024-09-10T10:02:09.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:09.651+0000 7f621c000640 1 -- start start 2024-09-10T10:02:09.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:09.651+0000 7f621c000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6214104010 0x7f621419a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:09.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:09.651+0000 7f621c000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f62141049c0 0x7f621419ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:09.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:09.651+0000 7f621c000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6214109140 0x7f621419efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:09.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:09.651+0000 7f621c000640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f6214114210 con 0x7f6214109140 2024-09-10T10:02:09.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:09.651+0000 7f621c000640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f6214114090 con 0x7f6214104010 2024-09-10T10:02:09.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:09.651+0000 7f621c000640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f6214114390 con 0x7f62141049c0 2024-09-10T10:02:09.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:09.651+0000 7f6219600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6214104010 0x7f621419a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:09.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:09.651+0000 7f6219600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6214104010 0x7f621419a6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:41640/0 (socket says 172.21.15.5:41640) 2024-09-10T10:02:09.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:09.651+0000 7f621a000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6214109140 0x7f621419efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:09.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:09.651+0000 7f6219600640 1 -- 172.21.15.5:0/2486414948 learned_addr learned my addr 172.21.15.5:0/2486414948 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T10:02:09.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:09.651+0000 7f6218c00640 1 --2- 172.21.15.5:0/2486414948 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f62141049c0 0x7f621419ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:09.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:09.651+0000 7f6219600640 1 -- 172.21.15.5:0/2486414948 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f62141049c0 msgr2=0x7f621419ac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:09.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:09.651+0000 7f6219600640 1 --2- 172.21.15.5:0/2486414948 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f62141049c0 0x7f621419ac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:09.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:09.651+0000 7f6219600640 1 -- 172.21.15.5:0/2486414948 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6214109140 msgr2=0x7f621419efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:09.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:09.651+0000 7f6219600640 1 --2- 172.21.15.5:0/2486414948 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6214109140 0x7f621419efc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:09.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:09.651+0000 7f6219600640 1 -- 172.21.15.5:0/2486414948 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f621419f6a0 con 0x7f6214104010 2024-09-10T10:02:09.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:09.651+0000 7f621a000640 1 --2- 172.21.15.5:0/2486414948 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6214109140 0x7f621419efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:02:09.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:09.651+0000 7f6219600640 1 --2- 172.21.15.5:0/2486414948 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6214104010 0x7f621419a6f0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f6208004770 tx=0x7f6208003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:09.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:09.657+0000 7f6212000640 1 -- 172.21.15.5:0/2486414948 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f620803f070 con 0x7f6214104010 2024-09-10T10:02:09.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:09.657+0000 7f6212000640 1 -- 172.21.15.5:0/2486414948 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6208031ca0 con 0x7f6214104010 2024-09-10T10:02:09.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:09.657+0000 7f6212000640 1 -- 172.21.15.5:0/2486414948 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6208035030 con 0x7f6214104010 2024-09-10T10:02:09.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:09.657+0000 7f621c000640 1 -- 172.21.15.5:0/2486414948 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f621419f930 con 0x7f6214104010 2024-09-10T10:02:09.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:09.657+0000 7f621c000640 1 -- 172.21.15.5:0/2486414948 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f621419fdc0 con 0x7f6214104010 2024-09-10T10:02:09.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:09.657+0000 7f621ec1b740 1 Processor -- start 2024-09-10T10:02:09.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:09.657+0000 7f621ec1b740 1 -- start start 2024-09-10T10:02:09.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:09.657+0000 7f621ec1b740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55747d6bcb90 0x55747d6bcf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:09.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:09.657+0000 7f621ec1b740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55747d6bd4b0 0x55747d6c7830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:09.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:09.657+0000 7f621ec1b740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55747d6c7d70 0x55747d6ca160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:09.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:09.657+0000 7f621ec1b740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55747d6233e0 con 0x55747d6c7d70 2024-09-10T10:02:09.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:09.657+0000 7f621ec1b740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55747d5cad80 con 0x55747d6bd4b0 2024-09-10T10:02:09.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:09.657+0000 7f621ec1b740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55747d61f3a0 con 0x55747d6bcb90 2024-09-10T10:02:09.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:09.657+0000 7f621a000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55747d6bcb90 0x55747d6bcf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:09.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:09.657+0000 7f6218c00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55747d6c7d70 0x55747d6ca160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:09.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:09.657+0000 7f621a000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55747d6bcb90 0x55747d6bcf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:37442/0 (socket says 172.21.15.5:37442) 2024-09-10T10:02:09.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:09.657+0000 7f621a000640 1 -- 172.21.15.5:0/222267131 learned_addr learned my addr 172.21.15.5:0/222267131 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T10:02:09.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:09.657+0000 7f6212000640 1 -- 172.21.15.5:0/2486414948 <== mon.2 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f6208042070 con 0x7f6214104010 2024-09-10T10:02:09.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:09.657+0000 7f621a000640 1 -- 172.21.15.5:0/222267131 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55747d6bd4b0 msgr2=0x55747d6c7830 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T10:02:09.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:09.657+0000 7f621a000640 1 --2- 172.21.15.5:0/222267131 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55747d6bd4b0 0x55747d6c7830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:09.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:09.657+0000 7f621a000640 1 -- 172.21.15.5:0/222267131 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55747d6c7d70 msgr2=0x55747d6ca160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:09.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:09.657+0000 7f621a000640 1 --2- 172.21.15.5:0/222267131 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55747d6c7d70 0x55747d6ca160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:09.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:09.657+0000 7f621a000640 1 -- 172.21.15.5:0/222267131 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55747d5e9ae0 con 0x55747d6bcb90 2024-09-10T10:02:09.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:09.662+0000 7f621a000640 1 --2- 172.21.15.5:0/222267131 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55747d6bcb90 0x55747d6bcf70 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f620c0107e0 tx=0x7f620c010ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6e3eaaf9ef9d1398 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:09.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:09.662+0000 7f6206600640 1 -- 172.21.15.5:0/222267131 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f620c003b70 con 0x55747d6bcb90 2024-09-10T10:02:09.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:09.662+0000 7f6206600640 1 -- 172.21.15.5:0/222267131 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f620c00ee10 con 0x55747d6bcb90 2024-09-10T10:02:09.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:09.662+0000 7f6206600640 1 -- 172.21.15.5:0/222267131 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f620c004770 con 0x55747d6bcb90 2024-09-10T10:02:09.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:09.662+0000 7f621ec1b740 1 -- 172.21.15.5:0/222267131 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55747d6bcb90 msgr2=0x55747d6bcf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:09.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:09.662+0000 7f621ec1b740 1 --2- 172.21.15.5:0/222267131 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55747d6bcb90 0x55747d6bcf70 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f620c0107e0 tx=0x7f620c010ca0 comp rx=0 tx=0).stop 2024-09-10T10:02:09.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:09.662+0000 7f6212000640 1 --2- 172.21.15.5:0/2486414948 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f61fc0a2c00 0x7f61fc0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:09.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:09.662+0000 7f621ec1b740 1 -- 172.21.15.5:0/222267131 shutdown_connections 2024-09-10T10:02:09.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:09.662+0000 7f621ec1b740 1 --2- 172.21.15.5:0/222267131 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55747d6c7d70 0x55747d6ca160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:09.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:09.662+0000 7f621ec1b740 1 --2- 172.21.15.5:0/222267131 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55747d6bd4b0 0x55747d6c7830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:09.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:09.662+0000 7f621ec1b740 1 --2- 172.21.15.5:0/222267131 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55747d6bcb90 0x55747d6bcf70 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:09.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:09.662+0000 7f621ec1b740 1 -- 172.21.15.5:0/222267131 >> 172.21.15.5:0/222267131 conn(0x55747d6b9120 msgr2=0x55747d6b9500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:09.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:09.662+0000 7f6212000640 1 -- 172.21.15.5:0/2486414948 <== mon.2 v2:172.21.15.102:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f62080efb60 con 0x7f6214104010 2024-09-10T10:02:09.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:09.662+0000 7f621a000640 1 --2- 172.21.15.5:0/2486414948 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f61fc0a2c00 0x7f61fc0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:09.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:09.662+0000 7f621ec1b740 1 -- 172.21.15.5:0/222267131 shutdown_connections 2024-09-10T10:02:09.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:09.662+0000 7f621ec1b740 1 -- 172.21.15.5:0/222267131 wait complete. 2024-09-10T10:02:09.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:09.662+0000 7f621ec1b740 1 -- start start 2024-09-10T10:02:09.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:09.662+0000 7f621a000640 1 --2- 172.21.15.5:0/2486414948 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f61fc0a2c00 0x7f61fc0a50c0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f620c005fd0 tx=0x7f620c009920 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:09.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:09.662+0000 7f621ec1b740 1 Processor -- start 2024-09-10T10:02:09.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:09.662+0000 7f621ec1b740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55747d6bcb90 0x55747d6c4d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:09.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:09.662+0000 7f621ec1b740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55747d6bd4b0 0x55747d6c5250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:09.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:09.662+0000 7f621ec1b740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55747d6c7d70 0x55747d6c5790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:09.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:09.662+0000 7f621ec1b740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55747d6ca9e0 con 0x55747d6bd4b0 2024-09-10T10:02:09.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:09.662+0000 7f621ec1b740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55747d5de0f0 con 0x55747d6c7d70 2024-09-10T10:02:09.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:09.662+0000 7f621ec1b740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55747d6cab60 con 0x55747d6bcb90 2024-09-10T10:02:09.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:09.662+0000 7f621ec1b740 10 client.0 _ll_register_callbacks cb 0x55747d5cf9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:02:09.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:09.662+0000 7f6218c00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55747d6bd4b0 0x55747d6c5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:09.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:09.662+0000 7f6219600640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55747d6bcb90 0x55747d6c4d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:09.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:09.662+0000 7f6218c00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55747d6bd4b0 0x55747d6c5250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57402/0 (socket says 172.21.15.5:57402) 2024-09-10T10:02:09.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:09.662+0000 7f6218c00640 1 -- 172.21.15.5:0/665186216 learned_addr learned my addr 172.21.15.5:0/665186216 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T10:02:09.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:09.662+0000 7f621a000640 1 --2- 172.21.15.5:0/665186216 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55747d6c7d70 0x55747d6c5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:09.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:09.662+0000 7f6219600640 1 -- 172.21.15.5:0/665186216 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55747d6c7d70 msgr2=0x55747d6c5790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:09.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:09.662+0000 7f6219600640 1 --2- 172.21.15.5:0/665186216 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55747d6c7d70 0x55747d6c5790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:09.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:09.662+0000 7f6219600640 1 -- 172.21.15.5:0/665186216 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55747d6bd4b0 msgr2=0x55747d6c5250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:09.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:09.662+0000 7f6219600640 1 --2- 172.21.15.5:0/665186216 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55747d6bd4b0 0x55747d6c5250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:09.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:09.662+0000 7f6219600640 1 -- 172.21.15.5:0/665186216 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55747d3943c0 con 0x55747d6bcb90 2024-09-10T10:02:09.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:09.662+0000 7f6218c00640 1 --2- 172.21.15.5:0/665186216 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55747d6bd4b0 0x55747d6c5250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:02:09.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:09.662+0000 7f6219600640 1 --2- 172.21.15.5:0/665186216 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55747d6bcb90 0x55747d6c4d10 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x55747d6b9f80 tx=0x7f62080bbb70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fb32940e7e660b7e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:09.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:09.662+0000 7f61faa00640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-09-10T10:02:09.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:09.667+0000 7f61faa00640 1 -- 172.21.15.5:0/665186216 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62080d6060 con 0x55747d6bcb90 2024-09-10T10:02:09.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:09.667+0000 7f621ec1b740 1 -- 172.21.15.5:0/665186216 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f620c00ee10 con 0x55747d6bcb90 2024-09-10T10:02:09.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:09.667+0000 7f61faa00640 1 -- 172.21.15.5:0/665186216 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f620803b4f0 con 0x55747d6bcb90 2024-09-10T10:02:09.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:09.667+0000 7f61faa00640 1 -- 172.21.15.5:0/665186216 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f6208049050 con 0x55747d6bcb90 2024-09-10T10:02:09.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:09.667+0000 7f621ec1b740 10 client.29833 fetch_fsmap learned FSMap version 42 2024-09-10T10:02:09.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:09.667+0000 7f621ec1b740 1 -- 172.21.15.5:0/665186216 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55747d6caa90 con 0x55747d6bcb90 2024-09-10T10:02:09.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:09.667+0000 7f61faa00640 1 -- 172.21.15.5:0/665186216 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f620c00ee10 con 0x55747d6bcb90 2024-09-10T10:02:09.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:09.667+0000 7f61faa00640 1 -- 172.21.15.5:0/665186216 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 42) ==== 8370+0+0 (secure 0 0 0) 0x7f62080bc1d0 con 0x55747d6bcb90 2024-09-10T10:02:09.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:09.667+0000 7f621ec1b740 10 client.29833 fetch_fsmap finished waiting for FSMap version 42 2024-09-10T10:02:09.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:09.667+0000 7f621ec1b740 10 client.29833 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T10:02:09.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:09.667+0000 7f621ec1b740 20 client.29833 populate_metadata read hostname 'smithi005' 2024-09-10T10:02:09.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:09.667+0000 7f621ec1b740 1 --2- 172.21.15.5:0/665186216 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55747d6d7c20 0x55747d6da050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:09.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:09.667+0000 7f621ec1b740 4 client.29833 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:02:09.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:09.667+0000 7f621ec1b740 1 -- 172.21.15.5:0/665186216 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55747d630950 con 0x55747d6d7c20 2024-09-10T10:02:09.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:09.667+0000 7f6218c00640 1 --2- 172.21.15.5:0/665186216 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55747d6d7c20 0x55747d6da050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:09.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:09.667+0000 7f6218c00640 1 --2- 172.21.15.5:0/665186216 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55747d6d7c20 0x55747d6da050 crc :-1 s=READY pgs=319 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6b8d1beccc15ce6d server_cookie=b572f0c7c472b920 in_seq=0 out_seq=0 2024-09-10T10:02:09.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:09.667+0000 7f61faa00640 10 client.29833 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T10:02:09.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:09.667+0000 7f61faa00640 1 -- 172.21.15.5:0/665186216 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55747d630950 con 0x55747d6d7c20 2024-09-10T10:02:09.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:09.667+0000 7f61faa00640 10 client.29833 handle_command_reply: tid=0 2024-09-10T10:02:09.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:09.667+0000 7f621ec1b740 1 client.29833 shutdown 2024-09-10T10:02:09.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:09.667+0000 7f621ec1b740 1 -- 172.21.15.5:0/665186216 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55747d6bcb90 msgr2=0x55747d6c4d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:09.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:09.667+0000 7f621ec1b740 1 --2- 172.21.15.5:0/665186216 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55747d6bcb90 0x55747d6c4d10 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x55747d6b9f80 tx=0x7f62080bbb70 comp rx=0 tx=0).stop 2024-09-10T10:02:09.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:09.667+0000 7f621ec1b740 1 -- 172.21.15.5:0/665186216 shutdown_connections 2024-09-10T10:02:09.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:09.667+0000 7f621ec1b740 1 --2- 172.21.15.5:0/665186216 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55747d6d7c20 0x55747d6da050 crc :-1 s=READY pgs=319 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:09.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:09.667+0000 7f621ec1b740 1 --2- 172.21.15.5:0/665186216 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55747d6c7d70 0x55747d6c5790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:09.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:09.667+0000 7f621ec1b740 1 --2- 172.21.15.5:0/665186216 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55747d6bd4b0 0x55747d6c5250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:09.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:09.667+0000 7f621ec1b740 1 --2- 172.21.15.5:0/665186216 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55747d6bcb90 0x55747d6c4d10 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:09.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:09.667+0000 7f61faa00640 0 client.29833 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T10:02:09.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:09.667+0000 7f621ec1b740 1 -- 172.21.15.5:0/665186216 >> 172.21.15.5:0/665186216 conn(0x55747d6b9120 msgr2=0x55747d6bee00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:09.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:09.667+0000 7f621ec1b740 1 -- 172.21.15.5:0/665186216 shutdown_connections 2024-09-10T10:02:09.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:09.667+0000 7f621ec1b740 1 -- 172.21.15.5:0/665186216 wait complete. 2024-09-10T10:02:09.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:09.667+0000 7f621ec1b740 20 client.29833 trim_cache size 0 max 16384 2024-09-10T10:02:09.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:09.686+0000 7f621ec1b740 1 Processor -- start 2024-09-10T10:02:09.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:09.686+0000 7f621ec1b740 1 -- start start 2024-09-10T10:02:09.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:09.686+0000 7f621ec1b740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55747d6bcb90 0x55747d6c55e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:09.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:09.686+0000 7f621ec1b740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55747d6bd4b0 0x55747d6df960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:09.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:09.686+0000 7f621ec1b740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55747d6c7d70 0x55747d6dfea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:09.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:09.686+0000 7f621ec1b740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f62141b1080 con 0x55747d6c7d70 2024-09-10T10:02:09.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:09.686+0000 7f621ec1b740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f62141b2bc0 con 0x55747d6bd4b0 2024-09-10T10:02:09.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:09.686+0000 7f621ec1b740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f62141b1750 con 0x55747d6bcb90 2024-09-10T10:02:09.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:09.686+0000 7f6219600640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55747d6bcb90 0x55747d6c55e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:09.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:09.686+0000 7f6219600640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55747d6bcb90 0x55747d6c55e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:37470/0 (socket says 172.21.15.5:37470) 2024-09-10T10:02:09.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:09.686+0000 7f6219600640 1 -- 172.21.15.5:0/654487829 learned_addr learned my addr 172.21.15.5:0/654487829 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T10:02:09.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:09.686+0000 7f621a000640 1 --2- 172.21.15.5:0/654487829 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55747d6c7d70 0x55747d6dfea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:09.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:09.686+0000 7f621a000640 1 -- 172.21.15.5:0/654487829 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55747d6bcb90 msgr2=0x55747d6c55e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:09.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:09.686+0000 7f621a000640 1 --2- 172.21.15.5:0/654487829 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55747d6bcb90 0x55747d6c55e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:09.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:09.686+0000 7f621a000640 1 -- 172.21.15.5:0/654487829 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55747d6bd4b0 msgr2=0x55747d6df960 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T10:02:09.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:09.686+0000 7f621a000640 1 --2- 172.21.15.5:0/654487829 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55747d6bd4b0 0x55747d6df960 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:09.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:09.686+0000 7f621a000640 1 -- 172.21.15.5:0/654487829 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55747d6da6d0 con 0x55747d6c7d70 2024-09-10T10:02:09.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:09.686+0000 7f621a000640 1 --2- 172.21.15.5:0/654487829 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55747d6c7d70 0x55747d6dfea0 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f620c011010 tx=0x7f620c0096a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=769f3706c3303ef2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:09.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:09.686+0000 7f6205c00640 1 -- 172.21.15.5:0/654487829 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f620c003e70 con 0x55747d6c7d70 2024-09-10T10:02:09.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:09.686+0000 7f6205c00640 1 -- 172.21.15.5:0/654487829 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f620c005530 con 0x55747d6c7d70 2024-09-10T10:02:09.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:09.686+0000 7f6205c00640 1 -- 172.21.15.5:0/654487829 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f620c01a980 con 0x55747d6c7d70 2024-09-10T10:02:09.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:09.691+0000 7f621ec1b740 1 -- 172.21.15.5:0/654487829 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55747d6c7d70 msgr2=0x55747d6dfea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:09.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:09.691+0000 7f621ec1b740 1 --2- 172.21.15.5:0/654487829 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55747d6c7d70 0x55747d6dfea0 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f620c011010 tx=0x7f620c0096a0 comp rx=0 tx=0).stop 2024-09-10T10:02:09.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:09.691+0000 7f621ec1b740 1 -- 172.21.15.5:0/654487829 shutdown_connections 2024-09-10T10:02:09.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:09.691+0000 7f621ec1b740 1 --2- 172.21.15.5:0/654487829 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55747d6c7d70 0x55747d6dfea0 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:09.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:09.691+0000 7f621ec1b740 1 --2- 172.21.15.5:0/654487829 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55747d6bd4b0 0x55747d6df960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:09.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:09.691+0000 7f621ec1b740 1 --2- 172.21.15.5:0/654487829 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55747d6bcb90 0x55747d6c55e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:09.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:09.691+0000 7f621ec1b740 1 -- 172.21.15.5:0/654487829 >> 172.21.15.5:0/654487829 conn(0x55747d6b9120 msgr2=0x55747d6bee00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:09.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:09.691+0000 7f621ec1b740 1 -- 172.21.15.5:0/654487829 shutdown_connections 2024-09-10T10:02:09.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:09.691+0000 7f621ec1b740 1 -- 172.21.15.5:0/654487829 wait complete. 2024-09-10T10:02:09.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:09.691+0000 7f621ec1b740 1 -- start start 2024-09-10T10:02:09.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:09.691+0000 7f621ec1b740 1 Processor -- start 2024-09-10T10:02:09.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:09.691+0000 7f621ec1b740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55747d6bcb90 0x55747d6cfff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:09.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:09.691+0000 7f621ec1b740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55747d6bd4b0 0x55747d6d0530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:09.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:09.691+0000 7f621ec1b740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55747d6c7d70 0x55747d6ccde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:09.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:09.691+0000 7f621ec1b740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f62141b24f0 con 0x55747d6c7d70 2024-09-10T10:02:09.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:09.691+0000 7f621ec1b740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f62141b1e20 con 0x55747d6bcb90 2024-09-10T10:02:09.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:09.691+0000 7f621ec1b740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f62141af060 con 0x55747d6bd4b0 2024-09-10T10:02:09.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:09.691+0000 7f621ec1b740 10 client.0 _ll_register_callbacks cb 0x7f62080b82a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:02:09.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:09.691+0000 7f621a000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55747d6c7d70 0x55747d6ccde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:09.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:09.691+0000 7f6218c00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55747d6bd4b0 0x55747d6d0530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:09.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:09.691+0000 7f6219600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55747d6bcb90 0x55747d6cfff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:09.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:09.691+0000 7f621a000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55747d6c7d70 0x55747d6ccde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57420/0 (socket says 172.21.15.5:57420) 2024-09-10T10:02:09.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:09.691+0000 7f621a000640 1 -- 172.21.15.5:0/844094582 learned_addr learned my addr 172.21.15.5:0/844094582 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T10:02:09.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:09.691+0000 7f6218c00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55747d6bd4b0 0x55747d6d0530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:37480/0 (socket says 172.21.15.5:37480) 2024-09-10T10:02:09.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:09.691+0000 7f6218c00640 1 -- 172.21.15.5:0/844094582 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55747d6bcb90 msgr2=0x55747d6cfff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:09.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:09.691+0000 7f6218c00640 1 --2- 172.21.15.5:0/844094582 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55747d6bcb90 0x55747d6cfff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:09.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:09.691+0000 7f6218c00640 1 -- 172.21.15.5:0/844094582 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55747d6c7d70 msgr2=0x55747d6ccde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:09.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:09.691+0000 7f6218c00640 1 --2- 172.21.15.5:0/844094582 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55747d6c7d70 0x55747d6ccde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:09.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:09.691+0000 7f6218c00640 1 -- 172.21.15.5:0/844094582 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55747d6cd5e0 con 0x55747d6bd4b0 2024-09-10T10:02:09.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:09.691+0000 7f6219600640 1 --2- 172.21.15.5:0/844094582 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55747d6bcb90 0x55747d6cfff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:02:09.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:09.691+0000 7f621a000640 1 --2- 172.21.15.5:0/844094582 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55747d6c7d70 0x55747d6ccde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:02:09.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:09.698+0000 7f6218c00640 1 --2- 172.21.15.5:0/844094582 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55747d6bd4b0 0x55747d6d0530 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x55747d6da8a0 tx=0x7f62000178d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=132df94591d8f3ea server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:09.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:09.698+0000 7f61faa00640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-09-10T10:02:09.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:09.698+0000 7f61faa00640 1 -- 172.21.15.5:0/844094582 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f620001d520 con 0x55747d6bd4b0 2024-09-10T10:02:09.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:09.698+0000 7f621ec1b740 1 -- 172.21.15.5:0/844094582 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f620c005530 con 0x55747d6bd4b0 2024-09-10T10:02:09.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:09.698+0000 7f61faa00640 1 -- 172.21.15.5:0/844094582 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f620000f040 con 0x55747d6bd4b0 2024-09-10T10:02:09.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:09.698+0000 7f61faa00640 1 -- 172.21.15.5:0/844094582 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f620001d6c0 con 0x55747d6bd4b0 2024-09-10T10:02:09.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:09.698+0000 7f621ec1b740 10 client.29839 fetch_fsmap learned FSMap version 42 2024-09-10T10:02:09.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:09.698+0000 7f621ec1b740 1 -- 172.21.15.5:0/844094582 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55747d743500 con 0x55747d6bd4b0 2024-09-10T10:02:09.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:09.698+0000 7f61faa00640 1 -- 172.21.15.5:0/844094582 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f620c005530 con 0x55747d6bd4b0 2024-09-10T10:02:09.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:09.698+0000 7f61faa00640 1 -- 172.21.15.5:0/844094582 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 42) ==== 8370+0+0 (secure 0 0 0) 0x7f6200030b90 con 0x55747d6bd4b0 2024-09-10T10:02:09.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:09.698+0000 7f621ec1b740 10 client.29839 fetch_fsmap finished waiting for FSMap version 42 2024-09-10T10:02:09.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:09.698+0000 7f621ec1b740 10 client.29839 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T10:02:09.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:09.698+0000 7f621ec1b740 20 client.29839 populate_metadata read hostname 'smithi005' 2024-09-10T10:02:09.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:09.698+0000 7f621ec1b740 1 --2- 172.21.15.5:0/844094582 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55747d743c30 0x55747d746050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:09.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:09.698+0000 7f621ec1b740 4 client.29839 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T10:02:09.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:09.698+0000 7f621ec1b740 1 -- 172.21.15.5:0/844094582 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55747d62ae60 con 0x55747d743c30 2024-09-10T10:02:09.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:09.698+0000 7f6219600640 1 --2- 172.21.15.5:0/844094582 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55747d743c30 0x55747d746050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:09.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:09.698+0000 7f6219600640 1 --2- 172.21.15.5:0/844094582 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55747d743c30 0x55747d746050 crc :-1 s=READY pgs=320 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b1153bb90db5b14f server_cookie=be56215a17f9c417 in_seq=0 out_seq=0 2024-09-10T10:02:09.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:09.698+0000 7f61faa00640 10 client.29839 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T10:02:09.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:09.698+0000 7f61faa00640 1 -- 172.21.15.5:0/844094582 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55747d62ae60 con 0x55747d743c30 2024-09-10T10:02:09.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:09.698+0000 7f61faa00640 10 client.29839 handle_command_reply: tid=0 2024-09-10T10:02:09.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:09.698+0000 7f621ec1b740 1 client.29839 shutdown 2024-09-10T10:02:09.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:09.698+0000 7f621ec1b740 1 -- 172.21.15.5:0/844094582 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55747d6bd4b0 msgr2=0x55747d6d0530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:09.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:09.698+0000 7f621ec1b740 1 --2- 172.21.15.5:0/844094582 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55747d6bd4b0 0x55747d6d0530 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x55747d6da8a0 tx=0x7f62000178d0 comp rx=0 tx=0).stop 2024-09-10T10:02:09.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:09.698+0000 7f621ec1b740 1 -- 172.21.15.5:0/844094582 shutdown_connections 2024-09-10T10:02:09.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:09.698+0000 7f621ec1b740 1 --2- 172.21.15.5:0/844094582 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55747d743c30 0x55747d746050 crc :-1 s=READY pgs=320 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:09.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:09.698+0000 7f621ec1b740 1 --2- 172.21.15.5:0/844094582 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55747d6c7d70 0x55747d6ccde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:09.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:09.698+0000 7f621ec1b740 1 --2- 172.21.15.5:0/844094582 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55747d6bd4b0 0x55747d6d0530 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:09.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:09.698+0000 7f621ec1b740 1 --2- 172.21.15.5:0/844094582 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55747d6bcb90 0x55747d6cfff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:09.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:09.698+0000 7f621ec1b740 1 -- 172.21.15.5:0/844094582 >> 172.21.15.5:0/844094582 conn(0x55747d6b9120 msgr2=0x55747d6b79a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:09.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:09.698+0000 7f61faa00640 0 client.29839 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T10:02:09.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:09.698+0000 7f621ec1b740 1 -- 172.21.15.5:0/844094582 shutdown_connections 2024-09-10T10:02:09.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:09.698+0000 7f621ec1b740 1 -- 172.21.15.5:0/844094582 wait complete. 2024-09-10T10:02:09.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:09.698+0000 7f621ec1b740 20 client.29839 trim_cache size 0 max 16384 2024-09-10T10:02:09.705 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-10T10:02:09.705 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "no active scrubs running", 2024-09-10T10:02:09.705 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": {} 2024-09-10T10:02:09.705 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-10T10:02:09.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:09.698+0000 7f621c000640 1 -- 172.21.15.5:0/2486414948 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f61fc0a2c00 msgr2=0x7f61fc0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:09.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:09.698+0000 7f621c000640 1 --2- 172.21.15.5:0/2486414948 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f61fc0a2c00 0x7f61fc0a50c0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f620c005fd0 tx=0x7f620c009920 comp rx=0 tx=0).stop 2024-09-10T10:02:09.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:09.698+0000 7f621c000640 1 -- 172.21.15.5:0/2486414948 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6214104010 msgr2=0x7f621419a6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:09.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:09.698+0000 7f621c000640 1 --2- 172.21.15.5:0/2486414948 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6214104010 0x7f621419a6f0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f6208004770 tx=0x7f6208003c10 comp rx=0 tx=0).stop 2024-09-10T10:02:09.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:09.704+0000 7f621c000640 1 -- 172.21.15.5:0/2486414948 shutdown_connections 2024-09-10T10:02:09.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:09.704+0000 7f621c000640 1 --2- 172.21.15.5:0/2486414948 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f61fc0a2c00 0x7f61fc0a50c0 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:09.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:09.704+0000 7f621c000640 1 --2- 172.21.15.5:0/2486414948 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6214109140 0x7f621419efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:09.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:09.704+0000 7f621c000640 1 --2- 172.21.15.5:0/2486414948 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f62141049c0 0x7f621419ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:09.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:09.704+0000 7f621c000640 1 --2- 172.21.15.5:0/2486414948 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6214104010 0x7f621419a6f0 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:09.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:09.704+0000 7f621c000640 1 -- 172.21.15.5:0/2486414948 >> 172.21.15.5:0/2486414948 conn(0x7f62140ffa50 msgr2=0x7f6214100dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:09.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:09.704+0000 7f621c000640 1 -- 172.21.15.5:0/2486414948 shutdown_connections 2024-09-10T10:02:09.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:09.704+0000 7f621c000640 1 -- 172.21.15.5:0/2486414948 wait complete. 2024-09-10T10:02:10.066 INFO:tasks.workunit.client.0.smithi005.stdout: 10 156090 13.48 MB/sec execute 468 sec latency 7929.115 ms 2024-09-10T10:02:10.076 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:02:09 smithi005 ceph-mon[30350]: pgmap v1107: 129 pgs: 129 active+clean; 742 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 348 KiB/s rd, 5.2 MiB/s wr, 223 op/s 2024-09-10T10:02:10.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:02:09 smithi159 ceph-mon[35513]: pgmap v1107: 129 pgs: 129 active+clean; 742 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 348 KiB/s rd, 5.2 MiB/s wr, 223 op/s 2024-09-10T10:02:10.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:02:09 smithi102 ceph-mon[35442]: pgmap v1107: 129 pgs: 129 active+clean; 742 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 348 KiB/s rd, 5.2 MiB/s wr, 223 op/s 2024-09-10T10:02:11.067 INFO:tasks.workunit.client.0.smithi005.stdout: 10 156090 13.45 MB/sec execute 469 sec latency 7033.459 ms 2024-09-10T10:02:12.005 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-10T10:02:12.005 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T10:02:12.066 INFO:tasks.workunit.client.0.smithi005.stdout: 10 156090 13.42 MB/sec execute 470 sec latency 8033.674 ms 2024-09-10T10:02:12.177 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:02:11 smithi005 ceph-mon[30350]: pgmap v1108: 129 pgs: 129 active+clean; 752 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 365 KiB/s rd, 5.9 MiB/s wr, 237 op/s 2024-09-10T10:02:12.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:02:11 smithi159 ceph-mon[35513]: pgmap v1108: 129 pgs: 129 active+clean; 752 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 365 KiB/s rd, 5.9 MiB/s wr, 237 op/s 2024-09-10T10:02:12.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:02:11 smithi102 ceph-mon[35442]: pgmap v1108: 129 pgs: 129 active+clean; 752 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 365 KiB/s rd, 5.9 MiB/s wr, 237 op/s 2024-09-10T10:02:12.558 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:02:13.066 INFO:tasks.workunit.client.0.smithi005.stdout: 10 156090 13.39 MB/sec execute 471 sec latency 9033.737 ms 2024-09-10T10:02:13.306 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:02:12 smithi005 ceph-mon[30350]: pgmap v1109: 129 pgs: 129 active+clean; 755 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 302 KiB/s rd, 4.9 MiB/s wr, 196 op/s 2024-09-10T10:02:13.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:02:12 smithi159 ceph-mon[35513]: pgmap v1109: 129 pgs: 129 active+clean; 755 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 302 KiB/s rd, 4.9 MiB/s wr, 196 op/s 2024-09-10T10:02:13.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:02:12 smithi102 ceph-mon[35442]: pgmap v1109: 129 pgs: 129 active+clean; 755 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 302 KiB/s rd, 4.9 MiB/s wr, 196 op/s 2024-09-10T10:02:14.066 INFO:tasks.workunit.client.0.smithi005.stdout: 10 156090 13.37 MB/sec execute 472 sec latency 10033.795 ms 2024-09-10T10:02:14.871 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 15.3 2024-09-10T10:02:14.871 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T10:02:15.066 INFO:tasks.workunit.client.0.smithi005.stdout: 10 156090 13.34 MB/sec execute 473 sec latency 11033.847 ms 2024-09-10T10:02:15.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:15.259+0000 7f165e000640 1 -- 172.21.15.5:0/4174397848 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1658074ee0 msgr2=0x7f165810dd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:15.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:15.259+0000 7f165e000640 1 --2- 172.21.15.5:0/4174397848 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1658074ee0 0x7f165810dd20 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f1644009860 tx=0x7f164402a3d0 comp rx=0 tx=0).stop 2024-09-10T10:02:15.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:15.260+0000 7f165e000640 1 -- 172.21.15.5:0/4174397848 shutdown_connections 2024-09-10T10:02:15.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:15.260+0000 7f165e000640 1 --2- 172.21.15.5:0/4174397848 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f165810e260 0x7f1658110650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:15.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:15.260+0000 7f165e000640 1 --2- 172.21.15.5:0/4174397848 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1658074ee0 0x7f165810dd20 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:15.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:15.260+0000 7f165e000640 1 --2- 172.21.15.5:0/4174397848 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f16580745c0 0x7f16580749a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:15.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:15.260+0000 7f165e000640 1 -- 172.21.15.5:0/4174397848 >> 172.21.15.5:0/4174397848 conn(0x7f16580fd740 msgr2=0x7f16580ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:15.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:15.260+0000 7f165e000640 1 -- 172.21.15.5:0/4174397848 shutdown_connections 2024-09-10T10:02:15.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:15.260+0000 7f165e000640 1 -- 172.21.15.5:0/4174397848 wait complete. 2024-09-10T10:02:15.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:15.261+0000 7f165e000640 1 Processor -- start 2024-09-10T10:02:15.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:15.261+0000 7f165e000640 1 -- start start 2024-09-10T10:02:15.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:15.261+0000 7f165e000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f16580745c0 0x7f165819a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:15.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:15.261+0000 7f165e000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1658074ee0 0x7f165819a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:15.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:15.261+0000 7f165e000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f165810e260 0x7f165819ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:15.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:15.262+0000 7f165e000640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1658114000 con 0x7f165810e260 2024-09-10T10:02:15.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:15.262+0000 7f165e000640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f1658113e80 con 0x7f1658074ee0 2024-09-10T10:02:15.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:15.262+0000 7f165e000640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f1658114180 con 0x7f16580745c0 2024-09-10T10:02:15.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:15.262+0000 7f165d600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f165810e260 0x7f165819ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:15.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:15.262+0000 7f165d600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f165810e260 0x7f165819ed70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36928/0 (socket says 172.21.15.5:36928) 2024-09-10T10:02:15.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:15.262+0000 7f165d600640 1 -- 172.21.15.5:0/4001730043 learned_addr learned my addr 172.21.15.5:0/4001730043 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T10:02:15.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:15.262+0000 7f1657e00640 1 --2- 172.21.15.5:0/4001730043 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1658074ee0 0x7f165819a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:15.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:15.262+0000 7f1657e00640 1 -- 172.21.15.5:0/4001730043 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f16580745c0 msgr2=0x7f165819a4a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T10:02:15.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:15.262+0000 7f1657e00640 1 --2- 172.21.15.5:0/4001730043 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f16580745c0 0x7f165819a4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:15.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:15.262+0000 7f1657e00640 1 -- 172.21.15.5:0/4001730043 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f165810e260 msgr2=0x7f165819ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:15.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:15.262+0000 7f1657e00640 1 --2- 172.21.15.5:0/4001730043 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f165810e260 0x7f165819ed70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:15.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:15.262+0000 7f1657e00640 1 -- 172.21.15.5:0/4001730043 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f165819f450 con 0x7f1658074ee0 2024-09-10T10:02:15.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:15.263+0000 7f1657e00640 1 --2- 172.21.15.5:0/4001730043 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1658074ee0 0x7f165819a9e0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f164402a8f0 tx=0x7f16440309a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:15.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:15.263+0000 7f1655600640 1 -- 172.21.15.5:0/4001730043 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1644040070 con 0x7f1658074ee0 2024-09-10T10:02:15.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:15.263+0000 7f165e000640 1 -- 172.21.15.5:0/4001730043 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f165819f6e0 con 0x7f1658074ee0 2024-09-10T10:02:15.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:15.263+0000 7f1655600640 1 -- 172.21.15.5:0/4001730043 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f16440055a0 con 0x7f1658074ee0 2024-09-10T10:02:15.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:15.263+0000 7f1655600640 1 -- 172.21.15.5:0/4001730043 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1644004920 con 0x7f1658074ee0 2024-09-10T10:02:15.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:15.263+0000 7f165e000640 1 -- 172.21.15.5:0/4001730043 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16581a7020 con 0x7f1658074ee0 2024-09-10T10:02:15.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:15.265+0000 7f165e000640 1 -- 172.21.15.5:0/4001730043 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f16580749a0 con 0x7f1658074ee0 2024-09-10T10:02:15.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:15.266+0000 7f1655600640 1 -- 172.21.15.5:0/4001730043 <== mon.2 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f1644043070 con 0x7f1658074ee0 2024-09-10T10:02:15.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:15.268+0000 7f1655600640 1 --2- 172.21.15.5:0/4001730043 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f16400a2b80 0x7f16400a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:15.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:15.268+0000 7f1655600640 1 -- 172.21.15.5:0/4001730043 <== mon.2 v2:172.21.15.102:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f16440f0150 con 0x7f1658074ee0 2024-09-10T10:02:15.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:15.269+0000 7f165cc00640 1 --2- 172.21.15.5:0/4001730043 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f16400a2b80 0x7f16400a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:15.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:15.269+0000 7f165cc00640 1 --2- 172.21.15.5:0/4001730043 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f16400a2b80 0x7f16400a5040 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f164c005fd0 tx=0x7f164c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:15.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:15.274+0000 7f1655600640 1 -- 172.21.15.5:0/4001730043 <== mon.2 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f16440bcc60 con 0x7f1658074ee0 2024-09-10T10:02:15.381 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:02:15.453 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:02:15 smithi005 ceph-mon[30350]: pgmap v1110: 129 pgs: 129 active+clean; 755 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 284 KiB/s rd, 4.8 MiB/s wr, 196 op/s 2024-09-10T10:02:15.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:15.515+0000 7f165e000640 1 -- 172.21.15.5:0/4001730043 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f16581093e0 con 0x7f1658074ee0 2024-09-10T10:02:15.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:15.516+0000 7f1655600640 1 -- 172.21.15.5:0/4001730043 <== mon.2 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 42 v42) ==== 94+0+11969 (secure 0 0 0) 0x7f16440bc560 con 0x7f1658074ee0 2024-09-10T10:02:15.518 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-10T10:02:15.518 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":42,"btime":"2024-09-10T09:55:06:670916+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14454,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/2970594311","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2970594311},{"type":"v1","addr":"172.21.15.5:6835","nonce":2970594311}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14514,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6839/3616502174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":3616502174},{"type":"v1","addr":"172.21.15.5:6839","nonce":3616502174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":42,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T09:35:29.626287+0000","modified":"2024-09-10T09:55:06.419829+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24542,"mds_1":24535,"mds_2":24511,"mds_3":24506,"mds_4":24443},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14520":{"gid":14520,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/2073599842","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":2073599842},{"type":"v1","addr":"172.21.15.5:6841","nonce":2073599842}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"k","rank":4,"incarnation":26,"state":"up:active","state_seq":24,"addr":"172.21.15.102:6833/2457007427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":2457007427},{"type":"v1","addr":"172.21.15.102:6833","nonce":2457007427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/356445159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":356445159},{"type":"v1","addr":"172.21.15.159:6833","nonce":356445159}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6837/2894314507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":2894314507},{"type":"v1","addr":"172.21.15.5:6837","nonce":2894314507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.102:6835/1647549111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":1647549111},{"type":"v1","addr":"172.21.15.102:6835","nonce":1647549111}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6835/952657242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":952657242},{"type":"v1","addr":"172.21.15.159:6835","nonce":952657242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":19,"addr":"172.21.15.102:6837/2689776764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":2689776764},{"type":"v1","addr":"172.21.15.102:6837","nonce":2689776764}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.159:6837/620627639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":620627639},{"type":"v1","addr":"172.21.15.159:6837","nonce":620627639}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24535":{"gid":24535,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":18,"addr":"172.21.15.159:6839/2073739599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":2073739599},{"type":"v1","addr":"172.21.15.159:6839","nonce":2073739599}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24542":{"gid":24542,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.102:6839/335409749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6838","nonce":335409749},{"type":"v1","addr":"172.21.15.102:6839","nonce":335409749}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24542,"qdb_cluster":[24542,24535,24511,24506,24443]},"id":1}]} 2024-09-10T10:02:15.518 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 42 2024-09-10T10:02:15.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:15.518+0000 7f165e000640 1 -- 172.21.15.5:0/4001730043 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f16400a2b80 msgr2=0x7f16400a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:15.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:15.518+0000 7f165e000640 1 --2- 172.21.15.5:0/4001730043 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f16400a2b80 0x7f16400a5040 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f164c005fd0 tx=0x7f164c009260 comp rx=0 tx=0).stop 2024-09-10T10:02:15.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:15.518+0000 7f165e000640 1 -- 172.21.15.5:0/4001730043 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1658074ee0 msgr2=0x7f165819a9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:15.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:15.518+0000 7f165e000640 1 --2- 172.21.15.5:0/4001730043 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1658074ee0 0x7f165819a9e0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f164402a8f0 tx=0x7f16440309a0 comp rx=0 tx=0).stop 2024-09-10T10:02:15.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:15.518+0000 7f165e000640 1 -- 172.21.15.5:0/4001730043 shutdown_connections 2024-09-10T10:02:15.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:15.518+0000 7f165e000640 1 --2- 172.21.15.5:0/4001730043 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f16400a2b80 0x7f16400a5040 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:15.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:15.518+0000 7f165e000640 1 --2- 172.21.15.5:0/4001730043 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f165810e260 0x7f165819ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:15.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:15.518+0000 7f165e000640 1 --2- 172.21.15.5:0/4001730043 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1658074ee0 0x7f165819a9e0 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:15.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:15.518+0000 7f165e000640 1 --2- 172.21.15.5:0/4001730043 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f16580745c0 0x7f165819a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:15.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:15.518+0000 7f165e000640 1 -- 172.21.15.5:0/4001730043 >> 172.21.15.5:0/4001730043 conn(0x7f16580fd740 msgr2=0x7f16580fe3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:15.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:15.518+0000 7f165e000640 1 -- 172.21.15.5:0/4001730043 shutdown_connections 2024-09-10T10:02:15.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:15.518+0000 7f165e000640 1 -- 172.21.15.5:0/4001730043 wait complete. 2024-09-10T10:02:15.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:02:15 smithi159 ceph-mon[35513]: pgmap v1110: 129 pgs: 129 active+clean; 755 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 284 KiB/s rd, 4.8 MiB/s wr, 196 op/s 2024-09-10T10:02:15.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:02:15 smithi102 ceph-mon[35442]: pgmap v1110: 129 pgs: 129 active+clean; 755 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 284 KiB/s rd, 4.8 MiB/s wr, 196 op/s 2024-09-10T10:02:16.066 INFO:tasks.workunit.client.0.smithi005.stdout: 10 156092 13.31 MB/sec execute 474 sec latency 12033.899 ms 2024-09-10T10:02:16.322 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:02:16 smithi005 ceph-mon[30350]: from='client.? 172.21.15.5:0/4001730043' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:02:16.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:16.330+0000 7fa019800640 1 -- 172.21.15.5:0/290149140 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa014074510 msgr2=0x7fa0140748f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:16.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:16.330+0000 7fa019800640 1 --2- 172.21.15.5:0/290149140 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa014074510 0x7fa0140748f0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7fa004009860 tx=0x7fa00402a3d0 comp rx=0 tx=0).stop 2024-09-10T10:02:16.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:16.330+0000 7fa019800640 1 -- 172.21.15.5:0/290149140 shutdown_connections 2024-09-10T10:02:16.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:16.330+0000 7fa019800640 1 --2- 172.21.15.5:0/290149140 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa01410e280 0x7fa014110670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:16.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:16.330+0000 7fa019800640 1 --2- 172.21.15.5:0/290149140 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa014074e30 0x7fa01410dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:16.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:16.330+0000 7fa019800640 1 --2- 172.21.15.5:0/290149140 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa014074510 0x7fa0140748f0 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:16.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:16.330+0000 7fa019800640 1 -- 172.21.15.5:0/290149140 >> 172.21.15.5:0/290149140 conn(0x7fa0140fd740 msgr2=0x7fa0140ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:16.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:16.330+0000 7fa019800640 1 -- 172.21.15.5:0/290149140 shutdown_connections 2024-09-10T10:02:16.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:16.330+0000 7fa019800640 1 -- 172.21.15.5:0/290149140 wait complete. 2024-09-10T10:02:16.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:16.331+0000 7fa019800640 1 Processor -- start 2024-09-10T10:02:16.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:16.332+0000 7fa019800640 1 -- start start 2024-09-10T10:02:16.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:16.332+0000 7fa019800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa014074510 0x7fa01419a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:16.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:16.332+0000 7fa019800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa014074e30 0x7fa01419a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:16.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:16.332+0000 7fa019800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa01410e280 0x7fa01419ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:16.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:16.332+0000 7fa019800640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa014113f70 con 0x7fa014074e30 2024-09-10T10:02:16.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:16.332+0000 7fa019800640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fa014113df0 con 0x7fa014074510 2024-09-10T10:02:16.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:16.332+0000 7fa019800640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fa0141140f0 con 0x7fa01410e280 2024-09-10T10:02:16.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:16.332+0000 7fa013400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa014074e30 0x7fa01419a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:16.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:16.332+0000 7fa013e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa014074510 0x7fa01419a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:16.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:16.332+0000 7fa013400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa014074e30 0x7fa01419a9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36938/0 (socket says 172.21.15.5:36938) 2024-09-10T10:02:16.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:16.332+0000 7fa013400640 1 -- 172.21.15.5:0/3562045046 learned_addr learned my addr 172.21.15.5:0/3562045046 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T10:02:16.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:16.333+0000 7fa018e00640 1 --2- 172.21.15.5:0/3562045046 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa01410e280 0x7fa01419ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:16.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:16.333+0000 7fa013400640 1 -- 172.21.15.5:0/3562045046 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa01410e280 msgr2=0x7fa01419ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:16.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:16.333+0000 7fa013400640 1 --2- 172.21.15.5:0/3562045046 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa01410e280 0x7fa01419ed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:16.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:16.333+0000 7fa013400640 1 -- 172.21.15.5:0/3562045046 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa014074510 msgr2=0x7fa01419a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:16.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:16.333+0000 7fa013400640 1 --2- 172.21.15.5:0/3562045046 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa014074510 0x7fa01419a470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:16.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:16.333+0000 7fa013400640 1 -- 172.21.15.5:0/3562045046 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa01419f420 con 0x7fa014074e30 2024-09-10T10:02:16.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:16.333+0000 7fa013e00640 1 --2- 172.21.15.5:0/3562045046 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa014074510 0x7fa01419a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:02:16.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:16.333+0000 7fa013400640 1 --2- 172.21.15.5:0/3562045046 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa014074e30 0x7fa01419a9b0 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7fa00000e9c0 tx=0x7fa00000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:16.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:16.333+0000 7fa010c00640 1 -- 172.21.15.5:0/3562045046 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa00000cde0 con 0x7fa014074e30 2024-09-10T10:02:16.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:16.334+0000 7fa010c00640 1 -- 172.21.15.5:0/3562045046 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa0000035f0 con 0x7fa014074e30 2024-09-10T10:02:16.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:16.334+0000 7fa010c00640 1 -- 172.21.15.5:0/3562045046 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa000010690 con 0x7fa014074e30 2024-09-10T10:02:16.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:16.334+0000 7fa019800640 1 -- 172.21.15.5:0/3562045046 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa01419f710 con 0x7fa014074e30 2024-09-10T10:02:16.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:16.334+0000 7fa019800640 1 -- 172.21.15.5:0/3562045046 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0141a6ff0 con 0x7fa014074e30 2024-09-10T10:02:16.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:16.335+0000 7fa019800640 1 -- 172.21.15.5:0/3562045046 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa0140748f0 con 0x7fa014074e30 2024-09-10T10:02:16.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:16.339+0000 7fa010c00640 1 -- 172.21.15.5:0/3562045046 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7fa0000037a0 con 0x7fa014074e30 2024-09-10T10:02:16.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:16.340+0000 7fa010c00640 1 --2- 172.21.15.5:0/3562045046 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f9ff80a2b80 0x7f9ff80a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:16.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:16.340+0000 7fa010c00640 1 -- 172.21.15.5:0/3562045046 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fa0000cf870 con 0x7fa014074e30 2024-09-10T10:02:16.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:16.340+0000 7fa013e00640 1 --2- 172.21.15.5:0/3562045046 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f9ff80a2b80 0x7f9ff80a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:16.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:16.341+0000 7fa013e00640 1 --2- 172.21.15.5:0/3562045046 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f9ff80a2b80 0x7f9ff80a5040 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fa004031000 tx=0x7fa004009530 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:16.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:16.344+0000 7fa010c00640 1 -- 172.21.15.5:0/3562045046 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa00009c380 con 0x7fa014074e30 2024-09-10T10:02:16.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:16.570+0000 7fa019800640 1 -- 172.21.15.5:0/3562045046 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa01402d7c0 con 0x7fa014074e30 2024-09-10T10:02:16.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:16.570+0000 7fa010c00640 1 -- 172.21.15.5:0/3562045046 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 42 v42) ==== 94+0+11969 (secure 0 0 0) 0x7fa00009bc80 con 0x7fa014074e30 2024-09-10T10:02:16.572 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-10T10:02:16.572 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":42,"btime":"2024-09-10T09:55:06:670916+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14454,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/2970594311","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2970594311},{"type":"v1","addr":"172.21.15.5:6835","nonce":2970594311}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14514,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6839/3616502174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":3616502174},{"type":"v1","addr":"172.21.15.5:6839","nonce":3616502174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":42,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T09:35:29.626287+0000","modified":"2024-09-10T09:55:06.419829+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24542,"mds_1":24535,"mds_2":24511,"mds_3":24506,"mds_4":24443},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14520":{"gid":14520,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/2073599842","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":2073599842},{"type":"v1","addr":"172.21.15.5:6841","nonce":2073599842}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"k","rank":4,"incarnation":26,"state":"up:active","state_seq":24,"addr":"172.21.15.102:6833/2457007427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":2457007427},{"type":"v1","addr":"172.21.15.102:6833","nonce":2457007427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/356445159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":356445159},{"type":"v1","addr":"172.21.15.159:6833","nonce":356445159}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6837/2894314507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":2894314507},{"type":"v1","addr":"172.21.15.5:6837","nonce":2894314507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.102:6835/1647549111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":1647549111},{"type":"v1","addr":"172.21.15.102:6835","nonce":1647549111}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6835/952657242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":952657242},{"type":"v1","addr":"172.21.15.159:6835","nonce":952657242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":19,"addr":"172.21.15.102:6837/2689776764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":2689776764},{"type":"v1","addr":"172.21.15.102:6837","nonce":2689776764}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.159:6837/620627639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":620627639},{"type":"v1","addr":"172.21.15.159:6837","nonce":620627639}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24535":{"gid":24535,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":18,"addr":"172.21.15.159:6839/2073739599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":2073739599},{"type":"v1","addr":"172.21.15.159:6839","nonce":2073739599}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24542":{"gid":24542,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.102:6839/335409749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6838","nonce":335409749},{"type":"v1","addr":"172.21.15.102:6839","nonce":335409749}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24542,"qdb_cluster":[24542,24535,24511,24506,24443]},"id":1}]} 2024-09-10T10:02:16.572 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 42 2024-09-10T10:02:16.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:16.573+0000 7fa019800640 1 -- 172.21.15.5:0/3562045046 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f9ff80a2b80 msgr2=0x7f9ff80a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:16.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:16.573+0000 7fa019800640 1 --2- 172.21.15.5:0/3562045046 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f9ff80a2b80 0x7f9ff80a5040 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fa004031000 tx=0x7fa004009530 comp rx=0 tx=0).stop 2024-09-10T10:02:16.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:16.573+0000 7fa019800640 1 -- 172.21.15.5:0/3562045046 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa014074e30 msgr2=0x7fa01419a9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:16.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:16.573+0000 7fa019800640 1 --2- 172.21.15.5:0/3562045046 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa014074e30 0x7fa01419a9b0 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7fa00000e9c0 tx=0x7fa00000ee80 comp rx=0 tx=0).stop 2024-09-10T10:02:16.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:16.573+0000 7fa019800640 1 -- 172.21.15.5:0/3562045046 shutdown_connections 2024-09-10T10:02:16.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:16.573+0000 7fa019800640 1 --2- 172.21.15.5:0/3562045046 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f9ff80a2b80 0x7f9ff80a5040 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:16.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:16.573+0000 7fa019800640 1 --2- 172.21.15.5:0/3562045046 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa01410e280 0x7fa01419ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:16.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:16.573+0000 7fa019800640 1 --2- 172.21.15.5:0/3562045046 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa014074e30 0x7fa01419a9b0 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:16.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:16.573+0000 7fa019800640 1 --2- 172.21.15.5:0/3562045046 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa014074510 0x7fa01419a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:16.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:16.573+0000 7fa019800640 1 -- 172.21.15.5:0/3562045046 >> 172.21.15.5:0/3562045046 conn(0x7fa0140fd740 msgr2=0x7fa0140fee90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:16.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:16.573+0000 7fa019800640 1 -- 172.21.15.5:0/3562045046 shutdown_connections 2024-09-10T10:02:16.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:16.573+0000 7fa019800640 1 -- 172.21.15.5:0/3562045046 wait complete. 2024-09-10T10:02:16.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:02:16 smithi159 ceph-mon[35513]: from='client.? 172.21.15.5:0/4001730043' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:02:16.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:02:16 smithi102 ceph-mon[35442]: from='client.? 172.21.15.5:0/4001730043' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:02:16.929 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-10T10:02:17.068 INFO:tasks.workunit.client.0.smithi005.stdout: 10 156092 13.28 MB/sec execute 475 sec latency 13035.812 ms 2024-09-10T10:02:17.417 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:02:17 smithi005 ceph-mon[30350]: pgmap v1111: 129 pgs: 129 active+clean; 760 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 364 KiB/s rd, 4.8 MiB/s wr, 206 op/s 2024-09-10T10:02:17.417 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:02:17 smithi005 ceph-mon[30350]: from='client.? 172.21.15.5:0/3562045046' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:02:17.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:02:17 smithi159 ceph-mon[35513]: pgmap v1111: 129 pgs: 129 active+clean; 760 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 364 KiB/s rd, 4.8 MiB/s wr, 206 op/s 2024-09-10T10:02:17.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:02:17 smithi159 ceph-mon[35513]: from='client.? 172.21.15.5:0/3562045046' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:02:17.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:02:17 smithi102 ceph-mon[35442]: pgmap v1111: 129 pgs: 129 active+clean; 760 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 364 KiB/s rd, 4.8 MiB/s wr, 206 op/s 2024-09-10T10:02:17.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:02:17 smithi102 ceph-mon[35442]: from='client.? 172.21.15.5:0/3562045046' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:02:17.832 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24542 quiesce db / --timeout 90 --expiration 135.3 --await-for 45.0' 2024-09-10T10:02:17.833 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24542 quiesce db / --timeout 90 --expiration 135.3 --await-for 45.0 2024-09-10T10:02:17.952 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:02:18.068 INFO:tasks.workunit.client.0.smithi005.stdout: 10 156092 13.25 MB/sec execute 476 sec latency 14036.140 ms 2024-09-10T10:02:19.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:18.895+0000 7f2595c00640 1 -- 172.21.15.5:0/1191250503 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2590102b90 msgr2=0x7f2590102f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:19.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:18.895+0000 7f2595c00640 1 --2- 172.21.15.5:0/1191250503 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2590102b90 0x7f2590102f70 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f2580009900 tx=0x7f258002a480 comp rx=0 tx=0).stop 2024-09-10T10:02:19.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:18.896+0000 7f2595c00640 1 -- 172.21.15.5:0/1191250503 shutdown_connections 2024-09-10T10:02:19.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:18.896+0000 7f2595c00640 1 --2- 172.21.15.5:0/1191250503 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f259010bfc0 0x7f259010e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:19.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:18.896+0000 7f2595c00640 1 --2- 172.21.15.5:0/1191250503 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f25901034b0 0x7f259010b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:19.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:18.896+0000 7f2595c00640 1 --2- 172.21.15.5:0/1191250503 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2590102b90 0x7f2590102f70 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:19.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:18.896+0000 7f2595c00640 1 -- 172.21.15.5:0/1191250503 >> 172.21.15.5:0/1191250503 conn(0x7f25900fc610 msgr2=0x7f25900fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:19.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:18.896+0000 7f2595c00640 1 -- 172.21.15.5:0/1191250503 shutdown_connections 2024-09-10T10:02:19.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:18.896+0000 7f2595c00640 1 -- 172.21.15.5:0/1191250503 wait complete. 2024-09-10T10:02:19.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:18.897+0000 7f2595c00640 1 Processor -- start 2024-09-10T10:02:19.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:18.897+0000 7f2595c00640 1 -- start start 2024-09-10T10:02:19.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:18.897+0000 7f2595c00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2590102b90 0x7f25901007f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:19.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:18.897+0000 7f2595c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f25901034b0 0x7f2590100d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:19.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:18.897+0000 7f2595c00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f259010bfc0 0x7f2590101270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:19.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:18.897+0000 7f2595c00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f2590111e10 con 0x7f2590102b90 2024-09-10T10:02:19.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:18.897+0000 7f2595c00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f2590111c90 con 0x7f25901034b0 2024-09-10T10:02:19.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:18.897+0000 7f2595c00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f2590111f90 con 0x7f259010bfc0 2024-09-10T10:02:19.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:18.898+0000 7f258fe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2590102b90 0x7f25901007f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:19.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:18.898+0000 7f258fe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2590102b90 0x7f25901007f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36948/0 (socket says 172.21.15.5:36948) 2024-09-10T10:02:19.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:18.898+0000 7f258fe00640 1 -- 172.21.15.5:0/959239847 learned_addr learned my addr 172.21.15.5:0/959239847 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T10:02:19.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:18.898+0000 7f2587e00640 1 --2- 172.21.15.5:0/959239847 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f25901034b0 0x7f2590100d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:19.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:18.898+0000 7f2595200640 1 --2- 172.21.15.5:0/959239847 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f259010bfc0 0x7f2590101270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:19.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:18.898+0000 7f258fe00640 1 -- 172.21.15.5:0/959239847 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f259010bfc0 msgr2=0x7f2590101270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:19.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:18.898+0000 7f258fe00640 1 --2- 172.21.15.5:0/959239847 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f259010bfc0 0x7f2590101270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:19.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:18.898+0000 7f258fe00640 1 -- 172.21.15.5:0/959239847 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f25901034b0 msgr2=0x7f2590100d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:19.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:18.898+0000 7f258fe00640 1 --2- 172.21.15.5:0/959239847 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f25901034b0 0x7f2590100d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:19.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:18.898+0000 7f258fe00640 1 -- 172.21.15.5:0/959239847 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25901a6760 con 0x7f2590102b90 2024-09-10T10:02:19.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:18.898+0000 7f258fe00640 1 --2- 172.21.15.5:0/959239847 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2590102b90 0x7f25901007f0 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f2580004770 tx=0x7f2580003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:19.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:18.899+0000 7f258d600640 1 -- 172.21.15.5:0/959239847 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f258003f070 con 0x7f2590102b90 2024-09-10T10:02:19.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:18.899+0000 7f258d600640 1 -- 172.21.15.5:0/959239847 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2580031ca0 con 0x7f2590102b90 2024-09-10T10:02:19.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:18.899+0000 7f258d600640 1 -- 172.21.15.5:0/959239847 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2580035030 con 0x7f2590102b90 2024-09-10T10:02:19.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:18.899+0000 7f2595c00640 1 -- 172.21.15.5:0/959239847 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25901a69f0 con 0x7f2590102b90 2024-09-10T10:02:19.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:18.899+0000 7f2595c00640 1 -- 172.21.15.5:0/959239847 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25901a6e80 con 0x7f2590102b90 2024-09-10T10:02:19.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:18.902+0000 7f258d600640 1 -- 172.21.15.5:0/959239847 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f2580042070 con 0x7f2590102b90 2024-09-10T10:02:19.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:18.902+0000 7f2599df2740 1 Processor -- start 2024-09-10T10:02:19.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:18.902+0000 7f2599df2740 1 -- start start 2024-09-10T10:02:19.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:18.902+0000 7f2599df2740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5625f4b06e40 0x5625f4b07220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:19.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:18.902+0000 7f2599df2740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5625f4b07760 0x5625f4b11ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:19.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:18.902+0000 7f2599df2740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5625f4b12020 0x5625f4b14410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:19.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:18.902+0000 7f2599df2740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5625f4a6d3e0 con 0x5625f4b06e40 2024-09-10T10:02:19.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:18.903+0000 7f2595200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5625f4b06e40 0x5625f4b07220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:19.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:18.903+0000 7f2599df2740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x5625f4a14d80 con 0x5625f4b07760 2024-09-10T10:02:19.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:18.903+0000 7f2599df2740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5625f4a693a0 con 0x5625f4b12020 2024-09-10T10:02:19.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:18.903+0000 7f2595200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5625f4b06e40 0x5625f4b07220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36962/0 (socket says 172.21.15.5:36962) 2024-09-10T10:02:19.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:18.903+0000 7f2595200640 1 -- 172.21.15.5:0/2153844375 learned_addr learned my addr 172.21.15.5:0/2153844375 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T10:02:19.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:18.903+0000 7f258fe00640 1 --2- 172.21.15.5:0/2153844375 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5625f4b07760 0x5625f4b11ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:19.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:18.903+0000 7f2587e00640 1 --2- 172.21.15.5:0/2153844375 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5625f4b12020 0x5625f4b14410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:19.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:18.903+0000 7f2595200640 1 -- 172.21.15.5:0/2153844375 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5625f4b12020 msgr2=0x5625f4b14410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:19.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:18.903+0000 7f2595200640 1 --2- 172.21.15.5:0/2153844375 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5625f4b12020 0x5625f4b14410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:19.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:18.903+0000 7f2595200640 1 -- 172.21.15.5:0/2153844375 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5625f4b07760 msgr2=0x5625f4b11ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:19.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:18.903+0000 7f2595200640 1 --2- 172.21.15.5:0/2153844375 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5625f4b07760 0x5625f4b11ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:19.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:18.903+0000 7f2595200640 1 -- 172.21.15.5:0/2153844375 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5625f4a33ae0 con 0x5625f4b06e40 2024-09-10T10:02:19.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:18.903+0000 7f258fe00640 1 --2- 172.21.15.5:0/2153844375 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5625f4b07760 0x5625f4b11ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:02:19.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:18.904+0000 7f258d600640 1 --2- 172.21.15.5:0/959239847 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f25740a2c00 0x7f25740a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:19.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:18.904+0000 7f2595200640 1 --2- 172.21.15.5:0/2153844375 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5625f4b06e40 0x5625f4b07220 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f2588009f50 tx=0x7f25880118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8896a08f8fb62fa1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:19.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:18.904+0000 7f258d600640 1 -- 172.21.15.5:0/959239847 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f25800ef6f0 con 0x7f2590102b90 2024-09-10T10:02:19.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:18.904+0000 7f258fe00640 1 --2- 172.21.15.5:0/959239847 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f25740a2c00 0x7f25740a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:19.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:18.904+0000 7f2585600640 1 -- 172.21.15.5:0/2153844375 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2588004d20 con 0x5625f4b06e40 2024-09-10T10:02:19.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:18.904+0000 7f2585600640 1 -- 172.21.15.5:0/2153844375 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2588004050 con 0x5625f4b06e40 2024-09-10T10:02:19.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:18.904+0000 7f2585600640 1 -- 172.21.15.5:0/2153844375 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2588005280 con 0x5625f4b06e40 2024-09-10T10:02:19.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:18.905+0000 7f2599df2740 1 -- 172.21.15.5:0/2153844375 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5625f4b06e40 msgr2=0x5625f4b07220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:19.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:18.905+0000 7f258fe00640 1 --2- 172.21.15.5:0/959239847 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f25740a2c00 0x7f25740a50c0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f25800bbb40 tx=0x7f2580009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:19.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:18.905+0000 7f2599df2740 1 --2- 172.21.15.5:0/2153844375 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5625f4b06e40 0x5625f4b07220 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f2588009f50 tx=0x7f25880118d0 comp rx=0 tx=0).stop 2024-09-10T10:02:19.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:18.905+0000 7f2599df2740 1 -- 172.21.15.5:0/2153844375 shutdown_connections 2024-09-10T10:02:19.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:18.905+0000 7f2599df2740 1 --2- 172.21.15.5:0/2153844375 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5625f4b12020 0x5625f4b14410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:19.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:18.905+0000 7f2599df2740 1 --2- 172.21.15.5:0/2153844375 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5625f4b07760 0x5625f4b11ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:19.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:18.905+0000 7f2599df2740 1 --2- 172.21.15.5:0/2153844375 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5625f4b06e40 0x5625f4b07220 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:19.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:18.905+0000 7f2599df2740 1 -- 172.21.15.5:0/2153844375 >> 172.21.15.5:0/2153844375 conn(0x5625f4b033d0 msgr2=0x5625f4b037b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:19.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:18.905+0000 7f2599df2740 1 -- 172.21.15.5:0/2153844375 shutdown_connections 2024-09-10T10:02:19.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:18.905+0000 7f2599df2740 1 -- 172.21.15.5:0/2153844375 wait complete. 2024-09-10T10:02:19.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:18.906+0000 7f2599df2740 1 -- start start 2024-09-10T10:02:19.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:18.906+0000 7f2599df2740 1 Processor -- start 2024-09-10T10:02:19.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:18.907+0000 7f2599df2740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5625f4b06e40 0x5625f4b0eec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:19.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:18.907+0000 7f2599df2740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5625f4b07760 0x5625f4b0f400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:19.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:18.907+0000 7f2599df2740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5625f4b12020 0x5625f4b0f940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:19.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:18.907+0000 7f2599df2740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5625f4b14bf0 con 0x5625f4b07760 2024-09-10T10:02:19.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:18.907+0000 7f2599df2740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x5625f4a280f0 con 0x5625f4b06e40 2024-09-10T10:02:19.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:18.907+0000 7f2599df2740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5625f4b14d70 con 0x5625f4b12020 2024-09-10T10:02:19.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:18.907+0000 7f2599df2740 10 client.0 _ll_register_callbacks cb 0x5625f4a0fbd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:02:19.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:18.907+0000 7f2595200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5625f4b06e40 0x5625f4b0eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:19.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:18.907+0000 7f2587e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5625f4b07760 0x5625f4b0f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:19.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:18.907+0000 7f2595200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5625f4b06e40 0x5625f4b0eec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:39726/0 (socket says 172.21.15.5:39726) 2024-09-10T10:02:19.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:18.907+0000 7f2595200640 1 -- 172.21.15.5:0/3421211760 learned_addr learned my addr 172.21.15.5:0/3421211760 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T10:02:19.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:18.907+0000 7f2595200640 1 --2- 172.21.15.5:0/3421211760 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5625f4b12020 0x5625f4b0f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:19.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:18.907+0000 7f2587e00640 1 -- 172.21.15.5:0/3421211760 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5625f4b12020 msgr2=0x5625f4b0f940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:19.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:18.907+0000 7f2587e00640 1 --2- 172.21.15.5:0/3421211760 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5625f4b12020 0x5625f4b0f940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:19.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:18.907+0000 7f2587e00640 1 -- 172.21.15.5:0/3421211760 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5625f4b06e40 msgr2=0x5625f4b0eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:19.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:18.907+0000 7f2587e00640 1 --2- 172.21.15.5:0/3421211760 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5625f4b06e40 0x5625f4b0eec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:19.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:18.907+0000 7f2587e00640 1 -- 172.21.15.5:0/3421211760 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5625f47de3c0 con 0x5625f4b07760 2024-09-10T10:02:19.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:18.908+0000 7f2587e00640 1 --2- 172.21.15.5:0/3421211760 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5625f4b07760 0x5625f4b0f400 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f2578005d00 tx=0x7f2578011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee4dd53eb6a4736 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:19.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:18.908+0000 7f257e000640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-10T10:02:19.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:18.908+0000 7f257e000640 1 -- 172.21.15.5:0/3421211760 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2578004020 con 0x5625f4b07760 2024-09-10T10:02:19.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:18.908+0000 7f257e000640 1 -- 172.21.15.5:0/3421211760 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f25780041c0 con 0x5625f4b07760 2024-09-10T10:02:19.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:18.908+0000 7f2599df2740 1 -- 172.21.15.5:0/3421211760 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2588004050 con 0x5625f4b07760 2024-09-10T10:02:19.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:18.908+0000 7f257e000640 1 -- 172.21.15.5:0/3421211760 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f2578024050 con 0x5625f4b07760 2024-09-10T10:02:19.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:18.908+0000 7f2599df2740 10 client.19806 fetch_fsmap learned FSMap version 42 2024-09-10T10:02:19.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:18.908+0000 7f2599df2740 1 -- 172.21.15.5:0/3421211760 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x5625f4b20c10 con 0x5625f4b07760 2024-09-10T10:02:19.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:18.909+0000 7f257e000640 1 -- 172.21.15.5:0/3421211760 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2588004050 con 0x5625f4b07760 2024-09-10T10:02:19.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:18.909+0000 7f257e000640 1 -- 172.21.15.5:0/3421211760 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 42) ==== 8370+0+0 (secure 0 0 0) 0x7f2578003950 con 0x5625f4b07760 2024-09-10T10:02:19.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:18.909+0000 7f2599df2740 10 client.19806 fetch_fsmap finished waiting for FSMap version 42 2024-09-10T10:02:19.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:18.909+0000 7f2599df2740 10 client.19806 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-09-10T10:02:19.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:18.909+0000 7f2599df2740 20 client.19806 populate_metadata read hostname 'smithi005' 2024-09-10T10:02:19.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:18.909+0000 7f2599df2740 1 --2- 172.21.15.5:0/3421211760 >> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] conn(0x5625f4b220d0 0x5625f4b24530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:19.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:18.909+0000 7f2599df2740 4 client.19806 mds_command: new command op to 24443 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:02:19.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:18.909+0000 7f2599df2740 1 -- 172.21.15.5:0/3421211760 --> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5625f4a7a950 con 0x5625f4b220d0 2024-09-10T10:02:19.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:18.910+0000 7f2595200640 1 --2- 172.21.15.5:0/3421211760 >> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] conn(0x5625f4b220d0 0x5625f4b24530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:19.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:18.910+0000 7f2595200640 1 --2- 172.21.15.5:0/3421211760 >> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] conn(0x5625f4b220d0 0x5625f4b24530 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=acca6574046e6df6 server_cookie=9e48465cbce85bc9 in_seq=0 out_seq=0 2024-09-10T10:02:19.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:18.910+0000 7f257e000640 10 client.19806 ms_handle_connect on v2:172.21.15.102:6832/2457007427 2024-09-10T10:02:19.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:18.911+0000 7f257e000640 1 -- 172.21.15.5:0/3421211760 <== mds.4 v2:172.21.15.102:6832/2457007427 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5625f4a7a950 con 0x5625f4b220d0 2024-09-10T10:02:19.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:18.911+0000 7f257e000640 10 client.19806 handle_command_reply: tid=0 2024-09-10T10:02:19.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:18.911+0000 7f2599df2740 1 client.19806 shutdown 2024-09-10T10:02:19.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:18.911+0000 7f2599df2740 1 -- 172.21.15.5:0/3421211760 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5625f4b07760 msgr2=0x5625f4b0f400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:19.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:18.911+0000 7f2599df2740 1 --2- 172.21.15.5:0/3421211760 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5625f4b07760 0x5625f4b0f400 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f2578005d00 tx=0x7f2578011810 comp rx=0 tx=0).stop 2024-09-10T10:02:19.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:18.911+0000 7f2599df2740 1 -- 172.21.15.5:0/3421211760 shutdown_connections 2024-09-10T10:02:19.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:18.911+0000 7f2599df2740 1 --2- 172.21.15.5:0/3421211760 >> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] conn(0x5625f4b220d0 0x5625f4b24530 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:19.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:18.911+0000 7f2599df2740 1 --2- 172.21.15.5:0/3421211760 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5625f4b12020 0x5625f4b0f940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:19.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:18.911+0000 7f2599df2740 1 --2- 172.21.15.5:0/3421211760 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5625f4b07760 0x5625f4b0f400 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:19.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:18.911+0000 7f2599df2740 1 --2- 172.21.15.5:0/3421211760 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5625f4b06e40 0x5625f4b0eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:19.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:18.911+0000 7f2599df2740 1 -- 172.21.15.5:0/3421211760 >> 172.21.15.5:0/3421211760 conn(0x5625f4b033d0 msgr2=0x5625f4b09100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:19.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:18.911+0000 7f257e000640 0 client.19806 ms_handle_reset on v2:172.21.15.102:6832/2457007427 2024-09-10T10:02:19.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:18.912+0000 7f2599df2740 1 -- 172.21.15.5:0/3421211760 shutdown_connections 2024-09-10T10:02:19.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:18.912+0000 7f2599df2740 1 -- 172.21.15.5:0/3421211760 wait complete. 2024-09-10T10:02:19.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:18.912+0000 7f2599df2740 20 client.19806 trim_cache size 0 max 16384 2024-09-10T10:02:19.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:18.937+0000 7f2599df2740 1 Processor -- start 2024-09-10T10:02:19.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:18.937+0000 7f2599df2740 1 -- start start 2024-09-10T10:02:19.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:18.938+0000 7f2599df2740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5625f4b06e40 0x5625f4b098a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:19.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:18.938+0000 7f2599df2740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5625f4b07760 0x5625f4b023a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:19.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:18.938+0000 7f2599df2740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5625f4b12020 0x5625f4b29e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:19.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:18.938+0000 7f2599df2740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f25901b0ee0 con 0x5625f4b06e40 2024-09-10T10:02:19.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:18.938+0000 7f2599df2740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f25901b2a20 con 0x5625f4b12020 2024-09-10T10:02:19.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:18.938+0000 7f2599df2740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f25901b15b0 con 0x5625f4b07760 2024-09-10T10:02:19.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:18.938+0000 7f2595200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5625f4b06e40 0x5625f4b098a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:19.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:18.938+0000 7f2595200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5625f4b06e40 0x5625f4b098a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36972/0 (socket says 172.21.15.5:36972) 2024-09-10T10:02:19.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:18.938+0000 7f2595200640 1 -- 172.21.15.5:0/311326778 learned_addr learned my addr 172.21.15.5:0/311326778 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T10:02:19.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:18.938+0000 7f2587e00640 1 --2- 172.21.15.5:0/311326778 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5625f4b07760 0x5625f4b023a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:19.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:18.938+0000 7f2595200640 1 --2- 172.21.15.5:0/311326778 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5625f4b12020 0x5625f4b29e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:19.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:18.939+0000 7f2595200640 1 -- 172.21.15.5:0/311326778 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5625f4b07760 msgr2=0x5625f4b023a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:19.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:18.939+0000 7f2595200640 1 --2- 172.21.15.5:0/311326778 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5625f4b07760 0x5625f4b023a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:19.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:18.939+0000 7f2595200640 1 -- 172.21.15.5:0/311326778 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5625f4b12020 msgr2=0x5625f4b29e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:19.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:18.939+0000 7f2595200640 1 --2- 172.21.15.5:0/311326778 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5625f4b12020 0x5625f4b29e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:19.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:18.939+0000 7f2595200640 1 -- 172.21.15.5:0/311326778 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5625f4b03200 con 0x5625f4b06e40 2024-09-10T10:02:19.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:18.939+0000 7f2587e00640 1 --2- 172.21.15.5:0/311326778 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5625f4b07760 0x5625f4b023a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:02:19.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:18.939+0000 7f2595200640 1 --2- 172.21.15.5:0/311326778 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5625f4b06e40 0x5625f4b098a0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f2588000f80 tx=0x7f2588009d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f2cfcc2485a626b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:19.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:18.939+0000 7f2584c00640 1 -- 172.21.15.5:0/311326778 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2588004520 con 0x5625f4b06e40 2024-09-10T10:02:19.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:18.939+0000 7f2584c00640 1 -- 172.21.15.5:0/311326778 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f258802b070 con 0x5625f4b06e40 2024-09-10T10:02:19.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:18.939+0000 7f2584c00640 1 -- 172.21.15.5:0/311326778 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2588027d40 con 0x5625f4b06e40 2024-09-10T10:02:19.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:18.940+0000 7f2599df2740 1 -- 172.21.15.5:0/311326778 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5625f4b06e40 msgr2=0x5625f4b098a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:19.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:18.940+0000 7f2599df2740 1 --2- 172.21.15.5:0/311326778 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5625f4b06e40 0x5625f4b098a0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f2588000f80 tx=0x7f2588009d00 comp rx=0 tx=0).stop 2024-09-10T10:02:19.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:18.940+0000 7f2599df2740 1 -- 172.21.15.5:0/311326778 shutdown_connections 2024-09-10T10:02:19.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:18.940+0000 7f2599df2740 1 --2- 172.21.15.5:0/311326778 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5625f4b12020 0x5625f4b29e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:19.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:18.940+0000 7f2599df2740 1 --2- 172.21.15.5:0/311326778 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5625f4b07760 0x5625f4b023a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:19.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:18.940+0000 7f2599df2740 1 --2- 172.21.15.5:0/311326778 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5625f4b06e40 0x5625f4b098a0 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:19.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:18.940+0000 7f2599df2740 1 -- 172.21.15.5:0/311326778 >> 172.21.15.5:0/311326778 conn(0x5625f4b033d0 msgr2=0x5625f4b01c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:19.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:18.940+0000 7f2599df2740 1 -- 172.21.15.5:0/311326778 shutdown_connections 2024-09-10T10:02:19.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:18.940+0000 7f2599df2740 1 -- 172.21.15.5:0/311326778 wait complete. 2024-09-10T10:02:19.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:18.941+0000 7f2599df2740 1 -- start start 2024-09-10T10:02:19.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:18.941+0000 7f2599df2740 1 Processor -- start 2024-09-10T10:02:19.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:18.941+0000 7f2599df2740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5625f4b06e40 0x5625f4b1a370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:19.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:18.941+0000 7f2599df2740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5625f4b07760 0x5625f4b1a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:19.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:18.941+0000 7f2599df2740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5625f4b12020 0x5625f4b15940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:19.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:18.941+0000 7f2599df2740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f25901b2350 con 0x5625f4b07760 2024-09-10T10:02:19.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:18.941+0000 7f2599df2740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f25901b1c80 con 0x5625f4b06e40 2024-09-10T10:02:19.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:18.941+0000 7f2599df2740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f25901aeec0 con 0x5625f4b12020 2024-09-10T10:02:19.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:18.941+0000 7f2599df2740 10 client.0 _ll_register_callbacks cb 0x7f257800bb00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:02:19.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:18.942+0000 7f2587e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5625f4b07760 0x5625f4b1a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:19.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:18.942+0000 7f2587e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5625f4b07760 0x5625f4b1a8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36984/0 (socket says 172.21.15.5:36984) 2024-09-10T10:02:19.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:18.942+0000 7f2587e00640 1 -- 172.21.15.5:0/3548902635 learned_addr learned my addr 172.21.15.5:0/3548902635 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T10:02:19.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:18.942+0000 7f2595200640 1 --2- 172.21.15.5:0/3548902635 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5625f4b06e40 0x5625f4b1a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:19.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:18.942+0000 7f2595200640 1 --2- 172.21.15.5:0/3548902635 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5625f4b12020 0x5625f4b15940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:19.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:18.942+0000 7f2587e00640 1 -- 172.21.15.5:0/3548902635 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5625f4b12020 msgr2=0x5625f4b15940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:19.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:18.942+0000 7f2587e00640 1 --2- 172.21.15.5:0/3548902635 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5625f4b12020 0x5625f4b15940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:19.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:18.942+0000 7f2587e00640 1 -- 172.21.15.5:0/3548902635 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5625f4b06e40 msgr2=0x5625f4b1a370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:19.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:18.942+0000 7f2587e00640 1 --2- 172.21.15.5:0/3548902635 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5625f4b06e40 0x5625f4b1a370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:19.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:18.942+0000 7f2587e00640 1 -- 172.21.15.5:0/3548902635 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5625f4b16170 con 0x5625f4b07760 2024-09-10T10:02:19.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:18.942+0000 7f2595200640 1 --2- 172.21.15.5:0/3548902635 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5625f4b12020 0x5625f4b15940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:02:19.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:18.943+0000 7f2587e00640 1 --2- 172.21.15.5:0/3548902635 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5625f4b07760 0x5625f4b1a8b0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x5625f4b10500 tx=0x7f2578005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2c6d3f66bf433eb server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:19.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:18.943+0000 7f257e000640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-10T10:02:19.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:18.943+0000 7f257e000640 1 -- 172.21.15.5:0/3548902635 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25780041f0 con 0x5625f4b07760 2024-09-10T10:02:19.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:18.943+0000 7f257e000640 1 -- 172.21.15.5:0/3548902635 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f2578004390 con 0x5625f4b07760 2024-09-10T10:02:19.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:18.943+0000 7f2599df2740 1 -- 172.21.15.5:0/3548902635 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f258802b070 con 0x5625f4b07760 2024-09-10T10:02:19.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:18.943+0000 7f257e000640 1 -- 172.21.15.5:0/3548902635 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f2578039050 con 0x5625f4b07760 2024-09-10T10:02:19.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:18.943+0000 7f2599df2740 10 client.19812 fetch_fsmap learned FSMap version 42 2024-09-10T10:02:19.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:18.943+0000 7f2599df2740 1 -- 172.21.15.5:0/3548902635 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x5625f4b8ca60 con 0x5625f4b07760 2024-09-10T10:02:19.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:18.944+0000 7f257e000640 1 -- 172.21.15.5:0/3548902635 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f258802b070 con 0x5625f4b07760 2024-09-10T10:02:19.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:18.944+0000 7f257e000640 1 -- 172.21.15.5:0/3548902635 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 42) ==== 8370+0+0 (secure 0 0 0) 0x7f2578005450 con 0x5625f4b07760 2024-09-10T10:02:19.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:18.944+0000 7f2599df2740 10 client.19812 fetch_fsmap finished waiting for FSMap version 42 2024-09-10T10:02:19.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:18.944+0000 7f2599df2740 10 client.19812 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-09-10T10:02:19.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:18.944+0000 7f2599df2740 20 client.19812 populate_metadata read hostname 'smithi005' 2024-09-10T10:02:19.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:18.944+0000 7f2599df2740 1 --2- 172.21.15.5:0/3548902635 >> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] conn(0x5625f4b8d210 0x5625f4b8f630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:19.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:18.944+0000 7f2599df2740 4 client.19812 mds_command: new command op to 24443 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T10:02:19.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:18.944+0000 7f2599df2740 1 -- 172.21.15.5:0/3548902635 --> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5625f4a74e60 con 0x5625f4b8d210 2024-09-10T10:02:19.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:18.945+0000 7f2595200640 1 --2- 172.21.15.5:0/3548902635 >> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] conn(0x5625f4b8d210 0x5625f4b8f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:19.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:18.945+0000 7f2595200640 1 --2- 172.21.15.5:0/3548902635 >> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] conn(0x5625f4b8d210 0x5625f4b8f630 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=641482b1ecd04ab5 server_cookie=712153a441711757 in_seq=0 out_seq=0 2024-09-10T10:02:19.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:18.945+0000 7f257e000640 10 client.19812 ms_handle_connect on v2:172.21.15.102:6832/2457007427 2024-09-10T10:02:19.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:18.946+0000 7f257e000640 1 -- 172.21.15.5:0/3548902635 <== mds.4 v2:172.21.15.102:6832/2457007427 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5625f4a74e60 con 0x5625f4b8d210 2024-09-10T10:02:19.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:18.946+0000 7f257e000640 10 client.19812 handle_command_reply: tid=0 2024-09-10T10:02:19.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:18.946+0000 7f2599df2740 1 client.19812 shutdown 2024-09-10T10:02:19.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:18.946+0000 7f2599df2740 1 -- 172.21.15.5:0/3548902635 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5625f4b07760 msgr2=0x5625f4b1a8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:19.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:18.946+0000 7f2599df2740 1 --2- 172.21.15.5:0/3548902635 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5625f4b07760 0x5625f4b1a8b0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x5625f4b10500 tx=0x7f2578005850 comp rx=0 tx=0).stop 2024-09-10T10:02:19.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:18.946+0000 7f2599df2740 1 -- 172.21.15.5:0/3548902635 shutdown_connections 2024-09-10T10:02:19.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:18.946+0000 7f2599df2740 1 --2- 172.21.15.5:0/3548902635 >> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] conn(0x5625f4b8d210 0x5625f4b8f630 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:19.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:18.946+0000 7f2599df2740 1 --2- 172.21.15.5:0/3548902635 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5625f4b12020 0x5625f4b15940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:19.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:18.946+0000 7f2599df2740 1 --2- 172.21.15.5:0/3548902635 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5625f4b07760 0x5625f4b1a8b0 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:19.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:18.946+0000 7f2599df2740 1 --2- 172.21.15.5:0/3548902635 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5625f4b06e40 0x5625f4b1a370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:19.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:18.946+0000 7f257e000640 0 client.19812 ms_handle_reset on v2:172.21.15.102:6832/2457007427 2024-09-10T10:02:19.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:18.946+0000 7f2599df2740 1 -- 172.21.15.5:0/3548902635 >> 172.21.15.5:0/3548902635 conn(0x5625f4b033d0 msgr2=0x5625f4b10f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:19.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:18.946+0000 7f2599df2740 1 -- 172.21.15.5:0/3548902635 shutdown_connections 2024-09-10T10:02:19.079 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-10T10:02:19.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:18.946+0000 7f2599df2740 1 -- 172.21.15.5:0/3548902635 wait complete. 2024-09-10T10:02:19.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:18.946+0000 7f2599df2740 20 client.19812 trim_cache size 0 max 16384 2024-09-10T10:02:19.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:18.947+0000 7f2595c00640 1 -- 172.21.15.5:0/959239847 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f25740a2c00 msgr2=0x7f25740a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:19.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:18.947+0000 7f2595c00640 1 --2- 172.21.15.5:0/959239847 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f25740a2c00 0x7f25740a50c0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f25800bbb40 tx=0x7f2580009260 comp rx=0 tx=0).stop 2024-09-10T10:02:19.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:18.947+0000 7f2595c00640 1 -- 172.21.15.5:0/959239847 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2590102b90 msgr2=0x7f25901007f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:19.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:18.947+0000 7f2595c00640 1 --2- 172.21.15.5:0/959239847 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2590102b90 0x7f25901007f0 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f2580004770 tx=0x7f2580003c10 comp rx=0 tx=0).stop 2024-09-10T10:02:19.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:18.947+0000 7f2595c00640 1 -- 172.21.15.5:0/959239847 shutdown_connections 2024-09-10T10:02:19.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:18.947+0000 7f2595c00640 1 --2- 172.21.15.5:0/959239847 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f25740a2c00 0x7f25740a50c0 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:19.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:18.947+0000 7f2595c00640 1 --2- 172.21.15.5:0/959239847 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f259010bfc0 0x7f2590101270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:19.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:18.947+0000 7f2595c00640 1 --2- 172.21.15.5:0/959239847 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f25901034b0 0x7f2590100d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:19.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:18.947+0000 7f2595c00640 1 --2- 172.21.15.5:0/959239847 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2590102b90 0x7f25901007f0 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:19.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:18.947+0000 7f2595c00640 1 -- 172.21.15.5:0/959239847 >> 172.21.15.5:0/959239847 conn(0x7f25900fc610 msgr2=0x7f25900fd2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:19.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:18.947+0000 7f2595c00640 1 -- 172.21.15.5:0/959239847 shutdown_connections 2024-09-10T10:02:19.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:18.947+0000 7f2595c00640 1 -- 172.21.15.5:0/959239847 wait complete. 2024-09-10T10:02:19.081 INFO:tasks.workunit.client.0.smithi005.stdout: 10 156096 13.23 MB/sec execute 477 sec latency 15036.212 ms 2024-09-10T10:02:19.435 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:02:19 smithi005 ceph-mon[30350]: pgmap v1112: 129 pgs: 129 active+clean; 762 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 187 KiB/s rd, 4.3 MiB/s wr, 148 op/s 2024-09-10T10:02:19.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:02:19 smithi159 ceph-mon[35513]: pgmap v1112: 129 pgs: 129 active+clean; 762 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 187 KiB/s rd, 4.3 MiB/s wr, 148 op/s 2024-09-10T10:02:19.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:02:19 smithi102 ceph-mon[35442]: pgmap v1112: 129 pgs: 129 active+clean; 762 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 187 KiB/s rd, 4.3 MiB/s wr, 148 op/s 2024-09-10T10:02:20.068 INFO:tasks.workunit.client.0.smithi005.stdout: 10 156096 13.20 MB/sec execute 478 sec latency 16036.274 ms 2024-09-10T10:02:20.280 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:02:20.308 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-10T10:02:21.068 INFO:tasks.workunit.client.0.smithi005.stdout: 10 156096 13.17 MB/sec execute 479 sec latency 17036.333 ms 2024-09-10T10:02:21.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:21.240+0000 7fa835600640 1 -- 172.21.15.5:0/4143197260 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa83010e210 msgr2=0x7fa830110660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:21.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:21.240+0000 7fa835600640 1 --2- 172.21.15.5:0/4143197260 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa83010e210 0x7fa830110660 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7fa828009ca0 tx=0x7fa82802a950 comp rx=0 tx=0).stop 2024-09-10T10:02:21.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:21.240+0000 7fa835600640 1 -- 172.21.15.5:0/4143197260 shutdown_connections 2024-09-10T10:02:21.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:21.240+0000 7fa835600640 1 --2- 172.21.15.5:0/4143197260 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa83010e210 0x7fa830110660 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:21.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:21.240+0000 7fa835600640 1 --2- 172.21.15.5:0/4143197260 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa830074f40 0x7fa83010dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:21.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:21.240+0000 7fa835600640 1 --2- 172.21.15.5:0/4143197260 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa830074620 0x7fa830074a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:21.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:21.240+0000 7fa835600640 1 -- 172.21.15.5:0/4143197260 >> 172.21.15.5:0/4143197260 conn(0x7fa8300fd710 msgr2=0x7fa8300ffb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:21.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:21.240+0000 7fa835600640 1 -- 172.21.15.5:0/4143197260 shutdown_connections 2024-09-10T10:02:21.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:21.240+0000 7fa835600640 1 -- 172.21.15.5:0/4143197260 wait complete. 2024-09-10T10:02:21.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:21.241+0000 7fa835600640 1 Processor -- start 2024-09-10T10:02:21.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:21.241+0000 7fa835600640 1 -- start start 2024-09-10T10:02:21.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:21.241+0000 7fa835600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa830074620 0x7fa83010db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:21.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:21.241+0000 7fa835600640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa830074f40 0x7fa83010ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:21.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:21.241+0000 7fa835600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa83010e210 0x7fa83010b190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:21.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:21.241+0000 7fa835600640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa830114010 con 0x7fa83010e210 2024-09-10T10:02:21.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:21.241+0000 7fa835600640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fa830113e90 con 0x7fa830074620 2024-09-10T10:02:21.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:21.241+0000 7fa835600640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fa830114190 con 0x7fa830074f40 2024-09-10T10:02:21.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:21.242+0000 7fa82fe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa830074620 0x7fa83010db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:21.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:21.242+0000 7fa82fe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa830074620 0x7fa83010db80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:60276/0 (socket says 172.21.15.5:60276) 2024-09-10T10:02:21.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:21.242+0000 7fa82fe00640 1 -- 172.21.15.5:0/3301664983 learned_addr learned my addr 172.21.15.5:0/3301664983 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T10:02:21.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:21.242+0000 7fa827e00640 1 --2- 172.21.15.5:0/3301664983 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa830074f40 0x7fa83010ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:21.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:21.242+0000 7fa834c00640 1 --2- 172.21.15.5:0/3301664983 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa83010e210 0x7fa83010b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:21.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:21.242+0000 7fa82fe00640 1 -- 172.21.15.5:0/3301664983 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa830074f40 msgr2=0x7fa83010ac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:21.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:21.242+0000 7fa82fe00640 1 --2- 172.21.15.5:0/3301664983 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa830074f40 0x7fa83010ac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:21.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:21.242+0000 7fa82fe00640 1 -- 172.21.15.5:0/3301664983 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa83010e210 msgr2=0x7fa83010b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:21.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:21.242+0000 7fa82fe00640 1 --2- 172.21.15.5:0/3301664983 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa83010e210 0x7fa83010b190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:21.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:21.242+0000 7fa82fe00640 1 -- 172.21.15.5:0/3301664983 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa83010ba50 con 0x7fa830074620 2024-09-10T10:02:21.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:21.243+0000 7fa834c00640 1 --2- 172.21.15.5:0/3301664983 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa83010e210 0x7fa83010b190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:02:21.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:21.243+0000 7fa82fe00640 1 --2- 172.21.15.5:0/3301664983 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa830074620 0x7fa83010db80 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7fa82000e9f0 tx=0x7fa82000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:21.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:21.243+0000 7fa82d600640 1 -- 172.21.15.5:0/3301664983 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa82000cd80 con 0x7fa830074620 2024-09-10T10:02:21.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:21.243+0000 7fa82d600640 1 -- 172.21.15.5:0/3301664983 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa820010910 con 0x7fa830074620 2024-09-10T10:02:21.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:21.243+0000 7fa82d600640 1 -- 172.21.15.5:0/3301664983 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa820018990 con 0x7fa830074620 2024-09-10T10:02:21.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:21.243+0000 7fa835600640 1 -- 172.21.15.5:0/3301664983 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8301aae30 con 0x7fa830074620 2024-09-10T10:02:21.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:21.243+0000 7fa835600640 1 -- 172.21.15.5:0/3301664983 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8301ab2f0 con 0x7fa830074620 2024-09-10T10:02:21.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:21.247+0000 7fa82d600640 1 -- 172.21.15.5:0/3301664983 <== mon.2 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7fa820010ac0 con 0x7fa830074620 2024-09-10T10:02:21.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:21.248+0000 7fa839837740 1 Processor -- start 2024-09-10T10:02:21.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:21.248+0000 7fa839837740 1 -- start start 2024-09-10T10:02:21.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:21.248+0000 7fa839837740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5558039af560 0x5558039af940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:21.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:21.248+0000 7fa839837740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5558039afe80 0x5558039ba200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:21.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:21.248+0000 7fa839837740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5558039ba740 0x5558039bcb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:21.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:21.248+0000 7fa839837740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x555803915530 con 0x5558039af560 2024-09-10T10:02:21.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:21.248+0000 7fa839837740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x5558038bc860 con 0x5558039afe80 2024-09-10T10:02:21.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:21.248+0000 7fa839837740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5558039114f0 con 0x5558039ba740 2024-09-10T10:02:21.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:21.248+0000 7fa834c00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5558039af560 0x5558039af940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:21.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:21.249+0000 7fa834c00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5558039af560 0x5558039af940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58116/0 (socket says 172.21.15.5:58116) 2024-09-10T10:02:21.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:21.249+0000 7fa834c00640 1 -- 172.21.15.5:0/2706190099 learned_addr learned my addr 172.21.15.5:0/2706190099 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T10:02:21.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:21.249+0000 7fa82fe00640 1 --2- 172.21.15.5:0/2706190099 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5558039afe80 0x5558039ba200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:21.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:21.249+0000 7fa827e00640 1 --2- 172.21.15.5:0/2706190099 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5558039ba740 0x5558039bcb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:21.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:21.249+0000 7fa834c00640 1 -- 172.21.15.5:0/2706190099 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5558039ba740 msgr2=0x5558039bcb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:21.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:21.249+0000 7fa834c00640 1 --2- 172.21.15.5:0/2706190099 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5558039ba740 0x5558039bcb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:21.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:21.249+0000 7fa834c00640 1 -- 172.21.15.5:0/2706190099 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5558039afe80 msgr2=0x5558039ba200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:21.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:21.249+0000 7fa834c00640 1 --2- 172.21.15.5:0/2706190099 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5558039afe80 0x5558039ba200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:21.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:21.249+0000 7fa834c00640 1 -- 172.21.15.5:0/2706190099 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555803686b90 con 0x5558039af560 2024-09-10T10:02:21.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:21.249+0000 7fa82fe00640 1 --2- 172.21.15.5:0/2706190099 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5558039afe80 0x5558039ba200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:02:21.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:21.250+0000 7fa82d600640 1 --2- 172.21.15.5:0/3301664983 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fa8140a2c00 0x7fa8140a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:21.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:21.250+0000 7fa82d600640 1 -- 172.21.15.5:0/3301664983 <== mon.2 v2:172.21.15.102:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fa8200cf970 con 0x7fa830074620 2024-09-10T10:02:21.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:21.250+0000 7fa834c00640 1 --2- 172.21.15.5:0/2706190099 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5558039af560 0x5558039af940 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7fa828009ca0 tx=0x7fa828009a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72ec2691900158af server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:21.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:21.250+0000 7fa82fe00640 1 --2- 172.21.15.5:0/3301664983 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fa8140a2c00 0x7fa8140a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:21.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:21.250+0000 7fa825200640 1 -- 172.21.15.5:0/2706190099 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa828034040 con 0x5558039af560 2024-09-10T10:02:21.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:21.250+0000 7fa825200640 1 -- 172.21.15.5:0/2706190099 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa828003e50 con 0x5558039af560 2024-09-10T10:02:21.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:21.251+0000 7fa825200640 1 -- 172.21.15.5:0/2706190099 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa828004330 con 0x5558039af560 2024-09-10T10:02:21.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:21.251+0000 7fa839837740 1 -- 172.21.15.5:0/2706190099 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5558039af560 msgr2=0x5558039af940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:21.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:21.251+0000 7fa839837740 1 --2- 172.21.15.5:0/2706190099 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5558039af560 0x5558039af940 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7fa828009ca0 tx=0x7fa828009a20 comp rx=0 tx=0).stop 2024-09-10T10:02:21.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:21.252+0000 7fa839837740 1 -- 172.21.15.5:0/2706190099 shutdown_connections 2024-09-10T10:02:21.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:21.252+0000 7fa839837740 1 --2- 172.21.15.5:0/2706190099 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5558039ba740 0x5558039bcb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:21.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:21.252+0000 7fa839837740 1 --2- 172.21.15.5:0/2706190099 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5558039afe80 0x5558039ba200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:21.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:21.252+0000 7fa839837740 1 --2- 172.21.15.5:0/2706190099 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5558039af560 0x5558039af940 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:21.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:21.252+0000 7fa839837740 1 -- 172.21.15.5:0/2706190099 >> 172.21.15.5:0/2706190099 conn(0x5558039abaf0 msgr2=0x5558039abed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:21.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:21.252+0000 7fa82fe00640 1 --2- 172.21.15.5:0/3301664983 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fa8140a2c00 0x7fa8140a50c0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fa820098750 tx=0x7fa820018420 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:21.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:21.252+0000 7fa839837740 1 -- 172.21.15.5:0/2706190099 shutdown_connections 2024-09-10T10:02:21.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:21.252+0000 7fa839837740 1 -- 172.21.15.5:0/2706190099 wait complete. 2024-09-10T10:02:21.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:21.252+0000 7fa839837740 1 -- start start 2024-09-10T10:02:21.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:21.253+0000 7fa839837740 1 Processor -- start 2024-09-10T10:02:21.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:21.254+0000 7fa839837740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5558039af560 0x5558039b7630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:21.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:21.254+0000 7fa839837740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5558039afe80 0x5558039b7b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:21.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:21.254+0000 7fa839837740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5558039ba740 0x5558039b80b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:21.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:21.254+0000 7fa839837740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5558039bd310 con 0x5558039ba740 2024-09-10T10:02:21.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:21.254+0000 7fa839837740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x5558038cfe60 con 0x5558039af560 2024-09-10T10:02:21.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:21.254+0000 7fa839837740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5558039bd490 con 0x5558039afe80 2024-09-10T10:02:21.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:21.254+0000 7fa839837740 10 client.0 _ll_register_callbacks cb 0x5558038d4cc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:02:21.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:21.254+0000 7fa834c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5558039af560 0x5558039b7630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:21.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:21.254+0000 7fa834c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5558039af560 0x5558039b7630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:60290/0 (socket says 172.21.15.5:60290) 2024-09-10T10:02:21.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:21.254+0000 7fa834c00640 1 -- 172.21.15.5:0/305526659 learned_addr learned my addr 172.21.15.5:0/305526659 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T10:02:21.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:21.254+0000 7fa834c00640 1 --2- 172.21.15.5:0/305526659 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5558039ba740 0x5558039b80b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:21.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:21.254+0000 7fa827e00640 1 --2- 172.21.15.5:0/305526659 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5558039afe80 0x5558039b7b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:21.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:21.255+0000 7fa834c00640 1 -- 172.21.15.5:0/305526659 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5558039afe80 msgr2=0x5558039b7b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:21.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:21.255+0000 7fa834c00640 1 --2- 172.21.15.5:0/305526659 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5558039afe80 0x5558039b7b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:21.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:21.255+0000 7fa834c00640 1 -- 172.21.15.5:0/305526659 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5558039ba740 msgr2=0x5558039b80b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:21.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:21.255+0000 7fa834c00640 1 --2- 172.21.15.5:0/305526659 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5558039ba740 0x5558039b80b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:21.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:21.255+0000 7fa834c00640 1 -- 172.21.15.5:0/305526659 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5558038db990 con 0x5558039af560 2024-09-10T10:02:21.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:21.255+0000 7fa834c00640 1 --2- 172.21.15.5:0/305526659 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5558039af560 0x5558039b7630 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7fa828009620 tx=0x7fa82802ff10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c3e978361a03bb0e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:21.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:21.255+0000 7fa81d600640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-09-10T10:02:21.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:21.255+0000 7fa81d600640 1 -- 172.21.15.5:0/305526659 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa828003970 con 0x5558039af560 2024-09-10T10:02:21.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:21.255+0000 7fa81d600640 1 -- 172.21.15.5:0/305526659 <== mon.2 v2:172.21.15.102:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fa828003b10 con 0x5558039af560 2024-09-10T10:02:21.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:21.255+0000 7fa839837740 1 -- 172.21.15.5:0/305526659 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa828003e50 con 0x5558039af560 2024-09-10T10:02:21.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:21.256+0000 7fa81d600640 1 -- 172.21.15.5:0/305526659 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7fa828004300 con 0x5558039af560 2024-09-10T10:02:21.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:21.256+0000 7fa839837740 10 client.29909 fetch_fsmap learned FSMap version 42 2024-09-10T10:02:21.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:21.256+0000 7fa839837740 1 -- 172.21.15.5:0/305526659 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x5558039bd310 con 0x5558039af560 2024-09-10T10:02:21.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:21.256+0000 7fa81d600640 1 -- 172.21.15.5:0/305526659 <== mon.2 v2:172.21.15.102:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa828003e50 con 0x5558039af560 2024-09-10T10:02:21.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:21.256+0000 7fa81d600640 1 -- 172.21.15.5:0/305526659 <== mon.2 v2:172.21.15.102:3300/0 5 ==== fsmap(e 42) ==== 8370+0+0 (secure 0 0 0) 0x7fa82804f040 con 0x5558039af560 2024-09-10T10:02:21.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:21.256+0000 7fa839837740 10 client.29909 fetch_fsmap finished waiting for FSMap version 42 2024-09-10T10:02:21.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:21.256+0000 7fa839837740 10 client.29909 resolve_mds: validated gid 24542 aka daemon mds.b 2024-09-10T10:02:21.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:21.256+0000 7fa839837740 20 client.29909 populate_metadata read hostname 'smithi005' 2024-09-10T10:02:21.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:21.256+0000 7fa839837740 1 --2- 172.21.15.5:0/305526659 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x5558039ca0a0 0x5558039cc490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:21.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:21.256+0000 7fa839837740 4 client.29909 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:02:21.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:21.256+0000 7fa839837740 1 -- 172.21.15.5:0/305526659 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5558038b8310 con 0x5558039ca0a0 2024-09-10T10:02:21.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:21.257+0000 7fa827e00640 1 --2- 172.21.15.5:0/305526659 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x5558039ca0a0 0x5558039cc490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:21.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:21.257+0000 7fa827e00640 1 --2- 172.21.15.5:0/305526659 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x5558039ca0a0 0x5558039cc490 crc :-1 s=READY pgs=321 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e6978017ae5acbf5 server_cookie=4fcd0049f8694bd8 in_seq=0 out_seq=0 2024-09-10T10:02:21.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:21.257+0000 7fa81d600640 10 client.29909 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T10:02:21.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:21.258+0000 7fa81d600640 1 -- 172.21.15.5:0/305526659 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5558038b8310 con 0x5558039ca0a0 2024-09-10T10:02:21.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:21.258+0000 7fa81d600640 10 client.29909 handle_command_reply: tid=0 2024-09-10T10:02:21.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:21.258+0000 7fa839837740 1 client.29909 shutdown 2024-09-10T10:02:21.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:21.259+0000 7fa839837740 1 -- 172.21.15.5:0/305526659 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5558039af560 msgr2=0x5558039b7630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:21.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:21.259+0000 7fa839837740 1 --2- 172.21.15.5:0/305526659 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5558039af560 0x5558039b7630 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7fa828009620 tx=0x7fa82802ff10 comp rx=0 tx=0).stop 2024-09-10T10:02:21.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:21.259+0000 7fa839837740 1 -- 172.21.15.5:0/305526659 shutdown_connections 2024-09-10T10:02:21.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:21.259+0000 7fa839837740 1 --2- 172.21.15.5:0/305526659 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x5558039ca0a0 0x5558039cc490 crc :-1 s=READY pgs=321 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:21.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:21.259+0000 7fa839837740 1 --2- 172.21.15.5:0/305526659 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5558039ba740 0x5558039b80b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:21.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:21.259+0000 7fa839837740 1 --2- 172.21.15.5:0/305526659 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5558039afe80 0x5558039b7b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:21.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:21.259+0000 7fa839837740 1 --2- 172.21.15.5:0/305526659 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5558039af560 0x5558039b7630 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:21.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:21.259+0000 7fa839837740 1 -- 172.21.15.5:0/305526659 >> 172.21.15.5:0/305526659 conn(0x5558039abaf0 msgr2=0x5558039b1820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:21.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:21.259+0000 7fa81d600640 0 client.29909 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T10:02:21.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:21.259+0000 7fa839837740 1 -- 172.21.15.5:0/305526659 shutdown_connections 2024-09-10T10:02:21.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:21.259+0000 7fa839837740 1 -- 172.21.15.5:0/305526659 wait complete. 2024-09-10T10:02:21.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:21.259+0000 7fa839837740 20 client.29909 trim_cache size 0 max 16384 2024-09-10T10:02:21.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:21.279+0000 7fa839837740 1 Processor -- start 2024-09-10T10:02:21.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:21.279+0000 7fa839837740 1 -- start start 2024-09-10T10:02:21.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:21.279+0000 7fa839837740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5558039af560 0x5558039b6570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:21.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:21.279+0000 7fa839837740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5558039afe80 0x5558039b2010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:21.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:21.279+0000 7fa839837740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5558039ba740 0x5558039d1b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:21.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:21.279+0000 7fa839837740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa8301b53e0 con 0x5558039ba740 2024-09-10T10:02:21.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:21.279+0000 7fa839837740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fa8301b6f20 con 0x5558039af560 2024-09-10T10:02:21.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:21.279+0000 7fa839837740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fa8301b5ab0 con 0x5558039afe80 2024-09-10T10:02:21.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:21.280+0000 7fa834c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5558039af560 0x5558039b6570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:21.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:21.280+0000 7fa834c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5558039af560 0x5558039b6570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:60292/0 (socket says 172.21.15.5:60292) 2024-09-10T10:02:21.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:21.280+0000 7fa834c00640 1 -- 172.21.15.5:0/3892140498 learned_addr learned my addr 172.21.15.5:0/3892140498 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T10:02:21.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:21.280+0000 7fa827e00640 1 --2- 172.21.15.5:0/3892140498 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5558039afe80 0x5558039b2010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:21.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:21.280+0000 7fa834c00640 1 --2- 172.21.15.5:0/3892140498 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5558039ba740 0x5558039d1b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:21.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:21.280+0000 7fa827e00640 1 -- 172.21.15.5:0/3892140498 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5558039af560 msgr2=0x5558039b6570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:21.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:21.280+0000 7fa827e00640 1 --2- 172.21.15.5:0/3892140498 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5558039af560 0x5558039b6570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:21.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:21.280+0000 7fa827e00640 1 -- 172.21.15.5:0/3892140498 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5558039ba740 msgr2=0x5558039d1b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:21.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:21.280+0000 7fa827e00640 1 --2- 172.21.15.5:0/3892140498 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5558039ba740 0x5558039d1b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:21.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:21.280+0000 7fa827e00640 1 -- 172.21.15.5:0/3892140498 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5558039a7530 con 0x5558039afe80 2024-09-10T10:02:21.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:21.280+0000 7fa827e00640 1 --2- 172.21.15.5:0/3892140498 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5558039afe80 0x5558039b2010 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x5558039b8c70 tx=0x7fa818005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=88a045e6821b7fd1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:21.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:21.281+0000 7fa81fe00640 1 -- 172.21.15.5:0/3892140498 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa818010430 con 0x5558039afe80 2024-09-10T10:02:21.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:21.281+0000 7fa81fe00640 1 -- 172.21.15.5:0/3892140498 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa818010ab0 con 0x5558039afe80 2024-09-10T10:02:21.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:21.281+0000 7fa81fe00640 1 -- 172.21.15.5:0/3892140498 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa81801eb60 con 0x5558039afe80 2024-09-10T10:02:21.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:21.282+0000 7fa839837740 1 -- 172.21.15.5:0/3892140498 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5558039afe80 msgr2=0x5558039b2010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:21.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:21.282+0000 7fa839837740 1 --2- 172.21.15.5:0/3892140498 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5558039afe80 0x5558039b2010 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x5558039b8c70 tx=0x7fa818005f90 comp rx=0 tx=0).stop 2024-09-10T10:02:21.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:21.282+0000 7fa839837740 1 -- 172.21.15.5:0/3892140498 shutdown_connections 2024-09-10T10:02:21.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:21.282+0000 7fa839837740 1 --2- 172.21.15.5:0/3892140498 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5558039ba740 0x5558039d1b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:21.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:21.282+0000 7fa839837740 1 --2- 172.21.15.5:0/3892140498 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5558039afe80 0x5558039b2010 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:21.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:21.282+0000 7fa839837740 1 --2- 172.21.15.5:0/3892140498 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5558039af560 0x5558039b6570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:21.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:21.282+0000 7fa839837740 1 -- 172.21.15.5:0/3892140498 >> 172.21.15.5:0/3892140498 conn(0x5558039abaf0 msgr2=0x5558039b95a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:21.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:21.282+0000 7fa839837740 1 -- 172.21.15.5:0/3892140498 shutdown_connections 2024-09-10T10:02:21.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:21.282+0000 7fa839837740 1 -- 172.21.15.5:0/3892140498 wait complete. 2024-09-10T10:02:21.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:21.282+0000 7fa839837740 1 -- start start 2024-09-10T10:02:21.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:21.283+0000 7fa839837740 1 Processor -- start 2024-09-10T10:02:21.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:21.283+0000 7fa839837740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5558039af560 0x5558039c26e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:21.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:21.283+0000 7fa839837740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5558039afe80 0x5558039c2c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:21.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:21.283+0000 7fa839837740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5558039ba740 0x5558039bf6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:21.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:21.283+0000 7fa839837740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa8301b6850 con 0x5558039afe80 2024-09-10T10:02:21.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:21.283+0000 7fa839837740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fa8301b6180 con 0x5558039af560 2024-09-10T10:02:21.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:21.283+0000 7fa839837740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fa8301b33c0 con 0x5558039ba740 2024-09-10T10:02:21.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:21.283+0000 7fa839837740 10 client.0 _ll_register_callbacks cb 0x7fa8180097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:02:21.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:21.284+0000 7fa827e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5558039afe80 0x5558039c2c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:21.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:21.284+0000 7fa834c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5558039af560 0x5558039c26e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:21.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:21.284+0000 7fa834c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5558039af560 0x5558039c26e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:60304/0 (socket says 172.21.15.5:60304) 2024-09-10T10:02:21.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:21.284+0000 7fa834c00640 1 -- 172.21.15.5:0/3619685148 learned_addr learned my addr 172.21.15.5:0/3619685148 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T10:02:21.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:21.284+0000 7fa834c00640 1 --2- 172.21.15.5:0/3619685148 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5558039ba740 0x5558039bf6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:21.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:21.284+0000 7fa834c00640 1 -- 172.21.15.5:0/3619685148 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5558039ba740 msgr2=0x5558039bf6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:21.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:21.284+0000 7fa834c00640 1 --2- 172.21.15.5:0/3619685148 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5558039ba740 0x5558039bf6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:21.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:21.284+0000 7fa834c00640 1 -- 172.21.15.5:0/3619685148 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5558039afe80 msgr2=0x5558039c2c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:21.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:21.284+0000 7fa834c00640 1 --2- 172.21.15.5:0/3619685148 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5558039afe80 0x5558039c2c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:21.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:21.284+0000 7fa834c00640 1 -- 172.21.15.5:0/3619685148 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5558039c3360 con 0x5558039af560 2024-09-10T10:02:21.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:21.284+0000 7fa827e00640 1 --2- 172.21.15.5:0/3619685148 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5558039afe80 0x5558039c2c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:02:21.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:21.285+0000 7fa81d600640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-09-10T10:02:21.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:21.284+0000 7fa834c00640 1 --2- 172.21.15.5:0/3619685148 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5558039af560 0x5558039c26e0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7fa828009620 tx=0x7fa82802fe70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a59e0e2140edaf9c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:21.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:21.285+0000 7fa81d600640 1 -- 172.21.15.5:0/3619685148 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa828061040 con 0x5558039af560 2024-09-10T10:02:21.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:21.285+0000 7fa81d600640 1 -- 172.21.15.5:0/3619685148 <== mon.2 v2:172.21.15.102:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fa828030650 con 0x5558039af560 2024-09-10T10:02:21.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:21.285+0000 7fa839837740 1 -- 172.21.15.5:0/3619685148 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa818010ab0 con 0x5558039af560 2024-09-10T10:02:21.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:21.285+0000 7fa81d600640 1 -- 172.21.15.5:0/3619685148 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7fa82802f420 con 0x5558039af560 2024-09-10T10:02:21.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:21.285+0000 7fa839837740 10 client.29915 fetch_fsmap learned FSMap version 42 2024-09-10T10:02:21.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:21.285+0000 7fa839837740 1 -- 172.21.15.5:0/3619685148 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x555803a35180 con 0x5558039af560 2024-09-10T10:02:21.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:21.286+0000 7fa81d600640 1 -- 172.21.15.5:0/3619685148 <== mon.2 v2:172.21.15.102:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa818010ab0 con 0x5558039af560 2024-09-10T10:02:21.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:21.286+0000 7fa81d600640 1 -- 172.21.15.5:0/3619685148 <== mon.2 v2:172.21.15.102:3300/0 5 ==== fsmap(e 42) ==== 8370+0+0 (secure 0 0 0) 0x7fa828042070 con 0x5558039af560 2024-09-10T10:02:21.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:21.286+0000 7fa839837740 10 client.29915 fetch_fsmap finished waiting for FSMap version 42 2024-09-10T10:02:21.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:21.286+0000 7fa839837740 10 client.29915 resolve_mds: validated gid 24542 aka daemon mds.b 2024-09-10T10:02:21.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:21.286+0000 7fa839837740 20 client.29915 populate_metadata read hostname 'smithi005' 2024-09-10T10:02:21.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:21.286+0000 7fa839837740 1 --2- 172.21.15.5:0/3619685148 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x555803a358b0 0x555803a37ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:21.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:21.286+0000 7fa839837740 4 client.29915 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 135.3, "await_for": 45.0}] 2024-09-10T10:02:21.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:21.286+0000 7fa839837740 1 -- 172.21.15.5:0/3619685148 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 135.3, "await_for": 45.0}) -- 0x555803922c60 con 0x555803a358b0 2024-09-10T10:02:21.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:21.286+0000 7fa827e00640 1 --2- 172.21.15.5:0/3619685148 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x555803a358b0 0x555803a37ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:21.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:21.287+0000 7fa827e00640 1 --2- 172.21.15.5:0/3619685148 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x555803a358b0 0x555803a37ca0 crc :-1 s=READY pgs=322 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6056bd6a3c01a9b4 server_cookie=93989379bd947235 in_seq=0 out_seq=0 2024-09-10T10:02:21.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:21.287+0000 7fa81d600640 10 client.29915 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T10:02:21.290 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:02:21.379 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:02:21 smithi005 ceph-mon[30350]: pgmap v1113: 129 pgs: 129 active+clean; 773 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 177 KiB/s rd, 5.1 MiB/s wr, 144 op/s 2024-09-10T10:02:21.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:02:21 smithi159 ceph-mon[35513]: pgmap v1113: 129 pgs: 129 active+clean; 773 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 177 KiB/s rd, 5.1 MiB/s wr, 144 op/s 2024-09-10T10:02:21.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:02:21 smithi102 ceph-mon[35442]: pgmap v1113: 129 pgs: 129 active+clean; 773 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 177 KiB/s rd, 5.1 MiB/s wr, 144 op/s 2024-09-10T10:02:22.069 INFO:tasks.workunit.client.0.smithi005.stdout: 10 156137 13.15 MB/sec execute 480 sec latency 18036.389 ms 2024-09-10T10:02:22.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:22.084+0000 7f4604a00640 1 -- 172.21.15.5:0/3183382714 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f460011b140 msgr2=0x7f460011d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:22.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:22.084+0000 7f4604a00640 1 --2- 172.21.15.5:0/3183382714 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f460011b140 0x7f460011d530 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f45f8009be0 tx=0x7f45f802a6e0 comp rx=0 tx=0).stop 2024-09-10T10:02:22.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:22.085+0000 7f4604a00640 1 -- 172.21.15.5:0/3183382714 shutdown_connections 2024-09-10T10:02:22.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:22.085+0000 7f4604a00640 1 --2- 172.21.15.5:0/3183382714 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f460011b140 0x7f460011d530 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:22.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:22.085+0000 7f4604a00640 1 --2- 172.21.15.5:0/3183382714 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4600109830 0x7f4600109c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:22.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:22.085+0000 7f4604a00640 1 --2- 172.21.15.5:0/3183382714 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f4600109450 0x7f4600110250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:22.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:22.085+0000 7f4604a00640 1 -- 172.21.15.5:0/3183382714 >> 172.21.15.5:0/3183382714 conn(0x7f460006e140 msgr2=0x7f460006e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:22.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:22.085+0000 7f4604a00640 1 -- 172.21.15.5:0/3183382714 shutdown_connections 2024-09-10T10:02:22.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:22.085+0000 7f4604a00640 1 -- 172.21.15.5:0/3183382714 wait complete. 2024-09-10T10:02:22.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:22.086+0000 7f4604a00640 1 Processor -- start 2024-09-10T10:02:22.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:22.087+0000 7f4604a00640 1 -- start start 2024-09-10T10:02:22.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:22.087+0000 7f4604a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4600109450 0x7f46001a7300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:22.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:22.087+0000 7f4604a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4600109830 0x7f46001a7840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:22.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:22.087+0000 7f4604a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f460011b140 0x7f46001abbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:22.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:22.087+0000 7f4604a00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f4600120de0 con 0x7f4600109830 2024-09-10T10:02:22.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:22.087+0000 7f4604a00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f4600120c60 con 0x7f460011b140 2024-09-10T10:02:22.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:22.087+0000 7f4604a00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f4600120f60 con 0x7f4600109450 2024-09-10T10:02:22.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:22.088+0000 7f45fde00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4600109450 0x7f46001a7300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:22.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:22.088+0000 7f45fde00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4600109450 0x7f46001a7300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:51286/0 (socket says 172.21.15.5:51286) 2024-09-10T10:02:22.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:22.088+0000 7f45fd400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4600109830 0x7f46001a7840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:22.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:22.088+0000 7f45fde00640 1 -- 172.21.15.5:0/32756320 learned_addr learned my addr 172.21.15.5:0/32756320 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T10:02:22.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:22.089+0000 7f45fe800640 1 --2- 172.21.15.5:0/32756320 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f460011b140 0x7f46001abbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:22.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:22.089+0000 7f45fde00640 1 -- 172.21.15.5:0/32756320 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f460011b140 msgr2=0x7f46001abbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:22.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:22.089+0000 7f45fde00640 1 --2- 172.21.15.5:0/32756320 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f460011b140 0x7f46001abbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:22.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:22.089+0000 7f45fde00640 1 -- 172.21.15.5:0/32756320 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4600109830 msgr2=0x7f46001a7840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:22.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:22.089+0000 7f45fde00640 1 --2- 172.21.15.5:0/32756320 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4600109830 0x7f46001a7840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:22.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:22.089+0000 7f45fde00640 1 -- 172.21.15.5:0/32756320 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46001ac2b0 con 0x7f4600109450 2024-09-10T10:02:22.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:22.089+0000 7f45fe800640 1 --2- 172.21.15.5:0/32756320 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f460011b140 0x7f46001abbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:02:22.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:22.090+0000 7f45fde00640 1 --2- 172.21.15.5:0/32756320 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4600109450 0x7f46001a7300 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f45f000e9c0 tx=0x7f45f000ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:22.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:22.090+0000 7f45f6a00640 1 -- 172.21.15.5:0/32756320 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45f000cde0 con 0x7f4600109450 2024-09-10T10:02:22.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:22.090+0000 7f45f6a00640 1 -- 172.21.15.5:0/32756320 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f45f00035f0 con 0x7f4600109450 2024-09-10T10:02:22.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:22.090+0000 7f45f6a00640 1 -- 172.21.15.5:0/32756320 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45f0010690 con 0x7f4600109450 2024-09-10T10:02:22.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:22.090+0000 7f4604a00640 1 -- 172.21.15.5:0/32756320 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46001ac5a0 con 0x7f4600109450 2024-09-10T10:02:22.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:22.091+0000 7f4604a00640 1 -- 172.21.15.5:0/32756320 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46001b3e80 con 0x7f4600109450 2024-09-10T10:02:22.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:22.098+0000 7f45f6a00640 1 -- 172.21.15.5:0/32756320 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f45f0004020 con 0x7f4600109450 2024-09-10T10:02:22.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:22.100+0000 7f45f6a00640 1 --2- 172.21.15.5:0/32756320 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f45e40a2d50 0x7f45e40a5210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:22.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:22.101+0000 7f45f6a00640 1 -- 172.21.15.5:0/32756320 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f45f00cf7f0 con 0x7f4600109450 2024-09-10T10:02:22.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:22.102+0000 7f46075d7740 1 Processor -- start 2024-09-10T10:02:22.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:22.102+0000 7f46075d7740 1 -- start start 2024-09-10T10:02:22.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:22.103+0000 7f46075d7740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564f7fbdfc70 0x564f7fbe0050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:22.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:22.103+0000 7f46075d7740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564f7fbe0590 0x564f7fbea910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:22.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:22.103+0000 7f46075d7740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564f7fbeae50 0x564f7fbed240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:22.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:22.103+0000 7f46075d7740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x564f7fb463e0 con 0x564f7fbdfc70 2024-09-10T10:02:22.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:22.103+0000 7f46075d7740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x564f7faedd80 con 0x564f7fbeae50 2024-09-10T10:02:22.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:22.103+0000 7f46075d7740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x564f7fb423a0 con 0x564f7fbe0590 2024-09-10T10:02:22.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:22.103+0000 7f45fd400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564f7fbe0590 0x564f7fbea910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:22.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:22.103+0000 7f45fe800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564f7fbeae50 0x564f7fbed240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:22.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:22.104+0000 7f45fe800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564f7fbeae50 0x564f7fbed240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:60324/0 (socket says 172.21.15.5:60324) 2024-09-10T10:02:22.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:22.104+0000 7f45fe800640 1 -- 172.21.15.5:0/3082303919 learned_addr learned my addr 172.21.15.5:0/3082303919 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T10:02:22.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:22.104+0000 7f45fe800640 1 -- 172.21.15.5:0/3082303919 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564f7fbe0590 msgr2=0x564f7fbea910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:22.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:22.104+0000 7f45fde00640 1 --2- 172.21.15.5:0/3082303919 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564f7fbdfc70 0x564f7fbe0050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:22.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:22.105+0000 7f45fe800640 1 --2- 172.21.15.5:0/3082303919 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564f7fbe0590 0x564f7fbea910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:22.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:22.105+0000 7f45fe800640 1 -- 172.21.15.5:0/3082303919 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564f7fbdfc70 msgr2=0x564f7fbe0050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:22.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:22.105+0000 7f45fe800640 1 --2- 172.21.15.5:0/3082303919 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564f7fbdfc70 0x564f7fbe0050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:22.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:22.105+0000 7f45fe800640 1 -- 172.21.15.5:0/3082303919 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564f7fb0cae0 con 0x564f7fbeae50 2024-09-10T10:02:22.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:22.105+0000 7f45fde00640 1 --2- 172.21.15.5:0/3082303919 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564f7fbdfc70 0x564f7fbe0050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:02:22.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:22.105+0000 7f45fd400640 1 --2- 172.21.15.5:0/3082303919 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564f7fbe0590 0x564f7fbea910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:02:22.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:22.106+0000 7f45fd400640 1 --2- 172.21.15.5:0/32756320 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f45e40a2d50 0x7f45e40a5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:22.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:22.108+0000 7f45fe800640 1 --2- 172.21.15.5:0/3082303919 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564f7fbeae50 0x564f7fbed240 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f45f8009be0 tx=0x7f45f8031550 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6106d4978ac4a610 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:22.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:22.108+0000 7f45eb000640 1 -- 172.21.15.5:0/3082303919 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45f8040030 con 0x564f7fbeae50 2024-09-10T10:02:22.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:22.109+0000 7f45eb000640 1 -- 172.21.15.5:0/3082303919 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f45f8030bf0 con 0x564f7fbeae50 2024-09-10T10:02:22.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:22.109+0000 7f45eb000640 1 -- 172.21.15.5:0/3082303919 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45f8036040 con 0x564f7fbeae50 2024-09-10T10:02:22.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:22.109+0000 7f45fd400640 1 --2- 172.21.15.5:0/32756320 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f45e40a2d50 0x7f45e40a5210 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x564f7fbdd060 tx=0x7f45ec008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:22.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:22.111+0000 7f46075d7740 1 -- 172.21.15.5:0/3082303919 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564f7fbeae50 msgr2=0x564f7fbed240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:22.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:22.111+0000 7f46075d7740 1 --2- 172.21.15.5:0/3082303919 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564f7fbeae50 0x564f7fbed240 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f45f8009be0 tx=0x7f45f8031550 comp rx=0 tx=0).stop 2024-09-10T10:02:22.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:22.111+0000 7f46075d7740 1 -- 172.21.15.5:0/3082303919 shutdown_connections 2024-09-10T10:02:22.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:22.111+0000 7f46075d7740 1 --2- 172.21.15.5:0/3082303919 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564f7fbeae50 0x564f7fbed240 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:22.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:22.111+0000 7f46075d7740 1 --2- 172.21.15.5:0/3082303919 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564f7fbe0590 0x564f7fbea910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:22.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:22.111+0000 7f46075d7740 1 --2- 172.21.15.5:0/3082303919 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564f7fbdfc70 0x564f7fbe0050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:22.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:22.111+0000 7f46075d7740 1 -- 172.21.15.5:0/3082303919 >> 172.21.15.5:0/3082303919 conn(0x564f7fbdc200 msgr2=0x564f7fbdc5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:22.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:22.112+0000 7f46075d7740 1 -- 172.21.15.5:0/3082303919 shutdown_connections 2024-09-10T10:02:22.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:22.112+0000 7f46075d7740 1 -- 172.21.15.5:0/3082303919 wait complete. 2024-09-10T10:02:22.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:22.113+0000 7f46075d7740 1 -- start start 2024-09-10T10:02:22.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:22.113+0000 7f46075d7740 1 Processor -- start 2024-09-10T10:02:22.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:22.116+0000 7f46075d7740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564f7fbdfc70 0x564f7fbe7cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:22.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:22.116+0000 7f46075d7740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564f7fbe0590 0x564f7fbe8210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:22.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:22.116+0000 7f46075d7740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564f7fbeae50 0x564f7fbe8750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:22.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:22.116+0000 7f45fd400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564f7fbe0590 0x564f7fbe8210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:22.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:22.116+0000 7f45fd400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564f7fbe0590 0x564f7fbe8210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:51304/0 (socket says 172.21.15.5:51304) 2024-09-10T10:02:22.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:22.117+0000 7f45fde00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564f7fbdfc70 0x564f7fbe7cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:22.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:22.117+0000 7f45fde00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564f7fbdfc70 0x564f7fbe7cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58180/0 (socket says 172.21.15.5:58180) 2024-09-10T10:02:22.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:22.117+0000 7f45fde00640 1 -- 172.21.15.5:0/2664026677 learned_addr learned my addr 172.21.15.5:0/2664026677 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T10:02:22.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:22.117+0000 7f46075d7740 1 -- 172.21.15.5:0/2664026677 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x564f7fbfea30 con 0x564f7fbdfc70 2024-09-10T10:02:22.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:22.117+0000 7f46075d7740 1 -- 172.21.15.5:0/2664026677 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x564f7fb010f0 con 0x564f7fbeae50 2024-09-10T10:02:22.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:22.117+0000 7f46075d7740 1 -- 172.21.15.5:0/2664026677 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x564f7fbfebb0 con 0x564f7fbe0590 2024-09-10T10:02:22.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:22.117+0000 7f45fe800640 1 --2- 172.21.15.5:0/2664026677 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564f7fbeae50 0x564f7fbe8750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:22.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:22.117+0000 7f46075d7740 10 client.0 _ll_register_callbacks cb 0x564f7fae8bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:02:22.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:22.117+0000 7f45fe800640 1 -- 172.21.15.5:0/2664026677 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564f7fbe0590 msgr2=0x564f7fbe8210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:22.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:22.117+0000 7f45fe800640 1 --2- 172.21.15.5:0/2664026677 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564f7fbe0590 0x564f7fbe8210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:22.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:22.117+0000 7f45fe800640 1 -- 172.21.15.5:0/2664026677 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564f7fbdfc70 msgr2=0x564f7fbe7cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:22.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:22.117+0000 7f45fe800640 1 --2- 172.21.15.5:0/2664026677 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564f7fbdfc70 0x564f7fbe7cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:22.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:22.117+0000 7f45fe800640 1 -- 172.21.15.5:0/2664026677 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564f7f8b73c0 con 0x564f7fbeae50 2024-09-10T10:02:22.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:22.118+0000 7f45fd400640 1 --2- 172.21.15.5:0/2664026677 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564f7fbe0590 0x564f7fbe8210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:02:22.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:22.118+0000 7f46075d7740 1 -- 172.21.15.5:0/2664026677 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f45f8030bf0 con 0x564f7fbeae50 2024-09-10T10:02:22.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:22.118+0000 7f45fe800640 1 --2- 172.21.15.5:0/2664026677 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564f7fbeae50 0x564f7fbe8750 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f45f8009b30 tx=0x7f45f80039b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ede08f682c4f7233 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:22.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:22.119+0000 7f45df400640 10 client.29927 ms_handle_connect on v2:172.21.15.102:3300/0 2024-09-10T10:02:22.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:22.119+0000 7f45df400640 1 -- 172.21.15.5:0/2664026677 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45f8036050 con 0x564f7fbeae50 2024-09-10T10:02:22.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:22.119+0000 7f45df400640 1 -- 172.21.15.5:0/2664026677 <== mon.2 v2:172.21.15.102:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f45f80303a0 con 0x564f7fbeae50 2024-09-10T10:02:22.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:22.119+0000 7f45df400640 1 -- 172.21.15.5:0/2664026677 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f45f8040040 con 0x564f7fbeae50 2024-09-10T10:02:22.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:22.119+0000 7f46075d7740 10 client.29927 fetch_fsmap learned FSMap version 42 2024-09-10T10:02:22.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:22.119+0000 7f46075d7740 1 -- 172.21.15.5:0/2664026677 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({fsmap=42}) -- 0x564f7fc0a860 con 0x564f7fbeae50 2024-09-10T10:02:22.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:22.120+0000 7f45df400640 1 -- 172.21.15.5:0/2664026677 <== mon.2 v2:172.21.15.102:3300/0 4 ==== fsmap(e 42) ==== 8370+0+0 (secure 0 0 0) 0x7f45f80305e0 con 0x564f7fbeae50 2024-09-10T10:02:22.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:22.120+0000 7f46075d7740 10 client.29927 fetch_fsmap finished waiting for FSMap version 42 2024-09-10T10:02:22.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:22.120+0000 7f46075d7740 10 client.29927 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-09-10T10:02:22.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:22.120+0000 7f46075d7740 20 client.29927 populate_metadata read hostname 'smithi005' 2024-09-10T10:02:22.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:22.120+0000 7f46075d7740 1 --2- 172.21.15.5:0/2664026677 >> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] conn(0x564f7fc0bbb0 0x564f7fc0dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:22.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:22.120+0000 7f46075d7740 4 client.29927 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:02:22.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:22.120+0000 7f46075d7740 1 -- 172.21.15.5:0/2664026677 --> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564f7fb53950 con 0x564f7fc0bbb0 2024-09-10T10:02:22.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:22.121+0000 7f45fde00640 1 --2- 172.21.15.5:0/2664026677 >> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] conn(0x564f7fc0bbb0 0x564f7fc0dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:22.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:22.121+0000 7f45fde00640 1 --2- 172.21.15.5:0/2664026677 >> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] conn(0x564f7fc0bbb0 0x564f7fc0dfa0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=cf6689206bd2aa20 server_cookie=767542da3f1ce46a in_seq=0 out_seq=0 2024-09-10T10:02:22.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:22.121+0000 7f45df400640 10 client.29927 ms_handle_connect on v2:172.21.15.102:6836/2689776764 2024-09-10T10:02:22.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:22.122+0000 7f45df400640 1 -- 172.21.15.5:0/2664026677 <== mds.3 v2:172.21.15.102:6836/2689776764 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564f7fb53950 con 0x564f7fc0bbb0 2024-09-10T10:02:22.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:22.122+0000 7f45df400640 10 client.29927 handle_command_reply: tid=0 2024-09-10T10:02:22.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:22.123+0000 7f46075d7740 1 client.29927 shutdown 2024-09-10T10:02:22.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:22.123+0000 7f46075d7740 1 -- 172.21.15.5:0/2664026677 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564f7fbeae50 msgr2=0x564f7fbe8750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:22.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:22.123+0000 7f46075d7740 1 --2- 172.21.15.5:0/2664026677 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564f7fbeae50 0x564f7fbe8750 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f45f8009b30 tx=0x7f45f80039b0 comp rx=0 tx=0).stop 2024-09-10T10:02:22.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:22.123+0000 7f46075d7740 1 -- 172.21.15.5:0/2664026677 shutdown_connections 2024-09-10T10:02:22.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:22.123+0000 7f46075d7740 1 --2- 172.21.15.5:0/2664026677 >> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] conn(0x564f7fc0bbb0 0x564f7fc0dfa0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:22.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:22.124+0000 7f46075d7740 1 --2- 172.21.15.5:0/2664026677 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564f7fbeae50 0x564f7fbe8750 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:22.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:22.124+0000 7f46075d7740 1 --2- 172.21.15.5:0/2664026677 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564f7fbe0590 0x564f7fbe8210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:22.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:22.124+0000 7f46075d7740 1 --2- 172.21.15.5:0/2664026677 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564f7fbdfc70 0x564f7fbe7cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:22.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:22.124+0000 7f46075d7740 1 -- 172.21.15.5:0/2664026677 >> 172.21.15.5:0/2664026677 conn(0x564f7fbdc200 msgr2=0x564f7fbe1f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:22.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:22.124+0000 7f45df400640 0 client.29927 ms_handle_reset on v2:172.21.15.102:6836/2689776764 2024-09-10T10:02:22.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:22.124+0000 7f46075d7740 1 -- 172.21.15.5:0/2664026677 shutdown_connections 2024-09-10T10:02:22.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:22.124+0000 7f46075d7740 1 -- 172.21.15.5:0/2664026677 wait complete. 2024-09-10T10:02:22.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:22.125+0000 7f46075d7740 20 client.29927 trim_cache size 0 max 16384 2024-09-10T10:02:22.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:22.153+0000 7f46075d7740 1 Processor -- start 2024-09-10T10:02:22.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:22.153+0000 7f46075d7740 1 -- start start 2024-09-10T10:02:22.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:22.153+0000 7f46075d7740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564f7fbdfc70 0x564f7fbe85a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:22.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:22.153+0000 7f46075d7740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564f7fbe0590 0x564f7fc10fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:22.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:22.153+0000 7f46075d7740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564f7fbeae50 0x564f7fc11500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:22.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:22.153+0000 7f46075d7740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f46001b5540 con 0x564f7fbeae50 2024-09-10T10:02:22.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:22.153+0000 7f46075d7740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f46001b7080 con 0x564f7fbe0590 2024-09-10T10:02:22.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:22.153+0000 7f46075d7740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f46001b5c10 con 0x564f7fbdfc70 2024-09-10T10:02:22.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:22.154+0000 7f45fd400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564f7fbe0590 0x564f7fc10fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:22.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:22.154+0000 7f45fde00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564f7fbdfc70 0x564f7fbe85a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:22.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:22.154+0000 7f45fd400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564f7fbe0590 0x564f7fc10fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:60350/0 (socket says 172.21.15.5:60350) 2024-09-10T10:02:22.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:22.154+0000 7f45fd400640 1 -- 172.21.15.5:0/117457680 learned_addr learned my addr 172.21.15.5:0/117457680 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T10:02:22.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:22.154+0000 7f45fd400640 1 -- 172.21.15.5:0/117457680 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564f7fbdfc70 msgr2=0x564f7fbe85a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:22.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:22.155+0000 7f45fd400640 1 --2- 172.21.15.5:0/117457680 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564f7fbdfc70 0x564f7fbe85a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:22.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:22.155+0000 7f45fd400640 1 -- 172.21.15.5:0/117457680 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564f7fbeae50 msgr2=0x564f7fc11500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:22.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:22.155+0000 7f45fd400640 1 --2- 172.21.15.5:0/117457680 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564f7fbeae50 0x564f7fc11500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:22.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:22.155+0000 7f45fd400640 1 -- 172.21.15.5:0/117457680 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564f7fbdfaa0 con 0x564f7fbe0590 2024-09-10T10:02:22.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:22.155+0000 7f45fde00640 1 --2- 172.21.15.5:0/117457680 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564f7fbdfc70 0x564f7fbe85a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:02:22.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:22.155+0000 7f45fd400640 1 --2- 172.21.15.5:0/117457680 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564f7fbe0590 0x564f7fc10fc0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f45ec006a50 tx=0x7f45ec006f50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=de85093b66319270 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:22.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:22.156+0000 7f45ea600640 1 -- 172.21.15.5:0/117457680 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45ec021470 con 0x564f7fbe0590 2024-09-10T10:02:22.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:22.156+0000 7f45ea600640 1 -- 172.21.15.5:0/117457680 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f45ec026070 con 0x564f7fbe0590 2024-09-10T10:02:22.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:22.156+0000 7f45ea600640 1 -- 172.21.15.5:0/117457680 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45ec0138b0 con 0x564f7fbe0590 2024-09-10T10:02:22.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:22.158+0000 7f46075d7740 1 -- 172.21.15.5:0/117457680 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564f7fbe0590 msgr2=0x564f7fc10fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:22.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:22.158+0000 7f46075d7740 1 --2- 172.21.15.5:0/117457680 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564f7fbe0590 0x564f7fc10fc0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f45ec006a50 tx=0x7f45ec006f50 comp rx=0 tx=0).stop 2024-09-10T10:02:22.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:22.158+0000 7f46075d7740 1 -- 172.21.15.5:0/117457680 shutdown_connections 2024-09-10T10:02:22.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:22.158+0000 7f46075d7740 1 --2- 172.21.15.5:0/117457680 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564f7fbeae50 0x564f7fc11500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:22.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:22.158+0000 7f46075d7740 1 --2- 172.21.15.5:0/117457680 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564f7fbe0590 0x564f7fc10fc0 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:22.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:22.158+0000 7f46075d7740 1 --2- 172.21.15.5:0/117457680 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564f7fbdfc70 0x564f7fbe85a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:22.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:22.158+0000 7f46075d7740 1 -- 172.21.15.5:0/117457680 >> 172.21.15.5:0/117457680 conn(0x564f7fbdc200 msgr2=0x564f7fbe9d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:22.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:22.158+0000 7f46075d7740 1 -- 172.21.15.5:0/117457680 shutdown_connections 2024-09-10T10:02:22.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:22.159+0000 7f46075d7740 1 -- 172.21.15.5:0/117457680 wait complete. 2024-09-10T10:02:22.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:22.159+0000 7f46075d7740 1 -- start start 2024-09-10T10:02:22.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:22.160+0000 7f46075d7740 1 Processor -- start 2024-09-10T10:02:22.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:22.160+0000 7f46075d7740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564f7fbdfc70 0x564f7fc11750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:22.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:22.160+0000 7f46075d7740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564f7fbe0590 0x564f7fc0eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:22.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:22.160+0000 7f46075d7740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564f7fbeae50 0x564f7fc0f030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:22.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:22.160+0000 7f46075d7740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f46001b69b0 con 0x564f7fbe0590 2024-09-10T10:02:22.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:22.160+0000 7f46075d7740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f46001b62e0 con 0x564f7fbeae50 2024-09-10T10:02:22.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:22.160+0000 7f46075d7740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f46001185f0 con 0x564f7fbdfc70 2024-09-10T10:02:22.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:22.160+0000 7f46075d7740 10 client.0 _ll_register_callbacks cb 0x7f45f8034070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:02:22.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:22.161+0000 7f45fde00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564f7fbdfc70 0x564f7fc11750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:22.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:22.161+0000 7f45fe800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564f7fbeae50 0x564f7fc0f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:22.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:22.161+0000 7f45fe800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564f7fbeae50 0x564f7fc0f030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:60362/0 (socket says 172.21.15.5:60362) 2024-09-10T10:02:22.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:22.161+0000 7f45fde00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564f7fbdfc70 0x564f7fc11750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:51312/0 (socket says 172.21.15.5:51312) 2024-09-10T10:02:22.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:22.161+0000 7f45fde00640 1 -- 172.21.15.5:0/2926278270 learned_addr learned my addr 172.21.15.5:0/2926278270 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T10:02:22.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:22.162+0000 7f45fde00640 1 -- 172.21.15.5:0/2926278270 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564f7fbeae50 msgr2=0x564f7fc0f030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:22.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:22.162+0000 7f45fde00640 1 --2- 172.21.15.5:0/2926278270 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564f7fbeae50 0x564f7fc0f030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:22.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:22.162+0000 7f45fde00640 1 -- 172.21.15.5:0/2926278270 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564f7fbe0590 msgr2=0x564f7fc0eaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:22.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:22.162+0000 7f45fde00640 1 --2- 172.21.15.5:0/2926278270 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564f7fbe0590 0x564f7fc0eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:22.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:22.162+0000 7f45fde00640 1 -- 172.21.15.5:0/2926278270 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564f7fbd7770 con 0x564f7fbdfc70 2024-09-10T10:02:22.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:22.162+0000 7f45fe800640 1 --2- 172.21.15.5:0/2926278270 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564f7fbeae50 0x564f7fc0f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:02:22.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:22.162+0000 7f45fde00640 1 --2- 172.21.15.5:0/2926278270 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564f7fbdfc70 0x564f7fc11750 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f45f00985d0 tx=0x7f45f00038a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=172f9d99116932a9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:22.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:22.162+0000 7f45df400640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-09-10T10:02:22.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:22.163+0000 7f45df400640 1 -- 172.21.15.5:0/2926278270 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45f009ffb0 con 0x564f7fbdfc70 2024-09-10T10:02:22.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:22.163+0000 7f45df400640 1 -- 172.21.15.5:0/2926278270 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f45f00b4e00 con 0x564f7fbdfc70 2024-09-10T10:02:22.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:22.163+0000 7f46075d7740 1 -- 172.21.15.5:0/2926278270 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f45ec026070 con 0x564f7fbdfc70 2024-09-10T10:02:22.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:22.163+0000 7f45df400640 1 -- 172.21.15.5:0/2926278270 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f45f00a4870 con 0x564f7fbdfc70 2024-09-10T10:02:22.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:22.163+0000 7f46075d7740 10 client.29899 fetch_fsmap learned FSMap version 42 2024-09-10T10:02:22.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:22.163+0000 7f46075d7740 1 -- 172.21.15.5:0/2926278270 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x564f7fc763b0 con 0x564f7fbdfc70 2024-09-10T10:02:22.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:22.164+0000 7f45df400640 1 -- 172.21.15.5:0/2926278270 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f45ec026070 con 0x564f7fbdfc70 2024-09-10T10:02:22.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:22.164+0000 7f45df400640 1 -- 172.21.15.5:0/2926278270 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 42) ==== 8370+0+0 (secure 0 0 0) 0x7f45f00cf3e0 con 0x564f7fbdfc70 2024-09-10T10:02:22.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:22.164+0000 7f46075d7740 10 client.29899 fetch_fsmap finished waiting for FSMap version 42 2024-09-10T10:02:22.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:22.164+0000 7f46075d7740 10 client.29899 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-09-10T10:02:22.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:22.164+0000 7f46075d7740 20 client.29899 populate_metadata read hostname 'smithi005' 2024-09-10T10:02:22.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:22.164+0000 7f46075d7740 1 --2- 172.21.15.5:0/2926278270 >> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] conn(0x564f7fc76ad0 0x564f7fc78ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:22.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:22.164+0000 7f46075d7740 4 client.29899 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T10:02:22.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:22.164+0000 7f46075d7740 1 -- 172.21.15.5:0/2926278270 --> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564f7fb4de60 con 0x564f7fc76ad0 2024-09-10T10:02:22.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:22.164+0000 7f45fd400640 1 --2- 172.21.15.5:0/2926278270 >> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] conn(0x564f7fc76ad0 0x564f7fc78ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:22.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:22.165+0000 7f45fd400640 1 --2- 172.21.15.5:0/2926278270 >> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] conn(0x564f7fc76ad0 0x564f7fc78ee0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=363ac34bbeed12e5 server_cookie=b7cf17121aadaaef in_seq=0 out_seq=0 2024-09-10T10:02:22.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:22.165+0000 7f45df400640 10 client.29899 ms_handle_connect on v2:172.21.15.102:6836/2689776764 2024-09-10T10:02:22.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:22.165+0000 7f45df400640 1 -- 172.21.15.5:0/2926278270 <== mds.3 v2:172.21.15.102:6836/2689776764 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564f7fb4de60 con 0x564f7fc76ad0 2024-09-10T10:02:22.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:22.165+0000 7f45df400640 10 client.29899 handle_command_reply: tid=0 2024-09-10T10:02:22.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:22.166+0000 7f46075d7740 1 client.29899 shutdown 2024-09-10T10:02:22.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:22.166+0000 7f46075d7740 1 -- 172.21.15.5:0/2926278270 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564f7fbdfc70 msgr2=0x564f7fc11750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:22.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:22.166+0000 7f46075d7740 1 --2- 172.21.15.5:0/2926278270 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564f7fbdfc70 0x564f7fc11750 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f45f00985d0 tx=0x7f45f00038a0 comp rx=0 tx=0).stop 2024-09-10T10:02:22.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:22.166+0000 7f46075d7740 1 -- 172.21.15.5:0/2926278270 shutdown_connections 2024-09-10T10:02:22.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:22.166+0000 7f46075d7740 1 --2- 172.21.15.5:0/2926278270 >> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] conn(0x564f7fc76ad0 0x564f7fc78ee0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:22.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:22.166+0000 7f46075d7740 1 --2- 172.21.15.5:0/2926278270 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564f7fbeae50 0x564f7fc0f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:22.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:22.166+0000 7f46075d7740 1 --2- 172.21.15.5:0/2926278270 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564f7fbe0590 0x564f7fc0eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:22.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:22.166+0000 7f45df400640 0 client.29899 ms_handle_reset on v2:172.21.15.102:6836/2689776764 2024-09-10T10:02:22.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:22.166+0000 7f46075d7740 1 --2- 172.21.15.5:0/2926278270 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564f7fbdfc70 0x564f7fc11750 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:22.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:22.166+0000 7f46075d7740 1 -- 172.21.15.5:0/2926278270 >> 172.21.15.5:0/2926278270 conn(0x564f7fbdc200 msgr2=0x564f7fbe9d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:22.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:22.167+0000 7f46075d7740 1 -- 172.21.15.5:0/2926278270 shutdown_connections 2024-09-10T10:02:22.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:22.167+0000 7f46075d7740 1 -- 172.21.15.5:0/2926278270 wait complete. 2024-09-10T10:02:22.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:22.167+0000 7f46075d7740 20 client.29899 trim_cache size 0 max 16384 2024-09-10T10:02:22.170 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-10T10:02:22.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:22.168+0000 7f4604a00640 1 -- 172.21.15.5:0/32756320 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f45e40a2d50 msgr2=0x7f45e40a5210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:22.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:22.168+0000 7f4604a00640 1 --2- 172.21.15.5:0/32756320 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f45e40a2d50 0x7f45e40a5210 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x564f7fbdd060 tx=0x7f45ec008040 comp rx=0 tx=0).stop 2024-09-10T10:02:22.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:22.168+0000 7f4604a00640 1 -- 172.21.15.5:0/32756320 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4600109450 msgr2=0x7f46001a7300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:22.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:22.168+0000 7f4604a00640 1 --2- 172.21.15.5:0/32756320 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4600109450 0x7f46001a7300 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f45f000e9c0 tx=0x7f45f000ee80 comp rx=0 tx=0).stop 2024-09-10T10:02:22.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:22.168+0000 7f4604a00640 1 -- 172.21.15.5:0/32756320 shutdown_connections 2024-09-10T10:02:22.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:22.168+0000 7f4604a00640 1 --2- 172.21.15.5:0/32756320 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f45e40a2d50 0x7f45e40a5210 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:22.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:22.168+0000 7f4604a00640 1 --2- 172.21.15.5:0/32756320 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f460011b140 0x7f46001abbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:22.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:22.168+0000 7f4604a00640 1 --2- 172.21.15.5:0/32756320 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4600109830 0x7f46001a7840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:22.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:22.168+0000 7f4604a00640 1 --2- 172.21.15.5:0/32756320 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f4600109450 0x7f46001a7300 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:22.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:22.168+0000 7f4604a00640 1 -- 172.21.15.5:0/32756320 >> 172.21.15.5:0/32756320 conn(0x7f460006e140 msgr2=0x7f460011bdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:22.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:22.168+0000 7f4604a00640 1 -- 172.21.15.5:0/32756320 shutdown_connections 2024-09-10T10:02:22.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:22.168+0000 7f4604a00640 1 -- 172.21.15.5:0/32756320 wait complete. 2024-09-10T10:02:23.069 INFO:tasks.workunit.client.0.smithi005.stdout: 10 156138 13.12 MB/sec execute 481 sec latency 19036.447 ms 2024-09-10T10:02:23.396 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:02:23 smithi005 ceph-mon[30350]: pgmap v1114: 129 pgs: 129 active+clean; 775 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 163 KiB/s rd, 4.1 MiB/s wr, 116 op/s 2024-09-10T10:02:23.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:02:23 smithi159 ceph-mon[35513]: pgmap v1114: 129 pgs: 129 active+clean; 775 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 163 KiB/s rd, 4.1 MiB/s wr, 116 op/s 2024-09-10T10:02:23.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:02:23 smithi102 ceph-mon[35442]: pgmap v1114: 129 pgs: 129 active+clean; 775 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 163 KiB/s rd, 4.1 MiB/s wr, 116 op/s 2024-09-10T10:02:23.660 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-10T10:02:24.069 INFO:tasks.workunit.client.0.smithi005.stdout: 10 156138 13.10 MB/sec execute 482 sec latency 20036.499 ms 2024-09-10T10:02:24.276 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:02:25.069 INFO:tasks.workunit.client.0.smithi005.stdout: 10 156138 13.07 MB/sec execute 483 sec latency 21036.554 ms 2024-09-10T10:02:25.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:25.219+0000 7fd673e00640 1 -- 172.21.15.5:0/2168951948 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd66c104010 msgr2=0x7fd66c1043f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:25.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:25.219+0000 7fd673e00640 1 --2- 172.21.15.5:0/2168951948 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd66c104010 0x7fd66c1043f0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7fd660009900 tx=0x7fd66002a400 comp rx=0 tx=0).stop 2024-09-10T10:02:25.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:25.219+0000 7fd673e00640 1 -- 172.21.15.5:0/2168951948 shutdown_connections 2024-09-10T10:02:25.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:25.219+0000 7fd673e00640 1 --2- 172.21.15.5:0/2168951948 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd66c109140 0x7fd66c110cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:25.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:25.219+0000 7fd673e00640 1 --2- 172.21.15.5:0/2168951948 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd66c1049c0 0x7fd66c108a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:25.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:25.219+0000 7fd673e00640 1 --2- 172.21.15.5:0/2168951948 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd66c104010 0x7fd66c1043f0 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:25.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:25.219+0000 7fd673e00640 1 -- 172.21.15.5:0/2168951948 >> 172.21.15.5:0/2168951948 conn(0x7fd66c0ffa50 msgr2=0x7fd66c101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:25.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:25.219+0000 7fd673e00640 1 -- 172.21.15.5:0/2168951948 shutdown_connections 2024-09-10T10:02:25.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:25.219+0000 7fd673e00640 1 -- 172.21.15.5:0/2168951948 wait complete. 2024-09-10T10:02:25.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:25.225+0000 7fd673e00640 1 Processor -- start 2024-09-10T10:02:25.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:25.225+0000 7fd673e00640 1 -- start start 2024-09-10T10:02:25.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:25.225+0000 7fd673e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd66c104010 0x7fd66c19a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:25.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:25.225+0000 7fd673e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd66c1049c0 0x7fd66c19ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:25.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:25.225+0000 7fd673e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd66c109140 0x7fd66c19ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:25.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:25.225+0000 7fd673e00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd66c1141c0 con 0x7fd66c1049c0 2024-09-10T10:02:25.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:25.225+0000 7fd673e00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fd66c114040 con 0x7fd66c104010 2024-09-10T10:02:25.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:25.225+0000 7fd673e00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fd66c114340 con 0x7fd66c109140 2024-09-10T10:02:25.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:25.225+0000 7fd670a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd66c1049c0 0x7fd66c19ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:25.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:25.225+0000 7fd671400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd66c104010 0x7fd66c19a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:25.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:25.225+0000 7fd670a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd66c1049c0 0x7fd66c19ab80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58206/0 (socket says 172.21.15.5:58206) 2024-09-10T10:02:25.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:25.225+0000 7fd670a00640 1 -- 172.21.15.5:0/4161891463 learned_addr learned my addr 172.21.15.5:0/4161891463 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T10:02:25.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:25.225+0000 7fd671400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd66c104010 0x7fd66c19a640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:60378/0 (socket says 172.21.15.5:60378) 2024-09-10T10:02:25.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:25.225+0000 7fd671e00640 1 --2- 172.21.15.5:0/4161891463 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd66c109140 0x7fd66c19ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:25.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:25.225+0000 7fd671400640 1 -- 172.21.15.5:0/4161891463 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd66c109140 msgr2=0x7fd66c19ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:25.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:25.225+0000 7fd671400640 1 --2- 172.21.15.5:0/4161891463 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd66c109140 0x7fd66c19ef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:25.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:25.225+0000 7fd671400640 1 -- 172.21.15.5:0/4161891463 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd66c1049c0 msgr2=0x7fd66c19ab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:25.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:25.225+0000 7fd671400640 1 --2- 172.21.15.5:0/4161891463 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd66c1049c0 0x7fd66c19ab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:25.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:25.225+0000 7fd671400640 1 -- 172.21.15.5:0/4161891463 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd66c19f5f0 con 0x7fd66c104010 2024-09-10T10:02:25.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:25.225+0000 7fd670a00640 1 --2- 172.21.15.5:0/4161891463 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd66c1049c0 0x7fd66c19ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:02:25.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:25.225+0000 7fd671400640 1 --2- 172.21.15.5:0/4161891463 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd66c104010 0x7fd66c19a640 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7fd66002a920 tx=0x7fd660003b20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:25.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:25.225+0000 7fd66a000640 1 -- 172.21.15.5:0/4161891463 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd66003f070 con 0x7fd66c104010 2024-09-10T10:02:25.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:25.225+0000 7fd66a000640 1 -- 172.21.15.5:0/4161891463 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd660031c90 con 0x7fd66c104010 2024-09-10T10:02:25.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:25.225+0000 7fd66a000640 1 -- 172.21.15.5:0/4161891463 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd660035030 con 0x7fd66c104010 2024-09-10T10:02:25.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:25.225+0000 7fd673e00640 1 -- 172.21.15.5:0/4161891463 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd66c19f880 con 0x7fd66c104010 2024-09-10T10:02:25.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:25.225+0000 7fd673e00640 1 -- 172.21.15.5:0/4161891463 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd66c1a71c0 con 0x7fd66c104010 2024-09-10T10:02:25.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:25.230+0000 7fd676a6a740 1 Processor -- start 2024-09-10T10:02:25.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:25.230+0000 7fd676a6a740 1 -- start start 2024-09-10T10:02:25.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:25.230+0000 7fd676a6a740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56232a7edc70 0x56232a7ee050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:25.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:25.230+0000 7fd676a6a740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56232a7ee590 0x56232a7f8910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:25.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:25.230+0000 7fd676a6a740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56232a7f8e50 0x56232a7fb240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:25.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:25.230+0000 7fd676a6a740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x56232a7543e0 con 0x56232a7f8e50 2024-09-10T10:02:25.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:25.230+0000 7fd676a6a740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x56232a6fbd80 con 0x56232a7ee590 2024-09-10T10:02:25.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:25.230+0000 7fd676a6a740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x56232a7503a0 con 0x56232a7edc70 2024-09-10T10:02:25.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:25.230+0000 7fd66a000640 1 -- 172.21.15.5:0/4161891463 <== mon.2 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7fd660042070 con 0x7fd66c104010 2024-09-10T10:02:25.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:25.230+0000 7fd671e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56232a7edc70 0x56232a7ee050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:25.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:25.230+0000 7fd671e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56232a7edc70 0x56232a7ee050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:51338/0 (socket says 172.21.15.5:51338) 2024-09-10T10:02:25.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:25.230+0000 7fd671e00640 1 -- 172.21.15.5:0/2104844686 learned_addr learned my addr 172.21.15.5:0/2104844686 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T10:02:25.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:25.230+0000 7fd670a00640 1 --2- 172.21.15.5:0/2104844686 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56232a7f8e50 0x56232a7fb240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:25.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:25.230+0000 7fd671400640 1 --2- 172.21.15.5:0/2104844686 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56232a7ee590 0x56232a7f8910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:25.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:25.230+0000 7fd671e00640 1 -- 172.21.15.5:0/2104844686 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56232a7ee590 msgr2=0x56232a7f8910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:25.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:25.230+0000 7fd671e00640 1 --2- 172.21.15.5:0/2104844686 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56232a7ee590 0x56232a7f8910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:25.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:25.230+0000 7fd671e00640 1 -- 172.21.15.5:0/2104844686 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56232a7f8e50 msgr2=0x56232a7fb240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:25.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:25.230+0000 7fd671e00640 1 --2- 172.21.15.5:0/2104844686 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56232a7f8e50 0x56232a7fb240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:25.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:25.230+0000 7fd671e00640 1 -- 172.21.15.5:0/2104844686 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56232a71aae0 con 0x56232a7edc70 2024-09-10T10:02:25.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:25.230+0000 7fd670a00640 1 --2- 172.21.15.5:0/2104844686 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56232a7f8e50 0x56232a7fb240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:02:25.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:25.230+0000 7fd671e00640 1 --2- 172.21.15.5:0/2104844686 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56232a7edc70 0x56232a7ee050 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fd664009f50 tx=0x7fd6640118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e0d21f4282bfc978 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:25.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:25.230+0000 7fd65e600640 1 -- 172.21.15.5:0/2104844686 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd664004b20 con 0x56232a7edc70 2024-09-10T10:02:25.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:25.230+0000 7fd65e600640 1 -- 172.21.15.5:0/2104844686 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd664004cc0 con 0x56232a7edc70 2024-09-10T10:02:25.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:25.230+0000 7fd65e600640 1 -- 172.21.15.5:0/2104844686 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd664005020 con 0x56232a7edc70 2024-09-10T10:02:25.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:25.230+0000 7fd66a000640 1 --2- 172.21.15.5:0/4161891463 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fd6540a2c00 0x7fd6540a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:25.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:25.230+0000 7fd676a6a740 1 -- 172.21.15.5:0/2104844686 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56232a7edc70 msgr2=0x56232a7ee050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:25.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:25.230+0000 7fd676a6a740 1 --2- 172.21.15.5:0/2104844686 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56232a7edc70 0x56232a7ee050 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fd664009f50 tx=0x7fd6640118d0 comp rx=0 tx=0).stop 2024-09-10T10:02:25.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:25.230+0000 7fd66a000640 1 -- 172.21.15.5:0/4161891463 <== mon.2 v2:172.21.15.102:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fd6600efd00 con 0x7fd66c104010 2024-09-10T10:02:25.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:25.230+0000 7fd676a6a740 1 -- 172.21.15.5:0/2104844686 shutdown_connections 2024-09-10T10:02:25.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:25.230+0000 7fd676a6a740 1 --2- 172.21.15.5:0/2104844686 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56232a7f8e50 0x56232a7fb240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:25.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:25.230+0000 7fd676a6a740 1 --2- 172.21.15.5:0/2104844686 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56232a7ee590 0x56232a7f8910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:25.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:25.230+0000 7fd676a6a740 1 --2- 172.21.15.5:0/2104844686 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56232a7edc70 0x56232a7ee050 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:25.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:25.230+0000 7fd671400640 1 --2- 172.21.15.5:0/4161891463 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fd6540a2c00 0x7fd6540a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:25.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:25.230+0000 7fd676a6a740 1 -- 172.21.15.5:0/2104844686 >> 172.21.15.5:0/2104844686 conn(0x56232a7ea200 msgr2=0x56232a7ea5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:25.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:25.230+0000 7fd676a6a740 1 -- 172.21.15.5:0/2104844686 shutdown_connections 2024-09-10T10:02:25.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:25.230+0000 7fd676a6a740 1 -- 172.21.15.5:0/2104844686 wait complete. 2024-09-10T10:02:25.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:25.230+0000 7fd671400640 1 --2- 172.21.15.5:0/4161891463 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fd6540a2c00 0x7fd6540a50c0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fd6600b8b90 tx=0x7fd66003b720 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:25.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:25.230+0000 7fd676a6a740 1 -- start start 2024-09-10T10:02:25.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:25.235+0000 7fd676a6a740 1 Processor -- start 2024-09-10T10:02:25.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:25.235+0000 7fd676a6a740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56232a7edc70 0x56232a7f5d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:25.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:25.235+0000 7fd676a6a740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56232a7ee590 0x56232a7f6260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:25.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:25.235+0000 7fd676a6a740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56232a7f8e50 0x56232a7f67a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:25.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:25.235+0000 7fd676a6a740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x56232a7fba20 con 0x56232a7f8e50 2024-09-10T10:02:25.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:25.235+0000 7fd676a6a740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x56232a70f0f0 con 0x56232a7edc70 2024-09-10T10:02:25.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:25.235+0000 7fd676a6a740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x56232a7fbba0 con 0x56232a7ee590 2024-09-10T10:02:25.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:25.235+0000 7fd676a6a740 10 client.0 _ll_register_callbacks cb 0x56232a6f6bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:02:25.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:25.235+0000 7fd671e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56232a7f8e50 0x56232a7f67a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:25.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:25.235+0000 7fd670a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56232a7ee590 0x56232a7f6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:25.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:25.235+0000 7fd670a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56232a7ee590 0x56232a7f6260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:51354/0 (socket says 172.21.15.5:51354) 2024-09-10T10:02:25.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:25.235+0000 7fd670a00640 1 -- 172.21.15.5:0/1769258095 learned_addr learned my addr 172.21.15.5:0/1769258095 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T10:02:25.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:25.235+0000 7fd671e00640 1 -- 172.21.15.5:0/1769258095 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56232a7ee590 msgr2=0x56232a7f6260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:25.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:25.235+0000 7fd671e00640 1 --2- 172.21.15.5:0/1769258095 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56232a7ee590 0x56232a7f6260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:25.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:25.235+0000 7fd671e00640 1 -- 172.21.15.5:0/1769258095 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56232a7edc70 msgr2=0x56232a7f5d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:25.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:25.235+0000 7fd671e00640 1 --2- 172.21.15.5:0/1769258095 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56232a7edc70 0x56232a7f5d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:25.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:25.235+0000 7fd671e00640 1 -- 172.21.15.5:0/1769258095 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56232a4c53c0 con 0x56232a7f8e50 2024-09-10T10:02:25.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:25.235+0000 7fd670a00640 1 --2- 172.21.15.5:0/1769258095 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56232a7ee590 0x56232a7f6260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:02:25.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:25.235+0000 7fd671e00640 1 --2- 172.21.15.5:0/1769258095 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56232a7f8e50 0x56232a7f67a0 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7fd664009f50 tx=0x7fd664004f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=779c07eaeb294ddd server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:25.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:25.235+0000 7fd652a00640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-10T10:02:25.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:25.237+0000 7fd652a00640 1 -- 172.21.15.5:0/1769258095 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd664028070 con 0x56232a7f8e50 2024-09-10T10:02:25.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:25.237+0000 7fd652a00640 1 -- 172.21.15.5:0/1769258095 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fd664016040 con 0x56232a7f8e50 2024-09-10T10:02:25.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:25.237+0000 7fd676a6a740 1 -- 172.21.15.5:0/1769258095 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd664004cc0 con 0x56232a7f8e50 2024-09-10T10:02:25.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:25.237+0000 7fd652a00640 1 -- 172.21.15.5:0/1769258095 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7fd664004000 con 0x56232a7f8e50 2024-09-10T10:02:25.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:25.237+0000 7fd676a6a740 10 client.19848 fetch_fsmap learned FSMap version 42 2024-09-10T10:02:25.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:25.237+0000 7fd676a6a740 1 -- 172.21.15.5:0/1769258095 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x56232a807a70 con 0x56232a7f8e50 2024-09-10T10:02:25.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:25.237+0000 7fd652a00640 1 -- 172.21.15.5:0/1769258095 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd664004cc0 con 0x56232a7f8e50 2024-09-10T10:02:25.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:25.237+0000 7fd652a00640 1 -- 172.21.15.5:0/1769258095 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 42) ==== 8370+0+0 (secure 0 0 0) 0x7fd66400f510 con 0x56232a7f8e50 2024-09-10T10:02:25.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:25.237+0000 7fd676a6a740 10 client.19848 fetch_fsmap finished waiting for FSMap version 42 2024-09-10T10:02:25.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:25.237+0000 7fd676a6a740 10 client.19848 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-10T10:02:25.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:25.237+0000 7fd676a6a740 20 client.19848 populate_metadata read hostname 'smithi005' 2024-09-10T10:02:25.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:25.237+0000 7fd676a6a740 1 --2- 172.21.15.5:0/1769258095 >> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] conn(0x56232a808e80 0x56232a80b270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:25.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:25.237+0000 7fd676a6a740 4 client.19848 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:02:25.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:25.237+0000 7fd676a6a740 1 -- 172.21.15.5:0/1769258095 --> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56232a761950 con 0x56232a808e80 2024-09-10T10:02:25.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:25.237+0000 7fd670a00640 1 --2- 172.21.15.5:0/1769258095 >> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] conn(0x56232a808e80 0x56232a80b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:25.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:25.237+0000 7fd670a00640 1 --2- 172.21.15.5:0/1769258095 >> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] conn(0x56232a808e80 0x56232a80b270 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=82977af7c3b8838a server_cookie=5d980c57dc46000a in_seq=0 out_seq=0 2024-09-10T10:02:25.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:25.237+0000 7fd652a00640 10 client.19848 ms_handle_connect on v2:172.21.15.159:6836/620627639 2024-09-10T10:02:25.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:25.237+0000 7fd652a00640 1 -- 172.21.15.5:0/1769258095 <== mds.2 v2:172.21.15.159:6836/620627639 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56232a761950 con 0x56232a808e80 2024-09-10T10:02:25.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:25.237+0000 7fd652a00640 10 client.19848 handle_command_reply: tid=0 2024-09-10T10:02:25.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:25.237+0000 7fd676a6a740 1 client.19848 shutdown 2024-09-10T10:02:25.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:25.237+0000 7fd676a6a740 1 -- 172.21.15.5:0/1769258095 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56232a7f8e50 msgr2=0x56232a7f67a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:25.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:25.237+0000 7fd676a6a740 1 --2- 172.21.15.5:0/1769258095 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56232a7f8e50 0x56232a7f67a0 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7fd664009f50 tx=0x7fd664004f30 comp rx=0 tx=0).stop 2024-09-10T10:02:25.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:25.237+0000 7fd676a6a740 1 -- 172.21.15.5:0/1769258095 shutdown_connections 2024-09-10T10:02:25.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:25.237+0000 7fd676a6a740 1 --2- 172.21.15.5:0/1769258095 >> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] conn(0x56232a808e80 0x56232a80b270 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:25.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:25.237+0000 7fd676a6a740 1 --2- 172.21.15.5:0/1769258095 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56232a7f8e50 0x56232a7f67a0 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:25.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:25.237+0000 7fd676a6a740 1 --2- 172.21.15.5:0/1769258095 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56232a7ee590 0x56232a7f6260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:25.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:25.237+0000 7fd676a6a740 1 --2- 172.21.15.5:0/1769258095 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56232a7edc70 0x56232a7f5d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:25.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:25.237+0000 7fd676a6a740 1 -- 172.21.15.5:0/1769258095 >> 172.21.15.5:0/1769258095 conn(0x56232a7ea200 msgr2=0x56232a7efee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:25.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:25.237+0000 7fd652a00640 0 client.19848 ms_handle_reset on v2:172.21.15.159:6836/620627639 2024-09-10T10:02:25.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:25.237+0000 7fd676a6a740 1 -- 172.21.15.5:0/1769258095 shutdown_connections 2024-09-10T10:02:25.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:25.237+0000 7fd676a6a740 1 -- 172.21.15.5:0/1769258095 wait complete. 2024-09-10T10:02:25.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:25.237+0000 7fd676a6a740 20 client.19848 trim_cache size 0 max 16384 2024-09-10T10:02:25.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:25.254+0000 7fd676a6a740 1 Processor -- start 2024-09-10T10:02:25.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:25.254+0000 7fd676a6a740 1 -- start start 2024-09-10T10:02:25.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:25.254+0000 7fd676a6a740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56232a7edc70 0x56232a7f06d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:25.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:25.254+0000 7fd676a6a740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56232a7ee590 0x56232a7f7580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:25.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:25.254+0000 7fd676a6a740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56232a7f8e50 0x56232a810a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:25.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:25.254+0000 7fd676a6a740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd66c1b1090 con 0x56232a7f8e50 2024-09-10T10:02:25.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:25.254+0000 7fd676a6a740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fd66c1b2bd0 con 0x56232a7edc70 2024-09-10T10:02:25.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:25.254+0000 7fd676a6a740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fd66c1b1760 con 0x56232a7ee590 2024-09-10T10:02:25.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:25.254+0000 7fd671e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56232a7edc70 0x56232a7f06d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:25.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:25.254+0000 7fd670a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56232a7ee590 0x56232a7f7580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:25.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:25.254+0000 7fd670a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56232a7ee590 0x56232a7f7580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:51358/0 (socket says 172.21.15.5:51358) 2024-09-10T10:02:25.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:25.254+0000 7fd671e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56232a7edc70 0x56232a7f06d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:60398/0 (socket says 172.21.15.5:60398) 2024-09-10T10:02:25.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:25.254+0000 7fd670a00640 1 -- 172.21.15.5:0/826810796 learned_addr learned my addr 172.21.15.5:0/826810796 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T10:02:25.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:25.254+0000 7fd671e00640 1 --2- 172.21.15.5:0/826810796 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56232a7f8e50 0x56232a810a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:25.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:25.254+0000 7fd670a00640 1 -- 172.21.15.5:0/826810796 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56232a7edc70 msgr2=0x56232a7f06d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:25.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:25.254+0000 7fd670a00640 1 --2- 172.21.15.5:0/826810796 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56232a7edc70 0x56232a7f06d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:25.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:25.254+0000 7fd670a00640 1 -- 172.21.15.5:0/826810796 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56232a7f8e50 msgr2=0x56232a810a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:25.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:25.254+0000 7fd670a00640 1 --2- 172.21.15.5:0/826810796 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56232a7f8e50 0x56232a810a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:25.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:25.254+0000 7fd670a00640 1 -- 172.21.15.5:0/826810796 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56232a7f8430 con 0x56232a7ee590 2024-09-10T10:02:25.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:25.254+0000 7fd670a00640 1 --2- 172.21.15.5:0/826810796 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56232a7ee590 0x56232a7f7580 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7fd658012920 tx=0x7fd658005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=778bf34b2bf5b932 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:25.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:25.254+0000 7fd65dc00640 1 -- 172.21.15.5:0/826810796 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd658004230 con 0x56232a7ee590 2024-09-10T10:02:25.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:25.254+0000 7fd65dc00640 1 -- 172.21.15.5:0/826810796 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd658011db0 con 0x56232a7ee590 2024-09-10T10:02:25.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:25.254+0000 7fd65dc00640 1 -- 172.21.15.5:0/826810796 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd658004a00 con 0x56232a7ee590 2024-09-10T10:02:25.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:25.254+0000 7fd676a6a740 1 -- 172.21.15.5:0/826810796 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56232a7ee590 msgr2=0x56232a7f7580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:25.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:25.254+0000 7fd676a6a740 1 --2- 172.21.15.5:0/826810796 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56232a7ee590 0x56232a7f7580 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7fd658012920 tx=0x7fd658005f90 comp rx=0 tx=0).stop 2024-09-10T10:02:25.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:25.254+0000 7fd676a6a740 1 -- 172.21.15.5:0/826810796 shutdown_connections 2024-09-10T10:02:25.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:25.254+0000 7fd676a6a740 1 --2- 172.21.15.5:0/826810796 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56232a7f8e50 0x56232a810a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:25.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:25.254+0000 7fd676a6a740 1 --2- 172.21.15.5:0/826810796 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56232a7ee590 0x56232a7f7580 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:25.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:25.254+0000 7fd676a6a740 1 --2- 172.21.15.5:0/826810796 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56232a7edc70 0x56232a7f06d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:25.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:25.254+0000 7fd676a6a740 1 -- 172.21.15.5:0/826810796 >> 172.21.15.5:0/826810796 conn(0x56232a7ea200 msgr2=0x56232a7f4ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:25.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:25.254+0000 7fd676a6a740 1 -- 172.21.15.5:0/826810796 shutdown_connections 2024-09-10T10:02:25.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:25.254+0000 7fd676a6a740 1 -- 172.21.15.5:0/826810796 wait complete. 2024-09-10T10:02:25.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:25.254+0000 7fd676a6a740 1 -- start start 2024-09-10T10:02:25.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:25.254+0000 7fd676a6a740 1 Processor -- start 2024-09-10T10:02:25.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:25.254+0000 7fd676a6a740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56232a7edc70 0x56232a86bd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:25.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:25.254+0000 7fd676a6a740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56232a7ee590 0x56232a86c280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:25.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:25.254+0000 7fd676a6a740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56232a7f8e50 0x56232a7fc3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:25.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:25.254+0000 7fd676a6a740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd66c1b2500 con 0x56232a7f8e50 2024-09-10T10:02:25.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:25.254+0000 7fd676a6a740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fd66c1b1e30 con 0x56232a7ee590 2024-09-10T10:02:25.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:25.254+0000 7fd676a6a740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fd66c1af070 con 0x56232a7edc70 2024-09-10T10:02:25.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:25.254+0000 7fd676a6a740 10 client.0 _ll_register_callbacks cb 0x7fd664009e20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:02:25.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:25.254+0000 7fd670a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56232a7ee590 0x56232a86c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:25.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:25.254+0000 7fd670a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56232a7ee590 0x56232a86c280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:60404/0 (socket says 172.21.15.5:60404) 2024-09-10T10:02:25.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:25.254+0000 7fd670a00640 1 -- 172.21.15.5:0/3375483649 learned_addr learned my addr 172.21.15.5:0/3375483649 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T10:02:25.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:25.254+0000 7fd671e00640 1 --2- 172.21.15.5:0/3375483649 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56232a7edc70 0x56232a86bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:25.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:25.254+0000 7fd671e00640 1 --2- 172.21.15.5:0/3375483649 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56232a7f8e50 0x56232a7fc3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:25.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:25.259+0000 7fd670a00640 1 -- 172.21.15.5:0/3375483649 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56232a7edc70 msgr2=0x56232a86bd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:25.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:25.259+0000 7fd670a00640 1 --2- 172.21.15.5:0/3375483649 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56232a7edc70 0x56232a86bd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:25.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:25.259+0000 7fd670a00640 1 -- 172.21.15.5:0/3375483649 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56232a7f8e50 msgr2=0x56232a7fc3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:25.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:25.259+0000 7fd670a00640 1 --2- 172.21.15.5:0/3375483649 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56232a7f8e50 0x56232a7fc3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:25.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:25.259+0000 7fd670a00640 1 -- 172.21.15.5:0/3375483649 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56232a7fca90 con 0x56232a7ee590 2024-09-10T10:02:25.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:25.259+0000 7fd671e00640 1 --2- 172.21.15.5:0/3375483649 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56232a7f8e50 0x56232a7fc3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:02:25.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:25.259+0000 7fd670a00640 1 --2- 172.21.15.5:0/3375483649 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56232a7ee590 0x56232a86c280 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7fd658014010 tx=0x7fd658012070 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2887850c904287fc server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:25.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:25.259+0000 7fd652a00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-09-10T10:02:25.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:25.259+0000 7fd652a00640 1 -- 172.21.15.5:0/3375483649 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd658005230 con 0x56232a7ee590 2024-09-10T10:02:25.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:25.259+0000 7fd676a6a740 1 -- 172.21.15.5:0/3375483649 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd658011db0 con 0x56232a7ee590 2024-09-10T10:02:25.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:25.259+0000 7fd652a00640 1 -- 172.21.15.5:0/3375483649 <== mon.2 v2:172.21.15.102:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fd6580118d0 con 0x56232a7ee590 2024-09-10T10:02:25.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:25.259+0000 7fd652a00640 1 -- 172.21.15.5:0/3375483649 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7fd658011ad0 con 0x56232a7ee590 2024-09-10T10:02:25.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:25.259+0000 7fd676a6a740 10 client.29948 fetch_fsmap learned FSMap version 42 2024-09-10T10:02:25.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:25.259+0000 7fd676a6a740 1 -- 172.21.15.5:0/3375483649 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x56232a873510 con 0x56232a7ee590 2024-09-10T10:02:25.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:25.259+0000 7fd652a00640 1 -- 172.21.15.5:0/3375483649 <== mon.2 v2:172.21.15.102:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd658011db0 con 0x56232a7ee590 2024-09-10T10:02:25.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:25.261+0000 7fd652a00640 1 -- 172.21.15.5:0/3375483649 <== mon.2 v2:172.21.15.102:3300/0 5 ==== fsmap(e 42) ==== 8370+0+0 (secure 0 0 0) 0x7fd658004cc0 con 0x56232a7ee590 2024-09-10T10:02:25.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:25.261+0000 7fd676a6a740 10 client.29948 fetch_fsmap finished waiting for FSMap version 42 2024-09-10T10:02:25.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:25.261+0000 7fd676a6a740 10 client.29948 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-10T10:02:25.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:25.261+0000 7fd676a6a740 20 client.29948 populate_metadata read hostname 'smithi005' 2024-09-10T10:02:25.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:25.261+0000 7fd676a6a740 1 --2- 172.21.15.5:0/3375483649 >> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] conn(0x56232a873c40 0x56232a876060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:25.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:25.261+0000 7fd676a6a740 4 client.29948 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T10:02:25.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:25.261+0000 7fd676a6a740 1 -- 172.21.15.5:0/3375483649 --> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56232a75be60 con 0x56232a873c40 2024-09-10T10:02:25.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:25.261+0000 7fd671e00640 1 --2- 172.21.15.5:0/3375483649 >> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] conn(0x56232a873c40 0x56232a876060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:25.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:25.261+0000 7fd671e00640 1 --2- 172.21.15.5:0/3375483649 >> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] conn(0x56232a873c40 0x56232a876060 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7305290de381c646 server_cookie=593d474fa735ef43 in_seq=0 out_seq=0 2024-09-10T10:02:25.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:25.261+0000 7fd652a00640 10 client.29948 ms_handle_connect on v2:172.21.15.159:6836/620627639 2024-09-10T10:02:25.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:25.261+0000 7fd652a00640 1 -- 172.21.15.5:0/3375483649 <== mds.2 v2:172.21.15.159:6836/620627639 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56232a75be60 con 0x56232a873c40 2024-09-10T10:02:25.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:25.261+0000 7fd652a00640 10 client.29948 handle_command_reply: tid=0 2024-09-10T10:02:25.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:25.261+0000 7fd676a6a740 1 client.29948 shutdown 2024-09-10T10:02:25.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:25.261+0000 7fd676a6a740 1 -- 172.21.15.5:0/3375483649 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56232a7ee590 msgr2=0x56232a86c280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:25.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:25.261+0000 7fd676a6a740 1 --2- 172.21.15.5:0/3375483649 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56232a7ee590 0x56232a86c280 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7fd658014010 tx=0x7fd658012070 comp rx=0 tx=0).stop 2024-09-10T10:02:25.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:25.261+0000 7fd676a6a740 1 -- 172.21.15.5:0/3375483649 shutdown_connections 2024-09-10T10:02:25.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:25.261+0000 7fd676a6a740 1 --2- 172.21.15.5:0/3375483649 >> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] conn(0x56232a873c40 0x56232a876060 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:25.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:25.261+0000 7fd676a6a740 1 --2- 172.21.15.5:0/3375483649 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56232a7f8e50 0x56232a7fc3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:25.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:25.261+0000 7fd676a6a740 1 --2- 172.21.15.5:0/3375483649 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56232a7ee590 0x56232a86c280 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:25.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:25.261+0000 7fd652a00640 0 client.29948 ms_handle_reset on v2:172.21.15.159:6836/620627639 2024-09-10T10:02:25.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:25.261+0000 7fd676a6a740 1 --2- 172.21.15.5:0/3375483649 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56232a7edc70 0x56232a86bd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:25.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:25.261+0000 7fd676a6a740 1 -- 172.21.15.5:0/3375483649 >> 172.21.15.5:0/3375483649 conn(0x56232a7ea200 msgr2=0x56232a7e8b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:25.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:25.261+0000 7fd676a6a740 1 -- 172.21.15.5:0/3375483649 shutdown_connections 2024-09-10T10:02:25.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:25.261+0000 7fd676a6a740 1 -- 172.21.15.5:0/3375483649 wait complete. 2024-09-10T10:02:25.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:25.261+0000 7fd676a6a740 20 client.29948 trim_cache size 0 max 16384 2024-09-10T10:02:25.268 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-10T10:02:25.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:25.261+0000 7fd673e00640 1 -- 172.21.15.5:0/4161891463 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fd6540a2c00 msgr2=0x7fd6540a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:25.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:25.261+0000 7fd673e00640 1 --2- 172.21.15.5:0/4161891463 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fd6540a2c00 0x7fd6540a50c0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fd6600b8b90 tx=0x7fd66003b720 comp rx=0 tx=0).stop 2024-09-10T10:02:25.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:25.261+0000 7fd673e00640 1 -- 172.21.15.5:0/4161891463 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd66c104010 msgr2=0x7fd66c19a640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:25.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:25.261+0000 7fd673e00640 1 --2- 172.21.15.5:0/4161891463 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd66c104010 0x7fd66c19a640 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7fd66002a920 tx=0x7fd660003b20 comp rx=0 tx=0).stop 2024-09-10T10:02:25.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:25.261+0000 7fd673e00640 1 -- 172.21.15.5:0/4161891463 shutdown_connections 2024-09-10T10:02:25.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:25.261+0000 7fd673e00640 1 --2- 172.21.15.5:0/4161891463 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fd6540a2c00 0x7fd6540a50c0 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:25.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:25.261+0000 7fd673e00640 1 --2- 172.21.15.5:0/4161891463 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd66c109140 0x7fd66c19ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:25.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:25.261+0000 7fd673e00640 1 --2- 172.21.15.5:0/4161891463 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd66c1049c0 0x7fd66c19ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:25.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:25.261+0000 7fd673e00640 1 --2- 172.21.15.5:0/4161891463 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd66c104010 0x7fd66c19a640 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:25.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:25.261+0000 7fd673e00640 1 -- 172.21.15.5:0/4161891463 >> 172.21.15.5:0/4161891463 conn(0x7fd66c0ffa50 msgr2=0x7fd66c100dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:25.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:25.261+0000 7fd673e00640 1 -- 172.21.15.5:0/4161891463 shutdown_connections 2024-09-10T10:02:25.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:25.261+0000 7fd673e00640 1 -- 172.21.15.5:0/4161891463 wait complete. 2024-09-10T10:02:25.368 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:02:25 smithi005 ceph-mon[30350]: pgmap v1115: 129 pgs: 129 active+clean; 760 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 3.9 MiB/s wr, 131 op/s 2024-09-10T10:02:25.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:02:25 smithi159 ceph-mon[35513]: pgmap v1115: 129 pgs: 129 active+clean; 760 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 3.9 MiB/s wr, 131 op/s 2024-09-10T10:02:25.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:02:25 smithi102 ceph-mon[35442]: pgmap v1115: 129 pgs: 129 active+clean; 760 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 3.9 MiB/s wr, 131 op/s 2024-09-10T10:02:26.069 INFO:tasks.workunit.client.0.smithi005.stdout: 10 156140 13.04 MB/sec execute 484 sec latency 21972.137 ms 2024-09-10T10:02:26.456 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-10T10:02:26.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:26.574+0000 7fa81d600640 1 -- 172.21.15.5:0/3619685148 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x555803922c60 con 0x555803a358b0 2024-09-10T10:02:26.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:26.574+0000 7fa81d600640 10 client.29915 handle_command_reply: tid=0 2024-09-10T10:02:26.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:26.574+0000 7fa839837740 1 client.29915 shutdown 2024-09-10T10:02:26.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:26.575+0000 7fa839837740 1 -- 172.21.15.5:0/3619685148 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5558039af560 msgr2=0x5558039c26e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:26.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:26.575+0000 7fa839837740 1 --2- 172.21.15.5:0/3619685148 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5558039af560 0x5558039c26e0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7fa828009620 tx=0x7fa82802fe70 comp rx=0 tx=0).stop 2024-09-10T10:02:26.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:26.575+0000 7fa839837740 1 -- 172.21.15.5:0/3619685148 shutdown_connections 2024-09-10T10:02:26.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:26.575+0000 7fa839837740 1 --2- 172.21.15.5:0/3619685148 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x555803a358b0 0x555803a37ca0 crc :-1 s=READY pgs=322 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:26.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:26.575+0000 7fa839837740 1 --2- 172.21.15.5:0/3619685148 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5558039ba740 0x5558039bf6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:26.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:26.575+0000 7fa839837740 1 --2- 172.21.15.5:0/3619685148 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5558039afe80 0x5558039c2c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:26.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:26.575+0000 7fa839837740 1 --2- 172.21.15.5:0/3619685148 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5558039af560 0x5558039c26e0 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:26.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:26.575+0000 7fa839837740 1 -- 172.21.15.5:0/3619685148 >> 172.21.15.5:0/3619685148 conn(0x5558039abaf0 msgr2=0x5558039aa450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:26.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:26.575+0000 7fa81d600640 0 client.29915 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T10:02:26.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:26.576+0000 7fa839837740 1 -- 172.21.15.5:0/3619685148 shutdown_connections 2024-09-10T10:02:26.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:26.576+0000 7fa839837740 1 -- 172.21.15.5:0/3619685148 wait complete. 2024-09-10T10:02:26.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:26.576+0000 7fa839837740 20 client.29915 trim_cache size 0 max 16384 2024-09-10T10:02:26.578 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-10T10:02:26.578 INFO:teuthology.orchestra.run.smithi005.stdout: "epoch": 42, 2024-09-10T10:02:26.578 INFO:teuthology.orchestra.run.smithi005.stdout: "leader": 24542, 2024-09-10T10:02:26.578 INFO:teuthology.orchestra.run.smithi005.stdout: "set_version": 82, 2024-09-10T10:02:26.578 INFO:teuthology.orchestra.run.smithi005.stdout: "sets": { 2024-09-10T10:02:26.579 INFO:teuthology.orchestra.run.smithi005.stdout: "b7188858": { 2024-09-10T10:02:26.579 INFO:teuthology.orchestra.run.smithi005.stdout: "version": 82, 2024-09-10T10:02:26.579 INFO:teuthology.orchestra.run.smithi005.stdout: "age_ref": 0.0, 2024-09-10T10:02:26.579 INFO:teuthology.orchestra.run.smithi005.stdout: "state": { 2024-09-10T10:02:26.579 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "QUIESCED", 2024-09-10T10:02:26.579 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0 2024-09-10T10:02:26.579 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:02:26.579 INFO:teuthology.orchestra.run.smithi005.stdout: "timeout": 90.0, 2024-09-10T10:02:26.579 INFO:teuthology.orchestra.run.smithi005.stdout: "expiration": 135.3, 2024-09-10T10:02:26.579 INFO:teuthology.orchestra.run.smithi005.stdout: "members": { 2024-09-10T10:02:26.579 INFO:teuthology.orchestra.run.smithi005.stdout: "file:/": { 2024-09-10T10:02:26.579 INFO:teuthology.orchestra.run.smithi005.stdout: "excluded": false, 2024-09-10T10:02:26.579 INFO:teuthology.orchestra.run.smithi005.stdout: "state": { 2024-09-10T10:02:26.579 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "QUIESCED", 2024-09-10T10:02:26.580 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0 2024-09-10T10:02:26.580 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T10:02:26.580 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T10:02:26.580 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T10:02:26.580 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T10:02:26.580 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T10:02:26.580 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-10T10:02:26.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:26.577+0000 7fa835600640 1 -- 172.21.15.5:0/3301664983 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fa8140a2c00 msgr2=0x7fa8140a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:26.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:26.577+0000 7fa835600640 1 --2- 172.21.15.5:0/3301664983 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fa8140a2c00 0x7fa8140a50c0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fa820098750 tx=0x7fa820018420 comp rx=0 tx=0).stop 2024-09-10T10:02:26.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:26.577+0000 7fa835600640 1 -- 172.21.15.5:0/3301664983 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa830074620 msgr2=0x7fa83010db80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:26.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:26.577+0000 7fa835600640 1 --2- 172.21.15.5:0/3301664983 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa830074620 0x7fa83010db80 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7fa82000e9f0 tx=0x7fa82000eeb0 comp rx=0 tx=0).stop 2024-09-10T10:02:26.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:26.578+0000 7fa835600640 1 -- 172.21.15.5:0/3301664983 shutdown_connections 2024-09-10T10:02:26.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:26.578+0000 7fa835600640 1 --2- 172.21.15.5:0/3301664983 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fa8140a2c00 0x7fa8140a50c0 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:26.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:26.578+0000 7fa835600640 1 --2- 172.21.15.5:0/3301664983 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa83010e210 0x7fa83010b190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:26.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:26.578+0000 7fa835600640 1 --2- 172.21.15.5:0/3301664983 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa830074f40 0x7fa83010ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:26.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:26.578+0000 7fa835600640 1 --2- 172.21.15.5:0/3301664983 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa830074620 0x7fa83010db80 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:26.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:26.578+0000 7fa835600640 1 -- 172.21.15.5:0/3301664983 >> 172.21.15.5:0/3301664983 conn(0x7fa8300fd710 msgr2=0x7fa8300fe3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:26.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:26.579+0000 7fa835600640 1 -- 172.21.15.5:0/3301664983 shutdown_connections 2024-09-10T10:02:26.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:26.580+0000 7fa835600640 1 -- 172.21.15.5:0/3301664983 wait complete. 2024-09-10T10:02:27.069 INFO:tasks.workunit.client.0.smithi005.stdout: 10 156147 13.02 MB/sec execute 485 sec latency 18737.447 ms 2024-09-10T10:02:27.327 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'b7188858', quiesce took 12.5 seconds. Will release after: 2.8000000000000007 2024-09-10T10:02:27.352 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:02:27.491 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:02:27 smithi005 ceph-mon[30350]: pgmap v1116: 129 pgs: 129 active+clean; 742 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 282 KiB/s rd, 3.7 MiB/s wr, 157 op/s 2024-09-10T10:02:27.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:02:27 smithi159 ceph-mon[35513]: pgmap v1116: 129 pgs: 129 active+clean; 742 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 282 KiB/s rd, 3.7 MiB/s wr, 157 op/s 2024-09-10T10:02:27.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:02:27 smithi102 ceph-mon[35442]: pgmap v1116: 129 pgs: 129 active+clean; 742 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 282 KiB/s rd, 3.7 MiB/s wr, 157 op/s 2024-09-10T10:02:27.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:27.905+0000 7f642ae00640 1 -- 172.21.15.5:0/2572181211 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6424077040 msgr2=0x7f64241124f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:27.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:27.905+0000 7f642ae00640 1 --2- 172.21.15.5:0/2572181211 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6424077040 0x7f64241124f0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f6410009900 tx=0x7f641002a410 comp rx=0 tx=0).stop 2024-09-10T10:02:27.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:27.905+0000 7f642ae00640 1 -- 172.21.15.5:0/2572181211 shutdown_connections 2024-09-10T10:02:27.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:27.905+0000 7f642ae00640 1 --2- 172.21.15.5:0/2572181211 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6424112a30 0x7f6424114e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:27.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:27.905+0000 7f642ae00640 1 --2- 172.21.15.5:0/2572181211 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6424077040 0x7f64241124f0 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:27.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:27.905+0000 7f642ae00640 1 --2- 172.21.15.5:0/2572181211 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6424076720 0x7f6424076b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:27.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:27.905+0000 7f642ae00640 1 -- 172.21.15.5:0/2572181211 >> 172.21.15.5:0/2572181211 conn(0x7f64240ffa90 msgr2=0x7f6424101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:27.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:27.905+0000 7f642ae00640 1 -- 172.21.15.5:0/2572181211 shutdown_connections 2024-09-10T10:02:27.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:27.905+0000 7f642ae00640 1 -- 172.21.15.5:0/2572181211 wait complete. 2024-09-10T10:02:27.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:27.906+0000 7f642ae00640 1 Processor -- start 2024-09-10T10:02:27.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:27.906+0000 7f642ae00640 1 -- start start 2024-09-10T10:02:27.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:27.907+0000 7f642ae00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6424076720 0x7f64241a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:27.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:27.907+0000 7f642ae00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6424077040 0x7f64241a4c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:27.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:27.907+0000 7f642ae00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6424112a30 0x7f642419e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:27.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:27.907+0000 7f642ae00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f64241185a0 con 0x7f6424076720 2024-09-10T10:02:27.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:27.907+0000 7f642ae00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f6424118420 con 0x7f6424112a30 2024-09-10T10:02:27.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:27.907+0000 7f642ae00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f6424118720 con 0x7f6424077040 2024-09-10T10:02:27.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:27.907+0000 7f6423e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6424076720 0x7f64241a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:27.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:27.907+0000 7f6428e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6424112a30 0x7f642419e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:27.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:27.907+0000 7f6423e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6424076720 0x7f64241a46f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58246/0 (socket says 172.21.15.5:58246) 2024-09-10T10:02:27.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:27.907+0000 7f6423e00640 1 -- 172.21.15.5:0/2990217425 learned_addr learned my addr 172.21.15.5:0/2990217425 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T10:02:27.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:27.907+0000 7f6423400640 1 --2- 172.21.15.5:0/2990217425 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6424077040 0x7f64241a4c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:27.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:27.907+0000 7f6423e00640 1 -- 172.21.15.5:0/2990217425 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6424077040 msgr2=0x7f64241a4c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:27.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:27.907+0000 7f6423e00640 1 --2- 172.21.15.5:0/2990217425 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6424077040 0x7f64241a4c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:27.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:27.907+0000 7f6423e00640 1 -- 172.21.15.5:0/2990217425 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6424112a30 msgr2=0x7f642419e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:27.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:27.907+0000 7f6423e00640 1 --2- 172.21.15.5:0/2990217425 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6424112a30 0x7f642419e8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:27.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:27.907+0000 7f6423e00640 1 -- 172.21.15.5:0/2990217425 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f642419efe0 con 0x7f6424076720 2024-09-10T10:02:27.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:27.908+0000 7f6423e00640 1 --2- 172.21.15.5:0/2990217425 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6424076720 0x7f64241a46f0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f641400d950 tx=0x7f641400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:27.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:27.908+0000 7f6420c00640 1 -- 172.21.15.5:0/2990217425 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6414017030 con 0x7f6424076720 2024-09-10T10:02:27.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:27.908+0000 7f6420c00640 1 -- 172.21.15.5:0/2990217425 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6414004020 con 0x7f6424076720 2024-09-10T10:02:27.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:27.908+0000 7f642ae00640 1 -- 172.21.15.5:0/2990217425 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f642419f2d0 con 0x7f6424076720 2024-09-10T10:02:27.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:27.908+0000 7f6420c00640 1 -- 172.21.15.5:0/2990217425 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64140047a0 con 0x7f6424076720 2024-09-10T10:02:27.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:27.908+0000 7f642ae00640 1 -- 172.21.15.5:0/2990217425 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64241ab5c0 con 0x7f6424076720 2024-09-10T10:02:27.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:27.911+0000 7f642da68740 1 Processor -- start 2024-09-10T10:02:27.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:27.911+0000 7f6420c00640 1 -- 172.21.15.5:0/2990217425 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f64140041d0 con 0x7f6424076720 2024-09-10T10:02:27.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:27.911+0000 7f642da68740 1 -- start start 2024-09-10T10:02:27.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:27.912+0000 7f642da68740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560cd9351c70 0x560cd9352050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:27.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:27.912+0000 7f642da68740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560cd9352590 0x560cd935c910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:27.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:27.912+0000 7f642da68740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560cd935ce50 0x560cd935f240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:27.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:27.912+0000 7f642da68740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x560cd92b83e0 con 0x560cd9351c70 2024-09-10T10:02:27.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:27.912+0000 7f642da68740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x560cd925fd80 con 0x560cd935ce50 2024-09-10T10:02:27.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:27.912+0000 7f642da68740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x560cd92b43a0 con 0x560cd9352590 2024-09-10T10:02:27.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:27.912+0000 7f6423e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560cd9352590 0x560cd935c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:27.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:27.912+0000 7f6423e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560cd9352590 0x560cd935c910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:51414/0 (socket says 172.21.15.5:51414) 2024-09-10T10:02:27.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:27.912+0000 7f6423e00640 1 -- 172.21.15.5:0/248718620 learned_addr learned my addr 172.21.15.5:0/248718620 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T10:02:27.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:27.912+0000 7f6423400640 1 --2- 172.21.15.5:0/248718620 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560cd935ce50 0x560cd935f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:27.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:27.912+0000 7f6428e00640 1 --2- 172.21.15.5:0/248718620 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560cd9351c70 0x560cd9352050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:27.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:27.913+0000 7f6423400640 1 -- 172.21.15.5:0/248718620 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560cd9352590 msgr2=0x560cd935c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:27.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:27.913+0000 7f6423400640 1 --2- 172.21.15.5:0/248718620 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560cd9352590 0x560cd935c910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:27.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:27.913+0000 7f6423400640 1 -- 172.21.15.5:0/248718620 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560cd9351c70 msgr2=0x560cd9352050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:27.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:27.913+0000 7f6423400640 1 --2- 172.21.15.5:0/248718620 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560cd9351c70 0x560cd9352050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:27.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:27.913+0000 7f6423400640 1 -- 172.21.15.5:0/248718620 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560cd927eae0 con 0x560cd935ce50 2024-09-10T10:02:27.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:27.913+0000 7f6423e00640 1 --2- 172.21.15.5:0/248718620 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560cd9352590 0x560cd935c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:02:27.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:27.913+0000 7f6428e00640 1 --2- 172.21.15.5:0/248718620 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560cd9351c70 0x560cd9352050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:02:27.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:27.913+0000 7f6423400640 1 --2- 172.21.15.5:0/248718620 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560cd935ce50 0x560cd935f240 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f6410005940 tx=0x7f6410005e80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c3589bde4f630109 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:27.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:27.913+0000 7f6420c00640 1 --2- 172.21.15.5:0/2990217425 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f64080a2b80 0x7f64080a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:27.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:27.913+0000 7f6420c00640 1 -- 172.21.15.5:0/2990217425 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f64140cedd0 con 0x7f6424076720 2024-09-10T10:02:27.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:27.913+0000 7f6419600640 1 -- 172.21.15.5:0/248718620 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6410040070 con 0x560cd935ce50 2024-09-10T10:02:27.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:27.913+0000 7f6428e00640 1 --2- 172.21.15.5:0/2990217425 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f64080a2b80 0x7f64080a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:27.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:27.914+0000 7f6419600640 1 -- 172.21.15.5:0/248718620 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f64100319f0 con 0x560cd935ce50 2024-09-10T10:02:27.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:27.914+0000 7f6419600640 1 -- 172.21.15.5:0/248718620 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6410035040 con 0x560cd935ce50 2024-09-10T10:02:27.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:27.914+0000 7f642da68740 1 -- 172.21.15.5:0/248718620 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560cd935ce50 msgr2=0x560cd935f240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:27.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:27.914+0000 7f6428e00640 1 --2- 172.21.15.5:0/2990217425 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f64080a2b80 0x7f64080a5040 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f642406b460 tx=0x7f641c009600 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:27.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:27.914+0000 7f642da68740 1 --2- 172.21.15.5:0/248718620 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560cd935ce50 0x560cd935f240 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f6410005940 tx=0x7f6410005e80 comp rx=0 tx=0).stop 2024-09-10T10:02:27.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:27.914+0000 7f642da68740 1 -- 172.21.15.5:0/248718620 shutdown_connections 2024-09-10T10:02:27.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:27.914+0000 7f642da68740 1 --2- 172.21.15.5:0/248718620 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560cd935ce50 0x560cd935f240 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:27.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:27.914+0000 7f642da68740 1 --2- 172.21.15.5:0/248718620 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560cd9352590 0x560cd935c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:27.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:27.914+0000 7f642da68740 1 --2- 172.21.15.5:0/248718620 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560cd9351c70 0x560cd9352050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:27.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:27.914+0000 7f642da68740 1 -- 172.21.15.5:0/248718620 >> 172.21.15.5:0/248718620 conn(0x560cd934e200 msgr2=0x560cd934e5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:27.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:27.915+0000 7f642da68740 1 -- 172.21.15.5:0/248718620 shutdown_connections 2024-09-10T10:02:27.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:27.915+0000 7f642da68740 1 -- 172.21.15.5:0/248718620 wait complete. 2024-09-10T10:02:27.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:27.915+0000 7f642da68740 1 -- start start 2024-09-10T10:02:27.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:27.915+0000 7f642da68740 1 Processor -- start 2024-09-10T10:02:27.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:27.916+0000 7f642da68740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560cd9351c70 0x560cd9359cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:27.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:27.916+0000 7f642da68740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560cd9352590 0x560cd935a230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:27.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:27.916+0000 7f642da68740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560cd935ce50 0x560cd935a770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:27.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:27.916+0000 7f642da68740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x560cd935fa20 con 0x560cd935ce50 2024-09-10T10:02:27.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:27.916+0000 7f642da68740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x560cd92730f0 con 0x560cd9352590 2024-09-10T10:02:27.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:27.916+0000 7f642da68740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x560cd935fba0 con 0x560cd9351c70 2024-09-10T10:02:27.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:27.916+0000 7f642da68740 10 client.0 _ll_register_callbacks cb 0x560cd925abd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:02:27.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:27.916+0000 7f6423e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560cd9351c70 0x560cd9359cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:27.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:27.916+0000 7f6428e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560cd935ce50 0x560cd935a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:27.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:27.916+0000 7f6423e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560cd9351c70 0x560cd9359cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:51428/0 (socket says 172.21.15.5:51428) 2024-09-10T10:02:27.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:27.916+0000 7f6423e00640 1 -- 172.21.15.5:0/2674473372 learned_addr learned my addr 172.21.15.5:0/2674473372 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T10:02:27.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:27.916+0000 7f6423400640 1 --2- 172.21.15.5:0/2674473372 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560cd9352590 0x560cd935a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:27.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:27.917+0000 7f6423e00640 1 -- 172.21.15.5:0/2674473372 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560cd9352590 msgr2=0x560cd935a230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:27.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:27.917+0000 7f6423e00640 1 --2- 172.21.15.5:0/2674473372 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560cd9352590 0x560cd935a230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:27.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:27.917+0000 7f6423e00640 1 -- 172.21.15.5:0/2674473372 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560cd935ce50 msgr2=0x560cd935a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:27.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:27.917+0000 7f6423e00640 1 --2- 172.21.15.5:0/2674473372 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560cd935ce50 0x560cd935a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:27.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:27.917+0000 7f6423e00640 1 -- 172.21.15.5:0/2674473372 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560cd90293c0 con 0x560cd9351c70 2024-09-10T10:02:27.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:27.917+0000 7f6423e00640 1 --2- 172.21.15.5:0/2674473372 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560cd9351c70 0x560cd9359cf0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f64140cf070 tx=0x7f6414004540 comp rx=0 tx=0).ready entity=mon.1 client_cookie=61879d6896c1a5fe server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:27.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:27.917+0000 7f640e000640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-09-10T10:02:27.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:27.917+0000 7f640e000640 1 -- 172.21.15.5:0/2674473372 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f641409ad20 con 0x560cd9351c70 2024-09-10T10:02:27.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:27.917+0000 7f642da68740 1 -- 172.21.15.5:0/2674473372 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f64100319f0 con 0x560cd9351c70 2024-09-10T10:02:27.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:27.917+0000 7f640e000640 1 -- 172.21.15.5:0/2674473372 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f64140a39c0 con 0x560cd9351c70 2024-09-10T10:02:27.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:27.918+0000 7f640e000640 1 -- 172.21.15.5:0/2674473372 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f64140b4050 con 0x560cd9351c70 2024-09-10T10:02:27.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:27.918+0000 7f642da68740 10 client.29929 fetch_fsmap learned FSMap version 42 2024-09-10T10:02:27.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:27.918+0000 7f642da68740 1 -- 172.21.15.5:0/2674473372 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x560cd936ba60 con 0x560cd9351c70 2024-09-10T10:02:27.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:27.918+0000 7f640e000640 1 -- 172.21.15.5:0/2674473372 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f64100319f0 con 0x560cd9351c70 2024-09-10T10:02:27.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:27.918+0000 7f640e000640 1 -- 172.21.15.5:0/2674473372 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 42) ==== 8370+0+0 (secure 0 0 0) 0x7f641409f110 con 0x560cd9351c70 2024-09-10T10:02:27.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:27.918+0000 7f642da68740 10 client.29929 fetch_fsmap finished waiting for FSMap version 42 2024-09-10T10:02:27.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:27.918+0000 7f642da68740 10 client.29929 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-10T10:02:27.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:27.918+0000 7f642da68740 20 client.29929 populate_metadata read hostname 'smithi005' 2024-09-10T10:02:27.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:27.918+0000 7f642da68740 1 --2- 172.21.15.5:0/2674473372 >> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] conn(0x560cd936ce70 0x560cd936f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:27.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:27.918+0000 7f642da68740 4 client.29929 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:02:27.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:27.918+0000 7f642da68740 1 -- 172.21.15.5:0/2674473372 --> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560cd92c5950 con 0x560cd936ce70 2024-09-10T10:02:27.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:27.919+0000 7f6423400640 1 --2- 172.21.15.5:0/2674473372 >> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] conn(0x560cd936ce70 0x560cd936f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:27.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:27.919+0000 7f6423400640 1 --2- 172.21.15.5:0/2674473372 >> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] conn(0x560cd936ce70 0x560cd936f2c0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7810ef79df726aab server_cookie=c31db47119ef544b in_seq=0 out_seq=0 2024-09-10T10:02:27.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:27.919+0000 7f640e000640 10 client.29929 ms_handle_connect on v2:172.21.15.159:6838/2073739599 2024-09-10T10:02:27.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:27.920+0000 7f640e000640 1 -- 172.21.15.5:0/2674473372 <== mds.1 v2:172.21.15.159:6838/2073739599 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560cd92c5950 con 0x560cd936ce70 2024-09-10T10:02:27.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:27.920+0000 7f640e000640 10 client.29929 handle_command_reply: tid=0 2024-09-10T10:02:27.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:27.920+0000 7f642da68740 1 client.29929 shutdown 2024-09-10T10:02:27.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:27.920+0000 7f642da68740 1 -- 172.21.15.5:0/2674473372 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560cd9351c70 msgr2=0x560cd9359cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:27.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:27.920+0000 7f642da68740 1 --2- 172.21.15.5:0/2674473372 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560cd9351c70 0x560cd9359cf0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f64140cf070 tx=0x7f6414004540 comp rx=0 tx=0).stop 2024-09-10T10:02:27.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:27.920+0000 7f642da68740 1 -- 172.21.15.5:0/2674473372 shutdown_connections 2024-09-10T10:02:27.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:27.920+0000 7f642da68740 1 --2- 172.21.15.5:0/2674473372 >> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] conn(0x560cd936ce70 0x560cd936f2c0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:27.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:27.920+0000 7f642da68740 1 --2- 172.21.15.5:0/2674473372 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560cd935ce50 0x560cd935a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:27.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:27.920+0000 7f642da68740 1 --2- 172.21.15.5:0/2674473372 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560cd9352590 0x560cd935a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:27.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:27.920+0000 7f642da68740 1 --2- 172.21.15.5:0/2674473372 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560cd9351c70 0x560cd9359cf0 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:27.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:27.920+0000 7f640e000640 0 client.29929 ms_handle_reset on v2:172.21.15.159:6838/2073739599 2024-09-10T10:02:27.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:27.920+0000 7f642da68740 1 -- 172.21.15.5:0/2674473372 >> 172.21.15.5:0/2674473372 conn(0x560cd934e200 msgr2=0x560cd9353f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:27.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:27.920+0000 7f642da68740 1 -- 172.21.15.5:0/2674473372 shutdown_connections 2024-09-10T10:02:27.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:27.920+0000 7f642da68740 1 -- 172.21.15.5:0/2674473372 wait complete. 2024-09-10T10:02:27.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:27.921+0000 7f642da68740 20 client.29929 trim_cache size 0 max 16384 2024-09-10T10:02:27.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:27.934+0000 7f642da68740 1 Processor -- start 2024-09-10T10:02:27.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:27.934+0000 7f642da68740 1 -- start start 2024-09-10T10:02:27.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:27.935+0000 7f642da68740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560cd9351c70 0x560cd935bc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:27.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:27.935+0000 7f642da68740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560cd9352590 0x560cd9374b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:27.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:27.935+0000 7f642da68740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560cd935ce50 0x560cd93750d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:27.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:27.935+0000 7f642da68740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f64241b54b0 con 0x560cd9352590 2024-09-10T10:02:27.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:27.935+0000 7f642da68740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f64241b6ff0 con 0x560cd935ce50 2024-09-10T10:02:27.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:27.935+0000 7f642da68740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f64241b5b80 con 0x560cd9351c70 2024-09-10T10:02:27.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:27.935+0000 7f6423400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560cd9352590 0x560cd9374b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:27.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:27.935+0000 7f6428e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560cd935ce50 0x560cd93750d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:27.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:27.935+0000 7f6423e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560cd9351c70 0x560cd935bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:27.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:27.935+0000 7f6428e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560cd935ce50 0x560cd93750d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:60462/0 (socket says 172.21.15.5:60462) 2024-09-10T10:02:27.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:27.935+0000 7f6423400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560cd9352590 0x560cd9374b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58276/0 (socket says 172.21.15.5:58276) 2024-09-10T10:02:27.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:27.935+0000 7f6428e00640 1 -- 172.21.15.5:0/3094279799 learned_addr learned my addr 172.21.15.5:0/3094279799 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T10:02:27.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:27.935+0000 7f6423e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560cd9351c70 0x560cd935bc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:51444/0 (socket says 172.21.15.5:51444) 2024-09-10T10:02:27.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:27.935+0000 7f6423e00640 1 -- 172.21.15.5:0/3094279799 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560cd935ce50 msgr2=0x560cd93750d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:27.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:27.935+0000 7f6423e00640 1 --2- 172.21.15.5:0/3094279799 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560cd935ce50 0x560cd93750d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:27.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:27.935+0000 7f6423e00640 1 -- 172.21.15.5:0/3094279799 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560cd9352590 msgr2=0x560cd9374b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:27.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:27.935+0000 7f6423e00640 1 --2- 172.21.15.5:0/3094279799 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560cd9352590 0x560cd9374b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:27.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:27.935+0000 7f6423e00640 1 -- 172.21.15.5:0/3094279799 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560cd934e030 con 0x560cd9351c70 2024-09-10T10:02:27.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:27.935+0000 7f6428e00640 1 --2- 172.21.15.5:0/3094279799 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560cd935ce50 0x560cd93750d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:02:27.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:27.936+0000 7f6423e00640 1 --2- 172.21.15.5:0/3094279799 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560cd9351c70 0x560cd935bc60 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f6414097bb0 tx=0x7f641409f590 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8e039526625e2dd8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:27.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:27.936+0000 7f6418c00640 1 -- 172.21.15.5:0/3094279799 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f641409ab40 con 0x560cd9351c70 2024-09-10T10:02:27.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:27.936+0000 7f6418c00640 1 -- 172.21.15.5:0/3094279799 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f641409b8e0 con 0x560cd9351c70 2024-09-10T10:02:27.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:27.936+0000 7f6418c00640 1 -- 172.21.15.5:0/3094279799 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64140d2500 con 0x560cd9351c70 2024-09-10T10:02:27.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:27.937+0000 7f642da68740 1 -- 172.21.15.5:0/3094279799 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560cd9351c70 msgr2=0x560cd935bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:27.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:27.937+0000 7f642da68740 1 --2- 172.21.15.5:0/3094279799 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560cd9351c70 0x560cd935bc60 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f6414097bb0 tx=0x7f641409f590 comp rx=0 tx=0).stop 2024-09-10T10:02:27.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:27.937+0000 7f642da68740 1 -- 172.21.15.5:0/3094279799 shutdown_connections 2024-09-10T10:02:27.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:27.937+0000 7f642da68740 1 --2- 172.21.15.5:0/3094279799 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560cd935ce50 0x560cd93750d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:27.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:27.937+0000 7f642da68740 1 --2- 172.21.15.5:0/3094279799 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560cd9352590 0x560cd9374b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:27.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:27.937+0000 7f642da68740 1 --2- 172.21.15.5:0/3094279799 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560cd9351c70 0x560cd935bc60 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:27.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:27.937+0000 7f642da68740 1 -- 172.21.15.5:0/3094279799 >> 172.21.15.5:0/3094279799 conn(0x560cd934e200 msgr2=0x560cd935b550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:27.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:27.937+0000 7f642da68740 1 -- 172.21.15.5:0/3094279799 shutdown_connections 2024-09-10T10:02:27.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:27.937+0000 7f642da68740 1 -- 172.21.15.5:0/3094279799 wait complete. 2024-09-10T10:02:27.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:27.937+0000 7f642da68740 1 -- start start 2024-09-10T10:02:27.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:27.937+0000 7f642da68740 1 Processor -- start 2024-09-10T10:02:27.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:27.938+0000 7f642da68740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560cd9351c70 0x560cd9365640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:27.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:27.938+0000 7f642da68740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560cd9352590 0x560cd9356b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:27.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:27.938+0000 7f642da68740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560cd935ce50 0x560cd9357100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:27.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:27.938+0000 7f642da68740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f64241b6920 con 0x560cd9351c70 2024-09-10T10:02:27.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:27.938+0000 7f642da68740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f64241b6250 con 0x560cd935ce50 2024-09-10T10:02:27.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:27.938+0000 7f642da68740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f64241b3490 con 0x560cd9352590 2024-09-10T10:02:27.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:27.938+0000 7f642da68740 10 client.0 _ll_register_callbacks cb 0x7f64140046c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:02:27.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:27.938+0000 7f6423e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560cd9351c70 0x560cd9365640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:27.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:27.938+0000 7f6428e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560cd935ce50 0x560cd9357100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:27.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:27.938+0000 7f6428e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560cd935ce50 0x560cd9357100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:60466/0 (socket says 172.21.15.5:60466) 2024-09-10T10:02:27.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:27.938+0000 7f6423e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560cd9351c70 0x560cd9365640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58290/0 (socket says 172.21.15.5:58290) 2024-09-10T10:02:27.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:27.938+0000 7f6428e00640 1 -- 172.21.15.5:0/2042999383 learned_addr learned my addr 172.21.15.5:0/2042999383 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T10:02:27.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:27.938+0000 7f6423400640 1 --2- 172.21.15.5:0/2042999383 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560cd9352590 0x560cd9356b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:27.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:27.939+0000 7f6428e00640 1 -- 172.21.15.5:0/2042999383 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560cd9352590 msgr2=0x560cd9356b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:27.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:27.939+0000 7f6428e00640 1 --2- 172.21.15.5:0/2042999383 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560cd9352590 0x560cd9356b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:27.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:27.939+0000 7f6428e00640 1 -- 172.21.15.5:0/2042999383 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560cd9351c70 msgr2=0x560cd9365640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:27.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:27.939+0000 7f6428e00640 1 --2- 172.21.15.5:0/2042999383 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560cd9351c70 0x560cd9365640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:27.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:27.939+0000 7f6428e00640 1 -- 172.21.15.5:0/2042999383 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560cd9357930 con 0x560cd935ce50 2024-09-10T10:02:27.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:27.939+0000 7f6423e00640 1 --2- 172.21.15.5:0/2042999383 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560cd9351c70 0x560cd9365640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:02:27.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:27.939+0000 7f6428e00640 1 --2- 172.21.15.5:0/2042999383 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560cd935ce50 0x560cd9357100 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f641c01c7a0 tx=0x7f641c014040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9c06c83112699e66 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:27.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:27.939+0000 7f640e000640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-09-10T10:02:27.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:27.939+0000 7f640e000640 1 -- 172.21.15.5:0/2042999383 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f641c010bb0 con 0x560cd935ce50 2024-09-10T10:02:27.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:27.939+0000 7f642da68740 1 -- 172.21.15.5:0/2042999383 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f641409b8e0 con 0x560cd935ce50 2024-09-10T10:02:27.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:27.939+0000 7f640e000640 1 -- 172.21.15.5:0/2042999383 <== mon.2 v2:172.21.15.102:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f641c010d50 con 0x560cd935ce50 2024-09-10T10:02:27.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:27.940+0000 7f640e000640 1 -- 172.21.15.5:0/2042999383 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f641c029410 con 0x560cd935ce50 2024-09-10T10:02:27.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:27.940+0000 7f642da68740 10 client.29966 fetch_fsmap learned FSMap version 42 2024-09-10T10:02:27.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:27.940+0000 7f642da68740 1 -- 172.21.15.5:0/2042999383 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x560cd93d7d10 con 0x560cd935ce50 2024-09-10T10:02:27.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:27.940+0000 7f640e000640 1 -- 172.21.15.5:0/2042999383 <== mon.2 v2:172.21.15.102:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f641409b8e0 con 0x560cd935ce50 2024-09-10T10:02:27.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:27.940+0000 7f640e000640 1 -- 172.21.15.5:0/2042999383 <== mon.2 v2:172.21.15.102:3300/0 5 ==== fsmap(e 42) ==== 8370+0+0 (secure 0 0 0) 0x7f641c00f040 con 0x560cd935ce50 2024-09-10T10:02:27.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:27.940+0000 7f642da68740 10 client.29966 fetch_fsmap finished waiting for FSMap version 42 2024-09-10T10:02:27.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:27.940+0000 7f642da68740 10 client.29966 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-10T10:02:27.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:27.940+0000 7f642da68740 20 client.29966 populate_metadata read hostname 'smithi005' 2024-09-10T10:02:27.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:27.940+0000 7f642da68740 1 --2- 172.21.15.5:0/2042999383 >> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] conn(0x560cd93d8460 0x560cd93da880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:27.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:27.940+0000 7f642da68740 4 client.29966 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T10:02:27.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:27.940+0000 7f642da68740 1 -- 172.21.15.5:0/2042999383 --> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560cd92bfe60 con 0x560cd93d8460 2024-09-10T10:02:27.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:27.941+0000 7f6423e00640 1 --2- 172.21.15.5:0/2042999383 >> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] conn(0x560cd93d8460 0x560cd93da880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:27.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:27.941+0000 7f6423e00640 1 --2- 172.21.15.5:0/2042999383 >> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] conn(0x560cd93d8460 0x560cd93da880 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f5e6b05120ec38c8 server_cookie=5b9f8ee2223e998e in_seq=0 out_seq=0 2024-09-10T10:02:27.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:27.941+0000 7f640e000640 10 client.29966 ms_handle_connect on v2:172.21.15.159:6838/2073739599 2024-09-10T10:02:27.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:27.941+0000 7f640e000640 1 -- 172.21.15.5:0/2042999383 <== mds.1 v2:172.21.15.159:6838/2073739599 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560cd92bfe60 con 0x560cd93d8460 2024-09-10T10:02:27.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:27.941+0000 7f640e000640 10 client.29966 handle_command_reply: tid=0 2024-09-10T10:02:27.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:27.941+0000 7f642da68740 1 client.29966 shutdown 2024-09-10T10:02:27.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:27.942+0000 7f642da68740 1 -- 172.21.15.5:0/2042999383 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560cd935ce50 msgr2=0x560cd9357100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:27.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:27.942+0000 7f642da68740 1 --2- 172.21.15.5:0/2042999383 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560cd935ce50 0x560cd9357100 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f641c01c7a0 tx=0x7f641c014040 comp rx=0 tx=0).stop 2024-09-10T10:02:27.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:27.942+0000 7f642da68740 1 -- 172.21.15.5:0/2042999383 shutdown_connections 2024-09-10T10:02:27.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:27.942+0000 7f642da68740 1 --2- 172.21.15.5:0/2042999383 >> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] conn(0x560cd93d8460 0x560cd93da880 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:27.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:27.942+0000 7f642da68740 1 --2- 172.21.15.5:0/2042999383 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x560cd935ce50 0x560cd9357100 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:27.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:27.942+0000 7f642da68740 1 --2- 172.21.15.5:0/2042999383 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x560cd9352590 0x560cd9356b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:27.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:27.942+0000 7f640e000640 0 client.29966 ms_handle_reset on v2:172.21.15.159:6838/2073739599 2024-09-10T10:02:27.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:27.942+0000 7f642da68740 1 --2- 172.21.15.5:0/2042999383 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560cd9351c70 0x560cd9365640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:27.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:27.942+0000 7f642da68740 1 -- 172.21.15.5:0/2042999383 >> 172.21.15.5:0/2042999383 conn(0x560cd934e200 msgr2=0x560cd9353ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:27.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:27.942+0000 7f642da68740 1 -- 172.21.15.5:0/2042999383 shutdown_connections 2024-09-10T10:02:27.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:27.942+0000 7f642da68740 1 -- 172.21.15.5:0/2042999383 wait complete. 2024-09-10T10:02:27.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:27.942+0000 7f642da68740 20 client.29966 trim_cache size 0 max 16384 2024-09-10T10:02:27.947 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-10T10:02:27.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:27.943+0000 7f642ae00640 1 -- 172.21.15.5:0/2990217425 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f64080a2b80 msgr2=0x7f64080a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:27.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:27.943+0000 7f642ae00640 1 --2- 172.21.15.5:0/2990217425 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f64080a2b80 0x7f64080a5040 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f642406b460 tx=0x7f641c009600 comp rx=0 tx=0).stop 2024-09-10T10:02:27.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:27.943+0000 7f642ae00640 1 -- 172.21.15.5:0/2990217425 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6424076720 msgr2=0x7f64241a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:27.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:27.943+0000 7f642ae00640 1 --2- 172.21.15.5:0/2990217425 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6424076720 0x7f64241a46f0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f641400d950 tx=0x7f641400de10 comp rx=0 tx=0).stop 2024-09-10T10:02:27.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:27.943+0000 7f642ae00640 1 -- 172.21.15.5:0/2990217425 shutdown_connections 2024-09-10T10:02:27.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:27.943+0000 7f642ae00640 1 --2- 172.21.15.5:0/2990217425 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f64080a2b80 0x7f64080a5040 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:27.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:27.943+0000 7f642ae00640 1 --2- 172.21.15.5:0/2990217425 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6424112a30 0x7f642419e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:27.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:27.943+0000 7f642ae00640 1 --2- 172.21.15.5:0/2990217425 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6424077040 0x7f64241a4c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:27.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:27.943+0000 7f642ae00640 1 --2- 172.21.15.5:0/2990217425 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6424076720 0x7f64241a46f0 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:27.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:27.943+0000 7f642ae00640 1 -- 172.21.15.5:0/2990217425 >> 172.21.15.5:0/2990217425 conn(0x7f64240ffa90 msgr2=0x7f6424101e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:27.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:27.943+0000 7f642ae00640 1 -- 172.21.15.5:0/2990217425 shutdown_connections 2024-09-10T10:02:27.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:27.943+0000 7f642ae00640 1 -- 172.21.15.5:0/2990217425 wait complete. 2024-09-10T10:02:28.069 INFO:tasks.workunit.client.0.smithi005.stdout: 10 156147 12.99 MB/sec execute 486 sec latency 6895.442 ms 2024-09-10T10:02:28.720 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-10T10:02:29.069 INFO:tasks.workunit.client.0.smithi005.stdout: 10 156147 12.96 MB/sec execute 487 sec latency 7895.522 ms 2024-09-10T10:02:29.138 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:02:29.478 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:02:29 smithi005 ceph-mon[30350]: pgmap v1117: 129 pgs: 129 active+clean; 731 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 247 KiB/s rd, 2.9 MiB/s wr, 144 op/s 2024-09-10T10:02:29.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:02:29 smithi159 ceph-mon[35513]: pgmap v1117: 129 pgs: 129 active+clean; 731 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 247 KiB/s rd, 2.9 MiB/s wr, 144 op/s 2024-09-10T10:02:29.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:02:29 smithi102 ceph-mon[35442]: pgmap v1117: 129 pgs: 129 active+clean; 731 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 247 KiB/s rd, 2.9 MiB/s wr, 144 op/s 2024-09-10T10:02:29.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:29.708+0000 7ff07ea00640 1 -- 172.21.15.5:0/4291963256 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff078101de0 msgr2=0x7ff0781021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:29.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:29.708+0000 7ff07ea00640 1 --2- 172.21.15.5:0/4291963256 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff078101de0 0x7ff0781021c0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7ff068009860 tx=0x7ff06802a340 comp rx=0 tx=0).stop 2024-09-10T10:02:29.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:29.708+0000 7ff07ea00640 1 -- 172.21.15.5:0/4291963256 shutdown_connections 2024-09-10T10:02:29.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:29.708+0000 7ff07ea00640 1 --2- 172.21.15.5:0/4291963256 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff078106f10 0x7ff07810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:29.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:29.708+0000 7ff07ea00640 1 --2- 172.21.15.5:0/4291963256 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff078102790 0x7ff0781067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:29.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:29.708+0000 7ff07ea00640 1 --2- 172.21.15.5:0/4291963256 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff078101de0 0x7ff0781021c0 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:29.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:29.708+0000 7ff07ea00640 1 -- 172.21.15.5:0/4291963256 >> 172.21.15.5:0/4291963256 conn(0x7ff0780fd8c0 msgr2=0x7ff0780ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:29.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:29.709+0000 7ff07ea00640 1 -- 172.21.15.5:0/4291963256 shutdown_connections 2024-09-10T10:02:29.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:29.709+0000 7ff07ea00640 1 -- 172.21.15.5:0/4291963256 wait complete. 2024-09-10T10:02:29.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:29.710+0000 7ff07ea00640 1 Processor -- start 2024-09-10T10:02:29.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:29.710+0000 7ff07ea00640 1 -- start start 2024-09-10T10:02:29.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:29.710+0000 7ff07ea00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff078101de0 0x7ff07819a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:29.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:29.710+0000 7ff07ea00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff078102790 0x7ff07819abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:29.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:29.711+0000 7ff07ea00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff078106f10 0x7ff07819ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:29.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:29.711+0000 7ff07ea00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff078111f70 con 0x7ff078101de0 2024-09-10T10:02:29.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:29.711+0000 7ff07ea00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7ff078111df0 con 0x7ff078102790 2024-09-10T10:02:29.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:29.711+0000 7ff07ea00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7ff0781120f0 con 0x7ff078106f10 2024-09-10T10:02:29.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:29.711+0000 7ff077e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff078101de0 0x7ff07819a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:29.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:29.711+0000 7ff077e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff078101de0 0x7ff07819a6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58310/0 (socket says 172.21.15.5:58310) 2024-09-10T10:02:29.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:29.711+0000 7ff077e00640 1 -- 172.21.15.5:0/3956931621 learned_addr learned my addr 172.21.15.5:0/3956931621 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T10:02:29.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:29.711+0000 7ff06f400640 1 --2- 172.21.15.5:0/3956931621 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff078102790 0x7ff07819abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:29.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:29.711+0000 7ff07ca00640 1 --2- 172.21.15.5:0/3956931621 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff078106f10 0x7ff07819ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:29.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:29.711+0000 7ff077e00640 1 -- 172.21.15.5:0/3956931621 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff078106f10 msgr2=0x7ff07819ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:29.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:29.711+0000 7ff077e00640 1 --2- 172.21.15.5:0/3956931621 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff078106f10 0x7ff07819ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:29.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:29.711+0000 7ff077e00640 1 -- 172.21.15.5:0/3956931621 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff078102790 msgr2=0x7ff07819abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:29.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:29.711+0000 7ff077e00640 1 --2- 172.21.15.5:0/3956931621 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff078102790 0x7ff07819abe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:29.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:29.711+0000 7ff077e00640 1 -- 172.21.15.5:0/3956931621 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff07819f650 con 0x7ff078101de0 2024-09-10T10:02:29.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:29.712+0000 7ff077e00640 1 --2- 172.21.15.5:0/3956931621 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff078101de0 0x7ff07819a6a0 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7ff068035040 tx=0x7ff068033d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:29.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:29.712+0000 7ff075600640 1 -- 172.21.15.5:0/3956931621 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff06803f020 con 0x7ff078101de0 2024-09-10T10:02:29.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:29.712+0000 7ff075600640 1 -- 172.21.15.5:0/3956931621 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff068032770 con 0x7ff078101de0 2024-09-10T10:02:29.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:29.712+0000 7ff075600640 1 -- 172.21.15.5:0/3956931621 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff068037040 con 0x7ff078101de0 2024-09-10T10:02:29.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:29.712+0000 7ff07ea00640 1 -- 172.21.15.5:0/3956931621 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff07819f8e0 con 0x7ff078101de0 2024-09-10T10:02:29.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:29.712+0000 7ff07ea00640 1 -- 172.21.15.5:0/3956931621 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0781a7220 con 0x7ff078101de0 2024-09-10T10:02:29.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:29.716+0000 7ff081427740 1 Processor -- start 2024-09-10T10:02:29.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:29.716+0000 7ff081427740 1 -- start start 2024-09-10T10:02:29.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:29.717+0000 7ff075600640 1 -- 172.21.15.5:0/3956931621 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7ff068042020 con 0x7ff078101de0 2024-09-10T10:02:29.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:29.717+0000 7ff081427740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5635635efc70 0x5635635f0050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:29.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:29.717+0000 7ff081427740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5635635f0590 0x5635635fa910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:29.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:29.717+0000 7ff081427740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5635635fae50 0x5635635fd240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:29.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:29.717+0000 7ff081427740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5635635563e0 con 0x5635635f0590 2024-09-10T10:02:29.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:29.717+0000 7ff081427740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x5635634fdd80 con 0x5635635fae50 2024-09-10T10:02:29.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:29.717+0000 7ff081427740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5635635523a0 con 0x5635635efc70 2024-09-10T10:02:29.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:29.717+0000 7ff07ca00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5635635efc70 0x5635635f0050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:29.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:29.717+0000 7ff077e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5635635f0590 0x5635635fa910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:29.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:29.717+0000 7ff07ca00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5635635efc70 0x5635635f0050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:51466/0 (socket says 172.21.15.5:51466) 2024-09-10T10:02:29.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:29.717+0000 7ff07ca00640 1 -- 172.21.15.5:0/2725155244 learned_addr learned my addr 172.21.15.5:0/2725155244 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T10:02:29.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:29.717+0000 7ff06f400640 1 --2- 172.21.15.5:0/2725155244 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5635635fae50 0x5635635fd240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:29.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:29.718+0000 7ff07ca00640 1 -- 172.21.15.5:0/2725155244 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5635635fae50 msgr2=0x5635635fd240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:29.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:29.718+0000 7ff07ca00640 1 --2- 172.21.15.5:0/2725155244 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5635635fae50 0x5635635fd240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:29.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:29.718+0000 7ff07ca00640 1 -- 172.21.15.5:0/2725155244 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5635635f0590 msgr2=0x5635635fa910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:29.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:29.718+0000 7ff07ca00640 1 --2- 172.21.15.5:0/2725155244 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5635635f0590 0x5635635fa910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:29.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:29.718+0000 7ff07ca00640 1 -- 172.21.15.5:0/2725155244 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56356351cae0 con 0x5635635efc70 2024-09-10T10:02:29.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:29.718+0000 7ff07ca00640 1 --2- 172.21.15.5:0/2725155244 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5635635efc70 0x5635635f0050 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7ff0700099f0 tx=0x7ff0700095c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e1880bcc4f608671 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:29.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:29.718+0000 7ff075600640 1 --2- 172.21.15.5:0/3956931621 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7ff05c0a2b80 0x7ff05c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:29.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:29.718+0000 7ff06d200640 1 -- 172.21.15.5:0/2725155244 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff07001d070 con 0x5635635efc70 2024-09-10T10:02:29.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:29.719+0000 7ff075600640 1 -- 172.21.15.5:0/3956931621 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7ff0680ef5f0 con 0x7ff078101de0 2024-09-10T10:02:29.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:29.719+0000 7ff06d200640 1 -- 172.21.15.5:0/2725155244 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff070003e40 con 0x5635635efc70 2024-09-10T10:02:29.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:29.719+0000 7ff06d200640 1 -- 172.21.15.5:0/2725155244 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff07000db00 con 0x5635635efc70 2024-09-10T10:02:29.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:29.719+0000 7ff077e00640 1 --2- 172.21.15.5:0/3956931621 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7ff05c0a2b80 0x7ff05c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:29.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:29.719+0000 7ff081427740 1 -- 172.21.15.5:0/2725155244 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5635635efc70 msgr2=0x5635635f0050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:29.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:29.719+0000 7ff081427740 1 --2- 172.21.15.5:0/2725155244 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5635635efc70 0x5635635f0050 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7ff0700099f0 tx=0x7ff0700095c0 comp rx=0 tx=0).stop 2024-09-10T10:02:29.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:29.719+0000 7ff077e00640 1 --2- 172.21.15.5:0/3956931621 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7ff05c0a2b80 0x7ff05c0a5040 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7ff0680ed070 tx=0x7ff0680f5190 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:29.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:29.720+0000 7ff081427740 1 -- 172.21.15.5:0/2725155244 shutdown_connections 2024-09-10T10:02:29.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:29.720+0000 7ff081427740 1 --2- 172.21.15.5:0/2725155244 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5635635fae50 0x5635635fd240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:29.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:29.720+0000 7ff081427740 1 --2- 172.21.15.5:0/2725155244 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5635635f0590 0x5635635fa910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:29.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:29.720+0000 7ff081427740 1 --2- 172.21.15.5:0/2725155244 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5635635efc70 0x5635635f0050 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:29.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:29.720+0000 7ff081427740 1 -- 172.21.15.5:0/2725155244 >> 172.21.15.5:0/2725155244 conn(0x5635635ec200 msgr2=0x5635635ec5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:29.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:29.720+0000 7ff081427740 1 -- 172.21.15.5:0/2725155244 shutdown_connections 2024-09-10T10:02:29.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:29.720+0000 7ff081427740 1 -- 172.21.15.5:0/2725155244 wait complete. 2024-09-10T10:02:29.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:29.720+0000 7ff081427740 1 -- start start 2024-09-10T10:02:29.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:29.720+0000 7ff081427740 1 Processor -- start 2024-09-10T10:02:29.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:29.721+0000 7ff081427740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5635635efc70 0x5635635f7d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:29.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:29.721+0000 7ff081427740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5635635f0590 0x5635635f8250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:29.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:29.721+0000 7ff081427740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5635635fae50 0x5635635f8790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:29.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:29.721+0000 7ff081427740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5635635fda20 con 0x5635635efc70 2024-09-10T10:02:29.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:29.721+0000 7ff081427740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x5635635110f0 con 0x5635635f0590 2024-09-10T10:02:29.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:29.721+0000 7ff081427740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5635635fdba0 con 0x5635635fae50 2024-09-10T10:02:29.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:29.721+0000 7ff081427740 10 client.0 _ll_register_callbacks cb 0x5635634f8bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:02:29.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:29.721+0000 7ff07ca00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5635635efc70 0x5635635f7d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:29.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:29.721+0000 7ff07ca00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5635635efc70 0x5635635f7d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58338/0 (socket says 172.21.15.5:58338) 2024-09-10T10:02:29.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:29.721+0000 7ff07ca00640 1 -- 172.21.15.5:0/1948387055 learned_addr learned my addr 172.21.15.5:0/1948387055 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T10:02:29.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:29.721+0000 7ff06f400640 1 --2- 172.21.15.5:0/1948387055 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5635635f0590 0x5635635f8250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:29.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:29.722+0000 7ff07ca00640 1 --2- 172.21.15.5:0/1948387055 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5635635fae50 0x5635635f8790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:29.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:29.722+0000 7ff07ca00640 1 -- 172.21.15.5:0/1948387055 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5635635fae50 msgr2=0x5635635f8790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:29.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:29.722+0000 7ff07ca00640 1 --2- 172.21.15.5:0/1948387055 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5635635fae50 0x5635635f8790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:29.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:29.722+0000 7ff07ca00640 1 -- 172.21.15.5:0/1948387055 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5635635f0590 msgr2=0x5635635f8250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:29.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:29.722+0000 7ff07ca00640 1 --2- 172.21.15.5:0/1948387055 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5635635f0590 0x5635635f8250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:29.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:29.722+0000 7ff07ca00640 1 -- 172.21.15.5:0/1948387055 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5635632c73c0 con 0x5635635efc70 2024-09-10T10:02:29.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:29.722+0000 7ff06f400640 1 --2- 172.21.15.5:0/1948387055 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5635635f0590 0x5635635f8250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:02:29.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:29.722+0000 7ff07ca00640 1 --2- 172.21.15.5:0/1948387055 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5635635efc70 0x5635635f7d10 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7ff070005b10 tx=0x7ff07000d780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9e88e487e0baa77 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:29.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:29.722+0000 7ff065600640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-10T10:02:29.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:29.722+0000 7ff065600640 1 -- 172.21.15.5:0/1948387055 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff07001d030 con 0x5635635efc70 2024-09-10T10:02:29.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:29.722+0000 7ff065600640 1 -- 172.21.15.5:0/1948387055 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7ff07000dcc0 con 0x5635635efc70 2024-09-10T10:02:29.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:29.722+0000 7ff081427740 1 -- 172.21.15.5:0/1948387055 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x563563563950 con 0x5635635efc70 2024-09-10T10:02:29.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:29.723+0000 7ff065600640 1 -- 172.21.15.5:0/1948387055 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7ff070004050 con 0x5635635efc70 2024-09-10T10:02:29.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:29.723+0000 7ff081427740 10 client.19884 fetch_fsmap learned FSMap version 42 2024-09-10T10:02:29.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:29.723+0000 7ff081427740 1 -- 172.21.15.5:0/1948387055 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x563563609a30 con 0x5635635efc70 2024-09-10T10:02:29.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:29.723+0000 7ff065600640 1 -- 172.21.15.5:0/1948387055 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x563563563950 con 0x5635635efc70 2024-09-10T10:02:29.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:29.723+0000 7ff065600640 1 -- 172.21.15.5:0/1948387055 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 42) ==== 8370+0+0 (secure 0 0 0) 0x7ff07000b040 con 0x5635635efc70 2024-09-10T10:02:29.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:29.723+0000 7ff081427740 10 client.19884 fetch_fsmap finished waiting for FSMap version 42 2024-09-10T10:02:29.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:29.723+0000 7ff081427740 10 client.19884 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T10:02:29.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:29.723+0000 7ff081427740 20 client.19884 populate_metadata read hostname 'smithi005' 2024-09-10T10:02:29.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:29.723+0000 7ff081427740 1 --2- 172.21.15.5:0/1948387055 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x56356360ae90 0x56356360d280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:29.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:29.723+0000 7ff081427740 4 client.19884 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:02:29.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:29.723+0000 7ff081427740 1 -- 172.21.15.5:0/1948387055 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56356355de60 con 0x56356360ae90 2024-09-10T10:02:29.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:29.724+0000 7ff06f400640 1 --2- 172.21.15.5:0/1948387055 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x56356360ae90 0x56356360d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:29.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:29.724+0000 7ff06f400640 1 --2- 172.21.15.5:0/1948387055 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x56356360ae90 0x56356360d280 crc :-1 s=READY pgs=323 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=40fbb23594f42d38 server_cookie=64917e9585d6c71e in_seq=0 out_seq=0 2024-09-10T10:02:29.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:29.724+0000 7ff065600640 10 client.19884 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T10:02:29.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:29.725+0000 7ff065600640 1 -- 172.21.15.5:0/1948387055 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56356355de60 con 0x56356360ae90 2024-09-10T10:02:29.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:29.725+0000 7ff065600640 10 client.19884 handle_command_reply: tid=0 2024-09-10T10:02:29.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:29.725+0000 7ff081427740 1 client.19884 shutdown 2024-09-10T10:02:29.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:29.725+0000 7ff081427740 1 -- 172.21.15.5:0/1948387055 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5635635efc70 msgr2=0x5635635f7d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:29.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:29.725+0000 7ff081427740 1 --2- 172.21.15.5:0/1948387055 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5635635efc70 0x5635635f7d10 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7ff070005b10 tx=0x7ff07000d780 comp rx=0 tx=0).stop 2024-09-10T10:02:29.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:29.726+0000 7ff081427740 1 -- 172.21.15.5:0/1948387055 shutdown_connections 2024-09-10T10:02:29.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:29.726+0000 7ff081427740 1 --2- 172.21.15.5:0/1948387055 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x56356360ae90 0x56356360d280 crc :-1 s=READY pgs=323 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:29.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:29.726+0000 7ff081427740 1 --2- 172.21.15.5:0/1948387055 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5635635fae50 0x5635635f8790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:29.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:29.726+0000 7ff081427740 1 --2- 172.21.15.5:0/1948387055 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5635635f0590 0x5635635f8250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:29.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:29.726+0000 7ff081427740 1 --2- 172.21.15.5:0/1948387055 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5635635efc70 0x5635635f7d10 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:29.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:29.726+0000 7ff081427740 1 -- 172.21.15.5:0/1948387055 >> 172.21.15.5:0/1948387055 conn(0x5635635ec200 msgr2=0x5635635eb380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:29.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:29.726+0000 7ff065600640 0 client.19884 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T10:02:29.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:29.726+0000 7ff081427740 1 -- 172.21.15.5:0/1948387055 shutdown_connections 2024-09-10T10:02:29.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:29.726+0000 7ff081427740 1 -- 172.21.15.5:0/1948387055 wait complete. 2024-09-10T10:02:29.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:29.726+0000 7ff081427740 20 client.19884 trim_cache size 0 max 16384 2024-09-10T10:02:29.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:29.740+0000 7ff081427740 1 Processor -- start 2024-09-10T10:02:29.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:29.740+0000 7ff081427740 1 -- start start 2024-09-10T10:02:29.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:29.740+0000 7ff081427740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5635635efc70 0x5635635f9c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:29.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:29.740+0000 7ff081427740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5635635f0590 0x5635636128b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:29.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:29.740+0000 7ff081427740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5635635fae50 0x563563612df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:29.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:29.740+0000 7ff081427740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff0781b10e0 con 0x5635635fae50 2024-09-10T10:02:29.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:29.740+0000 7ff081427740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7ff0781b2c20 con 0x5635635efc70 2024-09-10T10:02:29.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:29.740+0000 7ff081427740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7ff0781b17b0 con 0x5635635f0590 2024-09-10T10:02:29.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:29.741+0000 7ff07ca00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5635635fae50 0x563563612df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:29.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:29.741+0000 7ff07ca00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5635635efc70 0x5635635f9c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:29.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:29.741+0000 7ff06f400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5635635f0590 0x5635636128b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:29.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:29.741+0000 7ff07ca00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5635635efc70 0x5635635f9c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:60528/0 (socket says 172.21.15.5:60528) 2024-09-10T10:02:29.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:29.741+0000 7ff07ca00640 1 -- 172.21.15.5:0/2549258428 learned_addr learned my addr 172.21.15.5:0/2549258428 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T10:02:29.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:29.741+0000 7ff06f400640 1 -- 172.21.15.5:0/2549258428 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5635635efc70 msgr2=0x5635635f9c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:29.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:29.741+0000 7ff06f400640 1 --2- 172.21.15.5:0/2549258428 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5635635efc70 0x5635635f9c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:29.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:29.741+0000 7ff06f400640 1 -- 172.21.15.5:0/2549258428 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5635635fae50 msgr2=0x563563612df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:29.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:29.741+0000 7ff06f400640 1 --2- 172.21.15.5:0/2549258428 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5635635fae50 0x563563612df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:29.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:29.741+0000 7ff06f400640 1 -- 172.21.15.5:0/2549258428 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5635635e7770 con 0x5635635f0590 2024-09-10T10:02:29.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:29.742+0000 7ff06f400640 1 --2- 172.21.15.5:0/2549258428 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5635635f0590 0x5635636128b0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x5635635f92c0 tx=0x7ff060005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e7ee082fd9026c44 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:29.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:29.742+0000 7ff067e00640 1 -- 172.21.15.5:0/2549258428 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0600122d0 con 0x5635635f0590 2024-09-10T10:02:29.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:29.742+0000 7ff067e00640 1 -- 172.21.15.5:0/2549258428 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff06000fd70 con 0x5635635f0590 2024-09-10T10:02:29.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:29.742+0000 7ff067e00640 1 -- 172.21.15.5:0/2549258428 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff060005650 con 0x5635635f0590 2024-09-10T10:02:29.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:29.743+0000 7ff081427740 1 -- 172.21.15.5:0/2549258428 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5635635f0590 msgr2=0x5635636128b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:29.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:29.743+0000 7ff081427740 1 --2- 172.21.15.5:0/2549258428 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5635635f0590 0x5635636128b0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x5635635f92c0 tx=0x7ff060005f90 comp rx=0 tx=0).stop 2024-09-10T10:02:29.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:29.743+0000 7ff081427740 1 -- 172.21.15.5:0/2549258428 shutdown_connections 2024-09-10T10:02:29.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:29.743+0000 7ff081427740 1 --2- 172.21.15.5:0/2549258428 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5635635fae50 0x563563612df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:29.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:29.743+0000 7ff081427740 1 --2- 172.21.15.5:0/2549258428 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5635635f0590 0x5635636128b0 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:29.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:29.743+0000 7ff081427740 1 --2- 172.21.15.5:0/2549258428 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5635635efc70 0x5635635f9c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:29.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:29.743+0000 7ff081427740 1 -- 172.21.15.5:0/2549258428 >> 172.21.15.5:0/2549258428 conn(0x5635635ec200 msgr2=0x5635635f9570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:29.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:29.743+0000 7ff081427740 1 -- 172.21.15.5:0/2549258428 shutdown_connections 2024-09-10T10:02:29.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:29.743+0000 7ff081427740 1 -- 172.21.15.5:0/2549258428 wait complete. 2024-09-10T10:02:29.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:29.743+0000 7ff081427740 1 -- start start 2024-09-10T10:02:29.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:29.744+0000 7ff081427740 1 Processor -- start 2024-09-10T10:02:29.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:29.744+0000 7ff081427740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5635635efc70 0x563563602df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:29.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:29.744+0000 7ff081427740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5635635f0590 0x563563603330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:29.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:29.744+0000 7ff081427740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5635635fae50 0x5635635f3f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:29.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:29.744+0000 7ff081427740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff0781b2550 con 0x5635635f0590 2024-09-10T10:02:29.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:29.744+0000 7ff081427740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7ff0781b1e80 con 0x5635635fae50 2024-09-10T10:02:29.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:29.744+0000 7ff081427740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7ff0781af0c0 con 0x5635635efc70 2024-09-10T10:02:29.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:29.744+0000 7ff081427740 10 client.0 _ll_register_callbacks cb 0x7ff07001b070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:02:29.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:29.744+0000 7ff06f400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5635635f0590 0x563563603330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:29.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:29.744+0000 7ff06f400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5635635f0590 0x563563603330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58350/0 (socket says 172.21.15.5:58350) 2024-09-10T10:02:29.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:29.744+0000 7ff06f400640 1 -- 172.21.15.5:0/3346008573 learned_addr learned my addr 172.21.15.5:0/3346008573 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T10:02:29.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:29.745+0000 7ff07ca00640 1 --2- 172.21.15.5:0/3346008573 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5635635fae50 0x5635635f3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:29.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:29.745+0000 7ff07ca00640 1 --2- 172.21.15.5:0/3346008573 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5635635efc70 0x563563602df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:29.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:29.745+0000 7ff06f400640 1 -- 172.21.15.5:0/3346008573 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5635635efc70 msgr2=0x563563602df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:29.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:29.745+0000 7ff06f400640 1 --2- 172.21.15.5:0/3346008573 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5635635efc70 0x563563602df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:29.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:29.745+0000 7ff06f400640 1 -- 172.21.15.5:0/3346008573 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5635635fae50 msgr2=0x5635635f3f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:29.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:29.745+0000 7ff06f400640 1 --2- 172.21.15.5:0/3346008573 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5635635fae50 0x5635635f3f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:29.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:29.745+0000 7ff06f400640 1 -- 172.21.15.5:0/3346008573 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5635635f5c70 con 0x5635635f0590 2024-09-10T10:02:29.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:29.745+0000 7ff07ca00640 1 --2- 172.21.15.5:0/3346008573 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5635635fae50 0x5635635f3f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:02:29.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:29.745+0000 7ff06f400640 1 --2- 172.21.15.5:0/3346008573 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5635635f0590 0x563563603330 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7ff060009ed0 tx=0x7ff06000ff20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54fa08c582c4f75a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:29.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:29.745+0000 7ff065600640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-10T10:02:29.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:29.746+0000 7ff065600640 1 -- 172.21.15.5:0/3346008573 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0600122d0 con 0x5635635f0590 2024-09-10T10:02:29.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:29.746+0000 7ff081427740 1 -- 172.21.15.5:0/3346008573 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff06000fd70 con 0x5635635f0590 2024-09-10T10:02:29.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:29.746+0000 7ff065600640 1 -- 172.21.15.5:0/3346008573 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7ff06000f890 con 0x5635635f0590 2024-09-10T10:02:29.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:29.746+0000 7ff065600640 1 -- 172.21.15.5:0/3346008573 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7ff060012630 con 0x5635635f0590 2024-09-10T10:02:29.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:29.746+0000 7ff081427740 10 client.19890 fetch_fsmap learned FSMap version 42 2024-09-10T10:02:29.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:29.746+0000 7ff081427740 1 -- 172.21.15.5:0/3346008573 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x563563674dc0 con 0x5635635f0590 2024-09-10T10:02:29.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:29.746+0000 7ff065600640 1 -- 172.21.15.5:0/3346008573 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff06000fd70 con 0x5635635f0590 2024-09-10T10:02:29.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:29.747+0000 7ff065600640 1 -- 172.21.15.5:0/3346008573 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 42) ==== 8370+0+0 (secure 0 0 0) 0x7ff060030070 con 0x5635635f0590 2024-09-10T10:02:29.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:29.747+0000 7ff081427740 10 client.19890 fetch_fsmap finished waiting for FSMap version 42 2024-09-10T10:02:29.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:29.747+0000 7ff081427740 10 client.19890 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T10:02:29.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:29.747+0000 7ff081427740 20 client.19890 populate_metadata read hostname 'smithi005' 2024-09-10T10:02:29.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:29.747+0000 7ff081427740 1 --2- 172.21.15.5:0/3346008573 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x563563675540 0x563563677960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:29.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:29.747+0000 7ff081427740 4 client.19890 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T10:02:29.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:29.747+0000 7ff081427740 1 -- 172.21.15.5:0/3346008573 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5635634da010 con 0x563563675540 2024-09-10T10:02:29.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:29.747+0000 7ff07ca00640 1 --2- 172.21.15.5:0/3346008573 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x563563675540 0x563563677960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:29.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:29.748+0000 7ff07ca00640 1 --2- 172.21.15.5:0/3346008573 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x563563675540 0x563563677960 crc :-1 s=READY pgs=324 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9a8a913e14631ffc server_cookie=b07dd9b36acacc5 in_seq=0 out_seq=0 2024-09-10T10:02:29.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:29.748+0000 7ff065600640 10 client.19890 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T10:02:29.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:29.748+0000 7ff065600640 1 -- 172.21.15.5:0/3346008573 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5635634da010 con 0x563563675540 2024-09-10T10:02:29.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:29.748+0000 7ff065600640 10 client.19890 handle_command_reply: tid=0 2024-09-10T10:02:29.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:29.748+0000 7ff081427740 1 client.19890 shutdown 2024-09-10T10:02:29.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:29.748+0000 7ff081427740 1 -- 172.21.15.5:0/3346008573 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5635635f0590 msgr2=0x563563603330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:29.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:29.748+0000 7ff081427740 1 --2- 172.21.15.5:0/3346008573 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5635635f0590 0x563563603330 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7ff060009ed0 tx=0x7ff06000ff20 comp rx=0 tx=0).stop 2024-09-10T10:02:29.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:29.748+0000 7ff081427740 1 -- 172.21.15.5:0/3346008573 shutdown_connections 2024-09-10T10:02:29.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:29.749+0000 7ff081427740 1 --2- 172.21.15.5:0/3346008573 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x563563675540 0x563563677960 crc :-1 s=READY pgs=324 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:29.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:29.749+0000 7ff081427740 1 --2- 172.21.15.5:0/3346008573 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5635635fae50 0x5635635f3f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:29.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:29.749+0000 7ff081427740 1 --2- 172.21.15.5:0/3346008573 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5635635f0590 0x563563603330 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:29.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:29.749+0000 7ff081427740 1 --2- 172.21.15.5:0/3346008573 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5635635efc70 0x563563602df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:29.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:29.749+0000 7ff065600640 0 client.19890 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T10:02:29.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:29.749+0000 7ff081427740 1 -- 172.21.15.5:0/3346008573 >> 172.21.15.5:0/3346008573 conn(0x5635635ec200 msgr2=0x5635635f9d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:29.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:29.749+0000 7ff081427740 1 -- 172.21.15.5:0/3346008573 shutdown_connections 2024-09-10T10:02:29.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:29.749+0000 7ff081427740 1 -- 172.21.15.5:0/3346008573 wait complete. 2024-09-10T10:02:29.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:29.749+0000 7ff081427740 20 client.19890 trim_cache size 0 max 16384 2024-09-10T10:02:29.753 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-10T10:02:29.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:29.750+0000 7ff07ea00640 1 -- 172.21.15.5:0/3956931621 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7ff05c0a2b80 msgr2=0x7ff05c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:29.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:29.750+0000 7ff07ea00640 1 --2- 172.21.15.5:0/3956931621 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7ff05c0a2b80 0x7ff05c0a5040 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7ff0680ed070 tx=0x7ff0680f5190 comp rx=0 tx=0).stop 2024-09-10T10:02:29.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:29.750+0000 7ff07ea00640 1 -- 172.21.15.5:0/3956931621 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff078101de0 msgr2=0x7ff07819a6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:29.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:29.750+0000 7ff07ea00640 1 --2- 172.21.15.5:0/3956931621 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff078101de0 0x7ff07819a6a0 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7ff068035040 tx=0x7ff068033d90 comp rx=0 tx=0).stop 2024-09-10T10:02:29.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:29.750+0000 7ff07ea00640 1 -- 172.21.15.5:0/3956931621 shutdown_connections 2024-09-10T10:02:29.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:29.750+0000 7ff07ea00640 1 --2- 172.21.15.5:0/3956931621 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7ff05c0a2b80 0x7ff05c0a5040 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:29.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:29.750+0000 7ff07ea00640 1 --2- 172.21.15.5:0/3956931621 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff078106f10 0x7ff07819ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:29.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:29.750+0000 7ff07ea00640 1 --2- 172.21.15.5:0/3956931621 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff078102790 0x7ff07819abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:29.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:29.750+0000 7ff07ea00640 1 --2- 172.21.15.5:0/3956931621 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff078101de0 0x7ff07819a6a0 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:29.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:29.750+0000 7ff07ea00640 1 -- 172.21.15.5:0/3956931621 >> 172.21.15.5:0/3956931621 conn(0x7ff0780fd8c0 msgr2=0x7ff0780febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:29.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:29.750+0000 7ff07ea00640 1 -- 172.21.15.5:0/3956931621 shutdown_connections 2024-09-10T10:02:29.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:29.750+0000 7ff07ea00640 1 -- 172.21.15.5:0/3956931621 wait complete. 2024-09-10T10:02:30.069 INFO:tasks.workunit.client.0.smithi005.stdout: 10 156147 12.94 MB/sec execute 488 sec latency 8895.651 ms 2024-09-10T10:02:30.128 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T10:02:30.591 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:02:31.069 INFO:tasks.workunit.client.0.smithi005.stdout: 10 156147 12.91 MB/sec execute 489 sec latency 9895.788 ms 2024-09-10T10:02:31.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:31.194+0000 7f91c2800640 1 -- 172.21.15.5:0/672737989 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f91bc1079c0 msgr2=0x7f91bc107da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:31.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:31.194+0000 7f91c2800640 1 --2- 172.21.15.5:0/672737989 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f91bc1079c0 0x7f91bc107da0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f91b0009930 tx=0x7f91b002a420 comp rx=0 tx=0).stop 2024-09-10T10:02:31.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:31.194+0000 7f91c2800640 1 -- 172.21.15.5:0/672737989 shutdown_connections 2024-09-10T10:02:31.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:31.194+0000 7f91c2800640 1 --2- 172.21.15.5:0/672737989 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f91bc102360 0x7f91bc10e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:31.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:31.194+0000 7f91c2800640 1 --2- 172.21.15.5:0/672737989 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f91bc1019c0 0x7f91bc101e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:31.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:31.194+0000 7f91c2800640 1 --2- 172.21.15.5:0/672737989 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f91bc1079c0 0x7f91bc107da0 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:31.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:31.194+0000 7f91c2800640 1 -- 172.21.15.5:0/672737989 >> 172.21.15.5:0/672737989 conn(0x7f91bc0fd740 msgr2=0x7f91bc0ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:31.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:31.194+0000 7f91c2800640 1 -- 172.21.15.5:0/672737989 shutdown_connections 2024-09-10T10:02:31.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:31.194+0000 7f91c2800640 1 -- 172.21.15.5:0/672737989 wait complete. 2024-09-10T10:02:31.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:31.195+0000 7f91c2800640 1 Processor -- start 2024-09-10T10:02:31.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:31.195+0000 7f91c2800640 1 -- start start 2024-09-10T10:02:31.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:31.196+0000 7f91c2800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f91bc1019c0 0x7f91bc198290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:31.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:31.196+0000 7f91c2800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f91bc102360 0x7f91bc1987d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:31.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:31.196+0000 7f91c2800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f91bc1079c0 0x7f91bc19cb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:31.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:31.196+0000 7f91c2800640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f91bc111e10 con 0x7f91bc1019c0 2024-09-10T10:02:31.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:31.196+0000 7f91c2800640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f91bc111c90 con 0x7f91bc1079c0 2024-09-10T10:02:31.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:31.196+0000 7f91c2800640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f91bc111f90 con 0x7f91bc102360 2024-09-10T10:02:31.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:31.196+0000 7f91c1400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f91bc1019c0 0x7f91bc198290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:31.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:31.196+0000 7f91c1400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f91bc1019c0 0x7f91bc198290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49466/0 (socket says 172.21.15.5:49466) 2024-09-10T10:02:31.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:31.196+0000 7f91c1400640 1 -- 172.21.15.5:0/2380087336 learned_addr learned my addr 172.21.15.5:0/2380087336 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T10:02:31.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:31.196+0000 7f91c0a00640 1 --2- 172.21.15.5:0/2380087336 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f91bc102360 0x7f91bc1987d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:31.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:31.196+0000 7f91c1e00640 1 --2- 172.21.15.5:0/2380087336 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f91bc1079c0 0x7f91bc19cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:31.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:31.196+0000 7f91c1400640 1 -- 172.21.15.5:0/2380087336 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f91bc102360 msgr2=0x7f91bc1987d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:31.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:31.196+0000 7f91c1400640 1 --2- 172.21.15.5:0/2380087336 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f91bc102360 0x7f91bc1987d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:31.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:31.196+0000 7f91c1400640 1 -- 172.21.15.5:0/2380087336 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f91bc1079c0 msgr2=0x7f91bc19cb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:31.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:31.196+0000 7f91c1400640 1 --2- 172.21.15.5:0/2380087336 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f91bc1079c0 0x7f91bc19cb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:31.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:31.196+0000 7f91c1400640 1 -- 172.21.15.5:0/2380087336 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91bc19d240 con 0x7f91bc1019c0 2024-09-10T10:02:31.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:31.197+0000 7f91c1400640 1 --2- 172.21.15.5:0/2380087336 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f91bc1019c0 0x7f91bc198290 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f91b002a940 tx=0x7f91b0003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:31.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:31.197+0000 7f91ba000640 1 -- 172.21.15.5:0/2380087336 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91b003f070 con 0x7f91bc1019c0 2024-09-10T10:02:31.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:31.197+0000 7f91ba000640 1 -- 172.21.15.5:0/2380087336 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f91b0031c90 con 0x7f91bc1019c0 2024-09-10T10:02:31.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:31.197+0000 7f91ba000640 1 -- 172.21.15.5:0/2380087336 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91b0035030 con 0x7f91bc1019c0 2024-09-10T10:02:31.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:31.197+0000 7f91c2800640 1 -- 172.21.15.5:0/2380087336 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91bc19d4d0 con 0x7f91bc1019c0 2024-09-10T10:02:31.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:31.197+0000 7f91c2800640 1 -- 172.21.15.5:0/2380087336 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91bc19d960 con 0x7f91bc1019c0 2024-09-10T10:02:31.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:31.198+0000 7f91c2800640 1 -- 172.21.15.5:0/2380087336 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f91bc101e20 con 0x7f91bc1019c0 2024-09-10T10:02:31.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:31.200+0000 7f91ba000640 1 -- 172.21.15.5:0/2380087336 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f91b0042070 con 0x7f91bc1019c0 2024-09-10T10:02:31.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:31.201+0000 7f91ba000640 1 --2- 172.21.15.5:0/2380087336 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f91a40a2b80 0x7f91a40a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:31.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:31.202+0000 7f91ba000640 1 -- 172.21.15.5:0/2380087336 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f91b00efb20 con 0x7f91bc1019c0 2024-09-10T10:02:31.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:31.202+0000 7f91c0a00640 1 --2- 172.21.15.5:0/2380087336 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f91a40a2b80 0x7f91a40a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:31.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:31.202+0000 7f91c0a00640 1 --2- 172.21.15.5:0/2380087336 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f91a40a2b80 0x7f91a40a5040 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f91bc0ffd30 tx=0x7f91a800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:31.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:31.206+0000 7f91ba000640 1 -- 172.21.15.5:0/2380087336 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f91b00bbfe0 con 0x7f91bc1019c0 2024-09-10T10:02:31.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:31.430+0000 7f91c2800640 1 -- 172.21.15.5:0/2380087336 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f91bc109f30 con 0x7f91bc1019c0 2024-09-10T10:02:31.432 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:02:31 smithi005 ceph-mon[30350]: pgmap v1118: 129 pgs: 129 active+clean; 727 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 341 KiB/s rd, 2.4 MiB/s wr, 168 op/s 2024-09-10T10:02:31.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:31.431+0000 7f91ba000640 1 -- 172.21.15.5:0/2380087336 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 42 v42) ==== 94+0+11969 (secure 0 0 0) 0x7f91b00c02e0 con 0x7f91bc1019c0 2024-09-10T10:02:31.433 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-10T10:02:31.433 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":42,"btime":"2024-09-10T09:55:06:670916+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14454,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/2970594311","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2970594311},{"type":"v1","addr":"172.21.15.5:6835","nonce":2970594311}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14514,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6839/3616502174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":3616502174},{"type":"v1","addr":"172.21.15.5:6839","nonce":3616502174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":42,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T09:35:29.626287+0000","modified":"2024-09-10T09:55:06.419829+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24542,"mds_1":24535,"mds_2":24511,"mds_3":24506,"mds_4":24443},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14520":{"gid":14520,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/2073599842","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":2073599842},{"type":"v1","addr":"172.21.15.5:6841","nonce":2073599842}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"k","rank":4,"incarnation":26,"state":"up:active","state_seq":24,"addr":"172.21.15.102:6833/2457007427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":2457007427},{"type":"v1","addr":"172.21.15.102:6833","nonce":2457007427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/356445159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":356445159},{"type":"v1","addr":"172.21.15.159:6833","nonce":356445159}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6837/2894314507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":2894314507},{"type":"v1","addr":"172.21.15.5:6837","nonce":2894314507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.102:6835/1647549111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":1647549111},{"type":"v1","addr":"172.21.15.102:6835","nonce":1647549111}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6835/952657242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":952657242},{"type":"v1","addr":"172.21.15.159:6835","nonce":952657242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":19,"addr":"172.21.15.102:6837/2689776764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":2689776764},{"type":"v1","addr":"172.21.15.102:6837","nonce":2689776764}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.159:6837/620627639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":620627639},{"type":"v1","addr":"172.21.15.159:6837","nonce":620627639}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24535":{"gid":24535,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":18,"addr":"172.21.15.159:6839/2073739599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":2073739599},{"type":"v1","addr":"172.21.15.159:6839","nonce":2073739599}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24542":{"gid":24542,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.102:6839/335409749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6838","nonce":335409749},{"type":"v1","addr":"172.21.15.102:6839","nonce":335409749}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24542,"qdb_cluster":[24542,24535,24511,24506,24443]},"id":1}]} 2024-09-10T10:02:31.433 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 42 2024-09-10T10:02:31.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:31.434+0000 7f91c2800640 1 -- 172.21.15.5:0/2380087336 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f91a40a2b80 msgr2=0x7f91a40a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:31.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:31.434+0000 7f91c2800640 1 --2- 172.21.15.5:0/2380087336 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f91a40a2b80 0x7f91a40a5040 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f91bc0ffd30 tx=0x7f91a800b390 comp rx=0 tx=0).stop 2024-09-10T10:02:31.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:31.434+0000 7f91c2800640 1 -- 172.21.15.5:0/2380087336 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f91bc1019c0 msgr2=0x7f91bc198290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:31.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:31.434+0000 7f91c2800640 1 --2- 172.21.15.5:0/2380087336 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f91bc1019c0 0x7f91bc198290 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f91b002a940 tx=0x7f91b0003b90 comp rx=0 tx=0).stop 2024-09-10T10:02:31.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:31.434+0000 7f91c2800640 1 -- 172.21.15.5:0/2380087336 shutdown_connections 2024-09-10T10:02:31.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:31.434+0000 7f91c2800640 1 --2- 172.21.15.5:0/2380087336 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f91a40a2b80 0x7f91a40a5040 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:31.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:31.434+0000 7f91c2800640 1 --2- 172.21.15.5:0/2380087336 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f91bc1079c0 0x7f91bc19cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:31.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:31.434+0000 7f91c2800640 1 --2- 172.21.15.5:0/2380087336 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f91bc102360 0x7f91bc1987d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:31.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:31.434+0000 7f91c2800640 1 --2- 172.21.15.5:0/2380087336 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f91bc1019c0 0x7f91bc198290 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:31.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:31.434+0000 7f91c2800640 1 -- 172.21.15.5:0/2380087336 >> 172.21.15.5:0/2380087336 conn(0x7f91bc0fd740 msgr2=0x7f91bc0fdf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:31.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:31.434+0000 7f91c2800640 1 -- 172.21.15.5:0/2380087336 shutdown_connections 2024-09-10T10:02:31.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:31.434+0000 7f91c2800640 1 -- 172.21.15.5:0/2380087336 wait complete. 2024-09-10T10:02:31.492 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-10T10:02:31.492 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-10T10:02:31.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:02:31 smithi159 ceph-mon[35513]: pgmap v1118: 129 pgs: 129 active+clean; 727 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 341 KiB/s rd, 2.4 MiB/s wr, 168 op/s 2024-09-10T10:02:31.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:02:31 smithi102 ceph-mon[35442]: pgmap v1118: 129 pgs: 129 active+clean; 727 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 341 KiB/s rd, 2.4 MiB/s wr, 168 op/s 2024-09-10T10:02:32.070 INFO:tasks.workunit.client.0.smithi005.stdout: 10 156147 12.88 MB/sec execute 490 sec latency 10895.913 ms 2024-09-10T10:02:32.420 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:02:32.450 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24542 quiesce db --set-id b7188858 --release --await' 2024-09-10T10:02:32.450 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24542 quiesce db --set-id b7188858 --release --await 2024-09-10T10:02:32.500 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:02:32 smithi005 ceph-mon[30350]: from='client.? 172.21.15.5:0/2380087336' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:02:32.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:02:32 smithi159 ceph-mon[35513]: from='client.? 172.21.15.5:0/2380087336' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:02:32.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:02:32 smithi102 ceph-mon[35442]: from='client.? 172.21.15.5:0/2380087336' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:02:33.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:33.021+0000 7f62f7800640 1 -- 172.21.15.5:0/3757278356 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f62f0109120 msgr2=0x7f62f0110ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:33.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:33.021+0000 7f62f7800640 1 --2- 172.21.15.5:0/3757278356 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f62f0109120 0x7f62f0110ca0 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f62e8009ca0 tx=0x7f62e802a8c0 comp rx=0 tx=0).stop 2024-09-10T10:02:33.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:33.021+0000 7f62f7800640 1 -- 172.21.15.5:0/3757278356 shutdown_connections 2024-09-10T10:02:33.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:33.021+0000 7f62f7800640 1 --2- 172.21.15.5:0/3757278356 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f62f0109120 0x7f62f0110ca0 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:33.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:33.021+0000 7f62f7800640 1 --2- 172.21.15.5:0/3757278356 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f62f01049a0 0x7f62f01089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:33.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:33.021+0000 7f62f7800640 1 --2- 172.21.15.5:0/3757278356 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f62f0103ff0 0x7f62f01043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:33.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:33.021+0000 7f62f7800640 1 -- 172.21.15.5:0/3757278356 >> 172.21.15.5:0/3757278356 conn(0x7f62f00ffa90 msgr2=0x7f62f0101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:33.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:33.021+0000 7f62f7800640 1 -- 172.21.15.5:0/3757278356 shutdown_connections 2024-09-10T10:02:33.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:33.021+0000 7f62f7800640 1 -- 172.21.15.5:0/3757278356 wait complete. 2024-09-10T10:02:33.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:33.022+0000 7f62f7800640 1 Processor -- start 2024-09-10T10:02:33.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:33.022+0000 7f62f7800640 1 -- start start 2024-09-10T10:02:33.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:33.023+0000 7f62f7800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f62f0103ff0 0x7f62f019a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:33.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:33.023+0000 7f62f7800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f62f01049a0 0x7f62f019aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:33.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:33.023+0000 7f62f7800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f62f0109120 0x7f62f019ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:33.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:33.023+0000 7f62f7800640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f62f01141a0 con 0x7f62f0103ff0 2024-09-10T10:02:33.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:33.023+0000 7f62f7800640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f62f0114020 con 0x7f62f0109120 2024-09-10T10:02:33.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:33.023+0000 7f62f7800640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f62f0114320 con 0x7f62f01049a0 2024-09-10T10:02:33.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:33.023+0000 7f62f4e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f62f0103ff0 0x7f62f019a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:33.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:33.023+0000 7f62f4e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f62f0103ff0 0x7f62f019a660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49474/0 (socket says 172.21.15.5:49474) 2024-09-10T10:02:33.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:33.023+0000 7f62f4e00640 1 -- 172.21.15.5:0/3210463986 learned_addr learned my addr 172.21.15.5:0/3210463986 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T10:02:33.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:33.023+0000 7f62f5800640 1 --2- 172.21.15.5:0/3210463986 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f62f0109120 0x7f62f019ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:33.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:33.023+0000 7f62efe00640 1 --2- 172.21.15.5:0/3210463986 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f62f01049a0 0x7f62f019aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:33.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:33.023+0000 7f62f4e00640 1 -- 172.21.15.5:0/3210463986 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f62f01049a0 msgr2=0x7f62f019aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:33.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:33.023+0000 7f62f4e00640 1 --2- 172.21.15.5:0/3210463986 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f62f01049a0 0x7f62f019aba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:33.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:33.023+0000 7f62f4e00640 1 -- 172.21.15.5:0/3210463986 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f62f0109120 msgr2=0x7f62f019ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:33.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:33.023+0000 7f62f4e00640 1 --2- 172.21.15.5:0/3210463986 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f62f0109120 0x7f62f019ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:33.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:33.023+0000 7f62f4e00640 1 -- 172.21.15.5:0/3210463986 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62f019f610 con 0x7f62f0103ff0 2024-09-10T10:02:33.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:33.023+0000 7f62efe00640 1 --2- 172.21.15.5:0/3210463986 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f62f01049a0 0x7f62f019aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:02:33.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:33.024+0000 7f62f4e00640 1 --2- 172.21.15.5:0/3210463986 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f62f0103ff0 0x7f62f019a660 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f62e400e9f0 tx=0x7f62e400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:33.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:33.024+0000 7f62ed600640 1 -- 172.21.15.5:0/3210463986 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62e400cd80 con 0x7f62f0103ff0 2024-09-10T10:02:33.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:33.024+0000 7f62ed600640 1 -- 172.21.15.5:0/3210463986 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f62e4005500 con 0x7f62f0103ff0 2024-09-10T10:02:33.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:33.024+0000 7f62ed600640 1 -- 172.21.15.5:0/3210463986 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62e4010690 con 0x7f62f0103ff0 2024-09-10T10:02:33.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:33.024+0000 7f62f7800640 1 -- 172.21.15.5:0/3210463986 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62f019f900 con 0x7f62f0103ff0 2024-09-10T10:02:33.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:33.024+0000 7f62f7800640 1 -- 172.21.15.5:0/3210463986 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62f01a71e0 con 0x7f62f0103ff0 2024-09-10T10:02:33.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:33.028+0000 7f62fa28a740 1 Processor -- start 2024-09-10T10:02:33.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:33.028+0000 7f62fa28a740 1 -- start start 2024-09-10T10:02:33.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:33.029+0000 7f62fa28a740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e434e9aac0 0x55e434e9aea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:33.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:33.029+0000 7f62fa28a740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e434e9b3e0 0x55e434ea5760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:33.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:33.029+0000 7f62fa28a740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e434ea5ca0 0x55e434ea8090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:33.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:33.029+0000 7f62fa28a740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55e434e01290 con 0x55e434e9aac0 2024-09-10T10:02:33.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:33.029+0000 7f62fa28a740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55e434da8b40 con 0x55e434e9b3e0 2024-09-10T10:02:33.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:33.029+0000 7f62fa28a740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55e434dfd250 con 0x55e434ea5ca0 2024-09-10T10:02:33.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:33.029+0000 7f62ed600640 1 -- 172.21.15.5:0/3210463986 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f62e40046c0 con 0x7f62f0103ff0 2024-09-10T10:02:33.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:33.029+0000 7f62f5800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e434e9aac0 0x55e434e9aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:33.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:33.029+0000 7f62f5800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e434e9aac0 0x55e434e9aea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49482/0 (socket says 172.21.15.5:49482) 2024-09-10T10:02:33.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:33.029+0000 7f62f5800640 1 -- 172.21.15.5:0/725029398 learned_addr learned my addr 172.21.15.5:0/725029398 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T10:02:33.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:33.029+0000 7f62efe00640 1 --2- 172.21.15.5:0/725029398 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e434ea5ca0 0x55e434ea8090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:33.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:33.030+0000 7f62f4e00640 1 --2- 172.21.15.5:0/725029398 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e434e9b3e0 0x55e434ea5760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:33.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:33.030+0000 7f62f5800640 1 -- 172.21.15.5:0/725029398 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e434ea5ca0 msgr2=0x55e434ea8090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:33.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:33.030+0000 7f62f5800640 1 --2- 172.21.15.5:0/725029398 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e434ea5ca0 0x55e434ea8090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:33.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:33.030+0000 7f62f5800640 1 -- 172.21.15.5:0/725029398 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e434e9b3e0 msgr2=0x55e434ea5760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:33.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:33.030+0000 7f62f5800640 1 --2- 172.21.15.5:0/725029398 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e434e9b3e0 0x55e434ea5760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:33.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:33.030+0000 7f62f5800640 1 -- 172.21.15.5:0/725029398 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e434b726a0 con 0x55e434e9aac0 2024-09-10T10:02:33.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:33.030+0000 7f62f4e00640 1 --2- 172.21.15.5:0/725029398 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e434e9b3e0 0x55e434ea5760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:02:33.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:33.030+0000 7f62f5800640 1 --2- 172.21.15.5:0/725029398 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e434e9aac0 0x55e434e9aea0 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f62e8009ca0 tx=0x7f62e80093e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=360c6d5b7b8e29b0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:33.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:33.030+0000 7f62e2000640 1 -- 172.21.15.5:0/725029398 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62e8031db0 con 0x55e434e9aac0 2024-09-10T10:02:33.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:33.030+0000 7f62e2000640 1 -- 172.21.15.5:0/725029398 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f62e8036030 con 0x55e434e9aac0 2024-09-10T10:02:33.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:33.030+0000 7f62e2000640 1 -- 172.21.15.5:0/725029398 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62e80042d0 con 0x55e434e9aac0 2024-09-10T10:02:33.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:33.031+0000 7f62ed600640 1 --2- 172.21.15.5:0/3210463986 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f62d80a2b80 0x7f62d80a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:33.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:33.031+0000 7f62ed600640 1 -- 172.21.15.5:0/3210463986 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f62e40cf920 con 0x7f62f0103ff0 2024-09-10T10:02:33.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:33.031+0000 7f62f4e00640 1 --2- 172.21.15.5:0/3210463986 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f62d80a2b80 0x7f62d80a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:33.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:33.031+0000 7f62fa28a740 1 -- 172.21.15.5:0/725029398 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e434e9aac0 msgr2=0x55e434e9aea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:33.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:33.031+0000 7f62fa28a740 1 --2- 172.21.15.5:0/725029398 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e434e9aac0 0x55e434e9aea0 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f62e8009ca0 tx=0x7f62e80093e0 comp rx=0 tx=0).stop 2024-09-10T10:02:33.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:33.031+0000 7f62fa28a740 1 -- 172.21.15.5:0/725029398 shutdown_connections 2024-09-10T10:02:33.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:33.031+0000 7f62fa28a740 1 --2- 172.21.15.5:0/725029398 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e434ea5ca0 0x55e434ea8090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:33.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:33.031+0000 7f62fa28a740 1 --2- 172.21.15.5:0/725029398 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e434e9b3e0 0x55e434ea5760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:33.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:33.031+0000 7f62fa28a740 1 --2- 172.21.15.5:0/725029398 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e434e9aac0 0x55e434e9aea0 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:33.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:33.031+0000 7f62fa28a740 1 -- 172.21.15.5:0/725029398 >> 172.21.15.5:0/725029398 conn(0x55e434e97050 msgr2=0x55e434e97430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:33.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:33.032+0000 7f62fa28a740 1 -- 172.21.15.5:0/725029398 shutdown_connections 2024-09-10T10:02:33.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:33.032+0000 7f62fa28a740 1 -- 172.21.15.5:0/725029398 wait complete. 2024-09-10T10:02:33.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:33.032+0000 7f62f4e00640 1 --2- 172.21.15.5:0/3210463986 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f62d80a2b80 0x7f62d80a5040 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f62e409bcc0 tx=0x7f62e4004a40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:33.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:33.032+0000 7f62fa28a740 1 -- start start 2024-09-10T10:02:33.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:33.032+0000 7f62fa28a740 1 Processor -- start 2024-09-10T10:02:33.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:33.033+0000 7f62fa28a740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e434e9aac0 0x55e434ea55a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:33.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:33.033+0000 7f62fa28a740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e434e9b3e0 0x55e434ea2850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:33.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:33.033+0000 7f62fa28a740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e434ea5ca0 0x55e434ea2d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:33.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:33.033+0000 7f62fa28a740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55e434ea8790 con 0x55e434ea5ca0 2024-09-10T10:02:33.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:33.033+0000 7f62fa28a740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55e434dbbfa0 con 0x55e434e9aac0 2024-09-10T10:02:33.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:33.033+0000 7f62fa28a740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55e434ea8910 con 0x55e434e9b3e0 2024-09-10T10:02:33.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:33.033+0000 7f62fa28a740 10 client.0 _ll_register_callbacks cb 0x55e434b0fba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:02:33.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:33.033+0000 7f62f5800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e434e9aac0 0x55e434ea55a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:33.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:33.033+0000 7f62efe00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e434e9b3e0 0x55e434ea2850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:33.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:33.033+0000 7f62f5800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e434ea5ca0 0x55e434ea2d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:33.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:33.033+0000 7f62efe00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e434e9b3e0 0x55e434ea2850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:35310/0 (socket says 172.21.15.5:35310) 2024-09-10T10:02:33.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:33.033+0000 7f62efe00640 1 -- 172.21.15.5:0/2835983565 learned_addr learned my addr 172.21.15.5:0/2835983565 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T10:02:33.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:33.034+0000 7f62efe00640 1 -- 172.21.15.5:0/2835983565 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e434e9aac0 msgr2=0x55e434ea55a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:33.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:33.034+0000 7f62efe00640 1 --2- 172.21.15.5:0/2835983565 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e434e9aac0 0x55e434ea55a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:33.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:33.034+0000 7f62efe00640 1 -- 172.21.15.5:0/2835983565 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e434ea5ca0 msgr2=0x55e434ea2d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:33.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:33.034+0000 7f62efe00640 1 --2- 172.21.15.5:0/2835983565 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e434ea5ca0 0x55e434ea2d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:33.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:33.034+0000 7f62efe00640 1 -- 172.21.15.5:0/2835983565 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e434ea35e0 con 0x55e434e9b3e0 2024-09-10T10:02:33.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:33.034+0000 7f62efe00640 1 --2- 172.21.15.5:0/2835983565 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e434e9b3e0 0x55e434ea2850 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f62dc0126b0 tx=0x7f62dc012b70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3541313dc10bafd9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:33.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:33.034+0000 7f62d6a00640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-09-10T10:02:33.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:33.034+0000 7f62d6a00640 1 -- 172.21.15.5:0/2835983565 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62dc004240 con 0x55e434e9b3e0 2024-09-10T10:02:33.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:33.034+0000 7f62fa28a740 1 -- 172.21.15.5:0/2835983565 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f62e8036030 con 0x55e434e9b3e0 2024-09-10T10:02:33.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:33.034+0000 7f62d6a00640 1 -- 172.21.15.5:0/2835983565 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f62dc0043e0 con 0x55e434e9b3e0 2024-09-10T10:02:33.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:33.035+0000 7f62d6a00640 1 -- 172.21.15.5:0/2835983565 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f62dc025050 con 0x55e434e9b3e0 2024-09-10T10:02:33.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:33.035+0000 7f62fa28a740 10 client.29971 fetch_fsmap learned FSMap version 42 2024-09-10T10:02:33.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:33.035+0000 7f62fa28a740 1 -- 172.21.15.5:0/2835983565 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55e434eb4690 con 0x55e434e9b3e0 2024-09-10T10:02:33.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:33.035+0000 7f62d6a00640 1 -- 172.21.15.5:0/2835983565 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f62e8036030 con 0x55e434e9b3e0 2024-09-10T10:02:33.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:33.035+0000 7f62d6a00640 1 -- 172.21.15.5:0/2835983565 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 42) ==== 8370+0+0 (secure 0 0 0) 0x7f62dc00f460 con 0x55e434e9b3e0 2024-09-10T10:02:33.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:33.035+0000 7f62fa28a740 10 client.29971 fetch_fsmap finished waiting for FSMap version 42 2024-09-10T10:02:33.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:33.035+0000 7f62fa28a740 10 client.29971 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T10:02:33.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:33.035+0000 7f62fa28a740 20 client.29971 populate_metadata read hostname 'smithi005' 2024-09-10T10:02:33.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:33.035+0000 7f62fa28a740 1 --2- 172.21.15.5:0/2835983565 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55e434eb5aa0 0x55e434eb7ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:33.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:33.035+0000 7f62fa28a740 4 client.29971 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:02:33.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:33.035+0000 7f62fa28a740 1 -- 172.21.15.5:0/2835983565 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e434e0e800 con 0x55e434eb5aa0 2024-09-10T10:02:33.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:33.036+0000 7f62f5800640 1 --2- 172.21.15.5:0/2835983565 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55e434eb5aa0 0x55e434eb7ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:33.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:33.036+0000 7f62f5800640 1 --2- 172.21.15.5:0/2835983565 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55e434eb5aa0 0x55e434eb7ec0 crc :-1 s=READY pgs=325 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c8d6d2207ba2bd65 server_cookie=a50d8d2325266b9c in_seq=0 out_seq=0 2024-09-10T10:02:33.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:33.036+0000 7f62d6a00640 10 client.29971 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T10:02:33.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:33.037+0000 7f62d6a00640 1 -- 172.21.15.5:0/2835983565 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e434e0e800 con 0x55e434eb5aa0 2024-09-10T10:02:33.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:33.037+0000 7f62d6a00640 10 client.29971 handle_command_reply: tid=0 2024-09-10T10:02:33.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:33.037+0000 7f62fa28a740 1 client.29971 shutdown 2024-09-10T10:02:33.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:33.037+0000 7f62fa28a740 1 -- 172.21.15.5:0/2835983565 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e434e9b3e0 msgr2=0x55e434ea2850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:33.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:33.037+0000 7f62fa28a740 1 --2- 172.21.15.5:0/2835983565 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e434e9b3e0 0x55e434ea2850 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f62dc0126b0 tx=0x7f62dc012b70 comp rx=0 tx=0).stop 2024-09-10T10:02:33.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:33.037+0000 7f62fa28a740 1 -- 172.21.15.5:0/2835983565 shutdown_connections 2024-09-10T10:02:33.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:33.037+0000 7f62fa28a740 1 --2- 172.21.15.5:0/2835983565 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55e434eb5aa0 0x55e434eb7ec0 crc :-1 s=READY pgs=325 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:33.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:33.037+0000 7f62fa28a740 1 --2- 172.21.15.5:0/2835983565 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e434ea5ca0 0x55e434ea2d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:33.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:33.037+0000 7f62fa28a740 1 --2- 172.21.15.5:0/2835983565 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e434e9b3e0 0x55e434ea2850 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:33.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:33.037+0000 7f62fa28a740 1 --2- 172.21.15.5:0/2835983565 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e434e9aac0 0x55e434ea55a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:33.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:33.037+0000 7f62d6a00640 0 client.29971 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T10:02:33.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:33.037+0000 7f62fa28a740 1 -- 172.21.15.5:0/2835983565 >> 172.21.15.5:0/2835983565 conn(0x55e434e97050 msgr2=0x55e434e961c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:33.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:33.038+0000 7f62fa28a740 1 -- 172.21.15.5:0/2835983565 shutdown_connections 2024-09-10T10:02:33.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:33.038+0000 7f62fa28a740 1 -- 172.21.15.5:0/2835983565 wait complete. 2024-09-10T10:02:33.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:33.038+0000 7f62fa28a740 20 client.29971 trim_cache size 0 max 16384 2024-09-10T10:02:33.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:33.056+0000 7f62fa28a740 1 Processor -- start 2024-09-10T10:02:33.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:33.056+0000 7f62fa28a740 1 -- start start 2024-09-10T10:02:33.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:33.057+0000 7f62fa28a740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e434e9aac0 0x55e434e9dd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:33.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:33.057+0000 7f62fa28a740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e434e9b3e0 0x55e434ebd790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:33.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:33.057+0000 7f62fa28a740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e434ea5ca0 0x55e434ebdcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:33.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:33.057+0000 7f62fa28a740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f62f01b0fa0 con 0x55e434e9b3e0 2024-09-10T10:02:33.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:33.057+0000 7f62fa28a740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f62f01b2ae0 con 0x55e434ea5ca0 2024-09-10T10:02:33.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:33.057+0000 7f62fa28a740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f62f01b1670 con 0x55e434e9aac0 2024-09-10T10:02:33.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:33.057+0000 7f62efe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e434e9b3e0 0x55e434ebd790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:33.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:33.057+0000 7f62f5800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e434e9aac0 0x55e434e9dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:33.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:33.057+0000 7f62efe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e434e9b3e0 0x55e434ebd790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49494/0 (socket says 172.21.15.5:49494) 2024-09-10T10:02:33.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:33.057+0000 7f62f5800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e434e9aac0 0x55e434e9dd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:35326/0 (socket says 172.21.15.5:35326) 2024-09-10T10:02:33.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:33.057+0000 7f62efe00640 1 -- 172.21.15.5:0/3995250123 learned_addr learned my addr 172.21.15.5:0/3995250123 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T10:02:33.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:33.057+0000 7f62f5800640 1 --2- 172.21.15.5:0/3995250123 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e434ea5ca0 0x55e434ebdcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:33.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:33.058+0000 7f62f5800640 1 -- 172.21.15.5:0/3995250123 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e434ea5ca0 msgr2=0x55e434ebdcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:33.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:33.058+0000 7f62f5800640 1 --2- 172.21.15.5:0/3995250123 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e434ea5ca0 0x55e434ebdcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:33.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:33.058+0000 7f62f5800640 1 -- 172.21.15.5:0/3995250123 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e434e9b3e0 msgr2=0x55e434ebd790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:33.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:33.058+0000 7f62f5800640 1 --2- 172.21.15.5:0/3995250123 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e434e9b3e0 0x55e434ebd790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:33.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:33.058+0000 7f62f5800640 1 -- 172.21.15.5:0/3995250123 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e434e96e80 con 0x55e434e9aac0 2024-09-10T10:02:33.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:33.058+0000 7f62f5800640 1 --2- 172.21.15.5:0/3995250123 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e434e9aac0 0x55e434e9dd70 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f62e8009620 tx=0x7f62e8047040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dbd44976acf89361 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:33.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:33.058+0000 7f62e1600640 1 -- 172.21.15.5:0/3995250123 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62e804a040 con 0x55e434e9aac0 2024-09-10T10:02:33.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:33.058+0000 7f62e1600640 1 -- 172.21.15.5:0/3995250123 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f62e8036030 con 0x55e434e9aac0 2024-09-10T10:02:33.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:33.058+0000 7f62e1600640 1 -- 172.21.15.5:0/3995250123 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62e8030820 con 0x55e434e9aac0 2024-09-10T10:02:33.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:33.059+0000 7f62fa28a740 1 -- 172.21.15.5:0/3995250123 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e434e9aac0 msgr2=0x55e434e9dd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:33.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:33.059+0000 7f62fa28a740 1 --2- 172.21.15.5:0/3995250123 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e434e9aac0 0x55e434e9dd70 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f62e8009620 tx=0x7f62e8047040 comp rx=0 tx=0).stop 2024-09-10T10:02:33.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:33.059+0000 7f62fa28a740 1 -- 172.21.15.5:0/3995250123 shutdown_connections 2024-09-10T10:02:33.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:33.059+0000 7f62fa28a740 1 --2- 172.21.15.5:0/3995250123 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e434ea5ca0 0x55e434ebdcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:33.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:33.059+0000 7f62fa28a740 1 --2- 172.21.15.5:0/3995250123 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e434e9b3e0 0x55e434ebd790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:33.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:33.059+0000 7f62fa28a740 1 --2- 172.21.15.5:0/3995250123 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e434e9aac0 0x55e434e9dd70 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:33.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:33.059+0000 7f62fa28a740 1 -- 172.21.15.5:0/3995250123 >> 172.21.15.5:0/3995250123 conn(0x55e434e97050 msgr2=0x55e434ea48d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:33.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:33.059+0000 7f62fa28a740 1 -- 172.21.15.5:0/3995250123 shutdown_connections 2024-09-10T10:02:33.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:33.059+0000 7f62fa28a740 1 -- 172.21.15.5:0/3995250123 wait complete. 2024-09-10T10:02:33.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:33.060+0000 7f62fa28a740 1 -- start start 2024-09-10T10:02:33.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:33.060+0000 7f62fa28a740 1 Processor -- start 2024-09-10T10:02:33.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:33.060+0000 7f62fa28a740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e434e9aac0 0x55e434eadb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:33.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:33.060+0000 7f62fa28a740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e434e9b3e0 0x55e434eae070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:33.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:33.060+0000 7f62fa28a740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e434ea5ca0 0x55e434eaab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:33.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:33.060+0000 7f62fa28a740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f62f01b2410 con 0x55e434ea5ca0 2024-09-10T10:02:33.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:33.060+0000 7f62fa28a740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f62f01b1d40 con 0x55e434e9b3e0 2024-09-10T10:02:33.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:33.060+0000 7f62fa28a740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f62f01aef80 con 0x55e434e9aac0 2024-09-10T10:02:33.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:33.061+0000 7f62fa28a740 10 client.0 _ll_register_callbacks cb 0x55e434ea37b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:02:33.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:33.061+0000 7f62efe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e434e9b3e0 0x55e434eae070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:33.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:33.061+0000 7f62f5800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e434e9aac0 0x55e434eadb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:33.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:33.061+0000 7f62efe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e434e9b3e0 0x55e434eae070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:58458/0 (socket says 172.21.15.5:58458) 2024-09-10T10:02:33.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:33.061+0000 7f62efe00640 1 -- 172.21.15.5:0/2994276294 learned_addr learned my addr 172.21.15.5:0/2994276294 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T10:02:33.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:33.061+0000 7f62efe00640 1 -- 172.21.15.5:0/2994276294 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e434e9aac0 msgr2=0x55e434eadb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:33.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:33.061+0000 7f62efe00640 1 --2- 172.21.15.5:0/2994276294 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e434e9aac0 0x55e434eadb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:33.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:33.061+0000 7f62efe00640 1 -- 172.21.15.5:0/2994276294 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e434ea5ca0 msgr2=0x55e434eaab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:33.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:33.061+0000 7f62efe00640 1 --2- 172.21.15.5:0/2994276294 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e434ea5ca0 0x55e434eaab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:33.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:33.061+0000 7f62efe00640 1 -- 172.21.15.5:0/2994276294 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e434eae760 con 0x55e434e9b3e0 2024-09-10T10:02:33.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:33.061+0000 7f62f5800640 1 --2- 172.21.15.5:0/2994276294 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e434e9aac0 0x55e434eadb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:02:33.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:33.062+0000 7f62efe00640 1 --2- 172.21.15.5:0/2994276294 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e434e9b3e0 0x55e434eae070 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f62dc00f8b0 tx=0x7f62dc00f460 comp rx=0 tx=0).ready entity=mon.2 client_cookie=72c900f00996d35c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:33.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:33.062+0000 7f62d6a00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-09-10T10:02:33.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:33.062+0000 7f62d6a00640 1 -- 172.21.15.5:0/2994276294 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62dc022040 con 0x55e434e9b3e0 2024-09-10T10:02:33.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:33.062+0000 7f62fa28a740 1 -- 172.21.15.5:0/2994276294 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f62e8036030 con 0x55e434e9b3e0 2024-09-10T10:02:33.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:33.062+0000 7f62d6a00640 1 -- 172.21.15.5:0/2994276294 <== mon.2 v2:172.21.15.102:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f62dc0054c0 con 0x55e434e9b3e0 2024-09-10T10:02:33.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:33.062+0000 7f62d6a00640 1 -- 172.21.15.5:0/2994276294 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f62dc03b050 con 0x55e434e9b3e0 2024-09-10T10:02:33.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:33.062+0000 7f62fa28a740 10 client.30005 fetch_fsmap learned FSMap version 42 2024-09-10T10:02:33.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:33.062+0000 7f62fa28a740 1 -- 172.21.15.5:0/2994276294 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55e434f20da0 con 0x55e434e9b3e0 2024-09-10T10:02:33.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:33.063+0000 7f62d6a00640 1 -- 172.21.15.5:0/2994276294 <== mon.2 v2:172.21.15.102:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f62e8036030 con 0x55e434e9b3e0 2024-09-10T10:02:33.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:33.063+0000 7f62d6a00640 1 -- 172.21.15.5:0/2994276294 <== mon.2 v2:172.21.15.102:3300/0 5 ==== fsmap(e 42) ==== 8370+0+0 (secure 0 0 0) 0x7f62dc0035f0 con 0x55e434e9b3e0 2024-09-10T10:02:33.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:33.063+0000 7f62fa28a740 10 client.30005 fetch_fsmap finished waiting for FSMap version 42 2024-09-10T10:02:33.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:33.063+0000 7f62fa28a740 10 client.30005 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T10:02:33.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:33.063+0000 7f62fa28a740 20 client.30005 populate_metadata read hostname 'smithi005' 2024-09-10T10:02:33.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:33.063+0000 7f62fa28a740 1 --2- 172.21.15.5:0/2994276294 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55e434f21550 0x55e434f23970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:33.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:33.063+0000 7f62fa28a740 4 client.30005 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-10T10:02:33.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:33.063+0000 7f62fa28a740 1 -- 172.21.15.5:0/2994276294 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55e434e08d10 con 0x55e434f21550 2024-09-10T10:02:33.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:33.063+0000 7f62f5800640 1 --2- 172.21.15.5:0/2994276294 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55e434f21550 0x55e434f23970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:33.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:33.064+0000 7f62f5800640 1 --2- 172.21.15.5:0/2994276294 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55e434f21550 0x55e434f23970 crc :-1 s=READY pgs=326 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=661043925348ef23 server_cookie=b92edd9702f31dca in_seq=0 out_seq=0 2024-09-10T10:02:33.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:33.064+0000 7f62d6a00640 10 client.30005 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T10:02:33.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:33.065+0000 7f62d6a00640 1 -- 172.21.15.5:0/2994276294 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55e434e08d10 con 0x55e434f21550 2024-09-10T10:02:33.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:33.065+0000 7f62d6a00640 10 client.30005 handle_command_reply: tid=0 2024-09-10T10:02:33.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:33.065+0000 7f62fa28a740 1 client.30005 shutdown 2024-09-10T10:02:33.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:33.065+0000 7f62fa28a740 1 -- 172.21.15.5:0/2994276294 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e434e9b3e0 msgr2=0x55e434eae070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:33.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:33.065+0000 7f62fa28a740 1 --2- 172.21.15.5:0/2994276294 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e434e9b3e0 0x55e434eae070 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f62dc00f8b0 tx=0x7f62dc00f460 comp rx=0 tx=0).stop 2024-09-10T10:02:33.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:33.065+0000 7f62fa28a740 1 -- 172.21.15.5:0/2994276294 shutdown_connections 2024-09-10T10:02:33.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:33.065+0000 7f62fa28a740 1 --2- 172.21.15.5:0/2994276294 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55e434f21550 0x55e434f23970 crc :-1 s=READY pgs=326 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:33.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:33.065+0000 7f62fa28a740 1 --2- 172.21.15.5:0/2994276294 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e434ea5ca0 0x55e434eaab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:33.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:33.065+0000 7f62fa28a740 1 --2- 172.21.15.5:0/2994276294 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55e434e9b3e0 0x55e434eae070 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:33.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:33.065+0000 7f62fa28a740 1 --2- 172.21.15.5:0/2994276294 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55e434e9aac0 0x55e434eadb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:33.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:33.065+0000 7f62d6a00640 0 client.30005 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T10:02:33.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:33.065+0000 7f62fa28a740 1 -- 172.21.15.5:0/2994276294 >> 172.21.15.5:0/2994276294 conn(0x55e434e97050 msgr2=0x55e434e959a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:33.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:33.065+0000 7f62fa28a740 1 -- 172.21.15.5:0/2994276294 shutdown_connections 2024-09-10T10:02:33.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:33.066+0000 7f62fa28a740 1 -- 172.21.15.5:0/2994276294 wait complete. 2024-09-10T10:02:33.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:33.066+0000 7f62fa28a740 20 client.30005 trim_cache size 0 max 16384 2024-09-10T10:02:33.071 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-10T10:02:33.071 INFO:teuthology.orchestra.run.smithi005.stdout: "return_code": 0, 2024-09-10T10:02:33.071 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_tag": "4a1f5597-8801-4d08-ac33-4ad9220a7597", 2024-09-10T10:02:33.071 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": "asynchronous" 2024-09-10T10:02:33.071 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-10T10:02:33.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:33.066+0000 7f62f7800640 1 -- 172.21.15.5:0/3210463986 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f62d80a2b80 msgr2=0x7f62d80a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:33.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:33.066+0000 7f62f7800640 1 --2- 172.21.15.5:0/3210463986 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f62d80a2b80 0x7f62d80a5040 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f62e409bcc0 tx=0x7f62e4004a40 comp rx=0 tx=0).stop 2024-09-10T10:02:33.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:33.066+0000 7f62f7800640 1 -- 172.21.15.5:0/3210463986 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f62f0103ff0 msgr2=0x7f62f019a660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:33.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:33.066+0000 7f62f7800640 1 --2- 172.21.15.5:0/3210463986 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f62f0103ff0 0x7f62f019a660 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f62e400e9f0 tx=0x7f62e400edf0 comp rx=0 tx=0).stop 2024-09-10T10:02:33.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:33.067+0000 7f62f7800640 1 -- 172.21.15.5:0/3210463986 shutdown_connections 2024-09-10T10:02:33.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:33.067+0000 7f62f7800640 1 --2- 172.21.15.5:0/3210463986 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f62d80a2b80 0x7f62d80a5040 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:33.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:33.067+0000 7f62f7800640 1 --2- 172.21.15.5:0/3210463986 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f62f0109120 0x7f62f019ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:33.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:33.067+0000 7f62f7800640 1 --2- 172.21.15.5:0/3210463986 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f62f01049a0 0x7f62f019aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:33.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:33.067+0000 7f62f7800640 1 --2- 172.21.15.5:0/3210463986 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f62f0103ff0 0x7f62f019a660 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:33.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:33.067+0000 7f62f7800640 1 -- 172.21.15.5:0/3210463986 >> 172.21.15.5:0/3210463986 conn(0x7f62f00ffa90 msgr2=0x7f62f0100540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:33.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:33.067+0000 7f62f7800640 1 -- 172.21.15.5:0/3210463986 shutdown_connections 2024-09-10T10:02:33.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:33.067+0000 7f62f7800640 1 -- 172.21.15.5:0/3210463986 wait complete. 2024-09-10T10:02:33.073 INFO:tasks.workunit.client.0.smithi005.stdout: 10 156147 12.86 MB/sec execute 491 sec latency 11895.979 ms 2024-09-10T10:02:33.306 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:02:33 smithi005 ceph-mon[30350]: pgmap v1119: 129 pgs: 129 active+clean; 726 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 331 KiB/s rd, 852 KiB/s wr, 137 op/s 2024-09-10T10:02:33.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:02:33 smithi159 ceph-mon[35513]: pgmap v1119: 129 pgs: 129 active+clean; 726 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 331 KiB/s rd, 852 KiB/s wr, 137 op/s 2024-09-10T10:02:33.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:02:33 smithi102 ceph-mon[35442]: pgmap v1119: 129 pgs: 129 active+clean; 726 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 331 KiB/s rd, 852 KiB/s wr, 137 op/s 2024-09-10T10:02:33.836 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T10:02:33.851 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:02:34.069 INFO:tasks.workunit.client.0.smithi005.stdout: 10 156147 12.83 MB/sec execute 492 sec latency 12896.037 ms 2024-09-10T10:02:34.346 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:02:34 smithi005 ceph-mon[30350]: scrub queued for path: / 2024-09-10T10:02:34.346 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:02:34 smithi005 ceph-mon[30350]: scrub summary: idle+waiting paths [/] 2024-09-10T10:02:34.346 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:02:34 smithi005 ceph-mon[30350]: scrub summary: active paths [/] 2024-09-10T10:02:34.346 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:02:34 smithi005 ceph-mon[30350]: scrub summary: active paths [/] 2024-09-10T10:02:34.346 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:02:34 smithi005 ceph-mon[30350]: scrub summary: active paths [/] 2024-09-10T10:02:34.346 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:02:34 smithi005 ceph-mon[30350]: scrub summary: idle+waiting paths [/] 2024-09-10T10:02:34.346 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:02:34 smithi005 ceph-mon[30350]: scrub summary: idle+waiting paths [/] 2024-09-10T10:02:34.346 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:02:34 smithi005 ceph-mon[30350]: scrub summary: active paths [/] 2024-09-10T10:02:34.346 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:02:34 smithi005 ceph-mon[30350]: scrub summary: active paths [/] 2024-09-10T10:02:34.346 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:02:34 smithi005 ceph-mon[30350]: scrub summary: idle+waiting paths [/] 2024-09-10T10:02:34.347 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:02:34 smithi005 ceph-mon[30350]: scrub summary: active paths [/] 2024-09-10T10:02:34.347 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:02:34 smithi005 ceph-mon[30350]: scrub summary: idle+waiting paths [/] 2024-09-10T10:02:34.347 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:02:34 smithi005 ceph-mon[30350]: scrub summary: idle+waiting paths [/] 2024-09-10T10:02:34.347 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:02:34 smithi005 ceph-mon[30350]: scrub summary: active paths [/] 2024-09-10T10:02:34.347 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:02:34 smithi005 ceph-mon[30350]: scrub summary: idle+waiting paths [/] 2024-09-10T10:02:34.347 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:02:34 smithi005 ceph-mon[30350]: scrub summary: idle+waiting paths [/] 2024-09-10T10:02:34.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:34.465+0000 7f1a96000640 1 -- 172.21.15.5:0/3084978014 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1a90109140 msgr2=0x7f1a90110cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:34.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:34.465+0000 7f1a96000640 1 --2- 172.21.15.5:0/3084978014 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1a90109140 0x7f1a90110cc0 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f1a88009ca0 tx=0x7f1a8802a8c0 comp rx=0 tx=0).stop 2024-09-10T10:02:34.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:34.465+0000 7f1a96000640 1 -- 172.21.15.5:0/3084978014 shutdown_connections 2024-09-10T10:02:34.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:34.465+0000 7f1a96000640 1 --2- 172.21.15.5:0/3084978014 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1a90109140 0x7f1a90110cc0 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:34.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:34.465+0000 7f1a96000640 1 --2- 172.21.15.5:0/3084978014 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1a901049c0 0x7f1a90108a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:34.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:34.465+0000 7f1a96000640 1 --2- 172.21.15.5:0/3084978014 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1a90104010 0x7f1a901043f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:34.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:34.465+0000 7f1a96000640 1 -- 172.21.15.5:0/3084978014 >> 172.21.15.5:0/3084978014 conn(0x7f1a900ffa50 msgr2=0x7f1a90101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:34.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:34.465+0000 7f1a96000640 1 -- 172.21.15.5:0/3084978014 shutdown_connections 2024-09-10T10:02:34.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:34.465+0000 7f1a96000640 1 -- 172.21.15.5:0/3084978014 wait complete. 2024-09-10T10:02:34.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:34.466+0000 7f1a96000640 1 Processor -- start 2024-09-10T10:02:34.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:34.466+0000 7f1a96000640 1 -- start start 2024-09-10T10:02:34.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:34.466+0000 7f1a96000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1a90104010 0x7f1a9019a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:34.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:34.466+0000 7f1a96000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1a901049c0 0x7f1a9019ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:34.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:34.466+0000 7f1a96000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1a90109140 0x7f1a9019eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:34.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:34.466+0000 7f1a96000640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1a90114150 con 0x7f1a90104010 2024-09-10T10:02:34.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:34.466+0000 7f1a96000640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f1a90113fd0 con 0x7f1a90109140 2024-09-10T10:02:34.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:34.466+0000 7f1a96000640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f1a901142d0 con 0x7f1a901049c0 2024-09-10T10:02:34.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:34.467+0000 7f1a8f400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1a90104010 0x7f1a9019a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:34.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:34.467+0000 7f1a8f400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1a90104010 0x7f1a9019a5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49530/0 (socket says 172.21.15.5:49530) 2024-09-10T10:02:34.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:34.467+0000 7f1a8f400640 1 -- 172.21.15.5:0/4222484916 learned_addr learned my addr 172.21.15.5:0/4222484916 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T10:02:34.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:34.467+0000 7f1a8fe00640 1 --2- 172.21.15.5:0/4222484916 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1a90109140 0x7f1a9019eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:34.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:34.467+0000 7f1a86a00640 1 --2- 172.21.15.5:0/4222484916 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1a901049c0 0x7f1a9019ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:34.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:34.467+0000 7f1a8f400640 1 -- 172.21.15.5:0/4222484916 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1a901049c0 msgr2=0x7f1a9019ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:34.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:34.467+0000 7f1a8f400640 1 --2- 172.21.15.5:0/4222484916 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1a901049c0 0x7f1a9019ab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:34.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:34.467+0000 7f1a8f400640 1 -- 172.21.15.5:0/4222484916 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1a90109140 msgr2=0x7f1a9019eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:34.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:34.467+0000 7f1a8f400640 1 --2- 172.21.15.5:0/4222484916 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1a90109140 0x7f1a9019eec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:34.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:34.467+0000 7f1a8f400640 1 -- 172.21.15.5:0/4222484916 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a9019f5a0 con 0x7f1a90104010 2024-09-10T10:02:34.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:34.467+0000 7f1a8fe00640 1 --2- 172.21.15.5:0/4222484916 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1a90109140 0x7f1a9019eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:02:34.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:34.468+0000 7f1a8f400640 1 --2- 172.21.15.5:0/4222484916 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1a90104010 0x7f1a9019a5f0 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f1a8000d950 tx=0x7f1a8000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:34.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:34.468+0000 7f1a8cc00640 1 -- 172.21.15.5:0/4222484916 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a80017030 con 0x7f1a90104010 2024-09-10T10:02:34.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:34.468+0000 7f1a96000640 1 -- 172.21.15.5:0/4222484916 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a9019f890 con 0x7f1a90104010 2024-09-10T10:02:34.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:34.468+0000 7f1a8cc00640 1 -- 172.21.15.5:0/4222484916 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1a80004020 con 0x7f1a90104010 2024-09-10T10:02:34.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:34.468+0000 7f1a8cc00640 1 -- 172.21.15.5:0/4222484916 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a800047a0 con 0x7f1a90104010 2024-09-10T10:02:34.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:34.468+0000 7f1a96000640 1 -- 172.21.15.5:0/4222484916 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a901a7170 con 0x7f1a90104010 2024-09-10T10:02:34.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:34.471+0000 7f1a98c5f740 1 Processor -- start 2024-09-10T10:02:34.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:34.471+0000 7f1a8cc00640 1 -- 172.21.15.5:0/4222484916 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f1a800041d0 con 0x7f1a90104010 2024-09-10T10:02:34.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:34.472+0000 7f1a98c5f740 1 -- start start 2024-09-10T10:02:34.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:34.472+0000 7f1a98c5f740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5613eb497310 0x5613eb4976f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:34.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:34.472+0000 7f1a98c5f740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5613eb497c30 0x5613eb4a1fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:34.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:34.472+0000 7f1a98c5f740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5613eb4a24f0 0x5613eb4a48e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:34.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:34.472+0000 7f1a98c5f740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5613eb3fd650 con 0x5613eb497310 2024-09-10T10:02:34.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:34.472+0000 7f1a98c5f740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x5613eb3a4f10 con 0x5613eb4a24f0 2024-09-10T10:02:34.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:34.472+0000 7f1a98c5f740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5613eb3f9610 con 0x5613eb497c30 2024-09-10T10:02:34.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:34.472+0000 7f1a86a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5613eb4a24f0 0x5613eb4a48e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:34.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:34.472+0000 7f1a8fe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5613eb497310 0x5613eb4976f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:34.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:34.472+0000 7f1a86a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5613eb4a24f0 0x5613eb4a48e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:58496/0 (socket says 172.21.15.5:58496) 2024-09-10T10:02:34.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:34.472+0000 7f1a86a00640 1 -- 172.21.15.5:0/2620543494 learned_addr learned my addr 172.21.15.5:0/2620543494 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T10:02:34.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:34.473+0000 7f1a8f400640 1 --2- 172.21.15.5:0/2620543494 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5613eb497c30 0x5613eb4a1fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:34.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:34.473+0000 7f1a8fe00640 1 -- 172.21.15.5:0/2620543494 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5613eb497c30 msgr2=0x5613eb4a1fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:34.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:34.473+0000 7f1a8fe00640 1 --2- 172.21.15.5:0/2620543494 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5613eb497c30 0x5613eb4a1fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:34.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:34.473+0000 7f1a8fe00640 1 -- 172.21.15.5:0/2620543494 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5613eb4a24f0 msgr2=0x5613eb4a48e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:34.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:34.473+0000 7f1a8fe00640 1 --2- 172.21.15.5:0/2620543494 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5613eb4a24f0 0x5613eb4a48e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:34.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:34.473+0000 7f1a8fe00640 1 -- 172.21.15.5:0/2620543494 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5613eb10c580 con 0x5613eb497310 2024-09-10T10:02:34.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:34.473+0000 7f1a86a00640 1 --2- 172.21.15.5:0/2620543494 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5613eb4a24f0 0x5613eb4a48e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:02:34.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:34.473+0000 7f1a8fe00640 1 --2- 172.21.15.5:0/2620543494 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5613eb497310 0x5613eb4976f0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f1a88006010 tx=0x7f1a88009800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e591572bbba232b0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:34.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:34.473+0000 7f1a8cc00640 1 --2- 172.21.15.5:0/4222484916 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f1a740a2c00 0x7f1a740a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:34.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:34.474+0000 7f1a8cc00640 1 -- 172.21.15.5:0/4222484916 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f1a800ceda0 con 0x7f1a90104010 2024-09-10T10:02:34.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:34.474+0000 7f1a7fe00640 1 -- 172.21.15.5:0/2620543494 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a88003b70 con 0x5613eb497310 2024-09-10T10:02:34.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:34.474+0000 7f1a7fe00640 1 -- 172.21.15.5:0/2620543494 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1a88036040 con 0x5613eb497310 2024-09-10T10:02:34.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:34.474+0000 7f1a7fe00640 1 -- 172.21.15.5:0/2620543494 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a880046a0 con 0x5613eb497310 2024-09-10T10:02:34.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:34.474+0000 7f1a8f400640 1 --2- 172.21.15.5:0/4222484916 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f1a740a2c00 0x7f1a740a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:34.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:34.475+0000 7f1a8f400640 1 --2- 172.21.15.5:0/4222484916 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f1a740a2c00 0x7f1a740a50c0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f1a80097b80 tx=0x7f1a80000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:34.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:34.475+0000 7f1a98c5f740 1 -- 172.21.15.5:0/2620543494 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5613eb497310 msgr2=0x5613eb4976f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:34.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:34.475+0000 7f1a98c5f740 1 --2- 172.21.15.5:0/2620543494 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5613eb497310 0x5613eb4976f0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f1a88006010 tx=0x7f1a88009800 comp rx=0 tx=0).stop 2024-09-10T10:02:34.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:34.475+0000 7f1a98c5f740 1 -- 172.21.15.5:0/2620543494 shutdown_connections 2024-09-10T10:02:34.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:34.475+0000 7f1a98c5f740 1 --2- 172.21.15.5:0/2620543494 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5613eb4a24f0 0x5613eb4a48e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:34.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:34.475+0000 7f1a98c5f740 1 --2- 172.21.15.5:0/2620543494 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5613eb497c30 0x5613eb4a1fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:34.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:34.475+0000 7f1a98c5f740 1 --2- 172.21.15.5:0/2620543494 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5613eb497310 0x5613eb4976f0 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:34.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:34.475+0000 7f1a98c5f740 1 -- 172.21.15.5:0/2620543494 >> 172.21.15.5:0/2620543494 conn(0x5613eb4938a0 msgr2=0x5613eb493c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:34.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:34.475+0000 7f1a98c5f740 1 -- 172.21.15.5:0/2620543494 shutdown_connections 2024-09-10T10:02:34.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:34.475+0000 7f1a98c5f740 1 -- 172.21.15.5:0/2620543494 wait complete. 2024-09-10T10:02:34.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:34.476+0000 7f1a98c5f740 1 -- start start 2024-09-10T10:02:34.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:34.476+0000 7f1a98c5f740 1 Processor -- start 2024-09-10T10:02:34.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:34.477+0000 7f1a98c5f740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5613eb497310 0x5613eb49f450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:34.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:34.477+0000 7f1a98c5f740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5613eb497c30 0x5613eb49f990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:34.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:34.477+0000 7f1a98c5f740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5613eb4a24f0 0x5613eb49fed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:34.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:34.477+0000 7f1a98c5f740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5613eb4b5f50 con 0x5613eb497c30 2024-09-10T10:02:34.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:34.477+0000 7f1a98c5f740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x5613eb3b8350 con 0x5613eb497310 2024-09-10T10:02:34.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:34.477+0000 7f1a98c5f740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5613eb4b60d0 con 0x5613eb4a24f0 2024-09-10T10:02:34.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:34.477+0000 7f1a98c5f740 10 client.0 _ll_register_callbacks cb 0x5613eb3db290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:02:34.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:34.477+0000 7f1a8fe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5613eb497310 0x5613eb49f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:34.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:34.477+0000 7f1a8fe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5613eb497310 0x5613eb49f450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:58508/0 (socket says 172.21.15.5:58508) 2024-09-10T10:02:34.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:34.477+0000 7f1a8fe00640 1 -- 172.21.15.5:0/3433305544 learned_addr learned my addr 172.21.15.5:0/3433305544 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T10:02:34.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:34.477+0000 7f1a86a00640 1 --2- 172.21.15.5:0/3433305544 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5613eb497c30 0x5613eb49f990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:34.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:34.478+0000 7f1a8fe00640 1 --2- 172.21.15.5:0/3433305544 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5613eb4a24f0 0x5613eb49fed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:34.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:34.478+0000 7f1a86a00640 1 -- 172.21.15.5:0/3433305544 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5613eb4a24f0 msgr2=0x5613eb49fed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:34.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:34.478+0000 7f1a86a00640 1 --2- 172.21.15.5:0/3433305544 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5613eb4a24f0 0x5613eb49fed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:34.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:34.478+0000 7f1a86a00640 1 -- 172.21.15.5:0/3433305544 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5613eb497310 msgr2=0x5613eb49f450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:34.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:34.478+0000 7f1a86a00640 1 --2- 172.21.15.5:0/3433305544 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5613eb497310 0x5613eb49f450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:34.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:34.478+0000 7f1a86a00640 1 -- 172.21.15.5:0/3433305544 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5613eb1d6960 con 0x5613eb497c30 2024-09-10T10:02:34.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:34.478+0000 7f1a8fe00640 1 --2- 172.21.15.5:0/3433305544 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5613eb497310 0x5613eb49f450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:02:34.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:34.478+0000 7f1a86a00640 1 --2- 172.21.15.5:0/3433305544 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5613eb497c30 0x5613eb49f990 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f1a78009560 tx=0x7f1a78012b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5614a29bf12e33fd server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:34.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:34.478+0000 7f1a7cc00640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-10T10:02:34.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:34.479+0000 7f1a7cc00640 1 -- 172.21.15.5:0/3433305544 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a78004050 con 0x5613eb497c30 2024-09-10T10:02:34.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:34.479+0000 7f1a7cc00640 1 -- 172.21.15.5:0/3433305544 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f1a780041f0 con 0x5613eb497c30 2024-09-10T10:02:34.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:34.479+0000 7f1a98c5f740 1 -- 172.21.15.5:0/3433305544 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1a88036040 con 0x5613eb497c30 2024-09-10T10:02:34.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:34.479+0000 7f1a7cc00640 1 -- 172.21.15.5:0/3433305544 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f1a78025050 con 0x5613eb497c30 2024-09-10T10:02:34.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:34.479+0000 7f1a98c5f740 10 client.19923 fetch_fsmap learned FSMap version 42 2024-09-10T10:02:34.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:34.479+0000 7f1a98c5f740 1 -- 172.21.15.5:0/3433305544 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x5613eb4c1c30 con 0x5613eb497c30 2024-09-10T10:02:34.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:34.479+0000 7f1a7cc00640 1 -- 172.21.15.5:0/3433305544 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1a88036040 con 0x5613eb497c30 2024-09-10T10:02:34.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:34.479+0000 7f1a7cc00640 1 -- 172.21.15.5:0/3433305544 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 42) ==== 8370+0+0 (secure 0 0 0) 0x7f1a78003af0 con 0x5613eb497c30 2024-09-10T10:02:34.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:34.480+0000 7f1a98c5f740 10 client.19923 fetch_fsmap finished waiting for FSMap version 42 2024-09-10T10:02:34.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:34.480+0000 7f1a98c5f740 10 client.19923 resolve_mds: validated gid 24542 aka daemon mds.b 2024-09-10T10:02:34.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:34.480+0000 7f1a98c5f740 20 client.19923 populate_metadata read hostname 'smithi005' 2024-09-10T10:02:34.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:34.480+0000 7f1a98c5f740 1 --2- 172.21.15.5:0/3433305544 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x5613eb4c30f0 0x5613eb4c5520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:34.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:34.480+0000 7f1a98c5f740 4 client.19923 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:02:34.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:34.480+0000 7f1a98c5f740 1 -- 172.21.15.5:0/3433305544 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5613eb40abc0 con 0x5613eb4c30f0 2024-09-10T10:02:34.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:34.480+0000 7f1a8fe00640 1 --2- 172.21.15.5:0/3433305544 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x5613eb4c30f0 0x5613eb4c5520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:34.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:34.481+0000 7f1a8fe00640 1 --2- 172.21.15.5:0/3433305544 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x5613eb4c30f0 0x5613eb4c5520 crc :-1 s=READY pgs=327 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f735ee860ea95472 server_cookie=bd3431a2e67f9162 in_seq=0 out_seq=0 2024-09-10T10:02:34.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:34.481+0000 7f1a7cc00640 10 client.19923 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T10:02:34.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:34.481+0000 7f1a7cc00640 1 -- 172.21.15.5:0/3433305544 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5613eb40abc0 con 0x5613eb4c30f0 2024-09-10T10:02:34.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:34.481+0000 7f1a7cc00640 10 client.19923 handle_command_reply: tid=0 2024-09-10T10:02:34.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:34.481+0000 7f1a98c5f740 1 client.19923 shutdown 2024-09-10T10:02:34.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:34.482+0000 7f1a98c5f740 1 -- 172.21.15.5:0/3433305544 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5613eb497c30 msgr2=0x5613eb49f990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:34.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:34.482+0000 7f1a98c5f740 1 --2- 172.21.15.5:0/3433305544 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5613eb497c30 0x5613eb49f990 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f1a78009560 tx=0x7f1a78012b60 comp rx=0 tx=0).stop 2024-09-10T10:02:34.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:34.482+0000 7f1a98c5f740 1 -- 172.21.15.5:0/3433305544 shutdown_connections 2024-09-10T10:02:34.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:34.482+0000 7f1a98c5f740 1 --2- 172.21.15.5:0/3433305544 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x5613eb4c30f0 0x5613eb4c5520 crc :-1 s=READY pgs=327 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:34.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:34.482+0000 7f1a98c5f740 1 --2- 172.21.15.5:0/3433305544 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5613eb4a24f0 0x5613eb49fed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:34.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:34.482+0000 7f1a98c5f740 1 --2- 172.21.15.5:0/3433305544 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5613eb497c30 0x5613eb49f990 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:34.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:34.482+0000 7f1a98c5f740 1 --2- 172.21.15.5:0/3433305544 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5613eb497310 0x5613eb49f450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:34.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:34.482+0000 7f1a98c5f740 1 -- 172.21.15.5:0/3433305544 >> 172.21.15.5:0/3433305544 conn(0x5613eb4938a0 msgr2=0x5613eb492aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:34.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:34.482+0000 7f1a7cc00640 0 client.19923 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T10:02:34.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:34.482+0000 7f1a98c5f740 1 -- 172.21.15.5:0/3433305544 shutdown_connections 2024-09-10T10:02:34.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:34.482+0000 7f1a98c5f740 1 -- 172.21.15.5:0/3433305544 wait complete. 2024-09-10T10:02:34.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:34.482+0000 7f1a98c5f740 20 client.19923 trim_cache size 0 max 16384 2024-09-10T10:02:34.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:34.500+0000 7f1a98c5f740 1 Processor -- start 2024-09-10T10:02:34.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:34.501+0000 7f1a98c5f740 1 -- start start 2024-09-10T10:02:34.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:34.501+0000 7f1a98c5f740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5613eb497310 0x5613eb49fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:34.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:34.501+0000 7f1a98c5f740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5613eb497c30 0x5613eb4a12b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:34.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:34.501+0000 7f1a98c5f740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5613eb4a24f0 0x5613eb4cae30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:34.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:34.501+0000 7f1a98c5f740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1a901a87c0 con 0x5613eb4a24f0 2024-09-10T10:02:34.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:34.501+0000 7f1a98c5f740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f1a901aa300 con 0x5613eb497c30 2024-09-10T10:02:34.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:34.501+0000 7f1a98c5f740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f1a901a8e90 con 0x5613eb497310 2024-09-10T10:02:34.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:34.501+0000 7f1a8fe00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5613eb497310 0x5613eb49fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:34.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:34.501+0000 7f1a86a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5613eb497c30 0x5613eb4a12b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:34.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:34.501+0000 7f1a8fe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5613eb4a24f0 0x5613eb4cae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:34.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:34.501+0000 7f1a8fe00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5613eb497310 0x5613eb49fd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:35366/0 (socket says 172.21.15.5:35366) 2024-09-10T10:02:34.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:34.501+0000 7f1a8fe00640 1 -- 172.21.15.5:0/3170478386 learned_addr learned my addr 172.21.15.5:0/3170478386 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T10:02:34.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:34.501+0000 7f1a86a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5613eb497c30 0x5613eb4a12b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:58514/0 (socket says 172.21.15.5:58514) 2024-09-10T10:02:34.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:34.502+0000 7f1a86a00640 1 -- 172.21.15.5:0/3170478386 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5613eb497310 msgr2=0x5613eb49fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:34.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:34.502+0000 7f1a86a00640 1 --2- 172.21.15.5:0/3170478386 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5613eb497310 0x5613eb49fd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:34.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:34.502+0000 7f1a86a00640 1 -- 172.21.15.5:0/3170478386 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5613eb4a24f0 msgr2=0x5613eb4cae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:34.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:34.502+0000 7f1a86a00640 1 --2- 172.21.15.5:0/3170478386 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5613eb4a24f0 0x5613eb4cae30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:34.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:34.502+0000 7f1a86a00640 1 -- 172.21.15.5:0/3170478386 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5613eb499bf0 con 0x5613eb497c30 2024-09-10T10:02:34.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:34.502+0000 7f1a86a00640 1 --2- 172.21.15.5:0/3170478386 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5613eb497c30 0x5613eb4a12b0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x5613eb4a0a00 tx=0x7f1a78009640 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7af693d5b8ef043c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:34.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:34.502+0000 7f1a7f400640 1 -- 172.21.15.5:0/3170478386 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a7802d020 con 0x5613eb497c30 2024-09-10T10:02:34.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:34.503+0000 7f1a7f400640 1 -- 172.21.15.5:0/3170478386 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1a7800fe00 con 0x5613eb497c30 2024-09-10T10:02:34.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:34.503+0000 7f1a7f400640 1 -- 172.21.15.5:0/3170478386 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a780035f0 con 0x5613eb497c30 2024-09-10T10:02:34.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:34.503+0000 7f1a98c5f740 1 -- 172.21.15.5:0/3170478386 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5613eb497c30 msgr2=0x5613eb4a12b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:34.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:34.503+0000 7f1a98c5f740 1 --2- 172.21.15.5:0/3170478386 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5613eb497c30 0x5613eb4a12b0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x5613eb4a0a00 tx=0x7f1a78009640 comp rx=0 tx=0).stop 2024-09-10T10:02:34.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:34.503+0000 7f1a98c5f740 1 -- 172.21.15.5:0/3170478386 shutdown_connections 2024-09-10T10:02:34.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:34.503+0000 7f1a98c5f740 1 --2- 172.21.15.5:0/3170478386 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5613eb4a24f0 0x5613eb4cae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:34.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:34.503+0000 7f1a98c5f740 1 --2- 172.21.15.5:0/3170478386 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5613eb497c30 0x5613eb4a12b0 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:34.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:34.503+0000 7f1a98c5f740 1 --2- 172.21.15.5:0/3170478386 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5613eb497310 0x5613eb49fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:34.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:34.503+0000 7f1a98c5f740 1 -- 172.21.15.5:0/3170478386 >> 172.21.15.5:0/3170478386 conn(0x5613eb4938a0 msgr2=0x5613eb4a0ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:34.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:34.504+0000 7f1a98c5f740 1 -- 172.21.15.5:0/3170478386 shutdown_connections 2024-09-10T10:02:34.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:34.504+0000 7f1a98c5f740 1 -- 172.21.15.5:0/3170478386 wait complete. 2024-09-10T10:02:34.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:34.504+0000 7f1a98c5f740 1 -- start start 2024-09-10T10:02:34.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:34.504+0000 7f1a98c5f740 1 Processor -- start 2024-09-10T10:02:34.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:34.505+0000 7f1a98c5f740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5613eb497310 0x5613eb4bb320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:34.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:34.505+0000 7f1a98c5f740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5613eb497c30 0x5613eb4bb860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:34.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:34.505+0000 7f1a98c5f740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5613eb4a24f0 0x5613eb4b82f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:34.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:34.505+0000 7f1a98c5f740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1a901a9c30 con 0x5613eb497c30 2024-09-10T10:02:34.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:34.505+0000 7f1a98c5f740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f1a901a9560 con 0x5613eb497310 2024-09-10T10:02:34.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:34.505+0000 7f1a98c5f740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f1a90073810 con 0x5613eb4a24f0 2024-09-10T10:02:34.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:34.505+0000 7f1a98c5f740 10 client.0 _ll_register_callbacks cb 0x7f1a78009930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:02:34.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:34.505+0000 7f1a86a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5613eb497c30 0x5613eb4bb860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:34.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:34.505+0000 7f1a86a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5613eb497c30 0x5613eb4bb860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49564/0 (socket says 172.21.15.5:49564) 2024-09-10T10:02:34.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:34.505+0000 7f1a86a00640 1 -- 172.21.15.5:0/3934806351 learned_addr learned my addr 172.21.15.5:0/3934806351 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T10:02:34.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:34.505+0000 7f1a8fe00640 1 --2- 172.21.15.5:0/3934806351 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5613eb4a24f0 0x5613eb4b82f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:34.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:34.505+0000 7f1a8fe00640 1 --2- 172.21.15.5:0/3934806351 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5613eb497310 0x5613eb4bb320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:34.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:34.505+0000 7f1a86a00640 1 -- 172.21.15.5:0/3934806351 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5613eb4a24f0 msgr2=0x5613eb4b82f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:34.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:34.505+0000 7f1a86a00640 1 --2- 172.21.15.5:0/3934806351 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5613eb4a24f0 0x5613eb4b82f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:34.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:34.505+0000 7f1a86a00640 1 -- 172.21.15.5:0/3934806351 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5613eb497310 msgr2=0x5613eb4bb320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:34.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:34.505+0000 7f1a86a00640 1 --2- 172.21.15.5:0/3934806351 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5613eb497310 0x5613eb4bb320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:34.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:34.505+0000 7f1a86a00640 1 -- 172.21.15.5:0/3934806351 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5613eb4b8830 con 0x5613eb497c30 2024-09-10T10:02:34.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:34.505+0000 7f1a8fe00640 1 --2- 172.21.15.5:0/3934806351 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5613eb497310 0x5613eb4bb320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:02:34.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:34.506+0000 7f1a86a00640 1 --2- 172.21.15.5:0/3934806351 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5613eb497c30 0x5613eb4bb860 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f1a78003ec0 tx=0x7f1a7800fbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a896435d9fee612 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:34.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:34.506+0000 7f1a7cc00640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-10T10:02:34.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:34.506+0000 7f1a7cc00640 1 -- 172.21.15.5:0/3934806351 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a78036020 con 0x5613eb497c30 2024-09-10T10:02:34.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:34.506+0000 7f1a7cc00640 1 -- 172.21.15.5:0/3934806351 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f1a78022040 con 0x5613eb497c30 2024-09-10T10:02:34.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:34.506+0000 7f1a98c5f740 1 -- 172.21.15.5:0/3934806351 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1a7800fe00 con 0x5613eb497c30 2024-09-10T10:02:34.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:34.507+0000 7f1a7cc00640 1 -- 172.21.15.5:0/3934806351 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f1a78041050 con 0x5613eb497c30 2024-09-10T10:02:34.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:34.507+0000 7f1a98c5f740 10 client.19929 fetch_fsmap learned FSMap version 42 2024-09-10T10:02:34.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:34.507+0000 7f1a98c5f740 1 -- 172.21.15.5:0/3934806351 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x5613eb52e600 con 0x5613eb497c30 2024-09-10T10:02:34.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:34.507+0000 7f1a7cc00640 1 -- 172.21.15.5:0/3934806351 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1a7800fe00 con 0x5613eb497c30 2024-09-10T10:02:34.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:34.507+0000 7f1a7cc00640 1 -- 172.21.15.5:0/3934806351 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 42) ==== 8370+0+0 (secure 0 0 0) 0x7f1a780107d0 con 0x5613eb497c30 2024-09-10T10:02:34.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:34.507+0000 7f1a98c5f740 10 client.19929 fetch_fsmap finished waiting for FSMap version 42 2024-09-10T10:02:34.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:34.507+0000 7f1a98c5f740 10 client.19929 resolve_mds: validated gid 24542 aka daemon mds.b 2024-09-10T10:02:34.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:34.507+0000 7f1a98c5f740 20 client.19929 populate_metadata read hostname 'smithi005' 2024-09-10T10:02:34.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:34.507+0000 7f1a98c5f740 1 --2- 172.21.15.5:0/3934806351 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x5613eb52ec30 0x5613eb531050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:34.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:34.507+0000 7f1a98c5f740 4 client.19929 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "b7188858", "release": true, "await": true}] 2024-09-10T10:02:34.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:34.507+0000 7f1a98c5f740 1 -- 172.21.15.5:0/3934806351 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "quiesce db", "set_id": "b7188858", "release": true, "await": true}) -- 0x5613eb4050d0 con 0x5613eb52ec30 2024-09-10T10:02:34.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:34.508+0000 7f1a8fe00640 1 --2- 172.21.15.5:0/3934806351 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x5613eb52ec30 0x5613eb531050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:34.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:34.508+0000 7f1a8fe00640 1 --2- 172.21.15.5:0/3934806351 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x5613eb52ec30 0x5613eb531050 crc :-1 s=READY pgs=328 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bbcaa6117b91c360 server_cookie=25c3cf6a1dbddcad in_seq=0 out_seq=0 2024-09-10T10:02:34.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:34.508+0000 7f1a7cc00640 10 client.19929 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T10:02:34.536 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:02:34 smithi102 ceph-mon[35442]: scrub queued for path: / 2024-09-10T10:02:34.536 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:02:34 smithi102 ceph-mon[35442]: scrub summary: idle+waiting paths [/] 2024-09-10T10:02:34.536 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:02:34 smithi102 ceph-mon[35442]: scrub summary: active paths [/] 2024-09-10T10:02:34.536 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:02:34 smithi102 ceph-mon[35442]: scrub summary: active paths [/] 2024-09-10T10:02:34.536 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:02:34 smithi102 ceph-mon[35442]: scrub summary: active paths [/] 2024-09-10T10:02:34.536 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:02:34 smithi102 ceph-mon[35442]: scrub summary: idle+waiting paths [/] 2024-09-10T10:02:34.537 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:02:34 smithi102 ceph-mon[35442]: scrub summary: idle+waiting paths [/] 2024-09-10T10:02:34.537 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:02:34 smithi102 ceph-mon[35442]: scrub summary: active paths [/] 2024-09-10T10:02:34.537 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:02:34 smithi102 ceph-mon[35442]: scrub summary: active paths [/] 2024-09-10T10:02:34.537 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:02:34 smithi102 ceph-mon[35442]: scrub summary: idle+waiting paths [/] 2024-09-10T10:02:34.537 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:02:34 smithi102 ceph-mon[35442]: scrub summary: active paths [/] 2024-09-10T10:02:34.537 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:02:34 smithi102 ceph-mon[35442]: scrub summary: idle+waiting paths [/] 2024-09-10T10:02:34.537 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:02:34 smithi102 ceph-mon[35442]: scrub summary: idle+waiting paths [/] 2024-09-10T10:02:34.537 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:02:34 smithi102 ceph-mon[35442]: scrub summary: active paths [/] 2024-09-10T10:02:34.537 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:02:34 smithi102 ceph-mon[35442]: scrub summary: idle+waiting paths [/] 2024-09-10T10:02:34.537 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:02:34 smithi102 ceph-mon[35442]: scrub summary: idle+waiting paths [/] 2024-09-10T10:02:34.541 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:02:34.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:34.573+0000 7f1a7cc00640 1 -- 172.21.15.5:0/3934806351 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5613eb4050d0 con 0x5613eb52ec30 2024-09-10T10:02:34.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:34.573+0000 7f1a7cc00640 10 client.19929 handle_command_reply: tid=0 2024-09-10T10:02:34.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:34.573+0000 7f1a98c5f740 1 client.19929 shutdown 2024-09-10T10:02:34.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:34.573+0000 7f1a98c5f740 1 -- 172.21.15.5:0/3934806351 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5613eb497c30 msgr2=0x5613eb4bb860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:34.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:34.573+0000 7f1a98c5f740 1 --2- 172.21.15.5:0/3934806351 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5613eb497c30 0x5613eb4bb860 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f1a78003ec0 tx=0x7f1a7800fbb0 comp rx=0 tx=0).stop 2024-09-10T10:02:34.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:34.573+0000 7f1a98c5f740 1 -- 172.21.15.5:0/3934806351 shutdown_connections 2024-09-10T10:02:34.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:34.573+0000 7f1a98c5f740 1 --2- 172.21.15.5:0/3934806351 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x5613eb52ec30 0x5613eb531050 crc :-1 s=READY pgs=328 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:34.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:34.573+0000 7f1a98c5f740 1 --2- 172.21.15.5:0/3934806351 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5613eb4a24f0 0x5613eb4b82f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:34.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:34.573+0000 7f1a98c5f740 1 --2- 172.21.15.5:0/3934806351 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5613eb497c30 0x5613eb4bb860 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:34.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:34.573+0000 7f1a98c5f740 1 --2- 172.21.15.5:0/3934806351 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5613eb497310 0x5613eb4bb320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:34.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:34.574+0000 7f1a98c5f740 1 -- 172.21.15.5:0/3934806351 >> 172.21.15.5:0/3934806351 conn(0x5613eb4938a0 msgr2=0x5613eb492aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:34.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:34.574+0000 7f1a7cc00640 0 client.19929 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T10:02:34.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:34.574+0000 7f1a98c5f740 1 -- 172.21.15.5:0/3934806351 shutdown_connections 2024-09-10T10:02:34.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:34.574+0000 7f1a98c5f740 1 -- 172.21.15.5:0/3934806351 wait complete. 2024-09-10T10:02:34.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:34.574+0000 7f1a98c5f740 20 client.19929 trim_cache size 0 max 16384 2024-09-10T10:02:34.577 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-10T10:02:34.577 INFO:teuthology.orchestra.run.smithi005.stdout: "epoch": 42, 2024-09-10T10:02:34.577 INFO:teuthology.orchestra.run.smithi005.stdout: "leader": 24542, 2024-09-10T10:02:34.577 INFO:teuthology.orchestra.run.smithi005.stdout: "set_version": 84, 2024-09-10T10:02:34.577 INFO:teuthology.orchestra.run.smithi005.stdout: "sets": { 2024-09-10T10:02:34.577 INFO:teuthology.orchestra.run.smithi005.stdout: "b7188858": { 2024-09-10T10:02:34.577 INFO:teuthology.orchestra.run.smithi005.stdout: "version": 84, 2024-09-10T10:02:34.577 INFO:teuthology.orchestra.run.smithi005.stdout: "age_ref": 0.0, 2024-09-10T10:02:34.577 INFO:teuthology.orchestra.run.smithi005.stdout: "state": { 2024-09-10T10:02:34.577 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "RELEASED", 2024-09-10T10:02:34.577 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0 2024-09-10T10:02:34.578 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:02:34.578 INFO:teuthology.orchestra.run.smithi005.stdout: "timeout": 90.0, 2024-09-10T10:02:34.578 INFO:teuthology.orchestra.run.smithi005.stdout: "expiration": 135.3, 2024-09-10T10:02:34.578 INFO:teuthology.orchestra.run.smithi005.stdout: "members": { 2024-09-10T10:02:34.578 INFO:teuthology.orchestra.run.smithi005.stdout: "file:/": { 2024-09-10T10:02:34.578 INFO:teuthology.orchestra.run.smithi005.stdout: "excluded": false, 2024-09-10T10:02:34.578 INFO:teuthology.orchestra.run.smithi005.stdout: "state": { 2024-09-10T10:02:34.578 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "RELEASED", 2024-09-10T10:02:34.578 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0 2024-09-10T10:02:34.578 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T10:02:34.578 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T10:02:34.578 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T10:02:34.578 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T10:02:34.578 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T10:02:34.579 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-10T10:02:34.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:34.575+0000 7f1a96000640 1 -- 172.21.15.5:0/4222484916 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f1a740a2c00 msgr2=0x7f1a740a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:34.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:34.575+0000 7f1a96000640 1 --2- 172.21.15.5:0/4222484916 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f1a740a2c00 0x7f1a740a50c0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f1a80097b80 tx=0x7f1a80000bc0 comp rx=0 tx=0).stop 2024-09-10T10:02:34.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:34.575+0000 7f1a96000640 1 -- 172.21.15.5:0/4222484916 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1a90104010 msgr2=0x7f1a9019a5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:34.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:34.575+0000 7f1a96000640 1 --2- 172.21.15.5:0/4222484916 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1a90104010 0x7f1a9019a5f0 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f1a8000d950 tx=0x7f1a8000de10 comp rx=0 tx=0).stop 2024-09-10T10:02:34.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:34.575+0000 7f1a96000640 1 -- 172.21.15.5:0/4222484916 shutdown_connections 2024-09-10T10:02:34.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:34.575+0000 7f1a96000640 1 --2- 172.21.15.5:0/4222484916 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f1a740a2c00 0x7f1a740a50c0 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:34.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:34.575+0000 7f1a96000640 1 --2- 172.21.15.5:0/4222484916 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f1a90109140 0x7f1a9019eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:34.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:34.575+0000 7f1a96000640 1 --2- 172.21.15.5:0/4222484916 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f1a901049c0 0x7f1a9019ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:34.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:34.575+0000 7f1a96000640 1 --2- 172.21.15.5:0/4222484916 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1a90104010 0x7f1a9019a5f0 secure :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f1a8000d950 tx=0x7f1a8000de10 comp rx=0 tx=0).stop 2024-09-10T10:02:34.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:34.575+0000 7f1a96000640 1 -- 172.21.15.5:0/4222484916 >> 172.21.15.5:0/4222484916 conn(0x7f1a900ffa50 msgr2=0x7f1a90100d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:34.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:34.576+0000 7f1a96000640 1 -- 172.21.15.5:0/4222484916 shutdown_connections 2024-09-10T10:02:34.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:34.576+0000 7f1a96000640 1 -- 172.21.15.5:0/4222484916 wait complete. 2024-09-10T10:02:34.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:02:34 smithi159 ceph-mon[35513]: scrub queued for path: / 2024-09-10T10:02:34.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:02:34 smithi159 ceph-mon[35513]: scrub summary: idle+waiting paths [/] 2024-09-10T10:02:34.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:02:34 smithi159 ceph-mon[35513]: scrub summary: active paths [/] 2024-09-10T10:02:34.601 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:02:34 smithi159 ceph-mon[35513]: scrub summary: active paths [/] 2024-09-10T10:02:34.601 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:02:34 smithi159 ceph-mon[35513]: scrub summary: active paths [/] 2024-09-10T10:02:34.601 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:02:34 smithi159 ceph-mon[35513]: scrub summary: idle+waiting paths [/] 2024-09-10T10:02:34.601 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:02:34 smithi159 ceph-mon[35513]: scrub summary: idle+waiting paths [/] 2024-09-10T10:02:34.601 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:02:34 smithi159 ceph-mon[35513]: scrub summary: active paths [/] 2024-09-10T10:02:34.601 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:02:34 smithi159 ceph-mon[35513]: scrub summary: active paths [/] 2024-09-10T10:02:34.601 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:02:34 smithi159 ceph-mon[35513]: scrub summary: idle+waiting paths [/] 2024-09-10T10:02:34.601 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:02:34 smithi159 ceph-mon[35513]: scrub summary: active paths [/] 2024-09-10T10:02:34.601 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:02:34 smithi159 ceph-mon[35513]: scrub summary: idle+waiting paths [/] 2024-09-10T10:02:34.601 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:02:34 smithi159 ceph-mon[35513]: scrub summary: idle+waiting paths [/] 2024-09-10T10:02:34.601 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:02:34 smithi159 ceph-mon[35513]: scrub summary: active paths [/] 2024-09-10T10:02:34.601 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:02:34 smithi159 ceph-mon[35513]: scrub summary: idle+waiting paths [/] 2024-09-10T10:02:34.601 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:02:34 smithi159 ceph-mon[35513]: scrub summary: idle+waiting paths [/] 2024-09-10T10:02:35.070 INFO:tasks.workunit.client.0.smithi005.stdout: 10 156433 12.83 MB/sec execute 493 sec latency 13403.385 ms 2024-09-10T10:02:35.546 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:02:35 smithi005 ceph-mon[30350]: pgmap v1120: 129 pgs: 129 active+clean; 726 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 322 KiB/s rd, 401 KiB/s wr, 137 op/s 2024-09-10T10:02:35.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:02:35 smithi159 ceph-mon[35513]: pgmap v1120: 129 pgs: 129 active+clean; 726 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 322 KiB/s rd, 401 KiB/s wr, 137 op/s 2024-09-10T10:02:35.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:02:35 smithi102 ceph-mon[35442]: pgmap v1120: 129 pgs: 129 active+clean; 726 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 322 KiB/s rd, 401 KiB/s wr, 137 op/s 2024-09-10T10:02:35.846 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'b7188858', total seconds elapsed: 21.0 2024-09-10T10:02:36.070 INFO:tasks.workunit.client.0.smithi005.stdout: 10 157161 12.86 MB/sec execute 494 sec latency 54.791 ms 2024-09-10T10:02:36.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:36.090+0000 7fc4d0a00640 1 -- 172.21.15.5:0/2682822831 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc4cc101de0 msgr2=0x7fc4cc1021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:36.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:36.090+0000 7fc4d0a00640 1 --2- 172.21.15.5:0/2682822831 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc4cc101de0 0x7fc4cc1021c0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fc4bc009900 tx=0x7fc4bc02a3f0 comp rx=0 tx=0).stop 2024-09-10T10:02:36.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:36.090+0000 7fc4d0a00640 1 -- 172.21.15.5:0/2682822831 shutdown_connections 2024-09-10T10:02:36.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:36.090+0000 7fc4d0a00640 1 --2- 172.21.15.5:0/2682822831 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc4cc106f10 0x7fc4cc10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:36.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:36.090+0000 7fc4d0a00640 1 --2- 172.21.15.5:0/2682822831 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc4cc102790 0x7fc4cc1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:36.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:36.090+0000 7fc4d0a00640 1 --2- 172.21.15.5:0/2682822831 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc4cc101de0 0x7fc4cc1021c0 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:36.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:36.090+0000 7fc4d0a00640 1 -- 172.21.15.5:0/2682822831 >> 172.21.15.5:0/2682822831 conn(0x7fc4cc0fd8c0 msgr2=0x7fc4cc0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:36.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:36.090+0000 7fc4d0a00640 1 -- 172.21.15.5:0/2682822831 shutdown_connections 2024-09-10T10:02:36.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:36.090+0000 7fc4d0a00640 1 -- 172.21.15.5:0/2682822831 wait complete. 2024-09-10T10:02:36.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:36.091+0000 7fc4d0a00640 1 Processor -- start 2024-09-10T10:02:36.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:36.092+0000 7fc4d0a00640 1 -- start start 2024-09-10T10:02:36.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:36.092+0000 7fc4d0a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc4cc101de0 0x7fc4cc1984a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:36.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:36.092+0000 7fc4d0a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc4cc102790 0x7fc4cc1989e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:36.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:36.092+0000 7fc4d0a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc4cc106f10 0x7fc4cc19cd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:36.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:36.092+0000 7fc4d0a00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc4cc111fc0 con 0x7fc4cc101de0 2024-09-10T10:02:36.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:36.092+0000 7fc4d0a00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fc4cc111e40 con 0x7fc4cc102790 2024-09-10T10:02:36.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:36.092+0000 7fc4d0a00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fc4cc112140 con 0x7fc4cc106f10 2024-09-10T10:02:36.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:36.093+0000 7fc4c9400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc4cc102790 0x7fc4cc1989e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:36.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:36.093+0000 7fc4ca800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc4cc106f10 0x7fc4cc19cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:36.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:36.093+0000 7fc4ca800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc4cc106f10 0x7fc4cc19cd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:35390/0 (socket says 172.21.15.5:35390) 2024-09-10T10:02:36.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:36.093+0000 7fc4ca800640 1 -- 172.21.15.5:0/212032947 learned_addr learned my addr 172.21.15.5:0/212032947 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T10:02:36.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:36.093+0000 7fc4c9400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc4cc102790 0x7fc4cc1989e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:58544/0 (socket says 172.21.15.5:58544) 2024-09-10T10:02:36.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:36.093+0000 7fc4c9e00640 1 --2- 172.21.15.5:0/212032947 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc4cc101de0 0x7fc4cc1984a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:36.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:36.093+0000 7fc4c9400640 1 -- 172.21.15.5:0/212032947 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc4cc106f10 msgr2=0x7fc4cc19cd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:36.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:36.093+0000 7fc4c9400640 1 --2- 172.21.15.5:0/212032947 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc4cc106f10 0x7fc4cc19cd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:36.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:36.093+0000 7fc4c9400640 1 -- 172.21.15.5:0/212032947 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc4cc101de0 msgr2=0x7fc4cc1984a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:36.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:36.093+0000 7fc4c9400640 1 --2- 172.21.15.5:0/212032947 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc4cc101de0 0x7fc4cc1984a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:36.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:36.093+0000 7fc4c9400640 1 -- 172.21.15.5:0/212032947 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4cc19d450 con 0x7fc4cc102790 2024-09-10T10:02:36.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:36.093+0000 7fc4ca800640 1 --2- 172.21.15.5:0/212032947 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc4cc106f10 0x7fc4cc19cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:02:36.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:36.094+0000 7fc4c9400640 1 --2- 172.21.15.5:0/212032947 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc4cc102790 0x7fc4cc1989e0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7fc4b800e9f0 tx=0x7fc4b800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:36.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:36.094+0000 7fc4c2a00640 1 -- 172.21.15.5:0/212032947 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4b800ccc0 con 0x7fc4cc102790 2024-09-10T10:02:36.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:36.094+0000 7fc4c2a00640 1 -- 172.21.15.5:0/212032947 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc4b8010910 con 0x7fc4cc102790 2024-09-10T10:02:36.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:36.094+0000 7fc4c2a00640 1 -- 172.21.15.5:0/212032947 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4b8018960 con 0x7fc4cc102790 2024-09-10T10:02:36.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:36.094+0000 7fc4d0a00640 1 -- 172.21.15.5:0/212032947 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4cc19d740 con 0x7fc4cc102790 2024-09-10T10:02:36.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:36.094+0000 7fc4d0a00640 1 -- 172.21.15.5:0/212032947 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4cc074420 con 0x7fc4cc102790 2024-09-10T10:02:36.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:36.098+0000 7fc4c2a00640 1 -- 172.21.15.5:0/212032947 <== mon.2 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7fc4b8010ac0 con 0x7fc4cc102790 2024-09-10T10:02:36.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:36.099+0000 7fc4d343d740 1 Processor -- start 2024-09-10T10:02:36.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:36.100+0000 7fc4d343d740 1 -- start start 2024-09-10T10:02:36.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:36.100+0000 7fc4d343d740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55abd91d1b90 0x55abd91d1f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:36.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:36.100+0000 7fc4d343d740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55abd91d24b0 0x55abd91dc830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:36.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:36.100+0000 7fc4d343d740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55abd91dcd70 0x55abd91df160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:36.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:36.100+0000 7fc4d343d740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55abd91383e0 con 0x55abd91d1b90 2024-09-10T10:02:36.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:36.100+0000 7fc4d343d740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55abd90dfd80 con 0x55abd91d24b0 2024-09-10T10:02:36.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:36.100+0000 7fc4d343d740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55abd91343a0 con 0x55abd91dcd70 2024-09-10T10:02:36.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:36.100+0000 7fc4ca800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55abd91d1b90 0x55abd91d1f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:36.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:36.100+0000 7fc4c9e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55abd91d24b0 0x55abd91dc830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:36.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:36.101+0000 7fc4ca800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55abd91d1b90 0x55abd91d1f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49592/0 (socket says 172.21.15.5:49592) 2024-09-10T10:02:36.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:36.101+0000 7fc4c9e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55abd91d24b0 0x55abd91dc830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:58546/0 (socket says 172.21.15.5:58546) 2024-09-10T10:02:36.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:36.101+0000 7fc4ca800640 1 -- 172.21.15.5:0/2938452612 learned_addr learned my addr 172.21.15.5:0/2938452612 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T10:02:36.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:36.101+0000 7fc4c9400640 1 --2- 172.21.15.5:0/2938452612 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55abd91dcd70 0x55abd91df160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:36.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:36.101+0000 7fc4c2a00640 1 --2- 172.21.15.5:0/212032947 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fc4b00a2c00 0x7fc4b00a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:36.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:36.101+0000 7fc4c9e00640 1 -- 172.21.15.5:0/2938452612 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55abd91dcd70 msgr2=0x55abd91df160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:36.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:36.101+0000 7fc4c9e00640 1 --2- 172.21.15.5:0/2938452612 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55abd91dcd70 0x55abd91df160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:36.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:36.101+0000 7fc4c9e00640 1 -- 172.21.15.5:0/2938452612 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55abd91d1b90 msgr2=0x55abd91d1f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:36.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:36.101+0000 7fc4c9e00640 1 --2- 172.21.15.5:0/2938452612 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55abd91d1b90 0x55abd91d1f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:36.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:36.101+0000 7fc4c9e00640 1 -- 172.21.15.5:0/2938452612 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55abd90feae0 con 0x55abd91d24b0 2024-09-10T10:02:36.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:36.101+0000 7fc4c2a00640 1 -- 172.21.15.5:0/212032947 <== mon.2 v2:172.21.15.102:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fc4b80cf7d0 con 0x7fc4cc102790 2024-09-10T10:02:36.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:36.101+0000 7fc4ca800640 1 --2- 172.21.15.5:0/2938452612 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55abd91d1b90 0x55abd91d1f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:02:36.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:36.102+0000 7fc4c9e00640 1 --2- 172.21.15.5:0/2938452612 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55abd91d24b0 0x55abd91dc830 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7fc4bc02a9e0 tx=0x7fc4bc005e60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c22428de1f60c1cd server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:36.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:36.102+0000 7fc4ca800640 1 --2- 172.21.15.5:0/212032947 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fc4b00a2c00 0x7fc4b00a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:36.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:36.102+0000 7fc4b7000640 1 -- 172.21.15.5:0/2938452612 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4bc035030 con 0x55abd91d24b0 2024-09-10T10:02:36.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:36.102+0000 7fc4b7000640 1 -- 172.21.15.5:0/2938452612 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc4bc0043c0 con 0x55abd91d24b0 2024-09-10T10:02:36.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:36.102+0000 7fc4b7000640 1 -- 172.21.15.5:0/2938452612 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4bc0035f0 con 0x55abd91d24b0 2024-09-10T10:02:36.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:36.103+0000 7fc4d343d740 1 -- 172.21.15.5:0/2938452612 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55abd91d24b0 msgr2=0x55abd91dc830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:36.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:36.103+0000 7fc4d343d740 1 --2- 172.21.15.5:0/2938452612 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55abd91d24b0 0x55abd91dc830 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7fc4bc02a9e0 tx=0x7fc4bc005e60 comp rx=0 tx=0).stop 2024-09-10T10:02:36.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:36.103+0000 7fc4ca800640 1 --2- 172.21.15.5:0/212032947 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fc4b00a2c00 0x7fc4b00a50c0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fc4c4009a50 tx=0x7fc4c4004a40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:36.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:36.103+0000 7fc4d343d740 1 -- 172.21.15.5:0/2938452612 shutdown_connections 2024-09-10T10:02:36.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:36.103+0000 7fc4d343d740 1 --2- 172.21.15.5:0/2938452612 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55abd91dcd70 0x55abd91df160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:36.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:36.103+0000 7fc4d343d740 1 --2- 172.21.15.5:0/2938452612 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55abd91d24b0 0x55abd91dc830 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:36.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:36.103+0000 7fc4d343d740 1 --2- 172.21.15.5:0/2938452612 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55abd91d1b90 0x55abd91d1f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:36.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:36.103+0000 7fc4d343d740 1 -- 172.21.15.5:0/2938452612 >> 172.21.15.5:0/2938452612 conn(0x55abd91ce120 msgr2=0x55abd91ce500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:36.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:36.103+0000 7fc4d343d740 1 -- 172.21.15.5:0/2938452612 shutdown_connections 2024-09-10T10:02:36.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:36.103+0000 7fc4d343d740 1 -- 172.21.15.5:0/2938452612 wait complete. 2024-09-10T10:02:36.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:36.104+0000 7fc4d343d740 1 -- start start 2024-09-10T10:02:36.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:36.104+0000 7fc4d343d740 1 Processor -- start 2024-09-10T10:02:36.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:36.105+0000 7fc4d343d740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55abd91d1b90 0x55abd91d9c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:36.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:36.105+0000 7fc4d343d740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55abd91d24b0 0x55abd91da180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:36.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:36.105+0000 7fc4d343d740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55abd91dcd70 0x55abd91da6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:36.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:36.105+0000 7fc4d343d740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55abd91df9e0 con 0x55abd91dcd70 2024-09-10T10:02:36.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:36.105+0000 7fc4d343d740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55abd90f30f0 con 0x55abd91d24b0 2024-09-10T10:02:36.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:36.105+0000 7fc4d343d740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55abd91dfb60 con 0x55abd91d1b90 2024-09-10T10:02:36.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:36.105+0000 7fc4d343d740 10 client.0 _ll_register_callbacks cb 0x55abd90e49e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:02:36.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:36.105+0000 7fc4c9400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55abd91d24b0 0x55abd91da180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:36.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:36.105+0000 7fc4c9400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55abd91d24b0 0x55abd91da180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:58552/0 (socket says 172.21.15.5:58552) 2024-09-10T10:02:36.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:36.105+0000 7fc4c9400640 1 -- 172.21.15.5:0/716053560 learned_addr learned my addr 172.21.15.5:0/716053560 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T10:02:36.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:36.105+0000 7fc4c9e00640 1 --2- 172.21.15.5:0/716053560 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55abd91d1b90 0x55abd91d9c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:36.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:36.105+0000 7fc4ca800640 1 --2- 172.21.15.5:0/716053560 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55abd91dcd70 0x55abd91da6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:36.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:36.105+0000 7fc4c9400640 1 -- 172.21.15.5:0/716053560 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55abd91d1b90 msgr2=0x55abd91d9c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:36.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:36.105+0000 7fc4c9400640 1 --2- 172.21.15.5:0/716053560 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55abd91d1b90 0x55abd91d9c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:36.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:36.105+0000 7fc4c9400640 1 -- 172.21.15.5:0/716053560 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55abd91dcd70 msgr2=0x55abd91da6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:36.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:36.105+0000 7fc4c9400640 1 --2- 172.21.15.5:0/716053560 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55abd91dcd70 0x55abd91da6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:36.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:36.106+0000 7fc4c9400640 1 -- 172.21.15.5:0/716053560 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55abd8ea93c0 con 0x55abd91d24b0 2024-09-10T10:02:36.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:36.106+0000 7fc4ca800640 1 --2- 172.21.15.5:0/716053560 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55abd91dcd70 0x55abd91da6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:02:36.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:36.106+0000 7fc4c9400640 1 --2- 172.21.15.5:0/716053560 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55abd91d24b0 0x55abd91da180 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7fc4b800e9c0 tx=0x7fc4b8097fa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9f922154e74e90cb server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:36.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:36.106+0000 7fc4ab400640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-09-10T10:02:36.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:36.106+0000 7fc4ab400640 1 -- 172.21.15.5:0/716053560 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4b809b780 con 0x55abd91d24b0 2024-09-10T10:02:36.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:36.106+0000 7fc4ab400640 1 -- 172.21.15.5:0/716053560 <== mon.2 v2:172.21.15.102:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fc4b809b090 con 0x55abd91d24b0 2024-09-10T10:02:36.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:36.106+0000 7fc4d343d740 1 -- 172.21.15.5:0/716053560 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc4bc0043c0 con 0x55abd91d24b0 2024-09-10T10:02:36.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:36.106+0000 7fc4ab400640 1 -- 172.21.15.5:0/716053560 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7fc4b809b5a0 con 0x55abd91d24b0 2024-09-10T10:02:36.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:36.106+0000 7fc4d343d740 10 client.30035 fetch_fsmap learned FSMap version 42 2024-09-10T10:02:36.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:36.106+0000 7fc4d343d740 1 -- 172.21.15.5:0/716053560 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55abd91eb6d0 con 0x55abd91d24b0 2024-09-10T10:02:36.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:36.107+0000 7fc4ab400640 1 -- 172.21.15.5:0/716053560 <== mon.2 v2:172.21.15.102:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc4bc0043c0 con 0x55abd91d24b0 2024-09-10T10:02:36.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:36.107+0000 7fc4ab400640 1 -- 172.21.15.5:0/716053560 <== mon.2 v2:172.21.15.102:3300/0 5 ==== fsmap(e 42) ==== 8370+0+0 (secure 0 0 0) 0x7fc4b8018b00 con 0x55abd91d24b0 2024-09-10T10:02:36.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:36.107+0000 7fc4d343d740 10 client.30035 fetch_fsmap finished waiting for FSMap version 42 2024-09-10T10:02:36.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:36.107+0000 7fc4d343d740 10 client.30035 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T10:02:36.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:36.107+0000 7fc4d343d740 20 client.30035 populate_metadata read hostname 'smithi005' 2024-09-10T10:02:36.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:36.107+0000 7fc4d343d740 1 --2- 172.21.15.5:0/716053560 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55abd91ecae0 0x55abd91eeef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:36.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:36.107+0000 7fc4d343d740 4 client.30035 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:02:36.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:36.107+0000 7fc4d343d740 1 -- 172.21.15.5:0/716053560 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55abd9145950 con 0x55abd91ecae0 2024-09-10T10:02:36.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:36.108+0000 7fc4c9e00640 1 --2- 172.21.15.5:0/716053560 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55abd91ecae0 0x55abd91eeef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:36.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:36.108+0000 7fc4c9e00640 1 --2- 172.21.15.5:0/716053560 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55abd91ecae0 0x55abd91eeef0 crc :-1 s=READY pgs=329 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3e2b3b8a97f81a40 server_cookie=145cc2345a315737 in_seq=0 out_seq=0 2024-09-10T10:02:36.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:36.108+0000 7fc4ab400640 10 client.30035 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T10:02:36.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:36.109+0000 7fc4ab400640 1 -- 172.21.15.5:0/716053560 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55abd9145950 con 0x55abd91ecae0 2024-09-10T10:02:36.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:36.109+0000 7fc4ab400640 10 client.30035 handle_command_reply: tid=0 2024-09-10T10:02:36.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:36.109+0000 7fc4d343d740 1 client.30035 shutdown 2024-09-10T10:02:36.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:36.109+0000 7fc4d343d740 1 -- 172.21.15.5:0/716053560 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55abd91d24b0 msgr2=0x55abd91da180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:36.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:36.109+0000 7fc4d343d740 1 --2- 172.21.15.5:0/716053560 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55abd91d24b0 0x55abd91da180 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7fc4b800e9c0 tx=0x7fc4b8097fa0 comp rx=0 tx=0).stop 2024-09-10T10:02:36.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:36.110+0000 7fc4d343d740 1 -- 172.21.15.5:0/716053560 shutdown_connections 2024-09-10T10:02:36.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:36.110+0000 7fc4d343d740 1 --2- 172.21.15.5:0/716053560 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55abd91ecae0 0x55abd91eeef0 crc :-1 s=READY pgs=329 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:36.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:36.110+0000 7fc4d343d740 1 --2- 172.21.15.5:0/716053560 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55abd91dcd70 0x55abd91da6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:36.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:36.110+0000 7fc4d343d740 1 --2- 172.21.15.5:0/716053560 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55abd91d24b0 0x55abd91da180 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:36.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:36.110+0000 7fc4d343d740 1 --2- 172.21.15.5:0/716053560 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55abd91d1b90 0x55abd91d9c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:36.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:36.110+0000 7fc4ab400640 0 client.30035 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T10:02:36.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:36.110+0000 7fc4d343d740 1 -- 172.21.15.5:0/716053560 >> 172.21.15.5:0/716053560 conn(0x55abd91ce120 msgr2=0x55abd91d3e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:36.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:36.110+0000 7fc4d343d740 1 -- 172.21.15.5:0/716053560 shutdown_connections 2024-09-10T10:02:36.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:36.111+0000 7fc4d343d740 1 -- 172.21.15.5:0/716053560 wait complete. 2024-09-10T10:02:36.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:36.111+0000 7fc4d343d740 20 client.30035 trim_cache size 0 max 16384 2024-09-10T10:02:36.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:36.131+0000 7fc4d343d740 1 Processor -- start 2024-09-10T10:02:36.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:36.131+0000 7fc4d343d740 1 -- start start 2024-09-10T10:02:36.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:36.132+0000 7fc4d343d740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55abd91ce120 0x55abd91dbbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:36.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:36.132+0000 7fc4d343d740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55abd91d1b90 0x55abd91f47c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:36.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:36.132+0000 7fc4d343d740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55abd91d24b0 0x55abd91f4d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:36.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:36.132+0000 7fc4d343d740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc4cc1b1030 con 0x55abd91d1b90 2024-09-10T10:02:36.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:36.132+0000 7fc4d343d740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fc4cc1b2b70 con 0x55abd91d24b0 2024-09-10T10:02:36.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:36.132+0000 7fc4d343d740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fc4cc1b1700 con 0x55abd91ce120 2024-09-10T10:02:36.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:36.132+0000 7fc4ca800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55abd91d24b0 0x55abd91f4d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:36.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:36.132+0000 7fc4ca800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55abd91d24b0 0x55abd91f4d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:58560/0 (socket says 172.21.15.5:58560) 2024-09-10T10:02:36.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:36.132+0000 7fc4ca800640 1 -- 172.21.15.5:0/1242578557 learned_addr learned my addr 172.21.15.5:0/1242578557 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T10:02:36.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:36.132+0000 7fc4c9e00640 1 --2- 172.21.15.5:0/1242578557 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55abd91ce120 0x55abd91dbbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:36.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:36.133+0000 7fc4ca800640 1 -- 172.21.15.5:0/1242578557 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55abd91ce120 msgr2=0x55abd91dbbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:36.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:36.133+0000 7fc4ca800640 1 --2- 172.21.15.5:0/1242578557 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55abd91ce120 0x55abd91dbbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:36.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:36.133+0000 7fc4ca800640 1 -- 172.21.15.5:0/1242578557 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55abd91d1b90 msgr2=0x55abd91f47c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:36.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:36.133+0000 7fc4ca800640 1 --2- 172.21.15.5:0/1242578557 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55abd91d1b90 0x55abd91f47c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:36.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:36.133+0000 7fc4ca800640 1 -- 172.21.15.5:0/1242578557 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55abd91cedb0 con 0x55abd91d24b0 2024-09-10T10:02:36.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:36.133+0000 7fc4ca800640 1 --2- 172.21.15.5:0/1242578557 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55abd91d24b0 0x55abd91f4d00 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7fc4c4013250 tx=0x7fc4c4016000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f38a06a0874dd7d7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:36.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:36.134+0000 7fc4b6600640 1 -- 172.21.15.5:0/1242578557 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4c4003b70 con 0x55abd91d24b0 2024-09-10T10:02:36.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:36.134+0000 7fc4b6600640 1 -- 172.21.15.5:0/1242578557 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc4c4013280 con 0x55abd91d24b0 2024-09-10T10:02:36.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:36.134+0000 7fc4b6600640 1 -- 172.21.15.5:0/1242578557 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4c401f810 con 0x55abd91d24b0 2024-09-10T10:02:36.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:36.135+0000 7fc4d343d740 1 -- 172.21.15.5:0/1242578557 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55abd91d24b0 msgr2=0x55abd91f4d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:36.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:36.135+0000 7fc4d343d740 1 --2- 172.21.15.5:0/1242578557 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55abd91d24b0 0x55abd91f4d00 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7fc4c4013250 tx=0x7fc4c4016000 comp rx=0 tx=0).stop 2024-09-10T10:02:36.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:36.135+0000 7fc4d343d740 1 -- 172.21.15.5:0/1242578557 shutdown_connections 2024-09-10T10:02:36.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:36.135+0000 7fc4d343d740 1 --2- 172.21.15.5:0/1242578557 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55abd91d24b0 0x55abd91f4d00 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:36.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:36.135+0000 7fc4d343d740 1 --2- 172.21.15.5:0/1242578557 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55abd91d1b90 0x55abd91f47c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:36.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:36.135+0000 7fc4d343d740 1 --2- 172.21.15.5:0/1242578557 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55abd91ce120 0x55abd91dbbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:36.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:36.135+0000 7fc4d343d740 1 -- 172.21.15.5:0/1242578557 >> 172.21.15.5:0/1242578557 conn(0x55abd91d5630 msgr2=0x55abd91db4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:36.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:36.135+0000 7fc4d343d740 1 -- 172.21.15.5:0/1242578557 shutdown_connections 2024-09-10T10:02:36.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:36.135+0000 7fc4d343d740 1 -- 172.21.15.5:0/1242578557 wait complete. 2024-09-10T10:02:36.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:36.135+0000 7fc4d343d740 1 -- start start 2024-09-10T10:02:36.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:36.136+0000 7fc4d343d740 1 Processor -- start 2024-09-10T10:02:36.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:36.136+0000 7fc4d343d740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55abd91ce120 0x55abd91e0b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:36.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:36.136+0000 7fc4d343d740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55abd91d1b90 0x55abd91e10a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:36.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:36.136+0000 7fc4d343d740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55abd91d24b0 0x55abd91e15e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:36.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:36.136+0000 7fc4d343d740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc4cc1b24a0 con 0x55abd91d24b0 2024-09-10T10:02:36.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:36.136+0000 7fc4d343d740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fc4cc1b1dd0 con 0x55abd91d1b90 2024-09-10T10:02:36.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:36.136+0000 7fc4d343d740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fc4cc1af010 con 0x55abd91ce120 2024-09-10T10:02:36.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:36.136+0000 7fc4d343d740 10 client.0 _ll_register_callbacks cb 0x7fc4b8010e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:02:36.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:36.137+0000 7fc4c9400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55abd91d1b90 0x55abd91e10a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:36.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:36.137+0000 7fc4c9e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55abd91ce120 0x55abd91e0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:36.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:36.137+0000 7fc4c9400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55abd91d1b90 0x55abd91e10a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:58576/0 (socket says 172.21.15.5:58576) 2024-09-10T10:02:36.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:36.137+0000 7fc4c9400640 1 -- 172.21.15.5:0/1246710094 learned_addr learned my addr 172.21.15.5:0/1246710094 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T10:02:36.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:36.137+0000 7fc4ca800640 1 --2- 172.21.15.5:0/1246710094 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55abd91d24b0 0x55abd91e15e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:36.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:36.137+0000 7fc4c9400640 1 -- 172.21.15.5:0/1246710094 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55abd91ce120 msgr2=0x55abd91e0b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:36.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:36.137+0000 7fc4c9400640 1 --2- 172.21.15.5:0/1246710094 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55abd91ce120 0x55abd91e0b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:36.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:36.137+0000 7fc4c9400640 1 -- 172.21.15.5:0/1246710094 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55abd91d24b0 msgr2=0x55abd91e15e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:02:36.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:36.137+0000 7fc4c9400640 1 --2- 172.21.15.5:0/1246710094 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55abd91d24b0 0x55abd91e15e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:36.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:36.137+0000 7fc4c9400640 1 -- 172.21.15.5:0/1246710094 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55abd92567e0 con 0x55abd91d1b90 2024-09-10T10:02:36.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:36.137+0000 7fc4c9e00640 1 --2- 172.21.15.5:0/1246710094 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55abd91ce120 0x55abd91e0b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:02:36.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:36.137+0000 7fc4c9400640 1 --2- 172.21.15.5:0/1246710094 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55abd91d1b90 0x55abd91e10a0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7fc4b80cd250 tx=0x7fc4b8018d10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f1a7689484ec0270 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:02:36.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:36.137+0000 7fc4ab400640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-09-10T10:02:36.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:36.138+0000 7fc4ab400640 1 -- 172.21.15.5:0/1246710094 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4b809b780 con 0x55abd91d1b90 2024-09-10T10:02:36.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:36.138+0000 7fc4ab400640 1 -- 172.21.15.5:0/1246710094 <== mon.2 v2:172.21.15.102:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fc4b8010430 con 0x55abd91d1b90 2024-09-10T10:02:36.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:36.138+0000 7fc4d343d740 1 -- 172.21.15.5:0/1246710094 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc4c4013280 con 0x55abd91d1b90 2024-09-10T10:02:36.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:36.138+0000 7fc4ab400640 1 -- 172.21.15.5:0/1246710094 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7fc4b8097ce0 con 0x55abd91d1b90 2024-09-10T10:02:36.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:36.138+0000 7fc4d343d740 10 client.30041 fetch_fsmap learned FSMap version 42 2024-09-10T10:02:36.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:36.138+0000 7fc4d343d740 1 -- 172.21.15.5:0/1246710094 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55abd9257ad0 con 0x55abd91d1b90 2024-09-10T10:02:36.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:36.138+0000 7fc4ab400640 1 -- 172.21.15.5:0/1246710094 <== mon.2 v2:172.21.15.102:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc4c4013280 con 0x55abd91d1b90 2024-09-10T10:02:36.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:36.138+0000 7fc4ab400640 1 -- 172.21.15.5:0/1246710094 <== mon.2 v2:172.21.15.102:3300/0 5 ==== fsmap(e 42) ==== 8370+0+0 (secure 0 0 0) 0x7fc4b809ab00 con 0x55abd91d1b90 2024-09-10T10:02:36.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:36.138+0000 7fc4d343d740 10 client.30041 fetch_fsmap finished waiting for FSMap version 42 2024-09-10T10:02:36.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:36.138+0000 7fc4d343d740 10 client.30041 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T10:02:36.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:36.138+0000 7fc4d343d740 20 client.30041 populate_metadata read hostname 'smithi005' 2024-09-10T10:02:36.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:36.138+0000 7fc4d343d740 1 --2- 172.21.15.5:0/1246710094 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55abd91dcd70 0x55abd925a230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:02:36.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:36.139+0000 7fc4d343d740 4 client.30041 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T10:02:36.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:36.139+0000 7fc4d343d740 1 -- 172.21.15.5:0/1246710094 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55abd913fe60 con 0x55abd91dcd70 2024-09-10T10:02:36.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:36.139+0000 7fc4c9e00640 1 --2- 172.21.15.5:0/1246710094 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55abd91dcd70 0x55abd925a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:02:36.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:36.140+0000 7fc4c9e00640 1 --2- 172.21.15.5:0/1246710094 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55abd91dcd70 0x55abd925a230 crc :-1 s=READY pgs=330 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ed5e72a671fc1d3f server_cookie=986204acb4bc48dc in_seq=0 out_seq=0 2024-09-10T10:02:36.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:36.140+0000 7fc4ab400640 10 client.30041 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T10:02:36.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:36.140+0000 7fc4ab400640 1 -- 172.21.15.5:0/1246710094 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55abd913fe60 con 0x55abd91dcd70 2024-09-10T10:02:36.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:36.140+0000 7fc4ab400640 10 client.30041 handle_command_reply: tid=0 2024-09-10T10:02:36.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:36.140+0000 7fc4d343d740 1 client.30041 shutdown 2024-09-10T10:02:36.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:36.140+0000 7fc4d343d740 1 -- 172.21.15.5:0/1246710094 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55abd91d1b90 msgr2=0x55abd91e10a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:36.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:36.140+0000 7fc4d343d740 1 --2- 172.21.15.5:0/1246710094 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55abd91d1b90 0x55abd91e10a0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7fc4b80cd250 tx=0x7fc4b8018d10 comp rx=0 tx=0).stop 2024-09-10T10:02:36.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:36.141+0000 7fc4d343d740 1 -- 172.21.15.5:0/1246710094 shutdown_connections 2024-09-10T10:02:36.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:36.141+0000 7fc4d343d740 1 --2- 172.21.15.5:0/1246710094 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55abd91dcd70 0x55abd925a230 crc :-1 s=READY pgs=330 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:36.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:36.141+0000 7fc4d343d740 1 --2- 172.21.15.5:0/1246710094 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55abd91d24b0 0x55abd91e15e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:36.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:36.141+0000 7fc4d343d740 1 --2- 172.21.15.5:0/1246710094 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55abd91d1b90 0x55abd91e10a0 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:36.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:36.141+0000 7fc4d343d740 1 --2- 172.21.15.5:0/1246710094 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55abd91ce120 0x55abd91e0b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:36.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:36.141+0000 7fc4d343d740 1 -- 172.21.15.5:0/1246710094 >> 172.21.15.5:0/1246710094 conn(0x55abd91d5630 msgr2=0x55abd91da420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:36.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:36.141+0000 7fc4ab400640 0 client.30041 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T10:02:36.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:36.141+0000 7fc4d343d740 1 -- 172.21.15.5:0/1246710094 shutdown_connections 2024-09-10T10:02:36.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:36.141+0000 7fc4d343d740 1 -- 172.21.15.5:0/1246710094 wait complete. 2024-09-10T10:02:36.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:36.141+0000 7fc4d343d740 20 client.30041 trim_cache size 0 max 16384 2024-09-10T10:02:36.144 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-10T10:02:36.144 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-10T10:02:36.145 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2024-09-10T10:02:36.145 INFO:teuthology.orchestra.run.smithi005.stdout: "4a1f5597-8801-4d08-ac33-4ad9220a7597": { 2024-09-10T10:02:36.145 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2024-09-10T10:02:36.145 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "4a1f5597-8801-4d08-ac33-4ad9220a7597", 2024-09-10T10:02:36.145 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2024-09-10T10:02:36.145 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T10:02:36.145 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T10:02:36.145 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-10T10:02:36.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:36.142+0000 7fc4d0a00640 1 -- 172.21.15.5:0/212032947 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fc4b00a2c00 msgr2=0x7fc4b00a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:36.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:36.142+0000 7fc4d0a00640 1 --2- 172.21.15.5:0/212032947 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fc4b00a2c00 0x7fc4b00a50c0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fc4c4009a50 tx=0x7fc4c4004a40 comp rx=0 tx=0).stop 2024-09-10T10:02:36.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:36.142+0000 7fc4d0a00640 1 -- 172.21.15.5:0/212032947 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc4cc102790 msgr2=0x7fc4cc1989e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:02:36.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:36.142+0000 7fc4d0a00640 1 --2- 172.21.15.5:0/212032947 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc4cc102790 0x7fc4cc1989e0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7fc4b800e9f0 tx=0x7fc4b800eeb0 comp rx=0 tx=0).stop 2024-09-10T10:02:36.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:36.142+0000 7fc4d0a00640 1 -- 172.21.15.5:0/212032947 shutdown_connections 2024-09-10T10:02:36.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:36.142+0000 7fc4d0a00640 1 --2- 172.21.15.5:0/212032947 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fc4b00a2c00 0x7fc4b00a50c0 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:36.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:36.142+0000 7fc4d0a00640 1 --2- 172.21.15.5:0/212032947 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc4cc106f10 0x7fc4cc19cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:36.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:36.142+0000 7fc4d0a00640 1 --2- 172.21.15.5:0/212032947 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc4cc102790 0x7fc4cc1989e0 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:36.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:36.142+0000 7fc4d0a00640 1 --2- 172.21.15.5:0/212032947 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc4cc101de0 0x7fc4cc1984a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:02:36.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:36.142+0000 7fc4d0a00640 1 -- 172.21.15.5:0/212032947 >> 172.21.15.5:0/212032947 conn(0x7fc4cc0fd8c0 msgr2=0x7fc4cc0fec70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:02:36.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:36.142+0000 7fc4d0a00640 1 -- 172.21.15.5:0/212032947 shutdown_connections 2024-09-10T10:02:36.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:02:36.143+0000 7fc4d0a00640 1 -- 172.21.15.5:0/212032947 wait complete. 2024-09-10T10:02:37.070 INFO:tasks.workunit.client.0.smithi005.stdout: 10 157944 12.94 MB/sec execute 495 sec latency 59.955 ms 2024-09-10T10:02:37.158 INFO:tasks.cephfs.filesystem:scrub status for tag:4a1f5597-8801-4d08-ac33-4ad9220a7597 - {'path': '/', 'tag': '4a1f5597-8801-4d08-ac33-4ad9220a7597', 'options': 'recursive,force'} 2024-09-10T10:02:37.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:02:37 smithi005 ceph-mon[30350]: pgmap v1121: 129 pgs: 129 active+clean; 727 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 484 KiB/s rd, 366 KiB/s wr, 140 op/s 2024-09-10T10:02:37.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:02:37 smithi159 ceph-mon[35513]: pgmap v1121: 129 pgs: 129 active+clean; 727 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 484 KiB/s rd, 366 KiB/s wr, 140 op/s 2024-09-10T10:02:37.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:02:37 smithi102 ceph-mon[35442]: pgmap v1121: 129 pgs: 129 active+clean; 727 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 484 KiB/s rd, 366 KiB/s wr, 140 op/s 2024-09-10T10:02:38.070 INFO:tasks.workunit.client.0.smithi005.stdout: 10 158523 12.99 MB/sec execute 496 sec latency 60.884 ms 2024-09-10T10:02:38.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:02:38 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:02:38.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:02:38 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:02:38.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:02:38 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:02:39.070 INFO:tasks.workunit.client.0.smithi005.stdout: 10 159162 13.04 MB/sec execute 497 sec latency 52.887 ms 2024-09-10T10:02:39.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:02:39 smithi005 ceph-mon[30350]: pgmap v1122: 129 pgs: 129 active+clean; 727 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1005 KiB/s rd, 360 KiB/s wr, 121 op/s 2024-09-10T10:02:39.557 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:02:39 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:02:39.557 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:02:39 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:02:39.557 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:02:39 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T10:02:39.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:02:39 smithi159 ceph-mon[35513]: pgmap v1122: 129 pgs: 129 active+clean; 727 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1005 KiB/s rd, 360 KiB/s wr, 121 op/s 2024-09-10T10:02:39.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:02:39 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:02:39.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:02:39 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:02:39.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:02:39 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T10:02:39.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:02:39 smithi102 ceph-mon[35442]: pgmap v1122: 129 pgs: 129 active+clean; 727 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1005 KiB/s rd, 360 KiB/s wr, 121 op/s 2024-09-10T10:02:39.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:02:39 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:02:39.635 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:02:39 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:02:39.635 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:02:39 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T10:02:40.070 INFO:tasks.workunit.client.0.smithi005.stdout: 10 159625 13.06 MB/sec execute 498 sec latency 106.272 ms 2024-09-10T10:02:41.070 INFO:tasks.workunit.client.0.smithi005.stdout: 10 160162 13.09 MB/sec execute 499 sec latency 67.585 ms 2024-09-10T10:02:41.509 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:02:41 smithi005 ceph-mon[30350]: pgmap v1123: 129 pgs: 129 active+clean; 735 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 1.5 MiB/s wr, 282 op/s 2024-09-10T10:02:41.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:02:41 smithi159 ceph-mon[35513]: pgmap v1123: 129 pgs: 129 active+clean; 735 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 1.5 MiB/s wr, 282 op/s 2024-09-10T10:02:41.635 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:02:41 smithi102 ceph-mon[35442]: pgmap v1123: 129 pgs: 129 active+clean; 735 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 1.5 MiB/s wr, 282 op/s 2024-09-10T10:02:42.070 INFO:tasks.workunit.client.0.smithi005.stdout: 10 160656 13.10 MB/sec execute 500 sec latency 90.552 ms 2024-09-10T10:02:43.071 INFO:tasks.workunit.client.0.smithi005.stdout: 10 161314 13.14 MB/sec execute 501 sec latency 58.507 ms 2024-09-10T10:02:43.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:02:43 smithi005 ceph-mon[30350]: pgmap v1124: 129 pgs: 129 active+clean; 732 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 1.5 MiB/s wr, 260 op/s 2024-09-10T10:02:43.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:02:43 smithi159 ceph-mon[35513]: pgmap v1124: 129 pgs: 129 active+clean; 732 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 1.5 MiB/s wr, 260 op/s 2024-09-10T10:02:43.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:02:43 smithi102 ceph-mon[35442]: pgmap v1124: 129 pgs: 129 active+clean; 732 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 1.5 MiB/s wr, 260 op/s 2024-09-10T10:02:44.071 INFO:tasks.workunit.client.0.smithi005.stdout: 10 162030 13.19 MB/sec execute 502 sec latency 136.842 ms 2024-09-10T10:02:45.071 INFO:tasks.workunit.client.0.smithi005.stdout: 10 162853 13.27 MB/sec execute 503 sec latency 47.911 ms 2024-09-10T10:02:45.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:02:45 smithi005 ceph-mon[30350]: pgmap v1125: 129 pgs: 129 active+clean; 723 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 1.9 MiB/s wr, 306 op/s 2024-09-10T10:02:45.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:02:45 smithi159 ceph-mon[35513]: pgmap v1125: 129 pgs: 129 active+clean; 723 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 1.9 MiB/s wr, 306 op/s 2024-09-10T10:02:45.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:02:45 smithi102 ceph-mon[35442]: pgmap v1125: 129 pgs: 129 active+clean; 723 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 1.9 MiB/s wr, 306 op/s 2024-09-10T10:02:46.071 INFO:tasks.workunit.client.0.smithi005.stdout: 10 163414 13.31 MB/sec execute 504 sec latency 61.681 ms 2024-09-10T10:02:47.071 INFO:tasks.workunit.client.0.smithi005.stdout: 10 164023 13.35 MB/sec execute 505 sec latency 70.014 ms 2024-09-10T10:02:47.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:02:47 smithi005 ceph-mon[30350]: pgmap v1126: 129 pgs: 129 active+clean; 730 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 3.0 MiB/s wr, 449 op/s 2024-09-10T10:02:47.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:02:47 smithi005 ceph-mon[30350]: scrub summary: idle 2024-09-10T10:02:47.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:02:47 smithi005 ceph-mon[30350]: scrub summary: idle 2024-09-10T10:02:47.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:02:47 smithi159 ceph-mon[35513]: pgmap v1126: 129 pgs: 129 active+clean; 730 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 3.0 MiB/s wr, 449 op/s 2024-09-10T10:02:47.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:02:47 smithi159 ceph-mon[35513]: scrub summary: idle 2024-09-10T10:02:47.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:02:47 smithi159 ceph-mon[35513]: scrub summary: idle 2024-09-10T10:02:47.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:02:47 smithi102 ceph-mon[35442]: pgmap v1126: 129 pgs: 129 active+clean; 730 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 3.0 MiB/s wr, 449 op/s 2024-09-10T10:02:47.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:02:47 smithi102 ceph-mon[35442]: scrub summary: idle 2024-09-10T10:02:47.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:02:47 smithi102 ceph-mon[35442]: scrub summary: idle 2024-09-10T10:02:48.071 INFO:tasks.workunit.client.0.smithi005.stdout: 10 164393 13.35 MB/sec execute 506 sec latency 84.071 ms 2024-09-10T10:02:48.447 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:02:48 smithi005 ceph-mon[30350]: scrub summary: idle 2024-09-10T10:02:48.447 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:02:48 smithi005 ceph-mon[30350]: scrub summary: idle 2024-09-10T10:02:48.447 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:02:48 smithi005 ceph-mon[30350]: scrub summary: idle 2024-09-10T10:02:48.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:02:48 smithi159 ceph-mon[35513]: scrub summary: idle 2024-09-10T10:02:48.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:02:48 smithi159 ceph-mon[35513]: scrub summary: idle 2024-09-10T10:02:48.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:02:48 smithi159 ceph-mon[35513]: scrub summary: idle 2024-09-10T10:02:48.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:02:48 smithi102 ceph-mon[35442]: scrub summary: idle 2024-09-10T10:02:48.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:02:48 smithi102 ceph-mon[35442]: scrub summary: idle 2024-09-10T10:02:48.635 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:02:48 smithi102 ceph-mon[35442]: scrub summary: idle 2024-09-10T10:02:49.071 INFO:tasks.workunit.client.0.smithi005.stdout: 10 164954 13.38 MB/sec execute 507 sec latency 83.390 ms 2024-09-10T10:02:49.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:02:49 smithi005 ceph-mon[30350]: pgmap v1127: 129 pgs: 129 active+clean; 731 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 3.2 MiB/s wr, 438 op/s 2024-09-10T10:02:49.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:02:49 smithi159 ceph-mon[35513]: pgmap v1127: 129 pgs: 129 active+clean; 731 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 3.2 MiB/s wr, 438 op/s 2024-09-10T10:02:49.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:02:49 smithi102 ceph-mon[35442]: pgmap v1127: 129 pgs: 129 active+clean; 731 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 3.2 MiB/s wr, 438 op/s 2024-09-10T10:02:50.071 INFO:tasks.workunit.client.0.smithi005.stdout: 10 165603 13.42 MB/sec execute 508 sec latency 55.862 ms 2024-09-10T10:02:51.071 INFO:tasks.workunit.client.0.smithi005.stdout: 10 166517 13.48 MB/sec execute 509 sec latency 154.835 ms 2024-09-10T10:02:51.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:02:51 smithi005 ceph-mon[30350]: pgmap v1128: 129 pgs: 129 active+clean; 742 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 4.1 MiB/s wr, 565 op/s 2024-09-10T10:02:51.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:02:51 smithi159 ceph-mon[35513]: pgmap v1128: 129 pgs: 129 active+clean; 742 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 4.1 MiB/s wr, 565 op/s 2024-09-10T10:02:51.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:02:51 smithi102 ceph-mon[35442]: pgmap v1128: 129 pgs: 129 active+clean; 742 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 4.1 MiB/s wr, 565 op/s 2024-09-10T10:02:52.072 INFO:tasks.workunit.client.0.smithi005.stdout: 10 167209 13.54 MB/sec execute 510 sec latency 62.060 ms 2024-09-10T10:02:53.071 INFO:tasks.workunit.client.0.smithi005.stdout: 10 167756 13.59 MB/sec execute 511 sec latency 84.374 ms 2024-09-10T10:02:53.280 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:02:53 smithi005 ceph-mon[30350]: pgmap v1129: 129 pgs: 129 active+clean; 745 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 3.3 MiB/s wr, 423 op/s 2024-09-10T10:02:53.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:02:53 smithi159 ceph-mon[35513]: pgmap v1129: 129 pgs: 129 active+clean; 745 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 3.3 MiB/s wr, 423 op/s 2024-09-10T10:02:53.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:02:53 smithi102 ceph-mon[35442]: pgmap v1129: 129 pgs: 129 active+clean; 745 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 3.3 MiB/s wr, 423 op/s 2024-09-10T10:02:54.074 INFO:tasks.workunit.client.0.smithi005.stdout: 10 168194 13.60 MB/sec execute 512 sec latency 118.351 ms 2024-09-10T10:02:55.073 INFO:tasks.workunit.client.0.smithi005.stdout: 10 168869 13.65 MB/sec execute 513 sec latency 61.236 ms 2024-09-10T10:02:55.418 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:02:55 smithi005 ceph-mon[30350]: pgmap v1130: 129 pgs: 129 active+clean; 748 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 3.5 MiB/s wr, 433 op/s 2024-09-10T10:02:55.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:02:55 smithi159 ceph-mon[35513]: pgmap v1130: 129 pgs: 129 active+clean; 748 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 3.5 MiB/s wr, 433 op/s 2024-09-10T10:02:55.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:02:55 smithi102 ceph-mon[35442]: pgmap v1130: 129 pgs: 129 active+clean; 748 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 3.5 MiB/s wr, 433 op/s 2024-09-10T10:02:56.074 INFO:tasks.workunit.client.0.smithi005.stdout: 10 169310 13.67 MB/sec execute 514 sec latency 68.677 ms 2024-09-10T10:02:57.074 INFO:tasks.workunit.client.0.smithi005.stdout: 10 169900 13.69 MB/sec execute 515 sec latency 61.480 ms 2024-09-10T10:02:57.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:02:57 smithi005 ceph-mon[30350]: pgmap v1131: 129 pgs: 129 active+clean; 755 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 3.9 MiB/s wr, 502 op/s 2024-09-10T10:02:57.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:02:57 smithi159 ceph-mon[35513]: pgmap v1131: 129 pgs: 129 active+clean; 755 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 3.9 MiB/s wr, 502 op/s 2024-09-10T10:02:57.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:02:57 smithi102 ceph-mon[35442]: pgmap v1131: 129 pgs: 129 active+clean; 755 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 3.9 MiB/s wr, 502 op/s 2024-09-10T10:02:58.074 INFO:tasks.workunit.client.0.smithi005.stdout: 10 170855 13.75 MB/sec execute 516 sec latency 47.318 ms 2024-09-10T10:02:59.074 INFO:tasks.workunit.client.0.smithi005.stdout: 10 171491 13.81 MB/sec execute 517 sec latency 64.703 ms 2024-09-10T10:02:59.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:02:59 smithi005 ceph-mon[30350]: pgmap v1132: 129 pgs: 129 active+clean; 759 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 3.2 MiB/s wr, 393 op/s 2024-09-10T10:02:59.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:02:59 smithi159 ceph-mon[35513]: pgmap v1132: 129 pgs: 129 active+clean; 759 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 3.2 MiB/s wr, 393 op/s 2024-09-10T10:02:59.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:02:59 smithi102 ceph-mon[35442]: pgmap v1132: 129 pgs: 129 active+clean; 759 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 3.2 MiB/s wr, 393 op/s 2024-09-10T10:03:00.074 INFO:tasks.workunit.client.0.smithi005.stdout: 10 172027 13.84 MB/sec execute 518 sec latency 192.498 ms 2024-09-10T10:03:01.074 INFO:tasks.workunit.client.0.smithi005.stdout: 10 172611 13.87 MB/sec execute 519 sec latency 74.820 ms 2024-09-10T10:03:01.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:03:01 smithi005 ceph-mon[30350]: pgmap v1133: 129 pgs: 129 active+clean; 770 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 4.2 MiB/s wr, 539 op/s 2024-09-10T10:03:01.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:03:01 smithi159 ceph-mon[35513]: pgmap v1133: 129 pgs: 129 active+clean; 770 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 4.2 MiB/s wr, 539 op/s 2024-09-10T10:03:01.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:03:01 smithi102 ceph-mon[35442]: pgmap v1133: 129 pgs: 129 active+clean; 770 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 4.2 MiB/s wr, 539 op/s 2024-09-10T10:03:02.074 INFO:tasks.workunit.client.0.smithi005.stdout: 10 173314 13.92 MB/sec execute 520 sec latency 86.093 ms 2024-09-10T10:03:03.074 INFO:tasks.workunit.client.0.smithi005.stdout: 10 173843 13.94 MB/sec execute 521 sec latency 53.117 ms 2024-09-10T10:03:03.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:03:03 smithi005 ceph-mon[30350]: pgmap v1134: 129 pgs: 129 active+clean; 772 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 3.3 MiB/s wr, 404 op/s 2024-09-10T10:03:03.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:03:03 smithi159 ceph-mon[35513]: pgmap v1134: 129 pgs: 129 active+clean; 772 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 3.3 MiB/s wr, 404 op/s 2024-09-10T10:03:03.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:03:03 smithi102 ceph-mon[35442]: pgmap v1134: 129 pgs: 129 active+clean; 772 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 3.3 MiB/s wr, 404 op/s 2024-09-10T10:03:04.074 INFO:tasks.workunit.client.0.smithi005.stdout: 10 174455 13.99 MB/sec execute 522 sec latency 49.297 ms 2024-09-10T10:03:04.261 INFO:tasks.quiescer.fs.[cephfs]:Total duration (60) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-09-10T10:03:04.261 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 30.0 2024-09-10T10:03:04.262 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T10:03:04.827 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:03:05.075 INFO:tasks.workunit.client.0.smithi005.stdout: 10 175011 14.00 MB/sec execute 523 sec latency 68.708 ms 2024-09-10T10:03:05.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:03:05 smithi159 ceph-mon[35513]: pgmap v1135: 129 pgs: 129 active+clean; 778 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 3.5 MiB/s wr, 433 op/s 2024-09-10T10:03:05.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:03:05 smithi102 ceph-mon[35442]: pgmap v1135: 129 pgs: 129 active+clean; 778 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 3.5 MiB/s wr, 433 op/s 2024-09-10T10:03:05.737 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:03:05 smithi005 ceph-mon[30350]: pgmap v1135: 129 pgs: 129 active+clean; 778 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 3.5 MiB/s wr, 433 op/s 2024-09-10T10:03:06.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:06.044+0000 7f80de600640 1 -- 172.21.15.5:0/3689029805 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f80d8076720 msgr2=0x7f80d8076b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:06.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:06.044+0000 7f80de600640 1 --2- 172.21.15.5:0/3689029805 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f80d8076720 0x7f80d8076b00 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f80cc009ca0 tx=0x7f80cc02a8b0 comp rx=0 tx=0).stop 2024-09-10T10:03:06.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:06.044+0000 7f80de600640 1 -- 172.21.15.5:0/3689029805 shutdown_connections 2024-09-10T10:03:06.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:06.044+0000 7f80de600640 1 --2- 172.21.15.5:0/3689029805 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f80d8112870 0x7f80d8114c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:06.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:06.044+0000 7f80de600640 1 --2- 172.21.15.5:0/3689029805 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f80d8077040 0x7f80d8112330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:06.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:06.044+0000 7f80de600640 1 --2- 172.21.15.5:0/3689029805 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f80d8076720 0x7f80d8076b00 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:06.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:06.044+0000 7f80de600640 1 -- 172.21.15.5:0/3689029805 >> 172.21.15.5:0/3689029805 conn(0x7f80d80ff8d0 msgr2=0x7f80d8101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:06.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:06.044+0000 7f80de600640 1 -- 172.21.15.5:0/3689029805 shutdown_connections 2024-09-10T10:03:06.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:06.044+0000 7f80de600640 1 -- 172.21.15.5:0/3689029805 wait complete. 2024-09-10T10:03:06.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:06.045+0000 7f80de600640 1 Processor -- start 2024-09-10T10:03:06.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:06.046+0000 7f80de600640 1 -- start start 2024-09-10T10:03:06.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:06.046+0000 7f80de600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f80d8076720 0x7f80d81a4650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:06.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:06.046+0000 7f80de600640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f80d8077040 0x7f80d81a4b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:06.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:06.046+0000 7f80de600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f80d8112870 0x7f80d819e720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:06.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:06.046+0000 7f80de600640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f80d8118450 con 0x7f80d8076720 2024-09-10T10:03:06.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:06.046+0000 7f80de600640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f80d81182d0 con 0x7f80d8112870 2024-09-10T10:03:06.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:06.046+0000 7f80de600640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f80d81185d0 con 0x7f80d8077040 2024-09-10T10:03:06.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:06.046+0000 7f80dd200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f80d8076720 0x7f80d81a4650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:06.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:06.047+0000 7f80dd200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f80d8076720 0x7f80d81a4650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37114/0 (socket says 172.21.15.5:37114) 2024-09-10T10:03:06.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:06.047+0000 7f80dd200640 1 -- 172.21.15.5:0/2883429758 learned_addr learned my addr 172.21.15.5:0/2883429758 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T10:03:06.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:06.047+0000 7f80ddc00640 1 --2- 172.21.15.5:0/2883429758 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f80d8112870 0x7f80d819e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:06.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:06.047+0000 7f80ddc00640 1 -- 172.21.15.5:0/2883429758 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f80d8077040 msgr2=0x7f80d81a4b90 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-10T10:03:06.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:06.047+0000 7f80ddc00640 1 --2- 172.21.15.5:0/2883429758 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f80d8077040 0x7f80d81a4b90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:06.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:06.047+0000 7f80ddc00640 1 -- 172.21.15.5:0/2883429758 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f80d8076720 msgr2=0x7f80d81a4650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:06.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:06.047+0000 7f80ddc00640 1 --2- 172.21.15.5:0/2883429758 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f80d8076720 0x7f80d81a4650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:06.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:06.047+0000 7f80ddc00640 1 -- 172.21.15.5:0/2883429758 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80d819ef90 con 0x7f80d8112870 2024-09-10T10:03:06.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:06.047+0000 7f80dd200640 1 --2- 172.21.15.5:0/2883429758 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f80d8076720 0x7f80d81a4650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:03:06.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:06.048+0000 7f80ddc00640 1 --2- 172.21.15.5:0/2883429758 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f80d8112870 0x7f80d819e720 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f80d807aa10 tx=0x7f80d000edd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:06.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:06.048+0000 7f80d5600640 1 -- 172.21.15.5:0/2883429758 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80d000ccc0 con 0x7f80d8112870 2024-09-10T10:03:06.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:06.048+0000 7f80d5600640 1 -- 172.21.15.5:0/2883429758 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f80d00035f0 con 0x7f80d8112870 2024-09-10T10:03:06.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:06.048+0000 7f80d5600640 1 -- 172.21.15.5:0/2883429758 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80d000ce60 con 0x7f80d8112870 2024-09-10T10:03:06.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:06.048+0000 7f80de600640 1 -- 172.21.15.5:0/2883429758 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80d819f280 con 0x7f80d8112870 2024-09-10T10:03:06.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:06.049+0000 7f80de600640 1 -- 172.21.15.5:0/2883429758 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80d81ab400 con 0x7f80d8112870 2024-09-10T10:03:06.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:06.050+0000 7f80de600640 1 -- 172.21.15.5:0/2883429758 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f80d8077e60 con 0x7f80d8112870 2024-09-10T10:03:06.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:06.052+0000 7f80d5600640 1 -- 172.21.15.5:0/2883429758 <== mon.2 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f80d00037a0 con 0x7f80d8112870 2024-09-10T10:03:06.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:06.054+0000 7f80d5600640 1 --2- 172.21.15.5:0/2883429758 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f80c00a2c80 0x7f80c00a5140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:06.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:06.054+0000 7f80d5600640 1 -- 172.21.15.5:0/2883429758 <== mon.2 v2:172.21.15.102:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f80d00cf670 con 0x7f80d8112870 2024-09-10T10:03:06.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:06.054+0000 7f80dd200640 1 --2- 172.21.15.5:0/2883429758 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f80c00a2c80 0x7f80c00a5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:06.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:06.055+0000 7f80dd200640 1 --2- 172.21.15.5:0/2883429758 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f80c00a2c80 0x7f80c00a5140 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f80cc0308d0 tx=0x7f80cc005d80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:06.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:06.058+0000 7f80d5600640 1 -- 172.21.15.5:0/2883429758 <== mon.2 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f80d009c180 con 0x7f80d8112870 2024-09-10T10:03:06.075 INFO:tasks.workunit.client.0.smithi005.stdout: 10 175763 14.06 MB/sec execute 524 sec latency 46.939 ms 2024-09-10T10:03:06.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:06.314+0000 7f80de600640 1 -- 172.21.15.5:0/2883429758 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f80d8076b00 con 0x7f80d8112870 2024-09-10T10:03:06.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:06.315+0000 7f80d5600640 1 -- 172.21.15.5:0/2883429758 <== mon.2 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 42 v42) ==== 94+0+11969 (secure 0 0 0) 0x7f80d8076b00 con 0x7f80d8112870 2024-09-10T10:03:06.316 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-10T10:03:06.316 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":42,"btime":"2024-09-10T09:55:06:670916+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14454,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/2970594311","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2970594311},{"type":"v1","addr":"172.21.15.5:6835","nonce":2970594311}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14514,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6839/3616502174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":3616502174},{"type":"v1","addr":"172.21.15.5:6839","nonce":3616502174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":42,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T09:35:29.626287+0000","modified":"2024-09-10T09:55:06.419829+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24542,"mds_1":24535,"mds_2":24511,"mds_3":24506,"mds_4":24443},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14520":{"gid":14520,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/2073599842","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":2073599842},{"type":"v1","addr":"172.21.15.5:6841","nonce":2073599842}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"k","rank":4,"incarnation":26,"state":"up:active","state_seq":24,"addr":"172.21.15.102:6833/2457007427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":2457007427},{"type":"v1","addr":"172.21.15.102:6833","nonce":2457007427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/356445159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":356445159},{"type":"v1","addr":"172.21.15.159:6833","nonce":356445159}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6837/2894314507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":2894314507},{"type":"v1","addr":"172.21.15.5:6837","nonce":2894314507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.102:6835/1647549111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":1647549111},{"type":"v1","addr":"172.21.15.102:6835","nonce":1647549111}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6835/952657242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":952657242},{"type":"v1","addr":"172.21.15.159:6835","nonce":952657242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":19,"addr":"172.21.15.102:6837/2689776764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":2689776764},{"type":"v1","addr":"172.21.15.102:6837","nonce":2689776764}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.159:6837/620627639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":620627639},{"type":"v1","addr":"172.21.15.159:6837","nonce":620627639}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24535":{"gid":24535,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":18,"addr":"172.21.15.159:6839/2073739599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":2073739599},{"type":"v1","addr":"172.21.15.159:6839","nonce":2073739599}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24542":{"gid":24542,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.102:6839/335409749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6838","nonce":335409749},{"type":"v1","addr":"172.21.15.102:6839","nonce":335409749}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24542,"qdb_cluster":[24542,24535,24511,24506,24443]},"id":1}]} 2024-09-10T10:03:06.317 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 42 2024-09-10T10:03:06.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:06.318+0000 7f80de600640 1 -- 172.21.15.5:0/2883429758 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f80c00a2c80 msgr2=0x7f80c00a5140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:06.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:06.318+0000 7f80de600640 1 --2- 172.21.15.5:0/2883429758 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f80c00a2c80 0x7f80c00a5140 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f80cc0308d0 tx=0x7f80cc005d80 comp rx=0 tx=0).stop 2024-09-10T10:03:06.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:06.318+0000 7f80de600640 1 -- 172.21.15.5:0/2883429758 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f80d8112870 msgr2=0x7f80d819e720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:06.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:06.318+0000 7f80de600640 1 --2- 172.21.15.5:0/2883429758 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f80d8112870 0x7f80d819e720 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f80d807aa10 tx=0x7f80d000edd0 comp rx=0 tx=0).stop 2024-09-10T10:03:06.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:06.318+0000 7f80de600640 1 -- 172.21.15.5:0/2883429758 shutdown_connections 2024-09-10T10:03:06.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:06.318+0000 7f80de600640 1 --2- 172.21.15.5:0/2883429758 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f80c00a2c80 0x7f80c00a5140 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:06.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:06.318+0000 7f80de600640 1 --2- 172.21.15.5:0/2883429758 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f80d8112870 0x7f80d819e720 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:06.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:06.318+0000 7f80de600640 1 --2- 172.21.15.5:0/2883429758 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f80d8077040 0x7f80d81a4b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:06.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:06.318+0000 7f80de600640 1 --2- 172.21.15.5:0/2883429758 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f80d8076720 0x7f80d81a4650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:06.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:06.318+0000 7f80de600640 1 -- 172.21.15.5:0/2883429758 >> 172.21.15.5:0/2883429758 conn(0x7f80d80ff8d0 msgr2=0x7f80d8101cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:06.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:06.318+0000 7f80de600640 1 -- 172.21.15.5:0/2883429758 shutdown_connections 2024-09-10T10:03:06.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:06.319+0000 7f80de600640 1 -- 172.21.15.5:0/2883429758 wait complete. 2024-09-10T10:03:07.075 INFO:tasks.workunit.client.0.smithi005.stdout: 10 176390 14.10 MB/sec execute 525 sec latency 49.438 ms 2024-09-10T10:03:07.159 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T10:03:07.297 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24542 quiesce db / --timeout 90 --expiration 150.0 --await-for 45.0' 2024-09-10T10:03:07.298 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24542 quiesce db / --timeout 90 --expiration 150.0 --await-for 45.0 2024-09-10T10:03:07.557 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:03:07 smithi005 ceph-mon[30350]: pgmap v1136: 129 pgs: 129 active+clean; 786 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 4.0 MiB/s wr, 521 op/s 2024-09-10T10:03:07.626 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:03:07 smithi005 ceph-mon[30350]: from='client.? 172.21.15.5:0/2883429758' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:03:07.626 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:03:07 smithi159 ceph-mon[35513]: pgmap v1136: 129 pgs: 129 active+clean; 786 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 4.0 MiB/s wr, 521 op/s 2024-09-10T10:03:07.626 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:03:07 smithi159 ceph-mon[35513]: from='client.? 172.21.15.5:0/2883429758' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:03:07.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:03:07 smithi102 ceph-mon[35442]: pgmap v1136: 129 pgs: 129 active+clean; 786 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 4.0 MiB/s wr, 521 op/s 2024-09-10T10:03:07.635 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:03:07 smithi102 ceph-mon[35442]: from='client.? 172.21.15.5:0/2883429758' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:03:07.777 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:03:08.075 INFO:tasks.workunit.client.0.smithi005.stdout: 10 176916 14.13 MB/sec execute 526 sec latency 187.035 ms 2024-09-10T10:03:08.549 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:03:08.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:08.611+0000 7f5648600640 1 -- 172.21.15.5:0/3520262853 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f56401036e0 msgr2=0x7f5640107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:08.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:08.611+0000 7f5648600640 1 --2- 172.21.15.5:0/3520262853 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f56401036e0 0x7f5640107b90 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f5634009900 tx=0x7f563402a420 comp rx=0 tx=0).stop 2024-09-10T10:03:08.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:08.616+0000 7f5648600640 1 -- 172.21.15.5:0/3520262853 shutdown_connections 2024-09-10T10:03:08.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:08.616+0000 7f5648600640 1 --2- 172.21.15.5:0/3520262853 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f56401082a0 0x7f564010ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:08.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:08.616+0000 7f5648600640 1 --2- 172.21.15.5:0/3520262853 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f56401036e0 0x7f5640107b90 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:08.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:08.616+0000 7f5648600640 1 --2- 172.21.15.5:0/3520262853 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5640102d30 0x7f5640103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:08.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:08.616+0000 7f5648600640 1 -- 172.21.15.5:0/3520262853 >> 172.21.15.5:0/3520262853 conn(0x7f56400fc7d0 msgr2=0x7f56400febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:08.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:08.616+0000 7f5648600640 1 -- 172.21.15.5:0/3520262853 shutdown_connections 2024-09-10T10:03:08.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:08.616+0000 7f5648600640 1 -- 172.21.15.5:0/3520262853 wait complete. 2024-09-10T10:03:08.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:08.623+0000 7f5648600640 1 Processor -- start 2024-09-10T10:03:08.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:08.624+0000 7f5648600640 1 -- start start 2024-09-10T10:03:08.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:08.624+0000 7f5648600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5640102d30 0x7f56401174c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:08.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:08.624+0000 7f5648600640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f56401036e0 0x7f5640111540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:08.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:08.624+0000 7f5648600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f56401082a0 0x7f5640111a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:08.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:08.624+0000 7f5648600640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f564010ff20 con 0x7f56401082a0 2024-09-10T10:03:08.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:08.624+0000 7f5648600640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f564010fda0 con 0x7f5640102d30 2024-09-10T10:03:08.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:08.624+0000 7f5648600640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f56401100a0 con 0x7f56401036e0 2024-09-10T10:03:08.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:08.627+0000 7f5645c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5640102d30 0x7f56401174c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:08.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:08.627+0000 7f5645200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f56401036e0 0x7f5640111540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:08.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:08.627+0000 7f5645c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5640102d30 0x7f56401174c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:60614/0 (socket says 172.21.15.5:60614) 2024-09-10T10:03:08.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:08.627+0000 7f5645c00640 1 -- 172.21.15.5:0/1200850204 learned_addr learned my addr 172.21.15.5:0/1200850204 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T10:03:08.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:08.627+0000 7f5646600640 1 --2- 172.21.15.5:0/1200850204 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f56401082a0 0x7f5640111a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:08.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:08.627+0000 7f5645200640 1 -- 172.21.15.5:0/1200850204 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5640102d30 msgr2=0x7f56401174c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:08.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:08.627+0000 7f5645200640 1 --2- 172.21.15.5:0/1200850204 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5640102d30 0x7f56401174c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:08.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:08.628+0000 7f5645200640 1 -- 172.21.15.5:0/1200850204 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f56401082a0 msgr2=0x7f5640111a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:08.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:08.628+0000 7f5645200640 1 --2- 172.21.15.5:0/1200850204 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f56401082a0 0x7f5640111a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:08.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:08.628+0000 7f5645200640 1 -- 172.21.15.5:0/1200850204 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5640112040 con 0x7f56401036e0 2024-09-10T10:03:08.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:08.628+0000 7f5645200640 1 --2- 172.21.15.5:0/1200850204 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f56401036e0 0x7f5640111540 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f5634009230 tx=0x7f5634003b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:08.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:08.629+0000 7f563e000640 1 -- 172.21.15.5:0/1200850204 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f563403f070 con 0x7f56401036e0 2024-09-10T10:03:08.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:08.629+0000 7f563e000640 1 -- 172.21.15.5:0/1200850204 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5634031c20 con 0x7f56401036e0 2024-09-10T10:03:08.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:08.629+0000 7f563e000640 1 -- 172.21.15.5:0/1200850204 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5634035030 con 0x7f56401036e0 2024-09-10T10:03:08.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:08.629+0000 7f5648600640 1 -- 172.21.15.5:0/1200850204 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56401122d0 con 0x7f56401036e0 2024-09-10T10:03:08.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:08.629+0000 7f5648600640 1 -- 172.21.15.5:0/1200850204 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56401b3e40 con 0x7f56401036e0 2024-09-10T10:03:08.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:08.641+0000 7f564b040740 1 Processor -- start 2024-09-10T10:03:08.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:08.640+0000 7f563e000640 1 -- 172.21.15.5:0/1200850204 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f5634042070 con 0x7f56401036e0 2024-09-10T10:03:08.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:08.643+0000 7f564b040740 1 -- start start 2024-09-10T10:03:08.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:08.646+0000 7f564b040740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558a7a553b90 0x558a7a553f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:08.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:08.646+0000 7f564b040740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558a7a5544b0 0x558a7a55e830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:08.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:08.646+0000 7f564b040740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558a7a55ed70 0x558a7a561160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:08.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:08.646+0000 7f564b040740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x558a7a4ba3e0 con 0x558a7a553b90 2024-09-10T10:03:08.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:08.646+0000 7f564b040740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x558a7a461d80 con 0x558a7a55ed70 2024-09-10T10:03:08.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:08.646+0000 7f564b040740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x558a7a4b63a0 con 0x558a7a5544b0 2024-09-10T10:03:08.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:08.647+0000 7f5646600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558a7a553b90 0x558a7a553f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:08.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:08.647+0000 7f5646600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558a7a553b90 0x558a7a553f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37142/0 (socket says 172.21.15.5:37142) 2024-09-10T10:03:08.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:08.647+0000 7f5646600640 1 -- 172.21.15.5:0/614721827 learned_addr learned my addr 172.21.15.5:0/614721827 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T10:03:08.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:08.649+0000 7f5645200640 1 --2- 172.21.15.5:0/614721827 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558a7a55ed70 0x558a7a561160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:08.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:08.649+0000 7f5645c00640 1 --2- 172.21.15.5:0/614721827 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558a7a5544b0 0x558a7a55e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:08.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:08.652+0000 7f5646600640 1 -- 172.21.15.5:0/614721827 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558a7a5544b0 msgr2=0x558a7a55e830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:03:08.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:08.652+0000 7f5646600640 1 --2- 172.21.15.5:0/614721827 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558a7a5544b0 0x558a7a55e830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:08.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:08.652+0000 7f5646600640 1 -- 172.21.15.5:0/614721827 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558a7a55ed70 msgr2=0x558a7a561160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:03:08.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:08.652+0000 7f5646600640 1 --2- 172.21.15.5:0/614721827 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558a7a55ed70 0x558a7a561160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:08.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:08.652+0000 7f5646600640 1 -- 172.21.15.5:0/614721827 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558a7a480ae0 con 0x558a7a553b90 2024-09-10T10:03:08.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:08.653+0000 7f563e000640 1 --2- 172.21.15.5:0/1200850204 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f56280a2c50 0x7f56280a5110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:08.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:08.653+0000 7f563e000640 1 -- 172.21.15.5:0/1200850204 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f56340efbc0 con 0x7f56401036e0 2024-09-10T10:03:08.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:08.655+0000 7f5645c00640 1 --2- 172.21.15.5:0/1200850204 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f56280a2c50 0x7f56280a5110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:08.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:08.655+0000 7f5646600640 1 --2- 172.21.15.5:0/614721827 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558a7a553b90 0x558a7a553f70 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f562c00a840 tx=0x7f562c00ad00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7dfd62d514d39bb2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:08.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:08.657+0000 7f5632a00640 1 -- 172.21.15.5:0/614721827 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f562c015070 con 0x558a7a553b90 2024-09-10T10:03:08.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:08.657+0000 7f5632a00640 1 -- 172.21.15.5:0/614721827 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f562c00cb40 con 0x558a7a553b90 2024-09-10T10:03:08.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:08.657+0000 7f5632a00640 1 -- 172.21.15.5:0/614721827 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f562c0185c0 con 0x558a7a553b90 2024-09-10T10:03:08.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:08.657+0000 7f5645c00640 1 --2- 172.21.15.5:0/1200850204 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f56280a2c50 0x7f56280a5110 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x558a7a550f80 tx=0x7f5638009720 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:08.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:08.658+0000 7f564b040740 1 -- 172.21.15.5:0/614721827 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558a7a553b90 msgr2=0x558a7a553f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:08.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:08.658+0000 7f564b040740 1 --2- 172.21.15.5:0/614721827 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558a7a553b90 0x558a7a553f70 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f562c00a840 tx=0x7f562c00ad00 comp rx=0 tx=0).stop 2024-09-10T10:03:08.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:08.661+0000 7f564b040740 1 -- 172.21.15.5:0/614721827 shutdown_connections 2024-09-10T10:03:08.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:08.661+0000 7f564b040740 1 --2- 172.21.15.5:0/614721827 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558a7a55ed70 0x558a7a561160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:08.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:08.661+0000 7f564b040740 1 --2- 172.21.15.5:0/614721827 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558a7a5544b0 0x558a7a55e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:08.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:08.661+0000 7f564b040740 1 --2- 172.21.15.5:0/614721827 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558a7a553b90 0x558a7a553f70 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:08.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:08.661+0000 7f564b040740 1 -- 172.21.15.5:0/614721827 >> 172.21.15.5:0/614721827 conn(0x558a7a550120 msgr2=0x558a7a550500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:08.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:08.665+0000 7f564b040740 1 -- 172.21.15.5:0/614721827 shutdown_connections 2024-09-10T10:03:08.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:08.665+0000 7f564b040740 1 -- 172.21.15.5:0/614721827 wait complete. 2024-09-10T10:03:08.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:08.666+0000 7f564b040740 1 -- start start 2024-09-10T10:03:08.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:08.667+0000 7f564b040740 1 Processor -- start 2024-09-10T10:03:08.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:08.670+0000 7f564b040740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558a7a553b90 0x558a7a55bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:08.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:08.670+0000 7f564b040740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558a7a5544b0 0x558a7a55c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:08.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:08.670+0000 7f564b040740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558a7a55ed70 0x558a7a55c730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:08.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:08.671+0000 7f5645c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558a7a553b90 0x558a7a55bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:08.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:08.671+0000 7f5645c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558a7a553b90 0x558a7a55bcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:60630/0 (socket says 172.21.15.5:60630) 2024-09-10T10:03:08.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:08.674+0000 7f5646600640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558a7a55ed70 0x558a7a55c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:08.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:08.674+0000 7f5646600640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558a7a55ed70 0x558a7a55c730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:42128/0 (socket says 172.21.15.5:42128) 2024-09-10T10:03:08.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:08.674+0000 7f5645200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558a7a5544b0 0x558a7a55c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:08.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:08.675+0000 7f5645c00640 1 -- 172.21.15.5:0/3376230769 learned_addr learned my addr 172.21.15.5:0/3376230769 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T10:03:08.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:08.675+0000 7f564b040740 1 -- 172.21.15.5:0/3376230769 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x558a7a5619e0 con 0x558a7a5544b0 2024-09-10T10:03:08.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:08.675+0000 7f564b040740 1 -- 172.21.15.5:0/3376230769 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x558a7a4750f0 con 0x558a7a553b90 2024-09-10T10:03:08.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:08.675+0000 7f564b040740 1 -- 172.21.15.5:0/3376230769 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x558a7a561b60 con 0x558a7a55ed70 2024-09-10T10:03:08.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:08.676+0000 7f564b040740 10 client.0 _ll_register_callbacks cb 0x558a7a4669e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:03:08.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:08.676+0000 7f5645c00640 1 -- 172.21.15.5:0/3376230769 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558a7a55ed70 msgr2=0x558a7a55c730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:03:08.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:08.676+0000 7f5645c00640 1 --2- 172.21.15.5:0/3376230769 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558a7a55ed70 0x558a7a55c730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:08.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:08.676+0000 7f5645c00640 1 -- 172.21.15.5:0/3376230769 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558a7a5544b0 msgr2=0x558a7a55c1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:03:08.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:08.676+0000 7f5645c00640 1 --2- 172.21.15.5:0/3376230769 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558a7a5544b0 0x558a7a55c1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:08.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:08.676+0000 7f5645c00640 1 -- 172.21.15.5:0/3376230769 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558a7a22b3c0 con 0x558a7a553b90 2024-09-10T10:03:08.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:08.677+0000 7f5645c00640 1 --2- 172.21.15.5:0/3376230769 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558a7a553b90 0x558a7a55bcb0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f5638003ef0 tx=0x7f5638004ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=56dba283ffa7e5d9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:08.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:08.678+0000 7f5627400640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-09-10T10:03:08.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:08.678+0000 7f5627400640 1 -- 172.21.15.5:0/3376230769 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5638011850 con 0x558a7a553b90 2024-09-10T10:03:08.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:08.679+0000 7f5627400640 1 -- 172.21.15.5:0/3376230769 <== mon.2 v2:172.21.15.102:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f5638018030 con 0x558a7a553b90 2024-09-10T10:03:08.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:08.681+0000 7f564b040740 1 -- 172.21.15.5:0/3376230769 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f562c00cb40 con 0x558a7a553b90 2024-09-10T10:03:08.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:08.682+0000 7f5627400640 1 -- 172.21.15.5:0/3376230769 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f56380119f0 con 0x558a7a553b90 2024-09-10T10:03:08.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:08.682+0000 7f564b040740 10 client.30056 fetch_fsmap learned FSMap version 42 2024-09-10T10:03:08.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:08.682+0000 7f564b040740 1 -- 172.21.15.5:0/3376230769 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x558a7a56d700 con 0x558a7a553b90 2024-09-10T10:03:08.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:08.682+0000 7f5627400640 1 -- 172.21.15.5:0/3376230769 <== mon.2 v2:172.21.15.102:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f562c00cb40 con 0x558a7a553b90 2024-09-10T10:03:08.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:08.683+0000 7f5627400640 1 -- 172.21.15.5:0/3376230769 <== mon.2 v2:172.21.15.102:3300/0 5 ==== fsmap(e 42) ==== 8370+0+0 (secure 0 0 0) 0x7f563800f040 con 0x558a7a553b90 2024-09-10T10:03:08.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:08.688+0000 7f564b040740 10 client.30056 fetch_fsmap finished waiting for FSMap version 42 2024-09-10T10:03:08.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:08.688+0000 7f564b040740 10 client.30056 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T10:03:08.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:08.688+0000 7f564b040740 20 client.30056 populate_metadata read hostname 'smithi005' 2024-09-10T10:03:08.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:08.688+0000 7f564b040740 1 --2- 172.21.15.5:0/3376230769 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x558a7a56eb10 0x558a7a570fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:08.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:08.688+0000 7f564b040740 4 client.30056 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:03:08.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:08.688+0000 7f564b040740 1 -- 172.21.15.5:0/3376230769 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558a7a4c7950 con 0x558a7a56eb10 2024-09-10T10:03:08.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:08.688+0000 7f5645200640 1 --2- 172.21.15.5:0/3376230769 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x558a7a56eb10 0x558a7a570fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:08.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:08.692+0000 7f5645200640 1 --2- 172.21.15.5:0/3376230769 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x558a7a56eb10 0x558a7a570fb0 crc :-1 s=READY pgs=331 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c83617d48dd7fec8 server_cookie=c38c3d0449c6083a in_seq=0 out_seq=0 2024-09-10T10:03:08.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:08.692+0000 7f5627400640 10 client.30056 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T10:03:08.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:08.693+0000 7f5627400640 1 -- 172.21.15.5:0/3376230769 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558a7a4c7950 con 0x558a7a56eb10 2024-09-10T10:03:08.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:08.693+0000 7f5627400640 10 client.30056 handle_command_reply: tid=0 2024-09-10T10:03:08.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:08.693+0000 7f564b040740 1 client.30056 shutdown 2024-09-10T10:03:08.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:08.706+0000 7f564b040740 1 -- 172.21.15.5:0/3376230769 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558a7a553b90 msgr2=0x558a7a55bcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:08.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:08.706+0000 7f564b040740 1 --2- 172.21.15.5:0/3376230769 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558a7a553b90 0x558a7a55bcb0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f5638003ef0 tx=0x7f5638004ab0 comp rx=0 tx=0).stop 2024-09-10T10:03:08.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:08.706+0000 7f564b040740 1 -- 172.21.15.5:0/3376230769 shutdown_connections 2024-09-10T10:03:08.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:08.706+0000 7f564b040740 1 --2- 172.21.15.5:0/3376230769 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x558a7a56eb10 0x558a7a570fb0 crc :-1 s=READY pgs=331 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:08.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:08.706+0000 7f564b040740 1 --2- 172.21.15.5:0/3376230769 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558a7a55ed70 0x558a7a55c730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:08.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:08.706+0000 7f564b040740 1 --2- 172.21.15.5:0/3376230769 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558a7a5544b0 0x558a7a55c1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:08.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:08.706+0000 7f564b040740 1 --2- 172.21.15.5:0/3376230769 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558a7a553b90 0x558a7a55bcb0 secure :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f5638003ef0 tx=0x7f5638004ab0 comp rx=0 tx=0).stop 2024-09-10T10:03:08.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:08.706+0000 7f564b040740 1 -- 172.21.15.5:0/3376230769 >> 172.21.15.5:0/3376230769 conn(0x558a7a550120 msgr2=0x558a7a555e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:08.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:08.706+0000 7f5627400640 0 client.30056 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T10:03:08.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:08.707+0000 7f564b040740 1 -- 172.21.15.5:0/3376230769 shutdown_connections 2024-09-10T10:03:08.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:08.707+0000 7f564b040740 1 -- 172.21.15.5:0/3376230769 wait complete. 2024-09-10T10:03:08.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:08.707+0000 7f564b040740 20 client.30056 trim_cache size 0 max 16384 2024-09-10T10:03:08.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:08.748+0000 7f564b040740 1 Processor -- start 2024-09-10T10:03:08.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:08.748+0000 7f564b040740 1 -- start start 2024-09-10T10:03:08.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:08.749+0000 7f564b040740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558a7a5544b0 0x558a7a5565f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:08.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:08.749+0000 7f564b040740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558a7a55ed70 0x558a7a55c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:08.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:08.749+0000 7f564b040740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558a7a5768d0 0x558a7a576d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:08.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:08.749+0000 7f564b040740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f56401bdc80 con 0x558a7a55ed70 2024-09-10T10:03:08.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:08.749+0000 7f564b040740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f56401bf7c0 con 0x558a7a5544b0 2024-09-10T10:03:08.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:08.750+0000 7f5645c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558a7a5544b0 0x558a7a5565f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:08.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:08.750+0000 7f5645c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558a7a5544b0 0x558a7a5565f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:60646/0 (socket says 172.21.15.5:60646) 2024-09-10T10:03:08.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:08.750+0000 7f5645c00640 1 -- 172.21.15.5:0/2131610549 learned_addr learned my addr 172.21.15.5:0/2131610549 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T10:03:08.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:08.750+0000 7f5646600640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558a7a5768d0 0x558a7a576d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:08.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:08.750+0000 7f5645200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558a7a55ed70 0x558a7a55c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:08.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:08.751+0000 7f564b040740 1 -- 172.21.15.5:0/2131610549 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f56401be350 con 0x558a7a5768d0 2024-09-10T10:03:08.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:08.753+0000 7f5645c00640 1 -- 172.21.15.5:0/2131610549 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558a7a5768d0 msgr2=0x558a7a576d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:03:08.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:08.753+0000 7f5645c00640 1 --2- 172.21.15.5:0/2131610549 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558a7a5768d0 0x558a7a576d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:08.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:08.753+0000 7f5645c00640 1 -- 172.21.15.5:0/2131610549 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558a7a55ed70 msgr2=0x558a7a55c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:03:08.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:08.753+0000 7f5645c00640 1 --2- 172.21.15.5:0/2131610549 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558a7a55ed70 0x558a7a55c5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:08.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:08.753+0000 7f5645c00640 1 -- 172.21.15.5:0/2131610549 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558a7a54a470 con 0x558a7a5544b0 2024-09-10T10:03:08.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:08.754+0000 7f5645c00640 1 --2- 172.21.15.5:0/2131610549 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558a7a5544b0 0x558a7a5565f0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f5638012f40 tx=0x7f5638012490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c776f3d8872da32c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:08.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:08.754+0000 7f5632000640 1 -- 172.21.15.5:0/2131610549 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56380284a0 con 0x558a7a5544b0 2024-09-10T10:03:08.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:08.754+0000 7f5632000640 1 -- 172.21.15.5:0/2131610549 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5638037070 con 0x558a7a5544b0 2024-09-10T10:03:08.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:08.755+0000 7f564b040740 1 -- 172.21.15.5:0/2131610549 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558a7a5544b0 msgr2=0x558a7a5565f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:08.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:08.755+0000 7f564b040740 1 --2- 172.21.15.5:0/2131610549 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558a7a5544b0 0x558a7a5565f0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f5638012f40 tx=0x7f5638012490 comp rx=0 tx=0).stop 2024-09-10T10:03:08.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:08.756+0000 7f564b040740 1 -- 172.21.15.5:0/2131610549 shutdown_connections 2024-09-10T10:03:08.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:08.756+0000 7f564b040740 1 --2- 172.21.15.5:0/2131610549 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558a7a5768d0 0x558a7a576d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:08.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:08.756+0000 7f564b040740 1 --2- 172.21.15.5:0/2131610549 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558a7a55ed70 0x558a7a55c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:08.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:08.756+0000 7f564b040740 1 --2- 172.21.15.5:0/2131610549 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558a7a5544b0 0x558a7a5565f0 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:08.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:08.756+0000 7f564b040740 1 -- 172.21.15.5:0/2131610549 >> 172.21.15.5:0/2131610549 conn(0x558a7a550120 msgr2=0x558a7a55dd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:08.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:08.757+0000 7f564b040740 1 -- 172.21.15.5:0/2131610549 shutdown_connections 2024-09-10T10:03:08.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:08.757+0000 7f564b040740 1 -- 172.21.15.5:0/2131610549 wait complete. 2024-09-10T10:03:08.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:08.758+0000 7f564b040740 1 -- start start 2024-09-10T10:03:08.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:08.758+0000 7f564b040740 1 Processor -- start 2024-09-10T10:03:08.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:08.760+0000 7f564b040740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558a7a5544b0 0x558a7a565db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:08.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:08.760+0000 7f5645c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558a7a5544b0 0x558a7a565db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:08.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:08.760+0000 7f5645c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558a7a5544b0 0x558a7a565db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:60652/0 (socket says 172.21.15.5:60652) 2024-09-10T10:03:08.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:08.760+0000 7f5645c00640 1 -- 172.21.15.5:0/2491652303 learned_addr learned my addr 172.21.15.5:0/2491652303 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T10:03:08.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:08.760+0000 7f564b040740 1 --2- 172.21.15.5:0/2491652303 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558a7a55ed70 0x558a7a5662f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:08.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:08.761+0000 7f564b040740 1 --2- 172.21.15.5:0/2491652303 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558a7a5768d0 0x558a7a566830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:08.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:08.761+0000 7f5645200640 1 --2- 172.21.15.5:0/2491652303 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558a7a55ed70 0x558a7a5662f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:08.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:08.761+0000 7f5646600640 1 --2- 172.21.15.5:0/2491652303 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558a7a5768d0 0x558a7a566830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:08.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:08.762+0000 7f564b040740 1 -- 172.21.15.5:0/2491652303 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f56401bf0f0 con 0x558a7a55ed70 2024-09-10T10:03:08.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:08.762+0000 7f564b040740 1 -- 172.21.15.5:0/2491652303 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f56401bea20 con 0x558a7a5544b0 2024-09-10T10:03:08.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:08.762+0000 7f564b040740 1 -- 172.21.15.5:0/2491652303 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f56401bbc60 con 0x558a7a5768d0 2024-09-10T10:03:08.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:08.763+0000 7f564b040740 10 client.0 _ll_register_callbacks cb 0x7f5638003ba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:03:08.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:08.763+0000 7f5645200640 1 -- 172.21.15.5:0/2491652303 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558a7a5768d0 msgr2=0x558a7a566830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:03:08.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:08.763+0000 7f5645200640 1 --2- 172.21.15.5:0/2491652303 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558a7a5768d0 0x558a7a566830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:08.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:08.763+0000 7f5645200640 1 -- 172.21.15.5:0/2491652303 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558a7a5544b0 msgr2=0x558a7a565db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:03:08.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:08.763+0000 7f5645200640 1 --2- 172.21.15.5:0/2491652303 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558a7a5544b0 0x558a7a565db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:08.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:08.763+0000 7f5645200640 1 -- 172.21.15.5:0/2491652303 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558a7a567030 con 0x558a7a55ed70 2024-09-10T10:03:08.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:08.763+0000 7f564b040740 1 -- 172.21.15.5:0/2491652303 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5638037070 con 0x558a7a55ed70 2024-09-10T10:03:08.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:08.763+0000 7f5627400640 10 client.19962 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-10T10:03:08.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:08.763+0000 7f5645200640 1 --2- 172.21.15.5:0/2491652303 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558a7a55ed70 0x558a7a5662f0 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f56340ed320 tx=0x7f5634000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9405a387ac8cfa3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:08.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:08.764+0000 7f5627400640 1 -- 172.21.15.5:0/2491652303 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5634049040 con 0x558a7a55ed70 2024-09-10T10:03:08.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:08.764+0000 7f5627400640 1 -- 172.21.15.5:0/2491652303 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f56340ba0d0 con 0x558a7a55ed70 2024-09-10T10:03:08.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:08.764+0000 7f5627400640 1 -- 172.21.15.5:0/2491652303 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f56340bc730 con 0x558a7a55ed70 2024-09-10T10:03:08.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:08.764+0000 7f564b040740 10 client.19962 fetch_fsmap learned FSMap version 42 2024-09-10T10:03:08.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:08.764+0000 7f564b040740 1 -- 172.21.15.5:0/2491652303 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({fsmap=42}) -- 0x558a7a5eae20 con 0x558a7a55ed70 2024-09-10T10:03:08.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:08.765+0000 7f5627400640 1 -- 172.21.15.5:0/2491652303 <== mon.0 v2:172.21.15.5:3300/0 4 ==== fsmap(e 42) ==== 8370+0+0 (secure 0 0 0) 0x7f56340b86a0 con 0x558a7a55ed70 2024-09-10T10:03:08.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:08.765+0000 7f564b040740 10 client.19962 fetch_fsmap finished waiting for FSMap version 42 2024-09-10T10:03:08.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:08.765+0000 7f564b040740 10 client.19962 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T10:03:08.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:08.765+0000 7f564b040740 20 client.19962 populate_metadata read hostname 'smithi005' 2024-09-10T10:03:08.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:08.765+0000 7f564b040740 1 --2- 172.21.15.5:0/2491652303 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x558a7a5eb410 0x558a7a5ed830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:08.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:08.765+0000 7f564b040740 4 client.19962 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T10:03:08.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:08.765+0000 7f564b040740 1 -- 172.21.15.5:0/2491652303 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "scrub status"}) -- 0x558a7a4c1e60 con 0x558a7a5eb410 2024-09-10T10:03:08.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:08.766+0000 7f5645c00640 1 --2- 172.21.15.5:0/2491652303 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x558a7a5eb410 0x558a7a5ed830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:08.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:08.767+0000 7f5645c00640 1 --2- 172.21.15.5:0/2491652303 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x558a7a5eb410 0x558a7a5ed830 crc :-1 s=READY pgs=332 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3d76d7e494d5598b server_cookie=101290f7a58ce02e in_seq=0 out_seq=0 2024-09-10T10:03:08.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:08.767+0000 7f5627400640 10 client.19962 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T10:03:08.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:08.767+0000 7f5627400640 1 -- 172.21.15.5:0/2491652303 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x558a7a4c1e60 con 0x558a7a5eb410 2024-09-10T10:03:08.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:08.767+0000 7f5627400640 10 client.19962 handle_command_reply: tid=0 2024-09-10T10:03:08.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:08.767+0000 7f564b040740 1 client.19962 shutdown 2024-09-10T10:03:08.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:08.768+0000 7f564b040740 1 -- 172.21.15.5:0/2491652303 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558a7a55ed70 msgr2=0x558a7a5662f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:08.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:08.768+0000 7f564b040740 1 --2- 172.21.15.5:0/2491652303 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558a7a55ed70 0x558a7a5662f0 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f56340ed320 tx=0x7f5634000bc0 comp rx=0 tx=0).stop 2024-09-10T10:03:08.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:08.768+0000 7f564b040740 1 -- 172.21.15.5:0/2491652303 shutdown_connections 2024-09-10T10:03:08.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:08.768+0000 7f564b040740 1 --2- 172.21.15.5:0/2491652303 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x558a7a5eb410 0x558a7a5ed830 crc :-1 s=READY pgs=332 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:08.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:08.768+0000 7f564b040740 1 --2- 172.21.15.5:0/2491652303 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558a7a5768d0 0x558a7a566830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:08.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:08.768+0000 7f564b040740 1 --2- 172.21.15.5:0/2491652303 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558a7a55ed70 0x558a7a5662f0 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:08.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:08.768+0000 7f564b040740 1 --2- 172.21.15.5:0/2491652303 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558a7a5544b0 0x558a7a565db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:08.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:08.768+0000 7f564b040740 1 -- 172.21.15.5:0/2491652303 >> 172.21.15.5:0/2491652303 conn(0x558a7a550120 msgr2=0x558a7a55dd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:08.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:08.768+0000 7f5627400640 0 client.19962 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T10:03:08.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:08.769+0000 7f564b040740 1 -- 172.21.15.5:0/2491652303 shutdown_connections 2024-09-10T10:03:08.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:08.769+0000 7f564b040740 1 -- 172.21.15.5:0/2491652303 wait complete. 2024-09-10T10:03:08.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:08.770+0000 7f564b040740 20 client.19962 trim_cache size 0 max 16384 2024-09-10T10:03:08.772 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-10T10:03:08.772 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "no active scrubs running", 2024-09-10T10:03:08.772 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": {} 2024-09-10T10:03:08.772 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-10T10:03:08.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:08.772+0000 7f5648600640 1 -- 172.21.15.5:0/1200850204 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f56280a2c50 msgr2=0x7f56280a5110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:08.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:08.772+0000 7f5648600640 1 --2- 172.21.15.5:0/1200850204 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f56280a2c50 0x7f56280a5110 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x558a7a550f80 tx=0x7f5638009720 comp rx=0 tx=0).stop 2024-09-10T10:03:08.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:08.772+0000 7f5648600640 1 -- 172.21.15.5:0/1200850204 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f56401036e0 msgr2=0x7f5640111540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:08.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:08.772+0000 7f5648600640 1 --2- 172.21.15.5:0/1200850204 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f56401036e0 0x7f5640111540 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f5634009230 tx=0x7f5634003b90 comp rx=0 tx=0).stop 2024-09-10T10:03:08.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:08.772+0000 7f5648600640 1 -- 172.21.15.5:0/1200850204 shutdown_connections 2024-09-10T10:03:08.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:08.772+0000 7f5648600640 1 --2- 172.21.15.5:0/1200850204 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f56280a2c50 0x7f56280a5110 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:08.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:08.772+0000 7f5648600640 1 --2- 172.21.15.5:0/1200850204 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f56401082a0 0x7f5640111a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:08.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:08.772+0000 7f5648600640 1 --2- 172.21.15.5:0/1200850204 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f56401036e0 0x7f5640111540 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:08.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:08.772+0000 7f5648600640 1 --2- 172.21.15.5:0/1200850204 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5640102d30 0x7f56401174c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:08.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:08.772+0000 7f5648600640 1 -- 172.21.15.5:0/1200850204 >> 172.21.15.5:0/1200850204 conn(0x7f56400fc7d0 msgr2=0x7f5640105d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:08.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:08.773+0000 7f5648600640 1 -- 172.21.15.5:0/1200850204 shutdown_connections 2024-09-10T10:03:08.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:08.773+0000 7f5648600640 1 -- 172.21.15.5:0/1200850204 wait complete. 2024-09-10T10:03:09.077 INFO:tasks.workunit.client.0.smithi005.stdout: 10 176976 14.11 MB/sec execute 527 sec latency 875.009 ms 2024-09-10T10:03:09.544 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:03:09 smithi005 ceph-mon[30350]: pgmap v1137: 129 pgs: 129 active+clean; 786 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 3.5 MiB/s wr, 421 op/s 2024-09-10T10:03:09.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:03:09 smithi159 ceph-mon[35513]: pgmap v1137: 129 pgs: 129 active+clean; 786 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 3.5 MiB/s wr, 421 op/s 2024-09-10T10:03:09.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:03:09 smithi102 ceph-mon[35442]: pgmap v1137: 129 pgs: 129 active+clean; 786 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 3.5 MiB/s wr, 421 op/s 2024-09-10T10:03:10.076 INFO:tasks.workunit.client.0.smithi005.stdout: 10 177022 14.08 MB/sec execute 528 sec latency 1864.668 ms 2024-09-10T10:03:11.076 INFO:tasks.workunit.client.0.smithi005.stdout: 10 177022 14.05 MB/sec execute 529 sec latency 2864.712 ms 2024-09-10T10:03:11.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:03:11 smithi159 ceph-mon[35513]: pgmap v1138: 129 pgs: 129 active+clean; 784 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 5.4 MiB/s wr, 538 op/s 2024-09-10T10:03:11.617 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:03:11 smithi005 ceph-mon[30350]: pgmap v1138: 129 pgs: 129 active+clean; 784 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 5.4 MiB/s wr, 538 op/s 2024-09-10T10:03:11.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:03:11 smithi102 ceph-mon[35442]: pgmap v1138: 129 pgs: 129 active+clean; 784 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 5.4 MiB/s wr, 538 op/s 2024-09-10T10:03:11.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:11.856+0000 7fa511800640 1 -- 172.21.15.5:0/3444743768 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa50c101df0 msgr2=0x7fa50c1021d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:11.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:11.856+0000 7fa511800640 1 --2- 172.21.15.5:0/3444743768 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa50c101df0 0x7fa50c1021d0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7fa504009ca0 tx=0x7fa50402a8c0 comp rx=0 tx=0).stop 2024-09-10T10:03:11.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:11.856+0000 7fa511800640 1 -- 172.21.15.5:0/3444743768 shutdown_connections 2024-09-10T10:03:11.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:11.856+0000 7fa511800640 1 --2- 172.21.15.5:0/3444743768 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa50c106f20 0x7fa50c10eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:11.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:11.856+0000 7fa511800640 1 --2- 172.21.15.5:0/3444743768 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa50c1027a0 0x7fa50c1067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:11.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:11.856+0000 7fa511800640 1 --2- 172.21.15.5:0/3444743768 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa50c101df0 0x7fa50c1021d0 secure :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7fa504009ca0 tx=0x7fa50402a8c0 comp rx=0 tx=0).stop 2024-09-10T10:03:11.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:11.856+0000 7fa511800640 1 -- 172.21.15.5:0/3444743768 >> 172.21.15.5:0/3444743768 conn(0x7fa50c0fd8b0 msgr2=0x7fa50c0ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:11.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:11.857+0000 7fa511800640 1 -- 172.21.15.5:0/3444743768 shutdown_connections 2024-09-10T10:03:11.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:11.858+0000 7fa511800640 1 -- 172.21.15.5:0/3444743768 wait complete. 2024-09-10T10:03:11.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:11.859+0000 7fa511800640 1 Processor -- start 2024-09-10T10:03:11.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:11.859+0000 7fa511800640 1 -- start start 2024-09-10T10:03:11.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:11.860+0000 7fa511800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa50c1027a0 0x7fa50c198670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:11.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:11.860+0000 7fa511800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa50c106f20 0x7fa50c198bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:11.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:11.860+0000 7fa511800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa50c19cf40 0x7fa50c19d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:11.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:11.860+0000 7fa511800640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa50c111ff0 con 0x7fa50c1027a0 2024-09-10T10:03:11.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:11.860+0000 7fa511800640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fa50c111e70 con 0x7fa50c106f20 2024-09-10T10:03:11.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:11.860+0000 7fa511800640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fa50c112170 con 0x7fa50c19cf40 2024-09-10T10:03:11.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:11.861+0000 7fa50a000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa50c106f20 0x7fa50c198bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:11.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:11.861+0000 7fa50a000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa50c106f20 0x7fa50c198bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:43986/0 (socket says 172.21.15.5:43986) 2024-09-10T10:03:11.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:11.861+0000 7fa50a000640 1 -- 172.21.15.5:0/1587156680 learned_addr learned my addr 172.21.15.5:0/1587156680 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T10:03:11.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:11.861+0000 7fa50b400640 1 --2- 172.21.15.5:0/1587156680 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa50c19cf40 0x7fa50c19d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:11.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:11.861+0000 7fa50aa00640 1 --2- 172.21.15.5:0/1587156680 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa50c1027a0 0x7fa50c198670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:11.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:11.862+0000 7fa50a000640 1 -- 172.21.15.5:0/1587156680 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa50c19cf40 msgr2=0x7fa50c19d3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:11.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:11.862+0000 7fa50a000640 1 --2- 172.21.15.5:0/1587156680 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa50c19cf40 0x7fa50c19d3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:11.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:11.862+0000 7fa50a000640 1 -- 172.21.15.5:0/1587156680 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa50c1027a0 msgr2=0x7fa50c198670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:11.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:11.862+0000 7fa50a000640 1 --2- 172.21.15.5:0/1587156680 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa50c1027a0 0x7fa50c198670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:11.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:11.862+0000 7fa50a000640 1 -- 172.21.15.5:0/1587156680 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa50c19da00 con 0x7fa50c106f20 2024-09-10T10:03:11.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:11.863+0000 7fa50a000640 1 --2- 172.21.15.5:0/1587156680 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa50c106f20 0x7fa50c198bb0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7fa4f800b4a0 tx=0x7fa4f800b960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:11.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:11.864+0000 7fa503400640 1 -- 172.21.15.5:0/1587156680 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4f8014070 con 0x7fa50c106f20 2024-09-10T10:03:11.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:11.864+0000 7fa503400640 1 -- 172.21.15.5:0/1587156680 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa4f8010af0 con 0x7fa50c106f20 2024-09-10T10:03:11.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:11.864+0000 7fa503400640 1 -- 172.21.15.5:0/1587156680 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4f8017d10 con 0x7fa50c106f20 2024-09-10T10:03:11.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:11.865+0000 7fa511800640 1 -- 172.21.15.5:0/1587156680 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa50c19dcf0 con 0x7fa50c106f20 2024-09-10T10:03:11.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:11.866+0000 7fa511800640 1 -- 172.21.15.5:0/1587156680 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa50c06bea0 con 0x7fa50c106f20 2024-09-10T10:03:11.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:11.874+0000 7fa51425b740 1 Processor -- start 2024-09-10T10:03:11.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:11.874+0000 7fa51425b740 1 -- start start 2024-09-10T10:03:11.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:11.877+0000 7fa51425b740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ec09f8e930 0x55ec09f8d250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:11.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:11.877+0000 7fa51425b740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ec09f8d790 0x55ec09f8dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:11.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:11.877+0000 7fa51425b740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ec09fa0c30 0x55ec09fa3020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:11.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:11.877+0000 7fa51425b740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55ec09ef3530 con 0x55ec09f8e930 2024-09-10T10:03:11.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:11.877+0000 7fa51425b740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55ec09e9a860 con 0x55ec09fa0c30 2024-09-10T10:03:11.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:11.877+0000 7fa51425b740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55ec09eef4f0 con 0x55ec09f8d790 2024-09-10T10:03:11.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:11.878+0000 7fa50aa00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ec09f8d790 0x55ec09f8dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:11.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:11.878+0000 7fa50aa00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ec09f8d790 0x55ec09f8dbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:38510/0 (socket says 172.21.15.5:38510) 2024-09-10T10:03:11.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:11.878+0000 7fa50aa00640 1 -- 172.21.15.5:0/490895460 learned_addr learned my addr 172.21.15.5:0/490895460 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T10:03:11.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:11.878+0000 7fa503400640 1 -- 172.21.15.5:0/1587156680 <== mon.2 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7fa4f8017070 con 0x7fa50c106f20 2024-09-10T10:03:11.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:11.879+0000 7fa50b400640 1 --2- 172.21.15.5:0/490895460 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ec09f8e930 0x55ec09f8d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:11.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:11.880+0000 7fa50b400640 1 -- 172.21.15.5:0/490895460 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ec09f8d790 msgr2=0x55ec09f8dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:03:11.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:11.880+0000 7fa50b400640 1 --2- 172.21.15.5:0/490895460 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ec09f8d790 0x55ec09f8dbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:11.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:11.880+0000 7fa50b400640 1 -- 172.21.15.5:0/490895460 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ec09fa0c30 msgr2=0x55ec09fa3020 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T10:03:11.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:11.880+0000 7fa50b400640 1 --2- 172.21.15.5:0/490895460 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ec09fa0c30 0x55ec09fa3020 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:11.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:11.880+0000 7fa50b400640 1 -- 172.21.15.5:0/490895460 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ec09c64b90 con 0x55ec09f8e930 2024-09-10T10:03:11.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:11.881+0000 7fa503400640 1 --2- 172.21.15.5:0/1587156680 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fa4f00a2d10 0x7fa4f00a51d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:11.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:11.882+0000 7fa50b400640 1 --2- 172.21.15.5:0/490895460 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ec09f8e930 0x55ec09f8d250 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7fa4fc009b80 tx=0x7fa4fc011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f939bf6b17a8e7b5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:11.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:11.882+0000 7fa50aa00640 1 --2- 172.21.15.5:0/1587156680 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fa4f00a2d10 0x7fa4f00a51d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:11.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:11.883+0000 7fa503400640 1 -- 172.21.15.5:0/1587156680 <== mon.2 v2:172.21.15.102:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fa4f80cbe90 con 0x7fa50c106f20 2024-09-10T10:03:11.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:11.883+0000 7fa4f7e00640 1 -- 172.21.15.5:0/490895460 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4fc004030 con 0x55ec09f8e930 2024-09-10T10:03:11.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:11.883+0000 7fa4f7e00640 1 -- 172.21.15.5:0/490895460 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa4fc005500 con 0x55ec09f8e930 2024-09-10T10:03:11.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:11.883+0000 7fa4f7e00640 1 -- 172.21.15.5:0/490895460 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4fc004d60 con 0x55ec09f8e930 2024-09-10T10:03:11.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:11.884+0000 7fa51425b740 1 -- 172.21.15.5:0/490895460 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ec09f8e930 msgr2=0x55ec09f8d250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:11.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:11.885+0000 7fa51425b740 1 --2- 172.21.15.5:0/490895460 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ec09f8e930 0x55ec09f8d250 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7fa4fc009b80 tx=0x7fa4fc011810 comp rx=0 tx=0).stop 2024-09-10T10:03:11.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:11.885+0000 7fa51425b740 1 -- 172.21.15.5:0/490895460 shutdown_connections 2024-09-10T10:03:11.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:11.885+0000 7fa51425b740 1 --2- 172.21.15.5:0/490895460 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ec09fa0c30 0x55ec09fa3020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:11.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:11.885+0000 7fa51425b740 1 --2- 172.21.15.5:0/490895460 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ec09f8d790 0x55ec09f8dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:11.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:11.885+0000 7fa51425b740 1 --2- 172.21.15.5:0/490895460 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ec09f8e930 0x55ec09f8d250 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:11.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:11.885+0000 7fa51425b740 1 -- 172.21.15.5:0/490895460 >> 172.21.15.5:0/490895460 conn(0x55ec09f89940 msgr2=0x55ec09f89d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:11.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:11.886+0000 7fa51425b740 1 -- 172.21.15.5:0/490895460 shutdown_connections 2024-09-10T10:03:11.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:11.886+0000 7fa50aa00640 1 --2- 172.21.15.5:0/1587156680 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fa4f00a2d10 0x7fa4f00a51d0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fa504005e30 tx=0x7fa504009840 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:11.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:11.886+0000 7fa51425b740 1 -- 172.21.15.5:0/490895460 wait complete. 2024-09-10T10:03:11.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:11.887+0000 7fa51425b740 1 -- start start 2024-09-10T10:03:11.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:11.887+0000 7fa51425b740 1 Processor -- start 2024-09-10T10:03:11.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:11.888+0000 7fa51425b740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ec09f8d790 0x55ec09f910e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:11.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:11.888+0000 7fa51425b740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ec09f8e930 0x55ec09f91620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:11.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:11.888+0000 7fa50aa00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ec09f8d790 0x55ec09f910e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:11.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:11.888+0000 7fa50aa00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ec09f8d790 0x55ec09f910e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:38514/0 (socket says 172.21.15.5:38514) 2024-09-10T10:03:11.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:11.888+0000 7fa50aa00640 1 -- 172.21.15.5:0/895352819 learned_addr learned my addr 172.21.15.5:0/895352819 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T10:03:11.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:11.888+0000 7fa50a000640 1 --2- 172.21.15.5:0/895352819 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ec09f8e930 0x55ec09f91620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:11.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:11.889+0000 7fa51425b740 1 --2- 172.21.15.5:0/895352819 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ec09fa0c30 0x55ec09f91b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:11.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:11.889+0000 7fa50b400640 1 --2- 172.21.15.5:0/895352819 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ec09fa0c30 0x55ec09f91b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:11.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:11.890+0000 7fa51425b740 1 -- 172.21.15.5:0/895352819 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55ec09fa3960 con 0x55ec09fa0c30 2024-09-10T10:03:11.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:11.890+0000 7fa51425b740 1 -- 172.21.15.5:0/895352819 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55ec09eade60 con 0x55ec09f8e930 2024-09-10T10:03:11.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:11.891+0000 7fa51425b740 1 -- 172.21.15.5:0/895352819 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55ec09fa3ae0 con 0x55ec09f8d790 2024-09-10T10:03:11.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:11.892+0000 7fa50aa00640 1 -- 172.21.15.5:0/895352819 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ec09f8e930 msgr2=0x55ec09f91620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:03:11.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:11.892+0000 7fa50aa00640 1 --2- 172.21.15.5:0/895352819 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ec09f8e930 0x55ec09f91620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:11.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:11.892+0000 7fa50aa00640 1 -- 172.21.15.5:0/895352819 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ec09fa0c30 msgr2=0x55ec09f91b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:03:11.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:11.892+0000 7fa50aa00640 1 --2- 172.21.15.5:0/895352819 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ec09fa0c30 0x55ec09f91b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:11.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:11.892+0000 7fa50aa00640 1 -- 172.21.15.5:0/895352819 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ec09eb9990 con 0x55ec09f8d790 2024-09-10T10:03:11.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:11.892+0000 7fa50a000640 1 --2- 172.21.15.5:0/895352819 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ec09f8e930 0x55ec09f91620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:03:11.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:11.892+0000 7fa51425b740 10 client.0 _ll_register_callbacks cb 0x55ec09eb2cc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:03:11.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:11.892+0000 7fa51425b740 1 -- 172.21.15.5:0/895352819 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa4fc005500 con 0x55ec09f8d790 2024-09-10T10:03:11.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:11.892+0000 7fa50aa00640 1 --2- 172.21.15.5:0/895352819 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ec09f8d790 0x55ec09f910e0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7fa504035000 tx=0x7fa50402a8c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e6e26d6c1b75a42f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:11.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:11.893+0000 7fa4f4c00640 10 client.30046 ms_handle_connect on v2:172.21.15.159:3300/0 2024-09-10T10:03:11.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:11.894+0000 7fa4f4c00640 1 -- 172.21.15.5:0/895352819 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa504030780 con 0x55ec09f8d790 2024-09-10T10:03:11.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:11.894+0000 7fa4f4c00640 1 -- 172.21.15.5:0/895352819 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fa504031660 con 0x55ec09f8d790 2024-09-10T10:03:11.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:11.894+0000 7fa4f4c00640 1 -- 172.21.15.5:0/895352819 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7fa504031860 con 0x55ec09f8d790 2024-09-10T10:03:11.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:11.894+0000 7fa51425b740 10 client.30046 fetch_fsmap learned FSMap version 42 2024-09-10T10:03:11.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:11.894+0000 7fa51425b740 1 -- 172.21.15.5:0/895352819 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({fsmap=42}) -- 0x55ec09faf4e0 con 0x55ec09f8d790 2024-09-10T10:03:11.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:11.895+0000 7fa4f4c00640 1 -- 172.21.15.5:0/895352819 <== mon.1 v2:172.21.15.159:3300/0 4 ==== fsmap(e 42) ==== 8370+0+0 (secure 0 0 0) 0x7fa504050070 con 0x55ec09f8d790 2024-09-10T10:03:11.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:11.895+0000 7fa51425b740 10 client.30046 fetch_fsmap finished waiting for FSMap version 42 2024-09-10T10:03:11.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:11.895+0000 7fa51425b740 10 client.30046 resolve_mds: validated gid 24542 aka daemon mds.b 2024-09-10T10:03:11.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:11.895+0000 7fa51425b740 20 client.30046 populate_metadata read hostname 'smithi005' 2024-09-10T10:03:11.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:11.895+0000 7fa51425b740 1 --2- 172.21.15.5:0/895352819 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55ec09fb0830 0x55ec09fb2ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:11.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:11.896+0000 7fa51425b740 4 client.30046 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:03:11.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:11.896+0000 7fa51425b740 1 -- 172.21.15.5:0/895352819 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ec09e96310 con 0x55ec09fb0830 2024-09-10T10:03:11.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:11.896+0000 7fa50a000640 1 --2- 172.21.15.5:0/895352819 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55ec09fb0830 0x55ec09fb2ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:11.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:11.897+0000 7fa50a000640 1 --2- 172.21.15.5:0/895352819 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55ec09fb0830 0x55ec09fb2ca0 crc :-1 s=READY pgs=333 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5ea094ec06a17987 server_cookie=6907f3156db1f8c in_seq=0 out_seq=0 2024-09-10T10:03:11.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:11.897+0000 7fa4f4c00640 10 client.30046 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T10:03:11.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:11.898+0000 7fa4f4c00640 1 -- 172.21.15.5:0/895352819 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ec09e96310 con 0x55ec09fb0830 2024-09-10T10:03:11.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:11.898+0000 7fa4f4c00640 10 client.30046 handle_command_reply: tid=0 2024-09-10T10:03:11.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:11.898+0000 7fa51425b740 1 client.30046 shutdown 2024-09-10T10:03:11.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:11.899+0000 7fa51425b740 1 -- 172.21.15.5:0/895352819 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ec09f8d790 msgr2=0x55ec09f910e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:11.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:11.899+0000 7fa51425b740 1 --2- 172.21.15.5:0/895352819 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ec09f8d790 0x55ec09f910e0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7fa504035000 tx=0x7fa50402a8c0 comp rx=0 tx=0).stop 2024-09-10T10:03:11.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:11.899+0000 7fa51425b740 1 -- 172.21.15.5:0/895352819 shutdown_connections 2024-09-10T10:03:11.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:11.899+0000 7fa51425b740 1 --2- 172.21.15.5:0/895352819 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55ec09fb0830 0x55ec09fb2ca0 crc :-1 s=READY pgs=333 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:11.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:11.899+0000 7fa51425b740 1 --2- 172.21.15.5:0/895352819 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ec09fa0c30 0x55ec09f91b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:11.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:11.899+0000 7fa51425b740 1 --2- 172.21.15.5:0/895352819 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ec09f8e930 0x55ec09f91620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:11.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:11.899+0000 7fa51425b740 1 --2- 172.21.15.5:0/895352819 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ec09f8d790 0x55ec09f910e0 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:11.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:11.899+0000 7fa4f4c00640 0 client.30046 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T10:03:11.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:11.899+0000 7fa51425b740 1 -- 172.21.15.5:0/895352819 >> 172.21.15.5:0/895352819 conn(0x55ec09f89940 msgr2=0x55ec09f88c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:11.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:11.900+0000 7fa51425b740 1 -- 172.21.15.5:0/895352819 shutdown_connections 2024-09-10T10:03:11.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:11.900+0000 7fa51425b740 1 -- 172.21.15.5:0/895352819 wait complete. 2024-09-10T10:03:11.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:11.901+0000 7fa51425b740 20 client.30046 trim_cache size 0 max 16384 2024-09-10T10:03:11.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:11.935+0000 7fa51425b740 1 Processor -- start 2024-09-10T10:03:11.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:11.936+0000 7fa51425b740 1 -- start start 2024-09-10T10:03:11.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:11.936+0000 7fa51425b740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ec09f8d790 0x55ec09f88a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:11.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:11.936+0000 7fa51425b740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ec09f8e930 0x55ec09f91650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:11.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:11.936+0000 7fa51425b740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ec09fa0c30 0x55ec09f91b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:11.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:11.937+0000 7fa50a000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ec09f8e930 0x55ec09f91650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:11.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:11.937+0000 7fa50a000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ec09f8e930 0x55ec09f91650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:38522/0 (socket says 172.21.15.5:38522) 2024-09-10T10:03:11.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:11.937+0000 7fa50b400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ec09fa0c30 0x55ec09f91b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:11.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:11.937+0000 7fa50b400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ec09fa0c30 0x55ec09f91b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:43990/0 (socket says 172.21.15.5:43990) 2024-09-10T10:03:11.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:11.938+0000 7fa50aa00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ec09f8d790 0x55ec09f88a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:11.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:11.938+0000 7fa51425b740 1 -- 172.21.15.5:0/1123393690 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa50c075f10 con 0x55ec09f8d790 2024-09-10T10:03:11.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:11.938+0000 7fa51425b740 1 -- 172.21.15.5:0/1123393690 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fa50c1b75d0 con 0x55ec09fa0c30 2024-09-10T10:03:11.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:11.938+0000 7fa51425b740 1 -- 172.21.15.5:0/1123393690 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fa50c1b6200 con 0x55ec09f8e930 2024-09-10T10:03:11.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:11.938+0000 7fa50a000640 1 -- 172.21.15.5:0/1123393690 learned_addr learned my addr 172.21.15.5:0/1123393690 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T10:03:11.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:11.939+0000 7fa50a000640 1 -- 172.21.15.5:0/1123393690 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ec09fa0c30 msgr2=0x55ec09f91b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:03:11.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:11.939+0000 7fa50a000640 1 --2- 172.21.15.5:0/1123393690 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ec09fa0c30 0x55ec09f91b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:11.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:11.939+0000 7fa50a000640 1 -- 172.21.15.5:0/1123393690 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ec09f8d790 msgr2=0x55ec09f88a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:03:11.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:11.939+0000 7fa50a000640 1 --2- 172.21.15.5:0/1123393690 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ec09f8d790 0x55ec09f88a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:11.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:11.939+0000 7fa50a000640 1 -- 172.21.15.5:0/1123393690 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ec09fa3390 con 0x55ec09f8e930 2024-09-10T10:03:11.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:11.940+0000 7fa50a000640 1 --2- 172.21.15.5:0/1123393690 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ec09f8e930 0x55ec09f91650 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7fa4f800be60 tx=0x7fa4f80cbc50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9496a5280a119985 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:11.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:11.941+0000 7fa4f7400640 1 -- 172.21.15.5:0/1123393690 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4f80b2070 con 0x55ec09f8e930 2024-09-10T10:03:11.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:11.941+0000 7fa4f7400640 1 -- 172.21.15.5:0/1123393690 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa4f8010810 con 0x55ec09f8e930 2024-09-10T10:03:11.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:11.941+0000 7fa4f7400640 1 -- 172.21.15.5:0/1123393690 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4f80b2070 con 0x55ec09f8e930 2024-09-10T10:03:11.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:11.943+0000 7fa51425b740 1 -- 172.21.15.5:0/1123393690 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ec09f8e930 msgr2=0x55ec09f91650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:11.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:11.943+0000 7fa51425b740 1 --2- 172.21.15.5:0/1123393690 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ec09f8e930 0x55ec09f91650 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7fa4f800be60 tx=0x7fa4f80cbc50 comp rx=0 tx=0).stop 2024-09-10T10:03:11.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:11.943+0000 7fa51425b740 1 -- 172.21.15.5:0/1123393690 shutdown_connections 2024-09-10T10:03:11.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:11.943+0000 7fa51425b740 1 --2- 172.21.15.5:0/1123393690 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ec09fa0c30 0x55ec09f91b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:11.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:11.943+0000 7fa51425b740 1 --2- 172.21.15.5:0/1123393690 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ec09f8e930 0x55ec09f91650 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:11.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:11.943+0000 7fa51425b740 1 --2- 172.21.15.5:0/1123393690 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ec09f8d790 0x55ec09f88a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:11.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:11.943+0000 7fa51425b740 1 -- 172.21.15.5:0/1123393690 >> 172.21.15.5:0/1123393690 conn(0x55ec09f89940 msgr2=0x55ec09f8f660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:11.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:11.944+0000 7fa51425b740 1 -- 172.21.15.5:0/1123393690 shutdown_connections 2024-09-10T10:03:11.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:11.944+0000 7fa51425b740 1 -- 172.21.15.5:0/1123393690 wait complete. 2024-09-10T10:03:11.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:11.945+0000 7fa51425b740 1 -- start start 2024-09-10T10:03:11.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:11.945+0000 7fa51425b740 1 Processor -- start 2024-09-10T10:03:11.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:11.947+0000 7fa51425b740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ec09f8d790 0x55ec09fa9190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:11.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:11.947+0000 7fa51425b740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ec09f8e930 0x55ec09fa96d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:11.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:11.947+0000 7fa51425b740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ec09fa0c30 0x55ec09fa5720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:11.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:11.947+0000 7fa51425b740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa50c1b6f50 con 0x55ec09f8d790 2024-09-10T10:03:11.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:11.947+0000 7fa51425b740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fa50c1b6880 con 0x55ec09fa0c30 2024-09-10T10:03:11.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:11.947+0000 7fa51425b740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fa50c073ef0 con 0x55ec09f8e930 2024-09-10T10:03:11.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:11.947+0000 7fa51425b740 10 client.0 _ll_register_callbacks cb 0x7fa504009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:03:11.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:11.948+0000 7fa50a000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ec09f8e930 0x55ec09fa96d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:11.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:11.948+0000 7fa50a000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ec09f8e930 0x55ec09fa96d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:38538/0 (socket says 172.21.15.5:38538) 2024-09-10T10:03:11.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:11.948+0000 7fa50a000640 1 -- 172.21.15.5:0/3233772718 learned_addr learned my addr 172.21.15.5:0/3233772718 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T10:03:11.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:11.948+0000 7fa50aa00640 1 --2- 172.21.15.5:0/3233772718 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ec09f8d790 0x55ec09fa9190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:11.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:11.948+0000 7fa50b400640 1 --2- 172.21.15.5:0/3233772718 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ec09fa0c30 0x55ec09fa5720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:11.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:11.949+0000 7fa50a000640 1 -- 172.21.15.5:0/3233772718 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ec09fa0c30 msgr2=0x55ec09fa5720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:03:11.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:11.949+0000 7fa50a000640 1 --2- 172.21.15.5:0/3233772718 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ec09fa0c30 0x55ec09fa5720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:11.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:11.949+0000 7fa50a000640 1 -- 172.21.15.5:0/3233772718 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ec09f8d790 msgr2=0x55ec09fa9190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:03:11.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:11.949+0000 7fa50a000640 1 --2- 172.21.15.5:0/3233772718 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ec09f8d790 0x55ec09fa9190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:11.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:11.949+0000 7fa50a000640 1 -- 172.21.15.5:0/3233772718 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ec09fa5f50 con 0x55ec09f8e930 2024-09-10T10:03:11.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:11.950+0000 7fa50b400640 1 --2- 172.21.15.5:0/3233772718 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ec09fa0c30 0x55ec09fa5720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:03:11.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:11.951+0000 7fa50a000640 1 --2- 172.21.15.5:0/3233772718 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ec09f8e930 0x55ec09fa96d0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7fa4f8017fd0 tx=0x7fa4f80cc0b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9fe5fe21698909be server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:11.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:11.951+0000 7fa4f4c00640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-09-10T10:03:11.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:11.952+0000 7fa4f4c00640 1 -- 172.21.15.5:0/3233772718 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4f80b2050 con 0x55ec09f8e930 2024-09-10T10:03:11.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:11.952+0000 7fa4f4c00640 1 -- 172.21.15.5:0/3233772718 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fa4f80a0020 con 0x55ec09f8e930 2024-09-10T10:03:11.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:11.952+0000 7fa51425b740 1 -- 172.21.15.5:0/3233772718 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa4f8010810 con 0x55ec09f8e930 2024-09-10T10:03:11.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:11.952+0000 7fa4f4c00640 1 -- 172.21.15.5:0/3233772718 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7fa4f801e040 con 0x55ec09f8e930 2024-09-10T10:03:11.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:11.952+0000 7fa51425b740 10 client.30052 fetch_fsmap learned FSMap version 42 2024-09-10T10:03:11.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:11.953+0000 7fa51425b740 1 -- 172.21.15.5:0/3233772718 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55ec0a01c220 con 0x55ec09f8e930 2024-09-10T10:03:11.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:11.956+0000 7fa4f4c00640 1 -- 172.21.15.5:0/3233772718 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa4f80a8070 con 0x55ec09f8e930 2024-09-10T10:03:11.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:11.956+0000 7fa4f4c00640 1 -- 172.21.15.5:0/3233772718 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 42) ==== 8370+0+0 (secure 0 0 0) 0x7fa4f80b3ce0 con 0x55ec09f8e930 2024-09-10T10:03:11.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:11.956+0000 7fa51425b740 10 client.30052 fetch_fsmap finished waiting for FSMap version 42 2024-09-10T10:03:11.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:11.956+0000 7fa51425b740 10 client.30052 resolve_mds: validated gid 24542 aka daemon mds.b 2024-09-10T10:03:11.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:11.956+0000 7fa51425b740 20 client.30052 populate_metadata read hostname 'smithi005' 2024-09-10T10:03:11.958 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:11.956+0000 7fa51425b740 1 --2- 172.21.15.5:0/3233772718 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55ec0a01c8f0 0x55ec0a01ece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:11.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:11.956+0000 7fa51425b740 4 client.30052 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 150.0, "await_for": 45.0}] 2024-09-10T10:03:11.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:11.956+0000 7fa51425b740 1 -- 172.21.15.5:0/3233772718 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 150.0, "await_for": 45.0}) -- 0x7fa4f8010810 con 0x55ec0a01c8f0 2024-09-10T10:03:11.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:11.958+0000 7fa50aa00640 1 --2- 172.21.15.5:0/3233772718 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55ec0a01c8f0 0x55ec0a01ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:11.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:11.958+0000 7fa50aa00640 1 --2- 172.21.15.5:0/3233772718 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55ec0a01c8f0 0x55ec0a01ece0 crc :-1 s=READY pgs=334 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=920ea8c7389d4188 server_cookie=67495dca66356e3a in_seq=0 out_seq=0 2024-09-10T10:03:11.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:11.958+0000 7fa4f4c00640 10 client.30052 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T10:03:12.076 INFO:tasks.workunit.client.0.smithi005.stdout: 10 177023 14.03 MB/sec execute 530 sec latency 3240.608 ms 2024-09-10T10:03:12.544 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-10T10:03:12.544 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T10:03:12.934 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:03:13.076 INFO:tasks.workunit.client.0.smithi005.stdout: 10 177037 14.01 MB/sec execute 531 sec latency 3877.288 ms 2024-09-10T10:03:13.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:03:13 smithi159 ceph-mon[35513]: pgmap v1139: 129 pgs: 129 active+clean; 781 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 5.1 MiB/s wr, 401 op/s 2024-09-10T10:03:13.604 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:03:13 smithi005 ceph-mon[30350]: pgmap v1139: 129 pgs: 129 active+clean; 781 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 5.1 MiB/s wr, 401 op/s 2024-09-10T10:03:13.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:03:13 smithi102 ceph-mon[35442]: pgmap v1139: 129 pgs: 129 active+clean; 781 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 5.1 MiB/s wr, 401 op/s 2024-09-10T10:03:14.076 INFO:tasks.workunit.client.0.smithi005.stdout: 10 177038 13.98 MB/sec execute 532 sec latency 1102.078 ms 2024-09-10T10:03:14.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:13.773+0000 7f73d0200640 1 -- 172.21.15.5:0/945335025 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f73c8106f10 msgr2=0x7f73c810ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:14.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:13.773+0000 7f73d0200640 1 --2- 172.21.15.5:0/945335025 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f73c8106f10 0x7f73c810ea90 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f73c0009ca0 tx=0x7f73c002a8d0 comp rx=0 tx=0).stop 2024-09-10T10:03:14.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:13.774+0000 7f73d0200640 1 -- 172.21.15.5:0/945335025 shutdown_connections 2024-09-10T10:03:14.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:13.774+0000 7f73d0200640 1 --2- 172.21.15.5:0/945335025 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f73c8106f10 0x7f73c810ea90 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:14.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:13.774+0000 7f73d0200640 1 --2- 172.21.15.5:0/945335025 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f73c8102790 0x7f73c81067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:14.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:13.774+0000 7f73d0200640 1 --2- 172.21.15.5:0/945335025 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f73c8101de0 0x7f73c81021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:14.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:13.774+0000 7f73d0200640 1 -- 172.21.15.5:0/945335025 >> 172.21.15.5:0/945335025 conn(0x7f73c80fd8c0 msgr2=0x7f73c80ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:14.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:13.775+0000 7f73d0200640 1 -- 172.21.15.5:0/945335025 shutdown_connections 2024-09-10T10:03:14.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:13.775+0000 7f73d0200640 1 -- 172.21.15.5:0/945335025 wait complete. 2024-09-10T10:03:14.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:13.778+0000 7f73d0200640 1 Processor -- start 2024-09-10T10:03:14.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:13.779+0000 7f73d0200640 1 -- start start 2024-09-10T10:03:14.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:13.780+0000 7f73d0200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f73c8101de0 0x7f73c81a47d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:14.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:13.780+0000 7f73d0200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f73c8102790 0x7f73c81a4d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:14.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:13.780+0000 7f73d0200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f73c8106f10 0x7f73c810dc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:14.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:13.780+0000 7f73cd800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f73c8101de0 0x7f73c81a47d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:14.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:13.780+0000 7f73cd800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f73c8101de0 0x7f73c81a47d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:44020/0 (socket says 172.21.15.5:44020) 2024-09-10T10:03:14.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:13.780+0000 7f73cd800640 1 -- 172.21.15.5:0/1318002755 learned_addr learned my addr 172.21.15.5:0/1318002755 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T10:03:14.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:13.781+0000 7f73ce200640 1 --2- 172.21.15.5:0/1318002755 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f73c8106f10 0x7f73c810dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:14.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:13.781+0000 7f73cce00640 1 --2- 172.21.15.5:0/1318002755 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f73c8102790 0x7f73c81a4d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:14.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:13.780+0000 7f73d0200640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f73c8111f90 con 0x7f73c8102790 2024-09-10T10:03:14.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:13.781+0000 7f73d0200640 1 -- 172.21.15.5:0/1318002755 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f73c8111e10 con 0x7f73c8101de0 2024-09-10T10:03:14.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:13.781+0000 7f73d0200640 1 -- 172.21.15.5:0/1318002755 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f73c8112110 con 0x7f73c8106f10 2024-09-10T10:03:14.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:13.781+0000 7f73cd800640 1 -- 172.21.15.5:0/1318002755 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f73c8106f10 msgr2=0x7f73c810dc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:14.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:13.781+0000 7f73cd800640 1 --2- 172.21.15.5:0/1318002755 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f73c8106f10 0x7f73c810dc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:14.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:13.781+0000 7f73cd800640 1 -- 172.21.15.5:0/1318002755 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f73c8102790 msgr2=0x7f73c81a4d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:14.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:13.781+0000 7f73cd800640 1 --2- 172.21.15.5:0/1318002755 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f73c8102790 0x7f73c81a4d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:14.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:13.782+0000 7f73cd800640 1 -- 172.21.15.5:0/1318002755 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73c810e510 con 0x7f73c8101de0 2024-09-10T10:03:14.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:13.782+0000 7f73ce200640 1 --2- 172.21.15.5:0/1318002755 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f73c8106f10 0x7f73c810dc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:03:14.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:13.782+0000 7f73cd800640 1 --2- 172.21.15.5:0/1318002755 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f73c8101de0 0x7f73c81a47d0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f73bc00e9f0 tx=0x7f73bc00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:14.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:13.782+0000 7f73c6000640 1 -- 172.21.15.5:0/1318002755 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73bc00cd80 con 0x7f73c8101de0 2024-09-10T10:03:14.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:13.783+0000 7f73c6000640 1 -- 172.21.15.5:0/1318002755 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f73bc010910 con 0x7f73c8101de0 2024-09-10T10:03:14.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:13.783+0000 7f73c6000640 1 -- 172.21.15.5:0/1318002755 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73bc00cd80 con 0x7f73c8101de0 2024-09-10T10:03:14.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:13.783+0000 7f73d0200640 1 -- 172.21.15.5:0/1318002755 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73c810e800 con 0x7f73c8101de0 2024-09-10T10:03:14.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:13.786+0000 7f73d0200640 1 -- 172.21.15.5:0/1318002755 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73c81ab5a0 con 0x7f73c8101de0 2024-09-10T10:03:14.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:13.789+0000 7f73d0200640 1 -- 172.21.15.5:0/1318002755 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f73c81032c0 con 0x7f73c8101de0 2024-09-10T10:03:14.269 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-10T10:03:14.269 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":42,"btime":"2024-09-10T09:55:06:670916+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14454,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/2970594311","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2970594311},{"type":"v1","addr":"172.21.15.5:6835","nonce":2970594311}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14514,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6839/3616502174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":3616502174},{"type":"v1","addr":"172.21.15.5:6839","nonce":3616502174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":42,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T09:35:29.626287+0000","modified":"2024-09-10T09:55:06.419829+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24542,"mds_1":24535,"mds_2":24511,"mds_3":24506,"mds_4":24443},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14520":{"gid":14520,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/2073599842","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":2073599842},{"type":"v1","addr":"172.21.15.5:6841","nonce":2073599842}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"k","rank":4,"incarnation":26,"state":"up:active","state_seq":24,"addr":"172.21.15.102:6833/2457007427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":2457007427},{"type":"v1","addr":"172.21.15.102:6833","nonce":2457007427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/356445159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":356445159},{"type":"v1","addr":"172.21.15.159:6833","nonce":356445159}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6837/2894314507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":2894314507},{"type":"v1","addr":"172.21.15.5:6837","nonce":2894314507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.102:6835/1647549111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":1647549111},{"type":"v1","addr":"172.21.15.102:6835","nonce":1647549111}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6835/952657242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":952657242},{"type":"v1","addr":"172.21.15.159:6835","nonce":952657242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":19,"addr":"172.21.15.102:6837/2689776764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":2689776764},{"type":"v1","addr":"172.21.15.102:6837","nonce":2689776764}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.159:6837/620627639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":620627639},{"type":"v1","addr":"172.21.15.159:6837","nonce":620627639}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24535":{"gid":24535,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":18,"addr":"172.21.15.159:6839/2073739599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":2073739599},{"type":"v1","addr":"172.21.15.159:6839","nonce":2073739599}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24542":{"gid":24542,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.102:6839/335409749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6838","nonce":335409749},{"type":"v1","addr":"172.21.15.102:6839","nonce":335409749}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24542,"qdb_cluster":[24542,24535,24511,24506,24443]},"id":1}]} 2024-09-10T10:03:14.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:13.791+0000 7f73c6000640 1 -- 172.21.15.5:0/1318002755 <== mon.2 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f73bc010ac0 con 0x7f73c8101de0 2024-09-10T10:03:14.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:13.793+0000 7f73c6000640 1 --2- 172.21.15.5:0/1318002755 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f73b00a2e50 0x7f73b00a5310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:14.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:13.793+0000 7f73c6000640 1 -- 172.21.15.5:0/1318002755 <== mon.2 v2:172.21.15.102:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f73bc0cf490 con 0x7f73c8101de0 2024-09-10T10:03:14.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:13.794+0000 7f73cce00640 1 --2- 172.21.15.5:0/1318002755 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f73b00a2e50 0x7f73b00a5310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:14.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:13.795+0000 7f73cce00640 1 --2- 172.21.15.5:0/1318002755 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f73b00a2e50 0x7f73b00a5310 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f73c81a0050 tx=0x7f73b4008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:14.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:13.798+0000 7f73c6000640 1 -- 172.21.15.5:0/1318002755 <== mon.2 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f73bc09c000 con 0x7f73c8101de0 2024-09-10T10:03:14.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:14.122+0000 7f73d0200640 1 -- 172.21.15.5:0/1318002755 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f73c81ab8b0 con 0x7f73c8101de0 2024-09-10T10:03:14.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:14.123+0000 7f73c6000640 1 -- 172.21.15.5:0/1318002755 <== mon.2 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 42 v42) ==== 94+0+11969 (secure 0 0 0) 0x7f73bc0afe20 con 0x7f73c8101de0 2024-09-10T10:03:14.271 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 42 2024-09-10T10:03:14.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:14.123+0000 7f73d0200640 1 -- 172.21.15.5:0/1318002755 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f73b00a2e50 msgr2=0x7f73b00a5310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:14.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:14.123+0000 7f73d0200640 1 --2- 172.21.15.5:0/1318002755 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f73b00a2e50 0x7f73b00a5310 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f73c81a0050 tx=0x7f73b4008040 comp rx=0 tx=0).stop 2024-09-10T10:03:14.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:14.123+0000 7f73d0200640 1 -- 172.21.15.5:0/1318002755 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f73c8101de0 msgr2=0x7f73c81a47d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:14.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:14.123+0000 7f73d0200640 1 --2- 172.21.15.5:0/1318002755 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f73c8101de0 0x7f73c81a47d0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f73bc00e9f0 tx=0x7f73bc00eeb0 comp rx=0 tx=0).stop 2024-09-10T10:03:14.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:14.123+0000 7f73d0200640 1 -- 172.21.15.5:0/1318002755 shutdown_connections 2024-09-10T10:03:14.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:14.123+0000 7f73d0200640 1 --2- 172.21.15.5:0/1318002755 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f73b00a2e50 0x7f73b00a5310 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:14.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:14.123+0000 7f73d0200640 1 --2- 172.21.15.5:0/1318002755 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f73c8106f10 0x7f73c810dc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:14.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:14.123+0000 7f73d0200640 1 --2- 172.21.15.5:0/1318002755 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f73c8102790 0x7f73c81a4d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:14.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:14.123+0000 7f73d0200640 1 --2- 172.21.15.5:0/1318002755 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f73c8101de0 0x7f73c81a47d0 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:14.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:14.123+0000 7f73d0200640 1 -- 172.21.15.5:0/1318002755 >> 172.21.15.5:0/1318002755 conn(0x7f73c80fd8c0 msgr2=0x7f73c80ffcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:14.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:14.123+0000 7f73d0200640 1 -- 172.21.15.5:0/1318002755 shutdown_connections 2024-09-10T10:03:14.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:14.123+0000 7f73d0200640 1 -- 172.21.15.5:0/1318002755 wait complete. 2024-09-10T10:03:14.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:14.384+0000 7fa4f4c00640 1 -- 172.21.15.5:0/3233772718 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x7fa4f8010810 con 0x55ec0a01c8f0 2024-09-10T10:03:14.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:14.384+0000 7fa4f4c00640 10 client.30052 handle_command_reply: tid=0 2024-09-10T10:03:14.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:14.384+0000 7fa51425b740 1 client.30052 shutdown 2024-09-10T10:03:14.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:14.384+0000 7fa51425b740 1 -- 172.21.15.5:0/3233772718 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ec09f8e930 msgr2=0x55ec09fa96d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:14.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:14.384+0000 7fa51425b740 1 --2- 172.21.15.5:0/3233772718 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ec09f8e930 0x55ec09fa96d0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7fa4f8017fd0 tx=0x7fa4f80cc0b0 comp rx=0 tx=0).stop 2024-09-10T10:03:14.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:14.384+0000 7fa51425b740 1 -- 172.21.15.5:0/3233772718 shutdown_connections 2024-09-10T10:03:14.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:14.384+0000 7fa51425b740 1 --2- 172.21.15.5:0/3233772718 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55ec0a01c8f0 0x55ec0a01ece0 crc :-1 s=READY pgs=334 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:14.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:14.384+0000 7fa51425b740 1 --2- 172.21.15.5:0/3233772718 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ec09fa0c30 0x55ec09fa5720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:14.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:14.384+0000 7fa51425b740 1 --2- 172.21.15.5:0/3233772718 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ec09f8e930 0x55ec09fa96d0 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:14.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:14.384+0000 7fa51425b740 1 --2- 172.21.15.5:0/3233772718 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ec09f8d790 0x55ec09fa9190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:14.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:14.384+0000 7fa51425b740 1 -- 172.21.15.5:0/3233772718 >> 172.21.15.5:0/3233772718 conn(0x55ec09f89940 msgr2=0x55ec09fabc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:14.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:14.384+0000 7fa4f4c00640 0 client.30052 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T10:03:14.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:14.384+0000 7fa51425b740 1 -- 172.21.15.5:0/3233772718 shutdown_connections 2024-09-10T10:03:14.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:14.384+0000 7fa51425b740 1 -- 172.21.15.5:0/3233772718 wait complete. 2024-09-10T10:03:14.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:14.384+0000 7fa51425b740 20 client.30052 trim_cache size 0 max 16384 2024-09-10T10:03:14.387 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-10T10:03:14.388 INFO:teuthology.orchestra.run.smithi005.stdout: "epoch": 42, 2024-09-10T10:03:14.388 INFO:teuthology.orchestra.run.smithi005.stdout: "leader": 24542, 2024-09-10T10:03:14.388 INFO:teuthology.orchestra.run.smithi005.stdout: "set_version": 86, 2024-09-10T10:03:14.388 INFO:teuthology.orchestra.run.smithi005.stdout: "sets": { 2024-09-10T10:03:14.388 INFO:teuthology.orchestra.run.smithi005.stdout: "dfb92e25": { 2024-09-10T10:03:14.388 INFO:teuthology.orchestra.run.smithi005.stdout: "version": 86, 2024-09-10T10:03:14.388 INFO:teuthology.orchestra.run.smithi005.stdout: "age_ref": 0.0, 2024-09-10T10:03:14.388 INFO:teuthology.orchestra.run.smithi005.stdout: "state": { 2024-09-10T10:03:14.388 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "QUIESCED", 2024-09-10T10:03:14.388 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0 2024-09-10T10:03:14.388 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:03:14.388 INFO:teuthology.orchestra.run.smithi005.stdout: "timeout": 90.0, 2024-09-10T10:03:14.389 INFO:teuthology.orchestra.run.smithi005.stdout: "expiration": 150.0, 2024-09-10T10:03:14.389 INFO:teuthology.orchestra.run.smithi005.stdout: "members": { 2024-09-10T10:03:14.389 INFO:teuthology.orchestra.run.smithi005.stdout: "file:/": { 2024-09-10T10:03:14.389 INFO:teuthology.orchestra.run.smithi005.stdout: "excluded": false, 2024-09-10T10:03:14.389 INFO:teuthology.orchestra.run.smithi005.stdout: "state": { 2024-09-10T10:03:14.389 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "QUIESCED", 2024-09-10T10:03:14.389 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0 2024-09-10T10:03:14.389 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T10:03:14.389 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T10:03:14.389 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T10:03:14.389 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T10:03:14.389 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T10:03:14.389 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-10T10:03:14.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:14.384+0000 7fa511800640 1 -- 172.21.15.5:0/1587156680 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fa4f00a2d10 msgr2=0x7fa4f00a51d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:14.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:14.384+0000 7fa511800640 1 --2- 172.21.15.5:0/1587156680 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fa4f00a2d10 0x7fa4f00a51d0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fa504005e30 tx=0x7fa504009840 comp rx=0 tx=0).stop 2024-09-10T10:03:14.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:14.384+0000 7fa511800640 1 -- 172.21.15.5:0/1587156680 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa50c106f20 msgr2=0x7fa50c198bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:14.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:14.384+0000 7fa511800640 1 --2- 172.21.15.5:0/1587156680 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa50c106f20 0x7fa50c198bb0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7fa4f800b4a0 tx=0x7fa4f800b960 comp rx=0 tx=0).stop 2024-09-10T10:03:14.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:14.384+0000 7fa511800640 1 -- 172.21.15.5:0/1587156680 shutdown_connections 2024-09-10T10:03:14.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:14.384+0000 7fa511800640 1 --2- 172.21.15.5:0/1587156680 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fa4f00a2d10 0x7fa4f00a51d0 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:14.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:14.384+0000 7fa511800640 1 --2- 172.21.15.5:0/1587156680 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa50c19cf40 0x7fa50c19d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:14.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:14.384+0000 7fa511800640 1 --2- 172.21.15.5:0/1587156680 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa50c106f20 0x7fa50c198bb0 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:14.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:14.384+0000 7fa511800640 1 --2- 172.21.15.5:0/1587156680 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa50c1027a0 0x7fa50c198670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:14.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:14.384+0000 7fa511800640 1 -- 172.21.15.5:0/1587156680 >> 172.21.15.5:0/1587156680 conn(0x7fa50c0fd8b0 msgr2=0x7fa50c0feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:14.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:14.384+0000 7fa511800640 1 -- 172.21.15.5:0/1587156680 shutdown_connections 2024-09-10T10:03:14.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:14.384+0000 7fa511800640 1 -- 172.21.15.5:0/1587156680 wait complete. 2024-09-10T10:03:14.418 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:03:14 smithi005 ceph-mon[30350]: from='client.? 172.21.15.5:0/1318002755' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:03:14.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:03:14 smithi102 ceph-mon[35442]: from='client.? 172.21.15.5:0/1318002755' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:03:14.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:03:14 smithi159 ceph-mon[35513]: from='client.? 172.21.15.5:0/1318002755' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:03:15.076 INFO:tasks.workunit.client.0.smithi005.stdout: 10 177038 13.95 MB/sec execute 533 sec latency 2102.145 ms 2024-09-10T10:03:16.076 INFO:tasks.workunit.client.0.smithi005.stdout: 10 177038 13.93 MB/sec execute 534 sec latency 3102.200 ms 2024-09-10T10:03:16.292 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:03:15 smithi005 ceph-mon[30350]: pgmap v1140: 129 pgs: 129 active+clean; 779 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 5.7 MiB/s wr, 422 op/s 2024-09-10T10:03:16.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:03:15 smithi159 ceph-mon[35513]: pgmap v1140: 129 pgs: 129 active+clean; 779 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 5.7 MiB/s wr, 422 op/s 2024-09-10T10:03:16.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:03:15 smithi102 ceph-mon[35442]: pgmap v1140: 129 pgs: 129 active+clean; 779 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 5.7 MiB/s wr, 422 op/s 2024-09-10T10:03:17.076 INFO:tasks.workunit.client.0.smithi005.stdout: 10 177038 13.90 MB/sec execute 535 sec latency 4102.252 ms 2024-09-10T10:03:18.076 INFO:tasks.workunit.client.0.smithi005.stdout: 10 177038 13.87 MB/sec execute 536 sec latency 5102.315 ms 2024-09-10T10:03:18.202 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:03:17 smithi005 ceph-mon[30350]: pgmap v1141: 129 pgs: 129 active+clean; 784 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 6.2 MiB/s wr, 441 op/s 2024-09-10T10:03:18.274 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'dfb92e25', quiesce took 14.0 seconds. Will release after: 16.0 2024-09-10T10:03:18.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:03:17 smithi159 ceph-mon[35513]: pgmap v1141: 129 pgs: 129 active+clean; 784 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 6.2 MiB/s wr, 441 op/s 2024-09-10T10:03:18.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:03:17 smithi102 ceph-mon[35442]: pgmap v1141: 129 pgs: 129 active+clean; 784 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 6.2 MiB/s wr, 441 op/s 2024-09-10T10:03:19.086 INFO:tasks.workunit.client.0.smithi005.stdout: 10 177040 13.85 MB/sec execute 537 sec latency 6111.895 ms 2024-09-10T10:03:19.353 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-10T10:03:19.736 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:03:20.086 INFO:tasks.workunit.client.0.smithi005.stdout: 10 177040 13.82 MB/sec execute 538 sec latency 7111.947 ms 2024-09-10T10:03:20.308 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:03:19 smithi005 ceph-mon[30350]: pgmap v1142: 129 pgs: 129 active+clean; 787 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 6.0 MiB/s wr, 345 op/s 2024-09-10T10:03:20.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:03:19 smithi159 ceph-mon[35513]: pgmap v1142: 129 pgs: 129 active+clean; 787 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 6.0 MiB/s wr, 345 op/s 2024-09-10T10:03:20.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:03:19 smithi102 ceph-mon[35442]: pgmap v1142: 129 pgs: 129 active+clean; 787 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 6.0 MiB/s wr, 345 op/s 2024-09-10T10:03:21.086 INFO:tasks.workunit.client.0.smithi005.stdout: 10 177040 13.80 MB/sec execute 539 sec latency 8112.017 ms 2024-09-10T10:03:22.086 INFO:tasks.workunit.client.0.smithi005.stdout: 10 177040 13.77 MB/sec execute 540 sec latency 9112.078 ms 2024-09-10T10:03:22.170 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:03:21 smithi005 ceph-mon[30350]: pgmap v1143: 129 pgs: 129 active+clean; 798 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 7.6 MiB/s wr, 372 op/s 2024-09-10T10:03:22.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:03:21 smithi159 ceph-mon[35513]: pgmap v1143: 129 pgs: 129 active+clean; 798 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 7.6 MiB/s wr, 372 op/s 2024-09-10T10:03:22.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:22.349+0000 7f2d4c400640 1 -- 172.21.15.5:0/576619389 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2d441049a0 msgr2=0x7f2d441089f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:22.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:22.349+0000 7f2d4c400640 1 --2- 172.21.15.5:0/576619389 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2d441049a0 0x7f2d441089f0 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f2d30009900 tx=0x7f2d3002a3f0 comp rx=0 tx=0).stop 2024-09-10T10:03:22.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:22.349+0000 7f2d4c400640 1 -- 172.21.15.5:0/576619389 shutdown_connections 2024-09-10T10:03:22.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:22.349+0000 7f2d4c400640 1 --2- 172.21.15.5:0/576619389 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2d44109120 0x7f2d44110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:22.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:22.349+0000 7f2d4c400640 1 --2- 172.21.15.5:0/576619389 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2d441049a0 0x7f2d441089f0 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:22.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:22.349+0000 7f2d4c400640 1 --2- 172.21.15.5:0/576619389 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2d44103ff0 0x7f2d441043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:22.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:22.349+0000 7f2d4c400640 1 -- 172.21.15.5:0/576619389 >> 172.21.15.5:0/576619389 conn(0x7f2d440ffa90 msgr2=0x7f2d44101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:22.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:22.349+0000 7f2d4c400640 1 -- 172.21.15.5:0/576619389 shutdown_connections 2024-09-10T10:03:22.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:22.349+0000 7f2d4c400640 1 -- 172.21.15.5:0/576619389 wait complete. 2024-09-10T10:03:22.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:22.350+0000 7f2d4c400640 1 Processor -- start 2024-09-10T10:03:22.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:22.350+0000 7f2d4c400640 1 -- start start 2024-09-10T10:03:22.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:22.351+0000 7f2d4c400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2d44103ff0 0x7f2d4419a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:22.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:22.351+0000 7f2d4c400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2d441049a0 0x7f2d4419ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:22.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:22.351+0000 7f2d4c400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2d44109120 0x7f2d4419eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:22.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:22.351+0000 7f2d4c400640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f2d44114140 con 0x7f2d441049a0 2024-09-10T10:03:22.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:22.351+0000 7f2d4c400640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f2d44113fc0 con 0x7f2d44109120 2024-09-10T10:03:22.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:22.351+0000 7f2d4c400640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f2d441142c0 con 0x7f2d44103ff0 2024-09-10T10:03:22.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:22.351+0000 7f2d49000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2d441049a0 0x7f2d4419ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:22.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:22.351+0000 7f2d49000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2d441049a0 0x7f2d4419ab60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49204/0 (socket says 172.21.15.5:49204) 2024-09-10T10:03:22.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:22.351+0000 7f2d49000640 1 -- 172.21.15.5:0/3025375605 learned_addr learned my addr 172.21.15.5:0/3025375605 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T10:03:22.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:22.351+0000 7f2d49a00640 1 --2- 172.21.15.5:0/3025375605 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2d44103ff0 0x7f2d4419a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:22.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:22.351+0000 7f2d4a400640 1 --2- 172.21.15.5:0/3025375605 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2d44109120 0x7f2d4419eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:22.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:22.352+0000 7f2d49a00640 1 -- 172.21.15.5:0/3025375605 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2d44109120 msgr2=0x7f2d4419eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:22.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:22.352+0000 7f2d49a00640 1 --2- 172.21.15.5:0/3025375605 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2d44109120 0x7f2d4419eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:22.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:22.352+0000 7f2d49a00640 1 -- 172.21.15.5:0/3025375605 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2d441049a0 msgr2=0x7f2d4419ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:22.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:22.352+0000 7f2d49a00640 1 --2- 172.21.15.5:0/3025375605 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2d441049a0 0x7f2d4419ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:22.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:22.352+0000 7f2d49a00640 1 -- 172.21.15.5:0/3025375605 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d4419f5d0 con 0x7f2d44103ff0 2024-09-10T10:03:22.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:22.352+0000 7f2d49000640 1 --2- 172.21.15.5:0/3025375605 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2d441049a0 0x7f2d4419ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:03:22.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:22.352+0000 7f2d4a400640 1 --2- 172.21.15.5:0/3025375605 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2d44109120 0x7f2d4419eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:03:22.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:22.352+0000 7f2d49a00640 1 --2- 172.21.15.5:0/3025375605 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2d44103ff0 0x7f2d4419a620 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f2d3800e9f0 tx=0x7f2d3800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:22.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:22.352+0000 7f2d42000640 1 -- 172.21.15.5:0/3025375605 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d3800ccc0 con 0x7f2d44103ff0 2024-09-10T10:03:22.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:22.352+0000 7f2d4c400640 1 -- 172.21.15.5:0/3025375605 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d4419f8c0 con 0x7f2d44103ff0 2024-09-10T10:03:22.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:22.352+0000 7f2d42000640 1 -- 172.21.15.5:0/3025375605 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2d38005020 con 0x7f2d44103ff0 2024-09-10T10:03:22.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:22.352+0000 7f2d42000640 1 -- 172.21.15.5:0/3025375605 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d380049e0 con 0x7f2d44103ff0 2024-09-10T10:03:22.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:22.352+0000 7f2d4c400640 1 -- 172.21.15.5:0/3025375605 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d441a71a0 con 0x7f2d44103ff0 2024-09-10T10:03:22.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:22.357+0000 7f2d4eed5740 1 Processor -- start 2024-09-10T10:03:22.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:22.357+0000 7f2d4eed5740 1 -- start start 2024-09-10T10:03:22.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:22.357+0000 7f2d42000640 1 -- 172.21.15.5:0/3025375605 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f2d38005560 con 0x7f2d44103ff0 2024-09-10T10:03:22.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:22.357+0000 7f2d4eed5740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f4deba6c70 0x55f4deba7050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:22.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:22.357+0000 7f2d4eed5740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f4deba7590 0x55f4debb1910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:22.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:22.357+0000 7f2d4eed5740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f4debb1e50 0x55f4debb4240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:22.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:22.357+0000 7f2d4eed5740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55f4deb0d3e0 con 0x55f4debb1e50 2024-09-10T10:03:22.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:22.357+0000 7f2d4eed5740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55f4deab4d80 con 0x55f4deba6c70 2024-09-10T10:03:22.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:22.358+0000 7f2d4eed5740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55f4deb093a0 con 0x55f4deba7590 2024-09-10T10:03:22.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:22.358+0000 7f2d4a400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f4deba6c70 0x55f4deba7050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:22.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:22.358+0000 7f2d4a400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f4deba6c70 0x55f4deba7050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:36210/0 (socket says 172.21.15.5:36210) 2024-09-10T10:03:22.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:22.358+0000 7f2d4a400640 1 -- 172.21.15.5:0/870460328 learned_addr learned my addr 172.21.15.5:0/870460328 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T10:03:22.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:22.358+0000 7f2d49000640 1 --2- 172.21.15.5:0/870460328 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f4debb1e50 0x55f4debb4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:22.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:22.358+0000 7f2d49a00640 1 --2- 172.21.15.5:0/870460328 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f4deba7590 0x55f4debb1910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:22.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:22.358+0000 7f2d49000640 1 -- 172.21.15.5:0/870460328 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f4deba7590 msgr2=0x55f4debb1910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:03:22.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:22.358+0000 7f2d49000640 1 --2- 172.21.15.5:0/870460328 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f4deba7590 0x55f4debb1910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:22.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:22.358+0000 7f2d49000640 1 -- 172.21.15.5:0/870460328 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f4deba6c70 msgr2=0x55f4deba7050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:03:22.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:22.358+0000 7f2d49000640 1 --2- 172.21.15.5:0/870460328 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f4deba6c70 0x55f4deba7050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:22.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:22.358+0000 7f2d49000640 1 -- 172.21.15.5:0/870460328 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f4dead3ae0 con 0x55f4debb1e50 2024-09-10T10:03:22.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:22.358+0000 7f2d4a400640 1 --2- 172.21.15.5:0/870460328 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f4deba6c70 0x55f4deba7050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:03:22.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:22.359+0000 7f2d49000640 1 --2- 172.21.15.5:0/870460328 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f4debb1e50 0x55f4debb4240 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f2d30009230 tx=0x7f2d30005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=705f9dc769626dae server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:22.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:22.359+0000 7f2d42000640 1 --2- 172.21.15.5:0/3025375605 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f2d2c0a2b80 0x7f2d2c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:22.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:22.359+0000 7f2d42000640 1 -- 172.21.15.5:0/3025375605 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f2d380cf410 con 0x7f2d44103ff0 2024-09-10T10:03:22.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:22.360+0000 7f2d4a400640 1 --2- 172.21.15.5:0/3025375605 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f2d2c0a2b80 0x7f2d2c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:22.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:22.360+0000 7f2d36a00640 1 -- 172.21.15.5:0/870460328 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d30035040 con 0x55f4debb1e50 2024-09-10T10:03:22.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:22.360+0000 7f2d36a00640 1 -- 172.21.15.5:0/870460328 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2d30031730 con 0x55f4debb1e50 2024-09-10T10:03:22.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:22.360+0000 7f2d36a00640 1 -- 172.21.15.5:0/870460328 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d30005550 con 0x55f4debb1e50 2024-09-10T10:03:22.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:22.361+0000 7f2d4eed5740 1 -- 172.21.15.5:0/870460328 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f4debb1e50 msgr2=0x55f4debb4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:22.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:22.361+0000 7f2d4a400640 1 --2- 172.21.15.5:0/3025375605 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f2d2c0a2b80 0x7f2d2c0a5040 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f2d3c0092f0 tx=0x7f2d3c009600 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:22.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:22.361+0000 7f2d4eed5740 1 --2- 172.21.15.5:0/870460328 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f4debb1e50 0x55f4debb4240 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f2d30009230 tx=0x7f2d30005e80 comp rx=0 tx=0).stop 2024-09-10T10:03:22.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:22.361+0000 7f2d4eed5740 1 -- 172.21.15.5:0/870460328 shutdown_connections 2024-09-10T10:03:22.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:22.361+0000 7f2d4eed5740 1 --2- 172.21.15.5:0/870460328 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f4debb1e50 0x55f4debb4240 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:22.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:22.361+0000 7f2d4eed5740 1 --2- 172.21.15.5:0/870460328 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f4deba7590 0x55f4debb1910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:22.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:22.361+0000 7f2d4eed5740 1 --2- 172.21.15.5:0/870460328 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f4deba6c70 0x55f4deba7050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:22.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:22.361+0000 7f2d4eed5740 1 -- 172.21.15.5:0/870460328 >> 172.21.15.5:0/870460328 conn(0x55f4deba3200 msgr2=0x55f4deba35e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:22.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:22.361+0000 7f2d4eed5740 1 -- 172.21.15.5:0/870460328 shutdown_connections 2024-09-10T10:03:22.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:22.361+0000 7f2d4eed5740 1 -- 172.21.15.5:0/870460328 wait complete. 2024-09-10T10:03:22.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:22.361+0000 7f2d4eed5740 1 -- start start 2024-09-10T10:03:22.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:22.362+0000 7f2d4eed5740 1 Processor -- start 2024-09-10T10:03:22.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:22.362+0000 7f2d4eed5740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f4deba6c70 0x55f4debaedd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:22.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:22.362+0000 7f2d4eed5740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f4deba7590 0x55f4debaf310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:22.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:22.362+0000 7f2d4eed5740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f4debb1e50 0x55f4debaf850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:22.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:22.362+0000 7f2d4eed5740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55f4debb4a70 con 0x55f4deba6c70 2024-09-10T10:03:22.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:22.362+0000 7f2d4eed5740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55f4deac80f0 con 0x55f4debb1e50 2024-09-10T10:03:22.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:22.362+0000 7f2d4eed5740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55f4debb4bf0 con 0x55f4deba7590 2024-09-10T10:03:22.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:22.362+0000 7f2d4eed5740 10 client.0 _ll_register_callbacks cb 0x55f4deaafbd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:03:22.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:22.363+0000 7f2d49a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f4deba6c70 0x55f4debaedd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:22.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:22.363+0000 7f2d49000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f4deba7590 0x55f4debaf310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:22.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:22.363+0000 7f2d49a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f4deba6c70 0x55f4debaedd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49230/0 (socket says 172.21.15.5:49230) 2024-09-10T10:03:22.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:22.363+0000 7f2d49a00640 1 -- 172.21.15.5:0/2710885588 learned_addr learned my addr 172.21.15.5:0/2710885588 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T10:03:22.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:22.363+0000 7f2d4a400640 1 --2- 172.21.15.5:0/2710885588 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f4debb1e50 0x55f4debaf850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:22.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:22.363+0000 7f2d49a00640 1 -- 172.21.15.5:0/2710885588 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f4deba7590 msgr2=0x55f4debaf310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:03:22.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:22.363+0000 7f2d49a00640 1 --2- 172.21.15.5:0/2710885588 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f4deba7590 0x55f4debaf310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:22.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:22.363+0000 7f2d49a00640 1 -- 172.21.15.5:0/2710885588 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f4debb1e50 msgr2=0x55f4debaf850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:03:22.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:22.363+0000 7f2d49a00640 1 --2- 172.21.15.5:0/2710885588 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f4debb1e50 0x55f4debaf850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:22.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:22.363+0000 7f2d49a00640 1 -- 172.21.15.5:0/2710885588 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f4de87e3c0 con 0x55f4deba6c70 2024-09-10T10:03:22.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:22.363+0000 7f2d4a400640 1 --2- 172.21.15.5:0/2710885588 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f4debb1e50 0x55f4debaf850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:03:22.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:22.363+0000 7f2d49000640 1 --2- 172.21.15.5:0/2710885588 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f4deba7590 0x55f4debaf310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:03:22.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:22.363+0000 7f2d49a00640 1 --2- 172.21.15.5:0/2710885588 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f4deba6c70 0x55f4debaedd0 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f2d380981f0 tx=0x7f2d380974c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c2f63f605db7e53 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:22.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:22.363+0000 7f2d2b400640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-10T10:03:22.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:22.364+0000 7f2d2b400640 1 -- 172.21.15.5:0/2710885588 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d3801c070 con 0x55f4deba6c70 2024-09-10T10:03:22.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:22.364+0000 7f2d2b400640 1 -- 172.21.15.5:0/2710885588 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f2d380860c0 con 0x55f4deba6c70 2024-09-10T10:03:22.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:22.364+0000 7f2d4eed5740 1 -- 172.21.15.5:0/2710885588 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2d30031730 con 0x55f4deba6c70 2024-09-10T10:03:22.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:22.364+0000 7f2d2b400640 1 -- 172.21.15.5:0/2710885588 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f2d3809b180 con 0x55f4deba6c70 2024-09-10T10:03:22.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:22.364+0000 7f2d4eed5740 10 client.19986 fetch_fsmap learned FSMap version 42 2024-09-10T10:03:22.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:22.364+0000 7f2d4eed5740 1 -- 172.21.15.5:0/2710885588 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55f4debc0be0 con 0x55f4deba6c70 2024-09-10T10:03:22.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:22.364+0000 7f2d2b400640 1 -- 172.21.15.5:0/2710885588 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2d30031730 con 0x55f4deba6c70 2024-09-10T10:03:22.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:22.364+0000 7f2d2b400640 1 -- 172.21.15.5:0/2710885588 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 42) ==== 8370+0+0 (secure 0 0 0) 0x7f2d38004ca0 con 0x55f4deba6c70 2024-09-10T10:03:22.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:22.365+0000 7f2d4eed5740 10 client.19986 fetch_fsmap finished waiting for FSMap version 42 2024-09-10T10:03:22.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:22.365+0000 7f2d4eed5740 10 client.19986 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-09-10T10:03:22.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:22.365+0000 7f2d4eed5740 20 client.19986 populate_metadata read hostname 'smithi005' 2024-09-10T10:03:22.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:22.365+0000 7f2d4eed5740 1 --2- 172.21.15.5:0/2710885588 >> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] conn(0x55f4debc20a0 0x55f4debc4490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:22.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:22.365+0000 7f2d4eed5740 4 client.19986 mds_command: new command op to 24443 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:03:22.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:22.365+0000 7f2d4eed5740 1 -- 172.21.15.5:0/2710885588 --> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f4deb1a950 con 0x55f4debc20a0 2024-09-10T10:03:22.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:22.365+0000 7f2d49000640 1 --2- 172.21.15.5:0/2710885588 >> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] conn(0x55f4debc20a0 0x55f4debc4490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:22.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:22.366+0000 7f2d49000640 1 --2- 172.21.15.5:0/2710885588 >> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] conn(0x55f4debc20a0 0x55f4debc4490 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=1a6838e1773c326c server_cookie=60f5812ea813cea4 in_seq=0 out_seq=0 2024-09-10T10:03:22.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:22.366+0000 7f2d2b400640 10 client.19986 ms_handle_connect on v2:172.21.15.102:6832/2457007427 2024-09-10T10:03:22.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:22.366+0000 7f2d2b400640 1 -- 172.21.15.5:0/2710885588 <== mds.4 v2:172.21.15.102:6832/2457007427 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f4deb1a950 con 0x55f4debc20a0 2024-09-10T10:03:22.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:22.366+0000 7f2d2b400640 10 client.19986 handle_command_reply: tid=0 2024-09-10T10:03:22.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:22.367+0000 7f2d4eed5740 1 client.19986 shutdown 2024-09-10T10:03:22.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:22.367+0000 7f2d4eed5740 1 -- 172.21.15.5:0/2710885588 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f4deba6c70 msgr2=0x55f4debaedd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:22.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:22.367+0000 7f2d4eed5740 1 --2- 172.21.15.5:0/2710885588 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f4deba6c70 0x55f4debaedd0 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f2d380981f0 tx=0x7f2d380974c0 comp rx=0 tx=0).stop 2024-09-10T10:03:22.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:22.367+0000 7f2d4eed5740 1 -- 172.21.15.5:0/2710885588 shutdown_connections 2024-09-10T10:03:22.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:22.367+0000 7f2d4eed5740 1 --2- 172.21.15.5:0/2710885588 >> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] conn(0x55f4debc20a0 0x55f4debc4490 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:22.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:22.367+0000 7f2d4eed5740 1 --2- 172.21.15.5:0/2710885588 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f4debb1e50 0x55f4debaf850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:22.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:22.367+0000 7f2d4eed5740 1 --2- 172.21.15.5:0/2710885588 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f4deba7590 0x55f4debaf310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:22.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:22.367+0000 7f2d4eed5740 1 --2- 172.21.15.5:0/2710885588 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f4deba6c70 0x55f4debaedd0 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:22.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:22.367+0000 7f2d4eed5740 1 -- 172.21.15.5:0/2710885588 >> 172.21.15.5:0/2710885588 conn(0x55f4deba3200 msgr2=0x55f4deba8ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:22.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:22.367+0000 7f2d2b400640 0 client.19986 ms_handle_reset on v2:172.21.15.102:6832/2457007427 2024-09-10T10:03:22.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:22.367+0000 7f2d4eed5740 1 -- 172.21.15.5:0/2710885588 shutdown_connections 2024-09-10T10:03:22.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:22.367+0000 7f2d4eed5740 1 -- 172.21.15.5:0/2710885588 wait complete. 2024-09-10T10:03:22.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:22.367+0000 7f2d4eed5740 20 client.19986 trim_cache size 0 max 16384 2024-09-10T10:03:22.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:22.381+0000 7f2d4eed5740 1 Processor -- start 2024-09-10T10:03:22.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:22.381+0000 7f2d4eed5740 1 -- start start 2024-09-10T10:03:22.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:22.381+0000 7f2d4eed5740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f4deba6c70 0x55f4deba96d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:22.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:22.381+0000 7f2d4eed5740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f4deba7590 0x55f4debc9c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:22.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:22.381+0000 7f2d4eed5740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f4debb1e50 0x55f4debca1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:22.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:22.381+0000 7f2d4eed5740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f2d441acc50 con 0x55f4debb1e50 2024-09-10T10:03:22.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:22.381+0000 7f2d4eed5740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f2d441ae790 con 0x55f4deba7590 2024-09-10T10:03:22.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:22.381+0000 7f2d4eed5740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f2d441ad320 con 0x55f4deba6c70 2024-09-10T10:03:22.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:22.382+0000 7f2d49a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f4deba6c70 0x55f4deba96d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:22.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:22.382+0000 7f2d49a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f4deba6c70 0x55f4deba96d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:33878/0 (socket says 172.21.15.5:33878) 2024-09-10T10:03:22.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:22.382+0000 7f2d49a00640 1 -- 172.21.15.5:0/4005861984 learned_addr learned my addr 172.21.15.5:0/4005861984 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T10:03:22.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:22.382+0000 7f2d4a400640 1 --2- 172.21.15.5:0/4005861984 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f4debb1e50 0x55f4debca1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:22.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:22.382+0000 7f2d49000640 1 --2- 172.21.15.5:0/4005861984 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f4deba7590 0x55f4debc9c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:22.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:22.382+0000 7f2d49a00640 1 -- 172.21.15.5:0/4005861984 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f4deba7590 msgr2=0x55f4debc9c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:03:22.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:22.382+0000 7f2d49a00640 1 --2- 172.21.15.5:0/4005861984 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f4deba7590 0x55f4debc9c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:22.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:22.382+0000 7f2d49a00640 1 -- 172.21.15.5:0/4005861984 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f4debb1e50 msgr2=0x55f4debca1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:03:22.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:22.382+0000 7f2d49a00640 1 --2- 172.21.15.5:0/4005861984 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f4debb1e50 0x55f4debca1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:22.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:22.382+0000 7f2d49a00640 1 -- 172.21.15.5:0/4005861984 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f4deba6aa0 con 0x55f4deba6c70 2024-09-10T10:03:22.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:22.382+0000 7f2d49a00640 1 --2- 172.21.15.5:0/4005861984 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f4deba6c70 0x55f4deba96d0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f2d380981a0 tx=0x7f2d3800e940 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b344d9700d2def73 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:22.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:22.383+0000 7f2d36000640 1 -- 172.21.15.5:0/4005861984 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d3801c070 con 0x55f4deba6c70 2024-09-10T10:03:22.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:22.383+0000 7f2d36000640 1 -- 172.21.15.5:0/4005861984 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2d380051d0 con 0x55f4deba6c70 2024-09-10T10:03:22.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:22.383+0000 7f2d36000640 1 -- 172.21.15.5:0/4005861984 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d380cf200 con 0x55f4deba6c70 2024-09-10T10:03:22.386 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:03:21 smithi102 ceph-mon[35442]: pgmap v1143: 129 pgs: 129 active+clean; 798 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 7.6 MiB/s wr, 372 op/s 2024-09-10T10:03:22.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:22.383+0000 7f2d4eed5740 1 -- 172.21.15.5:0/4005861984 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f4deba6c70 msgr2=0x55f4deba96d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:22.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:22.383+0000 7f2d4eed5740 1 --2- 172.21.15.5:0/4005861984 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f4deba6c70 0x55f4deba96d0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f2d380981a0 tx=0x7f2d3800e940 comp rx=0 tx=0).stop 2024-09-10T10:03:22.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:22.383+0000 7f2d4eed5740 1 -- 172.21.15.5:0/4005861984 shutdown_connections 2024-09-10T10:03:22.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:22.383+0000 7f2d4eed5740 1 --2- 172.21.15.5:0/4005861984 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f4debb1e50 0x55f4debca1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:22.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:22.383+0000 7f2d4eed5740 1 --2- 172.21.15.5:0/4005861984 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f4deba7590 0x55f4debc9c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:22.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:22.383+0000 7f2d4eed5740 1 --2- 172.21.15.5:0/4005861984 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f4deba6c70 0x55f4deba96d0 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:22.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:22.383+0000 7f2d4eed5740 1 -- 172.21.15.5:0/4005861984 >> 172.21.15.5:0/4005861984 conn(0x55f4deba3200 msgr2=0x55f4debb0e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:22.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:22.384+0000 7f2d4eed5740 1 -- 172.21.15.5:0/4005861984 shutdown_connections 2024-09-10T10:03:22.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:22.384+0000 7f2d4eed5740 1 -- 172.21.15.5:0/4005861984 wait complete. 2024-09-10T10:03:22.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:22.384+0000 7f2d4eed5740 1 -- start start 2024-09-10T10:03:22.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:22.384+0000 7f2d4eed5740 1 Processor -- start 2024-09-10T10:03:22.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:22.385+0000 7f2d4eed5740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f4deba6c70 0x55f4debb9e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:22.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:22.385+0000 7f2d4eed5740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f4deba7590 0x55f4debba3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:22.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:22.385+0000 7f2d4eed5740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f4debb1e50 0x55f4debb53f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:22.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:22.385+0000 7f2d4eed5740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f2d441ae0c0 con 0x55f4deba6c70 2024-09-10T10:03:22.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:22.385+0000 7f2d4eed5740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f2d441ad9f0 con 0x55f4deba7590 2024-09-10T10:03:22.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:22.385+0000 7f2d4eed5740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f2d441aac30 con 0x55f4debb1e50 2024-09-10T10:03:22.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:22.385+0000 7f2d4eed5740 10 client.0 _ll_register_callbacks cb 0x7f2d380d2b40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:03:22.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:22.385+0000 7f2d49a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f4deba6c70 0x55f4debb9e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:22.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:22.385+0000 7f2d49a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f4deba6c70 0x55f4debb9e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49238/0 (socket says 172.21.15.5:49238) 2024-09-10T10:03:22.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:22.385+0000 7f2d49000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f4deba7590 0x55f4debba3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:22.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:22.385+0000 7f2d49a00640 1 -- 172.21.15.5:0/2242643934 learned_addr learned my addr 172.21.15.5:0/2242643934 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T10:03:22.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:22.385+0000 7f2d4a400640 1 --2- 172.21.15.5:0/2242643934 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f4debb1e50 0x55f4debb53f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:22.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:22.385+0000 7f2d49a00640 1 -- 172.21.15.5:0/2242643934 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f4debb1e50 msgr2=0x55f4debb53f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:03:22.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:22.385+0000 7f2d49a00640 1 --2- 172.21.15.5:0/2242643934 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f4debb1e50 0x55f4debb53f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:22.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:22.385+0000 7f2d49a00640 1 -- 172.21.15.5:0/2242643934 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f4deba7590 msgr2=0x55f4debba3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:03:22.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:22.385+0000 7f2d49a00640 1 --2- 172.21.15.5:0/2242643934 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f4deba7590 0x55f4debba3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:22.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:22.385+0000 7f2d49a00640 1 -- 172.21.15.5:0/2242643934 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f4deb9e770 con 0x55f4deba6c70 2024-09-10T10:03:22.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:22.386+0000 7f2d49a00640 1 --2- 172.21.15.5:0/2242643934 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f4deba6c70 0x55f4debb9e70 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f2d38097c80 tx=0x7f2d380054d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0faf044f1803a35 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:22.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:22.386+0000 7f2d2b400640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-10T10:03:22.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:22.386+0000 7f2d2b400640 1 -- 172.21.15.5:0/2242643934 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d3801c030 con 0x55f4deba6c70 2024-09-10T10:03:22.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:22.386+0000 7f2d2b400640 1 -- 172.21.15.5:0/2242643934 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f2d3809cb50 con 0x55f4deba6c70 2024-09-10T10:03:22.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:22.386+0000 7f2d4eed5740 1 -- 172.21.15.5:0/2242643934 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2d380051d0 con 0x55f4deba6c70 2024-09-10T10:03:22.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:22.386+0000 7f2d2b400640 1 -- 172.21.15.5:0/2242643934 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f2d380b9050 con 0x55f4deba6c70 2024-09-10T10:03:22.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:22.386+0000 7f2d4eed5740 10 client.19992 fetch_fsmap learned FSMap version 42 2024-09-10T10:03:22.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:22.386+0000 7f2d4eed5740 1 -- 172.21.15.5:0/2242643934 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55f4dec2c480 con 0x55f4deba6c70 2024-09-10T10:03:22.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:22.386+0000 7f2d2b400640 1 -- 172.21.15.5:0/2242643934 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2d380051d0 con 0x55f4deba6c70 2024-09-10T10:03:22.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:22.387+0000 7f2d2b400640 1 -- 172.21.15.5:0/2242643934 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 42) ==== 8370+0+0 (secure 0 0 0) 0x7f2d3809ac70 con 0x55f4deba6c70 2024-09-10T10:03:22.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:22.387+0000 7f2d4eed5740 10 client.19992 fetch_fsmap finished waiting for FSMap version 42 2024-09-10T10:03:22.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:22.387+0000 7f2d4eed5740 10 client.19992 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-09-10T10:03:22.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:22.387+0000 7f2d4eed5740 20 client.19992 populate_metadata read hostname 'smithi005' 2024-09-10T10:03:22.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:22.387+0000 7f2d4eed5740 1 --2- 172.21.15.5:0/2242643934 >> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] conn(0x55f4dec2c9c0 0x55f4dec2ede0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:22.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:22.387+0000 7f2d4eed5740 4 client.19992 mds_command: new command op to 24443 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T10:03:22.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:22.387+0000 7f2d4eed5740 1 -- 172.21.15.5:0/2242643934 --> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f4deb14e60 con 0x55f4dec2c9c0 2024-09-10T10:03:22.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:22.387+0000 7f2d49000640 1 --2- 172.21.15.5:0/2242643934 >> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] conn(0x55f4dec2c9c0 0x55f4dec2ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:22.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:22.388+0000 7f2d49000640 1 --2- 172.21.15.5:0/2242643934 >> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] conn(0x55f4dec2c9c0 0x55f4dec2ede0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=b4f11b012e8659ba server_cookie=2631b159f81efc7e in_seq=0 out_seq=0 2024-09-10T10:03:22.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:22.388+0000 7f2d2b400640 10 client.19992 ms_handle_connect on v2:172.21.15.102:6832/2457007427 2024-09-10T10:03:22.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:22.388+0000 7f2d2b400640 1 -- 172.21.15.5:0/2242643934 <== mds.4 v2:172.21.15.102:6832/2457007427 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f4deb14e60 con 0x55f4dec2c9c0 2024-09-10T10:03:22.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:22.388+0000 7f2d2b400640 10 client.19992 handle_command_reply: tid=0 2024-09-10T10:03:22.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:22.388+0000 7f2d4eed5740 1 client.19992 shutdown 2024-09-10T10:03:22.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:22.388+0000 7f2d4eed5740 1 -- 172.21.15.5:0/2242643934 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f4deba6c70 msgr2=0x55f4debb9e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:22.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:22.388+0000 7f2d4eed5740 1 --2- 172.21.15.5:0/2242643934 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f4deba6c70 0x55f4debb9e70 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f2d38097c80 tx=0x7f2d380054d0 comp rx=0 tx=0).stop 2024-09-10T10:03:22.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:22.388+0000 7f2d4eed5740 1 -- 172.21.15.5:0/2242643934 shutdown_connections 2024-09-10T10:03:22.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:22.388+0000 7f2d4eed5740 1 --2- 172.21.15.5:0/2242643934 >> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] conn(0x55f4dec2c9c0 0x55f4dec2ede0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:22.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:22.388+0000 7f2d4eed5740 1 --2- 172.21.15.5:0/2242643934 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f4debb1e50 0x55f4debb53f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:22.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:22.388+0000 7f2d4eed5740 1 --2- 172.21.15.5:0/2242643934 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f4deba7590 0x55f4debba3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:22.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:22.388+0000 7f2d4eed5740 1 --2- 172.21.15.5:0/2242643934 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f4deba6c70 0x55f4debb9e70 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:22.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:22.388+0000 7f2d2b400640 0 client.19992 ms_handle_reset on v2:172.21.15.102:6832/2457007427 2024-09-10T10:03:22.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:22.388+0000 7f2d4eed5740 1 -- 172.21.15.5:0/2242643934 >> 172.21.15.5:0/2242643934 conn(0x55f4deba3200 msgr2=0x55f4deba1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:22.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:22.389+0000 7f2d4eed5740 1 -- 172.21.15.5:0/2242643934 shutdown_connections 2024-09-10T10:03:22.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:22.389+0000 7f2d4eed5740 1 -- 172.21.15.5:0/2242643934 wait complete. 2024-09-10T10:03:22.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:22.389+0000 7f2d4eed5740 20 client.19992 trim_cache size 0 max 16384 2024-09-10T10:03:22.394 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-10T10:03:22.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:22.389+0000 7f2d4c400640 1 -- 172.21.15.5:0/3025375605 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f2d2c0a2b80 msgr2=0x7f2d2c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:22.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:22.389+0000 7f2d4c400640 1 --2- 172.21.15.5:0/3025375605 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f2d2c0a2b80 0x7f2d2c0a5040 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f2d3c0092f0 tx=0x7f2d3c009600 comp rx=0 tx=0).stop 2024-09-10T10:03:22.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:22.389+0000 7f2d4c400640 1 -- 172.21.15.5:0/3025375605 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2d44103ff0 msgr2=0x7f2d4419a620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:22.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:22.389+0000 7f2d4c400640 1 --2- 172.21.15.5:0/3025375605 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2d44103ff0 0x7f2d4419a620 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f2d3800e9f0 tx=0x7f2d3800eeb0 comp rx=0 tx=0).stop 2024-09-10T10:03:22.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:22.389+0000 7f2d4c400640 1 -- 172.21.15.5:0/3025375605 shutdown_connections 2024-09-10T10:03:22.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:22.389+0000 7f2d4c400640 1 --2- 172.21.15.5:0/3025375605 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f2d2c0a2b80 0x7f2d2c0a5040 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:22.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:22.389+0000 7f2d4c400640 1 --2- 172.21.15.5:0/3025375605 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2d44109120 0x7f2d4419eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:22.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:22.389+0000 7f2d4c400640 1 --2- 172.21.15.5:0/3025375605 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2d441049a0 0x7f2d4419ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:22.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:22.389+0000 7f2d4c400640 1 --2- 172.21.15.5:0/3025375605 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2d44103ff0 0x7f2d4419a620 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:22.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:22.390+0000 7f2d4c400640 1 -- 172.21.15.5:0/3025375605 >> 172.21.15.5:0/3025375605 conn(0x7f2d440ffa90 msgr2=0x7f2d44100d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:22.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:22.390+0000 7f2d4c400640 1 -- 172.21.15.5:0/3025375605 shutdown_connections 2024-09-10T10:03:22.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:22.390+0000 7f2d4c400640 1 -- 172.21.15.5:0/3025375605 wait complete. 2024-09-10T10:03:23.086 INFO:tasks.workunit.client.0.smithi005.stdout: 10 177040 13.75 MB/sec execute 541 sec latency 10112.140 ms 2024-09-10T10:03:23.449 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-10T10:03:24.046 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:03:23 smithi005 ceph-mon[30350]: pgmap v1144: 129 pgs: 129 active+clean; 800 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 555 KiB/s rd, 6.0 MiB/s wr, 234 op/s 2024-09-10T10:03:24.101 INFO:tasks.workunit.client.0.smithi005.stdout: 10 177041 13.72 MB/sec execute 542 sec latency 11126.007 ms 2024-09-10T10:03:24.243 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:03:24.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:03:23 smithi159 ceph-mon[35513]: pgmap v1144: 129 pgs: 129 active+clean; 800 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 555 KiB/s rd, 6.0 MiB/s wr, 234 op/s 2024-09-10T10:03:24.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:03:23 smithi102 ceph-mon[35442]: pgmap v1144: 129 pgs: 129 active+clean; 800 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 555 KiB/s rd, 6.0 MiB/s wr, 234 op/s 2024-09-10T10:03:25.100 INFO:tasks.workunit.client.0.smithi005.stdout: 10 177041 13.70 MB/sec execute 543 sec latency 12126.061 ms 2024-09-10T10:03:26.100 INFO:tasks.workunit.client.0.smithi005.stdout: 10 177041 13.67 MB/sec execute 544 sec latency 13126.119 ms 2024-09-10T10:03:26.306 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:03:25 smithi005 ceph-mon[30350]: pgmap v1145: 129 pgs: 129 active+clean; 798 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 474 KiB/s rd, 6.1 MiB/s wr, 243 op/s 2024-09-10T10:03:26.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:03:26 smithi159 ceph-mon[35513]: pgmap v1145: 129 pgs: 129 active+clean; 798 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 474 KiB/s rd, 6.1 MiB/s wr, 243 op/s 2024-09-10T10:03:26.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:03:26 smithi102 ceph-mon[35442]: pgmap v1145: 129 pgs: 129 active+clean; 798 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 474 KiB/s rd, 6.1 MiB/s wr, 243 op/s 2024-09-10T10:03:27.100 INFO:tasks.workunit.client.0.smithi005.stdout: 10 177041 13.65 MB/sec execute 545 sec latency 14126.169 ms 2024-09-10T10:03:27.306 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:03:27 smithi005 ceph-mon[30350]: pgmap v1146: 129 pgs: 129 active+clean; 794 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 463 KiB/s rd, 6.0 MiB/s wr, 245 op/s 2024-09-10T10:03:27.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:03:27 smithi159 ceph-mon[35513]: pgmap v1146: 129 pgs: 129 active+clean; 794 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 463 KiB/s rd, 6.0 MiB/s wr, 245 op/s 2024-09-10T10:03:27.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:03:27 smithi102 ceph-mon[35442]: pgmap v1146: 129 pgs: 129 active+clean; 794 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 463 KiB/s rd, 6.0 MiB/s wr, 245 op/s 2024-09-10T10:03:28.102 INFO:tasks.workunit.client.0.smithi005.stdout: 10 177041 13.62 MB/sec execute 546 sec latency 15126.229 ms 2024-09-10T10:03:28.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:28.145+0000 7f5b6f600640 1 -- 172.21.15.5:0/2263167077 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5b68077040 msgr2=0x7f5b681124f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:28.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:28.145+0000 7f5b6f600640 1 --2- 172.21.15.5:0/2263167077 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5b68077040 0x7f5b681124f0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f5b5c009860 tx=0x7f5b5c02a340 comp rx=0 tx=0).stop 2024-09-10T10:03:28.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:28.145+0000 7f5b6f600640 1 -- 172.21.15.5:0/2263167077 shutdown_connections 2024-09-10T10:03:28.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:28.145+0000 7f5b6f600640 1 --2- 172.21.15.5:0/2263167077 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5b68112a30 0x7f5b68114e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:28.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:28.145+0000 7f5b6f600640 1 --2- 172.21.15.5:0/2263167077 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5b68077040 0x7f5b681124f0 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:28.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:28.145+0000 7f5b6f600640 1 --2- 172.21.15.5:0/2263167077 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5b68076720 0x7f5b68076b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:28.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:28.145+0000 7f5b6f600640 1 -- 172.21.15.5:0/2263167077 >> 172.21.15.5:0/2263167077 conn(0x7f5b680ffa90 msgr2=0x7f5b68101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:28.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:28.145+0000 7f5b6f600640 1 -- 172.21.15.5:0/2263167077 shutdown_connections 2024-09-10T10:03:28.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:28.145+0000 7f5b6f600640 1 -- 172.21.15.5:0/2263167077 wait complete. 2024-09-10T10:03:28.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:28.146+0000 7f5b6f600640 1 Processor -- start 2024-09-10T10:03:28.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:28.146+0000 7f5b6f600640 1 -- start start 2024-09-10T10:03:28.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:28.147+0000 7f5b6f600640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5b68076720 0x7f5b681a4730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:28.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:28.147+0000 7f5b6f600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5b68077040 0x7f5b681a4c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:28.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:28.147+0000 7f5b6f600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5b68112a30 0x7f5b6819e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:28.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:28.147+0000 7f5b6f600640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f5b681185a0 con 0x7f5b68077040 2024-09-10T10:03:28.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:28.147+0000 7f5b6f600640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f5b68118420 con 0x7f5b68112a30 2024-09-10T10:03:28.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:28.147+0000 7f5b6f600640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f5b68118720 con 0x7f5b68076720 2024-09-10T10:03:28.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:28.147+0000 7f5b67e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5b68077040 0x7f5b681a4c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:28.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:28.147+0000 7f5b6cc00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5b68076720 0x7f5b681a4730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:28.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:28.147+0000 7f5b6cc00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5b68076720 0x7f5b681a4730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:33912/0 (socket says 172.21.15.5:33912) 2024-09-10T10:03:28.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:28.147+0000 7f5b6cc00640 1 -- 172.21.15.5:0/751468067 learned_addr learned my addr 172.21.15.5:0/751468067 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T10:03:28.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:28.147+0000 7f5b6cc00640 1 -- 172.21.15.5:0/751468067 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5b68112a30 msgr2=0x7f5b6819e8f0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-10T10:03:28.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:28.147+0000 7f5b6cc00640 1 --2- 172.21.15.5:0/751468067 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5b68112a30 0x7f5b6819e8f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:28.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:28.148+0000 7f5b6cc00640 1 -- 172.21.15.5:0/751468067 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5b68077040 msgr2=0x7f5b681a4c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:28.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:28.148+0000 7f5b6cc00640 1 --2- 172.21.15.5:0/751468067 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5b68077040 0x7f5b681a4c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:28.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:28.148+0000 7f5b6cc00640 1 -- 172.21.15.5:0/751468067 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b6819f020 con 0x7f5b68076720 2024-09-10T10:03:28.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:28.148+0000 7f5b67e00640 1 --2- 172.21.15.5:0/751468067 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5b68077040 0x7f5b681a4c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:03:28.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:28.148+0000 7f5b6cc00640 1 --2- 172.21.15.5:0/751468067 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5b68076720 0x7f5b681a4730 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f5b5400ea60 tx=0x7f5b5400ef20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:28.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:28.148+0000 7f5b65600640 1 -- 172.21.15.5:0/751468067 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b5400cde0 con 0x7f5b68076720 2024-09-10T10:03:28.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:28.148+0000 7f5b6f600640 1 -- 172.21.15.5:0/751468067 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b6819f310 con 0x7f5b68076720 2024-09-10T10:03:28.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:28.148+0000 7f5b65600640 1 -- 172.21.15.5:0/751468067 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5b540035f0 con 0x7f5b68076720 2024-09-10T10:03:28.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:28.148+0000 7f5b65600640 1 -- 172.21.15.5:0/751468067 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b54010670 con 0x7f5b68076720 2024-09-10T10:03:28.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:28.148+0000 7f5b6f600640 1 -- 172.21.15.5:0/751468067 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b681ab600 con 0x7f5b68076720 2024-09-10T10:03:28.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:28.151+0000 7f5b72262740 1 Processor -- start 2024-09-10T10:03:28.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:28.151+0000 7f5b72262740 1 -- start start 2024-09-10T10:03:28.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:28.152+0000 7f5b65600640 1 -- 172.21.15.5:0/751468067 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f5b540037a0 con 0x7f5b68076720 2024-09-10T10:03:28.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:28.152+0000 7f5b72262740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556059d19c70 0x556059d1a050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:28.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:28.152+0000 7f5b72262740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556059d1a590 0x556059d24910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:28.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:28.152+0000 7f5b72262740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556059d24e50 0x556059d27240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:28.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:28.152+0000 7f5b72262740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x556059c803e0 con 0x556059d1a590 2024-09-10T10:03:28.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:28.152+0000 7f5b72262740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x556059c27d80 con 0x556059d24e50 2024-09-10T10:03:28.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:28.152+0000 7f5b72262740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x556059c7c3a0 con 0x556059d19c70 2024-09-10T10:03:28.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:28.152+0000 7f5b6cc00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556059d1a590 0x556059d24910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:28.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:28.152+0000 7f5b6d600640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556059d19c70 0x556059d1a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:28.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:28.152+0000 7f5b6d600640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556059d19c70 0x556059d1a050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:33918/0 (socket says 172.21.15.5:33918) 2024-09-10T10:03:28.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:28.152+0000 7f5b6d600640 1 -- 172.21.15.5:0/2622465015 learned_addr learned my addr 172.21.15.5:0/2622465015 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T10:03:28.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:28.153+0000 7f5b6d600640 1 -- 172.21.15.5:0/2622465015 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556059d24e50 msgr2=0x556059d27240 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T10:03:28.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:28.153+0000 7f5b6d600640 1 --2- 172.21.15.5:0/2622465015 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556059d24e50 0x556059d27240 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:28.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:28.153+0000 7f5b6d600640 1 -- 172.21.15.5:0/2622465015 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556059d1a590 msgr2=0x556059d24910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:03:28.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:28.153+0000 7f5b6d600640 1 --2- 172.21.15.5:0/2622465015 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556059d1a590 0x556059d24910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:28.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:28.153+0000 7f5b6d600640 1 -- 172.21.15.5:0/2622465015 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556059c46ae0 con 0x556059d19c70 2024-09-10T10:03:28.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:28.153+0000 7f5b6d600640 1 --2- 172.21.15.5:0/2622465015 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556059d19c70 0x556059d1a050 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f5b60009e80 tx=0x7f5b6000e7c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=877bb05374fe8c8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:28.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:28.153+0000 7f5b59c00640 1 -- 172.21.15.5:0/2622465015 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b60003b70 con 0x556059d19c70 2024-09-10T10:03:28.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:28.153+0000 7f5b59c00640 1 -- 172.21.15.5:0/2622465015 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5b60003d10 con 0x556059d19c70 2024-09-10T10:03:28.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:28.153+0000 7f5b59c00640 1 -- 172.21.15.5:0/2622465015 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b6000cbf0 con 0x556059d19c70 2024-09-10T10:03:28.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:28.154+0000 7f5b65600640 1 --2- 172.21.15.5:0/751468067 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f5b500a2b80 0x7f5b500a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:28.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:28.154+0000 7f5b6cc00640 1 --2- 172.21.15.5:0/751468067 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f5b500a2b80 0x7f5b500a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:28.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:28.154+0000 7f5b65600640 1 -- 172.21.15.5:0/751468067 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f5b540cf7f0 con 0x7f5b68076720 2024-09-10T10:03:28.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:28.154+0000 7f5b72262740 1 -- 172.21.15.5:0/2622465015 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556059d19c70 msgr2=0x556059d1a050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:28.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:28.154+0000 7f5b72262740 1 --2- 172.21.15.5:0/2622465015 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556059d19c70 0x556059d1a050 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f5b60009e80 tx=0x7f5b6000e7c0 comp rx=0 tx=0).stop 2024-09-10T10:03:28.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:28.154+0000 7f5b72262740 1 -- 172.21.15.5:0/2622465015 shutdown_connections 2024-09-10T10:03:28.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:28.154+0000 7f5b72262740 1 --2- 172.21.15.5:0/2622465015 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556059d24e50 0x556059d27240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:28.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:28.154+0000 7f5b72262740 1 --2- 172.21.15.5:0/2622465015 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556059d1a590 0x556059d24910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:28.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:28.154+0000 7f5b72262740 1 --2- 172.21.15.5:0/2622465015 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556059d19c70 0x556059d1a050 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:28.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:28.154+0000 7f5b72262740 1 -- 172.21.15.5:0/2622465015 >> 172.21.15.5:0/2622465015 conn(0x556059d16200 msgr2=0x556059d165e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:28.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:28.154+0000 7f5b72262740 1 -- 172.21.15.5:0/2622465015 shutdown_connections 2024-09-10T10:03:28.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:28.154+0000 7f5b72262740 1 -- 172.21.15.5:0/2622465015 wait complete. 2024-09-10T10:03:28.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:28.155+0000 7f5b6cc00640 1 --2- 172.21.15.5:0/751468067 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f5b500a2b80 0x7f5b500a5040 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f5b540985d0 tx=0x7f5b540d5530 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:28.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:28.155+0000 7f5b72262740 1 -- start start 2024-09-10T10:03:28.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:28.155+0000 7f5b72262740 1 Processor -- start 2024-09-10T10:03:28.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:28.156+0000 7f5b72262740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556059d19c70 0x556059d21d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:28.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:28.156+0000 7f5b72262740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556059d1a590 0x556059d22280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:28.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:28.156+0000 7f5b72262740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556059d24e50 0x556059d227c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:28.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:28.156+0000 7f5b72262740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x556059d27a20 con 0x556059d24e50 2024-09-10T10:03:28.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:28.156+0000 7f5b72262740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x556059c3b0f0 con 0x556059d1a590 2024-09-10T10:03:28.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:28.156+0000 7f5b72262740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x556059d27ba0 con 0x556059d19c70 2024-09-10T10:03:28.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:28.156+0000 7f5b72262740 10 client.0 _ll_register_callbacks cb 0x556059c22bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:03:28.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:28.156+0000 7f5b6d600640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556059d19c70 0x556059d21d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:28.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:28.156+0000 7f5b6d600640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556059d19c70 0x556059d21d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:33920/0 (socket says 172.21.15.5:33920) 2024-09-10T10:03:28.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:28.156+0000 7f5b6d600640 1 -- 172.21.15.5:0/676544090 learned_addr learned my addr 172.21.15.5:0/676544090 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T10:03:28.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:28.156+0000 7f5b6d600640 1 --2- 172.21.15.5:0/676544090 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556059d24e50 0x556059d227c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:28.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:28.157+0000 7f5b6d600640 1 -- 172.21.15.5:0/676544090 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556059d1a590 msgr2=0x556059d22280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:03:28.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:28.157+0000 7f5b6d600640 1 --2- 172.21.15.5:0/676544090 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556059d1a590 0x556059d22280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:28.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:28.157+0000 7f5b6d600640 1 -- 172.21.15.5:0/676544090 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556059d24e50 msgr2=0x556059d227c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:03:28.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:28.157+0000 7f5b6d600640 1 --2- 172.21.15.5:0/676544090 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556059d24e50 0x556059d227c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:28.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:28.157+0000 7f5b6d600640 1 -- 172.21.15.5:0/676544090 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5560599f13c0 con 0x556059d19c70 2024-09-10T10:03:28.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:28.157+0000 7f5b6d600640 1 --2- 172.21.15.5:0/676544090 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556059d19c70 0x556059d21d40 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f5b60018040 tx=0x7f5b6000c750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=17a4fd04c32b0c5a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:28.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:28.157+0000 7f5b4e000640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-09-10T10:03:28.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:28.157+0000 7f5b4e000640 1 -- 172.21.15.5:0/676544090 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b60029020 con 0x556059d19c70 2024-09-10T10:03:28.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:28.157+0000 7f5b4e000640 1 -- 172.21.15.5:0/676544090 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f5b60004430 con 0x556059d19c70 2024-09-10T10:03:28.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:28.157+0000 7f5b72262740 1 -- 172.21.15.5:0/676544090 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5b60003d10 con 0x556059d19c70 2024-09-10T10:03:28.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:28.158+0000 7f5b4e000640 1 -- 172.21.15.5:0/676544090 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f5b60034050 con 0x556059d19c70 2024-09-10T10:03:28.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:28.158+0000 7f5b72262740 10 client.30088 fetch_fsmap learned FSMap version 42 2024-09-10T10:03:28.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:28.158+0000 7f5b72262740 1 -- 172.21.15.5:0/676544090 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x556059d33a60 con 0x556059d19c70 2024-09-10T10:03:28.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:28.158+0000 7f5b4e000640 1 -- 172.21.15.5:0/676544090 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5b60003d10 con 0x556059d19c70 2024-09-10T10:03:28.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:28.158+0000 7f5b4e000640 1 -- 172.21.15.5:0/676544090 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 42) ==== 8370+0+0 (secure 0 0 0) 0x7f5b6000b040 con 0x556059d19c70 2024-09-10T10:03:28.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:28.158+0000 7f5b72262740 10 client.30088 fetch_fsmap finished waiting for FSMap version 42 2024-09-10T10:03:28.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:28.158+0000 7f5b72262740 10 client.30088 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-09-10T10:03:28.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:28.158+0000 7f5b72262740 20 client.30088 populate_metadata read hostname 'smithi005' 2024-09-10T10:03:28.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:28.158+0000 7f5b72262740 1 --2- 172.21.15.5:0/676544090 >> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] conn(0x556059d34e10 0x556059d37200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:28.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:28.158+0000 7f5b72262740 4 client.30088 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:03:28.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:28.158+0000 7f5b72262740 1 -- 172.21.15.5:0/676544090 --> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556059c8d950 con 0x556059d34e10 2024-09-10T10:03:28.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:28.159+0000 7f5b67e00640 1 --2- 172.21.15.5:0/676544090 >> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] conn(0x556059d34e10 0x556059d37200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:28.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:28.159+0000 7f5b67e00640 1 --2- 172.21.15.5:0/676544090 >> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] conn(0x556059d34e10 0x556059d37200 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=3e1c97fd160b6aca server_cookie=759d5ed88707f4f8 in_seq=0 out_seq=0 2024-09-10T10:03:28.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:28.159+0000 7f5b4e000640 10 client.30088 ms_handle_connect on v2:172.21.15.102:6836/2689776764 2024-09-10T10:03:28.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:28.160+0000 7f5b4e000640 1 -- 172.21.15.5:0/676544090 <== mds.3 v2:172.21.15.102:6836/2689776764 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556059c8d950 con 0x556059d34e10 2024-09-10T10:03:28.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:28.160+0000 7f5b4e000640 10 client.30088 handle_command_reply: tid=0 2024-09-10T10:03:28.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:28.160+0000 7f5b72262740 1 client.30088 shutdown 2024-09-10T10:03:28.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:28.160+0000 7f5b72262740 1 -- 172.21.15.5:0/676544090 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556059d19c70 msgr2=0x556059d21d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:28.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:28.160+0000 7f5b72262740 1 --2- 172.21.15.5:0/676544090 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556059d19c70 0x556059d21d40 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f5b60018040 tx=0x7f5b6000c750 comp rx=0 tx=0).stop 2024-09-10T10:03:28.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:28.161+0000 7f5b72262740 1 -- 172.21.15.5:0/676544090 shutdown_connections 2024-09-10T10:03:28.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:28.161+0000 7f5b72262740 1 --2- 172.21.15.5:0/676544090 >> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] conn(0x556059d34e10 0x556059d37200 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:28.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:28.161+0000 7f5b72262740 1 --2- 172.21.15.5:0/676544090 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556059d24e50 0x556059d227c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:28.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:28.161+0000 7f5b72262740 1 --2- 172.21.15.5:0/676544090 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556059d1a590 0x556059d22280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:28.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:28.161+0000 7f5b72262740 1 --2- 172.21.15.5:0/676544090 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556059d19c70 0x556059d21d40 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:28.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:28.161+0000 7f5b4e000640 0 client.30088 ms_handle_reset on v2:172.21.15.102:6836/2689776764 2024-09-10T10:03:28.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:28.161+0000 7f5b72262740 1 -- 172.21.15.5:0/676544090 >> 172.21.15.5:0/676544090 conn(0x556059d16200 msgr2=0x556059d15380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:28.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:28.161+0000 7f5b72262740 1 -- 172.21.15.5:0/676544090 shutdown_connections 2024-09-10T10:03:28.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:28.161+0000 7f5b72262740 1 -- 172.21.15.5:0/676544090 wait complete. 2024-09-10T10:03:28.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:28.161+0000 7f5b72262740 20 client.30088 trim_cache size 0 max 16384 2024-09-10T10:03:28.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:28.186+0000 7f5b72262740 1 Processor -- start 2024-09-10T10:03:28.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:28.187+0000 7f5b72262740 1 -- start start 2024-09-10T10:03:28.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:28.187+0000 7f5b72262740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556059d19c70 0x556059d1c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:28.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:28.187+0000 7f5b72262740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556059d1a590 0x556059d20610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:28.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:28.187+0000 7f5b72262740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556059d24e50 0x556059d3c970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:28.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:28.187+0000 7f5b72262740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f5b681b5590 con 0x556059d1a590 2024-09-10T10:03:28.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:28.187+0000 7f5b72262740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f5b681b70d0 con 0x556059d19c70 2024-09-10T10:03:28.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:28.187+0000 7f5b72262740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f5b681b5c60 con 0x556059d24e50 2024-09-10T10:03:28.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:28.187+0000 7f5b67e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556059d1a590 0x556059d20610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:28.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:28.187+0000 7f5b67e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556059d1a590 0x556059d20610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49278/0 (socket says 172.21.15.5:49278) 2024-09-10T10:03:28.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:28.187+0000 7f5b67e00640 1 -- 172.21.15.5:0/2846637651 learned_addr learned my addr 172.21.15.5:0/2846637651 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T10:03:28.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:28.188+0000 7f5b67e00640 1 -- 172.21.15.5:0/2846637651 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556059d24e50 msgr2=0x556059d3c970 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T10:03:28.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:28.188+0000 7f5b67e00640 1 --2- 172.21.15.5:0/2846637651 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556059d24e50 0x556059d3c970 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:28.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:28.188+0000 7f5b67e00640 1 -- 172.21.15.5:0/2846637651 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556059d19c70 msgr2=0x556059d1c6d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T10:03:28.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:28.188+0000 7f5b67e00640 1 --2- 172.21.15.5:0/2846637651 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556059d19c70 0x556059d1c6d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:28.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:28.188+0000 7f5b67e00640 1 -- 172.21.15.5:0/2846637651 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556059d1a3c0 con 0x556059d1a590 2024-09-10T10:03:28.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:28.188+0000 7f5b67e00640 1 --2- 172.21.15.5:0/2846637651 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556059d1a590 0x556059d20610 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f5b5c009780 tx=0x7f5b5c000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53560c921f4fb448 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:28.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:28.188+0000 7f5b59200640 1 -- 172.21.15.5:0/2846637651 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b5c037040 con 0x556059d1a590 2024-09-10T10:03:28.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:28.188+0000 7f5b59200640 1 -- 172.21.15.5:0/2846637651 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5b5c0042c0 con 0x556059d1a590 2024-09-10T10:03:28.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:28.188+0000 7f5b59200640 1 -- 172.21.15.5:0/2846637651 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b5c003d50 con 0x556059d1a590 2024-09-10T10:03:28.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:28.189+0000 7f5b72262740 1 -- 172.21.15.5:0/2846637651 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556059d1a590 msgr2=0x556059d20610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:28.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:28.189+0000 7f5b72262740 1 --2- 172.21.15.5:0/2846637651 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556059d1a590 0x556059d20610 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f5b5c009780 tx=0x7f5b5c000bc0 comp rx=0 tx=0).stop 2024-09-10T10:03:28.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:28.189+0000 7f5b72262740 1 -- 172.21.15.5:0/2846637651 shutdown_connections 2024-09-10T10:03:28.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:28.189+0000 7f5b72262740 1 --2- 172.21.15.5:0/2846637651 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556059d24e50 0x556059d3c970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:28.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:28.189+0000 7f5b72262740 1 --2- 172.21.15.5:0/2846637651 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556059d1a590 0x556059d20610 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:28.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:28.189+0000 7f5b72262740 1 --2- 172.21.15.5:0/2846637651 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556059d19c70 0x556059d1c6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:28.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:28.189+0000 7f5b72262740 1 -- 172.21.15.5:0/2846637651 >> 172.21.15.5:0/2846637651 conn(0x556059d16200 msgr2=0x556059d15380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:28.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:28.189+0000 7f5b72262740 1 -- 172.21.15.5:0/2846637651 shutdown_connections 2024-09-10T10:03:28.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:28.190+0000 7f5b72262740 1 -- 172.21.15.5:0/2846637651 wait complete. 2024-09-10T10:03:28.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:28.190+0000 7f5b72262740 1 -- start start 2024-09-10T10:03:28.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:28.190+0000 7f5b72262740 1 Processor -- start 2024-09-10T10:03:28.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:28.191+0000 7f5b72262740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556059d19c70 0x556059d2d410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:28.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:28.191+0000 7f5b72262740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556059d1a590 0x556059d1e950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:28.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:28.191+0000 7f5b72262740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556059d24e50 0x556059d1ee90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:28.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:28.191+0000 7f5b72262740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f5b681b6a00 con 0x556059d19c70 2024-09-10T10:03:28.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:28.191+0000 7f5b72262740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f5b681b6330 con 0x556059d24e50 2024-09-10T10:03:28.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:28.191+0000 7f5b72262740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f5b681b3570 con 0x556059d1a590 2024-09-10T10:03:28.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:28.191+0000 7f5b72262740 10 client.0 _ll_register_callbacks cb 0x7f5b540d5400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:03:28.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:28.191+0000 7f5b6d600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556059d19c70 0x556059d2d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:28.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:28.191+0000 7f5b6d600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556059d19c70 0x556059d2d410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49292/0 (socket says 172.21.15.5:49292) 2024-09-10T10:03:28.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:28.191+0000 7f5b6d600640 1 -- 172.21.15.5:0/787444447 learned_addr learned my addr 172.21.15.5:0/787444447 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T10:03:28.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:28.192+0000 7f5b6d600640 1 --2- 172.21.15.5:0/787444447 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556059d24e50 0x556059d1ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:28.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:28.192+0000 7f5b6d600640 1 -- 172.21.15.5:0/787444447 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556059d1a590 msgr2=0x556059d1e950 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T10:03:28.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:28.192+0000 7f5b6d600640 1 --2- 172.21.15.5:0/787444447 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556059d1a590 0x556059d1e950 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:28.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:28.192+0000 7f5b6d600640 1 -- 172.21.15.5:0/787444447 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556059d24e50 msgr2=0x556059d1ee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:03:28.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:28.192+0000 7f5b6d600640 1 --2- 172.21.15.5:0/787444447 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556059d24e50 0x556059d1ee90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:28.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:28.192+0000 7f5b6d600640 1 -- 172.21.15.5:0/787444447 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556059d1f580 con 0x556059d19c70 2024-09-10T10:03:28.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:28.192+0000 7f5b6d600640 1 --2- 172.21.15.5:0/787444447 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556059d19c70 0x556059d2d410 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f5b6000ce80 tx=0x7f5b60009f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85aab6c67ee2bd88 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:28.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:28.192+0000 7f5b4e000640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-10T10:03:28.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:28.192+0000 7f5b4e000640 1 -- 172.21.15.5:0/787444447 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b60029040 con 0x556059d19c70 2024-09-10T10:03:28.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:28.192+0000 7f5b4e000640 1 -- 172.21.15.5:0/787444447 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f5b60009540 con 0x556059d19c70 2024-09-10T10:03:28.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:28.192+0000 7f5b72262740 1 -- 172.21.15.5:0/787444447 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5b5c0042c0 con 0x556059d19c70 2024-09-10T10:03:28.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:28.193+0000 7f5b4e000640 1 -- 172.21.15.5:0/787444447 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f5b60009740 con 0x556059d19c70 2024-09-10T10:03:28.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:28.193+0000 7f5b72262740 10 client.20010 fetch_fsmap learned FSMap version 42 2024-09-10T10:03:28.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:28.193+0000 7f5b72262740 1 -- 172.21.15.5:0/787444447 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x556059d9f800 con 0x556059d19c70 2024-09-10T10:03:28.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:28.193+0000 7f5b4e000640 1 -- 172.21.15.5:0/787444447 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5b5c0042c0 con 0x556059d19c70 2024-09-10T10:03:28.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:28.193+0000 7f5b4e000640 1 -- 172.21.15.5:0/787444447 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 42) ==== 8370+0+0 (secure 0 0 0) 0x7f5b6000b070 con 0x556059d19c70 2024-09-10T10:03:28.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:28.193+0000 7f5b72262740 10 client.20010 fetch_fsmap finished waiting for FSMap version 42 2024-09-10T10:03:28.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:28.193+0000 7f5b72262740 10 client.20010 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-09-10T10:03:28.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:28.193+0000 7f5b72262740 20 client.20010 populate_metadata read hostname 'smithi005' 2024-09-10T10:03:28.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:28.193+0000 7f5b72262740 1 --2- 172.21.15.5:0/787444447 >> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] conn(0x556059d9ff80 0x556059da23a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:28.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:28.193+0000 7f5b72262740 4 client.20010 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T10:03:28.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:28.193+0000 7f5b72262740 1 -- 172.21.15.5:0/787444447 --> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556059c87e60 con 0x556059d9ff80 2024-09-10T10:03:28.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:28.194+0000 7f5b67e00640 1 --2- 172.21.15.5:0/787444447 >> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] conn(0x556059d9ff80 0x556059da23a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:28.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:28.194+0000 7f5b4e000640 10 client.20010 ms_handle_connect on v2:172.21.15.102:6836/2689776764 2024-09-10T10:03:28.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:28.194+0000 7f5b67e00640 1 --2- 172.21.15.5:0/787444447 >> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] conn(0x556059d9ff80 0x556059da23a0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=31283870ca82d963 server_cookie=f58c5890cbe79aec in_seq=0 out_seq=0 2024-09-10T10:03:28.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:28.195+0000 7f5b4e000640 1 -- 172.21.15.5:0/787444447 <== mds.3 v2:172.21.15.102:6836/2689776764 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556059c87e60 con 0x556059d9ff80 2024-09-10T10:03:28.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:28.195+0000 7f5b4e000640 10 client.20010 handle_command_reply: tid=0 2024-09-10T10:03:28.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:28.195+0000 7f5b72262740 1 client.20010 shutdown 2024-09-10T10:03:28.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:28.195+0000 7f5b72262740 1 -- 172.21.15.5:0/787444447 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556059d19c70 msgr2=0x556059d2d410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:28.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:28.195+0000 7f5b72262740 1 --2- 172.21.15.5:0/787444447 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556059d19c70 0x556059d2d410 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f5b6000ce80 tx=0x7f5b60009f70 comp rx=0 tx=0).stop 2024-09-10T10:03:28.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:28.195+0000 7f5b72262740 1 -- 172.21.15.5:0/787444447 shutdown_connections 2024-09-10T10:03:28.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:28.195+0000 7f5b72262740 1 --2- 172.21.15.5:0/787444447 >> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] conn(0x556059d9ff80 0x556059da23a0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:28.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:28.195+0000 7f5b72262740 1 --2- 172.21.15.5:0/787444447 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x556059d24e50 0x556059d1ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:28.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:28.195+0000 7f5b4e000640 0 client.20010 ms_handle_reset on v2:172.21.15.102:6836/2689776764 2024-09-10T10:03:28.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:28.195+0000 7f5b72262740 1 --2- 172.21.15.5:0/787444447 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x556059d1a590 0x556059d1e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:28.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:28.195+0000 7f5b72262740 1 --2- 172.21.15.5:0/787444447 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556059d19c70 0x556059d2d410 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:28.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:28.195+0000 7f5b72262740 1 -- 172.21.15.5:0/787444447 >> 172.21.15.5:0/787444447 conn(0x556059d16200 msgr2=0x556059d21d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:28.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:28.195+0000 7f5b72262740 1 -- 172.21.15.5:0/787444447 shutdown_connections 2024-09-10T10:03:28.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:28.195+0000 7f5b72262740 1 -- 172.21.15.5:0/787444447 wait complete. 2024-09-10T10:03:28.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:28.195+0000 7f5b72262740 20 client.20010 trim_cache size 0 max 16384 2024-09-10T10:03:28.200 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-10T10:03:28.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:28.196+0000 7f5b6f600640 1 -- 172.21.15.5:0/751468067 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f5b500a2b80 msgr2=0x7f5b500a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:28.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:28.196+0000 7f5b6f600640 1 --2- 172.21.15.5:0/751468067 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f5b500a2b80 0x7f5b500a5040 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f5b540985d0 tx=0x7f5b540d5530 comp rx=0 tx=0).stop 2024-09-10T10:03:28.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:28.196+0000 7f5b6f600640 1 -- 172.21.15.5:0/751468067 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5b68076720 msgr2=0x7f5b681a4730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:28.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:28.196+0000 7f5b6f600640 1 --2- 172.21.15.5:0/751468067 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5b68076720 0x7f5b681a4730 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f5b5400ea60 tx=0x7f5b5400ef20 comp rx=0 tx=0).stop 2024-09-10T10:03:28.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:28.196+0000 7f5b6f600640 1 -- 172.21.15.5:0/751468067 shutdown_connections 2024-09-10T10:03:28.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:28.196+0000 7f5b6f600640 1 --2- 172.21.15.5:0/751468067 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f5b500a2b80 0x7f5b500a5040 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:28.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:28.196+0000 7f5b6f600640 1 --2- 172.21.15.5:0/751468067 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5b68112a30 0x7f5b6819e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:28.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:28.196+0000 7f5b6f600640 1 --2- 172.21.15.5:0/751468067 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5b68077040 0x7f5b681a4c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:28.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:28.196+0000 7f5b6f600640 1 --2- 172.21.15.5:0/751468067 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5b68076720 0x7f5b681a4730 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:28.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:28.196+0000 7f5b6f600640 1 -- 172.21.15.5:0/751468067 >> 172.21.15.5:0/751468067 conn(0x7f5b680ffa90 msgr2=0x7f5b68101e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:28.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:28.196+0000 7f5b6f600640 1 -- 172.21.15.5:0/751468067 shutdown_connections 2024-09-10T10:03:28.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:28.196+0000 7f5b6f600640 1 -- 172.21.15.5:0/751468067 wait complete. 2024-09-10T10:03:29.100 INFO:tasks.workunit.client.0.smithi005.stdout: 10 177041 13.60 MB/sec execute 547 sec latency 16126.288 ms 2024-09-10T10:03:29.321 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:03:29 smithi005 ceph-mon[30350]: pgmap v1147: 129 pgs: 129 active+clean; 794 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 234 KiB/s rd, 5.8 MiB/s wr, 186 op/s 2024-09-10T10:03:29.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:03:29 smithi159 ceph-mon[35513]: pgmap v1147: 129 pgs: 129 active+clean; 794 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 234 KiB/s rd, 5.8 MiB/s wr, 186 op/s 2024-09-10T10:03:29.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:03:29 smithi102 ceph-mon[35442]: pgmap v1147: 129 pgs: 129 active+clean; 794 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 234 KiB/s rd, 5.8 MiB/s wr, 186 op/s 2024-09-10T10:03:30.100 INFO:tasks.workunit.client.0.smithi005.stdout: 10 177041 13.57 MB/sec execute 548 sec latency 17126.446 ms 2024-09-10T10:03:30.177 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-10T10:03:30.650 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:03:31.101 INFO:tasks.workunit.client.0.smithi005.stdout: 10 177041 13.55 MB/sec execute 549 sec latency 18126.500 ms 2024-09-10T10:03:31.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:03:31 smithi005 ceph-mon[30350]: pgmap v1148: 129 pgs: 129 active+clean; 793 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 289 KiB/s rd, 6.3 MiB/s wr, 219 op/s 2024-09-10T10:03:31.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:03:31 smithi159 ceph-mon[35513]: pgmap v1148: 129 pgs: 129 active+clean; 793 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 289 KiB/s rd, 6.3 MiB/s wr, 219 op/s 2024-09-10T10:03:31.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:03:31 smithi102 ceph-mon[35442]: pgmap v1148: 129 pgs: 129 active+clean; 793 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 289 KiB/s rd, 6.3 MiB/s wr, 219 op/s 2024-09-10T10:03:32.100 INFO:tasks.workunit.client.0.smithi005.stdout: 10 177041 13.52 MB/sec execute 550 sec latency 19126.553 ms 2024-09-10T10:03:33.101 INFO:tasks.workunit.client.0.smithi005.stdout: 10 177041 13.50 MB/sec execute 551 sec latency 20126.627 ms 2024-09-10T10:03:33.420 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:03:33 smithi005 ceph-mon[30350]: pgmap v1149: 129 pgs: 129 active+clean; 790 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 184 KiB/s rd, 5.1 MiB/s wr, 201 op/s 2024-09-10T10:03:33.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:33.575+0000 7f7329200640 1 -- 172.21.15.5:0/3394478582 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7324074510 msgr2=0x7f73240748f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:33.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:33.575+0000 7f7329200640 1 --2- 172.21.15.5:0/3394478582 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7324074510 0x7f73240748f0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f7314009900 tx=0x7f731402a480 comp rx=0 tx=0).stop 2024-09-10T10:03:33.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:33.575+0000 7f7329200640 1 -- 172.21.15.5:0/3394478582 shutdown_connections 2024-09-10T10:03:33.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:33.575+0000 7f7329200640 1 --2- 172.21.15.5:0/3394478582 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f732410e280 0x7f7324110670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:33.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:33.575+0000 7f7329200640 1 --2- 172.21.15.5:0/3394478582 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7324074e30 0x7f732410dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:33.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:33.575+0000 7f7329200640 1 --2- 172.21.15.5:0/3394478582 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7324074510 0x7f73240748f0 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:33.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:33.575+0000 7f7329200640 1 -- 172.21.15.5:0/3394478582 >> 172.21.15.5:0/3394478582 conn(0x7f73240fd740 msgr2=0x7f73240ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:33.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:33.576+0000 7f7329200640 1 -- 172.21.15.5:0/3394478582 shutdown_connections 2024-09-10T10:03:33.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:33.576+0000 7f7329200640 1 -- 172.21.15.5:0/3394478582 wait complete. 2024-09-10T10:03:33.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:33.577+0000 7f7329200640 1 Processor -- start 2024-09-10T10:03:33.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:33.577+0000 7f7329200640 1 -- start start 2024-09-10T10:03:33.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:33.577+0000 7f7329200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7324074510 0x7f732419a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:33.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:33.577+0000 7f7329200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7324074e30 0x7f732419aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:33.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:33.577+0000 7f7329200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f732410e280 0x7f732419ee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:33.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:33.577+0000 7f7329200640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f7324113fc0 con 0x7f732410e280 2024-09-10T10:03:33.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:33.577+0000 7f7329200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f7324113e40 con 0x7f7324074510 2024-09-10T10:03:33.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:33.577+0000 7f7329200640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f7324114140 con 0x7f7324074e30 2024-09-10T10:03:33.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:33.578+0000 7f7323400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7324074510 0x7f732419a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:33.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:33.578+0000 7f7323400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7324074510 0x7f732419a540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:53998/0 (socket says 172.21.15.5:53998) 2024-09-10T10:03:33.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:33.578+0000 7f7323400640 1 -- 172.21.15.5:0/3015218661 learned_addr learned my addr 172.21.15.5:0/3015218661 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T10:03:33.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:33.578+0000 7f7323e00640 1 --2- 172.21.15.5:0/3015218661 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f732410e280 0x7f732419ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:33.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:33.578+0000 7f7322a00640 1 --2- 172.21.15.5:0/3015218661 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7324074e30 0x7f732419aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:33.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:33.578+0000 7f7323400640 1 -- 172.21.15.5:0/3015218661 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7324074e30 msgr2=0x7f732419aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:33.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:33.578+0000 7f7323400640 1 --2- 172.21.15.5:0/3015218661 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7324074e30 0x7f732419aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:33.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:33.578+0000 7f7323400640 1 -- 172.21.15.5:0/3015218661 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f732410e280 msgr2=0x7f732419ee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:33.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:33.578+0000 7f7323400640 1 --2- 172.21.15.5:0/3015218661 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f732410e280 0x7f732419ee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:33.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:33.578+0000 7f7323400640 1 -- 172.21.15.5:0/3015218661 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f732419f4f0 con 0x7f7324074510 2024-09-10T10:03:33.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:33.578+0000 7f7323400640 1 --2- 172.21.15.5:0/3015218661 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7324074510 0x7f732419a540 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f7314004770 tx=0x7f7314003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:33.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:33.579+0000 7f731be00640 1 -- 172.21.15.5:0/3015218661 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f731403f070 con 0x7f7324074510 2024-09-10T10:03:33.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:33.579+0000 7f7329200640 1 -- 172.21.15.5:0/3015218661 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f732419f780 con 0x7f7324074510 2024-09-10T10:03:33.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:33.579+0000 7f731be00640 1 -- 172.21.15.5:0/3015218661 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7314031ca0 con 0x7f7324074510 2024-09-10T10:03:33.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:33.579+0000 7f731be00640 1 -- 172.21.15.5:0/3015218661 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7314035030 con 0x7f7324074510 2024-09-10T10:03:33.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:33.579+0000 7f7329200640 1 -- 172.21.15.5:0/3015218661 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73241a70c0 con 0x7f7324074510 2024-09-10T10:03:33.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:33.582+0000 7f732d1e0740 1 Processor -- start 2024-09-10T10:03:33.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:33.582+0000 7f731be00640 1 -- 172.21.15.5:0/3015218661 <== mon.2 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f7314042070 con 0x7f7324074510 2024-09-10T10:03:33.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:33.582+0000 7f732d1e0740 1 -- start start 2024-09-10T10:03:33.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:33.583+0000 7f732d1e0740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5640698dce40 0x5640698dd220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:33.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:33.583+0000 7f732d1e0740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5640698dd760 0x5640698e7ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:33.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:33.583+0000 7f732d1e0740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5640698e8020 0x5640698ea410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:33.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:33.583+0000 7f732d1e0740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5640698433e0 con 0x5640698dce40 2024-09-10T10:03:33.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:33.583+0000 7f732d1e0740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x5640697ead80 con 0x5640698dd760 2024-09-10T10:03:33.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:33.583+0000 7f732d1e0740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x56406983f3a0 con 0x5640698e8020 2024-09-10T10:03:33.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:33.583+0000 7f7323400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5640698dd760 0x5640698e7ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:33.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:33.583+0000 7f7322a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5640698e8020 0x5640698ea410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:33.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:33.583+0000 7f7323400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5640698dd760 0x5640698e7ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:54012/0 (socket says 172.21.15.5:54012) 2024-09-10T10:03:33.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:33.583+0000 7f7323400640 1 -- 172.21.15.5:0/4021670447 learned_addr learned my addr 172.21.15.5:0/4021670447 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T10:03:33.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:33.583+0000 7f7323e00640 1 --2- 172.21.15.5:0/4021670447 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5640698dce40 0x5640698dd220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:33.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:33.584+0000 7f7322a00640 1 -- 172.21.15.5:0/4021670447 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5640698dd760 msgr2=0x5640698e7ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:03:33.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:33.584+0000 7f7322a00640 1 --2- 172.21.15.5:0/4021670447 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5640698dd760 0x5640698e7ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:33.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:33.584+0000 7f7322a00640 1 -- 172.21.15.5:0/4021670447 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5640698dce40 msgr2=0x5640698dd220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:03:33.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:33.584+0000 7f7322a00640 1 --2- 172.21.15.5:0/4021670447 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5640698dce40 0x5640698dd220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:33.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:33.584+0000 7f7322a00640 1 -- 172.21.15.5:0/4021670447 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564069809ae0 con 0x5640698e8020 2024-09-10T10:03:33.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:33.584+0000 7f7322a00640 1 --2- 172.21.15.5:0/4021670447 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5640698e8020 0x5640698ea410 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f7310009bf0 tx=0x7f7310011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=144951b8e203a0f0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:33.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:33.584+0000 7f731be00640 1 --2- 172.21.15.5:0/3015218661 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f73080a2b80 0x7f73080a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:33.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:33.584+0000 7f7318c00640 1 -- 172.21.15.5:0/4021670447 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7310004050 con 0x5640698e8020 2024-09-10T10:03:33.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:33.585+0000 7f731be00640 1 -- 172.21.15.5:0/3015218661 <== mon.2 v2:172.21.15.102:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f73140efa70 con 0x7f7324074510 2024-09-10T10:03:33.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:33.585+0000 7f7318c00640 1 -- 172.21.15.5:0/4021670447 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7310005500 con 0x5640698e8020 2024-09-10T10:03:33.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:33.585+0000 7f7318c00640 1 -- 172.21.15.5:0/4021670447 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73100035f0 con 0x5640698e8020 2024-09-10T10:03:33.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:33.585+0000 7f7323e00640 1 --2- 172.21.15.5:0/3015218661 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f73080a2b80 0x7f73080a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:33.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:33.585+0000 7f732d1e0740 1 -- 172.21.15.5:0/4021670447 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5640698e8020 msgr2=0x5640698ea410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:33.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:33.585+0000 7f732d1e0740 1 --2- 172.21.15.5:0/4021670447 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5640698e8020 0x5640698ea410 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f7310009bf0 tx=0x7f7310011810 comp rx=0 tx=0).stop 2024-09-10T10:03:33.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:33.586+0000 7f732d1e0740 1 -- 172.21.15.5:0/4021670447 shutdown_connections 2024-09-10T10:03:33.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:33.586+0000 7f732d1e0740 1 --2- 172.21.15.5:0/4021670447 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5640698e8020 0x5640698ea410 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:33.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:33.586+0000 7f732d1e0740 1 --2- 172.21.15.5:0/4021670447 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5640698dd760 0x5640698e7ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:33.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:33.586+0000 7f732d1e0740 1 --2- 172.21.15.5:0/4021670447 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5640698dce40 0x5640698dd220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:33.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:33.586+0000 7f732d1e0740 1 -- 172.21.15.5:0/4021670447 >> 172.21.15.5:0/4021670447 conn(0x5640698d93d0 msgr2=0x5640698d97b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:33.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:33.586+0000 7f7323e00640 1 --2- 172.21.15.5:0/3015218661 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f73080a2b80 0x7f73080a5040 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f73240754f0 tx=0x7f731c009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:33.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:33.586+0000 7f732d1e0740 1 -- 172.21.15.5:0/4021670447 shutdown_connections 2024-09-10T10:03:33.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:33.586+0000 7f732d1e0740 1 -- 172.21.15.5:0/4021670447 wait complete. 2024-09-10T10:03:33.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:33.586+0000 7f732d1e0740 1 -- start start 2024-09-10T10:03:33.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:33.587+0000 7f732d1e0740 1 Processor -- start 2024-09-10T10:03:33.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:33.587+0000 7f732d1e0740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5640698dce40 0x5640698e4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:33.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:33.587+0000 7f732d1e0740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5640698dd760 0x5640698e5400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:33.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:33.587+0000 7f732d1e0740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5640698e8020 0x5640698e5940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:33.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:33.587+0000 7f732d1e0740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5640698eabf0 con 0x5640698e8020 2024-09-10T10:03:33.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:33.587+0000 7f732d1e0740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x5640697fe0f0 con 0x5640698dce40 2024-09-10T10:03:33.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:33.587+0000 7f732d1e0740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5640698ead70 con 0x5640698dd760 2024-09-10T10:03:33.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:33.587+0000 7f732d1e0740 10 client.0 _ll_register_callbacks cb 0x5640697e5bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:03:33.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:33.587+0000 7f7323400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5640698dce40 0x5640698e4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:33.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:33.587+0000 7f7322a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5640698dd760 0x5640698e5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:33.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:33.587+0000 7f7323e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5640698e8020 0x5640698e5940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:33.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:33.587+0000 7f7323400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5640698dce40 0x5640698e4ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:54028/0 (socket says 172.21.15.5:54028) 2024-09-10T10:03:33.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:33.588+0000 7f7322a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5640698dd760 0x5640698e5400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:45748/0 (socket says 172.21.15.5:45748) 2024-09-10T10:03:33.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:33.588+0000 7f7323400640 1 -- 172.21.15.5:0/3818211211 learned_addr learned my addr 172.21.15.5:0/3818211211 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T10:03:33.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:33.588+0000 7f7323400640 1 -- 172.21.15.5:0/3818211211 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5640698dd760 msgr2=0x5640698e5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:03:33.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:33.588+0000 7f7323400640 1 --2- 172.21.15.5:0/3818211211 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5640698dd760 0x5640698e5400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:33.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:33.588+0000 7f7323400640 1 -- 172.21.15.5:0/3818211211 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5640698e8020 msgr2=0x5640698e5940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:03:33.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:33.588+0000 7f7323400640 1 --2- 172.21.15.5:0/3818211211 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5640698e8020 0x5640698e5940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:33.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:33.588+0000 7f7323400640 1 -- 172.21.15.5:0/3818211211 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5640695b43c0 con 0x5640698dce40 2024-09-10T10:03:33.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:33.588+0000 7f7323400640 1 --2- 172.21.15.5:0/3818211211 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5640698dce40 0x5640698e4ec0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f73140c4680 tx=0x7f73140b8350 comp rx=0 tx=0).ready entity=mon.2 client_cookie=deff1abd006fe6dc server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:33.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:33.588+0000 7f730d600640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-09-10T10:03:33.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:33.588+0000 7f730d600640 1 -- 172.21.15.5:0/3818211211 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73140efe60 con 0x5640698dce40 2024-09-10T10:03:33.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:33.589+0000 7f730d600640 1 -- 172.21.15.5:0/3818211211 <== mon.2 v2:172.21.15.102:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f73140b8660 con 0x5640698dce40 2024-09-10T10:03:33.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:33.589+0000 7f732d1e0740 1 -- 172.21.15.5:0/3818211211 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7310005500 con 0x5640698dce40 2024-09-10T10:03:33.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:33.589+0000 7f730d600640 1 -- 172.21.15.5:0/3818211211 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f7314049070 con 0x5640698dce40 2024-09-10T10:03:33.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:33.589+0000 7f732d1e0740 10 client.30119 fetch_fsmap learned FSMap version 42 2024-09-10T10:03:33.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:33.589+0000 7f732d1e0740 1 -- 172.21.15.5:0/3818211211 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x5640698f6be0 con 0x5640698dce40 2024-09-10T10:03:33.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:33.589+0000 7f730d600640 1 -- 172.21.15.5:0/3818211211 <== mon.2 v2:172.21.15.102:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7310005500 con 0x5640698dce40 2024-09-10T10:03:33.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:33.589+0000 7f730d600640 1 -- 172.21.15.5:0/3818211211 <== mon.2 v2:172.21.15.102:3300/0 5 ==== fsmap(e 42) ==== 8370+0+0 (secure 0 0 0) 0x7f731403b4f0 con 0x5640698dce40 2024-09-10T10:03:33.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:33.589+0000 7f732d1e0740 10 client.30119 fetch_fsmap finished waiting for FSMap version 42 2024-09-10T10:03:33.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:33.589+0000 7f732d1e0740 10 client.30119 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-10T10:03:33.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:33.589+0000 7f732d1e0740 20 client.30119 populate_metadata read hostname 'smithi005' 2024-09-10T10:03:33.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:33.589+0000 7f732d1e0740 1 --2- 172.21.15.5:0/3818211211 >> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] conn(0x5640698f8090 0x5640698fa500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:33.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:33.589+0000 7f732d1e0740 4 client.30119 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:03:33.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:33.589+0000 7f732d1e0740 1 -- 172.21.15.5:0/3818211211 --> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564069850950 con 0x5640698f8090 2024-09-10T10:03:33.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:33.590+0000 7f7322a00640 1 --2- 172.21.15.5:0/3818211211 >> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] conn(0x5640698f8090 0x5640698fa500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:33.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:33.591+0000 7f7322a00640 1 --2- 172.21.15.5:0/3818211211 >> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] conn(0x5640698f8090 0x5640698fa500 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c1a9efe2f0e3961c server_cookie=cfe890171af8d166 in_seq=0 out_seq=0 2024-09-10T10:03:33.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:33.591+0000 7f730d600640 10 client.30119 ms_handle_connect on v2:172.21.15.159:6836/620627639 2024-09-10T10:03:33.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:33.591+0000 7f730d600640 1 -- 172.21.15.5:0/3818211211 <== mds.2 v2:172.21.15.159:6836/620627639 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564069850950 con 0x5640698f8090 2024-09-10T10:03:33.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:33.591+0000 7f730d600640 10 client.30119 handle_command_reply: tid=0 2024-09-10T10:03:33.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:33.592+0000 7f732d1e0740 1 client.30119 shutdown 2024-09-10T10:03:33.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:33.592+0000 7f732d1e0740 1 -- 172.21.15.5:0/3818211211 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5640698dce40 msgr2=0x5640698e4ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:33.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:33.592+0000 7f732d1e0740 1 --2- 172.21.15.5:0/3818211211 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5640698dce40 0x5640698e4ec0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f73140c4680 tx=0x7f73140b8350 comp rx=0 tx=0).stop 2024-09-10T10:03:33.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:33.592+0000 7f732d1e0740 1 -- 172.21.15.5:0/3818211211 shutdown_connections 2024-09-10T10:03:33.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:33.592+0000 7f732d1e0740 1 --2- 172.21.15.5:0/3818211211 >> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] conn(0x5640698f8090 0x5640698fa500 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:33.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:33.592+0000 7f732d1e0740 1 --2- 172.21.15.5:0/3818211211 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5640698e8020 0x5640698e5940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:33.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:33.592+0000 7f732d1e0740 1 --2- 172.21.15.5:0/3818211211 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5640698dd760 0x5640698e5400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:33.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:33.592+0000 7f732d1e0740 1 --2- 172.21.15.5:0/3818211211 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5640698dce40 0x5640698e4ec0 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:33.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:33.592+0000 7f730d600640 0 client.30119 ms_handle_reset on v2:172.21.15.159:6836/620627639 2024-09-10T10:03:33.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:33.592+0000 7f732d1e0740 1 -- 172.21.15.5:0/3818211211 >> 172.21.15.5:0/3818211211 conn(0x5640698d93d0 msgr2=0x5640698df100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:33.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:33.592+0000 7f732d1e0740 1 -- 172.21.15.5:0/3818211211 shutdown_connections 2024-09-10T10:03:33.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:33.592+0000 7f732d1e0740 1 -- 172.21.15.5:0/3818211211 wait complete. 2024-09-10T10:03:33.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:33.592+0000 7f732d1e0740 20 client.30119 trim_cache size 0 max 16384 2024-09-10T10:03:33.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:03:33 smithi159 ceph-mon[35513]: pgmap v1149: 129 pgs: 129 active+clean; 790 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 184 KiB/s rd, 5.1 MiB/s wr, 201 op/s 2024-09-10T10:03:33.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:33.607+0000 7f732d1e0740 1 Processor -- start 2024-09-10T10:03:33.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:33.607+0000 7f732d1e0740 1 -- start start 2024-09-10T10:03:33.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:33.607+0000 7f732d1e0740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5640698dce40 0x5640698df8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:33.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:33.607+0000 7f732d1e0740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5640698dd760 0x5640698ffdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:33.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:33.607+0000 7f732d1e0740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5640698e8020 0x564069900310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:33.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:33.607+0000 7f732d1e0740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f73241b0ef0 con 0x5640698dce40 2024-09-10T10:03:33.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:33.607+0000 7f732d1e0740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f73241b2a30 con 0x5640698e8020 2024-09-10T10:03:33.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:33.607+0000 7f732d1e0740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f73241b15c0 con 0x5640698dd760 2024-09-10T10:03:33.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:33.607+0000 7f7323400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5640698dce40 0x5640698df8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:33.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:33.607+0000 7f7322a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5640698dd760 0x5640698ffdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:33.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:33.608+0000 7f7322a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5640698dd760 0x5640698ffdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:45750/0 (socket says 172.21.15.5:45750) 2024-09-10T10:03:33.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:33.608+0000 7f7322a00640 1 -- 172.21.15.5:0/2687576691 learned_addr learned my addr 172.21.15.5:0/2687576691 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T10:03:33.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:33.608+0000 7f7323e00640 1 --2- 172.21.15.5:0/2687576691 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5640698e8020 0x564069900310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:33.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:33.608+0000 7f7322a00640 1 -- 172.21.15.5:0/2687576691 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5640698e8020 msgr2=0x564069900310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:03:33.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:33.608+0000 7f7322a00640 1 --2- 172.21.15.5:0/2687576691 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5640698e8020 0x564069900310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:33.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:33.608+0000 7f7322a00640 1 -- 172.21.15.5:0/2687576691 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5640698dce40 msgr2=0x5640698df8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:03:33.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:33.608+0000 7f7322a00640 1 --2- 172.21.15.5:0/2687576691 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5640698dce40 0x5640698df8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:33.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:33.608+0000 7f7322a00640 1 -- 172.21.15.5:0/2687576691 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5640698dcc70 con 0x5640698dd760 2024-09-10T10:03:33.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:33.608+0000 7f7323400640 1 --2- 172.21.15.5:0/2687576691 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5640698dce40 0x5640698df8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:03:33.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:33.608+0000 7f7322a00640 1 --2- 172.21.15.5:0/2687576691 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5640698dd760 0x5640698ffdd0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f7310009bf0 tx=0x7f7310004b20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9ee31b4d24a0fb6d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:33.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:33.609+0000 7f730fe00640 1 -- 172.21.15.5:0/2687576691 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7310004250 con 0x5640698dd760 2024-09-10T10:03:33.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:33.609+0000 7f730fe00640 1 -- 172.21.15.5:0/2687576691 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f73100043f0 con 0x5640698dd760 2024-09-10T10:03:33.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:33.609+0000 7f730fe00640 1 -- 172.21.15.5:0/2687576691 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7310019430 con 0x5640698dd760 2024-09-10T10:03:33.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:33.609+0000 7f732d1e0740 1 -- 172.21.15.5:0/2687576691 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5640698dd760 msgr2=0x5640698ffdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:33.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:33.609+0000 7f732d1e0740 1 --2- 172.21.15.5:0/2687576691 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5640698dd760 0x5640698ffdd0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f7310009bf0 tx=0x7f7310004b20 comp rx=0 tx=0).stop 2024-09-10T10:03:33.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:33.609+0000 7f732d1e0740 1 -- 172.21.15.5:0/2687576691 shutdown_connections 2024-09-10T10:03:33.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:33.609+0000 7f732d1e0740 1 --2- 172.21.15.5:0/2687576691 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5640698e8020 0x564069900310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:33.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:33.609+0000 7f732d1e0740 1 --2- 172.21.15.5:0/2687576691 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5640698dd760 0x5640698ffdd0 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:33.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:33.609+0000 7f732d1e0740 1 --2- 172.21.15.5:0/2687576691 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5640698dce40 0x5640698df8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:33.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:33.609+0000 7f732d1e0740 1 -- 172.21.15.5:0/2687576691 >> 172.21.15.5:0/2687576691 conn(0x5640698d93d0 msgr2=0x5640698e6f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:33.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:33.610+0000 7f732d1e0740 1 -- 172.21.15.5:0/2687576691 shutdown_connections 2024-09-10T10:03:33.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:33.610+0000 7f732d1e0740 1 -- 172.21.15.5:0/2687576691 wait complete. 2024-09-10T10:03:33.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:33.610+0000 7f732d1e0740 1 -- start start 2024-09-10T10:03:33.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:33.610+0000 7f732d1e0740 1 Processor -- start 2024-09-10T10:03:33.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:33.611+0000 7f732d1e0740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5640698dce40 0x5640698f08e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:33.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:33.611+0000 7f732d1e0740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5640698dd760 0x5640698ebc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:33.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:33.611+0000 7f732d1e0740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5640698e8020 0x5640698ec3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:33.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:33.611+0000 7f732d1e0740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f73241b2360 con 0x5640698dd760 2024-09-10T10:03:33.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:33.611+0000 7f732d1e0740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f73241b1c90 con 0x5640698dce40 2024-09-10T10:03:33.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:33.611+0000 7f732d1e0740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f73241aeed0 con 0x5640698e8020 2024-09-10T10:03:33.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:33.611+0000 7f732d1e0740 10 client.0 _ll_register_callbacks cb 0x7f73140f5200 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:03:33.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:33.611+0000 7f7323400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5640698dce40 0x5640698f08e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:33.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:33.611+0000 7f7323400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5640698dce40 0x5640698f08e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:54052/0 (socket says 172.21.15.5:54052) 2024-09-10T10:03:33.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:33.611+0000 7f7323400640 1 -- 172.21.15.5:0/2657895428 learned_addr learned my addr 172.21.15.5:0/2657895428 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T10:03:33.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:33.611+0000 7f7322a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5640698dd760 0x5640698ebc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:33.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:33.611+0000 7f7323e00640 1 --2- 172.21.15.5:0/2657895428 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5640698e8020 0x5640698ec3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:33.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:33.612+0000 7f7323400640 1 -- 172.21.15.5:0/2657895428 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5640698e8020 msgr2=0x5640698ec3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:03:33.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:33.612+0000 7f7323400640 1 --2- 172.21.15.5:0/2657895428 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5640698e8020 0x5640698ec3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:33.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:33.612+0000 7f7323400640 1 -- 172.21.15.5:0/2657895428 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5640698dd760 msgr2=0x5640698ebc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:03:33.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:33.612+0000 7f7323400640 1 --2- 172.21.15.5:0/2657895428 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5640698dd760 0x5640698ebc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:33.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:33.612+0000 7f7323400640 1 -- 172.21.15.5:0/2657895428 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564069961da0 con 0x5640698dce40 2024-09-10T10:03:33.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:33.612+0000 7f7323400640 1 --2- 172.21.15.5:0/2657895428 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5640698dce40 0x5640698f08e0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f73140efda0 tx=0x7f73140ba500 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e38c1917f75a447b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:33.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:33.612+0000 7f730d600640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-09-10T10:03:33.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:33.612+0000 7f730d600640 1 -- 172.21.15.5:0/2657895428 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73140ab060 con 0x5640698dce40 2024-09-10T10:03:33.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:33.612+0000 7f730d600640 1 -- 172.21.15.5:0/2657895428 <== mon.2 v2:172.21.15.102:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f73140b6b60 con 0x5640698dce40 2024-09-10T10:03:33.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:33.612+0000 7f732d1e0740 1 -- 172.21.15.5:0/2657895428 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f73100043f0 con 0x5640698dce40 2024-09-10T10:03:33.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:33.612+0000 7f730d600640 1 -- 172.21.15.5:0/2657895428 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f73140b2050 con 0x5640698dce40 2024-09-10T10:03:33.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:33.612+0000 7f732d1e0740 10 client.30125 fetch_fsmap learned FSMap version 42 2024-09-10T10:03:33.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:33.612+0000 7f732d1e0740 1 -- 172.21.15.5:0/2657895428 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x564069962f80 con 0x5640698dce40 2024-09-10T10:03:33.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:33.613+0000 7f730d600640 1 -- 172.21.15.5:0/2657895428 <== mon.2 v2:172.21.15.102:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f73100043f0 con 0x5640698dce40 2024-09-10T10:03:33.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:33.613+0000 7f730d600640 1 -- 172.21.15.5:0/2657895428 <== mon.2 v2:172.21.15.102:3300/0 5 ==== fsmap(e 42) ==== 8370+0+0 (secure 0 0 0) 0x7f73140bb440 con 0x5640698dce40 2024-09-10T10:03:33.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:33.613+0000 7f732d1e0740 10 client.30125 fetch_fsmap finished waiting for FSMap version 42 2024-09-10T10:03:33.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:33.613+0000 7f732d1e0740 10 client.30125 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-10T10:03:33.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:33.613+0000 7f732d1e0740 20 client.30125 populate_metadata read hostname 'smithi005' 2024-09-10T10:03:33.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:33.613+0000 7f732d1e0740 1 --2- 172.21.15.5:0/2657895428 >> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] conn(0x564069963510 0x564069965900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:33.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:33.613+0000 7f732d1e0740 4 client.30125 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T10:03:33.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:33.613+0000 7f732d1e0740 1 -- 172.21.15.5:0/2657895428 --> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56406984ae60 con 0x564069963510 2024-09-10T10:03:33.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:33.613+0000 7f7322a00640 1 --2- 172.21.15.5:0/2657895428 >> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] conn(0x564069963510 0x564069965900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:33.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:33.614+0000 7f7322a00640 1 --2- 172.21.15.5:0/2657895428 >> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] conn(0x564069963510 0x564069965900 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a9608175582705bf server_cookie=873d7fdc1036dfdc in_seq=0 out_seq=0 2024-09-10T10:03:33.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:33.614+0000 7f730d600640 10 client.30125 ms_handle_connect on v2:172.21.15.159:6836/620627639 2024-09-10T10:03:33.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:33.614+0000 7f730d600640 1 -- 172.21.15.5:0/2657895428 <== mds.2 v2:172.21.15.159:6836/620627639 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56406984ae60 con 0x564069963510 2024-09-10T10:03:33.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:33.614+0000 7f730d600640 10 client.30125 handle_command_reply: tid=0 2024-09-10T10:03:33.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:33.614+0000 7f732d1e0740 1 client.30125 shutdown 2024-09-10T10:03:33.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:33.614+0000 7f732d1e0740 1 -- 172.21.15.5:0/2657895428 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5640698dce40 msgr2=0x5640698f08e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:33.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:33.614+0000 7f732d1e0740 1 --2- 172.21.15.5:0/2657895428 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5640698dce40 0x5640698f08e0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f73140efda0 tx=0x7f73140ba500 comp rx=0 tx=0).stop 2024-09-10T10:03:33.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:33.614+0000 7f732d1e0740 1 -- 172.21.15.5:0/2657895428 shutdown_connections 2024-09-10T10:03:33.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:33.614+0000 7f732d1e0740 1 --2- 172.21.15.5:0/2657895428 >> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] conn(0x564069963510 0x564069965900 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:33.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:33.614+0000 7f732d1e0740 1 --2- 172.21.15.5:0/2657895428 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5640698e8020 0x5640698ec3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:33.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:33.614+0000 7f732d1e0740 1 --2- 172.21.15.5:0/2657895428 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5640698dd760 0x5640698ebc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:33.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:33.614+0000 7f732d1e0740 1 --2- 172.21.15.5:0/2657895428 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5640698dce40 0x5640698f08e0 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:33.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:33.614+0000 7f730d600640 0 client.30125 ms_handle_reset on v2:172.21.15.159:6836/620627639 2024-09-10T10:03:33.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:33.614+0000 7f732d1e0740 1 -- 172.21.15.5:0/2657895428 >> 172.21.15.5:0/2657895428 conn(0x5640698d93d0 msgr2=0x5640698e6f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:33.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:33.615+0000 7f732d1e0740 1 -- 172.21.15.5:0/2657895428 shutdown_connections 2024-09-10T10:03:33.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:33.615+0000 7f732d1e0740 1 -- 172.21.15.5:0/2657895428 wait complete. 2024-09-10T10:03:33.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:33.615+0000 7f732d1e0740 20 client.30125 trim_cache size 0 max 16384 2024-09-10T10:03:33.620 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-10T10:03:33.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:33.616+0000 7f7329200640 1 -- 172.21.15.5:0/3015218661 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f73080a2b80 msgr2=0x7f73080a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:33.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:33.616+0000 7f7329200640 1 --2- 172.21.15.5:0/3015218661 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f73080a2b80 0x7f73080a5040 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f73240754f0 tx=0x7f731c009500 comp rx=0 tx=0).stop 2024-09-10T10:03:33.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:33.616+0000 7f7329200640 1 -- 172.21.15.5:0/3015218661 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7324074510 msgr2=0x7f732419a540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:33.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:33.616+0000 7f7329200640 1 --2- 172.21.15.5:0/3015218661 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7324074510 0x7f732419a540 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f7314004770 tx=0x7f7314003c10 comp rx=0 tx=0).stop 2024-09-10T10:03:33.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:33.616+0000 7f7329200640 1 -- 172.21.15.5:0/3015218661 shutdown_connections 2024-09-10T10:03:33.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:33.616+0000 7f7329200640 1 --2- 172.21.15.5:0/3015218661 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f73080a2b80 0x7f73080a5040 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:33.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:33.616+0000 7f7329200640 1 --2- 172.21.15.5:0/3015218661 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f732410e280 0x7f732419ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:33.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:33.616+0000 7f7329200640 1 --2- 172.21.15.5:0/3015218661 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7324074e30 0x7f732419aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:33.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:33.616+0000 7f7329200640 1 --2- 172.21.15.5:0/3015218661 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7324074510 0x7f732419a540 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:33.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:33.616+0000 7f7329200640 1 -- 172.21.15.5:0/3015218661 >> 172.21.15.5:0/3015218661 conn(0x7f73240fd740 msgr2=0x7f73240fe440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:33.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:33.616+0000 7f7329200640 1 -- 172.21.15.5:0/3015218661 shutdown_connections 2024-09-10T10:03:33.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:33.616+0000 7f7329200640 1 -- 172.21.15.5:0/3015218661 wait complete. 2024-09-10T10:03:33.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:03:33 smithi102 ceph-mon[35442]: pgmap v1149: 129 pgs: 129 active+clean; 790 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 184 KiB/s rd, 5.1 MiB/s wr, 201 op/s 2024-09-10T10:03:34.101 INFO:tasks.workunit.client.0.smithi005.stdout: 10 177041 13.47 MB/sec execute 552 sec latency 21127.603 ms 2024-09-10T10:03:34.275 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T10:03:34.659 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-10T10:03:34.737 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:03:35.102 INFO:tasks.workunit.client.0.smithi005.stdout: 10 177041 13.45 MB/sec execute 553 sec latency 22127.659 ms 2024-09-10T10:03:35.396 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:03:35 smithi005 ceph-mon[30350]: pgmap v1150: 129 pgs: 129 active+clean; 783 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 187 KiB/s rd, 4.8 MiB/s wr, 207 op/s 2024-09-10T10:03:35.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:35.414+0000 7fab1a400640 1 -- 172.21.15.5:0/1574479168 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fab0c09b600 msgr2=0x7fab0c09b9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:35.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:35.414+0000 7fab1a400640 1 --2- 172.21.15.5:0/1574479168 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fab0c09b600 0x7fab0c09b9e0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7fab08009900 tx=0x7fab0802a420 comp rx=0 tx=0).stop 2024-09-10T10:03:35.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:35.414+0000 7fab1a400640 1 -- 172.21.15.5:0/1574479168 shutdown_connections 2024-09-10T10:03:35.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:35.414+0000 7fab1a400640 1 --2- 172.21.15.5:0/1574479168 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fab0c005620 0x7fab0c0a2420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:35.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:35.414+0000 7fab1a400640 1 --2- 172.21.15.5:0/1574479168 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fab0c004c80 0x7fab0c0050e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:35.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:35.414+0000 7fab1a400640 1 --2- 172.21.15.5:0/1574479168 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fab0c09b600 0x7fab0c09b9e0 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:35.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:35.414+0000 7fab1a400640 1 -- 172.21.15.5:0/1574479168 >> 172.21.15.5:0/1574479168 conn(0x7fab0c00a1f0 msgr2=0x7fab0c00a600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:35.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:35.415+0000 7fab1a400640 1 -- 172.21.15.5:0/1574479168 shutdown_connections 2024-09-10T10:03:35.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:35.415+0000 7fab1a400640 1 -- 172.21.15.5:0/1574479168 wait complete. 2024-09-10T10:03:35.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:35.417+0000 7fab1a400640 1 Processor -- start 2024-09-10T10:03:35.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:35.419+0000 7fab1a400640 1 -- start start 2024-09-10T10:03:35.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:35.420+0000 7fab1a400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fab0c004c80 0x7fab0c132970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:35.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:35.420+0000 7fab1a400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fab0c005620 0x7fab0c132eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:35.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:35.420+0000 7fab1a400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fab0c09b600 0x7fab0c0a1540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:35.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:35.421+0000 7fab1a400640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fab0c0a59b0 con 0x7fab0c005620 2024-09-10T10:03:35.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:35.421+0000 7fab1a400640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fab0c0a5830 con 0x7fab0c004c80 2024-09-10T10:03:35.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:35.421+0000 7fab1a400640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fab0c0a5b30 con 0x7fab0c09b600 2024-09-10T10:03:35.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:35.421+0000 7fab13e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fab0c005620 0x7fab0c132eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:35.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:35.421+0000 7fab19a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fab0c09b600 0x7fab0c0a1540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:35.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:35.421+0000 7fab19a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fab0c09b600 0x7fab0c0a1540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:45768/0 (socket says 172.21.15.5:45768) 2024-09-10T10:03:35.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:35.421+0000 7fab19a00640 1 -- 172.21.15.5:0/1485904115 learned_addr learned my addr 172.21.15.5:0/1485904115 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T10:03:35.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:35.422+0000 7fab19a00640 1 -- 172.21.15.5:0/1485904115 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fab0c004c80 msgr2=0x7fab0c132970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:35.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:35.422+0000 7fab19a00640 1 --2- 172.21.15.5:0/1485904115 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fab0c004c80 0x7fab0c132970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:35.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:35.422+0000 7fab19a00640 1 -- 172.21.15.5:0/1485904115 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fab0c005620 msgr2=0x7fab0c132eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:35.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:35.422+0000 7fab19a00640 1 --2- 172.21.15.5:0/1485904115 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fab0c005620 0x7fab0c132eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:35.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:35.422+0000 7fab19a00640 1 -- 172.21.15.5:0/1485904115 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab0c0a1e00 con 0x7fab0c09b600 2024-09-10T10:03:35.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:35.423+0000 7fab19a00640 1 --2- 172.21.15.5:0/1485904115 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fab0c09b600 0x7fab0c0a1540 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7fab140518b0 tx=0x7fab140747f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:35.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:35.424+0000 7fab11600640 1 -- 172.21.15.5:0/1485904115 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab1407f430 con 0x7fab0c09b600 2024-09-10T10:03:35.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:35.425+0000 7fab11600640 1 -- 172.21.15.5:0/1485904115 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fab14084020 con 0x7fab0c09b600 2024-09-10T10:03:35.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:35.425+0000 7fab1a400640 1 -- 172.21.15.5:0/1485904115 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab0c0a2090 con 0x7fab0c09b600 2024-09-10T10:03:35.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:35.425+0000 7fab1a400640 1 -- 172.21.15.5:0/1485904115 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab0c12e4b0 con 0x7fab0c09b600 2024-09-10T10:03:35.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:35.425+0000 7fab11600640 1 -- 172.21.15.5:0/1485904115 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab1407e820 con 0x7fab0c09b600 2024-09-10T10:03:35.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:35.427+0000 7fab1a400640 1 -- 172.21.15.5:0/1485904115 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faaf4005290 con 0x7fab0c09b600 2024-09-10T10:03:35.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:35.435+0000 7fab11600640 1 -- 172.21.15.5:0/1485904115 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7fab1407e9c0 con 0x7fab0c09b600 2024-09-10T10:03:35.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:35.437+0000 7fab11600640 1 --2- 172.21.15.5:0/1485904115 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7faafc0a2ce0 0x7faafc0a51a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:35.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:35.437+0000 7fab11600640 1 -- 172.21.15.5:0/1485904115 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fab1413d0b0 con 0x7fab0c09b600 2024-09-10T10:03:35.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:35.438+0000 7fab11600640 1 -- 172.21.15.5:0/1485904115 <== mon.1 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fab1413d570 con 0x7fab0c09b600 2024-09-10T10:03:35.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:35.438+0000 7fab19000640 1 --2- 172.21.15.5:0/1485904115 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7faafc0a2ce0 0x7faafc0a51a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:35.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:35.439+0000 7fab19000640 1 --2- 172.21.15.5:0/1485904115 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7faafc0a2ce0 0x7faafc0a51a0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fab080096c0 tx=0x7fab08000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:35.508 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:03:35.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:03:35 smithi159 ceph-mon[35513]: pgmap v1150: 129 pgs: 129 active+clean; 783 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 187 KiB/s rd, 4.8 MiB/s wr, 207 op/s 2024-09-10T10:03:35.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:03:35 smithi102 ceph-mon[35442]: pgmap v1150: 129 pgs: 129 active+clean; 783 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 187 KiB/s rd, 4.8 MiB/s wr, 207 op/s 2024-09-10T10:03:35.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:35.712+0000 7fab1a400640 1 -- 172.21.15.5:0/1485904115 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7faaf4005580 con 0x7fab0c09b600 2024-09-10T10:03:35.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:35.713+0000 7fab11600640 1 -- 172.21.15.5:0/1485904115 <== mon.1 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 42 v42) ==== 94+0+11969 (secure 0 0 0) 0x7fab14109bc0 con 0x7fab0c09b600 2024-09-10T10:03:35.716 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-10T10:03:35.716 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":42,"btime":"2024-09-10T09:55:06:670916+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14454,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/2970594311","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2970594311},{"type":"v1","addr":"172.21.15.5:6835","nonce":2970594311}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14514,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6839/3616502174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":3616502174},{"type":"v1","addr":"172.21.15.5:6839","nonce":3616502174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":42,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T09:35:29.626287+0000","modified":"2024-09-10T09:55:06.419829+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24542,"mds_1":24535,"mds_2":24511,"mds_3":24506,"mds_4":24443},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14520":{"gid":14520,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/2073599842","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":2073599842},{"type":"v1","addr":"172.21.15.5:6841","nonce":2073599842}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"k","rank":4,"incarnation":26,"state":"up:active","state_seq":24,"addr":"172.21.15.102:6833/2457007427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":2457007427},{"type":"v1","addr":"172.21.15.102:6833","nonce":2457007427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/356445159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":356445159},{"type":"v1","addr":"172.21.15.159:6833","nonce":356445159}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6837/2894314507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":2894314507},{"type":"v1","addr":"172.21.15.5:6837","nonce":2894314507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.102:6835/1647549111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":1647549111},{"type":"v1","addr":"172.21.15.102:6835","nonce":1647549111}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6835/952657242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":952657242},{"type":"v1","addr":"172.21.15.159:6835","nonce":952657242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":19,"addr":"172.21.15.102:6837/2689776764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":2689776764},{"type":"v1","addr":"172.21.15.102:6837","nonce":2689776764}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.159:6837/620627639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":620627639},{"type":"v1","addr":"172.21.15.159:6837","nonce":620627639}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24535":{"gid":24535,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":18,"addr":"172.21.15.159:6839/2073739599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":2073739599},{"type":"v1","addr":"172.21.15.159:6839","nonce":2073739599}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24542":{"gid":24542,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.102:6839/335409749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6838","nonce":335409749},{"type":"v1","addr":"172.21.15.102:6839","nonce":335409749}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24542,"qdb_cluster":[24542,24535,24511,24506,24443]},"id":1}]} 2024-09-10T10:03:35.717 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 42 2024-09-10T10:03:35.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:35.716+0000 7fab1a400640 1 -- 172.21.15.5:0/1485904115 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7faafc0a2ce0 msgr2=0x7faafc0a51a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:35.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:35.716+0000 7fab1a400640 1 --2- 172.21.15.5:0/1485904115 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7faafc0a2ce0 0x7faafc0a51a0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fab080096c0 tx=0x7fab08000bc0 comp rx=0 tx=0).stop 2024-09-10T10:03:35.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:35.716+0000 7fab1a400640 1 -- 172.21.15.5:0/1485904115 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fab0c09b600 msgr2=0x7fab0c0a1540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:35.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:35.716+0000 7fab1a400640 1 --2- 172.21.15.5:0/1485904115 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fab0c09b600 0x7fab0c0a1540 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7fab140518b0 tx=0x7fab140747f0 comp rx=0 tx=0).stop 2024-09-10T10:03:35.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:35.716+0000 7fab1a400640 1 -- 172.21.15.5:0/1485904115 shutdown_connections 2024-09-10T10:03:35.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:35.716+0000 7fab1a400640 1 --2- 172.21.15.5:0/1485904115 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7faafc0a2ce0 0x7faafc0a51a0 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:35.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:35.716+0000 7fab1a400640 1 --2- 172.21.15.5:0/1485904115 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fab0c09b600 0x7fab0c0a1540 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:35.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:35.716+0000 7fab1a400640 1 --2- 172.21.15.5:0/1485904115 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fab0c005620 0x7fab0c132eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:35.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:35.716+0000 7fab1a400640 1 --2- 172.21.15.5:0/1485904115 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fab0c004c80 0x7fab0c132970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:35.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:35.716+0000 7fab1a400640 1 -- 172.21.15.5:0/1485904115 >> 172.21.15.5:0/1485904115 conn(0x7fab0c00a1f0 msgr2=0x7fab0c0932c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:35.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:35.716+0000 7fab1a400640 1 -- 172.21.15.5:0/1485904115 shutdown_connections 2024-09-10T10:03:35.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:35.716+0000 7fab1a400640 1 -- 172.21.15.5:0/1485904115 wait complete. 2024-09-10T10:03:36.102 INFO:tasks.workunit.client.0.smithi005.stdout: 10 177041 13.42 MB/sec execute 554 sec latency 23127.722 ms 2024-09-10T10:03:36.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:36.261+0000 7f12f8200640 1 -- 172.21.15.5:0/568981066 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f12f0109120 msgr2=0x7f12f0110ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:36.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:36.261+0000 7f12f8200640 1 --2- 172.21.15.5:0/568981066 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f12f0109120 0x7f12f0110ca0 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f12e8009be0 tx=0x7f12e802a6d0 comp rx=0 tx=0).stop 2024-09-10T10:03:36.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:36.261+0000 7f12f8200640 1 -- 172.21.15.5:0/568981066 shutdown_connections 2024-09-10T10:03:36.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:36.261+0000 7f12f8200640 1 --2- 172.21.15.5:0/568981066 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f12f0109120 0x7f12f0110ca0 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:36.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:36.261+0000 7f12f8200640 1 --2- 172.21.15.5:0/568981066 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f12f01049a0 0x7f12f01089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:36.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:36.261+0000 7f12f8200640 1 --2- 172.21.15.5:0/568981066 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f12f0103ff0 0x7f12f01043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:36.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:36.261+0000 7f12f8200640 1 -- 172.21.15.5:0/568981066 >> 172.21.15.5:0/568981066 conn(0x7f12f00ffa90 msgr2=0x7f12f0101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:36.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:36.262+0000 7f12f8200640 1 -- 172.21.15.5:0/568981066 shutdown_connections 2024-09-10T10:03:36.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:36.262+0000 7f12f8200640 1 -- 172.21.15.5:0/568981066 wait complete. 2024-09-10T10:03:36.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:36.262+0000 7f12f8200640 1 Processor -- start 2024-09-10T10:03:36.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:36.263+0000 7f12f8200640 1 -- start start 2024-09-10T10:03:36.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:36.263+0000 7f12f8200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f12f0103ff0 0x7f12f019a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:36.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:36.263+0000 7f12f8200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f12f01049a0 0x7f12f019ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:36.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:36.263+0000 7f12f8200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f12f0109120 0x7f12f019eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:36.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:36.263+0000 7f12f8200640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f12f0114130 con 0x7f12f01049a0 2024-09-10T10:03:36.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:36.263+0000 7f12f8200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f12f0113fb0 con 0x7f12f0109120 2024-09-10T10:03:36.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:36.263+0000 7f12f8200640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f12f01142b0 con 0x7f12f0103ff0 2024-09-10T10:03:36.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:36.263+0000 7f12f4e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f12f01049a0 0x7f12f019ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:36.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:36.263+0000 7f12f4e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f12f01049a0 0x7f12f019ab50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56386/0 (socket says 172.21.15.5:56386) 2024-09-10T10:03:36.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:36.263+0000 7f12f4e00640 1 -- 172.21.15.5:0/2451889983 learned_addr learned my addr 172.21.15.5:0/2451889983 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T10:03:36.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:36.263+0000 7f12f5800640 1 --2- 172.21.15.5:0/2451889983 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f12f0103ff0 0x7f12f019a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:36.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:36.264+0000 7f12f4e00640 1 -- 172.21.15.5:0/2451889983 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f12f0103ff0 msgr2=0x7f12f019a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:36.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:36.264+0000 7f12f4e00640 1 --2- 172.21.15.5:0/2451889983 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f12f0103ff0 0x7f12f019a610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:36.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:36.264+0000 7f12f4e00640 1 -- 172.21.15.5:0/2451889983 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f12f0109120 msgr2=0x7f12f019eee0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-10T10:03:36.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:36.264+0000 7f12f4e00640 1 --2- 172.21.15.5:0/2451889983 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f12f0109120 0x7f12f019eee0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:36.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:36.264+0000 7f12f4e00640 1 -- 172.21.15.5:0/2451889983 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12f019f5c0 con 0x7f12f01049a0 2024-09-10T10:03:36.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:36.264+0000 7f12f4e00640 1 --2- 172.21.15.5:0/2451889983 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f12f01049a0 0x7f12f019ab50 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f12dc00e9c0 tx=0x7f12dc00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:36.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:36.265+0000 7f12ee000640 1 -- 172.21.15.5:0/2451889983 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12dc00ccc0 con 0x7f12f01049a0 2024-09-10T10:03:36.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:36.265+0000 7f12ee000640 1 -- 172.21.15.5:0/2451889983 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f12dc005500 con 0x7f12f01049a0 2024-09-10T10:03:36.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:36.265+0000 7f12ee000640 1 -- 172.21.15.5:0/2451889983 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12dc010430 con 0x7f12f01049a0 2024-09-10T10:03:36.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:36.265+0000 7f12f8200640 1 -- 172.21.15.5:0/2451889983 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12f019f8b0 con 0x7f12f01049a0 2024-09-10T10:03:36.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:36.265+0000 7f12f8200640 1 -- 172.21.15.5:0/2451889983 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12f01a7190 con 0x7f12f01049a0 2024-09-10T10:03:36.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:36.268+0000 7f12fadf6740 1 Processor -- start 2024-09-10T10:03:36.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:36.269+0000 7f12ee000640 1 -- 172.21.15.5:0/2451889983 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f12dc0046c0 con 0x7f12f01049a0 2024-09-10T10:03:36.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:36.269+0000 7f12fadf6740 1 -- start start 2024-09-10T10:03:36.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:36.269+0000 7f12fadf6740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563202e9cc70 0x563202e9d050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:36.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:36.269+0000 7f12fadf6740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563202e9d590 0x563202ea7910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:36.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:36.269+0000 7f12fadf6740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563202ea7e50 0x563202eaa240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:36.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:36.269+0000 7f12fadf6740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x563202e033e0 con 0x563202e9cc70 2024-09-10T10:03:36.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:36.269+0000 7f12fadf6740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x563202daad80 con 0x563202ea7e50 2024-09-10T10:03:36.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:36.269+0000 7f12fadf6740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x563202dff3a0 con 0x563202e9d590 2024-09-10T10:03:36.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:36.269+0000 7f12f6200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563202e9cc70 0x563202e9d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:36.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:36.270+0000 7f12f5800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563202e9d590 0x563202ea7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:36.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:36.270+0000 7f12f4e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563202ea7e50 0x563202eaa240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:36.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:36.270+0000 7f12f6200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563202e9cc70 0x563202e9d050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56394/0 (socket says 172.21.15.5:56394) 2024-09-10T10:03:36.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:36.270+0000 7f12f6200640 1 -- 172.21.15.5:0/2173466301 learned_addr learned my addr 172.21.15.5:0/2173466301 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T10:03:36.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:36.270+0000 7f12f6200640 1 -- 172.21.15.5:0/2173466301 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563202e9d590 msgr2=0x563202ea7910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:03:36.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:36.270+0000 7f12f6200640 1 --2- 172.21.15.5:0/2173466301 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563202e9d590 0x563202ea7910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:36.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:36.270+0000 7f12f6200640 1 -- 172.21.15.5:0/2173466301 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563202ea7e50 msgr2=0x563202eaa240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:03:36.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:36.270+0000 7f12f6200640 1 --2- 172.21.15.5:0/2173466301 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563202ea7e50 0x563202eaa240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:36.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:36.270+0000 7f12f6200640 1 -- 172.21.15.5:0/2173466301 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563202dc9ae0 con 0x563202e9cc70 2024-09-10T10:03:36.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:36.270+0000 7f12ee000640 1 --2- 172.21.15.5:0/2451889983 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f12d80a2c00 0x7f12d80a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:36.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:36.270+0000 7f12f6200640 1 --2- 172.21.15.5:0/2173466301 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563202e9cc70 0x563202e9d050 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f12e8009b30 tx=0x7f12e8030a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4855524211c2dddf server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:36.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:36.271+0000 7f12ee000640 1 -- 172.21.15.5:0/2451889983 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f12dc0cf730 con 0x7f12f01049a0 2024-09-10T10:03:36.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:36.271+0000 7f12f5800640 1 --2- 172.21.15.5:0/2451889983 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f12d80a2c00 0x7f12d80a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:36.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:36.271+0000 7f12e2a00640 1 -- 172.21.15.5:0/2173466301 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12e8040020 con 0x563202e9cc70 2024-09-10T10:03:36.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:36.271+0000 7f12e2a00640 1 -- 172.21.15.5:0/2173466301 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f12e8030d50 con 0x563202e9cc70 2024-09-10T10:03:36.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:36.271+0000 7f12e2a00640 1 -- 172.21.15.5:0/2173466301 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12e8036040 con 0x563202e9cc70 2024-09-10T10:03:36.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:36.272+0000 7f12fadf6740 1 -- 172.21.15.5:0/2173466301 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563202e9cc70 msgr2=0x563202e9d050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:36.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:36.272+0000 7f12fadf6740 1 --2- 172.21.15.5:0/2173466301 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563202e9cc70 0x563202e9d050 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f12e8009b30 tx=0x7f12e8030a30 comp rx=0 tx=0).stop 2024-09-10T10:03:36.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:36.272+0000 7f12f5800640 1 --2- 172.21.15.5:0/2451889983 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f12d80a2c00 0x7f12d80a50c0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x563202e9a060 tx=0x7f12e4005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:36.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:36.272+0000 7f12fadf6740 1 -- 172.21.15.5:0/2173466301 shutdown_connections 2024-09-10T10:03:36.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:36.272+0000 7f12fadf6740 1 --2- 172.21.15.5:0/2173466301 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563202ea7e50 0x563202eaa240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:36.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:36.272+0000 7f12fadf6740 1 --2- 172.21.15.5:0/2173466301 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563202e9d590 0x563202ea7910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:36.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:36.272+0000 7f12fadf6740 1 --2- 172.21.15.5:0/2173466301 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563202e9cc70 0x563202e9d050 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:36.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:36.272+0000 7f12fadf6740 1 -- 172.21.15.5:0/2173466301 >> 172.21.15.5:0/2173466301 conn(0x563202e99200 msgr2=0x563202e995e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:36.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:36.272+0000 7f12fadf6740 1 -- 172.21.15.5:0/2173466301 shutdown_connections 2024-09-10T10:03:36.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:36.272+0000 7f12fadf6740 1 -- 172.21.15.5:0/2173466301 wait complete. 2024-09-10T10:03:36.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:36.272+0000 7f12fadf6740 1 -- start start 2024-09-10T10:03:36.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:36.273+0000 7f12fadf6740 1 Processor -- start 2024-09-10T10:03:36.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:36.273+0000 7f12fadf6740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563202e9cc70 0x563202ea4d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:36.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:36.273+0000 7f12fadf6740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563202e9d590 0x563202ea5260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:36.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:36.273+0000 7f12fadf6740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563202ea7e50 0x563202ea57a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:36.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:36.273+0000 7f12fadf6740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x563202eaaa20 con 0x563202e9d590 2024-09-10T10:03:36.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:36.273+0000 7f12fadf6740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x563202dbe0f0 con 0x563202e9cc70 2024-09-10T10:03:36.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:36.273+0000 7f12fadf6740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x563202eaaba0 con 0x563202ea7e50 2024-09-10T10:03:36.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:36.273+0000 7f12fadf6740 10 client.0 _ll_register_callbacks cb 0x563202da5bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:03:36.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:36.273+0000 7f12f5800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563202e9cc70 0x563202ea4d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:36.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:36.273+0000 7f12f5800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563202e9cc70 0x563202ea4d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:54116/0 (socket says 172.21.15.5:54116) 2024-09-10T10:03:36.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:36.273+0000 7f12f5800640 1 -- 172.21.15.5:0/1293449789 learned_addr learned my addr 172.21.15.5:0/1293449789 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T10:03:36.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:36.274+0000 7f12f6200640 1 --2- 172.21.15.5:0/1293449789 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563202ea7e50 0x563202ea57a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:36.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:36.274+0000 7f12f4e00640 1 --2- 172.21.15.5:0/1293449789 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563202e9d590 0x563202ea5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:36.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:36.274+0000 7f12f5800640 1 -- 172.21.15.5:0/1293449789 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563202ea7e50 msgr2=0x563202ea57a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:03:36.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:36.274+0000 7f12f5800640 1 --2- 172.21.15.5:0/1293449789 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563202ea7e50 0x563202ea57a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:36.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:36.274+0000 7f12f5800640 1 -- 172.21.15.5:0/1293449789 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563202e9d590 msgr2=0x563202ea5260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:03:36.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:36.274+0000 7f12f5800640 1 --2- 172.21.15.5:0/1293449789 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563202e9d590 0x563202ea5260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:36.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:36.274+0000 7f12f5800640 1 -- 172.21.15.5:0/1293449789 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563202b743c0 con 0x563202e9cc70 2024-09-10T10:03:36.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:36.274+0000 7f12f6200640 1 --2- 172.21.15.5:0/1293449789 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563202ea7e50 0x563202ea57a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:03:36.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:36.274+0000 7f12f4e00640 1 --2- 172.21.15.5:0/1293449789 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563202e9d590 0x563202ea5260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:03:36.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:36.274+0000 7f12f5800640 1 --2- 172.21.15.5:0/1293449789 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563202e9cc70 0x563202ea4d20 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f12e4009730 tx=0x7f12e4004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6ef22e137f97062f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:36.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:36.274+0000 7f12d7400640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-09-10T10:03:36.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:36.275+0000 7f12d7400640 1 -- 172.21.15.5:0/1293449789 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12e401f640 con 0x563202e9cc70 2024-09-10T10:03:36.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:36.275+0000 7f12fadf6740 1 -- 172.21.15.5:0/1293449789 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f12e8030d50 con 0x563202e9cc70 2024-09-10T10:03:36.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:36.275+0000 7f12d7400640 1 -- 172.21.15.5:0/1293449789 <== mon.2 v2:172.21.15.102:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f12e401f7e0 con 0x563202e9cc70 2024-09-10T10:03:36.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:36.275+0000 7f12d7400640 1 -- 172.21.15.5:0/1293449789 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f12e401f9e0 con 0x563202e9cc70 2024-09-10T10:03:36.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:36.275+0000 7f12fadf6740 10 client.30137 fetch_fsmap learned FSMap version 42 2024-09-10T10:03:36.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:36.275+0000 7f12fadf6740 1 -- 172.21.15.5:0/1293449789 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x563202eb6af0 con 0x563202e9cc70 2024-09-10T10:03:36.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:36.275+0000 7f12d7400640 1 -- 172.21.15.5:0/1293449789 <== mon.2 v2:172.21.15.102:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f12e8030d50 con 0x563202e9cc70 2024-09-10T10:03:36.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:36.276+0000 7f12d7400640 1 -- 172.21.15.5:0/1293449789 <== mon.2 v2:172.21.15.102:3300/0 5 ==== fsmap(e 42) ==== 8370+0+0 (secure 0 0 0) 0x7f12e4027070 con 0x563202e9cc70 2024-09-10T10:03:36.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:36.276+0000 7f12fadf6740 10 client.30137 fetch_fsmap finished waiting for FSMap version 42 2024-09-10T10:03:36.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:36.276+0000 7f12fadf6740 10 client.30137 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-10T10:03:36.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:36.276+0000 7f12fadf6740 20 client.30137 populate_metadata read hostname 'smithi005' 2024-09-10T10:03:36.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:36.276+0000 7f12fadf6740 1 --2- 172.21.15.5:0/1293449789 >> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] conn(0x563202eb7f00 0x563202eba3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:36.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:36.276+0000 7f12fadf6740 4 client.30137 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:03:36.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:36.276+0000 7f12fadf6740 1 -- 172.21.15.5:0/1293449789 --> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563202e10950 con 0x563202eb7f00 2024-09-10T10:03:36.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:36.276+0000 7f12f4e00640 1 --2- 172.21.15.5:0/1293449789 >> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] conn(0x563202eb7f00 0x563202eba3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:36.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:36.277+0000 7f12f4e00640 1 --2- 172.21.15.5:0/1293449789 >> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] conn(0x563202eb7f00 0x563202eba3a0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7171ae1076f07ee1 server_cookie=f8d44aad0f97ee61 in_seq=0 out_seq=0 2024-09-10T10:03:36.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:36.277+0000 7f12d7400640 10 client.30137 ms_handle_connect on v2:172.21.15.159:6838/2073739599 2024-09-10T10:03:36.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:36.277+0000 7f12d7400640 1 -- 172.21.15.5:0/1293449789 <== mds.1 v2:172.21.15.159:6838/2073739599 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563202e10950 con 0x563202eb7f00 2024-09-10T10:03:36.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:36.277+0000 7f12d7400640 10 client.30137 handle_command_reply: tid=0 2024-09-10T10:03:36.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:36.278+0000 7f12fadf6740 1 client.30137 shutdown 2024-09-10T10:03:36.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:36.278+0000 7f12fadf6740 1 -- 172.21.15.5:0/1293449789 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563202e9cc70 msgr2=0x563202ea4d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:36.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:36.278+0000 7f12fadf6740 1 --2- 172.21.15.5:0/1293449789 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563202e9cc70 0x563202ea4d20 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f12e4009730 tx=0x7f12e4004730 comp rx=0 tx=0).stop 2024-09-10T10:03:36.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:36.278+0000 7f12fadf6740 1 -- 172.21.15.5:0/1293449789 shutdown_connections 2024-09-10T10:03:36.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:36.278+0000 7f12fadf6740 1 --2- 172.21.15.5:0/1293449789 >> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] conn(0x563202eb7f00 0x563202eba3a0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:36.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:36.278+0000 7f12fadf6740 1 --2- 172.21.15.5:0/1293449789 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563202ea7e50 0x563202ea57a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:36.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:36.278+0000 7f12fadf6740 1 --2- 172.21.15.5:0/1293449789 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563202e9d590 0x563202ea5260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:36.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:36.278+0000 7f12fadf6740 1 --2- 172.21.15.5:0/1293449789 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563202e9cc70 0x563202ea4d20 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:36.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:36.278+0000 7f12fadf6740 1 -- 172.21.15.5:0/1293449789 >> 172.21.15.5:0/1293449789 conn(0x563202e99200 msgr2=0x563202e98380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:36.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:36.278+0000 7f12d7400640 0 client.30137 ms_handle_reset on v2:172.21.15.159:6838/2073739599 2024-09-10T10:03:36.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:36.278+0000 7f12fadf6740 1 -- 172.21.15.5:0/1293449789 shutdown_connections 2024-09-10T10:03:36.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:36.278+0000 7f12fadf6740 1 -- 172.21.15.5:0/1293449789 wait complete. 2024-09-10T10:03:36.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:36.278+0000 7f12fadf6740 20 client.30137 trim_cache size 0 max 16384 2024-09-10T10:03:36.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:36.292+0000 7f12fadf6740 1 Processor -- start 2024-09-10T10:03:36.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:36.292+0000 7f12fadf6740 1 -- start start 2024-09-10T10:03:36.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:36.292+0000 7f12fadf6740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563202e9cc70 0x563202e9f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:36.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:36.292+0000 7f12fadf6740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563202e9d590 0x563202ea6c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:36.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:36.292+0000 7f12fadf6740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563202ea7e50 0x563202ebfcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:36.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:36.293+0000 7f12fadf6740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f12f01b1070 con 0x563202e9d590 2024-09-10T10:03:36.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:36.293+0000 7f12fadf6740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f12f01b2bb0 con 0x563202ea7e50 2024-09-10T10:03:36.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:36.293+0000 7f12fadf6740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f12f01b1740 con 0x563202e9cc70 2024-09-10T10:03:36.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:36.293+0000 7f12f4e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563202e9d590 0x563202ea6c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:36.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:36.293+0000 7f12f4e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563202e9d590 0x563202ea6c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56414/0 (socket says 172.21.15.5:56414) 2024-09-10T10:03:36.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:36.293+0000 7f12f6200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563202ea7e50 0x563202ebfcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:36.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:36.293+0000 7f12f4e00640 1 -- 172.21.15.5:0/57575784 learned_addr learned my addr 172.21.15.5:0/57575784 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T10:03:36.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:36.293+0000 7f12f5800640 1 --2- 172.21.15.5:0/57575784 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563202e9cc70 0x563202e9f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:36.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:36.293+0000 7f12f4e00640 1 -- 172.21.15.5:0/57575784 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563202e9cc70 msgr2=0x563202e9f6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:03:36.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:36.293+0000 7f12f4e00640 1 --2- 172.21.15.5:0/57575784 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563202e9cc70 0x563202e9f6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:36.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:36.293+0000 7f12f4e00640 1 -- 172.21.15.5:0/57575784 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563202ea7e50 msgr2=0x563202ebfcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:03:36.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:36.293+0000 7f12f4e00640 1 --2- 172.21.15.5:0/57575784 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563202ea7e50 0x563202ebfcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:36.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:36.293+0000 7f12f4e00640 1 -- 172.21.15.5:0/57575784 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563202e94770 con 0x563202e9d590 2024-09-10T10:03:36.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:36.293+0000 7f12f5800640 1 --2- 172.21.15.5:0/57575784 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563202e9cc70 0x563202e9f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:03:36.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:36.293+0000 7f12f6200640 1 --2- 172.21.15.5:0/57575784 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563202ea7e50 0x563202ebfcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:03:36.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:36.294+0000 7f12f4e00640 1 --2- 172.21.15.5:0/57575784 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563202e9d590 0x563202ea6c90 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f12dc0ccde0 tx=0x7f12dc0cccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3ebc106a5eee51f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:36.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:36.294+0000 7f12e2000640 1 -- 172.21.15.5:0/57575784 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12dc09c240 con 0x563202e9d590 2024-09-10T10:03:36.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:36.294+0000 7f12e2000640 1 -- 172.21.15.5:0/57575784 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f12dc09bb40 con 0x563202e9d590 2024-09-10T10:03:36.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:36.294+0000 7f12e2000640 1 -- 172.21.15.5:0/57575784 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12dc0cf3a0 con 0x563202e9d590 2024-09-10T10:03:36.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:36.295+0000 7f12fadf6740 1 -- 172.21.15.5:0/57575784 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563202e9d590 msgr2=0x563202ea6c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:36.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:36.295+0000 7f12fadf6740 1 --2- 172.21.15.5:0/57575784 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563202e9d590 0x563202ea6c90 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f12dc0ccde0 tx=0x7f12dc0cccb0 comp rx=0 tx=0).stop 2024-09-10T10:03:36.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:36.295+0000 7f12fadf6740 1 -- 172.21.15.5:0/57575784 shutdown_connections 2024-09-10T10:03:36.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:36.295+0000 7f12fadf6740 1 --2- 172.21.15.5:0/57575784 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563202ea7e50 0x563202ebfcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:36.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:36.295+0000 7f12fadf6740 1 --2- 172.21.15.5:0/57575784 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563202e9d590 0x563202ea6c90 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:36.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:36.295+0000 7f12fadf6740 1 --2- 172.21.15.5:0/57575784 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563202e9cc70 0x563202e9f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:36.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:36.295+0000 7f12fadf6740 1 -- 172.21.15.5:0/57575784 >> 172.21.15.5:0/57575784 conn(0x563202e99200 msgr2=0x563202ea6580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:36.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:36.295+0000 7f12fadf6740 1 -- 172.21.15.5:0/57575784 shutdown_connections 2024-09-10T10:03:36.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:36.295+0000 7f12fadf6740 1 -- 172.21.15.5:0/57575784 wait complete. 2024-09-10T10:03:36.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:36.295+0000 7f12fadf6740 1 -- start start 2024-09-10T10:03:36.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:36.295+0000 7f12fadf6740 1 Processor -- start 2024-09-10T10:03:36.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:36.296+0000 7f12fadf6740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563202e9cc70 0x563202eafe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:36.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:36.296+0000 7f12fadf6740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563202e9d590 0x563202eb0360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:36.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:36.296+0000 7f12fadf6740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563202ea7e50 0x563202eab3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:36.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:36.296+0000 7f12fadf6740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f12f01b24e0 con 0x563202ea7e50 2024-09-10T10:03:36.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:36.296+0000 7f12fadf6740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f12f01b1e10 con 0x563202e9cc70 2024-09-10T10:03:36.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:36.296+0000 7f12fadf6740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f12f01af050 con 0x563202e9d590 2024-09-10T10:03:36.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:36.296+0000 7f12fadf6740 10 client.0 _ll_register_callbacks cb 0x7f12e4009ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:03:36.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:36.296+0000 7f12f5800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563202e9cc70 0x563202eafe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:36.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:36.296+0000 7f12f4e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563202e9d590 0x563202eb0360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:36.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:36.296+0000 7f12f5800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563202e9cc70 0x563202eafe20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:54140/0 (socket says 172.21.15.5:54140) 2024-09-10T10:03:36.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:36.296+0000 7f12f5800640 1 -- 172.21.15.5:0/90033322 learned_addr learned my addr 172.21.15.5:0/90033322 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T10:03:36.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:36.296+0000 7f12f6200640 1 --2- 172.21.15.5:0/90033322 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563202ea7e50 0x563202eab3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:36.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:36.297+0000 7f12f5800640 1 -- 172.21.15.5:0/90033322 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563202e9d590 msgr2=0x563202eb0360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:03:36.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:36.297+0000 7f12f5800640 1 --2- 172.21.15.5:0/90033322 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563202e9d590 0x563202eb0360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:36.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:36.297+0000 7f12f5800640 1 -- 172.21.15.5:0/90033322 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563202ea7e50 msgr2=0x563202eab3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:03:36.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:36.297+0000 7f12f5800640 1 --2- 172.21.15.5:0/90033322 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563202ea7e50 0x563202eab3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:36.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:36.297+0000 7f12f5800640 1 -- 172.21.15.5:0/90033322 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563202eabb70 con 0x563202e9cc70 2024-09-10T10:03:36.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:36.297+0000 7f12f6200640 1 --2- 172.21.15.5:0/90033322 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563202ea7e50 0x563202eab3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:03:36.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:36.297+0000 7f12f5800640 1 --2- 172.21.15.5:0/90033322 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563202e9cc70 0x563202eafe20 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f12e4000c00 tx=0x7f12e401d700 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2e1799726658a52d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:36.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:36.297+0000 7f12d7400640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-09-10T10:03:36.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:36.297+0000 7f12d7400640 1 -- 172.21.15.5:0/90033322 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12e4003e50 con 0x563202e9cc70 2024-09-10T10:03:36.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:36.297+0000 7f12d7400640 1 -- 172.21.15.5:0/90033322 <== mon.2 v2:172.21.15.102:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f12e401d7f0 con 0x563202e9cc70 2024-09-10T10:03:36.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:36.297+0000 7f12fadf6740 1 -- 172.21.15.5:0/90033322 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f12dc09bb40 con 0x563202e9cc70 2024-09-10T10:03:36.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:36.298+0000 7f12d7400640 1 -- 172.21.15.5:0/90033322 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f12e401d9f0 con 0x563202e9cc70 2024-09-10T10:03:36.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:36.298+0000 7f12fadf6740 10 client.30143 fetch_fsmap learned FSMap version 42 2024-09-10T10:03:36.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:36.298+0000 7f12fadf6740 1 -- 172.21.15.5:0/90033322 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x563202f226f0 con 0x563202e9cc70 2024-09-10T10:03:36.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:36.298+0000 7f12d7400640 1 -- 172.21.15.5:0/90033322 <== mon.2 v2:172.21.15.102:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f12dc09bb40 con 0x563202e9cc70 2024-09-10T10:03:36.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:36.298+0000 7f12d7400640 1 -- 172.21.15.5:0/90033322 <== mon.2 v2:172.21.15.102:3300/0 5 ==== fsmap(e 42) ==== 8370+0+0 (secure 0 0 0) 0x7f12e4027040 con 0x563202e9cc70 2024-09-10T10:03:36.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:36.298+0000 7f12fadf6740 10 client.30143 fetch_fsmap finished waiting for FSMap version 42 2024-09-10T10:03:36.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:36.298+0000 7f12fadf6740 10 client.30143 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-10T10:03:36.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:36.298+0000 7f12fadf6740 20 client.30143 populate_metadata read hostname 'smithi005' 2024-09-10T10:03:36.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:36.298+0000 7f12fadf6740 1 --2- 172.21.15.5:0/90033322 >> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] conn(0x563202f22e70 0x563202f25290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:36.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:36.298+0000 7f12fadf6740 4 client.30143 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T10:03:36.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:36.298+0000 7f12fadf6740 1 -- 172.21.15.5:0/90033322 --> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563202e0ae60 con 0x563202f22e70 2024-09-10T10:03:36.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:36.299+0000 7f12f4e00640 1 --2- 172.21.15.5:0/90033322 >> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] conn(0x563202f22e70 0x563202f25290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:36.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:36.299+0000 7f12f4e00640 1 --2- 172.21.15.5:0/90033322 >> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] conn(0x563202f22e70 0x563202f25290 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2635f601287c49c8 server_cookie=5fc106d8f44e6e8b in_seq=0 out_seq=0 2024-09-10T10:03:36.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:36.299+0000 7f12d7400640 10 client.30143 ms_handle_connect on v2:172.21.15.159:6838/2073739599 2024-09-10T10:03:36.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:36.299+0000 7f12d7400640 1 -- 172.21.15.5:0/90033322 <== mds.1 v2:172.21.15.159:6838/2073739599 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563202e0ae60 con 0x563202f22e70 2024-09-10T10:03:36.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:36.299+0000 7f12d7400640 10 client.30143 handle_command_reply: tid=0 2024-09-10T10:03:36.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:36.299+0000 7f12fadf6740 1 client.30143 shutdown 2024-09-10T10:03:36.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:36.300+0000 7f12fadf6740 1 -- 172.21.15.5:0/90033322 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563202e9cc70 msgr2=0x563202eafe20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:36.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:36.300+0000 7f12fadf6740 1 --2- 172.21.15.5:0/90033322 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563202e9cc70 0x563202eafe20 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f12e4000c00 tx=0x7f12e401d700 comp rx=0 tx=0).stop 2024-09-10T10:03:36.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:36.300+0000 7f12fadf6740 1 -- 172.21.15.5:0/90033322 shutdown_connections 2024-09-10T10:03:36.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:36.300+0000 7f12fadf6740 1 --2- 172.21.15.5:0/90033322 >> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] conn(0x563202f22e70 0x563202f25290 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:36.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:36.300+0000 7f12fadf6740 1 --2- 172.21.15.5:0/90033322 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563202ea7e50 0x563202eab3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:36.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:36.300+0000 7f12fadf6740 1 --2- 172.21.15.5:0/90033322 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x563202e9d590 0x563202eb0360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:36.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:36.300+0000 7f12fadf6740 1 --2- 172.21.15.5:0/90033322 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x563202e9cc70 0x563202eafe20 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:36.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:36.300+0000 7f12fadf6740 1 -- 172.21.15.5:0/90033322 >> 172.21.15.5:0/90033322 conn(0x563202e99200 msgr2=0x563202ea6da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:36.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:36.300+0000 7f12d7400640 0 client.30143 ms_handle_reset on v2:172.21.15.159:6838/2073739599 2024-09-10T10:03:36.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:36.300+0000 7f12fadf6740 1 -- 172.21.15.5:0/90033322 shutdown_connections 2024-09-10T10:03:36.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:36.300+0000 7f12fadf6740 1 -- 172.21.15.5:0/90033322 wait complete. 2024-09-10T10:03:36.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:36.300+0000 7f12fadf6740 20 client.30143 trim_cache size 0 max 16384 2024-09-10T10:03:36.306 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-10T10:03:36.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:36.301+0000 7f12f8200640 1 -- 172.21.15.5:0/2451889983 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f12d80a2c00 msgr2=0x7f12d80a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:36.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:36.301+0000 7f12f8200640 1 --2- 172.21.15.5:0/2451889983 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f12d80a2c00 0x7f12d80a50c0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x563202e9a060 tx=0x7f12e4005f90 comp rx=0 tx=0).stop 2024-09-10T10:03:36.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:36.301+0000 7f12f8200640 1 -- 172.21.15.5:0/2451889983 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f12f01049a0 msgr2=0x7f12f019ab50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:36.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:36.301+0000 7f12f8200640 1 --2- 172.21.15.5:0/2451889983 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f12f01049a0 0x7f12f019ab50 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f12dc00e9c0 tx=0x7f12dc00ee80 comp rx=0 tx=0).stop 2024-09-10T10:03:36.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:36.301+0000 7f12f8200640 1 -- 172.21.15.5:0/2451889983 shutdown_connections 2024-09-10T10:03:36.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:36.301+0000 7f12f8200640 1 --2- 172.21.15.5:0/2451889983 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f12d80a2c00 0x7f12d80a50c0 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:36.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:36.301+0000 7f12f8200640 1 --2- 172.21.15.5:0/2451889983 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f12f0109120 0x7f12f019eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:36.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:36.301+0000 7f12f8200640 1 --2- 172.21.15.5:0/2451889983 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f12f01049a0 0x7f12f019ab50 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:36.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:36.301+0000 7f12f8200640 1 --2- 172.21.15.5:0/2451889983 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f12f0103ff0 0x7f12f019a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:36.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:36.301+0000 7f12f8200640 1 -- 172.21.15.5:0/2451889983 >> 172.21.15.5:0/2451889983 conn(0x7f12f00ffa90 msgr2=0x7f12f01005b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:36.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:36.302+0000 7f12f8200640 1 -- 172.21.15.5:0/2451889983 shutdown_connections 2024-09-10T10:03:36.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:36.302+0000 7f12f8200640 1 -- 172.21.15.5:0/2451889983 wait complete. 2024-09-10T10:03:36.436 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:03:36 smithi005 ceph-mon[30350]: from='client.? 172.21.15.5:0/1485904115' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:03:36.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:03:36 smithi159 ceph-mon[35513]: from='client.? 172.21.15.5:0/1485904115' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:03:36.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:03:36 smithi102 ceph-mon[35442]: from='client.? 172.21.15.5:0/1485904115' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:03:37.102 INFO:tasks.workunit.client.0.smithi005.stdout: 10 177041 13.40 MB/sec execute 555 sec latency 24127.774 ms 2024-09-10T10:03:37.258 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:03:37 smithi005 ceph-mon[30350]: pgmap v1151: 129 pgs: 129 active+clean; 786 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 296 KiB/s rd, 4.9 MiB/s wr, 204 op/s 2024-09-10T10:03:37.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:03:37 smithi159 ceph-mon[35513]: pgmap v1151: 129 pgs: 129 active+clean; 786 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 296 KiB/s rd, 4.9 MiB/s wr, 204 op/s 2024-09-10T10:03:37.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:03:37 smithi102 ceph-mon[35442]: pgmap v1151: 129 pgs: 129 active+clean; 786 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 296 KiB/s rd, 4.9 MiB/s wr, 204 op/s 2024-09-10T10:03:37.730 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24542 quiesce db --set-id dfb92e25 --release --await' 2024-09-10T10:03:37.730 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24542 quiesce db --set-id dfb92e25 --release --await 2024-09-10T10:03:38.071 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-10T10:03:38.102 INFO:tasks.workunit.client.0.smithi005.stdout: 10 177041 13.38 MB/sec execute 556 sec latency 25127.827 ms 2024-09-10T10:03:38.170 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:03:39.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:39.067+0000 7f125b200640 1 -- 172.21.15.5:0/330271512 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f125c10c0a0 msgr2=0x7f125c06e4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:39.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:39.067+0000 7f125b200640 1 --2- 172.21.15.5:0/330271512 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f125c10c0a0 0x7f125c06e4f0 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f124c009900 tx=0x7f124c02a400 comp rx=0 tx=0).stop 2024-09-10T10:03:39.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:39.069+0000 7f125b200640 1 -- 172.21.15.5:0/330271512 shutdown_connections 2024-09-10T10:03:39.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:39.069+0000 7f125b200640 1 --2- 172.21.15.5:0/330271512 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f125c1169f0 0x7f125c118e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:39.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:39.069+0000 7f125b200640 1 --2- 172.21.15.5:0/330271512 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f125c06ea30 0x7f125c06ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:39.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:39.069+0000 7f125b200640 1 --2- 172.21.15.5:0/330271512 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f125c10c0a0 0x7f125c06e4f0 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:39.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:39.069+0000 7f125b200640 1 -- 172.21.15.5:0/330271512 >> 172.21.15.5:0/330271512 conn(0x7f125c073180 msgr2=0x7f125c073590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:39.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:39.070+0000 7f125b200640 1 -- 172.21.15.5:0/330271512 shutdown_connections 2024-09-10T10:03:39.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:39.070+0000 7f125b200640 1 -- 172.21.15.5:0/330271512 wait complete. 2024-09-10T10:03:39.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:39.071+0000 7f125b200640 1 Processor -- start 2024-09-10T10:03:39.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:39.072+0000 7f125b200640 1 -- start start 2024-09-10T10:03:39.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:39.074+0000 7f125b200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f125c06ea30 0x7f125c1a88e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:39.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:39.074+0000 7f125b200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f125c10c0a0 0x7f125c1a8e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:39.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:39.074+0000 7f1259e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f125c06ea30 0x7f125c1a88e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:39.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:39.074+0000 7f1259e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f125c06ea30 0x7f125c1a88e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:54154/0 (socket says 172.21.15.5:54154) 2024-09-10T10:03:39.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:39.074+0000 7f1259e00640 1 -- 172.21.15.5:0/1538688621 learned_addr learned my addr 172.21.15.5:0/1538688621 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T10:03:39.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:39.075+0000 7f125b200640 1 --2- 172.21.15.5:0/1538688621 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f125c1169f0 0x7f125c1a2aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:39.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:39.075+0000 7f125b200640 1 -- 172.21.15.5:0/1538688621 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f125c11c7b0 con 0x7f125c10c0a0 2024-09-10T10:03:39.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:39.075+0000 7f125b200640 1 -- 172.21.15.5:0/1538688621 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f125c11c630 con 0x7f125c06ea30 2024-09-10T10:03:39.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:39.075+0000 7f1259400640 1 --2- 172.21.15.5:0/1538688621 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f125c10c0a0 0x7f125c1a8e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:39.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:39.076+0000 7f125b200640 1 -- 172.21.15.5:0/1538688621 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f125c11c930 con 0x7f125c1169f0 2024-09-10T10:03:39.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:39.076+0000 7f125a800640 1 --2- 172.21.15.5:0/1538688621 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f125c1169f0 0x7f125c1a2aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:39.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:39.078+0000 7f1259e00640 1 -- 172.21.15.5:0/1538688621 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f125c1169f0 msgr2=0x7f125c1a2aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:39.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:39.078+0000 7f1259e00640 1 --2- 172.21.15.5:0/1538688621 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f125c1169f0 0x7f125c1a2aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:39.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:39.078+0000 7f1259e00640 1 -- 172.21.15.5:0/1538688621 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f125c10c0a0 msgr2=0x7f125c1a8e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:39.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:39.078+0000 7f1259e00640 1 --2- 172.21.15.5:0/1538688621 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f125c10c0a0 0x7f125c1a8e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:39.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:39.078+0000 7f1259e00640 1 -- 172.21.15.5:0/1538688621 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f125c1a31d0 con 0x7f125c06ea30 2024-09-10T10:03:39.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:39.078+0000 7f1259400640 1 --2- 172.21.15.5:0/1538688621 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f125c10c0a0 0x7f125c1a8e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:03:39.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:39.079+0000 7f1259e00640 1 --2- 172.21.15.5:0/1538688621 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f125c06ea30 0x7f125c1a88e0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f124c004770 tx=0x7f124c003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:39.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:39.080+0000 7f1252a00640 1 -- 172.21.15.5:0/1538688621 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f124c03f070 con 0x7f125c06ea30 2024-09-10T10:03:39.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:39.080+0000 7f125b200640 1 -- 172.21.15.5:0/1538688621 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f125c1a3460 con 0x7f125c06ea30 2024-09-10T10:03:39.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:39.081+0000 7f125b200640 1 -- 172.21.15.5:0/1538688621 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f125c1af7b0 con 0x7f125c06ea30 2024-09-10T10:03:39.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:39.082+0000 7f1252a00640 1 -- 172.21.15.5:0/1538688621 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f124c031ca0 con 0x7f125c06ea30 2024-09-10T10:03:39.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:39.082+0000 7f1252a00640 1 -- 172.21.15.5:0/1538688621 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f124c03f070 con 0x7f125c06ea30 2024-09-10T10:03:39.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:39.093+0000 7f1252a00640 1 -- 172.21.15.5:0/1538688621 <== mon.2 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f124c042070 con 0x7f125c06ea30 2024-09-10T10:03:39.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:39.095+0000 7f1252a00640 1 --2- 172.21.15.5:0/1538688621 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f12400a2e80 0x7f12400a5340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:39.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:39.095+0000 7f1252a00640 1 -- 172.21.15.5:0/1538688621 <== mon.2 v2:172.21.15.102:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f124c0ef540 con 0x7f125c06ea30 2024-09-10T10:03:39.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:39.096+0000 7f1263cad740 1 Processor -- start 2024-09-10T10:03:39.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:39.098+0000 7f1263cad740 1 -- start start 2024-09-10T10:03:39.102 INFO:tasks.workunit.client.0.smithi005.stdout: 10 177041 13.35 MB/sec execute 557 sec latency 26127.882 ms 2024-09-10T10:03:39.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:39.098+0000 7f1259400640 1 --2- 172.21.15.5:0/1538688621 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f12400a2e80 0x7f12400a5340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:39.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:39.098+0000 7f1263cad740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5601cb52f440 0x5601cb52f820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:39.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:39.098+0000 7f1263cad740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5601cb52fd60 0x5601cb5429d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:39.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:39.098+0000 7f1263cad740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5601cb542f10 0x5601cb545300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:39.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:39.098+0000 7f1263cad740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5601cb495650 con 0x5601cb52fd60 2024-09-10T10:03:39.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:39.098+0000 7f1263cad740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x5601cb43cf10 con 0x5601cb52f440 2024-09-10T10:03:39.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:39.098+0000 7f1263cad740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5601cb491610 con 0x5601cb542f10 2024-09-10T10:03:39.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:39.099+0000 7f1259400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5601cb52fd60 0x5601cb5429d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:39.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:39.099+0000 7f1259400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5601cb52fd60 0x5601cb5429d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56456/0 (socket says 172.21.15.5:56456) 2024-09-10T10:03:39.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:39.099+0000 7f1259400640 1 -- 172.21.15.5:0/764767052 learned_addr learned my addr 172.21.15.5:0/764767052 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T10:03:39.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:39.100+0000 7f1259e00640 1 --2- 172.21.15.5:0/764767052 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5601cb52f440 0x5601cb52f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:39.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:39.101+0000 7f1259e00640 1 -- 172.21.15.5:0/764767052 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5601cb542f10 msgr2=0x5601cb545300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:03:39.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:39.101+0000 7f125a800640 1 --2- 172.21.15.5:0/764767052 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5601cb542f10 0x5601cb545300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:39.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:39.101+0000 7f1259e00640 1 --2- 172.21.15.5:0/764767052 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5601cb542f10 0x5601cb545300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:39.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:39.101+0000 7f1259e00640 1 -- 172.21.15.5:0/764767052 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5601cb52fd60 msgr2=0x5601cb5429d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:03:39.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:39.101+0000 7f1259e00640 1 --2- 172.21.15.5:0/764767052 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5601cb52fd60 0x5601cb5429d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:39.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:39.101+0000 7f1259e00640 1 -- 172.21.15.5:0/764767052 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5601cb1a4580 con 0x5601cb52f440 2024-09-10T10:03:39.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:39.101+0000 7f125a800640 1 --2- 172.21.15.5:0/764767052 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5601cb542f10 0x5601cb545300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:03:39.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:39.102+0000 7f1259e00640 1 --2- 172.21.15.5:0/764767052 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5601cb52f440 0x5601cb52f820 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f124c000c00 tx=0x7f124c02a400 comp rx=0 tx=0).ready entity=mon.2 client_cookie=95eadcba84daa96 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:39.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:39.103+0000 7f1247400640 1 -- 172.21.15.5:0/764767052 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f124c0d9070 con 0x5601cb52f440 2024-09-10T10:03:39.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:39.103+0000 7f1247400640 1 -- 172.21.15.5:0/764767052 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f124c0bba10 con 0x5601cb52f440 2024-09-10T10:03:39.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:39.103+0000 7f1247400640 1 -- 172.21.15.5:0/764767052 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f124c0d9070 con 0x5601cb52f440 2024-09-10T10:03:39.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:39.105+0000 7f1263cad740 1 -- 172.21.15.5:0/764767052 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5601cb52f440 msgr2=0x5601cb52f820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:39.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:39.105+0000 7f1263cad740 1 --2- 172.21.15.5:0/764767052 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5601cb52f440 0x5601cb52f820 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f124c000c00 tx=0x7f124c02a400 comp rx=0 tx=0).stop 2024-09-10T10:03:39.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:39.106+0000 7f1263cad740 1 -- 172.21.15.5:0/764767052 shutdown_connections 2024-09-10T10:03:39.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:39.106+0000 7f1263cad740 1 --2- 172.21.15.5:0/764767052 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5601cb542f10 0x5601cb545300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:39.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:39.106+0000 7f1263cad740 1 --2- 172.21.15.5:0/764767052 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5601cb52fd60 0x5601cb5429d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:39.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:39.106+0000 7f1263cad740 1 --2- 172.21.15.5:0/764767052 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5601cb52f440 0x5601cb52f820 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:39.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:39.106+0000 7f1263cad740 1 -- 172.21.15.5:0/764767052 >> 172.21.15.5:0/764767052 conn(0x5601cb52bab0 msgr2=0x5601cb52be90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:39.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:39.114+0000 7f1263cad740 1 -- 172.21.15.5:0/764767052 shutdown_connections 2024-09-10T10:03:39.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:39.114+0000 7f1263cad740 1 -- 172.21.15.5:0/764767052 wait complete. 2024-09-10T10:03:39.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:39.114+0000 7f1263cad740 1 -- start start 2024-09-10T10:03:39.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:39.115+0000 7f1263cad740 1 Processor -- start 2024-09-10T10:03:39.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:39.115+0000 7f1259400640 1 --2- 172.21.15.5:0/1538688621 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f12400a2e80 0x7f12400a5340 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f12480096a0 tx=0x7f12480095a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:39.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:39.116+0000 7f1263cad740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5601cb52f440 0x5601cb53ff10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:39.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:39.116+0000 7f1263cad740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5601cb52fd60 0x5601cb540450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:39.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:39.116+0000 7f1263cad740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5601cb542f10 0x5601cb540990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:39.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:39.116+0000 7f125a800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5601cb542f10 0x5601cb540990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:39.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:39.117+0000 7f1259e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5601cb52f440 0x5601cb53ff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:39.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:39.117+0000 7f1259e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5601cb52f440 0x5601cb53ff10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:54176/0 (socket says 172.21.15.5:54176) 2024-09-10T10:03:39.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:39.117+0000 7f1259e00640 1 -- 172.21.15.5:0/2060697507 learned_addr learned my addr 172.21.15.5:0/2060697507 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T10:03:39.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:39.117+0000 7f1263cad740 1 -- 172.21.15.5:0/2060697507 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5601cb545840 con 0x5601cb542f10 2024-09-10T10:03:39.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:39.117+0000 7f1263cad740 1 -- 172.21.15.5:0/2060697507 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x5601cb450350 con 0x5601cb52f440 2024-09-10T10:03:39.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:39.117+0000 7f1263cad740 1 -- 172.21.15.5:0/2060697507 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5601cb5459c0 con 0x5601cb52fd60 2024-09-10T10:03:39.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:39.117+0000 7f1263cad740 10 client.0 _ll_register_callbacks cb 0x5601cb473290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:03:39.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:39.117+0000 7f1259e00640 1 -- 172.21.15.5:0/2060697507 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5601cb52fd60 msgr2=0x5601cb540450 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T10:03:39.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:39.117+0000 7f1259e00640 1 --2- 172.21.15.5:0/2060697507 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5601cb52fd60 0x5601cb540450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:39.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:39.117+0000 7f1259e00640 1 -- 172.21.15.5:0/2060697507 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5601cb542f10 msgr2=0x5601cb540990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:03:39.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:39.117+0000 7f1259e00640 1 --2- 172.21.15.5:0/2060697507 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5601cb542f10 0x5601cb540990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:39.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:39.117+0000 7f1259e00640 1 -- 172.21.15.5:0/2060697507 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5601cb26e960 con 0x5601cb52f440 2024-09-10T10:03:39.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:39.118+0000 7f1259e00640 1 --2- 172.21.15.5:0/2060697507 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5601cb52f440 0x5601cb53ff10 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f124c03b9a0 tx=0x7f124c0ef460 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d6ccb5d1d83a7186 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:39.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:39.118+0000 7f123be00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-09-10T10:03:39.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:39.118+0000 7f1263cad740 1 -- 172.21.15.5:0/2060697507 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f124c0bba10 con 0x5601cb52f440 2024-09-10T10:03:39.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:39.118+0000 7f123be00640 1 -- 172.21.15.5:0/2060697507 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f124c0ab060 con 0x5601cb52f440 2024-09-10T10:03:39.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:39.118+0000 7f123be00640 1 -- 172.21.15.5:0/2060697507 <== mon.2 v2:172.21.15.102:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f124c0b8080 con 0x5601cb52f440 2024-09-10T10:03:39.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:39.118+0000 7f123be00640 1 -- 172.21.15.5:0/2060697507 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f124c0b2050 con 0x5601cb52f440 2024-09-10T10:03:39.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:39.119+0000 7f1263cad740 10 client.30155 fetch_fsmap learned FSMap version 42 2024-09-10T10:03:39.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:39.119+0000 7f1263cad740 1 -- 172.21.15.5:0/2060697507 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({fsmap=42}) -- 0x5601cb545840 con 0x5601cb52f440 2024-09-10T10:03:39.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:39.120+0000 7f123be00640 1 -- 172.21.15.5:0/2060697507 <== mon.2 v2:172.21.15.102:3300/0 4 ==== fsmap(e 42) ==== 8370+0+0 (secure 0 0 0) 0x7f124c0c41f0 con 0x5601cb52f440 2024-09-10T10:03:39.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:39.120+0000 7f1263cad740 10 client.30155 fetch_fsmap finished waiting for FSMap version 42 2024-09-10T10:03:39.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:39.120+0000 7f1263cad740 10 client.30155 resolve_mds: validated gid 24542 aka daemon mds.b 2024-09-10T10:03:39.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:39.120+0000 7f1263cad740 20 client.30155 populate_metadata read hostname 'smithi005' 2024-09-10T10:03:39.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:39.120+0000 7f1263cad740 1 --2- 172.21.15.5:0/2060697507 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x5601cb5744d0 0x5601cb5768c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:39.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:39.120+0000 7f1263cad740 4 client.30155 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:03:39.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:39.120+0000 7f1263cad740 1 -- 172.21.15.5:0/2060697507 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5601cb4a2bc0 con 0x5601cb5744d0 2024-09-10T10:03:39.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:39.121+0000 7f1259400640 1 --2- 172.21.15.5:0/2060697507 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x5601cb5744d0 0x5601cb5768c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:39.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:39.124+0000 7f123be00640 10 client.30155 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T10:03:39.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:39.124+0000 7f1259400640 1 --2- 172.21.15.5:0/2060697507 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x5601cb5744d0 0x5601cb5768c0 crc :-1 s=READY pgs=335 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=559450b4a22d100e server_cookie=93cc7b785c4cbd7b in_seq=0 out_seq=0 2024-09-10T10:03:39.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:39.125+0000 7f123be00640 1 -- 172.21.15.5:0/2060697507 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5601cb4a2bc0 con 0x5601cb5744d0 2024-09-10T10:03:39.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:39.125+0000 7f123be00640 10 client.30155 handle_command_reply: tid=0 2024-09-10T10:03:39.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:39.125+0000 7f1263cad740 1 client.30155 shutdown 2024-09-10T10:03:39.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:39.126+0000 7f1263cad740 1 -- 172.21.15.5:0/2060697507 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5601cb52f440 msgr2=0x5601cb53ff10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:39.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:39.126+0000 7f1263cad740 1 --2- 172.21.15.5:0/2060697507 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5601cb52f440 0x5601cb53ff10 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f124c03b9a0 tx=0x7f124c0ef460 comp rx=0 tx=0).stop 2024-09-10T10:03:39.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:39.127+0000 7f1263cad740 1 -- 172.21.15.5:0/2060697507 shutdown_connections 2024-09-10T10:03:39.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:39.127+0000 7f1263cad740 1 --2- 172.21.15.5:0/2060697507 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x5601cb5744d0 0x5601cb5768c0 crc :-1 s=READY pgs=335 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:39.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:39.127+0000 7f123be00640 0 client.30155 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T10:03:39.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:39.127+0000 7f1263cad740 1 --2- 172.21.15.5:0/2060697507 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5601cb542f10 0x5601cb540990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:39.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:39.127+0000 7f1263cad740 1 --2- 172.21.15.5:0/2060697507 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5601cb52fd60 0x5601cb540450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:39.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:39.127+0000 7f1263cad740 1 --2- 172.21.15.5:0/2060697507 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5601cb52f440 0x5601cb53ff10 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:39.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:39.127+0000 7f1263cad740 1 -- 172.21.15.5:0/2060697507 >> 172.21.15.5:0/2060697507 conn(0x5601cb52bab0 msgr2=0x5601cb52acf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:39.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:39.128+0000 7f1263cad740 1 -- 172.21.15.5:0/2060697507 shutdown_connections 2024-09-10T10:03:39.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:39.128+0000 7f1263cad740 1 -- 172.21.15.5:0/2060697507 wait complete. 2024-09-10T10:03:39.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:39.129+0000 7f1263cad740 20 client.30155 trim_cache size 0 max 16384 2024-09-10T10:03:39.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:39.164+0000 7f1263cad740 1 Processor -- start 2024-09-10T10:03:39.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:39.164+0000 7f1263cad740 1 -- start start 2024-09-10T10:03:39.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:39.165+0000 7f1263cad740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5601cb52f440 0x5601cb52ab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:39.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:39.165+0000 7f1263cad740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5601cb52fd60 0x5601cb57c150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:39.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:39.165+0000 7f1263cad740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5601cb542f10 0x5601cb57c690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:39.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:39.166+0000 7f1259e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5601cb52f440 0x5601cb52ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:39.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:39.166+0000 7f1259e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5601cb52f440 0x5601cb52ab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:45854/0 (socket says 172.21.15.5:45854) 2024-09-10T10:03:39.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:39.166+0000 7f1259e00640 1 -- 172.21.15.5:0/2554656882 learned_addr learned my addr 172.21.15.5:0/2554656882 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T10:03:39.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:39.166+0000 7f1263cad740 1 -- 172.21.15.5:0/2554656882 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f123c00fc90 con 0x5601cb542f10 2024-09-10T10:03:39.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:39.166+0000 7f1263cad740 1 -- 172.21.15.5:0/2554656882 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f123c0117d0 con 0x5601cb52fd60 2024-09-10T10:03:39.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:39.166+0000 7f1263cad740 1 -- 172.21.15.5:0/2554656882 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f123c010360 con 0x5601cb52f440 2024-09-10T10:03:39.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:39.166+0000 7f1259400640 1 --2- 172.21.15.5:0/2554656882 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5601cb52fd60 0x5601cb57c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:39.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:39.166+0000 7f125a800640 1 --2- 172.21.15.5:0/2554656882 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5601cb542f10 0x5601cb57c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:39.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:39.167+0000 7f1259400640 1 -- 172.21.15.5:0/2554656882 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5601cb52f440 msgr2=0x5601cb52ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:03:39.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:39.167+0000 7f1259400640 1 --2- 172.21.15.5:0/2554656882 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5601cb52f440 0x5601cb52ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:39.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:39.167+0000 7f1259400640 1 -- 172.21.15.5:0/2554656882 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5601cb542f10 msgr2=0x5601cb57c690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:03:39.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:39.167+0000 7f1259400640 1 --2- 172.21.15.5:0/2554656882 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5601cb542f10 0x5601cb57c690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:39.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:39.167+0000 7f1259400640 1 -- 172.21.15.5:0/2554656882 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5601cb52ed10 con 0x5601cb52fd60 2024-09-10T10:03:39.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:39.167+0000 7f1259400640 1 --2- 172.21.15.5:0/2554656882 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5601cb52fd60 0x5601cb57c150 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x5601cb5770f0 tx=0x7f1248015070 comp rx=0 tx=0).ready entity=mon.2 client_cookie=559e7e735a87064f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:39.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:39.168+0000 7f1246a00640 1 -- 172.21.15.5:0/2554656882 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1248011e70 con 0x5601cb52fd60 2024-09-10T10:03:39.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:39.168+0000 7f1246a00640 1 -- 172.21.15.5:0/2554656882 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f12480287c0 con 0x5601cb52fd60 2024-09-10T10:03:39.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:39.169+0000 7f1263cad740 1 -- 172.21.15.5:0/2554656882 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5601cb52fd60 msgr2=0x5601cb57c150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:39.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:39.169+0000 7f1263cad740 1 --2- 172.21.15.5:0/2554656882 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5601cb52fd60 0x5601cb57c150 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x5601cb5770f0 tx=0x7f1248015070 comp rx=0 tx=0).stop 2024-09-10T10:03:39.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:39.169+0000 7f1263cad740 1 -- 172.21.15.5:0/2554656882 shutdown_connections 2024-09-10T10:03:39.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:39.169+0000 7f1263cad740 1 --2- 172.21.15.5:0/2554656882 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5601cb542f10 0x5601cb57c690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:39.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:39.169+0000 7f1263cad740 1 --2- 172.21.15.5:0/2554656882 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5601cb52fd60 0x5601cb57c150 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:39.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:39.169+0000 7f1263cad740 1 --2- 172.21.15.5:0/2554656882 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5601cb52f440 0x5601cb52ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:39.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:39.169+0000 7f1263cad740 1 -- 172.21.15.5:0/2554656882 >> 172.21.15.5:0/2554656882 conn(0x5601cb52bab0 msgr2=0x5601cb54e2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:39.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:39.170+0000 7f1263cad740 1 -- 172.21.15.5:0/2554656882 shutdown_connections 2024-09-10T10:03:39.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:39.170+0000 7f1263cad740 1 -- 172.21.15.5:0/2554656882 wait complete. 2024-09-10T10:03:39.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:39.171+0000 7f1263cad740 1 -- start start 2024-09-10T10:03:39.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:39.171+0000 7f1263cad740 1 Processor -- start 2024-09-10T10:03:39.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:39.174+0000 7f1263cad740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5601cb52f440 0x5601cb54b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:39.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:39.174+0000 7f1263cad740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5601cb52fd60 0x5601cb5474e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:39.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:39.176+0000 7f1259e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5601cb52f440 0x5601cb54b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:39.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:39.176+0000 7f1259e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5601cb52f440 0x5601cb54b1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56492/0 (socket says 172.21.15.5:56492) 2024-09-10T10:03:39.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:39.176+0000 7f1259e00640 1 -- 172.21.15.5:0/1218949104 learned_addr learned my addr 172.21.15.5:0/1218949104 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T10:03:39.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:39.177+0000 7f1259400640 1 --2- 172.21.15.5:0/1218949104 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5601cb52fd60 0x5601cb5474e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:39.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:39.177+0000 7f1263cad740 1 --2- 172.21.15.5:0/1218949104 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5601cb542f10 0x5601cb547a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:39.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:39.177+0000 7f1263cad740 1 -- 172.21.15.5:0/1218949104 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f123c011100 con 0x5601cb52f440 2024-09-10T10:03:39.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:39.177+0000 7f1263cad740 1 -- 172.21.15.5:0/1218949104 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f123c010a30 con 0x5601cb52fd60 2024-09-10T10:03:39.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:39.177+0000 7f1263cad740 1 -- 172.21.15.5:0/1218949104 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f123c00dc70 con 0x5601cb542f10 2024-09-10T10:03:39.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:39.177+0000 7f125a800640 1 --2- 172.21.15.5:0/1218949104 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5601cb542f10 0x5601cb547a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:39.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:39.177+0000 7f1263cad740 10 client.0 _ll_register_callbacks cb 0x7f124c0d7070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:03:39.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:39.178+0000 7f1259e00640 1 -- 172.21.15.5:0/1218949104 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5601cb542f10 msgr2=0x5601cb547a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:03:39.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:39.178+0000 7f1259e00640 1 --2- 172.21.15.5:0/1218949104 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5601cb542f10 0x5601cb547a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:39.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:39.178+0000 7f1259e00640 1 -- 172.21.15.5:0/1218949104 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5601cb52fd60 msgr2=0x5601cb5474e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:03:39.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:39.178+0000 7f1259e00640 1 --2- 172.21.15.5:0/1218949104 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5601cb52fd60 0x5601cb5474e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:39.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:39.178+0000 7f1259e00640 1 -- 172.21.15.5:0/1218949104 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5601cb548060 con 0x5601cb52f440 2024-09-10T10:03:39.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:39.179+0000 7f1259e00640 1 --2- 172.21.15.5:0/1218949104 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5601cb52f440 0x5601cb54b1a0 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f124c0b83e0 tx=0x7f124c009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18d96e7b72637d57 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:39.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:39.179+0000 7f123be00640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-10T10:03:39.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:39.179+0000 7f123be00640 1 -- 172.21.15.5:0/1218949104 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f124c0ab040 con 0x5601cb52f440 2024-09-10T10:03:39.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:39.179+0000 7f123be00640 1 -- 172.21.15.5:0/1218949104 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f124c0c43f0 con 0x5601cb52f440 2024-09-10T10:03:39.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:39.180+0000 7f1263cad740 1 -- 172.21.15.5:0/1218949104 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f12480287c0 con 0x5601cb52f440 2024-09-10T10:03:39.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:39.180+0000 7f123be00640 1 -- 172.21.15.5:0/1218949104 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f124c0d9070 con 0x5601cb52f440 2024-09-10T10:03:39.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:39.181+0000 7f1263cad740 10 client.20067 fetch_fsmap learned FSMap version 42 2024-09-10T10:03:39.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:39.181+0000 7f1263cad740 1 -- 172.21.15.5:0/1218949104 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x5601cb548030 con 0x5601cb52f440 2024-09-10T10:03:39.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:39.181+0000 7f1259400640 1 --2- 172.21.15.5:0/1218949104 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5601cb52fd60 0x5601cb5474e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:03:39.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:39.182+0000 7f123be00640 1 -- 172.21.15.5:0/1218949104 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f12480287c0 con 0x5601cb52f440 2024-09-10T10:03:39.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:39.183+0000 7f123be00640 1 -- 172.21.15.5:0/1218949104 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 42) ==== 8370+0+0 (secure 0 0 0) 0x7f124c0bae60 con 0x5601cb52f440 2024-09-10T10:03:39.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:39.183+0000 7f1263cad740 10 client.20067 fetch_fsmap finished waiting for FSMap version 42 2024-09-10T10:03:39.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:39.183+0000 7f1263cad740 10 client.20067 resolve_mds: validated gid 24542 aka daemon mds.b 2024-09-10T10:03:39.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:39.183+0000 7f1263cad740 20 client.20067 populate_metadata read hostname 'smithi005' 2024-09-10T10:03:39.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:39.183+0000 7f1263cad740 1 --2- 172.21.15.5:0/1218949104 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x5601cb5324b0 0x5601cb5348d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:39.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:39.183+0000 7f1263cad740 4 client.20067 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "dfb92e25", "release": true, "await": true}] 2024-09-10T10:03:39.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:39.183+0000 7f1263cad740 1 -- 172.21.15.5:0/1218949104 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "quiesce db", "set_id": "dfb92e25", "release": true, "await": true}) -- 0x5601cb49d0d0 con 0x5601cb5324b0 2024-09-10T10:03:39.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:39.184+0000 7f1259400640 1 --2- 172.21.15.5:0/1218949104 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x5601cb5324b0 0x5601cb5348d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:39.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:39.184+0000 7f1259400640 1 --2- 172.21.15.5:0/1218949104 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x5601cb5324b0 0x5601cb5348d0 crc :-1 s=READY pgs=336 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=87005ea4b68bdf12 server_cookie=209d6ea6fd21d461 in_seq=0 out_seq=0 2024-09-10T10:03:39.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:39.185+0000 7f123be00640 10 client.20067 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T10:03:39.191 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:03:39.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:39.235+0000 7f123be00640 1 -- 172.21.15.5:0/1218949104 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5601cb49d0d0 con 0x5601cb5324b0 2024-09-10T10:03:39.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:39.235+0000 7f123be00640 10 client.20067 handle_command_reply: tid=0 2024-09-10T10:03:39.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:39.235+0000 7f1263cad740 1 client.20067 shutdown 2024-09-10T10:03:39.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:39.236+0000 7f1263cad740 1 -- 172.21.15.5:0/1218949104 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5601cb52f440 msgr2=0x5601cb54b1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:39.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:39.236+0000 7f1263cad740 1 --2- 172.21.15.5:0/1218949104 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5601cb52f440 0x5601cb54b1a0 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f124c0b83e0 tx=0x7f124c009260 comp rx=0 tx=0).stop 2024-09-10T10:03:39.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:39.236+0000 7f1263cad740 1 -- 172.21.15.5:0/1218949104 shutdown_connections 2024-09-10T10:03:39.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:39.236+0000 7f1263cad740 1 --2- 172.21.15.5:0/1218949104 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x5601cb5324b0 0x5601cb5348d0 crc :-1 s=READY pgs=336 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:39.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:39.236+0000 7f1263cad740 1 --2- 172.21.15.5:0/1218949104 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5601cb542f10 0x5601cb547a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:39.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:39.236+0000 7f1263cad740 1 --2- 172.21.15.5:0/1218949104 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5601cb52fd60 0x5601cb5474e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:39.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:39.236+0000 7f123be00640 0 client.20067 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T10:03:39.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:39.236+0000 7f1263cad740 1 --2- 172.21.15.5:0/1218949104 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5601cb52f440 0x5601cb54b1a0 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:39.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:39.236+0000 7f1263cad740 1 -- 172.21.15.5:0/1218949104 >> 172.21.15.5:0/1218949104 conn(0x5601cb52bab0 msgr2=0x5601cb5316b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:39.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:39.237+0000 7f1263cad740 1 -- 172.21.15.5:0/1218949104 shutdown_connections 2024-09-10T10:03:39.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:39.237+0000 7f1263cad740 1 -- 172.21.15.5:0/1218949104 wait complete. 2024-09-10T10:03:39.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:39.240+0000 7f1263cad740 20 client.20067 trim_cache size 0 max 16384 2024-09-10T10:03:39.245 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-10T10:03:39.245 INFO:teuthology.orchestra.run.smithi005.stdout: "epoch": 42, 2024-09-10T10:03:39.245 INFO:teuthology.orchestra.run.smithi005.stdout: "leader": 24542, 2024-09-10T10:03:39.245 INFO:teuthology.orchestra.run.smithi005.stdout: "set_version": 88, 2024-09-10T10:03:39.245 INFO:teuthology.orchestra.run.smithi005.stdout: "sets": { 2024-09-10T10:03:39.245 INFO:teuthology.orchestra.run.smithi005.stdout: "dfb92e25": { 2024-09-10T10:03:39.245 INFO:teuthology.orchestra.run.smithi005.stdout: "version": 88, 2024-09-10T10:03:39.245 INFO:teuthology.orchestra.run.smithi005.stdout: "age_ref": 0.0, 2024-09-10T10:03:39.245 INFO:teuthology.orchestra.run.smithi005.stdout: "state": { 2024-09-10T10:03:39.245 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "RELEASED", 2024-09-10T10:03:39.246 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0 2024-09-10T10:03:39.246 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:03:39.246 INFO:teuthology.orchestra.run.smithi005.stdout: "timeout": 90.0, 2024-09-10T10:03:39.246 INFO:teuthology.orchestra.run.smithi005.stdout: "expiration": 150.0, 2024-09-10T10:03:39.246 INFO:teuthology.orchestra.run.smithi005.stdout: "members": { 2024-09-10T10:03:39.246 INFO:teuthology.orchestra.run.smithi005.stdout: "file:/": { 2024-09-10T10:03:39.246 INFO:teuthology.orchestra.run.smithi005.stdout: "excluded": false, 2024-09-10T10:03:39.246 INFO:teuthology.orchestra.run.smithi005.stdout: "state": { 2024-09-10T10:03:39.246 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "RELEASED", 2024-09-10T10:03:39.246 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0 2024-09-10T10:03:39.246 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T10:03:39.246 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T10:03:39.246 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T10:03:39.246 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T10:03:39.246 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T10:03:39.247 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-10T10:03:39.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:39.243+0000 7f125b200640 1 -- 172.21.15.5:0/1538688621 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f12400a2e80 msgr2=0x7f12400a5340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:39.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:39.243+0000 7f125b200640 1 --2- 172.21.15.5:0/1538688621 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f12400a2e80 0x7f12400a5340 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f12480096a0 tx=0x7f12480095a0 comp rx=0 tx=0).stop 2024-09-10T10:03:39.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:39.243+0000 7f125b200640 1 -- 172.21.15.5:0/1538688621 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f125c06ea30 msgr2=0x7f125c1a88e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:39.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:39.243+0000 7f125b200640 1 --2- 172.21.15.5:0/1538688621 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f125c06ea30 0x7f125c1a88e0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f124c004770 tx=0x7f124c003c10 comp rx=0 tx=0).stop 2024-09-10T10:03:39.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:39.244+0000 7f125b200640 1 -- 172.21.15.5:0/1538688621 shutdown_connections 2024-09-10T10:03:39.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:39.244+0000 7f125b200640 1 --2- 172.21.15.5:0/1538688621 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f12400a2e80 0x7f12400a5340 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:39.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:39.244+0000 7f125b200640 1 --2- 172.21.15.5:0/1538688621 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f125c1169f0 0x7f125c1a2aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:39.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:39.244+0000 7f125b200640 1 --2- 172.21.15.5:0/1538688621 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f125c10c0a0 0x7f125c1a8e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:39.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:39.244+0000 7f125b200640 1 --2- 172.21.15.5:0/1538688621 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f125c06ea30 0x7f125c1a88e0 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:39.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:39.244+0000 7f125b200640 1 -- 172.21.15.5:0/1538688621 >> 172.21.15.5:0/1538688621 conn(0x7f125c073180 msgr2=0x7f125c116e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:39.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:39.244+0000 7f125b200640 1 -- 172.21.15.5:0/1538688621 shutdown_connections 2024-09-10T10:03:39.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:39.244+0000 7f125b200640 1 -- 172.21.15.5:0/1538688621 wait complete. 2024-09-10T10:03:39.491 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:03:39 smithi005 ceph-mon[30350]: pgmap v1152: 129 pgs: 129 active+clean; 790 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 175 KiB/s rd, 4.7 MiB/s wr, 181 op/s 2024-09-10T10:03:39.491 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:03:39 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:03:39.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:03:39 smithi159 ceph-mon[35513]: pgmap v1152: 129 pgs: 129 active+clean; 790 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 175 KiB/s rd, 4.7 MiB/s wr, 181 op/s 2024-09-10T10:03:39.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:03:39 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:03:39.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:03:39 smithi102 ceph-mon[35442]: pgmap v1152: 129 pgs: 129 active+clean; 790 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 175 KiB/s rd, 4.7 MiB/s wr, 181 op/s 2024-09-10T10:03:39.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:03:39 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:03:40.102 INFO:tasks.workunit.client.0.smithi005.stdout: 10 177041 13.33 MB/sec execute 558 sec latency 27128.155 ms 2024-09-10T10:03:40.572 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'dfb92e25', total seconds elapsed: 36.3 2024-09-10T10:03:40.572 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 30.0 2024-09-10T10:03:40.573 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T10:03:41.102 INFO:tasks.workunit.client.0.smithi005.stdout: 10 177041 13.30 MB/sec execute 559 sec latency 28128.216 ms 2024-09-10T10:03:41.403 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:03:41 smithi005 ceph-mon[30350]: pgmap v1153: 129 pgs: 129 active+clean; 794 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 255 KiB/s rd, 4.6 MiB/s wr, 190 op/s 2024-09-10T10:03:41.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:03:41 smithi159 ceph-mon[35513]: pgmap v1153: 129 pgs: 129 active+clean; 794 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 255 KiB/s rd, 4.6 MiB/s wr, 190 op/s 2024-09-10T10:03:41.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:03:41 smithi102 ceph-mon[35442]: pgmap v1153: 129 pgs: 129 active+clean; 794 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 255 KiB/s rd, 4.6 MiB/s wr, 190 op/s 2024-09-10T10:03:41.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:41.669+0000 7f31c1c00640 1 -- 172.21.15.5:0/2848356919 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f31bc112a50 msgr2=0x7f31bc114e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:41.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:41.669+0000 7f31c1c00640 1 --2- 172.21.15.5:0/2848356919 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f31bc112a50 0x7f31bc114e40 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f31b4009ca0 tx=0x7f31b402a8c0 comp rx=0 tx=0).stop 2024-09-10T10:03:41.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:41.669+0000 7f31c1c00640 1 -- 172.21.15.5:0/2848356919 shutdown_connections 2024-09-10T10:03:41.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:41.669+0000 7f31c1c00640 1 --2- 172.21.15.5:0/2848356919 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f31bc112a50 0x7f31bc114e40 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:41.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:41.669+0000 7f31c1c00640 1 --2- 172.21.15.5:0/2848356919 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f31bc077040 0x7f31bc112510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:41.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:41.669+0000 7f31c1c00640 1 --2- 172.21.15.5:0/2848356919 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f31bc076720 0x7f31bc076b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:41.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:41.669+0000 7f31c1c00640 1 -- 172.21.15.5:0/2848356919 >> 172.21.15.5:0/2848356919 conn(0x7f31bc0ffa50 msgr2=0x7f31bc101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:41.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:41.670+0000 7f31c1c00640 1 -- 172.21.15.5:0/2848356919 shutdown_connections 2024-09-10T10:03:41.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:41.670+0000 7f31c1c00640 1 -- 172.21.15.5:0/2848356919 wait complete. 2024-09-10T10:03:41.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:41.671+0000 7f31c1c00640 1 Processor -- start 2024-09-10T10:03:41.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:41.671+0000 7f31c1c00640 1 -- start start 2024-09-10T10:03:41.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:41.671+0000 7f31c1c00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f31bc076720 0x7f31bc19eb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:41.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:41.671+0000 7f31c1c00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f31bc077040 0x7f31bc19f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:41.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:41.671+0000 7f31c1c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f31bc112a50 0x7f31bc1a3490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:41.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:41.671+0000 7f31c1c00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f31bc1185f0 con 0x7f31bc077040 2024-09-10T10:03:41.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:41.671+0000 7f31c1c00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f31bc118470 con 0x7f31bc112a50 2024-09-10T10:03:41.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:41.671+0000 7f31c1c00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f31bc118770 con 0x7f31bc076720 2024-09-10T10:03:41.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:41.672+0000 7f31baa00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f31bc076720 0x7f31bc19eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:41.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:41.672+0000 7f31baa00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f31bc076720 0x7f31bc19eb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:55992/0 (socket says 172.21.15.5:55992) 2024-09-10T10:03:41.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:41.672+0000 7f31baa00640 1 -- 172.21.15.5:0/468602211 learned_addr learned my addr 172.21.15.5:0/468602211 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T10:03:41.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:41.672+0000 7f31ba000640 1 --2- 172.21.15.5:0/468602211 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f31bc077040 0x7f31bc19f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:41.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:41.672+0000 7f31bb400640 1 --2- 172.21.15.5:0/468602211 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f31bc112a50 0x7f31bc1a3490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:41.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:41.672+0000 7f31baa00640 1 -- 172.21.15.5:0/468602211 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f31bc112a50 msgr2=0x7f31bc1a3490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:41.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:41.672+0000 7f31baa00640 1 --2- 172.21.15.5:0/468602211 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f31bc112a50 0x7f31bc1a3490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:41.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:41.672+0000 7f31baa00640 1 -- 172.21.15.5:0/468602211 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f31bc077040 msgr2=0x7f31bc19f0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:41.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:41.672+0000 7f31baa00640 1 --2- 172.21.15.5:0/468602211 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f31bc077040 0x7f31bc19f0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:41.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:41.672+0000 7f31baa00640 1 -- 172.21.15.5:0/468602211 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31bc1a3bc0 con 0x7f31bc076720 2024-09-10T10:03:41.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:41.673+0000 7f31baa00640 1 --2- 172.21.15.5:0/468602211 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f31bc076720 0x7f31bc19eb90 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f31ac00e9f0 tx=0x7f31ac00edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:41.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:41.673+0000 7f31b3400640 1 -- 172.21.15.5:0/468602211 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31ac00cd80 con 0x7f31bc076720 2024-09-10T10:03:41.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:41.673+0000 7f31b3400640 1 -- 172.21.15.5:0/468602211 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f31ac005500 con 0x7f31bc076720 2024-09-10T10:03:41.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:41.673+0000 7f31c1c00640 1 -- 172.21.15.5:0/468602211 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31bc1a3eb0 con 0x7f31bc076720 2024-09-10T10:03:41.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:41.673+0000 7f31b3400640 1 -- 172.21.15.5:0/468602211 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31ac010690 con 0x7f31bc076720 2024-09-10T10:03:41.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:41.673+0000 7f31c1c00640 1 -- 172.21.15.5:0/468602211 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31bc1ab740 con 0x7f31bc076720 2024-09-10T10:03:41.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:41.676+0000 7f31c46a9740 1 Processor -- start 2024-09-10T10:03:41.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:41.676+0000 7f31b3400640 1 -- 172.21.15.5:0/468602211 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f31ac0046c0 con 0x7f31bc076720 2024-09-10T10:03:41.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:41.676+0000 7f31c46a9740 1 -- start start 2024-09-10T10:03:41.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:41.677+0000 7f31c46a9740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5565b5a1dc70 0x5565b5a1e050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:41.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:41.677+0000 7f31c46a9740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5565b5a1e590 0x5565b5a28910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:41.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:41.677+0000 7f31c46a9740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5565b5a28e50 0x5565b5a2b240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:41.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:41.677+0000 7f31c46a9740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5565b59843e0 con 0x5565b5a28e50 2024-09-10T10:03:41.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:41.677+0000 7f31c46a9740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x5565b592bd80 con 0x5565b5a1e590 2024-09-10T10:03:41.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:41.677+0000 7f31c46a9740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5565b59803a0 con 0x5565b5a1dc70 2024-09-10T10:03:41.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:41.677+0000 7f31baa00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5565b5a1e590 0x5565b5a28910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:41.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:41.677+0000 7f31bb400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5565b5a1dc70 0x5565b5a1e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:41.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:41.677+0000 7f31baa00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5565b5a1e590 0x5565b5a28910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:53712/0 (socket says 172.21.15.5:53712) 2024-09-10T10:03:41.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:41.677+0000 7f31baa00640 1 -- 172.21.15.5:0/4114962992 learned_addr learned my addr 172.21.15.5:0/4114962992 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T10:03:41.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:41.677+0000 7f31ba000640 1 --2- 172.21.15.5:0/4114962992 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5565b5a28e50 0x5565b5a2b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:41.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:41.678+0000 7f31baa00640 1 -- 172.21.15.5:0/4114962992 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5565b5a1dc70 msgr2=0x5565b5a1e050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:03:41.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:41.678+0000 7f31baa00640 1 --2- 172.21.15.5:0/4114962992 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5565b5a1dc70 0x5565b5a1e050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:41.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:41.678+0000 7f31baa00640 1 -- 172.21.15.5:0/4114962992 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5565b5a28e50 msgr2=0x5565b5a2b240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:03:41.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:41.678+0000 7f31baa00640 1 --2- 172.21.15.5:0/4114962992 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5565b5a28e50 0x5565b5a2b240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:41.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:41.678+0000 7f31baa00640 1 -- 172.21.15.5:0/4114962992 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5565b594aae0 con 0x5565b5a1e590 2024-09-10T10:03:41.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:41.678+0000 7f31b3400640 1 --2- 172.21.15.5:0/468602211 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f31a00a2b80 0x7f31a00a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:41.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:41.678+0000 7f31b3400640 1 -- 172.21.15.5:0/468602211 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f31ac0cf920 con 0x7f31bc076720 2024-09-10T10:03:41.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:41.678+0000 7f31baa00640 1 --2- 172.21.15.5:0/4114962992 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5565b5a1e590 0x5565b5a28910 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f31ac0d5a90 tx=0x7f31ac0d5b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aac4320e42565e66 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:41.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:41.678+0000 7f31bb400640 1 --2- 172.21.15.5:0/468602211 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f31a00a2b80 0x7f31a00a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:41.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:41.678+0000 7f31a7e00640 1 -- 172.21.15.5:0/4114962992 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31ac097fd0 con 0x5565b5a1e590 2024-09-10T10:03:41.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:41.678+0000 7f31a7e00640 1 -- 172.21.15.5:0/4114962992 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f31ac09c430 con 0x5565b5a1e590 2024-09-10T10:03:41.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:41.678+0000 7f31a7e00640 1 -- 172.21.15.5:0/4114962992 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31ac09b690 con 0x5565b5a1e590 2024-09-10T10:03:41.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:41.679+0000 7f31bb400640 1 --2- 172.21.15.5:0/468602211 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f31a00a2b80 0x7f31a00a5040 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f31b4006010 tx=0x7f31b402f390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:41.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:41.679+0000 7f31c46a9740 1 -- 172.21.15.5:0/4114962992 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5565b5a1e590 msgr2=0x5565b5a28910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:41.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:41.679+0000 7f31c46a9740 1 --2- 172.21.15.5:0/4114962992 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5565b5a1e590 0x5565b5a28910 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f31ac0d5a90 tx=0x7f31ac0d5b80 comp rx=0 tx=0).stop 2024-09-10T10:03:41.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:41.679+0000 7f31c46a9740 1 -- 172.21.15.5:0/4114962992 shutdown_connections 2024-09-10T10:03:41.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:41.679+0000 7f31c46a9740 1 --2- 172.21.15.5:0/4114962992 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5565b5a28e50 0x5565b5a2b240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:41.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:41.679+0000 7f31c46a9740 1 --2- 172.21.15.5:0/4114962992 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5565b5a1e590 0x5565b5a28910 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:41.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:41.679+0000 7f31c46a9740 1 --2- 172.21.15.5:0/4114962992 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5565b5a1dc70 0x5565b5a1e050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:41.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:41.679+0000 7f31c46a9740 1 -- 172.21.15.5:0/4114962992 >> 172.21.15.5:0/4114962992 conn(0x5565b5a1a200 msgr2=0x5565b5a1a5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:41.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:41.679+0000 7f31c46a9740 1 -- 172.21.15.5:0/4114962992 shutdown_connections 2024-09-10T10:03:41.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:41.679+0000 7f31c46a9740 1 -- 172.21.15.5:0/4114962992 wait complete. 2024-09-10T10:03:41.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:41.680+0000 7f31c46a9740 1 -- start start 2024-09-10T10:03:41.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:41.680+0000 7f31c46a9740 1 Processor -- start 2024-09-10T10:03:41.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:41.681+0000 7f31c46a9740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5565b5a1dc70 0x5565b5a25dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:41.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:41.681+0000 7f31c46a9740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5565b5a1e590 0x5565b5a26300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:41.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:41.681+0000 7f31c46a9740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5565b5a28e50 0x5565b5a26840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:41.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:41.681+0000 7f31c46a9740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5565b5a2ba20 con 0x5565b5a28e50 2024-09-10T10:03:41.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:41.681+0000 7f31c46a9740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x5565b593f0f0 con 0x5565b5a1e590 2024-09-10T10:03:41.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:41.681+0000 7f31c46a9740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5565b5a2bba0 con 0x5565b5a1dc70 2024-09-10T10:03:41.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:41.681+0000 7f31c46a9740 10 client.0 _ll_register_callbacks cb 0x5565b5926bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:03:41.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:41.681+0000 7f31bb400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5565b5a28e50 0x5565b5a26840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:41.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:41.681+0000 7f31baa00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5565b5a1dc70 0x5565b5a25dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:41.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:41.681+0000 7f31baa00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5565b5a1dc70 0x5565b5a25dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:56016/0 (socket says 172.21.15.5:56016) 2024-09-10T10:03:41.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:41.681+0000 7f31baa00640 1 -- 172.21.15.5:0/266249253 learned_addr learned my addr 172.21.15.5:0/266249253 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T10:03:41.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:41.681+0000 7f31ba000640 1 --2- 172.21.15.5:0/266249253 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5565b5a1e590 0x5565b5a26300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:41.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:41.682+0000 7f31baa00640 1 -- 172.21.15.5:0/266249253 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5565b5a1e590 msgr2=0x5565b5a26300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:03:41.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:41.682+0000 7f31baa00640 1 --2- 172.21.15.5:0/266249253 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5565b5a1e590 0x5565b5a26300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:41.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:41.682+0000 7f31baa00640 1 -- 172.21.15.5:0/266249253 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5565b5a28e50 msgr2=0x5565b5a26840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:03:41.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:41.682+0000 7f31baa00640 1 --2- 172.21.15.5:0/266249253 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5565b5a28e50 0x5565b5a26840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:41.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:41.682+0000 7f31baa00640 1 -- 172.21.15.5:0/266249253 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5565b56f53c0 con 0x5565b5a1dc70 2024-09-10T10:03:41.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:41.682+0000 7f31baa00640 1 --2- 172.21.15.5:0/266249253 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5565b5a1dc70 0x5565b5a25dc0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f31ac0d5a60 tx=0x7f31ac00cf20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c5ba8211187ba739 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:41.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:41.682+0000 7f31a4c00640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-09-10T10:03:41.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:41.682+0000 7f31a4c00640 1 -- 172.21.15.5:0/266249253 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31ac09b870 con 0x5565b5a1dc70 2024-09-10T10:03:41.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:41.682+0000 7f31a4c00640 1 -- 172.21.15.5:0/266249253 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f31ac09bf90 con 0x5565b5a1dc70 2024-09-10T10:03:41.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:41.682+0000 7f31c46a9740 1 -- 172.21.15.5:0/266249253 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f31ac09c430 con 0x5565b5a1dc70 2024-09-10T10:03:41.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:41.682+0000 7f31a4c00640 1 -- 172.21.15.5:0/266249253 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f31ac09bb00 con 0x5565b5a1dc70 2024-09-10T10:03:41.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:41.682+0000 7f31c46a9740 10 client.30154 fetch_fsmap learned FSMap version 42 2024-09-10T10:03:41.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:41.682+0000 7f31c46a9740 1 -- 172.21.15.5:0/266249253 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x5565b5a2ba20 con 0x5565b5a1dc70 2024-09-10T10:03:41.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:41.683+0000 7f31a4c00640 1 -- 172.21.15.5:0/266249253 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f31ac09c430 con 0x5565b5a1dc70 2024-09-10T10:03:41.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:41.683+0000 7f31a4c00640 1 -- 172.21.15.5:0/266249253 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 42) ==== 8370+0+0 (secure 0 0 0) 0x7f31ac0a4510 con 0x5565b5a1dc70 2024-09-10T10:03:41.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:41.683+0000 7f31c46a9740 10 client.30154 fetch_fsmap finished waiting for FSMap version 42 2024-09-10T10:03:41.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:41.683+0000 7f31c46a9740 10 client.30154 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T10:03:41.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:41.683+0000 7f31c46a9740 20 client.30154 populate_metadata read hostname 'smithi005' 2024-09-10T10:03:41.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:41.683+0000 7f31c46a9740 1 --2- 172.21.15.5:0/266249253 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x5565b5a38cb0 0x5565b5a3b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:41.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:41.683+0000 7f31c46a9740 4 client.30154 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:03:41.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:41.683+0000 7f31c46a9740 1 -- 172.21.15.5:0/266249253 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5565b5991950 con 0x5565b5a38cb0 2024-09-10T10:03:41.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:41.683+0000 7f31ba000640 1 --2- 172.21.15.5:0/266249253 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x5565b5a38cb0 0x5565b5a3b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:41.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:41.684+0000 7f31ba000640 1 --2- 172.21.15.5:0/266249253 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x5565b5a38cb0 0x5565b5a3b0a0 crc :-1 s=READY pgs=337 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6f108b1c11f94443 server_cookie=ae2cb6133a1b1a57 in_seq=0 out_seq=0 2024-09-10T10:03:41.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:41.684+0000 7f31a4c00640 10 client.30154 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T10:03:41.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:41.685+0000 7f31a4c00640 1 -- 172.21.15.5:0/266249253 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5565b5991950 con 0x5565b5a38cb0 2024-09-10T10:03:41.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:41.685+0000 7f31a4c00640 10 client.30154 handle_command_reply: tid=0 2024-09-10T10:03:41.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:41.685+0000 7f31c46a9740 1 client.30154 shutdown 2024-09-10T10:03:41.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:41.685+0000 7f31c46a9740 1 -- 172.21.15.5:0/266249253 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5565b5a1dc70 msgr2=0x5565b5a25dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:41.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:41.685+0000 7f31c46a9740 1 --2- 172.21.15.5:0/266249253 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5565b5a1dc70 0x5565b5a25dc0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f31ac0d5a60 tx=0x7f31ac00cf20 comp rx=0 tx=0).stop 2024-09-10T10:03:41.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:41.685+0000 7f31c46a9740 1 -- 172.21.15.5:0/266249253 shutdown_connections 2024-09-10T10:03:41.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:41.685+0000 7f31c46a9740 1 --2- 172.21.15.5:0/266249253 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x5565b5a38cb0 0x5565b5a3b0a0 crc :-1 s=READY pgs=337 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:41.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:41.685+0000 7f31c46a9740 1 --2- 172.21.15.5:0/266249253 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5565b5a28e50 0x5565b5a26840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:41.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:41.685+0000 7f31c46a9740 1 --2- 172.21.15.5:0/266249253 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5565b5a1e590 0x5565b5a26300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:41.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:41.685+0000 7f31c46a9740 1 --2- 172.21.15.5:0/266249253 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5565b5a1dc70 0x5565b5a25dc0 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:41.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:41.685+0000 7f31c46a9740 1 -- 172.21.15.5:0/266249253 >> 172.21.15.5:0/266249253 conn(0x5565b5a1a200 msgr2=0x5565b5a1fee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:41.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:41.685+0000 7f31a4c00640 0 client.30154 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T10:03:41.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:41.685+0000 7f31c46a9740 1 -- 172.21.15.5:0/266249253 shutdown_connections 2024-09-10T10:03:41.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:41.685+0000 7f31c46a9740 1 -- 172.21.15.5:0/266249253 wait complete. 2024-09-10T10:03:41.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:41.685+0000 7f31c46a9740 20 client.30154 trim_cache size 0 max 16384 2024-09-10T10:03:41.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:41.700+0000 7f31c46a9740 1 Processor -- start 2024-09-10T10:03:41.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:41.700+0000 7f31c46a9740 1 -- start start 2024-09-10T10:03:41.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:41.700+0000 7f31c46a9740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5565b5a1dc70 0x5565b5a1fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:41.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:41.700+0000 7f31c46a9740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5565b5a1e590 0x5565b5a40810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:41.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:41.700+0000 7f31c46a9740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5565b5a28e50 0x5565b5a40d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:41.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:41.700+0000 7f31c46a9740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f31bc1b55c0 con 0x5565b5a1dc70 2024-09-10T10:03:41.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:41.700+0000 7f31c46a9740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f31bc1b7100 con 0x5565b5a28e50 2024-09-10T10:03:41.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:41.700+0000 7f31c46a9740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f31bc1b5c90 con 0x5565b5a1e590 2024-09-10T10:03:41.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:41.700+0000 7f31baa00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5565b5a1dc70 0x5565b5a1fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:41.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:41.700+0000 7f31ba000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5565b5a1e590 0x5565b5a40810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:41.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:41.700+0000 7f31ba000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5565b5a1e590 0x5565b5a40810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:56020/0 (socket says 172.21.15.5:56020) 2024-09-10T10:03:41.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:41.700+0000 7f31baa00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5565b5a1dc70 0x5565b5a1fdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35032/0 (socket says 172.21.15.5:35032) 2024-09-10T10:03:41.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:41.700+0000 7f31ba000640 1 -- 172.21.15.5:0/1807422305 learned_addr learned my addr 172.21.15.5:0/1807422305 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T10:03:41.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:41.700+0000 7f31bb400640 1 --2- 172.21.15.5:0/1807422305 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5565b5a28e50 0x5565b5a40d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:41.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:41.701+0000 7f31baa00640 1 -- 172.21.15.5:0/1807422305 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5565b5a1e590 msgr2=0x5565b5a40810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:03:41.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:41.701+0000 7f31baa00640 1 --2- 172.21.15.5:0/1807422305 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5565b5a1e590 0x5565b5a40810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:41.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:41.701+0000 7f31baa00640 1 -- 172.21.15.5:0/1807422305 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5565b5a28e50 msgr2=0x5565b5a40d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:03:41.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:41.701+0000 7f31baa00640 1 --2- 172.21.15.5:0/1807422305 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5565b5a28e50 0x5565b5a40d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:41.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:41.701+0000 7f31baa00640 1 -- 172.21.15.5:0/1807422305 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5565b5a15770 con 0x5565b5a1dc70 2024-09-10T10:03:41.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:41.701+0000 7f31ba000640 1 --2- 172.21.15.5:0/1807422305 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5565b5a1e590 0x5565b5a40810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:03:41.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:41.701+0000 7f31baa00640 1 --2- 172.21.15.5:0/1807422305 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5565b5a1dc70 0x5565b5a1fdd0 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f31ac0d5a90 tx=0x7f31ac09a240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac4bc054915393d9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:41.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:41.701+0000 7f31a7400640 1 -- 172.21.15.5:0/1807422305 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31ac09b870 con 0x5565b5a1dc70 2024-09-10T10:03:41.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:41.701+0000 7f31a7400640 1 -- 172.21.15.5:0/1807422305 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f31ac09c430 con 0x5565b5a1dc70 2024-09-10T10:03:41.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:41.701+0000 7f31a7400640 1 -- 172.21.15.5:0/1807422305 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31ac09b690 con 0x5565b5a1dc70 2024-09-10T10:03:41.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:41.702+0000 7f31c46a9740 1 -- 172.21.15.5:0/1807422305 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5565b5a1dc70 msgr2=0x5565b5a1fdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:41.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:41.702+0000 7f31c46a9740 1 --2- 172.21.15.5:0/1807422305 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5565b5a1dc70 0x5565b5a1fdd0 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f31ac0d5a90 tx=0x7f31ac09a240 comp rx=0 tx=0).stop 2024-09-10T10:03:41.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:41.702+0000 7f31c46a9740 1 -- 172.21.15.5:0/1807422305 shutdown_connections 2024-09-10T10:03:41.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:41.702+0000 7f31c46a9740 1 --2- 172.21.15.5:0/1807422305 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5565b5a28e50 0x5565b5a40d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:41.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:41.702+0000 7f31c46a9740 1 --2- 172.21.15.5:0/1807422305 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5565b5a1e590 0x5565b5a40810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:41.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:41.702+0000 7f31c46a9740 1 --2- 172.21.15.5:0/1807422305 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5565b5a1dc70 0x5565b5a1fdd0 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:41.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:41.702+0000 7f31c46a9740 1 -- 172.21.15.5:0/1807422305 >> 172.21.15.5:0/1807422305 conn(0x5565b5a1a200 msgr2=0x5565b5a1f6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:41.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:41.702+0000 7f31c46a9740 1 -- 172.21.15.5:0/1807422305 shutdown_connections 2024-09-10T10:03:41.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:41.702+0000 7f31c46a9740 1 -- 172.21.15.5:0/1807422305 wait complete. 2024-09-10T10:03:41.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:41.703+0000 7f31c46a9740 1 -- start start 2024-09-10T10:03:41.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:41.703+0000 7f31c46a9740 1 Processor -- start 2024-09-10T10:03:41.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:41.704+0000 7f31c46a9740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5565b5a1dc70 0x5565b5a2cce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:41.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:41.704+0000 7f31c46a9740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5565b5a1e590 0x5565b5a2d220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:41.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:41.704+0000 7f31c46a9740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5565b5a28e50 0x5565b5a2d760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:41.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:41.704+0000 7f31c46a9740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f31bc1b6a30 con 0x5565b5a1e590 2024-09-10T10:03:41.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:41.704+0000 7f31c46a9740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f31bc1b6360 con 0x5565b5a1dc70 2024-09-10T10:03:41.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:41.704+0000 7f31c46a9740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f31bc1b35a0 con 0x5565b5a28e50 2024-09-10T10:03:41.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:41.704+0000 7f31c46a9740 10 client.0 _ll_register_callbacks cb 0x7f31a8009570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:03:41.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:41.704+0000 7f31ba000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5565b5a1e590 0x5565b5a2d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:41.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:41.704+0000 7f31baa00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5565b5a1dc70 0x5565b5a2cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:41.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:41.704+0000 7f31baa00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5565b5a1dc70 0x5565b5a2cce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:53730/0 (socket says 172.21.15.5:53730) 2024-09-10T10:03:41.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:41.704+0000 7f31baa00640 1 -- 172.21.15.5:0/3389551012 learned_addr learned my addr 172.21.15.5:0/3389551012 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T10:03:41.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:41.704+0000 7f31bb400640 1 --2- 172.21.15.5:0/3389551012 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5565b5a28e50 0x5565b5a2d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:41.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:41.704+0000 7f31ba000640 1 -- 172.21.15.5:0/3389551012 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5565b5a28e50 msgr2=0x5565b5a2d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:03:41.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:41.704+0000 7f31ba000640 1 --2- 172.21.15.5:0/3389551012 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5565b5a28e50 0x5565b5a2d760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:41.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:41.704+0000 7f31ba000640 1 -- 172.21.15.5:0/3389551012 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5565b5a1dc70 msgr2=0x5565b5a2cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:03:41.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:41.704+0000 7f31ba000640 1 --2- 172.21.15.5:0/3389551012 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5565b5a1dc70 0x5565b5a2cce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:41.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:41.704+0000 7f31ba000640 1 -- 172.21.15.5:0/3389551012 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5565b5aa2980 con 0x5565b5a1e590 2024-09-10T10:03:41.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:41.705+0000 7f31ba000640 1 --2- 172.21.15.5:0/3389551012 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5565b5a1e590 0x5565b5a2d220 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f31a8009540 tx=0x7f31a80108f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3344a25d70ed7f6a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:41.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:41.705+0000 7f31a4c00640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-10T10:03:41.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:41.705+0000 7f31a4c00640 1 -- 172.21.15.5:0/3389551012 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31a801f810 con 0x5565b5a1e590 2024-09-10T10:03:41.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:41.705+0000 7f31a4c00640 1 -- 172.21.15.5:0/3389551012 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f31a801f9b0 con 0x5565b5a1e590 2024-09-10T10:03:41.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:41.705+0000 7f31c46a9740 1 -- 172.21.15.5:0/3389551012 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f31ac09c430 con 0x5565b5a1e590 2024-09-10T10:03:41.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:41.706+0000 7f31a4c00640 1 -- 172.21.15.5:0/3389551012 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f31a801fbb0 con 0x5565b5a1e590 2024-09-10T10:03:41.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:41.706+0000 7f31c46a9740 10 client.20085 fetch_fsmap learned FSMap version 42 2024-09-10T10:03:41.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:41.706+0000 7f31c46a9740 1 -- 172.21.15.5:0/3389551012 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x5565b5aa3cc0 con 0x5565b5a1e590 2024-09-10T10:03:41.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:41.706+0000 7f31a4c00640 1 -- 172.21.15.5:0/3389551012 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f31ac09c430 con 0x5565b5a1e590 2024-09-10T10:03:41.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:41.706+0000 7f31a4c00640 1 -- 172.21.15.5:0/3389551012 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 42) ==== 8370+0+0 (secure 0 0 0) 0x7f31a8025070 con 0x5565b5a1e590 2024-09-10T10:03:41.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:41.706+0000 7f31c46a9740 10 client.20085 fetch_fsmap finished waiting for FSMap version 42 2024-09-10T10:03:41.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:41.706+0000 7f31c46a9740 10 client.20085 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T10:03:41.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:41.706+0000 7f31c46a9740 20 client.20085 populate_metadata read hostname 'smithi005' 2024-09-10T10:03:41.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:41.706+0000 7f31c46a9740 1 --2- 172.21.15.5:0/3389551012 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x5565b5aa4470 0x5565b5aa6890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:41.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:41.706+0000 7f31c46a9740 4 client.20085 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T10:03:41.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:41.706+0000 7f31c46a9740 1 -- 172.21.15.5:0/3389551012 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5565b598be60 con 0x5565b5aa4470 2024-09-10T10:03:41.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:41.707+0000 7f31baa00640 1 --2- 172.21.15.5:0/3389551012 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x5565b5aa4470 0x5565b5aa6890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:41.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:41.707+0000 7f31baa00640 1 --2- 172.21.15.5:0/3389551012 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x5565b5aa4470 0x5565b5aa6890 crc :-1 s=READY pgs=338 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=72c66e3f37c81ad5 server_cookie=7c4a452d6ee00f63 in_seq=0 out_seq=0 2024-09-10T10:03:41.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:41.707+0000 7f31a4c00640 10 client.20085 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T10:03:41.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:41.707+0000 7f31a4c00640 1 -- 172.21.15.5:0/3389551012 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5565b598be60 con 0x5565b5aa4470 2024-09-10T10:03:41.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:41.707+0000 7f31a4c00640 10 client.20085 handle_command_reply: tid=0 2024-09-10T10:03:41.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:41.707+0000 7f31c46a9740 1 client.20085 shutdown 2024-09-10T10:03:41.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:41.708+0000 7f31c46a9740 1 -- 172.21.15.5:0/3389551012 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5565b5a1e590 msgr2=0x5565b5a2d220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:41.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:41.708+0000 7f31c46a9740 1 --2- 172.21.15.5:0/3389551012 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5565b5a1e590 0x5565b5a2d220 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f31a8009540 tx=0x7f31a80108f0 comp rx=0 tx=0).stop 2024-09-10T10:03:41.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:41.708+0000 7f31c46a9740 1 -- 172.21.15.5:0/3389551012 shutdown_connections 2024-09-10T10:03:41.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:41.708+0000 7f31c46a9740 1 --2- 172.21.15.5:0/3389551012 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x5565b5aa4470 0x5565b5aa6890 crc :-1 s=READY pgs=338 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:41.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:41.708+0000 7f31c46a9740 1 --2- 172.21.15.5:0/3389551012 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5565b5a28e50 0x5565b5a2d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:41.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:41.708+0000 7f31c46a9740 1 --2- 172.21.15.5:0/3389551012 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5565b5a1e590 0x5565b5a2d220 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:41.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:41.708+0000 7f31c46a9740 1 --2- 172.21.15.5:0/3389551012 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5565b5a1dc70 0x5565b5a2cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:41.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:41.708+0000 7f31a4c00640 0 client.20085 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T10:03:41.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:41.708+0000 7f31c46a9740 1 -- 172.21.15.5:0/3389551012 >> 172.21.15.5:0/3389551012 conn(0x5565b5a1a200 msgr2=0x5565b5a1fee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:41.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:41.708+0000 7f31c46a9740 1 -- 172.21.15.5:0/3389551012 shutdown_connections 2024-09-10T10:03:41.813 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-10T10:03:41.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:41.708+0000 7f31c46a9740 1 -- 172.21.15.5:0/3389551012 wait complete. 2024-09-10T10:03:41.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:41.708+0000 7f31c46a9740 20 client.20085 trim_cache size 0 max 16384 2024-09-10T10:03:41.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:41.709+0000 7f31c1c00640 1 -- 172.21.15.5:0/468602211 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f31a00a2b80 msgr2=0x7f31a00a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:41.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:41.709+0000 7f31c1c00640 1 --2- 172.21.15.5:0/468602211 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f31a00a2b80 0x7f31a00a5040 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f31b4006010 tx=0x7f31b402f390 comp rx=0 tx=0).stop 2024-09-10T10:03:41.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:41.709+0000 7f31c1c00640 1 -- 172.21.15.5:0/468602211 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f31bc076720 msgr2=0x7f31bc19eb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:41.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:41.709+0000 7f31c1c00640 1 --2- 172.21.15.5:0/468602211 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f31bc076720 0x7f31bc19eb90 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f31ac00e9f0 tx=0x7f31ac00edf0 comp rx=0 tx=0).stop 2024-09-10T10:03:41.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:41.709+0000 7f31c1c00640 1 -- 172.21.15.5:0/468602211 shutdown_connections 2024-09-10T10:03:41.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:41.709+0000 7f31c1c00640 1 --2- 172.21.15.5:0/468602211 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f31a00a2b80 0x7f31a00a5040 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:41.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:41.709+0000 7f31c1c00640 1 --2- 172.21.15.5:0/468602211 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f31bc112a50 0x7f31bc1a3490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:41.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:41.709+0000 7f31c1c00640 1 --2- 172.21.15.5:0/468602211 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f31bc077040 0x7f31bc19f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:41.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:41.709+0000 7f31c1c00640 1 --2- 172.21.15.5:0/468602211 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f31bc076720 0x7f31bc19eb90 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:41.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:41.709+0000 7f31c1c00640 1 -- 172.21.15.5:0/468602211 >> 172.21.15.5:0/468602211 conn(0x7f31bc0ffa50 msgr2=0x7f31bc101e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:41.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:41.709+0000 7f31c1c00640 1 -- 172.21.15.5:0/468602211 shutdown_connections 2024-09-10T10:03:41.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:41.709+0000 7f31c1c00640 1 -- 172.21.15.5:0/468602211 wait complete. 2024-09-10T10:03:42.102 INFO:tasks.workunit.client.0.smithi005.stdout: 10 177041 13.28 MB/sec execute 560 sec latency 29128.281 ms 2024-09-10T10:03:43.102 INFO:tasks.workunit.client.0.smithi005.stdout: 10 177041 13.26 MB/sec execute 561 sec latency 30128.356 ms 2024-09-10T10:03:43.367 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:03:43 smithi005 ceph-mon[30350]: pgmap v1154: 129 pgs: 129 active+clean; 791 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 200 KiB/s rd, 3.7 MiB/s wr, 151 op/s 2024-09-10T10:03:43.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:03:43 smithi159 ceph-mon[35513]: pgmap v1154: 129 pgs: 129 active+clean; 791 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 200 KiB/s rd, 3.7 MiB/s wr, 151 op/s 2024-09-10T10:03:43.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:03:43 smithi102 ceph-mon[35442]: pgmap v1154: 129 pgs: 129 active+clean; 791 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 200 KiB/s rd, 3.7 MiB/s wr, 151 op/s 2024-09-10T10:03:43.829 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:03:44.102 INFO:tasks.workunit.client.0.smithi005.stdout: 10 177042 13.23 MB/sec execute 562 sec latency 31076.675 ms 2024-09-10T10:03:44.735 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-10T10:03:44.736 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-10T10:03:45.103 INFO:tasks.workunit.client.0.smithi005.stdout: 10 177042 13.21 MB/sec execute 563 sec latency 32076.740 ms 2024-09-10T10:03:45.306 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:03:45 smithi005 ceph-mon[30350]: pgmap v1155: 129 pgs: 129 active+clean; 787 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 199 KiB/s rd, 3.4 MiB/s wr, 140 op/s 2024-09-10T10:03:45.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:03:45 smithi159 ceph-mon[35513]: pgmap v1155: 129 pgs: 129 active+clean; 787 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 199 KiB/s rd, 3.4 MiB/s wr, 140 op/s 2024-09-10T10:03:45.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:03:45 smithi102 ceph-mon[35442]: pgmap v1155: 129 pgs: 129 active+clean; 787 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 199 KiB/s rd, 3.4 MiB/s wr, 140 op/s 2024-09-10T10:03:46.102 INFO:tasks.workunit.client.0.smithi005.stdout: 10 177044 13.19 MB/sec execute 564 sec latency 32489.979 ms 2024-09-10T10:03:46.394 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:03:46 smithi005 ceph-mon[30350]: from='client.? 172.21.15.5:0/2844673652' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:03:46.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:45.575+0000 7ff068000640 1 -- 172.21.15.5:0/1545119226 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff060101df0 msgr2=0x7ff0601021d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:46.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:45.575+0000 7ff068000640 1 --2- 172.21.15.5:0/1545119226 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff060101df0 0x7ff0601021d0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7ff054009900 tx=0x7ff05402a3f0 comp rx=0 tx=0).stop 2024-09-10T10:03:46.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:45.575+0000 7ff068000640 1 -- 172.21.15.5:0/1545119226 shutdown_connections 2024-09-10T10:03:46.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:45.575+0000 7ff068000640 1 --2- 172.21.15.5:0/1545119226 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff060106f20 0x7ff06010eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:46.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:45.575+0000 7ff068000640 1 --2- 172.21.15.5:0/1545119226 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff0601027a0 0x7ff0601067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:46.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:45.575+0000 7ff068000640 1 --2- 172.21.15.5:0/1545119226 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff060101df0 0x7ff0601021d0 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:46.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:45.575+0000 7ff068000640 1 -- 172.21.15.5:0/1545119226 >> 172.21.15.5:0/1545119226 conn(0x7ff0600fd8b0 msgr2=0x7ff0600ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:46.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:45.575+0000 7ff068000640 1 -- 172.21.15.5:0/1545119226 shutdown_connections 2024-09-10T10:03:46.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:45.575+0000 7ff068000640 1 -- 172.21.15.5:0/1545119226 wait complete. 2024-09-10T10:03:46.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:45.575+0000 7ff068000640 1 Processor -- start 2024-09-10T10:03:46.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:45.575+0000 7ff068000640 1 -- start start 2024-09-10T10:03:46.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:45.575+0000 7ff068000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff060101df0 0x7ff06019a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:46.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:45.575+0000 7ff068000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff0601027a0 0x7ff06019ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:46.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:45.575+0000 7ff068000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff060106f20 0x7ff06019eea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:46.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:45.575+0000 7ff068000640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff060111f80 con 0x7ff0601027a0 2024-09-10T10:03:46.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:45.575+0000 7ff068000640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7ff060111e00 con 0x7ff060101df0 2024-09-10T10:03:46.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:45.575+0000 7ff068000640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7ff060112100 con 0x7ff060106f20 2024-09-10T10:03:46.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:45.575+0000 7ff064c00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff0601027a0 0x7ff06019ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:46.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:45.575+0000 7ff065600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff060101df0 0x7ff06019a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:46.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:45.575+0000 7ff064c00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff0601027a0 0x7ff06019ab10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35058/0 (socket says 172.21.15.5:35058) 2024-09-10T10:03:46.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:45.575+0000 7ff064c00640 1 -- 172.21.15.5:0/2844673652 learned_addr learned my addr 172.21.15.5:0/2844673652 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T10:03:46.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:45.575+0000 7ff066000640 1 --2- 172.21.15.5:0/2844673652 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff060106f20 0x7ff06019eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:46.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:45.575+0000 7ff065600640 1 -- 172.21.15.5:0/2844673652 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff060106f20 msgr2=0x7ff06019eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:46.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:45.575+0000 7ff065600640 1 --2- 172.21.15.5:0/2844673652 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff060106f20 0x7ff06019eea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:46.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:45.575+0000 7ff065600640 1 -- 172.21.15.5:0/2844673652 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff0601027a0 msgr2=0x7ff06019ab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:46.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:45.575+0000 7ff065600640 1 --2- 172.21.15.5:0/2844673652 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff0601027a0 0x7ff06019ab10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:46.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:45.575+0000 7ff065600640 1 -- 172.21.15.5:0/2844673652 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff06019f580 con 0x7ff060101df0 2024-09-10T10:03:46.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:45.575+0000 7ff064c00640 1 --2- 172.21.15.5:0/2844673652 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff0601027a0 0x7ff06019ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:03:46.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:45.575+0000 7ff065600640 1 --2- 172.21.15.5:0/2844673652 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff060101df0 0x7ff06019a5d0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7ff054005940 tx=0x7ff0540040c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:46.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:45.575+0000 7ff05e000640 1 -- 172.21.15.5:0/2844673652 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff05403f030 con 0x7ff060101df0 2024-09-10T10:03:46.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:45.580+0000 7ff068000640 1 -- 172.21.15.5:0/2844673652 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff06019f810 con 0x7ff060101df0 2024-09-10T10:03:46.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:45.580+0000 7ff05e000640 1 -- 172.21.15.5:0/2844673652 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff0540044c0 con 0x7ff060101df0 2024-09-10T10:03:46.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:45.580+0000 7ff05e000640 1 -- 172.21.15.5:0/2844673652 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff054035070 con 0x7ff060101df0 2024-09-10T10:03:46.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:45.580+0000 7ff068000640 1 -- 172.21.15.5:0/2844673652 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0601a7150 con 0x7ff060101df0 2024-09-10T10:03:46.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:45.580+0000 7ff068000640 1 -- 172.21.15.5:0/2844673652 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff0601021d0 con 0x7ff060101df0 2024-09-10T10:03:46.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:45.580+0000 7ff05e000640 1 -- 172.21.15.5:0/2844673652 <== mon.2 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7ff054042030 con 0x7ff060101df0 2024-09-10T10:03:46.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:45.580+0000 7ff05e000640 1 --2- 172.21.15.5:0/2844673652 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7ff0480a2b80 0x7ff0480a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:46.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:45.580+0000 7ff05e000640 1 -- 172.21.15.5:0/2844673652 <== mon.2 v2:172.21.15.102:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7ff0540efa10 con 0x7ff060101df0 2024-09-10T10:03:46.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:45.585+0000 7ff064c00640 1 --2- 172.21.15.5:0/2844673652 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7ff0480a2b80 0x7ff0480a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:46.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:45.585+0000 7ff064c00640 1 --2- 172.21.15.5:0/2844673652 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7ff0480a2b80 0x7ff0480a5040 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7ff0600fff80 tx=0x7ff04c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:46.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:45.585+0000 7ff05e000640 1 -- 172.21.15.5:0/2844673652 <== mon.2 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff0540bbed0 con 0x7ff060101df0 2024-09-10T10:03:46.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:45.812+0000 7ff068000640 1 -- 172.21.15.5:0/2844673652 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff06019b930 con 0x7ff060101df0 2024-09-10T10:03:46.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:45.813+0000 7ff05e000640 1 -- 172.21.15.5:0/2844673652 <== mon.2 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 42 v42) ==== 94+0+11969 (secure 0 0 0) 0x7ff0540c01d0 con 0x7ff060101df0 2024-09-10T10:03:46.446 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-10T10:03:46.446 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":42,"btime":"2024-09-10T09:55:06:670916+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14454,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/2970594311","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2970594311},{"type":"v1","addr":"172.21.15.5:6835","nonce":2970594311}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14514,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6839/3616502174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":3616502174},{"type":"v1","addr":"172.21.15.5:6839","nonce":3616502174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":42,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T09:35:29.626287+0000","modified":"2024-09-10T09:55:06.419829+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24542,"mds_1":24535,"mds_2":24511,"mds_3":24506,"mds_4":24443},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14520":{"gid":14520,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/2073599842","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":2073599842},{"type":"v1","addr":"172.21.15.5:6841","nonce":2073599842}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"k","rank":4,"incarnation":26,"state":"up:active","state_seq":24,"addr":"172.21.15.102:6833/2457007427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":2457007427},{"type":"v1","addr":"172.21.15.102:6833","nonce":2457007427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/356445159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":356445159},{"type":"v1","addr":"172.21.15.159:6833","nonce":356445159}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6837/2894314507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":2894314507},{"type":"v1","addr":"172.21.15.5:6837","nonce":2894314507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.102:6835/1647549111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":1647549111},{"type":"v1","addr":"172.21.15.102:6835","nonce":1647549111}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6835/952657242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":952657242},{"type":"v1","addr":"172.21.15.159:6835","nonce":952657242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":19,"addr":"172.21.15.102:6837/2689776764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":2689776764},{"type":"v1","addr":"172.21.15.102:6837","nonce":2689776764}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.159:6837/620627639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":620627639},{"type":"v1","addr":"172.21.15.159:6837","nonce":620627639}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24535":{"gid":24535,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":18,"addr":"172.21.15.159:6839/2073739599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":2073739599},{"type":"v1","addr":"172.21.15.159:6839","nonce":2073739599}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24542":{"gid":24542,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.102:6839/335409749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6838","nonce":335409749},{"type":"v1","addr":"172.21.15.102:6839","nonce":335409749}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24542,"qdb_cluster":[24542,24535,24511,24506,24443]},"id":1}]} 2024-09-10T10:03:46.446 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 42 2024-09-10T10:03:46.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:45.816+0000 7ff068000640 1 -- 172.21.15.5:0/2844673652 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7ff0480a2b80 msgr2=0x7ff0480a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:46.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:45.816+0000 7ff068000640 1 --2- 172.21.15.5:0/2844673652 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7ff0480a2b80 0x7ff0480a5040 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7ff0600fff80 tx=0x7ff04c005f90 comp rx=0 tx=0).stop 2024-09-10T10:03:46.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:45.816+0000 7ff068000640 1 -- 172.21.15.5:0/2844673652 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff060101df0 msgr2=0x7ff06019a5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:46.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:45.816+0000 7ff068000640 1 --2- 172.21.15.5:0/2844673652 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff060101df0 0x7ff06019a5d0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7ff054005940 tx=0x7ff0540040c0 comp rx=0 tx=0).stop 2024-09-10T10:03:46.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:45.816+0000 7ff068000640 1 -- 172.21.15.5:0/2844673652 shutdown_connections 2024-09-10T10:03:46.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:45.816+0000 7ff068000640 1 --2- 172.21.15.5:0/2844673652 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7ff0480a2b80 0x7ff0480a5040 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:46.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:45.816+0000 7ff068000640 1 --2- 172.21.15.5:0/2844673652 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff060106f20 0x7ff06019eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:46.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:45.816+0000 7ff068000640 1 --2- 172.21.15.5:0/2844673652 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff0601027a0 0x7ff06019ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:46.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:45.816+0000 7ff068000640 1 --2- 172.21.15.5:0/2844673652 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff060101df0 0x7ff06019a5d0 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:46.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:45.816+0000 7ff068000640 1 -- 172.21.15.5:0/2844673652 >> 172.21.15.5:0/2844673652 conn(0x7ff0600fd8b0 msgr2=0x7ff0600fe790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:46.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:45.816+0000 7ff068000640 1 -- 172.21.15.5:0/2844673652 shutdown_connections 2024-09-10T10:03:46.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:45.816+0000 7ff068000640 1 -- 172.21.15.5:0/2844673652 wait complete. 2024-09-10T10:03:46.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:03:46 smithi159 ceph-mon[35513]: from='client.? 172.21.15.5:0/2844673652' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:03:46.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:03:46 smithi102 ceph-mon[35442]: from='client.? 172.21.15.5:0/2844673652' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:03:47.103 INFO:tasks.workunit.client.0.smithi005.stdout: 10 177044 13.16 MB/sec execute 565 sec latency 30166.962 ms 2024-09-10T10:03:47.534 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:03:47 smithi005 ceph-mon[30350]: pgmap v1156: 129 pgs: 129 active+clean; 780 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 255 KiB/s rd, 3.0 MiB/s wr, 133 op/s 2024-09-10T10:03:47.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:03:47 smithi159 ceph-mon[35513]: pgmap v1156: 129 pgs: 129 active+clean; 780 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 255 KiB/s rd, 3.0 MiB/s wr, 133 op/s 2024-09-10T10:03:47.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:03:47 smithi102 ceph-mon[35442]: pgmap v1156: 129 pgs: 129 active+clean; 780 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 255 KiB/s rd, 3.0 MiB/s wr, 133 op/s 2024-09-10T10:03:47.900 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24542 quiesce db / --timeout 90 --expiration 150.0 --await-for 45.0' 2024-09-10T10:03:47.900 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24542 quiesce db / --timeout 90 --expiration 150.0 --await-for 45.0 2024-09-10T10:03:48.103 INFO:tasks.workunit.client.0.smithi005.stdout: 10 177394 13.17 MB/sec execute 566 sec latency 30570.018 ms 2024-09-10T10:03:48.363 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:03:48.453 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:03:49.105 INFO:tasks.workunit.client.0.smithi005.stdout: 10 177989 13.20 MB/sec execute 567 sec latency 88.182 ms 2024-09-10T10:03:49.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:49.267+0000 7f3873800640 1 -- 172.21.15.5:0/2706312104 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f386c103ff0 msgr2=0x7f386c1043d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:49.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:49.267+0000 7f3873800640 1 --2- 172.21.15.5:0/2706312104 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f386c103ff0 0x7f386c1043d0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f3860009900 tx=0x7f386002a3f0 comp rx=0 tx=0).stop 2024-09-10T10:03:49.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:49.268+0000 7f3873800640 1 -- 172.21.15.5:0/2706312104 shutdown_connections 2024-09-10T10:03:49.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:49.268+0000 7f3873800640 1 --2- 172.21.15.5:0/2706312104 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f386c109120 0x7f386c110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:49.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:49.268+0000 7f3873800640 1 --2- 172.21.15.5:0/2706312104 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f386c1049a0 0x7f386c1089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:49.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:49.268+0000 7f3873800640 1 --2- 172.21.15.5:0/2706312104 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f386c103ff0 0x7f386c1043d0 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:49.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:49.268+0000 7f3873800640 1 -- 172.21.15.5:0/2706312104 >> 172.21.15.5:0/2706312104 conn(0x7f386c0ffa90 msgr2=0x7f386c101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:49.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:49.268+0000 7f3873800640 1 -- 172.21.15.5:0/2706312104 shutdown_connections 2024-09-10T10:03:49.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:49.268+0000 7f3873800640 1 -- 172.21.15.5:0/2706312104 wait complete. 2024-09-10T10:03:49.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:49.269+0000 7f3873800640 1 Processor -- start 2024-09-10T10:03:49.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:49.269+0000 7f3873800640 1 -- start start 2024-09-10T10:03:49.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:49.270+0000 7f3873800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f386c103ff0 0x7f386c076700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:49.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:49.270+0000 7f3873800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f386c1049a0 0x7f386c076c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:49.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:49.270+0000 7f3873800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f386c109120 0x7f386c077180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:49.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:49.270+0000 7f3873800640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f386c1141a0 con 0x7f386c109120 2024-09-10T10:03:49.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:49.270+0000 7f3873800640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f386c114020 con 0x7f386c1049a0 2024-09-10T10:03:49.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:49.270+0000 7f3873800640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f386c114320 con 0x7f386c103ff0 2024-09-10T10:03:49.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:49.270+0000 7f3870e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f386c103ff0 0x7f386c076700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:49.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:49.270+0000 7f386be00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f386c1049a0 0x7f386c076c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:49.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:49.270+0000 7f386be00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f386c1049a0 0x7f386c076c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:53746/0 (socket says 172.21.15.5:53746) 2024-09-10T10:03:49.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:49.270+0000 7f3870e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f386c103ff0 0x7f386c076700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:56062/0 (socket says 172.21.15.5:56062) 2024-09-10T10:03:49.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:49.270+0000 7f386be00640 1 -- 172.21.15.5:0/779011700 learned_addr learned my addr 172.21.15.5:0/779011700 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T10:03:49.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:49.270+0000 7f3871800640 1 --2- 172.21.15.5:0/779011700 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f386c109120 0x7f386c077180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:49.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:49.271+0000 7f3870e00640 1 -- 172.21.15.5:0/779011700 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f386c1049a0 msgr2=0x7f386c076c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:49.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:49.271+0000 7f3870e00640 1 --2- 172.21.15.5:0/779011700 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f386c1049a0 0x7f386c076c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:49.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:49.271+0000 7f3870e00640 1 -- 172.21.15.5:0/779011700 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f386c109120 msgr2=0x7f386c077180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:49.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:49.271+0000 7f3870e00640 1 --2- 172.21.15.5:0/779011700 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f386c109120 0x7f386c077180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:49.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:49.271+0000 7f3870e00640 1 -- 172.21.15.5:0/779011700 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f386c1aaca0 con 0x7f386c103ff0 2024-09-10T10:03:49.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:49.271+0000 7f3871800640 1 --2- 172.21.15.5:0/779011700 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f386c109120 0x7f386c077180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:03:49.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:49.271+0000 7f386be00640 1 --2- 172.21.15.5:0/779011700 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f386c1049a0 0x7f386c076c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:03:49.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:49.271+0000 7f3870e00640 1 --2- 172.21.15.5:0/779011700 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f386c103ff0 0x7f386c076700 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f3860009230 tx=0x7f3860003b20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:49.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:49.271+0000 7f3869600640 1 -- 172.21.15.5:0/779011700 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f386003f070 con 0x7f386c103ff0 2024-09-10T10:03:49.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:49.272+0000 7f3873800640 1 -- 172.21.15.5:0/779011700 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f386c1aaf30 con 0x7f386c103ff0 2024-09-10T10:03:49.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:49.272+0000 7f3869600640 1 -- 172.21.15.5:0/779011700 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3860031c90 con 0x7f386c103ff0 2024-09-10T10:03:49.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:49.272+0000 7f3869600640 1 -- 172.21.15.5:0/779011700 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3860035030 con 0x7f386c103ff0 2024-09-10T10:03:49.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:49.272+0000 7f3873800640 1 -- 172.21.15.5:0/779011700 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f386c1ab3e0 con 0x7f386c103ff0 2024-09-10T10:03:49.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:49.275+0000 7f3869600640 1 -- 172.21.15.5:0/779011700 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f3860042070 con 0x7f386c103ff0 2024-09-10T10:03:49.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:49.276+0000 7f38763ca740 1 Processor -- start 2024-09-10T10:03:49.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:49.277+0000 7f38763ca740 1 -- start start 2024-09-10T10:03:49.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:49.277+0000 7f38763ca740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56469370eac0 0x56469370eea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:49.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:49.277+0000 7f38763ca740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56469370f3e0 0x564693719760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:49.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:49.277+0000 7f3869600640 1 --2- 172.21.15.5:0/779011700 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f38540a2c00 0x7f38540a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:49.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:49.277+0000 7f38763ca740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564693719ca0 0x56469371c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:49.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:49.277+0000 7f38763ca740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x564693675290 con 0x56469370eac0 2024-09-10T10:03:49.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:49.277+0000 7f38763ca740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x56469361cb40 con 0x56469370f3e0 2024-09-10T10:03:49.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:49.277+0000 7f38763ca740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x564693671250 con 0x564693719ca0 2024-09-10T10:03:49.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:49.277+0000 7f3869600640 1 -- 172.21.15.5:0/779011700 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f38600ef5f0 con 0x7f386c103ff0 2024-09-10T10:03:49.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:49.278+0000 7f3871800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56469370eac0 0x56469370eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:49.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:49.278+0000 7f3871800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564693719ca0 0x56469371c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:49.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:49.278+0000 7f3871800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564693719ca0 0x56469371c090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:56068/0 (socket says 172.21.15.5:56068) 2024-09-10T10:03:49.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:49.278+0000 7f3871800640 1 -- 172.21.15.5:0/3679304098 learned_addr learned my addr 172.21.15.5:0/3679304098 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T10:03:49.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:49.278+0000 7f3870e00640 1 --2- 172.21.15.5:0/779011700 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f38540a2c00 0x7f38540a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:49.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:49.278+0000 7f3871800640 1 -- 172.21.15.5:0/3679304098 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56469370f3e0 msgr2=0x564693719760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:03:49.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:49.278+0000 7f386be00640 1 --2- 172.21.15.5:0/3679304098 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56469370f3e0 0x564693719760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:49.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:49.278+0000 7f3871800640 1 --2- 172.21.15.5:0/3679304098 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56469370f3e0 0x564693719760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:49.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:49.278+0000 7f3871800640 1 -- 172.21.15.5:0/3679304098 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56469370eac0 msgr2=0x56469370eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:03:49.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:49.278+0000 7f3871800640 1 --2- 172.21.15.5:0/3679304098 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56469370eac0 0x56469370eea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:49.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:49.278+0000 7f3871800640 1 -- 172.21.15.5:0/3679304098 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5646933e66a0 con 0x564693719ca0 2024-09-10T10:03:49.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:49.279+0000 7f386be00640 1 --2- 172.21.15.5:0/3679304098 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56469370f3e0 0x564693719760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:03:49.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:49.279+0000 7f3871800640 1 --2- 172.21.15.5:0/3679304098 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564693719ca0 0x56469371c090 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f3864012ae0 tx=0x7f3864009500 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f2db28b3d931ab47 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:49.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:49.279+0000 7f385e000640 1 -- 172.21.15.5:0/3679304098 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3864012110 con 0x564693719ca0 2024-09-10T10:03:49.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:49.279+0000 7f385e000640 1 -- 172.21.15.5:0/3679304098 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f386400f940 con 0x564693719ca0 2024-09-10T10:03:49.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:49.279+0000 7f385e000640 1 -- 172.21.15.5:0/3679304098 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38640053a0 con 0x564693719ca0 2024-09-10T10:03:49.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:49.280+0000 7f3870e00640 1 --2- 172.21.15.5:0/779011700 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f38540a2c00 0x7f38540a50c0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f386002a9b0 tx=0x7f3860000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:49.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:49.281+0000 7f38763ca740 1 -- 172.21.15.5:0/3679304098 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564693719ca0 msgr2=0x56469371c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:49.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:49.281+0000 7f38763ca740 1 --2- 172.21.15.5:0/3679304098 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564693719ca0 0x56469371c090 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f3864012ae0 tx=0x7f3864009500 comp rx=0 tx=0).stop 2024-09-10T10:03:49.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:49.281+0000 7f38763ca740 1 -- 172.21.15.5:0/3679304098 shutdown_connections 2024-09-10T10:03:49.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:49.281+0000 7f38763ca740 1 --2- 172.21.15.5:0/3679304098 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x564693719ca0 0x56469371c090 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:49.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:49.281+0000 7f38763ca740 1 --2- 172.21.15.5:0/3679304098 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56469370f3e0 0x564693719760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:49.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:49.281+0000 7f38763ca740 1 --2- 172.21.15.5:0/3679304098 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56469370eac0 0x56469370eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:49.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:49.281+0000 7f38763ca740 1 -- 172.21.15.5:0/3679304098 >> 172.21.15.5:0/3679304098 conn(0x56469370b050 msgr2=0x56469370b430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:49.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:49.281+0000 7f38763ca740 1 -- 172.21.15.5:0/3679304098 shutdown_connections 2024-09-10T10:03:49.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:49.281+0000 7f38763ca740 1 -- 172.21.15.5:0/3679304098 wait complete. 2024-09-10T10:03:49.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:49.282+0000 7f38763ca740 1 -- start start 2024-09-10T10:03:49.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:49.282+0000 7f38763ca740 1 Processor -- start 2024-09-10T10:03:49.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:49.283+0000 7f38763ca740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56469370eac0 0x564693719630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:49.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:49.283+0000 7f38763ca740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56469370f3e0 0x5646937168e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:49.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:49.283+0000 7f38763ca740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564693719ca0 0x564693716e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:49.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:49.283+0000 7f38763ca740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x56469371c910 con 0x56469370f3e0 2024-09-10T10:03:49.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:49.283+0000 7f38763ca740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x56469362ffa0 con 0x564693719ca0 2024-09-10T10:03:49.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:49.283+0000 7f38763ca740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x56469371ca90 con 0x56469370eac0 2024-09-10T10:03:49.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:49.283+0000 7f38763ca740 10 client.0 _ll_register_callbacks cb 0x564693383ba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:03:49.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:49.283+0000 7f3871800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56469370eac0 0x564693719630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:49.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:49.283+0000 7f3871800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56469370eac0 0x564693719630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:56072/0 (socket says 172.21.15.5:56072) 2024-09-10T10:03:49.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:49.283+0000 7f3871800640 1 -- 172.21.15.5:0/3670705129 learned_addr learned my addr 172.21.15.5:0/3670705129 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T10:03:49.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:49.283+0000 7f3871800640 1 --2- 172.21.15.5:0/3670705129 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564693719ca0 0x564693716e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:49.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:49.283+0000 7f386be00640 1 --2- 172.21.15.5:0/3670705129 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56469370f3e0 0x5646937168e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:49.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:49.284+0000 7f3871800640 1 -- 172.21.15.5:0/3670705129 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564693719ca0 msgr2=0x564693716e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:03:49.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:49.284+0000 7f3871800640 1 --2- 172.21.15.5:0/3670705129 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564693719ca0 0x564693716e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:49.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:49.284+0000 7f3871800640 1 -- 172.21.15.5:0/3670705129 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56469370f3e0 msgr2=0x5646937168e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:03:49.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:49.284+0000 7f3871800640 1 --2- 172.21.15.5:0/3670705129 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56469370f3e0 0x5646937168e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:49.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:49.284+0000 7f3871800640 1 -- 172.21.15.5:0/3670705129 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564693717690 con 0x56469370eac0 2024-09-10T10:03:49.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:49.285+0000 7f386be00640 1 --2- 172.21.15.5:0/3670705129 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56469370f3e0 0x5646937168e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:03:49.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:49.285+0000 7f3871800640 1 --2- 172.21.15.5:0/3670705129 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56469370eac0 0x564693719630 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f3864005c40 tx=0x7f38640098e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=69e2dc22323abcaf server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:49.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:49.285+0000 7f3852a00640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-09-10T10:03:49.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:49.286+0000 7f3852a00640 1 -- 172.21.15.5:0/3670705129 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3864005120 con 0x56469370eac0 2024-09-10T10:03:49.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:49.286+0000 7f3852a00640 1 -- 172.21.15.5:0/3670705129 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f38640052c0 con 0x56469370eac0 2024-09-10T10:03:49.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:49.286+0000 7f38763ca740 1 -- 172.21.15.5:0/3670705129 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f386400f940 con 0x56469370eac0 2024-09-10T10:03:49.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:49.290+0000 7f3852a00640 1 -- 172.21.15.5:0/3670705129 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f38640054c0 con 0x56469370eac0 2024-09-10T10:03:49.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:49.290+0000 7f38763ca740 10 client.30178 fetch_fsmap learned FSMap version 42 2024-09-10T10:03:49.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:49.290+0000 7f38763ca740 1 -- 172.21.15.5:0/3670705129 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x564693728800 con 0x56469370eac0 2024-09-10T10:03:49.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:49.291+0000 7f3852a00640 1 -- 172.21.15.5:0/3670705129 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f386400f940 con 0x56469370eac0 2024-09-10T10:03:49.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:49.291+0000 7f3852a00640 1 -- 172.21.15.5:0/3670705129 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 42) ==== 8370+0+0 (secure 0 0 0) 0x7f386401fc20 con 0x56469370eac0 2024-09-10T10:03:49.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:49.291+0000 7f38763ca740 10 client.30178 fetch_fsmap finished waiting for FSMap version 42 2024-09-10T10:03:49.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:49.291+0000 7f38763ca740 10 client.30178 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T10:03:49.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:49.291+0000 7f38763ca740 20 client.30178 populate_metadata read hostname 'smithi005' 2024-09-10T10:03:49.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:49.291+0000 7f38763ca740 1 --2- 172.21.15.5:0/3670705129 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x564693729b50 0x56469372bf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:49.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:49.291+0000 7f38763ca740 4 client.30178 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:03:49.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:49.291+0000 7f38763ca740 1 -- 172.21.15.5:0/3670705129 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564693682800 con 0x564693729b50 2024-09-10T10:03:49.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:49.292+0000 7f386be00640 1 --2- 172.21.15.5:0/3670705129 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x564693729b50 0x56469372bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:49.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:49.293+0000 7f386be00640 1 --2- 172.21.15.5:0/3670705129 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x564693729b50 0x56469372bf40 crc :-1 s=READY pgs=339 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4691c5f8fe75848 server_cookie=b5942dbb503ef225 in_seq=0 out_seq=0 2024-09-10T10:03:49.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:49.293+0000 7f3852a00640 10 client.30178 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T10:03:49.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:49.293+0000 7f3852a00640 1 -- 172.21.15.5:0/3670705129 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564693682800 con 0x564693729b50 2024-09-10T10:03:49.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:49.293+0000 7f3852a00640 10 client.30178 handle_command_reply: tid=0 2024-09-10T10:03:49.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:49.294+0000 7f38763ca740 1 client.30178 shutdown 2024-09-10T10:03:49.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:49.294+0000 7f38763ca740 1 -- 172.21.15.5:0/3670705129 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56469370eac0 msgr2=0x564693719630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:49.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:49.294+0000 7f38763ca740 1 --2- 172.21.15.5:0/3670705129 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56469370eac0 0x564693719630 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f3864005c40 tx=0x7f38640098e0 comp rx=0 tx=0).stop 2024-09-10T10:03:49.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:49.294+0000 7f38763ca740 1 -- 172.21.15.5:0/3670705129 shutdown_connections 2024-09-10T10:03:49.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:49.294+0000 7f38763ca740 1 --2- 172.21.15.5:0/3670705129 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x564693729b50 0x56469372bf40 crc :-1 s=READY pgs=339 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:49.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:49.294+0000 7f38763ca740 1 --2- 172.21.15.5:0/3670705129 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564693719ca0 0x564693716e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:49.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:49.294+0000 7f38763ca740 1 --2- 172.21.15.5:0/3670705129 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56469370f3e0 0x5646937168e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:49.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:49.294+0000 7f38763ca740 1 --2- 172.21.15.5:0/3670705129 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56469370eac0 0x564693719630 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:49.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:49.294+0000 7f38763ca740 1 -- 172.21.15.5:0/3670705129 >> 172.21.15.5:0/3670705129 conn(0x56469370b050 msgr2=0x564693710d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:49.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:49.294+0000 7f3852a00640 0 client.30178 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T10:03:49.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:49.294+0000 7f38763ca740 1 -- 172.21.15.5:0/3670705129 shutdown_connections 2024-09-10T10:03:49.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:49.294+0000 7f38763ca740 1 -- 172.21.15.5:0/3670705129 wait complete. 2024-09-10T10:03:49.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:49.295+0000 7f38763ca740 20 client.30178 trim_cache size 0 max 16384 2024-09-10T10:03:49.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:49.320+0000 7f38763ca740 1 Processor -- start 2024-09-10T10:03:49.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:49.320+0000 7f38763ca740 1 -- start start 2024-09-10T10:03:49.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:49.320+0000 7f38763ca740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56469370eac0 0x564693718850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:49.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:49.320+0000 7f38763ca740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56469370f3e0 0x564693731630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:49.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:49.320+0000 7f38763ca740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564693719ca0 0x564693731b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:49.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:49.320+0000 7f38763ca740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f386c1b54f0 con 0x564693719ca0 2024-09-10T10:03:49.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:49.320+0000 7f38763ca740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f386c1b7030 con 0x56469370f3e0 2024-09-10T10:03:49.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:49.320+0000 7f38763ca740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f386c1b5bc0 con 0x56469370eac0 2024-09-10T10:03:49.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:49.321+0000 7f386be00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56469370f3e0 0x564693731630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:49.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:49.321+0000 7f3871800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56469370eac0 0x564693718850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:49.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:49.321+0000 7f386be00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56469370f3e0 0x564693731630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:53764/0 (socket says 172.21.15.5:53764) 2024-09-10T10:03:49.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:49.321+0000 7f386be00640 1 -- 172.21.15.5:0/268334073 learned_addr learned my addr 172.21.15.5:0/268334073 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T10:03:49.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:49.321+0000 7f3871800640 1 --2- 172.21.15.5:0/268334073 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564693719ca0 0x564693731b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:49.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:49.321+0000 7f386be00640 1 -- 172.21.15.5:0/268334073 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56469370eac0 msgr2=0x564693718850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:03:49.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:49.321+0000 7f386be00640 1 --2- 172.21.15.5:0/268334073 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56469370eac0 0x564693718850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:49.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:49.321+0000 7f386be00640 1 -- 172.21.15.5:0/268334073 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564693719ca0 msgr2=0x564693731b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:03:49.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:49.321+0000 7f386be00640 1 --2- 172.21.15.5:0/268334073 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564693719ca0 0x564693731b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:49.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:49.321+0000 7f386be00640 1 -- 172.21.15.5:0/268334073 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564693726550 con 0x56469370f3e0 2024-09-10T10:03:49.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:49.321+0000 7f3871800640 1 --2- 172.21.15.5:0/268334073 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56469370eac0 0x564693718850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:03:49.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:49.321+0000 7f386be00640 1 --2- 172.21.15.5:0/268334073 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56469370f3e0 0x564693731630 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x56469370beb0 tx=0x7f3858004410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5f7f25a92b08080d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:49.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:49.322+0000 7f385d600640 1 -- 172.21.15.5:0/268334073 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3858003b20 con 0x56469370f3e0 2024-09-10T10:03:49.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:49.322+0000 7f385d600640 1 -- 172.21.15.5:0/268334073 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3858003cc0 con 0x56469370f3e0 2024-09-10T10:03:49.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:49.322+0000 7f385d600640 1 -- 172.21.15.5:0/268334073 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f385801d7c0 con 0x56469370f3e0 2024-09-10T10:03:49.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:49.323+0000 7f38763ca740 1 -- 172.21.15.5:0/268334073 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56469370f3e0 msgr2=0x564693731630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:49.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:49.323+0000 7f38763ca740 1 --2- 172.21.15.5:0/268334073 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56469370f3e0 0x564693731630 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x56469370beb0 tx=0x7f3858004410 comp rx=0 tx=0).stop 2024-09-10T10:03:49.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:49.323+0000 7f38763ca740 1 -- 172.21.15.5:0/268334073 shutdown_connections 2024-09-10T10:03:49.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:49.323+0000 7f38763ca740 1 --2- 172.21.15.5:0/268334073 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564693719ca0 0x564693731b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:49.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:49.323+0000 7f38763ca740 1 --2- 172.21.15.5:0/268334073 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x56469370f3e0 0x564693731630 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:49.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:49.323+0000 7f38763ca740 1 --2- 172.21.15.5:0/268334073 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56469370eac0 0x564693718850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:49.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:49.323+0000 7f38763ca740 1 -- 172.21.15.5:0/268334073 >> 172.21.15.5:0/268334073 conn(0x56469370b050 msgr2=0x564693718140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:49.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:49.323+0000 7f38763ca740 1 -- 172.21.15.5:0/268334073 shutdown_connections 2024-09-10T10:03:49.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:49.323+0000 7f38763ca740 1 -- 172.21.15.5:0/268334073 wait complete. 2024-09-10T10:03:49.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:49.324+0000 7f38763ca740 1 -- start start 2024-09-10T10:03:49.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:49.324+0000 7f38763ca740 1 Processor -- start 2024-09-10T10:03:49.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:49.324+0000 7f38763ca740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56469370eac0 0x564693722580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:49.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:49.325+0000 7f38763ca740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56469370f3e0 0x56469371d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:49.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:49.325+0000 7f38763ca740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564693719ca0 0x56469371dea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:49.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:49.325+0000 7f38763ca740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f386c1b6960 con 0x56469370eac0 2024-09-10T10:03:49.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:49.325+0000 7f38763ca740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f386c1b6290 con 0x564693719ca0 2024-09-10T10:03:49.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:49.325+0000 7f38763ca740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f386c1b34d0 con 0x56469370f3e0 2024-09-10T10:03:49.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:49.325+0000 7f38763ca740 10 client.0 _ll_register_callbacks cb 0x7f38640125b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:03:49.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:49.325+0000 7f3871800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56469370eac0 0x564693722580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:49.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:49.325+0000 7f386be00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56469370f3e0 0x56469371d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:49.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:49.325+0000 7f386be00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56469370f3e0 0x56469371d8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:56082/0 (socket says 172.21.15.5:56082) 2024-09-10T10:03:49.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:49.325+0000 7f386be00640 1 -- 172.21.15.5:0/2376396939 learned_addr learned my addr 172.21.15.5:0/2376396939 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T10:03:49.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:49.325+0000 7f3871800640 1 --2- 172.21.15.5:0/2376396939 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564693719ca0 0x56469371dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:49.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:49.326+0000 7f3871800640 1 -- 172.21.15.5:0/2376396939 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56469370f3e0 msgr2=0x56469371d8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:03:49.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:49.326+0000 7f3871800640 1 --2- 172.21.15.5:0/2376396939 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56469370f3e0 0x56469371d8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:49.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:49.326+0000 7f3871800640 1 -- 172.21.15.5:0/2376396939 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564693719ca0 msgr2=0x56469371dea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:03:49.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:49.326+0000 7f3871800640 1 --2- 172.21.15.5:0/2376396939 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564693719ca0 0x56469371dea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:49.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:49.326+0000 7f3871800640 1 -- 172.21.15.5:0/2376396939 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56469371e6d0 con 0x56469370eac0 2024-09-10T10:03:49.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:49.326+0000 7f3871800640 1 --2- 172.21.15.5:0/2376396939 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56469370eac0 0x564693722580 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f3864015020 tx=0x7f386401b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed0aa859f93e7d4c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:49.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:49.326+0000 7f3852a00640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-10T10:03:49.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:49.327+0000 7f3852a00640 1 -- 172.21.15.5:0/2376396939 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3864031040 con 0x56469370eac0 2024-09-10T10:03:49.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:49.327+0000 7f38763ca740 1 -- 172.21.15.5:0/2376396939 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3858003cc0 con 0x56469370eac0 2024-09-10T10:03:49.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:49.327+0000 7f3852a00640 1 -- 172.21.15.5:0/2376396939 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f3864009a30 con 0x56469370eac0 2024-09-10T10:03:49.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:49.327+0000 7f3852a00640 1 -- 172.21.15.5:0/2376396939 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f3864012070 con 0x56469370eac0 2024-09-10T10:03:49.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:49.327+0000 7f38763ca740 10 client.20109 fetch_fsmap learned FSMap version 42 2024-09-10T10:03:49.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:49.327+0000 7f38763ca740 1 -- 172.21.15.5:0/2376396939 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x564693794750 con 0x56469370eac0 2024-09-10T10:03:49.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:49.328+0000 7f3852a00640 1 -- 172.21.15.5:0/2376396939 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3858003cc0 con 0x56469370eac0 2024-09-10T10:03:49.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:49.328+0000 7f3852a00640 1 -- 172.21.15.5:0/2376396939 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 42) ==== 8370+0+0 (secure 0 0 0) 0x7f386401f460 con 0x56469370eac0 2024-09-10T10:03:49.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:49.328+0000 7f38763ca740 10 client.20109 fetch_fsmap finished waiting for FSMap version 42 2024-09-10T10:03:49.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:49.328+0000 7f38763ca740 10 client.20109 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T10:03:49.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:49.328+0000 7f38763ca740 20 client.20109 populate_metadata read hostname 'smithi005' 2024-09-10T10:03:49.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:49.328+0000 7f38763ca740 1 --2- 172.21.15.5:0/2376396939 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x564693794e20 0x564693797240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:49.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:49.328+0000 7f38763ca740 4 client.20109 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-10T10:03:49.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:49.328+0000 7f38763ca740 1 -- 172.21.15.5:0/2376396939 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x56469367cd10 con 0x564693794e20 2024-09-10T10:03:49.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:49.329+0000 7f386be00640 1 --2- 172.21.15.5:0/2376396939 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x564693794e20 0x564693797240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:49.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:49.329+0000 7f386be00640 1 --2- 172.21.15.5:0/2376396939 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x564693794e20 0x564693797240 crc :-1 s=READY pgs=340 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=88b835e248d4911c server_cookie=466fa2adfc855f25 in_seq=0 out_seq=0 2024-09-10T10:03:49.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:49.329+0000 7f3852a00640 10 client.20109 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T10:03:49.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:49.330+0000 7f3852a00640 1 -- 172.21.15.5:0/2376396939 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x56469367cd10 con 0x564693794e20 2024-09-10T10:03:49.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:49.330+0000 7f3852a00640 10 client.20109 handle_command_reply: tid=0 2024-09-10T10:03:49.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:49.330+0000 7f38763ca740 1 client.20109 shutdown 2024-09-10T10:03:49.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:49.330+0000 7f38763ca740 1 -- 172.21.15.5:0/2376396939 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56469370eac0 msgr2=0x564693722580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:49.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:49.330+0000 7f38763ca740 1 --2- 172.21.15.5:0/2376396939 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56469370eac0 0x564693722580 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f3864015020 tx=0x7f386401b040 comp rx=0 tx=0).stop 2024-09-10T10:03:49.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:49.331+0000 7f38763ca740 1 -- 172.21.15.5:0/2376396939 shutdown_connections 2024-09-10T10:03:49.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:49.331+0000 7f38763ca740 1 --2- 172.21.15.5:0/2376396939 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x564693794e20 0x564693797240 crc :-1 s=READY pgs=340 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:49.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:49.331+0000 7f3852a00640 0 client.20109 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T10:03:49.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:49.331+0000 7f38763ca740 1 --2- 172.21.15.5:0/2376396939 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x564693719ca0 0x56469371dea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:49.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:49.331+0000 7f38763ca740 1 --2- 172.21.15.5:0/2376396939 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x56469370f3e0 0x56469371d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:49.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:49.331+0000 7f38763ca740 1 --2- 172.21.15.5:0/2376396939 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56469370eac0 0x564693722580 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:49.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:49.331+0000 7f38763ca740 1 -- 172.21.15.5:0/2376396939 >> 172.21.15.5:0/2376396939 conn(0x56469370b050 msgr2=0x564693710d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:49.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:49.331+0000 7f38763ca740 1 -- 172.21.15.5:0/2376396939 shutdown_connections 2024-09-10T10:03:49.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:49.331+0000 7f38763ca740 1 -- 172.21.15.5:0/2376396939 wait complete. 2024-09-10T10:03:49.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:49.331+0000 7f38763ca740 20 client.20109 trim_cache size 0 max 16384 2024-09-10T10:03:49.418 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-10T10:03:49.418 INFO:teuthology.orchestra.run.smithi005.stdout: "return_code": 0, 2024-09-10T10:03:49.418 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_tag": "70f0fb62-adae-493b-a035-7ec0d58003d4", 2024-09-10T10:03:49.419 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": "asynchronous" 2024-09-10T10:03:49.419 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-10T10:03:49.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:49.332+0000 7f3873800640 1 -- 172.21.15.5:0/779011700 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f38540a2c00 msgr2=0x7f38540a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:49.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:49.332+0000 7f3873800640 1 --2- 172.21.15.5:0/779011700 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f38540a2c00 0x7f38540a50c0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f386002a9b0 tx=0x7f3860000bc0 comp rx=0 tx=0).stop 2024-09-10T10:03:49.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:49.332+0000 7f3873800640 1 -- 172.21.15.5:0/779011700 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f386c103ff0 msgr2=0x7f386c076700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:49.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:49.332+0000 7f3873800640 1 --2- 172.21.15.5:0/779011700 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f386c103ff0 0x7f386c076700 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f3860009230 tx=0x7f3860003b20 comp rx=0 tx=0).stop 2024-09-10T10:03:49.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:49.332+0000 7f3873800640 1 -- 172.21.15.5:0/779011700 shutdown_connections 2024-09-10T10:03:49.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:49.332+0000 7f3873800640 1 --2- 172.21.15.5:0/779011700 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f38540a2c00 0x7f38540a50c0 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:49.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:49.332+0000 7f3873800640 1 --2- 172.21.15.5:0/779011700 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f386c109120 0x7f386c077180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:49.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:49.332+0000 7f3873800640 1 --2- 172.21.15.5:0/779011700 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f386c1049a0 0x7f386c076c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:49.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:49.332+0000 7f3873800640 1 --2- 172.21.15.5:0/779011700 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f386c103ff0 0x7f386c076700 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:49.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:49.332+0000 7f3873800640 1 -- 172.21.15.5:0/779011700 >> 172.21.15.5:0/779011700 conn(0x7f386c0ffa90 msgr2=0x7f386c100d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:49.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:49.332+0000 7f3873800640 1 -- 172.21.15.5:0/779011700 shutdown_connections 2024-09-10T10:03:49.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:49.333+0000 7f3873800640 1 -- 172.21.15.5:0/779011700 wait complete. 2024-09-10T10:03:49.420 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:03:49 smithi005 ceph-mon[30350]: pgmap v1157: 129 pgs: 129 active+clean; 775 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 2.1 MiB/s wr, 109 op/s 2024-09-10T10:03:49.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:03:49 smithi159 ceph-mon[35513]: pgmap v1157: 129 pgs: 129 active+clean; 775 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 2.1 MiB/s wr, 109 op/s 2024-09-10T10:03:49.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:03:49 smithi102 ceph-mon[35442]: pgmap v1157: 129 pgs: 129 active+clean; 775 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 2.1 MiB/s wr, 109 op/s 2024-09-10T10:03:50.103 INFO:tasks.workunit.client.0.smithi005.stdout: 10 178493 13.23 MB/sec execute 568 sec latency 67.373 ms 2024-09-10T10:03:50.293 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:03:50 smithi159 ceph-mon[35513]: scrub queued for path: / 2024-09-10T10:03:50.293 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:03:50 smithi159 ceph-mon[35513]: scrub summary: idle+waiting paths [/] 2024-09-10T10:03:50.293 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:03:50 smithi159 ceph-mon[35513]: scrub summary: active paths [/] 2024-09-10T10:03:50.293 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:03:50 smithi159 ceph-mon[35513]: scrub summary: active paths [/] 2024-09-10T10:03:50.293 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:03:50 smithi159 ceph-mon[35513]: scrub summary: active paths [/] 2024-09-10T10:03:50.293 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:03:50 smithi159 ceph-mon[35513]: scrub summary: idle+waiting paths [/] 2024-09-10T10:03:50.293 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:03:50 smithi159 ceph-mon[35513]: scrub summary: idle+waiting paths [/] 2024-09-10T10:03:50.343 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:03:50 smithi005 ceph-mon[30350]: scrub queued for path: / 2024-09-10T10:03:50.343 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:03:50 smithi005 ceph-mon[30350]: scrub summary: idle+waiting paths [/] 2024-09-10T10:03:50.343 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:03:50 smithi005 ceph-mon[30350]: scrub summary: active paths [/] 2024-09-10T10:03:50.343 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:03:50 smithi005 ceph-mon[30350]: scrub summary: active paths [/] 2024-09-10T10:03:50.343 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:03:50 smithi005 ceph-mon[30350]: scrub summary: active paths [/] 2024-09-10T10:03:50.343 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:03:50 smithi005 ceph-mon[30350]: scrub summary: idle+waiting paths [/] 2024-09-10T10:03:50.343 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:03:50 smithi005 ceph-mon[30350]: scrub summary: idle+waiting paths [/] 2024-09-10T10:03:50.343 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:03:50 smithi005 ceph-mon[30350]: scrub summary: active paths [/] 2024-09-10T10:03:50.343 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:03:50 smithi005 ceph-mon[30350]: scrub summary: active paths [/] 2024-09-10T10:03:50.343 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:03:50 smithi005 ceph-mon[30350]: scrub summary: idle+waiting paths [/] 2024-09-10T10:03:50.343 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:03:50 smithi005 ceph-mon[30350]: scrub summary: idle+waiting paths [/] 2024-09-10T10:03:50.344 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:03:50 smithi005 ceph-mon[30350]: scrub summary: active paths [/] 2024-09-10T10:03:50.344 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:03:50 smithi005 ceph-mon[30350]: scrub summary: active paths [/] 2024-09-10T10:03:50.344 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:03:50 smithi005 ceph-mon[30350]: scrub summary: idle+waiting paths [/] 2024-09-10T10:03:50.344 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:03:50 smithi005 ceph-mon[30350]: scrub summary: idle+waiting paths [/] 2024-09-10T10:03:50.344 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:03:50 smithi005 ceph-mon[30350]: scrub summary: active paths [/] 2024-09-10T10:03:50.344 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:03:50 smithi005 ceph-mon[30350]: scrub summary: idle+waiting paths [/] 2024-09-10T10:03:50.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:50.401+0000 7fd72f200640 1 -- 172.21.15.5:0/3968938517 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd7301007a0 msgr2=0x7fd730100b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:50.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:50.401+0000 7fd72f200640 1 --2- 172.21.15.5:0/3968938517 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd7301007a0 0x7fd730100b80 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7fd720009ca0 tx=0x7fd72002a8c0 comp rx=0 tx=0).stop 2024-09-10T10:03:50.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:50.401+0000 7fd72f200640 1 -- 172.21.15.5:0/3968938517 shutdown_connections 2024-09-10T10:03:50.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:50.401+0000 7fd72f200640 1 --2- 172.21.15.5:0/3968938517 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd73010e350 0x7fd730110740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:50.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:50.401+0000 7fd72f200640 1 --2- 172.21.15.5:0/3968938517 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd730101150 0x7fd73010de10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:50.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:50.401+0000 7fd72f200640 1 --2- 172.21.15.5:0/3968938517 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd7301007a0 0x7fd730100b80 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:50.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:50.401+0000 7fd72f200640 1 -- 172.21.15.5:0/3968938517 >> 172.21.15.5:0/3968938517 conn(0x7fd7300fc610 msgr2=0x7fd7300fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:50.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:50.401+0000 7fd72f200640 1 -- 172.21.15.5:0/3968938517 shutdown_connections 2024-09-10T10:03:50.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:50.401+0000 7fd72f200640 1 -- 172.21.15.5:0/3968938517 wait complete. 2024-09-10T10:03:50.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:50.402+0000 7fd72f200640 1 Processor -- start 2024-09-10T10:03:50.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:50.403+0000 7fd72f200640 1 -- start start 2024-09-10T10:03:50.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:50.403+0000 7fd72f200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd7301007a0 0x7fd73019a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:50.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:50.403+0000 7fd72f200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd730101150 0x7fd73019aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:50.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:50.403+0000 7fd72f200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd73010e350 0x7fd73019ee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:50.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:50.403+0000 7fd72f200640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd730114040 con 0x7fd730101150 2024-09-10T10:03:50.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:50.403+0000 7fd72f200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fd730113ec0 con 0x7fd73010e350 2024-09-10T10:03:50.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:50.403+0000 7fd72f200640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fd7301141c0 con 0x7fd7301007a0 2024-09-10T10:03:50.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:50.404+0000 7fd72de00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd7301007a0 0x7fd73019a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:50.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:50.404+0000 7fd72d400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd730101150 0x7fd73019aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:50.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:50.404+0000 7fd72e800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd73010e350 0x7fd73019ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:50.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:50.404+0000 7fd72e800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd73010e350 0x7fd73019ee10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:53802/0 (socket says 172.21.15.5:53802) 2024-09-10T10:03:50.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:50.404+0000 7fd72d400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd730101150 0x7fd73019aa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35102/0 (socket says 172.21.15.5:35102) 2024-09-10T10:03:50.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:50.404+0000 7fd72e800640 1 -- 172.21.15.5:0/3384396720 learned_addr learned my addr 172.21.15.5:0/3384396720 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T10:03:50.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:50.404+0000 7fd72e800640 1 -- 172.21.15.5:0/3384396720 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd7301007a0 msgr2=0x7fd73019a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:50.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:50.404+0000 7fd72e800640 1 --2- 172.21.15.5:0/3384396720 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd7301007a0 0x7fd73019a540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:50.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:50.404+0000 7fd72e800640 1 -- 172.21.15.5:0/3384396720 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd730101150 msgr2=0x7fd73019aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:50.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:50.404+0000 7fd72e800640 1 --2- 172.21.15.5:0/3384396720 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd730101150 0x7fd73019aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:50.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:50.404+0000 7fd72e800640 1 -- 172.21.15.5:0/3384396720 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd73019f4f0 con 0x7fd73010e350 2024-09-10T10:03:50.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:50.405+0000 7fd72e800640 1 --2- 172.21.15.5:0/3384396720 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd73010e350 0x7fd73019ee10 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7fd728005fd0 tx=0x7fd72800dbd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:50.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:50.405+0000 7fd726a00640 1 -- 172.21.15.5:0/3384396720 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd728017070 con 0x7fd73010e350 2024-09-10T10:03:50.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:50.405+0000 7fd726a00640 1 -- 172.21.15.5:0/3384396720 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd72800de20 con 0x7fd73010e350 2024-09-10T10:03:50.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:50.405+0000 7fd72f200640 1 -- 172.21.15.5:0/3384396720 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd73019f7e0 con 0x7fd73010e350 2024-09-10T10:03:50.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:50.405+0000 7fd726a00640 1 -- 172.21.15.5:0/3384396720 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd728003ca0 con 0x7fd73010e350 2024-09-10T10:03:50.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:50.406+0000 7fd72f200640 1 -- 172.21.15.5:0/3384396720 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7301a70c0 con 0x7fd73010e350 2024-09-10T10:03:50.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:50.410+0000 7fd726a00640 1 -- 172.21.15.5:0/3384396720 <== mon.2 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7fd728003ec0 con 0x7fd73010e350 2024-09-10T10:03:50.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:50.411+0000 7fd737bfd740 1 Processor -- start 2024-09-10T10:03:50.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:50.411+0000 7fd737bfd740 1 -- start start 2024-09-10T10:03:50.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:50.411+0000 7fd726a00640 1 --2- 172.21.15.5:0/3384396720 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fd7140a2c40 0x7fd7140a5100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:50.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:50.411+0000 7fd726a00640 1 -- 172.21.15.5:0/3384396720 <== mon.2 v2:172.21.15.102:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fd7280cecc0 con 0x7fd73010e350 2024-09-10T10:03:50.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:50.411+0000 7fd737bfd740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cf374c8480 0x55cf374c8860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:50.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:50.411+0000 7fd737bfd740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cf374c8da0 0x55cf374dba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:50.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:50.412+0000 7fd737bfd740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cf374dbf50 0x55cf374de340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:50.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:50.412+0000 7fd737bfd740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55cf3742e530 con 0x55cf374c8480 2024-09-10T10:03:50.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:50.412+0000 7fd737bfd740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55cf373d5860 con 0x55cf374c8da0 2024-09-10T10:03:50.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:50.412+0000 7fd737bfd740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55cf3742a4f0 con 0x55cf374dbf50 2024-09-10T10:03:50.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:50.412+0000 7fd72d400640 1 --2- 172.21.15.5:0/3384396720 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fd7140a2c40 0x7fd7140a5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:50.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:50.412+0000 7fd72de00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cf374c8480 0x55cf374c8860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:50.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:50.412+0000 7fd72de00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cf374c8480 0x55cf374c8860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35110/0 (socket says 172.21.15.5:35110) 2024-09-10T10:03:50.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:50.412+0000 7fd72de00640 1 -- 172.21.15.5:0/1094635691 learned_addr learned my addr 172.21.15.5:0/1094635691 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T10:03:50.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:50.412+0000 7fd72d400640 1 --2- 172.21.15.5:0/1094635691 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cf374c8da0 0x55cf374dba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:50.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:50.412+0000 7fd72e800640 1 --2- 172.21.15.5:0/1094635691 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cf374dbf50 0x55cf374de340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:50.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:50.413+0000 7fd72de00640 1 -- 172.21.15.5:0/1094635691 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cf374dbf50 msgr2=0x55cf374de340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:03:50.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:50.413+0000 7fd72de00640 1 --2- 172.21.15.5:0/1094635691 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cf374dbf50 0x55cf374de340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:50.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:50.413+0000 7fd72de00640 1 -- 172.21.15.5:0/1094635691 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cf374c8da0 msgr2=0x55cf374dba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:03:50.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:50.413+0000 7fd72de00640 1 --2- 172.21.15.5:0/1094635691 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cf374c8da0 0x55cf374dba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:50.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:50.413+0000 7fd72de00640 1 -- 172.21.15.5:0/1094635691 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cf3719fb90 con 0x55cf374c8480 2024-09-10T10:03:50.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:50.413+0000 7fd72d400640 1 --2- 172.21.15.5:0/1094635691 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cf374c8da0 0x55cf374dba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:03:50.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:50.413+0000 7fd72d400640 1 --2- 172.21.15.5:0/3384396720 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fd7140a2c40 0x7fd7140a5100 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7fd71c009710 tx=0x7fd71c009af0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:50.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:50.413+0000 7fd72de00640 1 --2- 172.21.15.5:0/1094635691 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cf374c8480 0x55cf374c8860 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7fd720009ca0 tx=0x7fd720009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=871b700998459560 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:50.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:50.414+0000 7fd71b400640 1 -- 172.21.15.5:0/1094635691 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd720031db0 con 0x55cf374c8480 2024-09-10T10:03:50.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:50.414+0000 7fd71b400640 1 -- 172.21.15.5:0/1094635691 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd720036030 con 0x55cf374c8480 2024-09-10T10:03:50.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:50.414+0000 7fd71b400640 1 -- 172.21.15.5:0/1094635691 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7200042a0 con 0x55cf374c8480 2024-09-10T10:03:50.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:50.415+0000 7fd737bfd740 1 -- 172.21.15.5:0/1094635691 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cf374c8480 msgr2=0x55cf374c8860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:50.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:50.415+0000 7fd737bfd740 1 --2- 172.21.15.5:0/1094635691 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cf374c8480 0x55cf374c8860 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7fd720009ca0 tx=0x7fd720009920 comp rx=0 tx=0).stop 2024-09-10T10:03:50.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:50.416+0000 7fd737bfd740 1 -- 172.21.15.5:0/1094635691 shutdown_connections 2024-09-10T10:03:50.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:50.416+0000 7fd737bfd740 1 --2- 172.21.15.5:0/1094635691 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cf374dbf50 0x55cf374de340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:50.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:50.416+0000 7fd737bfd740 1 --2- 172.21.15.5:0/1094635691 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cf374c8da0 0x55cf374dba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:50.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:50.416+0000 7fd737bfd740 1 --2- 172.21.15.5:0/1094635691 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cf374c8480 0x55cf374c8860 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:50.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:50.416+0000 7fd737bfd740 1 -- 172.21.15.5:0/1094635691 >> 172.21.15.5:0/1094635691 conn(0x55cf374c4af0 msgr2=0x55cf374c4ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:50.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:50.416+0000 7fd737bfd740 1 -- 172.21.15.5:0/1094635691 shutdown_connections 2024-09-10T10:03:50.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:50.416+0000 7fd737bfd740 1 -- 172.21.15.5:0/1094635691 wait complete. 2024-09-10T10:03:50.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:50.417+0000 7fd737bfd740 1 -- start start 2024-09-10T10:03:50.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:50.417+0000 7fd737bfd740 1 Processor -- start 2024-09-10T10:03:50.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:50.417+0000 7fd737bfd740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cf374c8480 0x55cf374d8ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:50.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:50.417+0000 7fd737bfd740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cf374c8da0 0x55cf374d9430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:50.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:50.418+0000 7fd737bfd740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cf374dbf50 0x55cf374d9970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:50.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:50.418+0000 7fd737bfd740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55cf374de8e0 con 0x55cf374c8da0 2024-09-10T10:03:50.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:50.418+0000 7fd737bfd740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55cf373e8e60 con 0x55cf374c8480 2024-09-10T10:03:50.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:50.418+0000 7fd737bfd740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55cf374dea60 con 0x55cf374dbf50 2024-09-10T10:03:50.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:50.418+0000 7fd72de00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cf374c8480 0x55cf374d8ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:50.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:50.418+0000 7fd72de00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cf374c8480 0x55cf374d8ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:53824/0 (socket says 172.21.15.5:53824) 2024-09-10T10:03:50.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:50.418+0000 7fd72de00640 1 -- 172.21.15.5:0/3018853758 learned_addr learned my addr 172.21.15.5:0/3018853758 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T10:03:50.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:50.418+0000 7fd72e800640 1 --2- 172.21.15.5:0/3018853758 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cf374dbf50 0x55cf374d9970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:50.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:50.418+0000 7fd737bfd740 10 client.0 _ll_register_callbacks cb 0x55cf373edcc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:03:50.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:50.418+0000 7fd72d400640 1 --2- 172.21.15.5:0/3018853758 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cf374c8da0 0x55cf374d9430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:50.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:50.418+0000 7fd72de00640 1 -- 172.21.15.5:0/3018853758 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cf374dbf50 msgr2=0x55cf374d9970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:03:50.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:50.418+0000 7fd72de00640 1 --2- 172.21.15.5:0/3018853758 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cf374dbf50 0x55cf374d9970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:50.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:50.418+0000 7fd72de00640 1 -- 172.21.15.5:0/3018853758 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cf374c8da0 msgr2=0x55cf374d9430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:03:50.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:50.418+0000 7fd72de00640 1 --2- 172.21.15.5:0/3018853758 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cf374c8da0 0x55cf374d9430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:50.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:50.419+0000 7fd72de00640 1 -- 172.21.15.5:0/3018853758 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cf373f4990 con 0x55cf374c8480 2024-09-10T10:03:50.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:50.419+0000 7fd72d400640 1 --2- 172.21.15.5:0/3018853758 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cf374c8da0 0x55cf374d9430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:03:50.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:50.419+0000 7fd72de00640 1 --2- 172.21.15.5:0/3018853758 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cf374c8480 0x55cf374d8ef0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7fd7200097b0 tx=0x7fd720003ba0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=70d88857ec5e2d6f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:50.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:50.419+0000 7fd70fe00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-09-10T10:03:50.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:50.420+0000 7fd70fe00640 1 -- 172.21.15.5:0/3018853758 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd720052020 con 0x55cf374c8480 2024-09-10T10:03:50.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:50.420+0000 7fd70fe00640 1 -- 172.21.15.5:0/3018853758 <== mon.2 v2:172.21.15.102:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fd720004040 con 0x55cf374c8480 2024-09-10T10:03:50.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:50.420+0000 7fd737bfd740 1 -- 172.21.15.5:0/3018853758 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd720036030 con 0x55cf374c8480 2024-09-10T10:03:50.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:50.420+0000 7fd70fe00640 1 -- 172.21.15.5:0/3018853758 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7fd72005d050 con 0x55cf374c8480 2024-09-10T10:03:50.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:50.421+0000 7fd737bfd740 10 client.30212 fetch_fsmap learned FSMap version 42 2024-09-10T10:03:50.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:50.421+0000 7fd737bfd740 1 -- 172.21.15.5:0/3018853758 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55cf374de8e0 con 0x55cf374c8480 2024-09-10T10:03:50.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:50.422+0000 7fd70fe00640 1 -- 172.21.15.5:0/3018853758 <== mon.2 v2:172.21.15.102:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd720036030 con 0x55cf374c8480 2024-09-10T10:03:50.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:50.422+0000 7fd70fe00640 1 -- 172.21.15.5:0/3018853758 <== mon.2 v2:172.21.15.102:3300/0 5 ==== fsmap(e 42) ==== 8370+0+0 (secure 0 0 0) 0x7fd72004d030 con 0x55cf374c8480 2024-09-10T10:03:50.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:50.422+0000 7fd737bfd740 10 client.30212 fetch_fsmap finished waiting for FSMap version 42 2024-09-10T10:03:50.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:50.422+0000 7fd737bfd740 10 client.30212 resolve_mds: validated gid 24542 aka daemon mds.b 2024-09-10T10:03:50.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:50.422+0000 7fd737bfd740 20 client.30212 populate_metadata read hostname 'smithi005' 2024-09-10T10:03:50.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:50.422+0000 7fd737bfd740 1 --2- 172.21.15.5:0/3018853758 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55cf374eb790 0x55cf374edc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:50.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:50.422+0000 7fd737bfd740 4 client.30212 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:03:50.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:50.422+0000 7fd737bfd740 1 -- 172.21.15.5:0/3018853758 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cf373d1310 con 0x55cf374eb790 2024-09-10T10:03:50.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:50.423+0000 7fd72d400640 1 --2- 172.21.15.5:0/3018853758 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55cf374eb790 0x55cf374edc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:50.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:50.424+0000 7fd72d400640 1 --2- 172.21.15.5:0/3018853758 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55cf374eb790 0x55cf374edc30 crc :-1 s=READY pgs=341 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=712182a7598d3735 server_cookie=b6f7f3e6c57e7ea5 in_seq=0 out_seq=0 2024-09-10T10:03:50.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:50.424+0000 7fd70fe00640 10 client.30212 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T10:03:50.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:50.425+0000 7fd70fe00640 1 -- 172.21.15.5:0/3018853758 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55cf373d1310 con 0x55cf374eb790 2024-09-10T10:03:50.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:50.425+0000 7fd70fe00640 10 client.30212 handle_command_reply: tid=0 2024-09-10T10:03:50.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:50.425+0000 7fd737bfd740 1 client.30212 shutdown 2024-09-10T10:03:50.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:50.426+0000 7fd737bfd740 1 -- 172.21.15.5:0/3018853758 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cf374c8480 msgr2=0x55cf374d8ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:50.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:50.426+0000 7fd737bfd740 1 --2- 172.21.15.5:0/3018853758 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cf374c8480 0x55cf374d8ef0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7fd7200097b0 tx=0x7fd720003ba0 comp rx=0 tx=0).stop 2024-09-10T10:03:50.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:50.427+0000 7fd737bfd740 1 -- 172.21.15.5:0/3018853758 shutdown_connections 2024-09-10T10:03:50.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:50.427+0000 7fd737bfd740 1 --2- 172.21.15.5:0/3018853758 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55cf374eb790 0x55cf374edc30 crc :-1 s=READY pgs=341 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:50.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:50.427+0000 7fd737bfd740 1 --2- 172.21.15.5:0/3018853758 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cf374dbf50 0x55cf374d9970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:50.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:50.427+0000 7fd737bfd740 1 --2- 172.21.15.5:0/3018853758 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cf374c8da0 0x55cf374d9430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:50.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:50.427+0000 7fd737bfd740 1 --2- 172.21.15.5:0/3018853758 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cf374c8480 0x55cf374d8ef0 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:50.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:50.427+0000 7fd737bfd740 1 -- 172.21.15.5:0/3018853758 >> 172.21.15.5:0/3018853758 conn(0x55cf374c4af0 msgr2=0x55cf374ca780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:50.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:50.428+0000 7fd70fe00640 0 client.30212 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T10:03:50.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:50.428+0000 7fd737bfd740 1 -- 172.21.15.5:0/3018853758 shutdown_connections 2024-09-10T10:03:50.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:50.428+0000 7fd737bfd740 1 -- 172.21.15.5:0/3018853758 wait complete. 2024-09-10T10:03:50.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:50.428+0000 7fd737bfd740 20 client.30212 trim_cache size 0 max 16384 2024-09-10T10:03:50.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:50.455+0000 7fd737bfd740 1 Processor -- start 2024-09-10T10:03:50.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:50.455+0000 7fd737bfd740 1 -- start start 2024-09-10T10:03:50.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:50.455+0000 7fd737bfd740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cf374c8480 0x55cf374f3500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:50.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:50.455+0000 7fd737bfd740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cf374c8da0 0x55cf374f3a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:50.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:50.455+0000 7fd737bfd740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cf374dbf50 0x55cf374d6150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:50.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:50.455+0000 7fd737bfd740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd7301aca40 con 0x55cf374c8da0 2024-09-10T10:03:50.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:50.455+0000 7fd737bfd740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fd7301ae580 con 0x55cf374dbf50 2024-09-10T10:03:50.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:50.455+0000 7fd737bfd740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fd7301ad110 con 0x55cf374c8480 2024-09-10T10:03:50.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:50.456+0000 7fd72d400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cf374c8da0 0x55cf374f3a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:50.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:50.456+0000 7fd72de00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cf374c8480 0x55cf374f3500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:50.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:50.456+0000 7fd72d400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cf374c8da0 0x55cf374f3a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35132/0 (socket says 172.21.15.5:35132) 2024-09-10T10:03:50.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:50.456+0000 7fd72d400640 1 -- 172.21.15.5:0/347072715 learned_addr learned my addr 172.21.15.5:0/347072715 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T10:03:50.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:50.456+0000 7fd72e800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cf374dbf50 0x55cf374d6150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:50.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:50.456+0000 7fd72e800640 1 -- 172.21.15.5:0/347072715 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cf374c8480 msgr2=0x55cf374f3500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:03:50.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:50.456+0000 7fd72e800640 1 --2- 172.21.15.5:0/347072715 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cf374c8480 0x55cf374f3500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:50.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:50.456+0000 7fd72e800640 1 -- 172.21.15.5:0/347072715 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cf374c8da0 msgr2=0x55cf374f3a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:03:50.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:50.456+0000 7fd72e800640 1 --2- 172.21.15.5:0/347072715 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cf374c8da0 0x55cf374f3a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:50.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:50.456+0000 7fd72e800640 1 -- 172.21.15.5:0/347072715 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cf374c4380 con 0x55cf374dbf50 2024-09-10T10:03:50.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:50.457+0000 7fd72e800640 1 --2- 172.21.15.5:0/347072715 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cf374dbf50 0x55cf374d6150 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7fd7280cc3d0 tx=0x7fd72809ab40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=198211bd6ead7518 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:50.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:50.457+0000 7fd71aa00640 1 -- 172.21.15.5:0/347072715 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd728020040 con 0x55cf374dbf50 2024-09-10T10:03:50.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:50.457+0000 7fd71aa00640 1 -- 172.21.15.5:0/347072715 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd7280a3da0 con 0x55cf374dbf50 2024-09-10T10:03:50.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:50.457+0000 7fd71aa00640 1 -- 172.21.15.5:0/347072715 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd728097850 con 0x55cf374dbf50 2024-09-10T10:03:50.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:50.458+0000 7fd737bfd740 1 -- 172.21.15.5:0/347072715 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cf374dbf50 msgr2=0x55cf374d6150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:50.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:50.458+0000 7fd737bfd740 1 --2- 172.21.15.5:0/347072715 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cf374dbf50 0x55cf374d6150 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7fd7280cc3d0 tx=0x7fd72809ab40 comp rx=0 tx=0).stop 2024-09-10T10:03:50.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:50.458+0000 7fd737bfd740 1 -- 172.21.15.5:0/347072715 shutdown_connections 2024-09-10T10:03:50.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:50.458+0000 7fd737bfd740 1 --2- 172.21.15.5:0/347072715 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cf374dbf50 0x55cf374d6150 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:50.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:50.458+0000 7fd737bfd740 1 --2- 172.21.15.5:0/347072715 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cf374c8da0 0x55cf374f3a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:50.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:50.458+0000 7fd737bfd740 1 --2- 172.21.15.5:0/347072715 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cf374c8480 0x55cf374f3500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:50.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:50.458+0000 7fd737bfd740 1 -- 172.21.15.5:0/347072715 >> 172.21.15.5:0/347072715 conn(0x55cf374c4af0 msgr2=0x55cf374d96d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:50.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:50.458+0000 7fd737bfd740 1 -- 172.21.15.5:0/347072715 shutdown_connections 2024-09-10T10:03:50.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:50.459+0000 7fd737bfd740 1 -- 172.21.15.5:0/347072715 wait complete. 2024-09-10T10:03:50.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:50.459+0000 7fd737bfd740 1 -- start start 2024-09-10T10:03:50.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:50.459+0000 7fd737bfd740 1 Processor -- start 2024-09-10T10:03:50.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:50.460+0000 7fd737bfd740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cf374c8480 0x55cf374e0270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:50.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:50.460+0000 7fd737bfd740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cf374c8da0 0x55cf374e07b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:50.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:50.460+0000 7fd737bfd740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cf374dbf50 0x55cf375554d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:50.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:50.460+0000 7fd737bfd740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd7301adeb0 con 0x55cf374c8da0 2024-09-10T10:03:50.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:50.460+0000 7fd737bfd740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fd7301ad7e0 con 0x55cf374c8480 2024-09-10T10:03:50.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:50.460+0000 7fd737bfd740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fd7301aaa20 con 0x55cf374dbf50 2024-09-10T10:03:50.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:50.460+0000 7fd737bfd740 10 client.0 _ll_register_callbacks cb 0x7fd71c0134e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:03:50.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:50.460+0000 7fd72de00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cf374c8480 0x55cf374e0270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:50.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:50.461+0000 7fd72de00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cf374c8480 0x55cf374e0270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:53836/0 (socket says 172.21.15.5:53836) 2024-09-10T10:03:50.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:50.461+0000 7fd72de00640 1 -- 172.21.15.5:0/2333513121 learned_addr learned my addr 172.21.15.5:0/2333513121 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T10:03:50.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:50.461+0000 7fd72d400640 1 --2- 172.21.15.5:0/2333513121 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cf374c8da0 0x55cf374e07b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:50.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:50.461+0000 7fd72e800640 1 --2- 172.21.15.5:0/2333513121 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cf374dbf50 0x55cf375554d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:50.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:50.461+0000 7fd72de00640 1 -- 172.21.15.5:0/2333513121 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cf374dbf50 msgr2=0x55cf375554d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:03:50.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:50.461+0000 7fd72de00640 1 --2- 172.21.15.5:0/2333513121 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cf374dbf50 0x55cf375554d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:50.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:50.461+0000 7fd72de00640 1 -- 172.21.15.5:0/2333513121 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cf374c8da0 msgr2=0x55cf374e07b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:03:50.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:50.461+0000 7fd72de00640 1 --2- 172.21.15.5:0/2333513121 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cf374c8da0 0x55cf374e07b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:50.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:50.461+0000 7fd72de00640 1 -- 172.21.15.5:0/2333513121 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cf37555b10 con 0x55cf374c8480 2024-09-10T10:03:50.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:50.462+0000 7fd72d400640 1 --2- 172.21.15.5:0/2333513121 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cf374c8da0 0x55cf374e07b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:03:50.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:50.462+0000 7fd72de00640 1 --2- 172.21.15.5:0/2333513121 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cf374c8480 0x55cf374e0270 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7fd720009600 tx=0x7fd720032ff0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c71481a70a494bf8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:50.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:50.462+0000 7fd70fe00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-09-10T10:03:50.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:50.462+0000 7fd70fe00640 1 -- 172.21.15.5:0/2333513121 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd720052040 con 0x55cf374c8480 2024-09-10T10:03:50.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:50.463+0000 7fd737bfd740 1 -- 172.21.15.5:0/2333513121 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd7280a3da0 con 0x55cf374c8480 2024-09-10T10:03:50.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:50.463+0000 7fd70fe00640 1 -- 172.21.15.5:0/2333513121 <== mon.2 v2:172.21.15.102:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fd72002f850 con 0x55cf374c8480 2024-09-10T10:03:50.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:50.463+0000 7fd70fe00640 1 -- 172.21.15.5:0/2333513121 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7fd720066050 con 0x55cf374c8480 2024-09-10T10:03:50.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:50.463+0000 7fd737bfd740 10 client.30218 fetch_fsmap learned FSMap version 42 2024-09-10T10:03:50.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:50.463+0000 7fd737bfd740 1 -- 172.21.15.5:0/2333513121 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55cf37556dd0 con 0x55cf374c8480 2024-09-10T10:03:50.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:50.463+0000 7fd70fe00640 1 -- 172.21.15.5:0/2333513121 <== mon.2 v2:172.21.15.102:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd7280a3da0 con 0x55cf374c8480 2024-09-10T10:03:50.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:50.464+0000 7fd70fe00640 1 -- 172.21.15.5:0/2333513121 <== mon.2 v2:172.21.15.102:3300/0 5 ==== fsmap(e 42) ==== 8370+0+0 (secure 0 0 0) 0x7fd72004d040 con 0x55cf374c8480 2024-09-10T10:03:50.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:50.464+0000 7fd737bfd740 10 client.30218 fetch_fsmap finished waiting for FSMap version 42 2024-09-10T10:03:50.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:50.464+0000 7fd737bfd740 10 client.30218 resolve_mds: validated gid 24542 aka daemon mds.b 2024-09-10T10:03:50.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:50.464+0000 7fd737bfd740 20 client.30218 populate_metadata read hostname 'smithi005' 2024-09-10T10:03:50.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:50.464+0000 7fd737bfd740 1 --2- 172.21.15.5:0/2333513121 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55cf37557480 0x55cf375598a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:50.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:50.464+0000 7fd737bfd740 4 client.30218 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 150.0, "await_for": 45.0}] 2024-09-10T10:03:50.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:50.464+0000 7fd737bfd740 1 -- 172.21.15.5:0/2333513121 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 150.0, "await_for": 45.0}) -- 0x55cf3743bc60 con 0x55cf37557480 2024-09-10T10:03:50.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:50.465+0000 7fd72d400640 1 --2- 172.21.15.5:0/2333513121 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55cf37557480 0x55cf375598a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:50.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:50.465+0000 7fd72d400640 1 --2- 172.21.15.5:0/2333513121 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55cf37557480 0x55cf375598a0 crc :-1 s=READY pgs=342 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6c343c5201f91cf0 server_cookie=b74d2ab0f929b360 in_seq=0 out_seq=0 2024-09-10T10:03:50.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:50.465+0000 7fd70fe00640 10 client.30218 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T10:03:50.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:03:50 smithi159 ceph-mon[35513]: scrub summary: active paths [/] 2024-09-10T10:03:50.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:03:50 smithi159 ceph-mon[35513]: scrub summary: active paths [/] 2024-09-10T10:03:50.601 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:03:50 smithi159 ceph-mon[35513]: scrub summary: idle+waiting paths [/] 2024-09-10T10:03:50.601 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:03:50 smithi159 ceph-mon[35513]: scrub summary: idle+waiting paths [/] 2024-09-10T10:03:50.601 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:03:50 smithi159 ceph-mon[35513]: scrub summary: active paths [/] 2024-09-10T10:03:50.601 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:03:50 smithi159 ceph-mon[35513]: scrub summary: active paths [/] 2024-09-10T10:03:50.601 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:03:50 smithi159 ceph-mon[35513]: scrub summary: idle+waiting paths [/] 2024-09-10T10:03:50.601 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:03:50 smithi159 ceph-mon[35513]: scrub summary: idle+waiting paths [/] 2024-09-10T10:03:50.601 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:03:50 smithi159 ceph-mon[35513]: scrub summary: active paths [/] 2024-09-10T10:03:50.601 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:03:50 smithi159 ceph-mon[35513]: scrub summary: idle+waiting paths [/] 2024-09-10T10:03:50.635 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:03:50 smithi102 ceph-mon[35442]: scrub queued for path: / 2024-09-10T10:03:50.635 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:03:50 smithi102 ceph-mon[35442]: scrub summary: idle+waiting paths [/] 2024-09-10T10:03:50.635 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:03:50 smithi102 ceph-mon[35442]: scrub summary: active paths [/] 2024-09-10T10:03:50.635 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:03:50 smithi102 ceph-mon[35442]: scrub summary: active paths [/] 2024-09-10T10:03:50.635 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:03:50 smithi102 ceph-mon[35442]: scrub summary: active paths [/] 2024-09-10T10:03:50.635 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:03:50 smithi102 ceph-mon[35442]: scrub summary: idle+waiting paths [/] 2024-09-10T10:03:50.635 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:03:50 smithi102 ceph-mon[35442]: scrub summary: idle+waiting paths [/] 2024-09-10T10:03:50.635 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:03:50 smithi102 ceph-mon[35442]: scrub summary: active paths [/] 2024-09-10T10:03:50.635 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:03:50 smithi102 ceph-mon[35442]: scrub summary: active paths [/] 2024-09-10T10:03:50.635 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:03:50 smithi102 ceph-mon[35442]: scrub summary: idle+waiting paths [/] 2024-09-10T10:03:50.635 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:03:50 smithi102 ceph-mon[35442]: scrub summary: idle+waiting paths [/] 2024-09-10T10:03:50.635 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:03:50 smithi102 ceph-mon[35442]: scrub summary: active paths [/] 2024-09-10T10:03:50.636 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:03:50 smithi102 ceph-mon[35442]: scrub summary: active paths [/] 2024-09-10T10:03:50.636 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:03:50 smithi102 ceph-mon[35442]: scrub summary: idle+waiting paths [/] 2024-09-10T10:03:50.636 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:03:50 smithi102 ceph-mon[35442]: scrub summary: idle+waiting paths [/] 2024-09-10T10:03:50.636 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:03:50 smithi102 ceph-mon[35442]: scrub summary: active paths [/] 2024-09-10T10:03:50.636 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:03:50 smithi102 ceph-mon[35442]: scrub summary: idle+waiting paths [/] 2024-09-10T10:03:50.945 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T10:03:51.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:51.010+0000 7fd70fe00640 1 -- 172.21.15.5:0/2333513121 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55cf3743bc60 con 0x55cf37557480 2024-09-10T10:03:51.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:51.010+0000 7fd70fe00640 10 client.30218 handle_command_reply: tid=0 2024-09-10T10:03:51.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:51.010+0000 7fd737bfd740 1 client.30218 shutdown 2024-09-10T10:03:51.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:51.010+0000 7fd737bfd740 1 -- 172.21.15.5:0/2333513121 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cf374c8480 msgr2=0x55cf374e0270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:51.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:51.010+0000 7fd737bfd740 1 --2- 172.21.15.5:0/2333513121 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cf374c8480 0x55cf374e0270 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7fd720009600 tx=0x7fd720032ff0 comp rx=0 tx=0).stop 2024-09-10T10:03:51.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:51.011+0000 7fd737bfd740 1 -- 172.21.15.5:0/2333513121 shutdown_connections 2024-09-10T10:03:51.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:51.011+0000 7fd737bfd740 1 --2- 172.21.15.5:0/2333513121 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55cf37557480 0x55cf375598a0 crc :-1 s=READY pgs=342 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:51.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:51.011+0000 7fd737bfd740 1 --2- 172.21.15.5:0/2333513121 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55cf374dbf50 0x55cf375554d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:51.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:51.011+0000 7fd737bfd740 1 --2- 172.21.15.5:0/2333513121 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cf374c8da0 0x55cf374e07b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:51.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:51.011+0000 7fd737bfd740 1 --2- 172.21.15.5:0/2333513121 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55cf374c8480 0x55cf374e0270 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:51.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:51.011+0000 7fd737bfd740 1 -- 172.21.15.5:0/2333513121 >> 172.21.15.5:0/2333513121 conn(0x55cf374c4af0 msgr2=0x55cf374d8eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:51.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:51.011+0000 7fd70fe00640 0 client.30218 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T10:03:51.014 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-10T10:03:51.014 INFO:teuthology.orchestra.run.smithi005.stdout: "epoch": 42, 2024-09-10T10:03:51.014 INFO:teuthology.orchestra.run.smithi005.stdout: "leader": 24542, 2024-09-10T10:03:51.014 INFO:teuthology.orchestra.run.smithi005.stdout: "set_version": 90, 2024-09-10T10:03:51.014 INFO:teuthology.orchestra.run.smithi005.stdout: "sets": { 2024-09-10T10:03:51.014 INFO:teuthology.orchestra.run.smithi005.stdout: "23d0f3ef": { 2024-09-10T10:03:51.014 INFO:teuthology.orchestra.run.smithi005.stdout: "version": 90, 2024-09-10T10:03:51.015 INFO:teuthology.orchestra.run.smithi005.stdout: "age_ref": 0.0, 2024-09-10T10:03:51.015 INFO:teuthology.orchestra.run.smithi005.stdout: "state": { 2024-09-10T10:03:51.015 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "QUIESCED", 2024-09-10T10:03:51.015 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0 2024-09-10T10:03:51.015 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:03:51.015 INFO:teuthology.orchestra.run.smithi005.stdout: "timeout": 90.0, 2024-09-10T10:03:51.015 INFO:teuthology.orchestra.run.smithi005.stdout: "expiration": 150.0, 2024-09-10T10:03:51.015 INFO:teuthology.orchestra.run.smithi005.stdout: "members": { 2024-09-10T10:03:51.015 INFO:teuthology.orchestra.run.smithi005.stdout: "file:/": { 2024-09-10T10:03:51.015 INFO:teuthology.orchestra.run.smithi005.stdout: "excluded": false, 2024-09-10T10:03:51.015 INFO:teuthology.orchestra.run.smithi005.stdout: "state": { 2024-09-10T10:03:51.015 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "QUIESCED", 2024-09-10T10:03:51.015 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0 2024-09-10T10:03:51.015 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T10:03:51.016 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T10:03:51.016 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T10:03:51.016 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T10:03:51.016 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T10:03:51.016 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-10T10:03:51.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:51.011+0000 7fd737bfd740 1 -- 172.21.15.5:0/2333513121 shutdown_connections 2024-09-10T10:03:51.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:51.011+0000 7fd737bfd740 1 -- 172.21.15.5:0/2333513121 wait complete. 2024-09-10T10:03:51.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:51.011+0000 7fd737bfd740 20 client.30218 trim_cache size 0 max 16384 2024-09-10T10:03:51.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:51.012+0000 7fd72f200640 1 -- 172.21.15.5:0/3384396720 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fd7140a2c40 msgr2=0x7fd7140a5100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:51.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:51.012+0000 7fd72f200640 1 --2- 172.21.15.5:0/3384396720 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fd7140a2c40 0x7fd7140a5100 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7fd71c009710 tx=0x7fd71c009af0 comp rx=0 tx=0).stop 2024-09-10T10:03:51.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:51.012+0000 7fd72f200640 1 -- 172.21.15.5:0/3384396720 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd73010e350 msgr2=0x7fd73019ee10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:51.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:51.013+0000 7fd72f200640 1 --2- 172.21.15.5:0/3384396720 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd73010e350 0x7fd73019ee10 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7fd728005fd0 tx=0x7fd72800dbd0 comp rx=0 tx=0).stop 2024-09-10T10:03:51.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:51.015+0000 7fd72f200640 1 -- 172.21.15.5:0/3384396720 shutdown_connections 2024-09-10T10:03:51.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:51.015+0000 7fd72f200640 1 --2- 172.21.15.5:0/3384396720 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fd7140a2c40 0x7fd7140a5100 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:51.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:51.015+0000 7fd72f200640 1 --2- 172.21.15.5:0/3384396720 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fd73010e350 0x7fd73019ee10 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:51.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:51.015+0000 7fd72f200640 1 --2- 172.21.15.5:0/3384396720 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd730101150 0x7fd73019aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:51.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:51.015+0000 7fd72f200640 1 --2- 172.21.15.5:0/3384396720 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fd7301007a0 0x7fd73019a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:51.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:51.015+0000 7fd72f200640 1 -- 172.21.15.5:0/3384396720 >> 172.21.15.5:0/3384396720 conn(0x7fd7300fc610 msgr2=0x7fd7300fea00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:51.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:51.015+0000 7fd72f200640 1 -- 172.21.15.5:0/3384396720 shutdown_connections 2024-09-10T10:03:51.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:51.016+0000 7fd72f200640 1 -- 172.21.15.5:0/3384396720 wait complete. 2024-09-10T10:03:51.103 INFO:tasks.workunit.client.0.smithi005.stdout: 10 178771 13.24 MB/sec execute 569 sec latency 587.032 ms 2024-09-10T10:03:51.298 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:03:51 smithi005 ceph-mon[30350]: scrub summary: idle+waiting paths [/] 2024-09-10T10:03:51.298 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:03:51 smithi005 ceph-mon[30350]: pgmap v1158: 129 pgs: 129 active+clean; 760 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.0 MiB/s wr, 194 op/s 2024-09-10T10:03:51.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:03:51 smithi159 ceph-mon[35513]: scrub summary: idle+waiting paths [/] 2024-09-10T10:03:51.601 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:03:51 smithi159 ceph-mon[35513]: pgmap v1158: 129 pgs: 129 active+clean; 760 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.0 MiB/s wr, 194 op/s 2024-09-10T10:03:51.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:03:51 smithi102 ceph-mon[35442]: scrub summary: idle+waiting paths [/] 2024-09-10T10:03:51.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:03:51 smithi102 ceph-mon[35442]: pgmap v1158: 129 pgs: 129 active+clean; 760 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.0 MiB/s wr, 194 op/s 2024-09-10T10:03:52.103 INFO:tasks.workunit.client.0.smithi005.stdout: 10 178771 13.22 MB/sec execute 570 sec latency 1587.399 ms 2024-09-10T10:03:52.786 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '23d0f3ef', quiesce took 12.2 seconds. Will release after: 17.8 2024-09-10T10:03:52.818 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:03:53.103 INFO:tasks.workunit.client.0.smithi005.stdout: 10 178771 13.20 MB/sec execute 571 sec latency 2587.536 ms 2024-09-10T10:03:53.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:53.453+0000 7f0bf9200640 1 -- 172.21.15.5:0/3878304177 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0bf4109140 msgr2=0x7f0bf4110cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:53.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:53.453+0000 7f0bf9200640 1 --2- 172.21.15.5:0/3878304177 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0bf4109140 0x7f0bf4110cc0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f0bec009ca0 tx=0x7f0bec02a8c0 comp rx=0 tx=0).stop 2024-09-10T10:03:53.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:53.453+0000 7f0bf9200640 1 -- 172.21.15.5:0/3878304177 shutdown_connections 2024-09-10T10:03:53.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:53.453+0000 7f0bf9200640 1 --2- 172.21.15.5:0/3878304177 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0bf4109140 0x7f0bf4110cc0 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:53.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:53.453+0000 7f0bf9200640 1 --2- 172.21.15.5:0/3878304177 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0bf41049c0 0x7f0bf4108a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:53.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:53.453+0000 7f0bf9200640 1 --2- 172.21.15.5:0/3878304177 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0bf4104010 0x7f0bf41043f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:53.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:53.453+0000 7f0bf9200640 1 -- 172.21.15.5:0/3878304177 >> 172.21.15.5:0/3878304177 conn(0x7f0bf40ffa50 msgr2=0x7f0bf4101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:53.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:53.453+0000 7f0bf9200640 1 -- 172.21.15.5:0/3878304177 shutdown_connections 2024-09-10T10:03:53.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:53.453+0000 7f0bf9200640 1 -- 172.21.15.5:0/3878304177 wait complete. 2024-09-10T10:03:53.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:53.454+0000 7f0bf9200640 1 Processor -- start 2024-09-10T10:03:53.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:53.454+0000 7f0bf9200640 1 -- start start 2024-09-10T10:03:53.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:53.454+0000 7f0bf9200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0bf4104010 0x7f0bf419a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:53.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:53.454+0000 7f0bf9200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0bf41049c0 0x7f0bf419ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:53.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:53.454+0000 7f0bf9200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0bf4109140 0x7f0bf419ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:53.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:53.454+0000 7f0bf9200640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f0bf41141c0 con 0x7f0bf4109140 2024-09-10T10:03:53.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:53.454+0000 7f0bf9200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f0bf4114040 con 0x7f0bf4104010 2024-09-10T10:03:53.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:53.454+0000 7f0bf9200640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f0bf4114340 con 0x7f0bf41049c0 2024-09-10T10:03:53.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:53.455+0000 7f0bf2800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0bf4109140 0x7f0bf419ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:53.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:53.455+0000 7f0bf1e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0bf4104010 0x7f0bf419a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:53.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:53.455+0000 7f0bebe00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0bf41049c0 0x7f0bf419ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:53.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:53.455+0000 7f0bf1e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0bf4104010 0x7f0bf419a6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:38136/0 (socket says 172.21.15.5:38136) 2024-09-10T10:03:53.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:53.455+0000 7f0bf2800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0bf4109140 0x7f0bf419ef90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39300/0 (socket says 172.21.15.5:39300) 2024-09-10T10:03:53.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:53.455+0000 7f0bf1e00640 1 -- 172.21.15.5:0/2454021960 learned_addr learned my addr 172.21.15.5:0/2454021960 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T10:03:53.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:53.455+0000 7f0bf1e00640 1 -- 172.21.15.5:0/2454021960 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0bf41049c0 msgr2=0x7f0bf419ac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:53.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:53.455+0000 7f0bf1e00640 1 --2- 172.21.15.5:0/2454021960 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0bf41049c0 0x7f0bf419ac00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:53.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:53.455+0000 7f0bf1e00640 1 -- 172.21.15.5:0/2454021960 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0bf4109140 msgr2=0x7f0bf419ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:53.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:53.455+0000 7f0bf1e00640 1 --2- 172.21.15.5:0/2454021960 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0bf4109140 0x7f0bf419ef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:53.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:53.455+0000 7f0bf1e00640 1 -- 172.21.15.5:0/2454021960 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0bf419f670 con 0x7f0bf4104010 2024-09-10T10:03:53.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:53.456+0000 7f0bf1e00640 1 --2- 172.21.15.5:0/2454021960 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0bf4104010 0x7f0bf419a6c0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f0be400e9f0 tx=0x7f0be400edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:53.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:53.456+0000 7f0beaa00640 1 -- 172.21.15.5:0/2454021960 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0be400cd80 con 0x7f0bf4104010 2024-09-10T10:03:53.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:53.456+0000 7f0beaa00640 1 -- 172.21.15.5:0/2454021960 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0be4005500 con 0x7f0bf4104010 2024-09-10T10:03:53.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:53.456+0000 7f0bf9200640 1 -- 172.21.15.5:0/2454021960 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0bf419f960 con 0x7f0bf4104010 2024-09-10T10:03:53.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:53.456+0000 7f0beaa00640 1 -- 172.21.15.5:0/2454021960 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0be4010690 con 0x7f0bf4104010 2024-09-10T10:03:53.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:53.456+0000 7f0bf9200640 1 -- 172.21.15.5:0/2454021960 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0bf41a7240 con 0x7f0bf4104010 2024-09-10T10:03:53.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:53.459+0000 7f0bfbe36740 1 Processor -- start 2024-09-10T10:03:53.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:53.459+0000 7f0bfbe36740 1 -- start start 2024-09-10T10:03:53.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:53.459+0000 7f0beaa00640 1 -- 172.21.15.5:0/2454021960 <== mon.2 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f0be40046c0 con 0x7f0bf4104010 2024-09-10T10:03:53.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:53.460+0000 7f0bfbe36740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562c80f33b90 0x562c80f33f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:53.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:53.460+0000 7f0bfbe36740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562c80f344b0 0x562c80f3e830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:53.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:53.460+0000 7f0bfbe36740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562c80f3ed70 0x562c80f41160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:53.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:53.460+0000 7f0bfbe36740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x562c80e9a3e0 con 0x562c80f344b0 2024-09-10T10:03:53.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:53.460+0000 7f0bfbe36740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x562c80e41d80 con 0x562c80f33b90 2024-09-10T10:03:53.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:53.460+0000 7f0bfbe36740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x562c80e963a0 con 0x562c80f3ed70 2024-09-10T10:03:53.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:53.460+0000 7f0bf1e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562c80f344b0 0x562c80f3e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:53.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:53.460+0000 7f0bf2800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562c80f33b90 0x562c80f33f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:53.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:53.460+0000 7f0bf1e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562c80f344b0 0x562c80f3e830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39302/0 (socket says 172.21.15.5:39302) 2024-09-10T10:03:53.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:53.460+0000 7f0bf1e00640 1 -- 172.21.15.5:0/2307298262 learned_addr learned my addr 172.21.15.5:0/2307298262 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T10:03:53.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:53.460+0000 7f0bebe00640 1 --2- 172.21.15.5:0/2307298262 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562c80f3ed70 0x562c80f41160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:53.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:53.461+0000 7f0bf2800640 1 -- 172.21.15.5:0/2307298262 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562c80f3ed70 msgr2=0x562c80f41160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:03:53.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:53.461+0000 7f0bf2800640 1 --2- 172.21.15.5:0/2307298262 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562c80f3ed70 0x562c80f41160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:53.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:53.461+0000 7f0bf2800640 1 -- 172.21.15.5:0/2307298262 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562c80f344b0 msgr2=0x562c80f3e830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:03:53.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:53.461+0000 7f0bf2800640 1 --2- 172.21.15.5:0/2307298262 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562c80f344b0 0x562c80f3e830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:53.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:53.461+0000 7f0bf2800640 1 -- 172.21.15.5:0/2307298262 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562c80e60ae0 con 0x562c80f33b90 2024-09-10T10:03:53.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:53.461+0000 7f0bebe00640 1 --2- 172.21.15.5:0/2307298262 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562c80f3ed70 0x562c80f41160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:03:53.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:53.461+0000 7f0bf1e00640 1 --2- 172.21.15.5:0/2307298262 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562c80f344b0 0x562c80f3e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:03:53.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:53.461+0000 7f0bf2800640 1 --2- 172.21.15.5:0/2307298262 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562c80f33b90 0x562c80f33f70 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f0bec005e30 tx=0x7f0bec0093e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=80bb28a47be7fafe server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:53.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:53.461+0000 7f0beaa00640 1 --2- 172.21.15.5:0/2454021960 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f0bd80a2b80 0x7f0bd80a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:53.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:53.461+0000 7f0be3400640 1 -- 172.21.15.5:0/2307298262 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bec031db0 con 0x562c80f33b90 2024-09-10T10:03:53.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:53.461+0000 7f0be3400640 1 -- 172.21.15.5:0/2307298262 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0bec036030 con 0x562c80f33b90 2024-09-10T10:03:53.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:53.461+0000 7f0be3400640 1 -- 172.21.15.5:0/2307298262 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bec004240 con 0x562c80f33b90 2024-09-10T10:03:53.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:53.462+0000 7f0beaa00640 1 -- 172.21.15.5:0/2454021960 <== mon.2 v2:172.21.15.102:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f0be40cf870 con 0x7f0bf4104010 2024-09-10T10:03:53.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:53.462+0000 7f0bf1e00640 1 --2- 172.21.15.5:0/2454021960 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f0bd80a2b80 0x7f0bd80a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:53.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:53.462+0000 7f0bfbe36740 1 -- 172.21.15.5:0/2307298262 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562c80f33b90 msgr2=0x562c80f33f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:53.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:53.462+0000 7f0bfbe36740 1 --2- 172.21.15.5:0/2307298262 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562c80f33b90 0x562c80f33f70 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f0bec005e30 tx=0x7f0bec0093e0 comp rx=0 tx=0).stop 2024-09-10T10:03:53.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:53.462+0000 7f0bfbe36740 1 -- 172.21.15.5:0/2307298262 shutdown_connections 2024-09-10T10:03:53.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:53.462+0000 7f0bf1e00640 1 --2- 172.21.15.5:0/2454021960 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f0bd80a2b80 0x7f0bd80a5040 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f0be40a43c0 tx=0x7f0be40d5530 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:53.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:53.462+0000 7f0bfbe36740 1 --2- 172.21.15.5:0/2307298262 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562c80f3ed70 0x562c80f41160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:53.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:53.462+0000 7f0bfbe36740 1 --2- 172.21.15.5:0/2307298262 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562c80f344b0 0x562c80f3e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:53.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:53.462+0000 7f0bfbe36740 1 --2- 172.21.15.5:0/2307298262 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562c80f33b90 0x562c80f33f70 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:53.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:53.462+0000 7f0bfbe36740 1 -- 172.21.15.5:0/2307298262 >> 172.21.15.5:0/2307298262 conn(0x562c80f30120 msgr2=0x562c80f30500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:53.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:53.462+0000 7f0bfbe36740 1 -- 172.21.15.5:0/2307298262 shutdown_connections 2024-09-10T10:03:53.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:53.463+0000 7f0bfbe36740 1 -- 172.21.15.5:0/2307298262 wait complete. 2024-09-10T10:03:53.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:53.463+0000 7f0bfbe36740 1 -- start start 2024-09-10T10:03:53.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:53.463+0000 7f0bfbe36740 1 Processor -- start 2024-09-10T10:03:53.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:53.464+0000 7f0bfbe36740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562c80f33b90 0x562c80f3bc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:53.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:53.464+0000 7f0bfbe36740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562c80f344b0 0x562c80f3c180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:53.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:53.464+0000 7f0bfbe36740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562c80f3ed70 0x562c80f3c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:53.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:53.464+0000 7f0bfbe36740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x562c80f41860 con 0x562c80f344b0 2024-09-10T10:03:53.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:53.464+0000 7f0bfbe36740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x562c80e550f0 con 0x562c80f33b90 2024-09-10T10:03:53.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:53.464+0000 7f0bfbe36740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x562c80f419e0 con 0x562c80f3ed70 2024-09-10T10:03:53.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:53.464+0000 7f0bfbe36740 10 client.0 _ll_register_callbacks cb 0x562c80e469e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:03:53.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:53.464+0000 7f0bebe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562c80f344b0 0x562c80f3c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:53.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:53.464+0000 7f0bf2800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562c80f33b90 0x562c80f3bc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:53.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:53.464+0000 7f0bebe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562c80f344b0 0x562c80f3c180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39306/0 (socket says 172.21.15.5:39306) 2024-09-10T10:03:53.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:53.464+0000 7f0bebe00640 1 -- 172.21.15.5:0/316409275 learned_addr learned my addr 172.21.15.5:0/316409275 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T10:03:53.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:53.464+0000 7f0bf2800640 1 --2- 172.21.15.5:0/316409275 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562c80f3ed70 0x562c80f3c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:53.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:53.464+0000 7f0bebe00640 1 -- 172.21.15.5:0/316409275 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562c80f3ed70 msgr2=0x562c80f3c6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:03:53.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:53.464+0000 7f0bebe00640 1 --2- 172.21.15.5:0/316409275 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562c80f3ed70 0x562c80f3c6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:53.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:53.464+0000 7f0bebe00640 1 -- 172.21.15.5:0/316409275 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562c80f33b90 msgr2=0x562c80f3bc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:03:53.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:53.464+0000 7f0bebe00640 1 --2- 172.21.15.5:0/316409275 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562c80f33b90 0x562c80f3bc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:53.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:53.464+0000 7f0bebe00640 1 -- 172.21.15.5:0/316409275 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562c80c0b3c0 con 0x562c80f344b0 2024-09-10T10:03:53.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:53.465+0000 7f0bebe00640 1 --2- 172.21.15.5:0/316409275 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562c80f344b0 0x562c80f3c180 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f0bdc012700 tx=0x7f0bdc012bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90d3ea37ce1ebc4a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:53.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:53.465+0000 7f0bd7e00640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-10T10:03:53.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:53.465+0000 7f0bd7e00640 1 -- 172.21.15.5:0/316409275 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bdc00f920 con 0x562c80f344b0 2024-09-10T10:03:53.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:53.465+0000 7f0bd7e00640 1 -- 172.21.15.5:0/316409275 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f0bdc00fac0 con 0x562c80f344b0 2024-09-10T10:03:53.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:53.465+0000 7f0bfbe36740 1 -- 172.21.15.5:0/316409275 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0bec036030 con 0x562c80f344b0 2024-09-10T10:03:53.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:53.465+0000 7f0bd7e00640 1 -- 172.21.15.5:0/316409275 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f0bdc00fcc0 con 0x562c80f344b0 2024-09-10T10:03:53.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:53.465+0000 7f0bfbe36740 10 client.20139 fetch_fsmap learned FSMap version 42 2024-09-10T10:03:53.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:53.465+0000 7f0bfbe36740 1 -- 172.21.15.5:0/316409275 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x562c80f4d550 con 0x562c80f344b0 2024-09-10T10:03:53.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:53.466+0000 7f0bd7e00640 1 -- 172.21.15.5:0/316409275 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0bec036030 con 0x562c80f344b0 2024-09-10T10:03:53.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:53.466+0000 7f0bd7e00640 1 -- 172.21.15.5:0/316409275 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 42) ==== 8370+0+0 (secure 0 0 0) 0x7f0bdc01c420 con 0x562c80f344b0 2024-09-10T10:03:53.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:53.466+0000 7f0bfbe36740 10 client.20139 fetch_fsmap finished waiting for FSMap version 42 2024-09-10T10:03:53.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:53.466+0000 7f0bfbe36740 10 client.20139 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T10:03:53.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:53.466+0000 7f0bfbe36740 20 client.20139 populate_metadata read hostname 'smithi005' 2024-09-10T10:03:53.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:53.466+0000 7f0bfbe36740 1 --2- 172.21.15.5:0/316409275 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x562c80f4ea10 0x562c80f50e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:53.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:53.466+0000 7f0bfbe36740 4 client.20139 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:03:53.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:53.466+0000 7f0bfbe36740 1 -- 172.21.15.5:0/316409275 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562c80ea7950 con 0x562c80f4ea10 2024-09-10T10:03:53.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:53.467+0000 7f0bf2800640 1 --2- 172.21.15.5:0/316409275 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x562c80f4ea10 0x562c80f50e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:53.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:53.467+0000 7f0bf2800640 1 --2- 172.21.15.5:0/316409275 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x562c80f4ea10 0x562c80f50e40 crc :-1 s=READY pgs=343 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e86f6a6433a8e89 server_cookie=eafeb748456e95fe in_seq=0 out_seq=0 2024-09-10T10:03:53.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:53.467+0000 7f0bd7e00640 10 client.20139 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T10:03:53.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:53.468+0000 7f0bd7e00640 1 -- 172.21.15.5:0/316409275 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562c80ea7950 con 0x562c80f4ea10 2024-09-10T10:03:53.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:53.468+0000 7f0bd7e00640 10 client.20139 handle_command_reply: tid=0 2024-09-10T10:03:53.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:53.468+0000 7f0bfbe36740 1 client.20139 shutdown 2024-09-10T10:03:53.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:53.468+0000 7f0bfbe36740 1 -- 172.21.15.5:0/316409275 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562c80f344b0 msgr2=0x562c80f3c180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:53.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:53.468+0000 7f0bfbe36740 1 --2- 172.21.15.5:0/316409275 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562c80f344b0 0x562c80f3c180 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f0bdc012700 tx=0x7f0bdc012bc0 comp rx=0 tx=0).stop 2024-09-10T10:03:53.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:53.468+0000 7f0bfbe36740 1 -- 172.21.15.5:0/316409275 shutdown_connections 2024-09-10T10:03:53.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:53.468+0000 7f0bfbe36740 1 --2- 172.21.15.5:0/316409275 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x562c80f4ea10 0x562c80f50e40 crc :-1 s=READY pgs=343 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:53.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:53.468+0000 7f0bfbe36740 1 --2- 172.21.15.5:0/316409275 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562c80f3ed70 0x562c80f3c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:53.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:53.468+0000 7f0bfbe36740 1 --2- 172.21.15.5:0/316409275 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562c80f344b0 0x562c80f3c180 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:53.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:53.468+0000 7f0bfbe36740 1 --2- 172.21.15.5:0/316409275 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562c80f33b90 0x562c80f3bc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:53.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:53.468+0000 7f0bfbe36740 1 -- 172.21.15.5:0/316409275 >> 172.21.15.5:0/316409275 conn(0x562c80f30120 msgr2=0x562c80f35e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:53.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:53.468+0000 7f0bd7e00640 0 client.20139 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T10:03:53.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:53.469+0000 7f0bfbe36740 1 -- 172.21.15.5:0/316409275 shutdown_connections 2024-09-10T10:03:53.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:53.469+0000 7f0bfbe36740 1 -- 172.21.15.5:0/316409275 wait complete. 2024-09-10T10:03:53.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:53.469+0000 7f0bfbe36740 20 client.20139 trim_cache size 0 max 16384 2024-09-10T10:03:53.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:53.487+0000 7f0bfbe36740 1 Processor -- start 2024-09-10T10:03:53.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:53.487+0000 7f0bfbe36740 1 -- start start 2024-09-10T10:03:53.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:53.487+0000 7f0bfbe36740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562c80f30120 0x562c80f365f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:53.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:53.487+0000 7f0bfbe36740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562c80f33b90 0x562c80f3dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:53.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:53.487+0000 7f0bfbe36740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562c80f344b0 0x562c80f56750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:53.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:53.487+0000 7f0bfbe36740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f0bf41b1030 con 0x562c80f344b0 2024-09-10T10:03:53.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:53.487+0000 7f0bfbe36740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f0bf41b2b70 con 0x562c80f33b90 2024-09-10T10:03:53.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:53.487+0000 7f0bfbe36740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f0bf41b1700 con 0x562c80f30120 2024-09-10T10:03:53.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:53.487+0000 7f0bf2800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562c80f344b0 0x562c80f56750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:53.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:53.487+0000 7f0bf2800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562c80f344b0 0x562c80f56750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39310/0 (socket says 172.21.15.5:39310) 2024-09-10T10:03:53.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:53.487+0000 7f0bf2800640 1 -- 172.21.15.5:0/12165434 learned_addr learned my addr 172.21.15.5:0/12165434 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T10:03:53.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:53.487+0000 7f0bebe00640 1 --2- 172.21.15.5:0/12165434 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562c80f33b90 0x562c80f3dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:53.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:53.488+0000 7f0bf2800640 1 --2- 172.21.15.5:0/12165434 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562c80f30120 0x562c80f365f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:53.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:53.488+0000 7f0bf2800640 1 -- 172.21.15.5:0/12165434 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562c80f30120 msgr2=0x562c80f365f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:03:53.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:53.488+0000 7f0bf2800640 1 --2- 172.21.15.5:0/12165434 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562c80f30120 0x562c80f365f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:53.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:53.488+0000 7f0bf2800640 1 -- 172.21.15.5:0/12165434 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562c80f33b90 msgr2=0x562c80f3dbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:03:53.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:53.488+0000 7f0bf2800640 1 --2- 172.21.15.5:0/12165434 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562c80f33b90 0x562c80f3dbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:53.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:53.488+0000 7f0bf2800640 1 -- 172.21.15.5:0/12165434 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562c80f311d0 con 0x562c80f344b0 2024-09-10T10:03:53.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:53.488+0000 7f0bf2800640 1 --2- 172.21.15.5:0/12165434 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562c80f344b0 0x562c80f56750 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f0bec005e30 tx=0x7f0bec02a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d84e65faa4e2818 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:53.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:53.488+0000 7f0be2a00640 1 -- 172.21.15.5:0/12165434 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bec031510 con 0x562c80f344b0 2024-09-10T10:03:53.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:53.488+0000 7f0be2a00640 1 -- 172.21.15.5:0/12165434 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0bec036040 con 0x562c80f344b0 2024-09-10T10:03:53.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:53.488+0000 7f0be2a00640 1 -- 172.21.15.5:0/12165434 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bec004630 con 0x562c80f344b0 2024-09-10T10:03:53.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:53.489+0000 7f0bfbe36740 1 -- 172.21.15.5:0/12165434 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562c80f344b0 msgr2=0x562c80f56750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:53.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:53.489+0000 7f0bfbe36740 1 --2- 172.21.15.5:0/12165434 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562c80f344b0 0x562c80f56750 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f0bec005e30 tx=0x7f0bec02a8c0 comp rx=0 tx=0).stop 2024-09-10T10:03:53.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:53.489+0000 7f0bfbe36740 1 -- 172.21.15.5:0/12165434 shutdown_connections 2024-09-10T10:03:53.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:53.489+0000 7f0bfbe36740 1 --2- 172.21.15.5:0/12165434 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562c80f344b0 0x562c80f56750 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:53.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:53.489+0000 7f0bfbe36740 1 --2- 172.21.15.5:0/12165434 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562c80f33b90 0x562c80f3dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:53.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:53.489+0000 7f0bfbe36740 1 --2- 172.21.15.5:0/12165434 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562c80f30120 0x562c80f365f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:53.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:53.489+0000 7f0bfbe36740 1 -- 172.21.15.5:0/12165434 >> 172.21.15.5:0/12165434 conn(0x562c80f37630 msgr2=0x562c80f3d4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:53.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:53.489+0000 7f0bfbe36740 1 -- 172.21.15.5:0/12165434 shutdown_connections 2024-09-10T10:03:53.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:53.489+0000 7f0bfbe36740 1 -- 172.21.15.5:0/12165434 wait complete. 2024-09-10T10:03:53.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:53.490+0000 7f0bfbe36740 1 -- start start 2024-09-10T10:03:53.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:53.490+0000 7f0bfbe36740 1 Processor -- start 2024-09-10T10:03:53.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:53.490+0000 7f0bfbe36740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562c80f30120 0x562c80f472f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:53.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:53.490+0000 7f0bfbe36740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562c80f33b90 0x562c80f42620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:53.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:53.490+0000 7f0bfbe36740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562c80f344b0 0x562c80f42b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:53.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:53.490+0000 7f0bfbe36740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f0bf41b24a0 con 0x562c80f30120 2024-09-10T10:03:53.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:53.490+0000 7f0bfbe36740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f0bf41b1dd0 con 0x562c80f344b0 2024-09-10T10:03:53.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:53.490+0000 7f0bfbe36740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f0bf41af010 con 0x562c80f33b90 2024-09-10T10:03:53.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:53.491+0000 7f0bfbe36740 10 client.0 _ll_register_callbacks cb 0x7f0bdc009e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:03:53.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:53.491+0000 7f0bf2800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562c80f30120 0x562c80f472f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:53.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:53.491+0000 7f0bf2800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562c80f30120 0x562c80f472f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39316/0 (socket says 172.21.15.5:39316) 2024-09-10T10:03:53.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:53.491+0000 7f0bf2800640 1 -- 172.21.15.5:0/2588488214 learned_addr learned my addr 172.21.15.5:0/2588488214 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T10:03:53.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:53.491+0000 7f0bebe00640 1 --2- 172.21.15.5:0/2588488214 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562c80f33b90 0x562c80f42620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:53.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:53.491+0000 7f0bf2800640 1 --2- 172.21.15.5:0/2588488214 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562c80f344b0 0x562c80f42b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:53.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:53.491+0000 7f0bf2800640 1 -- 172.21.15.5:0/2588488214 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562c80f33b90 msgr2=0x562c80f42620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:03:53.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:53.491+0000 7f0bf2800640 1 --2- 172.21.15.5:0/2588488214 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562c80f33b90 0x562c80f42620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:53.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:53.491+0000 7f0bf2800640 1 -- 172.21.15.5:0/2588488214 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562c80f344b0 msgr2=0x562c80f42b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:03:53.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:53.491+0000 7f0bf2800640 1 --2- 172.21.15.5:0/2588488214 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562c80f344b0 0x562c80f42b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:53.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:53.491+0000 7f0bf2800640 1 -- 172.21.15.5:0/2588488214 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562c80f43370 con 0x562c80f30120 2024-09-10T10:03:53.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:53.491+0000 7f0bebe00640 1 --2- 172.21.15.5:0/2588488214 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562c80f33b90 0x562c80f42620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:03:53.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:53.492+0000 7f0bf2800640 1 --2- 172.21.15.5:0/2588488214 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562c80f30120 0x562c80f472f0 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f0bec002230 tx=0x7f0bec02afc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa3caac9e0f28f35 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:03:53.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:53.492+0000 7f0bd7e00640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-10T10:03:53.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:53.492+0000 7f0bd7e00640 1 -- 172.21.15.5:0/2588488214 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bec06a060 con 0x562c80f30120 2024-09-10T10:03:53.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:53.492+0000 7f0bfbe36740 1 -- 172.21.15.5:0/2588488214 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0bec036040 con 0x562c80f30120 2024-09-10T10:03:53.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:53.492+0000 7f0bd7e00640 1 -- 172.21.15.5:0/2588488214 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f0bec0042b0 con 0x562c80f30120 2024-09-10T10:03:53.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:53.492+0000 7f0bd7e00640 1 -- 172.21.15.5:0/2588488214 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f0bec0044b0 con 0x562c80f30120 2024-09-10T10:03:53.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:53.492+0000 7f0bfbe36740 10 client.20145 fetch_fsmap learned FSMap version 42 2024-09-10T10:03:53.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:53.492+0000 7f0bfbe36740 1 -- 172.21.15.5:0/2588488214 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x562c80fb96a0 con 0x562c80f30120 2024-09-10T10:03:53.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:53.492+0000 7f0bd7e00640 1 -- 172.21.15.5:0/2588488214 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0bec036040 con 0x562c80f30120 2024-09-10T10:03:53.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:53.493+0000 7f0bd7e00640 1 -- 172.21.15.5:0/2588488214 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 42) ==== 8370+0+0 (secure 0 0 0) 0x7f0bec043070 con 0x562c80f30120 2024-09-10T10:03:53.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:53.493+0000 7f0bfbe36740 10 client.20145 fetch_fsmap finished waiting for FSMap version 42 2024-09-10T10:03:53.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:53.493+0000 7f0bfbe36740 10 client.20145 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T10:03:53.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:53.493+0000 7f0bfbe36740 20 client.20145 populate_metadata read hostname 'smithi005' 2024-09-10T10:03:53.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:53.493+0000 7f0bfbe36740 1 --2- 172.21.15.5:0/2588488214 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x562c80f3ed70 0x562c80fbba50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:03:53.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:53.493+0000 7f0bfbe36740 4 client.20145 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T10:03:53.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:53.493+0000 7f0bfbe36740 1 -- 172.21.15.5:0/2588488214 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "scrub status"}) -- 0x562c80ea1e60 con 0x562c80f3ed70 2024-09-10T10:03:53.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:53.493+0000 7f0bebe00640 1 --2- 172.21.15.5:0/2588488214 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x562c80f3ed70 0x562c80fbba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:03:53.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:53.494+0000 7f0bebe00640 1 --2- 172.21.15.5:0/2588488214 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x562c80f3ed70 0x562c80fbba50 crc :-1 s=READY pgs=344 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4d5294761bd8a205 server_cookie=9f0831a6a9c93df2 in_seq=0 out_seq=0 2024-09-10T10:03:53.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:53.494+0000 7f0bd7e00640 10 client.20145 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T10:03:53.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:53.494+0000 7f0bd7e00640 1 -- 172.21.15.5:0/2588488214 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x562c80ea1e60 con 0x562c80f3ed70 2024-09-10T10:03:53.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:53.494+0000 7f0bd7e00640 10 client.20145 handle_command_reply: tid=0 2024-09-10T10:03:53.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:53.494+0000 7f0bfbe36740 1 client.20145 shutdown 2024-09-10T10:03:53.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:53.494+0000 7f0bfbe36740 1 -- 172.21.15.5:0/2588488214 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562c80f30120 msgr2=0x562c80f472f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:53.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:53.494+0000 7f0bfbe36740 1 --2- 172.21.15.5:0/2588488214 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562c80f30120 0x562c80f472f0 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f0bec002230 tx=0x7f0bec02afc0 comp rx=0 tx=0).stop 2024-09-10T10:03:53.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:53.494+0000 7f0bfbe36740 1 -- 172.21.15.5:0/2588488214 shutdown_connections 2024-09-10T10:03:53.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:53.494+0000 7f0bfbe36740 1 --2- 172.21.15.5:0/2588488214 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x562c80f3ed70 0x562c80fbba50 crc :-1 s=READY pgs=344 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:53.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:53.494+0000 7f0bfbe36740 1 --2- 172.21.15.5:0/2588488214 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x562c80f344b0 0x562c80f42b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:53.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:53.494+0000 7f0bfbe36740 1 --2- 172.21.15.5:0/2588488214 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x562c80f33b90 0x562c80f42620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:53.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:53.494+0000 7f0bfbe36740 1 --2- 172.21.15.5:0/2588488214 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562c80f30120 0x562c80f472f0 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:53.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:53.494+0000 7f0bfbe36740 1 -- 172.21.15.5:0/2588488214 >> 172.21.15.5:0/2588488214 conn(0x562c80f37630 msgr2=0x562c80f3dcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:53.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:53.494+0000 7f0bd7e00640 0 client.20145 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T10:03:53.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:53.494+0000 7f0bfbe36740 1 -- 172.21.15.5:0/2588488214 shutdown_connections 2024-09-10T10:03:53.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:53.494+0000 7f0bfbe36740 1 -- 172.21.15.5:0/2588488214 wait complete. 2024-09-10T10:03:53.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:53.495+0000 7f0bfbe36740 20 client.20145 trim_cache size 0 max 16384 2024-09-10T10:03:53.500 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-10T10:03:53.500 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-10T10:03:53.500 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2024-09-10T10:03:53.500 INFO:teuthology.orchestra.run.smithi005.stdout: "70f0fb62-adae-493b-a035-7ec0d58003d4": { 2024-09-10T10:03:53.500 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2024-09-10T10:03:53.500 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "70f0fb62-adae-493b-a035-7ec0d58003d4", 2024-09-10T10:03:53.500 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2024-09-10T10:03:53.500 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T10:03:53.500 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T10:03:53.500 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-10T10:03:53.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:53.495+0000 7f0bf9200640 1 -- 172.21.15.5:0/2454021960 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f0bd80a2b80 msgr2=0x7f0bd80a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:53.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:53.495+0000 7f0bf9200640 1 --2- 172.21.15.5:0/2454021960 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f0bd80a2b80 0x7f0bd80a5040 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f0be40a43c0 tx=0x7f0be40d5530 comp rx=0 tx=0).stop 2024-09-10T10:03:53.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:53.495+0000 7f0bf9200640 1 -- 172.21.15.5:0/2454021960 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0bf4104010 msgr2=0x7f0bf419a6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:03:53.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:53.495+0000 7f0bf9200640 1 --2- 172.21.15.5:0/2454021960 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0bf4104010 0x7f0bf419a6c0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f0be400e9f0 tx=0x7f0be400edf0 comp rx=0 tx=0).stop 2024-09-10T10:03:53.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:53.495+0000 7f0bf9200640 1 -- 172.21.15.5:0/2454021960 shutdown_connections 2024-09-10T10:03:53.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:53.495+0000 7f0bf9200640 1 --2- 172.21.15.5:0/2454021960 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f0bd80a2b80 0x7f0bd80a5040 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:53.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:53.495+0000 7f0bf9200640 1 --2- 172.21.15.5:0/2454021960 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0bf4109140 0x7f0bf419ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:53.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:53.495+0000 7f0bf9200640 1 --2- 172.21.15.5:0/2454021960 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f0bf41049c0 0x7f0bf419ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:53.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:53.495+0000 7f0bf9200640 1 --2- 172.21.15.5:0/2454021960 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f0bf4104010 0x7f0bf419a6c0 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:03:53.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:53.495+0000 7f0bf9200640 1 -- 172.21.15.5:0/2454021960 >> 172.21.15.5:0/2454021960 conn(0x7f0bf40ffa50 msgr2=0x7f0bf4100db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:03:53.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:53.495+0000 7f0bf9200640 1 -- 172.21.15.5:0/2454021960 shutdown_connections 2024-09-10T10:03:53.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:03:53.495+0000 7f0bf9200640 1 -- 172.21.15.5:0/2454021960 wait complete. 2024-09-10T10:03:54.034 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:03:53 smithi005 ceph-mon[30350]: pgmap v1159: 129 pgs: 129 active+clean; 754 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 1.3 MiB/s wr, 188 op/s 2024-09-10T10:03:54.034 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:03:53 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T10:03:54.035 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:03:53 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T10:03:54.035 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:03:53 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:03:54.035 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:03:53 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:03:54.035 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:03:53 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T10:03:54.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:03:53 smithi159 ceph-mon[35513]: pgmap v1159: 129 pgs: 129 active+clean; 754 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 1.3 MiB/s wr, 188 op/s 2024-09-10T10:03:54.101 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:03:53 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T10:03:54.101 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:03:53 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T10:03:54.101 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:03:53 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:03:54.101 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:03:53 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:03:54.101 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:03:53 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T10:03:54.103 INFO:tasks.workunit.client.0.smithi005.stdout: 10 178771 13.17 MB/sec execute 572 sec latency 3587.656 ms 2024-09-10T10:03:54.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:03:53 smithi102 ceph-mon[35442]: pgmap v1159: 129 pgs: 129 active+clean; 754 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 1.3 MiB/s wr, 188 op/s 2024-09-10T10:03:54.135 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:03:53 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T10:03:54.135 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:03:53 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T10:03:54.135 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:03:53 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:03:54.135 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:03:53 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:03:54.135 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:03:53 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T10:03:54.499 INFO:tasks.cephfs.filesystem:scrub status for tag:70f0fb62-adae-493b-a035-7ec0d58003d4 - {'path': '/', 'tag': '70f0fb62-adae-493b-a035-7ec0d58003d4', 'options': 'recursive,force'} 2024-09-10T10:03:55.104 INFO:tasks.workunit.client.0.smithi005.stdout: 10 178771 13.15 MB/sec execute 573 sec latency 4587.749 ms 2024-09-10T10:03:56.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:03:55 smithi005 ceph-mon[30350]: pgmap v1160: 129 pgs: 129 active+clean; 755 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 1.3 MiB/s wr, 230 op/s 2024-09-10T10:03:56.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:03:55 smithi159 ceph-mon[35513]: pgmap v1160: 129 pgs: 129 active+clean; 755 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 1.3 MiB/s wr, 230 op/s 2024-09-10T10:03:56.103 INFO:tasks.workunit.client.0.smithi005.stdout: 10 178771 13.13 MB/sec execute 574 sec latency 5587.841 ms 2024-09-10T10:03:56.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:03:55 smithi102 ceph-mon[35442]: pgmap v1160: 129 pgs: 129 active+clean; 755 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 1.3 MiB/s wr, 230 op/s 2024-09-10T10:03:57.104 INFO:tasks.workunit.client.0.smithi005.stdout: 10 178771 13.10 MB/sec execute 575 sec latency 6588.022 ms 2024-09-10T10:03:58.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:03:57 smithi005 ceph-mon[30350]: pgmap v1161: 129 pgs: 129 active+clean; 756 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 1.2 MiB/s wr, 257 op/s 2024-09-10T10:03:58.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:03:57 smithi159 ceph-mon[35513]: pgmap v1161: 129 pgs: 129 active+clean; 756 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 1.2 MiB/s wr, 257 op/s 2024-09-10T10:03:58.104 INFO:tasks.workunit.client.0.smithi005.stdout: 10 178771 13.08 MB/sec execute 576 sec latency 7588.110 ms 2024-09-10T10:03:58.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:03:57 smithi102 ceph-mon[35442]: pgmap v1161: 129 pgs: 129 active+clean; 756 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 1.2 MiB/s wr, 257 op/s 2024-09-10T10:03:59.104 INFO:tasks.workunit.client.0.smithi005.stdout: 10 178771 13.06 MB/sec execute 577 sec latency 8588.249 ms 2024-09-10T10:04:00.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:03:59 smithi005 ceph-mon[30350]: pgmap v1162: 129 pgs: 129 active+clean; 756 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 1010 KiB/s wr, 234 op/s 2024-09-10T10:04:00.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:03:59 smithi159 ceph-mon[35513]: pgmap v1162: 129 pgs: 129 active+clean; 756 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 1010 KiB/s wr, 234 op/s 2024-09-10T10:04:00.104 INFO:tasks.workunit.client.0.smithi005.stdout: 10 178771 13.04 MB/sec execute 578 sec latency 9588.372 ms 2024-09-10T10:04:00.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:03:59 smithi102 ceph-mon[35442]: pgmap v1162: 129 pgs: 129 active+clean; 756 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 1010 KiB/s wr, 234 op/s 2024-09-10T10:04:01.131 INFO:tasks.workunit.client.0.smithi005.stdout: 10 178771 13.01 MB/sec execute 579 sec latency 10615.332 ms 2024-09-10T10:04:01.737 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:04:01 smithi005 ceph-mon[30350]: pgmap v1163: 129 pgs: 129 active+clean; 756 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 977 KiB/s wr, 235 op/s 2024-09-10T10:04:02.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:04:01 smithi159 ceph-mon[35513]: pgmap v1163: 129 pgs: 129 active+clean; 756 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 977 KiB/s wr, 235 op/s 2024-09-10T10:04:02.131 INFO:tasks.workunit.client.0.smithi005.stdout: 10 178771 12.99 MB/sec execute 580 sec latency 11615.396 ms 2024-09-10T10:04:02.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:04:01 smithi102 ceph-mon[35442]: pgmap v1163: 129 pgs: 129 active+clean; 756 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 977 KiB/s wr, 235 op/s 2024-09-10T10:04:02.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:04:02 smithi005 ceph-mon[30350]: scrub summary: idle 2024-09-10T10:04:02.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:04:02 smithi005 ceph-mon[30350]: scrub summary: idle 2024-09-10T10:04:02.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:04:02 smithi005 ceph-mon[30350]: scrub summary: idle 2024-09-10T10:04:02.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:04:02 smithi005 ceph-mon[30350]: scrub summary: idle 2024-09-10T10:04:02.807 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:04:02 smithi005 ceph-mon[30350]: scrub summary: idle 2024-09-10T10:04:03.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:04:02 smithi159 ceph-mon[35513]: scrub summary: idle 2024-09-10T10:04:03.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:04:02 smithi159 ceph-mon[35513]: scrub summary: idle 2024-09-10T10:04:03.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:04:02 smithi159 ceph-mon[35513]: scrub summary: idle 2024-09-10T10:04:03.101 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:04:02 smithi159 ceph-mon[35513]: scrub summary: idle 2024-09-10T10:04:03.101 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:04:02 smithi159 ceph-mon[35513]: scrub summary: idle 2024-09-10T10:04:03.131 INFO:tasks.workunit.client.0.smithi005.stdout: 10 178771 12.97 MB/sec execute 581 sec latency 12615.534 ms 2024-09-10T10:04:03.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:04:02 smithi102 ceph-mon[35442]: scrub summary: idle 2024-09-10T10:04:03.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:04:02 smithi102 ceph-mon[35442]: scrub summary: idle 2024-09-10T10:04:03.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:04:02 smithi102 ceph-mon[35442]: scrub summary: idle 2024-09-10T10:04:03.135 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:04:02 smithi102 ceph-mon[35442]: scrub summary: idle 2024-09-10T10:04:03.135 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:04:02 smithi102 ceph-mon[35442]: scrub summary: idle 2024-09-10T10:04:04.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:04:03 smithi005 ceph-mon[30350]: pgmap v1164: 129 pgs: 129 active+clean; 760 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 1.0 MiB/s wr, 144 op/s 2024-09-10T10:04:04.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:04:03 smithi159 ceph-mon[35513]: pgmap v1164: 129 pgs: 129 active+clean; 760 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 1.0 MiB/s wr, 144 op/s 2024-09-10T10:04:04.131 INFO:tasks.workunit.client.0.smithi005.stdout: 10 178771 12.95 MB/sec execute 582 sec latency 13615.680 ms 2024-09-10T10:04:04.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:04:03 smithi102 ceph-mon[35442]: pgmap v1164: 129 pgs: 129 active+clean; 760 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 1.0 MiB/s wr, 144 op/s 2024-09-10T10:04:05.131 INFO:tasks.workunit.client.0.smithi005.stdout: 10 178771 12.92 MB/sec execute 583 sec latency 14615.780 ms 2024-09-10T10:04:05.936 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:04:05 smithi102 ceph-mon[35442]: pgmap v1165: 129 pgs: 129 active+clean; 765 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.7 MiB/s wr, 128 op/s 2024-09-10T10:04:06.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:04:05 smithi005 ceph-mon[30350]: pgmap v1165: 129 pgs: 129 active+clean; 765 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.7 MiB/s wr, 128 op/s 2024-09-10T10:04:06.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:04:05 smithi159 ceph-mon[35513]: pgmap v1165: 129 pgs: 129 active+clean; 765 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.7 MiB/s wr, 128 op/s 2024-09-10T10:04:06.132 INFO:tasks.workunit.client.0.smithi005.stdout: 10 178771 12.90 MB/sec execute 584 sec latency 15615.914 ms 2024-09-10T10:04:07.132 INFO:tasks.workunit.client.0.smithi005.stdout: 10 178771 12.88 MB/sec execute 585 sec latency 16616.061 ms 2024-09-10T10:04:08.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:04:07 smithi005 ceph-mon[30350]: pgmap v1166: 129 pgs: 129 active+clean; 770 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 645 KiB/s rd, 2.2 MiB/s wr, 94 op/s 2024-09-10T10:04:08.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:04:07 smithi159 ceph-mon[35513]: pgmap v1166: 129 pgs: 129 active+clean; 770 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 645 KiB/s rd, 2.2 MiB/s wr, 94 op/s 2024-09-10T10:04:08.132 INFO:tasks.workunit.client.0.smithi005.stdout: 10 178771 12.86 MB/sec execute 586 sec latency 17616.207 ms 2024-09-10T10:04:08.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:04:07 smithi102 ceph-mon[35442]: pgmap v1166: 129 pgs: 129 active+clean; 770 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 645 KiB/s rd, 2.2 MiB/s wr, 94 op/s 2024-09-10T10:04:09.132 INFO:tasks.workunit.client.0.smithi005.stdout: 10 178771 12.83 MB/sec execute 587 sec latency 18616.357 ms 2024-09-10T10:04:10.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:04:09 smithi005 ceph-mon[30350]: pgmap v1167: 129 pgs: 129 active+clean; 769 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 2.1 MiB/s wr, 56 op/s 2024-09-10T10:04:10.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:04:09 smithi159 ceph-mon[35513]: pgmap v1167: 129 pgs: 129 active+clean; 769 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 2.1 MiB/s wr, 56 op/s 2024-09-10T10:04:10.132 INFO:tasks.workunit.client.0.smithi005.stdout: 10 178771 12.81 MB/sec execute 588 sec latency 19616.511 ms 2024-09-10T10:04:10.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:04:09 smithi102 ceph-mon[35442]: pgmap v1167: 129 pgs: 129 active+clean; 769 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 2.1 MiB/s wr, 56 op/s 2024-09-10T10:04:10.586 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T10:04:10.978 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:04:11.132 INFO:tasks.workunit.client.0.smithi005.stdout: 10 178771 12.79 MB/sec execute 589 sec latency 20616.627 ms 2024-09-10T10:04:11.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:11.526+0000 7fc09e600640 1 -- 172.21.15.5:0/3492543328 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc0981079c0 msgr2=0x7fc098107da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:11.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:11.526+0000 7fc09e600640 1 --2- 172.21.15.5:0/3492543328 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc0981079c0 0x7fc098107da0 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7fc088009930 tx=0x7fc08802a420 comp rx=0 tx=0).stop 2024-09-10T10:04:11.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:11.526+0000 7fc09e600640 1 -- 172.21.15.5:0/3492543328 shutdown_connections 2024-09-10T10:04:11.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:11.526+0000 7fc09e600640 1 --2- 172.21.15.5:0/3492543328 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc098102360 0x7fc09810e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:11.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:11.526+0000 7fc09e600640 1 --2- 172.21.15.5:0/3492543328 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc0981019c0 0x7fc098101e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:11.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:11.526+0000 7fc09e600640 1 --2- 172.21.15.5:0/3492543328 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc0981079c0 0x7fc098107da0 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:11.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:11.526+0000 7fc09e600640 1 -- 172.21.15.5:0/3492543328 >> 172.21.15.5:0/3492543328 conn(0x7fc0980fd740 msgr2=0x7fc0980ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:11.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:11.526+0000 7fc09e600640 1 -- 172.21.15.5:0/3492543328 shutdown_connections 2024-09-10T10:04:11.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:11.526+0000 7fc09e600640 1 -- 172.21.15.5:0/3492543328 wait complete. 2024-09-10T10:04:11.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:11.527+0000 7fc09e600640 1 Processor -- start 2024-09-10T10:04:11.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:11.527+0000 7fc09e600640 1 -- start start 2024-09-10T10:04:11.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:11.528+0000 7fc09e600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc0981019c0 0x7fc0980744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:11.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:11.528+0000 7fc09e600640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc098102360 0x7fc098074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:11.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:11.528+0000 7fc09e600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc0981079c0 0x7fc098074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:11.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:11.528+0000 7fc09e600640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc098111d90 con 0x7fc0981019c0 2024-09-10T10:04:11.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:11.528+0000 7fc09e600640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fc098111c10 con 0x7fc0981079c0 2024-09-10T10:04:11.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:11.528+0000 7fc09e600640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fc098111f10 con 0x7fc098102360 2024-09-10T10:04:11.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:11.528+0000 7fc09d200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc0981019c0 0x7fc0980744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:11.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:11.528+0000 7fc09d200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc0981019c0 0x7fc0980744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36008/0 (socket says 172.21.15.5:36008) 2024-09-10T10:04:11.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:11.528+0000 7fc09d200640 1 -- 172.21.15.5:0/2886897408 learned_addr learned my addr 172.21.15.5:0/2886897408 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T10:04:11.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:11.528+0000 7fc08fe00640 1 --2- 172.21.15.5:0/2886897408 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc098102360 0x7fc098074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:11.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:11.528+0000 7fc09dc00640 1 --2- 172.21.15.5:0/2886897408 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc0981079c0 0x7fc098074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:11.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:11.529+0000 7fc08fe00640 1 -- 172.21.15.5:0/2886897408 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc0981079c0 msgr2=0x7fc098074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:11.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:11.529+0000 7fc08fe00640 1 --2- 172.21.15.5:0/2886897408 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc0981079c0 0x7fc098074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:11.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:11.529+0000 7fc08fe00640 1 -- 172.21.15.5:0/2886897408 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc0981019c0 msgr2=0x7fc0980744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:11.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:11.529+0000 7fc08fe00640 1 --2- 172.21.15.5:0/2886897408 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc0981019c0 0x7fc0980744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:11.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:11.529+0000 7fc08fe00640 1 -- 172.21.15.5:0/2886897408 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0981a67b0 con 0x7fc098102360 2024-09-10T10:04:11.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:11.529+0000 7fc09d200640 1 --2- 172.21.15.5:0/2886897408 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc0981019c0 0x7fc0980744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:04:11.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:11.529+0000 7fc09dc00640 1 --2- 172.21.15.5:0/2886897408 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc0981079c0 0x7fc098074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:04:11.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:11.529+0000 7fc08fe00640 1 --2- 172.21.15.5:0/2886897408 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc098102360 0x7fc098074a30 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fc08000d6d0 tx=0x7fc08000db90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:11.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:11.529+0000 7fc096000640 1 -- 172.21.15.5:0/2886897408 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc080017070 con 0x7fc098102360 2024-09-10T10:04:11.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:11.529+0000 7fc09e600640 1 -- 172.21.15.5:0/2886897408 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0981a6aa0 con 0x7fc098102360 2024-09-10T10:04:11.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:11.530+0000 7fc096000640 1 -- 172.21.15.5:0/2886897408 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc080009820 con 0x7fc098102360 2024-09-10T10:04:11.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:11.530+0000 7fc096000640 1 -- 172.21.15.5:0/2886897408 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc080004cd0 con 0x7fc098102360 2024-09-10T10:04:11.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:11.530+0000 7fc09e600640 1 -- 172.21.15.5:0/2886897408 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0981a7030 con 0x7fc098102360 2024-09-10T10:04:11.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:11.531+0000 7fc09e600640 1 -- 172.21.15.5:0/2886897408 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc098101e20 con 0x7fc098102360 2024-09-10T10:04:11.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:11.533+0000 7fc096000640 1 -- 172.21.15.5:0/2886897408 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7fc0800099d0 con 0x7fc098102360 2024-09-10T10:04:11.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:11.534+0000 7fc096000640 1 --2- 172.21.15.5:0/2886897408 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fc07c0a2b80 0x7fc07c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:11.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:11.534+0000 7fc096000640 1 -- 172.21.15.5:0/2886897408 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fc0800cf330 con 0x7fc098102360 2024-09-10T10:04:11.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:11.534+0000 7fc09d200640 1 --2- 172.21.15.5:0/2886897408 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fc07c0a2b80 0x7fc07c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:11.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:11.535+0000 7fc09d200640 1 --2- 172.21.15.5:0/2886897408 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fc07c0a2b80 0x7fc07c0a5040 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fc088000c00 tx=0x7fc088009560 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:11.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:11.536+0000 7fc096000640 1 -- 172.21.15.5:0/2886897408 <== mon.1 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc08009b7f0 con 0x7fc098102360 2024-09-10T10:04:11.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:11.759+0000 7fc09e600640 1 -- 172.21.15.5:0/2886897408 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc098109f60 con 0x7fc098102360 2024-09-10T10:04:11.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:11.760+0000 7fc096000640 1 -- 172.21.15.5:0/2886897408 <== mon.1 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 42 v42) ==== 94+0+11969 (secure 0 0 0) 0x7fc08009faf0 con 0x7fc098102360 2024-09-10T10:04:11.762 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-10T10:04:11.762 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":42,"btime":"2024-09-10T09:55:06:670916+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14454,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/2970594311","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2970594311},{"type":"v1","addr":"172.21.15.5:6835","nonce":2970594311}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14514,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6839/3616502174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":3616502174},{"type":"v1","addr":"172.21.15.5:6839","nonce":3616502174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":42,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T09:35:29.626287+0000","modified":"2024-09-10T09:55:06.419829+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24542,"mds_1":24535,"mds_2":24511,"mds_3":24506,"mds_4":24443},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14520":{"gid":14520,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/2073599842","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":2073599842},{"type":"v1","addr":"172.21.15.5:6841","nonce":2073599842}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"k","rank":4,"incarnation":26,"state":"up:active","state_seq":24,"addr":"172.21.15.102:6833/2457007427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":2457007427},{"type":"v1","addr":"172.21.15.102:6833","nonce":2457007427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/356445159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":356445159},{"type":"v1","addr":"172.21.15.159:6833","nonce":356445159}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6837/2894314507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":2894314507},{"type":"v1","addr":"172.21.15.5:6837","nonce":2894314507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.102:6835/1647549111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":1647549111},{"type":"v1","addr":"172.21.15.102:6835","nonce":1647549111}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6835/952657242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":952657242},{"type":"v1","addr":"172.21.15.159:6835","nonce":952657242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":19,"addr":"172.21.15.102:6837/2689776764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":2689776764},{"type":"v1","addr":"172.21.15.102:6837","nonce":2689776764}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.159:6837/620627639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":620627639},{"type":"v1","addr":"172.21.15.159:6837","nonce":620627639}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24535":{"gid":24535,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":18,"addr":"172.21.15.159:6839/2073739599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":2073739599},{"type":"v1","addr":"172.21.15.159:6839","nonce":2073739599}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24542":{"gid":24542,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.102:6839/335409749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6838","nonce":335409749},{"type":"v1","addr":"172.21.15.102:6839","nonce":335409749}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24542,"qdb_cluster":[24542,24535,24511,24506,24443]},"id":1}]} 2024-09-10T10:04:11.762 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 42 2024-09-10T10:04:11.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:11.762+0000 7fc09e600640 1 -- 172.21.15.5:0/2886897408 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fc07c0a2b80 msgr2=0x7fc07c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:11.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:11.762+0000 7fc09e600640 1 --2- 172.21.15.5:0/2886897408 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fc07c0a2b80 0x7fc07c0a5040 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fc088000c00 tx=0x7fc088009560 comp rx=0 tx=0).stop 2024-09-10T10:04:11.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:11.762+0000 7fc09e600640 1 -- 172.21.15.5:0/2886897408 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc098102360 msgr2=0x7fc098074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:11.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:11.762+0000 7fc09e600640 1 --2- 172.21.15.5:0/2886897408 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc098102360 0x7fc098074a30 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fc08000d6d0 tx=0x7fc08000db90 comp rx=0 tx=0).stop 2024-09-10T10:04:11.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:11.763+0000 7fc09e600640 1 -- 172.21.15.5:0/2886897408 shutdown_connections 2024-09-10T10:04:11.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:11.763+0000 7fc09e600640 1 --2- 172.21.15.5:0/2886897408 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fc07c0a2b80 0x7fc07c0a5040 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:11.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:11.763+0000 7fc09e600640 1 --2- 172.21.15.5:0/2886897408 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fc0981079c0 0x7fc098074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:11.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:11.763+0000 7fc09e600640 1 --2- 172.21.15.5:0/2886897408 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fc098102360 0x7fc098074a30 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:11.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:11.763+0000 7fc09e600640 1 --2- 172.21.15.5:0/2886897408 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc0981019c0 0x7fc0980744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:11.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:11.763+0000 7fc09e600640 1 -- 172.21.15.5:0/2886897408 >> 172.21.15.5:0/2886897408 conn(0x7fc0980fd740 msgr2=0x7fc0980fe080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:11.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:11.763+0000 7fc09e600640 1 -- 172.21.15.5:0/2886897408 shutdown_connections 2024-09-10T10:04:11.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:11.763+0000 7fc09e600640 1 -- 172.21.15.5:0/2886897408 wait complete. 2024-09-10T10:04:12.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:04:11 smithi005 ceph-mon[30350]: pgmap v1168: 129 pgs: 129 active+clean; 769 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 2.1 MiB/s wr, 66 op/s 2024-09-10T10:04:12.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:04:11 smithi159 ceph-mon[35513]: pgmap v1168: 129 pgs: 129 active+clean; 769 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 2.1 MiB/s wr, 66 op/s 2024-09-10T10:04:12.132 INFO:tasks.workunit.client.0.smithi005.stdout: 10 178771 12.77 MB/sec execute 590 sec latency 21616.706 ms 2024-09-10T10:04:12.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:04:11 smithi102 ceph-mon[35442]: pgmap v1168: 129 pgs: 129 active+clean; 769 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 2.1 MiB/s wr, 66 op/s 2024-09-10T10:04:12.656 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24542 quiesce db --set-id 23d0f3ef --release --await' 2024-09-10T10:04:12.656 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24542 quiesce db --set-id 23d0f3ef --release --await 2024-09-10T10:04:13.033 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:04:13.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:04:12 smithi005 ceph-mon[30350]: from='client.? 172.21.15.5:0/2886897408' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:04:13.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:04:12 smithi159 ceph-mon[35513]: from='client.? 172.21.15.5:0/2886897408' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:04:13.133 INFO:tasks.workunit.client.0.smithi005.stdout: 10 178771 12.75 MB/sec execute 591 sec latency 22616.853 ms 2024-09-10T10:04:13.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:04:12 smithi102 ceph-mon[35442]: from='client.? 172.21.15.5:0/2886897408' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:04:13.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:13.651+0000 7f7297200640 1 -- 172.21.15.5:0/365854629 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7298076720 msgr2=0x7f7298076b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:13.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:13.651+0000 7f7297200640 1 --2- 172.21.15.5:0/365854629 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7298076720 0x7f7298076b00 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f7288009900 tx=0x7f728802a3f0 comp rx=0 tx=0).stop 2024-09-10T10:04:13.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:13.652+0000 7f7297200640 1 -- 172.21.15.5:0/365854629 shutdown_connections 2024-09-10T10:04:13.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:13.652+0000 7f7297200640 1 --2- 172.21.15.5:0/365854629 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7298112870 0x7f7298114c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:13.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:13.652+0000 7f7297200640 1 --2- 172.21.15.5:0/365854629 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7298077040 0x7f7298112330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:13.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:13.652+0000 7f7297200640 1 --2- 172.21.15.5:0/365854629 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7298076720 0x7f7298076b00 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:13.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:13.652+0000 7f7297200640 1 -- 172.21.15.5:0/365854629 >> 172.21.15.5:0/365854629 conn(0x7f72980ff8d0 msgr2=0x7f7298101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:13.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:13.652+0000 7f7297200640 1 -- 172.21.15.5:0/365854629 shutdown_connections 2024-09-10T10:04:13.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:13.652+0000 7f7297200640 1 -- 172.21.15.5:0/365854629 wait complete. 2024-09-10T10:04:13.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:13.653+0000 7f7297200640 1 Processor -- start 2024-09-10T10:04:13.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:13.653+0000 7f7297200640 1 -- start start 2024-09-10T10:04:13.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:13.654+0000 7f7297200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7298076720 0x7f72981a4650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:13.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:13.654+0000 7f7297200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7298077040 0x7f72981a4b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:13.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:13.654+0000 7f7297200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7298112870 0x7f729819e720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:13.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:13.654+0000 7f7297200640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f7298118450 con 0x7f7298112870 2024-09-10T10:04:13.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:13.654+0000 7f7297200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f72981182d0 con 0x7f7298077040 2024-09-10T10:04:13.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:13.654+0000 7f7297200640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f72981185d0 con 0x7f7298076720 2024-09-10T10:04:13.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:13.654+0000 7f7296800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7298112870 0x7f729819e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:13.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:13.654+0000 7f7295e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7298076720 0x7f72981a4650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:13.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:13.654+0000 7f7295e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7298076720 0x7f72981a4650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:34826/0 (socket says 172.21.15.5:34826) 2024-09-10T10:04:13.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:13.654+0000 7f7295e00640 1 -- 172.21.15.5:0/3339924162 learned_addr learned my addr 172.21.15.5:0/3339924162 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T10:04:13.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:13.654+0000 7f728fe00640 1 --2- 172.21.15.5:0/3339924162 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7298077040 0x7f72981a4b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:13.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:13.654+0000 7f7296800640 1 -- 172.21.15.5:0/3339924162 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7298076720 msgr2=0x7f72981a4650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:13.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:13.654+0000 7f7296800640 1 --2- 172.21.15.5:0/3339924162 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7298076720 0x7f72981a4650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:13.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:13.654+0000 7f7296800640 1 -- 172.21.15.5:0/3339924162 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7298077040 msgr2=0x7f72981a4b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:13.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:13.654+0000 7f7296800640 1 --2- 172.21.15.5:0/3339924162 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7298077040 0x7f72981a4b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:13.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:13.654+0000 7f7296800640 1 -- 172.21.15.5:0/3339924162 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f729819ef90 con 0x7f7298112870 2024-09-10T10:04:13.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:13.655+0000 7f7296800640 1 --2- 172.21.15.5:0/3339924162 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7298112870 0x7f729819e720 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f729807aa10 tx=0x7f729000ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:13.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:13.655+0000 7f728ea00640 1 -- 172.21.15.5:0/3339924162 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f729000cdf0 con 0x7f7298112870 2024-09-10T10:04:13.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:13.655+0000 7f7297200640 1 -- 172.21.15.5:0/3339924162 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f729819f280 con 0x7f7298112870 2024-09-10T10:04:13.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:13.655+0000 7f728ea00640 1 -- 172.21.15.5:0/3339924162 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7290005020 con 0x7f7298112870 2024-09-10T10:04:13.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:13.655+0000 7f728ea00640 1 -- 172.21.15.5:0/3339924162 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7290010690 con 0x7f7298112870 2024-09-10T10:04:13.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:13.655+0000 7f7297200640 1 -- 172.21.15.5:0/3339924162 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72981ab5d0 con 0x7f7298112870 2024-09-10T10:04:13.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:13.659+0000 7f729fab9740 1 Processor -- start 2024-09-10T10:04:13.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:13.659+0000 7f728ea00640 1 -- 172.21.15.5:0/3339924162 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f72900054d0 con 0x7f7298112870 2024-09-10T10:04:13.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:13.660+0000 7f729fab9740 1 -- start start 2024-09-10T10:04:13.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:13.660+0000 7f729fab9740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5651a4d76520 0x5651a4d76900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:13.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:13.660+0000 7f729fab9740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5651a4d76e40 0x5651a4d811c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:13.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:13.660+0000 7f729fab9740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5651a4d81700 0x5651a4d83af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:13.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:13.660+0000 7f729fab9740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5651a4cdc650 con 0x5651a4d81700 2024-09-10T10:04:13.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:13.660+0000 7f729fab9740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x5651a4c83f10 con 0x5651a4d76e40 2024-09-10T10:04:13.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:13.660+0000 7f729fab9740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5651a4cd8610 con 0x5651a4d76520 2024-09-10T10:04:13.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:13.660+0000 7f728fe00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5651a4d76520 0x5651a4d76900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:13.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:13.660+0000 7f7295e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5651a4d76e40 0x5651a4d811c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:13.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:13.660+0000 7f7295e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5651a4d76e40 0x5651a4d811c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:52640/0 (socket says 172.21.15.5:52640) 2024-09-10T10:04:13.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:13.660+0000 7f7295e00640 1 -- 172.21.15.5:0/2928669702 learned_addr learned my addr 172.21.15.5:0/2928669702 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T10:04:13.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:13.661+0000 7f728fe00640 1 --2- 172.21.15.5:0/2928669702 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5651a4d81700 0x5651a4d83af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:13.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:13.661+0000 7f728fe00640 1 -- 172.21.15.5:0/2928669702 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5651a4d76e40 msgr2=0x5651a4d811c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:13.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:13.661+0000 7f728fe00640 1 --2- 172.21.15.5:0/2928669702 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5651a4d76e40 0x5651a4d811c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:13.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:13.661+0000 7f728fe00640 1 -- 172.21.15.5:0/2928669702 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5651a4d81700 msgr2=0x5651a4d83af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:13.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:13.661+0000 7f728fe00640 1 --2- 172.21.15.5:0/2928669702 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5651a4d81700 0x5651a4d83af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:13.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:13.661+0000 7f728fe00640 1 -- 172.21.15.5:0/2928669702 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5651a49eb580 con 0x5651a4d76520 2024-09-10T10:04:13.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:13.661+0000 7f7295e00640 1 --2- 172.21.15.5:0/2928669702 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5651a4d76e40 0x5651a4d811c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:04:13.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:13.661+0000 7f728ea00640 1 --2- 172.21.15.5:0/3339924162 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f727c0a2c00 0x7f727c0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:13.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:13.661+0000 7f728fe00640 1 --2- 172.21.15.5:0/2928669702 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5651a4d76520 0x5651a4d76900 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f7280009fd0 tx=0x7f72800096c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f863232207a65ebf server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:13.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:13.661+0000 7f728ea00640 1 -- 172.21.15.5:0/3339924162 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f72900cf9d0 con 0x7f7298112870 2024-09-10T10:04:13.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:13.661+0000 7f7295e00640 1 --2- 172.21.15.5:0/3339924162 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f727c0a2c00 0x7f727c0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:13.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:13.662+0000 7f7287400640 1 -- 172.21.15.5:0/2928669702 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7280004050 con 0x5651a4d76520 2024-09-10T10:04:13.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:13.662+0000 7f7287400640 1 -- 172.21.15.5:0/2928669702 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7280005500 con 0x5651a4d76520 2024-09-10T10:04:13.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:13.662+0000 7f7287400640 1 -- 172.21.15.5:0/2928669702 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7280004dd0 con 0x5651a4d76520 2024-09-10T10:04:13.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:13.662+0000 7f729fab9740 1 -- 172.21.15.5:0/2928669702 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5651a4d76520 msgr2=0x5651a4d76900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:13.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:13.662+0000 7f729fab9740 1 --2- 172.21.15.5:0/2928669702 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5651a4d76520 0x5651a4d76900 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f7280009fd0 tx=0x7f72800096c0 comp rx=0 tx=0).stop 2024-09-10T10:04:13.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:13.662+0000 7f7295e00640 1 --2- 172.21.15.5:0/3339924162 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f727c0a2c00 0x7f727c0a50c0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x5651a4d73910 tx=0x7f72880046c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:13.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:13.663+0000 7f729fab9740 1 -- 172.21.15.5:0/2928669702 shutdown_connections 2024-09-10T10:04:13.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:13.663+0000 7f729fab9740 1 --2- 172.21.15.5:0/2928669702 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5651a4d81700 0x5651a4d83af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:13.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:13.663+0000 7f729fab9740 1 --2- 172.21.15.5:0/2928669702 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5651a4d76e40 0x5651a4d811c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:13.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:13.663+0000 7f729fab9740 1 --2- 172.21.15.5:0/2928669702 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5651a4d76520 0x5651a4d76900 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:13.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:13.663+0000 7f729fab9740 1 -- 172.21.15.5:0/2928669702 >> 172.21.15.5:0/2928669702 conn(0x5651a4d72ab0 msgr2=0x5651a4d72e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:13.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:13.663+0000 7f729fab9740 1 -- 172.21.15.5:0/2928669702 shutdown_connections 2024-09-10T10:04:13.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:13.663+0000 7f729fab9740 1 -- 172.21.15.5:0/2928669702 wait complete. 2024-09-10T10:04:13.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:13.663+0000 7f729fab9740 1 -- start start 2024-09-10T10:04:13.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:13.664+0000 7f729fab9740 1 Processor -- start 2024-09-10T10:04:13.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:13.664+0000 7f729fab9740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5651a4d76520 0x5651a4d7e670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:13.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:13.664+0000 7f729fab9740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5651a4d76e40 0x5651a4d7ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:13.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:13.664+0000 7f729fab9740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5651a4d81700 0x5651a4d7f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:13.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:13.664+0000 7f729fab9740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5651a4d842d0 con 0x5651a4d81700 2024-09-10T10:04:13.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:13.664+0000 7f729fab9740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x5651a4c97350 con 0x5651a4d76e40 2024-09-10T10:04:13.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:13.664+0000 7f729fab9740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x5651a4d84450 con 0x5651a4d76520 2024-09-10T10:04:13.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:13.664+0000 7f729fab9740 10 client.0 _ll_register_callbacks cb 0x5651a4cba290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:04:13.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:13.665+0000 7f7295e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5651a4d76520 0x5651a4d7e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:13.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:13.665+0000 7f7295e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5651a4d76520 0x5651a4d7e670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:34836/0 (socket says 172.21.15.5:34836) 2024-09-10T10:04:13.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:13.665+0000 7f7295e00640 1 -- 172.21.15.5:0/3779830342 learned_addr learned my addr 172.21.15.5:0/3779830342 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T10:04:13.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:13.665+0000 7f7296800640 1 --2- 172.21.15.5:0/3779830342 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5651a4d81700 0x5651a4d7f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:13.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:13.665+0000 7f728fe00640 1 --2- 172.21.15.5:0/3779830342 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5651a4d76e40 0x5651a4d7ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:13.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:13.665+0000 7f7295e00640 1 -- 172.21.15.5:0/3779830342 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5651a4d76e40 msgr2=0x5651a4d7ebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:13.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:13.665+0000 7f7295e00640 1 --2- 172.21.15.5:0/3779830342 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5651a4d76e40 0x5651a4d7ebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:13.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:13.665+0000 7f7295e00640 1 -- 172.21.15.5:0/3779830342 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5651a4d81700 msgr2=0x5651a4d7f0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:13.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:13.665+0000 7f7295e00640 1 --2- 172.21.15.5:0/3779830342 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5651a4d81700 0x5651a4d7f0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:13.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:13.665+0000 7f7295e00640 1 -- 172.21.15.5:0/3779830342 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5651a4ab5960 con 0x5651a4d76520 2024-09-10T10:04:13.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:13.665+0000 7f7295e00640 1 --2- 172.21.15.5:0/3779830342 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5651a4d76520 0x5651a4d7e670 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f7288004170 tx=0x7f7288009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4ba42cabe05241e8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:13.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:13.665+0000 7f727be00640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-09-10T10:04:13.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:13.666+0000 7f727be00640 1 -- 172.21.15.5:0/3779830342 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f728804e070 con 0x5651a4d76520 2024-09-10T10:04:13.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:13.666+0000 7f729fab9740 1 -- 172.21.15.5:0/3779830342 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7280005500 con 0x5651a4d76520 2024-09-10T10:04:13.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:13.666+0000 7f727be00640 1 -- 172.21.15.5:0/3779830342 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f72880037f0 con 0x5651a4d76520 2024-09-10T10:04:13.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:13.666+0000 7f727be00640 1 -- 172.21.15.5:0/3779830342 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f728805a050 con 0x5651a4d76520 2024-09-10T10:04:13.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:13.666+0000 7f729fab9740 10 client.30238 fetch_fsmap learned FSMap version 42 2024-09-10T10:04:13.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:13.666+0000 7f729fab9740 1 -- 172.21.15.5:0/3779830342 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x5651a4d8ffb0 con 0x5651a4d76520 2024-09-10T10:04:13.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:13.667+0000 7f727be00640 1 -- 172.21.15.5:0/3779830342 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7280005500 con 0x5651a4d76520 2024-09-10T10:04:13.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:13.667+0000 7f727be00640 1 -- 172.21.15.5:0/3779830342 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 42) ==== 8370+0+0 (secure 0 0 0) 0x7f728804aad0 con 0x5651a4d76520 2024-09-10T10:04:13.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:13.667+0000 7f729fab9740 10 client.30238 fetch_fsmap finished waiting for FSMap version 42 2024-09-10T10:04:13.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:13.667+0000 7f729fab9740 10 client.30238 resolve_mds: validated gid 24542 aka daemon mds.b 2024-09-10T10:04:13.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:13.667+0000 7f729fab9740 20 client.30238 populate_metadata read hostname 'smithi005' 2024-09-10T10:04:13.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:13.667+0000 7f729fab9740 1 --2- 172.21.15.5:0/3779830342 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x5651a4d913c0 0x5651a4d937f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:13.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:13.667+0000 7f729fab9740 4 client.30238 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:04:13.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:13.667+0000 7f729fab9740 1 -- 172.21.15.5:0/3779830342 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5651a4ce9bc0 con 0x5651a4d913c0 2024-09-10T10:04:13.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:13.667+0000 7f728fe00640 1 --2- 172.21.15.5:0/3779830342 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x5651a4d913c0 0x5651a4d937f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:13.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:13.668+0000 7f728fe00640 1 --2- 172.21.15.5:0/3779830342 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x5651a4d913c0 0x5651a4d937f0 crc :-1 s=READY pgs=345 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c56a2eee66bb465 server_cookie=785c813b20e09168 in_seq=0 out_seq=0 2024-09-10T10:04:13.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:13.668+0000 7f727be00640 10 client.30238 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T10:04:13.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:13.669+0000 7f727be00640 1 -- 172.21.15.5:0/3779830342 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5651a4ce9bc0 con 0x5651a4d913c0 2024-09-10T10:04:13.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:13.669+0000 7f727be00640 10 client.30238 handle_command_reply: tid=0 2024-09-10T10:04:13.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:13.669+0000 7f729fab9740 1 client.30238 shutdown 2024-09-10T10:04:13.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:13.669+0000 7f729fab9740 1 -- 172.21.15.5:0/3779830342 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5651a4d76520 msgr2=0x5651a4d7e670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:13.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:13.669+0000 7f729fab9740 1 --2- 172.21.15.5:0/3779830342 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5651a4d76520 0x5651a4d7e670 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f7288004170 tx=0x7f7288009260 comp rx=0 tx=0).stop 2024-09-10T10:04:13.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:13.669+0000 7f729fab9740 1 -- 172.21.15.5:0/3779830342 shutdown_connections 2024-09-10T10:04:13.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:13.669+0000 7f729fab9740 1 --2- 172.21.15.5:0/3779830342 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x5651a4d913c0 0x5651a4d937f0 crc :-1 s=READY pgs=345 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:13.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:13.669+0000 7f729fab9740 1 --2- 172.21.15.5:0/3779830342 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5651a4d81700 0x5651a4d7f0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:13.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:13.669+0000 7f729fab9740 1 --2- 172.21.15.5:0/3779830342 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5651a4d76e40 0x5651a4d7ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:13.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:13.669+0000 7f729fab9740 1 --2- 172.21.15.5:0/3779830342 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5651a4d76520 0x5651a4d7e670 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:13.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:13.669+0000 7f727be00640 0 client.30238 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T10:04:13.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:13.669+0000 7f729fab9740 1 -- 172.21.15.5:0/3779830342 >> 172.21.15.5:0/3779830342 conn(0x5651a4d72ab0 msgr2=0x5651a4d71cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:13.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:13.669+0000 7f729fab9740 1 -- 172.21.15.5:0/3779830342 shutdown_connections 2024-09-10T10:04:13.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:13.669+0000 7f729fab9740 1 -- 172.21.15.5:0/3779830342 wait complete. 2024-09-10T10:04:13.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:13.670+0000 7f729fab9740 20 client.30238 trim_cache size 0 max 16384 2024-09-10T10:04:13.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:13.688+0000 7f729fab9740 1 Processor -- start 2024-09-10T10:04:13.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:13.688+0000 7f729fab9740 1 -- start start 2024-09-10T10:04:13.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:13.688+0000 7f729fab9740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5651a4d76520 0x5651a4d78f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:13.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:13.688+0000 7f729fab9740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5651a4d76e40 0x5651a4d804d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:13.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:13.688+0000 7f729fab9740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5651a4d81700 0x5651a4d990c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:13.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:13.688+0000 7f729fab9740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f72981b5360 con 0x5651a4d76520 2024-09-10T10:04:13.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:13.688+0000 7f729fab9740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f72981b6ea0 con 0x5651a4d76e40 2024-09-10T10:04:13.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:13.688+0000 7f729fab9740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f72981b5a30 con 0x5651a4d81700 2024-09-10T10:04:13.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:13.689+0000 7f728fe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5651a4d76e40 0x5651a4d804d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:13.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:13.689+0000 7f7295e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5651a4d76520 0x5651a4d78f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:13.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:13.689+0000 7f7295e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5651a4d76520 0x5651a4d78f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36064/0 (socket says 172.21.15.5:36064) 2024-09-10T10:04:13.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:13.689+0000 7f7296800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5651a4d81700 0x5651a4d990c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:13.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:13.689+0000 7f7295e00640 1 -- 172.21.15.5:0/2357537807 learned_addr learned my addr 172.21.15.5:0/2357537807 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T10:04:13.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:13.689+0000 7f728fe00640 1 -- 172.21.15.5:0/2357537807 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5651a4d81700 msgr2=0x5651a4d990c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:13.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:13.689+0000 7f728fe00640 1 --2- 172.21.15.5:0/2357537807 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5651a4d81700 0x5651a4d990c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:13.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:13.689+0000 7f728fe00640 1 -- 172.21.15.5:0/2357537807 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5651a4d76520 msgr2=0x5651a4d78f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:13.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:13.689+0000 7f728fe00640 1 --2- 172.21.15.5:0/2357537807 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5651a4d76520 0x5651a4d78f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:13.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:13.689+0000 7f728fe00640 1 -- 172.21.15.5:0/2357537807 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5651a4d7d620 con 0x5651a4d76e40 2024-09-10T10:04:13.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:13.689+0000 7f7296800640 1 --2- 172.21.15.5:0/2357537807 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5651a4d81700 0x5651a4d990c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:04:13.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:13.689+0000 7f7295e00640 1 --2- 172.21.15.5:0/2357537807 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5651a4d76520 0x5651a4d78f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:04:13.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:13.689+0000 7f728fe00640 1 --2- 172.21.15.5:0/2357537807 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5651a4d76e40 0x5651a4d804d0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f7280004ab0 tx=0x7f7280021040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=30bfdae544796c25 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:13.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:13.690+0000 7f7286a00640 1 -- 172.21.15.5:0/2357537807 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7280004450 con 0x5651a4d76e40 2024-09-10T10:04:13.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:13.690+0000 7f7286a00640 1 -- 172.21.15.5:0/2357537807 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f728000fc70 con 0x5651a4d76e40 2024-09-10T10:04:13.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:13.690+0000 7f7286a00640 1 -- 172.21.15.5:0/2357537807 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72800035f0 con 0x5651a4d76e40 2024-09-10T10:04:13.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:13.690+0000 7f729fab9740 1 -- 172.21.15.5:0/2357537807 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5651a4d76e40 msgr2=0x5651a4d804d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:13.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:13.690+0000 7f729fab9740 1 --2- 172.21.15.5:0/2357537807 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5651a4d76e40 0x5651a4d804d0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f7280004ab0 tx=0x7f7280021040 comp rx=0 tx=0).stop 2024-09-10T10:04:13.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:13.691+0000 7f729fab9740 1 -- 172.21.15.5:0/2357537807 shutdown_connections 2024-09-10T10:04:13.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:13.691+0000 7f729fab9740 1 --2- 172.21.15.5:0/2357537807 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5651a4d81700 0x5651a4d990c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:13.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:13.691+0000 7f729fab9740 1 --2- 172.21.15.5:0/2357537807 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5651a4d76e40 0x5651a4d804d0 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:13.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:13.691+0000 7f729fab9740 1 --2- 172.21.15.5:0/2357537807 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5651a4d76520 0x5651a4d78f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:13.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:13.691+0000 7f729fab9740 1 -- 172.21.15.5:0/2357537807 >> 172.21.15.5:0/2357537807 conn(0x5651a4d72ab0 msgr2=0x5651a4d7fdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:13.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:13.691+0000 7f729fab9740 1 -- 172.21.15.5:0/2357537807 shutdown_connections 2024-09-10T10:04:13.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:13.691+0000 7f729fab9740 1 -- 172.21.15.5:0/2357537807 wait complete. 2024-09-10T10:04:13.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:13.691+0000 7f729fab9740 1 -- start start 2024-09-10T10:04:13.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:13.691+0000 7f729fab9740 1 Processor -- start 2024-09-10T10:04:13.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:13.692+0000 7f729fab9740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5651a4d76520 0x5651a4d85ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:13.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:13.692+0000 7f729fab9740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5651a4d76e40 0x5651a4d86530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:13.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:13.692+0000 7f729fab9740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5651a4d81700 0x5651a4df4050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:13.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:13.692+0000 7f729fab9740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f72981b67d0 con 0x5651a4d81700 2024-09-10T10:04:13.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:13.692+0000 7f729fab9740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f72981b6100 con 0x5651a4d76520 2024-09-10T10:04:13.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:13.692+0000 7f729fab9740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f72981b3340 con 0x5651a4d76e40 2024-09-10T10:04:13.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:13.692+0000 7f729fab9740 10 client.0 _ll_register_callbacks cb 0x7f7288009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:04:13.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:13.692+0000 7f7295e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5651a4d76520 0x5651a4d85ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:13.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:13.692+0000 7f7295e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5651a4d76520 0x5651a4d85ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:52674/0 (socket says 172.21.15.5:52674) 2024-09-10T10:04:13.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:13.692+0000 7f7295e00640 1 -- 172.21.15.5:0/3145203059 learned_addr learned my addr 172.21.15.5:0/3145203059 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T10:04:13.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:13.692+0000 7f728fe00640 1 --2- 172.21.15.5:0/3145203059 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5651a4d76e40 0x5651a4d86530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:13.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:13.692+0000 7f7296800640 1 --2- 172.21.15.5:0/3145203059 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5651a4d81700 0x5651a4df4050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:13.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:13.692+0000 7f7295e00640 1 -- 172.21.15.5:0/3145203059 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5651a4d76e40 msgr2=0x5651a4d86530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:13.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:13.692+0000 7f7295e00640 1 --2- 172.21.15.5:0/3145203059 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5651a4d76e40 0x5651a4d86530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:13.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:13.693+0000 7f7295e00640 1 -- 172.21.15.5:0/3145203059 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5651a4d81700 msgr2=0x5651a4df4050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:13.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:13.693+0000 7f7295e00640 1 --2- 172.21.15.5:0/3145203059 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5651a4d81700 0x5651a4df4050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:13.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:13.693+0000 7f7295e00640 1 -- 172.21.15.5:0/3145203059 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5651a4df4710 con 0x5651a4d76520 2024-09-10T10:04:13.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:13.693+0000 7f7295e00640 1 --2- 172.21.15.5:0/3145203059 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5651a4d76520 0x5651a4d85ff0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f728802a9e0 tx=0x7f7288005990 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c0b7a723afa0d0ee server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:13.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:13.693+0000 7f727be00640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-09-10T10:04:13.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:13.693+0000 7f727be00640 1 -- 172.21.15.5:0/3145203059 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f728804e050 con 0x5651a4d76520 2024-09-10T10:04:13.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:13.693+0000 7f729fab9740 1 -- 172.21.15.5:0/3145203059 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f728000fc70 con 0x5651a4d76520 2024-09-10T10:04:13.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:13.693+0000 7f727be00640 1 -- 172.21.15.5:0/3145203059 <== mon.2 v2:172.21.15.102:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f728803dd50 con 0x5651a4d76520 2024-09-10T10:04:13.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:13.694+0000 7f727be00640 1 -- 172.21.15.5:0/3145203059 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f728806d050 con 0x5651a4d76520 2024-09-10T10:04:13.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:13.694+0000 7f729fab9740 10 client.30260 fetch_fsmap learned FSMap version 42 2024-09-10T10:04:13.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:13.694+0000 7f729fab9740 1 -- 172.21.15.5:0/3145203059 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x5651a4dfcc70 con 0x5651a4d76520 2024-09-10T10:04:13.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:13.694+0000 7f727be00640 1 -- 172.21.15.5:0/3145203059 <== mon.2 v2:172.21.15.102:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f728000fc70 con 0x5651a4d76520 2024-09-10T10:04:13.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:13.694+0000 7f727be00640 1 -- 172.21.15.5:0/3145203059 <== mon.2 v2:172.21.15.102:3300/0 5 ==== fsmap(e 42) ==== 8370+0+0 (secure 0 0 0) 0x7f72880035f0 con 0x5651a4d76520 2024-09-10T10:04:13.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:13.694+0000 7f729fab9740 10 client.30260 fetch_fsmap finished waiting for FSMap version 42 2024-09-10T10:04:13.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:13.694+0000 7f729fab9740 10 client.30260 resolve_mds: validated gid 24542 aka daemon mds.b 2024-09-10T10:04:13.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:13.694+0000 7f729fab9740 20 client.30260 populate_metadata read hostname 'smithi005' 2024-09-10T10:04:13.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:13.694+0000 7f729fab9740 1 --2- 172.21.15.5:0/3145203059 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x5651a4dfd3e0 0x5651a4dff800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:13.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:13.694+0000 7f729fab9740 4 client.30260 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "23d0f3ef", "release": true, "await": true}] 2024-09-10T10:04:13.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:13.694+0000 7f729fab9740 1 -- 172.21.15.5:0/3145203059 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "quiesce db", "set_id": "23d0f3ef", "release": true, "await": true}) -- 0x5651a4ce40d0 con 0x5651a4dfd3e0 2024-09-10T10:04:13.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:13.695+0000 7f728fe00640 1 --2- 172.21.15.5:0/3145203059 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x5651a4dfd3e0 0x5651a4dff800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:13.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:13.695+0000 7f728fe00640 1 --2- 172.21.15.5:0/3145203059 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x5651a4dfd3e0 0x5651a4dff800 crc :-1 s=READY pgs=346 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=985ee14318b4ce34 server_cookie=cf8d081829ec0452 in_seq=0 out_seq=0 2024-09-10T10:04:13.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:13.695+0000 7f727be00640 10 client.30260 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T10:04:13.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:13.760+0000 7f727be00640 1 -- 172.21.15.5:0/3145203059 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5651a4ce40d0 con 0x5651a4dfd3e0 2024-09-10T10:04:13.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:13.760+0000 7f727be00640 10 client.30260 handle_command_reply: tid=0 2024-09-10T10:04:13.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:13.760+0000 7f729fab9740 1 client.30260 shutdown 2024-09-10T10:04:13.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:13.761+0000 7f729fab9740 1 -- 172.21.15.5:0/3145203059 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5651a4d76520 msgr2=0x5651a4d85ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:13.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:13.761+0000 7f729fab9740 1 --2- 172.21.15.5:0/3145203059 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5651a4d76520 0x5651a4d85ff0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f728802a9e0 tx=0x7f7288005990 comp rx=0 tx=0).stop 2024-09-10T10:04:13.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:13.761+0000 7f729fab9740 1 -- 172.21.15.5:0/3145203059 shutdown_connections 2024-09-10T10:04:13.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:13.761+0000 7f729fab9740 1 --2- 172.21.15.5:0/3145203059 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x5651a4dfd3e0 0x5651a4dff800 crc :-1 s=READY pgs=346 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:13.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:13.761+0000 7f729fab9740 1 --2- 172.21.15.5:0/3145203059 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5651a4d81700 0x5651a4df4050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:13.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:13.761+0000 7f729fab9740 1 --2- 172.21.15.5:0/3145203059 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x5651a4d76e40 0x5651a4d86530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:13.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:13.761+0000 7f729fab9740 1 --2- 172.21.15.5:0/3145203059 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x5651a4d76520 0x5651a4d85ff0 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:13.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:13.761+0000 7f729fab9740 1 -- 172.21.15.5:0/3145203059 >> 172.21.15.5:0/3145203059 conn(0x5651a4d72ab0 msgr2=0x5651a4d8cd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:13.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:13.761+0000 7f727be00640 0 client.30260 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T10:04:13.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:13.762+0000 7f729fab9740 1 -- 172.21.15.5:0/3145203059 shutdown_connections 2024-09-10T10:04:13.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:13.762+0000 7f729fab9740 1 -- 172.21.15.5:0/3145203059 wait complete. 2024-09-10T10:04:13.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:13.762+0000 7f729fab9740 20 client.30260 trim_cache size 0 max 16384 2024-09-10T10:04:13.764 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-10T10:04:13.764 INFO:teuthology.orchestra.run.smithi005.stdout: "epoch": 42, 2024-09-10T10:04:13.764 INFO:teuthology.orchestra.run.smithi005.stdout: "leader": 24542, 2024-09-10T10:04:13.764 INFO:teuthology.orchestra.run.smithi005.stdout: "set_version": 92, 2024-09-10T10:04:13.764 INFO:teuthology.orchestra.run.smithi005.stdout: "sets": { 2024-09-10T10:04:13.764 INFO:teuthology.orchestra.run.smithi005.stdout: "23d0f3ef": { 2024-09-10T10:04:13.765 INFO:teuthology.orchestra.run.smithi005.stdout: "version": 92, 2024-09-10T10:04:13.765 INFO:teuthology.orchestra.run.smithi005.stdout: "age_ref": 0.0, 2024-09-10T10:04:13.765 INFO:teuthology.orchestra.run.smithi005.stdout: "state": { 2024-09-10T10:04:13.765 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "RELEASED", 2024-09-10T10:04:13.765 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0 2024-09-10T10:04:13.765 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:13.765 INFO:teuthology.orchestra.run.smithi005.stdout: "timeout": 90.0, 2024-09-10T10:04:13.765 INFO:teuthology.orchestra.run.smithi005.stdout: "expiration": 150.0, 2024-09-10T10:04:13.765 INFO:teuthology.orchestra.run.smithi005.stdout: "members": { 2024-09-10T10:04:13.765 INFO:teuthology.orchestra.run.smithi005.stdout: "file:/": { 2024-09-10T10:04:13.765 INFO:teuthology.orchestra.run.smithi005.stdout: "excluded": false, 2024-09-10T10:04:13.766 INFO:teuthology.orchestra.run.smithi005.stdout: "state": { 2024-09-10T10:04:13.766 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "RELEASED", 2024-09-10T10:04:13.766 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0 2024-09-10T10:04:13.766 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T10:04:13.766 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T10:04:13.766 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T10:04:13.766 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T10:04:13.766 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T10:04:13.766 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-10T10:04:13.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:13.763+0000 7f7297200640 1 -- 172.21.15.5:0/3339924162 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f727c0a2c00 msgr2=0x7f727c0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:13.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:13.763+0000 7f7297200640 1 --2- 172.21.15.5:0/3339924162 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f727c0a2c00 0x7f727c0a50c0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x5651a4d73910 tx=0x7f72880046c0 comp rx=0 tx=0).stop 2024-09-10T10:04:13.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:13.763+0000 7f7297200640 1 -- 172.21.15.5:0/3339924162 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7298112870 msgr2=0x7f729819e720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:13.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:13.763+0000 7f7297200640 1 --2- 172.21.15.5:0/3339924162 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7298112870 0x7f729819e720 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f729807aa10 tx=0x7f729000ef10 comp rx=0 tx=0).stop 2024-09-10T10:04:13.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:13.763+0000 7f7297200640 1 -- 172.21.15.5:0/3339924162 shutdown_connections 2024-09-10T10:04:13.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:13.763+0000 7f7297200640 1 --2- 172.21.15.5:0/3339924162 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f727c0a2c00 0x7f727c0a50c0 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:13.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:13.763+0000 7f7297200640 1 --2- 172.21.15.5:0/3339924162 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7298112870 0x7f729819e720 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:13.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:13.763+0000 7f7297200640 1 --2- 172.21.15.5:0/3339924162 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7298077040 0x7f72981a4b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:13.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:13.763+0000 7f7297200640 1 --2- 172.21.15.5:0/3339924162 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7298076720 0x7f72981a4650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:13.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:13.763+0000 7f7297200640 1 -- 172.21.15.5:0/3339924162 >> 172.21.15.5:0/3339924162 conn(0x7f72980ff8d0 msgr2=0x7f7298101cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:13.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:13.764+0000 7f7297200640 1 -- 172.21.15.5:0/3339924162 shutdown_connections 2024-09-10T10:04:13.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:13.764+0000 7f7297200640 1 -- 172.21.15.5:0/3339924162 wait complete. 2024-09-10T10:04:13.784 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:04:13 smithi005 ceph-mon[30350]: pgmap v1169: 129 pgs: 129 active+clean; 769 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 2.1 MiB/s wr, 57 op/s 2024-09-10T10:04:14.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:04:13 smithi159 ceph-mon[35513]: pgmap v1169: 129 pgs: 129 active+clean; 769 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 2.1 MiB/s wr, 57 op/s 2024-09-10T10:04:14.132 INFO:tasks.workunit.client.0.smithi005.stdout: 10 179058 12.75 MB/sec execute 592 sec latency 23242.167 ms 2024-09-10T10:04:14.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:04:13 smithi102 ceph-mon[35442]: pgmap v1169: 129 pgs: 129 active+clean; 769 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 2.1 MiB/s wr, 57 op/s 2024-09-10T10:04:14.858 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '23d0f3ef', total seconds elapsed: 34.3 2024-09-10T10:04:15.133 INFO:tasks.workunit.client.0.smithi005.stdout: 10 179659 12.78 MB/sec execute 593 sec latency 56.466 ms 2024-09-10T10:04:16.057 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:04:15 smithi005 ceph-mon[30350]: pgmap v1170: 129 pgs: 129 active+clean; 769 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 1.6 MiB/s wr, 50 op/s 2024-09-10T10:04:16.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:04:15 smithi159 ceph-mon[35513]: pgmap v1170: 129 pgs: 129 active+clean; 769 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 1.6 MiB/s wr, 50 op/s 2024-09-10T10:04:16.133 INFO:tasks.workunit.client.0.smithi005.stdout: 10 180277 12.82 MB/sec execute 594 sec latency 58.498 ms 2024-09-10T10:04:16.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:04:15 smithi102 ceph-mon[35442]: pgmap v1170: 129 pgs: 129 active+clean; 769 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 1.6 MiB/s wr, 50 op/s 2024-09-10T10:04:17.133 INFO:tasks.workunit.client.0.smithi005.stdout: 10 180709 12.83 MB/sec execute 595 sec latency 82.182 ms 2024-09-10T10:04:18.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:04:17 smithi005 ceph-mon[30350]: pgmap v1171: 129 pgs: 129 active+clean; 771 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 513 KiB/s rd, 992 KiB/s wr, 60 op/s 2024-09-10T10:04:18.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:04:17 smithi159 ceph-mon[35513]: pgmap v1171: 129 pgs: 129 active+clean; 771 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 513 KiB/s rd, 992 KiB/s wr, 60 op/s 2024-09-10T10:04:18.133 INFO:tasks.workunit.client.0.smithi005.stdout: 10 181314 12.86 MB/sec execute 596 sec latency 88.099 ms 2024-09-10T10:04:18.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:04:17 smithi102 ceph-mon[35442]: pgmap v1171: 129 pgs: 129 active+clean; 771 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 513 KiB/s rd, 992 KiB/s wr, 60 op/s 2024-09-10T10:04:19.133 INFO:tasks.workunit.client.0.smithi005.stdout: 10 181905 12.89 MB/sec execute 597 sec latency 96.523 ms 2024-09-10T10:04:20.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:04:19 smithi005 ceph-mon[30350]: pgmap v1172: 129 pgs: 129 active+clean; 767 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 311 KiB/s wr, 59 op/s 2024-09-10T10:04:20.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:04:19 smithi159 ceph-mon[35513]: pgmap v1172: 129 pgs: 129 active+clean; 767 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 311 KiB/s wr, 59 op/s 2024-09-10T10:04:20.133 INFO:tasks.workunit.client.0.smithi005.stdout: 10 182687 12.94 MB/sec execute 598 sec latency 57.843 ms 2024-09-10T10:04:20.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:04:19 smithi102 ceph-mon[35442]: pgmap v1172: 129 pgs: 129 active+clean; 767 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 311 KiB/s wr, 59 op/s 2024-09-10T10:04:21.133 INFO:tasks.workunit.client.0.smithi005.stdout: 10 183397 12.98 MB/sec execute 599 sec latency 81.298 ms 2024-09-10T10:04:22.057 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:04:21 smithi005 ceph-mon[30350]: pgmap v1173: 129 pgs: 129 active+clean; 769 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 1.6 MiB/s wr, 250 op/s 2024-09-10T10:04:22.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:04:21 smithi159 ceph-mon[35513]: pgmap v1173: 129 pgs: 129 active+clean; 769 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 1.6 MiB/s wr, 250 op/s 2024-09-10T10:04:22.133 INFO:tasks.workunit.client.0.smithi005.stdout: 10 cleanup 600 sec 2024-09-10T10:04:22.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:04:21 smithi102 ceph-mon[35442]: pgmap v1173: 129 pgs: 129 active+clean; 769 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 1.6 MiB/s wr, 250 op/s 2024-09-10T10:04:23.133 INFO:tasks.workunit.client.0.smithi005.stdout: 7 cleanup 601 sec 2024-09-10T10:04:23.437 INFO:tasks.workunit.client.0.smithi005.stdout: 0 cleanup 601 sec 2024-09-10T10:04:23.437 INFO:tasks.workunit.client.0.smithi005.stdout: 2024-09-10T10:04:23.437 INFO:tasks.workunit.client.0.smithi005.stdout: Operation Count AvgLat MaxLat 2024-09-10T10:04:23.438 INFO:tasks.workunit.client.0.smithi005.stdout: ---------------------------------------- 2024-09-10T10:04:23.438 INFO:tasks.workunit.client.0.smithi005.stdout: NTCreateX 250428 10.639 40207.989 2024-09-10T10:04:23.438 INFO:tasks.workunit.client.0.smithi005.stdout: Close 184030 0.006 3.361 2024-09-10T10:04:23.438 INFO:tasks.workunit.client.0.smithi005.stdout: Rename 10636 47.235 39319.372 2024-09-10T10:04:23.438 INFO:tasks.workunit.client.0.smithi005.stdout: Unlink 50483 18.401 32873.239 2024-09-10T10:04:23.438 INFO:tasks.workunit.client.0.smithi005.stdout: Qpathinfo 227222 4.349 20826.361 2024-09-10T10:04:23.438 INFO:tasks.workunit.client.0.smithi005.stdout: Qfileinfo 39692 0.002 3.259 2024-09-10T10:04:23.438 INFO:tasks.workunit.client.0.smithi005.stdout: Qfsinfo 41624 2.929 18684.263 2024-09-10T10:04:23.438 INFO:tasks.workunit.client.0.smithi005.stdout: Sfileinfo 20442 13.142 40242.282 2024-09-10T10:04:23.438 INFO:tasks.workunit.client.0.smithi005.stdout: Find 87804 1.517 21972.130 2024-09-10T10:04:23.438 INFO:tasks.workunit.client.0.smithi005.stdout: WriteX 123898 0.928 33153.576 2024-09-10T10:04:23.438 INFO:tasks.workunit.client.0.smithi005.stdout: ReadX 393546 0.138 18518.952 2024-09-10T10:04:23.438 INFO:tasks.workunit.client.0.smithi005.stdout: LockX 820 3.338 23.801 2024-09-10T10:04:23.439 INFO:tasks.workunit.client.0.smithi005.stdout: UnlockX 820 2.790 21.495 2024-09-10T10:04:23.439 INFO:tasks.workunit.client.0.smithi005.stdout: Flush 17553 13.982 18625.913 2024-09-10T10:04:23.439 INFO:tasks.workunit.client.0.smithi005.stdout: 2024-09-10T10:04:23.439 INFO:tasks.workunit.client.0.smithi005.stdout:Throughput 12.9832 MB/sec 10 clients 10 procs max_latency=40242.286 ms 2024-09-10T10:04:23.440 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-09-10T10:04:23.440 DEBUG:teuthology.orchestra.run.smithi005:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-09-10T10:04:23.753 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:04:23 smithi005 ceph-mon[30350]: pgmap v1174: 129 pgs: 129 active+clean; 772 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 2.0 MiB/s wr, 292 op/s 2024-09-10T10:04:24.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:04:23 smithi159 ceph-mon[35513]: pgmap v1174: 129 pgs: 129 active+clean; 772 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 2.0 MiB/s wr, 292 op/s 2024-09-10T10:04:24.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:04:23 smithi102 ceph-mon[35442]: pgmap v1174: 129 pgs: 129 active+clean; 772 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 2.0 MiB/s wr, 292 op/s 2024-09-10T10:04:24.500 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T10:04:25.066 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:04:25.094 INFO:tasks.workunit:Stopping ['suites/dbench.sh'] on client.0... 2024-09-10T10:04:25.094 DEBUG:teuthology.orchestra.run.smithi005:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-09-10T10:04:25.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:25.692+0000 7f6869600640 1 -- 172.21.15.5:0/2278308819 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6864104010 msgr2=0x7f68641043f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:25.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:25.692+0000 7f6869600640 1 --2- 172.21.15.5:0/2278308819 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6864104010 0x7f68641043f0 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f6854009900 tx=0x7f685402a3f0 comp rx=0 tx=0).stop 2024-09-10T10:04:25.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:25.692+0000 7f6869600640 1 -- 172.21.15.5:0/2278308819 shutdown_connections 2024-09-10T10:04:25.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:25.692+0000 7f6869600640 1 --2- 172.21.15.5:0/2278308819 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6864109140 0x7f6864110cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:25.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:25.692+0000 7f6869600640 1 --2- 172.21.15.5:0/2278308819 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f68641049c0 0x7f6864108a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:25.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:25.692+0000 7f6869600640 1 --2- 172.21.15.5:0/2278308819 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6864104010 0x7f68641043f0 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:25.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:25.692+0000 7f6869600640 1 -- 172.21.15.5:0/2278308819 >> 172.21.15.5:0/2278308819 conn(0x7f68640ffa50 msgr2=0x7f6864101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:25.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:25.692+0000 7f6869600640 1 -- 172.21.15.5:0/2278308819 shutdown_connections 2024-09-10T10:04:25.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:25.692+0000 7f6869600640 1 -- 172.21.15.5:0/2278308819 wait complete. 2024-09-10T10:04:25.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:25.693+0000 7f6869600640 1 Processor -- start 2024-09-10T10:04:25.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:25.693+0000 7f6869600640 1 -- start start 2024-09-10T10:04:25.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:25.693+0000 7f6869600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6864104010 0x7f686419a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:25.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:25.694+0000 7f6869600640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f68641049c0 0x7f686419ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:25.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:25.694+0000 7f6869600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6864109140 0x7f686419eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:25.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:25.694+0000 7f6869600640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f6864114140 con 0x7f6864109140 2024-09-10T10:04:25.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:25.694+0000 7f6869600640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f6864113fc0 con 0x7f6864104010 2024-09-10T10:04:25.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:25.694+0000 7f6869600640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f68641142c0 con 0x7f68641049c0 2024-09-10T10:04:25.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:25.694+0000 7f6863400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6864109140 0x7f686419eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:25.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:25.694+0000 7f6862a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6864104010 0x7f686419a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:25.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:25.694+0000 7f6863400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6864109140 0x7f686419eed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35592/0 (socket says 172.21.15.5:35592) 2024-09-10T10:04:25.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:25.694+0000 7f6862000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f68641049c0 0x7f686419ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:25.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:25.694+0000 7f6863400640 1 -- 172.21.15.5:0/3604812929 learned_addr learned my addr 172.21.15.5:0/3604812929 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T10:04:25.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:25.694+0000 7f6862000640 1 -- 172.21.15.5:0/3604812929 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6864104010 msgr2=0x7f686419a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:25.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:25.694+0000 7f6862000640 1 --2- 172.21.15.5:0/3604812929 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6864104010 0x7f686419a600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:25.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:25.694+0000 7f6862000640 1 -- 172.21.15.5:0/3604812929 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6864109140 msgr2=0x7f686419eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:25.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:25.694+0000 7f6862000640 1 --2- 172.21.15.5:0/3604812929 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6864109140 0x7f686419eed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:25.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:25.694+0000 7f6862000640 1 -- 172.21.15.5:0/3604812929 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f686419f5b0 con 0x7f68641049c0 2024-09-10T10:04:25.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:25.695+0000 7f6862000640 1 --2- 172.21.15.5:0/3604812929 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f68641049c0 0x7f686419ab40 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f685000e9f0 tx=0x7f685000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:25.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:25.695+0000 7f685b400640 1 -- 172.21.15.5:0/3604812929 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f685000ccc0 con 0x7f68641049c0 2024-09-10T10:04:25.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:25.695+0000 7f6869600640 1 -- 172.21.15.5:0/3604812929 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f686419f8a0 con 0x7f68641049c0 2024-09-10T10:04:25.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:25.695+0000 7f685b400640 1 -- 172.21.15.5:0/3604812929 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6850010910 con 0x7f68641049c0 2024-09-10T10:04:25.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:25.695+0000 7f685b400640 1 -- 172.21.15.5:0/3604812929 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6850018960 con 0x7f68641049c0 2024-09-10T10:04:25.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:25.695+0000 7f6869600640 1 -- 172.21.15.5:0/3604812929 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68641a7180 con 0x7f68641049c0 2024-09-10T10:04:25.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:25.698+0000 7f686c21a740 1 Processor -- start 2024-09-10T10:04:25.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:25.698+0000 7f686c21a740 1 -- start start 2024-09-10T10:04:25.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:25.698+0000 7f685b400640 1 -- 172.21.15.5:0/3604812929 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f6850010ac0 con 0x7f68641049c0 2024-09-10T10:04:25.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:25.699+0000 7f686c21a740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c1ca99db90 0x55c1ca99df70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:25.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:25.699+0000 7f686c21a740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c1ca99e4b0 0x55c1ca9a8830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:25.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:25.699+0000 7f686c21a740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c1ca9a8d70 0x55c1ca9ab160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:25.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:25.699+0000 7f686c21a740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55c1ca9043e0 con 0x55c1ca9a8d70 2024-09-10T10:04:25.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:25.699+0000 7f686c21a740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55c1ca8abd80 con 0x55c1ca99e4b0 2024-09-10T10:04:25.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:25.699+0000 7f686c21a740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55c1ca9003a0 con 0x55c1ca99db90 2024-09-10T10:04:25.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:25.699+0000 7f6863400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c1ca99db90 0x55c1ca99df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:25.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:25.699+0000 7f6862a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c1ca99e4b0 0x55c1ca9a8830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:25.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:25.699+0000 7f6863400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c1ca99db90 0x55c1ca99df70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:39998/0 (socket says 172.21.15.5:39998) 2024-09-10T10:04:25.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:25.699+0000 7f6863400640 1 -- 172.21.15.5:0/1493533599 learned_addr learned my addr 172.21.15.5:0/1493533599 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T10:04:25.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:25.700+0000 7f6862000640 1 --2- 172.21.15.5:0/1493533599 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c1ca9a8d70 0x55c1ca9ab160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:25.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:25.700+0000 7f6862a00640 1 -- 172.21.15.5:0/1493533599 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c1ca99db90 msgr2=0x55c1ca99df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:25.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:25.700+0000 7f6862a00640 1 --2- 172.21.15.5:0/1493533599 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c1ca99db90 0x55c1ca99df70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:25.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:25.700+0000 7f6862a00640 1 -- 172.21.15.5:0/1493533599 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c1ca9a8d70 msgr2=0x55c1ca9ab160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:25.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:25.700+0000 7f6862a00640 1 --2- 172.21.15.5:0/1493533599 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c1ca9a8d70 0x55c1ca9ab160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:25.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:25.700+0000 7f6862a00640 1 -- 172.21.15.5:0/1493533599 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c1ca8caae0 con 0x55c1ca99e4b0 2024-09-10T10:04:25.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:25.700+0000 7f6862a00640 1 --2- 172.21.15.5:0/1493533599 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c1ca99e4b0 0x55c1ca9a8830 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f6854005940 tx=0x7f6854005e00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1bc464ab0648f890 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:25.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:25.701+0000 7f685b400640 1 --2- 172.21.15.5:0/3604812929 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f68480a2c00 0x7f68480a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:25.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:25.701+0000 7f684f400640 1 -- 172.21.15.5:0/1493533599 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6854040070 con 0x55c1ca99e4b0 2024-09-10T10:04:25.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:25.701+0000 7f684f400640 1 -- 172.21.15.5:0/1493533599 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f68540318f0 con 0x55c1ca99e4b0 2024-09-10T10:04:25.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:25.701+0000 7f684f400640 1 -- 172.21.15.5:0/1493533599 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6854035040 con 0x55c1ca99e4b0 2024-09-10T10:04:25.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:25.701+0000 7f685b400640 1 -- 172.21.15.5:0/3604812929 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f68500cf780 con 0x7f68641049c0 2024-09-10T10:04:25.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:25.701+0000 7f686c21a740 1 -- 172.21.15.5:0/1493533599 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c1ca99e4b0 msgr2=0x55c1ca9a8830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:25.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:25.701+0000 7f686c21a740 1 --2- 172.21.15.5:0/1493533599 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c1ca99e4b0 0x55c1ca9a8830 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f6854005940 tx=0x7f6854005e00 comp rx=0 tx=0).stop 2024-09-10T10:04:25.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:25.702+0000 7f6863400640 1 --2- 172.21.15.5:0/3604812929 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f68480a2c00 0x7f68480a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:25.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:25.702+0000 7f686c21a740 1 -- 172.21.15.5:0/1493533599 shutdown_connections 2024-09-10T10:04:25.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:25.702+0000 7f686c21a740 1 --2- 172.21.15.5:0/1493533599 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c1ca9a8d70 0x55c1ca9ab160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:25.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:25.702+0000 7f686c21a740 1 --2- 172.21.15.5:0/1493533599 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c1ca99e4b0 0x55c1ca9a8830 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:25.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:25.702+0000 7f686c21a740 1 --2- 172.21.15.5:0/1493533599 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c1ca99db90 0x55c1ca99df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:25.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:25.702+0000 7f686c21a740 1 -- 172.21.15.5:0/1493533599 >> 172.21.15.5:0/1493533599 conn(0x55c1ca99a120 msgr2=0x55c1ca99a500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:25.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:25.702+0000 7f686c21a740 1 -- 172.21.15.5:0/1493533599 shutdown_connections 2024-09-10T10:04:25.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:25.702+0000 7f686c21a740 1 -- 172.21.15.5:0/1493533599 wait complete. 2024-09-10T10:04:25.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:25.702+0000 7f686c21a740 1 -- start start 2024-09-10T10:04:25.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:25.702+0000 7f6863400640 1 --2- 172.21.15.5:0/3604812929 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f68480a2c00 0x7f68480a50c0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f685c005f40 tx=0x7f685c009680 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:25.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:25.703+0000 7f686c21a740 1 Processor -- start 2024-09-10T10:04:25.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:25.703+0000 7f686c21a740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c1ca99db90 0x55c1ca9a5c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:25.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:25.703+0000 7f686c21a740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c1ca99e4b0 0x55c1ca9a6150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:25.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:25.703+0000 7f686c21a740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c1ca9a8d70 0x55c1ca9a6690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:25.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:25.703+0000 7f686c21a740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55c1ca9ab9e0 con 0x55c1ca99e4b0 2024-09-10T10:04:25.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:25.703+0000 7f686c21a740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55c1ca8bf0f0 con 0x55c1ca99db90 2024-09-10T10:04:25.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:25.703+0000 7f686c21a740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55c1ca9abb60 con 0x55c1ca9a8d70 2024-09-10T10:04:25.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:25.703+0000 7f686c21a740 10 client.0 _ll_register_callbacks cb 0x55c1ca8b09e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:04:25.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:25.704+0000 7f6862000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c1ca99e4b0 0x55c1ca9a6150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:25.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:25.704+0000 7f6863400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c1ca9a8d70 0x55c1ca9a6690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:25.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:25.704+0000 7f6862a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c1ca99db90 0x55c1ca9a5c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:25.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:25.704+0000 7f6862000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c1ca99e4b0 0x55c1ca9a6150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35618/0 (socket says 172.21.15.5:35618) 2024-09-10T10:04:25.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:25.704+0000 7f6862000640 1 -- 172.21.15.5:0/3705374230 learned_addr learned my addr 172.21.15.5:0/3705374230 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T10:04:25.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:25.704+0000 7f6863400640 1 -- 172.21.15.5:0/3705374230 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c1ca99db90 msgr2=0x55c1ca9a5c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:25.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:25.704+0000 7f6863400640 1 --2- 172.21.15.5:0/3705374230 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c1ca99db90 0x55c1ca9a5c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:25.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:25.704+0000 7f6863400640 1 -- 172.21.15.5:0/3705374230 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c1ca99e4b0 msgr2=0x55c1ca9a6150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:25.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:25.704+0000 7f6863400640 1 --2- 172.21.15.5:0/3705374230 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c1ca99e4b0 0x55c1ca9a6150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:25.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:25.704+0000 7f6863400640 1 -- 172.21.15.5:0/3705374230 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c1ca6753c0 con 0x55c1ca9a8d70 2024-09-10T10:04:25.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:25.704+0000 7f6863400640 1 --2- 172.21.15.5:0/3705374230 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c1ca9a8d70 0x55c1ca9a6690 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f685c009270 tx=0x7f685c004ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7d893c0b0366c531 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:25.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:25.704+0000 7f6843e00640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-09-10T10:04:25.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:25.705+0000 7f6843e00640 1 -- 172.21.15.5:0/3705374230 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f685c004b20 con 0x55c1ca9a8d70 2024-09-10T10:04:25.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:25.705+0000 7f6843e00640 1 -- 172.21.15.5:0/3705374230 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f685c018030 con 0x55c1ca9a8d70 2024-09-10T10:04:25.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:25.705+0000 7f686c21a740 1 -- 172.21.15.5:0/3705374230 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f68540318f0 con 0x55c1ca9a8d70 2024-09-10T10:04:25.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:25.705+0000 7f6843e00640 1 -- 172.21.15.5:0/3705374230 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f685c033050 con 0x55c1ca9a8d70 2024-09-10T10:04:25.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:25.705+0000 7f686c21a740 10 client.30256 fetch_fsmap learned FSMap version 43 2024-09-10T10:04:25.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:25.705+0000 7f686c21a740 1 -- 172.21.15.5:0/3705374230 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55c1ca9b76a0 con 0x55c1ca9a8d70 2024-09-10T10:04:25.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:25.705+0000 7f6843e00640 1 -- 172.21.15.5:0/3705374230 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f68540318f0 con 0x55c1ca9a8d70 2024-09-10T10:04:25.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:25.706+0000 7f6843e00640 1 -- 172.21.15.5:0/3705374230 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 43) ==== 8374+0+0 (secure 0 0 0) 0x7f685c00f040 con 0x55c1ca9a8d70 2024-09-10T10:04:25.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:25.706+0000 7f686c21a740 10 client.30256 fetch_fsmap finished waiting for FSMap version 43 2024-09-10T10:04:25.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:25.706+0000 7f686c21a740 10 client.30256 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T10:04:25.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:25.706+0000 7f686c21a740 20 client.30256 populate_metadata read hostname 'smithi005' 2024-09-10T10:04:25.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:25.706+0000 7f686c21a740 1 --2- 172.21.15.5:0/3705374230 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55c1ca9b8b60 0x55c1ca9bb000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:25.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:25.706+0000 7f686c21a740 4 client.30256 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:04:25.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:25.706+0000 7f686c21a740 1 -- 172.21.15.5:0/3705374230 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c1ca911950 con 0x55c1ca9b8b60 2024-09-10T10:04:25.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:25.706+0000 7f6862a00640 1 --2- 172.21.15.5:0/3705374230 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55c1ca9b8b60 0x55c1ca9bb000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:25.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:25.707+0000 7f6862a00640 1 --2- 172.21.15.5:0/3705374230 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55c1ca9b8b60 0x55c1ca9bb000 crc :-1 s=READY pgs=347 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=934223a7695f6783 server_cookie=398df691483678e5 in_seq=0 out_seq=0 2024-09-10T10:04:25.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:25.707+0000 7f6843e00640 10 client.30256 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T10:04:25.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:25.708+0000 7f6843e00640 1 -- 172.21.15.5:0/3705374230 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c1ca911950 con 0x55c1ca9b8b60 2024-09-10T10:04:25.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:25.708+0000 7f6843e00640 10 client.30256 handle_command_reply: tid=0 2024-09-10T10:04:25.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:25.708+0000 7f686c21a740 1 client.30256 shutdown 2024-09-10T10:04:25.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:25.708+0000 7f686c21a740 1 -- 172.21.15.5:0/3705374230 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c1ca9a8d70 msgr2=0x55c1ca9a6690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:25.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:25.708+0000 7f686c21a740 1 --2- 172.21.15.5:0/3705374230 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c1ca9a8d70 0x55c1ca9a6690 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f685c009270 tx=0x7f685c004ab0 comp rx=0 tx=0).stop 2024-09-10T10:04:25.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:25.708+0000 7f686c21a740 1 -- 172.21.15.5:0/3705374230 shutdown_connections 2024-09-10T10:04:25.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:25.708+0000 7f686c21a740 1 --2- 172.21.15.5:0/3705374230 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55c1ca9b8b60 0x55c1ca9bb000 crc :-1 s=READY pgs=347 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:25.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:25.708+0000 7f686c21a740 1 --2- 172.21.15.5:0/3705374230 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c1ca9a8d70 0x55c1ca9a6690 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:25.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:25.708+0000 7f686c21a740 1 --2- 172.21.15.5:0/3705374230 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c1ca99e4b0 0x55c1ca9a6150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:25.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:25.708+0000 7f686c21a740 1 --2- 172.21.15.5:0/3705374230 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c1ca99db90 0x55c1ca9a5c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:25.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:25.708+0000 7f686c21a740 1 -- 172.21.15.5:0/3705374230 >> 172.21.15.5:0/3705374230 conn(0x55c1ca99a120 msgr2=0x55c1ca99fe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:25.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:25.708+0000 7f6843e00640 0 client.30256 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T10:04:25.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:25.708+0000 7f686c21a740 1 -- 172.21.15.5:0/3705374230 shutdown_connections 2024-09-10T10:04:25.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:25.708+0000 7f686c21a740 1 -- 172.21.15.5:0/3705374230 wait complete. 2024-09-10T10:04:25.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:25.708+0000 7f686c21a740 20 client.30256 trim_cache size 0 max 16384 2024-09-10T10:04:25.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:25.728+0000 7f686c21a740 1 Processor -- start 2024-09-10T10:04:25.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:25.728+0000 7f686c21a740 1 -- start start 2024-09-10T10:04:25.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:25.728+0000 7f686c21a740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c1ca99db90 0x55c1ca9a05f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:25.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:25.728+0000 7f686c21a740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c1ca99e4b0 0x55c1ca9c0920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:25.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:25.728+0000 7f686c21a740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c1ca9a8d70 0x55c1ca9c0e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:25.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:25.728+0000 7f686c21a740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f68641b1050 con 0x55c1ca9a8d70 2024-09-10T10:04:25.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:25.728+0000 7f686c21a740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f68641b2b90 con 0x55c1ca99db90 2024-09-10T10:04:25.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:25.728+0000 7f686c21a740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f68641b1720 con 0x55c1ca99e4b0 2024-09-10T10:04:25.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:25.729+0000 7f6862a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c1ca99db90 0x55c1ca9a05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:25.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:25.729+0000 7f6862a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c1ca99db90 0x55c1ca9a05f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:47470/0 (socket says 172.21.15.5:47470) 2024-09-10T10:04:25.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:25.729+0000 7f6862a00640 1 -- 172.21.15.5:0/861382557 learned_addr learned my addr 172.21.15.5:0/861382557 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T10:04:25.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:25.729+0000 7f6862000640 1 --2- 172.21.15.5:0/861382557 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c1ca99e4b0 0x55c1ca9c0920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:25.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:25.729+0000 7f6863400640 1 --2- 172.21.15.5:0/861382557 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c1ca9a8d70 0x55c1ca9c0e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:25.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:25.729+0000 7f6862a00640 1 -- 172.21.15.5:0/861382557 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c1ca99e4b0 msgr2=0x55c1ca9c0920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:25.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:25.729+0000 7f6862a00640 1 --2- 172.21.15.5:0/861382557 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c1ca99e4b0 0x55c1ca9c0920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:25.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:25.729+0000 7f6862a00640 1 -- 172.21.15.5:0/861382557 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c1ca9a8d70 msgr2=0x55c1ca9c0e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:25.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:25.729+0000 7f6862a00640 1 --2- 172.21.15.5:0/861382557 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c1ca9a8d70 0x55c1ca9c0e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:25.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:25.729+0000 7f6862a00640 1 -- 172.21.15.5:0/861382557 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c1ca9bb680 con 0x55c1ca99db90 2024-09-10T10:04:25.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:25.729+0000 7f6862000640 1 --2- 172.21.15.5:0/861382557 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c1ca99e4b0 0x55c1ca9c0920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:04:25.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:25.729+0000 7f6862a00640 1 --2- 172.21.15.5:0/861382557 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c1ca99db90 0x55c1ca9a05f0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f6854005940 tx=0x7f685402a3f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=884bd62db4878900 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:25.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:25.730+0000 7f684ea00640 1 -- 172.21.15.5:0/861382557 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6854040070 con 0x55c1ca99db90 2024-09-10T10:04:25.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:25.730+0000 7f684ea00640 1 -- 172.21.15.5:0/861382557 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f68540318f0 con 0x55c1ca99db90 2024-09-10T10:04:25.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:25.730+0000 7f684ea00640 1 -- 172.21.15.5:0/861382557 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6854035040 con 0x55c1ca99db90 2024-09-10T10:04:25.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:25.730+0000 7f686c21a740 1 -- 172.21.15.5:0/861382557 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c1ca99db90 msgr2=0x55c1ca9a05f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:25.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:25.730+0000 7f686c21a740 1 --2- 172.21.15.5:0/861382557 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c1ca99db90 0x55c1ca9a05f0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f6854005940 tx=0x7f685402a3f0 comp rx=0 tx=0).stop 2024-09-10T10:04:25.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:25.731+0000 7f686c21a740 1 -- 172.21.15.5:0/861382557 shutdown_connections 2024-09-10T10:04:25.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:25.731+0000 7f686c21a740 1 --2- 172.21.15.5:0/861382557 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c1ca9a8d70 0x55c1ca9c0e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:25.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:25.731+0000 7f686c21a740 1 --2- 172.21.15.5:0/861382557 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c1ca99e4b0 0x55c1ca9c0920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:25.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:25.731+0000 7f686c21a740 1 --2- 172.21.15.5:0/861382557 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c1ca99db90 0x55c1ca9a05f0 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:25.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:25.731+0000 7f686c21a740 1 -- 172.21.15.5:0/861382557 >> 172.21.15.5:0/861382557 conn(0x55c1ca99a120 msgr2=0x55c1ca9a44e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:25.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:25.731+0000 7f686c21a740 1 -- 172.21.15.5:0/861382557 shutdown_connections 2024-09-10T10:04:25.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:25.731+0000 7f686c21a740 1 -- 172.21.15.5:0/861382557 wait complete. 2024-09-10T10:04:25.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:25.731+0000 7f686c21a740 1 -- start start 2024-09-10T10:04:25.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:25.731+0000 7f686c21a740 1 Processor -- start 2024-09-10T10:04:25.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:25.732+0000 7f686c21a740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c1ca99db90 0x55c1ca9ace20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:25.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:25.732+0000 7f686c21a740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c1ca99e4b0 0x55c1ca9ad360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:25.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:25.732+0000 7f686c21a740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c1ca9a8d70 0x55c1ca9ad8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:25.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:25.732+0000 7f686c21a740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f68641b24c0 con 0x55c1ca99e4b0 2024-09-10T10:04:25.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:25.732+0000 7f686c21a740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f68641b1df0 con 0x55c1ca9a8d70 2024-09-10T10:04:25.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:25.732+0000 7f686c21a740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f68641af030 con 0x55c1ca99db90 2024-09-10T10:04:25.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:25.732+0000 7f686c21a740 10 client.0 _ll_register_callbacks cb 0x7f685c011690 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:04:25.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:25.732+0000 7f6862a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c1ca99db90 0x55c1ca9ace20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:25.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:25.732+0000 7f6862000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c1ca99e4b0 0x55c1ca9ad360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:25.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:25.732+0000 7f6863400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c1ca9a8d70 0x55c1ca9ad8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:25.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:25.732+0000 7f6862a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c1ca99db90 0x55c1ca9ace20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:40028/0 (socket says 172.21.15.5:40028) 2024-09-10T10:04:25.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:25.732+0000 7f6863400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c1ca9a8d70 0x55c1ca9ad8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:47484/0 (socket says 172.21.15.5:47484) 2024-09-10T10:04:25.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:25.732+0000 7f6862a00640 1 -- 172.21.15.5:0/2155978951 learned_addr learned my addr 172.21.15.5:0/2155978951 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T10:04:25.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:25.733+0000 7f6862a00640 1 -- 172.21.15.5:0/2155978951 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c1ca9a8d70 msgr2=0x55c1ca9ad8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:25.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:25.733+0000 7f6862a00640 1 --2- 172.21.15.5:0/2155978951 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c1ca9a8d70 0x55c1ca9ad8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:25.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:25.733+0000 7f6862a00640 1 -- 172.21.15.5:0/2155978951 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c1ca99e4b0 msgr2=0x55c1ca9ad360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:25.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:25.733+0000 7f6862a00640 1 --2- 172.21.15.5:0/2155978951 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c1ca99e4b0 0x55c1ca9ad360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:25.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:25.733+0000 7f6862a00640 1 -- 172.21.15.5:0/2155978951 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c1caa22be0 con 0x55c1ca99db90 2024-09-10T10:04:25.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:25.733+0000 7f6862a00640 1 --2- 172.21.15.5:0/2155978951 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c1ca99db90 0x55c1ca9ace20 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f6854000c00 tx=0x7f6854005780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d22213d22e0198bc server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:25.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:25.733+0000 7f6843e00640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-09-10T10:04:25.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:25.733+0000 7f6843e00640 1 -- 172.21.15.5:0/2155978951 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6854035040 con 0x55c1ca99db90 2024-09-10T10:04:25.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:25.733+0000 7f686c21a740 1 -- 172.21.15.5:0/2155978951 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f68540318f0 con 0x55c1ca99db90 2024-09-10T10:04:25.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:25.733+0000 7f6843e00640 1 -- 172.21.15.5:0/2155978951 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f6854003c90 con 0x55c1ca99db90 2024-09-10T10:04:25.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:25.734+0000 7f6843e00640 1 -- 172.21.15.5:0/2155978951 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f6854069050 con 0x55c1ca99db90 2024-09-10T10:04:25.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:25.734+0000 7f686c21a740 10 client.30262 fetch_fsmap learned FSMap version 43 2024-09-10T10:04:25.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:25.734+0000 7f686c21a740 1 -- 172.21.15.5:0/2155978951 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55c1caa23ef0 con 0x55c1ca99db90 2024-09-10T10:04:25.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:25.734+0000 7f6843e00640 1 -- 172.21.15.5:0/2155978951 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f68540318f0 con 0x55c1ca99db90 2024-09-10T10:04:25.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:25.734+0000 7f6843e00640 1 -- 172.21.15.5:0/2155978951 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 43) ==== 8374+0+0 (secure 0 0 0) 0x7f685405b860 con 0x55c1ca99db90 2024-09-10T10:04:25.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:25.734+0000 7f686c21a740 10 client.30262 fetch_fsmap finished waiting for FSMap version 43 2024-09-10T10:04:25.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:25.734+0000 7f686c21a740 10 client.30262 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T10:04:25.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:25.734+0000 7f686c21a740 20 client.30262 populate_metadata read hostname 'smithi005' 2024-09-10T10:04:25.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:25.734+0000 7f686c21a740 1 --2- 172.21.15.5:0/2155978951 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55c1caa24170 0x55c1caa26560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:25.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:25.734+0000 7f686c21a740 4 client.30262 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T10:04:25.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:25.734+0000 7f686c21a740 1 -- 172.21.15.5:0/2155978951 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55c1ca90be60 con 0x55c1caa24170 2024-09-10T10:04:25.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:25.735+0000 7f6862000640 1 --2- 172.21.15.5:0/2155978951 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55c1caa24170 0x55c1caa26560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:25.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:25.735+0000 7f6862000640 1 --2- 172.21.15.5:0/2155978951 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55c1caa24170 0x55c1caa26560 crc :-1 s=READY pgs=348 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c50658b4b33d5c43 server_cookie=c936ffc54ecb1cd4 in_seq=0 out_seq=0 2024-09-10T10:04:25.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:25.735+0000 7f6843e00640 10 client.30262 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T10:04:25.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:25.736+0000 7f6843e00640 1 -- 172.21.15.5:0/2155978951 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55c1ca90be60 con 0x55c1caa24170 2024-09-10T10:04:25.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:25.736+0000 7f6843e00640 10 client.30262 handle_command_reply: tid=0 2024-09-10T10:04:25.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:25.736+0000 7f686c21a740 1 client.30262 shutdown 2024-09-10T10:04:25.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:25.736+0000 7f686c21a740 1 -- 172.21.15.5:0/2155978951 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c1ca99db90 msgr2=0x55c1ca9ace20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:25.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:25.736+0000 7f686c21a740 1 --2- 172.21.15.5:0/2155978951 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c1ca99db90 0x55c1ca9ace20 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f6854000c00 tx=0x7f6854005780 comp rx=0 tx=0).stop 2024-09-10T10:04:25.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:25.736+0000 7f686c21a740 1 -- 172.21.15.5:0/2155978951 shutdown_connections 2024-09-10T10:04:25.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:25.736+0000 7f686c21a740 1 --2- 172.21.15.5:0/2155978951 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55c1caa24170 0x55c1caa26560 crc :-1 s=READY pgs=348 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:25.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:25.736+0000 7f686c21a740 1 --2- 172.21.15.5:0/2155978951 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55c1ca9a8d70 0x55c1ca9ad8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:25.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:25.736+0000 7f686c21a740 1 --2- 172.21.15.5:0/2155978951 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c1ca99e4b0 0x55c1ca9ad360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:25.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:25.736+0000 7f686c21a740 1 --2- 172.21.15.5:0/2155978951 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55c1ca99db90 0x55c1ca9ace20 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:25.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:25.736+0000 7f6843e00640 0 client.30262 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T10:04:25.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:25.736+0000 7f686c21a740 1 -- 172.21.15.5:0/2155978951 >> 172.21.15.5:0/2155978951 conn(0x55c1ca99a120 msgr2=0x55c1ca9a48c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:25.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:25.736+0000 7f686c21a740 1 -- 172.21.15.5:0/2155978951 shutdown_connections 2024-09-10T10:04:25.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:25.736+0000 7f686c21a740 1 -- 172.21.15.5:0/2155978951 wait complete. 2024-09-10T10:04:25.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:25.736+0000 7f686c21a740 20 client.30262 trim_cache size 0 max 16384 2024-09-10T10:04:25.741 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-10T10:04:25.741 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "no active scrubs running", 2024-09-10T10:04:25.741 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": {} 2024-09-10T10:04:25.741 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-10T10:04:25.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:25.737+0000 7f6869600640 1 -- 172.21.15.5:0/3604812929 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f68480a2c00 msgr2=0x7f68480a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:25.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:25.737+0000 7f6869600640 1 --2- 172.21.15.5:0/3604812929 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f68480a2c00 0x7f68480a50c0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f685c005f40 tx=0x7f685c009680 comp rx=0 tx=0).stop 2024-09-10T10:04:25.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:25.737+0000 7f6869600640 1 -- 172.21.15.5:0/3604812929 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f68641049c0 msgr2=0x7f686419ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:25.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:25.737+0000 7f6869600640 1 --2- 172.21.15.5:0/3604812929 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f68641049c0 0x7f686419ab40 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f685000e9f0 tx=0x7f685000eeb0 comp rx=0 tx=0).stop 2024-09-10T10:04:25.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:25.737+0000 7f6869600640 1 -- 172.21.15.5:0/3604812929 shutdown_connections 2024-09-10T10:04:25.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:25.737+0000 7f6869600640 1 --2- 172.21.15.5:0/3604812929 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f68480a2c00 0x7f68480a50c0 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:25.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:25.737+0000 7f6869600640 1 --2- 172.21.15.5:0/3604812929 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6864109140 0x7f686419eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:25.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:25.737+0000 7f6869600640 1 --2- 172.21.15.5:0/3604812929 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f68641049c0 0x7f686419ab40 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:25.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:25.737+0000 7f6869600640 1 --2- 172.21.15.5:0/3604812929 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6864104010 0x7f686419a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:25.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:25.737+0000 7f6869600640 1 -- 172.21.15.5:0/3604812929 >> 172.21.15.5:0/3604812929 conn(0x7f68640ffa50 msgr2=0x7f6864100e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:25.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:25.737+0000 7f6869600640 1 -- 172.21.15.5:0/3604812929 shutdown_connections 2024-09-10T10:04:25.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:25.738+0000 7f6869600640 1 -- 172.21.15.5:0/3604812929 wait complete. 2024-09-10T10:04:25.843 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:04:25 smithi005 ceph-mon[30350]: pgmap v1175: 129 pgs: 129 active+clean; 769 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 2.1 MiB/s wr, 304 op/s 2024-09-10T10:04:25.843 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:04:25 smithi005 ceph-mon[30350]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T10:04:26.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:04:25 smithi159 ceph-mon[35513]: pgmap v1175: 129 pgs: 129 active+clean; 769 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 2.1 MiB/s wr, 304 op/s 2024-09-10T10:04:26.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:04:25 smithi159 ceph-mon[35513]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T10:04:26.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:04:25 smithi102 ceph-mon[35442]: pgmap v1175: 129 pgs: 129 active+clean; 769 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 2.1 MiB/s wr, 304 op/s 2024-09-10T10:04:26.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:04:25 smithi102 ceph-mon[35442]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T10:04:26.240 DEBUG:teuthology.parallel:result is None 2024-09-10T10:04:26.241 DEBUG:teuthology.orchestra.run.smithi005:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-09-10T10:04:26.306 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-09-10T10:04:26.307 DEBUG:teuthology.run_tasks:Unwinding manager quiescer 2024-09-10T10:04:26.317 INFO:tasks.quiescer:joining Quiescers 2024-09-10T10:04:26.317 WARNING:tasks.quiescer:The quiescer is requested to stop, running cancel all 2024-09-10T10:04:26.317 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T10:04:26.684 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-10T10:04:26.684 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T10:04:26.804 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:04:27.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:27.412+0000 7f25bf800640 1 -- 172.21.15.5:0/493411927 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f25b8101df0 msgr2=0x7f25b81021d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:27.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:27.412+0000 7f25bf800640 1 --2- 172.21.15.5:0/493411927 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f25b8101df0 0x7f25b81021d0 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f25ac009900 tx=0x7f25ac02a410 comp rx=0 tx=0).stop 2024-09-10T10:04:27.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:27.412+0000 7f25bf800640 1 -- 172.21.15.5:0/493411927 shutdown_connections 2024-09-10T10:04:27.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:27.412+0000 7f25bf800640 1 --2- 172.21.15.5:0/493411927 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f25b8106f20 0x7f25b810eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:27.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:27.412+0000 7f25bf800640 1 --2- 172.21.15.5:0/493411927 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f25b81027a0 0x7f25b81067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:27.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:27.412+0000 7f25bf800640 1 --2- 172.21.15.5:0/493411927 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f25b8101df0 0x7f25b81021d0 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:27.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:27.412+0000 7f25bf800640 1 -- 172.21.15.5:0/493411927 >> 172.21.15.5:0/493411927 conn(0x7f25b80fd8b0 msgr2=0x7f25b80ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:27.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:27.412+0000 7f25bf800640 1 -- 172.21.15.5:0/493411927 shutdown_connections 2024-09-10T10:04:27.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:27.412+0000 7f25bf800640 1 -- 172.21.15.5:0/493411927 wait complete. 2024-09-10T10:04:27.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:27.413+0000 7f25bf800640 1 Processor -- start 2024-09-10T10:04:27.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:27.413+0000 7f25bf800640 1 -- start start 2024-09-10T10:04:27.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:27.414+0000 7f25bf800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f25b8101df0 0x7f25b819a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:27.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:27.414+0000 7f25bf800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f25b81027a0 0x7f25b819ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:27.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:27.414+0000 7f25bf800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f25b8106f20 0x7f25b819eea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:27.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:27.414+0000 7f25bf800640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f25b8111f80 con 0x7f25b81027a0 2024-09-10T10:04:27.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:27.414+0000 7f25bf800640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f25b8111e00 con 0x7f25b8101df0 2024-09-10T10:04:27.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:27.414+0000 7f25bf800640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f25b8112100 con 0x7f25b8106f20 2024-09-10T10:04:27.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:27.414+0000 7f25bce00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f25b8101df0 0x7f25b819a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:27.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:27.414+0000 7f25bce00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f25b8101df0 0x7f25b819a5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:47496/0 (socket says 172.21.15.5:47496) 2024-09-10T10:04:27.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:27.414+0000 7f25b7e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f25b81027a0 0x7f25b819ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:27.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:27.414+0000 7f25bce00640 1 -- 172.21.15.5:0/1408111900 learned_addr learned my addr 172.21.15.5:0/1408111900 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T10:04:27.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:27.414+0000 7f25bd800640 1 --2- 172.21.15.5:0/1408111900 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f25b8106f20 0x7f25b819eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:27.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:27.415+0000 7f25bce00640 1 -- 172.21.15.5:0/1408111900 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f25b8106f20 msgr2=0x7f25b819eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:27.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:27.415+0000 7f25bce00640 1 --2- 172.21.15.5:0/1408111900 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f25b8106f20 0x7f25b819eea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:27.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:27.415+0000 7f25bce00640 1 -- 172.21.15.5:0/1408111900 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f25b81027a0 msgr2=0x7f25b819ab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:27.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:27.415+0000 7f25bce00640 1 --2- 172.21.15.5:0/1408111900 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f25b81027a0 0x7f25b819ab10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:27.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:27.415+0000 7f25bce00640 1 -- 172.21.15.5:0/1408111900 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25b819f580 con 0x7f25b8101df0 2024-09-10T10:04:27.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:27.415+0000 7f25b7e00640 1 --2- 172.21.15.5:0/1408111900 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f25b81027a0 0x7f25b819ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:04:27.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:27.415+0000 7f25bce00640 1 --2- 172.21.15.5:0/1408111900 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f25b8101df0 0x7f25b819a5d0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f25ac006010 tx=0x7f25ac004040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:27.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:27.415+0000 7f25b5600640 1 -- 172.21.15.5:0/1408111900 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25ac03f030 con 0x7f25b8101df0 2024-09-10T10:04:27.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:27.415+0000 7f25b5600640 1 -- 172.21.15.5:0/1408111900 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f25ac004440 con 0x7f25b8101df0 2024-09-10T10:04:27.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:27.415+0000 7f25bf800640 1 -- 172.21.15.5:0/1408111900 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25b819f810 con 0x7f25b8101df0 2024-09-10T10:04:27.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:27.416+0000 7f25b5600640 1 -- 172.21.15.5:0/1408111900 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25ac035070 con 0x7f25b8101df0 2024-09-10T10:04:27.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:27.416+0000 7f25bf800640 1 -- 172.21.15.5:0/1408111900 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25b81a7150 con 0x7f25b8101df0 2024-09-10T10:04:27.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:27.417+0000 7f25bf800640 1 -- 172.21.15.5:0/1408111900 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f25b81032d0 con 0x7f25b8101df0 2024-09-10T10:04:27.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:27.419+0000 7f25b5600640 1 -- 172.21.15.5:0/1408111900 <== mon.2 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f25ac042030 con 0x7f25b8101df0 2024-09-10T10:04:27.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:27.420+0000 7f25b5600640 1 --2- 172.21.15.5:0/1408111900 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f25a00a2b80 0x7f25a00a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:27.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:27.420+0000 7f25b5600640 1 -- 172.21.15.5:0/1408111900 <== mon.2 v2:172.21.15.102:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f25ac0efab0 con 0x7f25b8101df0 2024-09-10T10:04:27.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:27.420+0000 7f25b7e00640 1 --2- 172.21.15.5:0/1408111900 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f25a00a2b80 0x7f25a00a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:27.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:27.421+0000 7f25b7e00640 1 --2- 172.21.15.5:0/1408111900 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f25a00a2b80 0x7f25a00a5040 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f25b80fff80 tx=0x7f25a4005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:27.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:27.422+0000 7f25b5600640 1 -- 172.21.15.5:0/1408111900 <== mon.2 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f25ac0bbf70 con 0x7f25b8101df0 2024-09-10T10:04:27.466 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:04:27.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:27.662+0000 7f25bf800640 1 -- 172.21.15.5:0/1408111900 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f25b81021d0 con 0x7f25b8101df0 2024-09-10T10:04:27.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:27.663+0000 7f25b5600640 1 -- 172.21.15.5:0/1408111900 <== mon.2 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) ==== 94+0+11973 (secure 0 0 0) 0x7f25b81021d0 con 0x7f25b8101df0 2024-09-10T10:04:27.665 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-10T10:04:27.666 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":44,"btime":"2024-09-10T10:04:26:724173+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14454,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/2970594311","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2970594311},{"type":"v1","addr":"172.21.15.5:6835","nonce":2970594311}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14514,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6839/3616502174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":3616502174},{"type":"v1","addr":"172.21.15.5:6839","nonce":3616502174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":44,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T09:35:29.626287+0000","modified":"2024-09-10T10:04:26.455678+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24542,"mds_1":24535,"mds_2":24511,"mds_3":24506,"mds_4":24443},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14520":{"gid":14520,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/2073599842","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":2073599842},{"type":"v1","addr":"172.21.15.5:6841","nonce":2073599842}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"k","rank":4,"incarnation":26,"state":"up:active","state_seq":24,"addr":"172.21.15.102:6833/2457007427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":2457007427},{"type":"v1","addr":"172.21.15.102:6833","nonce":2457007427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/356445159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":356445159},{"type":"v1","addr":"172.21.15.159:6833","nonce":356445159}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6837/2894314507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":2894314507},{"type":"v1","addr":"172.21.15.5:6837","nonce":2894314507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.102:6835/1647549111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":1647549111},{"type":"v1","addr":"172.21.15.102:6835","nonce":1647549111}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6835/952657242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":952657242},{"type":"v1","addr":"172.21.15.159:6835","nonce":952657242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":19,"addr":"172.21.15.102:6837/2689776764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":2689776764},{"type":"v1","addr":"172.21.15.102:6837","nonce":2689776764}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.159:6837/620627639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":620627639},{"type":"v1","addr":"172.21.15.159:6837","nonce":620627639}]},"join_fscid":-1,"export_targets":[0,1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24535":{"gid":24535,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":18,"addr":"172.21.15.159:6839/2073739599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":2073739599},{"type":"v1","addr":"172.21.15.159:6839","nonce":2073739599}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24542":{"gid":24542,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.102:6839/335409749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6838","nonce":335409749},{"type":"v1","addr":"172.21.15.102:6839","nonce":335409749}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24542,"qdb_cluster":[24542,24535,24511,24506,24443]},"id":1}]} 2024-09-10T10:04:27.666 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 44 2024-09-10T10:04:27.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:27.665+0000 7f25bf800640 1 -- 172.21.15.5:0/1408111900 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f25a00a2b80 msgr2=0x7f25a00a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:27.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:27.665+0000 7f25bf800640 1 --2- 172.21.15.5:0/1408111900 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f25a00a2b80 0x7f25a00a5040 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f25b80fff80 tx=0x7f25a4005f90 comp rx=0 tx=0).stop 2024-09-10T10:04:27.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:27.665+0000 7f25bf800640 1 -- 172.21.15.5:0/1408111900 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f25b8101df0 msgr2=0x7f25b819a5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:27.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:27.665+0000 7f25bf800640 1 --2- 172.21.15.5:0/1408111900 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f25b8101df0 0x7f25b819a5d0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f25ac006010 tx=0x7f25ac004040 comp rx=0 tx=0).stop 2024-09-10T10:04:27.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:27.665+0000 7f25bf800640 1 -- 172.21.15.5:0/1408111900 shutdown_connections 2024-09-10T10:04:27.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:27.665+0000 7f25bf800640 1 --2- 172.21.15.5:0/1408111900 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f25a00a2b80 0x7f25a00a5040 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:27.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:27.665+0000 7f25bf800640 1 --2- 172.21.15.5:0/1408111900 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f25b8106f20 0x7f25b819eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:27.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:27.665+0000 7f25bf800640 1 --2- 172.21.15.5:0/1408111900 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f25b81027a0 0x7f25b819ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:27.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:27.665+0000 7f25bf800640 1 --2- 172.21.15.5:0/1408111900 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f25b8101df0 0x7f25b819a5d0 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:27.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:27.666+0000 7f25bf800640 1 -- 172.21.15.5:0/1408111900 >> 172.21.15.5:0/1408111900 conn(0x7f25b80fd8b0 msgr2=0x7f25b80fe790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:27.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:27.666+0000 7f25bf800640 1 -- 172.21.15.5:0/1408111900 shutdown_connections 2024-09-10T10:04:27.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:27.666+0000 7f25bf800640 1 -- 172.21.15.5:0/1408111900 wait complete. 2024-09-10T10:04:27.910 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:04:27 smithi005 ceph-mon[30350]: pgmap v1176: 129 pgs: 129 active+clean; 740 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 2.5 MiB/s wr, 381 op/s 2024-09-10T10:04:27.910 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:04:27 smithi005 ceph-mon[30350]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T10:04:27.910 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:04:27 smithi005 ceph-mon[30350]: from='client.? 172.21.15.5:0/1408111900' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:04:28.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:04:27 smithi159 ceph-mon[35513]: pgmap v1176: 129 pgs: 129 active+clean; 740 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 2.5 MiB/s wr, 381 op/s 2024-09-10T10:04:28.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:04:27 smithi159 ceph-mon[35513]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T10:04:28.101 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:04:27 smithi159 ceph-mon[35513]: from='client.? 172.21.15.5:0/1408111900' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:04:28.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:04:27 smithi102 ceph-mon[35442]: pgmap v1176: 129 pgs: 129 active+clean; 740 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 2.5 MiB/s wr, 381 op/s 2024-09-10T10:04:28.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:04:27 smithi102 ceph-mon[35442]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T10:04:28.135 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:04:27 smithi102 ceph-mon[35442]: from='client.? 172.21.15.5:0/1408111900' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:04:29.059 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:04:28 smithi005 ceph-mon[30350]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T10:04:29.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:04:28 smithi159 ceph-mon[35513]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T10:04:29.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:04:28 smithi102 ceph-mon[35442]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T10:04:29.428 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24542 quiesce db --cancel --all' 2024-09-10T10:04:29.428 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24542 quiesce db --cancel --all 2024-09-10T10:04:29.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:29.781+0000 7f6df6a00640 1 -- 172.21.15.5:0/1583587805 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6df002fd30 con 0x7f6df810bfc0 2024-09-10T10:04:29.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:29.781+0000 7f6dfd800640 1 -- 172.21.15.5:0/1583587805 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6df810bfc0 msgr2=0x7f6df810e460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:29.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:29.781+0000 7f6dfd800640 1 --2- 172.21.15.5:0/1583587805 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6df810bfc0 0x7f6df810e460 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f6df0009ca0 tx=0x7f6df002a940 comp rx=0 tx=0).stop 2024-09-10T10:04:29.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:29.781+0000 7f6dfd800640 1 -- 172.21.15.5:0/1583587805 shutdown_connections 2024-09-10T10:04:29.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:29.781+0000 7f6dfd800640 1 --2- 172.21.15.5:0/1583587805 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6df810bfc0 0x7f6df810e460 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:29.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:29.781+0000 7f6dfd800640 1 --2- 172.21.15.5:0/1583587805 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6df81034b0 0x7f6df810b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:29.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:29.781+0000 7f6dfd800640 1 --2- 172.21.15.5:0/1583587805 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6df8102b90 0x7f6df8102f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:29.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:29.781+0000 7f6dfd800640 1 -- 172.21.15.5:0/1583587805 >> 172.21.15.5:0/1583587805 conn(0x7f6df80fc610 msgr2=0x7f6df80fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:29.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:29.781+0000 7f6dfd800640 1 -- 172.21.15.5:0/1583587805 shutdown_connections 2024-09-10T10:04:29.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:29.781+0000 7f6dfd800640 1 -- 172.21.15.5:0/1583587805 wait complete. 2024-09-10T10:04:29.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:29.781+0000 7f6dfd800640 1 Processor -- start 2024-09-10T10:04:29.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:29.781+0000 7f6dfd800640 1 -- start start 2024-09-10T10:04:29.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:29.781+0000 7f6dfd800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6df8102b90 0x7f6df819a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:29.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:29.781+0000 7f6dfd800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6df81034b0 0x7f6df819a980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:29.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:29.781+0000 7f6dfd800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6df810bfc0 0x7f6df819ed10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:29.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:29.787+0000 7f6dfd800640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f6df8111dd0 con 0x7f6df81034b0 2024-09-10T10:04:29.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:29.787+0000 7f6dfd800640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f6df8111c50 con 0x7f6df8102b90 2024-09-10T10:04:29.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:29.787+0000 7f6dfd800640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f6df8111f50 con 0x7f6df810bfc0 2024-09-10T10:04:29.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:29.787+0000 7f6df7e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6df8102b90 0x7f6df819a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:29.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:29.787+0000 7f6dfce00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6df810bfc0 0x7f6df819ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:29.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:29.787+0000 7f6dfce00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6df810bfc0 0x7f6df819ed10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:40064/0 (socket says 172.21.15.5:40064) 2024-09-10T10:04:29.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:29.787+0000 7f6dfce00640 1 -- 172.21.15.5:0/2860261820 learned_addr learned my addr 172.21.15.5:0/2860261820 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T10:04:29.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:29.787+0000 7f6df7400640 1 --2- 172.21.15.5:0/2860261820 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6df81034b0 0x7f6df819a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:29.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:29.787+0000 7f6dfce00640 1 -- 172.21.15.5:0/2860261820 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6df8102b90 msgr2=0x7f6df819a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:29.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:29.787+0000 7f6dfce00640 1 --2- 172.21.15.5:0/2860261820 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6df8102b90 0x7f6df819a440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:29.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:29.787+0000 7f6dfce00640 1 -- 172.21.15.5:0/2860261820 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6df81034b0 msgr2=0x7f6df819a980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:29.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:29.787+0000 7f6dfce00640 1 --2- 172.21.15.5:0/2860261820 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6df81034b0 0x7f6df819a980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:29.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:29.787+0000 7f6dfce00640 1 -- 172.21.15.5:0/2860261820 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6df819f3f0 con 0x7f6df810bfc0 2024-09-10T10:04:29.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:29.787+0000 7f6df7400640 1 --2- 172.21.15.5:0/2860261820 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6df81034b0 0x7f6df819a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:04:29.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:29.792+0000 7f6dfce00640 1 --2- 172.21.15.5:0/2860261820 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6df810bfc0 0x7f6df819ed10 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f6df0006010 tx=0x7f6df002f5a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:29.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:29.792+0000 7f6df4c00640 1 -- 172.21.15.5:0/2860261820 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6df0034040 con 0x7f6df810bfc0 2024-09-10T10:04:29.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:29.792+0000 7f6df4c00640 1 -- 172.21.15.5:0/2860261820 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6df0003e50 con 0x7f6df810bfc0 2024-09-10T10:04:29.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:29.792+0000 7f6df4c00640 1 -- 172.21.15.5:0/2860261820 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6df002fd30 con 0x7f6df810bfc0 2024-09-10T10:04:29.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:29.797+0000 7f6dfd800640 1 -- 172.21.15.5:0/2860261820 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6df819f680 con 0x7f6df810bfc0 2024-09-10T10:04:29.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:29.797+0000 7f6dfd800640 1 -- 172.21.15.5:0/2860261820 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6df819fb10 con 0x7f6df810bfc0 2024-09-10T10:04:29.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:29.797+0000 7f6dfd800640 1 -- 172.21.15.5:0/2860261820 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6df8102f70 con 0x7f6df810bfc0 2024-09-10T10:04:29.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:29.802+0000 7f6df4c00640 1 -- 172.21.15.5:0/2860261820 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f6df0037020 con 0x7f6df810bfc0 2024-09-10T10:04:29.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:29.802+0000 7f6df4c00640 1 --2- 172.21.15.5:0/2860261820 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f6ddc0a2cd0 0x7f6ddc0a5190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:29.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:29.805+0000 7f6df4c00640 1 -- 172.21.15.5:0/2860261820 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f6df00f0310 con 0x7f6df810bfc0 2024-09-10T10:04:29.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:29.805+0000 7f6df7e00640 1 --2- 172.21.15.5:0/2860261820 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f6ddc0a2cd0 0x7f6ddc0a5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:29.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:29.805+0000 7f6df4c00640 1 -- 172.21.15.5:0/2860261820 <== mon.1 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f6df00bcf00 con 0x7f6df810bfc0 2024-09-10T10:04:29.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:29.805+0000 7f6df7e00640 1 --2- 172.21.15.5:0/2860261820 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f6ddc0a2cd0 0x7f6ddc0a5190 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f6de8005fd0 tx=0x7f6de8005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:29.959 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:04:29 smithi005 ceph-mon[30350]: pgmap v1177: 129 pgs: 129 active+clean; 687 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 2.8 MiB/s wr, 406 op/s 2024-09-10T10:04:30.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:04:29 smithi159 ceph-mon[35513]: pgmap v1177: 129 pgs: 129 active+clean; 687 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 2.8 MiB/s wr, 406 op/s 2024-09-10T10:04:30.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:04:29 smithi102 ceph-mon[35442]: pgmap v1177: 129 pgs: 129 active+clean; 687 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 2.8 MiB/s wr, 406 op/s 2024-09-10T10:04:30.140 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:04:30.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:30.204+0000 7f6dfd800640 1 -- 172.21.15.5:0/2860261820 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6df81a7170 con 0x7f6df810bfc0 2024-09-10T10:04:30.209 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-10T10:04:30.209 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":46,"btime":"2024-09-10T10:04:29:760158+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14454,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/2970594311","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2970594311},{"type":"v1","addr":"172.21.15.5:6835","nonce":2970594311}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14514,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6839/3616502174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":3616502174},{"type":"v1","addr":"172.21.15.5:6839","nonce":3616502174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":46,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T09:35:29.626287+0000","modified":"2024-09-10T10:04:29.593937+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24542,"mds_1":24535,"mds_2":24511,"mds_3":24506,"mds_4":24443},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14520":{"gid":14520,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/2073599842","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":2073599842},{"type":"v1","addr":"172.21.15.5:6841","nonce":2073599842}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"k","rank":4,"incarnation":26,"state":"up:active","state_seq":24,"addr":"172.21.15.102:6833/2457007427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":2457007427},{"type":"v1","addr":"172.21.15.102:6833","nonce":2457007427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/356445159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":356445159},{"type":"v1","addr":"172.21.15.159:6833","nonce":356445159}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6837/2894314507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":2894314507},{"type":"v1","addr":"172.21.15.5:6837","nonce":2894314507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.102:6835/1647549111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":1647549111},{"type":"v1","addr":"172.21.15.102:6835","nonce":1647549111}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6835/952657242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":952657242},{"type":"v1","addr":"172.21.15.159:6835","nonce":952657242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":19,"addr":"172.21.15.102:6837/2689776764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":2689776764},{"type":"v1","addr":"172.21.15.102:6837","nonce":2689776764}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.159:6837/620627639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":620627639},{"type":"v1","addr":"172.21.15.159:6837","nonce":620627639}]},"join_fscid":-1,"export_targets":[0,1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24535":{"gid":24535,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":18,"addr":"172.21.15.159:6839/2073739599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":2073739599},{"type":"v1","addr":"172.21.15.159:6839","nonce":2073739599}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24542":{"gid":24542,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.102:6839/335409749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6838","nonce":335409749},{"type":"v1","addr":"172.21.15.102:6839","nonce":335409749}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24542,"qdb_cluster":[24542,24535,24511,24506,24443]},"id":1}]} 2024-09-10T10:04:30.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:30.204+0000 7f6df4c00640 1 -- 172.21.15.5:0/2860261820 <== mon.1 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 46 v46) ==== 94+0+11977 (secure 0 0 0) 0x7f6df00bc800 con 0x7f6df810bfc0 2024-09-10T10:04:30.210 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 46 2024-09-10T10:04:30.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:30.208+0000 7f6dfd800640 1 -- 172.21.15.5:0/2860261820 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f6ddc0a2cd0 msgr2=0x7f6ddc0a5190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:30.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:30.208+0000 7f6dfd800640 1 --2- 172.21.15.5:0/2860261820 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f6ddc0a2cd0 0x7f6ddc0a5190 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f6de8005fd0 tx=0x7f6de8005850 comp rx=0 tx=0).stop 2024-09-10T10:04:30.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:30.208+0000 7f6dfd800640 1 -- 172.21.15.5:0/2860261820 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6df810bfc0 msgr2=0x7f6df819ed10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:30.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:30.208+0000 7f6dfd800640 1 --2- 172.21.15.5:0/2860261820 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6df810bfc0 0x7f6df819ed10 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f6df0006010 tx=0x7f6df002f5a0 comp rx=0 tx=0).stop 2024-09-10T10:04:30.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:30.208+0000 7f6dfd800640 1 -- 172.21.15.5:0/2860261820 shutdown_connections 2024-09-10T10:04:30.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:30.208+0000 7f6dfd800640 1 --2- 172.21.15.5:0/2860261820 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f6ddc0a2cd0 0x7f6ddc0a5190 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:30.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:30.208+0000 7f6dfd800640 1 --2- 172.21.15.5:0/2860261820 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f6df810bfc0 0x7f6df819ed10 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:30.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:30.208+0000 7f6dfd800640 1 --2- 172.21.15.5:0/2860261820 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6df81034b0 0x7f6df819a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:30.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:30.208+0000 7f6dfd800640 1 --2- 172.21.15.5:0/2860261820 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f6df8102b90 0x7f6df819a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:30.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:30.208+0000 7f6dfd800640 1 -- 172.21.15.5:0/2860261820 >> 172.21.15.5:0/2860261820 conn(0x7f6df80fc610 msgr2=0x7f6df80fd210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:30.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:30.208+0000 7f6dfd800640 1 -- 172.21.15.5:0/2860261820 shutdown_connections 2024-09-10T10:04:30.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:30.208+0000 7f6dfd800640 1 -- 172.21.15.5:0/2860261820 wait complete. 2024-09-10T10:04:30.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:04:30 smithi005 ceph-mon[30350]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T10:04:30.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:04:30 smithi005 ceph-mon[30350]: from='client.? 172.21.15.5:0/2860261820' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:04:31.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:04:30 smithi159 ceph-mon[35513]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T10:04:31.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:04:30 smithi159 ceph-mon[35513]: from='client.? 172.21.15.5:0/2860261820' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:04:31.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:04:30 smithi102 ceph-mon[35442]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T10:04:31.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:04:30 smithi102 ceph-mon[35442]: from='client.? 172.21.15.5:0/2860261820' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:04:32.057 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:04:31 smithi005 ceph-mon[30350]: pgmap v1178: 129 pgs: 129 active+clean; 650 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 3.7 MiB/s wr, 444 op/s 2024-09-10T10:04:32.057 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:04:31 smithi005 ceph-mon[30350]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T10:04:32.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:04:31 smithi159 ceph-mon[35513]: pgmap v1178: 129 pgs: 129 active+clean; 650 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 3.7 MiB/s wr, 444 op/s 2024-09-10T10:04:32.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:04:31 smithi159 ceph-mon[35513]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T10:04:32.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:04:31 smithi102 ceph-mon[35442]: pgmap v1178: 129 pgs: 129 active+clean; 650 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 3.7 MiB/s wr, 444 op/s 2024-09-10T10:04:32.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:04:31 smithi102 ceph-mon[35442]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T10:04:34.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:04:33 smithi005 ceph-mon[30350]: pgmap v1179: 129 pgs: 129 active+clean; 653 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.1 MiB/s wr, 272 op/s 2024-09-10T10:04:34.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:04:33 smithi159 ceph-mon[35513]: pgmap v1179: 129 pgs: 129 active+clean; 653 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.1 MiB/s wr, 272 op/s 2024-09-10T10:04:34.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:04:33 smithi102 ceph-mon[35442]: pgmap v1179: 129 pgs: 129 active+clean; 653 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.1 MiB/s wr, 272 op/s 2024-09-10T10:04:35.194 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-10T10:04:35.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:34.575+0000 7fa7e2e00640 1 -- 172.21.15.5:0/68319426 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa7dc103e30 msgr2=0x7fa7dc104210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:35.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:34.575+0000 7fa7e2e00640 1 --2- 172.21.15.5:0/68319426 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa7dc103e30 0x7fa7dc104210 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7fa7d0013150 tx=0x7fa7d0032760 comp rx=0 tx=0).stop 2024-09-10T10:04:35.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:34.575+0000 7fa7e2e00640 1 -- 172.21.15.5:0/68319426 shutdown_connections 2024-09-10T10:04:35.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:34.575+0000 7fa7e2e00640 1 --2- 172.21.15.5:0/68319426 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa7dc108f60 0x7fa7dc110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:35.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:34.575+0000 7fa7e2e00640 1 --2- 172.21.15.5:0/68319426 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa7dc1047e0 0x7fa7dc108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:35.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:34.575+0000 7fa7e2e00640 1 --2- 172.21.15.5:0/68319426 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa7dc103e30 0x7fa7dc104210 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:35.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:34.575+0000 7fa7e2e00640 1 -- 172.21.15.5:0/68319426 >> 172.21.15.5:0/68319426 conn(0x7fa7dc0ff8d0 msgr2=0x7fa7dc101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:35.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:34.576+0000 7fa7e2e00640 1 -- 172.21.15.5:0/68319426 shutdown_connections 2024-09-10T10:04:35.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:34.576+0000 7fa7e2e00640 1 -- 172.21.15.5:0/68319426 wait complete. 2024-09-10T10:04:35.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:34.576+0000 7fa7e2e00640 1 Processor -- start 2024-09-10T10:04:35.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:34.576+0000 7fa7e2e00640 1 -- start start 2024-09-10T10:04:35.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:34.576+0000 7fa7e2e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa7dc103e30 0x7fa7dc196510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:35.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:34.576+0000 7fa7e2e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa7dc1047e0 0x7fa7dc196a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:35.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:34.576+0000 7fa7e2e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa7dc108f60 0x7fa7dc19a860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:35.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:34.576+0000 7fa7e2e00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa7dc078fb0 con 0x7fa7dc108f60 2024-09-10T10:04:35.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:34.576+0000 7fa7e2e00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fa7dc078e30 con 0x7fa7dc1047e0 2024-09-10T10:04:35.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:34.576+0000 7fa7e2e00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fa7dc079130 con 0x7fa7dc103e30 2024-09-10T10:04:35.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:34.576+0000 7fa7e1000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa7dc1047e0 0x7fa7dc196a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:35.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:34.576+0000 7fa7e1000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa7dc1047e0 0x7fa7dc196a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:49112/0 (socket says 172.21.15.5:49112) 2024-09-10T10:04:35.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:34.576+0000 7fa7e1000640 1 -- 172.21.15.5:0/1204373047 learned_addr learned my addr 172.21.15.5:0/1204373047 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T10:04:35.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:34.576+0000 7fa7e2400640 1 --2- 172.21.15.5:0/1204373047 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa7dc108f60 0x7fa7dc19a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:35.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:34.576+0000 7fa7e1a00640 1 --2- 172.21.15.5:0/1204373047 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa7dc103e30 0x7fa7dc196510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:35.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:34.576+0000 7fa7e1000640 1 -- 172.21.15.5:0/1204373047 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa7dc103e30 msgr2=0x7fa7dc196510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:35.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:34.576+0000 7fa7e1000640 1 --2- 172.21.15.5:0/1204373047 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa7dc103e30 0x7fa7dc196510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:35.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:34.576+0000 7fa7e1000640 1 -- 172.21.15.5:0/1204373047 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa7dc108f60 msgr2=0x7fa7dc19a860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:35.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:34.576+0000 7fa7e1000640 1 --2- 172.21.15.5:0/1204373047 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa7dc108f60 0x7fa7dc19a860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:35.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:34.576+0000 7fa7e1000640 1 -- 172.21.15.5:0/1204373047 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7dc19af40 con 0x7fa7dc1047e0 2024-09-10T10:04:35.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:34.576+0000 7fa7e1a00640 1 --2- 172.21.15.5:0/1204373047 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa7dc103e30 0x7fa7dc196510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:04:35.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:34.581+0000 7fa7e1000640 1 --2- 172.21.15.5:0/1204373047 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa7dc1047e0 0x7fa7dc196a50 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7fa7c800c920 tx=0x7fa7c800cde0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:35.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:34.581+0000 7fa7da000640 1 -- 172.21.15.5:0/1204373047 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7c8007c70 con 0x7fa7dc1047e0 2024-09-10T10:04:35.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:34.581+0000 7fa7da000640 1 -- 172.21.15.5:0/1204373047 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa7c8007e10 con 0x7fa7dc1047e0 2024-09-10T10:04:35.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:34.581+0000 7fa7da000640 1 -- 172.21.15.5:0/1204373047 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7c8018890 con 0x7fa7dc1047e0 2024-09-10T10:04:35.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:34.581+0000 7fa7e2e00640 1 -- 172.21.15.5:0/1204373047 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7dc19b230 con 0x7fa7dc1047e0 2024-09-10T10:04:35.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:34.581+0000 7fa7e2e00640 1 -- 172.21.15.5:0/1204373047 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7dc1a2b10 con 0x7fa7dc1047e0 2024-09-10T10:04:35.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:34.586+0000 7fa7e7015740 1 Processor -- start 2024-09-10T10:04:35.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:34.586+0000 7fa7e7015740 1 -- start start 2024-09-10T10:04:35.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:34.586+0000 7fa7da000640 1 -- 172.21.15.5:0/1204373047 <== mon.2 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7fa7c8018a70 con 0x7fa7dc1047e0 2024-09-10T10:04:35.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:34.586+0000 7fa7e7015740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f34148f880 0x55f34148fc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:35.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:34.586+0000 7fa7e7015740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f3414901a0 0x55f34149a520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:35.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:34.586+0000 7fa7e7015740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f34149aa60 0x55f34149ce50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:35.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:34.586+0000 7fa7e7015740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55f3413f6340 con 0x55f34148f880 2024-09-10T10:04:35.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:34.586+0000 7fa7e7015740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55f34139dbf0 con 0x55f34149aa60 2024-09-10T10:04:35.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:34.586+0000 7fa7e7015740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55f3413f2300 con 0x55f3414901a0 2024-09-10T10:04:35.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:34.586+0000 7fa7e1000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f34149aa60 0x55f34149ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:35.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:34.586+0000 7fa7e1000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f34149aa60 0x55f34149ce50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:49118/0 (socket says 172.21.15.5:49118) 2024-09-10T10:04:35.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:34.586+0000 7fa7e1000640 1 -- 172.21.15.5:0/3912490908 learned_addr learned my addr 172.21.15.5:0/3912490908 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T10:04:35.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:34.586+0000 7fa7e2400640 1 --2- 172.21.15.5:0/3912490908 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f34148f880 0x55f34148fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:35.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:34.586+0000 7fa7e1000640 1 -- 172.21.15.5:0/3912490908 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f3414901a0 msgr2=0x55f34149a520 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T10:04:35.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:34.586+0000 7fa7e1000640 1 --2- 172.21.15.5:0/3912490908 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f3414901a0 0x55f34149a520 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:35.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:34.586+0000 7fa7e1000640 1 -- 172.21.15.5:0/3912490908 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f34148f880 msgr2=0x55f34148fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:35.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:34.586+0000 7fa7e1000640 1 --2- 172.21.15.5:0/3912490908 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f34148f880 0x55f34148fc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:35.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:34.586+0000 7fa7e1000640 1 -- 172.21.15.5:0/3912490908 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f341167620 con 0x55f34149aa60 2024-09-10T10:04:35.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:34.588+0000 7fa7da000640 1 --2- 172.21.15.5:0/1204373047 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fa7c40a2cc0 0x7fa7c40a5180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:35.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:34.588+0000 7fa7e1000640 1 --2- 172.21.15.5:0/3912490908 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f34149aa60 0x55f34149ce50 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7fa7c80d75d0 tx=0x7fa7c80d76c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c8799e77af42532b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:35.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:34.588+0000 7fa7da000640 1 -- 172.21.15.5:0/1204373047 <== mon.2 v2:172.21.15.102:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fa7c80cf4d0 con 0x7fa7dc1047e0 2024-09-10T10:04:35.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:34.588+0000 7fa7e2400640 1 --2- 172.21.15.5:0/1204373047 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fa7c40a2cc0 0x7fa7c40a5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:35.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:34.588+0000 7fa7cea00640 1 -- 172.21.15.5:0/3912490908 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7c8097b80 con 0x55f34149aa60 2024-09-10T10:04:35.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:34.588+0000 7fa7cea00640 1 -- 172.21.15.5:0/3912490908 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa7c809bfe0 con 0x55f34149aa60 2024-09-10T10:04:35.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:34.588+0000 7fa7cea00640 1 -- 172.21.15.5:0/3912490908 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7c809b240 con 0x55f34149aa60 2024-09-10T10:04:35.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:34.588+0000 7fa7e7015740 1 -- 172.21.15.5:0/3912490908 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f34149aa60 msgr2=0x55f34149ce50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:35.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:34.588+0000 7fa7e7015740 1 --2- 172.21.15.5:0/3912490908 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f34149aa60 0x55f34149ce50 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7fa7c80d75d0 tx=0x7fa7c80d76c0 comp rx=0 tx=0).stop 2024-09-10T10:04:35.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:34.588+0000 7fa7e7015740 1 -- 172.21.15.5:0/3912490908 shutdown_connections 2024-09-10T10:04:35.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:34.588+0000 7fa7e7015740 1 --2- 172.21.15.5:0/3912490908 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f34149aa60 0x55f34149ce50 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:35.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:34.588+0000 7fa7e7015740 1 --2- 172.21.15.5:0/3912490908 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f3414901a0 0x55f34149a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:35.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:34.588+0000 7fa7e7015740 1 --2- 172.21.15.5:0/3912490908 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f34148f880 0x55f34148fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:35.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:34.588+0000 7fa7e7015740 1 -- 172.21.15.5:0/3912490908 >> 172.21.15.5:0/3912490908 conn(0x55f34148be10 msgr2=0x55f34148c1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:35.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:34.588+0000 7fa7e7015740 1 -- 172.21.15.5:0/3912490908 shutdown_connections 2024-09-10T10:04:35.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:34.588+0000 7fa7e7015740 1 -- 172.21.15.5:0/3912490908 wait complete. 2024-09-10T10:04:35.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:34.588+0000 7fa7e7015740 1 -- start start 2024-09-10T10:04:35.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:34.588+0000 7fa7e7015740 1 Processor -- start 2024-09-10T10:04:35.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:34.588+0000 7fa7e2400640 1 --2- 172.21.15.5:0/1204373047 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fa7c40a2cc0 0x7fa7c40a5180 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fa7d4005fd0 tx=0x7fa7d4009600 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:35.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:34.588+0000 7fa7e7015740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f34148f880 0x55f341497960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:35.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:34.588+0000 7fa7e7015740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f3414901a0 0x55f341497ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:35.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:34.588+0000 7fa7e7015740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f34149aa60 0x55f3414983e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:35.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:34.588+0000 7fa7e7015740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55f34149d630 con 0x55f34149aa60 2024-09-10T10:04:35.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:34.588+0000 7fa7e7015740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55f3413b1060 con 0x55f34148f880 2024-09-10T10:04:35.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:34.588+0000 7fa7e7015740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55f34149d7b0 con 0x55f3414901a0 2024-09-10T10:04:35.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:34.588+0000 7fa7e1a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f34148f880 0x55f341497960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:35.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:34.588+0000 7fa7e1a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f34148f880 0x55f341497960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:49128/0 (socket says 172.21.15.5:49128) 2024-09-10T10:04:35.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:34.588+0000 7fa7e1a00640 1 -- 172.21.15.5:0/219165352 learned_addr learned my addr 172.21.15.5:0/219165352 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T10:04:35.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:34.588+0000 7fa7e7015740 10 client.0 _ll_register_callbacks cb 0x55f34139faf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:04:35.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:34.588+0000 7fa7e1000640 1 --2- 172.21.15.5:0/219165352 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f3414901a0 0x55f341497ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:35.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:34.588+0000 7fa7e2400640 1 --2- 172.21.15.5:0/219165352 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f34149aa60 0x55f3414983e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:35.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:34.588+0000 7fa7e1a00640 1 -- 172.21.15.5:0/219165352 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f3414901a0 msgr2=0x55f341497ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:35.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:34.588+0000 7fa7e1a00640 1 --2- 172.21.15.5:0/219165352 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f3414901a0 0x55f341497ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:35.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:34.588+0000 7fa7e1a00640 1 -- 172.21.15.5:0/219165352 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f34149aa60 msgr2=0x55f3414983e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:35.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:34.588+0000 7fa7e1a00640 1 --2- 172.21.15.5:0/219165352 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f34149aa60 0x55f3414983e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:35.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:34.588+0000 7fa7e1a00640 1 -- 172.21.15.5:0/219165352 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f3414a6ba0 con 0x55f34148f880 2024-09-10T10:04:35.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:34.588+0000 7fa7e2400640 1 --2- 172.21.15.5:0/219165352 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f34149aa60 0x55f3414983e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:04:35.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:34.593+0000 7fa7e1a00640 1 --2- 172.21.15.5:0/219165352 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f34148f880 0x55f341497960 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7fa7d0013150 tx=0x7fa7d0011440 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d275dccf2615367 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:35.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:34.593+0000 7fa7c3400640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-09-10T10:04:35.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:34.593+0000 7fa7c3400640 1 -- 172.21.15.5:0/219165352 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7d0004b90 con 0x55f34148f880 2024-09-10T10:04:35.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:34.593+0000 7fa7e7015740 1 -- 172.21.15.5:0/219165352 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa7c809bfe0 con 0x55f34148f880 2024-09-10T10:04:35.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:34.593+0000 7fa7c3400640 1 -- 172.21.15.5:0/219165352 <== mon.2 v2:172.21.15.102:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fa7d0004020 con 0x55f34148f880 2024-09-10T10:04:35.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:34.593+0000 7fa7c3400640 1 -- 172.21.15.5:0/219165352 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7fa7d0056050 con 0x55f34148f880 2024-09-10T10:04:35.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:34.593+0000 7fa7e7015740 10 client.30302 fetch_fsmap learned FSMap version 47 2024-09-10T10:04:35.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:34.593+0000 7fa7e7015740 1 -- 172.21.15.5:0/219165352 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x55f3414a9510 con 0x55f34148f880 2024-09-10T10:04:35.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:34.593+0000 7fa7c3400640 1 -- 172.21.15.5:0/219165352 <== mon.2 v2:172.21.15.102:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa7c809bfe0 con 0x55f34148f880 2024-09-10T10:04:35.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:34.593+0000 7fa7c3400640 1 -- 172.21.15.5:0/219165352 <== mon.2 v2:172.21.15.102:3300/0 5 ==== fsmap(e 47) ==== 8398+0+0 (secure 0 0 0) 0x7fa7d00035f0 con 0x55f34148f880 2024-09-10T10:04:35.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:34.593+0000 7fa7e7015740 10 client.30302 fetch_fsmap finished waiting for FSMap version 47 2024-09-10T10:04:35.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:34.593+0000 7fa7e7015740 10 client.30302 resolve_mds: validated gid 24542 aka daemon mds.b 2024-09-10T10:04:35.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:34.593+0000 7fa7e7015740 20 client.30302 populate_metadata read hostname 'smithi005' 2024-09-10T10:04:35.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:34.593+0000 7fa7e7015740 1 --2- 172.21.15.5:0/219165352 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55f3414aaa20 0x55f3414ace70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:35.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:34.593+0000 7fa7e7015740 4 client.30302 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:04:35.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:34.593+0000 7fa7e7015740 1 -- 172.21.15.5:0/219165352 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f3414038b0 con 0x55f3414aaa20 2024-09-10T10:04:35.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:34.593+0000 7fa7e1000640 1 --2- 172.21.15.5:0/219165352 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55f3414aaa20 0x55f3414ace70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:35.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:34.593+0000 7fa7c3400640 10 client.30302 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T10:04:35.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:34.593+0000 7fa7e1000640 1 --2- 172.21.15.5:0/219165352 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55f3414aaa20 0x55f3414ace70 crc :-1 s=READY pgs=349 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f80d60f78c807e9e server_cookie=f4ea8663ce93d244 in_seq=0 out_seq=0 2024-09-10T10:04:35.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:34.593+0000 7fa7c3400640 1 -- 172.21.15.5:0/219165352 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f3414038b0 con 0x55f3414aaa20 2024-09-10T10:04:35.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:34.593+0000 7fa7c3400640 10 client.30302 handle_command_reply: tid=0 2024-09-10T10:04:35.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:34.593+0000 7fa7e7015740 1 client.30302 shutdown 2024-09-10T10:04:35.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:34.593+0000 7fa7e7015740 1 -- 172.21.15.5:0/219165352 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f34148f880 msgr2=0x55f341497960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:35.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:34.593+0000 7fa7e7015740 1 --2- 172.21.15.5:0/219165352 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f34148f880 0x55f341497960 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7fa7d0013150 tx=0x7fa7d0011440 comp rx=0 tx=0).stop 2024-09-10T10:04:35.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:34.593+0000 7fa7e7015740 1 -- 172.21.15.5:0/219165352 shutdown_connections 2024-09-10T10:04:35.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:34.593+0000 7fa7e7015740 1 --2- 172.21.15.5:0/219165352 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55f3414aaa20 0x55f3414ace70 crc :-1 s=READY pgs=349 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:35.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:34.593+0000 7fa7e7015740 1 --2- 172.21.15.5:0/219165352 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f34149aa60 0x55f3414983e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:35.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:34.593+0000 7fa7e7015740 1 --2- 172.21.15.5:0/219165352 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f3414901a0 0x55f341497ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:35.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:34.593+0000 7fa7e7015740 1 --2- 172.21.15.5:0/219165352 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f34148f880 0x55f341497960 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:35.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:34.593+0000 7fa7c3400640 0 client.30302 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T10:04:35.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:34.593+0000 7fa7e7015740 1 -- 172.21.15.5:0/219165352 >> 172.21.15.5:0/219165352 conn(0x55f34148be10 msgr2=0x55f341491b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:35.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:34.593+0000 7fa7e7015740 1 -- 172.21.15.5:0/219165352 shutdown_connections 2024-09-10T10:04:35.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:34.593+0000 7fa7e7015740 1 -- 172.21.15.5:0/219165352 wait complete. 2024-09-10T10:04:35.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:34.593+0000 7fa7e7015740 20 client.30302 trim_cache size 0 max 16384 2024-09-10T10:04:35.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:34.612+0000 7fa7e7015740 1 Processor -- start 2024-09-10T10:04:35.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:34.612+0000 7fa7e7015740 1 -- start start 2024-09-10T10:04:35.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:34.612+0000 7fa7e7015740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f34148f880 0x55f341499830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:35.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:34.612+0000 7fa7e7015740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f3414901a0 0x55f341496250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:35.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:34.612+0000 7fa7e7015740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f34149aa60 0x55f341498230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:35.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:34.612+0000 7fa7e7015740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa7dc1ac9c0 con 0x55f3414901a0 2024-09-10T10:04:35.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:34.612+0000 7fa7e7015740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fa7dc1ae5c0 con 0x55f34149aa60 2024-09-10T10:04:35.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:34.612+0000 7fa7e7015740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fa7dc1ad090 con 0x55f34148f880 2024-09-10T10:04:35.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:34.612+0000 7fa7e1a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f34148f880 0x55f341499830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:35.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:34.612+0000 7fa7e1a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f34148f880 0x55f341499830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:42314/0 (socket says 172.21.15.5:42314) 2024-09-10T10:04:35.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:34.612+0000 7fa7e1a00640 1 -- 172.21.15.5:0/1838019762 learned_addr learned my addr 172.21.15.5:0/1838019762 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T10:04:35.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:34.612+0000 7fa7e2400640 1 --2- 172.21.15.5:0/1838019762 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f34149aa60 0x55f341498230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:35.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:34.612+0000 7fa7e1a00640 1 -- 172.21.15.5:0/1838019762 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f34149aa60 msgr2=0x55f341498230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:35.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:34.612+0000 7fa7e1a00640 1 --2- 172.21.15.5:0/1838019762 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f34149aa60 0x55f341498230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:35.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:34.612+0000 7fa7e1a00640 1 -- 172.21.15.5:0/1838019762 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f3414901a0 msgr2=0x55f341496250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:35.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:34.612+0000 7fa7e1a00640 1 --2- 172.21.15.5:0/1838019762 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f3414901a0 0x55f341496250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:35.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:34.612+0000 7fa7e1a00640 1 -- 172.21.15.5:0/1838019762 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f34148b6a0 con 0x55f34148f880 2024-09-10T10:04:35.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:34.612+0000 7fa7e1a00640 1 --2- 172.21.15.5:0/1838019762 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f34148f880 0x55f341499830 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fa7d0013120 tx=0x7fa7d0003800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bc54985f3a6fb87a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:35.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:34.612+0000 7fa7ce000640 1 -- 172.21.15.5:0/1838019762 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7d005e020 con 0x55f34148f880 2024-09-10T10:04:35.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:34.617+0000 7fa7ce000640 1 -- 172.21.15.5:0/1838019762 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa7d0004b90 con 0x55f34148f880 2024-09-10T10:04:35.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:34.617+0000 7fa7ce000640 1 -- 172.21.15.5:0/1838019762 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7d0003ee0 con 0x55f34148f880 2024-09-10T10:04:35.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:34.617+0000 7fa7e7015740 1 -- 172.21.15.5:0/1838019762 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f34148f880 msgr2=0x55f341499830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:35.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:34.617+0000 7fa7e7015740 1 --2- 172.21.15.5:0/1838019762 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f34148f880 0x55f341499830 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fa7d0013120 tx=0x7fa7d0003800 comp rx=0 tx=0).stop 2024-09-10T10:04:35.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:34.617+0000 7fa7e7015740 1 -- 172.21.15.5:0/1838019762 shutdown_connections 2024-09-10T10:04:35.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:34.617+0000 7fa7e7015740 1 --2- 172.21.15.5:0/1838019762 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f34149aa60 0x55f341498230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:35.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:34.617+0000 7fa7e7015740 1 --2- 172.21.15.5:0/1838019762 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f3414901a0 0x55f341496250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:35.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:34.617+0000 7fa7e7015740 1 --2- 172.21.15.5:0/1838019762 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f34148f880 0x55f341499830 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:35.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:34.617+0000 7fa7e7015740 1 -- 172.21.15.5:0/1838019762 >> 172.21.15.5:0/1838019762 conn(0x55f34148be10 msgr2=0x55f341491b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:35.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:34.617+0000 7fa7e7015740 1 -- 172.21.15.5:0/1838019762 shutdown_connections 2024-09-10T10:04:35.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:34.617+0000 7fa7e7015740 1 -- 172.21.15.5:0/1838019762 wait complete. 2024-09-10T10:04:35.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:34.617+0000 7fa7e7015740 1 -- start start 2024-09-10T10:04:35.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:34.617+0000 7fa7e7015740 1 Processor -- start 2024-09-10T10:04:35.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:34.617+0000 7fa7e7015740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f34148f880 0x55f3414a2a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:35.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:34.617+0000 7fa7e7015740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f3414901a0 0x55f3414a2f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:35.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:34.617+0000 7fa7e7015740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f34149aa60 0x55f34149fa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:35.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:34.617+0000 7fa7e7015740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa7dc1adec0 con 0x55f34149aa60 2024-09-10T10:04:35.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:34.617+0000 7fa7e7015740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fa7dc1ad7c0 con 0x55f3414901a0 2024-09-10T10:04:35.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:34.617+0000 7fa7e7015740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fa7dc1aa9a0 con 0x55f34148f880 2024-09-10T10:04:35.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:34.617+0000 7fa7e7015740 10 client.0 _ll_register_callbacks cb 0x7fa7d00110d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:04:35.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:34.617+0000 7fa7e1000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f3414901a0 0x55f3414a2f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:35.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:34.617+0000 7fa7e1000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f3414901a0 0x55f3414a2f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:49150/0 (socket says 172.21.15.5:49150) 2024-09-10T10:04:35.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:34.617+0000 7fa7e1000640 1 -- 172.21.15.5:0/4199699875 learned_addr learned my addr 172.21.15.5:0/4199699875 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T10:04:35.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:34.617+0000 7fa7e2400640 1 --2- 172.21.15.5:0/4199699875 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f34149aa60 0x55f34149fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:35.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:34.617+0000 7fa7e1a00640 1 --2- 172.21.15.5:0/4199699875 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f34148f880 0x55f3414a2a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:35.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:34.617+0000 7fa7e1000640 1 -- 172.21.15.5:0/4199699875 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f34148f880 msgr2=0x55f3414a2a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:35.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:34.617+0000 7fa7e1000640 1 --2- 172.21.15.5:0/4199699875 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f34148f880 0x55f3414a2a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:35.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:34.617+0000 7fa7e1000640 1 -- 172.21.15.5:0/4199699875 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f34149aa60 msgr2=0x55f34149fa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:35.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:34.617+0000 7fa7e1000640 1 --2- 172.21.15.5:0/4199699875 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f34149aa60 0x55f34149fa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:35.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:34.617+0000 7fa7e1000640 1 -- 172.21.15.5:0/4199699875 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f3414a0060 con 0x55f3414901a0 2024-09-10T10:04:35.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:34.617+0000 7fa7e1a00640 1 --2- 172.21.15.5:0/4199699875 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f34148f880 0x55f3414a2a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:04:35.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:34.617+0000 7fa7e2400640 1 --2- 172.21.15.5:0/4199699875 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f34149aa60 0x55f34149fa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:04:35.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:34.617+0000 7fa7e1000640 1 --2- 172.21.15.5:0/4199699875 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f3414901a0 0x55f3414a2f90 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7fa7c8007590 tx=0x7fa7c80cf720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=57496d77ce3f6cd2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:35.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:34.617+0000 7fa7c3400640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-09-10T10:04:35.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:34.617+0000 7fa7c3400640 1 -- 172.21.15.5:0/4199699875 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7c809b420 con 0x55f3414901a0 2024-09-10T10:04:35.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:34.617+0000 7fa7c3400640 1 -- 172.21.15.5:0/4199699875 <== mon.2 v2:172.21.15.102:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fa7c8099be0 con 0x55f3414901a0 2024-09-10T10:04:35.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:34.617+0000 7fa7e7015740 1 -- 172.21.15.5:0/4199699875 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa7d0004b90 con 0x55f3414901a0 2024-09-10T10:04:35.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:34.617+0000 7fa7c3400640 1 -- 172.21.15.5:0/4199699875 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7fa7c809b6b0 con 0x55f3414901a0 2024-09-10T10:04:35.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:34.617+0000 7fa7e7015740 10 client.30308 fetch_fsmap learned FSMap version 47 2024-09-10T10:04:35.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:34.617+0000 7fa7e7015740 1 -- 172.21.15.5:0/4199699875 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x55f341516040 con 0x55f3414901a0 2024-09-10T10:04:35.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:34.617+0000 7fa7c3400640 1 -- 172.21.15.5:0/4199699875 <== mon.2 v2:172.21.15.102:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa7d0004b90 con 0x55f3414901a0 2024-09-10T10:04:35.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:34.617+0000 7fa7c3400640 1 -- 172.21.15.5:0/4199699875 <== mon.2 v2:172.21.15.102:3300/0 5 ==== fsmap(e 47) ==== 8398+0+0 (secure 0 0 0) 0x7fa7c80ba140 con 0x55f3414901a0 2024-09-10T10:04:35.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:34.617+0000 7fa7e7015740 10 client.30308 fetch_fsmap finished waiting for FSMap version 47 2024-09-10T10:04:35.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:34.617+0000 7fa7e7015740 10 client.30308 resolve_mds: validated gid 24542 aka daemon mds.b 2024-09-10T10:04:35.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:34.617+0000 7fa7e7015740 20 client.30308 populate_metadata read hostname 'smithi005' 2024-09-10T10:04:35.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:34.617+0000 7fa7e7015740 1 --2- 172.21.15.5:0/4199699875 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55f341516380 0x55f341518770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:35.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:34.617+0000 7fa7e7015740 4 client.30308 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "quiesce db", "cancel": true, "all": true}] 2024-09-10T10:04:35.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:34.617+0000 7fa7e7015740 1 -- 172.21.15.5:0/4199699875 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "quiesce db", "cancel": true, "all": true}) -- 0x55f3413fddc0 con 0x55f341516380 2024-09-10T10:04:35.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:34.617+0000 7fa7e1a00640 1 --2- 172.21.15.5:0/4199699875 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55f341516380 0x55f341518770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:35.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:34.622+0000 7fa7e1a00640 1 --2- 172.21.15.5:0/4199699875 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55f341516380 0x55f341518770 crc :-1 s=READY pgs=350 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c78637dfdb66e241 server_cookie=b94a843eb13bd30 in_seq=0 out_seq=0 2024-09-10T10:04:35.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:34.622+0000 7fa7c3400640 10 client.30308 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T10:04:35.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:34.622+0000 7fa7c3400640 1 -- 172.21.15.5:0/4199699875 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+80 (crc 0 0 0) 0x55f3413fddc0 con 0x55f341516380 2024-09-10T10:04:35.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:34.622+0000 7fa7c3400640 10 client.30308 handle_command_reply: tid=0 2024-09-10T10:04:35.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:34.622+0000 7fa7e7015740 1 client.30308 shutdown 2024-09-10T10:04:35.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:34.622+0000 7fa7e7015740 1 -- 172.21.15.5:0/4199699875 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f3414901a0 msgr2=0x55f3414a2f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:35.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:34.622+0000 7fa7e7015740 1 --2- 172.21.15.5:0/4199699875 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f3414901a0 0x55f3414a2f90 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7fa7c8007590 tx=0x7fa7c80cf720 comp rx=0 tx=0).stop 2024-09-10T10:04:35.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:34.622+0000 7fa7e7015740 1 -- 172.21.15.5:0/4199699875 shutdown_connections 2024-09-10T10:04:35.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:34.622+0000 7fa7e7015740 1 --2- 172.21.15.5:0/4199699875 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55f341516380 0x55f341518770 crc :-1 s=READY pgs=350 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:35.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:34.622+0000 7fa7e7015740 1 --2- 172.21.15.5:0/4199699875 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f34149aa60 0x55f34149fa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:35.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:34.622+0000 7fa7e7015740 1 --2- 172.21.15.5:0/4199699875 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55f3414901a0 0x55f3414a2f90 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:35.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:34.622+0000 7fa7e7015740 1 --2- 172.21.15.5:0/4199699875 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55f34148f880 0x55f3414a2a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:35.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:34.622+0000 7fa7e7015740 1 -- 172.21.15.5:0/4199699875 >> 172.21.15.5:0/4199699875 conn(0x55f34148be10 msgr2=0x55f34148a690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:35.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:34.622+0000 7fa7c3400640 0 client.30308 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T10:04:35.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:34.622+0000 7fa7e7015740 1 -- 172.21.15.5:0/4199699875 shutdown_connections 2024-09-10T10:04:35.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:34.622+0000 7fa7e7015740 1 -- 172.21.15.5:0/4199699875 wait complete. 2024-09-10T10:04:35.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:34.622+0000 7fa7e7015740 20 client.30308 trim_cache size 0 max 16384 2024-09-10T10:04:35.346 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-10T10:04:35.346 INFO:teuthology.orchestra.run.smithi005.stdout: "epoch": 47, 2024-09-10T10:04:35.346 INFO:teuthology.orchestra.run.smithi005.stdout: "leader": 24542, 2024-09-10T10:04:35.346 INFO:teuthology.orchestra.run.smithi005.stdout: "set_version": 92, 2024-09-10T10:04:35.346 INFO:teuthology.orchestra.run.smithi005.stdout: "sets": {} 2024-09-10T10:04:35.346 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-10T10:04:35.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:34.624+0000 7fa7e2e00640 1 -- 172.21.15.5:0/1204373047 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fa7c40a2cc0 msgr2=0x7fa7c40a5180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:35.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:34.624+0000 7fa7e2e00640 1 --2- 172.21.15.5:0/1204373047 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fa7c40a2cc0 0x7fa7c40a5180 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fa7d4005fd0 tx=0x7fa7d4009600 comp rx=0 tx=0).stop 2024-09-10T10:04:35.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:34.624+0000 7fa7e2e00640 1 -- 172.21.15.5:0/1204373047 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa7dc1047e0 msgr2=0x7fa7dc196a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:35.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:34.624+0000 7fa7e2e00640 1 --2- 172.21.15.5:0/1204373047 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa7dc1047e0 0x7fa7dc196a50 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7fa7c800c920 tx=0x7fa7c800cde0 comp rx=0 tx=0).stop 2024-09-10T10:04:35.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:34.624+0000 7fa7e2e00640 1 -- 172.21.15.5:0/1204373047 shutdown_connections 2024-09-10T10:04:35.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:34.624+0000 7fa7e2e00640 1 --2- 172.21.15.5:0/1204373047 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fa7c40a2cc0 0x7fa7c40a5180 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:35.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:34.624+0000 7fa7e2e00640 1 --2- 172.21.15.5:0/1204373047 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa7dc108f60 0x7fa7dc19a860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:35.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:34.624+0000 7fa7e2e00640 1 --2- 172.21.15.5:0/1204373047 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa7dc1047e0 0x7fa7dc196a50 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:35.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:34.624+0000 7fa7e2e00640 1 --2- 172.21.15.5:0/1204373047 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa7dc103e30 0x7fa7dc196510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:35.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:34.624+0000 7fa7e2e00640 1 -- 172.21.15.5:0/1204373047 >> 172.21.15.5:0/1204373047 conn(0x7fa7dc0ff8d0 msgr2=0x7fa7dc107810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:35.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:34.624+0000 7fa7e2e00640 1 -- 172.21.15.5:0/1204373047 shutdown_connections 2024-09-10T10:04:35.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:34.624+0000 7fa7e2e00640 1 -- 172.21.15.5:0/1204373047 wait complete. 2024-09-10T10:04:36.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:04:35 smithi005 ceph-mon[30350]: pgmap v1180: 129 pgs: 129 active+clean; 636 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.5 MiB/s wr, 249 op/s 2024-09-10T10:04:36.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:04:35 smithi159 ceph-mon[35513]: pgmap v1180: 129 pgs: 129 active+clean; 636 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.5 MiB/s wr, 249 op/s 2024-09-10T10:04:36.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:04:35 smithi102 ceph-mon[35442]: pgmap v1180: 129 pgs: 129 active+clean; 636 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.5 MiB/s wr, 249 op/s 2024-09-10T10:04:37.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:04:37 smithi005 ceph-mon[30350]: pgmap v1181: 129 pgs: 129 active+clean; 609 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.1 MiB/s wr, 280 op/s 2024-09-10T10:04:38.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:04:37 smithi159 ceph-mon[35513]: pgmap v1181: 129 pgs: 129 active+clean; 609 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.1 MiB/s wr, 280 op/s 2024-09-10T10:04:38.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:04:37 smithi102 ceph-mon[35442]: pgmap v1181: 129 pgs: 129 active+clean; 609 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.1 MiB/s wr, 280 op/s 2024-09-10T10:04:38.384 INFO:tasks.quiescer:done joining Quiescers 2024-09-10T10:04:38.385 DEBUG:teuthology.run_tasks:Unwinding manager fwd_scrub 2024-09-10T10:04:38.395 INFO:tasks.fwd_scrub:joining ForwardScrubbers 2024-09-10T10:04:38.414 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:04:39.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:39.176+0000 7efedec00640 1 -- 172.21.15.5:0/814683927 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efed81079c0 msgr2=0x7efed8107da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:39.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:39.176+0000 7efedec00640 1 --2- 172.21.15.5:0/814683927 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efed81079c0 0x7efed8107da0 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7efecc009930 tx=0x7efecc02a420 comp rx=0 tx=0).stop 2024-09-10T10:04:39.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:39.177+0000 7efedec00640 1 -- 172.21.15.5:0/814683927 shutdown_connections 2024-09-10T10:04:39.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:39.177+0000 7efedec00640 1 --2- 172.21.15.5:0/814683927 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efed8102360 0x7efed810e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:39.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:39.177+0000 7efedec00640 1 --2- 172.21.15.5:0/814683927 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efed81019c0 0x7efed8101e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:39.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:39.177+0000 7efedec00640 1 --2- 172.21.15.5:0/814683927 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efed81079c0 0x7efed8107da0 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:39.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:39.177+0000 7efedec00640 1 -- 172.21.15.5:0/814683927 >> 172.21.15.5:0/814683927 conn(0x7efed80fd740 msgr2=0x7efed80ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:39.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:39.177+0000 7efedec00640 1 -- 172.21.15.5:0/814683927 shutdown_connections 2024-09-10T10:04:39.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:39.177+0000 7efedec00640 1 -- 172.21.15.5:0/814683927 wait complete. 2024-09-10T10:04:39.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:39.178+0000 7efedec00640 1 Processor -- start 2024-09-10T10:04:39.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:39.178+0000 7efedec00640 1 -- start start 2024-09-10T10:04:39.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:39.178+0000 7efedec00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efed81019c0 0x7efed80744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:39.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:39.178+0000 7efedec00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efed8102360 0x7efed8074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:39.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:39.178+0000 7efedec00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efed81079c0 0x7efed8074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:39.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:39.178+0000 7efedec00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7efed8111dc0 con 0x7efed81019c0 2024-09-10T10:04:39.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:39.178+0000 7efedec00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7efed8111c40 con 0x7efed8102360 2024-09-10T10:04:39.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:39.178+0000 7efedec00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7efed8111f40 con 0x7efed81079c0 2024-09-10T10:04:39.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:39.179+0000 7efedd800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efed81019c0 0x7efed80744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:39.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:39.179+0000 7efedd800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efed81019c0 0x7efed80744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47916/0 (socket says 172.21.15.5:47916) 2024-09-10T10:04:39.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:39.179+0000 7efedd800640 1 -- 172.21.15.5:0/3514870853 learned_addr learned my addr 172.21.15.5:0/3514870853 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T10:04:39.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:39.179+0000 7efedce00640 1 --2- 172.21.15.5:0/3514870853 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efed8102360 0x7efed8074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:39.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:39.179+0000 7efede200640 1 --2- 172.21.15.5:0/3514870853 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efed81079c0 0x7efed8074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:39.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:39.179+0000 7efedd800640 1 -- 172.21.15.5:0/3514870853 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efed81079c0 msgr2=0x7efed8074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:39.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:39.179+0000 7efedd800640 1 --2- 172.21.15.5:0/3514870853 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efed81079c0 0x7efed8074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:39.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:39.179+0000 7efedd800640 1 -- 172.21.15.5:0/3514870853 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efed8102360 msgr2=0x7efed8074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:39.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:39.179+0000 7efedd800640 1 --2- 172.21.15.5:0/3514870853 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efed8102360 0x7efed8074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:39.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:39.179+0000 7efedd800640 1 -- 172.21.15.5:0/3514870853 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efed81a67e0 con 0x7efed81019c0 2024-09-10T10:04:39.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:39.179+0000 7efedd800640 1 --2- 172.21.15.5:0/3514870853 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efed81019c0 0x7efed80744f0 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7efecc009230 tx=0x7efecc004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:39.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:39.180+0000 7efed6000640 1 -- 172.21.15.5:0/3514870853 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efecc03f030 con 0x7efed81019c0 2024-09-10T10:04:39.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:39.180+0000 7efed6000640 1 -- 172.21.15.5:0/3514870853 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efecc003ad0 con 0x7efed81019c0 2024-09-10T10:04:39.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:39.180+0000 7efed6000640 1 -- 172.21.15.5:0/3514870853 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efecc035070 con 0x7efed81019c0 2024-09-10T10:04:39.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:39.180+0000 7efedec00640 1 -- 172.21.15.5:0/3514870853 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efed81a6a70 con 0x7efed81019c0 2024-09-10T10:04:39.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:39.180+0000 7efedec00640 1 -- 172.21.15.5:0/3514870853 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efed81a6f00 con 0x7efed81019c0 2024-09-10T10:04:39.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:39.183+0000 7efed6000640 1 -- 172.21.15.5:0/3514870853 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7efecc042030 con 0x7efed81019c0 2024-09-10T10:04:39.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:39.183+0000 7efee2c26740 1 Processor -- start 2024-09-10T10:04:39.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:39.183+0000 7efee2c26740 1 -- start start 2024-09-10T10:04:39.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:39.184+0000 7efee2c26740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55998be2ce40 0x55998be2d220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:39.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:39.184+0000 7efee2c26740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55998be2d760 0x55998be37ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:39.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:39.184+0000 7efee2c26740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55998be38020 0x55998be3a410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:39.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:39.184+0000 7efee2c26740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55998bd933e0 con 0x55998be38020 2024-09-10T10:04:39.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:39.184+0000 7efee2c26740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55998bd3ad80 con 0x55998be2d760 2024-09-10T10:04:39.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:39.184+0000 7efee2c26740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55998bd8f3a0 con 0x55998be2ce40 2024-09-10T10:04:39.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:39.184+0000 7efedd800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55998be2d760 0x55998be37ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:39.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:39.184+0000 7efedd800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55998be2d760 0x55998be37ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:49176/0 (socket says 172.21.15.5:49176) 2024-09-10T10:04:39.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:39.184+0000 7efedd800640 1 -- 172.21.15.5:0/4250915275 learned_addr learned my addr 172.21.15.5:0/4250915275 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T10:04:39.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:39.184+0000 7efedce00640 1 --2- 172.21.15.5:0/4250915275 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55998be38020 0x55998be3a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:39.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:39.185+0000 7efedd800640 1 -- 172.21.15.5:0/4250915275 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55998be2ce40 msgr2=0x55998be2d220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:39.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:39.185+0000 7efedd800640 1 --2- 172.21.15.5:0/4250915275 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55998be2ce40 0x55998be2d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:39.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:39.185+0000 7efedd800640 1 -- 172.21.15.5:0/4250915275 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55998be38020 msgr2=0x55998be3a410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:39.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:39.185+0000 7efedd800640 1 --2- 172.21.15.5:0/4250915275 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55998be38020 0x55998be3a410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:39.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:39.185+0000 7efedd800640 1 -- 172.21.15.5:0/4250915275 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55998bd59ae0 con 0x55998be2d760 2024-09-10T10:04:39.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:39.185+0000 7efed6000640 1 --2- 172.21.15.5:0/3514870853 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7efec00a2c00 0x7efec00a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:39.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:39.185+0000 7efedd800640 1 --2- 172.21.15.5:0/4250915275 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55998be2d760 0x55998be37ae0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7efecc0f5c20 tx=0x7efecc0f5d10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=84dd0e39062f9b4e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:39.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:39.185+0000 7efede200640 1 --2- 172.21.15.5:0/3514870853 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7efec00a2c00 0x7efec00a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:39.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:39.186+0000 7efed6000640 1 -- 172.21.15.5:0/3514870853 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7efecc0ef9b0 con 0x7efed81019c0 2024-09-10T10:04:39.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:39.186+0000 7efecaa00640 1 -- 172.21.15.5:0/4250915275 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efecc0b8110 con 0x55998be2d760 2024-09-10T10:04:39.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:39.186+0000 7efecaa00640 1 -- 172.21.15.5:0/4250915275 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efecc03b460 con 0x55998be2d760 2024-09-10T10:04:39.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:39.186+0000 7efecaa00640 1 -- 172.21.15.5:0/4250915275 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efecc0bb7d0 con 0x55998be2d760 2024-09-10T10:04:39.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:39.186+0000 7efee2c26740 1 -- 172.21.15.5:0/4250915275 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55998be2d760 msgr2=0x55998be37ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:39.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:39.186+0000 7efede200640 1 --2- 172.21.15.5:0/3514870853 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7efec00a2c00 0x7efec00a50c0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7efed0005fd0 tx=0x7efed0009a90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:39.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:39.186+0000 7efee2c26740 1 --2- 172.21.15.5:0/4250915275 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55998be2d760 0x55998be37ae0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7efecc0f5c20 tx=0x7efecc0f5d10 comp rx=0 tx=0).stop 2024-09-10T10:04:39.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:39.187+0000 7efee2c26740 1 -- 172.21.15.5:0/4250915275 shutdown_connections 2024-09-10T10:04:39.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:39.187+0000 7efee2c26740 1 --2- 172.21.15.5:0/4250915275 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55998be38020 0x55998be3a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:39.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:39.187+0000 7efee2c26740 1 --2- 172.21.15.5:0/4250915275 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55998be2d760 0x55998be37ae0 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:39.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:39.187+0000 7efee2c26740 1 --2- 172.21.15.5:0/4250915275 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55998be2ce40 0x55998be2d220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:39.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:39.187+0000 7efee2c26740 1 -- 172.21.15.5:0/4250915275 >> 172.21.15.5:0/4250915275 conn(0x55998be293d0 msgr2=0x55998be297b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:39.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:39.187+0000 7efee2c26740 1 -- 172.21.15.5:0/4250915275 shutdown_connections 2024-09-10T10:04:39.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:39.187+0000 7efee2c26740 1 -- 172.21.15.5:0/4250915275 wait complete. 2024-09-10T10:04:39.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:39.187+0000 7efee2c26740 1 -- start start 2024-09-10T10:04:39.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:39.188+0000 7efee2c26740 1 Processor -- start 2024-09-10T10:04:39.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:39.188+0000 7efee2c26740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55998be2ce40 0x55998be34ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:39.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:39.188+0000 7efee2c26740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55998be2d760 0x55998be35420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:39.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:39.188+0000 7efee2c26740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55998be38020 0x55998be35960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:39.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:39.188+0000 7efee2c26740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55998be3a9b0 con 0x55998be2d760 2024-09-10T10:04:39.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:39.188+0000 7efee2c26740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55998bd4e0f0 con 0x55998be2ce40 2024-09-10T10:04:39.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:39.188+0000 7efee2c26740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55998be3ab30 con 0x55998be38020 2024-09-10T10:04:39.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:39.188+0000 7efee2c26740 10 client.0 _ll_register_callbacks cb 0x55998bd35bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:04:39.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:39.188+0000 7efedce00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55998be2d760 0x55998be35420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:39.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:39.188+0000 7efedd800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55998be2ce40 0x55998be34ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:39.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:39.189+0000 7efedce00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55998be2d760 0x55998be35420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47926/0 (socket says 172.21.15.5:47926) 2024-09-10T10:04:39.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:39.189+0000 7efedce00640 1 -- 172.21.15.5:0/4129395042 learned_addr learned my addr 172.21.15.5:0/4129395042 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T10:04:39.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:39.189+0000 7efede200640 1 --2- 172.21.15.5:0/4129395042 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55998be38020 0x55998be35960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:39.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:39.189+0000 7efedce00640 1 -- 172.21.15.5:0/4129395042 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55998be38020 msgr2=0x55998be35960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:39.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:39.189+0000 7efedce00640 1 --2- 172.21.15.5:0/4129395042 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55998be38020 0x55998be35960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:39.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:39.189+0000 7efedce00640 1 -- 172.21.15.5:0/4129395042 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55998be2ce40 msgr2=0x55998be34ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:39.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:39.189+0000 7efedce00640 1 --2- 172.21.15.5:0/4129395042 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55998be2ce40 0x55998be34ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:39.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:39.189+0000 7efedce00640 1 -- 172.21.15.5:0/4129395042 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55998bb043c0 con 0x55998be2d760 2024-09-10T10:04:39.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:39.189+0000 7efedce00640 1 --2- 172.21.15.5:0/4129395042 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55998be2d760 0x55998be35420 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7efec400b950 tx=0x7efec400be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1958ea6a53a89df0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:39.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:39.189+0000 7efebf400640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-10T10:04:39.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:39.190+0000 7efebf400640 1 -- 172.21.15.5:0/4129395042 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efec401b440 con 0x55998be2d760 2024-09-10T10:04:39.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:39.190+0000 7efebf400640 1 -- 172.21.15.5:0/4129395042 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7efec401b5e0 con 0x55998be2d760 2024-09-10T10:04:39.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:39.190+0000 7efee2c26740 1 -- 172.21.15.5:0/4129395042 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55998bda0950 con 0x55998be2d760 2024-09-10T10:04:39.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:39.190+0000 7efebf400640 1 -- 172.21.15.5:0/4129395042 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7efec4024050 con 0x55998be2d760 2024-09-10T10:04:39.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:39.190+0000 7efee2c26740 10 client.20223 fetch_fsmap learned FSMap version 47 2024-09-10T10:04:39.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:39.190+0000 7efee2c26740 1 -- 172.21.15.5:0/4129395042 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x55998be46990 con 0x55998be2d760 2024-09-10T10:04:39.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:39.190+0000 7efebf400640 1 -- 172.21.15.5:0/4129395042 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55998bda0950 con 0x55998be2d760 2024-09-10T10:04:39.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:39.191+0000 7efebf400640 1 -- 172.21.15.5:0/4129395042 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 47) ==== 8398+0+0 (secure 0 0 0) 0x7efec4019020 con 0x55998be2d760 2024-09-10T10:04:39.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:39.191+0000 7efee2c26740 10 client.20223 fetch_fsmap finished waiting for FSMap version 47 2024-09-10T10:04:39.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:39.191+0000 7efee2c26740 10 client.20223 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-09-10T10:04:39.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:39.191+0000 7efee2c26740 20 client.20223 populate_metadata read hostname 'smithi005' 2024-09-10T10:04:39.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:39.191+0000 7efee2c26740 1 --2- 172.21.15.5:0/4129395042 >> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] conn(0x55998be47e50 0x55998be4a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:39.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:39.191+0000 7efee2c26740 4 client.20223 mds_command: new command op to 24443 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:04:39.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:39.191+0000 7efee2c26740 1 -- 172.21.15.5:0/4129395042 --> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55998bd9ae60 con 0x55998be47e50 2024-09-10T10:04:39.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:39.191+0000 7efedd800640 1 --2- 172.21.15.5:0/4129395042 >> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] conn(0x55998be47e50 0x55998be4a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:39.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:39.192+0000 7efedd800640 1 --2- 172.21.15.5:0/4129395042 >> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] conn(0x55998be47e50 0x55998be4a2d0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=5b551cd3745bf3eb server_cookie=c8318dc039df9756 in_seq=0 out_seq=0 2024-09-10T10:04:39.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:39.192+0000 7efebf400640 10 client.20223 ms_handle_connect on v2:172.21.15.102:6832/2457007427 2024-09-10T10:04:39.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:39.192+0000 7efebf400640 1 -- 172.21.15.5:0/4129395042 <== mds.4 v2:172.21.15.102:6832/2457007427 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55998bd9ae60 con 0x55998be47e50 2024-09-10T10:04:39.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:39.192+0000 7efebf400640 10 client.20223 handle_command_reply: tid=0 2024-09-10T10:04:39.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:39.193+0000 7efee2c26740 1 client.20223 shutdown 2024-09-10T10:04:39.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:39.193+0000 7efee2c26740 1 -- 172.21.15.5:0/4129395042 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55998be2d760 msgr2=0x55998be35420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:39.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:39.193+0000 7efee2c26740 1 --2- 172.21.15.5:0/4129395042 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55998be2d760 0x55998be35420 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7efec400b950 tx=0x7efec400be10 comp rx=0 tx=0).stop 2024-09-10T10:04:39.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:39.193+0000 7efee2c26740 1 -- 172.21.15.5:0/4129395042 shutdown_connections 2024-09-10T10:04:39.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:39.193+0000 7efee2c26740 1 --2- 172.21.15.5:0/4129395042 >> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] conn(0x55998be47e50 0x55998be4a2d0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:39.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:39.193+0000 7efee2c26740 1 --2- 172.21.15.5:0/4129395042 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55998be38020 0x55998be35960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:39.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:39.193+0000 7efee2c26740 1 --2- 172.21.15.5:0/4129395042 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55998be2d760 0x55998be35420 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:39.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:39.193+0000 7efee2c26740 1 --2- 172.21.15.5:0/4129395042 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55998be2ce40 0x55998be34ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:39.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:39.193+0000 7efebf400640 0 client.20223 ms_handle_reset on v2:172.21.15.102:6832/2457007427 2024-09-10T10:04:39.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:39.193+0000 7efee2c26740 1 -- 172.21.15.5:0/4129395042 >> 172.21.15.5:0/4129395042 conn(0x55998be293d0 msgr2=0x55998be2f0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:39.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:39.193+0000 7efee2c26740 1 -- 172.21.15.5:0/4129395042 shutdown_connections 2024-09-10T10:04:39.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:39.193+0000 7efee2c26740 1 -- 172.21.15.5:0/4129395042 wait complete. 2024-09-10T10:04:39.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:39.193+0000 7efee2c26740 20 client.20223 trim_cache size 0 max 16384 2024-09-10T10:04:39.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:39.208+0000 7efee2c26740 1 Processor -- start 2024-09-10T10:04:39.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:39.208+0000 7efee2c26740 1 -- start start 2024-09-10T10:04:39.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:39.208+0000 7efee2c26740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55998be2ce40 0x55998be2f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:39.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:39.208+0000 7efee2c26740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55998be2d760 0x55998be347a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:39.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:39.208+0000 7efee2c26740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55998be38020 0x55998be4fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:39.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:39.208+0000 7efee2c26740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7efed81b0f60 con 0x55998be38020 2024-09-10T10:04:39.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:39.208+0000 7efee2c26740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7efed81b2b60 con 0x55998be2d760 2024-09-10T10:04:39.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:39.208+0000 7efee2c26740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7efed81b1630 con 0x55998be2ce40 2024-09-10T10:04:39.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:39.209+0000 7efede200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55998be38020 0x55998be4fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:39.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:39.209+0000 7efedd800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55998be2ce40 0x55998be2f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:39.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:39.209+0000 7efedce00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55998be2d760 0x55998be347a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:39.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:39.209+0000 7efedd800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55998be2ce40 0x55998be2f8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:42356/0 (socket says 172.21.15.5:42356) 2024-09-10T10:04:39.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:39.209+0000 7efedd800640 1 -- 172.21.15.5:0/292358841 learned_addr learned my addr 172.21.15.5:0/292358841 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T10:04:39.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:39.209+0000 7efedd800640 1 -- 172.21.15.5:0/292358841 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55998be2d760 msgr2=0x55998be347a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:39.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:39.209+0000 7efedd800640 1 --2- 172.21.15.5:0/292358841 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55998be2d760 0x55998be347a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:39.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:39.209+0000 7efedd800640 1 -- 172.21.15.5:0/292358841 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55998be38020 msgr2=0x55998be4fbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:39.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:39.209+0000 7efedd800640 1 --2- 172.21.15.5:0/292358841 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55998be38020 0x55998be4fbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:39.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:39.209+0000 7efedd800640 1 -- 172.21.15.5:0/292358841 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55998be33e90 con 0x55998be2ce40 2024-09-10T10:04:39.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:39.209+0000 7efede200640 1 --2- 172.21.15.5:0/292358841 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55998be38020 0x55998be4fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:04:39.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:39.210+0000 7efedd800640 1 --2- 172.21.15.5:0/292358841 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55998be2ce40 0x55998be2f8a0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7efecc0bbb70 tx=0x7efecc0f5f60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b00cd82e76590ab7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:39.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:39.210+0000 7efeca000640 1 -- 172.21.15.5:0/292358841 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efecc049040 con 0x55998be2ce40 2024-09-10T10:04:39.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:39.210+0000 7efeca000640 1 -- 172.21.15.5:0/292358841 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efecc0bbe70 con 0x55998be2ce40 2024-09-10T10:04:39.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:39.210+0000 7efeca000640 1 -- 172.21.15.5:0/292358841 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efecc0bb9b0 con 0x55998be2ce40 2024-09-10T10:04:39.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:39.211+0000 7efee2c26740 1 -- 172.21.15.5:0/292358841 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55998be2ce40 msgr2=0x55998be2f8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:39.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:39.211+0000 7efee2c26740 1 --2- 172.21.15.5:0/292358841 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55998be2ce40 0x55998be2f8a0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7efecc0bbb70 tx=0x7efecc0f5f60 comp rx=0 tx=0).stop 2024-09-10T10:04:39.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:39.211+0000 7efee2c26740 1 -- 172.21.15.5:0/292358841 shutdown_connections 2024-09-10T10:04:39.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:39.211+0000 7efee2c26740 1 --2- 172.21.15.5:0/292358841 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55998be38020 0x55998be4fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:39.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:39.211+0000 7efee2c26740 1 --2- 172.21.15.5:0/292358841 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55998be2d760 0x55998be347a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:39.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:39.211+0000 7efee2c26740 1 --2- 172.21.15.5:0/292358841 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55998be2ce40 0x55998be2f8a0 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:39.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:39.211+0000 7efee2c26740 1 -- 172.21.15.5:0/292358841 >> 172.21.15.5:0/292358841 conn(0x55998be293d0 msgr2=0x55998be36f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:39.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:39.211+0000 7efee2c26740 1 -- 172.21.15.5:0/292358841 shutdown_connections 2024-09-10T10:04:39.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:39.211+0000 7efee2c26740 1 -- 172.21.15.5:0/292358841 wait complete. 2024-09-10T10:04:39.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:39.211+0000 7efee2c26740 1 -- start start 2024-09-10T10:04:39.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:39.211+0000 7efee2c26740 1 Processor -- start 2024-09-10T10:04:39.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:39.212+0000 7efee2c26740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55998be2ce40 0x55998be3ba40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:39.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:39.212+0000 7efee2c26740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55998be2d760 0x55998be3bf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:39.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:39.212+0000 7efee2c26740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55998be38020 0x55998be3c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:39.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:39.212+0000 7efee2c26740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7efed81b2460 con 0x55998be2d760 2024-09-10T10:04:39.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:39.212+0000 7efee2c26740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7efed81b1d60 con 0x55998be38020 2024-09-10T10:04:39.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:39.212+0000 7efee2c26740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7efed81aef40 con 0x55998be2ce40 2024-09-10T10:04:39.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:39.212+0000 7efee2c26740 10 client.0 _ll_register_callbacks cb 0x7efec4004120 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:04:39.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:39.212+0000 7efedce00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55998be2d760 0x55998be3bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:39.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:39.212+0000 7efedd800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55998be2ce40 0x55998be3ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:39.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:39.212+0000 7efedce00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55998be2d760 0x55998be3bf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47940/0 (socket says 172.21.15.5:47940) 2024-09-10T10:04:39.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:39.212+0000 7efedce00640 1 -- 172.21.15.5:0/1442887778 learned_addr learned my addr 172.21.15.5:0/1442887778 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T10:04:39.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:39.213+0000 7efede200640 1 --2- 172.21.15.5:0/1442887778 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55998be38020 0x55998be3c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:39.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:39.213+0000 7efedd800640 1 -- 172.21.15.5:0/1442887778 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55998be38020 msgr2=0x55998be3c4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:39.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:39.213+0000 7efedd800640 1 --2- 172.21.15.5:0/1442887778 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55998be38020 0x55998be3c4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:39.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:39.213+0000 7efedd800640 1 -- 172.21.15.5:0/1442887778 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55998be2d760 msgr2=0x55998be3bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:39.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:39.213+0000 7efedd800640 1 --2- 172.21.15.5:0/1442887778 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55998be2d760 0x55998be3bf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:39.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:39.213+0000 7efedd800640 1 -- 172.21.15.5:0/1442887778 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55998beba3c0 con 0x55998be2ce40 2024-09-10T10:04:39.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:39.213+0000 7efedce00640 1 --2- 172.21.15.5:0/1442887778 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55998be2d760 0x55998be3bf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:04:39.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:39.213+0000 7efedd800640 1 --2- 172.21.15.5:0/1442887778 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55998be2ce40 0x55998be3ba40 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7efecc0bc030 tx=0x7efecc009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b4086e915b07306b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:39.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:39.213+0000 7efebf400640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-09-10T10:04:39.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:39.213+0000 7efebf400640 1 -- 172.21.15.5:0/1442887778 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efecc049050 con 0x55998be2ce40 2024-09-10T10:04:39.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:39.214+0000 7efee2c26740 1 -- 172.21.15.5:0/1442887778 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55998bd17010 con 0x55998be2ce40 2024-09-10T10:04:39.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:39.214+0000 7efebf400640 1 -- 172.21.15.5:0/1442887778 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7efecc03bd40 con 0x55998be2ce40 2024-09-10T10:04:39.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:39.214+0000 7efebf400640 1 -- 172.21.15.5:0/1442887778 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7efecc0bbc40 con 0x55998be2ce40 2024-09-10T10:04:39.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:39.214+0000 7efee2c26740 10 client.30301 fetch_fsmap learned FSMap version 47 2024-09-10T10:04:39.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:39.214+0000 7efee2c26740 1 -- 172.21.15.5:0/1442887778 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x55998beba3c0 con 0x55998be2ce40 2024-09-10T10:04:39.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:39.214+0000 7efebf400640 1 -- 172.21.15.5:0/1442887778 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55998bd17010 con 0x55998be2ce40 2024-09-10T10:04:39.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:39.214+0000 7efebf400640 1 -- 172.21.15.5:0/1442887778 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 47) ==== 8398+0+0 (secure 0 0 0) 0x7efecc09ace0 con 0x55998be2ce40 2024-09-10T10:04:39.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:39.214+0000 7efee2c26740 10 client.30301 fetch_fsmap finished waiting for FSMap version 47 2024-09-10T10:04:39.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:39.214+0000 7efee2c26740 10 client.30301 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-09-10T10:04:39.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:39.214+0000 7efee2c26740 20 client.30301 populate_metadata read hostname 'smithi005' 2024-09-10T10:04:39.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:39.214+0000 7efee2c26740 1 --2- 172.21.15.5:0/1442887778 >> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] conn(0x55998bebb830 0x55998bebdc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:39.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:39.214+0000 7efee2c26740 4 client.30301 mds_command: new command op to 24443 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T10:04:39.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:39.214+0000 7efee2c26740 1 -- 172.21.15.5:0/1442887778 --> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55998bd64aa0 con 0x55998bebb830 2024-09-10T10:04:39.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:39.215+0000 7efedce00640 1 --2- 172.21.15.5:0/1442887778 >> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] conn(0x55998bebb830 0x55998bebdc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:39.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:39.215+0000 7efedce00640 1 --2- 172.21.15.5:0/1442887778 >> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] conn(0x55998bebb830 0x55998bebdc20 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=1e53ac4d955cde5 server_cookie=8c58f77c185e5959 in_seq=0 out_seq=0 2024-09-10T10:04:39.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:39.215+0000 7efebf400640 10 client.30301 ms_handle_connect on v2:172.21.15.102:6832/2457007427 2024-09-10T10:04:39.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:39.216+0000 7efebf400640 1 -- 172.21.15.5:0/1442887778 <== mds.4 v2:172.21.15.102:6832/2457007427 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55998bd64aa0 con 0x55998bebb830 2024-09-10T10:04:39.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:39.216+0000 7efebf400640 10 client.30301 handle_command_reply: tid=0 2024-09-10T10:04:39.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:39.216+0000 7efee2c26740 1 client.30301 shutdown 2024-09-10T10:04:39.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:39.216+0000 7efee2c26740 1 -- 172.21.15.5:0/1442887778 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55998be2ce40 msgr2=0x55998be3ba40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:39.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:39.216+0000 7efee2c26740 1 --2- 172.21.15.5:0/1442887778 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55998be2ce40 0x55998be3ba40 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7efecc0bc030 tx=0x7efecc009260 comp rx=0 tx=0).stop 2024-09-10T10:04:39.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:39.216+0000 7efee2c26740 1 -- 172.21.15.5:0/1442887778 shutdown_connections 2024-09-10T10:04:39.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:39.216+0000 7efee2c26740 1 --2- 172.21.15.5:0/1442887778 >> [v2:172.21.15.102:6832/2457007427,v1:172.21.15.102:6833/2457007427] conn(0x55998bebb830 0x55998bebdc20 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:39.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:39.216+0000 7efee2c26740 1 --2- 172.21.15.5:0/1442887778 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55998be38020 0x55998be3c4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:39.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:39.216+0000 7efee2c26740 1 --2- 172.21.15.5:0/1442887778 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55998be2d760 0x55998be3bf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:39.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:39.216+0000 7efee2c26740 1 --2- 172.21.15.5:0/1442887778 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55998be2ce40 0x55998be3ba40 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:39.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:39.216+0000 7efee2c26740 1 -- 172.21.15.5:0/1442887778 >> 172.21.15.5:0/1442887778 conn(0x55998be293d0 msgr2=0x55998be27c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:39.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:39.216+0000 7efebf400640 0 client.30301 ms_handle_reset on v2:172.21.15.102:6832/2457007427 2024-09-10T10:04:39.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:39.216+0000 7efee2c26740 1 -- 172.21.15.5:0/1442887778 shutdown_connections 2024-09-10T10:04:39.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:39.216+0000 7efee2c26740 1 -- 172.21.15.5:0/1442887778 wait complete. 2024-09-10T10:04:39.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:39.217+0000 7efee2c26740 20 client.30301 trim_cache size 0 max 16384 2024-09-10T10:04:39.220 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-10T10:04:39.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:39.217+0000 7efedec00640 1 -- 172.21.15.5:0/3514870853 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7efec00a2c00 msgr2=0x7efec00a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:39.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:39.217+0000 7efedec00640 1 --2- 172.21.15.5:0/3514870853 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7efec00a2c00 0x7efec00a50c0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7efed0005fd0 tx=0x7efed0009a90 comp rx=0 tx=0).stop 2024-09-10T10:04:39.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:39.217+0000 7efedec00640 1 -- 172.21.15.5:0/3514870853 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efed81019c0 msgr2=0x7efed80744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:39.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:39.217+0000 7efedec00640 1 --2- 172.21.15.5:0/3514870853 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efed81019c0 0x7efed80744f0 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7efecc009230 tx=0x7efecc004270 comp rx=0 tx=0).stop 2024-09-10T10:04:39.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:39.217+0000 7efedec00640 1 -- 172.21.15.5:0/3514870853 shutdown_connections 2024-09-10T10:04:39.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:39.217+0000 7efedec00640 1 --2- 172.21.15.5:0/3514870853 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7efec00a2c00 0x7efec00a50c0 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:39.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:39.217+0000 7efedec00640 1 --2- 172.21.15.5:0/3514870853 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7efed81079c0 0x7efed8074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:39.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:39.217+0000 7efedec00640 1 --2- 172.21.15.5:0/3514870853 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7efed8102360 0x7efed8074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:39.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:39.217+0000 7efedec00640 1 --2- 172.21.15.5:0/3514870853 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efed81019c0 0x7efed80744f0 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:39.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:39.217+0000 7efedec00640 1 -- 172.21.15.5:0/3514870853 >> 172.21.15.5:0/3514870853 conn(0x7efed80fd740 msgr2=0x7efed80fe060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:39.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:39.218+0000 7efedec00640 1 -- 172.21.15.5:0/3514870853 shutdown_connections 2024-09-10T10:04:39.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:39.218+0000 7efedec00640 1 -- 172.21.15.5:0/3514870853 wait complete. 2024-09-10T10:04:40.017 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-10T10:04:40.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:04:39 smithi005 ceph-mon[30350]: pgmap v1182: 129 pgs: 129 active+clean; 587 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 569 KiB/s rd, 4.4 MiB/s wr, 233 op/s 2024-09-10T10:04:40.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:04:39 smithi159 ceph-mon[35513]: pgmap v1182: 129 pgs: 129 active+clean; 587 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 569 KiB/s rd, 4.4 MiB/s wr, 233 op/s 2024-09-10T10:04:40.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:04:39 smithi102 ceph-mon[35442]: pgmap v1182: 129 pgs: 129 active+clean; 587 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 569 KiB/s rd, 4.4 MiB/s wr, 233 op/s 2024-09-10T10:04:40.434 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:04:41.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:41.113+0000 7f8883600640 1 -- 172.21.15.5:0/2089911457 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f887c077040 msgr2=0x7f887c1124f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:41.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:41.113+0000 7f8883600640 1 --2- 172.21.15.5:0/2089911457 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f887c077040 0x7f887c1124f0 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f8864009900 tx=0x7f886402a410 comp rx=0 tx=0).stop 2024-09-10T10:04:41.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:41.114+0000 7f8883600640 1 -- 172.21.15.5:0/2089911457 shutdown_connections 2024-09-10T10:04:41.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:41.114+0000 7f8883600640 1 --2- 172.21.15.5:0/2089911457 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f887c112a30 0x7f887c114e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:41.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:41.114+0000 7f8883600640 1 --2- 172.21.15.5:0/2089911457 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f887c077040 0x7f887c1124f0 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:41.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:41.114+0000 7f8883600640 1 --2- 172.21.15.5:0/2089911457 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f887c076720 0x7f887c076b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:41.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:41.114+0000 7f8883600640 1 -- 172.21.15.5:0/2089911457 >> 172.21.15.5:0/2089911457 conn(0x7f887c0ffa90 msgr2=0x7f887c101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:41.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:41.114+0000 7f8883600640 1 -- 172.21.15.5:0/2089911457 shutdown_connections 2024-09-10T10:04:41.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:41.114+0000 7f8883600640 1 -- 172.21.15.5:0/2089911457 wait complete. 2024-09-10T10:04:41.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:41.115+0000 7f8883600640 1 Processor -- start 2024-09-10T10:04:41.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:41.115+0000 7f8883600640 1 -- start start 2024-09-10T10:04:41.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:41.115+0000 7f8883600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f887c076720 0x7f887c1a46c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:41.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:41.115+0000 7f8883600640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f887c077040 0x7f887c1a4c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:41.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:41.115+0000 7f8883600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f887c112a30 0x7f887c19e880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:41.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:41.115+0000 7f8883600640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f887c1185f0 con 0x7f887c076720 2024-09-10T10:04:41.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:41.115+0000 7f8883600640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f887c118470 con 0x7f887c112a30 2024-09-10T10:04:41.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:41.115+0000 7f8883600640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f887c118770 con 0x7f887c077040 2024-09-10T10:04:41.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:41.115+0000 7f8880c00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f887c076720 0x7f887c1a46c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:41.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:41.115+0000 7f8880c00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f887c076720 0x7f887c1a46c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40242/0 (socket says 172.21.15.5:40242) 2024-09-10T10:04:41.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:41.115+0000 7f8880c00640 1 -- 172.21.15.5:0/1055480692 learned_addr learned my addr 172.21.15.5:0/1055480692 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T10:04:41.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:41.116+0000 7f8873e00640 1 --2- 172.21.15.5:0/1055480692 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f887c077040 0x7f887c1a4c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:41.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:41.116+0000 7f8881600640 1 --2- 172.21.15.5:0/1055480692 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f887c112a30 0x7f887c19e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:41.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:41.116+0000 7f8880c00640 1 -- 172.21.15.5:0/1055480692 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f887c077040 msgr2=0x7f887c1a4c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:41.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:41.116+0000 7f8880c00640 1 --2- 172.21.15.5:0/1055480692 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f887c077040 0x7f887c1a4c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:41.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:41.116+0000 7f8880c00640 1 -- 172.21.15.5:0/1055480692 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f887c112a30 msgr2=0x7f887c19e880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:41.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:41.116+0000 7f8880c00640 1 --2- 172.21.15.5:0/1055480692 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f887c112a30 0x7f887c19e880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:41.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:41.116+0000 7f8880c00640 1 -- 172.21.15.5:0/1055480692 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f887c19efb0 con 0x7f887c076720 2024-09-10T10:04:41.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:41.116+0000 7f8880c00640 1 --2- 172.21.15.5:0/1055480692 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f887c076720 0x7f887c1a46c0 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f886c00e9f0 tx=0x7f886c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:41.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:41.116+0000 7f887a000640 1 -- 172.21.15.5:0/1055480692 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f886c00cd80 con 0x7f887c076720 2024-09-10T10:04:41.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:41.116+0000 7f887a000640 1 -- 172.21.15.5:0/1055480692 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f886c010910 con 0x7f887c076720 2024-09-10T10:04:41.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:41.117+0000 7f8883600640 1 -- 172.21.15.5:0/1055480692 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f887c19f2a0 con 0x7f887c076720 2024-09-10T10:04:41.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:41.117+0000 7f887a000640 1 -- 172.21.15.5:0/1055480692 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f886c018990 con 0x7f887c076720 2024-09-10T10:04:41.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:41.117+0000 7f8883600640 1 -- 172.21.15.5:0/1055480692 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f887c1ab590 con 0x7f887c076720 2024-09-10T10:04:41.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:41.120+0000 7f887a000640 1 -- 172.21.15.5:0/1055480692 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f886c010ac0 con 0x7f887c076720 2024-09-10T10:04:41.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:41.120+0000 7f8886069740 1 Processor -- start 2024-09-10T10:04:41.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:41.120+0000 7f8886069740 1 -- start start 2024-09-10T10:04:41.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:41.120+0000 7f8886069740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bb3bf00c70 0x55bb3bf01050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:41.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:41.120+0000 7f8886069740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bb3bf01590 0x55bb3bf0b910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:41.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:41.120+0000 7f8886069740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bb3bf0be50 0x55bb3bf0e240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:41.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:41.120+0000 7f8886069740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55bb3be673e0 con 0x55bb3bf00c70 2024-09-10T10:04:41.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:41.120+0000 7f8886069740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55bb3be0ed80 con 0x55bb3bf0be50 2024-09-10T10:04:41.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:41.120+0000 7f8886069740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55bb3be633a0 con 0x55bb3bf01590 2024-09-10T10:04:41.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:41.121+0000 7f8881600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bb3bf00c70 0x55bb3bf01050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:41.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:41.121+0000 7f8881600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bb3bf00c70 0x55bb3bf01050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40244/0 (socket says 172.21.15.5:40244) 2024-09-10T10:04:41.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:41.121+0000 7f8881600640 1 -- 172.21.15.5:0/3687749645 learned_addr learned my addr 172.21.15.5:0/3687749645 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T10:04:41.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:41.121+0000 7f8880c00640 1 --2- 172.21.15.5:0/3687749645 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bb3bf01590 0x55bb3bf0b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:41.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:41.121+0000 7f8873e00640 1 --2- 172.21.15.5:0/3687749645 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bb3bf0be50 0x55bb3bf0e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:41.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:41.121+0000 7f8881600640 1 -- 172.21.15.5:0/3687749645 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bb3bf01590 msgr2=0x55bb3bf0b910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:41.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:41.121+0000 7f8881600640 1 --2- 172.21.15.5:0/3687749645 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bb3bf01590 0x55bb3bf0b910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:41.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:41.121+0000 7f8881600640 1 -- 172.21.15.5:0/3687749645 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bb3bf0be50 msgr2=0x55bb3bf0e240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:41.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:41.121+0000 7f8881600640 1 --2- 172.21.15.5:0/3687749645 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bb3bf0be50 0x55bb3bf0e240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:41.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:41.121+0000 7f8881600640 1 -- 172.21.15.5:0/3687749645 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bb3be2dae0 con 0x55bb3bf00c70 2024-09-10T10:04:41.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:41.121+0000 7f887a000640 1 --2- 172.21.15.5:0/1055480692 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f88600a2b80 0x7f88600a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:41.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:41.121+0000 7f887a000640 1 -- 172.21.15.5:0/1055480692 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f886c0cf7f0 con 0x7f887c076720 2024-09-10T10:04:41.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:41.122+0000 7f8880c00640 1 --2- 172.21.15.5:0/1055480692 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f88600a2b80 0x7f88600a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:41.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:41.122+0000 7f8881600640 1 --2- 172.21.15.5:0/3687749645 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bb3bf00c70 0x55bb3bf01050 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f8874009ec0 tx=0x7f88740107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7fffb192d7f6bcd server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:41.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:41.122+0000 7f8871c00640 1 -- 172.21.15.5:0/3687749645 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8874004d20 con 0x55bb3bf00c70 2024-09-10T10:04:41.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:41.122+0000 7f8871c00640 1 -- 172.21.15.5:0/3687749645 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8874003e50 con 0x55bb3bf00c70 2024-09-10T10:04:41.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:41.122+0000 7f8871c00640 1 -- 172.21.15.5:0/3687749645 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8874005280 con 0x55bb3bf00c70 2024-09-10T10:04:41.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:41.123+0000 7f8886069740 1 -- 172.21.15.5:0/3687749645 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bb3bf00c70 msgr2=0x55bb3bf01050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:41.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:41.123+0000 7f8880c00640 1 --2- 172.21.15.5:0/1055480692 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f88600a2b80 0x7f88600a5040 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f886c0cd270 tx=0x7f886c0d5550 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:41.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:41.123+0000 7f8886069740 1 --2- 172.21.15.5:0/3687749645 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bb3bf00c70 0x55bb3bf01050 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f8874009ec0 tx=0x7f88740107a0 comp rx=0 tx=0).stop 2024-09-10T10:04:41.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:41.123+0000 7f8886069740 1 -- 172.21.15.5:0/3687749645 shutdown_connections 2024-09-10T10:04:41.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:41.123+0000 7f8886069740 1 --2- 172.21.15.5:0/3687749645 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bb3bf0be50 0x55bb3bf0e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:41.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:41.123+0000 7f8886069740 1 --2- 172.21.15.5:0/3687749645 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bb3bf01590 0x55bb3bf0b910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:41.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:41.123+0000 7f8886069740 1 --2- 172.21.15.5:0/3687749645 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bb3bf00c70 0x55bb3bf01050 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:41.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:41.123+0000 7f8886069740 1 -- 172.21.15.5:0/3687749645 >> 172.21.15.5:0/3687749645 conn(0x55bb3befd200 msgr2=0x55bb3befd5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:41.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:41.123+0000 7f8886069740 1 -- 172.21.15.5:0/3687749645 shutdown_connections 2024-09-10T10:04:41.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:41.123+0000 7f8886069740 1 -- 172.21.15.5:0/3687749645 wait complete. 2024-09-10T10:04:41.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:41.123+0000 7f8886069740 1 -- start start 2024-09-10T10:04:41.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:41.124+0000 7f8886069740 1 Processor -- start 2024-09-10T10:04:41.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:41.124+0000 7f8886069740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bb3bf00c70 0x55bb3bf08cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:41.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:41.124+0000 7f8886069740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bb3bf01590 0x55bb3bf09230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:41.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:41.124+0000 7f8886069740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bb3bf0be50 0x55bb3bf09770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:41.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:41.124+0000 7f8886069740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55bb3bf0ea20 con 0x55bb3bf01590 2024-09-10T10:04:41.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:41.124+0000 7f8886069740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55bb3be220f0 con 0x55bb3bf00c70 2024-09-10T10:04:41.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:41.124+0000 7f8886069740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55bb3bf0eba0 con 0x55bb3bf0be50 2024-09-10T10:04:41.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:41.124+0000 7f8886069740 10 client.0 _ll_register_callbacks cb 0x55bb3be09bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:04:41.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:41.124+0000 7f8873e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bb3bf01590 0x55bb3bf09230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:41.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:41.124+0000 7f8873e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bb3bf01590 0x55bb3bf09230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40256/0 (socket says 172.21.15.5:40256) 2024-09-10T10:04:41.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:41.124+0000 7f8873e00640 1 -- 172.21.15.5:0/324726319 learned_addr learned my addr 172.21.15.5:0/324726319 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T10:04:41.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:41.124+0000 7f8881600640 1 --2- 172.21.15.5:0/324726319 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bb3bf00c70 0x55bb3bf08cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:41.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:41.125+0000 7f8873e00640 1 -- 172.21.15.5:0/324726319 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bb3bf0be50 msgr2=0x55bb3bf09770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:41.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:41.125+0000 7f8881600640 1 --2- 172.21.15.5:0/324726319 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bb3bf0be50 0x55bb3bf09770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:41.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:41.125+0000 7f8873e00640 1 --2- 172.21.15.5:0/324726319 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bb3bf0be50 0x55bb3bf09770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:41.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:41.125+0000 7f8873e00640 1 -- 172.21.15.5:0/324726319 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bb3bf00c70 msgr2=0x55bb3bf08cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:41.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:41.125+0000 7f8873e00640 1 --2- 172.21.15.5:0/324726319 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bb3bf00c70 0x55bb3bf08cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:41.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:41.125+0000 7f8873e00640 1 -- 172.21.15.5:0/324726319 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bb3bbd83c0 con 0x55bb3bf01590 2024-09-10T10:04:41.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:41.125+0000 7f8881600640 1 --2- 172.21.15.5:0/324726319 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bb3bf0be50 0x55bb3bf09770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T10:04:41.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:41.125+0000 7f8873e00640 1 --2- 172.21.15.5:0/324726319 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bb3bf01590 0x55bb3bf09230 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f8864033000 tx=0x7f8864004440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce7edac71aa276d3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:41.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:41.125+0000 7f886a000640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-10T10:04:41.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:41.125+0000 7f886a000640 1 -- 172.21.15.5:0/324726319 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8864035040 con 0x55bb3bf01590 2024-09-10T10:04:41.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:41.125+0000 7f8886069740 1 -- 172.21.15.5:0/324726319 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8874003e50 con 0x55bb3bf01590 2024-09-10T10:04:41.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:41.125+0000 7f886a000640 1 -- 172.21.15.5:0/324726319 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f8864005510 con 0x55bb3bf01590 2024-09-10T10:04:41.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:41.126+0000 7f886a000640 1 -- 172.21.15.5:0/324726319 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f886404f050 con 0x55bb3bf01590 2024-09-10T10:04:41.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:41.126+0000 7f8886069740 10 client.20241 fetch_fsmap learned FSMap version 47 2024-09-10T10:04:41.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:41.126+0000 7f8886069740 1 -- 172.21.15.5:0/324726319 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x55bb3bf1aa60 con 0x55bb3bf01590 2024-09-10T10:04:41.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:41.126+0000 7f886a000640 1 -- 172.21.15.5:0/324726319 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8874003e50 con 0x55bb3bf01590 2024-09-10T10:04:41.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:41.126+0000 7f886a000640 1 -- 172.21.15.5:0/324726319 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 47) ==== 8398+0+0 (secure 0 0 0) 0x7f8864031950 con 0x55bb3bf01590 2024-09-10T10:04:41.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:41.126+0000 7f8886069740 10 client.20241 fetch_fsmap finished waiting for FSMap version 47 2024-09-10T10:04:41.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:41.126+0000 7f8886069740 10 client.20241 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-09-10T10:04:41.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:41.126+0000 7f8886069740 20 client.20241 populate_metadata read hostname 'smithi005' 2024-09-10T10:04:41.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:41.126+0000 7f8886069740 1 --2- 172.21.15.5:0/324726319 >> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] conn(0x55bb3bf1be70 0x55bb3bf1e310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:41.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:41.126+0000 7f8886069740 4 client.20241 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:04:41.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:41.126+0000 7f8886069740 1 -- 172.21.15.5:0/324726319 --> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bb3be74950 con 0x55bb3bf1be70 2024-09-10T10:04:41.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:41.127+0000 7f8881600640 1 --2- 172.21.15.5:0/324726319 >> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] conn(0x55bb3bf1be70 0x55bb3bf1e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:41.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:41.127+0000 7f8881600640 1 --2- 172.21.15.5:0/324726319 >> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] conn(0x55bb3bf1be70 0x55bb3bf1e310 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=51d232c736bfdd0c server_cookie=2c9b523edffcb444 in_seq=0 out_seq=0 2024-09-10T10:04:41.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:41.127+0000 7f886a000640 10 client.20241 ms_handle_connect on v2:172.21.15.102:6836/2689776764 2024-09-10T10:04:41.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:41.128+0000 7f886a000640 1 -- 172.21.15.5:0/324726319 <== mds.3 v2:172.21.15.102:6836/2689776764 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55bb3be74950 con 0x55bb3bf1be70 2024-09-10T10:04:41.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:41.128+0000 7f886a000640 10 client.20241 handle_command_reply: tid=0 2024-09-10T10:04:41.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:41.128+0000 7f8886069740 1 client.20241 shutdown 2024-09-10T10:04:41.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:41.128+0000 7f8886069740 1 -- 172.21.15.5:0/324726319 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bb3bf01590 msgr2=0x55bb3bf09230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:41.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:41.128+0000 7f8886069740 1 --2- 172.21.15.5:0/324726319 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bb3bf01590 0x55bb3bf09230 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f8864033000 tx=0x7f8864004440 comp rx=0 tx=0).stop 2024-09-10T10:04:41.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:41.128+0000 7f8886069740 1 -- 172.21.15.5:0/324726319 shutdown_connections 2024-09-10T10:04:41.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:41.128+0000 7f8886069740 1 --2- 172.21.15.5:0/324726319 >> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] conn(0x55bb3bf1be70 0x55bb3bf1e310 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:41.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:41.128+0000 7f8886069740 1 --2- 172.21.15.5:0/324726319 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bb3bf0be50 0x55bb3bf09770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:41.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:41.128+0000 7f8886069740 1 --2- 172.21.15.5:0/324726319 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bb3bf01590 0x55bb3bf09230 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:41.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:41.128+0000 7f8886069740 1 --2- 172.21.15.5:0/324726319 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bb3bf00c70 0x55bb3bf08cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:41.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:41.128+0000 7f8886069740 1 -- 172.21.15.5:0/324726319 >> 172.21.15.5:0/324726319 conn(0x55bb3befd200 msgr2=0x55bb3befc380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:41.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:41.128+0000 7f886a000640 0 client.20241 ms_handle_reset on v2:172.21.15.102:6836/2689776764 2024-09-10T10:04:41.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:41.129+0000 7f8886069740 1 -- 172.21.15.5:0/324726319 shutdown_connections 2024-09-10T10:04:41.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:41.129+0000 7f8886069740 1 -- 172.21.15.5:0/324726319 wait complete. 2024-09-10T10:04:41.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:41.129+0000 7f8886069740 20 client.20241 trim_cache size 0 max 16384 2024-09-10T10:04:41.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:41.142+0000 7f8886069740 1 Processor -- start 2024-09-10T10:04:41.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:41.143+0000 7f8886069740 1 -- start start 2024-09-10T10:04:41.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:41.143+0000 7f8886069740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bb3bf00c70 0x55bb3bf0ac60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:41.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:41.143+0000 7f8886069740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bb3bf01590 0x55bb3bf23c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:41.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:41.143+0000 7f8886069740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bb3bf0be50 0x55bb3bf24160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:41.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:41.143+0000 7f8886069740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f887c1b5450 con 0x55bb3bf01590 2024-09-10T10:04:41.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:41.143+0000 7f8886069740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f887c1b7050 con 0x55bb3bf0be50 2024-09-10T10:04:41.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:41.143+0000 7f8886069740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f887c1b5b20 con 0x55bb3bf00c70 2024-09-10T10:04:41.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:41.143+0000 7f8873e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bb3bf01590 0x55bb3bf23c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:41.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:41.143+0000 7f8873e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bb3bf01590 0x55bb3bf23c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40272/0 (socket says 172.21.15.5:40272) 2024-09-10T10:04:41.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:41.143+0000 7f8873e00640 1 -- 172.21.15.5:0/3352179935 learned_addr learned my addr 172.21.15.5:0/3352179935 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T10:04:41.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:41.143+0000 7f8881600640 1 --2- 172.21.15.5:0/3352179935 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bb3bf00c70 0x55bb3bf0ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:41.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:41.143+0000 7f8881600640 1 --2- 172.21.15.5:0/3352179935 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bb3bf0be50 0x55bb3bf24160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:41.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:41.143+0000 7f8873e00640 1 -- 172.21.15.5:0/3352179935 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bb3bf00c70 msgr2=0x55bb3bf0ac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:41.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:41.143+0000 7f8873e00640 1 --2- 172.21.15.5:0/3352179935 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bb3bf00c70 0x55bb3bf0ac60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:41.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:41.143+0000 7f8873e00640 1 -- 172.21.15.5:0/3352179935 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bb3bf0be50 msgr2=0x55bb3bf24160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:41.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:41.143+0000 7f8873e00640 1 --2- 172.21.15.5:0/3352179935 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bb3bf0be50 0x55bb3bf24160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:41.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:41.144+0000 7f8873e00640 1 -- 172.21.15.5:0/3352179935 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bb3bef8770 con 0x55bb3bf01590 2024-09-10T10:04:41.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:41.144+0000 7f8881600640 1 --2- 172.21.15.5:0/3352179935 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bb3bf00c70 0x55bb3bf0ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:04:41.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:41.144+0000 7f8873e00640 1 --2- 172.21.15.5:0/3352179935 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bb3bf01590 0x55bb3bf23c20 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f8864009900 tx=0x7f886402a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=376b76208283830a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:41.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:41.144+0000 7f8871200640 1 -- 172.21.15.5:0/3352179935 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8864035030 con 0x55bb3bf01590 2024-09-10T10:04:41.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:41.144+0000 7f8871200640 1 -- 172.21.15.5:0/3352179935 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8864041070 con 0x55bb3bf01590 2024-09-10T10:04:41.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:41.144+0000 7f8871200640 1 -- 172.21.15.5:0/3352179935 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8864003b20 con 0x55bb3bf01590 2024-09-10T10:04:41.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:41.145+0000 7f8886069740 1 -- 172.21.15.5:0/3352179935 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bb3bf01590 msgr2=0x55bb3bf23c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:41.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:41.145+0000 7f8886069740 1 --2- 172.21.15.5:0/3352179935 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bb3bf01590 0x55bb3bf23c20 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f8864009900 tx=0x7f886402a410 comp rx=0 tx=0).stop 2024-09-10T10:04:41.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:41.145+0000 7f8886069740 1 -- 172.21.15.5:0/3352179935 shutdown_connections 2024-09-10T10:04:41.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:41.145+0000 7f8886069740 1 --2- 172.21.15.5:0/3352179935 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bb3bf0be50 0x55bb3bf24160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:41.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:41.145+0000 7f8886069740 1 --2- 172.21.15.5:0/3352179935 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bb3bf01590 0x55bb3bf23c20 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:41.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:41.145+0000 7f8886069740 1 --2- 172.21.15.5:0/3352179935 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bb3bf00c70 0x55bb3bf0ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:41.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:41.145+0000 7f8886069740 1 -- 172.21.15.5:0/3352179935 >> 172.21.15.5:0/3352179935 conn(0x55bb3befd200 msgr2=0x55bb3bf0a550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:41.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:41.145+0000 7f8886069740 1 -- 172.21.15.5:0/3352179935 shutdown_connections 2024-09-10T10:04:41.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:41.145+0000 7f8886069740 1 -- 172.21.15.5:0/3352179935 wait complete. 2024-09-10T10:04:41.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:41.145+0000 7f8886069740 1 -- start start 2024-09-10T10:04:41.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:41.145+0000 7f8886069740 1 Processor -- start 2024-09-10T10:04:41.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:41.146+0000 7f8886069740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bb3bf00c70 0x55bb3bf0fd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:41.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:41.146+0000 7f8886069740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bb3bf01590 0x55bb3bf10270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:41.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:41.146+0000 7f8886069740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bb3bf0be50 0x55bb3bf107b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:41.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:41.146+0000 7f8886069740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f887c1b6950 con 0x55bb3bf00c70 2024-09-10T10:04:41.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:41.146+0000 7f8886069740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f887c1b6250 con 0x55bb3bf01590 2024-09-10T10:04:41.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:41.146+0000 7f8886069740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f887c1b3430 con 0x55bb3bf0be50 2024-09-10T10:04:41.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:41.146+0000 7f8886069740 10 client.0 _ll_register_callbacks cb 0x7f8864009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:04:41.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:41.146+0000 7f8881600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bb3bf00c70 0x55bb3bf0fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:41.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:41.146+0000 7f8881600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bb3bf00c70 0x55bb3bf0fd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40274/0 (socket says 172.21.15.5:40274) 2024-09-10T10:04:41.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:41.146+0000 7f8881600640 1 -- 172.21.15.5:0/2463992756 learned_addr learned my addr 172.21.15.5:0/2463992756 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T10:04:41.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:41.146+0000 7f8873e00640 1 --2- 172.21.15.5:0/2463992756 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bb3bf01590 0x55bb3bf10270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:41.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:41.146+0000 7f8881600640 1 -- 172.21.15.5:0/2463992756 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bb3bf0be50 msgr2=0x55bb3bf107b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:41.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:41.146+0000 7f8881600640 1 --2- 172.21.15.5:0/2463992756 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bb3bf0be50 0x55bb3bf107b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:41.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:41.146+0000 7f8881600640 1 -- 172.21.15.5:0/2463992756 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bb3bf01590 msgr2=0x55bb3bf10270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:41.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:41.146+0000 7f8881600640 1 --2- 172.21.15.5:0/2463992756 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bb3bf01590 0x55bb3bf10270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:41.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:41.146+0000 7f8881600640 1 -- 172.21.15.5:0/2463992756 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bb3bf03ae0 con 0x55bb3bf00c70 2024-09-10T10:04:41.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:41.147+0000 7f8881600640 1 --2- 172.21.15.5:0/2463992756 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bb3bf00c70 0x55bb3bf0fd30 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f8874004630 tx=0x7f8874005bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7db016fbf0ab6203 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:41.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:41.147+0000 7f886a000640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-10T10:04:41.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:41.147+0000 7f886a000640 1 -- 172.21.15.5:0/2463992756 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f887402c070 con 0x55bb3bf00c70 2024-09-10T10:04:41.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:41.147+0000 7f886a000640 1 -- 172.21.15.5:0/2463992756 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f8874003970 con 0x55bb3bf00c70 2024-09-10T10:04:41.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:41.147+0000 7f8886069740 1 -- 172.21.15.5:0/2463992756 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8864041070 con 0x55bb3bf00c70 2024-09-10T10:04:41.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:41.147+0000 7f886a000640 1 -- 172.21.15.5:0/2463992756 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f8874010cd0 con 0x55bb3bf00c70 2024-09-10T10:04:41.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:41.147+0000 7f8886069740 10 client.20247 fetch_fsmap learned FSMap version 47 2024-09-10T10:04:41.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:41.147+0000 7f8886069740 1 -- 172.21.15.5:0/2463992756 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x55bb3bf86e90 con 0x55bb3bf00c70 2024-09-10T10:04:41.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:41.148+0000 7f886a000640 1 -- 172.21.15.5:0/2463992756 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8864041070 con 0x55bb3bf00c70 2024-09-10T10:04:41.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:41.148+0000 7f886a000640 1 -- 172.21.15.5:0/2463992756 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 47) ==== 8398+0+0 (secure 0 0 0) 0x7f8874028780 con 0x55bb3bf00c70 2024-09-10T10:04:41.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:41.148+0000 7f8886069740 10 client.20247 fetch_fsmap finished waiting for FSMap version 47 2024-09-10T10:04:41.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:41.148+0000 7f8886069740 10 client.20247 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.h 2024-09-10T10:04:41.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:41.148+0000 7f8886069740 20 client.20247 populate_metadata read hostname 'smithi005' 2024-09-10T10:04:41.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:41.148+0000 7f8886069740 1 --2- 172.21.15.5:0/2463992756 >> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] conn(0x55bb3bf87450 0x55bb3bf89870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:41.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:41.148+0000 7f8886069740 4 client.20247 mds_command: new command op to 24506 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T10:04:41.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:41.148+0000 7f8886069740 1 -- 172.21.15.5:0/2463992756 --> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55bb3be6ee60 con 0x55bb3bf87450 2024-09-10T10:04:41.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:41.148+0000 7f8873e00640 1 --2- 172.21.15.5:0/2463992756 >> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] conn(0x55bb3bf87450 0x55bb3bf89870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:41.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:41.149+0000 7f8873e00640 1 --2- 172.21.15.5:0/2463992756 >> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] conn(0x55bb3bf87450 0x55bb3bf89870 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=49784ac44b093a9d server_cookie=44af5b989690d0a2 in_seq=0 out_seq=0 2024-09-10T10:04:41.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:41.149+0000 7f886a000640 10 client.20247 ms_handle_connect on v2:172.21.15.102:6836/2689776764 2024-09-10T10:04:41.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:41.149+0000 7f886a000640 1 -- 172.21.15.5:0/2463992756 <== mds.3 v2:172.21.15.102:6836/2689776764 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55bb3be6ee60 con 0x55bb3bf87450 2024-09-10T10:04:41.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:41.149+0000 7f886a000640 10 client.20247 handle_command_reply: tid=0 2024-09-10T10:04:41.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:41.149+0000 7f8886069740 1 client.20247 shutdown 2024-09-10T10:04:41.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:41.149+0000 7f8886069740 1 -- 172.21.15.5:0/2463992756 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bb3bf00c70 msgr2=0x55bb3bf0fd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:41.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:41.149+0000 7f8886069740 1 --2- 172.21.15.5:0/2463992756 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bb3bf00c70 0x55bb3bf0fd30 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f8874004630 tx=0x7f8874005bd0 comp rx=0 tx=0).stop 2024-09-10T10:04:41.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:41.149+0000 7f8886069740 1 -- 172.21.15.5:0/2463992756 shutdown_connections 2024-09-10T10:04:41.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:41.149+0000 7f8886069740 1 --2- 172.21.15.5:0/2463992756 >> [v2:172.21.15.102:6836/2689776764,v1:172.21.15.102:6837/2689776764] conn(0x55bb3bf87450 0x55bb3bf89870 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:41.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:41.149+0000 7f8886069740 1 --2- 172.21.15.5:0/2463992756 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55bb3bf0be50 0x55bb3bf107b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:41.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:41.149+0000 7f8886069740 1 --2- 172.21.15.5:0/2463992756 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55bb3bf01590 0x55bb3bf10270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:41.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:41.149+0000 7f8886069740 1 --2- 172.21.15.5:0/2463992756 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bb3bf00c70 0x55bb3bf0fd30 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:41.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:41.149+0000 7f8886069740 1 -- 172.21.15.5:0/2463992756 >> 172.21.15.5:0/2463992756 conn(0x55bb3befd200 msgr2=0x55bb3bf0ad70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:41.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:41.149+0000 7f886a000640 0 client.20247 ms_handle_reset on v2:172.21.15.102:6836/2689776764 2024-09-10T10:04:41.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:41.150+0000 7f8886069740 1 -- 172.21.15.5:0/2463992756 shutdown_connections 2024-09-10T10:04:41.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:41.150+0000 7f8886069740 1 -- 172.21.15.5:0/2463992756 wait complete. 2024-09-10T10:04:41.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:41.150+0000 7f8886069740 20 client.20247 trim_cache size 0 max 16384 2024-09-10T10:04:41.270 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-10T10:04:41.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:41.150+0000 7f8883600640 1 -- 172.21.15.5:0/1055480692 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f88600a2b80 msgr2=0x7f88600a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:41.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:41.150+0000 7f8883600640 1 --2- 172.21.15.5:0/1055480692 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f88600a2b80 0x7f88600a5040 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f886c0cd270 tx=0x7f886c0d5550 comp rx=0 tx=0).stop 2024-09-10T10:04:41.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:41.150+0000 7f8883600640 1 -- 172.21.15.5:0/1055480692 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f887c076720 msgr2=0x7f887c1a46c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:41.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:41.150+0000 7f8883600640 1 --2- 172.21.15.5:0/1055480692 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f887c076720 0x7f887c1a46c0 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f886c00e9f0 tx=0x7f886c00eeb0 comp rx=0 tx=0).stop 2024-09-10T10:04:41.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:41.150+0000 7f8883600640 1 -- 172.21.15.5:0/1055480692 shutdown_connections 2024-09-10T10:04:41.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:41.150+0000 7f8883600640 1 --2- 172.21.15.5:0/1055480692 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f88600a2b80 0x7f88600a5040 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:41.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:41.150+0000 7f8883600640 1 --2- 172.21.15.5:0/1055480692 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f887c112a30 0x7f887c19e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:41.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:41.150+0000 7f8883600640 1 --2- 172.21.15.5:0/1055480692 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f887c077040 0x7f887c1a4c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:41.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:41.150+0000 7f8883600640 1 --2- 172.21.15.5:0/1055480692 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f887c076720 0x7f887c1a46c0 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:41.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:41.150+0000 7f8883600640 1 -- 172.21.15.5:0/1055480692 >> 172.21.15.5:0/1055480692 conn(0x7f887c0ffa90 msgr2=0x7f887c101e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:41.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:41.151+0000 7f8883600640 1 -- 172.21.15.5:0/1055480692 shutdown_connections 2024-09-10T10:04:41.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:41.151+0000 7f8883600640 1 -- 172.21.15.5:0/1055480692 wait complete. 2024-09-10T10:04:42.008 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:04:41 smithi005 ceph-mon[30350]: pgmap v1183: 129 pgs: 129 active+clean; 566 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 163 KiB/s rd, 4.3 MiB/s wr, 210 op/s 2024-09-10T10:04:42.049 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-10T10:04:42.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:04:41 smithi159 ceph-mon[35513]: pgmap v1183: 129 pgs: 129 active+clean; 566 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 163 KiB/s rd, 4.3 MiB/s wr, 210 op/s 2024-09-10T10:04:42.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:04:41 smithi102 ceph-mon[35442]: pgmap v1183: 129 pgs: 129 active+clean; 566 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 163 KiB/s rd, 4.3 MiB/s wr, 210 op/s 2024-09-10T10:04:42.427 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:04:42.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:42.985+0000 7f9895c00640 1 -- 172.21.15.5:0/228699810 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9890074ee0 msgr2=0x7f989010dd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:42.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:42.985+0000 7f9895c00640 1 --2- 172.21.15.5:0/228699810 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9890074ee0 0x7f989010dd20 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f9878009900 tx=0x7f987802a480 comp rx=0 tx=0).stop 2024-09-10T10:04:42.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:42.985+0000 7f9895c00640 1 -- 172.21.15.5:0/228699810 shutdown_connections 2024-09-10T10:04:42.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:42.985+0000 7f9895c00640 1 --2- 172.21.15.5:0/228699810 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f989010e260 0x7f9890110650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:42.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:42.985+0000 7f9895c00640 1 --2- 172.21.15.5:0/228699810 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9890074ee0 0x7f989010dd20 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:42.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:42.985+0000 7f9895c00640 1 --2- 172.21.15.5:0/228699810 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f98900745c0 0x7f98900749a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:42.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:42.985+0000 7f9895c00640 1 -- 172.21.15.5:0/228699810 >> 172.21.15.5:0/228699810 conn(0x7f98900fd740 msgr2=0x7f98900ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:42.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:42.985+0000 7f9895c00640 1 -- 172.21.15.5:0/228699810 shutdown_connections 2024-09-10T10:04:42.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:42.985+0000 7f9895c00640 1 -- 172.21.15.5:0/228699810 wait complete. 2024-09-10T10:04:42.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:42.986+0000 7f9895c00640 1 Processor -- start 2024-09-10T10:04:42.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:42.986+0000 7f9895c00640 1 -- start start 2024-09-10T10:04:42.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:42.986+0000 7f9895c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f98900745c0 0x7f989019a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:42.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:42.986+0000 7f9895c00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9890074ee0 0x7f989019aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:42.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:42.987+0000 7f9895c00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f989010e260 0x7f989019edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:42.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:42.987+0000 7f9895c00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f9890114000 con 0x7f989010e260 2024-09-10T10:04:42.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:42.987+0000 7f9895c00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f9890113e80 con 0x7f98900745c0 2024-09-10T10:04:42.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:42.987+0000 7f9895c00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f9890114180 con 0x7f9890074ee0 2024-09-10T10:04:42.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:42.987+0000 7f9887e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9890074ee0 0x7f989019aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:42.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:42.987+0000 7f9887e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9890074ee0 0x7f989019aa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:55314/0 (socket says 172.21.15.5:55314) 2024-09-10T10:04:42.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:42.987+0000 7f9895200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f989010e260 0x7f989019edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:42.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:42.987+0000 7f988fe00640 1 --2- 172.21.15.5:0/4005889295 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f98900745c0 0x7f989019a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:42.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:42.987+0000 7f9887e00640 1 -- 172.21.15.5:0/4005889295 learned_addr learned my addr 172.21.15.5:0/4005889295 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T10:04:42.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:42.987+0000 7f9887e00640 1 -- 172.21.15.5:0/4005889295 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f98900745c0 msgr2=0x7f989019a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:42.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:42.987+0000 7f9887e00640 1 --2- 172.21.15.5:0/4005889295 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f98900745c0 0x7f989019a4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:42.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:42.987+0000 7f9887e00640 1 -- 172.21.15.5:0/4005889295 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f989010e260 msgr2=0x7f989019edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:42.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:42.987+0000 7f9887e00640 1 --2- 172.21.15.5:0/4005889295 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f989010e260 0x7f989019edb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:42.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:42.987+0000 7f9887e00640 1 -- 172.21.15.5:0/4005889295 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f989019f490 con 0x7f9890074ee0 2024-09-10T10:04:42.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:42.988+0000 7f9887e00640 1 --2- 172.21.15.5:0/4005889295 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9890074ee0 0x7f989019aa20 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f9878004770 tx=0x7f9878003c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:42.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:42.988+0000 7f988d600640 1 -- 172.21.15.5:0/4005889295 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f987803f070 con 0x7f9890074ee0 2024-09-10T10:04:42.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:42.988+0000 7f9895c00640 1 -- 172.21.15.5:0/4005889295 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f989019f720 con 0x7f9890074ee0 2024-09-10T10:04:42.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:42.988+0000 7f9895c00640 1 -- 172.21.15.5:0/4005889295 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98901a7060 con 0x7f9890074ee0 2024-09-10T10:04:42.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:42.988+0000 7f988d600640 1 -- 172.21.15.5:0/4005889295 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9878031ca0 con 0x7f9890074ee0 2024-09-10T10:04:42.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:42.988+0000 7f988d600640 1 -- 172.21.15.5:0/4005889295 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9878049060 con 0x7f9890074ee0 2024-09-10T10:04:42.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:42.991+0000 7f9899e40740 1 Processor -- start 2024-09-10T10:04:42.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:42.991+0000 7f9899e40740 1 -- start start 2024-09-10T10:04:42.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:42.991+0000 7f988d600640 1 -- 172.21.15.5:0/4005889295 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f9878042070 con 0x7f9890074ee0 2024-09-10T10:04:42.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:42.991+0000 7f9899e40740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558ef16a9e40 0x558ef16aa220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:42.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:42.992+0000 7f9899e40740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558ef16aa760 0x558ef16b4ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:42.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:42.992+0000 7f9899e40740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558ef16b5020 0x558ef16b7410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:42.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:42.992+0000 7f9899e40740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x558ef16103e0 con 0x558ef16aa760 2024-09-10T10:04:42.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:42.992+0000 7f9899e40740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x558ef15b7d80 con 0x558ef16a9e40 2024-09-10T10:04:42.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:42.992+0000 7f9899e40740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x558ef160c3a0 con 0x558ef16b5020 2024-09-10T10:04:42.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:42.992+0000 7f988fe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558ef16aa760 0x558ef16b4ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:42.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:42.992+0000 7f9895200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558ef16a9e40 0x558ef16aa220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:43.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:42.992+0000 7f9887e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558ef16b5020 0x558ef16b7410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:43.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:42.992+0000 7f988fe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558ef16aa760 0x558ef16b4ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40296/0 (socket says 172.21.15.5:40296) 2024-09-10T10:04:43.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:42.992+0000 7f988fe00640 1 -- 172.21.15.5:0/3555617276 learned_addr learned my addr 172.21.15.5:0/3555617276 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T10:04:43.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:42.992+0000 7f988fe00640 1 -- 172.21.15.5:0/3555617276 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558ef16b5020 msgr2=0x558ef16b7410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:43.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:42.992+0000 7f988fe00640 1 --2- 172.21.15.5:0/3555617276 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558ef16b5020 0x558ef16b7410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:43.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:42.992+0000 7f988fe00640 1 -- 172.21.15.5:0/3555617276 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558ef16a9e40 msgr2=0x558ef16aa220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:43.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:42.992+0000 7f988fe00640 1 --2- 172.21.15.5:0/3555617276 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558ef16a9e40 0x558ef16aa220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:43.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:42.992+0000 7f988fe00640 1 -- 172.21.15.5:0/3555617276 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558ef15d6ae0 con 0x558ef16aa760 2024-09-10T10:04:43.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:42.992+0000 7f9895200640 1 --2- 172.21.15.5:0/3555617276 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558ef16a9e40 0x558ef16aa220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:04:43.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:42.993+0000 7f988fe00640 1 --2- 172.21.15.5:0/3555617276 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558ef16aa760 0x558ef16b4ae0 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f988000bd10 tx=0x7f98800107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5090d0e164f4aa5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:43.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:42.993+0000 7f9885600640 1 -- 172.21.15.5:0/3555617276 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9880004030 con 0x558ef16aa760 2024-09-10T10:04:43.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:42.993+0000 7f9885600640 1 -- 172.21.15.5:0/3555617276 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f98800041d0 con 0x558ef16aa760 2024-09-10T10:04:43.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:42.993+0000 7f9885600640 1 -- 172.21.15.5:0/3555617276 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9880011840 con 0x558ef16aa760 2024-09-10T10:04:43.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:42.993+0000 7f988d600640 1 --2- 172.21.15.5:0/4005889295 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f98740a2b80 0x7f98740a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:43.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:42.993+0000 7f988d600640 1 -- 172.21.15.5:0/4005889295 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f98780ef700 con 0x7f9890074ee0 2024-09-10T10:04:43.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:42.994+0000 7f9895200640 1 --2- 172.21.15.5:0/4005889295 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f98740a2b80 0x7f98740a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:43.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:42.994+0000 7f9899e40740 1 -- 172.21.15.5:0/3555617276 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558ef16aa760 msgr2=0x558ef16b4ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:43.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:42.994+0000 7f9899e40740 1 --2- 172.21.15.5:0/3555617276 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558ef16aa760 0x558ef16b4ae0 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f988000bd10 tx=0x7f98800107a0 comp rx=0 tx=0).stop 2024-09-10T10:04:43.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:42.994+0000 7f9899e40740 1 -- 172.21.15.5:0/3555617276 shutdown_connections 2024-09-10T10:04:43.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:42.994+0000 7f9899e40740 1 --2- 172.21.15.5:0/3555617276 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558ef16b5020 0x558ef16b7410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:43.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:42.994+0000 7f9899e40740 1 --2- 172.21.15.5:0/3555617276 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558ef16aa760 0x558ef16b4ae0 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:43.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:42.994+0000 7f9899e40740 1 --2- 172.21.15.5:0/3555617276 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558ef16a9e40 0x558ef16aa220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:43.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:42.994+0000 7f9899e40740 1 -- 172.21.15.5:0/3555617276 >> 172.21.15.5:0/3555617276 conn(0x558ef16a63d0 msgr2=0x558ef16a67b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:43.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:42.994+0000 7f9899e40740 1 -- 172.21.15.5:0/3555617276 shutdown_connections 2024-09-10T10:04:43.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:42.994+0000 7f9895200640 1 --2- 172.21.15.5:0/4005889295 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f98740a2b80 0x7f98740a5040 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f989006a6f0 tx=0x7f9888009600 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:43.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:42.994+0000 7f9899e40740 1 -- 172.21.15.5:0/3555617276 wait complete. 2024-09-10T10:04:43.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:42.995+0000 7f9899e40740 1 -- start start 2024-09-10T10:04:43.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:42.995+0000 7f9899e40740 1 Processor -- start 2024-09-10T10:04:43.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:42.995+0000 7f9899e40740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558ef16a9e40 0x558ef16b1e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:43.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:42.996+0000 7f9899e40740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558ef16aa760 0x558ef16b23b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:43.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:42.996+0000 7f9899e40740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558ef16b5020 0x558ef16b28f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:43.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:42.996+0000 7f9899e40740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x558ef16b7bf0 con 0x558ef16b5020 2024-09-10T10:04:43.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:42.996+0000 7f9899e40740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x558ef15cb0f0 con 0x558ef16aa760 2024-09-10T10:04:43.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:42.996+0000 7f9899e40740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x558ef16b7d70 con 0x558ef16a9e40 2024-09-10T10:04:43.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:42.996+0000 7f9899e40740 10 client.0 _ll_register_callbacks cb 0x558ef15b2bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:04:43.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:42.996+0000 7f9895200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558ef16b5020 0x558ef16b28f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:43.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:42.996+0000 7f988fe00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558ef16a9e40 0x558ef16b1e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:43.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:42.996+0000 7f9895200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558ef16b5020 0x558ef16b28f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40310/0 (socket says 172.21.15.5:40310) 2024-09-10T10:04:43.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:42.996+0000 7f9895200640 1 -- 172.21.15.5:0/2647630419 learned_addr learned my addr 172.21.15.5:0/2647630419 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T10:04:43.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:42.996+0000 7f9887e00640 1 --2- 172.21.15.5:0/2647630419 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558ef16aa760 0x558ef16b23b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:43.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:42.996+0000 7f9895200640 1 -- 172.21.15.5:0/2647630419 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558ef16a9e40 msgr2=0x558ef16b1e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:43.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:42.996+0000 7f9895200640 1 --2- 172.21.15.5:0/2647630419 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558ef16a9e40 0x558ef16b1e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:43.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:42.996+0000 7f9895200640 1 -- 172.21.15.5:0/2647630419 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558ef16aa760 msgr2=0x558ef16b23b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:43.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:42.996+0000 7f9895200640 1 --2- 172.21.15.5:0/2647630419 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558ef16aa760 0x558ef16b23b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:43.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:42.996+0000 7f9895200640 1 -- 172.21.15.5:0/2647630419 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558ef13813c0 con 0x558ef16b5020 2024-09-10T10:04:43.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:42.996+0000 7f988fe00640 1 --2- 172.21.15.5:0/2647630419 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558ef16a9e40 0x558ef16b1e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:04:43.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:42.996+0000 7f9887e00640 1 --2- 172.21.15.5:0/2647630419 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558ef16aa760 0x558ef16b23b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:04:43.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:42.997+0000 7f9895200640 1 --2- 172.21.15.5:0/2647630419 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558ef16b5020 0x558ef16b28f0 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f9888011e70 tx=0x7f9888004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa588a292c4eff6d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:43.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:42.997+0000 7f987e000640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-10T10:04:43.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:42.997+0000 7f987e000640 1 -- 172.21.15.5:0/2647630419 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9888020620 con 0x558ef16b5020 2024-09-10T10:04:43.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:42.997+0000 7f987e000640 1 -- 172.21.15.5:0/2647630419 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f98880207c0 con 0x558ef16b5020 2024-09-10T10:04:43.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:42.997+0000 7f9899e40740 1 -- 172.21.15.5:0/2647630419 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f98800041d0 con 0x558ef16b5020 2024-09-10T10:04:43.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:42.998+0000 7f987e000640 1 -- 172.21.15.5:0/2647630419 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f9888033050 con 0x558ef16b5020 2024-09-10T10:04:43.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:42.998+0000 7f9899e40740 10 client.20259 fetch_fsmap learned FSMap version 47 2024-09-10T10:04:43.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:42.998+0000 7f9899e40740 1 -- 172.21.15.5:0/2647630419 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x558ef16c3c80 con 0x558ef16b5020 2024-09-10T10:04:43.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:42.998+0000 7f987e000640 1 -- 172.21.15.5:0/2647630419 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f98800041d0 con 0x558ef16b5020 2024-09-10T10:04:43.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:42.998+0000 7f987e000640 1 -- 172.21.15.5:0/2647630419 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 47) ==== 8398+0+0 (secure 0 0 0) 0x7f988800f040 con 0x558ef16b5020 2024-09-10T10:04:43.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:42.998+0000 7f9899e40740 10 client.20259 fetch_fsmap finished waiting for FSMap version 47 2024-09-10T10:04:43.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:42.998+0000 7f9899e40740 10 client.20259 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-10T10:04:43.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:42.998+0000 7f9899e40740 20 client.20259 populate_metadata read hostname 'smithi005' 2024-09-10T10:04:43.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:42.998+0000 7f9899e40740 1 --2- 172.21.15.5:0/2647630419 >> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] conn(0x558ef16c5200 0x558ef16c76a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:43.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:42.998+0000 7f9899e40740 4 client.20259 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:04:43.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:42.998+0000 7f9899e40740 1 -- 172.21.15.5:0/2647630419 --> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558ef161d950 con 0x558ef16c5200 2024-09-10T10:04:43.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:42.999+0000 7f988fe00640 1 --2- 172.21.15.5:0/2647630419 >> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] conn(0x558ef16c5200 0x558ef16c76a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:43.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:42.999+0000 7f988fe00640 1 --2- 172.21.15.5:0/2647630419 >> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] conn(0x558ef16c5200 0x558ef16c76a0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2a324a356e83828c server_cookie=d4003ee1e8433d47 in_seq=0 out_seq=0 2024-09-10T10:04:43.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:42.999+0000 7f987e000640 10 client.20259 ms_handle_connect on v2:172.21.15.159:6836/620627639 2024-09-10T10:04:43.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:43.000+0000 7f987e000640 1 -- 172.21.15.5:0/2647630419 <== mds.2 v2:172.21.15.159:6836/620627639 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558ef161d950 con 0x558ef16c5200 2024-09-10T10:04:43.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:43.000+0000 7f987e000640 10 client.20259 handle_command_reply: tid=0 2024-09-10T10:04:43.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:43.000+0000 7f9899e40740 1 client.20259 shutdown 2024-09-10T10:04:43.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:43.000+0000 7f9899e40740 1 -- 172.21.15.5:0/2647630419 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558ef16b5020 msgr2=0x558ef16b28f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:43.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:43.000+0000 7f9899e40740 1 --2- 172.21.15.5:0/2647630419 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558ef16b5020 0x558ef16b28f0 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f9888011e70 tx=0x7f9888004a40 comp rx=0 tx=0).stop 2024-09-10T10:04:43.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:43.001+0000 7f9899e40740 1 -- 172.21.15.5:0/2647630419 shutdown_connections 2024-09-10T10:04:43.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:43.001+0000 7f9899e40740 1 --2- 172.21.15.5:0/2647630419 >> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] conn(0x558ef16c5200 0x558ef16c76a0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:43.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:43.001+0000 7f9899e40740 1 --2- 172.21.15.5:0/2647630419 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558ef16b5020 0x558ef16b28f0 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:43.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:43.001+0000 7f9899e40740 1 --2- 172.21.15.5:0/2647630419 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558ef16aa760 0x558ef16b23b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:43.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:43.001+0000 7f9899e40740 1 --2- 172.21.15.5:0/2647630419 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558ef16a9e40 0x558ef16b1e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:43.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:43.001+0000 7f9899e40740 1 -- 172.21.15.5:0/2647630419 >> 172.21.15.5:0/2647630419 conn(0x558ef16a63d0 msgr2=0x558ef16ac100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:43.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:43.001+0000 7f987e000640 0 client.20259 ms_handle_reset on v2:172.21.15.159:6836/620627639 2024-09-10T10:04:43.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:43.001+0000 7f9899e40740 1 -- 172.21.15.5:0/2647630419 shutdown_connections 2024-09-10T10:04:43.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:43.001+0000 7f9899e40740 1 -- 172.21.15.5:0/2647630419 wait complete. 2024-09-10T10:04:43.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:43.001+0000 7f9899e40740 20 client.20259 trim_cache size 0 max 16384 2024-09-10T10:04:43.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:43.015+0000 7f9899e40740 1 Processor -- start 2024-09-10T10:04:43.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:43.015+0000 7f9899e40740 1 -- start start 2024-09-10T10:04:43.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:43.015+0000 7f9899e40740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558ef16a9e40 0x558ef16ccfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:43.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:43.015+0000 7f9899e40740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558ef16aa760 0x558ef16cd4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:43.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:43.015+0000 7f9899e40740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558ef16b5020 0x558ef16af0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:43.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:43.016+0000 7f9899e40740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f986c00fcc0 con 0x558ef16b5020 2024-09-10T10:04:43.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:43.016+0000 7f9899e40740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f986c0118c0 con 0x558ef16a9e40 2024-09-10T10:04:43.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:43.016+0000 7f9899e40740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f986c010390 con 0x558ef16aa760 2024-09-10T10:04:43.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:43.016+0000 7f9887e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558ef16aa760 0x558ef16cd4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:43.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:43.016+0000 7f988fe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558ef16a9e40 0x558ef16ccfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:43.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:43.016+0000 7f988fe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558ef16a9e40 0x558ef16ccfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:44512/0 (socket says 172.21.15.5:44512) 2024-09-10T10:04:43.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:43.016+0000 7f988fe00640 1 -- 172.21.15.5:0/2771660892 learned_addr learned my addr 172.21.15.5:0/2771660892 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T10:04:43.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:43.016+0000 7f9895200640 1 --2- 172.21.15.5:0/2771660892 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558ef16b5020 0x558ef16af0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:43.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:43.016+0000 7f988fe00640 1 -- 172.21.15.5:0/2771660892 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558ef16aa760 msgr2=0x558ef16cd4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:43.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:43.016+0000 7f988fe00640 1 --2- 172.21.15.5:0/2771660892 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558ef16aa760 0x558ef16cd4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:43.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:43.016+0000 7f988fe00640 1 -- 172.21.15.5:0/2771660892 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558ef16b5020 msgr2=0x558ef16af0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:43.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:43.016+0000 7f988fe00640 1 --2- 172.21.15.5:0/2771660892 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558ef16b5020 0x558ef16af0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:43.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:43.016+0000 7f988fe00640 1 -- 172.21.15.5:0/2771660892 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558ef16b47e0 con 0x558ef16a9e40 2024-09-10T10:04:43.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:43.016+0000 7f9887e00640 1 --2- 172.21.15.5:0/2771660892 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558ef16aa760 0x558ef16cd4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:04:43.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:43.016+0000 7f9895200640 1 --2- 172.21.15.5:0/2771660892 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558ef16b5020 0x558ef16af0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:04:43.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:43.017+0000 7f988fe00640 1 --2- 172.21.15.5:0/2771660892 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558ef16a9e40 0x558ef16ccfb0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x558ef16a7230 tx=0x7f988000bc60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8700a646db4879e6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:43.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:43.017+0000 7f9884c00640 1 -- 172.21.15.5:0/2771660892 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9880005240 con 0x558ef16a9e40 2024-09-10T10:04:43.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:43.017+0000 7f9884c00640 1 -- 172.21.15.5:0/2771660892 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f988000b8e0 con 0x558ef16a9e40 2024-09-10T10:04:43.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:43.017+0000 7f9884c00640 1 -- 172.21.15.5:0/2771660892 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98800116a0 con 0x558ef16a9e40 2024-09-10T10:04:43.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:43.018+0000 7f9899e40740 1 -- 172.21.15.5:0/2771660892 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558ef16a9e40 msgr2=0x558ef16ccfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:43.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:43.018+0000 7f9899e40740 1 --2- 172.21.15.5:0/2771660892 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558ef16a9e40 0x558ef16ccfb0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x558ef16a7230 tx=0x7f988000bc60 comp rx=0 tx=0).stop 2024-09-10T10:04:43.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:43.018+0000 7f9899e40740 1 -- 172.21.15.5:0/2771660892 shutdown_connections 2024-09-10T10:04:43.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:43.018+0000 7f9899e40740 1 --2- 172.21.15.5:0/2771660892 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558ef16b5020 0x558ef16af0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:43.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:43.018+0000 7f9899e40740 1 --2- 172.21.15.5:0/2771660892 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558ef16aa760 0x558ef16cd4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:43.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:43.018+0000 7f9899e40740 1 --2- 172.21.15.5:0/2771660892 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558ef16a9e40 0x558ef16ccfb0 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:43.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:43.018+0000 7f9899e40740 1 -- 172.21.15.5:0/2771660892 >> 172.21.15.5:0/2771660892 conn(0x558ef16a63d0 msgr2=0x558ef16ac100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:43.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:43.018+0000 7f9899e40740 1 -- 172.21.15.5:0/2771660892 shutdown_connections 2024-09-10T10:04:43.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:43.018+0000 7f9899e40740 1 -- 172.21.15.5:0/2771660892 wait complete. 2024-09-10T10:04:43.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:43.018+0000 7f9899e40740 1 -- start start 2024-09-10T10:04:43.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:43.018+0000 7f9899e40740 1 Processor -- start 2024-09-10T10:04:43.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:43.019+0000 7f9899e40740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558ef16a9e40 0x558ef16bd330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:43.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:43.019+0000 7f9899e40740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558ef16aa760 0x558ef16bd870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:43.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:43.019+0000 7f9899e40740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558ef16b5020 0x558ef16ae8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:43.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:43.019+0000 7f9899e40740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f986c0111c0 con 0x558ef16a9e40 2024-09-10T10:04:43.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:43.019+0000 7f9899e40740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f986c010ac0 con 0x558ef16aa760 2024-09-10T10:04:43.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:43.019+0000 7f9899e40740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f986c00dca0 con 0x558ef16b5020 2024-09-10T10:04:43.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:43.019+0000 7f9899e40740 10 client.0 _ll_register_callbacks cb 0x7f9888011af0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:04:43.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:43.019+0000 7f988fe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558ef16a9e40 0x558ef16bd330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:43.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:43.019+0000 7f988fe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558ef16a9e40 0x558ef16bd330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40328/0 (socket says 172.21.15.5:40328) 2024-09-10T10:04:43.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:43.019+0000 7f988fe00640 1 -- 172.21.15.5:0/1726195788 learned_addr learned my addr 172.21.15.5:0/1726195788 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T10:04:43.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:43.019+0000 7f9895200640 1 --2- 172.21.15.5:0/1726195788 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558ef16b5020 0x558ef16ae8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:43.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:43.019+0000 7f9887e00640 1 --2- 172.21.15.5:0/1726195788 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558ef16aa760 0x558ef16bd870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:43.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:43.020+0000 7f9895200640 1 -- 172.21.15.5:0/1726195788 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558ef16aa760 msgr2=0x558ef16bd870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:43.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:43.020+0000 7f9895200640 1 --2- 172.21.15.5:0/1726195788 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558ef16aa760 0x558ef16bd870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:43.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:43.020+0000 7f9895200640 1 -- 172.21.15.5:0/1726195788 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558ef16a9e40 msgr2=0x558ef16bd330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:43.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:43.020+0000 7f9895200640 1 --2- 172.21.15.5:0/1726195788 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558ef16a9e40 0x558ef16bd330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:43.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:43.020+0000 7f9895200640 1 -- 172.21.15.5:0/1726195788 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558ef16af0e0 con 0x558ef16b5020 2024-09-10T10:04:43.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:43.020+0000 7f988fe00640 1 --2- 172.21.15.5:0/1726195788 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558ef16a9e40 0x558ef16bd330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:04:43.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:43.020+0000 7f9895200640 1 --2- 172.21.15.5:0/1726195788 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558ef16b5020 0x558ef16ae8b0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f988801de10 tx=0x7f9888013460 comp rx=0 tx=0).ready entity=mon.1 client_cookie=92c2c0fa950e59fe server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:43.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:43.020+0000 7f987e000640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-09-10T10:04:43.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:43.020+0000 7f987e000640 1 -- 172.21.15.5:0/1726195788 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9888004cb0 con 0x558ef16b5020 2024-09-10T10:04:43.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:43.020+0000 7f987e000640 1 -- 172.21.15.5:0/1726195788 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f9888003970 con 0x558ef16b5020 2024-09-10T10:04:43.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:43.020+0000 7f9899e40740 1 -- 172.21.15.5:0/1726195788 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f988000b8e0 con 0x558ef16b5020 2024-09-10T10:04:43.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:43.021+0000 7f987e000640 1 -- 172.21.15.5:0/1726195788 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f9888043050 con 0x558ef16b5020 2024-09-10T10:04:43.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:43.021+0000 7f9899e40740 10 client.30331 fetch_fsmap learned FSMap version 47 2024-09-10T10:04:43.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:43.021+0000 7f9899e40740 1 -- 172.21.15.5:0/1726195788 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x558ef172fba0 con 0x558ef16b5020 2024-09-10T10:04:43.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:43.021+0000 7f987e000640 1 -- 172.21.15.5:0/1726195788 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f988000b8e0 con 0x558ef16b5020 2024-09-10T10:04:43.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:43.021+0000 7f987e000640 1 -- 172.21.15.5:0/1726195788 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 47) ==== 8398+0+0 (secure 0 0 0) 0x7f988800f040 con 0x558ef16b5020 2024-09-10T10:04:43.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:43.021+0000 7f9899e40740 10 client.30331 fetch_fsmap finished waiting for FSMap version 47 2024-09-10T10:04:43.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:43.021+0000 7f9899e40740 10 client.30331 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-10T10:04:43.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:43.021+0000 7f9899e40740 20 client.30331 populate_metadata read hostname 'smithi005' 2024-09-10T10:04:43.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:43.021+0000 7f9899e40740 1 --2- 172.21.15.5:0/1726195788 >> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] conn(0x558ef1730340 0x558ef1732760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:43.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:43.021+0000 7f9899e40740 4 client.30331 mds_command: new command op to 24511 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T10:04:43.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:43.021+0000 7f9899e40740 1 -- 172.21.15.5:0/1726195788 --> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558ef1617e60 con 0x558ef1730340 2024-09-10T10:04:43.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:43.022+0000 7f988fe00640 1 --2- 172.21.15.5:0/1726195788 >> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] conn(0x558ef1730340 0x558ef1732760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:43.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:43.022+0000 7f988fe00640 1 --2- 172.21.15.5:0/1726195788 >> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] conn(0x558ef1730340 0x558ef1732760 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=21684cf7078949f7 server_cookie=15c9eb9e9ff08f51 in_seq=0 out_seq=0 2024-09-10T10:04:43.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:43.022+0000 7f987e000640 10 client.30331 ms_handle_connect on v2:172.21.15.159:6836/620627639 2024-09-10T10:04:43.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:43.023+0000 7f987e000640 1 -- 172.21.15.5:0/1726195788 <== mds.2 v2:172.21.15.159:6836/620627639 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558ef1617e60 con 0x558ef1730340 2024-09-10T10:04:43.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:43.023+0000 7f987e000640 10 client.30331 handle_command_reply: tid=0 2024-09-10T10:04:43.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:43.023+0000 7f9899e40740 1 client.30331 shutdown 2024-09-10T10:04:43.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:43.023+0000 7f9899e40740 1 -- 172.21.15.5:0/1726195788 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558ef16b5020 msgr2=0x558ef16ae8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:43.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:43.023+0000 7f9899e40740 1 --2- 172.21.15.5:0/1726195788 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558ef16b5020 0x558ef16ae8b0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f988801de10 tx=0x7f9888013460 comp rx=0 tx=0).stop 2024-09-10T10:04:43.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:43.023+0000 7f9899e40740 1 -- 172.21.15.5:0/1726195788 shutdown_connections 2024-09-10T10:04:43.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:43.023+0000 7f9899e40740 1 --2- 172.21.15.5:0/1726195788 >> [v2:172.21.15.159:6836/620627639,v1:172.21.15.159:6837/620627639] conn(0x558ef1730340 0x558ef1732760 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:43.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:43.023+0000 7f9899e40740 1 --2- 172.21.15.5:0/1726195788 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x558ef16b5020 0x558ef16ae8b0 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:43.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:43.023+0000 7f9899e40740 1 --2- 172.21.15.5:0/1726195788 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x558ef16aa760 0x558ef16bd870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:43.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:43.023+0000 7f9899e40740 1 --2- 172.21.15.5:0/1726195788 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558ef16a9e40 0x558ef16bd330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:43.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:43.023+0000 7f9899e40740 1 -- 172.21.15.5:0/1726195788 >> 172.21.15.5:0/1726195788 conn(0x558ef16a63d0 msgr2=0x558ef16b0740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:43.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:43.023+0000 7f987e000640 0 client.30331 ms_handle_reset on v2:172.21.15.159:6836/620627639 2024-09-10T10:04:43.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:43.023+0000 7f9899e40740 1 -- 172.21.15.5:0/1726195788 shutdown_connections 2024-09-10T10:04:43.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:43.023+0000 7f9899e40740 1 -- 172.21.15.5:0/1726195788 wait complete. 2024-09-10T10:04:43.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:43.023+0000 7f9899e40740 20 client.30331 trim_cache size 0 max 16384 2024-09-10T10:04:43.028 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-10T10:04:43.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:43.024+0000 7f9895c00640 1 -- 172.21.15.5:0/4005889295 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f98740a2b80 msgr2=0x7f98740a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:43.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:43.024+0000 7f9895c00640 1 --2- 172.21.15.5:0/4005889295 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f98740a2b80 0x7f98740a5040 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f989006a6f0 tx=0x7f9888009600 comp rx=0 tx=0).stop 2024-09-10T10:04:43.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:43.024+0000 7f9895c00640 1 -- 172.21.15.5:0/4005889295 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9890074ee0 msgr2=0x7f989019aa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:43.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:43.024+0000 7f9895c00640 1 --2- 172.21.15.5:0/4005889295 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9890074ee0 0x7f989019aa20 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f9878004770 tx=0x7f9878003c10 comp rx=0 tx=0).stop 2024-09-10T10:04:43.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:43.024+0000 7f9895c00640 1 -- 172.21.15.5:0/4005889295 shutdown_connections 2024-09-10T10:04:43.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:43.024+0000 7f9895c00640 1 --2- 172.21.15.5:0/4005889295 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f98740a2b80 0x7f98740a5040 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:43.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:43.024+0000 7f9895c00640 1 --2- 172.21.15.5:0/4005889295 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f989010e260 0x7f989019edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:43.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:43.024+0000 7f9895c00640 1 --2- 172.21.15.5:0/4005889295 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f9890074ee0 0x7f989019aa20 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:43.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:43.024+0000 7f9895c00640 1 --2- 172.21.15.5:0/4005889295 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f98900745c0 0x7f989019a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:43.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:43.024+0000 7f9895c00640 1 -- 172.21.15.5:0/4005889295 >> 172.21.15.5:0/4005889295 conn(0x7f98900fd740 msgr2=0x7f98900fe400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:43.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:43.024+0000 7f9895c00640 1 -- 172.21.15.5:0/4005889295 shutdown_connections 2024-09-10T10:04:43.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:43.024+0000 7f9895c00640 1 -- 172.21.15.5:0/4005889295 wait complete. 2024-09-10T10:04:43.774 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-10T10:04:44.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:04:43 smithi005 ceph-mon[30350]: pgmap v1184: 129 pgs: 129 active+clean; 553 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 201 KiB/s rd, 3.3 MiB/s wr, 162 op/s 2024-09-10T10:04:44.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:04:43 smithi159 ceph-mon[35513]: pgmap v1184: 129 pgs: 129 active+clean; 553 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 201 KiB/s rd, 3.3 MiB/s wr, 162 op/s 2024-09-10T10:04:44.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:04:43 smithi102 ceph-mon[35442]: pgmap v1184: 129 pgs: 129 active+clean; 553 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 201 KiB/s rd, 3.3 MiB/s wr, 162 op/s 2024-09-10T10:04:44.152 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:04:44.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:44.716+0000 7f8c6ec00640 1 -- 172.21.15.5:0/1946303652 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8c68077040 msgr2=0x7f8c68112510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:44.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:44.716+0000 7f8c6ec00640 1 --2- 172.21.15.5:0/1946303652 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8c68077040 0x7f8c68112510 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f8c50009900 tx=0x7f8c5002a410 comp rx=0 tx=0).stop 2024-09-10T10:04:44.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:44.716+0000 7f8c6ec00640 1 -- 172.21.15.5:0/1946303652 shutdown_connections 2024-09-10T10:04:44.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:44.716+0000 7f8c6ec00640 1 --2- 172.21.15.5:0/1946303652 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8c68112a50 0x7f8c68114e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:44.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:44.716+0000 7f8c6ec00640 1 --2- 172.21.15.5:0/1946303652 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8c68077040 0x7f8c68112510 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:44.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:44.716+0000 7f8c6ec00640 1 --2- 172.21.15.5:0/1946303652 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8c68076720 0x7f8c68076b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:44.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:44.716+0000 7f8c6ec00640 1 -- 172.21.15.5:0/1946303652 >> 172.21.15.5:0/1946303652 conn(0x7f8c680ffa50 msgr2=0x7f8c68101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:44.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:44.716+0000 7f8c6ec00640 1 -- 172.21.15.5:0/1946303652 shutdown_connections 2024-09-10T10:04:44.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:44.716+0000 7f8c6ec00640 1 -- 172.21.15.5:0/1946303652 wait complete. 2024-09-10T10:04:44.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:44.717+0000 7f8c6ec00640 1 Processor -- start 2024-09-10T10:04:44.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:44.717+0000 7f8c6ec00640 1 -- start start 2024-09-10T10:04:44.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:44.717+0000 7f8c6ec00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8c68076720 0x7f8c681a4860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:44.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:44.717+0000 7f8c6ec00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8c68077040 0x7f8c681a4da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:44.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:44.717+0000 7f8c6ec00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8c68112a50 0x7f8c6819ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:44.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:44.717+0000 7f8c6ec00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f8c68118630 con 0x7f8c68077040 2024-09-10T10:04:44.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:44.717+0000 7f8c6ec00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f8c681184b0 con 0x7f8c68112a50 2024-09-10T10:04:44.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:44.717+0000 7f8c6ec00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f8c681187b0 con 0x7f8c68076720 2024-09-10T10:04:44.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:44.718+0000 7f8c5fe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8c68077040 0x7f8c681a4da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:44.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:44.718+0000 7f8c5fe00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8c68077040 0x7f8c681a4da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40350/0 (socket says 172.21.15.5:40350) 2024-09-10T10:04:44.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:44.718+0000 7f8c5fe00640 1 -- 172.21.15.5:0/3848320474 learned_addr learned my addr 172.21.15.5:0/3848320474 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T10:04:44.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:44.718+0000 7f8c67e00640 1 --2- 172.21.15.5:0/3848320474 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8c68076720 0x7f8c681a4860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:44.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:44.718+0000 7f8c6cc00640 1 --2- 172.21.15.5:0/3848320474 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8c68112a50 0x7f8c6819ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:44.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:44.718+0000 7f8c5fe00640 1 -- 172.21.15.5:0/3848320474 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8c68076720 msgr2=0x7f8c681a4860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:44.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:44.718+0000 7f8c5fe00640 1 --2- 172.21.15.5:0/3848320474 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8c68076720 0x7f8c681a4860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:44.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:44.718+0000 7f8c5fe00640 1 -- 172.21.15.5:0/3848320474 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8c68112a50 msgr2=0x7f8c6819ea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:44.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:44.718+0000 7f8c5fe00640 1 --2- 172.21.15.5:0/3848320474 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8c68112a50 0x7f8c6819ea20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:44.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:44.718+0000 7f8c5fe00640 1 -- 172.21.15.5:0/3848320474 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c6819f150 con 0x7f8c68077040 2024-09-10T10:04:44.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:44.718+0000 7f8c6cc00640 1 --2- 172.21.15.5:0/3848320474 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8c68112a50 0x7f8c6819ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:04:44.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:44.718+0000 7f8c5fe00640 1 --2- 172.21.15.5:0/3848320474 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8c68077040 0x7f8c681a4da0 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f8c50004770 tx=0x7f8c50003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:44.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:44.719+0000 7f8c65600640 1 -- 172.21.15.5:0/3848320474 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c5003f070 con 0x7f8c68077040 2024-09-10T10:04:44.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:44.719+0000 7f8c65600640 1 -- 172.21.15.5:0/3848320474 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8c50031ca0 con 0x7f8c68077040 2024-09-10T10:04:44.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:44.719+0000 7f8c6ec00640 1 -- 172.21.15.5:0/3848320474 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c6819f3e0 con 0x7f8c68077040 2024-09-10T10:04:44.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:44.719+0000 7f8c65600640 1 -- 172.21.15.5:0/3848320474 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c50035030 con 0x7f8c68077040 2024-09-10T10:04:44.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:44.719+0000 7f8c6ec00640 1 -- 172.21.15.5:0/3848320474 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c681ab730 con 0x7f8c68077040 2024-09-10T10:04:44.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:44.722+0000 7f8c716ae740 1 Processor -- start 2024-09-10T10:04:44.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:44.722+0000 7f8c65600640 1 -- 172.21.15.5:0/3848320474 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f8c50042070 con 0x7f8c68077040 2024-09-10T10:04:44.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:44.722+0000 7f8c716ae740 1 -- start start 2024-09-10T10:04:44.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:44.723+0000 7f8c716ae740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d2717a9c70 0x55d2717aa050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:44.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:44.723+0000 7f8c716ae740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d2717aa590 0x55d2717b4910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:44.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:44.723+0000 7f8c716ae740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d2717b4e50 0x55d2717b7240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:44.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:44.723+0000 7f8c716ae740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55d2717103e0 con 0x55d2717b4e50 2024-09-10T10:04:44.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:44.723+0000 7f8c716ae740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55d2716b7d80 con 0x55d2717aa590 2024-09-10T10:04:44.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:44.723+0000 7f8c716ae740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55d27170c3a0 con 0x55d2717a9c70 2024-09-10T10:04:44.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:44.723+0000 7f8c67e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d2717aa590 0x55d2717b4910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:44.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:44.723+0000 7f8c6cc00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d2717a9c70 0x55d2717aa050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:44.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:44.723+0000 7f8c67e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d2717aa590 0x55d2717b4910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:44556/0 (socket says 172.21.15.5:44556) 2024-09-10T10:04:44.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:44.723+0000 7f8c67e00640 1 -- 172.21.15.5:0/489453588 learned_addr learned my addr 172.21.15.5:0/489453588 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T10:04:44.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:44.723+0000 7f8c5fe00640 1 --2- 172.21.15.5:0/489453588 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d2717b4e50 0x55d2717b7240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:44.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:44.723+0000 7f8c67e00640 1 -- 172.21.15.5:0/489453588 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d2717a9c70 msgr2=0x55d2717aa050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:44.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:44.723+0000 7f8c67e00640 1 --2- 172.21.15.5:0/489453588 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d2717a9c70 0x55d2717aa050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:44.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:44.724+0000 7f8c67e00640 1 -- 172.21.15.5:0/489453588 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d2717b4e50 msgr2=0x55d2717b7240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:44.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:44.724+0000 7f8c67e00640 1 --2- 172.21.15.5:0/489453588 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d2717b4e50 0x55d2717b7240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:44.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:44.724+0000 7f8c67e00640 1 -- 172.21.15.5:0/489453588 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d2716d6ae0 con 0x55d2717aa590 2024-09-10T10:04:44.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:44.724+0000 7f8c65600640 1 --2- 172.21.15.5:0/3848320474 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f8c4c0a2b80 0x7f8c4c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:44.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:44.724+0000 7f8c67e00640 1 --2- 172.21.15.5:0/489453588 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d2717aa590 0x55d2717b4910 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f8c58009ca0 tx=0x7f8c58011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9a39c01fa20452fe server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:44.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:44.724+0000 7f8c65600640 1 -- 172.21.15.5:0/3848320474 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f8c500efb20 con 0x7f8c68077040 2024-09-10T10:04:44.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:44.724+0000 7f8c6cc00640 1 --2- 172.21.15.5:0/3848320474 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f8c4c0a2b80 0x7f8c4c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:44.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:44.724+0000 7f8c5d200640 1 -- 172.21.15.5:0/489453588 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c58005220 con 0x55d2717aa590 2024-09-10T10:04:44.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:44.724+0000 7f8c5d200640 1 -- 172.21.15.5:0/489453588 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8c58004c80 con 0x55d2717aa590 2024-09-10T10:04:44.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:44.724+0000 7f8c5d200640 1 -- 172.21.15.5:0/489453588 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c58019800 con 0x55d2717aa590 2024-09-10T10:04:44.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:44.725+0000 7f8c716ae740 1 -- 172.21.15.5:0/489453588 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d2717aa590 msgr2=0x55d2717b4910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:44.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:44.725+0000 7f8c716ae740 1 --2- 172.21.15.5:0/489453588 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d2717aa590 0x55d2717b4910 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f8c58009ca0 tx=0x7f8c58011810 comp rx=0 tx=0).stop 2024-09-10T10:04:44.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:44.725+0000 7f8c716ae740 1 -- 172.21.15.5:0/489453588 shutdown_connections 2024-09-10T10:04:44.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:44.725+0000 7f8c716ae740 1 --2- 172.21.15.5:0/489453588 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d2717b4e50 0x55d2717b7240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:44.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:44.725+0000 7f8c716ae740 1 --2- 172.21.15.5:0/489453588 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d2717aa590 0x55d2717b4910 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:44.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:44.725+0000 7f8c716ae740 1 --2- 172.21.15.5:0/489453588 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d2717a9c70 0x55d2717aa050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:44.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:44.725+0000 7f8c716ae740 1 -- 172.21.15.5:0/489453588 >> 172.21.15.5:0/489453588 conn(0x55d2717a6200 msgr2=0x55d2717a65e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:44.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:44.725+0000 7f8c716ae740 1 -- 172.21.15.5:0/489453588 shutdown_connections 2024-09-10T10:04:44.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:44.725+0000 7f8c716ae740 1 -- 172.21.15.5:0/489453588 wait complete. 2024-09-10T10:04:44.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:44.726+0000 7f8c6cc00640 1 --2- 172.21.15.5:0/3848320474 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f8c4c0a2b80 0x7f8c4c0a5040 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f8c6806b460 tx=0x7f8c60009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:44.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:44.726+0000 7f8c716ae740 1 -- start start 2024-09-10T10:04:44.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:44.726+0000 7f8c716ae740 1 Processor -- start 2024-09-10T10:04:44.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:44.727+0000 7f8c716ae740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d2717a9c70 0x55d2717b1cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:44.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:44.727+0000 7f8c716ae740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d2717aa590 0x55d2717b2230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:44.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:44.727+0000 7f8c716ae740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d2717b4e50 0x55d2717b2770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:44.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:44.727+0000 7f8c716ae740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55d2717b7a20 con 0x55d2717aa590 2024-09-10T10:04:44.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:44.727+0000 7f8c716ae740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55d2716cb0f0 con 0x55d2717a9c70 2024-09-10T10:04:44.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:44.727+0000 7f8c716ae740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55d2717b7ba0 con 0x55d2717b4e50 2024-09-10T10:04:44.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:44.727+0000 7f8c716ae740 10 client.0 _ll_register_callbacks cb 0x55d2716b2bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:04:44.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:44.727+0000 7f8c67e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d2717a9c70 0x55d2717b1cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:44.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:44.727+0000 7f8c67e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d2717a9c70 0x55d2717b1cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:44572/0 (socket says 172.21.15.5:44572) 2024-09-10T10:04:44.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:44.727+0000 7f8c67e00640 1 -- 172.21.15.5:0/1130914539 learned_addr learned my addr 172.21.15.5:0/1130914539 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T10:04:44.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:44.727+0000 7f8c6cc00640 1 --2- 172.21.15.5:0/1130914539 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d2717b4e50 0x55d2717b2770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:44.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:44.727+0000 7f8c5fe00640 1 --2- 172.21.15.5:0/1130914539 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d2717aa590 0x55d2717b2230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:44.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:44.727+0000 7f8c67e00640 1 -- 172.21.15.5:0/1130914539 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d2717b4e50 msgr2=0x55d2717b2770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:44.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:44.727+0000 7f8c67e00640 1 --2- 172.21.15.5:0/1130914539 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d2717b4e50 0x55d2717b2770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:44.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:44.727+0000 7f8c67e00640 1 -- 172.21.15.5:0/1130914539 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d2717aa590 msgr2=0x55d2717b2230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:44.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:44.727+0000 7f8c67e00640 1 --2- 172.21.15.5:0/1130914539 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d2717aa590 0x55d2717b2230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:44.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:44.727+0000 7f8c67e00640 1 -- 172.21.15.5:0/1130914539 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d2714813c0 con 0x55d2717a9c70 2024-09-10T10:04:44.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:44.727+0000 7f8c5fe00640 1 --2- 172.21.15.5:0/1130914539 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d2717aa590 0x55d2717b2230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:04:44.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:44.728+0000 7f8c67e00640 1 --2- 172.21.15.5:0/1130914539 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d2717a9c70 0x55d2717b1cf0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f8c5801b040 tx=0x7f8c58005db0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=af563b0ecf288061 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:44.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:44.728+0000 7f8c55600640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-09-10T10:04:44.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:44.728+0000 7f8c55600640 1 -- 172.21.15.5:0/1130914539 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c58029070 con 0x55d2717a9c70 2024-09-10T10:04:44.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:44.728+0000 7f8c55600640 1 -- 172.21.15.5:0/1130914539 <== mon.2 v2:172.21.15.102:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f8c58019560 con 0x55d2717a9c70 2024-09-10T10:04:44.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:44.728+0000 7f8c716ae740 1 -- 172.21.15.5:0/1130914539 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8c58004c80 con 0x55d2717a9c70 2024-09-10T10:04:44.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:44.728+0000 7f8c55600640 1 -- 172.21.15.5:0/1130914539 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f8c58035050 con 0x55d2717a9c70 2024-09-10T10:04:44.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:44.728+0000 7f8c716ae740 10 client.30374 fetch_fsmap learned FSMap version 47 2024-09-10T10:04:44.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:44.728+0000 7f8c716ae740 1 -- 172.21.15.5:0/1130914539 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x55d2717c3a10 con 0x55d2717a9c70 2024-09-10T10:04:44.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:44.729+0000 7f8c55600640 1 -- 172.21.15.5:0/1130914539 <== mon.2 v2:172.21.15.102:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8c58004c80 con 0x55d2717a9c70 2024-09-10T10:04:44.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:44.729+0000 7f8c55600640 1 -- 172.21.15.5:0/1130914539 <== mon.2 v2:172.21.15.102:3300/0 5 ==== fsmap(e 47) ==== 8398+0+0 (secure 0 0 0) 0x7f8c58023840 con 0x55d2717a9c70 2024-09-10T10:04:44.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:44.729+0000 7f8c716ae740 10 client.30374 fetch_fsmap finished waiting for FSMap version 47 2024-09-10T10:04:44.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:44.729+0000 7f8c716ae740 10 client.30374 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-10T10:04:44.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:44.729+0000 7f8c716ae740 20 client.30374 populate_metadata read hostname 'smithi005' 2024-09-10T10:04:44.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:44.729+0000 7f8c716ae740 1 --2- 172.21.15.5:0/1130914539 >> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] conn(0x55d2717c4d60 0x55d2717c7150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:44.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:44.729+0000 7f8c716ae740 4 client.30374 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:04:44.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:44.729+0000 7f8c716ae740 1 -- 172.21.15.5:0/1130914539 --> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d27171d950 con 0x55d2717c4d60 2024-09-10T10:04:44.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:44.729+0000 7f8c5fe00640 1 --2- 172.21.15.5:0/1130914539 >> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] conn(0x55d2717c4d60 0x55d2717c7150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:44.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:44.730+0000 7f8c5fe00640 1 --2- 172.21.15.5:0/1130914539 >> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] conn(0x55d2717c4d60 0x55d2717c7150 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b49679abb3c4f626 server_cookie=6a21200a20fa15ff in_seq=0 out_seq=0 2024-09-10T10:04:44.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:44.730+0000 7f8c55600640 10 client.30374 ms_handle_connect on v2:172.21.15.159:6838/2073739599 2024-09-10T10:04:44.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:44.731+0000 7f8c55600640 1 -- 172.21.15.5:0/1130914539 <== mds.1 v2:172.21.15.159:6838/2073739599 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d27171d950 con 0x55d2717c4d60 2024-09-10T10:04:44.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:44.731+0000 7f8c55600640 10 client.30374 handle_command_reply: tid=0 2024-09-10T10:04:44.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:44.731+0000 7f8c716ae740 1 client.30374 shutdown 2024-09-10T10:04:44.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:44.731+0000 7f8c716ae740 1 -- 172.21.15.5:0/1130914539 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d2717a9c70 msgr2=0x55d2717b1cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:44.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:44.731+0000 7f8c716ae740 1 --2- 172.21.15.5:0/1130914539 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d2717a9c70 0x55d2717b1cf0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f8c5801b040 tx=0x7f8c58005db0 comp rx=0 tx=0).stop 2024-09-10T10:04:44.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:44.731+0000 7f8c716ae740 1 -- 172.21.15.5:0/1130914539 shutdown_connections 2024-09-10T10:04:44.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:44.731+0000 7f8c716ae740 1 --2- 172.21.15.5:0/1130914539 >> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] conn(0x55d2717c4d60 0x55d2717c7150 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:44.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:44.731+0000 7f8c716ae740 1 --2- 172.21.15.5:0/1130914539 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d2717b4e50 0x55d2717b2770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:44.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:44.731+0000 7f8c716ae740 1 --2- 172.21.15.5:0/1130914539 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d2717aa590 0x55d2717b2230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:44.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:44.731+0000 7f8c716ae740 1 --2- 172.21.15.5:0/1130914539 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d2717a9c70 0x55d2717b1cf0 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:44.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:44.731+0000 7f8c716ae740 1 -- 172.21.15.5:0/1130914539 >> 172.21.15.5:0/1130914539 conn(0x55d2717a6200 msgr2=0x55d2717abee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:44.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:44.731+0000 7f8c55600640 0 client.30374 ms_handle_reset on v2:172.21.15.159:6838/2073739599 2024-09-10T10:04:44.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:44.732+0000 7f8c716ae740 1 -- 172.21.15.5:0/1130914539 shutdown_connections 2024-09-10T10:04:44.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:44.732+0000 7f8c716ae740 1 -- 172.21.15.5:0/1130914539 wait complete. 2024-09-10T10:04:44.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:44.732+0000 7f8c716ae740 20 client.30374 trim_cache size 0 max 16384 2024-09-10T10:04:44.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:44.745+0000 7f8c716ae740 1 Processor -- start 2024-09-10T10:04:44.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:44.745+0000 7f8c716ae740 1 -- start start 2024-09-10T10:04:44.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:44.746+0000 7f8c716ae740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d2717a9c70 0x55d2717b1250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:44.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:44.746+0000 7f8c716ae740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d2717aa590 0x55d2717cc980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:44.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:44.746+0000 7f8c716ae740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d2717b4e50 0x55d2717ccec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:44.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:44.746+0000 7f8c716ae740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f8c681b5460 con 0x55d2717a9c70 2024-09-10T10:04:44.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:44.746+0000 7f8c716ae740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f8c681b7060 con 0x55d2717aa590 2024-09-10T10:04:44.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:44.746+0000 7f8c716ae740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f8c681b5b30 con 0x55d2717b4e50 2024-09-10T10:04:44.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:44.746+0000 7f8c67e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d2717a9c70 0x55d2717b1250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:44.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:44.746+0000 7f8c67e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d2717a9c70 0x55d2717b1250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40380/0 (socket says 172.21.15.5:40380) 2024-09-10T10:04:44.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:44.746+0000 7f8c5fe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d2717aa590 0x55d2717cc980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:44.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:44.746+0000 7f8c67e00640 1 -- 172.21.15.5:0/129040680 learned_addr learned my addr 172.21.15.5:0/129040680 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T10:04:44.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:44.746+0000 7f8c6cc00640 1 --2- 172.21.15.5:0/129040680 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d2717b4e50 0x55d2717ccec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:44.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:44.747+0000 7f8c5fe00640 1 -- 172.21.15.5:0/129040680 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d2717b4e50 msgr2=0x55d2717ccec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:44.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:44.747+0000 7f8c5fe00640 1 --2- 172.21.15.5:0/129040680 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d2717b4e50 0x55d2717ccec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:44.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:44.747+0000 7f8c5fe00640 1 -- 172.21.15.5:0/129040680 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d2717a9c70 msgr2=0x55d2717b1250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:44.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:44.747+0000 7f8c5fe00640 1 --2- 172.21.15.5:0/129040680 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d2717a9c70 0x55d2717b1250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:44.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:44.747+0000 7f8c5fe00640 1 -- 172.21.15.5:0/129040680 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d2717a1770 con 0x55d2717aa590 2024-09-10T10:04:44.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:44.747+0000 7f8c67e00640 1 --2- 172.21.15.5:0/129040680 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d2717a9c70 0x55d2717b1250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:04:44.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:44.747+0000 7f8c6cc00640 1 --2- 172.21.15.5:0/129040680 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d2717b4e50 0x55d2717ccec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:04:44.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:44.747+0000 7f8c5fe00640 1 --2- 172.21.15.5:0/129040680 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d2717aa590 0x55d2717cc980 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f8c500ed1a0 tx=0x7f8c500f3560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fad987cf536443ed server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:44.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:44.747+0000 7f8c57e00640 1 -- 172.21.15.5:0/129040680 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c50049070 con 0x55d2717aa590 2024-09-10T10:04:44.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:44.747+0000 7f8c57e00640 1 -- 172.21.15.5:0/129040680 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8c500bc6f0 con 0x55d2717aa590 2024-09-10T10:04:44.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:44.747+0000 7f8c57e00640 1 -- 172.21.15.5:0/129040680 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c500c4c60 con 0x55d2717aa590 2024-09-10T10:04:44.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:44.748+0000 7f8c716ae740 1 -- 172.21.15.5:0/129040680 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d2717aa590 msgr2=0x55d2717cc980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:44.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:44.748+0000 7f8c716ae740 1 --2- 172.21.15.5:0/129040680 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d2717aa590 0x55d2717cc980 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f8c500ed1a0 tx=0x7f8c500f3560 comp rx=0 tx=0).stop 2024-09-10T10:04:44.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:44.748+0000 7f8c716ae740 1 -- 172.21.15.5:0/129040680 shutdown_connections 2024-09-10T10:04:44.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:44.748+0000 7f8c716ae740 1 --2- 172.21.15.5:0/129040680 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d2717b4e50 0x55d2717ccec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:44.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:44.748+0000 7f8c716ae740 1 --2- 172.21.15.5:0/129040680 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d2717aa590 0x55d2717cc980 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:44.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:44.748+0000 7f8c716ae740 1 --2- 172.21.15.5:0/129040680 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d2717a9c70 0x55d2717b1250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:44.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:44.748+0000 7f8c716ae740 1 -- 172.21.15.5:0/129040680 >> 172.21.15.5:0/129040680 conn(0x55d2717a6200 msgr2=0x55d2717abee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:44.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:44.748+0000 7f8c716ae740 1 -- 172.21.15.5:0/129040680 shutdown_connections 2024-09-10T10:04:44.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:44.748+0000 7f8c716ae740 1 -- 172.21.15.5:0/129040680 wait complete. 2024-09-10T10:04:44.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:44.748+0000 7f8c716ae740 1 -- start start 2024-09-10T10:04:44.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:44.749+0000 7f8c716ae740 1 Processor -- start 2024-09-10T10:04:44.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:44.749+0000 7f8c716ae740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d2717a9c70 0x55d2717bd6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:44.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:44.749+0000 7f8c716ae740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d2717aa590 0x55d2717b89d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:44.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:44.749+0000 7f8c716ae740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d2717b4e50 0x55d2717b9160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:44.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:44.749+0000 7f8c716ae740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f8c681b6960 con 0x55d2717a9c70 2024-09-10T10:04:44.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:44.749+0000 7f8c716ae740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f8c681b6260 con 0x55d2717aa590 2024-09-10T10:04:44.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:44.749+0000 7f8c716ae740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f8c681b3440 con 0x55d2717b4e50 2024-09-10T10:04:44.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:44.749+0000 7f8c716ae740 10 client.0 _ll_register_callbacks cb 0x7f8c500f3430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:04:44.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:44.749+0000 7f8c67e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d2717a9c70 0x55d2717bd6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:44.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:44.749+0000 7f8c5fe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d2717aa590 0x55d2717b89d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:44.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:44.750+0000 7f8c67e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d2717a9c70 0x55d2717bd6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40390/0 (socket says 172.21.15.5:40390) 2024-09-10T10:04:44.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:44.750+0000 7f8c67e00640 1 -- 172.21.15.5:0/164187014 learned_addr learned my addr 172.21.15.5:0/164187014 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T10:04:44.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:44.750+0000 7f8c6cc00640 1 --2- 172.21.15.5:0/164187014 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d2717b4e50 0x55d2717b9160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:44.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:44.750+0000 7f8c67e00640 1 -- 172.21.15.5:0/164187014 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d2717b4e50 msgr2=0x55d2717b9160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:44.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:44.750+0000 7f8c67e00640 1 --2- 172.21.15.5:0/164187014 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d2717b4e50 0x55d2717b9160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:44.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:44.750+0000 7f8c67e00640 1 -- 172.21.15.5:0/164187014 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d2717aa590 msgr2=0x55d2717b89d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:44.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:44.750+0000 7f8c67e00640 1 --2- 172.21.15.5:0/164187014 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d2717aa590 0x55d2717b89d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:44.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:44.750+0000 7f8c67e00640 1 -- 172.21.15.5:0/164187014 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d27182e950 con 0x55d2717a9c70 2024-09-10T10:04:44.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:44.750+0000 7f8c5fe00640 1 --2- 172.21.15.5:0/164187014 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d2717aa590 0x55d2717b89d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:04:44.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:44.750+0000 7f8c67e00640 1 --2- 172.21.15.5:0/164187014 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d2717a9c70 0x55d2717bd6a0 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f8c58023be0 tx=0x7f8c580056d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e48e01153b9dcba5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:44.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:44.750+0000 7f8c55600640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-10T10:04:44.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:44.751+0000 7f8c55600640 1 -- 172.21.15.5:0/164187014 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c58029070 con 0x55d2717a9c70 2024-09-10T10:04:44.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:44.751+0000 7f8c716ae740 1 -- 172.21.15.5:0/164187014 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8c500bc6f0 con 0x55d2717a9c70 2024-09-10T10:04:44.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:44.751+0000 7f8c55600640 1 -- 172.21.15.5:0/164187014 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f8c58004450 con 0x55d2717a9c70 2024-09-10T10:04:44.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:44.751+0000 7f8c55600640 1 -- 172.21.15.5:0/164187014 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f8c58009af0 con 0x55d2717a9c70 2024-09-10T10:04:44.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:44.751+0000 7f8c716ae740 10 client.20283 fetch_fsmap learned FSMap version 47 2024-09-10T10:04:44.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:44.751+0000 7f8c716ae740 1 -- 172.21.15.5:0/164187014 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x55d27182fab0 con 0x55d2717a9c70 2024-09-10T10:04:44.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:44.751+0000 7f8c55600640 1 -- 172.21.15.5:0/164187014 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8c500bc6f0 con 0x55d2717a9c70 2024-09-10T10:04:44.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:44.752+0000 7f8c55600640 1 -- 172.21.15.5:0/164187014 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 47) ==== 8398+0+0 (secure 0 0 0) 0x7f8c580035f0 con 0x55d2717a9c70 2024-09-10T10:04:44.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:44.752+0000 7f8c716ae740 10 client.20283 fetch_fsmap finished waiting for FSMap version 47 2024-09-10T10:04:44.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:44.752+0000 7f8c716ae740 10 client.20283 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-10T10:04:44.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:44.752+0000 7f8c716ae740 20 client.20283 populate_metadata read hostname 'smithi005' 2024-09-10T10:04:44.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:44.752+0000 7f8c716ae740 1 --2- 172.21.15.5:0/164187014 >> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] conn(0x55d2718301f0 0x55d271832610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:44.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:44.752+0000 7f8c716ae740 4 client.20283 mds_command: new command op to 24535 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T10:04:44.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:44.752+0000 7f8c716ae740 1 -- 172.21.15.5:0/164187014 --> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d271717e60 con 0x55d2718301f0 2024-09-10T10:04:44.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:44.752+0000 7f8c5fe00640 1 --2- 172.21.15.5:0/164187014 >> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] conn(0x55d2718301f0 0x55d271832610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:44.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:44.752+0000 7f8c5fe00640 1 --2- 172.21.15.5:0/164187014 >> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] conn(0x55d2718301f0 0x55d271832610 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=36f465c6f2bafc4 server_cookie=20bd334dc7f221c8 in_seq=0 out_seq=0 2024-09-10T10:04:44.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:44.752+0000 7f8c55600640 10 client.20283 ms_handle_connect on v2:172.21.15.159:6838/2073739599 2024-09-10T10:04:44.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:44.753+0000 7f8c55600640 1 -- 172.21.15.5:0/164187014 <== mds.1 v2:172.21.15.159:6838/2073739599 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d271717e60 con 0x55d2718301f0 2024-09-10T10:04:44.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:44.753+0000 7f8c55600640 10 client.20283 handle_command_reply: tid=0 2024-09-10T10:04:44.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:44.753+0000 7f8c716ae740 1 client.20283 shutdown 2024-09-10T10:04:44.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:44.753+0000 7f8c716ae740 1 -- 172.21.15.5:0/164187014 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d2717a9c70 msgr2=0x55d2717bd6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:44.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:44.753+0000 7f8c716ae740 1 --2- 172.21.15.5:0/164187014 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d2717a9c70 0x55d2717bd6a0 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f8c58023be0 tx=0x7f8c580056d0 comp rx=0 tx=0).stop 2024-09-10T10:04:44.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:44.753+0000 7f8c716ae740 1 -- 172.21.15.5:0/164187014 shutdown_connections 2024-09-10T10:04:44.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:44.753+0000 7f8c716ae740 1 --2- 172.21.15.5:0/164187014 >> [v2:172.21.15.159:6838/2073739599,v1:172.21.15.159:6839/2073739599] conn(0x55d2718301f0 0x55d271832610 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:44.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:44.753+0000 7f8c716ae740 1 --2- 172.21.15.5:0/164187014 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55d2717b4e50 0x55d2717b9160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:44.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:44.753+0000 7f8c716ae740 1 --2- 172.21.15.5:0/164187014 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55d2717aa590 0x55d2717b89d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:44.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:44.753+0000 7f8c716ae740 1 --2- 172.21.15.5:0/164187014 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d2717a9c70 0x55d2717bd6a0 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:44.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:44.753+0000 7f8c716ae740 1 -- 172.21.15.5:0/164187014 >> 172.21.15.5:0/164187014 conn(0x55d2717a6200 msgr2=0x55d2717a4b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:44.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:44.753+0000 7f8c55600640 0 client.20283 ms_handle_reset on v2:172.21.15.159:6838/2073739599 2024-09-10T10:04:44.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:44.753+0000 7f8c716ae740 1 -- 172.21.15.5:0/164187014 shutdown_connections 2024-09-10T10:04:44.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:44.753+0000 7f8c716ae740 1 -- 172.21.15.5:0/164187014 wait complete. 2024-09-10T10:04:44.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:44.753+0000 7f8c716ae740 20 client.20283 trim_cache size 0 max 16384 2024-09-10T10:04:44.793 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-10T10:04:44.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:44.754+0000 7f8c6ec00640 1 -- 172.21.15.5:0/3848320474 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f8c4c0a2b80 msgr2=0x7f8c4c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:44.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:44.754+0000 7f8c6ec00640 1 --2- 172.21.15.5:0/3848320474 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f8c4c0a2b80 0x7f8c4c0a5040 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f8c6806b460 tx=0x7f8c60009500 comp rx=0 tx=0).stop 2024-09-10T10:04:44.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:44.754+0000 7f8c6ec00640 1 -- 172.21.15.5:0/3848320474 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8c68077040 msgr2=0x7f8c681a4da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:44.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:44.754+0000 7f8c6ec00640 1 --2- 172.21.15.5:0/3848320474 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8c68077040 0x7f8c681a4da0 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f8c50004770 tx=0x7f8c50003c10 comp rx=0 tx=0).stop 2024-09-10T10:04:44.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:44.754+0000 7f8c6ec00640 1 -- 172.21.15.5:0/3848320474 shutdown_connections 2024-09-10T10:04:44.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:44.754+0000 7f8c6ec00640 1 --2- 172.21.15.5:0/3848320474 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f8c4c0a2b80 0x7f8c4c0a5040 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:44.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:44.754+0000 7f8c6ec00640 1 --2- 172.21.15.5:0/3848320474 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f8c68112a50 0x7f8c6819ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:44.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:44.754+0000 7f8c6ec00640 1 --2- 172.21.15.5:0/3848320474 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8c68077040 0x7f8c681a4da0 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:44.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:44.754+0000 7f8c6ec00640 1 --2- 172.21.15.5:0/3848320474 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f8c68076720 0x7f8c681a4860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:44.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:44.754+0000 7f8c6ec00640 1 -- 172.21.15.5:0/3848320474 >> 172.21.15.5:0/3848320474 conn(0x7f8c680ffa50 msgr2=0x7f8c68101e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:44.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:44.754+0000 7f8c6ec00640 1 -- 172.21.15.5:0/3848320474 shutdown_connections 2024-09-10T10:04:44.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:44.754+0000 7f8c6ec00640 1 -- 172.21.15.5:0/3848320474 wait complete. 2024-09-10T10:04:45.589 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-10T10:04:45.973 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:04:46.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:04:45 smithi005 ceph-mon[30350]: pgmap v1185: 129 pgs: 129 active+clean; 553 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 197 KiB/s rd, 2.6 MiB/s wr, 141 op/s 2024-09-10T10:04:46.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:04:45 smithi159 ceph-mon[35513]: pgmap v1185: 129 pgs: 129 active+clean; 553 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 197 KiB/s rd, 2.6 MiB/s wr, 141 op/s 2024-09-10T10:04:46.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:04:45 smithi102 ceph-mon[35442]: pgmap v1185: 129 pgs: 129 active+clean; 553 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 197 KiB/s rd, 2.6 MiB/s wr, 141 op/s 2024-09-10T10:04:46.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:46.611+0000 7f3798200640 1 -- 172.21.15.5:0/739112608 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3790102790 msgr2=0x7f37901067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:46.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:46.611+0000 7f3798200640 1 --2- 172.21.15.5:0/739112608 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3790102790 0x7f37901067e0 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f377c009900 tx=0x7f377c02a3f0 comp rx=0 tx=0).stop 2024-09-10T10:04:46.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:46.611+0000 7f3798200640 1 -- 172.21.15.5:0/739112608 shutdown_connections 2024-09-10T10:04:46.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:46.611+0000 7f3798200640 1 --2- 172.21.15.5:0/739112608 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3790106f10 0x7f379010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:46.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:46.611+0000 7f3798200640 1 --2- 172.21.15.5:0/739112608 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3790102790 0x7f37901067e0 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:46.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:46.611+0000 7f3798200640 1 --2- 172.21.15.5:0/739112608 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3790101de0 0x7f37901021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:46.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:46.611+0000 7f3798200640 1 -- 172.21.15.5:0/739112608 >> 172.21.15.5:0/739112608 conn(0x7f37900fd8c0 msgr2=0x7f37900ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:46.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:46.611+0000 7f3798200640 1 -- 172.21.15.5:0/739112608 shutdown_connections 2024-09-10T10:04:46.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:46.611+0000 7f3798200640 1 -- 172.21.15.5:0/739112608 wait complete. 2024-09-10T10:04:46.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:46.612+0000 7f3798200640 1 Processor -- start 2024-09-10T10:04:46.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:46.612+0000 7f3798200640 1 -- start start 2024-09-10T10:04:46.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:46.612+0000 7f3798200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3790101de0 0x7f37900744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:46.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:46.612+0000 7f3798200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3790102790 0x7f3790074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:46.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:46.612+0000 7f3798200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3790106f10 0x7f3790074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:46.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:46.613+0000 7f3798200640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f3790111fc0 con 0x7f3790101de0 2024-09-10T10:04:46.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:46.613+0000 7f3798200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f3790111e40 con 0x7f3790102790 2024-09-10T10:04:46.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:46.613+0000 7f3798200640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f3790112140 con 0x7f3790106f10 2024-09-10T10:04:46.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:46.613+0000 7f3794e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3790102790 0x7f3790074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:46.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:46.613+0000 7f3795800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3790101de0 0x7f37900744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:46.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:46.613+0000 7f3794e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3790102790 0x7f3790074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:44610/0 (socket says 172.21.15.5:44610) 2024-09-10T10:04:46.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:46.613+0000 7f3794e00640 1 -- 172.21.15.5:0/3758521748 learned_addr learned my addr 172.21.15.5:0/3758521748 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T10:04:46.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:46.613+0000 7f3796200640 1 --2- 172.21.15.5:0/3758521748 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3790106f10 0x7f3790074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:46.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:46.613+0000 7f3794e00640 1 -- 172.21.15.5:0/3758521748 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3790106f10 msgr2=0x7f3790074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:46.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:46.613+0000 7f3794e00640 1 --2- 172.21.15.5:0/3758521748 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3790106f10 0x7f3790074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:46.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:46.613+0000 7f3794e00640 1 -- 172.21.15.5:0/3758521748 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3790101de0 msgr2=0x7f37900744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:46.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:46.613+0000 7f3794e00640 1 --2- 172.21.15.5:0/3758521748 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3790101de0 0x7f37900744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:46.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:46.613+0000 7f3794e00640 1 -- 172.21.15.5:0/3758521748 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37901aae10 con 0x7f3790102790 2024-09-10T10:04:46.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:46.614+0000 7f3794e00640 1 --2- 172.21.15.5:0/3758521748 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3790102790 0x7f3790074a30 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f377c004770 tx=0x7f377c003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:46.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:46.614+0000 7f378e000640 1 -- 172.21.15.5:0/3758521748 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f377c03f070 con 0x7f3790102790 2024-09-10T10:04:46.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:46.614+0000 7f3798200640 1 -- 172.21.15.5:0/3758521748 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37901aafe0 con 0x7f3790102790 2024-09-10T10:04:46.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:46.614+0000 7f378e000640 1 -- 172.21.15.5:0/3758521748 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f377c031ca0 con 0x7f3790102790 2024-09-10T10:04:46.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:46.614+0000 7f378e000640 1 -- 172.21.15.5:0/3758521748 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f377c035030 con 0x7f3790102790 2024-09-10T10:04:46.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:46.614+0000 7f3798200640 1 -- 172.21.15.5:0/3758521748 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37901ab490 con 0x7f3790102790 2024-09-10T10:04:46.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:46.618+0000 7f379ac49740 1 Processor -- start 2024-09-10T10:04:46.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:46.618+0000 7f379ac49740 1 -- start start 2024-09-10T10:04:46.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:46.619+0000 7f379ac49740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ac1a8b5c70 0x55ac1a8b6050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:46.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:46.619+0000 7f379ac49740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ac1a8b6590 0x55ac1a8c0910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:46.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:46.619+0000 7f379ac49740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ac1a8c0e50 0x55ac1a8c3240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:46.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:46.619+0000 7f379ac49740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55ac1a81c3e0 con 0x55ac1a8c0e50 2024-09-10T10:04:46.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:46.619+0000 7f379ac49740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55ac1a7c3d80 con 0x55ac1a8b6590 2024-09-10T10:04:46.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:46.619+0000 7f379ac49740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55ac1a8183a0 con 0x55ac1a8b5c70 2024-09-10T10:04:46.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:46.619+0000 7f3796200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ac1a8b5c70 0x55ac1a8b6050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:46.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:46.619+0000 7f3795800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ac1a8b6590 0x55ac1a8c0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:46.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:46.619+0000 7f3796200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ac1a8b5c70 0x55ac1a8b6050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:55440/0 (socket says 172.21.15.5:55440) 2024-09-10T10:04:46.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:46.619+0000 7f3796200640 1 -- 172.21.15.5:0/4001346846 learned_addr learned my addr 172.21.15.5:0/4001346846 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T10:04:46.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:46.619+0000 7f378e000640 1 -- 172.21.15.5:0/3758521748 <== mon.2 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f377c042070 con 0x7f3790102790 2024-09-10T10:04:46.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:46.619+0000 7f3796200640 1 -- 172.21.15.5:0/4001346846 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ac1a8b6590 msgr2=0x55ac1a8c0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:46.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:46.619+0000 7f3796200640 1 --2- 172.21.15.5:0/4001346846 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ac1a8b6590 0x55ac1a8c0910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:46.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:46.619+0000 7f3796200640 1 -- 172.21.15.5:0/4001346846 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ac1a8c0e50 msgr2=0x55ac1a8c3240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:46.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:46.619+0000 7f3796200640 1 --2- 172.21.15.5:0/4001346846 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ac1a8c0e50 0x55ac1a8c3240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:46.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:46.619+0000 7f3796200640 1 -- 172.21.15.5:0/4001346846 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ac1a7e2ae0 con 0x55ac1a8b5c70 2024-09-10T10:04:46.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:46.620+0000 7f3796200640 1 --2- 172.21.15.5:0/4001346846 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ac1a8b5c70 0x55ac1a8b6050 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f3788009f60 tx=0x7f3788009f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cb065d21f182b292 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:46.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:46.620+0000 7f3782a00640 1 -- 172.21.15.5:0/4001346846 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f378801e070 con 0x55ac1a8b5c70 2024-09-10T10:04:46.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:46.620+0000 7f3782a00640 1 -- 172.21.15.5:0/4001346846 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3788003c30 con 0x55ac1a8b5c70 2024-09-10T10:04:46.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:46.620+0000 7f3782a00640 1 -- 172.21.15.5:0/4001346846 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f378800f920 con 0x55ac1a8b5c70 2024-09-10T10:04:46.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:46.621+0000 7f378e000640 1 --2- 172.21.15.5:0/3758521748 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f37780a2b80 0x7f37780a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:46.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:46.621+0000 7f378e000640 1 -- 172.21.15.5:0/3758521748 <== mon.2 v2:172.21.15.102:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f377c0ef9e0 con 0x7f3790102790 2024-09-10T10:04:46.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:46.621+0000 7f379ac49740 1 -- 172.21.15.5:0/4001346846 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ac1a8b5c70 msgr2=0x55ac1a8b6050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:46.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:46.621+0000 7f379ac49740 1 --2- 172.21.15.5:0/4001346846 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ac1a8b5c70 0x55ac1a8b6050 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f3788009f60 tx=0x7f3788009f90 comp rx=0 tx=0).stop 2024-09-10T10:04:46.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:46.621+0000 7f3795800640 1 --2- 172.21.15.5:0/3758521748 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f37780a2b80 0x7f37780a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:46.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:46.621+0000 7f379ac49740 1 -- 172.21.15.5:0/4001346846 shutdown_connections 2024-09-10T10:04:46.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:46.621+0000 7f379ac49740 1 --2- 172.21.15.5:0/4001346846 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ac1a8c0e50 0x55ac1a8c3240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:46.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:46.621+0000 7f379ac49740 1 --2- 172.21.15.5:0/4001346846 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ac1a8b6590 0x55ac1a8c0910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:46.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:46.621+0000 7f379ac49740 1 --2- 172.21.15.5:0/4001346846 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ac1a8b5c70 0x55ac1a8b6050 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:46.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:46.621+0000 7f379ac49740 1 -- 172.21.15.5:0/4001346846 >> 172.21.15.5:0/4001346846 conn(0x55ac1a8b2200 msgr2=0x55ac1a8b25e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:46.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:46.621+0000 7f379ac49740 1 -- 172.21.15.5:0/4001346846 shutdown_connections 2024-09-10T10:04:46.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:46.621+0000 7f379ac49740 1 -- 172.21.15.5:0/4001346846 wait complete. 2024-09-10T10:04:46.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:46.622+0000 7f3795800640 1 --2- 172.21.15.5:0/3758521748 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f37780a2b80 0x7f37780a5040 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x55ac1a8b3060 tx=0x7f3784005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:46.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:46.622+0000 7f379ac49740 1 -- start start 2024-09-10T10:04:46.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:46.622+0000 7f379ac49740 1 Processor -- start 2024-09-10T10:04:46.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:46.622+0000 7f379ac49740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ac1a8b5c70 0x55ac1a8bde00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:46.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:46.622+0000 7f379ac49740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ac1a8b6590 0x55ac1a8be340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:46.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:46.622+0000 7f379ac49740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ac1a8c0e50 0x55ac1a8be880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:46.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:46.622+0000 7f379ac49740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55ac1a8c3a20 con 0x55ac1a8b6590 2024-09-10T10:04:46.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:46.622+0000 7f379ac49740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x55ac1a7d70f0 con 0x55ac1a8c0e50 2024-09-10T10:04:46.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:46.622+0000 7f379ac49740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x55ac1a8c3ba0 con 0x55ac1a8b5c70 2024-09-10T10:04:46.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:46.623+0000 7f379ac49740 10 client.0 _ll_register_callbacks cb 0x55ac1a7bebd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:04:46.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:46.623+0000 7f3794e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ac1a8b6590 0x55ac1a8be340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:46.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:46.623+0000 7f3795800640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ac1a8b5c70 0x55ac1a8bde00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:46.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:46.623+0000 7f3794e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ac1a8b6590 0x55ac1a8be340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40418/0 (socket says 172.21.15.5:40418) 2024-09-10T10:04:46.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:46.623+0000 7f3794e00640 1 -- 172.21.15.5:0/3165930904 learned_addr learned my addr 172.21.15.5:0/3165930904 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T10:04:46.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:46.623+0000 7f3795800640 1 --2- 172.21.15.5:0/3165930904 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ac1a8b5c70 0x55ac1a8bde00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:55448/0 (socket says 172.21.15.5:55448) 2024-09-10T10:04:46.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:46.623+0000 7f3796200640 1 --2- 172.21.15.5:0/3165930904 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ac1a8c0e50 0x55ac1a8be880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:46.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:46.623+0000 7f3795800640 1 -- 172.21.15.5:0/3165930904 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ac1a8c0e50 msgr2=0x55ac1a8be880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:46.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:46.623+0000 7f3795800640 1 --2- 172.21.15.5:0/3165930904 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ac1a8c0e50 0x55ac1a8be880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:46.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:46.623+0000 7f3795800640 1 -- 172.21.15.5:0/3165930904 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ac1a8b6590 msgr2=0x55ac1a8be340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:46.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:46.623+0000 7f3795800640 1 --2- 172.21.15.5:0/3165930904 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ac1a8b6590 0x55ac1a8be340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:46.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:46.623+0000 7f3795800640 1 -- 172.21.15.5:0/3165930904 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ac1a58d3c0 con 0x55ac1a8b5c70 2024-09-10T10:04:46.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:46.623+0000 7f3794e00640 1 --2- 172.21.15.5:0/3165930904 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ac1a8b6590 0x55ac1a8be340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:04:46.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:46.623+0000 7f3796200640 1 --2- 172.21.15.5:0/3165930904 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ac1a8c0e50 0x55ac1a8be880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:04:46.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:46.624+0000 7f3795800640 1 --2- 172.21.15.5:0/3165930904 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ac1a8b5c70 0x55ac1a8bde00 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f378401e9d0 tx=0x7f3784004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=edf95dbd6510d784 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:46.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:46.624+0000 7f3777400640 10 client.0 ms_handle_connect on v2:172.21.15.159:3300/0 2024-09-10T10:04:46.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:46.624+0000 7f3777400640 1 -- 172.21.15.5:0/3165930904 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f378402e060 con 0x55ac1a8b5c70 2024-09-10T10:04:46.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:46.624+0000 7f379ac49740 1 -- 172.21.15.5:0/3165930904 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3788003c30 con 0x55ac1a8b5c70 2024-09-10T10:04:46.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:46.624+0000 7f3777400640 1 -- 172.21.15.5:0/3165930904 <== mon.1 v2:172.21.15.159:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f3784018030 con 0x55ac1a8b5c70 2024-09-10T10:04:46.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:46.625+0000 7f3777400640 1 -- 172.21.15.5:0/3165930904 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f3784035050 con 0x55ac1a8b5c70 2024-09-10T10:04:46.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:46.625+0000 7f379ac49740 10 client.30361 fetch_fsmap learned FSMap version 47 2024-09-10T10:04:46.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:46.625+0000 7f379ac49740 1 -- 172.21.15.5:0/3165930904 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x55ac1a8cfb80 con 0x55ac1a8b5c70 2024-09-10T10:04:46.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:46.625+0000 7f3777400640 1 -- 172.21.15.5:0/3165930904 <== mon.1 v2:172.21.15.159:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3788003c30 con 0x55ac1a8b5c70 2024-09-10T10:04:46.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:46.625+0000 7f3777400640 1 -- 172.21.15.5:0/3165930904 <== mon.1 v2:172.21.15.159:3300/0 5 ==== fsmap(e 47) ==== 8398+0+0 (secure 0 0 0) 0x7f378401c430 con 0x55ac1a8b5c70 2024-09-10T10:04:46.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:46.625+0000 7f379ac49740 10 client.30361 fetch_fsmap finished waiting for FSMap version 47 2024-09-10T10:04:46.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:46.625+0000 7f379ac49740 10 client.30361 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T10:04:46.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:46.625+0000 7f379ac49740 20 client.30361 populate_metadata read hostname 'smithi005' 2024-09-10T10:04:46.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:46.625+0000 7f379ac49740 1 --2- 172.21.15.5:0/3165930904 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55ac1a8d1040 0x55ac1a8d34b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:46.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:46.625+0000 7f379ac49740 4 client.30361 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T10:04:46.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:46.625+0000 7f379ac49740 1 -- 172.21.15.5:0/3165930904 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ac1a829950 con 0x55ac1a8d1040 2024-09-10T10:04:46.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:46.626+0000 7f3794e00640 1 --2- 172.21.15.5:0/3165930904 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55ac1a8d1040 0x55ac1a8d34b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:46.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:46.626+0000 7f3794e00640 1 --2- 172.21.15.5:0/3165930904 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55ac1a8d1040 0x55ac1a8d34b0 crc :-1 s=READY pgs=351 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6dc86775c8e102f7 server_cookie=33c8446e72e90bdb in_seq=0 out_seq=0 2024-09-10T10:04:46.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:46.626+0000 7f3777400640 10 client.30361 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T10:04:46.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:46.627+0000 7f3777400640 1 -- 172.21.15.5:0/3165930904 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ac1a829950 con 0x55ac1a8d1040 2024-09-10T10:04:46.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:46.627+0000 7f3777400640 10 client.30361 handle_command_reply: tid=0 2024-09-10T10:04:46.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:46.627+0000 7f379ac49740 1 client.30361 shutdown 2024-09-10T10:04:46.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:46.627+0000 7f379ac49740 1 -- 172.21.15.5:0/3165930904 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ac1a8b5c70 msgr2=0x55ac1a8bde00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:46.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:46.627+0000 7f379ac49740 1 --2- 172.21.15.5:0/3165930904 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ac1a8b5c70 0x55ac1a8bde00 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f378401e9d0 tx=0x7f3784004730 comp rx=0 tx=0).stop 2024-09-10T10:04:46.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:46.627+0000 7f379ac49740 1 -- 172.21.15.5:0/3165930904 shutdown_connections 2024-09-10T10:04:46.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:46.627+0000 7f379ac49740 1 --2- 172.21.15.5:0/3165930904 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55ac1a8d1040 0x55ac1a8d34b0 crc :-1 s=READY pgs=351 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:46.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:46.627+0000 7f379ac49740 1 --2- 172.21.15.5:0/3165930904 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ac1a8c0e50 0x55ac1a8be880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:46.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:46.628+0000 7f379ac49740 1 --2- 172.21.15.5:0/3165930904 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ac1a8b6590 0x55ac1a8be340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:46.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:46.628+0000 7f3777400640 0 client.30361 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T10:04:46.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:46.628+0000 7f379ac49740 1 --2- 172.21.15.5:0/3165930904 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ac1a8b5c70 0x55ac1a8bde00 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:46.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:46.628+0000 7f379ac49740 1 -- 172.21.15.5:0/3165930904 >> 172.21.15.5:0/3165930904 conn(0x55ac1a8b2200 msgr2=0x55ac1a8b7ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:46.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:46.628+0000 7f379ac49740 1 -- 172.21.15.5:0/3165930904 shutdown_connections 2024-09-10T10:04:46.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:46.628+0000 7f379ac49740 1 -- 172.21.15.5:0/3165930904 wait complete. 2024-09-10T10:04:46.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:46.628+0000 7f379ac49740 20 client.30361 trim_cache size 0 max 16384 2024-09-10T10:04:46.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:46.642+0000 7f379ac49740 1 Processor -- start 2024-09-10T10:04:46.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:46.642+0000 7f379ac49740 1 -- start start 2024-09-10T10:04:46.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:46.642+0000 7f379ac49740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ac1a8b5c70 0x55ac1a8bd760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:46.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:46.642+0000 7f379ac49740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ac1a8b6590 0x55ac1a8d8d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:46.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:46.642+0000 7f379ac49740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ac1a8c0e50 0x55ac1a8d92c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:46.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:46.642+0000 7f379ac49740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f37901b54f0 con 0x55ac1a8b6590 2024-09-10T10:04:46.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:46.642+0000 7f379ac49740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f37901b70f0 con 0x55ac1a8c0e50 2024-09-10T10:04:46.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:46.642+0000 7f379ac49740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f37901b5bc0 con 0x55ac1a8b5c70 2024-09-10T10:04:46.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:46.642+0000 7f3794e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ac1a8b6590 0x55ac1a8d8d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:46.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:46.642+0000 7f3794e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ac1a8b6590 0x55ac1a8d8d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40424/0 (socket says 172.21.15.5:40424) 2024-09-10T10:04:46.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:46.642+0000 7f3794e00640 1 -- 172.21.15.5:0/446847525 learned_addr learned my addr 172.21.15.5:0/446847525 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T10:04:46.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:46.642+0000 7f3795800640 1 --2- 172.21.15.5:0/446847525 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ac1a8b5c70 0x55ac1a8bd760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:46.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:46.643+0000 7f3796200640 1 --2- 172.21.15.5:0/446847525 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ac1a8c0e50 0x55ac1a8d92c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:46.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:46.643+0000 7f3795800640 1 -- 172.21.15.5:0/446847525 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ac1a8c0e50 msgr2=0x55ac1a8d92c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:46.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:46.643+0000 7f3795800640 1 --2- 172.21.15.5:0/446847525 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ac1a8c0e50 0x55ac1a8d92c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:46.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:46.643+0000 7f3795800640 1 -- 172.21.15.5:0/446847525 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ac1a8b6590 msgr2=0x55ac1a8d8d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:46.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:46.643+0000 7f3795800640 1 --2- 172.21.15.5:0/446847525 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ac1a8b6590 0x55ac1a8d8d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:46.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:46.643+0000 7f3795800640 1 -- 172.21.15.5:0/446847525 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ac1a8b5aa0 con 0x55ac1a8b5c70 2024-09-10T10:04:46.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:46.643+0000 7f3796200640 1 --2- 172.21.15.5:0/446847525 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ac1a8c0e50 0x55ac1a8d92c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:04:46.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:46.643+0000 7f3794e00640 1 --2- 172.21.15.5:0/446847525 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ac1a8b6590 0x55ac1a8d8d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:04:46.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:46.643+0000 7f3795800640 1 --2- 172.21.15.5:0/446847525 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ac1a8b5c70 0x55ac1a8bd760 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f3784012d50 tx=0x7f378401edd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=64b3402cd5a901e6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:46.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:46.643+0000 7f3782000640 1 -- 172.21.15.5:0/446847525 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f378403e070 con 0x55ac1a8b5c70 2024-09-10T10:04:46.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:46.644+0000 7f3782000640 1 -- 172.21.15.5:0/446847525 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3784004520 con 0x55ac1a8b5c70 2024-09-10T10:04:46.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:46.644+0000 7f3782000640 1 -- 172.21.15.5:0/446847525 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3784003850 con 0x55ac1a8b5c70 2024-09-10T10:04:46.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:46.644+0000 7f379ac49740 1 -- 172.21.15.5:0/446847525 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ac1a8b5c70 msgr2=0x55ac1a8bd760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:46.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:46.644+0000 7f379ac49740 1 --2- 172.21.15.5:0/446847525 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ac1a8b5c70 0x55ac1a8bd760 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f3784012d50 tx=0x7f378401edd0 comp rx=0 tx=0).stop 2024-09-10T10:04:46.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:46.644+0000 7f379ac49740 1 -- 172.21.15.5:0/446847525 shutdown_connections 2024-09-10T10:04:46.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:46.644+0000 7f379ac49740 1 --2- 172.21.15.5:0/446847525 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ac1a8c0e50 0x55ac1a8d92c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:46.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:46.644+0000 7f379ac49740 1 --2- 172.21.15.5:0/446847525 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ac1a8b6590 0x55ac1a8d8d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:46.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:46.644+0000 7f379ac49740 1 --2- 172.21.15.5:0/446847525 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ac1a8b5c70 0x55ac1a8bd760 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:46.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:46.644+0000 7f379ac49740 1 -- 172.21.15.5:0/446847525 >> 172.21.15.5:0/446847525 conn(0x55ac1a8b2200 msgr2=0x55ac1a8be5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:46.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:46.644+0000 7f379ac49740 1 -- 172.21.15.5:0/446847525 shutdown_connections 2024-09-10T10:04:46.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:46.644+0000 7f379ac49740 1 -- 172.21.15.5:0/446847525 wait complete. 2024-09-10T10:04:46.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:46.645+0000 7f379ac49740 1 -- start start 2024-09-10T10:04:46.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:46.645+0000 7f379ac49740 1 Processor -- start 2024-09-10T10:04:46.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:46.645+0000 7f379ac49740 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ac1a8b5c70 0x55ac1a8c9040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:46.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:46.645+0000 7f379ac49740 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ac1a8b6590 0x55ac1a8c9580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:46.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:46.645+0000 7f379ac49740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ac1a8c0e50 0x55ac1a8ba6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:46.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:46.645+0000 7f379ac49740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f37901b69f0 con 0x55ac1a8c0e50 2024-09-10T10:04:46.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:46.645+0000 7f379ac49740 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f37901b62f0 con 0x55ac1a8b5c70 2024-09-10T10:04:46.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:46.645+0000 7f379ac49740 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f37901b34d0 con 0x55ac1a8b6590 2024-09-10T10:04:46.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:46.646+0000 7f379ac49740 10 client.0 _ll_register_callbacks cb 0x7f3784009ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T10:04:46.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:46.646+0000 7f3795800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ac1a8b5c70 0x55ac1a8c9040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:46.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:46.646+0000 7f3794e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ac1a8b6590 0x55ac1a8c9580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:46.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:46.646+0000 7f3795800640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ac1a8b5c70 0x55ac1a8c9040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:44656/0 (socket says 172.21.15.5:44656) 2024-09-10T10:04:46.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:46.646+0000 7f3795800640 1 -- 172.21.15.5:0/1119276326 learned_addr learned my addr 172.21.15.5:0/1119276326 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T10:04:46.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:46.646+0000 7f3796200640 1 --2- 172.21.15.5:0/1119276326 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ac1a8c0e50 0x55ac1a8ba6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:46.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:46.646+0000 7f3795800640 1 -- 172.21.15.5:0/1119276326 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ac1a8b6590 msgr2=0x55ac1a8c9580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:46.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:46.646+0000 7f3795800640 1 --2- 172.21.15.5:0/1119276326 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ac1a8b6590 0x55ac1a8c9580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:46.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:46.646+0000 7f3795800640 1 -- 172.21.15.5:0/1119276326 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ac1a8c0e50 msgr2=0x55ac1a8ba6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T10:04:46.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:46.646+0000 7f3795800640 1 --2- 172.21.15.5:0/1119276326 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ac1a8c0e50 0x55ac1a8ba6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:46.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:46.646+0000 7f3795800640 1 -- 172.21.15.5:0/1119276326 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ac1a8ad770 con 0x55ac1a8b5c70 2024-09-10T10:04:46.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:46.646+0000 7f3794e00640 1 --2- 172.21.15.5:0/1119276326 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ac1a8b6590 0x55ac1a8c9580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:04:46.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:46.647+0000 7f3795800640 1 --2- 172.21.15.5:0/1119276326 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ac1a8b5c70 0x55ac1a8c9040 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f3784019040 tx=0x7f378401e7c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bd8c547599ceaf9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:46.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:46.647+0000 7f3777400640 10 client.0 ms_handle_connect on v2:172.21.15.102:3300/0 2024-09-10T10:04:46.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:46.647+0000 7f3777400640 1 -- 172.21.15.5:0/1119276326 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f378403e070 con 0x55ac1a8b5c70 2024-09-10T10:04:46.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:46.647+0000 7f379ac49740 1 -- 172.21.15.5:0/1119276326 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3784004520 con 0x55ac1a8b5c70 2024-09-10T10:04:46.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:46.647+0000 7f3777400640 1 -- 172.21.15.5:0/1119276326 <== mon.2 v2:172.21.15.102:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f3784005040 con 0x55ac1a8b5c70 2024-09-10T10:04:46.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:46.647+0000 7f3777400640 1 -- 172.21.15.5:0/1119276326 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f378404b050 con 0x55ac1a8b5c70 2024-09-10T10:04:46.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:46.647+0000 7f379ac49740 10 client.30398 fetch_fsmap learned FSMap version 47 2024-09-10T10:04:46.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:46.647+0000 7f379ac49740 1 -- 172.21.15.5:0/1119276326 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x55ac1a93b670 con 0x55ac1a8b5c70 2024-09-10T10:04:46.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:46.648+0000 7f3777400640 1 -- 172.21.15.5:0/1119276326 <== mon.2 v2:172.21.15.102:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3784004520 con 0x55ac1a8b5c70 2024-09-10T10:04:46.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:46.648+0000 7f3777400640 1 -- 172.21.15.5:0/1119276326 <== mon.2 v2:172.21.15.102:3300/0 5 ==== fsmap(e 47) ==== 8398+0+0 (secure 0 0 0) 0x7f3784038030 con 0x55ac1a8b5c70 2024-09-10T10:04:46.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:46.648+0000 7f379ac49740 10 client.30398 fetch_fsmap finished waiting for FSMap version 47 2024-09-10T10:04:46.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:46.648+0000 7f379ac49740 10 client.30398 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T10:04:46.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:46.648+0000 7f379ac49740 20 client.30398 populate_metadata read hostname 'smithi005' 2024-09-10T10:04:46.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:46.648+0000 7f379ac49740 1 --2- 172.21.15.5:0/1119276326 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55ac1a93bcd0 0x55ac1a93e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:46.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:46.648+0000 7f379ac49740 4 client.30398 mds_command: new command op to 24542 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T10:04:46.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:46.648+0000 7f379ac49740 1 -- 172.21.15.5:0/1119276326 --> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ac1a823e60 con 0x55ac1a93bcd0 2024-09-10T10:04:46.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:46.648+0000 7f3794e00640 1 --2- 172.21.15.5:0/1119276326 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55ac1a93bcd0 0x55ac1a93e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:46.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:46.649+0000 7f3794e00640 1 --2- 172.21.15.5:0/1119276326 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55ac1a93bcd0 0x55ac1a93e0c0 crc :-1 s=READY pgs=352 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=366ec8c26cc499c server_cookie=5591ee1369042a82 in_seq=0 out_seq=0 2024-09-10T10:04:46.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:46.649+0000 7f3777400640 10 client.30398 ms_handle_connect on v2:172.21.15.102:6838/335409749 2024-09-10T10:04:46.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:46.649+0000 7f3777400640 1 -- 172.21.15.5:0/1119276326 <== mds.0 v2:172.21.15.102:6838/335409749 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ac1a823e60 con 0x55ac1a93bcd0 2024-09-10T10:04:46.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:46.649+0000 7f3777400640 10 client.30398 handle_command_reply: tid=0 2024-09-10T10:04:46.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:46.649+0000 7f379ac49740 1 client.30398 shutdown 2024-09-10T10:04:46.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:46.649+0000 7f379ac49740 1 -- 172.21.15.5:0/1119276326 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ac1a8b5c70 msgr2=0x55ac1a8c9040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:46.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:46.649+0000 7f379ac49740 1 --2- 172.21.15.5:0/1119276326 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ac1a8b5c70 0x55ac1a8c9040 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f3784019040 tx=0x7f378401e7c0 comp rx=0 tx=0).stop 2024-09-10T10:04:46.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:46.649+0000 7f379ac49740 1 -- 172.21.15.5:0/1119276326 shutdown_connections 2024-09-10T10:04:46.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:46.649+0000 7f379ac49740 1 --2- 172.21.15.5:0/1119276326 >> [v2:172.21.15.102:6838/335409749,v1:172.21.15.102:6839/335409749] conn(0x55ac1a93bcd0 0x55ac1a93e0c0 crc :-1 s=READY pgs=352 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:46.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:46.649+0000 7f379ac49740 1 --2- 172.21.15.5:0/1119276326 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ac1a8c0e50 0x55ac1a8ba6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:46.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:46.649+0000 7f379ac49740 1 --2- 172.21.15.5:0/1119276326 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x55ac1a8b6590 0x55ac1a8c9580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:46.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:46.649+0000 7f379ac49740 1 --2- 172.21.15.5:0/1119276326 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x55ac1a8b5c70 0x55ac1a8c9040 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:46.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:46.649+0000 7f3777400640 0 client.30398 ms_handle_reset on v2:172.21.15.102:6838/335409749 2024-09-10T10:04:46.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:46.649+0000 7f379ac49740 1 -- 172.21.15.5:0/1119276326 >> 172.21.15.5:0/1119276326 conn(0x55ac1a8b2200 msgr2=0x55ac1a8bf660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:46.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:46.650+0000 7f379ac49740 1 -- 172.21.15.5:0/1119276326 shutdown_connections 2024-09-10T10:04:46.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:46.650+0000 7f379ac49740 1 -- 172.21.15.5:0/1119276326 wait complete. 2024-09-10T10:04:46.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:46.650+0000 7f379ac49740 20 client.30398 trim_cache size 0 max 16384 2024-09-10T10:04:46.654 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-10T10:04:46.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:46.650+0000 7f3798200640 1 -- 172.21.15.5:0/3758521748 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f37780a2b80 msgr2=0x7f37780a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:46.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:46.650+0000 7f3798200640 1 --2- 172.21.15.5:0/3758521748 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f37780a2b80 0x7f37780a5040 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x55ac1a8b3060 tx=0x7f3784005f90 comp rx=0 tx=0).stop 2024-09-10T10:04:46.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:46.650+0000 7f3798200640 1 -- 172.21.15.5:0/3758521748 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3790102790 msgr2=0x7f3790074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:46.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:46.650+0000 7f3798200640 1 --2- 172.21.15.5:0/3758521748 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3790102790 0x7f3790074a30 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f377c004770 tx=0x7f377c003c10 comp rx=0 tx=0).stop 2024-09-10T10:04:46.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:46.650+0000 7f3798200640 1 -- 172.21.15.5:0/3758521748 shutdown_connections 2024-09-10T10:04:46.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:46.650+0000 7f3798200640 1 --2- 172.21.15.5:0/3758521748 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f37780a2b80 0x7f37780a5040 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:46.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:46.650+0000 7f3798200640 1 --2- 172.21.15.5:0/3758521748 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f3790106f10 0x7f3790074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:46.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:46.650+0000 7f3798200640 1 --2- 172.21.15.5:0/3758521748 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f3790102790 0x7f3790074a30 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:46.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:46.651+0000 7f3798200640 1 --2- 172.21.15.5:0/3758521748 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3790101de0 0x7f37900744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:46.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:46.651+0000 7f3798200640 1 -- 172.21.15.5:0/3758521748 >> 172.21.15.5:0/3758521748 conn(0x7f37900fd8c0 msgr2=0x7f37900fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:46.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:46.651+0000 7f3798200640 1 -- 172.21.15.5:0/3758521748 shutdown_connections 2024-09-10T10:04:46.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:46.651+0000 7f3798200640 1 -- 172.21.15.5:0/3758521748 wait complete. 2024-09-10T10:04:47.464 ERROR:tasks.fwd_scrub.fs.[cephfs]:exception: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph-c_1cf819b7212a6048e92fc78c4eff963344f6d56b/qa/tasks/fwd_scrub.py", line 35, in _run self.do_scrub() File "/home/teuthworker/src/git.ceph.com_ceph-c_1cf819b7212a6048e92fc78c4eff963344f6d56b/qa/tasks/fwd_scrub.py", line 49, in do_scrub self.sleep_unless_stopped(self.sleep_between_iterations) File "/home/teuthworker/src/git.ceph.com_ceph-c_1cf819b7212a6048e92fc78c4eff963344f6d56b/qa/tasks/thrasher.py", line 47, in sleep_unless_stopped raise self.Stopped() tasks.thrasher.ThrasherGreenlet.Stopped 2024-09-10T10:04:47.465 INFO:tasks.fwd_scrub:done joining 2024-09-10T10:04:47.465 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-09-10T10:04:47.475 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T10:04:47.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:47.635+0000 7fa169600640 1 -- 172.21.15.5:0/4067707753 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa164074fc0 msgr2=0x7fa16410df40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:47.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:47.635+0000 7fa169600640 1 --2- 172.21.15.5:0/4067707753 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa164074fc0 0x7fa16410df40 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7fa14c009900 tx=0x7fa14c02a3e0 comp rx=0 tx=0).stop 2024-09-10T10:04:47.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:47.635+0000 7fa169600640 1 -- 172.21.15.5:0/4067707753 shutdown_connections 2024-09-10T10:04:47.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:47.635+0000 7fa169600640 1 --2- 172.21.15.5:0/4067707753 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa164105340 0x7fa164110490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:47.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:47.635+0000 7fa169600640 1 --2- 172.21.15.5:0/4067707753 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa164074fc0 0x7fa16410df40 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:47.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:47.635+0000 7fa169600640 1 --2- 172.21.15.5:0/4067707753 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa164074680 0x7fa164074a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:47.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:47.635+0000 7fa169600640 1 -- 172.21.15.5:0/4067707753 >> 172.21.15.5:0/4067707753 conn(0x7fa1640fd110 msgr2=0x7fa1640ff530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:47.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:47.635+0000 7fa169600640 1 -- 172.21.15.5:0/4067707753 shutdown_connections 2024-09-10T10:04:47.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:47.635+0000 7fa169600640 1 -- 172.21.15.5:0/4067707753 wait complete. 2024-09-10T10:04:47.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:47.636+0000 7fa169600640 1 Processor -- start 2024-09-10T10:04:47.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:47.636+0000 7fa169600640 1 -- start start 2024-09-10T10:04:47.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:47.637+0000 7fa169600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa164074680 0x7fa16419e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:47.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:47.637+0000 7fa169600640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa164074fc0 0x7fa16419e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:47.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:47.637+0000 7fa169600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa1641a3940 0x7fa1641a5d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:47.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:47.637+0000 7fa169600640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa164113d90 con 0x7fa1641a3940 2024-09-10T10:04:47.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:47.637+0000 7fa169600640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fa164113c10 con 0x7fa164074680 2024-09-10T10:04:47.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:47.637+0000 7fa169600640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fa164113f10 con 0x7fa164074fc0 2024-09-10T10:04:47.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:47.637+0000 7fa168c00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa1641a3940 0x7fa1641a5d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:47.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:47.637+0000 7fa168c00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa1641a3940 0x7fa1641a5d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40442/0 (socket says 172.21.15.5:40442) 2024-09-10T10:04:47.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:47.637+0000 7fa15b400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa164074fc0 0x7fa16419e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:47.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:47.637+0000 7fa168c00640 1 -- 172.21.15.5:0/1218848762 learned_addr learned my addr 172.21.15.5:0/1218848762 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T10:04:47.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:47.637+0000 7fa163e00640 1 --2- 172.21.15.5:0/1218848762 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa164074680 0x7fa16419e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:47.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:47.637+0000 7fa168c00640 1 -- 172.21.15.5:0/1218848762 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa164074fc0 msgr2=0x7fa16419e8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:47.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:47.637+0000 7fa168c00640 1 --2- 172.21.15.5:0/1218848762 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa164074fc0 0x7fa16419e8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:47.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:47.637+0000 7fa168c00640 1 -- 172.21.15.5:0/1218848762 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa164074680 msgr2=0x7fa16419e3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:47.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:47.637+0000 7fa168c00640 1 --2- 172.21.15.5:0/1218848762 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa164074680 0x7fa16419e3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:47.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:47.637+0000 7fa168c00640 1 -- 172.21.15.5:0/1218848762 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1641a63a0 con 0x7fa1641a3940 2024-09-10T10:04:47.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:47.637+0000 7fa163e00640 1 --2- 172.21.15.5:0/1218848762 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa164074680 0x7fa16419e3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:04:47.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:47.638+0000 7fa168c00640 1 --2- 172.21.15.5:0/1218848762 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa1641a3940 0x7fa1641a5d30 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7fa15c00eb10 tx=0x7fa15c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:47.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:47.638+0000 7fa161600640 1 -- 172.21.15.5:0/1218848762 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa15c004b20 con 0x7fa1641a3940 2024-09-10T10:04:47.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:47.638+0000 7fa161600640 1 -- 172.21.15.5:0/1218848762 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa15c004cc0 con 0x7fa1641a3940 2024-09-10T10:04:47.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:47.638+0000 7fa169600640 1 -- 172.21.15.5:0/1218848762 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1641a6690 con 0x7fa1641a3940 2024-09-10T10:04:47.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:47.638+0000 7fa161600640 1 -- 172.21.15.5:0/1218848762 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa15c010760 con 0x7fa1641a3940 2024-09-10T10:04:47.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:47.638+0000 7fa169600640 1 -- 172.21.15.5:0/1218848762 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1641a6b60 con 0x7fa1641a3940 2024-09-10T10:04:47.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:47.639+0000 7fa169600640 1 -- 172.21.15.5:0/1218848762 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa164074a80 con 0x7fa1641a3940 2024-09-10T10:04:47.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:47.641+0000 7fa161600640 1 -- 172.21.15.5:0/1218848762 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7fa15c005020 con 0x7fa1641a3940 2024-09-10T10:04:47.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:47.642+0000 7fa161600640 1 --2- 172.21.15.5:0/1218848762 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fa1480a2b90 0x7fa1480a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:47.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:47.642+0000 7fa161600640 1 -- 172.21.15.5:0/1218848762 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fa15c0cfa40 con 0x7fa1641a3940 2024-09-10T10:04:47.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:47.643+0000 7fa163e00640 1 --2- 172.21.15.5:0/1218848762 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fa1480a2b90 0x7fa1480a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:47.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:47.643+0000 7fa163e00640 1 --2- 172.21.15.5:0/1218848762 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fa1480a2b90 0x7fa1480a5050 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fa154009fc0 tx=0x7fa154005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:47.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:47.646+0000 7fa161600640 1 -- 172.21.15.5:0/1218848762 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa15c09c550 con 0x7fa1641a3940 2024-09-10T10:04:47.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:47.834+0000 7fa169600640 1 -- 172.21.15.5:0/1218848762 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa16402d7c0 con 0x7fa1641a3940 2024-09-10T10:04:47.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:47.835+0000 7fa161600640 1 -- 172.21.15.5:0/1218848762 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v96) ==== 74+0+24846 (secure 0 0 0) 0x7fa15c09be50 con 0x7fa1641a3940 2024-09-10T10:04:47.837 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-10T10:04:47.837 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":96,"fsid":"bade4b50-6f56-11ef-bcea-c7b262605968","created":"2024-09-10T09:27:13.298498+0000","modified":"2024-09-10T09:37:07.520520+0000","last_up_change":"2024-09-10T09:34:30.730066+0000","last_in_change":"2024-09-10T09:34:07.340995+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T09:30:26.042255+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T09:35:24.461031+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T09:35:27.155349+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-10T09:35:35.121462+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"085749a8-e882-4377-92dd-34bb266c0f0b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":954535266},{"type":"v1","addr":"172.21.15.5:6803","nonce":954535266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":954535266},{"type":"v1","addr":"172.21.15.5:6805","nonce":954535266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":954535266},{"type":"v1","addr":"172.21.15.5:6809","nonce":954535266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":954535266},{"type":"v1","addr":"172.21.15.5:6807","nonce":954535266}]},"public_addr":"172.21.15.5:6803/954535266","cluster_addr":"172.21.15.5:6805/954535266","heartbeat_back_addr":"172.21.15.5:6809/954535266","heartbeat_front_addr":"172.21.15.5:6807/954535266","state":["exists","up"]},{"osd":1,"uuid":"0a76c795-2573-4a92-969a-2ec5dd65c094","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6800","nonce":733924375},{"type":"v1","addr":"172.21.15.102:6801","nonce":733924375}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6802","nonce":733924375},{"type":"v1","addr":"172.21.15.102:6803","nonce":733924375}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6806","nonce":733924375},{"type":"v1","addr":"172.21.15.102:6807","nonce":733924375}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6804","nonce":733924375},{"type":"v1","addr":"172.21.15.102:6805","nonce":733924375}]},"public_addr":"172.21.15.102:6801/733924375","cluster_addr":"172.21.15.102:6803/733924375","heartbeat_back_addr":"172.21.15.102:6807/733924375","heartbeat_front_addr":"172.21.15.102:6805/733924375","state":["exists","up"]},{"osd":2,"uuid":"0c98fcb8-60ab-4473-8995-1d9e97c577e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6800","nonce":2236204189},{"type":"v1","addr":"172.21.15.159:6801","nonce":2236204189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":2236204189},{"type":"v1","addr":"172.21.15.159:6803","nonce":2236204189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":2236204189},{"type":"v1","addr":"172.21.15.159:6807","nonce":2236204189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":2236204189},{"type":"v1","addr":"172.21.15.159:6805","nonce":2236204189}]},"public_addr":"172.21.15.159:6801/2236204189","cluster_addr":"172.21.15.159:6803/2236204189","heartbeat_back_addr":"172.21.15.159:6807/2236204189","heartbeat_front_addr":"172.21.15.159:6805/2236204189","state":["exists","up"]},{"osd":3,"uuid":"88eecdeb-5cee-4f78-a3f9-a66bf4bf772a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":2221468725},{"type":"v1","addr":"172.21.15.5:6811","nonce":2221468725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":2221468725},{"type":"v1","addr":"172.21.15.5:6813","nonce":2221468725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6816","nonce":2221468725},{"type":"v1","addr":"172.21.15.5:6817","nonce":2221468725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":2221468725},{"type":"v1","addr":"172.21.15.5:6815","nonce":2221468725}]},"public_addr":"172.21.15.5:6811/2221468725","cluster_addr":"172.21.15.5:6813/2221468725","heartbeat_back_addr":"172.21.15.5:6817/2221468725","heartbeat_front_addr":"172.21.15.5:6815/2221468725","state":["exists","up"]},{"osd":4,"uuid":"f2e7314d-7938-4668-9011-45c175f66b48","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6808","nonce":594465753},{"type":"v1","addr":"172.21.15.102:6809","nonce":594465753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6810","nonce":594465753},{"type":"v1","addr":"172.21.15.102:6811","nonce":594465753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6814","nonce":594465753},{"type":"v1","addr":"172.21.15.102:6815","nonce":594465753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6812","nonce":594465753},{"type":"v1","addr":"172.21.15.102:6813","nonce":594465753}]},"public_addr":"172.21.15.102:6809/594465753","cluster_addr":"172.21.15.102:6811/594465753","heartbeat_back_addr":"172.21.15.102:6815/594465753","heartbeat_front_addr":"172.21.15.102:6813/594465753","state":["exists","up"]},{"osd":5,"uuid":"0c0366d3-658b-4bf0-ba17-aff28a02e948","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":3230280402},{"type":"v1","addr":"172.21.15.159:6809","nonce":3230280402}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":3230280402},{"type":"v1","addr":"172.21.15.159:6811","nonce":3230280402}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":3230280402},{"type":"v1","addr":"172.21.15.159:6815","nonce":3230280402}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":3230280402},{"type":"v1","addr":"172.21.15.159:6813","nonce":3230280402}]},"public_addr":"172.21.15.159:6809/3230280402","cluster_addr":"172.21.15.159:6811/3230280402","heartbeat_back_addr":"172.21.15.159:6815/3230280402","heartbeat_front_addr":"172.21.15.159:6813/3230280402","state":["exists","up"]},{"osd":6,"uuid":"682670a6-51d5-46e8-af6e-dc2d8c1cf407","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":1907651140},{"type":"v1","addr":"172.21.15.5:6819","nonce":1907651140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":1907651140},{"type":"v1","addr":"172.21.15.5:6821","nonce":1907651140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":1907651140},{"type":"v1","addr":"172.21.15.5:6825","nonce":1907651140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":1907651140},{"type":"v1","addr":"172.21.15.5:6823","nonce":1907651140}]},"public_addr":"172.21.15.5:6819/1907651140","cluster_addr":"172.21.15.5:6821/1907651140","heartbeat_back_addr":"172.21.15.5:6825/1907651140","heartbeat_front_addr":"172.21.15.5:6823/1907651140","state":["exists","up"]},{"osd":7,"uuid":"8ae0f4a9-99e7-4563-aa2a-15169684a8b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6816","nonce":692040360},{"type":"v1","addr":"172.21.15.102:6817","nonce":692040360}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6818","nonce":692040360},{"type":"v1","addr":"172.21.15.102:6819","nonce":692040360}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6822","nonce":692040360},{"type":"v1","addr":"172.21.15.102:6823","nonce":692040360}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6820","nonce":692040360},{"type":"v1","addr":"172.21.15.102:6821","nonce":692040360}]},"public_addr":"172.21.15.102:6817/692040360","cluster_addr":"172.21.15.102:6819/692040360","heartbeat_back_addr":"172.21.15.102:6823/692040360","heartbeat_front_addr":"172.21.15.102:6821/692040360","state":["exists","up"]},{"osd":8,"uuid":"e37c845e-ced9-4413-9bbe-bc090537c2bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":2107291799},{"type":"v1","addr":"172.21.15.159:6817","nonce":2107291799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":2107291799},{"type":"v1","addr":"172.21.15.159:6819","nonce":2107291799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":2107291799},{"type":"v1","addr":"172.21.15.159:6823","nonce":2107291799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":2107291799},{"type":"v1","addr":"172.21.15.159:6821","nonce":2107291799}]},"public_addr":"172.21.15.159:6817/2107291799","cluster_addr":"172.21.15.159:6819/2107291799","heartbeat_back_addr":"172.21.15.159:6823/2107291799","heartbeat_front_addr":"172.21.15.159:6821/2107291799","state":["exists","up"]},{"osd":9,"uuid":"91a5b6cc-08da-47be-af12-d0896a849309","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":902431281},{"type":"v1","addr":"172.21.15.5:6827","nonce":902431281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":902431281},{"type":"v1","addr":"172.21.15.5:6829","nonce":902431281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":902431281},{"type":"v1","addr":"172.21.15.5:6833","nonce":902431281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":902431281},{"type":"v1","addr":"172.21.15.5:6831","nonce":902431281}]},"public_addr":"172.21.15.5:6827/902431281","cluster_addr":"172.21.15.5:6829/902431281","heartbeat_back_addr":"172.21.15.5:6833/902431281","heartbeat_front_addr":"172.21.15.5:6831/902431281","state":["exists","up"]},{"osd":10,"uuid":"2211030c-b00e-402b-b4d7-c587eb29a490","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6824","nonce":2853806211},{"type":"v1","addr":"172.21.15.102:6825","nonce":2853806211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6826","nonce":2853806211},{"type":"v1","addr":"172.21.15.102:6827","nonce":2853806211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6830","nonce":2853806211},{"type":"v1","addr":"172.21.15.102:6831","nonce":2853806211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6828","nonce":2853806211},{"type":"v1","addr":"172.21.15.102:6829","nonce":2853806211}]},"public_addr":"172.21.15.102:6825/2853806211","cluster_addr":"172.21.15.102:6827/2853806211","heartbeat_back_addr":"172.21.15.102:6831/2853806211","heartbeat_front_addr":"172.21.15.102:6829/2853806211","state":["exists","up"]},{"osd":11,"uuid":"4e9699ec-8ff2-4005-9d84-af5600b3a2ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6824","nonce":969941218},{"type":"v1","addr":"172.21.15.159:6825","nonce":969941218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":969941218},{"type":"v1","addr":"172.21.15.159:6827","nonce":969941218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6830","nonce":969941218},{"type":"v1","addr":"172.21.15.159:6831","nonce":969941218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":969941218},{"type":"v1","addr":"172.21.15.159:6829","nonce":969941218}]},"public_addr":"172.21.15.159:6825/969941218","cluster_addr":"172.21.15.159:6827/969941218","heartbeat_back_addr":"172.21.15.159:6831/969941218","heartbeat_front_addr":"172.21.15.159:6829/969941218","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:29:34.416490+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:30:00.246101+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:30:24.300464+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:30:54.663623+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:31:21.426292+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:31:46.738914+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:32:16.196571+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:32:42.549243+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:33:08.178651+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:33:37.477911+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:34:03.114026+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:34:28.891924+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4}]},{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"3.0","mappings":[{"from":4,"to":8}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":6}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9},{"from":5,"to":7}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]},{"pgid":"3.39","mappings":[{"from":5,"to":6}]},{"pgid":"4.8","mappings":[{"from":5,"to":8}]},{"pgid":"4.c","mappings":[{"from":9,"to":10}]},{"pgid":"4.18","mappings":[{"from":9,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.5:6801/1527422686":"2024-09-11T09:28:05.436742+0000","172.21.15.5:0/2276144345":"2024-09-11T09:28:05.436742+0000","172.21.15.5:0/412040104":"2024-09-11T09:28:05.436742+0000","172.21.15.5:6800/1527422686":"2024-09-11T09:28:05.436742+0000","172.21.15.5:0/1856581170":"2024-09-11T09:28:05.436742+0000","172.21.15.5:0/2847668298":"2024-09-11T09:27:37.778500+0000","172.21.15.5:6801/14425913":"2024-09-11T09:27:37.778500+0000","172.21.15.5:0/166910782":"2024-09-11T09:27:37.778500+0000","172.21.15.5:6800/14425913":"2024-09-11T09:27:37.778500+0000","172.21.15.5:0/1711858179":"2024-09-11T09:27:37.778500+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T10:04:47.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:47.838+0000 7fa169600640 1 -- 172.21.15.5:0/1218848762 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fa1480a2b90 msgr2=0x7fa1480a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:47.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:47.838+0000 7fa169600640 1 --2- 172.21.15.5:0/1218848762 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fa1480a2b90 0x7fa1480a5050 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fa154009fc0 tx=0x7fa154005f90 comp rx=0 tx=0).stop 2024-09-10T10:04:47.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:47.838+0000 7fa169600640 1 -- 172.21.15.5:0/1218848762 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa1641a3940 msgr2=0x7fa1641a5d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:47.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:47.838+0000 7fa169600640 1 --2- 172.21.15.5:0/1218848762 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa1641a3940 0x7fa1641a5d30 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7fa15c00eb10 tx=0x7fa15c00c2d0 comp rx=0 tx=0).stop 2024-09-10T10:04:47.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:47.838+0000 7fa169600640 1 -- 172.21.15.5:0/1218848762 shutdown_connections 2024-09-10T10:04:47.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:47.838+0000 7fa169600640 1 --2- 172.21.15.5:0/1218848762 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fa1480a2b90 0x7fa1480a5050 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:47.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:47.838+0000 7fa169600640 1 --2- 172.21.15.5:0/1218848762 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa1641a3940 0x7fa1641a5d30 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:47.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:47.838+0000 7fa169600640 1 --2- 172.21.15.5:0/1218848762 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fa164074fc0 0x7fa16419e8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:47.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:47.838+0000 7fa169600640 1 --2- 172.21.15.5:0/1218848762 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fa164074680 0x7fa16419e3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:47.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:47.838+0000 7fa169600640 1 -- 172.21.15.5:0/1218848762 >> 172.21.15.5:0/1218848762 conn(0x7fa1640fd110 msgr2=0x7fa1641100f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:47.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:47.838+0000 7fa169600640 1 -- 172.21.15.5:0/1218848762 shutdown_connections 2024-09-10T10:04:47.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:47.838+0000 7fa169600640 1 -- 172.21.15.5:0/1218848762 wait complete. 2024-09-10T10:04:47.855 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-10T09:30:26.042255+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-10T09:35:24.461031+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-10T09:35:27.155349+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-09-10T09:35:35.121462+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '92', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-09-10T10:04:47.856 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-10T10:04:48.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:48.017+0000 7ff8d2600640 1 -- 172.21.15.5:0/3378325404 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff8cc103d30 msgr2=0x7ff8cc104130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:48.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:48.017+0000 7ff8d2600640 1 --2- 172.21.15.5:0/3378325404 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff8cc103d30 0x7ff8cc104130 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7ff8bc009900 tx=0x7ff8bc02a410 comp rx=0 tx=0).stop 2024-09-10T10:04:48.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:48.017+0000 7ff8d2600640 1 -- 172.21.15.5:0/3378325404 shutdown_connections 2024-09-10T10:04:48.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:48.017+0000 7ff8d2600640 1 --2- 172.21.15.5:0/3378325404 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff8cc1058e0 0x7ff8cc10c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:48.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:48.017+0000 7ff8d2600640 1 --2- 172.21.15.5:0/3378325404 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff8cc104f20 0x7ff8cc1053a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:48.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:48.017+0000 7ff8d2600640 1 --2- 172.21.15.5:0/3378325404 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff8cc103d30 0x7ff8cc104130 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:48.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:48.017+0000 7ff8d2600640 1 -- 172.21.15.5:0/3378325404 >> 172.21.15.5:0/3378325404 conn(0x7ff8cc0ff470 msgr2=0x7ff8cc1018d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:48.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:48.018+0000 7ff8d2600640 1 -- 172.21.15.5:0/3378325404 shutdown_connections 2024-09-10T10:04:48.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:48.018+0000 7ff8d2600640 1 -- 172.21.15.5:0/3378325404 wait complete. 2024-09-10T10:04:48.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:48.018+0000 7ff8d2600640 1 Processor -- start 2024-09-10T10:04:48.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:48.018+0000 7ff8d2600640 1 -- start start 2024-09-10T10:04:48.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:48.019+0000 7ff8d2600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff8cc103d30 0x7ff8cc19a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:48.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:48.019+0000 7ff8d2600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff8cc104f20 0x7ff8cc19a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:48.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:48.019+0000 7ff8d2600640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff8cc1058e0 0x7ff8cc1a1670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:48.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:48.019+0000 7ff8d2600640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff8cc10f9e0 con 0x7ff8cc104f20 2024-09-10T10:04:48.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:48.019+0000 7ff8d2600640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7ff8cc10f860 con 0x7ff8cc103d30 2024-09-10T10:04:48.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:48.019+0000 7ff8d2600640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7ff8cc10fb60 con 0x7ff8cc1058e0 2024-09-10T10:04:48.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:48.019+0000 7ff8c3e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff8cc104f20 0x7ff8cc19a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:48.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:48.019+0000 7ff8c3e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff8cc104f20 0x7ff8cc19a600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40454/0 (socket says 172.21.15.5:40454) 2024-09-10T10:04:48.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:48.019+0000 7ff8c3e00640 1 -- 172.21.15.5:0/75174731 learned_addr learned my addr 172.21.15.5:0/75174731 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T10:04:48.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:48.019+0000 7ff8cb400640 1 --2- 172.21.15.5:0/75174731 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff8cc103d30 0x7ff8cc19a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:48.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:48.019+0000 7ff8cbe00640 1 --2- 172.21.15.5:0/75174731 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff8cc1058e0 0x7ff8cc1a1670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:48.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:48.019+0000 7ff8c3e00640 1 -- 172.21.15.5:0/75174731 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff8cc1058e0 msgr2=0x7ff8cc1a1670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:48.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:48.019+0000 7ff8c3e00640 1 --2- 172.21.15.5:0/75174731 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff8cc1058e0 0x7ff8cc1a1670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:48.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:48.019+0000 7ff8c3e00640 1 -- 172.21.15.5:0/75174731 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff8cc103d30 msgr2=0x7ff8cc19a0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:48.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:48.019+0000 7ff8c3e00640 1 --2- 172.21.15.5:0/75174731 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff8cc103d30 0x7ff8cc19a0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:48.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:48.019+0000 7ff8c3e00640 1 -- 172.21.15.5:0/75174731 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8cc1a1ce0 con 0x7ff8cc104f20 2024-09-10T10:04:48.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:48.019+0000 7ff8cb400640 1 --2- 172.21.15.5:0/75174731 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff8cc103d30 0x7ff8cc19a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:04:48.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:48.020+0000 7ff8c3e00640 1 --2- 172.21.15.5:0/75174731 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff8cc104f20 0x7ff8cc19a600 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7ff8b400e9f0 tx=0x7ff8b400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:48.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:48.020+0000 7ff8c8c00640 1 -- 172.21.15.5:0/75174731 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8b400cd80 con 0x7ff8cc104f20 2024-09-10T10:04:48.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:48.020+0000 7ff8c8c00640 1 -- 172.21.15.5:0/75174731 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff8b4010910 con 0x7ff8cc104f20 2024-09-10T10:04:48.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:48.020+0000 7ff8c8c00640 1 -- 172.21.15.5:0/75174731 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8b4018990 con 0x7ff8cc104f20 2024-09-10T10:04:48.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:48.020+0000 7ff8d2600640 1 -- 172.21.15.5:0/75174731 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8cc1a1fd0 con 0x7ff8cc104f20 2024-09-10T10:04:48.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:48.020+0000 7ff8d2600640 1 -- 172.21.15.5:0/75174731 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8cc1a2440 con 0x7ff8cc104f20 2024-09-10T10:04:48.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:48.021+0000 7ff8d2600640 1 -- 172.21.15.5:0/75174731 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff8cc10b170 con 0x7ff8cc104f20 2024-09-10T10:04:48.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:48.023+0000 7ff8c8c00640 1 -- 172.21.15.5:0/75174731 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7ff8b4010ac0 con 0x7ff8cc104f20 2024-09-10T10:04:48.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:48.024+0000 7ff8c8c00640 1 --2- 172.21.15.5:0/75174731 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7ff8b00a2c10 0x7ff8b00a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:48.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:48.024+0000 7ff8c8c00640 1 -- 172.21.15.5:0/75174731 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7ff8b40cf710 con 0x7ff8cc104f20 2024-09-10T10:04:48.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:48.025+0000 7ff8cb400640 1 --2- 172.21.15.5:0/75174731 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7ff8b00a2c10 0x7ff8b00a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:48.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:48.025+0000 7ff8cb400640 1 --2- 172.21.15.5:0/75174731 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7ff8b00a2c10 0x7ff8b00a50d0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7ff8bc02aa00 tx=0x7ff8bc0315b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:48.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:48.029+0000 7ff8c8c00640 1 -- 172.21.15.5:0/75174731 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff8b409bbd0 con 0x7ff8cc104f20 2024-09-10T10:04:48.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:04:47 smithi159 ceph-mon[35513]: pgmap v1186: 129 pgs: 129 active+clean; 551 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 204 KiB/s rd, 1.8 MiB/s wr, 117 op/s 2024-09-10T10:04:48.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:04:47 smithi102 ceph-mon[35442]: pgmap v1186: 129 pgs: 129 active+clean; 551 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 204 KiB/s rd, 1.8 MiB/s wr, 117 op/s 2024-09-10T10:04:48.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:48.213+0000 7ff8d2600640 1 -- 172.21.15.5:0/75174731 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7ff8cc062e90 con 0x7ff8cc104f20 2024-09-10T10:04:48.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:48.213+0000 7ff8c8c00640 1 -- 172.21.15.5:0/75174731 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v96) ==== 93+0+10 (secure 0 0 0) 0x7ff8b409fed0 con 0x7ff8cc104f20 2024-09-10T10:04:48.215 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 1 2024-09-10T10:04:48.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:48.215+0000 7ff8d2600640 1 -- 172.21.15.5:0/75174731 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7ff8b00a2c10 msgr2=0x7ff8b00a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:48.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:48.215+0000 7ff8d2600640 1 --2- 172.21.15.5:0/75174731 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7ff8b00a2c10 0x7ff8b00a50d0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7ff8bc02aa00 tx=0x7ff8bc0315b0 comp rx=0 tx=0).stop 2024-09-10T10:04:48.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:48.215+0000 7ff8d2600640 1 -- 172.21.15.5:0/75174731 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff8cc104f20 msgr2=0x7ff8cc19a600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:48.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:48.215+0000 7ff8d2600640 1 --2- 172.21.15.5:0/75174731 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff8cc104f20 0x7ff8cc19a600 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7ff8b400e9f0 tx=0x7ff8b400eeb0 comp rx=0 tx=0).stop 2024-09-10T10:04:48.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:48.215+0000 7ff8d2600640 1 -- 172.21.15.5:0/75174731 shutdown_connections 2024-09-10T10:04:48.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:48.215+0000 7ff8d2600640 1 --2- 172.21.15.5:0/75174731 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7ff8b00a2c10 0x7ff8b00a50d0 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:48.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:48.215+0000 7ff8d2600640 1 --2- 172.21.15.5:0/75174731 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff8cc1058e0 0x7ff8cc1a1670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:48.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:48.215+0000 7ff8d2600640 1 --2- 172.21.15.5:0/75174731 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff8cc104f20 0x7ff8cc19a600 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:48.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:48.216+0000 7ff8d2600640 1 --2- 172.21.15.5:0/75174731 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff8cc103d30 0x7ff8cc19a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:48.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:48.216+0000 7ff8d2600640 1 -- 172.21.15.5:0/75174731 >> 172.21.15.5:0/75174731 conn(0x7ff8cc0ff470 msgr2=0x7ff8cc108340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:48.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:48.216+0000 7ff8d2600640 1 -- 172.21.15.5:0/75174731 shutdown_connections 2024-09-10T10:04:48.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:48.216+0000 7ff8d2600640 1 -- 172.21.15.5:0/75174731 wait complete. 2024-09-10T10:04:48.231 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:04:47 smithi005 ceph-mon[30350]: pgmap v1186: 129 pgs: 129 active+clean; 551 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 204 KiB/s rd, 1.8 MiB/s wr, 117 op/s 2024-09-10T10:04:48.234 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-10T10:04:48.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:48.393+0000 7fe3c5200640 1 -- 172.21.15.5:0/4205432930 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe3c0109990 msgr2=0x7fe3c010be50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:48.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:48.393+0000 7fe3c5200640 1 --2- 172.21.15.5:0/4205432930 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe3c0109990 0x7fe3c010be50 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7fe3b8009ca0 tx=0x7fe3b802a950 comp rx=0 tx=0).stop 2024-09-10T10:04:48.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:48.394+0000 7fe3c5200640 1 -- 172.21.15.5:0/4205432930 shutdown_connections 2024-09-10T10:04:48.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:48.394+0000 7fe3c5200640 1 --2- 172.21.15.5:0/4205432930 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe3c0109990 0x7fe3c010be50 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:48.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:48.394+0000 7fe3c5200640 1 --2- 172.21.15.5:0/4205432930 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe3c0074bf0 0x7fe3c0075050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:48.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:48.394+0000 7fe3c5200640 1 --2- 172.21.15.5:0/4205432930 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe3c00761d0 0x7fe3c0074620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:48.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:48.394+0000 7fe3c5200640 1 -- 172.21.15.5:0/4205432930 >> 172.21.15.5:0/4205432930 conn(0x7fe3c00fd2a0 msgr2=0x7fe3c00ff6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:48.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:48.394+0000 7fe3c5200640 1 -- 172.21.15.5:0/4205432930 shutdown_connections 2024-09-10T10:04:48.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:48.394+0000 7fe3c5200640 1 -- 172.21.15.5:0/4205432930 wait complete. 2024-09-10T10:04:48.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:48.395+0000 7fe3c5200640 1 Processor -- start 2024-09-10T10:04:48.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:48.395+0000 7fe3c5200640 1 -- start start 2024-09-10T10:04:48.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:48.395+0000 7fe3c5200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe3c0074bf0 0x7fe3c019a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:48.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:48.395+0000 7fe3c5200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe3c00761d0 0x7fe3c019a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:48.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:48.395+0000 7fe3c5200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe3c0109990 0x7fe3c01a15c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:48.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:48.395+0000 7fe3c5200640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe3c010f9e0 con 0x7fe3c00761d0 2024-09-10T10:04:48.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:48.395+0000 7fe3c5200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fe3c010f860 con 0x7fe3c0074bf0 2024-09-10T10:04:48.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:48.395+0000 7fe3c5200640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fe3c010fb60 con 0x7fe3c0109990 2024-09-10T10:04:48.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:48.395+0000 7fe3bd400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe3c00761d0 0x7fe3c019a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:48.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:48.395+0000 7fe3bd400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe3c00761d0 0x7fe3c019a550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40484/0 (socket says 172.21.15.5:40484) 2024-09-10T10:04:48.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:48.395+0000 7fe3bd400640 1 -- 172.21.15.5:0/2141921761 learned_addr learned my addr 172.21.15.5:0/2141921761 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T10:04:48.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:48.395+0000 7fe3bde00640 1 --2- 172.21.15.5:0/2141921761 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe3c0074bf0 0x7fe3c019a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:48.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:48.395+0000 7fe3be800640 1 --2- 172.21.15.5:0/2141921761 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe3c0109990 0x7fe3c01a15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:48.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:48.396+0000 7fe3bd400640 1 -- 172.21.15.5:0/2141921761 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe3c0109990 msgr2=0x7fe3c01a15c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:48.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:48.396+0000 7fe3bd400640 1 --2- 172.21.15.5:0/2141921761 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe3c0109990 0x7fe3c01a15c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:48.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:48.396+0000 7fe3bd400640 1 -- 172.21.15.5:0/2141921761 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe3c0074bf0 msgr2=0x7fe3c019a010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:48.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:48.396+0000 7fe3bd400640 1 --2- 172.21.15.5:0/2141921761 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe3c0074bf0 0x7fe3c019a010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:48.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:48.396+0000 7fe3bd400640 1 -- 172.21.15.5:0/2141921761 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3c01a1c30 con 0x7fe3c00761d0 2024-09-10T10:04:48.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:48.396+0000 7fe3be800640 1 --2- 172.21.15.5:0/2141921761 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe3c0109990 0x7fe3c01a15c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:04:48.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:48.396+0000 7fe3bd400640 1 --2- 172.21.15.5:0/2141921761 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe3c00761d0 0x7fe3c019a550 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7fe3ac00ea60 tx=0x7fe3ac00ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:48.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:48.396+0000 7fe3b6a00640 1 -- 172.21.15.5:0/2141921761 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3ac00cde0 con 0x7fe3c00761d0 2024-09-10T10:04:48.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:48.396+0000 7fe3b6a00640 1 -- 172.21.15.5:0/2141921761 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe3ac0035f0 con 0x7fe3c00761d0 2024-09-10T10:04:48.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:48.396+0000 7fe3b6a00640 1 -- 172.21.15.5:0/2141921761 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3ac010670 con 0x7fe3c00761d0 2024-09-10T10:04:48.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:48.396+0000 7fe3c5200640 1 -- 172.21.15.5:0/2141921761 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3c01a1f20 con 0x7fe3c00761d0 2024-09-10T10:04:48.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:48.396+0000 7fe3c5200640 1 -- 172.21.15.5:0/2141921761 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3c01a2390 con 0x7fe3c00761d0 2024-09-10T10:04:48.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:48.397+0000 7fe3c5200640 1 -- 172.21.15.5:0/2141921761 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe3c0108750 con 0x7fe3c00761d0 2024-09-10T10:04:48.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:48.399+0000 7fe3b6a00640 1 -- 172.21.15.5:0/2141921761 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7fe3ac0037a0 con 0x7fe3c00761d0 2024-09-10T10:04:48.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:48.400+0000 7fe3b6a00640 1 --2- 172.21.15.5:0/2141921761 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fe3a40a2b90 0x7fe3a40a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:48.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:48.401+0000 7fe3b6a00640 1 -- 172.21.15.5:0/2141921761 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fe3ac0cf810 con 0x7fe3c00761d0 2024-09-10T10:04:48.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:48.401+0000 7fe3bde00640 1 --2- 172.21.15.5:0/2141921761 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fe3a40a2b90 0x7fe3a40a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:48.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:48.402+0000 7fe3bde00640 1 --2- 172.21.15.5:0/2141921761 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fe3a40a2b90 0x7fe3a40a5050 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fe3b0009bd0 tx=0x7fe3b0005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:48.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:48.405+0000 7fe3b6a00640 1 -- 172.21.15.5:0/2141921761 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe3ac09c320 con 0x7fe3c00761d0 2024-09-10T10:04:48.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:48.588+0000 7fe3c5200640 1 -- 172.21.15.5:0/2141921761 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fe3c0062e90 con 0x7fe3c00761d0 2024-09-10T10:04:48.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:48.589+0000 7fe3b6a00640 1 -- 172.21.15.5:0/2141921761 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v96) ==== 104+0+11 (secure 0 0 0) 0x7fe3ac086020 con 0x7fe3c00761d0 2024-09-10T10:04:48.590 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 32 2024-09-10T10:04:48.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:48.591+0000 7fe3c5200640 1 -- 172.21.15.5:0/2141921761 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fe3a40a2b90 msgr2=0x7fe3a40a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:48.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:48.591+0000 7fe3c5200640 1 --2- 172.21.15.5:0/2141921761 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fe3a40a2b90 0x7fe3a40a5050 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fe3b0009bd0 tx=0x7fe3b0005f90 comp rx=0 tx=0).stop 2024-09-10T10:04:48.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:48.591+0000 7fe3c5200640 1 -- 172.21.15.5:0/2141921761 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe3c00761d0 msgr2=0x7fe3c019a550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:48.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:48.591+0000 7fe3c5200640 1 --2- 172.21.15.5:0/2141921761 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe3c00761d0 0x7fe3c019a550 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7fe3ac00ea60 tx=0x7fe3ac00ef20 comp rx=0 tx=0).stop 2024-09-10T10:04:48.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:48.591+0000 7fe3c5200640 1 -- 172.21.15.5:0/2141921761 shutdown_connections 2024-09-10T10:04:48.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:48.591+0000 7fe3c5200640 1 --2- 172.21.15.5:0/2141921761 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fe3a40a2b90 0x7fe3a40a5050 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:48.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:48.591+0000 7fe3c5200640 1 --2- 172.21.15.5:0/2141921761 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fe3c0109990 0x7fe3c01a15c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:48.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:48.591+0000 7fe3c5200640 1 --2- 172.21.15.5:0/2141921761 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe3c00761d0 0x7fe3c019a550 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:48.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:48.591+0000 7fe3c5200640 1 --2- 172.21.15.5:0/2141921761 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fe3c0074bf0 0x7fe3c019a010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:48.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:48.591+0000 7fe3c5200640 1 -- 172.21.15.5:0/2141921761 >> 172.21.15.5:0/2141921761 conn(0x7fe3c00fd2a0 msgr2=0x7fe3c0105920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:48.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:48.591+0000 7fe3c5200640 1 -- 172.21.15.5:0/2141921761 shutdown_connections 2024-09-10T10:04:48.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:48.591+0000 7fe3c5200640 1 -- 172.21.15.5:0/2141921761 wait complete. 2024-09-10T10:04:48.608 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-10T10:04:48.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:48.772+0000 7f17fec00640 1 -- 172.21.15.5:0/3976327901 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f17f8104f10 msgr2=0x7f17f8105390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:48.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:48.772+0000 7f17fec00640 1 --2- 172.21.15.5:0/3976327901 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f17f8104f10 0x7f17f8105390 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f17e0009900 tx=0x7f17e002a410 comp rx=0 tx=0).stop 2024-09-10T10:04:48.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:48.772+0000 7f17fec00640 1 -- 172.21.15.5:0/3976327901 shutdown_connections 2024-09-10T10:04:48.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:48.772+0000 7f17fec00640 1 --2- 172.21.15.5:0/3976327901 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f17f81058d0 0x7f17f810c160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:48.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:48.772+0000 7f17fec00640 1 --2- 172.21.15.5:0/3976327901 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f17f8104f10 0x7f17f8105390 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:48.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:48.772+0000 7f17fec00640 1 --2- 172.21.15.5:0/3976327901 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f17f8103d20 0x7f17f8104120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:48.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:48.772+0000 7f17fec00640 1 -- 172.21.15.5:0/3976327901 >> 172.21.15.5:0/3976327901 conn(0x7f17f80ff4a0 msgr2=0x7f17f81018c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:48.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:48.772+0000 7f17fec00640 1 -- 172.21.15.5:0/3976327901 shutdown_connections 2024-09-10T10:04:48.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:48.772+0000 7f17fec00640 1 -- 172.21.15.5:0/3976327901 wait complete. 2024-09-10T10:04:48.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:48.773+0000 7f17fec00640 1 Processor -- start 2024-09-10T10:04:48.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:48.773+0000 7f17fec00640 1 -- start start 2024-09-10T10:04:48.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:48.773+0000 7f17fec00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f17f8103d20 0x7f17f819a070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:48.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:48.773+0000 7f17fec00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f17f8104f10 0x7f17f819a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:48.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:48.773+0000 7f17fec00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f17f81058d0 0x7f17f81a1620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:48.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:48.773+0000 7f17fec00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f17f810f9f0 con 0x7f17f8104f10 2024-09-10T10:04:48.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:48.773+0000 7f17fec00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f17f810f870 con 0x7f17f8103d20 2024-09-10T10:04:48.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:48.773+0000 7f17fec00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f17f810fb70 con 0x7f17f81058d0 2024-09-10T10:04:48.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:48.773+0000 7f17ef400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f17f8104f10 0x7f17f819a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:48.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:48.774+0000 7f17f7e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f17f8103d20 0x7f17f819a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:48.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:48.774+0000 7f17ef400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f17f8104f10 0x7f17f819a5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40498/0 (socket says 172.21.15.5:40498) 2024-09-10T10:04:48.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:48.774+0000 7f17ef400640 1 -- 172.21.15.5:0/3647544003 learned_addr learned my addr 172.21.15.5:0/3647544003 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T10:04:48.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:48.774+0000 7f17fcc00640 1 --2- 172.21.15.5:0/3647544003 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f17f81058d0 0x7f17f81a1620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:48.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:48.774+0000 7f17f7e00640 1 -- 172.21.15.5:0/3647544003 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f17f81058d0 msgr2=0x7f17f81a1620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:48.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:48.774+0000 7f17f7e00640 1 --2- 172.21.15.5:0/3647544003 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f17f81058d0 0x7f17f81a1620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:48.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:48.774+0000 7f17f7e00640 1 -- 172.21.15.5:0/3647544003 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f17f8104f10 msgr2=0x7f17f819a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:48.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:48.774+0000 7f17f7e00640 1 --2- 172.21.15.5:0/3647544003 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f17f8104f10 0x7f17f819a5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:48.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:48.774+0000 7f17f7e00640 1 -- 172.21.15.5:0/3647544003 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17f81a1c90 con 0x7f17f8103d20 2024-09-10T10:04:48.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:48.774+0000 7f17f7e00640 1 --2- 172.21.15.5:0/3647544003 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f17f8103d20 0x7f17f819a070 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f17e800e9c0 tx=0x7f17e800ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:48.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:48.774+0000 7f17f5600640 1 -- 172.21.15.5:0/3647544003 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17e800cd80 con 0x7f17f8103d20 2024-09-10T10:04:48.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:48.774+0000 7f17f5600640 1 -- 172.21.15.5:0/3647544003 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f17e8010910 con 0x7f17f8103d20 2024-09-10T10:04:48.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:48.774+0000 7f17fec00640 1 -- 172.21.15.5:0/3647544003 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17f81a1f80 con 0x7f17f8103d20 2024-09-10T10:04:48.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:48.774+0000 7f17f5600640 1 -- 172.21.15.5:0/3647544003 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17e8018980 con 0x7f17f8103d20 2024-09-10T10:04:48.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:48.775+0000 7f17fec00640 1 -- 172.21.15.5:0/3647544003 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17f81a23e0 con 0x7f17f8103d20 2024-09-10T10:04:48.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:48.775+0000 7f17fec00640 1 -- 172.21.15.5:0/3647544003 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f17f810b0c0 con 0x7f17f8103d20 2024-09-10T10:04:48.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:48.778+0000 7f17f5600640 1 -- 172.21.15.5:0/3647544003 <== mon.2 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f17e801f020 con 0x7f17f8103d20 2024-09-10T10:04:48.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:48.779+0000 7f17f5600640 1 --2- 172.21.15.5:0/3647544003 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f17dc0a2b90 0x7f17dc0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:48.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:48.779+0000 7f17f5600640 1 -- 172.21.15.5:0/3647544003 <== mon.2 v2:172.21.15.102:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f17e80cf9d0 con 0x7f17f8103d20 2024-09-10T10:04:48.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:48.779+0000 7f17ef400640 1 --2- 172.21.15.5:0/3647544003 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f17dc0a2b90 0x7f17dc0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:48.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:48.780+0000 7f17ef400640 1 --2- 172.21.15.5:0/3647544003 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f17dc0a2b90 0x7f17dc0a5050 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f17f819b580 tx=0x7f17e00315b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:48.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:48.781+0000 7f17f5600640 1 -- 172.21.15.5:0/3647544003 <== mon.2 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f17e809c4e0 con 0x7f17f8103d20 2024-09-10T10:04:48.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:48.970+0000 7f17fec00640 1 -- 172.21.15.5:0/3647544003 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f17f802d9e0 con 0x7f17f8103d20 2024-09-10T10:04:48.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:48.971+0000 7f17f5600640 1 -- 172.21.15.5:0/3647544003 <== mon.2 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v96) ==== 100+0+11 (secure 0 0 0) 0x7f17e809bde0 con 0x7f17f8103d20 2024-09-10T10:04:48.972 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 64 2024-09-10T10:04:48.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:48.973+0000 7f17fec00640 1 -- 172.21.15.5:0/3647544003 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f17dc0a2b90 msgr2=0x7f17dc0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:48.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:48.973+0000 7f17fec00640 1 --2- 172.21.15.5:0/3647544003 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f17dc0a2b90 0x7f17dc0a5050 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f17f819b580 tx=0x7f17e00315b0 comp rx=0 tx=0).stop 2024-09-10T10:04:48.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:48.973+0000 7f17fec00640 1 -- 172.21.15.5:0/3647544003 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f17f8103d20 msgr2=0x7f17f819a070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:48.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:48.973+0000 7f17fec00640 1 --2- 172.21.15.5:0/3647544003 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f17f8103d20 0x7f17f819a070 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f17e800e9c0 tx=0x7f17e800ee80 comp rx=0 tx=0).stop 2024-09-10T10:04:48.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:48.974+0000 7f17fec00640 1 -- 172.21.15.5:0/3647544003 shutdown_connections 2024-09-10T10:04:48.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:48.974+0000 7f17fec00640 1 --2- 172.21.15.5:0/3647544003 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f17dc0a2b90 0x7f17dc0a5050 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:48.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:48.974+0000 7f17fec00640 1 --2- 172.21.15.5:0/3647544003 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f17f81058d0 0x7f17f81a1620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:48.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:48.974+0000 7f17fec00640 1 --2- 172.21.15.5:0/3647544003 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f17f8104f10 0x7f17f819a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:48.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:48.974+0000 7f17fec00640 1 --2- 172.21.15.5:0/3647544003 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f17f8103d20 0x7f17f819a070 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:48.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:48.974+0000 7f17fec00640 1 -- 172.21.15.5:0/3647544003 >> 172.21.15.5:0/3647544003 conn(0x7f17f80ff4a0 msgr2=0x7f17f8108290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:48.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:48.974+0000 7f17fec00640 1 -- 172.21.15.5:0/3647544003 shutdown_connections 2024-09-10T10:04:48.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:48.974+0000 7f17fec00640 1 -- 172.21.15.5:0/3647544003 wait complete. 2024-09-10T10:04:48.990 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:04:48 smithi005 ceph-mon[30350]: from='client.? 172.21.15.5:0/1218848762' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T10:04:48.990 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:04:48 smithi005 ceph-mon[30350]: from='client.? 172.21.15.5:0/75174731' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-10T10:04:48.990 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:04:48 smithi005 ceph-mon[30350]: from='client.? 172.21.15.5:0/2141921761' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-10T10:04:48.992 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-09-10T10:04:49.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:04:48 smithi159 ceph-mon[35513]: from='client.? 172.21.15.5:0/1218848762' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T10:04:49.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:04:48 smithi159 ceph-mon[35513]: from='client.? 172.21.15.5:0/75174731' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-10T10:04:49.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:04:48 smithi159 ceph-mon[35513]: from='client.? 172.21.15.5:0/2141921761' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-10T10:04:49.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:04:48 smithi102 ceph-mon[35442]: from='client.? 172.21.15.5:0/1218848762' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T10:04:49.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:04:48 smithi102 ceph-mon[35442]: from='client.? 172.21.15.5:0/75174731' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-10T10:04:49.135 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:04:48 smithi102 ceph-mon[35442]: from='client.? 172.21.15.5:0/2141921761' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-10T10:04:49.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:49.152+0000 7f2ff3e00640 1 -- 172.21.15.5:0/3278384349 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2fec1058d0 msgr2=0x7f2fec10c160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:49.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:49.152+0000 7f2ff3e00640 1 --2- 172.21.15.5:0/3278384349 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2fec1058d0 0x7f2fec10c160 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f2fe4009900 tx=0x7f2fe402a400 comp rx=0 tx=0).stop 2024-09-10T10:04:49.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:49.153+0000 7f2ff3e00640 1 -- 172.21.15.5:0/3278384349 shutdown_connections 2024-09-10T10:04:49.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:49.153+0000 7f2ff3e00640 1 --2- 172.21.15.5:0/3278384349 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2fec1058d0 0x7f2fec10c160 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:49.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:49.153+0000 7f2ff3e00640 1 --2- 172.21.15.5:0/3278384349 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2fec104f10 0x7f2fec105390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:49.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:49.153+0000 7f2ff3e00640 1 --2- 172.21.15.5:0/3278384349 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2fec103d20 0x7f2fec104120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:49.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:49.153+0000 7f2ff3e00640 1 -- 172.21.15.5:0/3278384349 >> 172.21.15.5:0/3278384349 conn(0x7f2fec0ff4a0 msgr2=0x7f2fec1018c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:49.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:49.153+0000 7f2ff3e00640 1 -- 172.21.15.5:0/3278384349 shutdown_connections 2024-09-10T10:04:49.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:49.153+0000 7f2ff3e00640 1 -- 172.21.15.5:0/3278384349 wait complete. 2024-09-10T10:04:49.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:49.153+0000 7f2ff3e00640 1 Processor -- start 2024-09-10T10:04:49.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:49.154+0000 7f2ff3e00640 1 -- start start 2024-09-10T10:04:49.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:49.154+0000 7f2ff3e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2fec103d20 0x7f2fec199ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:49.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:49.154+0000 7f2ff3e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2fec104f10 0x7f2fec19a530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:49.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:49.154+0000 7f2ff3e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2fec1058d0 0x7f2fec1a15a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:49.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:49.154+0000 7f2ff3e00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f2fec10f9d0 con 0x7f2fec104f10 2024-09-10T10:04:49.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:49.154+0000 7f2ff3e00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f2fec10f850 con 0x7f2fec1058d0 2024-09-10T10:04:49.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:49.154+0000 7f2ff3e00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f2fec10fb50 con 0x7f2fec103d20 2024-09-10T10:04:49.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:49.154+0000 7f2ff0a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2fec104f10 0x7f2fec19a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:49.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:49.154+0000 7f2ff1400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2fec103d20 0x7f2fec199ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:49.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:49.154+0000 7f2ff0a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2fec104f10 0x7f2fec19a530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40506/0 (socket says 172.21.15.5:40506) 2024-09-10T10:04:49.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:49.154+0000 7f2ff1400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2fec103d20 0x7f2fec199ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.159:3300/0 says I am v2:172.21.15.5:55550/0 (socket says 172.21.15.5:55550) 2024-09-10T10:04:49.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:49.154+0000 7f2ff0a00640 1 -- 172.21.15.5:0/4233161491 learned_addr learned my addr 172.21.15.5:0/4233161491 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T10:04:49.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:49.154+0000 7f2ff1e00640 1 --2- 172.21.15.5:0/4233161491 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2fec1058d0 0x7f2fec1a15a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:49.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:49.155+0000 7f2ff1400640 1 -- 172.21.15.5:0/4233161491 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2fec1058d0 msgr2=0x7f2fec1a15a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:49.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:49.155+0000 7f2ff1400640 1 --2- 172.21.15.5:0/4233161491 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2fec1058d0 0x7f2fec1a15a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:49.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:49.155+0000 7f2ff1400640 1 -- 172.21.15.5:0/4233161491 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2fec104f10 msgr2=0x7f2fec19a530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:49.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:49.155+0000 7f2ff1400640 1 --2- 172.21.15.5:0/4233161491 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2fec104f10 0x7f2fec19a530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:49.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:49.155+0000 7f2ff1400640 1 -- 172.21.15.5:0/4233161491 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2fec1a1c10 con 0x7f2fec103d20 2024-09-10T10:04:49.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:49.155+0000 7f2ff1e00640 1 --2- 172.21.15.5:0/4233161491 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2fec1058d0 0x7f2fec1a15a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:04:49.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:49.155+0000 7f2ff0a00640 1 --2- 172.21.15.5:0/4233161491 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2fec104f10 0x7f2fec19a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:04:49.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:49.155+0000 7f2ff1400640 1 --2- 172.21.15.5:0/4233161491 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2fec103d20 0x7f2fec199ff0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f2fe0005fd0 tx=0x7f2fe000ef90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:49.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:49.155+0000 7f2fea000640 1 -- 172.21.15.5:0/4233161491 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fe0003970 con 0x7f2fec103d20 2024-09-10T10:04:49.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:49.155+0000 7f2ff3e00640 1 -- 172.21.15.5:0/4233161491 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2fec1a1f00 con 0x7f2fec103d20 2024-09-10T10:04:49.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:49.155+0000 7f2fea000640 1 -- 172.21.15.5:0/4233161491 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2fe0004030 con 0x7f2fec103d20 2024-09-10T10:04:49.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:49.155+0000 7f2fea000640 1 -- 172.21.15.5:0/4233161491 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fe00107f0 con 0x7f2fec103d20 2024-09-10T10:04:49.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:49.155+0000 7f2ff3e00640 1 -- 172.21.15.5:0/4233161491 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2fec1a23c0 con 0x7f2fec103d20 2024-09-10T10:04:49.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:49.156+0000 7f2ff3e00640 1 -- 172.21.15.5:0/4233161491 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2fec10b0c0 con 0x7f2fec103d20 2024-09-10T10:04:49.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:49.159+0000 7f2fea000640 1 -- 172.21.15.5:0/4233161491 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f2fe00041e0 con 0x7f2fec103d20 2024-09-10T10:04:49.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:49.160+0000 7f2fea000640 1 --2- 172.21.15.5:0/4233161491 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f2fd40a2c10 0x7f2fd40a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:49.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:49.160+0000 7f2fea000640 1 -- 172.21.15.5:0/4233161491 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f2fe00cfbf0 con 0x7f2fec103d20 2024-09-10T10:04:49.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:49.160+0000 7f2ff0a00640 1 --2- 172.21.15.5:0/4233161491 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f2fd40a2c10 0x7f2fd40a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:49.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:49.161+0000 7f2ff0a00640 1 --2- 172.21.15.5:0/4233161491 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f2fd40a2c10 0x7f2fd40a50d0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f2fec19b500 tx=0x7f2fd8005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:49.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:49.162+0000 7f2fea000640 1 -- 172.21.15.5:0/4233161491 <== mon.1 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f2fe009c700 con 0x7f2fec103d20 2024-09-10T10:04:49.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:49.346+0000 7f2ff3e00640 1 -- 172.21.15.5:0/4233161491 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f2fec02da00 con 0x7f2fec103d20 2024-09-10T10:04:49.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:49.346+0000 7f2fea000640 1 -- 172.21.15.5:0/4233161491 <== mon.1 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v96) ==== 103+0+11 (secure 0 0 0) 0x7f2fe00bc0c0 con 0x7f2fec103d20 2024-09-10T10:04:49.348 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 32 2024-09-10T10:04:49.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:49.349+0000 7f2ff3e00640 1 -- 172.21.15.5:0/4233161491 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f2fd40a2c10 msgr2=0x7f2fd40a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:49.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:49.349+0000 7f2ff3e00640 1 --2- 172.21.15.5:0/4233161491 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f2fd40a2c10 0x7f2fd40a50d0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f2fec19b500 tx=0x7f2fd8005f90 comp rx=0 tx=0).stop 2024-09-10T10:04:49.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:49.349+0000 7f2ff3e00640 1 -- 172.21.15.5:0/4233161491 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2fec103d20 msgr2=0x7f2fec199ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:49.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:49.349+0000 7f2ff3e00640 1 --2- 172.21.15.5:0/4233161491 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2fec103d20 0x7f2fec199ff0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f2fe0005fd0 tx=0x7f2fe000ef90 comp rx=0 tx=0).stop 2024-09-10T10:04:49.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:49.349+0000 7f2ff3e00640 1 -- 172.21.15.5:0/4233161491 shutdown_connections 2024-09-10T10:04:49.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:49.349+0000 7f2ff3e00640 1 --2- 172.21.15.5:0/4233161491 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f2fd40a2c10 0x7f2fd40a50d0 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:49.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:49.349+0000 7f2ff3e00640 1 --2- 172.21.15.5:0/4233161491 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2fec1058d0 0x7f2fec1a15a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:49.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:49.349+0000 7f2ff3e00640 1 --2- 172.21.15.5:0/4233161491 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2fec104f10 0x7f2fec19a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:49.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:49.349+0000 7f2ff3e00640 1 --2- 172.21.15.5:0/4233161491 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2fec103d20 0x7f2fec199ff0 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:49.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:49.349+0000 7f2ff3e00640 1 -- 172.21.15.5:0/4233161491 >> 172.21.15.5:0/4233161491 conn(0x7f2fec0ff4a0 msgr2=0x7f2fec108290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:49.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:49.349+0000 7f2ff3e00640 1 -- 172.21.15.5:0/4233161491 shutdown_connections 2024-09-10T10:04:49.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:49.349+0000 7f2ff3e00640 1 -- 172.21.15.5:0/4233161491 wait complete. 2024-09-10T10:04:49.367 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-09-10T10:04:49.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:49.525+0000 7fadc6c00640 1 -- 172.21.15.5:0/2522451224 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fadc01058e0 msgr2=0x7fadc010c170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:49.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:49.525+0000 7fadc6c00640 1 --2- 172.21.15.5:0/2522451224 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fadc01058e0 0x7fadc010c170 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7fadb8009ca0 tx=0x7fadb802a8c0 comp rx=0 tx=0).stop 2024-09-10T10:04:49.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:49.525+0000 7fadc6c00640 1 -- 172.21.15.5:0/2522451224 shutdown_connections 2024-09-10T10:04:49.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:49.525+0000 7fadc6c00640 1 --2- 172.21.15.5:0/2522451224 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fadc01058e0 0x7fadc010c170 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:49.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:49.525+0000 7fadc6c00640 1 --2- 172.21.15.5:0/2522451224 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fadc0104f20 0x7fadc01053a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:49.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:49.525+0000 7fadc6c00640 1 --2- 172.21.15.5:0/2522451224 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fadc0103d30 0x7fadc0104130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:49.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:49.525+0000 7fadc6c00640 1 -- 172.21.15.5:0/2522451224 >> 172.21.15.5:0/2522451224 conn(0x7fadc00ff4b0 msgr2=0x7fadc01018d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:49.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:49.525+0000 7fadc6c00640 1 -- 172.21.15.5:0/2522451224 shutdown_connections 2024-09-10T10:04:49.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:49.525+0000 7fadc6c00640 1 -- 172.21.15.5:0/2522451224 wait complete. 2024-09-10T10:04:49.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:49.526+0000 7fadc6c00640 1 Processor -- start 2024-09-10T10:04:49.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:49.526+0000 7fadc6c00640 1 -- start start 2024-09-10T10:04:49.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:49.527+0000 7fadc6c00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fadc0103d30 0x7fadc019a000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:49.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:49.527+0000 7fadc6c00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fadc0104f20 0x7fadc019a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:49.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:49.527+0000 7fadc6c00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fadc01058e0 0x7fadc01a15b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:49.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:49.527+0000 7fadc6c00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fadc010fa00 con 0x7fadc01058e0 2024-09-10T10:04:49.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:49.527+0000 7fadc6c00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fadc010f880 con 0x7fadc0103d30 2024-09-10T10:04:49.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:49.527+0000 7fadc6c00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fadc010fb80 con 0x7fadc0104f20 2024-09-10T10:04:49.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:49.527+0000 7fadc4c00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fadc01058e0 0x7fadc01a15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:49.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:49.527+0000 7fadb7e00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fadc0104f20 0x7fadc019a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:49.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:49.527+0000 7fadc4c00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fadc01058e0 0x7fadc01a15b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40520/0 (socket says 172.21.15.5:40520) 2024-09-10T10:04:49.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:49.527+0000 7fadc4c00640 1 -- 172.21.15.5:0/3449477826 learned_addr learned my addr 172.21.15.5:0/3449477826 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T10:04:49.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:49.527+0000 7fadbfe00640 1 --2- 172.21.15.5:0/3449477826 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fadc0103d30 0x7fadc019a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:49.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:49.527+0000 7fadb7e00640 1 -- 172.21.15.5:0/3449477826 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fadc0103d30 msgr2=0x7fadc019a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:49.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:49.527+0000 7fadb7e00640 1 --2- 172.21.15.5:0/3449477826 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fadc0103d30 0x7fadc019a000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:49.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:49.527+0000 7fadb7e00640 1 -- 172.21.15.5:0/3449477826 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fadc01058e0 msgr2=0x7fadc01a15b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:49.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:49.527+0000 7fadb7e00640 1 --2- 172.21.15.5:0/3449477826 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fadc01058e0 0x7fadc01a15b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:49.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:49.527+0000 7fadb7e00640 1 -- 172.21.15.5:0/3449477826 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fadc01a1c20 con 0x7fadc0104f20 2024-09-10T10:04:49.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:49.528+0000 7fadb7e00640 1 --2- 172.21.15.5:0/3449477826 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fadc0104f20 0x7fadc019a540 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7fada800e9f0 tx=0x7fada800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:49.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:49.528+0000 7fadbd600640 1 -- 172.21.15.5:0/3449477826 <== mon.1 v2:172.21.15.159:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fada800ccc0 con 0x7fadc0104f20 2024-09-10T10:04:49.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:49.528+0000 7fadbd600640 1 -- 172.21.15.5:0/3449477826 <== mon.1 v2:172.21.15.159:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fada8010910 con 0x7fadc0104f20 2024-09-10T10:04:49.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:49.528+0000 7fadc6c00640 1 -- 172.21.15.5:0/3449477826 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fadc01a1f10 con 0x7fadc0104f20 2024-09-10T10:04:49.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:49.528+0000 7fadbd600640 1 -- 172.21.15.5:0/3449477826 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fada8018960 con 0x7fadc0104f20 2024-09-10T10:04:49.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:49.528+0000 7fadc6c00640 1 -- 172.21.15.5:0/3449477826 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fadc01a23d0 con 0x7fadc0104f20 2024-09-10T10:04:49.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:49.529+0000 7fadc6c00640 1 -- 172.21.15.5:0/3449477826 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fadc010b050 con 0x7fadc0104f20 2024-09-10T10:04:49.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:49.531+0000 7fadbd600640 1 -- 172.21.15.5:0/3449477826 <== mon.1 v2:172.21.15.159:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7fada8010ac0 con 0x7fadc0104f20 2024-09-10T10:04:49.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:49.532+0000 7fadbd600640 1 --2- 172.21.15.5:0/3449477826 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fada40a2b90 0x7fada40a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:04:49.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:49.533+0000 7fadbd600640 1 -- 172.21.15.5:0/3449477826 <== mon.1 v2:172.21.15.159:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fada80cfce0 con 0x7fadc0104f20 2024-09-10T10:04:49.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:49.533+0000 7fadbfe00640 1 --2- 172.21.15.5:0/3449477826 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fada40a2b90 0x7fada40a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:04:49.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:49.533+0000 7fadbfe00640 1 --2- 172.21.15.5:0/3449477826 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fada40a2b90 0x7fada40a5050 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fadc0104d80 tx=0x7fadb0005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:04:49.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:49.535+0000 7fadbd600640 1 -- 172.21.15.5:0/3449477826 <== mon.1 v2:172.21.15.159:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fada809c1a0 con 0x7fadc0104f20 2024-09-10T10:04:49.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:49.774+0000 7fadc6c00640 1 -- 172.21.15.5:0/3449477826 --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) -- 0x7fadc002da10 con 0x7fadc0104f20 2024-09-10T10:04:49.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:49.784+0000 7fadbd600640 1 -- 172.21.15.5:0/3449477826 <== mon.1 v2:172.21.15.159:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v20) ==== 81+0+740362 (secure 0 0 0) 0x7fada80a04a0 con 0x7fadc0104f20 2024-09-10T10:04:49.797 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-10T10:04:49.797 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-10T10:04:49.797 INFO:teuthology.orchestra.run.smithi005.stdout: "epoch": 20, 2024-09-10T10:04:49.797 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:49.797 INFO:teuthology.orchestra.run.smithi005.stdout: "active_gid": 14150, 2024-09-10T10:04:49.797 INFO:teuthology.orchestra.run.smithi005.stdout: "active_name": "x", 2024-09-10T10:04:49.797 INFO:teuthology.orchestra.run.smithi005.stdout: "active_addrs": { 2024-09-10T10:04:49.797 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-09-10T10:04:49.797 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-10T10:04:49.797 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-09-10T10:04:49.797 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6800", 2024-09-10T10:04:49.798 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 2707348081 2024-09-10T10:04:49.798 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:49.798 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-10T10:04:49.798 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-09-10T10:04:49.798 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6801", 2024-09-10T10:04:49.798 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 2707348081 2024-09-10T10:04:49.798 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T10:04:49.798 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-10T10:04:49.798 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:49.798 INFO:teuthology.orchestra.run.smithi005.stdout: "active_addr": "172.21.15.5:6801/2707348081", 2024-09-10T10:04:49.798 INFO:teuthology.orchestra.run.smithi005.stdout: "active_change": "2024-09-10T09:28:05.436891+0000", 2024-09-10T10:04:49.798 INFO:teuthology.orchestra.run.smithi005.stdout: "active_mgr_features": 4540701547738038271, 2024-09-10T10:04:49.798 INFO:teuthology.orchestra.run.smithi005.stdout: "available": true, 2024-09-10T10:04:49.798 INFO:teuthology.orchestra.run.smithi005.stdout: "standbys": [ 2024-09-10T10:04:49.799 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-10T10:04:49.799 INFO:teuthology.orchestra.run.smithi005.stdout: "gid": 24109, 2024-09-10T10:04:49.799 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "z", 2024-09-10T10:04:49.799 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr_features": 4540701547738038271, 2024-09-10T10:04:49.799 INFO:teuthology.orchestra.run.smithi005.stdout: "available_modules": [ 2024-09-10T10:04:49.799 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-10T10:04:49.799 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "alerts", 2024-09-10T10:04:49.799 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-10T10:04:49.799 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-10T10:04:49.799 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-10T10:04:49.799 INFO:teuthology.orchestra.run.smithi005.stdout: "interval": { 2024-09-10T10:04:49.799 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "interval", 2024-09-10T10:04:49.799 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-10T10:04:49.799 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:49.799 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:49.800 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "60", 2024-09-10T10:04:49.800 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:49.800 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:49.800 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:49.800 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "How frequently to reexamine health status", 2024-09-10T10:04:49.800 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:49.800 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:49.800 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:49.800 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:49.800 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-10T10:04:49.800 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-10T10:04:49.800 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:49.800 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:49.801 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:49.801 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-10T10:04:49.801 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:49.801 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:49.801 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-10T10:04:49.801 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-10T10:04:49.801 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-10T10:04:49.801 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-10T10:04:49.801 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-10T10:04:49.801 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-10T10:04:49.801 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-10T10:04:49.801 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-10T10:04:49.801 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:49.801 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:49.801 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:49.802 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:49.802 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:49.802 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-10T10:04:49.802 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-10T10:04:49.802 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:49.802 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:49.802 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:49.802 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-10T10:04:49.802 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:49.802 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:49.802 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:49.802 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:49.802 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:49.802 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:49.802 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:49.803 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:49.803 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-10T10:04:49.803 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-10T10:04:49.803 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:49.803 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:49.803 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:49.803 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-10T10:04:49.803 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:49.803 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:49.803 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-10T10:04:49.803 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-10T10:04:49.803 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-10T10:04:49.803 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-10T10:04:49.804 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-10T10:04:49.804 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-10T10:04:49.804 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-10T10:04:49.804 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-10T10:04:49.804 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:49.804 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:49.804 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:49.804 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:49.804 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:49.804 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-10T10:04:49.804 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-10T10:04:49.804 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:49.804 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:49.805 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:49.805 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-10T10:04:49.805 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:49.805 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:49.805 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:49.805 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:49.805 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:49.805 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:49.805 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:49.805 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:49.805 INFO:teuthology.orchestra.run.smithi005.stdout: "smtp_destination": { 2024-09-10T10:04:49.805 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "smtp_destination", 2024-09-10T10:04:49.805 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:49.805 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:49.805 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:49.806 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-10T10:04:49.806 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:49.806 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:49.806 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:49.806 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Email address to send alerts to", 2024-09-10T10:04:49.806 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:49.806 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:49.806 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:49.806 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:49.806 INFO:teuthology.orchestra.run.smithi005.stdout: "smtp_from_name": { 2024-09-10T10:04:49.806 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "smtp_from_name", 2024-09-10T10:04:49.806 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:49.806 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:49.806 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:49.807 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "Ceph", 2024-09-10T10:04:49.807 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:49.807 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:49.807 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:49.807 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Email From: name", 2024-09-10T10:04:49.807 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:49.807 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:49.807 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:49.807 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:49.807 INFO:teuthology.orchestra.run.smithi005.stdout: "smtp_host": { 2024-09-10T10:04:49.807 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "smtp_host", 2024-09-10T10:04:49.807 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:49.807 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:49.807 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:49.808 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-10T10:04:49.808 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:49.808 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:49.808 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:49.808 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "SMTP server", 2024-09-10T10:04:49.808 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:49.808 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:49.808 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:49.808 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:49.808 INFO:teuthology.orchestra.run.smithi005.stdout: "smtp_password": { 2024-09-10T10:04:49.808 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "smtp_password", 2024-09-10T10:04:49.808 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:49.808 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:49.808 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:49.809 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-10T10:04:49.809 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:49.809 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:49.809 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:49.809 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Password to authenticate with", 2024-09-10T10:04:49.809 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:49.809 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:49.809 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:49.809 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:49.809 INFO:teuthology.orchestra.run.smithi005.stdout: "smtp_port": { 2024-09-10T10:04:49.809 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "smtp_port", 2024-09-10T10:04:49.809 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-10T10:04:49.809 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:49.810 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:49.810 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "465", 2024-09-10T10:04:49.810 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:49.810 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:49.810 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:49.810 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "SMTP port", 2024-09-10T10:04:49.810 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:49.810 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:49.810 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:49.810 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:49.810 INFO:teuthology.orchestra.run.smithi005.stdout: "smtp_sender": { 2024-09-10T10:04:49.810 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "smtp_sender", 2024-09-10T10:04:49.810 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:49.810 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:49.810 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:49.811 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-10T10:04:49.811 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:49.811 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:49.811 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:49.811 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "SMTP envelope sender", 2024-09-10T10:04:49.811 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:49.811 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:49.811 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:49.811 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:49.811 INFO:teuthology.orchestra.run.smithi005.stdout: "smtp_ssl": { 2024-09-10T10:04:49.811 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "smtp_ssl", 2024-09-10T10:04:49.811 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:49.811 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:49.811 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:49.812 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-10T10:04:49.812 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:49.812 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:49.812 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:49.812 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-10T10:04:49.812 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:49.812 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:49.812 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:49.812 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:49.812 INFO:teuthology.orchestra.run.smithi005.stdout: "smtp_user": { 2024-09-10T10:04:49.812 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "smtp_user", 2024-09-10T10:04:49.812 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:49.812 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:49.812 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:49.812 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-10T10:04:49.813 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:49.813 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:49.813 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:49.813 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "User to authenticate as", 2024-09-10T10:04:49.813 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:49.813 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:49.813 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:49.813 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T10:04:49.813 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T10:04:49.813 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:49.813 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-10T10:04:49.813 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "balancer", 2024-09-10T10:04:49.813 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-10T10:04:49.813 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-10T10:04:49.814 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-10T10:04:49.814 INFO:teuthology.orchestra.run.smithi005.stdout: "active": { 2024-09-10T10:04:49.814 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "active", 2024-09-10T10:04:49.814 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:49.814 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:49.814 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:49.814 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-10T10:04:49.814 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:49.814 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:49.814 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:49.814 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "automatically balance PGs across cluster", 2024-09-10T10:04:49.814 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:49.814 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:49.814 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:49.814 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:49.815 INFO:teuthology.orchestra.run.smithi005.stdout: "begin_time": { 2024-09-10T10:04:49.815 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "begin_time", 2024-09-10T10:04:49.815 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:49.815 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:49.815 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:49.815 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0000", 2024-09-10T10:04:49.815 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:49.815 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:49.815 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:49.815 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "beginning time of day to automatically balance", 2024-09-10T10:04:49.815 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-10T10:04:49.815 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:49.815 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:49.815 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:49.816 INFO:teuthology.orchestra.run.smithi005.stdout: "begin_weekday": { 2024-09-10T10:04:49.816 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "begin_weekday", 2024-09-10T10:04:49.816 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "uint", 2024-09-10T10:04:49.816 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:49.816 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:49.816 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-09-10T10:04:49.816 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "0", 2024-09-10T10:04:49.816 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "6", 2024-09-10T10:04:49.816 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:49.816 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-10T10:04:49.816 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-10T10:04:49.816 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:49.816 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:49.816 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:49.817 INFO:teuthology.orchestra.run.smithi005.stdout: "crush_compat_max_iterations": { 2024-09-10T10:04:49.817 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "crush_compat_max_iterations", 2024-09-10T10:04:49.817 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "uint", 2024-09-10T10:04:49.817 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:49.817 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:49.817 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "25", 2024-09-10T10:04:49.817 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "1", 2024-09-10T10:04:49.817 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "250", 2024-09-10T10:04:49.817 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:49.817 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-10T10:04:49.817 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:49.817 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:49.818 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:49.818 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:49.818 INFO:teuthology.orchestra.run.smithi005.stdout: "crush_compat_metrics": { 2024-09-10T10:04:49.818 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "crush_compat_metrics", 2024-09-10T10:04:49.818 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:49.818 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:49.818 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:49.818 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "pgs,objects,bytes", 2024-09-10T10:04:49.818 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:49.818 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:49.818 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:49.818 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-10T10:04:49.818 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-10T10:04:49.818 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:49.819 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:49.819 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:49.819 INFO:teuthology.orchestra.run.smithi005.stdout: "crush_compat_step": { 2024-09-10T10:04:49.819 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "crush_compat_step", 2024-09-10T10:04:49.819 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "float", 2024-09-10T10:04:49.819 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:49.819 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:49.819 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0.5", 2024-09-10T10:04:49.819 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "0.001", 2024-09-10T10:04:49.819 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "0.999", 2024-09-10T10:04:49.819 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:49.819 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "aggressiveness of optimization", 2024-09-10T10:04:49.819 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-10T10:04:49.819 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:49.819 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:49.820 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:49.820 INFO:teuthology.orchestra.run.smithi005.stdout: "end_time": { 2024-09-10T10:04:49.820 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "end_time", 2024-09-10T10:04:49.820 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:49.820 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:49.820 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:49.820 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "2359", 2024-09-10T10:04:49.820 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:49.820 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:49.820 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:49.820 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "ending time of day to automatically balance", 2024-09-10T10:04:49.820 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-10T10:04:49.820 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:49.820 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:49.821 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:49.821 INFO:teuthology.orchestra.run.smithi005.stdout: "end_weekday": { 2024-09-10T10:04:49.821 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "end_weekday", 2024-09-10T10:04:49.821 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "uint", 2024-09-10T10:04:49.821 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:49.821 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:49.821 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-09-10T10:04:49.821 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "0", 2024-09-10T10:04:49.821 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "6", 2024-09-10T10:04:49.821 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:49.821 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-10T10:04:49.821 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-10T10:04:49.821 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:49.822 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:49.822 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:49.822 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-10T10:04:49.822 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-10T10:04:49.822 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:49.822 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:49.822 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:49.822 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-10T10:04:49.822 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:49.822 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:49.822 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-10T10:04:49.822 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-10T10:04:49.822 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-10T10:04:49.822 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-10T10:04:49.822 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-10T10:04:49.823 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-10T10:04:49.823 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-10T10:04:49.823 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-10T10:04:49.823 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:49.823 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:49.823 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:49.823 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:49.823 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:49.823 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-10T10:04:49.823 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-10T10:04:49.823 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:49.823 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:49.823 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:49.823 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-10T10:04:49.824 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:49.824 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:49.824 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:49.824 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:49.824 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:49.824 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:49.824 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:49.824 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:49.824 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-10T10:04:49.824 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-10T10:04:49.824 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:49.824 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:49.824 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:49.825 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-10T10:04:49.825 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:49.825 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:49.825 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-10T10:04:49.825 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-10T10:04:49.825 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-10T10:04:49.825 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-10T10:04:49.825 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-10T10:04:49.825 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-10T10:04:49.825 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-10T10:04:49.825 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-10T10:04:49.825 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:49.825 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:49.825 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:49.826 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:49.826 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:49.826 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-10T10:04:49.826 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-10T10:04:49.826 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:49.826 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:49.826 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:49.826 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-10T10:04:49.826 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:49.826 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:49.826 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:49.826 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:49.826 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:49.827 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:49.827 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:49.827 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:49.827 INFO:teuthology.orchestra.run.smithi005.stdout: "min_score": { 2024-09-10T10:04:49.827 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "min_score", 2024-09-10T10:04:49.827 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "float", 2024-09-10T10:04:49.827 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:49.827 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:49.827 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-09-10T10:04:49.827 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:49.827 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:49.827 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:49.827 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-10T10:04:49.827 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:49.828 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:49.828 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:49.828 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:49.828 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": { 2024-09-10T10:04:49.828 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "mode", 2024-09-10T10:04:49.828 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:49.828 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:49.828 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:49.828 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "upmap", 2024-09-10T10:04:49.828 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:49.828 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:49.829 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-10T10:04:49.829 INFO:teuthology.orchestra.run.smithi005.stdout: "crush-compat", 2024-09-10T10:04:49.829 INFO:teuthology.orchestra.run.smithi005.stdout: "none", 2024-09-10T10:04:49.829 INFO:teuthology.orchestra.run.smithi005.stdout: "read", 2024-09-10T10:04:49.829 INFO:teuthology.orchestra.run.smithi005.stdout: "upmap", 2024-09-10T10:04:49.829 INFO:teuthology.orchestra.run.smithi005.stdout: "upmap-read" 2024-09-10T10:04:49.829 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-10T10:04:49.829 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Balancer mode", 2024-09-10T10:04:49.829 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:49.829 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:49.829 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:49.829 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:49.829 INFO:teuthology.orchestra.run.smithi005.stdout: "pool_ids": { 2024-09-10T10:04:49.830 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "pool_ids", 2024-09-10T10:04:49.830 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:49.830 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:49.830 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:49.830 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-10T10:04:49.830 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:49.830 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:49.830 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:49.830 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-10T10:04:49.830 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:49.830 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:49.830 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:49.830 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:49.830 INFO:teuthology.orchestra.run.smithi005.stdout: "sleep_interval": { 2024-09-10T10:04:49.831 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sleep_interval", 2024-09-10T10:04:49.831 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-10T10:04:49.831 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:49.831 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:49.831 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "60", 2024-09-10T10:04:49.831 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:49.831 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:49.831 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:49.831 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-10T10:04:49.831 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:49.831 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:49.831 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:49.831 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:49.831 INFO:teuthology.orchestra.run.smithi005.stdout: "upmap_max_deviation": { 2024-09-10T10:04:49.832 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "upmap_max_deviation", 2024-09-10T10:04:49.832 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-10T10:04:49.832 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:49.832 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:49.832 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "5", 2024-09-10T10:04:49.832 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "1", 2024-09-10T10:04:49.832 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:49.832 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:49.832 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-10T10:04:49.832 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-10T10:04:49.832 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:49.832 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:49.832 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:49.832 INFO:teuthology.orchestra.run.smithi005.stdout: "upmap_max_optimizations": { 2024-09-10T10:04:49.832 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "upmap_max_optimizations", 2024-09-10T10:04:49.833 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "uint", 2024-09-10T10:04:49.833 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:49.833 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:49.833 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10", 2024-09-10T10:04:49.833 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:49.833 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:49.833 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:49.833 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-10T10:04:49.833 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:49.833 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:49.833 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:49.833 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T10:04:49.833 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T10:04:49.833 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:49.834 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-10T10:04:49.834 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "cephadm", 2024-09-10T10:04:49.834 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-10T10:04:49.834 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-10T10:04:49.834 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-10T10:04:49.834 INFO:teuthology.orchestra.run.smithi005.stdout: "agent_down_multiplier": { 2024-09-10T10:04:49.834 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "agent_down_multiplier", 2024-09-10T10:04:49.834 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "float", 2024-09-10T10:04:49.834 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:49.834 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:49.834 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "3.0", 2024-09-10T10:04:49.834 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:49.834 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:49.834 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:49.834 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-10T10:04:49.835 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:49.835 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:49.835 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:49.835 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:49.835 INFO:teuthology.orchestra.run.smithi005.stdout: "agent_refresh_rate": { 2024-09-10T10:04:49.835 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "agent_refresh_rate", 2024-09-10T10:04:49.835 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-10T10:04:49.835 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:49.835 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:49.835 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "20", 2024-09-10T10:04:49.835 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:49.835 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:49.835 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:49.835 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-10T10:04:49.836 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:49.836 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:49.836 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:49.836 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:49.836 INFO:teuthology.orchestra.run.smithi005.stdout: "agent_starting_port": { 2024-09-10T10:04:49.836 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "agent_starting_port", 2024-09-10T10:04:49.836 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-10T10:04:49.836 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:49.836 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:49.836 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "4721", 2024-09-10T10:04:49.836 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:49.836 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:49.836 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:49.837 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-10T10:04:49.837 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:49.837 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:49.837 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:49.837 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:49.837 INFO:teuthology.orchestra.run.smithi005.stdout: "allow_ptrace": { 2024-09-10T10:04:49.837 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "allow_ptrace", 2024-09-10T10:04:49.837 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:49.837 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:49.837 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:49.837 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-10T10:04:49.837 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:49.837 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:49.838 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:49.838 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-10T10:04:49.838 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-10T10:04:49.838 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:49.838 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:49.838 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:49.838 INFO:teuthology.orchestra.run.smithi005.stdout: "autotune_interval": { 2024-09-10T10:04:49.838 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "autotune_interval", 2024-09-10T10:04:49.838 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-10T10:04:49.838 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:49.838 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:49.838 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "600", 2024-09-10T10:04:49.838 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:49.838 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:49.839 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:49.839 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-10T10:04:49.839 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:49.839 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:49.839 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:49.839 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:49.839 INFO:teuthology.orchestra.run.smithi005.stdout: "autotune_memory_target_ratio": { 2024-09-10T10:04:49.839 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "autotune_memory_target_ratio", 2024-09-10T10:04:49.839 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "float", 2024-09-10T10:04:49.839 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:49.839 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:49.839 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0.7", 2024-09-10T10:04:49.839 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:49.839 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:49.840 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:49.840 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-10T10:04:49.840 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:49.840 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:49.840 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:49.840 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:49.840 INFO:teuthology.orchestra.run.smithi005.stdout: "cephadm_log_destination": { 2024-09-10T10:04:49.840 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "cephadm_log_destination", 2024-09-10T10:04:49.840 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:49.840 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:49.840 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:49.840 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-10T10:04:49.840 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:49.840 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:49.841 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-10T10:04:49.841 INFO:teuthology.orchestra.run.smithi005.stdout: "file", 2024-09-10T10:04:49.841 INFO:teuthology.orchestra.run.smithi005.stdout: "file,syslog", 2024-09-10T10:04:49.841 INFO:teuthology.orchestra.run.smithi005.stdout: "syslog" 2024-09-10T10:04:49.841 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-10T10:04:49.841 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-10T10:04:49.841 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:49.841 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:49.841 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:49.841 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:49.841 INFO:teuthology.orchestra.run.smithi005.stdout: "cgroups_split": { 2024-09-10T10:04:49.841 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "cgroups_split", 2024-09-10T10:04:49.841 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:49.842 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:49.842 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:49.842 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-10T10:04:49.842 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:49.842 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:49.842 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:49.842 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-10T10:04:49.842 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:49.842 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:49.842 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:49.842 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:49.842 INFO:teuthology.orchestra.run.smithi005.stdout: "config_checks_enabled": { 2024-09-10T10:04:49.842 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "config_checks_enabled", 2024-09-10T10:04:49.842 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:49.842 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:49.843 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:49.843 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-10T10:04:49.843 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:49.843 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:49.843 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:49.843 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-10T10:04:49.843 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:49.843 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:49.843 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:49.843 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:49.843 INFO:teuthology.orchestra.run.smithi005.stdout: "config_dashboard": { 2024-09-10T10:04:49.843 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "config_dashboard", 2024-09-10T10:04:49.843 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:49.843 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:49.843 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:49.844 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-10T10:04:49.844 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:49.844 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:49.844 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:49.844 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-10T10:04:49.844 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:49.844 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:49.844 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:49.844 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:49.844 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_alertmanager": { 2024-09-10T10:04:49.844 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_alertmanager", 2024-09-10T10:04:49.844 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:49.844 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:49.844 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:49.844 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-09-10T10:04:49.845 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:49.845 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:49.845 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:49.845 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Prometheus container image", 2024-09-10T10:04:49.845 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:49.845 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:49.845 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:49.845 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:49.845 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_base": { 2024-09-10T10:04:49.845 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_base", 2024-09-10T10:04:49.845 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:49.845 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:49.845 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:49.845 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-10T10:04:49.845 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:49.846 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:49.846 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:49.846 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Container image name, without the tag", 2024-09-10T10:04:49.846 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:49.846 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:49.846 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:49.846 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:49.846 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_elasticsearch": { 2024-09-10T10:04:49.846 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_elasticsearch", 2024-09-10T10:04:49.846 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:49.846 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:49.846 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:49.846 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-10T10:04:49.846 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:49.846 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:49.847 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:49.847 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "elasticsearch container image", 2024-09-10T10:04:49.847 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:49.847 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:49.847 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:49.847 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:49.847 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_grafana": { 2024-09-10T10:04:49.847 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_grafana", 2024-09-10T10:04:49.847 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:49.847 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:49.847 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:49.847 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-09-10T10:04:49.847 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:49.847 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:49.847 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:49.848 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Prometheus container image", 2024-09-10T10:04:49.848 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:49.848 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:49.848 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:49.848 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:49.848 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_haproxy": { 2024-09-10T10:04:49.848 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_haproxy", 2024-09-10T10:04:49.848 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:49.848 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:49.848 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:49.848 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-10T10:04:49.848 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:49.848 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:49.848 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:49.849 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "HAproxy container image", 2024-09-10T10:04:49.849 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:49.849 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:49.849 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:49.849 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:49.849 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_jaeger_agent": { 2024-09-10T10:04:49.849 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_jaeger_agent", 2024-09-10T10:04:49.849 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:49.849 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:49.849 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:49.849 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-10T10:04:49.849 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:49.849 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:49.849 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:49.849 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Jaeger agent container image", 2024-09-10T10:04:49.850 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:49.850 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:49.850 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:49.850 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:49.850 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_jaeger_collector": { 2024-09-10T10:04:49.850 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_jaeger_collector", 2024-09-10T10:04:49.850 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:49.850 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:49.850 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:49.850 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-10T10:04:49.850 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:49.850 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:49.850 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:49.850 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Jaeger collector container image", 2024-09-10T10:04:49.850 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:49.851 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:49.851 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:49.851 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:49.851 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_jaeger_query": { 2024-09-10T10:04:49.851 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_jaeger_query", 2024-09-10T10:04:49.851 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:49.851 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:49.851 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:49.851 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-10T10:04:49.851 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:49.851 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:49.851 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:49.851 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Jaeger query container image", 2024-09-10T10:04:49.851 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:49.851 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:49.852 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:49.852 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:49.852 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_keepalived": { 2024-09-10T10:04:49.852 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_keepalived", 2024-09-10T10:04:49.852 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:49.852 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:49.852 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:49.852 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-10T10:04:49.852 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:49.852 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:49.852 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:49.852 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Keepalived container image", 2024-09-10T10:04:49.852 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:49.852 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:49.852 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:49.853 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:49.853 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_loki": { 2024-09-10T10:04:49.853 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_loki", 2024-09-10T10:04:49.853 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:49.853 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:49.853 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:49.853 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-10T10:04:49.853 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:49.853 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:49.853 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:49.853 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Loki container image", 2024-09-10T10:04:49.853 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:49.853 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:49.854 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:49.854 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:49.854 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_node_exporter": { 2024-09-10T10:04:49.854 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_node_exporter", 2024-09-10T10:04:49.854 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:49.854 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:49.854 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:49.854 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-09-10T10:04:49.854 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:49.854 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:49.854 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:49.855 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Prometheus container image", 2024-09-10T10:04:49.855 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:49.855 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:49.855 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:49.855 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:49.855 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_nvmeof": { 2024-09-10T10:04:49.855 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_nvmeof", 2024-09-10T10:04:49.855 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:49.855 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:49.856 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:49.856 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-09-10T10:04:49.856 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:49.856 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:49.856 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:49.856 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Nvme-of container image", 2024-09-10T10:04:49.856 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:49.856 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:49.856 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:49.856 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:49.857 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_prometheus": { 2024-09-10T10:04:49.857 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_prometheus", 2024-09-10T10:04:49.857 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:49.857 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:49.857 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:49.857 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-09-10T10:04:49.857 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:49.857 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:49.857 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:49.858 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Prometheus container image", 2024-09-10T10:04:49.858 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:49.858 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:49.858 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:49.858 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:49.858 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_promtail": { 2024-09-10T10:04:49.858 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_promtail", 2024-09-10T10:04:49.858 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:49.858 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:49.858 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:49.858 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-10T10:04:49.859 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:49.859 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:49.859 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:49.859 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Promtail container image", 2024-09-10T10:04:49.859 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:49.859 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:49.859 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:49.859 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:49.859 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_samba": { 2024-09-10T10:04:49.859 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_samba", 2024-09-10T10:04:49.859 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:49.859 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:49.860 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:49.860 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-10T10:04:49.860 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:49.860 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:49.860 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:49.860 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Samba/SMB container image", 2024-09-10T10:04:49.860 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:49.860 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:49.860 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:49.860 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:49.860 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_snmp_gateway": { 2024-09-10T10:04:49.860 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_snmp_gateway", 2024-09-10T10:04:49.860 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:49.861 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:49.861 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:49.861 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-10T10:04:49.861 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:49.861 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:49.861 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:49.861 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "SNMP Gateway container image", 2024-09-10T10:04:49.861 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:49.861 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:49.861 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:49.861 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:49.861 INFO:teuthology.orchestra.run.smithi005.stdout: "container_init": { 2024-09-10T10:04:49.861 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_init", 2024-09-10T10:04:49.861 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:49.862 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:49.862 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:49.862 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-10T10:04:49.862 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:49.862 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:49.862 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:49.862 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Run podman/docker with `--init`", 2024-09-10T10:04:49.862 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:49.862 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:49.862 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:49.862 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:49.862 INFO:teuthology.orchestra.run.smithi005.stdout: "daemon_cache_timeout": { 2024-09-10T10:04:49.862 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "daemon_cache_timeout", 2024-09-10T10:04:49.863 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-10T10:04:49.863 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:49.863 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:49.863 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "600", 2024-09-10T10:04:49.863 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:49.863 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:49.863 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:49.863 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-10T10:04:49.863 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:49.863 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:49.863 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:49.863 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:49.863 INFO:teuthology.orchestra.run.smithi005.stdout: "default_cephadm_command_timeout": { 2024-09-10T10:04:49.863 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "default_cephadm_command_timeout", 2024-09-10T10:04:49.864 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-10T10:04:49.864 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:49.864 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:49.864 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "900", 2024-09-10T10:04:49.864 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:49.864 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:49.864 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:49.864 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-10T10:04:49.864 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:49.864 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:49.864 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:49.864 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:49.864 INFO:teuthology.orchestra.run.smithi005.stdout: "default_registry": { 2024-09-10T10:04:49.865 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "default_registry", 2024-09-10T10:04:49.865 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:49.865 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:49.865 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:49.865 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "docker.io", 2024-09-10T10:04:49.865 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:49.865 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:49.865 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:49.865 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-10T10:04:49.865 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:49.865 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:49.865 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:49.865 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:49.866 INFO:teuthology.orchestra.run.smithi005.stdout: "device_cache_timeout": { 2024-09-10T10:04:49.866 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "device_cache_timeout", 2024-09-10T10:04:49.866 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-10T10:04:49.866 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:49.866 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:49.866 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "1800", 2024-09-10T10:04:49.866 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:49.866 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:49.866 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:49.866 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "seconds to cache device inventory", 2024-09-10T10:04:49.866 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:49.866 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:49.867 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:49.867 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:49.867 INFO:teuthology.orchestra.run.smithi005.stdout: "device_enhanced_scan": { 2024-09-10T10:04:49.867 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "device_enhanced_scan", 2024-09-10T10:04:49.867 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:49.867 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:49.867 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:49.867 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-10T10:04:49.867 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:49.867 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:49.867 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:49.867 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-10T10:04:49.867 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:49.868 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:49.868 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:49.868 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:49.868 INFO:teuthology.orchestra.run.smithi005.stdout: "facts_cache_timeout": { 2024-09-10T10:04:49.868 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "facts_cache_timeout", 2024-09-10T10:04:49.868 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-10T10:04:49.868 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:49.868 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:49.868 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "60", 2024-09-10T10:04:49.868 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:49.868 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:49.868 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:49.868 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "seconds to cache host facts data", 2024-09-10T10:04:49.868 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:49.869 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:49.869 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:49.869 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:49.869 INFO:teuthology.orchestra.run.smithi005.stdout: "grafana_dashboards_path": { 2024-09-10T10:04:49.869 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "grafana_dashboards_path", 2024-09-10T10:04:49.869 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:49.869 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:49.869 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:49.869 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-10T10:04:49.869 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:49.869 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:49.869 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:49.869 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-10T10:04:49.870 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:49.870 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:49.870 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:49.870 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:49.870 INFO:teuthology.orchestra.run.smithi005.stdout: "host_check_interval": { 2024-09-10T10:04:49.870 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "host_check_interval", 2024-09-10T10:04:49.870 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-10T10:04:49.870 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:49.870 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:49.870 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "600", 2024-09-10T10:04:49.870 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:49.870 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:49.870 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:49.870 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "how frequently to perform a host check", 2024-09-10T10:04:49.870 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:49.871 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:49.871 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:49.871 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:49.871 INFO:teuthology.orchestra.run.smithi005.stdout: "hw_monitoring": { 2024-09-10T10:04:49.871 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "hw_monitoring", 2024-09-10T10:04:49.871 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:49.871 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:49.871 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:49.871 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-10T10:04:49.871 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:49.871 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:49.871 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:49.871 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-10T10:04:49.872 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:49.872 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:49.872 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:49.872 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:49.872 INFO:teuthology.orchestra.run.smithi005.stdout: "inventory_list_all": { 2024-09-10T10:04:49.872 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "inventory_list_all", 2024-09-10T10:04:49.872 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:49.872 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:49.872 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:49.872 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-10T10:04:49.872 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:49.872 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:49.872 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:49.872 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-10T10:04:49.873 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:49.873 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:49.873 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:49.873 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:49.873 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-10T10:04:49.873 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-10T10:04:49.873 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:49.873 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:49.873 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:49.873 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-10T10:04:49.873 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:49.873 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:49.873 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-10T10:04:49.873 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-10T10:04:49.874 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-10T10:04:49.874 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-10T10:04:49.874 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-10T10:04:49.874 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-10T10:04:49.874 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-10T10:04:49.874 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-10T10:04:49.874 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:49.874 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:49.874 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:49.874 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:49.874 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:49.874 INFO:teuthology.orchestra.run.smithi005.stdout: "log_refresh_metadata": { 2024-09-10T10:04:49.874 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_refresh_metadata", 2024-09-10T10:04:49.875 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:49.875 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:49.875 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:49.875 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-10T10:04:49.875 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:49.875 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:49.875 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:49.875 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-10T10:04:49.875 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:49.875 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:49.875 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:49.875 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:49.875 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-10T10:04:49.875 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-10T10:04:49.876 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:49.876 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:49.876 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:49.876 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-10T10:04:49.876 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:49.876 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:49.876 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:49.876 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-10T10:04:49.876 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:49.876 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:49.876 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:49.876 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:49.876 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-10T10:04:49.877 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-10T10:04:49.877 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:49.877 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:49.877 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:49.877 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-10T10:04:49.877 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:49.877 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:49.877 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-10T10:04:49.877 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-10T10:04:49.877 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-10T10:04:49.877 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-10T10:04:49.877 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-10T10:04:49.877 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-10T10:04:49.878 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-10T10:04:49.878 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-10T10:04:49.878 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:49.878 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:49.878 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:49.878 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:49.878 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:49.878 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-10T10:04:49.878 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-10T10:04:49.878 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:49.878 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:49.878 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:49.878 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-10T10:04:49.878 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:49.878 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:49.879 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:49.879 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:49.879 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:49.879 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:49.879 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:49.879 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:49.879 INFO:teuthology.orchestra.run.smithi005.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-10T10:04:49.879 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-10T10:04:49.879 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:49.879 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:49.879 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:49.879 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-10T10:04:49.879 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:49.879 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:49.879 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:49.880 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-10T10:04:49.880 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:49.880 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:49.880 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:49.880 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:49.880 INFO:teuthology.orchestra.run.smithi005.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-10T10:04:49.880 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-10T10:04:49.880 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:49.880 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:49.880 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:49.880 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "*", 2024-09-10T10:04:49.880 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:49.880 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:49.880 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:49.881 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-10T10:04:49.881 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:49.881 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:49.881 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:49.881 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:49.881 INFO:teuthology.orchestra.run.smithi005.stdout: "max_count_per_host": { 2024-09-10T10:04:49.881 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "max_count_per_host", 2024-09-10T10:04:49.881 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-10T10:04:49.881 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:49.881 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:49.881 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10", 2024-09-10T10:04:49.881 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:49.881 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:49.882 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:49.882 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "max number of daemons per service per host", 2024-09-10T10:04:49.882 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:49.882 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:49.882 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:49.882 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:49.882 INFO:teuthology.orchestra.run.smithi005.stdout: "max_osd_draining_count": { 2024-09-10T10:04:49.882 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "max_osd_draining_count", 2024-09-10T10:04:49.882 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-10T10:04:49.882 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:49.882 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:49.882 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10", 2024-09-10T10:04:49.882 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:49.882 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:49.882 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:49.883 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-10T10:04:49.883 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:49.883 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:49.883 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:49.883 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:49.883 INFO:teuthology.orchestra.run.smithi005.stdout: "migration_current": { 2024-09-10T10:04:49.883 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "migration_current", 2024-09-10T10:04:49.883 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-10T10:04:49.883 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:49.883 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:49.883 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-10T10:04:49.883 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:49.883 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:49.883 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:49.883 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "internal - do not modify", 2024-09-10T10:04:49.884 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:49.884 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:49.884 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:49.884 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:49.884 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": { 2024-09-10T10:04:49.884 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "mode", 2024-09-10T10:04:49.884 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:49.884 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:49.884 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:49.884 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "root", 2024-09-10T10:04:49.884 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:49.884 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:49.884 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-10T10:04:49.884 INFO:teuthology.orchestra.run.smithi005.stdout: "cephadm-package", 2024-09-10T10:04:49.885 INFO:teuthology.orchestra.run.smithi005.stdout: "root" 2024-09-10T10:04:49.885 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-10T10:04:49.885 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "mode for remote execution of cephadm", 2024-09-10T10:04:49.885 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:49.885 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:49.885 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:49.885 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:49.885 INFO:teuthology.orchestra.run.smithi005.stdout: "oob_default_addr": { 2024-09-10T10:04:49.885 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "oob_default_addr", 2024-09-10T10:04:49.885 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:49.885 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:49.886 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:49.886 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "169.254.1.1", 2024-09-10T10:04:49.886 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:49.886 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:49.886 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:49.886 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-10T10:04:49.886 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:49.886 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:49.886 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:49.886 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:49.886 INFO:teuthology.orchestra.run.smithi005.stdout: "prometheus_alerts_path": { 2024-09-10T10:04:49.886 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "prometheus_alerts_path", 2024-09-10T10:04:49.886 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:49.887 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:49.887 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:49.887 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-10T10:04:49.887 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:49.887 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:49.887 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:49.887 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-10T10:04:49.887 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:49.887 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:49.887 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:49.887 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:49.887 INFO:teuthology.orchestra.run.smithi005.stdout: "registry_insecure": { 2024-09-10T10:04:49.887 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "registry_insecure", 2024-09-10T10:04:49.888 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:49.888 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:49.888 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:49.888 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-10T10:04:49.888 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:49.888 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:49.888 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:49.888 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-10T10:04:49.888 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:49.888 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:49.888 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:49.888 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:49.888 INFO:teuthology.orchestra.run.smithi005.stdout: "registry_password": { 2024-09-10T10:04:49.888 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "registry_password", 2024-09-10T10:04:49.888 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:49.889 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:49.889 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:49.889 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-10T10:04:49.889 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:49.889 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:49.889 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:49.889 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-10T10:04:49.889 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:49.889 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:49.889 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:49.889 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:49.889 INFO:teuthology.orchestra.run.smithi005.stdout: "registry_url": { 2024-09-10T10:04:49.889 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "registry_url", 2024-09-10T10:04:49.890 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:49.890 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:49.890 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:49.890 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-10T10:04:49.890 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:49.890 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:49.890 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:49.890 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-10T10:04:49.890 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:49.890 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:49.890 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:49.890 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:49.890 INFO:teuthology.orchestra.run.smithi005.stdout: "registry_username": { 2024-09-10T10:04:49.890 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "registry_username", 2024-09-10T10:04:49.891 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:49.891 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:49.891 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:49.891 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-10T10:04:49.891 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:49.891 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:49.891 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:49.891 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-10T10:04:49.891 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:49.891 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:49.891 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:49.891 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:49.891 INFO:teuthology.orchestra.run.smithi005.stdout: "secure_monitoring_stack": { 2024-09-10T10:04:49.892 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "secure_monitoring_stack", 2024-09-10T10:04:49.892 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:49.892 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:49.892 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:49.892 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-10T10:04:49.892 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:49.892 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:49.892 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:49.892 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-10T10:04:49.892 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:49.892 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:49.892 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:49.892 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:49.892 INFO:teuthology.orchestra.run.smithi005.stdout: "service_discovery_port": { 2024-09-10T10:04:49.893 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "service_discovery_port", 2024-09-10T10:04:49.893 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-10T10:04:49.893 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:49.893 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:49.893 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "8765", 2024-09-10T10:04:49.893 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:49.893 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:49.893 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:49.893 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "cephadm service discovery port", 2024-09-10T10:04:49.893 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:49.893 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:49.893 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:49.893 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:49.893 INFO:teuthology.orchestra.run.smithi005.stdout: "ssh_config_file": { 2024-09-10T10:04:49.894 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ssh_config_file", 2024-09-10T10:04:49.894 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:49.894 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:49.894 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:49.894 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-10T10:04:49.894 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:49.894 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:49.894 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:49.894 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-10T10:04:49.894 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:49.894 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:49.894 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:49.894 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:49.894 INFO:teuthology.orchestra.run.smithi005.stdout: "use_agent": { 2024-09-10T10:04:49.894 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "use_agent", 2024-09-10T10:04:49.895 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:49.895 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:49.895 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:49.895 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-10T10:04:49.895 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:49.895 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:49.895 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:49.895 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-10T10:04:49.895 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:49.895 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:49.895 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:49.895 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:49.895 INFO:teuthology.orchestra.run.smithi005.stdout: "use_repo_digest": { 2024-09-10T10:04:49.895 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "use_repo_digest", 2024-09-10T10:04:49.896 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:49.896 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:49.896 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:49.896 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-10T10:04:49.896 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:49.896 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:49.896 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:49.896 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-10T10:04:49.896 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:49.896 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:49.896 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:49.896 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:49.896 INFO:teuthology.orchestra.run.smithi005.stdout: "warn_on_failed_host_check": { 2024-09-10T10:04:49.896 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "warn_on_failed_host_check", 2024-09-10T10:04:49.897 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:49.897 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:49.897 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:49.897 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-10T10:04:49.897 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:49.897 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:49.897 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:49.897 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "raise a health warning if the host check fails", 2024-09-10T10:04:49.897 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:49.897 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:49.897 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:49.897 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:49.897 INFO:teuthology.orchestra.run.smithi005.stdout: "warn_on_stray_daemons": { 2024-09-10T10:04:49.898 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "warn_on_stray_daemons", 2024-09-10T10:04:49.898 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:49.898 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:49.898 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:49.898 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-10T10:04:49.898 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:49.898 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:49.898 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:49.898 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-10T10:04:49.898 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:49.898 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:49.898 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:49.898 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:49.898 INFO:teuthology.orchestra.run.smithi005.stdout: "warn_on_stray_hosts": { 2024-09-10T10:04:49.899 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "warn_on_stray_hosts", 2024-09-10T10:04:49.899 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:49.899 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:49.899 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:49.899 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-10T10:04:49.899 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:49.899 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:49.899 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:49.899 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-10T10:04:49.899 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:49.899 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:49.899 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:49.899 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T10:04:49.899 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T10:04:49.900 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:49.900 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-10T10:04:49.900 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "crash", 2024-09-10T10:04:49.900 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-10T10:04:49.900 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-10T10:04:49.900 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-10T10:04:49.900 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-10T10:04:49.900 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-10T10:04:49.900 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:49.900 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:49.900 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:49.900 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-10T10:04:49.900 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:49.900 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:49.901 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-10T10:04:49.901 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-10T10:04:49.901 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-10T10:04:49.901 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-10T10:04:49.901 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-10T10:04:49.901 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-10T10:04:49.901 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-10T10:04:49.901 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-10T10:04:49.901 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:49.901 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:49.901 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:49.901 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:49.901 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:49.901 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-10T10:04:49.901 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-10T10:04:49.902 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:49.902 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:49.902 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:49.902 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-10T10:04:49.902 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:49.902 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:49.902 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:49.902 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:49.902 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:49.902 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:49.902 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:49.902 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:49.902 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-10T10:04:49.902 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-10T10:04:49.902 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:49.903 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:49.903 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:49.903 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-10T10:04:49.903 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:49.903 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:49.903 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-10T10:04:49.903 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-10T10:04:49.903 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-10T10:04:49.903 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-10T10:04:49.903 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-10T10:04:49.903 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-10T10:04:49.903 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-10T10:04:49.903 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-10T10:04:49.903 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:49.903 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:49.904 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:49.904 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:49.904 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:49.904 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-10T10:04:49.904 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-10T10:04:49.904 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:49.904 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:49.904 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:49.904 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-10T10:04:49.904 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:49.904 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:49.904 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:49.904 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:49.904 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:49.905 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:49.905 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:49.905 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:49.905 INFO:teuthology.orchestra.run.smithi005.stdout: "retain_interval": { 2024-09-10T10:04:49.905 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "retain_interval", 2024-09-10T10:04:49.905 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-10T10:04:49.905 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:49.905 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:49.905 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "31536000", 2024-09-10T10:04:49.905 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:49.905 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:49.905 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:49.905 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-10T10:04:49.906 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:49.906 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:49.906 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:49.906 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:49.906 INFO:teuthology.orchestra.run.smithi005.stdout: "warn_recent_interval": { 2024-09-10T10:04:49.906 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "warn_recent_interval", 2024-09-10T10:04:49.906 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-10T10:04:49.906 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:49.906 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:49.906 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "1209600", 2024-09-10T10:04:49.906 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:49.906 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:49.906 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:49.906 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-10T10:04:49.906 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:49.907 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:49.907 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:49.907 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T10:04:49.907 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T10:04:49.907 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:49.907 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-10T10:04:49.907 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "dashboard", 2024-09-10T10:04:49.907 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-10T10:04:49.907 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-10T10:04:49.907 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-10T10:04:49.907 INFO:teuthology.orchestra.run.smithi005.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-10T10:04:49.907 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-10T10:04:49.907 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-10T10:04:49.907 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:49.908 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:49.908 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10", 2024-09-10T10:04:49.908 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:49.908 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:49.908 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:49.908 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:49.908 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:49.908 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:49.908 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:49.908 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:49.908 INFO:teuthology.orchestra.run.smithi005.stdout: "ALERTMANAGER_API_HOST": { 2024-09-10T10:04:49.908 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-10T10:04:49.908 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:49.908 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:49.909 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:49.909 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-10T10:04:49.909 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:49.909 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:49.909 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:49.909 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:49.909 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:49.909 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:49.909 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:49.909 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:49.909 INFO:teuthology.orchestra.run.smithi005.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-10T10:04:49.909 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-10T10:04:49.909 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:49.910 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:49.910 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:49.910 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-10T10:04:49.910 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:49.910 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:49.910 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:49.910 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:49.910 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:49.910 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:49.910 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:49.910 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:49.910 INFO:teuthology.orchestra.run.smithi005.stdout: "AUDIT_API_ENABLED": { 2024-09-10T10:04:49.910 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "AUDIT_API_ENABLED", 2024-09-10T10:04:49.910 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:49.910 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:49.911 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:49.911 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-10T10:04:49.911 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:49.911 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:49.911 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:49.911 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:49.911 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:49.911 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:49.911 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:49.911 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:49.911 INFO:teuthology.orchestra.run.smithi005.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-10T10:04:49.911 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-10T10:04:49.911 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:49.911 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:49.912 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:49.912 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-10T10:04:49.912 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:49.912 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:49.912 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:49.912 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:49.912 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:49.912 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:49.912 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:49.912 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:49.912 INFO:teuthology.orchestra.run.smithi005.stdout: "ENABLE_BROWSABLE_API": { 2024-09-10T10:04:49.912 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-10T10:04:49.912 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:49.913 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:49.913 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:49.913 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-10T10:04:49.913 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:49.913 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:49.913 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:49.913 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:49.913 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:49.913 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:49.913 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:49.913 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:49.913 INFO:teuthology.orchestra.run.smithi005.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-10T10:04:49.914 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-10T10:04:49.914 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:49.914 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:49.914 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:49.914 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-10T10:04:49.914 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:49.914 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:49.914 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:49.914 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:49.914 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:49.914 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:49.914 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:49.914 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:49.915 INFO:teuthology.orchestra.run.smithi005.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-10T10:04:49.915 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-10T10:04:49.915 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:49.915 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:49.915 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:49.915 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-10T10:04:49.915 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:49.915 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:49.915 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:49.915 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:49.915 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:49.915 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:49.915 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:49.915 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:49.916 INFO:teuthology.orchestra.run.smithi005.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-10T10:04:49.916 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-10T10:04:49.916 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:49.916 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:49.916 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:49.916 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-10T10:04:49.916 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:49.916 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:49.916 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:49.916 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:49.916 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:49.916 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:49.916 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:49.916 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:49.917 INFO:teuthology.orchestra.run.smithi005.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-10T10:04:49.917 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-10T10:04:49.917 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:49.917 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:49.917 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:49.917 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-10T10:04:49.917 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:49.917 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:49.917 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:49.917 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:49.917 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:49.917 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:49.917 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:49.918 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:49.918 INFO:teuthology.orchestra.run.smithi005.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-10T10:04:49.918 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-10T10:04:49.918 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:49.918 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:49.918 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:49.918 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-10T10:04:49.918 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:49.918 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:49.918 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:49.918 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:49.918 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:49.918 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:49.919 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:49.919 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:49.919 INFO:teuthology.orchestra.run.smithi005.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-10T10:04:49.919 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-10T10:04:49.919 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:49.919 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:49.919 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:49.919 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-10T10:04:49.919 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:49.919 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:49.919 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:49.919 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:49.919 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:49.920 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:49.920 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:49.920 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:49.920 INFO:teuthology.orchestra.run.smithi005.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-10T10:04:49.920 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-10T10:04:49.920 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:49.920 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:49.920 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:49.920 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-10T10:04:49.920 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:49.920 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:49.920 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:49.920 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:49.920 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:49.920 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:49.921 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:49.921 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:49.921 INFO:teuthology.orchestra.run.smithi005.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-10T10:04:49.921 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-10T10:04:49.921 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:49.921 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:49.921 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:49.921 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-10T10:04:49.921 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:49.921 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:49.921 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:49.921 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:49.921 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:49.921 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:49.921 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:49.921 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:49.922 INFO:teuthology.orchestra.run.smithi005.stdout: "GRAFANA_API_PASSWORD": { 2024-09-10T10:04:49.922 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-10T10:04:49.922 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:49.922 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:49.922 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:49.922 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "admin", 2024-09-10T10:04:49.922 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:49.922 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:49.922 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:49.922 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:49.922 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:49.922 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:49.922 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:49.922 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:49.922 INFO:teuthology.orchestra.run.smithi005.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-10T10:04:49.923 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-10T10:04:49.923 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:49.923 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:49.923 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:49.923 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-10T10:04:49.923 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:49.923 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:49.923 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:49.923 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:49.923 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:49.923 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:49.923 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:49.923 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:49.924 INFO:teuthology.orchestra.run.smithi005.stdout: "GRAFANA_API_URL": { 2024-09-10T10:04:49.924 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "GRAFANA_API_URL", 2024-09-10T10:04:49.924 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:49.924 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:49.924 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:49.924 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-10T10:04:49.924 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:49.924 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:49.924 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:49.924 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:49.924 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:49.924 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:49.924 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:49.924 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:49.925 INFO:teuthology.orchestra.run.smithi005.stdout: "GRAFANA_API_USERNAME": { 2024-09-10T10:04:49.925 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-10T10:04:49.925 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:49.925 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:49.925 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:49.925 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "admin", 2024-09-10T10:04:49.925 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:49.925 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:49.925 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:49.925 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:49.925 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:49.925 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:49.925 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:49.926 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:49.926 INFO:teuthology.orchestra.run.smithi005.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-10T10:04:49.926 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-10T10:04:49.926 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:49.926 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:49.926 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:49.926 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-10T10:04:49.926 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:49.926 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:49.926 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:49.926 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:49.926 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:49.926 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:49.926 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:49.927 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:49.927 INFO:teuthology.orchestra.run.smithi005.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-10T10:04:49.927 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-10T10:04:49.927 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:49.927 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:49.927 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:49.927 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-10T10:04:49.927 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:49.927 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:49.927 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:49.927 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:49.927 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:49.928 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:49.928 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:49.928 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:49.928 INFO:teuthology.orchestra.run.smithi005.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-10T10:04:49.928 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-10T10:04:49.928 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:49.928 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:49.928 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:49.928 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-10T10:04:49.928 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:49.928 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:49.928 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:49.928 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:49.928 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:49.929 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:49.929 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:49.929 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:49.929 INFO:teuthology.orchestra.run.smithi005.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-10T10:04:49.929 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-10T10:04:49.929 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:49.929 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:49.929 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:49.929 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-10T10:04:49.929 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:49.929 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:49.929 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:49.929 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:49.929 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:49.930 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:49.930 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:49.930 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:49.930 INFO:teuthology.orchestra.run.smithi005.stdout: "PROMETHEUS_API_HOST": { 2024-09-10T10:04:49.930 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-10T10:04:49.930 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:49.930 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:49.930 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:49.930 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-10T10:04:49.930 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:49.930 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:49.930 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:49.930 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:49.930 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:49.931 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:49.931 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:49.931 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:49.931 INFO:teuthology.orchestra.run.smithi005.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-10T10:04:49.931 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-10T10:04:49.931 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:49.931 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:49.931 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:49.931 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-10T10:04:49.931 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:49.931 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:49.931 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:49.931 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:49.931 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:49.932 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:49.932 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:49.932 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:49.932 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-10T10:04:49.932 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-10T10:04:49.932 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:49.932 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:49.932 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:49.932 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-10T10:04:49.932 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:49.932 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:49.932 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:49.932 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:49.932 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:49.933 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:49.933 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:49.933 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:49.933 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-10T10:04:49.933 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-10T10:04:49.933 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:49.933 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:49.933 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:49.933 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-10T10:04:49.933 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:49.933 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:49.933 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:49.933 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:49.934 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:49.934 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:49.934 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:49.934 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:49.934 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-10T10:04:49.934 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-10T10:04:49.934 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:49.934 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:49.934 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:49.934 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-10T10:04:49.934 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:49.934 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:49.934 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:49.934 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:49.935 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:49.935 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:49.935 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:49.935 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:49.935 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-10T10:04:49.935 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-10T10:04:49.935 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:49.935 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:49.935 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:49.935 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-10T10:04:49.935 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:49.935 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:49.936 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:49.936 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:49.936 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:49.936 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:49.936 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:49.936 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:49.936 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-10T10:04:49.936 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-10T10:04:49.936 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:49.936 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:49.936 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:49.936 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-10T10:04:49.937 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:49.937 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:49.937 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:49.937 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:49.937 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:49.937 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:49.937 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:49.937 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:49.937 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-10T10:04:49.937 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-10T10:04:49.937 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:49.937 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:49.938 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:49.938 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-10T10:04:49.938 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:49.938 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:49.938 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:49.938 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:49.938 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:49.938 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:49.938 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:49.938 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:49.938 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-10T10:04:49.938 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-10T10:04:49.938 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:49.939 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:49.939 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:49.939 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-10T10:04:49.939 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:49.939 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:49.939 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:49.939 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:49.939 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:49.939 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:49.939 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:49.939 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:49.939 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_ENABLED": { 2024-09-10T10:04:49.940 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-10T10:04:49.940 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:49.940 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:49.940 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:49.940 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-10T10:04:49.940 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:49.940 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:49.940 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:49.940 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:49.940 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:49.940 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:49.940 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:49.940 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:49.941 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-10T10:04:49.941 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-10T10:04:49.941 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:49.941 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:49.941 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:49.941 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-10T10:04:49.941 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:49.941 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:49.941 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:49.941 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:49.941 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:49.941 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:49.941 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:49.942 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:49.942 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-10T10:04:49.942 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-10T10:04:49.942 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-10T10:04:49.942 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:49.942 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:49.942 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10", 2024-09-10T10:04:49.942 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:49.942 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:49.942 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:49.942 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:49.942 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:49.942 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:49.943 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:49.943 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:49.943 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-10T10:04:49.943 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-10T10:04:49.943 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-10T10:04:49.943 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:49.943 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:49.943 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "8", 2024-09-10T10:04:49.943 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:49.943 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:49.943 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:49.943 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:49.943 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:49.944 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:49.944 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:49.944 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:49.944 INFO:teuthology.orchestra.run.smithi005.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-10T10:04:49.944 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-10T10:04:49.944 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-10T10:04:49.944 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:49.944 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:49.944 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "45", 2024-09-10T10:04:49.944 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:49.944 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:49.944 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:49.944 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:49.944 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:49.945 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:49.945 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:49.945 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:49.945 INFO:teuthology.orchestra.run.smithi005.stdout: "RGW_API_ACCESS_KEY": { 2024-09-10T10:04:49.945 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-10T10:04:49.945 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:49.945 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:49.945 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:49.945 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-10T10:04:49.945 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:49.945 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:49.945 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:49.945 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:49.945 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:49.946 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:49.946 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:49.946 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:49.946 INFO:teuthology.orchestra.run.smithi005.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-10T10:04:49.946 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-10T10:04:49.946 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:49.946 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:49.946 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:49.946 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "admin", 2024-09-10T10:04:49.946 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:49.946 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:49.946 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:49.946 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:49.946 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:49.947 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:49.947 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:49.947 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:49.947 INFO:teuthology.orchestra.run.smithi005.stdout: "RGW_API_SECRET_KEY": { 2024-09-10T10:04:49.947 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-10T10:04:49.947 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:49.947 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:49.947 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:49.947 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-10T10:04:49.947 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:49.947 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:49.947 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:49.947 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:49.948 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:49.948 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:49.948 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:49.948 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:49.948 INFO:teuthology.orchestra.run.smithi005.stdout: "RGW_API_SSL_VERIFY": { 2024-09-10T10:04:49.948 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-10T10:04:49.948 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:49.948 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:49.948 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:49.948 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-10T10:04:49.948 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:49.948 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:49.948 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:49.948 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:49.949 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:49.949 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:49.949 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:49.949 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:49.949 INFO:teuthology.orchestra.run.smithi005.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-10T10:04:49.949 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-10T10:04:49.949 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-10T10:04:49.949 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:49.949 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:49.949 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-09-10T10:04:49.949 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:49.949 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:49.949 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:49.949 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:49.950 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:49.950 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:49.950 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:49.950 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:49.950 INFO:teuthology.orchestra.run.smithi005.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-10T10:04:49.950 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-10T10:04:49.950 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-10T10:04:49.950 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:49.950 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:49.950 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10", 2024-09-10T10:04:49.950 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:49.950 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:49.950 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:49.950 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:49.951 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:49.951 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:49.951 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:49.951 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:49.951 INFO:teuthology.orchestra.run.smithi005.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-10T10:04:49.951 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-10T10:04:49.951 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-10T10:04:49.951 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:49.951 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:49.951 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "5", 2024-09-10T10:04:49.951 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:49.951 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:49.951 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:49.951 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:49.951 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:49.952 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:49.952 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:49.952 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:49.952 INFO:teuthology.orchestra.run.smithi005.stdout: "cross_origin_url": { 2024-09-10T10:04:49.952 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "cross_origin_url", 2024-09-10T10:04:49.952 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:49.952 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:49.952 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:49.952 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-10T10:04:49.952 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:49.952 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:49.952 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:49.952 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:49.952 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:49.953 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:49.953 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:49.953 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:49.953 INFO:teuthology.orchestra.run.smithi005.stdout: "crt_file": { 2024-09-10T10:04:49.953 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "crt_file", 2024-09-10T10:04:49.953 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:49.953 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:49.953 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:49.953 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-10T10:04:49.953 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:49.953 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:49.953 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:49.954 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:49.954 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:49.954 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:49.954 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:49.954 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:49.954 INFO:teuthology.orchestra.run.smithi005.stdout: "debug": { 2024-09-10T10:04:49.954 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "debug", 2024-09-10T10:04:49.954 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:49.954 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:49.954 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:49.954 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-10T10:04:49.954 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:49.955 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:49.955 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:49.955 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Enable/disable debug options", 2024-09-10T10:04:49.955 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:49.955 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:49.955 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:49.955 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:49.955 INFO:teuthology.orchestra.run.smithi005.stdout: "jwt_token_ttl": { 2024-09-10T10:04:49.955 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "jwt_token_ttl", 2024-09-10T10:04:49.955 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-10T10:04:49.955 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:49.955 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:49.955 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "28800", 2024-09-10T10:04:49.956 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:49.956 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:49.956 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:49.956 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:49.956 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:49.956 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:49.956 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:49.956 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:49.956 INFO:teuthology.orchestra.run.smithi005.stdout: "key_file": { 2024-09-10T10:04:49.956 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "key_file", 2024-09-10T10:04:49.956 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:49.956 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:49.957 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:49.957 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-10T10:04:49.957 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:49.957 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:49.957 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:49.957 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:49.957 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:49.957 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:49.957 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:49.957 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:49.957 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-10T10:04:49.957 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-10T10:04:49.957 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:49.958 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:49.958 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:49.958 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-10T10:04:49.958 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:49.958 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:49.958 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-10T10:04:49.958 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-10T10:04:49.958 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-10T10:04:49.958 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-10T10:04:49.958 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-10T10:04:49.958 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-10T10:04:49.958 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-10T10:04:49.958 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-10T10:04:49.958 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:49.959 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:49.959 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:49.959 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:49.959 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:49.959 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-10T10:04:49.959 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-10T10:04:49.959 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:49.959 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:49.959 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:49.959 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-10T10:04:49.959 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:49.959 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:49.959 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:49.959 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:49.960 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:49.960 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:49.960 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:49.960 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:49.960 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-10T10:04:49.960 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-10T10:04:49.960 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:49.960 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:49.960 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:49.960 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-10T10:04:49.960 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:49.960 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:49.960 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-10T10:04:49.961 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-10T10:04:49.961 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-10T10:04:49.961 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-10T10:04:49.961 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-10T10:04:49.961 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-10T10:04:49.961 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-10T10:04:49.961 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-10T10:04:49.961 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:49.961 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:49.961 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:49.961 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:49.961 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:49.961 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-10T10:04:49.961 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-10T10:04:49.962 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:49.962 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:49.962 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:49.962 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-10T10:04:49.962 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:49.962 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:49.962 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:49.962 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:49.962 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:49.962 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:49.962 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:49.962 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:49.962 INFO:teuthology.orchestra.run.smithi005.stdout: "motd": { 2024-09-10T10:04:49.962 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "motd", 2024-09-10T10:04:49.962 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:49.963 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:49.963 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:49.963 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-10T10:04:49.963 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:49.963 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:49.963 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:49.963 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "The message of the day", 2024-09-10T10:04:49.963 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:49.963 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:49.963 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:49.963 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:49.963 INFO:teuthology.orchestra.run.smithi005.stdout: "redirect_resolve_ip_addr": { 2024-09-10T10:04:49.963 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "redirect_resolve_ip_addr", 2024-09-10T10:04:49.963 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:49.963 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:49.964 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:49.964 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-10T10:04:49.964 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:49.964 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:49.964 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:49.964 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:49.964 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:49.964 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:49.964 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:49.964 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:49.964 INFO:teuthology.orchestra.run.smithi005.stdout: "server_addr": { 2024-09-10T10:04:49.964 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "server_addr", 2024-09-10T10:04:49.964 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:49.964 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:49.965 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:49.965 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "::", 2024-09-10T10:04:49.965 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:49.965 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:49.965 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:49.965 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:49.965 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:49.965 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:49.965 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:49.965 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:49.965 INFO:teuthology.orchestra.run.smithi005.stdout: "server_port": { 2024-09-10T10:04:49.965 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "server_port", 2024-09-10T10:04:49.965 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-10T10:04:49.965 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:49.966 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:49.966 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "8080", 2024-09-10T10:04:49.966 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:49.966 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:49.966 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:49.966 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:49.966 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:49.966 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:49.966 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:49.966 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:49.966 INFO:teuthology.orchestra.run.smithi005.stdout: "ssl": { 2024-09-10T10:04:49.966 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ssl", 2024-09-10T10:04:49.966 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:49.967 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:49.967 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:49.967 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-10T10:04:49.967 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:49.967 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:49.967 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:49.967 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:49.967 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:49.967 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:49.967 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:49.967 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:49.967 INFO:teuthology.orchestra.run.smithi005.stdout: "ssl_server_port": { 2024-09-10T10:04:49.967 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ssl_server_port", 2024-09-10T10:04:49.967 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-10T10:04:49.968 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:49.968 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:49.968 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "8443", 2024-09-10T10:04:49.968 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:49.968 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:49.968 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:49.968 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:49.968 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:49.968 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:49.968 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:49.968 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:49.968 INFO:teuthology.orchestra.run.smithi005.stdout: "standby_behaviour": { 2024-09-10T10:04:49.968 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "standby_behaviour", 2024-09-10T10:04:49.969 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:49.969 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:49.969 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:49.969 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "redirect", 2024-09-10T10:04:49.969 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:49.969 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:49.969 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-10T10:04:49.969 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-10T10:04:49.969 INFO:teuthology.orchestra.run.smithi005.stdout: "redirect" 2024-09-10T10:04:49.969 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-10T10:04:49.969 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:49.969 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:49.969 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:49.969 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:49.970 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:49.970 INFO:teuthology.orchestra.run.smithi005.stdout: "standby_error_status_code": { 2024-09-10T10:04:49.970 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "standby_error_status_code", 2024-09-10T10:04:49.970 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-10T10:04:49.970 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:49.970 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:49.970 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "500", 2024-09-10T10:04:49.970 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "400", 2024-09-10T10:04:49.970 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "599", 2024-09-10T10:04:49.970 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:49.970 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:49.970 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:49.970 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:49.970 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:49.970 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:49.971 INFO:teuthology.orchestra.run.smithi005.stdout: "url_prefix": { 2024-09-10T10:04:49.971 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "url_prefix", 2024-09-10T10:04:49.971 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:49.971 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:49.971 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:49.971 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-10T10:04:49.971 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:49.971 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:49.971 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:49.971 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:49.971 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:49.971 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:49.971 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:49.971 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T10:04:49.972 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T10:04:49.972 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:49.972 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-10T10:04:49.972 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "devicehealth", 2024-09-10T10:04:49.972 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-10T10:04:49.972 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-10T10:04:49.972 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-10T10:04:49.972 INFO:teuthology.orchestra.run.smithi005.stdout: "enable_monitoring": { 2024-09-10T10:04:49.972 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "enable_monitoring", 2024-09-10T10:04:49.972 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:49.972 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:49.972 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:49.972 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-10T10:04:49.972 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:49.973 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:49.973 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:49.973 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "monitor device health metrics", 2024-09-10T10:04:49.973 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:49.973 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:49.973 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:49.973 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:49.973 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-10T10:04:49.973 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-10T10:04:49.973 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:49.973 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:49.973 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:49.973 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-10T10:04:49.973 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:49.973 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:49.974 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-10T10:04:49.974 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-10T10:04:49.974 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-10T10:04:49.974 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-10T10:04:49.974 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-10T10:04:49.974 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-10T10:04:49.974 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-10T10:04:49.974 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-10T10:04:49.974 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:49.974 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:49.974 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:49.974 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:49.974 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:49.974 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-10T10:04:49.974 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-10T10:04:49.975 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:49.975 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:49.975 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:49.975 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-10T10:04:49.975 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:49.975 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:49.975 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:49.975 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:49.975 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:49.975 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:49.975 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:49.975 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:49.975 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-10T10:04:49.975 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-10T10:04:49.976 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:49.976 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:49.976 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:49.976 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-10T10:04:49.976 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:49.976 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:49.976 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-10T10:04:49.976 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-10T10:04:49.976 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-10T10:04:49.976 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-10T10:04:49.976 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-10T10:04:49.976 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-10T10:04:49.976 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-10T10:04:49.976 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-10T10:04:49.976 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:49.977 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:49.977 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:49.977 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:49.977 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:49.977 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-10T10:04:49.977 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-10T10:04:49.977 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:49.977 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:49.977 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:49.977 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-10T10:04:49.977 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:49.977 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:49.977 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:49.978 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:49.978 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:49.978 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:49.978 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:49.978 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:49.978 INFO:teuthology.orchestra.run.smithi005.stdout: "mark_out_threshold": { 2024-09-10T10:04:49.978 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "mark_out_threshold", 2024-09-10T10:04:49.978 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-10T10:04:49.978 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:49.978 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:49.978 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "2419200", 2024-09-10T10:04:49.978 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:49.978 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:49.978 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:49.978 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-10T10:04:49.979 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:49.979 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:49.979 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:49.979 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:49.979 INFO:teuthology.orchestra.run.smithi005.stdout: "pool_name": { 2024-09-10T10:04:49.979 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "pool_name", 2024-09-10T10:04:49.979 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:49.979 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:49.979 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:49.979 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "device_health_metrics", 2024-09-10T10:04:49.979 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:49.979 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:49.979 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:49.979 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-10T10:04:49.980 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:49.980 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:49.980 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:49.980 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:49.980 INFO:teuthology.orchestra.run.smithi005.stdout: "retention_period": { 2024-09-10T10:04:49.980 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "retention_period", 2024-09-10T10:04:49.980 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-10T10:04:49.980 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:49.980 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:49.980 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "15552000", 2024-09-10T10:04:49.980 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:49.980 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:49.980 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:49.981 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "how long to retain device health metrics", 2024-09-10T10:04:49.981 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:49.981 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:49.981 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:49.981 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:49.981 INFO:teuthology.orchestra.run.smithi005.stdout: "scrape_frequency": { 2024-09-10T10:04:49.981 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "scrape_frequency", 2024-09-10T10:04:49.981 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-10T10:04:49.981 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:49.981 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:49.981 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "86400", 2024-09-10T10:04:49.981 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:49.982 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:49.982 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:49.982 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-10T10:04:49.982 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:49.982 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:49.982 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:49.982 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:49.982 INFO:teuthology.orchestra.run.smithi005.stdout: "self_heal": { 2024-09-10T10:04:49.982 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "self_heal", 2024-09-10T10:04:49.982 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:49.982 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:49.982 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:49.983 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-10T10:04:49.983 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:49.983 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:49.983 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:49.983 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-10T10:04:49.983 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:49.983 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:49.983 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:49.983 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:49.983 INFO:teuthology.orchestra.run.smithi005.stdout: "sleep_interval": { 2024-09-10T10:04:49.984 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sleep_interval", 2024-09-10T10:04:49.984 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-10T10:04:49.984 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:49.984 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:49.984 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "600", 2024-09-10T10:04:49.984 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:49.984 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:49.984 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:49.984 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "how frequently to wake up and check device health", 2024-09-10T10:04:49.984 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:49.984 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:49.984 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:49.984 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:49.985 INFO:teuthology.orchestra.run.smithi005.stdout: "warn_threshold": { 2024-09-10T10:04:49.985 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "warn_threshold", 2024-09-10T10:04:49.985 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-10T10:04:49.985 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:49.985 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:49.985 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "7257600", 2024-09-10T10:04:49.985 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:49.985 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:49.985 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:49.985 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-10T10:04:49.985 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:49.985 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:49.986 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:49.986 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T10:04:49.986 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T10:04:49.986 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:49.986 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-10T10:04:49.986 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "diskprediction_local", 2024-09-10T10:04:49.986 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-10T10:04:49.986 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-10T10:04:49.986 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-10T10:04:49.986 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-10T10:04:49.986 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-10T10:04:49.986 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:49.986 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:49.987 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:49.987 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-10T10:04:49.987 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:49.987 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:49.987 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-10T10:04:49.987 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-10T10:04:49.987 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-10T10:04:49.987 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-10T10:04:49.987 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-10T10:04:49.987 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-10T10:04:49.987 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-10T10:04:49.988 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-10T10:04:49.988 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:49.988 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:49.988 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:49.988 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:49.988 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:49.988 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-10T10:04:49.988 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-10T10:04:49.988 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:49.988 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:49.988 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:49.988 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-10T10:04:49.989 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:49.989 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:49.989 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:49.989 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:49.989 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:49.989 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:49.989 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:49.989 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:49.989 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-10T10:04:49.989 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-10T10:04:49.989 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:49.989 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:49.989 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:49.990 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-10T10:04:49.990 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:49.990 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:49.990 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-10T10:04:49.990 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-10T10:04:49.990 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-10T10:04:49.990 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-10T10:04:49.990 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-10T10:04:49.990 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-10T10:04:49.990 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-10T10:04:49.990 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-10T10:04:49.990 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:49.990 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:49.991 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:49.991 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:49.991 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:49.991 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-10T10:04:49.991 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-10T10:04:49.991 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:49.991 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:49.991 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:49.991 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-10T10:04:49.991 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:49.991 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:49.991 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:49.991 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:49.991 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:49.992 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:49.992 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:49.992 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:49.992 INFO:teuthology.orchestra.run.smithi005.stdout: "predict_interval": { 2024-09-10T10:04:49.992 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "predict_interval", 2024-09-10T10:04:49.992 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:49.992 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:49.992 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:49.992 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "86400", 2024-09-10T10:04:49.992 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:49.992 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:49.992 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:49.992 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:49.993 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:49.993 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:49.993 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:49.993 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:49.993 INFO:teuthology.orchestra.run.smithi005.stdout: "predictor_model": { 2024-09-10T10:04:49.993 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "predictor_model", 2024-09-10T10:04:49.993 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:49.993 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:49.993 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:49.993 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "prophetstor", 2024-09-10T10:04:49.993 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:49.993 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:49.993 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:49.993 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:49.994 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:49.994 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:49.994 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:49.994 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:49.994 INFO:teuthology.orchestra.run.smithi005.stdout: "sleep_interval": { 2024-09-10T10:04:49.994 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sleep_interval", 2024-09-10T10:04:49.994 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:49.994 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:49.994 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:49.994 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "600", 2024-09-10T10:04:49.994 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:49.994 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:49.994 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:49.995 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:49.995 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:49.995 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:49.995 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:49.995 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T10:04:49.995 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T10:04:49.995 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:49.995 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-10T10:04:49.995 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "influx", 2024-09-10T10:04:49.995 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": false, 2024-09-10T10:04:49.995 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "influxdb python module not found", 2024-09-10T10:04:49.995 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-10T10:04:49.995 INFO:teuthology.orchestra.run.smithi005.stdout: "batch_size": { 2024-09-10T10:04:49.995 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "batch_size", 2024-09-10T10:04:49.995 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-10T10:04:49.996 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:49.996 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:49.996 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "5000", 2024-09-10T10:04:49.996 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:49.996 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:49.996 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:49.996 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-10T10:04:49.996 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:49.996 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:49.996 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:49.996 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:49.996 INFO:teuthology.orchestra.run.smithi005.stdout: "database": { 2024-09-10T10:04:49.997 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "database", 2024-09-10T10:04:49.997 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:49.997 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:49.997 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:49.997 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "ceph", 2024-09-10T10:04:49.997 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:49.997 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:49.997 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:49.997 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-10T10:04:49.997 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:49.997 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:49.997 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:49.997 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:49.997 INFO:teuthology.orchestra.run.smithi005.stdout: "hostname": { 2024-09-10T10:04:49.998 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "hostname", 2024-09-10T10:04:49.998 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:49.998 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:49.998 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:49.998 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-10T10:04:49.998 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:49.998 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:49.998 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:49.998 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "InfluxDB server hostname", 2024-09-10T10:04:49.998 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:49.998 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:49.998 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:49.998 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:49.998 INFO:teuthology.orchestra.run.smithi005.stdout: "interval": { 2024-09-10T10:04:49.999 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "interval", 2024-09-10T10:04:49.999 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-10T10:04:49.999 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:49.999 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:49.999 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "30", 2024-09-10T10:04:49.999 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "5", 2024-09-10T10:04:49.999 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:49.999 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:49.999 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-10T10:04:49.999 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:49.999 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:49.999 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:49.999 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.000 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-10T10:04:50.000 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-10T10:04:50.000 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.000 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.000 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.000 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-10T10:04:50.000 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.000 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.000 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-10T10:04:50.000 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-10T10:04:50.000 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-10T10:04:50.000 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-10T10:04:50.000 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-10T10:04:50.000 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-10T10:04:50.001 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-10T10:04:50.001 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-10T10:04:50.001 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.001 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.001 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.001 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.001 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.001 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-10T10:04:50.001 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-10T10:04:50.001 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.001 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.001 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.001 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-10T10:04:50.001 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.002 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.002 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.002 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.002 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.002 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.002 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.002 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.002 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-10T10:04:50.002 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-10T10:04:50.002 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.002 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.002 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.002 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-10T10:04:50.002 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.003 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.003 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-10T10:04:50.003 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-10T10:04:50.003 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-10T10:04:50.003 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-10T10:04:50.003 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-10T10:04:50.003 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-10T10:04:50.003 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-10T10:04:50.003 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-10T10:04:50.003 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.003 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.003 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.003 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.003 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.004 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-10T10:04:50.004 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-10T10:04:50.004 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.004 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.004 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.004 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-10T10:04:50.004 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.004 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.004 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.004 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.004 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.004 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.004 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.004 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.004 INFO:teuthology.orchestra.run.smithi005.stdout: "password": { 2024-09-10T10:04:50.005 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "password", 2024-09-10T10:04:50.005 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.005 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.005 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.005 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-10T10:04:50.005 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.005 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.005 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.005 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "password of InfluxDB server user", 2024-09-10T10:04:50.005 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.005 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.005 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.005 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.005 INFO:teuthology.orchestra.run.smithi005.stdout: "port": { 2024-09-10T10:04:50.006 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "port", 2024-09-10T10:04:50.006 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-10T10:04:50.006 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.006 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.006 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "8086", 2024-09-10T10:04:50.006 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.006 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.006 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.006 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "InfluxDB server port", 2024-09-10T10:04:50.006 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.006 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.006 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.006 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.006 INFO:teuthology.orchestra.run.smithi005.stdout: "ssl": { 2024-09-10T10:04:50.006 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ssl", 2024-09-10T10:04:50.007 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.007 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.007 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.007 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "false", 2024-09-10T10:04:50.007 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.007 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.007 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.007 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-10T10:04:50.007 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.007 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.007 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.007 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.007 INFO:teuthology.orchestra.run.smithi005.stdout: "threads": { 2024-09-10T10:04:50.007 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "threads", 2024-09-10T10:04:50.008 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-10T10:04:50.008 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.008 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.008 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "5", 2024-09-10T10:04:50.008 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "1", 2024-09-10T10:04:50.008 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "32", 2024-09-10T10:04:50.008 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.008 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-10T10:04:50.008 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.008 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.008 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.008 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.008 INFO:teuthology.orchestra.run.smithi005.stdout: "username": { 2024-09-10T10:04:50.008 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "username", 2024-09-10T10:04:50.009 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.009 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.009 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.009 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-10T10:04:50.009 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.009 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.009 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.009 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "username of InfluxDB server user", 2024-09-10T10:04:50.009 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.009 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.009 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.009 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.009 INFO:teuthology.orchestra.run.smithi005.stdout: "verify_ssl": { 2024-09-10T10:04:50.010 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "verify_ssl", 2024-09-10T10:04:50.010 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.010 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.010 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.010 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "true", 2024-09-10T10:04:50.010 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.010 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.010 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.010 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-10T10:04:50.010 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.010 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.010 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.010 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T10:04:50.011 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T10:04:50.011 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.011 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-10T10:04:50.011 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "insights", 2024-09-10T10:04:50.011 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-10T10:04:50.011 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-10T10:04:50.011 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-10T10:04:50.011 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-10T10:04:50.011 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-10T10:04:50.011 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.011 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.011 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.011 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-10T10:04:50.011 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.012 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.012 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-10T10:04:50.012 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-10T10:04:50.012 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-10T10:04:50.012 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-10T10:04:50.012 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-10T10:04:50.012 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-10T10:04:50.012 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-10T10:04:50.012 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-10T10:04:50.012 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.012 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.012 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.012 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.013 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.013 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-10T10:04:50.013 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-10T10:04:50.013 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.013 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.013 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.013 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-10T10:04:50.013 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.013 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.013 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.013 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.013 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.013 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.014 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.014 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.014 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-10T10:04:50.014 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-10T10:04:50.014 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.014 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.014 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.014 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-10T10:04:50.014 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.014 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.014 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-10T10:04:50.014 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-10T10:04:50.014 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-10T10:04:50.014 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-10T10:04:50.015 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-10T10:04:50.015 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-10T10:04:50.015 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-10T10:04:50.015 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-10T10:04:50.015 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.015 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.015 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.015 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.015 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.015 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-10T10:04:50.015 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-10T10:04:50.015 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.016 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.016 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.016 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-10T10:04:50.016 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.016 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.016 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.016 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.016 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.016 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.016 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.016 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T10:04:50.016 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T10:04:50.016 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.017 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-10T10:04:50.017 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "iostat", 2024-09-10T10:04:50.017 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-10T10:04:50.017 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-10T10:04:50.017 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-10T10:04:50.017 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-10T10:04:50.017 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-10T10:04:50.017 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.017 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.017 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.017 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-10T10:04:50.017 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.017 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.018 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-10T10:04:50.018 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-10T10:04:50.018 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-10T10:04:50.018 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-10T10:04:50.018 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-10T10:04:50.018 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-10T10:04:50.018 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-10T10:04:50.018 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-10T10:04:50.018 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.018 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.018 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.018 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.018 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.018 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-10T10:04:50.019 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-10T10:04:50.019 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.019 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.019 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.019 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-10T10:04:50.019 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.019 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.019 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.019 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.019 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.019 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.019 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.019 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.019 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-10T10:04:50.020 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-10T10:04:50.020 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.020 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.020 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.020 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-10T10:04:50.020 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.020 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.020 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-10T10:04:50.020 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-10T10:04:50.020 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-10T10:04:50.020 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-10T10:04:50.020 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-10T10:04:50.021 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-10T10:04:50.021 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-10T10:04:50.021 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-10T10:04:50.021 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.021 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.021 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.021 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.021 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.021 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-10T10:04:50.021 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-10T10:04:50.021 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.021 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.021 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.022 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-10T10:04:50.022 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.022 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.022 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.022 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.022 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.022 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.022 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.022 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T10:04:50.022 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T10:04:50.022 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.022 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-10T10:04:50.022 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "k8sevents", 2024-09-10T10:04:50.023 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-10T10:04:50.023 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-10T10:04:50.023 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-10T10:04:50.023 INFO:teuthology.orchestra.run.smithi005.stdout: "ceph_event_retention_days": { 2024-09-10T10:04:50.023 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ceph_event_retention_days", 2024-09-10T10:04:50.023 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-10T10:04:50.023 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.023 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.023 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "7", 2024-09-10T10:04:50.023 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.023 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.023 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.024 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Days to hold ceph event information within local cache", 2024-09-10T10:04:50.024 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.024 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.024 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.024 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.024 INFO:teuthology.orchestra.run.smithi005.stdout: "config_check_secs": { 2024-09-10T10:04:50.024 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "config_check_secs", 2024-09-10T10:04:50.024 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-10T10:04:50.024 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.024 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.024 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10", 2024-09-10T10:04:50.024 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "10", 2024-09-10T10:04:50.025 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.025 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.025 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-09-10T10:04:50.025 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.025 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.025 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.025 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.025 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-10T10:04:50.025 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-10T10:04:50.025 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.025 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.025 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.026 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-10T10:04:50.026 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.026 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.026 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-10T10:04:50.026 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-10T10:04:50.026 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-10T10:04:50.026 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-10T10:04:50.026 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-10T10:04:50.026 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-10T10:04:50.026 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-10T10:04:50.026 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-10T10:04:50.026 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.027 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.027 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.027 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.027 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.027 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-10T10:04:50.027 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-10T10:04:50.027 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.027 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.027 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.027 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-10T10:04:50.027 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.027 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.028 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.028 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.028 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.028 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.028 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.028 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.028 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-10T10:04:50.028 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-10T10:04:50.028 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.028 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.028 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.028 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-10T10:04:50.028 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.028 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.029 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-10T10:04:50.029 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-10T10:04:50.029 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-10T10:04:50.029 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-10T10:04:50.029 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-10T10:04:50.029 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-10T10:04:50.029 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-10T10:04:50.029 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-10T10:04:50.029 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.029 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.029 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.029 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.029 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.030 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-10T10:04:50.030 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-10T10:04:50.030 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.030 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.030 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.030 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-10T10:04:50.030 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.030 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.030 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.030 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.030 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.030 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.030 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.031 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T10:04:50.031 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T10:04:50.031 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.031 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-10T10:04:50.031 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "localpool", 2024-09-10T10:04:50.031 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-10T10:04:50.031 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-10T10:04:50.031 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-10T10:04:50.031 INFO:teuthology.orchestra.run.smithi005.stdout: "failure_domain": { 2024-09-10T10:04:50.031 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "failure_domain", 2024-09-10T10:04:50.031 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.031 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.031 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.032 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "host", 2024-09-10T10:04:50.032 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.032 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.032 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.032 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "failure domain for any created local pool", 2024-09-10T10:04:50.032 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-10T10:04:50.032 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.032 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.032 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.032 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-10T10:04:50.032 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-10T10:04:50.032 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.032 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.033 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.033 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-10T10:04:50.033 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.033 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.033 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-10T10:04:50.033 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-10T10:04:50.033 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-10T10:04:50.033 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-10T10:04:50.033 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-10T10:04:50.033 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-10T10:04:50.033 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-10T10:04:50.033 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-10T10:04:50.033 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.034 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.034 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.034 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.034 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.034 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-10T10:04:50.034 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-10T10:04:50.034 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.034 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.034 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.034 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-10T10:04:50.034 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.034 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.034 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.035 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.035 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.035 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.035 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.035 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.035 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-10T10:04:50.035 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-10T10:04:50.035 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.035 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.035 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.035 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-10T10:04:50.035 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.036 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.036 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-10T10:04:50.036 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-10T10:04:50.036 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-10T10:04:50.036 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-10T10:04:50.036 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-10T10:04:50.036 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-10T10:04:50.036 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-10T10:04:50.036 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-10T10:04:50.036 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.036 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.036 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.036 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.037 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.037 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-10T10:04:50.037 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-10T10:04:50.037 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.037 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.037 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.037 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-10T10:04:50.037 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.037 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.037 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.037 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.037 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.037 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.038 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.038 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.038 INFO:teuthology.orchestra.run.smithi005.stdout: "min_size": { 2024-09-10T10:04:50.038 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "min_size", 2024-09-10T10:04:50.038 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-10T10:04:50.038 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.038 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.038 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-10T10:04:50.038 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.038 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.038 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.038 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "default min_size for any created local pool", 2024-09-10T10:04:50.038 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-10T10:04:50.038 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.039 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.039 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.039 INFO:teuthology.orchestra.run.smithi005.stdout: "num_rep": { 2024-09-10T10:04:50.039 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "num_rep", 2024-09-10T10:04:50.039 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-10T10:04:50.039 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.039 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.039 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "3", 2024-09-10T10:04:50.039 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.039 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.039 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.039 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "default replica count for any created local pool", 2024-09-10T10:04:50.039 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.039 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.040 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.040 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.040 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_num": { 2024-09-10T10:04:50.040 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "pg_num", 2024-09-10T10:04:50.040 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-10T10:04:50.040 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.040 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.040 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "128", 2024-09-10T10:04:50.040 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.040 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.040 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.040 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "default pg_num for any created local pool", 2024-09-10T10:04:50.040 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.040 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.041 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.041 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.041 INFO:teuthology.orchestra.run.smithi005.stdout: "prefix": { 2024-09-10T10:04:50.041 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "prefix", 2024-09-10T10:04:50.041 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.041 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.041 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.041 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-10T10:04:50.041 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.041 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.041 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.041 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "name prefix for any created local pool", 2024-09-10T10:04:50.041 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.042 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.042 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.042 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.042 INFO:teuthology.orchestra.run.smithi005.stdout: "subtree": { 2024-09-10T10:04:50.042 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "subtree", 2024-09-10T10:04:50.042 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.042 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.042 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.042 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "rack", 2024-09-10T10:04:50.042 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.042 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.042 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.042 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-10T10:04:50.042 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-10T10:04:50.043 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.043 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.043 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T10:04:50.043 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T10:04:50.043 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.043 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-10T10:04:50.043 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "mds_autoscaler", 2024-09-10T10:04:50.043 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-10T10:04:50.043 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-10T10:04:50.043 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-10T10:04:50.043 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-10T10:04:50.043 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-10T10:04:50.043 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.043 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.044 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.044 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-10T10:04:50.044 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.044 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.044 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-10T10:04:50.044 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-10T10:04:50.044 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-10T10:04:50.044 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-10T10:04:50.044 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-10T10:04:50.044 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-10T10:04:50.044 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-10T10:04:50.044 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-10T10:04:50.044 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.044 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.045 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.045 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.045 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.045 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-10T10:04:50.045 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-10T10:04:50.045 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.045 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.045 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.045 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-10T10:04:50.045 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.045 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.045 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.046 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.046 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.046 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.046 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.046 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.046 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-10T10:04:50.046 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-10T10:04:50.046 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.046 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.046 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.046 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-10T10:04:50.046 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.046 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.046 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-10T10:04:50.047 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-10T10:04:50.047 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-10T10:04:50.047 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-10T10:04:50.047 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-10T10:04:50.047 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-10T10:04:50.047 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-10T10:04:50.047 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-10T10:04:50.047 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.047 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.047 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.047 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.047 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.047 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-10T10:04:50.047 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-10T10:04:50.048 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.048 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.048 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.048 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-10T10:04:50.048 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.048 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.048 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.048 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.048 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.048 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.048 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.048 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T10:04:50.048 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T10:04:50.048 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.049 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-10T10:04:50.049 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "mirroring", 2024-09-10T10:04:50.049 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-10T10:04:50.049 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-10T10:04:50.049 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-10T10:04:50.049 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-10T10:04:50.049 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-10T10:04:50.049 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.049 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.049 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.049 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-10T10:04:50.049 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.049 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.049 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-10T10:04:50.049 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-10T10:04:50.050 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-10T10:04:50.050 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-10T10:04:50.050 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-10T10:04:50.050 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-10T10:04:50.050 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-10T10:04:50.050 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-10T10:04:50.050 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.050 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.050 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.050 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.050 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.050 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-10T10:04:50.050 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-10T10:04:50.051 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.051 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.051 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.051 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-10T10:04:50.051 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.051 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.051 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.051 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.051 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.051 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.051 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.051 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.052 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-10T10:04:50.052 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-10T10:04:50.052 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.052 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.052 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.052 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-10T10:04:50.052 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.052 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.052 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-10T10:04:50.052 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-10T10:04:50.052 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-10T10:04:50.052 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-10T10:04:50.052 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-10T10:04:50.052 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-10T10:04:50.053 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-10T10:04:50.053 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-10T10:04:50.053 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.053 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.053 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.053 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.053 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.053 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-10T10:04:50.053 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-10T10:04:50.053 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.053 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.053 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.054 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-10T10:04:50.054 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.054 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.054 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.054 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.054 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.054 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.054 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.054 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T10:04:50.054 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T10:04:50.054 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.054 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-10T10:04:50.054 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "nfs", 2024-09-10T10:04:50.055 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-10T10:04:50.055 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-10T10:04:50.055 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-10T10:04:50.055 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-10T10:04:50.055 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-10T10:04:50.055 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.055 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.055 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.055 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-10T10:04:50.055 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.055 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.055 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-10T10:04:50.055 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-10T10:04:50.055 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-10T10:04:50.056 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-10T10:04:50.056 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-10T10:04:50.056 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-10T10:04:50.056 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-10T10:04:50.056 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-10T10:04:50.056 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.056 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.056 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.056 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.056 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.056 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-10T10:04:50.056 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-10T10:04:50.057 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.057 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.057 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.057 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-10T10:04:50.057 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.057 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.057 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.057 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.057 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.057 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.057 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.057 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.057 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-10T10:04:50.058 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-10T10:04:50.058 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.058 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.058 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.058 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-10T10:04:50.058 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.058 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.058 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-10T10:04:50.058 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-10T10:04:50.058 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-10T10:04:50.058 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-10T10:04:50.058 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-10T10:04:50.058 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-10T10:04:50.058 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-10T10:04:50.058 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-10T10:04:50.059 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.059 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.059 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.059 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.059 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.059 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-10T10:04:50.059 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-10T10:04:50.059 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.059 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.059 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.059 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-10T10:04:50.059 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.059 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.059 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.060 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.060 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.060 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.060 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.060 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T10:04:50.060 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T10:04:50.060 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.060 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-10T10:04:50.060 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "orchestrator", 2024-09-10T10:04:50.060 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-10T10:04:50.060 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-10T10:04:50.060 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-10T10:04:50.060 INFO:teuthology.orchestra.run.smithi005.stdout: "fail_fs": { 2024-09-10T10:04:50.060 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "fail_fs", 2024-09-10T10:04:50.060 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.061 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.061 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.061 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-10T10:04:50.061 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.061 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.061 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.061 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-10T10:04:50.061 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.061 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.061 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.061 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.061 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-10T10:04:50.061 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-10T10:04:50.061 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.062 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.062 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.062 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-10T10:04:50.062 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.062 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.062 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-10T10:04:50.062 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-10T10:04:50.062 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-10T10:04:50.062 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-10T10:04:50.062 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-10T10:04:50.062 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-10T10:04:50.062 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-10T10:04:50.062 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-10T10:04:50.062 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.062 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.063 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.063 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.063 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.063 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-10T10:04:50.063 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-10T10:04:50.063 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.063 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.063 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.063 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-10T10:04:50.063 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.063 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.063 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.063 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.063 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.063 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.064 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.064 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.064 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-10T10:04:50.064 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-10T10:04:50.064 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.064 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.064 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.064 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-10T10:04:50.064 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.064 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.064 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-10T10:04:50.064 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-10T10:04:50.064 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-10T10:04:50.064 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-10T10:04:50.065 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-10T10:04:50.065 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-10T10:04:50.065 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-10T10:04:50.065 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-10T10:04:50.065 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.065 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.065 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.065 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.065 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.065 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-10T10:04:50.065 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-10T10:04:50.065 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.065 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.066 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.066 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-10T10:04:50.066 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.066 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.066 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.066 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.066 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.066 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.066 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.066 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.066 INFO:teuthology.orchestra.run.smithi005.stdout: "orchestrator": { 2024-09-10T10:04:50.066 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "orchestrator", 2024-09-10T10:04:50.066 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.066 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.067 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.067 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-10T10:04:50.067 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.067 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.067 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-10T10:04:50.067 INFO:teuthology.orchestra.run.smithi005.stdout: "cephadm", 2024-09-10T10:04:50.067 INFO:teuthology.orchestra.run.smithi005.stdout: "rook", 2024-09-10T10:04:50.067 INFO:teuthology.orchestra.run.smithi005.stdout: "test_orchestrator" 2024-09-10T10:04:50.067 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-10T10:04:50.067 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Orchestrator backend", 2024-09-10T10:04:50.067 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.067 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.067 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.067 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T10:04:50.068 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T10:04:50.068 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.068 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-10T10:04:50.068 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "osd_perf_query", 2024-09-10T10:04:50.068 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-10T10:04:50.068 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-10T10:04:50.068 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-10T10:04:50.068 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-10T10:04:50.068 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-10T10:04:50.068 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.068 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.068 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.068 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-10T10:04:50.069 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.069 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.069 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-10T10:04:50.069 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-10T10:04:50.069 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-10T10:04:50.069 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-10T10:04:50.069 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-10T10:04:50.069 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-10T10:04:50.069 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-10T10:04:50.069 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-10T10:04:50.069 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.069 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.069 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.069 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.070 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.070 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-10T10:04:50.070 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-10T10:04:50.070 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.070 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.070 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.070 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-10T10:04:50.070 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.070 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.070 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.070 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.070 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.070 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.070 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.070 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.071 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-10T10:04:50.071 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-10T10:04:50.071 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.071 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.071 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.071 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-10T10:04:50.071 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.071 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.071 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-10T10:04:50.071 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-10T10:04:50.071 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-10T10:04:50.071 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-10T10:04:50.071 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-10T10:04:50.071 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-10T10:04:50.071 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-10T10:04:50.072 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-10T10:04:50.072 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.072 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.072 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.072 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.072 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.072 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-10T10:04:50.072 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-10T10:04:50.072 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.072 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.072 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.072 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-10T10:04:50.072 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.073 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.073 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.073 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.073 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.073 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.073 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.073 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T10:04:50.073 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T10:04:50.073 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.073 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-10T10:04:50.073 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "osd_support", 2024-09-10T10:04:50.073 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-10T10:04:50.073 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-10T10:04:50.073 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-10T10:04:50.074 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-10T10:04:50.074 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-10T10:04:50.074 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.074 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.074 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.074 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-10T10:04:50.074 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.074 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.074 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-10T10:04:50.074 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-10T10:04:50.074 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-10T10:04:50.074 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-10T10:04:50.074 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-10T10:04:50.074 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-10T10:04:50.075 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-10T10:04:50.075 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-10T10:04:50.075 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.075 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.075 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.075 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.075 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.075 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-10T10:04:50.075 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-10T10:04:50.075 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.075 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.075 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.075 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-10T10:04:50.075 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.075 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.076 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.076 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.076 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.076 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.076 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.076 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.076 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-10T10:04:50.076 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-10T10:04:50.076 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.076 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.076 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.076 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-10T10:04:50.076 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.076 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.077 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-10T10:04:50.077 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-10T10:04:50.077 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-10T10:04:50.077 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-10T10:04:50.077 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-10T10:04:50.077 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-10T10:04:50.077 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-10T10:04:50.077 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-10T10:04:50.077 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.077 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.077 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.077 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.077 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.078 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-10T10:04:50.078 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-10T10:04:50.078 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.078 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.078 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.078 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-10T10:04:50.078 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.078 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.078 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.078 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.078 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.078 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.078 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.078 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T10:04:50.079 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T10:04:50.079 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.079 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-10T10:04:50.079 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "pg_autoscaler", 2024-09-10T10:04:50.079 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-10T10:04:50.079 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-10T10:04:50.079 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-10T10:04:50.079 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-10T10:04:50.079 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-10T10:04:50.079 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.079 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.079 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.079 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-10T10:04:50.079 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.079 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.080 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-10T10:04:50.080 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-10T10:04:50.080 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-10T10:04:50.080 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-10T10:04:50.080 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-10T10:04:50.080 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-10T10:04:50.080 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-10T10:04:50.080 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-10T10:04:50.080 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.080 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.080 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.080 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.080 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.080 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-10T10:04:50.080 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-10T10:04:50.081 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.081 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.081 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.081 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-10T10:04:50.081 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.081 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.081 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.081 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.081 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.081 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.081 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.081 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.081 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-10T10:04:50.081 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-10T10:04:50.082 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.082 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.082 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.082 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-10T10:04:50.082 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.082 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.082 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-10T10:04:50.082 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-10T10:04:50.082 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-10T10:04:50.082 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-10T10:04:50.082 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-10T10:04:50.082 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-10T10:04:50.082 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-10T10:04:50.083 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-10T10:04:50.083 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.083 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.083 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.083 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.083 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.083 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-10T10:04:50.083 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-10T10:04:50.083 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.083 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.083 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.083 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-10T10:04:50.083 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.083 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.083 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.084 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.084 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.084 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.084 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.084 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.084 INFO:teuthology.orchestra.run.smithi005.stdout: "sleep_interval": { 2024-09-10T10:04:50.084 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sleep_interval", 2024-09-10T10:04:50.084 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-10T10:04:50.084 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.084 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.084 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "60", 2024-09-10T10:04:50.084 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.084 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.085 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.085 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.085 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.085 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.085 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.085 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.085 INFO:teuthology.orchestra.run.smithi005.stdout: "threshold": { 2024-09-10T10:04:50.085 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "threshold", 2024-09-10T10:04:50.085 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "float", 2024-09-10T10:04:50.085 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.085 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.085 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "3.0", 2024-09-10T10:04:50.085 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "1.0", 2024-09-10T10:04:50.086 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.086 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.086 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "scaling threshold", 2024-09-10T10:04:50.086 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-10T10:04:50.086 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.086 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.086 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T10:04:50.086 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T10:04:50.086 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.086 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-10T10:04:50.086 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "progress", 2024-09-10T10:04:50.086 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-10T10:04:50.087 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-10T10:04:50.087 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-10T10:04:50.087 INFO:teuthology.orchestra.run.smithi005.stdout: "allow_pg_recovery_event": { 2024-09-10T10:04:50.087 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "allow_pg_recovery_event", 2024-09-10T10:04:50.087 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.087 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.087 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.087 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-10T10:04:50.087 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.087 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.087 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.087 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-10T10:04:50.087 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.088 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.088 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.088 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.088 INFO:teuthology.orchestra.run.smithi005.stdout: "enabled": { 2024-09-10T10:04:50.088 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "enabled", 2024-09-10T10:04:50.088 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.088 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.088 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.088 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-10T10:04:50.088 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.088 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.088 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.088 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.088 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.089 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.089 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.089 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.089 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-10T10:04:50.089 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-10T10:04:50.089 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.089 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.089 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.089 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-10T10:04:50.089 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.089 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.089 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-10T10:04:50.089 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-10T10:04:50.090 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-10T10:04:50.090 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-10T10:04:50.090 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-10T10:04:50.090 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-10T10:04:50.090 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-10T10:04:50.090 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-10T10:04:50.090 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.090 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.090 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.090 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.090 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.090 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-10T10:04:50.090 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-10T10:04:50.091 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.091 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.091 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.091 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-10T10:04:50.091 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.091 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.091 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.091 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.091 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.091 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.091 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.091 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.091 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-10T10:04:50.091 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-10T10:04:50.092 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.092 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.092 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.092 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-10T10:04:50.092 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.092 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.092 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-10T10:04:50.092 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-10T10:04:50.092 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-10T10:04:50.092 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-10T10:04:50.092 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-10T10:04:50.092 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-10T10:04:50.092 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-10T10:04:50.092 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-10T10:04:50.093 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.093 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.093 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.093 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.093 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.093 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-10T10:04:50.093 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-10T10:04:50.093 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.093 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.093 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.093 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-10T10:04:50.093 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.093 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.094 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.094 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.094 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.094 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.094 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.094 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.094 INFO:teuthology.orchestra.run.smithi005.stdout: "max_completed_events": { 2024-09-10T10:04:50.094 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "max_completed_events", 2024-09-10T10:04:50.094 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-10T10:04:50.094 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.094 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.094 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "50", 2024-09-10T10:04:50.094 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.094 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.095 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.095 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "number of past completed events to remember", 2024-09-10T10:04:50.095 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.095 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.095 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.095 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.095 INFO:teuthology.orchestra.run.smithi005.stdout: "sleep_interval": { 2024-09-10T10:04:50.095 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sleep_interval", 2024-09-10T10:04:50.095 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-10T10:04:50.095 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.095 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.095 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "5", 2024-09-10T10:04:50.095 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.095 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.095 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.096 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "how long the module is going to sleep", 2024-09-10T10:04:50.096 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.096 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.096 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.096 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T10:04:50.096 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T10:04:50.096 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.096 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-10T10:04:50.096 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "prometheus", 2024-09-10T10:04:50.096 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-10T10:04:50.096 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-10T10:04:50.096 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-10T10:04:50.097 INFO:teuthology.orchestra.run.smithi005.stdout: "cache": { 2024-09-10T10:04:50.097 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "cache", 2024-09-10T10:04:50.097 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.097 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.097 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.097 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-10T10:04:50.097 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.097 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.097 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.097 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.097 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.097 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.097 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.097 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.098 INFO:teuthology.orchestra.run.smithi005.stdout: "exclude_perf_counters": { 2024-09-10T10:04:50.098 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "exclude_perf_counters", 2024-09-10T10:04:50.098 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.098 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.098 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.098 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-10T10:04:50.098 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.098 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.098 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.098 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-10T10:04:50.098 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-10T10:04:50.098 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.098 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.098 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.099 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-10T10:04:50.099 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-10T10:04:50.099 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.099 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.099 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.099 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-10T10:04:50.099 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.099 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.099 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-10T10:04:50.099 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-10T10:04:50.099 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-10T10:04:50.099 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-10T10:04:50.099 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-10T10:04:50.099 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-10T10:04:50.100 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-10T10:04:50.100 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-10T10:04:50.100 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.100 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.100 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.100 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.100 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.100 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-10T10:04:50.100 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-10T10:04:50.100 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.100 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.100 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.101 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-10T10:04:50.101 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.101 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.101 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.101 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.101 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.101 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.101 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.101 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.101 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-10T10:04:50.101 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-10T10:04:50.101 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.101 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.102 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.102 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-10T10:04:50.102 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.102 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.102 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-10T10:04:50.102 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-10T10:04:50.102 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-10T10:04:50.102 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-10T10:04:50.102 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-10T10:04:50.102 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-10T10:04:50.102 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-10T10:04:50.102 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-10T10:04:50.102 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.102 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.103 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.103 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.103 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.103 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-10T10:04:50.103 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-10T10:04:50.103 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.103 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.103 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.103 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-10T10:04:50.103 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.103 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.103 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.103 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.104 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.104 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.104 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.104 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.104 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_stats_pools": { 2024-09-10T10:04:50.104 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rbd_stats_pools", 2024-09-10T10:04:50.104 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.104 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.104 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.104 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-10T10:04:50.104 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.104 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.104 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.104 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.105 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.105 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.105 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.105 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.105 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-10T10:04:50.105 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-10T10:04:50.105 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-10T10:04:50.105 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.105 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.105 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "300", 2024-09-10T10:04:50.105 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.105 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.105 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.105 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.106 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.106 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.106 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.106 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.106 INFO:teuthology.orchestra.run.smithi005.stdout: "scrape_interval": { 2024-09-10T10:04:50.106 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "scrape_interval", 2024-09-10T10:04:50.106 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "float", 2024-09-10T10:04:50.106 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.106 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.106 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "15.0", 2024-09-10T10:04:50.106 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.106 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.106 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.106 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.107 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.107 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.107 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.107 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.107 INFO:teuthology.orchestra.run.smithi005.stdout: "server_addr": { 2024-09-10T10:04:50.107 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "server_addr", 2024-09-10T10:04:50.107 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.107 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.107 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.107 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "::", 2024-09-10T10:04:50.107 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.107 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.107 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.107 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-10T10:04:50.108 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.108 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.108 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.108 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.108 INFO:teuthology.orchestra.run.smithi005.stdout: "server_port": { 2024-09-10T10:04:50.108 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "server_port", 2024-09-10T10:04:50.108 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-10T10:04:50.108 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.108 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.108 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "9283", 2024-09-10T10:04:50.108 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.108 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.108 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.109 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-10T10:04:50.109 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.109 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.109 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.109 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.109 INFO:teuthology.orchestra.run.smithi005.stdout: "stale_cache_strategy": { 2024-09-10T10:04:50.109 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "stale_cache_strategy", 2024-09-10T10:04:50.109 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.109 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.109 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.109 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "log", 2024-09-10T10:04:50.109 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.109 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.109 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.110 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.110 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.110 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.110 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.110 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.110 INFO:teuthology.orchestra.run.smithi005.stdout: "standby_behaviour": { 2024-09-10T10:04:50.110 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "standby_behaviour", 2024-09-10T10:04:50.110 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.110 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.110 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.110 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "default", 2024-09-10T10:04:50.110 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.110 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.110 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-10T10:04:50.110 INFO:teuthology.orchestra.run.smithi005.stdout: "default", 2024-09-10T10:04:50.111 INFO:teuthology.orchestra.run.smithi005.stdout: "error" 2024-09-10T10:04:50.111 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-10T10:04:50.111 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.111 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.111 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.111 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.111 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.111 INFO:teuthology.orchestra.run.smithi005.stdout: "standby_error_status_code": { 2024-09-10T10:04:50.111 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "standby_error_status_code", 2024-09-10T10:04:50.111 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-10T10:04:50.111 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.111 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.111 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "500", 2024-09-10T10:04:50.111 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "400", 2024-09-10T10:04:50.111 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "599", 2024-09-10T10:04:50.111 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.112 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.112 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.112 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.112 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.112 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T10:04:50.112 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T10:04:50.112 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.112 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-10T10:04:50.112 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rbd_support", 2024-09-10T10:04:50.112 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-10T10:04:50.112 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-10T10:04:50.112 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-10T10:04:50.112 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-10T10:04:50.112 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-10T10:04:50.113 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.113 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.113 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.113 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-10T10:04:50.113 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.113 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.113 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-10T10:04:50.113 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-10T10:04:50.113 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-10T10:04:50.113 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-10T10:04:50.113 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-10T10:04:50.113 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-10T10:04:50.113 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-10T10:04:50.113 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-10T10:04:50.114 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.114 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.114 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.114 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.114 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.114 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-10T10:04:50.114 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-10T10:04:50.114 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.114 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.114 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.114 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-10T10:04:50.114 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.114 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.114 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.115 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.115 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.115 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.115 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.115 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.115 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-10T10:04:50.115 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-10T10:04:50.115 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.115 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.115 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.115 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-10T10:04:50.115 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.115 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.115 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-10T10:04:50.115 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-10T10:04:50.116 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-10T10:04:50.116 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-10T10:04:50.116 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-10T10:04:50.116 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-10T10:04:50.116 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-10T10:04:50.116 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-10T10:04:50.116 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.116 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.116 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.116 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.116 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.116 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-10T10:04:50.116 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-10T10:04:50.116 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.116 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.117 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.117 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-10T10:04:50.117 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.117 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.117 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.117 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.117 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.117 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.117 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.117 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.117 INFO:teuthology.orchestra.run.smithi005.stdout: "max_concurrent_snap_create": { 2024-09-10T10:04:50.117 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "max_concurrent_snap_create", 2024-09-10T10:04:50.117 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-10T10:04:50.118 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.118 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.118 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10", 2024-09-10T10:04:50.118 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.118 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.118 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.118 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.118 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.118 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.118 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.118 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.118 INFO:teuthology.orchestra.run.smithi005.stdout: "mirror_snapshot_schedule": { 2024-09-10T10:04:50.118 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "mirror_snapshot_schedule", 2024-09-10T10:04:50.118 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.119 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.119 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.119 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-10T10:04:50.119 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.119 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.119 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.119 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.119 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.119 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.119 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.119 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.119 INFO:teuthology.orchestra.run.smithi005.stdout: "trash_purge_schedule": { 2024-09-10T10:04:50.119 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "trash_purge_schedule", 2024-09-10T10:04:50.119 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.119 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.120 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.120 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-10T10:04:50.120 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.120 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.120 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.120 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.120 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.120 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.120 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.120 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T10:04:50.120 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T10:04:50.120 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.120 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-10T10:04:50.120 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "restful", 2024-09-10T10:04:50.120 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-10T10:04:50.121 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-10T10:04:50.121 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-10T10:04:50.121 INFO:teuthology.orchestra.run.smithi005.stdout: "enable_auth": { 2024-09-10T10:04:50.121 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "enable_auth", 2024-09-10T10:04:50.121 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.121 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.121 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.121 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-10T10:04:50.121 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.121 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.121 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.121 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.121 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.122 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.122 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.122 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.122 INFO:teuthology.orchestra.run.smithi005.stdout: "key_file": { 2024-09-10T10:04:50.122 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "key_file", 2024-09-10T10:04:50.122 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.122 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.122 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.122 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-10T10:04:50.122 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.122 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.122 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.122 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.122 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.123 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.123 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.123 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.123 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-10T10:04:50.123 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-10T10:04:50.123 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.123 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.123 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.123 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-10T10:04:50.124 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.124 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.124 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-10T10:04:50.124 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-10T10:04:50.124 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-10T10:04:50.124 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-10T10:04:50.124 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-10T10:04:50.124 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-10T10:04:50.124 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-10T10:04:50.124 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-10T10:04:50.124 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.124 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.124 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.125 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.125 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.125 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-10T10:04:50.125 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-10T10:04:50.125 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.125 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.125 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.125 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-10T10:04:50.125 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.125 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.125 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.125 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.125 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.126 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.126 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.126 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.126 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-10T10:04:50.126 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-10T10:04:50.126 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.126 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.126 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.126 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-10T10:04:50.126 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.126 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.126 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-10T10:04:50.126 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-10T10:04:50.127 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-10T10:04:50.127 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-10T10:04:50.127 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-10T10:04:50.127 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-10T10:04:50.127 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-10T10:04:50.127 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-10T10:04:50.127 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.127 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.127 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.127 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.127 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.127 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-10T10:04:50.128 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-10T10:04:50.128 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.128 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.128 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.128 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-10T10:04:50.128 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.128 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.128 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.128 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.128 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.128 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.128 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.128 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.129 INFO:teuthology.orchestra.run.smithi005.stdout: "server_addr": { 2024-09-10T10:04:50.129 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "server_addr", 2024-09-10T10:04:50.129 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.129 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.129 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.129 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-10T10:04:50.129 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.129 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.129 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.129 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.129 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.129 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.129 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.129 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.130 INFO:teuthology.orchestra.run.smithi005.stdout: "server_port": { 2024-09-10T10:04:50.130 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "server_port", 2024-09-10T10:04:50.130 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.130 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.130 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.130 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-10T10:04:50.130 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.130 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.130 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.130 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.130 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.130 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.131 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.131 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T10:04:50.131 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T10:04:50.131 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.131 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-10T10:04:50.131 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rgw", 2024-09-10T10:04:50.131 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-10T10:04:50.131 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-10T10:04:50.131 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-10T10:04:50.131 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-10T10:04:50.131 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-10T10:04:50.131 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.131 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.132 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.132 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-10T10:04:50.132 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.132 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.132 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-10T10:04:50.132 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-10T10:04:50.132 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-10T10:04:50.132 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-10T10:04:50.132 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-10T10:04:50.132 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-10T10:04:50.132 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-10T10:04:50.132 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-10T10:04:50.133 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.133 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.133 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.133 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.133 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.133 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-10T10:04:50.133 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-10T10:04:50.133 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.133 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.133 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.133 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-10T10:04:50.133 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.133 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.134 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.134 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.134 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.134 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.134 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.134 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.134 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-10T10:04:50.134 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-10T10:04:50.134 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.134 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.134 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.134 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-10T10:04:50.134 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.134 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.135 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-10T10:04:50.135 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-10T10:04:50.135 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-10T10:04:50.135 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-10T10:04:50.135 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-10T10:04:50.135 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-10T10:04:50.135 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-10T10:04:50.135 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-10T10:04:50.135 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.135 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.135 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.135 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.135 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.136 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-10T10:04:50.136 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-10T10:04:50.136 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.136 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.136 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.136 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-10T10:04:50.136 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.136 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.136 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.136 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.136 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.136 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.136 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.137 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T10:04:50.137 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T10:04:50.137 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.137 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-10T10:04:50.137 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rook", 2024-09-10T10:04:50.137 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-10T10:04:50.137 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-10T10:04:50.137 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-10T10:04:50.137 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-10T10:04:50.137 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-10T10:04:50.137 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.137 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.137 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.137 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-10T10:04:50.138 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.138 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.138 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-10T10:04:50.138 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-10T10:04:50.138 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-10T10:04:50.138 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-10T10:04:50.138 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-10T10:04:50.138 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-10T10:04:50.138 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-10T10:04:50.138 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-10T10:04:50.138 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.138 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.139 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.139 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.139 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.139 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-10T10:04:50.139 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-10T10:04:50.139 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.139 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.139 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.139 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-10T10:04:50.139 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.139 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.139 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.139 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.140 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.140 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.140 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.140 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.140 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-10T10:04:50.140 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-10T10:04:50.140 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.140 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.140 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.140 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-10T10:04:50.140 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.140 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.140 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-10T10:04:50.141 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-10T10:04:50.141 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-10T10:04:50.141 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-10T10:04:50.141 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-10T10:04:50.141 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-10T10:04:50.141 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-10T10:04:50.141 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-10T10:04:50.141 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.141 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.141 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.141 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.141 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.141 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-10T10:04:50.142 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-10T10:04:50.142 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.142 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.142 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.142 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-10T10:04:50.142 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.142 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.142 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.142 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.142 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.142 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.142 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.143 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.143 INFO:teuthology.orchestra.run.smithi005.stdout: "storage_class": { 2024-09-10T10:04:50.143 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "storage_class", 2024-09-10T10:04:50.143 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.143 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.143 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.143 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "local", 2024-09-10T10:04:50.143 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.143 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.143 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.143 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-10T10:04:50.143 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.143 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.143 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.144 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T10:04:50.144 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T10:04:50.144 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.144 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-10T10:04:50.144 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "selftest", 2024-09-10T10:04:50.144 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-10T10:04:50.144 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-10T10:04:50.144 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-10T10:04:50.144 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-10T10:04:50.144 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-10T10:04:50.144 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.144 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.144 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.145 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-10T10:04:50.145 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.145 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.145 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-10T10:04:50.145 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-10T10:04:50.145 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-10T10:04:50.145 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-10T10:04:50.145 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-10T10:04:50.145 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-10T10:04:50.145 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-10T10:04:50.145 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-10T10:04:50.145 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.146 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.146 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.146 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.146 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.146 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-10T10:04:50.146 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-10T10:04:50.146 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.146 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.146 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.146 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-10T10:04:50.146 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.146 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.146 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.147 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.147 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.147 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.147 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.147 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.147 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-10T10:04:50.147 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-10T10:04:50.147 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.147 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.147 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.147 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-10T10:04:50.147 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.147 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.147 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-10T10:04:50.148 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-10T10:04:50.148 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-10T10:04:50.148 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-10T10:04:50.148 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-10T10:04:50.148 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-10T10:04:50.148 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-10T10:04:50.148 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-10T10:04:50.148 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.148 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.148 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.148 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.148 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.148 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-10T10:04:50.148 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-10T10:04:50.149 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.149 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.149 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.149 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-10T10:04:50.149 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.149 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.149 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.149 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.149 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.149 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.149 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.149 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.150 INFO:teuthology.orchestra.run.smithi005.stdout: "roption1": { 2024-09-10T10:04:50.150 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "roption1", 2024-09-10T10:04:50.150 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.150 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.150 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.150 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-10T10:04:50.150 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.150 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.150 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.150 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.150 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.150 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.150 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.150 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.151 INFO:teuthology.orchestra.run.smithi005.stdout: "roption2": { 2024-09-10T10:04:50.151 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "roption2", 2024-09-10T10:04:50.151 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.151 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.151 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.151 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "xyz", 2024-09-10T10:04:50.151 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.151 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.151 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.151 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.151 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.151 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.151 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.152 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.152 INFO:teuthology.orchestra.run.smithi005.stdout: "rwoption1": { 2024-09-10T10:04:50.152 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rwoption1", 2024-09-10T10:04:50.152 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.152 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.152 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.152 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-10T10:04:50.152 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.152 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.152 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.152 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.152 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.152 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.152 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.152 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.153 INFO:teuthology.orchestra.run.smithi005.stdout: "rwoption2": { 2024-09-10T10:04:50.153 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rwoption2", 2024-09-10T10:04:50.153 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-10T10:04:50.153 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.153 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.153 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-10T10:04:50.153 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.153 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.153 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.153 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.153 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.153 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.153 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.154 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.154 INFO:teuthology.orchestra.run.smithi005.stdout: "rwoption3": { 2024-09-10T10:04:50.154 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rwoption3", 2024-09-10T10:04:50.154 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "float", 2024-09-10T10:04:50.154 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.154 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.154 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-10T10:04:50.154 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.154 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.154 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.154 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.154 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.154 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.154 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.154 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.155 INFO:teuthology.orchestra.run.smithi005.stdout: "rwoption4": { 2024-09-10T10:04:50.155 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rwoption4", 2024-09-10T10:04:50.155 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.155 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.155 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.155 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-10T10:04:50.155 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.155 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.155 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.155 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.155 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.155 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.155 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.155 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.155 INFO:teuthology.orchestra.run.smithi005.stdout: "rwoption5": { 2024-09-10T10:04:50.156 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rwoption5", 2024-09-10T10:04:50.156 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.156 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.156 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.156 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-10T10:04:50.156 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.156 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.156 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.156 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.156 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.156 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.156 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.156 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.156 INFO:teuthology.orchestra.run.smithi005.stdout: "rwoption6": { 2024-09-10T10:04:50.157 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rwoption6", 2024-09-10T10:04:50.157 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.157 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.157 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.157 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-10T10:04:50.157 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.157 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.157 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.157 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.157 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.157 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.157 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.157 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.158 INFO:teuthology.orchestra.run.smithi005.stdout: "rwoption7": { 2024-09-10T10:04:50.158 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rwoption7", 2024-09-10T10:04:50.158 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-10T10:04:50.158 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.158 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.158 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-10T10:04:50.158 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "1", 2024-09-10T10:04:50.158 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "42", 2024-09-10T10:04:50.158 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.158 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.158 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.158 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.158 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.158 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.158 INFO:teuthology.orchestra.run.smithi005.stdout: "testkey": { 2024-09-10T10:04:50.158 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "testkey", 2024-09-10T10:04:50.159 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.159 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.159 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.159 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-10T10:04:50.159 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.159 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.159 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.159 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.159 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.159 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.159 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.159 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.159 INFO:teuthology.orchestra.run.smithi005.stdout: "testlkey": { 2024-09-10T10:04:50.160 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "testlkey", 2024-09-10T10:04:50.160 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.160 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.160 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.160 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-10T10:04:50.160 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.160 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.160 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.160 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.160 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.160 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.160 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.160 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.160 INFO:teuthology.orchestra.run.smithi005.stdout: "testnewline": { 2024-09-10T10:04:50.161 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "testnewline", 2024-09-10T10:04:50.161 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.161 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.161 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.161 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-10T10:04:50.161 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.161 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.161 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.161 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.161 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.161 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.161 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.161 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T10:04:50.161 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T10:04:50.162 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.162 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-10T10:04:50.162 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "snap_schedule", 2024-09-10T10:04:50.162 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-10T10:04:50.162 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-10T10:04:50.162 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-10T10:04:50.162 INFO:teuthology.orchestra.run.smithi005.stdout: "allow_m_granularity": { 2024-09-10T10:04:50.162 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "allow_m_granularity", 2024-09-10T10:04:50.162 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.162 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.162 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.162 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-10T10:04:50.162 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.162 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.162 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.163 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "allow minute scheduled snapshots", 2024-09-10T10:04:50.163 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.163 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.163 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.163 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.163 INFO:teuthology.orchestra.run.smithi005.stdout: "dump_on_update": { 2024-09-10T10:04:50.163 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "dump_on_update", 2024-09-10T10:04:50.163 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.163 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.163 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.163 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-10T10:04:50.163 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.163 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.163 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.164 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "dump database to debug log on update", 2024-09-10T10:04:50.164 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.164 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.164 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.164 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.164 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-10T10:04:50.164 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-10T10:04:50.164 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.164 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.164 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.164 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-10T10:04:50.164 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.164 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.164 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-10T10:04:50.165 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-10T10:04:50.165 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-10T10:04:50.165 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-10T10:04:50.165 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-10T10:04:50.165 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-10T10:04:50.165 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-10T10:04:50.165 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-10T10:04:50.165 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.165 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.165 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.165 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.165 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.165 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-10T10:04:50.165 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-10T10:04:50.165 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.166 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.166 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.166 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-10T10:04:50.166 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.166 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.166 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.166 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.166 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.166 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.166 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.166 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.166 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-10T10:04:50.167 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-10T10:04:50.167 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.167 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.167 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.167 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-10T10:04:50.167 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.167 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.167 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-10T10:04:50.167 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-10T10:04:50.167 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-10T10:04:50.167 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-10T10:04:50.167 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-10T10:04:50.167 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-10T10:04:50.168 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-10T10:04:50.168 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-10T10:04:50.168 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.168 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.168 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.168 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.168 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.168 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-10T10:04:50.168 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-10T10:04:50.168 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.168 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.168 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.168 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-10T10:04:50.168 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.169 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.169 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.169 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.169 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.169 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.169 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.169 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T10:04:50.169 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T10:04:50.169 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.169 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-10T10:04:50.169 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "stats", 2024-09-10T10:04:50.169 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-10T10:04:50.169 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-10T10:04:50.169 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-10T10:04:50.170 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-10T10:04:50.170 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-10T10:04:50.170 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.170 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.170 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.170 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-10T10:04:50.170 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.170 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.170 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-10T10:04:50.170 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-10T10:04:50.170 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-10T10:04:50.170 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-10T10:04:50.170 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-10T10:04:50.171 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-10T10:04:50.171 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-10T10:04:50.171 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-10T10:04:50.171 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.171 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.171 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.171 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.171 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.171 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-10T10:04:50.171 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-10T10:04:50.171 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.171 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.171 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.171 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-10T10:04:50.171 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.172 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.172 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.172 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.172 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.172 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.172 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.172 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.172 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-10T10:04:50.172 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-10T10:04:50.172 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.172 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.172 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.172 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-10T10:04:50.172 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.173 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.173 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-10T10:04:50.173 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-10T10:04:50.173 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-10T10:04:50.173 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-10T10:04:50.173 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-10T10:04:50.173 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-10T10:04:50.173 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-10T10:04:50.173 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-10T10:04:50.173 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.173 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.173 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.173 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.173 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.174 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-10T10:04:50.174 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-10T10:04:50.174 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.174 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.174 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.174 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-10T10:04:50.174 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.174 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.174 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.174 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.174 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.174 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.174 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.174 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T10:04:50.175 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T10:04:50.175 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.175 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-10T10:04:50.175 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "status", 2024-09-10T10:04:50.175 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-10T10:04:50.175 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-10T10:04:50.175 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-10T10:04:50.175 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-10T10:04:50.175 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-10T10:04:50.175 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.175 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.175 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.175 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-10T10:04:50.176 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.176 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.176 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-10T10:04:50.176 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-10T10:04:50.176 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-10T10:04:50.176 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-10T10:04:50.176 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-10T10:04:50.176 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-10T10:04:50.176 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-10T10:04:50.176 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-10T10:04:50.176 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.176 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.176 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.177 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.177 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.177 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-10T10:04:50.177 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-10T10:04:50.177 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.177 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.177 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.177 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-10T10:04:50.177 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.177 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.177 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.177 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.178 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.178 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.178 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.178 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.178 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-10T10:04:50.178 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-10T10:04:50.178 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.178 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.178 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.178 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-10T10:04:50.178 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.178 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.178 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-10T10:04:50.179 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-10T10:04:50.179 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-10T10:04:50.179 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-10T10:04:50.179 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-10T10:04:50.179 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-10T10:04:50.179 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-10T10:04:50.179 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-10T10:04:50.179 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.179 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.179 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.179 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.179 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.179 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-10T10:04:50.180 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-10T10:04:50.180 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.180 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.180 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.180 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-10T10:04:50.180 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.180 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.180 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.180 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.180 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.180 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.180 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.180 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T10:04:50.180 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T10:04:50.181 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.181 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-10T10:04:50.181 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "telegraf", 2024-09-10T10:04:50.181 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-10T10:04:50.181 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-10T10:04:50.181 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-10T10:04:50.181 INFO:teuthology.orchestra.run.smithi005.stdout: "address": { 2024-09-10T10:04:50.181 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "address", 2024-09-10T10:04:50.181 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.181 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.181 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.181 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-10T10:04:50.181 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.181 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.181 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.182 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.182 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.182 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.182 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.182 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.182 INFO:teuthology.orchestra.run.smithi005.stdout: "interval": { 2024-09-10T10:04:50.182 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "interval", 2024-09-10T10:04:50.182 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-10T10:04:50.182 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.182 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.182 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "15", 2024-09-10T10:04:50.182 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.183 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.183 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.183 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.183 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.183 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.183 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.183 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.183 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-10T10:04:50.183 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-10T10:04:50.183 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.183 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.183 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.183 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-10T10:04:50.183 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.184 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.184 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-10T10:04:50.184 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-10T10:04:50.184 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-10T10:04:50.184 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-10T10:04:50.184 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-10T10:04:50.184 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-10T10:04:50.184 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-10T10:04:50.184 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-10T10:04:50.184 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.184 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.184 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.184 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.185 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.185 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-10T10:04:50.185 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-10T10:04:50.185 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.185 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.185 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.185 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-10T10:04:50.185 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.185 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.185 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.185 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.185 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.185 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.186 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.186 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.186 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-10T10:04:50.186 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-10T10:04:50.186 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.186 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.186 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.186 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-10T10:04:50.186 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.186 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.186 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-10T10:04:50.186 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-10T10:04:50.186 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-10T10:04:50.187 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-10T10:04:50.187 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-10T10:04:50.187 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-10T10:04:50.187 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-10T10:04:50.187 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-10T10:04:50.187 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.187 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.187 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.187 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.187 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.187 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-10T10:04:50.187 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-10T10:04:50.187 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.188 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.188 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.188 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-10T10:04:50.188 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.188 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.188 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.188 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.188 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.188 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.188 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.188 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T10:04:50.188 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T10:04:50.188 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.189 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-10T10:04:50.189 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "telemetry", 2024-09-10T10:04:50.189 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-10T10:04:50.189 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-10T10:04:50.189 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-10T10:04:50.189 INFO:teuthology.orchestra.run.smithi005.stdout: "channel_basic": { 2024-09-10T10:04:50.189 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "channel_basic", 2024-09-10T10:04:50.189 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.189 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.189 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.189 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-10T10:04:50.189 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.190 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.190 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.190 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-10T10:04:50.190 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.190 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.190 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.190 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.190 INFO:teuthology.orchestra.run.smithi005.stdout: "channel_crash": { 2024-09-10T10:04:50.190 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "channel_crash", 2024-09-10T10:04:50.190 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.190 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.190 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.190 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-10T10:04:50.191 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.191 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.191 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.191 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-10T10:04:50.191 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.191 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.191 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.191 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.191 INFO:teuthology.orchestra.run.smithi005.stdout: "channel_device": { 2024-09-10T10:04:50.191 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "channel_device", 2024-09-10T10:04:50.191 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.191 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.191 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.192 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-10T10:04:50.192 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.192 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.192 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.192 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-10T10:04:50.192 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.192 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.192 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.192 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.192 INFO:teuthology.orchestra.run.smithi005.stdout: "channel_ident": { 2024-09-10T10:04:50.192 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "channel_ident", 2024-09-10T10:04:50.192 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.192 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.193 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.193 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-10T10:04:50.193 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.193 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.193 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.193 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-10T10:04:50.193 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.193 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.193 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.193 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.193 INFO:teuthology.orchestra.run.smithi005.stdout: "channel_perf": { 2024-09-10T10:04:50.193 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "channel_perf", 2024-09-10T10:04:50.193 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.193 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.194 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.194 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-10T10:04:50.194 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.194 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.194 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.194 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-10T10:04:50.194 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.194 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.194 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.194 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.194 INFO:teuthology.orchestra.run.smithi005.stdout: "contact": { 2024-09-10T10:04:50.194 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "contact", 2024-09-10T10:04:50.194 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.194 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.194 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.195 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-10T10:04:50.195 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.195 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.195 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.195 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.195 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.195 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.195 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.195 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.195 INFO:teuthology.orchestra.run.smithi005.stdout: "description": { 2024-09-10T10:04:50.195 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "description", 2024-09-10T10:04:50.195 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.195 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.195 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.196 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-10T10:04:50.196 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.196 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.196 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.196 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.196 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.196 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.196 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.196 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.196 INFO:teuthology.orchestra.run.smithi005.stdout: "device_url": { 2024-09-10T10:04:50.196 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "device_url", 2024-09-10T10:04:50.196 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.196 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.197 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.197 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-10T10:04:50.197 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.197 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.197 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.197 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.197 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.197 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.197 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.197 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.197 INFO:teuthology.orchestra.run.smithi005.stdout: "enabled": { 2024-09-10T10:04:50.197 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "enabled", 2024-09-10T10:04:50.197 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.198 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.198 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.198 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-10T10:04:50.198 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.198 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.198 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.198 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.198 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.198 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.198 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.198 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.198 INFO:teuthology.orchestra.run.smithi005.stdout: "interval": { 2024-09-10T10:04:50.198 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "interval", 2024-09-10T10:04:50.199 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-10T10:04:50.199 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.199 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.199 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "24", 2024-09-10T10:04:50.199 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "8", 2024-09-10T10:04:50.199 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.199 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.199 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.199 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.199 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.199 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.199 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.199 INFO:teuthology.orchestra.run.smithi005.stdout: "last_opt_revision": { 2024-09-10T10:04:50.199 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "last_opt_revision", 2024-09-10T10:04:50.200 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-10T10:04:50.200 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.200 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.200 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "1", 2024-09-10T10:04:50.200 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.200 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.200 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.200 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.200 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.200 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.200 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.200 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.200 INFO:teuthology.orchestra.run.smithi005.stdout: "leaderboard": { 2024-09-10T10:04:50.200 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "leaderboard", 2024-09-10T10:04:50.201 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.201 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.201 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.201 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-10T10:04:50.201 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.201 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.201 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.201 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.201 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.201 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.201 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.201 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.201 INFO:teuthology.orchestra.run.smithi005.stdout: "leaderboard_description": { 2024-09-10T10:04:50.201 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "leaderboard_description", 2024-09-10T10:04:50.202 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.202 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.202 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.202 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-10T10:04:50.202 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.202 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.202 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.202 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.202 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.202 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.202 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.202 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.202 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-10T10:04:50.202 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-10T10:04:50.203 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.203 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.203 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.203 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-10T10:04:50.203 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.203 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.203 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-10T10:04:50.203 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-10T10:04:50.203 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-10T10:04:50.203 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-10T10:04:50.203 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-10T10:04:50.203 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-10T10:04:50.203 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-10T10:04:50.203 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-10T10:04:50.203 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.204 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.204 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.204 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.204 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.204 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-10T10:04:50.204 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-10T10:04:50.204 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.204 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.204 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.204 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-10T10:04:50.204 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.204 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.204 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.204 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.205 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.205 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.205 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.205 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.205 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-10T10:04:50.205 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-10T10:04:50.205 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.205 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.205 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.205 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-10T10:04:50.205 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.205 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.205 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-10T10:04:50.206 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-10T10:04:50.206 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-10T10:04:50.206 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-10T10:04:50.206 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-10T10:04:50.206 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-10T10:04:50.206 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-10T10:04:50.206 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-10T10:04:50.206 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.206 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.206 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.206 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.206 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.206 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-10T10:04:50.206 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-10T10:04:50.207 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.207 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.207 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.207 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-10T10:04:50.207 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.207 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.207 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.207 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.207 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.207 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.207 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.207 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.207 INFO:teuthology.orchestra.run.smithi005.stdout: "organization": { 2024-09-10T10:04:50.208 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "organization", 2024-09-10T10:04:50.208 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.208 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.208 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.208 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-10T10:04:50.208 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.208 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.208 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.208 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.208 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.208 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.208 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.208 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.208 INFO:teuthology.orchestra.run.smithi005.stdout: "proxy": { 2024-09-10T10:04:50.209 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "proxy", 2024-09-10T10:04:50.209 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.209 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.209 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.209 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-10T10:04:50.209 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.209 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.209 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.209 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.209 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.209 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.209 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.209 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.209 INFO:teuthology.orchestra.run.smithi005.stdout: "url": { 2024-09-10T10:04:50.210 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "url", 2024-09-10T10:04:50.210 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.210 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.210 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.210 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-10T10:04:50.210 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.210 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.210 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.210 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.210 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.210 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.210 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.210 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T10:04:50.210 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T10:04:50.211 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.211 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-10T10:04:50.211 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "test_orchestrator", 2024-09-10T10:04:50.211 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-10T10:04:50.211 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-10T10:04:50.211 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-10T10:04:50.211 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-10T10:04:50.211 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-10T10:04:50.211 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.211 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.211 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.211 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-10T10:04:50.211 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.211 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.211 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-10T10:04:50.212 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-10T10:04:50.212 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-10T10:04:50.212 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-10T10:04:50.212 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-10T10:04:50.212 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-10T10:04:50.212 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-10T10:04:50.212 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-10T10:04:50.212 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.212 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.212 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.212 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.212 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.212 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-10T10:04:50.212 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-10T10:04:50.213 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.213 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.213 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.213 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-10T10:04:50.213 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.213 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.213 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.213 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.213 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.213 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.213 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.213 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.213 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-10T10:04:50.213 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-10T10:04:50.214 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.214 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.214 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.214 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-10T10:04:50.214 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.214 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.214 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-10T10:04:50.214 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-10T10:04:50.214 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-10T10:04:50.214 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-10T10:04:50.214 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-10T10:04:50.214 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-10T10:04:50.214 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-10T10:04:50.214 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-10T10:04:50.215 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.215 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.215 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.215 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.215 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.215 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-10T10:04:50.215 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-10T10:04:50.215 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.215 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.215 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.215 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-10T10:04:50.215 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.216 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.216 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.216 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.216 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.216 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.216 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.216 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T10:04:50.216 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T10:04:50.216 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.216 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-10T10:04:50.216 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "volumes", 2024-09-10T10:04:50.216 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-10T10:04:50.216 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-10T10:04:50.216 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-10T10:04:50.217 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-10T10:04:50.217 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-10T10:04:50.217 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.217 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.217 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.217 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-10T10:04:50.217 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.217 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.217 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-10T10:04:50.217 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-10T10:04:50.217 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-10T10:04:50.217 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-10T10:04:50.217 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-10T10:04:50.218 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-10T10:04:50.218 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-10T10:04:50.218 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-10T10:04:50.218 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.218 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.218 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.218 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.218 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.218 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-10T10:04:50.218 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-10T10:04:50.218 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.218 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.218 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.218 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-10T10:04:50.219 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.219 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.219 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.219 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.219 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.219 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.219 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.219 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.219 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-10T10:04:50.219 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-10T10:04:50.219 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.219 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.219 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.219 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-10T10:04:50.220 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.220 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.220 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-10T10:04:50.220 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-10T10:04:50.220 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-10T10:04:50.220 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-10T10:04:50.220 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-10T10:04:50.220 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-10T10:04:50.220 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-10T10:04:50.220 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-10T10:04:50.220 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.220 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.220 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.221 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.221 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.221 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-10T10:04:50.221 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-10T10:04:50.221 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.221 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.221 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.221 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-10T10:04:50.221 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.221 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.221 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.221 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.221 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.221 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.221 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.222 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.222 INFO:teuthology.orchestra.run.smithi005.stdout: "max_concurrent_clones": { 2024-09-10T10:04:50.222 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "max_concurrent_clones", 2024-09-10T10:04:50.222 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-10T10:04:50.222 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.222 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.222 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "4", 2024-09-10T10:04:50.222 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.222 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.222 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.222 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-10T10:04:50.222 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.222 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.222 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.222 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.223 INFO:teuthology.orchestra.run.smithi005.stdout: "periodic_async_work": { 2024-09-10T10:04:50.223 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "periodic_async_work", 2024-09-10T10:04:50.223 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.223 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.223 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.223 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-10T10:04:50.223 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.223 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.223 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.223 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Periodically check for async work", 2024-09-10T10:04:50.223 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.223 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.223 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.224 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.224 INFO:teuthology.orchestra.run.smithi005.stdout: "snapshot_clone_delay": { 2024-09-10T10:04:50.224 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "snapshot_clone_delay", 2024-09-10T10:04:50.224 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-10T10:04:50.224 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.224 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.224 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-09-10T10:04:50.224 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.224 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.224 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.224 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-10T10:04:50.224 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.224 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.224 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.225 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.225 INFO:teuthology.orchestra.run.smithi005.stdout: "snapshot_clone_no_wait": { 2024-09-10T10:04:50.225 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "snapshot_clone_no_wait", 2024-09-10T10:04:50.225 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.225 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.225 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.225 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-10T10:04:50.225 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.225 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.225 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.225 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-10T10:04:50.225 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.225 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.226 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.226 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T10:04:50.226 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T10:04:50.226 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.226 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-10T10:04:50.226 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "zabbix", 2024-09-10T10:04:50.226 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-10T10:04:50.226 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-10T10:04:50.226 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-10T10:04:50.226 INFO:teuthology.orchestra.run.smithi005.stdout: "discovery_interval": { 2024-09-10T10:04:50.226 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "discovery_interval", 2024-09-10T10:04:50.226 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "uint", 2024-09-10T10:04:50.226 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.226 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.227 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "100", 2024-09-10T10:04:50.227 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.227 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.227 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.227 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.227 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.227 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.227 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.227 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.227 INFO:teuthology.orchestra.run.smithi005.stdout: "identifier": { 2024-09-10T10:04:50.227 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "identifier", 2024-09-10T10:04:50.227 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.227 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.228 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.228 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-10T10:04:50.228 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.228 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.228 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.228 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.228 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.228 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.228 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.228 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.228 INFO:teuthology.orchestra.run.smithi005.stdout: "interval": { 2024-09-10T10:04:50.228 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "interval", 2024-09-10T10:04:50.228 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-10T10:04:50.228 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.228 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.229 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "60", 2024-09-10T10:04:50.229 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.229 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.229 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.229 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.229 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.229 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.229 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.229 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.229 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-10T10:04:50.229 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-10T10:04:50.229 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.229 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.229 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.229 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-10T10:04:50.230 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.230 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.230 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-10T10:04:50.230 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-10T10:04:50.230 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-10T10:04:50.230 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-10T10:04:50.230 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-10T10:04:50.230 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-10T10:04:50.230 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-10T10:04:50.230 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-10T10:04:50.230 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.230 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.230 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.230 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.230 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.231 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-10T10:04:50.231 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-10T10:04:50.231 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.231 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.231 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.231 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-10T10:04:50.231 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.231 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.231 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.231 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.231 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.231 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.231 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.231 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.231 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-10T10:04:50.232 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-10T10:04:50.232 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.232 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.232 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.232 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-10T10:04:50.232 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.232 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.232 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-10T10:04:50.232 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-10T10:04:50.232 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-10T10:04:50.232 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-10T10:04:50.232 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-10T10:04:50.232 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-10T10:04:50.232 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-10T10:04:50.232 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-10T10:04:50.233 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.233 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.233 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.233 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.233 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.233 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-10T10:04:50.233 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-10T10:04:50.233 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.233 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.233 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.233 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-10T10:04:50.233 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.233 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.233 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.234 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.234 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.234 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.234 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.234 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.234 INFO:teuthology.orchestra.run.smithi005.stdout: "zabbix_host": { 2024-09-10T10:04:50.234 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "zabbix_host", 2024-09-10T10:04:50.234 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.234 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.234 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.234 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-10T10:04:50.234 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.235 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.235 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.235 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.235 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.235 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.235 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.235 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.235 INFO:teuthology.orchestra.run.smithi005.stdout: "zabbix_port": { 2024-09-10T10:04:50.235 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "zabbix_port", 2024-09-10T10:04:50.235 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-10T10:04:50.235 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.235 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.235 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10051", 2024-09-10T10:04:50.235 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.235 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.236 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.236 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.236 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.236 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.236 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.236 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.236 INFO:teuthology.orchestra.run.smithi005.stdout: "zabbix_sender": { 2024-09-10T10:04:50.236 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "zabbix_sender", 2024-09-10T10:04:50.236 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.236 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.236 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.236 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-10T10:04:50.236 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.236 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.237 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.237 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.237 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.237 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.237 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.237 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T10:04:50.237 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T10:04:50.237 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T10:04:50.237 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-10T10:04:50.237 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.237 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-10T10:04:50.238 INFO:teuthology.orchestra.run.smithi005.stdout: "gid": 24110, 2024-09-10T10:04:50.238 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "y", 2024-09-10T10:04:50.238 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr_features": 4540701547738038271, 2024-09-10T10:04:50.238 INFO:teuthology.orchestra.run.smithi005.stdout: "available_modules": [ 2024-09-10T10:04:50.238 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-10T10:04:50.238 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "alerts", 2024-09-10T10:04:50.238 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-10T10:04:50.238 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-10T10:04:50.238 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-10T10:04:50.238 INFO:teuthology.orchestra.run.smithi005.stdout: "interval": { 2024-09-10T10:04:50.238 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "interval", 2024-09-10T10:04:50.238 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-10T10:04:50.238 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.238 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.239 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "60", 2024-09-10T10:04:50.239 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.239 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.239 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.239 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "How frequently to reexamine health status", 2024-09-10T10:04:50.239 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.239 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.239 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.239 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.239 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-10T10:04:50.239 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-10T10:04:50.239 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.239 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.239 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.240 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-10T10:04:50.240 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.240 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.240 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-10T10:04:50.240 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-10T10:04:50.240 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-10T10:04:50.240 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-10T10:04:50.240 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-10T10:04:50.240 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-10T10:04:50.240 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-10T10:04:50.240 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-10T10:04:50.240 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.241 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.241 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.241 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.241 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.241 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-10T10:04:50.241 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-10T10:04:50.241 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.241 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.241 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.241 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-10T10:04:50.241 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.241 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.241 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.241 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.242 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.242 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.242 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.242 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.242 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-10T10:04:50.242 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-10T10:04:50.242 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.242 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.242 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.242 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-10T10:04:50.242 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.242 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.242 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-10T10:04:50.242 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-10T10:04:50.242 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-10T10:04:50.243 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-10T10:04:50.243 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-10T10:04:50.243 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-10T10:04:50.243 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-10T10:04:50.243 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-10T10:04:50.243 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.243 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.243 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.243 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.243 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.243 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-10T10:04:50.243 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-10T10:04:50.243 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.244 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.244 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.244 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-10T10:04:50.244 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.244 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.244 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.244 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.244 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.244 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.244 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.244 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.244 INFO:teuthology.orchestra.run.smithi005.stdout: "smtp_destination": { 2024-09-10T10:04:50.244 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "smtp_destination", 2024-09-10T10:04:50.244 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.244 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.245 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.245 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-10T10:04:50.245 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.245 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.245 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.245 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Email address to send alerts to", 2024-09-10T10:04:50.245 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.245 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.245 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.245 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.245 INFO:teuthology.orchestra.run.smithi005.stdout: "smtp_from_name": { 2024-09-10T10:04:50.245 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "smtp_from_name", 2024-09-10T10:04:50.245 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.245 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.246 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.246 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "Ceph", 2024-09-10T10:04:50.246 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.246 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.246 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.246 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Email From: name", 2024-09-10T10:04:50.246 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.246 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.246 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.246 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.246 INFO:teuthology.orchestra.run.smithi005.stdout: "smtp_host": { 2024-09-10T10:04:50.246 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "smtp_host", 2024-09-10T10:04:50.246 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.246 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.247 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.247 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-10T10:04:50.247 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.247 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.247 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.247 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "SMTP server", 2024-09-10T10:04:50.247 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.247 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.247 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.247 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.247 INFO:teuthology.orchestra.run.smithi005.stdout: "smtp_password": { 2024-09-10T10:04:50.247 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "smtp_password", 2024-09-10T10:04:50.247 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.247 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.248 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.248 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-10T10:04:50.248 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.248 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.248 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.248 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Password to authenticate with", 2024-09-10T10:04:50.248 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.248 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.248 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.248 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.248 INFO:teuthology.orchestra.run.smithi005.stdout: "smtp_port": { 2024-09-10T10:04:50.248 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "smtp_port", 2024-09-10T10:04:50.248 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-10T10:04:50.248 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.248 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.249 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "465", 2024-09-10T10:04:50.249 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.249 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.249 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.249 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "SMTP port", 2024-09-10T10:04:50.249 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.249 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.249 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.249 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.249 INFO:teuthology.orchestra.run.smithi005.stdout: "smtp_sender": { 2024-09-10T10:04:50.249 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "smtp_sender", 2024-09-10T10:04:50.249 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.249 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.249 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.250 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-10T10:04:50.250 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.250 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.250 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.250 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "SMTP envelope sender", 2024-09-10T10:04:50.250 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.250 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.250 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.250 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.250 INFO:teuthology.orchestra.run.smithi005.stdout: "smtp_ssl": { 2024-09-10T10:04:50.250 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "smtp_ssl", 2024-09-10T10:04:50.250 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.250 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.251 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.251 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-10T10:04:50.251 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.251 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.251 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.251 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-10T10:04:50.251 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.251 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.251 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.251 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.251 INFO:teuthology.orchestra.run.smithi005.stdout: "smtp_user": { 2024-09-10T10:04:50.251 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "smtp_user", 2024-09-10T10:04:50.251 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.252 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.252 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.252 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-10T10:04:50.252 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.252 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.252 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.252 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "User to authenticate as", 2024-09-10T10:04:50.252 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.252 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.252 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.252 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T10:04:50.252 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T10:04:50.252 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.253 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-10T10:04:50.253 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "balancer", 2024-09-10T10:04:50.253 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-10T10:04:50.253 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-10T10:04:50.253 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-10T10:04:50.253 INFO:teuthology.orchestra.run.smithi005.stdout: "active": { 2024-09-10T10:04:50.253 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "active", 2024-09-10T10:04:50.253 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.253 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.253 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.253 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-10T10:04:50.253 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.253 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.254 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.254 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "automatically balance PGs across cluster", 2024-09-10T10:04:50.254 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.254 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.254 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.254 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.254 INFO:teuthology.orchestra.run.smithi005.stdout: "begin_time": { 2024-09-10T10:04:50.254 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "begin_time", 2024-09-10T10:04:50.254 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.254 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.254 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.254 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0000", 2024-09-10T10:04:50.254 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.254 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.254 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.255 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "beginning time of day to automatically balance", 2024-09-10T10:04:50.255 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-10T10:04:50.255 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.255 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.255 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.255 INFO:teuthology.orchestra.run.smithi005.stdout: "begin_weekday": { 2024-09-10T10:04:50.255 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "begin_weekday", 2024-09-10T10:04:50.255 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "uint", 2024-09-10T10:04:50.255 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.255 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.255 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-09-10T10:04:50.255 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "0", 2024-09-10T10:04:50.255 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "6", 2024-09-10T10:04:50.255 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.255 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-10T10:04:50.256 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-10T10:04:50.256 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.256 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.256 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.256 INFO:teuthology.orchestra.run.smithi005.stdout: "crush_compat_max_iterations": { 2024-09-10T10:04:50.256 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "crush_compat_max_iterations", 2024-09-10T10:04:50.256 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "uint", 2024-09-10T10:04:50.256 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.256 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.256 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "25", 2024-09-10T10:04:50.256 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "1", 2024-09-10T10:04:50.256 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "250", 2024-09-10T10:04:50.256 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.256 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-10T10:04:50.257 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.257 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.257 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.257 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.257 INFO:teuthology.orchestra.run.smithi005.stdout: "crush_compat_metrics": { 2024-09-10T10:04:50.257 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "crush_compat_metrics", 2024-09-10T10:04:50.257 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.257 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.257 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.257 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "pgs,objects,bytes", 2024-09-10T10:04:50.257 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.257 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.257 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.258 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-10T10:04:50.258 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-10T10:04:50.258 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.258 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.258 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.258 INFO:teuthology.orchestra.run.smithi005.stdout: "crush_compat_step": { 2024-09-10T10:04:50.258 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "crush_compat_step", 2024-09-10T10:04:50.258 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "float", 2024-09-10T10:04:50.258 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.258 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.258 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0.5", 2024-09-10T10:04:50.258 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "0.001", 2024-09-10T10:04:50.258 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "0.999", 2024-09-10T10:04:50.259 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.259 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "aggressiveness of optimization", 2024-09-10T10:04:50.259 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-10T10:04:50.259 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.259 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.259 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.259 INFO:teuthology.orchestra.run.smithi005.stdout: "end_time": { 2024-09-10T10:04:50.259 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "end_time", 2024-09-10T10:04:50.259 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.259 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.259 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.259 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "2359", 2024-09-10T10:04:50.259 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.259 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.259 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.260 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "ending time of day to automatically balance", 2024-09-10T10:04:50.260 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-10T10:04:50.260 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.260 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.260 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.260 INFO:teuthology.orchestra.run.smithi005.stdout: "end_weekday": { 2024-09-10T10:04:50.260 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "end_weekday", 2024-09-10T10:04:50.260 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "uint", 2024-09-10T10:04:50.260 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.260 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.260 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-09-10T10:04:50.260 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "0", 2024-09-10T10:04:50.260 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "6", 2024-09-10T10:04:50.260 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.260 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-10T10:04:50.261 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-10T10:04:50.261 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.261 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.261 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.261 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-10T10:04:50.261 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-10T10:04:50.261 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.261 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.261 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.261 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-10T10:04:50.261 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.261 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.261 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-10T10:04:50.261 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-10T10:04:50.262 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-10T10:04:50.262 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-10T10:04:50.262 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-10T10:04:50.262 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-10T10:04:50.262 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-10T10:04:50.262 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-10T10:04:50.262 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.262 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.262 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.262 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.262 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.262 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-10T10:04:50.262 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-10T10:04:50.262 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.262 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.263 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.263 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-10T10:04:50.263 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.263 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.263 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.263 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.263 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.263 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.263 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.263 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.263 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-10T10:04:50.263 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-10T10:04:50.263 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.263 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.263 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.264 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-10T10:04:50.264 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.264 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.264 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-10T10:04:50.264 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-10T10:04:50.264 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-10T10:04:50.264 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-10T10:04:50.264 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-10T10:04:50.264 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-10T10:04:50.264 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-10T10:04:50.264 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-10T10:04:50.264 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.264 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.264 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.265 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.265 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.265 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-10T10:04:50.265 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-10T10:04:50.265 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.265 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.265 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.265 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-10T10:04:50.265 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.265 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.265 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.265 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.265 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.265 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.266 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.266 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.266 INFO:teuthology.orchestra.run.smithi005.stdout: "min_score": { 2024-09-10T10:04:50.266 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "min_score", 2024-09-10T10:04:50.266 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "float", 2024-09-10T10:04:50.266 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.266 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.266 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-09-10T10:04:50.266 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.266 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.266 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.266 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-10T10:04:50.267 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.267 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.267 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.267 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.267 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": { 2024-09-10T10:04:50.267 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "mode", 2024-09-10T10:04:50.267 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.267 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.267 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.267 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "upmap", 2024-09-10T10:04:50.267 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.267 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.267 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-10T10:04:50.268 INFO:teuthology.orchestra.run.smithi005.stdout: "crush-compat", 2024-09-10T10:04:50.268 INFO:teuthology.orchestra.run.smithi005.stdout: "none", 2024-09-10T10:04:50.268 INFO:teuthology.orchestra.run.smithi005.stdout: "read", 2024-09-10T10:04:50.268 INFO:teuthology.orchestra.run.smithi005.stdout: "upmap", 2024-09-10T10:04:50.268 INFO:teuthology.orchestra.run.smithi005.stdout: "upmap-read" 2024-09-10T10:04:50.268 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-10T10:04:50.268 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Balancer mode", 2024-09-10T10:04:50.268 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.268 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.268 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.268 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.268 INFO:teuthology.orchestra.run.smithi005.stdout: "pool_ids": { 2024-09-10T10:04:50.268 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "pool_ids", 2024-09-10T10:04:50.268 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.269 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.269 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.269 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-10T10:04:50.269 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.269 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.269 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.269 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-10T10:04:50.269 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.269 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.269 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.269 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.269 INFO:teuthology.orchestra.run.smithi005.stdout: "sleep_interval": { 2024-09-10T10:04:50.269 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sleep_interval", 2024-09-10T10:04:50.269 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-10T10:04:50.270 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.270 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.270 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "60", 2024-09-10T10:04:50.270 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.270 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.270 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.270 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-10T10:04:50.270 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.270 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.270 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.270 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.270 INFO:teuthology.orchestra.run.smithi005.stdout: "upmap_max_deviation": { 2024-09-10T10:04:50.270 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "upmap_max_deviation", 2024-09-10T10:04:50.270 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-10T10:04:50.270 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.270 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.271 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "5", 2024-09-10T10:04:50.271 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "1", 2024-09-10T10:04:50.271 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.271 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.271 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-10T10:04:50.271 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-10T10:04:50.271 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.271 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.271 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.271 INFO:teuthology.orchestra.run.smithi005.stdout: "upmap_max_optimizations": { 2024-09-10T10:04:50.271 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "upmap_max_optimizations", 2024-09-10T10:04:50.271 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "uint", 2024-09-10T10:04:50.271 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.271 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.271 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10", 2024-09-10T10:04:50.272 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.272 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.272 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.272 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-10T10:04:50.272 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.272 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.272 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.272 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T10:04:50.272 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T10:04:50.272 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.272 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-10T10:04:50.272 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "cephadm", 2024-09-10T10:04:50.272 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-10T10:04:50.272 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-10T10:04:50.272 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-10T10:04:50.273 INFO:teuthology.orchestra.run.smithi005.stdout: "agent_down_multiplier": { 2024-09-10T10:04:50.273 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "agent_down_multiplier", 2024-09-10T10:04:50.273 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "float", 2024-09-10T10:04:50.273 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.273 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.273 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "3.0", 2024-09-10T10:04:50.273 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.273 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.273 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.273 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-10T10:04:50.273 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.273 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.273 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.273 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.273 INFO:teuthology.orchestra.run.smithi005.stdout: "agent_refresh_rate": { 2024-09-10T10:04:50.274 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "agent_refresh_rate", 2024-09-10T10:04:50.274 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-10T10:04:50.274 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.274 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.274 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "20", 2024-09-10T10:04:50.274 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.274 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.274 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.274 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-10T10:04:50.274 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.274 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.274 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.274 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.274 INFO:teuthology.orchestra.run.smithi005.stdout: "agent_starting_port": { 2024-09-10T10:04:50.274 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "agent_starting_port", 2024-09-10T10:04:50.275 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-10T10:04:50.275 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.275 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.275 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "4721", 2024-09-10T10:04:50.275 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.275 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.275 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.275 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-10T10:04:50.275 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.275 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.275 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.275 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.275 INFO:teuthology.orchestra.run.smithi005.stdout: "allow_ptrace": { 2024-09-10T10:04:50.275 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "allow_ptrace", 2024-09-10T10:04:50.275 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.276 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.276 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.276 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-10T10:04:50.276 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.276 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.276 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.276 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-10T10:04:50.276 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-10T10:04:50.276 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.276 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.276 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.276 INFO:teuthology.orchestra.run.smithi005.stdout: "autotune_interval": { 2024-09-10T10:04:50.276 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "autotune_interval", 2024-09-10T10:04:50.277 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-10T10:04:50.277 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.277 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.277 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "600", 2024-09-10T10:04:50.277 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.277 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.277 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.277 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-10T10:04:50.277 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.277 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.277 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.277 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.277 INFO:teuthology.orchestra.run.smithi005.stdout: "autotune_memory_target_ratio": { 2024-09-10T10:04:50.278 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "autotune_memory_target_ratio", 2024-09-10T10:04:50.278 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "float", 2024-09-10T10:04:50.278 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.278 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.278 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0.7", 2024-09-10T10:04:50.278 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.278 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.278 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.278 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-10T10:04:50.278 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.278 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.278 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.278 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.278 INFO:teuthology.orchestra.run.smithi005.stdout: "cephadm_log_destination": { 2024-09-10T10:04:50.278 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "cephadm_log_destination", 2024-09-10T10:04:50.279 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.279 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.279 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.279 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-10T10:04:50.279 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.279 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.279 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-10T10:04:50.279 INFO:teuthology.orchestra.run.smithi005.stdout: "file", 2024-09-10T10:04:50.279 INFO:teuthology.orchestra.run.smithi005.stdout: "file,syslog", 2024-09-10T10:04:50.279 INFO:teuthology.orchestra.run.smithi005.stdout: "syslog" 2024-09-10T10:04:50.279 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-10T10:04:50.279 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-10T10:04:50.279 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.279 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.279 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.280 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.280 INFO:teuthology.orchestra.run.smithi005.stdout: "cgroups_split": { 2024-09-10T10:04:50.280 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "cgroups_split", 2024-09-10T10:04:50.280 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.280 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.280 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.280 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-10T10:04:50.280 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.280 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.280 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.280 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-10T10:04:50.280 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.280 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.281 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.281 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.281 INFO:teuthology.orchestra.run.smithi005.stdout: "config_checks_enabled": { 2024-09-10T10:04:50.281 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "config_checks_enabled", 2024-09-10T10:04:50.281 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.281 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.281 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.281 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-10T10:04:50.281 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.281 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.281 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.281 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-10T10:04:50.281 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.281 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.281 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.282 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.282 INFO:teuthology.orchestra.run.smithi005.stdout: "config_dashboard": { 2024-09-10T10:04:50.282 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "config_dashboard", 2024-09-10T10:04:50.282 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.282 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.282 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.282 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-10T10:04:50.282 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.282 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.282 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.282 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-10T10:04:50.282 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.282 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.282 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.283 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.283 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_alertmanager": { 2024-09-10T10:04:50.283 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_alertmanager", 2024-09-10T10:04:50.283 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.283 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.283 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.283 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-09-10T10:04:50.283 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.283 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.283 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.283 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Prometheus container image", 2024-09-10T10:04:50.283 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.283 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.283 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.283 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.284 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_base": { 2024-09-10T10:04:50.284 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_base", 2024-09-10T10:04:50.284 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.284 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.284 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.284 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-10T10:04:50.284 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.284 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.284 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.284 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Container image name, without the tag", 2024-09-10T10:04:50.284 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.284 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.284 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.284 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.284 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_elasticsearch": { 2024-09-10T10:04:50.285 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_elasticsearch", 2024-09-10T10:04:50.285 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.285 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.285 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.285 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-10T10:04:50.285 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.285 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.285 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.285 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "elasticsearch container image", 2024-09-10T10:04:50.285 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.285 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.285 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.285 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.285 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_grafana": { 2024-09-10T10:04:50.286 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_grafana", 2024-09-10T10:04:50.286 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.286 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.286 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.286 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-09-10T10:04:50.286 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.286 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.286 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.286 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Prometheus container image", 2024-09-10T10:04:50.286 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.286 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.286 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.286 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.287 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_haproxy": { 2024-09-10T10:04:50.287 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_haproxy", 2024-09-10T10:04:50.287 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.287 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.287 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.287 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-10T10:04:50.287 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.287 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.287 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.287 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "HAproxy container image", 2024-09-10T10:04:50.287 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.287 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.287 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.287 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.288 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_jaeger_agent": { 2024-09-10T10:04:50.288 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_jaeger_agent", 2024-09-10T10:04:50.288 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.288 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.288 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.288 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-10T10:04:50.288 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.288 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.288 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.288 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Jaeger agent container image", 2024-09-10T10:04:50.288 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.288 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.289 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.289 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.289 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_jaeger_collector": { 2024-09-10T10:04:50.289 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_jaeger_collector", 2024-09-10T10:04:50.289 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.289 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.289 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.289 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-10T10:04:50.289 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.289 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.289 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.289 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Jaeger collector container image", 2024-09-10T10:04:50.289 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.290 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.290 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.290 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.290 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_jaeger_query": { 2024-09-10T10:04:50.290 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_jaeger_query", 2024-09-10T10:04:50.290 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.290 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.290 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.290 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-10T10:04:50.290 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.290 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.290 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.290 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Jaeger query container image", 2024-09-10T10:04:50.291 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.291 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.291 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.291 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.291 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_keepalived": { 2024-09-10T10:04:50.291 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_keepalived", 2024-09-10T10:04:50.291 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.291 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.291 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.291 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-10T10:04:50.291 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.291 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.291 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.291 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Keepalived container image", 2024-09-10T10:04:50.292 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.292 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.292 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.292 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.292 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_loki": { 2024-09-10T10:04:50.292 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_loki", 2024-09-10T10:04:50.292 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.292 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.292 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.292 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-10T10:04:50.292 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.292 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.292 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.292 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Loki container image", 2024-09-10T10:04:50.293 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.293 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.293 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.293 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.293 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_node_exporter": { 2024-09-10T10:04:50.293 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_node_exporter", 2024-09-10T10:04:50.293 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.293 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.293 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.293 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-09-10T10:04:50.293 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.293 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.293 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.293 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Prometheus container image", 2024-09-10T10:04:50.294 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.294 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.294 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.294 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.294 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_nvmeof": { 2024-09-10T10:04:50.294 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_nvmeof", 2024-09-10T10:04:50.294 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.294 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.294 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.294 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-09-10T10:04:50.294 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.294 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.294 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.294 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Nvme-of container image", 2024-09-10T10:04:50.295 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.295 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.295 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.295 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.295 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_prometheus": { 2024-09-10T10:04:50.295 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_prometheus", 2024-09-10T10:04:50.295 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.295 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.295 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.295 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-09-10T10:04:50.295 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.295 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.295 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.295 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Prometheus container image", 2024-09-10T10:04:50.296 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.296 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.296 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.296 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.296 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_promtail": { 2024-09-10T10:04:50.296 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_promtail", 2024-09-10T10:04:50.296 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.296 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.296 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.296 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-10T10:04:50.296 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.296 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.296 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.296 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Promtail container image", 2024-09-10T10:04:50.297 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.297 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.297 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.297 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.297 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_samba": { 2024-09-10T10:04:50.297 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_samba", 2024-09-10T10:04:50.297 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.297 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.297 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.297 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-10T10:04:50.297 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.297 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.297 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.298 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Samba/SMB container image", 2024-09-10T10:04:50.298 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.298 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.298 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.298 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.298 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_snmp_gateway": { 2024-09-10T10:04:50.298 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_snmp_gateway", 2024-09-10T10:04:50.298 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.298 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.298 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.298 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-10T10:04:50.298 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.298 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.298 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.299 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "SNMP Gateway container image", 2024-09-10T10:04:50.299 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.299 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.299 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.299 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.299 INFO:teuthology.orchestra.run.smithi005.stdout: "container_init": { 2024-09-10T10:04:50.299 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_init", 2024-09-10T10:04:50.299 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.299 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.299 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.299 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-10T10:04:50.299 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.299 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.299 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.299 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Run podman/docker with `--init`", 2024-09-10T10:04:50.300 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.300 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.300 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.300 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.300 INFO:teuthology.orchestra.run.smithi005.stdout: "daemon_cache_timeout": { 2024-09-10T10:04:50.300 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "daemon_cache_timeout", 2024-09-10T10:04:50.300 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-10T10:04:50.300 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.300 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.300 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "600", 2024-09-10T10:04:50.300 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.300 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.300 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.301 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-10T10:04:50.301 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.301 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.301 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.301 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.301 INFO:teuthology.orchestra.run.smithi005.stdout: "default_cephadm_command_timeout": { 2024-09-10T10:04:50.301 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "default_cephadm_command_timeout", 2024-09-10T10:04:50.301 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-10T10:04:50.301 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.301 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.301 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "900", 2024-09-10T10:04:50.301 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.301 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.301 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.302 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-10T10:04:50.302 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.302 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.302 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.302 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.302 INFO:teuthology.orchestra.run.smithi005.stdout: "default_registry": { 2024-09-10T10:04:50.302 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "default_registry", 2024-09-10T10:04:50.302 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.302 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.302 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.302 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "docker.io", 2024-09-10T10:04:50.302 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.302 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.302 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.303 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-10T10:04:50.303 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.303 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.303 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.303 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.303 INFO:teuthology.orchestra.run.smithi005.stdout: "device_cache_timeout": { 2024-09-10T10:04:50.303 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "device_cache_timeout", 2024-09-10T10:04:50.303 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-10T10:04:50.303 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.303 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.303 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "1800", 2024-09-10T10:04:50.303 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.303 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.303 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.303 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "seconds to cache device inventory", 2024-09-10T10:04:50.304 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.304 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.304 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.304 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.304 INFO:teuthology.orchestra.run.smithi005.stdout: "device_enhanced_scan": { 2024-09-10T10:04:50.304 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "device_enhanced_scan", 2024-09-10T10:04:50.304 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.304 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.304 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.304 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-10T10:04:50.304 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.304 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.304 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.304 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-10T10:04:50.305 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.305 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.305 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.305 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.305 INFO:teuthology.orchestra.run.smithi005.stdout: "facts_cache_timeout": { 2024-09-10T10:04:50.305 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "facts_cache_timeout", 2024-09-10T10:04:50.305 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-10T10:04:50.305 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.305 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.305 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "60", 2024-09-10T10:04:50.305 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.305 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.305 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.305 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "seconds to cache host facts data", 2024-09-10T10:04:50.306 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.306 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.306 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.306 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.306 INFO:teuthology.orchestra.run.smithi005.stdout: "grafana_dashboards_path": { 2024-09-10T10:04:50.306 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "grafana_dashboards_path", 2024-09-10T10:04:50.306 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.306 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.306 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.306 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-10T10:04:50.306 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.306 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.306 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.307 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-10T10:04:50.307 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.307 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.307 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.307 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.307 INFO:teuthology.orchestra.run.smithi005.stdout: "host_check_interval": { 2024-09-10T10:04:50.307 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "host_check_interval", 2024-09-10T10:04:50.307 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-10T10:04:50.307 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.307 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.307 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "600", 2024-09-10T10:04:50.307 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.307 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.308 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.308 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "how frequently to perform a host check", 2024-09-10T10:04:50.308 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.308 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.308 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.308 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.308 INFO:teuthology.orchestra.run.smithi005.stdout: "hw_monitoring": { 2024-09-10T10:04:50.308 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "hw_monitoring", 2024-09-10T10:04:50.308 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.308 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.308 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.308 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-10T10:04:50.308 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.308 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.309 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.309 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-10T10:04:50.309 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.309 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.309 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.309 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.309 INFO:teuthology.orchestra.run.smithi005.stdout: "inventory_list_all": { 2024-09-10T10:04:50.309 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "inventory_list_all", 2024-09-10T10:04:50.309 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.309 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.309 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.309 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-10T10:04:50.309 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.309 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.310 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.310 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-10T10:04:50.310 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.310 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.310 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.310 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.310 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-10T10:04:50.310 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-10T10:04:50.310 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.310 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.310 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.310 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-10T10:04:50.310 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.310 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.311 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-10T10:04:50.311 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-10T10:04:50.311 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-10T10:04:50.311 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-10T10:04:50.311 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-10T10:04:50.311 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-10T10:04:50.311 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-10T10:04:50.311 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-10T10:04:50.311 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.311 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.311 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.311 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.311 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.311 INFO:teuthology.orchestra.run.smithi005.stdout: "log_refresh_metadata": { 2024-09-10T10:04:50.311 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_refresh_metadata", 2024-09-10T10:04:50.312 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.312 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.312 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.312 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-10T10:04:50.312 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.312 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.312 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.312 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-10T10:04:50.312 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.312 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.312 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.312 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.312 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-10T10:04:50.313 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-10T10:04:50.313 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.313 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.313 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.313 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-10T10:04:50.313 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.313 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.313 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.313 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-10T10:04:50.313 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.313 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.313 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.313 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.313 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-10T10:04:50.313 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-10T10:04:50.314 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.314 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.314 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.314 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-10T10:04:50.314 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.314 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.314 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-10T10:04:50.314 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-10T10:04:50.314 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-10T10:04:50.314 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-10T10:04:50.314 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-10T10:04:50.314 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-10T10:04:50.314 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-10T10:04:50.314 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-10T10:04:50.315 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.315 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.315 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.315 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.315 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.315 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-10T10:04:50.315 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-10T10:04:50.315 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.315 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.315 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.315 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-10T10:04:50.315 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.315 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.315 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.315 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.316 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.316 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.316 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.316 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.316 INFO:teuthology.orchestra.run.smithi005.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-10T10:04:50.316 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-10T10:04:50.316 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.316 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.316 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.316 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-10T10:04:50.316 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.316 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.316 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.317 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-10T10:04:50.317 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.317 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.317 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.317 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.317 INFO:teuthology.orchestra.run.smithi005.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-10T10:04:50.317 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-10T10:04:50.317 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.317 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.317 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.317 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "*", 2024-09-10T10:04:50.317 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.317 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.317 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.318 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-10T10:04:50.318 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.318 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.318 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.318 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.318 INFO:teuthology.orchestra.run.smithi005.stdout: "max_count_per_host": { 2024-09-10T10:04:50.318 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "max_count_per_host", 2024-09-10T10:04:50.318 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-10T10:04:50.318 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.318 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.318 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10", 2024-09-10T10:04:50.318 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.318 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.318 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.318 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "max number of daemons per service per host", 2024-09-10T10:04:50.319 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.319 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.319 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.319 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.319 INFO:teuthology.orchestra.run.smithi005.stdout: "max_osd_draining_count": { 2024-09-10T10:04:50.319 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "max_osd_draining_count", 2024-09-10T10:04:50.319 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-10T10:04:50.319 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.319 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.319 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10", 2024-09-10T10:04:50.319 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.319 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.319 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.319 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-10T10:04:50.319 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.320 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.320 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.320 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.320 INFO:teuthology.orchestra.run.smithi005.stdout: "migration_current": { 2024-09-10T10:04:50.320 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "migration_current", 2024-09-10T10:04:50.320 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-10T10:04:50.320 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.320 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.320 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-10T10:04:50.320 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.320 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.320 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.320 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "internal - do not modify", 2024-09-10T10:04:50.320 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.320 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.321 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.321 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.321 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": { 2024-09-10T10:04:50.321 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "mode", 2024-09-10T10:04:50.321 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.321 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.321 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.321 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "root", 2024-09-10T10:04:50.321 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.321 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.321 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-10T10:04:50.321 INFO:teuthology.orchestra.run.smithi005.stdout: "cephadm-package", 2024-09-10T10:04:50.321 INFO:teuthology.orchestra.run.smithi005.stdout: "root" 2024-09-10T10:04:50.321 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-10T10:04:50.322 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "mode for remote execution of cephadm", 2024-09-10T10:04:50.322 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.322 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.322 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.322 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.322 INFO:teuthology.orchestra.run.smithi005.stdout: "oob_default_addr": { 2024-09-10T10:04:50.322 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "oob_default_addr", 2024-09-10T10:04:50.322 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.322 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.322 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.322 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "169.254.1.1", 2024-09-10T10:04:50.322 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.322 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.322 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.322 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-10T10:04:50.323 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.323 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.323 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.323 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.323 INFO:teuthology.orchestra.run.smithi005.stdout: "prometheus_alerts_path": { 2024-09-10T10:04:50.323 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "prometheus_alerts_path", 2024-09-10T10:04:50.323 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.323 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.323 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.323 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-10T10:04:50.323 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.323 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.323 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.324 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-10T10:04:50.324 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.324 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.324 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.324 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.324 INFO:teuthology.orchestra.run.smithi005.stdout: "registry_insecure": { 2024-09-10T10:04:50.324 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "registry_insecure", 2024-09-10T10:04:50.324 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.324 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.324 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.324 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-10T10:04:50.324 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.324 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.324 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.325 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-10T10:04:50.325 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.325 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.325 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.325 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.325 INFO:teuthology.orchestra.run.smithi005.stdout: "registry_password": { 2024-09-10T10:04:50.325 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "registry_password", 2024-09-10T10:04:50.325 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.325 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.325 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.325 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-10T10:04:50.325 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.326 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.326 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.326 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-10T10:04:50.326 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.326 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.326 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.326 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.326 INFO:teuthology.orchestra.run.smithi005.stdout: "registry_url": { 2024-09-10T10:04:50.326 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "registry_url", 2024-09-10T10:04:50.326 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.326 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.327 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.327 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-10T10:04:50.327 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.327 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.327 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.327 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-10T10:04:50.327 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.327 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.327 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.327 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.327 INFO:teuthology.orchestra.run.smithi005.stdout: "registry_username": { 2024-09-10T10:04:50.327 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "registry_username", 2024-09-10T10:04:50.327 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.327 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.328 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.328 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-10T10:04:50.328 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.328 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.328 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.328 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-10T10:04:50.328 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.328 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.328 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.328 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.328 INFO:teuthology.orchestra.run.smithi005.stdout: "secure_monitoring_stack": { 2024-09-10T10:04:50.328 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "secure_monitoring_stack", 2024-09-10T10:04:50.328 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.328 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.329 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.329 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-10T10:04:50.329 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.329 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.329 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.329 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-10T10:04:50.329 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.329 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.329 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.329 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.329 INFO:teuthology.orchestra.run.smithi005.stdout: "service_discovery_port": { 2024-09-10T10:04:50.329 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "service_discovery_port", 2024-09-10T10:04:50.329 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-10T10:04:50.329 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.330 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.330 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "8765", 2024-09-10T10:04:50.330 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.330 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.330 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.330 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "cephadm service discovery port", 2024-09-10T10:04:50.330 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.330 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.330 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.330 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.330 INFO:teuthology.orchestra.run.smithi005.stdout: "ssh_config_file": { 2024-09-10T10:04:50.330 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ssh_config_file", 2024-09-10T10:04:50.330 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.330 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.331 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.331 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-10T10:04:50.331 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.331 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.331 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.331 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-10T10:04:50.331 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.331 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.331 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.331 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.331 INFO:teuthology.orchestra.run.smithi005.stdout: "use_agent": { 2024-09-10T10:04:50.331 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "use_agent", 2024-09-10T10:04:50.331 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.331 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.331 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.332 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-10T10:04:50.332 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.332 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.332 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.332 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-10T10:04:50.332 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.332 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.332 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.332 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.332 INFO:teuthology.orchestra.run.smithi005.stdout: "use_repo_digest": { 2024-09-10T10:04:50.332 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "use_repo_digest", 2024-09-10T10:04:50.332 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.332 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.332 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.333 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-10T10:04:50.333 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.333 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.333 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.333 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-10T10:04:50.333 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.333 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.333 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.333 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.333 INFO:teuthology.orchestra.run.smithi005.stdout: "warn_on_failed_host_check": { 2024-09-10T10:04:50.333 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "warn_on_failed_host_check", 2024-09-10T10:04:50.333 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.333 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.334 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.334 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-10T10:04:50.334 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.334 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.334 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.334 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "raise a health warning if the host check fails", 2024-09-10T10:04:50.334 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.334 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.334 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.334 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.334 INFO:teuthology.orchestra.run.smithi005.stdout: "warn_on_stray_daemons": { 2024-09-10T10:04:50.334 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "warn_on_stray_daemons", 2024-09-10T10:04:50.334 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.334 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.334 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.335 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-10T10:04:50.335 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.335 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.335 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.335 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-10T10:04:50.335 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.335 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.335 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.335 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.335 INFO:teuthology.orchestra.run.smithi005.stdout: "warn_on_stray_hosts": { 2024-09-10T10:04:50.335 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "warn_on_stray_hosts", 2024-09-10T10:04:50.335 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.335 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.336 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.336 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-10T10:04:50.336 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.336 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.336 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.336 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-10T10:04:50.336 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.336 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.336 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.336 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T10:04:50.336 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T10:04:50.336 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.336 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-10T10:04:50.336 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "crash", 2024-09-10T10:04:50.337 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-10T10:04:50.337 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-10T10:04:50.337 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-10T10:04:50.337 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-10T10:04:50.337 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-10T10:04:50.337 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.337 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.337 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.337 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-10T10:04:50.337 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.337 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.337 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-10T10:04:50.337 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-10T10:04:50.337 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-10T10:04:50.338 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-10T10:04:50.338 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-10T10:04:50.338 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-10T10:04:50.338 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-10T10:04:50.338 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-10T10:04:50.338 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.338 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.338 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.338 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.338 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.338 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-10T10:04:50.338 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-10T10:04:50.338 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.338 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.339 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.339 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-10T10:04:50.339 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.339 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.339 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.339 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.339 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.339 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.339 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.339 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.339 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-10T10:04:50.339 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-10T10:04:50.339 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.339 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.339 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.340 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-10T10:04:50.340 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.340 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.340 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-10T10:04:50.340 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-10T10:04:50.340 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-10T10:04:50.340 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-10T10:04:50.340 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-10T10:04:50.340 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-10T10:04:50.340 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-10T10:04:50.340 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-10T10:04:50.340 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.340 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.340 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.341 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.341 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.341 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-10T10:04:50.341 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-10T10:04:50.341 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.341 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.341 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.341 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-10T10:04:50.341 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.341 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.341 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.341 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.341 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.341 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.341 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.342 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.342 INFO:teuthology.orchestra.run.smithi005.stdout: "retain_interval": { 2024-09-10T10:04:50.342 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "retain_interval", 2024-09-10T10:04:50.342 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-10T10:04:50.342 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.342 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.342 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "31536000", 2024-09-10T10:04:50.342 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.342 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.342 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.342 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-10T10:04:50.342 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.342 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.342 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.342 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.343 INFO:teuthology.orchestra.run.smithi005.stdout: "warn_recent_interval": { 2024-09-10T10:04:50.343 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "warn_recent_interval", 2024-09-10T10:04:50.343 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-10T10:04:50.343 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.343 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.343 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "1209600", 2024-09-10T10:04:50.343 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.343 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.343 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.343 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-10T10:04:50.343 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.343 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.343 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.343 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T10:04:50.343 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T10:04:50.344 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.344 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-10T10:04:50.344 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "dashboard", 2024-09-10T10:04:50.344 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-10T10:04:50.344 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-10T10:04:50.344 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-10T10:04:50.344 INFO:teuthology.orchestra.run.smithi005.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-10T10:04:50.344 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-10T10:04:50.344 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-10T10:04:50.344 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.344 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.344 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10", 2024-09-10T10:04:50.344 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.344 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.344 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.344 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.345 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.345 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.345 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.345 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.345 INFO:teuthology.orchestra.run.smithi005.stdout: "ALERTMANAGER_API_HOST": { 2024-09-10T10:04:50.345 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-10T10:04:50.345 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.345 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.345 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.345 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-10T10:04:50.345 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.345 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.345 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.346 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.346 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.346 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.346 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.346 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.346 INFO:teuthology.orchestra.run.smithi005.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-10T10:04:50.346 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-10T10:04:50.346 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.346 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.346 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.346 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-10T10:04:50.346 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.346 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.346 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.346 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.347 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.347 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.347 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.347 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.347 INFO:teuthology.orchestra.run.smithi005.stdout: "AUDIT_API_ENABLED": { 2024-09-10T10:04:50.347 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "AUDIT_API_ENABLED", 2024-09-10T10:04:50.347 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.347 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.347 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.347 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-10T10:04:50.347 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.347 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.347 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.347 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.347 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.348 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.348 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.348 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.348 INFO:teuthology.orchestra.run.smithi005.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-10T10:04:50.348 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-10T10:04:50.348 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.348 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.348 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.348 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-10T10:04:50.348 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.348 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.348 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.348 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.348 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.348 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.349 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.349 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.349 INFO:teuthology.orchestra.run.smithi005.stdout: "ENABLE_BROWSABLE_API": { 2024-09-10T10:04:50.349 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-10T10:04:50.349 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.349 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.349 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.349 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-10T10:04:50.349 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.349 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.349 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.349 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.349 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.349 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.349 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.349 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.350 INFO:teuthology.orchestra.run.smithi005.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-10T10:04:50.350 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-10T10:04:50.350 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.350 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.350 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.350 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-10T10:04:50.350 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.350 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.350 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.350 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.350 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.350 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.350 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.350 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.351 INFO:teuthology.orchestra.run.smithi005.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-10T10:04:50.351 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-10T10:04:50.351 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.351 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.351 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.351 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-10T10:04:50.351 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.351 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.351 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.351 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.351 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.351 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.351 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.351 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.351 INFO:teuthology.orchestra.run.smithi005.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-10T10:04:50.351 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-10T10:04:50.352 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.352 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.352 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.352 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-10T10:04:50.352 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.352 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.352 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.352 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.352 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.352 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.352 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.352 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.352 INFO:teuthology.orchestra.run.smithi005.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-10T10:04:50.352 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-10T10:04:50.353 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.353 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.353 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.353 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-10T10:04:50.353 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.353 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.353 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.353 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.353 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.353 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.353 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.353 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.353 INFO:teuthology.orchestra.run.smithi005.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-10T10:04:50.353 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-10T10:04:50.354 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.354 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.354 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.354 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-10T10:04:50.354 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.354 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.354 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.354 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.354 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.354 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.354 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.354 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.354 INFO:teuthology.orchestra.run.smithi005.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-10T10:04:50.355 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-10T10:04:50.355 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.355 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.355 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.355 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-10T10:04:50.355 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.355 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.355 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.355 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.355 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.355 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.355 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.355 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.355 INFO:teuthology.orchestra.run.smithi005.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-10T10:04:50.356 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-10T10:04:50.356 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.356 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.356 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.356 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-10T10:04:50.356 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.356 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.356 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.356 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.356 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.356 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.356 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.356 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.357 INFO:teuthology.orchestra.run.smithi005.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-10T10:04:50.357 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-10T10:04:50.357 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.357 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.357 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.357 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-10T10:04:50.357 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.357 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.357 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.357 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.357 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.357 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.358 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.358 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.358 INFO:teuthology.orchestra.run.smithi005.stdout: "GRAFANA_API_PASSWORD": { 2024-09-10T10:04:50.358 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-10T10:04:50.358 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.358 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.358 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.358 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "admin", 2024-09-10T10:04:50.358 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.358 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.358 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.358 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.358 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.358 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.359 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.359 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.359 INFO:teuthology.orchestra.run.smithi005.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-10T10:04:50.359 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-10T10:04:50.359 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.359 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.359 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.359 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-10T10:04:50.359 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.359 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.359 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.359 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.359 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.359 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.360 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.360 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.360 INFO:teuthology.orchestra.run.smithi005.stdout: "GRAFANA_API_URL": { 2024-09-10T10:04:50.360 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "GRAFANA_API_URL", 2024-09-10T10:04:50.360 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.360 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.360 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.360 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-10T10:04:50.360 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.360 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.360 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.360 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.360 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.360 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.361 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.361 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.361 INFO:teuthology.orchestra.run.smithi005.stdout: "GRAFANA_API_USERNAME": { 2024-09-10T10:04:50.361 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-10T10:04:50.361 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.361 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.361 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.361 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "admin", 2024-09-10T10:04:50.361 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.361 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.361 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.361 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.361 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.361 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.362 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.362 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.362 INFO:teuthology.orchestra.run.smithi005.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-10T10:04:50.362 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-10T10:04:50.362 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.362 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.362 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.362 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-10T10:04:50.362 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.362 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.362 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.362 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.362 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.362 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.362 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.363 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.363 INFO:teuthology.orchestra.run.smithi005.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-10T10:04:50.363 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-10T10:04:50.363 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.363 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.363 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.363 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-10T10:04:50.363 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.363 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.363 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.363 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.363 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.364 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.364 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.364 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.364 INFO:teuthology.orchestra.run.smithi005.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-10T10:04:50.364 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-10T10:04:50.364 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.364 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.364 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.364 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-10T10:04:50.364 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.364 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.364 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.365 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.365 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.365 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.365 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.365 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.365 INFO:teuthology.orchestra.run.smithi005.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-10T10:04:50.365 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-10T10:04:50.365 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.365 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.365 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.365 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-10T10:04:50.365 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.365 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.366 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.366 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.366 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.366 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.366 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.366 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.366 INFO:teuthology.orchestra.run.smithi005.stdout: "PROMETHEUS_API_HOST": { 2024-09-10T10:04:50.366 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-10T10:04:50.366 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.366 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.366 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.366 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-10T10:04:50.366 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.367 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.367 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.367 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.367 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.367 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.367 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.367 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.367 INFO:teuthology.orchestra.run.smithi005.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-10T10:04:50.367 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-10T10:04:50.367 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.367 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.367 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.367 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-10T10:04:50.368 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.368 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.368 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.368 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.368 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.368 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.368 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.368 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.368 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-10T10:04:50.368 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-10T10:04:50.368 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.368 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.369 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.369 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-10T10:04:50.369 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.369 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.369 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.369 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.369 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.369 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.369 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.369 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.369 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-10T10:04:50.369 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-10T10:04:50.369 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.370 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.370 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.370 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-10T10:04:50.370 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.370 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.370 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.370 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.370 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.370 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.370 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.371 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.371 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-10T10:04:50.371 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-10T10:04:50.371 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.371 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.371 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.371 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-10T10:04:50.371 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.371 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.371 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.371 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.371 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.371 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.371 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.371 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.372 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-10T10:04:50.372 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-10T10:04:50.372 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.372 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.372 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.372 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-10T10:04:50.372 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.372 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.372 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.372 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.372 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.372 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.372 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.372 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.373 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-10T10:04:50.373 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-10T10:04:50.373 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.373 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.373 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.373 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-10T10:04:50.373 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.373 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.373 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.373 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.373 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.373 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.373 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.373 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.373 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-10T10:04:50.374 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-10T10:04:50.374 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.374 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.374 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.374 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-10T10:04:50.374 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.374 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.374 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.374 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.374 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.374 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.374 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.374 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.374 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-10T10:04:50.375 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-10T10:04:50.375 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.375 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.375 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.375 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-10T10:04:50.375 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.375 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.375 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.375 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.375 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.375 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.375 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.375 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.375 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_ENABLED": { 2024-09-10T10:04:50.375 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-10T10:04:50.376 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.376 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.376 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.376 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-10T10:04:50.376 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.376 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.376 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.376 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.376 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.376 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.376 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.376 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.376 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-10T10:04:50.376 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-10T10:04:50.377 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.377 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.377 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.377 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-10T10:04:50.377 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.377 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.377 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.377 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.377 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.377 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.377 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.377 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.378 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-10T10:04:50.378 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-10T10:04:50.378 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-10T10:04:50.378 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.378 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.378 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10", 2024-09-10T10:04:50.378 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.378 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.378 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.378 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.378 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.378 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.378 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.378 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.378 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-10T10:04:50.379 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-10T10:04:50.379 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-10T10:04:50.379 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.379 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.379 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "8", 2024-09-10T10:04:50.379 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.379 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.379 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.379 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.379 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.379 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.379 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.379 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.379 INFO:teuthology.orchestra.run.smithi005.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-10T10:04:50.379 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-10T10:04:50.380 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-10T10:04:50.380 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.380 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.380 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "45", 2024-09-10T10:04:50.380 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.380 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.380 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.380 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.380 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.380 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.380 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.380 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.380 INFO:teuthology.orchestra.run.smithi005.stdout: "RGW_API_ACCESS_KEY": { 2024-09-10T10:04:50.380 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-10T10:04:50.381 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.381 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.381 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.381 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-10T10:04:50.381 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.381 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.381 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.381 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.381 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.381 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.381 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.381 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.382 INFO:teuthology.orchestra.run.smithi005.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-10T10:04:50.382 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-10T10:04:50.382 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.382 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.382 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.382 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "admin", 2024-09-10T10:04:50.382 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.382 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.382 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.382 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.382 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.382 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.382 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.383 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.383 INFO:teuthology.orchestra.run.smithi005.stdout: "RGW_API_SECRET_KEY": { 2024-09-10T10:04:50.383 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-10T10:04:50.383 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.383 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.383 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.383 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-10T10:04:50.383 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.383 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.383 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.383 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.383 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.383 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.383 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.384 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.384 INFO:teuthology.orchestra.run.smithi005.stdout: "RGW_API_SSL_VERIFY": { 2024-09-10T10:04:50.384 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-10T10:04:50.384 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.384 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.384 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.384 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-10T10:04:50.384 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.384 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.384 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.384 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.384 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.384 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.384 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.385 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.385 INFO:teuthology.orchestra.run.smithi005.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-10T10:04:50.385 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-10T10:04:50.385 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-10T10:04:50.385 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.385 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.385 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-09-10T10:04:50.385 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.385 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.385 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.385 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.385 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.385 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.386 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.386 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.386 INFO:teuthology.orchestra.run.smithi005.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-10T10:04:50.386 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-10T10:04:50.386 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-10T10:04:50.386 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.386 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.386 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10", 2024-09-10T10:04:50.386 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.386 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.386 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.386 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.386 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.387 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.387 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.387 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.387 INFO:teuthology.orchestra.run.smithi005.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-10T10:04:50.387 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-10T10:04:50.387 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-10T10:04:50.387 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.387 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.387 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "5", 2024-09-10T10:04:50.387 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.387 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.387 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.387 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.387 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.388 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.388 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.388 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.388 INFO:teuthology.orchestra.run.smithi005.stdout: "cross_origin_url": { 2024-09-10T10:04:50.388 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "cross_origin_url", 2024-09-10T10:04:50.388 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.388 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.388 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.388 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-10T10:04:50.388 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.388 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.388 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.388 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.388 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.389 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.389 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.389 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.389 INFO:teuthology.orchestra.run.smithi005.stdout: "crt_file": { 2024-09-10T10:04:50.389 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "crt_file", 2024-09-10T10:04:50.389 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.389 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.389 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.389 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-10T10:04:50.389 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.389 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.389 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.389 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.389 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.390 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.390 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.390 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.390 INFO:teuthology.orchestra.run.smithi005.stdout: "debug": { 2024-09-10T10:04:50.390 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "debug", 2024-09-10T10:04:50.390 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.390 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.390 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.390 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-10T10:04:50.390 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.390 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.390 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.390 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Enable/disable debug options", 2024-09-10T10:04:50.390 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.391 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.391 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.391 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.391 INFO:teuthology.orchestra.run.smithi005.stdout: "jwt_token_ttl": { 2024-09-10T10:04:50.391 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "jwt_token_ttl", 2024-09-10T10:04:50.391 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-10T10:04:50.391 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.391 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.391 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "28800", 2024-09-10T10:04:50.391 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.391 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.391 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.391 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.392 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.392 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.392 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.392 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.392 INFO:teuthology.orchestra.run.smithi005.stdout: "key_file": { 2024-09-10T10:04:50.392 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "key_file", 2024-09-10T10:04:50.392 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.392 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.392 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.392 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-10T10:04:50.392 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.392 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.392 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.393 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.393 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.393 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.393 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.393 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.393 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-10T10:04:50.393 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-10T10:04:50.393 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.393 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.393 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.393 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-10T10:04:50.393 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.393 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.394 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-10T10:04:50.394 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-10T10:04:50.394 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-10T10:04:50.394 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-10T10:04:50.394 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-10T10:04:50.394 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-10T10:04:50.394 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-10T10:04:50.394 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-10T10:04:50.394 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.394 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.394 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.394 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.394 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.394 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-10T10:04:50.395 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-10T10:04:50.395 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.395 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.395 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.395 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-10T10:04:50.395 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.395 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.395 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.395 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.395 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.395 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.395 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.395 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.396 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-10T10:04:50.396 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-10T10:04:50.396 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.396 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.396 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.396 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-10T10:04:50.396 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.396 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.396 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-10T10:04:50.396 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-10T10:04:50.396 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-10T10:04:50.396 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-10T10:04:50.396 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-10T10:04:50.397 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-10T10:04:50.397 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-10T10:04:50.397 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-10T10:04:50.397 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.397 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.397 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.397 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.397 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.397 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-10T10:04:50.397 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-10T10:04:50.397 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.397 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.397 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.398 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-10T10:04:50.398 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.398 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.398 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.398 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.398 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.398 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.398 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.398 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.398 INFO:teuthology.orchestra.run.smithi005.stdout: "motd": { 2024-09-10T10:04:50.398 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "motd", 2024-09-10T10:04:50.398 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.398 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.398 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.398 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-10T10:04:50.399 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.399 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.399 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.399 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "The message of the day", 2024-09-10T10:04:50.399 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.399 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.399 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.399 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.399 INFO:teuthology.orchestra.run.smithi005.stdout: "redirect_resolve_ip_addr": { 2024-09-10T10:04:50.399 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "redirect_resolve_ip_addr", 2024-09-10T10:04:50.399 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.399 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.399 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.399 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-10T10:04:50.400 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.400 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.400 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.400 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.400 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.400 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.400 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.400 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.400 INFO:teuthology.orchestra.run.smithi005.stdout: "server_addr": { 2024-09-10T10:04:50.400 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "server_addr", 2024-09-10T10:04:50.400 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.400 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.400 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.400 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "::", 2024-09-10T10:04:50.401 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.401 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.401 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.401 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.401 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.401 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.401 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.401 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.401 INFO:teuthology.orchestra.run.smithi005.stdout: "server_port": { 2024-09-10T10:04:50.401 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "server_port", 2024-09-10T10:04:50.401 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-10T10:04:50.401 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.401 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.401 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "8080", 2024-09-10T10:04:50.402 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.402 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.402 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.402 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.402 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.402 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.402 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.402 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.402 INFO:teuthology.orchestra.run.smithi005.stdout: "ssl": { 2024-09-10T10:04:50.402 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ssl", 2024-09-10T10:04:50.402 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.402 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.402 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.402 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-10T10:04:50.402 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.403 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.403 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.403 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.403 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.403 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.403 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.403 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.403 INFO:teuthology.orchestra.run.smithi005.stdout: "ssl_server_port": { 2024-09-10T10:04:50.403 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ssl_server_port", 2024-09-10T10:04:50.403 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-10T10:04:50.403 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.403 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.403 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "8443", 2024-09-10T10:04:50.404 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.404 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.404 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.404 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.404 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.404 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.404 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.404 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.404 INFO:teuthology.orchestra.run.smithi005.stdout: "standby_behaviour": { 2024-09-10T10:04:50.404 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "standby_behaviour", 2024-09-10T10:04:50.404 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.404 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.404 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.404 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "redirect", 2024-09-10T10:04:50.405 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.405 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.405 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-10T10:04:50.405 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-10T10:04:50.405 INFO:teuthology.orchestra.run.smithi005.stdout: "redirect" 2024-09-10T10:04:50.405 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-10T10:04:50.405 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.405 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.405 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.405 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.405 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.405 INFO:teuthology.orchestra.run.smithi005.stdout: "standby_error_status_code": { 2024-09-10T10:04:50.405 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "standby_error_status_code", 2024-09-10T10:04:50.406 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-10T10:04:50.406 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.406 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.406 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "500", 2024-09-10T10:04:50.406 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "400", 2024-09-10T10:04:50.406 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "599", 2024-09-10T10:04:50.406 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.406 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.406 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.406 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.406 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.406 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.407 INFO:teuthology.orchestra.run.smithi005.stdout: "url_prefix": { 2024-09-10T10:04:50.407 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "url_prefix", 2024-09-10T10:04:50.407 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.407 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.407 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.407 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-10T10:04:50.407 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.407 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.407 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.407 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.407 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.407 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.407 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.408 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T10:04:50.408 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T10:04:50.408 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.408 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-10T10:04:50.408 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "devicehealth", 2024-09-10T10:04:50.408 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-10T10:04:50.408 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-10T10:04:50.408 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-10T10:04:50.408 INFO:teuthology.orchestra.run.smithi005.stdout: "enable_monitoring": { 2024-09-10T10:04:50.408 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "enable_monitoring", 2024-09-10T10:04:50.408 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.408 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.408 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.408 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-10T10:04:50.409 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.409 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.409 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.409 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "monitor device health metrics", 2024-09-10T10:04:50.409 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.409 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.409 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.409 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.409 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-10T10:04:50.409 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-10T10:04:50.409 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.409 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.409 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.409 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-10T10:04:50.410 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.410 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.410 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-10T10:04:50.410 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-10T10:04:50.410 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-10T10:04:50.410 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-10T10:04:50.410 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-10T10:04:50.410 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-10T10:04:50.410 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-10T10:04:50.410 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-10T10:04:50.410 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.410 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.410 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.411 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.411 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.411 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-10T10:04:50.411 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-10T10:04:50.411 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.411 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.411 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.411 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-10T10:04:50.411 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.411 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.411 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.411 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.412 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.412 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.412 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.412 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.412 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-10T10:04:50.412 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-10T10:04:50.412 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.412 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.412 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.412 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-10T10:04:50.412 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.412 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.412 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-10T10:04:50.413 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-10T10:04:50.413 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-10T10:04:50.413 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-10T10:04:50.413 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-10T10:04:50.413 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-10T10:04:50.413 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-10T10:04:50.413 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-10T10:04:50.413 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.413 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.413 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.413 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.413 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.413 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-10T10:04:50.414 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-10T10:04:50.414 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.414 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.414 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.414 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-10T10:04:50.414 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.414 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.414 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.414 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.414 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.414 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.414 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.414 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.414 INFO:teuthology.orchestra.run.smithi005.stdout: "mark_out_threshold": { 2024-09-10T10:04:50.415 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "mark_out_threshold", 2024-09-10T10:04:50.415 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-10T10:04:50.415 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.415 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.415 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "2419200", 2024-09-10T10:04:50.415 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.415 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.415 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.415 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-10T10:04:50.415 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.415 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.415 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.416 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.416 INFO:teuthology.orchestra.run.smithi005.stdout: "pool_name": { 2024-09-10T10:04:50.416 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "pool_name", 2024-09-10T10:04:50.416 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.416 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.416 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.416 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "device_health_metrics", 2024-09-10T10:04:50.416 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.416 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.416 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.416 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-10T10:04:50.416 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.416 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.417 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.417 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.417 INFO:teuthology.orchestra.run.smithi005.stdout: "retention_period": { 2024-09-10T10:04:50.417 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "retention_period", 2024-09-10T10:04:50.417 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-10T10:04:50.417 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.417 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.417 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "15552000", 2024-09-10T10:04:50.417 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.417 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.417 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.417 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "how long to retain device health metrics", 2024-09-10T10:04:50.418 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.418 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.418 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.418 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.418 INFO:teuthology.orchestra.run.smithi005.stdout: "scrape_frequency": { 2024-09-10T10:04:50.418 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "scrape_frequency", 2024-09-10T10:04:50.418 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-10T10:04:50.418 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.418 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.418 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "86400", 2024-09-10T10:04:50.418 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.418 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.419 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.419 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-10T10:04:50.419 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.419 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.419 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.419 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.419 INFO:teuthology.orchestra.run.smithi005.stdout: "self_heal": { 2024-09-10T10:04:50.419 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "self_heal", 2024-09-10T10:04:50.419 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.419 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.419 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.419 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-10T10:04:50.419 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.419 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.420 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.420 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-10T10:04:50.420 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.420 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.420 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.420 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.420 INFO:teuthology.orchestra.run.smithi005.stdout: "sleep_interval": { 2024-09-10T10:04:50.420 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sleep_interval", 2024-09-10T10:04:50.420 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-10T10:04:50.420 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.420 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.420 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "600", 2024-09-10T10:04:50.420 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.420 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.420 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.421 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "how frequently to wake up and check device health", 2024-09-10T10:04:50.421 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.421 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.421 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.421 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.421 INFO:teuthology.orchestra.run.smithi005.stdout: "warn_threshold": { 2024-09-10T10:04:50.421 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "warn_threshold", 2024-09-10T10:04:50.421 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-10T10:04:50.421 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.421 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.421 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "7257600", 2024-09-10T10:04:50.421 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.421 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.422 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.422 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-10T10:04:50.422 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.422 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.422 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.422 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T10:04:50.422 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T10:04:50.422 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.422 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-10T10:04:50.422 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "diskprediction_local", 2024-09-10T10:04:50.422 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-10T10:04:50.422 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-10T10:04:50.422 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-10T10:04:50.422 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-10T10:04:50.422 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-10T10:04:50.423 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.423 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.423 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.423 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-10T10:04:50.423 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.423 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.423 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-10T10:04:50.423 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-10T10:04:50.423 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-10T10:04:50.423 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-10T10:04:50.423 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-10T10:04:50.423 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-10T10:04:50.423 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-10T10:04:50.423 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-10T10:04:50.423 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.423 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.424 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.424 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.424 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.424 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-10T10:04:50.424 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-10T10:04:50.424 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.424 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.424 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.424 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-10T10:04:50.424 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.424 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.424 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.424 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.424 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.425 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.425 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.425 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.425 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-10T10:04:50.425 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-10T10:04:50.425 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.425 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.425 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.425 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-10T10:04:50.425 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.425 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.425 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-10T10:04:50.425 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-10T10:04:50.425 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-10T10:04:50.425 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-10T10:04:50.426 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-10T10:04:50.426 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-10T10:04:50.426 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-10T10:04:50.426 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-10T10:04:50.426 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.426 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.426 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.426 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.426 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.426 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-10T10:04:50.426 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-10T10:04:50.426 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.426 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.426 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.427 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-10T10:04:50.427 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.427 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.427 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.427 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.427 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.427 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.427 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.427 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.427 INFO:teuthology.orchestra.run.smithi005.stdout: "predict_interval": { 2024-09-10T10:04:50.427 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "predict_interval", 2024-09-10T10:04:50.427 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.427 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.427 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.427 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "86400", 2024-09-10T10:04:50.428 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.428 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.428 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.428 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.428 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.428 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.428 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.428 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.428 INFO:teuthology.orchestra.run.smithi005.stdout: "predictor_model": { 2024-09-10T10:04:50.428 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "predictor_model", 2024-09-10T10:04:50.428 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.428 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.428 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.428 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "prophetstor", 2024-09-10T10:04:50.428 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.429 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.429 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.429 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.429 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.429 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.429 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.429 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.429 INFO:teuthology.orchestra.run.smithi005.stdout: "sleep_interval": { 2024-09-10T10:04:50.429 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sleep_interval", 2024-09-10T10:04:50.429 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.429 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.429 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.429 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "600", 2024-09-10T10:04:50.429 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.429 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.430 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.430 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.430 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.430 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.430 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.430 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T10:04:50.430 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T10:04:50.430 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.430 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-10T10:04:50.430 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "influx", 2024-09-10T10:04:50.430 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": false, 2024-09-10T10:04:50.430 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "influxdb python module not found", 2024-09-10T10:04:50.430 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-10T10:04:50.430 INFO:teuthology.orchestra.run.smithi005.stdout: "batch_size": { 2024-09-10T10:04:50.430 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "batch_size", 2024-09-10T10:04:50.431 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-10T10:04:50.431 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.431 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.431 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "5000", 2024-09-10T10:04:50.431 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.431 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.431 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.431 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-10T10:04:50.431 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.431 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.431 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.431 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.431 INFO:teuthology.orchestra.run.smithi005.stdout: "database": { 2024-09-10T10:04:50.431 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "database", 2024-09-10T10:04:50.432 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.432 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.432 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.432 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "ceph", 2024-09-10T10:04:50.432 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.432 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.432 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.432 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-10T10:04:50.432 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.432 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.432 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.432 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.433 INFO:teuthology.orchestra.run.smithi005.stdout: "hostname": { 2024-09-10T10:04:50.433 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "hostname", 2024-09-10T10:04:50.433 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.433 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.433 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.433 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-10T10:04:50.433 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.433 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.433 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.433 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "InfluxDB server hostname", 2024-09-10T10:04:50.433 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.433 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.433 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.433 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.433 INFO:teuthology.orchestra.run.smithi005.stdout: "interval": { 2024-09-10T10:04:50.434 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "interval", 2024-09-10T10:04:50.434 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-10T10:04:50.434 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.434 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.434 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "30", 2024-09-10T10:04:50.434 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "5", 2024-09-10T10:04:50.434 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.434 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.434 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-10T10:04:50.434 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.434 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.434 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.434 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.434 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-10T10:04:50.435 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-10T10:04:50.435 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.435 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.435 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.435 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-10T10:04:50.435 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.435 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.435 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-10T10:04:50.435 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-10T10:04:50.435 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-10T10:04:50.435 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-10T10:04:50.435 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-10T10:04:50.435 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-10T10:04:50.435 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-10T10:04:50.435 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-10T10:04:50.436 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.436 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.436 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.436 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.436 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.436 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-10T10:04:50.436 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-10T10:04:50.436 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.436 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.436 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.436 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-10T10:04:50.436 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.436 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.436 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.436 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.437 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.437 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.437 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.437 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.437 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-10T10:04:50.437 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-10T10:04:50.437 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.437 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.437 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.437 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-10T10:04:50.437 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.437 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.437 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-10T10:04:50.437 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-10T10:04:50.437 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-10T10:04:50.438 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-10T10:04:50.438 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-10T10:04:50.438 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-10T10:04:50.438 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-10T10:04:50.438 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-10T10:04:50.438 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.438 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.438 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.438 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.438 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.438 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-10T10:04:50.438 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-10T10:04:50.438 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.438 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.438 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.439 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-10T10:04:50.439 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.439 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.439 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.439 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.439 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.439 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.439 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.439 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.439 INFO:teuthology.orchestra.run.smithi005.stdout: "password": { 2024-09-10T10:04:50.439 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "password", 2024-09-10T10:04:50.439 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.439 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.439 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.439 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-10T10:04:50.440 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.440 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.440 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.440 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "password of InfluxDB server user", 2024-09-10T10:04:50.440 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.440 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.440 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.440 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.440 INFO:teuthology.orchestra.run.smithi005.stdout: "port": { 2024-09-10T10:04:50.440 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "port", 2024-09-10T10:04:50.440 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-10T10:04:50.440 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.440 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.440 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "8086", 2024-09-10T10:04:50.440 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.441 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.441 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.441 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "InfluxDB server port", 2024-09-10T10:04:50.441 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.441 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.441 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.441 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.441 INFO:teuthology.orchestra.run.smithi005.stdout: "ssl": { 2024-09-10T10:04:50.441 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ssl", 2024-09-10T10:04:50.441 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.441 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.441 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.441 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "false", 2024-09-10T10:04:50.441 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.441 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.442 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.442 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-10T10:04:50.442 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.442 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.442 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.442 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.442 INFO:teuthology.orchestra.run.smithi005.stdout: "threads": { 2024-09-10T10:04:50.442 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "threads", 2024-09-10T10:04:50.442 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-10T10:04:50.442 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.442 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.442 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "5", 2024-09-10T10:04:50.442 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "1", 2024-09-10T10:04:50.442 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "32", 2024-09-10T10:04:50.442 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.443 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-10T10:04:50.443 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.443 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.443 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.443 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.443 INFO:teuthology.orchestra.run.smithi005.stdout: "username": { 2024-09-10T10:04:50.443 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "username", 2024-09-10T10:04:50.443 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.443 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.443 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.443 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-10T10:04:50.443 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.443 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.443 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.443 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "username of InfluxDB server user", 2024-09-10T10:04:50.444 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.444 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.444 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.444 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.444 INFO:teuthology.orchestra.run.smithi005.stdout: "verify_ssl": { 2024-09-10T10:04:50.444 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "verify_ssl", 2024-09-10T10:04:50.444 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.444 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.444 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.444 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "true", 2024-09-10T10:04:50.444 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.444 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.444 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.444 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-10T10:04:50.444 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.445 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.445 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.445 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T10:04:50.445 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T10:04:50.445 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.445 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-10T10:04:50.445 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "insights", 2024-09-10T10:04:50.445 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-10T10:04:50.445 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-10T10:04:50.445 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-10T10:04:50.445 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-10T10:04:50.445 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-10T10:04:50.445 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.445 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.446 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.446 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-10T10:04:50.446 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.446 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.446 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-10T10:04:50.446 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-10T10:04:50.446 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-10T10:04:50.446 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-10T10:04:50.446 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-10T10:04:50.446 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-10T10:04:50.446 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-10T10:04:50.446 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-10T10:04:50.446 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.446 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.447 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.447 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.447 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.447 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-10T10:04:50.447 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-10T10:04:50.447 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.447 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.447 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.447 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-10T10:04:50.447 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.447 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.447 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.447 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.447 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.447 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.448 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.448 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.448 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-10T10:04:50.448 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-10T10:04:50.448 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.448 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.448 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.448 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-10T10:04:50.448 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.448 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.448 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-10T10:04:50.448 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-10T10:04:50.448 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-10T10:04:50.448 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-10T10:04:50.449 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-10T10:04:50.449 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-10T10:04:50.449 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-10T10:04:50.449 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-10T10:04:50.449 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.449 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.449 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.449 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.449 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.449 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-10T10:04:50.449 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-10T10:04:50.449 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.449 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.449 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.449 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-10T10:04:50.450 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.450 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.450 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.450 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.450 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.450 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.450 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.450 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T10:04:50.450 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T10:04:50.450 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.450 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-10T10:04:50.450 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "iostat", 2024-09-10T10:04:50.450 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-10T10:04:50.450 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-10T10:04:50.450 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-10T10:04:50.451 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-10T10:04:50.451 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-10T10:04:50.451 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.451 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.451 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.451 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-10T10:04:50.451 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.451 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.451 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-10T10:04:50.451 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-10T10:04:50.451 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-10T10:04:50.451 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-10T10:04:50.451 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-10T10:04:50.451 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-10T10:04:50.451 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-10T10:04:50.452 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-10T10:04:50.452 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.452 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.452 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.452 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.452 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.452 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-10T10:04:50.452 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-10T10:04:50.452 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.452 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.452 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.452 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-10T10:04:50.452 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.453 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.453 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.453 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.453 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.453 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.453 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.453 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.453 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-10T10:04:50.453 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-10T10:04:50.453 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.453 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.453 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.453 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-10T10:04:50.453 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.453 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.454 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-10T10:04:50.454 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-10T10:04:50.454 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-10T10:04:50.454 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-10T10:04:50.454 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-10T10:04:50.454 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-10T10:04:50.454 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-10T10:04:50.454 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-10T10:04:50.454 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.454 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.454 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.454 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.454 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.454 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-10T10:04:50.454 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-10T10:04:50.455 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.455 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.455 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.455 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-10T10:04:50.455 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.455 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.455 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.455 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.455 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.455 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.455 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.455 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T10:04:50.455 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T10:04:50.455 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.455 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-10T10:04:50.456 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "k8sevents", 2024-09-10T10:04:50.456 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-10T10:04:50.456 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-10T10:04:50.456 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-10T10:04:50.456 INFO:teuthology.orchestra.run.smithi005.stdout: "ceph_event_retention_days": { 2024-09-10T10:04:50.456 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ceph_event_retention_days", 2024-09-10T10:04:50.456 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-10T10:04:50.456 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.456 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.456 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "7", 2024-09-10T10:04:50.456 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.456 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.456 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.456 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Days to hold ceph event information within local cache", 2024-09-10T10:04:50.456 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.457 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.457 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.457 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.457 INFO:teuthology.orchestra.run.smithi005.stdout: "config_check_secs": { 2024-09-10T10:04:50.457 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "config_check_secs", 2024-09-10T10:04:50.457 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-10T10:04:50.457 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.457 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.457 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10", 2024-09-10T10:04:50.457 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "10", 2024-09-10T10:04:50.457 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.457 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.457 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-09-10T10:04:50.457 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.457 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.458 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.458 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.458 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-10T10:04:50.458 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-10T10:04:50.458 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.458 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.458 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.458 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-10T10:04:50.458 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.458 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.458 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-10T10:04:50.458 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-10T10:04:50.458 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-10T10:04:50.458 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-10T10:04:50.458 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-10T10:04:50.459 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-10T10:04:50.459 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-10T10:04:50.459 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-10T10:04:50.459 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.459 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.459 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.459 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.459 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.459 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-10T10:04:50.459 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-10T10:04:50.459 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.459 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.459 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.459 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-10T10:04:50.459 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.460 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.460 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.460 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.460 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.460 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.460 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.460 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.460 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-10T10:04:50.460 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-10T10:04:50.460 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.460 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.460 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.460 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-10T10:04:50.460 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.461 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.461 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-10T10:04:50.461 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-10T10:04:50.461 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-10T10:04:50.461 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-10T10:04:50.461 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-10T10:04:50.461 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-10T10:04:50.461 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-10T10:04:50.461 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-10T10:04:50.461 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.461 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.461 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.461 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.461 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.461 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-10T10:04:50.462 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-10T10:04:50.462 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.462 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.462 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.462 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-10T10:04:50.462 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.462 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.462 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.462 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.462 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.462 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.462 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.462 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T10:04:50.462 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T10:04:50.462 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.463 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-10T10:04:50.463 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "localpool", 2024-09-10T10:04:50.463 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-10T10:04:50.463 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-10T10:04:50.463 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-10T10:04:50.463 INFO:teuthology.orchestra.run.smithi005.stdout: "failure_domain": { 2024-09-10T10:04:50.463 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "failure_domain", 2024-09-10T10:04:50.463 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.463 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.463 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.463 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "host", 2024-09-10T10:04:50.463 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.463 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.463 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.463 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "failure domain for any created local pool", 2024-09-10T10:04:50.464 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-10T10:04:50.464 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.464 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.464 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.464 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-10T10:04:50.464 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-10T10:04:50.464 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.464 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.464 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.464 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-10T10:04:50.464 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.464 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.464 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-10T10:04:50.464 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-10T10:04:50.465 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-10T10:04:50.465 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-10T10:04:50.465 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-10T10:04:50.465 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-10T10:04:50.465 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-10T10:04:50.465 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-10T10:04:50.465 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.465 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.465 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.465 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.465 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.465 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-10T10:04:50.465 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-10T10:04:50.465 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.466 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.466 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.466 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-10T10:04:50.466 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.466 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.466 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.466 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.466 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.466 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.466 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.466 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.466 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-10T10:04:50.466 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-10T10:04:50.466 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.466 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.467 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.467 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-10T10:04:50.467 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.467 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.467 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-10T10:04:50.467 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-10T10:04:50.467 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-10T10:04:50.467 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-10T10:04:50.467 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-10T10:04:50.467 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-10T10:04:50.467 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-10T10:04:50.467 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-10T10:04:50.467 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.467 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.467 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.468 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.468 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.468 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-10T10:04:50.468 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-10T10:04:50.468 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.468 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.468 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.468 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-10T10:04:50.468 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.468 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.468 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.468 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.468 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.468 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.468 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.469 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.469 INFO:teuthology.orchestra.run.smithi005.stdout: "min_size": { 2024-09-10T10:04:50.469 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "min_size", 2024-09-10T10:04:50.469 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-10T10:04:50.469 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.469 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.469 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-10T10:04:50.469 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.469 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.469 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.469 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "default min_size for any created local pool", 2024-09-10T10:04:50.469 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-10T10:04:50.469 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.469 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.469 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.469 INFO:teuthology.orchestra.run.smithi005.stdout: "num_rep": { 2024-09-10T10:04:50.470 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "num_rep", 2024-09-10T10:04:50.470 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-10T10:04:50.470 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.470 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.470 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "3", 2024-09-10T10:04:50.470 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.470 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.470 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.470 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "default replica count for any created local pool", 2024-09-10T10:04:50.470 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.470 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.470 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.470 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.470 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_num": { 2024-09-10T10:04:50.470 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "pg_num", 2024-09-10T10:04:50.471 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-10T10:04:50.471 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.471 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.471 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "128", 2024-09-10T10:04:50.471 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.471 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.471 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.471 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "default pg_num for any created local pool", 2024-09-10T10:04:50.471 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.471 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.472 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.472 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.472 INFO:teuthology.orchestra.run.smithi005.stdout: "prefix": { 2024-09-10T10:04:50.472 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "prefix", 2024-09-10T10:04:50.472 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.472 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.472 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.472 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-10T10:04:50.472 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.472 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.472 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.472 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "name prefix for any created local pool", 2024-09-10T10:04:50.472 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.472 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.473 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.473 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.473 INFO:teuthology.orchestra.run.smithi005.stdout: "subtree": { 2024-09-10T10:04:50.473 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "subtree", 2024-09-10T10:04:50.473 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.473 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.473 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.473 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "rack", 2024-09-10T10:04:50.473 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.473 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.473 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.473 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-10T10:04:50.473 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-10T10:04:50.473 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.474 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.474 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T10:04:50.474 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T10:04:50.474 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.474 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-10T10:04:50.474 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "mds_autoscaler", 2024-09-10T10:04:50.474 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-10T10:04:50.474 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-10T10:04:50.474 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-10T10:04:50.474 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-10T10:04:50.474 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-10T10:04:50.474 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.474 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.474 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.475 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-10T10:04:50.475 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.475 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.475 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-10T10:04:50.475 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-10T10:04:50.475 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-10T10:04:50.475 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-10T10:04:50.475 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-10T10:04:50.475 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-10T10:04:50.475 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-10T10:04:50.475 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-10T10:04:50.475 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.475 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.475 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.476 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.476 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.476 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-10T10:04:50.476 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-10T10:04:50.476 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.476 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.476 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.476 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-10T10:04:50.476 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.476 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.476 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.476 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.476 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.477 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.477 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.477 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.477 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-10T10:04:50.477 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-10T10:04:50.477 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.477 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.477 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.477 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-10T10:04:50.477 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.477 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.477 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-10T10:04:50.477 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-10T10:04:50.478 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-10T10:04:50.478 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-10T10:04:50.478 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-10T10:04:50.478 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-10T10:04:50.478 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-10T10:04:50.478 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-10T10:04:50.478 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.478 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.478 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.478 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.478 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.478 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-10T10:04:50.478 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-10T10:04:50.478 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.478 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.479 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.479 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-10T10:04:50.479 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.479 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.479 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.479 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.479 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.479 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.479 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.479 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T10:04:50.479 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T10:04:50.479 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.479 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-10T10:04:50.479 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "mirroring", 2024-09-10T10:04:50.479 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-10T10:04:50.480 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-10T10:04:50.480 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-10T10:04:50.480 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-10T10:04:50.480 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-10T10:04:50.480 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.480 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.480 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.480 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-10T10:04:50.480 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.480 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.480 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-10T10:04:50.480 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-10T10:04:50.480 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-10T10:04:50.480 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-10T10:04:50.480 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-10T10:04:50.481 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-10T10:04:50.481 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-10T10:04:50.481 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-10T10:04:50.481 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.481 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.481 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.481 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.481 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.481 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-10T10:04:50.481 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-10T10:04:50.481 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.481 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.481 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.481 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-10T10:04:50.481 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.481 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.482 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.482 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.482 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.482 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.482 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.482 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.482 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-10T10:04:50.482 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-10T10:04:50.482 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.482 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.482 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.482 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-10T10:04:50.482 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.482 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.482 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-10T10:04:50.483 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-10T10:04:50.483 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-10T10:04:50.483 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-10T10:04:50.483 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-10T10:04:50.483 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-10T10:04:50.483 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-10T10:04:50.483 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-10T10:04:50.483 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.483 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.483 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.483 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.483 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.483 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-10T10:04:50.483 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-10T10:04:50.483 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.484 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.484 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.484 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-10T10:04:50.484 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.484 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.484 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.484 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.484 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.484 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.484 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.484 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T10:04:50.484 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T10:04:50.484 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.484 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-10T10:04:50.484 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "nfs", 2024-09-10T10:04:50.485 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-10T10:04:50.485 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-10T10:04:50.485 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-10T10:04:50.485 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-10T10:04:50.485 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-10T10:04:50.485 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.485 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.485 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.485 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-10T10:04:50.485 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.485 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.485 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-10T10:04:50.485 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-10T10:04:50.485 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-10T10:04:50.486 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-10T10:04:50.486 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-10T10:04:50.486 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-10T10:04:50.486 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-10T10:04:50.486 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-10T10:04:50.486 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.486 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.486 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.486 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.486 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.486 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-10T10:04:50.486 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-10T10:04:50.487 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.487 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.487 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.487 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-10T10:04:50.487 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.487 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.487 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.487 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.487 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.487 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.487 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.487 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.487 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-10T10:04:50.488 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-10T10:04:50.488 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.488 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.488 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.488 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-10T10:04:50.488 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.488 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.488 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-10T10:04:50.488 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-10T10:04:50.488 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-10T10:04:50.488 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-10T10:04:50.488 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-10T10:04:50.488 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-10T10:04:50.489 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-10T10:04:50.489 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-10T10:04:50.489 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.489 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.489 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.489 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.489 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.489 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-10T10:04:50.489 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-10T10:04:50.489 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.489 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.489 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.489 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-10T10:04:50.489 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.490 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.490 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.490 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.490 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.490 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.490 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.490 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T10:04:50.490 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T10:04:50.490 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.490 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-10T10:04:50.490 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "orchestrator", 2024-09-10T10:04:50.490 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-10T10:04:50.490 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-10T10:04:50.491 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-10T10:04:50.491 INFO:teuthology.orchestra.run.smithi005.stdout: "fail_fs": { 2024-09-10T10:04:50.491 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "fail_fs", 2024-09-10T10:04:50.491 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.491 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.491 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.491 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-10T10:04:50.491 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.491 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.491 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.491 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-10T10:04:50.491 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.491 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.491 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.492 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.492 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-10T10:04:50.492 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-10T10:04:50.492 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.492 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.492 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.492 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-10T10:04:50.492 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.492 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.492 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-10T10:04:50.492 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-10T10:04:50.492 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-10T10:04:50.492 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-10T10:04:50.493 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-10T10:04:50.493 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-10T10:04:50.493 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-10T10:04:50.493 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-10T10:04:50.493 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.493 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.493 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.493 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.493 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.493 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-10T10:04:50.493 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-10T10:04:50.493 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.493 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.494 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.494 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-10T10:04:50.494 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.494 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.494 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.494 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.494 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.494 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.494 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.494 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.494 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-10T10:04:50.494 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-10T10:04:50.494 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.494 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.494 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.494 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-10T10:04:50.495 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.495 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.495 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-10T10:04:50.495 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-10T10:04:50.495 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-10T10:04:50.495 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-10T10:04:50.495 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-10T10:04:50.495 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-10T10:04:50.495 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-10T10:04:50.495 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-10T10:04:50.495 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.495 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.495 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.495 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.496 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.496 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-10T10:04:50.496 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-10T10:04:50.496 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.496 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.496 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.496 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-10T10:04:50.496 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.496 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.496 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.496 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.496 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.496 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.497 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.497 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.497 INFO:teuthology.orchestra.run.smithi005.stdout: "orchestrator": { 2024-09-10T10:04:50.497 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "orchestrator", 2024-09-10T10:04:50.497 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.497 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.497 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.497 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-10T10:04:50.497 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.497 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.497 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-10T10:04:50.497 INFO:teuthology.orchestra.run.smithi005.stdout: "cephadm", 2024-09-10T10:04:50.497 INFO:teuthology.orchestra.run.smithi005.stdout: "rook", 2024-09-10T10:04:50.497 INFO:teuthology.orchestra.run.smithi005.stdout: "test_orchestrator" 2024-09-10T10:04:50.498 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-10T10:04:50.498 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Orchestrator backend", 2024-09-10T10:04:50.498 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.498 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.498 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.498 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T10:04:50.498 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T10:04:50.498 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.498 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-10T10:04:50.498 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "osd_perf_query", 2024-09-10T10:04:50.498 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-10T10:04:50.498 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-10T10:04:50.498 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-10T10:04:50.499 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-10T10:04:50.499 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-10T10:04:50.499 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.499 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.499 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.499 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-10T10:04:50.499 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.499 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.499 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-10T10:04:50.499 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-10T10:04:50.499 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-10T10:04:50.499 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-10T10:04:50.499 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-10T10:04:50.499 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-10T10:04:50.499 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-10T10:04:50.500 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-10T10:04:50.500 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.500 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.500 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.500 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.500 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.500 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-10T10:04:50.500 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-10T10:04:50.500 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.500 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.500 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.500 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-10T10:04:50.500 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.500 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.501 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.501 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.501 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.501 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.501 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.501 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.501 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-10T10:04:50.501 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-10T10:04:50.501 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.501 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.501 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.501 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-10T10:04:50.501 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.501 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.502 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-10T10:04:50.502 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-10T10:04:50.502 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-10T10:04:50.502 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-10T10:04:50.502 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-10T10:04:50.502 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-10T10:04:50.502 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-10T10:04:50.502 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-10T10:04:50.502 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.502 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.502 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.502 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.502 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.502 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-10T10:04:50.502 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-10T10:04:50.503 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.503 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.503 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.503 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-10T10:04:50.503 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.503 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.503 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.503 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.503 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.503 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.503 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.503 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T10:04:50.503 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T10:04:50.503 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.503 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-10T10:04:50.504 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "osd_support", 2024-09-10T10:04:50.504 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-10T10:04:50.504 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-10T10:04:50.504 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-10T10:04:50.504 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-10T10:04:50.504 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-10T10:04:50.504 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.504 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.504 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.504 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-10T10:04:50.504 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.504 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.504 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-10T10:04:50.504 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-10T10:04:50.505 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-10T10:04:50.505 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-10T10:04:50.505 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-10T10:04:50.505 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-10T10:04:50.505 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-10T10:04:50.505 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-10T10:04:50.505 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.505 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.505 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.505 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.505 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.505 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-10T10:04:50.505 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-10T10:04:50.506 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.506 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.506 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.506 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-10T10:04:50.506 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.506 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.506 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.506 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.506 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.506 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.506 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.506 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.506 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-10T10:04:50.506 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-10T10:04:50.506 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.507 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.507 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.507 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-10T10:04:50.507 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.507 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.507 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-10T10:04:50.507 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-10T10:04:50.507 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-10T10:04:50.507 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-10T10:04:50.507 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-10T10:04:50.507 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-10T10:04:50.507 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-10T10:04:50.507 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-10T10:04:50.508 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.508 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.508 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.508 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.508 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.508 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-10T10:04:50.508 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-10T10:04:50.508 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.508 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.508 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.508 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-10T10:04:50.508 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.508 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.508 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.508 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.509 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.509 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.509 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.509 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T10:04:50.509 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T10:04:50.509 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.509 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-10T10:04:50.509 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "pg_autoscaler", 2024-09-10T10:04:50.509 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-10T10:04:50.509 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-10T10:04:50.509 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-10T10:04:50.509 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-10T10:04:50.509 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-10T10:04:50.509 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.509 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.510 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.510 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-10T10:04:50.510 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.510 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.510 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-10T10:04:50.510 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-10T10:04:50.510 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-10T10:04:50.510 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-10T10:04:50.510 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-10T10:04:50.510 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-10T10:04:50.510 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-10T10:04:50.510 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-10T10:04:50.510 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.510 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.511 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.511 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.511 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.511 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-10T10:04:50.511 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-10T10:04:50.511 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.511 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.511 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.511 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-10T10:04:50.511 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.511 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.511 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.511 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.511 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.512 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.512 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.512 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.512 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-10T10:04:50.512 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-10T10:04:50.512 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.512 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.512 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.512 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-10T10:04:50.512 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.512 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.512 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-10T10:04:50.512 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-10T10:04:50.512 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-10T10:04:50.512 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-10T10:04:50.513 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-10T10:04:50.513 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-10T10:04:50.513 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-10T10:04:50.513 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-10T10:04:50.513 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.513 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.513 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.513 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.513 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.513 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-10T10:04:50.513 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-10T10:04:50.513 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.514 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.514 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.514 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-10T10:04:50.514 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.514 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.514 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.514 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.514 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.514 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.514 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.514 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.514 INFO:teuthology.orchestra.run.smithi005.stdout: "sleep_interval": { 2024-09-10T10:04:50.514 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sleep_interval", 2024-09-10T10:04:50.515 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-10T10:04:50.515 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.515 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.515 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "60", 2024-09-10T10:04:50.515 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.515 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.515 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.515 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.515 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.515 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.515 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.515 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.515 INFO:teuthology.orchestra.run.smithi005.stdout: "threshold": { 2024-09-10T10:04:50.515 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "threshold", 2024-09-10T10:04:50.516 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "float", 2024-09-10T10:04:50.516 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.516 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.516 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "3.0", 2024-09-10T10:04:50.516 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "1.0", 2024-09-10T10:04:50.516 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.516 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.516 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "scaling threshold", 2024-09-10T10:04:50.516 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-10T10:04:50.516 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.516 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.516 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T10:04:50.516 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T10:04:50.517 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.517 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-10T10:04:50.517 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "progress", 2024-09-10T10:04:50.517 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-10T10:04:50.517 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-10T10:04:50.517 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-10T10:04:50.517 INFO:teuthology.orchestra.run.smithi005.stdout: "allow_pg_recovery_event": { 2024-09-10T10:04:50.517 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "allow_pg_recovery_event", 2024-09-10T10:04:50.517 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.517 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.517 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.517 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-10T10:04:50.517 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.518 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.518 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.518 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-10T10:04:50.518 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.518 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.518 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.518 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.518 INFO:teuthology.orchestra.run.smithi005.stdout: "enabled": { 2024-09-10T10:04:50.518 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "enabled", 2024-09-10T10:04:50.518 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.518 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.518 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.518 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-10T10:04:50.518 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.519 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.519 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.519 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.519 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.519 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.519 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.519 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.519 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-10T10:04:50.519 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-10T10:04:50.519 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.519 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.519 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.519 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-10T10:04:50.520 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.520 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.520 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-10T10:04:50.520 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-10T10:04:50.520 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-10T10:04:50.520 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-10T10:04:50.520 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-10T10:04:50.520 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-10T10:04:50.520 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-10T10:04:50.520 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-10T10:04:50.520 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.520 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.520 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.520 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.521 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.521 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-10T10:04:50.521 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-10T10:04:50.521 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.521 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.521 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.521 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-10T10:04:50.521 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.521 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.521 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.521 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.521 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.521 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.521 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.522 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.522 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-10T10:04:50.522 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-10T10:04:50.522 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.522 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.522 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.522 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-10T10:04:50.522 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.522 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.522 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-10T10:04:50.522 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-10T10:04:50.522 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-10T10:04:50.522 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-10T10:04:50.522 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-10T10:04:50.522 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-10T10:04:50.523 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-10T10:04:50.523 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-10T10:04:50.523 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.523 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.523 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.523 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.523 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.523 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-10T10:04:50.523 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-10T10:04:50.523 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.523 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.523 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.523 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-10T10:04:50.523 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.523 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.524 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.524 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.524 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.524 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.524 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.524 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.524 INFO:teuthology.orchestra.run.smithi005.stdout: "max_completed_events": { 2024-09-10T10:04:50.524 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "max_completed_events", 2024-09-10T10:04:50.524 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-10T10:04:50.524 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.524 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.524 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "50", 2024-09-10T10:04:50.524 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.524 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.525 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.525 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "number of past completed events to remember", 2024-09-10T10:04:50.525 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.525 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.525 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.525 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.525 INFO:teuthology.orchestra.run.smithi005.stdout: "sleep_interval": { 2024-09-10T10:04:50.525 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sleep_interval", 2024-09-10T10:04:50.525 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-10T10:04:50.525 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.525 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.525 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "5", 2024-09-10T10:04:50.525 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.525 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.526 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.526 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "how long the module is going to sleep", 2024-09-10T10:04:50.526 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.526 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.526 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.526 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T10:04:50.526 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T10:04:50.526 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.526 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-10T10:04:50.526 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "prometheus", 2024-09-10T10:04:50.526 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-10T10:04:50.526 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-10T10:04:50.526 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-10T10:04:50.526 INFO:teuthology.orchestra.run.smithi005.stdout: "cache": { 2024-09-10T10:04:50.526 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "cache", 2024-09-10T10:04:50.527 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.527 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.527 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.527 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-10T10:04:50.527 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.527 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.527 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.527 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.527 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.527 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.527 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.527 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.527 INFO:teuthology.orchestra.run.smithi005.stdout: "exclude_perf_counters": { 2024-09-10T10:04:50.528 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "exclude_perf_counters", 2024-09-10T10:04:50.528 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.528 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.528 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.528 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-10T10:04:50.528 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.528 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.528 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.528 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-10T10:04:50.528 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-10T10:04:50.528 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.528 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.528 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.529 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-10T10:04:50.529 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-10T10:04:50.529 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.529 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.529 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.529 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-10T10:04:50.529 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.529 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.529 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-10T10:04:50.529 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-10T10:04:50.529 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-10T10:04:50.529 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-10T10:04:50.529 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-10T10:04:50.530 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-10T10:04:50.530 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-10T10:04:50.530 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-10T10:04:50.530 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.530 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.530 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.530 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.530 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.530 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-10T10:04:50.530 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-10T10:04:50.530 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.530 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.530 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.531 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-10T10:04:50.531 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.531 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.531 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.531 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.531 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.531 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.531 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.531 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.531 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-10T10:04:50.531 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-10T10:04:50.531 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.531 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.532 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.532 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-10T10:04:50.532 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.532 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.532 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-10T10:04:50.532 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-10T10:04:50.532 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-10T10:04:50.532 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-10T10:04:50.532 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-10T10:04:50.532 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-10T10:04:50.532 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-10T10:04:50.532 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-10T10:04:50.532 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.533 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.533 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.533 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.533 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.533 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-10T10:04:50.533 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-10T10:04:50.533 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.533 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.533 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.533 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-10T10:04:50.533 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.533 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.533 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.533 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.534 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.534 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.534 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.534 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.534 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_stats_pools": { 2024-09-10T10:04:50.534 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rbd_stats_pools", 2024-09-10T10:04:50.534 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.534 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.534 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.534 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-10T10:04:50.534 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.534 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.534 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.535 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.535 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.535 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.535 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.535 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.535 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-10T10:04:50.535 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-10T10:04:50.535 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-10T10:04:50.535 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.535 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.535 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "300", 2024-09-10T10:04:50.535 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.535 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.535 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.535 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.536 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.536 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.536 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.536 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.536 INFO:teuthology.orchestra.run.smithi005.stdout: "scrape_interval": { 2024-09-10T10:04:50.536 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "scrape_interval", 2024-09-10T10:04:50.536 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "float", 2024-09-10T10:04:50.536 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.536 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.536 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "15.0", 2024-09-10T10:04:50.536 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.536 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.536 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.537 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.537 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.537 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.537 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.537 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.537 INFO:teuthology.orchestra.run.smithi005.stdout: "server_addr": { 2024-09-10T10:04:50.537 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "server_addr", 2024-09-10T10:04:50.537 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.537 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.537 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.537 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "::", 2024-09-10T10:04:50.537 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.537 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.538 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.538 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-10T10:04:50.538 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.538 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.538 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.538 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.538 INFO:teuthology.orchestra.run.smithi005.stdout: "server_port": { 2024-09-10T10:04:50.538 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "server_port", 2024-09-10T10:04:50.538 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-10T10:04:50.538 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.538 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.538 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "9283", 2024-09-10T10:04:50.538 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.538 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.539 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.539 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-10T10:04:50.539 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.539 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.539 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.539 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.539 INFO:teuthology.orchestra.run.smithi005.stdout: "stale_cache_strategy": { 2024-09-10T10:04:50.539 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "stale_cache_strategy", 2024-09-10T10:04:50.539 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.539 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.539 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.539 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "log", 2024-09-10T10:04:50.539 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.539 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.539 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.540 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.540 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.540 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.540 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.540 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.540 INFO:teuthology.orchestra.run.smithi005.stdout: "standby_behaviour": { 2024-09-10T10:04:50.540 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "standby_behaviour", 2024-09-10T10:04:50.540 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.540 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.540 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.540 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "default", 2024-09-10T10:04:50.540 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.540 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.541 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-10T10:04:50.541 INFO:teuthology.orchestra.run.smithi005.stdout: "default", 2024-09-10T10:04:50.541 INFO:teuthology.orchestra.run.smithi005.stdout: "error" 2024-09-10T10:04:50.541 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-10T10:04:50.541 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.541 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.541 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.541 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.541 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.541 INFO:teuthology.orchestra.run.smithi005.stdout: "standby_error_status_code": { 2024-09-10T10:04:50.541 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "standby_error_status_code", 2024-09-10T10:04:50.541 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-10T10:04:50.541 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.542 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.542 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "500", 2024-09-10T10:04:50.542 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "400", 2024-09-10T10:04:50.542 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "599", 2024-09-10T10:04:50.542 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.542 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.542 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.542 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.542 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.542 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T10:04:50.542 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T10:04:50.542 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.542 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-10T10:04:50.543 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rbd_support", 2024-09-10T10:04:50.543 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-10T10:04:50.543 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-10T10:04:50.543 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-10T10:04:50.543 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-10T10:04:50.543 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-10T10:04:50.543 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.543 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.543 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.543 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-10T10:04:50.543 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.543 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.543 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-10T10:04:50.543 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-10T10:04:50.544 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-10T10:04:50.544 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-10T10:04:50.544 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-10T10:04:50.544 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-10T10:04:50.544 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-10T10:04:50.544 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-10T10:04:50.544 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.544 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.544 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.544 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.544 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.544 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-10T10:04:50.544 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-10T10:04:50.544 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.544 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.545 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.545 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-10T10:04:50.545 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.545 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.545 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.545 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.545 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.545 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.545 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.545 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.545 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-10T10:04:50.545 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-10T10:04:50.545 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.545 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.546 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.546 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-10T10:04:50.546 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.546 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.546 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-10T10:04:50.546 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-10T10:04:50.546 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-10T10:04:50.546 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-10T10:04:50.546 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-10T10:04:50.546 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-10T10:04:50.546 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-10T10:04:50.546 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-10T10:04:50.546 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.547 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.547 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.547 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.547 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.547 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-10T10:04:50.547 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-10T10:04:50.547 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.547 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.547 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.547 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-10T10:04:50.547 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.547 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.547 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.548 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.548 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.548 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.548 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.548 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.548 INFO:teuthology.orchestra.run.smithi005.stdout: "max_concurrent_snap_create": { 2024-09-10T10:04:50.548 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "max_concurrent_snap_create", 2024-09-10T10:04:50.548 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-10T10:04:50.548 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.548 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.548 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10", 2024-09-10T10:04:50.548 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.548 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.549 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.549 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.549 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.549 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.549 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.549 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.549 INFO:teuthology.orchestra.run.smithi005.stdout: "mirror_snapshot_schedule": { 2024-09-10T10:04:50.549 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "mirror_snapshot_schedule", 2024-09-10T10:04:50.549 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.549 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.549 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.549 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-10T10:04:50.549 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.549 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.550 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.550 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.550 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.550 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.550 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.550 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.550 INFO:teuthology.orchestra.run.smithi005.stdout: "trash_purge_schedule": { 2024-09-10T10:04:50.550 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "trash_purge_schedule", 2024-09-10T10:04:50.550 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.550 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.550 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.550 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-10T10:04:50.550 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.551 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.551 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.551 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.551 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.551 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.551 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.551 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T10:04:50.551 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T10:04:50.551 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.551 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-10T10:04:50.551 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "restful", 2024-09-10T10:04:50.551 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-10T10:04:50.551 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-10T10:04:50.551 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-10T10:04:50.552 INFO:teuthology.orchestra.run.smithi005.stdout: "enable_auth": { 2024-09-10T10:04:50.552 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "enable_auth", 2024-09-10T10:04:50.552 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.552 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.552 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.552 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-10T10:04:50.552 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.552 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.552 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.552 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.552 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.552 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.552 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.553 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.553 INFO:teuthology.orchestra.run.smithi005.stdout: "key_file": { 2024-09-10T10:04:50.553 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "key_file", 2024-09-10T10:04:50.553 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.553 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.553 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.553 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-10T10:04:50.553 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.553 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.553 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.553 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.553 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.553 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.553 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.554 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.554 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-10T10:04:50.554 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-10T10:04:50.554 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.554 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.554 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.554 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-10T10:04:50.554 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.554 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.554 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-10T10:04:50.554 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-10T10:04:50.554 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-10T10:04:50.554 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-10T10:04:50.554 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-10T10:04:50.555 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-10T10:04:50.555 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-10T10:04:50.555 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-10T10:04:50.555 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.555 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.555 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.555 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.555 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.555 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-10T10:04:50.555 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-10T10:04:50.555 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.555 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.555 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.556 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-10T10:04:50.556 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.556 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.556 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.556 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.556 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.556 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.556 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.556 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.556 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-10T10:04:50.556 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-10T10:04:50.556 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.556 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.557 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.557 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-10T10:04:50.557 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.557 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.557 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-10T10:04:50.557 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-10T10:04:50.557 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-10T10:04:50.557 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-10T10:04:50.557 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-10T10:04:50.557 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-10T10:04:50.557 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-10T10:04:50.557 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-10T10:04:50.557 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.557 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.558 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.558 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.558 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.558 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-10T10:04:50.558 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-10T10:04:50.558 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.558 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.558 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.558 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-10T10:04:50.558 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.558 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.558 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.558 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.558 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.559 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.559 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.559 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.559 INFO:teuthology.orchestra.run.smithi005.stdout: "server_addr": { 2024-09-10T10:04:50.559 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "server_addr", 2024-09-10T10:04:50.559 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.559 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.559 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.559 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-10T10:04:50.559 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.559 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.559 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.559 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.559 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.560 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.560 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.560 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.560 INFO:teuthology.orchestra.run.smithi005.stdout: "server_port": { 2024-09-10T10:04:50.560 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "server_port", 2024-09-10T10:04:50.560 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.560 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.560 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.560 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-10T10:04:50.560 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.560 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.560 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.560 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.560 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.561 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.561 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.561 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T10:04:50.561 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T10:04:50.561 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.561 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-10T10:04:50.561 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rgw", 2024-09-10T10:04:50.561 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-10T10:04:50.561 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-10T10:04:50.561 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-10T10:04:50.561 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-10T10:04:50.561 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-10T10:04:50.561 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.562 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.562 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.562 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-10T10:04:50.562 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.562 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.562 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-10T10:04:50.562 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-10T10:04:50.562 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-10T10:04:50.562 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-10T10:04:50.562 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-10T10:04:50.562 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-10T10:04:50.562 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-10T10:04:50.562 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-10T10:04:50.562 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.563 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.563 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.563 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.563 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.563 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-10T10:04:50.563 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-10T10:04:50.563 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.563 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.563 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.563 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-10T10:04:50.563 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.563 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.563 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.564 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.564 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.564 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.564 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.564 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.564 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-10T10:04:50.564 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-10T10:04:50.564 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.564 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.564 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.564 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-10T10:04:50.564 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.564 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.564 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-10T10:04:50.565 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-10T10:04:50.565 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-10T10:04:50.565 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-10T10:04:50.565 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-10T10:04:50.565 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-10T10:04:50.565 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-10T10:04:50.565 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-10T10:04:50.565 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.565 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.565 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.565 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.565 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.565 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-10T10:04:50.565 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-10T10:04:50.566 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.566 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.566 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.566 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-10T10:04:50.566 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.566 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.566 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.566 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.566 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.566 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.566 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.566 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T10:04:50.566 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T10:04:50.567 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.567 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-10T10:04:50.567 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rook", 2024-09-10T10:04:50.567 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-10T10:04:50.567 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-10T10:04:50.567 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-10T10:04:50.567 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-10T10:04:50.567 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-10T10:04:50.567 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.567 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.567 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.567 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-10T10:04:50.568 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.568 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.568 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-10T10:04:50.568 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-10T10:04:50.568 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-10T10:04:50.568 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-10T10:04:50.568 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-10T10:04:50.568 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-10T10:04:50.568 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-10T10:04:50.568 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-10T10:04:50.568 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.568 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.568 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.568 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.568 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.569 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-10T10:04:50.569 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-10T10:04:50.569 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.569 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.569 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.569 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-10T10:04:50.569 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.569 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.569 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.569 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.569 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.569 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.569 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.570 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.570 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-10T10:04:50.570 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-10T10:04:50.570 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.570 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.570 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.570 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-10T10:04:50.570 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.570 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.570 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-10T10:04:50.570 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-10T10:04:50.570 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-10T10:04:50.570 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-10T10:04:50.570 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-10T10:04:50.570 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-10T10:04:50.571 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-10T10:04:50.571 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-10T10:04:50.571 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.571 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.571 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.571 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.571 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.571 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-10T10:04:50.571 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-10T10:04:50.571 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.571 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.571 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.571 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-10T10:04:50.571 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.571 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.571 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.572 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.572 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.572 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.572 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.572 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.572 INFO:teuthology.orchestra.run.smithi005.stdout: "storage_class": { 2024-09-10T10:04:50.572 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "storage_class", 2024-09-10T10:04:50.572 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.572 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.572 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.572 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "local", 2024-09-10T10:04:50.572 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.572 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.572 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.573 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-10T10:04:50.573 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.573 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.573 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.573 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T10:04:50.573 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T10:04:50.573 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.573 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-10T10:04:50.573 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "selftest", 2024-09-10T10:04:50.573 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-10T10:04:50.573 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-10T10:04:50.573 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-10T10:04:50.573 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-10T10:04:50.573 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-10T10:04:50.573 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.574 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.574 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.574 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-10T10:04:50.574 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.574 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.574 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-10T10:04:50.574 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-10T10:04:50.574 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-10T10:04:50.574 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-10T10:04:50.574 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-10T10:04:50.574 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-10T10:04:50.574 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-10T10:04:50.574 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-10T10:04:50.574 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.574 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.575 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.575 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.575 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.575 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-10T10:04:50.575 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-10T10:04:50.575 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.575 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.575 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.575 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-10T10:04:50.575 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.575 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.575 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.575 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.575 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.575 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.575 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.576 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.576 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-10T10:04:50.576 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-10T10:04:50.576 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.576 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.576 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.576 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-10T10:04:50.576 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.576 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.576 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-10T10:04:50.576 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-10T10:04:50.576 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-10T10:04:50.576 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-10T10:04:50.576 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-10T10:04:50.577 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-10T10:04:50.577 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-10T10:04:50.577 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-10T10:04:50.577 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.577 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.577 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.577 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.577 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.577 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-10T10:04:50.577 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-10T10:04:50.577 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.577 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.577 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.578 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-10T10:04:50.578 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.578 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.578 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.578 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.578 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.578 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.578 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.578 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.578 INFO:teuthology.orchestra.run.smithi005.stdout: "roption1": { 2024-09-10T10:04:50.578 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "roption1", 2024-09-10T10:04:50.578 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.578 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.578 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.578 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-10T10:04:50.579 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.579 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.579 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.579 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.579 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.579 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.579 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.579 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.579 INFO:teuthology.orchestra.run.smithi005.stdout: "roption2": { 2024-09-10T10:04:50.579 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "roption2", 2024-09-10T10:04:50.579 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.579 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.579 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.579 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "xyz", 2024-09-10T10:04:50.579 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.580 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.580 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.580 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.580 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.580 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.580 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.580 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.580 INFO:teuthology.orchestra.run.smithi005.stdout: "rwoption1": { 2024-09-10T10:04:50.580 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rwoption1", 2024-09-10T10:04:50.580 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.580 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.580 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.580 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-10T10:04:50.580 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.581 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.581 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.581 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.581 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.581 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.581 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.581 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.581 INFO:teuthology.orchestra.run.smithi005.stdout: "rwoption2": { 2024-09-10T10:04:50.581 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rwoption2", 2024-09-10T10:04:50.581 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-10T10:04:50.581 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.581 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.581 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-10T10:04:50.581 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.581 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.582 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.582 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.582 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.582 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.582 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.582 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.582 INFO:teuthology.orchestra.run.smithi005.stdout: "rwoption3": { 2024-09-10T10:04:50.582 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rwoption3", 2024-09-10T10:04:50.582 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "float", 2024-09-10T10:04:50.582 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.582 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.582 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-10T10:04:50.582 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.582 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.582 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.583 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.583 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.583 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.583 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.583 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.583 INFO:teuthology.orchestra.run.smithi005.stdout: "rwoption4": { 2024-09-10T10:04:50.583 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rwoption4", 2024-09-10T10:04:50.583 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.583 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.583 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.583 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-10T10:04:50.583 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.583 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.583 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.583 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.584 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.584 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.584 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.584 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.584 INFO:teuthology.orchestra.run.smithi005.stdout: "rwoption5": { 2024-09-10T10:04:50.584 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rwoption5", 2024-09-10T10:04:50.584 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.584 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.584 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.584 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-10T10:04:50.584 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.584 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.584 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.584 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.584 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.585 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.585 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.585 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.585 INFO:teuthology.orchestra.run.smithi005.stdout: "rwoption6": { 2024-09-10T10:04:50.585 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rwoption6", 2024-09-10T10:04:50.585 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.585 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.585 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.585 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-10T10:04:50.585 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.585 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.585 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.585 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.585 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.585 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.586 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.586 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.586 INFO:teuthology.orchestra.run.smithi005.stdout: "rwoption7": { 2024-09-10T10:04:50.586 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rwoption7", 2024-09-10T10:04:50.586 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-10T10:04:50.586 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.586 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.586 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-10T10:04:50.586 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "1", 2024-09-10T10:04:50.586 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "42", 2024-09-10T10:04:50.586 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.586 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.586 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.586 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.586 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.587 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.587 INFO:teuthology.orchestra.run.smithi005.stdout: "testkey": { 2024-09-10T10:04:50.587 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "testkey", 2024-09-10T10:04:50.587 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.587 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.587 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.587 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-10T10:04:50.587 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.587 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.587 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.587 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.587 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.587 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.588 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.588 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.588 INFO:teuthology.orchestra.run.smithi005.stdout: "testlkey": { 2024-09-10T10:04:50.588 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "testlkey", 2024-09-10T10:04:50.588 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.588 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.588 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.588 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-10T10:04:50.588 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.588 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.588 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.588 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.588 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.588 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.589 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.589 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.589 INFO:teuthology.orchestra.run.smithi005.stdout: "testnewline": { 2024-09-10T10:04:50.589 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "testnewline", 2024-09-10T10:04:50.589 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.589 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.589 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.589 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-10T10:04:50.589 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.589 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.589 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.589 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.589 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.589 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.589 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.590 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T10:04:50.590 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T10:04:50.590 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.590 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-10T10:04:50.590 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "snap_schedule", 2024-09-10T10:04:50.590 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-10T10:04:50.590 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-10T10:04:50.590 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-10T10:04:50.590 INFO:teuthology.orchestra.run.smithi005.stdout: "allow_m_granularity": { 2024-09-10T10:04:50.590 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "allow_m_granularity", 2024-09-10T10:04:50.590 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.590 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.590 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.590 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-10T10:04:50.590 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.591 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.591 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.591 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "allow minute scheduled snapshots", 2024-09-10T10:04:50.591 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.591 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.591 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.591 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.591 INFO:teuthology.orchestra.run.smithi005.stdout: "dump_on_update": { 2024-09-10T10:04:50.591 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "dump_on_update", 2024-09-10T10:04:50.591 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.591 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.591 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.591 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-10T10:04:50.591 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.592 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.592 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.592 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "dump database to debug log on update", 2024-09-10T10:04:50.592 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.592 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.592 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.592 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.592 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-10T10:04:50.592 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-10T10:04:50.592 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.592 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.592 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.592 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-10T10:04:50.592 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.592 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.593 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-10T10:04:50.593 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-10T10:04:50.593 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-10T10:04:50.593 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-10T10:04:50.593 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-10T10:04:50.593 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-10T10:04:50.593 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-10T10:04:50.593 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-10T10:04:50.593 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.593 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.593 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.593 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.593 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.594 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-10T10:04:50.594 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-10T10:04:50.594 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.594 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.594 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.594 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-10T10:04:50.594 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.594 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.594 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.594 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.594 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.594 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.594 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.594 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.594 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-10T10:04:50.595 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-10T10:04:50.595 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.595 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.595 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.595 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-10T10:04:50.595 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.595 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.595 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-10T10:04:50.595 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-10T10:04:50.595 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-10T10:04:50.595 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-10T10:04:50.595 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-10T10:04:50.595 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-10T10:04:50.595 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-10T10:04:50.595 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-10T10:04:50.596 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.596 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.596 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.596 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.596 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.596 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-10T10:04:50.596 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-10T10:04:50.596 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.596 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.596 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.596 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-10T10:04:50.596 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.596 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.596 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.597 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.597 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.597 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.597 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.597 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T10:04:50.597 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T10:04:50.597 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.597 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-10T10:04:50.597 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "stats", 2024-09-10T10:04:50.597 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-10T10:04:50.597 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-10T10:04:50.597 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-10T10:04:50.597 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-10T10:04:50.597 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-10T10:04:50.598 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.598 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.598 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.598 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-10T10:04:50.598 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.598 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.598 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-10T10:04:50.598 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-10T10:04:50.598 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-10T10:04:50.598 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-10T10:04:50.598 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-10T10:04:50.598 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-10T10:04:50.598 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-10T10:04:50.598 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-10T10:04:50.599 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.599 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.599 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.599 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.599 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.599 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-10T10:04:50.599 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-10T10:04:50.599 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.599 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.599 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.599 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-10T10:04:50.599 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.599 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.599 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.600 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.600 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.600 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.600 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.600 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.600 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-10T10:04:50.600 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-10T10:04:50.600 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.600 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.600 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.600 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-10T10:04:50.600 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.600 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.600 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-10T10:04:50.600 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-10T10:04:50.601 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-10T10:04:50.601 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-10T10:04:50.601 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-10T10:04:50.601 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-10T10:04:50.601 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-10T10:04:50.601 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-10T10:04:50.601 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.601 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.601 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.601 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.601 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.601 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-10T10:04:50.601 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-10T10:04:50.601 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.602 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.602 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.602 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-10T10:04:50.602 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.602 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.602 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.602 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.602 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.602 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.602 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.602 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T10:04:50.602 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T10:04:50.602 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.602 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-10T10:04:50.603 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "status", 2024-09-10T10:04:50.603 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-10T10:04:50.603 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-10T10:04:50.603 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-10T10:04:50.603 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-10T10:04:50.603 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-10T10:04:50.603 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.603 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.603 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.603 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-10T10:04:50.603 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.603 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.603 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-10T10:04:50.603 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-10T10:04:50.603 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-10T10:04:50.604 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-10T10:04:50.604 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-10T10:04:50.604 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-10T10:04:50.604 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-10T10:04:50.604 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-10T10:04:50.604 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.604 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.604 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.604 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.604 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.604 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-10T10:04:50.604 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-10T10:04:50.604 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.605 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.605 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.605 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-10T10:04:50.605 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.605 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.605 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.605 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.605 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.605 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.605 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.605 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.605 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-10T10:04:50.605 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-10T10:04:50.605 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.606 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.606 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.606 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-10T10:04:50.606 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.606 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.606 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-10T10:04:50.606 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-10T10:04:50.606 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-10T10:04:50.606 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-10T10:04:50.606 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-10T10:04:50.606 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-10T10:04:50.606 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-10T10:04:50.606 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-10T10:04:50.607 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.607 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.607 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.607 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.607 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.607 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-10T10:04:50.607 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-10T10:04:50.607 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.607 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.607 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.607 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-10T10:04:50.607 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.607 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.607 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.608 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.608 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.608 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.608 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.608 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T10:04:50.608 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T10:04:50.608 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.608 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-10T10:04:50.608 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "telegraf", 2024-09-10T10:04:50.608 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-10T10:04:50.608 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-10T10:04:50.608 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-10T10:04:50.608 INFO:teuthology.orchestra.run.smithi005.stdout: "address": { 2024-09-10T10:04:50.609 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "address", 2024-09-10T10:04:50.609 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.609 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.609 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.609 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-10T10:04:50.609 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.609 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.609 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.609 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.609 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.609 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.609 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.609 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.609 INFO:teuthology.orchestra.run.smithi005.stdout: "interval": { 2024-09-10T10:04:50.610 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "interval", 2024-09-10T10:04:50.610 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-10T10:04:50.610 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.610 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.610 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "15", 2024-09-10T10:04:50.610 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.610 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.610 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.610 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.610 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.610 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.610 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.610 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.610 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-10T10:04:50.611 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-10T10:04:50.611 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.611 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.611 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.611 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-10T10:04:50.611 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.611 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.611 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-10T10:04:50.611 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-10T10:04:50.611 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-10T10:04:50.611 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-10T10:04:50.611 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-10T10:04:50.611 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-10T10:04:50.612 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-10T10:04:50.612 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-10T10:04:50.612 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.612 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.612 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.612 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.612 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.612 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-10T10:04:50.612 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-10T10:04:50.612 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.612 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.612 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.612 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-10T10:04:50.613 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.613 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.613 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.613 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.613 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.613 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.613 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.613 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.613 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-10T10:04:50.613 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-10T10:04:50.613 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.613 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.613 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.613 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-10T10:04:50.614 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.614 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.614 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-10T10:04:50.614 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-10T10:04:50.614 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-10T10:04:50.614 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-10T10:04:50.614 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-10T10:04:50.614 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-10T10:04:50.614 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-10T10:04:50.614 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-10T10:04:50.614 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.614 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.614 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.614 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.614 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.615 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-10T10:04:50.615 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-10T10:04:50.615 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.615 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.615 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.615 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-10T10:04:50.615 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.615 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.615 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.615 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.615 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.615 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.615 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.615 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T10:04:50.616 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T10:04:50.616 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.616 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-10T10:04:50.616 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "telemetry", 2024-09-10T10:04:50.616 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-10T10:04:50.616 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-10T10:04:50.616 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-10T10:04:50.616 INFO:teuthology.orchestra.run.smithi005.stdout: "channel_basic": { 2024-09-10T10:04:50.616 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "channel_basic", 2024-09-10T10:04:50.616 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.616 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.616 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.616 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-10T10:04:50.616 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.617 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.617 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.617 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-10T10:04:50.617 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.617 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.617 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.617 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.617 INFO:teuthology.orchestra.run.smithi005.stdout: "channel_crash": { 2024-09-10T10:04:50.617 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "channel_crash", 2024-09-10T10:04:50.617 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.617 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.617 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.617 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-10T10:04:50.617 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.618 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.618 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.618 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-10T10:04:50.618 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.618 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.618 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.618 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.618 INFO:teuthology.orchestra.run.smithi005.stdout: "channel_device": { 2024-09-10T10:04:50.618 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "channel_device", 2024-09-10T10:04:50.618 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.618 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.618 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.618 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-10T10:04:50.618 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.619 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.619 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.619 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-10T10:04:50.619 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.619 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.619 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.619 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.619 INFO:teuthology.orchestra.run.smithi005.stdout: "channel_ident": { 2024-09-10T10:04:50.619 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "channel_ident", 2024-09-10T10:04:50.619 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.619 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.619 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.619 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-10T10:04:50.619 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.619 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.619 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.620 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-10T10:04:50.620 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.620 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.620 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.620 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.620 INFO:teuthology.orchestra.run.smithi005.stdout: "channel_perf": { 2024-09-10T10:04:50.620 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "channel_perf", 2024-09-10T10:04:50.620 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.620 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.620 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.620 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-10T10:04:50.620 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.620 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.620 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.620 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-10T10:04:50.621 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.621 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.621 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.621 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.621 INFO:teuthology.orchestra.run.smithi005.stdout: "contact": { 2024-09-10T10:04:50.621 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "contact", 2024-09-10T10:04:50.621 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.621 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.621 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.621 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-10T10:04:50.621 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.621 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.621 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.621 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.622 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.622 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.622 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.622 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.622 INFO:teuthology.orchestra.run.smithi005.stdout: "description": { 2024-09-10T10:04:50.622 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "description", 2024-09-10T10:04:50.622 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.622 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.622 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.622 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-10T10:04:50.622 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.622 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.622 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.622 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.622 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.623 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.623 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.623 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.623 INFO:teuthology.orchestra.run.smithi005.stdout: "device_url": { 2024-09-10T10:04:50.623 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "device_url", 2024-09-10T10:04:50.623 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.623 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.623 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.623 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-10T10:04:50.623 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.623 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.623 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.623 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.623 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.623 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.624 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.624 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.624 INFO:teuthology.orchestra.run.smithi005.stdout: "enabled": { 2024-09-10T10:04:50.624 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "enabled", 2024-09-10T10:04:50.624 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.624 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.624 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.624 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-10T10:04:50.624 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.624 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.624 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.624 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.624 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.624 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.625 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.625 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.625 INFO:teuthology.orchestra.run.smithi005.stdout: "interval": { 2024-09-10T10:04:50.625 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "interval", 2024-09-10T10:04:50.625 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-10T10:04:50.625 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.625 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.625 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "24", 2024-09-10T10:04:50.625 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "8", 2024-09-10T10:04:50.625 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.625 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.625 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.625 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.625 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.625 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.626 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.626 INFO:teuthology.orchestra.run.smithi005.stdout: "last_opt_revision": { 2024-09-10T10:04:50.626 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "last_opt_revision", 2024-09-10T10:04:50.626 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-10T10:04:50.626 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.626 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.626 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "1", 2024-09-10T10:04:50.626 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.626 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.626 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.626 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.626 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.626 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.626 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.626 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.627 INFO:teuthology.orchestra.run.smithi005.stdout: "leaderboard": { 2024-09-10T10:04:50.627 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "leaderboard", 2024-09-10T10:04:50.627 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.627 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.627 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.627 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-10T10:04:50.627 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.627 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.627 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.627 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.627 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.627 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.627 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.628 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.628 INFO:teuthology.orchestra.run.smithi005.stdout: "leaderboard_description": { 2024-09-10T10:04:50.628 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "leaderboard_description", 2024-09-10T10:04:50.628 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.628 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.628 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.628 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-10T10:04:50.628 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.628 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.628 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.628 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.628 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.628 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.628 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.629 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.629 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-10T10:04:50.629 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-10T10:04:50.629 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.629 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.629 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.629 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-10T10:04:50.629 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.629 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.629 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-10T10:04:50.629 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-10T10:04:50.629 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-10T10:04:50.629 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-10T10:04:50.629 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-10T10:04:50.629 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-10T10:04:50.630 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-10T10:04:50.630 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-10T10:04:50.630 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.630 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.630 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.630 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.630 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.630 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-10T10:04:50.630 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-10T10:04:50.630 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.630 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.630 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.630 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-10T10:04:50.630 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.630 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.631 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.631 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.631 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.631 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.631 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.631 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.631 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-10T10:04:50.631 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-10T10:04:50.631 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.631 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.631 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.631 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-10T10:04:50.631 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.631 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.631 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-10T10:04:50.631 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-10T10:04:50.632 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-10T10:04:50.632 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-10T10:04:50.632 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-10T10:04:50.632 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-10T10:04:50.632 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-10T10:04:50.632 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-10T10:04:50.632 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.632 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.632 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.632 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.632 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.632 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-10T10:04:50.632 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-10T10:04:50.632 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.632 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.633 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.633 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-10T10:04:50.633 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.633 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.633 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.633 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.633 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.633 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.633 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.633 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.633 INFO:teuthology.orchestra.run.smithi005.stdout: "organization": { 2024-09-10T10:04:50.633 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "organization", 2024-09-10T10:04:50.633 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.633 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.633 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.634 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-10T10:04:50.634 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.634 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.634 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.634 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.634 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.634 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.634 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.634 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.634 INFO:teuthology.orchestra.run.smithi005.stdout: "proxy": { 2024-09-10T10:04:50.634 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "proxy", 2024-09-10T10:04:50.634 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.634 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.634 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.634 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-10T10:04:50.634 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.635 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.635 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.635 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.635 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.635 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.635 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.635 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.635 INFO:teuthology.orchestra.run.smithi005.stdout: "url": { 2024-09-10T10:04:50.635 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "url", 2024-09-10T10:04:50.635 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.635 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.635 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.635 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-10T10:04:50.635 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.635 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.636 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.636 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.636 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.636 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.636 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.636 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T10:04:50.636 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T10:04:50.636 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.636 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-10T10:04:50.636 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "test_orchestrator", 2024-09-10T10:04:50.636 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-10T10:04:50.636 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-10T10:04:50.636 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-10T10:04:50.636 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-10T10:04:50.637 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-10T10:04:50.637 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.637 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.637 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.637 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-10T10:04:50.637 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.637 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.637 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-10T10:04:50.637 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-10T10:04:50.637 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-10T10:04:50.637 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-10T10:04:50.637 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-10T10:04:50.637 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-10T10:04:50.637 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-10T10:04:50.638 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-10T10:04:50.638 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.638 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.638 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.638 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.638 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.638 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-10T10:04:50.638 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-10T10:04:50.638 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.638 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.638 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.638 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-10T10:04:50.638 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.638 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.639 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.639 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.639 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.639 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.639 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.639 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.639 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-10T10:04:50.639 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-10T10:04:50.639 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.639 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.639 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.639 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-10T10:04:50.639 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.639 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.640 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-10T10:04:50.640 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-10T10:04:50.640 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-10T10:04:50.640 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-10T10:04:50.640 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-10T10:04:50.640 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-10T10:04:50.640 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-10T10:04:50.640 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-10T10:04:50.640 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.640 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.640 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.640 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.640 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.640 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-10T10:04:50.641 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-10T10:04:50.641 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.641 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.641 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.641 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-10T10:04:50.641 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.641 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.641 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.641 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.641 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.641 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.641 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.641 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T10:04:50.641 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T10:04:50.641 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.642 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-10T10:04:50.642 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "volumes", 2024-09-10T10:04:50.642 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-10T10:04:50.642 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-10T10:04:50.642 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-10T10:04:50.642 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-10T10:04:50.642 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-10T10:04:50.642 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.642 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.642 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.642 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-10T10:04:50.642 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.642 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.642 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-10T10:04:50.643 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-10T10:04:50.643 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-10T10:04:50.643 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-10T10:04:50.643 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-10T10:04:50.643 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-10T10:04:50.643 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-10T10:04:50.643 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-10T10:04:50.643 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.643 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.643 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.643 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.643 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.643 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-10T10:04:50.643 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-10T10:04:50.644 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.644 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.644 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.644 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-10T10:04:50.644 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.644 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.644 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.644 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.644 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.644 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.644 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.644 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.644 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-10T10:04:50.644 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-10T10:04:50.644 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.645 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.645 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.645 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-10T10:04:50.645 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.645 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.645 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-10T10:04:50.645 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-10T10:04:50.645 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-10T10:04:50.645 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-10T10:04:50.645 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-10T10:04:50.645 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-10T10:04:50.645 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-10T10:04:50.645 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-10T10:04:50.645 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.645 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.646 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.646 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.646 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.646 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-10T10:04:50.646 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-10T10:04:50.646 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.646 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.646 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.646 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-10T10:04:50.646 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.646 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.646 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.646 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.646 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.647 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.647 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.647 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.647 INFO:teuthology.orchestra.run.smithi005.stdout: "max_concurrent_clones": { 2024-09-10T10:04:50.647 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "max_concurrent_clones", 2024-09-10T10:04:50.647 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-10T10:04:50.647 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.647 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.647 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "4", 2024-09-10T10:04:50.647 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.647 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.647 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.647 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-10T10:04:50.647 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.648 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.648 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.648 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.648 INFO:teuthology.orchestra.run.smithi005.stdout: "periodic_async_work": { 2024-09-10T10:04:50.648 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "periodic_async_work", 2024-09-10T10:04:50.648 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.648 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.648 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.648 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-10T10:04:50.648 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.648 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.648 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.648 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Periodically check for async work", 2024-09-10T10:04:50.649 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.649 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.649 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.649 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.649 INFO:teuthology.orchestra.run.smithi005.stdout: "snapshot_clone_delay": { 2024-09-10T10:04:50.649 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "snapshot_clone_delay", 2024-09-10T10:04:50.649 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-10T10:04:50.649 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.649 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.649 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-09-10T10:04:50.649 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.649 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.649 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.650 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-10T10:04:50.650 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.650 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.650 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.650 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.650 INFO:teuthology.orchestra.run.smithi005.stdout: "snapshot_clone_no_wait": { 2024-09-10T10:04:50.650 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "snapshot_clone_no_wait", 2024-09-10T10:04:50.650 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.650 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.650 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.650 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-10T10:04:50.650 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.650 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.650 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.650 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-10T10:04:50.651 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.651 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.651 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.651 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T10:04:50.651 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T10:04:50.651 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.651 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-10T10:04:50.651 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "zabbix", 2024-09-10T10:04:50.651 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-10T10:04:50.651 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-10T10:04:50.651 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-10T10:04:50.651 INFO:teuthology.orchestra.run.smithi005.stdout: "discovery_interval": { 2024-09-10T10:04:50.651 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "discovery_interval", 2024-09-10T10:04:50.652 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "uint", 2024-09-10T10:04:50.652 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.652 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.652 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "100", 2024-09-10T10:04:50.652 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.652 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.652 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.652 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.652 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.652 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.652 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.652 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.652 INFO:teuthology.orchestra.run.smithi005.stdout: "identifier": { 2024-09-10T10:04:50.652 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "identifier", 2024-09-10T10:04:50.653 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.653 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.653 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.653 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-10T10:04:50.653 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.653 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.653 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.653 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.653 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.653 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.653 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.653 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.653 INFO:teuthology.orchestra.run.smithi005.stdout: "interval": { 2024-09-10T10:04:50.653 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "interval", 2024-09-10T10:04:50.654 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-10T10:04:50.654 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.654 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.654 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "60", 2024-09-10T10:04:50.654 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.654 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.654 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.654 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.654 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.654 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.654 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.654 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.654 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-10T10:04:50.654 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-10T10:04:50.655 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.655 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.655 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.655 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-10T10:04:50.655 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.655 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.655 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-10T10:04:50.655 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-10T10:04:50.655 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-10T10:04:50.655 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-10T10:04:50.655 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-10T10:04:50.655 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-10T10:04:50.655 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-10T10:04:50.656 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-10T10:04:50.656 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.656 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.656 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.656 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.656 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.656 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-10T10:04:50.656 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-10T10:04:50.656 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.656 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.656 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.656 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-10T10:04:50.656 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.656 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.657 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.657 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.657 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.657 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.657 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.657 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.657 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-10T10:04:50.657 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-10T10:04:50.657 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.657 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.657 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.657 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-10T10:04:50.658 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.658 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.658 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-10T10:04:50.658 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-10T10:04:50.658 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-10T10:04:50.658 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-10T10:04:50.658 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-10T10:04:50.658 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-10T10:04:50.658 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-10T10:04:50.658 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-10T10:04:50.658 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.658 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.658 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.658 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.659 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.659 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-10T10:04:50.659 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-10T10:04:50.659 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.659 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.659 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.659 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-10T10:04:50.659 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.659 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.659 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.659 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.659 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.660 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.660 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.660 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.660 INFO:teuthology.orchestra.run.smithi005.stdout: "zabbix_host": { 2024-09-10T10:04:50.660 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "zabbix_host", 2024-09-10T10:04:50.660 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.660 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.660 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.660 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-10T10:04:50.660 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.660 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.660 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.661 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.661 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.661 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.661 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.661 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.661 INFO:teuthology.orchestra.run.smithi005.stdout: "zabbix_port": { 2024-09-10T10:04:50.661 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "zabbix_port", 2024-09-10T10:04:50.661 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-10T10:04:50.661 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.661 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.661 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10051", 2024-09-10T10:04:50.661 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.661 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.661 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.661 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.662 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.662 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.662 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.662 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.662 INFO:teuthology.orchestra.run.smithi005.stdout: "zabbix_sender": { 2024-09-10T10:04:50.662 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "zabbix_sender", 2024-09-10T10:04:50.662 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.662 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.662 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.662 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-10T10:04:50.662 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.662 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.662 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.663 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.663 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.663 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.663 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.663 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T10:04:50.663 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T10:04:50.663 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T10:04:50.663 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-10T10:04:50.663 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T10:04:50.663 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-10T10:04:50.663 INFO:teuthology.orchestra.run.smithi005.stdout: "modules": [ 2024-09-10T10:04:50.663 INFO:teuthology.orchestra.run.smithi005.stdout: "cephadm", 2024-09-10T10:04:50.663 INFO:teuthology.orchestra.run.smithi005.stdout: "dashboard", 2024-09-10T10:04:50.664 INFO:teuthology.orchestra.run.smithi005.stdout: "iostat", 2024-09-10T10:04:50.664 INFO:teuthology.orchestra.run.smithi005.stdout: "nfs", 2024-09-10T10:04:50.664 INFO:teuthology.orchestra.run.smithi005.stdout: "restful" 2024-09-10T10:04:50.664 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-10T10:04:50.664 INFO:teuthology.orchestra.run.smithi005.stdout: "available_modules": [ 2024-09-10T10:04:50.664 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-10T10:04:50.664 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "alerts", 2024-09-10T10:04:50.664 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-10T10:04:50.664 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-10T10:04:50.664 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-10T10:04:50.664 INFO:teuthology.orchestra.run.smithi005.stdout: "interval": { 2024-09-10T10:04:50.664 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "interval", 2024-09-10T10:04:50.664 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-10T10:04:50.664 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.665 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.665 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "60", 2024-09-10T10:04:50.665 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.665 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.665 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.665 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "How frequently to reexamine health status", 2024-09-10T10:04:50.665 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.665 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.665 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.665 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.665 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-10T10:04:50.665 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-10T10:04:50.665 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.665 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.666 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.666 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-10T10:04:50.666 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.666 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.666 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-10T10:04:50.666 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-10T10:04:50.666 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-10T10:04:50.666 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-10T10:04:50.666 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-10T10:04:50.666 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-10T10:04:50.666 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-10T10:04:50.666 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-10T10:04:50.666 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.667 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.667 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.667 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.667 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.667 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-10T10:04:50.667 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-10T10:04:50.667 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.667 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.667 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.667 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-10T10:04:50.667 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.667 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.667 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.668 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.668 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.668 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.668 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.668 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.668 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-10T10:04:50.668 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-10T10:04:50.668 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.668 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.668 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.668 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-10T10:04:50.668 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.668 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.668 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-10T10:04:50.668 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-10T10:04:50.669 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-10T10:04:50.669 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-10T10:04:50.669 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-10T10:04:50.669 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-10T10:04:50.669 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-10T10:04:50.669 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-10T10:04:50.669 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.669 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.669 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.669 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.669 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.669 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-10T10:04:50.669 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-10T10:04:50.669 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.670 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.670 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.670 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-10T10:04:50.670 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.670 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.670 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.670 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.670 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.670 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.670 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.670 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.670 INFO:teuthology.orchestra.run.smithi005.stdout: "smtp_destination": { 2024-09-10T10:04:50.670 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "smtp_destination", 2024-09-10T10:04:50.670 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.670 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.671 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.671 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-10T10:04:50.671 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.671 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.671 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.671 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Email address to send alerts to", 2024-09-10T10:04:50.671 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.671 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.671 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.671 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.671 INFO:teuthology.orchestra.run.smithi005.stdout: "smtp_from_name": { 2024-09-10T10:04:50.671 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "smtp_from_name", 2024-09-10T10:04:50.671 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.671 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.671 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.672 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "Ceph", 2024-09-10T10:04:50.672 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.672 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.672 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.672 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Email From: name", 2024-09-10T10:04:50.672 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.672 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.672 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.672 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.672 INFO:teuthology.orchestra.run.smithi005.stdout: "smtp_host": { 2024-09-10T10:04:50.672 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "smtp_host", 2024-09-10T10:04:50.672 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.672 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.672 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.673 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-10T10:04:50.673 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.673 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.673 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.673 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "SMTP server", 2024-09-10T10:04:50.673 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.673 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.673 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.673 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.673 INFO:teuthology.orchestra.run.smithi005.stdout: "smtp_password": { 2024-09-10T10:04:50.673 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "smtp_password", 2024-09-10T10:04:50.673 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.673 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.673 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.673 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-10T10:04:50.673 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.674 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.674 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.674 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Password to authenticate with", 2024-09-10T10:04:50.674 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.674 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.674 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.674 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.674 INFO:teuthology.orchestra.run.smithi005.stdout: "smtp_port": { 2024-09-10T10:04:50.674 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "smtp_port", 2024-09-10T10:04:50.674 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-10T10:04:50.674 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.674 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.674 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "465", 2024-09-10T10:04:50.674 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.674 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.675 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.675 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "SMTP port", 2024-09-10T10:04:50.675 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.675 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.675 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.675 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.675 INFO:teuthology.orchestra.run.smithi005.stdout: "smtp_sender": { 2024-09-10T10:04:50.675 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "smtp_sender", 2024-09-10T10:04:50.675 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.675 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.675 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.675 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-10T10:04:50.675 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.675 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.676 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.676 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "SMTP envelope sender", 2024-09-10T10:04:50.676 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.676 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.676 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.676 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.676 INFO:teuthology.orchestra.run.smithi005.stdout: "smtp_ssl": { 2024-09-10T10:04:50.676 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "smtp_ssl", 2024-09-10T10:04:50.676 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.676 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.676 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.676 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-10T10:04:50.676 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.676 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.677 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.677 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-10T10:04:50.677 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.677 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.677 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.677 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.677 INFO:teuthology.orchestra.run.smithi005.stdout: "smtp_user": { 2024-09-10T10:04:50.677 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "smtp_user", 2024-09-10T10:04:50.677 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.677 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.677 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.677 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-10T10:04:50.677 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.678 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.678 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.678 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "User to authenticate as", 2024-09-10T10:04:50.678 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.678 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.678 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.678 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T10:04:50.678 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T10:04:50.678 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.678 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-10T10:04:50.678 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "balancer", 2024-09-10T10:04:50.678 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-10T10:04:50.678 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-10T10:04:50.678 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-10T10:04:50.679 INFO:teuthology.orchestra.run.smithi005.stdout: "active": { 2024-09-10T10:04:50.679 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "active", 2024-09-10T10:04:50.679 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.679 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.679 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.679 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-10T10:04:50.679 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.679 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.679 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.679 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "automatically balance PGs across cluster", 2024-09-10T10:04:50.679 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.679 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.679 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.679 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.679 INFO:teuthology.orchestra.run.smithi005.stdout: "begin_time": { 2024-09-10T10:04:50.680 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "begin_time", 2024-09-10T10:04:50.680 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.680 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.680 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.680 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0000", 2024-09-10T10:04:50.680 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.680 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.680 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.680 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "beginning time of day to automatically balance", 2024-09-10T10:04:50.680 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-10T10:04:50.680 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.680 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.680 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.680 INFO:teuthology.orchestra.run.smithi005.stdout: "begin_weekday": { 2024-09-10T10:04:50.680 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "begin_weekday", 2024-09-10T10:04:50.681 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "uint", 2024-09-10T10:04:50.681 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.681 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.681 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-09-10T10:04:50.681 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "0", 2024-09-10T10:04:50.681 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "6", 2024-09-10T10:04:50.681 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.681 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-10T10:04:50.681 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-10T10:04:50.681 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.681 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.681 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.681 INFO:teuthology.orchestra.run.smithi005.stdout: "crush_compat_max_iterations": { 2024-09-10T10:04:50.681 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "crush_compat_max_iterations", 2024-09-10T10:04:50.681 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "uint", 2024-09-10T10:04:50.682 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.682 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.682 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "25", 2024-09-10T10:04:50.682 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "1", 2024-09-10T10:04:50.682 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "250", 2024-09-10T10:04:50.682 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.682 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-10T10:04:50.682 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.682 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.682 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.682 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.682 INFO:teuthology.orchestra.run.smithi005.stdout: "crush_compat_metrics": { 2024-09-10T10:04:50.682 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "crush_compat_metrics", 2024-09-10T10:04:50.682 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.683 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.683 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.683 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "pgs,objects,bytes", 2024-09-10T10:04:50.683 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.683 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.683 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.683 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-10T10:04:50.683 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-10T10:04:50.683 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.683 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.683 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.683 INFO:teuthology.orchestra.run.smithi005.stdout: "crush_compat_step": { 2024-09-10T10:04:50.683 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "crush_compat_step", 2024-09-10T10:04:50.683 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "float", 2024-09-10T10:04:50.683 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.684 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.684 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0.5", 2024-09-10T10:04:50.684 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "0.001", 2024-09-10T10:04:50.684 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "0.999", 2024-09-10T10:04:50.684 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.684 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "aggressiveness of optimization", 2024-09-10T10:04:50.684 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-10T10:04:50.684 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.684 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.684 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.684 INFO:teuthology.orchestra.run.smithi005.stdout: "end_time": { 2024-09-10T10:04:50.684 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "end_time", 2024-09-10T10:04:50.684 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.684 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.684 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.685 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "2359", 2024-09-10T10:04:50.685 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.685 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.685 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.685 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "ending time of day to automatically balance", 2024-09-10T10:04:50.685 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-10T10:04:50.685 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.685 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.685 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.685 INFO:teuthology.orchestra.run.smithi005.stdout: "end_weekday": { 2024-09-10T10:04:50.685 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "end_weekday", 2024-09-10T10:04:50.685 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "uint", 2024-09-10T10:04:50.685 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.685 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.686 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-09-10T10:04:50.686 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "0", 2024-09-10T10:04:50.686 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "6", 2024-09-10T10:04:50.686 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.686 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-10T10:04:50.686 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-10T10:04:50.686 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.686 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.686 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.686 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-10T10:04:50.686 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-10T10:04:50.686 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.686 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.687 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.687 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-10T10:04:50.687 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.687 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.687 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-10T10:04:50.687 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-10T10:04:50.687 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-10T10:04:50.687 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-10T10:04:50.687 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-10T10:04:50.687 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-10T10:04:50.687 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-10T10:04:50.687 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-10T10:04:50.687 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.688 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.688 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.688 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.688 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.688 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-10T10:04:50.688 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-10T10:04:50.688 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.688 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.688 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.688 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-10T10:04:50.688 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.688 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.688 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.689 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.689 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.689 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.689 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.689 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.689 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-10T10:04:50.689 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-10T10:04:50.689 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.689 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.689 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.689 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-10T10:04:50.689 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.689 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.689 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-10T10:04:50.689 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-10T10:04:50.690 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-10T10:04:50.690 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-10T10:04:50.690 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-10T10:04:50.690 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-10T10:04:50.690 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-10T10:04:50.690 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-10T10:04:50.690 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.690 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.690 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.690 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.690 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.690 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-10T10:04:50.690 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-10T10:04:50.690 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.690 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.691 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.691 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-10T10:04:50.691 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.691 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.691 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.691 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.691 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.691 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.691 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.691 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.691 INFO:teuthology.orchestra.run.smithi005.stdout: "min_score": { 2024-09-10T10:04:50.691 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "min_score", 2024-09-10T10:04:50.691 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "float", 2024-09-10T10:04:50.691 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.691 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.691 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-09-10T10:04:50.692 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.692 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.692 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.692 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-10T10:04:50.692 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.692 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.692 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.692 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.692 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": { 2024-09-10T10:04:50.692 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "mode", 2024-09-10T10:04:50.692 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.692 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.692 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.692 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "upmap", 2024-09-10T10:04:50.692 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.693 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.693 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-10T10:04:50.693 INFO:teuthology.orchestra.run.smithi005.stdout: "crush-compat", 2024-09-10T10:04:50.693 INFO:teuthology.orchestra.run.smithi005.stdout: "none", 2024-09-10T10:04:50.693 INFO:teuthology.orchestra.run.smithi005.stdout: "read", 2024-09-10T10:04:50.693 INFO:teuthology.orchestra.run.smithi005.stdout: "upmap", 2024-09-10T10:04:50.693 INFO:teuthology.orchestra.run.smithi005.stdout: "upmap-read" 2024-09-10T10:04:50.693 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-10T10:04:50.693 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Balancer mode", 2024-09-10T10:04:50.693 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.693 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.693 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.693 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.694 INFO:teuthology.orchestra.run.smithi005.stdout: "pool_ids": { 2024-09-10T10:04:50.694 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "pool_ids", 2024-09-10T10:04:50.694 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.694 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.694 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.694 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-10T10:04:50.694 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.694 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.694 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.694 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-10T10:04:50.694 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.694 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.694 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.694 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.694 INFO:teuthology.orchestra.run.smithi005.stdout: "sleep_interval": { 2024-09-10T10:04:50.695 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sleep_interval", 2024-09-10T10:04:50.695 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-10T10:04:50.695 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.695 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.695 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "60", 2024-09-10T10:04:50.695 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.695 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.695 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.695 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-10T10:04:50.695 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.695 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.695 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.695 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.695 INFO:teuthology.orchestra.run.smithi005.stdout: "upmap_max_deviation": { 2024-09-10T10:04:50.695 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "upmap_max_deviation", 2024-09-10T10:04:50.696 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-10T10:04:50.696 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.696 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.696 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "5", 2024-09-10T10:04:50.696 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "1", 2024-09-10T10:04:50.696 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.696 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.696 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-10T10:04:50.696 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-10T10:04:50.696 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.696 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.696 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.696 INFO:teuthology.orchestra.run.smithi005.stdout: "upmap_max_optimizations": { 2024-09-10T10:04:50.696 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "upmap_max_optimizations", 2024-09-10T10:04:50.697 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "uint", 2024-09-10T10:04:50.697 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.697 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.697 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10", 2024-09-10T10:04:50.697 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.697 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.697 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.697 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-10T10:04:50.697 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.697 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.697 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.697 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T10:04:50.697 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T10:04:50.698 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.698 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-10T10:04:50.698 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "cephadm", 2024-09-10T10:04:50.698 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-10T10:04:50.698 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-10T10:04:50.698 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-10T10:04:50.698 INFO:teuthology.orchestra.run.smithi005.stdout: "agent_down_multiplier": { 2024-09-10T10:04:50.698 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "agent_down_multiplier", 2024-09-10T10:04:50.698 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "float", 2024-09-10T10:04:50.698 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.698 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.698 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "3.0", 2024-09-10T10:04:50.698 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.698 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.698 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.699 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-10T10:04:50.699 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.699 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.699 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.699 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.699 INFO:teuthology.orchestra.run.smithi005.stdout: "agent_refresh_rate": { 2024-09-10T10:04:50.699 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "agent_refresh_rate", 2024-09-10T10:04:50.699 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-10T10:04:50.699 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.699 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.699 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "20", 2024-09-10T10:04:50.699 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.699 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.699 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.700 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-10T10:04:50.700 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.700 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.700 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.700 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.700 INFO:teuthology.orchestra.run.smithi005.stdout: "agent_starting_port": { 2024-09-10T10:04:50.700 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "agent_starting_port", 2024-09-10T10:04:50.700 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-10T10:04:50.700 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.700 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.700 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "4721", 2024-09-10T10:04:50.700 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.700 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.700 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.701 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-10T10:04:50.701 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.701 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.701 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.701 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.701 INFO:teuthology.orchestra.run.smithi005.stdout: "allow_ptrace": { 2024-09-10T10:04:50.701 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "allow_ptrace", 2024-09-10T10:04:50.701 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.701 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.701 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.701 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-10T10:04:50.701 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.701 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.701 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.702 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-10T10:04:50.702 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-10T10:04:50.702 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.702 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.702 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.702 INFO:teuthology.orchestra.run.smithi005.stdout: "autotune_interval": { 2024-09-10T10:04:50.702 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "autotune_interval", 2024-09-10T10:04:50.702 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-10T10:04:50.702 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.702 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.702 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "600", 2024-09-10T10:04:50.702 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.702 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.703 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.703 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-10T10:04:50.703 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.703 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.703 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.703 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.703 INFO:teuthology.orchestra.run.smithi005.stdout: "autotune_memory_target_ratio": { 2024-09-10T10:04:50.703 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "autotune_memory_target_ratio", 2024-09-10T10:04:50.703 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "float", 2024-09-10T10:04:50.703 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.703 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.703 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0.7", 2024-09-10T10:04:50.703 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.704 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.704 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.704 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-10T10:04:50.704 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.704 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.704 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.704 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.704 INFO:teuthology.orchestra.run.smithi005.stdout: "cephadm_log_destination": { 2024-09-10T10:04:50.704 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "cephadm_log_destination", 2024-09-10T10:04:50.704 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.704 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.704 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.704 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-10T10:04:50.704 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.705 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.705 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-10T10:04:50.705 INFO:teuthology.orchestra.run.smithi005.stdout: "file", 2024-09-10T10:04:50.705 INFO:teuthology.orchestra.run.smithi005.stdout: "file,syslog", 2024-09-10T10:04:50.705 INFO:teuthology.orchestra.run.smithi005.stdout: "syslog" 2024-09-10T10:04:50.705 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-10T10:04:50.705 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-10T10:04:50.705 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.705 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.705 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.705 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.705 INFO:teuthology.orchestra.run.smithi005.stdout: "cgroups_split": { 2024-09-10T10:04:50.705 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "cgroups_split", 2024-09-10T10:04:50.706 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.706 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.706 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.706 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-10T10:04:50.706 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.706 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.706 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.706 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-10T10:04:50.706 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.706 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.706 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.706 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.706 INFO:teuthology.orchestra.run.smithi005.stdout: "config_checks_enabled": { 2024-09-10T10:04:50.707 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "config_checks_enabled", 2024-09-10T10:04:50.707 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.707 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.707 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.707 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-10T10:04:50.707 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.707 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.707 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.707 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-10T10:04:50.707 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.707 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.707 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.707 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.707 INFO:teuthology.orchestra.run.smithi005.stdout: "config_dashboard": { 2024-09-10T10:04:50.707 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "config_dashboard", 2024-09-10T10:04:50.708 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.708 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.708 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.708 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-10T10:04:50.708 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.708 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.708 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.708 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-10T10:04:50.708 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.708 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.708 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.708 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.708 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_alertmanager": { 2024-09-10T10:04:50.709 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_alertmanager", 2024-09-10T10:04:50.709 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.709 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.709 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.709 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-09-10T10:04:50.709 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.709 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.709 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.709 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Prometheus container image", 2024-09-10T10:04:50.709 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.709 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.709 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.709 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.710 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_base": { 2024-09-10T10:04:50.710 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_base", 2024-09-10T10:04:50.710 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.710 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.710 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.710 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-10T10:04:50.710 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.710 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.710 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.710 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Container image name, without the tag", 2024-09-10T10:04:50.710 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.710 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.710 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.710 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.710 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_elasticsearch": { 2024-09-10T10:04:50.711 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_elasticsearch", 2024-09-10T10:04:50.711 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.711 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.711 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.711 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-10T10:04:50.711 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.711 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.711 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.711 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "elasticsearch container image", 2024-09-10T10:04:50.711 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.711 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.711 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.711 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.711 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_grafana": { 2024-09-10T10:04:50.712 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_grafana", 2024-09-10T10:04:50.712 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.712 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.712 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.712 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-09-10T10:04:50.712 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.712 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.712 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.712 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Prometheus container image", 2024-09-10T10:04:50.712 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.712 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.712 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.713 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.713 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_haproxy": { 2024-09-10T10:04:50.713 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_haproxy", 2024-09-10T10:04:50.713 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.713 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.713 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.713 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-10T10:04:50.713 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.713 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.713 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.713 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "HAproxy container image", 2024-09-10T10:04:50.713 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.713 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.713 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.714 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.714 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_jaeger_agent": { 2024-09-10T10:04:50.714 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_jaeger_agent", 2024-09-10T10:04:50.714 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.714 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.714 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.714 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-10T10:04:50.714 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.714 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.714 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.714 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Jaeger agent container image", 2024-09-10T10:04:50.714 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.714 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.714 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.714 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.715 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_jaeger_collector": { 2024-09-10T10:04:50.715 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_jaeger_collector", 2024-09-10T10:04:50.715 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.715 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.715 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.715 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-10T10:04:50.715 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.715 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.715 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.715 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Jaeger collector container image", 2024-09-10T10:04:50.715 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.715 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.715 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.715 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.716 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_jaeger_query": { 2024-09-10T10:04:50.716 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_jaeger_query", 2024-09-10T10:04:50.716 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.716 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.716 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.716 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-10T10:04:50.716 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.716 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.716 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.716 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Jaeger query container image", 2024-09-10T10:04:50.716 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.716 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.716 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.716 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.717 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_keepalived": { 2024-09-10T10:04:50.717 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_keepalived", 2024-09-10T10:04:50.717 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.717 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.717 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.717 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-10T10:04:50.717 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.717 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.717 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.717 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Keepalived container image", 2024-09-10T10:04:50.717 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.717 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.717 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.717 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.718 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_loki": { 2024-09-10T10:04:50.718 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_loki", 2024-09-10T10:04:50.718 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.718 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.718 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.718 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-10T10:04:50.718 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.718 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.718 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.718 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Loki container image", 2024-09-10T10:04:50.718 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.718 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.718 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.718 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.719 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_node_exporter": { 2024-09-10T10:04:50.719 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_node_exporter", 2024-09-10T10:04:50.719 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.719 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.719 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.719 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-09-10T10:04:50.719 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.719 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.719 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.719 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Prometheus container image", 2024-09-10T10:04:50.719 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.719 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.720 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.720 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.720 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_nvmeof": { 2024-09-10T10:04:50.720 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_nvmeof", 2024-09-10T10:04:50.720 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.720 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.720 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.720 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-09-10T10:04:50.720 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.720 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.720 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.720 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Nvme-of container image", 2024-09-10T10:04:50.720 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.721 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.721 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.721 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.721 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_prometheus": { 2024-09-10T10:04:50.721 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_prometheus", 2024-09-10T10:04:50.721 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.721 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.721 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.721 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-09-10T10:04:50.721 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.721 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.721 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.721 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Prometheus container image", 2024-09-10T10:04:50.722 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.722 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.722 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.722 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.722 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_promtail": { 2024-09-10T10:04:50.722 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_promtail", 2024-09-10T10:04:50.722 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.722 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.722 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.722 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-10T10:04:50.722 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.722 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.722 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.723 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Promtail container image", 2024-09-10T10:04:50.723 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.723 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.723 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.723 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.723 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_samba": { 2024-09-10T10:04:50.723 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_samba", 2024-09-10T10:04:50.723 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.723 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.723 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.723 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-10T10:04:50.723 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.723 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.723 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.724 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Samba/SMB container image", 2024-09-10T10:04:50.724 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.724 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.724 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.724 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.724 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_snmp_gateway": { 2024-09-10T10:04:50.724 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_snmp_gateway", 2024-09-10T10:04:50.724 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.724 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.724 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.724 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-10T10:04:50.724 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.724 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.724 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.725 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "SNMP Gateway container image", 2024-09-10T10:04:50.725 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.725 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.725 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.725 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.725 INFO:teuthology.orchestra.run.smithi005.stdout: "container_init": { 2024-09-10T10:04:50.725 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_init", 2024-09-10T10:04:50.725 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.725 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.725 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.725 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-10T10:04:50.725 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.725 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.725 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.726 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Run podman/docker with `--init`", 2024-09-10T10:04:50.726 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.726 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.726 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.726 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.726 INFO:teuthology.orchestra.run.smithi005.stdout: "daemon_cache_timeout": { 2024-09-10T10:04:50.726 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "daemon_cache_timeout", 2024-09-10T10:04:50.726 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-10T10:04:50.726 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.726 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.726 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "600", 2024-09-10T10:04:50.726 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.726 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.727 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.727 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-10T10:04:50.727 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.727 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.727 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.727 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.727 INFO:teuthology.orchestra.run.smithi005.stdout: "default_cephadm_command_timeout": { 2024-09-10T10:04:50.727 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "default_cephadm_command_timeout", 2024-09-10T10:04:50.727 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-10T10:04:50.727 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.727 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.727 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "900", 2024-09-10T10:04:50.727 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.727 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.728 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.728 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-10T10:04:50.728 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.728 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.728 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.728 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.728 INFO:teuthology.orchestra.run.smithi005.stdout: "default_registry": { 2024-09-10T10:04:50.728 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "default_registry", 2024-09-10T10:04:50.728 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.728 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.728 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.728 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "docker.io", 2024-09-10T10:04:50.728 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.728 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.728 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.729 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-10T10:04:50.729 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.729 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.729 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.729 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.729 INFO:teuthology.orchestra.run.smithi005.stdout: "device_cache_timeout": { 2024-09-10T10:04:50.729 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "device_cache_timeout", 2024-09-10T10:04:50.729 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-10T10:04:50.729 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.729 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.729 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "1800", 2024-09-10T10:04:50.729 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.729 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.730 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.730 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "seconds to cache device inventory", 2024-09-10T10:04:50.730 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.730 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.730 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.730 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.730 INFO:teuthology.orchestra.run.smithi005.stdout: "device_enhanced_scan": { 2024-09-10T10:04:50.730 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "device_enhanced_scan", 2024-09-10T10:04:50.730 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.730 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.730 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.730 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-10T10:04:50.730 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.730 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.731 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.731 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-10T10:04:50.731 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.731 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.731 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.731 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.731 INFO:teuthology.orchestra.run.smithi005.stdout: "facts_cache_timeout": { 2024-09-10T10:04:50.731 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "facts_cache_timeout", 2024-09-10T10:04:50.731 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-10T10:04:50.731 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.731 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.731 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "60", 2024-09-10T10:04:50.731 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.731 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.731 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.732 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "seconds to cache host facts data", 2024-09-10T10:04:50.732 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.732 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.732 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.732 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.732 INFO:teuthology.orchestra.run.smithi005.stdout: "grafana_dashboards_path": { 2024-09-10T10:04:50.732 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "grafana_dashboards_path", 2024-09-10T10:04:50.732 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.732 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.732 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.732 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-10T10:04:50.732 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.732 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.732 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.733 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-10T10:04:50.733 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.733 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.733 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.733 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.733 INFO:teuthology.orchestra.run.smithi005.stdout: "host_check_interval": { 2024-09-10T10:04:50.733 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "host_check_interval", 2024-09-10T10:04:50.733 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-10T10:04:50.733 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.733 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.733 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "600", 2024-09-10T10:04:50.733 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.733 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.733 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.733 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "how frequently to perform a host check", 2024-09-10T10:04:50.734 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.734 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.734 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.734 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.734 INFO:teuthology.orchestra.run.smithi005.stdout: "hw_monitoring": { 2024-09-10T10:04:50.734 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "hw_monitoring", 2024-09-10T10:04:50.734 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.734 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.734 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.734 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-10T10:04:50.734 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.734 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.734 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.734 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-10T10:04:50.735 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.735 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.735 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.735 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.735 INFO:teuthology.orchestra.run.smithi005.stdout: "inventory_list_all": { 2024-09-10T10:04:50.735 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "inventory_list_all", 2024-09-10T10:04:50.735 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.735 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.735 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.735 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-10T10:04:50.735 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.735 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.735 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.735 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-10T10:04:50.736 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.736 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.736 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.736 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.736 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-10T10:04:50.736 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-10T10:04:50.736 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.736 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.736 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.736 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-10T10:04:50.736 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.736 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.736 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-10T10:04:50.736 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-10T10:04:50.737 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-10T10:04:50.737 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-10T10:04:50.737 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-10T10:04:50.737 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-10T10:04:50.737 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-10T10:04:50.737 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-10T10:04:50.737 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.737 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.737 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.737 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.737 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.737 INFO:teuthology.orchestra.run.smithi005.stdout: "log_refresh_metadata": { 2024-09-10T10:04:50.737 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_refresh_metadata", 2024-09-10T10:04:50.738 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.738 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.738 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.738 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-10T10:04:50.738 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.738 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.738 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.738 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-10T10:04:50.738 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.738 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.738 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.738 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.738 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-10T10:04:50.738 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-10T10:04:50.738 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.739 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.739 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.739 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-10T10:04:50.739 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.739 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.739 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.739 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-10T10:04:50.739 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.739 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.739 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.739 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.739 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-10T10:04:50.739 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-10T10:04:50.739 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.740 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.740 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.740 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-10T10:04:50.740 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.740 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.740 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-10T10:04:50.740 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-10T10:04:50.740 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-10T10:04:50.740 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-10T10:04:50.740 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-10T10:04:50.740 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-10T10:04:50.740 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-10T10:04:50.740 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-10T10:04:50.740 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.741 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.741 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.741 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.741 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.741 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-10T10:04:50.741 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-10T10:04:50.741 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.741 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.741 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.741 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-10T10:04:50.741 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.741 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.741 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.741 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.742 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.742 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.742 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.742 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.742 INFO:teuthology.orchestra.run.smithi005.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-10T10:04:50.742 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-10T10:04:50.742 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.742 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.742 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.742 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-10T10:04:50.742 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.742 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.742 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.742 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-10T10:04:50.742 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.743 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.743 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.743 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.743 INFO:teuthology.orchestra.run.smithi005.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-10T10:04:50.743 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-10T10:04:50.743 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.743 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.743 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.743 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "*", 2024-09-10T10:04:50.743 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.743 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.743 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.743 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-10T10:04:50.744 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.744 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.744 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.744 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.744 INFO:teuthology.orchestra.run.smithi005.stdout: "max_count_per_host": { 2024-09-10T10:04:50.744 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "max_count_per_host", 2024-09-10T10:04:50.744 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-10T10:04:50.744 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.744 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.744 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10", 2024-09-10T10:04:50.744 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.744 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.744 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.745 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "max number of daemons per service per host", 2024-09-10T10:04:50.745 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.745 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.745 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.745 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.745 INFO:teuthology.orchestra.run.smithi005.stdout: "max_osd_draining_count": { 2024-09-10T10:04:50.745 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "max_osd_draining_count", 2024-09-10T10:04:50.745 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-10T10:04:50.745 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.745 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.745 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10", 2024-09-10T10:04:50.745 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.745 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.746 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.746 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-10T10:04:50.746 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.746 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.746 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.746 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.746 INFO:teuthology.orchestra.run.smithi005.stdout: "migration_current": { 2024-09-10T10:04:50.746 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "migration_current", 2024-09-10T10:04:50.746 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-10T10:04:50.746 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.746 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.746 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-10T10:04:50.746 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.746 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.747 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.747 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "internal - do not modify", 2024-09-10T10:04:50.747 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.747 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.747 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.747 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.747 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": { 2024-09-10T10:04:50.747 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "mode", 2024-09-10T10:04:50.747 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.747 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.747 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.747 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "root", 2024-09-10T10:04:50.748 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.748 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.748 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-10T10:04:50.748 INFO:teuthology.orchestra.run.smithi005.stdout: "cephadm-package", 2024-09-10T10:04:50.748 INFO:teuthology.orchestra.run.smithi005.stdout: "root" 2024-09-10T10:04:50.748 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-10T10:04:50.748 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "mode for remote execution of cephadm", 2024-09-10T10:04:50.748 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.748 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.748 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.748 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.748 INFO:teuthology.orchestra.run.smithi005.stdout: "oob_default_addr": { 2024-09-10T10:04:50.748 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "oob_default_addr", 2024-09-10T10:04:50.748 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.749 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.749 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.749 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "169.254.1.1", 2024-09-10T10:04:50.749 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.749 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.749 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.749 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-10T10:04:50.749 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.749 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.749 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.749 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.749 INFO:teuthology.orchestra.run.smithi005.stdout: "prometheus_alerts_path": { 2024-09-10T10:04:50.749 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "prometheus_alerts_path", 2024-09-10T10:04:50.749 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.750 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.750 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.750 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-10T10:04:50.750 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.750 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.750 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.750 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-10T10:04:50.750 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.750 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.750 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.750 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.750 INFO:teuthology.orchestra.run.smithi005.stdout: "registry_insecure": { 2024-09-10T10:04:50.750 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "registry_insecure", 2024-09-10T10:04:50.751 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.751 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.751 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.751 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-10T10:04:50.751 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.751 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.751 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.751 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-10T10:04:50.751 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.751 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.751 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.751 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.751 INFO:teuthology.orchestra.run.smithi005.stdout: "registry_password": { 2024-09-10T10:04:50.751 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "registry_password", 2024-09-10T10:04:50.751 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.751 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.752 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.752 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-10T10:04:50.752 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.752 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.752 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.752 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-10T10:04:50.752 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.752 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.752 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.752 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.752 INFO:teuthology.orchestra.run.smithi005.stdout: "registry_url": { 2024-09-10T10:04:50.752 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "registry_url", 2024-09-10T10:04:50.752 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.752 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.753 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.753 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-10T10:04:50.753 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.753 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.753 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.753 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-10T10:04:50.753 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.753 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.753 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.753 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.753 INFO:teuthology.orchestra.run.smithi005.stdout: "registry_username": { 2024-09-10T10:04:50.753 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "registry_username", 2024-09-10T10:04:50.753 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.753 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.753 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.753 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-10T10:04:50.754 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.754 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.754 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.754 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-10T10:04:50.754 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.754 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.754 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.754 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.754 INFO:teuthology.orchestra.run.smithi005.stdout: "secure_monitoring_stack": { 2024-09-10T10:04:50.754 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "secure_monitoring_stack", 2024-09-10T10:04:50.754 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.754 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.754 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.755 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-10T10:04:50.755 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.755 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.755 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.755 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-10T10:04:50.755 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.755 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.755 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.755 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.755 INFO:teuthology.orchestra.run.smithi005.stdout: "service_discovery_port": { 2024-09-10T10:04:50.755 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "service_discovery_port", 2024-09-10T10:04:50.755 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-10T10:04:50.755 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.755 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.756 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "8765", 2024-09-10T10:04:50.756 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.756 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.756 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.756 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "cephadm service discovery port", 2024-09-10T10:04:50.756 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.756 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.756 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.756 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.756 INFO:teuthology.orchestra.run.smithi005.stdout: "ssh_config_file": { 2024-09-10T10:04:50.756 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ssh_config_file", 2024-09-10T10:04:50.756 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.757 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.757 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.757 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-10T10:04:50.757 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.757 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.757 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.757 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-10T10:04:50.757 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.757 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.757 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.757 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.757 INFO:teuthology.orchestra.run.smithi005.stdout: "use_agent": { 2024-09-10T10:04:50.757 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "use_agent", 2024-09-10T10:04:50.758 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.758 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.758 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.758 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-10T10:04:50.758 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.758 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.758 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.758 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-10T10:04:50.758 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.758 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.758 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.758 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.758 INFO:teuthology.orchestra.run.smithi005.stdout: "use_repo_digest": { 2024-09-10T10:04:50.758 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "use_repo_digest", 2024-09-10T10:04:50.759 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.759 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.759 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.759 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-10T10:04:50.759 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.759 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.759 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.759 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-10T10:04:50.759 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.759 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.759 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.759 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.759 INFO:teuthology.orchestra.run.smithi005.stdout: "warn_on_failed_host_check": { 2024-09-10T10:04:50.759 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "warn_on_failed_host_check", 2024-09-10T10:04:50.760 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.760 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.760 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.760 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-10T10:04:50.760 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.760 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.760 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.760 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "raise a health warning if the host check fails", 2024-09-10T10:04:50.760 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.760 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.760 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.760 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.760 INFO:teuthology.orchestra.run.smithi005.stdout: "warn_on_stray_daemons": { 2024-09-10T10:04:50.761 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "warn_on_stray_daemons", 2024-09-10T10:04:50.761 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.761 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.761 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.761 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-10T10:04:50.761 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.761 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.761 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.761 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-10T10:04:50.761 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.761 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.761 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.761 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.762 INFO:teuthology.orchestra.run.smithi005.stdout: "warn_on_stray_hosts": { 2024-09-10T10:04:50.762 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "warn_on_stray_hosts", 2024-09-10T10:04:50.762 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.762 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.762 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.762 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-10T10:04:50.762 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.762 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.762 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.762 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-10T10:04:50.762 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.762 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.762 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.762 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T10:04:50.763 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T10:04:50.763 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.763 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-10T10:04:50.763 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "crash", 2024-09-10T10:04:50.763 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-10T10:04:50.763 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-10T10:04:50.763 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-10T10:04:50.763 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-10T10:04:50.763 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-10T10:04:50.763 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.763 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.763 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.763 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-10T10:04:50.763 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.763 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.764 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-10T10:04:50.764 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-10T10:04:50.764 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-10T10:04:50.764 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-10T10:04:50.764 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-10T10:04:50.764 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-10T10:04:50.764 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-10T10:04:50.764 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-10T10:04:50.764 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.764 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.764 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.764 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.764 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.764 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-10T10:04:50.765 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-10T10:04:50.765 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.765 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.765 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.765 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-10T10:04:50.765 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.765 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.765 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.765 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.765 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.765 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.765 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.765 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.765 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-10T10:04:50.765 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-10T10:04:50.766 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.766 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.766 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.766 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-10T10:04:50.766 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.766 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.766 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-10T10:04:50.766 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-10T10:04:50.766 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-10T10:04:50.766 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-10T10:04:50.766 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-10T10:04:50.766 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-10T10:04:50.766 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-10T10:04:50.766 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-10T10:04:50.766 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.767 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.767 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.767 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.767 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.767 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-10T10:04:50.767 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-10T10:04:50.767 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.767 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.767 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.767 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-10T10:04:50.767 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.767 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.767 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.767 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.768 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.768 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.768 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.768 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.768 INFO:teuthology.orchestra.run.smithi005.stdout: "retain_interval": { 2024-09-10T10:04:50.768 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "retain_interval", 2024-09-10T10:04:50.768 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-10T10:04:50.768 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.768 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.768 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "31536000", 2024-09-10T10:04:50.768 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.768 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.768 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.768 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-10T10:04:50.769 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.769 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.769 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.769 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.769 INFO:teuthology.orchestra.run.smithi005.stdout: "warn_recent_interval": { 2024-09-10T10:04:50.769 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "warn_recent_interval", 2024-09-10T10:04:50.769 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-10T10:04:50.769 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.769 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.769 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "1209600", 2024-09-10T10:04:50.769 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.769 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.769 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.769 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-10T10:04:50.769 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.770 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.770 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.770 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T10:04:50.770 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T10:04:50.770 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.770 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-10T10:04:50.770 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "dashboard", 2024-09-10T10:04:50.770 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-10T10:04:50.770 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-10T10:04:50.770 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-10T10:04:50.770 INFO:teuthology.orchestra.run.smithi005.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-10T10:04:50.770 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-10T10:04:50.770 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-10T10:04:50.770 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.771 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.771 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10", 2024-09-10T10:04:50.771 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.771 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.771 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.771 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.771 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.771 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.771 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.771 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.771 INFO:teuthology.orchestra.run.smithi005.stdout: "ALERTMANAGER_API_HOST": { 2024-09-10T10:04:50.771 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-10T10:04:50.771 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.771 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.771 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.772 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-10T10:04:50.772 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.772 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.772 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.772 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.772 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.772 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.772 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.772 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.772 INFO:teuthology.orchestra.run.smithi005.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-10T10:04:50.772 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-10T10:04:50.772 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.772 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.772 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.773 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-10T10:04:50.773 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.773 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.773 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.773 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.773 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.773 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.773 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.773 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.773 INFO:teuthology.orchestra.run.smithi005.stdout: "AUDIT_API_ENABLED": { 2024-09-10T10:04:50.773 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "AUDIT_API_ENABLED", 2024-09-10T10:04:50.773 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.773 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.773 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.773 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-10T10:04:50.774 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.774 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.774 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.774 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.774 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.774 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.774 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.774 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.774 INFO:teuthology.orchestra.run.smithi005.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-10T10:04:50.774 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-10T10:04:50.774 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.774 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.774 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.774 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-10T10:04:50.774 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.775 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.775 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.775 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.775 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.775 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.775 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.775 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.775 INFO:teuthology.orchestra.run.smithi005.stdout: "ENABLE_BROWSABLE_API": { 2024-09-10T10:04:50.775 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-10T10:04:50.775 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.775 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.775 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.775 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-10T10:04:50.775 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.776 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.776 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.776 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.776 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.776 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.776 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.776 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.776 INFO:teuthology.orchestra.run.smithi005.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-10T10:04:50.776 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-10T10:04:50.776 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.776 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.776 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.776 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-10T10:04:50.777 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.777 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.777 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.777 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.777 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.777 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.777 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.777 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.777 INFO:teuthology.orchestra.run.smithi005.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-10T10:04:50.777 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-10T10:04:50.777 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.777 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.778 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.778 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-10T10:04:50.778 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.778 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.778 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.778 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.778 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.778 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.778 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.778 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.778 INFO:teuthology.orchestra.run.smithi005.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-10T10:04:50.778 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-10T10:04:50.778 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.778 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.778 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.779 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-10T10:04:50.779 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.779 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.779 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.779 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.779 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.779 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.779 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.779 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.779 INFO:teuthology.orchestra.run.smithi005.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-10T10:04:50.779 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-10T10:04:50.779 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.779 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.779 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.780 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-10T10:04:50.780 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.780 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.780 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.780 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.780 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.780 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.780 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.780 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.780 INFO:teuthology.orchestra.run.smithi005.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-10T10:04:50.780 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-10T10:04:50.780 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.780 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.780 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.780 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-10T10:04:50.781 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.781 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.781 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.781 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.781 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.781 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.781 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.781 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.781 INFO:teuthology.orchestra.run.smithi005.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-10T10:04:50.781 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-10T10:04:50.781 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.781 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.781 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.781 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-10T10:04:50.781 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.782 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.782 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.782 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.782 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.782 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.782 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.782 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.782 INFO:teuthology.orchestra.run.smithi005.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-10T10:04:50.782 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-10T10:04:50.782 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.782 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.782 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.782 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-10T10:04:50.782 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.782 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.783 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.783 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.783 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.783 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.783 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.783 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.783 INFO:teuthology.orchestra.run.smithi005.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-10T10:04:50.783 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-10T10:04:50.783 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.783 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.783 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.783 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-10T10:04:50.783 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.783 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.783 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.783 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.784 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.784 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.784 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.784 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.784 INFO:teuthology.orchestra.run.smithi005.stdout: "GRAFANA_API_PASSWORD": { 2024-09-10T10:04:50.784 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-10T10:04:50.784 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.784 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.784 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.784 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "admin", 2024-09-10T10:04:50.784 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.784 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.784 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.784 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.784 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.785 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.785 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.785 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.785 INFO:teuthology.orchestra.run.smithi005.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-10T10:04:50.785 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-10T10:04:50.785 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.785 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.785 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.785 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-10T10:04:50.785 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.785 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.785 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.785 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.785 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.785 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.786 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.786 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.786 INFO:teuthology.orchestra.run.smithi005.stdout: "GRAFANA_API_URL": { 2024-09-10T10:04:50.786 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "GRAFANA_API_URL", 2024-09-10T10:04:50.786 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.786 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.786 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.786 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-10T10:04:50.786 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.786 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.786 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.786 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.786 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.786 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.787 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.787 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.787 INFO:teuthology.orchestra.run.smithi005.stdout: "GRAFANA_API_USERNAME": { 2024-09-10T10:04:50.787 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-10T10:04:50.787 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.787 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.787 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.787 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "admin", 2024-09-10T10:04:50.787 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.787 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.787 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.787 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.787 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.787 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.788 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.788 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.788 INFO:teuthology.orchestra.run.smithi005.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-10T10:04:50.788 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-10T10:04:50.788 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.788 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.788 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.788 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-10T10:04:50.788 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.788 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.788 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.788 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.788 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.789 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.789 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.789 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.789 INFO:teuthology.orchestra.run.smithi005.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-10T10:04:50.789 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-10T10:04:50.789 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.789 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.789 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.789 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-10T10:04:50.789 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.789 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.789 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.789 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.789 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.790 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.790 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.790 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.790 INFO:teuthology.orchestra.run.smithi005.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-10T10:04:50.790 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-10T10:04:50.790 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.790 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.790 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.790 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-10T10:04:50.790 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.790 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.790 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.790 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.790 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.790 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.791 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.791 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.791 INFO:teuthology.orchestra.run.smithi005.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-10T10:04:50.791 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-10T10:04:50.791 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.791 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.791 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.791 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-10T10:04:50.791 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.791 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.791 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.791 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.791 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.791 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.791 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.791 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.792 INFO:teuthology.orchestra.run.smithi005.stdout: "PROMETHEUS_API_HOST": { 2024-09-10T10:04:50.792 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-10T10:04:50.792 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.792 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.792 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.792 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-10T10:04:50.792 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.792 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.792 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.792 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.792 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.792 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.792 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.792 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.793 INFO:teuthology.orchestra.run.smithi005.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-10T10:04:50.793 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-10T10:04:50.793 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.793 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.793 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.793 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-10T10:04:50.793 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.793 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.793 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.793 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.793 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.793 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.793 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.793 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.793 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-10T10:04:50.794 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-10T10:04:50.794 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.794 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.794 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.794 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-10T10:04:50.794 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.794 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.794 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.794 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.794 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.794 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.794 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.794 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.794 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-10T10:04:50.794 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-10T10:04:50.795 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.795 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.795 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.795 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-10T10:04:50.795 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.795 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.795 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.795 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.795 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.795 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.795 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.795 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.795 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-10T10:04:50.795 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-10T10:04:50.795 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.796 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.796 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.796 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-10T10:04:50.796 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.796 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.796 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.796 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.796 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.796 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.796 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.796 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.796 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-10T10:04:50.796 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-10T10:04:50.797 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.797 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.797 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.797 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-10T10:04:50.797 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.797 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.797 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.797 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.797 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.797 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.797 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.797 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.797 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-10T10:04:50.798 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-10T10:04:50.798 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.798 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.798 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.798 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-10T10:04:50.798 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.798 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.798 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.798 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.798 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.798 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.798 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.798 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.798 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-10T10:04:50.799 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-10T10:04:50.799 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.799 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.799 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.799 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-10T10:04:50.799 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.799 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.799 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.799 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.799 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.799 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.799 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.799 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.799 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-10T10:04:50.800 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-10T10:04:50.800 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.800 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.800 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.800 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-10T10:04:50.800 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.800 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.800 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.800 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.800 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.800 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.800 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.800 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.800 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_ENABLED": { 2024-09-10T10:04:50.801 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-10T10:04:50.801 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.801 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.801 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.801 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-10T10:04:50.801 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.801 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.801 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.801 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.801 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.801 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.801 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.801 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.801 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-10T10:04:50.802 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-10T10:04:50.802 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.802 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.802 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.802 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-10T10:04:50.802 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.802 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.802 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.802 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.802 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.802 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.802 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.802 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.802 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-10T10:04:50.803 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-10T10:04:50.803 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-10T10:04:50.803 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.803 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.803 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10", 2024-09-10T10:04:50.803 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.803 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.803 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.803 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.803 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.803 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.803 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.803 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.803 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-10T10:04:50.804 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-10T10:04:50.804 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-10T10:04:50.804 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.804 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.804 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "8", 2024-09-10T10:04:50.804 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.804 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.804 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.804 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.804 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.804 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.804 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.804 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.805 INFO:teuthology.orchestra.run.smithi005.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-10T10:04:50.805 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-10T10:04:50.805 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-10T10:04:50.805 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.805 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.805 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "45", 2024-09-10T10:04:50.805 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.805 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.805 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.805 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.805 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.805 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.805 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.805 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.806 INFO:teuthology.orchestra.run.smithi005.stdout: "RGW_API_ACCESS_KEY": { 2024-09-10T10:04:50.806 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-10T10:04:50.806 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.806 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.806 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.806 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-10T10:04:50.806 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.806 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.806 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.806 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.806 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.806 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.806 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.806 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.806 INFO:teuthology.orchestra.run.smithi005.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-10T10:04:50.807 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-10T10:04:50.807 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.807 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.807 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.807 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "admin", 2024-09-10T10:04:50.807 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.807 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.807 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.807 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.807 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.807 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.807 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.807 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.807 INFO:teuthology.orchestra.run.smithi005.stdout: "RGW_API_SECRET_KEY": { 2024-09-10T10:04:50.808 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-10T10:04:50.808 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.808 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.808 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.808 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-10T10:04:50.808 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.808 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.808 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.808 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.808 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.808 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.808 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.808 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.808 INFO:teuthology.orchestra.run.smithi005.stdout: "RGW_API_SSL_VERIFY": { 2024-09-10T10:04:50.808 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-10T10:04:50.809 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.809 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.809 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.809 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-10T10:04:50.809 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.809 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.809 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.809 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.809 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.809 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.809 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.809 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.809 INFO:teuthology.orchestra.run.smithi005.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-10T10:04:50.809 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-10T10:04:50.809 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-10T10:04:50.810 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.810 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.810 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-09-10T10:04:50.810 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.810 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.810 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.810 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.810 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.810 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.810 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.810 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.810 INFO:teuthology.orchestra.run.smithi005.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-10T10:04:50.810 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-10T10:04:50.810 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-10T10:04:50.810 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.811 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.811 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10", 2024-09-10T10:04:50.811 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.811 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.811 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.811 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.811 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.811 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.811 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.811 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.811 INFO:teuthology.orchestra.run.smithi005.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-10T10:04:50.811 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-10T10:04:50.811 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-10T10:04:50.811 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.811 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.812 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "5", 2024-09-10T10:04:50.812 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.812 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.812 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.812 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.812 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.812 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.812 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.812 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.812 INFO:teuthology.orchestra.run.smithi005.stdout: "cross_origin_url": { 2024-09-10T10:04:50.812 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "cross_origin_url", 2024-09-10T10:04:50.812 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.812 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.812 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.812 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-10T10:04:50.813 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.813 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.813 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.813 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.813 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.813 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.813 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.813 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.813 INFO:teuthology.orchestra.run.smithi005.stdout: "crt_file": { 2024-09-10T10:04:50.813 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "crt_file", 2024-09-10T10:04:50.813 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.813 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.813 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.813 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-10T10:04:50.814 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.814 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.814 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.814 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.814 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.814 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.814 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.814 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.814 INFO:teuthology.orchestra.run.smithi005.stdout: "debug": { 2024-09-10T10:04:50.814 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "debug", 2024-09-10T10:04:50.814 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.814 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.814 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.814 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-10T10:04:50.815 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.815 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.815 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.815 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Enable/disable debug options", 2024-09-10T10:04:50.815 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.815 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.815 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.815 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.815 INFO:teuthology.orchestra.run.smithi005.stdout: "jwt_token_ttl": { 2024-09-10T10:04:50.815 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "jwt_token_ttl", 2024-09-10T10:04:50.815 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-10T10:04:50.815 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.815 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.815 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "28800", 2024-09-10T10:04:50.816 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.816 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.816 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.816 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.816 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.816 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.816 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.816 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.816 INFO:teuthology.orchestra.run.smithi005.stdout: "key_file": { 2024-09-10T10:04:50.816 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "key_file", 2024-09-10T10:04:50.816 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.816 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.816 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.816 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-10T10:04:50.817 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.817 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.817 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.817 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.817 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.817 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.817 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.817 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.817 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-10T10:04:50.817 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-10T10:04:50.817 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.817 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.817 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.817 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-10T10:04:50.818 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.818 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.818 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-10T10:04:50.818 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-10T10:04:50.818 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-10T10:04:50.818 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-10T10:04:50.818 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-10T10:04:50.818 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-10T10:04:50.818 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-10T10:04:50.818 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-10T10:04:50.818 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.818 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.818 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.819 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.819 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.819 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-10T10:04:50.819 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-10T10:04:50.819 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.819 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.819 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.819 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-10T10:04:50.819 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.819 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.819 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.819 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.819 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.820 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.820 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.820 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.820 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-10T10:04:50.820 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-10T10:04:50.820 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.820 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.820 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.820 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-10T10:04:50.820 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.820 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.820 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-10T10:04:50.820 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-10T10:04:50.820 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-10T10:04:50.821 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-10T10:04:50.821 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-10T10:04:50.821 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-10T10:04:50.821 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-10T10:04:50.821 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-10T10:04:50.821 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.821 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.821 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.821 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.821 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.821 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-10T10:04:50.821 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-10T10:04:50.821 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.821 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.822 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.822 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-10T10:04:50.822 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.822 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.822 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.822 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.822 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.822 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.822 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.822 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.822 INFO:teuthology.orchestra.run.smithi005.stdout: "motd": { 2024-09-10T10:04:50.822 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "motd", 2024-09-10T10:04:50.822 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.822 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.822 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.823 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-10T10:04:50.823 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.823 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.823 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.823 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "The message of the day", 2024-09-10T10:04:50.823 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.823 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.823 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.823 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.823 INFO:teuthology.orchestra.run.smithi005.stdout: "redirect_resolve_ip_addr": { 2024-09-10T10:04:50.823 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "redirect_resolve_ip_addr", 2024-09-10T10:04:50.823 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.823 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.823 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.824 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-10T10:04:50.824 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.824 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.824 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.824 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.824 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.824 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.824 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.824 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.824 INFO:teuthology.orchestra.run.smithi005.stdout: "server_addr": { 2024-09-10T10:04:50.824 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "server_addr", 2024-09-10T10:04:50.824 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.824 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.824 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.825 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "::", 2024-09-10T10:04:50.825 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.825 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.825 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.825 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.825 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.825 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.825 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.825 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.825 INFO:teuthology.orchestra.run.smithi005.stdout: "server_port": { 2024-09-10T10:04:50.825 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "server_port", 2024-09-10T10:04:50.825 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-10T10:04:50.825 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.825 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.826 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "8080", 2024-09-10T10:04:50.826 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.826 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.826 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.826 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.826 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.826 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.826 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.826 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.826 INFO:teuthology.orchestra.run.smithi005.stdout: "ssl": { 2024-09-10T10:04:50.826 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ssl", 2024-09-10T10:04:50.826 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.826 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.826 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.827 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-10T10:04:50.827 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.827 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.827 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.827 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.827 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.827 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.827 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.827 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.827 INFO:teuthology.orchestra.run.smithi005.stdout: "ssl_server_port": { 2024-09-10T10:04:50.827 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ssl_server_port", 2024-09-10T10:04:50.827 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-10T10:04:50.827 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.827 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.828 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "8443", 2024-09-10T10:04:50.828 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.828 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.828 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.828 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.828 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.828 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.828 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.828 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.828 INFO:teuthology.orchestra.run.smithi005.stdout: "standby_behaviour": { 2024-09-10T10:04:50.828 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "standby_behaviour", 2024-09-10T10:04:50.828 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.828 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.828 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.829 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "redirect", 2024-09-10T10:04:50.829 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.829 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.829 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-10T10:04:50.829 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-10T10:04:50.829 INFO:teuthology.orchestra.run.smithi005.stdout: "redirect" 2024-09-10T10:04:50.829 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-10T10:04:50.829 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.829 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.829 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.829 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.829 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.829 INFO:teuthology.orchestra.run.smithi005.stdout: "standby_error_status_code": { 2024-09-10T10:04:50.829 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "standby_error_status_code", 2024-09-10T10:04:50.830 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-10T10:04:50.830 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.830 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.830 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "500", 2024-09-10T10:04:50.830 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "400", 2024-09-10T10:04:50.830 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "599", 2024-09-10T10:04:50.830 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.830 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.830 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.830 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.830 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.830 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.830 INFO:teuthology.orchestra.run.smithi005.stdout: "url_prefix": { 2024-09-10T10:04:50.830 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "url_prefix", 2024-09-10T10:04:50.830 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.831 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.831 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.831 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-10T10:04:50.831 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.831 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.831 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.831 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.831 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.831 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.831 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.831 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T10:04:50.831 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T10:04:50.831 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.831 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-10T10:04:50.831 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "devicehealth", 2024-09-10T10:04:50.831 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-10T10:04:50.832 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-10T10:04:50.832 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-10T10:04:50.832 INFO:teuthology.orchestra.run.smithi005.stdout: "enable_monitoring": { 2024-09-10T10:04:50.832 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "enable_monitoring", 2024-09-10T10:04:50.832 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.832 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.832 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.832 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-10T10:04:50.832 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.832 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.832 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.832 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "monitor device health metrics", 2024-09-10T10:04:50.832 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.832 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.833 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.833 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.833 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-10T10:04:50.833 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-10T10:04:50.833 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.833 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.833 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.833 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-10T10:04:50.833 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.833 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.833 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-10T10:04:50.833 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-10T10:04:50.833 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-10T10:04:50.833 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-10T10:04:50.833 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-10T10:04:50.833 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-10T10:04:50.834 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-10T10:04:50.834 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-10T10:04:50.834 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.834 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.834 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.834 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.834 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.834 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-10T10:04:50.834 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-10T10:04:50.834 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.834 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.834 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.834 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-10T10:04:50.834 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.834 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.835 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.835 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.835 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.835 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.835 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.835 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.835 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-10T10:04:50.835 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-10T10:04:50.835 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.835 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.835 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.835 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-10T10:04:50.835 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.835 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.836 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-10T10:04:50.836 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-10T10:04:50.836 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-10T10:04:50.836 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-10T10:04:50.836 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-10T10:04:50.836 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-10T10:04:50.836 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-10T10:04:50.836 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-10T10:04:50.836 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.836 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.836 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.836 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.836 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.836 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-10T10:04:50.836 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-10T10:04:50.837 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.837 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.837 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.837 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-10T10:04:50.837 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.837 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.837 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.837 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.837 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.837 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.837 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.837 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.837 INFO:teuthology.orchestra.run.smithi005.stdout: "mark_out_threshold": { 2024-09-10T10:04:50.837 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "mark_out_threshold", 2024-09-10T10:04:50.838 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-10T10:04:50.838 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.838 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.838 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "2419200", 2024-09-10T10:04:50.838 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.838 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.838 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.838 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-10T10:04:50.838 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.838 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.838 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.838 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.838 INFO:teuthology.orchestra.run.smithi005.stdout: "pool_name": { 2024-09-10T10:04:50.838 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "pool_name", 2024-09-10T10:04:50.838 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.839 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.839 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.839 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "device_health_metrics", 2024-09-10T10:04:50.839 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.839 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.839 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.839 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-10T10:04:50.839 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.839 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.839 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.839 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.839 INFO:teuthology.orchestra.run.smithi005.stdout: "retention_period": { 2024-09-10T10:04:50.839 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "retention_period", 2024-09-10T10:04:50.839 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-10T10:04:50.840 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.840 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.840 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "15552000", 2024-09-10T10:04:50.840 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.840 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.840 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.840 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "how long to retain device health metrics", 2024-09-10T10:04:50.840 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.840 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.840 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.840 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.840 INFO:teuthology.orchestra.run.smithi005.stdout: "scrape_frequency": { 2024-09-10T10:04:50.840 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "scrape_frequency", 2024-09-10T10:04:50.841 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-10T10:04:50.841 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.841 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.841 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "86400", 2024-09-10T10:04:50.841 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.841 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.841 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.841 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-10T10:04:50.841 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.841 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.841 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.841 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.841 INFO:teuthology.orchestra.run.smithi005.stdout: "self_heal": { 2024-09-10T10:04:50.841 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "self_heal", 2024-09-10T10:04:50.841 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.842 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.842 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.842 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-10T10:04:50.842 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.842 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.842 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.842 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-10T10:04:50.842 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.842 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.842 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.842 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.842 INFO:teuthology.orchestra.run.smithi005.stdout: "sleep_interval": { 2024-09-10T10:04:50.842 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sleep_interval", 2024-09-10T10:04:50.842 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-10T10:04:50.843 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.843 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.843 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "600", 2024-09-10T10:04:50.843 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.843 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.843 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.843 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "how frequently to wake up and check device health", 2024-09-10T10:04:50.843 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.843 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.843 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.843 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.843 INFO:teuthology.orchestra.run.smithi005.stdout: "warn_threshold": { 2024-09-10T10:04:50.843 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "warn_threshold", 2024-09-10T10:04:50.843 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-10T10:04:50.844 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.844 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.844 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "7257600", 2024-09-10T10:04:50.844 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.844 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.844 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.844 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-10T10:04:50.844 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.844 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.844 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.844 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T10:04:50.844 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T10:04:50.844 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.844 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-10T10:04:50.845 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "diskprediction_local", 2024-09-10T10:04:50.845 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-10T10:04:50.845 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-10T10:04:50.845 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-10T10:04:50.845 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-10T10:04:50.845 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-10T10:04:50.845 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.845 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.845 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.845 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-10T10:04:50.845 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.845 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.845 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-10T10:04:50.845 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-10T10:04:50.846 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-10T10:04:50.846 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-10T10:04:50.846 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-10T10:04:50.846 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-10T10:04:50.846 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-10T10:04:50.846 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-10T10:04:50.846 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.846 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.846 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.846 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.846 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.846 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-10T10:04:50.846 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-10T10:04:50.846 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.847 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.847 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.847 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-10T10:04:50.847 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.847 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.847 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.847 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.847 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.847 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.847 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.847 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.847 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-10T10:04:50.847 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-10T10:04:50.847 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.848 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.848 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.848 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-10T10:04:50.848 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.848 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.848 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-10T10:04:50.848 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-10T10:04:50.848 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-10T10:04:50.848 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-10T10:04:50.848 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-10T10:04:50.848 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-10T10:04:50.848 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-10T10:04:50.848 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-10T10:04:50.848 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.848 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.849 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.849 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.849 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.849 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-10T10:04:50.849 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-10T10:04:50.849 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.849 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.849 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.849 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-10T10:04:50.849 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.849 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.849 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.849 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.849 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.849 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.850 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.850 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.850 INFO:teuthology.orchestra.run.smithi005.stdout: "predict_interval": { 2024-09-10T10:04:50.850 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "predict_interval", 2024-09-10T10:04:50.850 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.850 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.850 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.850 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "86400", 2024-09-10T10:04:50.850 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.850 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.850 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.850 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.851 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.851 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.851 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.851 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.851 INFO:teuthology.orchestra.run.smithi005.stdout: "predictor_model": { 2024-09-10T10:04:50.851 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "predictor_model", 2024-09-10T10:04:50.851 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.851 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.851 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.851 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "prophetstor", 2024-09-10T10:04:50.851 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.851 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.851 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.851 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.852 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.852 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.852 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.852 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.852 INFO:teuthology.orchestra.run.smithi005.stdout: "sleep_interval": { 2024-09-10T10:04:50.852 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sleep_interval", 2024-09-10T10:04:50.852 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.852 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.852 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.852 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "600", 2024-09-10T10:04:50.852 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.852 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.852 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.852 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.852 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.853 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.853 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.853 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T10:04:50.853 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T10:04:50.853 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.853 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-10T10:04:50.853 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "influx", 2024-09-10T10:04:50.853 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": false, 2024-09-10T10:04:50.853 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "influxdb python module not found", 2024-09-10T10:04:50.853 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-10T10:04:50.853 INFO:teuthology.orchestra.run.smithi005.stdout: "batch_size": { 2024-09-10T10:04:50.853 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "batch_size", 2024-09-10T10:04:50.853 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-10T10:04:50.853 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.854 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.854 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "5000", 2024-09-10T10:04:50.854 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.854 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.854 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.854 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-10T10:04:50.854 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.854 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.854 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.854 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.854 INFO:teuthology.orchestra.run.smithi005.stdout: "database": { 2024-09-10T10:04:50.854 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "database", 2024-09-10T10:04:50.854 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.854 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.854 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.854 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "ceph", 2024-09-10T10:04:50.855 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.855 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.855 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.855 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-10T10:04:50.855 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.855 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.855 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.855 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.855 INFO:teuthology.orchestra.run.smithi005.stdout: "hostname": { 2024-09-10T10:04:50.855 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "hostname", 2024-09-10T10:04:50.855 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.855 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.855 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.855 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-10T10:04:50.855 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.856 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.856 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.856 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "InfluxDB server hostname", 2024-09-10T10:04:50.856 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.856 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.856 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.856 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.856 INFO:teuthology.orchestra.run.smithi005.stdout: "interval": { 2024-09-10T10:04:50.856 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "interval", 2024-09-10T10:04:50.856 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-10T10:04:50.856 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.856 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.856 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "30", 2024-09-10T10:04:50.857 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "5", 2024-09-10T10:04:50.857 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.857 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.857 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-10T10:04:50.857 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.857 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.857 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.857 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.857 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-10T10:04:50.857 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-10T10:04:50.857 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.857 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.857 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.857 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-10T10:04:50.858 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.858 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.858 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-10T10:04:50.858 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-10T10:04:50.858 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-10T10:04:50.858 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-10T10:04:50.858 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-10T10:04:50.858 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-10T10:04:50.858 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-10T10:04:50.858 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-10T10:04:50.858 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.858 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.858 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.858 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.858 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.859 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-10T10:04:50.859 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-10T10:04:50.859 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.859 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.859 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.859 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-10T10:04:50.859 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.859 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.859 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.859 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.859 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.859 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.859 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.859 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.860 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-10T10:04:50.860 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-10T10:04:50.860 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.860 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.860 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.860 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-10T10:04:50.860 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.860 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.860 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-10T10:04:50.860 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-10T10:04:50.860 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-10T10:04:50.860 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-10T10:04:50.860 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-10T10:04:50.860 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-10T10:04:50.861 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-10T10:04:50.861 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-10T10:04:50.861 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.861 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.861 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.861 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.861 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.861 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-10T10:04:50.861 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-10T10:04:50.861 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.861 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.861 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.861 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-10T10:04:50.861 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.862 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.862 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.862 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.862 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.862 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.862 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.862 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.862 INFO:teuthology.orchestra.run.smithi005.stdout: "password": { 2024-09-10T10:04:50.862 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "password", 2024-09-10T10:04:50.862 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.862 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.862 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.862 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-10T10:04:50.862 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.862 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.863 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.863 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "password of InfluxDB server user", 2024-09-10T10:04:50.863 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.863 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.863 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.863 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.863 INFO:teuthology.orchestra.run.smithi005.stdout: "port": { 2024-09-10T10:04:50.863 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "port", 2024-09-10T10:04:50.863 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-10T10:04:50.863 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.863 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.863 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "8086", 2024-09-10T10:04:50.863 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.863 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.863 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.864 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "InfluxDB server port", 2024-09-10T10:04:50.864 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.864 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.864 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.864 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.864 INFO:teuthology.orchestra.run.smithi005.stdout: "ssl": { 2024-09-10T10:04:50.864 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ssl", 2024-09-10T10:04:50.864 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.864 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.864 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.864 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "false", 2024-09-10T10:04:50.864 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.864 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.864 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.865 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-10T10:04:50.865 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.865 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.865 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.865 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.865 INFO:teuthology.orchestra.run.smithi005.stdout: "threads": { 2024-09-10T10:04:50.865 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "threads", 2024-09-10T10:04:50.865 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-10T10:04:50.865 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.865 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.865 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "5", 2024-09-10T10:04:50.865 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "1", 2024-09-10T10:04:50.865 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "32", 2024-09-10T10:04:50.865 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.865 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-10T10:04:50.866 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.866 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.866 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.866 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.866 INFO:teuthology.orchestra.run.smithi005.stdout: "username": { 2024-09-10T10:04:50.866 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "username", 2024-09-10T10:04:50.866 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.866 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.866 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.866 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-10T10:04:50.866 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.866 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.866 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.866 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "username of InfluxDB server user", 2024-09-10T10:04:50.867 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.867 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.867 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.867 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.867 INFO:teuthology.orchestra.run.smithi005.stdout: "verify_ssl": { 2024-09-10T10:04:50.867 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "verify_ssl", 2024-09-10T10:04:50.867 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.867 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.867 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.867 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "true", 2024-09-10T10:04:50.867 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.867 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.867 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.867 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-10T10:04:50.867 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.868 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.868 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.868 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T10:04:50.868 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T10:04:50.868 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.868 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-10T10:04:50.868 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "insights", 2024-09-10T10:04:50.868 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-10T10:04:50.868 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-10T10:04:50.868 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-10T10:04:50.868 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-10T10:04:50.868 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-10T10:04:50.868 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.868 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.869 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.869 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-10T10:04:50.869 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.869 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.869 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-10T10:04:50.869 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-10T10:04:50.869 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-10T10:04:50.869 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-10T10:04:50.869 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-10T10:04:50.869 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-10T10:04:50.869 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-10T10:04:50.869 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-10T10:04:50.869 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.869 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.869 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.869 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.870 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.870 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-10T10:04:50.870 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-10T10:04:50.870 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.870 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.870 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.870 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-10T10:04:50.870 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.870 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.870 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.870 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.870 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.870 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.870 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.870 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.871 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-10T10:04:50.871 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-10T10:04:50.871 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.871 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.871 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.871 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-10T10:04:50.871 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.871 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.871 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-10T10:04:50.871 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-10T10:04:50.871 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-10T10:04:50.871 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-10T10:04:50.871 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-10T10:04:50.872 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-10T10:04:50.872 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-10T10:04:50.872 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-10T10:04:50.872 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.872 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.872 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.872 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.872 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.872 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-10T10:04:50.872 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-10T10:04:50.872 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.872 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.872 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.872 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-10T10:04:50.872 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.873 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.873 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.873 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.873 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.873 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.873 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.873 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T10:04:50.873 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T10:04:50.873 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.873 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-10T10:04:50.873 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "iostat", 2024-09-10T10:04:50.873 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-10T10:04:50.873 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-10T10:04:50.873 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-10T10:04:50.874 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-10T10:04:50.874 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-10T10:04:50.874 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.874 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.874 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.874 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-10T10:04:50.874 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.874 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.874 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-10T10:04:50.874 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-10T10:04:50.874 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-10T10:04:50.874 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-10T10:04:50.874 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-10T10:04:50.874 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-10T10:04:50.874 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-10T10:04:50.875 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-10T10:04:50.875 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.875 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.875 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.875 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.875 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.875 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-10T10:04:50.875 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-10T10:04:50.875 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.875 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.875 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.875 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-10T10:04:50.875 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.875 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.875 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.876 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.876 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.876 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.876 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.876 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.876 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-10T10:04:50.876 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-10T10:04:50.876 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.876 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.876 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.876 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-10T10:04:50.876 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.876 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.876 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-10T10:04:50.877 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-10T10:04:50.877 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-10T10:04:50.877 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-10T10:04:50.877 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-10T10:04:50.877 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-10T10:04:50.877 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-10T10:04:50.877 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-10T10:04:50.877 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.877 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.877 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.877 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.877 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.877 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-10T10:04:50.877 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-10T10:04:50.878 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.878 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.878 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.878 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-10T10:04:50.878 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.878 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.878 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.878 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.878 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.878 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.878 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.878 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T10:04:50.878 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T10:04:50.878 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.879 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-10T10:04:50.879 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "k8sevents", 2024-09-10T10:04:50.879 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-10T10:04:50.879 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-10T10:04:50.879 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-10T10:04:50.879 INFO:teuthology.orchestra.run.smithi005.stdout: "ceph_event_retention_days": { 2024-09-10T10:04:50.879 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ceph_event_retention_days", 2024-09-10T10:04:50.879 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-10T10:04:50.879 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.879 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.879 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "7", 2024-09-10T10:04:50.879 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.879 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.879 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.880 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Days to hold ceph event information within local cache", 2024-09-10T10:04:50.880 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.880 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.880 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.880 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.880 INFO:teuthology.orchestra.run.smithi005.stdout: "config_check_secs": { 2024-09-10T10:04:50.880 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "config_check_secs", 2024-09-10T10:04:50.880 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-10T10:04:50.880 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.880 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.880 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10", 2024-09-10T10:04:50.880 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "10", 2024-09-10T10:04:50.880 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.880 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.881 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-09-10T10:04:50.881 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.881 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.881 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.881 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.881 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-10T10:04:50.881 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-10T10:04:50.881 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.881 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.881 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.881 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-10T10:04:50.881 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.881 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.881 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-10T10:04:50.882 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-10T10:04:50.882 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-10T10:04:50.882 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-10T10:04:50.882 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-10T10:04:50.882 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-10T10:04:50.882 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-10T10:04:50.882 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-10T10:04:50.882 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.882 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.882 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.882 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.882 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.882 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-10T10:04:50.883 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-10T10:04:50.883 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.883 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.883 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.883 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-10T10:04:50.883 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.883 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.883 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.883 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.883 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.883 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.883 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.883 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.884 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-10T10:04:50.884 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-10T10:04:50.884 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.884 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.884 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.884 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-10T10:04:50.884 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.884 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.884 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-10T10:04:50.884 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-10T10:04:50.884 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-10T10:04:50.884 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-10T10:04:50.884 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-10T10:04:50.884 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-10T10:04:50.885 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-10T10:04:50.885 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-10T10:04:50.885 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.885 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.885 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.885 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.885 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.885 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-10T10:04:50.885 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-10T10:04:50.885 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.885 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.885 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.885 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-10T10:04:50.885 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.886 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.886 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.886 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.886 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.886 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.886 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.886 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T10:04:50.886 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T10:04:50.886 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.886 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-10T10:04:50.886 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "localpool", 2024-09-10T10:04:50.886 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-10T10:04:50.886 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-10T10:04:50.886 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-10T10:04:50.887 INFO:teuthology.orchestra.run.smithi005.stdout: "failure_domain": { 2024-09-10T10:04:50.887 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "failure_domain", 2024-09-10T10:04:50.887 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.887 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.887 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.887 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "host", 2024-09-10T10:04:50.887 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.887 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.887 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.887 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "failure domain for any created local pool", 2024-09-10T10:04:50.887 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-10T10:04:50.887 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.887 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.888 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.888 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-10T10:04:50.888 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-10T10:04:50.888 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.888 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.888 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.888 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-10T10:04:50.888 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.888 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.888 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-10T10:04:50.888 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-10T10:04:50.888 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-10T10:04:50.888 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-10T10:04:50.889 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-10T10:04:50.889 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-10T10:04:50.889 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-10T10:04:50.889 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-10T10:04:50.889 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.889 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.889 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.889 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.889 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.889 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-10T10:04:50.889 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-10T10:04:50.889 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.889 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.890 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.890 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-10T10:04:50.890 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.890 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.890 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.890 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.890 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.890 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.890 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.890 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.890 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-10T10:04:50.890 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-10T10:04:50.890 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.891 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.891 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.891 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-10T10:04:50.891 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.891 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.891 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-10T10:04:50.891 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-10T10:04:50.891 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-10T10:04:50.891 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-10T10:04:50.891 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-10T10:04:50.891 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-10T10:04:50.891 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-10T10:04:50.891 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-10T10:04:50.891 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.892 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.892 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.892 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.892 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.892 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-10T10:04:50.892 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-10T10:04:50.892 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.892 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.892 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.892 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-10T10:04:50.892 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.892 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.893 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.893 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.893 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.893 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.893 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.893 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.893 INFO:teuthology.orchestra.run.smithi005.stdout: "min_size": { 2024-09-10T10:04:50.893 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "min_size", 2024-09-10T10:04:50.893 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-10T10:04:50.893 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.893 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.893 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-10T10:04:50.893 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.894 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.894 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.894 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "default min_size for any created local pool", 2024-09-10T10:04:50.894 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-10T10:04:50.894 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.894 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.894 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.894 INFO:teuthology.orchestra.run.smithi005.stdout: "num_rep": { 2024-09-10T10:04:50.894 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "num_rep", 2024-09-10T10:04:50.894 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-10T10:04:50.894 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.894 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.894 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "3", 2024-09-10T10:04:50.895 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.895 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.895 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.895 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "default replica count for any created local pool", 2024-09-10T10:04:50.895 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.895 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.895 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.895 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.895 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_num": { 2024-09-10T10:04:50.895 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "pg_num", 2024-09-10T10:04:50.895 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-10T10:04:50.895 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.895 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.895 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "128", 2024-09-10T10:04:50.896 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.896 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.896 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.896 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "default pg_num for any created local pool", 2024-09-10T10:04:50.896 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.896 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.896 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.896 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.896 INFO:teuthology.orchestra.run.smithi005.stdout: "prefix": { 2024-09-10T10:04:50.896 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "prefix", 2024-09-10T10:04:50.896 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.896 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.896 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.897 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-10T10:04:50.897 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.897 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.897 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.897 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "name prefix for any created local pool", 2024-09-10T10:04:50.897 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.897 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.897 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.897 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.897 INFO:teuthology.orchestra.run.smithi005.stdout: "subtree": { 2024-09-10T10:04:50.897 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "subtree", 2024-09-10T10:04:50.897 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.897 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.897 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.898 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "rack", 2024-09-10T10:04:50.898 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.898 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.898 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.898 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-10T10:04:50.898 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-10T10:04:50.898 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.898 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.898 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T10:04:50.898 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T10:04:50.898 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.898 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-10T10:04:50.898 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "mds_autoscaler", 2024-09-10T10:04:50.898 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-10T10:04:50.899 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-10T10:04:50.899 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-10T10:04:50.899 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-10T10:04:50.899 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-10T10:04:50.899 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.899 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.899 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.899 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-10T10:04:50.899 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.899 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.899 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-10T10:04:50.899 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-10T10:04:50.899 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-10T10:04:50.899 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-10T10:04:50.899 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-10T10:04:50.900 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-10T10:04:50.900 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-10T10:04:50.900 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-10T10:04:50.900 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.900 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.900 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.900 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.900 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.900 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-10T10:04:50.900 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-10T10:04:50.900 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.900 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.900 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.900 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-10T10:04:50.901 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.901 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.901 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.901 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.901 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.901 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.901 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.901 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.901 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-10T10:04:50.901 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-10T10:04:50.901 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.901 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.901 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.901 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-10T10:04:50.901 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.902 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.902 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-10T10:04:50.902 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-10T10:04:50.902 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-10T10:04:50.902 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-10T10:04:50.902 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-10T10:04:50.902 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-10T10:04:50.902 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-10T10:04:50.902 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-10T10:04:50.902 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.902 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.902 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.902 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.902 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.902 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-10T10:04:50.903 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-10T10:04:50.903 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.903 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.903 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.903 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-10T10:04:50.903 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.903 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.903 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.903 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.903 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.903 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.903 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.903 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T10:04:50.903 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T10:04:50.903 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.904 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-10T10:04:50.904 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "mirroring", 2024-09-10T10:04:50.904 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-10T10:04:50.904 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-10T10:04:50.904 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-10T10:04:50.904 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-10T10:04:50.904 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-10T10:04:50.904 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.904 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.904 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.904 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-10T10:04:50.904 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.904 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.904 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-10T10:04:50.904 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-10T10:04:50.905 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-10T10:04:50.905 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-10T10:04:50.905 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-10T10:04:50.905 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-10T10:04:50.905 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-10T10:04:50.905 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-10T10:04:50.905 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.905 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.905 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.905 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.905 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.905 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-10T10:04:50.905 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-10T10:04:50.906 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.906 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.906 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.906 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-10T10:04:50.906 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.906 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.906 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.906 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.906 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.906 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.906 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.906 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.906 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-10T10:04:50.906 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-10T10:04:50.907 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.907 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.907 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.907 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-10T10:04:50.907 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.907 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.907 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-10T10:04:50.907 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-10T10:04:50.907 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-10T10:04:50.907 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-10T10:04:50.907 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-10T10:04:50.907 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-10T10:04:50.907 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-10T10:04:50.907 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-10T10:04:50.908 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.908 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.908 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.908 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.908 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.908 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-10T10:04:50.908 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-10T10:04:50.908 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.908 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.908 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.908 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-10T10:04:50.908 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.908 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.908 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.908 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.909 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.909 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.909 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.909 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T10:04:50.909 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T10:04:50.909 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.909 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-10T10:04:50.909 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "nfs", 2024-09-10T10:04:50.909 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-10T10:04:50.909 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-10T10:04:50.909 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-10T10:04:50.909 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-10T10:04:50.909 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-10T10:04:50.909 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.910 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.910 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.910 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-10T10:04:50.910 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.910 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.910 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-10T10:04:50.910 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-10T10:04:50.910 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-10T10:04:50.910 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-10T10:04:50.910 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-10T10:04:50.910 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-10T10:04:50.910 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-10T10:04:50.910 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-10T10:04:50.910 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.911 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.911 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.911 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.911 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.911 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-10T10:04:50.911 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-10T10:04:50.911 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.911 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.911 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.911 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-10T10:04:50.911 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.911 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.911 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.911 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.912 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.912 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.912 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.912 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.912 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-10T10:04:50.912 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-10T10:04:50.912 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.912 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.912 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.912 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-10T10:04:50.912 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.912 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.912 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-10T10:04:50.912 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-10T10:04:50.912 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-10T10:04:50.913 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-10T10:04:50.913 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-10T10:04:50.913 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-10T10:04:50.913 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-10T10:04:50.913 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-10T10:04:50.913 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.913 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.913 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.913 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.913 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.913 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-10T10:04:50.913 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-10T10:04:50.913 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.913 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.914 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.914 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-10T10:04:50.914 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.914 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.914 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.914 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.914 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.914 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.914 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.914 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T10:04:50.914 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T10:04:50.914 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.914 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-10T10:04:50.915 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "orchestrator", 2024-09-10T10:04:50.915 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-10T10:04:50.915 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-10T10:04:50.915 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-10T10:04:50.915 INFO:teuthology.orchestra.run.smithi005.stdout: "fail_fs": { 2024-09-10T10:04:50.915 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "fail_fs", 2024-09-10T10:04:50.915 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.915 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.915 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.915 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-10T10:04:50.915 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.915 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.915 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.915 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-10T10:04:50.915 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.916 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.916 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.916 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.916 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-10T10:04:50.916 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-10T10:04:50.916 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.916 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.916 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.916 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-10T10:04:50.916 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.916 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.916 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-10T10:04:50.916 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-10T10:04:50.917 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-10T10:04:50.917 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-10T10:04:50.917 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-10T10:04:50.917 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-10T10:04:50.917 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-10T10:04:50.917 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-10T10:04:50.917 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.917 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.917 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.917 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.917 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.917 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-10T10:04:50.917 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-10T10:04:50.917 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.918 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.918 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.918 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-10T10:04:50.918 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.918 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.918 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.918 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.918 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.918 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.918 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.918 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.918 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-10T10:04:50.918 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-10T10:04:50.918 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.919 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.919 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.919 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-10T10:04:50.919 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.919 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.919 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-10T10:04:50.919 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-10T10:04:50.919 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-10T10:04:50.919 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-10T10:04:50.919 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-10T10:04:50.919 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-10T10:04:50.919 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-10T10:04:50.919 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-10T10:04:50.919 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.920 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.920 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.920 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.920 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.920 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-10T10:04:50.920 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-10T10:04:50.920 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.920 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.920 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.920 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-10T10:04:50.920 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.920 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.920 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.921 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.921 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.921 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.921 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.921 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.921 INFO:teuthology.orchestra.run.smithi005.stdout: "orchestrator": { 2024-09-10T10:04:50.921 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "orchestrator", 2024-09-10T10:04:50.921 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.921 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.921 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.921 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-10T10:04:50.921 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.921 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.921 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-10T10:04:50.922 INFO:teuthology.orchestra.run.smithi005.stdout: "cephadm", 2024-09-10T10:04:50.922 INFO:teuthology.orchestra.run.smithi005.stdout: "rook", 2024-09-10T10:04:50.922 INFO:teuthology.orchestra.run.smithi005.stdout: "test_orchestrator" 2024-09-10T10:04:50.922 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-10T10:04:50.922 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Orchestrator backend", 2024-09-10T10:04:50.922 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.922 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.922 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.922 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T10:04:50.922 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T10:04:50.922 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.922 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-10T10:04:50.922 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "osd_perf_query", 2024-09-10T10:04:50.922 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-10T10:04:50.923 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-10T10:04:50.923 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-10T10:04:50.923 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-10T10:04:50.923 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-10T10:04:50.923 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.923 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.923 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.923 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-10T10:04:50.923 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.923 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.923 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-10T10:04:50.923 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-10T10:04:50.923 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-10T10:04:50.924 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-10T10:04:50.924 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-10T10:04:50.924 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-10T10:04:50.924 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-10T10:04:50.924 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-10T10:04:50.924 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.924 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.924 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.924 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.924 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.924 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-10T10:04:50.924 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-10T10:04:50.924 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.924 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.925 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.925 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-10T10:04:50.925 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.925 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.925 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.925 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.925 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.925 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.925 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.925 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.925 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-10T10:04:50.925 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-10T10:04:50.925 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.926 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.926 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.926 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-10T10:04:50.926 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.926 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.926 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-10T10:04:50.926 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-10T10:04:50.926 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-10T10:04:50.926 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-10T10:04:50.926 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-10T10:04:50.926 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-10T10:04:50.926 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-10T10:04:50.926 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-10T10:04:50.926 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.927 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.927 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.927 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.927 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.927 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-10T10:04:50.927 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-10T10:04:50.927 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.927 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.927 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.927 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-10T10:04:50.927 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.927 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.927 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.927 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.928 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.928 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.928 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.928 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T10:04:50.928 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T10:04:50.928 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.928 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-10T10:04:50.928 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "osd_support", 2024-09-10T10:04:50.928 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-10T10:04:50.928 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-10T10:04:50.928 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-10T10:04:50.928 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-10T10:04:50.928 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-10T10:04:50.928 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.929 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.929 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.929 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-10T10:04:50.929 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.929 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.929 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-10T10:04:50.929 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-10T10:04:50.929 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-10T10:04:50.929 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-10T10:04:50.929 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-10T10:04:50.929 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-10T10:04:50.929 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-10T10:04:50.929 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-10T10:04:50.929 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.930 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.930 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.930 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.930 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.930 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-10T10:04:50.930 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-10T10:04:50.930 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.930 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.930 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.930 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-10T10:04:50.930 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.930 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.930 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.930 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.931 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.931 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.931 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.931 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.931 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-10T10:04:50.931 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-10T10:04:50.931 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.931 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.931 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.931 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-10T10:04:50.931 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.931 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.931 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-10T10:04:50.931 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-10T10:04:50.931 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-10T10:04:50.932 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-10T10:04:50.932 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-10T10:04:50.932 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-10T10:04:50.932 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-10T10:04:50.932 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-10T10:04:50.932 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.932 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.932 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.932 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.932 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.932 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-10T10:04:50.932 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-10T10:04:50.932 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.932 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.933 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.933 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-10T10:04:50.933 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.933 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.933 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.933 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.933 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.933 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.933 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.933 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T10:04:50.933 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T10:04:50.933 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.933 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-10T10:04:50.933 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "pg_autoscaler", 2024-09-10T10:04:50.934 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-10T10:04:50.934 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-10T10:04:50.934 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-10T10:04:50.934 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-10T10:04:50.934 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-10T10:04:50.934 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.934 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.934 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.934 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-10T10:04:50.934 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.934 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.934 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-10T10:04:50.934 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-10T10:04:50.934 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-10T10:04:50.935 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-10T10:04:50.935 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-10T10:04:50.935 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-10T10:04:50.935 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-10T10:04:50.935 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-10T10:04:50.935 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.935 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.935 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.935 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.935 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.935 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-10T10:04:50.935 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-10T10:04:50.935 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.936 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.936 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.936 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-10T10:04:50.936 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.936 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.936 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.936 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.936 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.936 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.936 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.936 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.936 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-10T10:04:50.936 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-10T10:04:50.936 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.937 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.937 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.937 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-10T10:04:50.937 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.937 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.937 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-10T10:04:50.937 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-10T10:04:50.937 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-10T10:04:50.937 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-10T10:04:50.937 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-10T10:04:50.937 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-10T10:04:50.937 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-10T10:04:50.937 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-10T10:04:50.937 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.938 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.938 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.938 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.938 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.938 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-10T10:04:50.938 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-10T10:04:50.938 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.938 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.938 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.938 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-10T10:04:50.938 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.938 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.938 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.938 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.938 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.939 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.939 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.939 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.939 INFO:teuthology.orchestra.run.smithi005.stdout: "sleep_interval": { 2024-09-10T10:04:50.939 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sleep_interval", 2024-09-10T10:04:50.939 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-10T10:04:50.939 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.939 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.939 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "60", 2024-09-10T10:04:50.939 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.939 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.939 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.939 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.939 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.940 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.940 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.940 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.940 INFO:teuthology.orchestra.run.smithi005.stdout: "threshold": { 2024-09-10T10:04:50.940 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "threshold", 2024-09-10T10:04:50.940 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "float", 2024-09-10T10:04:50.940 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.940 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.940 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "3.0", 2024-09-10T10:04:50.940 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "1.0", 2024-09-10T10:04:50.940 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.940 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.940 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "scaling threshold", 2024-09-10T10:04:50.940 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-10T10:04:50.940 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.941 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.941 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T10:04:50.941 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T10:04:50.941 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.941 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-10T10:04:50.941 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "progress", 2024-09-10T10:04:50.941 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-10T10:04:50.941 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-10T10:04:50.941 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-10T10:04:50.941 INFO:teuthology.orchestra.run.smithi005.stdout: "allow_pg_recovery_event": { 2024-09-10T10:04:50.941 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "allow_pg_recovery_event", 2024-09-10T10:04:50.941 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.941 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.941 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.941 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-10T10:04:50.942 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.942 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.942 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.942 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-10T10:04:50.942 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.942 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.942 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.942 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.942 INFO:teuthology.orchestra.run.smithi005.stdout: "enabled": { 2024-09-10T10:04:50.942 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "enabled", 2024-09-10T10:04:50.942 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.942 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.942 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.942 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-10T10:04:50.942 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.943 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.943 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.943 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.943 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.943 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.943 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.943 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.943 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-10T10:04:50.943 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-10T10:04:50.943 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.943 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.943 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.943 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-10T10:04:50.943 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.943 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.944 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-10T10:04:50.944 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-10T10:04:50.944 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-10T10:04:50.944 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-10T10:04:50.944 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-10T10:04:50.944 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-10T10:04:50.944 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-10T10:04:50.944 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-10T10:04:50.944 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.944 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.944 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.944 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.944 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.944 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-10T10:04:50.945 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-10T10:04:50.945 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.945 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.945 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.945 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-10T10:04:50.945 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.945 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.945 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.945 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.945 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.945 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.945 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.945 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.945 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-10T10:04:50.945 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-10T10:04:50.946 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.946 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.946 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.946 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-10T10:04:50.946 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.946 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.946 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-10T10:04:50.946 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-10T10:04:50.946 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-10T10:04:50.946 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-10T10:04:50.946 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-10T10:04:50.946 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-10T10:04:50.946 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-10T10:04:50.946 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-10T10:04:50.946 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.947 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.947 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.947 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.947 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.947 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-10T10:04:50.947 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-10T10:04:50.947 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.947 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.947 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.947 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-10T10:04:50.947 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.947 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.947 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.947 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.948 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.948 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.948 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.948 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.948 INFO:teuthology.orchestra.run.smithi005.stdout: "max_completed_events": { 2024-09-10T10:04:50.948 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "max_completed_events", 2024-09-10T10:04:50.948 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-10T10:04:50.948 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.948 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.948 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "50", 2024-09-10T10:04:50.948 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.948 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.948 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.948 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "number of past completed events to remember", 2024-09-10T10:04:50.949 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.949 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.949 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.949 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.949 INFO:teuthology.orchestra.run.smithi005.stdout: "sleep_interval": { 2024-09-10T10:04:50.949 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sleep_interval", 2024-09-10T10:04:50.949 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-10T10:04:50.949 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.949 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.949 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "5", 2024-09-10T10:04:50.949 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.949 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.949 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.950 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "how long the module is going to sleep", 2024-09-10T10:04:50.950 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.950 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.950 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.950 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T10:04:50.950 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T10:04:50.950 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.950 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-10T10:04:50.950 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "prometheus", 2024-09-10T10:04:50.950 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-10T10:04:50.950 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-10T10:04:50.950 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-10T10:04:50.950 INFO:teuthology.orchestra.run.smithi005.stdout: "cache": { 2024-09-10T10:04:50.951 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "cache", 2024-09-10T10:04:50.951 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.951 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.951 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.951 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-10T10:04:50.951 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.951 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.951 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.951 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.951 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.951 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.951 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.951 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.952 INFO:teuthology.orchestra.run.smithi005.stdout: "exclude_perf_counters": { 2024-09-10T10:04:50.952 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "exclude_perf_counters", 2024-09-10T10:04:50.952 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.952 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.952 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.952 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-10T10:04:50.952 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.952 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.952 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.952 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-10T10:04:50.952 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-10T10:04:50.952 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.952 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.952 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.953 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-10T10:04:50.953 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-10T10:04:50.953 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.953 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.953 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.953 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-10T10:04:50.953 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.953 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.953 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-10T10:04:50.953 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-10T10:04:50.953 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-10T10:04:50.953 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-10T10:04:50.953 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-10T10:04:50.954 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-10T10:04:50.954 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-10T10:04:50.954 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-10T10:04:50.954 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.954 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.954 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.954 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.954 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.954 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-10T10:04:50.954 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-10T10:04:50.954 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.954 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.954 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.954 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-10T10:04:50.955 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.955 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.955 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.955 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.955 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.955 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.955 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.955 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.955 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-10T10:04:50.955 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-10T10:04:50.955 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.955 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.955 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.956 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-10T10:04:50.956 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.956 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.956 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-10T10:04:50.956 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-10T10:04:50.956 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-10T10:04:50.956 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-10T10:04:50.956 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-10T10:04:50.956 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-10T10:04:50.956 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-10T10:04:50.956 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-10T10:04:50.956 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.956 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.956 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.957 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.957 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.957 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-10T10:04:50.957 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-10T10:04:50.957 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.957 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.957 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.957 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-10T10:04:50.957 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.957 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.957 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.957 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.958 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.958 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.958 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.958 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.958 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_stats_pools": { 2024-09-10T10:04:50.958 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rbd_stats_pools", 2024-09-10T10:04:50.958 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.958 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.958 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.958 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-10T10:04:50.958 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.958 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.958 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.958 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.959 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.959 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.959 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.959 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.959 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-10T10:04:50.959 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-10T10:04:50.959 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-10T10:04:50.959 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.959 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.959 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "300", 2024-09-10T10:04:50.959 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.959 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.959 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.959 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.959 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.960 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.960 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.960 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.960 INFO:teuthology.orchestra.run.smithi005.stdout: "scrape_interval": { 2024-09-10T10:04:50.960 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "scrape_interval", 2024-09-10T10:04:50.960 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "float", 2024-09-10T10:04:50.960 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.960 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.960 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "15.0", 2024-09-10T10:04:50.960 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.960 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.960 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.960 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.960 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.960 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.961 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.961 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.961 INFO:teuthology.orchestra.run.smithi005.stdout: "server_addr": { 2024-09-10T10:04:50.961 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "server_addr", 2024-09-10T10:04:50.961 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.961 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.961 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.961 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "::", 2024-09-10T10:04:50.961 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.961 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.961 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.961 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-10T10:04:50.961 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.961 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.961 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.962 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.962 INFO:teuthology.orchestra.run.smithi005.stdout: "server_port": { 2024-09-10T10:04:50.962 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "server_port", 2024-09-10T10:04:50.962 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-10T10:04:50.962 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.962 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.962 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "9283", 2024-09-10T10:04:50.962 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.962 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.962 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.962 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-10T10:04:50.962 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.962 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.962 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.962 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.963 INFO:teuthology.orchestra.run.smithi005.stdout: "stale_cache_strategy": { 2024-09-10T10:04:50.963 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "stale_cache_strategy", 2024-09-10T10:04:50.963 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.963 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.963 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.963 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "log", 2024-09-10T10:04:50.963 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.963 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.963 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.963 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.963 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.963 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.963 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.963 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.964 INFO:teuthology.orchestra.run.smithi005.stdout: "standby_behaviour": { 2024-09-10T10:04:50.964 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "standby_behaviour", 2024-09-10T10:04:50.964 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.964 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.964 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.964 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "default", 2024-09-10T10:04:50.964 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.964 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.964 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-10T10:04:50.964 INFO:teuthology.orchestra.run.smithi005.stdout: "default", 2024-09-10T10:04:50.964 INFO:teuthology.orchestra.run.smithi005.stdout: "error" 2024-09-10T10:04:50.964 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-10T10:04:50.964 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.964 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.965 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.965 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.965 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.965 INFO:teuthology.orchestra.run.smithi005.stdout: "standby_error_status_code": { 2024-09-10T10:04:50.965 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "standby_error_status_code", 2024-09-10T10:04:50.965 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-10T10:04:50.965 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.965 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.965 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "500", 2024-09-10T10:04:50.965 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "400", 2024-09-10T10:04:50.965 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "599", 2024-09-10T10:04:50.965 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.965 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.965 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.965 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.966 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.966 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T10:04:50.966 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T10:04:50.966 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.966 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-10T10:04:50.966 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rbd_support", 2024-09-10T10:04:50.966 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-10T10:04:50.966 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-10T10:04:50.966 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-10T10:04:50.966 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-10T10:04:50.966 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-10T10:04:50.966 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.966 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.966 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.967 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-10T10:04:50.967 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.967 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.967 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-10T10:04:50.967 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-10T10:04:50.967 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-10T10:04:50.967 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-10T10:04:50.967 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-10T10:04:50.967 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-10T10:04:50.967 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-10T10:04:50.967 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-10T10:04:50.967 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.967 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.968 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.968 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.968 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.968 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-10T10:04:50.968 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-10T10:04:50.968 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.968 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.968 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.968 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-10T10:04:50.968 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.968 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.968 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.968 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.968 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.969 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.969 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.969 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.969 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-10T10:04:50.969 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-10T10:04:50.969 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.969 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.969 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.969 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-10T10:04:50.969 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.969 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.969 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-10T10:04:50.969 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-10T10:04:50.969 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-10T10:04:50.970 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-10T10:04:50.970 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-10T10:04:50.970 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-10T10:04:50.970 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-10T10:04:50.970 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-10T10:04:50.970 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.970 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.970 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.970 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.970 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.970 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-10T10:04:50.970 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-10T10:04:50.970 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.970 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.970 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.970 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-10T10:04:50.971 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.971 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.971 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.971 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.971 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.971 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.971 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.971 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.971 INFO:teuthology.orchestra.run.smithi005.stdout: "max_concurrent_snap_create": { 2024-09-10T10:04:50.971 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "max_concurrent_snap_create", 2024-09-10T10:04:50.971 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-10T10:04:50.971 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.971 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.971 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10", 2024-09-10T10:04:50.971 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.972 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.972 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.972 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.972 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.972 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.972 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.972 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.972 INFO:teuthology.orchestra.run.smithi005.stdout: "mirror_snapshot_schedule": { 2024-09-10T10:04:50.972 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "mirror_snapshot_schedule", 2024-09-10T10:04:50.972 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.972 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.972 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.972 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-10T10:04:50.972 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.972 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.973 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.973 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.973 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.973 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.973 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.973 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.973 INFO:teuthology.orchestra.run.smithi005.stdout: "trash_purge_schedule": { 2024-09-10T10:04:50.973 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "trash_purge_schedule", 2024-09-10T10:04:50.973 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.973 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.973 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.973 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-10T10:04:50.973 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.973 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.973 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.974 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.974 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.974 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.974 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.974 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T10:04:50.974 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T10:04:50.974 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.974 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-10T10:04:50.974 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "restful", 2024-09-10T10:04:50.974 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-10T10:04:50.974 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-10T10:04:50.974 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-10T10:04:50.974 INFO:teuthology.orchestra.run.smithi005.stdout: "enable_auth": { 2024-09-10T10:04:50.974 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "enable_auth", 2024-09-10T10:04:50.975 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.975 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.975 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.975 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-10T10:04:50.975 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.975 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.975 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.975 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.975 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.975 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.975 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.975 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.975 INFO:teuthology.orchestra.run.smithi005.stdout: "key_file": { 2024-09-10T10:04:50.975 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "key_file", 2024-09-10T10:04:50.975 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.975 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.976 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.976 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-10T10:04:50.976 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.976 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.976 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.976 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.976 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.976 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.976 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.976 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.976 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-10T10:04:50.976 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-10T10:04:50.976 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.976 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.977 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.977 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-10T10:04:50.977 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.977 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.977 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-10T10:04:50.977 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-10T10:04:50.977 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-10T10:04:50.977 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-10T10:04:50.977 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-10T10:04:50.977 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-10T10:04:50.977 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-10T10:04:50.977 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-10T10:04:50.977 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.977 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.978 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.978 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.978 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.978 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-10T10:04:50.978 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-10T10:04:50.978 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.978 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.978 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.978 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-10T10:04:50.978 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.978 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.978 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.978 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.978 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.978 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.978 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.979 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.979 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-10T10:04:50.979 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-10T10:04:50.979 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.979 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.979 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.979 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-10T10:04:50.979 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.979 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.979 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-10T10:04:50.979 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-10T10:04:50.979 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-10T10:04:50.979 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-10T10:04:50.979 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-10T10:04:50.979 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-10T10:04:50.980 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-10T10:04:50.980 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-10T10:04:50.980 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.980 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.980 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.980 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.980 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.980 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-10T10:04:50.980 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-10T10:04:50.980 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.980 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.980 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.980 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-10T10:04:50.980 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.980 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.981 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.981 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.981 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.981 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.981 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.981 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.981 INFO:teuthology.orchestra.run.smithi005.stdout: "server_addr": { 2024-09-10T10:04:50.981 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "server_addr", 2024-09-10T10:04:50.981 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.981 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.981 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.981 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-10T10:04:50.981 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.981 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.982 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.982 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.982 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.982 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.982 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.982 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.982 INFO:teuthology.orchestra.run.smithi005.stdout: "server_port": { 2024-09-10T10:04:50.982 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "server_port", 2024-09-10T10:04:50.982 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.982 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.982 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.982 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-10T10:04:50.982 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.982 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.983 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.983 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.983 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.983 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.983 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.983 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T10:04:50.983 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T10:04:50.983 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.983 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-10T10:04:50.983 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rgw", 2024-09-10T10:04:50.983 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-10T10:04:50.983 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-10T10:04:50.983 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-10T10:04:50.984 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-10T10:04:50.984 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-10T10:04:50.984 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.984 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.984 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.984 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-10T10:04:50.984 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.984 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.984 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-10T10:04:50.984 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-10T10:04:50.984 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-10T10:04:50.984 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-10T10:04:50.984 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-10T10:04:50.985 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-10T10:04:50.985 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-10T10:04:50.985 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-10T10:04:50.985 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.985 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.985 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.985 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.985 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.985 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-10T10:04:50.985 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-10T10:04:50.985 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.985 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.985 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.985 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-10T10:04:50.986 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.986 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.986 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.986 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.986 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.986 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.986 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.986 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.986 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-10T10:04:50.986 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-10T10:04:50.986 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.986 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.986 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.986 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-10T10:04:50.987 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.987 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.987 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-10T10:04:50.987 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-10T10:04:50.987 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-10T10:04:50.987 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-10T10:04:50.987 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-10T10:04:50.987 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-10T10:04:50.987 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-10T10:04:50.987 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-10T10:04:50.987 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.987 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.987 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.987 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.988 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.988 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-10T10:04:50.988 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-10T10:04:50.988 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.988 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.988 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.988 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-10T10:04:50.988 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.988 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.988 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.988 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.988 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.988 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.989 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.989 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T10:04:50.989 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T10:04:50.989 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.989 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-10T10:04:50.989 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rook", 2024-09-10T10:04:50.989 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-10T10:04:50.989 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-10T10:04:50.989 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-10T10:04:50.989 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-10T10:04:50.989 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-10T10:04:50.989 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.989 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.990 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.990 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-10T10:04:50.990 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.990 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.990 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-10T10:04:50.990 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-10T10:04:50.990 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-10T10:04:50.990 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-10T10:04:50.990 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-10T10:04:50.990 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-10T10:04:50.990 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-10T10:04:50.990 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-10T10:04:50.990 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.991 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.991 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.991 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.991 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.991 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-10T10:04:50.991 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-10T10:04:50.991 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.991 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.991 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.991 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-10T10:04:50.991 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.991 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.991 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.992 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.992 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.992 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.992 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.992 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.992 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-10T10:04:50.992 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-10T10:04:50.992 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.992 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.992 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.992 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-10T10:04:50.992 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.992 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.993 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-10T10:04:50.993 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-10T10:04:50.993 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-10T10:04:50.993 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-10T10:04:50.993 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-10T10:04:50.993 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-10T10:04:50.993 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-10T10:04:50.993 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-10T10:04:50.993 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.993 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.993 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.993 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.993 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.993 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-10T10:04:50.994 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-10T10:04:50.994 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.994 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.994 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.994 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-10T10:04:50.994 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.994 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.994 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.994 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.994 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.994 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.994 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.994 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.995 INFO:teuthology.orchestra.run.smithi005.stdout: "storage_class": { 2024-09-10T10:04:50.995 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "storage_class", 2024-09-10T10:04:50.995 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.995 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.995 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:50.995 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "local", 2024-09-10T10:04:50.995 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.995 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.995 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.995 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-10T10:04:50.995 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.995 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.995 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.995 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T10:04:50.996 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T10:04:50.996 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.996 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-10T10:04:50.996 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "selftest", 2024-09-10T10:04:50.996 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-10T10:04:50.996 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-10T10:04:50.996 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-10T10:04:50.996 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-10T10:04:50.996 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-10T10:04:50.996 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.996 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.996 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.997 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-10T10:04:50.997 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.997 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.997 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-10T10:04:50.997 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-10T10:04:50.997 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-10T10:04:50.997 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-10T10:04:50.997 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-10T10:04:50.997 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-10T10:04:50.997 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-10T10:04:50.997 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-10T10:04:50.997 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.998 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.998 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.998 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.998 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.998 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-10T10:04:50.998 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-10T10:04:50.998 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:50.998 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.998 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.998 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-10T10:04:50.998 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.998 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.998 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:50.998 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:50.998 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:50.999 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:50.999 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:50.999 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:50.999 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-10T10:04:50.999 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-10T10:04:50.999 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:50.999 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:50.999 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:50.999 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-10T10:04:50.999 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:50.999 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:50.999 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-10T10:04:50.999 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-10T10:04:50.999 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-10T10:04:51.000 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-10T10:04:51.000 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-10T10:04:51.000 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-10T10:04:51.000 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-10T10:04:51.000 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-10T10:04:51.000 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:51.000 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:51.000 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:51.000 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:51.000 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:51.000 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-10T10:04:51.000 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-10T10:04:51.000 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:51.000 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:51.001 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:51.001 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-10T10:04:51.001 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:51.001 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:51.001 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:51.001 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:51.001 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:51.001 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:51.001 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:51.001 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:51.001 INFO:teuthology.orchestra.run.smithi005.stdout: "roption1": { 2024-09-10T10:04:51.001 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "roption1", 2024-09-10T10:04:51.001 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:51.001 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:51.002 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:51.002 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-10T10:04:51.002 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:51.002 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:51.002 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:51.002 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:51.002 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:51.002 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:51.002 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:51.002 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:51.002 INFO:teuthology.orchestra.run.smithi005.stdout: "roption2": { 2024-09-10T10:04:51.002 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "roption2", 2024-09-10T10:04:51.002 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:51.002 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:51.003 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:51.003 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "xyz", 2024-09-10T10:04:51.003 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:51.003 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:51.003 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:51.003 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:51.003 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:51.003 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:51.003 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:51.003 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:51.003 INFO:teuthology.orchestra.run.smithi005.stdout: "rwoption1": { 2024-09-10T10:04:51.003 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rwoption1", 2024-09-10T10:04:51.003 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:51.003 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:51.003 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:51.004 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-10T10:04:51.004 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:51.004 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:51.004 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:51.004 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:51.004 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:51.004 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:51.004 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:51.004 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:51.004 INFO:teuthology.orchestra.run.smithi005.stdout: "rwoption2": { 2024-09-10T10:04:51.004 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rwoption2", 2024-09-10T10:04:51.004 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-10T10:04:51.004 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:51.004 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:51.005 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-10T10:04:51.005 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:51.005 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:51.005 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:51.005 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:51.005 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:51.005 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:51.005 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:51.005 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:51.005 INFO:teuthology.orchestra.run.smithi005.stdout: "rwoption3": { 2024-09-10T10:04:51.005 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rwoption3", 2024-09-10T10:04:51.005 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "float", 2024-09-10T10:04:51.005 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:51.005 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:51.006 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-10T10:04:51.006 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:51.006 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:51.006 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:51.006 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:51.006 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:51.006 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:51.006 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:51.006 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:51.006 INFO:teuthology.orchestra.run.smithi005.stdout: "rwoption4": { 2024-09-10T10:04:51.006 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rwoption4", 2024-09-10T10:04:51.006 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:51.006 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:51.006 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:51.007 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-10T10:04:51.007 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:51.007 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:51.007 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:51.007 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:51.007 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:51.007 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:51.007 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:51.007 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:51.007 INFO:teuthology.orchestra.run.smithi005.stdout: "rwoption5": { 2024-09-10T10:04:51.007 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rwoption5", 2024-09-10T10:04:51.007 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:51.007 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:51.008 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:51.008 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-10T10:04:51.008 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:51.008 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:51.008 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:51.008 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:51.008 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:51.008 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:51.008 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:51.008 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:51.008 INFO:teuthology.orchestra.run.smithi005.stdout: "rwoption6": { 2024-09-10T10:04:51.008 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rwoption6", 2024-09-10T10:04:51.008 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:51.008 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:51.008 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:51.009 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-10T10:04:51.009 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:51.009 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:51.009 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:51.009 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:51.009 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:51.009 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:51.009 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:51.009 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:51.009 INFO:teuthology.orchestra.run.smithi005.stdout: "rwoption7": { 2024-09-10T10:04:51.009 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rwoption7", 2024-09-10T10:04:51.009 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-10T10:04:51.009 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:51.010 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:51.010 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-10T10:04:51.010 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "1", 2024-09-10T10:04:51.010 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "42", 2024-09-10T10:04:51.010 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:51.010 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:51.010 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:51.010 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:51.010 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:51.010 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:51.010 INFO:teuthology.orchestra.run.smithi005.stdout: "testkey": { 2024-09-10T10:04:51.010 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "testkey", 2024-09-10T10:04:51.010 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:51.010 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:51.010 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:51.011 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-10T10:04:51.011 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:51.011 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:51.011 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:51.011 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:51.011 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:51.011 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:51.011 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:51.011 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:51.011 INFO:teuthology.orchestra.run.smithi005.stdout: "testlkey": { 2024-09-10T10:04:51.011 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "testlkey", 2024-09-10T10:04:51.011 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:51.011 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:51.011 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:51.011 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-10T10:04:51.012 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:51.012 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:51.012 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:51.012 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:51.012 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:51.012 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:51.012 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:51.012 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:51.012 INFO:teuthology.orchestra.run.smithi005.stdout: "testnewline": { 2024-09-10T10:04:51.012 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "testnewline", 2024-09-10T10:04:51.012 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:51.012 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:51.012 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:51.012 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-10T10:04:51.012 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:51.013 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:51.013 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:51.013 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:51.013 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:51.013 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:51.013 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:51.013 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T10:04:51.013 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T10:04:51.013 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:51.013 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-10T10:04:51.013 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "snap_schedule", 2024-09-10T10:04:51.013 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-10T10:04:51.013 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-10T10:04:51.013 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-10T10:04:51.014 INFO:teuthology.orchestra.run.smithi005.stdout: "allow_m_granularity": { 2024-09-10T10:04:51.014 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "allow_m_granularity", 2024-09-10T10:04:51.014 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:51.014 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:51.014 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:51.014 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-10T10:04:51.014 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:51.014 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:51.014 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:51.014 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "allow minute scheduled snapshots", 2024-09-10T10:04:51.014 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:51.014 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:51.014 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:51.014 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:51.014 INFO:teuthology.orchestra.run.smithi005.stdout: "dump_on_update": { 2024-09-10T10:04:51.014 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "dump_on_update", 2024-09-10T10:04:51.015 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:51.015 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:51.015 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:51.015 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-10T10:04:51.015 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:51.015 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:51.015 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:51.015 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "dump database to debug log on update", 2024-09-10T10:04:51.015 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:51.015 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:51.015 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:51.015 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:51.015 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-10T10:04:51.015 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-10T10:04:51.015 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:51.016 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:51.016 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:51.016 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-10T10:04:51.016 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:51.016 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:51.016 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-10T10:04:51.016 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-10T10:04:51.016 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-10T10:04:51.016 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-10T10:04:51.016 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-10T10:04:51.016 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-10T10:04:51.016 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-10T10:04:51.016 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-10T10:04:51.017 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:51.017 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:51.017 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:51.017 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:51.017 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:51.017 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-10T10:04:51.017 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-10T10:04:51.017 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:51.017 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:51.017 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:51.017 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-10T10:04:51.017 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:51.018 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:51.018 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:51.018 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:51.018 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:51.018 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:51.018 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:51.018 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:51.018 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-10T10:04:51.018 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-10T10:04:51.018 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:51.018 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:51.018 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:51.018 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-10T10:04:51.018 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:51.019 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:51.019 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-10T10:04:51.019 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-10T10:04:51.019 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-10T10:04:51.019 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-10T10:04:51.019 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-10T10:04:51.019 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-10T10:04:51.019 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-10T10:04:51.019 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-10T10:04:51.019 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:51.019 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:51.019 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:51.019 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:51.020 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:51.020 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-10T10:04:51.020 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-10T10:04:51.020 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:51.020 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:51.020 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:51.020 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-10T10:04:51.020 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:51.020 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:51.020 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:51.020 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:51.020 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:51.020 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:51.020 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:51.021 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T10:04:51.021 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T10:04:51.021 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:51.021 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-10T10:04:51.021 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "stats", 2024-09-10T10:04:51.021 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-10T10:04:51.021 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-10T10:04:51.021 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-10T10:04:51.021 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-10T10:04:51.021 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-10T10:04:51.021 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:51.021 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:51.021 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:51.022 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-10T10:04:51.022 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:51.022 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:51.022 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-10T10:04:51.022 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-10T10:04:51.022 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-10T10:04:51.022 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-10T10:04:51.022 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-10T10:04:51.022 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-10T10:04:51.022 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-10T10:04:51.022 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-10T10:04:51.022 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:51.022 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:51.023 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:51.023 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:51.023 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:51.023 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-10T10:04:51.023 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-10T10:04:51.023 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:51.023 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:51.023 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:51.023 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-10T10:04:51.023 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:51.023 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:51.023 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:51.023 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:51.024 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:51.024 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:51.024 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:51.024 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:51.024 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-10T10:04:51.024 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-10T10:04:51.024 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:51.024 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:51.024 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:51.024 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-10T10:04:51.024 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:51.024 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:51.024 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-10T10:04:51.024 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-10T10:04:51.025 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-10T10:04:51.025 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-10T10:04:51.025 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-10T10:04:51.025 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-10T10:04:51.025 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-10T10:04:51.025 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-10T10:04:51.025 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:51.025 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:51.025 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:51.025 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:51.025 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:51.025 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-10T10:04:51.025 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-10T10:04:51.026 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:51.026 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:51.026 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:51.026 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-10T10:04:51.026 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:51.026 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:51.026 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:51.026 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:51.026 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:51.026 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:51.026 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:51.026 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T10:04:51.026 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T10:04:51.027 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:51.027 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-10T10:04:51.027 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "status", 2024-09-10T10:04:51.027 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-10T10:04:51.027 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-10T10:04:51.027 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-10T10:04:51.027 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-10T10:04:51.027 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-10T10:04:51.027 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:51.027 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:51.027 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:51.027 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-10T10:04:51.027 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:51.027 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:51.028 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-10T10:04:51.028 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-10T10:04:51.028 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-10T10:04:51.028 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-10T10:04:51.028 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-10T10:04:51.028 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-10T10:04:51.028 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-10T10:04:51.028 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-10T10:04:51.028 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:51.028 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:51.028 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:51.028 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:51.028 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:51.029 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-10T10:04:51.029 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-10T10:04:51.029 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:51.029 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:51.029 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:51.029 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-10T10:04:51.029 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:51.029 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:51.029 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:51.029 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:51.029 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:51.029 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:51.029 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:51.029 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:51.030 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-10T10:04:51.030 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-10T10:04:51.030 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:51.030 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:51.030 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:51.030 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-10T10:04:51.030 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:51.030 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:51.030 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-10T10:04:51.030 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-10T10:04:51.030 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-10T10:04:51.030 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-10T10:04:51.030 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-10T10:04:51.030 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-10T10:04:51.030 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-10T10:04:51.031 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-10T10:04:51.031 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:51.031 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:51.031 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:51.031 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:51.031 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:51.031 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-10T10:04:51.031 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-10T10:04:51.031 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:51.031 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:51.031 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:51.031 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-10T10:04:51.031 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:51.031 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:51.031 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:51.031 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:51.032 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:51.032 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:51.032 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:51.032 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T10:04:51.032 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T10:04:51.032 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:51.032 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-10T10:04:51.032 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "telegraf", 2024-09-10T10:04:51.032 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-10T10:04:51.032 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-10T10:04:51.032 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-10T10:04:51.032 INFO:teuthology.orchestra.run.smithi005.stdout: "address": { 2024-09-10T10:04:51.032 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "address", 2024-09-10T10:04:51.032 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:51.032 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:51.033 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:51.033 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-10T10:04:51.033 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:51.033 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:51.035 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:51.035 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:51.036 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:51.036 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:51.036 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:51.036 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:51.036 INFO:teuthology.orchestra.run.smithi005.stdout: "interval": { 2024-09-10T10:04:51.036 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "interval", 2024-09-10T10:04:51.036 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-10T10:04:51.036 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:51.036 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:51.036 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "15", 2024-09-10T10:04:51.036 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:51.036 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:51.036 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:51.036 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:51.037 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:51.037 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:51.037 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:51.037 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:51.037 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-10T10:04:51.037 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-10T10:04:51.037 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:51.037 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:51.037 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:51.037 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-10T10:04:51.037 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:51.037 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:51.037 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-10T10:04:51.037 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-10T10:04:51.037 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-10T10:04:51.038 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-10T10:04:51.038 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-10T10:04:51.038 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-10T10:04:51.038 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-10T10:04:51.038 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-10T10:04:51.038 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:51.038 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:51.038 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:51.038 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:51.038 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:51.038 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-10T10:04:51.038 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-10T10:04:51.038 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:51.038 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:51.039 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:51.039 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-10T10:04:51.039 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:51.039 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:51.039 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:51.039 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:51.039 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:51.039 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:51.039 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:51.039 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:51.039 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-10T10:04:51.039 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-10T10:04:51.039 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:51.039 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:51.039 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:51.040 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-10T10:04:51.040 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:51.040 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:51.040 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-10T10:04:51.040 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-10T10:04:51.040 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-10T10:04:51.040 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-10T10:04:51.040 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-10T10:04:51.040 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-10T10:04:51.040 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-10T10:04:51.040 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-10T10:04:51.040 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:51.040 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:51.040 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:51.040 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:51.041 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:51.041 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-10T10:04:51.041 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-10T10:04:51.041 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:51.041 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:51.041 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:51.041 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-10T10:04:51.041 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:51.041 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:51.041 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:51.041 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:51.041 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:51.041 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:51.041 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:51.041 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T10:04:51.042 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T10:04:51.042 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:51.042 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-10T10:04:51.042 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "telemetry", 2024-09-10T10:04:51.042 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-10T10:04:51.042 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-10T10:04:51.042 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-10T10:04:51.042 INFO:teuthology.orchestra.run.smithi005.stdout: "channel_basic": { 2024-09-10T10:04:51.042 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "channel_basic", 2024-09-10T10:04:51.042 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:51.042 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:51.042 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:51.042 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-10T10:04:51.042 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:51.042 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:51.043 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:51.043 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-10T10:04:51.043 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:51.043 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:51.043 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:51.043 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:51.043 INFO:teuthology.orchestra.run.smithi005.stdout: "channel_crash": { 2024-09-10T10:04:51.043 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "channel_crash", 2024-09-10T10:04:51.043 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:51.043 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:51.043 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:51.043 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-10T10:04:51.043 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:51.043 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:51.043 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:51.044 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-10T10:04:51.044 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:51.044 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:51.044 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:51.044 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:51.044 INFO:teuthology.orchestra.run.smithi005.stdout: "channel_device": { 2024-09-10T10:04:51.044 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "channel_device", 2024-09-10T10:04:51.044 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:51.044 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:51.044 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:51.044 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-10T10:04:51.044 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:51.044 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:51.044 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:51.044 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-10T10:04:51.045 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:51.045 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:51.045 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:51.045 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:51.045 INFO:teuthology.orchestra.run.smithi005.stdout: "channel_ident": { 2024-09-10T10:04:51.045 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "channel_ident", 2024-09-10T10:04:51.045 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:51.045 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:51.045 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:51.045 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-10T10:04:51.045 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:51.045 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:51.045 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:51.045 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-10T10:04:51.045 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:51.046 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:51.046 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:51.046 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:51.046 INFO:teuthology.orchestra.run.smithi005.stdout: "channel_perf": { 2024-09-10T10:04:51.046 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "channel_perf", 2024-09-10T10:04:51.046 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:51.046 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:51.046 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:51.046 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-10T10:04:51.046 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:51.046 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:51.046 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:51.046 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-10T10:04:51.046 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:51.047 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:51.047 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:51.047 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:51.047 INFO:teuthology.orchestra.run.smithi005.stdout: "contact": { 2024-09-10T10:04:51.047 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "contact", 2024-09-10T10:04:51.047 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:51.047 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:51.047 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:51.047 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-10T10:04:51.047 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:51.047 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:51.047 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:51.047 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:51.047 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:51.048 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:51.048 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:51.048 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:51.048 INFO:teuthology.orchestra.run.smithi005.stdout: "description": { 2024-09-10T10:04:51.048 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "description", 2024-09-10T10:04:51.048 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:51.048 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:51.048 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:51.048 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-10T10:04:51.048 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:51.048 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:51.048 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:51.048 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:51.048 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:51.049 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:51.049 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:51.049 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:51.049 INFO:teuthology.orchestra.run.smithi005.stdout: "device_url": { 2024-09-10T10:04:51.049 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "device_url", 2024-09-10T10:04:51.049 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:51.049 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:51.049 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:51.049 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-10T10:04:51.049 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:51.049 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:51.049 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:51.049 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:51.049 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:51.049 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:51.050 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:51.050 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:51.050 INFO:teuthology.orchestra.run.smithi005.stdout: "enabled": { 2024-09-10T10:04:51.050 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "enabled", 2024-09-10T10:04:51.050 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:51.050 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:51.050 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:51.050 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-10T10:04:51.050 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:51.050 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:51.050 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:51.050 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:51.050 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:51.050 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:51.051 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:51.051 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:51.051 INFO:teuthology.orchestra.run.smithi005.stdout: "interval": { 2024-09-10T10:04:51.051 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "interval", 2024-09-10T10:04:51.051 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-10T10:04:51.051 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:51.051 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:51.051 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "24", 2024-09-10T10:04:51.051 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "8", 2024-09-10T10:04:51.051 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:51.051 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:51.051 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:51.051 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:51.051 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:51.051 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:51.052 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:51.052 INFO:teuthology.orchestra.run.smithi005.stdout: "last_opt_revision": { 2024-09-10T10:04:51.052 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "last_opt_revision", 2024-09-10T10:04:51.052 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-10T10:04:51.052 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:51.052 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:51.052 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "1", 2024-09-10T10:04:51.052 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:51.052 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:51.052 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:51.052 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:51.052 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:51.052 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:51.052 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:51.052 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:51.053 INFO:teuthology.orchestra.run.smithi005.stdout: "leaderboard": { 2024-09-10T10:04:51.053 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "leaderboard", 2024-09-10T10:04:51.053 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:51.053 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:51.053 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:51.053 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-10T10:04:51.053 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:51.053 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:51.053 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:51.053 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:51.053 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:51.053 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:51.053 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:51.053 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:51.053 INFO:teuthology.orchestra.run.smithi005.stdout: "leaderboard_description": { 2024-09-10T10:04:51.053 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "leaderboard_description", 2024-09-10T10:04:51.054 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:51.054 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:51.054 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:51.054 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-10T10:04:51.054 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:51.054 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:51.054 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:51.054 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:51.054 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:51.054 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:51.054 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:51.054 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:51.054 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-10T10:04:51.054 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-10T10:04:51.054 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:51.055 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:51.055 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:51.055 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-10T10:04:51.055 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:51.055 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:51.055 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-10T10:04:51.055 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-10T10:04:51.055 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-10T10:04:51.055 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-10T10:04:51.055 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-10T10:04:51.055 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-10T10:04:51.055 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-10T10:04:51.055 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-10T10:04:51.055 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:51.056 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:51.056 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:51.056 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:51.056 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:51.056 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-10T10:04:51.056 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-10T10:04:51.056 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:51.056 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:51.056 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:51.056 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-10T10:04:51.056 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:51.056 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:51.056 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:51.056 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:51.056 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:51.057 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:51.057 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:51.057 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:51.057 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-10T10:04:51.057 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-10T10:04:51.057 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:51.057 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:51.057 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:51.057 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-10T10:04:51.057 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:51.057 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:51.057 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-10T10:04:51.057 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-10T10:04:51.057 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-10T10:04:51.057 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-10T10:04:51.058 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-10T10:04:51.058 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-10T10:04:51.058 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-10T10:04:51.058 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-10T10:04:51.058 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:51.058 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:51.058 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:51.058 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:51.058 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:51.058 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-10T10:04:51.058 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-10T10:04:51.058 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:51.058 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:51.058 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:51.058 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-10T10:04:51.059 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:51.059 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:51.059 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:51.059 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:51.059 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:51.059 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:51.059 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:51.059 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:51.059 INFO:teuthology.orchestra.run.smithi005.stdout: "organization": { 2024-09-10T10:04:51.059 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "organization", 2024-09-10T10:04:51.059 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:51.059 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:51.059 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:51.059 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-10T10:04:51.060 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:51.060 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:51.060 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:51.060 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:51.060 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:51.060 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:51.060 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:51.060 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:51.060 INFO:teuthology.orchestra.run.smithi005.stdout: "proxy": { 2024-09-10T10:04:51.060 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "proxy", 2024-09-10T10:04:51.060 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:51.060 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:51.060 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:51.060 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-10T10:04:51.060 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:51.061 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:51.061 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:51.061 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:51.061 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:51.061 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:51.061 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:51.061 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:51.061 INFO:teuthology.orchestra.run.smithi005.stdout: "url": { 2024-09-10T10:04:51.061 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "url", 2024-09-10T10:04:51.061 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:51.061 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:51.061 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:51.061 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-10T10:04:51.061 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:51.061 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:51.062 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:51.062 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:51.062 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:51.062 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:51.062 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:51.062 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T10:04:51.062 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T10:04:51.062 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:51.062 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-10T10:04:51.062 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "test_orchestrator", 2024-09-10T10:04:51.062 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-10T10:04:51.062 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-10T10:04:51.062 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-10T10:04:51.062 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-10T10:04:51.062 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-10T10:04:51.063 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:51.063 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:51.063 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:51.063 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-10T10:04:51.063 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:51.063 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:51.063 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-10T10:04:51.063 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-10T10:04:51.063 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-10T10:04:51.063 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-10T10:04:51.063 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-10T10:04:51.063 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-10T10:04:51.063 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-10T10:04:51.063 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-10T10:04:51.063 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:51.063 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:51.064 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:51.064 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:51.064 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:51.064 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-10T10:04:51.064 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-10T10:04:51.064 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:51.064 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:51.064 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:51.064 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-10T10:04:51.064 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:51.064 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:51.064 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:51.064 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:51.064 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:51.065 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:51.065 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:51.065 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:51.065 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-10T10:04:51.065 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-10T10:04:51.065 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:51.065 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:51.065 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:51.065 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-10T10:04:51.065 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:51.065 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:51.065 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-10T10:04:51.065 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-10T10:04:51.065 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-10T10:04:51.065 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-10T10:04:51.066 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-10T10:04:51.066 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-10T10:04:51.066 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-10T10:04:51.066 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-10T10:04:51.066 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:51.066 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:51.066 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:51.066 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:51.066 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:51.066 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-10T10:04:51.066 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-10T10:04:51.066 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:51.066 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:51.066 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:51.066 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-10T10:04:51.066 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:51.067 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:51.067 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:51.067 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:51.067 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:51.067 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:51.067 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:51.067 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T10:04:51.067 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T10:04:51.067 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:51.067 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-10T10:04:51.067 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "volumes", 2024-09-10T10:04:51.067 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-10T10:04:51.067 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-10T10:04:51.067 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-10T10:04:51.068 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-10T10:04:51.068 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-10T10:04:51.068 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:51.068 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:51.068 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:51.068 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-10T10:04:51.068 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:51.068 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:51.068 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-10T10:04:51.068 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-10T10:04:51.068 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-10T10:04:51.068 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-10T10:04:51.068 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-10T10:04:51.068 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-10T10:04:51.068 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-10T10:04:51.069 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-10T10:04:51.069 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:51.069 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:51.069 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:51.069 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:51.069 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:51.069 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-10T10:04:51.069 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-10T10:04:51.069 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:51.069 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:51.069 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:51.069 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-10T10:04:51.069 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:51.069 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:51.070 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:51.070 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:51.070 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:51.070 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:51.070 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:51.070 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:51.070 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-10T10:04:51.070 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-10T10:04:51.070 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:51.070 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:51.070 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:51.070 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-10T10:04:51.070 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:51.070 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:51.071 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-10T10:04:51.071 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-10T10:04:51.071 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-10T10:04:51.071 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-10T10:04:51.071 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-10T10:04:51.071 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-10T10:04:51.071 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-10T10:04:51.071 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-10T10:04:51.071 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:51.071 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:51.071 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:51.071 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:51.071 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:51.071 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-10T10:04:51.072 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-10T10:04:51.072 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:51.072 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:51.072 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:51.072 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-10T10:04:51.072 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:51.072 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:51.072 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:51.072 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:51.072 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:51.072 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:51.072 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:51.072 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:51.072 INFO:teuthology.orchestra.run.smithi005.stdout: "max_concurrent_clones": { 2024-09-10T10:04:51.073 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "max_concurrent_clones", 2024-09-10T10:04:51.073 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-10T10:04:51.073 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:51.073 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:51.073 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "4", 2024-09-10T10:04:51.073 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:51.073 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:51.073 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:51.073 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-10T10:04:51.073 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:51.073 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:51.073 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:51.073 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:51.074 INFO:teuthology.orchestra.run.smithi005.stdout: "periodic_async_work": { 2024-09-10T10:04:51.074 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "periodic_async_work", 2024-09-10T10:04:51.074 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:51.074 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:51.074 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:51.074 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-10T10:04:51.074 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:51.074 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:51.074 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:51.074 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Periodically check for async work", 2024-09-10T10:04:51.074 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:51.074 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:51.074 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:51.074 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:51.074 INFO:teuthology.orchestra.run.smithi005.stdout: "snapshot_clone_delay": { 2024-09-10T10:04:51.075 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "snapshot_clone_delay", 2024-09-10T10:04:51.075 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-10T10:04:51.075 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:51.075 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:51.075 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-09-10T10:04:51.075 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:51.075 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:51.075 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:51.075 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-10T10:04:51.075 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:51.075 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:51.075 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:51.075 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:51.075 INFO:teuthology.orchestra.run.smithi005.stdout: "snapshot_clone_no_wait": { 2024-09-10T10:04:51.076 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "snapshot_clone_no_wait", 2024-09-10T10:04:51.076 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:51.076 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:51.076 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:51.076 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-10T10:04:51.076 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:51.076 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:51.076 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:51.076 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-10T10:04:51.076 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:51.076 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:51.076 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:51.076 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T10:04:51.076 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T10:04:51.077 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:51.077 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-10T10:04:51.077 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "zabbix", 2024-09-10T10:04:51.077 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-10T10:04:51.077 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-10T10:04:51.077 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-10T10:04:51.077 INFO:teuthology.orchestra.run.smithi005.stdout: "discovery_interval": { 2024-09-10T10:04:51.077 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "discovery_interval", 2024-09-10T10:04:51.077 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "uint", 2024-09-10T10:04:51.077 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:51.077 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:51.077 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "100", 2024-09-10T10:04:51.077 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:51.078 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:51.078 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:51.078 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:51.078 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:51.078 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:51.078 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:51.078 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:51.078 INFO:teuthology.orchestra.run.smithi005.stdout: "identifier": { 2024-09-10T10:04:51.078 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "identifier", 2024-09-10T10:04:51.078 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:51.078 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:51.078 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:51.078 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-10T10:04:51.078 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:51.078 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:51.079 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:51.079 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:51.079 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:51.079 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:51.079 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:51.079 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:51.079 INFO:teuthology.orchestra.run.smithi005.stdout: "interval": { 2024-09-10T10:04:51.079 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "interval", 2024-09-10T10:04:51.079 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-10T10:04:51.079 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:51.079 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:51.079 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "60", 2024-09-10T10:04:51.079 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:51.079 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:51.080 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:51.080 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:51.080 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:51.080 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:51.080 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:51.080 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:51.080 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-10T10:04:51.080 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-10T10:04:51.080 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:51.080 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:51.080 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:51.080 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-10T10:04:51.080 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:51.081 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:51.081 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-10T10:04:51.081 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-10T10:04:51.081 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-10T10:04:51.081 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-10T10:04:51.081 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-10T10:04:51.081 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-10T10:04:51.081 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-10T10:04:51.081 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-10T10:04:51.081 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:51.081 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:51.081 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:51.081 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:51.081 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:51.082 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-10T10:04:51.082 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-10T10:04:51.082 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:51.082 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:51.082 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:51.082 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-10T10:04:51.082 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:51.082 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:51.082 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:51.082 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:51.082 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:51.082 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:51.082 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:51.082 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:51.083 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-10T10:04:51.083 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-10T10:04:51.083 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:51.083 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:51.083 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:51.083 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-10T10:04:51.083 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:51.083 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:51.083 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-10T10:04:51.083 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-10T10:04:51.083 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-10T10:04:51.083 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-10T10:04:51.083 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-10T10:04:51.083 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-10T10:04:51.084 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-10T10:04:51.084 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-10T10:04:51.084 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:51.084 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:51.084 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:51.084 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:51.084 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:51.084 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-10T10:04:51.084 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-10T10:04:51.084 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-10T10:04:51.084 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:51.084 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-10T10:04:51.084 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-10T10:04:51.085 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:51.085 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:51.085 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:51.085 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:51.085 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:51.085 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:51.085 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:51.085 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:51.085 INFO:teuthology.orchestra.run.smithi005.stdout: "zabbix_host": { 2024-09-10T10:04:51.085 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "zabbix_host", 2024-09-10T10:04:51.085 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:51.085 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:51.085 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:51.085 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-10T10:04:51.086 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:51.086 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:51.086 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:51.086 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:51.086 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:51.086 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:51.086 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:51.086 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:51.086 INFO:teuthology.orchestra.run.smithi005.stdout: "zabbix_port": { 2024-09-10T10:04:51.086 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "zabbix_port", 2024-09-10T10:04:51.086 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-10T10:04:51.086 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:51.086 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:51.086 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10051", 2024-09-10T10:04:51.086 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:51.087 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:51.087 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:51.087 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:51.087 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:51.087 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:51.087 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:51.087 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:51.087 INFO:teuthology.orchestra.run.smithi005.stdout: "zabbix_sender": { 2024-09-10T10:04:51.087 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "zabbix_sender", 2024-09-10T10:04:51.087 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-10T10:04:51.087 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-10T10:04:51.087 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-10T10:04:51.087 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-10T10:04:51.088 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-10T10:04:51.088 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-10T10:04:51.088 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-10T10:04:51.088 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-10T10:04:51.088 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-10T10:04:51.088 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-10T10:04:51.088 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-10T10:04:51.088 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T10:04:51.088 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T10:04:51.088 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T10:04:51.088 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-10T10:04:51.088 INFO:teuthology.orchestra.run.smithi005.stdout: "services": { 2024-09-10T10:04:51.088 INFO:teuthology.orchestra.run.smithi005.stdout: "dashboard": "https://172.21.15.5:8443/" 2024-09-10T10:04:51.088 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:51.089 INFO:teuthology.orchestra.run.smithi005.stdout: "always_on_modules": { 2024-09-10T10:04:51.089 INFO:teuthology.orchestra.run.smithi005.stdout: "octopus": [ 2024-09-10T10:04:51.089 INFO:teuthology.orchestra.run.smithi005.stdout: "balancer", 2024-09-10T10:04:51.089 INFO:teuthology.orchestra.run.smithi005.stdout: "crash", 2024-09-10T10:04:51.089 INFO:teuthology.orchestra.run.smithi005.stdout: "devicehealth", 2024-09-10T10:04:51.089 INFO:teuthology.orchestra.run.smithi005.stdout: "orchestrator", 2024-09-10T10:04:51.089 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_autoscaler", 2024-09-10T10:04:51.089 INFO:teuthology.orchestra.run.smithi005.stdout: "progress", 2024-09-10T10:04:51.089 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_support", 2024-09-10T10:04:51.089 INFO:teuthology.orchestra.run.smithi005.stdout: "status", 2024-09-10T10:04:51.089 INFO:teuthology.orchestra.run.smithi005.stdout: "telemetry", 2024-09-10T10:04:51.089 INFO:teuthology.orchestra.run.smithi005.stdout: "volumes" 2024-09-10T10:04:51.089 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-10T10:04:51.090 INFO:teuthology.orchestra.run.smithi005.stdout: "pacific": [ 2024-09-10T10:04:51.090 INFO:teuthology.orchestra.run.smithi005.stdout: "balancer", 2024-09-10T10:04:51.090 INFO:teuthology.orchestra.run.smithi005.stdout: "crash", 2024-09-10T10:04:51.090 INFO:teuthology.orchestra.run.smithi005.stdout: "devicehealth", 2024-09-10T10:04:51.090 INFO:teuthology.orchestra.run.smithi005.stdout: "orchestrator", 2024-09-10T10:04:51.090 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_autoscaler", 2024-09-10T10:04:51.090 INFO:teuthology.orchestra.run.smithi005.stdout: "progress", 2024-09-10T10:04:51.090 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_support", 2024-09-10T10:04:51.090 INFO:teuthology.orchestra.run.smithi005.stdout: "status", 2024-09-10T10:04:51.090 INFO:teuthology.orchestra.run.smithi005.stdout: "telemetry", 2024-09-10T10:04:51.090 INFO:teuthology.orchestra.run.smithi005.stdout: "volumes" 2024-09-10T10:04:51.090 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-10T10:04:51.090 INFO:teuthology.orchestra.run.smithi005.stdout: "quincy": [ 2024-09-10T10:04:51.090 INFO:teuthology.orchestra.run.smithi005.stdout: "balancer", 2024-09-10T10:04:51.090 INFO:teuthology.orchestra.run.smithi005.stdout: "crash", 2024-09-10T10:04:51.090 INFO:teuthology.orchestra.run.smithi005.stdout: "devicehealth", 2024-09-10T10:04:51.091 INFO:teuthology.orchestra.run.smithi005.stdout: "orchestrator", 2024-09-10T10:04:51.091 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_autoscaler", 2024-09-10T10:04:51.091 INFO:teuthology.orchestra.run.smithi005.stdout: "progress", 2024-09-10T10:04:51.091 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_support", 2024-09-10T10:04:51.091 INFO:teuthology.orchestra.run.smithi005.stdout: "status", 2024-09-10T10:04:51.091 INFO:teuthology.orchestra.run.smithi005.stdout: "telemetry", 2024-09-10T10:04:51.091 INFO:teuthology.orchestra.run.smithi005.stdout: "volumes" 2024-09-10T10:04:51.091 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-10T10:04:51.091 INFO:teuthology.orchestra.run.smithi005.stdout: "reef": [ 2024-09-10T10:04:51.091 INFO:teuthology.orchestra.run.smithi005.stdout: "balancer", 2024-09-10T10:04:51.091 INFO:teuthology.orchestra.run.smithi005.stdout: "crash", 2024-09-10T10:04:51.091 INFO:teuthology.orchestra.run.smithi005.stdout: "devicehealth", 2024-09-10T10:04:51.091 INFO:teuthology.orchestra.run.smithi005.stdout: "orchestrator", 2024-09-10T10:04:51.091 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_autoscaler", 2024-09-10T10:04:51.091 INFO:teuthology.orchestra.run.smithi005.stdout: "progress", 2024-09-10T10:04:51.092 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_support", 2024-09-10T10:04:51.092 INFO:teuthology.orchestra.run.smithi005.stdout: "status", 2024-09-10T10:04:51.092 INFO:teuthology.orchestra.run.smithi005.stdout: "telemetry", 2024-09-10T10:04:51.092 INFO:teuthology.orchestra.run.smithi005.stdout: "volumes" 2024-09-10T10:04:51.092 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-10T10:04:51.092 INFO:teuthology.orchestra.run.smithi005.stdout: "squid": [ 2024-09-10T10:04:51.092 INFO:teuthology.orchestra.run.smithi005.stdout: "balancer", 2024-09-10T10:04:51.092 INFO:teuthology.orchestra.run.smithi005.stdout: "crash", 2024-09-10T10:04:51.092 INFO:teuthology.orchestra.run.smithi005.stdout: "devicehealth", 2024-09-10T10:04:51.092 INFO:teuthology.orchestra.run.smithi005.stdout: "orchestrator", 2024-09-10T10:04:51.092 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_autoscaler", 2024-09-10T10:04:51.092 INFO:teuthology.orchestra.run.smithi005.stdout: "progress", 2024-09-10T10:04:51.092 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_support", 2024-09-10T10:04:51.093 INFO:teuthology.orchestra.run.smithi005.stdout: "status", 2024-09-10T10:04:51.093 INFO:teuthology.orchestra.run.smithi005.stdout: "telemetry", 2024-09-10T10:04:51.093 INFO:teuthology.orchestra.run.smithi005.stdout: "volumes" 2024-09-10T10:04:51.093 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-10T10:04:51.093 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:51.093 INFO:teuthology.orchestra.run.smithi005.stdout: "last_failure_osd_epoch": 3, 2024-09-10T10:04:51.093 INFO:teuthology.orchestra.run.smithi005.stdout: "active_clients": [ 2024-09-10T10:04:51.093 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-10T10:04:51.093 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "devicehealth", 2024-09-10T10:04:51.093 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-09-10T10:04:51.093 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-10T10:04:51.093 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-09-10T10:04:51.093 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:0", 2024-09-10T10:04:51.093 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 2690222669 2024-09-10T10:04:51.094 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T10:04:51.094 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-10T10:04:51.094 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:51.094 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-10T10:04:51.094 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "libcephsqlite", 2024-09-10T10:04:51.094 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-09-10T10:04:51.094 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-10T10:04:51.094 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-09-10T10:04:51.094 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:0", 2024-09-10T10:04:51.094 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 837438180 2024-09-10T10:04:51.094 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T10:04:51.094 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-10T10:04:51.094 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:51.094 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-10T10:04:51.095 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rbd_support", 2024-09-10T10:04:51.095 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-09-10T10:04:51.095 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-10T10:04:51.095 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-09-10T10:04:51.095 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:0", 2024-09-10T10:04:51.095 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 1121510866 2024-09-10T10:04:51.095 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T10:04:51.095 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-10T10:04:51.095 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:51.095 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-10T10:04:51.095 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "volumes", 2024-09-10T10:04:51.095 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-09-10T10:04:51.095 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-10T10:04:51.095 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-09-10T10:04:51.095 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:0", 2024-09-10T10:04:51.096 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 1314353105 2024-09-10T10:04:51.096 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T10:04:51.096 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-10T10:04:51.096 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T10:04:51.096 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-10T10:04:51.096 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-10T10:04:51.097 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:04:49 smithi102 ceph-mon[35442]: pgmap v1187: 129 pgs: 129 active+clean; 551 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 170 KiB/s rd, 1.0 MiB/s wr, 74 op/s 2024-09-10T10:04:51.097 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:04:49 smithi102 ceph-mon[35442]: from='client.? 172.21.15.5:0/3647544003' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-10T10:04:51.097 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:04:49 smithi102 ceph-mon[35442]: from='client.? 172.21.15.5:0/4233161491' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-09-10T10:04:51.098 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:04:49 smithi102 ceph-mon[35442]: from='client.? 172.21.15.5:0/3449477826' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-09-10T10:04:51.098 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:04:49 smithi159 ceph-mon[35513]: pgmap v1187: 129 pgs: 129 active+clean; 551 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 170 KiB/s rd, 1.0 MiB/s wr, 74 op/s 2024-09-10T10:04:51.098 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:04:49 smithi159 ceph-mon[35513]: from='client.? 172.21.15.5:0/3647544003' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-10T10:04:51.098 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:04:49 smithi159 ceph-mon[35513]: from='client.? 172.21.15.5:0/4233161491' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-09-10T10:04:51.098 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:04:49 smithi159 ceph-mon[35513]: from='client.? 172.21.15.5:0/3449477826' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-09-10T10:04:51.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:49.791+0000 7fadc6c00640 1 -- 172.21.15.5:0/3449477826 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fada40a2b90 msgr2=0x7fada40a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:51.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:49.791+0000 7fadc6c00640 1 --2- 172.21.15.5:0/3449477826 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fada40a2b90 0x7fada40a5050 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fadc0104d80 tx=0x7fadb0005f90 comp rx=0 tx=0).stop 2024-09-10T10:04:51.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:49.791+0000 7fadc6c00640 1 -- 172.21.15.5:0/3449477826 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fadc0104f20 msgr2=0x7fadc019a540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:04:51.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:49.791+0000 7fadc6c00640 1 --2- 172.21.15.5:0/3449477826 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fadc0104f20 0x7fadc019a540 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7fada800e9f0 tx=0x7fada800eeb0 comp rx=0 tx=0).stop 2024-09-10T10:04:51.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:49.792+0000 7fadc6c00640 1 -- 172.21.15.5:0/3449477826 shutdown_connections 2024-09-10T10:04:51.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:49.792+0000 7fadc6c00640 1 --2- 172.21.15.5:0/3449477826 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fada40a2b90 0x7fada40a5050 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:51.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:49.792+0000 7fadc6c00640 1 --2- 172.21.15.5:0/3449477826 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fadc01058e0 0x7fadc01a15b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:51.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:49.792+0000 7fadc6c00640 1 --2- 172.21.15.5:0/3449477826 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fadc0104f20 0x7fadc019a540 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:51.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:49.792+0000 7fadc6c00640 1 --2- 172.21.15.5:0/3449477826 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fadc0103d30 0x7fadc019a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:04:51.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:49.792+0000 7fadc6c00640 1 -- 172.21.15.5:0/3449477826 >> 172.21.15.5:0/3449477826 conn(0x7fadc00ff4b0 msgr2=0x7fadc0108220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:04:51.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:49.792+0000 7fadc6c00640 1 -- 172.21.15.5:0/3449477826 shutdown_connections 2024-09-10T10:04:51.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:04:49.792+0000 7fadc6c00640 1 -- 172.21.15.5:0/3449477826 wait complete. 2024-09-10T10:04:51.105 DEBUG:tasks.check_counter:Getting stats from a 2024-09-10T10:04:51.105 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph daemon mds.a perf dump 2024-09-10T10:04:51.105 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:04:49 smithi005 ceph-mon[30350]: pgmap v1187: 129 pgs: 129 active+clean; 551 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 170 KiB/s rd, 1.0 MiB/s wr, 74 op/s 2024-09-10T10:04:51.105 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:04:49 smithi005 ceph-mon[30350]: from='client.? 172.21.15.5:0/3647544003' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-10T10:04:51.105 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:04:49 smithi005 ceph-mon[30350]: from='client.? 172.21.15.5:0/4233161491' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-09-10T10:04:51.106 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:04:49 smithi005 ceph-mon[30350]: from='client.? 172.21.15.5:0/3449477826' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-09-10T10:04:51.494 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:04:51.965 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:04:51 smithi005 ceph-mon[30350]: pgmap v1188: 129 pgs: 129 active+clean; 551 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 378 KiB/s wr, 46 op/s 2024-09-10T10:04:52.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:04:51 smithi159 ceph-mon[35513]: pgmap v1188: 129 pgs: 129 active+clean; 551 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 378 KiB/s wr, 46 op/s 2024-09-10T10:04:52.110 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-10T10:04:52.111 INFO:teuthology.orchestra.run.smithi005.stdout: "AsyncMessenger::Worker-0": { 2024-09-10T10:04:52.111 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_messages": 2032, 2024-09-10T10:04:52.111 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_messages": 2034, 2024-09-10T10:04:52.111 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_bytes": 1671069, 2024-09-10T10:04:52.111 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_bytes": 646177, 2024-09-10T10:04:52.111 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_created_connections": 7, 2024-09-10T10:04:52.111 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_active_connections": 2, 2024-09-10T10:04:52.111 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_total_time": 0.273684334, 2024-09-10T10:04:52.111 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_send_time": 0.125378575, 2024-09-10T10:04:52.111 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_recv_time": 0.057744794, 2024-09-10T10:04:52.111 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_fast_dispatch_time": 0.064885010, 2024-09-10T10:04:52.111 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T10:04:52.112 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 2032, 2024-09-10T10:04:52.112 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.167429165, 2024-09-10T10:04:52.112 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000082396 2024-09-10T10:04:52.112 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:52.112 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_handle_ack_lat": { 2024-09-10T10:04:52.112 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-10T10:04:52.112 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-10T10:04:52.112 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-10T10:04:52.112 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:52.112 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-10T10:04:52.112 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_encrypted_bytes": 0 2024-09-10T10:04:52.112 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:52.112 INFO:teuthology.orchestra.run.smithi005.stdout: "AsyncMessenger::Worker-1": { 2024-09-10T10:04:52.113 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_messages": 3256, 2024-09-10T10:04:52.113 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_messages": 3612, 2024-09-10T10:04:52.113 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_bytes": 13413379, 2024-09-10T10:04:52.113 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_bytes": 1739603, 2024-09-10T10:04:52.113 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_created_connections": 5, 2024-09-10T10:04:52.113 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_active_connections": 4, 2024-09-10T10:04:52.113 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_total_time": 0.335619316, 2024-09-10T10:04:52.113 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_send_time": 0.111228522, 2024-09-10T10:04:52.113 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_recv_time": 0.108340267, 2024-09-10T10:04:52.113 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_fast_dispatch_time": 0.091172805, 2024-09-10T10:04:52.113 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T10:04:52.113 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 3611, 2024-09-10T10:04:52.113 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.202234991, 2024-09-10T10:04:52.114 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000056005 2024-09-10T10:04:52.114 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:52.114 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_handle_ack_lat": { 2024-09-10T10:04:52.114 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-10T10:04:52.114 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-10T10:04:52.114 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-10T10:04:52.114 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:52.114 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_encrypted_bytes": 3184, 2024-09-10T10:04:52.114 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_encrypted_bytes": 705728 2024-09-10T10:04:52.114 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:52.114 INFO:teuthology.orchestra.run.smithi005.stdout: "AsyncMessenger::Worker-2": { 2024-09-10T10:04:52.114 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_messages": 504, 2024-09-10T10:04:52.114 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_messages": 480, 2024-09-10T10:04:52.115 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_bytes": 973940, 2024-09-10T10:04:52.115 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_bytes": 293621, 2024-09-10T10:04:52.115 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_created_connections": 5, 2024-09-10T10:04:52.115 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_active_connections": 3, 2024-09-10T10:04:52.115 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_total_time": 0.096029236, 2024-09-10T10:04:52.115 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_send_time": 0.054962980, 2024-09-10T10:04:52.115 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_recv_time": 0.048768591, 2024-09-10T10:04:52.115 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_fast_dispatch_time": 0.000054489, 2024-09-10T10:04:52.115 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T10:04:52.115 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 479, 2024-09-10T10:04:52.115 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.037967293, 2024-09-10T10:04:52.115 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000079263 2024-09-10T10:04:52.116 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:52.116 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_handle_ack_lat": { 2024-09-10T10:04:52.116 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-10T10:04:52.116 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-10T10:04:52.116 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-10T10:04:52.116 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:52.116 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_encrypted_bytes": 972592, 2024-09-10T10:04:52.116 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_encrypted_bytes": 292656 2024-09-10T10:04:52.116 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:52.116 INFO:teuthology.orchestra.run.smithi005.stdout: "cct": { 2024-09-10T10:04:52.116 INFO:teuthology.orchestra.run.smithi005.stdout: "total_workers": 1, 2024-09-10T10:04:52.116 INFO:teuthology.orchestra.run.smithi005.stdout: "unhealthy_workers": 0 2024-09-10T10:04:52.116 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:52.117 INFO:teuthology.orchestra.run.smithi005.stdout: "finisher-MDSRank": { 2024-09-10T10:04:52.117 INFO:teuthology.orchestra.run.smithi005.stdout: "queue_len": 0, 2024-09-10T10:04:52.117 INFO:teuthology.orchestra.run.smithi005.stdout: "complete_latency": { 2024-09-10T10:04:52.117 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 8790, 2024-09-10T10:04:52.117 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.231642234, 2024-09-10T10:04:52.117 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000026352 2024-09-10T10:04:52.117 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T10:04:52.117 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:52.117 INFO:teuthology.orchestra.run.smithi005.stdout: "finisher-PurgeQueue": { 2024-09-10T10:04:52.117 INFO:teuthology.orchestra.run.smithi005.stdout: "queue_len": 0, 2024-09-10T10:04:52.117 INFO:teuthology.orchestra.run.smithi005.stdout: "complete_latency": { 2024-09-10T10:04:52.117 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-10T10:04:52.117 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-10T10:04:52.118 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-10T10:04:52.118 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T10:04:52.118 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:52.118 INFO:teuthology.orchestra.run.smithi005.stdout: "mds": { 2024-09-10T10:04:52.118 INFO:teuthology.orchestra.run.smithi005.stdout: "request": 0, 2024-09-10T10:04:52.118 INFO:teuthology.orchestra.run.smithi005.stdout: "reply": 0, 2024-09-10T10:04:52.118 INFO:teuthology.orchestra.run.smithi005.stdout: "reply_latency": { 2024-09-10T10:04:52.118 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-10T10:04:52.118 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-10T10:04:52.118 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-10T10:04:52.118 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:52.118 INFO:teuthology.orchestra.run.smithi005.stdout: "slow_reply": 0, 2024-09-10T10:04:52.118 INFO:teuthology.orchestra.run.smithi005.stdout: "forward": 0, 2024-09-10T10:04:52.118 INFO:teuthology.orchestra.run.smithi005.stdout: "dir_fetch_complete": 0, 2024-09-10T10:04:52.119 INFO:teuthology.orchestra.run.smithi005.stdout: "dir_fetch_keys": 0, 2024-09-10T10:04:52.119 INFO:teuthology.orchestra.run.smithi005.stdout: "dir_commit": 0, 2024-09-10T10:04:52.119 INFO:teuthology.orchestra.run.smithi005.stdout: "dir_split": 0, 2024-09-10T10:04:52.119 INFO:teuthology.orchestra.run.smithi005.stdout: "dir_merge": 0, 2024-09-10T10:04:52.119 INFO:teuthology.orchestra.run.smithi005.stdout: "inodes": 2640, 2024-09-10T10:04:52.119 INFO:teuthology.orchestra.run.smithi005.stdout: "inodes_top": 0, 2024-09-10T10:04:52.119 INFO:teuthology.orchestra.run.smithi005.stdout: "inodes_bottom": 2640, 2024-09-10T10:04:52.119 INFO:teuthology.orchestra.run.smithi005.stdout: "inodes_pin_tail": 0, 2024-09-10T10:04:52.119 INFO:teuthology.orchestra.run.smithi005.stdout: "inodes_pinned": 2640, 2024-09-10T10:04:52.119 INFO:teuthology.orchestra.run.smithi005.stdout: "inodes_expired": 0, 2024-09-10T10:04:52.119 INFO:teuthology.orchestra.run.smithi005.stdout: "inodes_with_caps": 0, 2024-09-10T10:04:52.119 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": 0, 2024-09-10T10:04:52.119 INFO:teuthology.orchestra.run.smithi005.stdout: "subtrees": 1, 2024-09-10T10:04:52.119 INFO:teuthology.orchestra.run.smithi005.stdout: "traverse": 0, 2024-09-10T10:04:52.120 INFO:teuthology.orchestra.run.smithi005.stdout: "traverse_hit": 0, 2024-09-10T10:04:52.120 INFO:teuthology.orchestra.run.smithi005.stdout: "traverse_forward": 0, 2024-09-10T10:04:52.120 INFO:teuthology.orchestra.run.smithi005.stdout: "traverse_discover": 0, 2024-09-10T10:04:52.120 INFO:teuthology.orchestra.run.smithi005.stdout: "traverse_dir_fetch": 0, 2024-09-10T10:04:52.120 INFO:teuthology.orchestra.run.smithi005.stdout: "traverse_remote_ino": 0, 2024-09-10T10:04:52.120 INFO:teuthology.orchestra.run.smithi005.stdout: "traverse_lock": 0, 2024-09-10T10:04:52.120 INFO:teuthology.orchestra.run.smithi005.stdout: "load_cent": 0, 2024-09-10T10:04:52.120 INFO:teuthology.orchestra.run.smithi005.stdout: "q": 0, 2024-09-10T10:04:52.120 INFO:teuthology.orchestra.run.smithi005.stdout: "exported": 0, 2024-09-10T10:04:52.120 INFO:teuthology.orchestra.run.smithi005.stdout: "exported_inodes": 0, 2024-09-10T10:04:52.120 INFO:teuthology.orchestra.run.smithi005.stdout: "imported": 0, 2024-09-10T10:04:52.120 INFO:teuthology.orchestra.run.smithi005.stdout: "imported_inodes": 0, 2024-09-10T10:04:52.120 INFO:teuthology.orchestra.run.smithi005.stdout: "openino_dir_fetch": 0, 2024-09-10T10:04:52.120 INFO:teuthology.orchestra.run.smithi005.stdout: "openino_backtrace_fetch": 0, 2024-09-10T10:04:52.121 INFO:teuthology.orchestra.run.smithi005.stdout: "openino_peer_discover": 0, 2024-09-10T10:04:52.121 INFO:teuthology.orchestra.run.smithi005.stdout: "root_rfiles": 0, 2024-09-10T10:04:52.121 INFO:teuthology.orchestra.run.smithi005.stdout: "root_rbytes": 0, 2024-09-10T10:04:52.121 INFO:teuthology.orchestra.run.smithi005.stdout: "root_rsnaps": 0, 2024-09-10T10:04:52.121 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_backtrace_fetch": 0, 2024-09-10T10:04:52.121 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_set_tag": 0, 2024-09-10T10:04:52.121 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_backtrace_repaired": 0, 2024-09-10T10:04:52.121 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_inotable_repaired": 0, 2024-09-10T10:04:52.121 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_dir_inodes": 0, 2024-09-10T10:04:52.121 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_dir_base_inodes": 0, 2024-09-10T10:04:52.121 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_dirfrag_rstats": 0, 2024-09-10T10:04:52.121 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_file_inodes": 0, 2024-09-10T10:04:52.121 INFO:teuthology.orchestra.run.smithi005.stdout: "handle_inode_file_caps": 0, 2024-09-10T10:04:52.122 INFO:teuthology.orchestra.run.smithi005.stdout: "ceph_cap_op_revoke": 0, 2024-09-10T10:04:52.122 INFO:teuthology.orchestra.run.smithi005.stdout: "ceph_cap_op_grant": 0, 2024-09-10T10:04:52.122 INFO:teuthology.orchestra.run.smithi005.stdout: "ceph_cap_op_trunc": 0, 2024-09-10T10:04:52.122 INFO:teuthology.orchestra.run.smithi005.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-10T10:04:52.122 INFO:teuthology.orchestra.run.smithi005.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-10T10:04:52.122 INFO:teuthology.orchestra.run.smithi005.stdout: "handle_client_caps": 0, 2024-09-10T10:04:52.122 INFO:teuthology.orchestra.run.smithi005.stdout: "handle_client_caps_dirty": 0, 2024-09-10T10:04:52.122 INFO:teuthology.orchestra.run.smithi005.stdout: "handle_client_cap_release": 0, 2024-09-10T10:04:52.122 INFO:teuthology.orchestra.run.smithi005.stdout: "process_request_cap_release": 0 2024-09-10T10:04:52.122 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:52.122 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_cache": { 2024-09-10T10:04:52.122 INFO:teuthology.orchestra.run.smithi005.stdout: "dir_update": 0, 2024-09-10T10:04:52.122 INFO:teuthology.orchestra.run.smithi005.stdout: "dir_update_receipt": 0, 2024-09-10T10:04:52.122 INFO:teuthology.orchestra.run.smithi005.stdout: "dir_try_discover": 0, 2024-09-10T10:04:52.122 INFO:teuthology.orchestra.run.smithi005.stdout: "dir_send_discover": 0, 2024-09-10T10:04:52.123 INFO:teuthology.orchestra.run.smithi005.stdout: "dir_handle_discover": 0, 2024-09-10T10:04:52.123 INFO:teuthology.orchestra.run.smithi005.stdout: "num_strays": 0, 2024-09-10T10:04:52.123 INFO:teuthology.orchestra.run.smithi005.stdout: "num_strays_delayed": 0, 2024-09-10T10:04:52.123 INFO:teuthology.orchestra.run.smithi005.stdout: "num_strays_enqueuing": 0, 2024-09-10T10:04:52.123 INFO:teuthology.orchestra.run.smithi005.stdout: "strays_created": 0, 2024-09-10T10:04:52.123 INFO:teuthology.orchestra.run.smithi005.stdout: "strays_enqueued": 0, 2024-09-10T10:04:52.123 INFO:teuthology.orchestra.run.smithi005.stdout: "strays_reintegrated": 0, 2024-09-10T10:04:52.123 INFO:teuthology.orchestra.run.smithi005.stdout: "strays_migrated": 0, 2024-09-10T10:04:52.123 INFO:teuthology.orchestra.run.smithi005.stdout: "num_recovering_processing": 0, 2024-09-10T10:04:52.123 INFO:teuthology.orchestra.run.smithi005.stdout: "num_recovering_enqueued": 0, 2024-09-10T10:04:52.123 INFO:teuthology.orchestra.run.smithi005.stdout: "num_recovering_prioritized": 0, 2024-09-10T10:04:52.123 INFO:teuthology.orchestra.run.smithi005.stdout: "recovery_started": 0, 2024-09-10T10:04:52.123 INFO:teuthology.orchestra.run.smithi005.stdout: "recovery_completed": 0, 2024-09-10T10:04:52.123 INFO:teuthology.orchestra.run.smithi005.stdout: "ireq_quiesce_path": 0, 2024-09-10T10:04:52.123 INFO:teuthology.orchestra.run.smithi005.stdout: "ireq_quiesce_inode": 0, 2024-09-10T10:04:52.124 INFO:teuthology.orchestra.run.smithi005.stdout: "ireq_enqueue_scrub": 0, 2024-09-10T10:04:52.124 INFO:teuthology.orchestra.run.smithi005.stdout: "ireq_exportdir": 0, 2024-09-10T10:04:52.124 INFO:teuthology.orchestra.run.smithi005.stdout: "ireq_flush": 0, 2024-09-10T10:04:52.124 INFO:teuthology.orchestra.run.smithi005.stdout: "ireq_fragmentdir": 0, 2024-09-10T10:04:52.124 INFO:teuthology.orchestra.run.smithi005.stdout: "ireq_fragstats": 0, 2024-09-10T10:04:52.124 INFO:teuthology.orchestra.run.smithi005.stdout: "ireq_inodestats": 0 2024-09-10T10:04:52.124 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:52.124 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_log": { 2024-09-10T10:04:52.124 INFO:teuthology.orchestra.run.smithi005.stdout: "evlrg": 0, 2024-09-10T10:04:52.124 INFO:teuthology.orchestra.run.smithi005.stdout: "evadd": 0, 2024-09-10T10:04:52.124 INFO:teuthology.orchestra.run.smithi005.stdout: "evex": 0, 2024-09-10T10:04:52.124 INFO:teuthology.orchestra.run.smithi005.stdout: "evtrm": 0, 2024-09-10T10:04:52.124 INFO:teuthology.orchestra.run.smithi005.stdout: "ev": 11025, 2024-09-10T10:04:52.124 INFO:teuthology.orchestra.run.smithi005.stdout: "evexg": 0, 2024-09-10T10:04:52.125 INFO:teuthology.orchestra.run.smithi005.stdout: "evexd": 0, 2024-09-10T10:04:52.125 INFO:teuthology.orchestra.run.smithi005.stdout: "segadd": 0, 2024-09-10T10:04:52.125 INFO:teuthology.orchestra.run.smithi005.stdout: "segex": 0, 2024-09-10T10:04:52.125 INFO:teuthology.orchestra.run.smithi005.stdout: "segtrm": 0, 2024-09-10T10:04:52.125 INFO:teuthology.orchestra.run.smithi005.stdout: "seg": 14, 2024-09-10T10:04:52.125 INFO:teuthology.orchestra.run.smithi005.stdout: "segmjr": 2, 2024-09-10T10:04:52.125 INFO:teuthology.orchestra.run.smithi005.stdout: "segexg": 0, 2024-09-10T10:04:52.125 INFO:teuthology.orchestra.run.smithi005.stdout: "segexd": 0, 2024-09-10T10:04:52.125 INFO:teuthology.orchestra.run.smithi005.stdout: "expos": 4194304, 2024-09-10T10:04:52.125 INFO:teuthology.orchestra.run.smithi005.stdout: "wrpos": 17793226, 2024-09-10T10:04:52.125 INFO:teuthology.orchestra.run.smithi005.stdout: "rdpos": 17790942, 2024-09-10T10:04:52.125 INFO:teuthology.orchestra.run.smithi005.stdout: "jlat": { 2024-09-10T10:04:52.125 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-10T10:04:52.125 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-10T10:04:52.125 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-10T10:04:52.126 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:52.126 INFO:teuthology.orchestra.run.smithi005.stdout: "replayed": 11025 2024-09-10T10:04:52.126 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:52.126 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_mem": { 2024-09-10T10:04:52.126 INFO:teuthology.orchestra.run.smithi005.stdout: "ino": 13, 2024-09-10T10:04:52.126 INFO:teuthology.orchestra.run.smithi005.stdout: "ino+": 2655, 2024-09-10T10:04:52.126 INFO:teuthology.orchestra.run.smithi005.stdout: "ino-": 2642, 2024-09-10T10:04:52.126 INFO:teuthology.orchestra.run.smithi005.stdout: "dir": 11, 2024-09-10T10:04:52.126 INFO:teuthology.orchestra.run.smithi005.stdout: "dir+": 24, 2024-09-10T10:04:52.126 INFO:teuthology.orchestra.run.smithi005.stdout: "dir-": 13, 2024-09-10T10:04:52.126 INFO:teuthology.orchestra.run.smithi005.stdout: "dn": 2640, 2024-09-10T10:04:52.126 INFO:teuthology.orchestra.run.smithi005.stdout: "dn+": 2672, 2024-09-10T10:04:52.126 INFO:teuthology.orchestra.run.smithi005.stdout: "dn-": 32, 2024-09-10T10:04:52.126 INFO:teuthology.orchestra.run.smithi005.stdout: "cap": 0, 2024-09-10T10:04:52.126 INFO:teuthology.orchestra.run.smithi005.stdout: "cap+": 0, 2024-09-10T10:04:52.127 INFO:teuthology.orchestra.run.smithi005.stdout: "cap-": 0, 2024-09-10T10:04:52.127 INFO:teuthology.orchestra.run.smithi005.stdout: "rss": 52512, 2024-09-10T10:04:52.127 INFO:teuthology.orchestra.run.smithi005.stdout: "heap": 215328 2024-09-10T10:04:52.127 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:52.127 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_server": { 2024-09-10T10:04:52.127 INFO:teuthology.orchestra.run.smithi005.stdout: "dispatch_client_request": 0, 2024-09-10T10:04:52.127 INFO:teuthology.orchestra.run.smithi005.stdout: "dispatch_server_request": 0, 2024-09-10T10:04:52.127 INFO:teuthology.orchestra.run.smithi005.stdout: "handle_client_request": 0, 2024-09-10T10:04:52.127 INFO:teuthology.orchestra.run.smithi005.stdout: "handle_client_session": 0, 2024-09-10T10:04:52.127 INFO:teuthology.orchestra.run.smithi005.stdout: "handle_peer_request": 0, 2024-09-10T10:04:52.127 INFO:teuthology.orchestra.run.smithi005.stdout: "req_create_latency": { 2024-09-10T10:04:52.127 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-10T10:04:52.127 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-10T10:04:52.127 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-10T10:04:52.127 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:52.127 INFO:teuthology.orchestra.run.smithi005.stdout: "req_getattr_latency": { 2024-09-10T10:04:52.128 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-10T10:04:52.128 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-10T10:04:52.128 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-10T10:04:52.128 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:52.128 INFO:teuthology.orchestra.run.smithi005.stdout: "req_getfilelock_latency": { 2024-09-10T10:04:52.128 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-10T10:04:52.128 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-10T10:04:52.128 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-10T10:04:52.128 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:52.128 INFO:teuthology.orchestra.run.smithi005.stdout: "req_link_latency": { 2024-09-10T10:04:52.128 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-10T10:04:52.128 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-10T10:04:52.128 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-10T10:04:52.128 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:52.129 INFO:teuthology.orchestra.run.smithi005.stdout: "req_lookup_latency": { 2024-09-10T10:04:52.129 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-10T10:04:52.129 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-10T10:04:52.129 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-10T10:04:52.129 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:52.129 INFO:teuthology.orchestra.run.smithi005.stdout: "req_lookuphash_latency": { 2024-09-10T10:04:52.129 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-10T10:04:52.129 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-10T10:04:52.129 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-10T10:04:52.129 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:52.129 INFO:teuthology.orchestra.run.smithi005.stdout: "req_lookupino_latency": { 2024-09-10T10:04:52.129 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-10T10:04:52.129 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-10T10:04:52.130 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-10T10:04:52.130 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:52.130 INFO:teuthology.orchestra.run.smithi005.stdout: "req_lookupname_latency": { 2024-09-10T10:04:52.130 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-10T10:04:52.130 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-10T10:04:52.130 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-10T10:04:52.130 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:52.130 INFO:teuthology.orchestra.run.smithi005.stdout: "req_lookupparent_latency": { 2024-09-10T10:04:52.130 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-10T10:04:52.130 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-10T10:04:52.130 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-10T10:04:52.130 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:52.130 INFO:teuthology.orchestra.run.smithi005.stdout: "req_lookupsnap_latency": { 2024-09-10T10:04:52.130 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-10T10:04:52.130 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-10T10:04:52.131 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-10T10:04:52.131 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:52.131 INFO:teuthology.orchestra.run.smithi005.stdout: "req_lssnap_latency": { 2024-09-10T10:04:52.131 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-10T10:04:52.131 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-10T10:04:52.131 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-10T10:04:52.131 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:52.131 INFO:teuthology.orchestra.run.smithi005.stdout: "req_mkdir_latency": { 2024-09-10T10:04:52.131 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-10T10:04:52.132 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-10T10:04:52.132 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-10T10:04:52.132 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:52.132 INFO:teuthology.orchestra.run.smithi005.stdout: "req_mknod_latency": { 2024-09-10T10:04:52.132 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-10T10:04:52.132 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-10T10:04:52.132 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-10T10:04:52.132 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:52.132 INFO:teuthology.orchestra.run.smithi005.stdout: "req_mksnap_latency": { 2024-09-10T10:04:52.132 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-10T10:04:52.132 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-10T10:04:52.132 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-10T10:04:52.132 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:52.132 INFO:teuthology.orchestra.run.smithi005.stdout: "req_open_latency": { 2024-09-10T10:04:52.133 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-10T10:04:52.133 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-10T10:04:52.133 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-10T10:04:52.133 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:52.133 INFO:teuthology.orchestra.run.smithi005.stdout: "req_readdir_latency": { 2024-09-10T10:04:52.133 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-10T10:04:52.133 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-10T10:04:52.133 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-10T10:04:52.133 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:52.133 INFO:teuthology.orchestra.run.smithi005.stdout: "req_rename_latency": { 2024-09-10T10:04:52.133 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-10T10:04:52.133 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-10T10:04:52.133 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-10T10:04:52.133 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:52.134 INFO:teuthology.orchestra.run.smithi005.stdout: "req_renamesnap_latency": { 2024-09-10T10:04:52.134 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-10T10:04:52.134 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-10T10:04:52.134 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-10T10:04:52.134 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:52.134 INFO:teuthology.orchestra.run.smithi005.stdout: "req_snapdiff_latency": { 2024-09-10T10:04:52.134 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-10T10:04:52.134 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-10T10:04:52.134 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-10T10:04:52.134 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:52.134 INFO:teuthology.orchestra.run.smithi005.stdout: "req_rmdir_latency": { 2024-09-10T10:04:52.134 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-10T10:04:52.134 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-10T10:04:52.135 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-10T10:04:52.135 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:52.135 INFO:teuthology.orchestra.run.smithi005.stdout: "req_rmsnap_latency": { 2024-09-10T10:04:52.135 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-10T10:04:52.135 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-10T10:04:52.135 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-10T10:04:52.135 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:52.135 INFO:teuthology.orchestra.run.smithi005.stdout: "req_rmxattr_latency": { 2024-09-10T10:04:52.135 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-10T10:04:52.135 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-10T10:04:52.135 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-10T10:04:52.135 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:52.135 INFO:teuthology.orchestra.run.smithi005.stdout: "req_setattr_latency": { 2024-09-10T10:04:52.135 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-10T10:04:52.136 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-10T10:04:52.136 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-10T10:04:52.136 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:52.136 INFO:teuthology.orchestra.run.smithi005.stdout: "req_setdirlayout_latency": { 2024-09-10T10:04:52.136 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-10T10:04:52.136 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-10T10:04:52.136 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-10T10:04:52.136 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:52.136 INFO:teuthology.orchestra.run.smithi005.stdout: "req_setfilelock_latency": { 2024-09-10T10:04:52.136 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-10T10:04:52.136 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-10T10:04:52.136 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-10T10:04:52.136 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:52.136 INFO:teuthology.orchestra.run.smithi005.stdout: "req_setlayout_latency": { 2024-09-10T10:04:52.137 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-10T10:04:52.137 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-10T10:04:52.137 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-10T10:04:52.137 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:52.137 INFO:teuthology.orchestra.run.smithi005.stdout: "req_setxattr_latency": { 2024-09-10T10:04:52.137 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-10T10:04:52.137 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-10T10:04:52.137 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-10T10:04:52.137 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:52.137 INFO:teuthology.orchestra.run.smithi005.stdout: "req_symlink_latency": { 2024-09-10T10:04:52.137 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-10T10:04:52.137 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-10T10:04:52.137 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-10T10:04:52.138 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:52.138 INFO:teuthology.orchestra.run.smithi005.stdout: "req_unlink_latency": { 2024-09-10T10:04:52.138 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-10T10:04:52.138 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-10T10:04:52.138 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-10T10:04:52.138 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:52.138 INFO:teuthology.orchestra.run.smithi005.stdout: "cap_revoke_eviction": 0, 2024-09-10T10:04:52.138 INFO:teuthology.orchestra.run.smithi005.stdout: "cap_acquisition_throttle": 0, 2024-09-10T10:04:52.138 INFO:teuthology.orchestra.run.smithi005.stdout: "req_getvxattr_latency": { 2024-09-10T10:04:52.138 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-10T10:04:52.138 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-10T10:04:52.138 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-10T10:04:52.138 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T10:04:52.138 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:52.139 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_sessions": { 2024-09-10T10:04:52.139 INFO:teuthology.orchestra.run.smithi005.stdout: "session_count": 1, 2024-09-10T10:04:52.139 INFO:teuthology.orchestra.run.smithi005.stdout: "session_add": 2, 2024-09-10T10:04:52.139 INFO:teuthology.orchestra.run.smithi005.stdout: "session_remove": 1, 2024-09-10T10:04:52.139 INFO:teuthology.orchestra.run.smithi005.stdout: "sessions_open": 1, 2024-09-10T10:04:52.139 INFO:teuthology.orchestra.run.smithi005.stdout: "sessions_stale": 0, 2024-09-10T10:04:52.139 INFO:teuthology.orchestra.run.smithi005.stdout: "total_load": 0, 2024-09-10T10:04:52.139 INFO:teuthology.orchestra.run.smithi005.stdout: "average_load": 0, 2024-09-10T10:04:52.139 INFO:teuthology.orchestra.run.smithi005.stdout: "avg_session_uptime": 2771, 2024-09-10T10:04:52.139 INFO:teuthology.orchestra.run.smithi005.stdout: "mdthresh_evicted": 0 2024-09-10T10:04:52.139 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:52.139 INFO:teuthology.orchestra.run.smithi005.stdout: "mempool": { 2024-09-10T10:04:52.139 INFO:teuthology.orchestra.run.smithi005.stdout: "bloom_filter_bytes": 0, 2024-09-10T10:04:52.139 INFO:teuthology.orchestra.run.smithi005.stdout: "bloom_filter_items": 0, 2024-09-10T10:04:52.139 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_alloc_bytes": 0, 2024-09-10T10:04:52.140 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_alloc_items": 0, 2024-09-10T10:04:52.140 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_data_bytes": 0, 2024-09-10T10:04:52.140 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_data_items": 0, 2024-09-10T10:04:52.140 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-10T10:04:52.140 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_onode_items": 0, 2024-09-10T10:04:52.140 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-10T10:04:52.140 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_meta_items": 0, 2024-09-10T10:04:52.140 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_other_bytes": 0, 2024-09-10T10:04:52.140 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_other_items": 0, 2024-09-10T10:04:52.140 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-10T10:04:52.140 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_buffer_items": 0, 2024-09-10T10:04:52.140 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_extent_bytes": 0, 2024-09-10T10:04:52.140 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_extent_items": 0, 2024-09-10T10:04:52.140 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_blob_bytes": 0, 2024-09-10T10:04:52.140 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_blob_items": 0, 2024-09-10T10:04:52.141 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-10T10:04:52.141 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_shared_blob_items": 0, 2024-09-10T10:04:52.141 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-10T10:04:52.141 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_inline_bl_items": 0, 2024-09-10T10:04:52.141 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_fsck_bytes": 0, 2024-09-10T10:04:52.141 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_fsck_items": 0, 2024-09-10T10:04:52.141 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_txc_bytes": 0, 2024-09-10T10:04:52.141 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_txc_items": 0, 2024-09-10T10:04:52.141 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-10T10:04:52.141 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_writing_deferred_items": 0, 2024-09-10T10:04:52.141 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_writing_bytes": 0, 2024-09-10T10:04:52.141 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_writing_items": 0, 2024-09-10T10:04:52.141 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_bytes": 0, 2024-09-10T10:04:52.141 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_items": 0, 2024-09-10T10:04:52.141 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_file_reader_bytes": 0, 2024-09-10T10:04:52.142 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_file_reader_items": 0, 2024-09-10T10:04:52.142 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_file_writer_bytes": 0, 2024-09-10T10:04:52.142 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_file_writer_items": 0, 2024-09-10T10:04:52.142 INFO:teuthology.orchestra.run.smithi005.stdout: "buffer_anon_bytes": 200309, 2024-09-10T10:04:52.142 INFO:teuthology.orchestra.run.smithi005.stdout: "buffer_anon_items": 58, 2024-09-10T10:04:52.142 INFO:teuthology.orchestra.run.smithi005.stdout: "buffer_meta_bytes": 0, 2024-09-10T10:04:52.142 INFO:teuthology.orchestra.run.smithi005.stdout: "buffer_meta_items": 0, 2024-09-10T10:04:52.142 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_bytes": 0, 2024-09-10T10:04:52.142 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_items": 0, 2024-09-10T10:04:52.142 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_mapbl_bytes": 0, 2024-09-10T10:04:52.142 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_mapbl_items": 0, 2024-09-10T10:04:52.142 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_pglog_bytes": 0, 2024-09-10T10:04:52.142 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_pglog_items": 0, 2024-09-10T10:04:52.142 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_bytes": 9152, 2024-09-10T10:04:52.142 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_items": 173, 2024-09-10T10:04:52.143 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_mapping_bytes": 0, 2024-09-10T10:04:52.143 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_mapping_items": 0, 2024-09-10T10:04:52.143 INFO:teuthology.orchestra.run.smithi005.stdout: "pgmap_bytes": 0, 2024-09-10T10:04:52.143 INFO:teuthology.orchestra.run.smithi005.stdout: "pgmap_items": 0, 2024-09-10T10:04:52.143 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_co_bytes": 1902152, 2024-09-10T10:04:52.143 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_co_items": 10688, 2024-09-10T10:04:52.143 INFO:teuthology.orchestra.run.smithi005.stdout: "unittest_1_bytes": 0, 2024-09-10T10:04:52.143 INFO:teuthology.orchestra.run.smithi005.stdout: "unittest_1_items": 0, 2024-09-10T10:04:52.143 INFO:teuthology.orchestra.run.smithi005.stdout: "unittest_2_bytes": 0, 2024-09-10T10:04:52.143 INFO:teuthology.orchestra.run.smithi005.stdout: "unittest_2_items": 0 2024-09-10T10:04:52.143 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:52.143 INFO:teuthology.orchestra.run.smithi005.stdout: "objecter": { 2024-09-10T10:04:52.143 INFO:teuthology.orchestra.run.smithi005.stdout: "op_active": 0, 2024-09-10T10:04:52.143 INFO:teuthology.orchestra.run.smithi005.stdout: "op_laggy": 0, 2024-09-10T10:04:52.144 INFO:teuthology.orchestra.run.smithi005.stdout: "op_send": 5286, 2024-09-10T10:04:52.144 INFO:teuthology.orchestra.run.smithi005.stdout: "op_send_bytes": 21, 2024-09-10T10:04:52.144 INFO:teuthology.orchestra.run.smithi005.stdout: "op_resend": 3, 2024-09-10T10:04:52.144 INFO:teuthology.orchestra.run.smithi005.stdout: "op_reply": 5286, 2024-09-10T10:04:52.144 INFO:teuthology.orchestra.run.smithi005.stdout: "op_latency": { 2024-09-10T10:04:52.144 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 5286, 2024-09-10T10:04:52.144 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 12.651743841, 2024-09-10T10:04:52.144 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.002393443 2024-09-10T10:04:52.144 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:52.144 INFO:teuthology.orchestra.run.smithi005.stdout: "op_inflight": 0, 2024-09-10T10:04:52.144 INFO:teuthology.orchestra.run.smithi005.stdout: "oplen_avg": { 2024-09-10T10:04:52.144 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 5286, 2024-09-10T10:04:52.144 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 5287 2024-09-10T10:04:52.144 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:52.144 INFO:teuthology.orchestra.run.smithi005.stdout: "op": 5286, 2024-09-10T10:04:52.145 INFO:teuthology.orchestra.run.smithi005.stdout: "op_r": 5286, 2024-09-10T10:04:52.145 INFO:teuthology.orchestra.run.smithi005.stdout: "op_w": 0, 2024-09-10T10:04:52.145 INFO:teuthology.orchestra.run.smithi005.stdout: "op_rmw": 0, 2024-09-10T10:04:52.145 INFO:teuthology.orchestra.run.smithi005.stdout: "op_pg": 0, 2024-09-10T10:04:52.145 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_stat": 3328, 2024-09-10T10:04:52.145 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_create": 0, 2024-09-10T10:04:52.145 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_read": 1956, 2024-09-10T10:04:52.145 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_write": 0, 2024-09-10T10:04:52.145 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_writefull": 0, 2024-09-10T10:04:52.145 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_writesame": 0, 2024-09-10T10:04:52.145 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_append": 0, 2024-09-10T10:04:52.145 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_zero": 0, 2024-09-10T10:04:52.145 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_truncate": 0, 2024-09-10T10:04:52.145 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_delete": 0, 2024-09-10T10:04:52.146 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_mapext": 0, 2024-09-10T10:04:52.146 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_sparse_read": 0, 2024-09-10T10:04:52.146 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_clonerange": 0, 2024-09-10T10:04:52.146 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_getxattr": 1, 2024-09-10T10:04:52.146 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_setxattr": 0, 2024-09-10T10:04:52.146 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_cmpxattr": 0, 2024-09-10T10:04:52.146 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_rmxattr": 0, 2024-09-10T10:04:52.146 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_resetxattrs": 0, 2024-09-10T10:04:52.146 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_call": 0, 2024-09-10T10:04:52.146 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_watch": 0, 2024-09-10T10:04:52.146 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_notify": 0, 2024-09-10T10:04:52.146 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_src_cmpxattr": 0, 2024-09-10T10:04:52.146 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_pgls": 0, 2024-09-10T10:04:52.146 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_pgls_filter": 0, 2024-09-10T10:04:52.147 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_other": 0, 2024-09-10T10:04:52.147 INFO:teuthology.orchestra.run.smithi005.stdout: "linger_active": 0, 2024-09-10T10:04:52.147 INFO:teuthology.orchestra.run.smithi005.stdout: "linger_send": 0, 2024-09-10T10:04:52.147 INFO:teuthology.orchestra.run.smithi005.stdout: "linger_resend": 0, 2024-09-10T10:04:52.147 INFO:teuthology.orchestra.run.smithi005.stdout: "linger_ping": 0, 2024-09-10T10:04:52.147 INFO:teuthology.orchestra.run.smithi005.stdout: "poolop_active": 0, 2024-09-10T10:04:52.147 INFO:teuthology.orchestra.run.smithi005.stdout: "poolop_send": 0, 2024-09-10T10:04:52.147 INFO:teuthology.orchestra.run.smithi005.stdout: "poolop_resend": 0, 2024-09-10T10:04:52.147 INFO:teuthology.orchestra.run.smithi005.stdout: "poolstat_active": 0, 2024-09-10T10:04:52.147 INFO:teuthology.orchestra.run.smithi005.stdout: "poolstat_send": 0, 2024-09-10T10:04:52.147 INFO:teuthology.orchestra.run.smithi005.stdout: "poolstat_resend": 0, 2024-09-10T10:04:52.147 INFO:teuthology.orchestra.run.smithi005.stdout: "statfs_active": 0, 2024-09-10T10:04:52.147 INFO:teuthology.orchestra.run.smithi005.stdout: "statfs_send": 0, 2024-09-10T10:04:52.147 INFO:teuthology.orchestra.run.smithi005.stdout: "statfs_resend": 0, 2024-09-10T10:04:52.147 INFO:teuthology.orchestra.run.smithi005.stdout: "command_active": 0, 2024-09-10T10:04:52.148 INFO:teuthology.orchestra.run.smithi005.stdout: "command_send": 0, 2024-09-10T10:04:52.148 INFO:teuthology.orchestra.run.smithi005.stdout: "command_resend": 0, 2024-09-10T10:04:52.148 INFO:teuthology.orchestra.run.smithi005.stdout: "map_epoch": 96, 2024-09-10T10:04:52.148 INFO:teuthology.orchestra.run.smithi005.stdout: "map_full": 0, 2024-09-10T10:04:52.148 INFO:teuthology.orchestra.run.smithi005.stdout: "map_inc": 2, 2024-09-10T10:04:52.148 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_sessions": 6, 2024-09-10T10:04:52.148 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_session_open": 8, 2024-09-10T10:04:52.148 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_session_close": 2, 2024-09-10T10:04:52.148 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_laggy": 0, 2024-09-10T10:04:52.148 INFO:teuthology.orchestra.run.smithi005.stdout: "omap_wr": 0, 2024-09-10T10:04:52.148 INFO:teuthology.orchestra.run.smithi005.stdout: "omap_rd": 2, 2024-09-10T10:04:52.148 INFO:teuthology.orchestra.run.smithi005.stdout: "omap_del": 0 2024-09-10T10:04:52.148 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:52.148 INFO:teuthology.orchestra.run.smithi005.stdout: "oft": { 2024-09-10T10:04:52.148 INFO:teuthology.orchestra.run.smithi005.stdout: "omap_total_objs": 0, 2024-09-10T10:04:52.149 INFO:teuthology.orchestra.run.smithi005.stdout: "omap_total_kv_pairs": 0, 2024-09-10T10:04:52.149 INFO:teuthology.orchestra.run.smithi005.stdout: "omap_total_updates": 0, 2024-09-10T10:04:52.149 INFO:teuthology.orchestra.run.smithi005.stdout: "omap_total_removes": 0 2024-09-10T10:04:52.149 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:52.149 INFO:teuthology.orchestra.run.smithi005.stdout: "osd-slow-ops": { 2024-09-10T10:04:52.149 INFO:teuthology.orchestra.run.smithi005.stdout: "slow_ops_count": 0 2024-09-10T10:04:52.149 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:52.149 INFO:teuthology.orchestra.run.smithi005.stdout: "purge_queue": { 2024-09-10T10:04:52.149 INFO:teuthology.orchestra.run.smithi005.stdout: "pq_executing_ops": 0, 2024-09-10T10:04:52.149 INFO:teuthology.orchestra.run.smithi005.stdout: "pq_executing_ops_high_water": 0, 2024-09-10T10:04:52.149 INFO:teuthology.orchestra.run.smithi005.stdout: "pq_executing": 0, 2024-09-10T10:04:52.149 INFO:teuthology.orchestra.run.smithi005.stdout: "pq_executing_high_water": 0, 2024-09-10T10:04:52.149 INFO:teuthology.orchestra.run.smithi005.stdout: "pq_executed_ops": 0, 2024-09-10T10:04:52.150 INFO:teuthology.orchestra.run.smithi005.stdout: "pq_executed": 0, 2024-09-10T10:04:52.150 INFO:teuthology.orchestra.run.smithi005.stdout: "pq_item_in_journal": 0 2024-09-10T10:04:52.150 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:52.150 INFO:teuthology.orchestra.run.smithi005.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-10T10:04:52.150 INFO:teuthology.orchestra.run.smithi005.stdout: "val": 0, 2024-09-10T10:04:52.150 INFO:teuthology.orchestra.run.smithi005.stdout: "max": 104857600, 2024-09-10T10:04:52.150 INFO:teuthology.orchestra.run.smithi005.stdout: "get_started": 0, 2024-09-10T10:04:52.150 INFO:teuthology.orchestra.run.smithi005.stdout: "get": 5788, 2024-09-10T10:04:52.150 INFO:teuthology.orchestra.run.smithi005.stdout: "get_sum": 15508731, 2024-09-10T10:04:52.150 INFO:teuthology.orchestra.run.smithi005.stdout: "get_or_fail_fail": 0, 2024-09-10T10:04:52.150 INFO:teuthology.orchestra.run.smithi005.stdout: "get_or_fail_success": 5788, 2024-09-10T10:04:52.150 INFO:teuthology.orchestra.run.smithi005.stdout: "take": 0, 2024-09-10T10:04:52.150 INFO:teuthology.orchestra.run.smithi005.stdout: "take_sum": 0, 2024-09-10T10:04:52.151 INFO:teuthology.orchestra.run.smithi005.stdout: "put": 5788, 2024-09-10T10:04:52.151 INFO:teuthology.orchestra.run.smithi005.stdout: "put_sum": 15508731, 2024-09-10T10:04:52.151 INFO:teuthology.orchestra.run.smithi005.stdout: "wait": { 2024-09-10T10:04:52.151 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-10T10:04:52.151 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-10T10:04:52.151 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-10T10:04:52.151 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T10:04:52.151 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:52.151 INFO:teuthology.orchestra.run.smithi005.stdout: "throttle-objecter_bytes": { 2024-09-10T10:04:52.151 INFO:teuthology.orchestra.run.smithi005.stdout: "val": 0, 2024-09-10T10:04:52.151 INFO:teuthology.orchestra.run.smithi005.stdout: "max": 104857600, 2024-09-10T10:04:52.151 INFO:teuthology.orchestra.run.smithi005.stdout: "get_started": 0, 2024-09-10T10:04:52.151 INFO:teuthology.orchestra.run.smithi005.stdout: "get": 0, 2024-09-10T10:04:52.151 INFO:teuthology.orchestra.run.smithi005.stdout: "get_sum": 0, 2024-09-10T10:04:52.151 INFO:teuthology.orchestra.run.smithi005.stdout: "get_or_fail_fail": 0, 2024-09-10T10:04:52.152 INFO:teuthology.orchestra.run.smithi005.stdout: "get_or_fail_success": 0, 2024-09-10T10:04:52.152 INFO:teuthology.orchestra.run.smithi005.stdout: "take": 5286, 2024-09-10T10:04:52.152 INFO:teuthology.orchestra.run.smithi005.stdout: "take_sum": 13600512, 2024-09-10T10:04:52.152 INFO:teuthology.orchestra.run.smithi005.stdout: "put": 5286, 2024-09-10T10:04:52.152 INFO:teuthology.orchestra.run.smithi005.stdout: "put_sum": 13600512, 2024-09-10T10:04:52.152 INFO:teuthology.orchestra.run.smithi005.stdout: "wait": { 2024-09-10T10:04:52.152 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-10T10:04:52.152 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-10T10:04:52.152 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-10T10:04:52.152 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T10:04:52.152 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:52.152 INFO:teuthology.orchestra.run.smithi005.stdout: "throttle-objecter_ops": { 2024-09-10T10:04:52.152 INFO:teuthology.orchestra.run.smithi005.stdout: "val": 0, 2024-09-10T10:04:52.153 INFO:teuthology.orchestra.run.smithi005.stdout: "max": 1024, 2024-09-10T10:04:52.153 INFO:teuthology.orchestra.run.smithi005.stdout: "get_started": 0, 2024-09-10T10:04:52.153 INFO:teuthology.orchestra.run.smithi005.stdout: "get": 0, 2024-09-10T10:04:52.153 INFO:teuthology.orchestra.run.smithi005.stdout: "get_sum": 0, 2024-09-10T10:04:52.153 INFO:teuthology.orchestra.run.smithi005.stdout: "get_or_fail_fail": 0, 2024-09-10T10:04:52.153 INFO:teuthology.orchestra.run.smithi005.stdout: "get_or_fail_success": 0, 2024-09-10T10:04:52.153 INFO:teuthology.orchestra.run.smithi005.stdout: "take": 5286, 2024-09-10T10:04:52.153 INFO:teuthology.orchestra.run.smithi005.stdout: "take_sum": 5286, 2024-09-10T10:04:52.153 INFO:teuthology.orchestra.run.smithi005.stdout: "put": 5286, 2024-09-10T10:04:52.153 INFO:teuthology.orchestra.run.smithi005.stdout: "put_sum": 5286, 2024-09-10T10:04:52.153 INFO:teuthology.orchestra.run.smithi005.stdout: "wait": { 2024-09-10T10:04:52.153 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-10T10:04:52.153 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-10T10:04:52.153 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-10T10:04:52.153 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T10:04:52.154 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:52.154 INFO:teuthology.orchestra.run.smithi005.stdout: "throttle-write_buf_throttle": { 2024-09-10T10:04:52.154 INFO:teuthology.orchestra.run.smithi005.stdout: "val": 0, 2024-09-10T10:04:52.154 INFO:teuthology.orchestra.run.smithi005.stdout: "max": 3758096384, 2024-09-10T10:04:52.154 INFO:teuthology.orchestra.run.smithi005.stdout: "get_started": 0, 2024-09-10T10:04:52.154 INFO:teuthology.orchestra.run.smithi005.stdout: "get": 0, 2024-09-10T10:04:52.154 INFO:teuthology.orchestra.run.smithi005.stdout: "get_sum": 0, 2024-09-10T10:04:52.154 INFO:teuthology.orchestra.run.smithi005.stdout: "get_or_fail_fail": 0, 2024-09-10T10:04:52.154 INFO:teuthology.orchestra.run.smithi005.stdout: "get_or_fail_success": 0, 2024-09-10T10:04:52.154 INFO:teuthology.orchestra.run.smithi005.stdout: "take": 0, 2024-09-10T10:04:52.154 INFO:teuthology.orchestra.run.smithi005.stdout: "take_sum": 0, 2024-09-10T10:04:52.154 INFO:teuthology.orchestra.run.smithi005.stdout: "put": 0, 2024-09-10T10:04:52.154 INFO:teuthology.orchestra.run.smithi005.stdout: "put_sum": 0, 2024-09-10T10:04:52.154 INFO:teuthology.orchestra.run.smithi005.stdout: "wait": { 2024-09-10T10:04:52.154 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-10T10:04:52.155 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-10T10:04:52.155 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-10T10:04:52.155 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T10:04:52.155 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:52.155 INFO:teuthology.orchestra.run.smithi005.stdout: "throttle-write_buf_throttle-0x55e170b76be0": { 2024-09-10T10:04:52.155 INFO:teuthology.orchestra.run.smithi005.stdout: "val": 0, 2024-09-10T10:04:52.155 INFO:teuthology.orchestra.run.smithi005.stdout: "max": 3758096384, 2024-09-10T10:04:52.155 INFO:teuthology.orchestra.run.smithi005.stdout: "get_started": 0, 2024-09-10T10:04:52.155 INFO:teuthology.orchestra.run.smithi005.stdout: "get": 0, 2024-09-10T10:04:52.155 INFO:teuthology.orchestra.run.smithi005.stdout: "get_sum": 0, 2024-09-10T10:04:52.155 INFO:teuthology.orchestra.run.smithi005.stdout: "get_or_fail_fail": 0, 2024-09-10T10:04:52.155 INFO:teuthology.orchestra.run.smithi005.stdout: "get_or_fail_success": 0, 2024-09-10T10:04:52.155 INFO:teuthology.orchestra.run.smithi005.stdout: "take": 0, 2024-09-10T10:04:52.155 INFO:teuthology.orchestra.run.smithi005.stdout: "take_sum": 0, 2024-09-10T10:04:52.155 INFO:teuthology.orchestra.run.smithi005.stdout: "put": 0, 2024-09-10T10:04:52.155 INFO:teuthology.orchestra.run.smithi005.stdout: "put_sum": 0, 2024-09-10T10:04:52.156 INFO:teuthology.orchestra.run.smithi005.stdout: "wait": { 2024-09-10T10:04:52.156 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-10T10:04:52.156 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-10T10:04:52.156 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-10T10:04:52.156 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T10:04:52.156 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T10:04:52.156 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-10T10:04:52.156 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:04:51 smithi102 ceph-mon[35442]: pgmap v1188: 129 pgs: 129 active+clean; 551 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 378 KiB/s wr, 46 op/s 2024-09-10T10:04:52.936 INFO:tasks.check_counter:Daemon mds.a mds.exported=0 2024-09-10T10:04:52.936 INFO:tasks.check_counter:Daemon mds.a mds.imported=0 2024-09-10T10:04:52.936 DEBUG:tasks.check_counter:Getting stats from d 2024-09-10T10:04:52.936 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph daemon mds.d perf dump 2024-09-10T10:04:53.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:04:52 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:04:53.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:04:52 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:04:53.306 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:04:52 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T10:04:53.388 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:04:53.884 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:04:53 smithi102 ceph-mon[35442]: pgmap v1189: 129 pgs: 129 active+clean; 551 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 73 KiB/s wr, 21 op/s 2024-09-10T10:04:53.885 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:04:53 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:04:53.885 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:04:53 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:04:53.885 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:04:53 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:04:53.885 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:04:53 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:04:53.915 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:04:53 smithi159 ceph-mon[35513]: pgmap v1189: 129 pgs: 129 active+clean; 551 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 73 KiB/s wr, 21 op/s 2024-09-10T10:04:53.915 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:04:53 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:04:53.915 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:04:53 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:04:53.915 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:04:53 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:04:53.915 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:04:53 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:04:54.116 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:04:53 smithi005 ceph-mon[30350]: pgmap v1189: 129 pgs: 129 active+clean; 551 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 73 KiB/s wr, 21 op/s 2024-09-10T10:04:54.116 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:04:53 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:04:54.117 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:04:53 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:04:54.117 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:04:53 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:04:54.117 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:04:53 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-10T10:04:54.264 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-10T10:04:54.264 INFO:teuthology.orchestra.run.smithi005.stdout: "AsyncMessenger::Worker-0": { 2024-09-10T10:04:54.264 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_messages": 1854, 2024-09-10T10:04:54.264 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_messages": 1856, 2024-09-10T10:04:54.264 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_bytes": 222774857, 2024-09-10T10:04:54.265 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_bytes": 589636, 2024-09-10T10:04:54.265 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_created_connections": 8, 2024-09-10T10:04:54.265 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_active_connections": 4, 2024-09-10T10:04:54.265 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_total_time": 0.528126701, 2024-09-10T10:04:54.265 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_send_time": 0.063235054, 2024-09-10T10:04:54.265 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_recv_time": 0.390834764, 2024-09-10T10:04:54.265 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_fast_dispatch_time": 0.055218422, 2024-09-10T10:04:54.265 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T10:04:54.265 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 1854, 2024-09-10T10:04:54.265 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.084009731, 2024-09-10T10:04:54.265 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000045312 2024-09-10T10:04:54.265 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:54.265 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_handle_ack_lat": { 2024-09-10T10:04:54.266 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-10T10:04:54.266 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-10T10:04:54.266 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-10T10:04:54.266 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:54.266 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-10T10:04:54.266 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_encrypted_bytes": 0 2024-09-10T10:04:54.266 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:54.266 INFO:teuthology.orchestra.run.smithi005.stdout: "AsyncMessenger::Worker-1": { 2024-09-10T10:04:54.266 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_messages": 2484, 2024-09-10T10:04:54.266 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_messages": 2455, 2024-09-10T10:04:54.266 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_bytes": 269302423, 2024-09-10T10:04:54.266 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_bytes": 923269, 2024-09-10T10:04:54.266 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_created_connections": 6, 2024-09-10T10:04:54.267 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_active_connections": 5, 2024-09-10T10:04:54.267 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_total_time": 0.697391564, 2024-09-10T10:04:54.267 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_send_time": 0.096155005, 2024-09-10T10:04:54.267 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_recv_time": 0.535190860, 2024-09-10T10:04:54.267 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_fast_dispatch_time": 0.058058530, 2024-09-10T10:04:54.267 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T10:04:54.267 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 2455, 2024-09-10T10:04:54.267 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.153676912, 2024-09-10T10:04:54.267 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000062597 2024-09-10T10:04:54.267 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:54.267 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_handle_ack_lat": { 2024-09-10T10:04:54.267 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-10T10:04:54.267 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-10T10:04:54.268 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-10T10:04:54.268 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:54.268 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_encrypted_bytes": 996336, 2024-09-10T10:04:54.268 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_encrypted_bytes": 297440 2024-09-10T10:04:54.268 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:54.268 INFO:teuthology.orchestra.run.smithi005.stdout: "AsyncMessenger::Worker-2": { 2024-09-10T10:04:54.268 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_messages": 2395, 2024-09-10T10:04:54.268 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_messages": 2758, 2024-09-10T10:04:54.268 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_bytes": 122383292, 2024-09-10T10:04:54.268 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_bytes": 1557086, 2024-09-10T10:04:54.268 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_created_connections": 7, 2024-09-10T10:04:54.268 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_active_connections": 5, 2024-09-10T10:04:54.268 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_total_time": 0.509750123, 2024-09-10T10:04:54.268 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_send_time": 0.160410224, 2024-09-10T10:04:54.269 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_recv_time": 0.247342666, 2024-09-10T10:04:54.269 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_fast_dispatch_time": 0.072272884, 2024-09-10T10:04:54.269 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T10:04:54.269 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 2756, 2024-09-10T10:04:54.269 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.169422213, 2024-09-10T10:04:54.269 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000061473 2024-09-10T10:04:54.269 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:54.269 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_handle_ack_lat": { 2024-09-10T10:04:54.269 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-10T10:04:54.269 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-10T10:04:54.269 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-10T10:04:54.269 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:54.269 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-10T10:04:54.270 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_encrypted_bytes": 796112 2024-09-10T10:04:54.270 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:54.270 INFO:teuthology.orchestra.run.smithi005.stdout: "cct": { 2024-09-10T10:04:54.270 INFO:teuthology.orchestra.run.smithi005.stdout: "total_workers": 1, 2024-09-10T10:04:54.270 INFO:teuthology.orchestra.run.smithi005.stdout: "unhealthy_workers": 0 2024-09-10T10:04:54.270 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:54.270 INFO:teuthology.orchestra.run.smithi005.stdout: "finisher-MDSRank": { 2024-09-10T10:04:54.270 INFO:teuthology.orchestra.run.smithi005.stdout: "queue_len": 0, 2024-09-10T10:04:54.270 INFO:teuthology.orchestra.run.smithi005.stdout: "complete_latency": { 2024-09-10T10:04:54.270 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 10436, 2024-09-10T10:04:54.270 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.374788697, 2024-09-10T10:04:54.270 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000035913 2024-09-10T10:04:54.270 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T10:04:54.270 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:54.270 INFO:teuthology.orchestra.run.smithi005.stdout: "finisher-PurgeQueue": { 2024-09-10T10:04:54.271 INFO:teuthology.orchestra.run.smithi005.stdout: "queue_len": 0, 2024-09-10T10:04:54.271 INFO:teuthology.orchestra.run.smithi005.stdout: "complete_latency": { 2024-09-10T10:04:54.271 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-10T10:04:54.271 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-10T10:04:54.271 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-10T10:04:54.271 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T10:04:54.271 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:54.271 INFO:teuthology.orchestra.run.smithi005.stdout: "mds": { 2024-09-10T10:04:54.271 INFO:teuthology.orchestra.run.smithi005.stdout: "request": 0, 2024-09-10T10:04:54.271 INFO:teuthology.orchestra.run.smithi005.stdout: "reply": 0, 2024-09-10T10:04:54.271 INFO:teuthology.orchestra.run.smithi005.stdout: "reply_latency": { 2024-09-10T10:04:54.271 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-10T10:04:54.271 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-10T10:04:54.271 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-10T10:04:54.272 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:54.272 INFO:teuthology.orchestra.run.smithi005.stdout: "slow_reply": 0, 2024-09-10T10:04:54.272 INFO:teuthology.orchestra.run.smithi005.stdout: "forward": 0, 2024-09-10T10:04:54.272 INFO:teuthology.orchestra.run.smithi005.stdout: "dir_fetch_complete": 0, 2024-09-10T10:04:54.272 INFO:teuthology.orchestra.run.smithi005.stdout: "dir_fetch_keys": 0, 2024-09-10T10:04:54.272 INFO:teuthology.orchestra.run.smithi005.stdout: "dir_commit": 0, 2024-09-10T10:04:54.272 INFO:teuthology.orchestra.run.smithi005.stdout: "dir_split": 0, 2024-09-10T10:04:54.272 INFO:teuthology.orchestra.run.smithi005.stdout: "dir_merge": 0, 2024-09-10T10:04:54.272 INFO:teuthology.orchestra.run.smithi005.stdout: "inodes": 61507, 2024-09-10T10:04:54.272 INFO:teuthology.orchestra.run.smithi005.stdout: "inodes_top": 24991, 2024-09-10T10:04:54.272 INFO:teuthology.orchestra.run.smithi005.stdout: "inodes_bottom": 36516, 2024-09-10T10:04:54.272 INFO:teuthology.orchestra.run.smithi005.stdout: "inodes_pin_tail": 0, 2024-09-10T10:04:54.272 INFO:teuthology.orchestra.run.smithi005.stdout: "inodes_pinned": 22601, 2024-09-10T10:04:54.272 INFO:teuthology.orchestra.run.smithi005.stdout: "inodes_expired": 0, 2024-09-10T10:04:54.272 INFO:teuthology.orchestra.run.smithi005.stdout: "inodes_with_caps": 0, 2024-09-10T10:04:54.273 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": 0, 2024-09-10T10:04:54.273 INFO:teuthology.orchestra.run.smithi005.stdout: "subtrees": 1, 2024-09-10T10:04:54.273 INFO:teuthology.orchestra.run.smithi005.stdout: "traverse": 0, 2024-09-10T10:04:54.273 INFO:teuthology.orchestra.run.smithi005.stdout: "traverse_hit": 0, 2024-09-10T10:04:54.273 INFO:teuthology.orchestra.run.smithi005.stdout: "traverse_forward": 0, 2024-09-10T10:04:54.273 INFO:teuthology.orchestra.run.smithi005.stdout: "traverse_discover": 0, 2024-09-10T10:04:54.273 INFO:teuthology.orchestra.run.smithi005.stdout: "traverse_dir_fetch": 0, 2024-09-10T10:04:54.273 INFO:teuthology.orchestra.run.smithi005.stdout: "traverse_remote_ino": 0, 2024-09-10T10:04:54.273 INFO:teuthology.orchestra.run.smithi005.stdout: "traverse_lock": 0, 2024-09-10T10:04:54.273 INFO:teuthology.orchestra.run.smithi005.stdout: "load_cent": 0, 2024-09-10T10:04:54.273 INFO:teuthology.orchestra.run.smithi005.stdout: "q": 0, 2024-09-10T10:04:54.273 INFO:teuthology.orchestra.run.smithi005.stdout: "exported": 0, 2024-09-10T10:04:54.273 INFO:teuthology.orchestra.run.smithi005.stdout: "exported_inodes": 0, 2024-09-10T10:04:54.273 INFO:teuthology.orchestra.run.smithi005.stdout: "imported": 0, 2024-09-10T10:04:54.274 INFO:teuthology.orchestra.run.smithi005.stdout: "imported_inodes": 0, 2024-09-10T10:04:54.274 INFO:teuthology.orchestra.run.smithi005.stdout: "openino_dir_fetch": 0, 2024-09-10T10:04:54.274 INFO:teuthology.orchestra.run.smithi005.stdout: "openino_backtrace_fetch": 0, 2024-09-10T10:04:54.274 INFO:teuthology.orchestra.run.smithi005.stdout: "openino_peer_discover": 0, 2024-09-10T10:04:54.274 INFO:teuthology.orchestra.run.smithi005.stdout: "root_rfiles": 0, 2024-09-10T10:04:54.274 INFO:teuthology.orchestra.run.smithi005.stdout: "root_rbytes": 0, 2024-09-10T10:04:54.274 INFO:teuthology.orchestra.run.smithi005.stdout: "root_rsnaps": 0, 2024-09-10T10:04:54.274 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_backtrace_fetch": 0, 2024-09-10T10:04:54.274 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_set_tag": 0, 2024-09-10T10:04:54.274 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_backtrace_repaired": 0, 2024-09-10T10:04:54.274 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_inotable_repaired": 0, 2024-09-10T10:04:54.274 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_dir_inodes": 0, 2024-09-10T10:04:54.274 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_dir_base_inodes": 0, 2024-09-10T10:04:54.274 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_dirfrag_rstats": 0, 2024-09-10T10:04:54.274 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_file_inodes": 0, 2024-09-10T10:04:54.275 INFO:teuthology.orchestra.run.smithi005.stdout: "handle_inode_file_caps": 0, 2024-09-10T10:04:54.275 INFO:teuthology.orchestra.run.smithi005.stdout: "ceph_cap_op_revoke": 0, 2024-09-10T10:04:54.275 INFO:teuthology.orchestra.run.smithi005.stdout: "ceph_cap_op_grant": 0, 2024-09-10T10:04:54.275 INFO:teuthology.orchestra.run.smithi005.stdout: "ceph_cap_op_trunc": 0, 2024-09-10T10:04:54.275 INFO:teuthology.orchestra.run.smithi005.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-10T10:04:54.275 INFO:teuthology.orchestra.run.smithi005.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-10T10:04:54.275 INFO:teuthology.orchestra.run.smithi005.stdout: "handle_client_caps": 0, 2024-09-10T10:04:54.275 INFO:teuthology.orchestra.run.smithi005.stdout: "handle_client_caps_dirty": 0, 2024-09-10T10:04:54.275 INFO:teuthology.orchestra.run.smithi005.stdout: "handle_client_cap_release": 0, 2024-09-10T10:04:54.275 INFO:teuthology.orchestra.run.smithi005.stdout: "process_request_cap_release": 0 2024-09-10T10:04:54.275 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:54.275 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_cache": { 2024-09-10T10:04:54.275 INFO:teuthology.orchestra.run.smithi005.stdout: "dir_update": 0, 2024-09-10T10:04:54.275 INFO:teuthology.orchestra.run.smithi005.stdout: "dir_update_receipt": 0, 2024-09-10T10:04:54.276 INFO:teuthology.orchestra.run.smithi005.stdout: "dir_try_discover": 0, 2024-09-10T10:04:54.276 INFO:teuthology.orchestra.run.smithi005.stdout: "dir_send_discover": 0, 2024-09-10T10:04:54.276 INFO:teuthology.orchestra.run.smithi005.stdout: "dir_handle_discover": 0, 2024-09-10T10:04:54.276 INFO:teuthology.orchestra.run.smithi005.stdout: "num_strays": 0, 2024-09-10T10:04:54.276 INFO:teuthology.orchestra.run.smithi005.stdout: "num_strays_delayed": 0, 2024-09-10T10:04:54.276 INFO:teuthology.orchestra.run.smithi005.stdout: "num_strays_enqueuing": 0, 2024-09-10T10:04:54.276 INFO:teuthology.orchestra.run.smithi005.stdout: "strays_created": 0, 2024-09-10T10:04:54.276 INFO:teuthology.orchestra.run.smithi005.stdout: "strays_enqueued": 0, 2024-09-10T10:04:54.276 INFO:teuthology.orchestra.run.smithi005.stdout: "strays_reintegrated": 0, 2024-09-10T10:04:54.276 INFO:teuthology.orchestra.run.smithi005.stdout: "strays_migrated": 0, 2024-09-10T10:04:54.276 INFO:teuthology.orchestra.run.smithi005.stdout: "num_recovering_processing": 0, 2024-09-10T10:04:54.276 INFO:teuthology.orchestra.run.smithi005.stdout: "num_recovering_enqueued": 0, 2024-09-10T10:04:54.276 INFO:teuthology.orchestra.run.smithi005.stdout: "num_recovering_prioritized": 0, 2024-09-10T10:04:54.276 INFO:teuthology.orchestra.run.smithi005.stdout: "recovery_started": 0, 2024-09-10T10:04:54.277 INFO:teuthology.orchestra.run.smithi005.stdout: "recovery_completed": 0, 2024-09-10T10:04:54.277 INFO:teuthology.orchestra.run.smithi005.stdout: "ireq_quiesce_path": 0, 2024-09-10T10:04:54.277 INFO:teuthology.orchestra.run.smithi005.stdout: "ireq_quiesce_inode": 0, 2024-09-10T10:04:54.277 INFO:teuthology.orchestra.run.smithi005.stdout: "ireq_enqueue_scrub": 0, 2024-09-10T10:04:54.277 INFO:teuthology.orchestra.run.smithi005.stdout: "ireq_exportdir": 0, 2024-09-10T10:04:54.277 INFO:teuthology.orchestra.run.smithi005.stdout: "ireq_flush": 0, 2024-09-10T10:04:54.277 INFO:teuthology.orchestra.run.smithi005.stdout: "ireq_fragmentdir": 0, 2024-09-10T10:04:54.277 INFO:teuthology.orchestra.run.smithi005.stdout: "ireq_fragstats": 0, 2024-09-10T10:04:54.277 INFO:teuthology.orchestra.run.smithi005.stdout: "ireq_inodestats": 0 2024-09-10T10:04:54.277 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:54.277 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_log": { 2024-09-10T10:04:54.277 INFO:teuthology.orchestra.run.smithi005.stdout: "evlrg": 0, 2024-09-10T10:04:54.277 INFO:teuthology.orchestra.run.smithi005.stdout: "evadd": 0, 2024-09-10T10:04:54.278 INFO:teuthology.orchestra.run.smithi005.stdout: "evex": 0, 2024-09-10T10:04:54.278 INFO:teuthology.orchestra.run.smithi005.stdout: "evtrm": 0, 2024-09-10T10:04:54.278 INFO:teuthology.orchestra.run.smithi005.stdout: "ev": 295089, 2024-09-10T10:04:54.278 INFO:teuthology.orchestra.run.smithi005.stdout: "evexg": 0, 2024-09-10T10:04:54.278 INFO:teuthology.orchestra.run.smithi005.stdout: "evexd": 0, 2024-09-10T10:04:54.278 INFO:teuthology.orchestra.run.smithi005.stdout: "segadd": 0, 2024-09-10T10:04:54.278 INFO:teuthology.orchestra.run.smithi005.stdout: "segex": 0, 2024-09-10T10:04:54.278 INFO:teuthology.orchestra.run.smithi005.stdout: "segtrm": 0, 2024-09-10T10:04:54.278 INFO:teuthology.orchestra.run.smithi005.stdout: "seg": 128, 2024-09-10T10:04:54.278 INFO:teuthology.orchestra.run.smithi005.stdout: "segmjr": 26, 2024-09-10T10:04:54.278 INFO:teuthology.orchestra.run.smithi005.stdout: "segexg": 0, 2024-09-10T10:04:54.278 INFO:teuthology.orchestra.run.smithi005.stdout: "segexd": 0, 2024-09-10T10:04:54.278 INFO:teuthology.orchestra.run.smithi005.stdout: "expos": 389328154, 2024-09-10T10:04:54.278 INFO:teuthology.orchestra.run.smithi005.stdout: "wrpos": 615928886, 2024-09-10T10:04:54.278 INFO:teuthology.orchestra.run.smithi005.stdout: "rdpos": 615927748, 2024-09-10T10:04:54.278 INFO:teuthology.orchestra.run.smithi005.stdout: "jlat": { 2024-09-10T10:04:54.279 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-10T10:04:54.279 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-10T10:04:54.279 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-10T10:04:54.279 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:54.279 INFO:teuthology.orchestra.run.smithi005.stdout: "replayed": 295089 2024-09-10T10:04:54.279 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:54.279 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_mem": { 2024-09-10T10:04:54.279 INFO:teuthology.orchestra.run.smithi005.stdout: "ino": 21, 2024-09-10T10:04:54.279 INFO:teuthology.orchestra.run.smithi005.stdout: "ino+": 61716, 2024-09-10T10:04:54.279 INFO:teuthology.orchestra.run.smithi005.stdout: "ino-": 61695, 2024-09-10T10:04:54.279 INFO:teuthology.orchestra.run.smithi005.stdout: "dir": 40, 2024-09-10T10:04:54.279 INFO:teuthology.orchestra.run.smithi005.stdout: "dir+": 368, 2024-09-10T10:04:54.279 INFO:teuthology.orchestra.run.smithi005.stdout: "dir-": 328, 2024-09-10T10:04:54.279 INFO:teuthology.orchestra.run.smithi005.stdout: "dn": 61507, 2024-09-10T10:04:54.280 INFO:teuthology.orchestra.run.smithi005.stdout: "dn+": 62834, 2024-09-10T10:04:54.280 INFO:teuthology.orchestra.run.smithi005.stdout: "dn-": 1327, 2024-09-10T10:04:54.280 INFO:teuthology.orchestra.run.smithi005.stdout: "cap": 0, 2024-09-10T10:04:54.280 INFO:teuthology.orchestra.run.smithi005.stdout: "cap+": 0, 2024-09-10T10:04:54.280 INFO:teuthology.orchestra.run.smithi005.stdout: "cap-": 0, 2024-09-10T10:04:54.280 INFO:teuthology.orchestra.run.smithi005.stdout: "rss": 107060, 2024-09-10T10:04:54.280 INFO:teuthology.orchestra.run.smithi005.stdout: "heap": 215328 2024-09-10T10:04:54.280 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:54.280 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_server": { 2024-09-10T10:04:54.281 INFO:teuthology.orchestra.run.smithi005.stdout: "dispatch_client_request": 0, 2024-09-10T10:04:54.281 INFO:teuthology.orchestra.run.smithi005.stdout: "dispatch_server_request": 0, 2024-09-10T10:04:54.281 INFO:teuthology.orchestra.run.smithi005.stdout: "handle_client_request": 0, 2024-09-10T10:04:54.281 INFO:teuthology.orchestra.run.smithi005.stdout: "handle_client_session": 0, 2024-09-10T10:04:54.281 INFO:teuthology.orchestra.run.smithi005.stdout: "handle_peer_request": 0, 2024-09-10T10:04:54.281 INFO:teuthology.orchestra.run.smithi005.stdout: "req_create_latency": { 2024-09-10T10:04:54.281 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-10T10:04:54.281 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-10T10:04:54.281 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-10T10:04:54.281 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:54.281 INFO:teuthology.orchestra.run.smithi005.stdout: "req_getattr_latency": { 2024-09-10T10:04:54.281 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-10T10:04:54.281 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-10T10:04:54.281 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-10T10:04:54.282 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:54.282 INFO:teuthology.orchestra.run.smithi005.stdout: "req_getfilelock_latency": { 2024-09-10T10:04:54.282 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-10T10:04:54.282 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-10T10:04:54.282 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-10T10:04:54.282 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:54.282 INFO:teuthology.orchestra.run.smithi005.stdout: "req_link_latency": { 2024-09-10T10:04:54.282 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-10T10:04:54.282 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-10T10:04:54.282 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-10T10:04:54.282 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:54.282 INFO:teuthology.orchestra.run.smithi005.stdout: "req_lookup_latency": { 2024-09-10T10:04:54.282 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-10T10:04:54.282 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-10T10:04:54.283 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-10T10:04:54.283 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:54.283 INFO:teuthology.orchestra.run.smithi005.stdout: "req_lookuphash_latency": { 2024-09-10T10:04:54.283 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-10T10:04:54.283 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-10T10:04:54.283 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-10T10:04:54.283 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:54.283 INFO:teuthology.orchestra.run.smithi005.stdout: "req_lookupino_latency": { 2024-09-10T10:04:54.283 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-10T10:04:54.283 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-10T10:04:54.283 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-10T10:04:54.283 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:54.283 INFO:teuthology.orchestra.run.smithi005.stdout: "req_lookupname_latency": { 2024-09-10T10:04:54.283 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-10T10:04:54.284 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-10T10:04:54.284 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-10T10:04:54.284 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:54.284 INFO:teuthology.orchestra.run.smithi005.stdout: "req_lookupparent_latency": { 2024-09-10T10:04:54.284 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-10T10:04:54.284 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-10T10:04:54.284 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-10T10:04:54.284 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:54.284 INFO:teuthology.orchestra.run.smithi005.stdout: "req_lookupsnap_latency": { 2024-09-10T10:04:54.284 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-10T10:04:54.284 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-10T10:04:54.284 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-10T10:04:54.284 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:54.285 INFO:teuthology.orchestra.run.smithi005.stdout: "req_lssnap_latency": { 2024-09-10T10:04:54.285 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-10T10:04:54.285 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-10T10:04:54.285 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-10T10:04:54.285 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:54.285 INFO:teuthology.orchestra.run.smithi005.stdout: "req_mkdir_latency": { 2024-09-10T10:04:54.285 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-10T10:04:54.286 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-10T10:04:54.286 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-10T10:04:54.286 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:54.286 INFO:teuthology.orchestra.run.smithi005.stdout: "req_mknod_latency": { 2024-09-10T10:04:54.286 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-10T10:04:54.286 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-10T10:04:54.286 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-10T10:04:54.286 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:54.286 INFO:teuthology.orchestra.run.smithi005.stdout: "req_mksnap_latency": { 2024-09-10T10:04:54.286 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-10T10:04:54.286 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-10T10:04:54.286 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-10T10:04:54.287 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:54.287 INFO:teuthology.orchestra.run.smithi005.stdout: "req_open_latency": { 2024-09-10T10:04:54.287 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-10T10:04:54.287 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-10T10:04:54.287 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-10T10:04:54.287 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:54.287 INFO:teuthology.orchestra.run.smithi005.stdout: "req_readdir_latency": { 2024-09-10T10:04:54.287 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-10T10:04:54.287 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-10T10:04:54.287 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-10T10:04:54.287 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:54.287 INFO:teuthology.orchestra.run.smithi005.stdout: "req_rename_latency": { 2024-09-10T10:04:54.287 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-10T10:04:54.288 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-10T10:04:54.288 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-10T10:04:54.288 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:54.288 INFO:teuthology.orchestra.run.smithi005.stdout: "req_renamesnap_latency": { 2024-09-10T10:04:54.288 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-10T10:04:54.288 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-10T10:04:54.288 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-10T10:04:54.288 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:54.288 INFO:teuthology.orchestra.run.smithi005.stdout: "req_snapdiff_latency": { 2024-09-10T10:04:54.288 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-10T10:04:54.288 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-10T10:04:54.288 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-10T10:04:54.288 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:54.288 INFO:teuthology.orchestra.run.smithi005.stdout: "req_rmdir_latency": { 2024-09-10T10:04:54.288 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-10T10:04:54.289 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-10T10:04:54.289 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-10T10:04:54.289 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:54.289 INFO:teuthology.orchestra.run.smithi005.stdout: "req_rmsnap_latency": { 2024-09-10T10:04:54.289 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-10T10:04:54.289 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-10T10:04:54.289 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-10T10:04:54.289 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:54.289 INFO:teuthology.orchestra.run.smithi005.stdout: "req_rmxattr_latency": { 2024-09-10T10:04:54.289 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-10T10:04:54.289 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-10T10:04:54.289 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-10T10:04:54.289 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:54.289 INFO:teuthology.orchestra.run.smithi005.stdout: "req_setattr_latency": { 2024-09-10T10:04:54.290 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-10T10:04:54.290 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-10T10:04:54.290 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-10T10:04:54.290 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:54.290 INFO:teuthology.orchestra.run.smithi005.stdout: "req_setdirlayout_latency": { 2024-09-10T10:04:54.290 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-10T10:04:54.290 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-10T10:04:54.290 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-10T10:04:54.290 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:54.290 INFO:teuthology.orchestra.run.smithi005.stdout: "req_setfilelock_latency": { 2024-09-10T10:04:54.290 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-10T10:04:54.290 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-10T10:04:54.290 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-10T10:04:54.290 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:54.291 INFO:teuthology.orchestra.run.smithi005.stdout: "req_setlayout_latency": { 2024-09-10T10:04:54.291 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-10T10:04:54.291 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-10T10:04:54.291 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-10T10:04:54.291 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:54.291 INFO:teuthology.orchestra.run.smithi005.stdout: "req_setxattr_latency": { 2024-09-10T10:04:54.291 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-10T10:04:54.291 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-10T10:04:54.291 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-10T10:04:54.291 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:54.291 INFO:teuthology.orchestra.run.smithi005.stdout: "req_symlink_latency": { 2024-09-10T10:04:54.291 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-10T10:04:54.291 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-10T10:04:54.292 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-10T10:04:54.292 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:54.292 INFO:teuthology.orchestra.run.smithi005.stdout: "req_unlink_latency": { 2024-09-10T10:04:54.292 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-10T10:04:54.292 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-10T10:04:54.292 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-10T10:04:54.292 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:54.292 INFO:teuthology.orchestra.run.smithi005.stdout: "cap_revoke_eviction": 0, 2024-09-10T10:04:54.292 INFO:teuthology.orchestra.run.smithi005.stdout: "cap_acquisition_throttle": 0, 2024-09-10T10:04:54.292 INFO:teuthology.orchestra.run.smithi005.stdout: "req_getvxattr_latency": { 2024-09-10T10:04:54.292 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-10T10:04:54.292 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-10T10:04:54.292 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-10T10:04:54.292 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T10:04:54.293 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:54.293 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_sessions": { 2024-09-10T10:04:54.293 INFO:teuthology.orchestra.run.smithi005.stdout: "session_count": 1, 2024-09-10T10:04:54.293 INFO:teuthology.orchestra.run.smithi005.stdout: "session_add": 1, 2024-09-10T10:04:54.293 INFO:teuthology.orchestra.run.smithi005.stdout: "session_remove": 0, 2024-09-10T10:04:54.293 INFO:teuthology.orchestra.run.smithi005.stdout: "sessions_open": 1, 2024-09-10T10:04:54.293 INFO:teuthology.orchestra.run.smithi005.stdout: "sessions_stale": 0, 2024-09-10T10:04:54.293 INFO:teuthology.orchestra.run.smithi005.stdout: "total_load": 0, 2024-09-10T10:04:54.293 INFO:teuthology.orchestra.run.smithi005.stdout: "average_load": 0, 2024-09-10T10:04:54.293 INFO:teuthology.orchestra.run.smithi005.stdout: "avg_session_uptime": 2775, 2024-09-10T10:04:54.293 INFO:teuthology.orchestra.run.smithi005.stdout: "mdthresh_evicted": 0 2024-09-10T10:04:54.293 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:54.293 INFO:teuthology.orchestra.run.smithi005.stdout: "mempool": { 2024-09-10T10:04:54.293 INFO:teuthology.orchestra.run.smithi005.stdout: "bloom_filter_bytes": 0, 2024-09-10T10:04:54.294 INFO:teuthology.orchestra.run.smithi005.stdout: "bloom_filter_items": 0, 2024-09-10T10:04:54.294 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_alloc_bytes": 0, 2024-09-10T10:04:54.294 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_alloc_items": 0, 2024-09-10T10:04:54.294 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_data_bytes": 0, 2024-09-10T10:04:54.294 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_data_items": 0, 2024-09-10T10:04:54.294 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-10T10:04:54.294 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_onode_items": 0, 2024-09-10T10:04:54.294 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-10T10:04:54.294 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_meta_items": 0, 2024-09-10T10:04:54.294 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_other_bytes": 0, 2024-09-10T10:04:54.294 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_other_items": 0, 2024-09-10T10:04:54.294 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-10T10:04:54.294 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_buffer_items": 0, 2024-09-10T10:04:54.294 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_extent_bytes": 0, 2024-09-10T10:04:54.294 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_extent_items": 0, 2024-09-10T10:04:54.295 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_blob_bytes": 0, 2024-09-10T10:04:54.295 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_blob_items": 0, 2024-09-10T10:04:54.295 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-10T10:04:54.295 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_shared_blob_items": 0, 2024-09-10T10:04:54.295 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-10T10:04:54.295 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_inline_bl_items": 0, 2024-09-10T10:04:54.295 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_fsck_bytes": 0, 2024-09-10T10:04:54.295 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_fsck_items": 0, 2024-09-10T10:04:54.295 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_txc_bytes": 0, 2024-09-10T10:04:54.295 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_txc_items": 0, 2024-09-10T10:04:54.295 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-10T10:04:54.295 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_writing_deferred_items": 0, 2024-09-10T10:04:54.295 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_writing_bytes": 0, 2024-09-10T10:04:54.295 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_writing_items": 0, 2024-09-10T10:04:54.295 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_bytes": 0, 2024-09-10T10:04:54.296 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_items": 0, 2024-09-10T10:04:54.296 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_file_reader_bytes": 0, 2024-09-10T10:04:54.296 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_file_reader_items": 0, 2024-09-10T10:04:54.296 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_file_writer_bytes": 0, 2024-09-10T10:04:54.296 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_file_writer_items": 0, 2024-09-10T10:04:54.296 INFO:teuthology.orchestra.run.smithi005.stdout: "buffer_anon_bytes": 242310, 2024-09-10T10:04:54.296 INFO:teuthology.orchestra.run.smithi005.stdout: "buffer_anon_items": 90, 2024-09-10T10:04:54.296 INFO:teuthology.orchestra.run.smithi005.stdout: "buffer_meta_bytes": 0, 2024-09-10T10:04:54.296 INFO:teuthology.orchestra.run.smithi005.stdout: "buffer_meta_items": 0, 2024-09-10T10:04:54.296 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_bytes": 0, 2024-09-10T10:04:54.296 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_items": 0, 2024-09-10T10:04:54.296 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_mapbl_bytes": 0, 2024-09-10T10:04:54.296 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_mapbl_items": 0, 2024-09-10T10:04:54.296 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_pglog_bytes": 0, 2024-09-10T10:04:54.296 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_pglog_items": 0, 2024-09-10T10:04:54.297 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_bytes": 9152, 2024-09-10T10:04:54.297 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_items": 173, 2024-09-10T10:04:54.297 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_mapping_bytes": 0, 2024-09-10T10:04:54.297 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_mapping_items": 0, 2024-09-10T10:04:54.297 INFO:teuthology.orchestra.run.smithi005.stdout: "pgmap_bytes": 0, 2024-09-10T10:04:54.297 INFO:teuthology.orchestra.run.smithi005.stdout: "pgmap_items": 0, 2024-09-10T10:04:54.297 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_co_bytes": 43404652, 2024-09-10T10:04:54.297 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_co_items": 247193, 2024-09-10T10:04:54.297 INFO:teuthology.orchestra.run.smithi005.stdout: "unittest_1_bytes": 0, 2024-09-10T10:04:54.297 INFO:teuthology.orchestra.run.smithi005.stdout: "unittest_1_items": 0, 2024-09-10T10:04:54.297 INFO:teuthology.orchestra.run.smithi005.stdout: "unittest_2_bytes": 0, 2024-09-10T10:04:54.297 INFO:teuthology.orchestra.run.smithi005.stdout: "unittest_2_items": 0 2024-09-10T10:04:54.297 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:54.297 INFO:teuthology.orchestra.run.smithi005.stdout: "objecter": { 2024-09-10T10:04:54.298 INFO:teuthology.orchestra.run.smithi005.stdout: "op_active": 0, 2024-09-10T10:04:54.298 INFO:teuthology.orchestra.run.smithi005.stdout: "op_laggy": 0, 2024-09-10T10:04:54.298 INFO:teuthology.orchestra.run.smithi005.stdout: "op_send": 6215, 2024-09-10T10:04:54.298 INFO:teuthology.orchestra.run.smithi005.stdout: "op_send_bytes": 21, 2024-09-10T10:04:54.298 INFO:teuthology.orchestra.run.smithi005.stdout: "op_resend": 3, 2024-09-10T10:04:54.298 INFO:teuthology.orchestra.run.smithi005.stdout: "op_reply": 6215, 2024-09-10T10:04:54.298 INFO:teuthology.orchestra.run.smithi005.stdout: "op_latency": { 2024-09-10T10:04:54.298 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 6215, 2024-09-10T10:04:54.298 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 14.906698365, 2024-09-10T10:04:54.298 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.002398503 2024-09-10T10:04:54.298 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:54.298 INFO:teuthology.orchestra.run.smithi005.stdout: "op_inflight": 0, 2024-09-10T10:04:54.298 INFO:teuthology.orchestra.run.smithi005.stdout: "oplen_avg": { 2024-09-10T10:04:54.298 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 6215, 2024-09-10T10:04:54.299 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 6216 2024-09-10T10:04:54.299 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:54.299 INFO:teuthology.orchestra.run.smithi005.stdout: "op": 6215, 2024-09-10T10:04:54.299 INFO:teuthology.orchestra.run.smithi005.stdout: "op_r": 6215, 2024-09-10T10:04:54.299 INFO:teuthology.orchestra.run.smithi005.stdout: "op_w": 0, 2024-09-10T10:04:54.299 INFO:teuthology.orchestra.run.smithi005.stdout: "op_rmw": 0, 2024-09-10T10:04:54.299 INFO:teuthology.orchestra.run.smithi005.stdout: "op_pg": 0, 2024-09-10T10:04:54.299 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_stat": 3371, 2024-09-10T10:04:54.299 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_create": 0, 2024-09-10T10:04:54.299 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_read": 2842, 2024-09-10T10:04:54.299 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_write": 0, 2024-09-10T10:04:54.299 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_writefull": 0, 2024-09-10T10:04:54.299 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_writesame": 0, 2024-09-10T10:04:54.299 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_append": 0, 2024-09-10T10:04:54.299 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_zero": 0, 2024-09-10T10:04:54.299 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_truncate": 0, 2024-09-10T10:04:54.300 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_delete": 0, 2024-09-10T10:04:54.300 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_mapext": 0, 2024-09-10T10:04:54.300 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_sparse_read": 0, 2024-09-10T10:04:54.300 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_clonerange": 0, 2024-09-10T10:04:54.300 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_getxattr": 1, 2024-09-10T10:04:54.300 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_setxattr": 0, 2024-09-10T10:04:54.300 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_cmpxattr": 0, 2024-09-10T10:04:54.300 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_rmxattr": 0, 2024-09-10T10:04:54.300 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_resetxattrs": 0, 2024-09-10T10:04:54.300 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_call": 0, 2024-09-10T10:04:54.300 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_watch": 0, 2024-09-10T10:04:54.300 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_notify": 0, 2024-09-10T10:04:54.300 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_src_cmpxattr": 0, 2024-09-10T10:04:54.300 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_pgls": 0, 2024-09-10T10:04:54.300 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_pgls_filter": 0, 2024-09-10T10:04:54.301 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_other": 0, 2024-09-10T10:04:54.301 INFO:teuthology.orchestra.run.smithi005.stdout: "linger_active": 0, 2024-09-10T10:04:54.301 INFO:teuthology.orchestra.run.smithi005.stdout: "linger_send": 0, 2024-09-10T10:04:54.301 INFO:teuthology.orchestra.run.smithi005.stdout: "linger_resend": 0, 2024-09-10T10:04:54.301 INFO:teuthology.orchestra.run.smithi005.stdout: "linger_ping": 0, 2024-09-10T10:04:54.301 INFO:teuthology.orchestra.run.smithi005.stdout: "poolop_active": 0, 2024-09-10T10:04:54.301 INFO:teuthology.orchestra.run.smithi005.stdout: "poolop_send": 0, 2024-09-10T10:04:54.301 INFO:teuthology.orchestra.run.smithi005.stdout: "poolop_resend": 0, 2024-09-10T10:04:54.302 INFO:teuthology.orchestra.run.smithi005.stdout: "poolstat_active": 0, 2024-09-10T10:04:54.302 INFO:teuthology.orchestra.run.smithi005.stdout: "poolstat_send": 0, 2024-09-10T10:04:54.302 INFO:teuthology.orchestra.run.smithi005.stdout: "poolstat_resend": 0, 2024-09-10T10:04:54.302 INFO:teuthology.orchestra.run.smithi005.stdout: "statfs_active": 0, 2024-09-10T10:04:54.302 INFO:teuthology.orchestra.run.smithi005.stdout: "statfs_send": 0, 2024-09-10T10:04:54.302 INFO:teuthology.orchestra.run.smithi005.stdout: "statfs_resend": 0, 2024-09-10T10:04:54.302 INFO:teuthology.orchestra.run.smithi005.stdout: "command_active": 0, 2024-09-10T10:04:54.303 INFO:teuthology.orchestra.run.smithi005.stdout: "command_send": 0, 2024-09-10T10:04:54.303 INFO:teuthology.orchestra.run.smithi005.stdout: "command_resend": 0, 2024-09-10T10:04:54.303 INFO:teuthology.orchestra.run.smithi005.stdout: "map_epoch": 96, 2024-09-10T10:04:54.303 INFO:teuthology.orchestra.run.smithi005.stdout: "map_full": 0, 2024-09-10T10:04:54.303 INFO:teuthology.orchestra.run.smithi005.stdout: "map_inc": 2, 2024-09-10T10:04:54.303 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_sessions": 11, 2024-09-10T10:04:54.303 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_session_open": 11, 2024-09-10T10:04:54.303 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_session_close": 0, 2024-09-10T10:04:54.303 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_laggy": 0, 2024-09-10T10:04:54.303 INFO:teuthology.orchestra.run.smithi005.stdout: "omap_wr": 0, 2024-09-10T10:04:54.303 INFO:teuthology.orchestra.run.smithi005.stdout: "omap_rd": 2, 2024-09-10T10:04:54.303 INFO:teuthology.orchestra.run.smithi005.stdout: "omap_del": 0 2024-09-10T10:04:54.304 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:54.304 INFO:teuthology.orchestra.run.smithi005.stdout: "oft": { 2024-09-10T10:04:54.304 INFO:teuthology.orchestra.run.smithi005.stdout: "omap_total_objs": 0, 2024-09-10T10:04:54.304 INFO:teuthology.orchestra.run.smithi005.stdout: "omap_total_kv_pairs": 0, 2024-09-10T10:04:54.304 INFO:teuthology.orchestra.run.smithi005.stdout: "omap_total_updates": 0, 2024-09-10T10:04:54.304 INFO:teuthology.orchestra.run.smithi005.stdout: "omap_total_removes": 0 2024-09-10T10:04:54.304 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:54.304 INFO:teuthology.orchestra.run.smithi005.stdout: "osd-slow-ops": { 2024-09-10T10:04:54.304 INFO:teuthology.orchestra.run.smithi005.stdout: "slow_ops_count": 0 2024-09-10T10:04:54.304 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:54.304 INFO:teuthology.orchestra.run.smithi005.stdout: "purge_queue": { 2024-09-10T10:04:54.304 INFO:teuthology.orchestra.run.smithi005.stdout: "pq_executing_ops": 0, 2024-09-10T10:04:54.304 INFO:teuthology.orchestra.run.smithi005.stdout: "pq_executing_ops_high_water": 0, 2024-09-10T10:04:54.304 INFO:teuthology.orchestra.run.smithi005.stdout: "pq_executing": 0, 2024-09-10T10:04:54.304 INFO:teuthology.orchestra.run.smithi005.stdout: "pq_executing_high_water": 0, 2024-09-10T10:04:54.305 INFO:teuthology.orchestra.run.smithi005.stdout: "pq_executed_ops": 0, 2024-09-10T10:04:54.305 INFO:teuthology.orchestra.run.smithi005.stdout: "pq_executed": 0, 2024-09-10T10:04:54.305 INFO:teuthology.orchestra.run.smithi005.stdout: "pq_item_in_journal": 0 2024-09-10T10:04:54.305 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:54.305 INFO:teuthology.orchestra.run.smithi005.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-10T10:04:54.305 INFO:teuthology.orchestra.run.smithi005.stdout: "val": 0, 2024-09-10T10:04:54.305 INFO:teuthology.orchestra.run.smithi005.stdout: "max": 104857600, 2024-09-10T10:04:54.305 INFO:teuthology.orchestra.run.smithi005.stdout: "get_started": 0, 2024-09-10T10:04:54.305 INFO:teuthology.orchestra.run.smithi005.stdout: "get": 6729, 2024-09-10T10:04:54.305 INFO:teuthology.orchestra.run.smithi005.stdout: "get_sum": 613825644, 2024-09-10T10:04:54.305 INFO:teuthology.orchestra.run.smithi005.stdout: "get_or_fail_fail": 0, 2024-09-10T10:04:54.305 INFO:teuthology.orchestra.run.smithi005.stdout: "get_or_fail_success": 6729, 2024-09-10T10:04:54.305 INFO:teuthology.orchestra.run.smithi005.stdout: "take": 0, 2024-09-10T10:04:54.306 INFO:teuthology.orchestra.run.smithi005.stdout: "take_sum": 0, 2024-09-10T10:04:54.306 INFO:teuthology.orchestra.run.smithi005.stdout: "put": 6729, 2024-09-10T10:04:54.306 INFO:teuthology.orchestra.run.smithi005.stdout: "put_sum": 613825644, 2024-09-10T10:04:54.306 INFO:teuthology.orchestra.run.smithi005.stdout: "wait": { 2024-09-10T10:04:54.306 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-10T10:04:54.306 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-10T10:04:54.306 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-10T10:04:54.306 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T10:04:54.306 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:54.306 INFO:teuthology.orchestra.run.smithi005.stdout: "throttle-objecter_bytes": { 2024-09-10T10:04:54.306 INFO:teuthology.orchestra.run.smithi005.stdout: "val": 0, 2024-09-10T10:04:54.306 INFO:teuthology.orchestra.run.smithi005.stdout: "max": 104857600, 2024-09-10T10:04:54.306 INFO:teuthology.orchestra.run.smithi005.stdout: "get_started": 0, 2024-09-10T10:04:54.306 INFO:teuthology.orchestra.run.smithi005.stdout: "get": 0, 2024-09-10T10:04:54.306 INFO:teuthology.orchestra.run.smithi005.stdout: "get_sum": 0, 2024-09-10T10:04:54.306 INFO:teuthology.orchestra.run.smithi005.stdout: "get_or_fail_fail": 0, 2024-09-10T10:04:54.307 INFO:teuthology.orchestra.run.smithi005.stdout: "get_or_fail_success": 0, 2024-09-10T10:04:54.307 INFO:teuthology.orchestra.run.smithi005.stdout: "take": 6215, 2024-09-10T10:04:54.307 INFO:teuthology.orchestra.run.smithi005.stdout: "take_sum": 611740352, 2024-09-10T10:04:54.307 INFO:teuthology.orchestra.run.smithi005.stdout: "put": 6215, 2024-09-10T10:04:54.307 INFO:teuthology.orchestra.run.smithi005.stdout: "put_sum": 611740352, 2024-09-10T10:04:54.307 INFO:teuthology.orchestra.run.smithi005.stdout: "wait": { 2024-09-10T10:04:54.307 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-10T10:04:54.307 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-10T10:04:54.307 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-10T10:04:54.307 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T10:04:54.307 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:54.307 INFO:teuthology.orchestra.run.smithi005.stdout: "throttle-objecter_ops": { 2024-09-10T10:04:54.307 INFO:teuthology.orchestra.run.smithi005.stdout: "val": 0, 2024-09-10T10:04:54.307 INFO:teuthology.orchestra.run.smithi005.stdout: "max": 1024, 2024-09-10T10:04:54.308 INFO:teuthology.orchestra.run.smithi005.stdout: "get_started": 0, 2024-09-10T10:04:54.308 INFO:teuthology.orchestra.run.smithi005.stdout: "get": 0, 2024-09-10T10:04:54.308 INFO:teuthology.orchestra.run.smithi005.stdout: "get_sum": 0, 2024-09-10T10:04:54.308 INFO:teuthology.orchestra.run.smithi005.stdout: "get_or_fail_fail": 0, 2024-09-10T10:04:54.308 INFO:teuthology.orchestra.run.smithi005.stdout: "get_or_fail_success": 0, 2024-09-10T10:04:54.308 INFO:teuthology.orchestra.run.smithi005.stdout: "take": 6215, 2024-09-10T10:04:54.308 INFO:teuthology.orchestra.run.smithi005.stdout: "take_sum": 6215, 2024-09-10T10:04:54.308 INFO:teuthology.orchestra.run.smithi005.stdout: "put": 6215, 2024-09-10T10:04:54.308 INFO:teuthology.orchestra.run.smithi005.stdout: "put_sum": 6215, 2024-09-10T10:04:54.308 INFO:teuthology.orchestra.run.smithi005.stdout: "wait": { 2024-09-10T10:04:54.308 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-10T10:04:54.308 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-10T10:04:54.308 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-10T10:04:54.308 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T10:04:54.308 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:54.309 INFO:teuthology.orchestra.run.smithi005.stdout: "throttle-write_buf_throttle": { 2024-09-10T10:04:54.309 INFO:teuthology.orchestra.run.smithi005.stdout: "val": 0, 2024-09-10T10:04:54.309 INFO:teuthology.orchestra.run.smithi005.stdout: "max": 3758096384, 2024-09-10T10:04:54.309 INFO:teuthology.orchestra.run.smithi005.stdout: "get_started": 0, 2024-09-10T10:04:54.309 INFO:teuthology.orchestra.run.smithi005.stdout: "get": 0, 2024-09-10T10:04:54.309 INFO:teuthology.orchestra.run.smithi005.stdout: "get_sum": 0, 2024-09-10T10:04:54.309 INFO:teuthology.orchestra.run.smithi005.stdout: "get_or_fail_fail": 0, 2024-09-10T10:04:54.309 INFO:teuthology.orchestra.run.smithi005.stdout: "get_or_fail_success": 0, 2024-09-10T10:04:54.309 INFO:teuthology.orchestra.run.smithi005.stdout: "take": 0, 2024-09-10T10:04:54.309 INFO:teuthology.orchestra.run.smithi005.stdout: "take_sum": 0, 2024-09-10T10:04:54.309 INFO:teuthology.orchestra.run.smithi005.stdout: "put": 0, 2024-09-10T10:04:54.309 INFO:teuthology.orchestra.run.smithi005.stdout: "put_sum": 0, 2024-09-10T10:04:54.309 INFO:teuthology.orchestra.run.smithi005.stdout: "wait": { 2024-09-10T10:04:54.309 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-10T10:04:54.309 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-10T10:04:54.309 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-10T10:04:54.310 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T10:04:54.310 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:54.310 INFO:teuthology.orchestra.run.smithi005.stdout: "throttle-write_buf_throttle-0x55edabb20be0": { 2024-09-10T10:04:54.310 INFO:teuthology.orchestra.run.smithi005.stdout: "val": 0, 2024-09-10T10:04:54.310 INFO:teuthology.orchestra.run.smithi005.stdout: "max": 3758096384, 2024-09-10T10:04:54.310 INFO:teuthology.orchestra.run.smithi005.stdout: "get_started": 0, 2024-09-10T10:04:54.310 INFO:teuthology.orchestra.run.smithi005.stdout: "get": 0, 2024-09-10T10:04:54.310 INFO:teuthology.orchestra.run.smithi005.stdout: "get_sum": 0, 2024-09-10T10:04:54.310 INFO:teuthology.orchestra.run.smithi005.stdout: "get_or_fail_fail": 0, 2024-09-10T10:04:54.310 INFO:teuthology.orchestra.run.smithi005.stdout: "get_or_fail_success": 0, 2024-09-10T10:04:54.310 INFO:teuthology.orchestra.run.smithi005.stdout: "take": 0, 2024-09-10T10:04:54.310 INFO:teuthology.orchestra.run.smithi005.stdout: "take_sum": 0, 2024-09-10T10:04:54.310 INFO:teuthology.orchestra.run.smithi005.stdout: "put": 0, 2024-09-10T10:04:54.310 INFO:teuthology.orchestra.run.smithi005.stdout: "put_sum": 0, 2024-09-10T10:04:54.311 INFO:teuthology.orchestra.run.smithi005.stdout: "wait": { 2024-09-10T10:04:54.311 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-10T10:04:54.311 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-10T10:04:54.311 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-10T10:04:54.311 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T10:04:54.311 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T10:04:54.311 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-10T10:04:54.958 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:04:54 smithi005 ceph-mon[30350]: Adjusting osd_memory_target on smithi005 to 177.5M 2024-09-10T10:04:54.958 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:04:54 smithi005 ceph-mon[30350]: Unable to set osd_memory_target on smithi005 to 186147225: error parsing value: Value '186147225' is below minimum 939524096 2024-09-10T10:04:55.079 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:04:54 smithi159 ceph-mon[35513]: Adjusting osd_memory_target on smithi005 to 177.5M 2024-09-10T10:04:55.079 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:04:54 smithi159 ceph-mon[35513]: Unable to set osd_memory_target on smithi005 to 186147225: error parsing value: Value '186147225' is below minimum 939524096 2024-09-10T10:04:55.106 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:04:54 smithi102 ceph-mon[35442]: Adjusting osd_memory_target on smithi005 to 177.5M 2024-09-10T10:04:55.106 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:04:54 smithi102 ceph-mon[35442]: Unable to set osd_memory_target on smithi005 to 186147225: error parsing value: Value '186147225' is below minimum 939524096 2024-09-10T10:04:55.195 INFO:tasks.check_counter:Daemon mds.d mds.exported=0 2024-09-10T10:04:55.195 INFO:tasks.check_counter:Daemon mds.d mds.imported=0 2024-09-10T10:04:55.195 DEBUG:tasks.check_counter:Getting stats from g 2024-09-10T10:04:55.195 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph daemon mds.g perf dump 2024-09-10T10:04:55.575 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:04:56.010 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:04:55 smithi005 ceph-mon[30350]: pgmap v1190: 129 pgs: 129 active+clean; 551 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 17 KiB/s wr, 12 op/s 2024-09-10T10:04:56.101 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:04:55 smithi159 ceph-mon[35513]: pgmap v1190: 129 pgs: 129 active+clean; 551 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 17 KiB/s wr, 12 op/s 2024-09-10T10:04:56.135 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:04:55 smithi102 ceph-mon[35442]: pgmap v1190: 129 pgs: 129 active+clean; 551 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 17 KiB/s wr, 12 op/s 2024-09-10T10:04:56.157 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-10T10:04:56.157 INFO:teuthology.orchestra.run.smithi005.stdout: "AsyncMessenger::Worker-0": { 2024-09-10T10:04:56.157 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_messages": 0, 2024-09-10T10:04:56.157 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_messages": 2, 2024-09-10T10:04:56.157 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_bytes": 0, 2024-09-10T10:04:56.157 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_bytes": 0, 2024-09-10T10:04:56.157 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_created_connections": 4, 2024-09-10T10:04:56.157 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_active_connections": 0, 2024-09-10T10:04:56.157 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_total_time": 0.003323518, 2024-09-10T10:04:56.158 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_send_time": 0.000000000, 2024-09-10T10:04:56.158 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_recv_time": 0.000535495, 2024-09-10T10:04:56.158 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-10T10:04:56.158 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T10:04:56.158 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-10T10:04:56.158 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-10T10:04:56.158 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-10T10:04:56.158 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:56.158 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_handle_ack_lat": { 2024-09-10T10:04:56.158 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-10T10:04:56.158 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-10T10:04:56.158 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-10T10:04:56.158 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:56.159 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-10T10:04:56.159 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_encrypted_bytes": 0 2024-09-10T10:04:56.159 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:56.159 INFO:teuthology.orchestra.run.smithi005.stdout: "AsyncMessenger::Worker-1": { 2024-09-10T10:04:56.159 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_messages": 482, 2024-09-10T10:04:56.159 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_messages": 475, 2024-09-10T10:04:56.159 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_bytes": 830752, 2024-09-10T10:04:56.159 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_bytes": 293360, 2024-09-10T10:04:56.159 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_created_connections": 2, 2024-09-10T10:04:56.159 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_active_connections": 1, 2024-09-10T10:04:56.159 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_total_time": 0.086478417, 2024-09-10T10:04:56.159 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_send_time": 0.047058594, 2024-09-10T10:04:56.159 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_recv_time": 0.046049795, 2024-09-10T10:04:56.160 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-10T10:04:56.160 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T10:04:56.160 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 475, 2024-09-10T10:04:56.160 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.042482916, 2024-09-10T10:04:56.160 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000089437 2024-09-10T10:04:56.160 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:56.160 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_handle_ack_lat": { 2024-09-10T10:04:56.160 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-10T10:04:56.160 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-10T10:04:56.160 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-10T10:04:56.160 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:56.160 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_encrypted_bytes": 830752, 2024-09-10T10:04:56.160 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_encrypted_bytes": 293360 2024-09-10T10:04:56.160 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:56.161 INFO:teuthology.orchestra.run.smithi005.stdout: "AsyncMessenger::Worker-2": { 2024-09-10T10:04:56.161 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_messages": 2, 2024-09-10T10:04:56.161 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_messages": 361, 2024-09-10T10:04:56.161 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_bytes": 320, 2024-09-10T10:04:56.161 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_bytes": 220944, 2024-09-10T10:04:56.161 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_created_connections": 4, 2024-09-10T10:04:56.161 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_active_connections": 2, 2024-09-10T10:04:56.161 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_total_time": 0.038414345, 2024-09-10T10:04:56.161 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_send_time": 0.032074161, 2024-09-10T10:04:56.161 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_recv_time": 0.002106638, 2024-09-10T10:04:56.161 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-10T10:04:56.161 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T10:04:56.161 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 359, 2024-09-10T10:04:56.162 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.025560465, 2024-09-10T10:04:56.162 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000071199 2024-09-10T10:04:56.162 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:56.162 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_handle_ack_lat": { 2024-09-10T10:04:56.162 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-10T10:04:56.162 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-10T10:04:56.162 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-10T10:04:56.162 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:56.162 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-10T10:04:56.162 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_encrypted_bytes": 220944 2024-09-10T10:04:56.162 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:56.162 INFO:teuthology.orchestra.run.smithi005.stdout: "cct": { 2024-09-10T10:04:56.162 INFO:teuthology.orchestra.run.smithi005.stdout: "total_workers": 0, 2024-09-10T10:04:56.163 INFO:teuthology.orchestra.run.smithi005.stdout: "unhealthy_workers": 0 2024-09-10T10:04:56.163 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:56.163 INFO:teuthology.orchestra.run.smithi005.stdout: "mempool": { 2024-09-10T10:04:56.163 INFO:teuthology.orchestra.run.smithi005.stdout: "bloom_filter_bytes": 0, 2024-09-10T10:04:56.163 INFO:teuthology.orchestra.run.smithi005.stdout: "bloom_filter_items": 0, 2024-09-10T10:04:56.163 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_alloc_bytes": 0, 2024-09-10T10:04:56.163 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_alloc_items": 0, 2024-09-10T10:04:56.163 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_data_bytes": 0, 2024-09-10T10:04:56.163 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_data_items": 0, 2024-09-10T10:04:56.163 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-10T10:04:56.163 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_onode_items": 0, 2024-09-10T10:04:56.163 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-10T10:04:56.163 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_meta_items": 0, 2024-09-10T10:04:56.163 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_other_bytes": 0, 2024-09-10T10:04:56.164 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_other_items": 0, 2024-09-10T10:04:56.164 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-10T10:04:56.164 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_buffer_items": 0, 2024-09-10T10:04:56.164 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_extent_bytes": 0, 2024-09-10T10:04:56.164 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_extent_items": 0, 2024-09-10T10:04:56.164 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_blob_bytes": 0, 2024-09-10T10:04:56.164 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_blob_items": 0, 2024-09-10T10:04:56.164 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-10T10:04:56.164 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_shared_blob_items": 0, 2024-09-10T10:04:56.164 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-10T10:04:56.164 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_inline_bl_items": 0, 2024-09-10T10:04:56.164 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_fsck_bytes": 0, 2024-09-10T10:04:56.164 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_fsck_items": 0, 2024-09-10T10:04:56.165 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_txc_bytes": 0, 2024-09-10T10:04:56.165 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_txc_items": 0, 2024-09-10T10:04:56.165 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-10T10:04:56.165 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_writing_deferred_items": 0, 2024-09-10T10:04:56.165 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_writing_bytes": 0, 2024-09-10T10:04:56.165 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_writing_items": 0, 2024-09-10T10:04:56.165 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_bytes": 0, 2024-09-10T10:04:56.165 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_items": 0, 2024-09-10T10:04:56.165 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_file_reader_bytes": 0, 2024-09-10T10:04:56.165 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_file_reader_items": 0, 2024-09-10T10:04:56.165 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_file_writer_bytes": 0, 2024-09-10T10:04:56.165 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_file_writer_items": 0, 2024-09-10T10:04:56.165 INFO:teuthology.orchestra.run.smithi005.stdout: "buffer_anon_bytes": 148721, 2024-09-10T10:04:56.165 INFO:teuthology.orchestra.run.smithi005.stdout: "buffer_anon_items": 30, 2024-09-10T10:04:56.165 INFO:teuthology.orchestra.run.smithi005.stdout: "buffer_meta_bytes": 0, 2024-09-10T10:04:56.166 INFO:teuthology.orchestra.run.smithi005.stdout: "buffer_meta_items": 0, 2024-09-10T10:04:56.166 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_bytes": 0, 2024-09-10T10:04:56.166 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_items": 0, 2024-09-10T10:04:56.166 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_mapbl_bytes": 0, 2024-09-10T10:04:56.166 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_mapbl_items": 0, 2024-09-10T10:04:56.166 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_pglog_bytes": 0, 2024-09-10T10:04:56.166 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_pglog_items": 0, 2024-09-10T10:04:56.166 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_bytes": 0, 2024-09-10T10:04:56.166 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_items": 0, 2024-09-10T10:04:56.166 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_mapping_bytes": 0, 2024-09-10T10:04:56.166 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_mapping_items": 0, 2024-09-10T10:04:56.166 INFO:teuthology.orchestra.run.smithi005.stdout: "pgmap_bytes": 0, 2024-09-10T10:04:56.166 INFO:teuthology.orchestra.run.smithi005.stdout: "pgmap_items": 0, 2024-09-10T10:04:56.166 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_co_bytes": 928, 2024-09-10T10:04:56.167 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_co_items": 2, 2024-09-10T10:04:56.167 INFO:teuthology.orchestra.run.smithi005.stdout: "unittest_1_bytes": 0, 2024-09-10T10:04:56.167 INFO:teuthology.orchestra.run.smithi005.stdout: "unittest_1_items": 0, 2024-09-10T10:04:56.167 INFO:teuthology.orchestra.run.smithi005.stdout: "unittest_2_bytes": 0, 2024-09-10T10:04:56.167 INFO:teuthology.orchestra.run.smithi005.stdout: "unittest_2_items": 0 2024-09-10T10:04:56.167 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:56.167 INFO:teuthology.orchestra.run.smithi005.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-10T10:04:56.167 INFO:teuthology.orchestra.run.smithi005.stdout: "val": 0, 2024-09-10T10:04:56.167 INFO:teuthology.orchestra.run.smithi005.stdout: "max": 104857600, 2024-09-10T10:04:56.167 INFO:teuthology.orchestra.run.smithi005.stdout: "get_started": 0, 2024-09-10T10:04:56.167 INFO:teuthology.orchestra.run.smithi005.stdout: "get": 480, 2024-09-10T10:04:56.167 INFO:teuthology.orchestra.run.smithi005.stdout: "get_sum": 760131, 2024-09-10T10:04:56.167 INFO:teuthology.orchestra.run.smithi005.stdout: "get_or_fail_fail": 0, 2024-09-10T10:04:56.167 INFO:teuthology.orchestra.run.smithi005.stdout: "get_or_fail_success": 480, 2024-09-10T10:04:56.167 INFO:teuthology.orchestra.run.smithi005.stdout: "take": 0, 2024-09-10T10:04:56.167 INFO:teuthology.orchestra.run.smithi005.stdout: "take_sum": 0, 2024-09-10T10:04:56.168 INFO:teuthology.orchestra.run.smithi005.stdout: "put": 480, 2024-09-10T10:04:56.168 INFO:teuthology.orchestra.run.smithi005.stdout: "put_sum": 760131, 2024-09-10T10:04:56.168 INFO:teuthology.orchestra.run.smithi005.stdout: "wait": { 2024-09-10T10:04:56.168 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-10T10:04:56.168 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-10T10:04:56.168 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-10T10:04:56.168 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T10:04:56.168 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T10:04:56.168 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-10T10:04:57.137 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.g 2024-09-10T10:04:57.138 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.g 2024-09-10T10:04:57.138 DEBUG:tasks.check_counter:Getting stats from j 2024-09-10T10:04:57.138 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph daemon mds.j perf dump 2024-09-10T10:04:57.527 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:04:57.995 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:04:57 smithi005 ceph-mon[30350]: pgmap v1191: 129 pgs: 129 active+clean; 551 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 17 KiB/s wr, 14 op/s 2024-09-10T10:04:57.995 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:04:57 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T10:04:57.996 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:04:57 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T10:04:57.996 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:04:57 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T10:04:57.996 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:04:57 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T10:04:57.996 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:04:57 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:04:57.996 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:04:57 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:04:57.996 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:04:57 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T10:04:58.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:04:57 smithi159 ceph-mon[35513]: pgmap v1191: 129 pgs: 129 active+clean; 551 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 17 KiB/s wr, 14 op/s 2024-09-10T10:04:58.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:04:57 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T10:04:58.101 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:04:57 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T10:04:58.101 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:04:57 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T10:04:58.101 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:04:57 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T10:04:58.101 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:04:57 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:04:58.101 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:04:57 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:04:58.101 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:04:57 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T10:04:58.135 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:04:57 smithi102 ceph-mon[35442]: pgmap v1191: 129 pgs: 129 active+clean; 551 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 17 KiB/s wr, 14 op/s 2024-09-10T10:04:58.135 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:04:57 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T10:04:58.135 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:04:57 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T10:04:58.135 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:04:57 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T10:04:58.135 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:04:57 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T10:04:58.135 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:04:57 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T10:04:58.135 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:04:57 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T10:04:58.135 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:04:57 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' 2024-09-10T10:04:58.141 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-10T10:04:58.141 INFO:teuthology.orchestra.run.smithi005.stdout: "AsyncMessenger::Worker-0": { 2024-09-10T10:04:58.141 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_messages": 485, 2024-09-10T10:04:58.142 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_messages": 480, 2024-09-10T10:04:58.142 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_bytes": 830256, 2024-09-10T10:04:58.142 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_bytes": 297088, 2024-09-10T10:04:58.142 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_created_connections": 4, 2024-09-10T10:04:58.142 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_active_connections": 0, 2024-09-10T10:04:58.142 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_total_time": 0.088539300, 2024-09-10T10:04:58.142 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_send_time": 0.048884650, 2024-09-10T10:04:58.142 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_recv_time": 0.044321935, 2024-09-10T10:04:58.142 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-10T10:04:58.142 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T10:04:58.142 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 479, 2024-09-10T10:04:58.142 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.040034490, 2024-09-10T10:04:58.143 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000083579 2024-09-10T10:04:58.143 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:58.143 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_handle_ack_lat": { 2024-09-10T10:04:58.143 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-10T10:04:58.143 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-10T10:04:58.143 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-10T10:04:58.143 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:58.143 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_encrypted_bytes": 830256, 2024-09-10T10:04:58.143 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_encrypted_bytes": 297088 2024-09-10T10:04:58.143 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:58.143 INFO:teuthology.orchestra.run.smithi005.stdout: "AsyncMessenger::Worker-1": { 2024-09-10T10:04:58.143 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_messages": 3, 2024-09-10T10:04:58.143 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_messages": 367, 2024-09-10T10:04:58.143 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_bytes": 480, 2024-09-10T10:04:58.143 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_bytes": 300880, 2024-09-10T10:04:58.144 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_created_connections": 5, 2024-09-10T10:04:58.144 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_active_connections": 2, 2024-09-10T10:04:58.144 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_total_time": 0.040530606, 2024-09-10T10:04:58.144 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_send_time": 0.033119609, 2024-09-10T10:04:58.144 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_recv_time": 0.002613008, 2024-09-10T10:04:58.144 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-10T10:04:58.144 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T10:04:58.144 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 365, 2024-09-10T10:04:58.144 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.026290047, 2024-09-10T10:04:58.144 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000072027 2024-09-10T10:04:58.144 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:58.144 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_handle_ack_lat": { 2024-09-10T10:04:58.145 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-10T10:04:58.145 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-10T10:04:58.145 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-10T10:04:58.145 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:58.145 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_encrypted_bytes": 480, 2024-09-10T10:04:58.145 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_encrypted_bytes": 300880 2024-09-10T10:04:58.145 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:58.145 INFO:teuthology.orchestra.run.smithi005.stdout: "AsyncMessenger::Worker-2": { 2024-09-10T10:04:58.145 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_messages": 4, 2024-09-10T10:04:58.145 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_messages": 4, 2024-09-10T10:04:58.145 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_bytes": 3024, 2024-09-10T10:04:58.145 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_bytes": 448, 2024-09-10T10:04:58.145 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_created_connections": 2, 2024-09-10T10:04:58.146 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_active_connections": 1, 2024-09-10T10:04:58.146 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_total_time": 0.003654576, 2024-09-10T10:04:58.146 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_send_time": 0.000071456, 2024-09-10T10:04:58.146 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_recv_time": 0.003520359, 2024-09-10T10:04:58.146 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-10T10:04:58.146 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T10:04:58.146 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 3, 2024-09-10T10:04:58.146 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.006154203, 2024-09-10T10:04:58.146 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.002051401 2024-09-10T10:04:58.146 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:58.146 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_handle_ack_lat": { 2024-09-10T10:04:58.146 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-10T10:04:58.146 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-10T10:04:58.146 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-10T10:04:58.147 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:58.147 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_encrypted_bytes": 3024, 2024-09-10T10:04:58.147 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_encrypted_bytes": 448 2024-09-10T10:04:58.147 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:58.147 INFO:teuthology.orchestra.run.smithi005.stdout: "cct": { 2024-09-10T10:04:58.147 INFO:teuthology.orchestra.run.smithi005.stdout: "total_workers": 0, 2024-09-10T10:04:58.147 INFO:teuthology.orchestra.run.smithi005.stdout: "unhealthy_workers": 0 2024-09-10T10:04:58.147 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:58.147 INFO:teuthology.orchestra.run.smithi005.stdout: "mempool": { 2024-09-10T10:04:58.147 INFO:teuthology.orchestra.run.smithi005.stdout: "bloom_filter_bytes": 0, 2024-09-10T10:04:58.147 INFO:teuthology.orchestra.run.smithi005.stdout: "bloom_filter_items": 0, 2024-09-10T10:04:58.147 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_alloc_bytes": 0, 2024-09-10T10:04:58.147 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_alloc_items": 0, 2024-09-10T10:04:58.148 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_data_bytes": 0, 2024-09-10T10:04:58.148 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_data_items": 0, 2024-09-10T10:04:58.148 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-10T10:04:58.148 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_onode_items": 0, 2024-09-10T10:04:58.148 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-10T10:04:58.148 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_meta_items": 0, 2024-09-10T10:04:58.148 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_other_bytes": 0, 2024-09-10T10:04:58.148 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_other_items": 0, 2024-09-10T10:04:58.148 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-10T10:04:58.148 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_buffer_items": 0, 2024-09-10T10:04:58.148 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_extent_bytes": 0, 2024-09-10T10:04:58.148 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_extent_items": 0, 2024-09-10T10:04:58.149 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_blob_bytes": 0, 2024-09-10T10:04:58.149 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_blob_items": 0, 2024-09-10T10:04:58.149 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-10T10:04:58.149 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_shared_blob_items": 0, 2024-09-10T10:04:58.149 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-10T10:04:58.149 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_inline_bl_items": 0, 2024-09-10T10:04:58.149 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_fsck_bytes": 0, 2024-09-10T10:04:58.149 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_fsck_items": 0, 2024-09-10T10:04:58.149 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_txc_bytes": 0, 2024-09-10T10:04:58.149 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_txc_items": 0, 2024-09-10T10:04:58.149 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-10T10:04:58.149 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_writing_deferred_items": 0, 2024-09-10T10:04:58.149 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_writing_bytes": 0, 2024-09-10T10:04:58.150 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_writing_items": 0, 2024-09-10T10:04:58.150 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_bytes": 0, 2024-09-10T10:04:58.150 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_items": 0, 2024-09-10T10:04:58.150 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_file_reader_bytes": 0, 2024-09-10T10:04:58.150 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_file_reader_items": 0, 2024-09-10T10:04:58.150 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_file_writer_bytes": 0, 2024-09-10T10:04:58.150 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_file_writer_items": 0, 2024-09-10T10:04:58.150 INFO:teuthology.orchestra.run.smithi005.stdout: "buffer_anon_bytes": 148704, 2024-09-10T10:04:58.150 INFO:teuthology.orchestra.run.smithi005.stdout: "buffer_anon_items": 29, 2024-09-10T10:04:58.150 INFO:teuthology.orchestra.run.smithi005.stdout: "buffer_meta_bytes": 0, 2024-09-10T10:04:58.150 INFO:teuthology.orchestra.run.smithi005.stdout: "buffer_meta_items": 0, 2024-09-10T10:04:58.150 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_bytes": 0, 2024-09-10T10:04:58.150 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_items": 0, 2024-09-10T10:04:58.151 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_mapbl_bytes": 0, 2024-09-10T10:04:58.151 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_mapbl_items": 0, 2024-09-10T10:04:58.151 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_pglog_bytes": 0, 2024-09-10T10:04:58.151 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_pglog_items": 0, 2024-09-10T10:04:58.151 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_bytes": 0, 2024-09-10T10:04:58.151 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_items": 0, 2024-09-10T10:04:58.151 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_mapping_bytes": 0, 2024-09-10T10:04:58.151 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_mapping_items": 0, 2024-09-10T10:04:58.151 INFO:teuthology.orchestra.run.smithi005.stdout: "pgmap_bytes": 0, 2024-09-10T10:04:58.151 INFO:teuthology.orchestra.run.smithi005.stdout: "pgmap_items": 0, 2024-09-10T10:04:58.151 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_co_bytes": 928, 2024-09-10T10:04:58.151 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_co_items": 2, 2024-09-10T10:04:58.151 INFO:teuthology.orchestra.run.smithi005.stdout: "unittest_1_bytes": 0, 2024-09-10T10:04:58.151 INFO:teuthology.orchestra.run.smithi005.stdout: "unittest_1_items": 0, 2024-09-10T10:04:58.152 INFO:teuthology.orchestra.run.smithi005.stdout: "unittest_2_bytes": 0, 2024-09-10T10:04:58.152 INFO:teuthology.orchestra.run.smithi005.stdout: "unittest_2_items": 0 2024-09-10T10:04:58.152 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-10T10:04:58.152 INFO:teuthology.orchestra.run.smithi005.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-10T10:04:58.152 INFO:teuthology.orchestra.run.smithi005.stdout: "val": 0, 2024-09-10T10:04:58.152 INFO:teuthology.orchestra.run.smithi005.stdout: "max": 104857600, 2024-09-10T10:04:58.152 INFO:teuthology.orchestra.run.smithi005.stdout: "get_started": 0, 2024-09-10T10:04:58.152 INFO:teuthology.orchestra.run.smithi005.stdout: "get": 488, 2024-09-10T10:04:58.152 INFO:teuthology.orchestra.run.smithi005.stdout: "get_sum": 761705, 2024-09-10T10:04:58.152 INFO:teuthology.orchestra.run.smithi005.stdout: "get_or_fail_fail": 0, 2024-09-10T10:04:58.152 INFO:teuthology.orchestra.run.smithi005.stdout: "get_or_fail_success": 488, 2024-09-10T10:04:58.152 INFO:teuthology.orchestra.run.smithi005.stdout: "take": 0, 2024-09-10T10:04:58.152 INFO:teuthology.orchestra.run.smithi005.stdout: "take_sum": 0, 2024-09-10T10:04:58.153 INFO:teuthology.orchestra.run.smithi005.stdout: "put": 488, 2024-09-10T10:04:58.153 INFO:teuthology.orchestra.run.smithi005.stdout: "put_sum": 761705, 2024-09-10T10:04:58.153 INFO:teuthology.orchestra.run.smithi005.stdout: "wait": { 2024-09-10T10:04:58.153 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-10T10:04:58.153 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-10T10:04:58.153 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-10T10:04:58.153 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T10:04:58.153 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-10T10:04:58.153 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-10T10:04:59.040 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2024-09-10T10:04:59.040 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2024-09-10T10:04:59.040 DEBUG:tasks.check_counter:Getting stats from b 2024-09-10T10:04:59.040 DEBUG:teuthology.orchestra.run.smithi102:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph daemon mds.b perf dump 2024-09-10T10:04:59.339 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T10:05:00.019 INFO:teuthology.orchestra.run.smithi102.stdout:{ 2024-09-10T10:05:00.019 INFO:teuthology.orchestra.run.smithi102.stdout: "AsyncMessenger::Worker-0": { 2024-09-10T10:05:00.019 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_recv_messages": 190032, 2024-09-10T10:05:00.020 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_messages": 270646, 2024-09-10T10:05:00.020 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_recv_bytes": 59192814, 2024-09-10T10:05:00.020 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_bytes": 275460857, 2024-09-10T10:05:00.020 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_created_connections": 30, 2024-09-10T10:05:00.020 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_active_connections": 4, 2024-09-10T10:05:00.020 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_total_time": 10.392104762, 2024-09-10T10:05:00.020 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_send_time": 4.736653790, 2024-09-10T10:05:00.020 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_recv_time": 6.545306677, 2024-09-10T10:05:00.020 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_fast_dispatch_time": 0.748080781, 2024-09-10T10:05:00.020 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T10:05:00.020 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 270648, 2024-09-10T10:05:00.020 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 7.546364248, 2024-09-10T10:05:00.021 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000027882 2024-09-10T10:05:00.021 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-10T10:05:00.021 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_handle_ack_lat": { 2024-09-10T10:05:00.021 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 229544, 2024-09-10T10:05:00.021 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.063999703, 2024-09-10T10:05:00.021 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000278 2024-09-10T10:05:00.021 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-10T10:05:00.021 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_recv_encrypted_bytes": 1082784, 2024-09-10T10:05:00.021 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_encrypted_bytes": 367680 2024-09-10T10:05:00.021 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-10T10:05:00.021 INFO:teuthology.orchestra.run.smithi102.stdout: "AsyncMessenger::Worker-1": { 2024-09-10T10:05:00.022 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_recv_messages": 46194, 2024-09-10T10:05:00.022 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_messages": 43572, 2024-09-10T10:05:00.022 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_recv_bytes": 13133401, 2024-09-10T10:05:00.022 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_bytes": 78138160, 2024-09-10T10:05:00.022 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_created_connections": 11, 2024-09-10T10:05:00.022 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_active_connections": 7, 2024-09-10T10:05:00.022 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_total_time": 3.454308223, 2024-09-10T10:05:00.023 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_send_time": 1.363012459, 2024-09-10T10:05:00.023 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_recv_time": 1.419346035, 2024-09-10T10:05:00.023 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_fast_dispatch_time": 0.764573340, 2024-09-10T10:05:00.024 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T10:05:00.024 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 43567, 2024-09-10T10:05:00.024 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 1.396463723, 2024-09-10T10:05:00.024 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000032053 2024-09-10T10:05:00.024 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-10T10:05:00.024 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_handle_ack_lat": { 2024-09-10T10:05:00.025 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 38972, 2024-09-10T10:05:00.025 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.007433341, 2024-09-10T10:05:00.028 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000190 2024-09-10T10:05:00.028 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-10T10:05:00.028 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_recv_encrypted_bytes": 160, 2024-09-10T10:05:00.028 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_encrypted_bytes": 732752 2024-09-10T10:05:00.028 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-10T10:05:00.029 INFO:teuthology.orchestra.run.smithi102.stdout: "AsyncMessenger::Worker-2": { 2024-09-10T10:05:00.029 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_recv_messages": 209746, 2024-09-10T10:05:00.029 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_messages": 156144, 2024-09-10T10:05:00.029 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_recv_bytes": 55513542, 2024-09-10T10:05:00.029 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_bytes": 90766755, 2024-09-10T10:05:00.030 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_created_connections": 318, 2024-09-10T10:05:00.030 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_active_connections": 5, 2024-09-10T10:05:00.030 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_total_time": 10.858963424, 2024-09-10T10:05:00.030 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_send_time": 4.184343148, 2024-09-10T10:05:00.030 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_recv_time": 7.671327010, 2024-09-10T10:05:00.030 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_fast_dispatch_time": 0.674886313, 2024-09-10T10:05:00.031 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T10:05:00.031 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 156143, 2024-09-10T10:05:00.031 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 3.533264022, 2024-09-10T10:05:00.031 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000022628 2024-09-10T10:05:00.031 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-10T10:05:00.032 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_handle_ack_lat": { 2024-09-10T10:05:00.032 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 324355, 2024-09-10T10:05:00.032 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.047519382, 2024-09-10T10:05:00.032 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000146 2024-09-10T10:05:00.032 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-10T10:05:00.033 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_recv_encrypted_bytes": 43056, 2024-09-10T10:05:00.033 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_encrypted_bytes": 5568 2024-09-10T10:05:00.033 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-10T10:05:00.033 INFO:teuthology.orchestra.run.smithi102.stdout: "cct": { 2024-09-10T10:05:00.033 INFO:teuthology.orchestra.run.smithi102.stdout: "total_workers": 1, 2024-09-10T10:05:00.034 INFO:teuthology.orchestra.run.smithi102.stdout: "unhealthy_workers": 0 2024-09-10T10:05:00.034 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-10T10:05:00.034 INFO:teuthology.orchestra.run.smithi102.stdout: "finisher-MDSRank": { 2024-09-10T10:05:00.034 INFO:teuthology.orchestra.run.smithi102.stdout: "queue_len": 0, 2024-09-10T10:05:00.034 INFO:teuthology.orchestra.run.smithi102.stdout: "complete_latency": { 2024-09-10T10:05:00.034 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 54974, 2024-09-10T10:05:00.035 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 35.756126149, 2024-09-10T10:05:00.035 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000650418 2024-09-10T10:05:00.035 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-10T10:05:00.035 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-10T10:05:00.035 INFO:teuthology.orchestra.run.smithi102.stdout: "finisher-PurgeQueue": { 2024-09-10T10:05:00.035 INFO:teuthology.orchestra.run.smithi102.stdout: "queue_len": 0, 2024-09-10T10:05:00.036 INFO:teuthology.orchestra.run.smithi102.stdout: "complete_latency": { 2024-09-10T10:05:00.036 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 27319, 2024-09-10T10:05:00.036 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 1.772434563, 2024-09-10T10:05:00.036 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000064879 2024-09-10T10:05:00.036 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-10T10:05:00.037 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-10T10:05:00.037 INFO:teuthology.orchestra.run.smithi102.stdout: "mds": { 2024-09-10T10:05:00.037 INFO:teuthology.orchestra.run.smithi102.stdout: "request": 107144, 2024-09-10T10:05:00.037 INFO:teuthology.orchestra.run.smithi102.stdout: "reply": 79399, 2024-09-10T10:05:00.037 INFO:teuthology.orchestra.run.smithi102.stdout: "reply_latency": { 2024-09-10T10:05:00.038 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 79399, 2024-09-10T10:05:00.038 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 421.976791335, 2024-09-10T10:05:00.038 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.005314636 2024-09-10T10:05:00.038 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-10T10:05:00.038 INFO:teuthology.orchestra.run.smithi102.stdout: "slow_reply": 0, 2024-09-10T10:05:00.039 INFO:teuthology.orchestra.run.smithi102.stdout: "forward": 27745, 2024-09-10T10:05:00.039 INFO:teuthology.orchestra.run.smithi102.stdout: "dir_fetch_complete": 0, 2024-09-10T10:05:00.039 INFO:teuthology.orchestra.run.smithi102.stdout: "dir_fetch_keys": 0, 2024-09-10T10:05:00.039 INFO:teuthology.orchestra.run.smithi102.stdout: "dir_commit": 12, 2024-09-10T10:05:00.039 INFO:teuthology.orchestra.run.smithi102.stdout: "dir_split": 0, 2024-09-10T10:05:00.039 INFO:teuthology.orchestra.run.smithi102.stdout: "dir_merge": 0, 2024-09-10T10:05:00.039 INFO:teuthology.orchestra.run.smithi102.stdout: "inodes": 30, 2024-09-10T10:05:00.040 INFO:teuthology.orchestra.run.smithi102.stdout: "inodes_top": 4, 2024-09-10T10:05:00.040 INFO:teuthology.orchestra.run.smithi102.stdout: "inodes_bottom": 7, 2024-09-10T10:05:00.040 INFO:teuthology.orchestra.run.smithi102.stdout: "inodes_pin_tail": 19, 2024-09-10T10:05:00.040 INFO:teuthology.orchestra.run.smithi102.stdout: "inodes_pinned": 24, 2024-09-10T10:05:00.040 INFO:teuthology.orchestra.run.smithi102.stdout: "inodes_expired": 206, 2024-09-10T10:05:00.040 INFO:teuthology.orchestra.run.smithi102.stdout: "inodes_with_caps": 2, 2024-09-10T10:05:00.040 INFO:teuthology.orchestra.run.smithi102.stdout: "caps": 2, 2024-09-10T10:05:00.040 INFO:teuthology.orchestra.run.smithi102.stdout: "subtrees": 5, 2024-09-10T10:05:00.040 INFO:teuthology.orchestra.run.smithi102.stdout: "traverse": 165405, 2024-09-10T10:05:00.040 INFO:teuthology.orchestra.run.smithi102.stdout: "traverse_hit": 68309, 2024-09-10T10:05:00.040 INFO:teuthology.orchestra.run.smithi102.stdout: "traverse_forward": 27745, 2024-09-10T10:05:00.040 INFO:teuthology.orchestra.run.smithi102.stdout: "traverse_discover": 16, 2024-09-10T10:05:00.040 INFO:teuthology.orchestra.run.smithi102.stdout: "traverse_dir_fetch": 0, 2024-09-10T10:05:00.041 INFO:teuthology.orchestra.run.smithi102.stdout: "traverse_remote_ino": 0, 2024-09-10T10:05:00.041 INFO:teuthology.orchestra.run.smithi102.stdout: "traverse_lock": 726, 2024-09-10T10:05:00.041 INFO:teuthology.orchestra.run.smithi102.stdout: "load_cent": 0, 2024-09-10T10:05:00.041 INFO:teuthology.orchestra.run.smithi102.stdout: "q": 0, 2024-09-10T10:05:00.041 INFO:teuthology.orchestra.run.smithi102.stdout: "exported": 11, 2024-09-10T10:05:00.041 INFO:teuthology.orchestra.run.smithi102.stdout: "exported_inodes": 127, 2024-09-10T10:05:00.041 INFO:teuthology.orchestra.run.smithi102.stdout: "imported": 3, 2024-09-10T10:05:00.041 INFO:teuthology.orchestra.run.smithi102.stdout: "imported_inodes": 105, 2024-09-10T10:05:00.041 INFO:teuthology.orchestra.run.smithi102.stdout: "openino_dir_fetch": 0, 2024-09-10T10:05:00.041 INFO:teuthology.orchestra.run.smithi102.stdout: "openino_backtrace_fetch": 1758, 2024-09-10T10:05:00.041 INFO:teuthology.orchestra.run.smithi102.stdout: "openino_peer_discover": 0, 2024-09-10T10:05:00.041 INFO:teuthology.orchestra.run.smithi102.stdout: "root_rfiles": 4, 2024-09-10T10:05:00.041 INFO:teuthology.orchestra.run.smithi102.stdout: "root_rbytes": 234, 2024-09-10T10:05:00.041 INFO:teuthology.orchestra.run.smithi102.stdout: "root_rsnaps": 0, 2024-09-10T10:05:00.042 INFO:teuthology.orchestra.run.smithi102.stdout: "scrub_backtrace_fetch": 1758, 2024-09-10T10:05:00.042 INFO:teuthology.orchestra.run.smithi102.stdout: "scrub_set_tag": 0, 2024-09-10T10:05:00.042 INFO:teuthology.orchestra.run.smithi102.stdout: "scrub_backtrace_repaired": 0, 2024-09-10T10:05:00.042 INFO:teuthology.orchestra.run.smithi102.stdout: "scrub_inotable_repaired": 0, 2024-09-10T10:05:00.042 INFO:teuthology.orchestra.run.smithi102.stdout: "scrub_dir_inodes": 358, 2024-09-10T10:05:00.042 INFO:teuthology.orchestra.run.smithi102.stdout: "scrub_dir_base_inodes": 28, 2024-09-10T10:05:00.042 INFO:teuthology.orchestra.run.smithi102.stdout: "scrub_dirfrag_rstats": 330, 2024-09-10T10:05:00.042 INFO:teuthology.orchestra.run.smithi102.stdout: "scrub_file_inodes": 1400, 2024-09-10T10:05:00.042 INFO:teuthology.orchestra.run.smithi102.stdout: "handle_inode_file_caps": 18, 2024-09-10T10:05:00.042 INFO:teuthology.orchestra.run.smithi102.stdout: "ceph_cap_op_revoke": 15601, 2024-09-10T10:05:00.042 INFO:teuthology.orchestra.run.smithi102.stdout: "ceph_cap_op_grant": 72515, 2024-09-10T10:05:00.042 INFO:teuthology.orchestra.run.smithi102.stdout: "ceph_cap_op_trunc": 0, 2024-09-10T10:05:00.042 INFO:teuthology.orchestra.run.smithi102.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-10T10:05:00.042 INFO:teuthology.orchestra.run.smithi102.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-10T10:05:00.043 INFO:teuthology.orchestra.run.smithi102.stdout: "handle_client_caps": 44448, 2024-09-10T10:05:00.043 INFO:teuthology.orchestra.run.smithi102.stdout: "handle_client_caps_dirty": 13569, 2024-09-10T10:05:00.043 INFO:teuthology.orchestra.run.smithi102.stdout: "handle_client_cap_release": 4662, 2024-09-10T10:05:00.043 INFO:teuthology.orchestra.run.smithi102.stdout: "process_request_cap_release": 54702 2024-09-10T10:05:00.043 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-10T10:05:00.043 INFO:teuthology.orchestra.run.smithi102.stdout: "mds_cache": { 2024-09-10T10:05:00.043 INFO:teuthology.orchestra.run.smithi102.stdout: "dir_update": 279, 2024-09-10T10:05:00.043 INFO:teuthology.orchestra.run.smithi102.stdout: "dir_update_receipt": 34, 2024-09-10T10:05:00.043 INFO:teuthology.orchestra.run.smithi102.stdout: "dir_try_discover": 24, 2024-09-10T10:05:00.043 INFO:teuthology.orchestra.run.smithi102.stdout: "dir_send_discover": 32, 2024-09-10T10:05:00.043 INFO:teuthology.orchestra.run.smithi102.stdout: "dir_handle_discover": 315, 2024-09-10T10:05:00.043 INFO:teuthology.orchestra.run.smithi102.stdout: "num_strays": 1, 2024-09-10T10:05:00.043 INFO:teuthology.orchestra.run.smithi102.stdout: "num_strays_delayed": 0, 2024-09-10T10:05:00.043 INFO:teuthology.orchestra.run.smithi102.stdout: "num_strays_enqueuing": 0, 2024-09-10T10:05:00.044 INFO:teuthology.orchestra.run.smithi102.stdout: "strays_created": 13646, 2024-09-10T10:05:00.044 INFO:teuthology.orchestra.run.smithi102.stdout: "strays_enqueued": 13645, 2024-09-10T10:05:00.044 INFO:teuthology.orchestra.run.smithi102.stdout: "strays_reintegrated": 0, 2024-09-10T10:05:00.044 INFO:teuthology.orchestra.run.smithi102.stdout: "strays_migrated": 0, 2024-09-10T10:05:00.044 INFO:teuthology.orchestra.run.smithi102.stdout: "num_recovering_processing": 0, 2024-09-10T10:05:00.044 INFO:teuthology.orchestra.run.smithi102.stdout: "num_recovering_enqueued": 0, 2024-09-10T10:05:00.044 INFO:teuthology.orchestra.run.smithi102.stdout: "num_recovering_prioritized": 0, 2024-09-10T10:05:00.044 INFO:teuthology.orchestra.run.smithi102.stdout: "recovery_started": 0, 2024-09-10T10:05:00.044 INFO:teuthology.orchestra.run.smithi102.stdout: "recovery_completed": 0, 2024-09-10T10:05:00.044 INFO:teuthology.orchestra.run.smithi102.stdout: "ireq_quiesce_path": 23, 2024-09-10T10:05:00.044 INFO:teuthology.orchestra.run.smithi102.stdout: "ireq_quiesce_inode": 2037, 2024-09-10T10:05:00.044 INFO:teuthology.orchestra.run.smithi102.stdout: "ireq_enqueue_scrub": 28, 2024-09-10T10:05:00.044 INFO:teuthology.orchestra.run.smithi102.stdout: "ireq_exportdir": 13, 2024-09-10T10:05:00.044 INFO:teuthology.orchestra.run.smithi102.stdout: "ireq_flush": 0, 2024-09-10T10:05:00.045 INFO:teuthology.orchestra.run.smithi102.stdout: "ireq_fragmentdir": 0, 2024-09-10T10:05:00.045 INFO:teuthology.orchestra.run.smithi102.stdout: "ireq_fragstats": 0, 2024-09-10T10:05:00.045 INFO:teuthology.orchestra.run.smithi102.stdout: "ireq_inodestats": 0 2024-09-10T10:05:00.045 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-10T10:05:00.045 INFO:teuthology.orchestra.run.smithi102.stdout: "mds_log": { 2024-09-10T10:05:00.045 INFO:teuthology.orchestra.run.smithi102.stdout: "evlrg": 0, 2024-09-10T10:05:00.045 INFO:teuthology.orchestra.run.smithi102.stdout: "evadd": 81538, 2024-09-10T10:05:00.045 INFO:teuthology.orchestra.run.smithi102.stdout: "evex": 0, 2024-09-10T10:05:00.045 INFO:teuthology.orchestra.run.smithi102.stdout: "evtrm": 0, 2024-09-10T10:05:00.045 INFO:teuthology.orchestra.run.smithi102.stdout: "ev": 81538, 2024-09-10T10:05:00.045 INFO:teuthology.orchestra.run.smithi102.stdout: "evexg": 0, 2024-09-10T10:05:00.046 INFO:teuthology.orchestra.run.smithi102.stdout: "evexd": 0, 2024-09-10T10:05:00.046 INFO:teuthology.orchestra.run.smithi102.stdout: "segadd": 100, 2024-09-10T10:05:00.046 INFO:teuthology.orchestra.run.smithi102.stdout: "segex": 0, 2024-09-10T10:05:00.046 INFO:teuthology.orchestra.run.smithi102.stdout: "segtrm": 0, 2024-09-10T10:05:00.046 INFO:teuthology.orchestra.run.smithi102.stdout: "seg": 100, 2024-09-10T10:05:00.046 INFO:teuthology.orchestra.run.smithi102.stdout: "segmjr": 8, 2024-09-10T10:05:00.046 INFO:teuthology.orchestra.run.smithi102.stdout: "segexg": 0, 2024-09-10T10:05:00.046 INFO:teuthology.orchestra.run.smithi102.stdout: "segexd": 0, 2024-09-10T10:05:00.046 INFO:teuthology.orchestra.run.smithi102.stdout: "expos": 4194304, 2024-09-10T10:05:00.046 INFO:teuthology.orchestra.run.smithi102.stdout: "wrpos": 147656294, 2024-09-10T10:05:00.046 INFO:teuthology.orchestra.run.smithi102.stdout: "rdpos": 0, 2024-09-10T10:05:00.046 INFO:teuthology.orchestra.run.smithi102.stdout: "jlat": { 2024-09-10T10:05:00.046 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 27019, 2024-09-10T10:05:00.046 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 103.130890234, 2024-09-10T10:05:00.047 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.003816976 2024-09-10T10:05:00.047 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-10T10:05:00.047 INFO:teuthology.orchestra.run.smithi102.stdout: "replayed": 0 2024-09-10T10:05:00.047 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-10T10:05:00.047 INFO:teuthology.orchestra.run.smithi102.stdout: "mds_mem": { 2024-09-10T10:05:00.047 INFO:teuthology.orchestra.run.smithi102.stdout: "ino": 31, 2024-09-10T10:05:00.047 INFO:teuthology.orchestra.run.smithi102.stdout: "ino+": 13788, 2024-09-10T10:05:00.047 INFO:teuthology.orchestra.run.smithi102.stdout: "ino-": 13757, 2024-09-10T10:05:00.047 INFO:teuthology.orchestra.run.smithi102.stdout: "dir": 34, 2024-09-10T10:05:00.047 INFO:teuthology.orchestra.run.smithi102.stdout: "dir+": 112, 2024-09-10T10:05:00.047 INFO:teuthology.orchestra.run.smithi102.stdout: "dir-": 78, 2024-09-10T10:05:00.047 INFO:teuthology.orchestra.run.smithi102.stdout: "dn": 30, 2024-09-10T10:05:00.047 INFO:teuthology.orchestra.run.smithi102.stdout: "dn+": 29238, 2024-09-10T10:05:00.048 INFO:teuthology.orchestra.run.smithi102.stdout: "dn-": 29208, 2024-09-10T10:05:00.048 INFO:teuthology.orchestra.run.smithi102.stdout: "cap": 2, 2024-09-10T10:05:00.048 INFO:teuthology.orchestra.run.smithi102.stdout: "cap+": 16780, 2024-09-10T10:05:00.048 INFO:teuthology.orchestra.run.smithi102.stdout: "cap-": 16778, 2024-09-10T10:05:00.048 INFO:teuthology.orchestra.run.smithi102.stdout: "rss": 71404, 2024-09-10T10:05:00.048 INFO:teuthology.orchestra.run.smithi102.stdout: "heap": 231712 2024-09-10T10:05:00.048 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-10T10:05:00.048 INFO:teuthology.orchestra.run.smithi102.stdout: "mds_server": { 2024-09-10T10:05:00.048 INFO:teuthology.orchestra.run.smithi102.stdout: "dispatch_client_request": 150039, 2024-09-10T10:05:00.048 INFO:teuthology.orchestra.run.smithi102.stdout: "dispatch_server_request": 3008, 2024-09-10T10:05:00.048 INFO:teuthology.orchestra.run.smithi102.stdout: "handle_client_request": 107144, 2024-09-10T10:05:00.048 INFO:teuthology.orchestra.run.smithi102.stdout: "handle_client_session": 1614, 2024-09-10T10:05:00.048 INFO:teuthology.orchestra.run.smithi102.stdout: "handle_peer_request": 15815, 2024-09-10T10:05:00.048 INFO:teuthology.orchestra.run.smithi102.stdout: "req_create_latency": { 2024-09-10T10:05:00.049 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 13609, 2024-09-10T10:05:00.049 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 203.925674691, 2024-09-10T10:05:00.049 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.014984618 2024-09-10T10:05:00.049 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-10T10:05:00.049 INFO:teuthology.orchestra.run.smithi102.stdout: "req_getattr_latency": { 2024-09-10T10:05:00.049 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 1882, 2024-09-10T10:05:00.049 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 25.245156290, 2024-09-10T10:05:00.049 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.013414004 2024-09-10T10:05:00.049 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-10T10:05:00.050 INFO:teuthology.orchestra.run.smithi102.stdout: "req_getfilelock_latency": { 2024-09-10T10:05:00.050 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-10T10:05:00.050 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-10T10:05:00.050 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-10T10:05:00.050 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-10T10:05:00.050 INFO:teuthology.orchestra.run.smithi102.stdout: "req_link_latency": { 2024-09-10T10:05:00.050 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-10T10:05:00.050 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-10T10:05:00.050 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-10T10:05:00.050 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-10T10:05:00.050 INFO:teuthology.orchestra.run.smithi102.stdout: "req_lookup_latency": { 2024-09-10T10:05:00.050 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 23310, 2024-09-10T10:05:00.050 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 17.786064503, 2024-09-10T10:05:00.051 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000763022 2024-09-10T10:05:00.051 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-10T10:05:00.051 INFO:teuthology.orchestra.run.smithi102.stdout: "req_lookuphash_latency": { 2024-09-10T10:05:00.051 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-10T10:05:00.051 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-10T10:05:00.051 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-10T10:05:00.051 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-10T10:05:00.051 INFO:teuthology.orchestra.run.smithi102.stdout: "req_lookupino_latency": { 2024-09-10T10:05:00.051 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 1, 2024-09-10T10:05:00.051 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000172734, 2024-09-10T10:05:00.051 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000172734 2024-09-10T10:05:00.051 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-10T10:05:00.051 INFO:teuthology.orchestra.run.smithi102.stdout: "req_lookupname_latency": { 2024-09-10T10:05:00.051 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-10T10:05:00.052 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-10T10:05:00.052 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-10T10:05:00.052 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-10T10:05:00.052 INFO:teuthology.orchestra.run.smithi102.stdout: "req_lookupparent_latency": { 2024-09-10T10:05:00.052 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-10T10:05:00.052 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-10T10:05:00.052 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-10T10:05:00.052 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-10T10:05:00.052 INFO:teuthology.orchestra.run.smithi102.stdout: "req_lookupsnap_latency": { 2024-09-10T10:05:00.052 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-10T10:05:00.053 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-10T10:05:00.054 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-10T10:05:00.054 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-10T10:05:00.054 INFO:teuthology.orchestra.run.smithi102.stdout: "req_lssnap_latency": { 2024-09-10T10:05:00.054 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-10T10:05:00.054 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-10T10:05:00.054 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-10T10:05:00.054 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-10T10:05:00.054 INFO:teuthology.orchestra.run.smithi102.stdout: "req_mkdir_latency": { 2024-09-10T10:05:00.055 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 20, 2024-09-10T10:05:00.055 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 26.508037980, 2024-09-10T10:05:00.055 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 1.325401899 2024-09-10T10:05:00.055 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-10T10:05:00.055 INFO:teuthology.orchestra.run.smithi102.stdout: "req_mknod_latency": { 2024-09-10T10:05:00.055 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-10T10:05:00.055 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-10T10:05:00.055 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-10T10:05:00.055 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-10T10:05:00.055 INFO:teuthology.orchestra.run.smithi102.stdout: "req_mksnap_latency": { 2024-09-10T10:05:00.055 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-10T10:05:00.055 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-10T10:05:00.055 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-10T10:05:00.056 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-10T10:05:00.056 INFO:teuthology.orchestra.run.smithi102.stdout: "req_open_latency": { 2024-09-10T10:05:00.056 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 1, 2024-09-10T10:05:00.056 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000230400, 2024-09-10T10:05:00.056 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000230400 2024-09-10T10:05:00.056 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-10T10:05:00.056 INFO:teuthology.orchestra.run.smithi102.stdout: "req_readdir_latency": { 2024-09-10T10:05:00.056 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 24659, 2024-09-10T10:05:00.056 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 67.898512319, 2024-09-10T10:05:00.056 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.002753498 2024-09-10T10:05:00.056 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-10T10:05:00.056 INFO:teuthology.orchestra.run.smithi102.stdout: "req_rename_latency": { 2024-09-10T10:05:00.056 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 1921, 2024-09-10T10:05:00.056 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 3.738341687, 2024-09-10T10:05:00.057 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.001946039 2024-09-10T10:05:00.057 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-10T10:05:00.057 INFO:teuthology.orchestra.run.smithi102.stdout: "req_renamesnap_latency": { 2024-09-10T10:05:00.057 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-10T10:05:00.057 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-10T10:05:00.057 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-10T10:05:00.057 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-10T10:05:00.057 INFO:teuthology.orchestra.run.smithi102.stdout: "req_snapdiff_latency": { 2024-09-10T10:05:00.057 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-10T10:05:00.057 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-10T10:05:00.057 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-10T10:05:00.057 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-10T10:05:00.057 INFO:teuthology.orchestra.run.smithi102.stdout: "req_rmdir_latency": { 2024-09-10T10:05:00.057 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 12, 2024-09-10T10:05:00.058 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.099542221, 2024-09-10T10:05:00.058 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.008295185 2024-09-10T10:05:00.058 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-10T10:05:00.058 INFO:teuthology.orchestra.run.smithi102.stdout: "req_rmsnap_latency": { 2024-09-10T10:05:00.058 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-10T10:05:00.058 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-10T10:05:00.058 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-10T10:05:00.058 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-10T10:05:00.058 INFO:teuthology.orchestra.run.smithi102.stdout: "req_rmxattr_latency": { 2024-09-10T10:05:00.058 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-10T10:05:00.058 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-10T10:05:00.058 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-10T10:05:00.058 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-10T10:05:00.058 INFO:teuthology.orchestra.run.smithi102.stdout: "req_setattr_latency": { 2024-09-10T10:05:00.059 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 17, 2024-09-10T10:05:00.059 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 31.744482693, 2024-09-10T10:05:00.059 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 1.867322511 2024-09-10T10:05:00.059 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-10T10:05:00.059 INFO:teuthology.orchestra.run.smithi102.stdout: "req_setdirlayout_latency": { 2024-09-10T10:05:00.059 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-10T10:05:00.059 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-10T10:05:00.059 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-10T10:05:00.059 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-10T10:05:00.059 INFO:teuthology.orchestra.run.smithi102.stdout: "req_setfilelock_latency": { 2024-09-10T10:05:00.059 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 324, 2024-09-10T10:05:00.059 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.126023931, 2024-09-10T10:05:00.059 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000388962 2024-09-10T10:05:00.060 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-10T10:05:00.060 INFO:teuthology.orchestra.run.smithi102.stdout: "req_setlayout_latency": { 2024-09-10T10:05:00.060 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-10T10:05:00.060 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-10T10:05:00.060 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-10T10:05:00.060 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-10T10:05:00.060 INFO:teuthology.orchestra.run.smithi102.stdout: "req_setxattr_latency": { 2024-09-10T10:05:00.060 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 9, 2024-09-10T10:05:00.060 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.020986333, 2024-09-10T10:05:00.060 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.002331814 2024-09-10T10:05:00.060 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-10T10:05:00.060 INFO:teuthology.orchestra.run.smithi102.stdout: "req_symlink_latency": { 2024-09-10T10:05:00.060 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-10T10:05:00.060 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-10T10:05:00.061 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-10T10:05:00.061 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-10T10:05:00.061 INFO:teuthology.orchestra.run.smithi102.stdout: "req_unlink_latency": { 2024-09-10T10:05:00.061 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 13634, 2024-09-10T10:05:00.061 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 44.883565553, 2024-09-10T10:05:00.061 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.003292032 2024-09-10T10:05:00.061 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-10T10:05:00.061 INFO:teuthology.orchestra.run.smithi102.stdout: "cap_revoke_eviction": 0, 2024-09-10T10:05:00.061 INFO:teuthology.orchestra.run.smithi102.stdout: "cap_acquisition_throttle": 0, 2024-09-10T10:05:00.061 INFO:teuthology.orchestra.run.smithi102.stdout: "req_getvxattr_latency": { 2024-09-10T10:05:00.061 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-10T10:05:00.061 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-10T10:05:00.061 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-10T10:05:00.061 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-10T10:05:00.061 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-10T10:05:00.062 INFO:teuthology.orchestra.run.smithi102.stdout: "mds_sessions": { 2024-09-10T10:05:00.062 INFO:teuthology.orchestra.run.smithi102.stdout: "session_count": 1, 2024-09-10T10:05:00.062 INFO:teuthology.orchestra.run.smithi102.stdout: "session_add": 4, 2024-09-10T10:05:00.062 INFO:teuthology.orchestra.run.smithi102.stdout: "session_remove": 3, 2024-09-10T10:05:00.062 INFO:teuthology.orchestra.run.smithi102.stdout: "sessions_open": 1, 2024-09-10T10:05:00.062 INFO:teuthology.orchestra.run.smithi102.stdout: "sessions_stale": 0, 2024-09-10T10:05:00.062 INFO:teuthology.orchestra.run.smithi102.stdout: "total_load": 4997, 2024-09-10T10:05:00.062 INFO:teuthology.orchestra.run.smithi102.stdout: "average_load": 4997, 2024-09-10T10:05:00.062 INFO:teuthology.orchestra.run.smithi102.stdout: "avg_session_uptime": 1602, 2024-09-10T10:05:00.062 INFO:teuthology.orchestra.run.smithi102.stdout: "mdthresh_evicted": 0 2024-09-10T10:05:00.062 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-10T10:05:00.062 INFO:teuthology.orchestra.run.smithi102.stdout: "mempool": { 2024-09-10T10:05:00.063 INFO:teuthology.orchestra.run.smithi102.stdout: "bloom_filter_bytes": 0, 2024-09-10T10:05:00.063 INFO:teuthology.orchestra.run.smithi102.stdout: "bloom_filter_items": 0, 2024-09-10T10:05:00.063 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_alloc_bytes": 0, 2024-09-10T10:05:00.063 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_alloc_items": 0, 2024-09-10T10:05:00.063 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_data_bytes": 0, 2024-09-10T10:05:00.063 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_data_items": 0, 2024-09-10T10:05:00.063 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-10T10:05:00.063 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_onode_items": 0, 2024-09-10T10:05:00.063 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-10T10:05:00.063 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_meta_items": 0, 2024-09-10T10:05:00.063 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_other_bytes": 0, 2024-09-10T10:05:00.063 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_other_items": 0, 2024-09-10T10:05:00.063 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-10T10:05:00.063 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_buffer_items": 0, 2024-09-10T10:05:00.063 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_extent_bytes": 0, 2024-09-10T10:05:00.064 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_extent_items": 0, 2024-09-10T10:05:00.064 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_blob_bytes": 0, 2024-09-10T10:05:00.064 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_blob_items": 0, 2024-09-10T10:05:00.064 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-10T10:05:00.064 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_shared_blob_items": 0, 2024-09-10T10:05:00.064 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-10T10:05:00.064 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_inline_bl_items": 0, 2024-09-10T10:05:00.064 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_fsck_bytes": 0, 2024-09-10T10:05:00.064 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_fsck_items": 0, 2024-09-10T10:05:00.064 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_txc_bytes": 0, 2024-09-10T10:05:00.064 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_txc_items": 0, 2024-09-10T10:05:00.064 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-10T10:05:00.064 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_writing_deferred_items": 0, 2024-09-10T10:05:00.064 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_writing_bytes": 0, 2024-09-10T10:05:00.064 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_writing_items": 0, 2024-09-10T10:05:00.065 INFO:teuthology.orchestra.run.smithi102.stdout: "bluefs_bytes": 0, 2024-09-10T10:05:00.065 INFO:teuthology.orchestra.run.smithi102.stdout: "bluefs_items": 0, 2024-09-10T10:05:00.065 INFO:teuthology.orchestra.run.smithi102.stdout: "bluefs_file_reader_bytes": 0, 2024-09-10T10:05:00.065 INFO:teuthology.orchestra.run.smithi102.stdout: "bluefs_file_reader_items": 0, 2024-09-10T10:05:00.065 INFO:teuthology.orchestra.run.smithi102.stdout: "bluefs_file_writer_bytes": 0, 2024-09-10T10:05:00.065 INFO:teuthology.orchestra.run.smithi102.stdout: "bluefs_file_writer_items": 0, 2024-09-10T10:05:00.065 INFO:teuthology.orchestra.run.smithi102.stdout: "buffer_anon_bytes": 302859, 2024-09-10T10:05:00.065 INFO:teuthology.orchestra.run.smithi102.stdout: "buffer_anon_items": 103, 2024-09-10T10:05:00.065 INFO:teuthology.orchestra.run.smithi102.stdout: "buffer_meta_bytes": 88, 2024-09-10T10:05:00.065 INFO:teuthology.orchestra.run.smithi102.stdout: "buffer_meta_items": 1, 2024-09-10T10:05:00.065 INFO:teuthology.orchestra.run.smithi102.stdout: "osd_bytes": 0, 2024-09-10T10:05:00.065 INFO:teuthology.orchestra.run.smithi102.stdout: "osd_items": 0, 2024-09-10T10:05:00.065 INFO:teuthology.orchestra.run.smithi102.stdout: "osd_mapbl_bytes": 0, 2024-09-10T10:05:00.065 INFO:teuthology.orchestra.run.smithi102.stdout: "osd_mapbl_items": 0, 2024-09-10T10:05:00.066 INFO:teuthology.orchestra.run.smithi102.stdout: "osd_pglog_bytes": 0, 2024-09-10T10:05:00.066 INFO:teuthology.orchestra.run.smithi102.stdout: "osd_pglog_items": 0, 2024-09-10T10:05:00.066 INFO:teuthology.orchestra.run.smithi102.stdout: "osdmap_bytes": 9152, 2024-09-10T10:05:00.066 INFO:teuthology.orchestra.run.smithi102.stdout: "osdmap_items": 173, 2024-09-10T10:05:00.066 INFO:teuthology.orchestra.run.smithi102.stdout: "osdmap_mapping_bytes": 0, 2024-09-10T10:05:00.066 INFO:teuthology.orchestra.run.smithi102.stdout: "osdmap_mapping_items": 0, 2024-09-10T10:05:00.066 INFO:teuthology.orchestra.run.smithi102.stdout: "pgmap_bytes": 0, 2024-09-10T10:05:00.066 INFO:teuthology.orchestra.run.smithi102.stdout: "pgmap_items": 0, 2024-09-10T10:05:00.066 INFO:teuthology.orchestra.run.smithi102.stdout: "mds_co_bytes": 140357, 2024-09-10T10:05:00.066 INFO:teuthology.orchestra.run.smithi102.stdout: "mds_co_items": 962, 2024-09-10T10:05:00.066 INFO:teuthology.orchestra.run.smithi102.stdout: "unittest_1_bytes": 0, 2024-09-10T10:05:00.066 INFO:teuthology.orchestra.run.smithi102.stdout: "unittest_1_items": 0, 2024-09-10T10:05:00.066 INFO:teuthology.orchestra.run.smithi102.stdout: "unittest_2_bytes": 0, 2024-09-10T10:05:00.066 INFO:teuthology.orchestra.run.smithi102.stdout: "unittest_2_items": 0 2024-09-10T10:05:00.067 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-10T10:05:00.067 INFO:teuthology.orchestra.run.smithi102.stdout: "objecter": { 2024-09-10T10:05:00.067 INFO:teuthology.orchestra.run.smithi102.stdout: "op_active": 0, 2024-09-10T10:05:00.067 INFO:teuthology.orchestra.run.smithi102.stdout: "op_laggy": 0, 2024-09-10T10:05:00.067 INFO:teuthology.orchestra.run.smithi102.stdout: "op_send": 59438, 2024-09-10T10:05:00.067 INFO:teuthology.orchestra.run.smithi102.stdout: "op_send_bytes": 146543326, 2024-09-10T10:05:00.067 INFO:teuthology.orchestra.run.smithi102.stdout: "op_resend": 37, 2024-09-10T10:05:00.067 INFO:teuthology.orchestra.run.smithi102.stdout: "op_reply": 59438, 2024-09-10T10:05:00.067 INFO:teuthology.orchestra.run.smithi102.stdout: "op_latency": { 2024-09-10T10:05:00.067 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 59438, 2024-09-10T10:05:00.067 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 381.691758741, 2024-09-10T10:05:00.067 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.006421679 2024-09-10T10:05:00.067 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-10T10:05:00.068 INFO:teuthology.orchestra.run.smithi102.stdout: "op_inflight": 0, 2024-09-10T10:05:00.068 INFO:teuthology.orchestra.run.smithi102.stdout: "oplen_avg": { 2024-09-10T10:05:00.068 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 59438, 2024-09-10T10:05:00.068 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 60694 2024-09-10T10:05:00.068 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-10T10:05:00.068 INFO:teuthology.orchestra.run.smithi102.stdout: "op": 59438, 2024-09-10T10:05:00.068 INFO:teuthology.orchestra.run.smithi102.stdout: "op_r": 2484, 2024-09-10T10:05:00.068 INFO:teuthology.orchestra.run.smithi102.stdout: "op_w": 56954, 2024-09-10T10:05:00.068 INFO:teuthology.orchestra.run.smithi102.stdout: "op_rmw": 0, 2024-09-10T10:05:00.068 INFO:teuthology.orchestra.run.smithi102.stdout: "op_pg": 0, 2024-09-10T10:05:00.068 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_stat": 0, 2024-09-10T10:05:00.068 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_create": 11, 2024-09-10T10:05:00.068 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_read": 698, 2024-09-10T10:05:00.068 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_write": 27689, 2024-09-10T10:05:00.068 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_writefull": 462, 2024-09-10T10:05:00.069 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_writesame": 0, 2024-09-10T10:05:00.069 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_append": 0, 2024-09-10T10:05:00.069 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_zero": 0, 2024-09-10T10:05:00.069 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_truncate": 0, 2024-09-10T10:05:00.069 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_delete": 27825, 2024-09-10T10:05:00.069 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_mapext": 0, 2024-09-10T10:05:00.069 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_sparse_read": 0, 2024-09-10T10:05:00.069 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_clonerange": 0, 2024-09-10T10:05:00.069 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_getxattr": 1786, 2024-09-10T10:05:00.069 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_setxattr": 22, 2024-09-10T10:05:00.069 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_cmpxattr": 0, 2024-09-10T10:05:00.069 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_rmxattr": 0, 2024-09-10T10:05:00.069 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_resetxattrs": 0, 2024-09-10T10:05:00.069 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_call": 0, 2024-09-10T10:05:00.070 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_watch": 0, 2024-09-10T10:05:00.070 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_notify": 0, 2024-09-10T10:05:00.070 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_src_cmpxattr": 0, 2024-09-10T10:05:00.070 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_pgls": 0, 2024-09-10T10:05:00.070 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_pgls_filter": 0, 2024-09-10T10:05:00.070 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_other": 0, 2024-09-10T10:05:00.070 INFO:teuthology.orchestra.run.smithi102.stdout: "linger_active": 0, 2024-09-10T10:05:00.070 INFO:teuthology.orchestra.run.smithi102.stdout: "linger_send": 0, 2024-09-10T10:05:00.070 INFO:teuthology.orchestra.run.smithi102.stdout: "linger_resend": 0, 2024-09-10T10:05:00.070 INFO:teuthology.orchestra.run.smithi102.stdout: "linger_ping": 0, 2024-09-10T10:05:00.070 INFO:teuthology.orchestra.run.smithi102.stdout: "poolop_active": 0, 2024-09-10T10:05:00.070 INFO:teuthology.orchestra.run.smithi102.stdout: "poolop_send": 0, 2024-09-10T10:05:00.070 INFO:teuthology.orchestra.run.smithi102.stdout: "poolop_resend": 0, 2024-09-10T10:05:00.070 INFO:teuthology.orchestra.run.smithi102.stdout: "poolstat_active": 0, 2024-09-10T10:05:00.070 INFO:teuthology.orchestra.run.smithi102.stdout: "poolstat_send": 0, 2024-09-10T10:05:00.071 INFO:teuthology.orchestra.run.smithi102.stdout: "poolstat_resend": 0, 2024-09-10T10:05:00.071 INFO:teuthology.orchestra.run.smithi102.stdout: "statfs_active": 0, 2024-09-10T10:05:00.071 INFO:teuthology.orchestra.run.smithi102.stdout: "statfs_send": 0, 2024-09-10T10:05:00.071 INFO:teuthology.orchestra.run.smithi102.stdout: "statfs_resend": 0, 2024-09-10T10:05:00.071 INFO:teuthology.orchestra.run.smithi102.stdout: "command_active": 0, 2024-09-10T10:05:00.071 INFO:teuthology.orchestra.run.smithi102.stdout: "command_send": 0, 2024-09-10T10:05:00.071 INFO:teuthology.orchestra.run.smithi102.stdout: "command_resend": 0, 2024-09-10T10:05:00.071 INFO:teuthology.orchestra.run.smithi102.stdout: "map_epoch": 96, 2024-09-10T10:05:00.071 INFO:teuthology.orchestra.run.smithi102.stdout: "map_full": 0, 2024-09-10T10:05:00.071 INFO:teuthology.orchestra.run.smithi102.stdout: "map_inc": 11, 2024-09-10T10:05:00.071 INFO:teuthology.orchestra.run.smithi102.stdout: "osd_sessions": 12, 2024-09-10T10:05:00.071 INFO:teuthology.orchestra.run.smithi102.stdout: "osd_session_open": 12, 2024-09-10T10:05:00.071 INFO:teuthology.orchestra.run.smithi102.stdout: "osd_session_close": 0, 2024-09-10T10:05:00.071 INFO:teuthology.orchestra.run.smithi102.stdout: "osd_laggy": 0, 2024-09-10T10:05:00.072 INFO:teuthology.orchestra.run.smithi102.stdout: "omap_wr": 1730, 2024-09-10T10:05:00.072 INFO:teuthology.orchestra.run.smithi102.stdout: "omap_rd": 0, 2024-09-10T10:05:00.072 INFO:teuthology.orchestra.run.smithi102.stdout: "omap_del": 471 2024-09-10T10:05:00.072 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-10T10:05:00.072 INFO:teuthology.orchestra.run.smithi102.stdout: "oft": { 2024-09-10T10:05:00.072 INFO:teuthology.orchestra.run.smithi102.stdout: "omap_total_objs": 1, 2024-09-10T10:05:00.072 INFO:teuthology.orchestra.run.smithi102.stdout: "omap_total_kv_pairs": 0, 2024-09-10T10:05:00.072 INFO:teuthology.orchestra.run.smithi102.stdout: "omap_total_updates": 0, 2024-09-10T10:05:00.072 INFO:teuthology.orchestra.run.smithi102.stdout: "omap_total_removes": 0 2024-09-10T10:05:00.072 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-10T10:05:00.072 INFO:teuthology.orchestra.run.smithi102.stdout: "osd-slow-ops": { 2024-09-10T10:05:00.072 INFO:teuthology.orchestra.run.smithi102.stdout: "slow_ops_count": 1858 2024-09-10T10:05:00.072 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-10T10:05:00.072 INFO:teuthology.orchestra.run.smithi102.stdout: "purge_queue": { 2024-09-10T10:05:00.073 INFO:teuthology.orchestra.run.smithi102.stdout: "pq_executing_ops": 0, 2024-09-10T10:05:00.073 INFO:teuthology.orchestra.run.smithi102.stdout: "pq_executing_ops_high_water": 10, 2024-09-10T10:05:00.073 INFO:teuthology.orchestra.run.smithi102.stdout: "pq_executing": 0, 2024-09-10T10:05:00.073 INFO:teuthology.orchestra.run.smithi102.stdout: "pq_executing_high_water": 7, 2024-09-10T10:05:00.073 INFO:teuthology.orchestra.run.smithi102.stdout: "pq_executed_ops": 27279, 2024-09-10T10:05:00.073 INFO:teuthology.orchestra.run.smithi102.stdout: "pq_executed": 13645, 2024-09-10T10:05:00.073 INFO:teuthology.orchestra.run.smithi102.stdout: "pq_item_in_journal": 0 2024-09-10T10:05:00.073 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-10T10:05:00.073 INFO:teuthology.orchestra.run.smithi102.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-10T10:05:00.073 INFO:teuthology.orchestra.run.smithi102.stdout: "val": 0, 2024-09-10T10:05:00.073 INFO:teuthology.orchestra.run.smithi102.stdout: "max": 104857600, 2024-09-10T10:05:00.073 INFO:teuthology.orchestra.run.smithi102.stdout: "get_started": 0, 2024-09-10T10:05:00.073 INFO:teuthology.orchestra.run.smithi102.stdout: "get": 445968, 2024-09-10T10:05:00.073 INFO:teuthology.orchestra.run.smithi102.stdout: "get_sum": 90275961, 2024-09-10T10:05:00.074 INFO:teuthology.orchestra.run.smithi102.stdout: "get_or_fail_fail": 0, 2024-09-10T10:05:00.074 INFO:teuthology.orchestra.run.smithi102.stdout: "get_or_fail_success": 445968, 2024-09-10T10:05:00.074 INFO:teuthology.orchestra.run.smithi102.stdout: "take": 0, 2024-09-10T10:05:00.074 INFO:teuthology.orchestra.run.smithi102.stdout: "take_sum": 0, 2024-09-10T10:05:00.074 INFO:teuthology.orchestra.run.smithi102.stdout: "put": 445968, 2024-09-10T10:05:00.074 INFO:teuthology.orchestra.run.smithi102.stdout: "put_sum": 90275961, 2024-09-10T10:05:00.074 INFO:teuthology.orchestra.run.smithi102.stdout: "wait": { 2024-09-10T10:05:00.074 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-10T10:05:00.074 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-10T10:05:00.074 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-10T10:05:00.074 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-10T10:05:00.074 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-10T10:05:00.074 INFO:teuthology.orchestra.run.smithi102.stdout: "throttle-objecter_bytes": { 2024-09-10T10:05:00.074 INFO:teuthology.orchestra.run.smithi102.stdout: "val": 0, 2024-09-10T10:05:00.074 INFO:teuthology.orchestra.run.smithi102.stdout: "max": 104857600, 2024-09-10T10:05:00.075 INFO:teuthology.orchestra.run.smithi102.stdout: "get_started": 0, 2024-09-10T10:05:00.075 INFO:teuthology.orchestra.run.smithi102.stdout: "get": 0, 2024-09-10T10:05:00.075 INFO:teuthology.orchestra.run.smithi102.stdout: "get_sum": 0, 2024-09-10T10:05:00.075 INFO:teuthology.orchestra.run.smithi102.stdout: "get_or_fail_fail": 0, 2024-09-10T10:05:00.075 INFO:teuthology.orchestra.run.smithi102.stdout: "get_or_fail_success": 0, 2024-09-10T10:05:00.075 INFO:teuthology.orchestra.run.smithi102.stdout: "take": 59438, 2024-09-10T10:05:00.075 INFO:teuthology.orchestra.run.smithi102.stdout: "take_sum": 148030543, 2024-09-10T10:05:00.075 INFO:teuthology.orchestra.run.smithi102.stdout: "put": 59438, 2024-09-10T10:05:00.075 INFO:teuthology.orchestra.run.smithi102.stdout: "put_sum": 148030543, 2024-09-10T10:05:00.075 INFO:teuthology.orchestra.run.smithi102.stdout: "wait": { 2024-09-10T10:05:00.075 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-10T10:05:00.075 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-10T10:05:00.075 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-10T10:05:00.076 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-10T10:05:00.076 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-10T10:05:00.076 INFO:teuthology.orchestra.run.smithi102.stdout: "throttle-objecter_ops": { 2024-09-10T10:05:00.076 INFO:teuthology.orchestra.run.smithi102.stdout: "val": 0, 2024-09-10T10:05:00.076 INFO:teuthology.orchestra.run.smithi102.stdout: "max": 1024, 2024-09-10T10:05:00.076 INFO:teuthology.orchestra.run.smithi102.stdout: "get_started": 0, 2024-09-10T10:05:00.076 INFO:teuthology.orchestra.run.smithi102.stdout: "get": 0, 2024-09-10T10:05:00.076 INFO:teuthology.orchestra.run.smithi102.stdout: "get_sum": 0, 2024-09-10T10:05:00.076 INFO:teuthology.orchestra.run.smithi102.stdout: "get_or_fail_fail": 0, 2024-09-10T10:05:00.076 INFO:teuthology.orchestra.run.smithi102.stdout: "get_or_fail_success": 0, 2024-09-10T10:05:00.076 INFO:teuthology.orchestra.run.smithi102.stdout: "take": 59438, 2024-09-10T10:05:00.076 INFO:teuthology.orchestra.run.smithi102.stdout: "take_sum": 59438, 2024-09-10T10:05:00.076 INFO:teuthology.orchestra.run.smithi102.stdout: "put": 59438, 2024-09-10T10:05:00.076 INFO:teuthology.orchestra.run.smithi102.stdout: "put_sum": 59438, 2024-09-10T10:05:00.076 INFO:teuthology.orchestra.run.smithi102.stdout: "wait": { 2024-09-10T10:05:00.077 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-10T10:05:00.077 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-10T10:05:00.077 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-10T10:05:00.077 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-10T10:05:00.077 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-10T10:05:00.077 INFO:teuthology.orchestra.run.smithi102.stdout: "throttle-write_buf_throttle": { 2024-09-10T10:05:00.077 INFO:teuthology.orchestra.run.smithi102.stdout: "val": 0, 2024-09-10T10:05:00.077 INFO:teuthology.orchestra.run.smithi102.stdout: "max": 3758096384, 2024-09-10T10:05:00.077 INFO:teuthology.orchestra.run.smithi102.stdout: "get_started": 0, 2024-09-10T10:05:00.077 INFO:teuthology.orchestra.run.smithi102.stdout: "get": 13645, 2024-09-10T10:05:00.077 INFO:teuthology.orchestra.run.smithi102.stdout: "get_sum": 1487217, 2024-09-10T10:05:00.077 INFO:teuthology.orchestra.run.smithi102.stdout: "get_or_fail_fail": 0, 2024-09-10T10:05:00.077 INFO:teuthology.orchestra.run.smithi102.stdout: "get_or_fail_success": 13645, 2024-09-10T10:05:00.077 INFO:teuthology.orchestra.run.smithi102.stdout: "take": 0, 2024-09-10T10:05:00.077 INFO:teuthology.orchestra.run.smithi102.stdout: "take_sum": 0, 2024-09-10T10:05:00.078 INFO:teuthology.orchestra.run.smithi102.stdout: "put": 670, 2024-09-10T10:05:00.078 INFO:teuthology.orchestra.run.smithi102.stdout: "put_sum": 1487217, 2024-09-10T10:05:00.078 INFO:teuthology.orchestra.run.smithi102.stdout: "wait": { 2024-09-10T10:05:00.078 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-10T10:05:00.078 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-10T10:05:00.078 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-10T10:05:00.078 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-10T10:05:00.078 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-10T10:05:00.078 INFO:teuthology.orchestra.run.smithi102.stdout: "throttle-write_buf_throttle-0x55e8167e2960": { 2024-09-10T10:05:00.078 INFO:teuthology.orchestra.run.smithi102.stdout: "val": 0, 2024-09-10T10:05:00.078 INFO:teuthology.orchestra.run.smithi102.stdout: "max": 3758096384, 2024-09-10T10:05:00.078 INFO:teuthology.orchestra.run.smithi102.stdout: "get_started": 0, 2024-09-10T10:05:00.078 INFO:teuthology.orchestra.run.smithi102.stdout: "get": 81538, 2024-09-10T10:05:00.078 INFO:teuthology.orchestra.run.smithi102.stdout: "get_sum": 143461990, 2024-09-10T10:05:00.078 INFO:teuthology.orchestra.run.smithi102.stdout: "get_or_fail_fail": 0, 2024-09-10T10:05:00.079 INFO:teuthology.orchestra.run.smithi102.stdout: "get_or_fail_success": 81538, 2024-09-10T10:05:00.079 INFO:teuthology.orchestra.run.smithi102.stdout: "take": 0, 2024-09-10T10:05:00.079 INFO:teuthology.orchestra.run.smithi102.stdout: "take_sum": 0, 2024-09-10T10:05:00.079 INFO:teuthology.orchestra.run.smithi102.stdout: "put": 27019, 2024-09-10T10:05:00.079 INFO:teuthology.orchestra.run.smithi102.stdout: "put_sum": 143461990, 2024-09-10T10:05:00.079 INFO:teuthology.orchestra.run.smithi102.stdout: "wait": { 2024-09-10T10:05:00.079 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-10T10:05:00.079 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-10T10:05:00.079 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-10T10:05:00.079 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-10T10:05:00.079 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-10T10:05:00.079 INFO:teuthology.orchestra.run.smithi102.stdout:} 2024-09-10T10:05:00.080 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:04:59 smithi102 ceph-mon[35442]: pgmap v1192: 129 pgs: 129 active+clean; 551 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 KiB/s wr, 9 op/s 2024-09-10T10:05:00.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:04:59 smithi159 ceph-mon[35513]: pgmap v1192: 129 pgs: 129 active+clean; 551 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 KiB/s wr, 9 op/s 2024-09-10T10:05:00.306 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:04:59 smithi005 ceph-mon[30350]: pgmap v1192: 129 pgs: 129 active+clean; 551 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 KiB/s wr, 9 op/s 2024-09-10T10:05:00.851 INFO:tasks.check_counter:Daemon mds.b mds.exported=11 2024-09-10T10:05:00.851 INFO:tasks.check_counter:Daemon mds.b mds.imported=3 2024-09-10T10:05:00.851 DEBUG:tasks.check_counter:Getting stats from e 2024-09-10T10:05:00.852 DEBUG:teuthology.orchestra.run.smithi102:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph daemon mds.e perf dump 2024-09-10T10:05:01.098 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T10:05:01.647 INFO:teuthology.orchestra.run.smithi102.stdout:{ 2024-09-10T10:05:01.648 INFO:teuthology.orchestra.run.smithi102.stdout: "AsyncMessenger::Worker-0": { 2024-09-10T10:05:01.648 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_recv_messages": 4232, 2024-09-10T10:05:01.648 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_messages": 4233, 2024-09-10T10:05:01.648 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_recv_bytes": 34771423, 2024-09-10T10:05:01.648 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_bytes": 1345465, 2024-09-10T10:05:01.648 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_created_connections": 8, 2024-09-10T10:05:01.648 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_active_connections": 4, 2024-09-10T10:05:01.648 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_total_time": 0.310950414, 2024-09-10T10:05:01.648 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_send_time": 0.097187537, 2024-09-10T10:05:01.648 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_recv_time": 0.092940201, 2024-09-10T10:05:01.649 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_fast_dispatch_time": 0.096974122, 2024-09-10T10:05:01.649 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T10:05:01.649 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 4231, 2024-09-10T10:05:01.649 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.115907951, 2024-09-10T10:05:01.649 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000027394 2024-09-10T10:05:01.649 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-10T10:05:01.649 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_handle_ack_lat": { 2024-09-10T10:05:01.649 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-10T10:05:01.649 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-10T10:05:01.649 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-10T10:05:01.649 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-10T10:05:01.649 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-10T10:05:01.649 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_encrypted_bytes": 0 2024-09-10T10:05:01.650 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-10T10:05:01.650 INFO:teuthology.orchestra.run.smithi102.stdout: "AsyncMessenger::Worker-1": { 2024-09-10T10:05:01.650 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_recv_messages": 458, 2024-09-10T10:05:01.650 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_messages": 822, 2024-09-10T10:05:01.650 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_recv_bytes": 29482240, 2024-09-10T10:05:01.650 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_bytes": 1032608, 2024-09-10T10:05:01.650 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_created_connections": 8, 2024-09-10T10:05:01.650 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_active_connections": 5, 2024-09-10T10:05:01.650 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_total_time": 0.065792908, 2024-09-10T10:05:01.650 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_send_time": 0.023861112, 2024-09-10T10:05:01.650 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_recv_time": 0.024984253, 2024-09-10T10:05:01.650 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_fast_dispatch_time": 0.011218292, 2024-09-10T10:05:01.651 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T10:05:01.651 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 820, 2024-09-10T10:05:01.651 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.032268859, 2024-09-10T10:05:01.651 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000039352 2024-09-10T10:05:01.651 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-10T10:05:01.651 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_handle_ack_lat": { 2024-09-10T10:05:01.651 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-10T10:05:01.651 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-10T10:05:01.651 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-10T10:05:01.651 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-10T10:05:01.651 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_recv_encrypted_bytes": 480, 2024-09-10T10:05:01.651 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_encrypted_bytes": 887856 2024-09-10T10:05:01.651 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-10T10:05:01.652 INFO:teuthology.orchestra.run.smithi102.stdout: "AsyncMessenger::Worker-2": { 2024-09-10T10:05:01.652 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_recv_messages": 1336, 2024-09-10T10:05:01.652 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_messages": 1307, 2024-09-10T10:05:01.652 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_recv_bytes": 38264152, 2024-09-10T10:05:01.652 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_bytes": 558522, 2024-09-10T10:05:01.652 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_created_connections": 6, 2024-09-10T10:05:01.652 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_active_connections": 4, 2024-09-10T10:05:01.652 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_total_time": 0.139813659, 2024-09-10T10:05:01.652 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_send_time": 0.044698691, 2024-09-10T10:05:01.652 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_recv_time": 0.079793503, 2024-09-10T10:05:01.652 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_fast_dispatch_time": 0.019539058, 2024-09-10T10:05:01.652 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T10:05:01.652 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 1307, 2024-09-10T10:05:01.652 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.061911213, 2024-09-10T10:05:01.653 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000047368 2024-09-10T10:05:01.653 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-10T10:05:01.653 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_handle_ack_lat": { 2024-09-10T10:05:01.653 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-10T10:05:01.653 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-10T10:05:01.653 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-10T10:05:01.653 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-10T10:05:01.653 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_recv_encrypted_bytes": 996608, 2024-09-10T10:05:01.653 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_encrypted_bytes": 298080 2024-09-10T10:05:01.653 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-10T10:05:01.653 INFO:teuthology.orchestra.run.smithi102.stdout: "cct": { 2024-09-10T10:05:01.653 INFO:teuthology.orchestra.run.smithi102.stdout: "total_workers": 1, 2024-09-10T10:05:01.653 INFO:teuthology.orchestra.run.smithi102.stdout: "unhealthy_workers": 0 2024-09-10T10:05:01.654 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-10T10:05:01.654 INFO:teuthology.orchestra.run.smithi102.stdout: "finisher-MDSRank": { 2024-09-10T10:05:01.654 INFO:teuthology.orchestra.run.smithi102.stdout: "queue_len": 0, 2024-09-10T10:05:01.654 INFO:teuthology.orchestra.run.smithi102.stdout: "complete_latency": { 2024-09-10T10:05:01.654 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 9316, 2024-09-10T10:05:01.654 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.192322817, 2024-09-10T10:05:01.654 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000020644 2024-09-10T10:05:01.654 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-10T10:05:01.654 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-10T10:05:01.654 INFO:teuthology.orchestra.run.smithi102.stdout: "finisher-PurgeQueue": { 2024-09-10T10:05:01.654 INFO:teuthology.orchestra.run.smithi102.stdout: "queue_len": 0, 2024-09-10T10:05:01.654 INFO:teuthology.orchestra.run.smithi102.stdout: "complete_latency": { 2024-09-10T10:05:01.654 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-10T10:05:01.654 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-10T10:05:01.655 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-10T10:05:01.655 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-10T10:05:01.655 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-10T10:05:01.655 INFO:teuthology.orchestra.run.smithi102.stdout: "mds": { 2024-09-10T10:05:01.655 INFO:teuthology.orchestra.run.smithi102.stdout: "request": 0, 2024-09-10T10:05:01.655 INFO:teuthology.orchestra.run.smithi102.stdout: "reply": 0, 2024-09-10T10:05:01.655 INFO:teuthology.orchestra.run.smithi102.stdout: "reply_latency": { 2024-09-10T10:05:01.655 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-10T10:05:01.655 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-10T10:05:01.655 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-10T10:05:01.655 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-10T10:05:01.655 INFO:teuthology.orchestra.run.smithi102.stdout: "slow_reply": 0, 2024-09-10T10:05:01.655 INFO:teuthology.orchestra.run.smithi102.stdout: "forward": 0, 2024-09-10T10:05:01.655 INFO:teuthology.orchestra.run.smithi102.stdout: "dir_fetch_complete": 0, 2024-09-10T10:05:01.655 INFO:teuthology.orchestra.run.smithi102.stdout: "dir_fetch_keys": 0, 2024-09-10T10:05:01.656 INFO:teuthology.orchestra.run.smithi102.stdout: "dir_commit": 0, 2024-09-10T10:05:01.656 INFO:teuthology.orchestra.run.smithi102.stdout: "dir_split": 0, 2024-09-10T10:05:01.656 INFO:teuthology.orchestra.run.smithi102.stdout: "dir_merge": 0, 2024-09-10T10:05:01.656 INFO:teuthology.orchestra.run.smithi102.stdout: "inodes": 9856, 2024-09-10T10:05:01.656 INFO:teuthology.orchestra.run.smithi102.stdout: "inodes_top": 0, 2024-09-10T10:05:01.656 INFO:teuthology.orchestra.run.smithi102.stdout: "inodes_bottom": 9856, 2024-09-10T10:05:01.656 INFO:teuthology.orchestra.run.smithi102.stdout: "inodes_pin_tail": 0, 2024-09-10T10:05:01.656 INFO:teuthology.orchestra.run.smithi102.stdout: "inodes_pinned": 9856, 2024-09-10T10:05:01.656 INFO:teuthology.orchestra.run.smithi102.stdout: "inodes_expired": 0, 2024-09-10T10:05:01.656 INFO:teuthology.orchestra.run.smithi102.stdout: "inodes_with_caps": 0, 2024-09-10T10:05:01.656 INFO:teuthology.orchestra.run.smithi102.stdout: "caps": 0, 2024-09-10T10:05:01.656 INFO:teuthology.orchestra.run.smithi102.stdout: "subtrees": 1, 2024-09-10T10:05:01.656 INFO:teuthology.orchestra.run.smithi102.stdout: "traverse": 0, 2024-09-10T10:05:01.657 INFO:teuthology.orchestra.run.smithi102.stdout: "traverse_hit": 0, 2024-09-10T10:05:01.657 INFO:teuthology.orchestra.run.smithi102.stdout: "traverse_forward": 0, 2024-09-10T10:05:01.657 INFO:teuthology.orchestra.run.smithi102.stdout: "traverse_discover": 0, 2024-09-10T10:05:01.657 INFO:teuthology.orchestra.run.smithi102.stdout: "traverse_dir_fetch": 0, 2024-09-10T10:05:01.657 INFO:teuthology.orchestra.run.smithi102.stdout: "traverse_remote_ino": 0, 2024-09-10T10:05:01.657 INFO:teuthology.orchestra.run.smithi102.stdout: "traverse_lock": 0, 2024-09-10T10:05:01.657 INFO:teuthology.orchestra.run.smithi102.stdout: "load_cent": 0, 2024-09-10T10:05:01.657 INFO:teuthology.orchestra.run.smithi102.stdout: "q": 0, 2024-09-10T10:05:01.657 INFO:teuthology.orchestra.run.smithi102.stdout: "exported": 0, 2024-09-10T10:05:01.657 INFO:teuthology.orchestra.run.smithi102.stdout: "exported_inodes": 0, 2024-09-10T10:05:01.657 INFO:teuthology.orchestra.run.smithi102.stdout: "imported": 0, 2024-09-10T10:05:01.657 INFO:teuthology.orchestra.run.smithi102.stdout: "imported_inodes": 0, 2024-09-10T10:05:01.657 INFO:teuthology.orchestra.run.smithi102.stdout: "openino_dir_fetch": 0, 2024-09-10T10:05:01.657 INFO:teuthology.orchestra.run.smithi102.stdout: "openino_backtrace_fetch": 0, 2024-09-10T10:05:01.657 INFO:teuthology.orchestra.run.smithi102.stdout: "openino_peer_discover": 0, 2024-09-10T10:05:01.658 INFO:teuthology.orchestra.run.smithi102.stdout: "root_rfiles": 0, 2024-09-10T10:05:01.658 INFO:teuthology.orchestra.run.smithi102.stdout: "root_rbytes": 0, 2024-09-10T10:05:01.658 INFO:teuthology.orchestra.run.smithi102.stdout: "root_rsnaps": 0, 2024-09-10T10:05:01.658 INFO:teuthology.orchestra.run.smithi102.stdout: "scrub_backtrace_fetch": 0, 2024-09-10T10:05:01.658 INFO:teuthology.orchestra.run.smithi102.stdout: "scrub_set_tag": 0, 2024-09-10T10:05:01.658 INFO:teuthology.orchestra.run.smithi102.stdout: "scrub_backtrace_repaired": 0, 2024-09-10T10:05:01.658 INFO:teuthology.orchestra.run.smithi102.stdout: "scrub_inotable_repaired": 0, 2024-09-10T10:05:01.658 INFO:teuthology.orchestra.run.smithi102.stdout: "scrub_dir_inodes": 0, 2024-09-10T10:05:01.658 INFO:teuthology.orchestra.run.smithi102.stdout: "scrub_dir_base_inodes": 0, 2024-09-10T10:05:01.658 INFO:teuthology.orchestra.run.smithi102.stdout: "scrub_dirfrag_rstats": 0, 2024-09-10T10:05:01.658 INFO:teuthology.orchestra.run.smithi102.stdout: "scrub_file_inodes": 0, 2024-09-10T10:05:01.658 INFO:teuthology.orchestra.run.smithi102.stdout: "handle_inode_file_caps": 0, 2024-09-10T10:05:01.658 INFO:teuthology.orchestra.run.smithi102.stdout: "ceph_cap_op_revoke": 0, 2024-09-10T10:05:01.658 INFO:teuthology.orchestra.run.smithi102.stdout: "ceph_cap_op_grant": 0, 2024-09-10T10:05:01.659 INFO:teuthology.orchestra.run.smithi102.stdout: "ceph_cap_op_trunc": 0, 2024-09-10T10:05:01.659 INFO:teuthology.orchestra.run.smithi102.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-10T10:05:01.659 INFO:teuthology.orchestra.run.smithi102.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-10T10:05:01.659 INFO:teuthology.orchestra.run.smithi102.stdout: "handle_client_caps": 0, 2024-09-10T10:05:01.659 INFO:teuthology.orchestra.run.smithi102.stdout: "handle_client_caps_dirty": 0, 2024-09-10T10:05:01.659 INFO:teuthology.orchestra.run.smithi102.stdout: "handle_client_cap_release": 0, 2024-09-10T10:05:01.659 INFO:teuthology.orchestra.run.smithi102.stdout: "process_request_cap_release": 0 2024-09-10T10:05:01.659 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-10T10:05:01.659 INFO:teuthology.orchestra.run.smithi102.stdout: "mds_cache": { 2024-09-10T10:05:01.659 INFO:teuthology.orchestra.run.smithi102.stdout: "dir_update": 0, 2024-09-10T10:05:01.659 INFO:teuthology.orchestra.run.smithi102.stdout: "dir_update_receipt": 0, 2024-09-10T10:05:01.659 INFO:teuthology.orchestra.run.smithi102.stdout: "dir_try_discover": 0, 2024-09-10T10:05:01.659 INFO:teuthology.orchestra.run.smithi102.stdout: "dir_send_discover": 0, 2024-09-10T10:05:01.660 INFO:teuthology.orchestra.run.smithi102.stdout: "dir_handle_discover": 0, 2024-09-10T10:05:01.660 INFO:teuthology.orchestra.run.smithi102.stdout: "num_strays": 0, 2024-09-10T10:05:01.660 INFO:teuthology.orchestra.run.smithi102.stdout: "num_strays_delayed": 0, 2024-09-10T10:05:01.660 INFO:teuthology.orchestra.run.smithi102.stdout: "num_strays_enqueuing": 0, 2024-09-10T10:05:01.660 INFO:teuthology.orchestra.run.smithi102.stdout: "strays_created": 0, 2024-09-10T10:05:01.660 INFO:teuthology.orchestra.run.smithi102.stdout: "strays_enqueued": 0, 2024-09-10T10:05:01.660 INFO:teuthology.orchestra.run.smithi102.stdout: "strays_reintegrated": 0, 2024-09-10T10:05:01.660 INFO:teuthology.orchestra.run.smithi102.stdout: "strays_migrated": 0, 2024-09-10T10:05:01.660 INFO:teuthology.orchestra.run.smithi102.stdout: "num_recovering_processing": 0, 2024-09-10T10:05:01.660 INFO:teuthology.orchestra.run.smithi102.stdout: "num_recovering_enqueued": 0, 2024-09-10T10:05:01.660 INFO:teuthology.orchestra.run.smithi102.stdout: "num_recovering_prioritized": 0, 2024-09-10T10:05:01.660 INFO:teuthology.orchestra.run.smithi102.stdout: "recovery_started": 0, 2024-09-10T10:05:01.660 INFO:teuthology.orchestra.run.smithi102.stdout: "recovery_completed": 0, 2024-09-10T10:05:01.661 INFO:teuthology.orchestra.run.smithi102.stdout: "ireq_quiesce_path": 0, 2024-09-10T10:05:01.661 INFO:teuthology.orchestra.run.smithi102.stdout: "ireq_quiesce_inode": 0, 2024-09-10T10:05:01.661 INFO:teuthology.orchestra.run.smithi102.stdout: "ireq_enqueue_scrub": 0, 2024-09-10T10:05:01.661 INFO:teuthology.orchestra.run.smithi102.stdout: "ireq_exportdir": 0, 2024-09-10T10:05:01.661 INFO:teuthology.orchestra.run.smithi102.stdout: "ireq_flush": 0, 2024-09-10T10:05:01.661 INFO:teuthology.orchestra.run.smithi102.stdout: "ireq_fragmentdir": 0, 2024-09-10T10:05:01.661 INFO:teuthology.orchestra.run.smithi102.stdout: "ireq_fragstats": 0, 2024-09-10T10:05:01.661 INFO:teuthology.orchestra.run.smithi102.stdout: "ireq_inodestats": 0 2024-09-10T10:05:01.661 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-10T10:05:01.661 INFO:teuthology.orchestra.run.smithi102.stdout: "mds_log": { 2024-09-10T10:05:01.661 INFO:teuthology.orchestra.run.smithi102.stdout: "evlrg": 0, 2024-09-10T10:05:01.661 INFO:teuthology.orchestra.run.smithi102.stdout: "evadd": 0, 2024-09-10T10:05:01.662 INFO:teuthology.orchestra.run.smithi102.stdout: "evex": 0, 2024-09-10T10:05:01.662 INFO:teuthology.orchestra.run.smithi102.stdout: "evtrm": 0, 2024-09-10T10:05:01.662 INFO:teuthology.orchestra.run.smithi102.stdout: "ev": 50281, 2024-09-10T10:05:01.662 INFO:teuthology.orchestra.run.smithi102.stdout: "evexg": 0, 2024-09-10T10:05:01.662 INFO:teuthology.orchestra.run.smithi102.stdout: "evexd": 0, 2024-09-10T10:05:01.662 INFO:teuthology.orchestra.run.smithi102.stdout: "segadd": 0, 2024-09-10T10:05:01.662 INFO:teuthology.orchestra.run.smithi102.stdout: "segex": 0, 2024-09-10T10:05:01.662 INFO:teuthology.orchestra.run.smithi102.stdout: "segtrm": 0, 2024-09-10T10:05:01.662 INFO:teuthology.orchestra.run.smithi102.stdout: "seg": 63, 2024-09-10T10:05:01.662 INFO:teuthology.orchestra.run.smithi102.stdout: "segmjr": 6, 2024-09-10T10:05:01.662 INFO:teuthology.orchestra.run.smithi102.stdout: "segexg": 0, 2024-09-10T10:05:01.662 INFO:teuthology.orchestra.run.smithi102.stdout: "segexd": 0, 2024-09-10T10:05:01.662 INFO:teuthology.orchestra.run.smithi102.stdout: "expos": 4194304, 2024-09-10T10:05:01.662 INFO:teuthology.orchestra.run.smithi102.stdout: "wrpos": 104168352, 2024-09-10T10:05:01.662 INFO:teuthology.orchestra.run.smithi102.stdout: "rdpos": 104167214, 2024-09-10T10:05:01.663 INFO:teuthology.orchestra.run.smithi102.stdout: "jlat": { 2024-09-10T10:05:01.663 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-10T10:05:01.663 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-10T10:05:01.663 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-10T10:05:01.663 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-10T10:05:01.663 INFO:teuthology.orchestra.run.smithi102.stdout: "replayed": 50281 2024-09-10T10:05:01.663 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-10T10:05:01.663 INFO:teuthology.orchestra.run.smithi102.stdout: "mds_mem": { 2024-09-10T10:05:01.663 INFO:teuthology.orchestra.run.smithi102.stdout: "ino": 13, 2024-09-10T10:05:01.663 INFO:teuthology.orchestra.run.smithi102.stdout: "ino+": 9873, 2024-09-10T10:05:01.663 INFO:teuthology.orchestra.run.smithi102.stdout: "ino-": 9860, 2024-09-10T10:05:01.663 INFO:teuthology.orchestra.run.smithi102.stdout: "dir": 11, 2024-09-10T10:05:01.663 INFO:teuthology.orchestra.run.smithi102.stdout: "dir+": 36, 2024-09-10T10:05:01.663 INFO:teuthology.orchestra.run.smithi102.stdout: "dir-": 25, 2024-09-10T10:05:01.664 INFO:teuthology.orchestra.run.smithi102.stdout: "dn": 9856, 2024-09-10T10:05:01.664 INFO:teuthology.orchestra.run.smithi102.stdout: "dn+": 10033, 2024-09-10T10:05:01.664 INFO:teuthology.orchestra.run.smithi102.stdout: "dn-": 177, 2024-09-10T10:05:01.664 INFO:teuthology.orchestra.run.smithi102.stdout: "cap": 0, 2024-09-10T10:05:01.664 INFO:teuthology.orchestra.run.smithi102.stdout: "cap+": 0, 2024-09-10T10:05:01.664 INFO:teuthology.orchestra.run.smithi102.stdout: "cap-": 0, 2024-09-10T10:05:01.664 INFO:teuthology.orchestra.run.smithi102.stdout: "rss": 62384, 2024-09-10T10:05:01.664 INFO:teuthology.orchestra.run.smithi102.stdout: "heap": 215328 2024-09-10T10:05:01.664 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-10T10:05:01.664 INFO:teuthology.orchestra.run.smithi102.stdout: "mds_server": { 2024-09-10T10:05:01.664 INFO:teuthology.orchestra.run.smithi102.stdout: "dispatch_client_request": 0, 2024-09-10T10:05:01.664 INFO:teuthology.orchestra.run.smithi102.stdout: "dispatch_server_request": 0, 2024-09-10T10:05:01.665 INFO:teuthology.orchestra.run.smithi102.stdout: "handle_client_request": 0, 2024-09-10T10:05:01.665 INFO:teuthology.orchestra.run.smithi102.stdout: "handle_client_session": 0, 2024-09-10T10:05:01.665 INFO:teuthology.orchestra.run.smithi102.stdout: "handle_peer_request": 0, 2024-09-10T10:05:01.665 INFO:teuthology.orchestra.run.smithi102.stdout: "req_create_latency": { 2024-09-10T10:05:01.665 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-10T10:05:01.665 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-10T10:05:01.665 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-10T10:05:01.665 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-10T10:05:01.665 INFO:teuthology.orchestra.run.smithi102.stdout: "req_getattr_latency": { 2024-09-10T10:05:01.665 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-10T10:05:01.665 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-10T10:05:01.665 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-10T10:05:01.666 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-10T10:05:01.666 INFO:teuthology.orchestra.run.smithi102.stdout: "req_getfilelock_latency": { 2024-09-10T10:05:01.666 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-10T10:05:01.666 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-10T10:05:01.666 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-10T10:05:01.666 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-10T10:05:01.666 INFO:teuthology.orchestra.run.smithi102.stdout: "req_link_latency": { 2024-09-10T10:05:01.666 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-10T10:05:01.666 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-10T10:05:01.666 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-10T10:05:01.666 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-10T10:05:01.666 INFO:teuthology.orchestra.run.smithi102.stdout: "req_lookup_latency": { 2024-09-10T10:05:01.666 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-10T10:05:01.666 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-10T10:05:01.667 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-10T10:05:01.667 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-10T10:05:01.667 INFO:teuthology.orchestra.run.smithi102.stdout: "req_lookuphash_latency": { 2024-09-10T10:05:01.667 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-10T10:05:01.667 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-10T10:05:01.667 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-10T10:05:01.667 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-10T10:05:01.667 INFO:teuthology.orchestra.run.smithi102.stdout: "req_lookupino_latency": { 2024-09-10T10:05:01.667 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-10T10:05:01.667 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-10T10:05:01.667 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-10T10:05:01.667 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-10T10:05:01.667 INFO:teuthology.orchestra.run.smithi102.stdout: "req_lookupname_latency": { 2024-09-10T10:05:01.667 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-10T10:05:01.668 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-10T10:05:01.668 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-10T10:05:01.668 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-10T10:05:01.668 INFO:teuthology.orchestra.run.smithi102.stdout: "req_lookupparent_latency": { 2024-09-10T10:05:01.668 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-10T10:05:01.668 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-10T10:05:01.668 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-10T10:05:01.668 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-10T10:05:01.668 INFO:teuthology.orchestra.run.smithi102.stdout: "req_lookupsnap_latency": { 2024-09-10T10:05:01.668 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-10T10:05:01.668 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-10T10:05:01.668 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-10T10:05:01.668 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-10T10:05:01.669 INFO:teuthology.orchestra.run.smithi102.stdout: "req_lssnap_latency": { 2024-09-10T10:05:01.669 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-10T10:05:01.669 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-10T10:05:01.669 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-10T10:05:01.669 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-10T10:05:01.669 INFO:teuthology.orchestra.run.smithi102.stdout: "req_mkdir_latency": { 2024-09-10T10:05:01.669 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-10T10:05:01.669 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-10T10:05:01.670 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-10T10:05:01.670 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-10T10:05:01.670 INFO:teuthology.orchestra.run.smithi102.stdout: "req_mknod_latency": { 2024-09-10T10:05:01.670 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-10T10:05:01.670 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-10T10:05:01.670 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-10T10:05:01.670 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-10T10:05:01.670 INFO:teuthology.orchestra.run.smithi102.stdout: "req_mksnap_latency": { 2024-09-10T10:05:01.670 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-10T10:05:01.670 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-10T10:05:01.670 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-10T10:05:01.670 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-10T10:05:01.670 INFO:teuthology.orchestra.run.smithi102.stdout: "req_open_latency": { 2024-09-10T10:05:01.671 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-10T10:05:01.671 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-10T10:05:01.671 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-10T10:05:01.671 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-10T10:05:01.671 INFO:teuthology.orchestra.run.smithi102.stdout: "req_readdir_latency": { 2024-09-10T10:05:01.671 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-10T10:05:01.671 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-10T10:05:01.671 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-10T10:05:01.671 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-10T10:05:01.671 INFO:teuthology.orchestra.run.smithi102.stdout: "req_rename_latency": { 2024-09-10T10:05:01.671 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-10T10:05:01.671 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-10T10:05:01.671 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-10T10:05:01.672 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-10T10:05:01.672 INFO:teuthology.orchestra.run.smithi102.stdout: "req_renamesnap_latency": { 2024-09-10T10:05:01.672 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-10T10:05:01.672 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-10T10:05:01.672 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-10T10:05:01.672 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-10T10:05:01.672 INFO:teuthology.orchestra.run.smithi102.stdout: "req_snapdiff_latency": { 2024-09-10T10:05:01.672 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-10T10:05:01.672 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-10T10:05:01.672 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-10T10:05:01.672 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-10T10:05:01.672 INFO:teuthology.orchestra.run.smithi102.stdout: "req_rmdir_latency": { 2024-09-10T10:05:01.672 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-10T10:05:01.673 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-10T10:05:01.673 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-10T10:05:01.673 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-10T10:05:01.673 INFO:teuthology.orchestra.run.smithi102.stdout: "req_rmsnap_latency": { 2024-09-10T10:05:01.673 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-10T10:05:01.673 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-10T10:05:01.673 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-10T10:05:01.673 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-10T10:05:01.673 INFO:teuthology.orchestra.run.smithi102.stdout: "req_rmxattr_latency": { 2024-09-10T10:05:01.673 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-10T10:05:01.673 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-10T10:05:01.673 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-10T10:05:01.674 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-10T10:05:01.674 INFO:teuthology.orchestra.run.smithi102.stdout: "req_setattr_latency": { 2024-09-10T10:05:01.674 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-10T10:05:01.674 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-10T10:05:01.674 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-10T10:05:01.674 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-10T10:05:01.674 INFO:teuthology.orchestra.run.smithi102.stdout: "req_setdirlayout_latency": { 2024-09-10T10:05:01.674 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-10T10:05:01.674 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-10T10:05:01.674 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-10T10:05:01.674 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-10T10:05:01.675 INFO:teuthology.orchestra.run.smithi102.stdout: "req_setfilelock_latency": { 2024-09-10T10:05:01.675 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-10T10:05:01.675 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-10T10:05:01.675 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-10T10:05:01.675 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-10T10:05:01.675 INFO:teuthology.orchestra.run.smithi102.stdout: "req_setlayout_latency": { 2024-09-10T10:05:01.675 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-10T10:05:01.675 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-10T10:05:01.675 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-10T10:05:01.675 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-10T10:05:01.675 INFO:teuthology.orchestra.run.smithi102.stdout: "req_setxattr_latency": { 2024-09-10T10:05:01.675 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-10T10:05:01.675 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-10T10:05:01.676 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-10T10:05:01.676 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-10T10:05:01.676 INFO:teuthology.orchestra.run.smithi102.stdout: "req_symlink_latency": { 2024-09-10T10:05:01.676 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-10T10:05:01.676 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-10T10:05:01.676 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-10T10:05:01.676 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-10T10:05:01.676 INFO:teuthology.orchestra.run.smithi102.stdout: "req_unlink_latency": { 2024-09-10T10:05:01.676 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-10T10:05:01.676 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-10T10:05:01.676 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-10T10:05:01.677 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-10T10:05:01.677 INFO:teuthology.orchestra.run.smithi102.stdout: "cap_revoke_eviction": 0, 2024-09-10T10:05:01.677 INFO:teuthology.orchestra.run.smithi102.stdout: "cap_acquisition_throttle": 0, 2024-09-10T10:05:01.677 INFO:teuthology.orchestra.run.smithi102.stdout: "req_getvxattr_latency": { 2024-09-10T10:05:01.677 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-10T10:05:01.677 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-10T10:05:01.677 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-10T10:05:01.677 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-10T10:05:01.677 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-10T10:05:01.677 INFO:teuthology.orchestra.run.smithi102.stdout: "mds_sessions": { 2024-09-10T10:05:01.678 INFO:teuthology.orchestra.run.smithi102.stdout: "session_count": 1, 2024-09-10T10:05:01.678 INFO:teuthology.orchestra.run.smithi102.stdout: "session_add": 2, 2024-09-10T10:05:01.678 INFO:teuthology.orchestra.run.smithi102.stdout: "session_remove": 1, 2024-09-10T10:05:01.678 INFO:teuthology.orchestra.run.smithi102.stdout: "sessions_open": 1, 2024-09-10T10:05:01.678 INFO:teuthology.orchestra.run.smithi102.stdout: "sessions_stale": 0, 2024-09-10T10:05:01.678 INFO:teuthology.orchestra.run.smithi102.stdout: "total_load": 0, 2024-09-10T10:05:01.678 INFO:teuthology.orchestra.run.smithi102.stdout: "average_load": 0, 2024-09-10T10:05:01.678 INFO:teuthology.orchestra.run.smithi102.stdout: "avg_session_uptime": 3017, 2024-09-10T10:05:01.678 INFO:teuthology.orchestra.run.smithi102.stdout: "mdthresh_evicted": 0 2024-09-10T10:05:01.678 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-10T10:05:01.678 INFO:teuthology.orchestra.run.smithi102.stdout: "mempool": { 2024-09-10T10:05:01.678 INFO:teuthology.orchestra.run.smithi102.stdout: "bloom_filter_bytes": 0, 2024-09-10T10:05:01.678 INFO:teuthology.orchestra.run.smithi102.stdout: "bloom_filter_items": 0, 2024-09-10T10:05:01.678 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_alloc_bytes": 0, 2024-09-10T10:05:01.679 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_alloc_items": 0, 2024-09-10T10:05:01.679 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_data_bytes": 0, 2024-09-10T10:05:01.679 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_data_items": 0, 2024-09-10T10:05:01.679 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-10T10:05:01.679 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_onode_items": 0, 2024-09-10T10:05:01.679 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-10T10:05:01.679 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_meta_items": 0, 2024-09-10T10:05:01.679 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_other_bytes": 0, 2024-09-10T10:05:01.679 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_other_items": 0, 2024-09-10T10:05:01.679 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-10T10:05:01.679 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_buffer_items": 0, 2024-09-10T10:05:01.679 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_extent_bytes": 0, 2024-09-10T10:05:01.679 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_extent_items": 0, 2024-09-10T10:05:01.679 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_blob_bytes": 0, 2024-09-10T10:05:01.679 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_blob_items": 0, 2024-09-10T10:05:01.680 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-10T10:05:01.680 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_shared_blob_items": 0, 2024-09-10T10:05:01.680 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-10T10:05:01.680 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_inline_bl_items": 0, 2024-09-10T10:05:01.680 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_fsck_bytes": 0, 2024-09-10T10:05:01.680 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_fsck_items": 0, 2024-09-10T10:05:01.680 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_txc_bytes": 0, 2024-09-10T10:05:01.680 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_txc_items": 0, 2024-09-10T10:05:01.680 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-10T10:05:01.680 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_writing_deferred_items": 0, 2024-09-10T10:05:01.680 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_writing_bytes": 0, 2024-09-10T10:05:01.680 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_writing_items": 0, 2024-09-10T10:05:01.680 INFO:teuthology.orchestra.run.smithi102.stdout: "bluefs_bytes": 0, 2024-09-10T10:05:01.680 INFO:teuthology.orchestra.run.smithi102.stdout: "bluefs_items": 0, 2024-09-10T10:05:01.680 INFO:teuthology.orchestra.run.smithi102.stdout: "bluefs_file_reader_bytes": 0, 2024-09-10T10:05:01.681 INFO:teuthology.orchestra.run.smithi102.stdout: "bluefs_file_reader_items": 0, 2024-09-10T10:05:01.681 INFO:teuthology.orchestra.run.smithi102.stdout: "bluefs_file_writer_bytes": 0, 2024-09-10T10:05:01.681 INFO:teuthology.orchestra.run.smithi102.stdout: "bluefs_file_writer_items": 0, 2024-09-10T10:05:01.681 INFO:teuthology.orchestra.run.smithi102.stdout: "buffer_anon_bytes": 232921, 2024-09-10T10:05:01.681 INFO:teuthology.orchestra.run.smithi102.stdout: "buffer_anon_items": 78, 2024-09-10T10:05:01.681 INFO:teuthology.orchestra.run.smithi102.stdout: "buffer_meta_bytes": 0, 2024-09-10T10:05:01.681 INFO:teuthology.orchestra.run.smithi102.stdout: "buffer_meta_items": 0, 2024-09-10T10:05:01.681 INFO:teuthology.orchestra.run.smithi102.stdout: "osd_bytes": 0, 2024-09-10T10:05:01.681 INFO:teuthology.orchestra.run.smithi102.stdout: "osd_items": 0, 2024-09-10T10:05:01.681 INFO:teuthology.orchestra.run.smithi102.stdout: "osd_mapbl_bytes": 0, 2024-09-10T10:05:01.681 INFO:teuthology.orchestra.run.smithi102.stdout: "osd_mapbl_items": 0, 2024-09-10T10:05:01.681 INFO:teuthology.orchestra.run.smithi102.stdout: "osd_pglog_bytes": 0, 2024-09-10T10:05:01.681 INFO:teuthology.orchestra.run.smithi102.stdout: "osd_pglog_items": 0, 2024-09-10T10:05:01.682 INFO:teuthology.orchestra.run.smithi102.stdout: "osdmap_bytes": 9152, 2024-09-10T10:05:01.682 INFO:teuthology.orchestra.run.smithi102.stdout: "osdmap_items": 173, 2024-09-10T10:05:01.682 INFO:teuthology.orchestra.run.smithi102.stdout: "osdmap_mapping_bytes": 0, 2024-09-10T10:05:01.682 INFO:teuthology.orchestra.run.smithi102.stdout: "osdmap_mapping_items": 0, 2024-09-10T10:05:01.682 INFO:teuthology.orchestra.run.smithi102.stdout: "pgmap_bytes": 0, 2024-09-10T10:05:01.682 INFO:teuthology.orchestra.run.smithi102.stdout: "pgmap_items": 0, 2024-09-10T10:05:01.682 INFO:teuthology.orchestra.run.smithi102.stdout: "mds_co_bytes": 6982216, 2024-09-10T10:05:01.682 INFO:teuthology.orchestra.run.smithi102.stdout: "mds_co_items": 39552, 2024-09-10T10:05:01.682 INFO:teuthology.orchestra.run.smithi102.stdout: "unittest_1_bytes": 0, 2024-09-10T10:05:01.682 INFO:teuthology.orchestra.run.smithi102.stdout: "unittest_1_items": 0, 2024-09-10T10:05:01.682 INFO:teuthology.orchestra.run.smithi102.stdout: "unittest_2_bytes": 0, 2024-09-10T10:05:01.682 INFO:teuthology.orchestra.run.smithi102.stdout: "unittest_2_items": 0 2024-09-10T10:05:01.682 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-10T10:05:01.682 INFO:teuthology.orchestra.run.smithi102.stdout: "objecter": { 2024-09-10T10:05:01.682 INFO:teuthology.orchestra.run.smithi102.stdout: "op_active": 0, 2024-09-10T10:05:01.683 INFO:teuthology.orchestra.run.smithi102.stdout: "op_laggy": 0, 2024-09-10T10:05:01.683 INFO:teuthology.orchestra.run.smithi102.stdout: "op_send": 5505, 2024-09-10T10:05:01.683 INFO:teuthology.orchestra.run.smithi102.stdout: "op_send_bytes": 21, 2024-09-10T10:05:01.683 INFO:teuthology.orchestra.run.smithi102.stdout: "op_resend": 3, 2024-09-10T10:05:01.683 INFO:teuthology.orchestra.run.smithi102.stdout: "op_reply": 5505, 2024-09-10T10:05:01.683 INFO:teuthology.orchestra.run.smithi102.stdout: "op_latency": { 2024-09-10T10:05:01.683 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 5505, 2024-09-10T10:05:01.683 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 8.945806361, 2024-09-10T10:05:01.683 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.001625032 2024-09-10T10:05:01.683 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-10T10:05:01.683 INFO:teuthology.orchestra.run.smithi102.stdout: "op_inflight": 0, 2024-09-10T10:05:01.683 INFO:teuthology.orchestra.run.smithi102.stdout: "oplen_avg": { 2024-09-10T10:05:01.683 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 5505, 2024-09-10T10:05:01.684 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 5506 2024-09-10T10:05:01.684 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-10T10:05:01.684 INFO:teuthology.orchestra.run.smithi102.stdout: "op": 5505, 2024-09-10T10:05:01.684 INFO:teuthology.orchestra.run.smithi102.stdout: "op_r": 5505, 2024-09-10T10:05:01.684 INFO:teuthology.orchestra.run.smithi102.stdout: "op_w": 0, 2024-09-10T10:05:01.684 INFO:teuthology.orchestra.run.smithi102.stdout: "op_rmw": 0, 2024-09-10T10:05:01.684 INFO:teuthology.orchestra.run.smithi102.stdout: "op_pg": 0, 2024-09-10T10:05:01.684 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_stat": 3349, 2024-09-10T10:05:01.684 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_create": 0, 2024-09-10T10:05:01.684 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_read": 2154, 2024-09-10T10:05:01.684 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_write": 0, 2024-09-10T10:05:01.684 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_writefull": 0, 2024-09-10T10:05:01.684 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_writesame": 0, 2024-09-10T10:05:01.685 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_append": 0, 2024-09-10T10:05:01.685 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_zero": 0, 2024-09-10T10:05:01.685 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_truncate": 0, 2024-09-10T10:05:01.685 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_delete": 0, 2024-09-10T10:05:01.685 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_mapext": 0, 2024-09-10T10:05:01.685 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_sparse_read": 0, 2024-09-10T10:05:01.685 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_clonerange": 0, 2024-09-10T10:05:01.685 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_getxattr": 1, 2024-09-10T10:05:01.685 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_setxattr": 0, 2024-09-10T10:05:01.685 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_cmpxattr": 0, 2024-09-10T10:05:01.685 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_rmxattr": 0, 2024-09-10T10:05:01.685 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_resetxattrs": 0, 2024-09-10T10:05:01.685 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_call": 0, 2024-09-10T10:05:01.686 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_watch": 0, 2024-09-10T10:05:01.686 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_notify": 0, 2024-09-10T10:05:01.686 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_src_cmpxattr": 0, 2024-09-10T10:05:01.686 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_pgls": 0, 2024-09-10T10:05:01.686 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_pgls_filter": 0, 2024-09-10T10:05:01.686 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_other": 0, 2024-09-10T10:05:01.686 INFO:teuthology.orchestra.run.smithi102.stdout: "linger_active": 0, 2024-09-10T10:05:01.686 INFO:teuthology.orchestra.run.smithi102.stdout: "linger_send": 0, 2024-09-10T10:05:01.686 INFO:teuthology.orchestra.run.smithi102.stdout: "linger_resend": 0, 2024-09-10T10:05:01.686 INFO:teuthology.orchestra.run.smithi102.stdout: "linger_ping": 0, 2024-09-10T10:05:01.686 INFO:teuthology.orchestra.run.smithi102.stdout: "poolop_active": 0, 2024-09-10T10:05:01.686 INFO:teuthology.orchestra.run.smithi102.stdout: "poolop_send": 0, 2024-09-10T10:05:01.686 INFO:teuthology.orchestra.run.smithi102.stdout: "poolop_resend": 0, 2024-09-10T10:05:01.686 INFO:teuthology.orchestra.run.smithi102.stdout: "poolstat_active": 0, 2024-09-10T10:05:01.687 INFO:teuthology.orchestra.run.smithi102.stdout: "poolstat_send": 0, 2024-09-10T10:05:01.687 INFO:teuthology.orchestra.run.smithi102.stdout: "poolstat_resend": 0, 2024-09-10T10:05:01.687 INFO:teuthology.orchestra.run.smithi102.stdout: "statfs_active": 0, 2024-09-10T10:05:01.687 INFO:teuthology.orchestra.run.smithi102.stdout: "statfs_send": 0, 2024-09-10T10:05:01.687 INFO:teuthology.orchestra.run.smithi102.stdout: "statfs_resend": 0, 2024-09-10T10:05:01.687 INFO:teuthology.orchestra.run.smithi102.stdout: "command_active": 0, 2024-09-10T10:05:01.687 INFO:teuthology.orchestra.run.smithi102.stdout: "command_send": 0, 2024-09-10T10:05:01.687 INFO:teuthology.orchestra.run.smithi102.stdout: "command_resend": 0, 2024-09-10T10:05:01.687 INFO:teuthology.orchestra.run.smithi102.stdout: "map_epoch": 96, 2024-09-10T10:05:01.687 INFO:teuthology.orchestra.run.smithi102.stdout: "map_full": 0, 2024-09-10T10:05:01.687 INFO:teuthology.orchestra.run.smithi102.stdout: "map_inc": 2, 2024-09-10T10:05:01.687 INFO:teuthology.orchestra.run.smithi102.stdout: "osd_sessions": 10, 2024-09-10T10:05:01.687 INFO:teuthology.orchestra.run.smithi102.stdout: "osd_session_open": 11, 2024-09-10T10:05:01.687 INFO:teuthology.orchestra.run.smithi102.stdout: "osd_session_close": 1, 2024-09-10T10:05:01.688 INFO:teuthology.orchestra.run.smithi102.stdout: "osd_laggy": 0, 2024-09-10T10:05:01.688 INFO:teuthology.orchestra.run.smithi102.stdout: "omap_wr": 0, 2024-09-10T10:05:01.688 INFO:teuthology.orchestra.run.smithi102.stdout: "omap_rd": 2, 2024-09-10T10:05:01.688 INFO:teuthology.orchestra.run.smithi102.stdout: "omap_del": 0 2024-09-10T10:05:01.688 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-10T10:05:01.688 INFO:teuthology.orchestra.run.smithi102.stdout: "oft": { 2024-09-10T10:05:01.688 INFO:teuthology.orchestra.run.smithi102.stdout: "omap_total_objs": 0, 2024-09-10T10:05:01.688 INFO:teuthology.orchestra.run.smithi102.stdout: "omap_total_kv_pairs": 0, 2024-09-10T10:05:01.688 INFO:teuthology.orchestra.run.smithi102.stdout: "omap_total_updates": 0, 2024-09-10T10:05:01.688 INFO:teuthology.orchestra.run.smithi102.stdout: "omap_total_removes": 0 2024-09-10T10:05:01.688 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-10T10:05:01.688 INFO:teuthology.orchestra.run.smithi102.stdout: "osd-slow-ops": { 2024-09-10T10:05:01.688 INFO:teuthology.orchestra.run.smithi102.stdout: "slow_ops_count": 0 2024-09-10T10:05:01.688 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-10T10:05:01.689 INFO:teuthology.orchestra.run.smithi102.stdout: "purge_queue": { 2024-09-10T10:05:01.689 INFO:teuthology.orchestra.run.smithi102.stdout: "pq_executing_ops": 0, 2024-09-10T10:05:01.689 INFO:teuthology.orchestra.run.smithi102.stdout: "pq_executing_ops_high_water": 0, 2024-09-10T10:05:01.689 INFO:teuthology.orchestra.run.smithi102.stdout: "pq_executing": 0, 2024-09-10T10:05:01.689 INFO:teuthology.orchestra.run.smithi102.stdout: "pq_executing_high_water": 0, 2024-09-10T10:05:01.689 INFO:teuthology.orchestra.run.smithi102.stdout: "pq_executed_ops": 0, 2024-09-10T10:05:01.689 INFO:teuthology.orchestra.run.smithi102.stdout: "pq_executed": 0, 2024-09-10T10:05:01.689 INFO:teuthology.orchestra.run.smithi102.stdout: "pq_item_in_journal": 0 2024-09-10T10:05:01.689 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-10T10:05:01.689 INFO:teuthology.orchestra.run.smithi102.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-10T10:05:01.689 INFO:teuthology.orchestra.run.smithi102.stdout: "val": 0, 2024-09-10T10:05:01.689 INFO:teuthology.orchestra.run.smithi102.stdout: "max": 104857600, 2024-09-10T10:05:01.689 INFO:teuthology.orchestra.run.smithi102.stdout: "get_started": 0, 2024-09-10T10:05:01.689 INFO:teuthology.orchestra.run.smithi102.stdout: "get": 6022, 2024-09-10T10:05:01.690 INFO:teuthology.orchestra.run.smithi102.stdout: "get_sum": 101946422, 2024-09-10T10:05:01.690 INFO:teuthology.orchestra.run.smithi102.stdout: "get_or_fail_fail": 0, 2024-09-10T10:05:01.690 INFO:teuthology.orchestra.run.smithi102.stdout: "get_or_fail_success": 6022, 2024-09-10T10:05:01.690 INFO:teuthology.orchestra.run.smithi102.stdout: "take": 0, 2024-09-10T10:05:01.690 INFO:teuthology.orchestra.run.smithi102.stdout: "take_sum": 0, 2024-09-10T10:05:01.690 INFO:teuthology.orchestra.run.smithi102.stdout: "put": 6022, 2024-09-10T10:05:01.690 INFO:teuthology.orchestra.run.smithi102.stdout: "put_sum": 101946422, 2024-09-10T10:05:01.690 INFO:teuthology.orchestra.run.smithi102.stdout: "wait": { 2024-09-10T10:05:01.690 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-10T10:05:01.690 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-10T10:05:01.690 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-10T10:05:01.690 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-10T10:05:01.690 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-10T10:05:01.691 INFO:teuthology.orchestra.run.smithi102.stdout: "throttle-objecter_bytes": { 2024-09-10T10:05:01.691 INFO:teuthology.orchestra.run.smithi102.stdout: "val": 0, 2024-09-10T10:05:01.691 INFO:teuthology.orchestra.run.smithi102.stdout: "max": 104857600, 2024-09-10T10:05:01.691 INFO:teuthology.orchestra.run.smithi102.stdout: "get_started": 0, 2024-09-10T10:05:01.691 INFO:teuthology.orchestra.run.smithi102.stdout: "get": 0, 2024-09-10T10:05:01.691 INFO:teuthology.orchestra.run.smithi102.stdout: "get_sum": 0, 2024-09-10T10:05:01.691 INFO:teuthology.orchestra.run.smithi102.stdout: "get_or_fail_fail": 0, 2024-09-10T10:05:01.691 INFO:teuthology.orchestra.run.smithi102.stdout: "get_or_fail_success": 0, 2024-09-10T10:05:01.691 INFO:teuthology.orchestra.run.smithi102.stdout: "take": 5505, 2024-09-10T10:05:01.691 INFO:teuthology.orchestra.run.smithi102.stdout: "take_sum": 99980667, 2024-09-10T10:05:01.691 INFO:teuthology.orchestra.run.smithi102.stdout: "put": 5505, 2024-09-10T10:05:01.691 INFO:teuthology.orchestra.run.smithi102.stdout: "put_sum": 99980667, 2024-09-10T10:05:01.691 INFO:teuthology.orchestra.run.smithi102.stdout: "wait": { 2024-09-10T10:05:01.692 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-10T10:05:01.692 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-10T10:05:01.692 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-10T10:05:01.692 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-10T10:05:01.692 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-10T10:05:01.692 INFO:teuthology.orchestra.run.smithi102.stdout: "throttle-objecter_ops": { 2024-09-10T10:05:01.692 INFO:teuthology.orchestra.run.smithi102.stdout: "val": 0, 2024-09-10T10:05:01.692 INFO:teuthology.orchestra.run.smithi102.stdout: "max": 1024, 2024-09-10T10:05:01.692 INFO:teuthology.orchestra.run.smithi102.stdout: "get_started": 0, 2024-09-10T10:05:01.692 INFO:teuthology.orchestra.run.smithi102.stdout: "get": 0, 2024-09-10T10:05:01.692 INFO:teuthology.orchestra.run.smithi102.stdout: "get_sum": 0, 2024-09-10T10:05:01.692 INFO:teuthology.orchestra.run.smithi102.stdout: "get_or_fail_fail": 0, 2024-09-10T10:05:01.693 INFO:teuthology.orchestra.run.smithi102.stdout: "get_or_fail_success": 0, 2024-09-10T10:05:01.693 INFO:teuthology.orchestra.run.smithi102.stdout: "take": 5505, 2024-09-10T10:05:01.693 INFO:teuthology.orchestra.run.smithi102.stdout: "take_sum": 5505, 2024-09-10T10:05:01.693 INFO:teuthology.orchestra.run.smithi102.stdout: "put": 5505, 2024-09-10T10:05:01.693 INFO:teuthology.orchestra.run.smithi102.stdout: "put_sum": 5505, 2024-09-10T10:05:01.693 INFO:teuthology.orchestra.run.smithi102.stdout: "wait": { 2024-09-10T10:05:01.693 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-10T10:05:01.693 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-10T10:05:01.693 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-10T10:05:01.693 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-10T10:05:01.693 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-10T10:05:01.693 INFO:teuthology.orchestra.run.smithi102.stdout: "throttle-write_buf_throttle": { 2024-09-10T10:05:01.693 INFO:teuthology.orchestra.run.smithi102.stdout: "val": 0, 2024-09-10T10:05:01.693 INFO:teuthology.orchestra.run.smithi102.stdout: "max": 3758096384, 2024-09-10T10:05:01.694 INFO:teuthology.orchestra.run.smithi102.stdout: "get_started": 0, 2024-09-10T10:05:01.694 INFO:teuthology.orchestra.run.smithi102.stdout: "get": 0, 2024-09-10T10:05:01.694 INFO:teuthology.orchestra.run.smithi102.stdout: "get_sum": 0, 2024-09-10T10:05:01.694 INFO:teuthology.orchestra.run.smithi102.stdout: "get_or_fail_fail": 0, 2024-09-10T10:05:01.694 INFO:teuthology.orchestra.run.smithi102.stdout: "get_or_fail_success": 0, 2024-09-10T10:05:01.694 INFO:teuthology.orchestra.run.smithi102.stdout: "take": 0, 2024-09-10T10:05:01.694 INFO:teuthology.orchestra.run.smithi102.stdout: "take_sum": 0, 2024-09-10T10:05:01.694 INFO:teuthology.orchestra.run.smithi102.stdout: "put": 0, 2024-09-10T10:05:01.694 INFO:teuthology.orchestra.run.smithi102.stdout: "put_sum": 0, 2024-09-10T10:05:01.694 INFO:teuthology.orchestra.run.smithi102.stdout: "wait": { 2024-09-10T10:05:01.694 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-10T10:05:01.694 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-10T10:05:01.694 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-10T10:05:01.694 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-10T10:05:01.695 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-10T10:05:01.695 INFO:teuthology.orchestra.run.smithi102.stdout: "throttle-write_buf_throttle-0x55ca16558b40": { 2024-09-10T10:05:01.695 INFO:teuthology.orchestra.run.smithi102.stdout: "val": 0, 2024-09-10T10:05:01.695 INFO:teuthology.orchestra.run.smithi102.stdout: "max": 3758096384, 2024-09-10T10:05:01.695 INFO:teuthology.orchestra.run.smithi102.stdout: "get_started": 0, 2024-09-10T10:05:01.695 INFO:teuthology.orchestra.run.smithi102.stdout: "get": 0, 2024-09-10T10:05:01.695 INFO:teuthology.orchestra.run.smithi102.stdout: "get_sum": 0, 2024-09-10T10:05:01.695 INFO:teuthology.orchestra.run.smithi102.stdout: "get_or_fail_fail": 0, 2024-09-10T10:05:01.695 INFO:teuthology.orchestra.run.smithi102.stdout: "get_or_fail_success": 0, 2024-09-10T10:05:01.695 INFO:teuthology.orchestra.run.smithi102.stdout: "take": 0, 2024-09-10T10:05:01.695 INFO:teuthology.orchestra.run.smithi102.stdout: "take_sum": 0, 2024-09-10T10:05:01.695 INFO:teuthology.orchestra.run.smithi102.stdout: "put": 0, 2024-09-10T10:05:01.695 INFO:teuthology.orchestra.run.smithi102.stdout: "put_sum": 0, 2024-09-10T10:05:01.696 INFO:teuthology.orchestra.run.smithi102.stdout: "wait": { 2024-09-10T10:05:01.696 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-10T10:05:01.696 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-10T10:05:01.696 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-10T10:05:01.696 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-10T10:05:01.696 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-10T10:05:01.696 INFO:teuthology.orchestra.run.smithi102.stdout:} 2024-09-10T10:05:02.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:05:01 smithi159 ceph-mon[35513]: pgmap v1193: 129 pgs: 129 active+clean; 551 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 426 B/s wr, 11 op/s 2024-09-10T10:05:02.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:05:01 smithi102 ceph-mon[35442]: pgmap v1193: 129 pgs: 129 active+clean; 551 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 426 B/s wr, 11 op/s 2024-09-10T10:05:02.306 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:05:01 smithi005 ceph-mon[30350]: pgmap v1193: 129 pgs: 129 active+clean; 551 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 426 B/s wr, 11 op/s 2024-09-10T10:05:02.396 INFO:tasks.check_counter:Daemon mds.e mds.exported=0 2024-09-10T10:05:02.396 INFO:tasks.check_counter:Daemon mds.e mds.imported=0 2024-09-10T10:05:02.396 DEBUG:tasks.check_counter:Getting stats from h 2024-09-10T10:05:02.396 DEBUG:teuthology.orchestra.run.smithi102:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph daemon mds.h perf dump 2024-09-10T10:05:02.642 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T10:05:03.179 INFO:teuthology.orchestra.run.smithi102.stdout:{ 2024-09-10T10:05:03.180 INFO:teuthology.orchestra.run.smithi102.stdout: "AsyncMessenger::Worker-0": { 2024-09-10T10:05:03.180 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_recv_messages": 29952, 2024-09-10T10:05:03.180 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_messages": 32909, 2024-09-10T10:05:03.180 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_recv_bytes": 9034025, 2024-09-10T10:05:03.180 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_bytes": 31942123, 2024-09-10T10:05:03.180 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_created_connections": 10, 2024-09-10T10:05:03.180 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_active_connections": 4, 2024-09-10T10:05:03.180 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_total_time": 2.397860400, 2024-09-10T10:05:03.180 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_send_time": 1.033635881, 2024-09-10T10:05:03.181 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_recv_time": 1.106617264, 2024-09-10T10:05:03.181 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_fast_dispatch_time": 0.345789418, 2024-09-10T10:05:03.181 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T10:05:03.181 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 32910, 2024-09-10T10:05:03.181 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 1.066933325, 2024-09-10T10:05:03.181 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000032419 2024-09-10T10:05:03.181 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-10T10:05:03.181 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_handle_ack_lat": { 2024-09-10T10:05:03.181 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 38995, 2024-09-10T10:05:03.181 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.007593548, 2024-09-10T10:05:03.181 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000194 2024-09-10T10:05:03.181 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-10T10:05:03.182 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_recv_encrypted_bytes": 1004112, 2024-09-10T10:05:03.182 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_encrypted_bytes": 311440 2024-09-10T10:05:03.182 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-10T10:05:03.182 INFO:teuthology.orchestra.run.smithi102.stdout: "AsyncMessenger::Worker-1": { 2024-09-10T10:05:03.182 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_recv_messages": 135120, 2024-09-10T10:05:03.182 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_messages": 112043, 2024-09-10T10:05:03.182 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_recv_bytes": 34606284, 2024-09-10T10:05:03.182 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_bytes": 69589840, 2024-09-10T10:05:03.182 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_created_connections": 16, 2024-09-10T10:05:03.182 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_active_connections": 7, 2024-09-10T10:05:03.182 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_total_time": 6.930352324, 2024-09-10T10:05:03.182 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_send_time": 2.602280481, 2024-09-10T10:05:03.182 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_recv_time": 4.866575715, 2024-09-10T10:05:03.183 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_fast_dispatch_time": 0.429663179, 2024-09-10T10:05:03.183 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T10:05:03.183 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 112041, 2024-09-10T10:05:03.183 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 3.015129586, 2024-09-10T10:05:03.183 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000026910 2024-09-10T10:05:03.183 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-10T10:05:03.183 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_handle_ack_lat": { 2024-09-10T10:05:03.183 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 216659, 2024-09-10T10:05:03.183 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.033632142, 2024-09-10T10:05:03.183 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000155 2024-09-10T10:05:03.183 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-10T10:05:03.183 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_recv_encrypted_bytes": 160, 2024-09-10T10:05:03.183 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_encrypted_bytes": 711760 2024-09-10T10:05:03.184 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-10T10:05:03.184 INFO:teuthology.orchestra.run.smithi102.stdout: "AsyncMessenger::Worker-2": { 2024-09-10T10:05:03.184 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_recv_messages": 121636, 2024-09-10T10:05:03.184 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_messages": 154460, 2024-09-10T10:05:03.184 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_recv_bytes": 36018058, 2024-09-10T10:05:03.184 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_bytes": 125464667, 2024-09-10T10:05:03.184 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_created_connections": 66, 2024-09-10T10:05:03.184 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_active_connections": 7, 2024-09-10T10:05:03.184 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_total_time": 6.670730296, 2024-09-10T10:05:03.184 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_send_time": 3.118909266, 2024-09-10T10:05:03.184 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_recv_time": 4.464252278, 2024-09-10T10:05:03.184 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_fast_dispatch_time": 0.267328907, 2024-09-10T10:05:03.185 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T10:05:03.185 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 154457, 2024-09-10T10:05:03.185 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 4.592159558, 2024-09-10T10:05:03.185 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000029730 2024-09-10T10:05:03.185 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-10T10:05:03.185 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_handle_ack_lat": { 2024-09-10T10:05:03.185 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 149630, 2024-09-10T10:05:03.185 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.037376067, 2024-09-10T10:05:03.185 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000249 2024-09-10T10:05:03.185 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-10T10:05:03.185 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_recv_encrypted_bytes": 3024, 2024-09-10T10:05:03.185 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_encrypted_bytes": 448 2024-09-10T10:05:03.186 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-10T10:05:03.186 INFO:teuthology.orchestra.run.smithi102.stdout: "cct": { 2024-09-10T10:05:03.186 INFO:teuthology.orchestra.run.smithi102.stdout: "total_workers": 1, 2024-09-10T10:05:03.186 INFO:teuthology.orchestra.run.smithi102.stdout: "unhealthy_workers": 0 2024-09-10T10:05:03.186 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-10T10:05:03.186 INFO:teuthology.orchestra.run.smithi102.stdout: "finisher-MDSRank": { 2024-09-10T10:05:03.186 INFO:teuthology.orchestra.run.smithi102.stdout: "queue_len": 0, 2024-09-10T10:05:03.186 INFO:teuthology.orchestra.run.smithi102.stdout: "complete_latency": { 2024-09-10T10:05:03.186 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 32446, 2024-09-10T10:05:03.186 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 22.384599371, 2024-09-10T10:05:03.186 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000689903 2024-09-10T10:05:03.186 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-10T10:05:03.186 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-10T10:05:03.186 INFO:teuthology.orchestra.run.smithi102.stdout: "finisher-PurgeQueue": { 2024-09-10T10:05:03.187 INFO:teuthology.orchestra.run.smithi102.stdout: "queue_len": 0, 2024-09-10T10:05:03.187 INFO:teuthology.orchestra.run.smithi102.stdout: "complete_latency": { 2024-09-10T10:05:03.187 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 15003, 2024-09-10T10:05:03.187 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.955350444, 2024-09-10T10:05:03.187 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000063677 2024-09-10T10:05:03.187 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-10T10:05:03.187 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-10T10:05:03.187 INFO:teuthology.orchestra.run.smithi102.stdout: "mds": { 2024-09-10T10:05:03.187 INFO:teuthology.orchestra.run.smithi102.stdout: "request": 59956, 2024-09-10T10:05:03.187 INFO:teuthology.orchestra.run.smithi102.stdout: "reply": 51188, 2024-09-10T10:05:03.188 INFO:teuthology.orchestra.run.smithi102.stdout: "reply_latency": { 2024-09-10T10:05:03.188 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 51188, 2024-09-10T10:05:03.188 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 585.832860039, 2024-09-10T10:05:03.188 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.011444730 2024-09-10T10:05:03.188 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-10T10:05:03.188 INFO:teuthology.orchestra.run.smithi102.stdout: "slow_reply": 0, 2024-09-10T10:05:03.188 INFO:teuthology.orchestra.run.smithi102.stdout: "forward": 8768, 2024-09-10T10:05:03.188 INFO:teuthology.orchestra.run.smithi102.stdout: "dir_fetch_complete": 0, 2024-09-10T10:05:03.188 INFO:teuthology.orchestra.run.smithi102.stdout: "dir_fetch_keys": 0, 2024-09-10T10:05:03.188 INFO:teuthology.orchestra.run.smithi102.stdout: "dir_commit": 11, 2024-09-10T10:05:03.188 INFO:teuthology.orchestra.run.smithi102.stdout: "dir_split": 0, 2024-09-10T10:05:03.188 INFO:teuthology.orchestra.run.smithi102.stdout: "dir_merge": 0, 2024-09-10T10:05:03.189 INFO:teuthology.orchestra.run.smithi102.stdout: "inodes": 13, 2024-09-10T10:05:03.189 INFO:teuthology.orchestra.run.smithi102.stdout: "inodes_top": 2, 2024-09-10T10:05:03.189 INFO:teuthology.orchestra.run.smithi102.stdout: "inodes_bottom": 1, 2024-09-10T10:05:03.189 INFO:teuthology.orchestra.run.smithi102.stdout: "inodes_pin_tail": 10, 2024-09-10T10:05:03.189 INFO:teuthology.orchestra.run.smithi102.stdout: "inodes_pinned": 10, 2024-09-10T10:05:03.189 INFO:teuthology.orchestra.run.smithi102.stdout: "inodes_expired": 86, 2024-09-10T10:05:03.189 INFO:teuthology.orchestra.run.smithi102.stdout: "inodes_with_caps": 0, 2024-09-10T10:05:03.189 INFO:teuthology.orchestra.run.smithi102.stdout: "caps": 0, 2024-09-10T10:05:03.189 INFO:teuthology.orchestra.run.smithi102.stdout: "subtrees": 4, 2024-09-10T10:05:03.189 INFO:teuthology.orchestra.run.smithi102.stdout: "traverse": 103234, 2024-09-10T10:05:03.189 INFO:teuthology.orchestra.run.smithi102.stdout: "traverse_hit": 58433, 2024-09-10T10:05:03.189 INFO:teuthology.orchestra.run.smithi102.stdout: "traverse_forward": 8767, 2024-09-10T10:05:03.189 INFO:teuthology.orchestra.run.smithi102.stdout: "traverse_discover": 31, 2024-09-10T10:05:03.190 INFO:teuthology.orchestra.run.smithi102.stdout: "traverse_dir_fetch": 0, 2024-09-10T10:05:03.190 INFO:teuthology.orchestra.run.smithi102.stdout: "traverse_remote_ino": 0, 2024-09-10T10:05:03.190 INFO:teuthology.orchestra.run.smithi102.stdout: "traverse_lock": 1683, 2024-09-10T10:05:03.190 INFO:teuthology.orchestra.run.smithi102.stdout: "load_cent": 0, 2024-09-10T10:05:03.190 INFO:teuthology.orchestra.run.smithi102.stdout: "q": 0, 2024-09-10T10:05:03.190 INFO:teuthology.orchestra.run.smithi102.stdout: "exported": 4, 2024-09-10T10:05:03.190 INFO:teuthology.orchestra.run.smithi102.stdout: "exported_inodes": 6, 2024-09-10T10:05:03.190 INFO:teuthology.orchestra.run.smithi102.stdout: "imported": 13, 2024-09-10T10:05:03.190 INFO:teuthology.orchestra.run.smithi102.stdout: "imported_inodes": 129, 2024-09-10T10:05:03.190 INFO:teuthology.orchestra.run.smithi102.stdout: "openino_dir_fetch": 0, 2024-09-10T10:05:03.190 INFO:teuthology.orchestra.run.smithi102.stdout: "openino_backtrace_fetch": 983, 2024-09-10T10:05:03.190 INFO:teuthology.orchestra.run.smithi102.stdout: "openino_peer_discover": 1, 2024-09-10T10:05:03.190 INFO:teuthology.orchestra.run.smithi102.stdout: "root_rfiles": 733, 2024-09-10T10:05:03.190 INFO:teuthology.orchestra.run.smithi102.stdout: "root_rbytes": 226265082, 2024-09-10T10:05:03.191 INFO:teuthology.orchestra.run.smithi102.stdout: "root_rsnaps": 0, 2024-09-10T10:05:03.191 INFO:teuthology.orchestra.run.smithi102.stdout: "scrub_backtrace_fetch": 983, 2024-09-10T10:05:03.191 INFO:teuthology.orchestra.run.smithi102.stdout: "scrub_set_tag": 0, 2024-09-10T10:05:03.191 INFO:teuthology.orchestra.run.smithi102.stdout: "scrub_backtrace_repaired": 0, 2024-09-10T10:05:03.191 INFO:teuthology.orchestra.run.smithi102.stdout: "scrub_inotable_repaired": 0, 2024-09-10T10:05:03.191 INFO:teuthology.orchestra.run.smithi102.stdout: "scrub_dir_inodes": 99, 2024-09-10T10:05:03.191 INFO:teuthology.orchestra.run.smithi102.stdout: "scrub_dir_base_inodes": 0, 2024-09-10T10:05:03.191 INFO:teuthology.orchestra.run.smithi102.stdout: "scrub_dirfrag_rstats": 99, 2024-09-10T10:05:03.191 INFO:teuthology.orchestra.run.smithi102.stdout: "scrub_file_inodes": 884, 2024-09-10T10:05:03.191 INFO:teuthology.orchestra.run.smithi102.stdout: "handle_inode_file_caps": 12, 2024-09-10T10:05:03.191 INFO:teuthology.orchestra.run.smithi102.stdout: "ceph_cap_op_revoke": 5236, 2024-09-10T10:05:03.191 INFO:teuthology.orchestra.run.smithi102.stdout: "ceph_cap_op_grant": 28692, 2024-09-10T10:05:03.191 INFO:teuthology.orchestra.run.smithi102.stdout: "ceph_cap_op_trunc": 0, 2024-09-10T10:05:03.191 INFO:teuthology.orchestra.run.smithi102.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-10T10:05:03.192 INFO:teuthology.orchestra.run.smithi102.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-10T10:05:03.192 INFO:teuthology.orchestra.run.smithi102.stdout: "handle_client_caps": 25033, 2024-09-10T10:05:03.192 INFO:teuthology.orchestra.run.smithi102.stdout: "handle_client_caps_dirty": 7760, 2024-09-10T10:05:03.192 INFO:teuthology.orchestra.run.smithi102.stdout: "handle_client_cap_release": 1426, 2024-09-10T10:05:03.192 INFO:teuthology.orchestra.run.smithi102.stdout: "process_request_cap_release": 36237 2024-09-10T10:05:03.192 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-10T10:05:03.192 INFO:teuthology.orchestra.run.smithi102.stdout: "mds_cache": { 2024-09-10T10:05:03.192 INFO:teuthology.orchestra.run.smithi102.stdout: "dir_update": 7, 2024-09-10T10:05:03.192 INFO:teuthology.orchestra.run.smithi102.stdout: "dir_update_receipt": 218, 2024-09-10T10:05:03.192 INFO:teuthology.orchestra.run.smithi102.stdout: "dir_try_discover": 117, 2024-09-10T10:05:03.192 INFO:teuthology.orchestra.run.smithi102.stdout: "dir_send_discover": 231, 2024-09-10T10:05:03.192 INFO:teuthology.orchestra.run.smithi102.stdout: "dir_handle_discover": 32, 2024-09-10T10:05:03.192 INFO:teuthology.orchestra.run.smithi102.stdout: "num_strays": 0, 2024-09-10T10:05:03.192 INFO:teuthology.orchestra.run.smithi102.stdout: "num_strays_delayed": 0, 2024-09-10T10:05:03.192 INFO:teuthology.orchestra.run.smithi102.stdout: "num_strays_enqueuing": 0, 2024-09-10T10:05:03.193 INFO:teuthology.orchestra.run.smithi102.stdout: "strays_created": 7401, 2024-09-10T10:05:03.193 INFO:teuthology.orchestra.run.smithi102.stdout: "strays_enqueued": 7401, 2024-09-10T10:05:03.193 INFO:teuthology.orchestra.run.smithi102.stdout: "strays_reintegrated": 0, 2024-09-10T10:05:03.193 INFO:teuthology.orchestra.run.smithi102.stdout: "strays_migrated": 0, 2024-09-10T10:05:03.193 INFO:teuthology.orchestra.run.smithi102.stdout: "num_recovering_processing": 0, 2024-09-10T10:05:03.193 INFO:teuthology.orchestra.run.smithi102.stdout: "num_recovering_enqueued": 0, 2024-09-10T10:05:03.193 INFO:teuthology.orchestra.run.smithi102.stdout: "num_recovering_prioritized": 0, 2024-09-10T10:05:03.193 INFO:teuthology.orchestra.run.smithi102.stdout: "recovery_started": 0, 2024-09-10T10:05:03.193 INFO:teuthology.orchestra.run.smithi102.stdout: "recovery_completed": 0, 2024-09-10T10:05:03.193 INFO:teuthology.orchestra.run.smithi102.stdout: "ireq_quiesce_path": 23, 2024-09-10T10:05:03.193 INFO:teuthology.orchestra.run.smithi102.stdout: "ireq_quiesce_inode": 1466, 2024-09-10T10:05:03.193 INFO:teuthology.orchestra.run.smithi102.stdout: "ireq_enqueue_scrub": 0, 2024-09-10T10:05:03.193 INFO:teuthology.orchestra.run.smithi102.stdout: "ireq_exportdir": 4, 2024-09-10T10:05:03.194 INFO:teuthology.orchestra.run.smithi102.stdout: "ireq_flush": 0, 2024-09-10T10:05:03.194 INFO:teuthology.orchestra.run.smithi102.stdout: "ireq_fragmentdir": 0, 2024-09-10T10:05:03.194 INFO:teuthology.orchestra.run.smithi102.stdout: "ireq_fragstats": 0, 2024-09-10T10:05:03.194 INFO:teuthology.orchestra.run.smithi102.stdout: "ireq_inodestats": 0 2024-09-10T10:05:03.194 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-10T10:05:03.194 INFO:teuthology.orchestra.run.smithi102.stdout: "mds_log": { 2024-09-10T10:05:03.194 INFO:teuthology.orchestra.run.smithi102.stdout: "evlrg": 0, 2024-09-10T10:05:03.194 INFO:teuthology.orchestra.run.smithi102.stdout: "evadd": 43840, 2024-09-10T10:05:03.194 INFO:teuthology.orchestra.run.smithi102.stdout: "evex": 0, 2024-09-10T10:05:03.194 INFO:teuthology.orchestra.run.smithi102.stdout: "evtrm": 0, 2024-09-10T10:05:03.194 INFO:teuthology.orchestra.run.smithi102.stdout: "ev": 43840, 2024-09-10T10:05:03.194 INFO:teuthology.orchestra.run.smithi102.stdout: "evexg": 0, 2024-09-10T10:05:03.194 INFO:teuthology.orchestra.run.smithi102.stdout: "evexd": 0, 2024-09-10T10:05:03.194 INFO:teuthology.orchestra.run.smithi102.stdout: "segadd": 59, 2024-09-10T10:05:03.194 INFO:teuthology.orchestra.run.smithi102.stdout: "segex": 0, 2024-09-10T10:05:03.195 INFO:teuthology.orchestra.run.smithi102.stdout: "segtrm": 0, 2024-09-10T10:05:03.195 INFO:teuthology.orchestra.run.smithi102.stdout: "seg": 59, 2024-09-10T10:05:03.195 INFO:teuthology.orchestra.run.smithi102.stdout: "segmjr": 5, 2024-09-10T10:05:03.195 INFO:teuthology.orchestra.run.smithi102.stdout: "segexg": 0, 2024-09-10T10:05:03.195 INFO:teuthology.orchestra.run.smithi102.stdout: "segexd": 0, 2024-09-10T10:05:03.195 INFO:teuthology.orchestra.run.smithi102.stdout: "expos": 4194304, 2024-09-10T10:05:03.195 INFO:teuthology.orchestra.run.smithi102.stdout: "wrpos": 88509846, 2024-09-10T10:05:03.195 INFO:teuthology.orchestra.run.smithi102.stdout: "rdpos": 0, 2024-09-10T10:05:03.195 INFO:teuthology.orchestra.run.smithi102.stdout: "jlat": { 2024-09-10T10:05:03.195 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 16316, 2024-09-10T10:05:03.195 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 92.119077978, 2024-09-10T10:05:03.195 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.005645935 2024-09-10T10:05:03.195 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-10T10:05:03.195 INFO:teuthology.orchestra.run.smithi102.stdout: "replayed": 0 2024-09-10T10:05:03.196 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-10T10:05:03.196 INFO:teuthology.orchestra.run.smithi102.stdout: "mds_mem": { 2024-09-10T10:05:03.196 INFO:teuthology.orchestra.run.smithi102.stdout: "ino": 19, 2024-09-10T10:05:03.196 INFO:teuthology.orchestra.run.smithi102.stdout: "ino+": 7463, 2024-09-10T10:05:03.196 INFO:teuthology.orchestra.run.smithi102.stdout: "ino-": 7444, 2024-09-10T10:05:03.196 INFO:teuthology.orchestra.run.smithi102.stdout: "dir": 18, 2024-09-10T10:05:03.196 INFO:teuthology.orchestra.run.smithi102.stdout: "dir+": 172, 2024-09-10T10:05:03.196 INFO:teuthology.orchestra.run.smithi102.stdout: "dir-": 154, 2024-09-10T10:05:03.196 INFO:teuthology.orchestra.run.smithi102.stdout: "dn": 13, 2024-09-10T10:05:03.196 INFO:teuthology.orchestra.run.smithi102.stdout: "dn+": 12852, 2024-09-10T10:05:03.196 INFO:teuthology.orchestra.run.smithi102.stdout: "dn-": 12839, 2024-09-10T10:05:03.196 INFO:teuthology.orchestra.run.smithi102.stdout: "cap": 0, 2024-09-10T10:05:03.196 INFO:teuthology.orchestra.run.smithi102.stdout: "cap+": 8621, 2024-09-10T10:05:03.196 INFO:teuthology.orchestra.run.smithi102.stdout: "cap-": 8621, 2024-09-10T10:05:03.197 INFO:teuthology.orchestra.run.smithi102.stdout: "rss": 67124, 2024-09-10T10:05:03.197 INFO:teuthology.orchestra.run.smithi102.stdout: "heap": 223520 2024-09-10T10:05:03.197 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-10T10:05:03.197 INFO:teuthology.orchestra.run.smithi102.stdout: "mds_server": { 2024-09-10T10:05:03.197 INFO:teuthology.orchestra.run.smithi102.stdout: "dispatch_client_request": 86840, 2024-09-10T10:05:03.197 INFO:teuthology.orchestra.run.smithi102.stdout: "dispatch_server_request": 5291, 2024-09-10T10:05:03.197 INFO:teuthology.orchestra.run.smithi102.stdout: "handle_client_request": 59956, 2024-09-10T10:05:03.197 INFO:teuthology.orchestra.run.smithi102.stdout: "handle_client_session": 1562, 2024-09-10T10:05:03.197 INFO:teuthology.orchestra.run.smithi102.stdout: "handle_peer_request": 13336, 2024-09-10T10:05:03.197 INFO:teuthology.orchestra.run.smithi102.stdout: "req_create_latency": { 2024-09-10T10:05:03.197 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 7314, 2024-09-10T10:05:03.197 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 314.374671229, 2024-09-10T10:05:03.198 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.042982591 2024-09-10T10:05:03.198 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-10T10:05:03.198 INFO:teuthology.orchestra.run.smithi102.stdout: "req_getattr_latency": { 2024-09-10T10:05:03.198 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 6801, 2024-09-10T10:05:03.198 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 55.270992906, 2024-09-10T10:05:03.198 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.008126892 2024-09-10T10:05:03.198 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-10T10:05:03.198 INFO:teuthology.orchestra.run.smithi102.stdout: "req_getfilelock_latency": { 2024-09-10T10:05:03.198 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-10T10:05:03.198 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-10T10:05:03.198 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-10T10:05:03.198 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-10T10:05:03.198 INFO:teuthology.orchestra.run.smithi102.stdout: "req_link_latency": { 2024-09-10T10:05:03.199 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-10T10:05:03.199 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-10T10:05:03.199 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-10T10:05:03.199 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-10T10:05:03.199 INFO:teuthology.orchestra.run.smithi102.stdout: "req_lookup_latency": { 2024-09-10T10:05:03.199 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 14755, 2024-09-10T10:05:03.199 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 14.146063858, 2024-09-10T10:05:03.199 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000958730 2024-09-10T10:05:03.199 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-10T10:05:03.199 INFO:teuthology.orchestra.run.smithi102.stdout: "req_lookuphash_latency": { 2024-09-10T10:05:03.199 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-10T10:05:03.199 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-10T10:05:03.199 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-10T10:05:03.199 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-10T10:05:03.200 INFO:teuthology.orchestra.run.smithi102.stdout: "req_lookupino_latency": { 2024-09-10T10:05:03.200 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-10T10:05:03.200 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-10T10:05:03.200 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-10T10:05:03.200 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-10T10:05:03.200 INFO:teuthology.orchestra.run.smithi102.stdout: "req_lookupname_latency": { 2024-09-10T10:05:03.200 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-10T10:05:03.200 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-10T10:05:03.200 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-10T10:05:03.200 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-10T10:05:03.200 INFO:teuthology.orchestra.run.smithi102.stdout: "req_lookupparent_latency": { 2024-09-10T10:05:03.200 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-10T10:05:03.200 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-10T10:05:03.200 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-10T10:05:03.201 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-10T10:05:03.201 INFO:teuthology.orchestra.run.smithi102.stdout: "req_lookupsnap_latency": { 2024-09-10T10:05:03.201 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-10T10:05:03.201 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-10T10:05:03.201 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-10T10:05:03.201 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-10T10:05:03.201 INFO:teuthology.orchestra.run.smithi102.stdout: "req_lssnap_latency": { 2024-09-10T10:05:03.201 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-10T10:05:03.201 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-10T10:05:03.201 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-10T10:05:03.201 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-10T10:05:03.201 INFO:teuthology.orchestra.run.smithi102.stdout: "req_mkdir_latency": { 2024-09-10T10:05:03.202 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-10T10:05:03.202 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-10T10:05:03.202 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-10T10:05:03.202 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-10T10:05:03.202 INFO:teuthology.orchestra.run.smithi102.stdout: "req_mknod_latency": { 2024-09-10T10:05:03.202 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-10T10:05:03.202 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-10T10:05:03.202 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-10T10:05:03.202 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-10T10:05:03.202 INFO:teuthology.orchestra.run.smithi102.stdout: "req_mksnap_latency": { 2024-09-10T10:05:03.202 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-10T10:05:03.202 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-10T10:05:03.202 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-10T10:05:03.202 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-10T10:05:03.202 INFO:teuthology.orchestra.run.smithi102.stdout: "req_open_latency": { 2024-09-10T10:05:03.203 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 1, 2024-09-10T10:05:03.203 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000592920, 2024-09-10T10:05:03.203 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000592920 2024-09-10T10:05:03.203 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-10T10:05:03.203 INFO:teuthology.orchestra.run.smithi102.stdout: "req_readdir_latency": { 2024-09-10T10:05:03.203 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 12078, 2024-09-10T10:05:03.203 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 24.286661981, 2024-09-10T10:05:03.203 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.002010818 2024-09-10T10:05:03.204 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-10T10:05:03.204 INFO:teuthology.orchestra.run.smithi102.stdout: "req_rename_latency": { 2024-09-10T10:05:03.204 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 2227, 2024-09-10T10:05:03.204 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 64.463329274, 2024-09-10T10:05:03.204 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.028946263 2024-09-10T10:05:03.204 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-10T10:05:03.204 INFO:teuthology.orchestra.run.smithi102.stdout: "req_renamesnap_latency": { 2024-09-10T10:05:03.204 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-10T10:05:03.204 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-10T10:05:03.204 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-10T10:05:03.204 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-10T10:05:03.204 INFO:teuthology.orchestra.run.smithi102.stdout: "req_snapdiff_latency": { 2024-09-10T10:05:03.204 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-10T10:05:03.204 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-10T10:05:03.205 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-10T10:05:03.205 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-10T10:05:03.205 INFO:teuthology.orchestra.run.smithi102.stdout: "req_rmdir_latency": { 2024-09-10T10:05:03.205 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 9, 2024-09-10T10:05:03.205 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.063512554, 2024-09-10T10:05:03.205 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.007056950 2024-09-10T10:05:03.205 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-10T10:05:03.205 INFO:teuthology.orchestra.run.smithi102.stdout: "req_rmsnap_latency": { 2024-09-10T10:05:03.205 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-10T10:05:03.205 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-10T10:05:03.205 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-10T10:05:03.205 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-10T10:05:03.205 INFO:teuthology.orchestra.run.smithi102.stdout: "req_rmxattr_latency": { 2024-09-10T10:05:03.205 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-10T10:05:03.205 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-10T10:05:03.206 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-10T10:05:03.206 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-10T10:05:03.206 INFO:teuthology.orchestra.run.smithi102.stdout: "req_setattr_latency": { 2024-09-10T10:05:03.206 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 27, 2024-09-10T10:05:03.206 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.257793076, 2024-09-10T10:05:03.206 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.009547891 2024-09-10T10:05:03.206 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-10T10:05:03.206 INFO:teuthology.orchestra.run.smithi102.stdout: "req_setdirlayout_latency": { 2024-09-10T10:05:03.206 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-10T10:05:03.206 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-10T10:05:03.206 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-10T10:05:03.206 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-10T10:05:03.206 INFO:teuthology.orchestra.run.smithi102.stdout: "req_setfilelock_latency": { 2024-09-10T10:05:03.206 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 584, 2024-09-10T10:05:03.207 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.346286108, 2024-09-10T10:05:03.207 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000592955 2024-09-10T10:05:03.207 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-10T10:05:03.207 INFO:teuthology.orchestra.run.smithi102.stdout: "req_setlayout_latency": { 2024-09-10T10:05:03.207 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-10T10:05:03.207 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-10T10:05:03.207 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-10T10:05:03.207 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-10T10:05:03.207 INFO:teuthology.orchestra.run.smithi102.stdout: "req_setxattr_latency": { 2024-09-10T10:05:03.207 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-10T10:05:03.207 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-10T10:05:03.207 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-10T10:05:03.207 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-10T10:05:03.207 INFO:teuthology.orchestra.run.smithi102.stdout: "req_symlink_latency": { 2024-09-10T10:05:03.207 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-10T10:05:03.208 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-10T10:05:03.208 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-10T10:05:03.208 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-10T10:05:03.208 INFO:teuthology.orchestra.run.smithi102.stdout: "req_unlink_latency": { 2024-09-10T10:05:03.208 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 7392, 2024-09-10T10:05:03.208 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 112.622956133, 2024-09-10T10:05:03.208 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.015235789 2024-09-10T10:05:03.208 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-10T10:05:03.208 INFO:teuthology.orchestra.run.smithi102.stdout: "cap_revoke_eviction": 0, 2024-09-10T10:05:03.208 INFO:teuthology.orchestra.run.smithi102.stdout: "cap_acquisition_throttle": 0, 2024-09-10T10:05:03.208 INFO:teuthology.orchestra.run.smithi102.stdout: "req_getvxattr_latency": { 2024-09-10T10:05:03.208 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-10T10:05:03.208 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-10T10:05:03.209 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-10T10:05:03.209 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-10T10:05:03.209 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-10T10:05:03.209 INFO:teuthology.orchestra.run.smithi102.stdout: "mds_sessions": { 2024-09-10T10:05:03.209 INFO:teuthology.orchestra.run.smithi102.stdout: "session_count": 1, 2024-09-10T10:05:03.209 INFO:teuthology.orchestra.run.smithi102.stdout: "session_add": 1, 2024-09-10T10:05:03.209 INFO:teuthology.orchestra.run.smithi102.stdout: "session_remove": 0, 2024-09-10T10:05:03.209 INFO:teuthology.orchestra.run.smithi102.stdout: "sessions_open": 1, 2024-09-10T10:05:03.209 INFO:teuthology.orchestra.run.smithi102.stdout: "sessions_stale": 0, 2024-09-10T10:05:03.209 INFO:teuthology.orchestra.run.smithi102.stdout: "total_load": 5096, 2024-09-10T10:05:03.209 INFO:teuthology.orchestra.run.smithi102.stdout: "average_load": 5096, 2024-09-10T10:05:03.209 INFO:teuthology.orchestra.run.smithi102.stdout: "avg_session_uptime": 1577, 2024-09-10T10:05:03.209 INFO:teuthology.orchestra.run.smithi102.stdout: "mdthresh_evicted": 0 2024-09-10T10:05:03.209 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-10T10:05:03.210 INFO:teuthology.orchestra.run.smithi102.stdout: "mempool": { 2024-09-10T10:05:03.210 INFO:teuthology.orchestra.run.smithi102.stdout: "bloom_filter_bytes": 0, 2024-09-10T10:05:03.210 INFO:teuthology.orchestra.run.smithi102.stdout: "bloom_filter_items": 0, 2024-09-10T10:05:03.210 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_alloc_bytes": 0, 2024-09-10T10:05:03.210 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_alloc_items": 0, 2024-09-10T10:05:03.210 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_data_bytes": 0, 2024-09-10T10:05:03.210 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_data_items": 0, 2024-09-10T10:05:03.210 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-10T10:05:03.210 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_onode_items": 0, 2024-09-10T10:05:03.210 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-10T10:05:03.210 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_meta_items": 0, 2024-09-10T10:05:03.210 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_other_bytes": 0, 2024-09-10T10:05:03.211 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_other_items": 0, 2024-09-10T10:05:03.211 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-10T10:05:03.211 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_buffer_items": 0, 2024-09-10T10:05:03.211 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_extent_bytes": 0, 2024-09-10T10:05:03.211 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_extent_items": 0, 2024-09-10T10:05:03.211 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_blob_bytes": 0, 2024-09-10T10:05:03.211 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_blob_items": 0, 2024-09-10T10:05:03.211 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-10T10:05:03.211 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_shared_blob_items": 0, 2024-09-10T10:05:03.211 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-10T10:05:03.211 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_inline_bl_items": 0, 2024-09-10T10:05:03.211 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_fsck_bytes": 0, 2024-09-10T10:05:03.211 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_fsck_items": 0, 2024-09-10T10:05:03.211 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_txc_bytes": 0, 2024-09-10T10:05:03.212 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_txc_items": 0, 2024-09-10T10:05:03.212 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-10T10:05:03.212 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_writing_deferred_items": 0, 2024-09-10T10:05:03.212 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_writing_bytes": 0, 2024-09-10T10:05:03.212 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_writing_items": 0, 2024-09-10T10:05:03.212 INFO:teuthology.orchestra.run.smithi102.stdout: "bluefs_bytes": 0, 2024-09-10T10:05:03.212 INFO:teuthology.orchestra.run.smithi102.stdout: "bluefs_items": 0, 2024-09-10T10:05:03.212 INFO:teuthology.orchestra.run.smithi102.stdout: "bluefs_file_reader_bytes": 0, 2024-09-10T10:05:03.212 INFO:teuthology.orchestra.run.smithi102.stdout: "bluefs_file_reader_items": 0, 2024-09-10T10:05:03.212 INFO:teuthology.orchestra.run.smithi102.stdout: "bluefs_file_writer_bytes": 0, 2024-09-10T10:05:03.212 INFO:teuthology.orchestra.run.smithi102.stdout: "bluefs_file_writer_items": 0, 2024-09-10T10:05:03.212 INFO:teuthology.orchestra.run.smithi102.stdout: "buffer_anon_bytes": 280115, 2024-09-10T10:05:03.212 INFO:teuthology.orchestra.run.smithi102.stdout: "buffer_anon_items": 103, 2024-09-10T10:05:03.212 INFO:teuthology.orchestra.run.smithi102.stdout: "buffer_meta_bytes": 176, 2024-09-10T10:05:03.213 INFO:teuthology.orchestra.run.smithi102.stdout: "buffer_meta_items": 2, 2024-09-10T10:05:03.213 INFO:teuthology.orchestra.run.smithi102.stdout: "osd_bytes": 0, 2024-09-10T10:05:03.213 INFO:teuthology.orchestra.run.smithi102.stdout: "osd_items": 0, 2024-09-10T10:05:03.213 INFO:teuthology.orchestra.run.smithi102.stdout: "osd_mapbl_bytes": 0, 2024-09-10T10:05:03.213 INFO:teuthology.orchestra.run.smithi102.stdout: "osd_mapbl_items": 0, 2024-09-10T10:05:03.213 INFO:teuthology.orchestra.run.smithi102.stdout: "osd_pglog_bytes": 0, 2024-09-10T10:05:03.213 INFO:teuthology.orchestra.run.smithi102.stdout: "osd_pglog_items": 0, 2024-09-10T10:05:03.213 INFO:teuthology.orchestra.run.smithi102.stdout: "osdmap_bytes": 9152, 2024-09-10T10:05:03.213 INFO:teuthology.orchestra.run.smithi102.stdout: "osdmap_items": 173, 2024-09-10T10:05:03.213 INFO:teuthology.orchestra.run.smithi102.stdout: "osdmap_mapping_bytes": 0, 2024-09-10T10:05:03.213 INFO:teuthology.orchestra.run.smithi102.stdout: "osdmap_mapping_items": 0, 2024-09-10T10:05:03.213 INFO:teuthology.orchestra.run.smithi102.stdout: "pgmap_bytes": 0, 2024-09-10T10:05:03.213 INFO:teuthology.orchestra.run.smithi102.stdout: "pgmap_items": 0, 2024-09-10T10:05:03.214 INFO:teuthology.orchestra.run.smithi102.stdout: "mds_co_bytes": 76904, 2024-09-10T10:05:03.214 INFO:teuthology.orchestra.run.smithi102.stdout: "mds_co_items": 362, 2024-09-10T10:05:03.214 INFO:teuthology.orchestra.run.smithi102.stdout: "unittest_1_bytes": 0, 2024-09-10T10:05:03.214 INFO:teuthology.orchestra.run.smithi102.stdout: "unittest_1_items": 0, 2024-09-10T10:05:03.214 INFO:teuthology.orchestra.run.smithi102.stdout: "unittest_2_bytes": 0, 2024-09-10T10:05:03.214 INFO:teuthology.orchestra.run.smithi102.stdout: "unittest_2_items": 0 2024-09-10T10:05:03.214 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-10T10:05:03.214 INFO:teuthology.orchestra.run.smithi102.stdout: "objecter": { 2024-09-10T10:05:03.214 INFO:teuthology.orchestra.run.smithi102.stdout: "op_active": 0, 2024-09-10T10:05:03.214 INFO:teuthology.orchestra.run.smithi102.stdout: "op_laggy": 0, 2024-09-10T10:05:03.214 INFO:teuthology.orchestra.run.smithi102.stdout: "op_send": 33609, 2024-09-10T10:05:03.214 INFO:teuthology.orchestra.run.smithi102.stdout: "op_send_bytes": 86041897, 2024-09-10T10:05:03.214 INFO:teuthology.orchestra.run.smithi102.stdout: "op_resend": 33, 2024-09-10T10:05:03.214 INFO:teuthology.orchestra.run.smithi102.stdout: "op_reply": 33609, 2024-09-10T10:05:03.214 INFO:teuthology.orchestra.run.smithi102.stdout: "op_latency": { 2024-09-10T10:05:03.215 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 33609, 2024-09-10T10:05:03.215 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 274.768093403, 2024-09-10T10:05:03.215 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.008175431 2024-09-10T10:05:03.215 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-10T10:05:03.215 INFO:teuthology.orchestra.run.smithi102.stdout: "op_inflight": 0, 2024-09-10T10:05:03.215 INFO:teuthology.orchestra.run.smithi102.stdout: "oplen_avg": { 2024-09-10T10:05:03.215 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 33609, 2024-09-10T10:05:03.215 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 34238 2024-09-10T10:05:03.215 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-10T10:05:03.215 INFO:teuthology.orchestra.run.smithi102.stdout: "op": 33609, 2024-09-10T10:05:03.215 INFO:teuthology.orchestra.run.smithi102.stdout: "op_r": 1367, 2024-09-10T10:05:03.215 INFO:teuthology.orchestra.run.smithi102.stdout: "op_w": 32242, 2024-09-10T10:05:03.215 INFO:teuthology.orchestra.run.smithi102.stdout: "op_rmw": 0, 2024-09-10T10:05:03.215 INFO:teuthology.orchestra.run.smithi102.stdout: "op_pg": 0, 2024-09-10T10:05:03.216 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_stat": 0, 2024-09-10T10:05:03.216 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_create": 10, 2024-09-10T10:05:03.216 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_read": 384, 2024-09-10T10:05:03.216 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_write": 16700, 2024-09-10T10:05:03.216 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_writefull": 254, 2024-09-10T10:05:03.216 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_writesame": 0, 2024-09-10T10:05:03.216 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_append": 0, 2024-09-10T10:05:03.216 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_zero": 0, 2024-09-10T10:05:03.216 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_truncate": 0, 2024-09-10T10:05:03.216 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_delete": 14825, 2024-09-10T10:05:03.216 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_mapext": 0, 2024-09-10T10:05:03.216 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_sparse_read": 0, 2024-09-10T10:05:03.216 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_clonerange": 0, 2024-09-10T10:05:03.217 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_getxattr": 983, 2024-09-10T10:05:03.217 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_setxattr": 20, 2024-09-10T10:05:03.217 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_cmpxattr": 0, 2024-09-10T10:05:03.217 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_rmxattr": 0, 2024-09-10T10:05:03.217 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_resetxattrs": 0, 2024-09-10T10:05:03.217 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_call": 0, 2024-09-10T10:05:03.217 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_watch": 0, 2024-09-10T10:05:03.217 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_notify": 0, 2024-09-10T10:05:03.217 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_src_cmpxattr": 0, 2024-09-10T10:05:03.217 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_pgls": 0, 2024-09-10T10:05:03.217 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_pgls_filter": 0, 2024-09-10T10:05:03.217 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_other": 0, 2024-09-10T10:05:03.217 INFO:teuthology.orchestra.run.smithi102.stdout: "linger_active": 0, 2024-09-10T10:05:03.217 INFO:teuthology.orchestra.run.smithi102.stdout: "linger_send": 0, 2024-09-10T10:05:03.217 INFO:teuthology.orchestra.run.smithi102.stdout: "linger_resend": 0, 2024-09-10T10:05:03.218 INFO:teuthology.orchestra.run.smithi102.stdout: "linger_ping": 0, 2024-09-10T10:05:03.218 INFO:teuthology.orchestra.run.smithi102.stdout: "poolop_active": 0, 2024-09-10T10:05:03.218 INFO:teuthology.orchestra.run.smithi102.stdout: "poolop_send": 0, 2024-09-10T10:05:03.218 INFO:teuthology.orchestra.run.smithi102.stdout: "poolop_resend": 0, 2024-09-10T10:05:03.218 INFO:teuthology.orchestra.run.smithi102.stdout: "poolstat_active": 0, 2024-09-10T10:05:03.218 INFO:teuthology.orchestra.run.smithi102.stdout: "poolstat_send": 0, 2024-09-10T10:05:03.218 INFO:teuthology.orchestra.run.smithi102.stdout: "poolstat_resend": 0, 2024-09-10T10:05:03.218 INFO:teuthology.orchestra.run.smithi102.stdout: "statfs_active": 0, 2024-09-10T10:05:03.218 INFO:teuthology.orchestra.run.smithi102.stdout: "statfs_send": 0, 2024-09-10T10:05:03.218 INFO:teuthology.orchestra.run.smithi102.stdout: "statfs_resend": 0, 2024-09-10T10:05:03.218 INFO:teuthology.orchestra.run.smithi102.stdout: "command_active": 0, 2024-09-10T10:05:03.218 INFO:teuthology.orchestra.run.smithi102.stdout: "command_send": 0, 2024-09-10T10:05:03.218 INFO:teuthology.orchestra.run.smithi102.stdout: "command_resend": 0, 2024-09-10T10:05:03.218 INFO:teuthology.orchestra.run.smithi102.stdout: "map_epoch": 96, 2024-09-10T10:05:03.218 INFO:teuthology.orchestra.run.smithi102.stdout: "map_full": 0, 2024-09-10T10:05:03.219 INFO:teuthology.orchestra.run.smithi102.stdout: "map_inc": 2, 2024-09-10T10:05:03.219 INFO:teuthology.orchestra.run.smithi102.stdout: "osd_sessions": 12, 2024-09-10T10:05:03.219 INFO:teuthology.orchestra.run.smithi102.stdout: "osd_session_open": 15, 2024-09-10T10:05:03.219 INFO:teuthology.orchestra.run.smithi102.stdout: "osd_session_close": 3, 2024-09-10T10:05:03.219 INFO:teuthology.orchestra.run.smithi102.stdout: "osd_laggy": 0, 2024-09-10T10:05:03.219 INFO:teuthology.orchestra.run.smithi102.stdout: "omap_wr": 793, 2024-09-10T10:05:03.219 INFO:teuthology.orchestra.run.smithi102.stdout: "omap_rd": 0, 2024-09-10T10:05:03.219 INFO:teuthology.orchestra.run.smithi102.stdout: "omap_del": 269 2024-09-10T10:05:03.219 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-10T10:05:03.219 INFO:teuthology.orchestra.run.smithi102.stdout: "oft": { 2024-09-10T10:05:03.219 INFO:teuthology.orchestra.run.smithi102.stdout: "omap_total_objs": 1, 2024-09-10T10:05:03.219 INFO:teuthology.orchestra.run.smithi102.stdout: "omap_total_kv_pairs": 0, 2024-09-10T10:05:03.219 INFO:teuthology.orchestra.run.smithi102.stdout: "omap_total_updates": 0, 2024-09-10T10:05:03.219 INFO:teuthology.orchestra.run.smithi102.stdout: "omap_total_removes": 0 2024-09-10T10:05:03.220 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-10T10:05:03.220 INFO:teuthology.orchestra.run.smithi102.stdout: "osd-slow-ops": { 2024-09-10T10:05:03.220 INFO:teuthology.orchestra.run.smithi102.stdout: "slow_ops_count": 1362 2024-09-10T10:05:03.220 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-10T10:05:03.220 INFO:teuthology.orchestra.run.smithi102.stdout: "purge_queue": { 2024-09-10T10:05:03.220 INFO:teuthology.orchestra.run.smithi102.stdout: "pq_executing_ops": 0, 2024-09-10T10:05:03.220 INFO:teuthology.orchestra.run.smithi102.stdout: "pq_executing_ops_high_water": 10, 2024-09-10T10:05:03.220 INFO:teuthology.orchestra.run.smithi102.stdout: "pq_executing": 0, 2024-09-10T10:05:03.220 INFO:teuthology.orchestra.run.smithi102.stdout: "pq_executing_high_water": 5, 2024-09-10T10:05:03.220 INFO:teuthology.orchestra.run.smithi102.stdout: "pq_executed_ops": 14793, 2024-09-10T10:05:03.220 INFO:teuthology.orchestra.run.smithi102.stdout: "pq_executed": 7401, 2024-09-10T10:05:03.220 INFO:teuthology.orchestra.run.smithi102.stdout: "pq_item_in_journal": 0 2024-09-10T10:05:03.220 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-10T10:05:03.220 INFO:teuthology.orchestra.run.smithi102.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-10T10:05:03.221 INFO:teuthology.orchestra.run.smithi102.stdout: "val": 0, 2024-09-10T10:05:03.221 INFO:teuthology.orchestra.run.smithi102.stdout: "max": 104857600, 2024-09-10T10:05:03.221 INFO:teuthology.orchestra.run.smithi102.stdout: "get_started": 0, 2024-09-10T10:05:03.221 INFO:teuthology.orchestra.run.smithi102.stdout: "get": 286704, 2024-09-10T10:05:03.221 INFO:teuthology.orchestra.run.smithi102.stdout: "get_sum": 55265394, 2024-09-10T10:05:03.221 INFO:teuthology.orchestra.run.smithi102.stdout: "get_or_fail_fail": 0, 2024-09-10T10:05:03.221 INFO:teuthology.orchestra.run.smithi102.stdout: "get_or_fail_success": 286704, 2024-09-10T10:05:03.221 INFO:teuthology.orchestra.run.smithi102.stdout: "take": 0, 2024-09-10T10:05:03.221 INFO:teuthology.orchestra.run.smithi102.stdout: "take_sum": 0, 2024-09-10T10:05:03.221 INFO:teuthology.orchestra.run.smithi102.stdout: "put": 286704, 2024-09-10T10:05:03.221 INFO:teuthology.orchestra.run.smithi102.stdout: "put_sum": 55265394, 2024-09-10T10:05:03.221 INFO:teuthology.orchestra.run.smithi102.stdout: "wait": { 2024-09-10T10:05:03.221 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-10T10:05:03.222 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-10T10:05:03.222 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-10T10:05:03.222 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-10T10:05:03.222 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-10T10:05:03.222 INFO:teuthology.orchestra.run.smithi102.stdout: "throttle-objecter_bytes": { 2024-09-10T10:05:03.222 INFO:teuthology.orchestra.run.smithi102.stdout: "val": 0, 2024-09-10T10:05:03.222 INFO:teuthology.orchestra.run.smithi102.stdout: "max": 104857600, 2024-09-10T10:05:03.222 INFO:teuthology.orchestra.run.smithi102.stdout: "get_started": 0, 2024-09-10T10:05:03.222 INFO:teuthology.orchestra.run.smithi102.stdout: "get": 0, 2024-09-10T10:05:03.222 INFO:teuthology.orchestra.run.smithi102.stdout: "get_sum": 0, 2024-09-10T10:05:03.222 INFO:teuthology.orchestra.run.smithi102.stdout: "get_or_fail_fail": 0, 2024-09-10T10:05:03.222 INFO:teuthology.orchestra.run.smithi102.stdout: "get_or_fail_success": 0, 2024-09-10T10:05:03.222 INFO:teuthology.orchestra.run.smithi102.stdout: "take": 33609, 2024-09-10T10:05:03.222 INFO:teuthology.orchestra.run.smithi102.stdout: "take_sum": 86848534, 2024-09-10T10:05:03.222 INFO:teuthology.orchestra.run.smithi102.stdout: "put": 33609, 2024-09-10T10:05:03.223 INFO:teuthology.orchestra.run.smithi102.stdout: "put_sum": 86848534, 2024-09-10T10:05:03.223 INFO:teuthology.orchestra.run.smithi102.stdout: "wait": { 2024-09-10T10:05:03.223 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-10T10:05:03.223 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-10T10:05:03.223 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-10T10:05:03.223 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-10T10:05:03.223 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-10T10:05:03.223 INFO:teuthology.orchestra.run.smithi102.stdout: "throttle-objecter_ops": { 2024-09-10T10:05:03.223 INFO:teuthology.orchestra.run.smithi102.stdout: "val": 0, 2024-09-10T10:05:03.223 INFO:teuthology.orchestra.run.smithi102.stdout: "max": 1024, 2024-09-10T10:05:03.223 INFO:teuthology.orchestra.run.smithi102.stdout: "get_started": 0, 2024-09-10T10:05:03.223 INFO:teuthology.orchestra.run.smithi102.stdout: "get": 0, 2024-09-10T10:05:03.223 INFO:teuthology.orchestra.run.smithi102.stdout: "get_sum": 0, 2024-09-10T10:05:03.223 INFO:teuthology.orchestra.run.smithi102.stdout: "get_or_fail_fail": 0, 2024-09-10T10:05:03.224 INFO:teuthology.orchestra.run.smithi102.stdout: "get_or_fail_success": 0, 2024-09-10T10:05:03.224 INFO:teuthology.orchestra.run.smithi102.stdout: "take": 33609, 2024-09-10T10:05:03.224 INFO:teuthology.orchestra.run.smithi102.stdout: "take_sum": 33609, 2024-09-10T10:05:03.224 INFO:teuthology.orchestra.run.smithi102.stdout: "put": 33609, 2024-09-10T10:05:03.224 INFO:teuthology.orchestra.run.smithi102.stdout: "put_sum": 33609, 2024-09-10T10:05:03.224 INFO:teuthology.orchestra.run.smithi102.stdout: "wait": { 2024-09-10T10:05:03.224 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-10T10:05:03.224 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-10T10:05:03.224 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-10T10:05:03.224 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-10T10:05:03.224 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-10T10:05:03.224 INFO:teuthology.orchestra.run.smithi102.stdout: "throttle-write_buf_throttle": { 2024-09-10T10:05:03.224 INFO:teuthology.orchestra.run.smithi102.stdout: "val": 0, 2024-09-10T10:05:03.224 INFO:teuthology.orchestra.run.smithi102.stdout: "max": 3758096384, 2024-09-10T10:05:03.225 INFO:teuthology.orchestra.run.smithi102.stdout: "get_started": 0, 2024-09-10T10:05:03.225 INFO:teuthology.orchestra.run.smithi102.stdout: "get": 7401, 2024-09-10T10:05:03.225 INFO:teuthology.orchestra.run.smithi102.stdout: "get_sum": 806637, 2024-09-10T10:05:03.225 INFO:teuthology.orchestra.run.smithi102.stdout: "get_or_fail_fail": 0, 2024-09-10T10:05:03.225 INFO:teuthology.orchestra.run.smithi102.stdout: "get_or_fail_success": 7401, 2024-09-10T10:05:03.225 INFO:teuthology.orchestra.run.smithi102.stdout: "take": 0, 2024-09-10T10:05:03.225 INFO:teuthology.orchestra.run.smithi102.stdout: "take_sum": 0, 2024-09-10T10:05:03.225 INFO:teuthology.orchestra.run.smithi102.stdout: "put": 384, 2024-09-10T10:05:03.225 INFO:teuthology.orchestra.run.smithi102.stdout: "put_sum": 806637, 2024-09-10T10:05:03.225 INFO:teuthology.orchestra.run.smithi102.stdout: "wait": { 2024-09-10T10:05:03.225 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-10T10:05:03.225 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-10T10:05:03.226 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-10T10:05:03.226 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-10T10:05:03.226 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-10T10:05:03.226 INFO:teuthology.orchestra.run.smithi102.stdout: "throttle-write_buf_throttle-0x555dbbeee960": { 2024-09-10T10:05:03.226 INFO:teuthology.orchestra.run.smithi102.stdout: "val": 0, 2024-09-10T10:05:03.226 INFO:teuthology.orchestra.run.smithi102.stdout: "max": 3758096384, 2024-09-10T10:05:03.226 INFO:teuthology.orchestra.run.smithi102.stdout: "get_started": 0, 2024-09-10T10:05:03.226 INFO:teuthology.orchestra.run.smithi102.stdout: "get": 43840, 2024-09-10T10:05:03.226 INFO:teuthology.orchestra.run.smithi102.stdout: "get_sum": 84315542, 2024-09-10T10:05:03.226 INFO:teuthology.orchestra.run.smithi102.stdout: "get_or_fail_fail": 0, 2024-09-10T10:05:03.226 INFO:teuthology.orchestra.run.smithi102.stdout: "get_or_fail_success": 43840, 2024-09-10T10:05:03.226 INFO:teuthology.orchestra.run.smithi102.stdout: "take": 0, 2024-09-10T10:05:03.226 INFO:teuthology.orchestra.run.smithi102.stdout: "take_sum": 0, 2024-09-10T10:05:03.226 INFO:teuthology.orchestra.run.smithi102.stdout: "put": 16316, 2024-09-10T10:05:03.227 INFO:teuthology.orchestra.run.smithi102.stdout: "put_sum": 84315542, 2024-09-10T10:05:03.227 INFO:teuthology.orchestra.run.smithi102.stdout: "wait": { 2024-09-10T10:05:03.227 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-10T10:05:03.227 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-10T10:05:03.227 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-10T10:05:03.227 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-10T10:05:03.227 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-10T10:05:03.227 INFO:teuthology.orchestra.run.smithi102.stdout:} 2024-09-10T10:05:04.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:05:03 smithi159 ceph-mon[35513]: pgmap v1194: 129 pgs: 129 active+clean; 551 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-10T10:05:04.112 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:05:03 smithi102 ceph-mon[35442]: pgmap v1194: 129 pgs: 129 active+clean; 551 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-10T10:05:04.141 INFO:tasks.check_counter:Daemon mds.h mds.exported=4 2024-09-10T10:05:04.141 INFO:tasks.check_counter:Daemon mds.h mds.imported=13 2024-09-10T10:05:04.141 DEBUG:tasks.check_counter:Getting stats from k 2024-09-10T10:05:04.142 DEBUG:teuthology.orchestra.run.smithi102:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph daemon mds.k perf dump 2024-09-10T10:05:04.306 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:05:03 smithi005 ceph-mon[30350]: pgmap v1194: 129 pgs: 129 active+clean; 551 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-10T10:05:04.385 INFO:teuthology.orchestra.run.smithi102.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T10:05:04.927 INFO:teuthology.orchestra.run.smithi102.stdout:{ 2024-09-10T10:05:04.928 INFO:teuthology.orchestra.run.smithi102.stdout: "AsyncMessenger::Worker-0": { 2024-09-10T10:05:04.928 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_recv_messages": 39452, 2024-09-10T10:05:04.928 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_messages": 32842, 2024-09-10T10:05:04.928 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_recv_bytes": 10965193, 2024-09-10T10:05:04.928 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_bytes": 13611268, 2024-09-10T10:05:04.928 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_created_connections": 38, 2024-09-10T10:05:04.928 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_active_connections": 5, 2024-09-10T10:05:04.928 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_total_time": 2.567209316, 2024-09-10T10:05:04.928 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_send_time": 1.122450672, 2024-09-10T10:05:04.928 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_recv_time": 1.677458757, 2024-09-10T10:05:04.928 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_fast_dispatch_time": 0.102239753, 2024-09-10T10:05:04.928 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T10:05:04.929 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 32840, 2024-09-10T10:05:04.929 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.816732153, 2024-09-10T10:05:04.929 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000024870 2024-09-10T10:05:04.929 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-10T10:05:04.929 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_handle_ack_lat": { 2024-09-10T10:05:04.929 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 65467, 2024-09-10T10:05:04.929 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.011178782, 2024-09-10T10:05:04.929 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000170 2024-09-10T10:05:04.929 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-10T10:05:04.929 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-10T10:05:04.929 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_encrypted_bytes": 0 2024-09-10T10:05:04.929 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-10T10:05:04.929 INFO:teuthology.orchestra.run.smithi102.stdout: "AsyncMessenger::Worker-1": { 2024-09-10T10:05:04.929 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_recv_messages": 25179, 2024-09-10T10:05:04.930 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_messages": 35653, 2024-09-10T10:05:04.930 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_recv_bytes": 8590524, 2024-09-10T10:05:04.930 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_bytes": 22592566, 2024-09-10T10:05:04.930 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_created_connections": 15, 2024-09-10T10:05:04.930 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_active_connections": 6, 2024-09-10T10:05:04.930 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_total_time": 1.420983394, 2024-09-10T10:05:04.930 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_send_time": 0.665501460, 2024-09-10T10:05:04.930 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_recv_time": 0.869704042, 2024-09-10T10:05:04.930 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_fast_dispatch_time": 0.050473738, 2024-09-10T10:05:04.930 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T10:05:04.930 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 35653, 2024-09-10T10:05:04.930 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 1.029835278, 2024-09-10T10:05:04.930 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000028884 2024-09-10T10:05:04.930 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-10T10:05:04.931 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_handle_ack_lat": { 2024-09-10T10:05:04.931 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 34843, 2024-09-10T10:05:04.931 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.008377877, 2024-09-10T10:05:04.931 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000240 2024-09-10T10:05:04.931 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-10T10:05:04.931 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_recv_encrypted_bytes": 1002048, 2024-09-10T10:05:04.931 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_encrypted_bytes": 316208 2024-09-10T10:05:04.931 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-10T10:05:04.931 INFO:teuthology.orchestra.run.smithi102.stdout: "AsyncMessenger::Worker-2": { 2024-09-10T10:05:04.931 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_recv_messages": 88562, 2024-09-10T10:05:04.931 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_messages": 46159, 2024-09-10T10:05:04.931 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_recv_bytes": 23922811, 2024-09-10T10:05:04.931 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_bytes": 18616904, 2024-09-10T10:05:04.931 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_created_connections": 44, 2024-09-10T10:05:04.932 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_active_connections": 7, 2024-09-10T10:05:04.932 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_total_time": 3.861036111, 2024-09-10T10:05:04.932 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_send_time": 1.413827319, 2024-09-10T10:05:04.932 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_recv_time": 3.133061238, 2024-09-10T10:05:04.932 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_running_fast_dispatch_time": 0.116356333, 2024-09-10T10:05:04.932 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T10:05:04.932 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 46157, 2024-09-10T10:05:04.932 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.884436642, 2024-09-10T10:05:04.932 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000019161 2024-09-10T10:05:04.932 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-10T10:05:04.932 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_handle_ack_lat": { 2024-09-10T10:05:04.932 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 126042, 2024-09-10T10:05:04.932 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.017642882, 2024-09-10T10:05:04.932 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000139 2024-09-10T10:05:04.933 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-10T10:05:04.933 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_recv_encrypted_bytes": 480, 2024-09-10T10:05:04.933 INFO:teuthology.orchestra.run.smithi102.stdout: "msgr_send_encrypted_bytes": 889200 2024-09-10T10:05:04.933 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-10T10:05:04.933 INFO:teuthology.orchestra.run.smithi102.stdout: "cct": { 2024-09-10T10:05:04.933 INFO:teuthology.orchestra.run.smithi102.stdout: "total_workers": 1, 2024-09-10T10:05:04.933 INFO:teuthology.orchestra.run.smithi102.stdout: "unhealthy_workers": 0 2024-09-10T10:05:04.933 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-10T10:05:04.933 INFO:teuthology.orchestra.run.smithi102.stdout: "finisher-MDSRank": { 2024-09-10T10:05:04.933 INFO:teuthology.orchestra.run.smithi102.stdout: "queue_len": 0, 2024-09-10T10:05:04.933 INFO:teuthology.orchestra.run.smithi102.stdout: "complete_latency": { 2024-09-10T10:05:04.933 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 6329, 2024-09-10T10:05:04.933 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 3.694356508, 2024-09-10T10:05:04.933 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000583718 2024-09-10T10:05:04.933 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-10T10:05:04.934 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-10T10:05:04.934 INFO:teuthology.orchestra.run.smithi102.stdout: "finisher-PurgeQueue": { 2024-09-10T10:05:04.934 INFO:teuthology.orchestra.run.smithi102.stdout: "queue_len": 0, 2024-09-10T10:05:04.934 INFO:teuthology.orchestra.run.smithi102.stdout: "complete_latency": { 2024-09-10T10:05:04.934 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 5653, 2024-09-10T10:05:04.934 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.307454667, 2024-09-10T10:05:04.934 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000054387 2024-09-10T10:05:04.934 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-10T10:05:04.934 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-10T10:05:04.934 INFO:teuthology.orchestra.run.smithi102.stdout: "mds": { 2024-09-10T10:05:04.934 INFO:teuthology.orchestra.run.smithi102.stdout: "request": 14860, 2024-09-10T10:05:04.935 INFO:teuthology.orchestra.run.smithi102.stdout: "reply": 14617, 2024-09-10T10:05:04.935 INFO:teuthology.orchestra.run.smithi102.stdout: "reply_latency": { 2024-09-10T10:05:04.935 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 14617, 2024-09-10T10:05:04.935 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 38.910234739, 2024-09-10T10:05:04.935 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.002661984 2024-09-10T10:05:04.935 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-10T10:05:04.935 INFO:teuthology.orchestra.run.smithi102.stdout: "slow_reply": 0, 2024-09-10T10:05:04.935 INFO:teuthology.orchestra.run.smithi102.stdout: "forward": 243, 2024-09-10T10:05:04.935 INFO:teuthology.orchestra.run.smithi102.stdout: "dir_fetch_complete": 0, 2024-09-10T10:05:04.935 INFO:teuthology.orchestra.run.smithi102.stdout: "dir_fetch_keys": 0, 2024-09-10T10:05:04.935 INFO:teuthology.orchestra.run.smithi102.stdout: "dir_commit": 11, 2024-09-10T10:05:04.935 INFO:teuthology.orchestra.run.smithi102.stdout: "dir_split": 0, 2024-09-10T10:05:04.935 INFO:teuthology.orchestra.run.smithi102.stdout: "dir_merge": 0, 2024-09-10T10:05:04.936 INFO:teuthology.orchestra.run.smithi102.stdout: "inodes": 19, 2024-09-10T10:05:04.936 INFO:teuthology.orchestra.run.smithi102.stdout: "inodes_top": 2, 2024-09-10T10:05:04.936 INFO:teuthology.orchestra.run.smithi102.stdout: "inodes_bottom": 17, 2024-09-10T10:05:04.936 INFO:teuthology.orchestra.run.smithi102.stdout: "inodes_pin_tail": 0, 2024-09-10T10:05:04.936 INFO:teuthology.orchestra.run.smithi102.stdout: "inodes_pinned": 14, 2024-09-10T10:05:04.936 INFO:teuthology.orchestra.run.smithi102.stdout: "inodes_expired": 76, 2024-09-10T10:05:04.936 INFO:teuthology.orchestra.run.smithi102.stdout: "inodes_with_caps": 2, 2024-09-10T10:05:04.936 INFO:teuthology.orchestra.run.smithi102.stdout: "caps": 2, 2024-09-10T10:05:04.936 INFO:teuthology.orchestra.run.smithi102.stdout: "subtrees": 6, 2024-09-10T10:05:04.936 INFO:teuthology.orchestra.run.smithi102.stdout: "traverse": 30794, 2024-09-10T10:05:04.936 INFO:teuthology.orchestra.run.smithi102.stdout: "traverse_hit": 11476, 2024-09-10T10:05:04.936 INFO:teuthology.orchestra.run.smithi102.stdout: "traverse_forward": 241, 2024-09-10T10:05:04.936 INFO:teuthology.orchestra.run.smithi102.stdout: "traverse_discover": 21, 2024-09-10T10:05:04.937 INFO:teuthology.orchestra.run.smithi102.stdout: "traverse_dir_fetch": 0, 2024-09-10T10:05:04.937 INFO:teuthology.orchestra.run.smithi102.stdout: "traverse_remote_ino": 0, 2024-09-10T10:05:04.937 INFO:teuthology.orchestra.run.smithi102.stdout: "traverse_lock": 0, 2024-09-10T10:05:04.937 INFO:teuthology.orchestra.run.smithi102.stdout: "load_cent": 0, 2024-09-10T10:05:04.937 INFO:teuthology.orchestra.run.smithi102.stdout: "q": 0, 2024-09-10T10:05:04.937 INFO:teuthology.orchestra.run.smithi102.stdout: "exported": 1, 2024-09-10T10:05:04.937 INFO:teuthology.orchestra.run.smithi102.stdout: "exported_inodes": 0, 2024-09-10T10:05:04.937 INFO:teuthology.orchestra.run.smithi102.stdout: "imported": 1, 2024-09-10T10:05:04.937 INFO:teuthology.orchestra.run.smithi102.stdout: "imported_inodes": 3, 2024-09-10T10:05:04.937 INFO:teuthology.orchestra.run.smithi102.stdout: "openino_dir_fetch": 0, 2024-09-10T10:05:04.937 INFO:teuthology.orchestra.run.smithi102.stdout: "openino_backtrace_fetch": 38, 2024-09-10T10:05:04.937 INFO:teuthology.orchestra.run.smithi102.stdout: "openino_peer_discover": 0, 2024-09-10T10:05:04.937 INFO:teuthology.orchestra.run.smithi102.stdout: "root_rfiles": 733, 2024-09-10T10:05:04.938 INFO:teuthology.orchestra.run.smithi102.stdout: "root_rbytes": 226265082, 2024-09-10T10:05:04.938 INFO:teuthology.orchestra.run.smithi102.stdout: "root_rsnaps": 0, 2024-09-10T10:05:04.938 INFO:teuthology.orchestra.run.smithi102.stdout: "scrub_backtrace_fetch": 38, 2024-09-10T10:05:04.938 INFO:teuthology.orchestra.run.smithi102.stdout: "scrub_set_tag": 0, 2024-09-10T10:05:04.938 INFO:teuthology.orchestra.run.smithi102.stdout: "scrub_backtrace_repaired": 0, 2024-09-10T10:05:04.938 INFO:teuthology.orchestra.run.smithi102.stdout: "scrub_inotable_repaired": 0, 2024-09-10T10:05:04.938 INFO:teuthology.orchestra.run.smithi102.stdout: "scrub_dir_inodes": 0, 2024-09-10T10:05:04.938 INFO:teuthology.orchestra.run.smithi102.stdout: "scrub_dir_base_inodes": 0, 2024-09-10T10:05:04.938 INFO:teuthology.orchestra.run.smithi102.stdout: "scrub_dirfrag_rstats": 0, 2024-09-10T10:05:04.938 INFO:teuthology.orchestra.run.smithi102.stdout: "scrub_file_inodes": 38, 2024-09-10T10:05:04.938 INFO:teuthology.orchestra.run.smithi102.stdout: "handle_inode_file_caps": 0, 2024-09-10T10:05:04.938 INFO:teuthology.orchestra.run.smithi102.stdout: "ceph_cap_op_revoke": 164, 2024-09-10T10:05:04.938 INFO:teuthology.orchestra.run.smithi102.stdout: "ceph_cap_op_grant": 8334, 2024-09-10T10:05:04.938 INFO:teuthology.orchestra.run.smithi102.stdout: "ceph_cap_op_trunc": 0, 2024-09-10T10:05:04.939 INFO:teuthology.orchestra.run.smithi102.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-10T10:05:04.940 INFO:teuthology.orchestra.run.smithi102.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-10T10:05:04.940 INFO:teuthology.orchestra.run.smithi102.stdout: "handle_client_caps": 630, 2024-09-10T10:05:04.940 INFO:teuthology.orchestra.run.smithi102.stdout: "handle_client_caps_dirty": 74, 2024-09-10T10:05:04.940 INFO:teuthology.orchestra.run.smithi102.stdout: "handle_client_cap_release": 155, 2024-09-10T10:05:04.940 INFO:teuthology.orchestra.run.smithi102.stdout: "process_request_cap_release": 13778 2024-09-10T10:05:04.940 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-10T10:05:04.940 INFO:teuthology.orchestra.run.smithi102.stdout: "mds_cache": { 2024-09-10T10:05:04.941 INFO:teuthology.orchestra.run.smithi102.stdout: "dir_update": 3, 2024-09-10T10:05:04.941 INFO:teuthology.orchestra.run.smithi102.stdout: "dir_update_receipt": 224, 2024-09-10T10:05:04.941 INFO:teuthology.orchestra.run.smithi102.stdout: "dir_try_discover": 121, 2024-09-10T10:05:04.941 INFO:teuthology.orchestra.run.smithi102.stdout: "dir_send_discover": 217, 2024-09-10T10:05:04.941 INFO:teuthology.orchestra.run.smithi102.stdout: "dir_handle_discover": 5, 2024-09-10T10:05:04.941 INFO:teuthology.orchestra.run.smithi102.stdout: "num_strays": 0, 2024-09-10T10:05:04.941 INFO:teuthology.orchestra.run.smithi102.stdout: "num_strays_delayed": 0, 2024-09-10T10:05:04.941 INFO:teuthology.orchestra.run.smithi102.stdout: "num_strays_enqueuing": 0, 2024-09-10T10:05:04.941 INFO:teuthology.orchestra.run.smithi102.stdout: "strays_created": 2630, 2024-09-10T10:05:04.941 INFO:teuthology.orchestra.run.smithi102.stdout: "strays_enqueued": 2630, 2024-09-10T10:05:04.942 INFO:teuthology.orchestra.run.smithi102.stdout: "strays_reintegrated": 0, 2024-09-10T10:05:04.942 INFO:teuthology.orchestra.run.smithi102.stdout: "strays_migrated": 0, 2024-09-10T10:05:04.942 INFO:teuthology.orchestra.run.smithi102.stdout: "num_recovering_processing": 0, 2024-09-10T10:05:04.942 INFO:teuthology.orchestra.run.smithi102.stdout: "num_recovering_enqueued": 0, 2024-09-10T10:05:04.942 INFO:teuthology.orchestra.run.smithi102.stdout: "num_recovering_prioritized": 0, 2024-09-10T10:05:04.942 INFO:teuthology.orchestra.run.smithi102.stdout: "recovery_started": 0, 2024-09-10T10:05:04.942 INFO:teuthology.orchestra.run.smithi102.stdout: "recovery_completed": 0, 2024-09-10T10:05:04.942 INFO:teuthology.orchestra.run.smithi102.stdout: "ireq_quiesce_path": 23, 2024-09-10T10:05:04.942 INFO:teuthology.orchestra.run.smithi102.stdout: "ireq_quiesce_inode": 582, 2024-09-10T10:05:04.942 INFO:teuthology.orchestra.run.smithi102.stdout: "ireq_enqueue_scrub": 0, 2024-09-10T10:05:04.942 INFO:teuthology.orchestra.run.smithi102.stdout: "ireq_exportdir": 1, 2024-09-10T10:05:04.942 INFO:teuthology.orchestra.run.smithi102.stdout: "ireq_flush": 0, 2024-09-10T10:05:04.942 INFO:teuthology.orchestra.run.smithi102.stdout: "ireq_fragmentdir": 0, 2024-09-10T10:05:04.943 INFO:teuthology.orchestra.run.smithi102.stdout: "ireq_fragstats": 0, 2024-09-10T10:05:04.943 INFO:teuthology.orchestra.run.smithi102.stdout: "ireq_inodestats": 0 2024-09-10T10:05:04.943 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-10T10:05:04.943 INFO:teuthology.orchestra.run.smithi102.stdout: "mds_log": { 2024-09-10T10:05:04.943 INFO:teuthology.orchestra.run.smithi102.stdout: "evlrg": 0, 2024-09-10T10:05:04.943 INFO:teuthology.orchestra.run.smithi102.stdout: "evadd": 11025, 2024-09-10T10:05:04.943 INFO:teuthology.orchestra.run.smithi102.stdout: "evex": 0, 2024-09-10T10:05:04.943 INFO:teuthology.orchestra.run.smithi102.stdout: "evtrm": 0, 2024-09-10T10:05:04.943 INFO:teuthology.orchestra.run.smithi102.stdout: "ev": 11025, 2024-09-10T10:05:04.943 INFO:teuthology.orchestra.run.smithi102.stdout: "evexg": 0, 2024-09-10T10:05:04.943 INFO:teuthology.orchestra.run.smithi102.stdout: "evexd": 0, 2024-09-10T10:05:04.943 INFO:teuthology.orchestra.run.smithi102.stdout: "segadd": 14, 2024-09-10T10:05:04.943 INFO:teuthology.orchestra.run.smithi102.stdout: "segex": 0, 2024-09-10T10:05:04.943 INFO:teuthology.orchestra.run.smithi102.stdout: "segtrm": 0, 2024-09-10T10:05:04.944 INFO:teuthology.orchestra.run.smithi102.stdout: "seg": 14, 2024-09-10T10:05:04.944 INFO:teuthology.orchestra.run.smithi102.stdout: "segmjr": 2, 2024-09-10T10:05:04.944 INFO:teuthology.orchestra.run.smithi102.stdout: "segexg": 0, 2024-09-10T10:05:04.944 INFO:teuthology.orchestra.run.smithi102.stdout: "segexd": 0, 2024-09-10T10:05:04.944 INFO:teuthology.orchestra.run.smithi102.stdout: "expos": 4194304, 2024-09-10T10:05:04.944 INFO:teuthology.orchestra.run.smithi102.stdout: "wrpos": 17793226, 2024-09-10T10:05:04.944 INFO:teuthology.orchestra.run.smithi102.stdout: "rdpos": 0, 2024-09-10T10:05:04.944 INFO:teuthology.orchestra.run.smithi102.stdout: "jlat": { 2024-09-10T10:05:04.944 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 2984, 2024-09-10T10:05:04.944 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 14.588926694, 2024-09-10T10:05:04.944 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.004889050 2024-09-10T10:05:04.944 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-10T10:05:04.944 INFO:teuthology.orchestra.run.smithi102.stdout: "replayed": 0 2024-09-10T10:05:04.944 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-10T10:05:04.945 INFO:teuthology.orchestra.run.smithi102.stdout: "mds_mem": { 2024-09-10T10:05:04.945 INFO:teuthology.orchestra.run.smithi102.stdout: "ino": 26, 2024-09-10T10:05:04.945 INFO:teuthology.orchestra.run.smithi102.stdout: "ino+": 2694, 2024-09-10T10:05:04.945 INFO:teuthology.orchestra.run.smithi102.stdout: "ino-": 2668, 2024-09-10T10:05:04.945 INFO:teuthology.orchestra.run.smithi102.stdout: "dir": 33, 2024-09-10T10:05:04.945 INFO:teuthology.orchestra.run.smithi102.stdout: "dir+": 161, 2024-09-10T10:05:04.945 INFO:teuthology.orchestra.run.smithi102.stdout: "dir-": 128, 2024-09-10T10:05:04.945 INFO:teuthology.orchestra.run.smithi102.stdout: "dn": 19, 2024-09-10T10:05:04.945 INFO:teuthology.orchestra.run.smithi102.stdout: "dn+": 4395, 2024-09-10T10:05:04.945 INFO:teuthology.orchestra.run.smithi102.stdout: "dn-": 4376, 2024-09-10T10:05:04.945 INFO:teuthology.orchestra.run.smithi102.stdout: "cap": 2, 2024-09-10T10:05:04.945 INFO:teuthology.orchestra.run.smithi102.stdout: "cap+": 2716, 2024-09-10T10:05:04.945 INFO:teuthology.orchestra.run.smithi102.stdout: "cap-": 2714, 2024-09-10T10:05:04.945 INFO:teuthology.orchestra.run.smithi102.stdout: "rss": 62360, 2024-09-10T10:05:04.946 INFO:teuthology.orchestra.run.smithi102.stdout: "heap": 223520 2024-09-10T10:05:04.946 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-10T10:05:04.946 INFO:teuthology.orchestra.run.smithi102.stdout: "mds_server": { 2024-09-10T10:05:04.946 INFO:teuthology.orchestra.run.smithi102.stdout: "dispatch_client_request": 27170, 2024-09-10T10:05:04.946 INFO:teuthology.orchestra.run.smithi102.stdout: "dispatch_server_request": 8, 2024-09-10T10:05:04.946 INFO:teuthology.orchestra.run.smithi102.stdout: "handle_client_request": 14860, 2024-09-10T10:05:04.946 INFO:teuthology.orchestra.run.smithi102.stdout: "handle_client_session": 25, 2024-09-10T10:05:04.946 INFO:teuthology.orchestra.run.smithi102.stdout: "handle_peer_request": 5293, 2024-09-10T10:05:04.946 INFO:teuthology.orchestra.run.smithi102.stdout: "req_create_latency": { 2024-09-10T10:05:04.946 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 2627, 2024-09-10T10:05:04.946 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 31.744533970, 2024-09-10T10:05:04.946 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.012083948 2024-09-10T10:05:04.947 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-10T10:05:04.947 INFO:teuthology.orchestra.run.smithi102.stdout: "req_getattr_latency": { 2024-09-10T10:05:04.947 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 84, 2024-09-10T10:05:04.947 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.054341185, 2024-09-10T10:05:04.947 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000646918 2024-09-10T10:05:04.947 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-10T10:05:04.947 INFO:teuthology.orchestra.run.smithi102.stdout: "req_getfilelock_latency": { 2024-09-10T10:05:04.947 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-10T10:05:04.947 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-10T10:05:04.947 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-10T10:05:04.947 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-10T10:05:04.947 INFO:teuthology.orchestra.run.smithi102.stdout: "req_link_latency": { 2024-09-10T10:05:04.947 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-10T10:05:04.948 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-10T10:05:04.948 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-10T10:05:04.948 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-10T10:05:04.948 INFO:teuthology.orchestra.run.smithi102.stdout: "req_lookup_latency": { 2024-09-10T10:05:04.948 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 3386, 2024-09-10T10:05:04.948 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.712488602, 2024-09-10T10:05:04.948 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000210421 2024-09-10T10:05:04.948 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-10T10:05:04.948 INFO:teuthology.orchestra.run.smithi102.stdout: "req_lookuphash_latency": { 2024-09-10T10:05:04.948 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-10T10:05:04.948 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-10T10:05:04.948 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-10T10:05:04.948 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-10T10:05:04.948 INFO:teuthology.orchestra.run.smithi102.stdout: "req_lookupino_latency": { 2024-09-10T10:05:04.948 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 1, 2024-09-10T10:05:04.949 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000122847, 2024-09-10T10:05:04.949 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000122847 2024-09-10T10:05:04.949 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-10T10:05:04.949 INFO:teuthology.orchestra.run.smithi102.stdout: "req_lookupname_latency": { 2024-09-10T10:05:04.949 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-10T10:05:04.949 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-10T10:05:04.949 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-10T10:05:04.949 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-10T10:05:04.949 INFO:teuthology.orchestra.run.smithi102.stdout: "req_lookupparent_latency": { 2024-09-10T10:05:04.949 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-10T10:05:04.949 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-10T10:05:04.949 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-10T10:05:04.949 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-10T10:05:04.949 INFO:teuthology.orchestra.run.smithi102.stdout: "req_lookupsnap_latency": { 2024-09-10T10:05:04.950 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-10T10:05:04.950 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-10T10:05:04.950 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-10T10:05:04.950 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-10T10:05:04.950 INFO:teuthology.orchestra.run.smithi102.stdout: "req_lssnap_latency": { 2024-09-10T10:05:04.950 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-10T10:05:04.950 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-10T10:05:04.950 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-10T10:05:04.951 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-10T10:05:04.951 INFO:teuthology.orchestra.run.smithi102.stdout: "req_mkdir_latency": { 2024-09-10T10:05:04.951 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-10T10:05:04.951 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-10T10:05:04.951 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-10T10:05:04.951 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-10T10:05:04.951 INFO:teuthology.orchestra.run.smithi102.stdout: "req_mknod_latency": { 2024-09-10T10:05:04.951 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-10T10:05:04.951 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-10T10:05:04.951 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-10T10:05:04.951 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-10T10:05:04.951 INFO:teuthology.orchestra.run.smithi102.stdout: "req_mksnap_latency": { 2024-09-10T10:05:04.951 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-10T10:05:04.952 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-10T10:05:04.952 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-10T10:05:04.952 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-10T10:05:04.952 INFO:teuthology.orchestra.run.smithi102.stdout: "req_open_latency": { 2024-09-10T10:05:04.952 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-10T10:05:04.952 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-10T10:05:04.952 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-10T10:05:04.952 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-10T10:05:04.952 INFO:teuthology.orchestra.run.smithi102.stdout: "req_readdir_latency": { 2024-09-10T10:05:04.952 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 5889, 2024-09-10T10:05:04.952 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 2.258828708, 2024-09-10T10:05:04.952 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000383567 2024-09-10T10:05:04.952 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-10T10:05:04.953 INFO:teuthology.orchestra.run.smithi102.stdout: "req_rename_latency": { 2024-09-10T10:05:04.953 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-10T10:05:04.953 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-10T10:05:04.953 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-10T10:05:04.953 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-10T10:05:04.953 INFO:teuthology.orchestra.run.smithi102.stdout: "req_renamesnap_latency": { 2024-09-10T10:05:04.953 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-10T10:05:04.953 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-10T10:05:04.953 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-10T10:05:04.953 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-10T10:05:04.953 INFO:teuthology.orchestra.run.smithi102.stdout: "req_snapdiff_latency": { 2024-09-10T10:05:04.953 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-10T10:05:04.953 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-10T10:05:04.954 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-10T10:05:04.954 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-10T10:05:04.954 INFO:teuthology.orchestra.run.smithi102.stdout: "req_rmdir_latency": { 2024-09-10T10:05:04.954 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-10T10:05:04.954 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-10T10:05:04.954 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-10T10:05:04.954 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-10T10:05:04.954 INFO:teuthology.orchestra.run.smithi102.stdout: "req_rmsnap_latency": { 2024-09-10T10:05:04.954 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-10T10:05:04.954 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-10T10:05:04.954 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-10T10:05:04.954 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-10T10:05:04.954 INFO:teuthology.orchestra.run.smithi102.stdout: "req_rmxattr_latency": { 2024-09-10T10:05:04.954 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-10T10:05:04.955 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-10T10:05:04.955 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-10T10:05:04.955 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-10T10:05:04.955 INFO:teuthology.orchestra.run.smithi102.stdout: "req_setattr_latency": { 2024-09-10T10:05:04.955 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-10T10:05:04.955 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-10T10:05:04.955 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-10T10:05:04.955 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-10T10:05:04.955 INFO:teuthology.orchestra.run.smithi102.stdout: "req_setdirlayout_latency": { 2024-09-10T10:05:04.955 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-10T10:05:04.955 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-10T10:05:04.955 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-10T10:05:04.955 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-10T10:05:04.955 INFO:teuthology.orchestra.run.smithi102.stdout: "req_setfilelock_latency": { 2024-09-10T10:05:04.955 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-10T10:05:04.956 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-10T10:05:04.956 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-10T10:05:04.956 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-10T10:05:04.956 INFO:teuthology.orchestra.run.smithi102.stdout: "req_setlayout_latency": { 2024-09-10T10:05:04.956 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-10T10:05:04.956 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-10T10:05:04.956 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-10T10:05:04.956 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-10T10:05:04.956 INFO:teuthology.orchestra.run.smithi102.stdout: "req_setxattr_latency": { 2024-09-10T10:05:04.956 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-10T10:05:04.956 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-10T10:05:04.956 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-10T10:05:04.956 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-10T10:05:04.956 INFO:teuthology.orchestra.run.smithi102.stdout: "req_symlink_latency": { 2024-09-10T10:05:04.956 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-10T10:05:04.957 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-10T10:05:04.957 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-10T10:05:04.957 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-10T10:05:04.957 INFO:teuthology.orchestra.run.smithi102.stdout: "req_unlink_latency": { 2024-09-10T10:05:04.957 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 2630, 2024-09-10T10:05:04.957 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 4.139919427, 2024-09-10T10:05:04.957 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.001574113 2024-09-10T10:05:04.957 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-10T10:05:04.957 INFO:teuthology.orchestra.run.smithi102.stdout: "cap_revoke_eviction": 0, 2024-09-10T10:05:04.957 INFO:teuthology.orchestra.run.smithi102.stdout: "cap_acquisition_throttle": 0, 2024-09-10T10:05:04.957 INFO:teuthology.orchestra.run.smithi102.stdout: "req_getvxattr_latency": { 2024-09-10T10:05:04.957 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-10T10:05:04.958 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-10T10:05:04.958 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-10T10:05:04.958 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-10T10:05:04.958 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-10T10:05:04.958 INFO:teuthology.orchestra.run.smithi102.stdout: "mds_sessions": { 2024-09-10T10:05:04.958 INFO:teuthology.orchestra.run.smithi102.stdout: "session_count": 1, 2024-09-10T10:05:04.958 INFO:teuthology.orchestra.run.smithi102.stdout: "session_add": 2, 2024-09-10T10:05:04.958 INFO:teuthology.orchestra.run.smithi102.stdout: "session_remove": 1, 2024-09-10T10:05:04.958 INFO:teuthology.orchestra.run.smithi102.stdout: "sessions_open": 1, 2024-09-10T10:05:04.958 INFO:teuthology.orchestra.run.smithi102.stdout: "sessions_stale": 0, 2024-09-10T10:05:04.958 INFO:teuthology.orchestra.run.smithi102.stdout: "total_load": 1411, 2024-09-10T10:05:04.958 INFO:teuthology.orchestra.run.smithi102.stdout: "average_load": 1411, 2024-09-10T10:05:04.958 INFO:teuthology.orchestra.run.smithi102.stdout: "avg_session_uptime": 1605, 2024-09-10T10:05:04.958 INFO:teuthology.orchestra.run.smithi102.stdout: "mdthresh_evicted": 0 2024-09-10T10:05:04.959 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-10T10:05:04.959 INFO:teuthology.orchestra.run.smithi102.stdout: "mempool": { 2024-09-10T10:05:04.959 INFO:teuthology.orchestra.run.smithi102.stdout: "bloom_filter_bytes": 0, 2024-09-10T10:05:04.959 INFO:teuthology.orchestra.run.smithi102.stdout: "bloom_filter_items": 0, 2024-09-10T10:05:04.959 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_alloc_bytes": 0, 2024-09-10T10:05:04.959 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_alloc_items": 0, 2024-09-10T10:05:04.959 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_data_bytes": 0, 2024-09-10T10:05:04.959 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_data_items": 0, 2024-09-10T10:05:04.959 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-10T10:05:04.959 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_onode_items": 0, 2024-09-10T10:05:04.959 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-10T10:05:04.959 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_meta_items": 0, 2024-09-10T10:05:04.959 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_other_bytes": 0, 2024-09-10T10:05:04.960 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_other_items": 0, 2024-09-10T10:05:04.960 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-10T10:05:04.960 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_cache_buffer_items": 0, 2024-09-10T10:05:04.960 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_extent_bytes": 0, 2024-09-10T10:05:04.960 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_extent_items": 0, 2024-09-10T10:05:04.960 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_blob_bytes": 0, 2024-09-10T10:05:04.960 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_blob_items": 0, 2024-09-10T10:05:04.960 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-10T10:05:04.960 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_shared_blob_items": 0, 2024-09-10T10:05:04.960 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-10T10:05:04.960 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_inline_bl_items": 0, 2024-09-10T10:05:04.960 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_fsck_bytes": 0, 2024-09-10T10:05:04.960 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_fsck_items": 0, 2024-09-10T10:05:04.960 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_txc_bytes": 0, 2024-09-10T10:05:04.960 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_txc_items": 0, 2024-09-10T10:05:04.961 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-10T10:05:04.961 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_writing_deferred_items": 0, 2024-09-10T10:05:04.961 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_writing_bytes": 0, 2024-09-10T10:05:04.961 INFO:teuthology.orchestra.run.smithi102.stdout: "bluestore_writing_items": 0, 2024-09-10T10:05:04.961 INFO:teuthology.orchestra.run.smithi102.stdout: "bluefs_bytes": 0, 2024-09-10T10:05:04.961 INFO:teuthology.orchestra.run.smithi102.stdout: "bluefs_items": 0, 2024-09-10T10:05:04.961 INFO:teuthology.orchestra.run.smithi102.stdout: "bluefs_file_reader_bytes": 0, 2024-09-10T10:05:04.961 INFO:teuthology.orchestra.run.smithi102.stdout: "bluefs_file_reader_items": 0, 2024-09-10T10:05:04.961 INFO:teuthology.orchestra.run.smithi102.stdout: "bluefs_file_writer_bytes": 0, 2024-09-10T10:05:04.961 INFO:teuthology.orchestra.run.smithi102.stdout: "bluefs_file_writer_items": 0, 2024-09-10T10:05:04.961 INFO:teuthology.orchestra.run.smithi102.stdout: "buffer_anon_bytes": 288517, 2024-09-10T10:05:04.961 INFO:teuthology.orchestra.run.smithi102.stdout: "buffer_anon_items": 106, 2024-09-10T10:05:04.961 INFO:teuthology.orchestra.run.smithi102.stdout: "buffer_meta_bytes": 0, 2024-09-10T10:05:04.961 INFO:teuthology.orchestra.run.smithi102.stdout: "buffer_meta_items": 0, 2024-09-10T10:05:04.962 INFO:teuthology.orchestra.run.smithi102.stdout: "osd_bytes": 0, 2024-09-10T10:05:04.962 INFO:teuthology.orchestra.run.smithi102.stdout: "osd_items": 0, 2024-09-10T10:05:04.962 INFO:teuthology.orchestra.run.smithi102.stdout: "osd_mapbl_bytes": 0, 2024-09-10T10:05:04.962 INFO:teuthology.orchestra.run.smithi102.stdout: "osd_mapbl_items": 0, 2024-09-10T10:05:04.962 INFO:teuthology.orchestra.run.smithi102.stdout: "osd_pglog_bytes": 0, 2024-09-10T10:05:04.962 INFO:teuthology.orchestra.run.smithi102.stdout: "osd_pglog_items": 0, 2024-09-10T10:05:04.962 INFO:teuthology.orchestra.run.smithi102.stdout: "osdmap_bytes": 9152, 2024-09-10T10:05:04.962 INFO:teuthology.orchestra.run.smithi102.stdout: "osdmap_items": 173, 2024-09-10T10:05:04.962 INFO:teuthology.orchestra.run.smithi102.stdout: "osdmap_mapping_bytes": 0, 2024-09-10T10:05:04.962 INFO:teuthology.orchestra.run.smithi102.stdout: "osdmap_mapping_items": 0, 2024-09-10T10:05:04.962 INFO:teuthology.orchestra.run.smithi102.stdout: "pgmap_bytes": 0, 2024-09-10T10:05:04.962 INFO:teuthology.orchestra.run.smithi102.stdout: "pgmap_items": 0, 2024-09-10T10:05:04.962 INFO:teuthology.orchestra.run.smithi102.stdout: "mds_co_bytes": 116293, 2024-09-10T10:05:04.962 INFO:teuthology.orchestra.run.smithi102.stdout: "mds_co_items": 465, 2024-09-10T10:05:04.963 INFO:teuthology.orchestra.run.smithi102.stdout: "unittest_1_bytes": 0, 2024-09-10T10:05:04.963 INFO:teuthology.orchestra.run.smithi102.stdout: "unittest_1_items": 0, 2024-09-10T10:05:04.963 INFO:teuthology.orchestra.run.smithi102.stdout: "unittest_2_bytes": 0, 2024-09-10T10:05:04.963 INFO:teuthology.orchestra.run.smithi102.stdout: "unittest_2_items": 0 2024-09-10T10:05:04.963 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-10T10:05:04.963 INFO:teuthology.orchestra.run.smithi102.stdout: "objecter": { 2024-09-10T10:05:04.963 INFO:teuthology.orchestra.run.smithi102.stdout: "op_active": 0, 2024-09-10T10:05:04.963 INFO:teuthology.orchestra.run.smithi102.stdout: "op_laggy": 0, 2024-09-10T10:05:04.963 INFO:teuthology.orchestra.run.smithi102.stdout: "op_send": 9240, 2024-09-10T10:05:04.963 INFO:teuthology.orchestra.run.smithi102.stdout: "op_send_bytes": 14137901, 2024-09-10T10:05:04.963 INFO:teuthology.orchestra.run.smithi102.stdout: "op_resend": 33, 2024-09-10T10:05:04.963 INFO:teuthology.orchestra.run.smithi102.stdout: "op_reply": 9240, 2024-09-10T10:05:04.963 INFO:teuthology.orchestra.run.smithi102.stdout: "op_latency": { 2024-09-10T10:05:04.963 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 9240, 2024-09-10T10:05:04.963 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 63.343639086, 2024-09-10T10:05:04.963 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.006855372 2024-09-10T10:05:04.964 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-10T10:05:04.964 INFO:teuthology.orchestra.run.smithi102.stdout: "op_inflight": 0, 2024-09-10T10:05:04.964 INFO:teuthology.orchestra.run.smithi102.stdout: "oplen_avg": { 2024-09-10T10:05:04.964 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 9240, 2024-09-10T10:05:04.964 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 9615 2024-09-10T10:05:04.964 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-10T10:05:04.964 INFO:teuthology.orchestra.run.smithi102.stdout: "op": 9240, 2024-09-10T10:05:04.964 INFO:teuthology.orchestra.run.smithi102.stdout: "op_r": 269, 2024-09-10T10:05:04.964 INFO:teuthology.orchestra.run.smithi102.stdout: "op_w": 8971, 2024-09-10T10:05:04.964 INFO:teuthology.orchestra.run.smithi102.stdout: "op_rmw": 0, 2024-09-10T10:05:04.964 INFO:teuthology.orchestra.run.smithi102.stdout: "op_pg": 0, 2024-09-10T10:05:04.964 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_stat": 0, 2024-09-10T10:05:04.964 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_create": 10, 2024-09-10T10:05:04.964 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_read": 231, 2024-09-10T10:05:04.965 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_write": 3215, 2024-09-10T10:05:04.965 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_writefull": 191, 2024-09-10T10:05:04.965 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_writesame": 0, 2024-09-10T10:05:04.965 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_append": 0, 2024-09-10T10:05:04.965 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_zero": 0, 2024-09-10T10:05:04.965 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_truncate": 0, 2024-09-10T10:05:04.965 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_delete": 5275, 2024-09-10T10:05:04.965 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_mapext": 0, 2024-09-10T10:05:04.965 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_sparse_read": 0, 2024-09-10T10:05:04.965 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_clonerange": 0, 2024-09-10T10:05:04.965 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_getxattr": 38, 2024-09-10T10:05:04.965 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_setxattr": 20, 2024-09-10T10:05:04.965 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_cmpxattr": 0, 2024-09-10T10:05:04.966 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_rmxattr": 0, 2024-09-10T10:05:04.966 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_resetxattrs": 0, 2024-09-10T10:05:04.966 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_call": 0, 2024-09-10T10:05:04.966 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_watch": 0, 2024-09-10T10:05:04.966 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_notify": 0, 2024-09-10T10:05:04.966 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_src_cmpxattr": 0, 2024-09-10T10:05:04.966 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_pgls": 0, 2024-09-10T10:05:04.966 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_pgls_filter": 0, 2024-09-10T10:05:04.966 INFO:teuthology.orchestra.run.smithi102.stdout: "osdop_other": 0, 2024-09-10T10:05:04.966 INFO:teuthology.orchestra.run.smithi102.stdout: "linger_active": 0, 2024-09-10T10:05:04.966 INFO:teuthology.orchestra.run.smithi102.stdout: "linger_send": 0, 2024-09-10T10:05:04.966 INFO:teuthology.orchestra.run.smithi102.stdout: "linger_resend": 0, 2024-09-10T10:05:04.966 INFO:teuthology.orchestra.run.smithi102.stdout: "linger_ping": 0, 2024-09-10T10:05:04.967 INFO:teuthology.orchestra.run.smithi102.stdout: "poolop_active": 0, 2024-09-10T10:05:04.967 INFO:teuthology.orchestra.run.smithi102.stdout: "poolop_send": 0, 2024-09-10T10:05:04.967 INFO:teuthology.orchestra.run.smithi102.stdout: "poolop_resend": 0, 2024-09-10T10:05:04.967 INFO:teuthology.orchestra.run.smithi102.stdout: "poolstat_active": 0, 2024-09-10T10:05:04.967 INFO:teuthology.orchestra.run.smithi102.stdout: "poolstat_send": 0, 2024-09-10T10:05:04.967 INFO:teuthology.orchestra.run.smithi102.stdout: "poolstat_resend": 0, 2024-09-10T10:05:04.967 INFO:teuthology.orchestra.run.smithi102.stdout: "statfs_active": 0, 2024-09-10T10:05:04.967 INFO:teuthology.orchestra.run.smithi102.stdout: "statfs_send": 0, 2024-09-10T10:05:04.967 INFO:teuthology.orchestra.run.smithi102.stdout: "statfs_resend": 0, 2024-09-10T10:05:04.967 INFO:teuthology.orchestra.run.smithi102.stdout: "command_active": 0, 2024-09-10T10:05:04.967 INFO:teuthology.orchestra.run.smithi102.stdout: "command_send": 0, 2024-09-10T10:05:04.967 INFO:teuthology.orchestra.run.smithi102.stdout: "command_resend": 0, 2024-09-10T10:05:04.967 INFO:teuthology.orchestra.run.smithi102.stdout: "map_epoch": 96, 2024-09-10T10:05:04.967 INFO:teuthology.orchestra.run.smithi102.stdout: "map_full": 0, 2024-09-10T10:05:04.967 INFO:teuthology.orchestra.run.smithi102.stdout: "map_inc": 2, 2024-09-10T10:05:04.968 INFO:teuthology.orchestra.run.smithi102.stdout: "osd_sessions": 12, 2024-09-10T10:05:04.968 INFO:teuthology.orchestra.run.smithi102.stdout: "osd_session_open": 17, 2024-09-10T10:05:04.968 INFO:teuthology.orchestra.run.smithi102.stdout: "osd_session_close": 5, 2024-09-10T10:05:04.968 INFO:teuthology.orchestra.run.smithi102.stdout: "osd_laggy": 0, 2024-09-10T10:05:04.968 INFO:teuthology.orchestra.run.smithi102.stdout: "omap_wr": 469, 2024-09-10T10:05:04.968 INFO:teuthology.orchestra.run.smithi102.stdout: "omap_rd": 0, 2024-09-10T10:05:04.968 INFO:teuthology.orchestra.run.smithi102.stdout: "omap_del": 166 2024-09-10T10:05:04.968 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-10T10:05:04.968 INFO:teuthology.orchestra.run.smithi102.stdout: "oft": { 2024-09-10T10:05:04.968 INFO:teuthology.orchestra.run.smithi102.stdout: "omap_total_objs": 1, 2024-09-10T10:05:04.968 INFO:teuthology.orchestra.run.smithi102.stdout: "omap_total_kv_pairs": 0, 2024-09-10T10:05:04.968 INFO:teuthology.orchestra.run.smithi102.stdout: "omap_total_updates": 0, 2024-09-10T10:05:04.968 INFO:teuthology.orchestra.run.smithi102.stdout: "omap_total_removes": 0 2024-09-10T10:05:04.968 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-10T10:05:04.969 INFO:teuthology.orchestra.run.smithi102.stdout: "osd-slow-ops": { 2024-09-10T10:05:04.969 INFO:teuthology.orchestra.run.smithi102.stdout: "slow_ops_count": 561 2024-09-10T10:05:04.969 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-10T10:05:04.969 INFO:teuthology.orchestra.run.smithi102.stdout: "purge_queue": { 2024-09-10T10:05:04.969 INFO:teuthology.orchestra.run.smithi102.stdout: "pq_executing_ops": 0, 2024-09-10T10:05:04.969 INFO:teuthology.orchestra.run.smithi102.stdout: "pq_executing_ops_high_water": 10, 2024-09-10T10:05:04.969 INFO:teuthology.orchestra.run.smithi102.stdout: "pq_executing": 0, 2024-09-10T10:05:04.969 INFO:teuthology.orchestra.run.smithi102.stdout: "pq_executing_high_water": 5, 2024-09-10T10:05:04.969 INFO:teuthology.orchestra.run.smithi102.stdout: "pq_executed_ops": 5260, 2024-09-10T10:05:04.969 INFO:teuthology.orchestra.run.smithi102.stdout: "pq_executed": 2630, 2024-09-10T10:05:04.969 INFO:teuthology.orchestra.run.smithi102.stdout: "pq_item_in_journal": 0 2024-09-10T10:05:04.969 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-10T10:05:04.969 INFO:teuthology.orchestra.run.smithi102.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-10T10:05:04.969 INFO:teuthology.orchestra.run.smithi102.stdout: "val": 0, 2024-09-10T10:05:04.969 INFO:teuthology.orchestra.run.smithi102.stdout: "max": 104857600, 2024-09-10T10:05:04.970 INFO:teuthology.orchestra.run.smithi102.stdout: "get_started": 0, 2024-09-10T10:05:04.970 INFO:teuthology.orchestra.run.smithi102.stdout: "get": 153189, 2024-09-10T10:05:04.970 INFO:teuthology.orchestra.run.smithi102.stdout: "get_sum": 29939833, 2024-09-10T10:05:04.970 INFO:teuthology.orchestra.run.smithi102.stdout: "get_or_fail_fail": 0, 2024-09-10T10:05:04.970 INFO:teuthology.orchestra.run.smithi102.stdout: "get_or_fail_success": 153189, 2024-09-10T10:05:04.970 INFO:teuthology.orchestra.run.smithi102.stdout: "take": 0, 2024-09-10T10:05:04.970 INFO:teuthology.orchestra.run.smithi102.stdout: "take_sum": 0, 2024-09-10T10:05:04.970 INFO:teuthology.orchestra.run.smithi102.stdout: "put": 153189, 2024-09-10T10:05:04.970 INFO:teuthology.orchestra.run.smithi102.stdout: "put_sum": 29939833, 2024-09-10T10:05:04.970 INFO:teuthology.orchestra.run.smithi102.stdout: "wait": { 2024-09-10T10:05:04.970 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-10T10:05:04.970 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-10T10:05:04.970 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-10T10:05:04.970 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-10T10:05:04.970 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-10T10:05:04.971 INFO:teuthology.orchestra.run.smithi102.stdout: "throttle-objecter_bytes": { 2024-09-10T10:05:04.971 INFO:teuthology.orchestra.run.smithi102.stdout: "val": 0, 2024-09-10T10:05:04.971 INFO:teuthology.orchestra.run.smithi102.stdout: "max": 104857600, 2024-09-10T10:05:04.971 INFO:teuthology.orchestra.run.smithi102.stdout: "get_started": 0, 2024-09-10T10:05:04.971 INFO:teuthology.orchestra.run.smithi102.stdout: "get": 0, 2024-09-10T10:05:04.971 INFO:teuthology.orchestra.run.smithi102.stdout: "get_sum": 0, 2024-09-10T10:05:04.971 INFO:teuthology.orchestra.run.smithi102.stdout: "get_or_fail_fail": 0, 2024-09-10T10:05:04.971 INFO:teuthology.orchestra.run.smithi102.stdout: "get_or_fail_success": 0, 2024-09-10T10:05:04.971 INFO:teuthology.orchestra.run.smithi102.stdout: "take": 9240, 2024-09-10T10:05:04.971 INFO:teuthology.orchestra.run.smithi102.stdout: "take_sum": 14424571, 2024-09-10T10:05:04.971 INFO:teuthology.orchestra.run.smithi102.stdout: "put": 9240, 2024-09-10T10:05:04.971 INFO:teuthology.orchestra.run.smithi102.stdout: "put_sum": 14424571, 2024-09-10T10:05:04.971 INFO:teuthology.orchestra.run.smithi102.stdout: "wait": { 2024-09-10T10:05:04.971 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-10T10:05:04.971 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-10T10:05:04.972 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-10T10:05:04.972 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-10T10:05:04.972 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-10T10:05:04.972 INFO:teuthology.orchestra.run.smithi102.stdout: "throttle-objecter_ops": { 2024-09-10T10:05:04.972 INFO:teuthology.orchestra.run.smithi102.stdout: "val": 0, 2024-09-10T10:05:04.972 INFO:teuthology.orchestra.run.smithi102.stdout: "max": 1024, 2024-09-10T10:05:04.972 INFO:teuthology.orchestra.run.smithi102.stdout: "get_started": 0, 2024-09-10T10:05:04.972 INFO:teuthology.orchestra.run.smithi102.stdout: "get": 0, 2024-09-10T10:05:04.972 INFO:teuthology.orchestra.run.smithi102.stdout: "get_sum": 0, 2024-09-10T10:05:04.972 INFO:teuthology.orchestra.run.smithi102.stdout: "get_or_fail_fail": 0, 2024-09-10T10:05:04.972 INFO:teuthology.orchestra.run.smithi102.stdout: "get_or_fail_success": 0, 2024-09-10T10:05:04.972 INFO:teuthology.orchestra.run.smithi102.stdout: "take": 9240, 2024-09-10T10:05:04.972 INFO:teuthology.orchestra.run.smithi102.stdout: "take_sum": 9240, 2024-09-10T10:05:04.972 INFO:teuthology.orchestra.run.smithi102.stdout: "put": 9240, 2024-09-10T10:05:04.972 INFO:teuthology.orchestra.run.smithi102.stdout: "put_sum": 9240, 2024-09-10T10:05:04.973 INFO:teuthology.orchestra.run.smithi102.stdout: "wait": { 2024-09-10T10:05:04.973 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-10T10:05:04.973 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-10T10:05:04.973 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-10T10:05:04.973 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-10T10:05:04.973 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-10T10:05:04.973 INFO:teuthology.orchestra.run.smithi102.stdout: "throttle-write_buf_throttle": { 2024-09-10T10:05:04.973 INFO:teuthology.orchestra.run.smithi102.stdout: "val": 0, 2024-09-10T10:05:04.973 INFO:teuthology.orchestra.run.smithi102.stdout: "max": 3758096384, 2024-09-10T10:05:04.973 INFO:teuthology.orchestra.run.smithi102.stdout: "get_started": 0, 2024-09-10T10:05:04.973 INFO:teuthology.orchestra.run.smithi102.stdout: "get": 2630, 2024-09-10T10:05:04.973 INFO:teuthology.orchestra.run.smithi102.stdout: "get_sum": 286670, 2024-09-10T10:05:04.973 INFO:teuthology.orchestra.run.smithi102.stdout: "get_or_fail_fail": 0, 2024-09-10T10:05:04.973 INFO:teuthology.orchestra.run.smithi102.stdout: "get_or_fail_success": 2630, 2024-09-10T10:05:04.973 INFO:teuthology.orchestra.run.smithi102.stdout: "take": 0, 2024-09-10T10:05:04.974 INFO:teuthology.orchestra.run.smithi102.stdout: "take_sum": 0, 2024-09-10T10:05:04.974 INFO:teuthology.orchestra.run.smithi102.stdout: "put": 231, 2024-09-10T10:05:04.974 INFO:teuthology.orchestra.run.smithi102.stdout: "put_sum": 286670, 2024-09-10T10:05:04.974 INFO:teuthology.orchestra.run.smithi102.stdout: "wait": { 2024-09-10T10:05:04.974 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-10T10:05:04.974 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-10T10:05:04.974 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-10T10:05:04.974 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-10T10:05:04.974 INFO:teuthology.orchestra.run.smithi102.stdout: }, 2024-09-10T10:05:04.974 INFO:teuthology.orchestra.run.smithi102.stdout: "throttle-write_buf_throttle-0x561fda58a960": { 2024-09-10T10:05:04.974 INFO:teuthology.orchestra.run.smithi102.stdout: "val": 0, 2024-09-10T10:05:04.974 INFO:teuthology.orchestra.run.smithi102.stdout: "max": 3758096384, 2024-09-10T10:05:04.974 INFO:teuthology.orchestra.run.smithi102.stdout: "get_started": 0, 2024-09-10T10:05:04.974 INFO:teuthology.orchestra.run.smithi102.stdout: "get": 11025, 2024-09-10T10:05:04.974 INFO:teuthology.orchestra.run.smithi102.stdout: "get_sum": 13598922, 2024-09-10T10:05:04.975 INFO:teuthology.orchestra.run.smithi102.stdout: "get_or_fail_fail": 0, 2024-09-10T10:05:04.975 INFO:teuthology.orchestra.run.smithi102.stdout: "get_or_fail_success": 11025, 2024-09-10T10:05:04.975 INFO:teuthology.orchestra.run.smithi102.stdout: "take": 0, 2024-09-10T10:05:04.975 INFO:teuthology.orchestra.run.smithi102.stdout: "take_sum": 0, 2024-09-10T10:05:04.975 INFO:teuthology.orchestra.run.smithi102.stdout: "put": 2984, 2024-09-10T10:05:04.975 INFO:teuthology.orchestra.run.smithi102.stdout: "put_sum": 13598922, 2024-09-10T10:05:04.975 INFO:teuthology.orchestra.run.smithi102.stdout: "wait": { 2024-09-10T10:05:04.975 INFO:teuthology.orchestra.run.smithi102.stdout: "avgcount": 0, 2024-09-10T10:05:04.975 INFO:teuthology.orchestra.run.smithi102.stdout: "sum": 0.000000000, 2024-09-10T10:05:04.975 INFO:teuthology.orchestra.run.smithi102.stdout: "avgtime": 0.000000000 2024-09-10T10:05:04.975 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-10T10:05:04.975 INFO:teuthology.orchestra.run.smithi102.stdout: } 2024-09-10T10:05:04.975 INFO:teuthology.orchestra.run.smithi102.stdout:} 2024-09-10T10:05:05.734 INFO:tasks.check_counter:Daemon mds.k mds.exported=1 2024-09-10T10:05:05.735 INFO:tasks.check_counter:Daemon mds.k mds.imported=1 2024-09-10T10:05:05.735 DEBUG:tasks.check_counter:Getting stats from c 2024-09-10T10:05:05.735 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph daemon mds.c perf dump 2024-09-10T10:05:06.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:05:05 smithi159 ceph-mon[35513]: pgmap v1195: 129 pgs: 129 active+clean; 551 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-10T10:05:06.126 INFO:teuthology.orchestra.run.smithi159.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.c/config 2024-09-10T10:05:06.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:05:05 smithi102 ceph-mon[35442]: pgmap v1195: 129 pgs: 129 active+clean; 551 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-10T10:05:06.306 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:05:05 smithi005 ceph-mon[30350]: pgmap v1195: 129 pgs: 129 active+clean; 551 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-10T10:05:06.877 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-09-10T10:05:06.877 INFO:teuthology.orchestra.run.smithi159.stdout: "AsyncMessenger::Worker-0": { 2024-09-10T10:05:06.877 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_messages": 609350, 2024-09-10T10:05:06.877 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_messages": 975443, 2024-09-10T10:05:06.877 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_bytes": 186007754, 2024-09-10T10:05:06.877 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_bytes": 783849073, 2024-09-10T10:05:06.877 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_created_connections": 20, 2024-09-10T10:05:06.878 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_active_connections": 6, 2024-09-10T10:05:06.878 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_total_time": 33.130735967, 2024-09-10T10:05:06.878 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_send_time": 15.564276140, 2024-09-10T10:05:06.878 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_recv_time": 20.204720984, 2024-09-10T10:05:06.878 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_fast_dispatch_time": 2.677759497, 2024-09-10T10:05:06.878 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T10:05:06.878 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 975440, 2024-09-10T10:05:06.878 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 45.238090560, 2024-09-10T10:05:06.878 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000046377 2024-09-10T10:05:06.878 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-10T10:05:06.878 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_handle_ack_lat": { 2024-09-10T10:05:06.878 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 775540, 2024-09-10T10:05:06.879 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.205288305, 2024-09-10T10:05:06.879 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000264 2024-09-10T10:05:06.879 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-10T10:05:06.879 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_encrypted_bytes": 1015136, 2024-09-10T10:05:06.879 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_encrypted_bytes": 324592 2024-09-10T10:05:06.879 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-10T10:05:06.879 INFO:teuthology.orchestra.run.smithi159.stdout: "AsyncMessenger::Worker-1": { 2024-09-10T10:05:06.879 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_messages": 163656, 2024-09-10T10:05:06.879 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_messages": 202793, 2024-09-10T10:05:06.880 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_bytes": 38570543, 2024-09-10T10:05:06.880 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_bytes": 388384926, 2024-09-10T10:05:06.880 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_created_connections": 10, 2024-09-10T10:05:06.880 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_active_connections": 7, 2024-09-10T10:05:06.880 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_total_time": 16.247968434, 2024-09-10T10:05:06.881 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_send_time": 7.762757611, 2024-09-10T10:05:06.881 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_recv_time": 4.979241467, 2024-09-10T10:05:06.881 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_fast_dispatch_time": 3.443789102, 2024-09-10T10:05:06.881 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T10:05:06.881 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 202792, 2024-09-10T10:05:06.881 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 10.211281163, 2024-09-10T10:05:06.881 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000050353 2024-09-10T10:05:06.881 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-10T10:05:06.881 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_handle_ack_lat": { 2024-09-10T10:05:06.881 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 166238, 2024-09-10T10:05:06.881 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.030708283, 2024-09-10T10:05:06.882 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000184 2024-09-10T10:05:06.882 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-10T10:05:06.882 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_encrypted_bytes": 3344, 2024-09-10T10:05:06.882 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_encrypted_bytes": 803984 2024-09-10T10:05:06.882 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-10T10:05:06.882 INFO:teuthology.orchestra.run.smithi159.stdout: "AsyncMessenger::Worker-2": { 2024-09-10T10:05:06.882 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_messages": 274100, 2024-09-10T10:05:06.882 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_messages": 360532, 2024-09-10T10:05:06.882 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_bytes": 67288450, 2024-09-10T10:05:06.882 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_bytes": 279555629, 2024-09-10T10:05:06.882 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_created_connections": 60, 2024-09-10T10:05:06.882 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_active_connections": 5, 2024-09-10T10:05:06.883 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_total_time": 17.641579651, 2024-09-10T10:05:06.883 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_send_time": 6.841844495, 2024-09-10T10:05:06.883 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_recv_time": 9.169117559, 2024-09-10T10:05:06.883 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_fast_dispatch_time": 2.488412089, 2024-09-10T10:05:06.883 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T10:05:06.883 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 360532, 2024-09-10T10:05:06.883 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 14.601240826, 2024-09-10T10:05:06.883 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000040499 2024-09-10T10:05:06.883 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-10T10:05:06.883 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_handle_ack_lat": { 2024-09-10T10:05:06.883 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 492176, 2024-09-10T10:05:06.884 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.081984195, 2024-09-10T10:05:06.884 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000166 2024-09-10T10:05:06.884 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-10T10:05:06.884 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-10T10:05:06.884 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_encrypted_bytes": 0 2024-09-10T10:05:06.884 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-10T10:05:06.884 INFO:teuthology.orchestra.run.smithi159.stdout: "cct": { 2024-09-10T10:05:06.884 INFO:teuthology.orchestra.run.smithi159.stdout: "total_workers": 1, 2024-09-10T10:05:06.884 INFO:teuthology.orchestra.run.smithi159.stdout: "unhealthy_workers": 0 2024-09-10T10:05:06.884 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-10T10:05:06.884 INFO:teuthology.orchestra.run.smithi159.stdout: "finisher-MDSRank": { 2024-09-10T10:05:06.885 INFO:teuthology.orchestra.run.smithi159.stdout: "queue_len": 0, 2024-09-10T10:05:06.885 INFO:teuthology.orchestra.run.smithi159.stdout: "complete_latency": { 2024-09-10T10:05:06.885 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 157719, 2024-09-10T10:05:06.885 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 212.504406158, 2024-09-10T10:05:06.885 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.001347360 2024-09-10T10:05:06.885 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-10T10:05:06.885 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-10T10:05:06.885 INFO:teuthology.orchestra.run.smithi159.stdout: "finisher-PurgeQueue": { 2024-09-10T10:05:06.885 INFO:teuthology.orchestra.run.smithi159.stdout: "queue_len": 0, 2024-09-10T10:05:06.885 INFO:teuthology.orchestra.run.smithi159.stdout: "complete_latency": { 2024-09-10T10:05:06.885 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 117805, 2024-09-10T10:05:06.885 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 20.890255648, 2024-09-10T10:05:06.886 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000177329 2024-09-10T10:05:06.886 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-10T10:05:06.886 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-10T10:05:06.886 INFO:teuthology.orchestra.run.smithi159.stdout: "mds": { 2024-09-10T10:05:06.886 INFO:teuthology.orchestra.run.smithi159.stdout: "request": 363861, 2024-09-10T10:05:06.886 INFO:teuthology.orchestra.run.smithi159.stdout: "reply": 351642, 2024-09-10T10:05:06.886 INFO:teuthology.orchestra.run.smithi159.stdout: "reply_latency": { 2024-09-10T10:05:06.886 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 351642, 2024-09-10T10:05:06.886 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 3487.758687140, 2024-09-10T10:05:06.886 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.009918492 2024-09-10T10:05:06.886 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-10T10:05:06.886 INFO:teuthology.orchestra.run.smithi159.stdout: "slow_reply": 0, 2024-09-10T10:05:06.886 INFO:teuthology.orchestra.run.smithi159.stdout: "forward": 12219, 2024-09-10T10:05:06.886 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_fetch_complete": 0, 2024-09-10T10:05:06.886 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_fetch_keys": 0, 2024-09-10T10:05:06.887 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_commit": 325, 2024-09-10T10:05:06.887 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_split": 26, 2024-09-10T10:05:06.887 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_merge": 27, 2024-09-10T10:05:06.887 INFO:teuthology.orchestra.run.smithi159.stdout: "inodes": 3590, 2024-09-10T10:05:06.887 INFO:teuthology.orchestra.run.smithi159.stdout: "inodes_top": 2158, 2024-09-10T10:05:06.887 INFO:teuthology.orchestra.run.smithi159.stdout: "inodes_bottom": 1414, 2024-09-10T10:05:06.887 INFO:teuthology.orchestra.run.smithi159.stdout: "inodes_pin_tail": 18, 2024-09-10T10:05:06.887 INFO:teuthology.orchestra.run.smithi159.stdout: "inodes_pinned": 506, 2024-09-10T10:05:06.887 INFO:teuthology.orchestra.run.smithi159.stdout: "inodes_expired": 423, 2024-09-10T10:05:06.887 INFO:teuthology.orchestra.run.smithi159.stdout: "inodes_with_caps": 0, 2024-09-10T10:05:06.887 INFO:teuthology.orchestra.run.smithi159.stdout: "caps": 0, 2024-09-10T10:05:06.887 INFO:teuthology.orchestra.run.smithi159.stdout: "subtrees": 5, 2024-09-10T10:05:06.887 INFO:teuthology.orchestra.run.smithi159.stdout: "traverse": 534802, 2024-09-10T10:05:06.888 INFO:teuthology.orchestra.run.smithi159.stdout: "traverse_hit": 325557, 2024-09-10T10:05:06.888 INFO:teuthology.orchestra.run.smithi159.stdout: "traverse_forward": 12218, 2024-09-10T10:05:06.888 INFO:teuthology.orchestra.run.smithi159.stdout: "traverse_discover": 23, 2024-09-10T10:05:06.888 INFO:teuthology.orchestra.run.smithi159.stdout: "traverse_dir_fetch": 0, 2024-09-10T10:05:06.888 INFO:teuthology.orchestra.run.smithi159.stdout: "traverse_remote_ino": 0, 2024-09-10T10:05:06.888 INFO:teuthology.orchestra.run.smithi159.stdout: "traverse_lock": 4039, 2024-09-10T10:05:06.888 INFO:teuthology.orchestra.run.smithi159.stdout: "load_cent": 0, 2024-09-10T10:05:06.888 INFO:teuthology.orchestra.run.smithi159.stdout: "q": 0, 2024-09-10T10:05:06.888 INFO:teuthology.orchestra.run.smithi159.stdout: "exported": 19, 2024-09-10T10:05:06.888 INFO:teuthology.orchestra.run.smithi159.stdout: "exported_inodes": 375, 2024-09-10T10:05:06.888 INFO:teuthology.orchestra.run.smithi159.stdout: "imported": 17, 2024-09-10T10:05:06.888 INFO:teuthology.orchestra.run.smithi159.stdout: "imported_inodes": 136, 2024-09-10T10:05:06.888 INFO:teuthology.orchestra.run.smithi159.stdout: "openino_dir_fetch": 0, 2024-09-10T10:05:06.888 INFO:teuthology.orchestra.run.smithi159.stdout: "openino_backtrace_fetch": 6772, 2024-09-10T10:05:06.888 INFO:teuthology.orchestra.run.smithi159.stdout: "openino_peer_discover": 0, 2024-09-10T10:05:06.889 INFO:teuthology.orchestra.run.smithi159.stdout: "root_rfiles": 733, 2024-09-10T10:05:06.889 INFO:teuthology.orchestra.run.smithi159.stdout: "root_rbytes": 226265082, 2024-09-10T10:05:06.889 INFO:teuthology.orchestra.run.smithi159.stdout: "root_rsnaps": 0, 2024-09-10T10:05:06.889 INFO:teuthology.orchestra.run.smithi159.stdout: "scrub_backtrace_fetch": 6772, 2024-09-10T10:05:06.889 INFO:teuthology.orchestra.run.smithi159.stdout: "scrub_set_tag": 0, 2024-09-10T10:05:06.889 INFO:teuthology.orchestra.run.smithi159.stdout: "scrub_backtrace_repaired": 0, 2024-09-10T10:05:06.889 INFO:teuthology.orchestra.run.smithi159.stdout: "scrub_inotable_repaired": 0, 2024-09-10T10:05:06.889 INFO:teuthology.orchestra.run.smithi159.stdout: "scrub_dir_inodes": 1081, 2024-09-10T10:05:06.889 INFO:teuthology.orchestra.run.smithi159.stdout: "scrub_dir_base_inodes": 0, 2024-09-10T10:05:06.889 INFO:teuthology.orchestra.run.smithi159.stdout: "scrub_dirfrag_rstats": 1081, 2024-09-10T10:05:06.889 INFO:teuthology.orchestra.run.smithi159.stdout: "scrub_file_inodes": 5691, 2024-09-10T10:05:06.889 INFO:teuthology.orchestra.run.smithi159.stdout: "handle_inode_file_caps": 40, 2024-09-10T10:05:06.889 INFO:teuthology.orchestra.run.smithi159.stdout: "ceph_cap_op_revoke": 32536, 2024-09-10T10:05:06.889 INFO:teuthology.orchestra.run.smithi159.stdout: "ceph_cap_op_grant": 248806, 2024-09-10T10:05:06.889 INFO:teuthology.orchestra.run.smithi159.stdout: "ceph_cap_op_trunc": 0, 2024-09-10T10:05:06.890 INFO:teuthology.orchestra.run.smithi159.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-10T10:05:06.890 INFO:teuthology.orchestra.run.smithi159.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-10T10:05:06.890 INFO:teuthology.orchestra.run.smithi159.stdout: "handle_client_caps": 120109, 2024-09-10T10:05:06.890 INFO:teuthology.orchestra.run.smithi159.stdout: "handle_client_caps_dirty": 45917, 2024-09-10T10:05:06.890 INFO:teuthology.orchestra.run.smithi159.stdout: "handle_client_cap_release": 13358, 2024-09-10T10:05:06.890 INFO:teuthology.orchestra.run.smithi159.stdout: "process_request_cap_release": 222964 2024-09-10T10:05:06.890 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-10T10:05:06.890 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_cache": { 2024-09-10T10:05:06.890 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_update": 26, 2024-09-10T10:05:06.890 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_update_receipt": 18, 2024-09-10T10:05:06.890 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_try_discover": 12, 2024-09-10T10:05:06.890 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_send_discover": 31, 2024-09-10T10:05:06.890 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_handle_discover": 366, 2024-09-10T10:05:06.890 INFO:teuthology.orchestra.run.smithi159.stdout: "num_strays": 8, 2024-09-10T10:05:06.890 INFO:teuthology.orchestra.run.smithi159.stdout: "num_strays_delayed": 0, 2024-09-10T10:05:06.890 INFO:teuthology.orchestra.run.smithi159.stdout: "num_strays_enqueuing": 0, 2024-09-10T10:05:06.891 INFO:teuthology.orchestra.run.smithi159.stdout: "strays_created": 61437, 2024-09-10T10:05:06.891 INFO:teuthology.orchestra.run.smithi159.stdout: "strays_enqueued": 61429, 2024-09-10T10:05:06.891 INFO:teuthology.orchestra.run.smithi159.stdout: "strays_reintegrated": 0, 2024-09-10T10:05:06.891 INFO:teuthology.orchestra.run.smithi159.stdout: "strays_migrated": 0, 2024-09-10T10:05:06.891 INFO:teuthology.orchestra.run.smithi159.stdout: "num_recovering_processing": 0, 2024-09-10T10:05:06.891 INFO:teuthology.orchestra.run.smithi159.stdout: "num_recovering_enqueued": 0, 2024-09-10T10:05:06.891 INFO:teuthology.orchestra.run.smithi159.stdout: "num_recovering_prioritized": 0, 2024-09-10T10:05:06.891 INFO:teuthology.orchestra.run.smithi159.stdout: "recovery_started": 0, 2024-09-10T10:05:06.891 INFO:teuthology.orchestra.run.smithi159.stdout: "recovery_completed": 0, 2024-09-10T10:05:06.891 INFO:teuthology.orchestra.run.smithi159.stdout: "ireq_quiesce_path": 23, 2024-09-10T10:05:06.891 INFO:teuthology.orchestra.run.smithi159.stdout: "ireq_quiesce_inode": 6386, 2024-09-10T10:05:06.891 INFO:teuthology.orchestra.run.smithi159.stdout: "ireq_enqueue_scrub": 0, 2024-09-10T10:05:06.891 INFO:teuthology.orchestra.run.smithi159.stdout: "ireq_exportdir": 20, 2024-09-10T10:05:06.891 INFO:teuthology.orchestra.run.smithi159.stdout: "ireq_flush": 0, 2024-09-10T10:05:06.892 INFO:teuthology.orchestra.run.smithi159.stdout: "ireq_fragmentdir": 53, 2024-09-10T10:05:06.892 INFO:teuthology.orchestra.run.smithi159.stdout: "ireq_fragstats": 0, 2024-09-10T10:05:06.892 INFO:teuthology.orchestra.run.smithi159.stdout: "ireq_inodestats": 0 2024-09-10T10:05:06.892 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-10T10:05:06.892 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_log": { 2024-09-10T10:05:06.892 INFO:teuthology.orchestra.run.smithi159.stdout: "evlrg": 0, 2024-09-10T10:05:06.892 INFO:teuthology.orchestra.run.smithi159.stdout: "evadd": 295089, 2024-09-10T10:05:06.892 INFO:teuthology.orchestra.run.smithi159.stdout: "evex": 184351, 2024-09-10T10:05:06.892 INFO:teuthology.orchestra.run.smithi159.stdout: "evtrm": 184351, 2024-09-10T10:05:06.892 INFO:teuthology.orchestra.run.smithi159.stdout: "ev": 110738, 2024-09-10T10:05:06.892 INFO:teuthology.orchestra.run.smithi159.stdout: "evexg": 0, 2024-09-10T10:05:06.892 INFO:teuthology.orchestra.run.smithi159.stdout: "evexd": 12290, 2024-09-10T10:05:06.892 INFO:teuthology.orchestra.run.smithi159.stdout: "segadd": 347, 2024-09-10T10:05:06.893 INFO:teuthology.orchestra.run.smithi159.stdout: "segex": 219, 2024-09-10T10:05:06.893 INFO:teuthology.orchestra.run.smithi159.stdout: "segtrm": 219, 2024-09-10T10:05:06.893 INFO:teuthology.orchestra.run.smithi159.stdout: "seg": 128, 2024-09-10T10:05:06.893 INFO:teuthology.orchestra.run.smithi159.stdout: "segmjr": 10, 2024-09-10T10:05:06.893 INFO:teuthology.orchestra.run.smithi159.stdout: "segexg": 0, 2024-09-10T10:05:06.893 INFO:teuthology.orchestra.run.smithi159.stdout: "segexd": 15, 2024-09-10T10:05:06.893 INFO:teuthology.orchestra.run.smithi159.stdout: "expos": 389328154, 2024-09-10T10:05:06.894 INFO:teuthology.orchestra.run.smithi159.stdout: "wrpos": 615928886, 2024-09-10T10:05:06.894 INFO:teuthology.orchestra.run.smithi159.stdout: "rdpos": 0, 2024-09-10T10:05:06.894 INFO:teuthology.orchestra.run.smithi159.stdout: "jlat": { 2024-09-10T10:05:06.894 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 89269, 2024-09-10T10:05:06.894 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 868.757801559, 2024-09-10T10:05:06.895 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.009731909 2024-09-10T10:05:06.895 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-10T10:05:06.895 INFO:teuthology.orchestra.run.smithi159.stdout: "replayed": 0 2024-09-10T10:05:06.895 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-10T10:05:06.895 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_mem": { 2024-09-10T10:05:06.895 INFO:teuthology.orchestra.run.smithi159.stdout: "ino": 33, 2024-09-10T10:05:06.896 INFO:teuthology.orchestra.run.smithi159.stdout: "ino+": 61740, 2024-09-10T10:05:06.896 INFO:teuthology.orchestra.run.smithi159.stdout: "ino-": 61707, 2024-09-10T10:05:06.896 INFO:teuthology.orchestra.run.smithi159.stdout: "dir": 53, 2024-09-10T10:05:06.896 INFO:teuthology.orchestra.run.smithi159.stdout: "dir+": 382, 2024-09-10T10:05:06.896 INFO:teuthology.orchestra.run.smithi159.stdout: "dir-": 329, 2024-09-10T10:05:06.897 INFO:teuthology.orchestra.run.smithi159.stdout: "dn": 3590, 2024-09-10T10:05:06.897 INFO:teuthology.orchestra.run.smithi159.stdout: "dn+": 116409, 2024-09-10T10:05:06.897 INFO:teuthology.orchestra.run.smithi159.stdout: "dn-": 112819, 2024-09-10T10:05:06.897 INFO:teuthology.orchestra.run.smithi159.stdout: "cap": 0, 2024-09-10T10:05:06.897 INFO:teuthology.orchestra.run.smithi159.stdout: "cap+": 65529, 2024-09-10T10:05:06.898 INFO:teuthology.orchestra.run.smithi159.stdout: "cap-": 65529, 2024-09-10T10:05:06.898 INFO:teuthology.orchestra.run.smithi159.stdout: "rss": 94524, 2024-09-10T10:05:06.898 INFO:teuthology.orchestra.run.smithi159.stdout: "heap": 223520 2024-09-10T10:05:06.898 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-10T10:05:06.898 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_server": { 2024-09-10T10:05:06.898 INFO:teuthology.orchestra.run.smithi159.stdout: "dispatch_client_request": 477252, 2024-09-10T10:05:06.899 INFO:teuthology.orchestra.run.smithi159.stdout: "dispatch_server_request": 13879, 2024-09-10T10:05:06.899 INFO:teuthology.orchestra.run.smithi159.stdout: "handle_client_request": 363861, 2024-09-10T10:05:06.899 INFO:teuthology.orchestra.run.smithi159.stdout: "handle_client_session": 8287, 2024-09-10T10:05:06.899 INFO:teuthology.orchestra.run.smithi159.stdout: "handle_peer_request": 32919, 2024-09-10T10:05:06.899 INFO:teuthology.orchestra.run.smithi159.stdout: "req_create_latency": { 2024-09-10T10:05:06.900 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 61545, 2024-09-10T10:05:06.900 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 1073.799375005, 2024-09-10T10:05:06.900 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.017447386 2024-09-10T10:05:06.900 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-10T10:05:06.900 INFO:teuthology.orchestra.run.smithi159.stdout: "req_getattr_latency": { 2024-09-10T10:05:06.900 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 16908, 2024-09-10T10:05:06.900 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 192.179065569, 2024-09-10T10:05:06.900 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.011366161 2024-09-10T10:05:06.901 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-10T10:05:06.901 INFO:teuthology.orchestra.run.smithi159.stdout: "req_getfilelock_latency": { 2024-09-10T10:05:06.901 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-10T10:05:06.901 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-10T10:05:06.901 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-10T10:05:06.901 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-10T10:05:06.901 INFO:teuthology.orchestra.run.smithi159.stdout: "req_link_latency": { 2024-09-10T10:05:06.901 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-10T10:05:06.901 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-10T10:05:06.901 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-10T10:05:06.901 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-10T10:05:06.901 INFO:teuthology.orchestra.run.smithi159.stdout: "req_lookup_latency": { 2024-09-10T10:05:06.901 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 103199, 2024-09-10T10:05:06.902 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 332.928486049, 2024-09-10T10:05:06.902 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.003226082 2024-09-10T10:05:06.902 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-10T10:05:06.902 INFO:teuthology.orchestra.run.smithi159.stdout: "req_lookuphash_latency": { 2024-09-10T10:05:06.902 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-10T10:05:06.902 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-10T10:05:06.902 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-10T10:05:06.902 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-10T10:05:06.902 INFO:teuthology.orchestra.run.smithi159.stdout: "req_lookupino_latency": { 2024-09-10T10:05:06.902 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-10T10:05:06.902 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-10T10:05:06.902 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-10T10:05:06.902 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-10T10:05:06.902 INFO:teuthology.orchestra.run.smithi159.stdout: "req_lookupname_latency": { 2024-09-10T10:05:06.903 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-10T10:05:06.903 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-10T10:05:06.903 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-10T10:05:06.903 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-10T10:05:06.903 INFO:teuthology.orchestra.run.smithi159.stdout: "req_lookupparent_latency": { 2024-09-10T10:05:06.903 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-10T10:05:06.903 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-10T10:05:06.903 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-10T10:05:06.903 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-10T10:05:06.904 INFO:teuthology.orchestra.run.smithi159.stdout: "req_lookupsnap_latency": { 2024-09-10T10:05:06.904 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-10T10:05:06.904 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-10T10:05:06.904 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-10T10:05:06.904 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-10T10:05:06.904 INFO:teuthology.orchestra.run.smithi159.stdout: "req_lssnap_latency": { 2024-09-10T10:05:06.904 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-10T10:05:06.904 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-10T10:05:06.905 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-10T10:05:06.905 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-10T10:05:06.905 INFO:teuthology.orchestra.run.smithi159.stdout: "req_mkdir_latency": { 2024-09-10T10:05:06.905 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 99, 2024-09-10T10:05:06.905 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.549756524, 2024-09-10T10:05:06.905 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.005553096 2024-09-10T10:05:06.905 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-10T10:05:06.905 INFO:teuthology.orchestra.run.smithi159.stdout: "req_mknod_latency": { 2024-09-10T10:05:06.905 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-10T10:05:06.905 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-10T10:05:06.905 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-10T10:05:06.905 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-10T10:05:06.905 INFO:teuthology.orchestra.run.smithi159.stdout: "req_mksnap_latency": { 2024-09-10T10:05:06.906 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-10T10:05:06.906 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-10T10:05:06.906 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-10T10:05:06.906 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-10T10:05:06.906 INFO:teuthology.orchestra.run.smithi159.stdout: "req_open_latency": { 2024-09-10T10:05:06.906 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 9, 2024-09-10T10:05:06.906 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.020620155, 2024-09-10T10:05:06.906 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.002291128 2024-09-10T10:05:06.906 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-10T10:05:06.906 INFO:teuthology.orchestra.run.smithi159.stdout: "req_readdir_latency": { 2024-09-10T10:05:06.906 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 92472, 2024-09-10T10:05:06.906 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 732.003317307, 2024-09-10T10:05:06.906 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.007915945 2024-09-10T10:05:06.906 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-10T10:05:06.906 INFO:teuthology.orchestra.run.smithi159.stdout: "req_rename_latency": { 2024-09-10T10:05:06.907 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 13845, 2024-09-10T10:05:06.907 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 327.034160614, 2024-09-10T10:05:06.907 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.023621102 2024-09-10T10:05:06.907 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-10T10:05:06.907 INFO:teuthology.orchestra.run.smithi159.stdout: "req_renamesnap_latency": { 2024-09-10T10:05:06.907 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-10T10:05:06.907 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-10T10:05:06.907 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-10T10:05:06.907 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-10T10:05:06.907 INFO:teuthology.orchestra.run.smithi159.stdout: "req_snapdiff_latency": { 2024-09-10T10:05:06.907 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-10T10:05:06.907 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-10T10:05:06.907 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-10T10:05:06.907 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-10T10:05:06.907 INFO:teuthology.orchestra.run.smithi159.stdout: "req_rmdir_latency": { 2024-09-10T10:05:06.908 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 94, 2024-09-10T10:05:06.908 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 1.094932946, 2024-09-10T10:05:06.908 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.011648222 2024-09-10T10:05:06.908 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-10T10:05:06.908 INFO:teuthology.orchestra.run.smithi159.stdout: "req_rmsnap_latency": { 2024-09-10T10:05:06.908 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-10T10:05:06.908 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-10T10:05:06.908 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-10T10:05:06.908 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-10T10:05:06.908 INFO:teuthology.orchestra.run.smithi159.stdout: "req_rmxattr_latency": { 2024-09-10T10:05:06.908 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-10T10:05:06.908 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-10T10:05:06.908 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-10T10:05:06.908 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-10T10:05:06.909 INFO:teuthology.orchestra.run.smithi159.stdout: "req_setattr_latency": { 2024-09-10T10:05:06.909 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 297, 2024-09-10T10:05:06.909 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 220.778395307, 2024-09-10T10:05:06.909 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.743361600 2024-09-10T10:05:06.909 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-10T10:05:06.909 INFO:teuthology.orchestra.run.smithi159.stdout: "req_setdirlayout_latency": { 2024-09-10T10:05:06.909 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-10T10:05:06.909 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-10T10:05:06.909 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-10T10:05:06.909 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-10T10:05:06.909 INFO:teuthology.orchestra.run.smithi159.stdout: "req_setfilelock_latency": { 2024-09-10T10:05:06.909 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 1820, 2024-09-10T10:05:06.909 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 5.116486826, 2024-09-10T10:05:06.910 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.002811256 2024-09-10T10:05:06.910 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-10T10:05:06.910 INFO:teuthology.orchestra.run.smithi159.stdout: "req_setlayout_latency": { 2024-09-10T10:05:06.910 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-10T10:05:06.910 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-10T10:05:06.910 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-10T10:05:06.910 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-10T10:05:06.910 INFO:teuthology.orchestra.run.smithi159.stdout: "req_setxattr_latency": { 2024-09-10T10:05:06.910 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-10T10:05:06.910 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-10T10:05:06.910 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-10T10:05:06.910 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-10T10:05:06.910 INFO:teuthology.orchestra.run.smithi159.stdout: "req_symlink_latency": { 2024-09-10T10:05:06.910 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-10T10:05:06.910 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-10T10:05:06.911 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-10T10:05:06.911 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-10T10:05:06.911 INFO:teuthology.orchestra.run.smithi159.stdout: "req_unlink_latency": { 2024-09-10T10:05:06.911 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 61354, 2024-09-10T10:05:06.911 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 602.254090838, 2024-09-10T10:05:06.911 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.009816052 2024-09-10T10:05:06.911 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-10T10:05:06.911 INFO:teuthology.orchestra.run.smithi159.stdout: "cap_revoke_eviction": 0, 2024-09-10T10:05:06.911 INFO:teuthology.orchestra.run.smithi159.stdout: "cap_acquisition_throttle": 0, 2024-09-10T10:05:06.911 INFO:teuthology.orchestra.run.smithi159.stdout: "req_getvxattr_latency": { 2024-09-10T10:05:06.911 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-10T10:05:06.911 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-10T10:05:06.911 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-10T10:05:06.911 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-10T10:05:06.911 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-10T10:05:06.912 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_sessions": { 2024-09-10T10:05:06.912 INFO:teuthology.orchestra.run.smithi159.stdout: "session_count": 1, 2024-09-10T10:05:06.912 INFO:teuthology.orchestra.run.smithi159.stdout: "session_add": 1, 2024-09-10T10:05:06.912 INFO:teuthology.orchestra.run.smithi159.stdout: "session_remove": 0, 2024-09-10T10:05:06.912 INFO:teuthology.orchestra.run.smithi159.stdout: "sessions_open": 1, 2024-09-10T10:05:06.912 INFO:teuthology.orchestra.run.smithi159.stdout: "sessions_stale": 0, 2024-09-10T10:05:06.912 INFO:teuthology.orchestra.run.smithi159.stdout: "total_load": 18930, 2024-09-10T10:05:06.912 INFO:teuthology.orchestra.run.smithi159.stdout: "average_load": 18930, 2024-09-10T10:05:06.912 INFO:teuthology.orchestra.run.smithi159.stdout: "avg_session_uptime": 1484, 2024-09-10T10:05:06.912 INFO:teuthology.orchestra.run.smithi159.stdout: "mdthresh_evicted": 0 2024-09-10T10:05:06.912 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-10T10:05:06.912 INFO:teuthology.orchestra.run.smithi159.stdout: "mempool": { 2024-09-10T10:05:06.912 INFO:teuthology.orchestra.run.smithi159.stdout: "bloom_filter_bytes": 0, 2024-09-10T10:05:06.912 INFO:teuthology.orchestra.run.smithi159.stdout: "bloom_filter_items": 0, 2024-09-10T10:05:06.912 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_alloc_bytes": 0, 2024-09-10T10:05:06.913 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_alloc_items": 0, 2024-09-10T10:05:06.913 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_data_bytes": 0, 2024-09-10T10:05:06.913 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_data_items": 0, 2024-09-10T10:05:06.913 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-10T10:05:06.913 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_onode_items": 0, 2024-09-10T10:05:06.913 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-10T10:05:06.913 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_meta_items": 0, 2024-09-10T10:05:06.913 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_other_bytes": 0, 2024-09-10T10:05:06.913 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_other_items": 0, 2024-09-10T10:05:06.913 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-10T10:05:06.913 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_buffer_items": 0, 2024-09-10T10:05:06.913 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_extent_bytes": 0, 2024-09-10T10:05:06.914 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_extent_items": 0, 2024-09-10T10:05:06.914 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_blob_bytes": 0, 2024-09-10T10:05:06.914 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_blob_items": 0, 2024-09-10T10:05:06.914 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-10T10:05:06.914 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_shared_blob_items": 0, 2024-09-10T10:05:06.914 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-10T10:05:06.914 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_inline_bl_items": 0, 2024-09-10T10:05:06.914 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_fsck_bytes": 0, 2024-09-10T10:05:06.914 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_fsck_items": 0, 2024-09-10T10:05:06.914 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_txc_bytes": 0, 2024-09-10T10:05:06.914 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_txc_items": 0, 2024-09-10T10:05:06.914 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-10T10:05:06.914 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_writing_deferred_items": 0, 2024-09-10T10:05:06.914 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_writing_bytes": 0, 2024-09-10T10:05:06.914 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_writing_items": 0, 2024-09-10T10:05:06.914 INFO:teuthology.orchestra.run.smithi159.stdout: "bluefs_bytes": 0, 2024-09-10T10:05:06.915 INFO:teuthology.orchestra.run.smithi159.stdout: "bluefs_items": 0, 2024-09-10T10:05:06.915 INFO:teuthology.orchestra.run.smithi159.stdout: "bluefs_file_reader_bytes": 0, 2024-09-10T10:05:06.915 INFO:teuthology.orchestra.run.smithi159.stdout: "bluefs_file_reader_items": 0, 2024-09-10T10:05:06.915 INFO:teuthology.orchestra.run.smithi159.stdout: "bluefs_file_writer_bytes": 0, 2024-09-10T10:05:06.915 INFO:teuthology.orchestra.run.smithi159.stdout: "bluefs_file_writer_items": 0, 2024-09-10T10:05:06.916 INFO:teuthology.orchestra.run.smithi159.stdout: "buffer_anon_bytes": 351319, 2024-09-10T10:05:06.916 INFO:teuthology.orchestra.run.smithi159.stdout: "buffer_anon_items": 139, 2024-09-10T10:05:06.916 INFO:teuthology.orchestra.run.smithi159.stdout: "buffer_meta_bytes": 528, 2024-09-10T10:05:06.916 INFO:teuthology.orchestra.run.smithi159.stdout: "buffer_meta_items": 6, 2024-09-10T10:05:06.916 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_bytes": 0, 2024-09-10T10:05:06.916 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_items": 0, 2024-09-10T10:05:06.917 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_mapbl_bytes": 0, 2024-09-10T10:05:06.917 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_mapbl_items": 0, 2024-09-10T10:05:06.917 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_pglog_bytes": 0, 2024-09-10T10:05:06.917 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_pglog_items": 0, 2024-09-10T10:05:06.917 INFO:teuthology.orchestra.run.smithi159.stdout: "osdmap_bytes": 9152, 2024-09-10T10:05:06.917 INFO:teuthology.orchestra.run.smithi159.stdout: "osdmap_items": 173, 2024-09-10T10:05:06.917 INFO:teuthology.orchestra.run.smithi159.stdout: "osdmap_mapping_bytes": 0, 2024-09-10T10:05:06.917 INFO:teuthology.orchestra.run.smithi159.stdout: "osdmap_mapping_items": 0, 2024-09-10T10:05:06.917 INFO:teuthology.orchestra.run.smithi159.stdout: "pgmap_bytes": 0, 2024-09-10T10:05:06.918 INFO:teuthology.orchestra.run.smithi159.stdout: "pgmap_items": 0, 2024-09-10T10:05:06.918 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_co_bytes": 2878905, 2024-09-10T10:05:06.918 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_co_items": 41023, 2024-09-10T10:05:06.918 INFO:teuthology.orchestra.run.smithi159.stdout: "unittest_1_bytes": 0, 2024-09-10T10:05:06.918 INFO:teuthology.orchestra.run.smithi159.stdout: "unittest_1_items": 0, 2024-09-10T10:05:06.918 INFO:teuthology.orchestra.run.smithi159.stdout: "unittest_2_bytes": 0, 2024-09-10T10:05:06.918 INFO:teuthology.orchestra.run.smithi159.stdout: "unittest_2_items": 0 2024-09-10T10:05:06.918 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-10T10:05:06.918 INFO:teuthology.orchestra.run.smithi159.stdout: "objecter": { 2024-09-10T10:05:06.918 INFO:teuthology.orchestra.run.smithi159.stdout: "op_active": 0, 2024-09-10T10:05:06.918 INFO:teuthology.orchestra.run.smithi159.stdout: "op_laggy": 0, 2024-09-10T10:05:06.918 INFO:teuthology.orchestra.run.smithi159.stdout: "op_send": 225899, 2024-09-10T10:05:06.918 INFO:teuthology.orchestra.run.smithi159.stdout: "op_send_bytes": 627849842, 2024-09-10T10:05:06.918 INFO:teuthology.orchestra.run.smithi159.stdout: "op_resend": 33, 2024-09-10T10:05:06.919 INFO:teuthology.orchestra.run.smithi159.stdout: "op_reply": 225899, 2024-09-10T10:05:06.919 INFO:teuthology.orchestra.run.smithi159.stdout: "op_latency": { 2024-09-10T10:05:06.919 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 225899, 2024-09-10T10:05:06.919 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 1721.291597803, 2024-09-10T10:05:06.919 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.007619739 2024-09-10T10:05:06.919 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-10T10:05:06.919 INFO:teuthology.orchestra.run.smithi159.stdout: "op_inflight": 0, 2024-09-10T10:05:06.919 INFO:teuthology.orchestra.run.smithi159.stdout: "oplen_avg": { 2024-09-10T10:05:06.919 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 225899, 2024-09-10T10:05:06.919 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 229344 2024-09-10T10:05:06.919 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-10T10:05:06.919 INFO:teuthology.orchestra.run.smithi159.stdout: "op": 225899, 2024-09-10T10:05:06.919 INFO:teuthology.orchestra.run.smithi159.stdout: "op_r": 8545, 2024-09-10T10:05:06.919 INFO:teuthology.orchestra.run.smithi159.stdout: "op_w": 217354, 2024-09-10T10:05:06.919 INFO:teuthology.orchestra.run.smithi159.stdout: "op_rmw": 0, 2024-09-10T10:05:06.920 INFO:teuthology.orchestra.run.smithi159.stdout: "op_pg": 0, 2024-09-10T10:05:06.920 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_stat": 1, 2024-09-10T10:05:06.920 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_create": 836, 2024-09-10T10:05:06.920 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_read": 1773, 2024-09-10T10:05:06.920 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_write": 91043, 2024-09-10T10:05:06.920 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_writefull": 886, 2024-09-10T10:05:06.920 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_writesame": 0, 2024-09-10T10:05:06.920 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_append": 0, 2024-09-10T10:05:06.920 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_zero": 0, 2024-09-10T10:05:06.920 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_truncate": 26, 2024-09-10T10:05:06.920 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_delete": 123221, 2024-09-10T10:05:06.920 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_mapext": 0, 2024-09-10T10:05:06.921 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_sparse_read": 0, 2024-09-10T10:05:06.921 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_clonerange": 0, 2024-09-10T10:05:06.921 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_getxattr": 6772, 2024-09-10T10:05:06.921 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_setxattr": 1306, 2024-09-10T10:05:06.921 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_cmpxattr": 0, 2024-09-10T10:05:06.921 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_rmxattr": 0, 2024-09-10T10:05:06.921 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_resetxattrs": 0, 2024-09-10T10:05:06.921 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_call": 0, 2024-09-10T10:05:06.921 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_watch": 0, 2024-09-10T10:05:06.921 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_notify": 0, 2024-09-10T10:05:06.921 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_src_cmpxattr": 0, 2024-09-10T10:05:06.921 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_pgls": 0, 2024-09-10T10:05:06.922 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_pgls_filter": 0, 2024-09-10T10:05:06.922 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_other": 0, 2024-09-10T10:05:06.922 INFO:teuthology.orchestra.run.smithi159.stdout: "linger_active": 0, 2024-09-10T10:05:06.922 INFO:teuthology.orchestra.run.smithi159.stdout: "linger_send": 0, 2024-09-10T10:05:06.922 INFO:teuthology.orchestra.run.smithi159.stdout: "linger_resend": 0, 2024-09-10T10:05:06.922 INFO:teuthology.orchestra.run.smithi159.stdout: "linger_ping": 0, 2024-09-10T10:05:06.922 INFO:teuthology.orchestra.run.smithi159.stdout: "poolop_active": 0, 2024-09-10T10:05:06.922 INFO:teuthology.orchestra.run.smithi159.stdout: "poolop_send": 0, 2024-09-10T10:05:06.922 INFO:teuthology.orchestra.run.smithi159.stdout: "poolop_resend": 0, 2024-09-10T10:05:06.922 INFO:teuthology.orchestra.run.smithi159.stdout: "poolstat_active": 0, 2024-09-10T10:05:06.922 INFO:teuthology.orchestra.run.smithi159.stdout: "poolstat_send": 0, 2024-09-10T10:05:06.922 INFO:teuthology.orchestra.run.smithi159.stdout: "poolstat_resend": 0, 2024-09-10T10:05:06.922 INFO:teuthology.orchestra.run.smithi159.stdout: "statfs_active": 0, 2024-09-10T10:05:06.922 INFO:teuthology.orchestra.run.smithi159.stdout: "statfs_send": 0, 2024-09-10T10:05:06.923 INFO:teuthology.orchestra.run.smithi159.stdout: "statfs_resend": 0, 2024-09-10T10:05:06.923 INFO:teuthology.orchestra.run.smithi159.stdout: "command_active": 0, 2024-09-10T10:05:06.923 INFO:teuthology.orchestra.run.smithi159.stdout: "command_send": 0, 2024-09-10T10:05:06.923 INFO:teuthology.orchestra.run.smithi159.stdout: "command_resend": 0, 2024-09-10T10:05:06.923 INFO:teuthology.orchestra.run.smithi159.stdout: "map_epoch": 96, 2024-09-10T10:05:06.923 INFO:teuthology.orchestra.run.smithi159.stdout: "map_full": 0, 2024-09-10T10:05:06.923 INFO:teuthology.orchestra.run.smithi159.stdout: "map_inc": 2, 2024-09-10T10:05:06.923 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_sessions": 12, 2024-09-10T10:05:06.923 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_session_open": 12, 2024-09-10T10:05:06.923 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_session_close": 0, 2024-09-10T10:05:06.923 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_laggy": 0, 2024-09-10T10:05:06.923 INFO:teuthology.orchestra.run.smithi159.stdout: "omap_wr": 2492, 2024-09-10T10:05:06.923 INFO:teuthology.orchestra.run.smithi159.stdout: "omap_rd": 0, 2024-09-10T10:05:06.924 INFO:teuthology.orchestra.run.smithi159.stdout: "omap_del": 988 2024-09-10T10:05:06.924 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-10T10:05:06.924 INFO:teuthology.orchestra.run.smithi159.stdout: "oft": { 2024-09-10T10:05:06.924 INFO:teuthology.orchestra.run.smithi159.stdout: "omap_total_objs": 1, 2024-09-10T10:05:06.924 INFO:teuthology.orchestra.run.smithi159.stdout: "omap_total_kv_pairs": 0, 2024-09-10T10:05:06.924 INFO:teuthology.orchestra.run.smithi159.stdout: "omap_total_updates": 0, 2024-09-10T10:05:06.924 INFO:teuthology.orchestra.run.smithi159.stdout: "omap_total_removes": 0 2024-09-10T10:05:06.924 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-10T10:05:06.924 INFO:teuthology.orchestra.run.smithi159.stdout: "osd-slow-ops": { 2024-09-10T10:05:06.924 INFO:teuthology.orchestra.run.smithi159.stdout: "slow_ops_count": 5396 2024-09-10T10:05:06.924 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-10T10:05:06.925 INFO:teuthology.orchestra.run.smithi159.stdout: "purge_queue": { 2024-09-10T10:05:06.925 INFO:teuthology.orchestra.run.smithi159.stdout: "pq_executing_ops": 0, 2024-09-10T10:05:06.925 INFO:teuthology.orchestra.run.smithi159.stdout: "pq_executing_ops_high_water": 10, 2024-09-10T10:05:06.925 INFO:teuthology.orchestra.run.smithi159.stdout: "pq_executing": 0, 2024-09-10T10:05:06.925 INFO:teuthology.orchestra.run.smithi159.stdout: "pq_executing_high_water": 9, 2024-09-10T10:05:06.925 INFO:teuthology.orchestra.run.smithi159.stdout: "pq_executed_ops": 122783, 2024-09-10T10:05:06.925 INFO:teuthology.orchestra.run.smithi159.stdout: "pq_executed": 61429, 2024-09-10T10:05:06.925 INFO:teuthology.orchestra.run.smithi159.stdout: "pq_item_in_journal": 0 2024-09-10T10:05:06.925 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-10T10:05:06.925 INFO:teuthology.orchestra.run.smithi159.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-10T10:05:06.925 INFO:teuthology.orchestra.run.smithi159.stdout: "val": 0, 2024-09-10T10:05:06.926 INFO:teuthology.orchestra.run.smithi159.stdout: "max": 104857600, 2024-09-10T10:05:06.926 INFO:teuthology.orchestra.run.smithi159.stdout: "get_started": 0, 2024-09-10T10:05:06.926 INFO:teuthology.orchestra.run.smithi159.stdout: "get": 1047102, 2024-09-10T10:05:06.926 INFO:teuthology.orchestra.run.smithi159.stdout: "get_sum": 205743146, 2024-09-10T10:05:06.926 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_fail": 0, 2024-09-10T10:05:06.926 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_success": 1047102, 2024-09-10T10:05:06.926 INFO:teuthology.orchestra.run.smithi159.stdout: "take": 0, 2024-09-10T10:05:06.926 INFO:teuthology.orchestra.run.smithi159.stdout: "take_sum": 0, 2024-09-10T10:05:06.926 INFO:teuthology.orchestra.run.smithi159.stdout: "put": 1047102, 2024-09-10T10:05:06.926 INFO:teuthology.orchestra.run.smithi159.stdout: "put_sum": 205743146, 2024-09-10T10:05:06.926 INFO:teuthology.orchestra.run.smithi159.stdout: "wait": { 2024-09-10T10:05:06.926 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-10T10:05:06.926 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-10T10:05:06.926 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-10T10:05:06.926 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-10T10:05:06.927 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-10T10:05:06.927 INFO:teuthology.orchestra.run.smithi159.stdout: "throttle-objecter_bytes": { 2024-09-10T10:05:06.927 INFO:teuthology.orchestra.run.smithi159.stdout: "val": 0, 2024-09-10T10:05:06.927 INFO:teuthology.orchestra.run.smithi159.stdout: "max": 104857600, 2024-09-10T10:05:06.927 INFO:teuthology.orchestra.run.smithi159.stdout: "get_started": 0, 2024-09-10T10:05:06.927 INFO:teuthology.orchestra.run.smithi159.stdout: "get": 0, 2024-09-10T10:05:06.927 INFO:teuthology.orchestra.run.smithi159.stdout: "get_sum": 0, 2024-09-10T10:05:06.927 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_fail": 0, 2024-09-10T10:05:06.927 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_success": 0, 2024-09-10T10:05:06.927 INFO:teuthology.orchestra.run.smithi159.stdout: "take": 225899, 2024-09-10T10:05:06.927 INFO:teuthology.orchestra.run.smithi159.stdout: "take_sum": 634545003, 2024-09-10T10:05:06.927 INFO:teuthology.orchestra.run.smithi159.stdout: "put": 225899, 2024-09-10T10:05:06.927 INFO:teuthology.orchestra.run.smithi159.stdout: "put_sum": 634545003, 2024-09-10T10:05:06.927 INFO:teuthology.orchestra.run.smithi159.stdout: "wait": { 2024-09-10T10:05:06.927 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-10T10:05:06.928 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-10T10:05:06.928 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-10T10:05:06.928 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-10T10:05:06.928 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-10T10:05:06.928 INFO:teuthology.orchestra.run.smithi159.stdout: "throttle-objecter_ops": { 2024-09-10T10:05:06.928 INFO:teuthology.orchestra.run.smithi159.stdout: "val": 0, 2024-09-10T10:05:06.928 INFO:teuthology.orchestra.run.smithi159.stdout: "max": 1024, 2024-09-10T10:05:06.928 INFO:teuthology.orchestra.run.smithi159.stdout: "get_started": 0, 2024-09-10T10:05:06.928 INFO:teuthology.orchestra.run.smithi159.stdout: "get": 0, 2024-09-10T10:05:06.928 INFO:teuthology.orchestra.run.smithi159.stdout: "get_sum": 0, 2024-09-10T10:05:06.928 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_fail": 0, 2024-09-10T10:05:06.928 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_success": 0, 2024-09-10T10:05:06.928 INFO:teuthology.orchestra.run.smithi159.stdout: "take": 225899, 2024-09-10T10:05:06.928 INFO:teuthology.orchestra.run.smithi159.stdout: "take_sum": 225899, 2024-09-10T10:05:06.929 INFO:teuthology.orchestra.run.smithi159.stdout: "put": 225899, 2024-09-10T10:05:06.929 INFO:teuthology.orchestra.run.smithi159.stdout: "put_sum": 225899, 2024-09-10T10:05:06.929 INFO:teuthology.orchestra.run.smithi159.stdout: "wait": { 2024-09-10T10:05:06.929 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-10T10:05:06.929 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-10T10:05:06.929 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-10T10:05:06.929 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-10T10:05:06.929 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-10T10:05:06.929 INFO:teuthology.orchestra.run.smithi159.stdout: "throttle-write_buf_throttle": { 2024-09-10T10:05:06.929 INFO:teuthology.orchestra.run.smithi159.stdout: "val": 0, 2024-09-10T10:05:06.930 INFO:teuthology.orchestra.run.smithi159.stdout: "max": 3758096384, 2024-09-10T10:05:06.930 INFO:teuthology.orchestra.run.smithi159.stdout: "get_started": 0, 2024-09-10T10:05:06.930 INFO:teuthology.orchestra.run.smithi159.stdout: "get": 61429, 2024-09-10T10:05:06.930 INFO:teuthology.orchestra.run.smithi159.stdout: "get_sum": 6695161, 2024-09-10T10:05:06.930 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_fail": 0, 2024-09-10T10:05:06.930 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_success": 61429, 2024-09-10T10:05:06.930 INFO:teuthology.orchestra.run.smithi159.stdout: "take": 0, 2024-09-10T10:05:06.930 INFO:teuthology.orchestra.run.smithi159.stdout: "take_sum": 0, 2024-09-10T10:05:06.930 INFO:teuthology.orchestra.run.smithi159.stdout: "put": 1774, 2024-09-10T10:05:06.930 INFO:teuthology.orchestra.run.smithi159.stdout: "put_sum": 6695161, 2024-09-10T10:05:06.930 INFO:teuthology.orchestra.run.smithi159.stdout: "wait": { 2024-09-10T10:05:06.930 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-10T10:05:06.930 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-10T10:05:06.930 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-10T10:05:06.930 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-10T10:05:06.931 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-10T10:05:06.931 INFO:teuthology.orchestra.run.smithi159.stdout: "throttle-write_buf_throttle-0x55625a9b4960": { 2024-09-10T10:05:06.931 INFO:teuthology.orchestra.run.smithi159.stdout: "val": 0, 2024-09-10T10:05:06.931 INFO:teuthology.orchestra.run.smithi159.stdout: "max": 3758096384, 2024-09-10T10:05:06.931 INFO:teuthology.orchestra.run.smithi159.stdout: "get_started": 0, 2024-09-10T10:05:06.931 INFO:teuthology.orchestra.run.smithi159.stdout: "get": 295089, 2024-09-10T10:05:06.931 INFO:teuthology.orchestra.run.smithi159.stdout: "get_sum": 611734582, 2024-09-10T10:05:06.931 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_fail": 0, 2024-09-10T10:05:06.931 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_success": 295089, 2024-09-10T10:05:06.931 INFO:teuthology.orchestra.run.smithi159.stdout: "take": 0, 2024-09-10T10:05:06.931 INFO:teuthology.orchestra.run.smithi159.stdout: "take_sum": 0, 2024-09-10T10:05:06.931 INFO:teuthology.orchestra.run.smithi159.stdout: "put": 89269, 2024-09-10T10:05:06.931 INFO:teuthology.orchestra.run.smithi159.stdout: "put_sum": 611734582, 2024-09-10T10:05:06.931 INFO:teuthology.orchestra.run.smithi159.stdout: "wait": { 2024-09-10T10:05:06.932 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-10T10:05:06.932 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-10T10:05:06.932 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-10T10:05:06.932 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-10T10:05:06.932 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-10T10:05:06.932 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-09-10T10:05:07.696 INFO:tasks.check_counter:Daemon mds.c mds.exported=19 2024-09-10T10:05:07.696 INFO:tasks.check_counter:Daemon mds.c mds.imported=17 2024-09-10T10:05:07.696 DEBUG:tasks.check_counter:Getting stats from f 2024-09-10T10:05:07.697 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph daemon mds.f perf dump 2024-09-10T10:05:07.944 INFO:teuthology.orchestra.run.smithi159.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.c/config 2024-09-10T10:05:08.061 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:05:07 smithi159 ceph-mon[35513]: pgmap v1196: 129 pgs: 129 active+clean; 551 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-09-10T10:05:08.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:05:07 smithi102 ceph-mon[35442]: pgmap v1196: 129 pgs: 129 active+clean; 551 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-09-10T10:05:08.306 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:05:07 smithi005 ceph-mon[30350]: pgmap v1196: 129 pgs: 129 active+clean; 551 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-09-10T10:05:08.509 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-09-10T10:05:08.509 INFO:teuthology.orchestra.run.smithi159.stdout: "AsyncMessenger::Worker-0": { 2024-09-10T10:05:08.509 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_messages": 3614, 2024-09-10T10:05:08.510 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_messages": 3614, 2024-09-10T10:05:08.510 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_bytes": 59790439, 2024-09-10T10:05:08.510 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_bytes": 1148438, 2024-09-10T10:05:08.510 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_created_connections": 8, 2024-09-10T10:05:08.510 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_active_connections": 4, 2024-09-10T10:05:08.510 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_total_time": 0.329415108, 2024-09-10T10:05:08.510 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_send_time": 0.087091140, 2024-09-10T10:05:08.511 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_recv_time": 0.118926891, 2024-09-10T10:05:08.511 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_fast_dispatch_time": 0.096944259, 2024-09-10T10:05:08.511 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T10:05:08.511 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 3613, 2024-09-10T10:05:08.511 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.126852558, 2024-09-10T10:05:08.511 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000035110 2024-09-10T10:05:08.511 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-10T10:05:08.512 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_handle_ack_lat": { 2024-09-10T10:05:08.512 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-10T10:05:08.512 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-10T10:05:08.512 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-10T10:05:08.512 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-10T10:05:08.512 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_encrypted_bytes": 3024, 2024-09-10T10:05:08.513 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_encrypted_bytes": 448 2024-09-10T10:05:08.513 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-10T10:05:08.513 INFO:teuthology.orchestra.run.smithi159.stdout: "AsyncMessenger::Worker-1": { 2024-09-10T10:05:08.513 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_messages": 1455, 2024-09-10T10:05:08.513 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_messages": 1819, 2024-09-10T10:05:08.513 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_bytes": 50701506, 2024-09-10T10:05:08.513 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_bytes": 1261640, 2024-09-10T10:05:08.514 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_created_connections": 8, 2024-09-10T10:05:08.514 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_active_connections": 6, 2024-09-10T10:05:08.514 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_total_time": 0.148640984, 2024-09-10T10:05:08.514 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_send_time": 0.049499928, 2024-09-10T10:05:08.514 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_recv_time": 0.053960509, 2024-09-10T10:05:08.514 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_fast_dispatch_time": 0.033813624, 2024-09-10T10:05:08.514 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T10:05:08.515 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 1817, 2024-09-10T10:05:08.515 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.074463870, 2024-09-10T10:05:08.515 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000040981 2024-09-10T10:05:08.515 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-10T10:05:08.515 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_handle_ack_lat": { 2024-09-10T10:05:08.515 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-10T10:05:08.515 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-10T10:05:08.515 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-10T10:05:08.516 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-10T10:05:08.516 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-10T10:05:08.516 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_encrypted_bytes": 799520 2024-09-10T10:05:08.516 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-10T10:05:08.516 INFO:teuthology.orchestra.run.smithi159.stdout: "AsyncMessenger::Worker-2": { 2024-09-10T10:05:08.516 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_messages": 1604, 2024-09-10T10:05:08.516 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_messages": 1576, 2024-09-10T10:05:08.517 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_bytes": 35689306, 2024-09-10T10:05:08.517 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_bytes": 644417, 2024-09-10T10:05:08.517 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_created_connections": 5, 2024-09-10T10:05:08.517 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_active_connections": 4, 2024-09-10T10:05:08.517 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_total_time": 0.167771077, 2024-09-10T10:05:08.517 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_send_time": 0.056967256, 2024-09-10T10:05:08.517 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_recv_time": 0.082479883, 2024-09-10T10:05:08.518 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_fast_dispatch_time": 0.028725603, 2024-09-10T10:05:08.518 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T10:05:08.518 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 1575, 2024-09-10T10:05:08.518 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.081740056, 2024-09-10T10:05:08.518 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000051898 2024-09-10T10:05:08.518 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-10T10:05:08.519 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_handle_ack_lat": { 2024-09-10T10:05:08.519 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-10T10:05:08.519 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-10T10:05:08.519 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-10T10:05:08.519 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-10T10:05:08.519 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_encrypted_bytes": 994016, 2024-09-10T10:05:08.520 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_encrypted_bytes": 298432 2024-09-10T10:05:08.520 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-10T10:05:08.520 INFO:teuthology.orchestra.run.smithi159.stdout: "cct": { 2024-09-10T10:05:08.520 INFO:teuthology.orchestra.run.smithi159.stdout: "total_workers": 1, 2024-09-10T10:05:08.520 INFO:teuthology.orchestra.run.smithi159.stdout: "unhealthy_workers": 0 2024-09-10T10:05:08.520 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-10T10:05:08.520 INFO:teuthology.orchestra.run.smithi159.stdout: "finisher-MDSRank": { 2024-09-10T10:05:08.521 INFO:teuthology.orchestra.run.smithi159.stdout: "queue_len": 0, 2024-09-10T10:05:08.521 INFO:teuthology.orchestra.run.smithi159.stdout: "complete_latency": { 2024-09-10T10:05:08.521 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 10406, 2024-09-10T10:05:08.521 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.233036679, 2024-09-10T10:05:08.521 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000022394 2024-09-10T10:05:08.521 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-10T10:05:08.521 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-10T10:05:08.521 INFO:teuthology.orchestra.run.smithi159.stdout: "finisher-PurgeQueue": { 2024-09-10T10:05:08.522 INFO:teuthology.orchestra.run.smithi159.stdout: "queue_len": 0, 2024-09-10T10:05:08.522 INFO:teuthology.orchestra.run.smithi159.stdout: "complete_latency": { 2024-09-10T10:05:08.522 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-10T10:05:08.522 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-10T10:05:08.522 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-10T10:05:08.522 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-10T10:05:08.522 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-10T10:05:08.523 INFO:teuthology.orchestra.run.smithi159.stdout: "mds": { 2024-09-10T10:05:08.523 INFO:teuthology.orchestra.run.smithi159.stdout: "request": 0, 2024-09-10T10:05:08.523 INFO:teuthology.orchestra.run.smithi159.stdout: "reply": 0, 2024-09-10T10:05:08.523 INFO:teuthology.orchestra.run.smithi159.stdout: "reply_latency": { 2024-09-10T10:05:08.523 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-10T10:05:08.523 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-10T10:05:08.523 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-10T10:05:08.524 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-10T10:05:08.524 INFO:teuthology.orchestra.run.smithi159.stdout: "slow_reply": 0, 2024-09-10T10:05:08.524 INFO:teuthology.orchestra.run.smithi159.stdout: "forward": 0, 2024-09-10T10:05:08.524 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_fetch_complete": 0, 2024-09-10T10:05:08.524 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_fetch_keys": 0, 2024-09-10T10:05:08.524 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_commit": 0, 2024-09-10T10:05:08.524 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_split": 0, 2024-09-10T10:05:08.525 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_merge": 0, 2024-09-10T10:05:08.525 INFO:teuthology.orchestra.run.smithi159.stdout: "inodes": 13678, 2024-09-10T10:05:08.525 INFO:teuthology.orchestra.run.smithi159.stdout: "inodes_top": 0, 2024-09-10T10:05:08.525 INFO:teuthology.orchestra.run.smithi159.stdout: "inodes_bottom": 13678, 2024-09-10T10:05:08.525 INFO:teuthology.orchestra.run.smithi159.stdout: "inodes_pin_tail": 0, 2024-09-10T10:05:08.526 INFO:teuthology.orchestra.run.smithi159.stdout: "inodes_pinned": 13678, 2024-09-10T10:05:08.526 INFO:teuthology.orchestra.run.smithi159.stdout: "inodes_expired": 0, 2024-09-10T10:05:08.526 INFO:teuthology.orchestra.run.smithi159.stdout: "inodes_with_caps": 0, 2024-09-10T10:05:08.526 INFO:teuthology.orchestra.run.smithi159.stdout: "caps": 0, 2024-09-10T10:05:08.526 INFO:teuthology.orchestra.run.smithi159.stdout: "subtrees": 2, 2024-09-10T10:05:08.526 INFO:teuthology.orchestra.run.smithi159.stdout: "traverse": 0, 2024-09-10T10:05:08.526 INFO:teuthology.orchestra.run.smithi159.stdout: "traverse_hit": 0, 2024-09-10T10:05:08.526 INFO:teuthology.orchestra.run.smithi159.stdout: "traverse_forward": 0, 2024-09-10T10:05:08.526 INFO:teuthology.orchestra.run.smithi159.stdout: "traverse_discover": 0, 2024-09-10T10:05:08.526 INFO:teuthology.orchestra.run.smithi159.stdout: "traverse_dir_fetch": 0, 2024-09-10T10:05:08.526 INFO:teuthology.orchestra.run.smithi159.stdout: "traverse_remote_ino": 0, 2024-09-10T10:05:08.526 INFO:teuthology.orchestra.run.smithi159.stdout: "traverse_lock": 0, 2024-09-10T10:05:08.527 INFO:teuthology.orchestra.run.smithi159.stdout: "load_cent": 0, 2024-09-10T10:05:08.527 INFO:teuthology.orchestra.run.smithi159.stdout: "q": 0, 2024-09-10T10:05:08.527 INFO:teuthology.orchestra.run.smithi159.stdout: "exported": 0, 2024-09-10T10:05:08.527 INFO:teuthology.orchestra.run.smithi159.stdout: "exported_inodes": 0, 2024-09-10T10:05:08.527 INFO:teuthology.orchestra.run.smithi159.stdout: "imported": 0, 2024-09-10T10:05:08.527 INFO:teuthology.orchestra.run.smithi159.stdout: "imported_inodes": 0, 2024-09-10T10:05:08.527 INFO:teuthology.orchestra.run.smithi159.stdout: "openino_dir_fetch": 0, 2024-09-10T10:05:08.527 INFO:teuthology.orchestra.run.smithi159.stdout: "openino_backtrace_fetch": 0, 2024-09-10T10:05:08.528 INFO:teuthology.orchestra.run.smithi159.stdout: "openino_peer_discover": 0, 2024-09-10T10:05:08.528 INFO:teuthology.orchestra.run.smithi159.stdout: "root_rfiles": 4, 2024-09-10T10:05:08.528 INFO:teuthology.orchestra.run.smithi159.stdout: "root_rbytes": 234, 2024-09-10T10:05:08.528 INFO:teuthology.orchestra.run.smithi159.stdout: "root_rsnaps": 0, 2024-09-10T10:05:08.528 INFO:teuthology.orchestra.run.smithi159.stdout: "scrub_backtrace_fetch": 0, 2024-09-10T10:05:08.528 INFO:teuthology.orchestra.run.smithi159.stdout: "scrub_set_tag": 0, 2024-09-10T10:05:08.528 INFO:teuthology.orchestra.run.smithi159.stdout: "scrub_backtrace_repaired": 0, 2024-09-10T10:05:08.528 INFO:teuthology.orchestra.run.smithi159.stdout: "scrub_inotable_repaired": 0, 2024-09-10T10:05:08.528 INFO:teuthology.orchestra.run.smithi159.stdout: "scrub_dir_inodes": 0, 2024-09-10T10:05:08.529 INFO:teuthology.orchestra.run.smithi159.stdout: "scrub_dir_base_inodes": 0, 2024-09-10T10:05:08.529 INFO:teuthology.orchestra.run.smithi159.stdout: "scrub_dirfrag_rstats": 0, 2024-09-10T10:05:08.529 INFO:teuthology.orchestra.run.smithi159.stdout: "scrub_file_inodes": 0, 2024-09-10T10:05:08.529 INFO:teuthology.orchestra.run.smithi159.stdout: "handle_inode_file_caps": 0, 2024-09-10T10:05:08.529 INFO:teuthology.orchestra.run.smithi159.stdout: "ceph_cap_op_revoke": 0, 2024-09-10T10:05:08.529 INFO:teuthology.orchestra.run.smithi159.stdout: "ceph_cap_op_grant": 0, 2024-09-10T10:05:08.529 INFO:teuthology.orchestra.run.smithi159.stdout: "ceph_cap_op_trunc": 0, 2024-09-10T10:05:08.529 INFO:teuthology.orchestra.run.smithi159.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-10T10:05:08.530 INFO:teuthology.orchestra.run.smithi159.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-10T10:05:08.530 INFO:teuthology.orchestra.run.smithi159.stdout: "handle_client_caps": 0, 2024-09-10T10:05:08.530 INFO:teuthology.orchestra.run.smithi159.stdout: "handle_client_caps_dirty": 0, 2024-09-10T10:05:08.530 INFO:teuthology.orchestra.run.smithi159.stdout: "handle_client_cap_release": 0, 2024-09-10T10:05:08.530 INFO:teuthology.orchestra.run.smithi159.stdout: "process_request_cap_release": 0 2024-09-10T10:05:08.530 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-10T10:05:08.531 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_cache": { 2024-09-10T10:05:08.531 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_update": 0, 2024-09-10T10:05:08.531 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_update_receipt": 0, 2024-09-10T10:05:08.531 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_try_discover": 0, 2024-09-10T10:05:08.531 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_send_discover": 0, 2024-09-10T10:05:08.531 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_handle_discover": 0, 2024-09-10T10:05:08.531 INFO:teuthology.orchestra.run.smithi159.stdout: "num_strays": 0, 2024-09-10T10:05:08.531 INFO:teuthology.orchestra.run.smithi159.stdout: "num_strays_delayed": 0, 2024-09-10T10:05:08.531 INFO:teuthology.orchestra.run.smithi159.stdout: "num_strays_enqueuing": 0, 2024-09-10T10:05:08.531 INFO:teuthology.orchestra.run.smithi159.stdout: "strays_created": 0, 2024-09-10T10:05:08.531 INFO:teuthology.orchestra.run.smithi159.stdout: "strays_enqueued": 0, 2024-09-10T10:05:08.532 INFO:teuthology.orchestra.run.smithi159.stdout: "strays_reintegrated": 0, 2024-09-10T10:05:08.532 INFO:teuthology.orchestra.run.smithi159.stdout: "strays_migrated": 0, 2024-09-10T10:05:08.532 INFO:teuthology.orchestra.run.smithi159.stdout: "num_recovering_processing": 0, 2024-09-10T10:05:08.532 INFO:teuthology.orchestra.run.smithi159.stdout: "num_recovering_enqueued": 0, 2024-09-10T10:05:08.532 INFO:teuthology.orchestra.run.smithi159.stdout: "num_recovering_prioritized": 0, 2024-09-10T10:05:08.532 INFO:teuthology.orchestra.run.smithi159.stdout: "recovery_started": 0, 2024-09-10T10:05:08.532 INFO:teuthology.orchestra.run.smithi159.stdout: "recovery_completed": 0, 2024-09-10T10:05:08.532 INFO:teuthology.orchestra.run.smithi159.stdout: "ireq_quiesce_path": 0, 2024-09-10T10:05:08.532 INFO:teuthology.orchestra.run.smithi159.stdout: "ireq_quiesce_inode": 0, 2024-09-10T10:05:08.532 INFO:teuthology.orchestra.run.smithi159.stdout: "ireq_enqueue_scrub": 0, 2024-09-10T10:05:08.532 INFO:teuthology.orchestra.run.smithi159.stdout: "ireq_exportdir": 0, 2024-09-10T10:05:08.532 INFO:teuthology.orchestra.run.smithi159.stdout: "ireq_flush": 0, 2024-09-10T10:05:08.533 INFO:teuthology.orchestra.run.smithi159.stdout: "ireq_fragmentdir": 0, 2024-09-10T10:05:08.533 INFO:teuthology.orchestra.run.smithi159.stdout: "ireq_fragstats": 0, 2024-09-10T10:05:08.533 INFO:teuthology.orchestra.run.smithi159.stdout: "ireq_inodestats": 0 2024-09-10T10:05:08.533 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-10T10:05:08.533 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_log": { 2024-09-10T10:05:08.533 INFO:teuthology.orchestra.run.smithi159.stdout: "evlrg": 0, 2024-09-10T10:05:08.533 INFO:teuthology.orchestra.run.smithi159.stdout: "evadd": 0, 2024-09-10T10:05:08.533 INFO:teuthology.orchestra.run.smithi159.stdout: "evex": 0, 2024-09-10T10:05:08.533 INFO:teuthology.orchestra.run.smithi159.stdout: "evtrm": 0, 2024-09-10T10:05:08.533 INFO:teuthology.orchestra.run.smithi159.stdout: "ev": 81538, 2024-09-10T10:05:08.533 INFO:teuthology.orchestra.run.smithi159.stdout: "evexg": 0, 2024-09-10T10:05:08.533 INFO:teuthology.orchestra.run.smithi159.stdout: "evexd": 0, 2024-09-10T10:05:08.534 INFO:teuthology.orchestra.run.smithi159.stdout: "segadd": 0, 2024-09-10T10:05:08.534 INFO:teuthology.orchestra.run.smithi159.stdout: "segex": 0, 2024-09-10T10:05:08.534 INFO:teuthology.orchestra.run.smithi159.stdout: "segtrm": 0, 2024-09-10T10:05:08.534 INFO:teuthology.orchestra.run.smithi159.stdout: "seg": 100, 2024-09-10T10:05:08.534 INFO:teuthology.orchestra.run.smithi159.stdout: "segmjr": 8, 2024-09-10T10:05:08.534 INFO:teuthology.orchestra.run.smithi159.stdout: "segexg": 0, 2024-09-10T10:05:08.534 INFO:teuthology.orchestra.run.smithi159.stdout: "segexd": 0, 2024-09-10T10:05:08.535 INFO:teuthology.orchestra.run.smithi159.stdout: "expos": 4194304, 2024-09-10T10:05:08.535 INFO:teuthology.orchestra.run.smithi159.stdout: "wrpos": 147656294, 2024-09-10T10:05:08.535 INFO:teuthology.orchestra.run.smithi159.stdout: "rdpos": 147655156, 2024-09-10T10:05:08.535 INFO:teuthology.orchestra.run.smithi159.stdout: "jlat": { 2024-09-10T10:05:08.535 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-10T10:05:08.535 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-10T10:05:08.535 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-10T10:05:08.535 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-10T10:05:08.535 INFO:teuthology.orchestra.run.smithi159.stdout: "replayed": 81538 2024-09-10T10:05:08.535 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-10T10:05:08.535 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_mem": { 2024-09-10T10:05:08.536 INFO:teuthology.orchestra.run.smithi159.stdout: "ino": 24, 2024-09-10T10:05:08.536 INFO:teuthology.orchestra.run.smithi159.stdout: "ino+": 13721, 2024-09-10T10:05:08.536 INFO:teuthology.orchestra.run.smithi159.stdout: "ino-": 13697, 2024-09-10T10:05:08.536 INFO:teuthology.orchestra.run.smithi159.stdout: "dir": 19, 2024-09-10T10:05:08.536 INFO:teuthology.orchestra.run.smithi159.stdout: "dir+": 46, 2024-09-10T10:05:08.536 INFO:teuthology.orchestra.run.smithi159.stdout: "dir-": 27, 2024-09-10T10:05:08.536 INFO:teuthology.orchestra.run.smithi159.stdout: "dn": 13678, 2024-09-10T10:05:08.536 INFO:teuthology.orchestra.run.smithi159.stdout: "dn+": 13955, 2024-09-10T10:05:08.536 INFO:teuthology.orchestra.run.smithi159.stdout: "dn-": 277, 2024-09-10T10:05:08.536 INFO:teuthology.orchestra.run.smithi159.stdout: "cap": 0, 2024-09-10T10:05:08.536 INFO:teuthology.orchestra.run.smithi159.stdout: "cap+": 0, 2024-09-10T10:05:08.536 INFO:teuthology.orchestra.run.smithi159.stdout: "cap-": 0, 2024-09-10T10:05:08.536 INFO:teuthology.orchestra.run.smithi159.stdout: "rss": 64128, 2024-09-10T10:05:08.537 INFO:teuthology.orchestra.run.smithi159.stdout: "heap": 215328 2024-09-10T10:05:08.537 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-10T10:05:08.537 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_server": { 2024-09-10T10:05:08.537 INFO:teuthology.orchestra.run.smithi159.stdout: "dispatch_client_request": 0, 2024-09-10T10:05:08.537 INFO:teuthology.orchestra.run.smithi159.stdout: "dispatch_server_request": 0, 2024-09-10T10:05:08.537 INFO:teuthology.orchestra.run.smithi159.stdout: "handle_client_request": 0, 2024-09-10T10:05:08.537 INFO:teuthology.orchestra.run.smithi159.stdout: "handle_client_session": 0, 2024-09-10T10:05:08.537 INFO:teuthology.orchestra.run.smithi159.stdout: "handle_peer_request": 0, 2024-09-10T10:05:08.537 INFO:teuthology.orchestra.run.smithi159.stdout: "req_create_latency": { 2024-09-10T10:05:08.537 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-10T10:05:08.537 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-10T10:05:08.537 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-10T10:05:08.537 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-10T10:05:08.538 INFO:teuthology.orchestra.run.smithi159.stdout: "req_getattr_latency": { 2024-09-10T10:05:08.538 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-10T10:05:08.538 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-10T10:05:08.538 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-10T10:05:08.538 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-10T10:05:08.538 INFO:teuthology.orchestra.run.smithi159.stdout: "req_getfilelock_latency": { 2024-09-10T10:05:08.538 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-10T10:05:08.538 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-10T10:05:08.538 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-10T10:05:08.538 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-10T10:05:08.538 INFO:teuthology.orchestra.run.smithi159.stdout: "req_link_latency": { 2024-09-10T10:05:08.538 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-10T10:05:08.538 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-10T10:05:08.538 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-10T10:05:08.538 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-10T10:05:08.539 INFO:teuthology.orchestra.run.smithi159.stdout: "req_lookup_latency": { 2024-09-10T10:05:08.539 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-10T10:05:08.539 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-10T10:05:08.539 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-10T10:05:08.539 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-10T10:05:08.539 INFO:teuthology.orchestra.run.smithi159.stdout: "req_lookuphash_latency": { 2024-09-10T10:05:08.539 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-10T10:05:08.539 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-10T10:05:08.539 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-10T10:05:08.539 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-10T10:05:08.539 INFO:teuthology.orchestra.run.smithi159.stdout: "req_lookupino_latency": { 2024-09-10T10:05:08.539 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-10T10:05:08.539 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-10T10:05:08.540 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-10T10:05:08.540 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-10T10:05:08.540 INFO:teuthology.orchestra.run.smithi159.stdout: "req_lookupname_latency": { 2024-09-10T10:05:08.540 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-10T10:05:08.540 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-10T10:05:08.540 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-10T10:05:08.540 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-10T10:05:08.540 INFO:teuthology.orchestra.run.smithi159.stdout: "req_lookupparent_latency": { 2024-09-10T10:05:08.540 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-10T10:05:08.540 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-10T10:05:08.540 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-10T10:05:08.540 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-10T10:05:08.540 INFO:teuthology.orchestra.run.smithi159.stdout: "req_lookupsnap_latency": { 2024-09-10T10:05:08.540 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-10T10:05:08.541 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-10T10:05:08.541 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-10T10:05:08.541 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-10T10:05:08.541 INFO:teuthology.orchestra.run.smithi159.stdout: "req_lssnap_latency": { 2024-09-10T10:05:08.541 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-10T10:05:08.541 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-10T10:05:08.541 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-10T10:05:08.541 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-10T10:05:08.542 INFO:teuthology.orchestra.run.smithi159.stdout: "req_mkdir_latency": { 2024-09-10T10:05:08.542 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-10T10:05:08.542 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-10T10:05:08.542 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-10T10:05:08.542 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-10T10:05:08.542 INFO:teuthology.orchestra.run.smithi159.stdout: "req_mknod_latency": { 2024-09-10T10:05:08.542 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-10T10:05:08.542 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-10T10:05:08.542 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-10T10:05:08.543 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-10T10:05:08.543 INFO:teuthology.orchestra.run.smithi159.stdout: "req_mksnap_latency": { 2024-09-10T10:05:08.543 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-10T10:05:08.543 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-10T10:05:08.543 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-10T10:05:08.543 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-10T10:05:08.543 INFO:teuthology.orchestra.run.smithi159.stdout: "req_open_latency": { 2024-09-10T10:05:08.543 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-10T10:05:08.543 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-10T10:05:08.543 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-10T10:05:08.543 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-10T10:05:08.543 INFO:teuthology.orchestra.run.smithi159.stdout: "req_readdir_latency": { 2024-09-10T10:05:08.544 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-10T10:05:08.544 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-10T10:05:08.544 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-10T10:05:08.544 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-10T10:05:08.544 INFO:teuthology.orchestra.run.smithi159.stdout: "req_rename_latency": { 2024-09-10T10:05:08.544 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-10T10:05:08.544 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-10T10:05:08.544 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-10T10:05:08.544 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-10T10:05:08.544 INFO:teuthology.orchestra.run.smithi159.stdout: "req_renamesnap_latency": { 2024-09-10T10:05:08.544 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-10T10:05:08.545 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-10T10:05:08.545 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-10T10:05:08.545 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-10T10:05:08.545 INFO:teuthology.orchestra.run.smithi159.stdout: "req_snapdiff_latency": { 2024-09-10T10:05:08.545 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-10T10:05:08.545 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-10T10:05:08.545 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-10T10:05:08.545 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-10T10:05:08.545 INFO:teuthology.orchestra.run.smithi159.stdout: "req_rmdir_latency": { 2024-09-10T10:05:08.545 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-10T10:05:08.545 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-10T10:05:08.545 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-10T10:05:08.545 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-10T10:05:08.545 INFO:teuthology.orchestra.run.smithi159.stdout: "req_rmsnap_latency": { 2024-09-10T10:05:08.546 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-10T10:05:08.546 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-10T10:05:08.546 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-10T10:05:08.546 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-10T10:05:08.546 INFO:teuthology.orchestra.run.smithi159.stdout: "req_rmxattr_latency": { 2024-09-10T10:05:08.546 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-10T10:05:08.546 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-10T10:05:08.547 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-10T10:05:08.547 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-10T10:05:08.547 INFO:teuthology.orchestra.run.smithi159.stdout: "req_setattr_latency": { 2024-09-10T10:05:08.547 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-10T10:05:08.547 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-10T10:05:08.547 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-10T10:05:08.547 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-10T10:05:08.547 INFO:teuthology.orchestra.run.smithi159.stdout: "req_setdirlayout_latency": { 2024-09-10T10:05:08.547 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-10T10:05:08.547 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-10T10:05:08.548 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-10T10:05:08.548 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-10T10:05:08.548 INFO:teuthology.orchestra.run.smithi159.stdout: "req_setfilelock_latency": { 2024-09-10T10:05:08.548 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-10T10:05:08.548 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-10T10:05:08.548 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-10T10:05:08.548 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-10T10:05:08.548 INFO:teuthology.orchestra.run.smithi159.stdout: "req_setlayout_latency": { 2024-09-10T10:05:08.548 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-10T10:05:08.548 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-10T10:05:08.549 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-10T10:05:08.549 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-10T10:05:08.549 INFO:teuthology.orchestra.run.smithi159.stdout: "req_setxattr_latency": { 2024-09-10T10:05:08.549 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-10T10:05:08.549 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-10T10:05:08.549 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-10T10:05:08.549 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-10T10:05:08.549 INFO:teuthology.orchestra.run.smithi159.stdout: "req_symlink_latency": { 2024-09-10T10:05:08.549 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-10T10:05:08.549 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-10T10:05:08.549 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-10T10:05:08.549 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-10T10:05:08.549 INFO:teuthology.orchestra.run.smithi159.stdout: "req_unlink_latency": { 2024-09-10T10:05:08.549 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-10T10:05:08.550 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-10T10:05:08.550 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-10T10:05:08.550 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-10T10:05:08.550 INFO:teuthology.orchestra.run.smithi159.stdout: "cap_revoke_eviction": 0, 2024-09-10T10:05:08.550 INFO:teuthology.orchestra.run.smithi159.stdout: "cap_acquisition_throttle": 0, 2024-09-10T10:05:08.550 INFO:teuthology.orchestra.run.smithi159.stdout: "req_getvxattr_latency": { 2024-09-10T10:05:08.550 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-10T10:05:08.550 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-10T10:05:08.550 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-10T10:05:08.550 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-10T10:05:08.550 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-10T10:05:08.550 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_sessions": { 2024-09-10T10:05:08.550 INFO:teuthology.orchestra.run.smithi159.stdout: "session_count": 1, 2024-09-10T10:05:08.551 INFO:teuthology.orchestra.run.smithi159.stdout: "session_add": 4, 2024-09-10T10:05:08.551 INFO:teuthology.orchestra.run.smithi159.stdout: "session_remove": 3, 2024-09-10T10:05:08.551 INFO:teuthology.orchestra.run.smithi159.stdout: "sessions_open": 2, 2024-09-10T10:05:08.551 INFO:teuthology.orchestra.run.smithi159.stdout: "sessions_stale": 0, 2024-09-10T10:05:08.551 INFO:teuthology.orchestra.run.smithi159.stdout: "total_load": 0, 2024-09-10T10:05:08.551 INFO:teuthology.orchestra.run.smithi159.stdout: "average_load": 0, 2024-09-10T10:05:08.551 INFO:teuthology.orchestra.run.smithi159.stdout: "avg_session_uptime": 4395, 2024-09-10T10:05:08.551 INFO:teuthology.orchestra.run.smithi159.stdout: "mdthresh_evicted": 0 2024-09-10T10:05:08.551 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-10T10:05:08.551 INFO:teuthology.orchestra.run.smithi159.stdout: "mempool": { 2024-09-10T10:05:08.551 INFO:teuthology.orchestra.run.smithi159.stdout: "bloom_filter_bytes": 0, 2024-09-10T10:05:08.551 INFO:teuthology.orchestra.run.smithi159.stdout: "bloom_filter_items": 0, 2024-09-10T10:05:08.551 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_alloc_bytes": 0, 2024-09-10T10:05:08.551 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_alloc_items": 0, 2024-09-10T10:05:08.552 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_data_bytes": 0, 2024-09-10T10:05:08.552 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_data_items": 0, 2024-09-10T10:05:08.552 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-10T10:05:08.552 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_onode_items": 0, 2024-09-10T10:05:08.552 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-10T10:05:08.552 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_meta_items": 0, 2024-09-10T10:05:08.552 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_other_bytes": 0, 2024-09-10T10:05:08.552 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_other_items": 0, 2024-09-10T10:05:08.552 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-10T10:05:08.552 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_buffer_items": 0, 2024-09-10T10:05:08.552 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_extent_bytes": 0, 2024-09-10T10:05:08.552 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_extent_items": 0, 2024-09-10T10:05:08.552 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_blob_bytes": 0, 2024-09-10T10:05:08.553 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_blob_items": 0, 2024-09-10T10:05:08.553 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-10T10:05:08.553 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_shared_blob_items": 0, 2024-09-10T10:05:08.553 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-10T10:05:08.553 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_inline_bl_items": 0, 2024-09-10T10:05:08.553 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_fsck_bytes": 0, 2024-09-10T10:05:08.553 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_fsck_items": 0, 2024-09-10T10:05:08.554 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_txc_bytes": 0, 2024-09-10T10:05:08.554 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_txc_items": 0, 2024-09-10T10:05:08.554 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-10T10:05:08.554 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_writing_deferred_items": 0, 2024-09-10T10:05:08.554 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_writing_bytes": 0, 2024-09-10T10:05:08.554 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_writing_items": 0, 2024-09-10T10:05:08.554 INFO:teuthology.orchestra.run.smithi159.stdout: "bluefs_bytes": 0, 2024-09-10T10:05:08.554 INFO:teuthology.orchestra.run.smithi159.stdout: "bluefs_items": 0, 2024-09-10T10:05:08.554 INFO:teuthology.orchestra.run.smithi159.stdout: "bluefs_file_reader_bytes": 0, 2024-09-10T10:05:08.555 INFO:teuthology.orchestra.run.smithi159.stdout: "bluefs_file_reader_items": 0, 2024-09-10T10:05:08.555 INFO:teuthology.orchestra.run.smithi159.stdout: "bluefs_file_writer_bytes": 0, 2024-09-10T10:05:08.555 INFO:teuthology.orchestra.run.smithi159.stdout: "bluefs_file_writer_items": 0, 2024-09-10T10:05:08.555 INFO:teuthology.orchestra.run.smithi159.stdout: "buffer_anon_bytes": 242604, 2024-09-10T10:05:08.555 INFO:teuthology.orchestra.run.smithi159.stdout: "buffer_anon_items": 87, 2024-09-10T10:05:08.555 INFO:teuthology.orchestra.run.smithi159.stdout: "buffer_meta_bytes": 0, 2024-09-10T10:05:08.555 INFO:teuthology.orchestra.run.smithi159.stdout: "buffer_meta_items": 0, 2024-09-10T10:05:08.555 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_bytes": 0, 2024-09-10T10:05:08.555 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_items": 0, 2024-09-10T10:05:08.556 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_mapbl_bytes": 0, 2024-09-10T10:05:08.556 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_mapbl_items": 0, 2024-09-10T10:05:08.556 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_pglog_bytes": 0, 2024-09-10T10:05:08.556 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_pglog_items": 0, 2024-09-10T10:05:08.556 INFO:teuthology.orchestra.run.smithi159.stdout: "osdmap_bytes": 9152, 2024-09-10T10:05:08.556 INFO:teuthology.orchestra.run.smithi159.stdout: "osdmap_items": 173, 2024-09-10T10:05:08.556 INFO:teuthology.orchestra.run.smithi159.stdout: "osdmap_mapping_bytes": 0, 2024-09-10T10:05:08.556 INFO:teuthology.orchestra.run.smithi159.stdout: "osdmap_mapping_items": 0, 2024-09-10T10:05:08.556 INFO:teuthology.orchestra.run.smithi159.stdout: "pgmap_bytes": 0, 2024-09-10T10:05:08.556 INFO:teuthology.orchestra.run.smithi159.stdout: "pgmap_items": 0, 2024-09-10T10:05:08.556 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_co_bytes": 9707661, 2024-09-10T10:05:08.556 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_co_items": 55124, 2024-09-10T10:05:08.556 INFO:teuthology.orchestra.run.smithi159.stdout: "unittest_1_bytes": 0, 2024-09-10T10:05:08.556 INFO:teuthology.orchestra.run.smithi159.stdout: "unittest_1_items": 0, 2024-09-10T10:05:08.557 INFO:teuthology.orchestra.run.smithi159.stdout: "unittest_2_bytes": 0, 2024-09-10T10:05:08.557 INFO:teuthology.orchestra.run.smithi159.stdout: "unittest_2_items": 0 2024-09-10T10:05:08.557 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-10T10:05:08.557 INFO:teuthology.orchestra.run.smithi159.stdout: "objecter": { 2024-09-10T10:05:08.557 INFO:teuthology.orchestra.run.smithi159.stdout: "op_active": 0, 2024-09-10T10:05:08.557 INFO:teuthology.orchestra.run.smithi159.stdout: "op_laggy": 0, 2024-09-10T10:05:08.557 INFO:teuthology.orchestra.run.smithi159.stdout: "op_send": 6151, 2024-09-10T10:05:08.557 INFO:teuthology.orchestra.run.smithi159.stdout: "op_send_bytes": 26, 2024-09-10T10:05:08.557 INFO:teuthology.orchestra.run.smithi159.stdout: "op_resend": 4, 2024-09-10T10:05:08.557 INFO:teuthology.orchestra.run.smithi159.stdout: "op_reply": 6151, 2024-09-10T10:05:08.557 INFO:teuthology.orchestra.run.smithi159.stdout: "op_latency": { 2024-09-10T10:05:08.558 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 6151, 2024-09-10T10:05:08.558 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 6.287874192, 2024-09-10T10:05:08.558 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.001022252 2024-09-10T10:05:08.558 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-10T10:05:08.558 INFO:teuthology.orchestra.run.smithi159.stdout: "op_inflight": 0, 2024-09-10T10:05:08.558 INFO:teuthology.orchestra.run.smithi159.stdout: "oplen_avg": { 2024-09-10T10:05:08.558 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 6151, 2024-09-10T10:05:08.558 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 6152 2024-09-10T10:05:08.558 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-10T10:05:08.558 INFO:teuthology.orchestra.run.smithi159.stdout: "op": 6151, 2024-09-10T10:05:08.558 INFO:teuthology.orchestra.run.smithi159.stdout: "op_r": 6151, 2024-09-10T10:05:08.558 INFO:teuthology.orchestra.run.smithi159.stdout: "op_w": 0, 2024-09-10T10:05:08.558 INFO:teuthology.orchestra.run.smithi159.stdout: "op_rmw": 0, 2024-09-10T10:05:08.559 INFO:teuthology.orchestra.run.smithi159.stdout: "op_pg": 0, 2024-09-10T10:05:08.559 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_stat": 3444, 2024-09-10T10:05:08.559 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_create": 0, 2024-09-10T10:05:08.559 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_read": 2704, 2024-09-10T10:05:08.559 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_write": 0, 2024-09-10T10:05:08.559 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_writefull": 0, 2024-09-10T10:05:08.559 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_writesame": 0, 2024-09-10T10:05:08.559 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_append": 0, 2024-09-10T10:05:08.559 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_zero": 0, 2024-09-10T10:05:08.559 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_truncate": 0, 2024-09-10T10:05:08.559 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_delete": 0, 2024-09-10T10:05:08.559 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_mapext": 0, 2024-09-10T10:05:08.559 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_sparse_read": 0, 2024-09-10T10:05:08.560 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_clonerange": 0, 2024-09-10T10:05:08.560 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_getxattr": 2, 2024-09-10T10:05:08.560 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_setxattr": 0, 2024-09-10T10:05:08.560 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_cmpxattr": 0, 2024-09-10T10:05:08.560 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_rmxattr": 0, 2024-09-10T10:05:08.560 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_resetxattrs": 0, 2024-09-10T10:05:08.560 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_call": 0, 2024-09-10T10:05:08.560 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_watch": 0, 2024-09-10T10:05:08.560 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_notify": 0, 2024-09-10T10:05:08.560 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_src_cmpxattr": 0, 2024-09-10T10:05:08.560 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_pgls": 0, 2024-09-10T10:05:08.560 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_pgls_filter": 0, 2024-09-10T10:05:08.561 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_other": 0, 2024-09-10T10:05:08.561 INFO:teuthology.orchestra.run.smithi159.stdout: "linger_active": 0, 2024-09-10T10:05:08.561 INFO:teuthology.orchestra.run.smithi159.stdout: "linger_send": 0, 2024-09-10T10:05:08.561 INFO:teuthology.orchestra.run.smithi159.stdout: "linger_resend": 0, 2024-09-10T10:05:08.561 INFO:teuthology.orchestra.run.smithi159.stdout: "linger_ping": 0, 2024-09-10T10:05:08.561 INFO:teuthology.orchestra.run.smithi159.stdout: "poolop_active": 0, 2024-09-10T10:05:08.561 INFO:teuthology.orchestra.run.smithi159.stdout: "poolop_send": 0, 2024-09-10T10:05:08.561 INFO:teuthology.orchestra.run.smithi159.stdout: "poolop_resend": 0, 2024-09-10T10:05:08.561 INFO:teuthology.orchestra.run.smithi159.stdout: "poolstat_active": 0, 2024-09-10T10:05:08.561 INFO:teuthology.orchestra.run.smithi159.stdout: "poolstat_send": 0, 2024-09-10T10:05:08.561 INFO:teuthology.orchestra.run.smithi159.stdout: "poolstat_resend": 0, 2024-09-10T10:05:08.562 INFO:teuthology.orchestra.run.smithi159.stdout: "statfs_active": 0, 2024-09-10T10:05:08.562 INFO:teuthology.orchestra.run.smithi159.stdout: "statfs_send": 0, 2024-09-10T10:05:08.562 INFO:teuthology.orchestra.run.smithi159.stdout: "statfs_resend": 0, 2024-09-10T10:05:08.562 INFO:teuthology.orchestra.run.smithi159.stdout: "command_active": 0, 2024-09-10T10:05:08.562 INFO:teuthology.orchestra.run.smithi159.stdout: "command_send": 0, 2024-09-10T10:05:08.562 INFO:teuthology.orchestra.run.smithi159.stdout: "command_resend": 0, 2024-09-10T10:05:08.562 INFO:teuthology.orchestra.run.smithi159.stdout: "map_epoch": 96, 2024-09-10T10:05:08.562 INFO:teuthology.orchestra.run.smithi159.stdout: "map_full": 0, 2024-09-10T10:05:08.562 INFO:teuthology.orchestra.run.smithi159.stdout: "map_inc": 2, 2024-09-10T10:05:08.562 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_sessions": 11, 2024-09-10T10:05:08.562 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_session_open": 11, 2024-09-10T10:05:08.562 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_session_close": 0, 2024-09-10T10:05:08.563 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_laggy": 0, 2024-09-10T10:05:08.563 INFO:teuthology.orchestra.run.smithi159.stdout: "omap_wr": 0, 2024-09-10T10:05:08.563 INFO:teuthology.orchestra.run.smithi159.stdout: "omap_rd": 2, 2024-09-10T10:05:08.563 INFO:teuthology.orchestra.run.smithi159.stdout: "omap_del": 0 2024-09-10T10:05:08.563 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-10T10:05:08.563 INFO:teuthology.orchestra.run.smithi159.stdout: "oft": { 2024-09-10T10:05:08.563 INFO:teuthology.orchestra.run.smithi159.stdout: "omap_total_objs": 0, 2024-09-10T10:05:08.563 INFO:teuthology.orchestra.run.smithi159.stdout: "omap_total_kv_pairs": 0, 2024-09-10T10:05:08.564 INFO:teuthology.orchestra.run.smithi159.stdout: "omap_total_updates": 0, 2024-09-10T10:05:08.564 INFO:teuthology.orchestra.run.smithi159.stdout: "omap_total_removes": 0 2024-09-10T10:05:08.564 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-10T10:05:08.564 INFO:teuthology.orchestra.run.smithi159.stdout: "osd-slow-ops": { 2024-09-10T10:05:08.564 INFO:teuthology.orchestra.run.smithi159.stdout: "slow_ops_count": 0 2024-09-10T10:05:08.564 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-10T10:05:08.564 INFO:teuthology.orchestra.run.smithi159.stdout: "purge_queue": { 2024-09-10T10:05:08.564 INFO:teuthology.orchestra.run.smithi159.stdout: "pq_executing_ops": 0, 2024-09-10T10:05:08.564 INFO:teuthology.orchestra.run.smithi159.stdout: "pq_executing_ops_high_water": 0, 2024-09-10T10:05:08.564 INFO:teuthology.orchestra.run.smithi159.stdout: "pq_executing": 0, 2024-09-10T10:05:08.564 INFO:teuthology.orchestra.run.smithi159.stdout: "pq_executing_high_water": 0, 2024-09-10T10:05:08.564 INFO:teuthology.orchestra.run.smithi159.stdout: "pq_executed_ops": 0, 2024-09-10T10:05:08.564 INFO:teuthology.orchestra.run.smithi159.stdout: "pq_executed": 0, 2024-09-10T10:05:08.565 INFO:teuthology.orchestra.run.smithi159.stdout: "pq_item_in_journal": 0 2024-09-10T10:05:08.565 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-10T10:05:08.565 INFO:teuthology.orchestra.run.smithi159.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-10T10:05:08.565 INFO:teuthology.orchestra.run.smithi159.stdout: "val": 0, 2024-09-10T10:05:08.565 INFO:teuthology.orchestra.run.smithi159.stdout: "max": 104857600, 2024-09-10T10:05:08.565 INFO:teuthology.orchestra.run.smithi159.stdout: "get_started": 0, 2024-09-10T10:05:08.565 INFO:teuthology.orchestra.run.smithi159.stdout: "get": 6669, 2024-09-10T10:05:08.565 INFO:teuthology.orchestra.run.smithi159.stdout: "get_sum": 145551573, 2024-09-10T10:05:08.565 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_fail": 0, 2024-09-10T10:05:08.565 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_success": 6669, 2024-09-10T10:05:08.565 INFO:teuthology.orchestra.run.smithi159.stdout: "take": 0, 2024-09-10T10:05:08.565 INFO:teuthology.orchestra.run.smithi159.stdout: "take_sum": 0, 2024-09-10T10:05:08.565 INFO:teuthology.orchestra.run.smithi159.stdout: "put": 6669, 2024-09-10T10:05:08.565 INFO:teuthology.orchestra.run.smithi159.stdout: "put_sum": 145551573, 2024-09-10T10:05:08.566 INFO:teuthology.orchestra.run.smithi159.stdout: "wait": { 2024-09-10T10:05:08.566 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-10T10:05:08.566 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-10T10:05:08.566 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-10T10:05:08.566 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-10T10:05:08.566 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-10T10:05:08.566 INFO:teuthology.orchestra.run.smithi159.stdout: "throttle-objecter_bytes": { 2024-09-10T10:05:08.566 INFO:teuthology.orchestra.run.smithi159.stdout: "val": 0, 2024-09-10T10:05:08.566 INFO:teuthology.orchestra.run.smithi159.stdout: "max": 104857600, 2024-09-10T10:05:08.566 INFO:teuthology.orchestra.run.smithi159.stdout: "get_started": 0, 2024-09-10T10:05:08.566 INFO:teuthology.orchestra.run.smithi159.stdout: "get": 0, 2024-09-10T10:05:08.566 INFO:teuthology.orchestra.run.smithi159.stdout: "get_sum": 0, 2024-09-10T10:05:08.566 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_fail": 0, 2024-09-10T10:05:08.566 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_success": 0, 2024-09-10T10:05:08.567 INFO:teuthology.orchestra.run.smithi159.stdout: "take": 6151, 2024-09-10T10:05:08.567 INFO:teuthology.orchestra.run.smithi159.stdout: "take_sum": 143481555, 2024-09-10T10:05:08.567 INFO:teuthology.orchestra.run.smithi159.stdout: "put": 6151, 2024-09-10T10:05:08.567 INFO:teuthology.orchestra.run.smithi159.stdout: "put_sum": 143481555, 2024-09-10T10:05:08.567 INFO:teuthology.orchestra.run.smithi159.stdout: "wait": { 2024-09-10T10:05:08.567 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-10T10:05:08.567 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-10T10:05:08.567 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-10T10:05:08.567 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-10T10:05:08.567 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-10T10:05:08.567 INFO:teuthology.orchestra.run.smithi159.stdout: "throttle-objecter_ops": { 2024-09-10T10:05:08.567 INFO:teuthology.orchestra.run.smithi159.stdout: "val": 0, 2024-09-10T10:05:08.567 INFO:teuthology.orchestra.run.smithi159.stdout: "max": 1024, 2024-09-10T10:05:08.567 INFO:teuthology.orchestra.run.smithi159.stdout: "get_started": 0, 2024-09-10T10:05:08.567 INFO:teuthology.orchestra.run.smithi159.stdout: "get": 0, 2024-09-10T10:05:08.568 INFO:teuthology.orchestra.run.smithi159.stdout: "get_sum": 0, 2024-09-10T10:05:08.568 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_fail": 0, 2024-09-10T10:05:08.568 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_success": 0, 2024-09-10T10:05:08.568 INFO:teuthology.orchestra.run.smithi159.stdout: "take": 6151, 2024-09-10T10:05:08.568 INFO:teuthology.orchestra.run.smithi159.stdout: "take_sum": 6151, 2024-09-10T10:05:08.568 INFO:teuthology.orchestra.run.smithi159.stdout: "put": 6151, 2024-09-10T10:05:08.568 INFO:teuthology.orchestra.run.smithi159.stdout: "put_sum": 6151, 2024-09-10T10:05:08.568 INFO:teuthology.orchestra.run.smithi159.stdout: "wait": { 2024-09-10T10:05:08.568 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-10T10:05:08.568 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-10T10:05:08.568 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-10T10:05:08.568 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-10T10:05:08.568 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-10T10:05:08.568 INFO:teuthology.orchestra.run.smithi159.stdout: "throttle-write_buf_throttle": { 2024-09-10T10:05:08.569 INFO:teuthology.orchestra.run.smithi159.stdout: "val": 0, 2024-09-10T10:05:08.569 INFO:teuthology.orchestra.run.smithi159.stdout: "max": 3758096384, 2024-09-10T10:05:08.569 INFO:teuthology.orchestra.run.smithi159.stdout: "get_started": 0, 2024-09-10T10:05:08.569 INFO:teuthology.orchestra.run.smithi159.stdout: "get": 0, 2024-09-10T10:05:08.569 INFO:teuthology.orchestra.run.smithi159.stdout: "get_sum": 0, 2024-09-10T10:05:08.569 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_fail": 0, 2024-09-10T10:05:08.569 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_success": 0, 2024-09-10T10:05:08.569 INFO:teuthology.orchestra.run.smithi159.stdout: "take": 0, 2024-09-10T10:05:08.569 INFO:teuthology.orchestra.run.smithi159.stdout: "take_sum": 0, 2024-09-10T10:05:08.569 INFO:teuthology.orchestra.run.smithi159.stdout: "put": 0, 2024-09-10T10:05:08.570 INFO:teuthology.orchestra.run.smithi159.stdout: "put_sum": 0, 2024-09-10T10:05:08.570 INFO:teuthology.orchestra.run.smithi159.stdout: "wait": { 2024-09-10T10:05:08.570 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-10T10:05:08.570 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-10T10:05:08.570 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-10T10:05:08.570 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-10T10:05:08.570 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-10T10:05:08.570 INFO:teuthology.orchestra.run.smithi159.stdout: "throttle-write_buf_throttle-0x55dd1706cb40": { 2024-09-10T10:05:08.570 INFO:teuthology.orchestra.run.smithi159.stdout: "val": 0, 2024-09-10T10:05:08.570 INFO:teuthology.orchestra.run.smithi159.stdout: "max": 3758096384, 2024-09-10T10:05:08.570 INFO:teuthology.orchestra.run.smithi159.stdout: "get_started": 0, 2024-09-10T10:05:08.570 INFO:teuthology.orchestra.run.smithi159.stdout: "get": 0, 2024-09-10T10:05:08.570 INFO:teuthology.orchestra.run.smithi159.stdout: "get_sum": 0, 2024-09-10T10:05:08.570 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_fail": 0, 2024-09-10T10:05:08.571 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_success": 0, 2024-09-10T10:05:08.571 INFO:teuthology.orchestra.run.smithi159.stdout: "take": 0, 2024-09-10T10:05:08.571 INFO:teuthology.orchestra.run.smithi159.stdout: "take_sum": 0, 2024-09-10T10:05:08.571 INFO:teuthology.orchestra.run.smithi159.stdout: "put": 0, 2024-09-10T10:05:08.571 INFO:teuthology.orchestra.run.smithi159.stdout: "put_sum": 0, 2024-09-10T10:05:08.571 INFO:teuthology.orchestra.run.smithi159.stdout: "wait": { 2024-09-10T10:05:08.571 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-10T10:05:08.571 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-10T10:05:08.571 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-10T10:05:08.571 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-10T10:05:08.571 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-10T10:05:08.571 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-09-10T10:05:09.385 INFO:tasks.check_counter:Daemon mds.f mds.exported=0 2024-09-10T10:05:09.385 INFO:tasks.check_counter:Daemon mds.f mds.imported=0 2024-09-10T10:05:09.385 DEBUG:tasks.check_counter:Getting stats from i 2024-09-10T10:05:09.386 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph daemon mds.i perf dump 2024-09-10T10:05:09.631 INFO:teuthology.orchestra.run.smithi159.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.c/config 2024-09-10T10:05:10.083 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:05:09 smithi159 ceph-mon[35513]: pgmap v1197: 129 pgs: 129 active+clean; 551 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-10T10:05:10.136 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:05:09 smithi102 ceph-mon[35442]: pgmap v1197: 129 pgs: 129 active+clean; 551 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-10T10:05:10.187 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-09-10T10:05:10.187 INFO:teuthology.orchestra.run.smithi159.stdout: "AsyncMessenger::Worker-0": { 2024-09-10T10:05:10.187 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_messages": 118526, 2024-09-10T10:05:10.187 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_messages": 162611, 2024-09-10T10:05:10.187 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_bytes": 36717200, 2024-09-10T10:05:10.188 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_bytes": 142811676, 2024-09-10T10:05:10.188 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_created_connections": 49, 2024-09-10T10:05:10.188 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_active_connections": 4, 2024-09-10T10:05:10.188 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_total_time": 7.010058084, 2024-09-10T10:05:10.188 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_send_time": 3.130405693, 2024-09-10T10:05:10.188 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_recv_time": 4.215414414, 2024-09-10T10:05:10.188 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_fast_dispatch_time": 0.470880880, 2024-09-10T10:05:10.188 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T10:05:10.188 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 162611, 2024-09-10T10:05:10.188 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 7.532118128, 2024-09-10T10:05:10.189 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000046319 2024-09-10T10:05:10.189 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-10T10:05:10.189 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_handle_ack_lat": { 2024-09-10T10:05:10.189 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 142836, 2024-09-10T10:05:10.189 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.040952202, 2024-09-10T10:05:10.189 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000286 2024-09-10T10:05:10.189 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-10T10:05:10.189 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_encrypted_bytes": 1018272, 2024-09-10T10:05:10.189 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_encrypted_bytes": 315824 2024-09-10T10:05:10.189 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-10T10:05:10.189 INFO:teuthology.orchestra.run.smithi159.stdout: "AsyncMessenger::Worker-1": { 2024-09-10T10:05:10.189 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_messages": 27959, 2024-09-10T10:05:10.189 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_messages": 28911, 2024-09-10T10:05:10.190 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_bytes": 6835144, 2024-09-10T10:05:10.190 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_bytes": 42216164, 2024-09-10T10:05:10.190 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_created_connections": 13, 2024-09-10T10:05:10.190 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_active_connections": 7, 2024-09-10T10:05:10.190 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_total_time": 2.127165574, 2024-09-10T10:05:10.190 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_send_time": 0.685807841, 2024-09-10T10:05:10.190 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_recv_time": 0.840529512, 2024-09-10T10:05:10.190 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_fast_dispatch_time": 0.622445949, 2024-09-10T10:05:10.190 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T10:05:10.190 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 28909, 2024-09-10T10:05:10.190 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 1.545264435, 2024-09-10T10:05:10.190 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000053452 2024-09-10T10:05:10.190 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-10T10:05:10.191 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_handle_ack_lat": { 2024-09-10T10:05:10.191 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 31882, 2024-09-10T10:05:10.191 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.005545447, 2024-09-10T10:05:10.191 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000173 2024-09-10T10:05:10.191 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-10T10:05:10.191 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-10T10:05:10.191 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_encrypted_bytes": 803728 2024-09-10T10:05:10.191 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-10T10:05:10.191 INFO:teuthology.orchestra.run.smithi159.stdout: "AsyncMessenger::Worker-2": { 2024-09-10T10:05:10.191 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_messages": 120056, 2024-09-10T10:05:10.191 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_messages": 81283, 2024-09-10T10:05:10.191 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_bytes": 32528849, 2024-09-10T10:05:10.192 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_bytes": 60984588, 2024-09-10T10:05:10.192 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_created_connections": 32, 2024-09-10T10:05:10.192 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_active_connections": 7, 2024-09-10T10:05:10.192 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_total_time": 6.720327671, 2024-09-10T10:05:10.192 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_send_time": 2.773738235, 2024-09-10T10:05:10.192 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_recv_time": 4.258120363, 2024-09-10T10:05:10.192 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_fast_dispatch_time": 0.572179861, 2024-09-10T10:05:10.192 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T10:05:10.192 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 81281, 2024-09-10T10:05:10.193 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 2.315678347, 2024-09-10T10:05:10.193 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000028489 2024-09-10T10:05:10.193 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-10T10:05:10.193 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_handle_ack_lat": { 2024-09-10T10:05:10.193 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 166331, 2024-09-10T10:05:10.193 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.024692146, 2024-09-10T10:05:10.193 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000148 2024-09-10T10:05:10.193 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-10T10:05:10.193 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-10T10:05:10.193 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_encrypted_bytes": 0 2024-09-10T10:05:10.193 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-10T10:05:10.194 INFO:teuthology.orchestra.run.smithi159.stdout: "cct": { 2024-09-10T10:05:10.194 INFO:teuthology.orchestra.run.smithi159.stdout: "total_workers": 1, 2024-09-10T10:05:10.194 INFO:teuthology.orchestra.run.smithi159.stdout: "unhealthy_workers": 0 2024-09-10T10:05:10.194 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-10T10:05:10.194 INFO:teuthology.orchestra.run.smithi159.stdout: "finisher-MDSRank": { 2024-09-10T10:05:10.194 INFO:teuthology.orchestra.run.smithi159.stdout: "queue_len": 0, 2024-09-10T10:05:10.194 INFO:teuthology.orchestra.run.smithi159.stdout: "complete_latency": { 2024-09-10T10:05:10.194 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 35414, 2024-09-10T10:05:10.194 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 29.451353158, 2024-09-10T10:05:10.194 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000831630 2024-09-10T10:05:10.194 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-10T10:05:10.194 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-10T10:05:10.194 INFO:teuthology.orchestra.run.smithi159.stdout: "finisher-PurgeQueue": { 2024-09-10T10:05:10.195 INFO:teuthology.orchestra.run.smithi159.stdout: "queue_len": 0, 2024-09-10T10:05:10.195 INFO:teuthology.orchestra.run.smithi159.stdout: "complete_latency": { 2024-09-10T10:05:10.195 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 19514, 2024-09-10T10:05:10.195 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 1.413558701, 2024-09-10T10:05:10.195 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000072438 2024-09-10T10:05:10.195 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-10T10:05:10.195 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-10T10:05:10.195 INFO:teuthology.orchestra.run.smithi159.stdout: "mds": { 2024-09-10T10:05:10.195 INFO:teuthology.orchestra.run.smithi159.stdout: "request": 63773, 2024-09-10T10:05:10.195 INFO:teuthology.orchestra.run.smithi159.stdout: "reply": 56113, 2024-09-10T10:05:10.195 INFO:teuthology.orchestra.run.smithi159.stdout: "reply_latency": { 2024-09-10T10:05:10.195 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 56113, 2024-09-10T10:05:10.195 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 426.989048284, 2024-09-10T10:05:10.196 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.007609449 2024-09-10T10:05:10.196 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-10T10:05:10.196 INFO:teuthology.orchestra.run.smithi159.stdout: "slow_reply": 0, 2024-09-10T10:05:10.196 INFO:teuthology.orchestra.run.smithi159.stdout: "forward": 7660, 2024-09-10T10:05:10.196 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_fetch_complete": 0, 2024-09-10T10:05:10.196 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_fetch_keys": 0, 2024-09-10T10:05:10.196 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_commit": 11, 2024-09-10T10:05:10.196 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_split": 0, 2024-09-10T10:05:10.196 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_merge": 0, 2024-09-10T10:05:10.196 INFO:teuthology.orchestra.run.smithi159.stdout: "inodes": 15, 2024-09-10T10:05:10.196 INFO:teuthology.orchestra.run.smithi159.stdout: "inodes_top": 3, 2024-09-10T10:05:10.196 INFO:teuthology.orchestra.run.smithi159.stdout: "inodes_bottom": 2, 2024-09-10T10:05:10.196 INFO:teuthology.orchestra.run.smithi159.stdout: "inodes_pin_tail": 10, 2024-09-10T10:05:10.197 INFO:teuthology.orchestra.run.smithi159.stdout: "inodes_pinned": 10, 2024-09-10T10:05:10.197 INFO:teuthology.orchestra.run.smithi159.stdout: "inodes_expired": 80, 2024-09-10T10:05:10.197 INFO:teuthology.orchestra.run.smithi159.stdout: "inodes_with_caps": 0, 2024-09-10T10:05:10.197 INFO:teuthology.orchestra.run.smithi159.stdout: "caps": 0, 2024-09-10T10:05:10.197 INFO:teuthology.orchestra.run.smithi159.stdout: "subtrees": 5, 2024-09-10T10:05:10.197 INFO:teuthology.orchestra.run.smithi159.stdout: "traverse": 94161, 2024-09-10T10:05:10.197 INFO:teuthology.orchestra.run.smithi159.stdout: "traverse_hit": 49715, 2024-09-10T10:05:10.197 INFO:teuthology.orchestra.run.smithi159.stdout: "traverse_forward": 7659, 2024-09-10T10:05:10.197 INFO:teuthology.orchestra.run.smithi159.stdout: "traverse_discover": 22, 2024-09-10T10:05:10.197 INFO:teuthology.orchestra.run.smithi159.stdout: "traverse_dir_fetch": 0, 2024-09-10T10:05:10.197 INFO:teuthology.orchestra.run.smithi159.stdout: "traverse_remote_ino": 0, 2024-09-10T10:05:10.197 INFO:teuthology.orchestra.run.smithi159.stdout: "traverse_lock": 696, 2024-09-10T10:05:10.197 INFO:teuthology.orchestra.run.smithi159.stdout: "load_cent": 0, 2024-09-10T10:05:10.198 INFO:teuthology.orchestra.run.smithi159.stdout: "q": 0, 2024-09-10T10:05:10.198 INFO:teuthology.orchestra.run.smithi159.stdout: "exported": 2, 2024-09-10T10:05:10.198 INFO:teuthology.orchestra.run.smithi159.stdout: "exported_inodes": 3, 2024-09-10T10:05:10.198 INFO:teuthology.orchestra.run.smithi159.stdout: "imported": 3, 2024-09-10T10:05:10.198 INFO:teuthology.orchestra.run.smithi159.stdout: "imported_inodes": 138, 2024-09-10T10:05:10.198 INFO:teuthology.orchestra.run.smithi159.stdout: "openino_dir_fetch": 0, 2024-09-10T10:05:10.198 INFO:teuthology.orchestra.run.smithi159.stdout: "openino_backtrace_fetch": 948, 2024-09-10T10:05:10.198 INFO:teuthology.orchestra.run.smithi159.stdout: "openino_peer_discover": 0, 2024-09-10T10:05:10.198 INFO:teuthology.orchestra.run.smithi159.stdout: "root_rfiles": 733, 2024-09-10T10:05:10.198 INFO:teuthology.orchestra.run.smithi159.stdout: "root_rbytes": 226265082, 2024-09-10T10:05:10.198 INFO:teuthology.orchestra.run.smithi159.stdout: "root_rsnaps": 0, 2024-09-10T10:05:10.198 INFO:teuthology.orchestra.run.smithi159.stdout: "scrub_backtrace_fetch": 948, 2024-09-10T10:05:10.198 INFO:teuthology.orchestra.run.smithi159.stdout: "scrub_set_tag": 0, 2024-09-10T10:05:10.198 INFO:teuthology.orchestra.run.smithi159.stdout: "scrub_backtrace_repaired": 0, 2024-09-10T10:05:10.199 INFO:teuthology.orchestra.run.smithi159.stdout: "scrub_inotable_repaired": 0, 2024-09-10T10:05:10.199 INFO:teuthology.orchestra.run.smithi159.stdout: "scrub_dir_inodes": 99, 2024-09-10T10:05:10.199 INFO:teuthology.orchestra.run.smithi159.stdout: "scrub_dir_base_inodes": 0, 2024-09-10T10:05:10.199 INFO:teuthology.orchestra.run.smithi159.stdout: "scrub_dirfrag_rstats": 99, 2024-09-10T10:05:10.199 INFO:teuthology.orchestra.run.smithi159.stdout: "scrub_file_inodes": 849, 2024-09-10T10:05:10.199 INFO:teuthology.orchestra.run.smithi159.stdout: "handle_inode_file_caps": 4, 2024-09-10T10:05:10.199 INFO:teuthology.orchestra.run.smithi159.stdout: "ceph_cap_op_revoke": 3898, 2024-09-10T10:05:10.199 INFO:teuthology.orchestra.run.smithi159.stdout: "ceph_cap_op_grant": 40301, 2024-09-10T10:05:10.199 INFO:teuthology.orchestra.run.smithi159.stdout: "ceph_cap_op_trunc": 0, 2024-09-10T10:05:10.199 INFO:teuthology.orchestra.run.smithi159.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-10T10:05:10.199 INFO:teuthology.orchestra.run.smithi159.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-10T10:05:10.199 INFO:teuthology.orchestra.run.smithi159.stdout: "handle_client_caps": 21978, 2024-09-10T10:05:10.199 INFO:teuthology.orchestra.run.smithi159.stdout: "handle_client_caps_dirty": 8611, 2024-09-10T10:05:10.200 INFO:teuthology.orchestra.run.smithi159.stdout: "handle_client_cap_release": 1889, 2024-09-10T10:05:10.200 INFO:teuthology.orchestra.run.smithi159.stdout: "process_request_cap_release": 34541 2024-09-10T10:05:10.200 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-10T10:05:10.200 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_cache": { 2024-09-10T10:05:10.200 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_update": 7, 2024-09-10T10:05:10.200 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_update_receipt": 221, 2024-09-10T10:05:10.200 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_try_discover": 119, 2024-09-10T10:05:10.200 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_send_discover": 221, 2024-09-10T10:05:10.200 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_handle_discover": 14, 2024-09-10T10:05:10.200 INFO:teuthology.orchestra.run.smithi159.stdout: "num_strays": 0, 2024-09-10T10:05:10.200 INFO:teuthology.orchestra.run.smithi159.stdout: "num_strays_delayed": 0, 2024-09-10T10:05:10.200 INFO:teuthology.orchestra.run.smithi159.stdout: "num_strays_enqueuing": 0, 2024-09-10T10:05:10.201 INFO:teuthology.orchestra.run.smithi159.stdout: "strays_created": 9846, 2024-09-10T10:05:10.201 INFO:teuthology.orchestra.run.smithi159.stdout: "strays_enqueued": 9846, 2024-09-10T10:05:10.201 INFO:teuthology.orchestra.run.smithi159.stdout: "strays_reintegrated": 0, 2024-09-10T10:05:10.201 INFO:teuthology.orchestra.run.smithi159.stdout: "strays_migrated": 0, 2024-09-10T10:05:10.201 INFO:teuthology.orchestra.run.smithi159.stdout: "num_recovering_processing": 0, 2024-09-10T10:05:10.201 INFO:teuthology.orchestra.run.smithi159.stdout: "num_recovering_enqueued": 0, 2024-09-10T10:05:10.201 INFO:teuthology.orchestra.run.smithi159.stdout: "num_recovering_prioritized": 0, 2024-09-10T10:05:10.201 INFO:teuthology.orchestra.run.smithi159.stdout: "recovery_started": 0, 2024-09-10T10:05:10.201 INFO:teuthology.orchestra.run.smithi159.stdout: "recovery_completed": 0, 2024-09-10T10:05:10.201 INFO:teuthology.orchestra.run.smithi159.stdout: "ireq_quiesce_path": 23, 2024-09-10T10:05:10.201 INFO:teuthology.orchestra.run.smithi159.stdout: "ireq_quiesce_inode": 1410, 2024-09-10T10:05:10.201 INFO:teuthology.orchestra.run.smithi159.stdout: "ireq_enqueue_scrub": 0, 2024-09-10T10:05:10.201 INFO:teuthology.orchestra.run.smithi159.stdout: "ireq_exportdir": 2, 2024-09-10T10:05:10.202 INFO:teuthology.orchestra.run.smithi159.stdout: "ireq_flush": 0, 2024-09-10T10:05:10.202 INFO:teuthology.orchestra.run.smithi159.stdout: "ireq_fragmentdir": 0, 2024-09-10T10:05:10.202 INFO:teuthology.orchestra.run.smithi159.stdout: "ireq_fragstats": 0, 2024-09-10T10:05:10.202 INFO:teuthology.orchestra.run.smithi159.stdout: "ireq_inodestats": 0 2024-09-10T10:05:10.202 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-10T10:05:10.202 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_log": { 2024-09-10T10:05:10.202 INFO:teuthology.orchestra.run.smithi159.stdout: "evlrg": 0, 2024-09-10T10:05:10.202 INFO:teuthology.orchestra.run.smithi159.stdout: "evadd": 50281, 2024-09-10T10:05:10.202 INFO:teuthology.orchestra.run.smithi159.stdout: "evex": 0, 2024-09-10T10:05:10.203 INFO:teuthology.orchestra.run.smithi159.stdout: "evtrm": 0, 2024-09-10T10:05:10.203 INFO:teuthology.orchestra.run.smithi159.stdout: "ev": 50281, 2024-09-10T10:05:10.203 INFO:teuthology.orchestra.run.smithi159.stdout: "evexg": 0, 2024-09-10T10:05:10.203 INFO:teuthology.orchestra.run.smithi159.stdout: "evexd": 0, 2024-09-10T10:05:10.203 INFO:teuthology.orchestra.run.smithi159.stdout: "segadd": 63, 2024-09-10T10:05:10.204 INFO:teuthology.orchestra.run.smithi159.stdout: "segex": 0, 2024-09-10T10:05:10.204 INFO:teuthology.orchestra.run.smithi159.stdout: "segtrm": 0, 2024-09-10T10:05:10.204 INFO:teuthology.orchestra.run.smithi159.stdout: "seg": 63, 2024-09-10T10:05:10.204 INFO:teuthology.orchestra.run.smithi159.stdout: "segmjr": 6, 2024-09-10T10:05:10.204 INFO:teuthology.orchestra.run.smithi159.stdout: "segexg": 0, 2024-09-10T10:05:10.204 INFO:teuthology.orchestra.run.smithi159.stdout: "segexd": 0, 2024-09-10T10:05:10.205 INFO:teuthology.orchestra.run.smithi159.stdout: "expos": 4194304, 2024-09-10T10:05:10.205 INFO:teuthology.orchestra.run.smithi159.stdout: "wrpos": 104168352, 2024-09-10T10:05:10.205 INFO:teuthology.orchestra.run.smithi159.stdout: "rdpos": 0, 2024-09-10T10:05:10.205 INFO:teuthology.orchestra.run.smithi159.stdout: "jlat": { 2024-09-10T10:05:10.205 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 17700, 2024-09-10T10:05:10.206 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 103.764107312, 2024-09-10T10:05:10.206 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.005862378 2024-09-10T10:05:10.206 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-10T10:05:10.206 INFO:teuthology.orchestra.run.smithi159.stdout: "replayed": 0 2024-09-10T10:05:10.206 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-10T10:05:10.206 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_mem": { 2024-09-10T10:05:10.207 INFO:teuthology.orchestra.run.smithi159.stdout: "ino": 21, 2024-09-10T10:05:10.207 INFO:teuthology.orchestra.run.smithi159.stdout: "ino+": 9907, 2024-09-10T10:05:10.207 INFO:teuthology.orchestra.run.smithi159.stdout: "ino-": 9886, 2024-09-10T10:05:10.207 INFO:teuthology.orchestra.run.smithi159.stdout: "dir": 28, 2024-09-10T10:05:10.207 INFO:teuthology.orchestra.run.smithi159.stdout: "dir+": 170, 2024-09-10T10:05:10.207 INFO:teuthology.orchestra.run.smithi159.stdout: "dir-": 142, 2024-09-10T10:05:10.208 INFO:teuthology.orchestra.run.smithi159.stdout: "dn": 15, 2024-09-10T10:05:10.208 INFO:teuthology.orchestra.run.smithi159.stdout: "dn+": 21249, 2024-09-10T10:05:10.208 INFO:teuthology.orchestra.run.smithi159.stdout: "dn-": 21234, 2024-09-10T10:05:10.208 INFO:teuthology.orchestra.run.smithi159.stdout: "cap": 0, 2024-09-10T10:05:10.208 INFO:teuthology.orchestra.run.smithi159.stdout: "cap+": 11088, 2024-09-10T10:05:10.208 INFO:teuthology.orchestra.run.smithi159.stdout: "cap-": 11088, 2024-09-10T10:05:10.208 INFO:teuthology.orchestra.run.smithi159.stdout: "rss": 68388, 2024-09-10T10:05:10.208 INFO:teuthology.orchestra.run.smithi159.stdout: "heap": 223520 2024-09-10T10:05:10.208 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-10T10:05:10.208 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_server": { 2024-09-10T10:05:10.208 INFO:teuthology.orchestra.run.smithi159.stdout: "dispatch_client_request": 83021, 2024-09-10T10:05:10.208 INFO:teuthology.orchestra.run.smithi159.stdout: "dispatch_server_request": 2133, 2024-09-10T10:05:10.209 INFO:teuthology.orchestra.run.smithi159.stdout: "handle_client_request": 63773, 2024-09-10T10:05:10.209 INFO:teuthology.orchestra.run.smithi159.stdout: "handle_client_session": 1443, 2024-09-10T10:05:10.209 INFO:teuthology.orchestra.run.smithi159.stdout: "handle_peer_request": 5618, 2024-09-10T10:05:10.209 INFO:teuthology.orchestra.run.smithi159.stdout: "req_create_latency": { 2024-09-10T10:05:10.209 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 9756, 2024-09-10T10:05:10.209 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 208.465464358, 2024-09-10T10:05:10.209 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.021367923 2024-09-10T10:05:10.209 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-10T10:05:10.209 INFO:teuthology.orchestra.run.smithi159.stdout: "req_getattr_latency": { 2024-09-10T10:05:10.209 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 2357, 2024-09-10T10:05:10.209 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 26.727612155, 2024-09-10T10:05:10.209 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.011339674 2024-09-10T10:05:10.209 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-10T10:05:10.210 INFO:teuthology.orchestra.run.smithi159.stdout: "req_getfilelock_latency": { 2024-09-10T10:05:10.210 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-10T10:05:10.210 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-10T10:05:10.210 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-10T10:05:10.210 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-10T10:05:10.210 INFO:teuthology.orchestra.run.smithi159.stdout: "req_link_latency": { 2024-09-10T10:05:10.210 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-10T10:05:10.210 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-10T10:05:10.210 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-10T10:05:10.210 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-10T10:05:10.210 INFO:teuthology.orchestra.run.smithi159.stdout: "req_lookup_latency": { 2024-09-10T10:05:10.210 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 17651, 2024-09-10T10:05:10.211 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 10.153850734, 2024-09-10T10:05:10.211 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000575256 2024-09-10T10:05:10.211 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-10T10:05:10.211 INFO:teuthology.orchestra.run.smithi159.stdout: "req_lookuphash_latency": { 2024-09-10T10:05:10.211 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-10T10:05:10.211 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-10T10:05:10.211 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-10T10:05:10.211 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-10T10:05:10.211 INFO:teuthology.orchestra.run.smithi159.stdout: "req_lookupino_latency": { 2024-09-10T10:05:10.211 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-10T10:05:10.211 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-10T10:05:10.211 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-10T10:05:10.211 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-10T10:05:10.212 INFO:teuthology.orchestra.run.smithi159.stdout: "req_lookupname_latency": { 2024-09-10T10:05:10.212 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-10T10:05:10.212 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-10T10:05:10.212 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-10T10:05:10.212 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-10T10:05:10.212 INFO:teuthology.orchestra.run.smithi159.stdout: "req_lookupparent_latency": { 2024-09-10T10:05:10.212 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-10T10:05:10.213 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-10T10:05:10.213 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-10T10:05:10.213 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-10T10:05:10.213 INFO:teuthology.orchestra.run.smithi159.stdout: "req_lookupsnap_latency": { 2024-09-10T10:05:10.213 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-10T10:05:10.214 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-10T10:05:10.214 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-10T10:05:10.214 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-10T10:05:10.214 INFO:teuthology.orchestra.run.smithi159.stdout: "req_lssnap_latency": { 2024-09-10T10:05:10.214 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-10T10:05:10.214 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-10T10:05:10.215 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-10T10:05:10.215 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-10T10:05:10.215 INFO:teuthology.orchestra.run.smithi159.stdout: "req_mkdir_latency": { 2024-09-10T10:05:10.215 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-10T10:05:10.215 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-10T10:05:10.215 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-10T10:05:10.215 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-10T10:05:10.215 INFO:teuthology.orchestra.run.smithi159.stdout: "req_mknod_latency": { 2024-09-10T10:05:10.215 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-10T10:05:10.215 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-10T10:05:10.215 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-10T10:05:10.215 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-10T10:05:10.216 INFO:teuthology.orchestra.run.smithi159.stdout: "req_mksnap_latency": { 2024-09-10T10:05:10.216 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-10T10:05:10.216 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-10T10:05:10.216 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-10T10:05:10.216 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-10T10:05:10.216 INFO:teuthology.orchestra.run.smithi159.stdout: "req_open_latency": { 2024-09-10T10:05:10.216 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-10T10:05:10.216 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-10T10:05:10.216 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-10T10:05:10.216 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-10T10:05:10.216 INFO:teuthology.orchestra.run.smithi159.stdout: "req_readdir_latency": { 2024-09-10T10:05:10.216 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 14346, 2024-09-10T10:05:10.216 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 57.298891415, 2024-09-10T10:05:10.217 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.003994067 2024-09-10T10:05:10.217 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-10T10:05:10.217 INFO:teuthology.orchestra.run.smithi159.stdout: "req_rename_latency": { 2024-09-10T10:05:10.217 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 1812, 2024-09-10T10:05:10.217 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 15.297553130, 2024-09-10T10:05:10.217 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.008442358 2024-09-10T10:05:10.217 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-10T10:05:10.217 INFO:teuthology.orchestra.run.smithi159.stdout: "req_renamesnap_latency": { 2024-09-10T10:05:10.218 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-10T10:05:10.218 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-10T10:05:10.218 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-10T10:05:10.218 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-10T10:05:10.218 INFO:teuthology.orchestra.run.smithi159.stdout: "req_snapdiff_latency": { 2024-09-10T10:05:10.218 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-10T10:05:10.218 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-10T10:05:10.218 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-10T10:05:10.219 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-10T10:05:10.219 INFO:teuthology.orchestra.run.smithi159.stdout: "req_rmdir_latency": { 2024-09-10T10:05:10.219 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 9, 2024-09-10T10:05:10.219 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.028905999, 2024-09-10T10:05:10.219 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.003211777 2024-09-10T10:05:10.219 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-10T10:05:10.219 INFO:teuthology.orchestra.run.smithi159.stdout: "req_rmsnap_latency": { 2024-09-10T10:05:10.219 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-10T10:05:10.219 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-10T10:05:10.219 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-10T10:05:10.219 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-10T10:05:10.219 INFO:teuthology.orchestra.run.smithi159.stdout: "req_rmxattr_latency": { 2024-09-10T10:05:10.219 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-10T10:05:10.219 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-10T10:05:10.220 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-10T10:05:10.220 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-10T10:05:10.220 INFO:teuthology.orchestra.run.smithi159.stdout: "req_setattr_latency": { 2024-09-10T10:05:10.220 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 25, 2024-09-10T10:05:10.220 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 13.882928270, 2024-09-10T10:05:10.220 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.555317130 2024-09-10T10:05:10.220 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-10T10:05:10.220 INFO:teuthology.orchestra.run.smithi159.stdout: "req_setdirlayout_latency": { 2024-09-10T10:05:10.220 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-10T10:05:10.220 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-10T10:05:10.220 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-10T10:05:10.220 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-10T10:05:10.220 INFO:teuthology.orchestra.run.smithi159.stdout: "req_setfilelock_latency": { 2024-09-10T10:05:10.221 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 320, 2024-09-10T10:05:10.221 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.171210632, 2024-09-10T10:05:10.221 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000535033 2024-09-10T10:05:10.221 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-10T10:05:10.221 INFO:teuthology.orchestra.run.smithi159.stdout: "req_setlayout_latency": { 2024-09-10T10:05:10.221 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-10T10:05:10.221 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-10T10:05:10.221 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-10T10:05:10.221 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-10T10:05:10.221 INFO:teuthology.orchestra.run.smithi159.stdout: "req_setxattr_latency": { 2024-09-10T10:05:10.221 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-10T10:05:10.221 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-10T10:05:10.221 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-10T10:05:10.222 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-10T10:05:10.222 INFO:teuthology.orchestra.run.smithi159.stdout: "req_symlink_latency": { 2024-09-10T10:05:10.222 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-10T10:05:10.222 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-10T10:05:10.222 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-10T10:05:10.222 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-10T10:05:10.222 INFO:teuthology.orchestra.run.smithi159.stdout: "req_unlink_latency": { 2024-09-10T10:05:10.222 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 9837, 2024-09-10T10:05:10.222 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 94.962631591, 2024-09-10T10:05:10.222 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.009653617 2024-09-10T10:05:10.222 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-10T10:05:10.222 INFO:teuthology.orchestra.run.smithi159.stdout: "cap_revoke_eviction": 0, 2024-09-10T10:05:10.223 INFO:teuthology.orchestra.run.smithi159.stdout: "cap_acquisition_throttle": 0, 2024-09-10T10:05:10.223 INFO:teuthology.orchestra.run.smithi159.stdout: "req_getvxattr_latency": { 2024-09-10T10:05:10.223 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-10T10:05:10.223 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-10T10:05:10.223 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-10T10:05:10.223 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-10T10:05:10.223 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-10T10:05:10.223 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_sessions": { 2024-09-10T10:05:10.223 INFO:teuthology.orchestra.run.smithi159.stdout: "session_count": 1, 2024-09-10T10:05:10.223 INFO:teuthology.orchestra.run.smithi159.stdout: "session_add": 2, 2024-09-10T10:05:10.223 INFO:teuthology.orchestra.run.smithi159.stdout: "session_remove": 1, 2024-09-10T10:05:10.223 INFO:teuthology.orchestra.run.smithi159.stdout: "sessions_open": 1, 2024-09-10T10:05:10.223 INFO:teuthology.orchestra.run.smithi159.stdout: "sessions_stale": 0, 2024-09-10T10:05:10.223 INFO:teuthology.orchestra.run.smithi159.stdout: "total_load": 5643, 2024-09-10T10:05:10.223 INFO:teuthology.orchestra.run.smithi159.stdout: "average_load": 5643, 2024-09-10T10:05:10.224 INFO:teuthology.orchestra.run.smithi159.stdout: "avg_session_uptime": 1485, 2024-09-10T10:05:10.224 INFO:teuthology.orchestra.run.smithi159.stdout: "mdthresh_evicted": 0 2024-09-10T10:05:10.224 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-10T10:05:10.224 INFO:teuthology.orchestra.run.smithi159.stdout: "mempool": { 2024-09-10T10:05:10.224 INFO:teuthology.orchestra.run.smithi159.stdout: "bloom_filter_bytes": 0, 2024-09-10T10:05:10.224 INFO:teuthology.orchestra.run.smithi159.stdout: "bloom_filter_items": 0, 2024-09-10T10:05:10.224 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_alloc_bytes": 0, 2024-09-10T10:05:10.224 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_alloc_items": 0, 2024-09-10T10:05:10.224 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_data_bytes": 0, 2024-09-10T10:05:10.224 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_data_items": 0, 2024-09-10T10:05:10.224 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-10T10:05:10.224 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_onode_items": 0, 2024-09-10T10:05:10.224 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-10T10:05:10.224 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_meta_items": 0, 2024-09-10T10:05:10.225 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_other_bytes": 0, 2024-09-10T10:05:10.225 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_other_items": 0, 2024-09-10T10:05:10.225 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-10T10:05:10.225 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_buffer_items": 0, 2024-09-10T10:05:10.225 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_extent_bytes": 0, 2024-09-10T10:05:10.225 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_extent_items": 0, 2024-09-10T10:05:10.225 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_blob_bytes": 0, 2024-09-10T10:05:10.225 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_blob_items": 0, 2024-09-10T10:05:10.225 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-10T10:05:10.225 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_shared_blob_items": 0, 2024-09-10T10:05:10.225 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-10T10:05:10.225 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_inline_bl_items": 0, 2024-09-10T10:05:10.225 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_fsck_bytes": 0, 2024-09-10T10:05:10.225 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_fsck_items": 0, 2024-09-10T10:05:10.226 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_txc_bytes": 0, 2024-09-10T10:05:10.226 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_txc_items": 0, 2024-09-10T10:05:10.226 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-10T10:05:10.226 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_writing_deferred_items": 0, 2024-09-10T10:05:10.226 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_writing_bytes": 0, 2024-09-10T10:05:10.226 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_writing_items": 0, 2024-09-10T10:05:10.226 INFO:teuthology.orchestra.run.smithi159.stdout: "bluefs_bytes": 0, 2024-09-10T10:05:10.226 INFO:teuthology.orchestra.run.smithi159.stdout: "bluefs_items": 0, 2024-09-10T10:05:10.226 INFO:teuthology.orchestra.run.smithi159.stdout: "bluefs_file_reader_bytes": 0, 2024-09-10T10:05:10.226 INFO:teuthology.orchestra.run.smithi159.stdout: "bluefs_file_reader_items": 0, 2024-09-10T10:05:10.226 INFO:teuthology.orchestra.run.smithi159.stdout: "bluefs_file_writer_bytes": 0, 2024-09-10T10:05:10.226 INFO:teuthology.orchestra.run.smithi159.stdout: "bluefs_file_writer_items": 0, 2024-09-10T10:05:10.226 INFO:teuthology.orchestra.run.smithi159.stdout: "buffer_anon_bytes": 304612, 2024-09-10T10:05:10.226 INFO:teuthology.orchestra.run.smithi159.stdout: "buffer_anon_items": 111, 2024-09-10T10:05:10.226 INFO:teuthology.orchestra.run.smithi159.stdout: "buffer_meta_bytes": 88, 2024-09-10T10:05:10.227 INFO:teuthology.orchestra.run.smithi159.stdout: "buffer_meta_items": 1, 2024-09-10T10:05:10.227 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_bytes": 0, 2024-09-10T10:05:10.227 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_items": 0, 2024-09-10T10:05:10.227 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_mapbl_bytes": 0, 2024-09-10T10:05:10.227 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_mapbl_items": 0, 2024-09-10T10:05:10.227 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_pglog_bytes": 0, 2024-09-10T10:05:10.227 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_pglog_items": 0, 2024-09-10T10:05:10.227 INFO:teuthology.orchestra.run.smithi159.stdout: "osdmap_bytes": 9152, 2024-09-10T10:05:10.227 INFO:teuthology.orchestra.run.smithi159.stdout: "osdmap_items": 173, 2024-09-10T10:05:10.227 INFO:teuthology.orchestra.run.smithi159.stdout: "osdmap_mapping_bytes": 0, 2024-09-10T10:05:10.227 INFO:teuthology.orchestra.run.smithi159.stdout: "osdmap_mapping_items": 0, 2024-09-10T10:05:10.227 INFO:teuthology.orchestra.run.smithi159.stdout: "pgmap_bytes": 0, 2024-09-10T10:05:10.227 INFO:teuthology.orchestra.run.smithi159.stdout: "pgmap_items": 0, 2024-09-10T10:05:10.227 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_co_bytes": 96160, 2024-09-10T10:05:10.228 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_co_items": 395, 2024-09-10T10:05:10.228 INFO:teuthology.orchestra.run.smithi159.stdout: "unittest_1_bytes": 0, 2024-09-10T10:05:10.228 INFO:teuthology.orchestra.run.smithi159.stdout: "unittest_1_items": 0, 2024-09-10T10:05:10.228 INFO:teuthology.orchestra.run.smithi159.stdout: "unittest_2_bytes": 0, 2024-09-10T10:05:10.228 INFO:teuthology.orchestra.run.smithi159.stdout: "unittest_2_items": 0 2024-09-10T10:05:10.228 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-10T10:05:10.228 INFO:teuthology.orchestra.run.smithi159.stdout: "objecter": { 2024-09-10T10:05:10.228 INFO:teuthology.orchestra.run.smithi159.stdout: "op_active": 0, 2024-09-10T10:05:10.228 INFO:teuthology.orchestra.run.smithi159.stdout: "op_laggy": 0, 2024-09-10T10:05:10.228 INFO:teuthology.orchestra.run.smithi159.stdout: "op_send": 39839, 2024-09-10T10:05:10.228 INFO:teuthology.orchestra.run.smithi159.stdout: "op_send_bytes": 102131760, 2024-09-10T10:05:10.228 INFO:teuthology.orchestra.run.smithi159.stdout: "op_resend": 31, 2024-09-10T10:05:10.228 INFO:teuthology.orchestra.run.smithi159.stdout: "op_reply": 39839, 2024-09-10T10:05:10.228 INFO:teuthology.orchestra.run.smithi159.stdout: "op_latency": { 2024-09-10T10:05:10.229 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 39839, 2024-09-10T10:05:10.229 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 330.163407981, 2024-09-10T10:05:10.229 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.008287442 2024-09-10T10:05:10.229 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-10T10:05:10.229 INFO:teuthology.orchestra.run.smithi159.stdout: "op_inflight": 0, 2024-09-10T10:05:10.229 INFO:teuthology.orchestra.run.smithi159.stdout: "oplen_avg": { 2024-09-10T10:05:10.229 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 39839, 2024-09-10T10:05:10.229 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 40450 2024-09-10T10:05:10.229 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-10T10:05:10.229 INFO:teuthology.orchestra.run.smithi159.stdout: "op": 39839, 2024-09-10T10:05:10.229 INFO:teuthology.orchestra.run.smithi159.stdout: "op_r": 1332, 2024-09-10T10:05:10.229 INFO:teuthology.orchestra.run.smithi159.stdout: "op_w": 38507, 2024-09-10T10:05:10.229 INFO:teuthology.orchestra.run.smithi159.stdout: "op_rmw": 0, 2024-09-10T10:05:10.229 INFO:teuthology.orchestra.run.smithi159.stdout: "op_pg": 0, 2024-09-10T10:05:10.229 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_stat": 0, 2024-09-10T10:05:10.230 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_create": 10, 2024-09-10T10:05:10.230 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_read": 384, 2024-09-10T10:05:10.230 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_write": 18084, 2024-09-10T10:05:10.230 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_writefull": 254, 2024-09-10T10:05:10.230 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_writesame": 0, 2024-09-10T10:05:10.230 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_append": 0, 2024-09-10T10:05:10.230 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_zero": 0, 2024-09-10T10:05:10.230 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_truncate": 0, 2024-09-10T10:05:10.230 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_delete": 19718, 2024-09-10T10:05:10.230 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_mapext": 0, 2024-09-10T10:05:10.230 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_sparse_read": 0, 2024-09-10T10:05:10.230 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_clonerange": 0, 2024-09-10T10:05:10.230 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_getxattr": 948, 2024-09-10T10:05:10.230 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_setxattr": 20, 2024-09-10T10:05:10.230 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_cmpxattr": 0, 2024-09-10T10:05:10.231 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_rmxattr": 0, 2024-09-10T10:05:10.231 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_resetxattrs": 0, 2024-09-10T10:05:10.231 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_call": 0, 2024-09-10T10:05:10.231 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_watch": 0, 2024-09-10T10:05:10.231 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_notify": 0, 2024-09-10T10:05:10.231 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_src_cmpxattr": 0, 2024-09-10T10:05:10.231 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_pgls": 0, 2024-09-10T10:05:10.231 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_pgls_filter": 0, 2024-09-10T10:05:10.231 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_other": 0, 2024-09-10T10:05:10.231 INFO:teuthology.orchestra.run.smithi159.stdout: "linger_active": 0, 2024-09-10T10:05:10.231 INFO:teuthology.orchestra.run.smithi159.stdout: "linger_send": 0, 2024-09-10T10:05:10.231 INFO:teuthology.orchestra.run.smithi159.stdout: "linger_resend": 0, 2024-09-10T10:05:10.232 INFO:teuthology.orchestra.run.smithi159.stdout: "linger_ping": 0, 2024-09-10T10:05:10.232 INFO:teuthology.orchestra.run.smithi159.stdout: "poolop_active": 0, 2024-09-10T10:05:10.232 INFO:teuthology.orchestra.run.smithi159.stdout: "poolop_send": 0, 2024-09-10T10:05:10.232 INFO:teuthology.orchestra.run.smithi159.stdout: "poolop_resend": 0, 2024-09-10T10:05:10.232 INFO:teuthology.orchestra.run.smithi159.stdout: "poolstat_active": 0, 2024-09-10T10:05:10.232 INFO:teuthology.orchestra.run.smithi159.stdout: "poolstat_send": 0, 2024-09-10T10:05:10.232 INFO:teuthology.orchestra.run.smithi159.stdout: "poolstat_resend": 0, 2024-09-10T10:05:10.232 INFO:teuthology.orchestra.run.smithi159.stdout: "statfs_active": 0, 2024-09-10T10:05:10.232 INFO:teuthology.orchestra.run.smithi159.stdout: "statfs_send": 0, 2024-09-10T10:05:10.232 INFO:teuthology.orchestra.run.smithi159.stdout: "statfs_resend": 0, 2024-09-10T10:05:10.232 INFO:teuthology.orchestra.run.smithi159.stdout: "command_active": 0, 2024-09-10T10:05:10.233 INFO:teuthology.orchestra.run.smithi159.stdout: "command_send": 0, 2024-09-10T10:05:10.233 INFO:teuthology.orchestra.run.smithi159.stdout: "command_resend": 0, 2024-09-10T10:05:10.233 INFO:teuthology.orchestra.run.smithi159.stdout: "map_epoch": 96, 2024-09-10T10:05:10.233 INFO:teuthology.orchestra.run.smithi159.stdout: "map_full": 0, 2024-09-10T10:05:10.233 INFO:teuthology.orchestra.run.smithi159.stdout: "map_inc": 2, 2024-09-10T10:05:10.233 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_sessions": 12, 2024-09-10T10:05:10.233 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_session_open": 15, 2024-09-10T10:05:10.233 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_session_close": 3, 2024-09-10T10:05:10.233 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_laggy": 0, 2024-09-10T10:05:10.233 INFO:teuthology.orchestra.run.smithi159.stdout: "omap_wr": 767, 2024-09-10T10:05:10.233 INFO:teuthology.orchestra.run.smithi159.stdout: "omap_rd": 0, 2024-09-10T10:05:10.233 INFO:teuthology.orchestra.run.smithi159.stdout: "omap_del": 265 2024-09-10T10:05:10.233 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-10T10:05:10.233 INFO:teuthology.orchestra.run.smithi159.stdout: "oft": { 2024-09-10T10:05:10.234 INFO:teuthology.orchestra.run.smithi159.stdout: "omap_total_objs": 1, 2024-09-10T10:05:10.234 INFO:teuthology.orchestra.run.smithi159.stdout: "omap_total_kv_pairs": 0, 2024-09-10T10:05:10.234 INFO:teuthology.orchestra.run.smithi159.stdout: "omap_total_updates": 0, 2024-09-10T10:05:10.234 INFO:teuthology.orchestra.run.smithi159.stdout: "omap_total_removes": 0 2024-09-10T10:05:10.234 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-10T10:05:10.234 INFO:teuthology.orchestra.run.smithi159.stdout: "osd-slow-ops": { 2024-09-10T10:05:10.234 INFO:teuthology.orchestra.run.smithi159.stdout: "slow_ops_count": 1306 2024-09-10T10:05:10.234 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-10T10:05:10.234 INFO:teuthology.orchestra.run.smithi159.stdout: "purge_queue": { 2024-09-10T10:05:10.234 INFO:teuthology.orchestra.run.smithi159.stdout: "pq_executing_ops": 0, 2024-09-10T10:05:10.234 INFO:teuthology.orchestra.run.smithi159.stdout: "pq_executing_ops_high_water": 10, 2024-09-10T10:05:10.234 INFO:teuthology.orchestra.run.smithi159.stdout: "pq_executing": 0, 2024-09-10T10:05:10.234 INFO:teuthology.orchestra.run.smithi159.stdout: "pq_executing_high_water": 7, 2024-09-10T10:05:10.234 INFO:teuthology.orchestra.run.smithi159.stdout: "pq_executed_ops": 19683, 2024-09-10T10:05:10.235 INFO:teuthology.orchestra.run.smithi159.stdout: "pq_executed": 9846, 2024-09-10T10:05:10.235 INFO:teuthology.orchestra.run.smithi159.stdout: "pq_item_in_journal": 0 2024-09-10T10:05:10.235 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-10T10:05:10.235 INFO:teuthology.orchestra.run.smithi159.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-10T10:05:10.235 INFO:teuthology.orchestra.run.smithi159.stdout: "val": 0, 2024-09-10T10:05:10.235 INFO:teuthology.orchestra.run.smithi159.stdout: "max": 104857600, 2024-09-10T10:05:10.235 INFO:teuthology.orchestra.run.smithi159.stdout: "get_started": 0, 2024-09-10T10:05:10.235 INFO:teuthology.orchestra.run.smithi159.stdout: "get": 266537, 2024-09-10T10:05:10.235 INFO:teuthology.orchestra.run.smithi159.stdout: "get_sum": 53498522, 2024-09-10T10:05:10.235 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_fail": 0, 2024-09-10T10:05:10.235 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_success": 266537, 2024-09-10T10:05:10.235 INFO:teuthology.orchestra.run.smithi159.stdout: "take": 0, 2024-09-10T10:05:10.235 INFO:teuthology.orchestra.run.smithi159.stdout: "take_sum": 0, 2024-09-10T10:05:10.236 INFO:teuthology.orchestra.run.smithi159.stdout: "put": 266537, 2024-09-10T10:05:10.236 INFO:teuthology.orchestra.run.smithi159.stdout: "put_sum": 53498522, 2024-09-10T10:05:10.236 INFO:teuthology.orchestra.run.smithi159.stdout: "wait": { 2024-09-10T10:05:10.236 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-10T10:05:10.236 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-10T10:05:10.236 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-10T10:05:10.236 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-10T10:05:10.236 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-10T10:05:10.236 INFO:teuthology.orchestra.run.smithi159.stdout: "throttle-objecter_bytes": { 2024-09-10T10:05:10.236 INFO:teuthology.orchestra.run.smithi159.stdout: "val": 0, 2024-09-10T10:05:10.236 INFO:teuthology.orchestra.run.smithi159.stdout: "max": 104857600, 2024-09-10T10:05:10.236 INFO:teuthology.orchestra.run.smithi159.stdout: "get_started": 0, 2024-09-10T10:05:10.237 INFO:teuthology.orchestra.run.smithi159.stdout: "get": 0, 2024-09-10T10:05:10.237 INFO:teuthology.orchestra.run.smithi159.stdout: "get_sum": 0, 2024-09-10T10:05:10.237 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_fail": 0, 2024-09-10T10:05:10.237 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_success": 0, 2024-09-10T10:05:10.237 INFO:teuthology.orchestra.run.smithi159.stdout: "take": 39839, 2024-09-10T10:05:10.237 INFO:teuthology.orchestra.run.smithi159.stdout: "take_sum": 103204902, 2024-09-10T10:05:10.237 INFO:teuthology.orchestra.run.smithi159.stdout: "put": 39839, 2024-09-10T10:05:10.237 INFO:teuthology.orchestra.run.smithi159.stdout: "put_sum": 103204902, 2024-09-10T10:05:10.237 INFO:teuthology.orchestra.run.smithi159.stdout: "wait": { 2024-09-10T10:05:10.237 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-10T10:05:10.237 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-10T10:05:10.237 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-10T10:05:10.238 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-10T10:05:10.238 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-10T10:05:10.238 INFO:teuthology.orchestra.run.smithi159.stdout: "throttle-objecter_ops": { 2024-09-10T10:05:10.238 INFO:teuthology.orchestra.run.smithi159.stdout: "val": 0, 2024-09-10T10:05:10.238 INFO:teuthology.orchestra.run.smithi159.stdout: "max": 1024, 2024-09-10T10:05:10.238 INFO:teuthology.orchestra.run.smithi159.stdout: "get_started": 0, 2024-09-10T10:05:10.238 INFO:teuthology.orchestra.run.smithi159.stdout: "get": 0, 2024-09-10T10:05:10.238 INFO:teuthology.orchestra.run.smithi159.stdout: "get_sum": 0, 2024-09-10T10:05:10.238 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_fail": 0, 2024-09-10T10:05:10.238 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_success": 0, 2024-09-10T10:05:10.238 INFO:teuthology.orchestra.run.smithi159.stdout: "take": 39839, 2024-09-10T10:05:10.238 INFO:teuthology.orchestra.run.smithi159.stdout: "take_sum": 39839, 2024-09-10T10:05:10.238 INFO:teuthology.orchestra.run.smithi159.stdout: "put": 39839, 2024-09-10T10:05:10.239 INFO:teuthology.orchestra.run.smithi159.stdout: "put_sum": 39839, 2024-09-10T10:05:10.239 INFO:teuthology.orchestra.run.smithi159.stdout: "wait": { 2024-09-10T10:05:10.239 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-10T10:05:10.239 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-10T10:05:10.239 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-10T10:05:10.239 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-10T10:05:10.239 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-10T10:05:10.239 INFO:teuthology.orchestra.run.smithi159.stdout: "throttle-write_buf_throttle": { 2024-09-10T10:05:10.239 INFO:teuthology.orchestra.run.smithi159.stdout: "val": 0, 2024-09-10T10:05:10.239 INFO:teuthology.orchestra.run.smithi159.stdout: "max": 3758096384, 2024-09-10T10:05:10.239 INFO:teuthology.orchestra.run.smithi159.stdout: "get_started": 0, 2024-09-10T10:05:10.239 INFO:teuthology.orchestra.run.smithi159.stdout: "get": 9846, 2024-09-10T10:05:10.240 INFO:teuthology.orchestra.run.smithi159.stdout: "get_sum": 1073142, 2024-09-10T10:05:10.240 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_fail": 0, 2024-09-10T10:05:10.240 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_success": 9846, 2024-09-10T10:05:10.240 INFO:teuthology.orchestra.run.smithi159.stdout: "take": 0, 2024-09-10T10:05:10.240 INFO:teuthology.orchestra.run.smithi159.stdout: "take_sum": 0, 2024-09-10T10:05:10.240 INFO:teuthology.orchestra.run.smithi159.stdout: "put": 384, 2024-09-10T10:05:10.240 INFO:teuthology.orchestra.run.smithi159.stdout: "put_sum": 1073142, 2024-09-10T10:05:10.240 INFO:teuthology.orchestra.run.smithi159.stdout: "wait": { 2024-09-10T10:05:10.240 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-10T10:05:10.240 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-10T10:05:10.240 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-10T10:05:10.240 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-10T10:05:10.240 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-10T10:05:10.241 INFO:teuthology.orchestra.run.smithi159.stdout: "throttle-write_buf_throttle-0x55780c71e960": { 2024-09-10T10:05:10.241 INFO:teuthology.orchestra.run.smithi159.stdout: "val": 0, 2024-09-10T10:05:10.241 INFO:teuthology.orchestra.run.smithi159.stdout: "max": 3758096384, 2024-09-10T10:05:10.241 INFO:teuthology.orchestra.run.smithi159.stdout: "get_started": 0, 2024-09-10T10:05:10.241 INFO:teuthology.orchestra.run.smithi159.stdout: "get": 50281, 2024-09-10T10:05:10.241 INFO:teuthology.orchestra.run.smithi159.stdout: "get_sum": 99974048, 2024-09-10T10:05:10.241 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_fail": 0, 2024-09-10T10:05:10.241 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_success": 50281, 2024-09-10T10:05:10.241 INFO:teuthology.orchestra.run.smithi159.stdout: "take": 0, 2024-09-10T10:05:10.241 INFO:teuthology.orchestra.run.smithi159.stdout: "take_sum": 0, 2024-09-10T10:05:10.241 INFO:teuthology.orchestra.run.smithi159.stdout: "put": 17700, 2024-09-10T10:05:10.241 INFO:teuthology.orchestra.run.smithi159.stdout: "put_sum": 99974048, 2024-09-10T10:05:10.241 INFO:teuthology.orchestra.run.smithi159.stdout: "wait": { 2024-09-10T10:05:10.242 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-10T10:05:10.242 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-10T10:05:10.242 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-10T10:05:10.242 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-10T10:05:10.242 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-10T10:05:10.242 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-09-10T10:05:10.306 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:05:09 smithi005 ceph-mon[30350]: pgmap v1197: 129 pgs: 129 active+clean; 551 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-10T10:05:10.974 INFO:tasks.check_counter:Daemon mds.i mds.exported=2 2024-09-10T10:05:10.974 INFO:tasks.check_counter:Daemon mds.i mds.imported=3 2024-09-10T10:05:10.974 DEBUG:tasks.check_counter:Getting stats from l 2024-09-10T10:05:10.975 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph daemon mds.l perf dump 2024-09-10T10:05:11.219 INFO:teuthology.orchestra.run.smithi159.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.c/config 2024-09-10T10:05:11.777 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-09-10T10:05:11.777 INFO:teuthology.orchestra.run.smithi159.stdout: "AsyncMessenger::Worker-0": { 2024-09-10T10:05:11.777 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_messages": 3079, 2024-09-10T10:05:11.777 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_messages": 3081, 2024-09-10T10:05:11.777 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_bytes": 17709448, 2024-09-10T10:05:11.777 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_bytes": 979123, 2024-09-10T10:05:11.777 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_created_connections": 8, 2024-09-10T10:05:11.777 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_active_connections": 4, 2024-09-10T10:05:11.777 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_total_time": 0.277254833, 2024-09-10T10:05:11.777 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_send_time": 0.089931008, 2024-09-10T10:05:11.778 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_recv_time": 0.078439029, 2024-09-10T10:05:11.778 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_fast_dispatch_time": 0.084429786, 2024-09-10T10:05:11.778 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T10:05:11.778 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 3079, 2024-09-10T10:05:11.778 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.121730595, 2024-09-10T10:05:11.778 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000039535 2024-09-10T10:05:11.778 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-10T10:05:11.778 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_handle_ack_lat": { 2024-09-10T10:05:11.778 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-10T10:05:11.778 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-10T10:05:11.778 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-10T10:05:11.778 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-10T10:05:11.778 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-10T10:05:11.778 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_encrypted_bytes": 0 2024-09-10T10:05:11.779 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-10T10:05:11.779 INFO:teuthology.orchestra.run.smithi159.stdout: "AsyncMessenger::Worker-1": { 2024-09-10T10:05:11.779 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_messages": 2160, 2024-09-10T10:05:11.779 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_messages": 2528, 2024-09-10T10:05:11.779 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_bytes": 55499885, 2024-09-10T10:05:11.779 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_bytes": 1488152, 2024-09-10T10:05:11.779 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_created_connections": 8, 2024-09-10T10:05:11.779 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_active_connections": 5, 2024-09-10T10:05:11.779 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_total_time": 0.209642634, 2024-09-10T10:05:11.779 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_send_time": 0.070138652, 2024-09-10T10:05:11.779 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_recv_time": 0.065881357, 2024-09-10T10:05:11.779 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_fast_dispatch_time": 0.057643130, 2024-09-10T10:05:11.779 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T10:05:11.780 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 2526, 2024-09-10T10:05:11.780 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.089762922, 2024-09-10T10:05:11.780 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000035535 2024-09-10T10:05:11.780 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-10T10:05:11.780 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_handle_ack_lat": { 2024-09-10T10:05:11.780 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-10T10:05:11.780 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-10T10:05:11.780 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-10T10:05:11.780 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-10T10:05:11.780 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-10T10:05:11.780 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_encrypted_bytes": 801840 2024-09-10T10:05:11.780 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-10T10:05:11.780 INFO:teuthology.orchestra.run.smithi159.stdout: "AsyncMessenger::Worker-2": { 2024-09-10T10:05:11.781 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_messages": 774, 2024-09-10T10:05:11.781 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_messages": 745, 2024-09-10T10:05:11.781 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_bytes": 13644840, 2024-09-10T10:05:11.781 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_bytes": 381738, 2024-09-10T10:05:11.781 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_created_connections": 6, 2024-09-10T10:05:11.781 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_active_connections": 4, 2024-09-10T10:05:11.781 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_total_time": 0.093084366, 2024-09-10T10:05:11.781 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_send_time": 0.035331012, 2024-09-10T10:05:11.781 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_recv_time": 0.057708812, 2024-09-10T10:05:11.781 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_running_fast_dispatch_time": 0.006092063, 2024-09-10T10:05:11.781 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T10:05:11.781 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 745, 2024-09-10T10:05:11.781 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.041189870, 2024-09-10T10:05:11.782 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000055288 2024-09-10T10:05:11.782 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-10T10:05:11.782 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_handle_ack_lat": { 2024-09-10T10:05:11.782 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-10T10:05:11.782 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-10T10:05:11.782 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-10T10:05:11.782 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-10T10:05:11.782 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_recv_encrypted_bytes": 998240, 2024-09-10T10:05:11.782 INFO:teuthology.orchestra.run.smithi159.stdout: "msgr_send_encrypted_bytes": 301920 2024-09-10T10:05:11.782 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-10T10:05:11.782 INFO:teuthology.orchestra.run.smithi159.stdout: "cct": { 2024-09-10T10:05:11.782 INFO:teuthology.orchestra.run.smithi159.stdout: "total_workers": 1, 2024-09-10T10:05:11.782 INFO:teuthology.orchestra.run.smithi159.stdout: "unhealthy_workers": 0 2024-09-10T10:05:11.782 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-10T10:05:11.783 INFO:teuthology.orchestra.run.smithi159.stdout: "finisher-MDSRank": { 2024-09-10T10:05:11.783 INFO:teuthology.orchestra.run.smithi159.stdout: "queue_len": 0, 2024-09-10T10:05:11.783 INFO:teuthology.orchestra.run.smithi159.stdout: "complete_latency": { 2024-09-10T10:05:11.783 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 9220, 2024-09-10T10:05:11.783 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.219093501, 2024-09-10T10:05:11.783 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000023762 2024-09-10T10:05:11.783 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-10T10:05:11.783 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-10T10:05:11.783 INFO:teuthology.orchestra.run.smithi159.stdout: "finisher-PurgeQueue": { 2024-09-10T10:05:11.783 INFO:teuthology.orchestra.run.smithi159.stdout: "queue_len": 0, 2024-09-10T10:05:11.783 INFO:teuthology.orchestra.run.smithi159.stdout: "complete_latency": { 2024-09-10T10:05:11.783 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-10T10:05:11.783 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-10T10:05:11.783 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-10T10:05:11.784 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-10T10:05:11.784 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-10T10:05:11.784 INFO:teuthology.orchestra.run.smithi159.stdout: "mds": { 2024-09-10T10:05:11.784 INFO:teuthology.orchestra.run.smithi159.stdout: "request": 0, 2024-09-10T10:05:11.784 INFO:teuthology.orchestra.run.smithi159.stdout: "reply": 0, 2024-09-10T10:05:11.784 INFO:teuthology.orchestra.run.smithi159.stdout: "reply_latency": { 2024-09-10T10:05:11.784 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-10T10:05:11.784 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-10T10:05:11.784 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-10T10:05:11.784 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-10T10:05:11.784 INFO:teuthology.orchestra.run.smithi159.stdout: "slow_reply": 0, 2024-09-10T10:05:11.784 INFO:teuthology.orchestra.run.smithi159.stdout: "forward": 0, 2024-09-10T10:05:11.784 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_fetch_complete": 0, 2024-09-10T10:05:11.785 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_fetch_keys": 0, 2024-09-10T10:05:11.785 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_commit": 0, 2024-09-10T10:05:11.785 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_split": 0, 2024-09-10T10:05:11.785 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_merge": 0, 2024-09-10T10:05:11.785 INFO:teuthology.orchestra.run.smithi159.stdout: "inodes": 7411, 2024-09-10T10:05:11.785 INFO:teuthology.orchestra.run.smithi159.stdout: "inodes_top": 0, 2024-09-10T10:05:11.785 INFO:teuthology.orchestra.run.smithi159.stdout: "inodes_bottom": 7411, 2024-09-10T10:05:11.785 INFO:teuthology.orchestra.run.smithi159.stdout: "inodes_pin_tail": 0, 2024-09-10T10:05:11.785 INFO:teuthology.orchestra.run.smithi159.stdout: "inodes_pinned": 7411, 2024-09-10T10:05:11.785 INFO:teuthology.orchestra.run.smithi159.stdout: "inodes_expired": 0, 2024-09-10T10:05:11.785 INFO:teuthology.orchestra.run.smithi159.stdout: "inodes_with_caps": 0, 2024-09-10T10:05:11.785 INFO:teuthology.orchestra.run.smithi159.stdout: "caps": 0, 2024-09-10T10:05:11.785 INFO:teuthology.orchestra.run.smithi159.stdout: "subtrees": 1, 2024-09-10T10:05:11.785 INFO:teuthology.orchestra.run.smithi159.stdout: "traverse": 0, 2024-09-10T10:05:11.786 INFO:teuthology.orchestra.run.smithi159.stdout: "traverse_hit": 0, 2024-09-10T10:05:11.786 INFO:teuthology.orchestra.run.smithi159.stdout: "traverse_forward": 0, 2024-09-10T10:05:11.786 INFO:teuthology.orchestra.run.smithi159.stdout: "traverse_discover": 0, 2024-09-10T10:05:11.786 INFO:teuthology.orchestra.run.smithi159.stdout: "traverse_dir_fetch": 0, 2024-09-10T10:05:11.786 INFO:teuthology.orchestra.run.smithi159.stdout: "traverse_remote_ino": 0, 2024-09-10T10:05:11.786 INFO:teuthology.orchestra.run.smithi159.stdout: "traverse_lock": 0, 2024-09-10T10:05:11.786 INFO:teuthology.orchestra.run.smithi159.stdout: "load_cent": 0, 2024-09-10T10:05:11.786 INFO:teuthology.orchestra.run.smithi159.stdout: "q": 0, 2024-09-10T10:05:11.786 INFO:teuthology.orchestra.run.smithi159.stdout: "exported": 0, 2024-09-10T10:05:11.786 INFO:teuthology.orchestra.run.smithi159.stdout: "exported_inodes": 0, 2024-09-10T10:05:11.786 INFO:teuthology.orchestra.run.smithi159.stdout: "imported": 0, 2024-09-10T10:05:11.786 INFO:teuthology.orchestra.run.smithi159.stdout: "imported_inodes": 0, 2024-09-10T10:05:11.786 INFO:teuthology.orchestra.run.smithi159.stdout: "openino_dir_fetch": 0, 2024-09-10T10:05:11.786 INFO:teuthology.orchestra.run.smithi159.stdout: "openino_backtrace_fetch": 0, 2024-09-10T10:05:11.787 INFO:teuthology.orchestra.run.smithi159.stdout: "openino_peer_discover": 0, 2024-09-10T10:05:11.787 INFO:teuthology.orchestra.run.smithi159.stdout: "root_rfiles": 0, 2024-09-10T10:05:11.787 INFO:teuthology.orchestra.run.smithi159.stdout: "root_rbytes": 0, 2024-09-10T10:05:11.787 INFO:teuthology.orchestra.run.smithi159.stdout: "root_rsnaps": 0, 2024-09-10T10:05:11.787 INFO:teuthology.orchestra.run.smithi159.stdout: "scrub_backtrace_fetch": 0, 2024-09-10T10:05:11.787 INFO:teuthology.orchestra.run.smithi159.stdout: "scrub_set_tag": 0, 2024-09-10T10:05:11.787 INFO:teuthology.orchestra.run.smithi159.stdout: "scrub_backtrace_repaired": 0, 2024-09-10T10:05:11.787 INFO:teuthology.orchestra.run.smithi159.stdout: "scrub_inotable_repaired": 0, 2024-09-10T10:05:11.787 INFO:teuthology.orchestra.run.smithi159.stdout: "scrub_dir_inodes": 0, 2024-09-10T10:05:11.787 INFO:teuthology.orchestra.run.smithi159.stdout: "scrub_dir_base_inodes": 0, 2024-09-10T10:05:11.787 INFO:teuthology.orchestra.run.smithi159.stdout: "scrub_dirfrag_rstats": 0, 2024-09-10T10:05:11.787 INFO:teuthology.orchestra.run.smithi159.stdout: "scrub_file_inodes": 0, 2024-09-10T10:05:11.787 INFO:teuthology.orchestra.run.smithi159.stdout: "handle_inode_file_caps": 0, 2024-09-10T10:05:11.787 INFO:teuthology.orchestra.run.smithi159.stdout: "ceph_cap_op_revoke": 0, 2024-09-10T10:05:11.787 INFO:teuthology.orchestra.run.smithi159.stdout: "ceph_cap_op_grant": 0, 2024-09-10T10:05:11.788 INFO:teuthology.orchestra.run.smithi159.stdout: "ceph_cap_op_trunc": 0, 2024-09-10T10:05:11.788 INFO:teuthology.orchestra.run.smithi159.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-10T10:05:11.788 INFO:teuthology.orchestra.run.smithi159.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-10T10:05:11.788 INFO:teuthology.orchestra.run.smithi159.stdout: "handle_client_caps": 0, 2024-09-10T10:05:11.788 INFO:teuthology.orchestra.run.smithi159.stdout: "handle_client_caps_dirty": 0, 2024-09-10T10:05:11.788 INFO:teuthology.orchestra.run.smithi159.stdout: "handle_client_cap_release": 0, 2024-09-10T10:05:11.788 INFO:teuthology.orchestra.run.smithi159.stdout: "process_request_cap_release": 0 2024-09-10T10:05:11.788 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-10T10:05:11.788 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_cache": { 2024-09-10T10:05:11.788 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_update": 0, 2024-09-10T10:05:11.788 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_update_receipt": 0, 2024-09-10T10:05:11.788 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_try_discover": 0, 2024-09-10T10:05:11.789 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_send_discover": 0, 2024-09-10T10:05:11.789 INFO:teuthology.orchestra.run.smithi159.stdout: "dir_handle_discover": 0, 2024-09-10T10:05:11.789 INFO:teuthology.orchestra.run.smithi159.stdout: "num_strays": 0, 2024-09-10T10:05:11.789 INFO:teuthology.orchestra.run.smithi159.stdout: "num_strays_delayed": 0, 2024-09-10T10:05:11.789 INFO:teuthology.orchestra.run.smithi159.stdout: "num_strays_enqueuing": 0, 2024-09-10T10:05:11.789 INFO:teuthology.orchestra.run.smithi159.stdout: "strays_created": 0, 2024-09-10T10:05:11.789 INFO:teuthology.orchestra.run.smithi159.stdout: "strays_enqueued": 0, 2024-09-10T10:05:11.789 INFO:teuthology.orchestra.run.smithi159.stdout: "strays_reintegrated": 0, 2024-09-10T10:05:11.789 INFO:teuthology.orchestra.run.smithi159.stdout: "strays_migrated": 0, 2024-09-10T10:05:11.789 INFO:teuthology.orchestra.run.smithi159.stdout: "num_recovering_processing": 0, 2024-09-10T10:05:11.789 INFO:teuthology.orchestra.run.smithi159.stdout: "num_recovering_enqueued": 0, 2024-09-10T10:05:11.789 INFO:teuthology.orchestra.run.smithi159.stdout: "num_recovering_prioritized": 0, 2024-09-10T10:05:11.789 INFO:teuthology.orchestra.run.smithi159.stdout: "recovery_started": 0, 2024-09-10T10:05:11.789 INFO:teuthology.orchestra.run.smithi159.stdout: "recovery_completed": 0, 2024-09-10T10:05:11.790 INFO:teuthology.orchestra.run.smithi159.stdout: "ireq_quiesce_path": 0, 2024-09-10T10:05:11.790 INFO:teuthology.orchestra.run.smithi159.stdout: "ireq_quiesce_inode": 0, 2024-09-10T10:05:11.790 INFO:teuthology.orchestra.run.smithi159.stdout: "ireq_enqueue_scrub": 0, 2024-09-10T10:05:11.790 INFO:teuthology.orchestra.run.smithi159.stdout: "ireq_exportdir": 0, 2024-09-10T10:05:11.790 INFO:teuthology.orchestra.run.smithi159.stdout: "ireq_flush": 0, 2024-09-10T10:05:11.790 INFO:teuthology.orchestra.run.smithi159.stdout: "ireq_fragmentdir": 0, 2024-09-10T10:05:11.790 INFO:teuthology.orchestra.run.smithi159.stdout: "ireq_fragstats": 0, 2024-09-10T10:05:11.790 INFO:teuthology.orchestra.run.smithi159.stdout: "ireq_inodestats": 0 2024-09-10T10:05:11.790 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-10T10:05:11.790 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_log": { 2024-09-10T10:05:11.790 INFO:teuthology.orchestra.run.smithi159.stdout: "evlrg": 0, 2024-09-10T10:05:11.790 INFO:teuthology.orchestra.run.smithi159.stdout: "evadd": 0, 2024-09-10T10:05:11.791 INFO:teuthology.orchestra.run.smithi159.stdout: "evex": 0, 2024-09-10T10:05:11.791 INFO:teuthology.orchestra.run.smithi159.stdout: "evtrm": 0, 2024-09-10T10:05:11.791 INFO:teuthology.orchestra.run.smithi159.stdout: "ev": 43840, 2024-09-10T10:05:11.791 INFO:teuthology.orchestra.run.smithi159.stdout: "evexg": 0, 2024-09-10T10:05:11.791 INFO:teuthology.orchestra.run.smithi159.stdout: "evexd": 0, 2024-09-10T10:05:11.791 INFO:teuthology.orchestra.run.smithi159.stdout: "segadd": 0, 2024-09-10T10:05:11.791 INFO:teuthology.orchestra.run.smithi159.stdout: "segex": 0, 2024-09-10T10:05:11.791 INFO:teuthology.orchestra.run.smithi159.stdout: "segtrm": 0, 2024-09-10T10:05:11.791 INFO:teuthology.orchestra.run.smithi159.stdout: "seg": 59, 2024-09-10T10:05:11.791 INFO:teuthology.orchestra.run.smithi159.stdout: "segmjr": 5, 2024-09-10T10:05:11.791 INFO:teuthology.orchestra.run.smithi159.stdout: "segexg": 0, 2024-09-10T10:05:11.791 INFO:teuthology.orchestra.run.smithi159.stdout: "segexd": 0, 2024-09-10T10:05:11.791 INFO:teuthology.orchestra.run.smithi159.stdout: "expos": 4194304, 2024-09-10T10:05:11.791 INFO:teuthology.orchestra.run.smithi159.stdout: "wrpos": 88509846, 2024-09-10T10:05:11.792 INFO:teuthology.orchestra.run.smithi159.stdout: "rdpos": 88509247, 2024-09-10T10:05:11.792 INFO:teuthology.orchestra.run.smithi159.stdout: "jlat": { 2024-09-10T10:05:11.792 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-10T10:05:11.792 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-10T10:05:11.792 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-10T10:05:11.792 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-10T10:05:11.792 INFO:teuthology.orchestra.run.smithi159.stdout: "replayed": 43840 2024-09-10T10:05:11.792 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-10T10:05:11.792 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_mem": { 2024-09-10T10:05:11.792 INFO:teuthology.orchestra.run.smithi159.stdout: "ino": 13, 2024-09-10T10:05:11.792 INFO:teuthology.orchestra.run.smithi159.stdout: "ino+": 7434, 2024-09-10T10:05:11.792 INFO:teuthology.orchestra.run.smithi159.stdout: "ino-": 7421, 2024-09-10T10:05:11.792 INFO:teuthology.orchestra.run.smithi159.stdout: "dir": 11, 2024-09-10T10:05:11.793 INFO:teuthology.orchestra.run.smithi159.stdout: "dir+": 43, 2024-09-10T10:05:11.793 INFO:teuthology.orchestra.run.smithi159.stdout: "dir-": 32, 2024-09-10T10:05:11.793 INFO:teuthology.orchestra.run.smithi159.stdout: "dn": 7411, 2024-09-10T10:05:11.793 INFO:teuthology.orchestra.run.smithi159.stdout: "dn+": 7577, 2024-09-10T10:05:11.793 INFO:teuthology.orchestra.run.smithi159.stdout: "dn-": 166, 2024-09-10T10:05:11.793 INFO:teuthology.orchestra.run.smithi159.stdout: "cap": 0, 2024-09-10T10:05:11.793 INFO:teuthology.orchestra.run.smithi159.stdout: "cap+": 0, 2024-09-10T10:05:11.793 INFO:teuthology.orchestra.run.smithi159.stdout: "cap-": 0, 2024-09-10T10:05:11.793 INFO:teuthology.orchestra.run.smithi159.stdout: "rss": 58388, 2024-09-10T10:05:11.793 INFO:teuthology.orchestra.run.smithi159.stdout: "heap": 215328 2024-09-10T10:05:11.793 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-10T10:05:11.793 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_server": { 2024-09-10T10:05:11.793 INFO:teuthology.orchestra.run.smithi159.stdout: "dispatch_client_request": 0, 2024-09-10T10:05:11.794 INFO:teuthology.orchestra.run.smithi159.stdout: "dispatch_server_request": 0, 2024-09-10T10:05:11.794 INFO:teuthology.orchestra.run.smithi159.stdout: "handle_client_request": 0, 2024-09-10T10:05:11.794 INFO:teuthology.orchestra.run.smithi159.stdout: "handle_client_session": 0, 2024-09-10T10:05:11.794 INFO:teuthology.orchestra.run.smithi159.stdout: "handle_peer_request": 0, 2024-09-10T10:05:11.794 INFO:teuthology.orchestra.run.smithi159.stdout: "req_create_latency": { 2024-09-10T10:05:11.794 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-10T10:05:11.794 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-10T10:05:11.794 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-10T10:05:11.794 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-10T10:05:11.794 INFO:teuthology.orchestra.run.smithi159.stdout: "req_getattr_latency": { 2024-09-10T10:05:11.794 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-10T10:05:11.795 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-10T10:05:11.795 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-10T10:05:11.795 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-10T10:05:11.795 INFO:teuthology.orchestra.run.smithi159.stdout: "req_getfilelock_latency": { 2024-09-10T10:05:11.795 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-10T10:05:11.795 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-10T10:05:11.795 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-10T10:05:11.795 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-10T10:05:11.795 INFO:teuthology.orchestra.run.smithi159.stdout: "req_link_latency": { 2024-09-10T10:05:11.795 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-10T10:05:11.795 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-10T10:05:11.795 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-10T10:05:11.796 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-10T10:05:11.796 INFO:teuthology.orchestra.run.smithi159.stdout: "req_lookup_latency": { 2024-09-10T10:05:11.796 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-10T10:05:11.796 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-10T10:05:11.796 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-10T10:05:11.796 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-10T10:05:11.796 INFO:teuthology.orchestra.run.smithi159.stdout: "req_lookuphash_latency": { 2024-09-10T10:05:11.796 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-10T10:05:11.796 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-10T10:05:11.796 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-10T10:05:11.796 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-10T10:05:11.796 INFO:teuthology.orchestra.run.smithi159.stdout: "req_lookupino_latency": { 2024-09-10T10:05:11.796 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-10T10:05:11.796 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-10T10:05:11.797 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-10T10:05:11.797 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-10T10:05:11.797 INFO:teuthology.orchestra.run.smithi159.stdout: "req_lookupname_latency": { 2024-09-10T10:05:11.797 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-10T10:05:11.797 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-10T10:05:11.797 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-10T10:05:11.797 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-10T10:05:11.797 INFO:teuthology.orchestra.run.smithi159.stdout: "req_lookupparent_latency": { 2024-09-10T10:05:11.797 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-10T10:05:11.797 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-10T10:05:11.797 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-10T10:05:11.797 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-10T10:05:11.797 INFO:teuthology.orchestra.run.smithi159.stdout: "req_lookupsnap_latency": { 2024-09-10T10:05:11.798 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-10T10:05:11.798 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-10T10:05:11.798 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-10T10:05:11.798 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-10T10:05:11.798 INFO:teuthology.orchestra.run.smithi159.stdout: "req_lssnap_latency": { 2024-09-10T10:05:11.798 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-10T10:05:11.798 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-10T10:05:11.798 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-10T10:05:11.798 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-10T10:05:11.798 INFO:teuthology.orchestra.run.smithi159.stdout: "req_mkdir_latency": { 2024-09-10T10:05:11.798 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-10T10:05:11.798 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-10T10:05:11.798 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-10T10:05:11.798 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-10T10:05:11.799 INFO:teuthology.orchestra.run.smithi159.stdout: "req_mknod_latency": { 2024-09-10T10:05:11.799 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-10T10:05:11.799 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-10T10:05:11.799 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-10T10:05:11.799 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-10T10:05:11.799 INFO:teuthology.orchestra.run.smithi159.stdout: "req_mksnap_latency": { 2024-09-10T10:05:11.799 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-10T10:05:11.799 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-10T10:05:11.799 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-10T10:05:11.799 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-10T10:05:11.799 INFO:teuthology.orchestra.run.smithi159.stdout: "req_open_latency": { 2024-09-10T10:05:11.799 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-10T10:05:11.799 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-10T10:05:11.800 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-10T10:05:11.800 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-10T10:05:11.800 INFO:teuthology.orchestra.run.smithi159.stdout: "req_readdir_latency": { 2024-09-10T10:05:11.800 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-10T10:05:11.800 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-10T10:05:11.800 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-10T10:05:11.800 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-10T10:05:11.800 INFO:teuthology.orchestra.run.smithi159.stdout: "req_rename_latency": { 2024-09-10T10:05:11.800 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-10T10:05:11.800 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-10T10:05:11.800 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-10T10:05:11.800 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-10T10:05:11.800 INFO:teuthology.orchestra.run.smithi159.stdout: "req_renamesnap_latency": { 2024-09-10T10:05:11.801 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-10T10:05:11.801 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-10T10:05:11.801 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-10T10:05:11.801 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-10T10:05:11.801 INFO:teuthology.orchestra.run.smithi159.stdout: "req_snapdiff_latency": { 2024-09-10T10:05:11.801 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-10T10:05:11.801 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-10T10:05:11.801 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-10T10:05:11.801 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-10T10:05:11.801 INFO:teuthology.orchestra.run.smithi159.stdout: "req_rmdir_latency": { 2024-09-10T10:05:11.801 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-10T10:05:11.801 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-10T10:05:11.801 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-10T10:05:11.801 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-10T10:05:11.802 INFO:teuthology.orchestra.run.smithi159.stdout: "req_rmsnap_latency": { 2024-09-10T10:05:11.802 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-10T10:05:11.802 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-10T10:05:11.802 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-10T10:05:11.802 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-10T10:05:11.802 INFO:teuthology.orchestra.run.smithi159.stdout: "req_rmxattr_latency": { 2024-09-10T10:05:11.802 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-10T10:05:11.802 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-10T10:05:11.802 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-10T10:05:11.802 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-10T10:05:11.802 INFO:teuthology.orchestra.run.smithi159.stdout: "req_setattr_latency": { 2024-09-10T10:05:11.802 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-10T10:05:11.802 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-10T10:05:11.803 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-10T10:05:11.803 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-10T10:05:11.803 INFO:teuthology.orchestra.run.smithi159.stdout: "req_setdirlayout_latency": { 2024-09-10T10:05:11.803 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-10T10:05:11.803 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-10T10:05:11.803 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-10T10:05:11.803 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-10T10:05:11.803 INFO:teuthology.orchestra.run.smithi159.stdout: "req_setfilelock_latency": { 2024-09-10T10:05:11.803 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-10T10:05:11.803 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-10T10:05:11.803 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-10T10:05:11.803 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-10T10:05:11.803 INFO:teuthology.orchestra.run.smithi159.stdout: "req_setlayout_latency": { 2024-09-10T10:05:11.803 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-10T10:05:11.804 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-10T10:05:11.804 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-10T10:05:11.804 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-10T10:05:11.804 INFO:teuthology.orchestra.run.smithi159.stdout: "req_setxattr_latency": { 2024-09-10T10:05:11.804 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-10T10:05:11.804 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-10T10:05:11.804 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-10T10:05:11.804 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-10T10:05:11.804 INFO:teuthology.orchestra.run.smithi159.stdout: "req_symlink_latency": { 2024-09-10T10:05:11.804 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-10T10:05:11.804 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-10T10:05:11.804 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-10T10:05:11.804 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-10T10:05:11.805 INFO:teuthology.orchestra.run.smithi159.stdout: "req_unlink_latency": { 2024-09-10T10:05:11.805 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-10T10:05:11.805 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-10T10:05:11.805 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-10T10:05:11.805 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-10T10:05:11.805 INFO:teuthology.orchestra.run.smithi159.stdout: "cap_revoke_eviction": 0, 2024-09-10T10:05:11.805 INFO:teuthology.orchestra.run.smithi159.stdout: "cap_acquisition_throttle": 0, 2024-09-10T10:05:11.805 INFO:teuthology.orchestra.run.smithi159.stdout: "req_getvxattr_latency": { 2024-09-10T10:05:11.805 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-10T10:05:11.805 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-10T10:05:11.805 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-10T10:05:11.805 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-10T10:05:11.805 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-10T10:05:11.806 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_sessions": { 2024-09-10T10:05:11.806 INFO:teuthology.orchestra.run.smithi159.stdout: "session_count": 1, 2024-09-10T10:05:11.806 INFO:teuthology.orchestra.run.smithi159.stdout: "session_add": 1, 2024-09-10T10:05:11.806 INFO:teuthology.orchestra.run.smithi159.stdout: "session_remove": 0, 2024-09-10T10:05:11.806 INFO:teuthology.orchestra.run.smithi159.stdout: "sessions_open": 1, 2024-09-10T10:05:11.806 INFO:teuthology.orchestra.run.smithi159.stdout: "sessions_stale": 0, 2024-09-10T10:05:11.806 INFO:teuthology.orchestra.run.smithi159.stdout: "total_load": 0, 2024-09-10T10:05:11.806 INFO:teuthology.orchestra.run.smithi159.stdout: "average_load": 0, 2024-09-10T10:05:11.806 INFO:teuthology.orchestra.run.smithi159.stdout: "avg_session_uptime": 3004, 2024-09-10T10:05:11.806 INFO:teuthology.orchestra.run.smithi159.stdout: "mdthresh_evicted": 0 2024-09-10T10:05:11.806 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-10T10:05:11.806 INFO:teuthology.orchestra.run.smithi159.stdout: "mempool": { 2024-09-10T10:05:11.806 INFO:teuthology.orchestra.run.smithi159.stdout: "bloom_filter_bytes": 0, 2024-09-10T10:05:11.806 INFO:teuthology.orchestra.run.smithi159.stdout: "bloom_filter_items": 0, 2024-09-10T10:05:11.807 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_alloc_bytes": 0, 2024-09-10T10:05:11.807 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_alloc_items": 0, 2024-09-10T10:05:11.807 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_data_bytes": 0, 2024-09-10T10:05:11.807 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_data_items": 0, 2024-09-10T10:05:11.807 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-10T10:05:11.807 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_onode_items": 0, 2024-09-10T10:05:11.807 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-10T10:05:11.807 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_meta_items": 0, 2024-09-10T10:05:11.807 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_other_bytes": 0, 2024-09-10T10:05:11.807 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_other_items": 0, 2024-09-10T10:05:11.807 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-10T10:05:11.807 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_cache_buffer_items": 0, 2024-09-10T10:05:11.808 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_extent_bytes": 0, 2024-09-10T10:05:11.808 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_extent_items": 0, 2024-09-10T10:05:11.808 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_blob_bytes": 0, 2024-09-10T10:05:11.808 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_blob_items": 0, 2024-09-10T10:05:11.808 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-10T10:05:11.808 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_shared_blob_items": 0, 2024-09-10T10:05:11.808 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-10T10:05:11.808 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_inline_bl_items": 0, 2024-09-10T10:05:11.808 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_fsck_bytes": 0, 2024-09-10T10:05:11.808 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_fsck_items": 0, 2024-09-10T10:05:11.808 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_txc_bytes": 0, 2024-09-10T10:05:11.808 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_txc_items": 0, 2024-09-10T10:05:11.808 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-10T10:05:11.808 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_writing_deferred_items": 0, 2024-09-10T10:05:11.809 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_writing_bytes": 0, 2024-09-10T10:05:11.809 INFO:teuthology.orchestra.run.smithi159.stdout: "bluestore_writing_items": 0, 2024-09-10T10:05:11.809 INFO:teuthology.orchestra.run.smithi159.stdout: "bluefs_bytes": 0, 2024-09-10T10:05:11.809 INFO:teuthology.orchestra.run.smithi159.stdout: "bluefs_items": 0, 2024-09-10T10:05:11.809 INFO:teuthology.orchestra.run.smithi159.stdout: "bluefs_file_reader_bytes": 0, 2024-09-10T10:05:11.809 INFO:teuthology.orchestra.run.smithi159.stdout: "bluefs_file_reader_items": 0, 2024-09-10T10:05:11.809 INFO:teuthology.orchestra.run.smithi159.stdout: "bluefs_file_writer_bytes": 0, 2024-09-10T10:05:11.809 INFO:teuthology.orchestra.run.smithi159.stdout: "bluefs_file_writer_items": 0, 2024-09-10T10:05:11.809 INFO:teuthology.orchestra.run.smithi159.stdout: "buffer_anon_bytes": 232919, 2024-09-10T10:05:11.809 INFO:teuthology.orchestra.run.smithi159.stdout: "buffer_anon_items": 78, 2024-09-10T10:05:11.809 INFO:teuthology.orchestra.run.smithi159.stdout: "buffer_meta_bytes": 0, 2024-09-10T10:05:11.809 INFO:teuthology.orchestra.run.smithi159.stdout: "buffer_meta_items": 0, 2024-09-10T10:05:11.809 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_bytes": 0, 2024-09-10T10:05:11.809 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_items": 0, 2024-09-10T10:05:11.810 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_mapbl_bytes": 0, 2024-09-10T10:05:11.810 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_mapbl_items": 0, 2024-09-10T10:05:11.810 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_pglog_bytes": 0, 2024-09-10T10:05:11.810 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_pglog_items": 0, 2024-09-10T10:05:11.810 INFO:teuthology.orchestra.run.smithi159.stdout: "osdmap_bytes": 9152, 2024-09-10T10:05:11.810 INFO:teuthology.orchestra.run.smithi159.stdout: "osdmap_items": 173, 2024-09-10T10:05:11.810 INFO:teuthology.orchestra.run.smithi159.stdout: "osdmap_mapping_bytes": 0, 2024-09-10T10:05:11.810 INFO:teuthology.orchestra.run.smithi159.stdout: "osdmap_mapping_items": 0, 2024-09-10T10:05:11.810 INFO:teuthology.orchestra.run.smithi159.stdout: "pgmap_bytes": 0, 2024-09-10T10:05:11.810 INFO:teuthology.orchestra.run.smithi159.stdout: "pgmap_items": 0, 2024-09-10T10:05:11.810 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_co_bytes": 5260936, 2024-09-10T10:05:11.810 INFO:teuthology.orchestra.run.smithi159.stdout: "mds_co_items": 29772, 2024-09-10T10:05:11.810 INFO:teuthology.orchestra.run.smithi159.stdout: "unittest_1_bytes": 0, 2024-09-10T10:05:11.811 INFO:teuthology.orchestra.run.smithi159.stdout: "unittest_1_items": 0, 2024-09-10T10:05:11.811 INFO:teuthology.orchestra.run.smithi159.stdout: "unittest_2_bytes": 0, 2024-09-10T10:05:11.811 INFO:teuthology.orchestra.run.smithi159.stdout: "unittest_2_items": 0 2024-09-10T10:05:11.811 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-10T10:05:11.811 INFO:teuthology.orchestra.run.smithi159.stdout: "objecter": { 2024-09-10T10:05:11.811 INFO:teuthology.orchestra.run.smithi159.stdout: "op_active": 0, 2024-09-10T10:05:11.811 INFO:teuthology.orchestra.run.smithi159.stdout: "op_laggy": 0, 2024-09-10T10:05:11.811 INFO:teuthology.orchestra.run.smithi159.stdout: "op_send": 5488, 2024-09-10T10:05:11.811 INFO:teuthology.orchestra.run.smithi159.stdout: "op_send_bytes": 21, 2024-09-10T10:05:11.811 INFO:teuthology.orchestra.run.smithi159.stdout: "op_resend": 3, 2024-09-10T10:05:11.811 INFO:teuthology.orchestra.run.smithi159.stdout: "op_reply": 5488, 2024-09-10T10:05:11.811 INFO:teuthology.orchestra.run.smithi159.stdout: "op_latency": { 2024-09-10T10:05:11.811 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 5488, 2024-09-10T10:05:11.811 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 4.588908308, 2024-09-10T10:05:11.812 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000836171 2024-09-10T10:05:11.812 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-10T10:05:11.812 INFO:teuthology.orchestra.run.smithi159.stdout: "op_inflight": 0, 2024-09-10T10:05:11.812 INFO:teuthology.orchestra.run.smithi159.stdout: "oplen_avg": { 2024-09-10T10:05:11.812 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 5488, 2024-09-10T10:05:11.812 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 5489 2024-09-10T10:05:11.812 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-10T10:05:11.812 INFO:teuthology.orchestra.run.smithi159.stdout: "op": 5488, 2024-09-10T10:05:11.812 INFO:teuthology.orchestra.run.smithi159.stdout: "op_r": 5488, 2024-09-10T10:05:11.812 INFO:teuthology.orchestra.run.smithi159.stdout: "op_w": 0, 2024-09-10T10:05:11.812 INFO:teuthology.orchestra.run.smithi159.stdout: "op_rmw": 0, 2024-09-10T10:05:11.812 INFO:teuthology.orchestra.run.smithi159.stdout: "op_pg": 0, 2024-09-10T10:05:11.812 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_stat": 3310, 2024-09-10T10:05:11.812 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_create": 0, 2024-09-10T10:05:11.813 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_read": 2176, 2024-09-10T10:05:11.813 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_write": 0, 2024-09-10T10:05:11.813 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_writefull": 0, 2024-09-10T10:05:11.813 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_writesame": 0, 2024-09-10T10:05:11.813 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_append": 0, 2024-09-10T10:05:11.813 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_zero": 0, 2024-09-10T10:05:11.813 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_truncate": 0, 2024-09-10T10:05:11.813 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_delete": 0, 2024-09-10T10:05:11.813 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_mapext": 0, 2024-09-10T10:05:11.813 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_sparse_read": 0, 2024-09-10T10:05:11.813 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_clonerange": 0, 2024-09-10T10:05:11.813 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_getxattr": 1, 2024-09-10T10:05:11.813 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_setxattr": 0, 2024-09-10T10:05:11.814 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_cmpxattr": 0, 2024-09-10T10:05:11.814 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_rmxattr": 0, 2024-09-10T10:05:11.814 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_resetxattrs": 0, 2024-09-10T10:05:11.814 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_call": 0, 2024-09-10T10:05:11.814 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_watch": 0, 2024-09-10T10:05:11.814 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_notify": 0, 2024-09-10T10:05:11.814 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_src_cmpxattr": 0, 2024-09-10T10:05:11.814 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_pgls": 0, 2024-09-10T10:05:11.814 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_pgls_filter": 0, 2024-09-10T10:05:11.814 INFO:teuthology.orchestra.run.smithi159.stdout: "osdop_other": 0, 2024-09-10T10:05:11.814 INFO:teuthology.orchestra.run.smithi159.stdout: "linger_active": 0, 2024-09-10T10:05:11.814 INFO:teuthology.orchestra.run.smithi159.stdout: "linger_send": 0, 2024-09-10T10:05:11.814 INFO:teuthology.orchestra.run.smithi159.stdout: "linger_resend": 0, 2024-09-10T10:05:11.815 INFO:teuthology.orchestra.run.smithi159.stdout: "linger_ping": 0, 2024-09-10T10:05:11.815 INFO:teuthology.orchestra.run.smithi159.stdout: "poolop_active": 0, 2024-09-10T10:05:11.815 INFO:teuthology.orchestra.run.smithi159.stdout: "poolop_send": 0, 2024-09-10T10:05:11.815 INFO:teuthology.orchestra.run.smithi159.stdout: "poolop_resend": 0, 2024-09-10T10:05:11.815 INFO:teuthology.orchestra.run.smithi159.stdout: "poolstat_active": 0, 2024-09-10T10:05:11.815 INFO:teuthology.orchestra.run.smithi159.stdout: "poolstat_send": 0, 2024-09-10T10:05:11.815 INFO:teuthology.orchestra.run.smithi159.stdout: "poolstat_resend": 0, 2024-09-10T10:05:11.815 INFO:teuthology.orchestra.run.smithi159.stdout: "statfs_active": 0, 2024-09-10T10:05:11.815 INFO:teuthology.orchestra.run.smithi159.stdout: "statfs_send": 0, 2024-09-10T10:05:11.815 INFO:teuthology.orchestra.run.smithi159.stdout: "statfs_resend": 0, 2024-09-10T10:05:11.815 INFO:teuthology.orchestra.run.smithi159.stdout: "command_active": 0, 2024-09-10T10:05:11.815 INFO:teuthology.orchestra.run.smithi159.stdout: "command_send": 0, 2024-09-10T10:05:11.815 INFO:teuthology.orchestra.run.smithi159.stdout: "command_resend": 0, 2024-09-10T10:05:11.815 INFO:teuthology.orchestra.run.smithi159.stdout: "map_epoch": 96, 2024-09-10T10:05:11.815 INFO:teuthology.orchestra.run.smithi159.stdout: "map_full": 0, 2024-09-10T10:05:11.816 INFO:teuthology.orchestra.run.smithi159.stdout: "map_inc": 2, 2024-09-10T10:05:11.816 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_sessions": 10, 2024-09-10T10:05:11.816 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_session_open": 12, 2024-09-10T10:05:11.816 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_session_close": 2, 2024-09-10T10:05:11.816 INFO:teuthology.orchestra.run.smithi159.stdout: "osd_laggy": 0, 2024-09-10T10:05:11.816 INFO:teuthology.orchestra.run.smithi159.stdout: "omap_wr": 0, 2024-09-10T10:05:11.816 INFO:teuthology.orchestra.run.smithi159.stdout: "omap_rd": 2, 2024-09-10T10:05:11.816 INFO:teuthology.orchestra.run.smithi159.stdout: "omap_del": 0 2024-09-10T10:05:11.816 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-10T10:05:11.816 INFO:teuthology.orchestra.run.smithi159.stdout: "oft": { 2024-09-10T10:05:11.816 INFO:teuthology.orchestra.run.smithi159.stdout: "omap_total_objs": 0, 2024-09-10T10:05:11.816 INFO:teuthology.orchestra.run.smithi159.stdout: "omap_total_kv_pairs": 0, 2024-09-10T10:05:11.816 INFO:teuthology.orchestra.run.smithi159.stdout: "omap_total_updates": 0, 2024-09-10T10:05:11.816 INFO:teuthology.orchestra.run.smithi159.stdout: "omap_total_removes": 0 2024-09-10T10:05:11.817 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-10T10:05:11.817 INFO:teuthology.orchestra.run.smithi159.stdout: "osd-slow-ops": { 2024-09-10T10:05:11.817 INFO:teuthology.orchestra.run.smithi159.stdout: "slow_ops_count": 0 2024-09-10T10:05:11.817 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-10T10:05:11.817 INFO:teuthology.orchestra.run.smithi159.stdout: "purge_queue": { 2024-09-10T10:05:11.817 INFO:teuthology.orchestra.run.smithi159.stdout: "pq_executing_ops": 0, 2024-09-10T10:05:11.817 INFO:teuthology.orchestra.run.smithi159.stdout: "pq_executing_ops_high_water": 0, 2024-09-10T10:05:11.817 INFO:teuthology.orchestra.run.smithi159.stdout: "pq_executing": 0, 2024-09-10T10:05:11.817 INFO:teuthology.orchestra.run.smithi159.stdout: "pq_executing_high_water": 0, 2024-09-10T10:05:11.817 INFO:teuthology.orchestra.run.smithi159.stdout: "pq_executed_ops": 0, 2024-09-10T10:05:11.817 INFO:teuthology.orchestra.run.smithi159.stdout: "pq_executed": 0, 2024-09-10T10:05:11.817 INFO:teuthology.orchestra.run.smithi159.stdout: "pq_item_in_journal": 0 2024-09-10T10:05:11.817 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-10T10:05:11.817 INFO:teuthology.orchestra.run.smithi159.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-10T10:05:11.818 INFO:teuthology.orchestra.run.smithi159.stdout: "val": 0, 2024-09-10T10:05:11.818 INFO:teuthology.orchestra.run.smithi159.stdout: "max": 104857600, 2024-09-10T10:05:11.818 INFO:teuthology.orchestra.run.smithi159.stdout: "get_started": 0, 2024-09-10T10:05:11.818 INFO:teuthology.orchestra.run.smithi159.stdout: "get": 6009, 2024-09-10T10:05:11.818 INFO:teuthology.orchestra.run.smithi159.stdout: "get_sum": 86283681, 2024-09-10T10:05:11.818 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_fail": 0, 2024-09-10T10:05:11.818 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_success": 6009, 2024-09-10T10:05:11.818 INFO:teuthology.orchestra.run.smithi159.stdout: "take": 0, 2024-09-10T10:05:11.818 INFO:teuthology.orchestra.run.smithi159.stdout: "take_sum": 0, 2024-09-10T10:05:11.818 INFO:teuthology.orchestra.run.smithi159.stdout: "put": 6009, 2024-09-10T10:05:11.818 INFO:teuthology.orchestra.run.smithi159.stdout: "put_sum": 86283681, 2024-09-10T10:05:11.818 INFO:teuthology.orchestra.run.smithi159.stdout: "wait": { 2024-09-10T10:05:11.818 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-10T10:05:11.818 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-10T10:05:11.819 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-10T10:05:11.819 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-10T10:05:11.819 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-10T10:05:11.819 INFO:teuthology.orchestra.run.smithi159.stdout: "throttle-objecter_bytes": { 2024-09-10T10:05:11.819 INFO:teuthology.orchestra.run.smithi159.stdout: "val": 0, 2024-09-10T10:05:11.819 INFO:teuthology.orchestra.run.smithi159.stdout: "max": 104857600, 2024-09-10T10:05:11.819 INFO:teuthology.orchestra.run.smithi159.stdout: "get_started": 0, 2024-09-10T10:05:11.819 INFO:teuthology.orchestra.run.smithi159.stdout: "get": 0, 2024-09-10T10:05:11.819 INFO:teuthology.orchestra.run.smithi159.stdout: "get_sum": 0, 2024-09-10T10:05:11.819 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_fail": 0, 2024-09-10T10:05:11.819 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_success": 0, 2024-09-10T10:05:11.819 INFO:teuthology.orchestra.run.smithi159.stdout: "take": 5488, 2024-09-10T10:05:11.819 INFO:teuthology.orchestra.run.smithi159.stdout: "take_sum": 84319523, 2024-09-10T10:05:11.819 INFO:teuthology.orchestra.run.smithi159.stdout: "put": 5488, 2024-09-10T10:05:11.819 INFO:teuthology.orchestra.run.smithi159.stdout: "put_sum": 84319523, 2024-09-10T10:05:11.820 INFO:teuthology.orchestra.run.smithi159.stdout: "wait": { 2024-09-10T10:05:11.820 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-10T10:05:11.820 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-10T10:05:11.820 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-10T10:05:11.820 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-10T10:05:11.820 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-10T10:05:11.820 INFO:teuthology.orchestra.run.smithi159.stdout: "throttle-objecter_ops": { 2024-09-10T10:05:11.820 INFO:teuthology.orchestra.run.smithi159.stdout: "val": 0, 2024-09-10T10:05:11.820 INFO:teuthology.orchestra.run.smithi159.stdout: "max": 1024, 2024-09-10T10:05:11.820 INFO:teuthology.orchestra.run.smithi159.stdout: "get_started": 0, 2024-09-10T10:05:11.820 INFO:teuthology.orchestra.run.smithi159.stdout: "get": 0, 2024-09-10T10:05:11.820 INFO:teuthology.orchestra.run.smithi159.stdout: "get_sum": 0, 2024-09-10T10:05:11.821 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_fail": 0, 2024-09-10T10:05:11.821 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_success": 0, 2024-09-10T10:05:11.821 INFO:teuthology.orchestra.run.smithi159.stdout: "take": 5488, 2024-09-10T10:05:11.821 INFO:teuthology.orchestra.run.smithi159.stdout: "take_sum": 5488, 2024-09-10T10:05:11.821 INFO:teuthology.orchestra.run.smithi159.stdout: "put": 5488, 2024-09-10T10:05:11.821 INFO:teuthology.orchestra.run.smithi159.stdout: "put_sum": 5488, 2024-09-10T10:05:11.821 INFO:teuthology.orchestra.run.smithi159.stdout: "wait": { 2024-09-10T10:05:11.821 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-10T10:05:11.821 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-10T10:05:11.821 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-10T10:05:11.821 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-10T10:05:11.821 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-10T10:05:11.821 INFO:teuthology.orchestra.run.smithi159.stdout: "throttle-write_buf_throttle": { 2024-09-10T10:05:11.821 INFO:teuthology.orchestra.run.smithi159.stdout: "val": 0, 2024-09-10T10:05:11.822 INFO:teuthology.orchestra.run.smithi159.stdout: "max": 3758096384, 2024-09-10T10:05:11.822 INFO:teuthology.orchestra.run.smithi159.stdout: "get_started": 0, 2024-09-10T10:05:11.822 INFO:teuthology.orchestra.run.smithi159.stdout: "get": 0, 2024-09-10T10:05:11.822 INFO:teuthology.orchestra.run.smithi159.stdout: "get_sum": 0, 2024-09-10T10:05:11.822 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_fail": 0, 2024-09-10T10:05:11.822 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_success": 0, 2024-09-10T10:05:11.822 INFO:teuthology.orchestra.run.smithi159.stdout: "take": 0, 2024-09-10T10:05:11.822 INFO:teuthology.orchestra.run.smithi159.stdout: "take_sum": 0, 2024-09-10T10:05:11.822 INFO:teuthology.orchestra.run.smithi159.stdout: "put": 0, 2024-09-10T10:05:11.822 INFO:teuthology.orchestra.run.smithi159.stdout: "put_sum": 0, 2024-09-10T10:05:11.822 INFO:teuthology.orchestra.run.smithi159.stdout: "wait": { 2024-09-10T10:05:11.822 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-10T10:05:11.822 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-10T10:05:11.822 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-10T10:05:11.823 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-10T10:05:11.823 INFO:teuthology.orchestra.run.smithi159.stdout: }, 2024-09-10T10:05:11.823 INFO:teuthology.orchestra.run.smithi159.stdout: "throttle-write_buf_throttle-0x559964bf6b40": { 2024-09-10T10:05:11.823 INFO:teuthology.orchestra.run.smithi159.stdout: "val": 0, 2024-09-10T10:05:11.823 INFO:teuthology.orchestra.run.smithi159.stdout: "max": 3758096384, 2024-09-10T10:05:11.823 INFO:teuthology.orchestra.run.smithi159.stdout: "get_started": 0, 2024-09-10T10:05:11.823 INFO:teuthology.orchestra.run.smithi159.stdout: "get": 0, 2024-09-10T10:05:11.823 INFO:teuthology.orchestra.run.smithi159.stdout: "get_sum": 0, 2024-09-10T10:05:11.823 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_fail": 0, 2024-09-10T10:05:11.823 INFO:teuthology.orchestra.run.smithi159.stdout: "get_or_fail_success": 0, 2024-09-10T10:05:11.823 INFO:teuthology.orchestra.run.smithi159.stdout: "take": 0, 2024-09-10T10:05:11.823 INFO:teuthology.orchestra.run.smithi159.stdout: "take_sum": 0, 2024-09-10T10:05:11.823 INFO:teuthology.orchestra.run.smithi159.stdout: "put": 0, 2024-09-10T10:05:11.823 INFO:teuthology.orchestra.run.smithi159.stdout: "put_sum": 0, 2024-09-10T10:05:11.823 INFO:teuthology.orchestra.run.smithi159.stdout: "wait": { 2024-09-10T10:05:11.824 INFO:teuthology.orchestra.run.smithi159.stdout: "avgcount": 0, 2024-09-10T10:05:11.824 INFO:teuthology.orchestra.run.smithi159.stdout: "sum": 0.000000000, 2024-09-10T10:05:11.824 INFO:teuthology.orchestra.run.smithi159.stdout: "avgtime": 0.000000000 2024-09-10T10:05:11.824 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-10T10:05:11.824 INFO:teuthology.orchestra.run.smithi159.stdout: } 2024-09-10T10:05:11.824 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-09-10T10:05:12.100 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:05:11 smithi159 ceph-mon[35513]: pgmap v1198: 129 pgs: 129 active+clean; 551 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-09-10T10:05:12.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:05:11 smithi102 ceph-mon[35442]: pgmap v1198: 129 pgs: 129 active+clean; 551 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-09-10T10:05:12.306 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:05:11 smithi005 ceph-mon[30350]: pgmap v1198: 129 pgs: 129 active+clean; 551 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 11 op/s 2024-09-10T10:05:12.619 INFO:tasks.check_counter:Daemon mds.l mds.exported=0 2024-09-10T10:05:12.619 INFO:tasks.check_counter:Daemon mds.l mds.imported=0 2024-09-10T10:05:12.620 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-09-10T10:05:12.631 INFO:tasks.kclient:Unmounting kernel clients... 2024-09-10T10:05:12.632 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-09-10T10:05:12.632 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:05:12.674 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-09-10T10:05:12.674 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T10:05:12.753 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph osd blocklist ls 2024-09-10T10:05:13.178 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:05:13.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:13.724+0000 7f8197200640 1 -- 172.21.15.5:0/2578766866 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8198102360 msgr2=0x7f819810e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:13.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:13.724+0000 7f8197200640 1 --2- 172.21.15.5:0/2578766866 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8198102360 0x7f819810e890 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f8190009cd0 tx=0x7f819002a8f0 comp rx=0 tx=0).stop 2024-09-10T10:05:13.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:13.724+0000 7f8197200640 1 -- 172.21.15.5:0/2578766866 shutdown_connections 2024-09-10T10:05:13.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:13.724+0000 7f8197200640 1 --2- 172.21.15.5:0/2578766866 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8198102360 0x7f819810e890 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:13.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:13.724+0000 7f8197200640 1 --2- 172.21.15.5:0/2578766866 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f81981019c0 0x7f8198101e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:13.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:13.724+0000 7f8197200640 1 --2- 172.21.15.5:0/2578766866 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f81981079c0 0x7f8198107da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:13.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:13.724+0000 7f8197200640 1 -- 172.21.15.5:0/2578766866 >> 172.21.15.5:0/2578766866 conn(0x7f81980fd740 msgr2=0x7f81980ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:05:13.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:13.724+0000 7f8197200640 1 -- 172.21.15.5:0/2578766866 shutdown_connections 2024-09-10T10:05:13.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:13.724+0000 7f8197200640 1 -- 172.21.15.5:0/2578766866 wait complete. 2024-09-10T10:05:13.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:13.725+0000 7f8197200640 1 Processor -- start 2024-09-10T10:05:13.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:13.725+0000 7f8197200640 1 -- start start 2024-09-10T10:05:13.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:13.726+0000 7f8197200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f81981019c0 0x7f819819a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:13.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:13.726+0000 7f8197200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8198102360 0x7f819819a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:13.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:13.726+0000 7f8197200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f81981079c0 0x7f819819ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:13.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:13.726+0000 7f8197200640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f8198111dc0 con 0x7f8198102360 2024-09-10T10:05:13.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:13.726+0000 7f8197200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f8198111c40 con 0x7f81981079c0 2024-09-10T10:05:13.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:13.726+0000 7f8197200640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f8198111f40 con 0x7f81981019c0 2024-09-10T10:05:13.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:13.726+0000 7f8195400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8198102360 0x7f819819a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:13.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:13.726+0000 7f8195400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8198102360 0x7f819819a9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35248/0 (socket says 172.21.15.5:35248) 2024-09-10T10:05:13.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:13.726+0000 7f8195400640 1 -- 172.21.15.5:0/1347298582 learned_addr learned my addr 172.21.15.5:0/1347298582 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T10:05:13.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:13.726+0000 7f8196800640 1 --2- 172.21.15.5:0/1347298582 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f81981079c0 0x7f819819ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:13.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:13.726+0000 7f8195e00640 1 --2- 172.21.15.5:0/1347298582 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f81981019c0 0x7f819819a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:13.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:13.726+0000 7f8195400640 1 -- 172.21.15.5:0/1347298582 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f81981019c0 msgr2=0x7f819819a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:13.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:13.726+0000 7f8195400640 1 --2- 172.21.15.5:0/1347298582 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f81981019c0 0x7f819819a4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:13.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:13.726+0000 7f8195400640 1 -- 172.21.15.5:0/1347298582 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f81981079c0 msgr2=0x7f819819ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:13.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:13.726+0000 7f8195400640 1 --2- 172.21.15.5:0/1347298582 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f81981079c0 0x7f819819ed70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:13.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:13.726+0000 7f8195400640 1 -- 172.21.15.5:0/1347298582 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f819819f450 con 0x7f8198102360 2024-09-10T10:05:13.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:13.727+0000 7f8195400640 1 --2- 172.21.15.5:0/1347298582 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8198102360 0x7f819819a9e0 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f818400d950 tx=0x7f818400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:13.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:13.727+0000 7f818ea00640 1 -- 172.21.15.5:0/1347298582 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8184017070 con 0x7f8198102360 2024-09-10T10:05:13.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:13.727+0000 7f818ea00640 1 -- 172.21.15.5:0/1347298582 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8184009c30 con 0x7f8198102360 2024-09-10T10:05:13.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:13.727+0000 7f8197200640 1 -- 172.21.15.5:0/1347298582 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f819819f740 con 0x7f8198102360 2024-09-10T10:05:13.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:13.727+0000 7f818ea00640 1 -- 172.21.15.5:0/1347298582 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f818400f6b0 con 0x7f8198102360 2024-09-10T10:05:13.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:13.727+0000 7f8197200640 1 -- 172.21.15.5:0/1347298582 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81981a7020 con 0x7f8198102360 2024-09-10T10:05:13.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:13.728+0000 7f8197200640 1 -- 172.21.15.5:0/1347298582 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8198101e20 con 0x7f8198102360 2024-09-10T10:05:13.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:13.731+0000 7f818ea00640 1 -- 172.21.15.5:0/1347298582 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f81840109d0 con 0x7f8198102360 2024-09-10T10:05:13.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:13.732+0000 7f818ea00640 1 --2- 172.21.15.5:0/1347298582 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f817c0a2b80 0x7f817c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:13.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:13.732+0000 7f818ea00640 1 -- 172.21.15.5:0/1347298582 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f81840cf070 con 0x7f8198102360 2024-09-10T10:05:13.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:13.732+0000 7f8195e00640 1 --2- 172.21.15.5:0/1347298582 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f817c0a2b80 0x7f817c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:13.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:13.733+0000 7f8195e00640 1 --2- 172.21.15.5:0/1347298582 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f817c0a2b80 0x7f817c0a5040 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f8188009b50 tx=0x7f8188005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:13.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:13.736+0000 7f818ea00640 1 -- 172.21.15.5:0/1347298582 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f8184009de0 con 0x7f8198102360 2024-09-10T10:05:13.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:13.935+0000 7f8197200640 1 -- 172.21.15.5:0/1347298582 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f81981a7350 con 0x7f8198102360 2024-09-10T10:05:13.940 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:05:13 smithi005 ceph-mon[30350]: pgmap v1199: 129 pgs: 129 active+clean; 551 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-10T10:05:13.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:13.935+0000 7f818ea00640 1 -- 172.21.15.5:0/1347298582 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v96) ==== 81+0+576 (secure 0 0 0) 0x7f818409b530 con 0x7f8198102360 2024-09-10T10:05:13.941 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5:6801/1527422686 2024-09-11T09:28:05.436742+0000 2024-09-10T10:05:13.941 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5:0/2276144345 2024-09-11T09:28:05.436742+0000 2024-09-10T10:05:13.941 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5:0/412040104 2024-09-11T09:28:05.436742+0000 2024-09-10T10:05:13.941 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5:6800/1527422686 2024-09-11T09:28:05.436742+0000 2024-09-10T10:05:13.941 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5:0/1856581170 2024-09-11T09:28:05.436742+0000 2024-09-10T10:05:13.941 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5:0/2847668298 2024-09-11T09:27:37.778500+0000 2024-09-10T10:05:13.942 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5:6801/14425913 2024-09-11T09:27:37.778500+0000 2024-09-10T10:05:13.942 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5:0/166910782 2024-09-11T09:27:37.778500+0000 2024-09-10T10:05:13.942 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5:6800/14425913 2024-09-11T09:27:37.778500+0000 2024-09-10T10:05:13.942 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5:0/1711858179 2024-09-11T09:27:37.778500+0000 2024-09-10T10:05:13.942 INFO:teuthology.orchestra.run.smithi005.stderr:listed 10 entries 2024-09-10T10:05:13.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:13.938+0000 7f8197200640 1 -- 172.21.15.5:0/1347298582 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f817c0a2b80 msgr2=0x7f817c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:13.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:13.938+0000 7f8197200640 1 --2- 172.21.15.5:0/1347298582 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f817c0a2b80 0x7f817c0a5040 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f8188009b50 tx=0x7f8188005f90 comp rx=0 tx=0).stop 2024-09-10T10:05:13.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:13.938+0000 7f8197200640 1 -- 172.21.15.5:0/1347298582 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8198102360 msgr2=0x7f819819a9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:13.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:13.938+0000 7f8197200640 1 --2- 172.21.15.5:0/1347298582 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8198102360 0x7f819819a9e0 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f818400d950 tx=0x7f818400de10 comp rx=0 tx=0).stop 2024-09-10T10:05:13.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:13.938+0000 7f8197200640 1 -- 172.21.15.5:0/1347298582 shutdown_connections 2024-09-10T10:05:13.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:13.938+0000 7f8197200640 1 --2- 172.21.15.5:0/1347298582 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f817c0a2b80 0x7f817c0a5040 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:13.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:13.938+0000 7f8197200640 1 --2- 172.21.15.5:0/1347298582 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f81981079c0 0x7f819819ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:13.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:13.938+0000 7f8197200640 1 --2- 172.21.15.5:0/1347298582 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8198102360 0x7f819819a9e0 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:13.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:13.938+0000 7f8197200640 1 --2- 172.21.15.5:0/1347298582 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f81981019c0 0x7f819819a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:13.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:13.938+0000 7f8197200640 1 -- 172.21.15.5:0/1347298582 >> 172.21.15.5:0/1347298582 conn(0x7f81980fd740 msgr2=0x7f819810d010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:05:13.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:13.938+0000 7f8197200640 1 -- 172.21.15.5:0/1347298582 shutdown_connections 2024-09-10T10:05:13.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:13.938+0000 7f8197200640 1 -- 172.21.15.5:0/1347298582 wait complete. 2024-09-10T10:05:14.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:05:13 smithi102 ceph-mon[35442]: pgmap v1199: 129 pgs: 129 active+clean; 551 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-10T10:05:14.352 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:05:13 smithi159 ceph-mon[35513]: pgmap v1199: 129 pgs: 129 active+clean; 551 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-10T10:05:14.715 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-10T10:05:14.715 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:05:14.716 DEBUG:teuthology.orchestra.run.smithi005:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-10T10:05:14.865 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi005.front.sepia.ceph.com 2024-09-10T10:05:14.865 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:05:14.865 DEBUG:teuthology.orchestra.run.smithi005:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-10T10:05:14.940 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:05:14.940 DEBUG:teuthology.orchestra.run.smithi005:> ip netns list 2024-09-10T10:05:15.021 INFO:teuthology.orchestra.run.smithi005.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-10T10:05:15.022 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:05:15.022 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T10:05:15.059 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:05:14 smithi005 ceph-mon[30350]: from='client.? 172.21.15.5:0/1347298582' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-10T10:05:15.152 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T10:05:15.152 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip link delete ceph-brx 2024-09-10T10:05:15.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:05:14 smithi159 ceph-mon[35513]: from='client.? 172.21.15.5:0/1347298582' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-10T10:05:15.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:05:14 smithi102 ceph-mon[35442]: from='client.? 172.21.15.5:0/1347298582' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-10T10:05:15.453 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-09-10T10:05:15.550 INFO:tasks.cephadm:Teardown begin 2024-09-10T10:05:15.551 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-10T10:05:15.551 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T10:05:15.932 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:05:16.253 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:05:15 smithi005 ceph-mon[30350]: pgmap v1200: 129 pgs: 129 active+clean; 551 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-10T10:05:16.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:05:15 smithi159 ceph-mon[35513]: pgmap v1200: 129 pgs: 129 active+clean; 551 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-10T10:05:16.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:05:15 smithi102 ceph-mon[35442]: pgmap v1200: 129 pgs: 129 active+clean; 551 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-10T10:05:16.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:16.543+0000 7ff758a00640 1 -- 172.21.15.5:0/2431278025 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff754101df0 msgr2=0x7ff7541021d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:16.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:16.543+0000 7ff758a00640 1 --2- 172.21.15.5:0/2431278025 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff754101df0 0x7ff7541021d0 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7ff744009900 tx=0x7ff74402a410 comp rx=0 tx=0).stop 2024-09-10T10:05:16.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:16.543+0000 7ff758a00640 1 -- 172.21.15.5:0/2431278025 shutdown_connections 2024-09-10T10:05:16.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:16.543+0000 7ff758a00640 1 --2- 172.21.15.5:0/2431278025 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff754106f20 0x7ff75410eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:16.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:16.543+0000 7ff758a00640 1 --2- 172.21.15.5:0/2431278025 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff7541027a0 0x7ff7541067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:16.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:16.543+0000 7ff758a00640 1 --2- 172.21.15.5:0/2431278025 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff754101df0 0x7ff7541021d0 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:16.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:16.543+0000 7ff758a00640 1 -- 172.21.15.5:0/2431278025 >> 172.21.15.5:0/2431278025 conn(0x7ff7540fd8b0 msgr2=0x7ff7540ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:05:16.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:16.543+0000 7ff758a00640 1 -- 172.21.15.5:0/2431278025 shutdown_connections 2024-09-10T10:05:16.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:16.543+0000 7ff758a00640 1 -- 172.21.15.5:0/2431278025 wait complete. 2024-09-10T10:05:16.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:16.544+0000 7ff758a00640 1 Processor -- start 2024-09-10T10:05:16.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:16.544+0000 7ff758a00640 1 -- start start 2024-09-10T10:05:16.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:16.544+0000 7ff758a00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff754101df0 0x7ff7541983f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:16.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:16.544+0000 7ff758a00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff7541027a0 0x7ff754198930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:16.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:16.544+0000 7ff758a00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff754106f20 0x7ff75419ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:16.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:16.544+0000 7ff758a00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff754111fd0 con 0x7ff754106f20 2024-09-10T10:05:16.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:16.544+0000 7ff758a00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7ff754111e50 con 0x7ff754101df0 2024-09-10T10:05:16.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:16.544+0000 7ff758a00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7ff754112150 con 0x7ff7541027a0 2024-09-10T10:05:16.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:16.545+0000 7ff751e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff754101df0 0x7ff7541983f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:16.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:16.545+0000 7ff752800640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff754106f20 0x7ff75419ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:16.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:16.545+0000 7ff751400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff7541027a0 0x7ff754198930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:16.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:16.545+0000 7ff751e00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff754101df0 0x7ff7541983f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:50304/0 (socket says 172.21.15.5:50304) 2024-09-10T10:05:16.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:16.545+0000 7ff751e00640 1 -- 172.21.15.5:0/4111611722 learned_addr learned my addr 172.21.15.5:0/4111611722 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T10:05:16.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:16.545+0000 7ff752800640 1 -- 172.21.15.5:0/4111611722 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff7541027a0 msgr2=0x7ff754198930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:16.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:16.545+0000 7ff752800640 1 --2- 172.21.15.5:0/4111611722 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff7541027a0 0x7ff754198930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:16.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:16.545+0000 7ff752800640 1 -- 172.21.15.5:0/4111611722 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff754101df0 msgr2=0x7ff7541983f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:16.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:16.545+0000 7ff752800640 1 --2- 172.21.15.5:0/4111611722 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff754101df0 0x7ff7541983f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:16.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:16.545+0000 7ff752800640 1 -- 172.21.15.5:0/4111611722 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff75419d3a0 con 0x7ff754106f20 2024-09-10T10:05:16.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:16.545+0000 7ff751e00640 1 --2- 172.21.15.5:0/4111611722 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff754101df0 0x7ff7541983f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:05:16.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:16.545+0000 7ff751400640 1 --2- 172.21.15.5:0/4111611722 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff7541027a0 0x7ff754198930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:05:16.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:16.546+0000 7ff752800640 1 --2- 172.21.15.5:0/4111611722 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff754106f20 0x7ff75419ccc0 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7ff74c00e940 tx=0x7ff74c00ee00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:16.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:16.546+0000 7ff74aa00640 1 -- 172.21.15.5:0/4111611722 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff74c004d20 con 0x7ff754106f20 2024-09-10T10:05:16.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:16.546+0000 7ff74aa00640 1 -- 172.21.15.5:0/4111611722 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff74c005020 con 0x7ff754106f20 2024-09-10T10:05:16.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:16.546+0000 7ff758a00640 1 -- 172.21.15.5:0/4111611722 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff75419d690 con 0x7ff754106f20 2024-09-10T10:05:16.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:16.546+0000 7ff74aa00640 1 -- 172.21.15.5:0/4111611722 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff74c0116b0 con 0x7ff754106f20 2024-09-10T10:05:16.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:16.546+0000 7ff758a00640 1 -- 172.21.15.5:0/4111611722 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7541a4f70 con 0x7ff754106f20 2024-09-10T10:05:16.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:16.547+0000 7ff758a00640 1 -- 172.21.15.5:0/4111611722 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff7541021d0 con 0x7ff754106f20 2024-09-10T10:05:16.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:16.549+0000 7ff74aa00640 1 -- 172.21.15.5:0/4111611722 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7ff74c0054d0 con 0x7ff754106f20 2024-09-10T10:05:16.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:16.550+0000 7ff74aa00640 1 --2- 172.21.15.5:0/4111611722 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7ff7380a2b80 0x7ff7380a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:16.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:16.550+0000 7ff74aa00640 1 -- 172.21.15.5:0/4111611722 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7ff74c0cf480 con 0x7ff754106f20 2024-09-10T10:05:16.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:16.550+0000 7ff751e00640 1 --2- 172.21.15.5:0/4111611722 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7ff7380a2b80 0x7ff7380a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:16.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:16.551+0000 7ff751e00640 1 --2- 172.21.15.5:0/4111611722 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7ff7380a2b80 0x7ff7380a5040 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7ff7440096c0 tx=0x7ff7440046c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:16.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:16.555+0000 7ff74aa00640 1 -- 172.21.15.5:0/4111611722 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff74c016030 con 0x7ff754106f20 2024-09-10T10:05:16.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:16.774+0000 7ff758a00640 1 -- 172.21.15.5:0/4111611722 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff75402d7c0 con 0x7ff754106f20 2024-09-10T10:05:16.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:16.775+0000 7ff74aa00640 1 -- 172.21.15.5:0/4111611722 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) ==== 94+0+11978 (secure 0 0 0) 0x7ff74c09b940 con 0x7ff754106f20 2024-09-10T10:05:16.778 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-10T10:05:16.778 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":47,"btime":"2024-09-10T10:04:30:774530+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14454,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/2970594311","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2970594311},{"type":"v1","addr":"172.21.15.5:6835","nonce":2970594311}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14514,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6839/3616502174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":3616502174},{"type":"v1","addr":"172.21.15.5:6839","nonce":3616502174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":47,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T09:35:29.626287+0000","modified":"2024-09-10T10:04:29.992750+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24542,"mds_1":24535,"mds_2":24511,"mds_3":24506,"mds_4":24443},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14520":{"gid":14520,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/2073599842","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":2073599842},{"type":"v1","addr":"172.21.15.5:6841","nonce":2073599842}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"k","rank":4,"incarnation":26,"state":"up:active","state_seq":24,"addr":"172.21.15.102:6833/2457007427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":2457007427},{"type":"v1","addr":"172.21.15.102:6833","nonce":2457007427}]},"join_fscid":-1,"export_targets":[3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/356445159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":356445159},{"type":"v1","addr":"172.21.15.159:6833","nonce":356445159}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6837/2894314507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":2894314507},{"type":"v1","addr":"172.21.15.5:6837","nonce":2894314507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.102:6835/1647549111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":1647549111},{"type":"v1","addr":"172.21.15.102:6835","nonce":1647549111}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6835/952657242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":952657242},{"type":"v1","addr":"172.21.15.159:6835","nonce":952657242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":19,"addr":"172.21.15.102:6837/2689776764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":2689776764},{"type":"v1","addr":"172.21.15.102:6837","nonce":2689776764}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.159:6837/620627639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":620627639},{"type":"v1","addr":"172.21.15.159:6837","nonce":620627639}]},"join_fscid":-1,"export_targets":[0,1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24535":{"gid":24535,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":18,"addr":"172.21.15.159:6839/2073739599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":2073739599},{"type":"v1","addr":"172.21.15.159:6839","nonce":2073739599}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24542":{"gid":24542,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.102:6839/335409749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6838","nonce":335409749},{"type":"v1","addr":"172.21.15.102:6839","nonce":335409749}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24542,"qdb_cluster":[24443,24506,24511,24535,24542]},"id":1}]} 2024-09-10T10:05:16.778 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 47 2024-09-10T10:05:16.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:16.778+0000 7ff758a00640 1 -- 172.21.15.5:0/4111611722 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7ff7380a2b80 msgr2=0x7ff7380a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:16.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:16.778+0000 7ff758a00640 1 --2- 172.21.15.5:0/4111611722 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7ff7380a2b80 0x7ff7380a5040 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7ff7440096c0 tx=0x7ff7440046c0 comp rx=0 tx=0).stop 2024-09-10T10:05:16.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:16.778+0000 7ff758a00640 1 -- 172.21.15.5:0/4111611722 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff754106f20 msgr2=0x7ff75419ccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:16.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:16.778+0000 7ff758a00640 1 --2- 172.21.15.5:0/4111611722 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff754106f20 0x7ff75419ccc0 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7ff74c00e940 tx=0x7ff74c00ee00 comp rx=0 tx=0).stop 2024-09-10T10:05:16.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:16.778+0000 7ff758a00640 1 -- 172.21.15.5:0/4111611722 shutdown_connections 2024-09-10T10:05:16.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:16.778+0000 7ff758a00640 1 --2- 172.21.15.5:0/4111611722 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7ff7380a2b80 0x7ff7380a5040 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:16.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:16.778+0000 7ff758a00640 1 --2- 172.21.15.5:0/4111611722 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff754106f20 0x7ff75419ccc0 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:16.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:16.778+0000 7ff758a00640 1 --2- 172.21.15.5:0/4111611722 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff7541027a0 0x7ff754198930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:16.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:16.778+0000 7ff758a00640 1 --2- 172.21.15.5:0/4111611722 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff754101df0 0x7ff7541983f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:16.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:16.778+0000 7ff758a00640 1 -- 172.21.15.5:0/4111611722 >> 172.21.15.5:0/4111611722 conn(0x7ff7540fd8b0 msgr2=0x7ff7540feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:05:16.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:16.778+0000 7ff758a00640 1 -- 172.21.15.5:0/4111611722 shutdown_connections 2024-09-10T10:05:16.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:16.778+0000 7ff758a00640 1 -- 172.21.15.5:0/4111611722 wait complete. 2024-09-10T10:05:17.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:05:16 smithi005 ceph-mon[30350]: from='client.? 172.21.15.5:0/4111611722' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:05:17.134 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:05:16 smithi102 ceph-mon[35442]: from='client.? 172.21.15.5:0/4111611722' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:05:17.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:05:16 smithi159 ceph-mon[35513]: from='client.? 172.21.15.5:0/4111611722' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:05:17.566 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T10:05:17.950 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:05:18.272 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:05:17 smithi005 ceph-mon[30350]: pgmap v1201: 129 pgs: 129 active+clean; 551 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-09-10T10:05:18.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:05:17 smithi159 ceph-mon[35513]: pgmap v1201: 129 pgs: 129 active+clean; 551 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-09-10T10:05:18.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:05:17 smithi102 ceph-mon[35442]: pgmap v1201: 129 pgs: 129 active+clean; 551 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-09-10T10:05:18.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:18.554+0000 7f7e4ca00640 1 -- 172.21.15.5:0/613056165 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7e481079c0 msgr2=0x7f7e48107da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:18.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:18.554+0000 7f7e4ca00640 1 --2- 172.21.15.5:0/613056165 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7e481079c0 0x7f7e48107da0 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f7e38009cd0 tx=0x7f7e3802a8f0 comp rx=0 tx=0).stop 2024-09-10T10:05:18.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:18.554+0000 7f7e4ca00640 1 -- 172.21.15.5:0/613056165 shutdown_connections 2024-09-10T10:05:18.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:18.554+0000 7f7e4ca00640 1 --2- 172.21.15.5:0/613056165 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7e48102360 0x7f7e4810e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:18.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:18.554+0000 7f7e4ca00640 1 --2- 172.21.15.5:0/613056165 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7e481019c0 0x7f7e48101e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:18.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:18.554+0000 7f7e4ca00640 1 --2- 172.21.15.5:0/613056165 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7e481079c0 0x7f7e48107da0 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:18.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:18.554+0000 7f7e4ca00640 1 -- 172.21.15.5:0/613056165 >> 172.21.15.5:0/613056165 conn(0x7f7e480fd740 msgr2=0x7f7e480ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:05:18.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:18.554+0000 7f7e4ca00640 1 -- 172.21.15.5:0/613056165 shutdown_connections 2024-09-10T10:05:18.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:18.554+0000 7f7e4ca00640 1 -- 172.21.15.5:0/613056165 wait complete. 2024-09-10T10:05:18.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:18.555+0000 7f7e4ca00640 1 Processor -- start 2024-09-10T10:05:18.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:18.555+0000 7f7e4ca00640 1 -- start start 2024-09-10T10:05:18.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:18.556+0000 7f7e4ca00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7e481019c0 0x7f7e4819a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:18.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:18.556+0000 7f7e4ca00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7e48102360 0x7f7e4819a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:18.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:18.556+0000 7f7e4ca00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7e481079c0 0x7f7e4819ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:18.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:18.556+0000 7f7e4ca00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f7e48111dc0 con 0x7f7e481079c0 2024-09-10T10:05:18.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:18.556+0000 7f7e4ca00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f7e48111c40 con 0x7f7e481019c0 2024-09-10T10:05:18.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:18.556+0000 7f7e4ca00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f7e48111f40 con 0x7f7e48102360 2024-09-10T10:05:18.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:18.556+0000 7f7e47200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7e481079c0 0x7f7e4819ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:18.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:18.556+0000 7f7e47200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7e481079c0 0x7f7e4819ed80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35274/0 (socket says 172.21.15.5:35274) 2024-09-10T10:05:18.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:18.556+0000 7f7e47200640 1 -- 172.21.15.5:0/2840844131 learned_addr learned my addr 172.21.15.5:0/2840844131 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T10:05:18.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:18.556+0000 7f7e3de00640 1 --2- 172.21.15.5:0/2840844131 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7e48102360 0x7f7e4819a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:18.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:18.556+0000 7f7e46800640 1 --2- 172.21.15.5:0/2840844131 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7e481019c0 0x7f7e4819a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:18.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:18.556+0000 7f7e47200640 1 -- 172.21.15.5:0/2840844131 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7e48102360 msgr2=0x7f7e4819a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:18.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:18.556+0000 7f7e47200640 1 --2- 172.21.15.5:0/2840844131 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7e48102360 0x7f7e4819a9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:18.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:18.556+0000 7f7e47200640 1 -- 172.21.15.5:0/2840844131 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7e481019c0 msgr2=0x7f7e4819a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:18.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:18.556+0000 7f7e47200640 1 --2- 172.21.15.5:0/2840844131 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7e481019c0 0x7f7e4819a4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:18.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:18.556+0000 7f7e47200640 1 -- 172.21.15.5:0/2840844131 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e4819f460 con 0x7f7e481079c0 2024-09-10T10:05:18.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:18.557+0000 7f7e47200640 1 --2- 172.21.15.5:0/2840844131 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7e481079c0 0x7f7e4819ed80 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f7e4000e970 tx=0x7f7e4000ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:18.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:18.557+0000 7f7e3fe00640 1 -- 172.21.15.5:0/2840844131 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e4000ccc0 con 0x7f7e481079c0 2024-09-10T10:05:18.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:18.557+0000 7f7e3fe00640 1 -- 172.21.15.5:0/2840844131 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7e4000ce60 con 0x7f7e481079c0 2024-09-10T10:05:18.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:18.557+0000 7f7e4ca00640 1 -- 172.21.15.5:0/2840844131 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e4819f750 con 0x7f7e481079c0 2024-09-10T10:05:18.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:18.557+0000 7f7e3fe00640 1 -- 172.21.15.5:0/2840844131 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e40010490 con 0x7f7e481079c0 2024-09-10T10:05:18.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:18.557+0000 7f7e4ca00640 1 -- 172.21.15.5:0/2840844131 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e481a7030 con 0x7f7e481079c0 2024-09-10T10:05:18.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:18.558+0000 7f7e3ca00640 1 -- 172.21.15.5:0/2840844131 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7e48101e20 con 0x7f7e481079c0 2024-09-10T10:05:18.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:18.560+0000 7f7e3fe00640 1 -- 172.21.15.5:0/2840844131 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f7e40005000 con 0x7f7e481079c0 2024-09-10T10:05:18.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:18.561+0000 7f7e3fe00640 1 --2- 172.21.15.5:0/2840844131 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f7e2c0a2b80 0x7f7e2c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:18.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:18.562+0000 7f7e3fe00640 1 -- 172.21.15.5:0/2840844131 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7f7e400cf830 con 0x7f7e481079c0 2024-09-10T10:05:18.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:18.562+0000 7f7e46800640 1 --2- 172.21.15.5:0/2840844131 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f7e2c0a2b80 0x7f7e2c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:18.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:18.562+0000 7f7e46800640 1 --2- 172.21.15.5:0/2840844131 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f7e2c0a2b80 0x7f7e2c0a5040 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f7e38004af0 tx=0x7f7e38004a40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:18.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:18.565+0000 7f7e3fe00640 1 -- 172.21.15.5:0/2840844131 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f7e4009c340 con 0x7f7e481079c0 2024-09-10T10:05:18.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:18.791+0000 7f7e3ca00640 1 -- 172.21.15.5:0/2840844131 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7e48062df0 con 0x7f7e481079c0 2024-09-10T10:05:18.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:18.792+0000 7f7e3fe00640 1 -- 172.21.15.5:0/2840844131 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) ==== 94+0+11978 (secure 0 0 0) 0x7f7e4009bc40 con 0x7f7e481079c0 2024-09-10T10:05:18.794 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-10T10:05:18.795 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":47,"btime":"2024-09-10T10:04:30:774530+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14454,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/2970594311","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2970594311},{"type":"v1","addr":"172.21.15.5:6835","nonce":2970594311}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14514,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6839/3616502174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":3616502174},{"type":"v1","addr":"172.21.15.5:6839","nonce":3616502174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":47,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T09:35:29.626287+0000","modified":"2024-09-10T10:04:29.992750+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24542,"mds_1":24535,"mds_2":24511,"mds_3":24506,"mds_4":24443},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14520":{"gid":14520,"name":"a","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/2073599842","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":2073599842},{"type":"v1","addr":"172.21.15.5:6841","nonce":2073599842}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"k","rank":4,"incarnation":26,"state":"up:active","state_seq":24,"addr":"172.21.15.102:6833/2457007427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6832","nonce":2457007427},{"type":"v1","addr":"172.21.15.102:6833","nonce":2457007427}]},"join_fscid":-1,"export_targets":[3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24460":{"gid":24460,"name":"l","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6833/356445159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6832","nonce":356445159},{"type":"v1","addr":"172.21.15.159:6833","nonce":356445159}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"d","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6837/2894314507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":2894314507},{"type":"v1","addr":"172.21.15.5:6837","nonce":2894314507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"e","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.102:6835/1647549111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6834","nonce":1647549111},{"type":"v1","addr":"172.21.15.102:6835","nonce":1647549111}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.159:6835/952657242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6834","nonce":952657242},{"type":"v1","addr":"172.21.15.159:6835","nonce":952657242}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24506":{"gid":24506,"name":"h","rank":3,"incarnation":24,"state":"up:active","state_seq":19,"addr":"172.21.15.102:6837/2689776764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6836","nonce":2689776764},{"type":"v1","addr":"172.21.15.102:6837","nonce":2689776764}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24511":{"gid":24511,"name":"c","rank":2,"incarnation":22,"state":"up:active","state_seq":19,"addr":"172.21.15.159:6837/620627639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6836","nonce":620627639},{"type":"v1","addr":"172.21.15.159:6837","nonce":620627639}]},"join_fscid":-1,"export_targets":[0,1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24535":{"gid":24535,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":18,"addr":"172.21.15.159:6839/2073739599","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6838","nonce":2073739599},{"type":"v1","addr":"172.21.15.159:6839","nonce":2073739599}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24542":{"gid":24542,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.102:6839/335409749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6838","nonce":335409749},{"type":"v1","addr":"172.21.15.102:6839","nonce":335409749}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24542,"qdb_cluster":[24443,24506,24511,24535,24542]},"id":1}]} 2024-09-10T10:05:18.795 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 47 2024-09-10T10:05:18.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:18.794+0000 7f7e3ca00640 1 -- 172.21.15.5:0/2840844131 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f7e2c0a2b80 msgr2=0x7f7e2c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:18.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:18.794+0000 7f7e3ca00640 1 --2- 172.21.15.5:0/2840844131 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f7e2c0a2b80 0x7f7e2c0a5040 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f7e38004af0 tx=0x7f7e38004a40 comp rx=0 tx=0).stop 2024-09-10T10:05:18.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:18.794+0000 7f7e3ca00640 1 -- 172.21.15.5:0/2840844131 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7e481079c0 msgr2=0x7f7e4819ed80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:18.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:18.794+0000 7f7e3ca00640 1 --2- 172.21.15.5:0/2840844131 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7e481079c0 0x7f7e4819ed80 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f7e4000e970 tx=0x7f7e4000ed70 comp rx=0 tx=0).stop 2024-09-10T10:05:18.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:18.794+0000 7f7e3ca00640 1 -- 172.21.15.5:0/2840844131 shutdown_connections 2024-09-10T10:05:18.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:18.794+0000 7f7e3ca00640 1 --2- 172.21.15.5:0/2840844131 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f7e2c0a2b80 0x7f7e2c0a5040 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:18.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:18.794+0000 7f7e3ca00640 1 --2- 172.21.15.5:0/2840844131 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7e481079c0 0x7f7e4819ed80 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:18.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:18.794+0000 7f7e3ca00640 1 --2- 172.21.15.5:0/2840844131 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f7e48102360 0x7f7e4819a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:18.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:18.794+0000 7f7e3ca00640 1 --2- 172.21.15.5:0/2840844131 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f7e481019c0 0x7f7e4819a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:18.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:18.794+0000 7f7e3ca00640 1 -- 172.21.15.5:0/2840844131 >> 172.21.15.5:0/2840844131 conn(0x7f7e480fd740 msgr2=0x7f7e480fdd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:05:18.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:18.795+0000 7f7e3ca00640 1 -- 172.21.15.5:0/2840844131 shutdown_connections 2024-09-10T10:05:18.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:18.795+0000 7f7e3ca00640 1 -- 172.21.15.5:0/2840844131 wait complete. 2024-09-10T10:05:19.195 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:05:18 smithi005 ceph-mon[30350]: from='client.? 172.21.15.5:0/2840844131' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:05:19.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:05:18 smithi159 ceph-mon[35513]: from='client.? 172.21.15.5:0/2840844131' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:05:19.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:05:18 smithi102 ceph-mon[35442]: from='client.? 172.21.15.5:0/2840844131' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T10:05:19.829 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-10T10:05:20.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:05:19 smithi005 ceph-mon[30350]: pgmap v1202: 129 pgs: 129 active+clean; 551 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 597 B/s wr, 10 op/s 2024-09-10T10:05:20.208 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:05:20.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:05:19 smithi159 ceph-mon[35513]: pgmap v1202: 129 pgs: 129 active+clean; 551 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 597 B/s wr, 10 op/s 2024-09-10T10:05:20.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:05:19 smithi102 ceph-mon[35442]: pgmap v1202: 129 pgs: 129 active+clean; 551 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 597 B/s wr, 10 op/s 2024-09-10T10:05:20.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:20.823+0000 7ff16e400640 1 -- 172.21.15.5:0/3574351295 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff168104010 msgr2=0x7ff1681043f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:20.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:20.823+0000 7ff16e400640 1 --2- 172.21.15.5:0/3574351295 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff168104010 0x7ff1681043f0 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7ff158009900 tx=0x7ff15802a3f0 comp rx=0 tx=0).stop 2024-09-10T10:05:20.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:20.823+0000 7ff16e400640 1 -- 172.21.15.5:0/3574351295 shutdown_connections 2024-09-10T10:05:20.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:20.823+0000 7ff16e400640 1 --2- 172.21.15.5:0/3574351295 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff168109140 0x7ff168110cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:20.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:20.823+0000 7ff16e400640 1 --2- 172.21.15.5:0/3574351295 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff1681049c0 0x7ff168108a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:20.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:20.823+0000 7ff16e400640 1 --2- 172.21.15.5:0/3574351295 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff168104010 0x7ff1681043f0 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:20.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:20.823+0000 7ff16e400640 1 -- 172.21.15.5:0/3574351295 >> 172.21.15.5:0/3574351295 conn(0x7ff1680ffa50 msgr2=0x7ff168101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:05:20.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:20.823+0000 7ff16e400640 1 -- 172.21.15.5:0/3574351295 shutdown_connections 2024-09-10T10:05:20.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:20.823+0000 7ff16e400640 1 -- 172.21.15.5:0/3574351295 wait complete. 2024-09-10T10:05:20.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:20.824+0000 7ff16e400640 1 Processor -- start 2024-09-10T10:05:20.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:20.824+0000 7ff16e400640 1 -- start start 2024-09-10T10:05:20.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:20.824+0000 7ff16e400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff168104010 0x7ff16819a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:20.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:20.824+0000 7ff16e400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff1681049c0 0x7ff16819ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:20.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:20.824+0000 7ff16e400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff168109140 0x7ff16819eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:20.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:20.824+0000 7ff16e400640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff168114190 con 0x7ff168104010 2024-09-10T10:05:20.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:20.824+0000 7ff16e400640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7ff168114010 con 0x7ff1681049c0 2024-09-10T10:05:20.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:20.824+0000 7ff16e400640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7ff168114310 con 0x7ff168109140 2024-09-10T10:05:20.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:20.825+0000 7ff167400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff168104010 0x7ff16819a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:20.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:20.825+0000 7ff167400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff168104010 0x7ff16819a600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56882/0 (socket says 172.21.15.5:56882) 2024-09-10T10:05:20.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:20.825+0000 7ff15fe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff1681049c0 0x7ff16819ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:20.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:20.825+0000 7ff167400640 1 -- 172.21.15.5:0/1256386847 learned_addr learned my addr 172.21.15.5:0/1256386847 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T10:05:20.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:20.825+0000 7ff167e00640 1 --2- 172.21.15.5:0/1256386847 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff168109140 0x7ff16819eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:20.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:20.825+0000 7ff167400640 1 -- 172.21.15.5:0/1256386847 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff168109140 msgr2=0x7ff16819eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:20.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:20.825+0000 7ff167400640 1 --2- 172.21.15.5:0/1256386847 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff168109140 0x7ff16819eed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:20.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:20.825+0000 7ff167400640 1 -- 172.21.15.5:0/1256386847 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff1681049c0 msgr2=0x7ff16819ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:20.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:20.825+0000 7ff167400640 1 --2- 172.21.15.5:0/1256386847 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff1681049c0 0x7ff16819ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:20.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:20.825+0000 7ff167400640 1 -- 172.21.15.5:0/1256386847 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff16819f5b0 con 0x7ff168104010 2024-09-10T10:05:20.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:20.825+0000 7ff15fe00640 1 --2- 172.21.15.5:0/1256386847 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff1681049c0 0x7ff16819ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:05:20.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:20.826+0000 7ff167400640 1 --2- 172.21.15.5:0/1256386847 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff168104010 0x7ff16819a600 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7ff158004770 tx=0x7ff158003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:20.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:20.826+0000 7ff164c00640 1 -- 172.21.15.5:0/1256386847 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff15803f070 con 0x7ff168104010 2024-09-10T10:05:20.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:20.826+0000 7ff164c00640 1 -- 172.21.15.5:0/1256386847 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff158031ca0 con 0x7ff168104010 2024-09-10T10:05:20.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:20.826+0000 7ff164c00640 1 -- 172.21.15.5:0/1256386847 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff158035030 con 0x7ff168104010 2024-09-10T10:05:20.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:20.826+0000 7ff16e400640 1 -- 172.21.15.5:0/1256386847 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff16819f840 con 0x7ff168104010 2024-09-10T10:05:20.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:20.826+0000 7ff16e400640 1 -- 172.21.15.5:0/1256386847 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1681a7180 con 0x7ff168104010 2024-09-10T10:05:20.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:20.827+0000 7ff16e400640 1 -- 172.21.15.5:0/1256386847 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff1681043f0 con 0x7ff168104010 2024-09-10T10:05:20.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:20.829+0000 7ff164c00640 1 -- 172.21.15.5:0/1256386847 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7ff158042070 con 0x7ff168104010 2024-09-10T10:05:20.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:20.831+0000 7ff164c00640 1 --2- 172.21.15.5:0/1256386847 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7ff14c0a2c00 0x7ff14c0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:20.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:20.831+0000 7ff164c00640 1 -- 172.21.15.5:0/1256386847 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7ff1580efb50 con 0x7ff168104010 2024-09-10T10:05:20.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:20.832+0000 7ff15fe00640 1 --2- 172.21.15.5:0/1256386847 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7ff14c0a2c00 0x7ff14c0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:20.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:20.832+0000 7ff15fe00640 1 --2- 172.21.15.5:0/1256386847 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7ff14c0a2c00 0x7ff14c0a50c0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7ff1680ffe70 tx=0x7ff15000b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:20.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:20.836+0000 7ff164c00640 1 -- 172.21.15.5:0/1256386847 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff1580bc010 con 0x7ff168104010 2024-09-10T10:05:21.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:21.022+0000 7ff16e400640 1 -- 172.21.15.5:0/1256386847 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff16810c320 con 0x7ff168104010 2024-09-10T10:05:21.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:21.024+0000 7ff164c00640 1 -- 172.21.15.5:0/1256386847 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v96) ==== 74+0+24846 (secure 0 0 0) 0x7ff1580c4e20 con 0x7ff168104010 2024-09-10T10:05:21.027 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-10T10:05:21.027 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":96,"fsid":"bade4b50-6f56-11ef-bcea-c7b262605968","created":"2024-09-10T09:27:13.298498+0000","modified":"2024-09-10T09:37:07.520520+0000","last_up_change":"2024-09-10T09:34:30.730066+0000","last_in_change":"2024-09-10T09:34:07.340995+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T09:30:26.042255+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T09:35:24.461031+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T09:35:27.155349+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-10T09:35:35.121462+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"085749a8-e882-4377-92dd-34bb266c0f0b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":954535266},{"type":"v1","addr":"172.21.15.5:6803","nonce":954535266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":954535266},{"type":"v1","addr":"172.21.15.5:6805","nonce":954535266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":954535266},{"type":"v1","addr":"172.21.15.5:6809","nonce":954535266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":954535266},{"type":"v1","addr":"172.21.15.5:6807","nonce":954535266}]},"public_addr":"172.21.15.5:6803/954535266","cluster_addr":"172.21.15.5:6805/954535266","heartbeat_back_addr":"172.21.15.5:6809/954535266","heartbeat_front_addr":"172.21.15.5:6807/954535266","state":["exists","up"]},{"osd":1,"uuid":"0a76c795-2573-4a92-969a-2ec5dd65c094","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6800","nonce":733924375},{"type":"v1","addr":"172.21.15.102:6801","nonce":733924375}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6802","nonce":733924375},{"type":"v1","addr":"172.21.15.102:6803","nonce":733924375}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6806","nonce":733924375},{"type":"v1","addr":"172.21.15.102:6807","nonce":733924375}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6804","nonce":733924375},{"type":"v1","addr":"172.21.15.102:6805","nonce":733924375}]},"public_addr":"172.21.15.102:6801/733924375","cluster_addr":"172.21.15.102:6803/733924375","heartbeat_back_addr":"172.21.15.102:6807/733924375","heartbeat_front_addr":"172.21.15.102:6805/733924375","state":["exists","up"]},{"osd":2,"uuid":"0c98fcb8-60ab-4473-8995-1d9e97c577e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6800","nonce":2236204189},{"type":"v1","addr":"172.21.15.159:6801","nonce":2236204189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":2236204189},{"type":"v1","addr":"172.21.15.159:6803","nonce":2236204189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":2236204189},{"type":"v1","addr":"172.21.15.159:6807","nonce":2236204189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":2236204189},{"type":"v1","addr":"172.21.15.159:6805","nonce":2236204189}]},"public_addr":"172.21.15.159:6801/2236204189","cluster_addr":"172.21.15.159:6803/2236204189","heartbeat_back_addr":"172.21.15.159:6807/2236204189","heartbeat_front_addr":"172.21.15.159:6805/2236204189","state":["exists","up"]},{"osd":3,"uuid":"88eecdeb-5cee-4f78-a3f9-a66bf4bf772a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":2221468725},{"type":"v1","addr":"172.21.15.5:6811","nonce":2221468725}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":2221468725},{"type":"v1","addr":"172.21.15.5:6813","nonce":2221468725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6816","nonce":2221468725},{"type":"v1","addr":"172.21.15.5:6817","nonce":2221468725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":2221468725},{"type":"v1","addr":"172.21.15.5:6815","nonce":2221468725}]},"public_addr":"172.21.15.5:6811/2221468725","cluster_addr":"172.21.15.5:6813/2221468725","heartbeat_back_addr":"172.21.15.5:6817/2221468725","heartbeat_front_addr":"172.21.15.5:6815/2221468725","state":["exists","up"]},{"osd":4,"uuid":"f2e7314d-7938-4668-9011-45c175f66b48","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6808","nonce":594465753},{"type":"v1","addr":"172.21.15.102:6809","nonce":594465753}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6810","nonce":594465753},{"type":"v1","addr":"172.21.15.102:6811","nonce":594465753}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6814","nonce":594465753},{"type":"v1","addr":"172.21.15.102:6815","nonce":594465753}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6812","nonce":594465753},{"type":"v1","addr":"172.21.15.102:6813","nonce":594465753}]},"public_addr":"172.21.15.102:6809/594465753","cluster_addr":"172.21.15.102:6811/594465753","heartbeat_back_addr":"172.21.15.102:6815/594465753","heartbeat_front_addr":"172.21.15.102:6813/594465753","state":["exists","up"]},{"osd":5,"uuid":"0c0366d3-658b-4bf0-ba17-aff28a02e948","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6808","nonce":3230280402},{"type":"v1","addr":"172.21.15.159:6809","nonce":3230280402}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6810","nonce":3230280402},{"type":"v1","addr":"172.21.15.159:6811","nonce":3230280402}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6814","nonce":3230280402},{"type":"v1","addr":"172.21.15.159:6815","nonce":3230280402}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6812","nonce":3230280402},{"type":"v1","addr":"172.21.15.159:6813","nonce":3230280402}]},"public_addr":"172.21.15.159:6809/3230280402","cluster_addr":"172.21.15.159:6811/3230280402","heartbeat_back_addr":"172.21.15.159:6815/3230280402","heartbeat_front_addr":"172.21.15.159:6813/3230280402","state":["exists","up"]},{"osd":6,"uuid":"682670a6-51d5-46e8-af6e-dc2d8c1cf407","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":1907651140},{"type":"v1","addr":"172.21.15.5:6819","nonce":1907651140}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":1907651140},{"type":"v1","addr":"172.21.15.5:6821","nonce":1907651140}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":1907651140},{"type":"v1","addr":"172.21.15.5:6825","nonce":1907651140}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":1907651140},{"type":"v1","addr":"172.21.15.5:6823","nonce":1907651140}]},"public_addr":"172.21.15.5:6819/1907651140","cluster_addr":"172.21.15.5:6821/1907651140","heartbeat_back_addr":"172.21.15.5:6825/1907651140","heartbeat_front_addr":"172.21.15.5:6823/1907651140","state":["exists","up"]},{"osd":7,"uuid":"8ae0f4a9-99e7-4563-aa2a-15169684a8b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6816","nonce":692040360},{"type":"v1","addr":"172.21.15.102:6817","nonce":692040360}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6818","nonce":692040360},{"type":"v1","addr":"172.21.15.102:6819","nonce":692040360}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6822","nonce":692040360},{"type":"v1","addr":"172.21.15.102:6823","nonce":692040360}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6820","nonce":692040360},{"type":"v1","addr":"172.21.15.102:6821","nonce":692040360}]},"public_addr":"172.21.15.102:6817/692040360","cluster_addr":"172.21.15.102:6819/692040360","heartbeat_back_addr":"172.21.15.102:6823/692040360","heartbeat_front_addr":"172.21.15.102:6821/692040360","state":["exists","up"]},{"osd":8,"uuid":"e37c845e-ced9-4413-9bbe-bc090537c2bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6816","nonce":2107291799},{"type":"v1","addr":"172.21.15.159:6817","nonce":2107291799}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6818","nonce":2107291799},{"type":"v1","addr":"172.21.15.159:6819","nonce":2107291799}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6822","nonce":2107291799},{"type":"v1","addr":"172.21.15.159:6823","nonce":2107291799}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6820","nonce":2107291799},{"type":"v1","addr":"172.21.15.159:6821","nonce":2107291799}]},"public_addr":"172.21.15.159:6817/2107291799","cluster_addr":"172.21.15.159:6819/2107291799","heartbeat_back_addr":"172.21.15.159:6823/2107291799","heartbeat_front_addr":"172.21.15.159:6821/2107291799","state":["exists","up"]},{"osd":9,"uuid":"91a5b6cc-08da-47be-af12-d0896a849309","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":902431281},{"type":"v1","addr":"172.21.15.5:6827","nonce":902431281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":902431281},{"type":"v1","addr":"172.21.15.5:6829","nonce":902431281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":902431281},{"type":"v1","addr":"172.21.15.5:6833","nonce":902431281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":902431281},{"type":"v1","addr":"172.21.15.5:6831","nonce":902431281}]},"public_addr":"172.21.15.5:6827/902431281","cluster_addr":"172.21.15.5:6829/902431281","heartbeat_back_addr":"172.21.15.5:6833/902431281","heartbeat_front_addr":"172.21.15.5:6831/902431281","state":["exists","up"]},{"osd":10,"uuid":"2211030c-b00e-402b-b4d7-c587eb29a490","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6824","nonce":2853806211},{"type":"v1","addr":"172.21.15.102:6825","nonce":2853806211}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6826","nonce":2853806211},{"type":"v1","addr":"172.21.15.102:6827","nonce":2853806211}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6830","nonce":2853806211},{"type":"v1","addr":"172.21.15.102:6831","nonce":2853806211}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6828","nonce":2853806211},{"type":"v1","addr":"172.21.15.102:6829","nonce":2853806211}]},"public_addr":"172.21.15.102:6825/2853806211","cluster_addr":"172.21.15.102:6827/2853806211","heartbeat_back_addr":"172.21.15.102:6831/2853806211","heartbeat_front_addr":"172.21.15.102:6829/2853806211","state":["exists","up"]},{"osd":11,"uuid":"4e9699ec-8ff2-4005-9d84-af5600b3a2ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":77,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6824","nonce":969941218},{"type":"v1","addr":"172.21.15.159:6825","nonce":969941218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6826","nonce":969941218},{"type":"v1","addr":"172.21.15.159:6827","nonce":969941218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6830","nonce":969941218},{"type":"v1","addr":"172.21.15.159:6831","nonce":969941218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6828","nonce":969941218},{"type":"v1","addr":"172.21.15.159:6829","nonce":969941218}]},"public_addr":"172.21.15.159:6825/969941218","cluster_addr":"172.21.15.159:6827/969941218","heartbeat_back_addr":"172.21.15.159:6831/969941218","heartbeat_front_addr":"172.21.15.159:6829/969941218","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:29:34.416490+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:30:00.246101+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:30:24.300464+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:30:54.663623+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:31:21.426292+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:31:46.738914+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:32:16.196571+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:32:42.549243+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:33:08.178651+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:33:37.477911+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:34:03.114026+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T09:34:28.891924+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4}]},{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"3.0","mappings":[{"from":4,"to":8}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":6}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9},{"from":5,"to":7}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]},{"pgid":"3.39","mappings":[{"from":5,"to":6}]},{"pgid":"4.8","mappings":[{"from":5,"to":8}]},{"pgid":"4.c","mappings":[{"from":9,"to":10}]},{"pgid":"4.18","mappings":[{"from":9,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.5:6801/1527422686":"2024-09-11T09:28:05.436742+0000","172.21.15.5:0/2276144345":"2024-09-11T09:28:05.436742+0000","172.21.15.5:0/412040104":"2024-09-11T09:28:05.436742+0000","172.21.15.5:6800/1527422686":"2024-09-11T09:28:05.436742+0000","172.21.15.5:0/1856581170":"2024-09-11T09:28:05.436742+0000","172.21.15.5:0/2847668298":"2024-09-11T09:27:37.778500+0000","172.21.15.5:6801/14425913":"2024-09-11T09:27:37.778500+0000","172.21.15.5:0/166910782":"2024-09-11T09:27:37.778500+0000","172.21.15.5:6800/14425913":"2024-09-11T09:27:37.778500+0000","172.21.15.5:0/1711858179":"2024-09-11T09:27:37.778500+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T10:05:21.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:21.028+0000 7ff16e400640 1 -- 172.21.15.5:0/1256386847 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7ff14c0a2c00 msgr2=0x7ff14c0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:21.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:21.028+0000 7ff16e400640 1 --2- 172.21.15.5:0/1256386847 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7ff14c0a2c00 0x7ff14c0a50c0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7ff1680ffe70 tx=0x7ff15000b410 comp rx=0 tx=0).stop 2024-09-10T10:05:21.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:21.028+0000 7ff16e400640 1 -- 172.21.15.5:0/1256386847 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff168104010 msgr2=0x7ff16819a600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:21.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:21.028+0000 7ff16e400640 1 --2- 172.21.15.5:0/1256386847 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff168104010 0x7ff16819a600 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7ff158004770 tx=0x7ff158003c10 comp rx=0 tx=0).stop 2024-09-10T10:05:21.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:21.028+0000 7ff16e400640 1 -- 172.21.15.5:0/1256386847 shutdown_connections 2024-09-10T10:05:21.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:21.028+0000 7ff16e400640 1 --2- 172.21.15.5:0/1256386847 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7ff14c0a2c00 0x7ff14c0a50c0 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:21.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:21.028+0000 7ff16e400640 1 --2- 172.21.15.5:0/1256386847 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff168109140 0x7ff16819eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:21.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:21.028+0000 7ff16e400640 1 --2- 172.21.15.5:0/1256386847 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff1681049c0 0x7ff16819ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:21.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:21.028+0000 7ff16e400640 1 --2- 172.21.15.5:0/1256386847 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff168104010 0x7ff16819a600 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:21.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:21.028+0000 7ff16e400640 1 -- 172.21.15.5:0/1256386847 >> 172.21.15.5:0/1256386847 conn(0x7ff1680ffa50 msgr2=0x7ff168100d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:05:21.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:21.028+0000 7ff16e400640 1 -- 172.21.15.5:0/1256386847 shutdown_connections 2024-09-10T10:05:21.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:21.028+0000 7ff16e400640 1 -- 172.21.15.5:0/1256386847 wait complete. 2024-09-10T10:05:21.782 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph fs fail cephfs 2024-09-10T10:05:22.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:05:21 smithi005 ceph-mon[30350]: pgmap v1203: 129 pgs: 129 active+clean; 551 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 1.2 KiB/s wr, 14 op/s 2024-09-10T10:05:22.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:05:21 smithi005 ceph-mon[30350]: from='client.? 172.21.15.5:0/1256386847' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T10:05:22.162 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:05:22.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:05:21 smithi159 ceph-mon[35513]: pgmap v1203: 129 pgs: 129 active+clean; 551 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 1.2 KiB/s wr, 14 op/s 2024-09-10T10:05:22.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:05:21 smithi159 ceph-mon[35513]: from='client.? 172.21.15.5:0/1256386847' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T10:05:22.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:05:21 smithi102 ceph-mon[35442]: pgmap v1203: 129 pgs: 129 active+clean; 551 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 1.2 KiB/s wr, 14 op/s 2024-09-10T10:05:22.385 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:05:21 smithi102 ceph-mon[35442]: from='client.? 172.21.15.5:0/1256386847' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T10:05:22.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:22.788+0000 7fbcdfc00640 1 -- 172.21.15.5:0/608893125 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbcd8109120 msgr2=0x7fbcd8110ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:22.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:22.788+0000 7fbcdfc00640 1 --2- 172.21.15.5:0/608893125 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbcd8109120 0x7fbcd8110ca0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7fbcd0009ca0 tx=0x7fbcd002a8c0 comp rx=0 tx=0).stop 2024-09-10T10:05:22.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:22.788+0000 7fbcdfc00640 1 -- 172.21.15.5:0/608893125 shutdown_connections 2024-09-10T10:05:22.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:22.788+0000 7fbcdfc00640 1 --2- 172.21.15.5:0/608893125 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbcd8109120 0x7fbcd8110ca0 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:22.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:22.788+0000 7fbcdfc00640 1 --2- 172.21.15.5:0/608893125 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbcd81049a0 0x7fbcd81089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:22.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:22.788+0000 7fbcdfc00640 1 --2- 172.21.15.5:0/608893125 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbcd8103ff0 0x7fbcd81043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:22.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:22.788+0000 7fbcdfc00640 1 -- 172.21.15.5:0/608893125 >> 172.21.15.5:0/608893125 conn(0x7fbcd80ffa90 msgr2=0x7fbcd8101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:05:22.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:22.788+0000 7fbcdfc00640 1 -- 172.21.15.5:0/608893125 shutdown_connections 2024-09-10T10:05:22.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:22.788+0000 7fbcdfc00640 1 -- 172.21.15.5:0/608893125 wait complete. 2024-09-10T10:05:22.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:22.789+0000 7fbcdfc00640 1 Processor -- start 2024-09-10T10:05:22.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:22.789+0000 7fbcdfc00640 1 -- start start 2024-09-10T10:05:22.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:22.789+0000 7fbcdfc00640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbcd8103ff0 0x7fbcd819a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:22.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:22.789+0000 7fbcdfc00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbcd81049a0 0x7fbcd819ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:22.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:22.789+0000 7fbcdfc00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbcd8109120 0x7fbcd819ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:22.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:22.789+0000 7fbcdfc00640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fbcd8114150 con 0x7fbcd8109120 2024-09-10T10:05:22.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:22.789+0000 7fbcdfc00640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7fbcd8113fd0 con 0x7fbcd81049a0 2024-09-10T10:05:22.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:22.789+0000 7fbcdfc00640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7fbcd81142d0 con 0x7fbcd8103ff0 2024-09-10T10:05:22.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:22.790+0000 7fbcddc00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbcd8109120 0x7fbcd819ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:22.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:22.790+0000 7fbccfe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbcd81049a0 0x7fbcd819ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:22.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:22.790+0000 7fbcddc00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbcd8109120 0x7fbcd819ef00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56892/0 (socket says 172.21.15.5:56892) 2024-09-10T10:05:22.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:22.790+0000 7fbccfe00640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbcd81049a0 0x7fbcd819ab70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.102:3300/0 says I am v2:172.21.15.5:39302/0 (socket says 172.21.15.5:39302) 2024-09-10T10:05:22.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:22.790+0000 7fbcddc00640 1 -- 172.21.15.5:0/686101987 learned_addr learned my addr 172.21.15.5:0/686101987 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T10:05:22.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:22.790+0000 7fbcdd200640 1 --2- 172.21.15.5:0/686101987 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbcd8103ff0 0x7fbcd819a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:22.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:22.790+0000 7fbccfe00640 1 -- 172.21.15.5:0/686101987 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbcd8103ff0 msgr2=0x7fbcd819a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:22.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:22.790+0000 7fbccfe00640 1 --2- 172.21.15.5:0/686101987 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbcd8103ff0 0x7fbcd819a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:22.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:22.790+0000 7fbccfe00640 1 -- 172.21.15.5:0/686101987 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbcd8109120 msgr2=0x7fbcd819ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:22.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:22.790+0000 7fbccfe00640 1 --2- 172.21.15.5:0/686101987 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbcd8109120 0x7fbcd819ef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:22.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:22.790+0000 7fbccfe00640 1 -- 172.21.15.5:0/686101987 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbcd819f5e0 con 0x7fbcd81049a0 2024-09-10T10:05:22.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:22.791+0000 7fbccfe00640 1 --2- 172.21.15.5:0/686101987 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbcd81049a0 0x7fbcd819ab70 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7fbcc000e9f0 tx=0x7fbcc000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:22.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:22.791+0000 7fbcd6000640 1 -- 172.21.15.5:0/686101987 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcc000ccc0 con 0x7fbcd81049a0 2024-09-10T10:05:22.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:22.791+0000 7fbcd6000640 1 -- 172.21.15.5:0/686101987 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbcc0010910 con 0x7fbcd81049a0 2024-09-10T10:05:22.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:22.791+0000 7fbcdfc00640 1 -- 172.21.15.5:0/686101987 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbcd819f8d0 con 0x7fbcd81049a0 2024-09-10T10:05:22.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:22.791+0000 7fbcd6000640 1 -- 172.21.15.5:0/686101987 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcc0018960 con 0x7fbcd81049a0 2024-09-10T10:05:22.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:22.791+0000 7fbcdfc00640 1 -- 172.21.15.5:0/686101987 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbcd81a71b0 con 0x7fbcd81049a0 2024-09-10T10:05:22.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:22.792+0000 7fbcdfc00640 1 -- 172.21.15.5:0/686101987 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbcd81043d0 con 0x7fbcd81049a0 2024-09-10T10:05:22.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:22.794+0000 7fbcd6000640 1 -- 172.21.15.5:0/686101987 <== mon.2 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7fbcc0010ac0 con 0x7fbcd81049a0 2024-09-10T10:05:22.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:22.795+0000 7fbcd6000640 1 --2- 172.21.15.5:0/686101987 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fbcbc0a2c00 0x7fbcbc0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:22.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:22.795+0000 7fbcd6000640 1 -- 172.21.15.5:0/686101987 <== mon.2 v2:172.21.15.102:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9245+0+0 (secure 0 0 0) 0x7fbcc00cfba0 con 0x7fbcd81049a0 2024-09-10T10:05:22.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:22.796+0000 7fbcdd200640 1 --2- 172.21.15.5:0/686101987 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fbcbc0a2c00 0x7fbcbc0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:22.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:22.796+0000 7fbcdd200640 1 --2- 172.21.15.5:0/686101987 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fbcbc0a2c00 0x7fbcbc0a50c0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fbcc8009bd0 tx=0x7fbcc8005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:22.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:22.797+0000 7fbcd6000640 1 -- 172.21.15.5:0/686101987 <== mon.2 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fbcc009c060 con 0x7fbcd81049a0 2024-09-10T10:05:23.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:23.021+0000 7fbcdfc00640 1 -- 172.21.15.5:0/686101987 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fbcd802d7c0 con 0x7fbcd81049a0 2024-09-10T10:05:23.032 INFO:teuthology.orchestra.run.smithi005.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-10T10:05:23.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:23.029+0000 7fbcd6000640 1 -- 172.21.15.5:0/686101987 <== mon.2 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v48) ==== 161+0+0 (secure 0 0 0) 0x7fbcc00a0360 con 0x7fbcd81049a0 2024-09-10T10:05:23.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:23.033+0000 7fbcdfc00640 1 -- 172.21.15.5:0/686101987 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fbcbc0a2c00 msgr2=0x7fbcbc0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:23.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:23.033+0000 7fbcdfc00640 1 --2- 172.21.15.5:0/686101987 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fbcbc0a2c00 0x7fbcbc0a50c0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fbcc8009bd0 tx=0x7fbcc8005f90 comp rx=0 tx=0).stop 2024-09-10T10:05:23.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:23.033+0000 7fbcdfc00640 1 -- 172.21.15.5:0/686101987 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbcd81049a0 msgr2=0x7fbcd819ab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:23.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:23.033+0000 7fbcdfc00640 1 --2- 172.21.15.5:0/686101987 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbcd81049a0 0x7fbcd819ab70 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7fbcc000e9f0 tx=0x7fbcc000eeb0 comp rx=0 tx=0).stop 2024-09-10T10:05:23.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:23.033+0000 7fbcdfc00640 1 -- 172.21.15.5:0/686101987 shutdown_connections 2024-09-10T10:05:23.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:23.033+0000 7fbcdfc00640 1 --2- 172.21.15.5:0/686101987 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7fbcbc0a2c00 0x7fbcbc0a50c0 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:23.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:23.033+0000 7fbcdfc00640 1 --2- 172.21.15.5:0/686101987 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbcd8109120 0x7fbcd819ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:23.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:23.033+0000 7fbcdfc00640 1 --2- 172.21.15.5:0/686101987 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7fbcd81049a0 0x7fbcd819ab70 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:23.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:23.033+0000 7fbcdfc00640 1 --2- 172.21.15.5:0/686101987 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7fbcd8103ff0 0x7fbcd819a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:23.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:23.033+0000 7fbcdfc00640 1 -- 172.21.15.5:0/686101987 >> 172.21.15.5:0/686101987 conn(0x7fbcd80ffa90 msgr2=0x7fbcd8100d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:05:23.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:23.033+0000 7fbcdfc00640 1 -- 172.21.15.5:0/686101987 shutdown_connections 2024-09-10T10:05:23.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:23.033+0000 7fbcdfc00640 1 -- 172.21.15.5:0/686101987 wait complete. 2024-09-10T10:05:23.306 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:05:23 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mon-a[30346]: 2024-09-10T10:05:23.022+0000 7f1336600640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-10T10:05:23.307 INFO:journalctl@ceph.mds.d.smithi005.stdout:Sep 10 10:05:23 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mds-d[71009]: ignoring --setuser ceph since I am not root 2024-09-10T10:05:23.307 INFO:journalctl@ceph.mds.d.smithi005.stdout:Sep 10 10:05:23 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mds-d[71009]: ignoring --setgroup ceph since I am not root 2024-09-10T10:05:23.307 INFO:journalctl@ceph.mds.d.smithi005.stdout:Sep 10 10:05:23 smithi005 ceph-mds[71013]: -- 172.21.15.5:0/2707381042 <== mon.2 v2:172.21.15.102:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55aaf614bc20 con 0x55aaf6128800 2024-09-10T10:05:23.307 INFO:journalctl@ceph.mds.d.smithi005.stdout:Sep 10 10:05:23 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mds-d[71009]: starting mds.d at 2024-09-10T10:05:23.307 INFO:journalctl@ceph.mds.a.smithi005.stdout:Sep 10 10:05:23 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mds-a[72209]: ignoring --setuser ceph since I am not root 2024-09-10T10:05:23.307 INFO:journalctl@ceph.mds.a.smithi005.stdout:Sep 10 10:05:23 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mds-a[72209]: ignoring --setgroup ceph since I am not root 2024-09-10T10:05:23.307 INFO:journalctl@ceph.mds.a.smithi005.stdout:Sep 10 10:05:23 smithi005 ceph-mds[72224]: -- 172.21.15.5:0/3443947740 <== mon.2 v2:172.21.15.102:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x560420ad9c20 con 0x560420ab6800 2024-09-10T10:05:23.307 INFO:journalctl@ceph.mds.a.smithi005.stdout:Sep 10 10:05:23 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mds-a[72209]: starting mds.a at 2024-09-10T10:05:23.384 INFO:journalctl@ceph.mds.e.smithi102.stdout:Sep 10 10:05:23 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mds-e[64592]: ignoring --setuser ceph since I am not root 2024-09-10T10:05:23.385 INFO:journalctl@ceph.mds.e.smithi102.stdout:Sep 10 10:05:23 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mds-e[64592]: ignoring --setgroup ceph since I am not root 2024-09-10T10:05:23.385 INFO:journalctl@ceph.mds.e.smithi102.stdout:Sep 10 10:05:23 smithi102 ceph-mds[64596]: -- 172.21.15.102:0/1528487775 <== mon.2 v2:172.21.15.102:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5613c5403860 con 0x5613c53e1000 2024-09-10T10:05:23.385 INFO:journalctl@ceph.mds.e.smithi102.stdout:Sep 10 10:05:23 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mds-e[64592]: starting mds.e at 2024-09-10T10:05:23.385 INFO:journalctl@ceph.mds.h.smithi102.stdout:Sep 10 10:05:23 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mds-h[64895]: ignoring --setuser ceph since I am not root 2024-09-10T10:05:23.385 INFO:journalctl@ceph.mds.h.smithi102.stdout:Sep 10 10:05:23 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mds-h[64895]: ignoring --setgroup ceph since I am not root 2024-09-10T10:05:23.385 INFO:journalctl@ceph.mds.h.smithi102.stdout:Sep 10 10:05:23 smithi102 ceph-mds[64899]: -- 172.21.15.102:0/3421493368 <== mon.2 v2:172.21.15.102:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55a81e90b860 con 0x55a81e8e8800 2024-09-10T10:05:23.385 INFO:journalctl@ceph.mds.h.smithi102.stdout:Sep 10 10:05:23 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mds-h[64895]: starting mds.h at 2024-09-10T10:05:23.386 INFO:journalctl@ceph.mds.b.smithi102.stdout:Sep 10 10:05:23 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mds-b[65198]: ignoring --setuser ceph since I am not root 2024-09-10T10:05:23.386 INFO:journalctl@ceph.mds.b.smithi102.stdout:Sep 10 10:05:23 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mds-b[65198]: ignoring --setgroup ceph since I am not root 2024-09-10T10:05:23.386 INFO:journalctl@ceph.mds.b.smithi102.stdout:Sep 10 10:05:23 smithi102 ceph-mds[65202]: -- 172.21.15.102:0/3417680246 <== mon.2 v2:172.21.15.102:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55ebbd521860 con 0x55ebbd4ff000 2024-09-10T10:05:23.386 INFO:journalctl@ceph.mds.b.smithi102.stdout:Sep 10 10:05:23 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mds-b[65198]: starting mds.b at 2024-09-10T10:05:23.386 INFO:journalctl@ceph.mds.k.smithi102.stdout:Sep 10 10:05:23 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mds-k[64279]: ignoring --setuser ceph since I am not root 2024-09-10T10:05:23.386 INFO:journalctl@ceph.mds.k.smithi102.stdout:Sep 10 10:05:23 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mds-k[64279]: ignoring --setgroup ceph since I am not root 2024-09-10T10:05:23.386 INFO:journalctl@ceph.mds.k.smithi102.stdout:Sep 10 10:05:23 smithi102 ceph-mds[64283]: -- 172.21.15.102:0/3707873981 <== mon.2 v2:172.21.15.102:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x558063fc1c20 con 0x558063f9f400 2024-09-10T10:05:23.386 INFO:journalctl@ceph.mds.k.smithi102.stdout:Sep 10 10:05:23 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mds-k[64279]: starting mds.k at 2024-09-10T10:05:23.601 INFO:journalctl@ceph.mds.f.smithi159.stdout:Sep 10 10:05:23 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mds-f[64973]: ignoring --setuser ceph since I am not root 2024-09-10T10:05:23.601 INFO:journalctl@ceph.mds.f.smithi159.stdout:Sep 10 10:05:23 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mds-f[64973]: ignoring --setgroup ceph since I am not root 2024-09-10T10:05:23.601 INFO:journalctl@ceph.mds.f.smithi159.stdout:Sep 10 10:05:23 smithi159 ceph-mds[64977]: -- 172.21.15.159:0/2311030973 <== mon.1 v2:172.21.15.159:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x560535631860 con 0x56053560f400 2024-09-10T10:05:23.601 INFO:journalctl@ceph.mds.f.smithi159.stdout:Sep 10 10:05:23 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mds-f[64973]: starting mds.f at 2024-09-10T10:05:23.601 INFO:journalctl@ceph.mds.l.smithi159.stdout:Sep 10 10:05:23 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mds-l[64670]: ignoring --setuser ceph since I am not root 2024-09-10T10:05:23.601 INFO:journalctl@ceph.mds.l.smithi159.stdout:Sep 10 10:05:23 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mds-l[64670]: ignoring --setgroup ceph since I am not root 2024-09-10T10:05:23.601 INFO:journalctl@ceph.mds.l.smithi159.stdout:Sep 10 10:05:23 smithi159 ceph-mds[64674]: -- 172.21.15.159:0/3633555401 <== mon.1 v2:172.21.15.159:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x562645486680 con 0x56264545a800 2024-09-10T10:05:23.601 INFO:journalctl@ceph.mds.l.smithi159.stdout:Sep 10 10:05:23 smithi159 ceph-mds[64674]: -- 172.21.15.159:0/3633555401 <== mon.1 v2:172.21.15.159:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56264547d860 con 0x56264545a800 2024-09-10T10:05:23.601 INFO:journalctl@ceph.mds.l.smithi159.stdout:Sep 10 10:05:23 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mds-l[64670]: starting mds.l at 2024-09-10T10:05:23.602 INFO:journalctl@ceph.mds.c.smithi159.stdout:Sep 10 10:05:23 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mds-c[65276]: ignoring --setuser ceph since I am not root 2024-09-10T10:05:23.602 INFO:journalctl@ceph.mds.c.smithi159.stdout:Sep 10 10:05:23 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mds-c[65276]: ignoring --setgroup ceph since I am not root 2024-09-10T10:05:23.602 INFO:journalctl@ceph.mds.c.smithi159.stdout:Sep 10 10:05:23 smithi159 ceph-mds[65280]: -- 172.21.15.159:0/1613700708 <== mon.1 v2:172.21.15.159:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55e7092f1c20 con 0x55e7092cf400 2024-09-10T10:05:23.602 INFO:journalctl@ceph.mds.c.smithi159.stdout:Sep 10 10:05:23 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mds-c[65276]: starting mds.c at 2024-09-10T10:05:23.602 INFO:journalctl@ceph.mds.i.smithi159.stdout:Sep 10 10:05:23 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mds-i[65577]: ignoring --setuser ceph since I am not root 2024-09-10T10:05:23.602 INFO:journalctl@ceph.mds.i.smithi159.stdout:Sep 10 10:05:23 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mds-i[65577]: ignoring --setgroup ceph since I am not root 2024-09-10T10:05:23.602 INFO:journalctl@ceph.mds.i.smithi159.stdout:Sep 10 10:05:23 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mds-i[65577]: starting mds.i at 2024-09-10T10:05:23.846 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph fs rm cephfs --yes-i-really-mean-it 2024-09-10T10:05:24.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:05:23 smithi005 ceph-mon[30350]: pgmap v1204: 129 pgs: 129 active+clean; 551 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2024-09-10T10:05:24.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:05:23 smithi005 ceph-mon[30350]: from='client.? 172.21.15.5:0/686101987' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-09-10T10:05:24.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:05:23 smithi005 ceph-mon[30350]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-09-10T10:05:24.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:05:23 smithi005 ceph-mon[30350]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-10T10:05:24.057 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:05:23 smithi005 ceph-mon[30350]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-10T10:05:24.057 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:05:23 smithi005 ceph-mon[30350]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-10T10:05:24.057 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:05:23 smithi005 ceph-mon[30350]: osdmap e97: 12 total, 12 up, 12 in 2024-09-10T10:05:24.057 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:05:23 smithi005 ceph-mon[30350]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-09-10T10:05:24.231 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:05:24.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:05:23 smithi159 ceph-mon[35513]: pgmap v1204: 129 pgs: 129 active+clean; 551 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2024-09-10T10:05:24.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:05:23 smithi159 ceph-mon[35513]: from='client.? 172.21.15.5:0/686101987' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-09-10T10:05:24.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:05:23 smithi159 ceph-mon[35513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-09-10T10:05:24.351 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:05:23 smithi159 ceph-mon[35513]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-10T10:05:24.351 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:05:23 smithi159 ceph-mon[35513]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-10T10:05:24.351 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:05:23 smithi159 ceph-mon[35513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-10T10:05:24.351 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:05:23 smithi159 ceph-mon[35513]: osdmap e97: 12 total, 12 up, 12 in 2024-09-10T10:05:24.351 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:05:23 smithi159 ceph-mon[35513]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-09-10T10:05:24.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:05:23 smithi102 ceph-mon[35442]: pgmap v1204: 129 pgs: 129 active+clean; 551 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2024-09-10T10:05:24.385 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:05:23 smithi102 ceph-mon[35442]: from='client.? 172.21.15.5:0/686101987' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-09-10T10:05:24.385 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:05:23 smithi102 ceph-mon[35442]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-09-10T10:05:24.385 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:05:23 smithi102 ceph-mon[35442]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-10T10:05:24.385 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:05:23 smithi102 ceph-mon[35442]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-10T10:05:24.385 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:05:23 smithi102 ceph-mon[35442]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-10T10:05:24.385 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:05:23 smithi102 ceph-mon[35442]: osdmap e97: 12 total, 12 up, 12 in 2024-09-10T10:05:24.385 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:05:23 smithi102 ceph-mon[35442]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-09-10T10:05:24.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:24.912+0000 7f5a05400640 1 -- 172.21.15.5:0/4253774060 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5a001049a0 msgr2=0x7f5a001089f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:24.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:24.914+0000 7f5a05400640 1 --2- 172.21.15.5:0/4253774060 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5a001049a0 0x7f5a001089f0 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f59e8009900 tx=0x7f59e802a410 comp rx=0 tx=0).stop 2024-09-10T10:05:24.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:24.914+0000 7f5a05400640 1 -- 172.21.15.5:0/4253774060 shutdown_connections 2024-09-10T10:05:24.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:24.914+0000 7f5a05400640 1 --2- 172.21.15.5:0/4253774060 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5a00109120 0x7f5a00110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:24.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:24.914+0000 7f5a05400640 1 --2- 172.21.15.5:0/4253774060 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5a001049a0 0x7f5a001089f0 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:24.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:24.914+0000 7f5a05400640 1 --2- 172.21.15.5:0/4253774060 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5a00103ff0 0x7f5a001043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:24.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:24.914+0000 7f5a05400640 1 -- 172.21.15.5:0/4253774060 >> 172.21.15.5:0/4253774060 conn(0x7f5a000ffa90 msgr2=0x7f5a00101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:05:24.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:24.914+0000 7f5a05400640 1 -- 172.21.15.5:0/4253774060 shutdown_connections 2024-09-10T10:05:24.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:24.914+0000 7f5a05400640 1 -- 172.21.15.5:0/4253774060 wait complete. 2024-09-10T10:05:24.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:24.914+0000 7f5a05400640 1 Processor -- start 2024-09-10T10:05:24.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:24.914+0000 7f5a05400640 1 -- start start 2024-09-10T10:05:24.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:24.915+0000 7f5a05400640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5a00103ff0 0x7f5a0019a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:24.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:24.915+0000 7f5a05400640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5a001049a0 0x7f5a0019ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:24.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:24.915+0000 7f5a05400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5a00109120 0x7f5a0019eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:24.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:24.915+0000 7f5a05400640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f5a00114190 con 0x7f5a001049a0 2024-09-10T10:05:24.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:24.915+0000 7f5a05400640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f5a00114010 con 0x7f5a00109120 2024-09-10T10:05:24.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:24.915+0000 7f5a05400640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f5a00114310 con 0x7f5a00103ff0 2024-09-10T10:05:24.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:24.915+0000 7f59f5e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5a001049a0 0x7f5a0019ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:24.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:24.915+0000 7f59f5e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5a001049a0 0x7f5a0019ab50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56932/0 (socket says 172.21.15.5:56932) 2024-09-10T10:05:24.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:24.915+0000 7f59f5e00640 1 -- 172.21.15.5:0/4046629371 learned_addr learned my addr 172.21.15.5:0/4046629371 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T10:05:24.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:24.915+0000 7f59ff200640 1 --2- 172.21.15.5:0/4046629371 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5a00109120 0x7f5a0019eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:24.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:24.915+0000 7f59fe800640 1 --2- 172.21.15.5:0/4046629371 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5a00103ff0 0x7f5a0019a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:24.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:24.915+0000 7f59ff200640 1 -- 172.21.15.5:0/4046629371 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5a00103ff0 msgr2=0x7f5a0019a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:24.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:24.915+0000 7f59ff200640 1 --2- 172.21.15.5:0/4046629371 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5a00103ff0 0x7f5a0019a610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:24.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:24.915+0000 7f59ff200640 1 -- 172.21.15.5:0/4046629371 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5a001049a0 msgr2=0x7f5a0019ab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:24.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:24.915+0000 7f59ff200640 1 --2- 172.21.15.5:0/4046629371 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5a001049a0 0x7f5a0019ab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:24.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:24.916+0000 7f59ff200640 1 -- 172.21.15.5:0/4046629371 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a0019f5c0 con 0x7f5a00109120 2024-09-10T10:05:24.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:24.916+0000 7f59f5e00640 1 --2- 172.21.15.5:0/4046629371 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5a001049a0 0x7f5a0019ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T10:05:24.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:24.916+0000 7f59ff200640 1 --2- 172.21.15.5:0/4046629371 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5a00109120 0x7f5a0019eee0 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f59f8009fd0 tx=0x7f59f800ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:24.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:24.916+0000 7f59f7e00640 1 -- 172.21.15.5:0/4046629371 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59f800ce20 con 0x7f5a00109120 2024-09-10T10:05:24.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:24.916+0000 7f5a05400640 1 -- 172.21.15.5:0/4046629371 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a0019f8b0 con 0x7f5a00109120 2024-09-10T10:05:24.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:24.916+0000 7f59f7e00640 1 -- 172.21.15.5:0/4046629371 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f59f8005020 con 0x7f5a00109120 2024-09-10T10:05:24.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:24.916+0000 7f59f7e00640 1 -- 172.21.15.5:0/4046629371 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59f8010720 con 0x7f5a00109120 2024-09-10T10:05:24.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:24.916+0000 7f5a05400640 1 -- 172.21.15.5:0/4046629371 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a001a7190 con 0x7f5a00109120 2024-09-10T10:05:24.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:24.917+0000 7f5a05400640 1 -- 172.21.15.5:0/4046629371 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5a001043d0 con 0x7f5a00109120 2024-09-10T10:05:24.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:24.920+0000 7f59f7e00640 1 -- 172.21.15.5:0/4046629371 <== mon.2 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f59f8005540 con 0x7f5a00109120 2024-09-10T10:05:24.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:24.921+0000 7f59f7e00640 1 --2- 172.21.15.5:0/4046629371 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f59e40a2b80 0x7f59e40a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:24.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:24.921+0000 7f59f7e00640 1 -- 172.21.15.5:0/4046629371 <== mon.2 v2:172.21.15.102:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9675+0+0 (secure 0 0 0) 0x7f59f80cfdd0 con 0x7f5a00109120 2024-09-10T10:05:24.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:24.921+0000 7f59fe800640 1 --2- 172.21.15.5:0/4046629371 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f59e40a2b80 0x7f59e40a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:24.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:24.923+0000 7f59fe800640 1 --2- 172.21.15.5:0/4046629371 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f59e40a2b80 0x7f59e40a5040 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f59f0009bd0 tx=0x7f59f0005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:24.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:24.924+0000 7f59f7e00640 1 -- 172.21.15.5:0/4046629371 <== mon.2 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f59f809c730 con 0x7f5a00109120 2024-09-10T10:05:25.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:25.151+0000 7f5a05400640 1 -- 172.21.15.5:0/4046629371 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f5a0010c3c0 con 0x7f5a00109120 2024-09-10T10:05:25.307 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:05:25 smithi005 ceph-mon[30350]: mds.? [v2:172.21.15.102:6832/2820751404,v1:172.21.15.102:6834/2820751404] up:boot 2024-09-10T10:05:25.307 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:05:25 smithi005 ceph-mon[30350]: mds.? [v2:172.21.15.102:6833/10354412,v1:172.21.15.102:6835/10354412] up:boot 2024-09-10T10:05:25.307 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:05:25 smithi005 ceph-mon[30350]: mds.? [v2:172.21.15.102:6836/845223456,v1:172.21.15.102:6839/845223456] up:boot 2024-09-10T10:05:25.307 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:05:25 smithi005 ceph-mon[30350]: mds.? [v2:172.21.15.102:6837/3540224563,v1:172.21.15.102:6838/3540224563] up:boot 2024-09-10T10:05:25.307 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:05:25 smithi005 ceph-mon[30350]: mds.? [v2:172.21.15.159:6833/2325799293,v1:172.21.15.159:6835/2325799293] up:boot 2024-09-10T10:05:25.307 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:05:25 smithi005 ceph-mon[30350]: mds.? [v2:172.21.15.159:6832/3483833315,v1:172.21.15.159:6834/3483833315] up:boot 2024-09-10T10:05:25.307 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:05:25 smithi005 ceph-mon[30350]: mds.? [v2:172.21.15.159:6836/2308510533,v1:172.21.15.159:6837/2308510533] up:boot 2024-09-10T10:05:25.307 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:05:25 smithi005 ceph-mon[30350]: mds.? [v2:172.21.15.159:6838/1997277768,v1:172.21.15.159:6839/1997277768] up:boot 2024-09-10T10:05:25.307 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:05:25 smithi005 ceph-mon[30350]: mds.? [v2:172.21.15.5:6836/3470057351,v1:172.21.15.5:6837/3470057351] up:boot 2024-09-10T10:05:25.307 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:05:25 smithi005 ceph-mon[30350]: mds.? [v2:172.21.15.5:6840/2124036495,v1:172.21.15.5:6841/2124036495] up:boot 2024-09-10T10:05:25.307 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:05:25 smithi005 ceph-mon[30350]: fsmap cephfs:0/5 12 up:standby, 5 failed 2024-09-10T10:05:25.307 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:05:25 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-10T10:05:25.307 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:05:25 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-10T10:05:25.308 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:05:25 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-10T10:05:25.308 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:05:25 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-10T10:05:25.308 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:05:25 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-10T10:05:25.308 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:05:25 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-10T10:05:25.308 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:05:25 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-10T10:05:25.308 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:05:25 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-10T10:05:25.308 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:05:25 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-10T10:05:25.308 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:05:25 smithi005 ceph-mon[30350]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-10T10:05:25.308 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:05:25 smithi005 ceph-mon[30350]: pgmap v1206: 129 pgs: 129 active+clean; 551 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 1.5 KiB/s wr, 12 op/s 2024-09-10T10:05:25.351 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:05:25 smithi159 ceph-mon[35513]: mds.? [v2:172.21.15.102:6832/2820751404,v1:172.21.15.102:6834/2820751404] up:boot 2024-09-10T10:05:25.351 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:05:25 smithi159 ceph-mon[35513]: mds.? [v2:172.21.15.102:6833/10354412,v1:172.21.15.102:6835/10354412] up:boot 2024-09-10T10:05:25.351 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:05:25 smithi159 ceph-mon[35513]: mds.? [v2:172.21.15.102:6836/845223456,v1:172.21.15.102:6839/845223456] up:boot 2024-09-10T10:05:25.351 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:05:25 smithi159 ceph-mon[35513]: mds.? [v2:172.21.15.102:6837/3540224563,v1:172.21.15.102:6838/3540224563] up:boot 2024-09-10T10:05:25.351 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:05:25 smithi159 ceph-mon[35513]: mds.? [v2:172.21.15.159:6833/2325799293,v1:172.21.15.159:6835/2325799293] up:boot 2024-09-10T10:05:25.351 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:05:25 smithi159 ceph-mon[35513]: mds.? [v2:172.21.15.159:6832/3483833315,v1:172.21.15.159:6834/3483833315] up:boot 2024-09-10T10:05:25.351 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:05:25 smithi159 ceph-mon[35513]: mds.? [v2:172.21.15.159:6836/2308510533,v1:172.21.15.159:6837/2308510533] up:boot 2024-09-10T10:05:25.351 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:05:25 smithi159 ceph-mon[35513]: mds.? [v2:172.21.15.159:6838/1997277768,v1:172.21.15.159:6839/1997277768] up:boot 2024-09-10T10:05:25.351 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:05:25 smithi159 ceph-mon[35513]: mds.? [v2:172.21.15.5:6836/3470057351,v1:172.21.15.5:6837/3470057351] up:boot 2024-09-10T10:05:25.351 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:05:25 smithi159 ceph-mon[35513]: mds.? [v2:172.21.15.5:6840/2124036495,v1:172.21.15.5:6841/2124036495] up:boot 2024-09-10T10:05:25.352 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:05:25 smithi159 ceph-mon[35513]: fsmap cephfs:0/5 12 up:standby, 5 failed 2024-09-10T10:05:25.352 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:05:25 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-10T10:05:25.352 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:05:25 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-10T10:05:25.352 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:05:25 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-10T10:05:25.352 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:05:25 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-10T10:05:25.352 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:05:25 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-10T10:05:25.352 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:05:25 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-10T10:05:25.352 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:05:25 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-10T10:05:25.352 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:05:25 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-10T10:05:25.352 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:05:25 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-10T10:05:25.352 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:05:25 smithi159 ceph-mon[35513]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-10T10:05:25.352 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:05:25 smithi159 ceph-mon[35513]: pgmap v1206: 129 pgs: 129 active+clean; 551 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 1.5 KiB/s wr, 12 op/s 2024-09-10T10:05:25.385 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:05:25 smithi102 ceph-mon[35442]: mds.? [v2:172.21.15.102:6832/2820751404,v1:172.21.15.102:6834/2820751404] up:boot 2024-09-10T10:05:25.385 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:05:25 smithi102 ceph-mon[35442]: mds.? [v2:172.21.15.102:6833/10354412,v1:172.21.15.102:6835/10354412] up:boot 2024-09-10T10:05:25.385 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:05:25 smithi102 ceph-mon[35442]: mds.? [v2:172.21.15.102:6836/845223456,v1:172.21.15.102:6839/845223456] up:boot 2024-09-10T10:05:25.385 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:05:25 smithi102 ceph-mon[35442]: mds.? [v2:172.21.15.102:6837/3540224563,v1:172.21.15.102:6838/3540224563] up:boot 2024-09-10T10:05:25.385 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:05:25 smithi102 ceph-mon[35442]: mds.? [v2:172.21.15.159:6833/2325799293,v1:172.21.15.159:6835/2325799293] up:boot 2024-09-10T10:05:25.385 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:05:25 smithi102 ceph-mon[35442]: mds.? [v2:172.21.15.159:6832/3483833315,v1:172.21.15.159:6834/3483833315] up:boot 2024-09-10T10:05:25.385 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:05:25 smithi102 ceph-mon[35442]: mds.? [v2:172.21.15.159:6836/2308510533,v1:172.21.15.159:6837/2308510533] up:boot 2024-09-10T10:05:25.385 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:05:25 smithi102 ceph-mon[35442]: mds.? [v2:172.21.15.159:6838/1997277768,v1:172.21.15.159:6839/1997277768] up:boot 2024-09-10T10:05:25.385 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:05:25 smithi102 ceph-mon[35442]: mds.? [v2:172.21.15.5:6836/3470057351,v1:172.21.15.5:6837/3470057351] up:boot 2024-09-10T10:05:25.385 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:05:25 smithi102 ceph-mon[35442]: mds.? [v2:172.21.15.5:6840/2124036495,v1:172.21.15.5:6841/2124036495] up:boot 2024-09-10T10:05:25.385 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:05:25 smithi102 ceph-mon[35442]: fsmap cephfs:0/5 12 up:standby, 5 failed 2024-09-10T10:05:25.386 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:05:25 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-10T10:05:25.386 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:05:25 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-10T10:05:25.386 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:05:25 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-10T10:05:25.386 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:05:25 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-10T10:05:25.386 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:05:25 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-10T10:05:25.386 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:05:25 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-10T10:05:25.386 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:05:25 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-10T10:05:25.386 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:05:25 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-10T10:05:25.386 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:05:25 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-10T10:05:25.386 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:05:25 smithi102 ceph-mon[35442]: from='mgr.14150 172.21.15.5:0/1405015223' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-10T10:05:25.386 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:05:25 smithi102 ceph-mon[35442]: pgmap v1206: 129 pgs: 129 active+clean; 551 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 1.5 KiB/s wr, 12 op/s 2024-09-10T10:05:26.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:26.039+0000 7f59f7e00640 1 -- 172.21.15.5:0/4046629371 <== mon.2 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v50) ==== 104+0+0 (secure 0 0 0) 0x7f59f809c030 con 0x7f5a00109120 2024-09-10T10:05:26.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:26.042+0000 7f5a05400640 1 -- 172.21.15.5:0/4046629371 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f59e40a2b80 msgr2=0x7f59e40a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:26.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:26.042+0000 7f5a05400640 1 --2- 172.21.15.5:0/4046629371 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f59e40a2b80 0x7f59e40a5040 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f59f0009bd0 tx=0x7f59f0005f90 comp rx=0 tx=0).stop 2024-09-10T10:05:26.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:26.042+0000 7f5a05400640 1 -- 172.21.15.5:0/4046629371 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5a00109120 msgr2=0x7f5a0019eee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:26.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:26.042+0000 7f5a05400640 1 --2- 172.21.15.5:0/4046629371 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5a00109120 0x7f5a0019eee0 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f59f8009fd0 tx=0x7f59f800ef10 comp rx=0 tx=0).stop 2024-09-10T10:05:26.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:26.043+0000 7f5a05400640 1 -- 172.21.15.5:0/4046629371 shutdown_connections 2024-09-10T10:05:26.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:26.043+0000 7f5a05400640 1 --2- 172.21.15.5:0/4046629371 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f59e40a2b80 0x7f59e40a5040 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:26.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:26.043+0000 7f5a05400640 1 --2- 172.21.15.5:0/4046629371 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f5a00109120 0x7f5a0019eee0 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:26.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:26.043+0000 7f5a05400640 1 --2- 172.21.15.5:0/4046629371 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5a001049a0 0x7f5a0019ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:26.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:26.043+0000 7f5a05400640 1 --2- 172.21.15.5:0/4046629371 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f5a00103ff0 0x7f5a0019a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:26.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:26.043+0000 7f5a05400640 1 -- 172.21.15.5:0/4046629371 >> 172.21.15.5:0/4046629371 conn(0x7f5a000ffa90 msgr2=0x7f5a00100d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:05:26.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:26.043+0000 7f5a05400640 1 -- 172.21.15.5:0/4046629371 shutdown_connections 2024-09-10T10:05:26.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:26.043+0000 7f5a05400640 1 -- 172.21.15.5:0/4046629371 wait complete. 2024-09-10T10:05:26.306 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:05:26 smithi005 ceph-mon[30350]: from='client.? 172.21.15.5:0/4046629371' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-09-10T10:05:26.306 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:05:26 smithi005 ceph-mon[30350]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-09-10T10:05:26.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:05:26 smithi159 ceph-mon[35513]: from='client.? 172.21.15.5:0/4046629371' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-09-10T10:05:26.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:05:26 smithi159 ceph-mon[35513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-09-10T10:05:26.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:05:26 smithi102 ceph-mon[35442]: from='client.? 172.21.15.5:0/4046629371' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-09-10T10:05:26.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:05:26 smithi102 ceph-mon[35442]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-09-10T10:05:26.937 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-10T10:05:27.306 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:05:27 smithi005 ceph-mon[30350]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-10T10:05:27.306 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:05:27 smithi005 ceph-mon[30350]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-10T10:05:27.306 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:05:27 smithi005 ceph-mon[30350]: Cluster is now healthy 2024-09-10T10:05:27.306 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:05:27 smithi005 ceph-mon[30350]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-09-10T10:05:27.307 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:05:27 smithi005 ceph-mon[30350]: fsmap 12 up:standby 2024-09-10T10:05:27.307 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:05:27 smithi005 ceph-mon[30350]: pgmap v1207: 129 pgs: 129 active+clean; 551 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2024-09-10T10:05:27.323 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:05:27.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:05:27 smithi159 ceph-mon[35513]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-10T10:05:27.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:05:27 smithi159 ceph-mon[35513]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-10T10:05:27.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:05:27 smithi159 ceph-mon[35513]: Cluster is now healthy 2024-09-10T10:05:27.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:05:27 smithi159 ceph-mon[35513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-09-10T10:05:27.351 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:05:27 smithi159 ceph-mon[35513]: fsmap 12 up:standby 2024-09-10T10:05:27.351 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:05:27 smithi159 ceph-mon[35513]: pgmap v1207: 129 pgs: 129 active+clean; 551 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2024-09-10T10:05:27.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:05:27 smithi102 ceph-mon[35442]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-10T10:05:27.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:05:27 smithi102 ceph-mon[35442]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-10T10:05:27.385 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:05:27 smithi102 ceph-mon[35442]: Cluster is now healthy 2024-09-10T10:05:27.385 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:05:27 smithi102 ceph-mon[35442]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-09-10T10:05:27.385 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:05:27 smithi102 ceph-mon[35442]: fsmap 12 up:standby 2024-09-10T10:05:27.385 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:05:27 smithi102 ceph-mon[35442]: pgmap v1207: 129 pgs: 129 active+clean; 551 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2024-09-10T10:05:27.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:27.876+0000 7febc1600640 1 -- 172.21.15.5:0/2616844295 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7febbc103ff0 msgr2=0x7febbc1043d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:27.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:27.876+0000 7febc1600640 1 --2- 172.21.15.5:0/2616844295 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7febbc103ff0 0x7febbc1043d0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7febac009900 tx=0x7febac02a3f0 comp rx=0 tx=0).stop 2024-09-10T10:05:27.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:27.876+0000 7febc1600640 1 -- 172.21.15.5:0/2616844295 shutdown_connections 2024-09-10T10:05:27.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:27.876+0000 7febc1600640 1 --2- 172.21.15.5:0/2616844295 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7febbc109120 0x7febbc110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:27.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:27.876+0000 7febc1600640 1 --2- 172.21.15.5:0/2616844295 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7febbc1049a0 0x7febbc1089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:27.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:27.876+0000 7febc1600640 1 --2- 172.21.15.5:0/2616844295 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7febbc103ff0 0x7febbc1043d0 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:27.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:27.876+0000 7febc1600640 1 -- 172.21.15.5:0/2616844295 >> 172.21.15.5:0/2616844295 conn(0x7febbc0ffa90 msgr2=0x7febbc101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:05:27.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:27.877+0000 7febc1600640 1 -- 172.21.15.5:0/2616844295 shutdown_connections 2024-09-10T10:05:27.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:27.877+0000 7febc1600640 1 -- 172.21.15.5:0/2616844295 wait complete. 2024-09-10T10:05:27.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:27.877+0000 7febc1600640 1 Processor -- start 2024-09-10T10:05:27.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:27.878+0000 7febc1600640 1 -- start start 2024-09-10T10:05:27.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:27.878+0000 7febc1600640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7febbc103ff0 0x7febbc19a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:27.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:27.878+0000 7febc1600640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7febbc1049a0 0x7febbc19ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:27.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:27.878+0000 7febc1600640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7febbc109120 0x7febbc19eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:27.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:27.878+0000 7febc1600640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7febbc114190 con 0x7febbc103ff0 2024-09-10T10:05:27.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:27.878+0000 7febc1600640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7febbc114010 con 0x7febbc109120 2024-09-10T10:05:27.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:27.878+0000 7febc1600640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7febbc114310 con 0x7febbc1049a0 2024-09-10T10:05:27.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:27.878+0000 7febbaa00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7febbc103ff0 0x7febbc19a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:27.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:27.878+0000 7febbaa00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7febbc103ff0 0x7febbc19a620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56954/0 (socket says 172.21.15.5:56954) 2024-09-10T10:05:27.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:27.878+0000 7febbaa00640 1 -- 172.21.15.5:0/218305368 learned_addr learned my addr 172.21.15.5:0/218305368 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T10:05:27.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:27.878+0000 7febbb400640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7febbc109120 0x7febbc19eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:27.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:27.879+0000 7febb3e00640 1 --2- 172.21.15.5:0/218305368 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7febbc1049a0 0x7febbc19ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:27.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:27.879+0000 7febbb400640 1 -- 172.21.15.5:0/218305368 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7febbc1049a0 msgr2=0x7febbc19ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:27.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:27.879+0000 7febbb400640 1 --2- 172.21.15.5:0/218305368 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7febbc1049a0 0x7febbc19ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:27.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:27.879+0000 7febbb400640 1 -- 172.21.15.5:0/218305368 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7febbc103ff0 msgr2=0x7febbc19a620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:27.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:27.879+0000 7febbb400640 1 --2- 172.21.15.5:0/218305368 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7febbc103ff0 0x7febbc19a620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:27.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:27.879+0000 7febbb400640 1 -- 172.21.15.5:0/218305368 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febbc19f5d0 con 0x7febbc109120 2024-09-10T10:05:27.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:27.879+0000 7febbb400640 1 --2- 172.21.15.5:0/218305368 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7febbc109120 0x7febbc19eef0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7febb400ec50 tx=0x7febb400c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:27.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:27.879+0000 7febb3400640 1 -- 172.21.15.5:0/218305368 <== mon.2 v2:172.21.15.102:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febb4003b70 con 0x7febbc109120 2024-09-10T10:05:27.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:27.879+0000 7febb3400640 1 -- 172.21.15.5:0/218305368 <== mon.2 v2:172.21.15.102:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7febb4003d10 con 0x7febbc109120 2024-09-10T10:05:27.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:27.880+0000 7febc1600640 1 -- 172.21.15.5:0/218305368 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7febbc19f8c0 con 0x7febbc109120 2024-09-10T10:05:27.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:27.880+0000 7febb3400640 1 -- 172.21.15.5:0/218305368 <== mon.2 v2:172.21.15.102:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febb4010890 con 0x7febbc109120 2024-09-10T10:05:27.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:27.880+0000 7febc1600640 1 -- 172.21.15.5:0/218305368 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7febbc1a71a0 con 0x7febbc109120 2024-09-10T10:05:27.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:27.881+0000 7febc1600640 1 -- 172.21.15.5:0/218305368 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7febbc1043d0 con 0x7febbc109120 2024-09-10T10:05:27.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:27.883+0000 7febb3400640 1 -- 172.21.15.5:0/218305368 <== mon.2 v2:172.21.15.102:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7febb4004b40 con 0x7febbc109120 2024-09-10T10:05:27.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:27.884+0000 7febb3400640 1 --2- 172.21.15.5:0/218305368 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7feba00a2c00 0x7feba00a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:27.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:27.884+0000 7febb3400640 1 -- 172.21.15.5:0/218305368 <== mon.2 v2:172.21.15.102:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9675+0+0 (secure 0 0 0) 0x7febb40cfbb0 con 0x7febbc109120 2024-09-10T10:05:27.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:27.884+0000 7febbaa00640 1 --2- 172.21.15.5:0/218305368 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7feba00a2c00 0x7feba00a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:27.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:27.885+0000 7febbaa00640 1 --2- 172.21.15.5:0/218305368 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7feba00a2c00 0x7feba00a50c0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7febac0096c0 tx=0x7febac0046c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:27.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:27.886+0000 7febb3400640 1 -- 172.21.15.5:0/218305368 <== mon.2 v2:172.21.15.102:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7febb409c5a0 con 0x7febbc109120 2024-09-10T10:05:28.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:28.075+0000 7febc1600640 1 -- 172.21.15.5:0/218305368 --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7febbc19b980 con 0x7febbc109120 2024-09-10T10:05:28.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:28.142+0000 7febb3400640 1 -- 172.21.15.5:0/218305368 <== mon.2 v2:172.21.15.102:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v98) ==== 181+0+0 (secure 0 0 0) 0x7febbc19b980 con 0x7febbc109120 2024-09-10T10:05:28.143 INFO:teuthology.orchestra.run.smithi005.stderr:pool 'cephfs_metadata' removed 2024-09-10T10:05:28.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:28.145+0000 7febc1600640 1 -- 172.21.15.5:0/218305368 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7feba00a2c00 msgr2=0x7feba00a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:28.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:28.145+0000 7febc1600640 1 --2- 172.21.15.5:0/218305368 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7feba00a2c00 0x7feba00a50c0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7febac0096c0 tx=0x7febac0046c0 comp rx=0 tx=0).stop 2024-09-10T10:05:28.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:28.145+0000 7febc1600640 1 -- 172.21.15.5:0/218305368 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7febbc109120 msgr2=0x7febbc19eef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:28.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:28.145+0000 7febc1600640 1 --2- 172.21.15.5:0/218305368 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7febbc109120 0x7febbc19eef0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7febb400ec50 tx=0x7febb400c6a0 comp rx=0 tx=0).stop 2024-09-10T10:05:28.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:28.145+0000 7febc1600640 1 -- 172.21.15.5:0/218305368 shutdown_connections 2024-09-10T10:05:28.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:28.145+0000 7febc1600640 1 --2- 172.21.15.5:0/218305368 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7feba00a2c00 0x7feba00a50c0 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:28.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:28.145+0000 7febc1600640 1 --2- 172.21.15.5:0/218305368 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7febbc109120 0x7febbc19eef0 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:28.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:28.145+0000 7febc1600640 1 --2- 172.21.15.5:0/218305368 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7febbc1049a0 0x7febbc19ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:28.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:28.145+0000 7febc1600640 1 --2- 172.21.15.5:0/218305368 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7febbc103ff0 0x7febbc19a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:28.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:28.145+0000 7febc1600640 1 -- 172.21.15.5:0/218305368 >> 172.21.15.5:0/218305368 conn(0x7febbc0ffa90 msgr2=0x7febbc100cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:05:28.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:28.145+0000 7febc1600640 1 -- 172.21.15.5:0/218305368 shutdown_connections 2024-09-10T10:05:28.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:28.145+0000 7febc1600640 1 -- 172.21.15.5:0/218305368 wait complete. 2024-09-10T10:05:28.384 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:05:28 smithi102 ceph-mon[35442]: from='client.? 172.21.15.5:0/218305368' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-10T10:05:28.385 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:05:28 smithi102 ceph-mon[35442]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-10T10:05:28.514 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:05:28 smithi005 ceph-mon[30350]: from='client.? 172.21.15.5:0/218305368' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-10T10:05:28.514 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:05:28 smithi005 ceph-mon[30350]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-10T10:05:28.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:05:28 smithi159 ceph-mon[35513]: from='client.? 172.21.15.5:0/218305368' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-10T10:05:28.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:05:28 smithi159 ceph-mon[35513]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-10T10:05:28.937 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-10T10:05:29.361 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:05:29.500 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:05:29 smithi005 ceph-mon[30350]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-09-10T10:05:29.500 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:05:29 smithi005 ceph-mon[30350]: osdmap e98: 12 total, 12 up, 12 in 2024-09-10T10:05:29.501 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:05:29 smithi005 ceph-mon[30350]: pgmap v1209: 97 pgs: 97 active+clean; 577 KiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:05:29.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:05:29 smithi159 ceph-mon[35513]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-09-10T10:05:29.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:05:29 smithi159 ceph-mon[35513]: osdmap e98: 12 total, 12 up, 12 in 2024-09-10T10:05:29.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:05:29 smithi159 ceph-mon[35513]: pgmap v1209: 97 pgs: 97 active+clean; 577 KiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:05:29.635 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:05:29 smithi102 ceph-mon[35442]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-09-10T10:05:29.635 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:05:29 smithi102 ceph-mon[35442]: osdmap e98: 12 total, 12 up, 12 in 2024-09-10T10:05:29.635 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:05:29 smithi102 ceph-mon[35442]: pgmap v1209: 97 pgs: 97 active+clean; 577 KiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:05:30.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:30.081+0000 7f234f200640 1 -- 172.21.15.5:0/995303501 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2348100a80 msgr2=0x7f2348100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:30.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:30.081+0000 7f234f200640 1 --2- 172.21.15.5:0/995303501 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2348100a80 0x7f2348100ee0 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f2330009930 tx=0x7f233002a420 comp rx=0 tx=0).stop 2024-09-10T10:05:30.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:30.081+0000 7f234f200640 1 -- 172.21.15.5:0/995303501 shutdown_connections 2024-09-10T10:05:30.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:30.081+0000 7f234f200640 1 --2- 172.21.15.5:0/995303501 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2348101420 0x7f234810d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:30.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:30.081+0000 7f234f200640 1 --2- 172.21.15.5:0/995303501 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2348100a80 0x7f2348100ee0 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:30.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:30.081+0000 7f234f200640 1 --2- 172.21.15.5:0/995303501 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2348106a80 0x7f2348106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:30.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:30.081+0000 7f234f200640 1 -- 172.21.15.5:0/995303501 >> 172.21.15.5:0/995303501 conn(0x7f23480fc7a0 msgr2=0x7f23480febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:05:30.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:30.081+0000 7f234f200640 1 -- 172.21.15.5:0/995303501 shutdown_connections 2024-09-10T10:05:30.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:30.081+0000 7f234f200640 1 -- 172.21.15.5:0/995303501 wait complete. 2024-09-10T10:05:30.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:30.082+0000 7f234f200640 1 Processor -- start 2024-09-10T10:05:30.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:30.083+0000 7f234f200640 1 -- start start 2024-09-10T10:05:30.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:30.083+0000 7f234f200640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2348100a80 0x7f234819a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:30.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:30.083+0000 7f234f200640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2348101420 0x7f234819abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:30.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:30.083+0000 7f234f200640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2348106a80 0x7f234819ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:30.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:30.083+0000 7f234f200640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f2348111f90 con 0x7f2348100a80 2024-09-10T10:05:30.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:30.083+0000 7f234f200640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7f2348111e10 con 0x7f2348101420 2024-09-10T10:05:30.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:30.083+0000 7f234f200640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7f2348112110 con 0x7f2348106a80 2024-09-10T10:05:30.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:30.083+0000 7f2347e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2348100a80 0x7f234819a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:30.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:30.083+0000 7f2347e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2348100a80 0x7f234819a680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56974/0 (socket says 172.21.15.5:56974) 2024-09-10T10:05:30.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:30.083+0000 7f2347e00640 1 -- 172.21.15.5:0/2669947175 learned_addr learned my addr 172.21.15.5:0/2669947175 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T10:05:30.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:30.083+0000 7f233f400640 1 --2- 172.21.15.5:0/2669947175 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2348101420 0x7f234819abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:30.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:30.083+0000 7f234d200640 1 --2- 172.21.15.5:0/2669947175 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2348106a80 0x7f234819ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:30.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:30.084+0000 7f2347e00640 1 -- 172.21.15.5:0/2669947175 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2348106a80 msgr2=0x7f234819ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:30.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:30.084+0000 7f2347e00640 1 --2- 172.21.15.5:0/2669947175 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2348106a80 0x7f234819ef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:30.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:30.084+0000 7f2347e00640 1 -- 172.21.15.5:0/2669947175 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2348101420 msgr2=0x7f234819abc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:30.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:30.084+0000 7f2347e00640 1 --2- 172.21.15.5:0/2669947175 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2348101420 0x7f234819abc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:30.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:30.084+0000 7f2347e00640 1 -- 172.21.15.5:0/2669947175 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f234819f630 con 0x7f2348100a80 2024-09-10T10:05:30.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:30.084+0000 7f2347e00640 1 --2- 172.21.15.5:0/2669947175 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2348100a80 0x7f234819a680 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f233800d950 tx=0x7f233800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:30.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:30.084+0000 7f2345600640 1 -- 172.21.15.5:0/2669947175 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2338017070 con 0x7f2348100a80 2024-09-10T10:05:30.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:30.084+0000 7f2345600640 1 -- 172.21.15.5:0/2669947175 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2338009c30 con 0x7f2348100a80 2024-09-10T10:05:30.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:30.084+0000 7f234f200640 1 -- 172.21.15.5:0/2669947175 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f234819f920 con 0x7f2348100a80 2024-09-10T10:05:30.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:30.084+0000 7f2345600640 1 -- 172.21.15.5:0/2669947175 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f233800f6b0 con 0x7f2348100a80 2024-09-10T10:05:30.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:30.085+0000 7f234f200640 1 -- 172.21.15.5:0/2669947175 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23481a7200 con 0x7f2348100a80 2024-09-10T10:05:30.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:30.086+0000 7f234f200640 1 -- 172.21.15.5:0/2669947175 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2348100ee0 con 0x7f2348100a80 2024-09-10T10:05:30.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:30.087+0000 7f2345600640 1 -- 172.21.15.5:0/2669947175 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f23380109d0 con 0x7f2348100a80 2024-09-10T10:05:30.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:30.089+0000 7f2345600640 1 --2- 172.21.15.5:0/2669947175 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f232c0a2c00 0x7f232c0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:30.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:30.089+0000 7f2345600640 1 -- 172.21.15.5:0/2669947175 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9121+0+0 (secure 0 0 0) 0x7f23380ceec0 con 0x7f2348100a80 2024-09-10T10:05:30.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:30.089+0000 7f233f400640 1 --2- 172.21.15.5:0/2669947175 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f232c0a2c00 0x7f232c0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:30.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:30.090+0000 7f233f400640 1 --2- 172.21.15.5:0/2669947175 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f232c0a2c00 0x7f232c0a50c0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f2330006010 tx=0x7f2330000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:30.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:30.093+0000 7f2345600640 1 -- 172.21.15.5:0/2669947175 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f2338009de0 con 0x7f2348100a80 2024-09-10T10:05:30.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:30.281+0000 7f234f200640 1 -- 172.21.15.5:0/2669947175 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f23481a7430 con 0x7f2348100a80 2024-09-10T10:05:31.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:31.270+0000 7f2345600640 1 -- 172.21.15.5:0/2669947175 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v99) ==== 169+0+0 (secure 0 0 0) 0x7f233809b3f0 con 0x7f2348100a80 2024-09-10T10:05:31.271 INFO:teuthology.orchestra.run.smithi005.stderr:pool 'cephfs_data' removed 2024-09-10T10:05:31.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:31.274+0000 7f234f200640 1 -- 172.21.15.5:0/2669947175 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f232c0a2c00 msgr2=0x7f232c0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:31.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:31.274+0000 7f234f200640 1 --2- 172.21.15.5:0/2669947175 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f232c0a2c00 0x7f232c0a50c0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f2330006010 tx=0x7f2330000bc0 comp rx=0 tx=0).stop 2024-09-10T10:05:31.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:31.274+0000 7f234f200640 1 -- 172.21.15.5:0/2669947175 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2348100a80 msgr2=0x7f234819a680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:31.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:31.274+0000 7f234f200640 1 --2- 172.21.15.5:0/2669947175 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2348100a80 0x7f234819a680 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f233800d950 tx=0x7f233800de10 comp rx=0 tx=0).stop 2024-09-10T10:05:31.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:31.274+0000 7f234f200640 1 -- 172.21.15.5:0/2669947175 shutdown_connections 2024-09-10T10:05:31.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:31.274+0000 7f234f200640 1 --2- 172.21.15.5:0/2669947175 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7f232c0a2c00 0x7f232c0a50c0 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:31.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:31.274+0000 7f234f200640 1 --2- 172.21.15.5:0/2669947175 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7f2348106a80 0x7f234819ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:31.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:31.274+0000 7f234f200640 1 --2- 172.21.15.5:0/2669947175 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7f2348101420 0x7f234819abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:31.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:31.274+0000 7f234f200640 1 --2- 172.21.15.5:0/2669947175 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2348100a80 0x7f234819a680 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:31.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:31.274+0000 7f234f200640 1 -- 172.21.15.5:0/2669947175 >> 172.21.15.5:0/2669947175 conn(0x7f23480fc7a0 msgr2=0x7f234810c150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:05:31.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:31.275+0000 7f234f200640 1 -- 172.21.15.5:0/2669947175 shutdown_connections 2024-09-10T10:05:31.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:31.275+0000 7f234f200640 1 -- 172.21.15.5:0/2669947175 wait complete. 2024-09-10T10:05:31.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:05:31 smithi005 ceph-mon[30350]: pgmap v1210: 97 pgs: 97 active+clean; 577 KiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:05:31.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:05:31 smithi005 ceph-mon[30350]: from='client.? 172.21.15.5:0/2669947175' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-10T10:05:31.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:05:31 smithi159 ceph-mon[35513]: pgmap v1210: 97 pgs: 97 active+clean; 577 KiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:05:31.601 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:05:31 smithi159 ceph-mon[35513]: from='client.? 172.21.15.5:0/2669947175' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-10T10:05:31.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:05:31 smithi102 ceph-mon[35442]: pgmap v1210: 97 pgs: 97 active+clean; 577 KiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:05:31.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:05:31 smithi102 ceph-mon[35442]: from='client.? 172.21.15.5:0/2669947175' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-10T10:05:32.007 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b shell --fsid bade4b50-6f56-11ef-bcea-c7b262605968 -- ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2024-09-10T10:05:32.387 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T10:05:32.533 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:05:32 smithi005 ceph-mon[30350]: from='client.? 172.21.15.5:0/2669947175' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-09-10T10:05:32.533 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:05:32 smithi005 ceph-mon[30350]: osdmap e99: 12 total, 12 up, 12 in 2024-09-10T10:05:32.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:05:32 smithi159 ceph-mon[35513]: from='client.? 172.21.15.5:0/2669947175' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-09-10T10:05:32.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:05:32 smithi159 ceph-mon[35513]: osdmap e99: 12 total, 12 up, 12 in 2024-09-10T10:05:32.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:05:32 smithi102 ceph-mon[35442]: from='client.? 172.21.15.5:0/2669947175' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-09-10T10:05:32.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:05:32 smithi102 ceph-mon[35442]: osdmap e99: 12 total, 12 up, 12 in 2024-09-10T10:05:32.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:32.942+0000 7ff9e5000640 1 -- 172.21.15.5:0/3952110215 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff9e0106f10 msgr2=0x7ff9e010ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:32.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:32.942+0000 7ff9e5000640 1 --2- 172.21.15.5:0/3952110215 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff9e0106f10 0x7ff9e010ea90 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7ff9d8009ca0 tx=0x7ff9d802a8c0 comp rx=0 tx=0).stop 2024-09-10T10:05:32.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:32.943+0000 7ff9e5000640 1 -- 172.21.15.5:0/3952110215 shutdown_connections 2024-09-10T10:05:32.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:32.943+0000 7ff9e5000640 1 --2- 172.21.15.5:0/3952110215 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff9e0106f10 0x7ff9e010ea90 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:32.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:32.943+0000 7ff9e5000640 1 --2- 172.21.15.5:0/3952110215 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff9e0102790 0x7ff9e01067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:32.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:32.943+0000 7ff9e5000640 1 --2- 172.21.15.5:0/3952110215 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff9e0101de0 0x7ff9e01021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:32.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:32.943+0000 7ff9e5000640 1 -- 172.21.15.5:0/3952110215 >> 172.21.15.5:0/3952110215 conn(0x7ff9e00fd8c0 msgr2=0x7ff9e00ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:05:32.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:32.943+0000 7ff9e5000640 1 -- 172.21.15.5:0/3952110215 shutdown_connections 2024-09-10T10:05:32.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:32.943+0000 7ff9e5000640 1 -- 172.21.15.5:0/3952110215 wait complete. 2024-09-10T10:05:32.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:32.944+0000 7ff9e5000640 1 Processor -- start 2024-09-10T10:05:32.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:32.944+0000 7ff9e5000640 1 -- start start 2024-09-10T10:05:32.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:32.944+0000 7ff9e5000640 1 --2- >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff9e0101de0 0x7ff9e01983c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:32.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:32.944+0000 7ff9e5000640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff9e0102790 0x7ff9e0198900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:32.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:32.944+0000 7ff9e5000640 1 --2- >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff9e0106f10 0x7ff9e019cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:32.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:32.944+0000 7ff9e5000640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff9e0111f70 con 0x7ff9e0102790 2024-09-10T10:05:32.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:32.944+0000 7ff9e5000640 1 -- --> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] -- mon_getmap magic: 0 -- 0x7ff9e0111df0 con 0x7ff9e0101de0 2024-09-10T10:05:32.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:32.944+0000 7ff9e5000640 1 -- --> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] -- mon_getmap magic: 0 -- 0x7ff9e01120f0 con 0x7ff9e0106f10 2024-09-10T10:05:32.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:32.944+0000 7ff9d7e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff9e0102790 0x7ff9e0198900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:32.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:32.944+0000 7ff9d7e00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff9e0102790 0x7ff9e0198900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33014/0 (socket says 172.21.15.5:33014) 2024-09-10T10:05:32.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:32.944+0000 7ff9d7e00640 1 -- 172.21.15.5:0/45692462 learned_addr learned my addr 172.21.15.5:0/45692462 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-10T10:05:32.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:32.944+0000 7ff9dde00640 1 --2- 172.21.15.5:0/45692462 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff9e0101de0 0x7ff9e01983c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:32.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:32.945+0000 7ff9de800640 1 --2- 172.21.15.5:0/45692462 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff9e0106f10 0x7ff9e019cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:32.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:32.945+0000 7ff9d7e00640 1 -- 172.21.15.5:0/45692462 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff9e0106f10 msgr2=0x7ff9e019cc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:32.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:32.945+0000 7ff9d7e00640 1 --2- 172.21.15.5:0/45692462 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff9e0106f10 0x7ff9e019cc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:32.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:32.945+0000 7ff9d7e00640 1 -- 172.21.15.5:0/45692462 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff9e0101de0 msgr2=0x7ff9e01983c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:32.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:32.945+0000 7ff9d7e00640 1 --2- 172.21.15.5:0/45692462 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff9e0101de0 0x7ff9e01983c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:32.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:32.945+0000 7ff9d7e00640 1 -- 172.21.15.5:0/45692462 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9e019d370 con 0x7ff9e0102790 2024-09-10T10:05:32.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:32.945+0000 7ff9de800640 1 --2- 172.21.15.5:0/45692462 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff9e0106f10 0x7ff9e019cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T10:05:32.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:32.945+0000 7ff9d7e00640 1 --2- 172.21.15.5:0/45692462 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff9e0102790 0x7ff9e0198900 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7ff9c800e9f0 tx=0x7ff9c800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:32.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:32.945+0000 7ff9d6a00640 1 -- 172.21.15.5:0/45692462 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9c800cd80 con 0x7ff9e0102790 2024-09-10T10:05:32.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:32.945+0000 7ff9d6a00640 1 -- 172.21.15.5:0/45692462 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff9c8010910 con 0x7ff9e0102790 2024-09-10T10:05:32.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:32.945+0000 7ff9d6a00640 1 -- 172.21.15.5:0/45692462 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9c8018990 con 0x7ff9e0102790 2024-09-10T10:05:32.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:32.945+0000 7ff9e5000640 1 -- 172.21.15.5:0/45692462 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9e019d660 con 0x7ff9e0102790 2024-09-10T10:05:32.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:32.946+0000 7ff9e5000640 1 -- 172.21.15.5:0/45692462 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9e01a4f40 con 0x7ff9e0102790 2024-09-10T10:05:32.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:32.947+0000 7ff9e5000640 1 -- 172.21.15.5:0/45692462 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff9e01021c0 con 0x7ff9e0102790 2024-09-10T10:05:32.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:32.948+0000 7ff9d6a00640 1 -- 172.21.15.5:0/45692462 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7ff9c8010ac0 con 0x7ff9e0102790 2024-09-10T10:05:32.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:32.950+0000 7ff9d6a00640 1 --2- 172.21.15.5:0/45692462 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7ff9c40a2c00 0x7ff9c40a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T10:05:32.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:32.950+0000 7ff9d6a00640 1 -- 172.21.15.5:0/45692462 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8496+0+0 (secure 0 0 0) 0x7ff9c80cf400 con 0x7ff9e0102790 2024-09-10T10:05:32.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:32.950+0000 7ff9dde00640 1 --2- 172.21.15.5:0/45692462 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7ff9c40a2c00 0x7ff9c40a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T10:05:32.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:32.951+0000 7ff9dde00640 1 --2- 172.21.15.5:0/45692462 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7ff9c40a2c00 0x7ff9c40a50c0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7ff9d0009bd0 tx=0x7ff9d0005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T10:05:32.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:32.954+0000 7ff9d6a00640 1 -- 172.21.15.5:0/45692462 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff9c809c220 con 0x7ff9e0102790 2024-09-10T10:05:33.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:33.141+0000 7ff9e5000640 1 -- 172.21.15.5:0/45692462 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) -- 0x7ff9e0109bc0 con 0x7ff9e0102790 2024-09-10T10:05:33.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:33.282+0000 7ff9d6a00640 1 -- 172.21.15.5:0/45692462 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v100) ==== 178+0+0 (secure 0 0 0) 0x7ff9c809bb20 con 0x7ff9e0102790 2024-09-10T10:05:33.283 INFO:teuthology.orchestra.run.smithi005.stderr:pool 'cephfs_data_ec' removed 2024-09-10T10:05:33.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:33.286+0000 7ff9e5000640 1 -- 172.21.15.5:0/45692462 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7ff9c40a2c00 msgr2=0x7ff9c40a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:33.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:33.286+0000 7ff9e5000640 1 --2- 172.21.15.5:0/45692462 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7ff9c40a2c00 0x7ff9c40a50c0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7ff9d0009bd0 tx=0x7ff9d0005f90 comp rx=0 tx=0).stop 2024-09-10T10:05:33.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:33.286+0000 7ff9e5000640 1 -- 172.21.15.5:0/45692462 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff9e0102790 msgr2=0x7ff9e0198900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T10:05:33.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:33.286+0000 7ff9e5000640 1 --2- 172.21.15.5:0/45692462 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff9e0102790 0x7ff9e0198900 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7ff9c800e9f0 tx=0x7ff9c800eeb0 comp rx=0 tx=0).stop 2024-09-10T10:05:33.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:33.286+0000 7ff9e5000640 1 -- 172.21.15.5:0/45692462 shutdown_connections 2024-09-10T10:05:33.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:33.286+0000 7ff9e5000640 1 --2- 172.21.15.5:0/45692462 >> [v2:172.21.15.5:6800/2707348081,v1:172.21.15.5:6801/2707348081] conn(0x7ff9c40a2c00 0x7ff9c40a50c0 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:33.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:33.286+0000 7ff9e5000640 1 --2- 172.21.15.5:0/45692462 >> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] conn(0x7ff9e0106f10 0x7ff9e019cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:33.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:33.286+0000 7ff9e5000640 1 --2- 172.21.15.5:0/45692462 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff9e0102790 0x7ff9e0198900 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:33.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:33.286+0000 7ff9e5000640 1 --2- 172.21.15.5:0/45692462 >> [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] conn(0x7ff9e0101de0 0x7ff9e01983c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T10:05:33.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:33.286+0000 7ff9e5000640 1 -- 172.21.15.5:0/45692462 >> 172.21.15.5:0/45692462 conn(0x7ff9e00fd8c0 msgr2=0x7ff9e00febd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T10:05:33.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:33.286+0000 7ff9e5000640 1 -- 172.21.15.5:0/45692462 shutdown_connections 2024-09-10T10:05:33.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-10T10:05:33.290+0000 7ff9e5000640 1 -- 172.21.15.5:0/45692462 wait complete. 2024-09-10T10:05:33.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:05:33 smithi005 ceph-mon[30350]: pgmap v1212: 33 pgs: 33 active+clean; 577 KiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:05:33.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:05:33 smithi005 ceph-mon[30350]: from='client.? 172.21.15.5:0/45692462' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-10T10:05:33.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:05:33 smithi159 ceph-mon[35513]: pgmap v1212: 33 pgs: 33 active+clean; 577 KiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:05:33.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:05:33 smithi159 ceph-mon[35513]: from='client.? 172.21.15.5:0/45692462' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-10T10:05:33.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:05:33 smithi102 ceph-mon[35442]: pgmap v1212: 33 pgs: 33 active+clean; 577 KiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T10:05:33.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:05:33 smithi102 ceph-mon[35442]: from='client.? 172.21.15.5:0/45692462' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-10T10:05:34.230 DEBUG:teuthology.orchestra.run.smithi005:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-10T10:05:34.287 DEBUG:teuthology.orchestra.run.smithi102:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-10T10:05:34.322 DEBUG:teuthology.orchestra.run.smithi159:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-10T10:05:34.358 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-09-10T10:05:34.358 DEBUG:teuthology.orchestra.run.smithi005:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-10T10:05:34.397 DEBUG:teuthology.orchestra.run.smithi102:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-10T10:05:34.417 DEBUG:teuthology.orchestra.run.smithi159:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-10T10:05:34.433 INFO:tasks.cephadm:Stopping all daemons... 2024-09-10T10:05:34.433 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-09-10T10:05:34.433 DEBUG:teuthology.orchestra.run.smithi005:> sudo systemctl stop ceph-bade4b50-6f56-11ef-bcea-c7b262605968@mon.a 2024-09-10T10:05:34.486 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:05:34 smithi005 ceph-mon[30350]: from='client.? 172.21.15.5:0/45692462' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]': finished 2024-09-10T10:05:34.486 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:05:34 smithi005 ceph-mon[30350]: osdmap e100: 12 total, 12 up, 12 in 2024-09-10T10:05:34.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:05:34 smithi159 ceph-mon[35513]: from='client.? 172.21.15.5:0/45692462' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]': finished 2024-09-10T10:05:34.600 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:05:34 smithi159 ceph-mon[35513]: osdmap e100: 12 total, 12 up, 12 in 2024-09-10T10:05:34.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:05:34 smithi102 ceph-mon[35442]: from='client.? 172.21.15.5:0/45692462' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]': finished 2024-09-10T10:05:34.634 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:05:34 smithi102 ceph-mon[35442]: osdmap e100: 12 total, 12 up, 12 in 2024-09-10T10:05:34.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:05:34 smithi005 systemd[1]: Stopping Ceph mon.a for bade4b50-6f56-11ef-bcea-c7b262605968... 2024-09-10T10:05:35.306 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:05:34 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mon-a[30346]: 2024-09-10T10:05:34.890+0000 7f133d400640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T10:05:35.307 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:05:34 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mon-a[30346]: 2024-09-10T10:05:34.890+0000 7f133d400640 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-09-10T10:05:35.307 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:05:35 smithi005 podman[143477]: 2024-09-10 10:05:35.126243895 +0000 UTC m=+0.403781587 container died 36f7420b52880ca9c13a94f30b28e7998c72e348aea621b0a554120f9a6a4df8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b, name=ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mon-a, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-10T10:05:35.708 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:05:35 smithi005 podman[143477]: 2024-09-10 10:05:35.686935333 +0000 UTC m=+0.964473062 container cleanup 36f7420b52880ca9c13a94f30b28e7998c72e348aea621b0a554120f9a6a4df8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b, name=ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mon-a, ceph=True, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-10T10:05:35.708 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 10 10:05:35 smithi005 bash[143477]: ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mon-a 2024-09-10T10:05:36.186 DEBUG:teuthology.orchestra.run.smithi005:> sudo pkill -f 'journalctl -f -n 0 -u ceph-bade4b50-6f56-11ef-bcea-c7b262605968@mon.a.service' 2024-09-10T10:05:36.273 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T10:05:36.273 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-09-10T10:05:36.273 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-09-10T10:05:36.273 DEBUG:teuthology.orchestra.run.smithi102:> sudo systemctl stop ceph-bade4b50-6f56-11ef-bcea-c7b262605968@mon.b 2024-09-10T10:05:36.635 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:05:36 smithi102 systemd[1]: Stopping Ceph mon.b for bade4b50-6f56-11ef-bcea-c7b262605968... 2024-09-10T10:05:37.135 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:05:36 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mon-b[35438]: 2024-09-10T10:05:36.803+0000 7f62ca723640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T10:05:37.135 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:05:36 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mon-b[35438]: 2024-09-10T10:05:36.803+0000 7f62ca723640 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-09-10T10:05:37.135 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:05:36 smithi102 podman[75152]: 2024-09-10 10:05:36.916816025 +0000 UTC m=+0.309227585 container died 36a004c0c1829086524204befeaf8e702aae1ff698b9d0024da65c178ff10952 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mon-b, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, ceph=True, GIT_CLEAN=True, io.buildah.version=1.37.2) 2024-09-10T10:05:37.478 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:05:37 smithi102 podman[75152]: 2024-09-10 10:05:37.462331045 +0000 UTC m=+0.854742610 container cleanup 36a004c0c1829086524204befeaf8e702aae1ff698b9d0024da65c178ff10952 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mon-b, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS) 2024-09-10T10:05:37.478 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:05:37 smithi102 bash[75152]: ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mon-b 2024-09-10T10:05:37.935 DEBUG:teuthology.orchestra.run.smithi102:> sudo pkill -f 'journalctl -f -n 0 -u ceph-bade4b50-6f56-11ef-bcea-c7b262605968@mon.b.service' 2024-09-10T10:05:37.958 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:05:37 smithi102 podman[75170]: 2024-09-10 10:05:37.714289321 +0000 UTC m=+0.792687252 container remove 36a004c0c1829086524204befeaf8e702aae1ff698b9d0024da65c178ff10952 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mon-b, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240731) 2024-09-10T10:05:37.958 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:05:37 smithi102 systemd[1]: ceph-bade4b50-6f56-11ef-bcea-c7b262605968@mon.b.service: Deactivated successfully. 2024-09-10T10:05:37.958 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:05:37 smithi102 systemd[1]: Stopped Ceph mon.b for bade4b50-6f56-11ef-bcea-c7b262605968. 2024-09-10T10:05:37.959 INFO:journalctl@ceph.mon.b.smithi102.stdout:Sep 10 10:05:37 smithi102 systemd[1]: ceph-bade4b50-6f56-11ef-bcea-c7b262605968@mon.b.service: Consumed 20.544s CPU time. 2024-09-10T10:05:38.020 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T10:05:38.020 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-09-10T10:05:38.020 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-09-10T10:05:38.020 DEBUG:teuthology.orchestra.run.smithi159:> sudo systemctl stop ceph-bade4b50-6f56-11ef-bcea-c7b262605968@mon.c 2024-09-10T10:05:38.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:05:38 smithi159 systemd[1]: Stopping Ceph mon.c for bade4b50-6f56-11ef-bcea-c7b262605968... 2024-09-10T10:05:38.850 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:05:38 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mon-c[35509]: 2024-09-10T10:05:38.477+0000 7f744a367640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T10:05:38.851 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:05:38 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mon-c[35509]: 2024-09-10T10:05:38.477+0000 7f744a367640 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-09-10T10:05:38.851 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:05:38 smithi159 podman[76468]: 2024-09-10 10:05:38.528403105 +0000 UTC m=+0.174889395 container died ae880eacaff22940e28d0d371e222998bd96cf665225f3c0d7ce6d6adf71ff25 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mon-c, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T10:05:39.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:05:39 smithi159 podman[76468]: 2024-09-10 10:05:39.119552532 +0000 UTC m=+0.766038821 container cleanup ae880eacaff22940e28d0d371e222998bd96cf665225f3c0d7ce6d6adf71ff25 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mon-c, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-09-10T10:05:39.350 INFO:journalctl@ceph.mon.c.smithi159.stdout:Sep 10 10:05:39 smithi159 bash[76468]: ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mon-c 2024-09-10T10:05:39.809 DEBUG:teuthology.orchestra.run.smithi159:> sudo pkill -f 'journalctl -f -n 0 -u ceph-bade4b50-6f56-11ef-bcea-c7b262605968@mon.c.service' 2024-09-10T10:05:39.853 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T10:05:39.853 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-09-10T10:05:39.853 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-09-10T10:05:39.853 DEBUG:teuthology.orchestra.run.smithi005:> sudo systemctl stop ceph-bade4b50-6f56-11ef-bcea-c7b262605968@mgr.x 2024-09-10T10:05:40.306 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 10 10:05:39 smithi005 systemd[1]: Stopping Ceph mgr.x for bade4b50-6f56-11ef-bcea-c7b262605968... 2024-09-10T10:05:40.806 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 10 10:05:40 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mgr-x[30575]: 2024-09-10T10:05:40.436+0000 7f5703c00640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T10:05:40.806 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 10 10:05:40 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mgr-x[30575]: 2024-09-10T10:05:40.436+0000 7f5703c00640 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2024-09-10T10:05:40.806 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 10 10:05:40 smithi005 podman[143593]: 2024-09-10 10:05:40.519389316 +0000 UTC m=+0.376492908 container died 65d4fbac1f457ada3a5151d42ec4b7808cb8e28032f36febab2944fc1446a53e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b, name=ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mgr-x, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS) 2024-09-10T10:05:41.556 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 10 10:05:41 smithi005 podman[143593]: 2024-09-10 10:05:41.118005821 +0000 UTC m=+0.975109396 container cleanup 65d4fbac1f457ada3a5151d42ec4b7808cb8e28032f36febab2944fc1446a53e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b, name=ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mgr-x, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True) 2024-09-10T10:05:41.556 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 10 10:05:41 smithi005 bash[143593]: ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mgr-x 2024-09-10T10:05:41.556 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 10 10:05:41 smithi005 podman[143605]: 2024-09-10 10:05:41.336841387 +0000 UTC m=+0.805012905 container remove 65d4fbac1f457ada3a5151d42ec4b7808cb8e28032f36febab2944fc1446a53e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b, name=ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mgr-x, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, ceph=True, org.label-schema.build-date=20240731) 2024-09-10T10:05:41.594 DEBUG:teuthology.orchestra.run.smithi005:> sudo pkill -f 'journalctl -f -n 0 -u ceph-bade4b50-6f56-11ef-bcea-c7b262605968@mgr.x.service' 2024-09-10T10:05:41.668 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T10:05:41.669 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-09-10T10:05:41.669 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-09-10T10:05:41.669 DEBUG:teuthology.orchestra.run.smithi102:> sudo systemctl stop ceph-bade4b50-6f56-11ef-bcea-c7b262605968@mgr.y 2024-09-10T10:05:41.998 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 10 10:05:41 smithi102 systemd[1]: Stopping Ceph mgr.y for bade4b50-6f56-11ef-bcea-c7b262605968... 2024-09-10T10:05:42.286 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 10 10:05:41 smithi102 podman[75285]: 2024-09-10 10:05:41.997290583 +0000 UTC m=+0.128105536 container died b5d70fe88cc5c41b2162b86c1aa35742629352abbe658eaa71fcef1aecb226d0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mgr-y, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-09-10T10:05:42.634 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 10 10:05:42 smithi102 podman[75285]: 2024-09-10 10:05:42.393815819 +0000 UTC m=+0.524630813 container cleanup b5d70fe88cc5c41b2162b86c1aa35742629352abbe658eaa71fcef1aecb226d0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mgr-y, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2) 2024-09-10T10:05:42.634 INFO:journalctl@ceph.mgr.y.smithi102.stdout:Sep 10 10:05:42 smithi102 bash[75285]: ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mgr-y 2024-09-10T10:05:42.816 DEBUG:teuthology.orchestra.run.smithi102:> sudo pkill -f 'journalctl -f -n 0 -u ceph-bade4b50-6f56-11ef-bcea-c7b262605968@mgr.y.service' 2024-09-10T10:05:42.851 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T10:05:42.851 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-09-10T10:05:42.851 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-09-10T10:05:42.851 DEBUG:teuthology.orchestra.run.smithi159:> sudo systemctl stop ceph-bade4b50-6f56-11ef-bcea-c7b262605968@mgr.z 2024-09-10T10:05:43.196 INFO:journalctl@ceph.mgr.z.smithi159.stdout:Sep 10 10:05:42 smithi159 systemd[1]: Stopping Ceph mgr.z for bade4b50-6f56-11ef-bcea-c7b262605968... 2024-09-10T10:05:43.527 INFO:journalctl@ceph.mgr.z.smithi159.stdout:Sep 10 10:05:43 smithi159 podman[76589]: 2024-09-10 10:05:43.196035607 +0000 UTC m=+0.147398825 container died 0d11d57166b6b33dfe0757c0b933af912dc56640a6ab7006781e8b50d8a1b02b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mgr-z, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, ceph=True) 2024-09-10T10:05:43.850 INFO:journalctl@ceph.mgr.z.smithi159.stdout:Sep 10 10:05:43 smithi159 podman[76589]: 2024-09-10 10:05:43.626043075 +0000 UTC m=+0.577406288 container cleanup 0d11d57166b6b33dfe0757c0b933af912dc56640a6ab7006781e8b50d8a1b02b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mgr-z, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, ceph=True, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T10:05:43.850 INFO:journalctl@ceph.mgr.z.smithi159.stdout:Sep 10 10:05:43 smithi159 bash[76589]: ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mgr-z 2024-09-10T10:05:44.093 DEBUG:teuthology.orchestra.run.smithi159:> sudo pkill -f 'journalctl -f -n 0 -u ceph-bade4b50-6f56-11ef-bcea-c7b262605968@mgr.z.service' 2024-09-10T10:05:44.128 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T10:05:44.128 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-09-10T10:05:44.128 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-09-10T10:05:44.128 DEBUG:teuthology.orchestra.run.smithi005:> sudo systemctl stop ceph-bade4b50-6f56-11ef-bcea-c7b262605968@osd.0 2024-09-10T10:05:44.556 INFO:journalctl@ceph.osd.0.smithi005.stdout:Sep 10 10:05:44 smithi005 systemd[1]: Stopping Ceph osd.0 for bade4b50-6f56-11ef-bcea-c7b262605968... 2024-09-10T10:05:45.056 INFO:journalctl@ceph.osd.0.smithi005.stdout:Sep 10 10:05:44 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-0[44625]: 2024-09-10T10:05:44.625+0000 7f7a29600640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T10:05:45.056 INFO:journalctl@ceph.osd.0.smithi005.stdout:Sep 10 10:05:44 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-0[44625]: 2024-09-10T10:05:44.625+0000 7f7a29600640 -1 osd.0 100 *** Got signal Terminated *** 2024-09-10T10:05:45.057 INFO:journalctl@ceph.osd.0.smithi005.stdout:Sep 10 10:05:44 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-0[44625]: 2024-09-10T10:05:44.625+0000 7f7a29600640 -1 osd.0 100 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-10T10:05:50.551 INFO:journalctl@ceph.osd.0.smithi005.stdout:Sep 10 10:05:50 smithi005 podman[143712]: 2024-09-10 10:05:50.264172281 +0000 UTC m=+5.851096462 container died e5730e5c6ffc67bf3df7113e3373b9d86613936057157e0b0906addfa0677222 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-0, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-09-10T10:05:50.806 INFO:journalctl@ceph.osd.0.smithi005.stdout:Sep 10 10:05:50 smithi005 podman[143712]: 2024-09-10 10:05:50.765108053 +0000 UTC m=+6.352032226 container cleanup e5730e5c6ffc67bf3df7113e3373b9d86613936057157e0b0906addfa0677222 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-0, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-10T10:05:50.806 INFO:journalctl@ceph.osd.0.smithi005.stdout:Sep 10 10:05:50 smithi005 bash[143712]: ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-0 2024-09-10T10:05:51.306 INFO:journalctl@ceph.osd.0.smithi005.stdout:Sep 10 10:05:51 smithi005 podman[143730]: 2024-09-10 10:05:51.034087522 +0000 UTC m=+0.760935725 container remove e5730e5c6ffc67bf3df7113e3373b9d86613936057157e0b0906addfa0677222 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD) 2024-09-10T10:05:51.788 INFO:journalctl@ceph.osd.0.smithi005.stdout:Sep 10 10:05:51 smithi005 podman[143815]: 2024-09-10 10:05:51.397789251 +0000 UTC m=+0.027275017 image pull 41d67112835771c3efca60c6b8fe297c7ddaecdc26cea26fd28aa18636e49005 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 2024-09-10T10:05:51.788 INFO:journalctl@ceph.osd.0.smithi005.stdout:Sep 10 10:05:51 smithi005 podman[143815]: 2024-09-10 10:05:51.526468401 +0000 UTC m=+0.155954174 container create 71878d039485d345b22b8775a206e6c285eeacbc0aa634b6aebcbe6c6e425dda (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-0-deactivate, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2) 2024-09-10T10:05:51.788 INFO:journalctl@ceph.osd.0.smithi005.stdout:Sep 10 10:05:51 smithi005 podman[143815]: 2024-09-10 10:05:51.787484468 +0000 UTC m=+0.416970233 container init 71878d039485d345b22b8775a206e6c285eeacbc0aa634b6aebcbe6c6e425dda (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-0-deactivate, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b) 2024-09-10T10:05:52.056 INFO:journalctl@ceph.osd.0.smithi005.stdout:Sep 10 10:05:51 smithi005 podman[143815]: 2024-09-10 10:05:51.794880796 +0000 UTC m=+0.424366575 container start 71878d039485d345b22b8775a206e6c285eeacbc0aa634b6aebcbe6c6e425dda (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-0-deactivate, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-09-10T10:05:52.057 INFO:journalctl@ceph.osd.0.smithi005.stdout:Sep 10 10:05:51 smithi005 podman[143815]: 2024-09-10 10:05:51.870571144 +0000 UTC m=+0.500056921 container attach 71878d039485d345b22b8775a206e6c285eeacbc0aa634b6aebcbe6c6e425dda (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-0-deactivate, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-09-10T10:05:52.382 INFO:journalctl@ceph.osd.0.smithi005.stdout:Sep 10 10:05:52 smithi005 podman[143815]: 2024-09-10 10:05:52.065177397 +0000 UTC m=+0.694663165 container died 71878d039485d345b22b8775a206e6c285eeacbc0aa634b6aebcbe6c6e425dda (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-0-deactivate, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b) 2024-09-10T10:05:52.692 DEBUG:teuthology.orchestra.run.smithi005:> sudo pkill -f 'journalctl -f -n 0 -u ceph-bade4b50-6f56-11ef-bcea-c7b262605968@osd.0.service' 2024-09-10T10:05:52.770 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T10:05:52.770 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-09-10T10:05:52.771 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-09-10T10:05:52.771 DEBUG:teuthology.orchestra.run.smithi102:> sudo systemctl stop ceph-bade4b50-6f56-11ef-bcea-c7b262605968@osd.1 2024-09-10T10:05:53.116 INFO:journalctl@ceph.osd.1.smithi102.stdout:Sep 10 10:05:52 smithi102 systemd[1]: Stopping Ceph osd.1 for bade4b50-6f56-11ef-bcea-c7b262605968... 2024-09-10T10:05:53.384 INFO:journalctl@ceph.osd.1.smithi102.stdout:Sep 10 10:05:53 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-1[39907]: 2024-09-10T10:05:53.114+0000 7f23f328d640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T10:05:53.385 INFO:journalctl@ceph.osd.1.smithi102.stdout:Sep 10 10:05:53 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-1[39907]: 2024-09-10T10:05:53.114+0000 7f23f328d640 -1 osd.1 100 *** Got signal Terminated *** 2024-09-10T10:05:53.385 INFO:journalctl@ceph.osd.1.smithi102.stdout:Sep 10 10:05:53 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-1[39907]: 2024-09-10T10:05:53.114+0000 7f23f328d640 -1 osd.1 100 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-10T10:05:58.860 INFO:journalctl@ceph.osd.1.smithi102.stdout:Sep 10 10:05:58 smithi102 podman[75408]: 2024-09-10 10:05:58.565439211 +0000 UTC m=+5.599233097 container died 4033f25c36fa19f275fabfd812254ba2faa6c6b98724b6e9b3ac33720ed8c7b7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-1, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-09-10T10:05:59.134 INFO:journalctl@ceph.osd.1.smithi102.stdout:Sep 10 10:05:58 smithi102 podman[75408]: 2024-09-10 10:05:58.957527826 +0000 UTC m=+5.991321712 container cleanup 4033f25c36fa19f275fabfd812254ba2faa6c6b98724b6e9b3ac33720ed8c7b7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-1, ceph=True, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-09-10T10:05:59.134 INFO:journalctl@ceph.osd.1.smithi102.stdout:Sep 10 10:05:58 smithi102 bash[75408]: ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-1 2024-09-10T10:05:59.574 INFO:journalctl@ceph.osd.1.smithi102.stdout:Sep 10 10:05:59 smithi102 podman[75423]: 2024-09-10 10:05:59.251087069 +0000 UTC m=+0.683447127 container remove 4033f25c36fa19f275fabfd812254ba2faa6c6b98724b6e9b3ac33720ed8c7b7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-1, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-09-10T10:05:59.884 INFO:journalctl@ceph.osd.1.smithi102.stdout:Sep 10 10:05:59 smithi102 podman[75512]: 2024-09-10 10:05:59.475983566 +0000 UTC m=+0.015640404 image pull 41d67112835771c3efca60c6b8fe297c7ddaecdc26cea26fd28aa18636e49005 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 2024-09-10T10:05:59.885 INFO:journalctl@ceph.osd.1.smithi102.stdout:Sep 10 10:05:59 smithi102 podman[75512]: 2024-09-10 10:05:59.601987594 +0000 UTC m=+0.141644431 container create 4702b915e33e245a043e3f7a656021a541e168262807d3f72eb555cb42f9e5d3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-1-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=) 2024-09-10T10:06:00.376 INFO:journalctl@ceph.osd.1.smithi102.stdout:Sep 10 10:05:59 smithi102 podman[75512]: 2024-09-10 10:05:59.905486077 +0000 UTC m=+0.445142923 container init 4702b915e33e245a043e3f7a656021a541e168262807d3f72eb555cb42f9e5d3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-1-deactivate, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0) 2024-09-10T10:06:00.376 INFO:journalctl@ceph.osd.1.smithi102.stdout:Sep 10 10:05:59 smithi102 podman[75512]: 2024-09-10 10:05:59.909410244 +0000 UTC m=+0.449067081 container start 4702b915e33e245a043e3f7a656021a541e168262807d3f72eb555cb42f9e5d3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-1-deactivate, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-09-10T10:06:00.376 INFO:journalctl@ceph.osd.1.smithi102.stdout:Sep 10 10:05:59 smithi102 podman[75512]: 2024-09-10 10:05:59.996847641 +0000 UTC m=+0.536504478 container attach 4702b915e33e245a043e3f7a656021a541e168262807d3f72eb555cb42f9e5d3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-1-deactivate, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, ceph=True) 2024-09-10T10:06:00.376 INFO:journalctl@ceph.osd.1.smithi102.stdout:Sep 10 10:06:00 smithi102 podman[75512]: 2024-09-10 10:06:00.108296493 +0000 UTC m=+0.647953340 container died 4702b915e33e245a043e3f7a656021a541e168262807d3f72eb555cb42f9e5d3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-1-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T10:06:00.824 DEBUG:teuthology.orchestra.run.smithi102:> sudo pkill -f 'journalctl -f -n 0 -u ceph-bade4b50-6f56-11ef-bcea-c7b262605968@osd.1.service' 2024-09-10T10:06:00.863 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T10:06:00.863 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-09-10T10:06:00.863 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-09-10T10:06:00.863 DEBUG:teuthology.orchestra.run.smithi159:> sudo systemctl stop ceph-bade4b50-6f56-11ef-bcea-c7b262605968@osd.2 2024-09-10T10:06:01.248 INFO:journalctl@ceph.osd.2.smithi159.stdout:Sep 10 10:06:00 smithi159 systemd[1]: Stopping Ceph osd.2 for bade4b50-6f56-11ef-bcea-c7b262605968... 2024-09-10T10:06:01.600 INFO:journalctl@ceph.osd.2.smithi159.stdout:Sep 10 10:06:01 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-2[40490]: 2024-09-10T10:06:01.246+0000 7f4175473640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T10:06:01.601 INFO:journalctl@ceph.osd.2.smithi159.stdout:Sep 10 10:06:01 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-2[40490]: 2024-09-10T10:06:01.246+0000 7f4175473640 -1 osd.2 100 *** Got signal Terminated *** 2024-09-10T10:06:01.601 INFO:journalctl@ceph.osd.2.smithi159.stdout:Sep 10 10:06:01 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-2[40490]: 2024-09-10T10:06:01.246+0000 7f4175473640 -1 osd.2 100 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-10T10:06:07.040 INFO:journalctl@ceph.osd.2.smithi159.stdout:Sep 10 10:06:06 smithi159 podman[76716]: 2024-09-10 10:06:06.730239618 +0000 UTC m=+5.669074738 container died 70e47655f99aac5d82a19eb9471dc07bb9695ca26903374e75888e34a26bafc4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-2, io.buildah.version=1.37.2, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b) 2024-09-10T10:06:07.350 INFO:journalctl@ceph.osd.2.smithi159.stdout:Sep 10 10:06:07 smithi159 podman[76716]: 2024-09-10 10:06:07.130351379 +0000 UTC m=+6.069186502 container cleanup 70e47655f99aac5d82a19eb9471dc07bb9695ca26903374e75888e34a26bafc4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-10T10:06:07.350 INFO:journalctl@ceph.osd.2.smithi159.stdout:Sep 10 10:06:07 smithi159 bash[76716]: ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-2 2024-09-10T10:06:07.799 INFO:journalctl@ceph.osd.2.smithi159.stdout:Sep 10 10:06:07 smithi159 podman[76741]: 2024-09-10 10:06:07.49052443 +0000 UTC m=+0.754657129 container remove 70e47655f99aac5d82a19eb9471dc07bb9695ca26903374e75888e34a26bafc4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b) 2024-09-10T10:06:08.086 INFO:journalctl@ceph.osd.2.smithi159.stdout:Sep 10 10:06:07 smithi159 podman[76832]: 2024-09-10 10:06:07.701578209 +0000 UTC m=+0.015894533 image pull 41d67112835771c3efca60c6b8fe297c7ddaecdc26cea26fd28aa18636e49005 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 2024-09-10T10:06:08.086 INFO:journalctl@ceph.osd.2.smithi159.stdout:Sep 10 10:06:07 smithi159 podman[76832]: 2024-09-10 10:06:07.824999529 +0000 UTC m=+0.139315854 container create b45c87186010331678c3d9356684d618416f7bbd72586620491e2adfa7b884e6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-2-deactivate, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-09-10T10:06:08.351 INFO:journalctl@ceph.osd.2.smithi159.stdout:Sep 10 10:06:08 smithi159 podman[76832]: 2024-09-10 10:06:08.08603605 +0000 UTC m=+0.400352374 container init b45c87186010331678c3d9356684d618416f7bbd72586620491e2adfa7b884e6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-2-deactivate, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_CLEAN=True, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, ceph=True) 2024-09-10T10:06:08.351 INFO:journalctl@ceph.osd.2.smithi159.stdout:Sep 10 10:06:08 smithi159 podman[76832]: 2024-09-10 10:06:08.089629272 +0000 UTC m=+0.403945597 container start b45c87186010331678c3d9356684d618416f7bbd72586620491e2adfa7b884e6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-2-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, ceph=True, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2) 2024-09-10T10:06:08.351 INFO:journalctl@ceph.osd.2.smithi159.stdout:Sep 10 10:06:08 smithi159 podman[76832]: 2024-09-10 10:06:08.219862088 +0000 UTC m=+0.534178425 container attach b45c87186010331678c3d9356684d618416f7bbd72586620491e2adfa7b884e6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-2-deactivate, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T10:06:08.351 INFO:journalctl@ceph.osd.2.smithi159.stdout:Sep 10 10:06:08 smithi159 podman[76832]: 2024-09-10 10:06:08.288159424 +0000 UTC m=+0.602475749 container died b45c87186010331678c3d9356684d618416f7bbd72586620491e2adfa7b884e6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-2-deactivate, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b) 2024-09-10T10:06:08.965 DEBUG:teuthology.orchestra.run.smithi159:> sudo pkill -f 'journalctl -f -n 0 -u ceph-bade4b50-6f56-11ef-bcea-c7b262605968@osd.2.service' 2024-09-10T10:06:09.005 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T10:06:09.005 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-09-10T10:06:09.005 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-09-10T10:06:09.005 DEBUG:teuthology.orchestra.run.smithi005:> sudo systemctl stop ceph-bade4b50-6f56-11ef-bcea-c7b262605968@osd.3 2024-09-10T10:06:09.465 INFO:journalctl@ceph.osd.3.smithi005.stdout:Sep 10 10:06:09 smithi005 systemd[1]: Stopping Ceph osd.3 for bade4b50-6f56-11ef-bcea-c7b262605968... 2024-09-10T10:06:09.806 INFO:journalctl@ceph.osd.3.smithi005.stdout:Sep 10 10:06:09 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-3[51376]: 2024-09-10T10:06:09.462+0000 7fb367000640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T10:06:09.806 INFO:journalctl@ceph.osd.3.smithi005.stdout:Sep 10 10:06:09 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-3[51376]: 2024-09-10T10:06:09.462+0000 7fb367000640 -1 osd.3 100 *** Got signal Terminated *** 2024-09-10T10:06:09.807 INFO:journalctl@ceph.osd.3.smithi005.stdout:Sep 10 10:06:09 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-3[51376]: 2024-09-10T10:06:09.462+0000 7fb367000640 -1 osd.3 100 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-10T10:06:12.101 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 10 10:06:11 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-11[60437]: 2024-09-10T10:06:11.722+0000 7fa78e4d3640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-10T10:05:49.428950+0000 front 2024-09-10T10:05:49.429031+0000 (oldest deadline 2024-09-10T10:06:11.128800+0000) 2024-09-10T10:06:12.719 INFO:journalctl@ceph.osd.5.smithi159.stdout:Sep 10 10:06:12 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-5[46947]: 2024-09-10T10:06:12.450+0000 7f87f1ed7640 -1 osd.5 100 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-10T10:05:48.238525+0000 front 2024-09-10T10:05:48.238552+0000 (oldest deadline 2024-09-10T10:06:11.738538+0000) 2024-09-10T10:06:13.100 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 10 10:06:12 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-11[60437]: 2024-09-10T10:06:12.718+0000 7fa78e4d3640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-10T10:05:49.428950+0000 front 2024-09-10T10:05:49.429031+0000 (oldest deadline 2024-09-10T10:06:11.128800+0000) 2024-09-10T10:06:13.557 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 10 10:06:13 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-9[65288]: 2024-09-10T10:06:13.223+0000 7f84af200640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-10T10:05:47.346189+0000 front 2024-09-10T10:05:47.346220+0000 (oldest deadline 2024-09-10T10:06:12.646451+0000) 2024-09-10T10:06:13.720 INFO:journalctl@ceph.osd.5.smithi159.stdout:Sep 10 10:06:13 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-5[46947]: 2024-09-10T10:06:13.430+0000 7f87f1ed7640 -1 osd.5 100 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-10T10:05:48.238525+0000 front 2024-09-10T10:05:48.238552+0000 (oldest deadline 2024-09-10T10:06:11.738538+0000) 2024-09-10T10:06:13.884 INFO:journalctl@ceph.osd.10.smithi102.stdout:Sep 10 10:06:13 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-10[59874]: 2024-09-10T10:06:13.440+0000 7f40f3335640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-10T10:05:49.575097+0000 front 2024-09-10T10:05:49.575141+0000 (oldest deadline 2024-09-10T10:06:13.075035+0000) 2024-09-10T10:06:14.100 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 10 10:06:13 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-11[60437]: 2024-09-10T10:06:13.719+0000 7fa78e4d3640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-10T10:05:49.428950+0000 front 2024-09-10T10:05:49.429031+0000 (oldest deadline 2024-09-10T10:06:11.128800+0000) 2024-09-10T10:06:14.557 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 10 10:06:14 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-9[65288]: 2024-09-10T10:06:14.223+0000 7f84af200640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-10T10:05:47.346189+0000 front 2024-09-10T10:05:47.346220+0000 (oldest deadline 2024-09-10T10:06:12.646451+0000) 2024-09-10T10:06:14.742 INFO:journalctl@ceph.osd.5.smithi159.stdout:Sep 10 10:06:14 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-5[46947]: 2024-09-10T10:06:14.384+0000 7f87f1ed7640 -1 osd.5 100 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-10T10:05:48.238525+0000 front 2024-09-10T10:05:48.238552+0000 (oldest deadline 2024-09-10T10:06:11.738538+0000) 2024-09-10T10:06:14.884 INFO:journalctl@ceph.osd.10.smithi102.stdout:Sep 10 10:06:14 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-10[59874]: 2024-09-10T10:06:14.427+0000 7f40f3335640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-10T10:05:49.575097+0000 front 2024-09-10T10:05:49.575141+0000 (oldest deadline 2024-09-10T10:06:13.075035+0000) 2024-09-10T10:06:15.100 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 10 10:06:14 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-11[60437]: 2024-09-10T10:06:14.741+0000 7fa78e4d3640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-10T10:05:49.428950+0000 front 2024-09-10T10:05:49.429031+0000 (oldest deadline 2024-09-10T10:06:11.128800+0000) 2024-09-10T10:06:15.437 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 10 10:06:15 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-9[65288]: 2024-09-10T10:06:15.213+0000 7f84af200640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-10T10:05:47.346189+0000 front 2024-09-10T10:05:47.346220+0000 (oldest deadline 2024-09-10T10:06:12.646451+0000) 2024-09-10T10:06:15.437 INFO:journalctl@ceph.osd.3.smithi005.stdout:Sep 10 10:06:15 smithi005 podman[143915]: 2024-09-10 10:06:15.122532271 +0000 UTC m=+5.820420767 container died 5d96fa43618b54dcde8983abda3fe927598fb1a572356e669d6d7353ee454f46 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-3, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-09-10T10:06:15.745 INFO:journalctl@ceph.osd.5.smithi159.stdout:Sep 10 10:06:15 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-5[46947]: 2024-09-10T10:06:15.389+0000 7f87f1ed7640 -1 osd.5 100 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-10T10:05:48.238525+0000 front 2024-09-10T10:05:48.238552+0000 (oldest deadline 2024-09-10T10:06:11.738538+0000) 2024-09-10T10:06:15.806 INFO:journalctl@ceph.osd.3.smithi005.stdout:Sep 10 10:06:15 smithi005 podman[143915]: 2024-09-10 10:06:15.660328049 +0000 UTC m=+6.358216545 container cleanup 5d96fa43618b54dcde8983abda3fe927598fb1a572356e669d6d7353ee454f46 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-3, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, ceph=True, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD) 2024-09-10T10:06:15.806 INFO:journalctl@ceph.osd.3.smithi005.stdout:Sep 10 10:06:15 smithi005 bash[143915]: ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-3 2024-09-10T10:06:15.884 INFO:journalctl@ceph.osd.10.smithi102.stdout:Sep 10 10:06:15 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-10[59874]: 2024-09-10T10:06:15.452+0000 7f40f3335640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-10T10:05:49.575097+0000 front 2024-09-10T10:05:49.575141+0000 (oldest deadline 2024-09-10T10:06:13.075035+0000) 2024-09-10T10:06:16.100 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 10 10:06:15 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-11[60437]: 2024-09-10T10:06:15.744+0000 7fa78e4d3640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-10T10:05:49.428950+0000 front 2024-09-10T10:05:49.429031+0000 (oldest deadline 2024-09-10T10:06:11.128800+0000) 2024-09-10T10:06:16.171 INFO:journalctl@ceph.osd.3.smithi005.stdout:Sep 10 10:06:15 smithi005 podman[143931]: 2024-09-10 10:06:15.85373911 +0000 UTC m=+0.722374899 container remove 5d96fa43618b54dcde8983abda3fe927598fb1a572356e669d6d7353ee454f46 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-3, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b) 2024-09-10T10:06:16.430 INFO:journalctl@ceph.osd.3.smithi005.stdout:Sep 10 10:06:16 smithi005 podman[144018]: 2024-09-10 10:06:16.292123597 +0000 UTC m=+0.027518082 image pull 41d67112835771c3efca60c6b8fe297c7ddaecdc26cea26fd28aa18636e49005 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 2024-09-10T10:06:16.430 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 10 10:06:16 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-9[65288]: 2024-09-10T10:06:16.170+0000 7f84af200640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-10T10:05:47.346189+0000 front 2024-09-10T10:05:47.346220+0000 (oldest deadline 2024-09-10T10:06:12.646451+0000) 2024-09-10T10:06:16.683 INFO:journalctl@ceph.osd.3.smithi005.stdout:Sep 10 10:06:16 smithi005 podman[144018]: 2024-09-10 10:06:16.430091813 +0000 UTC m=+0.165486287 container create 7a6d047226165b5d872216e8c74bf2337b1c1bcffbb432ff1bc4cec87299e4c5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-3-deactivate, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-09-10T10:06:16.708 INFO:journalctl@ceph.osd.5.smithi159.stdout:Sep 10 10:06:16 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-5[46947]: 2024-09-10T10:06:16.413+0000 7f87f1ed7640 -1 osd.5 100 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-10T10:05:48.238525+0000 front 2024-09-10T10:05:48.238552+0000 (oldest deadline 2024-09-10T10:06:11.738538+0000) 2024-09-10T10:06:16.884 INFO:journalctl@ceph.osd.10.smithi102.stdout:Sep 10 10:06:16 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-10[59874]: 2024-09-10T10:06:16.481+0000 7f40f3335640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-10T10:05:49.575097+0000 front 2024-09-10T10:05:49.575141+0000 (oldest deadline 2024-09-10T10:06:13.075035+0000) 2024-09-10T10:06:16.953 INFO:journalctl@ceph.osd.3.smithi005.stdout:Sep 10 10:06:16 smithi005 podman[144018]: 2024-09-10 10:06:16.682673979 +0000 UTC m=+0.418068476 container init 7a6d047226165b5d872216e8c74bf2337b1c1bcffbb432ff1bc4cec87299e4c5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-3-deactivate, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2) 2024-09-10T10:06:16.954 INFO:journalctl@ceph.osd.3.smithi005.stdout:Sep 10 10:06:16 smithi005 podman[144018]: 2024-09-10 10:06:16.689669687 +0000 UTC m=+0.425064171 container start 7a6d047226165b5d872216e8c74bf2337b1c1bcffbb432ff1bc4cec87299e4c5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-3-deactivate, org.label-schema.build-date=20240731, ceph=True, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.37.2) 2024-09-10T10:06:16.954 INFO:journalctl@ceph.osd.3.smithi005.stdout:Sep 10 10:06:16 smithi005 podman[144018]: 2024-09-10 10:06:16.782413068 +0000 UTC m=+0.517807545 container attach 7a6d047226165b5d872216e8c74bf2337b1c1bcffbb432ff1bc4cec87299e4c5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-3-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240731) 2024-09-10T10:06:17.100 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 10 10:06:16 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-11[60437]: 2024-09-10T10:06:16.707+0000 7fa78e4d3640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-10T10:05:49.428950+0000 front 2024-09-10T10:05:49.429031+0000 (oldest deadline 2024-09-10T10:06:11.128800+0000) 2024-09-10T10:06:17.306 INFO:journalctl@ceph.osd.3.smithi005.stdout:Sep 10 10:06:16 smithi005 podman[144018]: 2024-09-10 10:06:16.95333991 +0000 UTC m=+0.688734404 container died 7a6d047226165b5d872216e8c74bf2337b1c1bcffbb432ff1bc4cec87299e4c5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-3-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2) 2024-09-10T10:06:17.307 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 10 10:06:17 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-9[65288]: 2024-09-10T10:06:17.134+0000 7f84af200640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-10T10:05:47.346189+0000 front 2024-09-10T10:05:47.346220+0000 (oldest deadline 2024-09-10T10:06:12.646451+0000) 2024-09-10T10:06:17.795 DEBUG:teuthology.orchestra.run.smithi005:> sudo pkill -f 'journalctl -f -n 0 -u ceph-bade4b50-6f56-11ef-bcea-c7b262605968@osd.3.service' 2024-09-10T10:06:17.850 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 10 10:06:17 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-11[60437]: 2024-09-10T10:06:17.668+0000 7fa78e4d3640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-10T10:05:49.428950+0000 front 2024-09-10T10:05:49.429031+0000 (oldest deadline 2024-09-10T10:06:11.128800+0000) 2024-09-10T10:06:17.851 INFO:journalctl@ceph.osd.5.smithi159.stdout:Sep 10 10:06:17 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-5[46947]: 2024-09-10T10:06:17.440+0000 7f87f1ed7640 -1 osd.5 100 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-10T10:05:48.238525+0000 front 2024-09-10T10:05:48.238552+0000 (oldest deadline 2024-09-10T10:06:11.738538+0000) 2024-09-10T10:06:17.872 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T10:06:17.872 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-09-10T10:06:17.872 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-09-10T10:06:17.872 DEBUG:teuthology.orchestra.run.smithi102:> sudo systemctl stop ceph-bade4b50-6f56-11ef-bcea-c7b262605968@osd.4 2024-09-10T10:06:17.884 INFO:journalctl@ceph.osd.10.smithi102.stdout:Sep 10 10:06:17 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-10[59874]: 2024-09-10T10:06:17.453+0000 7f40f3335640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-10T10:05:49.575097+0000 front 2024-09-10T10:05:49.575141+0000 (oldest deadline 2024-09-10T10:06:13.075035+0000) 2024-09-10T10:06:18.191 INFO:journalctl@ceph.osd.4.smithi102.stdout:Sep 10 10:06:17 smithi102 systemd[1]: Stopping Ceph osd.4 for bade4b50-6f56-11ef-bcea-c7b262605968... 2024-09-10T10:06:18.556 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 10 10:06:18 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-9[65288]: 2024-09-10T10:06:18.121+0000 7f84af200640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-10T10:05:47.346189+0000 front 2024-09-10T10:05:47.346220+0000 (oldest deadline 2024-09-10T10:06:12.646451+0000) 2024-09-10T10:06:18.634 INFO:journalctl@ceph.osd.10.smithi102.stdout:Sep 10 10:06:18 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-10[59874]: 2024-09-10T10:06:18.403+0000 7f40f3335640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-10T10:05:49.575097+0000 front 2024-09-10T10:05:49.575141+0000 (oldest deadline 2024-09-10T10:06:13.075035+0000) 2024-09-10T10:06:18.635 INFO:journalctl@ceph.osd.4.smithi102.stdout:Sep 10 10:06:18 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-4[46359]: 2024-09-10T10:06:18.189+0000 7f004c1d0640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T10:06:18.635 INFO:journalctl@ceph.osd.4.smithi102.stdout:Sep 10 10:06:18 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-4[46359]: 2024-09-10T10:06:18.189+0000 7f004c1d0640 -1 osd.4 100 *** Got signal Terminated *** 2024-09-10T10:06:18.635 INFO:journalctl@ceph.osd.4.smithi102.stdout:Sep 10 10:06:18 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-4[46359]: 2024-09-10T10:06:18.189+0000 7f004c1d0640 -1 osd.4 100 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-10T10:06:18.693 INFO:journalctl@ceph.osd.5.smithi159.stdout:Sep 10 10:06:18 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-5[46947]: 2024-09-10T10:06:18.407+0000 7f87f1ed7640 -1 osd.5 100 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-10T10:05:48.238525+0000 front 2024-09-10T10:05:48.238552+0000 (oldest deadline 2024-09-10T10:06:11.738538+0000) 2024-09-10T10:06:19.100 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 10 10:06:18 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-11[60437]: 2024-09-10T10:06:18.692+0000 7fa78e4d3640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-10T10:05:49.428950+0000 front 2024-09-10T10:05:49.429031+0000 (oldest deadline 2024-09-10T10:06:11.128800+0000) 2024-09-10T10:06:19.556 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 10 10:06:19 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-9[65288]: 2024-09-10T10:06:19.136+0000 7f84af200640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-10T10:05:47.346189+0000 front 2024-09-10T10:05:47.346220+0000 (oldest deadline 2024-09-10T10:06:12.646451+0000) 2024-09-10T10:06:19.712 INFO:journalctl@ceph.osd.5.smithi159.stdout:Sep 10 10:06:19 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-5[46947]: 2024-09-10T10:06:19.399+0000 7f87f1ed7640 -1 osd.5 100 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-10T10:05:48.238525+0000 front 2024-09-10T10:05:48.238552+0000 (oldest deadline 2024-09-10T10:06:11.738538+0000) 2024-09-10T10:06:19.884 INFO:journalctl@ceph.osd.10.smithi102.stdout:Sep 10 10:06:19 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-10[59874]: 2024-09-10T10:06:19.445+0000 7f40f3335640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-10T10:05:49.575097+0000 front 2024-09-10T10:05:49.575141+0000 (oldest deadline 2024-09-10T10:06:13.075035+0000) 2024-09-10T10:06:19.884 INFO:journalctl@ceph.osd.10.smithi102.stdout:Sep 10 10:06:19 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-10[59874]: 2024-09-10T10:06:19.445+0000 7f40f3335640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-09-10T10:05:54.176094+0000 front 2024-09-10T10:05:54.176115+0000 (oldest deadline 2024-09-10T10:06:19.376783+0000) 2024-09-10T10:06:20.100 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 10 10:06:19 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-11[60437]: 2024-09-10T10:06:19.711+0000 7fa78e4d3640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-10T10:05:49.428950+0000 front 2024-09-10T10:05:49.429031+0000 (oldest deadline 2024-09-10T10:06:11.128800+0000) 2024-09-10T10:06:20.287 INFO:journalctl@ceph.osd.4.smithi102.stdout:Sep 10 10:06:19 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-4[46359]: 2024-09-10T10:06:19.919+0000 7f00447de640 -1 osd.4 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-09-10T10:05:56.199631+0000 front 2024-09-10T10:05:56.199661+0000 (oldest deadline 2024-09-10T10:06:19.099760+0000) 2024-09-10T10:06:20.556 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 10 10:06:20 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-9[65288]: 2024-09-10T10:06:20.160+0000 7f84af200640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-10T10:05:47.346189+0000 front 2024-09-10T10:05:47.346220+0000 (oldest deadline 2024-09-10T10:06:12.646451+0000) 2024-09-10T10:06:20.634 INFO:journalctl@ceph.osd.7.smithi102.stdout:Sep 10 10:06:20 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-7[52880]: 2024-09-10T10:06:20.286+0000 7ff941357640 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-09-10T10:05:55.423434+0000 front 2024-09-10T10:05:55.423389+0000 (oldest deadline 2024-09-10T10:06:19.424367+0000) 2024-09-10T10:06:20.635 INFO:journalctl@ceph.osd.10.smithi102.stdout:Sep 10 10:06:20 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-10[59874]: 2024-09-10T10:06:20.484+0000 7f40f3335640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-10T10:05:49.575097+0000 front 2024-09-10T10:05:49.575141+0000 (oldest deadline 2024-09-10T10:06:13.075035+0000) 2024-09-10T10:06:20.635 INFO:journalctl@ceph.osd.10.smithi102.stdout:Sep 10 10:06:20 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-10[59874]: 2024-09-10T10:06:20.484+0000 7f40f3335640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-09-10T10:05:54.176094+0000 front 2024-09-10T10:05:54.176115+0000 (oldest deadline 2024-09-10T10:06:19.376783+0000) 2024-09-10T10:06:20.683 INFO:journalctl@ceph.osd.5.smithi159.stdout:Sep 10 10:06:20 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-5[46947]: 2024-09-10T10:06:20.409+0000 7f87f1ed7640 -1 osd.5 100 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-10T10:05:48.238525+0000 front 2024-09-10T10:05:48.238552+0000 (oldest deadline 2024-09-10T10:06:11.738538+0000) 2024-09-10T10:06:21.100 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 10 10:06:20 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-11[60437]: 2024-09-10T10:06:20.682+0000 7fa78e4d3640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-10T10:05:49.428950+0000 front 2024-09-10T10:05:49.429031+0000 (oldest deadline 2024-09-10T10:06:11.128800+0000) 2024-09-10T10:06:21.252 INFO:journalctl@ceph.osd.4.smithi102.stdout:Sep 10 10:06:20 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-4[46359]: 2024-09-10T10:06:20.934+0000 7f00447de640 -1 osd.4 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-09-10T10:05:56.199631+0000 front 2024-09-10T10:05:56.199661+0000 (oldest deadline 2024-09-10T10:06:19.099760+0000) 2024-09-10T10:06:21.556 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 10 10:06:21 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-9[65288]: 2024-09-10T10:06:21.200+0000 7f84af200640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-10T10:05:47.346189+0000 front 2024-09-10T10:05:47.346220+0000 (oldest deadline 2024-09-10T10:06:12.646451+0000) 2024-09-10T10:06:21.600 INFO:journalctl@ceph.osd.5.smithi159.stdout:Sep 10 10:06:21 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-5[46947]: 2024-09-10T10:06:21.374+0000 7f87f1ed7640 -1 osd.5 100 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-10T10:05:48.238525+0000 front 2024-09-10T10:05:48.238552+0000 (oldest deadline 2024-09-10T10:06:11.738538+0000) 2024-09-10T10:06:21.601 INFO:journalctl@ceph.osd.8.smithi159.stdout:Sep 10 10:06:21 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-8[53439]: 2024-09-10T10:06:21.331+0000 7f7417907640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-09-10T10:05:57.316774+0000 front 2024-09-10T10:05:57.316749+0000 (oldest deadline 2024-09-10T10:06:20.816694+0000) 2024-09-10T10:06:21.634 INFO:journalctl@ceph.osd.7.smithi102.stdout:Sep 10 10:06:21 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-7[52880]: 2024-09-10T10:06:21.251+0000 7ff941357640 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-09-10T10:05:55.423434+0000 front 2024-09-10T10:05:55.423389+0000 (oldest deadline 2024-09-10T10:06:19.424367+0000) 2024-09-10T10:06:21.635 INFO:journalctl@ceph.osd.10.smithi102.stdout:Sep 10 10:06:21 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-10[59874]: 2024-09-10T10:06:21.440+0000 7f40f3335640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-10T10:05:49.575097+0000 front 2024-09-10T10:05:49.575141+0000 (oldest deadline 2024-09-10T10:06:13.075035+0000) 2024-09-10T10:06:21.635 INFO:journalctl@ceph.osd.10.smithi102.stdout:Sep 10 10:06:21 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-10[59874]: 2024-09-10T10:06:21.440+0000 7f40f3335640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-09-10T10:05:54.176094+0000 front 2024-09-10T10:05:54.176115+0000 (oldest deadline 2024-09-10T10:06:19.376783+0000) 2024-09-10T10:06:22.100 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 10 10:06:21 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-11[60437]: 2024-09-10T10:06:21.657+0000 7fa78e4d3640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-10T10:05:49.428950+0000 front 2024-09-10T10:05:49.429031+0000 (oldest deadline 2024-09-10T10:06:11.128800+0000) 2024-09-10T10:06:22.283 INFO:journalctl@ceph.osd.4.smithi102.stdout:Sep 10 10:06:21 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-4[46359]: 2024-09-10T10:06:21.945+0000 7f00447de640 -1 osd.4 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-09-10T10:05:56.199631+0000 front 2024-09-10T10:05:56.199661+0000 (oldest deadline 2024-09-10T10:06:19.099760+0000) 2024-09-10T10:06:22.556 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 10 10:06:22 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-9[65288]: 2024-09-10T10:06:22.230+0000 7f84af200640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-10T10:05:47.346189+0000 front 2024-09-10T10:05:47.346220+0000 (oldest deadline 2024-09-10T10:06:12.646451+0000) 2024-09-10T10:06:22.600 INFO:journalctl@ceph.osd.8.smithi159.stdout:Sep 10 10:06:22 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-8[53439]: 2024-09-10T10:06:22.323+0000 7f7417907640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-09-10T10:05:57.316774+0000 front 2024-09-10T10:05:57.316749+0000 (oldest deadline 2024-09-10T10:06:20.816694+0000) 2024-09-10T10:06:22.600 INFO:journalctl@ceph.osd.5.smithi159.stdout:Sep 10 10:06:22 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-5[46947]: 2024-09-10T10:06:22.390+0000 7f87f1ed7640 -1 osd.5 100 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-10T10:05:48.238525+0000 front 2024-09-10T10:05:48.238552+0000 (oldest deadline 2024-09-10T10:06:11.738538+0000) 2024-09-10T10:06:22.634 INFO:journalctl@ceph.osd.7.smithi102.stdout:Sep 10 10:06:22 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-7[52880]: 2024-09-10T10:06:22.282+0000 7ff941357640 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-09-10T10:05:55.423434+0000 front 2024-09-10T10:05:55.423389+0000 (oldest deadline 2024-09-10T10:06:19.424367+0000) 2024-09-10T10:06:22.635 INFO:journalctl@ceph.osd.10.smithi102.stdout:Sep 10 10:06:22 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-10[59874]: 2024-09-10T10:06:22.456+0000 7f40f3335640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-10T10:05:49.575097+0000 front 2024-09-10T10:05:49.575141+0000 (oldest deadline 2024-09-10T10:06:13.075035+0000) 2024-09-10T10:06:22.635 INFO:journalctl@ceph.osd.10.smithi102.stdout:Sep 10 10:06:22 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-10[59874]: 2024-09-10T10:06:22.456+0000 7f40f3335640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-09-10T10:05:54.176094+0000 front 2024-09-10T10:05:54.176115+0000 (oldest deadline 2024-09-10T10:06:19.376783+0000) 2024-09-10T10:06:23.100 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 10 10:06:22 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-11[60437]: 2024-09-10T10:06:22.609+0000 7fa78e4d3640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-10T10:05:49.428950+0000 front 2024-09-10T10:05:49.429031+0000 (oldest deadline 2024-09-10T10:06:11.128800+0000) 2024-09-10T10:06:23.134 INFO:journalctl@ceph.osd.4.smithi102.stdout:Sep 10 10:06:22 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-4[46359]: 2024-09-10T10:06:22.913+0000 7f00447de640 -1 osd.4 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-09-10T10:05:56.199631+0000 front 2024-09-10T10:05:56.199661+0000 (oldest deadline 2024-09-10T10:06:19.099760+0000) 2024-09-10T10:06:23.556 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 10 10:06:23 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-9[65288]: 2024-09-10T10:06:23.266+0000 7f84af200640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-10T10:05:47.346189+0000 front 2024-09-10T10:05:47.346220+0000 (oldest deadline 2024-09-10T10:06:12.646451+0000) 2024-09-10T10:06:23.556 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 10 10:06:23 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-9[65288]: 2024-09-10T10:06:23.266+0000 7f84af200640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-09-10T10:05:57.348185+0000 front 2024-09-10T10:05:57.348335+0000 (oldest deadline 2024-09-10T10:06:23.248210+0000) 2024-09-10T10:06:23.600 INFO:journalctl@ceph.osd.8.smithi159.stdout:Sep 10 10:06:23 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-8[53439]: 2024-09-10T10:06:23.340+0000 7f7417907640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-09-10T10:05:57.316774+0000 front 2024-09-10T10:05:57.316749+0000 (oldest deadline 2024-09-10T10:06:20.816694+0000) 2024-09-10T10:06:23.601 INFO:journalctl@ceph.osd.5.smithi159.stdout:Sep 10 10:06:23 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-5[46947]: 2024-09-10T10:06:23.406+0000 7f87f1ed7640 -1 osd.5 100 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-10T10:05:48.238525+0000 front 2024-09-10T10:05:48.238552+0000 (oldest deadline 2024-09-10T10:06:11.738538+0000) 2024-09-10T10:06:23.601 INFO:journalctl@ceph.osd.5.smithi159.stdout:Sep 10 10:06:23 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-5[46947]: 2024-09-10T10:06:23.406+0000 7f87f1ed7640 -1 osd.5 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-09-10T10:05:58.041061+0000 front 2024-09-10T10:05:58.041082+0000 (oldest deadline 2024-09-10T10:06:22.741042+0000) 2024-09-10T10:06:23.613 INFO:journalctl@ceph.osd.7.smithi102.stdout:Sep 10 10:06:23 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-7[52880]: 2024-09-10T10:06:23.267+0000 7ff941357640 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-09-10T10:05:55.423434+0000 front 2024-09-10T10:05:55.423389+0000 (oldest deadline 2024-09-10T10:06:19.424367+0000) 2024-09-10T10:06:23.614 INFO:journalctl@ceph.osd.10.smithi102.stdout:Sep 10 10:06:23 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-10[59874]: 2024-09-10T10:06:23.503+0000 7f40f3335640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-10T10:05:49.575097+0000 front 2024-09-10T10:05:49.575141+0000 (oldest deadline 2024-09-10T10:06:13.075035+0000) 2024-09-10T10:06:23.614 INFO:journalctl@ceph.osd.10.smithi102.stdout:Sep 10 10:06:23 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-10[59874]: 2024-09-10T10:06:23.503+0000 7f40f3335640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-09-10T10:05:54.176094+0000 front 2024-09-10T10:05:54.176115+0000 (oldest deadline 2024-09-10T10:06:19.376783+0000) 2024-09-10T10:06:23.885 INFO:journalctl@ceph.osd.4.smithi102.stdout:Sep 10 10:06:23 smithi102 podman[75640]: 2024-09-10 10:06:23.613120697 +0000 UTC m=+5.546098317 container died e39fd048c40172371094d745e324c8d3d448e61f13dff0610038a482310bb966 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-4, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True) 2024-09-10T10:06:24.100 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 10 10:06:23 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-11[60437]: 2024-09-10T10:06:23.644+0000 7fa78e4d3640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-10T10:05:49.428950+0000 front 2024-09-10T10:05:49.429031+0000 (oldest deadline 2024-09-10T10:06:11.128800+0000) 2024-09-10T10:06:24.224 INFO:journalctl@ceph.osd.4.smithi102.stdout:Sep 10 10:06:24 smithi102 podman[75640]: 2024-09-10 10:06:24.00718098 +0000 UTC m=+5.940158588 container cleanup e39fd048c40172371094d745e324c8d3d448e61f13dff0610038a482310bb966 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-4, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS) 2024-09-10T10:06:24.224 INFO:journalctl@ceph.osd.4.smithi102.stdout:Sep 10 10:06:24 smithi102 bash[75640]: ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-4 2024-09-10T10:06:24.539 INFO:journalctl@ceph.osd.7.smithi102.stdout:Sep 10 10:06:24 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-7[52880]: 2024-09-10T10:06:24.223+0000 7ff941357640 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-09-10T10:05:55.423434+0000 front 2024-09-10T10:05:55.423389+0000 (oldest deadline 2024-09-10T10:06:19.424367+0000) 2024-09-10T10:06:24.540 INFO:journalctl@ceph.osd.4.smithi102.stdout:Sep 10 10:06:24 smithi102 podman[75656]: 2024-09-10 10:06:24.275260256 +0000 UTC m=+0.663567814 container remove e39fd048c40172371094d745e324c8d3d448e61f13dff0610038a482310bb966 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-4, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-10T10:06:24.556 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 10 10:06:24 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-9[65288]: 2024-09-10T10:06:24.263+0000 7f84af200640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-10T10:05:47.346189+0000 front 2024-09-10T10:05:47.346220+0000 (oldest deadline 2024-09-10T10:06:12.646451+0000) 2024-09-10T10:06:24.556 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 10 10:06:24 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-9[65288]: 2024-09-10T10:06:24.263+0000 7f84af200640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-09-10T10:05:57.348185+0000 front 2024-09-10T10:05:57.348335+0000 (oldest deadline 2024-09-10T10:06:23.248210+0000) 2024-09-10T10:06:24.690 INFO:journalctl@ceph.osd.8.smithi159.stdout:Sep 10 10:06:24 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-8[53439]: 2024-09-10T10:06:24.365+0000 7f7417907640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-09-10T10:05:57.316774+0000 front 2024-09-10T10:05:57.316749+0000 (oldest deadline 2024-09-10T10:06:20.816694+0000) 2024-09-10T10:06:24.690 INFO:journalctl@ceph.osd.5.smithi159.stdout:Sep 10 10:06:24 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-5[46947]: 2024-09-10T10:06:24.422+0000 7f87f1ed7640 -1 osd.5 100 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-10T10:05:48.238525+0000 front 2024-09-10T10:05:48.238552+0000 (oldest deadline 2024-09-10T10:06:11.738538+0000) 2024-09-10T10:06:24.690 INFO:journalctl@ceph.osd.5.smithi159.stdout:Sep 10 10:06:24 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-5[46947]: 2024-09-10T10:06:24.422+0000 7f87f1ed7640 -1 osd.5 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-09-10T10:05:58.041061+0000 front 2024-09-10T10:05:58.041082+0000 (oldest deadline 2024-09-10T10:06:22.741042+0000) 2024-09-10T10:06:24.884 INFO:journalctl@ceph.osd.4.smithi102.stdout:Sep 10 10:06:24 smithi102 podman[75749]: 2024-09-10 10:06:24.487681036 +0000 UTC m=+0.015558219 image pull 41d67112835771c3efca60c6b8fe297c7ddaecdc26cea26fd28aa18636e49005 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 2024-09-10T10:06:24.885 INFO:journalctl@ceph.osd.4.smithi102.stdout:Sep 10 10:06:24 smithi102 podman[75749]: 2024-09-10 10:06:24.617830419 +0000 UTC m=+0.145707587 container create 52e68271c1bf29af85b8b66f78bd2ca763f5781308e7fb64b127c53bfd2ad1e5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-4-deactivate, io.buildah.version=1.37.2, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-10T10:06:24.885 INFO:journalctl@ceph.osd.10.smithi102.stdout:Sep 10 10:06:24 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-10[59874]: 2024-09-10T10:06:24.538+0000 7f40f3335640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-10T10:05:49.575097+0000 front 2024-09-10T10:05:49.575141+0000 (oldest deadline 2024-09-10T10:06:13.075035+0000) 2024-09-10T10:06:24.885 INFO:journalctl@ceph.osd.10.smithi102.stdout:Sep 10 10:06:24 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-10[59874]: 2024-09-10T10:06:24.538+0000 7f40f3335640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-09-10T10:05:54.176094+0000 front 2024-09-10T10:05:54.176115+0000 (oldest deadline 2024-09-10T10:06:19.376783+0000) 2024-09-10T10:06:25.056 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 10 10:06:24 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-6[58305]: 2024-09-10T10:06:24.584+0000 7f7322400640 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-09-10T10:05:56.623525+0000 front 2024-09-10T10:05:56.623092+0000 (oldest deadline 2024-09-10T10:06:24.224362+0000) 2024-09-10T10:06:25.100 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 10 10:06:24 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-11[60437]: 2024-09-10T10:06:24.689+0000 7fa78e4d3640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-10T10:05:49.428950+0000 front 2024-09-10T10:05:49.429031+0000 (oldest deadline 2024-09-10T10:06:11.128800+0000) 2024-09-10T10:06:25.221 INFO:journalctl@ceph.osd.4.smithi102.stdout:Sep 10 10:06:24 smithi102 podman[75749]: 2024-09-10 10:06:24.884587375 +0000 UTC m=+0.412464558 container init 52e68271c1bf29af85b8b66f78bd2ca763f5781308e7fb64b127c53bfd2ad1e5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-4-deactivate, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, io.buildah.version=1.37.2) 2024-09-10T10:06:25.221 INFO:journalctl@ceph.osd.4.smithi102.stdout:Sep 10 10:06:24 smithi102 podman[75749]: 2024-09-10 10:06:24.887865792 +0000 UTC m=+0.415742963 container start 52e68271c1bf29af85b8b66f78bd2ca763f5781308e7fb64b127c53bfd2ad1e5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-4-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, CEPH_POINT_RELEASE=) 2024-09-10T10:06:25.221 INFO:journalctl@ceph.osd.4.smithi102.stdout:Sep 10 10:06:24 smithi102 podman[75749]: 2024-09-10 10:06:24.995839834 +0000 UTC m=+0.523717003 container attach 52e68271c1bf29af85b8b66f78bd2ca763f5781308e7fb64b127c53bfd2ad1e5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-4-deactivate, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , ceph=True, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240731) 2024-09-10T10:06:25.222 INFO:journalctl@ceph.osd.4.smithi102.stdout:Sep 10 10:06:25 smithi102 podman[75749]: 2024-09-10 10:06:25.078559654 +0000 UTC m=+0.606436836 container died 52e68271c1bf29af85b8b66f78bd2ca763f5781308e7fb64b127c53bfd2ad1e5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-4-deactivate, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240731, GIT_CLEAN=True) 2024-09-10T10:06:25.507 INFO:journalctl@ceph.osd.7.smithi102.stdout:Sep 10 10:06:25 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-7[52880]: 2024-09-10T10:06:25.219+0000 7ff941357640 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-09-10T10:05:55.423434+0000 front 2024-09-10T10:05:55.423389+0000 (oldest deadline 2024-09-10T10:06:19.424367+0000) 2024-09-10T10:06:25.544 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 10 10:06:25 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-9[65288]: 2024-09-10T10:06:25.275+0000 7f84af200640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-10T10:05:47.346189+0000 front 2024-09-10T10:05:47.346220+0000 (oldest deadline 2024-09-10T10:06:12.646451+0000) 2024-09-10T10:06:25.544 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 10 10:06:25 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-9[65288]: 2024-09-10T10:06:25.275+0000 7f84af200640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-09-10T10:05:57.348185+0000 front 2024-09-10T10:05:57.348335+0000 (oldest deadline 2024-09-10T10:06:23.248210+0000) 2024-09-10T10:06:25.696 INFO:journalctl@ceph.osd.5.smithi159.stdout:Sep 10 10:06:25 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-5[46947]: 2024-09-10T10:06:25.469+0000 7f87f1ed7640 -1 osd.5 100 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-10T10:05:48.238525+0000 front 2024-09-10T10:05:48.238552+0000 (oldest deadline 2024-09-10T10:06:11.738538+0000) 2024-09-10T10:06:25.696 INFO:journalctl@ceph.osd.5.smithi159.stdout:Sep 10 10:06:25 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-5[46947]: 2024-09-10T10:06:25.469+0000 7f87f1ed7640 -1 osd.5 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-09-10T10:05:58.041061+0000 front 2024-09-10T10:05:58.041082+0000 (oldest deadline 2024-09-10T10:06:22.741042+0000) 2024-09-10T10:06:25.697 INFO:journalctl@ceph.osd.8.smithi159.stdout:Sep 10 10:06:25 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-8[53439]: 2024-09-10T10:06:25.370+0000 7f7417907640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-09-10T10:05:57.316774+0000 front 2024-09-10T10:05:57.316749+0000 (oldest deadline 2024-09-10T10:06:20.816694+0000) 2024-09-10T10:06:25.768 INFO:journalctl@ceph.osd.4.smithi102.stdout:Sep 10 10:06:25 smithi102 podman[75772]: 2024-09-10 10:06:25.74988229 +0000 UTC m=+0.665167275 container remove 52e68271c1bf29af85b8b66f78bd2ca763f5781308e7fb64b127c53bfd2ad1e5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-4-deactivate, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-09-10T10:06:25.768 INFO:journalctl@ceph.osd.10.smithi102.stdout:Sep 10 10:06:25 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-10[59874]: 2024-09-10T10:06:25.506+0000 7f40f3335640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-10T10:05:49.575097+0000 front 2024-09-10T10:05:49.575141+0000 (oldest deadline 2024-09-10T10:06:13.075035+0000) 2024-09-10T10:06:25.768 INFO:journalctl@ceph.osd.10.smithi102.stdout:Sep 10 10:06:25 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-10[59874]: 2024-09-10T10:06:25.506+0000 7f40f3335640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-09-10T10:05:54.176094+0000 front 2024-09-10T10:05:54.176115+0000 (oldest deadline 2024-09-10T10:06:19.376783+0000) 2024-09-10T10:06:25.774 DEBUG:teuthology.orchestra.run.smithi102:> sudo pkill -f 'journalctl -f -n 0 -u ceph-bade4b50-6f56-11ef-bcea-c7b262605968@osd.4.service' 2024-09-10T10:06:25.808 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 10 10:06:25 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-6[58305]: 2024-09-10T10:06:25.543+0000 7f7322400640 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-09-10T10:05:56.623525+0000 front 2024-09-10T10:05:56.623092+0000 (oldest deadline 2024-09-10T10:06:24.224362+0000) 2024-09-10T10:06:25.856 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T10:06:25.856 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-09-10T10:06:25.856 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-09-10T10:06:25.856 DEBUG:teuthology.orchestra.run.smithi159:> sudo systemctl stop ceph-bade4b50-6f56-11ef-bcea-c7b262605968@osd.5 2024-09-10T10:06:26.100 INFO:journalctl@ceph.osd.5.smithi159.stdout:Sep 10 10:06:25 smithi159 systemd[1]: Stopping Ceph osd.5 for bade4b50-6f56-11ef-bcea-c7b262605968... 2024-09-10T10:06:26.101 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 10 10:06:25 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-11[60437]: 2024-09-10T10:06:25.695+0000 7fa78e4d3640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-10T10:05:49.428950+0000 front 2024-09-10T10:05:49.429031+0000 (oldest deadline 2024-09-10T10:06:11.128800+0000) 2024-09-10T10:06:26.497 INFO:journalctl@ceph.osd.8.smithi159.stdout:Sep 10 10:06:26 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-8[53439]: 2024-09-10T10:06:26.322+0000 7f7417907640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-09-10T10:05:57.316774+0000 front 2024-09-10T10:05:57.316749+0000 (oldest deadline 2024-09-10T10:06:20.816694+0000) 2024-09-10T10:06:26.497 INFO:journalctl@ceph.osd.5.smithi159.stdout:Sep 10 10:06:26 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-5[46947]: 2024-09-10T10:06:26.176+0000 7f87f58c1640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T10:06:26.497 INFO:journalctl@ceph.osd.5.smithi159.stdout:Sep 10 10:06:26 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-5[46947]: 2024-09-10T10:06:26.176+0000 7f87f58c1640 -1 osd.5 100 *** Got signal Terminated *** 2024-09-10T10:06:26.497 INFO:journalctl@ceph.osd.5.smithi159.stdout:Sep 10 10:06:26 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-5[46947]: 2024-09-10T10:06:26.176+0000 7f87f58c1640 -1 osd.5 100 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-10T10:06:26.533 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 10 10:06:26 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-9[65288]: 2024-09-10T10:06:26.237+0000 7f84af200640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-10T10:05:47.346189+0000 front 2024-09-10T10:05:47.346220+0000 (oldest deadline 2024-09-10T10:06:12.646451+0000) 2024-09-10T10:06:26.533 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 10 10:06:26 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-9[65288]: 2024-09-10T10:06:26.237+0000 7f84af200640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-09-10T10:05:57.348185+0000 front 2024-09-10T10:05:57.348335+0000 (oldest deadline 2024-09-10T10:06:23.248210+0000) 2024-09-10T10:06:26.551 INFO:journalctl@ceph.osd.7.smithi102.stdout:Sep 10 10:06:26 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-7[52880]: 2024-09-10T10:06:26.195+0000 7ff941357640 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-09-10T10:05:55.423434+0000 front 2024-09-10T10:05:55.423389+0000 (oldest deadline 2024-09-10T10:06:19.424367+0000) 2024-09-10T10:06:26.806 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 10 10:06:26 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-6[58305]: 2024-09-10T10:06:26.532+0000 7f7322400640 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-09-10T10:05:56.623525+0000 front 2024-09-10T10:05:56.623092+0000 (oldest deadline 2024-09-10T10:06:24.224362+0000) 2024-09-10T10:06:26.850 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 10 10:06:26 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-11[60437]: 2024-09-10T10:06:26.731+0000 7fa78e4d3640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-10T10:05:49.428950+0000 front 2024-09-10T10:05:49.429031+0000 (oldest deadline 2024-09-10T10:06:11.128800+0000) 2024-09-10T10:06:26.850 INFO:journalctl@ceph.osd.5.smithi159.stdout:Sep 10 10:06:26 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-5[46947]: 2024-09-10T10:06:26.496+0000 7f87f1ed7640 -1 osd.5 100 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-10T10:05:48.238525+0000 front 2024-09-10T10:05:48.238552+0000 (oldest deadline 2024-09-10T10:06:11.738538+0000) 2024-09-10T10:06:26.851 INFO:journalctl@ceph.osd.5.smithi159.stdout:Sep 10 10:06:26 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-5[46947]: 2024-09-10T10:06:26.496+0000 7f87f1ed7640 -1 osd.5 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-09-10T10:05:58.041061+0000 front 2024-09-10T10:05:58.041082+0000 (oldest deadline 2024-09-10T10:06:22.741042+0000) 2024-09-10T10:06:26.884 INFO:journalctl@ceph.osd.10.smithi102.stdout:Sep 10 10:06:26 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-10[59874]: 2024-09-10T10:06:26.550+0000 7f40f3335640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-10T10:05:49.575097+0000 front 2024-09-10T10:05:49.575141+0000 (oldest deadline 2024-09-10T10:06:13.075035+0000) 2024-09-10T10:06:26.885 INFO:journalctl@ceph.osd.10.smithi102.stdout:Sep 10 10:06:26 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-10[59874]: 2024-09-10T10:06:26.550+0000 7f40f3335640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-09-10T10:05:54.176094+0000 front 2024-09-10T10:05:54.176115+0000 (oldest deadline 2024-09-10T10:06:19.376783+0000) 2024-09-10T10:06:27.526 INFO:journalctl@ceph.osd.7.smithi102.stdout:Sep 10 10:06:27 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-7[52880]: 2024-09-10T10:06:27.154+0000 7ff941357640 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-09-10T10:05:55.423434+0000 front 2024-09-10T10:05:55.423389+0000 (oldest deadline 2024-09-10T10:06:19.424367+0000) 2024-09-10T10:06:27.536 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 10 10:06:27 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-9[65288]: 2024-09-10T10:06:27.224+0000 7f84af200640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-10T10:05:47.346189+0000 front 2024-09-10T10:05:47.346220+0000 (oldest deadline 2024-09-10T10:06:12.646451+0000) 2024-09-10T10:06:27.536 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 10 10:06:27 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-9[65288]: 2024-09-10T10:06:27.224+0000 7f84af200640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-09-10T10:05:57.348185+0000 front 2024-09-10T10:05:57.348335+0000 (oldest deadline 2024-09-10T10:06:23.248210+0000) 2024-09-10T10:06:27.536 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 10 10:06:27 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-9[65288]: 2024-09-10T10:06:27.224+0000 7f84af200640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-10T10:06:03.249484+0000 front 2024-09-10T10:06:03.249185+0000 (oldest deadline 2024-09-10T10:06:26.749283+0000) 2024-09-10T10:06:27.696 INFO:journalctl@ceph.osd.8.smithi159.stdout:Sep 10 10:06:27 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-8[53439]: 2024-09-10T10:06:27.351+0000 7f7417907640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-09-10T10:05:57.316774+0000 front 2024-09-10T10:05:57.316749+0000 (oldest deadline 2024-09-10T10:06:20.816694+0000) 2024-09-10T10:06:27.696 INFO:journalctl@ceph.osd.5.smithi159.stdout:Sep 10 10:06:27 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-5[46947]: 2024-09-10T10:06:27.464+0000 7f87f1ed7640 -1 osd.5 100 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-10T10:05:48.238525+0000 front 2024-09-10T10:05:48.238552+0000 (oldest deadline 2024-09-10T10:06:11.738538+0000) 2024-09-10T10:06:27.696 INFO:journalctl@ceph.osd.5.smithi159.stdout:Sep 10 10:06:27 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-5[46947]: 2024-09-10T10:06:27.464+0000 7f87f1ed7640 -1 osd.5 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-09-10T10:05:58.041061+0000 front 2024-09-10T10:05:58.041082+0000 (oldest deadline 2024-09-10T10:06:22.741042+0000) 2024-09-10T10:06:27.806 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 10 10:06:27 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-6[58305]: 2024-09-10T10:06:27.535+0000 7f7322400640 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-09-10T10:05:56.623525+0000 front 2024-09-10T10:05:56.623092+0000 (oldest deadline 2024-09-10T10:06:24.224362+0000) 2024-09-10T10:06:27.884 INFO:journalctl@ceph.osd.10.smithi102.stdout:Sep 10 10:06:27 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-10[59874]: 2024-09-10T10:06:27.525+0000 7f40f3335640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-10T10:05:49.575097+0000 front 2024-09-10T10:05:49.575141+0000 (oldest deadline 2024-09-10T10:06:13.075035+0000) 2024-09-10T10:06:27.884 INFO:journalctl@ceph.osd.10.smithi102.stdout:Sep 10 10:06:27 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-10[59874]: 2024-09-10T10:06:27.525+0000 7f40f3335640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-09-10T10:05:54.176094+0000 front 2024-09-10T10:05:54.176115+0000 (oldest deadline 2024-09-10T10:06:19.376783+0000) 2024-09-10T10:06:28.100 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 10 10:06:27 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-11[60437]: 2024-09-10T10:06:27.694+0000 7fa78e4d3640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-10T10:05:49.428950+0000 front 2024-09-10T10:05:49.429031+0000 (oldest deadline 2024-09-10T10:06:11.128800+0000) 2024-09-10T10:06:28.384 INFO:journalctl@ceph.osd.7.smithi102.stdout:Sep 10 10:06:28 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-7[52880]: 2024-09-10T10:06:28.124+0000 7ff941357640 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-09-10T10:05:55.423434+0000 front 2024-09-10T10:05:55.423389+0000 (oldest deadline 2024-09-10T10:06:19.424367+0000) 2024-09-10T10:06:28.496 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 10 10:06:28 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-9[65288]: 2024-09-10T10:06:28.216+0000 7f84af200640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-10T10:05:47.346189+0000 front 2024-09-10T10:05:47.346220+0000 (oldest deadline 2024-09-10T10:06:12.646451+0000) 2024-09-10T10:06:28.496 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 10 10:06:28 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-9[65288]: 2024-09-10T10:06:28.216+0000 7f84af200640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-09-10T10:05:57.348185+0000 front 2024-09-10T10:05:57.348335+0000 (oldest deadline 2024-09-10T10:06:23.248210+0000) 2024-09-10T10:06:28.496 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 10 10:06:28 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-9[65288]: 2024-09-10T10:06:28.216+0000 7f84af200640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-10T10:06:03.249484+0000 front 2024-09-10T10:06:03.249185+0000 (oldest deadline 2024-09-10T10:06:26.749283+0000) 2024-09-10T10:06:28.600 INFO:journalctl@ceph.osd.5.smithi159.stdout:Sep 10 10:06:28 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-5[46947]: 2024-09-10T10:06:28.494+0000 7f87f1ed7640 -1 osd.5 100 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-10T10:05:48.238525+0000 front 2024-09-10T10:05:48.238552+0000 (oldest deadline 2024-09-10T10:06:11.738538+0000) 2024-09-10T10:06:28.600 INFO:journalctl@ceph.osd.5.smithi159.stdout:Sep 10 10:06:28 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-5[46947]: 2024-09-10T10:06:28.494+0000 7f87f1ed7640 -1 osd.5 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-09-10T10:05:58.041061+0000 front 2024-09-10T10:05:58.041082+0000 (oldest deadline 2024-09-10T10:06:22.741042+0000) 2024-09-10T10:06:28.600 INFO:journalctl@ceph.osd.8.smithi159.stdout:Sep 10 10:06:28 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-8[53439]: 2024-09-10T10:06:28.330+0000 7f7417907640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-09-10T10:05:57.316774+0000 front 2024-09-10T10:05:57.316749+0000 (oldest deadline 2024-09-10T10:06:20.816694+0000) 2024-09-10T10:06:28.601 INFO:journalctl@ceph.osd.8.smithi159.stdout:Sep 10 10:06:28 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-8[53439]: 2024-09-10T10:06:28.330+0000 7f7417907640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-10T10:06:04.917990+0000 front 2024-09-10T10:06:04.918013+0000 (oldest deadline 2024-09-10T10:06:27.818198+0000) 2024-09-10T10:06:28.806 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 10 10:06:28 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-6[58305]: 2024-09-10T10:06:28.495+0000 7f7322400640 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-09-10T10:05:56.623525+0000 front 2024-09-10T10:05:56.623092+0000 (oldest deadline 2024-09-10T10:06:24.224362+0000) 2024-09-10T10:06:28.806 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 10 10:06:28 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-6[58305]: 2024-09-10T10:06:28.495+0000 7f7322400640 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-10T10:06:04.225156+0000 front 2024-09-10T10:06:04.225182+0000 (oldest deadline 2024-09-10T10:06:27.725387+0000) 2024-09-10T10:06:28.884 INFO:journalctl@ceph.osd.10.smithi102.stdout:Sep 10 10:06:28 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-10[59874]: 2024-09-10T10:06:28.571+0000 7f40f3335640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-10T10:05:49.575097+0000 front 2024-09-10T10:05:49.575141+0000 (oldest deadline 2024-09-10T10:06:13.075035+0000) 2024-09-10T10:06:28.884 INFO:journalctl@ceph.osd.10.smithi102.stdout:Sep 10 10:06:28 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-10[59874]: 2024-09-10T10:06:28.571+0000 7f40f3335640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-09-10T10:05:54.176094+0000 front 2024-09-10T10:05:54.176115+0000 (oldest deadline 2024-09-10T10:06:19.376783+0000) 2024-09-10T10:06:29.100 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 10 10:06:28 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-11[60437]: 2024-09-10T10:06:28.683+0000 7fa78e4d3640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-10T10:05:49.428950+0000 front 2024-09-10T10:05:49.429031+0000 (oldest deadline 2024-09-10T10:06:11.128800+0000) 2024-09-10T10:06:29.519 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 10 10:06:29 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-9[65288]: 2024-09-10T10:06:29.176+0000 7f84af200640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-10T10:05:47.346189+0000 front 2024-09-10T10:05:47.346220+0000 (oldest deadline 2024-09-10T10:06:12.646451+0000) 2024-09-10T10:06:29.519 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 10 10:06:29 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-9[65288]: 2024-09-10T10:06:29.176+0000 7f84af200640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-09-10T10:05:57.348185+0000 front 2024-09-10T10:05:57.348335+0000 (oldest deadline 2024-09-10T10:06:23.248210+0000) 2024-09-10T10:06:29.519 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 10 10:06:29 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-9[65288]: 2024-09-10T10:06:29.176+0000 7f84af200640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-10T10:06:03.249484+0000 front 2024-09-10T10:06:03.249185+0000 (oldest deadline 2024-09-10T10:06:26.749283+0000) 2024-09-10T10:06:29.547 INFO:journalctl@ceph.osd.7.smithi102.stdout:Sep 10 10:06:29 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-7[52880]: 2024-09-10T10:06:29.160+0000 7ff941357640 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-09-10T10:05:55.423434+0000 front 2024-09-10T10:05:55.423389+0000 (oldest deadline 2024-09-10T10:06:19.424367+0000) 2024-09-10T10:06:29.547 INFO:journalctl@ceph.osd.7.smithi102.stdout:Sep 10 10:06:29 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-7[52880]: 2024-09-10T10:06:29.160+0000 7ff941357640 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-10T10:06:04.025965+0000 front 2024-09-10T10:06:04.025864+0000 (oldest deadline 2024-09-10T10:06:28.726043+0000) 2024-09-10T10:06:29.689 INFO:journalctl@ceph.osd.8.smithi159.stdout:Sep 10 10:06:29 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-8[53439]: 2024-09-10T10:06:29.368+0000 7f7417907640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-09-10T10:05:57.316774+0000 front 2024-09-10T10:05:57.316749+0000 (oldest deadline 2024-09-10T10:06:20.816694+0000) 2024-09-10T10:06:29.689 INFO:journalctl@ceph.osd.8.smithi159.stdout:Sep 10 10:06:29 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-8[53439]: 2024-09-10T10:06:29.368+0000 7f7417907640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-10T10:06:04.917990+0000 front 2024-09-10T10:06:04.918013+0000 (oldest deadline 2024-09-10T10:06:27.818198+0000) 2024-09-10T10:06:29.690 INFO:journalctl@ceph.osd.5.smithi159.stdout:Sep 10 10:06:29 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-5[46947]: 2024-09-10T10:06:29.471+0000 7f87f1ed7640 -1 osd.5 100 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-10T10:05:48.238525+0000 front 2024-09-10T10:05:48.238552+0000 (oldest deadline 2024-09-10T10:06:11.738538+0000) 2024-09-10T10:06:29.690 INFO:journalctl@ceph.osd.5.smithi159.stdout:Sep 10 10:06:29 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-5[46947]: 2024-09-10T10:06:29.471+0000 7f87f1ed7640 -1 osd.5 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-09-10T10:05:58.041061+0000 front 2024-09-10T10:05:58.041082+0000 (oldest deadline 2024-09-10T10:06:22.741042+0000) 2024-09-10T10:06:29.806 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 10 10:06:29 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-6[58305]: 2024-09-10T10:06:29.517+0000 7f7322400640 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-09-10T10:05:56.623525+0000 front 2024-09-10T10:05:56.623092+0000 (oldest deadline 2024-09-10T10:06:24.224362+0000) 2024-09-10T10:06:29.806 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 10 10:06:29 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-6[58305]: 2024-09-10T10:06:29.517+0000 7f7322400640 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-10T10:06:04.225156+0000 front 2024-09-10T10:06:04.225182+0000 (oldest deadline 2024-09-10T10:06:27.725387+0000) 2024-09-10T10:06:29.884 INFO:journalctl@ceph.osd.10.smithi102.stdout:Sep 10 10:06:29 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-10[59874]: 2024-09-10T10:06:29.546+0000 7f40f3335640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-10T10:05:49.575097+0000 front 2024-09-10T10:05:49.575141+0000 (oldest deadline 2024-09-10T10:06:13.075035+0000) 2024-09-10T10:06:29.885 INFO:journalctl@ceph.osd.10.smithi102.stdout:Sep 10 10:06:29 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-10[59874]: 2024-09-10T10:06:29.546+0000 7f40f3335640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-09-10T10:05:54.176094+0000 front 2024-09-10T10:05:54.176115+0000 (oldest deadline 2024-09-10T10:06:19.376783+0000) 2024-09-10T10:06:30.100 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 10 10:06:29 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-11[60437]: 2024-09-10T10:06:29.689+0000 7fa78e4d3640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-10T10:05:49.428950+0000 front 2024-09-10T10:05:49.429031+0000 (oldest deadline 2024-09-10T10:06:11.128800+0000) 2024-09-10T10:06:30.100 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 10 10:06:29 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-11[60437]: 2024-09-10T10:06:29.689+0000 7fa78e4d3640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-10T10:06:03.430910+0000 front 2024-09-10T10:06:03.430889+0000 (oldest deadline 2024-09-10T10:06:29.331091+0000) 2024-09-10T10:06:30.503 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 10 10:06:30 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-9[65288]: 2024-09-10T10:06:30.159+0000 7f84af200640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-10T10:05:47.346189+0000 front 2024-09-10T10:05:47.346220+0000 (oldest deadline 2024-09-10T10:06:12.646451+0000) 2024-09-10T10:06:30.503 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 10 10:06:30 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-9[65288]: 2024-09-10T10:06:30.159+0000 7f84af200640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-09-10T10:05:57.348185+0000 front 2024-09-10T10:05:57.348335+0000 (oldest deadline 2024-09-10T10:06:23.248210+0000) 2024-09-10T10:06:30.504 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 10 10:06:30 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-9[65288]: 2024-09-10T10:06:30.159+0000 7f84af200640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-10T10:06:03.249484+0000 front 2024-09-10T10:06:03.249185+0000 (oldest deadline 2024-09-10T10:06:26.749283+0000) 2024-09-10T10:06:30.593 INFO:journalctl@ceph.osd.7.smithi102.stdout:Sep 10 10:06:30 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-7[52880]: 2024-09-10T10:06:30.180+0000 7ff941357640 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-09-10T10:05:55.423434+0000 front 2024-09-10T10:05:55.423389+0000 (oldest deadline 2024-09-10T10:06:19.424367+0000) 2024-09-10T10:06:30.594 INFO:journalctl@ceph.osd.7.smithi102.stdout:Sep 10 10:06:30 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-7[52880]: 2024-09-10T10:06:30.180+0000 7ff941357640 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-10T10:06:04.025965+0000 front 2024-09-10T10:06:04.025864+0000 (oldest deadline 2024-09-10T10:06:28.726043+0000) 2024-09-10T10:06:30.686 INFO:journalctl@ceph.osd.8.smithi159.stdout:Sep 10 10:06:30 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-8[53439]: 2024-09-10T10:06:30.350+0000 7f7417907640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-09-10T10:05:57.316774+0000 front 2024-09-10T10:05:57.316749+0000 (oldest deadline 2024-09-10T10:06:20.816694+0000) 2024-09-10T10:06:30.686 INFO:journalctl@ceph.osd.8.smithi159.stdout:Sep 10 10:06:30 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-8[53439]: 2024-09-10T10:06:30.350+0000 7f7417907640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-10T10:06:04.917990+0000 front 2024-09-10T10:06:04.918013+0000 (oldest deadline 2024-09-10T10:06:27.818198+0000) 2024-09-10T10:06:30.686 INFO:journalctl@ceph.osd.5.smithi159.stdout:Sep 10 10:06:30 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-5[46947]: 2024-09-10T10:06:30.445+0000 7f87f1ed7640 -1 osd.5 100 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-10T10:05:48.238525+0000 front 2024-09-10T10:05:48.238552+0000 (oldest deadline 2024-09-10T10:06:11.738538+0000) 2024-09-10T10:06:30.686 INFO:journalctl@ceph.osd.5.smithi159.stdout:Sep 10 10:06:30 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-5[46947]: 2024-09-10T10:06:30.445+0000 7f87f1ed7640 -1 osd.5 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-09-10T10:05:58.041061+0000 front 2024-09-10T10:05:58.041082+0000 (oldest deadline 2024-09-10T10:06:22.741042+0000) 2024-09-10T10:06:30.686 INFO:journalctl@ceph.osd.5.smithi159.stdout:Sep 10 10:06:30 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-5[46947]: 2024-09-10T10:06:30.445+0000 7f87f1ed7640 -1 osd.5 100 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-10T10:06:04.942478+0000 front 2024-09-10T10:06:04.942499+0000 (oldest deadline 2024-09-10T10:06:29.642684+0000) 2024-09-10T10:06:30.806 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 10 10:06:30 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-6[58305]: 2024-09-10T10:06:30.502+0000 7f7322400640 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-09-10T10:05:56.623525+0000 front 2024-09-10T10:05:56.623092+0000 (oldest deadline 2024-09-10T10:06:24.224362+0000) 2024-09-10T10:06:30.806 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 10 10:06:30 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-6[58305]: 2024-09-10T10:06:30.502+0000 7f7322400640 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-10T10:06:04.225156+0000 front 2024-09-10T10:06:04.225182+0000 (oldest deadline 2024-09-10T10:06:27.725387+0000) 2024-09-10T10:06:30.884 INFO:journalctl@ceph.osd.10.smithi102.stdout:Sep 10 10:06:30 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-10[59874]: 2024-09-10T10:06:30.592+0000 7f40f3335640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-10T10:05:49.575097+0000 front 2024-09-10T10:05:49.575141+0000 (oldest deadline 2024-09-10T10:06:13.075035+0000) 2024-09-10T10:06:30.884 INFO:journalctl@ceph.osd.10.smithi102.stdout:Sep 10 10:06:30 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-10[59874]: 2024-09-10T10:06:30.592+0000 7f40f3335640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-09-10T10:05:54.176094+0000 front 2024-09-10T10:05:54.176115+0000 (oldest deadline 2024-09-10T10:06:19.376783+0000) 2024-09-10T10:06:31.100 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 10 10:06:30 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-11[60437]: 2024-09-10T10:06:30.684+0000 7fa78e4d3640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-10T10:05:49.428950+0000 front 2024-09-10T10:05:49.429031+0000 (oldest deadline 2024-09-10T10:06:11.128800+0000) 2024-09-10T10:06:31.100 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 10 10:06:30 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-11[60437]: 2024-09-10T10:06:30.684+0000 7fa78e4d3640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-10T10:06:03.430910+0000 front 2024-09-10T10:06:03.430889+0000 (oldest deadline 2024-09-10T10:06:29.331091+0000) 2024-09-10T10:06:31.502 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 10 10:06:31 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-9[65288]: 2024-09-10T10:06:31.135+0000 7f84af200640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-10T10:05:47.346189+0000 front 2024-09-10T10:05:47.346220+0000 (oldest deadline 2024-09-10T10:06:12.646451+0000) 2024-09-10T10:06:31.502 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 10 10:06:31 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-9[65288]: 2024-09-10T10:06:31.135+0000 7f84af200640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-09-10T10:05:57.348185+0000 front 2024-09-10T10:05:57.348335+0000 (oldest deadline 2024-09-10T10:06:23.248210+0000) 2024-09-10T10:06:31.502 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 10 10:06:31 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-9[65288]: 2024-09-10T10:06:31.135+0000 7f84af200640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-10T10:06:03.249484+0000 front 2024-09-10T10:06:03.249185+0000 (oldest deadline 2024-09-10T10:06:26.749283+0000) 2024-09-10T10:06:31.590 INFO:journalctl@ceph.osd.7.smithi102.stdout:Sep 10 10:06:31 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-7[52880]: 2024-09-10T10:06:31.176+0000 7ff941357640 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-09-10T10:05:55.423434+0000 front 2024-09-10T10:05:55.423389+0000 (oldest deadline 2024-09-10T10:06:19.424367+0000) 2024-09-10T10:06:31.590 INFO:journalctl@ceph.osd.7.smithi102.stdout:Sep 10 10:06:31 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-7[52880]: 2024-09-10T10:06:31.176+0000 7ff941357640 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-10T10:06:04.025965+0000 front 2024-09-10T10:06:04.025864+0000 (oldest deadline 2024-09-10T10:06:28.726043+0000) 2024-09-10T10:06:31.600 INFO:journalctl@ceph.osd.8.smithi159.stdout:Sep 10 10:06:31 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-8[53439]: 2024-09-10T10:06:31.307+0000 7f7417907640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-09-10T10:05:57.316774+0000 front 2024-09-10T10:05:57.316749+0000 (oldest deadline 2024-09-10T10:06:20.816694+0000) 2024-09-10T10:06:31.600 INFO:journalctl@ceph.osd.8.smithi159.stdout:Sep 10 10:06:31 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-8[53439]: 2024-09-10T10:06:31.307+0000 7f7417907640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-10T10:06:04.917990+0000 front 2024-09-10T10:06:04.918013+0000 (oldest deadline 2024-09-10T10:06:27.818198+0000) 2024-09-10T10:06:31.806 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 10 10:06:31 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-6[58305]: 2024-09-10T10:06:31.501+0000 7f7322400640 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-09-10T10:05:56.623525+0000 front 2024-09-10T10:05:56.623092+0000 (oldest deadline 2024-09-10T10:06:24.224362+0000) 2024-09-10T10:06:31.806 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 10 10:06:31 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-6[58305]: 2024-09-10T10:06:31.501+0000 7f7322400640 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-10T10:06:04.225156+0000 front 2024-09-10T10:06:04.225182+0000 (oldest deadline 2024-09-10T10:06:27.725387+0000) 2024-09-10T10:06:31.885 INFO:journalctl@ceph.osd.10.smithi102.stdout:Sep 10 10:06:31 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-10[59874]: 2024-09-10T10:06:31.589+0000 7f40f3335640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-10T10:05:49.575097+0000 front 2024-09-10T10:05:49.575141+0000 (oldest deadline 2024-09-10T10:06:13.075035+0000) 2024-09-10T10:06:31.885 INFO:journalctl@ceph.osd.10.smithi102.stdout:Sep 10 10:06:31 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-10[59874]: 2024-09-10T10:06:31.589+0000 7f40f3335640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-09-10T10:05:54.176094+0000 front 2024-09-10T10:05:54.176115+0000 (oldest deadline 2024-09-10T10:06:19.376783+0000) 2024-09-10T10:06:32.087 INFO:journalctl@ceph.osd.5.smithi159.stdout:Sep 10 10:06:31 smithi159 podman[76933]: 2024-09-10 10:06:31.785794219 +0000 UTC m=+5.734600017 container died b693c1ca5c75ca51ca15b3f2271b60529b3a3ba215684f86154a065088b1dc90 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-5, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.37.2, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-09-10T10:06:32.087 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 10 10:06:31 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-11[60437]: 2024-09-10T10:06:31.660+0000 7fa78e4d3640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-10T10:05:49.428950+0000 front 2024-09-10T10:05:49.429031+0000 (oldest deadline 2024-09-10T10:06:11.128800+0000) 2024-09-10T10:06:32.087 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 10 10:06:31 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-11[60437]: 2024-09-10T10:06:31.660+0000 7fa78e4d3640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-10T10:06:03.430910+0000 front 2024-09-10T10:06:03.430889+0000 (oldest deadline 2024-09-10T10:06:29.331091+0000) 2024-09-10T10:06:32.350 INFO:journalctl@ceph.osd.5.smithi159.stdout:Sep 10 10:06:32 smithi159 podman[76933]: 2024-09-10 10:06:32.185020606 +0000 UTC m=+6.133826405 container cleanup b693c1ca5c75ca51ca15b3f2271b60529b3a3ba215684f86154a065088b1dc90 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-5, GIT_CLEAN=True, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux ) 2024-09-10T10:06:32.350 INFO:journalctl@ceph.osd.5.smithi159.stdout:Sep 10 10:06:32 smithi159 bash[76933]: ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-5 2024-09-10T10:06:32.351 INFO:journalctl@ceph.osd.8.smithi159.stdout:Sep 10 10:06:32 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-8[53439]: 2024-09-10T10:06:32.311+0000 7f7417907640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-09-10T10:05:57.316774+0000 front 2024-09-10T10:05:57.316749+0000 (oldest deadline 2024-09-10T10:06:20.816694+0000) 2024-09-10T10:06:32.351 INFO:journalctl@ceph.osd.8.smithi159.stdout:Sep 10 10:06:32 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-8[53439]: 2024-09-10T10:06:32.311+0000 7f7417907640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-10T10:06:04.917990+0000 front 2024-09-10T10:06:04.918013+0000 (oldest deadline 2024-09-10T10:06:27.818198+0000) 2024-09-10T10:06:32.549 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 10 10:06:32 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-9[65288]: 2024-09-10T10:06:32.177+0000 7f84af200640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-10T10:05:47.346189+0000 front 2024-09-10T10:05:47.346220+0000 (oldest deadline 2024-09-10T10:06:12.646451+0000) 2024-09-10T10:06:32.549 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 10 10:06:32 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-9[65288]: 2024-09-10T10:06:32.177+0000 7f84af200640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-09-10T10:05:57.348185+0000 front 2024-09-10T10:05:57.348335+0000 (oldest deadline 2024-09-10T10:06:23.248210+0000) 2024-09-10T10:06:32.549 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 10 10:06:32 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-9[65288]: 2024-09-10T10:06:32.177+0000 7f84af200640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-10T10:06:03.249484+0000 front 2024-09-10T10:06:03.249185+0000 (oldest deadline 2024-09-10T10:06:26.749283+0000) 2024-09-10T10:06:32.562 INFO:journalctl@ceph.osd.7.smithi102.stdout:Sep 10 10:06:32 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-7[52880]: 2024-09-10T10:06:32.222+0000 7ff941357640 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-09-10T10:05:55.423434+0000 front 2024-09-10T10:05:55.423389+0000 (oldest deadline 2024-09-10T10:06:19.424367+0000) 2024-09-10T10:06:32.562 INFO:journalctl@ceph.osd.7.smithi102.stdout:Sep 10 10:06:32 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-7[52880]: 2024-09-10T10:06:32.222+0000 7ff941357640 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-10T10:06:04.025965+0000 front 2024-09-10T10:06:04.025864+0000 (oldest deadline 2024-09-10T10:06:28.726043+0000) 2024-09-10T10:06:32.761 INFO:journalctl@ceph.osd.5.smithi159.stdout:Sep 10 10:06:32 smithi159 podman[76948]: 2024-09-10 10:06:32.453001794 +0000 UTC m=+0.663903927 container remove b693c1ca5c75ca51ca15b3f2271b60529b3a3ba215684f86154a065088b1dc90 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-5, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-09-10T10:06:32.761 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 10 10:06:32 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-11[60437]: 2024-09-10T10:06:32.679+0000 7fa78e4d3640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-10T10:05:49.428950+0000 front 2024-09-10T10:05:49.429031+0000 (oldest deadline 2024-09-10T10:06:11.128800+0000) 2024-09-10T10:06:32.761 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 10 10:06:32 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-11[60437]: 2024-09-10T10:06:32.679+0000 7fa78e4d3640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-10T10:06:03.430910+0000 front 2024-09-10T10:06:03.430889+0000 (oldest deadline 2024-09-10T10:06:29.331091+0000) 2024-09-10T10:06:32.806 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 10 10:06:32 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-6[58305]: 2024-09-10T10:06:32.547+0000 7f7322400640 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-09-10T10:05:56.623525+0000 front 2024-09-10T10:05:56.623092+0000 (oldest deadline 2024-09-10T10:06:24.224362+0000) 2024-09-10T10:06:32.806 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 10 10:06:32 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-6[58305]: 2024-09-10T10:06:32.547+0000 7f7322400640 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-10T10:06:04.225156+0000 front 2024-09-10T10:06:04.225182+0000 (oldest deadline 2024-09-10T10:06:27.725387+0000) 2024-09-10T10:06:32.884 INFO:journalctl@ceph.osd.10.smithi102.stdout:Sep 10 10:06:32 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-10[59874]: 2024-09-10T10:06:32.560+0000 7f40f3335640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-10T10:05:49.575097+0000 front 2024-09-10T10:05:49.575141+0000 (oldest deadline 2024-09-10T10:06:13.075035+0000) 2024-09-10T10:06:32.884 INFO:journalctl@ceph.osd.10.smithi102.stdout:Sep 10 10:06:32 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-10[59874]: 2024-09-10T10:06:32.560+0000 7f40f3335640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-09-10T10:05:54.176094+0000 front 2024-09-10T10:05:54.176115+0000 (oldest deadline 2024-09-10T10:06:19.376783+0000) 2024-09-10T10:06:32.885 INFO:journalctl@ceph.osd.10.smithi102.stdout:Sep 10 10:06:32 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-10[59874]: 2024-09-10T10:06:32.560+0000 7f40f3335640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-10T10:06:03.378660+0000 front 2024-09-10T10:06:03.378719+0000 (oldest deadline 2024-09-10T10:06:32.178984+0000) 2024-09-10T10:06:33.074 INFO:journalctl@ceph.osd.5.smithi159.stdout:Sep 10 10:06:32 smithi159 podman[77037]: 2024-09-10 10:06:32.663443177 +0000 UTC m=+0.015797405 image pull 41d67112835771c3efca60c6b8fe297c7ddaecdc26cea26fd28aa18636e49005 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 2024-09-10T10:06:33.074 INFO:journalctl@ceph.osd.5.smithi159.stdout:Sep 10 10:06:32 smithi159 podman[77037]: 2024-09-10 10:06:32.792595783 +0000 UTC m=+0.144950001 container create 4b4f11dea0dfb9a7f7587126859dc85a171fe1fd8ece9faf58a7aee77696c452 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-5-deactivate, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-09-10T10:06:33.074 INFO:journalctl@ceph.osd.5.smithi159.stdout:Sep 10 10:06:33 smithi159 podman[77037]: 2024-09-10 10:06:33.073449699 +0000 UTC m=+0.425803916 container init 4b4f11dea0dfb9a7f7587126859dc85a171fe1fd8ece9faf58a7aee77696c452 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-5-deactivate, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.schema-version=1.0) 2024-09-10T10:06:33.350 INFO:journalctl@ceph.osd.8.smithi159.stdout:Sep 10 10:06:33 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-8[53439]: 2024-09-10T10:06:33.279+0000 7f7417907640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-09-10T10:05:57.316774+0000 front 2024-09-10T10:05:57.316749+0000 (oldest deadline 2024-09-10T10:06:20.816694+0000) 2024-09-10T10:06:33.350 INFO:journalctl@ceph.osd.8.smithi159.stdout:Sep 10 10:06:33 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-8[53439]: 2024-09-10T10:06:33.279+0000 7f7417907640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-10T10:06:04.917990+0000 front 2024-09-10T10:06:04.918013+0000 (oldest deadline 2024-09-10T10:06:27.818198+0000) 2024-09-10T10:06:33.351 INFO:journalctl@ceph.osd.5.smithi159.stdout:Sep 10 10:06:33 smithi159 podman[77037]: 2024-09-10 10:06:33.076684359 +0000 UTC m=+0.429038577 container start 4b4f11dea0dfb9a7f7587126859dc85a171fe1fd8ece9faf58a7aee77696c452 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-5-deactivate, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, io.buildah.version=1.37.2, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T10:06:33.351 INFO:journalctl@ceph.osd.5.smithi159.stdout:Sep 10 10:06:33 smithi159 podman[77037]: 2024-09-10 10:06:33.173404911 +0000 UTC m=+0.525759139 container attach 4b4f11dea0dfb9a7f7587126859dc85a171fe1fd8ece9faf58a7aee77696c452 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-5-deactivate, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, io.buildah.version=1.37.2) 2024-09-10T10:06:33.351 INFO:journalctl@ceph.osd.5.smithi159.stdout:Sep 10 10:06:33 smithi159 podman[77037]: 2024-09-10 10:06:33.268054895 +0000 UTC m=+0.620409116 container died 4b4f11dea0dfb9a7f7587126859dc85a171fe1fd8ece9faf58a7aee77696c452 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-5-deactivate, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-09-10T10:06:33.533 INFO:journalctl@ceph.osd.7.smithi102.stdout:Sep 10 10:06:33 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-7[52880]: 2024-09-10T10:06:33.254+0000 7ff941357640 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-09-10T10:05:55.423434+0000 front 2024-09-10T10:05:55.423389+0000 (oldest deadline 2024-09-10T10:06:19.424367+0000) 2024-09-10T10:06:33.533 INFO:journalctl@ceph.osd.7.smithi102.stdout:Sep 10 10:06:33 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-7[52880]: 2024-09-10T10:06:33.254+0000 7ff941357640 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-10T10:06:04.025965+0000 front 2024-09-10T10:06:04.025864+0000 (oldest deadline 2024-09-10T10:06:28.726043+0000) 2024-09-10T10:06:33.556 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 10 10:06:33 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-9[65288]: 2024-09-10T10:06:33.204+0000 7f84af200640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-10T10:05:47.346189+0000 front 2024-09-10T10:05:47.346220+0000 (oldest deadline 2024-09-10T10:06:12.646451+0000) 2024-09-10T10:06:33.556 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 10 10:06:33 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-9[65288]: 2024-09-10T10:06:33.204+0000 7f84af200640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-09-10T10:05:57.348185+0000 front 2024-09-10T10:05:57.348335+0000 (oldest deadline 2024-09-10T10:06:23.248210+0000) 2024-09-10T10:06:33.556 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 10 10:06:33 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-9[65288]: 2024-09-10T10:06:33.204+0000 7f84af200640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-10T10:06:03.249484+0000 front 2024-09-10T10:06:03.249185+0000 (oldest deadline 2024-09-10T10:06:26.749283+0000) 2024-09-10T10:06:33.884 INFO:journalctl@ceph.osd.10.smithi102.stdout:Sep 10 10:06:33 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-10[59874]: 2024-09-10T10:06:33.532+0000 7f40f3335640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-10T10:05:49.575097+0000 front 2024-09-10T10:05:49.575141+0000 (oldest deadline 2024-09-10T10:06:13.075035+0000) 2024-09-10T10:06:33.885 INFO:journalctl@ceph.osd.10.smithi102.stdout:Sep 10 10:06:33 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-10[59874]: 2024-09-10T10:06:33.532+0000 7f40f3335640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-09-10T10:05:54.176094+0000 front 2024-09-10T10:05:54.176115+0000 (oldest deadline 2024-09-10T10:06:19.376783+0000) 2024-09-10T10:06:33.885 INFO:journalctl@ceph.osd.10.smithi102.stdout:Sep 10 10:06:33 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-10[59874]: 2024-09-10T10:06:33.532+0000 7f40f3335640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-10T10:06:03.378660+0000 front 2024-09-10T10:06:03.378719+0000 (oldest deadline 2024-09-10T10:06:32.178984+0000) 2024-09-10T10:06:33.953 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 10 10:06:33 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-11[60437]: 2024-09-10T10:06:33.713+0000 7fa78e4d3640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-10T10:05:49.428950+0000 front 2024-09-10T10:05:49.429031+0000 (oldest deadline 2024-09-10T10:06:11.128800+0000) 2024-09-10T10:06:33.953 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 10 10:06:33 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-11[60437]: 2024-09-10T10:06:33.713+0000 7fa78e4d3640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-10T10:06:03.430910+0000 front 2024-09-10T10:06:03.430889+0000 (oldest deadline 2024-09-10T10:06:29.331091+0000) 2024-09-10T10:06:33.968 DEBUG:teuthology.orchestra.run.smithi159:> sudo pkill -f 'journalctl -f -n 0 -u ceph-bade4b50-6f56-11ef-bcea-c7b262605968@osd.5.service' 2024-09-10T10:06:34.009 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T10:06:34.009 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-09-10T10:06:34.009 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-09-10T10:06:34.009 DEBUG:teuthology.orchestra.run.smithi005:> sudo systemctl stop ceph-bade4b50-6f56-11ef-bcea-c7b262605968@osd.6 2024-09-10T10:06:34.056 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 10 10:06:33 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-6[58305]: 2024-09-10T10:06:33.567+0000 7f7322400640 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-09-10T10:05:56.623525+0000 front 2024-09-10T10:05:56.623092+0000 (oldest deadline 2024-09-10T10:06:24.224362+0000) 2024-09-10T10:06:34.056 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 10 10:06:33 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-6[58305]: 2024-09-10T10:06:33.567+0000 7f7322400640 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-10T10:06:04.225156+0000 front 2024-09-10T10:06:04.225182+0000 (oldest deadline 2024-09-10T10:06:27.725387+0000) 2024-09-10T10:06:34.476 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 10 10:06:34 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-9[65288]: 2024-09-10T10:06:34.163+0000 7f84af200640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-10T10:05:47.346189+0000 front 2024-09-10T10:05:47.346220+0000 (oldest deadline 2024-09-10T10:06:12.646451+0000) 2024-09-10T10:06:34.476 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 10 10:06:34 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-9[65288]: 2024-09-10T10:06:34.163+0000 7f84af200640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-09-10T10:05:57.348185+0000 front 2024-09-10T10:05:57.348335+0000 (oldest deadline 2024-09-10T10:06:23.248210+0000) 2024-09-10T10:06:34.476 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 10 10:06:34 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-9[65288]: 2024-09-10T10:06:34.163+0000 7f84af200640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-10T10:06:03.249484+0000 front 2024-09-10T10:06:03.249185+0000 (oldest deadline 2024-09-10T10:06:26.749283+0000) 2024-09-10T10:06:34.477 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 10 10:06:34 smithi005 systemd[1]: Stopping Ceph osd.6 for bade4b50-6f56-11ef-bcea-c7b262605968... 2024-09-10T10:06:34.477 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 10 10:06:34 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-6[58305]: 2024-09-10T10:06:34.475+0000 7f732ba00640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T10:06:34.477 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 10 10:06:34 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-6[58305]: 2024-09-10T10:06:34.475+0000 7f732ba00640 -1 osd.6 100 *** Got signal Terminated *** 2024-09-10T10:06:34.477 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 10 10:06:34 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-6[58305]: 2024-09-10T10:06:34.475+0000 7f732ba00640 -1 osd.6 100 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-10T10:06:34.552 INFO:journalctl@ceph.osd.7.smithi102.stdout:Sep 10 10:06:34 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-7[52880]: 2024-09-10T10:06:34.257+0000 7ff941357640 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-09-10T10:05:55.423434+0000 front 2024-09-10T10:05:55.423389+0000 (oldest deadline 2024-09-10T10:06:19.424367+0000) 2024-09-10T10:06:34.552 INFO:journalctl@ceph.osd.7.smithi102.stdout:Sep 10 10:06:34 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-7[52880]: 2024-09-10T10:06:34.257+0000 7ff941357640 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-10T10:06:04.025965+0000 front 2024-09-10T10:06:04.025864+0000 (oldest deadline 2024-09-10T10:06:28.726043+0000) 2024-09-10T10:06:34.600 INFO:journalctl@ceph.osd.8.smithi159.stdout:Sep 10 10:06:34 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-8[53439]: 2024-09-10T10:06:34.255+0000 7f7417907640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-09-10T10:05:57.316774+0000 front 2024-09-10T10:05:57.316749+0000 (oldest deadline 2024-09-10T10:06:20.816694+0000) 2024-09-10T10:06:34.600 INFO:journalctl@ceph.osd.8.smithi159.stdout:Sep 10 10:06:34 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-8[53439]: 2024-09-10T10:06:34.255+0000 7f7417907640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-10T10:06:04.917990+0000 front 2024-09-10T10:06:04.918013+0000 (oldest deadline 2024-09-10T10:06:27.818198+0000) 2024-09-10T10:06:34.806 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 10 10:06:34 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-6[58305]: 2024-09-10T10:06:34.534+0000 7f7322400640 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-09-10T10:05:56.623525+0000 front 2024-09-10T10:05:56.623092+0000 (oldest deadline 2024-09-10T10:06:24.224362+0000) 2024-09-10T10:06:34.806 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 10 10:06:34 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-6[58305]: 2024-09-10T10:06:34.534+0000 7f7322400640 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-10T10:06:04.225156+0000 front 2024-09-10T10:06:04.225182+0000 (oldest deadline 2024-09-10T10:06:27.725387+0000) 2024-09-10T10:06:34.884 INFO:journalctl@ceph.osd.10.smithi102.stdout:Sep 10 10:06:34 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-10[59874]: 2024-09-10T10:06:34.550+0000 7f40f3335640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-10T10:05:49.575097+0000 front 2024-09-10T10:05:49.575141+0000 (oldest deadline 2024-09-10T10:06:13.075035+0000) 2024-09-10T10:06:34.884 INFO:journalctl@ceph.osd.10.smithi102.stdout:Sep 10 10:06:34 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-10[59874]: 2024-09-10T10:06:34.550+0000 7f40f3335640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-09-10T10:05:54.176094+0000 front 2024-09-10T10:05:54.176115+0000 (oldest deadline 2024-09-10T10:06:19.376783+0000) 2024-09-10T10:06:34.885 INFO:journalctl@ceph.osd.10.smithi102.stdout:Sep 10 10:06:34 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-10[59874]: 2024-09-10T10:06:34.550+0000 7f40f3335640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-10T10:06:03.378660+0000 front 2024-09-10T10:06:03.378719+0000 (oldest deadline 2024-09-10T10:06:32.178984+0000) 2024-09-10T10:06:35.100 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 10 10:06:34 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-11[60437]: 2024-09-10T10:06:34.724+0000 7fa78e4d3640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-10T10:05:49.428950+0000 front 2024-09-10T10:05:49.429031+0000 (oldest deadline 2024-09-10T10:06:11.128800+0000) 2024-09-10T10:06:35.100 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 10 10:06:34 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-11[60437]: 2024-09-10T10:06:34.724+0000 7fa78e4d3640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-10T10:06:03.430910+0000 front 2024-09-10T10:06:03.430889+0000 (oldest deadline 2024-09-10T10:06:29.331091+0000) 2024-09-10T10:06:35.529 INFO:journalctl@ceph.osd.7.smithi102.stdout:Sep 10 10:06:35 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-7[52880]: 2024-09-10T10:06:35.266+0000 7ff941357640 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-09-10T10:05:55.423434+0000 front 2024-09-10T10:05:55.423389+0000 (oldest deadline 2024-09-10T10:06:19.424367+0000) 2024-09-10T10:06:35.529 INFO:journalctl@ceph.osd.7.smithi102.stdout:Sep 10 10:06:35 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-7[52880]: 2024-09-10T10:06:35.266+0000 7ff941357640 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-10T10:06:04.025965+0000 front 2024-09-10T10:06:04.025864+0000 (oldest deadline 2024-09-10T10:06:28.726043+0000) 2024-09-10T10:06:35.556 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 10 10:06:35 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-9[65288]: 2024-09-10T10:06:35.135+0000 7f84af200640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-10T10:05:47.346189+0000 front 2024-09-10T10:05:47.346220+0000 (oldest deadline 2024-09-10T10:06:12.646451+0000) 2024-09-10T10:06:35.556 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 10 10:06:35 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-9[65288]: 2024-09-10T10:06:35.135+0000 7f84af200640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-09-10T10:05:57.348185+0000 front 2024-09-10T10:05:57.348335+0000 (oldest deadline 2024-09-10T10:06:23.248210+0000) 2024-09-10T10:06:35.556 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 10 10:06:35 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-9[65288]: 2024-09-10T10:06:35.135+0000 7f84af200640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-10T10:06:03.249484+0000 front 2024-09-10T10:06:03.249185+0000 (oldest deadline 2024-09-10T10:06:26.749283+0000) 2024-09-10T10:06:35.600 INFO:journalctl@ceph.osd.8.smithi159.stdout:Sep 10 10:06:35 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-8[53439]: 2024-09-10T10:06:35.270+0000 7f7417907640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-09-10T10:05:57.316774+0000 front 2024-09-10T10:05:57.316749+0000 (oldest deadline 2024-09-10T10:06:20.816694+0000) 2024-09-10T10:06:35.600 INFO:journalctl@ceph.osd.8.smithi159.stdout:Sep 10 10:06:35 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-8[53439]: 2024-09-10T10:06:35.270+0000 7f7417907640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-10T10:06:04.917990+0000 front 2024-09-10T10:06:04.918013+0000 (oldest deadline 2024-09-10T10:06:27.818198+0000) 2024-09-10T10:06:35.884 INFO:journalctl@ceph.osd.10.smithi102.stdout:Sep 10 10:06:35 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-10[59874]: 2024-09-10T10:06:35.528+0000 7f40f3335640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-10T10:05:49.575097+0000 front 2024-09-10T10:05:49.575141+0000 (oldest deadline 2024-09-10T10:06:13.075035+0000) 2024-09-10T10:06:35.885 INFO:journalctl@ceph.osd.10.smithi102.stdout:Sep 10 10:06:35 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-10[59874]: 2024-09-10T10:06:35.528+0000 7f40f3335640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-09-10T10:05:54.176094+0000 front 2024-09-10T10:05:54.176115+0000 (oldest deadline 2024-09-10T10:06:19.376783+0000) 2024-09-10T10:06:35.885 INFO:journalctl@ceph.osd.10.smithi102.stdout:Sep 10 10:06:35 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-10[59874]: 2024-09-10T10:06:35.528+0000 7f40f3335640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-10T10:06:03.378660+0000 front 2024-09-10T10:06:03.378719+0000 (oldest deadline 2024-09-10T10:06:32.178984+0000) 2024-09-10T10:06:36.056 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 10 10:06:35 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-6[58305]: 2024-09-10T10:06:35.559+0000 7f7322400640 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-09-10T10:05:56.623525+0000 front 2024-09-10T10:05:56.623092+0000 (oldest deadline 2024-09-10T10:06:24.224362+0000) 2024-09-10T10:06:36.056 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 10 10:06:35 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-6[58305]: 2024-09-10T10:06:35.559+0000 7f7322400640 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-10T10:06:04.225156+0000 front 2024-09-10T10:06:04.225182+0000 (oldest deadline 2024-09-10T10:06:27.725387+0000) 2024-09-10T10:06:36.100 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 10 10:06:35 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-11[60437]: 2024-09-10T10:06:35.693+0000 7fa78e4d3640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-10T10:05:49.428950+0000 front 2024-09-10T10:05:49.429031+0000 (oldest deadline 2024-09-10T10:06:11.128800+0000) 2024-09-10T10:06:36.100 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 10 10:06:35 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-11[60437]: 2024-09-10T10:06:35.693+0000 7fa78e4d3640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-10T10:06:03.430910+0000 front 2024-09-10T10:06:03.430889+0000 (oldest deadline 2024-09-10T10:06:29.331091+0000) 2024-09-10T10:06:36.536 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 10 10:06:36 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-9[65288]: 2024-09-10T10:06:36.178+0000 7f84af200640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-10T10:05:47.346189+0000 front 2024-09-10T10:05:47.346220+0000 (oldest deadline 2024-09-10T10:06:12.646451+0000) 2024-09-10T10:06:36.536 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 10 10:06:36 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-9[65288]: 2024-09-10T10:06:36.178+0000 7f84af200640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-09-10T10:05:57.348185+0000 front 2024-09-10T10:05:57.348335+0000 (oldest deadline 2024-09-10T10:06:23.248210+0000) 2024-09-10T10:06:36.537 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 10 10:06:36 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-9[65288]: 2024-09-10T10:06:36.178+0000 7f84af200640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-10T10:06:03.249484+0000 front 2024-09-10T10:06:03.249185+0000 (oldest deadline 2024-09-10T10:06:26.749283+0000) 2024-09-10T10:06:36.556 INFO:journalctl@ceph.osd.7.smithi102.stdout:Sep 10 10:06:36 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-7[52880]: 2024-09-10T10:06:36.252+0000 7ff941357640 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-09-10T10:05:55.423434+0000 front 2024-09-10T10:05:55.423389+0000 (oldest deadline 2024-09-10T10:06:19.424367+0000) 2024-09-10T10:06:36.556 INFO:journalctl@ceph.osd.7.smithi102.stdout:Sep 10 10:06:36 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-7[52880]: 2024-09-10T10:06:36.252+0000 7ff941357640 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-10T10:06:04.025965+0000 front 2024-09-10T10:06:04.025864+0000 (oldest deadline 2024-09-10T10:06:28.726043+0000) 2024-09-10T10:06:36.600 INFO:journalctl@ceph.osd.8.smithi159.stdout:Sep 10 10:06:36 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-8[53439]: 2024-09-10T10:06:36.238+0000 7f7417907640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-09-10T10:05:57.316774+0000 front 2024-09-10T10:05:57.316749+0000 (oldest deadline 2024-09-10T10:06:20.816694+0000) 2024-09-10T10:06:36.600 INFO:journalctl@ceph.osd.8.smithi159.stdout:Sep 10 10:06:36 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-8[53439]: 2024-09-10T10:06:36.238+0000 7f7417907640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-10T10:06:04.917990+0000 front 2024-09-10T10:06:04.918013+0000 (oldest deadline 2024-09-10T10:06:27.818198+0000) 2024-09-10T10:06:36.806 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 10 10:06:36 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-6[58305]: 2024-09-10T10:06:36.535+0000 7f7322400640 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-09-10T10:05:56.623525+0000 front 2024-09-10T10:05:56.623092+0000 (oldest deadline 2024-09-10T10:06:24.224362+0000) 2024-09-10T10:06:36.806 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 10 10:06:36 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-6[58305]: 2024-09-10T10:06:36.535+0000 7f7322400640 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-10T10:06:04.225156+0000 front 2024-09-10T10:06:04.225182+0000 (oldest deadline 2024-09-10T10:06:27.725387+0000) 2024-09-10T10:06:36.885 INFO:journalctl@ceph.osd.10.smithi102.stdout:Sep 10 10:06:36 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-10[59874]: 2024-09-10T10:06:36.555+0000 7f40f3335640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-10T10:05:49.575097+0000 front 2024-09-10T10:05:49.575141+0000 (oldest deadline 2024-09-10T10:06:13.075035+0000) 2024-09-10T10:06:36.885 INFO:journalctl@ceph.osd.10.smithi102.stdout:Sep 10 10:06:36 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-10[59874]: 2024-09-10T10:06:36.555+0000 7f40f3335640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-09-10T10:05:54.176094+0000 front 2024-09-10T10:05:54.176115+0000 (oldest deadline 2024-09-10T10:06:19.376783+0000) 2024-09-10T10:06:36.885 INFO:journalctl@ceph.osd.10.smithi102.stdout:Sep 10 10:06:36 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-10[59874]: 2024-09-10T10:06:36.555+0000 7f40f3335640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-10T10:06:03.378660+0000 front 2024-09-10T10:06:03.378719+0000 (oldest deadline 2024-09-10T10:06:32.178984+0000) 2024-09-10T10:06:37.100 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 10 10:06:36 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-11[60437]: 2024-09-10T10:06:36.665+0000 7fa78e4d3640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-10T10:05:49.428950+0000 front 2024-09-10T10:05:49.429031+0000 (oldest deadline 2024-09-10T10:06:11.128800+0000) 2024-09-10T10:06:37.100 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 10 10:06:36 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-11[60437]: 2024-09-10T10:06:36.665+0000 7fa78e4d3640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-10T10:06:03.430910+0000 front 2024-09-10T10:06:03.430889+0000 (oldest deadline 2024-09-10T10:06:29.331091+0000) 2024-09-10T10:06:37.100 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 10 10:06:36 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-11[60437]: 2024-09-10T10:06:36.665+0000 7fa78e4d3640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-10T10:06:13.432384+0000 front 2024-09-10T10:06:13.432489+0000 (oldest deadline 2024-09-10T10:06:35.732466+0000) 2024-09-10T10:06:37.493 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 10 10:06:37 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-9[65288]: 2024-09-10T10:06:37.216+0000 7f84af200640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-10T10:05:47.346189+0000 front 2024-09-10T10:05:47.346220+0000 (oldest deadline 2024-09-10T10:06:12.646451+0000) 2024-09-10T10:06:37.493 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 10 10:06:37 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-9[65288]: 2024-09-10T10:06:37.216+0000 7f84af200640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-09-10T10:05:57.348185+0000 front 2024-09-10T10:05:57.348335+0000 (oldest deadline 2024-09-10T10:06:23.248210+0000) 2024-09-10T10:06:37.493 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 10 10:06:37 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-9[65288]: 2024-09-10T10:06:37.216+0000 7f84af200640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-10T10:06:03.249484+0000 front 2024-09-10T10:06:03.249185+0000 (oldest deadline 2024-09-10T10:06:26.749283+0000) 2024-09-10T10:06:37.558 INFO:journalctl@ceph.osd.7.smithi102.stdout:Sep 10 10:06:37 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-7[52880]: 2024-09-10T10:06:37.282+0000 7ff941357640 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-09-10T10:05:55.423434+0000 front 2024-09-10T10:05:55.423389+0000 (oldest deadline 2024-09-10T10:06:19.424367+0000) 2024-09-10T10:06:37.558 INFO:journalctl@ceph.osd.7.smithi102.stdout:Sep 10 10:06:37 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-7[52880]: 2024-09-10T10:06:37.282+0000 7ff941357640 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-10T10:06:04.025965+0000 front 2024-09-10T10:06:04.025864+0000 (oldest deadline 2024-09-10T10:06:28.726043+0000) 2024-09-10T10:06:37.600 INFO:journalctl@ceph.osd.8.smithi159.stdout:Sep 10 10:06:37 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-8[53439]: 2024-09-10T10:06:37.268+0000 7f7417907640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-09-10T10:05:57.316774+0000 front 2024-09-10T10:05:57.316749+0000 (oldest deadline 2024-09-10T10:06:20.816694+0000) 2024-09-10T10:06:37.601 INFO:journalctl@ceph.osd.8.smithi159.stdout:Sep 10 10:06:37 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-8[53439]: 2024-09-10T10:06:37.268+0000 7f7417907640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-10T10:06:04.917990+0000 front 2024-09-10T10:06:04.918013+0000 (oldest deadline 2024-09-10T10:06:27.818198+0000) 2024-09-10T10:06:37.601 INFO:journalctl@ceph.osd.8.smithi159.stdout:Sep 10 10:06:37 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-8[53439]: 2024-09-10T10:06:37.268+0000 7f7417907640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-10T10:06:13.119605+0000 front 2024-09-10T10:06:13.119551+0000 (oldest deadline 2024-09-10T10:06:36.619665+0000) 2024-09-10T10:06:37.806 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 10 10:06:37 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-6[58305]: 2024-09-10T10:06:37.492+0000 7f7322400640 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-09-10T10:05:56.623525+0000 front 2024-09-10T10:05:56.623092+0000 (oldest deadline 2024-09-10T10:06:24.224362+0000) 2024-09-10T10:06:37.806 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 10 10:06:37 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-6[58305]: 2024-09-10T10:06:37.492+0000 7f7322400640 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-10T10:06:04.225156+0000 front 2024-09-10T10:06:04.225182+0000 (oldest deadline 2024-09-10T10:06:27.725387+0000) 2024-09-10T10:06:37.884 INFO:journalctl@ceph.osd.10.smithi102.stdout:Sep 10 10:06:37 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-10[59874]: 2024-09-10T10:06:37.557+0000 7f40f3335640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-10T10:05:49.575097+0000 front 2024-09-10T10:05:49.575141+0000 (oldest deadline 2024-09-10T10:06:13.075035+0000) 2024-09-10T10:06:37.885 INFO:journalctl@ceph.osd.10.smithi102.stdout:Sep 10 10:06:37 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-10[59874]: 2024-09-10T10:06:37.557+0000 7f40f3335640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-09-10T10:05:54.176094+0000 front 2024-09-10T10:05:54.176115+0000 (oldest deadline 2024-09-10T10:06:19.376783+0000) 2024-09-10T10:06:37.885 INFO:journalctl@ceph.osd.10.smithi102.stdout:Sep 10 10:06:37 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-10[59874]: 2024-09-10T10:06:37.557+0000 7f40f3335640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-10T10:06:03.378660+0000 front 2024-09-10T10:06:03.378719+0000 (oldest deadline 2024-09-10T10:06:32.178984+0000) 2024-09-10T10:06:37.885 INFO:journalctl@ceph.osd.10.smithi102.stdout:Sep 10 10:06:37 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-10[59874]: 2024-09-10T10:06:37.557+0000 7f40f3335640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-10T10:06:12.179545+0000 front 2024-09-10T10:06:12.179581+0000 (oldest deadline 2024-09-10T10:06:36.879462+0000) 2024-09-10T10:06:38.100 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 10 10:06:37 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-11[60437]: 2024-09-10T10:06:37.712+0000 7fa78e4d3640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-10T10:05:49.428950+0000 front 2024-09-10T10:05:49.429031+0000 (oldest deadline 2024-09-10T10:06:11.128800+0000) 2024-09-10T10:06:38.101 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 10 10:06:37 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-11[60437]: 2024-09-10T10:06:37.712+0000 7fa78e4d3640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-10T10:06:03.430910+0000 front 2024-09-10T10:06:03.430889+0000 (oldest deadline 2024-09-10T10:06:29.331091+0000) 2024-09-10T10:06:38.101 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 10 10:06:37 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-11[60437]: 2024-09-10T10:06:37.712+0000 7fa78e4d3640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-10T10:06:13.432384+0000 front 2024-09-10T10:06:13.432489+0000 (oldest deadline 2024-09-10T10:06:35.732466+0000) 2024-09-10T10:06:38.556 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 10 10:06:38 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-9[65288]: 2024-09-10T10:06:38.263+0000 7f84af200640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-10T10:05:47.346189+0000 front 2024-09-10T10:05:47.346220+0000 (oldest deadline 2024-09-10T10:06:12.646451+0000) 2024-09-10T10:06:38.557 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 10 10:06:38 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-9[65288]: 2024-09-10T10:06:38.263+0000 7f84af200640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-09-10T10:05:57.348185+0000 front 2024-09-10T10:05:57.348335+0000 (oldest deadline 2024-09-10T10:06:23.248210+0000) 2024-09-10T10:06:38.557 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 10 10:06:38 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-9[65288]: 2024-09-10T10:06:38.263+0000 7f84af200640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-10T10:06:03.249484+0000 front 2024-09-10T10:06:03.249185+0000 (oldest deadline 2024-09-10T10:06:26.749283+0000) 2024-09-10T10:06:38.557 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 10 10:06:38 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-6[58305]: 2024-09-10T10:06:38.511+0000 7f7322400640 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-09-10T10:05:56.623525+0000 front 2024-09-10T10:05:56.623092+0000 (oldest deadline 2024-09-10T10:06:24.224362+0000) 2024-09-10T10:06:38.557 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 10 10:06:38 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-6[58305]: 2024-09-10T10:06:38.511+0000 7f7322400640 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-10T10:06:04.225156+0000 front 2024-09-10T10:06:04.225182+0000 (oldest deadline 2024-09-10T10:06:27.725387+0000) 2024-09-10T10:06:38.557 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 10 10:06:38 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-6[58305]: 2024-09-10T10:06:38.511+0000 7f7322400640 -1 osd.6 100 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-10T10:06:13.026976+0000 front 2024-09-10T10:06:13.027029+0000 (oldest deadline 2024-09-10T10:06:37.727173+0000) 2024-09-10T10:06:38.600 INFO:journalctl@ceph.osd.8.smithi159.stdout:Sep 10 10:06:38 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-8[53439]: 2024-09-10T10:06:38.235+0000 7f7417907640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-09-10T10:05:57.316774+0000 front 2024-09-10T10:05:57.316749+0000 (oldest deadline 2024-09-10T10:06:20.816694+0000) 2024-09-10T10:06:38.600 INFO:journalctl@ceph.osd.8.smithi159.stdout:Sep 10 10:06:38 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-8[53439]: 2024-09-10T10:06:38.235+0000 7f7417907640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-10T10:06:04.917990+0000 front 2024-09-10T10:06:04.918013+0000 (oldest deadline 2024-09-10T10:06:27.818198+0000) 2024-09-10T10:06:38.601 INFO:journalctl@ceph.osd.8.smithi159.stdout:Sep 10 10:06:38 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-8[53439]: 2024-09-10T10:06:38.235+0000 7f7417907640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-10T10:06:13.119605+0000 front 2024-09-10T10:06:13.119551+0000 (oldest deadline 2024-09-10T10:06:36.619665+0000) 2024-09-10T10:06:38.634 INFO:journalctl@ceph.osd.10.smithi102.stdout:Sep 10 10:06:38 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-10[59874]: 2024-09-10T10:06:38.520+0000 7f40f3335640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-10T10:05:49.575097+0000 front 2024-09-10T10:05:49.575141+0000 (oldest deadline 2024-09-10T10:06:13.075035+0000) 2024-09-10T10:06:38.634 INFO:journalctl@ceph.osd.10.smithi102.stdout:Sep 10 10:06:38 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-10[59874]: 2024-09-10T10:06:38.520+0000 7f40f3335640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-09-10T10:05:54.176094+0000 front 2024-09-10T10:05:54.176115+0000 (oldest deadline 2024-09-10T10:06:19.376783+0000) 2024-09-10T10:06:38.635 INFO:journalctl@ceph.osd.10.smithi102.stdout:Sep 10 10:06:38 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-10[59874]: 2024-09-10T10:06:38.520+0000 7f40f3335640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-10T10:06:03.378660+0000 front 2024-09-10T10:06:03.378719+0000 (oldest deadline 2024-09-10T10:06:32.178984+0000) 2024-09-10T10:06:38.635 INFO:journalctl@ceph.osd.10.smithi102.stdout:Sep 10 10:06:38 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-10[59874]: 2024-09-10T10:06:38.520+0000 7f40f3335640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-10T10:06:12.179545+0000 front 2024-09-10T10:06:12.179581+0000 (oldest deadline 2024-09-10T10:06:36.879462+0000) 2024-09-10T10:06:38.635 INFO:journalctl@ceph.osd.7.smithi102.stdout:Sep 10 10:06:38 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-7[52880]: 2024-09-10T10:06:38.331+0000 7ff941357640 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-09-10T10:05:55.423434+0000 front 2024-09-10T10:05:55.423389+0000 (oldest deadline 2024-09-10T10:06:19.424367+0000) 2024-09-10T10:06:38.635 INFO:journalctl@ceph.osd.7.smithi102.stdout:Sep 10 10:06:38 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-7[52880]: 2024-09-10T10:06:38.331+0000 7ff941357640 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-10T10:06:04.025965+0000 front 2024-09-10T10:06:04.025864+0000 (oldest deadline 2024-09-10T10:06:28.726043+0000) 2024-09-10T10:06:38.635 INFO:journalctl@ceph.osd.7.smithi102.stdout:Sep 10 10:06:38 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-7[52880]: 2024-09-10T10:06:38.331+0000 7ff941357640 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-10T10:06:12.827133+0000 front 2024-09-10T10:06:12.827237+0000 (oldest deadline 2024-09-10T10:06:38.027668+0000) 2024-09-10T10:06:39.100 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 10 10:06:38 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-11[60437]: 2024-09-10T10:06:38.752+0000 7fa78e4d3640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-10T10:05:49.428950+0000 front 2024-09-10T10:05:49.429031+0000 (oldest deadline 2024-09-10T10:06:11.128800+0000) 2024-09-10T10:06:39.101 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 10 10:06:38 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-11[60437]: 2024-09-10T10:06:38.752+0000 7fa78e4d3640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-10T10:06:03.430910+0000 front 2024-09-10T10:06:03.430889+0000 (oldest deadline 2024-09-10T10:06:29.331091+0000) 2024-09-10T10:06:39.101 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 10 10:06:38 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-11[60437]: 2024-09-10T10:06:38.752+0000 7fa78e4d3640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-10T10:06:13.432384+0000 front 2024-09-10T10:06:13.432489+0000 (oldest deadline 2024-09-10T10:06:35.732466+0000) 2024-09-10T10:06:39.556 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 10 10:06:39 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-9[65288]: 2024-09-10T10:06:39.249+0000 7f84af200640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-10T10:05:47.346189+0000 front 2024-09-10T10:05:47.346220+0000 (oldest deadline 2024-09-10T10:06:12.646451+0000) 2024-09-10T10:06:39.556 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 10 10:06:39 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-9[65288]: 2024-09-10T10:06:39.249+0000 7f84af200640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-09-10T10:05:57.348185+0000 front 2024-09-10T10:05:57.348335+0000 (oldest deadline 2024-09-10T10:06:23.248210+0000) 2024-09-10T10:06:39.556 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 10 10:06:39 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-9[65288]: 2024-09-10T10:06:39.249+0000 7f84af200640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-10T10:06:03.249484+0000 front 2024-09-10T10:06:03.249185+0000 (oldest deadline 2024-09-10T10:06:26.749283+0000) 2024-09-10T10:06:39.557 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 10 10:06:39 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-9[65288]: 2024-09-10T10:06:39.249+0000 7f84af200640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-10T10:06:12.650860+0000 front 2024-09-10T10:06:12.650900+0000 (oldest deadline 2024-09-10T10:06:38.551142+0000) 2024-09-10T10:06:39.600 INFO:journalctl@ceph.osd.8.smithi159.stdout:Sep 10 10:06:39 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-8[53439]: 2024-09-10T10:06:39.258+0000 7f7417907640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-09-10T10:05:57.316774+0000 front 2024-09-10T10:05:57.316749+0000 (oldest deadline 2024-09-10T10:06:20.816694+0000) 2024-09-10T10:06:39.600 INFO:journalctl@ceph.osd.8.smithi159.stdout:Sep 10 10:06:39 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-8[53439]: 2024-09-10T10:06:39.258+0000 7f7417907640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-10T10:06:04.917990+0000 front 2024-09-10T10:06:04.918013+0000 (oldest deadline 2024-09-10T10:06:27.818198+0000) 2024-09-10T10:06:39.600 INFO:journalctl@ceph.osd.8.smithi159.stdout:Sep 10 10:06:39 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-8[53439]: 2024-09-10T10:06:39.258+0000 7f7417907640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-10T10:06:13.119605+0000 front 2024-09-10T10:06:13.119551+0000 (oldest deadline 2024-09-10T10:06:36.619665+0000) 2024-09-10T10:06:39.634 INFO:journalctl@ceph.osd.10.smithi102.stdout:Sep 10 10:06:39 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-10[59874]: 2024-09-10T10:06:39.535+0000 7f40f3335640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-10T10:05:49.575097+0000 front 2024-09-10T10:05:49.575141+0000 (oldest deadline 2024-09-10T10:06:13.075035+0000) 2024-09-10T10:06:39.634 INFO:journalctl@ceph.osd.10.smithi102.stdout:Sep 10 10:06:39 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-10[59874]: 2024-09-10T10:06:39.535+0000 7f40f3335640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-09-10T10:05:54.176094+0000 front 2024-09-10T10:05:54.176115+0000 (oldest deadline 2024-09-10T10:06:19.376783+0000) 2024-09-10T10:06:39.635 INFO:journalctl@ceph.osd.10.smithi102.stdout:Sep 10 10:06:39 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-10[59874]: 2024-09-10T10:06:39.535+0000 7f40f3335640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-10T10:06:03.378660+0000 front 2024-09-10T10:06:03.378719+0000 (oldest deadline 2024-09-10T10:06:32.178984+0000) 2024-09-10T10:06:39.635 INFO:journalctl@ceph.osd.10.smithi102.stdout:Sep 10 10:06:39 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-10[59874]: 2024-09-10T10:06:39.535+0000 7f40f3335640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-10T10:06:12.179545+0000 front 2024-09-10T10:06:12.179581+0000 (oldest deadline 2024-09-10T10:06:36.879462+0000) 2024-09-10T10:06:39.635 INFO:journalctl@ceph.osd.7.smithi102.stdout:Sep 10 10:06:39 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-7[52880]: 2024-09-10T10:06:39.328+0000 7ff941357640 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-09-10T10:05:55.423434+0000 front 2024-09-10T10:05:55.423389+0000 (oldest deadline 2024-09-10T10:06:19.424367+0000) 2024-09-10T10:06:39.635 INFO:journalctl@ceph.osd.7.smithi102.stdout:Sep 10 10:06:39 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-7[52880]: 2024-09-10T10:06:39.328+0000 7ff941357640 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-10T10:06:04.025965+0000 front 2024-09-10T10:06:04.025864+0000 (oldest deadline 2024-09-10T10:06:28.726043+0000) 2024-09-10T10:06:39.635 INFO:journalctl@ceph.osd.7.smithi102.stdout:Sep 10 10:06:39 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-7[52880]: 2024-09-10T10:06:39.328+0000 7ff941357640 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-10T10:06:12.827133+0000 front 2024-09-10T10:06:12.827237+0000 (oldest deadline 2024-09-10T10:06:38.027668+0000) 2024-09-10T10:06:40.100 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 10 10:06:39 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-11[60437]: 2024-09-10T10:06:39.774+0000 7fa78e4d3640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-10T10:05:49.428950+0000 front 2024-09-10T10:05:49.429031+0000 (oldest deadline 2024-09-10T10:06:11.128800+0000) 2024-09-10T10:06:40.100 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 10 10:06:39 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-11[60437]: 2024-09-10T10:06:39.774+0000 7fa78e4d3640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-10T10:06:03.430910+0000 front 2024-09-10T10:06:03.430889+0000 (oldest deadline 2024-09-10T10:06:29.331091+0000) 2024-09-10T10:06:40.100 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 10 10:06:39 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-11[60437]: 2024-09-10T10:06:39.774+0000 7fa78e4d3640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-10T10:06:13.432384+0000 front 2024-09-10T10:06:13.432489+0000 (oldest deadline 2024-09-10T10:06:35.732466+0000) 2024-09-10T10:06:40.556 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 10 10:06:40 smithi005 podman[144117]: 2024-09-10 10:06:40.126590995 +0000 UTC m=+5.820189433 container died d61f28e1ca26dd5eeb0ee55b3daef19ee8906fab4731d944539e2f5a0a14fe9e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-6, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-09-10T10:06:40.557 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 10 10:06:40 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-9[65288]: 2024-09-10T10:06:40.276+0000 7f84af200640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-10T10:05:47.346189+0000 front 2024-09-10T10:05:47.346220+0000 (oldest deadline 2024-09-10T10:06:12.646451+0000) 2024-09-10T10:06:40.557 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 10 10:06:40 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-9[65288]: 2024-09-10T10:06:40.276+0000 7f84af200640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-09-10T10:05:57.348185+0000 front 2024-09-10T10:05:57.348335+0000 (oldest deadline 2024-09-10T10:06:23.248210+0000) 2024-09-10T10:06:40.557 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 10 10:06:40 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-9[65288]: 2024-09-10T10:06:40.276+0000 7f84af200640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-10T10:06:03.249484+0000 front 2024-09-10T10:06:03.249185+0000 (oldest deadline 2024-09-10T10:06:26.749283+0000) 2024-09-10T10:06:40.557 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 10 10:06:40 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-9[65288]: 2024-09-10T10:06:40.276+0000 7f84af200640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-10T10:06:12.650860+0000 front 2024-09-10T10:06:12.650900+0000 (oldest deadline 2024-09-10T10:06:38.551142+0000) 2024-09-10T10:06:40.600 INFO:journalctl@ceph.osd.8.smithi159.stdout:Sep 10 10:06:40 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-8[53439]: 2024-09-10T10:06:40.251+0000 7f7417907640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-09-10T10:05:57.316774+0000 front 2024-09-10T10:05:57.316749+0000 (oldest deadline 2024-09-10T10:06:20.816694+0000) 2024-09-10T10:06:40.601 INFO:journalctl@ceph.osd.8.smithi159.stdout:Sep 10 10:06:40 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-8[53439]: 2024-09-10T10:06:40.251+0000 7f7417907640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-10T10:06:04.917990+0000 front 2024-09-10T10:06:04.918013+0000 (oldest deadline 2024-09-10T10:06:27.818198+0000) 2024-09-10T10:06:40.601 INFO:journalctl@ceph.osd.8.smithi159.stdout:Sep 10 10:06:40 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-8[53439]: 2024-09-10T10:06:40.251+0000 7f7417907640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-10T10:06:13.119605+0000 front 2024-09-10T10:06:13.119551+0000 (oldest deadline 2024-09-10T10:06:36.619665+0000) 2024-09-10T10:06:40.634 INFO:journalctl@ceph.osd.7.smithi102.stdout:Sep 10 10:06:40 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-7[52880]: 2024-09-10T10:06:40.332+0000 7ff941357640 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-09-10T10:05:55.423434+0000 front 2024-09-10T10:05:55.423389+0000 (oldest deadline 2024-09-10T10:06:19.424367+0000) 2024-09-10T10:06:40.635 INFO:journalctl@ceph.osd.7.smithi102.stdout:Sep 10 10:06:40 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-7[52880]: 2024-09-10T10:06:40.332+0000 7ff941357640 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-10T10:06:04.025965+0000 front 2024-09-10T10:06:04.025864+0000 (oldest deadline 2024-09-10T10:06:28.726043+0000) 2024-09-10T10:06:40.635 INFO:journalctl@ceph.osd.7.smithi102.stdout:Sep 10 10:06:40 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-7[52880]: 2024-09-10T10:06:40.332+0000 7ff941357640 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-10T10:06:12.827133+0000 front 2024-09-10T10:06:12.827237+0000 (oldest deadline 2024-09-10T10:06:38.027668+0000) 2024-09-10T10:06:40.635 INFO:journalctl@ceph.osd.10.smithi102.stdout:Sep 10 10:06:40 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-10[59874]: 2024-09-10T10:06:40.514+0000 7f40f3335640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-10T10:05:49.575097+0000 front 2024-09-10T10:05:49.575141+0000 (oldest deadline 2024-09-10T10:06:13.075035+0000) 2024-09-10T10:06:40.635 INFO:journalctl@ceph.osd.10.smithi102.stdout:Sep 10 10:06:40 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-10[59874]: 2024-09-10T10:06:40.514+0000 7f40f3335640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-09-10T10:05:54.176094+0000 front 2024-09-10T10:05:54.176115+0000 (oldest deadline 2024-09-10T10:06:19.376783+0000) 2024-09-10T10:06:40.635 INFO:journalctl@ceph.osd.10.smithi102.stdout:Sep 10 10:06:40 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-10[59874]: 2024-09-10T10:06:40.514+0000 7f40f3335640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-10T10:06:03.378660+0000 front 2024-09-10T10:06:03.378719+0000 (oldest deadline 2024-09-10T10:06:32.178984+0000) 2024-09-10T10:06:40.635 INFO:journalctl@ceph.osd.10.smithi102.stdout:Sep 10 10:06:40 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-10[59874]: 2024-09-10T10:06:40.514+0000 7f40f3335640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-10T10:06:12.179545+0000 front 2024-09-10T10:06:12.179581+0000 (oldest deadline 2024-09-10T10:06:36.879462+0000) 2024-09-10T10:06:40.851 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 10 10:06:40 smithi005 podman[144117]: 2024-09-10 10:06:40.830432494 +0000 UTC m=+6.524030922 container cleanup d61f28e1ca26dd5eeb0ee55b3daef19ee8906fab4731d944539e2f5a0a14fe9e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-6, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T10:06:40.851 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 10 10:06:40 smithi005 bash[144117]: ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-6 2024-09-10T10:06:41.100 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 10 10:06:40 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-11[60437]: 2024-09-10T10:06:40.781+0000 7fa78e4d3640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-10T10:05:49.428950+0000 front 2024-09-10T10:05:49.429031+0000 (oldest deadline 2024-09-10T10:06:11.128800+0000) 2024-09-10T10:06:41.100 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 10 10:06:40 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-11[60437]: 2024-09-10T10:06:40.781+0000 7fa78e4d3640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-10T10:06:03.430910+0000 front 2024-09-10T10:06:03.430889+0000 (oldest deadline 2024-09-10T10:06:29.331091+0000) 2024-09-10T10:06:41.101 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 10 10:06:40 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-11[60437]: 2024-09-10T10:06:40.781+0000 7fa78e4d3640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-10T10:06:13.432384+0000 front 2024-09-10T10:06:13.432489+0000 (oldest deadline 2024-09-10T10:06:35.732466+0000) 2024-09-10T10:06:41.544 INFO:journalctl@ceph.osd.7.smithi102.stdout:Sep 10 10:06:41 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-7[52880]: 2024-09-10T10:06:41.291+0000 7ff941357640 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-09-10T10:05:55.423434+0000 front 2024-09-10T10:05:55.423389+0000 (oldest deadline 2024-09-10T10:06:19.424367+0000) 2024-09-10T10:06:41.544 INFO:journalctl@ceph.osd.7.smithi102.stdout:Sep 10 10:06:41 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-7[52880]: 2024-09-10T10:06:41.291+0000 7ff941357640 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-10T10:06:04.025965+0000 front 2024-09-10T10:06:04.025864+0000 (oldest deadline 2024-09-10T10:06:28.726043+0000) 2024-09-10T10:06:41.544 INFO:journalctl@ceph.osd.7.smithi102.stdout:Sep 10 10:06:41 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-7[52880]: 2024-09-10T10:06:41.291+0000 7ff941357640 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-10T10:06:12.827133+0000 front 2024-09-10T10:06:12.827237+0000 (oldest deadline 2024-09-10T10:06:38.027668+0000) 2024-09-10T10:06:41.556 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 10 10:06:41 smithi005 podman[144136]: 2024-09-10 10:06:41.113422905 +0000 UTC m=+0.983252822 container remove d61f28e1ca26dd5eeb0ee55b3daef19ee8906fab4731d944539e2f5a0a14fe9e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-6, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, io.buildah.version=1.37.2) 2024-09-10T10:06:41.556 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 10 10:06:41 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-9[65288]: 2024-09-10T10:06:41.280+0000 7f84af200640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-10T10:05:47.346189+0000 front 2024-09-10T10:05:47.346220+0000 (oldest deadline 2024-09-10T10:06:12.646451+0000) 2024-09-10T10:06:41.556 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 10 10:06:41 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-9[65288]: 2024-09-10T10:06:41.281+0000 7f84af200640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-09-10T10:05:57.348185+0000 front 2024-09-10T10:05:57.348335+0000 (oldest deadline 2024-09-10T10:06:23.248210+0000) 2024-09-10T10:06:41.556 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 10 10:06:41 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-9[65288]: 2024-09-10T10:06:41.281+0000 7f84af200640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-10T10:06:03.249484+0000 front 2024-09-10T10:06:03.249185+0000 (oldest deadline 2024-09-10T10:06:26.749283+0000) 2024-09-10T10:06:41.556 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 10 10:06:41 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-9[65288]: 2024-09-10T10:06:41.281+0000 7f84af200640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-10T10:06:12.650860+0000 front 2024-09-10T10:06:12.650900+0000 (oldest deadline 2024-09-10T10:06:38.551142+0000) 2024-09-10T10:06:41.600 INFO:journalctl@ceph.osd.8.smithi159.stdout:Sep 10 10:06:41 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-8[53439]: 2024-09-10T10:06:41.281+0000 7f7417907640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-09-10T10:05:57.316774+0000 front 2024-09-10T10:05:57.316749+0000 (oldest deadline 2024-09-10T10:06:20.816694+0000) 2024-09-10T10:06:41.600 INFO:journalctl@ceph.osd.8.smithi159.stdout:Sep 10 10:06:41 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-8[53439]: 2024-09-10T10:06:41.281+0000 7f7417907640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-10T10:06:04.917990+0000 front 2024-09-10T10:06:04.918013+0000 (oldest deadline 2024-09-10T10:06:27.818198+0000) 2024-09-10T10:06:41.600 INFO:journalctl@ceph.osd.8.smithi159.stdout:Sep 10 10:06:41 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-8[53439]: 2024-09-10T10:06:41.281+0000 7f7417907640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-10T10:06:13.119605+0000 front 2024-09-10T10:06:13.119551+0000 (oldest deadline 2024-09-10T10:06:36.619665+0000) 2024-09-10T10:06:41.802 INFO:journalctl@ceph.osd.10.smithi102.stdout:Sep 10 10:06:41 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-10[59874]: 2024-09-10T10:06:41.542+0000 7f40f3335640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-10T10:05:49.575097+0000 front 2024-09-10T10:05:49.575141+0000 (oldest deadline 2024-09-10T10:06:13.075035+0000) 2024-09-10T10:06:41.802 INFO:journalctl@ceph.osd.10.smithi102.stdout:Sep 10 10:06:41 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-10[59874]: 2024-09-10T10:06:41.542+0000 7f40f3335640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-09-10T10:05:54.176094+0000 front 2024-09-10T10:05:54.176115+0000 (oldest deadline 2024-09-10T10:06:19.376783+0000) 2024-09-10T10:06:41.802 INFO:journalctl@ceph.osd.10.smithi102.stdout:Sep 10 10:06:41 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-10[59874]: 2024-09-10T10:06:41.542+0000 7f40f3335640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-10T10:06:03.378660+0000 front 2024-09-10T10:06:03.378719+0000 (oldest deadline 2024-09-10T10:06:32.178984+0000) 2024-09-10T10:06:41.803 INFO:journalctl@ceph.osd.10.smithi102.stdout:Sep 10 10:06:41 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-10[59874]: 2024-09-10T10:06:41.542+0000 7f40f3335640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-10T10:06:12.179545+0000 front 2024-09-10T10:06:12.179581+0000 (oldest deadline 2024-09-10T10:06:36.879462+0000) 2024-09-10T10:06:41.961 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 10 10:06:41 smithi005 podman[144223]: 2024-09-10 10:06:41.557388831 +0000 UTC m=+0.027487622 image pull 41d67112835771c3efca60c6b8fe297c7ddaecdc26cea26fd28aa18636e49005 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 2024-09-10T10:06:41.961 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 10 10:06:41 smithi005 podman[144223]: 2024-09-10 10:06:41.815965128 +0000 UTC m=+0.286063913 container create 87a08269c919e637377cdfd14364fcd34d9aa68d3042014d66b271e815b6a7a1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-6-deactivate, ceph=True, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-09-10T10:06:42.100 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 10 10:06:41 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-11[60437]: 2024-09-10T10:06:41.820+0000 7fa78e4d3640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-10T10:05:49.428950+0000 front 2024-09-10T10:05:49.429031+0000 (oldest deadline 2024-09-10T10:06:11.128800+0000) 2024-09-10T10:06:42.101 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 10 10:06:41 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-11[60437]: 2024-09-10T10:06:41.820+0000 7fa78e4d3640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-10T10:06:03.430910+0000 front 2024-09-10T10:06:03.430889+0000 (oldest deadline 2024-09-10T10:06:29.331091+0000) 2024-09-10T10:06:42.101 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 10 10:06:41 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-11[60437]: 2024-09-10T10:06:41.820+0000 7fa78e4d3640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-10T10:06:13.432384+0000 front 2024-09-10T10:06:13.432489+0000 (oldest deadline 2024-09-10T10:06:35.732466+0000) 2024-09-10T10:06:42.236 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 10 10:06:42 smithi005 podman[144223]: 2024-09-10 10:06:42.135467297 +0000 UTC m=+0.605566076 container init 87a08269c919e637377cdfd14364fcd34d9aa68d3042014d66b271e815b6a7a1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-6-deactivate, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_CLEAN=True, ceph=True, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b) 2024-09-10T10:06:42.236 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 10 10:06:42 smithi005 podman[144223]: 2024-09-10 10:06:42.14246414 +0000 UTC m=+0.612562920 container start 87a08269c919e637377cdfd14364fcd34d9aa68d3042014d66b271e815b6a7a1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-6-deactivate, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.vendor=CentOS) 2024-09-10T10:06:42.236 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 10 10:06:42 smithi005 podman[144223]: 2024-09-10 10:06:42.235338456 +0000 UTC m=+0.705437245 container attach 87a08269c919e637377cdfd14364fcd34d9aa68d3042014d66b271e815b6a7a1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-6-deactivate, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-10T10:06:42.545 INFO:journalctl@ceph.osd.7.smithi102.stdout:Sep 10 10:06:42 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-7[52880]: 2024-09-10T10:06:42.273+0000 7ff941357640 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-09-10T10:05:55.423434+0000 front 2024-09-10T10:05:55.423389+0000 (oldest deadline 2024-09-10T10:06:19.424367+0000) 2024-09-10T10:06:42.545 INFO:journalctl@ceph.osd.7.smithi102.stdout:Sep 10 10:06:42 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-7[52880]: 2024-09-10T10:06:42.273+0000 7ff941357640 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-10T10:06:04.025965+0000 front 2024-09-10T10:06:04.025864+0000 (oldest deadline 2024-09-10T10:06:28.726043+0000) 2024-09-10T10:06:42.545 INFO:journalctl@ceph.osd.7.smithi102.stdout:Sep 10 10:06:42 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-7[52880]: 2024-09-10T10:06:42.273+0000 7ff941357640 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-10T10:06:12.827133+0000 front 2024-09-10T10:06:12.827237+0000 (oldest deadline 2024-09-10T10:06:38.027668+0000) 2024-09-10T10:06:42.556 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 10 10:06:42 smithi005 podman[144246]: 2024-09-10 10:06:42.453108653 +0000 UTC m=+0.026686968 container died 87a08269c919e637377cdfd14364fcd34d9aa68d3042014d66b271e815b6a7a1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-6-deactivate, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , ceph=True, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-10T10:06:42.557 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 10 10:06:42 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-9[65288]: 2024-09-10T10:06:42.272+0000 7f84af200640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-10T10:05:47.346189+0000 front 2024-09-10T10:05:47.346220+0000 (oldest deadline 2024-09-10T10:06:12.646451+0000) 2024-09-10T10:06:42.557 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 10 10:06:42 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-9[65288]: 2024-09-10T10:06:42.272+0000 7f84af200640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-09-10T10:05:57.348185+0000 front 2024-09-10T10:05:57.348335+0000 (oldest deadline 2024-09-10T10:06:23.248210+0000) 2024-09-10T10:06:42.557 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 10 10:06:42 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-9[65288]: 2024-09-10T10:06:42.272+0000 7f84af200640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-10T10:06:03.249484+0000 front 2024-09-10T10:06:03.249185+0000 (oldest deadline 2024-09-10T10:06:26.749283+0000) 2024-09-10T10:06:42.557 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 10 10:06:42 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-9[65288]: 2024-09-10T10:06:42.272+0000 7f84af200640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-10T10:06:12.650860+0000 front 2024-09-10T10:06:12.650900+0000 (oldest deadline 2024-09-10T10:06:38.551142+0000) 2024-09-10T10:06:42.600 INFO:journalctl@ceph.osd.8.smithi159.stdout:Sep 10 10:06:42 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-8[53439]: 2024-09-10T10:06:42.256+0000 7f7417907640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-09-10T10:05:57.316774+0000 front 2024-09-10T10:05:57.316749+0000 (oldest deadline 2024-09-10T10:06:20.816694+0000) 2024-09-10T10:06:42.600 INFO:journalctl@ceph.osd.8.smithi159.stdout:Sep 10 10:06:42 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-8[53439]: 2024-09-10T10:06:42.256+0000 7f7417907640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-10T10:06:04.917990+0000 front 2024-09-10T10:06:04.918013+0000 (oldest deadline 2024-09-10T10:06:27.818198+0000) 2024-09-10T10:06:42.601 INFO:journalctl@ceph.osd.8.smithi159.stdout:Sep 10 10:06:42 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-8[53439]: 2024-09-10T10:06:42.256+0000 7f7417907640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-10T10:06:13.119605+0000 front 2024-09-10T10:06:13.119551+0000 (oldest deadline 2024-09-10T10:06:36.619665+0000) 2024-09-10T10:06:42.884 INFO:journalctl@ceph.osd.10.smithi102.stdout:Sep 10 10:06:42 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-10[59874]: 2024-09-10T10:06:42.543+0000 7f40f3335640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-10T10:05:49.575097+0000 front 2024-09-10T10:05:49.575141+0000 (oldest deadline 2024-09-10T10:06:13.075035+0000) 2024-09-10T10:06:42.884 INFO:journalctl@ceph.osd.10.smithi102.stdout:Sep 10 10:06:42 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-10[59874]: 2024-09-10T10:06:42.543+0000 7f40f3335640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-09-10T10:05:54.176094+0000 front 2024-09-10T10:05:54.176115+0000 (oldest deadline 2024-09-10T10:06:19.376783+0000) 2024-09-10T10:06:42.885 INFO:journalctl@ceph.osd.10.smithi102.stdout:Sep 10 10:06:42 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-10[59874]: 2024-09-10T10:06:42.543+0000 7f40f3335640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-10T10:06:03.378660+0000 front 2024-09-10T10:06:03.378719+0000 (oldest deadline 2024-09-10T10:06:32.178984+0000) 2024-09-10T10:06:42.885 INFO:journalctl@ceph.osd.10.smithi102.stdout:Sep 10 10:06:42 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-10[59874]: 2024-09-10T10:06:42.543+0000 7f40f3335640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-10T10:06:12.179545+0000 front 2024-09-10T10:06:12.179581+0000 (oldest deadline 2024-09-10T10:06:36.879462+0000) 2024-09-10T10:06:43.100 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 10 10:06:42 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-11[60437]: 2024-09-10T10:06:42.771+0000 7fa78e4d3640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-10T10:05:49.428950+0000 front 2024-09-10T10:05:49.429031+0000 (oldest deadline 2024-09-10T10:06:11.128800+0000) 2024-09-10T10:06:43.100 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 10 10:06:42 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-11[60437]: 2024-09-10T10:06:42.771+0000 7fa78e4d3640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-10T10:06:03.430910+0000 front 2024-09-10T10:06:03.430889+0000 (oldest deadline 2024-09-10T10:06:29.331091+0000) 2024-09-10T10:06:43.100 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 10 10:06:42 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-11[60437]: 2024-09-10T10:06:42.771+0000 7fa78e4d3640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-10T10:06:13.432384+0000 front 2024-09-10T10:06:13.432489+0000 (oldest deadline 2024-09-10T10:06:35.732466+0000) 2024-09-10T10:06:43.174 DEBUG:teuthology.orchestra.run.smithi005:> sudo pkill -f 'journalctl -f -n 0 -u ceph-bade4b50-6f56-11ef-bcea-c7b262605968@osd.6.service' 2024-09-10T10:06:43.251 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T10:06:43.251 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-09-10T10:06:43.251 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-09-10T10:06:43.251 DEBUG:teuthology.orchestra.run.smithi102:> sudo systemctl stop ceph-bade4b50-6f56-11ef-bcea-c7b262605968@osd.7 2024-09-10T10:06:43.545 INFO:journalctl@ceph.osd.7.smithi102.stdout:Sep 10 10:06:43 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-7[52880]: 2024-09-10T10:06:43.241+0000 7ff941357640 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-09-10T10:05:55.423434+0000 front 2024-09-10T10:05:55.423389+0000 (oldest deadline 2024-09-10T10:06:19.424367+0000) 2024-09-10T10:06:43.546 INFO:journalctl@ceph.osd.7.smithi102.stdout:Sep 10 10:06:43 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-7[52880]: 2024-09-10T10:06:43.241+0000 7ff941357640 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-10T10:06:04.025965+0000 front 2024-09-10T10:06:04.025864+0000 (oldest deadline 2024-09-10T10:06:28.726043+0000) 2024-09-10T10:06:43.546 INFO:journalctl@ceph.osd.7.smithi102.stdout:Sep 10 10:06:43 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-7[52880]: 2024-09-10T10:06:43.241+0000 7ff941357640 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-10T10:06:12.827133+0000 front 2024-09-10T10:06:12.827237+0000 (oldest deadline 2024-09-10T10:06:38.027668+0000) 2024-09-10T10:06:43.546 INFO:journalctl@ceph.osd.7.smithi102.stdout:Sep 10 10:06:43 smithi102 systemd[1]: Stopping Ceph osd.7 for bade4b50-6f56-11ef-bcea-c7b262605968... 2024-09-10T10:06:43.556 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 10 10:06:43 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-9[65288]: 2024-09-10T10:06:43.255+0000 7f84af200640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-10T10:05:47.346189+0000 front 2024-09-10T10:05:47.346220+0000 (oldest deadline 2024-09-10T10:06:12.646451+0000) 2024-09-10T10:06:43.556 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 10 10:06:43 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-9[65288]: 2024-09-10T10:06:43.255+0000 7f84af200640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-09-10T10:05:57.348185+0000 front 2024-09-10T10:05:57.348335+0000 (oldest deadline 2024-09-10T10:06:23.248210+0000) 2024-09-10T10:06:43.556 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 10 10:06:43 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-9[65288]: 2024-09-10T10:06:43.255+0000 7f84af200640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-10T10:06:03.249484+0000 front 2024-09-10T10:06:03.249185+0000 (oldest deadline 2024-09-10T10:06:26.749283+0000) 2024-09-10T10:06:43.556 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 10 10:06:43 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-9[65288]: 2024-09-10T10:06:43.255+0000 7f84af200640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-10T10:06:12.650860+0000 front 2024-09-10T10:06:12.650900+0000 (oldest deadline 2024-09-10T10:06:38.551142+0000) 2024-09-10T10:06:43.600 INFO:journalctl@ceph.osd.8.smithi159.stdout:Sep 10 10:06:43 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-8[53439]: 2024-09-10T10:06:43.276+0000 7f7417907640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-09-10T10:05:57.316774+0000 front 2024-09-10T10:05:57.316749+0000 (oldest deadline 2024-09-10T10:06:20.816694+0000) 2024-09-10T10:06:43.600 INFO:journalctl@ceph.osd.8.smithi159.stdout:Sep 10 10:06:43 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-8[53439]: 2024-09-10T10:06:43.276+0000 7f7417907640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-10T10:06:04.917990+0000 front 2024-09-10T10:06:04.918013+0000 (oldest deadline 2024-09-10T10:06:27.818198+0000) 2024-09-10T10:06:43.600 INFO:journalctl@ceph.osd.8.smithi159.stdout:Sep 10 10:06:43 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-8[53439]: 2024-09-10T10:06:43.276+0000 7f7417907640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-10T10:06:13.119605+0000 front 2024-09-10T10:06:13.119551+0000 (oldest deadline 2024-09-10T10:06:36.619665+0000) 2024-09-10T10:06:43.884 INFO:journalctl@ceph.osd.7.smithi102.stdout:Sep 10 10:06:43 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-7[52880]: 2024-09-10T10:06:43.544+0000 7ff944d41640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T10:06:43.885 INFO:journalctl@ceph.osd.7.smithi102.stdout:Sep 10 10:06:43 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-7[52880]: 2024-09-10T10:06:43.544+0000 7ff944d41640 -1 osd.7 100 *** Got signal Terminated *** 2024-09-10T10:06:43.885 INFO:journalctl@ceph.osd.7.smithi102.stdout:Sep 10 10:06:43 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-7[52880]: 2024-09-10T10:06:43.544+0000 7ff944d41640 -1 osd.7 100 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-10T10:06:43.885 INFO:journalctl@ceph.osd.10.smithi102.stdout:Sep 10 10:06:43 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-10[59874]: 2024-09-10T10:06:43.559+0000 7f40f3335640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-10T10:05:49.575097+0000 front 2024-09-10T10:05:49.575141+0000 (oldest deadline 2024-09-10T10:06:13.075035+0000) 2024-09-10T10:06:43.885 INFO:journalctl@ceph.osd.10.smithi102.stdout:Sep 10 10:06:43 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-10[59874]: 2024-09-10T10:06:43.559+0000 7f40f3335640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-09-10T10:05:54.176094+0000 front 2024-09-10T10:05:54.176115+0000 (oldest deadline 2024-09-10T10:06:19.376783+0000) 2024-09-10T10:06:43.885 INFO:journalctl@ceph.osd.10.smithi102.stdout:Sep 10 10:06:43 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-10[59874]: 2024-09-10T10:06:43.559+0000 7f40f3335640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-10T10:06:03.378660+0000 front 2024-09-10T10:06:03.378719+0000 (oldest deadline 2024-09-10T10:06:32.178984+0000) 2024-09-10T10:06:43.885 INFO:journalctl@ceph.osd.10.smithi102.stdout:Sep 10 10:06:43 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-10[59874]: 2024-09-10T10:06:43.559+0000 7f40f3335640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-10T10:06:12.179545+0000 front 2024-09-10T10:06:12.179581+0000 (oldest deadline 2024-09-10T10:06:36.879462+0000) 2024-09-10T10:06:44.100 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 10 10:06:43 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-11[60437]: 2024-09-10T10:06:43.732+0000 7fa78e4d3640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-10T10:05:49.428950+0000 front 2024-09-10T10:05:49.429031+0000 (oldest deadline 2024-09-10T10:06:11.128800+0000) 2024-09-10T10:06:44.100 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 10 10:06:43 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-11[60437]: 2024-09-10T10:06:43.732+0000 7fa78e4d3640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-10T10:06:03.430910+0000 front 2024-09-10T10:06:03.430889+0000 (oldest deadline 2024-09-10T10:06:29.331091+0000) 2024-09-10T10:06:44.100 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 10 10:06:43 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-11[60437]: 2024-09-10T10:06:43.732+0000 7fa78e4d3640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-10T10:06:13.432384+0000 front 2024-09-10T10:06:13.432489+0000 (oldest deadline 2024-09-10T10:06:35.732466+0000) 2024-09-10T10:06:44.530 INFO:journalctl@ceph.osd.7.smithi102.stdout:Sep 10 10:06:44 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-7[52880]: 2024-09-10T10:06:44.278+0000 7ff941357640 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-09-10T10:05:55.423434+0000 front 2024-09-10T10:05:55.423389+0000 (oldest deadline 2024-09-10T10:06:19.424367+0000) 2024-09-10T10:06:44.530 INFO:journalctl@ceph.osd.7.smithi102.stdout:Sep 10 10:06:44 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-7[52880]: 2024-09-10T10:06:44.278+0000 7ff941357640 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-10T10:06:04.025965+0000 front 2024-09-10T10:06:04.025864+0000 (oldest deadline 2024-09-10T10:06:28.726043+0000) 2024-09-10T10:06:44.530 INFO:journalctl@ceph.osd.7.smithi102.stdout:Sep 10 10:06:44 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-7[52880]: 2024-09-10T10:06:44.278+0000 7ff941357640 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-10T10:06:12.827133+0000 front 2024-09-10T10:06:12.827237+0000 (oldest deadline 2024-09-10T10:06:38.027668+0000) 2024-09-10T10:06:44.530 INFO:journalctl@ceph.osd.7.smithi102.stdout:Sep 10 10:06:44 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-7[52880]: 2024-09-10T10:06:44.278+0000 7ff941357640 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.102:6812 osd.4 since back 2024-09-10T10:06:18.528387+0000 front 2024-09-10T10:06:18.528339+0000 (oldest deadline 2024-09-10T10:06:43.828582+0000) 2024-09-10T10:06:44.556 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 10 10:06:44 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-9[65288]: 2024-09-10T10:06:44.221+0000 7f84af200640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-10T10:05:47.346189+0000 front 2024-09-10T10:05:47.346220+0000 (oldest deadline 2024-09-10T10:06:12.646451+0000) 2024-09-10T10:06:44.556 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 10 10:06:44 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-9[65288]: 2024-09-10T10:06:44.221+0000 7f84af200640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-09-10T10:05:57.348185+0000 front 2024-09-10T10:05:57.348335+0000 (oldest deadline 2024-09-10T10:06:23.248210+0000) 2024-09-10T10:06:44.556 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 10 10:06:44 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-9[65288]: 2024-09-10T10:06:44.221+0000 7f84af200640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-10T10:06:03.249484+0000 front 2024-09-10T10:06:03.249185+0000 (oldest deadline 2024-09-10T10:06:26.749283+0000) 2024-09-10T10:06:44.556 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 10 10:06:44 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-9[65288]: 2024-09-10T10:06:44.221+0000 7f84af200640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-10T10:06:12.650860+0000 front 2024-09-10T10:06:12.650900+0000 (oldest deadline 2024-09-10T10:06:38.551142+0000) 2024-09-10T10:06:44.600 INFO:journalctl@ceph.osd.8.smithi159.stdout:Sep 10 10:06:44 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-8[53439]: 2024-09-10T10:06:44.234+0000 7f7417907640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-09-10T10:05:57.316774+0000 front 2024-09-10T10:05:57.316749+0000 (oldest deadline 2024-09-10T10:06:20.816694+0000) 2024-09-10T10:06:44.600 INFO:journalctl@ceph.osd.8.smithi159.stdout:Sep 10 10:06:44 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-8[53439]: 2024-09-10T10:06:44.234+0000 7f7417907640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-10T10:06:04.917990+0000 front 2024-09-10T10:06:04.918013+0000 (oldest deadline 2024-09-10T10:06:27.818198+0000) 2024-09-10T10:06:44.600 INFO:journalctl@ceph.osd.8.smithi159.stdout:Sep 10 10:06:44 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-8[53439]: 2024-09-10T10:06:44.234+0000 7f7417907640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-10T10:06:13.119605+0000 front 2024-09-10T10:06:13.119551+0000 (oldest deadline 2024-09-10T10:06:36.619665+0000) 2024-09-10T10:06:44.884 INFO:journalctl@ceph.osd.10.smithi102.stdout:Sep 10 10:06:44 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-10[59874]: 2024-09-10T10:06:44.528+0000 7f40f3335640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-10T10:05:49.575097+0000 front 2024-09-10T10:05:49.575141+0000 (oldest deadline 2024-09-10T10:06:13.075035+0000) 2024-09-10T10:06:44.885 INFO:journalctl@ceph.osd.10.smithi102.stdout:Sep 10 10:06:44 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-10[59874]: 2024-09-10T10:06:44.528+0000 7f40f3335640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-09-10T10:05:54.176094+0000 front 2024-09-10T10:05:54.176115+0000 (oldest deadline 2024-09-10T10:06:19.376783+0000) 2024-09-10T10:06:44.885 INFO:journalctl@ceph.osd.10.smithi102.stdout:Sep 10 10:06:44 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-10[59874]: 2024-09-10T10:06:44.528+0000 7f40f3335640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-10T10:06:03.378660+0000 front 2024-09-10T10:06:03.378719+0000 (oldest deadline 2024-09-10T10:06:32.178984+0000) 2024-09-10T10:06:44.885 INFO:journalctl@ceph.osd.10.smithi102.stdout:Sep 10 10:06:44 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-10[59874]: 2024-09-10T10:06:44.528+0000 7f40f3335640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-10T10:06:12.179545+0000 front 2024-09-10T10:06:12.179581+0000 (oldest deadline 2024-09-10T10:06:36.879462+0000) 2024-09-10T10:06:44.885 INFO:journalctl@ceph.osd.10.smithi102.stdout:Sep 10 10:06:44 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-10[59874]: 2024-09-10T10:06:44.528+0000 7f40f3335640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.102:6812 osd.4 since back 2024-09-10T10:06:22.581662+0000 front 2024-09-10T10:06:22.581640+0000 (oldest deadline 2024-09-10T10:06:44.281620+0000) 2024-09-10T10:06:45.100 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 10 10:06:44 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-11[60437]: 2024-09-10T10:06:44.705+0000 7fa78e4d3640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-10T10:05:49.428950+0000 front 2024-09-10T10:05:49.429031+0000 (oldest deadline 2024-09-10T10:06:11.128800+0000) 2024-09-10T10:06:45.101 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 10 10:06:44 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-11[60437]: 2024-09-10T10:06:44.705+0000 7fa78e4d3640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-10T10:06:03.430910+0000 front 2024-09-10T10:06:03.430889+0000 (oldest deadline 2024-09-10T10:06:29.331091+0000) 2024-09-10T10:06:45.101 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 10 10:06:44 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-11[60437]: 2024-09-10T10:06:44.705+0000 7fa78e4d3640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-10T10:06:13.432384+0000 front 2024-09-10T10:06:13.432489+0000 (oldest deadline 2024-09-10T10:06:35.732466+0000) 2024-09-10T10:06:45.101 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 10 10:06:44 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-11[60437]: 2024-09-10T10:06:44.705+0000 7fa78e4d3640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.102:6812 osd.4 since back 2024-09-10T10:06:22.734557+0000 front 2024-09-10T10:06:22.734365+0000 (oldest deadline 2024-09-10T10:06:44.434370+0000) 2024-09-10T10:06:45.556 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 10 10:06:45 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-9[65288]: 2024-09-10T10:06:45.219+0000 7f84af200640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-10T10:05:47.346189+0000 front 2024-09-10T10:05:47.346220+0000 (oldest deadline 2024-09-10T10:06:12.646451+0000) 2024-09-10T10:06:45.556 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 10 10:06:45 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-9[65288]: 2024-09-10T10:06:45.219+0000 7f84af200640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-09-10T10:05:57.348185+0000 front 2024-09-10T10:05:57.348335+0000 (oldest deadline 2024-09-10T10:06:23.248210+0000) 2024-09-10T10:06:45.556 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 10 10:06:45 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-9[65288]: 2024-09-10T10:06:45.219+0000 7f84af200640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-10T10:06:03.249484+0000 front 2024-09-10T10:06:03.249185+0000 (oldest deadline 2024-09-10T10:06:26.749283+0000) 2024-09-10T10:06:45.556 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 10 10:06:45 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-9[65288]: 2024-09-10T10:06:45.219+0000 7f84af200640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-10T10:06:12.650860+0000 front 2024-09-10T10:06:12.650900+0000 (oldest deadline 2024-09-10T10:06:38.551142+0000) 2024-09-10T10:06:45.557 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 10 10:06:45 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-9[65288]: 2024-09-10T10:06:45.219+0000 7f84af200640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.102:6812 osd.4 since back 2024-09-10T10:06:18.551683+0000 front 2024-09-10T10:06:18.551825+0000 (oldest deadline 2024-09-10T10:06:44.451846+0000) 2024-09-10T10:06:45.600 INFO:journalctl@ceph.osd.8.smithi159.stdout:Sep 10 10:06:45 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-8[53439]: 2024-09-10T10:06:45.227+0000 7f7417907640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-09-10T10:05:57.316774+0000 front 2024-09-10T10:05:57.316749+0000 (oldest deadline 2024-09-10T10:06:20.816694+0000) 2024-09-10T10:06:45.600 INFO:journalctl@ceph.osd.8.smithi159.stdout:Sep 10 10:06:45 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-8[53439]: 2024-09-10T10:06:45.227+0000 7f7417907640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-10T10:06:04.917990+0000 front 2024-09-10T10:06:04.918013+0000 (oldest deadline 2024-09-10T10:06:27.818198+0000) 2024-09-10T10:06:45.600 INFO:journalctl@ceph.osd.8.smithi159.stdout:Sep 10 10:06:45 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-8[53439]: 2024-09-10T10:06:45.227+0000 7f7417907640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-10T10:06:13.119605+0000 front 2024-09-10T10:06:13.119551+0000 (oldest deadline 2024-09-10T10:06:36.619665+0000) 2024-09-10T10:06:45.601 INFO:journalctl@ceph.osd.8.smithi159.stdout:Sep 10 10:06:45 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-8[53439]: 2024-09-10T10:06:45.227+0000 7f7417907640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.102:6812 osd.4 since back 2024-09-10T10:06:21.821307+0000 front 2024-09-10T10:06:21.821315+0000 (oldest deadline 2024-09-10T10:06:44.721409+0000) 2024-09-10T10:06:45.634 INFO:journalctl@ceph.osd.10.smithi102.stdout:Sep 10 10:06:45 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-10[59874]: 2024-09-10T10:06:45.552+0000 7f40f3335640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-10T10:05:49.575097+0000 front 2024-09-10T10:05:49.575141+0000 (oldest deadline 2024-09-10T10:06:13.075035+0000) 2024-09-10T10:06:45.634 INFO:journalctl@ceph.osd.10.smithi102.stdout:Sep 10 10:06:45 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-10[59874]: 2024-09-10T10:06:45.552+0000 7f40f3335640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-09-10T10:05:54.176094+0000 front 2024-09-10T10:05:54.176115+0000 (oldest deadline 2024-09-10T10:06:19.376783+0000) 2024-09-10T10:06:45.635 INFO:journalctl@ceph.osd.10.smithi102.stdout:Sep 10 10:06:45 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-10[59874]: 2024-09-10T10:06:45.552+0000 7f40f3335640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-10T10:06:03.378660+0000 front 2024-09-10T10:06:03.378719+0000 (oldest deadline 2024-09-10T10:06:32.178984+0000) 2024-09-10T10:06:45.635 INFO:journalctl@ceph.osd.10.smithi102.stdout:Sep 10 10:06:45 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-10[59874]: 2024-09-10T10:06:45.552+0000 7f40f3335640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-10T10:06:12.179545+0000 front 2024-09-10T10:06:12.179581+0000 (oldest deadline 2024-09-10T10:06:36.879462+0000) 2024-09-10T10:06:45.635 INFO:journalctl@ceph.osd.10.smithi102.stdout:Sep 10 10:06:45 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-10[59874]: 2024-09-10T10:06:45.552+0000 7f40f3335640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.102:6812 osd.4 since back 2024-09-10T10:06:22.581662+0000 front 2024-09-10T10:06:22.581640+0000 (oldest deadline 2024-09-10T10:06:44.281620+0000) 2024-09-10T10:06:45.635 INFO:journalctl@ceph.osd.7.smithi102.stdout:Sep 10 10:06:45 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-7[52880]: 2024-09-10T10:06:45.317+0000 7ff941357640 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-09-10T10:05:55.423434+0000 front 2024-09-10T10:05:55.423389+0000 (oldest deadline 2024-09-10T10:06:19.424367+0000) 2024-09-10T10:06:45.635 INFO:journalctl@ceph.osd.7.smithi102.stdout:Sep 10 10:06:45 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-7[52880]: 2024-09-10T10:06:45.318+0000 7ff941357640 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-10T10:06:04.025965+0000 front 2024-09-10T10:06:04.025864+0000 (oldest deadline 2024-09-10T10:06:28.726043+0000) 2024-09-10T10:06:45.635 INFO:journalctl@ceph.osd.7.smithi102.stdout:Sep 10 10:06:45 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-7[52880]: 2024-09-10T10:06:45.318+0000 7ff941357640 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-10T10:06:12.827133+0000 front 2024-09-10T10:06:12.827237+0000 (oldest deadline 2024-09-10T10:06:38.027668+0000) 2024-09-10T10:06:45.635 INFO:journalctl@ceph.osd.7.smithi102.stdout:Sep 10 10:06:45 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-7[52880]: 2024-09-10T10:06:45.318+0000 7ff941357640 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.102:6812 osd.4 since back 2024-09-10T10:06:18.528387+0000 front 2024-09-10T10:06:18.528339+0000 (oldest deadline 2024-09-10T10:06:43.828582+0000) 2024-09-10T10:06:46.100 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 10 10:06:45 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-11[60437]: 2024-09-10T10:06:45.656+0000 7fa78e4d3640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-10T10:05:49.428950+0000 front 2024-09-10T10:05:49.429031+0000 (oldest deadline 2024-09-10T10:06:11.128800+0000) 2024-09-10T10:06:46.101 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 10 10:06:45 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-11[60437]: 2024-09-10T10:06:45.656+0000 7fa78e4d3640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-10T10:06:03.430910+0000 front 2024-09-10T10:06:03.430889+0000 (oldest deadline 2024-09-10T10:06:29.331091+0000) 2024-09-10T10:06:46.101 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 10 10:06:45 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-11[60437]: 2024-09-10T10:06:45.656+0000 7fa78e4d3640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-10T10:06:13.432384+0000 front 2024-09-10T10:06:13.432489+0000 (oldest deadline 2024-09-10T10:06:35.732466+0000) 2024-09-10T10:06:46.101 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 10 10:06:45 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-11[60437]: 2024-09-10T10:06:45.656+0000 7fa78e4d3640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.102:6812 osd.4 since back 2024-09-10T10:06:22.734557+0000 front 2024-09-10T10:06:22.734365+0000 (oldest deadline 2024-09-10T10:06:44.434370+0000) 2024-09-10T10:06:46.556 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 10 10:06:46 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-9[65288]: 2024-09-10T10:06:46.242+0000 7f84af200640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-10T10:05:47.346189+0000 front 2024-09-10T10:05:47.346220+0000 (oldest deadline 2024-09-10T10:06:12.646451+0000) 2024-09-10T10:06:46.556 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 10 10:06:46 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-9[65288]: 2024-09-10T10:06:46.242+0000 7f84af200640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-09-10T10:05:57.348185+0000 front 2024-09-10T10:05:57.348335+0000 (oldest deadline 2024-09-10T10:06:23.248210+0000) 2024-09-10T10:06:46.556 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 10 10:06:46 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-9[65288]: 2024-09-10T10:06:46.242+0000 7f84af200640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-10T10:06:03.249484+0000 front 2024-09-10T10:06:03.249185+0000 (oldest deadline 2024-09-10T10:06:26.749283+0000) 2024-09-10T10:06:46.557 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 10 10:06:46 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-9[65288]: 2024-09-10T10:06:46.242+0000 7f84af200640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-10T10:06:12.650860+0000 front 2024-09-10T10:06:12.650900+0000 (oldest deadline 2024-09-10T10:06:38.551142+0000) 2024-09-10T10:06:46.557 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 10 10:06:46 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-9[65288]: 2024-09-10T10:06:46.242+0000 7f84af200640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.102:6812 osd.4 since back 2024-09-10T10:06:18.551683+0000 front 2024-09-10T10:06:18.551825+0000 (oldest deadline 2024-09-10T10:06:44.451846+0000) 2024-09-10T10:06:46.601 INFO:journalctl@ceph.osd.8.smithi159.stdout:Sep 10 10:06:46 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-8[53439]: 2024-09-10T10:06:46.199+0000 7f7417907640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-09-10T10:05:57.316774+0000 front 2024-09-10T10:05:57.316749+0000 (oldest deadline 2024-09-10T10:06:20.816694+0000) 2024-09-10T10:06:46.601 INFO:journalctl@ceph.osd.8.smithi159.stdout:Sep 10 10:06:46 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-8[53439]: 2024-09-10T10:06:46.199+0000 7f7417907640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-10T10:06:04.917990+0000 front 2024-09-10T10:06:04.918013+0000 (oldest deadline 2024-09-10T10:06:27.818198+0000) 2024-09-10T10:06:46.601 INFO:journalctl@ceph.osd.8.smithi159.stdout:Sep 10 10:06:46 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-8[53439]: 2024-09-10T10:06:46.199+0000 7f7417907640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-10T10:06:13.119605+0000 front 2024-09-10T10:06:13.119551+0000 (oldest deadline 2024-09-10T10:06:36.619665+0000) 2024-09-10T10:06:46.601 INFO:journalctl@ceph.osd.8.smithi159.stdout:Sep 10 10:06:46 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-8[53439]: 2024-09-10T10:06:46.199+0000 7f7417907640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.102:6812 osd.4 since back 2024-09-10T10:06:21.821307+0000 front 2024-09-10T10:06:21.821315+0000 (oldest deadline 2024-09-10T10:06:44.721409+0000) 2024-09-10T10:06:46.634 INFO:journalctl@ceph.osd.10.smithi102.stdout:Sep 10 10:06:46 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-10[59874]: 2024-09-10T10:06:46.542+0000 7f40f3335640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-10T10:05:49.575097+0000 front 2024-09-10T10:05:49.575141+0000 (oldest deadline 2024-09-10T10:06:13.075035+0000) 2024-09-10T10:06:46.635 INFO:journalctl@ceph.osd.10.smithi102.stdout:Sep 10 10:06:46 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-10[59874]: 2024-09-10T10:06:46.542+0000 7f40f3335640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-09-10T10:05:54.176094+0000 front 2024-09-10T10:05:54.176115+0000 (oldest deadline 2024-09-10T10:06:19.376783+0000) 2024-09-10T10:06:46.635 INFO:journalctl@ceph.osd.10.smithi102.stdout:Sep 10 10:06:46 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-10[59874]: 2024-09-10T10:06:46.542+0000 7f40f3335640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-10T10:06:03.378660+0000 front 2024-09-10T10:06:03.378719+0000 (oldest deadline 2024-09-10T10:06:32.178984+0000) 2024-09-10T10:06:46.635 INFO:journalctl@ceph.osd.10.smithi102.stdout:Sep 10 10:06:46 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-10[59874]: 2024-09-10T10:06:46.542+0000 7f40f3335640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-10T10:06:12.179545+0000 front 2024-09-10T10:06:12.179581+0000 (oldest deadline 2024-09-10T10:06:36.879462+0000) 2024-09-10T10:06:46.635 INFO:journalctl@ceph.osd.10.smithi102.stdout:Sep 10 10:06:46 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-10[59874]: 2024-09-10T10:06:46.542+0000 7f40f3335640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.102:6812 osd.4 since back 2024-09-10T10:06:22.581662+0000 front 2024-09-10T10:06:22.581640+0000 (oldest deadline 2024-09-10T10:06:44.281620+0000) 2024-09-10T10:06:46.635 INFO:journalctl@ceph.osd.7.smithi102.stdout:Sep 10 10:06:46 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-7[52880]: 2024-09-10T10:06:46.304+0000 7ff941357640 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-09-10T10:05:55.423434+0000 front 2024-09-10T10:05:55.423389+0000 (oldest deadline 2024-09-10T10:06:19.424367+0000) 2024-09-10T10:06:46.635 INFO:journalctl@ceph.osd.7.smithi102.stdout:Sep 10 10:06:46 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-7[52880]: 2024-09-10T10:06:46.304+0000 7ff941357640 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-10T10:06:04.025965+0000 front 2024-09-10T10:06:04.025864+0000 (oldest deadline 2024-09-10T10:06:28.726043+0000) 2024-09-10T10:06:46.635 INFO:journalctl@ceph.osd.7.smithi102.stdout:Sep 10 10:06:46 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-7[52880]: 2024-09-10T10:06:46.304+0000 7ff941357640 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-10T10:06:12.827133+0000 front 2024-09-10T10:06:12.827237+0000 (oldest deadline 2024-09-10T10:06:38.027668+0000) 2024-09-10T10:06:46.635 INFO:journalctl@ceph.osd.7.smithi102.stdout:Sep 10 10:06:46 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-7[52880]: 2024-09-10T10:06:46.304+0000 7ff941357640 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.102:6812 osd.4 since back 2024-09-10T10:06:18.528387+0000 front 2024-09-10T10:06:18.528339+0000 (oldest deadline 2024-09-10T10:06:43.828582+0000) 2024-09-10T10:06:47.100 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 10 10:06:46 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-11[60437]: 2024-09-10T10:06:46.621+0000 7fa78e4d3640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-10T10:05:49.428950+0000 front 2024-09-10T10:05:49.429031+0000 (oldest deadline 2024-09-10T10:06:11.128800+0000) 2024-09-10T10:06:47.100 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 10 10:06:46 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-11[60437]: 2024-09-10T10:06:46.621+0000 7fa78e4d3640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-10T10:06:03.430910+0000 front 2024-09-10T10:06:03.430889+0000 (oldest deadline 2024-09-10T10:06:29.331091+0000) 2024-09-10T10:06:47.101 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 10 10:06:46 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-11[60437]: 2024-09-10T10:06:46.621+0000 7fa78e4d3640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-10T10:06:13.432384+0000 front 2024-09-10T10:06:13.432489+0000 (oldest deadline 2024-09-10T10:06:35.732466+0000) 2024-09-10T10:06:47.101 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 10 10:06:46 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-11[60437]: 2024-09-10T10:06:46.621+0000 7fa78e4d3640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.102:6812 osd.4 since back 2024-09-10T10:06:22.734557+0000 front 2024-09-10T10:06:22.734365+0000 (oldest deadline 2024-09-10T10:06:44.434370+0000) 2024-09-10T10:06:47.556 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 10 10:06:47 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-9[65288]: 2024-09-10T10:06:47.271+0000 7f84af200640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-10T10:05:47.346189+0000 front 2024-09-10T10:05:47.346220+0000 (oldest deadline 2024-09-10T10:06:12.646451+0000) 2024-09-10T10:06:47.556 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 10 10:06:47 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-9[65288]: 2024-09-10T10:06:47.271+0000 7f84af200640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-09-10T10:05:57.348185+0000 front 2024-09-10T10:05:57.348335+0000 (oldest deadline 2024-09-10T10:06:23.248210+0000) 2024-09-10T10:06:47.556 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 10 10:06:47 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-9[65288]: 2024-09-10T10:06:47.271+0000 7f84af200640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-10T10:06:03.249484+0000 front 2024-09-10T10:06:03.249185+0000 (oldest deadline 2024-09-10T10:06:26.749283+0000) 2024-09-10T10:06:47.557 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 10 10:06:47 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-9[65288]: 2024-09-10T10:06:47.271+0000 7f84af200640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-10T10:06:12.650860+0000 front 2024-09-10T10:06:12.650900+0000 (oldest deadline 2024-09-10T10:06:38.551142+0000) 2024-09-10T10:06:47.557 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 10 10:06:47 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-9[65288]: 2024-09-10T10:06:47.271+0000 7f84af200640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.102:6812 osd.4 since back 2024-09-10T10:06:18.551683+0000 front 2024-09-10T10:06:18.551825+0000 (oldest deadline 2024-09-10T10:06:44.451846+0000) 2024-09-10T10:06:47.558 INFO:journalctl@ceph.osd.7.smithi102.stdout:Sep 10 10:06:47 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-7[52880]: 2024-09-10T10:06:47.256+0000 7ff941357640 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-09-10T10:05:55.423434+0000 front 2024-09-10T10:05:55.423389+0000 (oldest deadline 2024-09-10T10:06:19.424367+0000) 2024-09-10T10:06:47.558 INFO:journalctl@ceph.osd.7.smithi102.stdout:Sep 10 10:06:47 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-7[52880]: 2024-09-10T10:06:47.256+0000 7ff941357640 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-10T10:06:04.025965+0000 front 2024-09-10T10:06:04.025864+0000 (oldest deadline 2024-09-10T10:06:28.726043+0000) 2024-09-10T10:06:47.558 INFO:journalctl@ceph.osd.7.smithi102.stdout:Sep 10 10:06:47 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-7[52880]: 2024-09-10T10:06:47.256+0000 7ff941357640 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-10T10:06:12.827133+0000 front 2024-09-10T10:06:12.827237+0000 (oldest deadline 2024-09-10T10:06:38.027668+0000) 2024-09-10T10:06:47.558 INFO:journalctl@ceph.osd.7.smithi102.stdout:Sep 10 10:06:47 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-7[52880]: 2024-09-10T10:06:47.256+0000 7ff941357640 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.102:6812 osd.4 since back 2024-09-10T10:06:18.528387+0000 front 2024-09-10T10:06:18.528339+0000 (oldest deadline 2024-09-10T10:06:43.828582+0000) 2024-09-10T10:06:47.577 INFO:journalctl@ceph.osd.8.smithi159.stdout:Sep 10 10:06:47 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-8[53439]: 2024-09-10T10:06:47.207+0000 7f7417907640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-09-10T10:05:57.316774+0000 front 2024-09-10T10:05:57.316749+0000 (oldest deadline 2024-09-10T10:06:20.816694+0000) 2024-09-10T10:06:47.577 INFO:journalctl@ceph.osd.8.smithi159.stdout:Sep 10 10:06:47 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-8[53439]: 2024-09-10T10:06:47.207+0000 7f7417907640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-10T10:06:04.917990+0000 front 2024-09-10T10:06:04.918013+0000 (oldest deadline 2024-09-10T10:06:27.818198+0000) 2024-09-10T10:06:47.577 INFO:journalctl@ceph.osd.8.smithi159.stdout:Sep 10 10:06:47 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-8[53439]: 2024-09-10T10:06:47.207+0000 7f7417907640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-10T10:06:13.119605+0000 front 2024-09-10T10:06:13.119551+0000 (oldest deadline 2024-09-10T10:06:36.619665+0000) 2024-09-10T10:06:47.577 INFO:journalctl@ceph.osd.8.smithi159.stdout:Sep 10 10:06:47 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-8[53439]: 2024-09-10T10:06:47.207+0000 7f7417907640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.102:6812 osd.4 since back 2024-09-10T10:06:21.821307+0000 front 2024-09-10T10:06:21.821315+0000 (oldest deadline 2024-09-10T10:06:44.721409+0000) 2024-09-10T10:06:47.850 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 10 10:06:47 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-11[60437]: 2024-09-10T10:06:47.575+0000 7fa78e4d3640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-10T10:05:49.428950+0000 front 2024-09-10T10:05:49.429031+0000 (oldest deadline 2024-09-10T10:06:11.128800+0000) 2024-09-10T10:06:47.850 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 10 10:06:47 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-11[60437]: 2024-09-10T10:06:47.575+0000 7fa78e4d3640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-10T10:06:03.430910+0000 front 2024-09-10T10:06:03.430889+0000 (oldest deadline 2024-09-10T10:06:29.331091+0000) 2024-09-10T10:06:47.850 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 10 10:06:47 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-11[60437]: 2024-09-10T10:06:47.575+0000 7fa78e4d3640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-10T10:06:13.432384+0000 front 2024-09-10T10:06:13.432489+0000 (oldest deadline 2024-09-10T10:06:35.732466+0000) 2024-09-10T10:06:47.850 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 10 10:06:47 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-11[60437]: 2024-09-10T10:06:47.575+0000 7fa78e4d3640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.102:6812 osd.4 since back 2024-09-10T10:06:22.734557+0000 front 2024-09-10T10:06:22.734365+0000 (oldest deadline 2024-09-10T10:06:44.434370+0000) 2024-09-10T10:06:47.884 INFO:journalctl@ceph.osd.10.smithi102.stdout:Sep 10 10:06:47 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-10[59874]: 2024-09-10T10:06:47.557+0000 7f40f3335640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-10T10:05:49.575097+0000 front 2024-09-10T10:05:49.575141+0000 (oldest deadline 2024-09-10T10:06:13.075035+0000) 2024-09-10T10:06:47.885 INFO:journalctl@ceph.osd.10.smithi102.stdout:Sep 10 10:06:47 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-10[59874]: 2024-09-10T10:06:47.557+0000 7f40f3335640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-09-10T10:05:54.176094+0000 front 2024-09-10T10:05:54.176115+0000 (oldest deadline 2024-09-10T10:06:19.376783+0000) 2024-09-10T10:06:47.885 INFO:journalctl@ceph.osd.10.smithi102.stdout:Sep 10 10:06:47 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-10[59874]: 2024-09-10T10:06:47.557+0000 7f40f3335640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-10T10:06:03.378660+0000 front 2024-09-10T10:06:03.378719+0000 (oldest deadline 2024-09-10T10:06:32.178984+0000) 2024-09-10T10:06:47.885 INFO:journalctl@ceph.osd.10.smithi102.stdout:Sep 10 10:06:47 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-10[59874]: 2024-09-10T10:06:47.557+0000 7f40f3335640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-10T10:06:12.179545+0000 front 2024-09-10T10:06:12.179581+0000 (oldest deadline 2024-09-10T10:06:36.879462+0000) 2024-09-10T10:06:47.885 INFO:journalctl@ceph.osd.10.smithi102.stdout:Sep 10 10:06:47 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-10[59874]: 2024-09-10T10:06:47.557+0000 7f40f3335640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.102:6812 osd.4 since back 2024-09-10T10:06:22.581662+0000 front 2024-09-10T10:06:22.581640+0000 (oldest deadline 2024-09-10T10:06:44.281620+0000) 2024-09-10T10:06:48.534 INFO:journalctl@ceph.osd.7.smithi102.stdout:Sep 10 10:06:48 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-7[52880]: 2024-09-10T10:06:48.282+0000 7ff941357640 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-09-10T10:05:55.423434+0000 front 2024-09-10T10:05:55.423389+0000 (oldest deadline 2024-09-10T10:06:19.424367+0000) 2024-09-10T10:06:48.535 INFO:journalctl@ceph.osd.7.smithi102.stdout:Sep 10 10:06:48 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-7[52880]: 2024-09-10T10:06:48.282+0000 7ff941357640 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-10T10:06:04.025965+0000 front 2024-09-10T10:06:04.025864+0000 (oldest deadline 2024-09-10T10:06:28.726043+0000) 2024-09-10T10:06:48.535 INFO:journalctl@ceph.osd.7.smithi102.stdout:Sep 10 10:06:48 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-7[52880]: 2024-09-10T10:06:48.282+0000 7ff941357640 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-10T10:06:12.827133+0000 front 2024-09-10T10:06:12.827237+0000 (oldest deadline 2024-09-10T10:06:38.027668+0000) 2024-09-10T10:06:48.535 INFO:journalctl@ceph.osd.7.smithi102.stdout:Sep 10 10:06:48 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-7[52880]: 2024-09-10T10:06:48.282+0000 7ff941357640 -1 osd.7 100 heartbeat_check: no reply from 172.21.15.102:6812 osd.4 since back 2024-09-10T10:06:18.528387+0000 front 2024-09-10T10:06:18.528339+0000 (oldest deadline 2024-09-10T10:06:43.828582+0000) 2024-09-10T10:06:48.556 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 10 10:06:48 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-9[65288]: 2024-09-10T10:06:48.245+0000 7f84af200640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-10T10:05:47.346189+0000 front 2024-09-10T10:05:47.346220+0000 (oldest deadline 2024-09-10T10:06:12.646451+0000) 2024-09-10T10:06:48.556 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 10 10:06:48 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-9[65288]: 2024-09-10T10:06:48.245+0000 7f84af200640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-09-10T10:05:57.348185+0000 front 2024-09-10T10:05:57.348335+0000 (oldest deadline 2024-09-10T10:06:23.248210+0000) 2024-09-10T10:06:48.556 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 10 10:06:48 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-9[65288]: 2024-09-10T10:06:48.245+0000 7f84af200640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-10T10:06:03.249484+0000 front 2024-09-10T10:06:03.249185+0000 (oldest deadline 2024-09-10T10:06:26.749283+0000) 2024-09-10T10:06:48.556 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 10 10:06:48 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-9[65288]: 2024-09-10T10:06:48.245+0000 7f84af200640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-10T10:06:12.650860+0000 front 2024-09-10T10:06:12.650900+0000 (oldest deadline 2024-09-10T10:06:38.551142+0000) 2024-09-10T10:06:48.557 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 10 10:06:48 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-9[65288]: 2024-09-10T10:06:48.245+0000 7f84af200640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.102:6812 osd.4 since back 2024-09-10T10:06:18.551683+0000 front 2024-09-10T10:06:18.551825+0000 (oldest deadline 2024-09-10T10:06:44.451846+0000) 2024-09-10T10:06:48.563 INFO:journalctl@ceph.osd.8.smithi159.stdout:Sep 10 10:06:48 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-8[53439]: 2024-09-10T10:06:48.167+0000 7f7417907640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-09-10T10:05:57.316774+0000 front 2024-09-10T10:05:57.316749+0000 (oldest deadline 2024-09-10T10:06:20.816694+0000) 2024-09-10T10:06:48.563 INFO:journalctl@ceph.osd.8.smithi159.stdout:Sep 10 10:06:48 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-8[53439]: 2024-09-10T10:06:48.167+0000 7f7417907640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-10T10:06:04.917990+0000 front 2024-09-10T10:06:04.918013+0000 (oldest deadline 2024-09-10T10:06:27.818198+0000) 2024-09-10T10:06:48.563 INFO:journalctl@ceph.osd.8.smithi159.stdout:Sep 10 10:06:48 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-8[53439]: 2024-09-10T10:06:48.167+0000 7f7417907640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-10T10:06:13.119605+0000 front 2024-09-10T10:06:13.119551+0000 (oldest deadline 2024-09-10T10:06:36.619665+0000) 2024-09-10T10:06:48.563 INFO:journalctl@ceph.osd.8.smithi159.stdout:Sep 10 10:06:48 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-8[53439]: 2024-09-10T10:06:48.167+0000 7f7417907640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.102:6812 osd.4 since back 2024-09-10T10:06:21.821307+0000 front 2024-09-10T10:06:21.821315+0000 (oldest deadline 2024-09-10T10:06:44.721409+0000) 2024-09-10T10:06:48.850 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 10 10:06:48 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-11[60437]: 2024-09-10T10:06:48.561+0000 7fa78e4d3640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-10T10:05:49.428950+0000 front 2024-09-10T10:05:49.429031+0000 (oldest deadline 2024-09-10T10:06:11.128800+0000) 2024-09-10T10:06:48.850 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 10 10:06:48 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-11[60437]: 2024-09-10T10:06:48.561+0000 7fa78e4d3640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-10T10:06:03.430910+0000 front 2024-09-10T10:06:03.430889+0000 (oldest deadline 2024-09-10T10:06:29.331091+0000) 2024-09-10T10:06:48.851 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 10 10:06:48 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-11[60437]: 2024-09-10T10:06:48.561+0000 7fa78e4d3640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-10T10:06:13.432384+0000 front 2024-09-10T10:06:13.432489+0000 (oldest deadline 2024-09-10T10:06:35.732466+0000) 2024-09-10T10:06:48.851 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 10 10:06:48 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-11[60437]: 2024-09-10T10:06:48.561+0000 7fa78e4d3640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.102:6812 osd.4 since back 2024-09-10T10:06:22.734557+0000 front 2024-09-10T10:06:22.734365+0000 (oldest deadline 2024-09-10T10:06:44.434370+0000) 2024-09-10T10:06:48.884 INFO:journalctl@ceph.osd.10.smithi102.stdout:Sep 10 10:06:48 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-10[59874]: 2024-09-10T10:06:48.533+0000 7f40f3335640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-10T10:05:49.575097+0000 front 2024-09-10T10:05:49.575141+0000 (oldest deadline 2024-09-10T10:06:13.075035+0000) 2024-09-10T10:06:48.885 INFO:journalctl@ceph.osd.10.smithi102.stdout:Sep 10 10:06:48 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-10[59874]: 2024-09-10T10:06:48.533+0000 7f40f3335640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-09-10T10:05:54.176094+0000 front 2024-09-10T10:05:54.176115+0000 (oldest deadline 2024-09-10T10:06:19.376783+0000) 2024-09-10T10:06:48.885 INFO:journalctl@ceph.osd.10.smithi102.stdout:Sep 10 10:06:48 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-10[59874]: 2024-09-10T10:06:48.533+0000 7f40f3335640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-10T10:06:03.378660+0000 front 2024-09-10T10:06:03.378719+0000 (oldest deadline 2024-09-10T10:06:32.178984+0000) 2024-09-10T10:06:48.885 INFO:journalctl@ceph.osd.10.smithi102.stdout:Sep 10 10:06:48 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-10[59874]: 2024-09-10T10:06:48.533+0000 7f40f3335640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-10T10:06:12.179545+0000 front 2024-09-10T10:06:12.179581+0000 (oldest deadline 2024-09-10T10:06:36.879462+0000) 2024-09-10T10:06:48.885 INFO:journalctl@ceph.osd.10.smithi102.stdout:Sep 10 10:06:48 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-10[59874]: 2024-09-10T10:06:48.533+0000 7f40f3335640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.102:6812 osd.4 since back 2024-09-10T10:06:22.581662+0000 front 2024-09-10T10:06:22.581640+0000 (oldest deadline 2024-09-10T10:06:44.281620+0000) 2024-09-10T10:06:49.437 INFO:journalctl@ceph.osd.7.smithi102.stdout:Sep 10 10:06:49 smithi102 podman[75862]: 2024-09-10 10:06:49.170163996 +0000 UTC m=+5.732855313 container died e6c83a922b2e73774d0578221e966f85fa59b8fd3a2babc210f83ea2ee2843dc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-7, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True) 2024-09-10T10:06:49.513 INFO:journalctl@ceph.osd.8.smithi159.stdout:Sep 10 10:06:49 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-8[53439]: 2024-09-10T10:06:49.156+0000 7f7417907640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-09-10T10:05:57.316774+0000 front 2024-09-10T10:05:57.316749+0000 (oldest deadline 2024-09-10T10:06:20.816694+0000) 2024-09-10T10:06:49.513 INFO:journalctl@ceph.osd.8.smithi159.stdout:Sep 10 10:06:49 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-8[53439]: 2024-09-10T10:06:49.156+0000 7f7417907640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-10T10:06:04.917990+0000 front 2024-09-10T10:06:04.918013+0000 (oldest deadline 2024-09-10T10:06:27.818198+0000) 2024-09-10T10:06:49.513 INFO:journalctl@ceph.osd.8.smithi159.stdout:Sep 10 10:06:49 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-8[53439]: 2024-09-10T10:06:49.156+0000 7f7417907640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-10T10:06:13.119605+0000 front 2024-09-10T10:06:13.119551+0000 (oldest deadline 2024-09-10T10:06:36.619665+0000) 2024-09-10T10:06:49.514 INFO:journalctl@ceph.osd.8.smithi159.stdout:Sep 10 10:06:49 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-8[53439]: 2024-09-10T10:06:49.156+0000 7f7417907640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.102:6812 osd.4 since back 2024-09-10T10:06:21.821307+0000 front 2024-09-10T10:06:21.821315+0000 (oldest deadline 2024-09-10T10:06:44.721409+0000) 2024-09-10T10:06:49.556 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 10 10:06:49 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-9[65288]: 2024-09-10T10:06:49.202+0000 7f84af200640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-10T10:05:47.346189+0000 front 2024-09-10T10:05:47.346220+0000 (oldest deadline 2024-09-10T10:06:12.646451+0000) 2024-09-10T10:06:49.556 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 10 10:06:49 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-9[65288]: 2024-09-10T10:06:49.202+0000 7f84af200640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-09-10T10:05:57.348185+0000 front 2024-09-10T10:05:57.348335+0000 (oldest deadline 2024-09-10T10:06:23.248210+0000) 2024-09-10T10:06:49.556 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 10 10:06:49 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-9[65288]: 2024-09-10T10:06:49.202+0000 7f84af200640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-10T10:06:03.249484+0000 front 2024-09-10T10:06:03.249185+0000 (oldest deadline 2024-09-10T10:06:26.749283+0000) 2024-09-10T10:06:49.556 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 10 10:06:49 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-9[65288]: 2024-09-10T10:06:49.202+0000 7f84af200640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-10T10:06:12.650860+0000 front 2024-09-10T10:06:12.650900+0000 (oldest deadline 2024-09-10T10:06:38.551142+0000) 2024-09-10T10:06:49.556 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 10 10:06:49 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-9[65288]: 2024-09-10T10:06:49.202+0000 7f84af200640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.102:6812 osd.4 since back 2024-09-10T10:06:18.551683+0000 front 2024-09-10T10:06:18.551825+0000 (oldest deadline 2024-09-10T10:06:44.451846+0000) 2024-09-10T10:06:49.821 INFO:journalctl@ceph.osd.7.smithi102.stdout:Sep 10 10:06:49 smithi102 podman[75862]: 2024-09-10 10:06:49.535896301 +0000 UTC m=+6.098587607 container cleanup e6c83a922b2e73774d0578221e966f85fa59b8fd3a2babc210f83ea2ee2843dc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-7, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-10T10:06:49.821 INFO:journalctl@ceph.osd.7.smithi102.stdout:Sep 10 10:06:49 smithi102 bash[75862]: ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-7 2024-09-10T10:06:49.821 INFO:journalctl@ceph.osd.10.smithi102.stdout:Sep 10 10:06:49 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-10[59874]: 2024-09-10T10:06:49.582+0000 7f40f3335640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-10T10:05:49.575097+0000 front 2024-09-10T10:05:49.575141+0000 (oldest deadline 2024-09-10T10:06:13.075035+0000) 2024-09-10T10:06:49.822 INFO:journalctl@ceph.osd.10.smithi102.stdout:Sep 10 10:06:49 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-10[59874]: 2024-09-10T10:06:49.582+0000 7f40f3335640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-09-10T10:05:54.176094+0000 front 2024-09-10T10:05:54.176115+0000 (oldest deadline 2024-09-10T10:06:19.376783+0000) 2024-09-10T10:06:49.822 INFO:journalctl@ceph.osd.10.smithi102.stdout:Sep 10 10:06:49 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-10[59874]: 2024-09-10T10:06:49.582+0000 7f40f3335640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-10T10:06:03.378660+0000 front 2024-09-10T10:06:03.378719+0000 (oldest deadline 2024-09-10T10:06:32.178984+0000) 2024-09-10T10:06:49.822 INFO:journalctl@ceph.osd.10.smithi102.stdout:Sep 10 10:06:49 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-10[59874]: 2024-09-10T10:06:49.582+0000 7f40f3335640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-10T10:06:12.179545+0000 front 2024-09-10T10:06:12.179581+0000 (oldest deadline 2024-09-10T10:06:36.879462+0000) 2024-09-10T10:06:49.822 INFO:journalctl@ceph.osd.10.smithi102.stdout:Sep 10 10:06:49 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-10[59874]: 2024-09-10T10:06:49.582+0000 7f40f3335640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.102:6812 osd.4 since back 2024-09-10T10:06:22.581662+0000 front 2024-09-10T10:06:22.581640+0000 (oldest deadline 2024-09-10T10:06:44.281620+0000) 2024-09-10T10:06:49.850 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 10 10:06:49 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-11[60437]: 2024-09-10T10:06:49.512+0000 7fa78e4d3640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-10T10:05:49.428950+0000 front 2024-09-10T10:05:49.429031+0000 (oldest deadline 2024-09-10T10:06:11.128800+0000) 2024-09-10T10:06:49.850 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 10 10:06:49 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-11[60437]: 2024-09-10T10:06:49.512+0000 7fa78e4d3640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-10T10:06:03.430910+0000 front 2024-09-10T10:06:03.430889+0000 (oldest deadline 2024-09-10T10:06:29.331091+0000) 2024-09-10T10:06:49.850 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 10 10:06:49 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-11[60437]: 2024-09-10T10:06:49.512+0000 7fa78e4d3640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-10T10:06:13.432384+0000 front 2024-09-10T10:06:13.432489+0000 (oldest deadline 2024-09-10T10:06:35.732466+0000) 2024-09-10T10:06:49.850 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 10 10:06:49 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-11[60437]: 2024-09-10T10:06:49.512+0000 7fa78e4d3640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.102:6812 osd.4 since back 2024-09-10T10:06:22.734557+0000 front 2024-09-10T10:06:22.734365+0000 (oldest deadline 2024-09-10T10:06:44.434370+0000) 2024-09-10T10:06:50.126 INFO:journalctl@ceph.osd.7.smithi102.stdout:Sep 10 10:06:49 smithi102 podman[75877]: 2024-09-10 10:06:49.820397037 +0000 UTC m=+0.652246384 container remove e6c83a922b2e73774d0578221e966f85fa59b8fd3a2babc210f83ea2ee2843dc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-7, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.37.2) 2024-09-10T10:06:50.385 INFO:journalctl@ceph.osd.7.smithi102.stdout:Sep 10 10:06:50 smithi102 podman[75967]: 2024-09-10 10:06:50.029017166 +0000 UTC m=+0.016047126 image pull 41d67112835771c3efca60c6b8fe297c7ddaecdc26cea26fd28aa18636e49005 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 2024-09-10T10:06:50.385 INFO:journalctl@ceph.osd.7.smithi102.stdout:Sep 10 10:06:50 smithi102 podman[75967]: 2024-09-10 10:06:50.154721682 +0000 UTC m=+0.141751631 container create 4beefc19bf98b3d295fd9eed1b234933884d9b5315780e90eb27af2227897377 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-7-deactivate, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731) 2024-09-10T10:06:50.538 INFO:journalctl@ceph.osd.8.smithi159.stdout:Sep 10 10:06:50 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-8[53439]: 2024-09-10T10:06:50.114+0000 7f7417907640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-09-10T10:05:57.316774+0000 front 2024-09-10T10:05:57.316749+0000 (oldest deadline 2024-09-10T10:06:20.816694+0000) 2024-09-10T10:06:50.538 INFO:journalctl@ceph.osd.8.smithi159.stdout:Sep 10 10:06:50 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-8[53439]: 2024-09-10T10:06:50.114+0000 7f7417907640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-10T10:06:04.917990+0000 front 2024-09-10T10:06:04.918013+0000 (oldest deadline 2024-09-10T10:06:27.818198+0000) 2024-09-10T10:06:50.538 INFO:journalctl@ceph.osd.8.smithi159.stdout:Sep 10 10:06:50 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-8[53439]: 2024-09-10T10:06:50.114+0000 7f7417907640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-10T10:06:13.119605+0000 front 2024-09-10T10:06:13.119551+0000 (oldest deadline 2024-09-10T10:06:36.619665+0000) 2024-09-10T10:06:50.539 INFO:journalctl@ceph.osd.8.smithi159.stdout:Sep 10 10:06:50 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-8[53439]: 2024-09-10T10:06:50.114+0000 7f7417907640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.102:6812 osd.4 since back 2024-09-10T10:06:21.821307+0000 front 2024-09-10T10:06:21.821315+0000 (oldest deadline 2024-09-10T10:06:44.721409+0000) 2024-09-10T10:06:50.556 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 10 10:06:50 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-9[65288]: 2024-09-10T10:06:50.172+0000 7f84af200640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-10T10:05:47.346189+0000 front 2024-09-10T10:05:47.346220+0000 (oldest deadline 2024-09-10T10:06:12.646451+0000) 2024-09-10T10:06:50.556 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 10 10:06:50 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-9[65288]: 2024-09-10T10:06:50.172+0000 7f84af200640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-09-10T10:05:57.348185+0000 front 2024-09-10T10:05:57.348335+0000 (oldest deadline 2024-09-10T10:06:23.248210+0000) 2024-09-10T10:06:50.556 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 10 10:06:50 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-9[65288]: 2024-09-10T10:06:50.172+0000 7f84af200640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-10T10:06:03.249484+0000 front 2024-09-10T10:06:03.249185+0000 (oldest deadline 2024-09-10T10:06:26.749283+0000) 2024-09-10T10:06:50.556 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 10 10:06:50 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-9[65288]: 2024-09-10T10:06:50.172+0000 7f84af200640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-10T10:06:12.650860+0000 front 2024-09-10T10:06:12.650900+0000 (oldest deadline 2024-09-10T10:06:38.551142+0000) 2024-09-10T10:06:50.556 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 10 10:06:50 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-9[65288]: 2024-09-10T10:06:50.172+0000 7f84af200640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.102:6812 osd.4 since back 2024-09-10T10:06:18.551683+0000 front 2024-09-10T10:06:18.551825+0000 (oldest deadline 2024-09-10T10:06:44.451846+0000) 2024-09-10T10:06:50.850 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 10 10:06:50 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-11[60437]: 2024-09-10T10:06:50.537+0000 7fa78e4d3640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-10T10:05:49.428950+0000 front 2024-09-10T10:05:49.429031+0000 (oldest deadline 2024-09-10T10:06:11.128800+0000) 2024-09-10T10:06:50.850 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 10 10:06:50 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-11[60437]: 2024-09-10T10:06:50.537+0000 7fa78e4d3640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-10T10:06:03.430910+0000 front 2024-09-10T10:06:03.430889+0000 (oldest deadline 2024-09-10T10:06:29.331091+0000) 2024-09-10T10:06:50.850 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 10 10:06:50 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-11[60437]: 2024-09-10T10:06:50.537+0000 7fa78e4d3640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-10T10:06:13.432384+0000 front 2024-09-10T10:06:13.432489+0000 (oldest deadline 2024-09-10T10:06:35.732466+0000) 2024-09-10T10:06:50.851 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 10 10:06:50 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-11[60437]: 2024-09-10T10:06:50.537+0000 7fa78e4d3640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.102:6812 osd.4 since back 2024-09-10T10:06:22.734557+0000 front 2024-09-10T10:06:22.734365+0000 (oldest deadline 2024-09-10T10:06:44.434370+0000) 2024-09-10T10:06:50.885 INFO:journalctl@ceph.osd.10.smithi102.stdout:Sep 10 10:06:50 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-10[59874]: 2024-09-10T10:06:50.548+0000 7f40f3335640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-10T10:05:49.575097+0000 front 2024-09-10T10:05:49.575141+0000 (oldest deadline 2024-09-10T10:06:13.075035+0000) 2024-09-10T10:06:50.885 INFO:journalctl@ceph.osd.10.smithi102.stdout:Sep 10 10:06:50 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-10[59874]: 2024-09-10T10:06:50.548+0000 7f40f3335640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-09-10T10:05:54.176094+0000 front 2024-09-10T10:05:54.176115+0000 (oldest deadline 2024-09-10T10:06:19.376783+0000) 2024-09-10T10:06:50.885 INFO:journalctl@ceph.osd.10.smithi102.stdout:Sep 10 10:06:50 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-10[59874]: 2024-09-10T10:06:50.548+0000 7f40f3335640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-10T10:06:03.378660+0000 front 2024-09-10T10:06:03.378719+0000 (oldest deadline 2024-09-10T10:06:32.178984+0000) 2024-09-10T10:06:50.885 INFO:journalctl@ceph.osd.10.smithi102.stdout:Sep 10 10:06:50 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-10[59874]: 2024-09-10T10:06:50.548+0000 7f40f3335640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-10T10:06:12.179545+0000 front 2024-09-10T10:06:12.179581+0000 (oldest deadline 2024-09-10T10:06:36.879462+0000) 2024-09-10T10:06:50.885 INFO:journalctl@ceph.osd.10.smithi102.stdout:Sep 10 10:06:50 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-10[59874]: 2024-09-10T10:06:50.548+0000 7f40f3335640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.102:6812 osd.4 since back 2024-09-10T10:06:22.581662+0000 front 2024-09-10T10:06:22.581640+0000 (oldest deadline 2024-09-10T10:06:44.281620+0000) 2024-09-10T10:06:50.885 INFO:journalctl@ceph.osd.7.smithi102.stdout:Sep 10 10:06:50 smithi102 podman[75967]: 2024-09-10 10:06:50.415606883 +0000 UTC m=+0.402636836 container init 4beefc19bf98b3d295fd9eed1b234933884d9b5315780e90eb27af2227897377 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-7-deactivate, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD) 2024-09-10T10:06:50.885 INFO:journalctl@ceph.osd.7.smithi102.stdout:Sep 10 10:06:50 smithi102 podman[75967]: 2024-09-10 10:06:50.418935446 +0000 UTC m=+0.405965396 container start 4beefc19bf98b3d295fd9eed1b234933884d9b5315780e90eb27af2227897377 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-7-deactivate, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-09-10T10:06:50.885 INFO:journalctl@ceph.osd.7.smithi102.stdout:Sep 10 10:06:50 smithi102 podman[75967]: 2024-09-10 10:06:50.490751661 +0000 UTC m=+0.477781626 container attach 4beefc19bf98b3d295fd9eed1b234933884d9b5315780e90eb27af2227897377 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-7-deactivate, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.vendor=CentOS, ceph=True) 2024-09-10T10:06:50.885 INFO:journalctl@ceph.osd.7.smithi102.stdout:Sep 10 10:06:50 smithi102 podman[75967]: 2024-09-10 10:06:50.606358473 +0000 UTC m=+0.593388431 container died 4beefc19bf98b3d295fd9eed1b234933884d9b5315780e90eb27af2227897377 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-7-deactivate, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b) 2024-09-10T10:06:51.276 DEBUG:teuthology.orchestra.run.smithi102:> sudo pkill -f 'journalctl -f -n 0 -u ceph-bade4b50-6f56-11ef-bcea-c7b262605968@osd.7.service' 2024-09-10T10:06:51.315 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T10:06:51.315 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-09-10T10:06:51.315 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-09-10T10:06:51.316 DEBUG:teuthology.orchestra.run.smithi159:> sudo systemctl stop ceph-bade4b50-6f56-11ef-bcea-c7b262605968@osd.8 2024-09-10T10:06:51.342 INFO:journalctl@ceph.osd.8.smithi159.stdout:Sep 10 10:06:51 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-8[53439]: 2024-09-10T10:06:51.082+0000 7f7417907640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-09-10T10:05:57.316774+0000 front 2024-09-10T10:05:57.316749+0000 (oldest deadline 2024-09-10T10:06:20.816694+0000) 2024-09-10T10:06:51.342 INFO:journalctl@ceph.osd.8.smithi159.stdout:Sep 10 10:06:51 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-8[53439]: 2024-09-10T10:06:51.082+0000 7f7417907640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-10T10:06:04.917990+0000 front 2024-09-10T10:06:04.918013+0000 (oldest deadline 2024-09-10T10:06:27.818198+0000) 2024-09-10T10:06:51.342 INFO:journalctl@ceph.osd.8.smithi159.stdout:Sep 10 10:06:51 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-8[53439]: 2024-09-10T10:06:51.083+0000 7f7417907640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-10T10:06:13.119605+0000 front 2024-09-10T10:06:13.119551+0000 (oldest deadline 2024-09-10T10:06:36.619665+0000) 2024-09-10T10:06:51.342 INFO:journalctl@ceph.osd.8.smithi159.stdout:Sep 10 10:06:51 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-8[53439]: 2024-09-10T10:06:51.083+0000 7f7417907640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.102:6812 osd.4 since back 2024-09-10T10:06:21.821307+0000 front 2024-09-10T10:06:21.821315+0000 (oldest deadline 2024-09-10T10:06:44.721409+0000) 2024-09-10T10:06:51.556 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 10 10:06:51 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-9[65288]: 2024-09-10T10:06:51.147+0000 7f84af200640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-10T10:05:47.346189+0000 front 2024-09-10T10:05:47.346220+0000 (oldest deadline 2024-09-10T10:06:12.646451+0000) 2024-09-10T10:06:51.556 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 10 10:06:51 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-9[65288]: 2024-09-10T10:06:51.147+0000 7f84af200640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-09-10T10:05:57.348185+0000 front 2024-09-10T10:05:57.348335+0000 (oldest deadline 2024-09-10T10:06:23.248210+0000) 2024-09-10T10:06:51.556 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 10 10:06:51 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-9[65288]: 2024-09-10T10:06:51.147+0000 7f84af200640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-10T10:06:03.249484+0000 front 2024-09-10T10:06:03.249185+0000 (oldest deadline 2024-09-10T10:06:26.749283+0000) 2024-09-10T10:06:51.557 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 10 10:06:51 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-9[65288]: 2024-09-10T10:06:51.147+0000 7f84af200640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-10T10:06:12.650860+0000 front 2024-09-10T10:06:12.650900+0000 (oldest deadline 2024-09-10T10:06:38.551142+0000) 2024-09-10T10:06:51.557 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 10 10:06:51 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-9[65288]: 2024-09-10T10:06:51.147+0000 7f84af200640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.102:6812 osd.4 since back 2024-09-10T10:06:18.551683+0000 front 2024-09-10T10:06:18.551825+0000 (oldest deadline 2024-09-10T10:06:44.451846+0000) 2024-09-10T10:06:51.600 INFO:journalctl@ceph.osd.8.smithi159.stdout:Sep 10 10:06:51 smithi159 systemd[1]: Stopping Ceph osd.8 for bade4b50-6f56-11ef-bcea-c7b262605968... 2024-09-10T10:06:51.601 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 10 10:06:51 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-11[60437]: 2024-09-10T10:06:51.537+0000 7fa78e4d3640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-10T10:05:49.428950+0000 front 2024-09-10T10:05:49.429031+0000 (oldest deadline 2024-09-10T10:06:11.128800+0000) 2024-09-10T10:06:51.601 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 10 10:06:51 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-11[60437]: 2024-09-10T10:06:51.537+0000 7fa78e4d3640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-10T10:06:03.430910+0000 front 2024-09-10T10:06:03.430889+0000 (oldest deadline 2024-09-10T10:06:29.331091+0000) 2024-09-10T10:06:51.601 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 10 10:06:51 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-11[60437]: 2024-09-10T10:06:51.537+0000 7fa78e4d3640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-10T10:06:13.432384+0000 front 2024-09-10T10:06:13.432489+0000 (oldest deadline 2024-09-10T10:06:35.732466+0000) 2024-09-10T10:06:51.601 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 10 10:06:51 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-11[60437]: 2024-09-10T10:06:51.537+0000 7fa78e4d3640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.102:6812 osd.4 since back 2024-09-10T10:06:22.734557+0000 front 2024-09-10T10:06:22.734365+0000 (oldest deadline 2024-09-10T10:06:44.434370+0000) 2024-09-10T10:06:51.885 INFO:journalctl@ceph.osd.10.smithi102.stdout:Sep 10 10:06:51 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-10[59874]: 2024-09-10T10:06:51.565+0000 7f40f3335640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-10T10:05:49.575097+0000 front 2024-09-10T10:05:49.575141+0000 (oldest deadline 2024-09-10T10:06:13.075035+0000) 2024-09-10T10:06:51.885 INFO:journalctl@ceph.osd.10.smithi102.stdout:Sep 10 10:06:51 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-10[59874]: 2024-09-10T10:06:51.565+0000 7f40f3335640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-09-10T10:05:54.176094+0000 front 2024-09-10T10:05:54.176115+0000 (oldest deadline 2024-09-10T10:06:19.376783+0000) 2024-09-10T10:06:51.885 INFO:journalctl@ceph.osd.10.smithi102.stdout:Sep 10 10:06:51 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-10[59874]: 2024-09-10T10:06:51.565+0000 7f40f3335640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-10T10:06:03.378660+0000 front 2024-09-10T10:06:03.378719+0000 (oldest deadline 2024-09-10T10:06:32.178984+0000) 2024-09-10T10:06:51.885 INFO:journalctl@ceph.osd.10.smithi102.stdout:Sep 10 10:06:51 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-10[59874]: 2024-09-10T10:06:51.565+0000 7f40f3335640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-10T10:06:12.179545+0000 front 2024-09-10T10:06:12.179581+0000 (oldest deadline 2024-09-10T10:06:36.879462+0000) 2024-09-10T10:06:51.885 INFO:journalctl@ceph.osd.10.smithi102.stdout:Sep 10 10:06:51 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-10[59874]: 2024-09-10T10:06:51.565+0000 7f40f3335640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.102:6812 osd.4 since back 2024-09-10T10:06:22.581662+0000 front 2024-09-10T10:06:22.581640+0000 (oldest deadline 2024-09-10T10:06:44.281620+0000) 2024-09-10T10:06:52.100 INFO:journalctl@ceph.osd.8.smithi159.stdout:Sep 10 10:06:51 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-8[53439]: 2024-09-10T10:06:51.629+0000 7f741f2f9640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T10:06:52.100 INFO:journalctl@ceph.osd.8.smithi159.stdout:Sep 10 10:06:51 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-8[53439]: 2024-09-10T10:06:51.629+0000 7f741f2f9640 -1 osd.8 100 *** Got signal Terminated *** 2024-09-10T10:06:52.100 INFO:journalctl@ceph.osd.8.smithi159.stdout:Sep 10 10:06:51 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-8[53439]: 2024-09-10T10:06:51.629+0000 7f741f2f9640 -1 osd.8 100 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-10T10:06:52.525 INFO:journalctl@ceph.osd.8.smithi159.stdout:Sep 10 10:06:52 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-8[53439]: 2024-09-10T10:06:52.116+0000 7f7417907640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-09-10T10:05:57.316774+0000 front 2024-09-10T10:05:57.316749+0000 (oldest deadline 2024-09-10T10:06:20.816694+0000) 2024-09-10T10:06:52.525 INFO:journalctl@ceph.osd.8.smithi159.stdout:Sep 10 10:06:52 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-8[53439]: 2024-09-10T10:06:52.116+0000 7f7417907640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-10T10:06:04.917990+0000 front 2024-09-10T10:06:04.918013+0000 (oldest deadline 2024-09-10T10:06:27.818198+0000) 2024-09-10T10:06:52.525 INFO:journalctl@ceph.osd.8.smithi159.stdout:Sep 10 10:06:52 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-8[53439]: 2024-09-10T10:06:52.116+0000 7f7417907640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-10T10:06:13.119605+0000 front 2024-09-10T10:06:13.119551+0000 (oldest deadline 2024-09-10T10:06:36.619665+0000) 2024-09-10T10:06:52.526 INFO:journalctl@ceph.osd.8.smithi159.stdout:Sep 10 10:06:52 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-8[53439]: 2024-09-10T10:06:52.116+0000 7f7417907640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.102:6812 osd.4 since back 2024-09-10T10:06:21.821307+0000 front 2024-09-10T10:06:21.821315+0000 (oldest deadline 2024-09-10T10:06:44.721409+0000) 2024-09-10T10:06:52.556 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 10 10:06:52 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-9[65288]: 2024-09-10T10:06:52.154+0000 7f84af200640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-10T10:05:47.346189+0000 front 2024-09-10T10:05:47.346220+0000 (oldest deadline 2024-09-10T10:06:12.646451+0000) 2024-09-10T10:06:52.556 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 10 10:06:52 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-9[65288]: 2024-09-10T10:06:52.154+0000 7f84af200640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-09-10T10:05:57.348185+0000 front 2024-09-10T10:05:57.348335+0000 (oldest deadline 2024-09-10T10:06:23.248210+0000) 2024-09-10T10:06:52.556 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 10 10:06:52 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-9[65288]: 2024-09-10T10:06:52.154+0000 7f84af200640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-10T10:06:03.249484+0000 front 2024-09-10T10:06:03.249185+0000 (oldest deadline 2024-09-10T10:06:26.749283+0000) 2024-09-10T10:06:52.556 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 10 10:06:52 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-9[65288]: 2024-09-10T10:06:52.154+0000 7f84af200640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-10T10:06:12.650860+0000 front 2024-09-10T10:06:12.650900+0000 (oldest deadline 2024-09-10T10:06:38.551142+0000) 2024-09-10T10:06:52.557 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 10 10:06:52 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-9[65288]: 2024-09-10T10:06:52.154+0000 7f84af200640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.102:6812 osd.4 since back 2024-09-10T10:06:18.551683+0000 front 2024-09-10T10:06:18.551825+0000 (oldest deadline 2024-09-10T10:06:44.451846+0000) 2024-09-10T10:06:52.850 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 10 10:06:52 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-11[60437]: 2024-09-10T10:06:52.524+0000 7fa78e4d3640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-10T10:05:49.428950+0000 front 2024-09-10T10:05:49.429031+0000 (oldest deadline 2024-09-10T10:06:11.128800+0000) 2024-09-10T10:06:52.850 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 10 10:06:52 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-11[60437]: 2024-09-10T10:06:52.524+0000 7fa78e4d3640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-10T10:06:03.430910+0000 front 2024-09-10T10:06:03.430889+0000 (oldest deadline 2024-09-10T10:06:29.331091+0000) 2024-09-10T10:06:52.850 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 10 10:06:52 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-11[60437]: 2024-09-10T10:06:52.524+0000 7fa78e4d3640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-10T10:06:13.432384+0000 front 2024-09-10T10:06:13.432489+0000 (oldest deadline 2024-09-10T10:06:35.732466+0000) 2024-09-10T10:06:52.850 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 10 10:06:52 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-11[60437]: 2024-09-10T10:06:52.524+0000 7fa78e4d3640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.102:6812 osd.4 since back 2024-09-10T10:06:22.734557+0000 front 2024-09-10T10:06:22.734365+0000 (oldest deadline 2024-09-10T10:06:44.434370+0000) 2024-09-10T10:06:52.884 INFO:journalctl@ceph.osd.10.smithi102.stdout:Sep 10 10:06:52 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-10[59874]: 2024-09-10T10:06:52.604+0000 7f40f3335640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-10T10:05:49.575097+0000 front 2024-09-10T10:05:49.575141+0000 (oldest deadline 2024-09-10T10:06:13.075035+0000) 2024-09-10T10:06:52.884 INFO:journalctl@ceph.osd.10.smithi102.stdout:Sep 10 10:06:52 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-10[59874]: 2024-09-10T10:06:52.604+0000 7f40f3335640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-09-10T10:05:54.176094+0000 front 2024-09-10T10:05:54.176115+0000 (oldest deadline 2024-09-10T10:06:19.376783+0000) 2024-09-10T10:06:52.884 INFO:journalctl@ceph.osd.10.smithi102.stdout:Sep 10 10:06:52 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-10[59874]: 2024-09-10T10:06:52.604+0000 7f40f3335640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-10T10:06:03.378660+0000 front 2024-09-10T10:06:03.378719+0000 (oldest deadline 2024-09-10T10:06:32.178984+0000) 2024-09-10T10:06:52.885 INFO:journalctl@ceph.osd.10.smithi102.stdout:Sep 10 10:06:52 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-10[59874]: 2024-09-10T10:06:52.604+0000 7f40f3335640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-10T10:06:12.179545+0000 front 2024-09-10T10:06:12.179581+0000 (oldest deadline 2024-09-10T10:06:36.879462+0000) 2024-09-10T10:06:52.885 INFO:journalctl@ceph.osd.10.smithi102.stdout:Sep 10 10:06:52 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-10[59874]: 2024-09-10T10:06:52.604+0000 7f40f3335640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.102:6812 osd.4 since back 2024-09-10T10:06:22.581662+0000 front 2024-09-10T10:06:22.581640+0000 (oldest deadline 2024-09-10T10:06:44.281620+0000) 2024-09-10T10:06:53.556 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 10 10:06:53 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-9[65288]: 2024-09-10T10:06:53.142+0000 7f84af200640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-10T10:05:47.346189+0000 front 2024-09-10T10:05:47.346220+0000 (oldest deadline 2024-09-10T10:06:12.646451+0000) 2024-09-10T10:06:53.557 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 10 10:06:53 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-9[65288]: 2024-09-10T10:06:53.142+0000 7f84af200640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-09-10T10:05:57.348185+0000 front 2024-09-10T10:05:57.348335+0000 (oldest deadline 2024-09-10T10:06:23.248210+0000) 2024-09-10T10:06:53.557 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 10 10:06:53 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-9[65288]: 2024-09-10T10:06:53.142+0000 7f84af200640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-10T10:06:03.249484+0000 front 2024-09-10T10:06:03.249185+0000 (oldest deadline 2024-09-10T10:06:26.749283+0000) 2024-09-10T10:06:53.557 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 10 10:06:53 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-9[65288]: 2024-09-10T10:06:53.142+0000 7f84af200640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-10T10:06:12.650860+0000 front 2024-09-10T10:06:12.650900+0000 (oldest deadline 2024-09-10T10:06:38.551142+0000) 2024-09-10T10:06:53.557 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 10 10:06:53 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-9[65288]: 2024-09-10T10:06:53.142+0000 7f84af200640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.102:6812 osd.4 since back 2024-09-10T10:06:18.551683+0000 front 2024-09-10T10:06:18.551825+0000 (oldest deadline 2024-09-10T10:06:44.451846+0000) 2024-09-10T10:06:53.569 INFO:journalctl@ceph.osd.8.smithi159.stdout:Sep 10 10:06:53 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-8[53439]: 2024-09-10T10:06:53.140+0000 7f7417907640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-09-10T10:05:57.316774+0000 front 2024-09-10T10:05:57.316749+0000 (oldest deadline 2024-09-10T10:06:20.816694+0000) 2024-09-10T10:06:53.569 INFO:journalctl@ceph.osd.8.smithi159.stdout:Sep 10 10:06:53 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-8[53439]: 2024-09-10T10:06:53.140+0000 7f7417907640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-10T10:06:04.917990+0000 front 2024-09-10T10:06:04.918013+0000 (oldest deadline 2024-09-10T10:06:27.818198+0000) 2024-09-10T10:06:53.569 INFO:journalctl@ceph.osd.8.smithi159.stdout:Sep 10 10:06:53 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-8[53439]: 2024-09-10T10:06:53.140+0000 7f7417907640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-10T10:06:13.119605+0000 front 2024-09-10T10:06:13.119551+0000 (oldest deadline 2024-09-10T10:06:36.619665+0000) 2024-09-10T10:06:53.570 INFO:journalctl@ceph.osd.8.smithi159.stdout:Sep 10 10:06:53 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-8[53439]: 2024-09-10T10:06:53.140+0000 7f7417907640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.102:6812 osd.4 since back 2024-09-10T10:06:21.821307+0000 front 2024-09-10T10:06:21.821315+0000 (oldest deadline 2024-09-10T10:06:44.721409+0000) 2024-09-10T10:06:53.570 INFO:journalctl@ceph.osd.8.smithi159.stdout:Sep 10 10:06:53 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-8[53439]: 2024-09-10T10:06:53.140+0000 7f7417907640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.159:6812 osd.5 since back 2024-09-10T10:06:29.322920+0000 front 2024-09-10T10:06:29.322884+0000 (oldest deadline 2024-09-10T10:06:52.222898+0000) 2024-09-10T10:06:53.850 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 10 10:06:53 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-11[60437]: 2024-09-10T10:06:53.568+0000 7fa78e4d3640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-10T10:05:49.428950+0000 front 2024-09-10T10:05:49.429031+0000 (oldest deadline 2024-09-10T10:06:11.128800+0000) 2024-09-10T10:06:53.851 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 10 10:06:53 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-11[60437]: 2024-09-10T10:06:53.568+0000 7fa78e4d3640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-10T10:06:03.430910+0000 front 2024-09-10T10:06:03.430889+0000 (oldest deadline 2024-09-10T10:06:29.331091+0000) 2024-09-10T10:06:53.851 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 10 10:06:53 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-11[60437]: 2024-09-10T10:06:53.568+0000 7fa78e4d3640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-10T10:06:13.432384+0000 front 2024-09-10T10:06:13.432489+0000 (oldest deadline 2024-09-10T10:06:35.732466+0000) 2024-09-10T10:06:53.851 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 10 10:06:53 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-11[60437]: 2024-09-10T10:06:53.568+0000 7fa78e4d3640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.102:6812 osd.4 since back 2024-09-10T10:06:22.734557+0000 front 2024-09-10T10:06:22.734365+0000 (oldest deadline 2024-09-10T10:06:44.434370+0000) 2024-09-10T10:06:53.884 INFO:journalctl@ceph.osd.10.smithi102.stdout:Sep 10 10:06:53 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-10[59874]: 2024-09-10T10:06:53.596+0000 7f40f3335640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-10T10:05:49.575097+0000 front 2024-09-10T10:05:49.575141+0000 (oldest deadline 2024-09-10T10:06:13.075035+0000) 2024-09-10T10:06:53.885 INFO:journalctl@ceph.osd.10.smithi102.stdout:Sep 10 10:06:53 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-10[59874]: 2024-09-10T10:06:53.596+0000 7f40f3335640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-09-10T10:05:54.176094+0000 front 2024-09-10T10:05:54.176115+0000 (oldest deadline 2024-09-10T10:06:19.376783+0000) 2024-09-10T10:06:53.885 INFO:journalctl@ceph.osd.10.smithi102.stdout:Sep 10 10:06:53 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-10[59874]: 2024-09-10T10:06:53.596+0000 7f40f3335640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-10T10:06:03.378660+0000 front 2024-09-10T10:06:03.378719+0000 (oldest deadline 2024-09-10T10:06:32.178984+0000) 2024-09-10T10:06:53.885 INFO:journalctl@ceph.osd.10.smithi102.stdout:Sep 10 10:06:53 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-10[59874]: 2024-09-10T10:06:53.596+0000 7f40f3335640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-10T10:06:12.179545+0000 front 2024-09-10T10:06:12.179581+0000 (oldest deadline 2024-09-10T10:06:36.879462+0000) 2024-09-10T10:06:53.885 INFO:journalctl@ceph.osd.10.smithi102.stdout:Sep 10 10:06:53 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-10[59874]: 2024-09-10T10:06:53.596+0000 7f40f3335640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.102:6812 osd.4 since back 2024-09-10T10:06:22.581662+0000 front 2024-09-10T10:06:22.581640+0000 (oldest deadline 2024-09-10T10:06:44.281620+0000) 2024-09-10T10:06:54.522 INFO:journalctl@ceph.osd.8.smithi159.stdout:Sep 10 10:06:54 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-8[53439]: 2024-09-10T10:06:54.119+0000 7f7417907640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-09-10T10:05:57.316774+0000 front 2024-09-10T10:05:57.316749+0000 (oldest deadline 2024-09-10T10:06:20.816694+0000) 2024-09-10T10:06:54.523 INFO:journalctl@ceph.osd.8.smithi159.stdout:Sep 10 10:06:54 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-8[53439]: 2024-09-10T10:06:54.119+0000 7f7417907640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-10T10:06:04.917990+0000 front 2024-09-10T10:06:04.918013+0000 (oldest deadline 2024-09-10T10:06:27.818198+0000) 2024-09-10T10:06:54.523 INFO:journalctl@ceph.osd.8.smithi159.stdout:Sep 10 10:06:54 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-8[53439]: 2024-09-10T10:06:54.119+0000 7f7417907640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-10T10:06:13.119605+0000 front 2024-09-10T10:06:13.119551+0000 (oldest deadline 2024-09-10T10:06:36.619665+0000) 2024-09-10T10:06:54.523 INFO:journalctl@ceph.osd.8.smithi159.stdout:Sep 10 10:06:54 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-8[53439]: 2024-09-10T10:06:54.119+0000 7f7417907640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.102:6812 osd.4 since back 2024-09-10T10:06:21.821307+0000 front 2024-09-10T10:06:21.821315+0000 (oldest deadline 2024-09-10T10:06:44.721409+0000) 2024-09-10T10:06:54.523 INFO:journalctl@ceph.osd.8.smithi159.stdout:Sep 10 10:06:54 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-8[53439]: 2024-09-10T10:06:54.119+0000 7f7417907640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.159:6812 osd.5 since back 2024-09-10T10:06:29.322920+0000 front 2024-09-10T10:06:29.322884+0000 (oldest deadline 2024-09-10T10:06:52.222898+0000) 2024-09-10T10:06:54.556 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 10 10:06:54 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-9[65288]: 2024-09-10T10:06:54.169+0000 7f84af200640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-10T10:05:47.346189+0000 front 2024-09-10T10:05:47.346220+0000 (oldest deadline 2024-09-10T10:06:12.646451+0000) 2024-09-10T10:06:54.556 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 10 10:06:54 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-9[65288]: 2024-09-10T10:06:54.169+0000 7f84af200640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-09-10T10:05:57.348185+0000 front 2024-09-10T10:05:57.348335+0000 (oldest deadline 2024-09-10T10:06:23.248210+0000) 2024-09-10T10:06:54.556 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 10 10:06:54 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-9[65288]: 2024-09-10T10:06:54.169+0000 7f84af200640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-10T10:06:03.249484+0000 front 2024-09-10T10:06:03.249185+0000 (oldest deadline 2024-09-10T10:06:26.749283+0000) 2024-09-10T10:06:54.556 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 10 10:06:54 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-9[65288]: 2024-09-10T10:06:54.169+0000 7f84af200640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-10T10:06:12.650860+0000 front 2024-09-10T10:06:12.650900+0000 (oldest deadline 2024-09-10T10:06:38.551142+0000) 2024-09-10T10:06:54.556 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 10 10:06:54 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-9[65288]: 2024-09-10T10:06:54.169+0000 7f84af200640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.102:6812 osd.4 since back 2024-09-10T10:06:18.551683+0000 front 2024-09-10T10:06:18.551825+0000 (oldest deadline 2024-09-10T10:06:44.451846+0000) 2024-09-10T10:06:54.850 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 10 10:06:54 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-11[60437]: 2024-09-10T10:06:54.521+0000 7fa78e4d3640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-10T10:05:49.428950+0000 front 2024-09-10T10:05:49.429031+0000 (oldest deadline 2024-09-10T10:06:11.128800+0000) 2024-09-10T10:06:54.851 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 10 10:06:54 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-11[60437]: 2024-09-10T10:06:54.521+0000 7fa78e4d3640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-10T10:06:03.430910+0000 front 2024-09-10T10:06:03.430889+0000 (oldest deadline 2024-09-10T10:06:29.331091+0000) 2024-09-10T10:06:54.851 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 10 10:06:54 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-11[60437]: 2024-09-10T10:06:54.521+0000 7fa78e4d3640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-10T10:06:13.432384+0000 front 2024-09-10T10:06:13.432489+0000 (oldest deadline 2024-09-10T10:06:35.732466+0000) 2024-09-10T10:06:54.851 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 10 10:06:54 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-11[60437]: 2024-09-10T10:06:54.521+0000 7fa78e4d3640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.102:6812 osd.4 since back 2024-09-10T10:06:22.734557+0000 front 2024-09-10T10:06:22.734365+0000 (oldest deadline 2024-09-10T10:06:44.434370+0000) 2024-09-10T10:06:54.851 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 10 10:06:54 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-11[60437]: 2024-09-10T10:06:54.521+0000 7fa78e4d3640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.159:6812 osd.5 since back 2024-09-10T10:06:29.635809+0000 front 2024-09-10T10:06:29.635901+0000 (oldest deadline 2024-09-10T10:06:53.735941+0000) 2024-09-10T10:06:54.884 INFO:journalctl@ceph.osd.10.smithi102.stdout:Sep 10 10:06:54 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-10[59874]: 2024-09-10T10:06:54.618+0000 7f40f3335640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-10T10:05:49.575097+0000 front 2024-09-10T10:05:49.575141+0000 (oldest deadline 2024-09-10T10:06:13.075035+0000) 2024-09-10T10:06:54.884 INFO:journalctl@ceph.osd.10.smithi102.stdout:Sep 10 10:06:54 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-10[59874]: 2024-09-10T10:06:54.618+0000 7f40f3335640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-09-10T10:05:54.176094+0000 front 2024-09-10T10:05:54.176115+0000 (oldest deadline 2024-09-10T10:06:19.376783+0000) 2024-09-10T10:06:54.885 INFO:journalctl@ceph.osd.10.smithi102.stdout:Sep 10 10:06:54 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-10[59874]: 2024-09-10T10:06:54.618+0000 7f40f3335640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-10T10:06:03.378660+0000 front 2024-09-10T10:06:03.378719+0000 (oldest deadline 2024-09-10T10:06:32.178984+0000) 2024-09-10T10:06:54.885 INFO:journalctl@ceph.osd.10.smithi102.stdout:Sep 10 10:06:54 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-10[59874]: 2024-09-10T10:06:54.618+0000 7f40f3335640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-10T10:06:12.179545+0000 front 2024-09-10T10:06:12.179581+0000 (oldest deadline 2024-09-10T10:06:36.879462+0000) 2024-09-10T10:06:54.885 INFO:journalctl@ceph.osd.10.smithi102.stdout:Sep 10 10:06:54 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-10[59874]: 2024-09-10T10:06:54.618+0000 7f40f3335640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.102:6812 osd.4 since back 2024-09-10T10:06:22.581662+0000 front 2024-09-10T10:06:22.581640+0000 (oldest deadline 2024-09-10T10:06:44.281620+0000) 2024-09-10T10:06:54.885 INFO:journalctl@ceph.osd.10.smithi102.stdout:Sep 10 10:06:54 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-10[59874]: 2024-09-10T10:06:54.618+0000 7f40f3335640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.159:6812 osd.5 since back 2024-09-10T10:06:28.382594+0000 front 2024-09-10T10:06:28.382505+0000 (oldest deadline 2024-09-10T10:06:53.682352+0000) 2024-09-10T10:06:55.350 INFO:journalctl@ceph.osd.8.smithi159.stdout:Sep 10 10:06:55 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-8[53439]: 2024-09-10T10:06:55.071+0000 7f7417907640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-09-10T10:05:57.316774+0000 front 2024-09-10T10:05:57.316749+0000 (oldest deadline 2024-09-10T10:06:20.816694+0000) 2024-09-10T10:06:55.350 INFO:journalctl@ceph.osd.8.smithi159.stdout:Sep 10 10:06:55 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-8[53439]: 2024-09-10T10:06:55.071+0000 7f7417907640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-10T10:06:04.917990+0000 front 2024-09-10T10:06:04.918013+0000 (oldest deadline 2024-09-10T10:06:27.818198+0000) 2024-09-10T10:06:55.351 INFO:journalctl@ceph.osd.8.smithi159.stdout:Sep 10 10:06:55 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-8[53439]: 2024-09-10T10:06:55.071+0000 7f7417907640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-10T10:06:13.119605+0000 front 2024-09-10T10:06:13.119551+0000 (oldest deadline 2024-09-10T10:06:36.619665+0000) 2024-09-10T10:06:55.351 INFO:journalctl@ceph.osd.8.smithi159.stdout:Sep 10 10:06:55 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-8[53439]: 2024-09-10T10:06:55.071+0000 7f7417907640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.102:6812 osd.4 since back 2024-09-10T10:06:21.821307+0000 front 2024-09-10T10:06:21.821315+0000 (oldest deadline 2024-09-10T10:06:44.721409+0000) 2024-09-10T10:06:55.351 INFO:journalctl@ceph.osd.8.smithi159.stdout:Sep 10 10:06:55 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-8[53439]: 2024-09-10T10:06:55.071+0000 7f7417907640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.159:6812 osd.5 since back 2024-09-10T10:06:29.322920+0000 front 2024-09-10T10:06:29.322884+0000 (oldest deadline 2024-09-10T10:06:52.222898+0000) 2024-09-10T10:06:55.556 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 10 10:06:55 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-9[65288]: 2024-09-10T10:06:55.146+0000 7f84af200640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-10T10:05:47.346189+0000 front 2024-09-10T10:05:47.346220+0000 (oldest deadline 2024-09-10T10:06:12.646451+0000) 2024-09-10T10:06:55.556 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 10 10:06:55 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-9[65288]: 2024-09-10T10:06:55.146+0000 7f84af200640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-09-10T10:05:57.348185+0000 front 2024-09-10T10:05:57.348335+0000 (oldest deadline 2024-09-10T10:06:23.248210+0000) 2024-09-10T10:06:55.556 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 10 10:06:55 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-9[65288]: 2024-09-10T10:06:55.146+0000 7f84af200640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-10T10:06:03.249484+0000 front 2024-09-10T10:06:03.249185+0000 (oldest deadline 2024-09-10T10:06:26.749283+0000) 2024-09-10T10:06:55.556 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 10 10:06:55 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-9[65288]: 2024-09-10T10:06:55.146+0000 7f84af200640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-10T10:06:12.650860+0000 front 2024-09-10T10:06:12.650900+0000 (oldest deadline 2024-09-10T10:06:38.551142+0000) 2024-09-10T10:06:55.557 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 10 10:06:55 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-9[65288]: 2024-09-10T10:06:55.146+0000 7f84af200640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.102:6812 osd.4 since back 2024-09-10T10:06:18.551683+0000 front 2024-09-10T10:06:18.551825+0000 (oldest deadline 2024-09-10T10:06:44.451846+0000) 2024-09-10T10:06:55.850 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 10 10:06:55 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-11[60437]: 2024-09-10T10:06:55.523+0000 7fa78e4d3640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-10T10:05:49.428950+0000 front 2024-09-10T10:05:49.429031+0000 (oldest deadline 2024-09-10T10:06:11.128800+0000) 2024-09-10T10:06:55.851 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 10 10:06:55 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-11[60437]: 2024-09-10T10:06:55.523+0000 7fa78e4d3640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-10T10:06:03.430910+0000 front 2024-09-10T10:06:03.430889+0000 (oldest deadline 2024-09-10T10:06:29.331091+0000) 2024-09-10T10:06:55.851 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 10 10:06:55 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-11[60437]: 2024-09-10T10:06:55.523+0000 7fa78e4d3640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-10T10:06:13.432384+0000 front 2024-09-10T10:06:13.432489+0000 (oldest deadline 2024-09-10T10:06:35.732466+0000) 2024-09-10T10:06:55.851 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 10 10:06:55 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-11[60437]: 2024-09-10T10:06:55.523+0000 7fa78e4d3640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.102:6812 osd.4 since back 2024-09-10T10:06:22.734557+0000 front 2024-09-10T10:06:22.734365+0000 (oldest deadline 2024-09-10T10:06:44.434370+0000) 2024-09-10T10:06:55.851 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 10 10:06:55 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-11[60437]: 2024-09-10T10:06:55.523+0000 7fa78e4d3640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.159:6812 osd.5 since back 2024-09-10T10:06:29.635809+0000 front 2024-09-10T10:06:29.635901+0000 (oldest deadline 2024-09-10T10:06:53.735941+0000) 2024-09-10T10:06:55.884 INFO:journalctl@ceph.osd.10.smithi102.stdout:Sep 10 10:06:55 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-10[59874]: 2024-09-10T10:06:55.629+0000 7f40f3335640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-10T10:05:49.575097+0000 front 2024-09-10T10:05:49.575141+0000 (oldest deadline 2024-09-10T10:06:13.075035+0000) 2024-09-10T10:06:55.884 INFO:journalctl@ceph.osd.10.smithi102.stdout:Sep 10 10:06:55 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-10[59874]: 2024-09-10T10:06:55.629+0000 7f40f3335640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-09-10T10:05:54.176094+0000 front 2024-09-10T10:05:54.176115+0000 (oldest deadline 2024-09-10T10:06:19.376783+0000) 2024-09-10T10:06:55.885 INFO:journalctl@ceph.osd.10.smithi102.stdout:Sep 10 10:06:55 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-10[59874]: 2024-09-10T10:06:55.629+0000 7f40f3335640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-10T10:06:03.378660+0000 front 2024-09-10T10:06:03.378719+0000 (oldest deadline 2024-09-10T10:06:32.178984+0000) 2024-09-10T10:06:55.885 INFO:journalctl@ceph.osd.10.smithi102.stdout:Sep 10 10:06:55 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-10[59874]: 2024-09-10T10:06:55.629+0000 7f40f3335640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-10T10:06:12.179545+0000 front 2024-09-10T10:06:12.179581+0000 (oldest deadline 2024-09-10T10:06:36.879462+0000) 2024-09-10T10:06:55.885 INFO:journalctl@ceph.osd.10.smithi102.stdout:Sep 10 10:06:55 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-10[59874]: 2024-09-10T10:06:55.629+0000 7f40f3335640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.102:6812 osd.4 since back 2024-09-10T10:06:22.581662+0000 front 2024-09-10T10:06:22.581640+0000 (oldest deadline 2024-09-10T10:06:44.281620+0000) 2024-09-10T10:06:55.885 INFO:journalctl@ceph.osd.10.smithi102.stdout:Sep 10 10:06:55 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-10[59874]: 2024-09-10T10:06:55.629+0000 7f40f3335640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.159:6812 osd.5 since back 2024-09-10T10:06:28.382594+0000 front 2024-09-10T10:06:28.382505+0000 (oldest deadline 2024-09-10T10:06:53.682352+0000) 2024-09-10T10:06:56.350 INFO:journalctl@ceph.osd.8.smithi159.stdout:Sep 10 10:06:56 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-8[53439]: 2024-09-10T10:06:56.074+0000 7f7417907640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-09-10T10:05:57.316774+0000 front 2024-09-10T10:05:57.316749+0000 (oldest deadline 2024-09-10T10:06:20.816694+0000) 2024-09-10T10:06:56.351 INFO:journalctl@ceph.osd.8.smithi159.stdout:Sep 10 10:06:56 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-8[53439]: 2024-09-10T10:06:56.074+0000 7f7417907640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-10T10:06:04.917990+0000 front 2024-09-10T10:06:04.918013+0000 (oldest deadline 2024-09-10T10:06:27.818198+0000) 2024-09-10T10:06:56.351 INFO:journalctl@ceph.osd.8.smithi159.stdout:Sep 10 10:06:56 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-8[53439]: 2024-09-10T10:06:56.074+0000 7f7417907640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-10T10:06:13.119605+0000 front 2024-09-10T10:06:13.119551+0000 (oldest deadline 2024-09-10T10:06:36.619665+0000) 2024-09-10T10:06:56.351 INFO:journalctl@ceph.osd.8.smithi159.stdout:Sep 10 10:06:56 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-8[53439]: 2024-09-10T10:06:56.074+0000 7f7417907640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.102:6812 osd.4 since back 2024-09-10T10:06:21.821307+0000 front 2024-09-10T10:06:21.821315+0000 (oldest deadline 2024-09-10T10:06:44.721409+0000) 2024-09-10T10:06:56.351 INFO:journalctl@ceph.osd.8.smithi159.stdout:Sep 10 10:06:56 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-8[53439]: 2024-09-10T10:06:56.074+0000 7f7417907640 -1 osd.8 100 heartbeat_check: no reply from 172.21.15.159:6812 osd.5 since back 2024-09-10T10:06:29.322920+0000 front 2024-09-10T10:06:29.322884+0000 (oldest deadline 2024-09-10T10:06:52.222898+0000) 2024-09-10T10:06:56.556 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 10 10:06:56 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-9[65288]: 2024-09-10T10:06:56.153+0000 7f84af200640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-10T10:05:47.346189+0000 front 2024-09-10T10:05:47.346220+0000 (oldest deadline 2024-09-10T10:06:12.646451+0000) 2024-09-10T10:06:56.556 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 10 10:06:56 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-9[65288]: 2024-09-10T10:06:56.153+0000 7f84af200640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-09-10T10:05:57.348185+0000 front 2024-09-10T10:05:57.348335+0000 (oldest deadline 2024-09-10T10:06:23.248210+0000) 2024-09-10T10:06:56.557 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 10 10:06:56 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-9[65288]: 2024-09-10T10:06:56.153+0000 7f84af200640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-10T10:06:03.249484+0000 front 2024-09-10T10:06:03.249185+0000 (oldest deadline 2024-09-10T10:06:26.749283+0000) 2024-09-10T10:06:56.557 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 10 10:06:56 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-9[65288]: 2024-09-10T10:06:56.153+0000 7f84af200640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-10T10:06:12.650860+0000 front 2024-09-10T10:06:12.650900+0000 (oldest deadline 2024-09-10T10:06:38.551142+0000) 2024-09-10T10:06:56.557 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 10 10:06:56 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-9[65288]: 2024-09-10T10:06:56.153+0000 7f84af200640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.102:6812 osd.4 since back 2024-09-10T10:06:18.551683+0000 front 2024-09-10T10:06:18.551825+0000 (oldest deadline 2024-09-10T10:06:44.451846+0000) 2024-09-10T10:06:56.851 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 10 10:06:56 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-11[60437]: 2024-09-10T10:06:56.504+0000 7fa78e4d3640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-10T10:05:49.428950+0000 front 2024-09-10T10:05:49.429031+0000 (oldest deadline 2024-09-10T10:06:11.128800+0000) 2024-09-10T10:06:56.851 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 10 10:06:56 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-11[60437]: 2024-09-10T10:06:56.504+0000 7fa78e4d3640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-10T10:06:03.430910+0000 front 2024-09-10T10:06:03.430889+0000 (oldest deadline 2024-09-10T10:06:29.331091+0000) 2024-09-10T10:06:56.851 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 10 10:06:56 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-11[60437]: 2024-09-10T10:06:56.504+0000 7fa78e4d3640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-10T10:06:13.432384+0000 front 2024-09-10T10:06:13.432489+0000 (oldest deadline 2024-09-10T10:06:35.732466+0000) 2024-09-10T10:06:56.851 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 10 10:06:56 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-11[60437]: 2024-09-10T10:06:56.504+0000 7fa78e4d3640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.102:6812 osd.4 since back 2024-09-10T10:06:22.734557+0000 front 2024-09-10T10:06:22.734365+0000 (oldest deadline 2024-09-10T10:06:44.434370+0000) 2024-09-10T10:06:56.851 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 10 10:06:56 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-11[60437]: 2024-09-10T10:06:56.504+0000 7fa78e4d3640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.159:6812 osd.5 since back 2024-09-10T10:06:29.635809+0000 front 2024-09-10T10:06:29.635901+0000 (oldest deadline 2024-09-10T10:06:53.735941+0000) 2024-09-10T10:06:56.885 INFO:journalctl@ceph.osd.10.smithi102.stdout:Sep 10 10:06:56 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-10[59874]: 2024-09-10T10:06:56.628+0000 7f40f3335640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-10T10:05:49.575097+0000 front 2024-09-10T10:05:49.575141+0000 (oldest deadline 2024-09-10T10:06:13.075035+0000) 2024-09-10T10:06:56.885 INFO:journalctl@ceph.osd.10.smithi102.stdout:Sep 10 10:06:56 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-10[59874]: 2024-09-10T10:06:56.628+0000 7f40f3335640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-09-10T10:05:54.176094+0000 front 2024-09-10T10:05:54.176115+0000 (oldest deadline 2024-09-10T10:06:19.376783+0000) 2024-09-10T10:06:56.885 INFO:journalctl@ceph.osd.10.smithi102.stdout:Sep 10 10:06:56 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-10[59874]: 2024-09-10T10:06:56.628+0000 7f40f3335640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-10T10:06:03.378660+0000 front 2024-09-10T10:06:03.378719+0000 (oldest deadline 2024-09-10T10:06:32.178984+0000) 2024-09-10T10:06:56.885 INFO:journalctl@ceph.osd.10.smithi102.stdout:Sep 10 10:06:56 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-10[59874]: 2024-09-10T10:06:56.628+0000 7f40f3335640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-10T10:06:12.179545+0000 front 2024-09-10T10:06:12.179581+0000 (oldest deadline 2024-09-10T10:06:36.879462+0000) 2024-09-10T10:06:56.885 INFO:journalctl@ceph.osd.10.smithi102.stdout:Sep 10 10:06:56 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-10[59874]: 2024-09-10T10:06:56.628+0000 7f40f3335640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.102:6812 osd.4 since back 2024-09-10T10:06:22.581662+0000 front 2024-09-10T10:06:22.581640+0000 (oldest deadline 2024-09-10T10:06:44.281620+0000) 2024-09-10T10:06:56.885 INFO:journalctl@ceph.osd.10.smithi102.stdout:Sep 10 10:06:56 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-10[59874]: 2024-09-10T10:06:56.628+0000 7f40f3335640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.159:6812 osd.5 since back 2024-09-10T10:06:28.382594+0000 front 2024-09-10T10:06:28.382505+0000 (oldest deadline 2024-09-10T10:06:53.682352+0000) 2024-09-10T10:06:57.322 INFO:journalctl@ceph.osd.8.smithi159.stdout:Sep 10 10:06:57 smithi159 podman[77139]: 2024-09-10 10:06:57.033010236 +0000 UTC m=+5.522188056 container died 18cf8016a8927279fc127f18712659a255f80e7a0ae715e99f6fd0392541c73a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-8, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD) 2024-09-10T10:06:57.556 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 10 10:06:57 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-9[65288]: 2024-09-10T10:06:57.142+0000 7f84af200640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-10T10:05:47.346189+0000 front 2024-09-10T10:05:47.346220+0000 (oldest deadline 2024-09-10T10:06:12.646451+0000) 2024-09-10T10:06:57.556 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 10 10:06:57 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-9[65288]: 2024-09-10T10:06:57.142+0000 7f84af200640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-09-10T10:05:57.348185+0000 front 2024-09-10T10:05:57.348335+0000 (oldest deadline 2024-09-10T10:06:23.248210+0000) 2024-09-10T10:06:57.556 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 10 10:06:57 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-9[65288]: 2024-09-10T10:06:57.142+0000 7f84af200640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-10T10:06:03.249484+0000 front 2024-09-10T10:06:03.249185+0000 (oldest deadline 2024-09-10T10:06:26.749283+0000) 2024-09-10T10:06:57.556 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 10 10:06:57 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-9[65288]: 2024-09-10T10:06:57.142+0000 7f84af200640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-10T10:06:12.650860+0000 front 2024-09-10T10:06:12.650900+0000 (oldest deadline 2024-09-10T10:06:38.551142+0000) 2024-09-10T10:06:57.556 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 10 10:06:57 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-9[65288]: 2024-09-10T10:06:57.142+0000 7f84af200640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.102:6812 osd.4 since back 2024-09-10T10:06:18.551683+0000 front 2024-09-10T10:06:18.551825+0000 (oldest deadline 2024-09-10T10:06:44.451846+0000) 2024-09-10T10:06:57.600 INFO:journalctl@ceph.osd.8.smithi159.stdout:Sep 10 10:06:57 smithi159 podman[77139]: 2024-09-10 10:06:57.4208552 +0000 UTC m=+5.910033020 container cleanup 18cf8016a8927279fc127f18712659a255f80e7a0ae715e99f6fd0392541c73a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-8, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, ceph=True, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2) 2024-09-10T10:06:57.600 INFO:journalctl@ceph.osd.8.smithi159.stdout:Sep 10 10:06:57 smithi159 bash[77139]: ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-8 2024-09-10T10:06:57.600 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 10 10:06:57 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-11[60437]: 2024-09-10T10:06:57.466+0000 7fa78e4d3640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-10T10:05:49.428950+0000 front 2024-09-10T10:05:49.429031+0000 (oldest deadline 2024-09-10T10:06:11.128800+0000) 2024-09-10T10:06:57.601 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 10 10:06:57 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-11[60437]: 2024-09-10T10:06:57.466+0000 7fa78e4d3640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-10T10:06:03.430910+0000 front 2024-09-10T10:06:03.430889+0000 (oldest deadline 2024-09-10T10:06:29.331091+0000) 2024-09-10T10:06:57.601 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 10 10:06:57 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-11[60437]: 2024-09-10T10:06:57.466+0000 7fa78e4d3640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-10T10:06:13.432384+0000 front 2024-09-10T10:06:13.432489+0000 (oldest deadline 2024-09-10T10:06:35.732466+0000) 2024-09-10T10:06:57.601 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 10 10:06:57 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-11[60437]: 2024-09-10T10:06:57.466+0000 7fa78e4d3640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.102:6812 osd.4 since back 2024-09-10T10:06:22.734557+0000 front 2024-09-10T10:06:22.734365+0000 (oldest deadline 2024-09-10T10:06:44.434370+0000) 2024-09-10T10:06:57.601 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 10 10:06:57 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-11[60437]: 2024-09-10T10:06:57.466+0000 7fa78e4d3640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.159:6812 osd.5 since back 2024-09-10T10:06:29.635809+0000 front 2024-09-10T10:06:29.635901+0000 (oldest deadline 2024-09-10T10:06:53.735941+0000) 2024-09-10T10:06:57.884 INFO:journalctl@ceph.osd.10.smithi102.stdout:Sep 10 10:06:57 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-10[59874]: 2024-09-10T10:06:57.602+0000 7f40f3335640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-10T10:05:49.575097+0000 front 2024-09-10T10:05:49.575141+0000 (oldest deadline 2024-09-10T10:06:13.075035+0000) 2024-09-10T10:06:57.885 INFO:journalctl@ceph.osd.10.smithi102.stdout:Sep 10 10:06:57 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-10[59874]: 2024-09-10T10:06:57.602+0000 7f40f3335640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-09-10T10:05:54.176094+0000 front 2024-09-10T10:05:54.176115+0000 (oldest deadline 2024-09-10T10:06:19.376783+0000) 2024-09-10T10:06:57.885 INFO:journalctl@ceph.osd.10.smithi102.stdout:Sep 10 10:06:57 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-10[59874]: 2024-09-10T10:06:57.602+0000 7f40f3335640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-10T10:06:03.378660+0000 front 2024-09-10T10:06:03.378719+0000 (oldest deadline 2024-09-10T10:06:32.178984+0000) 2024-09-10T10:06:57.885 INFO:journalctl@ceph.osd.10.smithi102.stdout:Sep 10 10:06:57 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-10[59874]: 2024-09-10T10:06:57.602+0000 7f40f3335640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-10T10:06:12.179545+0000 front 2024-09-10T10:06:12.179581+0000 (oldest deadline 2024-09-10T10:06:36.879462+0000) 2024-09-10T10:06:57.885 INFO:journalctl@ceph.osd.10.smithi102.stdout:Sep 10 10:06:57 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-10[59874]: 2024-09-10T10:06:57.602+0000 7f40f3335640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.102:6812 osd.4 since back 2024-09-10T10:06:22.581662+0000 front 2024-09-10T10:06:22.581640+0000 (oldest deadline 2024-09-10T10:06:44.281620+0000) 2024-09-10T10:06:57.885 INFO:journalctl@ceph.osd.10.smithi102.stdout:Sep 10 10:06:57 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-10[59874]: 2024-09-10T10:06:57.602+0000 7f40f3335640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.159:6812 osd.5 since back 2024-09-10T10:06:28.382594+0000 front 2024-09-10T10:06:28.382505+0000 (oldest deadline 2024-09-10T10:06:53.682352+0000) 2024-09-10T10:06:57.981 INFO:journalctl@ceph.osd.8.smithi159.stdout:Sep 10 10:06:57 smithi159 podman[77155]: 2024-09-10 10:06:57.663877196 +0000 UTC m=+0.625618990 container remove 18cf8016a8927279fc127f18712659a255f80e7a0ae715e99f6fd0392541c73a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-8, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-09-10T10:06:58.337 INFO:journalctl@ceph.osd.8.smithi159.stdout:Sep 10 10:06:57 smithi159 podman[77243]: 2024-09-10 10:06:57.981386202 +0000 UTC m=+0.108447271 container create 329a426a9bdff5e5fd9272afeee4d1da34fc81f645a0c7f85f7260b8b979828f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-8-deactivate, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-09-10T10:06:58.337 INFO:journalctl@ceph.osd.8.smithi159.stdout:Sep 10 10:06:57 smithi159 podman[77243]: 2024-09-10 10:06:57.889312329 +0000 UTC m=+0.016373349 image pull 41d67112835771c3efca60c6b8fe297c7ddaecdc26cea26fd28aa18636e49005 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 2024-09-10T10:06:58.337 INFO:journalctl@ceph.osd.8.smithi159.stdout:Sep 10 10:06:58 smithi159 podman[77243]: 2024-09-10 10:06:58.217206278 +0000 UTC m=+0.344267284 container init 329a426a9bdff5e5fd9272afeee4d1da34fc81f645a0c7f85f7260b8b979828f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-8-deactivate, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-09-10T10:06:58.338 INFO:journalctl@ceph.osd.8.smithi159.stdout:Sep 10 10:06:58 smithi159 podman[77243]: 2024-09-10 10:06:58.220607267 +0000 UTC m=+0.347668280 container start 329a426a9bdff5e5fd9272afeee4d1da34fc81f645a0c7f85f7260b8b979828f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-8-deactivate, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-10T10:06:58.338 INFO:journalctl@ceph.osd.8.smithi159.stdout:Sep 10 10:06:58 smithi159 podman[77243]: 2024-09-10 10:06:58.325381056 +0000 UTC m=+0.452442078 container attach 329a426a9bdff5e5fd9272afeee4d1da34fc81f645a0c7f85f7260b8b979828f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-8-deactivate, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.build-date=20240731, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b) 2024-09-10T10:06:58.556 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 10 10:06:58 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-9[65288]: 2024-09-10T10:06:58.139+0000 7f84af200640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-10T10:05:47.346189+0000 front 2024-09-10T10:05:47.346220+0000 (oldest deadline 2024-09-10T10:06:12.646451+0000) 2024-09-10T10:06:58.557 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 10 10:06:58 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-9[65288]: 2024-09-10T10:06:58.139+0000 7f84af200640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-09-10T10:05:57.348185+0000 front 2024-09-10T10:05:57.348335+0000 (oldest deadline 2024-09-10T10:06:23.248210+0000) 2024-09-10T10:06:58.557 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 10 10:06:58 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-9[65288]: 2024-09-10T10:06:58.139+0000 7f84af200640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-10T10:06:03.249484+0000 front 2024-09-10T10:06:03.249185+0000 (oldest deadline 2024-09-10T10:06:26.749283+0000) 2024-09-10T10:06:58.557 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 10 10:06:58 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-9[65288]: 2024-09-10T10:06:58.139+0000 7f84af200640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-10T10:06:12.650860+0000 front 2024-09-10T10:06:12.650900+0000 (oldest deadline 2024-09-10T10:06:38.551142+0000) 2024-09-10T10:06:58.557 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 10 10:06:58 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-9[65288]: 2024-09-10T10:06:58.139+0000 7f84af200640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.102:6812 osd.4 since back 2024-09-10T10:06:18.551683+0000 front 2024-09-10T10:06:18.551825+0000 (oldest deadline 2024-09-10T10:06:44.451846+0000) 2024-09-10T10:06:58.557 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 10 10:06:58 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-9[65288]: 2024-09-10T10:06:58.139+0000 7f84af200640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.159:6812 osd.5 since back 2024-09-10T10:06:26.153334+0000 front 2024-09-10T10:06:26.153303+0000 (oldest deadline 2024-09-10T10:06:57.354360+0000) 2024-09-10T10:06:58.601 INFO:journalctl@ceph.osd.8.smithi159.stdout:Sep 10 10:06:58 smithi159 podman[77243]: 2024-09-10 10:06:58.405962236 +0000 UTC m=+0.533023259 container died 329a426a9bdff5e5fd9272afeee4d1da34fc81f645a0c7f85f7260b8b979828f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-8-deactivate, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, ceph=True, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.build-date=20240731) 2024-09-10T10:06:58.601 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 10 10:06:58 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-11[60437]: 2024-09-10T10:06:58.510+0000 7fa78e4d3640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-10T10:05:49.428950+0000 front 2024-09-10T10:05:49.429031+0000 (oldest deadline 2024-09-10T10:06:11.128800+0000) 2024-09-10T10:06:58.601 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 10 10:06:58 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-11[60437]: 2024-09-10T10:06:58.510+0000 7fa78e4d3640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-10T10:06:03.430910+0000 front 2024-09-10T10:06:03.430889+0000 (oldest deadline 2024-09-10T10:06:29.331091+0000) 2024-09-10T10:06:58.601 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 10 10:06:58 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-11[60437]: 2024-09-10T10:06:58.510+0000 7fa78e4d3640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-10T10:06:13.432384+0000 front 2024-09-10T10:06:13.432489+0000 (oldest deadline 2024-09-10T10:06:35.732466+0000) 2024-09-10T10:06:58.601 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 10 10:06:58 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-11[60437]: 2024-09-10T10:06:58.510+0000 7fa78e4d3640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.102:6812 osd.4 since back 2024-09-10T10:06:22.734557+0000 front 2024-09-10T10:06:22.734365+0000 (oldest deadline 2024-09-10T10:06:44.434370+0000) 2024-09-10T10:06:58.601 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 10 10:06:58 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-11[60437]: 2024-09-10T10:06:58.510+0000 7fa78e4d3640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.159:6812 osd.5 since back 2024-09-10T10:06:29.635809+0000 front 2024-09-10T10:06:29.635901+0000 (oldest deadline 2024-09-10T10:06:53.735941+0000) 2024-09-10T10:06:59.104 DEBUG:teuthology.orchestra.run.smithi159:> sudo pkill -f 'journalctl -f -n 0 -u ceph-bade4b50-6f56-11ef-bcea-c7b262605968@osd.8.service' 2024-09-10T10:06:59.134 INFO:journalctl@ceph.osd.10.smithi102.stdout:Sep 10 10:06:58 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-10[59874]: 2024-09-10T10:06:58.635+0000 7f40f3335640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-10T10:05:49.575097+0000 front 2024-09-10T10:05:49.575141+0000 (oldest deadline 2024-09-10T10:06:13.075035+0000) 2024-09-10T10:06:59.134 INFO:journalctl@ceph.osd.10.smithi102.stdout:Sep 10 10:06:58 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-10[59874]: 2024-09-10T10:06:58.635+0000 7f40f3335640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-09-10T10:05:54.176094+0000 front 2024-09-10T10:05:54.176115+0000 (oldest deadline 2024-09-10T10:06:19.376783+0000) 2024-09-10T10:06:59.135 INFO:journalctl@ceph.osd.10.smithi102.stdout:Sep 10 10:06:58 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-10[59874]: 2024-09-10T10:06:58.635+0000 7f40f3335640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-10T10:06:03.378660+0000 front 2024-09-10T10:06:03.378719+0000 (oldest deadline 2024-09-10T10:06:32.178984+0000) 2024-09-10T10:06:59.135 INFO:journalctl@ceph.osd.10.smithi102.stdout:Sep 10 10:06:58 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-10[59874]: 2024-09-10T10:06:58.635+0000 7f40f3335640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-10T10:06:12.179545+0000 front 2024-09-10T10:06:12.179581+0000 (oldest deadline 2024-09-10T10:06:36.879462+0000) 2024-09-10T10:06:59.135 INFO:journalctl@ceph.osd.10.smithi102.stdout:Sep 10 10:06:58 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-10[59874]: 2024-09-10T10:06:58.635+0000 7f40f3335640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.102:6812 osd.4 since back 2024-09-10T10:06:22.581662+0000 front 2024-09-10T10:06:22.581640+0000 (oldest deadline 2024-09-10T10:06:44.281620+0000) 2024-09-10T10:06:59.135 INFO:journalctl@ceph.osd.10.smithi102.stdout:Sep 10 10:06:58 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-10[59874]: 2024-09-10T10:06:58.635+0000 7f40f3335640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.159:6812 osd.5 since back 2024-09-10T10:06:28.382594+0000 front 2024-09-10T10:06:28.382505+0000 (oldest deadline 2024-09-10T10:06:53.682352+0000) 2024-09-10T10:06:59.184 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T10:06:59.185 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-09-10T10:06:59.185 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-09-10T10:06:59.185 DEBUG:teuthology.orchestra.run.smithi005:> sudo systemctl stop ceph-bade4b50-6f56-11ef-bcea-c7b262605968@osd.9 2024-09-10T10:06:59.556 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 10 10:06:59 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-9[65288]: 2024-09-10T10:06:59.138+0000 7f84af200640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-10T10:05:47.346189+0000 front 2024-09-10T10:05:47.346220+0000 (oldest deadline 2024-09-10T10:06:12.646451+0000) 2024-09-10T10:06:59.556 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 10 10:06:59 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-9[65288]: 2024-09-10T10:06:59.138+0000 7f84af200640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-09-10T10:05:57.348185+0000 front 2024-09-10T10:05:57.348335+0000 (oldest deadline 2024-09-10T10:06:23.248210+0000) 2024-09-10T10:06:59.556 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 10 10:06:59 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-9[65288]: 2024-09-10T10:06:59.138+0000 7f84af200640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-10T10:06:03.249484+0000 front 2024-09-10T10:06:03.249185+0000 (oldest deadline 2024-09-10T10:06:26.749283+0000) 2024-09-10T10:06:59.557 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 10 10:06:59 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-9[65288]: 2024-09-10T10:06:59.138+0000 7f84af200640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-10T10:06:12.650860+0000 front 2024-09-10T10:06:12.650900+0000 (oldest deadline 2024-09-10T10:06:38.551142+0000) 2024-09-10T10:06:59.557 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 10 10:06:59 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-9[65288]: 2024-09-10T10:06:59.138+0000 7f84af200640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.102:6812 osd.4 since back 2024-09-10T10:06:18.551683+0000 front 2024-09-10T10:06:18.551825+0000 (oldest deadline 2024-09-10T10:06:44.451846+0000) 2024-09-10T10:06:59.557 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 10 10:06:59 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-9[65288]: 2024-09-10T10:06:59.138+0000 7f84af200640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.159:6812 osd.5 since back 2024-09-10T10:06:26.153334+0000 front 2024-09-10T10:06:26.153303+0000 (oldest deadline 2024-09-10T10:06:57.354360+0000) 2024-09-10T10:06:59.557 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 10 10:06:59 smithi005 systemd[1]: Stopping Ceph osd.9 for bade4b50-6f56-11ef-bcea-c7b262605968... 2024-09-10T10:06:59.851 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 10 10:06:59 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-11[60437]: 2024-09-10T10:06:59.502+0000 7fa78e4d3640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-10T10:05:49.428950+0000 front 2024-09-10T10:05:49.429031+0000 (oldest deadline 2024-09-10T10:06:11.128800+0000) 2024-09-10T10:06:59.851 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 10 10:06:59 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-11[60437]: 2024-09-10T10:06:59.502+0000 7fa78e4d3640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-10T10:06:03.430910+0000 front 2024-09-10T10:06:03.430889+0000 (oldest deadline 2024-09-10T10:06:29.331091+0000) 2024-09-10T10:06:59.851 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 10 10:06:59 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-11[60437]: 2024-09-10T10:06:59.502+0000 7fa78e4d3640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-10T10:06:13.432384+0000 front 2024-09-10T10:06:13.432489+0000 (oldest deadline 2024-09-10T10:06:35.732466+0000) 2024-09-10T10:06:59.851 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 10 10:06:59 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-11[60437]: 2024-09-10T10:06:59.502+0000 7fa78e4d3640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.102:6812 osd.4 since back 2024-09-10T10:06:22.734557+0000 front 2024-09-10T10:06:22.734365+0000 (oldest deadline 2024-09-10T10:06:44.434370+0000) 2024-09-10T10:06:59.851 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 10 10:06:59 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-11[60437]: 2024-09-10T10:06:59.502+0000 7fa78e4d3640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.159:6812 osd.5 since back 2024-09-10T10:06:29.635809+0000 front 2024-09-10T10:06:29.635901+0000 (oldest deadline 2024-09-10T10:06:53.735941+0000) 2024-09-10T10:06:59.884 INFO:journalctl@ceph.osd.10.smithi102.stdout:Sep 10 10:06:59 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-10[59874]: 2024-09-10T10:06:59.613+0000 7f40f3335640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-10T10:05:49.575097+0000 front 2024-09-10T10:05:49.575141+0000 (oldest deadline 2024-09-10T10:06:13.075035+0000) 2024-09-10T10:06:59.885 INFO:journalctl@ceph.osd.10.smithi102.stdout:Sep 10 10:06:59 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-10[59874]: 2024-09-10T10:06:59.613+0000 7f40f3335640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-09-10T10:05:54.176094+0000 front 2024-09-10T10:05:54.176115+0000 (oldest deadline 2024-09-10T10:06:19.376783+0000) 2024-09-10T10:06:59.885 INFO:journalctl@ceph.osd.10.smithi102.stdout:Sep 10 10:06:59 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-10[59874]: 2024-09-10T10:06:59.613+0000 7f40f3335640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-10T10:06:03.378660+0000 front 2024-09-10T10:06:03.378719+0000 (oldest deadline 2024-09-10T10:06:32.178984+0000) 2024-09-10T10:06:59.885 INFO:journalctl@ceph.osd.10.smithi102.stdout:Sep 10 10:06:59 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-10[59874]: 2024-09-10T10:06:59.613+0000 7f40f3335640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-10T10:06:12.179545+0000 front 2024-09-10T10:06:12.179581+0000 (oldest deadline 2024-09-10T10:06:36.879462+0000) 2024-09-10T10:06:59.885 INFO:journalctl@ceph.osd.10.smithi102.stdout:Sep 10 10:06:59 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-10[59874]: 2024-09-10T10:06:59.613+0000 7f40f3335640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.102:6812 osd.4 since back 2024-09-10T10:06:22.581662+0000 front 2024-09-10T10:06:22.581640+0000 (oldest deadline 2024-09-10T10:06:44.281620+0000) 2024-09-10T10:06:59.885 INFO:journalctl@ceph.osd.10.smithi102.stdout:Sep 10 10:06:59 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-10[59874]: 2024-09-10T10:06:59.613+0000 7f40f3335640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.159:6812 osd.5 since back 2024-09-10T10:06:28.382594+0000 front 2024-09-10T10:06:28.382505+0000 (oldest deadline 2024-09-10T10:06:53.682352+0000) 2024-09-10T10:07:00.056 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 10 10:06:59 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-9[65288]: 2024-09-10T10:06:59.601+0000 7f84b8800640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T10:07:00.056 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 10 10:06:59 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-9[65288]: 2024-09-10T10:06:59.601+0000 7f84b8800640 -1 osd.9 100 *** Got signal Terminated *** 2024-09-10T10:07:00.056 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 10 10:06:59 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-9[65288]: 2024-09-10T10:06:59.601+0000 7f84b8800640 -1 osd.9 100 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-10T10:07:00.556 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 10 10:07:00 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-9[65288]: 2024-09-10T10:07:00.154+0000 7f84af200640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-10T10:05:47.346189+0000 front 2024-09-10T10:05:47.346220+0000 (oldest deadline 2024-09-10T10:06:12.646451+0000) 2024-09-10T10:07:00.557 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 10 10:07:00 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-9[65288]: 2024-09-10T10:07:00.154+0000 7f84af200640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-09-10T10:05:57.348185+0000 front 2024-09-10T10:05:57.348335+0000 (oldest deadline 2024-09-10T10:06:23.248210+0000) 2024-09-10T10:07:00.557 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 10 10:07:00 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-9[65288]: 2024-09-10T10:07:00.154+0000 7f84af200640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-10T10:06:03.249484+0000 front 2024-09-10T10:06:03.249185+0000 (oldest deadline 2024-09-10T10:06:26.749283+0000) 2024-09-10T10:07:00.557 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 10 10:07:00 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-9[65288]: 2024-09-10T10:07:00.154+0000 7f84af200640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-10T10:06:12.650860+0000 front 2024-09-10T10:06:12.650900+0000 (oldest deadline 2024-09-10T10:06:38.551142+0000) 2024-09-10T10:07:00.557 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 10 10:07:00 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-9[65288]: 2024-09-10T10:07:00.154+0000 7f84af200640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.102:6812 osd.4 since back 2024-09-10T10:06:18.551683+0000 front 2024-09-10T10:06:18.551825+0000 (oldest deadline 2024-09-10T10:06:44.451846+0000) 2024-09-10T10:07:00.557 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 10 10:07:00 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-9[65288]: 2024-09-10T10:07:00.154+0000 7f84af200640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.159:6812 osd.5 since back 2024-09-10T10:06:26.153334+0000 front 2024-09-10T10:06:26.153303+0000 (oldest deadline 2024-09-10T10:06:57.354360+0000) 2024-09-10T10:07:00.600 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 10 10:07:00 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-11[60437]: 2024-09-10T10:07:00.469+0000 7fa78e4d3640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-10T10:05:49.428950+0000 front 2024-09-10T10:05:49.429031+0000 (oldest deadline 2024-09-10T10:06:11.128800+0000) 2024-09-10T10:07:00.600 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 10 10:07:00 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-11[60437]: 2024-09-10T10:07:00.469+0000 7fa78e4d3640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-10T10:06:03.430910+0000 front 2024-09-10T10:06:03.430889+0000 (oldest deadline 2024-09-10T10:06:29.331091+0000) 2024-09-10T10:07:00.601 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 10 10:07:00 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-11[60437]: 2024-09-10T10:07:00.469+0000 7fa78e4d3640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-10T10:06:13.432384+0000 front 2024-09-10T10:06:13.432489+0000 (oldest deadline 2024-09-10T10:06:35.732466+0000) 2024-09-10T10:07:00.601 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 10 10:07:00 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-11[60437]: 2024-09-10T10:07:00.469+0000 7fa78e4d3640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.102:6812 osd.4 since back 2024-09-10T10:06:22.734557+0000 front 2024-09-10T10:06:22.734365+0000 (oldest deadline 2024-09-10T10:06:44.434370+0000) 2024-09-10T10:07:00.601 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 10 10:07:00 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-11[60437]: 2024-09-10T10:07:00.469+0000 7fa78e4d3640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.159:6812 osd.5 since back 2024-09-10T10:06:29.635809+0000 front 2024-09-10T10:06:29.635901+0000 (oldest deadline 2024-09-10T10:06:53.735941+0000) 2024-09-10T10:07:01.134 INFO:journalctl@ceph.osd.10.smithi102.stdout:Sep 10 10:07:00 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-10[59874]: 2024-09-10T10:07:00.641+0000 7f40f3335640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-10T10:05:49.575097+0000 front 2024-09-10T10:05:49.575141+0000 (oldest deadline 2024-09-10T10:06:13.075035+0000) 2024-09-10T10:07:01.135 INFO:journalctl@ceph.osd.10.smithi102.stdout:Sep 10 10:07:00 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-10[59874]: 2024-09-10T10:07:00.641+0000 7f40f3335640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-09-10T10:05:54.176094+0000 front 2024-09-10T10:05:54.176115+0000 (oldest deadline 2024-09-10T10:06:19.376783+0000) 2024-09-10T10:07:01.135 INFO:journalctl@ceph.osd.10.smithi102.stdout:Sep 10 10:07:00 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-10[59874]: 2024-09-10T10:07:00.641+0000 7f40f3335640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-10T10:06:03.378660+0000 front 2024-09-10T10:06:03.378719+0000 (oldest deadline 2024-09-10T10:06:32.178984+0000) 2024-09-10T10:07:01.135 INFO:journalctl@ceph.osd.10.smithi102.stdout:Sep 10 10:07:00 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-10[59874]: 2024-09-10T10:07:00.641+0000 7f40f3335640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-10T10:06:12.179545+0000 front 2024-09-10T10:06:12.179581+0000 (oldest deadline 2024-09-10T10:06:36.879462+0000) 2024-09-10T10:07:01.135 INFO:journalctl@ceph.osd.10.smithi102.stdout:Sep 10 10:07:00 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-10[59874]: 2024-09-10T10:07:00.641+0000 7f40f3335640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.102:6812 osd.4 since back 2024-09-10T10:06:22.581662+0000 front 2024-09-10T10:06:22.581640+0000 (oldest deadline 2024-09-10T10:06:44.281620+0000) 2024-09-10T10:07:01.135 INFO:journalctl@ceph.osd.10.smithi102.stdout:Sep 10 10:07:00 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-10[59874]: 2024-09-10T10:07:00.641+0000 7f40f3335640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.159:6812 osd.5 since back 2024-09-10T10:06:28.382594+0000 front 2024-09-10T10:06:28.382505+0000 (oldest deadline 2024-09-10T10:06:53.682352+0000) 2024-09-10T10:07:01.556 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 10 10:07:01 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-9[65288]: 2024-09-10T10:07:01.155+0000 7f84af200640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-10T10:05:47.346189+0000 front 2024-09-10T10:05:47.346220+0000 (oldest deadline 2024-09-10T10:06:12.646451+0000) 2024-09-10T10:07:01.556 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 10 10:07:01 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-9[65288]: 2024-09-10T10:07:01.155+0000 7f84af200640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-09-10T10:05:57.348185+0000 front 2024-09-10T10:05:57.348335+0000 (oldest deadline 2024-09-10T10:06:23.248210+0000) 2024-09-10T10:07:01.556 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 10 10:07:01 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-9[65288]: 2024-09-10T10:07:01.155+0000 7f84af200640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-10T10:06:03.249484+0000 front 2024-09-10T10:06:03.249185+0000 (oldest deadline 2024-09-10T10:06:26.749283+0000) 2024-09-10T10:07:01.556 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 10 10:07:01 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-9[65288]: 2024-09-10T10:07:01.155+0000 7f84af200640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-10T10:06:12.650860+0000 front 2024-09-10T10:06:12.650900+0000 (oldest deadline 2024-09-10T10:06:38.551142+0000) 2024-09-10T10:07:01.557 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 10 10:07:01 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-9[65288]: 2024-09-10T10:07:01.155+0000 7f84af200640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.102:6812 osd.4 since back 2024-09-10T10:06:18.551683+0000 front 2024-09-10T10:06:18.551825+0000 (oldest deadline 2024-09-10T10:06:44.451846+0000) 2024-09-10T10:07:01.557 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 10 10:07:01 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-9[65288]: 2024-09-10T10:07:01.155+0000 7f84af200640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.159:6812 osd.5 since back 2024-09-10T10:06:26.153334+0000 front 2024-09-10T10:06:26.153303+0000 (oldest deadline 2024-09-10T10:06:57.354360+0000) 2024-09-10T10:07:01.850 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 10 10:07:01 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-11[60437]: 2024-09-10T10:07:01.490+0000 7fa78e4d3640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-10T10:05:49.428950+0000 front 2024-09-10T10:05:49.429031+0000 (oldest deadline 2024-09-10T10:06:11.128800+0000) 2024-09-10T10:07:01.851 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 10 10:07:01 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-11[60437]: 2024-09-10T10:07:01.490+0000 7fa78e4d3640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-10T10:06:03.430910+0000 front 2024-09-10T10:06:03.430889+0000 (oldest deadline 2024-09-10T10:06:29.331091+0000) 2024-09-10T10:07:01.851 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 10 10:07:01 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-11[60437]: 2024-09-10T10:07:01.490+0000 7fa78e4d3640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-10T10:06:13.432384+0000 front 2024-09-10T10:06:13.432489+0000 (oldest deadline 2024-09-10T10:06:35.732466+0000) 2024-09-10T10:07:01.851 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 10 10:07:01 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-11[60437]: 2024-09-10T10:07:01.490+0000 7fa78e4d3640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.102:6812 osd.4 since back 2024-09-10T10:06:22.734557+0000 front 2024-09-10T10:06:22.734365+0000 (oldest deadline 2024-09-10T10:06:44.434370+0000) 2024-09-10T10:07:01.851 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 10 10:07:01 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-11[60437]: 2024-09-10T10:07:01.490+0000 7fa78e4d3640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.159:6812 osd.5 since back 2024-09-10T10:06:29.635809+0000 front 2024-09-10T10:06:29.635901+0000 (oldest deadline 2024-09-10T10:06:53.735941+0000) 2024-09-10T10:07:02.134 INFO:journalctl@ceph.osd.10.smithi102.stdout:Sep 10 10:07:01 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-10[59874]: 2024-09-10T10:07:01.634+0000 7f40f3335640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-10T10:05:49.575097+0000 front 2024-09-10T10:05:49.575141+0000 (oldest deadline 2024-09-10T10:06:13.075035+0000) 2024-09-10T10:07:02.135 INFO:journalctl@ceph.osd.10.smithi102.stdout:Sep 10 10:07:01 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-10[59874]: 2024-09-10T10:07:01.634+0000 7f40f3335640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-09-10T10:05:54.176094+0000 front 2024-09-10T10:05:54.176115+0000 (oldest deadline 2024-09-10T10:06:19.376783+0000) 2024-09-10T10:07:02.135 INFO:journalctl@ceph.osd.10.smithi102.stdout:Sep 10 10:07:01 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-10[59874]: 2024-09-10T10:07:01.634+0000 7f40f3335640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-10T10:06:03.378660+0000 front 2024-09-10T10:06:03.378719+0000 (oldest deadline 2024-09-10T10:06:32.178984+0000) 2024-09-10T10:07:02.135 INFO:journalctl@ceph.osd.10.smithi102.stdout:Sep 10 10:07:01 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-10[59874]: 2024-09-10T10:07:01.634+0000 7f40f3335640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-10T10:06:12.179545+0000 front 2024-09-10T10:06:12.179581+0000 (oldest deadline 2024-09-10T10:06:36.879462+0000) 2024-09-10T10:07:02.135 INFO:journalctl@ceph.osd.10.smithi102.stdout:Sep 10 10:07:01 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-10[59874]: 2024-09-10T10:07:01.634+0000 7f40f3335640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.102:6812 osd.4 since back 2024-09-10T10:06:22.581662+0000 front 2024-09-10T10:06:22.581640+0000 (oldest deadline 2024-09-10T10:06:44.281620+0000) 2024-09-10T10:07:02.135 INFO:journalctl@ceph.osd.10.smithi102.stdout:Sep 10 10:07:01 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-10[59874]: 2024-09-10T10:07:01.634+0000 7f40f3335640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.159:6812 osd.5 since back 2024-09-10T10:06:28.382594+0000 front 2024-09-10T10:06:28.382505+0000 (oldest deadline 2024-09-10T10:06:53.682352+0000) 2024-09-10T10:07:02.556 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 10 10:07:02 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-9[65288]: 2024-09-10T10:07:02.168+0000 7f84af200640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-10T10:05:47.346189+0000 front 2024-09-10T10:05:47.346220+0000 (oldest deadline 2024-09-10T10:06:12.646451+0000) 2024-09-10T10:07:02.557 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 10 10:07:02 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-9[65288]: 2024-09-10T10:07:02.168+0000 7f84af200640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-09-10T10:05:57.348185+0000 front 2024-09-10T10:05:57.348335+0000 (oldest deadline 2024-09-10T10:06:23.248210+0000) 2024-09-10T10:07:02.557 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 10 10:07:02 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-9[65288]: 2024-09-10T10:07:02.168+0000 7f84af200640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-10T10:06:03.249484+0000 front 2024-09-10T10:06:03.249185+0000 (oldest deadline 2024-09-10T10:06:26.749283+0000) 2024-09-10T10:07:02.557 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 10 10:07:02 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-9[65288]: 2024-09-10T10:07:02.168+0000 7f84af200640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-10T10:06:12.650860+0000 front 2024-09-10T10:06:12.650900+0000 (oldest deadline 2024-09-10T10:06:38.551142+0000) 2024-09-10T10:07:02.557 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 10 10:07:02 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-9[65288]: 2024-09-10T10:07:02.168+0000 7f84af200640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.102:6812 osd.4 since back 2024-09-10T10:06:18.551683+0000 front 2024-09-10T10:06:18.551825+0000 (oldest deadline 2024-09-10T10:06:44.451846+0000) 2024-09-10T10:07:02.557 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 10 10:07:02 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-9[65288]: 2024-09-10T10:07:02.168+0000 7f84af200640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.159:6812 osd.5 since back 2024-09-10T10:06:26.153334+0000 front 2024-09-10T10:06:26.153303+0000 (oldest deadline 2024-09-10T10:06:57.354360+0000) 2024-09-10T10:07:02.850 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 10 10:07:02 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-11[60437]: 2024-09-10T10:07:02.516+0000 7fa78e4d3640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-10T10:05:49.428950+0000 front 2024-09-10T10:05:49.429031+0000 (oldest deadline 2024-09-10T10:06:11.128800+0000) 2024-09-10T10:07:02.850 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 10 10:07:02 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-11[60437]: 2024-09-10T10:07:02.516+0000 7fa78e4d3640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-10T10:06:03.430910+0000 front 2024-09-10T10:06:03.430889+0000 (oldest deadline 2024-09-10T10:06:29.331091+0000) 2024-09-10T10:07:02.850 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 10 10:07:02 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-11[60437]: 2024-09-10T10:07:02.516+0000 7fa78e4d3640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-10T10:06:13.432384+0000 front 2024-09-10T10:06:13.432489+0000 (oldest deadline 2024-09-10T10:06:35.732466+0000) 2024-09-10T10:07:02.851 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 10 10:07:02 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-11[60437]: 2024-09-10T10:07:02.516+0000 7fa78e4d3640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.102:6812 osd.4 since back 2024-09-10T10:06:22.734557+0000 front 2024-09-10T10:06:22.734365+0000 (oldest deadline 2024-09-10T10:06:44.434370+0000) 2024-09-10T10:07:02.851 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 10 10:07:02 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-11[60437]: 2024-09-10T10:07:02.516+0000 7fa78e4d3640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.159:6812 osd.5 since back 2024-09-10T10:06:29.635809+0000 front 2024-09-10T10:06:29.635901+0000 (oldest deadline 2024-09-10T10:06:53.735941+0000) 2024-09-10T10:07:02.851 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 10 10:07:02 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-11[60437]: 2024-09-10T10:07:02.516+0000 7fa78e4d3640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.5:6822 osd.6 since back 2024-09-10T10:06:38.937377+0000 front 2024-09-10T10:06:38.937447+0000 (oldest deadline 2024-09-10T10:07:01.837229+0000) 2024-09-10T10:07:02.884 INFO:journalctl@ceph.osd.10.smithi102.stdout:Sep 10 10:07:02 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-10[59874]: 2024-09-10T10:07:02.605+0000 7f40f3335640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-10T10:05:49.575097+0000 front 2024-09-10T10:05:49.575141+0000 (oldest deadline 2024-09-10T10:06:13.075035+0000) 2024-09-10T10:07:02.885 INFO:journalctl@ceph.osd.10.smithi102.stdout:Sep 10 10:07:02 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-10[59874]: 2024-09-10T10:07:02.605+0000 7f40f3335640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-09-10T10:05:54.176094+0000 front 2024-09-10T10:05:54.176115+0000 (oldest deadline 2024-09-10T10:06:19.376783+0000) 2024-09-10T10:07:02.885 INFO:journalctl@ceph.osd.10.smithi102.stdout:Sep 10 10:07:02 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-10[59874]: 2024-09-10T10:07:02.605+0000 7f40f3335640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-10T10:06:03.378660+0000 front 2024-09-10T10:06:03.378719+0000 (oldest deadline 2024-09-10T10:06:32.178984+0000) 2024-09-10T10:07:02.885 INFO:journalctl@ceph.osd.10.smithi102.stdout:Sep 10 10:07:02 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-10[59874]: 2024-09-10T10:07:02.605+0000 7f40f3335640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-10T10:06:12.179545+0000 front 2024-09-10T10:06:12.179581+0000 (oldest deadline 2024-09-10T10:06:36.879462+0000) 2024-09-10T10:07:02.885 INFO:journalctl@ceph.osd.10.smithi102.stdout:Sep 10 10:07:02 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-10[59874]: 2024-09-10T10:07:02.605+0000 7f40f3335640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.102:6812 osd.4 since back 2024-09-10T10:06:22.581662+0000 front 2024-09-10T10:06:22.581640+0000 (oldest deadline 2024-09-10T10:06:44.281620+0000) 2024-09-10T10:07:02.885 INFO:journalctl@ceph.osd.10.smithi102.stdout:Sep 10 10:07:02 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-10[59874]: 2024-09-10T10:07:02.605+0000 7f40f3335640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.159:6812 osd.5 since back 2024-09-10T10:06:28.382594+0000 front 2024-09-10T10:06:28.382505+0000 (oldest deadline 2024-09-10T10:06:53.682352+0000) 2024-09-10T10:07:03.556 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 10 10:07:03 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-9[65288]: 2024-09-10T10:07:03.159+0000 7f84af200640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-10T10:05:47.346189+0000 front 2024-09-10T10:05:47.346220+0000 (oldest deadline 2024-09-10T10:06:12.646451+0000) 2024-09-10T10:07:03.557 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 10 10:07:03 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-9[65288]: 2024-09-10T10:07:03.159+0000 7f84af200640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-09-10T10:05:57.348185+0000 front 2024-09-10T10:05:57.348335+0000 (oldest deadline 2024-09-10T10:06:23.248210+0000) 2024-09-10T10:07:03.557 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 10 10:07:03 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-9[65288]: 2024-09-10T10:07:03.159+0000 7f84af200640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-10T10:06:03.249484+0000 front 2024-09-10T10:06:03.249185+0000 (oldest deadline 2024-09-10T10:06:26.749283+0000) 2024-09-10T10:07:03.557 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 10 10:07:03 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-9[65288]: 2024-09-10T10:07:03.159+0000 7f84af200640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-10T10:06:12.650860+0000 front 2024-09-10T10:06:12.650900+0000 (oldest deadline 2024-09-10T10:06:38.551142+0000) 2024-09-10T10:07:03.557 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 10 10:07:03 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-9[65288]: 2024-09-10T10:07:03.159+0000 7f84af200640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.102:6812 osd.4 since back 2024-09-10T10:06:18.551683+0000 front 2024-09-10T10:06:18.551825+0000 (oldest deadline 2024-09-10T10:06:44.451846+0000) 2024-09-10T10:07:03.557 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 10 10:07:03 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-9[65288]: 2024-09-10T10:07:03.159+0000 7f84af200640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.159:6812 osd.5 since back 2024-09-10T10:06:26.153334+0000 front 2024-09-10T10:06:26.153303+0000 (oldest deadline 2024-09-10T10:06:57.354360+0000) 2024-09-10T10:07:03.850 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 10 10:07:03 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-11[60437]: 2024-09-10T10:07:03.494+0000 7fa78e4d3640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-10T10:05:49.428950+0000 front 2024-09-10T10:05:49.429031+0000 (oldest deadline 2024-09-10T10:06:11.128800+0000) 2024-09-10T10:07:03.851 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 10 10:07:03 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-11[60437]: 2024-09-10T10:07:03.494+0000 7fa78e4d3640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-10T10:06:03.430910+0000 front 2024-09-10T10:06:03.430889+0000 (oldest deadline 2024-09-10T10:06:29.331091+0000) 2024-09-10T10:07:03.851 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 10 10:07:03 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-11[60437]: 2024-09-10T10:07:03.494+0000 7fa78e4d3640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-10T10:06:13.432384+0000 front 2024-09-10T10:06:13.432489+0000 (oldest deadline 2024-09-10T10:06:35.732466+0000) 2024-09-10T10:07:03.851 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 10 10:07:03 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-11[60437]: 2024-09-10T10:07:03.494+0000 7fa78e4d3640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.102:6812 osd.4 since back 2024-09-10T10:06:22.734557+0000 front 2024-09-10T10:06:22.734365+0000 (oldest deadline 2024-09-10T10:06:44.434370+0000) 2024-09-10T10:07:03.851 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 10 10:07:03 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-11[60437]: 2024-09-10T10:07:03.494+0000 7fa78e4d3640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.159:6812 osd.5 since back 2024-09-10T10:06:29.635809+0000 front 2024-09-10T10:06:29.635901+0000 (oldest deadline 2024-09-10T10:06:53.735941+0000) 2024-09-10T10:07:03.851 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 10 10:07:03 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-11[60437]: 2024-09-10T10:07:03.494+0000 7fa78e4d3640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.5:6822 osd.6 since back 2024-09-10T10:06:38.937377+0000 front 2024-09-10T10:06:38.937447+0000 (oldest deadline 2024-09-10T10:07:01.837229+0000) 2024-09-10T10:07:03.884 INFO:journalctl@ceph.osd.10.smithi102.stdout:Sep 10 10:07:03 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-10[59874]: 2024-09-10T10:07:03.625+0000 7f40f3335640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-10T10:05:49.575097+0000 front 2024-09-10T10:05:49.575141+0000 (oldest deadline 2024-09-10T10:06:13.075035+0000) 2024-09-10T10:07:03.885 INFO:journalctl@ceph.osd.10.smithi102.stdout:Sep 10 10:07:03 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-10[59874]: 2024-09-10T10:07:03.625+0000 7f40f3335640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-09-10T10:05:54.176094+0000 front 2024-09-10T10:05:54.176115+0000 (oldest deadline 2024-09-10T10:06:19.376783+0000) 2024-09-10T10:07:03.885 INFO:journalctl@ceph.osd.10.smithi102.stdout:Sep 10 10:07:03 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-10[59874]: 2024-09-10T10:07:03.625+0000 7f40f3335640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-10T10:06:03.378660+0000 front 2024-09-10T10:06:03.378719+0000 (oldest deadline 2024-09-10T10:06:32.178984+0000) 2024-09-10T10:07:03.885 INFO:journalctl@ceph.osd.10.smithi102.stdout:Sep 10 10:07:03 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-10[59874]: 2024-09-10T10:07:03.625+0000 7f40f3335640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-10T10:06:12.179545+0000 front 2024-09-10T10:06:12.179581+0000 (oldest deadline 2024-09-10T10:06:36.879462+0000) 2024-09-10T10:07:03.885 INFO:journalctl@ceph.osd.10.smithi102.stdout:Sep 10 10:07:03 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-10[59874]: 2024-09-10T10:07:03.625+0000 7f40f3335640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.102:6812 osd.4 since back 2024-09-10T10:06:22.581662+0000 front 2024-09-10T10:06:22.581640+0000 (oldest deadline 2024-09-10T10:06:44.281620+0000) 2024-09-10T10:07:03.885 INFO:journalctl@ceph.osd.10.smithi102.stdout:Sep 10 10:07:03 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-10[59874]: 2024-09-10T10:07:03.625+0000 7f40f3335640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.159:6812 osd.5 since back 2024-09-10T10:06:28.382594+0000 front 2024-09-10T10:06:28.382505+0000 (oldest deadline 2024-09-10T10:06:53.682352+0000) 2024-09-10T10:07:03.885 INFO:journalctl@ceph.osd.10.smithi102.stdout:Sep 10 10:07:03 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-10[59874]: 2024-09-10T10:07:03.625+0000 7f40f3335640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.5:6822 osd.6 since back 2024-09-10T10:06:38.184236+0000 front 2024-09-10T10:06:38.184302+0000 (oldest deadline 2024-09-10T10:07:02.884090+0000) 2024-09-10T10:07:04.557 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 10 10:07:04 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-9[65288]: 2024-09-10T10:07:04.157+0000 7f84af200640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-10T10:05:47.346189+0000 front 2024-09-10T10:05:47.346220+0000 (oldest deadline 2024-09-10T10:06:12.646451+0000) 2024-09-10T10:07:04.557 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 10 10:07:04 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-9[65288]: 2024-09-10T10:07:04.157+0000 7f84af200640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-09-10T10:05:57.348185+0000 front 2024-09-10T10:05:57.348335+0000 (oldest deadline 2024-09-10T10:06:23.248210+0000) 2024-09-10T10:07:04.557 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 10 10:07:04 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-9[65288]: 2024-09-10T10:07:04.157+0000 7f84af200640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-10T10:06:03.249484+0000 front 2024-09-10T10:06:03.249185+0000 (oldest deadline 2024-09-10T10:06:26.749283+0000) 2024-09-10T10:07:04.557 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 10 10:07:04 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-9[65288]: 2024-09-10T10:07:04.157+0000 7f84af200640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-10T10:06:12.650860+0000 front 2024-09-10T10:06:12.650900+0000 (oldest deadline 2024-09-10T10:06:38.551142+0000) 2024-09-10T10:07:04.557 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 10 10:07:04 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-9[65288]: 2024-09-10T10:07:04.157+0000 7f84af200640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.102:6812 osd.4 since back 2024-09-10T10:06:18.551683+0000 front 2024-09-10T10:06:18.551825+0000 (oldest deadline 2024-09-10T10:06:44.451846+0000) 2024-09-10T10:07:04.557 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 10 10:07:04 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-9[65288]: 2024-09-10T10:07:04.157+0000 7f84af200640 -1 osd.9 100 heartbeat_check: no reply from 172.21.15.159:6812 osd.5 since back 2024-09-10T10:06:26.153334+0000 front 2024-09-10T10:06:26.153303+0000 (oldest deadline 2024-09-10T10:06:57.354360+0000) 2024-09-10T10:07:04.850 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 10 10:07:04 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-11[60437]: 2024-09-10T10:07:04.463+0000 7fa78e4d3640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-10T10:05:49.428950+0000 front 2024-09-10T10:05:49.429031+0000 (oldest deadline 2024-09-10T10:06:11.128800+0000) 2024-09-10T10:07:04.851 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 10 10:07:04 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-11[60437]: 2024-09-10T10:07:04.463+0000 7fa78e4d3640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-10T10:06:03.430910+0000 front 2024-09-10T10:06:03.430889+0000 (oldest deadline 2024-09-10T10:06:29.331091+0000) 2024-09-10T10:07:04.851 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 10 10:07:04 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-11[60437]: 2024-09-10T10:07:04.463+0000 7fa78e4d3640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-10T10:06:13.432384+0000 front 2024-09-10T10:06:13.432489+0000 (oldest deadline 2024-09-10T10:06:35.732466+0000) 2024-09-10T10:07:04.851 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 10 10:07:04 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-11[60437]: 2024-09-10T10:07:04.463+0000 7fa78e4d3640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.102:6812 osd.4 since back 2024-09-10T10:06:22.734557+0000 front 2024-09-10T10:06:22.734365+0000 (oldest deadline 2024-09-10T10:06:44.434370+0000) 2024-09-10T10:07:04.851 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 10 10:07:04 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-11[60437]: 2024-09-10T10:07:04.463+0000 7fa78e4d3640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.159:6812 osd.5 since back 2024-09-10T10:06:29.635809+0000 front 2024-09-10T10:06:29.635901+0000 (oldest deadline 2024-09-10T10:06:53.735941+0000) 2024-09-10T10:07:04.851 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 10 10:07:04 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-11[60437]: 2024-09-10T10:07:04.463+0000 7fa78e4d3640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.5:6822 osd.6 since back 2024-09-10T10:06:38.937377+0000 front 2024-09-10T10:06:38.937447+0000 (oldest deadline 2024-09-10T10:07:01.837229+0000) 2024-09-10T10:07:05.135 INFO:journalctl@ceph.osd.10.smithi102.stdout:Sep 10 10:07:04 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-10[59874]: 2024-09-10T10:07:04.633+0000 7f40f3335640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-10T10:05:49.575097+0000 front 2024-09-10T10:05:49.575141+0000 (oldest deadline 2024-09-10T10:06:13.075035+0000) 2024-09-10T10:07:05.135 INFO:journalctl@ceph.osd.10.smithi102.stdout:Sep 10 10:07:04 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-10[59874]: 2024-09-10T10:07:04.633+0000 7f40f3335640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-09-10T10:05:54.176094+0000 front 2024-09-10T10:05:54.176115+0000 (oldest deadline 2024-09-10T10:06:19.376783+0000) 2024-09-10T10:07:05.135 INFO:journalctl@ceph.osd.10.smithi102.stdout:Sep 10 10:07:04 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-10[59874]: 2024-09-10T10:07:04.633+0000 7f40f3335640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-10T10:06:03.378660+0000 front 2024-09-10T10:06:03.378719+0000 (oldest deadline 2024-09-10T10:06:32.178984+0000) 2024-09-10T10:07:05.135 INFO:journalctl@ceph.osd.10.smithi102.stdout:Sep 10 10:07:04 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-10[59874]: 2024-09-10T10:07:04.633+0000 7f40f3335640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-10T10:06:12.179545+0000 front 2024-09-10T10:06:12.179581+0000 (oldest deadline 2024-09-10T10:06:36.879462+0000) 2024-09-10T10:07:05.136 INFO:journalctl@ceph.osd.10.smithi102.stdout:Sep 10 10:07:04 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-10[59874]: 2024-09-10T10:07:04.633+0000 7f40f3335640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.102:6812 osd.4 since back 2024-09-10T10:06:22.581662+0000 front 2024-09-10T10:06:22.581640+0000 (oldest deadline 2024-09-10T10:06:44.281620+0000) 2024-09-10T10:07:05.136 INFO:journalctl@ceph.osd.10.smithi102.stdout:Sep 10 10:07:04 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-10[59874]: 2024-09-10T10:07:04.633+0000 7f40f3335640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.159:6812 osd.5 since back 2024-09-10T10:06:28.382594+0000 front 2024-09-10T10:06:28.382505+0000 (oldest deadline 2024-09-10T10:06:53.682352+0000) 2024-09-10T10:07:05.136 INFO:journalctl@ceph.osd.10.smithi102.stdout:Sep 10 10:07:04 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-10[59874]: 2024-09-10T10:07:04.633+0000 7f40f3335640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.5:6822 osd.6 since back 2024-09-10T10:06:38.184236+0000 front 2024-09-10T10:06:38.184302+0000 (oldest deadline 2024-09-10T10:07:02.884090+0000) 2024-09-10T10:07:05.667 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 10 10:07:05 smithi005 podman[144326]: 2024-09-10 10:07:05.344329828 +0000 UTC m=+5.870930537 container died fd7e7a76bf30f683eaa8857577127ebb20156033da30f283668cc7a3b91f6eb2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-9, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-10T10:07:05.850 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 10 10:07:05 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-11[60437]: 2024-09-10T10:07:05.443+0000 7fa78e4d3640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-10T10:05:49.428950+0000 front 2024-09-10T10:05:49.429031+0000 (oldest deadline 2024-09-10T10:06:11.128800+0000) 2024-09-10T10:07:05.850 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 10 10:07:05 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-11[60437]: 2024-09-10T10:07:05.443+0000 7fa78e4d3640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-10T10:06:03.430910+0000 front 2024-09-10T10:06:03.430889+0000 (oldest deadline 2024-09-10T10:06:29.331091+0000) 2024-09-10T10:07:05.850 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 10 10:07:05 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-11[60437]: 2024-09-10T10:07:05.443+0000 7fa78e4d3640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-10T10:06:13.432384+0000 front 2024-09-10T10:06:13.432489+0000 (oldest deadline 2024-09-10T10:06:35.732466+0000) 2024-09-10T10:07:05.851 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 10 10:07:05 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-11[60437]: 2024-09-10T10:07:05.443+0000 7fa78e4d3640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.102:6812 osd.4 since back 2024-09-10T10:06:22.734557+0000 front 2024-09-10T10:06:22.734365+0000 (oldest deadline 2024-09-10T10:06:44.434370+0000) 2024-09-10T10:07:05.851 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 10 10:07:05 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-11[60437]: 2024-09-10T10:07:05.443+0000 7fa78e4d3640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.159:6812 osd.5 since back 2024-09-10T10:06:29.635809+0000 front 2024-09-10T10:06:29.635901+0000 (oldest deadline 2024-09-10T10:06:53.735941+0000) 2024-09-10T10:07:05.851 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 10 10:07:05 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-11[60437]: 2024-09-10T10:07:05.443+0000 7fa78e4d3640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.5:6822 osd.6 since back 2024-09-10T10:06:38.937377+0000 front 2024-09-10T10:06:38.937447+0000 (oldest deadline 2024-09-10T10:07:01.837229+0000) 2024-09-10T10:07:05.884 INFO:journalctl@ceph.osd.10.smithi102.stdout:Sep 10 10:07:05 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-10[59874]: 2024-09-10T10:07:05.602+0000 7f40f3335640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-10T10:05:49.575097+0000 front 2024-09-10T10:05:49.575141+0000 (oldest deadline 2024-09-10T10:06:13.075035+0000) 2024-09-10T10:07:05.885 INFO:journalctl@ceph.osd.10.smithi102.stdout:Sep 10 10:07:05 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-10[59874]: 2024-09-10T10:07:05.602+0000 7f40f3335640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-09-10T10:05:54.176094+0000 front 2024-09-10T10:05:54.176115+0000 (oldest deadline 2024-09-10T10:06:19.376783+0000) 2024-09-10T10:07:05.885 INFO:journalctl@ceph.osd.10.smithi102.stdout:Sep 10 10:07:05 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-10[59874]: 2024-09-10T10:07:05.602+0000 7f40f3335640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-10T10:06:03.378660+0000 front 2024-09-10T10:06:03.378719+0000 (oldest deadline 2024-09-10T10:06:32.178984+0000) 2024-09-10T10:07:05.885 INFO:journalctl@ceph.osd.10.smithi102.stdout:Sep 10 10:07:05 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-10[59874]: 2024-09-10T10:07:05.602+0000 7f40f3335640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-10T10:06:12.179545+0000 front 2024-09-10T10:06:12.179581+0000 (oldest deadline 2024-09-10T10:06:36.879462+0000) 2024-09-10T10:07:05.885 INFO:journalctl@ceph.osd.10.smithi102.stdout:Sep 10 10:07:05 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-10[59874]: 2024-09-10T10:07:05.602+0000 7f40f3335640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.102:6812 osd.4 since back 2024-09-10T10:06:22.581662+0000 front 2024-09-10T10:06:22.581640+0000 (oldest deadline 2024-09-10T10:06:44.281620+0000) 2024-09-10T10:07:05.885 INFO:journalctl@ceph.osd.10.smithi102.stdout:Sep 10 10:07:05 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-10[59874]: 2024-09-10T10:07:05.602+0000 7f40f3335640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.159:6812 osd.5 since back 2024-09-10T10:06:28.382594+0000 front 2024-09-10T10:06:28.382505+0000 (oldest deadline 2024-09-10T10:06:53.682352+0000) 2024-09-10T10:07:05.885 INFO:journalctl@ceph.osd.10.smithi102.stdout:Sep 10 10:07:05 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-10[59874]: 2024-09-10T10:07:05.602+0000 7f40f3335640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.5:6822 osd.6 since back 2024-09-10T10:06:38.184236+0000 front 2024-09-10T10:06:38.184302+0000 (oldest deadline 2024-09-10T10:07:02.884090+0000) 2024-09-10T10:07:05.984 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 10 10:07:05 smithi005 podman[144326]: 2024-09-10 10:07:05.790770804 +0000 UTC m=+6.317371511 container cleanup fd7e7a76bf30f683eaa8857577127ebb20156033da30f283668cc7a3b91f6eb2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-9, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, ceph=True, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-10T10:07:05.984 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 10 10:07:05 smithi005 bash[144326]: ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-9 2024-09-10T10:07:06.306 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 10 10:07:05 smithi005 podman[144343]: 2024-09-10 10:07:05.983430336 +0000 UTC m=+0.628918946 container remove fd7e7a76bf30f683eaa8857577127ebb20156033da30f283668cc7a3b91f6eb2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-9, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=) 2024-09-10T10:07:06.812 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 10 10:07:06 smithi005 podman[144425]: 2024-09-10 10:07:06.462513955 +0000 UTC m=+0.027542176 image pull 41d67112835771c3efca60c6b8fe297c7ddaecdc26cea26fd28aa18636e49005 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 2024-09-10T10:07:06.812 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 10 10:07:06 smithi005 podman[144425]: 2024-09-10 10:07:06.576142637 +0000 UTC m=+0.141170833 container create 8553c2a1ce6cc7bf71c6e233302c2f85bc86ffd1b904c883448747a9cd56d28d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-9-deactivate, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T10:07:06.812 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 10 10:07:06 smithi005 podman[144425]: 2024-09-10 10:07:06.81163145 +0000 UTC m=+0.376659672 container init 8553c2a1ce6cc7bf71c6e233302c2f85bc86ffd1b904c883448747a9cd56d28d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-9-deactivate, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True) 2024-09-10T10:07:06.850 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 10 10:07:06 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-11[60437]: 2024-09-10T10:07:06.411+0000 7fa78e4d3640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-10T10:05:49.428950+0000 front 2024-09-10T10:05:49.429031+0000 (oldest deadline 2024-09-10T10:06:11.128800+0000) 2024-09-10T10:07:06.850 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 10 10:07:06 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-11[60437]: 2024-09-10T10:07:06.411+0000 7fa78e4d3640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-10T10:06:03.430910+0000 front 2024-09-10T10:06:03.430889+0000 (oldest deadline 2024-09-10T10:06:29.331091+0000) 2024-09-10T10:07:06.850 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 10 10:07:06 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-11[60437]: 2024-09-10T10:07:06.411+0000 7fa78e4d3640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-10T10:06:13.432384+0000 front 2024-09-10T10:06:13.432489+0000 (oldest deadline 2024-09-10T10:06:35.732466+0000) 2024-09-10T10:07:06.851 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 10 10:07:06 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-11[60437]: 2024-09-10T10:07:06.411+0000 7fa78e4d3640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.102:6812 osd.4 since back 2024-09-10T10:06:22.734557+0000 front 2024-09-10T10:06:22.734365+0000 (oldest deadline 2024-09-10T10:06:44.434370+0000) 2024-09-10T10:07:06.851 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 10 10:07:06 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-11[60437]: 2024-09-10T10:07:06.411+0000 7fa78e4d3640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.159:6812 osd.5 since back 2024-09-10T10:06:29.635809+0000 front 2024-09-10T10:06:29.635901+0000 (oldest deadline 2024-09-10T10:06:53.735941+0000) 2024-09-10T10:07:06.851 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 10 10:07:06 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-11[60437]: 2024-09-10T10:07:06.411+0000 7fa78e4d3640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.5:6822 osd.6 since back 2024-09-10T10:06:38.937377+0000 front 2024-09-10T10:06:38.937447+0000 (oldest deadline 2024-09-10T10:07:01.837229+0000) 2024-09-10T10:07:06.885 INFO:journalctl@ceph.osd.10.smithi102.stdout:Sep 10 10:07:06 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-10[59874]: 2024-09-10T10:07:06.607+0000 7f40f3335640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-10T10:05:49.575097+0000 front 2024-09-10T10:05:49.575141+0000 (oldest deadline 2024-09-10T10:06:13.075035+0000) 2024-09-10T10:07:06.885 INFO:journalctl@ceph.osd.10.smithi102.stdout:Sep 10 10:07:06 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-10[59874]: 2024-09-10T10:07:06.607+0000 7f40f3335640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-09-10T10:05:54.176094+0000 front 2024-09-10T10:05:54.176115+0000 (oldest deadline 2024-09-10T10:06:19.376783+0000) 2024-09-10T10:07:06.885 INFO:journalctl@ceph.osd.10.smithi102.stdout:Sep 10 10:07:06 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-10[59874]: 2024-09-10T10:07:06.607+0000 7f40f3335640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-10T10:06:03.378660+0000 front 2024-09-10T10:06:03.378719+0000 (oldest deadline 2024-09-10T10:06:32.178984+0000) 2024-09-10T10:07:06.885 INFO:journalctl@ceph.osd.10.smithi102.stdout:Sep 10 10:07:06 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-10[59874]: 2024-09-10T10:07:06.607+0000 7f40f3335640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-10T10:06:12.179545+0000 front 2024-09-10T10:06:12.179581+0000 (oldest deadline 2024-09-10T10:06:36.879462+0000) 2024-09-10T10:07:06.885 INFO:journalctl@ceph.osd.10.smithi102.stdout:Sep 10 10:07:06 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-10[59874]: 2024-09-10T10:07:06.607+0000 7f40f3335640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.102:6812 osd.4 since back 2024-09-10T10:06:22.581662+0000 front 2024-09-10T10:06:22.581640+0000 (oldest deadline 2024-09-10T10:06:44.281620+0000) 2024-09-10T10:07:06.885 INFO:journalctl@ceph.osd.10.smithi102.stdout:Sep 10 10:07:06 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-10[59874]: 2024-09-10T10:07:06.607+0000 7f40f3335640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.159:6812 osd.5 since back 2024-09-10T10:06:28.382594+0000 front 2024-09-10T10:06:28.382505+0000 (oldest deadline 2024-09-10T10:06:53.682352+0000) 2024-09-10T10:07:06.885 INFO:journalctl@ceph.osd.10.smithi102.stdout:Sep 10 10:07:06 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-10[59874]: 2024-09-10T10:07:06.607+0000 7f40f3335640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.5:6822 osd.6 since back 2024-09-10T10:06:38.184236+0000 front 2024-09-10T10:06:38.184302+0000 (oldest deadline 2024-09-10T10:07:02.884090+0000) 2024-09-10T10:07:07.090 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 10 10:07:06 smithi005 podman[144425]: 2024-09-10 10:07:06.818724939 +0000 UTC m=+0.383753136 container start 8553c2a1ce6cc7bf71c6e233302c2f85bc86ffd1b904c883448747a9cd56d28d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-9-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-09-10T10:07:07.090 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 10 10:07:06 smithi005 podman[144425]: 2024-09-10 10:07:06.911392267 +0000 UTC m=+0.476420465 container attach 8553c2a1ce6cc7bf71c6e233302c2f85bc86ffd1b904c883448747a9cd56d28d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-9-deactivate, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-09-10T10:07:07.423 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 10 10:07:07 smithi005 podman[144425]: 2024-09-10 10:07:07.090143971 +0000 UTC m=+0.655172185 container died 8553c2a1ce6cc7bf71c6e233302c2f85bc86ffd1b904c883448747a9cd56d28d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-9-deactivate, io.buildah.version=1.37.2, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-10T10:07:07.755 DEBUG:teuthology.orchestra.run.smithi005:> sudo pkill -f 'journalctl -f -n 0 -u ceph-bade4b50-6f56-11ef-bcea-c7b262605968@osd.9.service' 2024-09-10T10:07:07.830 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T10:07:07.830 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-09-10T10:07:07.830 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-09-10T10:07:07.830 DEBUG:teuthology.orchestra.run.smithi102:> sudo systemctl stop ceph-bade4b50-6f56-11ef-bcea-c7b262605968@osd.10 2024-09-10T10:07:07.850 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 10 10:07:07 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-11[60437]: 2024-09-10T10:07:07.448+0000 7fa78e4d3640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-10T10:05:49.428950+0000 front 2024-09-10T10:05:49.429031+0000 (oldest deadline 2024-09-10T10:06:11.128800+0000) 2024-09-10T10:07:07.851 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 10 10:07:07 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-11[60437]: 2024-09-10T10:07:07.448+0000 7fa78e4d3640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-10T10:06:03.430910+0000 front 2024-09-10T10:06:03.430889+0000 (oldest deadline 2024-09-10T10:06:29.331091+0000) 2024-09-10T10:07:07.851 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 10 10:07:07 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-11[60437]: 2024-09-10T10:07:07.448+0000 7fa78e4d3640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-10T10:06:13.432384+0000 front 2024-09-10T10:06:13.432489+0000 (oldest deadline 2024-09-10T10:06:35.732466+0000) 2024-09-10T10:07:07.851 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 10 10:07:07 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-11[60437]: 2024-09-10T10:07:07.448+0000 7fa78e4d3640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.102:6812 osd.4 since back 2024-09-10T10:06:22.734557+0000 front 2024-09-10T10:06:22.734365+0000 (oldest deadline 2024-09-10T10:06:44.434370+0000) 2024-09-10T10:07:07.851 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 10 10:07:07 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-11[60437]: 2024-09-10T10:07:07.448+0000 7fa78e4d3640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.159:6812 osd.5 since back 2024-09-10T10:06:29.635809+0000 front 2024-09-10T10:06:29.635901+0000 (oldest deadline 2024-09-10T10:06:53.735941+0000) 2024-09-10T10:07:07.851 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 10 10:07:07 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-11[60437]: 2024-09-10T10:07:07.448+0000 7fa78e4d3640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.5:6822 osd.6 since back 2024-09-10T10:06:38.937377+0000 front 2024-09-10T10:06:38.937447+0000 (oldest deadline 2024-09-10T10:07:01.837229+0000) 2024-09-10T10:07:07.878 INFO:journalctl@ceph.osd.10.smithi102.stdout:Sep 10 10:07:07 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-10[59874]: 2024-09-10T10:07:07.625+0000 7f40f3335640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-10T10:05:49.575097+0000 front 2024-09-10T10:05:49.575141+0000 (oldest deadline 2024-09-10T10:06:13.075035+0000) 2024-09-10T10:07:07.879 INFO:journalctl@ceph.osd.10.smithi102.stdout:Sep 10 10:07:07 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-10[59874]: 2024-09-10T10:07:07.625+0000 7f40f3335640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-09-10T10:05:54.176094+0000 front 2024-09-10T10:05:54.176115+0000 (oldest deadline 2024-09-10T10:06:19.376783+0000) 2024-09-10T10:07:07.879 INFO:journalctl@ceph.osd.10.smithi102.stdout:Sep 10 10:07:07 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-10[59874]: 2024-09-10T10:07:07.625+0000 7f40f3335640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-10T10:06:03.378660+0000 front 2024-09-10T10:06:03.378719+0000 (oldest deadline 2024-09-10T10:06:32.178984+0000) 2024-09-10T10:07:07.879 INFO:journalctl@ceph.osd.10.smithi102.stdout:Sep 10 10:07:07 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-10[59874]: 2024-09-10T10:07:07.625+0000 7f40f3335640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-10T10:06:12.179545+0000 front 2024-09-10T10:06:12.179581+0000 (oldest deadline 2024-09-10T10:06:36.879462+0000) 2024-09-10T10:07:07.879 INFO:journalctl@ceph.osd.10.smithi102.stdout:Sep 10 10:07:07 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-10[59874]: 2024-09-10T10:07:07.625+0000 7f40f3335640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.102:6812 osd.4 since back 2024-09-10T10:06:22.581662+0000 front 2024-09-10T10:06:22.581640+0000 (oldest deadline 2024-09-10T10:06:44.281620+0000) 2024-09-10T10:07:07.879 INFO:journalctl@ceph.osd.10.smithi102.stdout:Sep 10 10:07:07 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-10[59874]: 2024-09-10T10:07:07.625+0000 7f40f3335640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.159:6812 osd.5 since back 2024-09-10T10:06:28.382594+0000 front 2024-09-10T10:06:28.382505+0000 (oldest deadline 2024-09-10T10:06:53.682352+0000) 2024-09-10T10:07:07.879 INFO:journalctl@ceph.osd.10.smithi102.stdout:Sep 10 10:07:07 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-10[59874]: 2024-09-10T10:07:07.625+0000 7f40f3335640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.5:6822 osd.6 since back 2024-09-10T10:06:38.184236+0000 front 2024-09-10T10:06:38.184302+0000 (oldest deadline 2024-09-10T10:07:02.884090+0000) 2024-09-10T10:07:08.134 INFO:journalctl@ceph.osd.10.smithi102.stdout:Sep 10 10:07:07 smithi102 systemd[1]: Stopping Ceph osd.10 for bade4b50-6f56-11ef-bcea-c7b262605968... 2024-09-10T10:07:08.850 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 10 10:07:08 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-11[60437]: 2024-09-10T10:07:08.449+0000 7fa78e4d3640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-10T10:05:49.428950+0000 front 2024-09-10T10:05:49.429031+0000 (oldest deadline 2024-09-10T10:06:11.128800+0000) 2024-09-10T10:07:08.850 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 10 10:07:08 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-11[60437]: 2024-09-10T10:07:08.449+0000 7fa78e4d3640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-10T10:06:03.430910+0000 front 2024-09-10T10:06:03.430889+0000 (oldest deadline 2024-09-10T10:06:29.331091+0000) 2024-09-10T10:07:08.851 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 10 10:07:08 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-11[60437]: 2024-09-10T10:07:08.449+0000 7fa78e4d3640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-10T10:06:13.432384+0000 front 2024-09-10T10:06:13.432489+0000 (oldest deadline 2024-09-10T10:06:35.732466+0000) 2024-09-10T10:07:08.851 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 10 10:07:08 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-11[60437]: 2024-09-10T10:07:08.449+0000 7fa78e4d3640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.102:6812 osd.4 since back 2024-09-10T10:06:22.734557+0000 front 2024-09-10T10:06:22.734365+0000 (oldest deadline 2024-09-10T10:06:44.434370+0000) 2024-09-10T10:07:08.851 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 10 10:07:08 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-11[60437]: 2024-09-10T10:07:08.449+0000 7fa78e4d3640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.159:6812 osd.5 since back 2024-09-10T10:06:29.635809+0000 front 2024-09-10T10:06:29.635901+0000 (oldest deadline 2024-09-10T10:06:53.735941+0000) 2024-09-10T10:07:08.851 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 10 10:07:08 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-11[60437]: 2024-09-10T10:07:08.449+0000 7fa78e4d3640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.5:6822 osd.6 since back 2024-09-10T10:06:38.937377+0000 front 2024-09-10T10:06:38.937447+0000 (oldest deadline 2024-09-10T10:07:01.837229+0000) 2024-09-10T10:07:08.885 INFO:journalctl@ceph.osd.10.smithi102.stdout:Sep 10 10:07:08 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-10[59874]: 2024-09-10T10:07:08.431+0000 7f40f6d1f640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T10:07:08.885 INFO:journalctl@ceph.osd.10.smithi102.stdout:Sep 10 10:07:08 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-10[59874]: 2024-09-10T10:07:08.431+0000 7f40f6d1f640 -1 osd.10 100 *** Got signal Terminated *** 2024-09-10T10:07:08.885 INFO:journalctl@ceph.osd.10.smithi102.stdout:Sep 10 10:07:08 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-10[59874]: 2024-09-10T10:07:08.431+0000 7f40f6d1f640 -1 osd.10 100 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-10T10:07:08.885 INFO:journalctl@ceph.osd.10.smithi102.stdout:Sep 10 10:07:08 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-10[59874]: 2024-09-10T10:07:08.597+0000 7f40f3335640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-10T10:05:49.575097+0000 front 2024-09-10T10:05:49.575141+0000 (oldest deadline 2024-09-10T10:06:13.075035+0000) 2024-09-10T10:07:08.885 INFO:journalctl@ceph.osd.10.smithi102.stdout:Sep 10 10:07:08 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-10[59874]: 2024-09-10T10:07:08.597+0000 7f40f3335640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-09-10T10:05:54.176094+0000 front 2024-09-10T10:05:54.176115+0000 (oldest deadline 2024-09-10T10:06:19.376783+0000) 2024-09-10T10:07:08.885 INFO:journalctl@ceph.osd.10.smithi102.stdout:Sep 10 10:07:08 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-10[59874]: 2024-09-10T10:07:08.597+0000 7f40f3335640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-10T10:06:03.378660+0000 front 2024-09-10T10:06:03.378719+0000 (oldest deadline 2024-09-10T10:06:32.178984+0000) 2024-09-10T10:07:08.885 INFO:journalctl@ceph.osd.10.smithi102.stdout:Sep 10 10:07:08 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-10[59874]: 2024-09-10T10:07:08.597+0000 7f40f3335640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-10T10:06:12.179545+0000 front 2024-09-10T10:06:12.179581+0000 (oldest deadline 2024-09-10T10:06:36.879462+0000) 2024-09-10T10:07:08.885 INFO:journalctl@ceph.osd.10.smithi102.stdout:Sep 10 10:07:08 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-10[59874]: 2024-09-10T10:07:08.597+0000 7f40f3335640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.102:6812 osd.4 since back 2024-09-10T10:06:22.581662+0000 front 2024-09-10T10:06:22.581640+0000 (oldest deadline 2024-09-10T10:06:44.281620+0000) 2024-09-10T10:07:08.885 INFO:journalctl@ceph.osd.10.smithi102.stdout:Sep 10 10:07:08 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-10[59874]: 2024-09-10T10:07:08.597+0000 7f40f3335640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.159:6812 osd.5 since back 2024-09-10T10:06:28.382594+0000 front 2024-09-10T10:06:28.382505+0000 (oldest deadline 2024-09-10T10:06:53.682352+0000) 2024-09-10T10:07:08.885 INFO:journalctl@ceph.osd.10.smithi102.stdout:Sep 10 10:07:08 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-10[59874]: 2024-09-10T10:07:08.597+0000 7f40f3335640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.5:6822 osd.6 since back 2024-09-10T10:06:38.184236+0000 front 2024-09-10T10:06:38.184302+0000 (oldest deadline 2024-09-10T10:07:02.884090+0000) 2024-09-10T10:07:09.850 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 10 10:07:09 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-11[60437]: 2024-09-10T10:07:09.459+0000 7fa78e4d3640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-10T10:05:49.428950+0000 front 2024-09-10T10:05:49.429031+0000 (oldest deadline 2024-09-10T10:06:11.128800+0000) 2024-09-10T10:07:09.851 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 10 10:07:09 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-11[60437]: 2024-09-10T10:07:09.459+0000 7fa78e4d3640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-10T10:06:03.430910+0000 front 2024-09-10T10:06:03.430889+0000 (oldest deadline 2024-09-10T10:06:29.331091+0000) 2024-09-10T10:07:09.851 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 10 10:07:09 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-11[60437]: 2024-09-10T10:07:09.459+0000 7fa78e4d3640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-10T10:06:13.432384+0000 front 2024-09-10T10:06:13.432489+0000 (oldest deadline 2024-09-10T10:06:35.732466+0000) 2024-09-10T10:07:09.851 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 10 10:07:09 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-11[60437]: 2024-09-10T10:07:09.459+0000 7fa78e4d3640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.102:6812 osd.4 since back 2024-09-10T10:06:22.734557+0000 front 2024-09-10T10:06:22.734365+0000 (oldest deadline 2024-09-10T10:06:44.434370+0000) 2024-09-10T10:07:09.851 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 10 10:07:09 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-11[60437]: 2024-09-10T10:07:09.459+0000 7fa78e4d3640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.159:6812 osd.5 since back 2024-09-10T10:06:29.635809+0000 front 2024-09-10T10:06:29.635901+0000 (oldest deadline 2024-09-10T10:06:53.735941+0000) 2024-09-10T10:07:09.851 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 10 10:07:09 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-11[60437]: 2024-09-10T10:07:09.459+0000 7fa78e4d3640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.5:6822 osd.6 since back 2024-09-10T10:06:38.937377+0000 front 2024-09-10T10:06:38.937447+0000 (oldest deadline 2024-09-10T10:07:01.837229+0000) 2024-09-10T10:07:09.851 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 10 10:07:09 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-11[60437]: 2024-09-10T10:07:09.459+0000 7fa78e4d3640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.102:6820 osd.7 since back 2024-09-10T10:06:47.638708+0000 front 2024-09-10T10:06:47.638696+0000 (oldest deadline 2024-09-10T10:07:09.338624+0000) 2024-09-10T10:07:09.884 INFO:journalctl@ceph.osd.10.smithi102.stdout:Sep 10 10:07:09 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-10[59874]: 2024-09-10T10:07:09.556+0000 7f40f3335640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-10T10:05:49.575097+0000 front 2024-09-10T10:05:49.575141+0000 (oldest deadline 2024-09-10T10:06:13.075035+0000) 2024-09-10T10:07:09.884 INFO:journalctl@ceph.osd.10.smithi102.stdout:Sep 10 10:07:09 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-10[59874]: 2024-09-10T10:07:09.556+0000 7f40f3335640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-09-10T10:05:54.176094+0000 front 2024-09-10T10:05:54.176115+0000 (oldest deadline 2024-09-10T10:06:19.376783+0000) 2024-09-10T10:07:09.884 INFO:journalctl@ceph.osd.10.smithi102.stdout:Sep 10 10:07:09 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-10[59874]: 2024-09-10T10:07:09.556+0000 7f40f3335640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-10T10:06:03.378660+0000 front 2024-09-10T10:06:03.378719+0000 (oldest deadline 2024-09-10T10:06:32.178984+0000) 2024-09-10T10:07:09.885 INFO:journalctl@ceph.osd.10.smithi102.stdout:Sep 10 10:07:09 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-10[59874]: 2024-09-10T10:07:09.556+0000 7f40f3335640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-10T10:06:12.179545+0000 front 2024-09-10T10:06:12.179581+0000 (oldest deadline 2024-09-10T10:06:36.879462+0000) 2024-09-10T10:07:09.885 INFO:journalctl@ceph.osd.10.smithi102.stdout:Sep 10 10:07:09 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-10[59874]: 2024-09-10T10:07:09.556+0000 7f40f3335640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.102:6812 osd.4 since back 2024-09-10T10:06:22.581662+0000 front 2024-09-10T10:06:22.581640+0000 (oldest deadline 2024-09-10T10:06:44.281620+0000) 2024-09-10T10:07:09.885 INFO:journalctl@ceph.osd.10.smithi102.stdout:Sep 10 10:07:09 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-10[59874]: 2024-09-10T10:07:09.556+0000 7f40f3335640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.159:6812 osd.5 since back 2024-09-10T10:06:28.382594+0000 front 2024-09-10T10:06:28.382505+0000 (oldest deadline 2024-09-10T10:06:53.682352+0000) 2024-09-10T10:07:09.885 INFO:journalctl@ceph.osd.10.smithi102.stdout:Sep 10 10:07:09 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-10[59874]: 2024-09-10T10:07:09.556+0000 7f40f3335640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.5:6822 osd.6 since back 2024-09-10T10:06:38.184236+0000 front 2024-09-10T10:06:38.184302+0000 (oldest deadline 2024-09-10T10:07:02.884090+0000) 2024-09-10T10:07:10.850 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 10 10:07:10 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-11[60437]: 2024-09-10T10:07:10.488+0000 7fa78e4d3640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-10T10:05:49.428950+0000 front 2024-09-10T10:05:49.429031+0000 (oldest deadline 2024-09-10T10:06:11.128800+0000) 2024-09-10T10:07:10.851 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 10 10:07:10 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-11[60437]: 2024-09-10T10:07:10.488+0000 7fa78e4d3640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-10T10:06:03.430910+0000 front 2024-09-10T10:06:03.430889+0000 (oldest deadline 2024-09-10T10:06:29.331091+0000) 2024-09-10T10:07:10.851 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 10 10:07:10 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-11[60437]: 2024-09-10T10:07:10.488+0000 7fa78e4d3640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-10T10:06:13.432384+0000 front 2024-09-10T10:06:13.432489+0000 (oldest deadline 2024-09-10T10:06:35.732466+0000) 2024-09-10T10:07:10.851 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 10 10:07:10 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-11[60437]: 2024-09-10T10:07:10.488+0000 7fa78e4d3640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.102:6812 osd.4 since back 2024-09-10T10:06:22.734557+0000 front 2024-09-10T10:06:22.734365+0000 (oldest deadline 2024-09-10T10:06:44.434370+0000) 2024-09-10T10:07:10.851 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 10 10:07:10 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-11[60437]: 2024-09-10T10:07:10.488+0000 7fa78e4d3640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.159:6812 osd.5 since back 2024-09-10T10:06:29.635809+0000 front 2024-09-10T10:06:29.635901+0000 (oldest deadline 2024-09-10T10:06:53.735941+0000) 2024-09-10T10:07:10.851 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 10 10:07:10 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-11[60437]: 2024-09-10T10:07:10.488+0000 7fa78e4d3640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.5:6822 osd.6 since back 2024-09-10T10:06:38.937377+0000 front 2024-09-10T10:06:38.937447+0000 (oldest deadline 2024-09-10T10:07:01.837229+0000) 2024-09-10T10:07:10.851 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 10 10:07:10 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-11[60437]: 2024-09-10T10:07:10.488+0000 7fa78e4d3640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.102:6820 osd.7 since back 2024-09-10T10:06:47.638708+0000 front 2024-09-10T10:06:47.638696+0000 (oldest deadline 2024-09-10T10:07:09.338624+0000) 2024-09-10T10:07:10.884 INFO:journalctl@ceph.osd.10.smithi102.stdout:Sep 10 10:07:10 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-10[59874]: 2024-09-10T10:07:10.533+0000 7f40f3335640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-10T10:05:49.575097+0000 front 2024-09-10T10:05:49.575141+0000 (oldest deadline 2024-09-10T10:06:13.075035+0000) 2024-09-10T10:07:10.885 INFO:journalctl@ceph.osd.10.smithi102.stdout:Sep 10 10:07:10 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-10[59874]: 2024-09-10T10:07:10.533+0000 7f40f3335640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-09-10T10:05:54.176094+0000 front 2024-09-10T10:05:54.176115+0000 (oldest deadline 2024-09-10T10:06:19.376783+0000) 2024-09-10T10:07:10.885 INFO:journalctl@ceph.osd.10.smithi102.stdout:Sep 10 10:07:10 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-10[59874]: 2024-09-10T10:07:10.533+0000 7f40f3335640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-10T10:06:03.378660+0000 front 2024-09-10T10:06:03.378719+0000 (oldest deadline 2024-09-10T10:06:32.178984+0000) 2024-09-10T10:07:10.885 INFO:journalctl@ceph.osd.10.smithi102.stdout:Sep 10 10:07:10 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-10[59874]: 2024-09-10T10:07:10.533+0000 7f40f3335640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-10T10:06:12.179545+0000 front 2024-09-10T10:06:12.179581+0000 (oldest deadline 2024-09-10T10:06:36.879462+0000) 2024-09-10T10:07:10.885 INFO:journalctl@ceph.osd.10.smithi102.stdout:Sep 10 10:07:10 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-10[59874]: 2024-09-10T10:07:10.533+0000 7f40f3335640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.102:6812 osd.4 since back 2024-09-10T10:06:22.581662+0000 front 2024-09-10T10:06:22.581640+0000 (oldest deadline 2024-09-10T10:06:44.281620+0000) 2024-09-10T10:07:10.885 INFO:journalctl@ceph.osd.10.smithi102.stdout:Sep 10 10:07:10 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-10[59874]: 2024-09-10T10:07:10.533+0000 7f40f3335640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.159:6812 osd.5 since back 2024-09-10T10:06:28.382594+0000 front 2024-09-10T10:06:28.382505+0000 (oldest deadline 2024-09-10T10:06:53.682352+0000) 2024-09-10T10:07:10.885 INFO:journalctl@ceph.osd.10.smithi102.stdout:Sep 10 10:07:10 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-10[59874]: 2024-09-10T10:07:10.533+0000 7f40f3335640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.5:6822 osd.6 since back 2024-09-10T10:06:38.184236+0000 front 2024-09-10T10:06:38.184302+0000 (oldest deadline 2024-09-10T10:07:02.884090+0000) 2024-09-10T10:07:11.786 INFO:journalctl@ceph.osd.10.smithi102.stdout:Sep 10 10:07:11 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-10[59874]: 2024-09-10T10:07:11.530+0000 7f40f3335640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-10T10:05:49.575097+0000 front 2024-09-10T10:05:49.575141+0000 (oldest deadline 2024-09-10T10:06:13.075035+0000) 2024-09-10T10:07:11.786 INFO:journalctl@ceph.osd.10.smithi102.stdout:Sep 10 10:07:11 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-10[59874]: 2024-09-10T10:07:11.530+0000 7f40f3335640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-09-10T10:05:54.176094+0000 front 2024-09-10T10:05:54.176115+0000 (oldest deadline 2024-09-10T10:06:19.376783+0000) 2024-09-10T10:07:11.786 INFO:journalctl@ceph.osd.10.smithi102.stdout:Sep 10 10:07:11 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-10[59874]: 2024-09-10T10:07:11.530+0000 7f40f3335640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-10T10:06:03.378660+0000 front 2024-09-10T10:06:03.378719+0000 (oldest deadline 2024-09-10T10:06:32.178984+0000) 2024-09-10T10:07:11.786 INFO:journalctl@ceph.osd.10.smithi102.stdout:Sep 10 10:07:11 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-10[59874]: 2024-09-10T10:07:11.530+0000 7f40f3335640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-10T10:06:12.179545+0000 front 2024-09-10T10:06:12.179581+0000 (oldest deadline 2024-09-10T10:06:36.879462+0000) 2024-09-10T10:07:11.786 INFO:journalctl@ceph.osd.10.smithi102.stdout:Sep 10 10:07:11 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-10[59874]: 2024-09-10T10:07:11.530+0000 7f40f3335640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.102:6812 osd.4 since back 2024-09-10T10:06:22.581662+0000 front 2024-09-10T10:06:22.581640+0000 (oldest deadline 2024-09-10T10:06:44.281620+0000) 2024-09-10T10:07:11.787 INFO:journalctl@ceph.osd.10.smithi102.stdout:Sep 10 10:07:11 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-10[59874]: 2024-09-10T10:07:11.530+0000 7f40f3335640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.159:6812 osd.5 since back 2024-09-10T10:06:28.382594+0000 front 2024-09-10T10:06:28.382505+0000 (oldest deadline 2024-09-10T10:06:53.682352+0000) 2024-09-10T10:07:11.787 INFO:journalctl@ceph.osd.10.smithi102.stdout:Sep 10 10:07:11 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-10[59874]: 2024-09-10T10:07:11.530+0000 7f40f3335640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.5:6822 osd.6 since back 2024-09-10T10:06:38.184236+0000 front 2024-09-10T10:06:38.184302+0000 (oldest deadline 2024-09-10T10:07:02.884090+0000) 2024-09-10T10:07:11.850 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 10 10:07:11 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-11[60437]: 2024-09-10T10:07:11.464+0000 7fa78e4d3640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-10T10:05:49.428950+0000 front 2024-09-10T10:05:49.429031+0000 (oldest deadline 2024-09-10T10:06:11.128800+0000) 2024-09-10T10:07:11.850 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 10 10:07:11 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-11[60437]: 2024-09-10T10:07:11.464+0000 7fa78e4d3640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-10T10:06:03.430910+0000 front 2024-09-10T10:06:03.430889+0000 (oldest deadline 2024-09-10T10:06:29.331091+0000) 2024-09-10T10:07:11.851 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 10 10:07:11 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-11[60437]: 2024-09-10T10:07:11.464+0000 7fa78e4d3640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-10T10:06:13.432384+0000 front 2024-09-10T10:06:13.432489+0000 (oldest deadline 2024-09-10T10:06:35.732466+0000) 2024-09-10T10:07:11.851 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 10 10:07:11 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-11[60437]: 2024-09-10T10:07:11.464+0000 7fa78e4d3640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.102:6812 osd.4 since back 2024-09-10T10:06:22.734557+0000 front 2024-09-10T10:06:22.734365+0000 (oldest deadline 2024-09-10T10:06:44.434370+0000) 2024-09-10T10:07:11.851 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 10 10:07:11 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-11[60437]: 2024-09-10T10:07:11.464+0000 7fa78e4d3640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.159:6812 osd.5 since back 2024-09-10T10:06:29.635809+0000 front 2024-09-10T10:06:29.635901+0000 (oldest deadline 2024-09-10T10:06:53.735941+0000) 2024-09-10T10:07:11.851 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 10 10:07:11 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-11[60437]: 2024-09-10T10:07:11.464+0000 7fa78e4d3640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.5:6822 osd.6 since back 2024-09-10T10:06:38.937377+0000 front 2024-09-10T10:06:38.937447+0000 (oldest deadline 2024-09-10T10:07:01.837229+0000) 2024-09-10T10:07:11.851 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 10 10:07:11 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-11[60437]: 2024-09-10T10:07:11.464+0000 7fa78e4d3640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.102:6820 osd.7 since back 2024-09-10T10:06:47.638708+0000 front 2024-09-10T10:06:47.638696+0000 (oldest deadline 2024-09-10T10:07:09.338624+0000) 2024-09-10T10:07:12.850 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 10 10:07:12 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-11[60437]: 2024-09-10T10:07:12.430+0000 7fa78e4d3640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-10T10:05:49.428950+0000 front 2024-09-10T10:05:49.429031+0000 (oldest deadline 2024-09-10T10:06:11.128800+0000) 2024-09-10T10:07:12.850 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 10 10:07:12 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-11[60437]: 2024-09-10T10:07:12.430+0000 7fa78e4d3640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-10T10:06:03.430910+0000 front 2024-09-10T10:06:03.430889+0000 (oldest deadline 2024-09-10T10:06:29.331091+0000) 2024-09-10T10:07:12.851 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 10 10:07:12 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-11[60437]: 2024-09-10T10:07:12.430+0000 7fa78e4d3640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-10T10:06:13.432384+0000 front 2024-09-10T10:06:13.432489+0000 (oldest deadline 2024-09-10T10:06:35.732466+0000) 2024-09-10T10:07:12.851 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 10 10:07:12 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-11[60437]: 2024-09-10T10:07:12.430+0000 7fa78e4d3640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.102:6812 osd.4 since back 2024-09-10T10:06:22.734557+0000 front 2024-09-10T10:06:22.734365+0000 (oldest deadline 2024-09-10T10:06:44.434370+0000) 2024-09-10T10:07:12.851 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 10 10:07:12 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-11[60437]: 2024-09-10T10:07:12.430+0000 7fa78e4d3640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.159:6812 osd.5 since back 2024-09-10T10:06:29.635809+0000 front 2024-09-10T10:06:29.635901+0000 (oldest deadline 2024-09-10T10:06:53.735941+0000) 2024-09-10T10:07:12.851 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 10 10:07:12 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-11[60437]: 2024-09-10T10:07:12.430+0000 7fa78e4d3640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.5:6822 osd.6 since back 2024-09-10T10:06:38.937377+0000 front 2024-09-10T10:06:38.937447+0000 (oldest deadline 2024-09-10T10:07:01.837229+0000) 2024-09-10T10:07:12.851 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 10 10:07:12 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-11[60437]: 2024-09-10T10:07:12.430+0000 7fa78e4d3640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.102:6820 osd.7 since back 2024-09-10T10:06:47.638708+0000 front 2024-09-10T10:06:47.638696+0000 (oldest deadline 2024-09-10T10:07:09.338624+0000) 2024-09-10T10:07:12.885 INFO:journalctl@ceph.osd.10.smithi102.stdout:Sep 10 10:07:12 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-10[59874]: 2024-09-10T10:07:12.497+0000 7f40f3335640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-10T10:05:49.575097+0000 front 2024-09-10T10:05:49.575141+0000 (oldest deadline 2024-09-10T10:06:13.075035+0000) 2024-09-10T10:07:12.885 INFO:journalctl@ceph.osd.10.smithi102.stdout:Sep 10 10:07:12 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-10[59874]: 2024-09-10T10:07:12.497+0000 7f40f3335640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.102:6804 osd.1 since back 2024-09-10T10:05:54.176094+0000 front 2024-09-10T10:05:54.176115+0000 (oldest deadline 2024-09-10T10:06:19.376783+0000) 2024-09-10T10:07:12.885 INFO:journalctl@ceph.osd.10.smithi102.stdout:Sep 10 10:07:12 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-10[59874]: 2024-09-10T10:07:12.497+0000 7f40f3335640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-10T10:06:03.378660+0000 front 2024-09-10T10:06:03.378719+0000 (oldest deadline 2024-09-10T10:06:32.178984+0000) 2024-09-10T10:07:12.885 INFO:journalctl@ceph.osd.10.smithi102.stdout:Sep 10 10:07:12 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-10[59874]: 2024-09-10T10:07:12.497+0000 7f40f3335640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-10T10:06:12.179545+0000 front 2024-09-10T10:06:12.179581+0000 (oldest deadline 2024-09-10T10:06:36.879462+0000) 2024-09-10T10:07:12.885 INFO:journalctl@ceph.osd.10.smithi102.stdout:Sep 10 10:07:12 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-10[59874]: 2024-09-10T10:07:12.497+0000 7f40f3335640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.102:6812 osd.4 since back 2024-09-10T10:06:22.581662+0000 front 2024-09-10T10:06:22.581640+0000 (oldest deadline 2024-09-10T10:06:44.281620+0000) 2024-09-10T10:07:12.885 INFO:journalctl@ceph.osd.10.smithi102.stdout:Sep 10 10:07:12 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-10[59874]: 2024-09-10T10:07:12.497+0000 7f40f3335640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.159:6812 osd.5 since back 2024-09-10T10:06:28.382594+0000 front 2024-09-10T10:06:28.382505+0000 (oldest deadline 2024-09-10T10:06:53.682352+0000) 2024-09-10T10:07:12.885 INFO:journalctl@ceph.osd.10.smithi102.stdout:Sep 10 10:07:12 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-10[59874]: 2024-09-10T10:07:12.497+0000 7f40f3335640 -1 osd.10 100 heartbeat_check: no reply from 172.21.15.5:6822 osd.6 since back 2024-09-10T10:06:38.184236+0000 front 2024-09-10T10:06:38.184302+0000 (oldest deadline 2024-09-10T10:07:02.884090+0000) 2024-09-10T10:07:13.850 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 10 10:07:13 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-11[60437]: 2024-09-10T10:07:13.422+0000 7fa78e4d3640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-10T10:05:49.428950+0000 front 2024-09-10T10:05:49.429031+0000 (oldest deadline 2024-09-10T10:06:11.128800+0000) 2024-09-10T10:07:13.851 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 10 10:07:13 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-11[60437]: 2024-09-10T10:07:13.422+0000 7fa78e4d3640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-10T10:06:03.430910+0000 front 2024-09-10T10:06:03.430889+0000 (oldest deadline 2024-09-10T10:06:29.331091+0000) 2024-09-10T10:07:13.851 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 10 10:07:13 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-11[60437]: 2024-09-10T10:07:13.422+0000 7fa78e4d3640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-10T10:06:13.432384+0000 front 2024-09-10T10:06:13.432489+0000 (oldest deadline 2024-09-10T10:06:35.732466+0000) 2024-09-10T10:07:13.851 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 10 10:07:13 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-11[60437]: 2024-09-10T10:07:13.422+0000 7fa78e4d3640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.102:6812 osd.4 since back 2024-09-10T10:06:22.734557+0000 front 2024-09-10T10:06:22.734365+0000 (oldest deadline 2024-09-10T10:06:44.434370+0000) 2024-09-10T10:07:13.851 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 10 10:07:13 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-11[60437]: 2024-09-10T10:07:13.422+0000 7fa78e4d3640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.159:6812 osd.5 since back 2024-09-10T10:06:29.635809+0000 front 2024-09-10T10:06:29.635901+0000 (oldest deadline 2024-09-10T10:06:53.735941+0000) 2024-09-10T10:07:13.851 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 10 10:07:13 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-11[60437]: 2024-09-10T10:07:13.422+0000 7fa78e4d3640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.5:6822 osd.6 since back 2024-09-10T10:06:38.937377+0000 front 2024-09-10T10:06:38.937447+0000 (oldest deadline 2024-09-10T10:07:01.837229+0000) 2024-09-10T10:07:13.851 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 10 10:07:13 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-11[60437]: 2024-09-10T10:07:13.422+0000 7fa78e4d3640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.102:6820 osd.7 since back 2024-09-10T10:06:47.638708+0000 front 2024-09-10T10:06:47.638696+0000 (oldest deadline 2024-09-10T10:07:09.338624+0000) 2024-09-10T10:07:14.135 INFO:journalctl@ceph.osd.10.smithi102.stdout:Sep 10 10:07:13 smithi102 podman[76068]: 2024-09-10 10:07:13.864059855 +0000 UTC m=+5.843874581 container died ac91c3d5affc936dfe32c0aa8d9baa05981495812bd231ea9359995e8d254fc5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-10, ceph=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-09-10T10:07:14.580 INFO:journalctl@ceph.osd.10.smithi102.stdout:Sep 10 10:07:14 smithi102 podman[76068]: 2024-09-10 10:07:14.222728911 +0000 UTC m=+6.202543637 container cleanup ac91c3d5affc936dfe32c0aa8d9baa05981495812bd231ea9359995e8d254fc5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-10, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-10T10:07:14.580 INFO:journalctl@ceph.osd.10.smithi102.stdout:Sep 10 10:07:14 smithi102 bash[76068]: ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-10 2024-09-10T10:07:14.850 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 10 10:07:14 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-11[60437]: 2024-09-10T10:07:14.393+0000 7fa78e4d3640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-10T10:05:49.428950+0000 front 2024-09-10T10:05:49.429031+0000 (oldest deadline 2024-09-10T10:06:11.128800+0000) 2024-09-10T10:07:14.851 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 10 10:07:14 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-11[60437]: 2024-09-10T10:07:14.393+0000 7fa78e4d3640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-10T10:06:03.430910+0000 front 2024-09-10T10:06:03.430889+0000 (oldest deadline 2024-09-10T10:06:29.331091+0000) 2024-09-10T10:07:14.851 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 10 10:07:14 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-11[60437]: 2024-09-10T10:07:14.393+0000 7fa78e4d3640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-10T10:06:13.432384+0000 front 2024-09-10T10:06:13.432489+0000 (oldest deadline 2024-09-10T10:06:35.732466+0000) 2024-09-10T10:07:14.851 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 10 10:07:14 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-11[60437]: 2024-09-10T10:07:14.393+0000 7fa78e4d3640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.102:6812 osd.4 since back 2024-09-10T10:06:22.734557+0000 front 2024-09-10T10:06:22.734365+0000 (oldest deadline 2024-09-10T10:06:44.434370+0000) 2024-09-10T10:07:14.851 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 10 10:07:14 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-11[60437]: 2024-09-10T10:07:14.393+0000 7fa78e4d3640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.159:6812 osd.5 since back 2024-09-10T10:06:29.635809+0000 front 2024-09-10T10:06:29.635901+0000 (oldest deadline 2024-09-10T10:06:53.735941+0000) 2024-09-10T10:07:14.851 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 10 10:07:14 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-11[60437]: 2024-09-10T10:07:14.393+0000 7fa78e4d3640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.5:6822 osd.6 since back 2024-09-10T10:06:38.937377+0000 front 2024-09-10T10:06:38.937447+0000 (oldest deadline 2024-09-10T10:07:01.837229+0000) 2024-09-10T10:07:14.851 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 10 10:07:14 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-11[60437]: 2024-09-10T10:07:14.393+0000 7fa78e4d3640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.102:6820 osd.7 since back 2024-09-10T10:06:47.638708+0000 front 2024-09-10T10:06:47.638696+0000 (oldest deadline 2024-09-10T10:07:09.338624+0000) 2024-09-10T10:07:14.884 INFO:journalctl@ceph.osd.10.smithi102.stdout:Sep 10 10:07:14 smithi102 podman[76096]: 2024-09-10 10:07:14.579992047 +0000 UTC m=+0.704393987 container remove ac91c3d5affc936dfe32c0aa8d9baa05981495812bd231ea9359995e8d254fc5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-10, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-09-10T10:07:15.203 INFO:journalctl@ceph.osd.10.smithi102.stdout:Sep 10 10:07:14 smithi102 podman[76179]: 2024-09-10 10:07:14.796614441 +0000 UTC m=+0.015575847 image pull 41d67112835771c3efca60c6b8fe297c7ddaecdc26cea26fd28aa18636e49005 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 2024-09-10T10:07:15.203 INFO:journalctl@ceph.osd.10.smithi102.stdout:Sep 10 10:07:14 smithi102 podman[76179]: 2024-09-10 10:07:14.916275568 +0000 UTC m=+0.135236968 container create 92c217ffbfcc1223c294ee226cace6624053f33ab93a1c843190a75596a7d2b9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-10-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, CEPH_POINT_RELEASE=) 2024-09-10T10:07:15.203 INFO:journalctl@ceph.osd.10.smithi102.stdout:Sep 10 10:07:15 smithi102 podman[76179]: 2024-09-10 10:07:15.127120586 +0000 UTC m=+0.346081996 container init 92c217ffbfcc1223c294ee226cace6624053f33ab93a1c843190a75596a7d2b9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-10-deactivate, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.build-date=20240731, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b) 2024-09-10T10:07:15.203 INFO:journalctl@ceph.osd.10.smithi102.stdout:Sep 10 10:07:15 smithi102 podman[76179]: 2024-09-10 10:07:15.130494863 +0000 UTC m=+0.349456271 container start 92c217ffbfcc1223c294ee226cace6624053f33ab93a1c843190a75596a7d2b9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-10-deactivate, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2) 2024-09-10T10:07:15.203 INFO:journalctl@ceph.osd.10.smithi102.stdout:Sep 10 10:07:15 smithi102 podman[76179]: 2024-09-10 10:07:15.201897587 +0000 UTC m=+0.420858999 container attach 92c217ffbfcc1223c294ee226cace6624053f33ab93a1c843190a75596a7d2b9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-10-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731) 2024-09-10T10:07:15.600 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 10 10:07:15 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-11[60437]: 2024-09-10T10:07:15.346+0000 7fa78e4d3640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-10T10:05:49.428950+0000 front 2024-09-10T10:05:49.429031+0000 (oldest deadline 2024-09-10T10:06:11.128800+0000) 2024-09-10T10:07:15.600 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 10 10:07:15 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-11[60437]: 2024-09-10T10:07:15.346+0000 7fa78e4d3640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-10T10:06:03.430910+0000 front 2024-09-10T10:06:03.430889+0000 (oldest deadline 2024-09-10T10:06:29.331091+0000) 2024-09-10T10:07:15.601 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 10 10:07:15 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-11[60437]: 2024-09-10T10:07:15.346+0000 7fa78e4d3640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-10T10:06:13.432384+0000 front 2024-09-10T10:06:13.432489+0000 (oldest deadline 2024-09-10T10:06:35.732466+0000) 2024-09-10T10:07:15.601 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 10 10:07:15 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-11[60437]: 2024-09-10T10:07:15.346+0000 7fa78e4d3640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.102:6812 osd.4 since back 2024-09-10T10:06:22.734557+0000 front 2024-09-10T10:06:22.734365+0000 (oldest deadline 2024-09-10T10:06:44.434370+0000) 2024-09-10T10:07:15.601 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 10 10:07:15 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-11[60437]: 2024-09-10T10:07:15.346+0000 7fa78e4d3640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.159:6812 osd.5 since back 2024-09-10T10:06:29.635809+0000 front 2024-09-10T10:06:29.635901+0000 (oldest deadline 2024-09-10T10:06:53.735941+0000) 2024-09-10T10:07:15.601 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 10 10:07:15 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-11[60437]: 2024-09-10T10:07:15.346+0000 7fa78e4d3640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.5:6822 osd.6 since back 2024-09-10T10:06:38.937377+0000 front 2024-09-10T10:06:38.937447+0000 (oldest deadline 2024-09-10T10:07:01.837229+0000) 2024-09-10T10:07:15.601 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 10 10:07:15 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-11[60437]: 2024-09-10T10:07:15.346+0000 7fa78e4d3640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.102:6820 osd.7 since back 2024-09-10T10:06:47.638708+0000 front 2024-09-10T10:06:47.638696+0000 (oldest deadline 2024-09-10T10:07:09.338624+0000) 2024-09-10T10:07:15.621 INFO:journalctl@ceph.osd.10.smithi102.stdout:Sep 10 10:07:15 smithi102 podman[76179]: 2024-09-10 10:07:15.32013308 +0000 UTC m=+0.539094489 container died 92c217ffbfcc1223c294ee226cace6624053f33ab93a1c843190a75596a7d2b9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-10-deactivate, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-09-10T10:07:15.904 DEBUG:teuthology.orchestra.run.smithi102:> sudo pkill -f 'journalctl -f -n 0 -u ceph-bade4b50-6f56-11ef-bcea-c7b262605968@osd.10.service' 2024-09-10T10:07:15.945 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T10:07:15.945 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-09-10T10:07:15.945 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-09-10T10:07:15.945 DEBUG:teuthology.orchestra.run.smithi159:> sudo systemctl stop ceph-bade4b50-6f56-11ef-bcea-c7b262605968@osd.11 2024-09-10T10:07:16.274 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 10 10:07:16 smithi159 systemd[1]: Stopping Ceph osd.11 for bade4b50-6f56-11ef-bcea-c7b262605968... 2024-09-10T10:07:16.600 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 10 10:07:16 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-11[60437]: 2024-09-10T10:07:16.272+0000 7fa795ec5640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T10:07:16.601 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 10 10:07:16 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-11[60437]: 2024-09-10T10:07:16.272+0000 7fa795ec5640 -1 osd.11 100 *** Got signal Terminated *** 2024-09-10T10:07:16.601 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 10 10:07:16 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-11[60437]: 2024-09-10T10:07:16.272+0000 7fa795ec5640 -1 osd.11 100 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-10T10:07:16.601 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 10 10:07:16 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-11[60437]: 2024-09-10T10:07:16.338+0000 7fa78e4d3640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-10T10:05:49.428950+0000 front 2024-09-10T10:05:49.429031+0000 (oldest deadline 2024-09-10T10:06:11.128800+0000) 2024-09-10T10:07:16.601 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 10 10:07:16 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-11[60437]: 2024-09-10T10:07:16.338+0000 7fa78e4d3640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-10T10:06:03.430910+0000 front 2024-09-10T10:06:03.430889+0000 (oldest deadline 2024-09-10T10:06:29.331091+0000) 2024-09-10T10:07:16.601 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 10 10:07:16 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-11[60437]: 2024-09-10T10:07:16.338+0000 7fa78e4d3640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-10T10:06:13.432384+0000 front 2024-09-10T10:06:13.432489+0000 (oldest deadline 2024-09-10T10:06:35.732466+0000) 2024-09-10T10:07:16.601 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 10 10:07:16 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-11[60437]: 2024-09-10T10:07:16.338+0000 7fa78e4d3640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.102:6812 osd.4 since back 2024-09-10T10:06:22.734557+0000 front 2024-09-10T10:06:22.734365+0000 (oldest deadline 2024-09-10T10:06:44.434370+0000) 2024-09-10T10:07:16.601 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 10 10:07:16 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-11[60437]: 2024-09-10T10:07:16.338+0000 7fa78e4d3640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.159:6812 osd.5 since back 2024-09-10T10:06:29.635809+0000 front 2024-09-10T10:06:29.635901+0000 (oldest deadline 2024-09-10T10:06:53.735941+0000) 2024-09-10T10:07:16.601 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 10 10:07:16 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-11[60437]: 2024-09-10T10:07:16.338+0000 7fa78e4d3640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.5:6822 osd.6 since back 2024-09-10T10:06:38.937377+0000 front 2024-09-10T10:06:38.937447+0000 (oldest deadline 2024-09-10T10:07:01.837229+0000) 2024-09-10T10:07:16.601 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 10 10:07:16 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-11[60437]: 2024-09-10T10:07:16.338+0000 7fa78e4d3640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.102:6820 osd.7 since back 2024-09-10T10:06:47.638708+0000 front 2024-09-10T10:06:47.638696+0000 (oldest deadline 2024-09-10T10:07:09.338624+0000) 2024-09-10T10:07:17.600 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 10 10:07:17 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-11[60437]: 2024-09-10T10:07:17.305+0000 7fa78e4d3640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-10T10:05:49.428950+0000 front 2024-09-10T10:05:49.429031+0000 (oldest deadline 2024-09-10T10:06:11.128800+0000) 2024-09-10T10:07:17.601 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 10 10:07:17 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-11[60437]: 2024-09-10T10:07:17.305+0000 7fa78e4d3640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-10T10:06:03.430910+0000 front 2024-09-10T10:06:03.430889+0000 (oldest deadline 2024-09-10T10:06:29.331091+0000) 2024-09-10T10:07:17.601 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 10 10:07:17 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-11[60437]: 2024-09-10T10:07:17.305+0000 7fa78e4d3640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-10T10:06:13.432384+0000 front 2024-09-10T10:06:13.432489+0000 (oldest deadline 2024-09-10T10:06:35.732466+0000) 2024-09-10T10:07:17.601 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 10 10:07:17 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-11[60437]: 2024-09-10T10:07:17.305+0000 7fa78e4d3640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.102:6812 osd.4 since back 2024-09-10T10:06:22.734557+0000 front 2024-09-10T10:06:22.734365+0000 (oldest deadline 2024-09-10T10:06:44.434370+0000) 2024-09-10T10:07:17.601 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 10 10:07:17 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-11[60437]: 2024-09-10T10:07:17.305+0000 7fa78e4d3640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.159:6812 osd.5 since back 2024-09-10T10:06:29.635809+0000 front 2024-09-10T10:06:29.635901+0000 (oldest deadline 2024-09-10T10:06:53.735941+0000) 2024-09-10T10:07:17.601 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 10 10:07:17 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-11[60437]: 2024-09-10T10:07:17.305+0000 7fa78e4d3640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.5:6822 osd.6 since back 2024-09-10T10:06:38.937377+0000 front 2024-09-10T10:06:38.937447+0000 (oldest deadline 2024-09-10T10:07:01.837229+0000) 2024-09-10T10:07:17.601 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 10 10:07:17 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-11[60437]: 2024-09-10T10:07:17.305+0000 7fa78e4d3640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.102:6820 osd.7 since back 2024-09-10T10:06:47.638708+0000 front 2024-09-10T10:06:47.638696+0000 (oldest deadline 2024-09-10T10:07:09.338624+0000) 2024-09-10T10:07:18.600 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 10 10:07:18 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-11[60437]: 2024-09-10T10:07:18.271+0000 7fa78e4d3640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-10T10:05:49.428950+0000 front 2024-09-10T10:05:49.429031+0000 (oldest deadline 2024-09-10T10:06:11.128800+0000) 2024-09-10T10:07:18.601 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 10 10:07:18 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-11[60437]: 2024-09-10T10:07:18.271+0000 7fa78e4d3640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-10T10:06:03.430910+0000 front 2024-09-10T10:06:03.430889+0000 (oldest deadline 2024-09-10T10:06:29.331091+0000) 2024-09-10T10:07:18.601 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 10 10:07:18 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-11[60437]: 2024-09-10T10:07:18.271+0000 7fa78e4d3640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-10T10:06:13.432384+0000 front 2024-09-10T10:06:13.432489+0000 (oldest deadline 2024-09-10T10:06:35.732466+0000) 2024-09-10T10:07:18.601 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 10 10:07:18 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-11[60437]: 2024-09-10T10:07:18.271+0000 7fa78e4d3640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.102:6812 osd.4 since back 2024-09-10T10:06:22.734557+0000 front 2024-09-10T10:06:22.734365+0000 (oldest deadline 2024-09-10T10:06:44.434370+0000) 2024-09-10T10:07:18.601 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 10 10:07:18 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-11[60437]: 2024-09-10T10:07:18.271+0000 7fa78e4d3640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.159:6812 osd.5 since back 2024-09-10T10:06:29.635809+0000 front 2024-09-10T10:06:29.635901+0000 (oldest deadline 2024-09-10T10:06:53.735941+0000) 2024-09-10T10:07:18.601 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 10 10:07:18 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-11[60437]: 2024-09-10T10:07:18.271+0000 7fa78e4d3640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.5:6822 osd.6 since back 2024-09-10T10:06:38.937377+0000 front 2024-09-10T10:06:38.937447+0000 (oldest deadline 2024-09-10T10:07:01.837229+0000) 2024-09-10T10:07:18.601 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 10 10:07:18 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-11[60437]: 2024-09-10T10:07:18.271+0000 7fa78e4d3640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.102:6820 osd.7 since back 2024-09-10T10:06:47.638708+0000 front 2024-09-10T10:06:47.638696+0000 (oldest deadline 2024-09-10T10:07:09.338624+0000) 2024-09-10T10:07:19.601 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 10 10:07:19 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-11[60437]: 2024-09-10T10:07:19.235+0000 7fa78e4d3640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-10T10:05:49.428950+0000 front 2024-09-10T10:05:49.429031+0000 (oldest deadline 2024-09-10T10:06:11.128800+0000) 2024-09-10T10:07:19.601 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 10 10:07:19 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-11[60437]: 2024-09-10T10:07:19.235+0000 7fa78e4d3640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-10T10:06:03.430910+0000 front 2024-09-10T10:06:03.430889+0000 (oldest deadline 2024-09-10T10:06:29.331091+0000) 2024-09-10T10:07:19.601 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 10 10:07:19 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-11[60437]: 2024-09-10T10:07:19.235+0000 7fa78e4d3640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-10T10:06:13.432384+0000 front 2024-09-10T10:06:13.432489+0000 (oldest deadline 2024-09-10T10:06:35.732466+0000) 2024-09-10T10:07:19.601 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 10 10:07:19 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-11[60437]: 2024-09-10T10:07:19.235+0000 7fa78e4d3640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.102:6812 osd.4 since back 2024-09-10T10:06:22.734557+0000 front 2024-09-10T10:06:22.734365+0000 (oldest deadline 2024-09-10T10:06:44.434370+0000) 2024-09-10T10:07:19.601 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 10 10:07:19 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-11[60437]: 2024-09-10T10:07:19.235+0000 7fa78e4d3640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.159:6812 osd.5 since back 2024-09-10T10:06:29.635809+0000 front 2024-09-10T10:06:29.635901+0000 (oldest deadline 2024-09-10T10:06:53.735941+0000) 2024-09-10T10:07:19.601 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 10 10:07:19 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-11[60437]: 2024-09-10T10:07:19.235+0000 7fa78e4d3640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.5:6822 osd.6 since back 2024-09-10T10:06:38.937377+0000 front 2024-09-10T10:06:38.937447+0000 (oldest deadline 2024-09-10T10:07:01.837229+0000) 2024-09-10T10:07:19.601 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 10 10:07:19 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-11[60437]: 2024-09-10T10:07:19.235+0000 7fa78e4d3640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.102:6820 osd.7 since back 2024-09-10T10:06:47.638708+0000 front 2024-09-10T10:06:47.638696+0000 (oldest deadline 2024-09-10T10:07:09.338624+0000) 2024-09-10T10:07:20.600 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 10 10:07:20 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-11[60437]: 2024-09-10T10:07:20.274+0000 7fa78e4d3640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-10T10:05:49.428950+0000 front 2024-09-10T10:05:49.429031+0000 (oldest deadline 2024-09-10T10:06:11.128800+0000) 2024-09-10T10:07:20.601 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 10 10:07:20 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-11[60437]: 2024-09-10T10:07:20.274+0000 7fa78e4d3640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.159:6804 osd.2 since back 2024-09-10T10:06:03.430910+0000 front 2024-09-10T10:06:03.430889+0000 (oldest deadline 2024-09-10T10:06:29.331091+0000) 2024-09-10T10:07:20.601 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 10 10:07:20 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-11[60437]: 2024-09-10T10:07:20.274+0000 7fa78e4d3640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-10T10:06:13.432384+0000 front 2024-09-10T10:06:13.432489+0000 (oldest deadline 2024-09-10T10:06:35.732466+0000) 2024-09-10T10:07:20.601 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 10 10:07:20 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-11[60437]: 2024-09-10T10:07:20.274+0000 7fa78e4d3640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.102:6812 osd.4 since back 2024-09-10T10:06:22.734557+0000 front 2024-09-10T10:06:22.734365+0000 (oldest deadline 2024-09-10T10:06:44.434370+0000) 2024-09-10T10:07:20.601 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 10 10:07:20 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-11[60437]: 2024-09-10T10:07:20.274+0000 7fa78e4d3640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.159:6812 osd.5 since back 2024-09-10T10:06:29.635809+0000 front 2024-09-10T10:06:29.635901+0000 (oldest deadline 2024-09-10T10:06:53.735941+0000) 2024-09-10T10:07:20.601 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 10 10:07:20 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-11[60437]: 2024-09-10T10:07:20.274+0000 7fa78e4d3640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.5:6822 osd.6 since back 2024-09-10T10:06:38.937377+0000 front 2024-09-10T10:06:38.937447+0000 (oldest deadline 2024-09-10T10:07:01.837229+0000) 2024-09-10T10:07:20.601 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 10 10:07:20 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-11[60437]: 2024-09-10T10:07:20.274+0000 7fa78e4d3640 -1 osd.11 100 heartbeat_check: no reply from 172.21.15.102:6820 osd.7 since back 2024-09-10T10:06:47.638708+0000 front 2024-09-10T10:06:47.638696+0000 (oldest deadline 2024-09-10T10:07:09.338624+0000) 2024-09-10T10:07:22.031 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 10 10:07:21 smithi159 podman[77359]: 2024-09-10 10:07:21.702788969 +0000 UTC m=+5.556951295 container died 6596a70bb228bbd28fb8d93de39ae2addfa1796e1a3fb54ec1eb04c320104487 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-11, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0) 2024-09-10T10:07:22.031 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 10 10:07:22 smithi159 podman[77359]: 2024-09-10 10:07:22.030582601 +0000 UTC m=+5.884744923 container cleanup 6596a70bb228bbd28fb8d93de39ae2addfa1796e1a3fb54ec1eb04c320104487 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-11, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-09-10T10:07:22.350 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 10 10:07:22 smithi159 bash[77359]: ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-11 2024-09-10T10:07:22.350 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 10 10:07:22 smithi159 podman[77376]: 2024-09-10 10:07:22.256960396 +0000 UTC m=+0.548229791 container remove 6596a70bb228bbd28fb8d93de39ae2addfa1796e1a3fb54ec1eb04c320104487 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-11, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2) 2024-09-10T10:07:22.960 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 10 10:07:22 smithi159 podman[77464]: 2024-09-10 10:07:22.608945217 +0000 UTC m=+0.016024701 image pull 41d67112835771c3efca60c6b8fe297c7ddaecdc26cea26fd28aa18636e49005 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 2024-09-10T10:07:22.960 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 10 10:07:22 smithi159 podman[77464]: 2024-09-10 10:07:22.747216959 +0000 UTC m=+0.154296432 container create f0e9eaf0e1094b5dfd5bac711e6a77bc2aea896f1d4d0d921a888a706472a037 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-11-deactivate, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, ceph=True) 2024-09-10T10:07:22.960 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 10 10:07:22 smithi159 podman[77464]: 2024-09-10 10:07:22.959956483 +0000 UTC m=+0.367035958 container init f0e9eaf0e1094b5dfd5bac711e6a77bc2aea896f1d4d0d921a888a706472a037 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-11-deactivate, io.buildah.version=1.37.2, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T10:07:23.351 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 10 10:07:22 smithi159 podman[77464]: 2024-09-10 10:07:22.963372881 +0000 UTC m=+0.370452360 container start f0e9eaf0e1094b5dfd5bac711e6a77bc2aea896f1d4d0d921a888a706472a037 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-11-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-09-10T10:07:23.351 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 10 10:07:23 smithi159 podman[77464]: 2024-09-10 10:07:23.051512668 +0000 UTC m=+0.458592143 container attach f0e9eaf0e1094b5dfd5bac711e6a77bc2aea896f1d4d0d921a888a706472a037 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-11-deactivate, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=) 2024-09-10T10:07:23.351 INFO:journalctl@ceph.osd.11.smithi159.stdout:Sep 10 10:07:23 smithi159 podman[77464]: 2024-09-10 10:07:23.157518509 +0000 UTC m=+0.564597996 container died f0e9eaf0e1094b5dfd5bac711e6a77bc2aea896f1d4d0d921a888a706472a037 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-bade4b50-6f56-11ef-bcea-c7b262605968-osd-11-deactivate, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b) 2024-09-10T10:07:23.753 DEBUG:teuthology.orchestra.run.smithi159:> sudo pkill -f 'journalctl -f -n 0 -u ceph-bade4b50-6f56-11ef-bcea-c7b262605968@osd.11.service' 2024-09-10T10:07:23.792 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T10:07:23.792 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-09-10T10:07:23.792 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-09-10T10:07:23.792 DEBUG:teuthology.orchestra.run.smithi005:> sudo systemctl stop ceph-bade4b50-6f56-11ef-bcea-c7b262605968@mds.a 2024-09-10T10:07:24.306 INFO:journalctl@ceph.mds.a.smithi005.stdout:Sep 10 10:07:23 smithi005 systemd[1]: Stopping Ceph mds.a for bade4b50-6f56-11ef-bcea-c7b262605968... 2024-09-10T10:07:24.806 INFO:journalctl@ceph.mds.a.smithi005.stdout:Sep 10 10:07:24 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mds-a[72209]: 2024-09-10T10:07:24.337+0000 7f6900c00640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T10:07:24.807 INFO:journalctl@ceph.mds.a.smithi005.stdout:Sep 10 10:07:24 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mds-a[72209]: 2024-09-10T10:07:24.337+0000 7f6900c00640 -1 mds.a *** got signal Terminated *** 2024-09-10T10:07:27.468 INFO:journalctl@ceph.mds.a.smithi005.stdout:Sep 10 10:07:27 smithi005 podman[144526]: 2024-09-10 10:07:27.147526199 +0000 UTC m=+3.070969159 container died 0d820c5c04148b88bbaffdf4547e35e1f21930e21f65207af0c8df24b76b32da (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mds-a, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, ceph=True, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-09-10T10:07:27.806 INFO:journalctl@ceph.mds.a.smithi005.stdout:Sep 10 10:07:27 smithi005 podman[144526]: 2024-09-10 10:07:27.665462754 +0000 UTC m=+3.588905710 container cleanup 0d820c5c04148b88bbaffdf4547e35e1f21930e21f65207af0c8df24b76b32da (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mds-a, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True) 2024-09-10T10:07:27.806 INFO:journalctl@ceph.mds.a.smithi005.stdout:Sep 10 10:07:27 smithi005 bash[144526]: ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mds-a 2024-09-10T10:07:28.145 INFO:journalctl@ceph.mds.a.smithi005.stdout:Sep 10 10:07:27 smithi005 podman[144537]: 2024-09-10 10:07:27.891690499 +0000 UTC m=+0.732271947 container remove 0d820c5c04148b88bbaffdf4547e35e1f21930e21f65207af0c8df24b76b32da (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mds-a, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-09-10T10:07:28.173 DEBUG:teuthology.orchestra.run.smithi005:> sudo pkill -f 'journalctl -f -n 0 -u ceph-bade4b50-6f56-11ef-bcea-c7b262605968@mds.a.service' 2024-09-10T10:07:28.246 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T10:07:28.246 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-09-10T10:07:28.246 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-09-10T10:07:28.246 DEBUG:teuthology.orchestra.run.smithi005:> sudo systemctl stop ceph-bade4b50-6f56-11ef-bcea-c7b262605968@mds.d 2024-09-10T10:07:28.556 INFO:journalctl@ceph.mds.d.smithi005.stdout:Sep 10 10:07:28 smithi005 systemd[1]: Stopping Ceph mds.d for bade4b50-6f56-11ef-bcea-c7b262605968... 2024-09-10T10:07:29.056 INFO:journalctl@ceph.mds.d.smithi005.stdout:Sep 10 10:07:28 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mds-d[71009]: 2024-09-10T10:07:28.639+0000 7f9ce9600640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T10:07:29.056 INFO:journalctl@ceph.mds.d.smithi005.stdout:Sep 10 10:07:28 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mds-d[71009]: 2024-09-10T10:07:28.639+0000 7f9ce9600640 -1 mds.d *** got signal Terminated *** 2024-09-10T10:07:31.455 INFO:journalctl@ceph.mds.d.smithi005.stdout:Sep 10 10:07:31 smithi005 podman[144645]: 2024-09-10 10:07:31.151621789 +0000 UTC m=+2.632148173 container died cbc4e0ec9a31bd14cfe177b45d10e437d01edf3925620126f547534fa1c141b8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mds-d, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731) 2024-09-10T10:07:31.806 INFO:journalctl@ceph.mds.d.smithi005.stdout:Sep 10 10:07:31 smithi005 podman[144645]: 2024-09-10 10:07:31.646850927 +0000 UTC m=+3.127377318 container cleanup cbc4e0ec9a31bd14cfe177b45d10e437d01edf3925620126f547534fa1c141b8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mds-d, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, ceph=True) 2024-09-10T10:07:31.806 INFO:journalctl@ceph.mds.d.smithi005.stdout:Sep 10 10:07:31 smithi005 bash[144645]: ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mds-d 2024-09-10T10:07:32.079 DEBUG:teuthology.orchestra.run.smithi005:> sudo pkill -f 'journalctl -f -n 0 -u ceph-bade4b50-6f56-11ef-bcea-c7b262605968@mds.d.service' 2024-09-10T10:07:32.149 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T10:07:32.149 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-09-10T10:07:32.149 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-09-10T10:07:32.149 DEBUG:teuthology.orchestra.run.smithi005:> sudo systemctl stop ceph-bade4b50-6f56-11ef-bcea-c7b262605968@mds.g 2024-09-10T10:07:32.533 INFO:journalctl@ceph.mds.g.smithi005.stdout:Sep 10 10:07:32 smithi005 systemd[1]: Stopping Ceph mds.g for bade4b50-6f56-11ef-bcea-c7b262605968... 2024-09-10T10:07:32.806 INFO:journalctl@ceph.mds.g.smithi005.stdout:Sep 10 10:07:32 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mds-g[71868]: 2024-09-10T10:07:32.528+0000 7f7d21c00640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T10:07:32.806 INFO:journalctl@ceph.mds.g.smithi005.stdout:Sep 10 10:07:32 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mds-g[71868]: 2024-09-10T10:07:32.528+0000 7f7d21c00640 -1 mds.g *** got signal Terminated *** 2024-09-10T10:07:35.437 INFO:journalctl@ceph.mds.g.smithi005.stdout:Sep 10 10:07:35 smithi005 podman[144767]: 2024-09-10 10:07:35.173022255 +0000 UTC m=+2.750484941 container died 3935bf04438d53b284953ea3deadc3df1c7acdb661804bc556c30cfe0915efad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mds-g, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T10:07:35.806 INFO:journalctl@ceph.mds.g.smithi005.stdout:Sep 10 10:07:35 smithi005 podman[144767]: 2024-09-10 10:07:35.625307268 +0000 UTC m=+3.202769963 container cleanup 3935bf04438d53b284953ea3deadc3df1c7acdb661804bc556c30cfe0915efad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mds-g, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True) 2024-09-10T10:07:35.807 INFO:journalctl@ceph.mds.g.smithi005.stdout:Sep 10 10:07:35 smithi005 bash[144767]: ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mds-g 2024-09-10T10:07:36.112 INFO:journalctl@ceph.mds.g.smithi005.stdout:Sep 10 10:07:35 smithi005 podman[144781]: 2024-09-10 10:07:35.853386533 +0000 UTC m=+0.673434424 container remove 3935bf04438d53b284953ea3deadc3df1c7acdb661804bc556c30cfe0915efad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mds-g, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-09-10T10:07:36.120 DEBUG:teuthology.orchestra.run.smithi005:> sudo pkill -f 'journalctl -f -n 0 -u ceph-bade4b50-6f56-11ef-bcea-c7b262605968@mds.g.service' 2024-09-10T10:07:36.188 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T10:07:36.188 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-09-10T10:07:36.188 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-09-10T10:07:36.189 DEBUG:teuthology.orchestra.run.smithi005:> sudo systemctl stop ceph-bade4b50-6f56-11ef-bcea-c7b262605968@mds.j 2024-09-10T10:07:36.556 INFO:journalctl@ceph.mds.j.smithi005.stdout:Sep 10 10:07:36 smithi005 systemd[1]: Stopping Ceph mds.j for bade4b50-6f56-11ef-bcea-c7b262605968... 2024-09-10T10:07:37.056 INFO:journalctl@ceph.mds.j.smithi005.stdout:Sep 10 10:07:36 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mds-j[70631]: 2024-09-10T10:07:36.582+0000 7f567d000640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T10:07:37.056 INFO:journalctl@ceph.mds.j.smithi005.stdout:Sep 10 10:07:36 smithi005 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mds-j[70631]: 2024-09-10T10:07:36.582+0000 7f567d000640 -1 mds.j *** got signal Terminated *** 2024-09-10T10:07:40.916 INFO:journalctl@ceph.mds.j.smithi005.stdout:Sep 10 10:07:40 smithi005 podman[144888]: 2024-09-10 10:07:40.601587414 +0000 UTC m=+4.132823742 container died b66502621e8fee3f628304fa814bfd3b260c93dee478196b082cced14005f7ec (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mds-j, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T10:07:41.306 INFO:journalctl@ceph.mds.j.smithi005.stdout:Sep 10 10:07:41 smithi005 podman[144888]: 2024-09-10 10:07:41.104842304 +0000 UTC m=+4.636078613 container cleanup b66502621e8fee3f628304fa814bfd3b260c93dee478196b082cced14005f7ec (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mds-j, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b) 2024-09-10T10:07:41.306 INFO:journalctl@ceph.mds.j.smithi005.stdout:Sep 10 10:07:41 smithi005 bash[144888]: ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mds-j 2024-09-10T10:07:41.504 DEBUG:teuthology.orchestra.run.smithi005:> sudo pkill -f 'journalctl -f -n 0 -u ceph-bade4b50-6f56-11ef-bcea-c7b262605968@mds.j.service' 2024-09-10T10:07:41.573 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T10:07:41.573 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-09-10T10:07:41.573 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-09-10T10:07:41.574 DEBUG:teuthology.orchestra.run.smithi102:> sudo systemctl stop ceph-bade4b50-6f56-11ef-bcea-c7b262605968@mds.b 2024-09-10T10:07:41.884 INFO:journalctl@ceph.mds.b.smithi102.stdout:Sep 10 10:07:41 smithi102 systemd[1]: Stopping Ceph mds.b for bade4b50-6f56-11ef-bcea-c7b262605968... 2024-09-10T10:07:42.135 INFO:journalctl@ceph.mds.b.smithi102.stdout:Sep 10 10:07:41 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mds-b[65198]: 2024-09-10T10:07:41.914+0000 7f8f27839640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T10:07:42.135 INFO:journalctl@ceph.mds.b.smithi102.stdout:Sep 10 10:07:41 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mds-b[65198]: 2024-09-10T10:07:41.914+0000 7f8f27839640 -1 mds.b *** got signal Terminated *** 2024-09-10T10:07:43.385 INFO:journalctl@ceph.mds.b.smithi102.stdout:Sep 10 10:07:43 smithi102 podman[76282]: 2024-09-10 10:07:43.111818521 +0000 UTC m=+1.337608622 container died 1283ac49c22fa17d6f39a54224510397b8507457965833fa2e2526ed63d98d70 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mds-b, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b) 2024-09-10T10:07:43.741 INFO:journalctl@ceph.mds.b.smithi102.stdout:Sep 10 10:07:43 smithi102 podman[76282]: 2024-09-10 10:07:43.51512335 +0000 UTC m=+1.740913490 container cleanup 1283ac49c22fa17d6f39a54224510397b8507457965833fa2e2526ed63d98d70 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mds-b, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-10T10:07:43.742 INFO:journalctl@ceph.mds.b.smithi102.stdout:Sep 10 10:07:43 smithi102 bash[76282]: ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mds-b 2024-09-10T10:07:43.898 DEBUG:teuthology.orchestra.run.smithi102:> sudo pkill -f 'journalctl -f -n 0 -u ceph-bade4b50-6f56-11ef-bcea-c7b262605968@mds.b.service' 2024-09-10T10:07:43.935 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T10:07:43.935 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-09-10T10:07:43.935 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-09-10T10:07:43.936 DEBUG:teuthology.orchestra.run.smithi102:> sudo systemctl stop ceph-bade4b50-6f56-11ef-bcea-c7b262605968@mds.e 2024-09-10T10:07:44.361 INFO:journalctl@ceph.mds.e.smithi102.stdout:Sep 10 10:07:44 smithi102 systemd[1]: Stopping Ceph mds.e for bade4b50-6f56-11ef-bcea-c7b262605968... 2024-09-10T10:07:44.634 INFO:journalctl@ceph.mds.e.smithi102.stdout:Sep 10 10:07:44 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mds-e[64592]: 2024-09-10T10:07:44.360+0000 7fd8ae0e2640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T10:07:44.634 INFO:journalctl@ceph.mds.e.smithi102.stdout:Sep 10 10:07:44 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mds-e[64592]: 2024-09-10T10:07:44.360+0000 7fd8ae0e2640 -1 mds.e *** got signal Terminated *** 2024-09-10T10:07:47.361 INFO:journalctl@ceph.mds.e.smithi102.stdout:Sep 10 10:07:47 smithi102 podman[76413]: 2024-09-10 10:07:47.107855604 +0000 UTC m=+2.955686639 container died b23eaeab4c90e6fb404c7bb6e4895de9ec81a002cf262d21f24bf4b3f93147de (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mds-e, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b) 2024-09-10T10:07:47.634 INFO:journalctl@ceph.mds.e.smithi102.stdout:Sep 10 10:07:47 smithi102 podman[76413]: 2024-09-10 10:07:47.443999422 +0000 UTC m=+3.291830457 container cleanup b23eaeab4c90e6fb404c7bb6e4895de9ec81a002cf262d21f24bf4b3f93147de (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mds-e, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-09-10T10:07:47.635 INFO:journalctl@ceph.mds.e.smithi102.stdout:Sep 10 10:07:47 smithi102 bash[76413]: ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mds-e 2024-09-10T10:07:47.874 DEBUG:teuthology.orchestra.run.smithi102:> sudo pkill -f 'journalctl -f -n 0 -u ceph-bade4b50-6f56-11ef-bcea-c7b262605968@mds.e.service' 2024-09-10T10:07:47.896 INFO:journalctl@ceph.mds.e.smithi102.stdout:Sep 10 10:07:47 smithi102 podman[76425]: 2024-09-10 10:07:47.636822503 +0000 UTC m=+0.527586590 container remove b23eaeab4c90e6fb404c7bb6e4895de9ec81a002cf262d21f24bf4b3f93147de (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mds-e, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-09-10T10:07:47.896 INFO:journalctl@ceph.mds.e.smithi102.stdout:Sep 10 10:07:47 smithi102 systemd[1]: ceph-bade4b50-6f56-11ef-bcea-c7b262605968@mds.e.service: Deactivated successfully. 2024-09-10T10:07:47.896 INFO:journalctl@ceph.mds.e.smithi102.stdout:Sep 10 10:07:47 smithi102 systemd[1]: Stopped Ceph mds.e for bade4b50-6f56-11ef-bcea-c7b262605968. 2024-09-10T10:07:47.896 INFO:journalctl@ceph.mds.e.smithi102.stdout:Sep 10 10:07:47 smithi102 systemd[1]: ceph-bade4b50-6f56-11ef-bcea-c7b262605968@mds.e.service: Consumed 11.124s CPU time. 2024-09-10T10:07:47.907 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T10:07:47.908 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-09-10T10:07:47.908 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-09-10T10:07:47.908 DEBUG:teuthology.orchestra.run.smithi102:> sudo systemctl stop ceph-bade4b50-6f56-11ef-bcea-c7b262605968@mds.h 2024-09-10T10:07:48.223 INFO:journalctl@ceph.mds.h.smithi102.stdout:Sep 10 10:07:47 smithi102 systemd[1]: Stopping Ceph mds.h for bade4b50-6f56-11ef-bcea-c7b262605968... 2024-09-10T10:07:48.634 INFO:journalctl@ceph.mds.h.smithi102.stdout:Sep 10 10:07:48 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mds-h[64895]: 2024-09-10T10:07:48.222+0000 7f440f3f3640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T10:07:48.635 INFO:journalctl@ceph.mds.h.smithi102.stdout:Sep 10 10:07:48 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mds-h[64895]: 2024-09-10T10:07:48.222+0000 7f440f3f3640 -1 mds.h *** got signal Terminated *** 2024-09-10T10:07:51.385 INFO:journalctl@ceph.mds.h.smithi102.stdout:Sep 10 10:07:51 smithi102 podman[76537]: 2024-09-10 10:07:51.112446486 +0000 UTC m=+2.989614654 container died 0989d5e3d2ed54de7dd4a556306c46317db8753e6660efdb0f601ba19a420317 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mds-h, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, ceph=True, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b) 2024-09-10T10:07:51.724 INFO:journalctl@ceph.mds.h.smithi102.stdout:Sep 10 10:07:51 smithi102 podman[76537]: 2024-09-10 10:07:51.481304307 +0000 UTC m=+3.358472474 container cleanup 0989d5e3d2ed54de7dd4a556306c46317db8753e6660efdb0f601ba19a420317 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mds-h, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.build-date=20240731, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-09-10T10:07:51.724 INFO:journalctl@ceph.mds.h.smithi102.stdout:Sep 10 10:07:51 smithi102 bash[76537]: ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mds-h 2024-09-10T10:07:51.861 DEBUG:teuthology.orchestra.run.smithi102:> sudo pkill -f 'journalctl -f -n 0 -u ceph-bade4b50-6f56-11ef-bcea-c7b262605968@mds.h.service' 2024-09-10T10:07:51.893 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T10:07:51.893 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-09-10T10:07:51.893 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-09-10T10:07:51.894 DEBUG:teuthology.orchestra.run.smithi102:> sudo systemctl stop ceph-bade4b50-6f56-11ef-bcea-c7b262605968@mds.k 2024-09-10T10:07:52.135 INFO:journalctl@ceph.mds.k.smithi102.stdout:Sep 10 10:07:51 smithi102 systemd[1]: Stopping Ceph mds.k for bade4b50-6f56-11ef-bcea-c7b262605968... 2024-09-10T10:07:52.592 INFO:journalctl@ceph.mds.k.smithi102.stdout:Sep 10 10:07:52 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mds-k[64279]: 2024-09-10T10:07:52.234+0000 7f2635627640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T10:07:52.592 INFO:journalctl@ceph.mds.k.smithi102.stdout:Sep 10 10:07:52 smithi102 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mds-k[64279]: 2024-09-10T10:07:52.234+0000 7f2635627640 -1 mds.k *** got signal Terminated *** 2024-09-10T10:07:55.384 INFO:journalctl@ceph.mds.k.smithi102.stdout:Sep 10 10:07:55 smithi102 podman[76656]: 2024-09-10 10:07:55.109193304 +0000 UTC m=+3.002335256 container died a2e3425919b256e9588efcab9ebf4425d6025cc88b7afac9602aafb62003d167 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mds-k, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T10:07:55.761 INFO:journalctl@ceph.mds.k.smithi102.stdout:Sep 10 10:07:55 smithi102 podman[76656]: 2024-09-10 10:07:55.501999454 +0000 UTC m=+3.395141409 container cleanup a2e3425919b256e9588efcab9ebf4425d6025cc88b7afac9602aafb62003d167 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mds-k, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-09-10T10:07:55.762 INFO:journalctl@ceph.mds.k.smithi102.stdout:Sep 10 10:07:55 smithi102 bash[76656]: ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mds-k 2024-09-10T10:07:55.881 DEBUG:teuthology.orchestra.run.smithi102:> sudo pkill -f 'journalctl -f -n 0 -u ceph-bade4b50-6f56-11ef-bcea-c7b262605968@mds.k.service' 2024-09-10T10:07:55.913 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T10:07:55.914 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-09-10T10:07:55.914 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-09-10T10:07:55.914 DEBUG:teuthology.orchestra.run.smithi159:> sudo systemctl stop ceph-bade4b50-6f56-11ef-bcea-c7b262605968@mds.c 2024-09-10T10:07:56.272 INFO:journalctl@ceph.mds.c.smithi159.stdout:Sep 10 10:07:55 smithi159 systemd[1]: Stopping Ceph mds.c for bade4b50-6f56-11ef-bcea-c7b262605968... 2024-09-10T10:07:56.600 INFO:journalctl@ceph.mds.c.smithi159.stdout:Sep 10 10:07:56 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mds-c[65276]: 2024-09-10T10:07:56.270+0000 7fb6bf234640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T10:07:56.600 INFO:journalctl@ceph.mds.c.smithi159.stdout:Sep 10 10:07:56 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mds-c[65276]: 2024-09-10T10:07:56.270+0000 7fb6bf234640 -1 mds.c *** got signal Terminated *** 2024-09-10T10:07:59.405 INFO:journalctl@ceph.mds.c.smithi159.stdout:Sep 10 10:07:59 smithi159 podman[77569]: 2024-09-10 10:07:59.126592488 +0000 UTC m=+3.019856952 container died d0d410063ea5681cb353cad93f1b4c5eb2f621442e3d855122b32108daaea28c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mds-c, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-09-10T10:07:59.755 INFO:journalctl@ceph.mds.c.smithi159.stdout:Sep 10 10:07:59 smithi159 podman[77569]: 2024-09-10 10:07:59.503932626 +0000 UTC m=+3.397197073 container cleanup d0d410063ea5681cb353cad93f1b4c5eb2f621442e3d855122b32108daaea28c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mds-c, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, ceph=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-09-10T10:07:59.756 INFO:journalctl@ceph.mds.c.smithi159.stdout:Sep 10 10:07:59 smithi159 bash[77569]: ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mds-c 2024-09-10T10:07:59.882 DEBUG:teuthology.orchestra.run.smithi159:> sudo pkill -f 'journalctl -f -n 0 -u ceph-bade4b50-6f56-11ef-bcea-c7b262605968@mds.c.service' 2024-09-10T10:07:59.918 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T10:07:59.918 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-09-10T10:07:59.919 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-09-10T10:07:59.919 DEBUG:teuthology.orchestra.run.smithi159:> sudo systemctl stop ceph-bade4b50-6f56-11ef-bcea-c7b262605968@mds.f 2024-09-10T10:08:00.100 INFO:journalctl@ceph.mds.f.smithi159.stdout:Sep 10 10:07:59 smithi159 systemd[1]: Stopping Ceph mds.f for bade4b50-6f56-11ef-bcea-c7b262605968... 2024-09-10T10:08:00.600 INFO:journalctl@ceph.mds.f.smithi159.stdout:Sep 10 10:08:00 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mds-f[64973]: 2024-09-10T10:08:00.224+0000 7fd145789640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T10:08:00.601 INFO:journalctl@ceph.mds.f.smithi159.stdout:Sep 10 10:08:00 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mds-f[64973]: 2024-09-10T10:08:00.224+0000 7fd145789640 -1 mds.f *** got signal Terminated *** 2024-09-10T10:08:03.392 INFO:journalctl@ceph.mds.f.smithi159.stdout:Sep 10 10:08:03 smithi159 podman[77701]: 2024-09-10 10:08:03.136470438 +0000 UTC m=+3.001898274 container died 206912c328a0446e5b2f47ba768c5cca7fa401e0ed50f09a8e9132b94a26182f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mds-f, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-10T10:08:03.692 INFO:journalctl@ceph.mds.f.smithi159.stdout:Sep 10 10:08:03 smithi159 podman[77701]: 2024-09-10 10:08:03.47396549 +0000 UTC m=+3.339393329 container cleanup 206912c328a0446e5b2f47ba768c5cca7fa401e0ed50f09a8e9132b94a26182f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mds-f, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-09-10T10:08:03.692 INFO:journalctl@ceph.mds.f.smithi159.stdout:Sep 10 10:08:03 smithi159 bash[77701]: ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mds-f 2024-09-10T10:08:03.944 INFO:journalctl@ceph.mds.f.smithi159.stdout:Sep 10 10:08:03 smithi159 podman[77714]: 2024-09-10 10:08:03.691760304 +0000 UTC m=+0.552047003 container remove 206912c328a0446e5b2f47ba768c5cca7fa401e0ed50f09a8e9132b94a26182f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mds-f, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0) 2024-09-10T10:08:03.948 DEBUG:teuthology.orchestra.run.smithi159:> sudo pkill -f 'journalctl -f -n 0 -u ceph-bade4b50-6f56-11ef-bcea-c7b262605968@mds.f.service' 2024-09-10T10:08:03.981 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T10:08:03.981 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-09-10T10:08:03.981 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-09-10T10:08:03.982 DEBUG:teuthology.orchestra.run.smithi159:> sudo systemctl stop ceph-bade4b50-6f56-11ef-bcea-c7b262605968@mds.i 2024-09-10T10:08:04.296 INFO:journalctl@ceph.mds.i.smithi159.stdout:Sep 10 10:08:04 smithi159 systemd[1]: Stopping Ceph mds.i for bade4b50-6f56-11ef-bcea-c7b262605968... 2024-09-10T10:08:04.600 INFO:journalctl@ceph.mds.i.smithi159.stdout:Sep 10 10:08:04 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mds-i[65577]: 2024-09-10T10:08:04.294+0000 7feb131ca640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T10:08:04.600 INFO:journalctl@ceph.mds.i.smithi159.stdout:Sep 10 10:08:04 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mds-i[65577]: 2024-09-10T10:08:04.294+0000 7feb131ca640 -1 mds.i *** got signal Terminated *** 2024-09-10T10:08:07.404 INFO:journalctl@ceph.mds.i.smithi159.stdout:Sep 10 10:08:07 smithi159 podman[77825]: 2024-09-10 10:08:07.132652914 +0000 UTC m=+2.938020909 container died dfe08e65daed26646feb9f6ede92ac54dda8f31d601c8c9167bd3ae02807dde0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mds-i, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.schema-version=1.0) 2024-09-10T10:08:07.663 INFO:journalctl@ceph.mds.i.smithi159.stdout:Sep 10 10:08:07 smithi159 podman[77825]: 2024-09-10 10:08:07.477627559 +0000 UTC m=+3.282995551 container cleanup dfe08e65daed26646feb9f6ede92ac54dda8f31d601c8c9167bd3ae02807dde0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mds-i, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-09-10T10:08:07.663 INFO:journalctl@ceph.mds.i.smithi159.stdout:Sep 10 10:08:07 smithi159 bash[77825]: ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mds-i 2024-09-10T10:08:07.899 DEBUG:teuthology.orchestra.run.smithi159:> sudo pkill -f 'journalctl -f -n 0 -u ceph-bade4b50-6f56-11ef-bcea-c7b262605968@mds.i.service' 2024-09-10T10:08:07.920 INFO:journalctl@ceph.mds.i.smithi159.stdout:Sep 10 10:08:07 smithi159 podman[77837]: 2024-09-10 10:08:07.662267241 +0000 UTC m=+0.525814339 container remove dfe08e65daed26646feb9f6ede92ac54dda8f31d601c8c9167bd3ae02807dde0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mds-i, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True) 2024-09-10T10:08:07.920 INFO:journalctl@ceph.mds.i.smithi159.stdout:Sep 10 10:08:07 smithi159 systemd[1]: ceph-bade4b50-6f56-11ef-bcea-c7b262605968@mds.i.service: Deactivated successfully. 2024-09-10T10:08:07.921 INFO:journalctl@ceph.mds.i.smithi159.stdout:Sep 10 10:08:07 smithi159 systemd[1]: Stopped Ceph mds.i for bade4b50-6f56-11ef-bcea-c7b262605968. 2024-09-10T10:08:07.921 INFO:journalctl@ceph.mds.i.smithi159.stdout:Sep 10 10:08:07 smithi159 systemd[1]: ceph-bade4b50-6f56-11ef-bcea-c7b262605968@mds.i.service: Consumed 2min 54.664s CPU time. 2024-09-10T10:08:07.932 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T10:08:07.932 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-09-10T10:08:07.933 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-09-10T10:08:07.933 DEBUG:teuthology.orchestra.run.smithi159:> sudo systemctl stop ceph-bade4b50-6f56-11ef-bcea-c7b262605968@mds.l 2024-09-10T10:08:08.274 INFO:journalctl@ceph.mds.l.smithi159.stdout:Sep 10 10:08:08 smithi159 systemd[1]: Stopping Ceph mds.l for bade4b50-6f56-11ef-bcea-c7b262605968... 2024-09-10T10:08:08.565 INFO:journalctl@ceph.mds.l.smithi159.stdout:Sep 10 10:08:08 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mds-l[64670]: 2024-09-10T10:08:08.273+0000 7f6b70374640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T10:08:08.565 INFO:journalctl@ceph.mds.l.smithi159.stdout:Sep 10 10:08:08 smithi159 ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mds-l[64670]: 2024-09-10T10:08:08.273+0000 7f6b70374640 -1 mds.l *** got signal Terminated *** 2024-09-10T10:08:11.413 INFO:journalctl@ceph.mds.l.smithi159.stdout:Sep 10 10:08:11 smithi159 podman[77946]: 2024-09-10 10:08:11.131253154 +0000 UTC m=+2.974107081 container died 8b40c3762f535df0c1c66505a9fe26fede5d1ab0eaaa2235af7675efa86092c1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mds-l, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-09-10T10:08:11.774 INFO:journalctl@ceph.mds.l.smithi159.stdout:Sep 10 10:08:11 smithi159 podman[77946]: 2024-09-10 10:08:11.522668938 +0000 UTC m=+3.365522871 container cleanup 8b40c3762f535df0c1c66505a9fe26fede5d1ab0eaaa2235af7675efa86092c1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mds-l, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.build-date=20240731) 2024-09-10T10:08:11.774 INFO:journalctl@ceph.mds.l.smithi159.stdout:Sep 10 10:08:11 smithi159 bash[77946]: ceph-bade4b50-6f56-11ef-bcea-c7b262605968-mds-l 2024-09-10T10:08:11.900 DEBUG:teuthology.orchestra.run.smithi159:> sudo pkill -f 'journalctl -f -n 0 -u ceph-bade4b50-6f56-11ef-bcea-c7b262605968@mds.l.service' 2024-09-10T10:08:11.932 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T10:08:11.933 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-09-10T10:08:11.933 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid bade4b50-6f56-11ef-bcea-c7b262605968 --force --keep-logs 2024-09-10T10:08:12.214 INFO:teuthology.orchestra.run.smithi005.stdout:Deleting cluster with fsid: bade4b50-6f56-11ef-bcea-c7b262605968 2024-09-10T10:08:16.064 DEBUG:teuthology.orchestra.run.smithi102:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid bade4b50-6f56-11ef-bcea-c7b262605968 --force --keep-logs 2024-09-10T10:08:16.262 INFO:teuthology.orchestra.run.smithi102.stdout:Deleting cluster with fsid: bade4b50-6f56-11ef-bcea-c7b262605968 2024-09-10T10:08:18.783 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid bade4b50-6f56-11ef-bcea-c7b262605968 --force --keep-logs 2024-09-10T10:08:18.983 INFO:teuthology.orchestra.run.smithi159.stdout:Deleting cluster with fsid: bade4b50-6f56-11ef-bcea-c7b262605968 2024-09-10T10:08:21.549 DEBUG:teuthology.orchestra.run.smithi005:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-10T10:08:21.612 DEBUG:teuthology.orchestra.run.smithi102:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-10T10:08:21.643 DEBUG:teuthology.orchestra.run.smithi159:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-10T10:08:21.671 INFO:tasks.cephadm:Archiving crash dumps... 2024-09-10T10:08:21.673 DEBUG:teuthology.misc:Transferring archived files from smithi005:/var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/crash to /home/teuthworker/archive/rishabh-2024-09-10_06:58:10-fs-wip-rishabh-testing-20240909.165402-squid-distro-default-smithi/7898972/remote/smithi005/crash 2024-09-10T10:08:21.674 DEBUG:teuthology.orchestra.run.smithi005:> sudo tar c -f - -C /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/crash -- . 2024-09-10T10:08:21.745 INFO:teuthology.orchestra.run.smithi005.stderr:tar: /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/crash: Cannot open: No such file or directory 2024-09-10T10:08:21.746 INFO:teuthology.orchestra.run.smithi005.stderr:tar: Error is not recoverable: exiting now 2024-09-10T10:08:21.749 DEBUG:teuthology.misc:Transferring archived files from smithi102:/var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/crash to /home/teuthworker/archive/rishabh-2024-09-10_06:58:10-fs-wip-rishabh-testing-20240909.165402-squid-distro-default-smithi/7898972/remote/smithi102/crash 2024-09-10T10:08:21.750 DEBUG:teuthology.orchestra.run.smithi102:> sudo tar c -f - -C /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/crash -- . 2024-09-10T10:08:21.796 INFO:teuthology.orchestra.run.smithi102.stderr:tar: /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/crash: Cannot open: No such file or directory 2024-09-10T10:08:21.796 INFO:teuthology.orchestra.run.smithi102.stderr:tar: Error is not recoverable: exiting now 2024-09-10T10:08:21.798 DEBUG:teuthology.misc:Transferring archived files from smithi159:/var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/crash to /home/teuthworker/archive/rishabh-2024-09-10_06:58:10-fs-wip-rishabh-testing-20240909.165402-squid-distro-default-smithi/7898972/remote/smithi159/crash 2024-09-10T10:08:21.799 DEBUG:teuthology.orchestra.run.smithi159:> sudo tar c -f - -C /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/crash -- . 2024-09-10T10:08:21.837 INFO:teuthology.orchestra.run.smithi159.stderr:tar: /var/lib/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/crash: Cannot open: No such file or directory 2024-09-10T10:08:21.838 INFO:teuthology.orchestra.run.smithi159.stderr:tar: Error is not recoverable: exiting now 2024-09-10T10:08:21.839 INFO:tasks.cephadm:Checking cluster log for badness... 2024-09-10T10:08:21.839 DEBUG:teuthology.orchestra.run.smithi005:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'online, but wants' | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Degraded data redundancy' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-09-10T10:08:21.923 INFO:tasks.cephadm:Compressing logs... 2024-09-10T10:08:21.924 DEBUG:teuthology.orchestra.run.smithi005:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-10T10:08:21.928 DEBUG:teuthology.orchestra.run.smithi102:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-10T10:08:21.931 DEBUG:teuthology.orchestra.run.smithi159:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-10T10:08:21.958 INFO:teuthology.orchestra.run.smithi159.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-09-10T10:08:21.959 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/ceph-volume.log 2024-09-10T10:08:21.959 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/ceph.audit.log 2024-09-10T10:08:21.961 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/ceph-mds.i.log 2024-09-10T10:08:21.961 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/ceph-mon.c.log 2024-09-10T10:08:21.961 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/ceph-mds.i.log: gzip -5 --verbose -- /var/log/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/ceph.cephadm.log 2024-09-10T10:08:21.961 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/ceph-osd.11.log 2024-09-10T10:08:21.963 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/ceph-osd.2.log 2024-09-10T10:08:21.964 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/ceph-osd.11.log: gzip -5 --verbose -- /var/log/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/ceph-osd.5.log 2024-09-10T10:08:21.964 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/ceph.log 2024-09-10T10:08:21.964 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/ceph-mds.l.log 2024-09-10T10:08:21.964 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/ceph-osd.8.log 2024-09-10T10:08:21.964 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/ceph-mds.l.log: gzip -5 --verbose -- /var/log/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/ceph-mgr.z.log 2024-09-10T10:08:21.964 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/ceph-osd.8.log: gzip -5 --verbose -- /var/log/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/ceph-mds.c.log 2024-09-10T10:08:21.964 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/ceph-mgr.z.log: gzip -5 --verbose -- /var/log/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/ceph-mds.f.log 2024-09-10T10:08:21.964 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-10T10:08:21.968 INFO:teuthology.orchestra.run.smithi102.stderr:find: ‘/var/log/rbd-target-api’gzip -5: No such file or directory --verbose 2024-09-10T10:08:21.968 INFO:teuthology.orchestra.run.smithi102.stderr: -- /var/log/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/ceph-osd.10.log 2024-09-10T10:08:21.968 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/ceph-osd.1.log 2024-09-10T10:08:21.970 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/ceph-osd.7.log 2024-09-10T10:08:21.970 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/ceph-osd.10.log: /var/log/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/ceph-volume.log 2024-09-10T10:08:21.970 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/ceph-mds.b.log 2024-09-10T10:08:21.973 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/ceph-mds.k.log 2024-09-10T10:08:21.973 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/ceph-osd.4.log 2024-09-10T10:08:21.973 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/ceph-mds.k.log: gzip -5 --verbose -- /var/log/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/ceph.audit.log 2024-09-10T10:08:21.973 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/ceph.cephadm.log 2024-09-10T10:08:21.973 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/ceph-mgr.y.log 2024-09-10T10:08:21.973 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/ceph.log 2024-09-10T10:08:21.973 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/ceph-mds.h.log 2024-09-10T10:08:21.973 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/ceph-mon.b.log 2024-09-10T10:08:21.974 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/ceph-mds.h.log: gzip -5 --verbose -- /var/log/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/ceph-mds.e.log 2024-09-10T10:08:21.974 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-10T10:08:21.978 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.log 2024-09-10T10:08:21.979 INFO:teuthology.orchestra.run.smithi005.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-09-10T10:08:21.979 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/ceph-osd.0.log 2024-09-10T10:08:21.980 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.log: gzip -5 --verbose -- /var/log/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/ceph-osd.9.log 2024-09-10T10:08:21.981 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/ceph-volume.log 2024-09-10T10:08:21.981 INFO:teuthology.orchestra.run.smithi005.stderr: 92.5% -- replaced with /var/log/ceph/ceph-client.admin.log.gz 2024-09-10T10:08:21.982 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/ceph-osd.9.log: gzip -5 --verbose -- /var/log/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/ceph.audit.log 2024-09-10T10:08:21.983 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/ceph-mds.a.log 2024-09-10T10:08:21.984 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/ceph.cephadm.log 2024-09-10T10:08:21.985 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/ceph-mon.a.log 2024-09-10T10:08:21.986 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/ceph-mds.g.log 2024-09-10T10:08:21.987 INFO:teuthology.orchestra.run.smithi005.stderr: 83.0% -- replaced with /var/log/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/ceph.cephadm.log.gz 2024-09-10T10:08:21.987 INFO:teuthology.orchestra.run.smithi005.stderr: 90.3% -- replaced with /var/log/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/ceph.audit.log.gz 2024-09-10T10:08:21.987 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/ceph-osd.6.log 2024-09-10T10:08:21.988 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/ceph.log 2024-09-10T10:08:21.991 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/ceph-mds.g.log: /var/log/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/ceph-osd.6.log: /var/log/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/ceph-mds.d.log 2024-09-10T10:08:21.995 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/ceph-mds.d.log: gzip -5 --verbose -- /var/log/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/ceph-mgr.x.log 2024-09-10T10:08:21.996 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/ceph-mds.j.log 2024-09-10T10:08:21.997 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/ceph-mgr.x.log: 94.5% -- replaced with /var/log/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/ceph-mds.g.log.gz 2024-09-10T10:08:21.999 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/ceph-osd.3.log 2024-09-10T10:08:21.999 INFO:teuthology.orchestra.run.smithi005.stderr: 90.9% -- replaced with /var/log/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/ceph-volume.log.gz 2024-09-10T10:08:22.001 INFO:teuthology.orchestra.run.smithi005.stderr:gzip/var/log/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/ceph-mds.j.log: /var/log/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/ceph-osd.3.log: -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-10T10:08:22.007 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/cephadm.log: 87.0% -- replaced with /var/log/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/ceph.log.gz 2024-09-10T10:08:22.010 INFO:teuthology.orchestra.run.smithi005.stderr: 94.6% -- replaced with /var/log/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/ceph-mds.j.log.gz 2024-09-10T10:08:22.026 INFO:teuthology.orchestra.run.smithi005.stderr: 94.8% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-10T10:08:22.172 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/ceph-mds.f.log: /var/log/ceph/cephadm.log: 93.6% -- replaced with /var/log/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/ceph-mgr.z.log.gz 2024-09-10T10:08:22.173 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/ceph-mds.e.log: /var/log/ceph/cephadm.log: 93.6% -- replaced with /var/log/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/ceph-mgr.y.log.gz 2024-09-10T10:08:22.509 INFO:teuthology.orchestra.run.smithi005.stderr: 90.1% -- replaced with /var/log/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/ceph-mgr.x.log.gz 2024-09-10T10:08:22.589 INFO:teuthology.orchestra.run.smithi159.stderr: 92.3% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-10T10:08:22.664 INFO:teuthology.orchestra.run.smithi159.stderr: 83.2% -- replaced with /var/log/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/ceph.cephadm.log.gz 2024-09-10T10:08:22.681 INFO:teuthology.orchestra.run.smithi159.stderr: 90.4% -- replaced with /var/log/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/ceph.audit.log.gz 2024-09-10T10:08:22.715 INFO:teuthology.orchestra.run.smithi159.stderr: 87.1% -- replaced with /var/log/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/ceph.log.gz 2024-09-10T10:08:22.822 INFO:teuthology.orchestra.run.smithi005.stderr: 95.5% -- replaced with /var/log/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/ceph-mds.a.log.gz 2024-09-10T10:08:23.394 INFO:teuthology.orchestra.run.smithi102.stderr: 83.3% -- replaced with /var/log/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/ceph.cephadm.log.gz 2024-09-10T10:08:23.411 INFO:teuthology.orchestra.run.smithi102.stderr: 91.8% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-10T10:08:23.506 INFO:teuthology.orchestra.run.smithi102.stderr: 90.4% -- replaced with /var/log/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/ceph.audit.log.gz 2024-09-10T10:08:23.563 INFO:teuthology.orchestra.run.smithi102.stderr: 87.0% -- replaced with /var/log/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/ceph.log.gz 2024-09-10T10:08:24.106 INFO:teuthology.orchestra.run.smithi102.stderr: 91.0% -- replaced with /var/log/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/ceph-volume.log.gz 2024-09-10T10:08:24.268 INFO:teuthology.orchestra.run.smithi005.stderr: 90.1% -- replaced with /var/log/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/ceph-mon.a.log.gz 2024-09-10T10:08:24.363 INFO:teuthology.orchestra.run.smithi159.stderr: 91.1% -- replaced with /var/log/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/ceph-volume.log.gz 2024-09-10T10:08:28.409 INFO:teuthology.orchestra.run.smithi102.stderr: 91.6% -- replaced with /var/log/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/ceph-mon.b.log.gz 2024-09-10T10:08:38.393 INFO:teuthology.orchestra.run.smithi005.stderr: 94.9% -- replaced with /var/log/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/ceph-mds.d.log.gz 2024-09-10T10:08:47.995 INFO:teuthology.orchestra.run.smithi102.stderr: 95.6% -- replaced with /var/log/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/ceph-mds.e.log.gz 2024-09-10T10:09:00.267 INFO:teuthology.orchestra.run.smithi005.stderr: 91.6% -- replaced with /var/log/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/ceph-osd.0.log.gz 2024-09-10T10:09:08.761 INFO:teuthology.orchestra.run.smithi005.stderr: 91.3% -- replaced with /var/log/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/ceph-osd.6.log.gz 2024-09-10T10:09:14.728 INFO:teuthology.orchestra.run.smithi005.stderr: 91.3% -- replaced with /var/log/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/ceph-osd.3.log.gz 2024-09-10T10:09:18.274 INFO:teuthology.orchestra.run.smithi005.stderr: 91.2% -- replaced with /var/log/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/ceph-osd.9.log.gz 2024-09-10T10:09:18.278 INFO:teuthology.orchestra.run.smithi005.stderr: 2024-09-10T10:09:18.279 INFO:teuthology.orchestra.run.smithi005.stderr:real 0m56.321s 2024-09-10T10:09:18.279 INFO:teuthology.orchestra.run.smithi005.stderr:user 3m18.610s 2024-09-10T10:09:18.279 INFO:teuthology.orchestra.run.smithi005.stderr:sys 0m15.516s 2024-09-10T10:09:30.100 INFO:teuthology.orchestra.run.smithi159.stderr: 92.9% -- replaced with /var/log/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/ceph-mon.c.log.gz 2024-09-10T10:09:42.017 INFO:teuthology.orchestra.run.smithi102.stderr: 91.7% -- replaced with /var/log/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/ceph-osd.1.log.gz 2024-09-10T10:09:56.147 INFO:teuthology.orchestra.run.smithi102.stderr: 91.3% -- replaced with /var/log/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/ceph-osd.7.log.gz 2024-09-10T10:09:56.844 INFO:teuthology.orchestra.run.smithi102.stderr: 94.3% -- replaced with /var/log/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/ceph-mds.k.log.gz 2024-09-10T10:10:24.166 INFO:teuthology.orchestra.run.smithi102.stderr: 91.4% -- replaced with /var/log/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/ceph-osd.10.log.gz 2024-09-10T10:10:28.281 INFO:teuthology.orchestra.run.smithi159.stderr: 95.5% -- replaced with /var/log/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/ceph-mds.l.log.gz 2024-09-10T10:10:37.616 INFO:teuthology.orchestra.run.smithi102.stderr: 93.8% -- replaced with /var/log/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/ceph-mds.b.log.gz 2024-09-10T10:10:50.913 INFO:teuthology.orchestra.run.smithi159.stderr: 95.5% -- replaced with /var/log/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/ceph-mds.f.log.gz 2024-09-10T10:11:24.944 INFO:teuthology.orchestra.run.smithi102.stderr: 91.4% -- replaced with /var/log/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/ceph-osd.4.log.gz 2024-09-10T10:11:26.375 INFO:teuthology.orchestra.run.smithi102.stderr: 93.7% -- replaced with /var/log/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/ceph-mds.h.log.gz 2024-09-10T10:11:26.378 INFO:teuthology.orchestra.run.smithi102.stderr: 2024-09-10T10:11:26.378 INFO:teuthology.orchestra.run.smithi102.stderr:real 3m4.433s 2024-09-10T10:11:26.379 INFO:teuthology.orchestra.run.smithi102.stderr:user 4m39.810s 2024-09-10T10:11:26.379 INFO:teuthology.orchestra.run.smithi102.stderr:sys 0m10.300s 2024-09-10T10:16:13.526 INFO:teuthology.orchestra.run.smithi159.stderr: 91.5% -- replaced with /var/log/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/ceph-osd.11.log.gz 2024-09-10T10:17:37.882 INFO:teuthology.orchestra.run.smithi159.stderr: 91.1% -- replaced with /var/log/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/ceph-osd.8.log.gz 2024-09-10T10:18:15.154 INFO:teuthology.orchestra.run.smithi159.stderr: 91.3% -- replaced with /var/log/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/ceph-osd.2.log.gz 2024-09-10T10:20:48.252 INFO:teuthology.orchestra.run.smithi159.stderr: 93.6% -- replaced with /var/log/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/ceph-mds.i.log.gz 2024-09-10T10:21:15.439 INFO:teuthology.orchestra.run.smithi159.stderr: 91.1% -- replaced with /var/log/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/ceph-osd.5.log.gz 2024-09-10T10:25:29.658 INFO:teuthology.orchestra.run.smithi159.stderr: 93.2% -- replaced with /var/log/ceph/bade4b50-6f56-11ef-bcea-c7b262605968/ceph-mds.c.log.gz 2024-09-10T10:25:29.687 INFO:teuthology.orchestra.run.smithi159.stderr: 2024-09-10T10:25:29.687 INFO:teuthology.orchestra.run.smithi159.stderr:real 17m7.714s 2024-09-10T10:25:29.687 INFO:teuthology.orchestra.run.smithi159.stderr:user 8m17.329s 2024-09-10T10:25:29.687 INFO:teuthology.orchestra.run.smithi159.stderr:sys 0m22.282s 2024-09-10T10:25:29.688 INFO:tasks.cephadm:Archiving logs... 2024-09-10T10:25:29.689 DEBUG:teuthology.misc:Transferring archived files from smithi005:/var/log/ceph to /home/teuthworker/archive/rishabh-2024-09-10_06:58:10-fs-wip-rishabh-testing-20240909.165402-squid-distro-default-smithi/7898972/remote/smithi005/log 2024-09-10T10:25:29.690 DEBUG:teuthology.orchestra.run.smithi005:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-10T10:26:00.531 DEBUG:teuthology.misc:Transferring archived files from smithi102:/var/log/ceph to /home/teuthworker/archive/rishabh-2024-09-10_06:58:10-fs-wip-rishabh-testing-20240909.165402-squid-distro-default-smithi/7898972/remote/smithi102/log 2024-09-10T10:26:00.532 DEBUG:teuthology.orchestra.run.smithi102:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-10T10:26:44.742 DEBUG:teuthology.misc:Transferring archived files from smithi159:/var/log/ceph to /home/teuthworker/archive/rishabh-2024-09-10_06:58:10-fs-wip-rishabh-testing-20240909.165402-squid-distro-default-smithi/7898972/remote/smithi159/log 2024-09-10T10:26:44.743 DEBUG:teuthology.orchestra.run.smithi159:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-10T10:28:02.925 INFO:tasks.cephadm:Removing cluster... 2024-09-10T10:28:02.926 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid bade4b50-6f56-11ef-bcea-c7b262605968 --force 2024-09-10T10:28:03.212 INFO:teuthology.orchestra.run.smithi005.stdout:Deleting cluster with fsid: bade4b50-6f56-11ef-bcea-c7b262605968 2024-09-10T10:28:04.451 DEBUG:teuthology.orchestra.run.smithi102:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid bade4b50-6f56-11ef-bcea-c7b262605968 --force 2024-09-10T10:28:04.670 INFO:teuthology.orchestra.run.smithi102.stdout:Deleting cluster with fsid: bade4b50-6f56-11ef-bcea-c7b262605968 2024-09-10T10:28:05.297 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid bade4b50-6f56-11ef-bcea-c7b262605968 --force 2024-09-10T10:28:05.785 INFO:teuthology.orchestra.run.smithi159.stdout:Deleting cluster with fsid: bade4b50-6f56-11ef-bcea-c7b262605968 2024-09-10T10:28:06.518 INFO:tasks.cephadm:Removing cephadm ... 2024-09-10T10:28:06.518 DEBUG:teuthology.orchestra.run.smithi005:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-10T10:28:06.557 DEBUG:teuthology.orchestra.run.smithi102:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-10T10:28:06.577 DEBUG:teuthology.orchestra.run.smithi159:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-10T10:28:06.593 INFO:tasks.cephadm:Teardown complete 2024-09-10T10:28:06.593 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-09-10T10:28:06.605 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-09-10T10:28:06.605 DEBUG:teuthology.orchestra.run.smithi005:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-10T10:28:06.609 DEBUG:teuthology.orchestra.run.smithi102:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-10T10:28:06.619 DEBUG:teuthology.orchestra.run.smithi159:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-10T10:28:06.771 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-10T10:28:06.782 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-10T10:28:06.782 DEBUG:teuthology.orchestra.run.smithi005:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-10T10:28:06.786 DEBUG:teuthology.orchestra.run.smithi102:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-10T10:28:06.789 DEBUG:teuthology.orchestra.run.smithi159:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-10T10:28:06.803 INFO:teuthology.orchestra.run.smithi102.stderr:bash: line 1: ntpq: command not found 2024-09-10T10:28:06.803 INFO:teuthology.orchestra.run.smithi159.stderr:bash: line 1: ntpq: command not found 2024-09-10T10:28:06.816 INFO:teuthology.orchestra.run.smithi005.stderr:bash: line 1: ntpq: command not found 2024-09-10T10:28:06.823 INFO:teuthology.orchestra.run.smithi005.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-10T10:28:06.823 INFO:teuthology.orchestra.run.smithi005.stdout:=============================================================================== 2024-09-10T10:28:06.823 INFO:teuthology.orchestra.run.smithi005.stdout:^* hv01.front.sepia.ceph.com 3 6 377 9 -410us[ -418us] +/- 49ms 2024-09-10T10:28:06.823 INFO:teuthology.orchestra.run.smithi005.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 9 +362us[ +362us] +/- 63ms 2024-09-10T10:28:06.823 INFO:teuthology.orchestra.run.smithi005.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 11 +240us[ +232us] +/- 77ms 2024-09-10T10:28:06.823 INFO:teuthology.orchestra.run.smithi005.stdout:^? hv04.front.sepia.ceph.com 0 10 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T10:28:06.832 INFO:teuthology.orchestra.run.smithi102.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-10T10:28:06.832 INFO:teuthology.orchestra.run.smithi102.stdout:=============================================================================== 2024-09-10T10:28:06.833 INFO:teuthology.orchestra.run.smithi102.stdout:^* hv01.front.sepia.ceph.com 3 6 377 11 -415us[ -430us] +/- 49ms 2024-09-10T10:28:06.833 INFO:teuthology.orchestra.run.smithi102.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 11 +359us[ +344us] +/- 63ms 2024-09-10T10:28:06.833 INFO:teuthology.orchestra.run.smithi102.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 12 +222us[ +207us] +/- 76ms 2024-09-10T10:28:06.833 INFO:teuthology.orchestra.run.smithi102.stdout:^? hv04.front.sepia.ceph.com 0 10 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T10:28:06.838 INFO:teuthology.orchestra.run.smithi159.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-10T10:28:06.838 INFO:teuthology.orchestra.run.smithi159.stdout:=============================================================================== 2024-09-10T10:28:06.839 INFO:teuthology.orchestra.run.smithi159.stdout:^* hv01.front.sepia.ceph.com 3 7 377 7 -371us[ -373us] +/- 49ms 2024-09-10T10:28:06.839 INFO:teuthology.orchestra.run.smithi159.stdout:^+ hv02.front.sepia.ceph.com 3 7 377 18 +357us[ +356us] +/- 63ms 2024-09-10T10:28:06.839 INFO:teuthology.orchestra.run.smithi159.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 12 +207us[ +205us] +/- 76ms 2024-09-10T10:28:06.839 INFO:teuthology.orchestra.run.smithi159.stdout:^? hv04.front.sepia.ceph.com 0 10 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T10:28:06.840 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-10T10:28:06.849 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-10T10:28:06.850 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-10T10:28:06.875 DEBUG:teuthology.orchestra.run.smithi005:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-10T10:28:07.023 DEBUG:teuthology.orchestra.run.smithi102:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-10T10:28:07.147 DEBUG:teuthology.orchestra.run.smithi159:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-10T10:28:07.249 DEBUG:teuthology.orchestra.run.smithi005:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-10T10:28:07.312 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T10:28:07.312 DEBUG:teuthology.orchestra.run.smithi102:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-10T10:28:07.344 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T10:28:07.345 DEBUG:teuthology.orchestra.run.smithi159:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-10T10:28:07.373 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T10:28:07.373 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-10T10:28:07.385 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-10T10:28:07.425 INFO:teuthology.task.internal:Duration was 4114.996156 seconds 2024-09-10T10:28:07.426 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-10T10:28:07.458 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-10T10:28:07.458 DEBUG:teuthology.orchestra.run.smithi005:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-10T10:28:07.462 DEBUG:teuthology.orchestra.run.smithi102:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-10T10:28:07.465 DEBUG:teuthology.orchestra.run.smithi159:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-10T10:28:07.521 INFO:teuthology.orchestra.run.smithi159.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-10T10:28:07.526 INFO:teuthology.orchestra.run.smithi102.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-10T10:28:07.546 INFO:teuthology.orchestra.run.smithi005.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-10T10:28:07.907 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-10T10:28:07.907 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi005.front.sepia.ceph.com 2024-09-10T10:28:07.907 DEBUG:teuthology.orchestra.run.smithi005:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-10T10:28:07.958 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi102.front.sepia.ceph.com 2024-09-10T10:28:07.958 DEBUG:teuthology.orchestra.run.smithi102:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-10T10:28:08.000 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi159.front.sepia.ceph.com 2024-09-10T10:28:08.001 DEBUG:teuthology.orchestra.run.smithi159:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-10T10:28:08.032 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-10T10:28:08.032 DEBUG:teuthology.orchestra.run.smithi005:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-10T10:28:08.036 DEBUG:teuthology.orchestra.run.smithi102:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-10T10:28:08.043 DEBUG:teuthology.orchestra.run.smithi159:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-10T10:28:08.103 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-10T10:28:08.104 DEBUG:teuthology.orchestra.run.smithi005:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-10T10:28:08.108 DEBUG:teuthology.orchestra.run.smithi102:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-10T10:28:08.114 DEBUG:teuthology.orchestra.run.smithi159:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-10T10:28:10.642 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-10T10:28:10.653 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-10T10:28:10.653 DEBUG:teuthology.orchestra.run.smithi005:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-10T10:28:10.709 DEBUG:teuthology.orchestra.run.smithi102:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-10T10:28:10.748 DEBUG:teuthology.orchestra.run.smithi159:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-10T10:28:10.781 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-10T10:28:10.792 DEBUG:teuthology.orchestra.run.smithi005:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-10T10:28:10.796 DEBUG:teuthology.orchestra.run.smithi102:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-10T10:28:10.799 DEBUG:teuthology.orchestra.run.smithi159:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-10T10:28:10.826 INFO:teuthology.orchestra.run.smithi102.stdout:kernel.core_pattern = core 2024-09-10T10:28:10.862 INFO:teuthology.orchestra.run.smithi005.stdout:kernel.core_pattern = core 2024-09-10T10:28:10.888 INFO:teuthology.orchestra.run.smithi159.stdout:kernel.core_pattern = core 2024-09-10T10:28:10.908 DEBUG:teuthology.orchestra.run.smithi005:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-10T10:28:10.941 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T10:28:10.941 DEBUG:teuthology.orchestra.run.smithi102:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-10T10:28:10.957 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T10:28:10.957 DEBUG:teuthology.orchestra.run.smithi159:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-10T10:28:10.975 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T10:28:10.975 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-10T10:28:10.986 INFO:teuthology.task.internal:Transferring archived files... 2024-09-10T10:28:10.987 DEBUG:teuthology.misc:Transferring archived files from smithi005:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2024-09-10_06:58:10-fs-wip-rishabh-testing-20240909.165402-squid-distro-default-smithi/7898972/remote/smithi005 2024-09-10T10:28:10.987 DEBUG:teuthology.orchestra.run.smithi005:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-10T10:28:11.104 DEBUG:teuthology.misc:Transferring archived files from smithi102:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2024-09-10_06:58:10-fs-wip-rishabh-testing-20240909.165402-squid-distro-default-smithi/7898972/remote/smithi102 2024-09-10T10:28:11.104 DEBUG:teuthology.orchestra.run.smithi102:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-10T10:28:11.141 DEBUG:teuthology.misc:Transferring archived files from smithi159:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2024-09-10_06:58:10-fs-wip-rishabh-testing-20240909.165402-squid-distro-default-smithi/7898972/remote/smithi159 2024-09-10T10:28:11.141 DEBUG:teuthology.orchestra.run.smithi159:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-10T10:28:11.182 INFO:teuthology.task.internal:Removing archive directory... 2024-09-10T10:28:11.182 DEBUG:teuthology.orchestra.run.smithi005:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-10T10:28:11.186 DEBUG:teuthology.orchestra.run.smithi102:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-10T10:28:11.188 DEBUG:teuthology.orchestra.run.smithi159:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-10T10:28:11.234 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-10T10:28:11.249 INFO:teuthology.task.internal:Not uploading archives. 2024-09-10T10:28:11.249 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-10T10:28:11.292 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-10T10:28:11.293 DEBUG:teuthology.orchestra.run.smithi005:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-10T10:28:11.297 DEBUG:teuthology.orchestra.run.smithi102:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-10T10:28:11.299 DEBUG:teuthology.orchestra.run.smithi159:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-10T10:28:11.312 INFO:teuthology.orchestra.run.smithi102.stdout: 262165 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 10 10:28 /home/ubuntu/cephtest 2024-09-10T10:28:11.316 INFO:teuthology.orchestra.run.smithi159.stdout: 262165 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 10 10:28 /home/ubuntu/cephtest 2024-09-10T10:28:11.328 INFO:teuthology.orchestra.run.smithi005.stdout: 393233 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 10 10:28 /home/ubuntu/cephtest 2024-09-10T10:28:11.330 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-10T10:28:11.344 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-10T10:28:11.420 INFO:teuthology.run:Summary data: client.0-kernel-sha1: 2ca44a2f35ca8f36983aaaa46d49312239ef519a description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/legacy wsync/yes} objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/no 5-quiesce/with-quiesce 6-workunit/suites/dbench}} duration: 4114.996156454086 flavor: default owner: scheduled_rishabh@teuthology success: true 2024-09-10T10:28:11.421 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-10T10:28:11.512 INFO:teuthology.run:pass